packages feed

crypton 0.34 → 2.1.7

raw patch · 556 files changed

This diff is very large; some files are shown as “too large to diff”. Download the raw patch for the complete diff.

Files

CHANGELOG.md view
@@ -1,3 +1,1838 @@+# CHANGELOG for crypton++## 2.1.7++One fix: RSA-PSS verification was accepting a signature RFC 8017 says to+refuse.  It is a conformance fault rather than a forgery -- producing such a+signature takes the private key, since it is the signer who chooses the+encoding, and a third party holding a valid signature cannot turn it into+one of these.++* fix(pss): RSA-PSS verification refuses an encoding with a bit set outside+  `emBits`, as RFC 8017 9.1.2 step 6 requires.  Step 9 clears those bits in+  DB and crypton did that; clearing is not checking, so an encoding the+  standard calls inconsistent verified as though it were sound -- the bit+  that made it wrong was thrown away before anything looked at it.  Only the+  signer can produce such a signature, since it takes the private key to+  sign a chosen encoding, so this is conformance rather than forgery.  Found+  with tlsfuzzer, in the `xor 0x80 at 0` case of+  `test-tls13-certificate-verify.py`, while testing hs-tls++## 2.1.6++Two things a caller could walk into, the licence field saying what the tree+actually holds, and the C building without a warning.++* fix(pbkdf2): an output length of zero no longer takes the process down.+  `tryFastPBKDF2_*` passed it through to C, where `assert(out && nout)`+  aborted -- crypton's C is built without `NDEBUG`, so its assertions are+  live in a release.  `Crypto.KDF.PBKDF2.tryGenerate` has always answered+  with an empty result for the same request, and the fast paths now agree+* fix(p256): `Crypto.PubKey.ECC.P256.scalarInv` returns on a zero scalar+  rather than looping for ever.  `scalarFromBinary` accepts any 256 bits, so+  a zero scalar is easy to come by, and the binary extended Euclid behind+  `scalarInv` has no exit for it: zero stays even and is halved for ever.+  A hang inside a foreign call is not interruptible, so `System.Timeout` was+  no help either.  It now answers zero, which is what the function already+  answered for the other input with no inverse, and what `scalarInvSafe`+  answers for both.  crypton's own ECDSA was never exposed: it rejects a+  zero scalar before inverting, and uses `scalarInvSafe`+* doc(cabal): the `license:` field says what the tree holds --+  `BSD-3-Clause AND MIT AND ISC` -- and `license-files:` lists the five+  licence texts, where a tool looking for licences will find them.  The+  parts of `cbits/aes/gcm_fused_x86.c` that follow picotls's `fusion` now+  carry its MIT notice beside the file.  **Nothing is required of a user+  that was not required before**: crypton's own code is BSD-3-Clause as it+  always was, and the MIT and ISC code was already in the tree -- the field+  was silent about it.  Raised by Joey Hess in #232, and settled with the+  help of Kazuho Oku, who divided `fusion` between what derives from+  OpenSSL and what does not, and rewrote the former upstream+* fix(c): the sanitizer build is quiet again.  `crypton_sha256_finalize` and+  `crypton_sha512_finalize` say that their pointers are never null, which+  they never were.  gcc's `-Wstringop-overflow` had been reporting them as+  writing "into a region of size 0" at "address zero" under+  `-fsanitize=undefined`: UndefinedBehaviorSanitizer inserts a null check+  before `memcpy`, because glibc declares `memcpy` nonnull, and the check+  puts a null path in front of the warning pass.  Stating the contract+  removes the path rather than the warning, and costs nothing -- compiled as+  the package compiles it, the assembly is identical either way+* fix(pbkdf2): an instantiation whose digest is larger than its block is+  refused where it is written rather than after it has overflowed.  The+  macro that builds the three PBKDF2 variants shortens a long key by hashing+  it into a buffer the size of the block, and asserted afterwards that the+  result fitted -- afterwards being too late, since the write has already+  happened.  The check is a compile-time one now.  The three that exist are+  unaffected: SHA-1, SHA-256 and SHA-512 have digests of 20, 32 and 64 bytes+  against blocks of 64, 64 and 128++## 2.1.5++crypton 2.1.3 and 2.1.4 cannot be built with GCC 14 or newer; it was+reported from a Fedora 43 system, which ships GCC 15.  This release is that+fix, and two things that came with it.++* fix(x86): the C builds with GCC 14 and newer again.+  `crypton_sha1_x86_do_chunk` was declared taking `const uint32_t buf[16]`+  while every caller passes a `const uint8_t *`, and GCC 14 made+  `-Wincompatible-pointer-types` an error by default where GCC 13 only warns.+  The declaration now says `const uint8_t buf[64]`, which is the block size+  SHA-1 actually takes and what the two sibling functions already said.+  Reported as #282 and fixed in #284, both by @tbidne, who bisected it to+  the commit that introduced the declaration.  CI now builds the C with+  gcc-14 as well, so the next one of these is caught before release+* perf(armv8): AES-GCM is about a quarter faster on AArch64, which puts it+  ahead of OpenSSL 4.0.3 rather than behind it -- 1.15 at AES-128 and 1.06+  at AES-256 on an Apple M4, from 0.86.  The GHASH no longer keeps H the way+  GCM writes it; it is twisted once at key setup so that GCM's bit+  reflection is already undone, which turns a reduction of some twenty-five+  shifts and XORs into two PMULL and six EOR, and makes Karatsuba worth+  taking -- three multiplications a block rather than four.  Against the+  previous code over 16 KiB messages: 1.34 at AES-128 and 1.23 at AES-256 on+  an M4, and 1.25 across the three key sizes on a Neoverse N2.  The scheme+  is ARM's, from the BSD-3-Clause part of+  https://github.com/ARM-software/AArch64cryptolib+* test(armv8): the constant-time harness runs on AArch64, where it never had.+  It was pinned to one x86-64 job, so the AArch64 AES and GHASH had never+  been put to it; they are now, and they let no secret decide a branch or an+  address.  Running it somewhere new also found a fault in the harness+  itself: it counted the frame `--track-origins` prints to say where a value+  came from as a place that branched on a secret, which invented a finding+  rather than hiding one++## 2.1.4++2.1.3 could not be built from Hackage at all in the default configuration,+and is deprecated there.  This release is that fix and three more.++* fix(cabal): the source distribution carries `cbits/p256/p256_verify.h`.+  No field named it, so it was absent from the 2.1.3 tarball, and+  `cbits/p256/p256_ec.c` includes it whenever `support_s2n_bignum` is on --+  which is every x86-64 and aarch64 machine that leaves the flag alone.  The+  package built perfectly from a git checkout and not at all from Hackage.+  Reported as #270 by Laurent P. Rene de Cotret on the day 2.1.3 went out,+  fixed by gev in #271+* fix(armv8): the C builds with gcc before 13 again.  A function that uses an+  AArch64 extension says so with `__attribute__((target(...)))`, and the+  spelling used -- `target("+sha3")` -- is one clang has always taken and gcc+  learned in 13.  Before that the extension never reaches the function and an+  `always_inline` intrinsic that needs it cannot be inlined, which stops the+  build rather than slowing it.  Naming the architecture beside the extension+  is understood by both compilers at every version, so gcc is given that+  spelling.  Reported as #273 by gev, against 2.1.1, 2.1.2 and 2.1.3+* fix(sha256): SHA-256 on AArch64 is no longer five times slower than it was+  in 2.1.2 -- 644 MB/s against 3396 over 16 KiB on an Apple M4.  The+  CRYPTOGAMS assembly picks its path from `crypton_armcap_P` rather than from+  a flag in the C, and the bit was set only while that flag was still+  unresolved; 2.1.3 added a constructor that resolves it before anything+  runs, so the bit was never set and the assembly took its generic path on+  every processor.  SHA-1 was unaffected, its constructor setting the+  corresponding bit itself, and the SHA-512 assembly is x86 only.  No test+  could have caught this: the answers were right all along, only slow+* test(ci): three jobs for the three ways the above went unnoticed.  One+  builds the source distribution and then builds the library from it+  somewhere other than the checkout, since nothing had ever built a tarball+  and listing one is not building it.  One installs gcc-12 and builds the C+  with it, the runner's own gcc being 13, which is why a report covering+  three releases never reproduced here.  Both are verified against the bug+  they exist for: each was red before its fix and green after++# CHANGELOG for crypton++## 2.1.3++* fix(number): the arithmetic crypton falls back to when it is built without+  GMP no longer walks off the end of a buffer.  `fillPtr` writes a number out+  starting at its last byte and stops at offset zero, so a number of no bytes+  -- which is what zero is -- started it at minus one, and it never stopped.+  The same build also had `exponentiation` fail to terminate on a negative+  exponent, stepping between -1 and -2 until the stack ran out, where+  `expFast` and `expSafe` both reach it; `numBits` fail to terminate on a+  negative number; `numBits 0` answer one where GMP answers zero, so that+  `numBytes 0` claimed a byte that is not there; and a modulus of one answer+  one rather than zero in two places.  Every corner is now held to what the+  GMP build answers on the same input, and `-integer-gmp` has a CI job so+  that it stays that way+* fix(cabal): `-fsupport_sse` no longer selects the SSE BLAKE2 and Argon2+  sources on architectures that have no SSE, where they cannot compile, and+  `-fold_toolchain_inliner` links again -- one of the sixteen inline+  definitions in `cbits/decaf/include/word.h` had lost its `static`, which+  `-fgnu89-inline` turns into a duplicate symbol.  Both flags now have a CI+  job+* fix(p256): `crypton_p256_shl` and `crypton_p256_shr` no longer shift a+  digit by its own width, which the standard leaves undefined and which x86+  and ARM answer differently.  Nothing in the library calls either, which is+  why the sanitizers had not reported them: they can only report what runs+* fix(headers): `crypton_skein256.h` and `crypton_skein512.h` declared six+  functions under a misspelling of the package's own name, so the six the C+  defines had no prototype at all.  `crypton_chacha.h` and `crypton_salsa.h`+  each typedef'd a union to the bare name `block`, so no translation unit+  could include both; they are `crypton_chacha_block` and+  `crypton_salsa_block` now+* security(c): seven places that erase key material and then let the memory+  die -- five that `memset` a buffer and `free` it on the next line, two that+  clear a recoded scalar in a local going out of scope -- now write through a+  volatile pointer, which a compiler may not remove.  clang at -O2 was+  keeping all seven, but that is its choice rather than a guarantee.  RSA-2048+  signing is unchanged at 1479.3 microseconds against 1480.0 on an Apple M4+* perf(ed448): the scalar arithmetic on Apple Silicon runs on 64-bit limbs+  rather than 32-bit ones.  decaf sized its field limbs from the architecture+  and its scalar limbs from a macro it worked out from the compiler, and the+  two disagreed wherever `uint_fast32_t` is four bytes, so the same aarch64+  CPU took 64-bit field limbs and 32-bit scalar limbs on macOS and 64-bit for+  both on Linux.  Ed448 signing is 5.5% quicker for it, 20.69 to 19.56+  microseconds on an M4+* perf(p256): the one addition in each scalar multiplication that can be a+  point added to itself goes through a complete formula rather than being+  detected and worked around.  Kyle Butt pointed out that the comb's table is+  affine, so the same three numbers are the point in projective coordinates+  and in Jacobian ones, which is what lets a comb built on Jacobian+  arithmetic step into the formula for one addition.  It costs about a third+  of a percent, and buys an argument a reader had to follow becoming a+  comparison a machine can run+* doc(hash): the Haddock for `Context` says that it is not erased when it+  is finished with, what survives in it, and why scrubbing every one is not+  done -- it costs about 70% of a 32-byte hash, where the allocation is most+  of the work+* test(c): the C is now checked in CI for things the test suite cannot ask+  about: whether it is the same on a 32-bit machine and on a big-endian one,+  whether a private key ever decides a branch or an address, what a secret+  leaves behind in memory, and whether anything breaks on generated input.+  Each of the last four carries a probe that is deliberately wrong and has to+  be reported, because a check that has quietly stopped working otherwise+  reads as a clean bill of health -- which, four times over the course of+  this work, is exactly what it did++* doc(cabal): every dependency has an upper bound, which `cabal check` had+  been asking for, and `bytestring` is no longer listed twice in the same+  `build-depends`++* fix(c): the table that says which AES implementation to call is filled in+  once, before there is a second thread, rather than on every+  `crypton_aes_initkey`.  Two threads taking a key at the same time were+  writing the whole table at the same time, which ThreadSanitizer reports+  forty-four times over for eight threads doing nothing else.  The same for+  the flags that say whether to use the ARMv8 SHA-1, SHA-256 and SHA-512+  instructions.  Nothing has ever come of it -- the values written are the+  same ones every time and the table starts out holding valid generic+  implementations -- but it is a race the standard gives no meaning to.+  Taking an AES key is 6.7% quicker for not doing the work again: 72.1 to+  67.3 nanoseconds on an Apple M4+* fix(c): the C no longer reads a word off a caller's pointer by casting it,+  which the standard leaves undefined at an address the word type is not+  aligned for and which UndefinedBehaviorSanitizer reported on seventy-eight+  lines.  `crypton_align.h`'s accessors go through `memcpy`, the ten hash+  implementations read their block a word at a time rather than pointing at+  it as though it were an array of words, and `block128` is packed so that a+  caller's pointer may be one.  The non-aligned trampolines those hashes kept+  for the case are gone with it.  Measured on an Apple M4, every hash and+  AES-GCM is where it was, within a tenth of a per cent.  The sanitizers now+  run in CI with nothing turned off+* fix(c): two left shifts the C standard leaves undefined, found by building+  the C with UndefinedBehaviorSanitizer and running the test suite.  One+  shifted a carry into the sign bit of a signed 64-bit digit in+  `cbits/p256/p256.c`, the other shifted a negative value in+  `cbits/decaf/ed448goldilocks/decaf.c`.  Neither miscomputes on any compiler+  crypton is built with, and the values are unchanged; what they were was a+  licence the standard gives the compiler and no reason to give it+* security(cipher): a message of 2^32 bytes or more no longer has its length+  truncated on the way to the C, which takes its lengths as `uint32_t`.  It+  used to be: `Crypto.Cipher.ChaCha.combine` given 2^32 + 64 bytes enciphered+  64 of them and returned the rest as it found the buffer -- 4 GiB of zeros+  where the ciphertext should have been, with nothing returned to say so.+  `Crypto.Hash` has cut its work into 2 GiB pieces for this reason since+  before crypton; nothing else did.++  Where the C carries its state in a context and can simply be called again,+  the work is now cut up the same way and a long message is enciphered:+  `Crypto.Cipher.ChaCha`, `Crypto.Cipher.Salsa`, `Crypto.Cipher.XSalsa`,+  `Crypto.Cipher.RC4`, `Crypto.MAC.Poly1305`, and AES-GCM's incremental+  interface -- which is what `Crypto.Cipher.ChaChaPoly1305` and+  `Crypto.MAC.KMAC` reach it through.++  Where it cannot -- the one-call AEADs, which do the whole message in one+  call, and the AES modes, which are handed the IV and do not hand it back --+  the message is refused: `CryptoError_ParameterInvalid` from+  `Crypto.Cipher.AES.GCM` and `Crypto.Cipher.ChaCha.Poly1305`, and an error+  from AES ECB, CBC, CTR, XTS, OCB and CCM.  ECB, CBC and XTS count blocks+  rather than bytes, so their limit is sixteen times further out.+  `Crypto.Cipher.AESGCMSIV` already refused, and still does+* perf(p256): the comb that multiplies the base point takes five bits of the+  scalar at a time from each of two blocks rather than four, over the signed+  all-bits-set representation, so the table stays the same size while the+  loop goes from 32 steps to 26: 25 doublings and 52 mixed additions against+  31 and 64, which is 19% fewer field operations.  An ECDSA P-256 signature+  goes 25.52 to 22.47 microseconds on an Apple M4.  This is the C path, which+  x86-64 and AArch64 do not take -- they have the vendored assembly -- so it+  is for i386, armv7, riscv64, ppc64le, s390x and the rest.  The arrangement+  was suggested by Kyle Butt.++  One scalar below the order makes the last addition of the comb add a point+  to itself, which the formulas there cannot do; it was found by searching+  the sign patterns rather than by sampling, and the recoder now reports it+  and the answer is the doubling.  Every other addition is ruled out, the 48+  from step two upwards by parity and size and the two of step one by+  exhausting the 2^20 sign patterns that could reach them+* security(aes): `Crypto.Cipher.AES.GCM` refuses a nonce of no bytes, which+  its four functions used to accept.  SP 800-38D 5.2.1.1 asks for at least+  one byte, and with none GCM's pre-counter block is zero, so the tag of a+  message is `GHASH_H(A, C) XOR E(K, 0^128)` -- and `E(K, 0^128)` is the+  GHASH key `H` itself.  One full tag therefore gives `H` away; `H` belongs+  to the key rather than to the nonce, so an attacker who has it, and one+  genuine message under any nonce, can forge a tag that verifies for data of+  their own under that nonce, twelve-byte ones included.  `encrypt` and+  `decryptWithTag` now throw `CryptoError_IvSizeInvalid`, `decrypt` gives+  `Nothing` and `encryptWithMask` gives `False` and writes nothing.  Only the+  empty nonce is refused; every other length stays allowed.  The general+  interface has refused it since 2.1.0 and this module did not.  Affects+  2.1.0, 2.1.1 and 2.1.2.  Reported by arybczak in+  [#249](https://github.com/kazu-yamamoto/crypton/issues/249)+* perf(rsa): the multiply-accumulate at the bottom of the modular+  exponentiation takes four limbs to an iteration on AArch64, with the flags+  carrying through two long chains -- one for the low halves of the four+  products, one for the high halves a place up -- where the compiler writes+  a chain per limb and spends two instructions moving each carry between the+  flags and a register.  Measured on an Apple M4, an RSA-2048 signature goes+  593.0 to 515.9 microseconds through the Haskell API and the exponentiation+  itself 590.1 to 514.9.  The ragged end of a row -- three limbs, two or+  one -- is written out the same way rather than handed back to C, which+  matters because mont_sqr asks for every length from n-1 down to 1: that is+  a further two per cent of the exponentiation, 514.2 to 504.9.  The+  arrangement follows addMulVVWx in Go's crypto/internal/fips140/bigmod,+  which is BSD-3-Clause as this library is; cbits/LICENSE.go carries its+  notice.  Three other arrangements were tried and measured worse, and the+  comment above the function says which and why, so that they are not tried+  again+* perf(rsa): a Montgomery multiplication no longer clears its 2n-limb+  scratch first.  The first row of the product lands on empty space, so it+  is written rather than added to, and every row after it reads only limbs+  an earlier row has already put there.  Worth between a half and one per+  cent of an RSA-2048 signature on an Apple M4, which is less than it+  sounds like it should be: the clearing was cheap, and what it cost was+  mostly the row that had to add to zeros+* perf(rsa): the R^2 that Montgomery arithmetic needs before it can start is+  built by squaring rather than by doubling a bit at a time.  Write a value+  as 2^(lgR + d) mod m; a Montgomery squaring divides by R, so it takes that+  to 2^(lgR + 2d) and doubles d.  The climb from R to R^2 is then the binary+  expansion of lgR -- ten squarings for a 1024-bit modulus, where there were+  a thousand and twenty-five doublings.  On an Apple M4 the setup goes 24.74+  to 2.07 microseconds, and an RSA-2048 signature does it twice, once for+  each CRT half: the signature goes 512.3 to 466.9 through the Haskell API+  and the two exponentiations 502.6 to 457.0.  Curves that go through+  crypton_ecc.c pay the same setup once per context and gain the same.+  Public-key operations still go through GMP and do not change+* perf(ecdsa): P-256 verification multiplies both scalars at once, in+  variable time, where it used to do two constant-time multiplications and+  add the results.  Everything a verification touches is public -- the+  message, the signature and the public key -- so the constant-time work+  there was paid for nothing, and the two multiplications can share their+  doublings besides.  Both scalars go into non-adjacent form, width 7 for+  the base point, whose odd multiples are a table in the library, and width+  5 for the public key, whose eight are built per call; one pass down the+  digits does a doubling at every step and an addition where a digit is not+  zero.  Measured through the Haskell API on an Apple M4, a verification+  goes 30.37 to 25.48 microseconds, and the multiplication itself 29.94 to+  25.34; on an x86-64, 85.04 to 71.53.  Signing is untouched.  s2n-bignum's+  point addition is correct except when its two arguments are the same+  point, which is the side condition its proof carries, so a sum that comes+  out as the point at infinity from arguments that were not is given up on+  and the constant-time pair answers instead -- 1203 cases covering that,+  including 41 that take the fallback, agree with the old answers on both+  architectures+* perf(gcm): AES-GCM uses the 512-bit form of the AES and carry-less+  multiply instructions where the processor has them and they are worth+  having, which is Ice Lake and Zen 5 onwards.  Four blocks to an+  instruction where the 256-bit form takes two; a group is thirty-two+  blocks in eight registers and its GHASH is two passes of sixteen, since+  sixteen is how many powers of H the table holds.  Measured on GitHub's+  runners over 16 KiB, AES-128-GCM and AES-256-GCM in MB/s: an EPYC 9V45+  goes 9616 to 14268 and 8422 to 12674, a Xeon 6973P-C 8095 to 9848 and+  7187 to 8323, a Xeon 8573C 6983 to 8447 and 6166 to 7113.  Zen 4 is left+  on the 256-bit path -- there the 512-bit instructions are two passes+  through a 256-bit datapath, and AES-GCM measured slightly slower -- and+  the run-time check asks for three more bits of XCR0 as well as the+  instruction bits, since a machine can report these and still fault on+  them+* perf(ed25519): the base point multiplication goes through s2n-bignum,+  which signing does twice -- once for the nonce's point and once for the+  public key, which `sign` derives from the secret key every time rather+  than trusting the one it is handed.  Measured through the Haskell API on+  an Apple M4, signing goes 11.92 to 7.27 microseconds and `toPublic` 5.97+  to 3.52; at the C level on an x86-64 without ADX, where the `_alt` form+  runs, a public key goes 13.25 to 9.93 and a signature with the key in+  hand 14.74 to 11.34.  Verification is untouched: it multiplies two+  scalars at once and crypton's variable-time code is ahead of the+  constant-time assembly there.  ed25519-donna's table stays for every+  other architecture, and 5000 key and signature pairs agree between the+  two paths on both architectures+* perf(rsa): the masked scan of the exponentiation's table goes four limbs+  at a time on x86-64 with AVX2.  Every entry of the table is read and a+  mask keeps the one the window asks for, which is what keeps the address+  stream off the exponent, and at RSA-2048's CRT size that is two kilobytes+  read per window with 256 windows to an exponentiation -- 11% of the whole+  where s2n-bignum's multiplication runs, measured by taking the scan out+  altogether.  With the vector form almost all of it comes back: one+  RSA-2048 CRT private operation goes 812.4 to 720.8 microseconds against a+  707.1 floor with no scan at all, and on the same machine without ADX+  1646.1 to 1599.3.  AArch64 keeps the scalar form, where the same+  measurement puts the scan at 1% and there is nothing to win.  The+  processor is asked once per call, and without the AVX2 bit, or built+  without `use_target_attributes`, the scalar scan is what runs+* perf(rsa): the modular exponentiation squares into the other of its two+  buffers and swaps them, rather than squaring into one and copying it back.+  That is a copy of the modulus' width saved 1280 times per exponentiation,+  and which of the three buffers a pointer names is nobody's secret, so+  nothing about the timing changes.  Measured by thread CPU time, best of+  many, on one RSA-2048 CRT private operation: an Apple M4 goes 604.3 to+  595.9 microseconds and an x86-64 without ADX 1678.0 to 1665.6.  Where+  s2n-bignum's multiplication runs the difference is below the noise, that+  multiplication being most of the time there.  Also writes down what a+  five-bit window is worth, which was measured and is not taken: 3.5% on the+  M4, 1% the wrong way on an older x86-64, and 7% the wrong way wherever the+  assembly runs, because a table twice as long is a masked scan twice as long++## 2.1.2++* perf(gcm): AES-GCM uses the 256-bit form of the AES and carry-less+  multiply instructions where the processor has them, which is Zen 3 and Ice+  Lake onwards.  Measured on an EPYC 7763 over 16 KiB, in the library as+  cabal builds it, AES-128-GCM goes 4245 to 6042 MB/s and AES-256-GCM 3932+  to 5463 -- 1.42 and 1.39 times, and ahead of OpenSSL 3.0.13 on that+  machine, which reaches 4266 and 3946.  `crypton_cpu.c` asks the processor+  first and everything else is unchanged+* perf(x25519): X25519 goes through s2n-bignum, and key generation reads a+  table rather than multiplying the base point 9 the general way, which is+  what crypton did for want of anything else.  Measured through the Haskell+  API on an Apple M4, the shared secret goes 19.87 to 13.78 microseconds and+  a key generation 19.92 to 3.83 -- on x86-64 the C level is 41.19 to 26.96+  and 41.18 to 8.54.  The RFC 7748 vectors say the answers are the same+* perf(ecdsa): inverting modulo the group order, which ECDSA does once per+  signature and once per verification, takes a fixed number of division+  steps instead of a whole exponentiation.  Measured on an Apple M4:+  P-256 6.02 to 0.80 microseconds, P-384 31.3 to 1.20, P-521 63.2 to 2.05.+  Through the Haskell API that is ECDSA P-256 signing 11.77 to 7.10, which+  is faster than OpenSSL on that machine, verification 36.75 to 32.10, and+  P-384 signing 171.6 to 151.3.  `inverseSafe` carries it, so DSA, ElGamal,+  RSA's `qinv` and `Crypto.PubKey.ECC.Prim` get it too; it checks its answer+  by multiplying out, as it always has, and falls back where that fails+* perf(rsa): the modular exponentiation's Montgomery multiplication and+  square go through s2n-bignum on x86-64 with ADX, which is twice as fast as+  the C there because the C cannot form the two carry chains `ADCX` and+  `ADOX` give: measured on an EPYC 7763 at 1024 bits, 0.4832 to 0.2479+  microseconds for a multiplication and 0.3984 to 0.1994 for a square.  The+  window, the table and its masked scan are untouched, and so is every other+  size and architecture -- on AArch64 the C measures faster than the+  assembly, so nothing is even vendored for it+* perf(p256): ECDSA signing is 2.4 times faster and verification 2.2, which+  finishes what the two entries below began.  Signing and key generation go+  through s2n-bignum's fixed-base routine, which reads a table of multiples+  of the base point that `cbits/p256/gen_base_table.py` builds and that the+  test suite checks against crypton's own answer; verification goes through+  that one and the variable-point one, with the addition of the two left+  where it was.  Measured through the Haskell API on an Apple M4, signing+  goes 28.55 to 12.03 microseconds and verification 82.68 to 37.73, which+  leaves both within a tenth of OpenSSL on that machine where they were at+  four tenths of it.  The table costs 52 KiB of constant data, against the+  2.4 KiB of the one it replaces+* perf(ecc): P-384 and P-521 go through s2n-bignum as well, where the curve+  is exactly one of those two.  Measured through the Haskell API on an Apple+  M4, ECDH P-384 goes 549.5 to 75.4 microseconds and ECDSA P-384+  verification 772.4 to 295.8; at the C level P-384 is 7x and P-521 between+  9 and 10x, on both architectures.  Signing does not move: there is no+  fixed-base routine upstream for these two, so it keeps crypton's comb.+  Every other curve `crypton_ecc_mul` is asked about, including these two+  named with a different a or b, goes on to the C as before+* perf(p256): ECDH is 2.7 times faster on x86-64 and AArch64.  The+  variable-point scalar multiplication now goes through AWS's+  [s2n-bignum](https://github.com/awslabs/s2n-bignum), vendored in+  `cbits/s2n`: hand-written assembly, constant-time, and carrying a+  machine-checked proof in HOL-Light that it computes what it says.  Measured+  through the Haskell API on an Apple M4, ECDH P-256 goes 57.50 to 21.32+  microseconds; at the C level it is 2.6x there, 3.2x on an x86-64 with ADX+  and 2.6x on the `_alt` path taken where `CPUID` does not report it.  Its licence is `Apache-2.0 OR ISC OR MIT-0`,+  which is what makes this possible at all -- OpenSSL's and BoringSSL's+  `ecp_nistz256` is Apache-2.0 only.  Every other architecture keeps the C,+  as does Windows for now, and `-f-support_s2n_bignum` turns it off+* perf(p256): the field inversion takes the least squarings an exponent of+  256 bits can be done in.  It built the low 94 ones of p-2 in a second+  accumulator and multiplied the two at the end, which cost 287 squarings+  where 255 will do; the exponent is now built left to right from the shape+  of p-2 in one pass.  Measured on an Apple M4 the inversion goes 4.224 ->+  3.765 microseconds, about 11%, which is 0.9% of an ECDH since that is where+  the inversion sits.  The same 13 multiplications either way++## 2.1.1++* docs(rsa): the haddock says what the optional blinder covers -- that the+  private exponent is not what is at risk, `expSafe` keeping its value out of+  the work, and that what a blinder covers is the input, which without one is+  the ciphertext as it arrived and so a number an attacker may have chosen.+  The eight places taking a `Maybe Blinder` point at t'Blinder' rather than+  repeating half of it; the four in `Crypto.PubKey.RSA.PSS` said nothing at+  all before++* feat(chachapoly): `Crypto.Cipher.ChaCha.Poly1305`, which does a whole+  ChaCha20-Poly1305 message in one call, the shape `Crypto.Cipher.AES.GCM`+  has.  `Crypto.Cipher.ChaChaPoly1305` takes a message in steps, which is+  right when it arrives in pieces and is eight foreign calls and the+  allocations between them when it was already whole.  Measured on an Apple+  M4 through the Haskell interface, a 100-byte message goes 0.97 -> 0.415+  microseconds and a 1400-byte one 2.89 -> 2.36.  The nonce is the twelve+  bytes RFC 8439 defines; eight is the other ChaCha construction and is+  refused rather than quietly encrypted under a scheme nobody asked for++* feat(gcm): `Crypto.Cipher.AES.GCM.decryptWithTag`, which decrypts and hands+  back the tag it computed rather than comparing it.  For a protocol that+  carries the tag apart from the ciphertext, where `decrypt` -- which wants+  the two together -- does not fit.  It returns an `AuthTag`, whose `Eq` is a+  constant-time comparison, so the safe way to use it is also the obvious one++* feat(ecdsa): `Crypto.PubKey.ECDSA` gains the deterministic nonce of RFC+  6979, which `Crypto.PubKey.ECC.ECDSA` already had.  The fast module was the+  one without it, so moving to it for the speed meant giving up the one+  protection against the mistake that hands over an ECDSA private key.  Three+  new names: `deterministicNonce`, and `signDeterministic` and+  `signDigestDeterministic` over it.  Held to the implementation in+  `Crypto.PubKey.ECC.ECDSA`, which is itself held to the vectors in the RFC,+  on P-256, P-384 and P-521 with SHA-1 through SHA-512++## 2.1.0++* perf(p256): a signed five-bit window for the variable-point scalar+  multiplication, which is what ECDH and ECDSA signing spend their time in.+  The scalar is recoded into 52 digits, every one of them odd, so the table+  holds only the odd multiples P, 3P, ..., 31P and a negative digit costs a+  negation of y, which is free.  The main loop goes from 252 doublings and 64+  additions to 255 and 51, and -- because no digit is zero and no partial sum+  is the infinity -- it drops the masks that stood in for infinity on every+  iteration.  The table is built so that each pair of neighbouring odd+  multiples comes out of one doubling and one addition that shares everything+  but a squaring and a multiplication between X+P and X-P.  Counted exactly,+  the field multiplications and squarings go 3477 -> 3326.  Measured on an+  idle Intel Haswell, thirty runs each, ECDH is 158.5 -> 153.3 microseconds,+  about 4%; on an Apple M4 under desktop load the difference did not come out+  of the noise.  One scalar, 30, would have reached the last addition with+  the accumulator equal to the point being added, which these formulas cannot+  do; the recoder detects that from the scalar and the last iteration doubles+  instead.  Suggested by Kyle Butt++* perf(gcm): GHASH takes the ciphertext from the output buffer.  A group's+  multiplies are issued between the rounds of the group after it, and the+  blocks were copied into six registers' worth of scratch to wait there --+  six stores a group for bytes that had just been written to the output+  anyway.  The multiplies read the output instead, which is what picotls's+  fusion does.  On an Intel Haswell this is worth two to three points against+  fusion between 400 and 1440 bytes, and it removes the queue from the+  AES-128 path++* perf(gcm): decryption takes the fused path too, on both x86-64 and+  AArch64.  It had been left on the generic framing, so a received packet+  cost what a sent one did before any of this: measured at 100 bytes, three+  times what encrypting the same packet cost on either.  It is the simpler of+  the two -- what GHASH absorbs is the ciphertext, and the ciphertext is the+  input, so the multiplies need not wait on the AES and nothing is carried+  between groups.  On an Intel Haswell, 100 bytes goes 0.165 -> 0.050+  microseconds and 1440 bytes 0.362 -> 0.290; on an Apple M4, 0.230 -> 0.077+  and 0.396 -> 0.321.  Decrypting is now about what encrypting is rather than+  three times it.  The tag is still compared a byte at a time over its whole+  length whichever way the answer goes++* perf(gcm): let the one-call interface specialise.  `gcmFullEncrypt`,+  `gcmFullDecrypt` and `gcmFullEncryptMask` take three `ByteArrayAccess`+  arguments and were marked `NOINLINE`, which is this module's habit and is+  right for a wrapper that is called once; these are called once a packet.+  With no specialisation every `withByteArray` and every `length` went through+  a dictionary, and on an Apple M4 that measured **0.15 of the 0.265+  microseconds** a 100-byte packet cost through the Haskell interface -- more+  than the encryption it wrapped.  Marked `INLINABLE` so the caller can+  specialise them, 100 bytes falls to 0.128, where the same work measured in C+  is 0.114: the Haskell layer costs 0.014 rather than 0.15++* perf(gcm): build the length block and the initial counter in registers.+  The length block -- the two bit counts GHASH ends on -- was assembled by+  sixteen byte stores to the stack and read back, which measured about 9 of+  the 58 nanoseconds a 100-byte packet cost.  Reversed the way every block is+  on its way to GHASH, that block is just the two counts as little endian+  words with the message's in the low half, so one `_mm_set_epi64x` makes it+  and no shuffle is needed.  The initial counter likewise: the nonce is read+  where it lies and masked, rather than in three pieces of four bytes.  On an+  Intel Haswell a 100-byte packet goes from 0.058 to 0.049 microseconds.  With+  this every length measured is at 90 per cent of fusion's speed or better --+  100 bytes 90 and 95 with the header protection mask, 200 bytes 96, 1440+  bytes 94, 16 KiB 95 -- where the series began at 31 per cent for 100 bytes++* perf(gcm): read a short block without going through the stack.  Zeroing+  sixteen bytes, copying the block in and loading them back is three trips to+  memory with a store the load must wait for, and at packet lengths that was a+  tenth of the call.  The sixteen bytes are read where they lie and what is+  above the length masked off, which is safe everywhere except at the end of a+  page -- and a block near the end of a page whose own bytes stop short of it+  is read aligned, which cannot leave the page, and shuffled down.  This is+  how picotls's fusion does it.  On an Intel Haswell a 100-byte packet goes+  from 0.0625 to 0.058 microseconds, 79 per cent of fusion's speed against 73,+  and with the header protection mask 83+* perf(gcm): the tail of a message gets what the groups already had.  The+  six-wide pass that finishes a message was still running its rounds from a+  loop over a count held in the key, so the compiler could not place the+  waiting multiplies between them, and the blocks it produced went through an+  array indexed by a loop variable, which it cannot see through -- each block+  then reloaded its own keystream from memory.  Written out for the ten rounds+  of AES-128, with the multiplies at slots named at compile time, and the+  blocks taken from the registers the pass left them in.  The pass itself+  falls from about 29 to 6 nanoseconds; on an Intel Haswell 112 bytes is 13+  per cent faster and 400 bytes goes from 81 to 86 per cent of fusion's speed++* perf(gcm): give E(K,Y0) a lane that would otherwise sit idle, and stop+  copying the last short block through the stack.  Six blocks are in flight+  whatever the message length, so a message leaving a tail of four blocks or+  fewer has lanes to spare; the block that masks the tag rides in one of them+  instead of taking ten rounds nothing overlaps, which at 100 bytes measured+  9.3 of 78.9 nanoseconds.  And the last short block was stored to the stack+  and copied back, when the tag that follows it is about to overwrite the+  bytes above it anyway -- where there are sixteen to spare, one store does.+  On an Intel Haswell, 100 bytes goes from 60 to 69 per cent of fusion's speed+  and 200 bytes from 68 to 82++* perf(gcm): build the counter block without leaving the vector registers,+  and keep each power of H beside its Karatsuba term.  Both came from reading+  what picotls's `fusion` does differently.  The counter was being stepped in+  a general register, byte swapped there and inserted into a vector one, which+  is a move across register files for every lane and six to a group; held+  byte reversed in a vector register instead, `_mm_add_epi32` steps the low+  thirty-two bits and wraps them where GCM wants, and a shuffle puts the bytes+  back.  The powers were in two arrays 256 bytes apart, so a multiply touched+  two cache lines for operands it always wants together; they are now+  adjacent.  On an Intel Haswell a 1200-byte message goes from 0.311 to 0.281+  microseconds and 1440 bytes from 79 to 90 per cent of fusion's speed.  The+  multiplies are also now genuinely issued between the AES rounds, which the+  comment claimed and the generated code did not do: a test before each one+  ended the basic block the scheduler works inside, and the first group is+  peeled so that there is nothing to test++* perf(gcm): the same for AArch64, where what costs is the framing rather+  than a missing fast path.  `armv8_impl.c` already encrypts eight blocks at a+  time and folds their GHASH into one reduction; what sat outside it was the+  additional data, the tag and the counter, each reached through the branch+  table so that the running state went back to memory between them and a+  one-block header paid a reduction of its own.  Measured on an Apple M4, that+  framing was 0.07 of the 0.112 microseconds a 100-byte packet cost -- more+  than the encryption of the packet itself.  Taking the whole message in one+  call, with the tag and the counter in registers from end to end and the+  additional data and the length block riding in the same batches as the+  ciphertext: a 100-byte packet 3.0x, 200 bytes 2.6x, 400 bytes 1.6x, 1200+  bytes 1.30x, 1440 bytes 1.23x, and level from about 6 KB up.  With the QUIC+  header protection mask, 100 bytes is 3.3x.  Unlike x86-64 there is no length+  above which something else is faster, because there is no vendored assembly+  on this side to hand a long message to, so every length goes this way.  Held+  against the interface it replaces on two key sizes, seven lengths of+  additional data, fifteen message lengths up to 16 KB, three tag lengths and+  every sample offset that fits++* perf(gcm): a fused AES-GCM for x86-64, for messages short enough that the+  stitched assembly will not take them.  That assembly refuses anything under+  288 bytes, so until now a QUIC packet paid for the AES key schedule and the+  GHASH one block at a time, through a branch table that put the 128-bit state+  back in memory at every step: a 100-byte packet cost 0.153 us of which the+  encryption was a small part.  This is the design Kazuho Oku sets out for+  picotls's `fusion` -- keep AES-NI issuing every clock, six blocks in flight,+  and fit the additional data, the tag and the QUIC header protection mask+  into the gaps between the rounds -- written in C with intrinsics, for the+  reason he gives: what is complicated here is the scheduling, and it has to+  stay readable to stay correct.  On an Intel Haswell a 100-byte packet goes+  from 0.153 to 0.082 us and a 1200-byte one from 0.373 to 0.317, and the+  header protection mask becomes **free** wherever it can be taken: 400 bytes+  is 0.131 with it and 0.131 without, against 0.181 and 0.177, because it+  rides in a lane of the AES pipeline that the message length leaves idle+  rather than taking a block of its own.  It can be taken there only when the+  sample lies in output already written and the two key schedules are the same+  length, which is what TLS and QUIC do; otherwise it is computed after the+  tag, where everything it may cover exists.  Above+  1536 bytes the assembly is faster -- by 12 per cent at 3 KB and 20 at 16 KB+  -- so longer messages still go there and nothing about TLS-sized records+  changes.  The powers of H are built once per key, sixteen of them, which+  adds 512 bytes to what a key holds and no parameter to any interface: a+  power per block of the message would fold the whole GHASH into one reduction+  but would make that state grow with the longest message a caller might send.+  Held against the incremental interface on every combination of three key+  sizes, seven lengths of additional data, twelve message lengths and three+  tag lengths++* fix(cpu): stop reading Intel's SDBG bit as AMD's XOP, which crashed SHA-512+  and ChaCha20 on Broadwell and later.  The vendored assembly dispatches on a+  capability word this library fills, and reads bit 11 of its second dword as+  XOP -- which is CPUID leaf 0x80000001 ECX bit 11, an AMD extended leaf.+  crypton put the raw leaf 1 ECX there, whose bit 11 is SDBG, the silicon+  debug interface, which Intel has reported since Broadwell.  On such a+  processor `sha512-x86_64.S` took `.Lxop_shortcut` and+  `chacha-x86_64.S` took `.Lcrypton_chacha20_asm_4xop`, and the first+  `vprotq` is an invalid opcode: SIGILL, for SHA-512 and SHA-384, and for+  ChaCha20 and so ChaCha20-Poly1305.  OpenSSL clears leaf 1's bit 11 before+  merging the real flag in; crypton now clears it and leaves it clear, so the+  XOP paths are never taken.  Nothing is lost -- XOP ran from AMD's Bulldozer+  to Excavator and Zen dropped it -- and it is the reason the AVX-512 bits+  beside it are cleared as well.  Reported by @lucasdicioccio, who+  disassembled the trap+  [#202](https://github.com/kazu-yamamoto/crypton/issues/202)++* feat(aes): `encryptWithMask`, for the QUIC header protection mask.  QUIC+  takes the sample for its header protection from the ciphertext, so the mask+  cannot be had before the encryption -- but it can be had before coming back.+  `newHeaderKey` builds the second key schedule once, where `quic` builds it+  per packet, and `encryptWithMask` seals the message and writes the sixteen+  bytes of mask, both into buffers the caller already has, so that nothing is+  allocated for either.  On an Apple M4 the mask then costs about 0.02 us+  against 0.09 to 0.11 asked for separately: a 1440-byte packet goes from+  0.473 to 0.382 us and a 100-byte one from 0.343 to 0.260.  Most of that is+  the allocations rather than the crossing -- a version returning the two as+  bytearrays was measured at 0.419 and 0.299, so it recovered less than a+  third of it -- and the AES block itself is under two nanoseconds++* feat(aes): `Crypto.Cipher.AES.GCM`, for many short messages under one key.+  The interface in `Crypto.Cipher.Types` builds a state from the key *and* the+  nonce and then walks it through appending the additional data, encrypting+  and finalizing, copying the 320-byte state at each step.  For a stream that+  is nothing next to the encryption; for a datagram it is most of the work.+  Measured on an Apple M4, a 1440-byte packet with a 20-byte header took+  1.30 us, of which 0.17 us was the encryption: the AES key schedule and the+  table of multiples of `H` were rebuilt for every nonce although both depend+  on the key alone, and three state copies and four foreign calls carried the+  rest.  A `Context` now holds what the key determines and is built once, and+  `encrypt` takes a nonce and a whole message and answers in one call, giving+  the ciphertext with the tag after it -- the shape a packet wants.  `decrypt`+  takes that shape back and compares the tag itself, in C, looking at every+  byte either way.  1440 bytes: 1.30 to 0.37 us, 3.5x; 100 bytes 0.83 to 0.23;+  a 16 KiB TLS record 2.86 to 2.20, where the saving is the setup rather than+  the call.  A message of 4 KiB or less goes through an unsafe foreign call,+  which is worth 0.075 us and is only right because such a call is over+  quickly; anything longer keeps the safe one.  This computes what the general+  interface computes, which the tests hold it to on the same vectors+* Breaking change: fix(chachapoly1305): take a checked key, so that+  initializing cannot fail.  The nonce was already a checked type, built by+  `nonce8`, `nonce12` or `nonce24`, so the key length was the only way+  `initialize` and `initializeX` could fail -- and callers answered that with+  `throwCryptoError`, `tls` among them, where+  `noFail (ChaChaPoly1305.nonce12 nonce >>= ChaChaPoly1305.initialize key)`+  re-checked a length once per record for a key fixed for the connection.+  There is now a `Key` with `key` to build one, and+  `initialize :: Key -> Nonce -> State` and+  `initializeX :: Key -> XNonce -> State` are total.+  `aeadChacha20poly1305Init` is unchanged and still reports a bad key length.+  This also closes the last of #28: `initFromRootState` wrapped a+  `throwCryptoError` around a `B.take 32`, and the Poly1305 key type now has a+  home in a hidden module so the modules here that know the length can say so+  [#193](https://github.com/kazu-yamamoto/crypton/issues/193)++* feat(hash): Skein with the digest size as a type parameter.  Skein is+  defined for any digest size and the C here has always taken one -- the+  length goes to `crypton_skein512_init` and `crypton_skein512_finalize`, and+  the output is produced in counter mode for as many blocks as are asked for+  -- but Haskell could only reach the four sizes that had a type of their own.+  `Skein256 (bitlen :: Nat)` and `Skein512 (bitlen :: Nat)` take any, in the+  manner `SHAKE128` and `SHAKE256` already did; `Skein512 512` is+  `Skein512_512`, which the tests hold it to, and the named types are+  untouched.  This also brought back the `Skein256-160` and `Skein512-160`+  known-answer vectors, which had been commented out of the test suite for+  want of a type to run them against.  One large digest is a good deal cheaper+  than the same bytes from repeated small ones: 512 KiB at 947 MB/s in one+  digest against 172 MB/s as 8192 separate `Skein512_512` ones, on an M4+  [#56](https://github.com/kazu-yamamoto/crypton/issues/56)++## 2.0.0++* fix(docs): export the names the documentation already referred to.+  `Crypto.Number.ModArithmetic` throws `CoprimesAssertionError` and+  `ModulusAssertionError` from `inverseCoprimes` and `squareRoot`, and said so+  in the haddock, without exporting either, so a caller could not name the+  exception it was told to expect; `Crypto.PubKey.Rabin.Types.generatePrimes`+  takes a `PrimeCondition` in its exported signature and that synonym was not+  exported either.  All three are now exported+  [#195](https://github.com/kazu-yamamoto/crypton/pull/195)++* Breaking change: fix(bcrypt): refuse a cost bcrypt does not have rather than+  substituting one.  A cost below 4 came back as a cost-10 hash and a cost+  above 31 as a cost-31 one, with nothing said either way, so a caller asking+  for something bcrypt does not do was answered with something else and had no+  way to tell -- `hashPassword 3` and `hashPassword 10` returned the same+  thing.  Both ends are now reported as `CryptoError_ParameterInvalid`, which+  is what every other KDF here already did for a refused parameter.+  `hashPassword` can therefore fail where it could not before, so+  `tryHashPassword` is added beside it; the salt it generates is always the+  right length, so the cost is the only thing it can report+  [#59](https://github.com/kazu-yamamoto/crypton/issues/59)++* Breaking change: fix(poly1305): take a checked key, so that initializing+  cannot fail.  A Poly1305 key is thirty-two bytes and nothing else about it+  can be wrong, so `initialize` returning a `CryptoFailable` put an error case+  in front of every caller for a length most of them know is right -- and they+  answered it with `throwCryptoError`, this library included: the one in+  `Crypto.Cipher.ChaChaPoly1305` guarded a `B.take 32`, and `auth` did not+  even do that, it called `error`.  There is now a `Key` with `key` to build+  one, the length is checked there, and `initialize :: Key -> State` and+  `auth :: Key -> ba -> Auth` are total.  A caller checks once and then+  initializes as often as it likes with nothing to handle.  `initialize k`+  becomes `initialize <$> key k` where the length is unknown, and where it is+  known the check moves to where the key is made.  `Key` has no `Show`, as+  key material should not+  [#28](https://github.com/kazu-yamamoto/crypton/issues/28)++* fix(pubkey): stop printing private keys.  `Show` is what `print`, a message+  built with `error`, an exception and a test framework's failure output all+  reach for, so it is the instance a key travels on when nobody meant to send+  it anywhere; the library already kept that promise for the secret keys held+  in a `ScrubbedBytes`, and the documentation for `ScrubbedBytes` advertises+  it, while ten other types printed theirs in full.  Those ten --+  `RSA.PrivateKey` and `RSA.KeyPair`, `DSA.PrivateKey` and `DSA.KeyPair`,+  `ECDSA.PrivateKey` and `ECDSA.KeyPair`, `DH.PrivateNumber`, and the+  `PrivateKey` of `Rabin.Basic`, `Rabin.Modified` and `Rabin.RW` -- now render+  the public part and `<secret>` for the rest.  Nothing else about them+  changes: `Read`, `Eq`, `Data`, `Generic` and `NFData` are all still derived.+  The new `Crypto.Debug` exports a class `DebugShow` whose `debugShow` returns+  exactly what the derived `Show` used to return, for those ten and for the+  five `ScrubbedBytes` secret keys as well, which never had a `Show` that+  spoke.  **Code that serialized a key through `show` has to say `debugShow`+  instead**; `read (debugShow k) == k` still holds, but `read` given the+  output of `show` will now fail at run time, which the compiler cannot point+  at+  [#72](https://github.com/kazu-yamamoto/crypton/issues/72)++* deprecate(ecc): the curves over a binary field.  They are obsolete, they are+  the curves a cofactor makes delicate, and pyca/cryptography deprecated them+  for removal in the release that fixed CVE-2026-26007.  A `DEPRECATED` pragma+  now covers the eighteen `SEC_t*` constructors of `CurveName` and the+  eighteen types of the same names in `Crypto.ECC.Simple.Types`; nothing is+  removed, so the only effect is a warning where one of them is named, and+  they will go in a later major version.  The prime curves with a cofactor,+  `SEC_p112r2` and `SEC_p128r2`, are not deprecated: the check above covers+  them+  [#66](https://github.com/kazu-yamamoto/crypton/issues/66)++* fix(ecc): refuse a public point outside the prime-order subgroup.  A point+  that satisfies the curve equation is not necessarily in the subgroup the+  base point generates; the two coincide only where the cofactor is 1.  Of the+  curves in `CurveName` twenty have a cofactor -- the eighteen binary ones,+  and `SEC_p112r2` and `SEC_p128r2`, whose cofactor is 4 -- and on those the+  other party could offer a point of small order, at which point the value+  that came back depended on our private number only through its residue+  modulo that order, and offering it and watching the answer handed them those+  bits.  This is the flaw pyca/cryptography fixed as CVE-2026-26007; what is+  fixed here is the same one, found by following that report.+  `Crypto.PubKey.ECC.DH.getShared` and `tryGetShared`, and+  `Crypto.ECC.Simple.Prim.pointFromIntegers`, now require the point to be in+  the subgroup and report `CryptoError_PointSubgroupInvalid` when it is not.+  The check is `isPointInSubgroup`, newly exported from both prim modules:+  where the cofactor is 1 it answers without work, and otherwise it multiplies+  by the group order and requires the point at infinity, which is what+  OpenSSL's `EC_KEY_check_key` does and costs one further scalar+  multiplication -- an exchange on an affected curve is about twice the price,+  and one on every other curve is unchanged.  The typed `Crypto.ECC` interface+  offers only cofactor-1 curves and dedicated implementations, so nothing+  reaching elliptic curves through it, `tls` among them, was affected++* perf(p256): inline the field arithmetic on AArch64.  `felem_mul` and+  `felem_square` end in `felem_reduce_degree`, a carry chain the whole width+  of the number, and that chain is what their latency is: one product feeding+  the next costs 18.1 ns on an Apple M4, while four independent ones cost 11.4+  ns each.  The curve arithmetic has independent products to offer -- the two+  squarings that open a point doubling, the multiplication and the squaring+  that close it -- but only if the compiler inlines the reduction instead of+  calling it, since a call is a fence.  Plain `inline` does not change its+  mind; `always_inline` does, and it is worth asking where there are registers+  to hold two carry chains at once and not where there are not: 1.23x on an+  M4, 1.12x and 1.05x on a Neoverse under clang and gcc, and 0.95x and 0.82x+  on x86-64, whose fifteen general-purpose registers are not enough.  So it is+  gated on the architecture and x86-64 is left byte-identical.  ECDH P-256 on+  an M4: 69.16 to 56.24 us, against openssl's 24.68, so 0.36 becomes 0.44;+  ECDSA P-256 signing and verification move with it.  The cost is code, 30 to+  116 kilobytes of it+  [#188](https://github.com/kazu-yamamoto/crypton/pull/188)+* perf(number): count bytes from the bit count, not from base 256.  `numBytes`+  asked GMP how many base-256 digits a number has, and GHC's bignum answers+  that by dividing the number down to nothing, one digit at a time, where the+  same question in base two is a look at the highest limb.  On a 2048-bit+  `Integer` that is 1.65 us against 0.01.  Every serialization here asks for+  the size before it allocates and `i2ospOf` asks twice, so the cost landed on+  every RSA, DSA and DH operation leaving the `Integer` world: `i2ospOf_` at+  256 bytes goes from 2.87 to 0.09 us and RSA-2048 verification from 18.4 to+  15.5 us on an M4.  Signing moves by a percent; it is two exponentiations and+  hardly touches this+  [#187](https://github.com/kazu-yamamoto/crypton/pull/187)+* perf(ecc): stop sharing the doublings in the double multiplication.+  `pointAddTwoMuls` was Shamir's trick, one pass over the bits of both scalars+  at once in `Integer` arithmetic, which is the right trade when the two+  multiplications would cost the same.  They have not for a while: `pointMul`+  goes to C, and over a prime field it multiplies the base point through a+  table of its multiples at about a third of the price -- and the base point+  is one of the two, since ECDSA verification is the only caller.  Doing them+  separately and adding: ECDSA P-384 verification 1397 to 698 us on an M4 in+  the typed API, 9839 to 738 in the older one, and a curve over a binary field+  641 ms to 2.6.  P-256 keeps the double multiplication it has in C+  [#186](https://github.com/kazu-yamamoto/crypton/pull/186)+* perf(sha3): take the CRYPTOGAMS Keccak for x86-64 as well.  The same module+  as [#181](https://github.com/kazu-yamamoto/crypton/pull/181) on the other+  architecture, and the reason it was not taken at the time was a measurement+  taken on the wrong machine: crypton on Apple silicon against openssl on+  x86-64, which said there was nothing to gain.  Measured on one machine there+  was: SHA3-256 on an EPYC 7763 goes from 109 to 421 MB/s, against openssl's+  426, so 0.26 becomes 0.99+  [#184](https://github.com/kazu-yamamoto/crypton/pull/184)+* perf(sha1): take the CRYPTOGAMS SHA-1 for AArch64.  The instructions are the+  ones [#170](https://github.com/kazu-yamamoto/crypton/pull/170) put in, and+  the arrangement is what the module has over them: the message schedule of+  the next four rounds runs against the rounds of this one, which is not+  something a C function is going to be made to do --+  [#179](https://github.com/kazu-yamamoto/crypton/pull/179) tried the one+  thing C can do here, handing over a run of blocks, and on this processor it+  measured nothing at all.  The entry point for processors that have the+  instructions is not exported, so the module's own dispatch picks it and the+  answer to the runtime question goes into the word that dispatch reads.  On+  Apple silicon: 3155 to 3379 MB/s at 16 KiB, against openssl's 3350 on the+  same machine, so where this was at 0.94 it is now a shade ahead.  The+  intrinsics stay for the block a message ends with, and for any processor+  that has the instructions but is built without the assembly+  [#182](https://github.com/kazu-yamamoto/crypton/pull/182)+* perf(sha3): take the CRYPTOGAMS Keccak for AArch64.  The instructions are+  the ones [#171](https://github.com/kazu-yamamoto/crypton/pull/171) put in --+  EOR3, RAX1, XAR and BCAX -- and what this module does with them is take a+  run of blocks rather than one at a time, and schedule the round it is in+  against the next one.  The absorb loop hands over the whole run, which also+  drops the alignment trampoline on that path: the assembly reads the message+  as bytes.  SHA3-256 on Apple silicon: 1002 to 1104 MB/s at 16 KiB, against+  openssl's 1058 on the same machine.  Only the absorb side is handed over;+  the squeeze, which SHAKE uses to produce output, is entangled with this+  side's buffer bookkeeping and is not where the time goes+  [#181](https://github.com/kazu-yamamoto/crypton/pull/181)+* perf(xts): double the XTS tweak in the integer registers.  The tweak+  advances by doubling in GF(2^128) once per block, and it was doing that in a+  vector register: six operations on the same units that are running the+  rounds and the exclusive ors, in a chain where each waits for the one+  before.  On a processor whose AES is fast that is not a detail -- taking the+  doubling out of a diagnostic build, which gives the wrong answer but says+  where the time goes, left XTS running at the speed of ECB.  It costs three+  integer operations instead, and the integer units have nothing else to do+  here; what crosses over is one move per block.  On x86-64 that also gets+  eight values out of a register file with sixteen entries, so the round keys+  stay where they were.  AES-128-XTS at 16 KiB: 9644 to 18646 MB/s on Apple+  silicon and 4504 to 7660 on a Haswell-generation x86-64, against openssl's+  17382 and 6997 on the same machines, so both are now a little ahead where+  they were at 0.55 and 0.64.  No assembly: the AArch64 module in CRYPTOGAMS+  has no XTS, and the x86-64 one's is inside a module this does not otherwise+  want+  [#180](https://github.com/kazu-yamamoto/crypton/pull/180)+* perf(sha1): hand the SHA-1 block loop a run of blocks rather than one at a+  time.  A block at a time means the state goes out to memory and comes back+  either side of every block, with the two shuffles that put it in the order+  the instructions want; against the hundred-odd cycles a block costs with the+  SHA extensions that is most of what stood between this and openssl.  On an+  EPYC 7763: 1364 to 1677 MB/s, against openssl's 1670 on the same machine,+  and on a Xeon 8370C 1506 to 1619.  On Apple silicon it measures nothing at+  all -- that processor hides the cost -- and is kept there only so the two+  paths have one shape.  The intended file for this was CRYPTOGAMS'+  `sha1-x86_64.pl`, which turns out to be the 2006 scalar implementation: no+  SSSE3, no AVX, no SHA extensions.  Processors without the extensions are+  therefore where they were, 664 MB/s against openssl's 791 on a Haswell+  [#179](https://github.com/kazu-yamamoto/crypton/pull/179)+* perf(sha2): take the CRYPTOGAMS SHA-256 and SHA-512 for x86-64.  One+  generator gives both, as on AArch64, and each dispatches on what the+  processor has: the SHA extensions, AVX2, AVX, SSSE3 or plain integer code.+  That replaces everything written here for x86-64 -- `sha256_x86.c` and+  `sha512_x86.c` go -- since it is ahead of all of it either way.  At 16 KiB:+  on an EPYC 7763, SHA-256 1430 to 1584 MB/s and SHA-512 423 to 769; on a+  Haswell-generation part, SHA-256 318 to 379 and SHA-512 488 to 593, where+  openssl reports 378 and 589.  The block loops hand over the whole run of+  blocks rather than one at a time, and the alignment trampoline goes with it.+  This also fixes a bug in the capability word+  [#176](https://github.com/kazu-yamamoto/crypton/pull/176) added: bit 29 of+  leaf 7 EBX is the SHA extensions, not an AVX-512 bit, and was being cleared+  along with them -- which cost the SHA-256 assembly two thirds of its speed+  on a processor that has them, and which no machine here could have shown,+  since none has them+  [#178](https://github.com/kazu-yamamoto/crypton/pull/178)+* perf(chacha): take the CRYPTOGAMS ChaCha20 for x86-64 as well.  The C here+  vectorises from eight blocks up and takes anything shorter one block at a+  time, so a message of a few hundred bytes -- a QUIC packet, a small TLS+  record -- ran at a fifth of the bulk rate.  The module has vector code for+  those lengths and is a few per cent ahead in bulk besides: 494 to 1091 MB/s+  at 256 bytes, 477 to 701 at 128, and 2201 to 2374 at 16 KiB, which is+  openssl's 2389 on the same machine.  It is handed everything from one block+  up, where the AArch64 module is handed nothing below three, that one's+  scalar path measuring level with the C.  Keystream generation is still the+  C on both, having no input to exclusive-or+  [#177](https://github.com/kazu-yamamoto/crypton/pull/177)+* perf(poly1305): take the CRYPTOGAMS Poly1305 for x86-64 as well.  The same+  module for the other architecture, through the same three functions, so what+  this adds is the capability word: where the AArch64 one reads+  `crypton_armcap_P`, this one reads `crypton_ia32cap_P`, which is cpuid's own+  words in the order OpenSSL keeps them, filled with the bits for anything the+  operating system will not preserve cleared.  What it brings over the AVX2+  written here is a hand-scheduled scalar path, which is what a message of a+  few hundred bytes actually uses, and an AVX path for machines with no AVX2:+  on a Haswell-generation x86-64, 4298 to 5345 MB/s at 16 KiB and 556 to 1573+  at 64 bytes, against the roughly 5270 openssl reaches there.  `poly1305_avx2.c`+  goes the way the NEON did.  The module's AVX-512 paths are not taken: the+  generator chooses what to emit from the version of the assembler it is told+  about, and it is now told one that predates them, no machine here being able+  to run them and an assembler still in use being unable to assemble them.+  Pinning that version also makes the checked-in assembly independent of the+  host that produced it+  [#176](https://github.com/kazu-yamamoto/crypton/pull/176)+* perf(sha256): take the CRYPTOGAMS SHA-256 for AArch64.  The instructions are+  the ones the intrinsics here already use; what the module does with them is+  schedule them across a whole run of blocks rather than one at a time, and+  keep the message schedule of the next block moving while the rounds of this+  one are still going, which a function that is handed one block and returns+  cannot do whatever it is written in.  So the block loop hands over the whole+  run, which also drops the alignment trampoline on this path -- the assembly+  reads the message as bytes and wants neither the alignment nor the copy.  On+  Apple silicon: 2637 to 3279 MB/s at 16 KiB, against openssl's 3323 on the+  same machine, and 1576 to 1966 at 64 bytes.  SHA-512, which the same+  generator emits, is not taken: 1876 here against openssl's 1880, the+  ARMv8.2 instructions for it having gone in with #110+  [#175](https://github.com/kazu-yamamoto/crypton/pull/175)+* perf(poly1305): take the CRYPTOGAMS Poly1305 for AArch64.  One+  multiplication modulo 2^130 - 5 depends on the one before it, so what there+  is to win is in how the multiplies and the carries are laid against each+  other, and in keeping the accumulator in whichever base costs less: the+  module works in base 2^64 while the message is short and switches to base+  2^26 for the four-way vector loop, deciding that for itself.  Unlike the+  other two it is the whole of the arithmetic rather than a bulk loop bolted+  to the side, so the context now holds either the 26-bit limbs the C works in+  or the 192 bytes the assembly keeps, as a union, and grows from 84 bytes to+  232.  On Apple silicon: 4269 to 8060 MB/s at 16 KiB and 1542 to 4355 at 64+  bytes, and ChaCha20-Poly1305 together, which is what this is for, 1416 to+  2284 against openssl's 2180 on the same machine.  `poly1305_neon.c`, which+  [#169](https://github.com/kazu-yamamoto/crypton/pull/169) added, goes: the+  assembly is faster at every length on every target that gets it, and the+  scalar C remains for the targets that do not.  The tests came first and+  found that the chunking property here had been testing nothing -- it used+  the all-zero key, whose r is zero, so both sides were the nonce whatever+  they did, which is how it came to feed the chunks to `update` in reverse+  order and pass+  [#174](https://github.com/kazu-yamamoto/crypton/pull/174)+* perf(chacha): take the CRYPTOGAMS ChaCha20 for AArch64.  The vector+  registers hold four ChaCha states and there is no room for a fifth, so once+  four blocks are in flight the only place further parallelism can come from+  is the integer side: that module runs a fifth block through the general+  registers alongside four in the vector ones, and above 512 bytes two+  alongside six.  Which register holds which word is the whole of the trick+  and C has no way to say it, which is why the intrinsics here sat at about+  0.63 of what openssl gets out of this very file.  On Apple silicon, a+  message per call: 1911 to 3069 MB/s at 512 bytes, 1913 to 3093 at 4 KiB and+  2056 to 3112 at 64 KiB, against openssl 3.6's 3164 on the same machine,+  which is this code.  It is handed only the states it fits -- twenty rounds,+  a 256-bit key, and as many blocks as the 32-bit counter has room for, since+  crypton's counter is 64 bits wide and carries where the assembly wraps --+  and nothing below 192 bytes, where its own vector path starts.  The tests+  came first: the properties here generated one shape of state, so the+  256-bit constants were never exercised by them+  [#173](https://github.com/kazu-yamamoto/crypton/pull/173)+* perf(gcm): take the CRYPTOGAMS stitched AES-GCM for x86-64, which is the+  first assembly in the package.  Counter-mode AES and GHASH do not compete+  for the same execution ports, so a loop that interleaves them at+  instruction granularity runs both in about the time the rounds alone take;+  written in C that interleaving does not survive the compiler, which sinks+  every multiply to the end of the group, and the disassembly of what+  [#160](https://github.com/kazu-yamamoto/crypton/pull/160) produced says so.+  On a Haswell-generation x86-64, a message per call, AES-128-GCM: 2672 to+  3172 MB/s at 1152 bytes, 3394 to 4271 at 4 KiB and 3657 to 5110 at 16 KiB,+  where openssl speed on the same machine reports 4896; decryption within a+  couple of points of that, and AES-256-GCM 3147 to 4297 at 16 KiB against+  openssl's 4206.  `cbits/asm` holds the module, the translator it needs and the+  generated assembly, one file per object format, so that building needs no+  perl; `cbits/asm/README.md` records where it came from and what was done to+  it, which is to rename the entry points, a program linking both crypton+  and openssl being entitled to object to two definitions of+  `aesni_gcm_encrypt`.  What the assembly reads is laid out OpenSSL's way and+  is built per message in `cbits/aes/gcm_x86_asm.c`, the powers of H being+  the ones crypton already has, shifted up a bit.  Short messages are not+  handed over at all.  `cabal-version` is now 3.0, for `asm-sources`+  [#172](https://github.com/kazu-yamamoto/crypton/pull/172)+* perf(sha3): use the ARMv8.2 SHA-3 instructions.  Keccak was the plain C+  everywhere, a round at a time over tables of rotation amounts and lane+  positions, at half of what openssl manages on the same machine.  EOR3,+  RAX1, XAR and BCAX exist for exactly this permutation and take a round from+  around a hundred and fifty operations to sixty-six; they come with the+  SHA-512 extension the tree already asks for.  Rho and pi move one lane of+  every row into every other row, so the round cannot be done in place, and+  four rounds go in an iteration, which is worth a fifth over one.  SHA3-256+  551 to 991 MB/s (openssl 1064), SHAKE128 700 to 1166, Keccak-256 559 to+  944.  The body is generated from the definitions in FIPS 202 rather than+  copied in, and the script that worked out the rotations and the lane+  permutation checked itself against the published digests of the empty+  string and of "abc" before emitting any C, which is how a first attempt+  with chi reading lanes another row had already overwritten was caught.  x86+  is untouched: nothing there has instructions for this+  [#171](https://github.com/kazu-yamamoto/crypton/pull/171)+* perf(sha1): use the ARMv8 SHA-1 instructions.  The AArch64 paths for+  SHA-256 and SHA-512 went in with #104 and #110 and x86 got its SHA-1+  instructions in [#165](https://github.com/kazu-yamamoto/crypton/pull/165),+  but the AArch64 SHA-1 ones were never used -- and they are part of the same+  optional feature as the SHA-256 ones, so every processor that has those has+  these.  SHA1C, SHA1P and SHA1M each do four rounds with one of the three+  round functions, SHA1H carries E from one group to the next, and SHA1SU0+  and SHA1SU1 do the message schedule between them.  On Apple silicon: 1272+  to 3180 MB/s, against openssl 3.6's 3350 on the same machine.  Checked+  against the hardware rather than through an emulation of the instructions,+  the machine here having them: the digests agree with the generic+  implementation over every message length from 0 to 2000, with each input+  split in two updates+  [#170](https://github.com/kazu-yamamoto/crypton/pull/170)+* perf(poly1305): four blocks at a time with NEON.  AArch64 had only the+  scalar loop, whose five 26-bit limbs and 32-bit multiplies are the shape a+  32-bit machine wants.  This is the arithmetic of the AVX2 path in NEON,+  written as a transliteration of that file rather than a fresh formulation,+  since the maths there is already pinned by the known-answer tests; what+  differs is the width, AVX2 holding four 64-bit products in a register where+  NEON holds two, so each product becomes a pair and the limbs are packed+  back into four 32-bit lanes before the next multiply.  On Apple silicon:+  Poly1305 2783 to 4840 MB/s, and ChaCha20-Poly1305 together 1164 to 1368.+  Checked against the scalar implementation over forty keys and every message+  length from 0 to 400, with each input split in two updates.  Also measured+  and left alone: BLAKE2b at 1612 MB/s against openssl's 1378, the reference+  C being the faster of the two+  [#169](https://github.com/kazu-yamamoto/crypton/pull/169)+* perf(modes): stop the generic cipher modes allocating per byte.  Counter+  mode with a cipher whose modes are not in C ran at a third of what the same+  cipher managed in ECB, and at an eighth for Blowfish, for two reasons+  outside the cipher.  The counters were built one at a time by `ivAdd`,+  which allocates a block and walks the whole width of the counter from the+  original for each of them; they are now one buffer filled in place.  And+  the exclusive or was `Data.ByteArray`'s, which walks a byte at a time+  through an IO applicative -- 420 MB of heap for 8 MiB of counter mode,+  against 17 MB for the same data through ECB, which is fifty bytes allocated+  per byte produced and cost more than the cipher did.  There is a+  `crypton_memxor` to call instead, a pass of words, which the modes and CMAC+  use.  The serial modes also took each block as a copy and take shared+  slices now.  On Apple silicon, counter mode: Camellia-128 79.7 to 285.9+  MB/s, Blowfish 60.4 to 282.6, DES 46.2 to 114.9, CAST5 40.2 to 86.6,+  Twofish-128 37.4 to 57.6, 3DES 25.2 to 36.3, and CBC and CMAC by a third to+  a half as much again.  AES is unchanged: its modes are in C and never came+  this way+  [#168](https://github.com/kazu-yamamoto/crypton/pull/168)+* build: compile the C at -O3, which is what came of looking at P-256 against+  openssl.  The comparison in the problem list was wrong -- a base point+  multiplication here against openssl's ECDH, which is a variable point one --+  and measured properly P-256 is 2.8 to 3.3 times slower rather than the 1.27+  claimed.  The time is in the field arithmetic, five 51-bit limbs in+  Montgomery form at 44.4 ns a multiplication, against hand-written assembly+  using `mulx`, `adcx` and `adox`; a four-limb saturated Montgomery+  multiplication written in C to see what a compiler would give measured 41.3+  ns, so that is not the way in.  What did move is the optimisation level GHC+  passes: a P-256 base point multiplication goes from 71.0 to 59.8 us on x86-64+  and 26.0 to 24.3 on Apple silicon, AES-128-GCM from 3455 to 3708 MB/s and+  AES-128-OCB from 2187 to 2484, with ChaCha20, Poly1305, SHA-1 and MD5 within+  a couple of per cent either way.  The masked selections in the curve and+  field code compile to no conditional jumps at either level+  [#167](https://github.com/kazu-yamamoto/crypton/pull/167)+* refactor(aes): drop the keystream generator nobody can call.  `genCTR` and+  `genCounter` are exported from a module in `other-modules`, so nothing+  outside the library could reach them and nothing inside used them; the only+  mention left was a test commented out since the cryptonite days.  They were+  also the slowest thing in the file, a block at a time through the+  single-block entry point at 715 MB/s where counter mode does 5788, and the+  three ways of fixing that are each worse than removing them: counter mode+  over zeros costs Apple silicon a fifth, counters through ECB costs both, and+  a keystream loop written out per key size is eighty lines for an API no+  caller can see.  Also declares `crypton_aes_encrypt_ctr` and+  `crypton_aes_encrypt_c32` in the header, which had them defined and imported+  but never declared+  [#166](https://github.com/kazu-yamamoto/crypton/pull/166)+* perf(sha1): use the Intel SHA extensions on x86-64.  The extension that+  carries the SHA-256 instructions carries four for SHA-1 as well, and the same+  cpuid bit answers for both, so this is one file and one branch.  On an AMD+  EPYC 7763: 725.1 to 1363.8 MB/s, against openssl's 1668.2 on the same+  machine.  1.9x, where the SHA-256 instructions were worth 4.8x -- SHA-1's+  rounds are cheaper to begin with, so there is less for an instruction to+  replace.  The sequence was checked by replacing the four instructions with C+  that follows the SDM and comparing against the generic implementation over+  every length from 0 to 1024, which found the same missing schedule step+  [#155](https://github.com/kazu-yamamoto/crypton/pull/155) had+  [#165](https://github.com/kazu-yamamoto/crypton/pull/165)+* docs(sidechannel): say what the modules that still work in `Integer` keep+  from the clock, and fix the two places where something could be done about+  it.  ElGamal inverted the shared secret with the extended Euclidean+  algorithm, whose steps follow the bits it is given -- the modulus is prime,+  so Fermat reaches it.  Its signing inverts the ephemeral value modulo an even+  number, where Fermat does not reach, so `sign` blinds instead: the algorithm+  is handed that value times a fresh random unit and the blinder divided out+  afterwards.  What is left is written down rather than fixed -- the Jacobi+  symbols Rabin takes modulo its private primes, and the cost of `Integer`+  arithmetic following the size of the numbers -- and `Crypto.Cipher.AES` now+  says which implementation a machine gets and that the fallback, being+  table-driven, is not constant time+  [#164](https://github.com/kazu-yamamoto/crypton/pull/164)+* perf(poly1305): shorten the carry chain and stop the AVX2 loop spilling.  The+  carries go in pairs, since the two halves of that chain do not depend on each+  other; the powers of r are read from memory, there being sixteen registers+  and ten of them wanted for the accumulator and the products; and the message+  is added limb by limb as the block comes apart rather than five limbs being+  formed first.  4203 to 4452 MB/s, and ChaCha20-Poly1305 together from 1412 to+  1488.  What is left is the instruction count: 107 per 64 bytes, of which 25+  are the multiply+  [#163](https://github.com/kazu-yamamoto/crypton/pull/163)+* perf(chacha): combine as the keystream comes out of the registers.  All three+  vector implementations wrote it to a buffer on the stack and read it back to+  exclusive-or it with the input, which is a pass over every byte for something+  the registers were already holding.  2128 to 2230 MB/s on x86-64, and nothing+  on Apple silicon, where the round trip was free.  Measured while doing it:+  the AVX2 path already did eight blocks at a time, and what is left of the gap+  to openssl in this AEAD is Poly1305 rather than the cipher+  [#162](https://github.com/kazu-yamamoto/crypton/pull/162)+* perf(sha): compute the message schedule in vector registers on x86.  SHA-512+  has no instruction there and SHA-256 has none on a processor older than+  Goldmont or Zen, which includes the Ice Lake and Cascade Lake server parts.+  The rounds are a chain and stay where they are; the schedule is a quarter of+  the work, comes out four words at a time and depends on nothing but the+  message, so it goes into the vector registers and runs alongside rounds that+  need the general ones.  SHA-256 228 to 314 MB/s, SHA-512 354 to 480+  [#161](https://github.com/kazu-yamamoto/crypton/pull/161)+* perf(gcm): take the GHASH of the group before, alongside this group's rounds.+  Held a group apart the multiply and the rounds run through each other, where+  in step neither could start until the other finished.  With it, the multiply+  called directly rather than through a branch pointer the compiler cannot see+  through, and the round keys read from memory rather than spilled: AES-128-GCM+  2797 to 3458 MB/s and AES-256-GCM 2458 to 3053.  openssl does 4895 and 4205+  on the same machine; the rest of that is instruction-level interleaving,+  which does not survive being written in intrinsics+  [#160](https://github.com/kazu-yamamoto/crypton/pull/160)+* perf(ocb): drive OCB through the ECB paths a group at a time.  It ran one+  block at a time through the single-block entry point and so cost four times+  what GCM costs, for a mode that does less work than GCM.  The offsets have to+  be worked out in order but the block cipher calls under them do not depend on+  each other, so eight go through ECB together.  OCB-128 1130 to 3500 MB/s on+  Apple silicon and 694 to 2173 on x86-64, the authenticated data 1141 to 5900+  and 692 to 2526.  CCM is unchanged and stays that way: what is left there is+  CBC-MAC, where each block waits for the one before it+  [#159](https://github.com/kazu-yamamoto/crypton/pull/159)+* test(aes): run the XTS vectors, and add OCB and CCM at 192 and 256 bits.  The+  XTS known-answer tests never ran: the call was commented out and the test it+  would have called did not compile, so vectors at both key sizes sat in the+  tree unused.  XTS is defined only for a 128-bit block, which the general KAT+  runner cannot promise, so it gains a counterpart for a cipher that can.  OCB+  and CCM had vectors at 128 bits only.  2613 examples to 2679+  [#158](https://github.com/kazu-yamamoto/crypton/pull/158)+* perf(aes): build the AArch64 key schedule with the instructions rather than+  the S-box table.  The AArch64 path expanded a key by calling the generic+  implementation and then inverting the round keys, so every schedule went+  through sixteen lookups at addresses derived from the key -- a small thing+  next to the per-block indexing the extensions exist to remove, but a key+  schedule is what an attacker most wants out of a cache, and x86 has never+  needed the table.  AArch64 has no counterpart to AESKEYGENASSIST, but AESE+  against a zero key is SubBytes and ShiftRows, and a word given to it in all+  four columns comes back as SubWord in each of them.  The words stay in+  vector registers throughout, which is what makes it free: moving each one to+  a general register for the instruction and back cost more than the+  instruction did, 87 to 144 ns for an AES-128 schedule, where keeping them in+  registers gives 81.4+  [#157](https://github.com/kazu-yamamoto/crypton/pull/157)+* perf(aes): AES-192 through the processor's AES instructions.  Every 192-bit+  slot in the branch table was left at the generic code, on x86 and on AArch64+  alike, so a 192-bit key got the table-driven software AES while 128 and 256+  got the instructions.  It was 164 times slower for counter mode on the x86+  machine measured and 62 on Apple silicon, and it was also the only key size+  whose data path indexes a table with bytes derived from the key -- a caller+  who picks AES-192 over AES-128 for a wider margin was quietly given a weaker+  one.  Counter mode then GCM, before and after: Apple silicon 152.7 to 9452.0+  MB/s and 112.3 to 7049.3, x86-64 40.4 to 6635.1 and 39.9 to 2633.5.  Both+  implementations were already written once per key size, so this instantiates+  them again at twelve rounds; x86 also needed the 192-bit schedule, which+  does not fall into 128-bit pieces the way the other two do+  [#156](https://github.com/kazu-yamamoto/crypton/pull/156)+* perf(sha256): use the Intel SHA extensions on x86-64, which is what issue+  [#31](https://github.com/kazu-yamamoto/crypton/issues/31) reports -- SHA-256+  four to eight times slower than sha256sum and openssl, both of which use the+  processor's instructions.  AArch64 got its instructions in #104 and is at+  parity with them; x86 had nothing.  SHA256RNDS2 does two rounds at a time and+  SHA256MSG1 and SHA256MSG2 help with the message schedule, so a block costs+  four groups of sixteen instructions instead of sixty-four rounds of scalar+  work.  On an AMD EPYC 9V74: 338.3 to 1612.7 MB/s, against openssl's 1783.8 on+  the same machine.  The extensions arrived with Goldmont and Ice Lake at Intel+  and with Zen at AMD, far later than AES-NI, so a processor without them is+  ordinary rather than ancient: the code sits behind a target attribute and a+  cpuid question, and the plain C stays for everything else+  [#155](https://github.com/kazu-yamamoto/crypton/pull/155)+* perf(bcrypt): Blowfish, and the key setup bcrypt wraps it in, in C.  bcrypt+  is a cost parameter and a promise that the cost is paid, and what pays it is+  the Blowfish key schedule; in Haskell that cost about twice what the usual+  implementations charge, so a hash of a given length of time had to be asked+  for with a lower cost than elsewhere.  Cost 8 goes from 25.97 to 9.98 ms,+  cost 10 from 102.01 to 39.79, cost 12 from 418.78 to 159.41, `bcrypt_pbkdf`+  from 109.76 to 40.38, and Blowfish over 4 KiB from 0.05 to 0.01 -- at cost+  10 that is 39.8 ms against the 52 `htpasswd` takes on the same machine.  The+  Haskell cipher goes with it, so there is one implementation rather than two,+  and nothing exposed changes+  [#154](https://github.com/kazu-yamamoto/crypton/pull/154)+* perf(prime): fewer Miller-Rabin rounds for a candidate nobody chose.  A+  number handed over may have been built to pass, and against that the only+  thing to go on is that a round catches three quarters of the composites+  there are, so `isProbablyPrime`, `findPrimeFrom` and `findPrimeFromWith`,+  which all take their number from the caller, keep their thirty rounds.  A+  candidate drawn here is the case Damgard, Landrock and Pomerance worked out+  and Table 4.4 of the Handbook of Applied Cryptography tabulates:+  `generatePrime` and `generateSafePrime` now use twice what it asks for one+  chance in 2^80, capped at the thirty they had, which leaves the chance far+  under one in 2^100 at every size.  With the candidates held fixed,+  `generatePrime 1024` goes from 28.5 to 16.9 ms and an RSA-2048 key from 52.9+  to 39.2+  [#153](https://github.com/kazu-yamamoto/crypton/pull/153)+* fix(rsa): work the private exponent out without the extended Euclidean+  algorithm.  The modulus is the secret there, so multiplying the value by a+  random number hides nothing; what does is that `e` is public.  Whatever `d`+  is, `e * d = 1 + k * phi` for some `k` under `e`, and reading that modulo+  `e` gives `k` as an inverse modulo a number of a handful of bits, which for+  a prime `e` is Fermat; `d` is then an exact division.  What phi touches is a+  remainder and a division, and nothing in either follows it+  [#152](https://github.com/kazu-yamamoto/crypton/pull/152)+* perf(f2m): ask aarch64 for its carry-less multiply as well.  #148 used PMULL+  only where the compiler had been told the machine has the crypto+  extensions, which is so on Apple and not on a Linux built for the bare+  ARMv8 baseline, though every processor that runs such a build has it.  It is+  now compiled behind an attribute and the machine asked at run time, through+  the auxiliary vector on Linux and Android, elf_aux_info on FreeBSD and a+  sysctl on Apple: sect283k1 421.2 to 168.9 us there, sect571r1 2289.2 to+  579.9+  [#151](https://github.com/kazu-yamamoto/crypton/pull/151)+* refactor(ecc): one multiplication for both of the curve APIs, and one place+  for each buffer's size.  Which path a point multiplication takes was written+  out twice, and the copy in `Crypto.ECC.Simple.Prim` cannot be reached from+  outside the library on a curve over a binary field, so the suite never ran+  it; it moves to the internal module both already share, which makes the copy+  nobody can call the same code everybody runs.  The two buffers for a C call+  that still had their size written out separately from the offsets into them+  now take both from one list, as the one that was wrong in #141 does -- the+  note there records that neither valgrind nor the debug RTS catches that+  mistake, both having been tried+  [#150](https://github.com/kazu-yamamoto/crypton/pull/150)+* perf(f2m): use the x86 carry-less multiply where the processor has it.+  PCLMULQDQ is not part of the x86-64 baseline, so the cpuid the package+  already runs for AES-NI reports one more bit and the multiplication that+  uses the instruction sits behind an attribute.  Measured through Rosetta,+  which translates rather than runs it, so the ratio is what to read:+  sect283k1 560.4 to 177.5 us, sect571r1 3049.6 to 623.9+  [#149](https://github.com/kazu-yamamoto/crypton/pull/149)+* perf(f2m): do the binary field arithmetic in C.  The ladder of #142 spent+  nearly all its time on one thing -- a carry-less multiplication, which+  ordinary arithmetic does not give and which in Haskell was `Integer` shifts+  and exclusive ors, about 4 us for a 283-bit multiplication.  The field and+  the ladder over it are now C, with the processor's instruction where there+  is one and four interleaved groups of bits where there is not, folding for+  the reduction and Fermat for the inverse.  sect163k1 3431 to 67.4 us with+  the instruction and 117.3 without, sect283k1 10181 to 157.6 and 386.2,+  sect571r1 40469 to 521.6 and 2101.0.  It is also constant time, which the+  Haskell ladder was not+  [#148](https://github.com/kazu-yamamoto/crypton/pull/148)+* perf(bignum): start the doubling for `R^2 mod m` at the highest power of two+  under the modulus rather than at one, which for a modulus that fills its+  limbs is half the steps.  Two to three percent of a curve operation, and+  every curve operation and every `expSafe` pays for it once.  Folding instead+  of Montgomery for the primes shaped `2^k - c` was written and measured+  alongside it and is not here: it is slower in this representation, 87.8 ns+  against 76.8 for a 521-bit multiplication, because the shift down by `k`+  costs more than the reduction pass it replaces when `k` does not land on a+  limb boundary+  [#147](https://github.com/kazu-yamamoto/crypton/pull/147)+* perf(ecc): keep a table of the multiples of each curve's base point, which+  is the point signing and making a key multiply and the only one worth a+  table.  A multiplication with it is one addition per four bits and no+  doublings: secp256k1 211.1 to 59.8 us, secp384r1 519.3 to 144.2, secp521r1+  1047.7 to 283.0, and ECDSA P-384 signing 556.4 to 179.9 on both elliptic+  curve APIs, which share the table.  A table is built when a curve is first+  asked for one -- 2.8 ms for secp256k1, 5.5 for secp384r1, 10.6 for secp521r1+  -- and is 221 KB and 456 KB for the last two, so it pays for itself after+  about fifteen multiplications+  [#146](https://github.com/kazu-yamamoto/crypton/pull/146)+* perf(bignum): take the limbs four and two at a time as well as eight in the+  loop every modular multiplication is built out of.  Four and six limbs, which+  is what most of the curves want, fell entirely to the one-at-a-time tail+  before: a field multiplication at six limbs goes from about 58 to 49 ns,+  secp384r1 scalar multiplication from 596.7 to 519.3 us and ECDSA P-384+  signing from 645.0 to 556.4.  Specialising the sizes further, which is what a+  generated implementation would do, measures about 4% more and is not here+  [#145](https://github.com/kazu-yamamoto/crypton/pull/145)+* fix(rsa): keep the blinding factor out of the extended Euclidean algorithm.+  The blinder is a random number and its inverse, and the inverse went through+  an algorithm whose steps follow the number handed to it -- the number the+  blinding rests on, and unlike the other inverses this one is worked out once+  per operation rather than once per key.  `n` being composite leaves no+  Fermat to fall back on, so the algorithm is handed the factor multiplied by+  sixteen fresh random bytes and its answer multiplied by them again, which+  leaves the inverse wanted and shows the algorithm nothing to do with it.  In+  IO, where every draw of randomness goes to the system, `generateBlinder`+  goes from 74 to about 120 us and a PKCS#1 v1.5 `signSafer` from 719 to about+  765; under a DRG the caller carries, 24.7 to 24.9+  [#144](https://github.com/kazu-yamamoto/crypton/pull/144)+* fix(rsa): work `qinv` out without the extended Euclidean algorithm.  Making+  a key inverts one prime modulo the other and both of them are the key+  itself, so that inverse is now Fermat's little theorem through `expSafe`,+  which the other prime being prime allows: 308.6 us against 10.1, on a key+  that takes tens of milliseconds to make.  Making a key cannot be constant+  time -- the search for the primes takes as long as it takes -- but what that+  leaks is about the search rather than about the primes it settles on, and+  the haddock now says which is which+  [#143](https://github.com/kazu-yamamoto/crypton/pull/143)+* perf(ecc): a ladder for the curves over a binary field.  These were the last+  multiplication whose cost followed the scalar: an affine double-and-add, one+  addition for every bit that was set and none for the others, which on+  sect283k1 ran from 9665 us for a scalar with two bits set to 17958 for one+  with 270.  It is now Montgomery's ladder, which carries the multiples of two+  consecutive numbers -- their difference being the point is what lets it+  carry only their x coordinates -- and spends one addition and one doubling+  on every bit whichever way it goes, working the y out at the end from the+  two x it is left with, so one division does for the whole multiplication+  where the affine code had one per step.  The multiplication is now flat, and+  quicker: sect163k1 4428 to 3431 us, sect233r1 9304 to 6806, sect283k1 13797+  to 10181, sect409k1 30826 to 20584, sect571r1 61931 to 40469.  Uniform is+  not constant time -- these are `Integer` operations, whose cost follows the+  values -- and the point with no x, which is its own negation, keeps the code+  that was there+  [#142](https://github.com/kazu-yamamoto/crypton/pull/142)+* perf(ecc): multiply points in C on curves over a prime field.  P-256 has had+  a C implementation all along; every other prime curve -- P-384, P-521,+  secp256k1 and the rest -- multiplied points with `Integer` arithmetic, which+  cannot be constant time, since what an `Integer` operation costs follows the+  value it is given.  The C walks four bits of scalar at a time, taking the+  multiple to add from a table of sixteen that it reads by touching every+  entry and keeping one with a mask, and its addition and doubling are the+  complete formulas of Renes, Costello and Batina, which answer for every pair+  of points with no case to choose between.  A P-384 multiplication goes from+  1557 to 585 us and no longer follows the scalar, ECDSA P-384 signing from+  1700 to 636 us, P-521 from 1942 to 1123.  Binary curves are unchanged, and a+  point that is not on the curve keeps the answer the Haskell gives it+  [#141](https://github.com/kazu-yamamoto/crypton/pull/141)+* fix(ecc): add at every bit in the prime-curve multiplication, which laziness+  was skipping.  The multiplication adds at every bit, set or not, so that its+  cost follows the width of the curve's order rather than the scalar -- but+  the addition was a binding only one branch of the following `if` used, so at+  a bit that was not set it stayed a thunk and was never worked out.  The cost+  followed the number of bits set in the scalar, which is the nonce when+  signing and the private key in ECDH: on P-384, 765.8 us for a scalar with+  two bits set against 1671.5 for one with 383, in a straight line between.+  Both copies of the multiplication had it, so both elliptic curve APIs were+  affected on every prime curve but P-256+  [#140](https://github.com/kazu-yamamoto/crypton/pull/140)+* fix(ecdsa): keep the P-256 signature out of `Integer` arithmetic.  The+  scalar handed to the C implementation was reduced with `mod`, a division,+  whose steps follow the number being divided -- the nonce when signing, the+  private key in ECDH.  Twice the order is more than 256 bits hold, so a+  scalar that fits is brought under the order by one masked subtraction+  instead.  The second half of a signature, `kInv * (z + r * d)`, was+  `Integer` arithmetic as well, and now goes through `scalarAdd` and+  `scalarMul`, which on P-256 are the C implementation's fixed-width+  arithmetic.  What is left on that curve is the conversion between `Integer`+  and fixed-width scalars, which is also what it costs: signing goes from 34.1+  to 39.3 us, and ECDH and the other curves are unchanged+  [#139](https://github.com/kazu-yamamoto/crypton/pull/139)+* fix(dsa,ecdsa): invert the signing nonce without a side channel.  Both+  inverted it with the extended Euclidean algorithm, whose step count and+  branches follow the bits of what it is given -- and a handful of signatures+  whose nonces are partly known give the private key away, so the nonce is+  worth as much as the key.  `Crypto.Number.ModArithmetic.inverseSafe` works+  the inverse out with Fermat's little theorem through `expSafe` instead,+  falling back on `inverse` when the modulus turns out not to be prime, so+  every answer is the one it was.  On P-256 the C implementation does it.+  Signing costs a little more: ECDSA P-256 30.6 to 34.1 us, ECDSA P-384 678.7+  to 703.4, DSA-2048 422.5 to 437.1.  Verification inverts a value that+  arrives in the signature and is left alone+  [#138](https://github.com/kazu-yamamoto/crypton/pull/138)+* perf(number): square, and multiply, faster in `expSafe`.  The product and+  the Montgomery reduction are now a full product followed by a reduction+  rather than interleaved, built out of one loop that takes its limbs eight at+  a time, and squaring works out only the products on one side of the diagonal+  and doubles their sum.  At 2048 bits the constant-time exponentiation goes+  from 3.59 to 2.15 ms, which is 1.4x GMP's own rather than 2.4x; RSA-2048+  signing goes from 0.80 to 0.63 ms and DH-2048 `getShared` from 2.43 to 1.67+  [#137](https://github.com/kazu-yamamoto/crypton/pull/137)+* fix(number): make `expSafe` hide the exponent again.  It asked integer-gmp+  for `powModSecInteger` and fell back on the ordinary `powModInteger` when+  that was missing; since integer-gmp 1.1 it is always missing, so on every+  GHC this package supports `expSafe` was the same windowed exponentiation as+  `expFast`, table indexed by the exponent's bits, for RSA, DSA, DH, ElGamal+  and Rabin alike.  It now goes to C: four bits of exponent at a time, the+  table of sixteen read by touching every entry and keeping one with a mask,+  and a Montgomery multiplication whose final subtraction is masked too.  The+  exponent's length is still visible, rounded up to a whole 64-bit word, which+  is what GMP's own `mpz_powm_sec` lets slip.  Hiding the exponent costs 1.6x+  at 512 bits and 2.4x at 2048: RSA-2048 signing goes from 0.46 to 0.80 ms and+  DH-2048 `getShared` from 1.04 to 2.43+  [#136](https://github.com/kazu-yamamoto/crypton/pull/136)+* perf(prime): stop running a Fermat test that Miller-Rabin subsumes.  Every+  candidate was tested to base 2 before the Miller-Rabin rounds, which begin+  with the same base and prove more; the primes it passed paid for it twice+  and the composites it caught were nearly all caught by trial division first.+  RSA-2048 key generation goes from 55.0 to 32.8 ms+  [#135](https://github.com/kazu-yamamoto/crypton/pull/135)+* perf(f2m): reduce the binary field by folding the top back in rather than+  taking a step per bit of excess, square a byte at a time through a table of+  the patterns a byte spreads into, and take four bits of a multiplier at a+  time rather than one.  On the 283-bit field, squaring goes from 5440 to 2068+  ns and multiplication from 7526 to 4086.  A scalar multiplication there is+  still affine, so it inverts once per addition, which is where its time now+  goes+  [#134](https://github.com/kazu-yamamoto/crypton/pull/134)+* perf(ecc): fold instead of dividing in the generic prime-curve arithmetic,+  and add the point being multiplied as the affine point it is.  These primes+  are `2^k - c` with `c` far smaller, so the top half of a product folds back+  in with a shift, a multiplication and an addition, where dividing costs four+  times as much -- above 256 bits, below which the folding costs more than it+  saves.  P-521 scalar multiplication goes from 892 to 492 us and P-384 from+  684 to 572+  [#133](https://github.com/kazu-yamamoto/crypton/pull/133)+* perf(ecc): route P-256 through the C implementation the library already had.+  `Crypto.PubKey.ECDSA` reached `cbits/p256`; `Crypto.PubKey.ECC.*`, the older+  and more widely used API, never did.  ECDSA signing goes from 590 to 28.7 us,+  verification from 726 to 91.4, and `getShared` from 1177 to 96.  On P-256+  that multiplication is now constant time, where the generic code branches on+  the scalar at every bit+  [#132](https://github.com/kazu-yamamoto/crypton/pull/132)+* Breaking change: perf(camellia): put Camellia in C, 40 to 321 MiB/s.  The+  round function ran a byte at a time in Haskell; generating the tables that+  take a byte straight to its contribution gained 14%, and the rest was the+  language.  Input that is not a whole number of blocks now raises, where the+  tail of the answer used to be uninitialised memory+  [#131](https://github.com/kazu-yamamoto/crypton/pull/131)+* Breaking change: perf(twofish): walk the blocks once and carry them in words+  rather than appending each result to what came before and going through lists+  per block.  2 MiB goes from 0.14 to 56 MiB/s, and the rate no longer falls as+  the message grows.  Input that is not a whole number of blocks now raises,+  where it used to come back longer than it went in+  [#130](https://github.com/kazu-yamamoto/crypton/pull/130)+* perf(modes): cut the message without copying the rest of it in the generic+  block cipher modes, which every cipher but AES uses, and hand whole slices to+  the cipher in the modes whose blocks do not depend on one another.  Camellia+  in CBC goes from 1.8 to 22.9 MiB/s at 1 MiB, DES CBC decryption from 0.5 to+  83, and every figure is now flat in the message length where it used to fall+  [#129](https://github.com/kazu-yamamoto/crypton/pull/129)+* Breaking change: perf(des): put DES in C.  It was carried over lists of+  `Bool`, one cons cell per bit, with the key schedule recomputed for every+  block: 0.04 MiB/s, and 3DES 0.013, against 105 and 41 for OpenSSL.  They are+  now 112 and 37.  Input that is not a whole number of blocks now raises, where+  the tail of the answer used to be uninitialised memory+  [#128](https://github.com/kazu-yamamoto/crypton/pull/128)+* perf(cmac): slice the message rather than copying what is left of it once per+  block, and chain through CBC, which is what CMAC's chaining is.  A MAC over+  4 MiB goes from 0.36 to 1628 MiB/s, which is the speed of AES-CBC itself+  [#127](https://github.com/kazu-yamamoto/crypton/pull/127)+* fix(rabin): decode OAEP without early exits, as+  `Crypto.PubKey.RSA.OAEP.unpad` has since #91.  The difference is not+  measurable against the cost of mask generation, and is structural: the scan+  across the padding no longer depends on the data+  [#126](https://github.com/kazu-yamamoto/crypton/pull/126)+* Breaking change: fix(rabin): refuse a ciphertext or a signature that is not+  below the modulus, and a ciphertext carrying a leading zero octet.  Squaring+  and the square roots that undo it work modulo n, so Basic and Rabin-Williams+  decrypted `c + n` to whatever `c` decrypted to, and all three schemes verified+  `s + n`, and `-s`, wherever they verified `s`.  `Basic.signWith` also refuses a+  padding whose first octet is zero, which the signature cannot carry: about one+  signature in 256 was one its own `verify` rejected+  [#125](https://github.com/kazu-yamamoto/crypton/pull/125)+* fix(prime): derive the Miller-Rabin witnesses from the number being tested and+  from a secret drawn once per process.  They came from one generator made once+  and shared by every call, so the witnesses for one number were the witnesses+  for every number, and testing a number again told the caller nothing it had+  not already been told.  This is the path every GHC since 9.0 takes, integer-gmp+  1.1 having no Miller-Rabin of its own+  [#124](https://github.com/kazu-yamamoto/crypton/pull/124)+* docs(elgamal): say what `signWith` requires of its ephemeral value: the range+  is 1 to p-2, not the "between 0 and p-1" the haddock claimed, and the value is+  a private key that a signature discloses if it is reused or revealed+  [#123](https://github.com/kazu-yamamoto/crypton/pull/123)+* Breaking change: fix(afis): give `split` and `merge` one answer for a parameter+  they cannot use.  They had four between them, including a division by zero for+  an expand count of zero and, for a count of one, handing the diffused data back+  as though it were the secret+  [#122](https://github.com/kazu-yamamoto/crypton/pull/122)+* Breaking change: fix(rsa): refuse a ciphertext or a signature whose integer+  representative is not below the modulus, which RFC 8017 requires in sections+  5.1.2 and 5.2.2.  `PKCS15.decrypt` and `OAEP.decrypt` decrypted `c + n` to the+  same message as `c`, and `PSS.verifyDigest` accepted `s + n` wherever it+  accepted `s`+  [#121](https://github.com/kazu-yamamoto/crypton/pull/121)+* fix(otp): search the HOTP resynchronization window without early exits.  The+  time taken read out both where in the window the client's counter was found+  and how many of the submitted values were right -- the second of which the+  answer itself does not give, being `Nothing` either way.  A call now costs one+  HMAC per counter in the window plus one per extra value, every time+  [#120](https://github.com/kazu-yamamoto/crypton/pull/120)+* Breaking change: fix(kdf): report a refused parameter as a `CryptoError` rather+  than as an `ErrorCall` carrying a string, with a `'`-suffixed variant of each+  entry point returning `CryptoFailable`.  PBKDF2 had no validation at all: a+  negative output length reached `memSet` and killed the process with SIGBUS, and+  an iteration count of zero returned 32 bytes of zeroes+  [#119](https://github.com/kazu-yamamoto/crypton/pull/119)+* perf(xts): take eight blocks at a time on AArch64 and x86-64, and dispatch XTS+  decryption through the branch table, which it had never used.  AArch64 goes+  from 1200 to 7742 MiB/s encrypting and 1166 to 7763 decrypting, x86-64 from+  1220 to 3464 and from 594 to 3461+  [#118](https://github.com/kazu-yamamoto/crypton/pull/118)+* perf(poly1305): take four blocks at a time with AVX2 on x86-64, folding the+  lanes back together weighted by the powers of r.  1347 to 4137 MiB/s+  [#117](https://github.com/kazu-yamamoto/crypton/pull/117)+* perf(ecc): work in Jacobian coordinates in both generic prime-field scalar+  multiplications, and say in `Crypto.ECC` which curves branch on a secret+  scalar.  P-384 and P-521 ECDSA are 2.3x: signing goes from 3.36 to 1.46 ms and+  from 5.96 to 2.61 ms.  P-256, which has its own C implementation, is unaffected+  [#116](https://github.com/kazu-yamamoto/crypton/pull/116)+* Breaking change: fix(padding): bound PKCS#7 padding by the block rather than by+  the whole input, which had let a block of sixteen accept a claim of twenty, and+  refuse a `ZERO` size of zero rather than dividing by it.  What `ZERO` can and+  cannot undo is now written down+  [#115](https://github.com/kazu-yamamoto/crypton/pull/115)+* perf(gcm): give x86 its own GCM decryption loop.  It fell to the generic one,+  which calls the block function once per block, and ran at a quarter the speed+  of encryption; both directions now take eight blocks at a time and fold their+  GHASH into one reduction.  AES-256-GCM decryption goes from 561 to 2733 MiB/s+  and AES-128 from 667 to 3150+  [#114](https://github.com/kazu-yamamoto/crypton/pull/114)+* perf(chacha): take eight blocks at a time with AVX2 where the machine has it,+  with the cpuid and XGETBV checks that decide.  ChaCha20 on x86-64 goes from+  900 to 2074 MiB/s+  [#113](https://github.com/kazu-yamamoto/crypton/pull/113)+* perf(chacha): do four blocks at a time with SSE2 on x86-64, where the cipher+  had no vector code at all.  ChaCha20 goes from 493 to 900 MiB/s+  [#112](https://github.com/kazu-yamamoto/crypton/pull/112)+* perf(chacha): do four blocks at a time with NEON on AArch64.  ChaCha20 goes+  from 1025 to 1955 MiB/s+  [#111](https://github.com/kazu-yamamoto/crypton/pull/111)+* feat(sha512): use the ARMv8.2 SHA-512 instructions on AArch64, which SHA-384+  and the truncated SHA-512/t variants share.  Hashing 1 MiB goes from 1.53 ms+  to 597 us.  The extension is optional, so it is asked for at runtime on both+  Apple and Linux rather than assumed+  [#110](https://github.com/kazu-yamamoto/crypton/pull/110)+* perf(gcm): drive GCM from AArch64 rather than the generic loop, with a group+  of eight blocks folding into a single GHASH reduction.  AES-128-GCM goes from+  4030 to 8266 MiB/s and AES-256 from 4043 to 7172+  [#109](https://github.com/kazu-yamamoto/crypton/pull/109)+* perf(aes): specialise the AArch64 code by key size and interleave eight+  blocks, and give CTR its own loop.  AES-256 ECB goes from 3886 to 15991+  MiB/s, CTR from 2935 to 13567 and CBC decryption from 4366 to 15807+  [#108](https://github.com/kazu-yamamoto/crypton/pull/108)++* perf(aes): build the AES-NI paths on Windows, which was missing from the list of+  systems that compile them.  Windows builds have been doing AES, and GHASH with it,+  in the generic C+  [#107](https://github.com/kazu-yamamoto/crypton/pull/107)+* fix(armv8): compile the AArch64 sources on a toolchain whose baseline lacks the+  crypto extensions.  They had not built with GCC on AArch64 Linux since #100; CI now+  builds and tests there+  [#106](https://github.com/kazu-yamamoto/crypton/pull/106)+* perf(gcm): fold four GHASH blocks into one reduction.  AES-256-GCM is 1.6x at 1 KiB+  and 2.6x at 64 KiB on Apple silicon, and the x86 paths gain the same structure+  [#105](https://github.com/kazu-yamamoto/crypton/pull/105)+* perf(sha256): use the ARMv8 SHA-2 instructions on AArch64.  SHA-256 and SHA-224 are+  5.5x+  [#104](https://github.com/kazu-yamamoto/crypton/pull/104)+* ci: keep the macOS jobs from queueing behind each other, and supersede a branch's+  earlier run+  [#103](https://github.com/kazu-yamamoto/crypton/pull/103)+* perf(aes): use PMULL for GHASH on AArch64+  [#102](https://github.com/kazu-yamamoto/crypton/pull/102)+* ci: ask cabal where its caches live rather than assuming, and keep the build+  products in the cache+  [#101](https://github.com/kazu-yamamoto/crypton/pull/101)+* perf(aes): use the ARMv8 cryptographic extensions on AArch64.  With the GHASH work+  in #102 and #105, AES-256-ECB goes from 121 to 2992 MiB/s and AES-256-GCM from 92 to+  2318 MiB/s on Apple silicon+  [#100](https://github.com/kazu-yamamoto/crypton/pull/100)+* build(bench): move the benchmarks from gauge, which is no longer maintained, to+  tasty-bench, and let them resolve on a current GHC+  [#99](https://github.com/kazu-yamamoto/crypton/pull/99)+* Breaking change: fix(padding): reject a `PKCS7` block size outside 1..255.  `pad`+  raises and `unpad` returns `Nothing`, where both previously narrowed the size to a+  `Word8` and silently agreed on the wrong value+  [#98](https://github.com/kazu-yamamoto/crypton/pull/98)+* feat(elgamal): fix `Crypto.PubKey.ElGamal` and expose it+  [#97](https://github.com/kazu-yamamoto/crypton/pull/97)+* docs(bcrypt): say that only the first 72 bytes of a password count+  [#96](https://github.com/kazu-yamamoto/crypton/pull/96)+* test: move the test suite from tasty to hspec, with hspec-discover.  `cabal-version`+  is now 2.0+  [#95](https://github.com/kazu-yamamoto/crypton/pull/95)++* feat(aead): add `tryAeadSimpleDecrypt`, which takes the tag length as its own argument instead of reading it off the supplied tag+  [#94](https://github.com/kazu-yamamoto/crypton/pull/94)+* Breaking change: feat(dh): add `tryGetShared` to `Crypto.PubKey.DH` and `Crypto.PubKey.ECC.DH`, reporting a rejected peer value as `CryptoFailable`; `getShared` is now defined in terms of it and so raises a `CryptoError` rather than an `ErrorCall`+  [#93](https://github.com/kazu-yamamoto/crypton/pull/93)+* fix(otp): compare TOTP candidates without an early exit+  [#92](https://github.com/kazu-yamamoto/crypton/pull/92)+* fix(rsa): drop the early exits from PKCS#1 v1.5 and OAEP unpadding+  [#91](https://github.com/kazu-yamamoto/crypton/pull/91)+* Breaking change: fix(argon2): report invalid options as `CryptoFailed` rather than raising, adding `CryptoError_ParameterInvalid` to `CryptoError`+  [#90](https://github.com/kazu-yamamoto/crypton/pull/90)+* Breaking change: fix(dh): validate the peer public number, and size the shared secret from `p` rather than `params_bits`+  [#89](https://github.com/kazu-yamamoto/crypton/pull/89)+* fix(dsa): do not crash on values that are not invertible modulo `q`+  [#88](https://github.com/kazu-yamamoto/crypton/pull/88)+* Breaking change: fix(ecdh): validate the peer point before the exchange+  [#87](https://github.com/kazu-yamamoto/crypton/pull/87)+* Breaking change: fix(pkcs15): reject PKCS#1 v1.5 signatures of the wrong length or out of range+  [#86](https://github.com/kazu-yamamoto/crypton/pull/86)+* Breaking change: fix(otp): require a digest long enough for RFC 4226 dynamic truncation, which was reading past the end of the MAC+  [#85](https://github.com/kazu-yamamoto/crypton/pull/85)+* fix(ecc): accept zero-x P-256 shared secret+  [#84](https://github.com/kazu-yamamoto/crypton/pull/84)+* fix(p256): accept valid edge-case points+  [#83](https://github.com/kazu-yamamoto/crypton/pull/83)+* Breaking change: fix(hkdf): enforce output length limit+  [#82](https://github.com/kazu-yamamoto/crypton/pull/82)+* Breaking change: fix(ed25519): reject non-canonical signatures+  [#81](https://github.com/kazu-yamamoto/crypton/pull/81)+* Support GHC 9.14; `tested-with` now covers 9.10.2, 9.12.4 and 9.14.1+  [#74](https://github.com/kazu-yamamoto/crypton/pull/74)++### API changes++* New exports: `Crypto.OTP.minimumDigestSize`, `Crypto.PubKey.DH.tryGetShared`,+  `Crypto.PubKey.ECC.DH.tryGetShared`, `Crypto.Cipher.Types.AEAD.tryAeadSimpleDecrypt`,+  `Crypto.Number.ModArithmetic.inverseSafe`, `Crypto.PubKey.ECC.Prim.scalarInverse`,+  `scalarAdd` and `scalarMul`, `Crypto.PubKey.ECC.P256.scalarReduce`,+  and the whole of `Crypto.PubKey.ElGamal`, which was present but not exposed.+  The variant of an entry point that reports a refusal rather than raising is+  named `try` followed by the name it varies, `tryExpand` beside `expand`.  A+  trailing apostrophe was the obvious spelling and is what these were called+  until shortly before release; it collides too easily, since a caller that+  imports one of these modules unqualified and has its own `expand'` or+  `split'` no longer compiles, and `tls` did.  `Safe` was considered and set+  aside: this library already uses that suffix for something else, in+  `Crypto.Number.ModArithmetic.expSafe` and `inverseSafe` and in+  `Crypto.PubKey.ECC.P256.scalarInvSafe`, where it means the value being+  worked on stays out of the timing.+  The KDFs gained a variant of each entry point that can refuse its parameters,+  returning `CryptoFailable` instead of raising: `Crypto.KDF.Scrypt.tryGenerate`,+  `Crypto.KDF.BCrypt.tryBcrypt`, `Crypto.KDF.BCryptPBKDF.tryGenerate` and+  `tryHashInternal`, `Crypto.KDF.HKDF.tryExpand`, `Crypto.KDF.PBKDF2.tryGenerate` and+  `tryFastPBKDF2_SHA1`, `tryFastPBKDF2_SHA256` and `tryFastPBKDF2_SHA512`, and+  `Crypto.Data.AFIS.trySplit` and `tryMerge`.  These are additions and break nothing.+* Breaking change: `CryptoError_ParameterInvalid` is added to `CryptoError`.  It is+  appended, so the `Enum` values of the existing constructors are unchanged, but an+  exhaustive `case` without a wildcard will warn.  Adding a constructor to an exported+  datatype is what requires a major version bump under the PVP, which would have been+  1.2.0; this release goes to 2.0.0.  Everything else below changes behaviour rather+  than types.+* Breaking change: `getShared` in both DH modules raises a `CryptoError` where it+  previously raised an `ErrorCall`, since it is now defined in terms of `tryGetShared`.+  The same is now true of `Crypto.KDF.Scrypt.generate`, `Crypto.KDF.BCrypt.bcrypt`,+  `Crypto.KDF.BCryptPBKDF.generate` and `hashInternal`, and `Crypto.Data.AFIS.split`+  and `merge`, each of which is defined in terms of the variant above.+* Breaking change: input that used to be accepted is now rejected -- a digest shorter+  than 20 bytes in `Crypto.OTP.hotp`, a signature of the wrong length or out of range+  in `Crypto.PubKey.RSA.PKCS15.verify`, an off-curve peer point or a peer public number+  outside `1 < y < p-1` in `getShared`, an output beyond 255 blocks in+  `Crypto.KDF.HKDF.expand`, a non-canonical Ed25519 signature, and `Options` the+  implementation refuses in `Crypto.KDF.Argon2.hash`.+* Breaking change: a value at or above the modulus is now rejected where it used to be+  reduced and accepted -- a ciphertext in `Crypto.PubKey.RSA.PKCS15.decrypt` and+  `Crypto.PubKey.RSA.OAEP.decrypt`, a signature in `Crypto.PubKey.RSA.PSS.verify`, and+  both, along with a negated signature and a ciphertext with a leading zero octet, in+  the three `Crypto.PubKey.Rabin.*` schemes.+* Breaking change: parameters that used to be accepted are now refused -- an iteration+  count below one or a negative output length in `Crypto.KDF.PBKDF2`, an expand count+  below two or a secret of no bytes in `Crypto.Data.AFIS`, a `PKCS7` claim longer than+  the block and a `ZERO` size of zero in `Crypto.Data.Padding`, and a signature padding+  whose first octet is zero in `Crypto.PubKey.Rabin.Basic.signWith`.+* Breaking change: DES, 3DES, Twofish and Camellia now raise on input that is not a+  whole number of blocks, as AES already did.  Before, DES and Camellia returned an+  answer whose tail was never written -- uninitialised memory -- and Twofish returned+  more than it was given, the missing bytes read as zero.+* Breaking change: `Crypto.Data.Padding.pad` raises on a `PKCS7` block size outside+  1..255, and `unpad` returns `Nothing` for one, where both used to narrow the size to+  a `Word8` and hand back something other than what was padded.+* No exported function changed its signature.++## 1.1.5++* fix(aead): reject undersized tags+  [#80](https://github.com/kazu-yamamoto/crypton/pull/80)+* fix(aes): refuse a zero-length AES-GCM IV+  [#79](https://github.com/kazu-yamamoto/crypton/pull/79)+* fix(p256): prevent crashes when validating valid points+  [#78](https://github.com/kazu-yamamoto/crypton/pull/78)+* feat(asn1): add SHA-3 HashAlgorithmASN1 instances for PKCS#1 v1.5+  [#77](https://github.com/kazu-yamamoto/crypton/pull/77)+* OCB3 conformance+  [#76](https://github.com/kazu-yamamoto/crypton/pull/76)++## 1.1.4++* Generic instance for RSA PublicKey and PrivateKey++## 1.1.3++* Ensure that `pointAdd` in `PubKey.ECC.P256` treats the point at infinity as the additive identity.+  [#73](https://github.com/kazu-yamamoto/crypton/pull/73)++## 1.1.2++* Preparing `ram` v0.22.+* Generalizing RSA encrypt/decrypt to manipulate ScrubbedBytes directly.++## 1.1.1++* On iOS, ScrubbedBytes based hashing is used for seedNew. On other+  plateforms, entropy is used directly as used to be.+  [#71](https://github.com/kazu-yamamoto/crypton/pull/71)++## 1.1.0++* Removing "basement" and "memory".+  [#67](https://github.com/kazu-yamamoto/crypton/pull/67)+++## 1.0.7++* Stop depending on basement, use upstream dependencies instead+* Stop transitively depending on basement by depending on ram.++## 1.0.6++* Fix test failures on less common 64-bit arches.+  [#65](https://github.com/kazu-yamamoto/crypton/pull/65)++## 1.0.5++* Setter/Getter for ChaCha counter.+  [#63](https://github.com/kazu-yamamoto/crypton/pull/63)+* Add simple interface to generate full blocks+  [#60](https://github.com/kazu-yamamoto/crypton/pull/60)+* Avoid `ghc-prim` dependency.+  [#61](https://github.com/kazu-yamamoto/crypton/pull/61)++## 1.0.4++* Ed448.sign: avoid extra re-derive of public key.+  [#48](https://github.com/kazu-yamamoto/crypton/pull/48)++## 1.0.3++* Make sign of Ed25519/Ed448 safer. The public key parameter is+  ignored and its public key is generated from the secret key+  parameter to prevent Double Public Key Signing Function Oracle+  Attack.+  [#47](https://github.com/kazu-yamamoto/crypton/pull/47)++## 1.0.2++* Deterministic Nonce Generation for ECDSA+  [#46](https://github.com/kazu-yamamoto/crypton/pull/46)+* ECDSA Signature Normalization.+  [#45](https://github.com/kazu-yamamoto/crypton/pull/45)+* Add Full Test Suite from RFC 6979.+  [#44](https://github.com/kazu-yamamoto/crypton/pull/44)+* ECDSA with Public Key Recovery.+  [#43](https://github.com/kazu-yamamoto/crypton/pull/43)+* Providing necessary features for HPKE.+  [#42](https://github.com/kazu-yamamoto/crypton/pull/42)++## 1.0.1++* Update decaf library.+  [#38](https://github.com/kazu-yamamoto/crypton/pull/38)+* Add TypeOperators language extension to EdDSA.hs.+  [#36](https://github.com/kazu-yamamoto/crypton/pull/36)++## 1.0.0++* Versions follow the standard version policy.+* Removing pthread stuff.+  [#32](https://github.com/kazu-yamamoto/crypton/pull/32)+ ## 0.34  * Hashing getRandomBytes before using as Seed for ChaChaDRG
Crypto/Cipher/AES.hs view
@@ -1,22 +1,43 @@+{-# LANGUAGE CPP #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.Cipher.AES -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : stable -- Portability : good-{-# LANGUAGE CPP #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.Cipher.AES-    ( AES128-    , AES192-    , AES256-    ) where+--+-- AES, in the modes "Crypto.Cipher.Types" defines.+--+-- == Which implementation runs+--+-- Where the processor has instructions for AES -- AES-NI on x86-64, the+-- cryptographic extensions on AArch64 -- every key size and every mode here+-- goes through them, and a block costs the same whatever the key and the data+-- are.+--+-- Where it does not, the fallback is the table-driven code in+-- @cbits\/aes\/generic.c@, which indexes a 256-byte table with bytes derived+-- from the key and from the block.  That is the cache-timing exposure the+-- instructions exist to remove, and on such a machine AES here is not+-- constant time.  Every x86-64 part since about 2010 and every AArch64 one in+-- ordinary use has the instructions.+--+-- 'Crypto.System.CPU.processorOptions' says which of the two a given machine+-- got: @AESNI@ in that list means the processor's AES instructions, on either+-- architecture.+module Crypto.Cipher.AES (+    AES128,+    AES192,+    AES256,+) where -import Crypto.Error+import Crypto.Cipher.AES.Primitive import Crypto.Cipher.Types-import Crypto.Cipher.Utils import Crypto.Cipher.Types.Block-import Crypto.Cipher.AES.Primitive+import Crypto.Cipher.Utils+import Crypto.Error import Crypto.Internal.Imports  -- | AES with 128 bit key@@ -32,20 +53,19 @@     deriving (NFData)  instance Cipher AES128 where-    cipherName    _ = "AES128"+    cipherName _ = "AES128"     cipherKeySize _ = KeySizeFixed 16-    cipherInit k    = AES128 <$> (initAES =<< validateKeySize (undefined :: AES128) k)+    cipherInit k = AES128 <$> (initAES =<< validateKeySize (undefined :: AES128) k)  instance Cipher AES192 where-    cipherName    _ = "AES192"+    cipherName _ = "AES192"     cipherKeySize _ = KeySizeFixed 24-    cipherInit k    = AES192 <$> (initAES =<< validateKeySize (undefined :: AES192) k)+    cipherInit k = AES192 <$> (initAES =<< validateKeySize (undefined :: AES192) k)  instance Cipher AES256 where-    cipherName    _ = "AES256"+    cipherName _ = "AES256"     cipherKeySize _ = KeySizeFixed 32-    cipherInit k    = AES256 <$> (initAES =<< validateKeySize (undefined :: AES256) k)-+    cipherInit k = AES256 <$> (initAES =<< validateKeySize (undefined :: AES256) k)  #define INSTANCE_BLOCKCIPHER(CSTR) \ instance BlockCipher CSTR where \@@ -55,7 +75,7 @@     ; cbcEncrypt (CSTR aes) (IV iv) = encryptCBC aes (IV iv) \     ; cbcDecrypt (CSTR aes) (IV iv) = decryptCBC aes (IV iv) \     ; ctrCombine (CSTR aes) (IV iv) = encryptCTR aes (IV iv) \-    ; aeadInit AEAD_GCM (CSTR aes) iv = CryptoPassed $ AEAD (gcmMode aes) (gcmInit aes iv) \+    ; aeadInit AEAD_GCM (CSTR aes) iv = gcmAeadInit aes iv \     ; aeadInit AEAD_OCB (CSTR aes) iv = CryptoPassed $ AEAD (ocbMode aes) (ocbInit aes iv) \     ; aeadInit (AEAD_CCM n m l) (CSTR aes) iv = AEAD (ccmMode aes) <$> ccmInit aes iv n m l \     ; aeadInit _        _          _  = CryptoFailed CryptoError_AEADModeNotSupported \
+ Crypto/Cipher/AES/GCM.hs view
@@ -0,0 +1,256 @@+-- |+-- Module      : Crypto.Cipher.AES.GCM+-- License     : BSD-style+-- Maintainer  : Kazu Yamamoto <kazu@iij.ad.jp>+-- Stability   : experimental+-- Portability : unknown+--+-- AES-GCM for callers that send many short messages under one key, which is+-- what a datagram transport does.+--+-- The interface in "Crypto.Cipher.Types" builds a state from the key /and/+-- the nonce and then walks it through appending the additional data,+-- encrypting and finalizing, copying the state at each step.  For a stream+-- that is nothing next to the encryption.  For a QUIC packet it is most of+-- the work: the key schedule and the table of multiples of @H@ depend on the+-- key alone, and rebuilding them for every nonce costs more than encrypting+-- 1440 bytes.+--+-- So here a t'Context' is built from the key once and holds both, and+-- 'encrypt' takes a nonce and a whole message and answers in one call.+--+-- > ctx <- throwCryptoError <$> pure (newContext key)+-- > let packet = encrypt ctx nonce header plaintext 16+--+-- This runs on AES-NI and carry-less multiply, or on the ARMv8 cryptographic+-- extension, and makes no branch and no memory access that depends on the key+-- or on the data.  Where the processor has neither, AES falls back to a table+-- driven implementation that is /not/ constant time; see the side channels+-- section of the README, and 'Crypto.System.CPU.processorOptions' for which is+-- in use.+--+-- The result is the ciphertext with the tag after it, which is the shape a+-- packet wants.  'decrypt' takes that shape back, compares the tag itself and+-- answers 'Nothing' when it does not match.+--+-- This computes the same thing as the general interface; the tests hold it to+-- that on the same vectors.+module Crypto.Cipher.AES.GCM (+    Context,+    newContext,+    encrypt,+    decrypt,+    decryptWithTag,++    -- * Header protection+    HeaderKey,+    newHeaderKey,+    encryptWithMask,+) where++import Crypto.Cipher.AES.Primitive (+    AES,+    AESGCMKey,+    gcmFullDecrypt,+    gcmFullDecryptTag,+    gcmFullEncrypt,+    gcmFullEncryptMask,+    gcmKeyInit,+    initAES,+ )+import Crypto.Cipher.Types (AuthTag)+import Crypto.Cipher.Types.AEAD (minimumTagLength)+import Crypto.Error+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess)+import qualified Crypto.Internal.ByteArray as B+import Data.Word (Word8)+import Foreign.Ptr (Ptr)++-- | Everything a key determines: the AES key schedule and the table of+-- multiples of @H@.  Build it once and encrypt as many messages under it as+-- the key is good for.+data Context = Context !AES !AESGCMKey++-- | Take a key of 16, 24 or 32 bytes.  Any other length is reported as+-- 'CryptoError_KeySizeInvalid'.+newContext :: ByteArrayAccess key => key -> CryptoFailable Context+newContext k = do+    aes <- initAES k+    return $ Context aes (gcmKeyInit aes)++-- | Encrypt one message: the nonce, the additional data that is+-- authenticated but not encrypted, the plaintext, and how many bytes of tag+-- to produce, which GCM allows between 4 and 16.  Any other length throws+-- 'CryptoError_AuthenticationTagSizeInvalid', and so does 'decryptWithTag'.+--+-- The answer is the ciphertext followed by the tag.+--+-- A nonce must not be used twice with the same t'Context'.  Twelve bytes is+-- the size GCM is defined for and the only one that does not cost a further+-- pass.  A nonce of no bytes is refused: it would hand out the key GCM+-- authenticates with, so 'encrypt' and 'decryptWithTag' throw+-- 'CryptoError_IvSizeInvalid' for it, 'decrypt' gives 'Nothing' and+-- 'encryptWithMask' gives 'False'.+{-# INLINABLE encrypt #-}+encrypt+    :: ( ByteArrayAccess nonce+       , ByteArrayAccess aad+       , ByteArrayAccess ba+       , ByteArray output+       )+    => Context+    -> nonce+    -> aad+    -> ba+    -> Int+    -> output+encrypt (Context aes gk) nonce aad input taglen+    | tooLongForC aad input =+        throwCryptoError (CryptoFailed CryptoError_ParameterInvalid)+    | badNonce nonce =+        throwCryptoError (CryptoFailed CryptoError_IvSizeInvalid)+    | badTagLength taglen =+        throwCryptoError (CryptoFailed CryptoError_AuthenticationTagSizeInvalid)+    | otherwise = gcmFullEncrypt aes gk nonce aad input taglen++-- | Decrypt one message, in the shape 'encrypt' produced: the ciphertext with+-- its tag after it.  The tag is compared here, every byte of it whatever the+-- answer, and a message whose tag does not match gives 'Nothing' rather than+-- the plaintext.+--+-- 'Nothing' also comes back when the nonce has no bytes, the input is+-- shorter than the tag, or the tag length is outside 4 to 16.+{-# INLINABLE decrypt #-}+decrypt+    :: (ByteArrayAccess nonce, ByteArrayAccess aad, ByteArray ba)+    => Context+    -> nonce+    -> aad+    -> ba+    -> Int+    -> Maybe ba+decrypt (Context aes gk) nonce aad input taglen+    | tooLongForC aad input = Nothing+    | badNonce nonce || badTagLength taglen || B.length input < taglen = Nothing+    | otherwise = gcmFullDecrypt aes gk nonce aad body tag+  where+    (body, tag) = B.splitAt (B.length input - taglen) input++-- | Decrypt one message, the tag kept apart, and hand back the tag this end+-- computed.+--+-- For a caller whose protocol hands it the tag separately from the+-- ciphertext, so that 'decrypt' -- which wants the two together and compares+-- them itself -- does not fit.  Compare the two tags with '=='; the 'Eq'+-- instance of t'AuthTag' is a constant-time comparison, and taking them apart+-- to compare the bytes is how this goes wrong.+--+-- Nothing here says whether the message is authentic.  Until the comparison+-- is made and has come out equal, what this returns is not plaintext, it is+-- what the ciphertext turns into, and a caller must not act on it.+{-# INLINABLE decryptWithTag #-}+decryptWithTag+    :: (ByteArrayAccess nonce, ByteArrayAccess aad, ByteArray ba)+    => Context+    -> nonce+    -> aad+    -> ba+    -> Int+    -> (ba, AuthTag)+decryptWithTag (Context aes gk) nonce aad input taglen+    | tooLongForC aad input =+        throwCryptoError (CryptoFailed CryptoError_ParameterInvalid)+    | badNonce nonce =+        throwCryptoError (CryptoFailed CryptoError_IvSizeInvalid)+    | badTagLength taglen =+        throwCryptoError (CryptoFailed CryptoError_AuthenticationTagSizeInvalid)+    | otherwise = gcmFullDecryptTag aes gk nonce aad input taglen++----------------------------------------------------------------++-- | The key schedule for header protection, which QUIC keeps separately from+-- the one it encrypts with.  Built once, like a t'Context'.+newtype HeaderKey = HeaderKey AES++-- | Take a header protection key of 16, 24 or 32 bytes.+newHeaderKey :: ByteArrayAccess key => key -> CryptoFailable HeaderKey+newHeaderKey k = HeaderKey <$> initAES k++-- | Encrypt one message and, from a sample of the ciphertext it just+-- produced, make the header protection mask -- in one call, into two buffers+-- the caller already has.+--+-- QUIC takes its sample from the ciphertext, so the mask cannot be had before+-- the encryption.  It can be had before coming back, and with the buffers+-- already there nothing is allocated for either.  On an Apple M4 the mask+-- then costs about 0.02 us, where asking for it separately costs 0.11.+--+-- The sealed message wants @length input + taglen@ bytes and the mask+-- sixteen.  @sampleOffset@ says where the sixteen bytes of sample begin in+-- the sealed message, counting the tag as part of it.+--+-- 'False' comes back, and nothing is written, when the nonce has no bytes,+-- the sample would not fit, or the tag length is outside 4 to 16.+{-# INLINABLE encryptWithMask #-}+encryptWithMask+    :: (ByteArrayAccess nonce, ByteArrayAccess aad, ByteArrayAccess ba)+    => Context+    -> HeaderKey+    -> nonce+    -> aad+    -> ba+    -> Int+    -- ^ tag length+    -> Int+    -- ^ sample offset+    -> Ptr Word8+    -- ^ where the sealed message goes+    -> Ptr Word8+    -- ^ where the sixteen bytes of mask go+    -> IO Bool+encryptWithMask (Context aes gk) (HeaderKey hp) nonce aad input taglen off outp maskp+    | tooLongForC aad input = return False+    | badNonce nonce = return False+    | off < 0 || badTagLength taglen || off + 16 > B.length input + taglen =+        return False+    | otherwise = do+        gcmFullEncryptMask aes gk hp nonce aad input taglen off outp maskp+        return True++-- | The C behind all four takes its lengths as @uint32_t@, so a message or+-- its additional data from 2^32 bytes up cannot be handed to it: the length+-- would be truncated and most of the buffer left untouched, with nothing to+-- say so.  There is no splitting the work here -- the C does the whole+-- message in one call, tag and all -- so such a message is refused.+tooLongForC+    :: (ByteArrayAccess aad, ByteArrayAccess ba) => aad -> ba -> Bool+tooLongForC aad input =+    B.overCLength (B.length aad) || B.overCLength (B.length input)++-- | SP 800-38D 5.2.1.1 asks for at least one byte of IV, and this is why.+--+-- GCM builds its pre-counter block from a nonce that is not twelve bytes as+-- @J0 = GHASH_H(IV || 0^s || [0]_64 || [len(IV)]_64)@.  For an empty IV that+-- input is one block of zeros, so @J0@ is zero, and the tag of a message+-- becomes @GHASH_H(A, C) XOR E(K, 0^128)@ -- where @E(K, 0^128)@ is the+-- definition of @H@.  The tag of an empty message under an empty nonce is+-- therefore @H@ itself, and any other full tag gives @H@ as the root of a+-- known polynomial.+--+-- @H@ belongs to the key, not to the nonce.  An attacker holding it, plus one+-- genuine message under any nonce, has @E(K, J0)@ for that nonce and can make+-- a tag that verifies for data of their own -- under a correct twelve-byte+-- nonce, and for every nonce they have seen.  One encryption with an empty+-- nonce and a full tag ends the authenticity of everything under the key.+--+-- 'Crypto.Cipher.AES.Primitive.gcmAeadInit' refuses the empty IV for this+-- reason, and these four have to as well.  Only the empty one is refused:+-- SP 800-38D allows every length from one byte up.+badNonce :: ByteArrayAccess nonce => nonce -> Bool+badNonce nonce = B.length nonce == 0++-- | GCM makes a sixteen-byte tag and a shorter one is a prefix of it.  Below+-- 'minimumTagLength' it authenticates next to nothing, and past sixteen the+-- C code would read beyond the tag it computed.+badTagLength :: Int -> Bool+badTagLength t = t < minimumTagLength || t > 16
Crypto/Cipher/AES/Primitive.hs view
@@ -1,645 +1,1105 @@-{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE ViewPatterns #-}-{-# LANGUAGE BangPatterns #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}--- |--- Module      : Crypto.Cipher.AES.Primitive--- License     : BSD-style--- Maintainer  : Vincent Hanquez <vincent@snarc.org>--- Stability   : stable--- Portability : good----module Crypto.Cipher.AES.Primitive-    (-    -- * Block cipher data types-      AES--    -- * Authenticated encryption block cipher types-    , AESGCM-    , AESOCB--    -- * Creation-    , initAES--    -- * Miscellanea-    , genCTR-    , genCounter--    -- * Encryption-    , encryptECB-    , encryptCBC-    , encryptCTR-    , encryptXTS--    -- * Decryption-    , decryptECB-    , decryptCBC-    , decryptCTR-    , decryptXTS--    -- * CTR with 32-bit wrapping-    , combineC32--    -- * Incremental GCM-    , gcmMode-    , gcmInit--    -- * Incremental OCB-    , ocbMode-    , ocbInit--    -- * CCM-    , ccmMode-    , ccmInit-    ) where--import           Data.Word-import           Foreign.Ptr-import           Foreign.C.Types-import           Foreign.C.String--import           Crypto.Error-import           Crypto.Cipher.Types-import           Crypto.Cipher.Types.Block (IV(..))-import           Crypto.Internal.Compat-import           Crypto.Internal.Imports-import           Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, ScrubbedBytes, withByteArray)-import qualified Crypto.Internal.ByteArray as B--instance Cipher AES where-    cipherName    _ = "AES"-    cipherKeySize _ = KeySizeEnum [16,24,32]-    cipherInit k    = initAES k--instance BlockCipher AES where-    blockSize _ = 16-    ecbEncrypt = encryptECB-    ecbDecrypt = decryptECB-    cbcEncrypt = encryptCBC-    cbcDecrypt = decryptCBC-    ctrCombine = encryptCTR-    aeadInit AEAD_GCM aes iv = CryptoPassed $ AEAD (gcmMode aes) (gcmInit aes iv)-    aeadInit AEAD_OCB aes iv = CryptoPassed $ AEAD (ocbMode aes) (ocbInit aes iv)-    aeadInit (AEAD_CCM n m l) aes iv = AEAD (ccmMode aes) <$> ccmInit aes iv n m l-    aeadInit _        _   _  = CryptoFailed CryptoError_AEADModeNotSupported-instance BlockCipher128 AES where-    xtsEncrypt = encryptXTS-    xtsDecrypt = decryptXTS---- | Create an AES AEAD implementation for GCM-gcmMode :: AES -> AEADModeImpl AESGCM-gcmMode aes = AEADModeImpl-    { aeadImplAppendHeader = gcmAppendAAD-    , aeadImplEncrypt      = gcmAppendEncrypt aes-    , aeadImplDecrypt      = gcmAppendDecrypt aes-    , aeadImplFinalize     = gcmFinish aes-    }---- | Create an AES AEAD implementation for OCB-ocbMode :: AES -> AEADModeImpl AESOCB-ocbMode aes = AEADModeImpl-    { aeadImplAppendHeader = ocbAppendAAD aes-    , aeadImplEncrypt      = ocbAppendEncrypt aes-    , aeadImplDecrypt      = ocbAppendDecrypt aes-    , aeadImplFinalize     = ocbFinish aes-    }---- | Create an AES AEAD implementation for CCM-ccmMode :: AES -> AEADModeImpl AESCCM-ccmMode aes = AEADModeImpl-    { aeadImplAppendHeader = ccmAppendAAD aes-    , aeadImplEncrypt      = ccmEncrypt aes-    , aeadImplDecrypt      = ccmDecrypt aes-    , aeadImplFinalize     = ccmFinish aes-    }----- | AES Context (pre-processed key)-newtype AES = AES ScrubbedBytes-    deriving (NFData)---- | AESGCM State-newtype AESGCM = AESGCM ScrubbedBytes-    deriving (NFData)---- | AESOCB State-newtype AESOCB = AESOCB ScrubbedBytes-    deriving (NFData)---- | AESCCM State-newtype AESCCM = AESCCM ScrubbedBytes-    deriving (NFData)--sizeGCM :: Int-sizeGCM = 320--sizeOCB :: Int-sizeOCB = 160--sizeCCM :: Int-sizeCCM = 80--keyToPtr :: AES -> (Ptr AES -> IO a) -> IO a-keyToPtr (AES b) f = withByteArray b (f . castPtr)--ivToPtr :: ByteArrayAccess iv => iv -> (Ptr Word8 -> IO a) -> IO a-ivToPtr iv f = withByteArray iv (f . castPtr)---ivCopyPtr :: IV AES -> (Ptr Word8 -> IO a) -> IO (a, IV AES)-ivCopyPtr (IV iv) f = (\(x,y) -> (x, IV y)) `fmap` copyAndModify iv f-  where-    copyAndModify :: ByteArray ba => ba -> (Ptr Word8 -> IO a) -> IO (a, ba)-    copyAndModify ba f' = B.copyRet ba f'--withKeyAndIV :: ByteArrayAccess iv => AES -> iv -> (Ptr AES -> Ptr Word8 -> IO a) -> IO a-withKeyAndIV ctx iv f = keyToPtr ctx $ \kptr -> ivToPtr iv $ \ivp -> f kptr ivp--withKey2AndIV :: ByteArrayAccess iv => AES -> AES -> iv -> (Ptr AES -> Ptr AES -> Ptr Word8 -> IO a) -> IO a-withKey2AndIV key1 key2 iv f =-    keyToPtr key1 $ \kptr1 -> keyToPtr key2 $ \kptr2 -> ivToPtr iv $ \ivp -> f kptr1 kptr2 ivp--withGCMKeyAndCopySt :: AES -> AESGCM -> (Ptr AESGCM -> Ptr AES -> IO a) -> IO (a, AESGCM)-withGCMKeyAndCopySt aes (AESGCM gcmSt) f =-    keyToPtr aes $ \aesPtr -> do-        newSt <- B.copy gcmSt (\_ -> return ())-        a     <- withByteArray newSt $ \gcmStPtr -> f (castPtr gcmStPtr) aesPtr-        return (a, AESGCM newSt)--withNewGCMSt :: AESGCM -> (Ptr AESGCM -> IO ()) -> IO AESGCM-withNewGCMSt (AESGCM gcmSt) f = B.copy gcmSt (f . castPtr) >>= \sm2 -> return (AESGCM sm2)--withOCBKeyAndCopySt :: AES -> AESOCB -> (Ptr AESOCB -> Ptr AES -> IO a) -> IO (a, AESOCB)-withOCBKeyAndCopySt aes (AESOCB gcmSt) f =-    keyToPtr aes $ \aesPtr -> do-        newSt <- B.copy gcmSt (\_ -> return ())-        a     <- withByteArray newSt $ \gcmStPtr -> f (castPtr gcmStPtr) aesPtr-        return (a, AESOCB newSt)--withCCMKeyAndCopySt :: AES -> AESCCM -> (Ptr AESCCM -> Ptr AES -> IO a) -> IO (a, AESCCM)-withCCMKeyAndCopySt aes (AESCCM ccmSt) f =-    keyToPtr aes $ \aesPtr -> do-        newSt <- B.copy ccmSt (\_ -> return ())-        a     <- withByteArray newSt $ \ccmStPtr -> f (castPtr ccmStPtr) aesPtr-        return (a, AESCCM newSt)---- | Initialize a new context with a key------ Key needs to be of length 16, 24 or 32 bytes. Any other values will return failure-initAES :: ByteArrayAccess key => key -> CryptoFailable AES-initAES k-    | len == 16 = CryptoPassed $ initWithRounds 10-    | len == 24 = CryptoPassed $ initWithRounds 12-    | len == 32 = CryptoPassed $ initWithRounds 14-    | otherwise = CryptoFailed CryptoError_KeySizeInvalid-  where len = B.length k-        initWithRounds nbR = AES $ B.allocAndFreeze (16+2*2*16*nbR) aesInit-        aesInit ptr = withByteArray k $ \ikey ->-            c_aes_init (castPtr ptr) (castPtr ikey) (fromIntegral len)---- | encrypt using Electronic Code Book (ECB)-{-# NOINLINE encryptECB #-}-encryptECB :: ByteArray ba => AES -> ba -> ba-encryptECB = doECB c_aes_encrypt_ecb---- | encrypt using Cipher Block Chaining (CBC)-{-# NOINLINE encryptCBC #-}-encryptCBC :: ByteArray ba-           => AES        -- ^ AES Context-           -> IV AES     -- ^ Initial vector of AES block size-           -> ba         -- ^ plaintext-           -> ba         -- ^ ciphertext-encryptCBC = doCBC c_aes_encrypt_cbc---- | generate a counter mode pad. this is generally xor-ed to an input--- to make the standard counter mode block operations.------ if the length requested is not a multiple of the block cipher size,--- more data will be returned, so that the returned bytearray is--- a multiple of the block cipher size.-{-# NOINLINE genCTR #-}-genCTR :: ByteArray ba-       => AES    -- ^ Cipher Key.-       -> IV AES -- ^ usually a 128 bit integer.-       -> Int    -- ^ length of bytes required.-       -> ba-genCTR ctx (IV iv) len-    | len <= 0  = B.empty-    | otherwise = B.allocAndFreeze (nbBlocks * 16) generate-  where generate o = withKeyAndIV ctx iv $ \k i -> c_aes_gen_ctr (castPtr o) k i (fromIntegral nbBlocks)-        (nbBlocks',r) = len `quotRem` 16-        nbBlocks = if r == 0 then nbBlocks' else nbBlocks' + 1---- | generate a counter mode pad. this is generally xor-ed to an input--- to make the standard counter mode block operations.------ if the length requested is not a multiple of the block cipher size,--- more data will be returned, so that the returned bytearray is--- a multiple of the block cipher size.------ Similiar to 'genCTR' but also return the next IV for continuation-{-# NOINLINE genCounter #-}-genCounter :: ByteArray ba-           => AES-           -> IV AES-           -> Int-           -> (ba, IV AES)-genCounter ctx iv len-    | len <= 0  = (B.empty, iv)-    | otherwise = unsafeDoIO $-        keyToPtr ctx $ \k ->-        ivCopyPtr iv $ \i ->-        B.alloc outputLength $ \o -> do-            c_aes_gen_ctr_cont (castPtr o) k i (fromIntegral nbBlocks)-  where-        (nbBlocks',r) = len `quotRem` 16-        nbBlocks = if r == 0 then nbBlocks' else nbBlocks' + 1-        outputLength = nbBlocks * 16--{- TODO: when genCTR has same AESIV requirements for IV, add the following rules:- - RULES "snd . genCounter" forall ctx iv len .  snd (genCounter ctx iv len) = genCTR ctx iv len- -}---- | encrypt using Counter mode (CTR)------ in CTR mode encryption and decryption is the same operation.-{-# NOINLINE encryptCTR #-}-encryptCTR :: ByteArray ba-           => AES        -- ^ AES Context-           -> IV AES     -- ^ initial vector of AES block size (usually representing a 128 bit integer)-           -> ba         -- ^ plaintext input-           -> ba         -- ^ ciphertext output-encryptCTR ctx iv input-    | len <= 0          = B.empty-    | B.length iv /= 16 = error $ "AES error: IV length must be block size (16). Its length is: " ++ (show $ B.length iv)-    | otherwise = B.allocAndFreeze len doEncrypt-  where doEncrypt o = withKeyAndIV ctx iv $ \k v -> withByteArray input $ \i ->-                      c_aes_encrypt_ctr (castPtr o) k v i (fromIntegral len)-        len = B.length input---- | encrypt using XTS------ the first key is the normal block encryption key--- the second key is used for the initial block tweak-{-# NOINLINE encryptXTS #-}-encryptXTS :: ByteArray ba-           => (AES,AES)  -- ^ AES cipher and tweak context-           -> IV AES     -- ^ a 128 bits IV, typically a sector or a block offset in XTS-           -> Word32     -- ^ number of rounds to skip, also seen a 16 byte offset in the sector or block.-           -> ba         -- ^ input to encrypt-           -> ba         -- ^ output encrypted-encryptXTS = doXTS c_aes_encrypt_xts---- | decrypt using Electronic Code Book (ECB)-{-# NOINLINE decryptECB #-}-decryptECB :: ByteArray ba => AES -> ba -> ba-decryptECB = doECB c_aes_decrypt_ecb---- | decrypt using Cipher block chaining (CBC)-{-# NOINLINE decryptCBC #-}-decryptCBC :: ByteArray ba => AES -> IV AES -> ba -> ba-decryptCBC = doCBC c_aes_decrypt_cbc---- | decrypt using Counter mode (CTR).------ in CTR mode encryption and decryption is the same operation.-decryptCTR :: ByteArray ba-           => AES        -- ^ AES Context-           -> IV AES     -- ^ initial vector, usually representing a 128 bit integer-           -> ba         -- ^ ciphertext input-           -> ba         -- ^ plaintext output-decryptCTR = encryptCTR---- | decrypt using XTS-{-# NOINLINE decryptXTS #-}-decryptXTS :: ByteArray ba-           => (AES,AES)  -- ^ AES cipher and tweak context-           -> IV AES     -- ^ a 128 bits IV, typically a sector or a block offset in XTS-           -> Word32     -- ^ number of rounds to skip, also seen a 16 byte offset in the sector or block.-           -> ba         -- ^ input to decrypt-           -> ba         -- ^ output decrypted-decryptXTS = doXTS c_aes_decrypt_xts---- | encrypt/decrypt using Counter mode (32-bit wrapping used in AES-GCM-SIV)-{-# NOINLINE combineC32 #-}-combineC32 :: ByteArray ba-           => AES        -- ^ AES Context-           -> IV AES     -- ^ initial vector of AES block size (usually representing a 128 bit integer)-           -> ba         -- ^ plaintext input-           -> ba         -- ^ ciphertext output-combineC32 ctx iv input-    | len <= 0          = B.empty-    | B.length iv /= 16 = error $ "AES error: IV length must be block size (16). Its length is: " ++ show (B.length iv)-    | otherwise = B.allocAndFreeze len doEncrypt-  where doEncrypt o = withKeyAndIV ctx iv $ \k v -> withByteArray input $ \i ->-                      c_aes_encrypt_c32 (castPtr o) k v i (fromIntegral len)-        len = B.length input--{-# INLINE doECB #-}-doECB :: ByteArray ba-      => (Ptr b -> Ptr AES -> CString -> CUInt -> IO ())-      -> AES -> ba -> ba-doECB f ctx input-    | len == 0     = B.empty-    | r /= 0       = error $ "Encryption error: input length must be a multiple of block size (16). Its length is: " ++ (show len)-    | otherwise    =-        B.allocAndFreeze len $ \o ->-        keyToPtr ctx         $ \k ->-        withByteArray input  $ \i ->-            f (castPtr o) k i (fromIntegral nbBlocks)-  where (nbBlocks, r) = len `quotRem` 16-        len           = B.length input--{-# INLINE doCBC #-}-doCBC :: ByteArray ba-      => (Ptr b -> Ptr AES -> Ptr Word8 -> CString -> CUInt -> IO ())-      -> AES -> IV AES -> ba -> ba-doCBC f ctx (IV iv) input-    | len == 0  = B.empty-    | r /= 0    = error $ "Encryption error: input length must be a multiple of block size (16). Its length is: " ++ (show len)-    | otherwise = B.allocAndFreeze len $ \o ->-                  withKeyAndIV ctx iv $ \k v ->-                  withByteArray input $ \i ->-                  f (castPtr o) k v i (fromIntegral nbBlocks)-  where (nbBlocks, r) = len `quotRem` 16-        len           = B.length input--{-# INLINE doXTS #-}-doXTS :: ByteArray ba-      => (Ptr b -> Ptr AES -> Ptr AES -> Ptr Word8 -> CUInt -> CString -> CUInt -> IO ())-      -> (AES, AES)-      -> IV AES-      -> Word32-      -> ba-      -> ba-doXTS f (key1,key2) iv spoint input-    | len == 0  = B.empty-    | r /= 0    = error $ "Encryption error: input length must be a multiple of block size (16) for now. Its length is: " ++ (show len)-    | otherwise = B.allocAndFreeze len $ \o -> withKey2AndIV key1 key2 iv $ \k1 k2 v -> withByteArray input $ \i ->-            f (castPtr o) k1 k2 v (fromIntegral spoint) i (fromIntegral nbBlocks)-  where (nbBlocks, r) = len `quotRem` 16-        len           = B.length input----------------------------------------------------------------------------- GCM----------------------------------------------------------------------------- | initialize a gcm context-{-# NOINLINE gcmInit #-}-gcmInit :: ByteArrayAccess iv => AES -> iv -> AESGCM-gcmInit ctx iv = unsafeDoIO $ do-    sm <- B.alloc sizeGCM $ \gcmStPtr ->-            withKeyAndIV ctx iv $ \k v ->-            c_aes_gcm_init (castPtr gcmStPtr) k v (fromIntegral $ B.length iv)-    return $ AESGCM sm---- | append data which is only going to be authenticated to the GCM context.------ needs to happen after initialization and before appending encryption/decryption data.-{-# NOINLINE gcmAppendAAD #-}-gcmAppendAAD :: ByteArrayAccess aad => AESGCM -> aad -> AESGCM-gcmAppendAAD gcmSt input = unsafeDoIO doAppend-  where doAppend =-            withNewGCMSt gcmSt $ \gcmStPtr ->-            withByteArray input $ \i ->-            c_aes_gcm_aad gcmStPtr i (fromIntegral $ B.length input)---- | append data to encrypt and append to the GCM context------ the bytearray needs to be a multiple of AES block size, unless it's the last call to this function.--- needs to happen after AAD appending, or after initialization if no AAD data.-{-# NOINLINE gcmAppendEncrypt #-}-gcmAppendEncrypt :: ByteArray ba => AES -> AESGCM -> ba -> (ba, AESGCM)-gcmAppendEncrypt ctx gcm input = unsafeDoIO $ withGCMKeyAndCopySt ctx gcm doEnc-  where len = B.length input-        doEnc gcmStPtr aesPtr =-            B.alloc len $ \o ->-            withByteArray input $ \i ->-            c_aes_gcm_encrypt (castPtr o) gcmStPtr aesPtr i (fromIntegral len)---- | append data to decrypt and append to the GCM context------ the bytearray needs to be a multiple of AES block size, unless it's the last call to this function.--- needs to happen after AAD appending, or after initialization if no AAD data.-{-# NOINLINE gcmAppendDecrypt #-}-gcmAppendDecrypt :: ByteArray ba => AES -> AESGCM -> ba -> (ba, AESGCM)-gcmAppendDecrypt ctx gcm input = unsafeDoIO $ withGCMKeyAndCopySt ctx gcm doDec-  where len = B.length input-        doDec gcmStPtr aesPtr =-            B.alloc len $ \o ->-            withByteArray input $ \i ->-            c_aes_gcm_decrypt (castPtr o) gcmStPtr aesPtr i (fromIntegral len)---- | Generate the Tag from GCM context-{-# NOINLINE gcmFinish #-}-gcmFinish :: AES -> AESGCM -> Int -> AuthTag-gcmFinish ctx gcm taglen = AuthTag $ B.take taglen computeTag-  where computeTag = B.allocAndFreeze 16 $ \t ->-                        withGCMKeyAndCopySt ctx gcm (c_aes_gcm_finish (castPtr t)) >> return ()----------------------------------------------------------------------------- OCB v3----------------------------------------------------------------------------- | initialize an ocb context-{-# NOINLINE ocbInit #-}-ocbInit :: ByteArrayAccess iv => AES -> iv -> AESOCB-ocbInit ctx iv = unsafeDoIO $ do-    sm <- B.alloc sizeOCB $ \ocbStPtr ->-            withKeyAndIV ctx iv $ \k v ->-            c_aes_ocb_init (castPtr ocbStPtr) k v (fromIntegral $ B.length iv)-    return $ AESOCB sm---- | append data which is going to just be authenticated to the OCB context.------ need to happen after initialization and before appending encryption/decryption data.-{-# NOINLINE ocbAppendAAD #-}-ocbAppendAAD :: ByteArrayAccess aad => AES -> AESOCB -> aad -> AESOCB-ocbAppendAAD ctx ocb input = unsafeDoIO (snd `fmap` withOCBKeyAndCopySt ctx ocb doAppend)-  where doAppend ocbStPtr aesPtr =-            withByteArray input $ \i ->-            c_aes_ocb_aad ocbStPtr aesPtr i (fromIntegral $ B.length input)---- | append data to encrypt and append to the OCB context------ the bytearray needs to be a multiple of the AES block size, unless it's the last call to this function.--- need to happen after AAD appending, or after initialization if no AAD data.-{-# NOINLINE ocbAppendEncrypt #-}-ocbAppendEncrypt :: ByteArray ba => AES -> AESOCB -> ba -> (ba, AESOCB)-ocbAppendEncrypt ctx ocb input = unsafeDoIO $ withOCBKeyAndCopySt ctx ocb doEnc-  where len = B.length input-        doEnc ocbStPtr aesPtr =-            B.alloc len $ \o ->-            withByteArray input $ \i ->-            c_aes_ocb_encrypt (castPtr o) ocbStPtr aesPtr i (fromIntegral len)---- | append data to decrypt and append to the OCB context------ the bytearray needs to be a multiple of the AES block size, unless it's the last call to this function.--- need to happen after AAD appending, or after initialization if no AAD data.-{-# NOINLINE ocbAppendDecrypt #-}-ocbAppendDecrypt :: ByteArray ba => AES -> AESOCB -> ba -> (ba, AESOCB)-ocbAppendDecrypt ctx ocb input = unsafeDoIO $ withOCBKeyAndCopySt ctx ocb doDec-  where len = B.length input-        doDec ocbStPtr aesPtr =-            B.alloc len $ \o ->-            withByteArray input $ \i ->-            c_aes_ocb_decrypt (castPtr o) ocbStPtr aesPtr i (fromIntegral len)---- | Generate the Tag from OCB context-{-# NOINLINE ocbFinish #-}-ocbFinish :: AES -> AESOCB -> Int -> AuthTag-ocbFinish ctx ocb taglen = AuthTag $ B.take taglen computeTag-  where computeTag = B.allocAndFreeze 16 $ \t ->-                        withOCBKeyAndCopySt ctx ocb (c_aes_ocb_finish (castPtr t)) >> return ()--ccmGetM :: CCM_M -> Int-ccmGetL :: CCM_L -> Int-ccmGetM m = case m of-  CCM_M4 -> 4-  CCM_M6 -> 6-  CCM_M8 -> 8-  CCM_M10 -> 10-  CCM_M12 -> 12-  CCM_M14 -> 14-  CCM_M16 -> 16--ccmGetL l = case l of-  CCM_L2 -> 2-  CCM_L3 -> 3-  CCM_L4 -> 4---- | initialize a ccm context-{-# NOINLINE ccmInit #-}-ccmInit :: ByteArrayAccess iv => AES -> iv -> Int -> CCM_M -> CCM_L -> CryptoFailable AESCCM-ccmInit ctx iv n m l-    | 15 - li /= B.length iv = CryptoFailed CryptoError_IvSizeInvalid-    | otherwise = unsafeDoIO $ do-          sm <- B.alloc sizeCCM $ \ccmStPtr ->-            withKeyAndIV ctx iv $ \k v ->-            c_aes_ccm_init (castPtr ccmStPtr) k v (fromIntegral $ B.length iv) (fromIntegral n) (fromIntegral mi) (fromIntegral li)-          return $ CryptoPassed (AESCCM sm)-  where-    mi = ccmGetM m-    li = ccmGetL l---- | append data which is only going to be authenticated to the CCM context.------ needs to happen after initialization and before appending encryption/decryption data.-{-# NOINLINE ccmAppendAAD #-}-ccmAppendAAD :: ByteArrayAccess aad => AES -> AESCCM -> aad -> AESCCM-ccmAppendAAD ctx ccm input = unsafeDoIO $ snd <$> withCCMKeyAndCopySt ctx ccm doAppend-  where doAppend ccmStPtr aesPtr =-            withByteArray input $ \i -> c_aes_ccm_aad ccmStPtr aesPtr i (fromIntegral $ B.length input)---- | append data to encrypt and append to the CCM context------ the bytearray needs to be a multiple of AES block size, unless it's the last call to this function.--- needs to happen after AAD appending, or after initialization if no AAD data.-{-# NOINLINE ccmEncrypt #-}-ccmEncrypt :: ByteArray ba => AES -> AESCCM -> ba -> (ba, AESCCM)-ccmEncrypt ctx ccm input = unsafeDoIO $ withCCMKeyAndCopySt ctx ccm cbcmacAndIv-  where len = B.length input-        cbcmacAndIv ccmStPtr aesPtr =-            B.alloc len $ \o ->-            withByteArray input $ \i ->-            c_aes_ccm_encrypt (castPtr o) ccmStPtr aesPtr i (fromIntegral len)---- | append data to decrypt and append to the CCM context------ the bytearray needs to be a multiple of AES block size, unless it's the last call to this function.--- needs to happen after AAD appending, or after initialization if no AAD data.-{-# NOINLINE ccmDecrypt #-}-ccmDecrypt :: ByteArray ba => AES -> AESCCM -> ba -> (ba, AESCCM)-ccmDecrypt ctx ccm input = unsafeDoIO $ withCCMKeyAndCopySt ctx ccm cbcmacAndIv-  where len = B.length input-        cbcmacAndIv ccmStPtr aesPtr =-            B.alloc len $ \o ->-            withByteArray input $ \i ->-            c_aes_ccm_decrypt (castPtr o) ccmStPtr aesPtr i (fromIntegral len)---- | Generate the Tag from CCM context-{-# NOINLINE ccmFinish #-}-ccmFinish :: AES -> AESCCM -> Int -> AuthTag-ccmFinish ctx ccm taglen = AuthTag $ B.take taglen computeTag-  where computeTag = B.allocAndFreeze 16 $ \t ->-                        withCCMKeyAndCopySt ctx ccm (c_aes_ccm_finish (castPtr t)) >> return ()---------------------------------------------------------------------------foreign import ccall "crypton_aes.h crypton_aes_initkey"-    c_aes_init :: Ptr AES -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_encrypt_ecb"-    c_aes_encrypt_ecb :: CString -> Ptr AES -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_decrypt_ecb"-    c_aes_decrypt_ecb :: CString -> Ptr AES -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_encrypt_cbc"-    c_aes_encrypt_cbc :: CString -> Ptr AES -> Ptr Word8 -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_decrypt_cbc"-    c_aes_decrypt_cbc :: CString -> Ptr AES -> Ptr Word8 -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_encrypt_xts"-    c_aes_encrypt_xts :: CString -> Ptr AES -> Ptr AES -> Ptr Word8 -> CUInt -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_decrypt_xts"-    c_aes_decrypt_xts :: CString -> Ptr AES -> Ptr AES -> Ptr Word8 -> CUInt -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_gen_ctr"-    c_aes_gen_ctr :: CString -> Ptr AES -> Ptr Word8 -> CUInt -> IO ()--foreign import ccall unsafe "crypton_aes.h crypton_aes_gen_ctr_cont"-    c_aes_gen_ctr_cont :: CString -> Ptr AES -> Ptr Word8 -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_encrypt_ctr"-    c_aes_encrypt_ctr :: CString -> Ptr AES -> Ptr Word8 -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_encrypt_c32"-    c_aes_encrypt_c32 :: CString -> Ptr AES -> Ptr Word8 -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_gcm_init"-    c_aes_gcm_init :: Ptr AESGCM -> Ptr AES -> Ptr Word8 -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_gcm_aad"-    c_aes_gcm_aad :: Ptr AESGCM -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_gcm_encrypt"-    c_aes_gcm_encrypt :: CString -> Ptr AESGCM -> Ptr AES -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_gcm_decrypt"-    c_aes_gcm_decrypt :: CString -> Ptr AESGCM -> Ptr AES -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_gcm_finish"-    c_aes_gcm_finish :: CString -> Ptr AESGCM -> Ptr AES -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_ocb_init"-    c_aes_ocb_init :: Ptr AESOCB -> Ptr AES -> Ptr Word8 -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_ocb_aad"-    c_aes_ocb_aad :: Ptr AESOCB -> Ptr AES -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_ocb_encrypt"-    c_aes_ocb_encrypt :: CString -> Ptr AESOCB -> Ptr AES -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_ocb_decrypt"-    c_aes_ocb_decrypt :: CString -> Ptr AESOCB -> Ptr AES -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_ocb_finish"-    c_aes_ocb_finish :: CString -> Ptr AESOCB -> Ptr AES -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_ccm_init"-    c_aes_ccm_init :: Ptr AESCCM -> Ptr AES -> Ptr Word8 -> CUInt -> CUInt -> CInt -> CInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_ccm_aad"-    c_aes_ccm_aad :: Ptr AESCCM -> Ptr AES -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_ccm_encrypt"-    c_aes_ccm_encrypt :: CString -> Ptr AESCCM -> Ptr AES -> CString -> CUInt -> IO ()--foreign import ccall "crypton_aes.h crypton_aes_ccm_decrypt"-    c_aes_ccm_decrypt :: CString -> Ptr AESCCM -> Ptr AES -> CString -> CUInt -> IO ()+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# LANGUAGE ViewPatterns #-}++-- |+-- Module      : Crypto.Cipher.AES.Primitive+-- License     : BSD-style+-- Maintainer  : Vincent Hanquez <vincent@snarc.org>+-- Stability   : stable+-- Portability : good+module Crypto.Cipher.AES.Primitive (+    -- * Block cipher data types+    AES,++    -- * Authenticated encryption block cipher types+    AESGCM,+    AESOCB,++    -- * Creation+    initAES,++    -- * Miscellanea++    -- * Encryption+    encryptECB,+    encryptCBC,+    encryptCTR,+    encryptXTS,++    -- * Decryption+    decryptECB,+    decryptCBC,+    decryptCTR,+    decryptXTS,++    -- * CTR with 32-bit wrapping+    combineC32,++    -- * Incremental GCM+    gcmMode,+    gcmInit,+    AESGCMKey,+    gcmKeyInit,+    gcmFullEncrypt,+    gcmFullEncryptMask,+    gcmFullDecrypt,+    gcmFullDecryptTag,+    gcmAeadInit,++    -- * Incremental OCB+    ocbMode,+    ocbModeWithTagLength,+    ocbInit,+    ocbInitWithTagLength,++    -- * CCM+    ccmMode,+    ccmInit,+) where++import Data.Word+import Foreign.C.String+import Foreign.C.Types+import Foreign.Ptr++import Crypto.Cipher.Types+import Crypto.Cipher.Types.Block (IV (..))+import Crypto.Error+import Crypto.Internal.ByteArray (+    ByteArray,+    ByteArrayAccess,+    ScrubbedBytes,+    withByteArray,+ )+import qualified Crypto.Internal.ByteArray as B+import Crypto.Internal.Compat+import Crypto.Internal.Imports++instance Cipher AES where+    cipherName _ = "AES"+    cipherKeySize _ = KeySizeEnum [16, 24, 32]+    cipherInit k = initAES k++instance BlockCipher AES where+    blockSize _ = 16+    ecbEncrypt = encryptECB+    ecbDecrypt = decryptECB+    cbcEncrypt = encryptCBC+    cbcDecrypt = decryptCBC+    ctrCombine = encryptCTR+    aeadInit AEAD_GCM aes iv = gcmAeadInit aes iv+    aeadInit AEAD_OCB aes iv = CryptoPassed $ AEAD (ocbMode aes) (ocbInit aes iv)+    aeadInit (AEAD_CCM n m l) aes iv = AEAD (ccmMode aes) <$> ccmInit aes iv n m l+    aeadInit _ _ _ = CryptoFailed CryptoError_AEADModeNotSupported+instance BlockCipher128 AES where+    xtsEncrypt = encryptXTS+    xtsDecrypt = decryptXTS++-- | Create an AES AEAD context for GCM, refusing the zero-length IV that+-- SP 800-38D 5.2.1.1 forbids: any length other than 96 bits is fed to+-- GHASH, and for the empty IV that makes J0 the GHASH of the empty+-- string, which leaks the authentication key.+gcmAeadInit :: ByteArrayAccess iv => AES -> iv -> CryptoFailable (AEAD c)+gcmAeadInit aes iv+    | B.length iv == 0 = CryptoFailed CryptoError_IvSizeInvalid+    | otherwise = CryptoPassed $ AEAD (gcmMode aes) (gcmInit aes iv)++-- | Create an AES AEAD implementation for GCM+gcmMode :: AES -> AEADModeImpl AESGCM+gcmMode aes =+    AEADModeImpl+        { aeadImplAppendHeader = gcmAppendAAD+        , aeadImplEncrypt = gcmAppendEncrypt aes+        , aeadImplDecrypt = gcmAppendDecrypt aes+        , aeadImplFinalize = gcmFinish aes+        }++-- | Create an AES AEAD implementation for OCB+ocbMode :: AES -> AEADModeImpl AESOCB+ocbMode aes =+    AEADModeImpl+        { aeadImplAppendHeader = ocbAppendAAD aes+        , aeadImplEncrypt = ocbAppendEncrypt aes+        , aeadImplDecrypt = ocbAppendDecrypt aes+        , aeadImplFinalize = ocbFinish aes+        }++ocbModeWithTagLength :: AES -> Int -> AEADModeImpl AESOCB+ocbModeWithTagLength aes taglen =+    AEADModeImpl+        { aeadImplAppendHeader = ocbAppendAAD aes+        , aeadImplEncrypt = ocbAppendEncrypt aes+        , aeadImplDecrypt = ocbAppendDecrypt aes+        , aeadImplFinalize = \ocb _ -> ocbFinish aes ocb taglen+        }++-- | Create an AES AEAD implementation for CCM+ccmMode :: AES -> AEADModeImpl AESCCM+ccmMode aes =+    AEADModeImpl+        { aeadImplAppendHeader = ccmAppendAAD aes+        , aeadImplEncrypt = ccmEncrypt aes+        , aeadImplDecrypt = ccmDecrypt aes+        , aeadImplFinalize = ccmFinish aes+        }++-- | AES Context (pre-processed key)+newtype AES = AES ScrubbedBytes+    deriving (NFData)++-- | AESGCM State+newtype AESGCM = AESGCM ScrubbedBytes+    deriving (NFData)++-- | AESOCB State+newtype AESOCB = AESOCB ScrubbedBytes+    deriving (NFData)++-- | AESCCM State+newtype AESCCM = AESCCM ScrubbedBytes+    deriving (NFData)++sizeGCM :: Int+sizeGCM = 320++-- | The size of what a key determines, which is the 320 bytes above and the+-- powers of H the fused path reads: sixteen of them, and sixteen more for+-- the term the Karatsuba multiplication would otherwise work out every time.+-- The same on every platform, so that this is one number rather than one per+-- architecture; the powers are filled only where that path is compiled in.+sizeGCMKey :: Int+sizeGCMKey = 832++sizeOCB :: Int+sizeOCB = 160++sizeCCM :: Int+sizeCCM = 80++keyToPtr :: AES -> (Ptr AES -> IO a) -> IO a+keyToPtr (AES b) f = withByteArray b (f . castPtr)++ivToPtr :: ByteArrayAccess iv => iv -> (Ptr Word8 -> IO a) -> IO a+ivToPtr iv f = withByteArray iv (f . castPtr)++withKeyAndIV+    :: ByteArrayAccess iv => AES -> iv -> (Ptr AES -> Ptr Word8 -> IO a) -> IO a+withKeyAndIV ctx iv f = keyToPtr ctx $ \kptr -> ivToPtr iv $ \ivp -> f kptr ivp++withKey2AndIV+    :: ByteArrayAccess iv+    => AES -> AES -> iv -> (Ptr AES -> Ptr AES -> Ptr Word8 -> IO a) -> IO a+withKey2AndIV key1 key2 iv f =+    keyToPtr key1 $ \kptr1 -> keyToPtr key2 $ \kptr2 -> ivToPtr iv $ \ivp -> f kptr1 kptr2 ivp++withGCMKeyAndCopySt+    :: AES -> AESGCM -> (Ptr AESGCM -> Ptr AES -> IO a) -> IO (a, AESGCM)+withGCMKeyAndCopySt aes (AESGCM gcmSt) f =+    keyToPtr aes $ \aesPtr -> do+        newSt <- B.copy gcmSt (\_ -> return ())+        a <- withByteArray newSt $ \gcmStPtr -> f (castPtr gcmStPtr) aesPtr+        return (a, AESGCM newSt)++withNewGCMSt :: AESGCM -> (Ptr AESGCM -> IO ()) -> IO AESGCM+withNewGCMSt (AESGCM gcmSt) f = B.copy gcmSt (f . castPtr) >>= \sm2 -> return (AESGCM sm2)++withOCBKeyAndCopySt+    :: AES -> AESOCB -> (Ptr AESOCB -> Ptr AES -> IO a) -> IO (a, AESOCB)+withOCBKeyAndCopySt aes (AESOCB gcmSt) f =+    keyToPtr aes $ \aesPtr -> do+        newSt <- B.copy gcmSt (\_ -> return ())+        a <- withByteArray newSt $ \gcmStPtr -> f (castPtr gcmStPtr) aesPtr+        return (a, AESOCB newSt)++withCCMKeyAndCopySt+    :: AES -> AESCCM -> (Ptr AESCCM -> Ptr AES -> IO a) -> IO (a, AESCCM)+withCCMKeyAndCopySt aes (AESCCM ccmSt) f =+    keyToPtr aes $ \aesPtr -> do+        newSt <- B.copy ccmSt (\_ -> return ())+        a <- withByteArray newSt $ \ccmStPtr -> f (castPtr ccmStPtr) aesPtr+        return (a, AESCCM newSt)++-- | Initialize a new context with a key+--+-- Key needs to be of length 16, 24 or 32 bytes. Any other values will return failure+initAES :: ByteArrayAccess key => key -> CryptoFailable AES+initAES k+    | len == 16 = CryptoPassed $ initWithRounds 10+    | len == 24 = CryptoPassed $ initWithRounds 12+    | len == 32 = CryptoPassed $ initWithRounds 14+    | otherwise = CryptoFailed CryptoError_KeySizeInvalid+  where+    len = B.length k+    initWithRounds nbR = AES $ B.allocAndFreeze (16 + 2 * 2 * 16 * nbR) aesInit+    aesInit ptr = withByteArray k $ \ikey ->+        c_aes_init (castPtr ptr) (castPtr ikey) (fromIntegral len)++-- | encrypt using Electronic Code Book (ECB)+{-# NOINLINE encryptECB #-}+encryptECB :: ByteArray ba => AES -> ba -> ba+encryptECB = doECB c_aes_encrypt_ecb++-- | encrypt using Cipher Block Chaining (CBC)+{-# NOINLINE encryptCBC #-}+encryptCBC+    :: ByteArray ba+    => AES+    -- ^ AES Context+    -> IV AES+    -- ^ Initial vector of AES block size+    -> ba+    -- ^ plaintext+    -> ba+    -- ^ ciphertext+encryptCBC = doCBC c_aes_encrypt_cbc++-- | encrypt using Counter mode (CTR)+--+-- in CTR mode encryption and decryption is the same operation.+{-# NOINLINE encryptCTR #-}+encryptCTR+    :: ByteArray ba+    => AES+    -- ^ AES Context+    -> IV AES+    -- ^ initial vector of AES block size (usually representing a 128 bit integer)+    -> ba+    -- ^ plaintext input+    -> ba+    -- ^ ciphertext output+encryptCTR ctx iv input+    | len <= 0 = B.empty+    | B.overCLength len = error tooLongMessage+    | B.length iv /= 16 =+        error $+            "AES error: IV length must be block size (16). Its length is: "+                ++ (show $ B.length iv)+    | otherwise = B.allocAndFreeze len doEncrypt+  where+    doEncrypt o = withKeyAndIV ctx iv $ \k v -> withByteArray input $ \i ->+        c_aes_encrypt_ctr (castPtr o) k v i (fromIntegral len)+    len = B.length input++-- | encrypt using XTS+--+-- the first key is the normal block encryption key+-- the second key is used for the initial block tweak+{-# NOINLINE encryptXTS #-}+encryptXTS+    :: ByteArray ba+    => (AES, AES)+    -- ^ AES cipher and tweak context+    -> IV AES+    -- ^ a 128 bits IV, typically a sector or a block offset in XTS+    -> Word32+    -- ^ number of rounds to skip, also seen a 16 byte offset in the sector or block.+    -> ba+    -- ^ input to encrypt+    -> ba+    -- ^ output encrypted+encryptXTS = doXTS c_aes_encrypt_xts++-- | decrypt using Electronic Code Book (ECB)+{-# NOINLINE decryptECB #-}+decryptECB :: ByteArray ba => AES -> ba -> ba+decryptECB = doECB c_aes_decrypt_ecb++-- | decrypt using Cipher block chaining (CBC)+{-# NOINLINE decryptCBC #-}+decryptCBC :: ByteArray ba => AES -> IV AES -> ba -> ba+decryptCBC = doCBC c_aes_decrypt_cbc++-- | decrypt using Counter mode (CTR).+--+-- in CTR mode encryption and decryption is the same operation.+decryptCTR+    :: ByteArray ba+    => AES+    -- ^ AES Context+    -> IV AES+    -- ^ initial vector, usually representing a 128 bit integer+    -> ba+    -- ^ ciphertext input+    -> ba+    -- ^ plaintext output+decryptCTR = encryptCTR++-- | decrypt using XTS+{-# NOINLINE decryptXTS #-}+decryptXTS+    :: ByteArray ba+    => (AES, AES)+    -- ^ AES cipher and tweak context+    -> IV AES+    -- ^ a 128 bits IV, typically a sector or a block offset in XTS+    -> Word32+    -- ^ number of rounds to skip, also seen a 16 byte offset in the sector or block.+    -> ba+    -- ^ input to decrypt+    -> ba+    -- ^ output decrypted+decryptXTS = doXTS c_aes_decrypt_xts++-- | encrypt/decrypt using Counter mode (32-bit wrapping used in AES-GCM-SIV)+{-# NOINLINE combineC32 #-}+combineC32+    :: ByteArray ba+    => AES+    -- ^ AES Context+    -> IV AES+    -- ^ initial vector of AES block size (usually representing a 128 bit integer)+    -> ba+    -- ^ plaintext input+    -> ba+    -- ^ ciphertext output+combineC32 ctx iv input+    | len <= 0 = B.empty+    | B.length iv /= 16 =+        error $+            "AES error: IV length must be block size (16). Its length is: "+                ++ show (B.length iv)+    | otherwise = B.allocAndFreeze len doEncrypt+  where+    doEncrypt o = withKeyAndIV ctx iv $ \k v -> withByteArray input $ \i ->+        c_aes_encrypt_c32 (castPtr o) k v i (fromIntegral len)+    len = B.length input++-- | What the AES modes say when a message cannot be given to the C, whose+-- lengths are @uint32_t@.  Above that the length is truncated on the way+-- down and most of the buffer is left as it was found, with nothing to say+-- so, which is worse than refusing.+--+-- Unlike the stream ciphers, these cannot be done in pieces: the C is handed+-- the IV and does not hand it back, so a second call would start from the+-- wrong place.  ECB, CBC and XTS count blocks rather than bytes, so their+-- limit is sixteen times further out than CTR's.+tooLongMessage :: String+tooLongMessage =+    "AES error: message too long for this implementation, whose C takes its "+        ++ "lengths as uint32_t"++{-# INLINE doECB #-}+doECB+    :: ByteArray ba+    => (Ptr b -> Ptr AES -> CString -> CUInt -> IO ())+    -> AES+    -> ba+    -> ba+doECB f ctx input+    | B.overCLength nbBlocks = error tooLongMessage+    | len == 0 = B.empty+    | r /= 0 =+        error $+            "Encryption error: input length must be a multiple of block size (16). Its length is: "+                ++ (show len)+    | otherwise =+        B.allocAndFreeze len $ \o ->+            keyToPtr ctx $ \k ->+                withByteArray input $ \i ->+                    f (castPtr o) k i (fromIntegral nbBlocks)+  where+    (nbBlocks, r) = len `quotRem` 16+    len = B.length input++{-# INLINE doCBC #-}+doCBC+    :: ByteArray ba+    => (Ptr b -> Ptr AES -> Ptr Word8 -> CString -> CUInt -> IO ())+    -> AES+    -> IV AES+    -> ba+    -> ba+doCBC f ctx (IV iv) input+    | B.overCLength nbBlocks = error tooLongMessage+    | len == 0 = B.empty+    | r /= 0 =+        error $+            "Encryption error: input length must be a multiple of block size (16). Its length is: "+                ++ (show len)+    | otherwise = B.allocAndFreeze len $ \o ->+        withKeyAndIV ctx iv $ \k v ->+            withByteArray input $ \i ->+                f (castPtr o) k v i (fromIntegral nbBlocks)+  where+    (nbBlocks, r) = len `quotRem` 16+    len = B.length input++{-# INLINE doXTS #-}+doXTS+    :: ByteArray ba+    => (Ptr b -> Ptr AES -> Ptr AES -> Ptr Word8 -> CUInt -> CString -> CUInt -> IO ())+    -> (AES, AES)+    -> IV AES+    -> Word32+    -> ba+    -> ba+doXTS f (key1, key2) iv spoint input+    | B.overCLength nbBlocks = error tooLongMessage+    | len == 0 = B.empty+    | r /= 0 =+        error $+            "Encryption error: input length must be a multiple of block size (16) for now. Its length is: "+                ++ (show len)+    | otherwise = B.allocAndFreeze len $ \o -> withKey2AndIV key1 key2 iv $ \k1 k2 v -> withByteArray input $ \i ->+        f (castPtr o) k1 k2 v (fromIntegral spoint) i (fromIntegral nbBlocks)+  where+    (nbBlocks, r) = len `quotRem` 16+    len = B.length input++------------------------------------------------------------------------+-- GCM+------------------------------------------------------------------------++-- | initialize a gcm context+{-# NOINLINE gcmInit #-}+gcmInit :: ByteArrayAccess iv => AES -> iv -> AESGCM+gcmInit ctx iv = unsafeDoIO $ do+    sm <- B.alloc sizeGCM $ \gcmStPtr ->+        withKeyAndIV ctx iv $ \k v ->+            c_aes_gcm_init (castPtr gcmStPtr) k v (fromIntegral $ B.length iv)+    return $ AESGCM sm++-- | How long a message may be and still be handed to an unsafe foreign call.+-- Four kibibytes is about half a microsecond of work, and it takes in a+-- datagram of any size a network will carry.+shortMessage :: Int+shortMessage = 4096++-- | The part of a GCM state the key alone determines: H, which is the key+-- applied to a block of zeroes, and the table of its multiples.  That is 256+-- of the 320 bytes of a GCM state, and it is the same for every message sent+-- under one key, so a caller that keeps a key can build this once rather than+-- once for every message.+newtype AESGCMKey = AESGCMKey ScrubbedBytes++-- | Build the key part of a GCM state.+{-# NOINLINE gcmKeyInit #-}+gcmKeyInit :: AES -> AESGCMKey+gcmKeyInit ctx = AESGCMKey $ B.allocAndFreeze sizeGCMKey $ \p ->+    keyToPtr ctx $ \k -> c_aes_gcm_key_init (castPtr p) k++-- | Authenticate and encrypt one message in a single call: the nonce, the+-- additional data, the plaintext and the tag, with no state crossing back+-- into Haskell in between.  The result is the ciphertext followed by the tag.+{-# INLINABLE gcmFullEncrypt #-}+gcmFullEncrypt+    :: (ByteArrayAccess iv, ByteArrayAccess aad, ByteArrayAccess ba, ByteArray output)+    => AES -> AESGCMKey -> iv -> aad -> ba -> Int -> output+gcmFullEncrypt ctx (AESGCMKey gk) iv aad input taglen =+    B.allocAndFreeze (B.length input + taglen) $ \out ->+        B.withByteArray gk $ \gkp ->+            keyToPtr ctx $ \k ->+                B.withByteArray iv $ \ivp ->+                    B.withByteArray aad $ \aadp ->+                        B.withByteArray input $ \inp ->+                            call+                                out+                                (castPtr gkp)+                                k+                                ivp+                                (fromIntegral $ B.length iv)+                                aadp+                                (fromIntegral $ B.length aad)+                                inp+                                (fromIntegral $ B.length input)+                                (fromIntegral taglen)+  where+    -- An unsafe call keeps a capability for as long as it runs, so it is only+    -- right for work that is over quickly.  A message this side of+    -- 'shortMessage' is, and it is the short ones the saving matters for: a+    -- safe call costs about 0.075 us whatever the length, which is a fifth of+    -- a 1440-byte packet and a percent of a 16 KiB record.+    call+        | B.length input <= shortMessage = c_aes_gcm_full_encrypt_unsafe+        | otherwise = c_aes_gcm_full_encrypt++-- | Encrypt, and from a sample of the ciphertext just produced make the+-- header protection mask, into buffers the caller owns.  QUIC takes its+-- sample from the ciphertext, so the mask cannot be had before the+-- encryption; it can be had before coming back, and with the buffers already+-- there nothing is allocated for either.+--+-- @sampleoff@ is where the sixteen bytes of sample begin in the output.+{-# INLINABLE gcmFullEncryptMask #-}+gcmFullEncryptMask+    :: (ByteArrayAccess iv, ByteArrayAccess aad, ByteArrayAccess ba)+    => AES+    -> AESGCMKey+    -> AES+    -> iv+    -> aad+    -> ba+    -> Int+    -> Int+    -> Ptr Word8+    -> Ptr Word8+    -> IO ()+gcmFullEncryptMask ctx (AESGCMKey gk) hpctx iv aad input taglen sampleoff outp maskp =+    B.withByteArray gk $ \gkp ->+        keyToPtr ctx $ \k ->+            keyToPtr hpctx $ \hk ->+                B.withByteArray iv $ \ivp ->+                    B.withByteArray aad $ \aadp ->+                        B.withByteArray input $ \inp ->+                            call+                                outp+                                (castPtr gkp)+                                k+                                ivp+                                (fromIntegral $ B.length iv)+                                aadp+                                (fromIntegral $ B.length aad)+                                inp+                                (fromIntegral $ B.length input)+                                (fromIntegral taglen)+                                hk+                                (fromIntegral sampleoff)+                                maskp+  where+    call+        | B.length input <= shortMessage = c_aes_gcm_full_encrypt_mask_unsafe+        | otherwise = c_aes_gcm_full_encrypt_mask++-- | The same the other way, with the tag compared here rather than by the+-- caller: 'Nothing' when it does not match, and every byte of it is looked at+-- either way.  The ciphertext comes in without its tag, which is given+-- separately.+{-# INLINABLE gcmFullDecrypt #-}+gcmFullDecrypt+    :: ( ByteArrayAccess iv+       , ByteArrayAccess aad+       , ByteArrayAccess ba+       , ByteArrayAccess tag+       , ByteArray output+       )+    => AES -> AESGCMKey -> iv -> aad -> ba -> tag -> Maybe output+gcmFullDecrypt ctx (AESGCMKey gk) iv aad input tag = unsafeDoIO $ do+    (r, out) <- B.allocRet (B.length input) $ \outp ->+        B.withByteArray gk $ \gkp ->+            keyToPtr ctx $ \k ->+                B.withByteArray iv $ \ivp ->+                    B.withByteArray aad $ \aadp ->+                        B.withByteArray input $ \inp ->+                            B.withByteArray tag $ \tagp ->+                                call+                                    outp+                                    (castPtr gkp)+                                    k+                                    ivp+                                    (fromIntegral $ B.length iv)+                                    aadp+                                    (fromIntegral $ B.length aad)+                                    inp+                                    (fromIntegral $ B.length input)+                                    tagp+                                    (fromIntegral $ B.length tag)+    return $ if r /= 0 then Just out else Nothing+  where+    call+        | B.length input <= shortMessage = c_aes_gcm_full_decrypt_unsafe+        | otherwise = c_aes_gcm_full_decrypt++-- | Decrypt one message and hand back the tag that was computed over it,+-- rather than comparing it here.+--+-- For a caller that holds the expected tag in a form of its own and will+-- compare it itself.  Compare the two t'AuthTag's with '==', whose instance+-- for that type is a constant-time comparison; taking them apart and+-- comparing the bytes is how this goes wrong.+--+-- Where the tag simply arrives after the ciphertext, 'gcmFullDecrypt' is the+-- one to use: it compares in C and never puts a tag in the caller's hands.+{-# INLINABLE gcmFullDecryptTag #-}+gcmFullDecryptTag+    :: ( ByteArrayAccess iv+       , ByteArrayAccess aad+       , ByteArrayAccess ba+       , ByteArray output+       )+    => AES -> AESGCMKey -> iv -> aad -> ba -> Int -> (output, AuthTag)+gcmFullDecryptTag ctx (AESGCMKey gk) iv aad input taglen = unsafeDoIO $ do+    (tagbs, out) <- B.allocRet (B.length input) $ \outp ->+        B.alloc taglen $ \tagp ->+            B.withByteArray gk $ \gkp ->+                keyToPtr ctx $ \k ->+                    B.withByteArray iv $ \ivp ->+                        B.withByteArray aad $ \aadp ->+                            B.withByteArray input $ \inp ->+                                call+                                    outp+                                    tagp+                                    (castPtr gkp)+                                    k+                                    ivp+                                    (fromIntegral $ B.length iv)+                                    aadp+                                    (fromIntegral $ B.length aad)+                                    inp+                                    (fromIntegral $ B.length input)+                                    (fromIntegral taglen)+    return (out, AuthTag $ B.convert (tagbs :: B.Bytes))+  where+    call+        | B.length input <= shortMessage = c_aes_gcm_full_decrypt_tag_unsafe+        | otherwise = c_aes_gcm_full_decrypt_tag++-- | append data which is only going to be authenticated to the GCM context.+--+-- needs to happen after initialization and before appending encryption/decryption data.+{-# NOINLINE gcmAppendAAD #-}+gcmAppendAAD :: ByteArrayAccess aad => AESGCM -> aad -> AESGCM+gcmAppendAAD gcmSt input = unsafeDoIO doAppend+  where+    doAppend =+        withNewGCMSt gcmSt $ \gcmStPtr ->+            withByteArray input $ \i ->+                B.inCLengths (B.length input) $ \off n ->+                    c_aes_gcm_aad gcmStPtr (i `plusPtr` off) (fromIntegral n)++-- | append data to encrypt and append to the GCM context+--+-- the bytearray needs to be a multiple of AES block size, unless it's the last call to this function.+-- needs to happen after AAD appending, or after initialization if no AAD data.+{-# NOINLINE gcmAppendEncrypt #-}+gcmAppendEncrypt :: ByteArray ba => AES -> AESGCM -> ba -> (ba, AESGCM)+gcmAppendEncrypt ctx gcm input = unsafeDoIO $ withGCMKeyAndCopySt ctx gcm doEnc+  where+    len = B.length input+    doEnc gcmStPtr aesPtr =+        B.alloc len $ \o ->+            withByteArray input $ \i ->+                B.inCLengths len $ \off n ->+                    c_aes_gcm_encrypt+                        (castPtr o `plusPtr` off)+                        gcmStPtr+                        aesPtr+                        (i `plusPtr` off)+                        (fromIntegral n)++-- | append data to decrypt and append to the GCM context+--+-- the bytearray needs to be a multiple of AES block size, unless it's the last call to this function.+-- needs to happen after AAD appending, or after initialization if no AAD data.+{-# NOINLINE gcmAppendDecrypt #-}+gcmAppendDecrypt :: ByteArray ba => AES -> AESGCM -> ba -> (ba, AESGCM)+gcmAppendDecrypt ctx gcm input = unsafeDoIO $ withGCMKeyAndCopySt ctx gcm doDec+  where+    len = B.length input+    doDec gcmStPtr aesPtr =+        B.alloc len $ \o ->+            withByteArray input $ \i ->+                B.inCLengths len $ \off n ->+                    c_aes_gcm_decrypt+                        (castPtr o `plusPtr` off)+                        gcmStPtr+                        aesPtr+                        (i `plusPtr` off)+                        (fromIntegral n)++-- | Generate the Tag from GCM context+{-# NOINLINE gcmFinish #-}+gcmFinish :: AES -> AESGCM -> Int -> AuthTag+gcmFinish ctx gcm taglen = AuthTag $ B.take taglen computeTag+  where+    computeTag = B.allocAndFreeze 16 $ \t ->+        withGCMKeyAndCopySt ctx gcm (c_aes_gcm_finish (castPtr t)) >> return ()++------------------------------------------------------------------------+-- OCB v3+------------------------------------------------------------------------++-- | initialize an ocb context+{-# NOINLINE ocbInit #-}+ocbInit :: ByteArrayAccess iv => AES -> iv -> AESOCB+ocbInit ctx iv = unsafeDoIO $ do+    sm <- B.alloc sizeOCB $ \ocbStPtr ->+        withKeyAndIV ctx iv $ \k v ->+            c_aes_ocb_init+                (castPtr ocbStPtr)+                k+                v+                (fromIntegral $ B.length iv)+                16+    return $ AESOCB sm++-- | initialize an OCB context with a fixed authentication tag length.+--+-- The tag length is expressed in bytes and must be in [0..16].+-- The IV length must be in [1..15] bytes per RFC 7253.+{-# NOINLINE ocbInitWithTagLength #-}+ocbInitWithTagLength+    :: ByteArrayAccess iv => AES -> iv -> Int -> CryptoFailable AESOCB+ocbInitWithTagLength ctx iv taglen+    | taglen < 0 || taglen > 16 =+        CryptoFailed CryptoError_AuthenticationTagSizeInvalid+    | ivlen < 1 || ivlen > 15 = CryptoFailed CryptoError_IvSizeInvalid+    | otherwise = CryptoPassed $ unsafeDoIO $ do+        sm <- B.alloc sizeOCB $ \ocbStPtr ->+            withKeyAndIV ctx iv $ \k v ->+                c_aes_ocb_init+                    (castPtr ocbStPtr)+                    k+                    v+                    (fromIntegral ivlen)+                    (fromIntegral taglen)+        return $ AESOCB sm+  where+    ivlen = B.length iv++-- | append data which is going to just be authenticated to the OCB context.+--+-- need to happen after initialization and before appending encryption/decryption data.+{-# NOINLINE ocbAppendAAD #-}+ocbAppendAAD :: ByteArrayAccess aad => AES -> AESOCB -> aad -> AESOCB+ocbAppendAAD ctx ocb input+    | B.overCLength (B.length input) = error tooLongMessage+    | otherwise = unsafeDoIO (snd `fmap` withOCBKeyAndCopySt ctx ocb doAppend)+  where+    doAppend ocbStPtr aesPtr =+        withByteArray input $ \i ->+            c_aes_ocb_aad ocbStPtr aesPtr i (fromIntegral $ B.length input)++-- | append data to encrypt and append to the OCB context+--+-- the bytearray needs to be a multiple of the AES block size, unless it's the last call to this function.+-- need to happen after AAD appending, or after initialization if no AAD data.+{-# NOINLINE ocbAppendEncrypt #-}+ocbAppendEncrypt :: ByteArray ba => AES -> AESOCB -> ba -> (ba, AESOCB)+ocbAppendEncrypt ctx ocb input+    | B.overCLength (B.length input) = error tooLongMessage+    | otherwise = unsafeDoIO $ withOCBKeyAndCopySt ctx ocb doEnc+  where+    len = B.length input+    doEnc ocbStPtr aesPtr =+        B.alloc len $ \o ->+            withByteArray input $ \i ->+                c_aes_ocb_encrypt (castPtr o) ocbStPtr aesPtr i (fromIntegral len)++-- | append data to decrypt and append to the OCB context+--+-- the bytearray needs to be a multiple of the AES block size, unless it's the last call to this function.+-- need to happen after AAD appending, or after initialization if no AAD data.+{-# NOINLINE ocbAppendDecrypt #-}+ocbAppendDecrypt :: ByteArray ba => AES -> AESOCB -> ba -> (ba, AESOCB)+ocbAppendDecrypt ctx ocb input+    | B.overCLength (B.length input) = error tooLongMessage+    | otherwise = unsafeDoIO $ withOCBKeyAndCopySt ctx ocb doDec+  where+    len = B.length input+    doDec ocbStPtr aesPtr =+        B.alloc len $ \o ->+            withByteArray input $ \i ->+                c_aes_ocb_decrypt (castPtr o) ocbStPtr aesPtr i (fromIntegral len)++-- | Generate the Tag from OCB context+{-# NOINLINE ocbFinish #-}+ocbFinish :: AES -> AESOCB -> Int -> AuthTag+ocbFinish ctx ocb taglen = AuthTag $ B.take taglen computeTag+  where+    computeTag = B.allocAndFreeze 16 $ \t ->+        withOCBKeyAndCopySt ctx ocb (c_aes_ocb_finish (castPtr t)) >> return ()++ccmGetM :: CCM_M -> Int+ccmGetL :: CCM_L -> Int+ccmGetM m = case m of+    CCM_M4 -> 4+    CCM_M6 -> 6+    CCM_M8 -> 8+    CCM_M10 -> 10+    CCM_M12 -> 12+    CCM_M14 -> 14+    CCM_M16 -> 16++ccmGetL l = case l of+    CCM_L2 -> 2+    CCM_L3 -> 3+    CCM_L4 -> 4++-- | initialize a ccm context+{-# NOINLINE ccmInit #-}+ccmInit+    :: ByteArrayAccess iv+    => AES -> iv -> Int -> CCM_M -> CCM_L -> CryptoFailable AESCCM+ccmInit ctx iv n m l+    | 15 - li /= B.length iv = CryptoFailed CryptoError_IvSizeInvalid+    | otherwise = unsafeDoIO $ do+        sm <- B.alloc sizeCCM $ \ccmStPtr ->+            withKeyAndIV ctx iv $ \k v ->+                c_aes_ccm_init+                    (castPtr ccmStPtr)+                    k+                    v+                    (fromIntegral $ B.length iv)+                    (fromIntegral n)+                    (fromIntegral mi)+                    (fromIntegral li)+        return $ CryptoPassed (AESCCM sm)+  where+    mi = ccmGetM m+    li = ccmGetL l++-- | append data which is only going to be authenticated to the CCM context.+--+-- needs to happen after initialization and before appending encryption/decryption data.+{-# NOINLINE ccmAppendAAD #-}+ccmAppendAAD :: ByteArrayAccess aad => AES -> AESCCM -> aad -> AESCCM+ccmAppendAAD ctx ccm input+    | B.overCLength (B.length input) = error tooLongMessage+    | otherwise = unsafeDoIO $ snd <$> withCCMKeyAndCopySt ctx ccm doAppend+  where+    doAppend ccmStPtr aesPtr =+        withByteArray input $ \i -> c_aes_ccm_aad ccmStPtr aesPtr i (fromIntegral $ B.length input)++-- | append data to encrypt and append to the CCM context+--+-- the bytearray needs to be a multiple of AES block size, unless it's the last call to this function.+-- needs to happen after AAD appending, or after initialization if no AAD data.+{-# NOINLINE ccmEncrypt #-}+ccmEncrypt :: ByteArray ba => AES -> AESCCM -> ba -> (ba, AESCCM)+ccmEncrypt ctx ccm input+    | B.overCLength (B.length input) = error tooLongMessage+    | otherwise = unsafeDoIO $ withCCMKeyAndCopySt ctx ccm cbcmacAndIv+  where+    len = B.length input+    cbcmacAndIv ccmStPtr aesPtr =+        B.alloc len $ \o ->+            withByteArray input $ \i ->+                c_aes_ccm_encrypt (castPtr o) ccmStPtr aesPtr i (fromIntegral len)++-- | append data to decrypt and append to the CCM context+--+-- the bytearray needs to be a multiple of AES block size, unless it's the last call to this function.+-- needs to happen after AAD appending, or after initialization if no AAD data.+{-# NOINLINE ccmDecrypt #-}+ccmDecrypt :: ByteArray ba => AES -> AESCCM -> ba -> (ba, AESCCM)+ccmDecrypt ctx ccm input+    | B.overCLength (B.length input) = error tooLongMessage+    | otherwise = unsafeDoIO $ withCCMKeyAndCopySt ctx ccm cbcmacAndIv+  where+    len = B.length input+    cbcmacAndIv ccmStPtr aesPtr =+        B.alloc len $ \o ->+            withByteArray input $ \i ->+                c_aes_ccm_decrypt (castPtr o) ccmStPtr aesPtr i (fromIntegral len)++-- | Generate the Tag from CCM context+{-# NOINLINE ccmFinish #-}+ccmFinish :: AES -> AESCCM -> Int -> AuthTag+ccmFinish ctx ccm taglen = AuthTag $ B.take taglen computeTag+  where+    computeTag = B.allocAndFreeze 16 $ \t ->+        withCCMKeyAndCopySt ctx ccm (c_aes_ccm_finish (castPtr t)) >> return ()++------------------------------------------------------------------------+foreign import ccall "crypton_aes.h crypton_aes_initkey"+    c_aes_init :: Ptr AES -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_encrypt_ecb"+    c_aes_encrypt_ecb :: CString -> Ptr AES -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_decrypt_ecb"+    c_aes_decrypt_ecb :: CString -> Ptr AES -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_encrypt_cbc"+    c_aes_encrypt_cbc+        :: CString -> Ptr AES -> Ptr Word8 -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_decrypt_cbc"+    c_aes_decrypt_cbc+        :: CString -> Ptr AES -> Ptr Word8 -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_encrypt_xts"+    c_aes_encrypt_xts+        :: CString -> Ptr AES -> Ptr AES -> Ptr Word8 -> CUInt -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_decrypt_xts"+    c_aes_decrypt_xts+        :: CString -> Ptr AES -> Ptr AES -> Ptr Word8 -> CUInt -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_encrypt_ctr"+    c_aes_encrypt_ctr+        :: CString -> Ptr AES -> Ptr Word8 -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_encrypt_c32"+    c_aes_encrypt_c32+        :: CString -> Ptr AES -> Ptr Word8 -> CString -> CUInt -> IO ()++foreign import ccall unsafe "crypton_aes.h crypton_aes_gcm_key_init"+    c_aes_gcm_key_init :: Ptr AESGCM -> Ptr AES -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_gcm_full_encrypt"+    c_aes_gcm_full_encrypt+        :: Ptr Word8+        -> Ptr AESGCM+        -> Ptr AES+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> CUInt+        -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_gcm_full_decrypt"+    c_aes_gcm_full_decrypt+        :: Ptr Word8+        -> Ptr AESGCM+        -> Ptr AES+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> IO CInt++foreign import ccall "crypton_aes.h crypton_aes_gcm_full_decrypt_tag"+    c_aes_gcm_full_decrypt_tag+        :: Ptr Word8+        -> Ptr Word8+        -> Ptr AESGCM+        -> Ptr AES+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> CUInt+        -> IO ()++foreign import ccall unsafe "crypton_aes.h crypton_aes_gcm_full_decrypt_tag"+    c_aes_gcm_full_decrypt_tag_unsafe+        :: Ptr Word8+        -> Ptr Word8+        -> Ptr AESGCM+        -> Ptr AES+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> CUInt+        -> IO ()++foreign import ccall unsafe "crypton_aes.h crypton_aes_gcm_full_encrypt"+    c_aes_gcm_full_encrypt_unsafe+        :: Ptr Word8+        -> Ptr AESGCM+        -> Ptr AES+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> CUInt+        -> IO ()++foreign import ccall unsafe "crypton_aes.h crypton_aes_gcm_full_decrypt"+    c_aes_gcm_full_decrypt_unsafe+        :: Ptr Word8+        -> Ptr AESGCM+        -> Ptr AES+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> IO CInt++foreign import ccall "crypton_aes.h crypton_aes_gcm_full_encrypt_mask"+    c_aes_gcm_full_encrypt_mask+        :: Ptr Word8+        -> Ptr AESGCM+        -> Ptr AES+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> CUInt+        -> Ptr AES+        -> CUInt+        -> Ptr Word8+        -> IO ()++foreign import ccall unsafe "crypton_aes.h crypton_aes_gcm_full_encrypt_mask"+    c_aes_gcm_full_encrypt_mask_unsafe+        :: Ptr Word8+        -> Ptr AESGCM+        -> Ptr AES+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> CUInt+        -> Ptr AES+        -> CUInt+        -> Ptr Word8+        -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_gcm_init"+    c_aes_gcm_init :: Ptr AESGCM -> Ptr AES -> Ptr Word8 -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_gcm_aad"+    c_aes_gcm_aad :: Ptr AESGCM -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_gcm_encrypt"+    c_aes_gcm_encrypt+        :: CString -> Ptr AESGCM -> Ptr AES -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_gcm_decrypt"+    c_aes_gcm_decrypt+        :: CString -> Ptr AESGCM -> Ptr AES -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_gcm_finish"+    c_aes_gcm_finish :: CString -> Ptr AESGCM -> Ptr AES -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_ocb_init"+    c_aes_ocb_init+        :: Ptr AESOCB -> Ptr AES -> Ptr Word8 -> CUInt -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_ocb_aad"+    c_aes_ocb_aad :: Ptr AESOCB -> Ptr AES -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_ocb_encrypt"+    c_aes_ocb_encrypt+        :: CString -> Ptr AESOCB -> Ptr AES -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_ocb_decrypt"+    c_aes_ocb_decrypt+        :: CString -> Ptr AESOCB -> Ptr AES -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_ocb_finish"+    c_aes_ocb_finish :: CString -> Ptr AESOCB -> Ptr AES -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_ccm_init"+    c_aes_ccm_init+        :: Ptr AESCCM -> Ptr AES -> Ptr Word8 -> CUInt -> CUInt -> CInt -> CInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_ccm_aad"+    c_aes_ccm_aad :: Ptr AESCCM -> Ptr AES -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_ccm_encrypt"+    c_aes_ccm_encrypt+        :: CString -> Ptr AESCCM -> Ptr AES -> CString -> CUInt -> IO ()++foreign import ccall "crypton_aes.h crypton_aes_ccm_decrypt"+    c_aes_ccm_decrypt+        :: CString -> Ptr AESCCM -> Ptr AES -> CString -> CUInt -> IO ()  foreign import ccall "crypton_aes.h crypton_aes_ccm_finish"     c_aes_ccm_finish :: CString -> Ptr AESCCM -> Ptr AES -> IO ()
Crypto/Cipher/AESGCMSIV.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.Cipher.AESGCMSIV -- License     : BSD-style@@ -16,28 +19,27 @@ -- -- The specification allows inputs up to 2^36 bytes but this implementation -- requires AAD and plaintext/ciphertext to be both smaller than 2^32 bytes.-{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.Cipher.AESGCMSIV-    ( Nonce-    , nonce-    , generateNonce-    , encrypt-    , decrypt-    ) where+module Crypto.Cipher.AESGCMSIV (+    Nonce,+    nonce,+    generateNonce,+    encrypt,+    decrypt,+) where  import Data.Bits+import Data.Maybe import Data.Word -import Foreign.C.Types import Foreign.C.String+import Foreign.C.Types import Foreign.Ptr (Ptr, plusPtr) import Foreign.Storable (peekElemOff, poke, pokeElemOff) -import           Data.ByteArray+import Data.ByteArray (ByteArray, ByteArrayAccess, Bytes, ScrubbedBytes) import qualified Data.ByteArray as B-import           Data.Memory.Endian (toLE)-import           Data.Memory.PtrMethods (memXor)+import Data.Memory.Endian (toLE)+import Data.Memory.PtrMethods (memXor)  import Crypto.Cipher.AES.Primitive import Crypto.Cipher.Types@@ -45,7 +47,6 @@ import Crypto.Internal.Compat (unsafeDoIO) import Crypto.Random - -- 12-byte nonces  -- | Nonce value for AES-GCM-SIV, always 12 bytes.@@ -55,26 +56,27 @@ nonce :: ByteArrayAccess iv => iv -> CryptoFailable Nonce nonce iv     | B.length iv == 12 = CryptoPassed (Nonce $ B.convert iv)-    | otherwise         = CryptoFailed CryptoError_IvSizeInvalid+    | otherwise = CryptoFailed CryptoError_IvSizeInvalid  -- | Generate a random nonce for use with AES-GCM-SIV. generateNonce :: MonadRandom m => m Nonce generateNonce = Nonce <$> getRandomBytes 12 - -- POLYVAL (mutable context)  newtype Polyval = Polyval Bytes  polyvalInit :: ScrubbedBytes -> IO Polyval polyvalInit h = Polyval <$> doInit-  where doInit = B.alloc 272 $ \pctx -> B.withByteArray h $ \ph ->-            c_aes_polyval_init pctx ph+  where+    doInit = B.alloc 272 $ \pctx -> B.withByteArray h $ \ph ->+        c_aes_polyval_init pctx ph  polyvalUpdate :: ByteArrayAccess ba => Polyval -> ba -> IO () polyvalUpdate (Polyval ctx) bs = B.withByteArray ctx $ \pctx ->     B.withByteArray bs $ \pbs -> c_aes_polyval_update pctx pbs sz-  where sz = fromIntegral (B.length bs)+  where+    sz = fromIntegral (B.length bs)  polyvalFinalize :: Polyval -> IO ScrubbedBytes polyvalFinalize (Polyval ctx) = B.alloc 16 $ \dst ->@@ -89,35 +91,35 @@ foreign import ccall unsafe "crypton_aes.h crypton_aes_polyval_finalize"     c_aes_polyval_finalize :: Ptr Polyval -> CString -> IO () - -- Key Generation  le32iv :: Word32 -> Nonce -> Bytes le32iv n (Nonce iv) = B.allocAndFreeze 16 $ \ptr -> do     poke ptr (toLE n)-    copyByteArrayToPtr iv (ptr `plusPtr` 4)+    B.copyByteArrayToPtr iv (ptr `plusPtr` 4)  deriveKeys :: BlockCipher128 aes => aes -> Nonce -> (ScrubbedBytes, AES) deriveKeys aes iv =     case cipherKeySize aes of-        KeySizeFixed sz | sz `mod` 8 == 0 ->-            let mak = buildKey [0 .. 1]-                key = buildKey [2 .. fromIntegral (sz `div` 8) + 1]-                mek = throwCryptoError (cipherInit key)-             in (mak, mek)+        KeySizeFixed sz+            | sz `mod` 8 == 0 ->+                let mak = buildKey [0 .. 1]+                    key = buildKey [2 .. fromIntegral (sz `div` 8) + 1]+                    mek = throwCryptoError (cipherInit key)+                 in (mak, mek)         _ -> error "AESGCMSIV: invalid cipher"   where-    idx n = ecbEncrypt aes (le32iv n iv) `takeView` 8+    idx n = ecbEncrypt aes (le32iv n iv) `B.takeView` 8     buildKey = B.concat . map idx - -- Encryption and decryption  lengthInvalid :: ByteArrayAccess ba => ba -> Bool lengthInvalid bs     | finiteBitSize len > 32 = len >= 1 `unsafeShiftL` 32-    | otherwise              = False-  where len = B.length bs+    | otherwise = False+  where+    len = B.length bs  -- | AEAD encryption with the specified key and nonce.  The key must be given -- as an initialized 'Crypto.Cipher.AES.AES128' or 'Crypto.Cipher.AES.AES256'@@ -125,8 +127,9 @@ -- -- Lengths of additional data and plaintext must be less than 2^32 bytes, -- otherwise an exception is thrown.-encrypt :: (BlockCipher128 aes, ByteArrayAccess aad, ByteArray ba)-        => aes -> Nonce -> aad -> ba -> (AuthTag, ba)+encrypt+    :: (BlockCipher128 aes, ByteArrayAccess aad, ByteArray ba)+    => aes -> Nonce -> aad -> ba -> (AuthTag, ba) encrypt aes iv aad plaintext     | lengthInvalid aad = error "AESGCMSIV: aad is too large"     | lengthInvalid plaintext = error "AESGCMSIV: plaintext is too large"@@ -143,12 +146,13 @@ -- -- Lengths of additional data and ciphertext must be less than 2^32 bytes, -- otherwise an exception is thrown.-decrypt :: (BlockCipher128 aes, ByteArrayAccess aad, ByteArray ba)-        => aes -> Nonce -> aad -> ba -> AuthTag -> Maybe ba+decrypt+    :: (BlockCipher128 aes, ByteArrayAccess aad, ByteArray ba)+    => aes -> Nonce -> aad -> ba -> AuthTag -> Maybe ba decrypt aes iv aad ciphertext (AuthTag tag)     | lengthInvalid aad = error "AESGCMSIV: aad is too large"     | lengthInvalid ciphertext = error "AESGCMSIV: ciphertext is too large"-    | tag `constEq` buildTag mek ss iv = Just plaintext+    | tag `B.constEq` buildTag mek ss iv = Just plaintext     | otherwise = Nothing   where     (mak, mek) = deriveKeys aes iv@@ -156,18 +160,19 @@     plaintext = combineC32 mek (transformTag tag) ciphertext  -- Calculate S_s = POLYVAL(mak, X_1, X_2, ...).-getSs :: (ByteArrayAccess aad, ByteArrayAccess ba)-      => ScrubbedBytes -> aad -> ba -> ScrubbedBytes+getSs+    :: (ByteArrayAccess aad, ByteArrayAccess ba)+    => ScrubbedBytes -> aad -> ba -> ScrubbedBytes getSs mak aad plaintext = unsafeDoIO $ do     ctx <- polyvalInit mak     polyvalUpdate ctx aad     polyvalUpdate ctx plaintext-    polyvalUpdate ctx (lb :: Bytes)  -- the "length block"+    polyvalUpdate ctx (lb :: Bytes) -- the "length block"     polyvalFinalize ctx   where     lb = B.allocAndFreeze 16 $ \ptr -> do-            pokeElemOff ptr 0 (toLE64 $ B.length aad)-            pokeElemOff ptr 1 (toLE64 $ B.length plaintext)+        pokeElemOff ptr 0 (toLE64 $ B.length aad)+        pokeElemOff ptr 1 (toLE64 $ B.length plaintext)     toLE64 x = toLE (fromIntegral x * 8 :: Word64)  -- XOR the first 12 bytes of S_s with the nonce and clear the most significant@@ -175,10 +180,10 @@ tagInput :: ScrubbedBytes -> Nonce -> Bytes tagInput ss (Nonce iv) =     B.copyAndFreeze ss $ \ptr ->-    B.withByteArray iv $ \ivPtr -> do-        memXor ptr ptr ivPtr 12-        b <- peekElemOff ptr 15-        pokeElemOff ptr 15 (b .&. (0x7f :: Word8))+        B.withByteArray iv $ \ivPtr -> do+            memXor ptr ptr ivPtr 12+            b <- peekElemOff ptr 15+            pokeElemOff ptr 15 (b .&. (0x7f :: Word8))  -- Encrypt the result with AES using the message-encryption key to produce the -- tag.@@ -190,4 +195,5 @@ transformTag :: Bytes -> IV AES transformTag tag = toIV $ B.copyAndFreeze tag $ \ptr ->     peekElemOff ptr 15 >>= pokeElemOff ptr 15 . (.|. (0x80 :: Word8))-  where toIV bs = let Just iv = makeIV (bs :: Bytes) in iv+  where+    toIV bs = fromJust $ makeIV (bs :: Bytes)
Crypto/Cipher/Blowfish.hs view
@@ -1,23 +1,23 @@+{-# LANGUAGE CPP #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.Cipher.Blowfish -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : stable -- Portability : good----{-# LANGUAGE CPP #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.Cipher.Blowfish-    ( Blowfish-    , Blowfish64-    , Blowfish128-    , Blowfish256-    , Blowfish448-    ) where+module Crypto.Cipher.Blowfish (+    Blowfish,+    Blowfish64,+    Blowfish128,+    Blowfish256,+    Blowfish448,+) where -import Crypto.Internal.Imports-import Crypto.Cipher.Types import Crypto.Cipher.Blowfish.Primitive+import Crypto.Cipher.Types+import Crypto.Internal.Imports  -- | variable keyed blowfish state newtype Blowfish = Blowfish Context@@ -40,9 +40,9 @@     deriving (NFData)  instance Cipher Blowfish where-    cipherName _    = "blowfish"+    cipherName _ = "blowfish"     cipherKeySize _ = KeySizeRange 6 56-    cipherInit k    = Blowfish `fmap` initBlowfish k+    cipherInit k = Blowfish `fmap` initBlowfish k  instance BlockCipher Blowfish where     blockSize _ = 8
− Crypto/Cipher/Blowfish/Box.hs 
@@ -1,296 +0,0 @@--- |--- Module      : Crypto.Cipher.Blowfish.Box--- License     : BSD-style--- Stability   : experimental--- Portability : Good-{-# LANGUAGE MagicHash #-}-module Crypto.Cipher.Blowfish.Box-    (   KeySchedule(..)-    ,   createKeySchedule-    ,   copyKeySchedule-    ) where--import           Crypto.Internal.WordArray (MutableArray32,-                                            mutableArray32FromAddrBE,-                                            mutableArrayRead32,-                                            mutableArrayWrite32)--newtype KeySchedule = KeySchedule MutableArray32---- | Copy the state of one key schedule into the other.---   The first parameter is the destination and the second the source.-copyKeySchedule :: KeySchedule -> KeySchedule -> IO ()-copyKeySchedule (KeySchedule dst) (KeySchedule src) = loop 0-  where-    loop 1042 = return ()-    loop i    = do-        w32 <-mutableArrayRead32 src i-        mutableArrayWrite32 dst i w32-        loop (i + 1)---- | Create a key schedule mutable array of the pbox followed by--- all the sboxes.-createKeySchedule :: IO KeySchedule-createKeySchedule = KeySchedule `fmap` mutableArray32FromAddrBE 1042 "\-    \\x24\x3f\x6a\x88\x85\xa3\x08\xd3\x13\x19\x8a\x2e\x03\x70\x73\x44\-    \\xa4\x09\x38\x22\x29\x9f\x31\xd0\x08\x2e\xfa\x98\xec\x4e\x6c\x89\-    \\x45\x28\x21\xe6\x38\xd0\x13\x77\xbe\x54\x66\xcf\x34\xe9\x0c\x6c\-    \\xc0\xac\x29\xb7\xc9\x7c\x50\xdd\x3f\x84\xd5\xb5\xb5\x47\x09\x17\-    \\x92\x16\xd5\xd9\x89\x79\xfb\x1b\-    \\xd1\x31\x0b\xa6\x98\xdf\xb5\xac\x2f\xfd\x72\xdb\xd0\x1a\xdf\xb7\-    \\xb8\xe1\xaf\xed\x6a\x26\x7e\x96\xba\x7c\x90\x45\xf1\x2c\x7f\x99\-    \\x24\xa1\x99\x47\xb3\x91\x6c\xf7\x08\x01\xf2\xe2\x85\x8e\xfc\x16\-    \\x63\x69\x20\xd8\x71\x57\x4e\x69\xa4\x58\xfe\xa3\xf4\x93\x3d\x7e\-    \\x0d\x95\x74\x8f\x72\x8e\xb6\x58\x71\x8b\xcd\x58\x82\x15\x4a\xee\-    \\x7b\x54\xa4\x1d\xc2\x5a\x59\xb5\x9c\x30\xd5\x39\x2a\xf2\x60\x13\-    \\xc5\xd1\xb0\x23\x28\x60\x85\xf0\xca\x41\x79\x18\xb8\xdb\x38\xef\-    \\x8e\x79\xdc\xb0\x60\x3a\x18\x0e\x6c\x9e\x0e\x8b\xb0\x1e\x8a\x3e\-    \\xd7\x15\x77\xc1\xbd\x31\x4b\x27\x78\xaf\x2f\xda\x55\x60\x5c\x60\-    \\xe6\x55\x25\xf3\xaa\x55\xab\x94\x57\x48\x98\x62\x63\xe8\x14\x40\-    \\x55\xca\x39\x6a\x2a\xab\x10\xb6\xb4\xcc\x5c\x34\x11\x41\xe8\xce\-    \\xa1\x54\x86\xaf\x7c\x72\xe9\x93\xb3\xee\x14\x11\x63\x6f\xbc\x2a\-    \\x2b\xa9\xc5\x5d\x74\x18\x31\xf6\xce\x5c\x3e\x16\x9b\x87\x93\x1e\-    \\xaf\xd6\xba\x33\x6c\x24\xcf\x5c\x7a\x32\x53\x81\x28\x95\x86\x77\-    \\x3b\x8f\x48\x98\x6b\x4b\xb9\xaf\xc4\xbf\xe8\x1b\x66\x28\x21\x93\-    \\x61\xd8\x09\xcc\xfb\x21\xa9\x91\x48\x7c\xac\x60\x5d\xec\x80\x32\-    \\xef\x84\x5d\x5d\xe9\x85\x75\xb1\xdc\x26\x23\x02\xeb\x65\x1b\x88\-    \\x23\x89\x3e\x81\xd3\x96\xac\xc5\x0f\x6d\x6f\xf3\x83\xf4\x42\x39\-    \\x2e\x0b\x44\x82\xa4\x84\x20\x04\x69\xc8\xf0\x4a\x9e\x1f\x9b\x5e\-    \\x21\xc6\x68\x42\xf6\xe9\x6c\x9a\x67\x0c\x9c\x61\xab\xd3\x88\xf0\-    \\x6a\x51\xa0\xd2\xd8\x54\x2f\x68\x96\x0f\xa7\x28\xab\x51\x33\xa3\-    \\x6e\xef\x0b\x6c\x13\x7a\x3b\xe4\xba\x3b\xf0\x50\x7e\xfb\x2a\x98\-    \\xa1\xf1\x65\x1d\x39\xaf\x01\x76\x66\xca\x59\x3e\x82\x43\x0e\x88\-    \\x8c\xee\x86\x19\x45\x6f\x9f\xb4\x7d\x84\xa5\xc3\x3b\x8b\x5e\xbe\-    \\xe0\x6f\x75\xd8\x85\xc1\x20\x73\x40\x1a\x44\x9f\x56\xc1\x6a\xa6\-    \\x4e\xd3\xaa\x62\x36\x3f\x77\x06\x1b\xfe\xdf\x72\x42\x9b\x02\x3d\-    \\x37\xd0\xd7\x24\xd0\x0a\x12\x48\xdb\x0f\xea\xd3\x49\xf1\xc0\x9b\-    \\x07\x53\x72\xc9\x80\x99\x1b\x7b\x25\xd4\x79\xd8\xf6\xe8\xde\xf7\-    \\xe3\xfe\x50\x1a\xb6\x79\x4c\x3b\x97\x6c\xe0\xbd\x04\xc0\x06\xba\-    \\xc1\xa9\x4f\xb6\x40\x9f\x60\xc4\x5e\x5c\x9e\xc2\x19\x6a\x24\x63\-    \\x68\xfb\x6f\xaf\x3e\x6c\x53\xb5\x13\x39\xb2\xeb\x3b\x52\xec\x6f\-    \\x6d\xfc\x51\x1f\x9b\x30\x95\x2c\xcc\x81\x45\x44\xaf\x5e\xbd\x09\-    \\xbe\xe3\xd0\x04\xde\x33\x4a\xfd\x66\x0f\x28\x07\x19\x2e\x4b\xb3\-    \\xc0\xcb\xa8\x57\x45\xc8\x74\x0f\xd2\x0b\x5f\x39\xb9\xd3\xfb\xdb\-    \\x55\x79\xc0\xbd\x1a\x60\x32\x0a\xd6\xa1\x00\xc6\x40\x2c\x72\x79\-    \\x67\x9f\x25\xfe\xfb\x1f\xa3\xcc\x8e\xa5\xe9\xf8\xdb\x32\x22\xf8\-    \\x3c\x75\x16\xdf\xfd\x61\x6b\x15\x2f\x50\x1e\xc8\xad\x05\x52\xab\-    \\x32\x3d\xb5\xfa\xfd\x23\x87\x60\x53\x31\x7b\x48\x3e\x00\xdf\x82\-    \\x9e\x5c\x57\xbb\xca\x6f\x8c\xa0\x1a\x87\x56\x2e\xdf\x17\x69\xdb\-    \\xd5\x42\xa8\xf6\x28\x7e\xff\xc3\xac\x67\x32\xc6\x8c\x4f\x55\x73\-    \\x69\x5b\x27\xb0\xbb\xca\x58\xc8\xe1\xff\xa3\x5d\xb8\xf0\x11\xa0\-    \\x10\xfa\x3d\x98\xfd\x21\x83\xb8\x4a\xfc\xb5\x6c\x2d\xd1\xd3\x5b\-    \\x9a\x53\xe4\x79\xb6\xf8\x45\x65\xd2\x8e\x49\xbc\x4b\xfb\x97\x90\-    \\xe1\xdd\xf2\xda\xa4\xcb\x7e\x33\x62\xfb\x13\x41\xce\xe4\xc6\xe8\-    \\xef\x20\xca\xda\x36\x77\x4c\x01\xd0\x7e\x9e\xfe\x2b\xf1\x1f\xb4\-    \\x95\xdb\xda\x4d\xae\x90\x91\x98\xea\xad\x8e\x71\x6b\x93\xd5\xa0\-    \\xd0\x8e\xd1\xd0\xaf\xc7\x25\xe0\x8e\x3c\x5b\x2f\x8e\x75\x94\xb7\-    \\x8f\xf6\xe2\xfb\xf2\x12\x2b\x64\x88\x88\xb8\x12\x90\x0d\xf0\x1c\-    \\x4f\xad\x5e\xa0\x68\x8f\xc3\x1c\xd1\xcf\xf1\x91\xb3\xa8\xc1\xad\-    \\x2f\x2f\x22\x18\xbe\x0e\x17\x77\xea\x75\x2d\xfe\x8b\x02\x1f\xa1\-    \\xe5\xa0\xcc\x0f\xb5\x6f\x74\xe8\x18\xac\xf3\xd6\xce\x89\xe2\x99\-    \\xb4\xa8\x4f\xe0\xfd\x13\xe0\xb7\x7c\xc4\x3b\x81\xd2\xad\xa8\xd9\-    \\x16\x5f\xa2\x66\x80\x95\x77\x05\x93\xcc\x73\x14\x21\x1a\x14\x77\-    \\xe6\xad\x20\x65\x77\xb5\xfa\x86\xc7\x54\x42\xf5\xfb\x9d\x35\xcf\-    \\xeb\xcd\xaf\x0c\x7b\x3e\x89\xa0\xd6\x41\x1b\xd3\xae\x1e\x7e\x49\-    \\x00\x25\x0e\x2d\x20\x71\xb3\x5e\x22\x68\x00\xbb\x57\xb8\xe0\xaf\-    \\x24\x64\x36\x9b\xf0\x09\xb9\x1e\x55\x63\x91\x1d\x59\xdf\xa6\xaa\-    \\x78\xc1\x43\x89\xd9\x5a\x53\x7f\x20\x7d\x5b\xa2\x02\xe5\xb9\xc5\-    \\x83\x26\x03\x76\x62\x95\xcf\xa9\x11\xc8\x19\x68\x4e\x73\x4a\x41\-    \\xb3\x47\x2d\xca\x7b\x14\xa9\x4a\x1b\x51\x00\x52\x9a\x53\x29\x15\-    \\xd6\x0f\x57\x3f\xbc\x9b\xc6\xe4\x2b\x60\xa4\x76\x81\xe6\x74\x00\-    \\x08\xba\x6f\xb5\x57\x1b\xe9\x1f\xf2\x96\xec\x6b\x2a\x0d\xd9\x15\-    \\xb6\x63\x65\x21\xe7\xb9\xf9\xb6\xff\x34\x05\x2e\xc5\x85\x56\x64\-    \\x53\xb0\x2d\x5d\xa9\x9f\x8f\xa1\x08\xba\x47\x99\x6e\x85\x07\x6a\-    \\x4b\x7a\x70\xe9\xb5\xb3\x29\x44\xdb\x75\x09\x2e\xc4\x19\x26\x23\-    \\xad\x6e\xa6\xb0\x49\xa7\xdf\x7d\x9c\xee\x60\xb8\x8f\xed\xb2\x66\-    \\xec\xaa\x8c\x71\x69\x9a\x17\xff\x56\x64\x52\x6c\xc2\xb1\x9e\xe1\-    \\x19\x36\x02\xa5\x75\x09\x4c\x29\xa0\x59\x13\x40\xe4\x18\x3a\x3e\-    \\x3f\x54\x98\x9a\x5b\x42\x9d\x65\x6b\x8f\xe4\xd6\x99\xf7\x3f\xd6\-    \\xa1\xd2\x9c\x07\xef\xe8\x30\xf5\x4d\x2d\x38\xe6\xf0\x25\x5d\xc1\-    \\x4c\xdd\x20\x86\x84\x70\xeb\x26\x63\x82\xe9\xc6\x02\x1e\xcc\x5e\-    \\x09\x68\x6b\x3f\x3e\xba\xef\xc9\x3c\x97\x18\x14\x6b\x6a\x70\xa1\-    \\x68\x7f\x35\x84\x52\xa0\xe2\x86\xb7\x9c\x53\x05\xaa\x50\x07\x37\-    \\x3e\x07\x84\x1c\x7f\xde\xae\x5c\x8e\x7d\x44\xec\x57\x16\xf2\xb8\-    \\xb0\x3a\xda\x37\xf0\x50\x0c\x0d\xf0\x1c\x1f\x04\x02\x00\xb3\xff\-    \\xae\x0c\xf5\x1a\x3c\xb5\x74\xb2\x25\x83\x7a\x58\xdc\x09\x21\xbd\-    \\xd1\x91\x13\xf9\x7c\xa9\x2f\xf6\x94\x32\x47\x73\x22\xf5\x47\x01\-    \\x3a\xe5\xe5\x81\x37\xc2\xda\xdc\xc8\xb5\x76\x34\x9a\xf3\xdd\xa7\-    \\xa9\x44\x61\x46\x0f\xd0\x03\x0e\xec\xc8\xc7\x3e\xa4\x75\x1e\x41\-    \\xe2\x38\xcd\x99\x3b\xea\x0e\x2f\x32\x80\xbb\xa1\x18\x3e\xb3\x31\-    \\x4e\x54\x8b\x38\x4f\x6d\xb9\x08\x6f\x42\x0d\x03\xf6\x0a\x04\xbf\-    \\x2c\xb8\x12\x90\x24\x97\x7c\x79\x56\x79\xb0\x72\xbc\xaf\x89\xaf\-    \\xde\x9a\x77\x1f\xd9\x93\x08\x10\xb3\x8b\xae\x12\xdc\xcf\x3f\x2e\-    \\x55\x12\x72\x1f\x2e\x6b\x71\x24\x50\x1a\xdd\xe6\x9f\x84\xcd\x87\-    \\x7a\x58\x47\x18\x74\x08\xda\x17\xbc\x9f\x9a\xbc\xe9\x4b\x7d\x8c\-    \\xec\x7a\xec\x3a\xdb\x85\x1d\xfa\x63\x09\x43\x66\xc4\x64\xc3\xd2\-    \\xef\x1c\x18\x47\x32\x15\xd9\x08\xdd\x43\x3b\x37\x24\xc2\xba\x16\-    \\x12\xa1\x4d\x43\x2a\x65\xc4\x51\x50\x94\x00\x02\x13\x3a\xe4\xdd\-    \\x71\xdf\xf8\x9e\x10\x31\x4e\x55\x81\xac\x77\xd6\x5f\x11\x19\x9b\-    \\x04\x35\x56\xf1\xd7\xa3\xc7\x6b\x3c\x11\x18\x3b\x59\x24\xa5\x09\-    \\xf2\x8f\xe6\xed\x97\xf1\xfb\xfa\x9e\xba\xbf\x2c\x1e\x15\x3c\x6e\-    \\x86\xe3\x45\x70\xea\xe9\x6f\xb1\x86\x0e\x5e\x0a\x5a\x3e\x2a\xb3\-    \\x77\x1f\xe7\x1c\x4e\x3d\x06\xfa\x29\x65\xdc\xb9\x99\xe7\x1d\x0f\-    \\x80\x3e\x89\xd6\x52\x66\xc8\x25\x2e\x4c\xc9\x78\x9c\x10\xb3\x6a\-    \\xc6\x15\x0e\xba\x94\xe2\xea\x78\xa5\xfc\x3c\x53\x1e\x0a\x2d\xf4\-    \\xf2\xf7\x4e\xa7\x36\x1d\x2b\x3d\x19\x39\x26\x0f\x19\xc2\x79\x60\-    \\x52\x23\xa7\x08\xf7\x13\x12\xb6\xeb\xad\xfe\x6e\xea\xc3\x1f\x66\-    \\xe3\xbc\x45\x95\xa6\x7b\xc8\x83\xb1\x7f\x37\xd1\x01\x8c\xff\x28\-    \\xc3\x32\xdd\xef\xbe\x6c\x5a\xa5\x65\x58\x21\x85\x68\xab\x98\x02\-    \\xee\xce\xa5\x0f\xdb\x2f\x95\x3b\x2a\xef\x7d\xad\x5b\x6e\x2f\x84\-    \\x15\x21\xb6\x28\x29\x07\x61\x70\xec\xdd\x47\x75\x61\x9f\x15\x10\-    \\x13\xcc\xa8\x30\xeb\x61\xbd\x96\x03\x34\xfe\x1e\xaa\x03\x63\xcf\-    \\xb5\x73\x5c\x90\x4c\x70\xa2\x39\xd5\x9e\x9e\x0b\xcb\xaa\xde\x14\-    \\xee\xcc\x86\xbc\x60\x62\x2c\xa7\x9c\xab\x5c\xab\xb2\xf3\x84\x6e\-    \\x64\x8b\x1e\xaf\x19\xbd\xf0\xca\xa0\x23\x69\xb9\x65\x5a\xbb\x50\-    \\x40\x68\x5a\x32\x3c\x2a\xb4\xb3\x31\x9e\xe9\xd5\xc0\x21\xb8\xf7\-    \\x9b\x54\x0b\x19\x87\x5f\xa0\x99\x95\xf7\x99\x7e\x62\x3d\x7d\xa8\-    \\xf8\x37\x88\x9a\x97\xe3\x2d\x77\x11\xed\x93\x5f\x16\x68\x12\x81\-    \\x0e\x35\x88\x29\xc7\xe6\x1f\xd6\x96\xde\xdf\xa1\x78\x58\xba\x99\-    \\x57\xf5\x84\xa5\x1b\x22\x72\x63\x9b\x83\xc3\xff\x1a\xc2\x46\x96\-    \\xcd\xb3\x0a\xeb\x53\x2e\x30\x54\x8f\xd9\x48\xe4\x6d\xbc\x31\x28\-    \\x58\xeb\xf2\xef\x34\xc6\xff\xea\xfe\x28\xed\x61\xee\x7c\x3c\x73\-    \\x5d\x4a\x14\xd9\xe8\x64\xb7\xe3\x42\x10\x5d\x14\x20\x3e\x13\xe0\-    \\x45\xee\xe2\xb6\xa3\xaa\xab\xea\xdb\x6c\x4f\x15\xfa\xcb\x4f\xd0\-    \\xc7\x42\xf4\x42\xef\x6a\xbb\xb5\x65\x4f\x3b\x1d\x41\xcd\x21\x05\-    \\xd8\x1e\x79\x9e\x86\x85\x4d\xc7\xe4\x4b\x47\x6a\x3d\x81\x62\x50\-    \\xcf\x62\xa1\xf2\x5b\x8d\x26\x46\xfc\x88\x83\xa0\xc1\xc7\xb6\xa3\-    \\x7f\x15\x24\xc3\x69\xcb\x74\x92\x47\x84\x8a\x0b\x56\x92\xb2\x85\-    \\x09\x5b\xbf\x00\xad\x19\x48\x9d\x14\x62\xb1\x74\x23\x82\x0e\x00\-    \\x58\x42\x8d\x2a\x0c\x55\xf5\xea\x1d\xad\xf4\x3e\x23\x3f\x70\x61\-    \\x33\x72\xf0\x92\x8d\x93\x7e\x41\xd6\x5f\xec\xf1\x6c\x22\x3b\xdb\-    \\x7c\xde\x37\x59\xcb\xee\x74\x60\x40\x85\xf2\xa7\xce\x77\x32\x6e\-    \\xa6\x07\x80\x84\x19\xf8\x50\x9e\xe8\xef\xd8\x55\x61\xd9\x97\x35\-    \\xa9\x69\xa7\xaa\xc5\x0c\x06\xc2\x5a\x04\xab\xfc\x80\x0b\xca\xdc\-    \\x9e\x44\x7a\x2e\xc3\x45\x34\x84\xfd\xd5\x67\x05\x0e\x1e\x9e\xc9\-    \\xdb\x73\xdb\xd3\x10\x55\x88\xcd\x67\x5f\xda\x79\xe3\x67\x43\x40\-    \\xc5\xc4\x34\x65\x71\x3e\x38\xd8\x3d\x28\xf8\x9e\xf1\x6d\xff\x20\-    \\x15\x3e\x21\xe7\x8f\xb0\x3d\x4a\xe6\xe3\x9f\x2b\xdb\x83\xad\xf7\-    \\xe9\x3d\x5a\x68\x94\x81\x40\xf7\xf6\x4c\x26\x1c\x94\x69\x29\x34\-    \\x41\x15\x20\xf7\x76\x02\xd4\xf7\xbc\xf4\x6b\x2e\xd4\xa2\x00\x68\-    \\xd4\x08\x24\x71\x33\x20\xf4\x6a\x43\xb7\xd4\xb7\x50\x00\x61\xaf\-    \\x1e\x39\xf6\x2e\x97\x24\x45\x46\x14\x21\x4f\x74\xbf\x8b\x88\x40\-    \\x4d\x95\xfc\x1d\x96\xb5\x91\xaf\x70\xf4\xdd\xd3\x66\xa0\x2f\x45\-    \\xbf\xbc\x09\xec\x03\xbd\x97\x85\x7f\xac\x6d\xd0\x31\xcb\x85\x04\-    \\x96\xeb\x27\xb3\x55\xfd\x39\x41\xda\x25\x47\xe6\xab\xca\x0a\x9a\-    \\x28\x50\x78\x25\x53\x04\x29\xf4\x0a\x2c\x86\xda\xe9\xb6\x6d\xfb\-    \\x68\xdc\x14\x62\xd7\x48\x69\x00\x68\x0e\xc0\xa4\x27\xa1\x8d\xee\-    \\x4f\x3f\xfe\xa2\xe8\x87\xad\x8c\xb5\x8c\xe0\x06\x7a\xf4\xd6\xb6\-    \\xaa\xce\x1e\x7c\xd3\x37\x5f\xec\xce\x78\xa3\x99\x40\x6b\x2a\x42\-    \\x20\xfe\x9e\x35\xd9\xf3\x85\xb9\xee\x39\xd7\xab\x3b\x12\x4e\x8b\-    \\x1d\xc9\xfa\xf7\x4b\x6d\x18\x56\x26\xa3\x66\x31\xea\xe3\x97\xb2\-    \\x3a\x6e\xfa\x74\xdd\x5b\x43\x32\x68\x41\xe7\xf7\xca\x78\x20\xfb\-    \\xfb\x0a\xf5\x4e\xd8\xfe\xb3\x97\x45\x40\x56\xac\xba\x48\x95\x27\-    \\x55\x53\x3a\x3a\x20\x83\x8d\x87\xfe\x6b\xa9\xb7\xd0\x96\x95\x4b\-    \\x55\xa8\x67\xbc\xa1\x15\x9a\x58\xcc\xa9\x29\x63\x99\xe1\xdb\x33\-    \\xa6\x2a\x4a\x56\x3f\x31\x25\xf9\x5e\xf4\x7e\x1c\x90\x29\x31\x7c\-    \\xfd\xf8\xe8\x02\x04\x27\x2f\x70\x80\xbb\x15\x5c\x05\x28\x2c\xe3\-    \\x95\xc1\x15\x48\xe4\xc6\x6d\x22\x48\xc1\x13\x3f\xc7\x0f\x86\xdc\-    \\x07\xf9\xc9\xee\x41\x04\x1f\x0f\x40\x47\x79\xa4\x5d\x88\x6e\x17\-    \\x32\x5f\x51\xeb\xd5\x9b\xc0\xd1\xf2\xbc\xc1\x8f\x41\x11\x35\x64\-    \\x25\x7b\x78\x34\x60\x2a\x9c\x60\xdf\xf8\xe8\xa3\x1f\x63\x6c\x1b\-    \\x0e\x12\xb4\xc2\x02\xe1\x32\x9e\xaf\x66\x4f\xd1\xca\xd1\x81\x15\-    \\x6b\x23\x95\xe0\x33\x3e\x92\xe1\x3b\x24\x0b\x62\xee\xbe\xb9\x22\-    \\x85\xb2\xa2\x0e\xe6\xba\x0d\x99\xde\x72\x0c\x8c\x2d\xa2\xf7\x28\-    \\xd0\x12\x78\x45\x95\xb7\x94\xfd\x64\x7d\x08\x62\xe7\xcc\xf5\xf0\-    \\x54\x49\xa3\x6f\x87\x7d\x48\xfa\xc3\x9d\xfd\x27\xf3\x3e\x8d\x1e\-    \\x0a\x47\x63\x41\x99\x2e\xff\x74\x3a\x6f\x6e\xab\xf4\xf8\xfd\x37\-    \\xa8\x12\xdc\x60\xa1\xeb\xdd\xf8\x99\x1b\xe1\x4c\xdb\x6e\x6b\x0d\-    \\xc6\x7b\x55\x10\x6d\x67\x2c\x37\x27\x65\xd4\x3b\xdc\xd0\xe8\x04\-    \\xf1\x29\x0d\xc7\xcc\x00\xff\xa3\xb5\x39\x0f\x92\x69\x0f\xed\x0b\-    \\x66\x7b\x9f\xfb\xce\xdb\x7d\x9c\xa0\x91\xcf\x0b\xd9\x15\x5e\xa3\-    \\xbb\x13\x2f\x88\x51\x5b\xad\x24\x7b\x94\x79\xbf\x76\x3b\xd6\xeb\-    \\x37\x39\x2e\xb3\xcc\x11\x59\x79\x80\x26\xe2\x97\xf4\x2e\x31\x2d\-    \\x68\x42\xad\xa7\xc6\x6a\x2b\x3b\x12\x75\x4c\xcc\x78\x2e\xf1\x1c\-    \\x6a\x12\x42\x37\xb7\x92\x51\xe7\x06\xa1\xbb\xe6\x4b\xfb\x63\x50\-    \\x1a\x6b\x10\x18\x11\xca\xed\xfa\x3d\x25\xbd\xd8\xe2\xe1\xc3\xc9\-    \\x44\x42\x16\x59\x0a\x12\x13\x86\xd9\x0c\xec\x6e\xd5\xab\xea\x2a\-    \\x64\xaf\x67\x4e\xda\x86\xa8\x5f\xbe\xbf\xe9\x88\x64\xe4\xc3\xfe\-    \\x9d\xbc\x80\x57\xf0\xf7\xc0\x86\x60\x78\x7b\xf8\x60\x03\x60\x4d\-    \\xd1\xfd\x83\x46\xf6\x38\x1f\xb0\x77\x45\xae\x04\xd7\x36\xfc\xcc\-    \\x83\x42\x6b\x33\xf0\x1e\xab\x71\xb0\x80\x41\x87\x3c\x00\x5e\x5f\-    \\x77\xa0\x57\xbe\xbd\xe8\xae\x24\x55\x46\x42\x99\xbf\x58\x2e\x61\-    \\x4e\x58\xf4\x8f\xf2\xdd\xfd\xa2\xf4\x74\xef\x38\x87\x89\xbd\xc2\-    \\x53\x66\xf9\xc3\xc8\xb3\x8e\x74\xb4\x75\xf2\x55\x46\xfc\xd9\xb9\-    \\x7a\xeb\x26\x61\x8b\x1d\xdf\x84\x84\x6a\x0e\x79\x91\x5f\x95\xe2\-    \\x46\x6e\x59\x8e\x20\xb4\x57\x70\x8c\xd5\x55\x91\xc9\x02\xde\x4c\-    \\xb9\x0b\xac\xe1\xbb\x82\x05\xd0\x11\xa8\x62\x48\x75\x74\xa9\x9e\-    \\xb7\x7f\x19\xb6\xe0\xa9\xdc\x09\x66\x2d\x09\xa1\xc4\x32\x46\x33\-    \\xe8\x5a\x1f\x02\x09\xf0\xbe\x8c\x4a\x99\xa0\x25\x1d\x6e\xfe\x10\-    \\x1a\xb9\x3d\x1d\x0b\xa5\xa4\xdf\xa1\x86\xf2\x0f\x28\x68\xf1\x69\-    \\xdc\xb7\xda\x83\x57\x39\x06\xfe\xa1\xe2\xce\x9b\x4f\xcd\x7f\x52\-    \\x50\x11\x5e\x01\xa7\x06\x83\xfa\xa0\x02\xb5\xc4\x0d\xe6\xd0\x27\-    \\x9a\xf8\x8c\x27\x77\x3f\x86\x41\xc3\x60\x4c\x06\x61\xa8\x06\xb5\-    \\xf0\x17\x7a\x28\xc0\xf5\x86\xe0\x00\x60\x58\xaa\x30\xdc\x7d\x62\-    \\x11\xe6\x9e\xd7\x23\x38\xea\x63\x53\xc2\xdd\x94\xc2\xc2\x16\x34\-    \\xbb\xcb\xee\x56\x90\xbc\xb6\xde\xeb\xfc\x7d\xa1\xce\x59\x1d\x76\-    \\x6f\x05\xe4\x09\x4b\x7c\x01\x88\x39\x72\x0a\x3d\x7c\x92\x7c\x24\-    \\x86\xe3\x72\x5f\x72\x4d\x9d\xb9\x1a\xc1\x5b\xb4\xd3\x9e\xb8\xfc\-    \\xed\x54\x55\x78\x08\xfc\xa5\xb5\xd8\x3d\x7c\xd3\x4d\xad\x0f\xc4\-    \\x1e\x50\xef\x5e\xb1\x61\xe6\xf8\xa2\x85\x14\xd9\x6c\x51\x13\x3c\-    \\x6f\xd5\xc7\xe7\x56\xe1\x4e\xc4\x36\x2a\xbf\xce\xdd\xc6\xc8\x37\-    \\xd7\x9a\x32\x34\x92\x63\x82\x12\x67\x0e\xfa\x8e\x40\x60\x00\xe0\-    \\x3a\x39\xce\x37\xd3\xfa\xf5\xcf\xab\xc2\x77\x37\x5a\xc5\x2d\x1b\-    \\x5c\xb0\x67\x9e\x4f\xa3\x37\x42\xd3\x82\x27\x40\x99\xbc\x9b\xbe\-    \\xd5\x11\x8e\x9d\xbf\x0f\x73\x15\xd6\x2d\x1c\x7e\xc7\x00\xc4\x7b\-    \\xb7\x8c\x1b\x6b\x21\xa1\x90\x45\xb2\x6e\xb1\xbe\x6a\x36\x6e\xb4\-    \\x57\x48\xab\x2f\xbc\x94\x6e\x79\xc6\xa3\x76\xd2\x65\x49\xc2\xc8\-    \\x53\x0f\xf8\xee\x46\x8d\xde\x7d\xd5\x73\x0a\x1d\x4c\xd0\x4d\xc6\-    \\x29\x39\xbb\xdb\xa9\xba\x46\x50\xac\x95\x26\xe8\xbe\x5e\xe3\x04\-    \\xa1\xfa\xd5\xf0\x6a\x2d\x51\x9a\x63\xef\x8c\xe2\x9a\x86\xee\x22\-    \\xc0\x89\xc2\xb8\x43\x24\x2e\xf6\xa5\x1e\x03\xaa\x9c\xf2\xd0\xa4\-    \\x83\xc0\x61\xba\x9b\xe9\x6a\x4d\x8f\xe5\x15\x50\xba\x64\x5b\xd6\-    \\x28\x26\xa2\xf9\xa7\x3a\x3a\xe1\x4b\xa9\x95\x86\xef\x55\x62\xe9\-    \\xc7\x2f\xef\xd3\xf7\x52\xf7\xda\x3f\x04\x6f\x69\x77\xfa\x0a\x59\-    \\x80\xe4\xa9\x15\x87\xb0\x86\x01\x9b\x09\xe6\xad\x3b\x3e\xe5\x93\-    \\xe9\x90\xfd\x5a\x9e\x34\xd7\x97\x2c\xf0\xb7\xd9\x02\x2b\x8b\x51\-    \\x96\xd5\xac\x3a\x01\x7d\xa6\x7d\xd1\xcf\x3e\xd6\x7c\x7d\x2d\x28\-    \\x1f\x9f\x25\xcf\xad\xf2\xb8\x9b\x5a\xd6\xb4\x72\x5a\x88\xf5\x4c\-    \\xe0\x29\xac\x71\xe0\x19\xa5\xe6\x47\xb0\xac\xfd\xed\x93\xfa\x9b\-    \\xe8\xd3\xc4\x8d\x28\x3b\x57\xcc\xf8\xd5\x66\x29\x79\x13\x2e\x28\-    \\x78\x5f\x01\x91\xed\x75\x60\x55\xf7\x96\x0e\x44\xe3\xd3\x5e\x8c\-    \\x15\x05\x6d\xd4\x88\xf4\x6d\xba\x03\xa1\x61\x25\x05\x64\xf0\xbd\-    \\xc3\xeb\x9e\x15\x3c\x90\x57\xa2\x97\x27\x1a\xec\xa9\x3a\x07\x2a\-    \\x1b\x3f\x6d\x9b\x1e\x63\x21\xf5\xf5\x9c\x66\xfb\x26\xdc\xf3\x19\-    \\x75\x33\xd9\x28\xb1\x55\xfd\xf5\x03\x56\x34\x82\x8a\xba\x3c\xbb\-    \\x28\x51\x77\x11\xc2\x0a\xd9\xf8\xab\xcc\x51\x67\xcc\xad\x92\x5f\-    \\x4d\xe8\x17\x51\x38\x30\xdc\x8e\x37\x9d\x58\x62\x93\x20\xf9\x91\-    \\xea\x7a\x90\xc2\xfb\x3e\x7b\xce\x51\x21\xce\x64\x77\x4f\xbe\x32\-    \\xa8\xb6\xe3\x7e\xc3\x29\x3d\x46\x48\xde\x53\x69\x64\x13\xe6\x80\-    \\xa2\xae\x08\x10\xdd\x6d\xb2\x24\x69\x85\x2d\xfd\x09\x07\x21\x66\-    \\xb3\x9a\x46\x0a\x64\x45\xc0\xdd\x58\x6c\xde\xcf\x1c\x20\xc8\xae\-    \\x5b\xbe\xf7\xdd\x1b\x58\x8d\x40\xcc\xd2\x01\x7f\x6b\xb4\xe3\xbb\-    \\xdd\xa2\x6a\x7e\x3a\x59\xff\x45\x3e\x35\x0a\x44\xbc\xb4\xcd\xd5\-    \\x72\xea\xce\xa8\xfa\x64\x84\xbb\x8d\x66\x12\xae\xbf\x3c\x6f\x47\-    \\xd2\x9b\xe4\x63\x54\x2f\x5d\x9e\xae\xc2\x77\x1b\xf6\x4e\x63\x70\-    \\x74\x0e\x0d\x8d\xe7\x5b\x13\x57\xf8\x72\x16\x71\xaf\x53\x7d\x5d\-    \\x40\x40\xcb\x08\x4e\xb4\xe2\xcc\x34\xd2\x46\x6a\x01\x15\xaf\x84\-    \\xe1\xb0\x04\x28\x95\x98\x3a\x1d\x06\xb8\x9f\xb4\xce\x6e\xa0\x48\-    \\x6f\x3f\x3b\x82\x35\x20\xab\x82\x01\x1a\x1d\x4b\x27\x72\x27\xf8\-    \\x61\x15\x60\xb1\xe7\x93\x3f\xdc\xbb\x3a\x79\x2b\x34\x45\x25\xbd\-    \\xa0\x88\x39\xe1\x51\xce\x79\x4b\x2f\x32\xc9\xb7\xa0\x1f\xba\xc9\-    \\xe0\x1c\xc8\x7e\xbc\xc7\xd1\xf6\xcf\x01\x11\xc3\xa1\xe8\xaa\xc7\-    \\x1a\x90\x87\x49\xd4\x4f\xbd\x9a\xd0\xda\xde\xcb\xd5\x0a\xda\x38\-    \\x03\x39\xc3\x2a\xc6\x91\x36\x67\x8d\xf9\x31\x7c\xe0\xb1\x2b\x4f\-    \\xf7\x9e\x59\xb7\x43\xf5\xbb\x3a\xf2\xd5\x19\xff\x27\xd9\x45\x9c\-    \\xbf\x97\x22\x2c\x15\xe6\xfc\x2a\x0f\x91\xfc\x71\x9b\x94\x15\x25\-    \\xfa\xe5\x93\x61\xce\xb6\x9c\xeb\xc2\xa8\x64\x59\x12\xba\xa8\xd1\-    \\xb6\xc1\x07\x5e\xe3\x05\x6a\x0c\x10\xd2\x50\x65\xcb\x03\xa4\x42\-    \\xe0\xec\x6e\x0e\x16\x98\xdb\x3b\x4c\x98\xa0\xbe\x32\x78\xe9\x64\-    \\x9f\x1f\x95\x32\xe0\xd3\x92\xdf\xd3\xa0\x34\x2b\x89\x71\xf2\x1e\-    \\x1b\x0a\x74\x41\x4b\xa3\x34\x8c\xc5\xbe\x71\x20\xc3\x76\x32\xd8\-    \\xdf\x35\x9f\x8d\x9b\x99\x2f\x2e\xe6\x0b\x6f\x47\x0f\xe3\xf1\x1d\-    \\xe5\x4c\xda\x54\x1e\xda\xd8\x91\xce\x62\x79\xcf\xcd\x3e\x7e\x6f\-    \\x16\x18\xb1\x66\xfd\x2c\x1d\x05\x84\x8f\xd2\xc5\xf6\xfb\x22\x99\-    \\xf5\x23\xf3\x57\xa6\x32\x76\x23\x93\xa8\x35\x31\x56\xcc\xcd\x02\-    \\xac\xf0\x81\x62\x5a\x75\xeb\xb5\x6e\x16\x36\x97\x88\xd2\x73\xcc\-    \\xde\x96\x62\x92\x81\xb9\x49\xd0\x4c\x50\x90\x1b\x71\xc6\x56\x14\-    \\xe6\xc6\xc7\xbd\x32\x7a\x14\x0a\x45\xe1\xd0\x06\xc3\xf2\x7b\x9a\-    \\xc9\xaa\x53\xfd\x62\xa8\x0f\x00\xbb\x25\xbf\xe2\x35\xbd\xd2\xf6\-    \\x71\x12\x69\x05\xb2\x04\x02\x22\xb6\xcb\xcf\x7c\xcd\x76\x9c\x2b\-    \\x53\x11\x3e\xc0\x16\x40\xe3\xd3\x38\xab\xbd\x60\x25\x47\xad\xf0\-    \\xba\x38\x20\x9c\xf7\x46\xce\x76\x77\xaf\xa1\xc5\x20\x75\x60\x60\-    \\x85\xcb\xfe\x4e\x8a\xe8\x8d\xd8\x7a\xaa\xf9\xb0\x4c\xf9\xaa\x7e\-    \\x19\x48\xc2\x5c\x02\xfb\x8a\x8c\x01\xc3\x6a\xe4\xd6\xeb\xe1\xf9\-    \\x90\xd4\xf8\x69\xa6\x5c\xde\xa0\x3f\x09\x25\x2d\xc2\x08\xe6\x9f\-    \\xb7\x4e\x61\x32\xce\x77\xe2\x5b\x57\x8f\xdf\xe3\x3a\xc3\x72\xe6\-    \"#
Crypto/Cipher/Blowfish/Primitive.hs view
@@ -1,9 +1,3 @@--- |--- Module      : Crypto.Cipher.Blowfish.Primitive--- License     : BSD-style--- Stability   : experimental--- Portability : Good- -- Rewritten by Vincent Hanquez (c) 2015 --              Lars Petersen (c) 2018 --@@ -12,247 +6,145 @@ --      based on: BlowfishAux.hs (C) 2002 HardCore SoftWare, Doug Hoyte --           (as found in Crypto-4.2.4) {-# LANGUAGE BangPatterns #-}-module Crypto.Cipher.Blowfish.Primitive-    ( Context-    , initBlowfish-    , encrypt-    , decrypt-    , KeySchedule-    , createKeySchedule-    , freezeKeySchedule-    , expandKey-    , expandKeyWithSalt-    , cipherBlockMutable-    ) where -import           Control.Monad              (when)-import           Data.Bits-import           Data.Memory.Endian-import           Data.Word+-- |+-- Module      : Crypto.Cipher.Blowfish.Primitive+-- License     : BSD-style+-- Stability   : experimental+-- Portability : Good+--+-- The cipher itself is in C, as is the key setup bcrypt wraps around it:+-- what the schedule costs is the whole of what bcrypt is for, and in Haskell+-- it cost about twice what the usual implementations do.+module Crypto.Cipher.Blowfish.Primitive (+    Context,+    initBlowfish,+    encrypt,+    decrypt,+    bcryptHash,+    bcryptPbkdfHash,+) where -import           Crypto.Cipher.Blowfish.Box-import           Crypto.Error-import           Crypto.Internal.ByteArray  (ByteArray, ByteArrayAccess)-import qualified Crypto.Internal.ByteArray  as B-import           Crypto.Internal.Compat-import           Crypto.Internal.Imports-import           Crypto.Internal.WordArray+import Crypto.Error+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, ScrubbedBytes)+import qualified Crypto.Internal.ByteArray as B+import Crypto.Internal.Compat+import Crypto.Internal.Imports+import Foreign.C.Types (CInt (..))+import Foreign.Ptr (Ptr) -newtype Context = Context Array32+-- | The key schedule: the P array and the four S boxes, as the C keeps them.+newtype Context = Context ScrubbedBytes  instance NFData Context where     rnf a = a `seq` () +-- | How many bytes of schedule the C wants: eighteen words and four boxes of+-- two hundred and fifty-six.+contextSize :: Int+contextSize = (18 + 4 * 256) * 4+ -- | Initialize a new Blowfish context from a key. -- -- key needs to be between 0 and 448 bits. initBlowfish :: ByteArrayAccess key => key -> CryptoFailable Context initBlowfish key     | B.length key > (448 `div` 8) = CryptoFailed CryptoError_KeySizeInvalid-    | otherwise                    = CryptoPassed $ unsafeDoIO $ do-        ks <- createKeySchedule-        expandKey ks key-        freezeKeySchedule ks---- | Get an immutable Blowfish context by freezing a mutable key schedule.-freezeKeySchedule :: KeySchedule -> IO Context-freezeKeySchedule (KeySchedule ma) = Context `fmap` mutableArray32Freeze ma--expandKey :: (ByteArrayAccess key) => KeySchedule -> key -> IO ()-expandKey ks@(KeySchedule ma) key = do-    when (B.length key > 0) $ iterKeyStream key 0 0 $ \i l r a0 a1 cont-> do-        mutableArrayWriteXor32 ma i l-        mutableArrayWriteXor32 ma (i + 1) r-        when (i + 2 < 18) (cont a0 a1)-    loop 0 0 0-    where-        loop i l r = do-            n <- cipherBlockMutable ks (fromIntegral l `shiftL` 32 .|. fromIntegral r)-            let nl = fromIntegral (n `shiftR` 32)-                nr = fromIntegral (n .&. 0xffffffff)-            mutableArrayWrite32 ma i nl-            mutableArrayWrite32 ma (i + 1) nr-            when (i < 18 + 1024) (loop (i + 2) nl nr)--expandKeyWithSalt :: (ByteArrayAccess key, ByteArrayAccess salt)-    => KeySchedule-    -> key-    -> salt-    -> IO ()-expandKeyWithSalt ks key salt-    | B.length salt == 16 = expandKeyWithSalt128 ks key (fromBE $ B.toW64BE salt 0) (fromBE $ B.toW64BE salt 8)-    | otherwise           = expandKeyWithSaltAny ks key salt--expandKeyWithSaltAny :: (ByteArrayAccess key, ByteArrayAccess salt)-    => KeySchedule         -- ^ The key schedule-    -> key                 -- ^ The key-    -> salt                -- ^ The salt-    -> IO ()-expandKeyWithSaltAny ks@(KeySchedule ma) key salt = do-    when (B.length key > 0) $ iterKeyStream key 0 0 $ \i l r a0 a1 cont-> do-        mutableArrayWriteXor32 ma i l-        mutableArrayWriteXor32 ma (i + 1) r-        when (i + 2 < 18) (cont a0 a1)-    -- Go through the entire key schedule overwriting the P-Array and S-Boxes-    when (B.length salt > 0) $ iterKeyStream salt 0 0 $ \i l r a0 a1 cont-> do-        let l' = xor l a0-        let r' = xor r a1-        n <- cipherBlockMutable ks (fromIntegral l' `shiftL` 32 .|. fromIntegral r')-        let nl = fromIntegral (n `shiftR` 32)-            nr = fromIntegral (n .&. 0xffffffff)-        mutableArrayWrite32 ma i nl-        mutableArrayWrite32 ma (i + 1) nr-        when (i + 2 < 18 + 1024) (cont nl nr)--expandKeyWithSalt128 :: ByteArrayAccess ba-    => KeySchedule         -- ^ The key schedule-    -> ba                  -- ^ The key-    -> Word64              -- ^ First word of the salt-    -> Word64              -- ^ Second word of the salt-    -> IO ()-expandKeyWithSalt128 ks@(KeySchedule ma) key salt1 salt2 = do-    when (B.length key > 0) $ iterKeyStream key 0 0 $ \i l r a0 a1 cont-> do-        mutableArrayWriteXor32 ma i l-        mutableArrayWriteXor32 ma (i + 1) r-        when (i + 2 < 18) (cont a0 a1)-    -- Go through the entire key schedule overwriting the P-Array and S-Boxes-    loop 0 salt1 salt1 salt2-    where-        loop i input slt1 slt2-            | i == 1042   = return ()-            | otherwise = do-                n <- cipherBlockMutable ks input-                let nl = fromIntegral (n `shiftR` 32)-                    nr = fromIntegral (n .&. 0xffffffff)-                mutableArrayWrite32 ma i     nl-                mutableArrayWrite32 ma (i+1) nr-                loop (i+2) (n `xor` slt2) slt2 slt1+    | otherwise = CryptoPassed $+        unsafeDoIO $+            fmap Context $+                B.alloc contextSize $ \ctx ->+                    B.withByteArray key $ \k ->+                        c_blowfish_init ctx k (fromIntegral (B.length key))  -- | Encrypt blocks -- -- Input need to be a multiple of 8 bytes encrypt :: ByteArray ba => Context -> ba -> ba-encrypt ctx ba-    | B.length ba == 0         = B.empty-    | B.length ba `mod` 8 /= 0 = error "invalid data length"-    | otherwise                = B.mapAsWord64 (cipherBlock ctx False) ba+encrypt = through c_blowfish_encrypt  -- | Decrypt blocks -- -- Input need to be a multiple of 8 bytes decrypt :: ByteArray ba => Context -> ba -> ba-decrypt ctx ba-    | B.length ba == 0         = B.empty-    | B.length ba `mod` 8 /= 0 = error "invalid data length"-    | otherwise                = B.mapAsWord64 (cipherBlock ctx True) ba+decrypt = through c_blowfish_decrypt --- | Encrypt or decrypt a single block of 64 bits.------ The inverse argument decides whether to encrypt or decrypt.-cipherBlock :: Context -> Bool -> Word64 -> Word64-cipherBlock (Context ar) inverse input = doRound input 0-    where-    -- | Transform the input over 16 rounds-    doRound :: Word64 -> Int -> Word64-    doRound !i roundIndex-        | roundIndex == 16 =-            let final = (fromIntegral (p 16) `shiftL` 32) .|. fromIntegral (p 17)-             in rotateL (i `xor` final) 32-        | otherwise     =-            let newr = fromIntegral (i `shiftR` 32) `xor` p roundIndex-                newi = ((i `shiftL` 32) `xor` f newr) .|. fromIntegral newr-             in doRound newi (roundIndex+1)+through+    :: ByteArray ba+    => (Ptr Word8 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO ())+    -> Context+    -> ba+    -> ba+through f (Context ctx) input+    | len `mod` 8 /= 0 =+        error "Crypto.Cipher.Blowfish: input length must be a multiple of 8"+    | otherwise = unsafeDoIO $+        B.alloc len $ \out ->+            B.withByteArray ctx $ \c ->+                B.withByteArray input $ \i -> f c out i (fromIntegral len)+  where+    len = B.length input -    -- | The Blowfish Feistel function F-    f   :: Word32 -> Word64-    f t = let a = s0 (0xff .&. (t `shiftR` 24))-              b = s1 (0xff .&. (t `shiftR` 16))-              c = s2 (0xff .&. (t `shiftR` 8))-              d = s3 (0xff .&.  t)-           in fromIntegral (((a + b) `xor` c) + d) `shiftL` 32+-- | What bcrypt does with Blowfish: the key setup that costs what the cost+-- says, and then the sixty-four encryptions.  The answer is 24 bytes, of+-- which bcrypt keeps 23.+--+-- The salt has to be 16 bytes and the key 1 to 73, which is a password of at+-- most 72 with the zero byte the original implementation appends.  'Nothing'+-- means it was given something else.+bcryptHash+    :: (ByteArrayAccess salt, ByteArrayAccess key, ByteArray output)+    => Int+    -- ^ the cost, between 4 and 31+    -> salt+    -> key+    -> Maybe output+bcryptHash cost salt key+    | cost < 4 || cost > 31 = Nothing+    | B.length salt /= 16 = Nothing+    | B.length key < 1 || B.length key > 73 = Nothing+    | otherwise = unsafeDoIO $ do+        (r, out) <- B.allocRet 24 $ \o ->+            B.withByteArray salt $ \s ->+                B.withByteArray key $ \k ->+                    c_bcrypt o (fromIntegral cost) s k (fromIntegral (B.length key))+        return $ if r == 0 then Just out else Nothing -    -- | S-Box arrays, each containing 256 32-bit words-    --   The first 18 words contain the P-Array of subkeys-    s0, s1, s2, s3 :: Word32 -> Word32-    s0 i            = arrayRead32 ar (fromIntegral i + 18)-    s1 i            = arrayRead32 ar (fromIntegral i + 274)-    s2 i            = arrayRead32 ar (fromIntegral i + 530)-    s3 i            = arrayRead32 ar (fromIntegral i + 786)-    p              :: Int -> Word32-    p i | inverse   = arrayRead32 ar (17 - i)-        | otherwise = arrayRead32 ar i+foreign import ccall unsafe "crypton_blowfish_init"+    c_blowfish_init :: Ptr Word8 -> Ptr Word8 -> Word32 -> IO () --- | Blowfish encrypt a Word using the current state of the key schedule-cipherBlockMutable :: KeySchedule -> Word64 -> IO Word64-cipherBlockMutable (KeySchedule ma) input = doRound input 0-    where-    -- | Transform the input over 16 rounds-    doRound !i roundIndex-        | roundIndex == 16 = do-            pVal1 <- mutableArrayRead32 ma 16-            pVal2 <- mutableArrayRead32 ma 17-            let final = (fromIntegral pVal1 `shiftL` 32) .|. fromIntegral pVal2-            return $ rotateL (i `xor` final) 32-        | otherwise     = do-            pVal <- mutableArrayRead32 ma roundIndex-            let newr = fromIntegral (i `shiftR` 32) `xor` pVal-            newr' <- f newr-            let newi = ((i `shiftL` 32) `xor` newr') .|. fromIntegral newr-            doRound newi (roundIndex+1)+foreign import ccall unsafe "crypton_blowfish_encrypt"+    c_blowfish_encrypt :: Ptr Word8 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO () -    -- | The Blowfish Feistel function F-    f   :: Word32 -> IO Word64-    f t = do-        a <- s0 (0xff .&. (t `shiftR` 24))-        b <- s1 (0xff .&. (t `shiftR` 16))-        c <- s2 (0xff .&. (t `shiftR` 8))-        d <- s3 (0xff .&.  t)-        return (fromIntegral (((a + b) `xor` c) + d) `shiftL` 32)+foreign import ccall unsafe "crypton_blowfish_decrypt"+    c_blowfish_decrypt :: Ptr Word8 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO () -    -- | S-Box arrays, each containing 256 32-bit words-    --   The first 18 words contain the P-Array of subkeys-    s0, s1, s2, s3 :: Word32 -> IO Word32-    s0 i = mutableArrayRead32 ma (fromIntegral i + 18)-    s1 i = mutableArrayRead32 ma (fromIntegral i + 274)-    s2 i = mutableArrayRead32 ma (fromIntegral i + 530)-    s3 i = mutableArrayRead32 ma (fromIntegral i + 786)+-- the work is what the cost says, so this one may take a while: it is a safe+-- call, which lets the other capabilities carry on while it does+foreign import ccall safe "crypton_bcrypt"+    c_bcrypt :: Ptr Word8 -> Word32 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO CInt -iterKeyStream :: (ByteArrayAccess x)-    => x-    -> Word32-    -> Word32-    -> (Int -> Word32 -> Word32 -> Word32 -> Word32 -> (Word32 -> Word32 -> IO ()) -> IO ())+-- | What bcrypt_pbkdf does with Blowfish: the same key setup sixty-four times+-- over, and then the four blocks of its own magic.  Writes 32 bytes where it+-- is pointed, which is what the caller of this one wants.+bcryptPbkdfHash+    :: (ByteArrayAccess pass, ByteArrayAccess salt)+    => pass+    -> salt+    -> Ptr Word8     -> IO ()-iterKeyStream x a0 a1 g = f 0 0 a0 a1-    where-        len          = B.length x-        -- Avoiding the modulo operation when interating over the ring-        -- buffer is assumed to be more efficient here. All other-        -- implementations do this, too. The branch prediction shall prefer-        -- the branch with the increment.-        n j          = if j + 1 >= len then 0 else j + 1-        f i j0 b0 b1 = g i l r b0 b1 (f (i + 2) j8)-            where-                j1 = n j0-                j2 = n j1-                j3 = n j2-                j4 = n j3-                j5 = n j4-                j6 = n j5-                j7 = n j6-                j8 = n j7-                x0 = fromIntegral (B.index x j0)-                x1 = fromIntegral (B.index x j1)-                x2 = fromIntegral (B.index x j2)-                x3 = fromIntegral (B.index x j3)-                x4 = fromIntegral (B.index x j4)-                x5 = fromIntegral (B.index x j5)-                x6 = fromIntegral (B.index x j6)-                x7 = fromIntegral (B.index x j7)-                l  = shiftL x0 24 .|. shiftL x1 16 .|. shiftL x2 8 .|. x3-                r  = shiftL x4 24 .|. shiftL x5 16 .|. shiftL x6 8 .|. x7-{-# INLINE iterKeyStream #-}--- Benchmarking shows that GHC considers this function too big to inline--- although forcing inlining causes an actual improvement.--- It is assumed that all function calls (especially the continuation)--- collapse into a tight loop after inlining.+bcryptPbkdfHash pass salt out =+    B.withByteArray pass $ \p ->+        B.withByteArray salt $ \s -> do+            _ <-+                c_bcrypt_pbkdf_hash+                    out+                    p+                    (fromIntegral (B.length pass))+                    s+                    (fromIntegral (B.length salt))+            return ()++foreign import ccall safe "crypton_bcrypt_pbkdf_hash"+    c_bcrypt_pbkdf_hash+        :: Ptr Word8 -> Ptr Word8 -> Word32 -> Ptr Word8 -> Word32 -> IO CInt
Crypto/Cipher/CAST5.hs view
@@ -4,15 +4,14 @@ -- Maintainer  : Olivier Chéron <olivier.cheron@gmail.com> -- Stability   : stable -- Portability : good----module Crypto.Cipher.CAST5-    ( CAST5-    ) where+module Crypto.Cipher.CAST5 (+    CAST5,+) where -import           Crypto.Error-import           Crypto.Cipher.Types-import           Crypto.Cipher.CAST5.Primitive-import           Crypto.Internal.ByteArray (ByteArrayAccess)+import Crypto.Cipher.CAST5.Primitive+import Crypto.Cipher.Types+import Crypto.Error+import Crypto.Internal.ByteArray (ByteArrayAccess) import qualified Crypto.Internal.ByteArray as B  -- | CAST5 block cipher (also known as CAST-128).  Key is between@@ -20,9 +19,9 @@ newtype CAST5 = CAST5 Key  instance Cipher CAST5 where-    cipherName    _ = "CAST5"+    cipherName _ = "CAST5"     cipherKeySize _ = KeySizeRange 5 16-    cipherInit      = initCAST5+    cipherInit = initCAST5  instance BlockCipher CAST5 where     blockSize _ = 8@@ -31,12 +30,12 @@  initCAST5 :: ByteArrayAccess key => key -> CryptoFailable CAST5 initCAST5 bs-    | len <   5 = CryptoFailed CryptoError_KeySizeInvalid-    | len <  16 = CryptoPassed (CAST5 $ buildKey short padded)+    | len < 5 = CryptoFailed CryptoError_KeySizeInvalid+    | len < 16 = CryptoPassed (CAST5 $ buildKey short padded)     | len == 16 = CryptoPassed (CAST5 $ buildKey False bs)     | otherwise = CryptoFailed CryptoError_KeySizeInvalid   where-    len   = B.length bs+    len = B.length bs     short = len <= 10      padded :: B.Bytes
Crypto/Cipher/CAST5/Primitive.hs view
@@ -1,21 +1,20 @@ {-# LANGUAGE MagicHash #-}  -----------------------------------------------------------------------------++-----------------------------------------------------------------------------+ -- | -- Module      :  Crypto.Cipher.CAST5.Primitive -- License     :  BSD-style -- -- Haskell implementation of the CAST-128 Encryption Algorithm------------------------------------------------------------------------------------module Crypto.Cipher.CAST5.Primitive-    ( encrypt-    , decrypt-    , Key()-    , buildKey-    ) where+module Crypto.Cipher.CAST5.Primitive (+    encrypt,+    decrypt,+    Key (),+    buildKey,+) where  import Control.Monad (void, (>=>)) @@ -23,23 +22,28 @@ import Data.Memory.Endian import Data.Word -import           Crypto.Internal.ByteArray (ByteArrayAccess)+import Crypto.Internal.ByteArray (ByteArrayAccess) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Internal.WordArray-+import Crypto.Internal.WordArray  -- Data Types -data P = P {-# UNPACK #-} !Word32 -- left word-           {-# UNPACK #-} !Word32 -- right word+data P+    = P+        {-# UNPACK #-} !Word32 -- left word+        {-# UNPACK #-} !Word32 -- right word -data Q = Q {-# UNPACK #-} !Word32 {-# UNPACK #-} !Word32-           {-# UNPACK #-} !Word32 {-# UNPACK #-} !Word32+data Q+    = Q+        {-# UNPACK #-} !Word32+        {-# UNPACK #-} !Word32+        {-# UNPACK #-} !Word32+        {-# UNPACK #-} !Word32  -- | All subkeys for 12 or 16 rounds-data Key = K12 {-# UNPACK #-} !Array32 -- [ km1, kr1, km2, kr2, ..., km12, kr12 ]-         | K16 {-# UNPACK #-} !Array32 -- [ km1, kr1, km2, kr2, ..., km16, kr16 ]-+data Key+    = K12 {-# UNPACK #-} !Array32 -- [ km1, kr1, km2, kr2, ..., km12, kr12 ]+    | K16 {-# UNPACK #-} !Array32 -- [ km1, kr1, km2, kr2, ..., km16, kr16 ]  -- Big-endian Transformations @@ -53,10 +57,9 @@ decomp32 x =     let a = fromIntegral (x `shiftR` 24)         b = fromIntegral (x `shiftR` 16)-        c = fromIntegral (x `shiftR`  8)+        c = fromIntegral (x `shiftR` 8)         d = fromIntegral x-    in (a, b, c, d)-+     in (a, b, c, d)  -- Encryption @@ -67,19 +70,18 @@ cast_enc :: Key -> P -> P cast_enc (K12 a) (P l0 r0) = P r12 r11   where-    r1  = type1 a 0  l0  r0-    r2  = type2 a 2  r0  r1-    r3  = type3 a 4  r1  r2-    r4  = type1 a 6  r2  r3-    r5  = type2 a 8  r3  r4-    r6  = type3 a 10 r4  r5-    r7  = type1 a 12 r5  r6-    r8  = type2 a 14 r6  r7-    r9  = type3 a 16 r7  r8-    r10 = type1 a 18 r8  r9-    r11 = type2 a 20 r9  r10+    r1 = type1 a 0 l0 r0+    r2 = type2 a 2 r0 r1+    r3 = type3 a 4 r1 r2+    r4 = type1 a 6 r2 r3+    r5 = type2 a 8 r3 r4+    r6 = type3 a 10 r4 r5+    r7 = type1 a 12 r5 r6+    r8 = type2 a 14 r6 r7+    r9 = type3 a 16 r7 r8+    r10 = type1 a 18 r8 r9+    r11 = type2 a 20 r9 r10     r12 = type3 a 22 r10 r11- cast_enc (K16 a) p = P r16 r15   where     P r12 r11 = cast_enc (K12 a) p@@ -99,18 +101,17 @@ cast_dec (K12 a) (P r12 r11) = P l0 r0   where     r10 = type3 a 22 r12 r11-    r9  = type2 a 20 r11 r10-    r8  = type1 a 18 r10 r9-    r7  = type3 a 16 r9  r8-    r6  = type2 a 14 r8  r7-    r5  = type1 a 12 r7  r6-    r4  = type3 a 10 r6  r5-    r3  = type2 a 8  r5  r4-    r2  = type1 a 6  r4  r3-    r1  = type3 a 4  r3  r2-    r0  = type2 a 2  r2  r1-    l0  = type1 a 0  r1  r0-+    r9 = type2 a 20 r11 r10+    r8 = type1 a 18 r10 r9+    r7 = type3 a 16 r9 r8+    r6 = type2 a 14 r8 r7+    r5 = type1 a 12 r7 r6+    r4 = type3 a 10 r6 r5+    r3 = type2 a 8 r5 r4+    r2 = type1 a 6 r4 r3+    r1 = type3 a 4 r3 r2+    r0 = type2 a 2 r2 r1+    l0 = type1 a 0 r1 r0 cast_dec (K16 a) (P r16 r15) = cast_dec (K12 a) (P r12 r11)   where     r14 = type1 a 30 r16 r15@@ -118,7 +119,6 @@     r12 = type2 a 26 r14 r13     r11 = type1 a 24 r13 r12 - -- Non-Identical Rounds  type1 :: Array32 -> Int -> Word32 -> Word32 -> Word32@@ -145,14 +145,17 @@         (ja, jb, jc, jd) = decomp32 j      in l `xor` (((sbox_s1 ja + sbox_s2 jb) `xor` sbox_s3 jc) - sbox_s4 jd) - -- Key Schedule  -- | Precompute "masking" and "rotation" subkeys-buildKey :: ByteArrayAccess key-         => Bool -- ^ @True@ for short keys that only need 12 rounds-         -> key  -- ^ Input key padded to 16 bytes-         -> Key  -- ^ Output data structure+buildKey+    :: ByteArrayAccess key+    => Bool+    -- ^ @True@ for short keys that only need 12 rounds+    -> key+    -- ^ Input key padded to 16 bytes+    -> Key+    -- ^ Output data structure buildKey isShort key =     let P x0123 x4567 = decomp64 (fromBE $ B.toW64BE key 0)         P x89AB xCDEF = decomp64 (fromBE $ B.toW64BE key 8)@@ -160,12 +163,10 @@  keySchedule :: Bool -> Q -> Key keySchedule isShort x-    | isShort   = K12 $ allocArray32AndFreeze 24 $ \ma ->+    | isShort = K12 $ allocArray32AndFreeze 24 $ \ma ->         void (steps123 ma 0 x >>= skip4 >>= steps123 ma 1)-     | otherwise = K16 $ allocArray32AndFreeze 32 $ \ma ->         void (steps123 ma 0 x >>= step4 ma 24 >>= steps123 ma 1 >>= step4 ma 25)-   where     sbox_s56785 a b c d e = sbox_s5 a `xor` sbox_s6 b `xor` sbox_s7 c `xor` sbox_s8 d `xor` sbox_s5 e     sbox_s56786 a b c d e = sbox_s5 a `xor` sbox_s6 b `xor` sbox_s7 c `xor` sbox_s8 d `xor` sbox_s6 e@@ -279,7 +280,9 @@ sbox_s1 :: Word8 -> Word32 sbox_s1 i = arrayRead32 t (fromIntegral i)   where-    t = array32FromAddrBE 256+    t =+        array32FromAddrBE+            256             "\x30\xfb\x40\xd4\x9f\xa0\xff\x0b\x6b\xec\xcd\x2f\x3f\x25\x8c\x7a\x1e\x21\x3f\x2f\x9c\x00\x4d\xd3\x60\x03\xe5\x40\xcf\x9f\xc9\x49\             \\xbf\xd4\xaf\x27\x88\xbb\xbd\xb5\xe2\x03\x40\x90\x98\xd0\x96\x75\x6e\x63\xa0\xe0\x15\xc3\x61\xd2\xc2\xe7\x66\x1d\x22\xd4\xff\x8e\             \\x28\x68\x3b\x6f\xc0\x7f\xd0\x59\xff\x23\x79\xc8\x77\x5f\x50\xe2\x43\xc3\x40\xd3\xdf\x2f\x86\x56\x88\x7c\xa4\x1a\xa2\xd2\xbd\x2d\@@ -316,7 +319,9 @@ sbox_s2 :: Word8 -> Word32 sbox_s2 i = arrayRead32 t (fromIntegral i)   where-    t = array32FromAddrBE 256+    t =+        array32FromAddrBE+            256             "\x1f\x20\x10\x94\xef\x0b\xa7\x5b\x69\xe3\xcf\x7e\x39\x3f\x43\x80\xfe\x61\xcf\x7a\xee\xc5\x20\x7a\x55\x88\x9c\x94\x72\xfc\x06\x51\             \\xad\xa7\xef\x79\x4e\x1d\x72\x35\xd5\x5a\x63\xce\xde\x04\x36\xba\x99\xc4\x30\xef\x5f\x0c\x07\x94\x18\xdc\xdb\x7d\xa1\xd6\xef\xf3\             \\xa0\xb5\x2f\x7b\x59\xe8\x36\x05\xee\x15\xb0\x94\xe9\xff\xd9\x09\xdc\x44\x00\x86\xef\x94\x44\x59\xba\x83\xcc\xb3\xe0\xc3\xcd\xfb\@@ -353,7 +358,9 @@ sbox_s3 :: Word8 -> Word32 sbox_s3 i = arrayRead32 t (fromIntegral i)   where-    t = array32FromAddrBE 256+    t =+        array32FromAddrBE+            256             "\x8d\xef\xc2\x40\x25\xfa\x5d\x9f\xeb\x90\x3d\xbf\xe8\x10\xc9\x07\x47\x60\x7f\xff\x36\x9f\xe4\x4b\x8c\x1f\xc6\x44\xae\xce\xca\x90\             \\xbe\xb1\xf9\xbf\xee\xfb\xca\xea\xe8\xcf\x19\x50\x51\xdf\x07\xae\x92\x0e\x88\x06\xf0\xad\x05\x48\xe1\x3c\x8d\x83\x92\x70\x10\xd5\             \\x11\x10\x7d\x9f\x07\x64\x7d\xb9\xb2\xe3\xe4\xd4\x3d\x4f\x28\x5e\xb9\xaf\xa8\x20\xfa\xde\x82\xe0\xa0\x67\x26\x8b\x82\x72\x79\x2e\@@ -390,7 +397,9 @@ sbox_s4 :: Word8 -> Word32 sbox_s4 i = arrayRead32 t (fromIntegral i)   where-    t = array32FromAddrBE 256+    t =+        array32FromAddrBE+            256             "\x9d\xb3\x04\x20\x1f\xb6\xe9\xde\xa7\xbe\x7b\xef\xd2\x73\xa2\x98\x4a\x4f\x7b\xdb\x64\xad\x8c\x57\x85\x51\x04\x43\xfa\x02\x0e\xd1\             \\x7e\x28\x7a\xff\xe6\x0f\xb6\x63\x09\x5f\x35\xa1\x79\xeb\xf1\x20\xfd\x05\x9d\x43\x64\x97\xb7\xb1\xf3\x64\x1f\x63\x24\x1e\x4a\xdf\             \\x28\x14\x7f\x5f\x4f\xa2\xb8\xcd\xc9\x43\x00\x40\x0c\xc3\x22\x20\xfd\xd3\x0b\x30\xc0\xa5\x37\x4f\x1d\x2d\x00\xd9\x24\x14\x7b\x15\@@ -427,7 +436,9 @@ sbox_s5 :: Word8 -> Word32 sbox_s5 i = arrayRead32 t (fromIntegral i)   where-    t = array32FromAddrBE 256+    t =+        array32FromAddrBE+            256             "\x7e\xc9\x0c\x04\x2c\x6e\x74\xb9\x9b\x0e\x66\xdf\xa6\x33\x79\x11\xb8\x6a\x7f\xff\x1d\xd3\x58\xf5\x44\xdd\x9d\x44\x17\x31\x16\x7f\             \\x08\xfb\xf1\xfa\xe7\xf5\x11\xcc\xd2\x05\x1b\x00\x73\x5a\xba\x00\x2a\xb7\x22\xd8\x38\x63\x81\xcb\xac\xf6\x24\x3a\x69\xbe\xfd\x7a\             \\xe6\xa2\xe7\x7f\xf0\xc7\x20\xcd\xc4\x49\x48\x16\xcc\xf5\xc1\x80\x38\x85\x16\x40\x15\xb0\xa8\x48\xe6\x8b\x18\xcb\x4c\xaa\xde\xff\@@ -464,7 +475,9 @@ sbox_s6 :: Word8 -> Word32 sbox_s6 i = arrayRead32 t (fromIntegral i)   where-    t = array32FromAddrBE 256+    t =+        array32FromAddrBE+            256             "\xf6\xfa\x8f\x9d\x2c\xac\x6c\xe1\x4c\xa3\x48\x67\xe2\x33\x7f\x7c\x95\xdb\x08\xe7\x01\x68\x43\xb4\xec\xed\x5c\xbc\x32\x55\x53\xac\             \\xbf\x9f\x09\x60\xdf\xa1\xe2\xed\x83\xf0\x57\x9d\x63\xed\x86\xb9\x1a\xb6\xa6\xb8\xde\x5e\xbe\x39\xf3\x8f\xf7\x32\x89\x89\xb1\x38\             \\x33\xf1\x49\x61\xc0\x19\x37\xbd\xf5\x06\xc6\xda\xe4\x62\x5e\x7e\xa3\x08\xea\x99\x4e\x23\xe3\x3c\x79\xcb\xd7\xcc\x48\xa1\x43\x67\@@ -501,7 +514,9 @@ sbox_s7 :: Word8 -> Word32 sbox_s7 i = arrayRead32 t (fromIntegral i)   where-    t = array32FromAddrBE 256+    t =+        array32FromAddrBE+            256             "\x85\xe0\x40\x19\x33\x2b\xf5\x67\x66\x2d\xbf\xff\xcf\xc6\x56\x93\x2a\x8d\x7f\x6f\xab\x9b\xc9\x12\xde\x60\x08\xa1\x20\x28\xda\x1f\             \\x02\x27\xbc\xe7\x4d\x64\x29\x16\x18\xfa\xc3\x00\x50\xf1\x8b\x82\x2c\xb2\xcb\x11\xb2\x32\xe7\x5c\x4b\x36\x95\xf2\xb2\x87\x07\xde\             \\xa0\x5f\xbc\xf6\xcd\x41\x81\xe9\xe1\x50\x21\x0c\xe2\x4e\xf1\xbd\xb1\x68\xc3\x81\xfd\xe4\xe7\x89\x5c\x79\xb0\xd8\x1e\x8b\xfd\x43\@@ -538,7 +553,9 @@ sbox_s8 :: Word8 -> Word32 sbox_s8 i = arrayRead32 t (fromIntegral i)   where-    t = array32FromAddrBE 256+    t =+        array32FromAddrBE+            256             "\xe2\x16\x30\x0d\xbb\xdd\xff\xfc\xa7\xeb\xda\xbd\x35\x64\x80\x95\x77\x89\xf8\xb7\xe6\xc1\x12\x1b\x0e\x24\x16\x00\x05\x2c\xe8\xb5\             \\x11\xa9\xcf\xb0\xe5\x95\x2f\x11\xec\xe7\x99\x0a\x93\x86\xd1\x74\x2a\x42\x93\x1c\x76\xe3\x81\x11\xb1\x2d\xef\x3a\x37\xdd\xdd\xfc\             \\xde\x9a\xde\xb1\x0a\x0c\xc3\x2c\xbe\x19\x70\x29\x84\xa0\x09\x40\xbb\x24\x3a\x0f\xb4\xd1\x37\xcf\xb4\x4e\x79\xf0\x04\x9e\xed\xfd\
Crypto/Cipher/Camellia.hs view
@@ -6,10 +6,9 @@ -- Portability : Good -- -- Camellia support. only 128 bit variant available for now.--module Crypto.Cipher.Camellia-    ( Camellia128-    ) where+module Crypto.Cipher.Camellia (+    Camellia128,+) where  import Crypto.Cipher.Camellia.Primitive import Crypto.Cipher.Types@@ -18,9 +17,9 @@ newtype Camellia128 = Camellia128 Camellia  instance Cipher Camellia128 where-    cipherName    _ = "Camellia128"+    cipherName _ = "Camellia128"     cipherKeySize _ = KeySizeFixed 16-    cipherInit k    = Camellia128 `fmap` initCamellia k+    cipherInit k = Camellia128 `fmap` initCamellia k  instance BlockCipher Camellia128 where     blockSize _ = 16
Crypto/Cipher/Camellia/Primitive.hs view
@@ -1,3 +1,4 @@+{-# LANGUAGE ForeignFunctionInterface #-}  -- | -- Module      : Crypto.Cipher.Camellia.Primitive@@ -6,278 +7,76 @@ -- Stability   : experimental -- Portability : Good --+-- Camellia with a 128-bit key, over the C in @cbits/crypton_camellia.c@.+-- -- This only cover Camellia 128 bits for now. The API will change once -- 192 and 256 mode are implemented too.-{-# LANGUAGE MagicHash #-}-module Crypto.Cipher.Camellia.Primitive-    ( Camellia-    , initCamellia-    , encrypt-    , decrypt-    ) where--import           Data.Word-import           Data.Bits+module Crypto.Cipher.Camellia.Primitive (+    Camellia,+    initCamellia,+    encrypt,+    decrypt,+) where -import           Crypto.Error-import           Crypto.Internal.ByteArray (ByteArrayAccess, ByteArray)+import Crypto.Error+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, Bytes) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Internal.Words-import           Crypto.Internal.WordArray-import           Data.Memory.Endian--data Mode = Decrypt | Encrypt--w64tow128 :: (Word64, Word64) -> Word128-w64tow128 (x1, x2) = Word128 x1 x2--w64tow8 :: Word64 -> (Word8, Word8, Word8, Word8, Word8, Word8, Word8, Word8)-w64tow8 x = (t1, t2, t3, t4, t5, t6, t7, t8)-    where-        t1 = fromIntegral (x `shiftR` 56)-        t2 = fromIntegral (x `shiftR` 48)-        t3 = fromIntegral (x `shiftR` 40)-        t4 = fromIntegral (x `shiftR` 32)-        t5 = fromIntegral (x `shiftR` 24)-        t6 = fromIntegral (x `shiftR` 16)-        t7 = fromIntegral (x `shiftR` 8)-        t8 = fromIntegral (x)--w8tow64 :: (Word8, Word8, Word8, Word8, Word8, Word8, Word8, Word8) -> Word64-w8tow64 (t1,t2,t3,t4,t5,t6,t7,t8) =-    (fromIntegral t1 `shiftL` 56) .|.-    (fromIntegral t2 `shiftL` 48) .|.-    (fromIntegral t3 `shiftL` 40) .|.-    (fromIntegral t4 `shiftL` 32) .|.-    (fromIntegral t5 `shiftL` 24) .|.-    (fromIntegral t6 `shiftL` 16) .|.-    (fromIntegral t7 `shiftL` 8)  .|.-    (fromIntegral t8)--sbox :: Int -> Word8-sbox = arrayRead8 t-  where t = array8-            "\x70\x82\x2c\xec\xb3\x27\xc0\xe5\xe4\x85\x57\x35\xea\x0c\xae\x41\-            \\x23\xef\x6b\x93\x45\x19\xa5\x21\xed\x0e\x4f\x4e\x1d\x65\x92\xbd\-            \\x86\xb8\xaf\x8f\x7c\xeb\x1f\xce\x3e\x30\xdc\x5f\x5e\xc5\x0b\x1a\-            \\xa6\xe1\x39\xca\xd5\x47\x5d\x3d\xd9\x01\x5a\xd6\x51\x56\x6c\x4d\-            \\x8b\x0d\x9a\x66\xfb\xcc\xb0\x2d\x74\x12\x2b\x20\xf0\xb1\x84\x99\-            \\xdf\x4c\xcb\xc2\x34\x7e\x76\x05\x6d\xb7\xa9\x31\xd1\x17\x04\xd7\-            \\x14\x58\x3a\x61\xde\x1b\x11\x1c\x32\x0f\x9c\x16\x53\x18\xf2\x22\-            \\xfe\x44\xcf\xb2\xc3\xb5\x7a\x91\x24\x08\xe8\xa8\x60\xfc\x69\x50\-            \\xaa\xd0\xa0\x7d\xa1\x89\x62\x97\x54\x5b\x1e\x95\xe0\xff\x64\xd2\-            \\x10\xc4\x00\x48\xa3\xf7\x75\xdb\x8a\x03\xe6\xda\x09\x3f\xdd\x94\-            \\x87\x5c\x83\x02\xcd\x4a\x90\x33\x73\x67\xf6\xf3\x9d\x7f\xbf\xe2\-            \\x52\x9b\xd8\x26\xc8\x37\xc6\x3b\x81\x96\x6f\x4b\x13\xbe\x63\x2e\-            \\xe9\x79\xa7\x8c\x9f\x6e\xbc\x8e\x29\xf5\xf9\xb6\x2f\xfd\xb4\x59\-            \\x78\x98\x06\x6a\xe7\x46\x71\xba\xd4\x25\xab\x42\x88\xa2\x8d\xfa\-            \\x72\x07\xb9\x55\xf8\xee\xac\x0a\x36\x49\x2a\x68\x3c\x38\xf1\xa4\-            \\x40\x28\xd3\x7b\xbb\xc9\x43\xc1\x15\xe3\xad\xf4\x77\xc7\x80\x9e"#--sbox1 :: Word8 -> Word8-sbox1 x = sbox (fromIntegral x)--sbox2 :: Word8 -> Word8-sbox2 x = sbox1 x `rotateL` 1--sbox3 :: Word8 -> Word8-sbox3 x = sbox1 x `rotateL` 7--sbox4 :: Word8 -> Word8-sbox4 x = sbox1 (x `rotateL` 1)--sigma1, sigma2, sigma3, sigma4, sigma5, sigma6 :: Word64-sigma1 = 0xA09E667F3BCC908B-sigma2 = 0xB67AE8584CAA73B2-sigma3 = 0xC6EF372FE94F82BE-sigma4 = 0x54FF53A5F1D36F1C-sigma5 = 0x10E527FADE682D1D-sigma6 = 0xB05688C2B3E6C1FD--rotl128 :: Word128 -> Int -> Word128-rotl128 v               0  = v-rotl128 (Word128 x1 x2) 64 = Word128 x2 x1--rotl128 v@(Word128 x1 x2) w-    | w > 64    = (v `rotl128` 64) `rotl128` (w - 64)-    | otherwise = Word128 (x1high .|. x2low) (x2high .|. x1low)-        where-            splitBits i = (i .&. complement x, i .&. x)-                where x = 2 ^ w - 1-            (x1high, x1low) = splitBits (x1 `rotateL` w)-            (x2high, x2low) = splitBits (x2 `rotateL` w)---- | Camellia context-data Camellia = Camellia-    { k  :: Array64-    , kw :: Array64-    , ke :: Array64-    }+import Crypto.Internal.Compat (unsafeDoIO)+import Data.Word+import Foreign.Ptr (Ptr) -setKeyInterim :: ByteArrayAccess key => key -> (Word128, Word128, Word128, Word128)-setKeyInterim keyseed = (w64tow128 kL, w64tow128 kR, w64tow128 kA, w64tow128 kB)-  where kL = (fromBE $ B.toW64BE keyseed 0, fromBE $ B.toW64BE keyseed 8)-        kR = (0, 0)+-- | The subkeys of RFC 3713 section 2.2: kw, k and ke, as 26 64-bit words.+newtype Camellia = Camellia Bytes+    deriving (Eq) -        kA = let d1 = (fst kL `xor` fst kR)-                 d2 = (snd kL `xor` snd kR)-                 d3 = d2 `xor` feistel d1 sigma1-                 d4 = d1 `xor` feistel d3 sigma2-                 d5 = d4 `xor` (fst kL)-                 d6 = d3 `xor` (snd kL)-                 d7 = d6 `xor` feistel d5 sigma3-                 d8 = d5 `xor` feistel d7 sigma4-              in (d8, d7)+scheduleSize :: Int+scheduleSize = 26 * 8 -        kB = let d1 = (fst kA `xor` fst kR)-                 d2 = (snd kA `xor` snd kR)-                 d3 = d2 `xor` feistel d1 sigma5-                 d4 = d1 `xor` feistel d3 sigma6-              in (d4, d3)+blockBytes :: Int+blockBytes = 16 --- | Initialize a 128-bit key------ Return the initialized key or a error message if the given --- keyseed was not 16-bytes in length.-initCamellia :: ByteArray key-             => key -- ^ The key to create the camellia context-             -> CryptoFailable Camellia+-- | Initialize a 128-bit key.+initCamellia :: ByteArrayAccess key => key -> CryptoFailable Camellia initCamellia key-    | B.length key /= 16 = CryptoFailed $ CryptoError_KeySizeInvalid-    | otherwise          =-        let (kL, _, kA, _) = setKeyInterim key in--        let (Word128 kw1 kw2) = (kL `rotl128` 0) in-        let (Word128 k1 k2)   = (kA `rotl128` 0) in-        let (Word128 k3 k4)   = (kL `rotl128` 15) in-        let (Word128 k5 k6)   = (kA `rotl128` 15) in-        let (Word128 ke1 ke2) = (kA `rotl128` 30) in --ke1 = (KA <<<  30) >> 64; ke2 = (KA <<<  30) & MASK64;-        let (Word128 k7 k8)   = (kL `rotl128` 45) in --k7  = (KL <<<  45) >> 64; k8  = (KL <<<  45) & MASK64;-        let (Word128 k9 _)    = (kA `rotl128` 45) in --k9  = (KA <<<  45) >> 64;-        let (Word128 _ k10)   = (kL `rotl128` 60) in-        let (Word128 k11 k12) = (kA `rotl128` 60) in-        let (Word128 ke3 ke4) = (kL `rotl128` 77) in-        let (Word128 k13 k14) = (kL `rotl128` 94) in-        let (Word128 k15 k16) = (kA `rotl128` 94) in-        let (Word128 k17 k18) = (kL `rotl128` 111) in-        let (Word128 kw3 kw4) = (kA `rotl128` 111) in--        CryptoPassed $ Camellia-            { kw = array64 4 [ kw1, kw2, kw3, kw4 ]-            , ke = array64 4 [ ke1, ke2, ke3, ke4 ]-            , k  = array64 18 [ k1, k2, k3, k4, k5, k6, k7, k8, k9, k10, k11, k12, k13, k14, k15, k16, k17, k18 ]-            }--feistel :: Word64 -> Word64 -> Word64-feistel fin sk = -    let x = fin `xor` sk in-    let (t1, t2, t3, t4, t5, t6, t7, t8) = w64tow8 x in-    let t1' = sbox1 t1 in-    let t2' = sbox2 t2 in-    let t3' = sbox3 t3 in-    let t4' = sbox4 t4 in-    let t5' = sbox2 t5 in-    let t6' = sbox3 t6 in-    let t7' = sbox4 t7 in-    let t8' = sbox1 t8 in-    let y1 = t1' `xor` t3' `xor` t4' `xor` t6' `xor` t7' `xor` t8' in-    let y2 = t1' `xor` t2' `xor` t4' `xor` t5' `xor` t7' `xor` t8' in-    let y3 = t1' `xor` t2' `xor` t3' `xor` t5' `xor` t6' `xor` t8' in-    let y4 = t2' `xor` t3' `xor` t4' `xor` t5' `xor` t6' `xor` t7' in-    let y5 = t1' `xor` t2' `xor` t6' `xor` t7' `xor` t8' in-    let y6 = t2' `xor` t3' `xor` t5' `xor` t7' `xor` t8' in-    let y7 = t3' `xor` t4' `xor` t5' `xor` t6' `xor` t8' in-    let y8 = t1' `xor` t4' `xor` t5' `xor` t6' `xor` t7' in-    w8tow64 (y1, y2, y3, y4, y5, y6, y7, y8)--fl :: Word64 -> Word64 -> Word64-fl fin sk =-    let (x1, x2) = w64to32 fin in-    let (k1, k2) = w64to32 sk in-    let y2 = x2 `xor` ((x1 .&. k1) `rotateL` 1) in-    let y1 = x1 `xor` (y2 .|. k2) in-    w32to64 (y1, y2)--flinv :: Word64 -> Word64 -> Word64-flinv fin sk =-    let (y1, y2) = w64to32 fin in-    let (k1, k2) = w64to32 sk in-    let x1 = y1 `xor` (y2 .|. k2) in-    let x2 = y2 `xor` ((x1 .&. k1) `rotateL` 1) in-    w32to64 (x1, x2)--{- in decrypt mode 0->17 1->16 ... -}-getKeyK :: Mode -> Camellia -> Int -> Word64-getKeyK Encrypt key i = k key `arrayRead64` i-getKeyK Decrypt key i = k key `arrayRead64` (17 - i)--{- in decrypt mode 0->3 1->2 2->1 3->0 -}-getKeyKe :: Mode -> Camellia -> Int -> Word64-getKeyKe Encrypt key i = ke key `arrayRead64` i-getKeyKe Decrypt key i = ke key `arrayRead64` (3 - i)--{- in decrypt mode 0->2 1->3 2->0 3->1 -}-getKeyKw :: Mode -> Camellia -> Int -> Word64-getKeyKw Encrypt key i = (kw key) `arrayRead64` i-getKeyKw Decrypt key i = (kw key) `arrayRead64` ((i + 2) `mod` 4)--{- perform the following-    D2 = D2 ^ F(D1, k1);     // Round 1-    D1 = D1 ^ F(D2, k2);     // Round 2-    D2 = D2 ^ F(D1, k3);     // Round 3-    D1 = D1 ^ F(D2, k4);     // Round 4-    D2 = D2 ^ F(D1, k5);     // Round 5-    D1 = D1 ^ F(D2, k6);     // Round 6- -}-doBlockRound :: Mode -> Camellia -> Word64 -> Word64 -> Int -> (Word64, Word64)-doBlockRound mode key d1 d2 i =-    let r1 = d2 `xor` feistel d1 (getKeyK mode key (0+i)) in     {- Round 1+i -}-    let r2 = d1 `xor` feistel r1 (getKeyK mode key (1+i)) in     {- Round 2+i -}-    let r3 = r1 `xor` feistel r2 (getKeyK mode key (2+i)) in     {- Round 3+i -}-    let r4 = r2 `xor` feistel r3 (getKeyK mode key (3+i)) in     {- Round 4+i -}-    let r5 = r3 `xor` feistel r4 (getKeyK mode key (4+i)) in     {- Round 5+i -}-    let r6 = r4 `xor` feistel r5 (getKeyK mode key (5+i)) in     {- Round 6+i -}-    (r6, r5)--doBlock :: Mode -> Camellia -> Word128 -> Word128-doBlock mode key (Word128 d1 d2) =-    let d1a = d1 `xor` (getKeyKw mode key 0) in {- Prewhitening -}-    let d2a = d2 `xor` (getKeyKw mode key 1) in--    let (d1b, d2b) = doBlockRound mode key d1a d2a 0 in--    let d1c = fl    d1b (getKeyKe mode key 0) in {- FL -}-    let d2c = flinv d2b (getKeyKe mode key 1) in {- FLINV -}--    let (d1d, d2d) = doBlockRound mode key d1c d2c 6 in--    let d1e = fl    d1d (getKeyKe mode key 2) in {- FL -}-    let d2e = flinv d2d (getKeyKe mode key 3) in {- FLINV -}+    | B.length key /= 16 = CryptoFailed CryptoError_KeySizeInvalid+    | otherwise =+        CryptoPassed $+            Camellia $+                B.allocAndFreeze scheduleSize $ \ks ->+                    B.withByteArray key $ \k -> c_camellia_init ks k -    let (d1f, d2f) = doBlockRound mode key d1e d2e 12 in+-- | Encrypt the given input, which has to be a whole number of blocks.+encrypt :: ByteArray ba => Camellia -> ba -> ba+encrypt = run c_camellia_encrypt -    let d2g = d2f `xor` (getKeyKw mode key 2) in {- Postwhitening -}-    let d1g = d1f `xor` (getKeyKw mode key 3) in-    w64tow128 (d2g, d1g)+-- | Decrypt the given input, which has to be a whole number of blocks.+decrypt :: ByteArray ba => Camellia -> ba -> ba+decrypt = run c_camellia_decrypt -{- encryption for 128 bits blocks -}-encryptBlock :: Camellia -> Word128 -> Word128-encryptBlock = doBlock Encrypt+run+    :: ByteArray ba+    => (Ptr Word8 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO ())+    -> Camellia+    -> ba+    -> ba+run f (Camellia sched) input+    | len `mod` blockBytes /= 0 =+        error $+            "Crypto.Cipher.Camellia: input length must be a multiple of block size (16). Its length is: "+                ++ show len+    | otherwise = unsafeDoIO $+        B.alloc len $ \out ->+            B.withByteArray sched $ \ks ->+                B.withByteArray input $ \inp ->+                    f out ks inp (fromIntegral (len `div` blockBytes))+  where+    len = B.length input -{- decryption for 128 bits blocks -}-decryptBlock :: Camellia -> Word128 -> Word128-decryptBlock = doBlock Decrypt+foreign import ccall unsafe "crypton_camellia.h crypton_camellia_init"+    c_camellia_init :: Ptr Word8 -> Ptr Word8 -> IO () --- | Encrypts the given ByteString using the given Key-encrypt :: ByteArray ba-        => Camellia     -- ^ The key to use-        -> ba           -- ^ The data to encrypt-        -> ba-encrypt key = B.mapAsWord128 (encryptBlock key)+foreign import ccall unsafe "crypton_camellia.h crypton_camellia_encrypt"+    c_camellia_encrypt :: Ptr Word8 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO () --- | Decrypts the given ByteString using the given Key-decrypt :: ByteArray ba-        => Camellia     -- ^ The key to use-        -> ba           -- ^ The data to decrypt-        -> ba-decrypt key = B.mapAsWord128 (decryptBlock key)+foreign import ccall unsafe "crypton_camellia.h crypton_camellia_decrypt"+    c_camellia_decrypt :: Ptr Word8 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO ()
Crypto/Cipher/ChaCha.hs view
@@ -1,30 +1,40 @@+{-# LANGUAGE CApiFFI #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.Cipher.ChaCha -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : stable -- Portability : good----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.Cipher.ChaCha-    ( initialize-    , initializeX-    , combine-    , generate-    , State+module Crypto.Cipher.ChaCha (+    initialize,+    initializeX,+    combine,+    generate,+    State,+     -- * Simple interface for DRG purpose-    , initializeSimple-    , generateSimple-    , StateSimple-    ) where+    initializeSimple,+    generateSimple,+    StateSimple, -import           Crypto.Internal.ByteArray (ByteArrayAccess, ByteArray, ScrubbedBytes)+    -- * Seeking and cursor for DRG purposes+    generateSimpleBlock,+    ChaChaState (..),+) where++import Crypto.Internal.ByteArray (+    ByteArray,+    ByteArrayAccess,+    ScrubbedBytes,+ ) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Internal.Compat-import           Crypto.Internal.Imports-import           Foreign.Ptr-import           Foreign.C.Types+import Crypto.Internal.Compat+import Crypto.Internal.Imports+import Foreign.C.Types+import Foreign.Ptr  -- | ChaCha context newtype State = State ScrubbedBytes@@ -34,111 +44,217 @@ newtype StateSimple = StateSimple ScrubbedBytes -- just ChaCha's state     deriving (NFData) +class ChaChaState a where+    getCounter64 :: a -> Word64+    setCounter64 :: Word64 -> a -> a+    getCounter32 :: a -> Word32+    setCounter32 :: Word32 -> a -> a++instance ChaChaState State where+    getCounter64 (State st) = getCounter64' st ccrypton_chacha_get_state+    setCounter64 n (State st) = State $ setCounter64' n st ccrypton_chacha_get_state+    getCounter32 (State st) = getCounter32' st ccrypton_chacha_get_state+    setCounter32 n (State st) = State $ setCounter32' n st ccrypton_chacha_get_state++instance ChaChaState StateSimple where+    getCounter64 (StateSimple st) = getCounter64' st id+    setCounter64 n (StateSimple st) = StateSimple $ setCounter64' n st id+    getCounter32 (StateSimple st) = getCounter32' st id+    setCounter32 n (StateSimple st) = StateSimple $ setCounter32' n st id++getCounter64' :: ScrubbedBytes -> (Ptr a -> Ptr StateSimple) -> Word64+getCounter64' currSt conv =+    unsafeDoIO $ do+        B.withByteArray currSt $ \stPtr ->+            ccrypton_chacha_counter64 $ conv stPtr++getCounter32' :: ScrubbedBytes -> (Ptr a -> Ptr StateSimple) -> Word32+getCounter32' currSt conv =+    unsafeDoIO $ do+        B.withByteArray currSt $ \stPtr ->+            ccrypton_chacha_counter32 $ conv stPtr++setCounter64'+    :: Word64 -> ScrubbedBytes -> (Ptr a -> Ptr StateSimple) -> ScrubbedBytes+setCounter64' newCounter prevSt conv =+    unsafeDoIO $ do+        newSt <- B.copy prevSt (\_ -> return ())+        B.withByteArray newSt $ \stPtr ->+            ccrypton_chacha_set_counter64 (conv stPtr) newCounter+        return newSt++setCounter32'+    :: Word32 -> ScrubbedBytes -> (Ptr a -> Ptr StateSimple) -> ScrubbedBytes+setCounter32' newCounter prevSt conv =+    unsafeDoIO $ do+        newSt <- B.copy prevSt (\_ -> return ())+        B.withByteArray newSt $ \stPtr ->+            ccrypton_chacha_set_counter32 (conv stPtr) newCounter+        return newSt+ -- | Initialize a new ChaCha context with the number of rounds, -- the key and the nonce associated.-initialize :: (ByteArrayAccess key, ByteArrayAccess nonce)-           => Int   -- ^ number of rounds (8,12,20)-           -> key   -- ^ the key (128 or 256 bits)-           -> nonce -- ^ the nonce (64 or 96 bits)-           -> State -- ^ the initial ChaCha state+-- To use ChaCha20 defined in RFC 8439, 20, 256bits-key and 96-bits nonce must be used.+initialize+    :: (ByteArrayAccess key, ByteArrayAccess nonce)+    => Int+    -- ^ number of rounds (8,12,20)+    -> key+    -- ^ the key (128 or 256 bits)+    -> nonce+    -- ^ the nonce (64 or 96 bits)+    -> State+    -- ^ the initial ChaCha state initialize nbRounds key nonce-    | kLen `notElem` [16,32]          = error "ChaCha: key length should be 128 or 256 bits"-    | nonceLen `notElem` [8,12]       = error "ChaCha: nonce length should be 64 or 96 bits"-    | nbRounds `notElem` [8,12,20]    = error "ChaCha: rounds should be 8, 12 or 20"-    | otherwise                       = unsafeDoIO $ do+    | kLen `notElem` [16, 32] =+        error "ChaCha: key length should be 128 or 256 bits"+    | nonceLen `notElem` [8, 12] =+        error "ChaCha: nonce length should be 64 or 96 bits"+    | nbRounds `notElem` [8, 12, 20] = error "ChaCha: rounds should be 8, 12 or 20"+    | otherwise = unsafeDoIO $ do         stPtr <- B.alloc 132 $ \stPtr ->-            B.withByteArray nonce $ \noncePtr  ->-            B.withByteArray key   $ \keyPtr ->-                ccrypton_chacha_init stPtr  nbRounds kLen keyPtr nonceLen noncePtr+            B.withByteArray nonce $ \noncePtr ->+                B.withByteArray key $ \keyPtr ->+                    ccrypton_chacha_init stPtr nbRounds kLen keyPtr nonceLen noncePtr         return $ State stPtr-  where kLen     = B.length key-        nonceLen = B.length nonce+  where+    kLen = B.length key+    nonceLen = B.length nonce  -- | Initialize a new XChaCha context with the number of rounds, -- the key and the nonce associated. -- -- An XChaCha state can be used like a regular ChaCha state after initialisation.-initializeX :: (ByteArrayAccess key, ByteArrayAccess nonce)-            => Int   -- ^ number of rounds (8,12,20)-            -> key   -- ^ the key (256 bits)-            -> nonce -- ^ the nonce (192 bits)-            -> State -- ^ the initial ChaCha state+initializeX+    :: (ByteArrayAccess key, ByteArrayAccess nonce)+    => Int+    -- ^ number of rounds (8,12,20)+    -> key+    -- ^ the key (256 bits)+    -> nonce+    -- ^ the nonce (192 bits)+    -> State+    -- ^ the initial ChaCha state initializeX nbRounds key nonce-    | kLen /= 32                      = error "XChaCha: key length should be 256 bits"-    | nonceLen /= 24                  = error "XChaCha: nonce length should be 192 bits"-    | nbRounds `notElem` [8,12,20]    = error "XChaCha: rounds should be 8, 12 or 20"-    | otherwise                       = unsafeDoIO $ do+    | kLen /= 32 =+        error "XChaCha: key length should be 256 bits"+    | nonceLen /= 24 =+        error "XChaCha: nonce length should be 192 bits"+    | nbRounds `notElem` [8, 12, 20] =+        error "XChaCha: rounds should be 8, 12 or 20"+    | otherwise = unsafeDoIO $ do         stPtr <- B.alloc 132 $ \stPtr ->-            B.withByteArray nonce $ \noncePtr  ->-            B.withByteArray key   $ \keyPtr ->-                ccrypton_xchacha_init stPtr nbRounds keyPtr noncePtr+            B.withByteArray nonce $ \noncePtr ->+                B.withByteArray key $ \keyPtr ->+                    ccrypton_xchacha_init stPtr nbRounds keyPtr noncePtr         return $ State stPtr-  where kLen     = B.length key-        nonceLen = B.length nonce+  where+    kLen = B.length key+    nonceLen = B.length nonce  -- | Initialize simple ChaCha State -- -- The seed need to be at least 40 bytes long-initializeSimple :: ByteArrayAccess seed-                 => seed -- ^ a 40 bytes long seed-                 -> StateSimple+initializeSimple+    :: ByteArrayAccess seed+    => seed+    -- ^ a 40 bytes long seed+    -> StateSimple initializeSimple seed     | sLen < 40 = error "ChaCha Random: seed length should be 40 bytes"     | otherwise = unsafeDoIO $ do         stPtr <- B.alloc 64 $ \stPtr ->-                    B.withByteArray seed $ \seedPtr ->-                        ccrypton_chacha_init_core stPtr 32 seedPtr 8 (seedPtr `plusPtr` 32)+            B.withByteArray seed $ \seedPtr ->+                ccrypton_chacha_init_core stPtr 32 seedPtr 8 (seedPtr `plusPtr` 32)         return $ StateSimple stPtr   where     sLen = B.length seed  -- | Combine the chacha output and an arbitrary message with a xor, -- and return the combined output and the new state.-combine :: ByteArray ba-        => State       -- ^ the current ChaCha state-        -> ba          -- ^ the source to xor with the generator-        -> (ba, State)+combine+    :: ByteArray ba+    => State+    -- ^ the current ChaCha state+    -> ba+    -- ^ the source to xor with the generator+    -> (ba, State) combine prevSt@(State prevStMem) src     | B.null src = (B.empty, prevSt)-    | otherwise  = unsafeDoIO $ do+    | otherwise = unsafeDoIO $ do         (out, st) <- B.copyRet prevStMem $ \ctx ->-            B.alloc (B.length src) $ \dstPtr ->-            B.withByteArray src    $ \srcPtr ->-                ccrypton_chacha_combine dstPtr ctx srcPtr (fromIntegral $ B.length src)+            B.alloc n $ \dstPtr ->+                B.withByteArray src $ \srcPtr ->+                    -- in pieces the C's uint32_t length can hold; it carries+                    -- the state in ctx, so it can simply be called again+                    B.inCLengths n $ \off len ->+                        ccrypton_chacha_combine+                            (dstPtr `plusPtr` off)+                            ctx+                            (srcPtr `plusPtr` off)+                            (fromIntegral len)         return (out, State st)+  where+    n = B.length src  -- | Generate a number of bytes from the ChaCha output directly-generate :: ByteArray ba-         => State -- ^ the current ChaCha state-         -> Int   -- ^ the length of data to generate-         -> (ba, State)+generate+    :: ByteArray ba+    => State+    -- ^ the current ChaCha state+    -> Int+    -- ^ the length of data to generate+    -> (ba, State) generate prevSt@(State prevStMem) len-    | len <= 0  = (B.empty, prevSt)+    | len <= 0 = (B.empty, prevSt)     | otherwise = unsafeDoIO $ do         (out, st) <- B.copyRet prevStMem $ \ctx ->             B.alloc len $ \dstPtr ->-                ccrypton_chacha_generate dstPtr ctx (fromIntegral len)+                B.inCLengths len $ \off n ->+                    ccrypton_chacha_generate+                        (dstPtr `plusPtr` off)+                        ctx+                        (fromIntegral n)         return (out, State st)  -- | similar to 'generate' but assume certains values-generateSimple :: ByteArray ba-               => StateSimple-               -> Int-               -> (ba, StateSimple)+generateSimple+    :: ByteArray ba+    => StateSimple+    -> Int+    -> (ba, StateSimple) generateSimple (StateSimple prevSt) nbBytes = unsafeDoIO $ do-    newSt  <- B.copy prevSt (\_ -> return ())+    newSt <- B.copy prevSt (\_ -> return ())     output <- B.alloc nbBytes $ \dstPtr ->         B.withByteArray newSt $ \stPtr ->             ccrypton_chacha_random 8 dstPtr stPtr (fromIntegral nbBytes)     return (output, StateSimple newSt) -foreign import ccall "crypton_chacha_init_core"-    ccrypton_chacha_init_core :: Ptr StateSimple -> Int -> Ptr Word8 -> Int -> Ptr Word8 -> IO ()+-- | similar to 'generate' but accepts a number of rounds, and always generates+--   64 bytes (a single block)+generateSimpleBlock+    :: ByteArray ba+    => Word8+    -> StateSimple+    -> (ba, StateSimple)+generateSimpleBlock nbRounds (StateSimple prevSt)+    | nbRounds `notElem` [8, 12, 20] = error "ChaCha: rounds should be 8, 12 or 20"+    | otherwise = unsafeDoIO $ do+        newSt <- B.copy prevSt (\_ -> return ())+        output <- B.alloc 64 $ \dstPtr ->+            B.withByteArray newSt $ \stPtr ->+                ccrypton_chacha_generate_simple_block dstPtr stPtr nbRounds+        return (output, StateSimple newSt) -foreign import ccall "crypton_chacha_init"-    ccrypton_chacha_init :: Ptr State -> Int -> Int -> Ptr Word8 -> Int -> Ptr Word8 -> IO ()+foreign import ccall unsafe "crypton_chacha_init_core"+    ccrypton_chacha_init_core+        :: Ptr StateSimple -> Int -> Ptr Word8 -> Int -> Ptr Word8 -> IO () -foreign import ccall "crypton_xchacha_init"+foreign import ccall unsafe "crypton_chacha_init"+    ccrypton_chacha_init+        :: Ptr State -> Int -> Int -> Ptr Word8 -> Int -> Ptr Word8 -> IO ()++foreign import ccall unsafe "crypton_xchacha_init"     ccrypton_xchacha_init :: Ptr State -> Int -> Ptr Word8 -> Ptr Word8 -> IO ()  foreign import ccall "crypton_chacha_combine"@@ -150,3 +266,21 @@ foreign import ccall "crypton_chacha_random"     ccrypton_chacha_random :: Int -> Ptr Word8 -> Ptr StateSimple -> CUInt -> IO () +foreign import ccall unsafe "crypton_chacha_counter64"+    ccrypton_chacha_counter64 :: Ptr StateSimple -> IO Word64++foreign import ccall unsafe "crypton_chacha_set_counter64"+    ccrypton_chacha_set_counter64 :: Ptr StateSimple -> Word64 -> IO ()++foreign import ccall unsafe "crypton_chacha_counter32"+    ccrypton_chacha_counter32 :: Ptr StateSimple -> IO Word32++foreign import ccall unsafe "crypton_chacha_set_counter32"+    ccrypton_chacha_set_counter32 :: Ptr StateSimple -> Word32 -> IO ()++foreign import ccall unsafe "crypton_chacha_generate_simple_block"+    ccrypton_chacha_generate_simple_block+        :: Ptr Word8 -> Ptr StateSimple -> Word8 -> IO ()++foreign import capi unsafe "crypton_chacha.h crypton_chacha_get_state"+    ccrypton_chacha_get_state :: Ptr State -> Ptr StateSimple
+ Crypto/Cipher/ChaCha/Poly1305.hs view
@@ -0,0 +1,335 @@+-- |+-- Module      : Crypto.Cipher.ChaCha.Poly1305+-- License     : BSD-style+-- Maintainer  : Kazu Yamamoto <kazu@iij.ad.jp>+-- Stability   : experimental+-- Portability : Good+--+-- ChaCha20-Poly1305 (RFC 8439) a message at a time.+--+-- "Crypto.Cipher.ChaChaPoly1305" takes a message in pieces: a state is+-- started, the additional data appended, the body encrypted and the tag+-- taken, each a step of its own.  That is what a protocol wants when the+-- message arrives in pieces, and it is eight foreign calls and the+-- allocations between them when the message was already whole.+--+-- Here the whole message goes in one call.+--+-- The functions are the same shape as "Crypto.Cipher.AES.GCM", so a protocol+-- that offers both ciphers can hold them the same way.+module Crypto.Cipher.ChaCha.Poly1305 (+    Context,+    newContext,+    encrypt,+    decrypt,+    decryptWithTag,+) where++import Crypto.Cipher.Types (AuthTag (..))+import Crypto.Error+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess)+import qualified Crypto.Internal.ByteArray as B+import Crypto.Internal.Compat (unsafeDoIO)+import Crypto.Internal.Imports+import Foreign.C.Types (CInt (..), CUInt (..))+import Foreign.Ptr (Ptr, plusPtr)++-- | A key, checked once.+--+-- ChaCha20-Poly1305 has nothing to precompute from a key: the one-time+-- Poly1305 key comes from the nonce, so it differs for every message.  This+-- holds the thirty-two bytes and the knowledge that they are thirty-two, and+-- exists so that the interface is the one "Crypto.Cipher.AES.GCM" has.+newtype Context = Context B.ScrubbedBytes++instance NFData Context where+    rnf (Context k) = k `seq` ()++-- | Take a key of 32 bytes.  Any other length is reported as+-- 'CryptoError_KeySizeInvalid'.+newContext :: ByteArrayAccess key => key -> CryptoFailable Context+newContext k+    | B.length k /= 32 = CryptoFailed CryptoError_KeySizeInvalid+    | otherwise = CryptoPassed $ Context (B.convert k)+{-# INLINABLE newContext #-}++-- | Encrypt one message.  The result is the ciphertext with the tag after it,+-- which is the shape 'decrypt' expects.+--+-- The nonce is the twelve bytes RFC 8439 defines; any other length gives+-- 'CryptoError_IvSizeInvalid'.  RFC 8439 requires a 16-byte tag.+{-# INLINABLE encrypt #-}+encrypt+    :: ( ByteArrayAccess nonce+       , ByteArrayAccess aad+       , ByteArrayAccess ba+       , ByteArray output+       )+    => Context+    -> nonce+    -> aad+    -> ba+    -> Int+    -> CryptoFailable output+encrypt (Context k) nonce aad input taglen+    | tooLongForC aad input = CryptoFailed CryptoError_ParameterInvalid+    | not (validNonce nonce) = CryptoFailed CryptoError_IvSizeInvalid+    | badTag taglen = CryptoFailed CryptoError_AuthenticationTagSizeInvalid+    | otherwise =+        CryptoPassed $+            unsafeDoIO $+                B.alloc (B.length input + taglen) $ \out ->+                    B.withByteArray k $ \kp ->+                        B.withByteArray nonce $ \np ->+                            B.withByteArray aad $ \ap ->+                                B.withByteArray input $ \ip ->+                                    (callE (B.length input))+                                        out+                                        (out `plusPtr` B.length input)+                                        (fromIntegral taglen)+                                        kp+                                        np+                                        (fromIntegral $ B.length nonce)+                                        ap+                                        (fromIntegral $ B.length aad)+                                        ip+                                        (fromIntegral $ B.length input)++-- | Decrypt one message, in the shape 'encrypt' produced: the ciphertext with+-- its tag after it.  The tag is compared here, every byte of it whatever the+-- answer, and a message whose tag does not match gives 'Nothing' rather than+-- the plaintext.+--+-- 'Nothing' also comes back when the input is shorter than the tag, or the+-- nonce is not twelve bytes.+{-# INLINABLE decrypt #-}+decrypt+    :: (ByteArrayAccess nonce, ByteArrayAccess aad, ByteArray ba)+    => Context+    -> nonce+    -> aad+    -> ba+    -> Int+    -> Maybe ba+decrypt (Context k) nonce aad input taglen+    | tooLongForC aad input = Nothing+    | not (validNonce nonce) = Nothing+    | badTag taglen || B.length input < taglen = Nothing+    | otherwise = unsafeDoIO $ do+        (r, out) <- B.allocRet bodylen $ \outp ->+            B.withByteArray k $ \kp ->+                B.withByteArray nonce $ \np ->+                    B.withByteArray aad $ \ap ->+                        B.withByteArray body $ \ip ->+                            B.withByteArray tag $ \tp ->+                                (callD bodylen)+                                    outp+                                    tp+                                    (fromIntegral taglen)+                                    kp+                                    np+                                    (fromIntegral $ B.length nonce)+                                    ap+                                    (fromIntegral $ B.length aad)+                                    ip+                                    (fromIntegral bodylen)+        return $ if r /= 0 then Just out else Nothing+  where+    bodylen = B.length input - taglen+    (body, tag) = B.splitAt bodylen input++-- | Decrypt one message, the tag kept apart, and hand back the tag this end+-- computed.+--+-- For a caller whose protocol carries the tag separately from the ciphertext,+-- so that 'decrypt' -- which wants the two together and compares them itself+-- -- does not fit.  Compare the two tags with '=='; the 'Eq' instance of+-- t'AuthTag' is a constant-time comparison, and taking them apart to compare+-- the bytes is how this goes wrong.+--+-- Nothing here says whether the message is authentic.  Until the comparison+-- is made and has come out equal, what this returns is not plaintext, it is+-- what the ciphertext turns into, and a caller must not act on it.+{-# INLINABLE decryptWithTag #-}+decryptWithTag+    :: (ByteArrayAccess nonce, ByteArrayAccess aad, ByteArray ba)+    => Context+    -> nonce+    -> aad+    -> ba+    -> Int+    -> CryptoFailable (ba, AuthTag)+decryptWithTag (Context k) nonce aad input taglen+    | tooLongForC aad input = CryptoFailed CryptoError_ParameterInvalid+    | not (validNonce nonce) = CryptoFailed CryptoError_IvSizeInvalid+    | badTag taglen = CryptoFailed CryptoError_AuthenticationTagSizeInvalid+    | otherwise = CryptoPassed $ unsafeDoIO $ do+        (tagbs, out) <- B.allocRet (B.length input) $ \outp ->+            B.alloc taglen $ \tagp ->+                B.withByteArray k $ \kp ->+                    B.withByteArray nonce $ \np ->+                        B.withByteArray aad $ \ap ->+                            B.withByteArray input $ \ip ->+                                (callT (B.length input))+                                    outp+                                    tagp+                                    (fromIntegral taglen)+                                    kp+                                    np+                                    (fromIntegral $ B.length nonce)+                                    ap+                                    (fromIntegral $ B.length aad)+                                    ip+                                    (fromIntegral $ B.length input)+        return (out, AuthTag $ B.convert (tagbs :: B.Bytes))++-- | The C takes its lengths as @uint32_t@, so a message or its additional+-- data from 2^32 bytes up cannot be handed to it: the length would be+-- truncated and most of the buffer left untouched, with nothing to say so.+-- The C does the whole message in one call, so there is no splitting it.+tooLongForC+    :: (ByteArrayAccess aad, ByteArrayAccess ba) => aad -> ba -> Bool+tooLongForC aad input =+    B.overCLength (B.length aad) || B.overCLength (B.length input)++-- RFC 8439 is the twelve-byte nonce.  ChaCha20 will take eight, but that is+-- the other construction, with a 64-bit block counter, and it is not what+-- this AEAD is defined over -- so it is refused here rather than quietly+-- encrypting under a scheme nobody asked for.+validNonce :: ByteArrayAccess nonce => nonce -> Bool+validNonce n = B.length n == 12++badTag :: Int -> Bool+badTag t = t /= 16++-- | An unsafe call keeps a capability for as long as it runs, so it is only+-- for a message short enough that the run is short.  Four kibibytes is what+-- the AES side uses, and it takes in a datagram of any size a network will+-- carry.+shortMessage :: Int+shortMessage = 4096++callE :: Int -> CEncrypt+callE n+    | n <= shortMessage = c_chachapoly_encrypt_unsafe+    | otherwise = c_chachapoly_encrypt++callD :: Int -> CDecrypt+callD n+    | n <= shortMessage = c_chachapoly_decrypt_unsafe+    | otherwise = c_chachapoly_decrypt++callT :: Int -> CEncrypt+callT n+    | n <= shortMessage = c_chachapoly_decrypt_tag_unsafe+    | otherwise = c_chachapoly_decrypt_tag++type CEncrypt =+    Ptr Word8+    -> Ptr Word8+    -> CUInt+    -> Ptr Word8+    -> Ptr Word8+    -> CUInt+    -> Ptr Word8+    -> CUInt+    -> Ptr Word8+    -> CUInt+    -> IO ()++type CDecrypt =+    Ptr Word8+    -> Ptr Word8+    -> CUInt+    -> Ptr Word8+    -> Ptr Word8+    -> CUInt+    -> Ptr Word8+    -> CUInt+    -> Ptr Word8+    -> CUInt+    -> IO CInt++foreign import ccall "crypton_chachapoly.h crypton_chachapoly_encrypt"+    c_chachapoly_encrypt+        :: Ptr Word8+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> IO ()++foreign import ccall unsafe "crypton_chachapoly.h crypton_chachapoly_encrypt"+    c_chachapoly_encrypt_unsafe+        :: Ptr Word8+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> IO ()++foreign import ccall "crypton_chachapoly.h crypton_chachapoly_decrypt"+    c_chachapoly_decrypt+        :: Ptr Word8+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> IO CInt++foreign import ccall unsafe "crypton_chachapoly.h crypton_chachapoly_decrypt"+    c_chachapoly_decrypt_unsafe+        :: Ptr Word8+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> IO CInt++foreign import ccall "crypton_chachapoly.h crypton_chachapoly_decrypt_tag"+    c_chachapoly_decrypt_tag+        :: Ptr Word8+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> IO ()++foreign import ccall unsafe "crypton_chachapoly.h crypton_chachapoly_decrypt_tag"+    c_chachapoly_decrypt_tag_unsafe+        :: Ptr Word8+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> Ptr Word8+        -> CUInt+        -> IO ()
Crypto/Cipher/ChaChaPoly1305.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.Cipher.ChaChaPoly1305 -- License     : BSD-style@@ -6,7 +8,7 @@ -- Portability : good -- -- A simple AEAD scheme using ChaCha20 and Poly1305. See--- <https://tools.ietf.org/html/rfc7539 RFC 7539>.+-- <https://tools.ietf.org/html/rfc8439 RFC 8439>. -- -- The State is not modified in place, so each function changing the State, -- returns a new State.@@ -28,69 +30,86 @@ -- >    -> ByteString -- input plaintext to be encrypted -- >    -> CryptoFailable ByteString -- ciphertext with a 128-bit tag attached -- >encrypt nonce key header plaintext = do--- >    st1 <- C.nonce12 nonce >>= C.initialize key+-- >    st1 <- C.initialize <$> C.key key <*> C.nonce12 nonce -- >    let -- >        st2 = C.finalizeAAD $ C.appendAAD header st1 -- >        (out, st3) = C.encrypt plaintext st2 -- >        auth = C.finalize st3 -- >    return $ out `B.append` Data.ByteArray.convert auth----module Crypto.Cipher.ChaChaPoly1305-    ( State-    , Nonce-    , XNonce-    , nonce12-    , nonce8-    , nonce24-    , incrementNonce-    , initialize-    , initializeX-    , appendAAD-    , finalizeAAD-    , encrypt-    , decrypt-    , finalize-    ) where+module Crypto.Cipher.ChaChaPoly1305 (+    -- * AEAD+    ChaCha20Poly1305,+    aeadChacha20poly1305Init, -import           Control.Monad             (when)-import           Crypto.Internal.ByteArray (ByteArrayAccess, ByteArray, Bytes, ScrubbedBytes)-import qualified Crypto.Internal.ByteArray as B-import           Crypto.Internal.Imports-import           Crypto.Error+    -- * Low level+    State,+    Key,+    key,+    Nonce,+    XNonce,+    nonce12,+    nonce8,+    nonce24,+    incrementNonce,+    initialize,+    initializeX,+    appendAAD,+    finalizeAAD,+    encrypt,+    decrypt,+    finalize,+) where++import Control.Monad (when) import qualified Crypto.Cipher.ChaCha as ChaCha-import qualified Crypto.MAC.Poly1305  as Poly1305-import           Data.Memory.Endian+import Crypto.Cipher.Types+import Crypto.Error+import Crypto.Internal.ByteArray (+    ByteArray,+    ByteArrayAccess,+    Bytes,+    ScrubbedBytes,+ )+import qualified Crypto.Internal.ByteArray as B+import Crypto.Internal.Imports+import qualified Crypto.Internal.Poly1305 as PolyKey+import qualified Crypto.MAC.Poly1305 as Poly1305 import qualified Data.ByteArray.Pack as P-import           Foreign.Ptr-import           Foreign.Storable+import Data.Memory.Endian+import Foreign.Ptr+import Foreign.Storable  -- | A ChaChaPoly1305 State. -- -- The state is immutable, and only new state can be created-data State = State !ChaCha.State-                   !Poly1305.State-                   !Word64 -- AAD length-                   !Word64 -- ciphertext length+data State+    = State+        !ChaCha.State+        !Poly1305.State+        !Word64 -- AAD length+        !Word64 -- ciphertext length +-- | A ChaChaPoly1305 State.+type ChaCha20Poly1305 = State+ -- | Valid Nonce for ChaChaPoly1305. -- -- It can be created with 'nonce8' or 'nonce12' data Nonce = Nonce8 Bytes | Nonce12 Bytes  instance ByteArrayAccess Nonce where-  length (Nonce8  n) = B.length n-  length (Nonce12 n) = B.length n+    length (Nonce8 n) = B.length n+    length (Nonce12 n) = B.length n -  withByteArray (Nonce8  n) = B.withByteArray n-  withByteArray (Nonce12 n) = B.withByteArray n+    withByteArray (Nonce8 n) = B.withByteArray n+    withByteArray (Nonce12 n) = B.withByteArray n  -- | Extended nonce for XChaChaPoly1305. newtype XNonce = Nonce24 Bytes  instance ByteArrayAccess XNonce where-  length (Nonce24 n) = B.length n-  withByteArray (Nonce24 n) = B.withByteArray n-+    length (Nonce24 n) = B.length n+    withByteArray (Nonce24 n) = B.withByteArray n  -- Based on the following pseudo code: --@@ -108,7 +127,7 @@ pad16 :: Word64 -> Bytes pad16 n     | modLen == 0 = B.empty-    | otherwise   = B.replicate (16 - modLen) 0+    | otherwise = B.replicate (16 - modLen) 0   where     modLen = fromIntegral (n `mod` 16) @@ -116,78 +135,83 @@ nonce12 :: ByteArrayAccess iv => iv -> CryptoFailable Nonce nonce12 iv     | B.length iv /= 12 = CryptoFailed CryptoError_IvSizeInvalid-    | otherwise         = CryptoPassed . Nonce12 . B.convert $ iv+    | otherwise = CryptoPassed . Nonce12 . B.convert $ iv  -- | 8 bytes IV, nonce constructor-nonce8 :: ByteArrayAccess ba-       => ba -- ^ 4 bytes constant-       -> ba -- ^ 8 bytes IV-       -> CryptoFailable Nonce+nonce8+    :: ByteArrayAccess ba+    => ba+    -- ^ 4 bytes constant+    -> ba+    -- ^ 8 bytes IV+    -> CryptoFailable Nonce nonce8 constant iv     | B.length constant /= 4 = CryptoFailed CryptoError_IvSizeInvalid-    | B.length iv       /= 8 = CryptoFailed CryptoError_IvSizeInvalid-    | otherwise              = CryptoPassed . Nonce8 . B.concat $ [constant, iv]+    | B.length iv /= 8 = CryptoFailed CryptoError_IvSizeInvalid+    | otherwise = CryptoPassed . Nonce8 . B.concat $ [constant, iv]  -- | 24 bytes IV, extended nonce constructor-nonce24 :: ByteArrayAccess ba-        => ba -> CryptoFailable XNonce+nonce24+    :: ByteArrayAccess ba+    => ba -> CryptoFailable XNonce nonce24 iv     | B.length iv /= 24 = CryptoFailed CryptoError_IvSizeInvalid-    | otherwise         = CryptoPassed . Nonce24 . B.convert $ iv+    | otherwise = CryptoPassed . Nonce24 . B.convert $ iv  -- | Increment a nonce incrementNonce :: Nonce -> Nonce-incrementNonce (Nonce8  n) = Nonce8  $ incrementNonce' n 4+incrementNonce (Nonce8 n) = Nonce8 $ incrementNonce' n 4 incrementNonce (Nonce12 n) = Nonce12 $ incrementNonce' n 0  incrementNonce' :: Bytes -> Int -> Bytes incrementNonce' b offset = B.copyAndFreeze b $ \s ->     loop s (s `plusPtr` offset)-    where-      loop :: Ptr Word8 -> Ptr Word8 -> IO ()-      loop s p-          | s == (p `plusPtr` (B.length b - offset - 1)) = peek s >>= poke s . (+) 1-          | otherwise = do-              r <- (+) 1 <$> peek p-              poke p r-              when (r == 0) $ loop s (p `plusPtr` 1)+  where+    loop :: Ptr Word8 -> Ptr Word8 -> IO ()+    loop s p+        | s == (p `plusPtr` (B.length b - offset - 1)) = peek s >>= poke s . (+) 1+        | otherwise = do+            r <- (+) 1 <$> peek p+            poke p r+            when (r == 0) $ loop s (p `plusPtr` 1)  -- | Initialize a new ChaChaPoly1305 State -- -- The key length need to be 256 bits, and the nonce -- procured using either `nonce8` or `nonce12`-initialize :: ByteArrayAccess key-           => key -> Nonce -> CryptoFailable State-initialize key (Nonce8  nonce) = initialize' key nonce-initialize key (Nonce12 nonce) = initialize' key nonce+-- | A ChaCha20Poly1305 key: thirty-two bytes, checked once here rather than+-- at every use, so that 'initialize' and 'initializeX' cannot fail.+newtype Key = Key ScrubbedBytes+    deriving (ByteArrayAccess, Eq, NFData) +-- | Take thirty-two bytes for a key.  A different length is reported as+-- 'CryptoError_KeySizeInvalid'; nothing else about a key can be wrong.+key :: ByteArrayAccess ba => ba -> CryptoFailable Key+key k+    | B.length k /= 32 = CryptoFailed CryptoError_KeySizeInvalid+    | otherwise = CryptoPassed $ Key $ B.convert k -initialize' :: ByteArrayAccess key-            => key -> Bytes -> CryptoFailable State-initialize' key nonce-    | B.length key /= 32 = CryptoFailed CryptoError_KeySizeInvalid-    | otherwise          = CryptoPassed $ initFromRootState rootState-  where rootState = ChaCha.initialize 20 key nonce+initialize :: Key -> Nonce -> State+initialize k (Nonce8 nonce) = initialize' k nonce+initialize k (Nonce12 nonce) = initialize' k nonce +initialize' :: Key -> Bytes -> State+initialize' k nonce = initFromRootState (ChaCha.initialize 20 k nonce)  initFromRootState :: ChaCha.State -> State initFromRootState rootState = State encState polyState 0 0   where     (polyKey, encState) = ChaCha.generate rootState 64-    polyState           = throwCryptoError $ Poly1305.initialize (B.take 32 polyKey :: ScrubbedBytes)-+    -- 64 bytes are generated so the ChaCha state advances a whole block, and+    -- the first 32 of them are the key, so there is no length left to check+    polyState = Poly1305.initialize (PolyKey.Key (B.take 32 polyKey))  -- | Initialize a new XChaChaPoly1305 State -- -- The key length needs to be 256 bits, and the nonce -- procured using `nonce24`.-initializeX :: ByteArrayAccess key-            => key -> XNonce -> CryptoFailable State-initializeX key (Nonce24 nonce)-    | B.length key /= 32 = CryptoFailed CryptoError_KeySizeInvalid-    | otherwise          = CryptoPassed $ initFromRootState rootState-  where rootState = ChaCha.initializeX 20 key nonce-+initializeX :: Key -> XNonce -> State+initializeX k (Nonce24 nonce) = initFromRootState (ChaCha.initializeX 20 k nonce)  -- | Append Authenticated Data to the State and return -- the new modified State.@@ -199,7 +223,7 @@     State encState newMacState newLength plainLength   where     newMacState = Poly1305.update macState ba-    newLength   = aadLength + fromIntegral (B.length ba)+    newLength = aadLength + fromIntegral (B.length ba)  -- | Finalize the Authenticated Data and return the finalized State finalizeAAD :: State -> State@@ -215,8 +239,8 @@     (output, State newEncState newMacState aadLength newPlainLength)   where     (output, newEncState) = ChaCha.combine encState input-    newMacState           = Poly1305.update macState output-    newPlainLength        = plainLength + fromIntegral (B.length input)+    newMacState = Poly1305.update macState output+    newPlainLength = plainLength + fromIntegral (B.length input)  -- | Decrypt a piece of data and returns the decrypted Data and the -- updated State.@@ -225,13 +249,32 @@     (output, State newEncState newMacState aadLength newPlainLength)   where     (output, newEncState) = ChaCha.combine encState input-    newMacState           = Poly1305.update macState input-    newPlainLength        = plainLength + fromIntegral (B.length input)+    newMacState = Poly1305.update macState input+    newPlainLength = plainLength + fromIntegral (B.length input)  -- | Generate an authentication tag from the State. finalize :: State -> Poly1305.Auth finalize (State _ macState aadLength plainLength) =-    Poly1305.finalize $ Poly1305.updates macState-        [ pad16 plainLength-        , either (error "finalize: internal error") id $ P.fill 16 (P.putStorable (toLE aadLength) >> P.putStorable (toLE plainLength))-        ]+    Poly1305.finalize $+        Poly1305.updates+            macState+            [ pad16 plainLength+            , either (error "finalize: internal error") id $+                P.fill 16 (P.putStorable (toLE aadLength) >> P.putStorable (toLE plainLength))+            ]++-- | Setting up AEAD for ChaCha20Poly1305.+aeadChacha20poly1305Init+    :: (ByteArrayAccess k, ByteArrayAccess n)+    => k -> n -> CryptoFailable (AEAD ChaCha20Poly1305)+aeadChacha20poly1305Init k nonce = do+    st0 <- initialize <$> key k <*> nonce12 nonce+    return $ AEAD model st0+  where+    model =+        AEADModeImpl+            { aeadImplAppendHeader = \st aad -> finalizeAAD $ appendAAD aad st+            , aeadImplEncrypt = \st plain -> encrypt plain st+            , aeadImplDecrypt = \st cipher -> decrypt cipher st+            , aeadImplFinalize = \st _ -> let Poly1305.Auth tag = finalize st in AuthTag tag+            }
Crypto/Cipher/DES.hs view
@@ -5,35 +5,34 @@ -- Stability   : stable -- Portability : good ---module Crypto.Cipher.DES-    ( DES-    ) where+-- DES, which is here because callers still meet it rather than because it+-- should be chosen: its 56-bit key is exhaustible.  Prefer "Crypto.Cipher.AES".+module Crypto.Cipher.DES (+    DES,+) where -import           Data.Word-import           Crypto.Error-import           Crypto.Cipher.Types-import           Crypto.Cipher.DES.Primitive-import           Crypto.Internal.ByteArray (ByteArrayAccess)+import Crypto.Cipher.DES.Primitive+import Crypto.Cipher.Types+import Crypto.Error+import Crypto.Internal.ByteArray (ByteArrayAccess) import qualified Crypto.Internal.ByteArray as B-import           Data.Memory.Endian  -- | DES Context-data DES = DES Word64+data DES = DES Schedule Schedule     deriving (Eq)  instance Cipher DES where-    cipherName    _ = "DES"+    cipherName _ = "DES"     cipherKeySize _ = KeySizeFixed 8-    cipherInit k    = initDES k+    cipherInit k = initDES k  instance BlockCipher DES where     blockSize _ = 8-    ecbEncrypt (DES key) = B.mapAsWord64 (unBlock . encrypt key . Block)-    ecbDecrypt (DES key) = B.mapAsWord64 (unBlock . decrypt key . Block)+    ecbEncrypt (DES enc _) = ecb enc+    ecbDecrypt (DES _ dec) = ecb dec  initDES :: ByteArrayAccess key => key -> CryptoFailable DES initDES k-    | len == 8  = CryptoPassed $ DES key-    | otherwise = CryptoFailed $ CryptoError_KeySizeInvalid-  where len = B.length k-        key = fromBE $ B.toW64BE k 0+    | B.length k == 8 =+        CryptoPassed $ DES (schedule [(Encrypt, k)]) (schedule [(Decrypt, k)])+    | otherwise = CryptoFailed CryptoError_KeySizeInvalid
Crypto/Cipher/DES/Primitive.hs view
@@ -1,223 +1,83 @@-{-# LANGUAGE FlexibleInstances #-}+{-# LANGUAGE ForeignFunctionInterface #-} ------------------------------------------------------------------------------ -- |--- Module      :  Crypto.Cipher.DES.Primitive--- License     :  BSD-style+-- Module      : Crypto.Cipher.DES.Primitive+-- License     : BSD-style+-- Stability   : experimental+-- Portability : Good ----- This module is copy of DES module from Crypto package.--- http://hackage.haskell.org/package/Crypto+-- The DES block operation, as FIPS 46-3 defines it, over the C in+-- @cbits/crypton_des.c@. -----------------------------------------------------------------------------------module Crypto.Cipher.DES.Primitive-    ( encrypt-    , decrypt-    , Block(..)-    ) where+-- A t'Schedule' holds the round keys of one or more stages in the order they+-- are applied, which is what lets single DES and the three stage constructions+-- share one entry point.+module Crypto.Cipher.DES.Primitive (+    Schedule,+    Direction (..),+    schedule,+    ecb,+) where +import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, Bytes)+import qualified Crypto.Internal.ByteArray as B+import Crypto.Internal.Compat (unsafeDoIO) import Data.Word-import Data.Bits---- | a DES block (64 bits)-newtype Block = Block { unBlock :: Word64 }--type Rotation = Int-type Key     = Word64--type Bits4  = [Bool]-type Bits6  = [Bool]-type Bits32 = [Bool]-type Bits48 = [Bool]-type Bits56 = [Bool]-type Bits64 = [Bool]--desXor :: [Bool] -> [Bool] -> [Bool]-desXor a b = zipWith (/=) a b--desRotate :: [Bool] -> Int -> [Bool]-desRotate bits rot = drop rot' bits ++ take rot' bits-  where rot' = rot `mod` length bits--bitify :: Word64 -> Bits64-bitify w = map (\b -> w .&. (shiftL 1 b) /= 0) [63,62..0]--unbitify :: Bits64 -> Word64-unbitify bs = foldl (\i b -> if b then 1 + shiftL i 1 else shiftL i 1) 0 bs--initial_permutation :: Bits64 -> Bits64-initial_permutation mb = map ((!!) mb) i- where i = [57, 49, 41, 33, 25, 17,  9, 1, 59, 51, 43, 35, 27, 19, 11, 3,-            61, 53, 45, 37, 29, 21, 13, 5, 63, 55, 47, 39, 31, 23, 15, 7,-            56, 48, 40, 32, 24, 16,  8, 0, 58, 50, 42, 34, 26, 18, 10, 2,-            60, 52, 44, 36, 28, 20, 12, 4, 62, 54, 46, 38, 30, 22, 14, 6]--{--"\x39\x31\x29\x21\x19\x11\x09\x01\x3b\x33\x2b\x23\x1b\x13\-\\x0b\x03\x3d\x35\x2d\x25\x1d\x15\x0d\x05\x3f\x37\x2f\x27\-\\x1f\x17\x0f\x07\x38\x30\x28\x20\x18\x10\x08\x00\x3a\x32\-\\x2a\x22\x1a\x12\x0a\x02\x3c\x34\x2c\x24\x1c\x14\x0c\x04\-\\x3e\x36\x2e\x26\x1e\x16\x0e\x06"--}--key_transformation :: Bits64 -> Bits56-key_transformation kb = map ((!!) kb) i- where i = [56, 48, 40, 32, 24, 16,  8,  0, 57, 49, 41, 33, 25, 17,-             9,  1, 58, 50, 42, 34, 26, 18, 10,  2, 59, 51, 43, 35,-            62, 54, 46, 38, 30, 22, 14,  6, 61, 53, 45, 37, 29, 21,-            13,  5, 60, 52, 44, 36, 28, 20, 12,  4, 27, 19, 11,  3]-{--"\x38\x30\x28\x20\x18\x10\x08\x00\x39\x31\x29\x21\x19\x11\-\\x09\x01\x3a\x32\x2a\x22\x1a\x12\x0a\x02\x3b\x33\x2b\x23\-\\x3e\x36\x2e\x26\x1e\x16\x0e\x06\x3d\x35\x2d\x25\x1d\x15\-\\x0d\x05\x3c\x34\x2c\x24\x1c\x14\x0c\x04\x1b\x13\x0b\x03"--}---des_enc :: Block -> Key -> Block-des_enc = do_des [1,2,4,6,8,10,12,14,15,17,19,21,23,25,27,28]--des_dec :: Block -> Key -> Block-des_dec = do_des [28,27,25,23,21,19,17,15,14,12,10,8,6,4,2,1]--do_des :: [Rotation] -> Block -> Key -> Block-do_des rots (Block m) k = Block $ des_work rots (takeDrop 32 mb) kb- where kb = key_transformation $ bitify k-       mb = initial_permutation $ bitify m--des_work :: [Rotation] -> (Bits32, Bits32) -> Bits56 -> Word64-des_work [] (ml, mr) _ = unbitify $ final_perm $ (mr ++ ml)-des_work (r:rs) mb kb = des_work rs mb' kb- where mb' = do_round r mb kb--do_round :: Rotation -> (Bits32, Bits32) -> Bits56 -> (Bits32, Bits32)-do_round r (ml, mr) kb = (mr, m')- where kb' = get_key kb r-       comp_kb = compression_permutation kb'-       expa_mr = expansion_permutation mr-       res = comp_kb `desXor` expa_mr-       res' = tail $ iterate (trans 6) ([], res)-       trans n (_, b) = (take n b, drop n b)-       res_s = concat $ zipWith (\f (x,_) -> f x) [s_box_1, s_box_2,-                                                   s_box_3, s_box_4,-                                                   s_box_5, s_box_6,-                                                   s_box_7, s_box_8] res'-       res_p = p_box res_s-       m' = res_p `desXor` ml--get_key :: Bits56 -> Rotation -> Bits56-get_key kb r = kb'- where (kl, kr) = takeDrop 28 kb-       kb' = desRotate kl r ++ desRotate kr r--compression_permutation :: Bits56 -> Bits48-compression_permutation kb = map ((!!) kb) i- where i = [13, 16, 10, 23,  0,  4,  2, 27, 14,  5, 20,  9,-            22, 18, 11,  3, 25,  7, 15,  6, 26, 19, 12,  1,-            40, 51, 30, 36, 46, 54, 29, 39, 50, 44, 32, 47,-            43, 48, 38, 55, 33, 52, 45, 41, 49, 35, 28, 31]--expansion_permutation :: Bits32 -> Bits48-expansion_permutation mb = map ((!!) mb) i- where i = [31,  0,  1,  2,  3,  4,  3,  4,  5,  6,  7,  8,-             7,  8,  9, 10, 11, 12, 11, 12, 13, 14, 15, 16,-            15, 16, 17, 18, 19, 20, 19, 20, 21, 22, 23, 24,-            23, 24, 25, 26, 27, 28, 27, 28, 29, 30, 31,  0]--s_box :: [[Word8]] -> Bits6 -> Bits4-s_box s [a,b,c,d,e,f] = to_bool 4 $ (s !! row) !! col- where row = sum $ zipWith numericise [a,f]     [1, 0]-       col = sum $ zipWith numericise [b,c,d,e] [3, 2, 1, 0]-       numericise :: Bool -> Int -> Int-       numericise = (\x y -> if x then 2^y else 0)--       to_bool :: Int -> Word8 -> [Bool]-       to_bool 0 _ = []-       to_bool n i = ((i .&. 8) == 8):to_bool (n-1) (shiftL i 1)-s_box _ _             = error "DES: internal error bits6 more than 6 elements"--s_box_1 :: Bits6 -> Bits4-s_box_1 = s_box i- where i = [[14,  4, 13,  1,  2, 15, 11,  8,  3, 10,  6, 12,  5,  9,  0,  7],-            [ 0, 15,  7,  4, 14,  2, 13,  1, 10,  6, 12, 11,  9,  5,  3,  8],-            [ 4,  1, 14,  8, 13,  6,  2, 11, 15, 12,  9,  7,  3, 10,  5,  0],-            [15, 12,  8,  2,  4,  9,  1,  7,  5, 11,  3, 14, 10,  0,  6, 13]]--s_box_2 :: Bits6 -> Bits4-s_box_2 = s_box i- where i = [[15,  1,  8, 14,  6, 11,  3,  4,  9,  7,  2, 13, 12,  0,  5, 10],-            [3,  13,  4,  7, 15,  2,  8, 14, 12,  0,  1, 10,  6,  9,  11, 5],-            [0,  14,  7, 11, 10,  4, 13,  1,  5,  8, 12,  6,  9,  3,  2, 15],-            [13,  8, 10,  1,  3, 15,  4,  2, 11,  6,  7, 12,  0,  5,  14, 9]]--s_box_3 :: Bits6 -> Bits4-s_box_3 = s_box i- where i = [[10,  0,  9, 14 , 6,  3, 15,  5,  1, 13, 12,  7, 11,  4,  2,  8],-            [13,  7,  0,  9,  3,  4,  6, 10,  2,  8,  5, 14, 12, 11, 15,  1],-            [13,  6,  4,  9,  8, 15,  3,  0, 11,  1,  2, 12,  5, 10, 14,  7],-            [1,  10, 13,  0,  6,  9,  8,  7,  4, 15, 14,  3, 11,  5,  2, 12]]--s_box_4 :: Bits6 -> Bits4-s_box_4 = s_box i- where i = [[7,  13, 14,  3,  0,  6,  9, 10,  1,  2,  8,  5, 11, 12,  4, 15],-            [13,  8, 11,  5,  6, 15,  0,  3,  4,  7,  2, 12,  1, 10, 14,  9],-            [10,  6,  9,  0, 12, 11,  7, 13, 15,  1,  3, 14,  5,  2,  8,  4],-            [3,  15,  0,  6, 10,  1, 13,  8,  9,  4,  5, 11, 12,  7,  2, 14]]--s_box_5 :: Bits6 -> Bits4-s_box_5 = s_box i- where i = [[2,  12,  4,  1,  7, 10, 11,  6,  8,  5,  3, 15, 13,  0, 14,  9],-            [14, 11,  2, 12,  4,  7, 13,  1,  5,  0, 15, 10,  3,  9,  8,  6],-            [4,   2,  1, 11, 10, 13,  7,  8, 15,  9, 12,  5,  6,  3,  0, 14],-            [11,  8, 12,  7,  1, 14,  2, 13,  6, 15,  0,  9, 10,  4,  5,  3]]--s_box_6 :: Bits6 -> Bits4-s_box_6 = s_box i- where i = [[12,  1, 10, 15,  9,  2,  6,  8,  0, 13,  3,  4, 14,  7,  5, 11],-            [10, 15,  4,  2,  7, 12,  9,  5,  6,  1, 13, 14,  0, 11,  3,  8],-            [9,  14, 15,  5,  2,  8, 12,  3,  7,  0,  4, 10,  1, 13, 11,  6],-            [4,  3,   2, 12,  9,  5, 15, 10, 11, 14,  1,  7,  6,  0,  8, 13]]+import Foreign.C.Types (CInt (..))+import Foreign.Ptr (Ptr, plusPtr) -s_box_7 :: Bits6 -> Bits4-s_box_7 = s_box i- where i = [[4,  11,  2, 14, 15,  0,  8, 13,  3, 12,  9,  7,  5, 10,  6,  1],-            [13, 0,  11,  7,  4,  9,  1, 10, 14,  3,  5, 12,  2, 15,  8,  6],-            [1,  4,  11, 13, 12,  3,  7, 14, 10, 15,  6,  8,  0,  5,  9,  2],-            [6,  11, 13,  8,  1,  4, 10,  7,  9,  5,  0, 15, 14,  2,  3, 12]]+-- | Which way a stage runs.+data Direction = Encrypt | Decrypt+    deriving (Show, Eq) -s_box_8 :: Bits6 -> Bits4-s_box_8 = s_box i- where i = [[13,  2,  8,  4,  6, 15, 11,  1, 10,  9,  3, 14,  5,  0, 12,  7],-            [1,  15, 13,  8, 10,  3,  7,  4, 12,  5,  6, 11,  0, 14,  9,  2],-            [7,  11,  4,  1,  9, 12, 14,  2,  0,  6, 10, 13, 15,  3,  5,  8],-            [2,   1, 14,  7,  4, 10,  8, 13, 15, 12,  9,  0,  3,  5,  6, 11]]+-- | The round keys of one or more stages, in the order they are applied.+newtype Schedule = Schedule Bytes+    deriving (Eq) -p_box :: Bits32 -> Bits32-p_box kb = map ((!!) kb) i- where i = [15, 6, 19, 20, 28, 11, 27, 16,  0, 14, 22, 25,  4, 17, 30,  9,-             1, 7, 23, 13, 31, 26,  2,  8, 18, 12, 29,  5, 21, 10,  3, 24]+-- | Bytes per stage: sixteen rounds of eight six-bit values.+stageSize :: Int+stageSize = 16 * 8 -final_perm :: Bits64 -> Bits64-final_perm kb = map ((!!) kb) i- where i = [39, 7, 47, 15, 55, 23, 63, 31, 38, 6, 46, 14, 54, 22, 62, 30,-            37, 5, 45, 13, 53, 21, 61, 29, 36, 4, 44, 12, 52, 20, 60, 28,-            35, 3, 43, 11, 51, 19, 59, 27, 34, 2, 42, 10, 50, 18, 58, 26,-            33, 1, 41,  9, 49, 17, 57, 25, 32, 0, 40 , 8, 48, 16, 56, 24]+-- | The block size DES works in.+blockBytes :: Int+blockBytes = 8 -takeDrop :: Int -> [a] -> ([a], [a])-takeDrop _ [] = ([], [])-takeDrop 0 xs = ([], xs)-takeDrop n (x:xs) = (x:ys, zs)- where (ys, zs) = takeDrop (n-1) xs+-- | Build the schedule for a sequence of stages, each an eight byte key and+-- the direction that stage runs in.  Shorter keys are rejected by the callers,+-- which know their own size; the bytes past the eighth are not read.+schedule :: ByteArrayAccess key => [(Direction, key)] -> Schedule+schedule stages =+    Schedule $ B.allocAndFreeze (stageSize * length stages) $ \dst ->+        mapM_ (uncurry (one dst)) (zip [0 ..] stages)+  where+    one dst i (dir, key) =+        B.withByteArray key $ \k ->+            c_des_init (dst `plusPtr` (i * stageSize)) k (reverseFlag dir)+    reverseFlag Encrypt = 0+    reverseFlag Decrypt = 1 +-- | Apply every stage of the schedule, in order, to each block of the input.+ecb :: ByteArray ba => Schedule -> ba -> ba+ecb (Schedule sched) input+    | len `mod` blockBytes /= 0 =+        error $+            "Crypto.Cipher.DES: input length must be a multiple of block size (8). Its length is: "+                ++ show len+    | otherwise = unsafeDoIO $+        B.alloc len $ \out ->+            B.withByteArray sched $ \ks ->+                B.withByteArray input $ \inp ->+                    c_des_ecb+                        out+                        ks+                        (fromIntegral (B.length sched `div` stageSize))+                        inp+                        (fromIntegral (len `div` blockBytes))+  where+    len = B.length input --- | Basic DES encryption which takes a key and a block of plaintext--- and returns the encrypted block of ciphertext according to the standard.-encrypt :: Word64 -> Block -> Block-encrypt = flip des_enc+foreign import ccall unsafe "crypton_des.h crypton_des_init"+    c_des_init :: Ptr Word8 -> Ptr Word8 -> CInt -> IO () --- | Basic DES decryption which takes a key and a block of ciphertext and--- returns the decrypted block of plaintext according to the standard.-decrypt :: Word64 -> Block -> Block-decrypt = flip des_dec+foreign import ccall unsafe "crypton_des.h crypton_des_ecb"+    c_des_ecb :: Ptr Word8 -> Ptr Word8 -> Word32 -> Ptr Word8 -> Word32 -> IO ()
Crypto/Cipher/RC4.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.Cipher.RC4 -- License     : BSD-style@@ -11,23 +14,24 @@ -- Initial FFI implementation by Peter White <peter@janrain.com> -- -- Reorganized and simplified to have an opaque context.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.Cipher.RC4-    ( initialize-    , combine-    , generate-    , State-    ) where+module Crypto.Cipher.RC4 (+    initialize,+    combine,+    generate,+    State,+) where -import           Data.Word-import           Foreign.Ptr-import           Crypto.Internal.ByteArray (ScrubbedBytes, ByteArray, ByteArrayAccess)+import Crypto.Internal.ByteArray (+    ByteArray,+    ByteArrayAccess,+    ScrubbedBytes,+ ) import qualified Crypto.Internal.ByteArray as B+import Data.Word+import Foreign.Ptr -import           Crypto.Internal.Compat-import           Crypto.Internal.Imports+import Crypto.Internal.Compat+import Crypto.Internal.Imports  -- | The encryption state for RC4 --@@ -36,29 +40,41 @@ -- and change in future versions.  The bytearray should not be used as input to -- cryptographic algorithms. newtype State = State ScrubbedBytes-    deriving (ByteArrayAccess,NFData)+    deriving (ByteArrayAccess, NFData)  -- | C Call for initializing the encryptor foreign import ccall unsafe "crypton_rc4.h crypton_rc4_init"-    c_rc4_init :: Ptr Word8 -- ^ The rc4 key-               -> Word32    -- ^ The key length-               -> Ptr State -- ^ The context-               -> IO ()+    c_rc4_init+        :: Ptr Word8+        -- ^ The rc4 key+        -> Word32+        -- ^ The key length+        -> Ptr State+        -- ^ The context+        -> IO ()  foreign import ccall unsafe "crypton_rc4.h crypton_rc4_combine"-    c_rc4_combine :: Ptr State        -- ^ Pointer to the permutation-                  -> Ptr Word8      -- ^ Pointer to the clear text-                  -> Word32         -- ^ Length of the clear text-                  -> Ptr Word8      -- ^ Output buffer-                  -> IO ()+    c_rc4_combine+        :: Ptr State+        -- ^ Pointer to the permutation+        -> Ptr Word8+        -- ^ Pointer to the clear text+        -> Word32+        -- ^ Length of the clear text+        -> Ptr Word8+        -- ^ Output buffer+        -> IO ()  -- | RC4 context initialization. -- -- seed the context with an initial key. the key size need to be -- adequate otherwise security takes a hit.-initialize :: ByteArrayAccess key-           => key   -- ^ The key-           -> State -- ^ The RC4 context with the key mixed in+initialize+    :: ByteArrayAccess key+    => key+    -- ^ The key+    -> State+    -- ^ The RC4 context with the key mixed in initialize key = unsafeDoIO $ do     st <- B.alloc 264 $ \stPtr ->         B.withByteArray key $ \keyPtr -> c_rc4_init keyPtr (fromIntegral $ B.length key) (castPtr stPtr)@@ -70,15 +86,27 @@ generate ctx len = combine ctx (B.zero len)  -- | RC4 xor combination of the rc4 stream with an input-combine :: ByteArray ba-        => State               -- ^ rc4 context-        -> ba                  -- ^ input-        -> (State, ba)         -- ^ new rc4 context, and the output+combine+    :: ByteArray ba+    => State+    -- ^ rc4 context+    -> ba+    -- ^ input+    -> (State, ba)+    -- ^ new rc4 context, and the output combine (State prevSt) clearText = unsafeDoIO $-    B.allocRet len            $ \outptr ->-    B.withByteArray clearText $ \clearPtr -> do-        st <- B.copy prevSt $ \stPtr ->-                c_rc4_combine (castPtr stPtr) clearPtr (fromIntegral len) outptr-        return $! State st-    --return $! (State st, B.PS outfptr 0 len)-  where len = B.length clearText+    B.allocRet len $ \outptr ->+        B.withByteArray clearText $ \clearPtr -> do+            st <- B.copy prevSt $ \stPtr ->+                -- in pieces the C's uint32_t length can hold; the state it+                -- keeps means it can simply be called again+                B.inCLengths len $ \off n ->+                    c_rc4_combine+                        (castPtr stPtr)+                        (clearPtr `plusPtr` off)+                        (fromIntegral n)+                        (outptr `plusPtr` off)+            return $! State st+  where+    -- return $! (State st, B.PS outfptr 0 len)+    len = B.length clearText
Crypto/Cipher/Salsa.hs view
@@ -1,25 +1,29 @@+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.Cipher.Salsa -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : stable -- Portability : good----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.Cipher.Salsa-    ( initialize-    , combine-    , generate-    , State(..)-    ) where+module Crypto.Cipher.Salsa (+    initialize,+    combine,+    generate,+    State (..),+) where -import           Crypto.Internal.ByteArray (ByteArrayAccess, ByteArray, ScrubbedBytes)+import Crypto.Internal.ByteArray (+    ByteArray,+    ByteArrayAccess,+    ScrubbedBytes,+ ) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Internal.Compat-import           Crypto.Internal.Imports-import           Foreign.Ptr-import           Foreign.C.Types+import Crypto.Internal.Compat+import Crypto.Internal.Imports+import Foreign.C.Types+import Foreign.Ptr  -- | Salsa context newtype State = State ScrubbedBytes@@ -27,54 +31,82 @@  -- | Initialize a new Salsa context with the number of rounds, -- the key and the nonce associated.-initialize :: (ByteArrayAccess key, ByteArrayAccess nonce)-           => Int    -- ^ number of rounds (8,12,20)-           -> key    -- ^ the key (128 or 256 bits)-           -> nonce  -- ^ the nonce (64 or 96 bits)-           -> State  -- ^ the initial Salsa state+initialize+    :: (ByteArrayAccess key, ByteArrayAccess nonce)+    => Int+    -- ^ number of rounds (8,12,20)+    -> key+    -- ^ the key (128 or 256 bits)+    -> nonce+    -- ^ the nonce (64 or 96 bits)+    -> State+    -- ^ the initial Salsa state initialize nbRounds key nonce-    | kLen `notElem` [16,32]          = error "Salsa: key length should be 128 or 256 bits"-    | nonceLen `notElem` [8,12]       = error "Salsa: nonce length should be 64 or 96 bits"-    | nbRounds `notElem` [8,12,20]    = error "Salsa: rounds should be 8, 12 or 20"+    | kLen `notElem` [16, 32] =+        error "Salsa: key length should be 128 or 256 bits"+    | nonceLen `notElem` [8, 12] =+        error "Salsa: nonce length should be 64 or 96 bits"+    | nbRounds `notElem` [8, 12, 20] = error "Salsa: rounds should be 8, 12 or 20"     | otherwise = unsafeDoIO $ do         stPtr <- B.alloc 132 $ \stPtr ->-            B.withByteArray nonce $ \noncePtr  ->-            B.withByteArray key   $ \keyPtr ->-                ccrypton_salsa_init stPtr nbRounds kLen keyPtr nonceLen noncePtr+            B.withByteArray nonce $ \noncePtr ->+                B.withByteArray key $ \keyPtr ->+                    ccrypton_salsa_init stPtr nbRounds kLen keyPtr nonceLen noncePtr         return $ State stPtr-  where kLen     = B.length key-        nonceLen = B.length nonce+  where+    kLen = B.length key+    nonceLen = B.length nonce  -- | Combine the salsa output and an arbitrary message with a xor, -- and return the combined output and the new state.-combine :: ByteArray ba-        => State      -- ^ the current Salsa state-        -> ba         -- ^ the source to xor with the generator-        -> (ba, State)+combine+    :: ByteArray ba+    => State+    -- ^ the current Salsa state+    -> ba+    -- ^ the source to xor with the generator+    -> (ba, State) combine prevSt@(State prevStMem) src     | B.null src = (B.empty, prevSt)-    | otherwise  = unsafeDoIO $ do+    | otherwise = unsafeDoIO $ do         (out, st) <- B.copyRet prevStMem $ \ctx ->-            B.alloc (B.length src) $ \dstPtr ->-            B.withByteArray src    $ \srcPtr -> do-                ccrypton_salsa_combine dstPtr ctx srcPtr (fromIntegral $ B.length src)+            B.alloc n $ \dstPtr ->+                B.withByteArray src $ \srcPtr ->+                    -- in pieces the C's uint32_t length can hold; it carries+                    -- the state in ctx, so it can simply be called again+                    B.inCLengths n $ \off len ->+                        ccrypton_salsa_combine+                            (dstPtr `plusPtr` off)+                            ctx+                            (srcPtr `plusPtr` off)+                            (fromIntegral len)         return (out, State st)+  where+    n = B.length src  -- | Generate a number of bytes from the Salsa output directly-generate :: ByteArray ba-         => State -- ^ the current Salsa state-         -> Int   -- ^ the length of data to generate-         -> (ba, State)+generate+    :: ByteArray ba+    => State+    -- ^ the current Salsa state+    -> Int+    -- ^ the length of data to generate+    -> (ba, State) generate prevSt@(State prevStMem) len-    | len <= 0  = (B.empty, prevSt)+    | len <= 0 = (B.empty, prevSt)     | otherwise = unsafeDoIO $ do         (out, st) <- B.copyRet prevStMem $ \ctx ->             B.alloc len $ \dstPtr ->-                ccrypton_salsa_generate dstPtr ctx (fromIntegral len)+                B.inCLengths len $ \off n ->+                    ccrypton_salsa_generate+                        (dstPtr `plusPtr` off)+                        ctx+                        (fromIntegral n)         return (out, State st)  foreign import ccall "crypton_salsa_init"-    ccrypton_salsa_init :: Ptr State -> Int -> Int -> Ptr Word8 -> Int -> Ptr Word8 -> IO ()+    ccrypton_salsa_init+        :: Ptr State -> Int -> Int -> Ptr Word8 -> Int -> Ptr Word8 -> IO ()  foreign import ccall "crypton_salsa_combine"     ccrypton_salsa_combine :: Ptr Word8 -> Ptr State -> Ptr Word8 -> CUInt -> IO ()
Crypto/Cipher/TripleDES.hs view
@@ -3,88 +3,114 @@ -- License     : BSD-style -- Stability   : experimental -- Portability : ???--module Crypto.Cipher.TripleDES-    ( DES_EEE3-    , DES_EDE3-    , DES_EEE2-    , DES_EDE2-    ) where+module Crypto.Cipher.TripleDES (+    DES_EEE3,+    DES_EDE3,+    DES_EEE2,+    DES_EDE2,+) where -import           Data.Word-import           Crypto.Error-import           Crypto.Cipher.Types-import           Crypto.Cipher.DES.Primitive-import           Crypto.Internal.ByteArray (ByteArrayAccess)+import Crypto.Cipher.DES.Primitive+import Crypto.Cipher.Types+import Crypto.Error+import Crypto.Internal.ByteArray (ByteArrayAccess, ScrubbedBytes) import qualified Crypto.Internal.ByteArray as B-import           Data.Memory.Endian  -- | 3DES with 3 different keys used all in the same direction-data DES_EEE3 = DES_EEE3 Word64 Word64 Word64+data DES_EEE3 = DES_EEE3 Schedule Schedule     deriving (Eq)  -- | 3DES with 3 different keys used in alternative direction-data DES_EDE3 = DES_EDE3 Word64 Word64 Word64 +data DES_EDE3 = DES_EDE3 Schedule Schedule     deriving (Eq)  -- | 3DES where the first and third keys are equal, used in the same direction-data DES_EEE2 = DES_EEE2 Word64 Word64 -- key1 and key3 are equal+data DES_EEE2 = DES_EEE2 Schedule Schedule     deriving (Eq)  -- | 3DES where the first and third keys are equal, used in alternative direction-data DES_EDE2 = DES_EDE2 Word64 Word64 -- key1 and key3 are equal+data DES_EDE2 = DES_EDE2 Schedule Schedule     deriving (Eq)  instance Cipher DES_EEE3 where-    cipherName    _ = "3DES_EEE"+    cipherName _ = "3DES_EEE"     cipherKeySize _ = KeySizeFixed 24-    cipherInit k    = init3DES DES_EEE3 k+    cipherInit k = init3DES DES_EEE3 Encrypt k  instance Cipher DES_EDE3 where-    cipherName    _ = "3DES_EDE"+    cipherName _ = "3DES_EDE"     cipherKeySize _ = KeySizeFixed 24-    cipherInit k    = init3DES DES_EDE3 k+    cipherInit k = init3DES DES_EDE3 Decrypt k  instance Cipher DES_EDE2 where-    cipherName    _ = "2DES_EDE"+    cipherName _ = "2DES_EDE"     cipherKeySize _ = KeySizeFixed 16-    cipherInit k    = init2DES DES_EDE2 k+    cipherInit k = init2DES DES_EDE2 Decrypt k  instance Cipher DES_EEE2 where-    cipherName    _ = "2DES_EEE"+    cipherName _ = "2DES_EEE"     cipherKeySize _ = KeySizeFixed 16-    cipherInit k    = init2DES DES_EEE2 k+    cipherInit k = init2DES DES_EEE2 Encrypt k  instance BlockCipher DES_EEE3 where     blockSize _ = 8-    ecbEncrypt (DES_EEE3 k1 k2 k3) = B.mapAsWord64 (unBlock . (encrypt k3 . encrypt k2 . encrypt k1) . Block)-    ecbDecrypt (DES_EEE3 k1 k2 k3) = B.mapAsWord64 (unBlock . (decrypt k1 . decrypt k2 . decrypt k3) . Block)+    ecbEncrypt (DES_EEE3 enc _) = ecb enc+    ecbDecrypt (DES_EEE3 _ dec) = ecb dec  instance BlockCipher DES_EDE3 where     blockSize _ = 8-    ecbEncrypt (DES_EDE3 k1 k2 k3) = B.mapAsWord64 (unBlock . (encrypt k3 . decrypt k2 . encrypt k1) . Block)-    ecbDecrypt (DES_EDE3 k1 k2 k3) = B.mapAsWord64 (unBlock . (decrypt k1 . encrypt k2 . decrypt k3) . Block)+    ecbEncrypt (DES_EDE3 enc _) = ecb enc+    ecbDecrypt (DES_EDE3 _ dec) = ecb dec  instance BlockCipher DES_EEE2 where     blockSize _ = 8-    ecbEncrypt (DES_EEE2 k1 k2) = B.mapAsWord64 (unBlock . (encrypt k1 . encrypt k2 . encrypt k1) . Block)-    ecbDecrypt (DES_EEE2 k1 k2) = B.mapAsWord64 (unBlock . (decrypt k1 . decrypt k2 . decrypt k1) . Block)+    ecbEncrypt (DES_EEE2 enc _) = ecb enc+    ecbDecrypt (DES_EEE2 _ dec) = ecb dec  instance BlockCipher DES_EDE2 where     blockSize _ = 8-    ecbEncrypt (DES_EDE2 k1 k2) = B.mapAsWord64 (unBlock . (encrypt k1 . decrypt k2 . encrypt k1) . Block)-    ecbDecrypt (DES_EDE2 k1 k2) = B.mapAsWord64 (unBlock . (decrypt k1 . encrypt k2 . decrypt k1) . Block)+    ecbEncrypt (DES_EDE2 enc _) = ecb enc+    ecbDecrypt (DES_EDE2 _ dec) = ecb dec -init3DES :: ByteArrayAccess key => (Word64 -> Word64 -> Word64 -> a) -> key -> CryptoFailable a-init3DES constr k-    | len == 24 = CryptoPassed $ constr k1 k2 k3+-- | The schedules of a three stage cipher, for both directions.+--+-- The outer stages encrypt and the middle one goes whichever way the+-- construction says; decrypting is the same three stages in the opposite+-- order, each the other way round.+stages+    :: ByteArrayAccess key+    => Direction+    -- ^ the direction of the middle stage when encrypting+    -> (key, key, key)+    -> (Schedule, Schedule)+stages mid (k1, k2, k3) =+    ( schedule [(Encrypt, k1), (mid, k2), (Encrypt, k3)]+    , schedule [(Decrypt, k3), (opposite mid, k2), (Decrypt, k1)]+    )+  where+    opposite Encrypt = Decrypt+    opposite Decrypt = Encrypt++init3DES+    :: ByteArrayAccess key+    => (Schedule -> Schedule -> a) -> Direction -> key -> CryptoFailable a+init3DES constr mid k+    | B.length k == 24 =+        CryptoPassed $ uncurry constr $ stages mid (part 0, part 8, part 16)     | otherwise = CryptoFailed CryptoError_KeySizeInvalid-  where len = B.length k-        (k1, k2, k3) = (fromBE $ B.toW64BE k 0, fromBE $ B.toW64BE k 8, fromBE $ B.toW64BE k 16)+  where+    part = keyPart k -init2DES :: ByteArrayAccess key => (Word64 -> Word64 -> a) -> key -> CryptoFailable a-init2DES constr k-    | len == 16 = CryptoPassed $ constr k1 k2+init2DES+    :: ByteArrayAccess key+    => (Schedule -> Schedule -> a) -> Direction -> key -> CryptoFailable a+init2DES constr mid k+    | B.length k == 16 =+        CryptoPassed $ uncurry constr $ stages mid (part 0, part 8, part 0)     | otherwise = CryptoFailed CryptoError_KeySizeInvalid-  where len = B.length k-        (k1, k2) = (fromBE $ B.toW64BE k 0, fromBE $ B.toW64BE k 8)+  where+    part = keyPart k++-- | The eight bytes of a key that start at the given offset.+keyPart :: ByteArrayAccess key => key -> Int -> ScrubbedBytes+keyPart k i = B.take 8 $ B.drop i (B.convert k :: ScrubbedBytes)
Crypto/Cipher/Twofish.hs view
@@ -1,8 +1,8 @@-module Crypto.Cipher.Twofish-    ( Twofish128-    , Twofish192-    , Twofish256-    ) where+module Crypto.Cipher.Twofish (+    Twofish128,+    Twofish192,+    Twofish256,+) where  import Crypto.Cipher.Twofish.Primitive import Crypto.Cipher.Types@@ -11,35 +11,38 @@ newtype Twofish128 = Twofish128 Twofish  instance Cipher Twofish128 where-    cipherName    _ = "Twofish128"+    cipherName _ = "Twofish128"     cipherKeySize _ = KeySizeFixed 16-    cipherInit key  = Twofish128 <$> (initTwofish =<< validateKeySize (undefined :: Twofish128) key)+    cipherInit key =+        Twofish128 <$> (initTwofish =<< validateKeySize (undefined :: Twofish128) key)  instance BlockCipher Twofish128 where-    blockSize                 _ = 16+    blockSize _ = 16     ecbEncrypt (Twofish128 key) = encrypt key     ecbDecrypt (Twofish128 key) = decrypt key  newtype Twofish192 = Twofish192 Twofish  instance Cipher Twofish192 where-    cipherName    _ = "Twofish192"+    cipherName _ = "Twofish192"     cipherKeySize _ = KeySizeFixed 24-    cipherInit key  = Twofish192 <$> (initTwofish =<< validateKeySize (undefined :: Twofish192) key)+    cipherInit key =+        Twofish192 <$> (initTwofish =<< validateKeySize (undefined :: Twofish192) key)  instance BlockCipher Twofish192 where-    blockSize                 _ = 16+    blockSize _ = 16     ecbEncrypt (Twofish192 key) = encrypt key     ecbDecrypt (Twofish192 key) = decrypt key  newtype Twofish256 = Twofish256 Twofish  instance Cipher Twofish256 where-    cipherName    _ = "Twofish256"+    cipherName _ = "Twofish256"     cipherKeySize _ = KeySizeFixed 32-    cipherInit key  = Twofish256 <$> (initTwofish =<< validateKeySize (undefined :: Twofish256) key)+    cipherInit key =+        Twofish256 <$> (initTwofish =<< validateKeySize (undefined :: Twofish256) key)  instance BlockCipher Twofish256 where-    blockSize                 _ = 16+    blockSize _ = 16     ecbEncrypt (Twofish256 key) = encrypt key     ecbDecrypt (Twofish256 key) = decrypt key
Crypto/Cipher/Twofish/Primitive.hs view
@@ -1,39 +1,46 @@-{-# LANGUAGE MagicHash #-} {-# LANGUAGE BangPatterns #-}-module Crypto.Cipher.Twofish.Primitive-    ( Twofish-    , initTwofish-    , encrypt-    , decrypt-    ) where+{-# LANGUAGE MagicHash #-}+{-# OPTIONS_GHC -Wno-incomplete-uni-patterns #-} -import           Crypto.Error-import           Crypto.Internal.ByteArray (ByteArray)+module Crypto.Cipher.Twofish.Primitive (+    Twofish,+    initTwofish,+    encrypt,+    decrypt,+) where++import Crypto.Error+import Crypto.Internal.ByteArray (ByteArray) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Internal.WordArray-import           Data.Word-import           Data.Bits-import           Data.List+import Crypto.Internal.WordArray+import Crypto.Internal.Words (Word128 (..))+import Data.Bits+import Data.List (foldl')+import Data.Word+import Prelude hiding (foldl')  -- Based on the Golang referance implementation -- https://github.com/golang/crypto/blob/master/twofish/twofish.go - -- BlockSize is the constant block size of Twofish. blockSize :: Int blockSize = 16  mdsPolynomial, rsPolynomial :: Word32 mdsPolynomial = 0x169 -- x^8 + x^6 + x^5 + x^3 + 1, see [TWOFISH] 4.2-rsPolynomial = 0x14d  -- x^8 + x^6 + x^3 + x^2 + 1, see [TWOFISH] 4.3+rsPolynomial = 0x14d -- x^8 + x^6 + x^3 + x^2 + 1, see [TWOFISH] 4.3 -data Twofish = Twofish { s :: (Array32, Array32, Array32, Array32)-                       , k :: Array32 }+data Twofish = Twofish+    { s :: (Array32, Array32, Array32, Array32)+    , k :: Array32+    }  data ByteSize = Bytes16 | Bytes24 | Bytes32 deriving (Eq) -data KeyPackage ba = KeyPackage { rawKeyBytes :: ba-                                , byteSize :: ByteSize }+data KeyPackage ba = KeyPackage+    { rawKeyBytes :: ba+    , byteSize :: ByteSize+    }  buildPackage :: ByteArray ba => ba -> Maybe (KeyPackage ba) buildPackage key@@ -46,89 +53,180 @@ -- -- Return the initialized key or a error message if the given -- keyseed was not 16-bytes in length.-initTwofish :: ByteArray key-            => key -- ^ The key to create the twofish context-            -> CryptoFailable Twofish+initTwofish+    :: ByteArray key+    => key+    -- ^ The key to create the twofish context+    -> CryptoFailable Twofish initTwofish key =-    case buildPackage key of Nothing -> CryptoFailed CryptoError_KeySizeInvalid-                             Just keyPackage -> CryptoPassed Twofish { k = generatedK, s = generatedS }-                                  where generatedK = array32 40 $ genK keyPackage-                                        generatedS = genSboxes keyPackage $ sWords key+    case buildPackage key of+        Nothing -> CryptoFailed CryptoError_KeySizeInvalid+        Just keyPackage -> CryptoPassed Twofish{k = generatedK, s = generatedS}+          where+            generatedK = array32 40 $ genK keyPackage+            generatedS = genSboxes keyPackage $ sWords key -mapBlocks :: ByteArray ba => (ba -> ba) -> ba -> ba+-- | Run a block operation over every block of the input.+--+-- 'B.mapAsWord128' walks the input and the output once each, where taking a+-- block off the front and appending the result copied the whole of both, once+-- per block.+mapBlocks :: ByteArray ba => (Word128 -> Word128) -> ba -> ba mapBlocks operation input-    | B.null rest = blockOutput-    | otherwise = blockOutput `B.append` mapBlocks operation rest-        where (block, rest) = B.splitAt blockSize input-              blockOutput = operation block+    | B.length input `mod` blockSize /= 0 =+        error $+            "Crypto.Cipher.Twofish: input length must be a multiple of block size (16). Its length is: "+                ++ show (B.length input)+    | otherwise = B.mapAsWord128 operation input +-- | The four little-endian words of a block, from the two big-endian words+-- t'Word128' is read as.+load32ls :: Word128 -> (Word32, Word32, Word32, Word32)+load32ls (Word128 hi lo) =+    ( byteSwap32 (fromIntegral (hi `shiftR` 32))+    , byteSwap32 (fromIntegral hi)+    , byteSwap32 (fromIntegral (lo `shiftR` 32))+    , byteSwap32 (fromIntegral lo)+    )++store32ls :: (Word32, Word32, Word32, Word32) -> Word128+store32ls (a, b, c, d) = Word128 (pair a b) (pair c d)+  where+    pair x y =+        (fromIntegral (byteSwap32 x) `shiftL` 32) .|. fromIntegral (byteSwap32 y)+ -- | Encrypts the given ByteString using the given Key-encrypt :: ByteArray ba-        => Twofish     -- ^ The key to use-        -> ba           -- ^ The data to encrypt-        -> ba+encrypt+    :: ByteArray ba+    => Twofish+    -- ^ The key to use+    -> ba+    -- ^ The data to encrypt+    -> ba encrypt cipher = mapBlocks (encryptBlock cipher) -encryptBlock :: ByteArray ba => Twofish -> ba -> ba-encryptBlock Twofish { s = (s1, s2, s3, s4), k = ks } message = store32ls ts-    where (a, b, c, d) = load32ls message-          a' = a `xor` arrayRead32 ks 0-          b' = b `xor` arrayRead32 ks 1-          c' = c `xor` arrayRead32 ks 2-          d' = d `xor` arrayRead32 ks 3-          (!a'', !b'', !c'', !d'') = foldl' shuffle (a', b', c', d') [0..7]-          ts = (c'' `xor` arrayRead32 ks 4, d'' `xor` arrayRead32 ks 5, a'' `xor` arrayRead32 ks 6, b'' `xor` arrayRead32 ks 7)+encryptBlock :: Twofish -> Word128 -> Word128+encryptBlock Twofish{s = (s1, s2, s3, s4), k = ks} message = store32ls ts+  where+    (a, b, c, d) = load32ls message+    a' = a `xor` arrayRead32 ks 0+    b' = b `xor` arrayRead32 ks 1+    c' = c `xor` arrayRead32 ks 2+    d' = d `xor` arrayRead32 ks 3+    (!a'', !b'', !c'', !d'') = foldl' shuffle (a', b', c', d') [0 .. 7]+    ts =+        ( c'' `xor` arrayRead32 ks 4+        , d'' `xor` arrayRead32 ks 5+        , a'' `xor` arrayRead32 ks 6+        , b'' `xor` arrayRead32 ks 7+        ) -          shuffle :: (Word32, Word32, Word32, Word32) -> Int -> (Word32, Word32, Word32, Word32)-          shuffle (!retA, !retB, !retC, !retD) ind = (retA', retB', retC', retD')-            where [k0, k1, k2, k3] = fmap (\offset -> arrayRead32 ks $ (8 + 4 * ind) + offset) [0..3]-                  t2 = byteIndex s2 retB `xor` byteIndex s3 (shiftR retB 8) `xor` byteIndex s4 (shiftR retB 16) `xor` byteIndex s1 (shiftR retB 24)-                  t1 = (byteIndex s1 retA `xor` byteIndex s2 (shiftR retA 8) `xor` byteIndex s3 (shiftR retA 16) `xor` byteIndex s4 (shiftR retA 24)) + t2-                  retC' = rotateR (retC `xor` (t1 + k0)) 1-                  retD' = rotateL retD 1 `xor` (t1 + t2 + k1)-                  t2' = byteIndex s2 retD' `xor` byteIndex s3 (shiftR retD' 8) `xor` byteIndex s4 (shiftR retD' 16) `xor` byteIndex s1 (shiftR retD' 24)-                  t1' = (byteIndex s1 retC' `xor` byteIndex s2 (shiftR retC' 8) `xor` byteIndex s3 (shiftR retC' 16) `xor` byteIndex s4 (shiftR retC' 24)) + t2'-                  retA' = rotateR (retA `xor` (t1' + k2)) 1-                  retB' = rotateL retB 1 `xor` (t1' + t2' + k3)+    shuffle+        :: (Word32, Word32, Word32, Word32) -> Int -> (Word32, Word32, Word32, Word32)+    shuffle (!retA, !retB, !retC, !retD) ind = (retA', retB', retC', retD')+      where+        [k0, k1, k2, k3] = fmap (\offset -> arrayRead32 ks $ (8 + 4 * ind) + offset) [0 .. 3]+        t2 =+            byteIndex s2 retB+                `xor` byteIndex s3 (shiftR retB 8)+                `xor` byteIndex s4 (shiftR retB 16)+                `xor` byteIndex s1 (shiftR retB 24)+        t1 =+            ( byteIndex s1 retA+                `xor` byteIndex s2 (shiftR retA 8)+                `xor` byteIndex s3 (shiftR retA 16)+                `xor` byteIndex s4 (shiftR retA 24)+            )+                + t2+        retC' = rotateR (retC `xor` (t1 + k0)) 1+        retD' = rotateL retD 1 `xor` (t1 + t2 + k1)+        t2' =+            byteIndex s2 retD'+                `xor` byteIndex s3 (shiftR retD' 8)+                `xor` byteIndex s4 (shiftR retD' 16)+                `xor` byteIndex s1 (shiftR retD' 24)+        t1' =+            ( byteIndex s1 retC'+                `xor` byteIndex s2 (shiftR retC' 8)+                `xor` byteIndex s3 (shiftR retC' 16)+                `xor` byteIndex s4 (shiftR retC' 24)+            )+                + t2'+        retA' = rotateR (retA `xor` (t1' + k2)) 1+        retB' = rotateL retB 1 `xor` (t1' + t2' + k3)  -- Unsafe, no bounds checking byteIndex :: Array32 -> Word32 -> Word32-byteIndex xs ind  = arrayRead32 xs $ fromIntegral byte-    where byte = ind `mod` 256+byteIndex xs ind = arrayRead32 xs $ fromIntegral byte+  where+    byte = ind `mod` 256  -- | Decrypts the given ByteString using the given Key-decrypt :: ByteArray ba-        => Twofish     -- ^ The key to use-        -> ba           -- ^ The data to decrypt-        -> ba+decrypt+    :: ByteArray ba+    => Twofish+    -- ^ The key to use+    -> ba+    -- ^ The data to decrypt+    -> ba decrypt cipher = mapBlocks (decryptBlock cipher)  {- decryption for 128 bits blocks -}-decryptBlock :: ByteArray ba => Twofish -> ba -> ba-decryptBlock Twofish { s = (s1, s2, s3, s4), k = ks } message = store32ls ixs-    where (a, b, c, d) = load32ls message-          a' = c `xor` arrayRead32 ks 6-          b' = d `xor` arrayRead32 ks 7-          c' = a `xor` arrayRead32 ks 4-          d' = b `xor` arrayRead32 ks 5-          (!a'', !b'', !c'', !d'') = foldl' unshuffle (a', b', c', d') [8, 7..1]-          ixs = (a'' `xor` arrayRead32 ks 0, b'' `xor` arrayRead32 ks 1, c'' `xor` arrayRead32 ks 2, d'' `xor` arrayRead32 ks 3)+decryptBlock :: Twofish -> Word128 -> Word128+decryptBlock Twofish{s = (s1, s2, s3, s4), k = ks} message = store32ls ixs+  where+    (a, b, c, d) = load32ls message+    a' = c `xor` arrayRead32 ks 6+    b' = d `xor` arrayRead32 ks 7+    c' = a `xor` arrayRead32 ks 4+    d' = b `xor` arrayRead32 ks 5+    (!a'', !b'', !c'', !d'') = foldl' unshuffle (a', b', c', d') [8, 7 .. 1]+    ixs =+        ( a'' `xor` arrayRead32 ks 0+        , b'' `xor` arrayRead32 ks 1+        , c'' `xor` arrayRead32 ks 2+        , d'' `xor` arrayRead32 ks 3+        ) -          unshuffle :: (Word32, Word32, Word32, Word32) -> Int -> (Word32, Word32, Word32, Word32)-          unshuffle (!retA, !retB, !retC, !retD) ind = (retA', retB', retC', retD')-            where [k0, k1, k2, k3] = fmap (\offset -> arrayRead32 ks $ (4 + 4 * ind) + offset) [0..3]-                  t2 = byteIndex s2 retD `xor` byteIndex s3 (shiftR retD 8) `xor` byteIndex s4 (shiftR retD 16) `xor` byteIndex s1 (shiftR retD 24)-                  t1 = (byteIndex s1 retC `xor` byteIndex s2 (shiftR retC 8) `xor` byteIndex s3 (shiftR retC 16) `xor` byteIndex s4 (shiftR retC 24)) + t2-                  retA' = rotateL retA 1 `xor` (t1 + k2)-                  retB' = rotateR (retB `xor` (t2 + t1 + k3)) 1-                  t2' = byteIndex s2 retB' `xor` byteIndex s3 (shiftR retB' 8) `xor` byteIndex s4 (shiftR retB' 16) `xor` byteIndex s1 (shiftR retB' 24)-                  t1' = (byteIndex s1 retA' `xor` byteIndex s2 (shiftR retA' 8) `xor` byteIndex s3 (shiftR retA' 16) `xor` byteIndex s4 (shiftR retA' 24)) + t2'-                  retC' = rotateL retC 1 `xor` (t1' + k0)-                  retD' = rotateR (retD `xor` (t2' + t1' + k1)) 1+    unshuffle+        :: (Word32, Word32, Word32, Word32) -> Int -> (Word32, Word32, Word32, Word32)+    unshuffle (!retA, !retB, !retC, !retD) ind = (retA', retB', retC', retD')+      where+        [k0, k1, k2, k3] = fmap (\offset -> arrayRead32 ks $ (4 + 4 * ind) + offset) [0 .. 3]+        t2 =+            byteIndex s2 retD+                `xor` byteIndex s3 (shiftR retD 8)+                `xor` byteIndex s4 (shiftR retD 16)+                `xor` byteIndex s1 (shiftR retD 24)+        t1 =+            ( byteIndex s1 retC+                `xor` byteIndex s2 (shiftR retC 8)+                `xor` byteIndex s3 (shiftR retC 16)+                `xor` byteIndex s4 (shiftR retC 24)+            )+                + t2+        retA' = rotateL retA 1 `xor` (t1 + k2)+        retB' = rotateR (retB `xor` (t2 + t1 + k3)) 1+        t2' =+            byteIndex s2 retB'+                `xor` byteIndex s3 (shiftR retB' 8)+                `xor` byteIndex s4 (shiftR retB' 16)+                `xor` byteIndex s1 (shiftR retB' 24)+        t1' =+            ( byteIndex s1 retA'+                `xor` byteIndex s2 (shiftR retA' 8)+                `xor` byteIndex s3 (shiftR retA' 16)+                `xor` byteIndex s4 (shiftR retA' 24)+            )+                + t2'+        retC' = rotateL retC 1 `xor` (t1' + k0)+        retD' = rotateR (retD `xor` (t2' + t1' + k1)) 1  sbox0 :: Int -> Word8 sbox0 = arrayRead8 t-    where t = array8+  where+    t =+        array8             "\xa9\x67\xb3\xe8\x04\xfd\xa3\x76\x9a\x92\x80\x78\xe4\xdd\xd1\x38\             \\x0d\xc6\x35\x98\x18\xf7\xec\x6c\x43\x75\x37\x26\xfa\x13\x94\x48\             \\xf2\xd0\x8b\x30\x84\x54\xdf\x23\x19\x5b\x3d\x59\xf3\xae\xa2\x82\@@ -148,7 +246,9 @@  sbox1 :: Int -> Word8 sbox1 = arrayRead8 t-    where t = array8+  where+    t =+        array8             "\x75\xf3\xc6\xf4\xdb\x7b\xfb\xc8\x4a\xd3\xe6\x6b\x45\x7d\xe8\x4b\             \\xd6\x32\xd8\xfd\x37\x71\xf1\xe1\x30\x0f\xf8\x1b\x87\xfa\x06\x3f\             \\x5e\xba\xae\x5b\x8a\x00\xbc\x9d\x6d\xc1\xb1\x0e\x80\x5d\xd2\xd5\@@ -167,145 +267,261 @@             \\xd7\x61\x1e\xb4\x50\x04\xf6\xc2\x16\x25\x86\x56\x55\x09\xbe\x91"#  rs :: [[Word8]]-rs = [ [0x01, 0xA4, 0x55, 0x87, 0x5A, 0x58, 0xDB, 0x9E]-     , [0xA4, 0x56, 0x82, 0xF3, 0x1E, 0xC6, 0x68, 0xE5]-     , [0x02, 0xA1, 0xFC, 0xC1, 0x47, 0xAE, 0x3D, 0x19]-     , [0xA4, 0x55, 0x87, 0x5A, 0x58, 0xDB, 0x9E, 0x03] ]----load32ls :: ByteArray ba => ba -> (Word32, Word32, Word32, Word32)-load32ls message = (intify q1, intify q2, intify q3, intify q4)-    where (half1, half2) = B.splitAt 8 message-          (q1, q2) = B.splitAt 4 half1-          (q3, q4) = B.splitAt 4 half2--          intify :: ByteArray ba => ba -> Word32-          intify bytes = foldl' (\int (!word, !ind) -> int .|. shiftL (fromIntegral word) (ind * 8) ) 0 (zip (B.unpack bytes) [0..])--store32ls :: ByteArray ba => (Word32, Word32, Word32, Word32) -> ba-store32ls (a, b, c, d) = B.pack $ concatMap splitWordl [a, b, c, d]-    where splitWordl :: Word32 -> [Word8]-          splitWordl w = fmap (\ind -> fromIntegral $ shiftR w (8 * ind)) [0..3]-+rs =+    [ [0x01, 0xA4, 0x55, 0x87, 0x5A, 0x58, 0xDB, 0x9E]+    , [0xA4, 0x56, 0x82, 0xF3, 0x1E, 0xC6, 0x68, 0xE5]+    , [0x02, 0xA1, 0xFC, 0xC1, 0x47, 0xAE, 0x3D, 0x19]+    , [0xA4, 0x55, 0x87, 0x5A, 0x58, 0xDB, 0x9E, 0x03]+    ]  -- Create S words sWords :: ByteArray ba => ba -> [Word8] sWords key = sWord-    where word64Count = B.length key `div` 2-          sWord = concatMap (\wordIndex ->-                        map (\rsRow ->-                            foldl' (\acc (!rsVal, !colIndex) ->+  where+    word64Count = B.length key `div` 2+    sWord =+        concatMap+            ( \wordIndex ->+                map+                    ( \rsRow ->+                        foldl'+                            ( \acc (!rsVal, !colIndex) ->                                 acc `xor` gfMult rsPolynomial (B.index key $ 8 * wordIndex + colIndex) rsVal-                                ) 0 (zip rsRow [0..])-                            ) rs-                    ) [0..word64Count - 1]+                            )+                            0+                            (zip rsRow [0 ..])+                    )+                    rs+            )+            [0 .. word64Count - 1]  data Column = Zero | One | Two | Three deriving (Show, Eq, Enum, Bounded)  genSboxes :: KeyPackage ba -> [Word8] -> (Array32, Array32, Array32, Array32) genSboxes keyPackage ws = (mkArray b0', mkArray b1', mkArray b2', mkArray b3')-    where range = [0..255]-          mkArray = array32 256-          [w0, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, w11, w12, w13, w14, w15] = take 16 ws-          (b0', b1', b2', b3') = sboxBySize $ byteSize keyPackage--          sboxBySize :: ByteSize -> ([Word32], [Word32], [Word32], [Word32])-          sboxBySize Bytes16 = (b0, b1, b2, b3)-            where !b0 = fmap mapper range-                    where mapper :: Int -> Word32-                          mapper byte = mdsColumnMult ((sbox1 . fromIntegral) ((sbox0 . fromIntegral $ sbox0 byte `xor` w0) `xor` w4)) Zero-                  !b1 = fmap mapper range-                    where mapper byte = mdsColumnMult ((sbox0 . fromIntegral) ((sbox0 . fromIntegral $ sbox1 byte `xor` w1) `xor` w5)) One-                  !b2 = fmap mapper range-                    where mapper byte = mdsColumnMult ((sbox1 . fromIntegral) ((sbox1 . fromIntegral $ sbox0 byte `xor` w2) `xor` w6)) Two-                  !b3 = fmap mapper range-                    where mapper byte = mdsColumnMult ((sbox0 . fromIntegral) ((sbox1 . fromIntegral $ sbox1 byte `xor` w3) `xor` w7)) Three--          sboxBySize Bytes24 = (b0, b1, b2, b3)-            where !b0 = fmap mapper range-                    where mapper byte = mdsColumnMult ((sbox1 . fromIntegral) ((sbox0 . fromIntegral) ((sbox0 . fromIntegral $ sbox1 byte `xor` w0) `xor` w4) `xor` w8)) Zero-                  !b1 = fmap mapper range-                    where mapper byte = mdsColumnMult ((sbox0 . fromIntegral) ((sbox0 . fromIntegral) ((sbox1 . fromIntegral $ sbox1 byte `xor` w1) `xor` w5) `xor` w9)) One-                  !b2 = fmap mapper range-                    where mapper byte = mdsColumnMult ((sbox1 . fromIntegral) ((sbox1 . fromIntegral) ((sbox0 . fromIntegral $ sbox0 byte `xor` w2) `xor` w6) `xor` w10)) Two-                  !b3 = fmap mapper range-                    where mapper byte = mdsColumnMult ((sbox0 . fromIntegral) ((sbox1 . fromIntegral) ((sbox1 . fromIntegral $ sbox0 byte `xor` w3) `xor` w7) `xor` w11)) Three--          sboxBySize Bytes32 = (b0, b1, b2, b3)-            where !b0 = fmap mapper range-                    where mapper byte = mdsColumnMult ((sbox1 . fromIntegral) ((sbox0 . fromIntegral) ((sbox0 . fromIntegral) ((sbox1 . fromIntegral $ sbox1 byte `xor` w0) `xor` w4) `xor` w8) `xor` w12)) Zero-                  !b1 = fmap mapper range-                    where mapper byte = mdsColumnMult ((sbox0 . fromIntegral) ((sbox0 . fromIntegral) ((sbox1 . fromIntegral) ((sbox1 . fromIntegral $ sbox0 byte `xor` w1) `xor` w5) `xor` w9) `xor` w13)) One-                  !b2 = fmap mapper range-                    where mapper byte = mdsColumnMult ((sbox1 . fromIntegral) ((sbox1 . fromIntegral) ((sbox0 . fromIntegral) ((sbox0 . fromIntegral $ sbox0 byte `xor` w2) `xor` w6) `xor` w10) `xor` w14)) Two-                  !b3 = fmap mapper range-                    where mapper byte = mdsColumnMult ((sbox0 . fromIntegral) ((sbox1 . fromIntegral) ((sbox1 . fromIntegral) ((sbox0 . fromIntegral $ sbox1 byte `xor` w3) `xor` w7) `xor` w11) `xor` w15)) Three--genK :: (ByteArray ba) => KeyPackage ba -> [Word32]-genK keyPackage = concatMap makeTuple [0..19]-    where makeTuple :: Word8 -> [Word32]-          makeTuple idx = [a + b', rotateL (2 * b' + a) 9]-            where tmp1 = replicate 4 $ 2 * idx-                  tmp2 = fmap (+1) tmp1-                  a = h tmp1 keyPackage 0-                  b = h tmp2 keyPackage 1-                  b' = rotateL b 8+  where+    range = [0 .. 255]+    mkArray = array32 256+    [w0, w1, w2, w3, w4, w5, w6, w7, w8, w9, w10, w11, w12, w13, w14, w15] = take 16 ws+    (b0', b1', b2', b3') = sboxBySize $ byteSize keyPackage -h :: (ByteArray ba) => [Word8] -> KeyPackage ba -> Int -> Word32-h input keyPackage offset =  foldl' xorMdsColMult 0 $ zip [y0f, y1f, y2f, y3f] $ enumFrom Zero-    where key = rawKeyBytes keyPackage-          [y0, y1, y2, y3] = take 4 input-          (!y0f, !y1f, !y2f, !y3f) = run (y0, y1, y2, y3) $ byteSize keyPackage+    sboxBySize :: ByteSize -> ([Word32], [Word32], [Word32], [Word32])+    sboxBySize Bytes16 = (b0, b1, b2, b3)+      where+        !b0 = fmap mapper range+          where+            mapper :: Int -> Word32+            mapper byte =+                mdsColumnMult+                    ((sbox1 . fromIntegral) ((sbox0 . fromIntegral $ sbox0 byte `xor` w0) `xor` w4))+                    Zero+        !b1 = fmap mapper range+          where+            mapper byte =+                mdsColumnMult+                    ((sbox0 . fromIntegral) ((sbox0 . fromIntegral $ sbox1 byte `xor` w1) `xor` w5))+                    One+        !b2 = fmap mapper range+          where+            mapper byte =+                mdsColumnMult+                    ((sbox1 . fromIntegral) ((sbox1 . fromIntegral $ sbox0 byte `xor` w2) `xor` w6))+                    Two+        !b3 = fmap mapper range+          where+            mapper byte =+                mdsColumnMult+                    ((sbox0 . fromIntegral) ((sbox1 . fromIntegral $ sbox1 byte `xor` w3) `xor` w7))+                    Three+    sboxBySize Bytes24 = (b0, b1, b2, b3)+      where+        !b0 = fmap mapper range+          where+            mapper byte =+                mdsColumnMult+                    ( (sbox1 . fromIntegral)+                        ( (sbox0 . fromIntegral) ((sbox0 . fromIntegral $ sbox1 byte `xor` w0) `xor` w4)+                            `xor` w8+                        )+                    )+                    Zero+        !b1 = fmap mapper range+          where+            mapper byte =+                mdsColumnMult+                    ( (sbox0 . fromIntegral)+                        ( (sbox0 . fromIntegral) ((sbox1 . fromIntegral $ sbox1 byte `xor` w1) `xor` w5)+                            `xor` w9+                        )+                    )+                    One+        !b2 = fmap mapper range+          where+            mapper byte =+                mdsColumnMult+                    ( (sbox1 . fromIntegral)+                        ( (sbox1 . fromIntegral) ((sbox0 . fromIntegral $ sbox0 byte `xor` w2) `xor` w6)+                            `xor` w10+                        )+                    )+                    Two+        !b3 = fmap mapper range+          where+            mapper byte =+                mdsColumnMult+                    ( (sbox0 . fromIntegral)+                        ( (sbox1 . fromIntegral) ((sbox1 . fromIntegral $ sbox0 byte `xor` w3) `xor` w7)+                            `xor` w11+                        )+                    )+                    Three+    sboxBySize Bytes32 = (b0, b1, b2, b3)+      where+        !b0 = fmap mapper range+          where+            mapper byte =+                mdsColumnMult+                    ( (sbox1 . fromIntegral)+                        ( (sbox0 . fromIntegral)+                            ( (sbox0 . fromIntegral) ((sbox1 . fromIntegral $ sbox1 byte `xor` w0) `xor` w4)+                                `xor` w8+                            )+                            `xor` w12+                        )+                    )+                    Zero+        !b1 = fmap mapper range+          where+            mapper byte =+                mdsColumnMult+                    ( (sbox0 . fromIntegral)+                        ( (sbox0 . fromIntegral)+                            ( (sbox1 . fromIntegral) ((sbox1 . fromIntegral $ sbox0 byte `xor` w1) `xor` w5)+                                `xor` w9+                            )+                            `xor` w13+                        )+                    )+                    One+        !b2 = fmap mapper range+          where+            mapper byte =+                mdsColumnMult+                    ( (sbox1 . fromIntegral)+                        ( (sbox1 . fromIntegral)+                            ( (sbox0 . fromIntegral) ((sbox0 . fromIntegral $ sbox0 byte `xor` w2) `xor` w6)+                                `xor` w10+                            )+                            `xor` w14+                        )+                    )+                    Two+        !b3 = fmap mapper range+          where+            mapper byte =+                mdsColumnMult+                    ( (sbox0 . fromIntegral)+                        ( (sbox1 . fromIntegral)+                            ( (sbox1 . fromIntegral) ((sbox0 . fromIntegral $ sbox1 byte `xor` w3) `xor` w7)+                                `xor` w11+                            )+                            `xor` w15+                        )+                    )+                    Three -          run :: (Word8, Word8, Word8, Word8) -> ByteSize -> (Word8, Word8, Word8, Word8)-          run (!y0'', !y1'', !y2'', !y3'') Bytes32 = run (y0', y1', y2', y3') Bytes24-            where y0' = sbox1 (fromIntegral y0'') `xor` B.index key (4 * (6 + offset) + 0)-                  y1' = sbox0 (fromIntegral y1'') `xor` B.index key (4 * (6 + offset) + 1)-                  y2' = sbox0 (fromIntegral y2'') `xor` B.index key (4 * (6 + offset) + 2)-                  y3' = sbox1 (fromIntegral y3'') `xor` B.index key (4 * (6 + offset) + 3)+genK :: ByteArray ba => KeyPackage ba -> [Word32]+genK keyPackage = concatMap makeTuple [0 .. 19]+  where+    makeTuple :: Word8 -> [Word32]+    makeTuple idx = [a + b', rotateL (2 * b' + a) 9]+      where+        tmp1 = replicate 4 $ 2 * idx+        tmp2 = fmap (+ 1) tmp1+        a = h tmp1 keyPackage 0+        b = h tmp2 keyPackage 1+        b' = rotateL b 8 -          run (!y0'', !y1'', !y2'', !y3'') Bytes24 = run (y0', y1', y2', y3') Bytes16-            where y0' = sbox1 (fromIntegral y0'') `xor` B.index key (4 * (4 + offset) + 0)-                  y1' = sbox1 (fromIntegral y1'') `xor` B.index key (4 * (4 + offset) + 1)-                  y2' = sbox0 (fromIntegral y2'') `xor` B.index key (4 * (4 + offset) + 2)-                  y3' = sbox0 (fromIntegral y3'') `xor` B.index key (4 * (4 + offset) + 3)+h :: ByteArray ba => [Word8] -> KeyPackage ba -> Int -> Word32+h input keyPackage offset = foldl' xorMdsColMult 0 $ zip [y0f, y1f, y2f, y3f] $ enumFrom Zero+  where+    key = rawKeyBytes keyPackage+    [y0, y1, y2, y3] = take 4 input+    (!y0f, !y1f, !y2f, !y3f) = run (y0, y1, y2, y3) $ byteSize keyPackage -          run (!y0'', !y1'', !y2'', !y3'') Bytes16 = (y0', y1', y2', y3')-            where y0' = sbox1 . fromIntegral $ (sbox0 . fromIntegral $ (sbox0 (fromIntegral y0'') `xor` B.index key (4 * (2 + offset) + 0))) `xor` B.index key (4 * (0 + offset) + 0)-                  y1' = sbox0 . fromIntegral $ (sbox0 . fromIntegral $ (sbox1 (fromIntegral y1'') `xor` B.index key (4 * (2 + offset) + 1))) `xor` B.index key (4 * (0 + offset) + 1)-                  y2' = sbox1 . fromIntegral $ (sbox1 . fromIntegral $ (sbox0 (fromIntegral y2'') `xor` B.index key (4 * (2 + offset) + 2))) `xor` B.index key (4 * (0 + offset) + 2)-                  y3' = sbox0 . fromIntegral $ (sbox1 . fromIntegral $ (sbox1 (fromIntegral y3'') `xor` B.index key (4 * (2 + offset) + 3))) `xor` B.index key (4 * (0 + offset) + 3)+    run :: (Word8, Word8, Word8, Word8) -> ByteSize -> (Word8, Word8, Word8, Word8)+    run (!y0'', !y1'', !y2'', !y3'') Bytes32 = run (y0', y1', y2', y3') Bytes24+      where+        y0' = sbox1 (fromIntegral y0'') `xor` B.index key (4 * (6 + offset) + 0)+        y1' = sbox0 (fromIntegral y1'') `xor` B.index key (4 * (6 + offset) + 1)+        y2' = sbox0 (fromIntegral y2'') `xor` B.index key (4 * (6 + offset) + 2)+        y3' = sbox1 (fromIntegral y3'') `xor` B.index key (4 * (6 + offset) + 3)+    run (!y0'', !y1'', !y2'', !y3'') Bytes24 = run (y0', y1', y2', y3') Bytes16+      where+        y0' = sbox1 (fromIntegral y0'') `xor` B.index key (4 * (4 + offset) + 0)+        y1' = sbox1 (fromIntegral y1'') `xor` B.index key (4 * (4 + offset) + 1)+        y2' = sbox0 (fromIntegral y2'') `xor` B.index key (4 * (4 + offset) + 2)+        y3' = sbox0 (fromIntegral y3'') `xor` B.index key (4 * (4 + offset) + 3)+    run (!y0'', !y1'', !y2'', !y3'') Bytes16 = (y0', y1', y2', y3')+      where+        y0' =+            sbox1 . fromIntegral $+                ( sbox0 . fromIntegral $+                    (sbox0 (fromIntegral y0'') `xor` B.index key (4 * (2 + offset) + 0))+                )+                    `xor` B.index key (4 * (0 + offset) + 0)+        y1' =+            sbox0 . fromIntegral $+                ( sbox0 . fromIntegral $+                    (sbox1 (fromIntegral y1'') `xor` B.index key (4 * (2 + offset) + 1))+                )+                    `xor` B.index key (4 * (0 + offset) + 1)+        y2' =+            sbox1 . fromIntegral $+                ( sbox1 . fromIntegral $+                    (sbox0 (fromIntegral y2'') `xor` B.index key (4 * (2 + offset) + 2))+                )+                    `xor` B.index key (4 * (0 + offset) + 2)+        y3' =+            sbox0 . fromIntegral $+                ( sbox1 . fromIntegral $+                    (sbox1 (fromIntegral y3'') `xor` B.index key (4 * (2 + offset) + 3))+                )+                    `xor` B.index key (4 * (0 + offset) + 3) -          xorMdsColMult :: Word32 -> (Word8, Column) -> Word32-          xorMdsColMult acc wordAndIndex = acc `xor` uncurry mdsColumnMult wordAndIndex+    xorMdsColMult :: Word32 -> (Word8, Column) -> Word32+    xorMdsColMult acc wordAndIndex = acc `xor` uncurry mdsColumnMult wordAndIndex  mdsColumnMult :: Word8 -> Column -> Word32 mdsColumnMult !byte !col =-    case col of Zero  -> input .|. rotateL mul5B 8 .|. rotateL mulEF 16 .|. rotateL mulEF 24-                One   -> mulEF .|. rotateL mulEF 8 .|. rotateL mul5B 16 .|. rotateL input 24-                Two   -> mul5B .|. rotateL mulEF 8 .|. rotateL input 16 .|. rotateL mulEF 24-                Three -> mul5B .|. rotateL input 8 .|. rotateL mulEF 16 .|. rotateL mul5B 24-        where input = fromIntegral byte-              mul5B = fromIntegral $ gfMult mdsPolynomial byte 0x5B-              mulEF = fromIntegral $ gfMult mdsPolynomial byte 0xEF+    case col of+        Zero -> input .|. rotateL mul5B 8 .|. rotateL mulEF 16 .|. rotateL mulEF 24+        One -> mulEF .|. rotateL mulEF 8 .|. rotateL mul5B 16 .|. rotateL input 24+        Two -> mul5B .|. rotateL mulEF 8 .|. rotateL input 16 .|. rotateL mulEF 24+        Three -> mul5B .|. rotateL input 8 .|. rotateL mulEF 16 .|. rotateL mul5B 24+  where+    input = fromIntegral byte+    mul5B = fromIntegral $ gfMult mdsPolynomial byte 0x5B+    mulEF = fromIntegral $ gfMult mdsPolynomial byte 0xEF -tupInd :: (Bits b) => b -> (a, a) -> a+tupInd :: Bits b => b -> (a, a) -> a tupInd b     | testBit b 0 = snd     | otherwise = fst  gfMult :: Word32 -> Word8 -> Word8 -> Word8 gfMult p a b = fromIntegral $ run a b' p' result 0-    where b' = (0, fromIntegral b)-          p' = (0, p)-          result = 0+  where+    b' = (0, fromIntegral b)+    p' = (0, p)+    result = 0 -          run :: Word8 -> (Word32, Word32) -> (Word32, Word32) -> Word32 -> Int -> Word32-          run a' b'' p'' result' count =-            if count == 7+    run :: Word8 -> (Word32, Word32) -> (Word32, Word32) -> Word32 -> Int -> Word32+    run a' b'' p'' result' count =+        if count == 7             then result''             else run a'' b''' p'' result'' (count + 1)-                where result'' = result' `xor` tupInd (a' .&. 1) b''-                      a'' = shiftR a' 1-                      b''' = (fst b'', tupInd (shiftR (snd b'') 7) p'' `xor` shiftL (snd b'') 1)+      where+        result'' = result' `xor` tupInd (a' .&. 1) b''+        a'' = shiftR a' 1+        b''' = (fst b'', tupInd (shiftR (snd b'') 7) p'' `xor` shiftL (snd b'') 1)
Crypto/Cipher/Types.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE DeriveDataTypeable #-}+ -- | -- Module      : Crypto.Cipher.Types -- License     : BSD-style@@ -6,34 +8,34 @@ -- Portability : Excellent -- -- Symmetric cipher basic types----{-# LANGUAGE DeriveDataTypeable #-}-module Crypto.Cipher.Types-    (+module Crypto.Cipher.Types (     -- * Cipher classes-      Cipher(..)-    , BlockCipher(..)-    , BlockCipher128(..)-    , StreamCipher(..)-    , DataUnitOffset-    , KeySizeSpecifier(..)+    Cipher (..),+    BlockCipher (..),+    BlockCipher128 (..),+    StreamCipher (..),+    DataUnitOffset,+    KeySizeSpecifier (..),     -- , cfb8Encrypt     -- , cfb8Decrypt+     -- * AEAD functions-    , AEADMode(..)-    , CCM_M(..)-    , CCM_L(..)-    , module Crypto.Cipher.Types.AEAD+    AEADMode (..),+    CCM_M (..),+    CCM_L (..),+    module Crypto.Cipher.Types.AEAD,+     -- * Initial Vector type and constructor-    , IV-    , makeIV-    , nullIV-    , ivAdd+    IV,+    makeIV,+    nullIV,+    ivAdd,+     -- * Authentification Tag-    , AuthTag(..)-    ) where+    AuthTag (..),+) where +import Crypto.Cipher.Types.AEAD import Crypto.Cipher.Types.Base import Crypto.Cipher.Types.Block import Crypto.Cipher.Types.Stream-import Crypto.Cipher.Types.AEAD
Crypto/Cipher/Types/AEAD.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE ExistentialQuantification #-}+{-# LANGUAGE Rank2Types #-}+ -- | -- Module      : Crypto.Cipher.Types.AEAD -- License     : BSD-style@@ -6,69 +9,133 @@ -- Portability : Excellent -- -- AEAD cipher basic types----{-# LANGUAGE ExistentialQuantification #-}-{-# LANGUAGE Rank2Types #-} module Crypto.Cipher.Types.AEAD where -import           Crypto.Cipher.Types.Base-import           Crypto.Internal.ByteArray (ByteArrayAccess, ByteArray)+import Crypto.Cipher.Types.Base+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Internal.Imports+import Crypto.Internal.Imports  -- | AEAD Implementation data AEADModeImpl st = AEADModeImpl-    { aeadImplAppendHeader :: forall ba . ByteArrayAccess ba => st -> ba -> st-    , aeadImplEncrypt      :: forall ba . ByteArray ba => st -> ba -> (ba, st)-    , aeadImplDecrypt      :: forall ba . ByteArray ba => st -> ba -> (ba, st)-    , aeadImplFinalize     :: st -> Int -> AuthTag+    { aeadImplAppendHeader :: forall ba. ByteArrayAccess ba => st -> ba -> st+    -- ^ Adding associated\/additional data to the AEAD context.+    , aeadImplEncrypt :: forall ba. ByteArray ba => st -> ba -> (ba, st)+    -- ^ Encrypiting plaintext and update the AEAD context.+    , aeadImplDecrypt :: forall ba. ByteArray ba => st -> ba -> (ba, st)+    -- ^ Decrypting ciphertext and update the AEAD context.+    , aeadImplFinalize :: st -> Int -> AuthTag+    -- ^ Finalizing the AEAD context and returning the authentication tag.     } --- | Authenticated Encryption with Associated Data algorithms-data AEAD cipher = forall st . AEAD+-- | Algorithm and context for AEAD(Authenticated Encryption with Associated\/Additional Data)+data AEAD cipher = forall st. AEAD     { aeadModeImpl :: AEADModeImpl st-    , aeadState    :: !st+    , aeadState :: !st     } --- | Append some header information to an AEAD context+-- | Adding associated\/additional data to the AEAD context. aeadAppendHeader :: ByteArrayAccess aad => AEAD cipher -> aad -> AEAD cipher aeadAppendHeader (AEAD impl st) aad = AEAD impl $ aeadImplAppendHeader impl st aad --- | Encrypt some data and update the AEAD context+-- | Encrypting plaintext  and update the AEAD context. aeadEncrypt :: ByteArray ba => AEAD cipher -> ba -> (ba, AEAD cipher) aeadEncrypt (AEAD impl st) ba = second (AEAD impl) $ aeadImplEncrypt impl st ba --- | Decrypt some data and update the AEAD context+-- | Decrypting ciphertext and update the AEAD context. aeadDecrypt :: ByteArray ba => AEAD cipher -> ba -> (ba, AEAD cipher) aeadDecrypt (AEAD impl st) ba = second (AEAD impl) $ aeadImplDecrypt impl st ba --- | Finalize the AEAD context and return the authentication tag+-- | Finalizing the AEAD context and returning the authentication tag. aeadFinalize :: AEAD cipher -> Int -> AuthTag aeadFinalize (AEAD impl st) = aeadImplFinalize impl st --- | Simple AEAD encryption-aeadSimpleEncrypt :: (ByteArrayAccess aad, ByteArray ba)-                  => AEAD a        -- ^ A new AEAD Context-                  -> aad           -- ^ Optional Authentication data header-                  -> ba            -- ^ Optional Plaintext-                  -> Int           -- ^ Tag length-                  -> (AuthTag, ba) -- ^ Authentication tag and ciphertext+-- | Simple AEAD encryption.+aeadSimpleEncrypt+    :: (ByteArrayAccess aad, ByteArray ba)+    => AEAD a+    -- ^ An AEAD Context+    -> aad+    -- ^ Associated\/additional data+    -> ba+    -- ^ Plaintext+    -> Int+    -- ^ Tag length+    -> (AuthTag, ba)+    -- ^ Authentication tag and ciphertext aeadSimpleEncrypt aeadIni header input taglen = (tag, output)-  where aead                = aeadAppendHeader aeadIni header-        (output, aeadFinal) = aeadEncrypt aead input-        tag                 = aeadFinalize aeadFinal taglen+  where+    aead = aeadAppendHeader aeadIni header+    (output, aeadFinal) = aeadEncrypt aead input+    tag = aeadFinalize aeadFinal taglen --- | Simple AEAD decryption-aeadSimpleDecrypt :: (ByteArrayAccess aad, ByteArray ba)-                  => AEAD a        -- ^ A new AEAD Context-                  -> aad           -- ^ Optional Authentication data header-                  -> ba            -- ^ Ciphertext-                  -> AuthTag       -- ^ The authentication tag-                  -> Maybe ba      -- ^ Plaintext+-- | The shortest authentication tag any mode here produces, four bytes+-- (@CCM_M4@).  Modes that truncate their tag -- GCM and OCB3 -- will+-- compute one of whatever length they are asked for, down to nothing, so+-- 'aeadSimpleDecrypt' refuses a shorter tag than this rather than+-- authenticate fewer bytes than any supported mode ever emits.+minimumTagLength :: Int+minimumTagLength = 4++-- | Simple AEAD decryptio.+--+-- The number of bytes compared is the length of @authTag@.  That is the+-- caller's choice of tag length, so a caller reading a tag off the wire+-- must check its length against the one it expects: passing an+-- attacker-supplied tag straight in lets the attacker pick how much of it+-- is verified.  Tags shorter than 'minimumTagLength' are rejected+-- outright.+aeadSimpleDecrypt+    :: (ByteArrayAccess aad, ByteArray ba)+    => AEAD a+    -- ^ An AEAD Context+    -> aad+    -- ^ Associated\/additional data+    -> ba+    -- ^ Ciphertext+    -> AuthTag+    -- ^ The authentication tag+    -> Maybe ba+    -- ^ Plaintext aeadSimpleDecrypt aeadIni header input authTag+    | B.length authTag < minimumTagLength = Nothing     | tag == authTag = Just output-    | otherwise      = Nothing-  where aead                = aeadAppendHeader aeadIni header-        (output, aeadFinal) = aeadDecrypt aead input-        tag                 = aeadFinalize aeadFinal (B.length authTag)+    | otherwise = Nothing+  where+    aead = aeadAppendHeader aeadIni header+    (output, aeadFinal) = aeadDecrypt aead input+    tag = aeadFinalize aeadFinal (B.length authTag) +-- | Simple AEAD decryption with the tag length given by the caller.+--+-- 'aeadSimpleDecrypt' authenticates as many octets as the tag it is handed is+-- long.  That is the caller's choice only for as long as the tag is: one read+-- off the wire is the peer's, and an attacker who truncates it picks how much+-- of it gets verified, down to 'minimumTagLength'.+--+-- Here the length is a separate argument and a tag that is not exactly that+-- long is refused before anything is compared, so the peer cannot weaken the+-- check.  Prefer this wherever the tag is attacker reachable.+tryAeadSimpleDecrypt+    :: (ByteArrayAccess aad, ByteArray ba)+    => AEAD a+    -- ^ An AEAD Context+    -> aad+    -- ^ Associated\/additional data+    -> ba+    -- ^ Ciphertext+    -> Int+    -- ^ The tag length to authenticate, which the tag must match+    -> AuthTag+    -- ^ The authentication tag+    -> Maybe ba+    -- ^ Plaintext+tryAeadSimpleDecrypt aeadIni header input taglen authTag+    | taglen < minimumTagLength = Nothing+    | B.length authTag /= taglen = Nothing+    | tag == authTag = Just output+    | otherwise = Nothing+  where+    aead = aeadAppendHeader aeadIni header+    (output, aeadFinal) = aeadDecrypt aead input+    tag = aeadFinalize aeadFinal taglen
Crypto/Cipher/Types/Base.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE ExistentialQuantification #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.Cipher.Types.Base -- License     : BSD-style@@ -6,60 +9,63 @@ -- Portability : Excellent -- -- Symmetric cipher basic types----{-# LANGUAGE ExistentialQuantification #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.Cipher.Types.Base-    ( KeySizeSpecifier(..)-    , Cipher(..)-    , AuthTag(..)-    , AEADMode(..)-    , CCM_M(..)-    , CCM_L(..)-    , DataUnitOffset-    ) where+module Crypto.Cipher.Types.Base (+    KeySizeSpecifier (..),+    Cipher (..),+    AuthTag (..),+    AEADMode (..),+    CCM_M (..),+    CCM_L (..),+    DataUnitOffset,+) where -import           Data.Word-import           Crypto.Internal.ByteArray (Bytes, ByteArrayAccess, ByteArray)+import Crypto.Error+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, Bytes) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Internal.DeepSeq-import           Crypto.Error+import Crypto.Internal.DeepSeq+import Data.Word  -- | Different specifier for key size in bytes-data KeySizeSpecifier =-      KeySizeRange Int Int -- ^ in the range [min,max]-    | KeySizeEnum  [Int]   -- ^ one of the specified values-    | KeySizeFixed Int     -- ^ a specific size-    deriving (Show,Eq)+data KeySizeSpecifier+    = -- | in the range [min,max]+      KeySizeRange Int Int+    | -- | one of the specified values+      KeySizeEnum [Int]+    | -- | a specific size+      KeySizeFixed Int+    deriving (Show, Eq)  -- | Offset inside an XTS data unit, measured in block size. type DataUnitOffset = Word32  -- | Authentication Tag for AE cipher mode-newtype AuthTag = AuthTag { unAuthTag :: Bytes }+newtype AuthTag = AuthTag {unAuthTag :: Bytes}     deriving (Show, ByteArrayAccess, NFData)  instance Eq AuthTag where     (AuthTag a) == (AuthTag b) = B.constEq a b -data CCM_M = CCM_M4 | CCM_M6 | CCM_M8 | CCM_M10 | CCM_M12 | CCM_M14 | CCM_M16 deriving (Show, Eq)+data CCM_M = CCM_M4 | CCM_M6 | CCM_M8 | CCM_M10 | CCM_M12 | CCM_M14 | CCM_M16+    deriving (Show, Eq) data CCM_L = CCM_L2 | CCM_L3 | CCM_L4 deriving (Show, Eq)  -- | AEAD Mode-data AEADMode =-      AEAD_OCB -- OCB3+data AEADMode+    = AEAD_OCB -- OCB3     | AEAD_CCM Int CCM_M CCM_L     | AEAD_EAX     | AEAD_CWC     | AEAD_GCM-    deriving (Show,Eq)+    deriving (Show, Eq)  -- | Symmetric cipher class. class Cipher cipher where     -- | Initialize a cipher context from a key-    cipherInit    :: ByteArray key => key -> CryptoFailable cipher+    cipherInit :: ByteArray key => key -> CryptoFailable cipher+     -- | Cipher name-    cipherName    :: cipher -> String+    cipherName :: cipher -> String+     -- | return the size of the key required for this cipher.     -- Some cipher accept any size for key     cipherKeySize :: cipher -> KeySizeSpecifier
Crypto/Cipher/Types/Block.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE ExistentialQuantification #-}+{-# LANGUAGE MultiParamTypeClasses #-}+{-# LANGUAGE Rank2Types #-}+{-# LANGUAGE ViewPatterns #-}+ -- | -- Module      : Crypto.Cipher.Types.Block -- License     : BSD-style@@ -6,51 +11,57 @@ -- Portability : Excellent -- -- Block cipher basic types----{-# LANGUAGE MultiParamTypeClasses #-}-{-# LANGUAGE ExistentialQuantification #-}-{-# LANGUAGE ViewPatterns #-}-{-# LANGUAGE Rank2Types #-}-module Crypto.Cipher.Types.Block-    (+module Crypto.Cipher.Types.Block (     -- * BlockCipher-      BlockCipher(..)-    , BlockCipher128(..)+    BlockCipher (..),+    BlockCipher128 (..),+     -- * Initialization vector (IV)-    , IV(..)-    , makeIV-    , nullIV-    , ivAdd+    IV (..),+    makeIV,+    nullIV,+    ivAdd,+     -- * XTS-    , XTS+    XTS,+     -- * AEAD-    , AEAD(..)+    AEAD (..),     -- , AEADState(..)-    , AEADModeImpl(..)-    , aeadAppendHeader-    , aeadEncrypt-    , aeadDecrypt-    , aeadFinalize+    AEADModeImpl (..),+    aeadAppendHeader,+    aeadEncrypt,+    aeadDecrypt,+    aeadFinalize,+     -- * CFB 8 bits-    --, cfb8Encrypt-    --, cfb8Decrypt-    ) where+) where -import           Data.Word-import           Crypto.Error-import           Crypto.Cipher.Types.Base-import           Crypto.Cipher.Types.GF-import           Crypto.Cipher.Types.AEAD-import           Crypto.Cipher.Types.Utils+-- , cfb8Encrypt+-- , cfb8Decrypt -import           Crypto.Internal.ByteArray (ByteArrayAccess, ByteArray, withByteArray, Bytes)+import Crypto.Cipher.Types.AEAD+import Crypto.Cipher.Types.Base+import Crypto.Cipher.Types.GF+import Crypto.Error+import Data.Word++import Crypto.Internal.ByteArray (+    ByteArray,+    ByteArrayAccess,+    Bytes,+    withByteArray,+ ) import qualified Crypto.Internal.ByteArray as B+import Data.ByteString (ByteString)+import qualified Data.ByteString as S -import           Foreign.Ptr-import           Foreign.Storable+import Foreign.Marshal.Utils (copyBytes)+import Foreign.Ptr+import Foreign.Storable  -- | an IV parametrized by the cipher-data IV c = forall byteArray . ByteArray byteArray => IV !byteArray+data IV c = forall byteArray. ByteArray byteArray => IV !byteArray  instance BlockCipher c => ByteArrayAccess (IV c) where     withByteArray (IV z) f = withByteArray z f@@ -59,16 +70,21 @@     (IV a) == (IV b) = B.eq a b  -- | XTS callback-type XTS ba cipher = (cipher, cipher)-                  -> IV cipher        -- ^ Usually represent the Data Unit (e.g. disk sector)-                  -> DataUnitOffset   -- ^ Offset in the data unit in number of blocks-                  -> ba               -- ^ Data-                  -> ba               -- ^ Processed Data+type XTS ba cipher =+    (cipher, cipher)+    -> IV cipher+    -- ^ Usually represent the Data Unit (e.g. disk sector)+    -> DataUnitOffset+    -- ^ Offset in the data unit in number of blocks+    -> ba+    -- ^ Data+    -> ba+    -- ^ Processed Data  -- | Symmetric block cipher class class Cipher cipher => BlockCipher cipher where     -- | Return the size of block required for this block cipher-    blockSize    :: cipher -> Int+    blockSize :: cipher -> Int      -- | Encrypt blocks     --@@ -85,6 +101,7 @@     -- input need to be a multiple of the blocksize     cbcEncrypt :: ByteArray ba => cipher -> IV cipher -> ba -> ba     cbcEncrypt = cbcEncryptGeneric+     -- | decrypt using the CBC mode.     --     -- input need to be a multiple of the blocksize@@ -96,6 +113,7 @@     -- input need to be a multiple of the blocksize     cfbEncrypt :: ByteArray ba => cipher -> IV cipher -> ba -> ba     cfbEncrypt = cfbEncryptGeneric+     -- | decrypt using the CFB mode.     --     -- input need to be a multiple of the blocksize@@ -116,7 +134,8 @@     -- | Initialize a new AEAD State     --     -- When Nothing is returns, it means the mode is not handled.-    aeadInit :: ByteArrayAccess iv => AEADMode -> cipher -> iv -> CryptoFailable (AEAD cipher)+    aeadInit+        :: ByteArrayAccess iv => AEADMode -> cipher -> iv -> CryptoFailable (AEAD cipher)     aeadInit _ _ _ = CryptoFailed CryptoError_AEADModeNotSupported  -- | class of block cipher with a 128 bits block size@@ -125,118 +144,223 @@     --     -- input need to be a multiple of the blocksize, and the cipher     -- need to process 128 bits block only-    xtsEncrypt :: ByteArray ba-               => (cipher, cipher)-               -> IV cipher        -- ^ Usually represent the Data Unit (e.g. disk sector)-               -> DataUnitOffset   -- ^ Offset in the data unit in number of blocks-               -> ba               -- ^ Plaintext-               -> ba               -- ^ Ciphertext+    xtsEncrypt+        :: ByteArray ba+        => (cipher, cipher)+        -> IV cipher+        -- ^ Usually represent the Data Unit (e.g. disk sector)+        -> DataUnitOffset+        -- ^ Offset in the data unit in number of blocks+        -> ba+        -- ^ Plaintext+        -> ba+        -- ^ Ciphertext     xtsEncrypt = xtsEncryptGeneric      -- | decrypt using the XTS mode.     --     -- input need to be a multiple of the blocksize, and the cipher     -- need to process 128 bits block only-    xtsDecrypt :: ByteArray ba-               => (cipher, cipher)-               -> IV cipher        -- ^ Usually represent the Data Unit (e.g. disk sector)-               -> DataUnitOffset   -- ^ Offset in the data unit in number of blocks-               -> ba               -- ^ Ciphertext-               -> ba               -- ^ Plaintext+    xtsDecrypt+        :: ByteArray ba+        => (cipher, cipher)+        -> IV cipher+        -- ^ Usually represent the Data Unit (e.g. disk sector)+        -> DataUnitOffset+        -- ^ Offset in the data unit in number of blocks+        -> ba+        -- ^ Ciphertext+        -> ba+        -- ^ Plaintext     xtsDecrypt = xtsDecryptGeneric  -- | Create an IV for a specified block cipher makeIV :: (ByteArrayAccess b, BlockCipher c) => b -> Maybe (IV c) makeIV b = toIV undefined-  where toIV :: BlockCipher c => c -> Maybe (IV c)-        toIV cipher-          | B.length b == sz = Just $ IV (B.convert b :: Bytes)-          | otherwise        = Nothing-          where sz = blockSize cipher+  where+    toIV :: BlockCipher c => c -> Maybe (IV c)+    toIV cipher+        | B.length b == sz = Just $ IV (B.convert b :: Bytes)+        | otherwise = Nothing+      where+        sz = blockSize cipher  -- | Create an IV that is effectively representing the number 0 nullIV :: BlockCipher c => IV c nullIV = toIV undefined-  where toIV :: BlockCipher c => c -> IV c-        toIV cipher = IV (B.zero (blockSize cipher) :: Bytes)+  where+    toIV :: BlockCipher c => c -> IV c+    toIV cipher = IV (B.zero (blockSize cipher) :: Bytes)  -- | Increment an IV by a number. -- -- Assume the IV is in Big Endian format. ivAdd :: IV c -> Int -> IV c ivAdd (IV b) i = IV $ copy b-  where copy :: ByteArray bs => bs -> bs-        copy bs = B.copyAndFreeze bs $ loop i (B.length bs - 1)+  where+    copy :: ByteArray bs => bs -> bs+    copy bs = B.copyAndFreeze bs $ loop i (B.length bs - 1) -        loop :: Int -> Int -> Ptr Word8 -> IO ()-        loop acc ofs p-            | ofs < 0   = return ()-            | otherwise = do-                v <- peek (p `plusPtr` ofs) :: IO Word8-                let accv    = acc + fromIntegral v-                    (hi,lo) = accv `divMod` 256-                poke (p `plusPtr` ofs) (fromIntegral lo :: Word8)-                loop hi (ofs - 1) p+    loop :: Int -> Int -> Ptr Word8 -> IO ()+    loop acc ofs p+        | ofs < 0 = return ()+        | otherwise = do+            v <- peek (p `plusPtr` ofs) :: IO Word8+            let accv = acc + fromIntegral v+                (hi, lo) = accv `divMod` 256+            poke (p `plusPtr` ofs) (fromIntegral lo :: Word8)+            loop hi (ofs - 1) p -cbcEncryptGeneric :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba-cbcEncryptGeneric cipher ivini input = mconcat $ doEnc ivini $ chunk (blockSize cipher) input-  where doEnc _  []     = []-        doEnc iv (i:is) =-            let o = ecbEncrypt cipher $ B.xor iv i-             in o : doEnc (IV o) is+cbcEncryptGeneric+    :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba+cbcEncryptGeneric cipher ivini input =+    B.concat $ doEnc ivini $ slices (blockSize cipher) input+  where+    -- the blocks of the message as shared slices rather than copies: each+    -- block already costs an exclusive or and a call into the cipher, both of+    -- which allocate, and the chain makes it one block at a time+    doEnc _ [] = []+    doEnc iv (i : is) =+        let o = ecbEncrypt cipher (B.bxor iv i) `asTypeOf` input+         in o : doEnc (IV o) is -cbcDecryptGeneric :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba-cbcDecryptGeneric cipher ivini input = mconcat $ doDec ivini $ chunk (blockSize cipher) input+-- | How many blocks to hand the cipher at a time in the modes whose blocks do+-- not depend on one another.  Enough that the cost of a call disappears, few+-- enough that what it copies stays in cache.+blocksPerCall :: Int+blocksPerCall = 2048++-- | The input in slices of that many blocks.  A ByteString shares where+-- 'B.splitAt' copies the rest of the message, once per slice.+slices :: ByteArray ba => Int -> ba -> [ByteString]+slices bytes input = go (B.convert input)   where-        doDec _  []     = []-        doDec iv (i:is) =-            let o = B.xor iv $ ecbDecrypt cipher i-             in o : doDec (IV i) is+    go bs+        | S.null bs = []+        | otherwise = let (hd, tl) = S.splitAt bytes bs in hd : go tl -cfbEncryptGeneric :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba-cfbEncryptGeneric cipher ivini input = mconcat $ doEnc ivini $ chunk (blockSize cipher) input+-- | The previous ciphertext block of every block in a slice: the incoming IV,+-- and then the slice itself one block short.+shiftedBy :: BlockCipher cipher => Int -> IV cipher -> ByteString -> ByteString+shiftedBy bsz iv c = S.append (B.convert iv) (S.take (S.length c - bsz) c)++-- | The last whole block of a slice, which is where the next one carries on+-- from.+lastBlockOf :: Int -> ByteString -> IV cipher+lastBlockOf bsz c = IV (B.convert (S.drop (S.length c - bsz) c) :: Bytes)++-- | Decryption does not chain: @P_i@ is @D(C_i)@ exclusive-ored with+-- @C_(i-1)@, so a whole slice is decrypted in one call and exclusive-ored with+-- the ciphertext moved along by a block.+cbcDecryptGeneric+    :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba+cbcDecryptGeneric cipher ivini input =+    B.concat $ doDec ivini $ slices (blocksPerCall * bsz) input   where-        doEnc _  []     = []-        doEnc (IV iv) (i:is) =-            let o = B.xor i $ ecbEncrypt cipher iv-             in o : doEnc (IV o) is+    bsz = blockSize cipher+    conv x = B.convert x `asTypeOf` input+    xorB a b = B.bxor a b `asTypeOf` input+    doDec _ [] = []+    doDec iv (c : cs) =+        xorB (ecbDecrypt cipher (conv c)) (conv (shiftedBy bsz iv c))+            : doDec (lastBlockOf bsz c) cs -cfbDecryptGeneric :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba-cfbDecryptGeneric cipher ivini input = mconcat $ doDec ivini $ chunk (blockSize cipher) input+cfbEncryptGeneric+    :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba+cfbEncryptGeneric cipher ivini input =+    B.concat $ doEnc ivini $ slices (blockSize cipher) input   where-        doDec _  []     = []-        doDec (IV iv) (i:is) =-            let o = B.xor i $ ecbEncrypt cipher iv-             in o : doDec (IV i) is+    doEnc _ [] = []+    doEnc (IV iv) (i : is) =+        let o = B.bxor i (ecbEncrypt cipher iv) `asTypeOf` input+         in o : doEnc (IV o) is -ctrCombineGeneric :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba-ctrCombineGeneric cipher ivini input = mconcat $ doCnt ivini $ chunk (blockSize cipher) input-  where doCnt _  [] = []-        doCnt iv@(IV ivd) (i:is) =-            let ivEnc = ecbEncrypt cipher ivd-             in B.xor i ivEnc : doCnt (ivAdd iv 1) is+-- | Nor does this one: @P_i@ is @C_i@ exclusive-ored with @E(C_(i-1))@, and+-- what gets encrypted is again the ciphertext moved along by a block.+cfbDecryptGeneric+    :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba+cfbDecryptGeneric cipher ivini input =+    B.concat $ doDec ivini $ slices (blocksPerCall * bsz) input+  where+    bsz = blockSize cipher+    conv x = B.convert x `asTypeOf` input+    xorB a b = B.bxor a b `asTypeOf` input+    doDec _ [] = []+    doDec iv (c : cs) =+        xorB (conv c) (ecbEncrypt cipher (conv (shiftedBy bsz iv c)))+            : doDec (lastBlockOf bsz c) cs +-- | The counters do not depend on the message at all, so a slice of them is+-- built and encrypted in one call.+ctrCombineGeneric+    :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba+ctrCombineGeneric cipher ivini input =+    B.concat $ doCnt ivini $ slices (blocksPerCall * bsz) input+  where+    bsz = blockSize cipher+    conv x = B.convert x `asTypeOf` input+    xorB a b = B.bxor a b `asTypeOf` input+    doCnt _ [] = []+    doCnt iv (m : ms) =+        xorB (conv m) (ecbEncrypt cipher (counters iv n `asTypeOf` input))+            : doCnt (ivAdd iv n) ms+      where+        n = (S.length m + bsz - 1) `div` bsz++-- | The counters for a slice: the given one, then each next as the one before+-- it plus one.+--+-- One buffer, filled in place.  Asking 'ivAdd' for each of them separately+-- allocated a block per block and walked the whole width of the counter from+-- the original every time, which cost more than the cipher did: counter mode+-- ran at a quarter of what the same cipher managed in ECB, and at an eighth+-- for Blowfish.+counters :: (ByteArray ba, BlockCipher cipher) => IV cipher -> Int -> ba+counters iv n = B.allocAndFreeze (n * bsz) fill+  where+    bsz = B.length iv++    fill p = do+        B.copyByteArrayToPtr iv p+        let go k prev+                | k >= n = return ()+                | otherwise = do+                    let this = prev `plusPtr` bsz+                    copyBytes this prev bsz+                    increment this (bsz - 1)+                    go (k + 1) this+        go 1 p++    increment p ofs+        | ofs < 0 = return ()+        | otherwise = do+            v <- peek (p `plusPtr` ofs) :: IO Word8+            poke (p `plusPtr` ofs) (v + 1)+            if v == 0xff then increment p (ofs - 1) else return ()+ xtsEncryptGeneric :: (ByteArray ba, BlockCipher128 cipher) => XTS ba cipher xtsEncryptGeneric = xtsGeneric ecbEncrypt  xtsDecryptGeneric :: (ByteArray ba, BlockCipher128 cipher) => XTS ba cipher xtsDecryptGeneric = xtsGeneric ecbDecrypt -xtsGeneric :: (ByteArray ba, BlockCipher128 cipher)-           => (cipher -> ba -> ba)-           -> (cipher, cipher)-           -> IV cipher-           -> DataUnitOffset-           -> ba-           -> ba+xtsGeneric+    :: (ByteArray ba, BlockCipher128 cipher)+    => (cipher -> ba -> ba)+    -> (cipher, cipher)+    -> IV cipher+    -> DataUnitOffset+    -> ba+    -> ba xtsGeneric f (cipher, tweakCipher) (IV iv) sPoint input =-    mconcat $ doXts iniTweak $ chunk (blockSize cipher) input-  where encTweak = ecbEncrypt tweakCipher iv-        iniTweak = iterate xtsGFMul encTweak !! fromIntegral sPoint-        doXts _     []     = []-        doXts tweak (i:is) =-            let o = B.xor (f cipher $ B.xor i tweak) tweak-             in o : doXts (xtsGFMul tweak) is+    B.concat $ doXts iniTweak $ slices (blockSize cipher) input+  where+    encTweak = ecbEncrypt tweakCipher iv+    iniTweak = iterate xtsGFMul encTweak !! fromIntegral sPoint+    doXts _ [] = []+    doXts tweak (i : is) =+        let o = B.bxor (f cipher (B.bxor i tweak)) tweak `asTypeOf` input+         in o : doXts (xtsGFMul tweak) is  {- -- | Encrypt using CFB mode in 8 bit output
Crypto/Cipher/Types/GF.hs view
@@ -6,19 +6,17 @@ -- Portability : Excellent -- -- Slow Galois Field arithmetic for generic XTS and GCM implementation----module Crypto.Cipher.Types.GF-    (+module Crypto.Cipher.Types.GF (     -- * XTS support-      xtsGFMul-    ) where+    xtsGFMul,+) where -import           Crypto.Internal.Imports-import           Crypto.Internal.ByteArray (ByteArray, withByteArray)+import Crypto.Internal.ByteArray (ByteArray, withByteArray) import qualified Crypto.Internal.ByteArray as B-import           Foreign.Storable-import           Foreign.Ptr-import           Data.Bits+import Crypto.Internal.Imports+import Data.Bits+import Foreign.Ptr+import Foreign.Storable  -- | Compute the gfmul with the XTS polynomial --@@ -29,19 +27,22 @@ xtsGFMul b     | len == 16 =         B.allocAndFreeze len $ \dst ->-        withByteArray b      $ \src -> do-            (hi,lo) <- gf <$> peek (castPtr src) <*> peek (castPtr src `plusPtr` 8)-            poke (castPtr dst) lo-            poke (castPtr dst `plusPtr` 8) hi+            withByteArray b $ \src -> do+                (hi, lo) <- gf <$> peek (castPtr src) <*> peek (castPtr src `plusPtr` 8)+                poke (castPtr dst) lo+                poke (castPtr dst `plusPtr` 8) hi     | otherwise = error "unsupported block size in GF"-  where gf :: Word64 -> Word64 -> (Word64, Word64)-        gf srcLo srcHi =-            ((if carryLo then (.|. 1) else id) (srcHi `shiftL` 1)-            ,(if carryHi then xor 0x87 else id) $ (srcLo `shiftL` 1)-            )-          where carryHi = srcHi `testBit` 63 -                carryLo = srcLo `testBit` 63-        len = B.length b+  where+    gf :: Word64 -> Word64 -> (Word64, Word64)+    gf srcLo srcHi =+        ( (if carryLo then (.|. 1) else id) (srcHi `shiftL` 1)+        , (if carryHi then xor 0x87 else id) $ (srcLo `shiftL` 1)+        )+      where+        carryHi = srcHi `testBit` 63+        carryLo = srcLo `testBit` 63+    len = B.length b+ {- 	const uint64_t gf_mask = cpu_to_le64(0x8000000000000000ULL); 	uint64_t r = ((a->q[1] & gf_mask) ? cpu_to_le64(0x87) : 0);
Crypto/Cipher/Types/Stream.hs view
@@ -6,10 +6,9 @@ -- Portability : Excellent -- -- Stream cipher basic types----module Crypto.Cipher.Types.Stream-    ( StreamCipher(..)-    ) where+module Crypto.Cipher.Types.Stream (+    StreamCipher (..),+) where  import Crypto.Cipher.Types.Base import Crypto.Internal.ByteArray (ByteArray)
Crypto/Cipher/Types/Utils.hs view
@@ -6,16 +6,23 @@ -- Portability : Excellent -- -- Basic utility for cipher related stuff--- module Crypto.Cipher.Types.Utils where -import           Crypto.Internal.ByteArray (ByteArray)+import Crypto.Internal.ByteArray (ByteArray) import qualified Crypto.Internal.ByteArray as B+import Data.ByteString (ByteString)+import qualified Data.ByteString as S  -- | Chunk some input byte array into @sz byte list of byte array.+--+-- The input is held as a 'ByteString' while it is cut up, because+-- 'Crypto.Internal.ByteArray.splitAt' copies both halves whatever the type+-- underneath: cutting a block off the front that way copies the rest of the+-- message, once per block, and so the message about n/2 times.  A ByteString+-- shares instead, and only the blocks themselves are copied out. chunk :: ByteArray b => Int -> b -> [b]-chunk sz bs = split bs-  where split b | B.length b <= sz = [b]-                | otherwise        =-                        let (b1, b2) = B.splitAt sz b-                         in b1 : split b2+chunk sz bs = map B.convert (split (B.convert bs :: ByteString))+  where+    split b+        | S.length b <= sz = [b]+        | otherwise = let (b1, b2) = S.splitAt sz b in b1 : split b2
Crypto/Cipher/Utils.hs view
@@ -1,18 +1,21 @@-module Crypto.Cipher.Utils-    ( validateKeySize-    ) where+module Crypto.Cipher.Utils (+    validateKeySize,+) where -import Crypto.Error import Crypto.Cipher.Types+import Crypto.Error  import Data.ByteArray as BA -validateKeySize :: (ByteArrayAccess key, Cipher cipher) => cipher -> key -> CryptoFailable key-validateKeySize c k = if validKeyLength-                      then CryptoPassed k-                      else CryptoFailed CryptoError_KeySizeInvalid-  where keyLength = BA.length k-        validKeyLength = case cipherKeySize c of-          KeySizeRange low high -> keyLength >= low && keyLength <= high-          KeySizeEnum lengths -> keyLength `elem` lengths-          KeySizeFixed s -> keyLength == s+validateKeySize+    :: (ByteArrayAccess key, Cipher cipher) => cipher -> key -> CryptoFailable key+validateKeySize c k =+    if validKeyLength+        then CryptoPassed k+        else CryptoFailed CryptoError_KeySizeInvalid+  where+    keyLength = BA.length k+    validKeyLength = case cipherKeySize c of+        KeySizeRange low high -> keyLength >= low && keyLength <= high+        KeySizeEnum lengths -> keyLength `elem` lengths+        KeySizeFixed s -> keyLength == s
Crypto/Cipher/XSalsa.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE ForeignFunctionInterface #-}+ -- | -- Module      : Crypto.Cipher.XSalsa -- License     : BSD-style@@ -8,42 +10,48 @@ -- Implementation of XSalsa20 algorithm -- <https://cr.yp.to/snuffle/xsalsa-20081128.pdf> -- Based on the Salsa20 algorithm with 256 bit key extended with 192 bit nonce--{-# LANGUAGE ForeignFunctionInterface #-}-module Crypto.Cipher.XSalsa-    ( initialize-    , derive-    , combine-    , generate-    , State-    ) where+module Crypto.Cipher.XSalsa (+    initialize,+    derive,+    combine,+    generate,+    State,+) where -import           Crypto.Internal.ByteArray (ByteArrayAccess)+import Crypto.Cipher.Salsa hiding (initialize)+import Crypto.Internal.ByteArray (ByteArrayAccess) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Internal.Compat-import           Crypto.Internal.Imports-import           Foreign.Ptr-import           Crypto.Cipher.Salsa hiding (initialize)+import Crypto.Internal.Compat+import Crypto.Internal.Imports+import Foreign.Ptr  -- | Initialize a new XSalsa context with the number of rounds, -- the key and the nonce associated.-initialize :: (ByteArrayAccess key, ByteArrayAccess nonce)-           => Int    -- ^ number of rounds (8,12,20)-           -> key    -- ^ the key (256 bits)-           -> nonce  -- ^ the nonce (192 bits)-           -> State  -- ^ the initial XSalsa state+initialize+    :: (ByteArrayAccess key, ByteArrayAccess nonce)+    => Int+    -- ^ number of rounds (8,12,20)+    -> key+    -- ^ the key (256 bits)+    -> nonce+    -- ^ the nonce (192 bits)+    -> State+    -- ^ the initial XSalsa state initialize nbRounds key nonce-    | kLen /= 32                      = error "XSalsa: key length should be 256 bits"-    | nonceLen /= 24                  = error "XSalsa: nonce length should be 192 bits"-    | nbRounds `notElem` [8,12,20]    = error "XSalsa: rounds should be 8, 12 or 20"+    | kLen /= 32 =+        error "XSalsa: key length should be 256 bits"+    | nonceLen /= 24 =+        error "XSalsa: nonce length should be 192 bits"+    | nbRounds `notElem` [8, 12, 20] = error "XSalsa: rounds should be 8, 12 or 20"     | otherwise = unsafeDoIO $ do         stPtr <- B.alloc 132 $ \stPtr ->-            B.withByteArray nonce $ \noncePtr  ->-            B.withByteArray key   $ \keyPtr ->-                ccrypton_xsalsa_init stPtr nbRounds kLen keyPtr nonceLen noncePtr+            B.withByteArray nonce $ \noncePtr ->+                B.withByteArray key $ \keyPtr ->+                    ccrypton_xsalsa_init stPtr nbRounds kLen keyPtr nonceLen noncePtr         return $ State stPtr-  where kLen     = B.length key-        nonceLen = B.length nonce+  where+    kLen = B.length key+    nonceLen = B.length nonce  -- | Use an already initialized context and new nonce material to derive another -- XSalsa context.@@ -55,21 +63,27 @@ -- -- The output context always uses the same number of rounds as the input -- context.-derive :: ByteArrayAccess nonce-       => State  -- ^ base XSalsa state-       -> nonce  -- ^ the remainder nonce (128 bits)-       -> State  -- ^ the new XSalsa state+derive+    :: ByteArrayAccess nonce+    => State+    -- ^ base XSalsa state+    -> nonce+    -- ^ the remainder nonce (128 bits)+    -> State+    -- ^ the new XSalsa state derive (State stPtr') nonce     | nonceLen /= 16 = error "XSalsa: nonce length should be 128 bits"     | otherwise = unsafeDoIO $ do         stPtr <- B.copy stPtr' $ \stPtr ->-            B.withByteArray nonce $ \noncePtr  ->+            B.withByteArray nonce $ \noncePtr ->                 ccrypton_xsalsa_derive stPtr nonceLen noncePtr         return $ State stPtr-  where nonceLen = B.length nonce+  where+    nonceLen = B.length nonce  foreign import ccall "crypton_xsalsa_init"-    ccrypton_xsalsa_init :: Ptr State -> Int -> Int -> Ptr Word8 -> Int -> Ptr Word8 -> IO ()+    ccrypton_xsalsa_init+        :: Ptr State -> Int -> Int -> Ptr Word8 -> Int -> Ptr Word8 -> IO ()  foreign import ccall "crypton_xsalsa_derive"     ccrypton_xsalsa_derive :: Ptr State -> Int -> Ptr Word8 -> IO ()
Crypto/ConstructHash/MiyaguchiPreneel.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.ConstructHash.MiyaguchiPreneel -- License     : BSD-style@@ -7,62 +9,70 @@ -- -- Provide the hash function construction method from block cipher -- <https://en.wikipedia.org/wiki/One-way_compression_function>----{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.ConstructHash.MiyaguchiPreneel-       ( compute, compute'-       , MiyaguchiPreneel-       ) where+module Crypto.ConstructHash.MiyaguchiPreneel (+    compute,+    compute',+    MiyaguchiPreneel,+) where -import           Data.List (foldl')+import Data.List (foldl')+import Prelude hiding (foldl') -import           Crypto.Data.Padding (pad, Format (ZERO))-import           Crypto.Cipher.Types-import           Crypto.Error (throwCryptoError)-import           Crypto.Internal.ByteArray (ByteArrayAccess, ByteArray, Bytes)+import Crypto.Cipher.Types+import Crypto.Cipher.Types.Utils (chunk)+import Crypto.Data.Padding (Format (ZERO), pad)+import Crypto.Error (throwCryptoError)+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, Bytes) import qualified Crypto.Internal.ByteArray as B - newtype MiyaguchiPreneel a = MP Bytes     deriving (ByteArrayAccess)  instance Eq (MiyaguchiPreneel a) where-    MP b1 == MP b2  =  B.constEq b1 b2-+    MP b1 == MP b2 = B.constEq b1 b2  -- | Compute Miyaguchi-Preneel one way compress using the supplied block cipher.-compute' :: (ByteArrayAccess bin, BlockCipher cipher)-         => (Bytes -> cipher)       -- ^ key build function to compute Miyaguchi-Preneel. care about block-size and key-size-         -> bin                     -- ^ input message-         -> MiyaguchiPreneel cipher -- ^ output tag-compute' g = MP . foldl' (step $ g) (B.replicate bsz 0) . chunks . pad (ZERO bsz) . B.convert+compute'+    :: (ByteArrayAccess bin, BlockCipher cipher)+    => (Bytes -> cipher)+    -- ^ key build function to compute Miyaguchi-Preneel. care about block-size and key-size+    -> bin+    -- ^ input message+    -> MiyaguchiPreneel cipher+    -- ^ output tag+compute' g =+    MP . foldl' (step $ g) (B.replicate bsz 0) . chunks . pad (ZERO bsz) . B.convert   where-    bsz = blockSize ( g B.empty {- dummy to get block size -} )+    bsz = blockSize (g B.empty {- dummy to get block size -})+    -- 'chunk' slices rather than splitting the message, which copied whatever+    -- was left of it once per block     chunks msg-      | B.null msg  =  []-      | otherwise  =   (hd :: Bytes) : chunks tl-      where-        (hd, tl) = B.splitAt bsz msg+        | B.null msg = []+        | otherwise = chunk bsz (msg :: Bytes)  -- | Compute Miyaguchi-Preneel one way compress using the inferred block cipher. --   Only safe when KEY-SIZE equals to BLOCK-SIZE. -- --   Simple usage /mp' msg :: MiyaguchiPreneel AES128/-compute :: (ByteArrayAccess bin, BlockCipher cipher)-        => bin                     -- ^ input message-        -> MiyaguchiPreneel cipher -- ^ output tag+compute+    :: (ByteArrayAccess bin, BlockCipher cipher)+    => bin+    -- ^ input message+    -> MiyaguchiPreneel cipher+    -- ^ output tag compute = compute' $ throwCryptoError . cipherInit  -- | computation step of Miyaguchi-Preneel-step :: (ByteArray ba, BlockCipher k)-     => (ba -> k)-     -> ba-     -> ba-     -> ba+step+    :: (ByteArray ba, BlockCipher k)+    => (ba -> k)+    -> ba+    -> ba+    -> ba step g iv msg =     ecbEncrypt k msg `bxor` iv `bxor` msg   where     k = g iv  bxor :: ByteArray ba => ba -> ba -> ba-bxor = B.xor+bxor = B.bxor
Crypto/Data/AFIS.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE ScopedTypeVariables #-}+ -- | -- Module      : Crypto.Data.AFIS -- License     : BSD-style@@ -11,26 +13,27 @@ -- The algorithm bloats an arbitrary secret with many bits that are necessary for -- the recovery of the key (merge), and allow greater way to permanently -- destroy a key stored on disk.----{-# LANGUAGE ScopedTypeVariables #-}-module Crypto.Data.AFIS-    ( split-    , merge-    ) where+module Crypto.Data.AFIS (+    split,+    trySplit,+    merge,+    tryMerge,+) where -import           Crypto.Hash-import           Crypto.Random.Types-import           Crypto.Internal.Compat-import           Control.Monad (forM_, foldM)-import           Data.Word-import           Data.Bits-import           Foreign.Storable-import           Foreign.Ptr+import Control.Monad (foldM, forM_)+import Crypto.Error+import Crypto.Hash+import Crypto.Internal.Compat+import Crypto.Random.Types+import Data.Bits+import Data.Word+import Foreign.Ptr+import Foreign.Storable -import           Crypto.Internal.ByteArray (ByteArray, Bytes, MemView(..))+import Crypto.Internal.ByteArray (ByteArray, Bytes, MemView (..)) import qualified Crypto.Internal.ByteArray as B -import           Data.Memory.PtrMethods (memSet, memCopy)+import Data.Memory.PtrMethods (memCopy, memSet)  -- | Split data to diffused data, using a random generator and -- an hash algorithm.@@ -50,56 +53,118 @@ -- where acc is : --   acc(n+1) = hash (n ++ rand(n)) ^ acc(n) ---split :: (ByteArray ba, HashAlgorithm hash, DRG rng)-      => hash  -- ^ Hash algorithm to use as diffuser-      -> rng   -- ^ Random generator to use-      -> Int   -- ^ Number of times to diffuse the data.-      -> ba    -- ^ original data to diffuse.-      -> (ba, rng)         -- ^ The diffused data-{-# NOINLINE split #-}-split hashAlg rng expandTimes src-    | expandTimes <= 1 = error "invalid expandTimes value"-    | otherwise        = unsafeDoIO $ do+-- The data has to be at least one byte long and the number of times to diffuse+-- it at least two; anything else raises 'CryptoError_ParameterInvalid', which+-- 'trySplit' reports as 'CryptoFailed' instead.+split+    :: (ByteArray ba, HashAlgorithm hash, DRG rng)+    => hash+    -- ^ Hash algorithm to use as diffuser+    -> rng+    -- ^ Random generator to use+    -> Int+    -- ^ Number of times to diffuse the data.+    -> ba+    -- ^ original data to diffuse.+    -> (ba, rng)+    -- ^ The diffused data+split hashAlg rng expandTimes src =+    throwCryptoError (trySplit hashAlg rng expandTimes src)++-- | Split data to diffused data, reporting parameters the splitter cannot work+-- with rather than raising.+--+-- See 'split'.+trySplit+    :: (ByteArray ba, HashAlgorithm hash, DRG rng)+    => hash+    -- ^ Hash algorithm to use as diffuser+    -> rng+    -- ^ Random generator to use+    -> Int+    -- ^ Number of times to diffuse the data.+    -> ba+    -- ^ original data to diffuse.+    -> CryptoFailable (ba, rng)+    -- ^ The diffused data+{-# NOINLINE trySplit #-}+trySplit hashAlg rng expandTimes src+    | expandTimes < 2 = CryptoFailed CryptoError_ParameterInvalid+    -- an empty secret splits into nothing at all, which merge cannot undo+    | blockSize == 0 = CryptoFailed CryptoError_ParameterInvalid+    | otherwise = CryptoPassed $ unsafeDoIO $ do         (rng', bs) <- B.allocRet diffusedLen runOp         return (bs, rng')-  where diffusedLen = blockSize * expandTimes-        blockSize   = B.length src-        runOp dstPtr = do-            let lastBlock = dstPtr `plusPtr` (blockSize * (expandTimes-1))-            memSet lastBlock 0 blockSize-            let randomBlockPtrs = map (plusPtr dstPtr . (*) blockSize) [0..(expandTimes-2)]-            rng' <- foldM fillRandomBlock rng randomBlockPtrs-            mapM_ (addRandomBlock lastBlock) randomBlockPtrs-            B.withByteArray src $ \srcPtr -> xorMem srcPtr lastBlock blockSize-            return rng'-        addRandomBlock lastBlock blockPtr = do-            xorMem blockPtr lastBlock blockSize-            diffuse hashAlg lastBlock blockSize-        fillRandomBlock g blockPtr = do-            let (rand :: Bytes, g') = randomBytesGenerate blockSize g-            B.withByteArray rand $ \randPtr -> memCopy blockPtr randPtr blockSize-            return g'+  where+    diffusedLen = blockSize * expandTimes+    blockSize = B.length src+    runOp dstPtr = do+        let lastBlock = dstPtr `plusPtr` (blockSize * (expandTimes - 1))+        memSet lastBlock 0 blockSize+        let randomBlockPtrs = map (plusPtr dstPtr . (*) blockSize) [0 .. (expandTimes - 2)]+        rng' <- foldM fillRandomBlock rng randomBlockPtrs+        mapM_ (addRandomBlock lastBlock) randomBlockPtrs+        B.withByteArray src $ \srcPtr -> xorMem srcPtr lastBlock blockSize+        return rng'+    addRandomBlock lastBlock blockPtr = do+        xorMem blockPtr lastBlock blockSize+        diffuse hashAlg lastBlock blockSize+    fillRandomBlock g blockPtr = do+        let (rand :: Bytes, g') = randomBytesGenerate blockSize g+        B.withByteArray rand $ \randPtr -> memCopy blockPtr randPtr blockSize+        return g'  -- | Merge previously diffused data back to the original data.-merge :: (ByteArray ba, HashAlgorithm hash)-      => hash  -- ^ Hash algorithm used as diffuser-      -> Int   -- ^ Number of times to un-diffuse the data-      -> ba    -- ^ Diffused data-      -> ba    -- ^ Original data-{-# NOINLINE merge #-}-merge hashAlg expandTimes bs-    | r /= 0            = error "diffused data not a multiple of expandTimes"-    | originalSize <= 0 = error "diffused data null"-    | otherwise         = B.allocAndFreeze originalSize $ \dstPtr ->+--+-- The diffused data has to be a non-empty multiple of the number of times it+-- was diffused, and that number at least two -- the same values 'split'+-- accepts.  Anything else raises 'CryptoError_ParameterInvalid', which+-- 'tryMerge' reports as 'CryptoFailed' instead.+merge+    :: (ByteArray ba, HashAlgorithm hash)+    => hash+    -- ^ Hash algorithm used as diffuser+    -> Int+    -- ^ Number of times to un-diffuse the data+    -> ba+    -- ^ Diffused data+    -> ba+    -- ^ Original data+merge hashAlg expandTimes bs =+    throwCryptoError (tryMerge hashAlg expandTimes bs)++-- | Merge previously diffused data back to the original data, reporting+-- parameters the merger cannot work with rather than raising.+--+-- See 'merge'.+tryMerge+    :: (ByteArray ba, HashAlgorithm hash)+    => hash+    -- ^ Hash algorithm used as diffuser+    -> Int+    -- ^ Number of times to un-diffuse the data+    -> ba+    -- ^ Diffused data+    -> CryptoFailable ba+    -- ^ Original data+{-# NOINLINE tryMerge #-}+tryMerge hashAlg expandTimes bs+    -- guards the quotRem below, which for zero would divide by zero; a count+    -- of one would return the diffused data itself as the secret+    | expandTimes < 2 = CryptoFailed CryptoError_ParameterInvalid+    | r /= 0 = CryptoFailed CryptoError_ParameterInvalid+    | originalSize <= 0 = CryptoFailed CryptoError_ParameterInvalid+    | otherwise = CryptoPassed $ B.allocAndFreeze originalSize $ \dstPtr ->         B.withByteArray bs $ \srcPtr -> do             memSet dstPtr 0 originalSize-            forM_ [0..(expandTimes-2)] $ \i -> do+            forM_ [0 .. (expandTimes - 2)] $ \i -> do                 xorMem (srcPtr `plusPtr` (i * originalSize)) dstPtr originalSize                 diffuse hashAlg dstPtr originalSize-            xorMem (srcPtr `plusPtr` ((expandTimes-1) * originalSize)) dstPtr originalSize+            xorMem (srcPtr `plusPtr` ((expandTimes - 1) * originalSize)) dstPtr originalSize             return ()-  where (originalSize,r) = len `quotRem` expandTimes-        len              = B.length bs+  where+    (originalSize, r) = len `quotRem` expandTimes+    len = B.length bs  -- | inplace Xor with an input -- dst = src `xor` dst@@ -107,42 +172,51 @@ xorMem src dst sz     | sz `mod` 64 == 0 = loop 8 (castPtr src :: Ptr Word64) (castPtr dst) sz     | sz `mod` 32 == 0 = loop 4 (castPtr src :: Ptr Word32) (castPtr dst) sz-    | otherwise        = loop 1 (src :: Ptr Word8) dst sz-  where loop _    _ _ 0 = return ()-        loop incr s d n = do a <- peek s-                             b <- peek d-                             poke d (a `xor` b)-                             loop incr (s `plusPtr` incr) (d `plusPtr` incr) (n-incr)+    | otherwise = loop 1 (src :: Ptr Word8) dst sz+  where+    loop _ _ _ 0 = return ()+    loop incr s d n = do+        a <- peek s+        b <- peek d+        poke d (a `xor` b)+        loop incr (s `plusPtr` incr) (d `plusPtr` incr) (n - incr) -diffuse :: HashAlgorithm hash-        => hash      -- ^ Hash function to use as diffuser-        -> Ptr Word8 -- ^ buffer to diffuse, modify in place-        -> Int       -- ^ length of buffer to diffuse-        -> IO ()+diffuse+    :: HashAlgorithm hash+    => hash+    -- ^ Hash function to use as diffuser+    -> Ptr Word8+    -- ^ buffer to diffuse, modify in place+    -> Int+    -- ^ length of buffer to diffuse+    -> IO () diffuse hashAlg src sz = loop src 0-  where (full,pad) = sz `quotRem` digestSize -        loop s i-            | i < full = do h <- hashBlock i s digestSize-                            B.withByteArray h $ \hPtr -> memCopy s hPtr digestSize-                            loop (s `plusPtr` digestSize) (i+1)-            | pad /= 0 = do h <- hashBlock i s pad-                            B.withByteArray h $ \hPtr -> memCopy s hPtr pad-                            return ()-            | otherwise = return ()+  where+    (full, pad) = sz `quotRem` digestSize+    loop s i+        | i < full = do+            h <- hashBlock i s digestSize+            B.withByteArray h $ \hPtr -> memCopy s hPtr digestSize+            loop (s `plusPtr` digestSize) (i + 1)+        | pad /= 0 = do+            h <- hashBlock i s pad+            B.withByteArray h $ \hPtr -> memCopy s hPtr pad+            return ()+        | otherwise = return () -        digestSize = hashDigestSize hashAlg+    digestSize = hashDigestSize hashAlg -        -- Hash [ BE32(n), (p .. p+hashSz) ]-        hashBlock n p hashSz = do-            let ctx = hashInitWith hashAlg-            return $! hashFinalize $ hashUpdate (hashUpdate ctx (be32 n)) (MemView p hashSz)+    -- Hash [ BE32(n), (p .. p+hashSz) ]+    hashBlock n p hashSz = do+        let ctx = hashInitWith hashAlg+        return $! hashFinalize $ hashUpdate (hashUpdate ctx (be32 n)) (MemView p hashSz) -        be32 :: Int -> Bytes-        be32 n = B.allocAndFreeze 4 $ \ptr -> do-            poke ptr               (f8 (n `shiftR` 24))-            poke (ptr `plusPtr` 1) (f8 (n `shiftR` 16))-            poke (ptr `plusPtr` 2) (f8 (n `shiftR` 8))-            poke (ptr `plusPtr` 3) (f8 n)-          where-                f8 :: Int -> Word8-                f8 = fromIntegral+    be32 :: Int -> Bytes+    be32 n = B.allocAndFreeze 4 $ \ptr -> do+        poke ptr (f8 (n `shiftR` 24))+        poke (ptr `plusPtr` 1) (f8 (n `shiftR` 16))+        poke (ptr `plusPtr` 2) (f8 (n `shiftR` 8))+        poke (ptr `plusPtr` 3) (f8 n)+      where+        f8 :: Int -> Word8+        f8 = fromIntegral
Crypto/Data/Padding.hs view
@@ -7,59 +7,111 @@ -- -- Various cryptographic padding commonly used for block ciphers -- or asymmetric systems.----module Crypto.Data.Padding-    ( Format(..)-    , pad-    , unpad-    ) where+module Crypto.Data.Padding (+    Format (..),+    pad,+    unpad,+) where -import           Data.ByteArray (ByteArray, Bytes)+import Data.ByteArray (ByteArray, Bytes) import qualified Data.ByteArray as B  -- | Format of padding-data Format =-      PKCS5     -- ^ PKCS5: PKCS7 with hardcoded size of 8-    | PKCS7 Int -- ^ PKCS7 with padding size between 1 and 255-    | ZERO Int  -- ^ zero padding with block size+data Format+    = -- | PKCS5: PKCS7 with hardcoded size of 8+      PKCS5+    | -- | PKCS7 with padding size between 1 and 255+      PKCS7 Int+    | -- | Zero padding with block size, which must be at least 1.+      --+      -- Zero padding does not say how much of it there is, so 'unpad' cannot+      -- undo 'pad': see 'unpad'.+      ZERO Int     deriving (Show, Eq) +-- | Is this a block size PKCS7 can describe?+--+-- The padding octet carries the number of octets added, so it cannot describe+-- a block longer than 255, and a block of zero has nothing to describe.+-- Outside that range the octet would be computed as an 'Int' and then narrowed+-- to a 'Data.Word.Word8', which wraps: 'pad' and 'unpad' would agree on the+-- wrapped value and hand back something other than what was padded.+pkcs7SizeValid :: Int -> Bool+pkcs7SizeValid sz = sz >= 1 && sz <= 255++-- | Is this a block size 'ZERO' can use?+--+-- Nothing is written into the padding, so there is no upper bound to match+-- the one 'PKCS7' has; but a block of zero or fewer octets is not a block,+-- and the length is taken modulo it.+zeroSizeValid :: Int -> Bool+zeroSizeValid sz = sz >= 1+ -- | Apply some pad to a bytearray+--+-- A 'PKCS7' block size outside 1..255, or a 'ZERO' block size below 1, raises+-- an 'error'; 'unpad' reports the same condition as 'Nothing'. pad :: ByteArray byteArray => Format -> byteArray -> byteArray-pad  PKCS5     bin = pad (PKCS7 8) bin-pad (PKCS7 sz) bin = bin `B.append` paddingString+pad PKCS5 bin = pad (PKCS7 8) bin+pad (PKCS7 sz) bin+    | not (pkcs7SizeValid sz) =+        error $+            "Crypto.Data.Padding: PKCS7 block size "+                ++ show sz+                ++ " is not between 1 and 255"+    | otherwise = bin `B.append` paddingString   where     paddingString = B.replicate paddingByte (fromIntegral paddingByte)-    paddingByte   = sz - (B.length bin `mod` sz)-pad (ZERO sz)  bin = bin `B.append` paddingString+    paddingByte = sz - (B.length bin `mod` sz)+pad (ZERO sz) bin+    | not (zeroSizeValid sz) =+        error $+            "Crypto.Data.Padding: ZERO block size "+                ++ show sz+                ++ " is not at least 1"+    | otherwise = bin `B.append` paddingString   where     paddingString = B.replicate paddingSz 0     paddingSz-      | len == 0   =  sz-      | m == 0     =  0-      | otherwise  =  sz - m+        | len == 0 = sz+        | m == 0 = 0+        | otherwise = sz - m     m = len `mod` sz     len = B.length bin  -- | Try to remove some padding from a bytearray.+--+-- 'PKCS7' padding says how long it is, so this undoes 'pad' exactly.+--+-- 'ZERO' padding says nothing, and 'pad' adds none at all when the input is+-- already a multiple of the block size, so there is no way to tell padding+-- from data that happens to end in zero octets.  This therefore does not undo+-- 'pad': it returns the input unchanged when the last octet is not zero, and+-- 'Nothing' when it is, rather than guess and hand back less than it was+-- given.  Zero padding is only usable where the original length is known by+-- other means. unpad :: ByteArray byteArray => Format -> byteArray -> Maybe byteArray-unpad  PKCS5     bin = unpad (PKCS7 8) bin+unpad PKCS5 bin = unpad (PKCS7 8) bin unpad (PKCS7 sz) bin-    | len == 0                           = Nothing-    | (len `mod` sz) /= 0                = Nothing-    | paddingSz < 1 || paddingSz > len   = Nothing+    | not (pkcs7SizeValid sz) = Nothing+    | len == 0 = Nothing+    | (len `mod` sz) /= 0 = Nothing+    -- the padded length is a multiple of the block size and the padding is+    -- what was added to reach it, so it is never more than one block+    | paddingSz < 1 || paddingSz > sz = Nothing     | paddingWitness `B.constEq` padding = Just content-    | otherwise                          = Nothing+    | otherwise = Nothing   where-    len         = B.length bin+    len = B.length bin     paddingByte = B.index bin (len - 1)-    paddingSz   = fromIntegral paddingByte+    paddingSz = fromIntegral paddingByte     (content, padding) = B.splitAt (len - paddingSz) bin-    paddingWitness     = B.replicate paddingSz paddingByte :: Bytes-unpad (ZERO sz)  bin-    | len == 0                           = Nothing-    | (len `mod` sz) /= 0                = Nothing-    | B.index bin (len - 1) /= 0         = Just bin-    | otherwise                          = Nothing+    paddingWitness = B.replicate paddingSz paddingByte :: Bytes+unpad (ZERO sz) bin+    | not (zeroSizeValid sz) = Nothing+    | len == 0 = Nothing+    | (len `mod` sz) /= 0 = Nothing+    | B.index bin (len - 1) /= 0 = Just bin+    | otherwise = Nothing   where-    len         = B.length bin+    len = B.length bin
+ Crypto/Debug.hs view
@@ -0,0 +1,52 @@+-- |+-- Module      : Crypto.Debug+-- License     : BSD-style+-- Maintainer  : Kazu Yamamoto <kazu@iij.ad.jp>+-- Stability   : experimental+-- Portability : unknown+--+-- Printing secret key material, on purpose.+--+-- The 'Show' instance of a type that holds a secret does not print it.  That+-- is deliberate: 'Show' is what @print@, a message built with @error@, an+-- exception and a test framework's failure output all reach for, and a+-- private key reaching a log or a bug report that way is an accident nobody+-- asked for.  Those instances render the public part and write @\<secret\>@+-- for the rest.+--+-- This module is how you print one when printing it is what you mean.  What+-- 'debugShow' returns is what the derived 'Show' used to return, so for the+-- types that still have a 'Read' instance+--+-- > read (debugShow k) == k+--+-- and a call site that was serializing a key through @show@ moves by one+-- word.+--+-- Needing 'debugShow' in scope is the record of the intent: nothing here is+-- exported anywhere else, so a search for this module finds every place a key+-- can be revealed.  Do not leave a call to it where production code runs.+module Crypto.Debug (+    DebugShow (..),+    debugShowBytes,+) where++import Data.Bits (shiftR, (.&.))+import qualified Data.ByteArray as BA+import Data.Word (Word8)++-- | Rendering a value with its secret in place.+class DebugShow a where+    -- | Render the value, secret included.+    debugShow :: a -> String++-- | Render a secret that is held as bytes, in hexadecimal.  The secret keys+-- that keep theirs in a @ScrubbedBytes@ never had a 'Show' that printed it,+-- so unlike the rest of this module what comes back is for reading and not+-- for 'Prelude.read'.+debugShowBytes :: BA.ByteArrayAccess ba => String -> ba -> String+debugShowBytes con b = con ++ (' ' : concatMap hex (BA.unpack b))+  where+    hex :: Word8 -> String+    hex w = [digit (w `shiftR` 4), digit (w .&. 0x0f)]+    digit n = "0123456789abcdef" !! fromIntegral n
Crypto/ECC.hs view
@@ -1,3 +1,9 @@+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.ECC -- License     : BSD-style@@ -7,68 +13,93 @@ -- -- Elliptic Curve Cryptography ---{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE FlexibleContexts #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-{-# LANGUAGE TypeFamilies #-}-{-# LANGUAGE ScopedTypeVariables #-}-module Crypto.ECC-    ( Curve_P256R1(..)-    , Curve_P384R1(..)-    , Curve_P521R1(..)-    , Curve_X25519(..)-    , Curve_X448(..)-    , Curve_Edwards25519(..)-    , EllipticCurve(..)-    , EllipticCurveDH(..)-    , EllipticCurveArith(..)-    , EllipticCurveBasepointArith(..)-    , KeyPair(..)-    , SharedSecret(..)-    ) where+-- == Timing+--+-- t'Curve_P256R1' reaches a dedicated implementation whose scalar+-- multiplication does not branch on the scalar.  t'Curve_P384R1' and+-- t'Curve_P521R1' do not: they are built on "Crypto.ECC.Simple.Prim", whose+-- scalar multiplication is a double-and-add over @Integer@ and is+-- documented there as vulnerable to timing attacks.+--+-- That matters wherever the scalar is secret, which is both operations that+-- have one: 'ecdh', which multiplies by the private key, and ECDSA signing,+-- which multiplies by the secret nonce.  Verification and public-key+-- derivation work on values an attacker already has, so they are unaffected.+--+-- Note also that @Integer@ arithmetic is variable-time underneath, so no+-- curve built on "Crypto.ECC.Simple.Prim" can be made constant-time without+-- leaving it.  Where that matters, use t'Curve_P256R1', t'Curve_X25519',+-- t'Curve_X448' or t'Curve_Edwards25519'.+module Crypto.ECC (+    Curve_P256R1 (..),+    Curve_P384R1 (..),+    Curve_P521R1 (..),+    Curve_X25519 (..),+    Curve_X448 (..),+    Curve_Edwards25519 (..),+    EllipticCurve (..),+    EllipticCurveDH (..),+    EllipticCurveArith (..),+    EllipticCurveBasepointArith (..),+    KeyPair (..),+    SharedSecret (..),+) where -import qualified Crypto.PubKey.ECC.P256 as P256 import qualified Crypto.ECC.Edwards25519 as Edwards25519-import qualified Crypto.ECC.Simple.Types as Simple import qualified Crypto.ECC.Simple.Prim as Simple-import           Crypto.Random-import           Crypto.Error-import           Crypto.Internal.Imports-import           Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, ScrubbedBytes)+import qualified Crypto.ECC.Simple.Types as Simple+import Crypto.Error+import Crypto.Internal.ByteArray (+    ByteArray,+    ByteArrayAccess,+    ScrubbedBytes,+ ) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Number.Basic (numBits)-import           Crypto.Number.Serialize (i2ospOf_, os2ip)+import Crypto.Internal.Imports+import Crypto.Number.Basic (numBits)+import Crypto.Number.Serialize (i2ospOf_, os2ip) import qualified Crypto.Number.Serialize.LE as LE import qualified Crypto.PubKey.Curve25519 as X25519 import qualified Crypto.PubKey.Curve448 as X448-import           Data.ByteArray (convert)-import           Data.Data (Data())-import           Data.Kind (Type)-import           Data.Proxy+import qualified Crypto.PubKey.ECC.P256 as P256+import Crypto.Random+import Data.ByteArray (convert)+import Data.Data (Data ())+import Data.Kind (Type)+import Data.Proxy  -- | An elliptic curve key pair composed of the private part (a scalar), and -- the associated point. data KeyPair curve = KeyPair-    { keypairGetPublic  :: !(Point curve)+    { keypairGetPublic :: !(Point curve)     , keypairGetPrivate :: !(Scalar curve)     } +-- | Secret shared via key exchange newtype SharedSecret = SharedSecret ScrubbedBytes     deriving (Eq, ByteArrayAccess, NFData) +instance Semigroup SharedSecret where+    SharedSecret x <> SharedSecret y = SharedSecret (x <> y)++instance Monoid SharedSecret where+    mempty = SharedSecret mempty+ class EllipticCurve curve where     -- | Point on an Elliptic Curve-    type Point curve  :: Type+    type Point curve :: Type      -- | Scalar in the Elliptic Curve domain     type Scalar curve :: Type      -- | Generate a new random scalar on the curve.     -- The scalar will represent a number between 1 and the order of the curve non included-    curveGenerateScalar :: MonadRandom randomly => proxy curve -> randomly (Scalar curve)+    curveGenerateScalar+        :: MonadRandom randomly => proxy curve -> randomly (Scalar curve)      -- | Generate a new random keypair-    curveGenerateKeyPair :: MonadRandom randomly => proxy curve -> randomly (KeyPair curve)+    curveGenerateKeyPair+        :: MonadRandom randomly => proxy curve -> randomly (KeyPair curve)      -- | Get the curve size in bits     curveSizeBits :: proxy curve -> Int@@ -79,6 +110,15 @@     -- | Try to decode the binary form of an elliptic curve point     decodePoint :: ByteArray bs => proxy curve -> bs -> CryptoFailable (Point curve) +    -- | Encode an elliptic curve scalar into big-endian form+    encodeScalar :: ByteArray bs => proxy curve -> Scalar curve -> bs++    -- | Try to decode the big-endian form of an elliptic curve scalar+    decodeScalar+        :: ByteArray bs => proxy curve -> bs -> CryptoFailable (Scalar curve)++    scalarToPoint :: proxy curve -> Scalar curve -> Point curve+ class EllipticCurve curve => EllipticCurveDH curve where     -- | Generate a Diffie hellman secret value.     --@@ -100,7 +140,8 @@     -- This additional test avoids risks existing with function 'ecdhRaw'.     -- Implementations always return a 'CryptoError' instead of a special     -- value or an exception.-    ecdh :: proxy curve -> Scalar curve -> Point curve -> CryptoFailable SharedSecret+    ecdh+        :: proxy curve -> Scalar curve -> Point curve -> CryptoFailable SharedSecret  class (EllipticCurve curve, Eq (Point curve)) => EllipticCurveArith curve where     -- | Add points on a curve@@ -115,7 +156,10 @@ --   -- | Scalar Inverse --   scalarInverse :: Scalar curve -> Scalar curve -class (EllipticCurveArith curve, Eq (Scalar curve)) => EllipticCurveBasepointArith curve where+class+    (EllipticCurveArith curve, Eq (Scalar curve)) =>+    EllipticCurveBasepointArith curve+    where     -- | Get the curve order size in bits     curveOrderBits :: proxy curve -> Int @@ -123,15 +167,10 @@     pointBaseSmul :: proxy curve -> Scalar curve -> Point curve      -- | Multiply the point @p@ with @s2@ and add a lifted to curve value @s1@-    pointsSmulVarTime :: proxy curve -> Scalar curve -> Scalar curve -> Point curve -> Point curve+    pointsSmulVarTime+        :: proxy curve -> Scalar curve -> Scalar curve -> Point curve -> Point curve     pointsSmulVarTime prx s1 s2 p = pointAdd prx (pointBaseSmul prx s1) (pointSmul prx s2 p) -    -- | Encode an elliptic curve scalar into big-endian form-    encodeScalar :: ByteArray bs => proxy curve -> Scalar curve -> bs--    -- | Try to decode the big-endian form of an elliptic curve scalar-    decodeScalar :: ByteArray bs => proxy curve -> bs -> CryptoFailable (Scalar curve)-     -- | Convert an elliptic curve scalar to an integer     scalarToInteger :: proxy curve -> Scalar curve -> Integer @@ -148,7 +187,7 @@ -- -- also known as P256 data Curve_P256R1 = Curve_P256R1-    deriving (Show,Data)+    deriving (Show, Data)  instance EllipticCurve Curve_P256R1 where     type Point Curve_P256R1 = P256.Point@@ -156,7 +195,8 @@     curveSizeBits _ = 256     curveGenerateScalar _ = P256.scalarGenerate     curveGenerateKeyPair _ = toKeyPair <$> P256.scalarGenerate-      where toKeyPair scalar = KeyPair (P256.toPoint scalar) scalar+      where+        toKeyPair scalar = KeyPair (P256.toPoint scalar) scalar     encodePoint _ p = mxy       where         mxy :: forall bs. ByteArray bs => bs@@ -167,33 +207,47 @@             xy = P256.pointToBinary p     decodePoint _ mxy = case B.uncons mxy of         Nothing -> CryptoFailed CryptoError_PointSizeInvalid-        Just (m,xy)+        Just (m, xy)             -- uncompressed             | m == 4 -> P256.pointFromBinary xy             | otherwise -> CryptoFailed CryptoError_PointFormatInvalid+    encodeScalar _ = P256.scalarToBinary+    decodeScalar _ = P256.scalarFromBinary+    scalarToPoint _ = P256.toPoint  instance EllipticCurveArith Curve_P256R1 where-    pointAdd  _ a b = P256.pointAdd a b+    pointAdd _ a b = P256.pointAdd a b     pointNegate _ p = P256.pointNegate p     pointSmul _ s p = P256.pointMul s p  instance EllipticCurveDH Curve_P256R1 where     ecdhRaw _ s p = SharedSecret $ P256.pointDh s p-    ecdh  prx s p = checkNonZeroDH (ecdhRaw prx s p) +    -- An all-zero x-coordinate can be valid. Since P-256's group has prime+    -- order n, s * P is the identity only when P is the identity or the+    -- 256-bit scalar s is zero or n.+    ecdh _ s p+        | P256.pointIsAtInfinity p+            || P256.scalarIsZero s+            || P256.scalarCmp s P256.scalarN == EQ =+            CryptoFailed CryptoError_ScalarMultiplicationInvalid+        | otherwise = CryptoPassed $ SharedSecret $ P256.pointDh s p+ instance EllipticCurveBasepointArith Curve_P256R1 where     curveOrderBits _ = 256     pointBaseSmul _ = P256.toPoint     pointsSmulVarTime _ = P256.pointsMulVarTime-    encodeScalar _ = P256.scalarToBinary-    decodeScalar _ = P256.scalarFromBinary     scalarToInteger _ = P256.scalarToInteger     scalarFromInteger _ = P256.scalarFromInteger     scalarAdd _ = P256.scalarAdd     scalarMul _ = P256.scalarMul +-- | NIST P-384.+--+-- Scalar multiplication branches on the scalar; see the note on timing+-- at the head of this module. data Curve_P384R1 = Curve_P384R1-    deriving (Show,Data)+    deriving (Show, Data)  instance EllipticCurve Curve_P384R1 where     type Point Curve_P384R1 = Simple.Point Simple.SEC_p384r1@@ -201,9 +255,13 @@     curveSizeBits _ = 384     curveGenerateScalar _ = Simple.scalarGenerate     curveGenerateKeyPair _ = toKeyPair <$> Simple.scalarGenerate-      where toKeyPair scalar = KeyPair (Simple.pointBaseMul scalar) scalar+      where+        toKeyPair scalar = KeyPair (Simple.pointBaseMul scalar) scalar     encodePoint _ point = encodeECPoint point     decodePoint _ bs = decodeECPoint bs+    encodeScalar _ = ecScalarToBinary+    decodeScalar _ = ecScalarFromBinary+    scalarToPoint _ = Simple.pointBaseMul  instance EllipticCurveArith Curve_P384R1 where     pointAdd _ a b = Simple.pointAdd a b@@ -219,15 +277,17 @@     curveOrderBits _ = 384     pointBaseSmul _ = Simple.pointBaseMul     pointsSmulVarTime _ = ecPointsMulVarTime-    encodeScalar _ = ecScalarToBinary-    decodeScalar _ = ecScalarFromBinary     scalarToInteger _ = ecScalarToInteger     scalarFromInteger _ = ecScalarFromInteger     scalarAdd _ = ecScalarAdd     scalarMul _ = ecScalarMul +-- | NIST P-521.+--+-- Scalar multiplication branches on the scalar; see the note on timing+-- at the head of this module. data Curve_P521R1 = Curve_P521R1-    deriving (Show,Data)+    deriving (Show, Data)  instance EllipticCurve Curve_P521R1 where     type Point Curve_P521R1 = Simple.Point Simple.SEC_p521r1@@ -235,9 +295,13 @@     curveSizeBits _ = 521     curveGenerateScalar _ = Simple.scalarGenerate     curveGenerateKeyPair _ = toKeyPair <$> Simple.scalarGenerate-      where toKeyPair scalar = KeyPair (Simple.pointBaseMul scalar) scalar+      where+        toKeyPair scalar = KeyPair (Simple.pointBaseMul scalar) scalar     encodePoint _ point = encodeECPoint point     decodePoint _ bs = decodeECPoint bs+    encodeScalar _ = ecScalarToBinary+    decodeScalar _ = ecScalarFromBinary+    scalarToPoint _ = Simple.pointBaseMul  instance EllipticCurveArith Curve_P521R1 where     pointAdd _ a b = Simple.pointAdd a b@@ -253,15 +317,13 @@     curveOrderBits _ = 521     pointBaseSmul _ = Simple.pointBaseMul     pointsSmulVarTime _ = ecPointsMulVarTime-    encodeScalar _ = ecScalarToBinary-    decodeScalar _ = ecScalarFromBinary     scalarToInteger _ = ecScalarToInteger     scalarFromInteger _ = ecScalarFromInteger     scalarAdd _ = ecScalarAdd     scalarMul _ = ecScalarMul  data Curve_X25519 = Curve_X25519-    deriving (Show,Data)+    deriving (Show, Data)  instance EllipticCurve Curve_X25519 where     type Point Curve_X25519 = X25519.PublicKey@@ -273,14 +335,18 @@         return $ KeyPair (X25519.toPublic s) s     encodePoint _ p = B.convert p     decodePoint _ bs = X25519.publicKey bs+    encodeScalar _ s = convert s+    decodeScalar _ bs = X25519.secretKey bs+    scalarToPoint _ s = X25519.toPublic s  instance EllipticCurveDH Curve_X25519 where     ecdhRaw _ s p = SharedSecret $ convert secret-      where secret = X25519.dh p s+      where+        secret = X25519.dh p s     ecdh prx s p = checkNonZeroDH (ecdhRaw prx s p)  data Curve_X448 = Curve_X448-    deriving (Show,Data)+    deriving (Show, Data)  instance EllipticCurve Curve_X448 where     type Point Curve_X448 = X448.PublicKey@@ -292,14 +358,18 @@         return $ KeyPair (X448.toPublic s) s     encodePoint _ p = B.convert p     decodePoint _ bs = X448.publicKey bs+    encodeScalar _ s = convert s+    decodeScalar _ bs = X448.secretKey bs+    scalarToPoint _ s = X448.toPublic s  instance EllipticCurveDH Curve_X448 where     ecdhRaw _ s p = SharedSecret $ convert secret-      where secret = X448.dh p s+      where+        secret = X448.dh p s     ecdh prx s p = checkNonZeroDH (ecdhRaw prx s p)  data Curve_Edwards25519 = Curve_Edwards25519-    deriving (Show,Data)+    deriving (Show, Data)  instance EllipticCurve Curve_Edwards25519 where     type Point Curve_Edwards25519 = Edwards25519.Point@@ -307,9 +377,15 @@     curveSizeBits _ = 255     curveGenerateScalar _ = Edwards25519.scalarGenerate     curveGenerateKeyPair _ = toKeyPair <$> Edwards25519.scalarGenerate-      where toKeyPair scalar = KeyPair (Edwards25519.toPoint scalar) scalar+      where+        toKeyPair scalar = KeyPair (Edwards25519.toPoint scalar) scalar     encodePoint _ point = Edwards25519.pointEncode point     decodePoint _ bs = Edwards25519.pointDecode bs+    encodeScalar _ = B.reverse . Edwards25519.scalarEncode+    decodeScalar _ bs+        | B.length bs == 32 = Edwards25519.scalarDecodeLong (B.reverse bs)+        | otherwise = CryptoFailed CryptoError_SecretKeySizeInvalid+    scalarToPoint _ = Edwards25519.toPoint  instance EllipticCurveArith Curve_Edwards25519 where     pointAdd _ a b = Edwards25519.pointAdd a b@@ -320,10 +396,6 @@     curveOrderBits _ = 253     pointBaseSmul _ = Edwards25519.toPoint     pointsSmulVarTime _ = Edwards25519.pointsMulVarTime-    encodeScalar _ = B.reverse . Edwards25519.scalarEncode-    decodeScalar _ bs-        | B.length bs == 32 = Edwards25519.scalarDecodeLong (B.reverse bs)-        | otherwise         = CryptoFailed CryptoError_SecretKeySizeInvalid     scalarToInteger _ s = LE.os2ip (Edwards25519.scalarEncode s :: B.Bytes)     scalarFromInteger _ i =         case LE.i2ospOf 32 i of@@ -335,15 +407,18 @@ checkNonZeroDH :: SharedSecret -> CryptoFailable SharedSecret checkNonZeroDH s@(SharedSecret b)     | B.constAllZero b = CryptoFailed CryptoError_ScalarMultiplicationInvalid-    | otherwise        = CryptoPassed s+    | otherwise = CryptoPassed s -encodeECShared :: Simple.Curve curve => Proxy curve -> Simple.Point curve -> CryptoFailable SharedSecret-encodeECShared _   Simple.PointO      = CryptoFailed CryptoError_ScalarMultiplicationInvalid+encodeECShared+    :: Simple.Curve curve+    => Proxy curve -> Simple.Point curve -> CryptoFailable SharedSecret+encodeECShared _ Simple.PointO = CryptoFailed CryptoError_ScalarMultiplicationInvalid encodeECShared prx (Simple.Point x _) = CryptoPassed . SharedSecret $ i2ospOf_ (Simple.curveSizeBytes prx) x -encodeECPoint :: forall curve bs . (Simple.Curve curve, ByteArray bs) => Simple.Point curve -> bs-encodeECPoint Simple.PointO      = error "encodeECPoint: cannot serialize point at infinity"-encodeECPoint (Simple.Point x y) = B.concat [uncompressed,xb,yb]+encodeECPoint+    :: forall curve bs. (Simple.Curve curve, ByteArray bs) => Simple.Point curve -> bs+encodeECPoint Simple.PointO = error "encodeECPoint: cannot serialize point at infinity"+encodeECPoint (Simple.Point x y) = B.concat [uncompressed, xb, yb]   where     size = Simple.curveSizeBytes (Proxy :: Proxy curve)     uncompressed, xb, yb :: bs@@ -351,58 +426,79 @@     xb = i2ospOf_ size x     yb = i2ospOf_ size y -decodeECPoint :: (Simple.Curve curve, ByteArray bs) => bs -> CryptoFailable (Simple.Point curve)+decodeECPoint+    :: (Simple.Curve curve, ByteArray bs) => bs -> CryptoFailable (Simple.Point curve) decodeECPoint mxy = case B.uncons mxy of-    Nothing     -> CryptoFailed CryptoError_PointSizeInvalid-    Just (m,xy)+    Nothing -> CryptoFailed CryptoError_PointSizeInvalid+    Just (m, xy)         -- uncompressed         | m == 4 ->             let siz = B.length xy `div` 2-                (xb,yb) = B.splitAt siz xy+                (xb, yb) = B.splitAt siz xy                 x = os2ip xb                 y = os2ip yb-             in Simple.pointFromIntegers (x,y)+             in Simple.pointFromIntegers (x, y)         | otherwise -> CryptoFailed CryptoError_PointFormatInvalid -ecPointsMulVarTime :: forall curve . Simple.Curve curve-                   => Simple.Scalar curve-                   -> Simple.Scalar curve -> Simple.Point curve-                   -> Simple.Point curve+ecPointsMulVarTime+    :: forall curve+     . Simple.Curve curve+    => Simple.Scalar curve+    -> Simple.Scalar curve+    -> Simple.Point curve+    -> Simple.Point curve ecPointsMulVarTime n1 = Simple.pointAddTwoMuls n1 g-  where g = Simple.curveEccG $ Simple.curveParameters (Proxy :: Proxy curve)+  where+    g = Simple.curveEccG $ Simple.curveParameters (Proxy :: Proxy curve) -ecScalarFromBinary :: forall curve bs . (Simple.Curve curve, ByteArrayAccess bs)-                   => bs -> CryptoFailable (Simple.Scalar curve)+ecScalarFromBinary+    :: forall curve bs+     . (Simple.Curve curve, ByteArrayAccess bs)+    => bs -> CryptoFailable (Simple.Scalar curve) ecScalarFromBinary ba     | B.length ba /= size = CryptoFailed CryptoError_SecretKeySizeInvalid-    | otherwise           = CryptoPassed (Simple.Scalar $ os2ip ba)-  where size = ecCurveOrderBytes (Proxy :: Proxy curve)+    | otherwise = CryptoPassed (Simple.Scalar $ os2ip ba)+  where+    size = ecCurveOrderBytes (Proxy :: Proxy curve) -ecScalarToBinary :: forall curve bs . (Simple.Curve curve, ByteArray bs)-                 => Simple.Scalar curve -> bs+ecScalarToBinary+    :: forall curve bs+     . (Simple.Curve curve, ByteArray bs)+    => Simple.Scalar curve -> bs ecScalarToBinary (Simple.Scalar s) = i2ospOf_ size s-  where size = ecCurveOrderBytes (Proxy :: Proxy curve)+  where+    size = ecCurveOrderBytes (Proxy :: Proxy curve) -ecScalarFromInteger :: forall curve . Simple.Curve curve-                    => Integer -> CryptoFailable (Simple.Scalar curve)+ecScalarFromInteger+    :: forall curve+     . Simple.Curve curve+    => Integer -> CryptoFailable (Simple.Scalar curve) ecScalarFromInteger s     | numBits s > nb = CryptoFailed CryptoError_SecretKeySizeInvalid-    | otherwise      = CryptoPassed (Simple.Scalar s)-  where nb = 8 * ecCurveOrderBytes (Proxy :: Proxy curve)+    | otherwise = CryptoPassed (Simple.Scalar s)+  where+    nb = 8 * ecCurveOrderBytes (Proxy :: Proxy curve)  ecScalarToInteger :: Simple.Scalar curve -> Integer ecScalarToInteger (Simple.Scalar s) = s  ecCurveOrderBytes :: Simple.Curve c => proxy c -> Int ecCurveOrderBytes prx = (numBits n + 7) `div` 8-  where n = Simple.curveEccN $ Simple.curveParameters prx+  where+    n = Simple.curveEccN $ Simple.curveParameters prx -ecScalarAdd :: forall curve . Simple.Curve curve-            => Simple.Scalar curve -> Simple.Scalar curve -> Simple.Scalar curve+ecScalarAdd+    :: forall curve+     . Simple.Curve curve+    => Simple.Scalar curve -> Simple.Scalar curve -> Simple.Scalar curve ecScalarAdd (Simple.Scalar a) (Simple.Scalar b) = Simple.Scalar ((a + b) `mod` n)-  where n = Simple.curveEccN $ Simple.curveParameters (Proxy :: Proxy curve)+  where+    n = Simple.curveEccN $ Simple.curveParameters (Proxy :: Proxy curve) -ecScalarMul :: forall curve . Simple.Curve curve-            => Simple.Scalar curve -> Simple.Scalar curve -> Simple.Scalar curve+ecScalarMul+    :: forall curve+     . Simple.Curve curve+    => Simple.Scalar curve -> Simple.Scalar curve -> Simple.Scalar curve ecScalarMul (Simple.Scalar a) (Simple.Scalar b) = Simple.Scalar ((a * b) `mod` n)-  where n = Simple.curveEccN $ Simple.curveParameters (Proxy :: Proxy curve)+  where+    n = Simple.curveEccN $ Simple.curveParameters (Proxy :: Proxy curve)
Crypto/ECC/Edwards25519.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.ECC.Edwards25519 -- License     : BSD-style@@ -48,55 +50,55 @@ -- 3. Because of modular reduction in this implementation it is not -- possible to multiply points directly by scalars like 8.s or L. -- This has to be decomposed into several steps.----{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.ECC.Edwards25519-    ( Scalar-    , Point+module Crypto.ECC.Edwards25519 (+    Scalar,+    Point,+     -- * Scalars-    , scalarGenerate-    , scalarDecodeLong-    , scalarEncode+    scalarGenerate,+    scalarDecodeLong,+    scalarEncode,+     -- * Points-    , pointDecode-    , pointEncode-    , pointHasPrimeOrder+    pointDecode,+    pointEncode,+    pointHasPrimeOrder,+     -- * Arithmetic functions-    , toPoint-    , scalarAdd-    , scalarMul-    , pointNegate-    , pointAdd-    , pointDouble-    , pointMul-    , pointMulByCofactor-    , pointsMulVarTime-    ) where+    toPoint,+    scalarAdd,+    scalarMul,+    pointNegate,+    pointAdd,+    pointDouble,+    pointMul,+    pointMulByCofactor,+    pointsMulVarTime,+) where -import           Data.Word-import           Foreign.C.Types-import           Foreign.Ptr+import Data.Word+import Foreign.C.Types+import Foreign.Ptr -import           Crypto.Error-import           Crypto.Internal.ByteArray (Bytes, ScrubbedBytes, withByteArray)+import Crypto.Error+import Crypto.Internal.ByteArray (Bytes, ScrubbedBytes, withByteArray) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Internal.Compat-import           Crypto.Internal.Imports-import           Crypto.Random-+import Crypto.Internal.Compat+import Crypto.Internal.Imports+import Crypto.Random  scalarArraySize :: Int scalarArraySize = 40 -- maximum [9 * 4 {- 32 bits -}, 5 * 8 {- 64 bits -}]  -- | A scalar modulo prime order of curve edwards25519. newtype Scalar = Scalar ScrubbedBytes-    deriving (Show,NFData)+    deriving (Show, NFData)  instance Eq Scalar where     (Scalar s1) == (Scalar s2) = unsafeDoIO $         withByteArray s1 $ \ps1 ->-        withByteArray s2 $ \ps2 ->-            fmap (/= 0) (ed25519_scalar_eq ps1 ps2)+            withByteArray s2 $ \ps2 ->+                fmap (/= 0) (ed25519_scalar_eq ps1 ps2)     {-# NOINLINE (==) #-}  pointArraySize :: Int@@ -104,19 +106,20 @@  -- | A point on curve edwards25519. newtype Point = Point Bytes-    deriving NFData+    deriving (NFData)  instance Show Point where     showsPrec d p =         let bs = pointEncode p :: Bytes-         in showParen (d > 10) $ showString "Point "-                               . shows (B.convertToBase B.Base16 bs :: Bytes)+         in showParen (d > 10) $+                showString "Point "+                    . shows (B.convertToBase B.Base16 bs :: Bytes)  instance Eq Point where     (Point p1) == (Point p2) = unsafeDoIO $         withByteArray p1 $ \pp1 ->-        withByteArray p2 $ \pp2 ->-            fmap (/= 0) (ed25519_point_eq pp1 pp2)+            withByteArray p2 $ \pp2 ->+                fmap (/= 0) (ed25519_point_eq pp1 pp2)     {-# NOINLINE (==) #-}  -- | Generate a random scalar.@@ -148,12 +151,12 @@ scalarDecodeLong :: B.ByteArrayAccess bs => bs -> CryptoFailable Scalar scalarDecodeLong bs     | B.length bs > 64 = CryptoFailed CryptoError_EcScalarOutOfBounds-    | otherwise        = unsafeDoIO $ withByteArray bs initialize+    | otherwise = unsafeDoIO $ withByteArray bs initialize   where     len = fromIntegral $ B.length bs     initialize inp = do         s <- B.alloc scalarArraySize $ \ps ->-                 ed25519_scalar_decode_long ps inp len+            ed25519_scalar_decode_long ps inp len         return $ CryptoPassed (Scalar s) {-# NOINLINE scalarDecodeLong #-} @@ -162,16 +165,16 @@ scalarAdd (Scalar a) (Scalar b) =     Scalar $ B.allocAndFreeze scalarArraySize $ \out ->         withByteArray a $ \pa ->-        withByteArray b $ \pb ->-             ed25519_scalar_add out pa pb+            withByteArray b $ \pb ->+                ed25519_scalar_add out pa pb  -- | Multiply two scalars. scalarMul :: Scalar -> Scalar -> Scalar scalarMul (Scalar a) (Scalar b) =     Scalar $ B.allocAndFreeze scalarArraySize $ \out ->         withByteArray a $ \pa ->-        withByteArray b $ \pb ->-             ed25519_scalar_mul out pa pb+            withByteArray b $ \pb ->+                ed25519_scalar_mul out pa pb  -- | Multiplies a scalar with the curve base point. toPoint :: Scalar -> Point@@ -188,7 +191,7 @@ pointEncode (Point p) =     B.allocAndFreeze 32 $ \out ->         withByteArray p $ \pp ->-             ed25519_point_encode out pp+            ed25519_point_encode out pp  -- | Deserialize a 32-byte array as a point, ensuring the point is -- valid on edwards25519.@@ -197,13 +200,14 @@ pointDecode :: B.ByteArrayAccess bs => bs -> CryptoFailable Point pointDecode bs     | B.length bs == 32 = unsafeDoIO $ withByteArray bs initialize-    | otherwise         = CryptoFailed CryptoError_PointSizeInvalid+    | otherwise = CryptoFailed CryptoError_PointSizeInvalid   where     initialize inp = do         (res, p) <- B.allocRet pointArraySize $ \pp ->-                        ed25519_point_decode_vartime pp inp-        if res == 0 then return $ CryptoFailed CryptoError_PointCoordinatesInvalid-                    else return $ CryptoPassed (Point p)+            ed25519_point_decode_vartime pp inp+        if res == 0+            then return $ CryptoFailed CryptoError_PointCoordinatesInvalid+            else return $ CryptoPassed (Point p) {-# NOINLINE pointDecode #-}  -- | Test whether a point belongs to the prime-order subgroup@@ -224,15 +228,15 @@ pointNegate (Point a) =     Point $ B.allocAndFreeze pointArraySize $ \out ->         withByteArray a $ \pa ->-             ed25519_point_negate out pa+            ed25519_point_negate out pa  -- | Add two points. pointAdd :: Point -> Point -> Point pointAdd (Point a) (Point b) =     Point $ B.allocAndFreeze pointArraySize $ \out ->         withByteArray a $ \pa ->-        withByteArray b $ \pb ->-             ed25519_point_add out pa pb+            withByteArray b $ \pb ->+                ed25519_point_add out pa pb  -- | Add a point to itself. --@@ -243,7 +247,7 @@ pointDouble (Point a) =     Point $ B.allocAndFreeze pointArraySize $ \out ->         withByteArray a $ \pa ->-             ed25519_point_double out pa+            ed25519_point_double out pa  -- | Multiply a point by h = 8. --@@ -254,7 +258,7 @@ pointMulByCofactor (Point a) =     Point $ B.allocAndFreeze pointArraySize $ \out ->         withByteArray a $ \pa ->-             ed25519_point_mul_by_cofactor out pa+            ed25519_point_mul_by_cofactor out pa  -- | Scalar multiplication over curve edwards25519. --@@ -265,8 +269,8 @@ pointMul (Scalar scalar) (Point base) =     Point $ B.allocAndFreeze pointArraySize $ \out ->         withByteArray scalar $ \pscalar ->-        withByteArray base   $ \pbase   ->-             ed25519_point_scalarmul out pbase pscalar+            withByteArray base $ \pbase ->+                ed25519_point_scalarmul out pbase pscalar  -- | Multiply the point @p@ with @s2@ and add a lifted to curve value @s1@. --@@ -279,92 +283,108 @@ pointsMulVarTime (Scalar s1) (Scalar s2) (Point p) =     Point $ B.allocAndFreeze pointArraySize $ \out ->         withByteArray s1 $ \ps1 ->-        withByteArray s2 $ \ps2 ->-        withByteArray p  $ \pp  ->-             ed25519_base_double_scalarmul_vartime out ps1 pp ps2+            withByteArray s2 $ \ps2 ->+                withByteArray p $ \pp ->+                    ed25519_base_double_scalarmul_vartime out ps1 pp ps2  foreign import ccall unsafe "crypton_ed25519_scalar_eq"-    ed25519_scalar_eq :: Ptr Scalar-                      -> Ptr Scalar-                      -> IO CInt+    ed25519_scalar_eq+        :: Ptr Scalar+        -> Ptr Scalar+        -> IO CInt  foreign import ccall unsafe "crypton_ed25519_scalar_encode"-    ed25519_scalar_encode :: Ptr Word8-                          -> Ptr Scalar-                          -> IO ()+    ed25519_scalar_encode+        :: Ptr Word8+        -> Ptr Scalar+        -> IO ()  foreign import ccall unsafe "crypton_ed25519_scalar_decode_long"-    ed25519_scalar_decode_long :: Ptr Scalar-                               -> Ptr Word8-                               -> CSize-                               -> IO ()+    ed25519_scalar_decode_long+        :: Ptr Scalar+        -> Ptr Word8+        -> CSize+        -> IO ()  foreign import ccall unsafe "crypton_ed25519_scalar_add"-    ed25519_scalar_add :: Ptr Scalar -- sum-                       -> Ptr Scalar -- a-                       -> Ptr Scalar -- b-                       -> IO ()+    ed25519_scalar_add+        :: Ptr Scalar -- sum+        -> Ptr Scalar -- a+        -> Ptr Scalar -- b+        -> IO ()  foreign import ccall unsafe "crypton_ed25519_scalar_mul"-    ed25519_scalar_mul :: Ptr Scalar -- out-                       -> Ptr Scalar -- a-                       -> Ptr Scalar -- b-                       -> IO ()+    ed25519_scalar_mul+        :: Ptr Scalar -- out+        -> Ptr Scalar -- a+        -> Ptr Scalar -- b+        -> IO ()  foreign import ccall unsafe "crypton_ed25519_point_encode"-    ed25519_point_encode :: Ptr Word8-                         -> Ptr Point-                         -> IO ()+    ed25519_point_encode+        :: Ptr Word8+        -> Ptr Point+        -> IO ()  foreign import ccall unsafe "crypton_ed25519_point_decode_vartime"-    ed25519_point_decode_vartime :: Ptr Point-                                 -> Ptr Word8-                                 -> IO CInt+    ed25519_point_decode_vartime+        :: Ptr Point+        -> Ptr Word8+        -> IO CInt  foreign import ccall unsafe "crypton_ed25519_point_eq"-    ed25519_point_eq :: Ptr Point-                     -> Ptr Point-                     -> IO CInt+    ed25519_point_eq+        :: Ptr Point+        -> Ptr Point+        -> IO CInt  foreign import ccall "crypton_ed25519_point_has_prime_order"-    ed25519_point_has_prime_order :: Ptr Point-                                  -> IO CInt+    ed25519_point_has_prime_order+        :: Ptr Point+        -> IO CInt  foreign import ccall unsafe "crypton_ed25519_point_negate"-    ed25519_point_negate :: Ptr Point -- minus_a-                         -> Ptr Point -- a-                         -> IO ()+    ed25519_point_negate+        :: Ptr Point -- minus_a+        -> Ptr Point -- a+        -> IO ()  foreign import ccall unsafe "crypton_ed25519_point_add"-    ed25519_point_add :: Ptr Point -- sum-                      -> Ptr Point -- a-                      -> Ptr Point -- b-                      -> IO ()+    ed25519_point_add+        :: Ptr Point -- sum+        -> Ptr Point -- a+        -> Ptr Point -- b+        -> IO ()  foreign import ccall unsafe "crypton_ed25519_point_double"-    ed25519_point_double :: Ptr Point -- two_a-                         -> Ptr Point -- a-                         -> IO ()+    ed25519_point_double+        :: Ptr Point -- two_a+        -> Ptr Point -- a+        -> IO ()  foreign import ccall unsafe "crypton_ed25519_point_mul_by_cofactor"-    ed25519_point_mul_by_cofactor :: Ptr Point -- eight_a-                                  -> Ptr Point -- a-                                  -> IO ()+    ed25519_point_mul_by_cofactor+        :: Ptr Point -- eight_a+        -> Ptr Point -- a+        -> IO ()  foreign import ccall "crypton_ed25519_point_base_scalarmul"-    ed25519_point_base_scalarmul :: Ptr Point  -- scaled-                                 -> Ptr Scalar -- scalar-                                 -> IO ()+    ed25519_point_base_scalarmul+        :: Ptr Point -- scaled+        -> Ptr Scalar -- scalar+        -> IO ()  foreign import ccall "crypton_ed25519_point_scalarmul"-    ed25519_point_scalarmul :: Ptr Point  -- scaled-                            -> Ptr Point  -- base-                            -> Ptr Scalar -- scalar-                            -> IO ()+    ed25519_point_scalarmul+        :: Ptr Point -- scaled+        -> Ptr Point -- base+        -> Ptr Scalar -- scalar+        -> IO ()  foreign import ccall "crypton_ed25519_base_double_scalarmul_vartime"-    ed25519_base_double_scalarmul_vartime :: Ptr Point  -- combo-                                          -> Ptr Scalar -- scalar1-                                          -> Ptr Point  -- base2-                                          -> Ptr Scalar -- scalar2-                                          -> IO ()+    ed25519_base_double_scalarmul_vartime+        :: Ptr Point -- combo+        -> Ptr Scalar -- scalar1+        -> Ptr Point -- base2+        -> Ptr Scalar -- scalar2+        -> IO ()
Crypto/ECC/Simple/Prim.hs view
@@ -1,56 +1,66 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE ScopedTypeVariables #-}+ -- | Elliptic Curve Arithmetic. -- -- /WARNING:/ These functions are vulnerable to timing attacks.-{-# LANGUAGE ScopedTypeVariables #-}-module Crypto.ECC.Simple.Prim-    ( scalarGenerate-    , scalarFromInteger-    , pointAdd-    , pointNegate-    , pointDouble-    , pointBaseMul-    , pointMul-    , pointAddTwoMuls-    , pointFromIntegers-    , isPointAtInfinity-    , isPointValid-    ) where+module Crypto.ECC.Simple.Prim (+    scalarGenerate,+    scalarFromInteger,+    pointAdd,+    pointNegate,+    pointDouble,+    pointBaseMul,+    pointMul,+    pointAddTwoMuls,+    pointFromIntegers,+    isPointAtInfinity,+    isPointValid,+    isPointInSubgroup,+) where -import Data.Maybe-import Data.Proxy-import Crypto.Number.ModArithmetic-import Crypto.Number.F2m-import Crypto.Number.Generate (generateBetween) import Crypto.ECC.Simple.Types import Crypto.Error+import Crypto.Internal.ECC (CurveField (..), MulResult (..), curveMul)+import Crypto.Number.Basic (numBits)+import Crypto.Number.F2m+import Crypto.Number.Generate (generateBetween)+import Crypto.Number.ModArithmetic import Crypto.Random+import Data.Bits (shiftL, shiftR, testBit, (.&.)) +import Data.Maybe+import Data.Proxy+ -- | Generate a valid scalar for a specific Curve-scalarGenerate :: forall randomly curve . (MonadRandom randomly, Curve curve) => randomly (Scalar curve)+scalarGenerate+    :: forall randomly curve+     . (MonadRandom randomly, Curve curve) => randomly (Scalar curve) scalarGenerate =     Scalar <$> generateBetween 1 (n - 1)   where     n = curveEccN $ curveParameters (Proxy :: Proxy curve) -scalarFromInteger :: forall curve . Curve curve => Integer -> CryptoFailable (Scalar curve)+scalarFromInteger+    :: forall curve. Curve curve => Integer -> CryptoFailable (Scalar curve) scalarFromInteger n-    | n < 0  || n >= mx = CryptoFailed $ CryptoError_EcScalarOutOfBounds-    | otherwise         = CryptoPassed $ Scalar n+    | n < 0 || n >= mx = CryptoFailed $ CryptoError_EcScalarOutOfBounds+    | otherwise = CryptoPassed $ Scalar n   where     mx = case curveType (Proxy :: Proxy curve) of-            CurveBinary (CurveBinaryParam b) -> b-            CurvePrime (CurvePrimeParam p)   -> p+        CurveBinary (CurveBinaryParam b) -> b+        CurvePrime (CurvePrimeParam p) -> p ---TODO: Extract helper function for `fromMaybe PointO...`+-- TODO: Extract helper function for `fromMaybe PointO...`  -- | Elliptic Curve point negation: -- @pointNegate p@ returns point @q@ such that @pointAdd p q == PointO@. pointNegate :: Curve curve => Point curve -> Point curve-pointNegate        PointO     = PointO+pointNegate PointO = PointO pointNegate point@(Point x y) =     case curveType point of         CurvePrime (CurvePrimeParam p) -> Point x (p - y)-        CurveBinary {} -> Point x (x `addF2m` y)+        CurveBinary{} -> Point x (x `addF2m` y)  -- | Elliptic Curve point addition. --@@ -60,13 +70,13 @@ pointAdd PointO q = q pointAdd p PointO = p pointAdd p q-  | p == q             = pointDouble p-  | p == pointNegate q = PointO+    | p == q = pointDouble p+    | p == pointNegate q = PointO pointAdd point@(Point xp yp) (Point xq yq) =     case ty of         CurvePrime (CurvePrimeParam pr) -> fromMaybe PointO $ do             s <- divmod (yp - yq) (xp - xq) pr-            let xr = (s ^ (2::Int) - xp - xq) `mod` pr+            let xr = (s ^ (2 :: Int) - xp - xq) `mod` pr                 yr = (s * (xp - xr) - yp) `mod` pr             return $ Point xr yr         CurveBinary (CurveBinaryParam fx) -> fromMaybe PointO $ do@@ -77,7 +87,7 @@   where     ty = curveType point     cc = curveParameters point-    a  = curveEccA cc+    a = curveEccA cc  -- | Elliptic Curve point doubling. --@@ -94,19 +104,18 @@ -- >    s = xp + (yp / xp) -- >    xr = s ^ 2 + s + a -- >    yr = xp ^ 2 + (s+1) * xr--- pointDouble :: Curve curve => Point curve -> Point curve pointDouble PointO = PointO pointDouble point@(Point xp yp) =     case ty of         CurvePrime (CurvePrimeParam pr) -> fromMaybe PointO $ do-            lambda <- divmod (3 * xp ^ (2::Int) + a) (2 * yp) pr-            let xr = (lambda ^ (2::Int) - 2 * xp) `mod` pr+            lambda <- divmod (3 * xp ^ (2 :: Int) + a) (2 * yp) pr+            let xr = (lambda ^ (2 :: Int) - 2 * xp) `mod` pr                 yr = (lambda * (xp - xr) - yp) `mod` pr             return $ Point xr yr         CurveBinary (CurveBinaryParam fx)-            | xp == 0    -> PointO-            | otherwise  -> fromMaybe PointO $ do+            | xp == 0 -> PointO+            | otherwise -> fromMaybe PointO $ do                 s <- return . addF2m xp =<< divF2m fx yp xp                 let xr = mulF2m fx s s `addF2m` s `addF2m` a                     yr = mulF2m fx xp xp `addF2m` mulF2m fx xr (s `addF2m` 1)@@ -114,7 +123,7 @@   where     ty = curveType point     cc = curveParameters point-    a  = curveEccA cc+    a = curveEccA cc  -- | Elliptic curve point multiplication using the base --@@ -122,53 +131,251 @@ pointBaseMul :: Curve curve => Scalar curve -> Point curve pointBaseMul n = pointMul n (curveEccG $ curveParameters (Proxy :: Proxy curve)) --- | Elliptic curve point multiplication (double and add algorithm).+-- | Elliptic curve point multiplication. ----- /WARNING:/ Vulnerable to timing attacks.-pointMul :: Curve curve => Scalar curve -> Point curve -> Point curve+-- Over a prime field this goes to C, four bits of scalar at a time, with the+-- multiple to add taken from a table read by touching every entry of it.+-- Over a binary field it also goes to C, as Montgomery's ladder: it carries+-- the x coordinates of two consecutive multiples -- their difference being+-- the point is what lets it carry no more than that -- and spends one+-- addition and one doubling on every bit whichever way the bit goes, with the+-- two exchanged by a mask rather than chosen by a branch.  Either way the work+-- follows the width of the curve's order and not the scalar.+--+-- What falls back on the 'Integer' arithmetic below is a point that is not on+-- the curve, the one point of a binary curve that has no x, and a prime the C+-- will not take.+--+-- Multiplying the base point of a curve over a prime field -- which is what+-- signing and making a key do, and nothing else does -- goes through a table+-- of its multiples, built when that curve is first asked for one and kept+-- afterwards.  The build is a few milliseconds and the table a few hundred+-- kilobytes, and a multiplication that uses it takes about a third of what+-- one without it takes.+--+-- /WARNING:/ What is left of the 'Integer' arithmetic below -- a point off+-- the curve, the one point of a binary curve with no x, a prime or a+-- polynomial the C will not take -- has uniform operation counts at best, and+-- uniform operation counts are not constant time: those operations cost what+-- the values they are given cost.  See the note in+-- "Crypto.ECC".+pointMul+    :: forall curve. Curve curve => Scalar curve -> Point curve -> Point curve pointMul _ PointO = PointO pointMul (Scalar n) p-    | n == 0    = PointO-    | n == 1    = p-    | odd n     = pointAdd p (pointMul (Scalar (n - 1)) p)-    | otherwise = pointMul (Scalar (n `div` 2)) (pointDouble p)+    | n == 0 = PointO+    | n < 0 = pointNegate (pointMul (Scalar (negate n) :: Scalar curve) p)+    | otherwise =+        case curveType (Proxy :: Proxy curve) of+            CurvePrime (CurvePrimeParam pr) -> primeMul pr+            CurveBinary (CurveBinaryParam fx) -> binaryMul fx+  where+    cc = curveParameters (Proxy :: Proxy curve)+    a = curveEccA cc+    -- Count to the width of the order, which is public, so a scalar in range+    -- -- which is every secret one -- takes the same number of steps whatever+    -- it is.  A scalar may still be given out of range, and then the count has+    -- to follow it or the high bits would be dropped.+    bits = max (integerBits n) (integerBits (curveEccN cc)) --- | Elliptic curve double-scalar multiplication (uses Shamir's trick).+    -- The C answers for a point on the curve; anything else keeps the+    -- answers it has always had from the code below.+    primeMul pr = case p of+        Point px py+            | isPointValid (Proxy :: Proxy curve) px py ->+                answer slow $+                    curveMul+                        (Prime pr a (curveEccB cc))+                        (curveEccN cc)+                        n+                        px+                        py+                        (p == curveEccG cc)+        _ -> slow+      where+        slow = jacobianMul pr a bits n p++    -- The ladder answers for a point on the curve that has an x; the one+    -- point with no x, and anything off the curve, keep what they had.+    binaryMul fx = case p of+        Point px py+            | isPointValid (Proxy :: Proxy curve) px py ->+                answer (affineMul n p) $+                    curveMul (Binary fx (curveEccB cc)) (curveEccN cc) n px py False+        _ -> affineMul n p++    -- what the C could not take goes back to the code that was here before+    answer fallback r = case r of+        MulPoint x y -> Point x y+        MulInfinity -> PointO+        MulUnsupported -> fallback++    affineMul k q+        | k == 0 = PointO+        | k == 1 = q+        | odd k = pointAdd q (affineMul (k - 1) q)+        | otherwise = affineMul (k `div` 2) (pointDouble q)++-- | Number of bits needed to write n, for n > 0.+integerBits :: Integer -> Int+integerBits = go 0+  where+    go acc 0 = acc+    go acc k = go (acc + 1) (k `div` 2)++-- | A point in Jacobian coordinates: @(X, Y, Z)@ stands for the affine+-- @(X\/Z^2, Y\/Z^3)@, and @JPointO@ for the point at infinity.  Only ever+-- used inside this module, since t'Point' is what the curve exposes.+data JPoint = JPointO | JPoint !Integer !Integer !Integer++-- | The prime, the width to fold at, and what to fold back in.  A @c@ of zero+-- says to divide instead, either because the prime has no such shape or+-- because it is too small for folding to pay: @c@ has to be under half the+-- width, or folding would not shrink the number, and below 256 bits the+-- handful of 'Integer' operations folding takes costs more than the division+-- it saves -- measured on P-192, where folding is 14% slower. --+-- Most curve primes are @2^k - c@ with @c@ far smaller than the prime, and+-- then reducing is a shift, a multiplication by @c@ and an addition, where+-- dividing a number twice the width costs about four times as much.+data Field = Field !Integer !Int !Integer++mkField :: Integer -> Field+mkField p+    | p > 0 && c > 0 && 2 * numBits c <= k && k >= 256 = Field p k c+    | otherwise = Field p 0 0+  where+    k = numBits p+    c = (1 `shiftL` k) - p++fieldPrime :: Field -> Integer+fieldPrime (Field p _ _) = p++fieldReduce :: Field -> Integer -> Integer+fieldReduce (Field p k c) x+    | c == 0 || x < 0 = x `mod` p+    | otherwise = trim (fold x)+  where+    mask = (1 `shiftL` k) - 1+    fold v+        | v > mask = fold ((v `shiftR` k) * c + (v .&. mask))+        | otherwise = v+    trim v+        | v >= p = trim (v - p)+        | otherwise = v+{-# INLINE fieldReduce #-}++jacobianMul+    :: Integer -> Integer -> Int -> Integer -> Point curve -> Point curve+jacobianMul _ _ _ _ PointO = PointO+jacobianMul pr a bits n (Point px py) = fromJacobian f (go (bits - 1) JPointO)+  where+    f = mkField pr++    -- The bangs are what make the addition happen at every bit.  Without+    -- them the one that is not taken stays a thunk and is never worked out,+    -- so the multiplication costs a step for every bit that is set rather+    -- than for every bit there is, and a single measurement tells an attacker+    -- how many bits of the scalar are set.+    go i acc+        | i < 0 = acc+        | otherwise =+            let !d = jDouble f a acc+                !s = jAddAffine f a d px py+             in go (i - 1) (if testBit n i then s else d)++jDouble :: Field -> Integer -> JPoint -> JPoint+jDouble _ _ JPointO = JPointO+jDouble f a (JPoint x y z)+    | y == 0 = JPointO+    | otherwise = JPoint x3 y3 z3+  where+    red = fieldReduce f+    yy = red (y * y)+    delta = red (4 * x * yy)+    zz = red (z * z)+    m = red (3 * x * x + a * zz * zz)+    x3 = red (m * m - 2 * delta)+    y3 = red (m * (delta - x3) - 8 * yy * yy)+    z3 = red (2 * y * z)++-- | Add a point whose z is one, which is what a scalar multiplication always+-- adds: u1 is x1, s1 is y1, and z3 is one multiplication rather than two.+jAddAffine :: Field -> Integer -> JPoint -> Integer -> Integer -> JPoint+jAddAffine _ _ JPointO x2 y2 = JPoint x2 y2 1+jAddAffine f a p@(JPoint x1 y1 z1) x2 y2+    | h /= 0 = JPoint x3 y3 z3+    | r /= 0 = JPointO+    | otherwise = jDouble f a p+  where+    red = fieldReduce f+    z1s = red (z1 * z1)+    u2 = red (x2 * z1s)+    s2 = red (y2 * z1s * z1)+    h = red (u2 - x1)+    r = red (s2 - y1)+    h2 = red (h * h)+    h3 = red (h2 * h)+    x3 = red (r * r - h3 - 2 * x1 * h2)+    y3 = red (r * (x1 * h2 - x3) - y1 * h3)+    z3 = red (h * z1)++fromJacobian :: Field -> JPoint -> Point curve+fromJacobian _ JPointO = PointO+fromJacobian f (JPoint x y z) =+    case inverse z (fieldPrime f) of+        Nothing -> PointO+        Just zi ->+            let red = fieldReduce f+                zi2 = red (zi * zi)+             in Point (red (x * zi2)) (red (y * zi2 * zi))++-- | Elliptic curve double-scalar multiplication.+-- -- > pointAddTwoMuls n1 p1 n2 p2 == pointAdd (pointMul n1 p1) -- >                                         (pointMul n2 p2) --+-- which is how it is done: the two multiplications separately, and then one+-- addition.+--+-- This used to be Shamir's trick, one pass over the bits of both scalars at+-- once, which shares the doublings between them and is the right thing to do+-- when the two multiplications would cost the same.  They no longer do.+-- 'pointMul' goes to C, and over a prime field it multiplies the base point+-- through a table of its multiples, which is a third of the price of an+-- ordinary multiplication -- and the base point is one of the two here,+-- since ECDSA verification is what asks for this.  Sharing the doublings+-- with a pass in "Integer" arithmetic gives that up and more: on P-384 it+-- costs twice what two multiplications in C cost, and on the curves over a+-- binary field, whose addition needs an inversion where C has a ladder that+-- needs none, it costs two hundred times as much.+-- -- /WARNING:/ Vulnerable to timing attacks.-pointAddTwoMuls :: Curve curve => Scalar curve -> Point curve -> Scalar curve -> Point curve -> Point curve-pointAddTwoMuls _  PointO _  PointO = PointO-pointAddTwoMuls _  PointO n2 p2     = pointMul n2 p2-pointAddTwoMuls n1 p1     _  PointO = pointMul n1 p1-pointAddTwoMuls (Scalar n1) p1 (Scalar n2) p2 = go (n1, n2)-  where-    p0 = pointAdd p1 p2--    go (0,  0 ) = PointO-    go (k1, k2) =-        let q = pointDouble $ go (k1 `div` 2, k2 `div` 2)-        in case (odd k1, odd k2) of-            (True  , True  ) -> pointAdd p0 q-            (True  , False ) -> pointAdd p1 q-            (False , True  ) -> pointAdd p2 q-            (False , False ) -> q+pointAddTwoMuls+    :: forall curve+     . Curve curve+    => Scalar curve -> Point curve -> Scalar curve -> Point curve -> Point curve+pointAddTwoMuls n1 p1 n2 p2 = pointAdd (pointMul n1 p1) (pointMul n2 p2)  -- | Check if a point is the point at infinity. isPointAtInfinity :: Point curve -> Bool isPointAtInfinity PointO = True-isPointAtInfinity _      = False+isPointAtInfinity _ = False  -- | Make a point on a curve from integer (x,y) coordinate -- -- if the point is not valid related to the curve then an error is -- returned instead of a point-pointFromIntegers :: forall curve . Curve curve => (Integer, Integer) -> CryptoFailable (Point curve)-pointFromIntegers (x,y)-    | isPointValid (Proxy :: Proxy curve) x y = CryptoPassed $ Point x y-    | otherwise                               = CryptoFailed $ CryptoError_PointCoordinatesInvalid+pointFromIntegers+    :: forall curve. Curve curve => (Integer, Integer) -> CryptoFailable (Point curve)+pointFromIntegers (x, y)+    | not (isPointValid (Proxy :: Proxy curve) x y) =+        CryptoFailed CryptoError_PointCoordinatesInvalid+    | not (isPointInSubgroup (Proxy :: Proxy curve) p) =+        CryptoFailed CryptoError_PointSubgroupInvalid+    | otherwise = CryptoPassed p+  where+    p = Point x y  -- | check if a point is on specific curve --@@ -181,23 +388,42 @@ isPointValid proxy x y =     case ty of         CurvePrime (CurvePrimeParam p) ->-            let a  = curveEccA cc-                b  = curveEccB cc+            let a = curveEccA cc+                b = curveEccB cc                 eqModP z1 z2 = (z1 `mod` p) == (z2 `mod` p)                 isValid e = e >= 0 && e < p              in isValid x && isValid y && (y ^ (2 :: Int)) `eqModP` (x ^ (3 :: Int) + a * x + b)         CurveBinary (CurveBinaryParam fx) ->-            let a  = curveEccA cc-                b  = curveEccB cc+            let a = curveEccA cc+                b = curveEccB cc                 add = addF2m                 mul = mulF2m fx                 isValid e = modF2m fx e == e-             in and [ isValid x+             in and+                    [ isValid x                     , isValid y                     , ((((x `add` a) `mul` x `add` y) `mul` x) `add` b `add` (squareF2m fx y)) == 0                     ]   where     ty = curveType proxy+    cc = curveParameters proxy++-- | Check that a point is in the subgroup the base point generates, which is+-- the further check 'isPointValid' does not make.  A point that is on the+-- curve but outside that subgroup answers a multiplication modulo an order+-- smaller than the group's, so the multiplier -- a private number, where the+-- point came from a peer -- is revealed modulo that small order.+--+-- Where the cofactor is 1 the subgroup is the whole curve group and the+-- answer is 'True' for any point on the curve, at no cost.  Otherwise the+-- point is multiplied by the group order and the answer is whether that+-- reaches the point at infinity, which costs one scalar multiplication.+isPointInSubgroup+    :: forall proxy curve. Curve curve => proxy curve -> Point curve -> Bool+isPointInSubgroup proxy p+    | curveEccH cc == 1 = True+    | otherwise = pointMul (Scalar (curveEccN cc) :: Scalar curve) p == PointO+  where     cc = curveParameters proxy  -- | div and mod
Crypto/ECC/Simple/Types.hs view
@@ -1,5 +1,7 @@ {-# LANGUAGE DeriveDataTypeable #-} {-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# OPTIONS_GHC -fno-warn-missing-signatures #-}+ -- | -- Module      : Crypto.ECC.Simple.Types -- License     : BSD-style@@ -9,57 +11,56 @@ -- -- References: --   <https://tools.ietf.org/html/rfc5915>----{-# OPTIONS_GHC -fno-warn-missing-signatures #-}-module Crypto.ECC.Simple.Types-    ( Curve(..)-    , Point(..)-    , Scalar(..)-    , CurveType(..)-    , CurveBinaryParam(..)-    , CurvePrimeParam(..)-    , curveSizeBits-    , curveSizeBytes-    , CurveParameters(..)+module Crypto.ECC.Simple.Types (+    Curve (..),+    Point (..),+    Scalar (..),+    CurveType (..),+    CurveBinaryParam (..),+    CurvePrimeParam (..),+    curveSizeBits,+    curveSizeBytes,+    CurveParameters (..),+     -- * Specific curves definition-    , SEC_p112r1(..)-    , SEC_p112r2(..)-    , SEC_p128r1(..)-    , SEC_p128r2(..)-    , SEC_p160k1(..)-    , SEC_p160r1(..)-    , SEC_p160r2(..)-    , SEC_p192k1(..)-    , SEC_p192r1(..) -- aka prime192v1-    , SEC_p224k1(..)-    , SEC_p224r1(..)-    , SEC_p256k1(..)-    , SEC_p256r1(..) -- aka prime256v1-    , SEC_p384r1(..)-    , SEC_p521r1(..)-    , SEC_t113r1(..)-    , SEC_t113r2(..)-    , SEC_t131r1(..)-    , SEC_t131r2(..)-    , SEC_t163k1(..)-    , SEC_t163r1(..)-    , SEC_t163r2(..)-    , SEC_t193r1(..)-    , SEC_t193r2(..)-    , SEC_t233k1(..) -- aka NIST K-233-    , SEC_t233r1(..)-    , SEC_t239k1(..)-    , SEC_t283k1(..)-    , SEC_t283r1(..)-    , SEC_t409k1(..)-    , SEC_t409r1(..)-    , SEC_t571k1(..)-    , SEC_t571r1(..)-    ) where+    SEC_p112r1 (..),+    SEC_p112r2 (..),+    SEC_p128r1 (..),+    SEC_p128r2 (..),+    SEC_p160k1 (..),+    SEC_p160r1 (..),+    SEC_p160r2 (..),+    SEC_p192k1 (..),+    SEC_p192r1 (..), -- aka prime192v1+    SEC_p224k1 (..),+    SEC_p224r1 (..),+    SEC_p256k1 (..),+    SEC_p256r1 (..), -- aka prime256v1+    SEC_p384r1 (..),+    SEC_p521r1 (..),+    SEC_t113r1 (..),+    SEC_t113r2 (..),+    SEC_t131r1 (..),+    SEC_t131r2 (..),+    SEC_t163k1 (..),+    SEC_t163r1 (..),+    SEC_t163r2 (..),+    SEC_t193r1 (..),+    SEC_t193r2 (..),+    SEC_t233k1 (..), -- aka NIST K-233+    SEC_t233r1 (..),+    SEC_t239k1 (..),+    SEC_t283k1 (..),+    SEC_t283r1 (..),+    SEC_t409k1 (..),+    SEC_t409r1 (..),+    SEC_t571k1 (..),+    SEC_t571r1 (..),+) where -import           Data.Data-import           Crypto.Internal.Imports-import           Crypto.Number.Basic (numBits)+import Crypto.Internal.Imports+import Crypto.Number.Basic (numBits)+import Data.Data  class Curve curve where     curveParameters :: proxy curve -> CurveParameters curve@@ -69,7 +70,7 @@ curveSizeBits :: Curve curve => proxy curve -> Int curveSizeBits proxy =     case curveType proxy of-        CurvePrime (CurvePrimeParam p)   -> numBits p+        CurvePrime (CurvePrimeParam p) -> numBits p         CurveBinary (CurveBinaryParam c) -> numBits c - 1  -- | get the size of the curve in bytes@@ -79,72 +80,92 @@ -- | Define common parameters in a curve definition -- of the form: y^2 = x^3 + ax + b. data CurveParameters curve = CurveParameters-    { curveEccA :: Integer     -- ^ curve parameter a-    , curveEccB :: Integer     -- ^ curve parameter b-    , curveEccG :: Point curve -- ^ base point-    , curveEccN :: Integer     -- ^ order of G-    , curveEccH :: Integer     -- ^ cofactor-    } deriving (Show,Eq,Data)+    { curveEccA :: Integer+    -- ^ curve parameter a+    , curveEccB :: Integer+    -- ^ curve parameter b+    , curveEccG :: Point curve+    -- ^ base point+    , curveEccN :: Integer+    -- ^ order of G+    , curveEccH :: Integer+    -- ^ cofactor+    }+    deriving (Show, Eq, Data)  newtype CurveBinaryParam = CurveBinaryParam Integer-    deriving (Show,Read,Eq,Data)+    deriving (Show, Read, Eq, Data)  newtype CurvePrimeParam = CurvePrimeParam Integer-    deriving (Show,Read,Eq,Data)+    deriving (Show, Read, Eq, Data) -data CurveType =-      CurveBinary CurveBinaryParam+data CurveType+    = CurveBinary CurveBinaryParam     | CurvePrime CurvePrimeParam-    deriving (Show,Read,Eq,Data)+    deriving (Show, Read, Eq, Data)  -- | ECC Private Number newtype Scalar curve = Scalar Integer-    deriving (Show,Read,Eq,Data,NFData)+    deriving (Show, Read, Eq, Data, NFData)  -- | Define a point on a curve.-data Point curve =-      Point Integer Integer-    | PointO -- ^ Point at Infinity-    deriving (Show,Read,Eq,Data)+data Point curve+    = Point Integer Integer+    | -- | Point at Infinity+      PointO+    deriving (Show, Read, Eq, Data)  instance NFData (Point curve) where     rnf (Point x y) = x `seq` y `seq` ()     rnf PointO = () -data SEC_p112r1 = SEC_p112r1 deriving (Show,Read,Eq)-data SEC_p112r2 = SEC_p112r2 deriving (Show,Read,Eq)-data SEC_p128r1 = SEC_p128r1 deriving (Show,Read,Eq)-data SEC_p128r2 = SEC_p128r2 deriving (Show,Read,Eq)-data SEC_p160k1 = SEC_p160k1 deriving (Show,Read,Eq)-data SEC_p160r1 = SEC_p160r1 deriving (Show,Read,Eq)-data SEC_p160r2 = SEC_p160r2 deriving (Show,Read,Eq)-data SEC_p192k1 = SEC_p192k1 deriving (Show,Read,Eq)-data SEC_p192r1 = SEC_p192r1 deriving (Show,Read,Eq)-data SEC_p224k1 = SEC_p224k1 deriving (Show,Read,Eq)-data SEC_p224r1 = SEC_p224r1 deriving (Show,Read,Eq)-data SEC_p256k1 = SEC_p256k1 deriving (Show,Read,Eq)-data SEC_p256r1 = SEC_p256r1 deriving (Show,Read,Eq)-data SEC_p384r1 = SEC_p384r1 deriving (Show,Read,Eq)-data SEC_p521r1 = SEC_p521r1 deriving (Show,Read,Eq)-data SEC_t113r1 = SEC_t113r1 deriving (Show,Read,Eq)-data SEC_t113r2 = SEC_t113r2 deriving (Show,Read,Eq)-data SEC_t131r1 = SEC_t131r1 deriving (Show,Read,Eq)-data SEC_t131r2 = SEC_t131r2 deriving (Show,Read,Eq)-data SEC_t163k1 = SEC_t163k1 deriving (Show,Read,Eq)-data SEC_t163r1 = SEC_t163r1 deriving (Show,Read,Eq)-data SEC_t163r2 = SEC_t163r2 deriving (Show,Read,Eq)-data SEC_t193r1 = SEC_t193r1 deriving (Show,Read,Eq)-data SEC_t193r2 = SEC_t193r2 deriving (Show,Read,Eq)-data SEC_t233k1 = SEC_t233k1 deriving (Show,Read,Eq)-data SEC_t233r1 = SEC_t233r1 deriving (Show,Read,Eq)-data SEC_t239k1 = SEC_t239k1 deriving (Show,Read,Eq)-data SEC_t283k1 = SEC_t283k1 deriving (Show,Read,Eq)-data SEC_t283r1 = SEC_t283r1 deriving (Show,Read,Eq)-data SEC_t409k1 = SEC_t409k1 deriving (Show,Read,Eq)-data SEC_t409r1 = SEC_t409r1 deriving (Show,Read,Eq)-data SEC_t571k1 = SEC_t571k1 deriving (Show,Read,Eq)-data SEC_t571r1 = SEC_t571r1 deriving (Show,Read,Eq)+data SEC_p112r1 = SEC_p112r1 deriving (Show, Read, Eq)+data SEC_p112r2 = SEC_p112r2 deriving (Show, Read, Eq)+data SEC_p128r1 = SEC_p128r1 deriving (Show, Read, Eq)+data SEC_p128r2 = SEC_p128r2 deriving (Show, Read, Eq)+data SEC_p160k1 = SEC_p160k1 deriving (Show, Read, Eq)+data SEC_p160r1 = SEC_p160r1 deriving (Show, Read, Eq)+data SEC_p160r2 = SEC_p160r2 deriving (Show, Read, Eq)+data SEC_p192k1 = SEC_p192k1 deriving (Show, Read, Eq)+data SEC_p192r1 = SEC_p192r1 deriving (Show, Read, Eq)+data SEC_p224k1 = SEC_p224k1 deriving (Show, Read, Eq)+data SEC_p224r1 = SEC_p224r1 deriving (Show, Read, Eq)+data SEC_p256k1 = SEC_p256k1 deriving (Show, Read, Eq)+data SEC_p256r1 = SEC_p256r1 deriving (Show, Read, Eq)+data SEC_p384r1 = SEC_p384r1 deriving (Show, Read, Eq)+data SEC_p521r1 = SEC_p521r1 deriving (Show, Read, Eq)+data SEC_t113r1 = SEC_t113r1 deriving (Show, Read, Eq)+data SEC_t113r2 = SEC_t113r2 deriving (Show, Read, Eq)+data SEC_t131r1 = SEC_t131r1 deriving (Show, Read, Eq)+data SEC_t131r2 = SEC_t131r2 deriving (Show, Read, Eq)+data SEC_t163k1 = SEC_t163k1 deriving (Show, Read, Eq)+data SEC_t163r1 = SEC_t163r1 deriving (Show, Read, Eq)+data SEC_t163r2 = SEC_t163r2 deriving (Show, Read, Eq)+data SEC_t193r1 = SEC_t193r1 deriving (Show, Read, Eq)+data SEC_t193r2 = SEC_t193r2 deriving (Show, Read, Eq)+data SEC_t233k1 = SEC_t233k1 deriving (Show, Read, Eq)+data SEC_t233r1 = SEC_t233r1 deriving (Show, Read, Eq)+data SEC_t239k1 = SEC_t239k1 deriving (Show, Read, Eq)+data SEC_t283k1 = SEC_t283k1 deriving (Show, Read, Eq)+data SEC_t283r1 = SEC_t283r1 deriving (Show, Read, Eq)+data SEC_t409k1 = SEC_t409k1 deriving (Show, Read, Eq)+data SEC_t409r1 = SEC_t409r1 deriving (Show, Read, Eq)+data SEC_t571k1 = SEC_t571k1 deriving (Show, Read, Eq)+data SEC_t571r1 = SEC_t571r1 deriving (Show, Read, Eq) +{-# DEPRECATED+    SEC_t113r1, SEC_t113r2, SEC_t131r1, SEC_t131r2, SEC_t163k1, SEC_t163r1,+    SEC_t163r2, SEC_t193r1, SEC_t193r2, SEC_t233k1, SEC_t233r1, SEC_t239k1,+    SEC_t283k1, SEC_t283r1, SEC_t409k1, SEC_t409r1, SEC_t571k1, SEC_t571r1+    [ "This curve is over a binary field, and those are obsolete."+    , "They are also the curves whose cofactor is not 1, so a point from"+    , "a peer needs the subgroup check that costs a further scalar"+    , "multiplication; pyca/cryptography deprecated them for removal in"+    , "the release that fixed CVE-2026-26007.  This one will go in a"+    , "later major version of crypton.  Prefer a prime curve, or X25519."+    ]+    #-}+ -- | Define names for known recommended curves. instance Curve SEC_p112r1 where     curveType _ = typeSEC_p112r1@@ -318,299 +339,468 @@ -}  typeSEC_p112r1 = CurvePrime $ CurvePrimeParam 0xdb7c2abf62e35e668076bead208b-paramSEC_p112r1 = CurveParameters-    { curveEccA = 0xdb7c2abf62e35e668076bead2088-    , curveEccB = 0x659ef8ba043916eede8911702b22-    , curveEccG = Point 0x09487239995a5ee76b55f9c2f098-                    0xa89ce5af8724c0a23e0e0ff77500-    , curveEccN = 0xdb7c2abf62e35e7628dfac6561c5-    , curveEccH = 1-    }+paramSEC_p112r1 =+    CurveParameters+        { curveEccA = 0xdb7c2abf62e35e668076bead2088+        , curveEccB = 0x659ef8ba043916eede8911702b22+        , curveEccG =+            Point+                0x09487239995a5ee76b55f9c2f098+                0xa89ce5af8724c0a23e0e0ff77500+        , curveEccN = 0xdb7c2abf62e35e7628dfac6561c5+        , curveEccH = 1+        } typeSEC_p112r2 = CurvePrime $ CurvePrimeParam 0xdb7c2abf62e35e668076bead208b-paramSEC_p112r2 = CurveParameters-    { curveEccA = 0x6127c24c05f38a0aaaf65c0ef02c-    , curveEccB = 0x51def1815db5ed74fcc34c85d709-    , curveEccG = Point 0x4ba30ab5e892b4e1649dd0928643-                    0xadcd46f5882e3747def36e956e97-    , curveEccN = 0x36df0aafd8b8d7597ca10520d04b-    , curveEccH = 4-    }+paramSEC_p112r2 =+    CurveParameters+        { curveEccA = 0x6127c24c05f38a0aaaf65c0ef02c+        , curveEccB = 0x51def1815db5ed74fcc34c85d709+        , curveEccG =+            Point+                0x4ba30ab5e892b4e1649dd0928643+                0xadcd46f5882e3747def36e956e97+        , curveEccN = 0x36df0aafd8b8d7597ca10520d04b+        , curveEccH = 4+        } typeSEC_p128r1 = CurvePrime $ CurvePrimeParam 0xfffffffdffffffffffffffffffffffff-paramSEC_p128r1 = CurveParameters-    { curveEccA = 0xfffffffdfffffffffffffffffffffffc-    , curveEccB = 0xe87579c11079f43dd824993c2cee5ed3-    , curveEccG = Point 0x161ff7528b899b2d0c28607ca52c5b86-                    0xcf5ac8395bafeb13c02da292dded7a83-    , curveEccN = 0xfffffffe0000000075a30d1b9038a115-    , curveEccH = 1-    }+paramSEC_p128r1 =+    CurveParameters+        { curveEccA = 0xfffffffdfffffffffffffffffffffffc+        , curveEccB = 0xe87579c11079f43dd824993c2cee5ed3+        , curveEccG =+            Point+                0x161ff7528b899b2d0c28607ca52c5b86+                0xcf5ac8395bafeb13c02da292dded7a83+        , curveEccN = 0xfffffffe0000000075a30d1b9038a115+        , curveEccH = 1+        } typeSEC_p128r2 = CurvePrime $ CurvePrimeParam 0xfffffffdffffffffffffffffffffffff-paramSEC_p128r2 = CurveParameters-    { curveEccA = 0xd6031998d1b3bbfebf59cc9bbff9aee1-    , curveEccB = 0x5eeefca380d02919dc2c6558bb6d8a5d-    , curveEccG = Point 0x7b6aa5d85e572983e6fb32a7cdebc140-                    0x27b6916a894d3aee7106fe805fc34b44-    , curveEccN = 0x3fffffff7fffffffbe0024720613b5a3-    , curveEccH = 4-    }+paramSEC_p128r2 =+    CurveParameters+        { curveEccA = 0xd6031998d1b3bbfebf59cc9bbff9aee1+        , curveEccB = 0x5eeefca380d02919dc2c6558bb6d8a5d+        , curveEccG =+            Point+                0x7b6aa5d85e572983e6fb32a7cdebc140+                0x27b6916a894d3aee7106fe805fc34b44+        , curveEccN = 0x3fffffff7fffffffbe0024720613b5a3+        , curveEccH = 4+        } typeSEC_p160k1 = CurvePrime $ CurvePrimeParam 0x00fffffffffffffffffffffffffffffffeffffac73-paramSEC_p160k1 = CurveParameters-    { curveEccA = 0x000000000000000000000000000000000000000000-    , curveEccB = 0x000000000000000000000000000000000000000007-    , curveEccG = Point 0x003b4c382ce37aa192a4019e763036f4f5dd4d7ebb-                    0x00938cf935318fdced6bc28286531733c3f03c4fee-    , curveEccN = 0x0100000000000000000001b8fa16dfab9aca16b6b3-    , curveEccH = 1-    }+paramSEC_p160k1 =+    CurveParameters+        { curveEccA = 0x000000000000000000000000000000000000000000+        , curveEccB = 0x000000000000000000000000000000000000000007+        , curveEccG =+            Point+                0x003b4c382ce37aa192a4019e763036f4f5dd4d7ebb+                0x00938cf935318fdced6bc28286531733c3f03c4fee+        , curveEccN = 0x0100000000000000000001b8fa16dfab9aca16b6b3+        , curveEccH = 1+        } typeSEC_p160r1 = CurvePrime $ CurvePrimeParam 0x00ffffffffffffffffffffffffffffffff7fffffff-paramSEC_p160r1 = CurveParameters-    { curveEccA = 0x00ffffffffffffffffffffffffffffffff7ffffffc-    , curveEccB = 0x001c97befc54bd7a8b65acf89f81d4d4adc565fa45-    , curveEccG = Point 0x004a96b5688ef573284664698968c38bb913cbfc82-                    0x0023a628553168947d59dcc912042351377ac5fb32-    , curveEccN = 0x0100000000000000000001f4c8f927aed3ca752257-    , curveEccH = 1-    }+paramSEC_p160r1 =+    CurveParameters+        { curveEccA = 0x00ffffffffffffffffffffffffffffffff7ffffffc+        , curveEccB = 0x001c97befc54bd7a8b65acf89f81d4d4adc565fa45+        , curveEccG =+            Point+                0x004a96b5688ef573284664698968c38bb913cbfc82+                0x0023a628553168947d59dcc912042351377ac5fb32+        , curveEccN = 0x0100000000000000000001f4c8f927aed3ca752257+        , curveEccH = 1+        } typeSEC_p160r2 = CurvePrime $ CurvePrimeParam 0x00fffffffffffffffffffffffffffffffeffffac73-paramSEC_p160r2 = CurveParameters-    { curveEccA = 0x00fffffffffffffffffffffffffffffffeffffac70-    , curveEccB = 0x00b4e134d3fb59eb8bab57274904664d5af50388ba-    , curveEccG = Point 0x0052dcb034293a117e1f4ff11b30f7199d3144ce6d-                    0x00feaffef2e331f296e071fa0df9982cfea7d43f2e-    , curveEccN = 0x0100000000000000000000351ee786a818f3a1a16b-    , curveEccH = 1-    }-typeSEC_p192k1 = CurvePrime $ CurvePrimeParam 0xfffffffffffffffffffffffffffffffffffffffeffffee37-paramSEC_p192k1 = CurveParameters-    { curveEccA = 0x000000000000000000000000000000000000000000000000-    , curveEccB = 0x000000000000000000000000000000000000000000000003-    , curveEccG = Point 0xdb4ff10ec057e9ae26b07d0280b7f4341da5d1b1eae06c7d-                    0x9b2f2f6d9c5628a7844163d015be86344082aa88d95e2f9d-    , curveEccN = 0xfffffffffffffffffffffffe26f2fc170f69466a74defd8d-    , curveEccH = 1-    }-typeSEC_p192r1 = CurvePrime $ CurvePrimeParam 0xfffffffffffffffffffffffffffffffeffffffffffffffff-paramSEC_p192r1 = CurveParameters-    { curveEccA = 0xfffffffffffffffffffffffffffffffefffffffffffffffc-    , curveEccB = 0x64210519e59c80e70fa7e9ab72243049feb8deecc146b9b1-    , curveEccG = Point 0x188da80eb03090f67cbf20eb43a18800f4ff0afd82ff1012-                    0x07192b95ffc8da78631011ed6b24cdd573f977a11e794811-    , curveEccN = 0xffffffffffffffffffffffff99def836146bc9b1b4d22831-    , curveEccH = 1-    }-typeSEC_p224k1 = CurvePrime $ CurvePrimeParam 0x00fffffffffffffffffffffffffffffffffffffffffffffffeffffe56d-paramSEC_p224k1 = CurveParameters-    { curveEccA = 0x0000000000000000000000000000000000000000000000000000000000-    , curveEccB = 0x0000000000000000000000000000000000000000000000000000000005-    , curveEccG = Point 0x00a1455b334df099df30fc28a169a467e9e47075a90f7e650eb6b7a45c-                    0x007e089fed7fba344282cafbd6f7e319f7c0b0bd59e2ca4bdb556d61a5-    , curveEccN = 0x010000000000000000000000000001dce8d2ec6184caf0a971769fb1f7-    , curveEccH = 1-    }-typeSEC_p224r1 = CurvePrime $ CurvePrimeParam 0xffffffffffffffffffffffffffffffff000000000000000000000001-paramSEC_p224r1 = CurveParameters-    { curveEccA = 0xfffffffffffffffffffffffffffffffefffffffffffffffffffffffe-    , curveEccB = 0xb4050a850c04b3abf54132565044b0b7d7bfd8ba270b39432355ffb4-    , curveEccG = Point 0xb70e0cbd6bb4bf7f321390b94a03c1d356c21122343280d6115c1d21-                    0xbd376388b5f723fb4c22dfe6cd4375a05a07476444d5819985007e34-    , curveEccN = 0xffffffffffffffffffffffffffff16a2e0b8f03e13dd29455c5c2a3d-    , curveEccH = 1-    }-typeSEC_p256k1 = CurvePrime $ CurvePrimeParam 0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f-paramSEC_p256k1 = CurveParameters-    { curveEccA = 0x0000000000000000000000000000000000000000000000000000000000000000-    , curveEccB = 0x0000000000000000000000000000000000000000000000000000000000000007-    , curveEccG = Point 0x79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798-                    0x483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8-    , curveEccN = 0xfffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141-    , curveEccH = 1-    }-typeSEC_p256r1 = CurvePrime $ CurvePrimeParam 0xffffffff00000001000000000000000000000000ffffffffffffffffffffffff-paramSEC_p256r1 = CurveParameters-    { curveEccA = 0xffffffff00000001000000000000000000000000fffffffffffffffffffffffc-    , curveEccB = 0x5ac635d8aa3a93e7b3ebbd55769886bc651d06b0cc53b0f63bce3c3e27d2604b-    , curveEccG = Point 0x6b17d1f2e12c4247f8bce6e563a440f277037d812deb33a0f4a13945d898c296-                    0x4fe342e2fe1a7f9b8ee7eb4a7c0f9e162bce33576b315ececbb6406837bf51f5-    , curveEccN = 0xffffffff00000000ffffffffffffffffbce6faada7179e84f3b9cac2fc632551-    , curveEccH = 1-    }-typeSEC_p384r1 = CurvePrime $ CurvePrimeParam 0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffeffffffff0000000000000000ffffffff-paramSEC_p384r1 = CurveParameters-    { curveEccA = 0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffeffffffff0000000000000000fffffffc-    , curveEccB = 0xb3312fa7e23ee7e4988e056be3f82d19181d9c6efe8141120314088f5013875ac656398d8a2ed19d2a85c8edd3ec2aef-    , curveEccG = Point 0xaa87ca22be8b05378eb1c71ef320ad746e1d3b628ba79b9859f741e082542a385502f25dbf55296c3a545e3872760ab7-                    0x3617de4a96262c6f5d9e98bf9292dc29f8f41dbd289a147ce9da3113b5f0b8c00a60b1ce1d7e819d7a431d7c90ea0e5f-    , curveEccN = 0xffffffffffffffffffffffffffffffffffffffffffffffffc7634d81f4372ddf581a0db248b0a77aecec196accc52973-    , curveEccH = 1-    }-typeSEC_p521r1 = CurvePrime $ CurvePrimeParam 0x01ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff-paramSEC_p521r1 = CurveParameters-    { curveEccA = 0x01fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc-    , curveEccB = 0x0051953eb9618e1c9a1f929a21a0b68540eea2da725b99b315f3b8b489918ef109e156193951ec7e937b1652c0bd3bb1bf073573df883d2c34f1ef451fd46b503f00-    , curveEccG = Point 0x00c6858e06b70404e9cd9e3ecb662395b4429c648139053fb521f828af606b4d3dbaa14b5e77efe75928fe1dc127a2ffa8de3348b3c1856a429bf97e7e31c2e5bd66-                    0x011839296a789a3bc0045c8a5fb42c7d1bd998f54449579b446817afbd17273e662c97ee72995ef42640c550b9013fad0761353c7086a272c24088be94769fd16650-    , curveEccN = 0x01fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffa51868783bf2f966b7fcc0148f709a5d03bb5c9b8899c47aebb6fb71e91386409-    , curveEccH = 1-    }+paramSEC_p160r2 =+    CurveParameters+        { curveEccA = 0x00fffffffffffffffffffffffffffffffeffffac70+        , curveEccB = 0x00b4e134d3fb59eb8bab57274904664d5af50388ba+        , curveEccG =+            Point+                0x0052dcb034293a117e1f4ff11b30f7199d3144ce6d+                0x00feaffef2e331f296e071fa0df9982cfea7d43f2e+        , curveEccN = 0x0100000000000000000000351ee786a818f3a1a16b+        , curveEccH = 1+        }+typeSEC_p192k1 =+    CurvePrime $ CurvePrimeParam 0xfffffffffffffffffffffffffffffffffffffffeffffee37+paramSEC_p192k1 =+    CurveParameters+        { curveEccA = 0x000000000000000000000000000000000000000000000000+        , curveEccB = 0x000000000000000000000000000000000000000000000003+        , curveEccG =+            Point+                0xdb4ff10ec057e9ae26b07d0280b7f4341da5d1b1eae06c7d+                0x9b2f2f6d9c5628a7844163d015be86344082aa88d95e2f9d+        , curveEccN = 0xfffffffffffffffffffffffe26f2fc170f69466a74defd8d+        , curveEccH = 1+        }+typeSEC_p192r1 =+    CurvePrime $ CurvePrimeParam 0xfffffffffffffffffffffffffffffffeffffffffffffffff+paramSEC_p192r1 =+    CurveParameters+        { curveEccA = 0xfffffffffffffffffffffffffffffffefffffffffffffffc+        , curveEccB = 0x64210519e59c80e70fa7e9ab72243049feb8deecc146b9b1+        , curveEccG =+            Point+                0x188da80eb03090f67cbf20eb43a18800f4ff0afd82ff1012+                0x07192b95ffc8da78631011ed6b24cdd573f977a11e794811+        , curveEccN = 0xffffffffffffffffffffffff99def836146bc9b1b4d22831+        , curveEccH = 1+        }+typeSEC_p224k1 =+    CurvePrime $+        CurvePrimeParam 0x00fffffffffffffffffffffffffffffffffffffffffffffffeffffe56d+paramSEC_p224k1 =+    CurveParameters+        { curveEccA = 0x0000000000000000000000000000000000000000000000000000000000+        , curveEccB = 0x0000000000000000000000000000000000000000000000000000000005+        , curveEccG =+            Point+                0x00a1455b334df099df30fc28a169a467e9e47075a90f7e650eb6b7a45c+                0x007e089fed7fba344282cafbd6f7e319f7c0b0bd59e2ca4bdb556d61a5+        , curveEccN = 0x010000000000000000000000000001dce8d2ec6184caf0a971769fb1f7+        , curveEccH = 1+        }+typeSEC_p224r1 =+    CurvePrime $+        CurvePrimeParam 0xffffffffffffffffffffffffffffffff000000000000000000000001+paramSEC_p224r1 =+    CurveParameters+        { curveEccA = 0xfffffffffffffffffffffffffffffffefffffffffffffffffffffffe+        , curveEccB = 0xb4050a850c04b3abf54132565044b0b7d7bfd8ba270b39432355ffb4+        , curveEccG =+            Point+                0xb70e0cbd6bb4bf7f321390b94a03c1d356c21122343280d6115c1d21+                0xbd376388b5f723fb4c22dfe6cd4375a05a07476444d5819985007e34+        , curveEccN = 0xffffffffffffffffffffffffffff16a2e0b8f03e13dd29455c5c2a3d+        , curveEccH = 1+        }+typeSEC_p256k1 =+    CurvePrime $+        CurvePrimeParam+            0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f+paramSEC_p256k1 =+    CurveParameters+        { curveEccA = 0x0000000000000000000000000000000000000000000000000000000000000000+        , curveEccB = 0x0000000000000000000000000000000000000000000000000000000000000007+        , curveEccG =+            Point+                0x79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798+                0x483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8+        , curveEccN = 0xfffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141+        , curveEccH = 1+        }+typeSEC_p256r1 =+    CurvePrime $+        CurvePrimeParam+            0xffffffff00000001000000000000000000000000ffffffffffffffffffffffff+paramSEC_p256r1 =+    CurveParameters+        { curveEccA = 0xffffffff00000001000000000000000000000000fffffffffffffffffffffffc+        , curveEccB = 0x5ac635d8aa3a93e7b3ebbd55769886bc651d06b0cc53b0f63bce3c3e27d2604b+        , curveEccG =+            Point+                0x6b17d1f2e12c4247f8bce6e563a440f277037d812deb33a0f4a13945d898c296+                0x4fe342e2fe1a7f9b8ee7eb4a7c0f9e162bce33576b315ececbb6406837bf51f5+        , curveEccN = 0xffffffff00000000ffffffffffffffffbce6faada7179e84f3b9cac2fc632551+        , curveEccH = 1+        }+typeSEC_p384r1 =+    CurvePrime $+        CurvePrimeParam+            0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffeffffffff0000000000000000ffffffff+paramSEC_p384r1 =+    CurveParameters+        { curveEccA =+            0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffeffffffff0000000000000000fffffffc+        , curveEccB =+            0xb3312fa7e23ee7e4988e056be3f82d19181d9c6efe8141120314088f5013875ac656398d8a2ed19d2a85c8edd3ec2aef+        , curveEccG =+            Point+                0xaa87ca22be8b05378eb1c71ef320ad746e1d3b628ba79b9859f741e082542a385502f25dbf55296c3a545e3872760ab7+                0x3617de4a96262c6f5d9e98bf9292dc29f8f41dbd289a147ce9da3113b5f0b8c00a60b1ce1d7e819d7a431d7c90ea0e5f+        , curveEccN =+            0xffffffffffffffffffffffffffffffffffffffffffffffffc7634d81f4372ddf581a0db248b0a77aecec196accc52973+        , curveEccH = 1+        }+typeSEC_p521r1 =+    CurvePrime $+        CurvePrimeParam+            0x01ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff+paramSEC_p521r1 =+    CurveParameters+        { curveEccA =+            0x01fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc+        , curveEccB =+            0x0051953eb9618e1c9a1f929a21a0b68540eea2da725b99b315f3b8b489918ef109e156193951ec7e937b1652c0bd3bb1bf073573df883d2c34f1ef451fd46b503f00+        , curveEccG =+            Point+                0x00c6858e06b70404e9cd9e3ecb662395b4429c648139053fb521f828af606b4d3dbaa14b5e77efe75928fe1dc127a2ffa8de3348b3c1856a429bf97e7e31c2e5bd66+                0x011839296a789a3bc0045c8a5fb42c7d1bd998f54449579b446817afbd17273e662c97ee72995ef42640c550b9013fad0761353c7086a272c24088be94769fd16650+        , curveEccN =+            0x01fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffa51868783bf2f966b7fcc0148f709a5d03bb5c9b8899c47aebb6fb71e91386409+        , curveEccH = 1+        } typeSEC_t113r1 = CurveBinary $ CurveBinaryParam 0x020000000000000000000000000201-paramSEC_t113r1 = CurveParameters-    { curveEccA = 0x003088250ca6e7c7fe649ce85820f7-    , curveEccB = 0x00e8bee4d3e2260744188be0e9c723-    , curveEccG = Point 0x009d73616f35f4ab1407d73562c10f-                    0x00a52830277958ee84d1315ed31886-    , curveEccN = 0x0100000000000000d9ccec8a39e56f-    , curveEccH = 2-    }+paramSEC_t113r1 =+    CurveParameters+        { curveEccA = 0x003088250ca6e7c7fe649ce85820f7+        , curveEccB = 0x00e8bee4d3e2260744188be0e9c723+        , curveEccG =+            Point+                0x009d73616f35f4ab1407d73562c10f+                0x00a52830277958ee84d1315ed31886+        , curveEccN = 0x0100000000000000d9ccec8a39e56f+        , curveEccH = 2+        } typeSEC_t113r2 = CurveBinary $ CurveBinaryParam 0x020000000000000000000000000201-paramSEC_t113r2 = CurveParameters-    { curveEccA = 0x00689918dbec7e5a0dd6dfc0aa55c7-    , curveEccB = 0x0095e9a9ec9b297bd4bf36e059184f-    , curveEccG = Point 0x01a57a6a7b26ca5ef52fcdb8164797-                    0x00b3adc94ed1fe674c06e695baba1d-    , curveEccN = 0x010000000000000108789b2496af93-    , curveEccH = 2-    }+paramSEC_t113r2 =+    CurveParameters+        { curveEccA = 0x00689918dbec7e5a0dd6dfc0aa55c7+        , curveEccB = 0x0095e9a9ec9b297bd4bf36e059184f+        , curveEccG =+            Point+                0x01a57a6a7b26ca5ef52fcdb8164797+                0x00b3adc94ed1fe674c06e695baba1d+        , curveEccN = 0x010000000000000108789b2496af93+        , curveEccH = 2+        } typeSEC_t131r1 = CurveBinary $ CurveBinaryParam 0x080000000000000000000000000000010d-paramSEC_t131r1 = CurveParameters-    { curveEccA = 0x07a11b09a76b562144418ff3ff8c2570b8-    , curveEccB = 0x0217c05610884b63b9c6c7291678f9d341-    , curveEccG = Point 0x0081baf91fdf9833c40f9c181343638399-                    0x078c6e7ea38c001f73c8134b1b4ef9e150-    , curveEccN = 0x0400000000000000023123953a9464b54d-    , curveEccH = 2-    }+paramSEC_t131r1 =+    CurveParameters+        { curveEccA = 0x07a11b09a76b562144418ff3ff8c2570b8+        , curveEccB = 0x0217c05610884b63b9c6c7291678f9d341+        , curveEccG =+            Point+                0x0081baf91fdf9833c40f9c181343638399+                0x078c6e7ea38c001f73c8134b1b4ef9e150+        , curveEccN = 0x0400000000000000023123953a9464b54d+        , curveEccH = 2+        } typeSEC_t131r2 = CurveBinary $ CurveBinaryParam 0x080000000000000000000000000000010d-paramSEC_t131r2 = CurveParameters-    { curveEccA = 0x03e5a88919d7cafcbf415f07c2176573b2-    , curveEccB = 0x04b8266a46c55657ac734ce38f018f2192-    , curveEccG = Point 0x0356dcd8f2f95031ad652d23951bb366a8-                    0x0648f06d867940a5366d9e265de9eb240f-    , curveEccN = 0x0400000000000000016954a233049ba98f-    , curveEccH = 2-    }+paramSEC_t131r2 =+    CurveParameters+        { curveEccA = 0x03e5a88919d7cafcbf415f07c2176573b2+        , curveEccB = 0x04b8266a46c55657ac734ce38f018f2192+        , curveEccG =+            Point+                0x0356dcd8f2f95031ad652d23951bb366a8+                0x0648f06d867940a5366d9e265de9eb240f+        , curveEccN = 0x0400000000000000016954a233049ba98f+        , curveEccH = 2+        } typeSEC_t163k1 = CurveBinary $ CurveBinaryParam 0x0800000000000000000000000000000000000000c9-paramSEC_t163k1 = CurveParameters-    { curveEccA = 0x000000000000000000000000000000000000000001-    , curveEccB = 0x000000000000000000000000000000000000000001-    , curveEccG = Point 0x02fe13c0537bbc11acaa07d793de4e6d5e5c94eee8-                    0x0289070fb05d38ff58321f2e800536d538ccdaa3d9-    , curveEccN = 0x04000000000000000000020108a2e0cc0d99f8a5ef-    , curveEccH = 2-    }+paramSEC_t163k1 =+    CurveParameters+        { curveEccA = 0x000000000000000000000000000000000000000001+        , curveEccB = 0x000000000000000000000000000000000000000001+        , curveEccG =+            Point+                0x02fe13c0537bbc11acaa07d793de4e6d5e5c94eee8+                0x0289070fb05d38ff58321f2e800536d538ccdaa3d9+        , curveEccN = 0x04000000000000000000020108a2e0cc0d99f8a5ef+        , curveEccH = 2+        } typeSEC_t163r1 = CurveBinary $ CurveBinaryParam 0x0800000000000000000000000000000000000000c9-paramSEC_t163r1 = CurveParameters-    { curveEccA = 0x07b6882caaefa84f9554ff8428bd88e246d2782ae2-    , curveEccB = 0x0713612dcddcb40aab946bda29ca91f73af958afd9-    , curveEccG = Point 0x0369979697ab43897789566789567f787a7876a654-                    0x00435edb42efafb2989d51fefce3c80988f41ff883-    , curveEccN = 0x03ffffffffffffffffffff48aab689c29ca710279b-    , curveEccH = 2-    }+paramSEC_t163r1 =+    CurveParameters+        { curveEccA = 0x07b6882caaefa84f9554ff8428bd88e246d2782ae2+        , curveEccB = 0x0713612dcddcb40aab946bda29ca91f73af958afd9+        , curveEccG =+            Point+                0x0369979697ab43897789566789567f787a7876a654+                0x00435edb42efafb2989d51fefce3c80988f41ff883+        , curveEccN = 0x03ffffffffffffffffffff48aab689c29ca710279b+        , curveEccH = 2+        } typeSEC_t163r2 = CurveBinary $ CurveBinaryParam 0x0800000000000000000000000000000000000000c9-paramSEC_t163r2 = CurveParameters-    { curveEccA = 0x000000000000000000000000000000000000000001-    , curveEccB = 0x020a601907b8c953ca1481eb10512f78744a3205fd-    , curveEccG = Point 0x03f0eba16286a2d57ea0991168d4994637e8343e36-                    0x00d51fbc6c71a0094fa2cdd545b11c5c0c797324f1-    , curveEccN = 0x040000000000000000000292fe77e70c12a4234c33-    , curveEccH = 2-    }-typeSEC_t193r1 = CurveBinary $ CurveBinaryParam 0x02000000000000000000000000000000000000000000008001-paramSEC_t193r1 = CurveParameters-    { curveEccA = 0x0017858feb7a98975169e171f77b4087de098ac8a911df7b01-    , curveEccB = 0x00fdfb49bfe6c3a89facadaa7a1e5bbc7cc1c2e5d831478814-    , curveEccG = Point 0x01f481bc5f0ff84a74ad6cdf6fdef4bf6179625372d8c0c5e1-                    0x0025e399f2903712ccf3ea9e3a1ad17fb0b3201b6af7ce1b05-    , curveEccN = 0x01000000000000000000000000c7f34a778f443acc920eba49-    , curveEccH = 2-    }-typeSEC_t193r2 = CurveBinary $ CurveBinaryParam 0x02000000000000000000000000000000000000000000008001-paramSEC_t193r2 = CurveParameters-    { curveEccA = 0x0163f35a5137c2ce3ea6ed8667190b0bc43ecd69977702709b-    , curveEccB = 0x00c9bb9e8927d4d64c377e2ab2856a5b16e3efb7f61d4316ae-    , curveEccG = Point 0x00d9b67d192e0367c803f39e1a7e82ca14a651350aae617e8f-                    0x01ce94335607c304ac29e7defbd9ca01f596f927224cdecf6c-    , curveEccN = 0x010000000000000000000000015aab561b005413ccd4ee99d5-    , curveEccH = 2-    }-typeSEC_t233k1 = CurveBinary $ CurveBinaryParam 0x020000000000000000000000000000000000000004000000000000000001-paramSEC_t233k1 = CurveParameters-    { curveEccA = 0x000000000000000000000000000000000000000000000000000000000000-    , curveEccB = 0x000000000000000000000000000000000000000000000000000000000001-    , curveEccG = Point 0x017232ba853a7e731af129f22ff4149563a419c26bf50a4c9d6eefad6126-                    0x01db537dece819b7f70f555a67c427a8cd9bf18aeb9b56e0c11056fae6a3-    , curveEccN = 0x008000000000000000000000000000069d5bb915bcd46efb1ad5f173abdf-    , curveEccH = 4-    }-typeSEC_t233r1 = CurveBinary $ CurveBinaryParam 0x020000000000000000000000000000000000000004000000000000000001-paramSEC_t233r1 = CurveParameters-    { curveEccA = 0x000000000000000000000000000000000000000000000000000000000001-    , curveEccB = 0x0066647ede6c332c7f8c0923bb58213b333b20e9ce4281fe115f7d8f90ad-    , curveEccG = Point 0x00fac9dfcbac8313bb2139f1bb755fef65bc391f8b36f8f8eb7371fd558b-                    0x01006a08a41903350678e58528bebf8a0beff867a7ca36716f7e01f81052-    , curveEccN = 0x01000000000000000000000000000013e974e72f8a6922031d2603cfe0d7-    , curveEccH = 2-    }-typeSEC_t239k1 = CurveBinary $ CurveBinaryParam 0x800000000000000000004000000000000000000000000000000000000001-paramSEC_t239k1 = CurveParameters-    { curveEccA = 0x000000000000000000000000000000000000000000000000000000000000-    , curveEccB = 0x000000000000000000000000000000000000000000000000000000000001-    , curveEccG = Point 0x29a0b6a887a983e9730988a68727a8b2d126c44cc2cc7b2a6555193035dc-                    0x76310804f12e549bdb011c103089e73510acb275fc312a5dc6b76553f0ca-    , curveEccN = 0x2000000000000000000000000000005a79fec67cb6e91f1c1da800e478a5-    , curveEccH = 4-    }-typeSEC_t283k1 = CurveBinary $ CurveBinaryParam 0x0800000000000000000000000000000000000000000000000000000000000000000010a1-paramSEC_t283k1 = CurveParameters-    { curveEccA = 0x000000000000000000000000000000000000000000000000000000000000000000000000-    , curveEccB = 0x000000000000000000000000000000000000000000000000000000000000000000000001-    , curveEccG = Point 0x0503213f78ca44883f1a3b8162f188e553cd265f23c1567a16876913b0c2ac2458492836-                    0x01ccda380f1c9e318d90f95d07e5426fe87e45c0e8184698e45962364e34116177dd2259-    , curveEccN = 0x01ffffffffffffffffffffffffffffffffffe9ae2ed07577265dff7f94451e061e163c61-    , curveEccH = 4-    }-typeSEC_t283r1 = CurveBinary $ CurveBinaryParam 0x0800000000000000000000000000000000000000000000000000000000000000000010a1-paramSEC_t283r1 = CurveParameters-    { curveEccA = 0x000000000000000000000000000000000000000000000000000000000000000000000001-    , curveEccB = 0x027b680ac8b8596da5a4af8a19a0303fca97fd7645309fa2a581485af6263e313b79a2f5-    , curveEccG = Point 0x05f939258db7dd90e1934f8c70b0dfec2eed25b8557eac9c80e2e198f8cdbecd86b12053-                    0x03676854fe24141cb98fe6d4b20d02b4516ff702350eddb0826779c813f0df45be8112f4-    , curveEccN = 0x03ffffffffffffffffffffffffffffffffffef90399660fc938a90165b042a7cefadb307-    , curveEccH = 2-    }-typeSEC_t409k1 = CurveBinary $ CurveBinaryParam 0x02000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000001-paramSEC_t409k1 = CurveParameters-    { curveEccA = 0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000-    , curveEccB = 0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001-    , curveEccG = Point 0x0060f05f658f49c1ad3ab1890f7184210efd0987e307c84c27accfb8f9f67cc2c460189eb5aaaa62ee222eb1b35540cfe9023746-                    0x01e369050b7c4e42acba1dacbf04299c3460782f918ea427e6325165e9ea10e3da5f6c42e9c55215aa9ca27a5863ec48d8e0286b-    , curveEccN = 0x007ffffffffffffffffffffffffffffffffffffffffffffffffffe5f83b2d4ea20400ec4557d5ed3e3e7ca5b4b5c83b8e01e5fcf-    , curveEccH = 4-    }-typeSEC_t409r1 = CurveBinary $ CurveBinaryParam 0x02000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000001-paramSEC_t409r1 = CurveParameters-    { curveEccA = 0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001-    , curveEccB = 0x0021a5c2c8ee9feb5c4b9a753b7b476b7fd6422ef1f3dd674761fa99d6ac27c8a9a197b272822f6cd57a55aa4f50ae317b13545f-    , curveEccG = Point 0x015d4860d088ddb3496b0c6064756260441cde4af1771d4db01ffe5b34e59703dc255a868a1180515603aeab60794e54bb7996a7-                    0x0061b1cfab6be5f32bbfa78324ed106a7636b9c5a7bd198d0158aa4f5488d08f38514f1fdf4b4f40d2181b3681c364ba0273c706-    , curveEccN = 0x010000000000000000000000000000000000000000000000000001e2aad6a612f33307be5fa47c3c9e052f838164cd37d9a21173-    , curveEccH = 2-    }-typeSEC_t571k1 = CurveBinary $ CurveBinaryParam 0x080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000425-paramSEC_t571k1 = CurveParameters-    { curveEccA = 0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000-    , curveEccB = 0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001-    , curveEccG = Point 0x026eb7a859923fbc82189631f8103fe4ac9ca2970012d5d46024804801841ca44370958493b205e647da304db4ceb08cbbd1ba39494776fb988b47174dca88c7e2945283a01c8972-                    0x0349dc807f4fbf374f4aeade3bca95314dd58cec9f307a54ffc61efc006d8a2c9d4979c0ac44aea74fbebbb9f772aedcb620b01a7ba7af1b320430c8591984f601cd4c143ef1c7a3-    , curveEccN = 0x020000000000000000000000000000000000000000000000000000000000000000000000131850e1f19a63e4b391a8db917f4138b630d84be5d639381e91deb45cfe778f637c1001-    , curveEccH = 4-    }-typeSEC_t571r1 = CurveBinary $ CurveBinaryParam 0x080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000425-paramSEC_t571r1 = CurveParameters-    { curveEccA = 0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001-    , curveEccB = 0x02f40e7e2221f295de297117b7f3d62f5c6a97ffcb8ceff1cd6ba8ce4a9a18ad84ffabbd8efa59332be7ad6756a66e294afd185a78ff12aa520e4de739baca0c7ffeff7f2955727a-    , curveEccG = Point 0x0303001d34b856296c16c0d40d3cd7750a93d1d2955fa80aa5f40fc8db7b2abdbde53950f4c0d293cdd711a35b67fb1499ae60038614f1394abfa3b4c850d927e1e7769c8eec2d19-                    0x037bf27342da639b6dccfffeb73d69d78c6c27a6009cbbca1980f8533921e8a684423e43bab08a576291af8f461bb2a8b3531d2f0485c19b16e2f1516e23dd3c1a4827af1b8ac15b-    , curveEccN = 0x03ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe661ce18ff55987308059b186823851ec7dd9ca1161de93d5174d66e8382e9bb2fe84e47-    , curveEccH = 2-    }+paramSEC_t163r2 =+    CurveParameters+        { curveEccA = 0x000000000000000000000000000000000000000001+        , curveEccB = 0x020a601907b8c953ca1481eb10512f78744a3205fd+        , curveEccG =+            Point+                0x03f0eba16286a2d57ea0991168d4994637e8343e36+                0x00d51fbc6c71a0094fa2cdd545b11c5c0c797324f1+        , curveEccN = 0x040000000000000000000292fe77e70c12a4234c33+        , curveEccH = 2+        }+typeSEC_t193r1 =+    CurveBinary $+        CurveBinaryParam 0x02000000000000000000000000000000000000000000008001+paramSEC_t193r1 =+    CurveParameters+        { curveEccA = 0x0017858feb7a98975169e171f77b4087de098ac8a911df7b01+        , curveEccB = 0x00fdfb49bfe6c3a89facadaa7a1e5bbc7cc1c2e5d831478814+        , curveEccG =+            Point+                0x01f481bc5f0ff84a74ad6cdf6fdef4bf6179625372d8c0c5e1+                0x0025e399f2903712ccf3ea9e3a1ad17fb0b3201b6af7ce1b05+        , curveEccN = 0x01000000000000000000000000c7f34a778f443acc920eba49+        , curveEccH = 2+        }+typeSEC_t193r2 =+    CurveBinary $+        CurveBinaryParam 0x02000000000000000000000000000000000000000000008001+paramSEC_t193r2 =+    CurveParameters+        { curveEccA = 0x0163f35a5137c2ce3ea6ed8667190b0bc43ecd69977702709b+        , curveEccB = 0x00c9bb9e8927d4d64c377e2ab2856a5b16e3efb7f61d4316ae+        , curveEccG =+            Point+                0x00d9b67d192e0367c803f39e1a7e82ca14a651350aae617e8f+                0x01ce94335607c304ac29e7defbd9ca01f596f927224cdecf6c+        , curveEccN = 0x010000000000000000000000015aab561b005413ccd4ee99d5+        , curveEccH = 2+        }+typeSEC_t233k1 =+    CurveBinary $+        CurveBinaryParam 0x020000000000000000000000000000000000000004000000000000000001+paramSEC_t233k1 =+    CurveParameters+        { curveEccA = 0x000000000000000000000000000000000000000000000000000000000000+        , curveEccB = 0x000000000000000000000000000000000000000000000000000000000001+        , curveEccG =+            Point+                0x017232ba853a7e731af129f22ff4149563a419c26bf50a4c9d6eefad6126+                0x01db537dece819b7f70f555a67c427a8cd9bf18aeb9b56e0c11056fae6a3+        , curveEccN = 0x008000000000000000000000000000069d5bb915bcd46efb1ad5f173abdf+        , curveEccH = 4+        }+typeSEC_t233r1 =+    CurveBinary $+        CurveBinaryParam 0x020000000000000000000000000000000000000004000000000000000001+paramSEC_t233r1 =+    CurveParameters+        { curveEccA = 0x000000000000000000000000000000000000000000000000000000000001+        , curveEccB = 0x0066647ede6c332c7f8c0923bb58213b333b20e9ce4281fe115f7d8f90ad+        , curveEccG =+            Point+                0x00fac9dfcbac8313bb2139f1bb755fef65bc391f8b36f8f8eb7371fd558b+                0x01006a08a41903350678e58528bebf8a0beff867a7ca36716f7e01f81052+        , curveEccN = 0x01000000000000000000000000000013e974e72f8a6922031d2603cfe0d7+        , curveEccH = 2+        }+typeSEC_t239k1 =+    CurveBinary $+        CurveBinaryParam 0x800000000000000000004000000000000000000000000000000000000001+paramSEC_t239k1 =+    CurveParameters+        { curveEccA = 0x000000000000000000000000000000000000000000000000000000000000+        , curveEccB = 0x000000000000000000000000000000000000000000000000000000000001+        , curveEccG =+            Point+                0x29a0b6a887a983e9730988a68727a8b2d126c44cc2cc7b2a6555193035dc+                0x76310804f12e549bdb011c103089e73510acb275fc312a5dc6b76553f0ca+        , curveEccN = 0x2000000000000000000000000000005a79fec67cb6e91f1c1da800e478a5+        , curveEccH = 4+        }+typeSEC_t283k1 =+    CurveBinary $+        CurveBinaryParam+            0x0800000000000000000000000000000000000000000000000000000000000000000010a1+paramSEC_t283k1 =+    CurveParameters+        { curveEccA =+            0x000000000000000000000000000000000000000000000000000000000000000000000000+        , curveEccB =+            0x000000000000000000000000000000000000000000000000000000000000000000000001+        , curveEccG =+            Point+                0x0503213f78ca44883f1a3b8162f188e553cd265f23c1567a16876913b0c2ac2458492836+                0x01ccda380f1c9e318d90f95d07e5426fe87e45c0e8184698e45962364e34116177dd2259+        , curveEccN =+            0x01ffffffffffffffffffffffffffffffffffe9ae2ed07577265dff7f94451e061e163c61+        , curveEccH = 4+        }+typeSEC_t283r1 =+    CurveBinary $+        CurveBinaryParam+            0x0800000000000000000000000000000000000000000000000000000000000000000010a1+paramSEC_t283r1 =+    CurveParameters+        { curveEccA =+            0x000000000000000000000000000000000000000000000000000000000000000000000001+        , curveEccB =+            0x027b680ac8b8596da5a4af8a19a0303fca97fd7645309fa2a581485af6263e313b79a2f5+        , curveEccG =+            Point+                0x05f939258db7dd90e1934f8c70b0dfec2eed25b8557eac9c80e2e198f8cdbecd86b12053+                0x03676854fe24141cb98fe6d4b20d02b4516ff702350eddb0826779c813f0df45be8112f4+        , curveEccN =+            0x03ffffffffffffffffffffffffffffffffffef90399660fc938a90165b042a7cefadb307+        , curveEccH = 2+        }+typeSEC_t409k1 =+    CurveBinary $+        CurveBinaryParam+            0x02000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000001+paramSEC_t409k1 =+    CurveParameters+        { curveEccA =+            0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000+        , curveEccB =+            0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001+        , curveEccG =+            Point+                0x0060f05f658f49c1ad3ab1890f7184210efd0987e307c84c27accfb8f9f67cc2c460189eb5aaaa62ee222eb1b35540cfe9023746+                0x01e369050b7c4e42acba1dacbf04299c3460782f918ea427e6325165e9ea10e3da5f6c42e9c55215aa9ca27a5863ec48d8e0286b+        , curveEccN =+            0x007ffffffffffffffffffffffffffffffffffffffffffffffffffe5f83b2d4ea20400ec4557d5ed3e3e7ca5b4b5c83b8e01e5fcf+        , curveEccH = 4+        }+typeSEC_t409r1 =+    CurveBinary $+        CurveBinaryParam+            0x02000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000001+paramSEC_t409r1 =+    CurveParameters+        { curveEccA =+            0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001+        , curveEccB =+            0x0021a5c2c8ee9feb5c4b9a753b7b476b7fd6422ef1f3dd674761fa99d6ac27c8a9a197b272822f6cd57a55aa4f50ae317b13545f+        , curveEccG =+            Point+                0x015d4860d088ddb3496b0c6064756260441cde4af1771d4db01ffe5b34e59703dc255a868a1180515603aeab60794e54bb7996a7+                0x0061b1cfab6be5f32bbfa78324ed106a7636b9c5a7bd198d0158aa4f5488d08f38514f1fdf4b4f40d2181b3681c364ba0273c706+        , curveEccN =+            0x010000000000000000000000000000000000000000000000000001e2aad6a612f33307be5fa47c3c9e052f838164cd37d9a21173+        , curveEccH = 2+        }+typeSEC_t571k1 =+    CurveBinary $+        CurveBinaryParam+            0x080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000425+paramSEC_t571k1 =+    CurveParameters+        { curveEccA =+            0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000+        , curveEccB =+            0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001+        , curveEccG =+            Point+                0x026eb7a859923fbc82189631f8103fe4ac9ca2970012d5d46024804801841ca44370958493b205e647da304db4ceb08cbbd1ba39494776fb988b47174dca88c7e2945283a01c8972+                0x0349dc807f4fbf374f4aeade3bca95314dd58cec9f307a54ffc61efc006d8a2c9d4979c0ac44aea74fbebbb9f772aedcb620b01a7ba7af1b320430c8591984f601cd4c143ef1c7a3+        , curveEccN =+            0x020000000000000000000000000000000000000000000000000000000000000000000000131850e1f19a63e4b391a8db917f4138b630d84be5d639381e91deb45cfe778f637c1001+        , curveEccH = 4+        }+typeSEC_t571r1 =+    CurveBinary $+        CurveBinaryParam+            0x080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000425+paramSEC_t571r1 =+    CurveParameters+        { curveEccA =+            0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001+        , curveEccB =+            0x02f40e7e2221f295de297117b7f3d62f5c6a97ffcb8ceff1cd6ba8ce4a9a18ad84ffabbd8efa59332be7ad6756a66e294afd185a78ff12aa520e4de739baca0c7ffeff7f2955727a+        , curveEccG =+            Point+                0x0303001d34b856296c16c0d40d3cd7750a93d1d2955fa80aa5f40fc8db7b2abdbde53950f4c0d293cdd711a35b67fb1499ae60038614f1394abfa3b4c850d927e1e7769c8eec2d19+                0x037bf27342da639b6dccfffeb73d69d78c6c27a6009cbbca1980f8533921e8a684423e43bab08a576291af8f461bb2a8b3531d2f0485c19b16e2f1516e23dd3c1a4827af1b8ac15b+        , curveEccN =+            0x03ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe661ce18ff55987308059b186823851ec7dd9ca1161de93d5174d66e8382e9bb2fe84e47+        , curveEccH = 2+        }
Crypto/Error.hs view
@@ -4,9 +4,8 @@ -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : Stable -- Portability : Excellent----module Crypto.Error-    ( module Crypto.Error.Types-    ) where+module Crypto.Error (+    module Crypto.Error.Types,+) where  import Crypto.Error.Types
Crypto/Error/Types.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Error.Types -- License     : BSD-style@@ -6,53 +9,55 @@ -- Portability : Good -- -- Cryptographic Error enumeration and handling----{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE TypeFamilies       #-}-module Crypto.Error.Types-    ( CryptoError(..)-    , CryptoFailable(..)-    , throwCryptoErrorIO-    , throwCryptoError-    , onCryptoFailure-    , eitherCryptoError-    , maybeCryptoError-    ) where+module Crypto.Error.Types (+    CryptoError (..),+    CryptoFailable (..),+    throwCryptoErrorIO,+    throwCryptoError,+    onCryptoFailure,+    eitherCryptoError,+    maybeCryptoError,+) where  import qualified Control.Exception as E-import           Data.Data--import           Basement.Monad (MonadFailure(..))+import Data.Data  -- | Enumeration of all possible errors that can be found in this library-data CryptoError =-    -- symmetric cipher errors+data CryptoError+    = -- symmetric cipher errors       CryptoError_KeySizeInvalid     | CryptoError_IvSizeInvalid     | CryptoError_SeedSizeInvalid     | CryptoError_AEADModeNotSupported-    -- public key cryptography error-    | CryptoError_SecretKeySizeInvalid+    | -- public key cryptography error+      CryptoError_SecretKeySizeInvalid     | CryptoError_SecretKeyStructureInvalid     | CryptoError_PublicKeySizeInvalid     | CryptoError_SharedSecretSizeInvalid-    -- elliptic cryptography error-    | CryptoError_EcScalarOutOfBounds+    | -- elliptic cryptography error+      CryptoError_EcScalarOutOfBounds     | CryptoError_PointSizeInvalid     | CryptoError_PointFormatInvalid     | CryptoError_PointFormatUnsupported     | CryptoError_PointCoordinatesInvalid     | CryptoError_ScalarMultiplicationInvalid-    -- Message authentification error-    | CryptoError_MacKeyInvalid+    | -- Message authentification error+      CryptoError_MacKeyInvalid     | CryptoError_AuthenticationTagSizeInvalid-    -- Prime generation error-    | CryptoError_PrimeSizeInvalid-    -- Parameter errors-    | CryptoError_SaltTooSmall+    | -- Prime generation error+      CryptoError_PrimeSizeInvalid+    | -- Parameter errors+      CryptoError_SaltTooSmall     | CryptoError_OutputLengthTooSmall     | CryptoError_OutputLengthTooBig-    deriving (Show,Eq,Enum,Data)+    | -- | A parameter is outside the range the algorithm accepts.  Appended to+      -- keep the 'Enum' values of the constructors above unchanged.+      CryptoError_ParameterInvalid+    | -- | A point satisfies the curve equation but lies outside the subgroup+      -- the base point generates, so multiplying it would answer modulo a+      -- small order.  Appended for the same reason as the constructor above.+      CryptoError_PointSubgroupInvalid+    deriving (Show, Eq, Enum, Data)  instance E.Exception CryptoError @@ -63,23 +68,22 @@ -- * 'CryptoPassed' : The computation succeeded, and contains the result of the computation -- -- * 'CryptoFailed' : The computation failed, and contains the cryptographic error associated----data CryptoFailable a =-      CryptoPassed a+data CryptoFailable a+    = CryptoPassed a     | CryptoFailed CryptoError     deriving (Show)  instance Eq a => Eq (CryptoFailable a) where-    (==) (CryptoPassed a)  (CryptoPassed b)  = a == b+    (==) (CryptoPassed a) (CryptoPassed b) = a == b     (==) (CryptoFailed e1) (CryptoFailed e2) = e1 == e2-    (==) _                 _                 = False+    (==) _ _ = False  instance Functor CryptoFailable where     fmap f (CryptoPassed a) = CryptoPassed (f a)     fmap _ (CryptoFailed r) = CryptoFailed r  instance Applicative CryptoFailable where-    pure a     = CryptoPassed a+    pure a = CryptoPassed a     (<*>) fm m = fm >>= \p -> m >>= \r2 -> return (p r2) instance Monad CryptoFailable where     return = pure@@ -88,10 +92,6 @@             CryptoPassed a -> m2 a             CryptoFailed e -> CryptoFailed e -instance MonadFailure CryptoFailable where-    type Failure CryptoFailable = CryptoError-    mFail = CryptoFailed- -- | Throw an CryptoError as exception on CryptoFailed result, -- otherwise return the computed value throwCryptoErrorIO :: CryptoFailable a -> IO a@@ -105,8 +105,8 @@  -- | Simple 'either' like combinator for CryptoFailable type onCryptoFailure :: (CryptoError -> r) -> (a -> r) -> CryptoFailable a -> r-onCryptoFailure onError _         (CryptoFailed e) = onError e-onCryptoFailure _       onSuccess (CryptoPassed r) = onSuccess r+onCryptoFailure onError _ (CryptoFailed e) = onError e+onCryptoFailure _ onSuccess (CryptoPassed r) = onSuccess r  -- | Transform a CryptoFailable to an Either eitherCryptoError :: CryptoFailable a -> Either CryptoError a
Crypto/Hash.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE ScopedTypeVariables #-}+ -- | -- Module      : Crypto.Hash -- License     : BSD-style@@ -15,54 +18,51 @@ -- > -- > hexSha3_512 :: ByteString -> String -- > hexSha3_512 bs = show (hash bs :: Digest SHA3_512)----{-# LANGUAGE ScopedTypeVariables #-}-{-# LANGUAGE BangPatterns        #-}-module Crypto.Hash-    (+module Crypto.Hash (     -- * Types-      Context-    , Digest+    Context,+    Digest,+     -- * Functions-    , digestFromByteString+    digestFromByteString,+     -- * Hash methods parametrized by algorithm-    , hashInitWith-    , hashWith-    , hashPrefixWith+    hashInitWith,+    hashWith,+    hashPrefixWith,+     -- * Hash methods-    , hashInit-    , hashUpdates-    , hashUpdate-    , hashFinalize-    , hashFinalizePrefix-    , hashBlockSize-    , hashDigestSize-    , hash-    , hashPrefix-    , hashlazy+    hashInit,+    hashUpdates,+    hashUpdate,+    hashFinalize,+    hashFinalizePrefix,+    hashBlockSize,+    hashDigestSize,+    hash,+    hashPrefix,+    hashlazy,+     -- * Hash algorithms-    , module Crypto.Hash.Algorithms-    ) where+    module Crypto.Hash.Algorithms,+) where -import           Basement.Types.OffsetSize (CountOf (..))-import           Basement.Block (Block, unsafeFreeze)-import           Basement.Block.Mutable (copyFromPtr, new)-import           Crypto.Internal.Compat (unsafeDoIO)-import           Crypto.Hash.Types-import           Crypto.Hash.Algorithms-import           Foreign.Ptr (Ptr, plusPtr)-import           Crypto.Internal.ByteArray (ByteArrayAccess)+import Crypto.Hash.Algorithms+import Crypto.Hash.Types+import Crypto.Internal.ByteArray (ByteArrayAccess, allocAndFreezePrim) import qualified Crypto.Internal.ByteArray as B import qualified Data.ByteString.Lazy as L-import           Data.Word (Word8)-import           Data.Int (Int32)+import Data.Int (Int32)+import qualified Foreign.Marshal.Utils as FMU+import Foreign.Ptr (Ptr, castPtr, plusPtr)  -- | Hash a strict bytestring into a digest. hash :: (ByteArrayAccess ba, HashAlgorithm a) => ba -> Digest a hash bs = hashFinalize $ hashUpdate hashInit bs  -- | Hash the first N bytes of a bytestring, with code path independent from N.-hashPrefix :: (ByteArrayAccess ba, HashAlgorithmPrefix a) => ba -> Int -> Digest a+hashPrefix+    :: (ByteArrayAccess ba, HashAlgorithmPrefix a) => ba -> Int -> Digest a hashPrefix = hashFinalizePrefix hashInit  -- | Hash a lazy bytestring into a digest.@@ -70,24 +70,27 @@ hashlazy lbs = hashFinalize $ hashUpdates hashInit (L.toChunks lbs)  -- | Initialize a new context for this hash algorithm-hashInit :: forall a . HashAlgorithm a => Context a+hashInit :: forall a. HashAlgorithm a => Context a hashInit = Context $ B.allocAndFreeze (hashInternalContextSize (undefined :: a)) $ \(ptr :: Ptr (Context a)) ->     hashInternalInit ptr  -- | run hashUpdates on one single bytestring and return the updated context.-hashUpdate :: (ByteArrayAccess ba, HashAlgorithm a) => Context a -> ba -> Context a+hashUpdate+    :: (ByteArrayAccess ba, HashAlgorithm a) => Context a -> ba -> Context a hashUpdate ctx b-    | B.null b  = ctx+    | B.null b = ctx     | otherwise = hashUpdates ctx [b]  -- | Update the context with a list of strict bytestring, -- and return a new context with the updates.-hashUpdates :: forall a ba . (HashAlgorithm a, ByteArrayAccess ba)-            => Context a-            -> [ba]-            -> Context a+hashUpdates+    :: forall a ba+     . (HashAlgorithm a, ByteArrayAccess ba)+    => Context a+    -> [ba]+    -> Context a hashUpdates c l-    | null ls   = c+    | null ls = c     | otherwise = Context $ B.copyAndFreeze c $ \(ctx :: Ptr (Context a)) ->         mapM_ (\b -> B.withByteArray b (processBlocks ctx (B.length b))) ls   where@@ -97,18 +100,24 @@         | bytesLeft == 0 = return ()         | otherwise = do             hashInternalUpdate ctx dataPtr (fromIntegral actuallyProcessed)-            processBlocks ctx (bytesLeft - actuallyProcessed) (dataPtr `plusPtr` actuallyProcessed)-        where-            actuallyProcessed = min bytesLeft (fromIntegral (maxBound :: Int32))+            processBlocks+                ctx+                (bytesLeft - actuallyProcessed)+                (dataPtr `plusPtr` actuallyProcessed)+      where+        actuallyProcessed = min bytesLeft (fromIntegral (maxBound :: Int32))  -- | Finalize a context and return a digest.-hashFinalize :: forall a . HashAlgorithm a-             => Context a-             -> Digest a-hashFinalize !c =-    Digest $ B.allocAndFreeze (hashDigestSize (undefined :: a)) $ \(dig :: Ptr (Digest a)) -> do-        ((!_) :: B.Bytes) <- B.copy c $ \(ctx :: Ptr (Context a)) -> hashInternalFinalize ctx dig-        return ()+hashFinalize+    :: forall a+     . HashAlgorithm a+    => Context a+    -> Digest a+hashFinalize !c = Digest $+    allocAndFreezePrim (hashDigestSize (undefined :: a)) $+        \(dig :: Ptr (Digest a)) -> do+            ((!_) :: B.Bytes) <- B.copy c $ \(ctx :: Ptr (Context a)) -> hashInternalFinalize ctx dig+            return ()  -- | Update the context with the first N bytes of a bytestring and return the -- digest.  The code path is independent from N but much slower than a normal@@ -116,17 +125,25 @@ -- order to exclude a variable padding, without leaking the padding length.  The -- begining of the message, never impacted by the padding, should preferably go -- through 'hashUpdate' for better performance.-hashFinalizePrefix :: forall a ba . (HashAlgorithmPrefix a, ByteArrayAccess ba)-                   => Context a-                   -> ba-                   -> Int-                   -> Digest a-hashFinalizePrefix !c b len =-    Digest $ B.allocAndFreeze (hashDigestSize (undefined :: a)) $ \(dig :: Ptr (Digest a)) -> do-        ((!_) :: B.Bytes) <- B.copy c $ \(ctx :: Ptr (Context a)) ->-            B.withByteArray b $ \d ->-                hashInternalFinalizePrefix ctx d (fromIntegral $ B.length b) (fromIntegral len) dig-        return ()+hashFinalizePrefix+    :: forall a ba+     . (HashAlgorithmPrefix a, ByteArrayAccess ba)+    => Context a+    -> ba+    -> Int+    -> Digest a+hashFinalizePrefix !c b len = Digest $+    allocAndFreezePrim (hashDigestSize (undefined :: a)) $+        \(dig :: Ptr (Digest a)) -> do+            ((!_) :: B.Bytes) <- B.copy c $ \(ctx :: Ptr (Context a)) ->+                B.withByteArray b $ \d ->+                    hashInternalFinalizePrefix+                        ctx+                        d+                        (fromIntegral $ B.length b)+                        (fromIntegral len)+                        dig+            return ()  -- | Initialize a new context for a specified hash algorithm hashInitWith :: HashAlgorithm alg => alg -> Context alg@@ -137,25 +154,24 @@ hashWith _ = hash  -- | Run the 'hashPrefix' function but takes an explicit hash algorithm parameter-hashPrefixWith :: (ByteArrayAccess ba, HashAlgorithmPrefix alg) => alg -> ba -> Int -> Digest alg+hashPrefixWith+    :: (ByteArrayAccess ba, HashAlgorithmPrefix alg) => alg -> ba -> Int -> Digest alg hashPrefixWith _ = hashPrefix  -- | Try to transform a bytearray into a Digest of specific algorithm. -- -- If the digest is not the right size for the algorithm specified, then -- Nothing is returned.-digestFromByteString :: forall a ba . (HashAlgorithm a, ByteArrayAccess ba) => ba -> Maybe (Digest a)+digestFromByteString+    :: forall a ba. (HashAlgorithm a, ByteArrayAccess ba) => ba -> Maybe (Digest a) digestFromByteString = from undefined   where-        from :: a -> ba -> Maybe (Digest a)-        from alg bs-            | B.length bs == (hashDigestSize alg) = Just $ Digest $ unsafeDoIO $ copyBytes bs-            | otherwise                           = Nothing+    from :: a -> ba -> Maybe (Digest a)+    from alg bs+        | B.length bs == (hashDigestSize alg) =+            Just $ Digest $ copyByteArray bs+        | otherwise = Nothing -        copyBytes :: ba -> IO (Block Word8)-        copyBytes ba = do-            muArray <- new count-            B.withByteArray ba $ \ptr -> copyFromPtr ptr muArray 0 count-            unsafeFreeze muArray-          where-            count = CountOf (B.length ba)+    copyByteArray ba = allocAndFreezePrim (B.length ba) $ \dst ->+        B.withByteArray ba $ \src ->+            FMU.copyBytes dst (castPtr src) (B.length ba)
Crypto/Hash/Algorithms.hs view
@@ -6,75 +6,79 @@ -- Portability : unknown -- -- Definitions of known hash algorithms----module Crypto.Hash.Algorithms-    ( HashAlgorithm-    , HashAlgorithmPrefix+module Crypto.Hash.Algorithms (+    HashAlgorithm,+    HashAlgorithmPrefix,+     -- * Hash algorithms-    , Blake2s_160(..)-    , Blake2s_224(..)-    , Blake2s_256(..)-    , Blake2sp_224(..)-    , Blake2sp_256(..)-    , Blake2b_160(..)-    , Blake2b_224(..)-    , Blake2b_256(..)-    , Blake2b_384(..)-    , Blake2b_512(..)-    , Blake2bp_512(..)-    , MD2(..)-    , MD4(..)-    , MD5(..)-    , SHA1(..)-    , SHA224(..)-    , SHA256(..)-    , SHA384(..)-    , SHA512(..)-    , SHA512t_224(..)-    , SHA512t_256(..)-    , RIPEMD160(..)-    , Tiger(..)-    , Keccak_224(..)-    , Keccak_256(..)-    , Keccak_384(..)-    , Keccak_512(..)-    , SHA3_224(..)-    , SHA3_256(..)-    , SHA3_384(..)-    , SHA3_512(..)-    , SHAKE128(..)-    , SHAKE256(..)-    , Blake2b(..), Blake2bp(..)-    , Blake2s(..), Blake2sp(..)-    , Skein256_224(..)-    , Skein256_256(..)-    , Skein512_224(..)-    , Skein512_256(..)-    , Skein512_384(..)-    , Skein512_512(..)-    , Whirlpool(..)-    ) where+    Blake2s_160 (..),+    Blake2s_224 (..),+    Blake2s_256 (..),+    Blake2sp_224 (..),+    Blake2sp_256 (..),+    Blake2b_160 (..),+    Blake2b_224 (..),+    Blake2b_256 (..),+    Blake2b_384 (..),+    Blake2b_512 (..),+    Blake2bp_512 (..),+    MD2 (..),+    MD4 (..),+    MD5 (..),+    SHA1 (..),+    SHA224 (..),+    SHA256 (..),+    SHA384 (..),+    SHA512 (..),+    SHA512t_224 (..),+    SHA512t_256 (..),+    RIPEMD160 (..),+    Tiger (..),+    Keccak_224 (..),+    Keccak_256 (..),+    Keccak_384 (..),+    Keccak_512 (..),+    SHA3_224 (..),+    SHA3_256 (..),+    SHA3_384 (..),+    SHA3_512 (..),+    SHAKE128 (..),+    SHAKE256 (..),+    Blake2b (..),+    Blake2bp (..),+    Blake2s (..),+    Blake2sp (..),+    Skein256 (..),+    Skein256_224 (..),+    Skein256_256 (..),+    Skein512 (..),+    Skein512_224 (..),+    Skein512_256 (..),+    Skein512_384 (..),+    Skein512_512 (..),+    Whirlpool (..),+) where -import           Crypto.Hash.Types (HashAlgorithm, HashAlgorithmPrefix)-import           Crypto.Hash.Blake2s-import           Crypto.Hash.Blake2sp-import           Crypto.Hash.Blake2b-import           Crypto.Hash.Blake2bp-import           Crypto.Hash.MD2-import           Crypto.Hash.MD4-import           Crypto.Hash.MD5-import           Crypto.Hash.SHA1-import           Crypto.Hash.SHA224-import           Crypto.Hash.SHA256-import           Crypto.Hash.SHA384-import           Crypto.Hash.SHA512-import           Crypto.Hash.SHA512t-import           Crypto.Hash.SHA3-import           Crypto.Hash.Keccak-import           Crypto.Hash.RIPEMD160-import           Crypto.Hash.Tiger-import           Crypto.Hash.Skein256-import           Crypto.Hash.Skein512-import           Crypto.Hash.Whirlpool-import           Crypto.Hash.SHAKE-import           Crypto.Hash.Blake2+import Crypto.Hash.Blake2+import Crypto.Hash.Blake2b+import Crypto.Hash.Blake2bp+import Crypto.Hash.Blake2s+import Crypto.Hash.Blake2sp+import Crypto.Hash.Keccak+import Crypto.Hash.MD2+import Crypto.Hash.MD4+import Crypto.Hash.MD5+import Crypto.Hash.RIPEMD160+import Crypto.Hash.SHA1+import Crypto.Hash.SHA224+import Crypto.Hash.SHA256+import Crypto.Hash.SHA3+import Crypto.Hash.SHA384+import Crypto.Hash.SHA512+import Crypto.Hash.SHA512t+import Crypto.Hash.SHAKE+import Crypto.Hash.Skein256+import Crypto.Hash.Skein512+import Crypto.Hash.Tiger+import Crypto.Hash.Types (HashAlgorithm, HashAlgorithmPrefix)+import Crypto.Hash.Whirlpool
Crypto/Hash/Blake2.hs view
@@ -1,3 +1,10 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE KindSignatures #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.Blake2 -- License     : BSD-style@@ -25,31 +32,23 @@ --      id-blake2s224 | 32-bit |   2**112  |         28  | --      id-blake2s256 | 32-bit |   2**128  |         32  | --     ---------------+--------+-----------+-------------+----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE ScopedTypeVariables #-}-{-# LANGUAGE KindSignatures #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.Blake2-    ( HashBlake2(..)-    , Blake2s(..)-    , Blake2sp(..)-    , Blake2b(..)-    , Blake2bp(..)-    ) where+module Crypto.Hash.Blake2 (+    HashBlake2 (..),+    Blake2s (..),+    Blake2sp (..),+    Blake2b (..),+    Blake2bp (..),+) where -import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)-import           GHC.TypeLits (Nat, KnownNat)-import           Crypto.Internal.Nat+import Crypto.Hash.Types+import Crypto.Internal.Nat+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)+import GHC.TypeLits (KnownNat, Nat)  -- | Typeclass for the Blake2 family of digest functions. class HashAlgorithm a => HashBlake2 a where-     -- | Init Blake2 algorithm with the specified key of the specified length.     -- The key length is specified in bytes.     blake2InternalKeyedInit :: Ptr (Context a) -> Ptr Word8 -> Word32 -> IO ()@@ -63,28 +62,38 @@ -- * Blake2s 160 -- * Blake2s 224 -- * Blake2s 256--- data Blake2s (bitlen :: Nat) = Blake2s-    deriving (Show,Data)+    deriving (Show, Data) -instance (IsDivisibleBy8 bitlen, KnownNat bitlen, IsAtLeast bitlen 8, IsAtMost bitlen 256)-      => HashAlgorithm (Blake2s bitlen)-      where-    type HashBlockSize           (Blake2s bitlen) = 64-    type HashDigestSize          (Blake2s bitlen) = Div8 bitlen+instance+    ( IsDivisibleBy8 bitlen+    , KnownNat bitlen+    , IsAtLeast bitlen 8+    , IsAtMost bitlen 256+    )+    => HashAlgorithm (Blake2s bitlen)+    where+    type HashBlockSize (Blake2s bitlen) = 64+    type HashDigestSize (Blake2s bitlen) = Div8 bitlen     type HashInternalContextSize (Blake2s bitlen) = 136-    hashBlockSize  _          = 64-    hashDigestSize _          = byteLen (Proxy :: Proxy bitlen)+    hashBlockSize _ = 64+    hashDigestSize _ = byteLen (Proxy :: Proxy bitlen)     hashInternalContextSize _ = 136-    hashInternalInit p        = c_blake2s_init p (integralNatVal (Proxy :: Proxy bitlen))-    hashInternalUpdate        = c_blake2s_update-    hashInternalFinalize p    = c_blake2s_finalize p (integralNatVal (Proxy :: Proxy bitlen))+    hashInternalInit p = c_blake2s_init p (integralNatVal (Proxy :: Proxy bitlen))+    hashInternalUpdate = c_blake2s_update+    hashInternalFinalize p = c_blake2s_finalize p (integralNatVal (Proxy :: Proxy bitlen)) -instance (IsDivisibleBy8 bitlen, KnownNat bitlen, IsAtLeast bitlen 8, IsAtMost bitlen 256)-      => HashBlake2 (Blake2s bitlen)-      where+instance+    ( IsDivisibleBy8 bitlen+    , KnownNat bitlen+    , IsAtLeast bitlen 8+    , IsAtMost bitlen 256+    )+    => HashBlake2 (Blake2s bitlen)+    where     blake2InternalKeyedInit p = c_blake2s_init_key p outLen-        where outLen = integralNatVal (Proxy :: Proxy bitlen)+      where+        outLen = integralNatVal (Proxy :: Proxy bitlen)  foreign import ccall unsafe "crypton_blake2s_init"     c_blake2s_init :: Ptr (Context a) -> Word32 -> IO ()@@ -106,28 +115,38 @@ -- * Blake2b 256 -- * Blake2b 384 -- * Blake2b 512--- data Blake2b (bitlen :: Nat) = Blake2b-    deriving (Show,Data)+    deriving (Show, Data) -instance (IsDivisibleBy8 bitlen, KnownNat bitlen, IsAtLeast bitlen 8, IsAtMost bitlen 512)-      => HashAlgorithm (Blake2b bitlen)-      where-    type HashBlockSize           (Blake2b bitlen) = 128-    type HashDigestSize          (Blake2b bitlen) = Div8 bitlen+instance+    ( IsDivisibleBy8 bitlen+    , KnownNat bitlen+    , IsAtLeast bitlen 8+    , IsAtMost bitlen 512+    )+    => HashAlgorithm (Blake2b bitlen)+    where+    type HashBlockSize (Blake2b bitlen) = 128+    type HashDigestSize (Blake2b bitlen) = Div8 bitlen     type HashInternalContextSize (Blake2b bitlen) = 248-    hashBlockSize  _          = 128-    hashDigestSize _          = byteLen (Proxy :: Proxy bitlen)+    hashBlockSize _ = 128+    hashDigestSize _ = byteLen (Proxy :: Proxy bitlen)     hashInternalContextSize _ = 248-    hashInternalInit p        = c_blake2b_init p (integralNatVal (Proxy :: Proxy bitlen))-    hashInternalUpdate        = c_blake2b_update-    hashInternalFinalize p    = c_blake2b_finalize p (integralNatVal (Proxy :: Proxy bitlen))+    hashInternalInit p = c_blake2b_init p (integralNatVal (Proxy :: Proxy bitlen))+    hashInternalUpdate = c_blake2b_update+    hashInternalFinalize p = c_blake2b_finalize p (integralNatVal (Proxy :: Proxy bitlen)) -instance (IsDivisibleBy8 bitlen, KnownNat bitlen, IsAtLeast bitlen 8, IsAtMost bitlen 512)-      => HashBlake2 (Blake2b bitlen)-      where+instance+    ( IsDivisibleBy8 bitlen+    , KnownNat bitlen+    , IsAtLeast bitlen 8+    , IsAtMost bitlen 512+    )+    => HashBlake2 (Blake2b bitlen)+    where     blake2InternalKeyedInit p = c_blake2b_init_key p outLen-        where outLen = integralNatVal (Proxy :: Proxy bitlen)+      where+        outLen = integralNatVal (Proxy :: Proxy bitlen)  foreign import ccall unsafe "crypton_blake2b_init"     c_blake2b_init :: Ptr (Context a) -> Word32 -> IO ()@@ -139,26 +158,37 @@     c_blake2b_finalize :: Ptr (Context a) -> Word32 -> Ptr (Digest a) -> IO ()  data Blake2sp (bitlen :: Nat) = Blake2sp-    deriving (Show,Data)+    deriving (Show, Data) -instance (IsDivisibleBy8 bitlen, KnownNat bitlen, IsAtLeast bitlen 8, IsAtMost bitlen 256)-      => HashAlgorithm (Blake2sp bitlen)-      where-    type HashBlockSize           (Blake2sp bitlen) = 64-    type HashDigestSize          (Blake2sp bitlen) = Div8 bitlen+instance+    ( IsDivisibleBy8 bitlen+    , KnownNat bitlen+    , IsAtLeast bitlen 8+    , IsAtMost bitlen 256+    )+    => HashAlgorithm (Blake2sp bitlen)+    where+    type HashBlockSize (Blake2sp bitlen) = 64+    type HashDigestSize (Blake2sp bitlen) = Div8 bitlen     type HashInternalContextSize (Blake2sp bitlen) = 2185-    hashBlockSize  _          = 64-    hashDigestSize _          = byteLen (Proxy :: Proxy bitlen)+    hashBlockSize _ = 64+    hashDigestSize _ = byteLen (Proxy :: Proxy bitlen)     hashInternalContextSize _ = 2185-    hashInternalInit p        = c_blake2sp_init p (integralNatVal (Proxy :: Proxy bitlen))-    hashInternalUpdate        = c_blake2sp_update-    hashInternalFinalize p    = c_blake2sp_finalize p (integralNatVal (Proxy :: Proxy bitlen))+    hashInternalInit p = c_blake2sp_init p (integralNatVal (Proxy :: Proxy bitlen))+    hashInternalUpdate = c_blake2sp_update+    hashInternalFinalize p = c_blake2sp_finalize p (integralNatVal (Proxy :: Proxy bitlen)) -instance (IsDivisibleBy8 bitlen, KnownNat bitlen, IsAtLeast bitlen 8, IsAtMost bitlen 256)-      => HashBlake2 (Blake2sp bitlen)-      where+instance+    ( IsDivisibleBy8 bitlen+    , KnownNat bitlen+    , IsAtLeast bitlen 8+    , IsAtMost bitlen 256+    )+    => HashBlake2 (Blake2sp bitlen)+    where     blake2InternalKeyedInit p = c_blake2sp_init_key p outLen-        where outLen = integralNatVal (Proxy :: Proxy bitlen)+      where+        outLen = integralNatVal (Proxy :: Proxy bitlen)  foreign import ccall unsafe "crypton_blake2sp_init"     c_blake2sp_init :: Ptr (Context a) -> Word32 -> IO ()@@ -170,26 +200,37 @@     c_blake2sp_finalize :: Ptr (Context a) -> Word32 -> Ptr (Digest a) -> IO ()  data Blake2bp (bitlen :: Nat) = Blake2bp-    deriving (Show,Data)+    deriving (Show, Data) -instance (IsDivisibleBy8 bitlen, KnownNat bitlen, IsAtLeast bitlen 8, IsAtMost bitlen 512)-      => HashAlgorithm (Blake2bp bitlen)-      where-    type HashBlockSize           (Blake2bp bitlen) = 128-    type HashDigestSize          (Blake2bp bitlen) = Div8 bitlen+instance+    ( IsDivisibleBy8 bitlen+    , KnownNat bitlen+    , IsAtLeast bitlen 8+    , IsAtMost bitlen 512+    )+    => HashAlgorithm (Blake2bp bitlen)+    where+    type HashBlockSize (Blake2bp bitlen) = 128+    type HashDigestSize (Blake2bp bitlen) = Div8 bitlen     type HashInternalContextSize (Blake2bp bitlen) = 2325-    hashBlockSize  _          = 128-    hashDigestSize _          = byteLen (Proxy :: Proxy bitlen)+    hashBlockSize _ = 128+    hashDigestSize _ = byteLen (Proxy :: Proxy bitlen)     hashInternalContextSize _ = 2325-    hashInternalInit p        = c_blake2bp_init p (integralNatVal (Proxy :: Proxy bitlen))-    hashInternalUpdate        = c_blake2bp_update-    hashInternalFinalize p    = c_blake2bp_finalize p (integralNatVal (Proxy :: Proxy bitlen))+    hashInternalInit p = c_blake2bp_init p (integralNatVal (Proxy :: Proxy bitlen))+    hashInternalUpdate = c_blake2bp_update+    hashInternalFinalize p = c_blake2bp_finalize p (integralNatVal (Proxy :: Proxy bitlen)) -instance (IsDivisibleBy8 bitlen, KnownNat bitlen, IsAtLeast bitlen 8, IsAtMost bitlen 512)-      => HashBlake2 (Blake2bp bitlen)-      where+instance+    ( IsDivisibleBy8 bitlen+    , KnownNat bitlen+    , IsAtLeast bitlen 8+    , IsAtMost bitlen 512+    )+    => HashBlake2 (Blake2bp bitlen)+    where     blake2InternalKeyedInit p = c_blake2bp_init_key p outLen-        where outLen = integralNatVal (Proxy :: Proxy bitlen)+      where+        outLen = integralNatVal (Proxy :: Proxy bitlen)  foreign import ccall unsafe "crypton_blake2bp_init"     c_blake2bp_init :: Ptr (Context a) -> Word32 -> IO ()
Crypto/Hash/Blake2b.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.Blake2b -- License     : BSD-style@@ -7,96 +12,93 @@ -- -- Module containing the binding functions to work with the -- Blake2b cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.Blake2b-    (  Blake2b_160 (..), Blake2b_224 (..), Blake2b_256 (..), Blake2b_384 (..), Blake2b_512 (..)-    ) where--import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+module Crypto.Hash.Blake2b (+    Blake2b_160 (..),+    Blake2b_224 (..),+    Blake2b_256 (..),+    Blake2b_384 (..),+    Blake2b_512 (..),+) where +import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | Blake2b (160 bits) cryptographic hash algorithm data Blake2b_160 = Blake2b_160-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Blake2b_160 where-    type HashBlockSize           Blake2b_160 = 128-    type HashDigestSize          Blake2b_160 = 20+    type HashBlockSize Blake2b_160 = 128+    type HashDigestSize Blake2b_160 = 20     type HashInternalContextSize Blake2b_160 = 248-    hashBlockSize  _          = 128-    hashDigestSize _          = 20+    hashBlockSize _ = 128+    hashDigestSize _ = 20     hashInternalContextSize _ = 248-    hashInternalInit p        = c_blake2b_init p 160-    hashInternalUpdate        = c_blake2b_update-    hashInternalFinalize p    = c_blake2b_finalize p 160+    hashInternalInit p = c_blake2b_init p 160+    hashInternalUpdate = c_blake2b_update+    hashInternalFinalize p = c_blake2b_finalize p 160  -- | Blake2b (224 bits) cryptographic hash algorithm data Blake2b_224 = Blake2b_224-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Blake2b_224 where-    type HashBlockSize           Blake2b_224 = 128-    type HashDigestSize          Blake2b_224 = 28+    type HashBlockSize Blake2b_224 = 128+    type HashDigestSize Blake2b_224 = 28     type HashInternalContextSize Blake2b_224 = 248-    hashBlockSize  _          = 128-    hashDigestSize _          = 28+    hashBlockSize _ = 128+    hashDigestSize _ = 28     hashInternalContextSize _ = 248-    hashInternalInit p        = c_blake2b_init p 224-    hashInternalUpdate        = c_blake2b_update-    hashInternalFinalize p    = c_blake2b_finalize p 224+    hashInternalInit p = c_blake2b_init p 224+    hashInternalUpdate = c_blake2b_update+    hashInternalFinalize p = c_blake2b_finalize p 224  -- | Blake2b (256 bits) cryptographic hash algorithm data Blake2b_256 = Blake2b_256-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Blake2b_256 where-    type HashBlockSize           Blake2b_256 = 128-    type HashDigestSize          Blake2b_256 = 32+    type HashBlockSize Blake2b_256 = 128+    type HashDigestSize Blake2b_256 = 32     type HashInternalContextSize Blake2b_256 = 248-    hashBlockSize  _          = 128-    hashDigestSize _          = 32+    hashBlockSize _ = 128+    hashDigestSize _ = 32     hashInternalContextSize _ = 248-    hashInternalInit p        = c_blake2b_init p 256-    hashInternalUpdate        = c_blake2b_update-    hashInternalFinalize p    = c_blake2b_finalize p 256+    hashInternalInit p = c_blake2b_init p 256+    hashInternalUpdate = c_blake2b_update+    hashInternalFinalize p = c_blake2b_finalize p 256  -- | Blake2b (384 bits) cryptographic hash algorithm data Blake2b_384 = Blake2b_384-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Blake2b_384 where-    type HashBlockSize           Blake2b_384 = 128-    type HashDigestSize          Blake2b_384 = 48+    type HashBlockSize Blake2b_384 = 128+    type HashDigestSize Blake2b_384 = 48     type HashInternalContextSize Blake2b_384 = 248-    hashBlockSize  _          = 128-    hashDigestSize _          = 48+    hashBlockSize _ = 128+    hashDigestSize _ = 48     hashInternalContextSize _ = 248-    hashInternalInit p        = c_blake2b_init p 384-    hashInternalUpdate        = c_blake2b_update-    hashInternalFinalize p    = c_blake2b_finalize p 384+    hashInternalInit p = c_blake2b_init p 384+    hashInternalUpdate = c_blake2b_update+    hashInternalFinalize p = c_blake2b_finalize p 384  -- | Blake2b (512 bits) cryptographic hash algorithm data Blake2b_512 = Blake2b_512-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Blake2b_512 where-    type HashBlockSize           Blake2b_512 = 128-    type HashDigestSize          Blake2b_512 = 64+    type HashBlockSize Blake2b_512 = 128+    type HashDigestSize Blake2b_512 = 64     type HashInternalContextSize Blake2b_512 = 248-    hashBlockSize  _          = 128-    hashDigestSize _          = 64+    hashBlockSize _ = 128+    hashDigestSize _ = 64     hashInternalContextSize _ = 248-    hashInternalInit p        = c_blake2b_init p 512-    hashInternalUpdate        = c_blake2b_update-    hashInternalFinalize p    = c_blake2b_finalize p 512-+    hashInternalInit p = c_blake2b_init p 512+    hashInternalUpdate = c_blake2b_update+    hashInternalFinalize p = c_blake2b_finalize p 512  foreign import ccall unsafe "crypton_blake2b_init"     c_blake2b_init :: Ptr (Context a) -> Word32 -> IO ()
Crypto/Hash/Blake2bp.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.Blake2bp -- License     : BSD-style@@ -7,36 +12,29 @@ -- -- Module containing the binding functions to work with the -- Blake2bp cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.Blake2bp-    (  Blake2bp_512 (..)-    ) where--import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+module Crypto.Hash.Blake2bp (+    Blake2bp_512 (..),+) where +import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | Blake2bp (512 bits) cryptographic hash algorithm data Blake2bp_512 = Blake2bp_512-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Blake2bp_512 where-    type HashBlockSize           Blake2bp_512 = 128-    type HashDigestSize          Blake2bp_512 = 64+    type HashBlockSize Blake2bp_512 = 128+    type HashDigestSize Blake2bp_512 = 64     type HashInternalContextSize Blake2bp_512 = 1768-    hashBlockSize  _          = 128-    hashDigestSize _          = 64+    hashBlockSize _ = 128+    hashDigestSize _ = 64     hashInternalContextSize _ = 1768-    hashInternalInit p        = c_blake2bp_init p 512-    hashInternalUpdate        = c_blake2bp_update-    hashInternalFinalize p    = c_blake2bp_finalize p 512-+    hashInternalInit p = c_blake2bp_init p 512+    hashInternalUpdate = c_blake2bp_update+    hashInternalFinalize p = c_blake2bp_finalize p 512  foreign import ccall unsafe "crypton_blake2bp_init"     c_blake2bp_init :: Ptr (Context a) -> Word32 -> IO ()
Crypto/Hash/Blake2s.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.Blake2s -- License     : BSD-style@@ -7,66 +12,61 @@ -- -- Module containing the binding functions to work with the -- Blake2s cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.Blake2s-    (  Blake2s_160 (..), Blake2s_224 (..), Blake2s_256 (..)-    ) where--import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+module Crypto.Hash.Blake2s (+    Blake2s_160 (..),+    Blake2s_224 (..),+    Blake2s_256 (..),+) where +import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | Blake2s (160 bits) cryptographic hash algorithm data Blake2s_160 = Blake2s_160-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Blake2s_160 where-    type HashBlockSize           Blake2s_160 = 64-    type HashDigestSize          Blake2s_160 = 20+    type HashBlockSize Blake2s_160 = 64+    type HashDigestSize Blake2s_160 = 20     type HashInternalContextSize Blake2s_160 = 136-    hashBlockSize  _          = 64-    hashDigestSize _          = 20+    hashBlockSize _ = 64+    hashDigestSize _ = 20     hashInternalContextSize _ = 136-    hashInternalInit p        = c_blake2s_init p 160-    hashInternalUpdate        = c_blake2s_update-    hashInternalFinalize p    = c_blake2s_finalize p 160+    hashInternalInit p = c_blake2s_init p 160+    hashInternalUpdate = c_blake2s_update+    hashInternalFinalize p = c_blake2s_finalize p 160  -- | Blake2s (224 bits) cryptographic hash algorithm data Blake2s_224 = Blake2s_224-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Blake2s_224 where-    type HashBlockSize           Blake2s_224 = 64-    type HashDigestSize          Blake2s_224 = 28+    type HashBlockSize Blake2s_224 = 64+    type HashDigestSize Blake2s_224 = 28     type HashInternalContextSize Blake2s_224 = 136-    hashBlockSize  _          = 64-    hashDigestSize _          = 28+    hashBlockSize _ = 64+    hashDigestSize _ = 28     hashInternalContextSize _ = 136-    hashInternalInit p        = c_blake2s_init p 224-    hashInternalUpdate        = c_blake2s_update-    hashInternalFinalize p    = c_blake2s_finalize p 224+    hashInternalInit p = c_blake2s_init p 224+    hashInternalUpdate = c_blake2s_update+    hashInternalFinalize p = c_blake2s_finalize p 224  -- | Blake2s (256 bits) cryptographic hash algorithm data Blake2s_256 = Blake2s_256-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Blake2s_256 where-    type HashBlockSize           Blake2s_256 = 64-    type HashDigestSize          Blake2s_256 = 32+    type HashBlockSize Blake2s_256 = 64+    type HashDigestSize Blake2s_256 = 32     type HashInternalContextSize Blake2s_256 = 136-    hashBlockSize  _          = 64-    hashDigestSize _          = 32+    hashBlockSize _ = 64+    hashDigestSize _ = 32     hashInternalContextSize _ = 136-    hashInternalInit p        = c_blake2s_init p 256-    hashInternalUpdate        = c_blake2s_update-    hashInternalFinalize p    = c_blake2s_finalize p 256-+    hashInternalInit p = c_blake2s_init p 256+    hashInternalUpdate = c_blake2s_update+    hashInternalFinalize p = c_blake2s_finalize p 256  foreign import ccall unsafe "crypton_blake2s_init"     c_blake2s_init :: Ptr (Context a) -> Word32 -> IO ()
Crypto/Hash/Blake2sp.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.Blake2sp -- License     : BSD-style@@ -7,51 +12,45 @@ -- -- Module containing the binding functions to work with the -- Blake2sp cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.Blake2sp-    (  Blake2sp_224 (..), Blake2sp_256 (..)-    ) where--import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+module Crypto.Hash.Blake2sp (+    Blake2sp_224 (..),+    Blake2sp_256 (..),+) where +import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | Blake2sp (224 bits) cryptographic hash algorithm data Blake2sp_224 = Blake2sp_224-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Blake2sp_224 where-    type HashBlockSize           Blake2sp_224 = 64-    type HashDigestSize          Blake2sp_224 = 28+    type HashBlockSize Blake2sp_224 = 64+    type HashDigestSize Blake2sp_224 = 28     type HashInternalContextSize Blake2sp_224 = 1752-    hashBlockSize  _          = 64-    hashDigestSize _          = 28+    hashBlockSize _ = 64+    hashDigestSize _ = 28     hashInternalContextSize _ = 1752-    hashInternalInit p        = c_blake2sp_init p 224-    hashInternalUpdate        = c_blake2sp_update-    hashInternalFinalize p    = c_blake2sp_finalize p 224+    hashInternalInit p = c_blake2sp_init p 224+    hashInternalUpdate = c_blake2sp_update+    hashInternalFinalize p = c_blake2sp_finalize p 224  -- | Blake2sp (256 bits) cryptographic hash algorithm data Blake2sp_256 = Blake2sp_256-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Blake2sp_256 where-    type HashBlockSize           Blake2sp_256 = 64-    type HashDigestSize          Blake2sp_256 = 32+    type HashBlockSize Blake2sp_256 = 64+    type HashDigestSize Blake2sp_256 = 32     type HashInternalContextSize Blake2sp_256 = 1752-    hashBlockSize  _          = 64-    hashDigestSize _          = 32+    hashBlockSize _ = 64+    hashDigestSize _ = 32     hashInternalContextSize _ = 1752-    hashInternalInit p        = c_blake2sp_init p 256-    hashInternalUpdate        = c_blake2sp_update-    hashInternalFinalize p    = c_blake2sp_finalize p 256-+    hashInternalInit p = c_blake2sp_init p 256+    hashInternalUpdate = c_blake2sp_update+    hashInternalFinalize p = c_blake2sp_finalize p 256  foreign import ccall unsafe "crypton_blake2sp_init"     c_blake2sp_init :: Ptr (Context a) -> Word32 -> IO ()
Crypto/Hash/IO.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# LANGUAGE ScopedTypeVariables #-}+ -- | -- Module      : Crypto.Hash.IO -- License     : BSD-style@@ -6,22 +9,20 @@ -- Portability : unknown -- -- Generalized impure cryptographic hash interface----{-# LANGUAGE GeneralizedNewtypeDeriving #-}-{-# LANGUAGE ScopedTypeVariables        #-}-module Crypto.Hash.IO-    ( HashAlgorithm(..)-    , MutableContext-    , hashMutableInit-    , hashMutableInitWith-    , hashMutableUpdate-    , hashMutableFinalize-    , hashMutableReset-    ) where+module Crypto.Hash.IO (+    HashAlgorithm (..),+    MutableContext,+    hashMutableInit,+    hashMutableInitWith,+    hashMutableUpdate,+    hashMutableFinalize,+    hashMutableReset,+) where -import           Crypto.Hash.Types+import Crypto.Hash.Types+import Crypto.Internal.ByteArray (allocAndFreezePrimIO) import qualified Crypto.Internal.ByteArray as B-import           Foreign.Ptr+import Foreign.Ptr  -- | A Mutable hash context --@@ -38,8 +39,10 @@ hashMutableInit :: HashAlgorithm alg => IO (MutableContext alg) hashMutableInit = doInit undefined B.alloc   where-        doInit :: HashAlgorithm a => a -> (Int -> (Ptr (Context a) -> IO ()) -> IO B.Bytes) -> IO (MutableContext a)-        doInit alg alloc = MutableContext `fmap` alloc (hashInternalContextSize alg) hashInternalInit+    doInit+        :: HashAlgorithm a+        => a -> (Int -> (Ptr (Context a) -> IO ()) -> IO B.Bytes) -> IO (MutableContext a)+    doInit alg alloc = MutableContext `fmap` alloc (hashInternalContextSize alg) hashInternalInit  -- | Create a new mutable hash context. --@@ -48,23 +51,32 @@ hashMutableInitWith _ = hashMutableInit  -- | Update a mutable hash context in place-hashMutableUpdate :: (B.ByteArrayAccess ba, HashAlgorithm a) => MutableContext a -> ba -> IO ()+hashMutableUpdate+    :: (B.ByteArrayAccess ba, HashAlgorithm a) => MutableContext a -> ba -> IO () hashMutableUpdate mc dat = doUpdate mc (B.withByteArray mc)-  where doUpdate :: HashAlgorithm a => MutableContext a -> ((Ptr (Context a) -> IO ()) -> IO ()) -> IO ()-        doUpdate _ withCtx =-            withCtx             $ \ctx ->-            B.withByteArray dat $ \d   ->+  where+    doUpdate+        :: HashAlgorithm a+        => MutableContext a -> ((Ptr (Context a) -> IO ()) -> IO ()) -> IO ()+    doUpdate _ withCtx =+        withCtx $ \ctx ->+            B.withByteArray dat $ \d ->                 hashInternalUpdate ctx d (fromIntegral $ B.length dat)  -- | Finalize a mutable hash context and compute a digest-hashMutableFinalize :: forall a . HashAlgorithm a => MutableContext a -> IO (Digest a)+hashMutableFinalize+    :: forall a. HashAlgorithm a => MutableContext a -> IO (Digest a) hashMutableFinalize mc = do-    b <- B.alloc (hashDigestSize (undefined :: a)) $ \dig -> B.withByteArray mc $ \(ctx :: Ptr (Context a)) -> hashInternalFinalize ctx dig-    return $ Digest b+    ba <- allocAndFreezePrimIO (hashDigestSize (undefined :: a)) $+        \(dig :: Ptr (Digest a)) ->+            B.withByteArray mc $ \(ctx :: Ptr (Context a)) -> hashInternalFinalize ctx dig+    return (Digest ba)  -- | Reset the mutable context to the initial state of the hash hashMutableReset :: HashAlgorithm a => MutableContext a -> IO () hashMutableReset mc = doReset mc (B.withByteArray mc)   where-    doReset :: HashAlgorithm a => MutableContext a -> ((Ptr (Context a) -> IO ()) -> IO ()) -> IO ()+    doReset+        :: HashAlgorithm a+        => MutableContext a -> ((Ptr (Context a) -> IO ()) -> IO ()) -> IO ()     doReset _ withCtx = withCtx hashInternalInit
Crypto/Hash/Keccak.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.Keccak -- License     : BSD-style@@ -7,81 +12,77 @@ -- -- Module containing the binding functions to work with the -- Keccak cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.Keccak-    (  Keccak_224 (..), Keccak_256 (..), Keccak_384 (..), Keccak_512 (..)-    ) where--import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+module Crypto.Hash.Keccak (+    Keccak_224 (..),+    Keccak_256 (..),+    Keccak_384 (..),+    Keccak_512 (..),+) where +import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | Keccak (224 bits) cryptographic hash algorithm data Keccak_224 = Keccak_224-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Keccak_224 where-    type HashBlockSize           Keccak_224 = 144-    type HashDigestSize          Keccak_224 = 28+    type HashBlockSize Keccak_224 = 144+    type HashDigestSize Keccak_224 = 28     type HashInternalContextSize Keccak_224 = 352-    hashBlockSize  _          = 144-    hashDigestSize _          = 28+    hashBlockSize _ = 144+    hashDigestSize _ = 28     hashInternalContextSize _ = 352-    hashInternalInit p        = c_keccak_init p 224-    hashInternalUpdate        = c_keccak_update-    hashInternalFinalize p    = c_keccak_finalize p 224+    hashInternalInit p = c_keccak_init p 224+    hashInternalUpdate = c_keccak_update+    hashInternalFinalize p = c_keccak_finalize p 224  -- | Keccak (256 bits) cryptographic hash algorithm data Keccak_256 = Keccak_256-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Keccak_256 where-    type HashBlockSize           Keccak_256 = 136-    type HashDigestSize          Keccak_256 = 32+    type HashBlockSize Keccak_256 = 136+    type HashDigestSize Keccak_256 = 32     type HashInternalContextSize Keccak_256 = 344-    hashBlockSize  _          = 136-    hashDigestSize _          = 32+    hashBlockSize _ = 136+    hashDigestSize _ = 32     hashInternalContextSize _ = 344-    hashInternalInit p        = c_keccak_init p 256-    hashInternalUpdate        = c_keccak_update-    hashInternalFinalize p    = c_keccak_finalize p 256+    hashInternalInit p = c_keccak_init p 256+    hashInternalUpdate = c_keccak_update+    hashInternalFinalize p = c_keccak_finalize p 256  -- | Keccak (384 bits) cryptographic hash algorithm data Keccak_384 = Keccak_384-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Keccak_384 where-    type HashBlockSize           Keccak_384 = 104-    type HashDigestSize          Keccak_384 = 48+    type HashBlockSize Keccak_384 = 104+    type HashDigestSize Keccak_384 = 48     type HashInternalContextSize Keccak_384 = 312-    hashBlockSize  _          = 104-    hashDigestSize _          = 48+    hashBlockSize _ = 104+    hashDigestSize _ = 48     hashInternalContextSize _ = 312-    hashInternalInit p        = c_keccak_init p 384-    hashInternalUpdate        = c_keccak_update-    hashInternalFinalize p    = c_keccak_finalize p 384+    hashInternalInit p = c_keccak_init p 384+    hashInternalUpdate = c_keccak_update+    hashInternalFinalize p = c_keccak_finalize p 384  -- | Keccak (512 bits) cryptographic hash algorithm data Keccak_512 = Keccak_512-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Keccak_512 where-    type HashBlockSize           Keccak_512 = 72-    type HashDigestSize          Keccak_512 = 64+    type HashBlockSize Keccak_512 = 72+    type HashDigestSize Keccak_512 = 64     type HashInternalContextSize Keccak_512 = 280-    hashBlockSize  _          = 72-    hashDigestSize _          = 64+    hashBlockSize _ = 72+    hashDigestSize _ = 64     hashInternalContextSize _ = 280-    hashInternalInit p        = c_keccak_init p 512-    hashInternalUpdate        = c_keccak_update-    hashInternalFinalize p    = c_keccak_finalize p 512-+    hashInternalInit p = c_keccak_init p 512+    hashInternalUpdate = c_keccak_update+    hashInternalFinalize p = c_keccak_finalize p 512  foreign import ccall unsafe "crypton_keccak_init"     c_keccak_init :: Ptr (Context a) -> Word32 -> IO ()
Crypto/Hash/MD2.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.MD2 -- License     : BSD-style@@ -7,35 +12,30 @@ -- -- Module containing the binding functions to work with the -- MD2 cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.MD2 ( MD2 (..) ) where+module Crypto.Hash.MD2 (MD2 (..)) where -import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | MD2 cryptographic hash algorithm data MD2 = MD2-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm MD2 where-    type HashBlockSize           MD2 = 16-    type HashDigestSize          MD2 = 16+    type HashBlockSize MD2 = 16+    type HashDigestSize MD2 = 16     type HashInternalContextSize MD2 = 96-    hashBlockSize  _          = 16-    hashDigestSize _          = 16+    hashBlockSize _ = 16+    hashDigestSize _ = 16     hashInternalContextSize _ = 96-    hashInternalInit          = c_md2_init-    hashInternalUpdate        = c_md2_update-    hashInternalFinalize      = c_md2_finalize+    hashInternalInit = c_md2_init+    hashInternalUpdate = c_md2_update+    hashInternalFinalize = c_md2_finalize  foreign import ccall unsafe "crypton_md2_init"-    c_md2_init :: Ptr (Context a)-> IO ()+    c_md2_init :: Ptr (Context a) -> IO ()  foreign import ccall "crypton_md2_update"     c_md2_update :: Ptr (Context a) -> Ptr Word8 -> Word32 -> IO ()
Crypto/Hash/MD4.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.MD4 -- License     : BSD-style@@ -7,35 +12,30 @@ -- -- Module containing the binding functions to work with the -- MD4 cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.MD4 ( MD4 (..) ) where+module Crypto.Hash.MD4 (MD4 (..)) where -import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | MD4 cryptographic hash algorithm data MD4 = MD4-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm MD4 where-    type HashBlockSize           MD4 = 64-    type HashDigestSize          MD4 = 16+    type HashBlockSize MD4 = 64+    type HashDigestSize MD4 = 16     type HashInternalContextSize MD4 = 96-    hashBlockSize  _          = 64-    hashDigestSize _          = 16+    hashBlockSize _ = 64+    hashDigestSize _ = 16     hashInternalContextSize _ = 96-    hashInternalInit          = c_md4_init-    hashInternalUpdate        = c_md4_update-    hashInternalFinalize      = c_md4_finalize+    hashInternalInit = c_md4_init+    hashInternalUpdate = c_md4_update+    hashInternalFinalize = c_md4_finalize  foreign import ccall unsafe "crypton_md4_init"-    c_md4_init :: Ptr (Context a)-> IO ()+    c_md4_init :: Ptr (Context a) -> IO ()  foreign import ccall "crypton_md4_update"     c_md4_update :: Ptr (Context a) -> Ptr Word8 -> Word32 -> IO ()
Crypto/Hash/MD5.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.MD5 -- License     : BSD-style@@ -7,38 +12,33 @@ -- -- Module containing the binding functions to work with the -- MD5 cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.MD5 ( MD5 (..) ) where+module Crypto.Hash.MD5 (MD5 (..)) where -import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | MD5 cryptographic hash algorithm data MD5 = MD5-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm MD5 where-    type HashBlockSize           MD5 = 64-    type HashDigestSize          MD5 = 16+    type HashBlockSize MD5 = 64+    type HashDigestSize MD5 = 16     type HashInternalContextSize MD5 = 96-    hashBlockSize  _          = 64-    hashDigestSize _          = 16+    hashBlockSize _ = 64+    hashDigestSize _ = 16     hashInternalContextSize _ = 96-    hashInternalInit          = c_md5_init-    hashInternalUpdate        = c_md5_update-    hashInternalFinalize      = c_md5_finalize+    hashInternalInit = c_md5_init+    hashInternalUpdate = c_md5_update+    hashInternalFinalize = c_md5_finalize  instance HashAlgorithmPrefix MD5 where     hashInternalFinalizePrefix = c_md5_finalize_prefix  foreign import ccall unsafe "crypton_md5_init"-    c_md5_init :: Ptr (Context a)-> IO ()+    c_md5_init :: Ptr (Context a) -> IO ()  foreign import ccall "crypton_md5_update"     c_md5_update :: Ptr (Context a) -> Ptr Word8 -> Word32 -> IO ()@@ -47,4 +47,5 @@     c_md5_finalize :: Ptr (Context a) -> Ptr (Digest a) -> IO ()  foreign import ccall "crypton_md5_finalize_prefix"-    c_md5_finalize_prefix :: Ptr (Context a) -> Ptr Word8 -> Word32 -> Word32 -> Ptr (Digest a) -> IO ()+    c_md5_finalize_prefix+        :: Ptr (Context a) -> Ptr Word8 -> Word32 -> Word32 -> Ptr (Digest a) -> IO ()
Crypto/Hash/RIPEMD160.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.RIPEMD160 -- License     : BSD-style@@ -7,35 +12,30 @@ -- -- Module containing the binding functions to work with the -- RIPEMD160 cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.RIPEMD160 ( RIPEMD160 (..) ) where+module Crypto.Hash.RIPEMD160 (RIPEMD160 (..)) where -import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | RIPEMD160 cryptographic hash algorithm data RIPEMD160 = RIPEMD160-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm RIPEMD160 where-    type HashBlockSize           RIPEMD160 = 64-    type HashDigestSize          RIPEMD160 = 20+    type HashBlockSize RIPEMD160 = 64+    type HashDigestSize RIPEMD160 = 20     type HashInternalContextSize RIPEMD160 = 128-    hashBlockSize  _          = 64-    hashDigestSize _          = 20+    hashBlockSize _ = 64+    hashDigestSize _ = 20     hashInternalContextSize _ = 128-    hashInternalInit          = c_ripemd160_init-    hashInternalUpdate        = c_ripemd160_update-    hashInternalFinalize      = c_ripemd160_finalize+    hashInternalInit = c_ripemd160_init+    hashInternalUpdate = c_ripemd160_update+    hashInternalFinalize = c_ripemd160_finalize  foreign import ccall unsafe "crypton_ripemd160_init"-    c_ripemd160_init :: Ptr (Context a)-> IO ()+    c_ripemd160_init :: Ptr (Context a) -> IO ()  foreign import ccall "crypton_ripemd160_update"     c_ripemd160_update :: Ptr (Context a) -> Ptr Word8 -> Word32 -> IO ()
Crypto/Hash/SHA1.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.SHA1 -- License     : BSD-style@@ -7,38 +12,33 @@ -- -- Module containing the binding functions to work with the -- SHA1 cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.SHA1 ( SHA1 (..) ) where+module Crypto.Hash.SHA1 (SHA1 (..)) where -import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | SHA1 cryptographic hash algorithm data SHA1 = SHA1-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm SHA1 where-    type HashBlockSize           SHA1 = 64-    type HashDigestSize          SHA1 = 20+    type HashBlockSize SHA1 = 64+    type HashDigestSize SHA1 = 20     type HashInternalContextSize SHA1 = 96-    hashBlockSize  _          = 64-    hashDigestSize _          = 20+    hashBlockSize _ = 64+    hashDigestSize _ = 20     hashInternalContextSize _ = 96-    hashInternalInit          = c_sha1_init-    hashInternalUpdate        = c_sha1_update-    hashInternalFinalize      = c_sha1_finalize+    hashInternalInit = c_sha1_init+    hashInternalUpdate = c_sha1_update+    hashInternalFinalize = c_sha1_finalize  instance HashAlgorithmPrefix SHA1 where     hashInternalFinalizePrefix = c_sha1_finalize_prefix  foreign import ccall unsafe "crypton_sha1_init"-    c_sha1_init :: Ptr (Context a)-> IO ()+    c_sha1_init :: Ptr (Context a) -> IO ()  foreign import ccall "crypton_sha1_update"     c_sha1_update :: Ptr (Context a) -> Ptr Word8 -> Word32 -> IO ()@@ -47,4 +47,5 @@     c_sha1_finalize :: Ptr (Context a) -> Ptr (Digest a) -> IO ()  foreign import ccall "crypton_sha1_finalize_prefix"-    c_sha1_finalize_prefix :: Ptr (Context a) -> Ptr Word8 -> Word32 -> Word32 -> Ptr (Digest a) -> IO ()+    c_sha1_finalize_prefix+        :: Ptr (Context a) -> Ptr Word8 -> Word32 -> Word32 -> Ptr (Digest a) -> IO ()
Crypto/Hash/SHA224.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.SHA224 -- License     : BSD-style@@ -7,38 +12,33 @@ -- -- Module containing the binding functions to work with the -- SHA224 cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.SHA224 ( SHA224 (..) ) where+module Crypto.Hash.SHA224 (SHA224 (..)) where -import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | SHA224 cryptographic hash algorithm data SHA224 = SHA224-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm SHA224 where-    type HashBlockSize           SHA224 = 64-    type HashDigestSize          SHA224 = 28+    type HashBlockSize SHA224 = 64+    type HashDigestSize SHA224 = 28     type HashInternalContextSize SHA224 = 192-    hashBlockSize  _          = 64-    hashDigestSize _          = 28+    hashBlockSize _ = 64+    hashDigestSize _ = 28     hashInternalContextSize _ = 192-    hashInternalInit          = c_sha224_init-    hashInternalUpdate        = c_sha224_update-    hashInternalFinalize      = c_sha224_finalize+    hashInternalInit = c_sha224_init+    hashInternalUpdate = c_sha224_update+    hashInternalFinalize = c_sha224_finalize  instance HashAlgorithmPrefix SHA224 where     hashInternalFinalizePrefix = c_sha224_finalize_prefix  foreign import ccall unsafe "crypton_sha224_init"-    c_sha224_init :: Ptr (Context a)-> IO ()+    c_sha224_init :: Ptr (Context a) -> IO ()  foreign import ccall "crypton_sha224_update"     c_sha224_update :: Ptr (Context a) -> Ptr Word8 -> Word32 -> IO ()@@ -47,4 +47,5 @@     c_sha224_finalize :: Ptr (Context a) -> Ptr (Digest a) -> IO ()  foreign import ccall "crypton_sha224_finalize_prefix"-    c_sha224_finalize_prefix :: Ptr (Context a) -> Ptr Word8 -> Word32 -> Word32 -> Ptr (Digest a) -> IO ()+    c_sha224_finalize_prefix+        :: Ptr (Context a) -> Ptr Word8 -> Word32 -> Word32 -> Ptr (Digest a) -> IO ()
Crypto/Hash/SHA256.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.SHA256 -- License     : BSD-style@@ -7,38 +12,33 @@ -- -- Module containing the binding functions to work with the -- SHA256 cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.SHA256 ( SHA256 (..) ) where+module Crypto.Hash.SHA256 (SHA256 (..)) where -import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | SHA256 cryptographic hash algorithm data SHA256 = SHA256-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm SHA256 where-    type HashBlockSize           SHA256 = 64-    type HashDigestSize          SHA256 = 32+    type HashBlockSize SHA256 = 64+    type HashDigestSize SHA256 = 32     type HashInternalContextSize SHA256 = 192-    hashBlockSize  _          = 64-    hashDigestSize _          = 32+    hashBlockSize _ = 64+    hashDigestSize _ = 32     hashInternalContextSize _ = 192-    hashInternalInit          = c_sha256_init-    hashInternalUpdate        = c_sha256_update-    hashInternalFinalize      = c_sha256_finalize+    hashInternalInit = c_sha256_init+    hashInternalUpdate = c_sha256_update+    hashInternalFinalize = c_sha256_finalize  instance HashAlgorithmPrefix SHA256 where     hashInternalFinalizePrefix = c_sha256_finalize_prefix  foreign import ccall unsafe "crypton_sha256_init"-    c_sha256_init :: Ptr (Context a)-> IO ()+    c_sha256_init :: Ptr (Context a) -> IO ()  foreign import ccall "crypton_sha256_update"     c_sha256_update :: Ptr (Context a) -> Ptr Word8 -> Word32 -> IO ()@@ -47,4 +47,5 @@     c_sha256_finalize :: Ptr (Context a) -> Ptr (Digest a) -> IO ()  foreign import ccall "crypton_sha256_finalize_prefix"-    c_sha256_finalize_prefix :: Ptr (Context a) -> Ptr Word8 -> Word32 -> Word32 -> Ptr (Digest a) -> IO ()+    c_sha256_finalize_prefix+        :: Ptr (Context a) -> Ptr Word8 -> Word32 -> Word32 -> Ptr (Digest a) -> IO ()
Crypto/Hash/SHA3.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.SHA3 -- License     : BSD-style@@ -7,81 +12,77 @@ -- -- Module containing the binding functions to work with the -- SHA3 cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.SHA3-    (  SHA3_224 (..), SHA3_256 (..), SHA3_384 (..), SHA3_512 (..)-    ) where--import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+module Crypto.Hash.SHA3 (+    SHA3_224 (..),+    SHA3_256 (..),+    SHA3_384 (..),+    SHA3_512 (..),+) where +import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | SHA3 (224 bits) cryptographic hash algorithm data SHA3_224 = SHA3_224-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm SHA3_224 where-    type HashBlockSize           SHA3_224 = 144-    type HashDigestSize          SHA3_224 = 28+    type HashBlockSize SHA3_224 = 144+    type HashDigestSize SHA3_224 = 28     type HashInternalContextSize SHA3_224 = 352-    hashBlockSize  _          = 144-    hashDigestSize _          = 28+    hashBlockSize _ = 144+    hashDigestSize _ = 28     hashInternalContextSize _ = 352-    hashInternalInit p        = c_sha3_init p 224-    hashInternalUpdate        = c_sha3_update-    hashInternalFinalize p    = c_sha3_finalize p 224+    hashInternalInit p = c_sha3_init p 224+    hashInternalUpdate = c_sha3_update+    hashInternalFinalize p = c_sha3_finalize p 224  -- | SHA3 (256 bits) cryptographic hash algorithm data SHA3_256 = SHA3_256-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm SHA3_256 where-    type HashBlockSize           SHA3_256 = 136-    type HashDigestSize          SHA3_256 = 32+    type HashBlockSize SHA3_256 = 136+    type HashDigestSize SHA3_256 = 32     type HashInternalContextSize SHA3_256 = 344-    hashBlockSize  _          = 136-    hashDigestSize _          = 32+    hashBlockSize _ = 136+    hashDigestSize _ = 32     hashInternalContextSize _ = 344-    hashInternalInit p        = c_sha3_init p 256-    hashInternalUpdate        = c_sha3_update-    hashInternalFinalize p    = c_sha3_finalize p 256+    hashInternalInit p = c_sha3_init p 256+    hashInternalUpdate = c_sha3_update+    hashInternalFinalize p = c_sha3_finalize p 256  -- | SHA3 (384 bits) cryptographic hash algorithm data SHA3_384 = SHA3_384-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm SHA3_384 where-    type HashBlockSize           SHA3_384 = 104-    type HashDigestSize          SHA3_384 = 48+    type HashBlockSize SHA3_384 = 104+    type HashDigestSize SHA3_384 = 48     type HashInternalContextSize SHA3_384 = 312-    hashBlockSize  _          = 104-    hashDigestSize _          = 48+    hashBlockSize _ = 104+    hashDigestSize _ = 48     hashInternalContextSize _ = 312-    hashInternalInit p        = c_sha3_init p 384-    hashInternalUpdate        = c_sha3_update-    hashInternalFinalize p    = c_sha3_finalize p 384+    hashInternalInit p = c_sha3_init p 384+    hashInternalUpdate = c_sha3_update+    hashInternalFinalize p = c_sha3_finalize p 384  -- | SHA3 (512 bits) cryptographic hash algorithm data SHA3_512 = SHA3_512-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm SHA3_512 where-    type HashBlockSize           SHA3_512 = 72-    type HashDigestSize          SHA3_512 = 64+    type HashBlockSize SHA3_512 = 72+    type HashDigestSize SHA3_512 = 64     type HashInternalContextSize SHA3_512 = 280-    hashBlockSize  _          = 72-    hashDigestSize _          = 64+    hashBlockSize _ = 72+    hashDigestSize _ = 64     hashInternalContextSize _ = 280-    hashInternalInit p        = c_sha3_init p 512-    hashInternalUpdate        = c_sha3_update-    hashInternalFinalize p    = c_sha3_finalize p 512-+    hashInternalInit p = c_sha3_init p 512+    hashInternalUpdate = c_sha3_update+    hashInternalFinalize p = c_sha3_finalize p 512  foreign import ccall unsafe "crypton_sha3_init"     c_sha3_init :: Ptr (Context a) -> Word32 -> IO ()
Crypto/Hash/SHA384.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.SHA384 -- License     : BSD-style@@ -7,38 +12,33 @@ -- -- Module containing the binding functions to work with the -- SHA384 cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.SHA384 ( SHA384 (..) ) where+module Crypto.Hash.SHA384 (SHA384 (..)) where -import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | SHA384 cryptographic hash algorithm data SHA384 = SHA384-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm SHA384 where-    type HashBlockSize           SHA384 = 128-    type HashDigestSize          SHA384 = 48+    type HashBlockSize SHA384 = 128+    type HashDigestSize SHA384 = 48     type HashInternalContextSize SHA384 = 256-    hashBlockSize  _          = 128-    hashDigestSize _          = 48+    hashBlockSize _ = 128+    hashDigestSize _ = 48     hashInternalContextSize _ = 256-    hashInternalInit          = c_sha384_init-    hashInternalUpdate        = c_sha384_update-    hashInternalFinalize      = c_sha384_finalize+    hashInternalInit = c_sha384_init+    hashInternalUpdate = c_sha384_update+    hashInternalFinalize = c_sha384_finalize  instance HashAlgorithmPrefix SHA384 where     hashInternalFinalizePrefix = c_sha384_finalize_prefix  foreign import ccall unsafe "crypton_sha384_init"-    c_sha384_init :: Ptr (Context a)-> IO ()+    c_sha384_init :: Ptr (Context a) -> IO ()  foreign import ccall "crypton_sha384_update"     c_sha384_update :: Ptr (Context a) -> Ptr Word8 -> Word32 -> IO ()@@ -47,4 +47,5 @@     c_sha384_finalize :: Ptr (Context a) -> Ptr (Digest a) -> IO ()  foreign import ccall "crypton_sha384_finalize_prefix"-    c_sha384_finalize_prefix :: Ptr (Context a) -> Ptr Word8 -> Word32 -> Word32 -> Ptr (Digest a) -> IO ()+    c_sha384_finalize_prefix+        :: Ptr (Context a) -> Ptr Word8 -> Word32 -> Word32 -> Ptr (Digest a) -> IO ()
Crypto/Hash/SHA512.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.SHA512 -- License     : BSD-style@@ -7,38 +12,33 @@ -- -- Module containing the binding functions to work with the -- SHA512 cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.SHA512 ( SHA512 (..) ) where+module Crypto.Hash.SHA512 (SHA512 (..)) where -import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | SHA512 cryptographic hash algorithm data SHA512 = SHA512-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm SHA512 where-    type HashBlockSize           SHA512 = 128-    type HashDigestSize          SHA512 = 64+    type HashBlockSize SHA512 = 128+    type HashDigestSize SHA512 = 64     type HashInternalContextSize SHA512 = 256-    hashBlockSize  _          = 128-    hashDigestSize _          = 64+    hashBlockSize _ = 128+    hashDigestSize _ = 64     hashInternalContextSize _ = 256-    hashInternalInit          = c_sha512_init-    hashInternalUpdate        = c_sha512_update-    hashInternalFinalize      = c_sha512_finalize+    hashInternalInit = c_sha512_init+    hashInternalUpdate = c_sha512_update+    hashInternalFinalize = c_sha512_finalize  instance HashAlgorithmPrefix SHA512 where     hashInternalFinalizePrefix = c_sha512_finalize_prefix  foreign import ccall unsafe "crypton_sha512_init"-    c_sha512_init :: Ptr (Context a)-> IO ()+    c_sha512_init :: Ptr (Context a) -> IO ()  foreign import ccall "crypton_sha512_update"     c_sha512_update :: Ptr (Context a) -> Ptr Word8 -> Word32 -> IO ()@@ -47,4 +47,5 @@     c_sha512_finalize :: Ptr (Context a) -> Ptr (Digest a) -> IO ()  foreign import ccall "crypton_sha512_finalize_prefix"-    c_sha512_finalize_prefix :: Ptr (Context a) -> Ptr Word8 -> Word32 -> Word32 -> Ptr (Digest a) -> IO ()+    c_sha512_finalize_prefix+        :: Ptr (Context a) -> Ptr Word8 -> Word32 -> Word32 -> Ptr (Digest a) -> IO ()
Crypto/Hash/SHA512t.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.SHA512t -- License     : BSD-style@@ -7,51 +12,45 @@ -- -- Module containing the binding functions to work with the -- SHA512t cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.SHA512t-    (  SHA512t_224 (..), SHA512t_256 (..)-    ) where--import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+module Crypto.Hash.SHA512t (+    SHA512t_224 (..),+    SHA512t_256 (..),+) where +import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | SHA512t (224 bits) cryptographic hash algorithm data SHA512t_224 = SHA512t_224-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm SHA512t_224 where-    type HashBlockSize           SHA512t_224 = 128-    type HashDigestSize          SHA512t_224 = 28+    type HashBlockSize SHA512t_224 = 128+    type HashDigestSize SHA512t_224 = 28     type HashInternalContextSize SHA512t_224 = 256-    hashBlockSize  _          = 128-    hashDigestSize _          = 28+    hashBlockSize _ = 128+    hashDigestSize _ = 28     hashInternalContextSize _ = 256-    hashInternalInit p        = c_sha512t_init p 224-    hashInternalUpdate        = c_sha512t_update-    hashInternalFinalize p    = c_sha512t_finalize p 224+    hashInternalInit p = c_sha512t_init p 224+    hashInternalUpdate = c_sha512t_update+    hashInternalFinalize p = c_sha512t_finalize p 224  -- | SHA512t (256 bits) cryptographic hash algorithm data SHA512t_256 = SHA512t_256-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm SHA512t_256 where-    type HashBlockSize           SHA512t_256 = 128-    type HashDigestSize          SHA512t_256 = 32+    type HashBlockSize SHA512t_256 = 128+    type HashDigestSize SHA512t_256 = 32     type HashInternalContextSize SHA512t_256 = 256-    hashBlockSize  _          = 128-    hashDigestSize _          = 32+    hashBlockSize _ = 128+    hashDigestSize _ = 32     hashInternalContextSize _ = 256-    hashInternalInit p        = c_sha512t_init p 256-    hashInternalUpdate        = c_sha512t_update-    hashInternalFinalize p    = c_sha512t_finalize p 256-+    hashInternalInit p = c_sha512t_init p 256+    hashInternalUpdate = c_sha512t_update+    hashInternalFinalize p = c_sha512t_finalize p 256  foreign import ccall unsafe "crypton_sha512t_init"     c_sha512t_init :: Ptr (Context a) -> Word32 -> IO ()
Crypto/Hash/SHAKE.hs view
@@ -1,3 +1,12 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE KindSignatures #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE TypeOperators #-}+{-# LANGUAGE UndecidableInstances #-}+ -- | -- Module      : Crypto.Hash.SHAKE -- License     : BSD-style@@ -7,56 +16,50 @@ -- -- Module containing the binding functions to work with the -- SHA3 extendable output functions (SHAKE).----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE KindSignatures #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE ScopedTypeVariables #-}-{-# LANGUAGE TypeOperators #-}-{-# LANGUAGE TypeFamilies #-}-{-# LANGUAGE UndecidableInstances #-}-module Crypto.Hash.SHAKE-    (  SHAKE128 (..), SHAKE256 (..), HashSHAKE (..)-    ) where+module Crypto.Hash.SHAKE (+    SHAKE128 (..),+    SHAKE256 (..),+    HashSHAKE (..),+) where -import           Control.Monad (when)-import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr, castPtr)-import           Foreign.Storable (Storable(..))-import           Data.Bits-import           Data.Data-import           Data.Word (Word8, Word32)+import Control.Monad (when)+import Crypto.Hash.Types+import Data.Bits+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr, castPtr)+import Foreign.Storable (Storable (..)) -import           GHC.TypeLits (Nat, KnownNat, type (+))-import           Crypto.Internal.Nat+import Crypto.Internal.Nat+import GHC.TypeLits (KnownNat, Nat, type (+))  -- | Type class of SHAKE algorithms. class HashAlgorithm a => HashSHAKE a where     -- | Alternate finalization needed for cSHAKE     cshakeInternalFinalize :: Ptr (Context a) -> Ptr (Digest a) -> IO ()+     -- | Get the digest bit length     cshakeOutputLength :: a -> Int  -- | SHAKE128 (128 bits) extendable output function.  Supports an arbitrary -- digest size, to be specified as a type parameter of kind 'Nat'. ----- Note: outputs from @'SHAKE128' n@ and @'SHAKE128' m@ for the same input are+-- Note: outputs from @t'SHAKE128' n@ and @t'SHAKE128' m@ for the same input are -- correlated (one being a prefix of the other).  Results are unrelated to--- 'SHAKE256' results.+-- t'SHAKE256' results. data SHAKE128 (bitlen :: Nat) = SHAKE128     deriving (Show, Data)  instance KnownNat bitlen => HashAlgorithm (SHAKE128 bitlen) where-    type HashBlockSize           (SHAKE128 bitlen)  = 168-    type HashDigestSize          (SHAKE128 bitlen) = Div8 (bitlen + 7)+    type HashBlockSize (SHAKE128 bitlen) = 168+    type HashDigestSize (SHAKE128 bitlen) = Div8 (bitlen + 7)     type HashInternalContextSize (SHAKE128 bitlen) = 376-    hashBlockSize  _          = 168-    hashDigestSize _          = byteLen (Proxy :: Proxy bitlen)+    hashBlockSize _ = 168+    hashDigestSize _ = byteLen (Proxy :: Proxy bitlen)     hashInternalContextSize _ = 376-    hashInternalInit p        = c_sha3_init p 128-    hashInternalUpdate        = c_sha3_update-    hashInternalFinalize      = shakeFinalizeOutput (Proxy :: Proxy bitlen)+    hashInternalInit p = c_sha3_init p 128+    hashInternalUpdate = c_sha3_update+    hashInternalFinalize = shakeFinalizeOutput (Proxy :: Proxy bitlen)  instance KnownNat bitlen => HashSHAKE (SHAKE128 bitlen) where     cshakeInternalFinalize = cshakeFinalizeOutput (Proxy :: Proxy bitlen)@@ -65,42 +68,44 @@ -- | SHAKE256 (256 bits) extendable output function.  Supports an arbitrary -- digest size, to be specified as a type parameter of kind 'Nat'. ----- Note: outputs from @'SHAKE256' n@ and @'SHAKE256' m@ for the same input are+-- Note: outputs from @t'SHAKE256' n@ and @t'SHAKE256' m@ for the same input are -- correlated (one being a prefix of the other).  Results are unrelated to--- 'SHAKE128' results.+-- t'SHAKE128' results. data SHAKE256 (bitlen :: Nat) = SHAKE256     deriving (Show, Data)  instance KnownNat bitlen => HashAlgorithm (SHAKE256 bitlen) where-    type HashBlockSize           (SHAKE256 bitlen) = 136-    type HashDigestSize          (SHAKE256 bitlen) = Div8 (bitlen + 7)+    type HashBlockSize (SHAKE256 bitlen) = 136+    type HashDigestSize (SHAKE256 bitlen) = Div8 (bitlen + 7)     type HashInternalContextSize (SHAKE256 bitlen) = 344-    hashBlockSize  _          = 136-    hashDigestSize _          = byteLen (Proxy :: Proxy bitlen)+    hashBlockSize _ = 136+    hashDigestSize _ = byteLen (Proxy :: Proxy bitlen)     hashInternalContextSize _ = 344-    hashInternalInit p        = c_sha3_init p 256-    hashInternalUpdate        = c_sha3_update-    hashInternalFinalize      = shakeFinalizeOutput (Proxy :: Proxy bitlen)+    hashInternalInit p = c_sha3_init p 256+    hashInternalUpdate = c_sha3_update+    hashInternalFinalize = shakeFinalizeOutput (Proxy :: Proxy bitlen)  instance KnownNat bitlen => HashSHAKE (SHAKE256 bitlen) where     cshakeInternalFinalize = cshakeFinalizeOutput (Proxy :: Proxy bitlen)     cshakeOutputLength _ = integralNatVal (Proxy :: Proxy bitlen) -shakeFinalizeOutput :: KnownNat bitlen-                    => proxy bitlen-                    -> Ptr (Context a)-                    -> Ptr (Digest a)-                    -> IO ()+shakeFinalizeOutput+    :: KnownNat bitlen+    => proxy bitlen+    -> Ptr (Context a)+    -> Ptr (Digest a)+    -> IO () shakeFinalizeOutput d ctx dig = do     c_sha3_finalize_shake ctx     c_sha3_output ctx dig (byteLen d)     shakeTruncate d (castPtr dig) -cshakeFinalizeOutput :: KnownNat bitlen-                     => proxy bitlen-                     -> Ptr (Context a)-                     -> Ptr (Digest a)-                     -> IO ()+cshakeFinalizeOutput+    :: KnownNat bitlen+    => proxy bitlen+    -> Ptr (Context a)+    -> Ptr (Digest a)+    -> IO () cshakeFinalizeOutput d ctx dig = do     c_sha3_finalize_cshake ctx     c_sha3_output ctx dig (byteLen d)
Crypto/Hash/Skein256.hs view
@@ -1,3 +1,12 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE KindSignatures #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE TypeOperators #-}+{-# LANGUAGE UndecidableInstances #-}+ -- | -- Module      : Crypto.Hash.Skein256 -- License     : BSD-style@@ -7,51 +16,80 @@ -- -- Module containing the binding functions to work with the -- Skein256 cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.Skein256-    (  Skein256_224 (..), Skein256_256 (..)-    ) where--import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+module Crypto.Hash.Skein256 (+    Skein256 (..),+    Skein256_224 (..),+    Skein256_256 (..),+) where +import Crypto.Hash.Types+import Crypto.Internal.Nat+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)+import GHC.TypeLits (KnownNat, Nat, type (+))  -- | Skein256 (224 bits) cryptographic hash algorithm data Skein256_224 = Skein256_224-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Skein256_224 where-    type HashBlockSize           Skein256_224 = 32-    type HashDigestSize          Skein256_224 = 28+    type HashBlockSize Skein256_224 = 32+    type HashDigestSize Skein256_224 = 28     type HashInternalContextSize Skein256_224 = 96-    hashBlockSize  _          = 32-    hashDigestSize _          = 28+    hashBlockSize _ = 32+    hashDigestSize _ = 28     hashInternalContextSize _ = 96-    hashInternalInit p        = c_skein256_init p 224-    hashInternalUpdate        = c_skein256_update-    hashInternalFinalize p    = c_skein256_finalize p 224+    hashInternalInit p = c_skein256_init p 224+    hashInternalUpdate = c_skein256_update+    hashInternalFinalize p = c_skein256_finalize p 224  -- | Skein256 (256 bits) cryptographic hash algorithm data Skein256_256 = Skein256_256-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Skein256_256 where-    type HashBlockSize           Skein256_256 = 32-    type HashDigestSize          Skein256_256 = 32+    type HashBlockSize Skein256_256 = 32+    type HashDigestSize Skein256_256 = 32     type HashInternalContextSize Skein256_256 = 96-    hashBlockSize  _          = 32-    hashDigestSize _          = 32+    hashBlockSize _ = 32+    hashDigestSize _ = 32     hashInternalContextSize _ = 96-    hashInternalInit p        = c_skein256_init p 256-    hashInternalUpdate        = c_skein256_update-    hashInternalFinalize p    = c_skein256_finalize p 256+    hashInternalInit p = c_skein256_init p 256+    hashInternalUpdate = c_skein256_update+    hashInternalFinalize p = c_skein256_finalize p 256 +-- | Skein256 with the digest size given as a type parameter of kind 'Nat',+-- in bits.  @t'Skein256' 256@ is @t'Skein256_256'@; the sizes with a type of+-- their own+-- above are there for their names, and this one also takes the sizes that+-- have none.+--+-- A size that is not a whole number of bytes is rounded up to the next one,+-- as the implementation underneath does.+--+-- The output is produced in counter mode, a block of it per Threefish call,+-- so one large digest is a good deal cheaper than the same number of bytes+-- taken from repeated small ones: on an Apple M4, 512 KiB arrives at 947 MB/s+-- in one digest against 172 MB/s as 8192 separate @t'Skein256_256'@ ones.+--+-- Note the digest size goes into the configuration block, so it changes the+-- value the message is hashed from: a longer digest is /not/ an extension of+-- a shorter one.  That is the opposite of how t'Crypto.Hash.SHAKE.SHAKE128'+-- behaves.+data Skein256 (bitlen :: Nat) = Skein256+    deriving (Show, Data)++instance KnownNat bitlen => HashAlgorithm (Skein256 bitlen) where+    type HashBlockSize (Skein256 bitlen) = 32+    type HashDigestSize (Skein256 bitlen) = Div8 (bitlen + 7)+    type HashInternalContextSize (Skein256 bitlen) = 96+    hashBlockSize _ = 32+    hashDigestSize _ = byteLen (Proxy :: Proxy bitlen)+    hashInternalContextSize _ = 96+    hashInternalInit p = c_skein256_init p (integralNatVal (Proxy :: Proxy bitlen))+    hashInternalUpdate = c_skein256_update+    hashInternalFinalize p = c_skein256_finalize p (integralNatVal (Proxy :: Proxy bitlen))  foreign import ccall unsafe "crypton_skein256_init"     c_skein256_init :: Ptr (Context a) -> Word32 -> IO ()
Crypto/Hash/Skein512.hs view
@@ -1,3 +1,12 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE KindSignatures #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE TypeOperators #-}+{-# LANGUAGE UndecidableInstances #-}+ -- | -- Module      : Crypto.Hash.Skein512 -- License     : BSD-style@@ -7,81 +16,112 @@ -- -- Module containing the binding functions to work with the -- Skein512 cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.Skein512-    (  Skein512_224 (..), Skein512_256 (..), Skein512_384 (..), Skein512_512 (..)-    ) where--import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+module Crypto.Hash.Skein512 (+    Skein512 (..),+    Skein512_224 (..),+    Skein512_256 (..),+    Skein512_384 (..),+    Skein512_512 (..),+) where +import Crypto.Hash.Types+import Crypto.Internal.Nat+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)+import GHC.TypeLits (KnownNat, Nat, type (+))  -- | Skein512 (224 bits) cryptographic hash algorithm data Skein512_224 = Skein512_224-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Skein512_224 where-    type HashBlockSize           Skein512_224 = 64-    type HashDigestSize          Skein512_224 = 28+    type HashBlockSize Skein512_224 = 64+    type HashDigestSize Skein512_224 = 28     type HashInternalContextSize Skein512_224 = 160-    hashBlockSize  _          = 64-    hashDigestSize _          = 28+    hashBlockSize _ = 64+    hashDigestSize _ = 28     hashInternalContextSize _ = 160-    hashInternalInit p        = c_skein512_init p 224-    hashInternalUpdate        = c_skein512_update-    hashInternalFinalize p    = c_skein512_finalize p 224+    hashInternalInit p = c_skein512_init p 224+    hashInternalUpdate = c_skein512_update+    hashInternalFinalize p = c_skein512_finalize p 224  -- | Skein512 (256 bits) cryptographic hash algorithm data Skein512_256 = Skein512_256-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Skein512_256 where-    type HashBlockSize           Skein512_256 = 64-    type HashDigestSize          Skein512_256 = 32+    type HashBlockSize Skein512_256 = 64+    type HashDigestSize Skein512_256 = 32     type HashInternalContextSize Skein512_256 = 160-    hashBlockSize  _          = 64-    hashDigestSize _          = 32+    hashBlockSize _ = 64+    hashDigestSize _ = 32     hashInternalContextSize _ = 160-    hashInternalInit p        = c_skein512_init p 256-    hashInternalUpdate        = c_skein512_update-    hashInternalFinalize p    = c_skein512_finalize p 256+    hashInternalInit p = c_skein512_init p 256+    hashInternalUpdate = c_skein512_update+    hashInternalFinalize p = c_skein512_finalize p 256  -- | Skein512 (384 bits) cryptographic hash algorithm data Skein512_384 = Skein512_384-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Skein512_384 where-    type HashBlockSize           Skein512_384 = 64-    type HashDigestSize          Skein512_384 = 48+    type HashBlockSize Skein512_384 = 64+    type HashDigestSize Skein512_384 = 48     type HashInternalContextSize Skein512_384 = 160-    hashBlockSize  _          = 64-    hashDigestSize _          = 48+    hashBlockSize _ = 64+    hashDigestSize _ = 48     hashInternalContextSize _ = 160-    hashInternalInit p        = c_skein512_init p 384-    hashInternalUpdate        = c_skein512_update-    hashInternalFinalize p    = c_skein512_finalize p 384+    hashInternalInit p = c_skein512_init p 384+    hashInternalUpdate = c_skein512_update+    hashInternalFinalize p = c_skein512_finalize p 384  -- | Skein512 (512 bits) cryptographic hash algorithm data Skein512_512 = Skein512_512-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Skein512_512 where-    type HashBlockSize           Skein512_512 = 64-    type HashDigestSize          Skein512_512 = 64+    type HashBlockSize Skein512_512 = 64+    type HashDigestSize Skein512_512 = 64     type HashInternalContextSize Skein512_512 = 160-    hashBlockSize  _          = 64-    hashDigestSize _          = 64+    hashBlockSize _ = 64+    hashDigestSize _ = 64     hashInternalContextSize _ = 160-    hashInternalInit p        = c_skein512_init p 512-    hashInternalUpdate        = c_skein512_update-    hashInternalFinalize p    = c_skein512_finalize p 512+    hashInternalInit p = c_skein512_init p 512+    hashInternalUpdate = c_skein512_update+    hashInternalFinalize p = c_skein512_finalize p 512 +-- | Skein512 with the digest size given as a type parameter of kind 'Nat',+-- in bits.  @t'Skein512' 512@ is @t'Skein512_512'@; the sizes with a type of+-- their own+-- above are there for their names, and this one also takes the sizes that+-- have none.+--+-- A size that is not a whole number of bytes is rounded up to the next one,+-- as the implementation underneath does.+--+-- The output is produced in counter mode, a block of it per Threefish call,+-- so one large digest is a good deal cheaper than the same number of bytes+-- taken from repeated small ones: on an Apple M4, 512 KiB arrives at 947 MB/s+-- in one digest against 172 MB/s as 8192 separate @t'Skein512_512'@ ones.+--+-- Note the digest size goes into the configuration block, so it changes the+-- value the message is hashed from: a longer digest is /not/ an extension of+-- a shorter one.  That is the opposite of how t'Crypto.Hash.SHAKE.SHAKE128'+-- behaves.+data Skein512 (bitlen :: Nat) = Skein512+    deriving (Show, Data)++instance KnownNat bitlen => HashAlgorithm (Skein512 bitlen) where+    type HashBlockSize (Skein512 bitlen) = 64+    type HashDigestSize (Skein512 bitlen) = Div8 (bitlen + 7)+    type HashInternalContextSize (Skein512 bitlen) = 160+    hashBlockSize _ = 64+    hashDigestSize _ = byteLen (Proxy :: Proxy bitlen)+    hashInternalContextSize _ = 160+    hashInternalInit p = c_skein512_init p (integralNatVal (Proxy :: Proxy bitlen))+    hashInternalUpdate = c_skein512_update+    hashInternalFinalize p = c_skein512_finalize p (integralNatVal (Proxy :: Proxy bitlen))  foreign import ccall unsafe "crypton_skein512_init"     c_skein512_init :: Ptr (Context a) -> Word32 -> IO ()
Crypto/Hash/Tiger.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.Tiger -- License     : BSD-style@@ -7,35 +12,30 @@ -- -- Module containing the binding functions to work with the -- Tiger cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.Tiger ( Tiger (..) ) where+module Crypto.Hash.Tiger (Tiger (..)) where -import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | Tiger cryptographic hash algorithm data Tiger = Tiger-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Tiger where-    type HashBlockSize           Tiger = 64-    type HashDigestSize          Tiger = 24+    type HashBlockSize Tiger = 64+    type HashDigestSize Tiger = 24     type HashInternalContextSize Tiger = 96-    hashBlockSize  _          = 64-    hashDigestSize _          = 24+    hashBlockSize _ = 64+    hashDigestSize _ = 24     hashInternalContextSize _ = 96-    hashInternalInit          = c_tiger_init-    hashInternalUpdate        = c_tiger_update-    hashInternalFinalize      = c_tiger_finalize+    hashInternalInit = c_tiger_init+    hashInternalUpdate = c_tiger_update+    hashInternalFinalize = c_tiger_finalize  foreign import ccall unsafe "crypton_tiger_init"-    c_tiger_init :: Ptr (Context a)-> IO ()+    c_tiger_init :: Ptr (Context a) -> IO ()  foreign import ccall "crypton_tiger_update"     c_tiger_update :: Ptr (Context a) -> Ptr Word8 -> Word32 -> IO ()
Crypto/Hash/Types.hs view
@@ -1,3 +1,10 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# LANGUAGE RoleAnnotations #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.Types -- License     : BSD-style@@ -6,31 +13,36 @@ -- Portability : unknown -- -- Crypto hash types definitions----{-# LANGUAGE GeneralizedNewtypeDeriving #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE ScopedTypeVariables #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.Types-    ( HashAlgorithm(..)-    , HashAlgorithmPrefix(..)-    , Context(..)-    , Digest(..)-    ) where+module Crypto.Hash.Types (+    HashAlgorithm (..),+    HashAlgorithmPrefix (..),+    Context (..),+    Digest (..),+) where -import           Crypto.Internal.Imports-import           Crypto.Internal.ByteArray (ByteArrayAccess, Bytes)+import Control.DeepSeq (deepseq)+import Control.Monad.Primitive (PrimMonad (..))+import Control.Monad.ST+import Crypto.Internal.ByteArray (ByteArrayAccess (..), Bytes) import qualified Crypto.Internal.ByteArray as B-import           Control.Monad.ST-import           Data.Char (digitToInt, isHexDigit)-import           Foreign.Ptr (Ptr)-import           Basement.Block (Block, unsafeFreeze)-import           Basement.Block.Mutable (MutableBlock, new, unsafeWrite)-import           Basement.NormalForm (deepseq)-import           Basement.Types.OffsetSize (CountOf(..), Offset(..))-import           GHC.TypeLits (Nat)-import           Data.Data (Data)+import Crypto.Internal.Imports+import Data.Base16.Types (extractBase16)+import Data.ByteString (ByteString)+import Data.ByteString.Base16 (encodeBase16)+import Data.Char (digitToInt, isHexDigit)+import Data.Data (Data)+import Data.Primitive.ByteArray (+    ByteArray,+    MutableByteArray,+    newPinnedByteArray,+    sizeofByteArray,+    unsafeFreezeByteArray,+    withByteArrayContents,+    writeByteArray,+ )+import qualified Data.Text as Text+import Foreign.Ptr (Ptr, castPtr)+import GHC.TypeLits (Nat)  -- | Class representing hashing algorithms. --@@ -40,23 +52,30 @@ class HashAlgorithm a where     -- | Associated type for the block size of the hash algorithm     type HashBlockSize a :: Nat+     -- | Associated type for the digest size of the hash algorithm     type HashDigestSize a :: Nat+     -- | Associated type for the internal context size of the hash algorithm     type HashInternalContextSize a :: Nat      -- | Get the block size of a hash algorithm-    hashBlockSize           :: a -> Int+    hashBlockSize :: a -> Int+     -- | Get the digest size of a hash algorithm-    hashDigestSize          :: a -> Int+    hashDigestSize :: a -> Int+     -- | Get the size of the context used for a hash algorithm     hashInternalContextSize :: a -> Int-    --hashAlgorithmFromProxy  :: Proxy a -> a +    -- hashAlgorithmFromProxy  :: Proxy a -> a+     -- | Initialize a context pointer to the initial state of a hash algorithm-    hashInternalInit     :: Ptr (Context a) -> IO ()+    hashInternalInit :: Ptr (Context a) -> IO ()+     -- | Update the context with some raw data-    hashInternalUpdate   :: Ptr (Context a) -> Ptr Word8 -> Word32 -> IO ()+    hashInternalUpdate :: Ptr (Context a) -> Ptr Word8 -> Word32 -> IO ()+     -- | Finalize the context and set the digest raw memory to the right value     hashInternalFinalize :: Ptr (Context a) -> Ptr (Digest a) -> IO () @@ -65,11 +84,13 @@     -- | Update the context with the first N bytes of a buffer and finalize this     -- context.  The code path executed is independent from N and depends only     -- on the complete buffer length.-    hashInternalFinalizePrefix :: Ptr (Context a)-                               -> Ptr Word8 -> Word32-                               -> Word32-                               -> Ptr (Digest a)-                               -> IO ()+    hashInternalFinalizePrefix+        :: Ptr (Context a)+        -> Ptr Word8+        -> Word32+        -> Word32+        -> Ptr (Digest a)+        -> IO ()  {- hashContextGetAlgorithm :: HashAlgorithm a => Context a -> a@@ -82,42 +103,75 @@ -- layout is architecture dependent, may contain uninitialized data fragments, -- and change in future versions.  The bytearray should not be used as input to -- cryptographic algorithms.+--+-- __A context is not erased when it is finished with.__  A hash algorithm+-- buffers its input a block at a time, and finalizing does not clear what is+-- left there.  How much survives depends on where the message ended relative+-- to the block: with SHA-256, a 32-byte message is still in the context in+-- full afterwards, and a 100-byte one leaves its last 36 bytes.+-- @hashFinalize@ works on a copy, so the caller's own context keeps what it+-- had as well.  Nothing clears either of them: this is 'Bytes' rather than+-- @ScrubbedBytes@, and the C clears nothing.  They go to the garbage+-- collector as they are, and a core file, a crash dump or a swapped page can+-- carry them away afterwards.+--+-- That is a deliberate trade rather than an oversight, and there is no way+-- to ask for the other side of it: no operation here clears a context.+-- Scrubbing them all was measured at about 70% of a 32-byte hash and a third+-- of an incremental one, because the allocation is most of the work when the+-- message is short -- and short hashes are the common case, in HMAC, in+-- HKDF, and anywhere a key or an identifier is hashed.  A 64 KB hash does+-- not notice it.  Anything that must not be left in memory this way is+-- better not hashed through this interface at all. newtype Context a = Context Bytes-    deriving (ByteArrayAccess,NFData)+    deriving (ByteArrayAccess, NFData)  -- | Represent a digest for a given hash algorithm. -- -- This type is an instance of 'ByteArrayAccess' from package--- <https://hackage.haskell.org/package/memory memory>.+-- <https://hackage.haskell.org/package/ram ram>. -- Module "Data.ByteArray" provides many primitives to work with those values -- including conversion to other types. -- -- Creating a digest from a bytearray is also possible with function -- 'Crypto.Hash.digestFromByteString'.-newtype Digest a = Digest (Block Word8)-    deriving (Eq,Ord,ByteArrayAccess, Data)+newtype Digest a = Digest ByteArray+    deriving (Eq, Ord, Data) +type role Digest nominal+ instance NFData (Digest a) where     rnf (Digest u) = u `deepseq` () +instance ByteArrayAccess (Digest a) where+    length (Digest ba) = sizeofByteArray ba+    withByteArray (Digest ba) f = withByteArrayContents ba (f . castPtr)+ instance Show (Digest a) where-    show (Digest bs) = map (toEnum . fromIntegral)-                     $ B.unpack (B.convertToBase B.Base16 bs :: Bytes)+    show d =+        Text.unpack (extractBase16 $ encodeBase16 (B.convert d :: ByteString))  instance HashAlgorithm a => Read (Digest a) where-    readsPrec _ str = runST $ do mut <- new (CountOf len)-                                 loop mut len str+    readsPrec _ str = runST $ do+        mut <- newPinnedByteArray len+        loop len mut len str       where         len = hashDigestSize (undefined :: a) -        loop :: MutableBlock Word8 s -> Int -> String -> ST s [(Digest a, String)]-        loop mut 0   cs          = (\b -> [(Digest b, cs)]) <$> unsafeFreeze mut-        loop _   _   []          = return []-        loop _   _   [_]         = return []-        loop mut n   (c:(d:ds))-            | not (isHexDigit c) = return []-            | not (isHexDigit d) = return []-            | otherwise          = do-                let w8 = fromIntegral $ digitToInt c * 16 + digitToInt d-                unsafeWrite mut (Offset $ len - n) w8-                loop mut (n - 1) ds+loop+    :: Int+    -> MutableByteArray (PrimState (ST s))+    -> Int+    -> String+    -> ST s [(Digest a, String)]+loop _ mut 0 cs = (\b -> [(Digest b, cs)]) <$> unsafeFreezeByteArray mut+loop _ _ _ [] = return []+loop _ _ _ [_] = return []+loop len mut n (c : (d : ds))+    | not (isHexDigit c) = return []+    | not (isHexDigit d) = return []+    | otherwise = do+        let w8 :: Word8+            w8 = fromIntegral $ digitToInt c * 16 + digitToInt d+        writeByteArray mut (len - n) w8+        loop len mut (n - 1) ds
Crypto/Hash/Whirlpool.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE TypeFamilies #-}+ -- | -- Module      : Crypto.Hash.Whirlpool -- License     : BSD-style@@ -7,35 +12,30 @@ -- -- Module containing the binding functions to work with the -- Whirlpool cryptographic hash.----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeFamilies #-}-module Crypto.Hash.Whirlpool ( Whirlpool (..) ) where+module Crypto.Hash.Whirlpool (Whirlpool (..)) where -import           Crypto.Hash.Types-import           Foreign.Ptr (Ptr)-import           Data.Data-import           Data.Word (Word8, Word32)+import Crypto.Hash.Types+import Data.Data+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr)  -- | Whirlpool cryptographic hash algorithm data Whirlpool = Whirlpool-    deriving (Show,Data)+    deriving (Show, Data)  instance HashAlgorithm Whirlpool where-    type HashBlockSize           Whirlpool = 64-    type HashDigestSize          Whirlpool = 64+    type HashBlockSize Whirlpool = 64+    type HashDigestSize Whirlpool = 64     type HashInternalContextSize Whirlpool = 168-    hashBlockSize  _          = 64-    hashDigestSize _          = 64+    hashBlockSize _ = 64+    hashDigestSize _ = 64     hashInternalContextSize _ = 168-    hashInternalInit          = c_whirlpool_init-    hashInternalUpdate        = c_whirlpool_update-    hashInternalFinalize      = c_whirlpool_finalize+    hashInternalInit = c_whirlpool_init+    hashInternalUpdate = c_whirlpool_update+    hashInternalFinalize = c_whirlpool_finalize  foreign import ccall unsafe "crypton_whirlpool_init"-    c_whirlpool_init :: Ptr (Context a)-> IO ()+    c_whirlpool_init :: Ptr (Context a) -> IO ()  foreign import ccall "crypton_whirlpool_update"     c_whirlpool_update :: Ptr (Context a) -> Ptr Word8 -> Word32 -> IO ()
Crypto/Internal/Builder.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE BangPatterns #-}+ -- | -- Module      : Crypto.Internal.Builder -- License     : BSD-style@@ -8,31 +10,30 @@ -- Delaying and merging ByteArray allocations.  This is similar to module -- "Data.ByteArray.Pack" except the total length is computed automatically based -- on what is appended.----{-# LANGUAGE BangPatterns #-}-module Crypto.Internal.Builder-    ( Builder-    , buildAndFreeze-    , builderLength-    , byte-    , bytes-    , zero-    ) where+module Crypto.Internal.Builder (+    Builder,+    buildAndFreeze,+    builderLength,+    byte,+    bytes,+    zero,+) where -import           Data.ByteArray (ByteArray, ByteArrayAccess)+import Data.ByteArray (ByteArray, ByteArrayAccess) import qualified Data.ByteArray as B-import           Data.Memory.PtrMethods (memSet)+import Data.Memory.PtrMethods (memSet) -import           Foreign.Ptr (Ptr, plusPtr)-import           Foreign.Storable (poke)+import Foreign.Ptr (Ptr, plusPtr)+import Foreign.Storable (poke) -import           Crypto.Internal.Imports hiding (empty)+import Crypto.Internal.Imports hiding (empty) -data Builder =  Builder !Int (Ptr Word8 -> IO ())  -- size and initializer+data Builder = Builder !Int (Ptr Word8 -> IO ()) -- size and initializer  instance Semigroup Builder where     (Builder s1 f1) <> (Builder s2 f2) = Builder (s1 + s2) f-      where f p = f1 p >> f2 (p `plusPtr` s1)+      where+        f p = f1 p >> f2 (p `plusPtr` s1)  builderLength :: Builder -> Int builderLength (Builder s _) = s
Crypto/Internal/ByteArray.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE BangPatterns #-}+{-# OPTIONS_HADDOCK hide #-}+ -- | -- Module      : Crypto.Internal.ByteArray -- License     : BSD-style@@ -6,34 +9,115 @@ -- Portability : Good -- -- Simple and efficient byte array types----{-# LANGUAGE BangPatterns #-}-{-# OPTIONS_HADDOCK hide #-}-module Crypto.Internal.ByteArray-    ( module Data.ByteArray-    , module Data.ByteArray.Mapping-    , module Data.ByteArray.Encoding-    , constAllZero-    ) where+module Crypto.Internal.ByteArray (+    module Data.ByteArray,+    module Data.ByteArray.Mapping,+    module Data.ByteArray.Encoding,+    constAllZero,+    overCLength,+    inCLengths,+    allocAndFreezePrimIO,+    allocAndFreezePrim,+    bxor,+) where  import Data.ByteArray-import Data.ByteArray.Mapping import Data.ByteArray.Encoding+import Data.ByteArray.Mapping  import Data.Bits ((.|.))-import Data.Word (Word8)-import Foreign.Ptr (Ptr)+import qualified Data.Primitive.ByteArray as Prim+import Data.Word (Word32, Word8)+import Foreign.Ptr (Ptr, castPtr) import Foreign.Storable (peekByteOff)  import Crypto.Internal.Compat (unsafeDoIO) +-- | Whether a length is too large to reach the C, which takes its lengths as+-- @uint32_t@.+--+-- From 2^32 up the value is truncated on the way down, and the C then works+-- on the low bits of it and leaves the rest of the buffer as it found it --+-- which for a fresh allocation is zeros.  What comes back is as long as the+-- caller asked for, with nothing to say that most of it was never written:+-- a 4 GiB message through Crypto.Cipher.ChaCha.combine came back with 2^32+-- bytes of zeros where the ciphertext should have been.+--+-- Every place that hands a caller's length to the C either turns it away+-- with this or cuts the work into pieces small enough to pass.+--+-- The round trip through 'Word32' rather than a comparison against 2^32,+-- which a 32-bit 'Int' cannot hold.  There every non-negative 'Int' passes,+-- which is the right answer: there is no such buffer to be had.+overCLength :: Int -> Bool+overCLength n = fromIntegral (fromIntegral n :: Word32) /= n++-- | Walk a length in pieces small enough to reach the C, calling the action+-- with the offset and the size of each.+--+-- The same 2 GiB step "Crypto.Hash" has always taken, and for the same+-- reason: the C takes its lengths as @uint32_t@, and a 32-bit 'Int' cannot+-- hold a whole one either.  This is for the C that keeps its state in a+-- context and can simply be called again -- the stream ciphers, the MACs --+-- where a long message can be enciphered in pieces rather than refused.+inCLengths :: Int -> (Int -> Int -> IO ()) -> IO ()+inCLengths total f = go 0+  where+    go !off+        | off >= total = return ()+        | otherwise = f off n >> go (off + n)+      where+        !n = min (total - off) cChunk++-- | The step 'inCLengths' takes: the largest multiple of 64 that a signed+-- 32-bit integer holds.+--+-- Under 2^31 because a 32-bit 'Int' cannot hold more, and a+-- multiple of 64 because some of the C this feeds -- the AEAD modes -- will+-- take a piece that is not a whole number of blocks only as the last one.+cChunk :: Int+cChunk = 0x7fffffc0++-- | Allocate a pinned 'Prim.ByteArray' of the given size, populate it via a+-- 'Ptr', then freeze and return it.  The pointer must not be retained after+-- the action returns.+allocAndFreezePrimIO :: Int -> (Ptr p -> IO ()) -> IO Prim.ByteArray+allocAndFreezePrimIO n f = do+    mba <- Prim.newPinnedByteArray n+    f (castPtr (Prim.mutableByteArrayContents mba))+    Prim.unsafeFreezeByteArray mba++-- | The allocation is strictly local,+-- the computation is deterministic, and no IO effects escape.+allocAndFreezePrim :: Int -> (Ptr p -> IO ()) -> Prim.ByteArray+allocAndFreezePrim n = unsafeDoIO . allocAndFreezePrimIO n+ constAllZero :: ByteArrayAccess ba => ba -> Bool constAllZero b = unsafeDoIO $ withByteArray b $ \p -> loop p 0 0   where     loop :: Ptr b -> Int -> Word8 -> IO Bool     loop p i !acc-        | i == len  = return $! acc == 0+        | i == len = return $! acc == 0         | otherwise = do             e <- peekByteOff p i-            loop p (i+1) (acc .|. e)+            loop p (i + 1) (acc .|. e)     len = Data.ByteArray.length b++-- | @a@ exclusive-ored with @b@, as long as the shorter of the two.+--+-- 'Data.ByteArray.xor' does this a byte at a time through an IO applicative,+-- which allocates about fifty bytes of heap for every byte it produces.  That+-- is more than a block cipher costs: it was four fifths of the time counter+-- mode spent on anything but AES, whose modes are in C and do not come this+-- way.+bxor :: (ByteArrayAccess a, ByteArrayAccess b, ByteArray c) => a -> b -> c+bxor a b = unsafeDoIO $+    alloc n $ \pd ->+        withByteArray a $ \pa ->+            withByteArray b $ \pb ->+                c_memxor pd pa pb (fromIntegral n)+  where+    n = min (Data.ByteArray.length a) (Data.ByteArray.length b)++foreign import ccall unsafe "crypton_memxor.h crypton_memxor"+    c_memxor :: Ptr Word8 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO ()
Crypto/Internal/Compat.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE CPP #-}+ -- | -- Module      : Crypto.Internal.Compat -- License     : BSD-style@@ -7,17 +9,15 @@ -- -- This module tries to keep all the difference between versions of base -- or other needed packages, so that modules don't need to use CPP.----{-# LANGUAGE CPP #-}-module Crypto.Internal.Compat-    ( unsafeDoIO-    , popCount-    , byteSwap64-    ) where+module Crypto.Internal.Compat (+    unsafeDoIO,+    popCount,+    byteSwap64,+) where -import System.IO.Unsafe-import Data.Word import Data.Bits+import Data.Word+import System.IO.Unsafe  -- | Perform io for hashes that do allocation and FFI. -- 'unsafeDupablePerformIO' is used when possible as the
Crypto/Internal/CompatPrim.hs view
@@ -1,3 +1,8 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE CPP #-}+{-# LANGUAGE MagicHash #-}+{-# LANGUAGE UnboxedTuples #-}+ -- | -- Module      : Crypto.Internal.CompatPrim -- License     : BSD-style@@ -10,27 +15,22 @@ -- -- Note that MagicHash and CPP conflicts in places, making it "more interesting" -- to write compat code for primitives.----{-# LANGUAGE CPP #-}-{-# LANGUAGE BangPatterns #-}-{-# LANGUAGE MagicHash #-}-{-# LANGUAGE UnboxedTuples #-}-module Crypto.Internal.CompatPrim-    ( be32Prim-    , le32Prim-    , byteswap32Prim-    , booleanPrim-    , convert4To32-    ) where+module Crypto.Internal.CompatPrim (+    be32Prim,+    le32Prim,+    byteswap32Prim,+    booleanPrim,+    convert4To32,+) where  #if !defined(ARCH_IS_LITTLE_ENDIAN) && !defined(ARCH_IS_BIG_ENDIAN) import Data.Memory.Endian (getSystemEndianness, Endianness(..)) #endif  #if __GLASGOW_HASKELL__ >= 902-import GHC.Prim+import GHC.Exts #else-import GHC.Prim hiding (Word32#)+import GHC.Exts hiding (Word32#) type Word32# = Word# #endif @@ -68,8 +68,12 @@ #endif  -- | Combine 4 word8 [a,b,c,d] to a word32 representing [a,b,c,d]-convert4To32 :: Word# -> Word# -> Word# -> Word#-             -> Word#+convert4To32+    :: Word#+    -> Word#+    -> Word#+    -> Word#+    -> Word# convert4To32 a b c d = or# (or# c1 c2) (or# c3 c4)   where #ifdef ARCH_IS_LITTLE_ENDIAN
Crypto/Internal/DeepSeq.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE CPP #-}+ -- | -- Module      : Crypto.Internal.DeepSeq -- License     : BSD-style@@ -8,11 +10,9 @@ -- Simple abstraction module to allow compilation without deepseq -- by defining our own NFData class if not compiling with deepseq -- support.----{-# LANGUAGE CPP #-}-module Crypto.Internal.DeepSeq-    ( NFData(..)-    ) where+module Crypto.Internal.DeepSeq (+    NFData (..),+) where  #ifdef WITH_DEEPSEQ_SUPPORT import Control.DeepSeq
+ Crypto/Internal/ECC.hs view
@@ -0,0 +1,524 @@+{-# LANGUAGE BangPatterns #-}++-- |+-- Module      : Crypto.Internal.ECC+-- License     : BSD-style+-- Maintainer  : Kazu Yamamoto <kazu@iij.ad.jp>+-- Stability   : experimental+-- Portability : Good+--+-- The C scalar multiplication for curves over a prime field, which both of+-- the elliptic curve APIs reach for.+module Crypto.Internal.ECC (+    MulResult (..),+    CurveField (..),+    curveMul,+    primeCurveMul,+    primeCurveTableMul,+    baseTable,+    binaryCurveMul,+    binaryCurveC,+) where++import Crypto.Internal.Compat (unsafeDoIO)+import Crypto.Number.Basic (numBits, numBytes)+import Crypto.Number.F2m (addF2m, divF2m, mulF2m, squareF2m)+import qualified Crypto.Number.Serialize.Internal as Internal+import Crypto.PubKey.ECC.Types (+    Curve (..),+    CurveCommon (..),+    CurveName,+    CurvePrime (..),+    Point (..),+    getCurveByName,+ )+import Data.Bits (testBit)+import Data.Word (Word32, Word8)+import Foreign.C.Types (CInt (..))+import Foreign.ForeignPtr (ForeignPtr, mallocForeignPtrBytes, withForeignPtr)+import Foreign.Marshal.Alloc (allocaBytes)+import Foreign.Ptr (Ptr, plusPtr)++-- | What the C made of it.+data MulResult+    = -- | the point it arrived at+      MulPoint !Integer !Integer+    | -- | the point at infinity, which has no coordinates+      MulInfinity+    | -- | not something the C works with, so the caller has to+      MulUnsupported+    deriving (Show, Eq)++-- | Multiply a point by a scalar on the curve @y^2 = x^3 + a*x + b@ over the+-- field of @p@, which has to be an odd prime.  The point has to be on the+-- curve and not the point at infinity, and its coordinates, @a@ and @b@ have+-- to be under @p@; the caller has all of that to hand and the C does not+-- check it.+--+-- The scalar is walked four bits at a time over the whole of the width asked+-- for, so its value is hidden but that width is not.  Ask for the width of+-- the curve's order, which is public, and every scalar in range costs the+-- same.+primeCurveMul+    :: Integer+    -- ^ p+    -> Integer+    -- ^ a+    -> Integer+    -- ^ b+    -> Int+    -- ^ how many bytes of scalar to walk+    -> Integer+    -- ^ the scalar+    -> Integer+    -- ^ the point's x+    -> Integer+    -- ^ the point's y+    -> MulResult+primeCurveMul p a b klen k px py+    | p <= 0 || even p || klen <= 0 || k < 0 = MulUnsupported+    | otherwise = unsafeDoIO $+        allocaBytes (sum widths) $ \base -> case scanl plusPtr base widths of+            (outx : outy : cx : cy : ca : cb : cp : ck : _) -> do+                _ <- Internal.i2ospOf px cx plen+                _ <- Internal.i2ospOf py cy plen+                _ <- Internal.i2ospOf a ca plen+                _ <- Internal.i2ospOf b cb plen+                _ <- Internal.i2ospOf p cp plen+                _ <- Internal.i2ospOf k ck klen+                r <-+                    c_ecc_mul+                        outx+                        outy+                        cx+                        cy+                        ck+                        (fromIntegral klen)+                        ca+                        cb+                        cp+                        (fromIntegral plen)+                -- the scalar is the caller's secret, and this is the last place+                -- it is written out in the clear+                Internal.i2ospOf 0 ck klen >> return ()+                case r of+                    0 -> do+                        !x <- Internal.os2ip outx plen+                        !y <- Internal.os2ip outy plen+                        return (MulPoint x y)+                    1 -> return MulInfinity+                    _ -> return MulUnsupported+            _ -> return MulUnsupported -- there are eight, but say so anyway+  where+    !plen = numBytes p+    -- What the buffer holds, in this order: the two coordinates out, the two+    -- in, a, b, the prime, and the scalar.  The room to take and where each+    -- one starts both come from here, so they cannot drift apart.+    --+    -- They did once, and nothing caught it: the memory is a pinned array on+    -- the GHC heap, so writing past it is invisible to valgrind, which sees+    -- one large allocation, and to the sanity checks of the debug RTS, which+    -- found nothing when the mistake was put back to try them.  One runner+    -- out of eighteen died of it and the rest went green.  The way to be+    -- right about this is not to have two numbers to keep the same.+    widths = [plen, plen, plen, plen, plen, plen, plen, klen]++foreign import ccall unsafe "crypton_ecc_table_size"+    c_ecc_table_size :: Word32 -> Word32 -> Word32++foreign import ccall safe "crypton_ecc_table_build"+    c_ecc_table_build+        :: Ptr Word8+        -> Ptr Word8+        -> Ptr Word8+        -> Word32+        -> Ptr Word8+        -> Ptr Word8+        -> Ptr Word8+        -> Word32+        -> IO CInt++foreign import ccall safe "crypton_ecc_table_mul"+    c_ecc_table_mul+        :: Ptr Word8+        -> Ptr Word8+        -> Ptr Word8+        -> Ptr Word8+        -> Word32+        -> Ptr Word8+        -> Ptr Word8+        -> Ptr Word8+        -> Word32+        -> IO CInt++foreign import ccall safe "crypton_ecc_mul"+    c_ecc_mul+        :: Ptr Word8+        -> Ptr Word8+        -> Ptr Word8+        -> Ptr Word8+        -> Ptr Word8+        -> Word32+        -> Ptr Word8+        -> Ptr Word8+        -> Ptr Word8+        -> Word32+        -> IO CInt++-- | What a curve is made of, as much of it as a multiplication needs.+data CurveField+    = -- | over a prime field: the prime, a and b+      Prime !Integer !Integer !Integer+    | -- | over a binary field: the polynomial and b+      Binary !Integer !Integer+    deriving (Show, Eq)++-- | Multiply a point by a scalar, through the C wherever the C takes it.+--+-- Both elliptic curve APIs come here, so that the decision -- the table for a+-- base point, the C for anything else, what is left over -- is made once and+-- in one place.  One of those APIs cannot be reached from outside the library+-- on a curve over a binary field, and this is how that copy stays the same+-- code as the copy everybody runs.+--+-- The caller has seen to it that the point is on the curve, which is what the+-- C takes for granted, and deals with 'MulUnsupported' in whatever way it+-- has.+curveMul+    :: CurveField+    -> Integer+    -- ^ the order of the curve+    -> Integer+    -- ^ the scalar+    -> Integer+    -- ^ the point's x+    -> Integer+    -- ^ the point's y+    -> Bool+    -- ^ whether that point is the curve's base point+    -> MulResult+curveMul field order k px py isBase = case field of+    Prime p a b+        | isBase+        , klen == numBytes order+        , Just table <- baseTable p a b klen px py ->+            primeCurveTableMul table p a b klen k+        | otherwise -> primeCurveMul p a b klen k px py+    Binary fx b+        | px == 0 -> MulUnsupported -- its own negation, and easier the long way+        | otherwise -> case binaryCurveC fx b klen k px py of+            -- the ladder in Haskell, for a field the C will not take+            MulUnsupported -> binaryCurveMul fx b (klen * 8) k px py+            r -> r+  where+    -- Walk the width of the order, which is public, so a scalar in range --+    -- which is every secret one -- costs the same whatever it is.  A scalar+    -- may still be given out of range, and then the width has to follow it or+    -- the high bits would be dropped.+    !klen = max (numBytes k) (numBytes order)++-- | The table for the base point of a curve the library knows, which is the+-- point signing and making a key multiply and the only point worth keeping a+-- table for.  The curves are told apart by their numbers, which are public,+-- so both of the elliptic curve APIs find the same table.+--+-- Each is built when it is first wanted and kept for as long as the program+-- runs, and a curve nobody multiplies the base point of never has one built.+-- Building costs 2.8 ms for secp256k1, 5.5 for secp384r1 and 10.6 for+-- secp521r1, and the last two take 221 KB and 456 KB.  A multiplication with+-- the table takes about a third of what one without it takes, so the build+-- pays for itself after about fifteen of them: a program that signs many+-- times wins, and one that signs once and exits does not.+baseTable+    :: Integer+    -- ^ p+    -> Integer+    -- ^ a+    -> Integer+    -- ^ b+    -> Int+    -- ^ how many bytes of scalar are wanted+    -> Integer+    -- ^ the base point's x+    -> Integer+    -- ^ the base point's y+    -> Maybe (ForeignPtr Word8)+baseTable p a b klen gx gy =+    case lookup (p, a, b, klen, gx, gy) baseTables of+        Just table -> table+        Nothing -> Nothing++type TableKey = (Integer, Integer, Integer, Int, Integer, Integer)++baseTables :: [(TableKey, Maybe (ForeignPtr Word8))]+baseTables =+    [ ((p, a, b, klen, gx, gy), primeCurveTable p a b klen gx gy)+    | name <- [minBound .. maxBound] :: [CurveName]+    , CurveFP (CurvePrime p cc) <- [getCurveByName name]+    , Point gx gy <- [ecc_g cc]+    , let a = ecc_a cc+    , let b = ecc_b cc+    , let klen = numBytes (ecc_n cc)+    ]+{-# NOINLINE baseTables #-}++-- | The multiples of a point that 'primeCurveTableMul' wants: for every four+-- bits of a scalar, the sixteen points those bits can call for.  Building it+-- costs a few thousand point operations, and what it saves is all the+-- doublings of every multiplication that uses it, so it is worth keeping for+-- as long as the point is -- which for a curve's base point is forever.+--+-- The arguments are as for 'primeCurveMul'.  'Nothing' means the C would not+-- take them.+primeCurveTable+    :: Integer+    -- ^ p+    -> Integer+    -- ^ a+    -> Integer+    -- ^ b+    -> Int+    -- ^ how many bytes of scalar the table is to cover+    -> Integer+    -- ^ the point's x+    -> Integer+    -- ^ the point's y+    -> Maybe (ForeignPtr Word8)+primeCurveTable p a b klen px py+    | p <= 0 || even p || klen <= 0 || size == 0 = Nothing+    | otherwise = unsafeDoIO $ do+        table <- mallocForeignPtrBytes (fromIntegral size)+        allocaBytes (sum widths) $ \base -> case scanl plusPtr base widths of+            (cx : cy : ca : cb : cp : _) -> do+                _ <- Internal.i2ospOf px cx plen+                _ <- Internal.i2ospOf py cy plen+                _ <- Internal.i2ospOf a ca plen+                _ <- Internal.i2ospOf b cb plen+                _ <- Internal.i2ospOf p cp plen+                r <- withForeignPtr table $ \t ->+                    c_ecc_table_build+                        t+                        cx+                        cy+                        (fromIntegral klen)+                        ca+                        cb+                        cp+                        (fromIntegral plen)+                return $ if r == 0 then Just table else Nothing+            _ -> return Nothing -- there are five, but say so anyway+  where+    !plen = numBytes p+    !size = c_ecc_table_size (fromIntegral plen) (fromIntegral klen)+    -- the point, a, b and the prime, all of the prime's width.  The room to+    -- take and where each one starts both come from here, so they cannot+    -- drift apart: they did once, and nothing caught it -- see the note on+    -- primeCurveMul.+    widths = [plen, plen, plen, plen, plen]++-- | Multiply the point a table was built for by a scalar of the width the+-- table was built for.  One addition for every four bits and no doublings.+primeCurveTableMul+    :: ForeignPtr Word8+    -- ^ the table+    -> Integer+    -- ^ p+    -> Integer+    -- ^ a+    -> Integer+    -- ^ b+    -> Int+    -- ^ the width the table was built for+    -> Integer+    -- ^ the scalar+    -> MulResult+primeCurveTableMul table p a b klen k+    | p <= 0 || even p || klen <= 0 || k < 0 = MulUnsupported+    | otherwise = unsafeDoIO $+        allocaBytes (sum widths) $ \base -> case scanl plusPtr base widths of+            (outx : outy : ca : cb : cp : ck : _) -> do+                _ <- Internal.i2ospOf a ca plen+                _ <- Internal.i2ospOf b cb plen+                _ <- Internal.i2ospOf p cp plen+                _ <- Internal.i2ospOf k ck klen+                r <- withForeignPtr table $ \t ->+                    c_ecc_table_mul+                        outx+                        outy+                        t+                        ck+                        (fromIntegral klen)+                        ca+                        cb+                        cp+                        (fromIntegral plen)+                Internal.i2ospOf 0 ck klen >> return ()+                case r of+                    0 -> do+                        !x <- Internal.os2ip outx plen+                        !y <- Internal.os2ip outy plen+                        return (MulPoint x y)+                    1 -> return MulInfinity+                    _ -> return MulUnsupported+            _ -> return MulUnsupported -- there are six, but say so anyway+  where+    !plen = numBytes p+    widths = [plen, plen, plen, plen, plen, klen]++-- | Multiply a point by a scalar on the curve @y^2 + x*y = x^3 + a*x^2 + b@+-- over the binary field of @fx@, by Montgomery's ladder.+--+-- The ladder carries the multiples of two consecutive numbers, whose+-- difference is therefore the point itself, and every bit of the scalar costs+-- one addition and one doubling of them whichever way it goes.  Only the x+-- coordinates are carried -- the difference being known is what lets them be+-- -- and the y is worked out at the end from the two of them, which is what+-- makes the coordinates projective: one division for the whole+-- multiplication rather than one for every step.+--+-- The point has to be on the curve and to have an x, which is what the+-- caller has to hand: the one point with no x is its own negation and is+-- easier multiplied the long way.  The scalar is walked over the whole of+-- the width asked for, so its value is hidden but that width is not.+binaryCurveMul+    :: Integer+    -- ^ the polynomial the field is over+    -> Integer+    -- ^ b+    -> Int+    -- ^ how many bits of scalar to walk+    -> Integer+    -- ^ the scalar+    -> Integer+    -- ^ the point's x+    -> Integer+    -- ^ the point's y+    -> MulResult+binaryCurveMul fx b bits k x y+    | bits <= 0 || k < 0 || x == 0 = MulUnsupported+    | otherwise = recover (go (bits - 1) (1, 0) (x, 1))+  where+    infixl 6 .+.+    (.+.) = addF2m+    sqr = squareF2m fx+    mul = mulF2m fx++    -- The two of them added, which the difference between them being the+    -- point makes possible from their x coordinates alone.  It does not+    -- matter which way round they come.+    madd (xa, za) (xb, zb) =+        let t1 = mul xa zb+            t2 = mul xb za+            z = sqr (t1 .+. t2)+         in (mul x z .+. mul t1 t2, z)++    -- One of them doubled.+    mdouble (xa, za) =+        let xa2 = sqr xa+            za2 = sqr za+         in (sqr xa2 .+. mul b (sqr za2), mul xa2 za2)++    -- Nothing is at infinity to begin with and the point is next to it, and+    -- from there each bit takes the pair to twice where it was.  The bangs+    -- are what make both halves happen: without them the one the bit does not+    -- call for would stay a thunk, and the work would follow the scalar.+    go i p1 p2+        | i < 0 = (p1, p2)+        | testBit k i =+            let !s = madd p1 p2+                !d = mdouble p2+             in go (i - 1) s d+        | otherwise =+            let !s = madd p1 p2+                !d = mdouble p1+             in go (i - 1) d s++    -- x1 is the answer and x2 is one point further on; together with the+    -- point they give the y that the ladder does not carry.+    recover ((x1, z1), (x2, z2))+        | z1 == 0 = MulInfinity -- the multiple is at infinity+        | z2 == 0 = MulPoint x (x .+. y) -- the one after it is, so this is -P+        | otherwise = case (divF2m fx x1 z1, divF2m fx x2 z2) of+            (Just xa, Just xb) ->+                let u = xa .+. x+                    v = xb .+. x+                    inner = mul u v .+. sqr x .+. y+                 in case divF2m fx (mul u inner) x of+                        Just w -> MulPoint xa (w .+. y)+                        Nothing -> MulUnsupported+            _ -> MulUnsupported++-- | Multiply a point by a scalar on a curve over a binary field, in C.+--+-- The ladder is the same one 'binaryCurveMul' walks, but the field arithmetic+-- is carry-less multiplication -- the processor's where it has it, and four+-- interleaved groups of bits where it does not -- rather than 'Integer'+-- shifts and exclusive ors, and nothing in it branches on the scalar or+-- indexes memory with it.+--+-- The point has to be on the curve and to have an x, and the scalar is walked+-- over the whole of the width asked for, as for 'primeCurveMul'.+binaryCurveC+    :: Integer+    -- ^ the polynomial the field is over+    -> Integer+    -- ^ b+    -> Int+    -- ^ how many bytes of scalar to walk+    -> Integer+    -- ^ the scalar+    -> Integer+    -- ^ the point's x+    -> Integer+    -- ^ the point's y+    -> MulResult+binaryCurveC fx b klen k px py+    | fx <= 1 || klen <= 0 || k < 0 || px <= 0 || flen <= 0 = MulUnsupported+    | otherwise = unsafeDoIO $+        allocaBytes (sum widths) $ \base -> case scanl plusPtr base widths of+            (outx : outy : cx : cy : cb : cf : ck : _) -> do+                _ <- Internal.i2ospOf px cx flen+                _ <- Internal.i2ospOf py cy flen+                _ <- Internal.i2ospOf b cb flen+                _ <- Internal.i2ospOf fx cf fxlen+                _ <- Internal.i2ospOf k ck klen+                r <-+                    c_f2m_mul+                        outx+                        outy+                        cx+                        cy+                        ck+                        (fromIntegral klen)+                        cb+                        (fromIntegral flen)+                        cf+                        (fromIntegral fxlen)+                Internal.i2ospOf 0 ck klen >> return ()+                case r of+                    0 -> do+                        !x <- Internal.os2ip outx flen+                        !y <- Internal.os2ip outy flen+                        return (MulPoint x y)+                    1 -> return MulInfinity+                    _ -> return MulUnsupported+            _ -> return MulUnsupported -- there are seven, but say so anyway+  where+    -- the field is the degree of the polynomial, which is one under its width+    !flen = (numBits fx - 1 + 7) `div` 8+    !fxlen = numBytes fx+    widths = [flen, flen, flen, flen, flen, fxlen, klen]++foreign import ccall safe "crypton_f2m_mul"+    c_f2m_mul+        :: Ptr Word8+        -> Ptr Word8+        -> Ptr Word8+        -> Ptr Word8+        -> Ptr Word8+        -> Word32+        -> Ptr Word8+        -> Word32+        -> Ptr Word8+        -> Word32+        -> IO CInt
+ Crypto/Internal/Endian.hs view
@@ -0,0 +1,43 @@+{-# LANGUAGE CPP #-}++-- |+-- Module      : Crypto.Internal.Endian+-- License     : BSD-style+-- Maintainer  : Vincent Hanquez <vincent@snarc.org>+-- Stability   : stable+-- Portability : good+module Crypto.Internal.Endian (+    fromBE64,+    toBE64,+    fromLE64,+    toLE64,+) where++import Crypto.Internal.Compat (byteSwap64)+import Data.Word (Word64)++#ifdef ARCH_IS_LITTLE_ENDIAN+fromLE64 :: Word64 -> Word64+fromLE64 = id++toLE64 :: Word64 -> Word64+toLE64 = id++fromBE64 :: Word64 -> Word64+fromBE64 = byteSwap64++toBE64 :: Word64 -> Word64+toBE64 = byteSwap64+#else+fromLE64 :: Word64 -> Word64+fromLE64 = byteSwap64++toLE64 :: Word64 -> Word64+toLE64 = byteSwap64++fromBE64 :: Word64 -> Word64+fromBE64 = id++toBE64 :: Word64 -> Word64+toBE64 = id+#endif
Crypto/Internal/Imports.hs view
@@ -1,20 +1,20 @@+{-# LANGUAGE CPP #-}+ -- | -- Module      : Crypto.Internal.Imports -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : unknown----{-# LANGUAGE CPP #-}-module Crypto.Internal.Imports-    ( module X-    ) where+module Crypto.Internal.Imports (+    module X,+) where -import Data.Word               as X+import Data.Word as X #if !(MIN_VERSION_base(4,11,0)) import Data.Semigroup          as X (Semigroup(..)) #endif-import Control.Applicative     as X-import Control.Monad           as X (forM, forM_, void)-import Control.Arrow           as X (first, second)+import Control.Applicative as X+import Control.Arrow as X (first, second)+import Control.Monad as X (forM, forM_, void) import Crypto.Internal.DeepSeq as X
Crypto/Internal/Nat.hs view
@@ -33,7 +33,7 @@     IsLE bitlen n 'False = 'False #endif --- | ensure the given `bitlen` is lesser or equal to `n`+-- | ensure the given @bitlen@ is lesser or equal to @n@ -- type IsAtMost  (bitlen :: Nat) (n :: Nat) = IsLE bitlen n (bitlen <=? n) ~ 'True @@ -48,7 +48,7 @@     IsGE bitlen n 'False = 'False #endif --- | ensure the given `bitlen` is greater or equal to `n`+-- | ensure the given @bitlen@ is greater or equal to @n@ -- type IsAtLeast (bitlen :: Nat) (n :: Nat) = IsGE bitlen n (n <=? bitlen) ~ 'True @@ -208,6 +208,6 @@     Mod8 63 = 7     Mod8 n = Mod8 (n - 64) --- | ensure the given `bitlen` is divisible by 8+-- | ensure the given @bitlen@ is divisible by 8 -- type IsDivisibleBy8 bitLen = IsDiv8 bitLen bitLen ~ 'True
+ Crypto/Internal/Poly1305.hs view
@@ -0,0 +1,37 @@+{-# LANGUAGE GeneralizedNewtypeDeriving #-}++-- |+-- Module      : Crypto.Internal.Poly1305+-- License     : BSD-style+-- Maintainer  : Kazu Yamamoto <kazu@iij.ad.jp>+-- Stability   : experimental+-- Portability : unknown+--+-- The Poly1305 key with its constructor, for the modules here that build one+-- from bytes whose length they already know.  "Crypto.MAC.Poly1305" exports+-- the type without the constructor, so that outside this library a key can+-- only be made by 'key', which checks.+module Crypto.Internal.Poly1305 (+    Key (..),+    key,+) where++import Crypto.Error+import Crypto.Internal.ByteArray (ByteArrayAccess, ScrubbedBytes)+import qualified Crypto.Internal.ByteArray as B+import Crypto.Internal.DeepSeq++-- | A Poly1305 key: thirty-two bytes, and the length is checked here rather+-- than at every use.  'Crypto.MAC.Poly1305.initialize' and+-- 'Crypto.MAC.Poly1305.auth' take one of these and cannot fail, so a caller+-- that holds a key does not carry an error case for a length it already knows+-- is right.+newtype Key = Key ScrubbedBytes+    deriving (ByteArrayAccess, Eq, NFData)++-- | Take thirty-two bytes for a key.  A different length is reported as+-- 'CryptoError_MacKeyInvalid'; nothing else about a key can be wrong.+key :: ByteArrayAccess ba => ba -> CryptoFailable Key+key k+    | B.length k /= 32 = CryptoFailed CryptoError_MacKeyInvalid+    | otherwise = CryptoPassed $ Key $ B.convert k
Crypto/Internal/WordArray.hs view
@@ -1,3 +1,7 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE MagicHash #-}+{-# LANGUAGE UnboxedTuples #-}+ -- | -- Module      : Crypto.Internal.WordArray -- License     : BSD-style@@ -9,37 +13,32 @@ -- with limited safety for internal use. -- -- The array produced should never be exposed to the user directly.----{-# LANGUAGE BangPatterns #-}-{-# LANGUAGE MagicHash #-}-{-# LANGUAGE UnboxedTuples #-}-module Crypto.Internal.WordArray-    ( Array8-    , Array32-    , Array64-    , MutableArray32-    , array8-    , array32-    , array32FromAddrBE-    , allocArray32AndFreeze-    , mutableArray32-    , array64-    , arrayRead8-    , arrayRead32-    , arrayRead64-    , mutableArrayRead32-    , mutableArrayWrite32-    , mutableArrayWriteXor32-    , mutableArray32FromAddrBE-    , mutableArray32Freeze-    ) where+module Crypto.Internal.WordArray (+    Array8,+    Array32,+    Array64,+    MutableArray32,+    array8,+    array32,+    array32FromAddrBE,+    allocArray32AndFreeze,+    mutableArray32,+    array64,+    arrayRead8,+    arrayRead32,+    arrayRead64,+    mutableArrayRead32,+    mutableArrayWrite32,+    mutableArrayWriteXor32,+    mutableArray32FromAddrBE,+    mutableArray32Freeze,+) where -import Data.Word-import Data.Bits (xor) import Crypto.Internal.Compat import Crypto.Internal.CompatPrim-import GHC.Prim-import GHC.Types+import Data.Bits (xor)+import Data.Word+import GHC.Base import GHC.Word  -- | Array of Word8@@ -81,15 +80,15 @@     case newAlignedPinnedByteArray# (n *# 8#) 8# s of         (# s', mbarr #) -> loop 0# s' mbarr l   where-        loop _ st mb [] = freezeArray mb st-        loop i st mb ((W64# x):xs)-            | booleanPrim (i ==# n) = freezeArray mb st-            | otherwise =-                let !st' = writeWord64Array# mb i x st-                 in loop (i +# 1#) st' mb xs-        freezeArray mb st =-            case unsafeFreezeByteArray# mb st of-                (# st', b #) -> (# st', Array64 b #)+    loop _ st mb [] = freezeArray mb st+    loop i st mb ((W64# x) : xs)+        | booleanPrim (i ==# n) = freezeArray mb st+        | otherwise =+            let !st' = writeWord64Array# mb i x st+             in loop (i +# 1#) st' mb xs+    freezeArray mb st =+        case unsafeFreezeByteArray# mb st of+            (# st', b #) -> (# st', Array64 b #) {-# NOINLINE array64 #-}  -- | Create a Mutable Array of Word32 of specific size from a list of Word32@@ -98,12 +97,12 @@     case newAlignedPinnedByteArray# (n *# 4#) 4# s of         (# s', mbarr #) -> loop 0# s' mbarr l   where-        loop _ st mb [] = (# st, MutableArray32 mb #)-        loop i st mb ((W32# x):xs)-            | booleanPrim (i ==# n) = (# st, MutableArray32 mb #)-            | otherwise =-                let !st' = writeWord32Array# mb i x st-                 in loop (i +# 1#) st' mb xs+    loop _ st mb [] = (# st, MutableArray32 mb #)+    loop i st mb ((W32# x) : xs)+        | booleanPrim (i ==# n) = (# st, MutableArray32 mb #)+        | otherwise =+            let !st' = writeWord32Array# mb i x st+             in loop (i +# 1#) st' mb xs  -- | Create a Mutable Array of BE Word32 aliasing an Addr mutableArray32FromAddrBE :: Int -> Addr# -> IO MutableArray32@@ -111,11 +110,11 @@     case newAlignedPinnedByteArray# (n *# 4#) 4# s of         (# s', mbarr #) -> loop 0# s' mbarr   where-        loop i st mb-            | booleanPrim (i ==# n) = (# st, MutableArray32 mb #)-            | otherwise             =-                let !st' = writeWord32Array# mb i (be32Prim (indexWord32OffAddr# a i)) st-                 in loop (i +# 1#) st' mb+    loop i st mb+        | booleanPrim (i ==# n) = (# st, MutableArray32 mb #)+        | otherwise =+            let !st' = writeWord32Array# mb i (be32Prim (indexWord32OffAddr# a i)) st+             in loop (i +# 1#) st' mb  -- | freeze a Mutable Array of Word32 into a immutable Array of Word32 mutableArray32Freeze :: MutableArray32 -> IO Array32
Crypto/Internal/Words.hs view
@@ -6,16 +6,15 @@ -- Portability : unknown -- -- Extra Word size----module Crypto.Internal.Words-    ( Word128(..)-    , w64to32-    , w32to64-    ) where+module Crypto.Internal.Words (+    Word128 (..),+    w64to32,+    w32to64,+) where -import Data.Word import Data.Bits import Data.Memory.ExtendedWords+import Data.Word  -- | Split a 'Word64' into the highest and lowest 'Word32' w64to32 :: Word64 -> (Word32, Word32)
Crypto/KDF/Argon2.hs view
@@ -11,73 +11,77 @@ -- -- File started from Argon2.hs, from Oliver Charles -- at https://github.com/ocharles/argon2----module Crypto.KDF.Argon2-    (-      Options(..)-    , TimeCost-    , MemoryCost-    , Parallelism-    , Variant(..)-    , Version(..)-    , defaultOptions+module Crypto.KDF.Argon2 (+    Options (..),+    TimeCost,+    MemoryCost,+    Parallelism,+    Variant (..),+    Version (..),+    defaultOptions,+     -- * Hashing function-    , hash-    ) where+    hash,+) where -import           Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess)+import Crypto.Error+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Error-import           Control.Monad (when)-import           Data.Word-import           Foreign.C-import           Foreign.Ptr+import Crypto.Internal.Compat (unsafeDoIO)+import Data.Word+import Foreign.C+import Foreign.Ptr  -- | Which variant of Argon2 to use. You should choose the variant that is most -- applicable to your intention to hash inputs.-data Variant =-      Argon2d  -- ^ Argon2d is faster than Argon2i and uses data-depending memory access,-               -- which makes it suitable for cryptocurrencies and applications with no-               -- threats from side-channel timing attacks.-    | Argon2i  -- ^ Argon2i uses data-independent memory access, which is preferred-               -- for password hashing and password-based key derivation. Argon2i-               -- is slower as it makes more passes over the memory to protect from-               -- tradeoff attacks.-    | Argon2id -- ^ Argon2id is a hybrid of Argon2i and Argon2d, using a combination-               -- of data-depending and data-independent memory accesses, which gives-               -- some of Argon2i's resistance to side-channel cache timing attacks-               -- and much of Argon2d's resistance to GPU cracking attacks-    deriving (Eq,Ord,Read,Show,Enum,Bounded)+data Variant+    = -- | Argon2d is faster than Argon2i and uses data-depending memory access,+      -- which makes it suitable for cryptocurrencies and applications with no+      -- threats from side-channel timing attacks.+      Argon2d+    | -- | Argon2i uses data-independent memory access, which is preferred+      -- for password hashing and password-based key derivation. Argon2i+      -- is slower as it makes more passes over the memory to protect from+      -- tradeoff attacks.+      Argon2i+    | -- | Argon2id is a hybrid of Argon2i and Argon2d, using a combination+      -- of data-depending and data-independent memory accesses, which gives+      -- some of Argon2i's resistance to side-channel cache timing attacks+      -- and much of Argon2d's resistance to GPU cracking attacks+      Argon2id+    deriving (Eq, Ord, Read, Show, Enum, Bounded)  -- | Which version of Argon2 to use data Version = Version10 | Version13-    deriving (Eq,Ord,Read,Show,Enum,Bounded)+    deriving (Eq, Ord, Read, Show, Enum, Bounded)  -- | The time cost, which defines the amount of computation realized and therefore the execution time, given in number of iterations. ----- 'FFI.ARGON2_MIN_TIME' <= 'hashIterations' <= 'FFI.ARGON2_MAX_TIME'+-- 'FFI.ARGON2_MIN_TIME' <= 'iterations' <= 'FFI.ARGON2_MAX_TIME' type TimeCost = Word32  -- | The memory cost, which defines the memory usage, given in kibibytes. ----- max 'FFI.ARGON2_MIN_MEMORY' (8 * 'hashParallelism') <= 'hashMemory' <= 'FFI.ARGON2_MAX_MEMORY'+-- max 'FFI.ARGON2_MIN_MEMORY' (8 * 'parallelism') <= 'memory' <= 'FFI.ARGON2_MAX_MEMORY' type MemoryCost = Word32  -- | A parallelism degree, which defines the number of parallel threads. ----- 'FFI.ARGON2_MIN_LANES' <= 'hashParallelism' <= 'FFI.ARGON2_MAX_LANES' && 'FFI.ARGON_MIN_THREADS' <= 'hashParallelism' <= 'FFI.ARGON2_MAX_THREADS'+-- 'FFI.ARGON2_MIN_LANES' <= 'parallelism' <= 'FFI.ARGON2_MAX_LANES' && 'FFI.ARGON_MIN_THREADS' <= 'parallelism' <= 'FFI.ARGON2_MAX_THREADS' type Parallelism = Word32  -- | Parameters that can be adjusted to change the runtime performance of the -- hashing. data Options = Options-    { iterations  :: !TimeCost-    , memory      :: !MemoryCost+    { iterations :: !TimeCost+    , memory :: !MemoryCost     , parallelism :: !Parallelism-    , variant     :: !Variant     -- ^ Which variant of Argon2 to use.-    , version     :: !Version     -- ^ Which version of Argon2 to use.+    , variant :: !Variant+    -- ^ Which variant of Argon2 to use.+    , version :: !Version+    -- ^ Which version of Argon2 to use.     }-    deriving (Eq,Ord,Read,Show)+    deriving (Eq, Ord, Read, Show)  saltMinLength :: Int saltMinLength = 8@@ -92,38 +96,52 @@  defaultOptions :: Options defaultOptions =-    Options { iterations  = 1-            , memory      = 2 ^ (17 :: Int)-            , parallelism = 4-            , variant     = Argon2i-            , version     = Version13-            }+    Options+        { iterations = 1+        , memory = 2 ^ (17 :: Int)+        , parallelism = 4+        , variant = Argon2i+        , version = Version13+        } -hash :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out)-     => Options-     -> password-     -> salt-     -> Int-     -> CryptoFailable out+-- | Hash a password with Argon2.+--+-- Options the underlying implementation refuses -- iterations, memory or+-- parallelism outside the range it accepts -- are reported as+-- 'CryptoError_ParameterInvalid'.+hash+    :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out)+    => Options+    -> password+    -> salt+    -> Int+    -> CryptoFailable out hash options password salt outLen-    | saltLen < saltMinLength  = CryptoFailed CryptoError_SaltTooSmall+    | saltLen < saltMinLength = CryptoFailed CryptoError_SaltTooSmall     | outLen < outputMinLength = CryptoFailed CryptoError_OutputLengthTooSmall     | outLen > outputMaxLength = CryptoFailed CryptoError_OutputLengthTooBig-    | otherwise                = CryptoPassed $ B.allocAndFreeze outLen $ \out -> do-        res <- B.withByteArray password $ \pPass ->-               B.withByteArray salt     $ \pSalt ->-                    argon2_hash (iterations options)-                                (memory options)-                                (parallelism options)-                                pPass-                                (csizeOfInt passwordLen)-                                pSalt-                                (csizeOfInt saltLen)-                                out-                                (csizeOfInt outLen)-                                (cOfVariant $ variant options)-                                (cOfVersion $ version options)-        when (res /= 0) $ error "argon2: hash: internal error"+    | otherwise = unsafeDoIO $ do+        -- the bounds on iterations, memory and parallelism are checked by the+        -- C implementation, which reports them through its return code+        (res, out) <- B.allocRet outLen $ \pOut ->+            B.withByteArray password $ \pPass ->+                B.withByteArray salt $ \pSalt ->+                    argon2_hash+                        (iterations options)+                        (memory options)+                        (parallelism options)+                        pPass+                        (csizeOfInt passwordLen)+                        pSalt+                        (csizeOfInt saltLen)+                        pOut+                        (csizeOfInt outLen)+                        (cOfVariant $ variant options)+                        (cOfVersion $ version options)+        return $+            if res == 0+                then CryptoPassed out+                else CryptoFailed CryptoError_ParameterInvalid   where     saltLen = B.length salt     passwordLen = B.length password@@ -140,18 +158,24 @@ cOfVersion Version13 = 0x13  cOfVariant :: Variant -> CVariant-cOfVariant Argon2d  = 0-cOfVariant Argon2i  = 1+cOfVariant Argon2d = 0+cOfVariant Argon2i = 1 cOfVariant Argon2id = 2  csizeOfInt :: Int -> CSize csizeOfInt = fromIntegral  foreign import ccall unsafe "crypton_argon2_hash"-    argon2_hash :: Word32 -> Word32 -> Word32-                -> Ptr Pass -> CSize-                -> Ptr Salt -> CSize-                -> Ptr HashOut -> CSize-                -> CVariant-                -> CVersion-                -> IO CInt+    argon2_hash+        :: Word32+        -> Word32+        -> Word32+        -> Ptr Pass+        -> CSize+        -> Ptr Salt+        -> CSize+        -> Ptr HashOut+        -> CSize+        -> CVariant+        -> CVersion+        -> IO CInt
Crypto/KDF/BCrypt.hs view
@@ -1,4 +1,3 @@- -- | Password encoding and validation using bcrypt. -- -- Example usage:@@ -38,32 +37,43 @@ -- if passwords are UTF-8 encoded (which they should be) and less than 256 -- characters long. --+-- Only the first 72 bytes of a password are used.  The rest is silently+-- ignored, so two passwords sharing a 72-byte prefix produce the same hash and+-- validate against each other.  That is what the original implementation does+-- and is kept for compatibility, but it means a longer+-- passphrase buys nothing past that point, and the limit is on /bytes/ rather+-- than characters -- a UTF-8 passphrase reaches it sooner than its length in+-- characters suggests.  Where passwords may be longer, hash them to a fixed+-- size first, or use "Crypto.KDF.Argon2" or "Crypto.KDF.Scrypt", which have no+-- such limit.+-- -- The cost parameter can be between 4 and 31 inclusive, but anything less than -- 10 is probably not strong enough. High values may be prohibitively slow -- depending on your hardware. Choose the highest value you can without having -- an unacceptable impact on your users. The cost parameter can also be varied -- depending on the account, since it is unique to an individual hash.--module Crypto.KDF.BCrypt-    ( hashPassword-    , validatePassword-    , validatePasswordEither-    , bcrypt-    )+module Crypto.KDF.BCrypt (+    hashPassword,+    tryHashPassword,+    validatePassword,+    validatePasswordEither,+    bcrypt,+    tryBcrypt,+) where -import           Control.Monad                    (forM_, unless, when)-import           Crypto.Cipher.Blowfish.Primitive (Context, createKeySchedule,-                                                   encrypt, expandKey,-                                                   expandKeyWithSalt,-                                                   freezeKeySchedule)-import           Crypto.Internal.Compat-import           Crypto.Random                    (MonadRandom, getRandomBytes)-import           Data.ByteArray                   (ByteArray, ByteArrayAccess,-                                                   Bytes)-import qualified Data.ByteArray                   as B-import           Data.ByteArray.Encoding-import           Data.Char+import Control.Monad (unless, when)+import Crypto.Cipher.Blowfish.Primitive (bcryptHash)+import Crypto.Error+import Crypto.Random (MonadRandom, getRandomBytes)+import Data.ByteArray (+    ByteArray,+    ByteArrayAccess,+    Bytes,+ )+import qualified Data.ByteArray as B+import Data.ByteArray.Encoding+import Data.Char  data BCryptHash = BCH Char Int Bytes Bytes @@ -73,117 +83,156 @@ -- -- Each increment of the cost approximately doubles the time taken. -- The 16 bytes of random salt will be generated internally.-hashPassword :: (MonadRandom m, ByteArray password, ByteArray hash)-             => Int-             -- ^ The cost parameter. Should be between 4 and 31 (inclusive).-             -- Values which lie outside this range will be adjusted accordingly.-             -> password-             -- ^ The password. Should be the UTF-8 encoded bytes of the password text.-             -> m hash-             -- ^ The bcrypt hash in standard format.-hashPassword cost password = do+--+-- A cost outside 4 to 31 raises 'CryptoError_ParameterInvalid';+-- 'tryHashPassword' reports it instead.+hashPassword+    :: (MonadRandom m, ByteArray password, ByteArray hash)+    => Int+    -- ^ The cost parameter. Must be between 4 and 31 inclusive; anything+    -- else is refused.+    -> password+    -- ^ The password. Should be the UTF-8 encoded bytes of the password text.+    -- Only the first 72 bytes are used; see the module documentation.+    -> m hash+    -- ^ The bcrypt hash in standard format.+hashPassword cost password = throwCryptoError <$> tryHashPassword cost password++-- | Create a bcrypt hash for a password with a provided cost value,+-- reporting a cost the implementation refuses rather than raising.+--+-- The salt is generated internally and is always the right length, so the+-- cost is the only thing here that can be wrong.+tryHashPassword+    :: (MonadRandom m, ByteArray password, ByteArray hash)+    => Int+    -- ^ The cost parameter. Must be between 4 and 31 inclusive; anything+    -- else is reported.+    -> password+    -- ^ The password. Should be the UTF-8 encoded bytes of the password text.+    -- Only the first 72 bytes are used; see the module documentation.+    -> m (CryptoFailable hash)+    -- ^ The bcrypt hash in standard format.+tryHashPassword cost password = do     salt <- getRandomBytes 16-    return $ bcrypt cost (salt :: Bytes) password+    return $ tryBcrypt cost (salt :: Bytes) password  -- | Create a bcrypt hash for a password with a provided cost value and salt. ----- Cost value under 4 will be automatically adjusted back to 10 for safety reason.-bcrypt :: (ByteArray salt, ByteArray password, ByteArray output)-       => Int-       -- ^ The cost parameter. Should be between 4 and 31 (inclusive).-       -- Values which lie outside this range will be adjusted accordingly.-       -> salt-       -- ^ The salt. Must be 16 bytes in length or an error will be raised.-       -> password-       -- ^ The password. Should be the UTF-8 encoded bytes of the password text.-       -> output-       -- ^ The bcrypt hash in standard format.-bcrypt cost salt password = B.concat [header, B.snoc costBytes dollar, b64 salt, b64 hash]+-- A cost outside 4 to 31, or a salt that is not 16 bytes long, raises+-- 'CryptoError_ParameterInvalid'; 'tryBcrypt' reports the same conditions as+-- 'CryptoFailed'.+bcrypt+    :: (ByteArray salt, ByteArray password, ByteArray output)+    => Int+    -- ^ The cost parameter. Must be between 4 and 31 inclusive; anything+    -- else is refused.+    -> salt+    -- ^ The salt. Must be 16 bytes in length or an error will be raised.+    -> password+    -- ^ The password. Should be the UTF-8 encoded bytes of the password text.+    -- Only the first 72 bytes are used; see the module documentation.+    -> output+    -- ^ The bcrypt hash in standard format.+bcrypt cost salt password = throwCryptoError (tryBcrypt cost salt password)++-- | Create a bcrypt hash for a password with a provided cost value and salt,+-- reporting a parameter the implementation refuses rather than raising.+--+-- bcrypt is defined for a cost of 4 to 31, and a cost outside that is+-- reported rather than replaced by one inside it: a caller that asks for+-- something this does not do should hear so, not receive a hash at a cost it+-- did not choose.+tryBcrypt+    :: (ByteArray salt, ByteArray password, ByteArray output)+    => Int+    -- ^ The cost parameter. Must be between 4 and 31 inclusive; anything+    -- else is refused.+    -> salt+    -- ^ The salt. Must be 16 bytes in length.+    -> password+    -- ^ The password. Should be the UTF-8 encoded bytes of the password text.+    -- Only the first 72 bytes are used; see the module documentation.+    -> CryptoFailable output+    -- ^ The bcrypt hash in standard format.+tryBcrypt cost salt password+    | cost < 4 || cost > 31 = CryptoFailed CryptoError_ParameterInvalid+    | B.length salt /= 16 = CryptoFailed CryptoError_ParameterInvalid+    | otherwise =+        CryptoPassed $+            B.concat [header, B.snoc costBytes dollar, b64 salt, b64 hash]   where-    hash   = rawHash 'b' realCost salt password+    hash = rawHash 'b' cost salt password     header = B.pack [dollar, fromIntegral (ord '2'), fromIntegral (ord 'b'), dollar]     dollar = fromIntegral (ord '$')-    zero   = fromIntegral (ord '0')-    costBytes  = B.pack [zero + fromIntegral (realCost `div` 10), zero + fromIntegral (realCost `mod` 10)]-    realCost-        | cost < 4  = 10 -- 4 is virtually pointless so go for 10-        | cost > 31 = 31-        | otherwise = cost+    zero = fromIntegral (ord '0')+    costBytes =+        B.pack+            [ zero + fromIntegral (cost `div` 10)+            , zero + fromIntegral (cost `mod` 10)+            ] -    b64 :: (ByteArray ba) => ba -> ba+    b64 :: ByteArray ba => ba -> ba     b64 = convertToBase Base64OpenBSD  -- | Check a password against a stored bcrypt hash when authenticating a user. -- -- Returns @False@ if the password doesn't match the hash, or if the hash is -- invalid or an unsupported version.-validatePassword :: (ByteArray password, ByteArray hash) => password -> hash -> Bool+--+-- Only the first 72 bytes of the password are compared; see the module+-- documentation.+validatePassword+    :: (ByteArray password, ByteArray hash) => password -> hash -> Bool validatePassword password bcHash = either (const False) id (validatePasswordEither password bcHash)  -- | Check a password against a bcrypt hash -- -- As for @validatePassword@ but will provide error information if the hash is invalid or--- an unsupported version.-validatePasswordEither :: (ByteArray password, ByteArray hash) => password -> hash -> Either String Bool+-- an unsupported version.  The same 72-byte limit applies.+validatePasswordEither+    :: (ByteArray password, ByteArray hash) => password -> hash -> Either String Bool validatePasswordEither password bcHash = do     BCH version cost salt hash <- parseBCryptHash bcHash     return $ (rawHash version cost salt password :: Bytes) `B.constEq` hash -rawHash :: (ByteArrayAccess salt, ByteArray password, ByteArray output) => Char -> Int -> salt -> password -> output-rawHash _ cost salt password = B.take 23 hash -- Another compatibility bug. Ignore last byte of hash+rawHash+    :: (ByteArrayAccess salt, ByteArray password, ByteArray output)+    => Char -> Int -> salt -> password -> output+rawHash _ cost salt password = case bcryptHash cost salt key of+    Just hash -> B.take 23 hash -- Another compatibility bug. Ignore last byte of hash+    Nothing -> error "bcrypt: the cost or the salt is not one bcrypt takes"   where-    hash = loop (0 :: Int) orpheanBeholder--    loop i input-        | i < 64    = loop (i+1) (encrypt ctx input)-        | otherwise = input-     -- Truncate the password if necessary and append a null byte for C compatibility-    key = B.snoc (B.take 72 password) 0--    ctx = expensiveBlowfishContext key salt cost--    -- The BCrypt plaintext: "OrpheanBeholderScryDoubt"-    orpheanBeholder = B.pack [79,114,112,104,101,97,110,66,101,104,111,108,100,101,114,83,99,114,121,68,111,117,98,116]+    key = B.snoc (B.take 72 (B.convert password :: Bytes)) 0  -- "$2a$10$XajjQvNhvvRt5GSeFk1xFeyqRrsxkhBkUiQeg0dt.wU1qD4aFDcga"-parseBCryptHash :: (ByteArray ba) => ba -> Either String BCryptHash+parseBCryptHash :: ByteArray ba => ba -> Either String BCryptHash parseBCryptHash bc = do-    unless (B.length bc == 60      &&-            B.index bc 0 == dollar &&-            B.index bc 1 == fromIntegral (ord '2') &&-            B.index bc 3 == dollar &&-            B.index bc 6 == dollar) (Left "Invalid hash format")-    unless (version == 'b' || version == 'a' || version == 'y') (Left ("Unsupported minor version: " ++ [version]))-    when (costTens > 3 || cost > 31 || cost < 4)  (Left "Invalid bcrypt cost")+    unless+        ( B.length bc == 60+            && B.index bc 0 == dollar+            && B.index bc 1 == fromIntegral (ord '2')+            && B.index bc 3 == dollar+            && B.index bc 6 == dollar+        )+        (Left "Invalid hash format")+    unless+        (version == 'b' || version == 'a' || version == 'y')+        (Left ("Unsupported minor version: " ++ [version]))+    when (costTens > 3 || cost > 31 || cost < 4) (Left "Invalid bcrypt cost")     (salt, hash) <- decodeSaltHash (B.drop 7 bc)     return (BCH version cost salt hash)   where-    dollar    = fromIntegral (ord '$')-    zero      = ord '0'-    costTens  = fromIntegral (B.index bc 4) - zero+    dollar = fromIntegral (ord '$')+    zero = ord '0'+    costTens = fromIntegral (B.index bc 4) - zero     costUnits = fromIntegral (B.index bc 5) - zero-    version   = chr (fromIntegral (B.index bc 2))-    cost      = costUnits + 10*costTens :: Int+    version = chr (fromIntegral (B.index bc 2))+    cost = costUnits + 10 * costTens :: Int      decodeSaltHash saltHash = do         let (s, h) = B.splitAt 22 saltHash         salt <- convertFromBase Base64OpenBSD s         hash <- convertFromBase Base64OpenBSD h         return (salt, hash)---- | Create a key schedule for the BCrypt "EKS" version.------ Salt must be a 128-bit byte array.--- Cost must be between 4 and 31 inclusive--- See <https://www.usenix.org/conference/1999-usenix-annual-technical-conference/future-adaptable-password-scheme>-expensiveBlowfishContext :: (ByteArrayAccess key, ByteArrayAccess salt) => key-> salt -> Int -> Context-expensiveBlowfishContext keyBytes saltBytes cost-  | B.length saltBytes /= 16 = error "bcrypt salt must be 16 bytes"-  | otherwise = unsafeDoIO $ do-        ks <- createKeySchedule-        expandKeyWithSalt ks keyBytes saltBytes-        forM_ [1..2^cost :: Int] $ \_ -> do-            expandKey ks keyBytes-            expandKey ks saltBytes-        freezeKeySchedule ks
Crypto/KDF/BCryptPBKDF.hs view
@@ -6,182 +6,187 @@ -- -- Port of the bcrypt_pbkdf key derivation function from OpenBSD -- as described at <http://man.openbsd.org/bcrypt_pbkdf.3>.-module Crypto.KDF.BCryptPBKDF-    ( Parameters (..)-    , generate-    , hashInternal-    )+module Crypto.KDF.BCryptPBKDF (+    Parameters (..),+    generate,+    tryGenerate,+    hashInternal,+    tryHashInternal,+) where -import           Basement.Block                   (MutableBlock)-import qualified Basement.Block                   as Block-import qualified Basement.Block.Mutable           as Block-import           Basement.Monad                   (PrimState)-import           Basement.Types.OffsetSize        (CountOf (..), Offset (..))-import           Control.Exception                (finally)-import           Control.Monad                    (when)-import qualified Crypto.Cipher.Blowfish.Box       as Blowfish-import qualified Crypto.Cipher.Blowfish.Primitive as Blowfish-import           Crypto.Hash.Algorithms           (SHA512 (..))-import           Crypto.Hash.Types                (Context,-                                                   hashDigestSize,-                                                   hashInternalContextSize,-                                                   hashInternalFinalize,-                                                   hashInternalInit,-                                                   hashInternalUpdate)-import           Crypto.Internal.Compat           (unsafeDoIO)-import           Data.Bits-import qualified Data.ByteArray                   as B-import           Data.Foldable                    (forM_)-import           Data.Memory.PtrMethods           (memCopy, memSet, memXor)-import           Data.Word-import           Foreign.Ptr                      (Ptr, castPtr)-import           Foreign.Storable                 (peekByteOff, pokeByteOff)+import qualified Control.Exception as E+import Control.Monad (when)+import Crypto.Cipher.Blowfish.Primitive (bcryptPbkdfHash)+import Crypto.Error+import Crypto.Hash.Algorithms (SHA512 (..))+import Crypto.Hash.Types (+    Context,+    hashDigestSize,+    hashInternalContextSize,+    hashInternalFinalize,+    hashInternalInit,+    hashInternalUpdate,+ )+import Crypto.Internal.Compat (unsafeDoIO)+import Data.Bits+import qualified Data.ByteArray as B+import qualified Data.ByteString.Internal as BSI+import Data.Foldable (forM_)+import Data.Memory.PtrMethods (memCopy, memSet, memXor)+import Data.Word+import Foreign.ForeignPtr (ForeignPtr, mallocForeignPtrBytes, withForeignPtr)+import Foreign.Ptr (Ptr, castPtr)+import Foreign.Storable (peekByteOff, pokeByteOff)  data Parameters = Parameters-  { iterCounts   :: Int -- ^ The number of user-defined iterations for the algorithm-                        --   (must be > 0)-  , outputLength :: Int -- ^ The number of bytes to generate out of BCryptPBKDF-                        --   (must be in 1..1024)-  } deriving (Eq, Ord, Show)+    { iterCounts :: Int+    -- ^ The number of user-defined iterations for the algorithm+    --   (must be > 0)+    , outputLength :: Int+    -- ^ The number of bytes to generate out of BCryptPBKDF+    --   (must be in 1..1024)+    }+    deriving (Eq, Ord, Show)  -- | Derive a key of specified length using the bcrypt_pbkdf algorithm.-generate :: (B.ByteArray pass, B.ByteArray salt, B.ByteArray output)-       => Parameters-       -> pass-       -> salt-       -> output-generate params pass salt-    | iterCounts params < 1       = error "BCryptPBKDF: iterCounts must be > 0"-    | keyLen < 1 || keyLen > 1024 = error "BCryptPBKDF: outputLength must be in 1..1024"-    | otherwise                   = B.unsafeCreate keyLen deriveKey+--+-- Parameters outside the ranges documented for t'Parameters' raise+-- 'CryptoError_ParameterInvalid'; 'tryGenerate' reports the same condition as+-- 'CryptoFailed'.+generate+    :: (B.ByteArray pass, B.ByteArray salt, B.ByteArray output)+    => Parameters+    -> pass+    -> salt+    -> output+generate params pass salt = throwCryptoError (tryGenerate params pass salt)++-- | Derive a key of specified length using the bcrypt_pbkdf algorithm,+-- reporting parameters the implementation refuses rather than raising.+tryGenerate+    :: (B.ByteArray pass, B.ByteArray salt, B.ByteArray output)+    => Parameters+    -> pass+    -> salt+    -> CryptoFailable output+tryGenerate params pass salt+    | iterCounts params < 1 = CryptoFailed CryptoError_ParameterInvalid+    | keyLen < 1 || keyLen > 1024 = CryptoFailed CryptoError_ParameterInvalid+    | otherwise = CryptoPassed $ B.unsafeCreate keyLen deriveKey   where     outLen, tmpLen, blkLen, keyLen, passLen, saltLen, ctxLen, hashLen, blocks :: Int-    outLen  = 32-    tmpLen  = 32-    blkLen  = 4+    outLen = 32+    tmpLen = 32+    blkLen = 4     passLen = B.length pass     saltLen = B.length salt-    keyLen  = outputLength params-    ctxLen  = hashInternalContextSize SHA512+    keyLen = outputLength params+    ctxLen = hashInternalContextSize SHA512     hashLen = hashDigestSize SHA512 -- 64-    blocks  = (keyLen + outLen - 1) `div` outLen+    blocks = (keyLen + outLen - 1) `div` outLen      deriveKey :: Ptr Word8 -> IO ()     deriveKey keyPtr = do-        -- Allocate all necessary memory. The algorihm shall not allocate-        -- any more dynamic memory after this point. Blocks need to be pinned-        -- as pointers to them are passed to the SHA512 implementation.-        ksClean        <- Blowfish.createKeySchedule-        ksDirty        <- Blowfish.createKeySchedule-        ctxMBlock      <- Block.newPinned (CountOf ctxLen  :: CountOf Word8)-        outMBlock      <- Block.newPinned (CountOf outLen  :: CountOf Word8)-        tmpMBlock      <- Block.newPinned (CountOf tmpLen  :: CountOf Word8)-        blkMBlock      <- Block.newPinned (CountOf blkLen  :: CountOf Word8)-        passHashMBlock <- Block.newPinned (CountOf hashLen :: CountOf Word8)-        saltHashMBlock <- Block.newPinned (CountOf hashLen :: CountOf Word8)+        -- Allocate all necessary memory. The algorithm shall not allocate+        -- any more dynamic memory after this point. ForeignPtrs allocate+        -- pinned memory, so raw pointers to them are stable.+        ctxFP <- mallocForeignPtrBytes ctxLen :: IO (ForeignPtr Word8)+        outFP <- mallocForeignPtrBytes outLen :: IO (ForeignPtr Word8)+        tmpFP <- mallocForeignPtrBytes tmpLen :: IO (ForeignPtr Word8)+        blkFP <- mallocForeignPtrBytes blkLen :: IO (ForeignPtr Word8)+        passHashFP <- mallocForeignPtrBytes hashLen :: IO (ForeignPtr Word8)+        saltHashFP <- mallocForeignPtrBytes hashLen :: IO (ForeignPtr Word8)         -- Finally erase all memory areas that contain information from         -- which the derived key could be reconstructed.-        -- As all MutableBlocks are pinned it shall be guaranteed that-        -- no temporary trampoline buffers are allocated.-        finallyErase outMBlock $ finallyErase passHashMBlock $-            B.withByteArray pass                $ \passPtr->-            B.withByteArray salt                $ \saltPtr->-            Block.withMutablePtr ctxMBlock      $ \ctxPtr->-            Block.withMutablePtr outMBlock      $ \outPtr->-            Block.withMutablePtr tmpMBlock      $ \tmpPtr->-            Block.withMutablePtr blkMBlock      $ \blkPtr->-            Block.withMutablePtr passHashMBlock $ \passHashPtr->-            Block.withMutablePtr saltHashMBlock $ \saltHashPtr-> do-                -- Hash the password.-                let shaPtr = castPtr ctxPtr :: Ptr (Context SHA512)-                hashInternalInit     shaPtr-                hashInternalUpdate   shaPtr passPtr (fromIntegral passLen)-                hashInternalFinalize shaPtr (castPtr passHashPtr)-                passHashBlock <- Block.unsafeFreeze passHashMBlock-                forM_ [1..blocks] $ \block-> do-                    -- Poke the increased block counter.-                    Block.unsafeWrite blkMBlock 0 (fromIntegral $ block `shiftR` 24)-                    Block.unsafeWrite blkMBlock 1 (fromIntegral $ block `shiftR` 16)-                    Block.unsafeWrite blkMBlock 2 (fromIntegral $ block `shiftR`  8)-                    Block.unsafeWrite blkMBlock 3 (fromIntegral $ block `shiftR`  0)-                    -- First round (slightly different).-                    hashInternalInit     shaPtr-                    hashInternalUpdate   shaPtr saltPtr (fromIntegral saltLen)-                    hashInternalUpdate   shaPtr blkPtr  (fromIntegral blkLen)-                    hashInternalFinalize shaPtr (castPtr saltHashPtr)-                    Block.unsafeFreeze saltHashMBlock >>= \x-> do-                        Blowfish.copyKeySchedule ksDirty ksClean-                        hashInternalMutable ksDirty passHashBlock x tmpMBlock-                    memCopy outPtr tmpPtr outLen-                    -- Remaining rounds.-                    forM_ [2..iterCounts params] $ const $ do-                        hashInternalInit     shaPtr-                        hashInternalUpdate   shaPtr tmpPtr (fromIntegral tmpLen)-                        hashInternalFinalize shaPtr (castPtr saltHashPtr)-                        Block.unsafeFreeze saltHashMBlock >>= \x-> do-                            Blowfish.copyKeySchedule ksDirty ksClean-                            hashInternalMutable ksDirty passHashBlock x tmpMBlock-                        memXor outPtr outPtr tmpPtr outLen-                    -- Spread the current out buffer evenly over the key buffer.-                    -- After both loops have run every byte of the key buffer-                    -- will have been written to exactly once and every byte-                    -- of the output will have been used.-                    forM_ [0..outLen - 1] $ \outIdx-> do-                        let keyIdx = outIdx * blocks + block - 1-                        when (keyIdx < keyLen) $ do-                            w8 <- peekByteOff outPtr outIdx :: IO Word8-                            pokeByteOff keyPtr keyIdx w8+        finallyErase outFP outLen $+            finallyErase passHashFP hashLen $+                B.withByteArray pass $ \passPtr ->+                    B.withByteArray salt $ \saltPtr ->+                        withForeignPtr ctxFP $ \ctxPtr' ->+                            withForeignPtr outFP $ \outPtr ->+                                withForeignPtr tmpFP $ \tmpPtr ->+                                    withForeignPtr blkFP $ \blkPtr ->+                                        withForeignPtr passHashFP $ \passHashPtr ->+                                            withForeignPtr saltHashFP $ \saltHashPtr -> do+                                                -- Hash the password.+                                                let shaPtr = castPtr ctxPtr' :: Ptr (Context SHA512)+                                                hashInternalInit shaPtr+                                                hashInternalUpdate shaPtr passPtr (fromIntegral passLen)+                                                hashInternalFinalize shaPtr (castPtr passHashPtr)+                                                -- Create a stable ByteString view of the password hash+                                                -- (passHashFP is not modified after this point).+                                                let passHashBS = BSI.fromForeignPtr passHashFP 0 hashLen+                                                forM_ [1 .. blocks] $ \block -> do+                                                    -- Poke the increased block counter.+                                                    pokeByteOff blkPtr 0 (fromIntegral (block `shiftR` 24) :: Word8)+                                                    pokeByteOff blkPtr 1 (fromIntegral (block `shiftR` 16) :: Word8)+                                                    pokeByteOff blkPtr 2 (fromIntegral (block `shiftR` 8) :: Word8)+                                                    pokeByteOff blkPtr 3 (fromIntegral (block `shiftR` 0 :: Int) :: Word8)+                                                    -- First round (slightly different).+                                                    hashInternalInit shaPtr+                                                    hashInternalUpdate shaPtr saltPtr (fromIntegral saltLen)+                                                    hashInternalUpdate shaPtr blkPtr (fromIntegral blkLen)+                                                    hashInternalFinalize shaPtr (castPtr saltHashPtr)+                                                    let saltHashBS = BSI.fromForeignPtr saltHashFP 0 hashLen+                                                    hashInternalMutable passHashBS saltHashBS tmpPtr+                                                    memCopy outPtr tmpPtr outLen+                                                    -- Remaining rounds.+                                                    forM_ [2 .. iterCounts params] $ const $ do+                                                        hashInternalInit shaPtr+                                                        hashInternalUpdate shaPtr tmpPtr (fromIntegral tmpLen)+                                                        hashInternalFinalize shaPtr (castPtr saltHashPtr)+                                                        let saltHashBS2 = BSI.fromForeignPtr saltHashFP 0 hashLen+                                                        hashInternalMutable passHashBS saltHashBS2 tmpPtr+                                                        memXor outPtr outPtr tmpPtr outLen+                                                    -- Spread the current out buffer evenly over the key buffer.+                                                    -- After both loops have run every byte of the key buffer+                                                    -- will have been written to exactly once and every byte+                                                    -- of the output will have been used.+                                                    forM_ [0 .. outLen - 1] $ \outIdx -> do+                                                        let keyIdx = outIdx * blocks + block - 1+                                                        when (keyIdx < keyLen) $ do+                                                            w8 <- peekByteOff outPtr outIdx :: IO Word8+                                                            pokeByteOff keyPtr keyIdx w8  -- | Internal hash function used by `generate`. -- -- Normal users should not need this.-hashInternal :: (B.ByteArrayAccess pass, B.ByteArrayAccess salt, B.ByteArray output)+--+-- Inputs that are not 512 bits long raise 'CryptoError_ParameterInvalid';+-- 'tryHashInternal' reports the same condition as 'CryptoFailed'.+hashInternal+    :: (B.ByteArrayAccess pass, B.ByteArrayAccess salt, B.ByteArray output)     => pass     -> salt     -> output-hashInternal passHash saltHash-    | B.length passHash /= 64 = error "passHash must be 512 bits"-    | B.length saltHash /= 64 = error "saltHash must be 512 bits"-    | otherwise = unsafeDoIO $ do-        ks0 <- Blowfish.createKeySchedule-        outMBlock <- Block.newPinned 32-        hashInternalMutable ks0 passHash saltHash outMBlock-        B.convert `fmap` Block.freeze outMBlock+hashInternal passHash saltHash =+    throwCryptoError (tryHashInternal passHash saltHash) -hashInternalMutable :: (B.ByteArrayAccess pass, B.ByteArrayAccess salt)-    => Blowfish.KeySchedule-    -> pass+-- | Internal hash function used by 'tryGenerate', reporting inputs the+-- implementation refuses rather than raising.+--+-- Normal users should not need this.+tryHashInternal+    :: (B.ByteArrayAccess pass, B.ByteArrayAccess salt, B.ByteArray output)+    => pass     -> salt-    -> MutableBlock Word8 (PrimState IO)+    -> CryptoFailable output+tryHashInternal passHash saltHash+    | B.length passHash /= 64 = CryptoFailed CryptoError_ParameterInvalid+    | B.length saltHash /= 64 = CryptoFailed CryptoError_ParameterInvalid+    | otherwise = CryptoPassed $ unsafeDoIO $ do+        B.alloc 32 $ \outPtr -> hashInternalMutable passHash saltHash outPtr++hashInternalMutable+    :: (B.ByteArrayAccess pass, B.ByteArrayAccess salt)+    => pass+    -> salt+    -> Ptr Word8     -> IO ()-hashInternalMutable bfks passHash saltHash outMBlock = do-    Blowfish.expandKeyWithSalt bfks passHash saltHash-    forM_ [0..63 :: Int] $ const $ do-        Blowfish.expandKey bfks saltHash-        Blowfish.expandKey bfks passHash-    -- "OxychromaticBlowfishSwatDynamite" represented as 4 Word64 in big-endian.-    store  0 =<< cipher 64 0x4f78796368726f6d-    store  8 =<< cipher 64 0x61746963426c6f77-    store 16 =<< cipher 64 0x6669736853776174-    store 24 =<< cipher 64 0x44796e616d697465-    where-        store :: Offset Word8 -> Word64 -> IO ()-        store o w64 = do-            Block.unsafeWrite outMBlock (o + 0) (fromIntegral $ w64 `shiftR` 32)-            Block.unsafeWrite outMBlock (o + 1) (fromIntegral $ w64 `shiftR` 40)-            Block.unsafeWrite outMBlock (o + 2) (fromIntegral $ w64 `shiftR` 48)-            Block.unsafeWrite outMBlock (o + 3) (fromIntegral $ w64 `shiftR` 56)-            Block.unsafeWrite outMBlock (o + 4) (fromIntegral $ w64 `shiftR`  0)-            Block.unsafeWrite outMBlock (o + 5) (fromIntegral $ w64 `shiftR`  8)-            Block.unsafeWrite outMBlock (o + 6) (fromIntegral $ w64 `shiftR` 16)-            Block.unsafeWrite outMBlock (o + 7) (fromIntegral $ w64 `shiftR` 24)-        cipher :: Int -> Word64 -> IO Word64-        cipher 0 block = return block-        cipher i block = Blowfish.cipherBlockMutable bfks block >>= cipher (i - 1)+hashInternalMutable passHash saltHash outPtr =+    bcryptPbkdfHash passHash saltHash outPtr -finallyErase :: MutableBlock Word8 (PrimState IO) -> IO () -> IO ()-finallyErase mblock action =-    action `finally` Block.withMutablePtr mblock (\ptr-> memSet ptr 0 len)-    where-        CountOf len = Block.mutableLengthBytes mblock+finallyErase :: ForeignPtr Word8 -> Int -> IO () -> IO ()+finallyErase fp len action =+    action `E.finally` withForeignPtr fp (\ptr -> memSet ptr 0 len)
Crypto/KDF/HKDF.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE ScopedTypeVariables #-}+ -- | -- Module      : Crypto.KDF.HKDF -- License     : BSD-style@@ -8,77 +11,122 @@ -- Key Derivation Function based on HMAC -- -- See RFC5869----{-# LANGUAGE BangPatterns #-}-module Crypto.KDF.HKDF-    ( PRK-    , extract-    , extractSkip-    , expand-    ) where+module Crypto.KDF.HKDF (+    PRK,+    extract,+    extractSkip,+    expand,+    tryExpand,+    toPRK,+) where -import           Data.Word-import           Crypto.Hash-import           Crypto.MAC.HMAC-import           Crypto.Internal.ByteArray (ScrubbedBytes, ByteArray, ByteArrayAccess)+import Crypto.Error+import Crypto.Hash+import Crypto.Internal.ByteArray (+    ByteArray,+    ByteArrayAccess,+    ScrubbedBytes,+ ) import qualified Crypto.Internal.ByteArray as B+import Crypto.MAC.HMAC+import Data.Word  -- | Pseudo Random Key data PRK a = PRK (HMAC a) | PRK_NoExpand ScrubbedBytes     deriving (Eq)  instance ByteArrayAccess (PRK a) where-    length (PRK hm)          = B.length hm+    length (PRK hm) = B.length hm     length (PRK_NoExpand sb) = B.length sb-    withByteArray (PRK hm)          = B.withByteArray hm+    withByteArray (PRK hm) = B.withByteArray hm     withByteArray (PRK_NoExpand sb) = B.withByteArray sb  -- | Extract a Pseudo Random Key using the parameter and the underlaying hash mechanism-extract :: (HashAlgorithm a, ByteArrayAccess salt, ByteArrayAccess ikm)-        => salt  -- ^ Salt-        -> ikm   -- ^ Input Keying Material-        -> PRK a -- ^ Pseudo random key+extract+    :: (HashAlgorithm a, ByteArrayAccess salt, ByteArrayAccess ikm)+    => salt+    -- ^ Salt+    -> ikm+    -- ^ Input Keying Material+    -> PRK a+    -- ^ Pseudo random key extract salt ikm = PRK $ hmac salt ikm  -- | Create a PRK directly from the input key material. -- -- Only use when guaranteed to have a good quality and random data to use directly as key. -- This effectively skip a HMAC with key=salt and data=key.-extractSkip :: ByteArrayAccess ikm-            => ikm-            -> PRK a+extractSkip+    :: ByteArrayAccess ikm+    => ikm+    -> PRK a extractSkip ikm = PRK_NoExpand $ B.convert ikm  -- | Expand key material of specific length out of the parameters-expand :: (HashAlgorithm a, ByteArrayAccess info, ByteArray out)-       => PRK a      -- ^ Pseudo Random Key-       -> info       -- ^ Optional context and application specific information-       -> Int        -- ^ Output length in bytes-       -> out        -- ^ Output data+--+-- Requests exceeding the RFC 5869 limit of @255 * HashLen@ raise+-- 'CryptoError_OutputLengthTooBig'; 'tryExpand' reports the same condition as+-- 'CryptoFailed'.+expand+    :: forall a info out+     . (HashAlgorithm a, ByteArrayAccess info, ByteArray out)+    => PRK a+    -- ^ Pseudo Random Key+    -> info+    -- ^ Optional context and application specific information+    -> Int+    -- ^ Output length in bytes+    -> out+    -- ^ Output data expand prkAt infoAt outputLength =-    let hF = hFGet prkAt-     in B.concat $ loop hF B.empty outputLength 1+    throwCryptoError (tryExpand prkAt infoAt outputLength)++-- | Expand key material of specific length out of the parameters, reporting a+-- length the RFC refuses rather than raising.+tryExpand+    :: forall a info out+     . (HashAlgorithm a, ByteArrayAccess info, ByteArray out)+    => PRK a+    -- ^ Pseudo Random Key+    -> info+    -- ^ Optional context and application specific information+    -> Int+    -- ^ Output length in bytes+    -> CryptoFailable out+    -- ^ Output data+tryExpand prkAt infoAt outputLength+    | outputLength > 255 * hashDigestSize (undefined :: a) =+        CryptoFailed CryptoError_OutputLengthTooBig+    | otherwise =+        let hF = hFGet prkAt+         in CryptoPassed $ B.concat $ loop hF B.empty outputLength 1   where     hFGet :: (HashAlgorithm a, ByteArrayAccess b) => PRK a -> (b -> HMAC a)     hFGet prk = case prk of-             PRK hmacKey      -> hmac hmacKey-             PRK_NoExpand ikm -> hmac ikm+        PRK hmacKey -> hmac hmacKey+        PRK_NoExpand ikm -> hmac ikm      info :: ScrubbedBytes     info = B.convert infoAt -    loop :: HashAlgorithm a-         => (ScrubbedBytes -> HMAC a)-         -> ScrubbedBytes-         -> Int-         -> Word8-         -> [ScrubbedBytes]+    loop+        :: HashAlgorithm a+        => (ScrubbedBytes -> HMAC a)+        -> ScrubbedBytes+        -> Int+        -> Word8+        -> [ScrubbedBytes]     loop hF tim1 n i-        | n <= 0    = []+        | n <= 0 = []         | otherwise =-            let input   = B.concat [tim1,info,B.singleton i] :: ScrubbedBytes-                ti      = B.convert $ hF input+            let input = B.concat [tim1, info, B.singleton i] :: ScrubbedBytes+                ti = B.convert $ hF input                 hashLen = B.length ti-                r       = n - hashLen+                r = n - hashLen              in (if n >= hashLen then ti else B.take n ti)-              : loop hF ti r (i+1)+                    : loop hF ti r (i + 1)++toPRK :: (HashAlgorithm a, ByteArrayAccess ba) => ba -> Maybe (PRK a)+toPRK bs = case digestFromByteString bs of+    Nothing -> Nothing+    Just digest -> Just $ PRK $ HMAC digest
Crypto/KDF/PBKDF2.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE ForeignFunctionInterface #-}+ -- | -- Module      : Crypto.KDF.PBKDF2 -- License     : BSD-style@@ -6,67 +9,105 @@ -- Portability : unknown -- -- Password Based Key Derivation Function 2----{-# LANGUAGE BangPatterns #-}-{-# LANGUAGE ForeignFunctionInterface #-}--module Crypto.KDF.PBKDF2-    ( PRF-    , prfHMAC-    , Parameters(..)-    , generate-    , fastPBKDF2_SHA1-    , fastPBKDF2_SHA256-    , fastPBKDF2_SHA512-    ) where+module Crypto.KDF.PBKDF2 (+    PRF,+    prfHMAC,+    Parameters (..),+    generate,+    tryGenerate,+    fastPBKDF2_SHA1,+    tryFastPBKDF2_SHA1,+    fastPBKDF2_SHA256,+    tryFastPBKDF2_SHA256,+    fastPBKDF2_SHA512,+    tryFastPBKDF2_SHA512,+) where -import           Data.Word-import           Data.Bits-import           Foreign.Marshal.Alloc-import           Foreign.Ptr (plusPtr, Ptr)-import           Foreign.C.Types (CUInt(..), CSize(..))+import Data.Bits+import Data.Word+import Foreign.C.Types (CSize (..), CUInt (..))+import Foreign.Marshal.Alloc+import Foreign.Ptr (Ptr, plusPtr) -import           Crypto.Hash (HashAlgorithm)+import Crypto.Error+import Crypto.Hash (HashAlgorithm) import qualified Crypto.MAC.HMAC as HMAC -import           Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, Bytes)+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, Bytes) import qualified Crypto.Internal.ByteArray as B-import           Data.Memory.PtrMethods+import Data.Memory.PtrMethods  -- | The PRF used for PBKDF2 type PRF password =-       password -- ^ the password parameters-    -> Bytes    -- ^ the content-    -> Bytes    -- ^ prf(password,content)+    password+    -- ^ the password parameters+    -> Bytes+    -- ^ the content+    -> Bytes+    -- ^ prf(password,content)  -- | PRF for PBKDF2 using HMAC with the hash algorithm as parameter-prfHMAC :: (HashAlgorithm a, ByteArrayAccess password)-        => a-        -> PRF password+prfHMAC+    :: (HashAlgorithm a, ByteArrayAccess password)+    => a+    -> PRF password prfHMAC alg k = hmacIncr alg (HMAC.initialize k)-  where hmacIncr :: HashAlgorithm a => a -> HMAC.Context a -> (Bytes -> Bytes)-        hmacIncr _ !ctx = \b -> B.convert $ HMAC.finalize $ HMAC.update ctx b+  where+    hmacIncr :: HashAlgorithm a => a -> HMAC.Context a -> (Bytes -> Bytes)+    hmacIncr _ !ctx = \b -> B.convert $ HMAC.finalize $ HMAC.update ctx b  -- | Parameters for PBKDF2 data Parameters = Parameters-    { iterCounts   :: Int -- ^ the number of user-defined iterations for the algorithms. e.g. WPA2 uses 4000.-    , outputLength :: Int -- ^ the number of bytes to generate out of PBKDF2+    { iterCounts :: Int+    -- ^ the number of user-defined iterations for the algorithms. e.g. WPA2 uses 4000.+    --   (must be > 0)+    , outputLength :: Int+    -- ^ the number of bytes to generate out of PBKDF2+    --   (must not be negative)     } +-- | Report parameters no PBKDF2 entry point accepts.+--+-- An iteration count below one derives a key that is not a key at all, and a+-- negative output length asks for a buffer that cannot be allocated.+validateParameters :: Parameters -> Maybe CryptoError+validateParameters params+    | iterCounts params < 1 = Just CryptoError_ParameterInvalid+    | outputLength params < 0 = Just CryptoError_ParameterInvalid+    | otherwise = Nothing+ -- | generate the pbkdf2 key derivation function from the output-generate :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray ba)-         => PRF password-         -> Parameters-         -> password-         -> salt-         -> ba+--+-- Parameters outside the ranges documented for t'Parameters' raise+-- 'CryptoError_ParameterInvalid'; 'tryGenerate' reports the same condition as+-- 'CryptoFailed'.+generate+    :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray ba)+    => PRF password+    -> Parameters+    -> password+    -> salt+    -> ba generate prf params password salt =-    B.allocAndFreeze (outputLength params) $ \p -> do+    throwCryptoError (tryGenerate prf params password salt)++-- | generate the pbkdf2 key derivation function from the output, reporting+-- parameters the implementation refuses rather than raising.+tryGenerate+    :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray ba)+    => PRF password+    -> Parameters+    -> password+    -> salt+    -> CryptoFailable ba+tryGenerate prf params password salt+    | Just err <- validateParameters params = CryptoFailed err+    | otherwise = CryptoPassed $ B.allocAndFreeze (outputLength params) $ \p -> do         memSet p 0 (outputLength params)         loop 1 (outputLength params) p   where     !runPRF = prf password-    !hLen   = B.length $ runPRF B.empty+    !hLen = B.length $ runPRF B.empty      -- run the following f function on each complete chunk.     -- when having an incomplete chunk, we call partial.@@ -76,100 +117,175 @@     -- U1 = PRF(pass,salt || BE32(i))     -- Uc = PRF(pass,Uc-1)     loop iterNb len p-        | len == 0   = return ()+        | len == 0 = return ()         | len < hLen = partial iterNb len p-        | otherwise  = do-            let applyMany 0 _     = return ()+        | otherwise = do+            let applyMany 0 _ = return ()                 applyMany i uprev = do                     let uData = runPRF uprev                     B.withByteArray uData $ \u -> memXor p p u hLen-                    applyMany (i-1) uData+                    applyMany (i - 1) uData             applyMany (iterCounts params) (B.convert salt `B.append` toBS iterNb)-            loop (iterNb+1) (len - hLen) (p `plusPtr` hLen)+            loop (iterNb + 1) (len - hLen) (p `plusPtr` hLen)      partial iterNb len p = allocaBytesAligned hLen 8 $ \tmp -> do         let applyMany :: Int -> Bytes -> IO ()-            applyMany 0 _     = return ()+            applyMany 0 _ = return ()             applyMany i uprev = do                 let uData = runPRF uprev                 B.withByteArray uData $ \u -> memXor tmp tmp u hLen-                applyMany (i-1) uData+                applyMany (i - 1) uData         memSet tmp 0 hLen         applyMany (iterCounts params) (B.convert salt `B.append` toBS iterNb)         memCopy p tmp len      -- big endian encoding of Word32     toBS :: ByteArray ba => Word32 -> ba-    toBS w = B.pack [a,b,c,d]-      where a = fromIntegral (w `shiftR` 24)-            b = fromIntegral ((w `shiftR` 16) .&. 0xff)-            c = fromIntegral ((w `shiftR` 8) .&. 0xff)-            d = fromIntegral (w .&. 0xff)-{-# NOINLINE generate #-}+    toBS w = B.pack [a, b, c, d]+      where+        a = fromIntegral (w `shiftR` 24)+        b = fromIntegral ((w `shiftR` 16) .&. 0xff)+        c = fromIntegral ((w `shiftR` 8) .&. 0xff)+        d = fromIntegral (w .&. 0xff)+{-# NOINLINE tryGenerate #-} -fastPBKDF2_SHA1 :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out)-                => Parameters-                -> password-                -> salt-                -> out+-- | PBKDF2 with HMAC-SHA1, using the bundled C implementation.+--+-- Parameters outside the ranges documented for t'Parameters' raise+-- 'CryptoError_ParameterInvalid'; 'tryFastPBKDF2_SHA1' reports the same condition+-- as 'CryptoFailed'.+fastPBKDF2_SHA1+    :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out)+    => Parameters+    -> password+    -> salt+    -> out fastPBKDF2_SHA1 params password salt =-    B.allocAndFreeze (outputLength params) $ \outPtr ->-    B.withByteArray password $ \passPtr ->-    B.withByteArray salt $ \saltPtr ->-        c_crypton_fastpbkdf2_hmac_sha1-            passPtr (fromIntegral $ B.length password)-            saltPtr (fromIntegral $ B.length salt)-            (fromIntegral $ iterCounts params)-            outPtr (fromIntegral $ outputLength params)+    throwCryptoError (tryFastPBKDF2_SHA1 params password salt) -fastPBKDF2_SHA256 :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out)-                  => Parameters-                  -> password-                  -> salt-                  -> out+-- | PBKDF2 with HMAC-SHA1, reporting parameters the implementation refuses+-- rather than raising.+tryFastPBKDF2_SHA1+    :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out)+    => Parameters+    -> password+    -> salt+    -> CryptoFailable out+tryFastPBKDF2_SHA1 params password salt+    | Just err <- validateParameters params = CryptoFailed err+    | otherwise = CryptoPassed $ B.allocAndFreeze (outputLength params) $ \outPtr ->+        B.withByteArray password $ \passPtr ->+            B.withByteArray salt $ \saltPtr ->+                c_crypton_fastpbkdf2_hmac_sha1+                    passPtr+                    (fromIntegral $ B.length password)+                    saltPtr+                    (fromIntegral $ B.length salt)+                    (fromIntegral $ iterCounts params)+                    outPtr+                    (fromIntegral $ outputLength params)++-- | PBKDF2 with HMAC-SHA256, using the bundled C implementation.+--+-- Parameters outside the ranges documented for t'Parameters' raise+-- 'CryptoError_ParameterInvalid'; 'tryFastPBKDF2_SHA256' reports the same condition+-- as 'CryptoFailed'.+fastPBKDF2_SHA256+    :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out)+    => Parameters+    -> password+    -> salt+    -> out fastPBKDF2_SHA256 params password salt =-    B.allocAndFreeze (outputLength params) $ \outPtr ->-    B.withByteArray password $ \passPtr ->-    B.withByteArray salt $ \saltPtr ->-        c_crypton_fastpbkdf2_hmac_sha256-            passPtr (fromIntegral $ B.length password)-            saltPtr (fromIntegral $ B.length salt)-            (fromIntegral $ iterCounts params)-            outPtr (fromIntegral $ outputLength params)+    throwCryptoError (tryFastPBKDF2_SHA256 params password salt) -fastPBKDF2_SHA512 :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out)-                  => Parameters-                  -> password-                  -> salt-                  -> out+-- | PBKDF2 with HMAC-SHA256, reporting parameters the implementation refuses+-- rather than raising.+tryFastPBKDF2_SHA256+    :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out)+    => Parameters+    -> password+    -> salt+    -> CryptoFailable out+tryFastPBKDF2_SHA256 params password salt+    | Just err <- validateParameters params = CryptoFailed err+    | otherwise = CryptoPassed $ B.allocAndFreeze (outputLength params) $ \outPtr ->+        B.withByteArray password $ \passPtr ->+            B.withByteArray salt $ \saltPtr ->+                c_crypton_fastpbkdf2_hmac_sha256+                    passPtr+                    (fromIntegral $ B.length password)+                    saltPtr+                    (fromIntegral $ B.length salt)+                    (fromIntegral $ iterCounts params)+                    outPtr+                    (fromIntegral $ outputLength params)++-- | PBKDF2 with HMAC-SHA512, using the bundled C implementation.+--+-- Parameters outside the ranges documented for t'Parameters' raise+-- 'CryptoError_ParameterInvalid'; 'tryFastPBKDF2_SHA512' reports the same condition+-- as 'CryptoFailed'.+fastPBKDF2_SHA512+    :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out)+    => Parameters+    -> password+    -> salt+    -> out fastPBKDF2_SHA512 params password salt =-    B.allocAndFreeze (outputLength params) $ \outPtr ->-    B.withByteArray password $ \passPtr ->-    B.withByteArray salt $ \saltPtr ->-        c_crypton_fastpbkdf2_hmac_sha512-            passPtr (fromIntegral $ B.length password)-            saltPtr (fromIntegral $ B.length salt)-            (fromIntegral $ iterCounts params)-            outPtr (fromIntegral $ outputLength params)+    throwCryptoError (tryFastPBKDF2_SHA512 params password salt) +-- | PBKDF2 with HMAC-SHA512, reporting parameters the implementation refuses+-- rather than raising.+tryFastPBKDF2_SHA512+    :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out)+    => Parameters+    -> password+    -> salt+    -> CryptoFailable out+tryFastPBKDF2_SHA512 params password salt+    | Just err <- validateParameters params = CryptoFailed err+    | otherwise = CryptoPassed $ B.allocAndFreeze (outputLength params) $ \outPtr ->+        B.withByteArray password $ \passPtr ->+            B.withByteArray salt $ \saltPtr ->+                c_crypton_fastpbkdf2_hmac_sha512+                    passPtr+                    (fromIntegral $ B.length password)+                    saltPtr+                    (fromIntegral $ B.length salt)+                    (fromIntegral $ iterCounts params)+                    outPtr+                    (fromIntegral $ outputLength params)  foreign import ccall unsafe "crypton_pbkdf2.h crypton_fastpbkdf2_hmac_sha1"-    c_crypton_fastpbkdf2_hmac_sha1 :: Ptr Word8 -> CSize-                                      -> Ptr Word8 -> CSize-                                      -> CUInt-                                      -> Ptr Word8 -> CSize-                                      -> IO ()+    c_crypton_fastpbkdf2_hmac_sha1+        :: Ptr Word8+        -> CSize+        -> Ptr Word8+        -> CSize+        -> CUInt+        -> Ptr Word8+        -> CSize+        -> IO ()  foreign import ccall unsafe "crypton_pbkdf2.h crypton_fastpbkdf2_hmac_sha256"-    c_crypton_fastpbkdf2_hmac_sha256 :: Ptr Word8 -> CSize-                                        -> Ptr Word8 -> CSize-                                        -> CUInt-                                        -> Ptr Word8 -> CSize-                                        -> IO ()+    c_crypton_fastpbkdf2_hmac_sha256+        :: Ptr Word8+        -> CSize+        -> Ptr Word8+        -> CSize+        -> CUInt+        -> Ptr Word8+        -> CSize+        -> IO ()  foreign import ccall unsafe "crypton_pbkdf2.h crypton_fastpbkdf2_hmac_sha512"-    c_crypton_fastpbkdf2_hmac_sha512 :: Ptr Word8 -> CSize-                                        -> Ptr Word8 -> CSize-                                        -> CUInt-                                        -> Ptr Word8 -> CSize-                                        -> IO ()+    c_crypton_fastpbkdf2_hmac_sha512+        :: Ptr Word8+        -> CSize+        -> Ptr Word8+        -> CSize+        -> CUInt+        -> Ptr Word8+        -> CSize+        -> IO ()
Crypto/KDF/Scrypt.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE ForeignFunctionInterface #-}+ -- | -- Module      : Crypto.KDF.Scrypt -- License     : BSD-style@@ -8,57 +11,85 @@ -- Scrypt key derivation function as defined in Colin Percival's paper -- "Stronger Key Derivation via Sequential Memory-Hard Functions" -- <http://www.tarsnap.com/scrypt/scrypt.pdf>.----{-# LANGUAGE BangPatterns #-}-{-# LANGUAGE ForeignFunctionInterface #-}-module Crypto.KDF.Scrypt-    ( Parameters(..)-    , generate-    ) where+module Crypto.KDF.Scrypt (+    Parameters (..),+    generate,+    tryGenerate,+) where -import           Data.Word-import           Foreign.Marshal.Alloc-import           Foreign.Ptr (Ptr, plusPtr)-import           Control.Monad (forM_)+import Control.Monad (forM_)+import Data.Word+import Foreign.Marshal.Alloc+import Foreign.Ptr (Ptr, plusPtr) -import           Crypto.Hash (SHA256(..))-import qualified Crypto.KDF.PBKDF2 as PBKDF2-import           Crypto.Internal.Compat (popCount, unsafeDoIO)-import           Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess)+import Crypto.Error+import Crypto.Hash (SHA256 (..))+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess) import qualified Crypto.Internal.ByteArray as B+import Crypto.Internal.Compat (popCount, unsafeDoIO)+import qualified Crypto.KDF.PBKDF2 as PBKDF2  -- | Parameters for Scrypt data Parameters = Parameters-    { n            :: Word64 -- ^ Cpu/Memory cost ratio. must be a power of 2 greater than 1. also known as N.-    , r            :: Int    -- ^ Must satisfy r * p < 2^30-    , p            :: Int    -- ^ Must satisfy r * p < 2^30-    , outputLength :: Int    -- ^ the number of bytes to generate out of Scrypt+    { n :: Word64+    -- ^ Cpu/Memory cost ratio. must be a power of 2 greater than 1. also known as N.+    , r :: Int+    -- ^ Must satisfy r * p < 2^30+    , p :: Int+    -- ^ Must satisfy r * p < 2^30+    , outputLength :: Int+    -- ^ the number of bytes to generate out of Scrypt     }  foreign import ccall "crypton_scrypt_smix"-    ccrypton_scrypt_smix :: Ptr Word8 -> Word32 -> Word64 -> Ptr Word8 -> Ptr Word8 -> IO ()+    ccrypton_scrypt_smix+        :: Ptr Word8 -> Word32 -> Word64 -> Ptr Word8 -> Ptr Word8 -> IO ()  -- | Generate the scrypt key derivation data-generate :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray output)-         => Parameters-         -> password-         -> salt-         -> output-generate params password salt-    | r params * p params >= 0x40000000 =-        error "Scrypt: invalid parameters: r and p constraint"-    | popCount (n params) /= 1 =-        error "Scrypt: invalid parameters: n not a power of 2"-    | otherwise = unsafeDoIO $ do+--+-- Parameters the implementation refuses raise a 'CryptoError'; 'tryGenerate'+-- reports the same condition as 'CryptoFailed'.+generate+    :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray output)+    => Parameters+    -> password+    -> salt+    -> output+generate params password salt = throwCryptoError (tryGenerate params password salt)++-- | Generate the scrypt key derivation data, reporting parameters the+-- implementation refuses rather than raising.+--+-- @n@ has to be a power of two, and @r@ times @p@ has to stay below 2^30.+tryGenerate+    :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray output)+    => Parameters+    -> password+    -> salt+    -> CryptoFailable output+tryGenerate params password salt+    | r params * p params >= 0x40000000 = CryptoFailed CryptoError_ParameterInvalid+    | popCount (n params) /= 1 = CryptoFailed CryptoError_ParameterInvalid+    | otherwise = CryptoPassed $ unsafeDoIO $ do         let b = PBKDF2.generate prf (PBKDF2.Parameters 1 intLen) password salt :: B.Bytes         newSalt <- B.copy b $ \bPtr ->-            allocaBytesAligned (128*(fromIntegral $ n params)*(r params)) 8 $ \v ->-            allocaBytesAligned (256*r params + 64) 8 $ \xy -> do-                forM_ [0..(p params-1)] $ \i ->-                    ccrypton_scrypt_smix (bPtr `plusPtr` (i * 128 * (r params)))-                                            (fromIntegral $ r params) (n params) v xy+            allocaBytesAligned (128 * (fromIntegral $ n params) * (r params)) 8 $ \v ->+                allocaBytesAligned (256 * r params + 64) 8 $ \xy -> do+                    forM_ [0 .. (p params - 1)] $ \i ->+                        ccrypton_scrypt_smix+                            (bPtr `plusPtr` (i * 128 * (r params)))+                            (fromIntegral $ r params)+                            (n params)+                            v+                            xy -        return $ PBKDF2.generate prf (PBKDF2.Parameters 1 (outputLength params)) password (newSalt :: B.Bytes)-  where prf    = PBKDF2.prfHMAC SHA256-        intLen = p params * 128 * r params-{-# NOINLINE generate #-}+        return $+            PBKDF2.generate+                prf+                (PBKDF2.Parameters 1 (outputLength params))+                password+                (newSalt :: B.Bytes)+  where+    prf = PBKDF2.prfHMAC SHA256+    intLen = p params * 128 * r params+{-# NOINLINE tryGenerate #-}
Crypto/MAC/CMAC.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.MAC.CMAC -- License     : BSD-style@@ -8,20 +11,20 @@ -- Provide the CMAC (Cipher based Message Authentification Code) base algorithm. -- <http://en.wikipedia.org/wiki/CMAC> -- <http://csrc.nist.gov/publications/nistpubs/800-38B/SP_800-38B.pdf>----{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.MAC.CMAC-    ( cmac-    , CMAC-    , subKeys-    ) where+module Crypto.MAC.CMAC (+    cmac,+    CMAC,+    subKeys,+) where -import           Data.Word-import           Data.Bits (setBit, testBit, shiftL)-import           Data.List (foldl')+import Data.Bits (setBit, shiftL, testBit)+import Data.ByteString (ByteString)+import qualified Data.ByteString as S+import Data.Word -import           Crypto.Cipher.Types-import           Crypto.Internal.ByteArray (ByteArrayAccess, ByteArray, Bytes)+import Crypto.Cipher.Types+import Crypto.Cipher.Types.Block (IV (..))+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, Bytes) import qualified Crypto.Internal.ByteArray as B  -- | Authentication code@@ -29,70 +32,100 @@     deriving (ByteArrayAccess)  instance Eq (CMAC a) where-  CMAC b1 == CMAC b2  =  B.constEq b1 b2+    CMAC b1 == CMAC b2 = B.constEq b1 b2  -- | compute a MAC using the supplied cipher-cmac :: (ByteArrayAccess bin, BlockCipher cipher)-     => cipher      -- ^ key to compute CMAC with-     -> bin         -- ^ input message-     -> CMAC cipher -- ^ output tag-cmac k msg =-    CMAC $ foldl' (\c m -> ecbEncrypt k $ bxor c m) zeroV ms+cmac+    :: (ByteArrayAccess bin, BlockCipher cipher)+    => cipher+    -- ^ key to compute CMAC with+    -> bin+    -- ^ input message+    -> CMAC cipher+    -- ^ output tag+cmac k msg = CMAC $ B.convert $ step (chain zeroV whole) final   where     bytes = blockSize k-    zeroV = B.replicate bytes 0 :: Bytes+    zeroV = S.replicate bytes 0     (k1, k2) = subKeys k-    ms = cmacChunks k k1 k2 $ B.convert msg -cmacChunks :: (BlockCipher k, ByteArray ba) => k -> ba -> ba -> ba -> [ba]-cmacChunks k k1 k2  =  rec'  where-    rec' msg-      | B.null tl  =  if lack == 0-                      then  [bxor k1 hd]-                      else  [bxor k2 $ hd `B.append` B.pack (0x80 : replicate (lack - 1) 0)]-      | otherwise  =        hd : rec' tl-      where-          bytes = blockSize k-          (hd, tl) = B.splitAt bytes msg-          lack = bytes - B.length hd+    -- The message is held as a ByteString and sliced, never consumed.  'Bytes'+    -- has no shared representation, so splitting one repeatedly -- which is+    -- what this used to do, once per block -- copied whatever was left of the+    -- message each time, and so the message about n/2 times in all.+    msgBytes = B.convert msg :: ByteString+    msgLen = S.length msgBytes +    -- the last block is the one the subkeys are for, and it is a whole block+    -- only when there is one to be had+    lastLen+        | msgLen > 0 && msgLen `mod` bytes == 0 = bytes+        | otherwise = msgLen `mod` bytes+    (whole, rest) = S.splitAt (msgLen - lastLen) msgBytes+    final+        | lastLen == bytes = bxor k1 rest+        | otherwise =+            bxor k2 $+                S.concat [rest, S.singleton 0x80, S.replicate (bytes - lastLen - 1) 0]++    -- CMAC chains its blocks the way CBC does, so the running state is the+    -- last ciphertext block of a CBC encryption.  Handing the cipher a chunk+    -- at a time rather than a block at a time is what makes that worth saying:+    -- for AES it reaches the C implementation of CBC, where a block at a time+    -- reached a foreign call per sixteen bytes.+    chunkBytes = bytes * 2048+    chain !c bs+        | S.null bs = c+        | otherwise =+            let (hd, tl) = S.splitAt chunkBytes bs+                out = cbcEncrypt k (IV c) hd+             in chain (S.drop (S.length hd - bytes) out) tl++    step c m = ecbEncrypt k (bxor c m) :: ByteString+ -- | make sub-keys used in CMAC-subKeys :: (BlockCipher k, ByteArray ba)-        => k         -- ^ key to compute CMAC with-        -> (ba, ba)  -- ^ sub-keys to compute CMAC-subKeys k = (k1, k2)   where+subKeys+    :: (BlockCipher k, ByteArray ba)+    => k+    -- ^ key to compute CMAC with+    -> (ba, ba)+    -- ^ sub-keys to compute CMAC+subKeys k = (k1, k2)+  where     ipt = cipherIPT k     k0 = ecbEncrypt k $ B.replicate (blockSize k) 0     k1 = subKey ipt k0     k2 = subKey ipt k1  -- polynomial multiply operation to culculate subkey-subKey :: (ByteArray ba) => [Word8] -> ba -> ba-subKey ipt ws  =  case B.unpack ws of-    []                  ->  B.empty-    w:_  | testBit w 7  ->  B.pack ipt `bxor` shiftL1 ws-         | otherwise    ->  shiftL1 ws+subKey :: ByteArray ba => [Word8] -> ba -> ba+subKey ipt ws = case B.unpack ws of+    [] -> B.empty+    w : _+        | testBit w 7 -> B.pack ipt `bxor` shiftL1 ws+        | otherwise -> shiftL1 ws -shiftL1 :: (ByteArray ba) => ba -> ba+shiftL1 :: ByteArray ba => ba -> ba shiftL1 = B.pack . shiftL1W . B.unpack  shiftL1W :: [Word8] -> [Word8]-shiftL1W []         =  []-shiftL1W ws@(_:ns)  =  rec' $ zip ws (ns ++ [0])   where-    rec'  []         =  []-    rec' ((x,y):ps)  =  w : rec' ps+shiftL1W [] = []+shiftL1W ws@(_ : ns) = rec' $ zip ws (ns ++ [0])+  where+    rec' [] = []+    rec' ((x, y) : ps) = w : rec' ps       where-          w | testBit y 7  =  setBit sl1 0-            | otherwise    =  sl1-            where     sl1 = shiftL x 1+        w+            | testBit y 7 = setBit sl1 0+            | otherwise = sl1+          where+            sl1 = shiftL x 1  bxor :: ByteArray ba => ba -> ba -> ba-bxor = B.xor-+bxor = B.bxor  ----- - cipherIPT :: BlockCipher k => k -> [Word8] cipherIPT = expandIPT . blockSize @@ -104,29 +137,44 @@ -- It represents that the smallest irreducible binary polynomial of degree 128 -- is x^128 + x^7 + x^2 + x^1 + 1. data IPolynomial-  = Q Int Int Int+    = Q Int Int Int+ ---  | T Int  iPolynomial :: Int -> Maybe IPolynomial-iPolynomial = d  where-    d   64  =  Just $ Q 4 3 1-    d  128  =  Just $ Q 7 2 1-    d    _  =  Nothing+iPolynomial = d+  where+    d 64 = Just $ Q 4 3 1+    d 128 = Just $ Q 7 2 1+    d _ = Nothing  -- Expand a tail bit pattern of irreducible binary polynomial expandIPT :: Int -> [Word8]-expandIPT bytes = expandIPT' bytes ipt  where-    ipt = maybe (error $ "Irreducible binary polynomial not defined against " ++ show nb ++ " bit") id-          $ iPolynomial nb+expandIPT bytes = expandIPT' bytes ipt+  where+    ipt =+        maybe+            ( error $+                "Irreducible binary polynomial not defined against " ++ show nb ++ " bit"+            )+            id+            $ iPolynomial nb     nb = bytes * 8  -- Expand a tail bit pattern of irreducible binary polynomial-expandIPT' :: Int         -- ^ width in byte-           -> IPolynomial -- ^ irreducible binary polynomial definition-           -> [Word8]     -- ^ result bit pattern+expandIPT'+    :: Int+    -- ^ width in byte+    -> IPolynomial+    -- ^ irreducible binary polynomial definition+    -> [Word8]+    -- ^ result bit pattern expandIPT' bytes (Q x y z) =     reverse . setB x . setB y . setB z . setB 0 $ replicate bytes 0   where-    setB i ws =  hd ++ setBit (head tl) r : tail tl  where+    setB i ws = case tl of+        (a : as) -> hd ++ setBit a r : as+        _ -> error "expandIPT'"+      where         (q, r) = i `quotRem` 8         (hd, tl) = splitAt q ws
Crypto/MAC/HMAC.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.MAC.HMAC -- License     : BSD-style@@ -7,117 +10,136 @@ -- -- Provide the HMAC (Hash based Message Authentification Code) base algorithm. -- <http://en.wikipedia.org/wiki/HMAC>----{-# LANGUAGE BangPatterns #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.MAC.HMAC-    ( hmac-    , hmacLazy-    , HMAC(..)+module Crypto.MAC.HMAC (+    hmac,+    hmacLazy,+    HMAC (..),+     -- * Incremental-    , Context(..)-    , initialize-    , update-    , updates-    , finalize-    ) where+    Context (..),+    initialize,+    update,+    updates,+    finalize,+) where -import           Crypto.Hash hiding (Context)+import Crypto.Hash hiding (Context) import qualified Crypto.Hash as Hash (Context)-import           Crypto.Hash.IO-import           Crypto.Internal.ByteArray (ScrubbedBytes, ByteArrayAccess)+import Crypto.Hash.IO+import Crypto.Internal.ByteArray (ByteArrayAccess, ScrubbedBytes) import qualified Crypto.Internal.ByteArray as B-import           Data.Memory.PtrMethods-import           Crypto.Internal.Compat+import Crypto.Internal.Compat import qualified Data.ByteString.Lazy as L+import Data.Memory.PtrMethods  -- | Represent an HMAC that is a phantom type with the hash used to produce the mac. -- -- The Eq instance is constant time.  No Show instance is provided, to avoid -- printing by mistake.-newtype HMAC a = HMAC { hmacGetDigest :: Digest a }+newtype HMAC a = HMAC {hmacGetDigest :: Digest a}     deriving (ByteArrayAccess)  instance Eq (HMAC a) where     (HMAC b1) == (HMAC b2) = B.constEq b1 b2  -- | Compute a MAC using the supplied hashing function-hmac :: (ByteArrayAccess key, ByteArrayAccess message, HashAlgorithm a)-     => key     -- ^ Secret key-     -> message -- ^ Message to MAC-     -> HMAC a+hmac+    :: (ByteArrayAccess key, ByteArrayAccess message, HashAlgorithm a)+    => key+    -- ^ Secret key+    -> message+    -- ^ Message to MAC+    -> HMAC a hmac secret msg = finalize $ updates (initialize secret) [msg]  -- | Compute a MAC using the supplied hashing function, for a lazy input-hmacLazy :: (ByteArrayAccess key, HashAlgorithm a)-     => key     -- ^ Secret key-     -> L.ByteString -- ^ Message to MAC-     -> HMAC a+hmacLazy+    :: (ByteArrayAccess key, HashAlgorithm a)+    => key+    -- ^ Secret key+    -> L.ByteString+    -- ^ Message to MAC+    -> HMAC a hmacLazy secret msg = finalize $ updates (initialize secret) (L.toChunks msg)  -- | Represent an ongoing HMAC state, that can be appended with 'update'--- and finalize to an HMAC with 'hmacFinalize'+-- and finalize to an HMAC with 'finalize' data Context hashalg = Context !(Hash.Context hashalg) !(Hash.Context hashalg)  -- | Initialize a new incremental HMAC context-initialize :: (ByteArrayAccess key, HashAlgorithm a)-           => key       -- ^ Secret key-           -> Context a+initialize+    :: (ByteArrayAccess key, HashAlgorithm a)+    => key+    -- ^ Secret key+    -> Context a initialize secret = unsafeDoIO (doHashAlg undefined)   where-        doHashAlg :: HashAlgorithm a => a -> IO (Context a)-        doHashAlg alg = do-            !withKey <- case B.length secret `compare` blockSize of-                            EQ -> return $ B.withByteArray secret-                            LT -> do key <- B.alloc blockSize $ \k -> do-                                        memSet k 0 blockSize-                                        B.withByteArray secret $ \s -> memCopy k s (B.length secret)-                                     return $ B.withByteArray (key :: ScrubbedBytes)-                            GT -> do-                                -- hash the secret key-                                ctx <- hashMutableInitWith alg-                                hashMutableUpdate ctx secret-                                digest <- hashMutableFinalize ctx-                                hashMutableReset ctx-                                -- pad it if necessary-                                if digestSize < blockSize-                                    then do-                                        key <- B.alloc blockSize $ \k -> do-                                            memSet k 0 blockSize-                                            B.withByteArray digest $ \s -> memCopy k s (B.length digest)-                                        return $ B.withByteArray (key :: ScrubbedBytes)-                                    else-                                       return $ B.withByteArray digest-            (inner, outer) <- withKey $ \keyPtr ->-                (,) <$> B.alloc blockSize (\p -> memXorWith p 0x36 keyPtr blockSize)-                    <*> B.alloc blockSize (\p -> memXorWith p 0x5c keyPtr blockSize)-            return $ Context (hashUpdates initCtx [outer :: ScrubbedBytes])-                             (hashUpdates initCtx [inner :: ScrubbedBytes])-          where -                blockSize  = hashBlockSize alg-                digestSize = hashDigestSize alg-                initCtx    = hashInitWith alg+    doHashAlg :: HashAlgorithm a => a -> IO (Context a)+    doHashAlg alg = do+        !withKey <- case B.length secret `compare` blockSize of+            EQ -> return $ B.withByteArray secret+            LT -> do+                key <- B.alloc blockSize $ \k -> do+                    memSet k 0 blockSize+                    B.withByteArray secret $ \s -> memCopy k s (B.length secret)+                return $ B.withByteArray (key :: ScrubbedBytes)+            GT -> do+                -- hash the secret key+                ctx <- hashMutableInitWith alg+                hashMutableUpdate ctx secret+                digest <- hashMutableFinalize ctx+                hashMutableReset ctx+                -- pad it if necessary+                if digestSize < blockSize+                    then do+                        key <- B.alloc blockSize $ \k -> do+                            memSet k 0 blockSize+                            B.withByteArray digest $ \s -> memCopy k s (B.length digest)+                        return $ B.withByteArray (key :: ScrubbedBytes)+                    else+                        return $ B.withByteArray digest+        (inner, outer) <- withKey $ \keyPtr ->+            (,)+                <$> B.alloc blockSize (\p -> memXorWith p 0x36 keyPtr blockSize)+                <*> B.alloc blockSize (\p -> memXorWith p 0x5c keyPtr blockSize)+        return $+            Context+                (hashUpdates initCtx [outer :: ScrubbedBytes])+                (hashUpdates initCtx [inner :: ScrubbedBytes])+      where+        blockSize = hashBlockSize alg+        digestSize = hashDigestSize alg+        initCtx = hashInitWith alg {-# NOINLINE initialize #-}  -- | Incrementally update a HMAC context-update :: (ByteArrayAccess message, HashAlgorithm a)-       => Context a  -- ^ Current HMAC context-       -> message    -- ^ Message to append to the MAC-       -> Context a  -- ^ Updated HMAC context+update+    :: (ByteArrayAccess message, HashAlgorithm a)+    => Context a+    -- ^ Current HMAC context+    -> message+    -- ^ Message to append to the MAC+    -> Context a+    -- ^ Updated HMAC context update (Context octx ictx) msg =     Context octx (hashUpdate ictx msg)  -- | Increamentally update a HMAC context with multiple inputs-updates :: (ByteArrayAccess message, HashAlgorithm a)-        => Context a -- ^ Current HMAC context-        -> [message] -- ^ Messages to append to the MAC-        -> Context a -- ^ Updated HMAC context+updates+    :: (ByteArrayAccess message, HashAlgorithm a)+    => Context a+    -- ^ Current HMAC context+    -> [message]+    -- ^ Messages to append to the MAC+    -> Context a+    -- ^ Updated HMAC context updates (Context octx ictx) msgs =     Context octx (hashUpdates ictx msgs)  -- | Finalize a HMAC context and return the HMAC.-finalize :: HashAlgorithm a-         => Context a-         -> HMAC a+finalize+    :: HashAlgorithm a+    => Context a+    -> HMAC a finalize (Context octx ictx) =     HMAC $ hashFinalize $ hashUpdates octx [hashFinalize ictx]
Crypto/MAC/KMAC.hs view
@@ -1,3 +1,7 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# LANGUAGE ScopedTypeVariables #-}+ -- | -- Module      : Crypto.MAC.KMAC -- License     : BSD-style@@ -7,38 +11,41 @@ -- -- Provide the KMAC (Keccak Message Authentication Code) algorithm, derived from -- the SHA-3 base algorithm Keccak and defined in NIST SP800-185.----{-# LANGUAGE BangPatterns #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-{-# LANGUAGE ScopedTypeVariables #-}-module Crypto.MAC.KMAC-    ( HashSHAKE-    , kmac-    , KMAC(..)+module Crypto.MAC.KMAC (+    HashSHAKE,+    kmac,+    KMAC (..),+     -- * Incremental-    , Context-    , initialize-    , update-    , updates-    , finalize-    ) where+    Context,+    initialize,+    update,+    updates,+    finalize,+) where  import qualified Crypto.Hash as H-import           Crypto.Hash.SHAKE (HashSHAKE(..))-import           Crypto.Hash.Types (HashAlgorithm(..), Digest(..))+import Crypto.Hash.SHAKE (HashSHAKE (..))+import Crypto.Hash.Types (Digest (..), HashAlgorithm (..)) import qualified Crypto.Hash.Types as H-import           Crypto.Internal.Builder-import           Crypto.Internal.Imports-import           Foreign.Ptr (Ptr)-import           Data.Bits (shiftR)-import           Data.ByteArray (ByteArrayAccess)+import Crypto.Internal.Builder+import Crypto.Internal.ByteArray (allocAndFreezePrim)+import Crypto.Internal.Imports+import Data.Bits (shiftR)+import Data.ByteArray (ByteArrayAccess) import qualified Data.ByteArray as B-+import Foreign.Ptr (Ptr)  -- cSHAKE -cshakeInit :: forall a name string prefix . (HashSHAKE a, ByteArrayAccess name, ByteArrayAccess string, ByteArrayAccess prefix)-           => name -> string -> prefix -> H.Context a+cshakeInit+    :: forall a name string prefix+     . ( HashSHAKE a+       , ByteArrayAccess name+       , ByteArrayAccess string+       , ByteArrayAccess prefix+       )+    => name -> string -> prefix -> H.Context a cshakeInit n s p = H.Context $ B.allocAndFreeze c $ \(ptr :: Ptr (H.Context a)) -> do     hashInternalInit ptr     B.withByteArray b $ \d -> hashInternalUpdate ptr d (fromIntegral $ B.length b)@@ -49,24 +56,28 @@     x = encodeString n <> encodeString s     b = buildAndFreeze (bytepad x w) :: B.Bytes -cshakeUpdate :: (HashSHAKE a, ByteArrayAccess ba)-             => H.Context a -> ba -> H.Context a+cshakeUpdate+    :: (HashSHAKE a, ByteArrayAccess ba)+    => H.Context a -> ba -> H.Context a cshakeUpdate = H.hashUpdate -cshakeUpdates :: (HashSHAKE a, ByteArrayAccess ba)-              => H.Context a -> [ba] -> H.Context a+cshakeUpdates+    :: (HashSHAKE a, ByteArrayAccess ba)+    => H.Context a -> [ba] -> H.Context a cshakeUpdates = H.hashUpdates -cshakeFinalize :: forall a suffix . (HashSHAKE a, ByteArrayAccess suffix)-               => H.Context a -> suffix -> Digest a-cshakeFinalize !c s =-    Digest $ B.allocAndFreeze (hashDigestSize (undefined :: a)) $ \dig -> do-        ((!_) :: B.Bytes) <- B.copy c $ \(ctx :: Ptr (H.Context a)) -> do-            B.withByteArray s $ \d ->-                hashInternalUpdate ctx d (fromIntegral $ B.length s)-            cshakeInternalFinalize ctx dig-        return ()-+cshakeFinalize+    :: forall a suffix+     . (HashSHAKE a, ByteArrayAccess suffix)+    => H.Context a -> suffix -> Digest a+cshakeFinalize !c s = Digest $+    allocAndFreezePrim (hashDigestSize (undefined :: a)) $+        \(dig :: Ptr (Digest a)) -> do+            ((!_) :: B.Bytes) <- B.copy c $ \(ctx :: Ptr (H.Context a)) -> do+                B.withByteArray s $ \d ->+                    hashInternalUpdate ctx d (fromIntegral $ B.length s)+                cshakeInternalFinalize ctx dig+            return ()  -- KMAC @@ -75,28 +86,31 @@ -- -- The Eq instance is constant time.  No Show instance is provided, to avoid -- printing by mistake.-newtype KMAC a = KMAC { kmacGetDigest :: Digest a }-    deriving (ByteArrayAccess,NFData)+newtype KMAC a = KMAC {kmacGetDigest :: Digest a}+    deriving (ByteArrayAccess, NFData)  instance Eq (KMAC a) where     (KMAC b1) == (KMAC b2) = B.constEq b1 b2  -- | Compute a KMAC using the supplied customization string and key.-kmac :: (HashSHAKE a, ByteArrayAccess string, ByteArrayAccess key, ByteArrayAccess ba)-     => string -> key -> ba -> KMAC a+kmac+    :: (HashSHAKE a, ByteArrayAccess string, ByteArrayAccess key, ByteArrayAccess ba)+    => string -> key -> ba -> KMAC a kmac str key msg = finalize $ updates (initialize str key) [msg]  -- | Represent an ongoing KMAC state, that can be appended with 'update' and--- finalized to a 'KMAC' with 'finalize'.+-- finalized to a t'KMAC' with 'finalize'. newtype Context a = Context (H.Context a)  -- | Initialize a new incremental KMAC context with the supplied customization -- string and key.-initialize :: forall a string key . (HashSHAKE a, ByteArrayAccess string, ByteArrayAccess key)-           => string -> key -> Context a+initialize+    :: forall a string key+     . (HashSHAKE a, ByteArrayAccess string, ByteArrayAccess key)+    => string -> key -> Context a initialize str key = Context $ cshakeInit n str p   where-    n = B.pack [75,77,65,67] :: B.Bytes  -- "KMAC"+    n = B.pack [75, 77, 65, 67] :: B.Bytes -- "KMAC"     w = hashBlockSize (undefined :: a)     p = buildAndFreeze (bytepad (encodeString key) w) :: B.ScrubbedBytes @@ -109,13 +123,12 @@ updates (Context ctx) = Context . cshakeUpdates ctx  -- | Finalize a KMAC context and return the KMAC.-finalize :: forall a . HashSHAKE a => Context a -> KMAC a+finalize :: forall a. HashSHAKE a => Context a -> KMAC a finalize (Context ctx) = KMAC $ cshakeFinalize ctx suffix   where     l = cshakeOutputLength (undefined :: a)     suffix = buildAndFreeze (rightEncode l) :: B.Bytes - -- Utilities  bytepad :: Builder -> Int -> Builder@@ -131,14 +144,15 @@ leftEncode x = byte len <> digits   where     digits = i2osp x-    len    = fromIntegral (builderLength digits)+    len = fromIntegral (builderLength digits)  rightEncode :: Int -> Builder rightEncode x = digits <> byte len   where     digits = i2osp x-    len    = fromIntegral (builderLength digits)+    len = fromIntegral (builderLength digits)  i2osp :: Int -> Builder-i2osp i | i >= 256  = i2osp (shiftR i 8) <> byte (fromIntegral i)-        | otherwise = byte (fromIntegral i)+i2osp i+    | i >= 256 = i2osp (shiftR i 8) <> byte (fromIntegral i)+    | otherwise = byte (fromIntegral i)
Crypto/MAC/KeyedBlake2.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# LANGUAGE ScopedTypeVariables #-}+ -- | -- Module      : Crypto.MAC.KeyedBlake2 -- License     : BSD-style@@ -7,33 +10,29 @@ -- -- Expose a MAC interface to the keyed Blake2 algorithms -- defined in RFC 7693.----{-# LANGUAGE GeneralizedNewtypeDeriving #-}-{-# LANGUAGE ScopedTypeVariables #-}+module Crypto.MAC.KeyedBlake2 (+    HashBlake2,+    KeyedBlake2 (..),+    keyedBlake2,+    keyedBlake2Lazy, -module Crypto.MAC.KeyedBlake2-    ( HashBlake2-    , KeyedBlake2(..)-    , keyedBlake2-    , keyedBlake2Lazy     -- * Incremental-    , Context-    , initialize-    , update-    , updates-    , finalize-    ) where+    Context,+    initialize,+    update,+    updates,+    finalize,+) where  import qualified Crypto.Hash as H+import Crypto.Hash.Blake2 import qualified Crypto.Hash.Types as H-import           Crypto.Hash.Blake2-import           Crypto.Internal.DeepSeq (NFData)+import Crypto.Internal.DeepSeq (NFData)+import Data.ByteArray (ByteArrayAccess) import qualified Data.ByteArray as B-import           Data.ByteArray (ByteArrayAccess) import qualified Data.ByteString.Lazy as L -import           Foreign.Ptr (Ptr)-+import Foreign.Ptr (Ptr)  -- Keyed Blake2b @@ -42,28 +41,31 @@ -- -- The Eq instance is constant time.  No Show instance is provided, to avoid -- printing by mistake.-newtype KeyedBlake2 a = KeyedBlake2 { keyedBlake2GetDigest :: H.Digest a }-    deriving (ByteArrayAccess,NFData)+newtype KeyedBlake2 a = KeyedBlake2 {keyedBlake2GetDigest :: H.Digest a}+    deriving (ByteArrayAccess, NFData)  instance Eq (KeyedBlake2 a) where     KeyedBlake2 x == KeyedBlake2 y = B.constEq x y  -- | Represent an ongoing Blake2 state, that can be appended with 'update' and--- finalized to a 'KeyedBlake2' with 'finalize'.+-- finalized to a t'KeyedBlake2' with 'finalize'. newtype Context a = Context (H.Context a)  -- | Initialize a new incremental keyed Blake2 context with the supplied key.-initialize :: forall a key . (HashBlake2 a, ByteArrayAccess key)-           => key -> Context a+initialize+    :: forall a key+     . (HashBlake2 a, ByteArrayAccess key)+    => key -> Context a initialize k = Context $ H.Context $ B.allocAndFreeze ctxSz performInit-    where ctxSz = H.hashInternalContextSize (undefined :: a)-          digestSz = H.hashDigestSize (undefined :: a)-          -- cap the number of key bytes at digestSz,-          -- since that's the maximal key size-          keyByteLen = min (B.length k) digestSz-          performInit :: Ptr (H.Context a) -> IO ()-          performInit ptr = B.withByteArray k-            $ \keyPtr -> blake2InternalKeyedInit ptr keyPtr (fromIntegral keyByteLen)+  where+    ctxSz = H.hashInternalContextSize (undefined :: a)+    digestSz = H.hashDigestSize (undefined :: a)+    -- cap the number of key bytes at digestSz,+    -- since that's the maximal key size+    keyByteLen = min (B.length k) digestSz+    performInit :: Ptr (H.Context a) -> IO ()+    performInit ptr = B.withByteArray k $+        \keyPtr -> blake2InternalKeyedInit ptr keyPtr (fromIntegral keyByteLen)  -- | Incrementally update a keyed Blake2 context. update :: (HashBlake2 a, ByteArrayAccess ba) => Context a -> ba -> Context a@@ -78,11 +80,13 @@ finalize (Context ctx) = KeyedBlake2 $ H.hashFinalize ctx  -- | Compute a Blake2 MAC using the supplied key.-keyedBlake2 :: (HashBlake2 a, ByteArrayAccess key, ByteArrayAccess ba)-            => key -> ba -> KeyedBlake2 a+keyedBlake2+    :: (HashBlake2 a, ByteArrayAccess key, ByteArrayAccess ba)+    => key -> ba -> KeyedBlake2 a keyedBlake2 key msg = finalize $ update (initialize key) msg  -- | Compute a Blake2 MAC using the supplied key, for a lazy input.-keyedBlake2Lazy :: (HashBlake2 a, ByteArrayAccess key)-            => key -> L.ByteString -> KeyedBlake2 a+keyedBlake2Lazy+    :: (HashBlake2 a, ByteArrayAccess key)+    => key -> L.ByteString -> KeyedBlake2 a keyedBlake2Lazy key msg = finalize $ updates (initialize key) (L.toChunks msg)
Crypto/MAC/Poly1305.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}  -- | -- Module      : Crypto.MAC.Poly1305@@ -7,30 +9,36 @@ -- Portability : unknown -- -- Poly1305 implementation----{-# LANGUAGE ForeignFunctionInterface #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.MAC.Poly1305-    ( Ctx-    , State-    , Auth(..)-    , authTag+module Crypto.MAC.Poly1305 (+    Ctx,+    State,+    Key,+    key,+    Auth (..),+    authTag,+     -- * Incremental MAC Functions-    , initialize -- :: State-    , update     -- :: State -> ByteString -> State-    , updates    -- :: State -> [ByteString] -> State-    , finalize   -- :: State -> Auth+    initialize, -- :: State+    update, -- :: State -> ByteString -> State+    updates, -- :: State -> [ByteString] -> State+    finalize, -- :: State -> Auth+     -- * One-pass MAC function-    , auth-    ) where+    auth,+) where -import           Foreign.Ptr-import           Foreign.C.Types-import           Data.Word-import           Crypto.Internal.ByteArray (ByteArrayAccess, ScrubbedBytes, Bytes)+import Crypto.Error+import Crypto.Internal.ByteArray (+    ByteArrayAccess,+    Bytes,+    ScrubbedBytes,+ ) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Internal.DeepSeq-import           Crypto.Error+import Crypto.Internal.DeepSeq+import Crypto.Internal.Poly1305 (Key (..), key)+import Data.Word+import Foreign.C.Types+import Foreign.Ptr  -- | Poly1305 State --@@ -43,20 +51,27 @@  -- | Poly1305 State. use State instead of Ctx type Ctx = State+ {-# DEPRECATED Ctx "use Poly1305 State instead" #-}  -- | Poly1305 Auth newtype Auth = Auth Bytes-    deriving (ByteArrayAccess,NFData)+    deriving (ByteArrayAccess, NFData)  authTag :: ByteArrayAccess b => b -> CryptoFailable Auth authTag b     | B.length b /= 16 = CryptoFailed $ CryptoError_AuthenticationTagSizeInvalid-    | otherwise        = CryptoPassed $ Auth $ B.convert b+    | otherwise = CryptoPassed $ Auth $ B.convert b  instance Eq Auth where     (Auth a1) == (Auth a2) = B.constEq a1 a2 +-- | @sizeof(poly1305_ctx)@: the accumulator and the key, either as the+-- limbs the C implementation works in or as the state the assembly keeps,+-- and the buffer for a partial block.  See @cbits/crypton_poly1305.h@.+sizeCtx :: Int+sizeCtx = 232+ foreign import ccall unsafe "crypton_poly1305.h crypton_poly1305_init"     c_poly1305_init :: Ptr State -> Ptr Word8 -> IO () @@ -67,50 +82,56 @@     c_poly1305_finalize :: Ptr Word8 -> Ptr State -> IO ()  -- | initialize a Poly1305 context-initialize :: ByteArrayAccess key-           => key-           -> CryptoFailable State-initialize key-    | B.length key /= 32 = CryptoFailed $ CryptoError_MacKeyInvalid-    | otherwise          = CryptoPassed $ State $ B.allocAndFreeze 84 $ \ctxPtr ->-        B.withByteArray key $ \keyPtr ->-            c_poly1305_init (castPtr ctxPtr) keyPtr+initialize :: Key -> State+initialize k = State $ B.allocAndFreeze sizeCtx $ \ctxPtr ->+    B.withByteArray k $ \keyPtr ->+        c_poly1305_init (castPtr ctxPtr) keyPtr {-# NOINLINE initialize #-}  -- | update a context with a bytestring update :: ByteArrayAccess ba => State -> ba -> State update (State prevCtx) d = State $ B.copyAndFreeze prevCtx $ \ctxPtr ->     B.withByteArray d $ \dataPtr ->-        c_poly1305_update (castPtr ctxPtr) dataPtr (fromIntegral $ B.length d)+        -- in pieces the C's uint32_t length can hold; the context carries+        -- across, so it can simply be called again+        B.inCLengths (B.length d) $ \off n ->+            c_poly1305_update (castPtr ctxPtr) (dataPtr `plusPtr` off) (fromIntegral n) {-# NOINLINE update #-}  -- | updates a context with multiples bytestring updates :: ByteArrayAccess ba => State -> [ba] -> State updates (State prevCtx) d = State $ B.copyAndFreeze prevCtx (loop d)-  where loop []     _      = return ()-        loop (x:xs) ctxPtr = do-            B.withByteArray x $ \dataPtr -> c_poly1305_update ctxPtr dataPtr (fromIntegral $ B.length x)-            loop xs ctxPtr+  where+    loop [] _ = return ()+    loop (x : xs) ctxPtr = do+        B.withByteArray x $ \dataPtr ->+            B.inCLengths (B.length x) $ \off n ->+                c_poly1305_update ctxPtr (dataPtr `plusPtr` off) (fromIntegral n)+        loop xs ctxPtr {-# NOINLINE updates #-}  -- | finalize the context into a digest bytestring finalize :: State -> Auth finalize (State prevCtx) = Auth $ B.allocAndFreeze 16 $ \dst -> do-    _ <- B.copy prevCtx (\ctxPtr -> c_poly1305_finalize dst (castPtr ctxPtr)) :: IO ScrubbedBytes+    _ <-+        B.copy prevCtx (\ctxPtr -> c_poly1305_finalize dst (castPtr ctxPtr))+            :: IO ScrubbedBytes     return () {-# NOINLINE finalize #-}  -- | One-pass authorization creation-auth :: (ByteArrayAccess key, ByteArrayAccess ba) => key -> ba -> Auth-auth key d-    | B.length key /= 32 = error "Poly1305: key length expected 32 bytes"-    | otherwise          = Auth $ B.allocAndFreeze 16 $ \dst -> do-        _ <- B.alloc 84 (onCtx dst) :: IO ScrubbedBytes-        return ()+auth :: ByteArrayAccess ba => Key -> ba -> Auth+auth k d = Auth $ B.allocAndFreeze 16 $ \dst -> do+    _ <- B.alloc sizeCtx (onCtx dst) :: IO ScrubbedBytes+    return ()   where-        onCtx dst ctxPtr =-            B.withByteArray key $ \keyPtr -> do-                c_poly1305_init (castPtr ctxPtr) keyPtr-                B.withByteArray d $ \dataPtr ->-                    c_poly1305_update (castPtr ctxPtr) dataPtr (fromIntegral $ B.length d)-                c_poly1305_finalize dst (castPtr ctxPtr)+    onCtx dst ctxPtr =+        B.withByteArray k $ \keyPtr -> do+            c_poly1305_init (castPtr ctxPtr) keyPtr+            B.withByteArray d $ \dataPtr ->+                B.inCLengths (B.length d) $ \off n ->+                    c_poly1305_update+                        (castPtr ctxPtr)+                        (dataPtr `plusPtr` off)+                        (fromIntegral n)+            c_poly1305_finalize dst (castPtr ctxPtr)
Crypto/Number/Basic.hs view
@@ -1,20 +1,20 @@+{-# LANGUAGE BangPatterns #-}+ -- | -- Module      : Crypto.Number.Basic -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good--{-# LANGUAGE BangPatterns #-}-module Crypto.Number.Basic-    ( sqrti-    , gcde-    , areEven-    , log2-    , numBits-    , numBytes-    , asPowerOf2AndOdd-    ) where+module Crypto.Number.Basic (+    sqrti,+    gcde,+    areEven,+    log2,+    numBits,+    numBytes,+    asPowerOf2AndOdd,+) where  import Data.Bits @@ -25,46 +25,48 @@ -- and use a dichotomy algorithm to compute the bound relatively efficiently. sqrti :: Integer -> (Integer, Integer) sqrti i-    | i < 0     = error "cannot compute negative square root"-    | i == 0    = (0,0)-    | i == 1    = (1,1)-    | i == 2    = (1,2)+    | i < 0 = error "cannot compute negative square root"+    | i == 0 = (0, 0)+    | i == 1 = (1, 1)+    | i == 2 = (1, 2)     | otherwise = loop x0-        where-            nbdigits = length $ show i-            x0n = (if even nbdigits then nbdigits - 2 else nbdigits - 1) `div` 2-            x0  = if even nbdigits then 2 * 10 ^ x0n else 6 * 10 ^ x0n-            loop x = case compare (sq x) i of-                LT -> iterUp x-                EQ -> (x, x)-                GT -> iterDown x-            iterUp lb = if sq ub >= i then iter lb ub else iterUp ub-                where ub = lb * 2-            iterDown ub = if sq lb >= i then iterDown lb else iter lb ub-                where lb = ub `div` 2-            iter lb ub-                | lb == ub   = (lb, ub)-                | lb+1 == ub = (lb, ub)-                | otherwise  =-                    let d = (ub - lb) `div` 2 in-                    if sq (lb + d) >= i-                        then iter lb (ub-d)-                        else iter (lb+d) ub-            sq a = a * a+  where+    nbdigits = length $ show i+    x0n = (if even nbdigits then nbdigits - 2 else nbdigits - 1) `div` 2+    x0 = if even nbdigits then 2 * 10 ^ x0n else 6 * 10 ^ x0n+    loop x = case compare (sq x) i of+        LT -> iterUp x+        EQ -> (x, x)+        GT -> iterDown x+    iterUp lb = if sq ub >= i then iter lb ub else iterUp ub+      where+        ub = lb * 2+    iterDown ub = if sq lb >= i then iterDown lb else iter lb ub+      where+        lb = ub `div` 2+    iter lb ub+        | lb == ub = (lb, ub)+        | lb + 1 == ub = (lb, ub)+        | otherwise =+            let d = (ub - lb) `div` 2+             in if sq (lb + d) >= i+                    then iter lb (ub - d)+                    else iter (lb + d) ub+    sq a = a * a  -- | Get the extended GCD of two integer using integer divMod ----- gcde 'a' 'b' find (x,y,gcd(a,b)) where ax + by = d---+-- gcde @a@ @b@ find (x,y,gcd(a,b)) where ax + by = d gcde :: Integer -> Integer -> (Integer, Integer, Integer)-gcde a b = onGmpUnsupported (gmpGcde a b) $-    if d < 0 then (-x,-y,-d) else (x,y,d)+gcde a b =+    onGmpUnsupported (gmpGcde a b) $+        if d < 0 then (-x, -y, -d) else (x, y, d)   where-    (d, x, y)                     = f (a,1,0) (b,0,1)-    f t              (0, _, _)    = t+    (d, x, y) = f (a, 1, 0) (b, 0, 1)+    f t (0, _, _) = t     f (a', sa, ta) t@(b', sb, tb) =-        let (q, r) = a' `divMod` b' in-        f t (r, sa - (q * sb), ta - (q * tb))+        let (q, r) = a' `divMod` b'+         in f t (r, sa - (q * sb), ta - (q * tb))  -- | Check if a list of integer are all even areEven :: [Integer] -> Bool@@ -75,7 +77,7 @@ log2 n = onGmpUnsupported (gmpLog2 n) $ imLog 2 n   where     -- http://www.haskell.org/pipermail/haskell-cafe/2008-February/039465.html-    imLog b x = if x < b then 0 else (x `div` b^l) `doDiv` l+    imLog b x = if x < b then 0 else (x `div` b ^ l) `doDiv` l       where         l = 2 * imLog (b * b) x         doDiv x' l' = if x' < b then l' else (x' `div` b) `doDiv` (l' + 1)@@ -83,34 +85,65 @@  -- | Compute the number of bits for an integer numBits :: Integer -> Int-numBits n = gmpSizeInBits n `onGmpUnsupported` (if n == 0 then 1 else computeBits 0 n)-  where computeBits !acc i-            | q == 0 =-                if r >= 0x80 then acc+8-                else if r >= 0x40 then acc+7-                else if r >= 0x20 then acc+6-                else if r >= 0x10 then acc+5-                else if r >= 0x08 then acc+4-                else if r >= 0x04 then acc+3-                else if r >= 0x02 then acc+2-                else if r >= 0x01 then acc+1-                else acc -- should be catch by previous loop-            | otherwise = computeBits (acc+8) q-          where (q,r) = i `divMod` 256+-- GMP sizes the magnitude and calls zero zero bits, and every caller here --+-- 'numBytes' above all -- is written against that.  The fallback used to+-- answer 1 for zero, and to divide a negative number by 256 forever, because+-- the quotient never reaches zero.+numBits n =+    gmpSizeInBits n `onGmpUnsupported` (if n == 0 then 0 else computeBits 0 (abs n))+  where+    computeBits !acc i+        | q == 0 =+            if r >= 0x80+                then acc + 8+                else+                    if r >= 0x40+                        then acc + 7+                        else+                            if r >= 0x20+                                then acc + 6+                                else+                                    if r >= 0x10+                                        then acc + 5+                                        else+                                            if r >= 0x08+                                                then acc + 4+                                                else+                                                    if r >= 0x04+                                                        then acc + 3+                                                        else+                                                            if r >= 0x02+                                                                then acc + 2+                                                                else+                                                                    if r >= 0x01+                                                                        then acc + 1+                                                                        else acc -- should be catch by previous loop+        | otherwise = computeBits (acc + 8) q+      where+        (q, r) = i `divMod` 256  -- | Compute the number of bytes for an integer+--+-- Out of 'numBits' rather than out of GMP's own count in base 256.  GHC's+-- bignum sizes a number in base two by looking at its highest limb, and in+-- any other base -- 256 included -- by dividing the number down to nothing:+-- on a 2048-bit modulus that is 1.65 us against 0.01, and every serialization+-- here asks for the size before it allocates.  Eight bits to the byte does+-- the rest. numBytes :: Integer -> Int-numBytes n = gmpSizeInBytes n `onGmpUnsupported` ((numBits n + 7) `div` 8)+numBytes n = (numBits n + 7) `div` 8  -- | Express an integer as an odd number and a power of 2 asPowerOf2AndOdd :: Integer -> (Int, Integer) asPowerOf2AndOdd a-    | a == 0       = (0, 0)-    | odd a        = (0, a)-    | a < 0        = let (e, a1) = asPowerOf2AndOdd $ abs a in (e, -a1)+    | a == 0 = (0, 0)+    | odd a = (0, a)+    | a < 0 = let (e, a1) = asPowerOf2AndOdd $ abs a in (e, -a1)     | isPowerOf2 a = (log2 a, 1)-    | otherwise    = loop a 0-        where      -          isPowerOf2 n = (n /= 0) && ((n .&. (n - 1)) == 0)-          loop n pw = if n `mod` 2 == 0 then loop (n `div` 2) (pw + 1)-                      else (pw, n)+    | otherwise = loop a 0+  where+    isPowerOf2 n = (n /= 0) && ((n .&. (n - 1)) == 0)+    loop n pw =+        if n `mod` 2 == 0+            then loop (n `div` 2) (pw + 1)+            else (pw, n)
Crypto/Number/Compat.hs view
@@ -1,31 +1,32 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE CPP #-}+{-# LANGUAGE MagicHash #-}+{-# LANGUAGE UnboxedTuples #-}+{-# OPTIONS_GHC -Wno-deprecations #-}+ -- | -- Module      : Crypto.Number.Compat -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----{-# LANGUAGE CPP           #-}-{-# LANGUAGE MagicHash     #-}-{-# LANGUAGE BangPatterns  #-}-{-# LANGUAGE UnboxedTuples #-}-module Crypto.Number.Compat-    ( GmpSupported(..)-    , onGmpUnsupported-    , gmpGcde-    , gmpLog2-    , gmpPowModSecInteger-    , gmpPowModInteger-    , gmpInverse-    , gmpNextPrime-    , gmpTestPrimeMillerRabin-    , gmpSizeInBytes-    , gmpSizeInBits-    , gmpExportInteger-    , gmpExportIntegerLE-    , gmpImportInteger-    , gmpImportIntegerLE-    ) where+module Crypto.Number.Compat (+    GmpSupported (..),+    onGmpUnsupported,+    gmpGcde,+    gmpLog2,+    gmpPowModSecInteger,+    gmpPowModInteger,+    gmpInverse,+    gmpNextPrime,+    gmpTestPrimeMillerRabin,+    gmpSizeInBytes,+    gmpSizeInBits,+    gmpExportInteger,+    gmpExportIntegerLE,+    gmpImportInteger,+    gmpImportIntegerLE,+) where  #ifndef MIN_VERSION_integer_gmp #define MIN_VERSION_integer_gmp(a,b,c) 0@@ -37,17 +38,18 @@ import GHC.Integer.Logarithms (integerLog2#) #endif import Data.Word-import GHC.Ptr (Ptr(..))+import GHC.Ptr (Ptr (..))  -- | GMP Supported / Unsupported-data GmpSupported a = GmpSupported a-                    | GmpUnsupported-                    deriving (Show,Eq)+data GmpSupported a+    = GmpSupported a+    | GmpUnsupported+    deriving (Show, Eq)  -- | Simple combinator in case the operation is not supported through GMP onGmpUnsupported :: GmpSupported a -> a -> a onGmpUnsupported (GmpSupported a) _ = a-onGmpUnsupported GmpUnsupported   f = f+onGmpUnsupported GmpUnsupported f = f  -- | Compute the GCDE of a two integer through GMP gmpGcde :: Integer -> Integer -> GmpSupported (Integer, Integer, Integer)
Crypto/Number/F2m.hs view
@@ -7,33 +7,47 @@ -- -- This module provides basic arithmetic operations over F₂m. Performance is -- not optimal and it doesn't provide protection against timing--- attacks. The 'm' parameter is implicitly derived from the irreducible+-- attacks. The @m@ parameter is implicitly derived from the irreducible -- polynomial where applicable.--module Crypto.Number.F2m-    ( BinaryPolynomial-    , addF2m-    , mulF2m-    , squareF2m'-    , squareF2m-    , powF2m-    , modF2m-    , sqrtF2m-    , invF2m-    , divF2m-    ) where+module Crypto.Number.F2m (+    BinaryPolynomial,+    addF2m,+    mulF2m,+    squareF2m',+    squareF2m,+    powF2m,+    modF2m,+    sqrtF2m,+    invF2m,+    divF2m,+    quadraticF2m,+) where -import Data.Bits (xor, shift, testBit, setBit)-import Data.List+import Crypto.Internal.WordArray import Crypto.Number.Basic+import Data.Bits (+    setBit,+    shift,+    shiftL,+    shiftR,+    testBit,+    unsafeShiftR,+    xor,+    (.&.),+    (.|.),+ )+import Data.List (foldl')+import Data.Word (Word32)+import Prelude hiding (foldl')  -- | Binary Polynomial represented by an integer type BinaryPolynomial = Integer  -- | Addition over F₂m. This is just a synonym of 'xor'.-addF2m :: Integer-       -> Integer-       -> Integer+addF2m+    :: Integer+    -> Integer+    -> Integer addF2m = xor {-# INLINE addF2m #-} @@ -41,50 +55,127 @@ -- -- This function is undefined for negative arguments, because their bit -- representation is platform-dependent. Zero modulus is also prohibited.-modF2m :: BinaryPolynomial -- ^ Modulus-       -> Integer-       -> Integer+modF2m+    :: BinaryPolynomial+    -- ^ Modulus+    -> Integer+    -> Integer modF2m fx i-    | fx < 0 || i < 0 = error "modF2m: negative number represent no binary polynomial"-    | fx == 0         = error "modF2m: cannot divide by zero polynomial"-    | fx == 1         = 0-    | otherwise       = go i+    | fx < 0 || i < 0 =+        error "modF2m: negative number represent no binary polynomial"+    | fx == 0 = error "modF2m: cannot divide by zero polynomial"+    | fx == 1 = 0+    | otherwise = case tailExponents fx of+        Just es -> fold es i+        Nothing -> go i+  where+    lfx = log2 fx+    -- one bit at a time, for a modulus with too many terms to be worth the+    -- other way+    go n+        | s == 0 = n `addF2m` fx+        | s < 0 = n+        | otherwise = go $ n `addF2m` shift fx s       where-        lfx = log2 fx-        go n | s == 0    = n `addF2m` fx-             | s < 0     = n-             | otherwise = go $ n `addF2m` shift fx s-                where s = log2 n - lfx+        s = log2 n - lfx++    -- x^m is the rest of the modulus, so everything above bit m folds back in+    -- as a copy of the number's top shifted by each of the modulus's lower+    -- exponents: a handful of shifts, where the loop above takes one step per+    -- bit of excess+    mask = (1 `shiftL` lfx) - 1+    fold es n+        | n <= mask = n+        | otherwise =+            fold+                es+                (foldl' (\acc e -> acc `xor` (hi `shiftL` e)) (n .&. mask) es)+      where+        hi = n `shiftR` lfx {-# INLINE modF2m #-} +-- | The exponents of a modulus below its leading term, when there are few+-- enough of them to reduce with.+--+-- Every binary curve in use has a trinomial or a pentanomial here, which is+-- three or five exponents; sixteen is the point past which folding stops being+-- the cheaper way.+tailExponents :: BinaryPolynomial -> Maybe [Int]+tailExponents fx = go (log2 fx - 1) 0 []+  where+    go i n acc+        | i < 0 = Just acc+        | n > 16 = Nothing+        | testBit fx i = go (i - 1) (n + 1 :: Int) (i : acc)+        | otherwise = go (i - 1) n acc+ -- | Multiplication over F₂m. -- -- This function is undefined for negative arguments, because their bit -- representation is platform-dependent. Zero modulus is also prohibited.-mulF2m :: BinaryPolynomial -- ^ Modulus-       -> Integer-       -> Integer-       -> Integer+mulF2m+    :: BinaryPolynomial+    -- ^ Modulus+    -> Integer+    -> Integer+    -> Integer mulF2m fx n1 n2-    |    fx < 0-      || n1 < 0-      || n2 < 0 = error "mulF2m: negative number represent no binary polynomial"-    | fx == 0   = error "mulF2m: cannot multiply modulo zero polynomial"-    | otherwise = modF2m fx $ go (if n2 `mod` 2 == 1 then n1 else 0) (log2 n2)-      where-        go n s | s == 0  = n-               | otherwise = if testBit n2 s-                                then go (n `addF2m` shift n1 s) (s - 1)-                                else go n (s - 1)-{-# INLINABLE mulF2m #-}+    | fx < 0+        || n1 < 0+        || n2 < 0 =+        error "mulF2m: negative number represent no binary polynomial"+    | fx == 0 = error "mulF2m: cannot multiply modulo zero polynomial"+    | otherwise = modF2m fx (go n2 0 0)+  where+    -- Four bits of the multiplier at a time, against the sixteen multiples of+    -- n1 that four bits can ask for.  A bit at a time is four times the+    -- shifting and exclusive-oring, and each of those allocates.+    go 0 _ acc = acc+    go v sh acc =+        go (v `shiftR` 4) (sh + 4) (acc `xor` (multiple (v .&. 0xf) `shiftL` sh)) +    m2 = n1 `shiftL` 1+    m4 = n1 `shiftL` 2+    m8 = n1 `shiftL` 3+    m3 = m2 `xor` n1+    m5 = m4 `xor` n1+    m6 = m4 `xor` m2+    m7 = m6 `xor` n1+    m9 = m8 `xor` n1+    m10 = m8 `xor` m2+    m11 = m10 `xor` n1+    m12 = m8 `xor` m4+    m13 = m12 `xor` n1+    m14 = m12 `xor` m2+    m15 = m14 `xor` n1++    multiple 1 = n1+    multiple 2 = m2+    multiple 3 = m3+    multiple 4 = m4+    multiple 5 = m5+    multiple 6 = m6+    multiple 7 = m7+    multiple 8 = m8+    multiple 9 = m9+    multiple 10 = m10+    multiple 11 = m11+    multiple 12 = m12+    multiple 13 = m13+    multiple 14 = m14+    multiple 15 = m15+    multiple _ = 0+{-# INLINEABLE mulF2m #-}+ -- | Squaring over F₂m. -- -- This function is undefined for negative arguments, because their bit -- representation is platform-dependent. Zero modulus is also prohibited.-squareF2m :: BinaryPolynomial -- ^ Modulus-          -> Integer-          -> Integer+squareF2m+    :: BinaryPolynomial+    -- ^ Modulus+    -> Integer+    -> Integer squareF2m fx = modF2m fx . squareF2m' {-# INLINE squareF2m #-} @@ -95,75 +186,141 @@ -- -- This function is undefined for negative arguments, because their bit -- representation is platform-dependent.-squareF2m' :: Integer-           -> Integer+squareF2m'+    :: Integer+    -> Integer squareF2m' n-    | n < 0     = error "mulF2m: negative number represent no binary polynomial"-    | otherwise = foldl' (\acc s -> if testBit n s then setBit acc (2 * s) else acc) 0 [0 .. log2 n]+    | n < 0 = error "mulF2m: negative number represent no binary polynomial"+    | otherwise = go n 0 0+  where+    -- A byte at a time, through a table of the sixteen-bit patterns a byte+    -- spreads into.  A bit at a time is eight times the work, and setting a+    -- bit of an Integer allocates another one.+    go 0 _ acc = acc+    go v sh acc =+        go+            (v `shiftR` 8)+            (sh + 16)+            ( acc+                .|. (fromIntegral (arrayRead32 spreadTable (fromIntegral (v .&. 0xff))) `shiftL` sh)+            )++-- | Each byte, with a zero inserted between every pair of its bits.+spreadTable :: Array32+spreadTable = array32 256 [spread b | b <- [0 .. 255]]+  where+    spread :: Int -> Word32+    spread b =+        foldl' (\acc i -> if testBit b i then setBit acc (2 * i) else acc) 0 [0 .. 7]+{-# NOINLINE spreadTable #-}+ {-# INLINE squareF2m' #-}  -- | Exponentiation in F₂m by computing @a^b mod fx@. -- -- This implements an exponentiation by squaring based solution. It inherits the -- same restrictions as 'squareF2m'. Negative exponents are disallowed.-powF2m :: BinaryPolynomial -- ^Modulus-       -> Integer          -- ^a-       -> Integer          -- ^b-       -> Integer+powF2m+    :: BinaryPolynomial+    -- ^ Modulus+    -> Integer+    -- ^ a+    -> Integer+    -- ^ b+    -> Integer powF2m fx a b-  | b < 0     = error "powF2m: negative exponents disallowed"-  | b == 0    = if fx > 1 then 1 else 0-  | even b    = squareF2m fx x-  | otherwise = mulF2m fx a (squareF2m' x)-  where x = powF2m fx a (b `div` 2)+    | b < 0 = error "powF2m: negative exponents disallowed"+    | b == 0 = if fx > 1 then 1 else 0+    | even b = squareF2m fx x+    | otherwise = mulF2m fx a (squareF2m' x)+  where+    x = powF2m fx a (b `div` 2)  -- | Square rooot in F₂m. -- -- We exploit the fact that @a^(2^m) = a@, or in particular, @a^(2^m - 1) = 1@ -- from a classical result by Lagrange. Thus the square root is simply @a^(2^(m -- - 1))@.-sqrtF2m :: BinaryPolynomial -- ^Modulus-        -> Integer          -- ^a-        -> Integer+sqrtF2m+    :: BinaryPolynomial+    -- ^ Modulus+    -> Integer+    -- ^ a+    -> Integer sqrtF2m fx a = go (log2 fx - 1) a-  where go 0 x = x-        go n x = go (n - 1) (squareF2m fx x)+  where+    go 0 x = x+    go n x = go (n - 1) (squareF2m fx x)  -- | Extended GCD algorithm for polynomials. For @a@ and @b@ returns @(g, u, v)@ such that @a * u + b * v == g@. -- -- Reference: https://en.wikipedia.org/wiki/Polynomial_greatest_common_divisor#B.C3.A9zout.27s_identity_and_extended_GCD_algorithm-gcdF2m :: Integer-       -> Integer-       -> (Integer, Integer, Integer)+gcdF2m+    :: Integer+    -> Integer+    -> (Integer, Integer, Integer) gcdF2m a b = go (a, b, 1, 0, 0, 1)   where-    go (g, 0, u, _, v, _)-        = (g, u, v)-    go (r0, r1, s0, s1, t0, t1)-        = go (r1, r0 `addF2m` shift r1 j, s1, s0 `addF2m` shift s1 j, t1, t0 `addF2m` shift t1 j)-            where j = max 0 (log2 r0 - log2 r1)+    go (g, 0, u, _, v, _) =+        (g, u, v)+    go (r0, r1, s0, s1, t0, t1) =+        go+            ( r1+            , r0 `addF2m` shift r1 j+            , s1+            , s0 `addF2m` shift s1 j+            , t1+            , t0 `addF2m` shift t1 j+            )+      where+        j = max 0 (log2 r0 - log2 r1)  -- | Modular inversion over F₂m. -- If @n@ doesn't have an inverse, 'Nothing' is returned. -- -- This function is undefined for negative arguments, because their bit -- representation is platform-dependent. Zero modulus is also prohibited.-invF2m :: BinaryPolynomial -- ^ Modulus-       -> Integer-       -> Maybe Integer+invF2m+    :: BinaryPolynomial+    -- ^ Modulus+    -> Integer+    -> Maybe Integer invF2m fx n = if g == 1 then Just (modF2m fx u) else Nothing   where     (g, u, _) = gcdF2m n fx-{-# INLINABLE invF2m #-}+{-# INLINEABLE invF2m #-}  -- | Division over F₂m. If the dividend doesn't have an inverse it returns -- 'Nothing'. -- -- This function is undefined for negative arguments, because their bit -- representation is platform-dependent. Zero modulus is also prohibited.-divF2m :: BinaryPolynomial -- ^ Modulus-       -> Integer          -- ^ Dividend-       -> Integer          -- ^ Divisor-       -> Maybe Integer    -- ^ Quotient+divF2m+    :: BinaryPolynomial+    -- ^ Modulus+    -> Integer+    -- ^ Dividend+    -> Integer+    -- ^ Divisor+    -> Maybe Integer+    -- ^ Quotient divF2m fx n1 n2 = mulF2m fx n1 <$> invF2m fx n2 {-# INLINE divF2m #-}++traceF2m :: BinaryPolynomial -> Integer -> Integer+traceF2m fx = foldr addF2m 0 . take (log2 fx) . iterate (squareF2m fx)+{-# INLINE traceF2m #-}++halfTraceF2m :: BinaryPolynomial -> Integer -> Integer+halfTraceF2m fx =+    foldr addF2m 0+        . take (1 + log2 fx `unsafeShiftR` 1)+        . iterate (squareF2m fx . squareF2m fx)+{-# INLINE halfTraceF2m #-}++-- | Solve a quadratic equation of the form @x^2 + x = a@ in F₂m.+quadraticF2m :: BinaryPolynomial -> Integer -> Maybe Integer+quadraticF2m fx a+    | traceF2m fx a == 0 = Just $ halfTraceF2m fx a+    | otherwise = Nothing+{-# INLINEABLE quadraticF2m #-}
Crypto/Number/Generate.hs view
@@ -4,31 +4,32 @@ -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good--module Crypto.Number.Generate-    ( GenTopPolicy(..)-    , generateParams-    , generateMax-    , generateBetween-    ) where+module Crypto.Number.Generate (+    GenTopPolicy (..),+    generateParams,+    generatePrefix,+    generateMax,+    generateBetween,+) where -import           Crypto.Internal.Imports-import           Crypto.Number.Basic-import           Crypto.Number.Serialize-import           Crypto.Random.Types-import           Control.Monad (when)-import           Foreign.Ptr-import           Foreign.Storable-import           Data.Bits ((.|.), (.&.), shiftL, complement, testBit)-import           Crypto.Internal.ByteArray (ScrubbedBytes)+import Control.Monad (when)+import Crypto.Internal.ByteArray (ScrubbedBytes) import qualified Crypto.Internal.ByteArray as B-+import Crypto.Internal.Imports+import Crypto.Number.Basic+import Crypto.Number.Serialize+import Crypto.Random.Types+import Data.Bits (complement, shiftL, testBit, unsafeShiftR, (.&.), (.|.))+import Foreign.Ptr+import Foreign.Storable  -- | Top bits policy when generating a number-data GenTopPolicy =-      SetHighest    -- ^ set the highest bit-    | SetTwoHighest -- ^ set the two highest bit-    deriving (Show,Eq)+data GenTopPolicy+    = -- | set the highest bit+      SetHighest+    | -- | set the two highest bit+      SetTwoHighest+    deriving (Show, Eq)  -- | Generate a number for a specific size of bits, -- and optionaly set bottom and top bits@@ -38,27 +39,31 @@ -- -- If @generateOdd is set to 'True', then the number generated -- is guaranteed to be odd. Otherwise it will be whatever is generated----generateParams :: MonadRandom m-               => Int                -- ^ number of bits-               -> Maybe GenTopPolicy -- ^ top bit policy-               -> Bool               -- ^ force the number to be odd-               -> m Integer+generateParams+    :: MonadRandom m+    => Int+    -- ^ number of bits+    -> Maybe GenTopPolicy+    -- ^ top bit policy+    -> Bool+    -- ^ force the number to be odd+    -> m Integer generateParams bits genTopPolicy generateOdd     | bits <= 0 = return 0     | otherwise = os2ip . tweak <$> getRandomBytes bytes   where     tweak :: ScrubbedBytes -> ScrubbedBytes     tweak orig = B.copyAndFreeze orig $ \p0 -> do-        let p1   = p0 `plusPtr` 1+        let p1 = p0 `plusPtr` 1             pEnd = p0 `plusPtr` (bytes - 1)         case genTopPolicy of-            Nothing             -> return ()-            Just SetHighest     -> p0 |= (1 `shiftL` bit)+            Nothing -> return ()+            Just SetHighest -> p0 |= (1 `shiftL` bit)             Just SetTwoHighest-                | bit == 0      -> do p0 $= 0x1-                                      p1 |= 0x80-                | otherwise     -> p0 |= (0x3 `shiftL` (bit - 1))+                | bit == 0 -> do+                    p0 $= 0x1+                    p1 |= 0x80+                | otherwise -> p0 |= (0x3 `shiftL` (bit - 1))         p0 &= (complement $ mask)         when generateOdd (pEnd |= 0x1) @@ -71,52 +76,81 @@     (&=) :: Ptr Word8 -> Word8 -> IO ()     (&=) p w = peek p >>= \v -> poke p (v .&. w) -    bytes = (bits + 7) `div` 8;-    bit   = (bits - 1) `mod` 8;-    mask  = 0xff `shiftL` (bit + 1);+    bytes = (bits + 7) `div` 8+    bit = (bits - 1) `mod` 8+    mask = 0xff `shiftL` (bit + 1) +-- | Generate a number for a specific size of bits.+--+-- * @'generateParams' n Nothing False@ generates bytes and uses the suffix of @n@ bits+-- * @'generatePrefix' n@ generates bytes and uses the prefix of @n@ bits+generatePrefix :: MonadRandom m => Int -> m Integer+generatePrefix bits+    | bits <= 0 = return 0+    | otherwise = do+        let (count, offset) = (bits + 7) `divMod` 8+        bytes <- getRandomBytes count+        return $ os2ip (bytes :: ScrubbedBytes) `unsafeShiftR` (7 - offset)+ -- | Generate a positive integer x, s.t. 0 <= x < range-generateMax :: MonadRandom m-            => Integer  -- ^ range-            -> m Integer+generateMax+    :: MonadRandom m+    => Integer+    -- ^ range+    -> m Integer generateMax range-    | range <= 1      = return 0-    | range < 127     = generateSimple+    | range <= 1 = return 0+    | range < 127 = generateSimple     | canOverGenerate = loopGenerateOver tries-    | otherwise       = loopGenerate tries+    | otherwise = loopGenerate tries   where-        -- this "generator" is mostly for quickcheck benefits. it'll be biased if-        -- range is not a multiple of 2, but overall, no security should be-        -- assumed for a number between 0 and 127.-        generateSimple = flip mod range `fmap` generateParams bits Nothing False+    -- this "generator" is mostly for quickcheck benefits. it'll be biased if+    -- range is not a multiple of 2, but overall, no security should be+    -- assumed for a number between 0 and 127.+    generateSimple = flip mod range `fmap` generateParams bits Nothing False -        loopGenerate count-            | count == 0 = error $ "internal: generateMax(" ++ show range ++ " bits=" ++ show bits ++ ") (normal) doesn't seems to work properly"-            | otherwise  = do-                r <- generateParams bits Nothing False-                if isValid r then return r else loopGenerate (count-1)+    loopGenerate count+        | count == 0 =+            error $+                "internal: generateMax("+                    ++ show range+                    ++ " bits="+                    ++ show bits+                    ++ ") (normal) doesn't seems to work properly"+        | otherwise = do+            r <- generateParams bits Nothing False+            if isValid r then return r else loopGenerate (count - 1) -        loopGenerateOver count-            | count == 0 = error $ "internal: generateMax(" ++ show range ++ " bits=" ++ show bits ++ ") (over) doesn't seems to work properly"-            | otherwise  = do-                r <- generateParams (bits+1) Nothing False-                let r2 = r - range-                    r3 = r2 - range-                if isValid r-                    then return r-                    else if isValid r2+    loopGenerateOver count+        | count == 0 =+            error $+                "internal: generateMax("+                    ++ show range+                    ++ " bits="+                    ++ show bits+                    ++ ") (over) doesn't seems to work properly"+        | otherwise = do+            r <- generateParams (bits + 1) Nothing False+            let r2 = r - range+                r3 = r2 - range+            if isValid r+                then return r+                else+                    if isValid r2                         then return r2-                        else if isValid r3-                            then return r3-                            else loopGenerateOver (count-1)+                        else+                            if isValid r3+                                then return r3+                                else loopGenerateOver (count - 1) -        bits            = numBits range-        canOverGenerate = bits > 3 && not (range `testBit` (bits-2)) && not (range `testBit` (bits-3))+    bits = numBits range+    canOverGenerate =+        bits > 3 && not (range `testBit` (bits - 2)) && not (range `testBit` (bits - 3)) -        isValid n = n < range+    isValid n = n < range -        tries :: Int-        tries = 100+    tries :: Int+    tries = 100  -- | generate a number between the inclusive bound [low,high]. generateBetween :: MonadRandom m => Integer -> Integer -> m Integer
Crypto/Number/ModArithmetic.hs view
@@ -1,34 +1,60 @@ {-# LANGUAGE BangPatterns #-}+ -- | -- Module      : Crypto.Number.ModArithmetic -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good+--+-- Modular arithmetic on 'Integer'.+--+-- == What an 'Integer' shows+--+-- An 'Integer' is as long as its value needs, and every operation on one+-- costs what that length says.  A secret that happens to be short is+-- multiplied, reduced and compared in fewer words than a full-length one, and+-- the difference is there to be measured.  'expSafe' and 'inverseSafe' keep+-- the /value/ of an exponent or of a number being inverted out of the work+-- they do, and that is as far as an 'Integer' can be taken: hiding the length+-- as well means a fixed-width representation, which is what the curve modules+-- and 'expSafe' itself use underneath.+module Crypto.Number.ModArithmetic (+    -- * Exceptions+    CoprimesAssertionError (..),+    ModulusAssertionError (..), -module Crypto.Number.ModArithmetic-    (     -- * Exponentiation-      expSafe-    , expFast+    expSafe,+    expFast,+     -- * Inverse computing-    , inverse-    , inverseCoprimes-    , inverseFermat+    inverse,+    inverseSafe,+    inverseCoprimes,+    inverseFermat,+     -- * Squares-    , jacobi-    , squareRoot-    ) where+    jacobi,+    squareRoot,+) where -import Control.Exception (throw, Exception)+import qualified Control.Exception as E+import Crypto.Internal.Compat (unsafeDoIO) import Crypto.Number.Basic import Crypto.Number.Compat+import qualified Crypto.Number.Serialize.Internal as Internal+import Data.Memory.PtrMethods (memSet)+import Data.Word (Word32, Word8)+import Foreign.C.Types (CInt (..))+import Foreign.Marshal.Alloc (allocaBytes)+import Foreign.Ptr (Ptr, plusPtr)  -- | Raised when two numbers are supposed to be coprimes but are not. data CoprimesAssertionError = CoprimesAssertionError     deriving (Show) -instance Exception CoprimesAssertionError+instance E.Exception CoprimesAssertionError  -- | Compute the modular exponentiation of base^exponent using -- algorithms design to avoid side channels and timing measurement@@ -36,66 +62,155 @@ -- Modulo need to be odd otherwise the normal fast modular exponentiation -- is used. ----- When used with integer-simple, this function is not different--- from expFast, and thus provide the same unstudied and dubious--- timing and side channels claims.+-- With an odd modulo the work is done in C, four bits of exponent at a time:+-- four squarings and one multiplication by a small power of the base, taken+-- from a table of sixteen which is read by touching every entry and keeping+-- one of them with a mask.  So each group of four bits costs the same five+-- multiplications and the same sixteen reads whatever those bits are, and+-- nothing branches on the exponent or indexes memory with it. ----- Before GHC 8.4.2, powModSecInteger is missing from integer-gmp,--- so expSafe has the same security as expFast.-expSafe :: Integer -- ^ base-        -> Integer -- ^ exponent-        -> Integer -- ^ modulo-        -> Integer -- ^ result+-- What the exponent still shows is its length: it is rounded up to a whole+-- 64-bit word and every bit of that is walked over, so its value is hidden+-- but its size is not.  The @mpz_powm_sec@ of GMP, which GHC stopped+-- offering in integer-gmp 1.1 and which this replaces, hides exactly as much.+--+-- The base is taken to be public -- in this library it is a ciphertext, a+-- public value from a peer, or a generator -- and is reduced modulo the+-- modulus in the ordinary way first.+--+-- Hiding the exponent has a price: against the windowed exponentiation of+-- GMP, which is what this function used to end up calling, a 2048-bit+-- modulus costs somewhat over twice as much.+expSafe+    :: Integer+    -- ^ base+    -> Integer+    -- ^ exponent+    -> Integer+    -- ^ modulo+    -> Integer+    -- ^ result expSafe b e m-    | odd m     = gmpPowModSecInteger b e m `onGmpUnsupported`-                  (gmpPowModInteger b e m   `onGmpUnsupported`-                  exponentiation b e m)-    | otherwise = gmpPowModInteger b e m    `onGmpUnsupported`-                  exponentiation b e m+    | odd m && m > 1 && e >= 0 =+        gmpPowModSecInteger b e m `onGmpUnsupported` expSec (b `mod` m) e m+    -- a modulus of one, and a negative exponent asking for an inverse, are+    -- left to the path they have always taken+    | otherwise =+        gmpPowModInteger b e m+            `onGmpUnsupported` exponentiation b e m +-- | The windowed exponentiation itself, in C.  The base has to be reduced+-- already, the exponent to be zero or more, and the modulus odd and above+-- one.+expSec :: Integer -> Integer -> Integer -> Integer+expSec b e m = unsafeDoIO $+    allocaBytes (sum widths) $ \start -> case scanl plusPtr start widths of+        (out : base : expo : modu : _) -> do+            _ <- Internal.i2ospOf b base mLen+            _ <- Internal.i2ospOf e expo eLen+            _ <- Internal.i2ospOf m modu mLen+            r <-+                c_powm_sec+                    out+                    base+                    (fromIntegral mLen)+                    expo+                    (fromIntegral eLen)+                    modu+                    (fromIntegral mLen)+            -- the exponent is the caller's secret, and this is the last place it+            -- is written out in the clear+            memSet expo 0 eLen+            if r == 0+                then do+                    !v <- Internal.os2ip out mLen+                    return v+                else+                    return+                        ( gmpPowModInteger b e m+                            `onGmpUnsupported` exponentiation b e m+                        )+        _ -> return 0 -- there are four, but say so anyway+  where+    !mLen = numBytes m+    -- the answer, the base, the exponent and the modulus.  The room to take+    -- and where each one starts both come from here, so they cannot drift+    -- apart.+    widths = [mLen, mLen, eLen, mLen]+    -- whole words of exponent, so that the count of them says as little as+    -- what GMP's own secure exponentiation lets slip+    !eLen = 8 * ((numBytes e + 7) `div` 8)++foreign import ccall safe "crypton_powm_sec"+    c_powm_sec+        :: Ptr Word8+        -> Ptr Word8+        -> Word32+        -> Ptr Word8+        -> Word32+        -> Ptr Word8+        -> Word32+        -> IO CInt+ -- | Compute the modular exponentiation of base^exponent using -- the fastest algorithm without any consideration for -- hiding parameters. -- -- Use this function when all the parameters are public, -- otherwise 'expSafe' should be preferred.-expFast :: Integer -- ^ base-        -> Integer -- ^ exponent-        -> Integer -- ^ modulo-        -> Integer -- ^ result+expFast+    :: Integer+    -- ^ base+    -> Integer+    -- ^ exponent+    -> Integer+    -- ^ modulo+    -> Integer+    -- ^ result expFast b e m = gmpPowModInteger b e m `onGmpUnsupported` exponentiation b e m  -- | @exponentiation@ computes modular exponentiation as /b^e mod m/ -- using repetitive squaring.+--+-- The corner cases are held to what GMP answers, since that is what this+-- computes on every build that has it: a modulus of one is zero whatever+-- else is asked, and a negative exponent is a request for the inverse of+-- the base raised to its magnitude, which is zero when no inverse exists.+-- Read literally, the recursion below walked a negative exponent from -1 to+-- -2 and back for as long as the stack held. exponentiation :: Integer -> Integer -> Integer -> Integer exponentiation b e m-    | b == 1    = b-    | e == 0    = 1-    | e == 1    = b `mod` m-    | even e    = let p = exponentiation b (e `div` 2) m `mod` m-                   in (p^(2::Integer)) `mod` m-    | otherwise = (b * exponentiation b (e-1) m) `mod` m+    | m == 1 = 0+    | e < 0 =+        maybe 0 (\bInv -> exponentiation bInv (negate e) m) (inverse (b `mod` m) m)+    | b == 1 = 1+    | e == 0 = 1+    | e == 1 = b `mod` m+    | even e =+        let p = exponentiation b (e `div` 2) m `mod` m+         in (p ^ (2 :: Integer)) `mod` m+    | otherwise = (b * exponentiation b (e - 1) m) `mod` m  -- | @inverse@ computes the modular inverse as in /g^(-1) mod m/. inverse :: Integer -> Integer -> Maybe Integer inverse g m = gmpInverse g m `onGmpUnsupported` v   where     v-        | d > 1     = Nothing+        | d > 1 = Nothing         | otherwise = Just (x `mod` m)-    (x,_,d) = gcde g m+    (x, _, d) = gcde g m  -- | Compute the modular inverse of two coprime numbers. -- This is equivalent to inverse except that the result -- is known to exists. -- -- If the numbers are not defined as coprime, this function--- will raise a 'CoprimesAssertionError'.+-- will raise a t'CoprimesAssertionError'. inverseCoprimes :: Integer -> Integer -> Integer inverseCoprimes g m =     case inverse g m of-        Nothing -> throw CoprimesAssertionError-        Just i  -> i+        Nothing -> E.throw CoprimesAssertionError+        Just i -> i  -- | Computes the Jacobi symbol (a/n). -- 0 ≤ a < n; n ≥ 3 and odd.@@ -106,58 +221,116 @@ -- See algorithm 2.149 in "Handbook of Applied Cryptography" by Alfred J. Menezes et al. jacobi :: Integer -> Integer -> Maybe Integer jacobi a n-    | n < 3 || even n  = Nothing+    | n < 3 || even n = Nothing     | a == 0 || a == 1 = Just a-    | n <= a           = jacobi (a `mod` n) n-    | a < 0            =-      let b = if n `mod` 4 == 1 then 1 else -1-       in fmap (*b) (jacobi (-a) n)-    | otherwise        =-      let (e, a1) = asPowerOf2AndOdd a-          nMod8   = n `mod` 8-          nMod4   = n `mod` 4-          a1Mod4  = a1 `mod` 4-          s'      = if even e || nMod8 == 1 || nMod8 == 7 then 1 else -1-          s       = if nMod4 == 3 && a1Mod4 == 3 then -s' else s'-          n1      = n `mod` a1-       in if a1 == 1 then Just s-          else fmap (*s) (jacobi n1 a1)+    | n <= a = jacobi (a `mod` n) n+    | a < 0 =+        let b = if n `mod` 4 == 1 then 1 else -1+         in fmap (* b) (jacobi (-a) n)+    | otherwise =+        let (e, a1) = asPowerOf2AndOdd a+            nMod8 = n `mod` 8+            nMod4 = n `mod` 4+            a1Mod4 = a1 `mod` 4+            s' = if even e || nMod8 == 1 || nMod8 == 7 then 1 else -1+            s = if nMod4 == 3 && a1Mod4 == 3 then -s' else s'+            n1 = n `mod` a1+         in if a1 == 1+                then Just s+                else fmap (* s) (jacobi n1 a1)  -- | Modular inverse using Fermat's little theorem.  This works only when -- the modulus is prime but avoids side channels like in 'expSafe'. inverseFermat :: Integer -> Integer -> Integer inverseFermat g p = expSafe g (p - 2) p +-- | @inverseSafe@ computes the modular inverse without letting the number+-- being inverted steer how long the work takes, which is what 'inverse' does:+-- the extended Euclidean algorithm takes a number of steps that follows the+-- bits it is given, and a nonce inverted that way has been taken apart before+-- by watching the steps go by.+--+-- The answer comes from a fixed number of division steps where the assembly+-- for them is built, and from 'inverseFermat' where it is not.  Either way it+-- is checked here by multiplying out: neither one says when the number has no+-- inverse -- the first returns something that is not one and the second+-- returns something that is not one either -- so the check is what makes this+-- agree with 'inverse' on every input, and 'inverse' is asked when it fails.+-- That fallback is reached only by parameters that are already broken.+--+-- The division steps cost about a twentieth of the exponentiation: on an+-- Apple M4, inverting modulo the P-256 group order is 0.80 microseconds+-- against 6.02, and modulo the P-521 one 2.05 against 63.2.+inverseSafe :: Integer -> Integer -> Maybe Integer+inverseSafe g m+    | m > 1 && (g * r) `mod` m == 1 = Just r+    | otherwise = inverse g m+  where+    r = case inverseSec g m of+        Just v -> v+        Nothing -> inverseFermat g m++-- | The inverse in a fixed number of division steps, from the vendored+-- assembly.  'Nothing' when that is not built, when the modulus is even --+-- where the routine answers without saying it cannot -- or when the numbers+-- are larger than it keeps room for.  The answer is not checked here; the+-- caller does that.+inverseSec :: Integer -> Integer -> Maybe Integer+inverseSec g m+    | m <= 1 || even m || g < 0 = Nothing+    | otherwise = unsafeDoIO $+        allocaBytes (3 * mLen) $ \out -> do+            let gp = out `plusPtr` mLen+                mp = gp `plusPtr` mLen+            _ <- Internal.i2ospOf (g `mod` m) gp mLen+            _ <- Internal.i2ospOf m mp mLen+            r <- c_modinv_sec out gp mp (fromIntegral mLen)+            if r == 0+                then do+                    !v <- Internal.os2ip out mLen+                    return (Just v)+                else return Nothing+  where+    !mLen = numBytes m++foreign import ccall unsafe "crypton_modinv_sec"+    c_modinv_sec+        :: Ptr Word8+        -> Ptr Word8+        -> Ptr Word8+        -> Word32+        -> IO CInt+ -- | Raised when the assumption about the modulus is invalid. data ModulusAssertionError = ModulusAssertionError     deriving (Show) -instance Exception ModulusAssertionError+instance E.Exception ModulusAssertionError  -- | Modular square root of @g@ modulo a prime @p@. -- -- If the modulus is found not to be prime, the function will raise a--- 'ModulusAssertionError'.+-- t'ModulusAssertionError'. -- -- This implementation is variable time and should be used with public -- parameters only. squareRoot :: Integer -> Integer -> Maybe Integer squareRoot p-    | p < 2     = throw ModulusAssertionError+    | p < 2 = E.throw ModulusAssertionError     | otherwise =         case p `divMod` 8 of-           (v, 3) -> method1 (2 * v + 1)-           (v, 7) -> method1 (2 * v + 2)-           (u, 5) -> method2 u-           (_, 1) -> tonelliShanks p-           (0, 2) -> \a -> Just (if even a then 0 else 1)-           _      -> throw ModulusAssertionError-+            (v, 3) -> method1 (2 * v + 1)+            (v, 7) -> method1 (2 * v + 2)+            (u, 5) -> method2 u+            (_, 1) -> tonelliShanks p+            (0, 2) -> \a -> Just (if even a then 0 else 1)+            _ -> E.throw ModulusAssertionError   where     x `eqMod` y = (x - y) `mod` p == 0 -    validate g y | (y * y) `eqMod` g = Just y-                 | otherwise         = Nothing+    validate g y+        | (y * y) `eqMod` g = Just y+        | otherwise = Nothing      -- p == 4u + 3 and u' == u + 1     method1 u' g =@@ -174,20 +347,24 @@  tonelliShanks :: Integer -> Integer -> Maybe Integer tonelliShanks p a-    | aa == 0   = Just 0+    | aa == 0 = Just 0     | otherwise =         case expFast aa p2 p of-            b | b == p1   -> Nothing-              | b == 1    -> Just $ go (expFast aa ((s + 1) `div` 2) p)-                                       (expFast aa s p)-                                       (expFast n  s p)-                                       e-              | otherwise -> throw ModulusAssertionError+            b+                | b == p1 -> Nothing+                | b == 1 ->+                    Just $+                        go+                            (expFast aa ((s + 1) `div` 2) p)+                            (expFast aa s p)+                            (expFast n s p)+                            e+                | otherwise -> E.throw ModulusAssertionError   where     aa = a `mod` p     p1 = p - 1     p2 = p1 `div` 2-    n  = findN 2+    n = findN 2      x `mul` y = (x * y) `mod` p @@ -199,15 +376,15 @@     -- find a quadratic non-residue     findN i         | expFast i p2 p == p1 = i-        | otherwise            = findN (i + 1)+        | otherwise = findN (i + 1)      -- find m such that b^(2^m) == 1 (mod p)     findM b i-        | b == 1    = i+        | b == 1 = i         | otherwise = findM (b `mul` b) (i + 1)      go !x b g !r-        | b == 1    = x+        | b == 1 = x         | otherwise =             let r' = findM b 0                 z = pow2m (r - r' - 1) g
Crypto/Number/Nat.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE TypeOperators #-}+ -- | -- Module      : Crypto.Number.Nat -- License     : BSD-style@@ -26,38 +29,39 @@ -- -- Function @withDivisibleBy8@ above returns 'Nothing' when the argument @len@ -- is negative or not divisible by 8.-{-# LANGUAGE DataKinds #-}-{-# LANGUAGE TypeOperators #-}-module Crypto.Number.Nat-    ( type IsDivisibleBy8-    , type IsAtMost, type IsAtLeast-    , isDivisibleBy8-    , isAtMost-    , isAtLeast-    ) where+module Crypto.Number.Nat (+    type IsDivisibleBy8,+    type IsAtMost,+    type IsAtLeast,+    isDivisibleBy8,+    isAtMost,+    isAtLeast,+) where -import           Data.Type.Equality-import           GHC.TypeLits-import           Unsafe.Coerce (unsafeCoerce)+import Data.Type.Equality+import GHC.TypeLits+import Unsafe.Coerce (unsafeCoerce) -import           Crypto.Internal.Nat+import Crypto.Internal.Nat  -- | get a runtime proof that the constraint @'IsDivisibleBy8' n@ is satified isDivisibleBy8 :: KnownNat n => proxy n -> Maybe (IsDiv8 n n :~: 'True) isDivisibleBy8 n     | mod (natVal n) 8 == 0 = Just (unsafeCoerce Refl)-    | otherwise             = Nothing+    | otherwise = Nothing  -- | get a runtime proof that the constraint @'IsAtMost' value bound@ is -- satified-isAtMost :: (KnownNat value, KnownNat bound)-         => proxy value -> proxy' bound -> Maybe ((value <=? bound) :~: 'True)+isAtMost+    :: (KnownNat value, KnownNat bound)+    => proxy value -> proxy' bound -> Maybe ((value <=? bound) :~: 'True) isAtMost x y-    | natVal x <= natVal y  = Just (unsafeCoerce Refl)-    | otherwise             = Nothing+    | natVal x <= natVal y = Just (unsafeCoerce Refl)+    | otherwise = Nothing  -- | get a runtime proof that the constraint @'IsAtLeast' value bound@ is -- satified-isAtLeast :: (KnownNat value, KnownNat bound)-          => proxy value -> proxy' bound -> Maybe ((bound <=? value) :~: 'True)+isAtLeast+    :: (KnownNat value, KnownNat bound)+    => proxy value -> proxy' bound -> Maybe ((bound <=? value) :~: 'True) isAtLeast = flip isAtMost
Crypto/Number/Prime.hs view
@@ -1,45 +1,91 @@+{-# LANGUAGE BangPatterns #-}+ -- | -- Module      : Crypto.Number.Prime -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good--{-# LANGUAGE BangPatterns #-}-module Crypto.Number.Prime-    (-      generatePrime-    , generateSafePrime-    , isProbablyPrime-    , findPrimeFrom-    , findPrimeFromWith-    , primalityTestMillerRabin-    , primalityTestNaive-    , primalityTestFermat-    , isCoprime-    ) where+module Crypto.Number.Prime (+    generatePrime,+    generateSafePrime,+    isProbablyPrime,+    findPrimeFrom,+    findPrimeFromWith,+    primalityTestMillerRabin,+    primalityTestNaive,+    primalityTestFermat,+    isCoprime,+) where +import Crypto.Error+import Crypto.Number.Basic (gcde, sqrti) import Crypto.Number.Compat import Crypto.Number.Generate-import Crypto.Number.Basic (sqrti, gcde) import Crypto.Number.ModArithmetic (expSafe)-import Crypto.Random.Types+import Crypto.Number.Serialize (i2osp) import Crypto.Random.Probabilistic-import Crypto.Error+import Crypto.Random.Types +import Crypto.Internal.ByteArray (Bytes)+ import Data.Bits  -- | Returns if the number is probably prime.--- First a list of small primes are implicitely tested for divisibility,--- then a fermat primality test is used with arbitrary numbers and--- then the Miller Rabin algorithm is used with an accuracy of 30 recursions.+--+-- The small primes are tested for divisibility first, and then the+-- Miller-Rabin algorithm with an accuracy of 30 rounds.+--+-- A Fermat test of fifty consecutive bases used to run between the two.  It+-- ruled out nothing Miller-Rabin does not: a strong probable prime to a base+-- is a Fermat probable prime to that base, and the converse is what Carmichael+-- numbers are.  What it cost was fifty modular exponentiations on every number+-- that turned out to be prime -- 2167 of the 3480 microseconds spent on a+-- 512-bit prime, and about two thirds of the time to generate one. isProbablyPrime :: Integer -> Bool-isProbablyPrime !n+isProbablyPrime = probablyPrime 30++-- | The same, with the number of rounds said outright.+--+-- Thirty rounds is what a number from anywhere gets: whoever handed it over+-- may have built it to pass, and against that the only thing to go on is that+-- each round with a base drawn at random catches three quarters of the+-- composites there are, whatever the number is.  Thirty of them leave one+-- chance in 2^60.+probablyPrime :: Int -> Integer -> Bool+probablyPrime rounds !n+    | n < 2 = False     | any (\p -> p `divides` n) (filter (< n) firstPrimes) = False-    | n >= 2 && n <= 2903                                  = True-    | primalityTestFermat 50 (n `div` 2) n                 = primalityTestMillerRabin 30 n-    | otherwise                                            = False+    | n <= 2903 = True+    | otherwise = primalityTestMillerRabin rounds n +-- | How many rounds a candidate drawn here needs.+--+-- A number nobody chose is a different matter from one somebody did.  The+-- composites that survive a round are rare, and the ones that survive several+-- are rarer than the bound above says: Damgard, Landrock and Pomerance+-- worked out how much rarer for a candidate drawn at random, and Table 4.4 of+-- the Handbook of Applied Cryptography puts their numbers in a table -- two+-- rounds at 1300 bits, three at 850, five at 550, and so on, for one chance+-- in 2^80.+--+-- This is twice that, and never more than the thirty a number from anywhere+-- gets, which leaves the chance far under one in 2^100 at every size.  It is+-- what makes generating a prime worth doing: the thirty rounds were half the+-- time it took.+roundsForDrawn :: Int -> Int+roundsForDrawn bits+    | bits >= 1300 = 6+    | bits >= 850 = 8+    | bits >= 650 = 10+    | bits >= 550 = 12+    | bits >= 450 = 14+    | bits >= 400 = 16+    | bits >= 350 = 18+    | bits >= 300 = 20+    | bits >= 250 = 24+    | otherwise = 30+ -- | Generate a prime number of the required bitsize (i.e. in the range -- [2^(b-1)+2^(b-2), 2^b)). --@@ -50,14 +96,16 @@ -- the proper size. generatePrime :: MonadRandom m => Int -> m Integer generatePrime bits = do-    if bits < 5 then-        throwCryptoError $ CryptoFailed $ CryptoError_PrimeSizeInvalid-    else do-        sp <- generateParams bits (Just SetTwoHighest) True-        let prime = findPrimeFrom sp-        if prime < 1 `shiftL` bits then-            return $ prime-        else generatePrime bits+    if bits < 5+        then+            throwCryptoError $ CryptoFailed $ CryptoError_PrimeSizeInvalid+        else do+            sp <- generateParams bits (Just SetTwoHighest) True+            let prime = findPrimeFromDrawn (roundsForDrawn bits) sp+            if prime < 1 `shiftL` bits+                then+                    return $ prime+                else generatePrime bits  -- | Generate a prime number of the form 2p+1 where p is also prime. -- it is also knowed as a Sophie Germaine prime or safe prime.@@ -69,28 +117,45 @@ -- 6 bits, as the smallest safe prime with the two highest bits set is 59. generateSafePrime :: MonadRandom m => Int -> m Integer generateSafePrime bits = do-    if bits < 6 then-        throwCryptoError $ CryptoFailed $ CryptoError_PrimeSizeInvalid-    else do-        sp <- generateParams bits (Just SetTwoHighest) True-        let p = findPrimeFromWith (\i -> isProbablyPrime (2*i+1)) (sp `div` 2)-        let val = 2 * p + 1-        if val < 1 `shiftL` bits then-            return $ val-        else generateSafePrime bits+    if bits < 6+        then+            throwCryptoError $ CryptoFailed $ CryptoError_PrimeSizeInvalid+        else do+            sp <- generateParams bits (Just SetTwoHighest) True+            let rounds = roundsForDrawn bits+                p =+                    findPrimeFromWithRounds+                        rounds+                        (\i -> probablyPrime rounds (2 * i + 1))+                        (sp `div` 2)+            let val = 2 * p + 1+            if val < 1 `shiftL` bits+                then+                    return $ val+                else generateSafePrime bits  -- | Find a prime from a starting point where the property hold. findPrimeFromWith :: (Integer -> Bool) -> Integer -> Integer-findPrimeFromWith prop !n-    | even n        = findPrimeFromWith prop (n+1)-    | otherwise     =-        if not (isProbablyPrime n)-            then findPrimeFromWith prop (n+2)+findPrimeFromWith = findPrimeFromWithRounds 30++-- | The same, with the number of rounds said outright: the walk starts where+-- the caller says, and only a caller that drew that starting point itself is+-- entitled to the smaller number.+findPrimeFromWithRounds :: Int -> (Integer -> Bool) -> Integer -> Integer+findPrimeFromWithRounds rounds prop !n+    | even n = findPrimeFromWithRounds rounds prop (n + 1)+    | otherwise =+        if not (probablyPrime rounds n)+            then findPrimeFromWithRounds rounds prop (n + 2)             else                 if prop n                     then n-                    else findPrimeFromWith prop (n+2)+                    else findPrimeFromWithRounds rounds prop (n + 2) +-- | Find a prime from a starting point that the caller drew itself.+findPrimeFromDrawn :: Int -> Integer -> Integer+findPrimeFromDrawn rounds = findPrimeFromWithRounds rounds (\_ -> True)+ -- | Find a prime from a starting point with no specific property. findPrimeFrom :: Integer -> Integer findPrimeFrom n =@@ -100,48 +165,56 @@  -- | Miller Rabin algorithm return if the number is probably prime or composite. -- the tries parameter is the number of recursion, that determines the accuracy of the test.+--+-- The witnesses are drawn from a generator derived from @n@ itself and from a+-- secret drawn once per process: testing the same number twice gives the same+-- answer, testing two numbers draws independent witnesses for each, and an+-- attacker choosing the number cannot tell which witnesses it will face. primalityTestMillerRabin :: Int -> Integer -> Bool primalityTestMillerRabin tries !n =     case gmpTestPrimeMillerRabin tries n of         GmpSupported b -> b-        GmpUnsupported -> probabilistic run+        -- the material is forced only once a witness is drawn, which the+        -- guards in run reach only for an odd n above 3+        GmpUnsupported -> probabilisticFrom (i2osp n :: Bytes) run   where     run-        | n <= 3     = error "Miller-Rabin requires tested value to be > 3"-        | even n     = return False+        | n <= 3 = error "Miller-Rabin requires tested value to be > 3"+        | even n = return False         | tries <= 0 = error "Miller-Rabin tries need to be > 0"-        | otherwise  = loop <$> generateTries tries+        | otherwise = loop <$> generateTries tries -    !nm1 = n-1-    !nm2 = n-2+    !nm1 = n - 1+    !nm2 = n - 2 -    (!s,!d) = (factorise 0 nm1)+    (!s, !d) = (factorise 0 nm1)      generateTries 0 = return []     generateTries t = do-        v  <- generateBetween 2 nm2-        vs <- generateTries (t-1)-        return (v:vs)+        v <- generateBetween 2 nm2+        vs <- generateTries (t - 1)+        return (v : vs)      -- factorise n-1 into the form 2^s*d     factorise :: Integer -> Integer -> (Integer, Integer)     factorise !si !vi         | vi `testBit` 0 = (si, vi)-        | otherwise     = factorise (si+1) (vi `shiftR` 1) -- probably faster to not shift v continuously, but just once.+        | otherwise = factorise (si + 1) (vi `shiftR` 1) -- probably faster to not shift v continuously, but just once.     expmod = expSafe      -- when iteration reach zero, we have a probable prime-    loop []     = True-    loop (w:ws) = let x = expmod w d n-                   in if x == (1 :: Integer) || x == nm1-                          then loop ws-                          else loop' ws ((x*x) `mod` n) 1+    loop [] = True+    loop (w : ws) =+        let x = expmod w d n+         in if x == (1 :: Integer) || x == nm1+                then loop ws+                else loop' ws ((x * x) `mod` n) 1      -- loop from 1 to s-1. if we reach the end then it's composite     loop' ws !x2 !r-        | r == s    = False-        | x2 == 1   = False-        | x2 /= nm1 = loop' ws ((x2*x2) `mod` n) (r+1)+        | r == s = False+        | x2 == 1 = False+        | x2 /= nm1 = loop' ws ((x2 * x2) `mod` n) (r + 1)         | otherwise = loop ws  {-@@ -157,77 +230,461 @@ -- | Probabilitic Test using Fermat primility test. -- Beware of Carmichael numbers that are Fermat liars, i.e. this test -- is useless for them. always combines with some other test.-primalityTestFermat :: Int -- ^ number of iterations of the algorithm-                    -> Integer -- ^ starting a-                    -> Integer -- ^ number to test for primality-                    -> Bool-primalityTestFermat n a p = and $ map expTest [a..(a+fromIntegral n)]-    where !pm1 = p-1-          expTest i = expSafe i pm1 p == 1+primalityTestFermat+    :: Int+    -- ^ number of iterations of the algorithm+    -> Integer+    -- ^ starting a+    -> Integer+    -- ^ number to test for primality+    -> Bool+primalityTestFermat n a p = and $ map expTest [a .. (a + fromIntegral n)]+  where+    !pm1 = p - 1+    expTest i = expSafe i pm1 p == 1  -- | Test naively is integer is prime. -- while naive, we skip even number and stop iteration at i > sqrt(n) primalityTestNaive :: Integer -> Bool primalityTestNaive n-    | n <= 1    = False-    | n == 2    = True-    | even n    = False+    | n <= 1 = False+    | n == 2 = True+    | even n = False     | otherwise = search 3-        where !ubound = snd $ sqrti n-              search !i-                  | i > ubound    = True-                  | i `divides` n = False-                  | otherwise     = search (i+2)+  where+    !ubound = snd $ sqrti n+    search !i+        | i > ubound = True+        | i `divides` n = False+        | otherwise = search (i + 2)  -- | Test is two integer are coprime to each other isCoprime :: Integer -> Integer -> Bool-isCoprime m n = case gcde m n of (_,_,d) -> d == 1+isCoprime m n = case gcde m n of (_, _, d) -> d == 1  -- | List of the first primes till 2903. firstPrimes :: [Integer] firstPrimes =-    [ 2    , 3    , 5    , 7    , 11   , 13   , 17   , 19   , 23   , 29-    , 31   , 37   , 41   , 43   , 47   , 53   , 59   , 61   , 67   , 71-    , 73   , 79   , 83   , 89   , 97   , 101  , 103  , 107  , 109  , 113-    , 127  , 131  , 137  , 139  , 149  , 151  , 157  , 163  , 167  , 173-    , 179  , 181  , 191  , 193  , 197  , 199  , 211  , 223  , 227  , 229-    , 233  , 239  , 241  , 251  , 257  , 263  , 269  , 271  , 277  , 281-    , 283  , 293  , 307  , 311  , 313  , 317  , 331  , 337  , 347  , 349-    , 353  , 359  , 367  , 373  , 379  , 383  , 389  , 397  , 401  , 409-    , 419  , 421  , 431  , 433  , 439  , 443  , 449  , 457  , 461  , 463-    , 467  , 479  , 487  , 491  , 499  , 503  , 509  , 521  , 523  , 541-    , 547  , 557  , 563  , 569  , 571  , 577  , 587  , 593  , 599  , 601-    , 607  , 613  , 617  , 619  , 631  , 641  , 643  , 647  , 653  , 659-    , 661  , 673  , 677  , 683  , 691  , 701  , 709  , 719  , 727  , 733-    , 739  , 743  , 751  , 757  , 761  , 769  , 773  , 787  , 797  , 809-    , 811  , 821  , 823  , 827  , 829  , 839  , 853  , 857  , 859  , 863-    , 877  , 881  , 883  , 887  , 907  , 911  , 919  , 929  , 937  , 941-    , 947  , 953  , 967  , 971  , 977  , 983  , 991  , 997  , 1009 , 1013-    , 1019 , 1021 , 1031 , 1033 , 1039 , 1049 , 1051 , 1061 , 1063 , 1069-    , 1087 , 1091 , 1093 , 1097 , 1103 , 1109 , 1117 , 1123 , 1129 , 1151-    , 1153 , 1163 , 1171 , 1181 , 1187 , 1193 , 1201 , 1213 , 1217 , 1223-    , 1229 , 1231 , 1237 , 1249 , 1259 , 1277 , 1279 , 1283 , 1289 , 1291-    , 1297 , 1301 , 1303 , 1307 , 1319 , 1321 , 1327 , 1361 , 1367 , 1373-    , 1381 , 1399 , 1409 , 1423 , 1427 , 1429 , 1433 , 1439 , 1447 , 1451-    , 1453 , 1459 , 1471 , 1481 , 1483 , 1487 , 1489 , 1493 , 1499 , 1511-    , 1523 , 1531 , 1543 , 1549 , 1553 , 1559 , 1567 , 1571 , 1579 , 1583-    , 1597 , 1601 , 1607 , 1609 , 1613 , 1619 , 1621 , 1627 , 1637 , 1657-    , 1663 , 1667 , 1669 , 1693 , 1697 , 1699 , 1709 , 1721 , 1723 , 1733-    , 1741 , 1747 , 1753 , 1759 , 1777 , 1783 , 1787 , 1789 , 1801 , 1811-    , 1823 , 1831 , 1847 , 1861 , 1867 , 1871 , 1873 , 1877 , 1879 , 1889-    , 1901 , 1907 , 1913 , 1931 , 1933 , 1949 , 1951 , 1973 , 1979 , 1987-    , 1993 , 1997 , 1999 , 2003 , 2011 , 2017 , 2027 , 2029 , 2039 , 2053-    , 2063 , 2069 , 2081 , 2083 , 2087 , 2089 , 2099 , 2111 , 2113 , 2129-    , 2131 , 2137 , 2141 , 2143 , 2153 , 2161 , 2179 , 2203 , 2207 , 2213-    , 2221 , 2237 , 2239 , 2243 , 2251 , 2267 , 2269 , 2273 , 2281 , 2287-    , 2293 , 2297 , 2309 , 2311 , 2333 , 2339 , 2341 , 2347 , 2351 , 2357-    , 2371 , 2377 , 2381 , 2383 , 2389 , 2393 , 2399 , 2411 , 2417 , 2423-    , 2437 , 2441 , 2447 , 2459 , 2467 , 2473 , 2477 , 2503 , 2521 , 2531-    , 2539 , 2543 , 2549 , 2551 , 2557 , 2579 , 2591 , 2593 , 2609 , 2617-    , 2621 , 2633 , 2647 , 2657 , 2659 , 2663 , 2671 , 2677 , 2683 , 2687-    , 2689 , 2693 , 2699 , 2707 , 2711 , 2713 , 2719 , 2729 , 2731 , 2741-    , 2749 , 2753 , 2767 , 2777 , 2789 , 2791 , 2797 , 2801 , 2803 , 2819-    , 2833 , 2837 , 2843 , 2851 , 2857 , 2861 , 2879 , 2887 , 2897 , 2903+    [ 2+    , 3+    , 5+    , 7+    , 11+    , 13+    , 17+    , 19+    , 23+    , 29+    , 31+    , 37+    , 41+    , 43+    , 47+    , 53+    , 59+    , 61+    , 67+    , 71+    , 73+    , 79+    , 83+    , 89+    , 97+    , 101+    , 103+    , 107+    , 109+    , 113+    , 127+    , 131+    , 137+    , 139+    , 149+    , 151+    , 157+    , 163+    , 167+    , 173+    , 179+    , 181+    , 191+    , 193+    , 197+    , 199+    , 211+    , 223+    , 227+    , 229+    , 233+    , 239+    , 241+    , 251+    , 257+    , 263+    , 269+    , 271+    , 277+    , 281+    , 283+    , 293+    , 307+    , 311+    , 313+    , 317+    , 331+    , 337+    , 347+    , 349+    , 353+    , 359+    , 367+    , 373+    , 379+    , 383+    , 389+    , 397+    , 401+    , 409+    , 419+    , 421+    , 431+    , 433+    , 439+    , 443+    , 449+    , 457+    , 461+    , 463+    , 467+    , 479+    , 487+    , 491+    , 499+    , 503+    , 509+    , 521+    , 523+    , 541+    , 547+    , 557+    , 563+    , 569+    , 571+    , 577+    , 587+    , 593+    , 599+    , 601+    , 607+    , 613+    , 617+    , 619+    , 631+    , 641+    , 643+    , 647+    , 653+    , 659+    , 661+    , 673+    , 677+    , 683+    , 691+    , 701+    , 709+    , 719+    , 727+    , 733+    , 739+    , 743+    , 751+    , 757+    , 761+    , 769+    , 773+    , 787+    , 797+    , 809+    , 811+    , 821+    , 823+    , 827+    , 829+    , 839+    , 853+    , 857+    , 859+    , 863+    , 877+    , 881+    , 883+    , 887+    , 907+    , 911+    , 919+    , 929+    , 937+    , 941+    , 947+    , 953+    , 967+    , 971+    , 977+    , 983+    , 991+    , 997+    , 1009+    , 1013+    , 1019+    , 1021+    , 1031+    , 1033+    , 1039+    , 1049+    , 1051+    , 1061+    , 1063+    , 1069+    , 1087+    , 1091+    , 1093+    , 1097+    , 1103+    , 1109+    , 1117+    , 1123+    , 1129+    , 1151+    , 1153+    , 1163+    , 1171+    , 1181+    , 1187+    , 1193+    , 1201+    , 1213+    , 1217+    , 1223+    , 1229+    , 1231+    , 1237+    , 1249+    , 1259+    , 1277+    , 1279+    , 1283+    , 1289+    , 1291+    , 1297+    , 1301+    , 1303+    , 1307+    , 1319+    , 1321+    , 1327+    , 1361+    , 1367+    , 1373+    , 1381+    , 1399+    , 1409+    , 1423+    , 1427+    , 1429+    , 1433+    , 1439+    , 1447+    , 1451+    , 1453+    , 1459+    , 1471+    , 1481+    , 1483+    , 1487+    , 1489+    , 1493+    , 1499+    , 1511+    , 1523+    , 1531+    , 1543+    , 1549+    , 1553+    , 1559+    , 1567+    , 1571+    , 1579+    , 1583+    , 1597+    , 1601+    , 1607+    , 1609+    , 1613+    , 1619+    , 1621+    , 1627+    , 1637+    , 1657+    , 1663+    , 1667+    , 1669+    , 1693+    , 1697+    , 1699+    , 1709+    , 1721+    , 1723+    , 1733+    , 1741+    , 1747+    , 1753+    , 1759+    , 1777+    , 1783+    , 1787+    , 1789+    , 1801+    , 1811+    , 1823+    , 1831+    , 1847+    , 1861+    , 1867+    , 1871+    , 1873+    , 1877+    , 1879+    , 1889+    , 1901+    , 1907+    , 1913+    , 1931+    , 1933+    , 1949+    , 1951+    , 1973+    , 1979+    , 1987+    , 1993+    , 1997+    , 1999+    , 2003+    , 2011+    , 2017+    , 2027+    , 2029+    , 2039+    , 2053+    , 2063+    , 2069+    , 2081+    , 2083+    , 2087+    , 2089+    , 2099+    , 2111+    , 2113+    , 2129+    , 2131+    , 2137+    , 2141+    , 2143+    , 2153+    , 2161+    , 2179+    , 2203+    , 2207+    , 2213+    , 2221+    , 2237+    , 2239+    , 2243+    , 2251+    , 2267+    , 2269+    , 2273+    , 2281+    , 2287+    , 2293+    , 2297+    , 2309+    , 2311+    , 2333+    , 2339+    , 2341+    , 2347+    , 2351+    , 2357+    , 2371+    , 2377+    , 2381+    , 2383+    , 2389+    , 2393+    , 2399+    , 2411+    , 2417+    , 2423+    , 2437+    , 2441+    , 2447+    , 2459+    , 2467+    , 2473+    , 2477+    , 2503+    , 2521+    , 2531+    , 2539+    , 2543+    , 2549+    , 2551+    , 2557+    , 2579+    , 2591+    , 2593+    , 2609+    , 2617+    , 2621+    , 2633+    , 2647+    , 2657+    , 2659+    , 2663+    , 2671+    , 2677+    , 2683+    , 2687+    , 2689+    , 2693+    , 2699+    , 2707+    , 2711+    , 2713+    , 2719+    , 2729+    , 2731+    , 2741+    , 2749+    , 2753+    , 2767+    , 2777+    , 2789+    , 2791+    , 2797+    , 2801+    , 2803+    , 2819+    , 2833+    , 2837+    , 2843+    , 2851+    , 2857+    , 2861+    , 2879+    , 2887+    , 2897+    , 2903     ]  {-# INLINE divides #-}
Crypto/Number/Serialize.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE BangPatterns #-}+ -- | -- Module      : Crypto.Number.Serialize -- License     : BSD-style@@ -6,17 +8,16 @@ -- Portability : Good -- -- Fast serialization primitives for integer-{-# LANGUAGE BangPatterns #-}-module Crypto.Number.Serialize-    ( i2osp-    , os2ip-    , i2ospOf-    , i2ospOf_-    ) where+module Crypto.Number.Serialize (+    i2osp,+    os2ip,+    i2ospOf,+    i2ospOf_,+) where -import           Crypto.Number.Basic-import           Crypto.Internal.Compat (unsafeDoIO) import qualified Crypto.Internal.ByteArray as B+import Crypto.Internal.Compat (unsafeDoIO)+import Crypto.Number.Basic import qualified Crypto.Number.Serialize.Internal as Internal  -- | @os2ip@ converts a byte string into a positive integer.@@ -27,23 +28,24 @@ -- -- The first byte is MSB (most significant byte); the last byte is the LSB (least significant byte) i2osp :: B.ByteArray ba => Integer -> ba-i2osp 0 = B.allocAndFreeze 1  (\p -> Internal.i2osp 0 p 1 >> return ())+i2osp 0 = B.allocAndFreeze 1 (\p -> Internal.i2osp 0 p 1 >> return ()) i2osp m = B.allocAndFreeze sz (\p -> Internal.i2osp m p sz >> return ())   where-        !sz = numBytes m+    !sz = numBytes m  -- | Just like 'i2osp', but takes an extra parameter for size. -- If the number is too big to fit in @len@ bytes, 'Nothing' is returned -- otherwise the number is padded with 0 to fit the @len@ required.-{-# INLINABLE i2ospOf #-}+{-# INLINEABLE i2ospOf #-} i2ospOf :: B.ByteArray ba => Int -> Integer -> Maybe ba i2ospOf len m-    | len <= 0  = Nothing-    | m < 0     = Nothing-    | sz > len  = Nothing-    | otherwise = Just $ B.unsafeCreate len (\p -> Internal.i2ospOf m p len >> return ())+    | len <= 0 = Nothing+    | m < 0 = Nothing+    | sz > len = Nothing+    | otherwise =+        Just $ B.unsafeCreate len (\p -> Internal.i2ospOf m p len >> return ())   where-        !sz = numBytes m+    !sz = numBytes m  -- | Just like 'i2ospOf' except that it doesn't expect a failure: i.e. -- an integer larger than the number of output bytes requested.
Crypto/Number/Serialize/Internal.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE BangPatterns #-}+ -- | -- Module      : Crypto.Number.Serialize.Internal -- License     : BSD-style@@ -6,20 +8,19 @@ -- Portability : Good -- -- Fast serialization primitives for integer using raw pointers-{-# LANGUAGE BangPatterns #-}-module Crypto.Number.Serialize.Internal-    ( i2osp-    , i2ospOf-    , os2ip-    ) where+module Crypto.Number.Serialize.Internal (+    i2osp,+    i2ospOf,+    os2ip,+) where -import           Crypto.Number.Compat-import           Crypto.Number.Basic-import           Data.Bits-import           Data.Memory.PtrMethods-import           Data.Word (Word8)-import           Foreign.Ptr-import           Foreign.Storable+import Crypto.Number.Basic+import Crypto.Number.Compat+import Data.Bits+import Data.Memory.PtrMethods+import Data.Word (Word8)+import Foreign.Ptr+import Foreign.Storable  -- | Fill a pointer with the big endian binary representation of an integer --@@ -30,47 +31,52 @@ i2osp :: Integer -> Ptr Word8 -> Int -> IO Int i2osp m ptr ptrSz     | ptrSz <= 0 = return 0-    | m < 0      = return 0-    | m == 0     = pokeByteOff ptr 0 (0 :: Word8) >> return 1+    | m < 0 = return 0+    | m == 0 = pokeByteOff ptr 0 (0 :: Word8) >> return 1     | ptrSz < sz = return 0-    | otherwise  = fillPtr ptr sz m >> return sz+    | otherwise = fillPtr ptr sz m >> return sz   where-    !sz    = numBytes m+    !sz = numBytes m  -- | Similar to 'i2osp', except it will pad any remaining space with zero. i2ospOf :: Integer -> Ptr Word8 -> Int -> IO Int i2ospOf m ptr ptrSz     | ptrSz <= 0 = return 0-    | m < 0      = return 0+    | m < 0 = return 0     | ptrSz < sz = return 0-    | otherwise  = do+    | otherwise = do         memSet ptr 0 ptrSz         fillPtr (ptr `plusPtr` padSz) sz m         return ptrSz   where-    !sz    = numBytes m+    !sz = numBytes m     !padSz = ptrSz - sz  fillPtr :: Ptr Word8 -> Int -> Integer -> IO ()-fillPtr p sz m = gmpExportInteger m p `onGmpUnsupported` export (sz-1) m+fillPtr p sz m+    -- zero is no bytes wide, and the callers above have already written the+    -- room out as zeros.  Without this the loop below starts at offset -1,+    -- never meets the 0 it stops at, and walks backwards out of the buffer.+    | sz <= 0 = return ()+    | otherwise = gmpExportInteger m p `onGmpUnsupported` export (sz - 1) m   where     export ofs i-        | ofs == 0  = pokeByteOff p ofs (fromIntegral i :: Word8)+        | ofs == 0 = pokeByteOff p ofs (fromIntegral i :: Word8)         | otherwise = do             let (i', b) = i `divMod` 256             pokeByteOff p ofs (fromIntegral b :: Word8)-            export (ofs-1) i'+            export (ofs - 1) i'  -- | Transform a big endian binary integer representation pointed by a pointer and a size -- into an integer os2ip :: Ptr Word8 -> Int -> IO Integer os2ip ptr ptrSz     | ptrSz <= 0 = return 0-    | otherwise  = gmpImportInteger ptrSz ptr `onGmpUnsupported` loop 0 0 ptr+    | otherwise = gmpImportInteger ptrSz ptr `onGmpUnsupported` loop 0 0 ptr   where     loop :: Integer -> Int -> Ptr Word8 -> IO Integer     loop !acc i !p         | i == ptrSz = return acc-        | otherwise  = do+        | otherwise = do             w <- peekByteOff p i :: IO Word8-            loop ((acc `shiftL` 8) .|. fromIntegral w) (i+1) p+            loop ((acc `shiftL` 8) .|. fromIntegral w) (i + 1) p
Crypto/Number/Serialize/Internal/LE.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE BangPatterns #-}+ -- | -- Module      : Crypto.Number.Serialize.Internal.LE -- License     : BSD-style@@ -6,20 +8,19 @@ -- Portability : Good -- -- Fast serialization primitives for integer using raw pointers (little endian)-{-# LANGUAGE BangPatterns #-}-module Crypto.Number.Serialize.Internal.LE-    ( i2osp-    , i2ospOf-    , os2ip-    ) where+module Crypto.Number.Serialize.Internal.LE (+    i2osp,+    i2ospOf,+    os2ip,+) where -import           Crypto.Number.Compat-import           Crypto.Number.Basic-import           Data.Bits-import           Data.Memory.PtrMethods-import           Data.Word (Word8)-import           Foreign.Ptr-import           Foreign.Storable+import Crypto.Number.Basic+import Crypto.Number.Compat+import Data.Bits+import Data.Memory.PtrMethods+import Data.Word (Word8)+import Foreign.Ptr+import Foreign.Storable  -- | Fill a pointer with the little endian binary representation of an integer --@@ -30,25 +31,25 @@ i2osp :: Integer -> Ptr Word8 -> Int -> IO Int i2osp m ptr ptrSz     | ptrSz <= 0 = return 0-    | m < 0      = return 0-    | m == 0     = pokeByteOff ptr 0 (0 :: Word8) >> return 1+    | m < 0 = return 0+    | m == 0 = pokeByteOff ptr 0 (0 :: Word8) >> return 1     | ptrSz < sz = return 0-    | otherwise  = fillPtr ptr sz m >> return sz+    | otherwise = fillPtr ptr sz m >> return sz   where-    !sz    = numBytes m+    !sz = numBytes m  -- | Similar to 'i2osp', except it will pad any remaining space with zero. i2ospOf :: Integer -> Ptr Word8 -> Int -> IO Int i2ospOf m ptr ptrSz     | ptrSz <= 0 = return 0-    | m < 0      = return 0+    | m < 0 = return 0     | ptrSz < sz = return 0-    | otherwise  = do+    | otherwise = do         memSet ptr 0 ptrSz         fillPtr ptr sz m         return ptrSz   where-    !sz    = numBytes m+    !sz = numBytes m  fillPtr :: Ptr Word8 -> Int -> Integer -> IO () fillPtr p sz m = gmpExportIntegerLE m p `onGmpUnsupported` export 0 m@@ -58,18 +59,19 @@         | otherwise = do             let (i', b) = i `divMod` 256             pokeByteOff p ofs (fromIntegral b :: Word8)-            export (ofs+1) i'+            export (ofs + 1) i'  -- | Transform a little endian binary integer representation pointed by a -- pointer and a size into an integer os2ip :: Ptr Word8 -> Int -> IO Integer os2ip ptr ptrSz     | ptrSz <= 0 = return 0-    | otherwise  = gmpImportIntegerLE ptrSz ptr `onGmpUnsupported` loop 0 (ptrSz-1) ptr+    | otherwise =+        gmpImportIntegerLE ptrSz ptr `onGmpUnsupported` loop 0 (ptrSz - 1) ptr   where     loop :: Integer -> Int -> Ptr Word8 -> IO Integer     loop !acc i !p-        | i < 0      = return acc-        | otherwise  = do+        | i < 0 = return acc+        | otherwise = do             w <- peekByteOff p i :: IO Word8-            loop ((acc `shiftL` 8) .|. fromIntegral w) (i-1) p+            loop ((acc `shiftL` 8) .|. fromIntegral w) (i - 1) p
Crypto/Number/Serialize/LE.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE BangPatterns #-}+ -- | -- Module      : Crypto.Number.Serialize.LE -- License     : BSD-style@@ -6,17 +8,16 @@ -- Portability : Good -- -- Fast serialization primitives for integer (little endian)-{-# LANGUAGE BangPatterns #-}-module Crypto.Number.Serialize.LE-    ( i2osp-    , os2ip-    , i2ospOf-    , i2ospOf_-    ) where+module Crypto.Number.Serialize.LE (+    i2osp,+    os2ip,+    i2ospOf,+    i2ospOf_,+) where -import           Crypto.Number.Basic-import           Crypto.Internal.Compat (unsafeDoIO) import qualified Crypto.Internal.ByteArray as B+import Crypto.Internal.Compat (unsafeDoIO)+import Crypto.Number.Basic import qualified Crypto.Number.Serialize.Internal.LE as Internal  -- | @os2ip@ converts a byte string into a positive integer.@@ -27,23 +28,24 @@ -- -- The first byte is LSB (least significant byte); the last byte is the MSB (most significant byte) i2osp :: B.ByteArray ba => Integer -> ba-i2osp 0 = B.allocAndFreeze 1  (\p -> Internal.i2osp 0 p 1 >> return ())+i2osp 0 = B.allocAndFreeze 1 (\p -> Internal.i2osp 0 p 1 >> return ()) i2osp m = B.allocAndFreeze sz (\p -> Internal.i2osp m p sz >> return ())   where-        !sz = numBytes m+    !sz = numBytes m  -- | Just like 'i2osp', but takes an extra parameter for size. -- If the number is too big to fit in @len@ bytes, 'Nothing' is returned -- otherwise the number is padded with 0 to fit the @len@ required.-{-# INLINABLE i2ospOf #-}+{-# INLINEABLE i2ospOf #-} i2ospOf :: B.ByteArray ba => Int -> Integer -> Maybe ba i2ospOf len m-    | len <= 0  = Nothing-    | m < 0     = Nothing-    | sz > len  = Nothing-    | otherwise = Just $ B.unsafeCreate len (\p -> Internal.i2ospOf m p len >> return ())+    | len <= 0 = Nothing+    | m < 0 = Nothing+    | sz > len = Nothing+    | otherwise =+        Just $ B.unsafeCreate len (\p -> Internal.i2ospOf m p len >> return ())   where-        !sz = numBytes m+    !sz = numBytes m  -- | Just like 'i2ospOf' except that it doesn't expect a failure: i.e. -- an integer larger than the number of output bytes requested.
Crypto/OTP.hs view
@@ -1,3 +1,4 @@+{-# LANGUAGE BangPatterns #-} {-# LANGUAGE ScopedTypeVariables #-}  -- | One-time password implementation as defined by the@@ -25,33 +26,32 @@ -- >>> import Data.Time.Clock.POSIX -- >>> -- >>> let getOTPTime = getPOSIXTime >>= \t -> return (floor t :: OTPTime)-----module Crypto.OTP-    ( OTP-    , OTPDigits (..)-    , OTPTime-    , hotp-    , resynchronize-    , totp-    , totpVerify-    , TOTPParams-    , ClockSkew (..)-    , defaultTOTPParams-    , mkTOTPParams-    )+module Crypto.OTP (+    OTP,+    OTPDigits (..),+    OTPTime,+    minimumDigestSize,+    hotp,+    resynchronize,+    totp,+    totpVerify,+    TOTPParams,+    ClockSkew (..),+    defaultTOTPParams,+    mkTOTPParams,+) where -import           Data.Bits (shiftL, (.&.), (.|.))-import           Data.ByteArray.Mapping (fromW64BE)-import           Data.List (elemIndex)-import           Data.Word-import           Control.Monad (unless)-import           Crypto.Hash (HashAlgorithm, SHA1(..))-import           Crypto.MAC.HMAC-import           Crypto.Internal.ByteArray (ByteArrayAccess, Bytes)+import Control.Monad (unless)+import Crypto.Hash (HashAlgorithm, SHA1 (..), hashDigestSize)+import Crypto.Internal.ByteArray (ByteArrayAccess, Bytes) import qualified Crypto.Internal.ByteArray as B-+import Crypto.MAC.HMAC+import Data.Bits (complement, shiftL, shiftR, xor, (.&.), (.|.))+import Data.ByteArray.Mapping (fromW64BE)+import Data.List (foldl')+import Data.Word+import Prelude hiding (foldl')  -- | A one-time password which is a sequence of 4 to 9 digits. type OTP = Word32@@ -63,7 +63,24 @@ -- | An integral time value in seconds. type OTPTime = Word64 -hotp :: forall hash key. (HashAlgorithm hash, ByteArrayAccess key)+-- | The smallest hash digest 'hotp' can be used with, in bytes.+--+-- RFC 4226 section 5.3 defines dynamic truncation over the 20-byte HMAC-SHA-1+-- output: the offset is the low four bits of the last byte, so it selects any+-- of the first 16 bytes, and four bytes are then read starting there.  The+-- highest byte that can be reached is therefore byte 18, and a shorter digest+-- would make that read run off the end of the MAC.+minimumDigestSize :: Int+minimumDigestSize = 20++-- | Calculate an HOTP value as defined by RFC 4226.+--+-- The hash must produce a digest of at least 'minimumDigestSize' bytes, which+-- is what the dynamic truncation step is defined over; 'error' is raised+-- otherwise.+hotp+    :: forall hash key+     . (HashAlgorithm hash, ByteArrayAccess key)     => hash     -> OTPDigits     -- ^ Number of digits in the HOTP value extracted from the calculated HMAC@@ -73,18 +90,35 @@     -- ^ Counter value synchronized between the client and server     -> OTP     -- ^ The HOTP value-hotp _ d k c = dt `mod` digitsPower d+hotp _ d k c+    | macLen < minimumDigestSize =+        error $+            "Crypto.OTP.hotp: hash digest is "+                ++ show macLen+                ++ " bytes, but at least "+                ++ show minimumDigestSize+                ++ " are required"+    | otherwise = dt `mod` digitsPower d   where     mac = hmac k (fromW64BE c :: Bytes) :: HMAC hash-    offset = fromIntegral (B.index mac (B.length mac - 1) .&. 0xf)-    dt = (fromIntegral (B.index mac offset       .&. 0x7f) `shiftL` 24) .|.-         (fromIntegral (B.index mac (offset + 1) .&. 0xff) `shiftL` 16) .|.-         (fromIntegral (B.index mac (offset + 2) .&. 0xff) `shiftL`  8) .|.-         fromIntegral  (B.index mac (offset + 3) .&. 0xff)+    macLen = B.length mac+    offset = fromIntegral (B.index mac (macLen - 1) .&. 0xf)+    dt =+        (fromIntegral (B.index mac offset .&. 0x7f) `shiftL` 24)+            .|. (fromIntegral (B.index mac (offset + 1) .&. 0xff) `shiftL` 16)+            .|. (fromIntegral (B.index mac (offset + 2) .&. 0xff) `shiftL` 8)+            .|. fromIntegral (B.index mac (offset + 3) .&. 0xff)  -- | Attempt to resynchronize the server's counter value -- with the client, given a sequence of HOTP values.-resynchronize :: (HashAlgorithm hash, ByteArrayAccess key)+--+-- Every counter in the window is tried and every submitted value is compared,+-- whatever matches, so the time taken does not depend on where in the window+-- the client's counter was found, nor on how many of the submitted values were+-- right.  The cost of a call is therefore one HMAC per counter in the window+-- plus one per extra value, every time.+resynchronize+    :: (HashAlgorithm hash, ByteArrayAccess key)     => hash     -> OTPDigits     -> Word16@@ -100,17 +134,46 @@     -> Maybe Word64     -- ^ The new counter value, synchronized with the client's current counter     -- or Nothing if the submitted OTP values didn't match anywhere within the window-resynchronize h d s k c (p1, extras) = do-    offBy <- fmap fromIntegral (elemIndex p1 range)-    checkExtraOtps (c + offBy + 1) extras+resynchronize h d s k c (p1, extras)+    | accepted == 0 = Nothing+    | otherwise = Just (afterFirst + fromIntegral (length extras))   where-    checkExtraOtps ctr [] = Just ctr-    checkExtraOtps ctr (p:ps)-        | hotp h d k ctr /= p = Nothing-        | otherwise           = checkExtraOtps (ctr + 1) ps+    -- Every counter in the window is tried and every extra value is compared,+    -- whatever matches: the search does not stop at the first hit and the+    -- check of the extra values does not stop at the first miss.  Each skipped+    -- counter used to save an HMAC, so the time taken revealed where in the+    -- window the client's counter sat and how many of its extra values were+    -- right -- the second of which a client that submits guesses cannot learn+    -- from the answer itself, since that is 'Nothing' either way.+    accepted = matched .&. extrasMatched -    range = map (hotp h d k)[c..c + fromIntegral s]+    range = map (hotp h d k) [c .. c + fromIntegral s] +    -- the offset of the first match, accumulated without stopping there+    (matched, offset) = foldl' pick (0, 0) (zip [0 ..] range)+    pick (!m, !off) (i, candidate) = (m .|. hit, off .|. (hit .&. i))+      where+        -- zero once something has matched, so only the first match counts+        hit = eqMask candidate p1 .&. complement m++    -- the counter the first submitted value matched, plus one+    afterFirst = c + offset + 1++    -- the counters continue past the window, and wrap where the old+    -- 'checkExtraOtps' wrapped+    extrasMatched =+        foldl' step (complement 0) (zip (iterate (+ 1) afterFirst) extras)+    step acc (ctr, p) = acc .&. eqMask (hotp h d k ctr) p++-- | All ones when the two values are equal, zero otherwise, without branching+-- on either of them.+eqMask :: OTP -> OTP -> Word64+eqMask a b = negate (fromIntegral (1 - nonZero))+  where+    v = a `xor` b+    -- 0 when v is zero, 1 otherwise+    nonZero = (v .|. negate v) `shiftR` 31+ digitsPower :: OTPDigits -> Word32 digitsPower OTP4 = 10000 digitsPower OTP5 = 100000@@ -119,17 +182,18 @@ digitsPower OTP8 = 100000000 digitsPower OTP9 = 1000000000 - data TOTPParams h = TP !h !OTPTime !Word16 !OTPDigits !ClockSkew deriving (Show) -data ClockSkew = NoSkew | OneStep | TwoSteps | ThreeSteps | FourSteps deriving (Enum, Show)+data ClockSkew = NoSkew | OneStep | TwoSteps | ThreeSteps | FourSteps+    deriving (Enum, Show)  -- | The default TOTP configuration. defaultTOTPParams :: TOTPParams SHA1 defaultTOTPParams = TP SHA1 0 30 OTP6 TwoSteps  -- | Create a TOTP configuration with customized parameters.-mkTOTPParams :: (HashAlgorithm hash)+mkTOTPParams+    :: HashAlgorithm hash     => hash     -> OTPTime     -- ^ The T0 parameter in seconds. This is the Unix time from which to start@@ -146,10 +210,18 @@ mkTOTPParams h t0 x d skew = do     unless (x > 0) (Left "Time step must be greater than zero")     unless (x <= 300) (Left "Time step cannot be greater than 300 seconds")+    unless+        (hashDigestSize h >= minimumDigestSize)+        ( Left $+            "Hash digest must be at least "+                ++ show minimumDigestSize+                ++ " bytes"+        )     return (TP h t0 x d skew)  -- | Calculate a totp value for the given time.-totp :: (HashAlgorithm hash, ByteArrayAccess key)+totp+    :: (HashAlgorithm hash, ByteArrayAccess key)     => TOTPParams hash     -> key     -- ^ The shared secret@@ -161,18 +233,25 @@  -- | Check a supplied TOTP value is valid for the given time, -- within the window defined by the skew parameter.-totpVerify :: (HashAlgorithm hash, ByteArrayAccess key)+totpVerify+    :: (HashAlgorithm hash, ByteArrayAccess key)     => TOTPParams hash     -> key     -> OTPTime     -> OTP     -> Bool-totpVerify (TP h t0 x d skew) k now otp = otp `elem` map (hotp h d k) (range window [])+totpVerify (TP h t0 x d skew) k now otp = matched /= 0   where     t = timeToCounter now t0 x     window = fromIntegral (fromEnum skew)     range 0 acc = t : acc-    range n acc = range (n-1) ((t-n) : (t+n) : acc)+    range n acc = range (n - 1) ((t - n) : (t + n) : acc)++    -- every candidate is compared, and none of the comparisons stops early, so+    -- neither which step matched nor how far a mismatch got is visible in how+    -- long this takes+    matched = foldl' step 0 (map (hotp h d k) (range window []))+    step acc candidate = acc .|. eqMask candidate otp  timeToCounter :: Word64 -> Word64 -> Word16 -> Word64 timeToCounter now t0 x = (now - t0) `div` fromIntegral x
Crypto/PubKey/Curve25519.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# LANGUAGE ScopedTypeVariables #-}+ -- | -- Module      : Crypto.PubKey.Curve25519 -- License     : BSD-style@@ -6,55 +9,62 @@ -- Portability : unknown -- -- Curve25519 support----{-# LANGUAGE GeneralizedNewtypeDeriving #-}-{-# LANGUAGE MagicHash #-}-{-# LANGUAGE ScopedTypeVariables #-}-module Crypto.PubKey.Curve25519-    ( SecretKey-    , PublicKey-    , DhSecret+module Crypto.PubKey.Curve25519 (+    SecretKey,+    PublicKey,+    DhSecret,+     -- * Smart constructors-    , dhSecret-    , publicKey-    , secretKey+    dhSecret,+    publicKey,+    secretKey,+     -- * Methods-    , dh-    , toPublic-    , generateSecretKey-    ) where+    dh,+    toPublic,+    generateSecretKey,+) where -import           Data.Bits-import           Data.Word-import           Foreign.Ptr-import           Foreign.Storable-import           GHC.Ptr+import Crypto.Debug (DebugShow (..), debugShowBytes)+import Data.Bits+import Data.Word+import Foreign.Ptr+import Foreign.Storable -import           Crypto.Error-import           Crypto.Internal.Compat-import           Crypto.Internal.Imports-import           Crypto.Internal.ByteArray (ByteArrayAccess, ScrubbedBytes, Bytes, withByteArray)+import Crypto.Error+import Crypto.Internal.ByteArray (+    ByteArrayAccess,+    Bytes,+    ScrubbedBytes,+    withByteArray,+ ) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Random+import Crypto.Internal.Compat+import Crypto.Internal.Imports+import Crypto.Random  -- | A Curve25519 Secret key newtype SecretKey = SecretKey ScrubbedBytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData) +instance DebugShow SecretKey where+    debugShow = debugShowBytes "SecretKey"+ -- | A Curve25519 public key newtype PublicKey = PublicKey Bytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData)  -- | A Curve25519 Diffie Hellman secret related to a -- public key and a secret key. newtype DhSecret = DhSecret ScrubbedBytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData)  -- | Try to build a public key from a bytearray publicKey :: ByteArrayAccess bs => bs -> CryptoFailable PublicKey publicKey bs-    | B.length bs == 32 = CryptoPassed $ PublicKey $ B.copyAndFreeze bs (\_ -> return ())-    | otherwise         = CryptoFailed CryptoError_PublicKeySizeInvalid+    | B.length bs == 32 =+        CryptoPassed $ PublicKey $ B.copyAndFreeze bs (\_ -> return ())+    | otherwise = CryptoFailed CryptoError_PublicKeySizeInvalid  -- | Try to build a secret key from a bytearray secretKey :: ByteArrayAccess bs => bs -> CryptoFailable SecretKey@@ -67,14 +77,14 @@                 else return $ CryptoFailed CryptoError_SecretKeyStructureInvalid     | otherwise = CryptoFailed CryptoError_SecretKeySizeInvalid   where-        --  e[0] &= 0xf8;-        --  e[31] &= 0x7f;-        --  e[31] |= 40;-        isValidPtr :: Ptr Word8 -> IO Bool-        isValidPtr _ = do-            --b0  <- peekElemOff inp 0-            --b31 <- peekElemOff inp 31-            return True+    --  e[0] &= 0xf8;+    --  e[31] &= 0x7f;+    --  e[31] |= 40;+    isValidPtr :: Ptr Word8 -> IO Bool+    isValidPtr _ = do+        -- b0  <- peekElemOff inp 0+        -- b31 <- peekElemOff inp 31+        return True {-             return $ and [ testBit b0  0 == False                          , testBit b0  1 == False@@ -88,29 +98,33 @@ -- | Create a DhSecret from a bytearray object dhSecret :: ByteArrayAccess b => b -> CryptoFailable DhSecret dhSecret bs-    | B.length bs == 32 = CryptoPassed $ DhSecret $ B.copyAndFreeze bs (\_ -> return ())-    | otherwise         = CryptoFailed CryptoError_SharedSecretSizeInvalid+    | B.length bs == 32 =+        CryptoPassed $ DhSecret $ B.copyAndFreeze bs (\_ -> return ())+    | otherwise = CryptoFailed CryptoError_SharedSecretSizeInvalid  -- | Compute the Diffie Hellman secret from a public key and a secret key. -- -- This implementation may return an all-zero value as it does not check for -- the condition. dh :: PublicKey -> SecretKey -> DhSecret-dh (PublicKey pub) (SecretKey sec) = DhSecret <$>-    B.allocAndFreeze 32        $ \result ->-    withByteArray sec          $ \psec   ->-    withByteArray pub          $ \ppub   ->-        ccrypton_curve25519 result psec ppub+dh (PublicKey pub) (SecretKey sec) = DhSecret+    <$> B.allocAndFreeze 32+    $ \result ->+        withByteArray sec $ \psec ->+            withByteArray pub $ \ppub ->+                ccrypton_x25519 result psec ppub {-# NOINLINE dh #-}  -- | Create a public key from a secret key+-- The base point does not go in: where the assembly is built there is a table+-- for this, and it is four to five times less work than multiplying the point+-- 9 the general way. toPublic :: SecretKey -> PublicKey-toPublic (SecretKey sec) = PublicKey <$>-    B.allocAndFreeze 32     $ \result ->-    withByteArray sec       $ \psec   ->-        ccrypton_curve25519 result psec basePoint-  where-        basePoint = Ptr "\x09\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"#+toPublic (SecretKey sec) = PublicKey+    <$> B.allocAndFreeze 32+    $ \result ->+        withByteArray sec $ \psec ->+            ccrypton_x25519_base result psec {-# NOINLINE toPublic #-}  -- | Generate a secret key.@@ -125,8 +139,20 @@     modifyByte :: Ptr Word8 -> Int -> (Word8 -> Word8) -> IO ()     modifyByte p n f = peekByteOff p n >>= pokeByteOff p n . f -foreign import ccall "crypton_curve25519_donna"-    ccrypton_curve25519 :: Ptr Word8 -- ^ public-                           -> Ptr Word8 -- ^ secret-                           -> Ptr Word8 -- ^ basepoint-                           -> IO ()+foreign import ccall "crypton_x25519"+    ccrypton_x25519+        :: Ptr Word8+        -- ^ public+        -> Ptr Word8+        -- ^ secret+        -> Ptr Word8+        -- ^ basepoint+        -> IO ()++foreign import ccall "crypton_x25519_base"+    ccrypton_x25519_base+        :: Ptr Word8+        -- ^ public+        -> Ptr Word8+        -- ^ secret+        -> IO ()
Crypto/PubKey/Curve448.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.PubKey.Curve448 -- License     : BSD-style@@ -10,50 +12,60 @@ -- Internally uses Decaf point compression to omit the cofactor -- and implementation by Mike Hamburg.  Externally API and -- data types are compatible with the encoding specified in RFC 7748.----{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.PubKey.Curve448-    ( SecretKey-    , PublicKey-    , DhSecret+module Crypto.PubKey.Curve448 (+    SecretKey,+    PublicKey,+    DhSecret,+     -- * Smart constructors-    , dhSecret-    , publicKey-    , secretKey+    dhSecret,+    publicKey,+    secretKey,+     -- * Methods-    , dh-    , toPublic-    , generateSecretKey-    ) where+    dh,+    toPublic,+    generateSecretKey,+) where -import           Data.Word-import           Foreign.Ptr+import Crypto.Debug (DebugShow (..), debugShowBytes)+import Data.Word+import Foreign.Ptr -import           Crypto.Error-import           Crypto.Random-import           Crypto.Internal.Compat-import           Crypto.Internal.Imports-import           Crypto.Internal.ByteArray (ByteArrayAccess, ScrubbedBytes, Bytes, withByteArray)+import Crypto.Error+import Crypto.Internal.ByteArray (+    ByteArrayAccess,+    Bytes,+    ScrubbedBytes,+    withByteArray,+ ) import qualified Crypto.Internal.ByteArray as B+import Crypto.Internal.Compat+import Crypto.Internal.Imports+import Crypto.Random  -- | A Curve448 Secret key newtype SecretKey = SecretKey ScrubbedBytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData) +instance DebugShow SecretKey where+    debugShow = debugShowBytes "SecretKey"+ -- | A Curve448 public key newtype PublicKey = PublicKey Bytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData)  -- | A Curve448 Diffie Hellman secret related to a -- public key and a secret key. newtype DhSecret = DhSecret ScrubbedBytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData)  -- | Try to build a public key from a bytearray publicKey :: ByteArrayAccess bs => bs -> CryptoFailable PublicKey publicKey bs-    | B.length bs == x448_bytes = CryptoPassed $ PublicKey $ B.copyAndFreeze bs (\_ -> return ())-    | otherwise                 = CryptoFailed CryptoError_PublicKeySizeInvalid+    | B.length bs == x448_bytes =+        CryptoPassed $ PublicKey $ B.copyAndFreeze bs (\_ -> return ())+    | otherwise = CryptoFailed CryptoError_PublicKeySizeInvalid  -- | Try to build a secret key from a bytearray secretKey :: ByteArrayAccess bs => bs -> CryptoFailable SecretKey@@ -66,35 +78,38 @@                 else return $ CryptoFailed CryptoError_SecretKeyStructureInvalid     | otherwise = CryptoFailed CryptoError_SecretKeySizeInvalid   where-        isValidPtr :: Ptr Word8 -> IO Bool-        isValidPtr _ =-            return True+    isValidPtr :: Ptr Word8 -> IO Bool+    isValidPtr _ =+        return True {-# NOINLINE secretKey #-}  -- | Create a DhSecret from a bytearray object dhSecret :: ByteArrayAccess b => b -> CryptoFailable DhSecret dhSecret bs-    | B.length bs == x448_bytes = CryptoPassed $ DhSecret $ B.copyAndFreeze bs (\_ -> return ())-    | otherwise                 = CryptoFailed CryptoError_SharedSecretSizeInvalid+    | B.length bs == x448_bytes =+        CryptoPassed $ DhSecret $ B.copyAndFreeze bs (\_ -> return ())+    | otherwise = CryptoFailed CryptoError_SharedSecretSizeInvalid  -- | Compute the Diffie Hellman secret from a public key and a secret key. -- -- This implementation may return an all-zero value as it does not check for -- the condition. dh :: PublicKey -> SecretKey -> DhSecret-dh (PublicKey pub) (SecretKey sec) = DhSecret <$>-    B.allocAndFreeze x448_bytes $ \result ->-    withByteArray sec           $ \psec   ->-    withByteArray pub           $ \ppub   ->-        decaf_x448 result ppub psec+dh (PublicKey pub) (SecretKey sec) = DhSecret+    <$> B.allocAndFreeze x448_bytes+    $ \result ->+        withByteArray sec $ \psec ->+            withByteArray pub $ \ppub ->+                decaf_x448 result ppub psec {-# NOINLINE dh #-}  -- | Create a public key from a secret key toPublic :: SecretKey -> PublicKey-toPublic (SecretKey sec) = PublicKey <$>-    B.allocAndFreeze x448_bytes     $ \result ->-    withByteArray sec               $ \psec   ->-        decaf_x448_derive_public_key result psec+toPublic (SecretKey sec) = PublicKey+    <$> B.allocAndFreeze x448_bytes+    $ \result ->+        withByteArray sec $ \psec ->+            decaf_x448_derive_public_key result psec {-# NOINLINE toPublic #-}  -- | Generate a secret key.@@ -105,12 +120,19 @@ x448_bytes = 448 `quot` 8  foreign import ccall "crypton_decaf_x448"-    decaf_x448 :: Ptr Word8 -- ^ public-               -> Ptr Word8 -- ^ basepoint-               -> Ptr Word8 -- ^ secret-               -> IO ()+    decaf_x448+        :: Ptr Word8+        -- ^ public+        -> Ptr Word8+        -- ^ basepoint+        -> Ptr Word8+        -- ^ secret+        -> IO ()  foreign import ccall "crypton_decaf_x448_derive_public_key"-    decaf_x448_derive_public_key :: Ptr Word8 -- ^ public-                                 -> Ptr Word8 -- ^ secret-                                 -> IO ()+    decaf_x448_derive_public_key+        :: Ptr Word8+        -- ^ public+        -> Ptr Word8+        -- ^ secret+        -> IO ()
Crypto/PubKey/DH.hs view
@@ -1,28 +1,36 @@+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.PubKey.DH -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.PubKey.DH-    ( Params(..)-    , PublicNumber(..)-    , PrivateNumber(..)-    , SharedKey(..)-    , generateParams-    , generatePrivate-    , calculatePublic-    , generatePublic-    , getShared-    ) where+module Crypto.PubKey.DH (+    Params (..),+    PublicNumber (..),+    PrivateNumber (..),+    SharedKey (..),+    generateParams,+    generatePrivate,+    calculatePublic,+    generatePublic,+    getShared,+    tryGetShared,+) where +import Crypto.Debug (DebugShow (..))+import Crypto.Error (+    CryptoError (..),+    CryptoFailable (..),+    throwCryptoError,+ ) import Crypto.Internal.Imports+import Crypto.Number.Basic (numBytes)+import Crypto.Number.Generate (generateMax) import Crypto.Number.ModArithmetic (expSafe) import Crypto.Number.Prime (generateSafePrime)-import Crypto.Number.Generate (generateMax) import Crypto.Number.Serialize (i2ospOf_) import Crypto.Random.Types import Data.ByteArray (ByteArrayAccess, ScrubbedBytes)@@ -33,29 +41,41 @@     { params_p :: Integer     , params_g :: Integer     , params_bits :: Int-    } deriving (Show,Read,Eq,Data)+    }+    deriving (Show, Read, Eq, Data)  instance NFData Params where     rnf (Params p g bits) = rnf p `seq` rnf g `seq` bits `seq` ()  -- | Represent Diffie Hellman public number Y. newtype PublicNumber = PublicNumber Integer-    deriving (Show,Read,Eq,Enum,Real,Num,Ord,NFData)+    deriving (Show, Read, Eq, Enum, Real, Num, Ord, NFData)  -- | Represent Diffie Hellman private number X. newtype PrivateNumber = PrivateNumber Integer-    deriving (Show,Read,Eq,Enum,Real,Num,Ord,NFData)+    deriving (Read, Eq, Enum, Real, Num, Ord, NFData) +-- | The number is not shown.  Use 'Crypto.Debug.debugShow' to see it.+instance Show PrivateNumber where+    show _ = "PrivateNumber <secret>"++instance DebugShow PrivateNumber where+    debugShow (PrivateNumber n) =+        showString "PrivateNumber " . showsPrec 11 n $ ""+ -- | Represent Diffie Hellman shared secret. newtype SharedKey = SharedKey ScrubbedBytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData)  -- | generate params from a specific generator (2 or 5 are common values) -- we generate a safe prime (a prime number of the form 2p+1 where p is also prime)-generateParams :: MonadRandom m =>-                  Int                   -- ^ number of bits-               -> Integer               -- ^ generator-               -> m Params+generateParams+    :: MonadRandom m+    => Int+    -- ^ number of bits+    -> Integer+    -- ^ generator+    -> m Params generateParams bits generator =     (\p -> Params p generator bits) <$> generateSafePrime bits @@ -75,8 +95,33 @@ -- DEPRECATED use calculatePublic generatePublic :: Params -> PrivateNumber -> PublicNumber generatePublic = calculatePublic+ -- commented until 0.3 {-# DEPRECATED generatePublic "use calculatePublic" #-}  -- | generate a shared key using our private number and the other party public number+--+-- This raises the 'CryptoError' that 'tryGetShared' reports.  Use 'tryGetShared'+-- where the failure has to be handled. getShared :: Params -> PrivateNumber -> PublicNumber -> SharedKey-getShared (Params p _ bits) (PrivateNumber x) (PublicNumber y) = SharedKey $ i2ospOf_ ((bits + 7) `div` 8) $ expSafe y x p+getShared params x y = throwCryptoError $ tryGetShared params x y++-- | generate a shared key using our private number and the other party public+-- number, reporting a rejected public number instead of raising.+--+-- The public number comes from the other party, so it is checked to satisfy+-- @1 < y < p-1@ as RFC 7919 section 5.1 requires.  The excluded values+-- generate the subgroup @{1}@ or @{1, p-1}@, so the shared secret they produce+-- is one of a handful of constants and carries none of our private number's+-- secrecy.  A value outside that range is reported as+-- 'CryptoError_ParameterInvalid'.+--+-- Note this is the only check made here: it does not establish that @y@ lies+-- in the subgroup generated by @g@, which needs the subgroup order that+-- t'Params' does not carry.+tryGetShared+    :: Params -> PrivateNumber -> PublicNumber -> CryptoFailable SharedKey+tryGetShared (Params p _ _) (PrivateNumber x) (PublicNumber y)+    | y <= 1 || y >= p - 1 = CryptoFailed CryptoError_ParameterInvalid+    -- the size of p, not params_bits: only p and g travel on the wire, so a+    -- caller-supplied bit size can disagree with p+    | otherwise = CryptoPassed $ SharedKey $ i2ospOf_ (numBytes p) $ expSafe y x p
Crypto/PubKey/DSA.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE DeriveDataTypeable #-}+ -- | -- Module      : Crypto.PubKey.DSA -- License     : BSD-style@@ -6,37 +8,57 @@ -- Portability : Good -- -- An implementation of the Digital Signature Algorithm (DSA)-{-# LANGUAGE DeriveDataTypeable #-}-module Crypto.PubKey.DSA-    ( Params(..)-    , Signature(..)-    , PublicKey(..)-    , PrivateKey(..)-    , PublicNumber-    , PrivateNumber+--+-- == What is kept from the clock, and what is not+--+-- Signing keeps the private number and the ephemeral @k@ out of the two+-- places whose duration would otherwise follow them: the exponentiation is+-- 'Crypto.Number.ModArithmetic.expSafe', which walks the exponent a fixed+-- four bits at a time, and @k@ is inverted by Fermat's little theorem rather+-- than by the extended Euclidean algorithm, whose number of steps follows the+-- bits it is given.+--+-- What is left is the arithmetic around them.  @x * r@, the addition and the+-- reduction modulo @q@ are 'Integer' operations, and an 'Integer' costs what+-- its size says: a private number that happens to be short is multiplied in+-- fewer words than a full-length one.  The same holds in+-- "Crypto.PubKey.ElGamal" and "Crypto.PubKey.Rabin.Basic".  Removing it means+-- leaving 'Integer' for a fixed-width representation, which is what+-- "Crypto.PubKey.RSA" does for its exponentiation and the curve modules do+-- throughout; there is nothing a caller can do about it from here.+module Crypto.PubKey.DSA (+    Params (..),+    Signature (..),+    PublicKey (..),+    PrivateKey (..),+    PublicNumber,+    PrivateNumber,+     -- * Generation-    , generatePrivate-    , calculatePublic+    generatePrivate,+    calculatePublic,+     -- * Signature primitive-    , sign-    , signWith+    sign,+    signWith,+     -- * Verification primitive-    , verify-    -- * Key pair-    , KeyPair(..)-    , toPublicKey-    , toPrivateKey-    ) where+    verify, +    -- * Key pair+    KeyPair (..),+    toPublicKey,+    toPrivateKey,+) where +import Crypto.Debug (DebugShow (..)) import Data.Data-import Data.Maybe -import Crypto.Number.ModArithmetic (expFast, expSafe, inverse)-import Crypto.Number.Generate+import Crypto.Hash import Crypto.Internal.ByteArray (ByteArrayAccess) import Crypto.Internal.Imports-import Crypto.Hash+import Crypto.Number.Generate+import Crypto.Number.ModArithmetic (expFast, expSafe, inverse, inverseSafe) import Crypto.PubKey.Internal (dsaTruncHash) import Crypto.Random.Types @@ -48,28 +70,38 @@  -- | Represent DSA parameters namely P, G, and Q. data Params = Params-    { params_p :: Integer -- ^ DSA p-    , params_g :: Integer -- ^ DSA g-    , params_q :: Integer -- ^ DSA q-    } deriving (Show,Read,Eq,Data)+    { params_p :: Integer+    -- ^ DSA p+    , params_g :: Integer+    -- ^ DSA g+    , params_q :: Integer+    -- ^ DSA q+    }+    deriving (Show, Read, Eq, Data)  instance NFData Params where     rnf (Params p g q) = p `seq` g `seq` q `seq` ()  -- | Represent a DSA signature namely R and S. data Signature = Signature-    { sign_r :: Integer -- ^ DSA r-    , sign_s :: Integer -- ^ DSA s-    } deriving (Show,Read,Eq,Data)+    { sign_r :: Integer+    -- ^ DSA r+    , sign_s :: Integer+    -- ^ DSA s+    }+    deriving (Show, Read, Eq, Data)  instance NFData Signature where     rnf (Signature r s) = r `seq` s `seq` ()  -- | Represent a DSA public key. data PublicKey = PublicKey-    { public_params :: Params       -- ^ DSA parameters-    , public_y      :: PublicNumber -- ^ DSA public Y-    } deriving (Show,Read,Eq,Data)+    { public_params :: Params+    -- ^ DSA parameters+    , public_y :: PublicNumber+    -- ^ DSA public Y+    }+    deriving (Show, Read, Eq, Data)  instance NFData PublicKey where     rnf (PublicKey params y) = y `seq` params `seq` ()@@ -79,17 +111,57 @@ -- Only x need to be secret. -- the DSA parameters are publicly shared with the other side. data PrivateKey = PrivateKey-    { private_params :: Params        -- ^ DSA parameters-    , private_x      :: PrivateNumber -- ^ DSA private X-    } deriving (Show,Read,Eq,Data)+    { private_params :: Params+    -- ^ DSA parameters+    , private_x :: PrivateNumber+    -- ^ DSA private X+    }+    deriving (Read, Eq, Data) +-- | The parameters are shown; @private_x@ is not.  Use+-- 'Crypto.Debug.debugShow' to see it.+instance Show PrivateKey where+    showsPrec d k =+        showParen (d > 10) $+            showString "PrivateKey {private_params = "+                . shows (private_params k)+                . showString ", private_x = <secret>}"++instance DebugShow PrivateKey where+    debugShow k =+        showString "PrivateKey {private_params = "+            . shows (private_params k)+            . showString ", private_x = "+            . shows (private_x k)+            . showChar '}'+            $ ""+ instance NFData PrivateKey where     rnf (PrivateKey params x) = x `seq` params `seq` ()  -- | Represent a DSA key pair data KeyPair = KeyPair Params PublicNumber PrivateNumber-    deriving (Show,Read,Eq,Data)+    deriving (Read, Eq, Data) +instance Show KeyPair where+    showsPrec d (KeyPair params y _) =+        showParen (d > 10) $+            showString "KeyPair "+                . showsPrec 11 params+                . showChar ' '+                . showsPrec 11 y+                . showString " <secret>"++instance DebugShow KeyPair where+    debugShow (KeyPair params y x) =+        showString "KeyPair "+            . showsPrec 11 params+            . showChar ' '+            . showsPrec 11 y+            . showChar ' '+            . showsPrec 11 x+            $ ""+ instance NFData KeyPair where     rnf (KeyPair params y x) = x `seq` y `seq` params `seq` () @@ -111,44 +183,61 @@ calculatePublic (Params p g _) x = expSafe g x p  -- | sign message using the private key and an explicit k number.-signWith :: (ByteArrayAccess msg, HashAlgorithm hash)-         => Integer         -- ^ k random number-         -> PrivateKey      -- ^ private key-         -> hash            -- ^ hash function-         -> msg             -- ^ message to sign-         -> Maybe Signature-signWith k pk hashAlg msg-    | r == 0 || s == 0  = Nothing-    | otherwise         = Just $ Signature r s-    where -- parameters-          (Params p g q) = private_params pk-          x              = private_x pk-          -- compute r,s-          kInv      = fromJust $ inverse k q-          hm        = dsaTruncHash hashAlg msg q-          r         = expSafe g k p `mod` q-          s         = (kInv * (hm + x * r)) `mod` q+signWith+    :: (ByteArrayAccess msg, HashAlgorithm hash)+    => Integer+    -- ^ k random number+    -> PrivateKey+    -- ^ private key+    -> hash+    -- ^ hash function+    -> msg+    -- ^ message to sign+    -> Maybe Signature+signWith k pk hashAlg msg = do+    -- k comes from the caller and is only invertible when it is coprime with+    -- q, which the caller cannot check without knowing q is prime.  It is also+    -- a secret worth as much as the private key, so it is inverted without+    -- the extended Euclidean algorithm, whose steps follow the bits of what+    -- it is given+    kInv <- inverseSafe k q+    let hm = dsaTruncHash hashAlg msg q+        r = expSafe g k p `mod` q+        s = (kInv * (hm + x * r)) `mod` q+    if r == 0 || s == 0 then Nothing else Just $ Signature r s+  where+    -- parameters+    (Params p g q) = private_params pk+    x = private_x pk  -- | sign message using the private key.-sign :: (ByteArrayAccess msg, HashAlgorithm hash, MonadRandom m) => PrivateKey -> hash -> msg -> m Signature+sign+    :: (ByteArrayAccess msg, HashAlgorithm hash, MonadRandom m)+    => PrivateKey -> hash -> msg -> m Signature sign pk hashAlg msg = do     k <- generateMax q     case signWith k pk hashAlg msg of-        Nothing  -> sign pk hashAlg msg+        Nothing -> sign pk hashAlg msg         Just sig -> return sig   where     (Params _ _ q) = private_params pk  -- | verify a bytestring using the public key.-verify :: (ByteArrayAccess msg, HashAlgorithm hash) => hash -> PublicKey -> Signature -> msg -> Bool+verify+    :: (ByteArrayAccess msg, HashAlgorithm hash)+    => hash -> PublicKey -> Signature -> msg -> Bool verify hashAlg pk (Signature r s) m     -- Reject the signature if either 0 < r < q or 0 < s < q is not satisfied.     | r <= 0 || r >= q || s <= 0 || s >= q = False-    | otherwise                            = v == r-    where (Params p g q) = public_params pk-          y       = public_y pk-          hm      = dsaTruncHash hashAlg m q-          w       = fromJust $ inverse s q-          u1      = (hm*w) `mod` q-          u2      = (r*w) `mod` q-          v       = ((expFast g u1 p) * (expFast y u2 p)) `mod` p `mod` q+    -- s is invertible for every 0 < s < q when q is prime, but the parameters+    -- arrive with the public key and a composite q admits an s that is not+    | otherwise = maybe False (r ==) v+  where+    (Params p g q) = public_params pk+    y = public_y pk+    hm = dsaTruncHash hashAlg m q+    v = do+        w <- inverse s q+        let u1 = (hm * w) `mod` q+            u2 = (r * w) `mod` q+        return $ ((expFast g u1 p) * (expFast y u2 p)) `mod` p `mod` q
Crypto/PubKey/ECC/DH.hs view
@@ -6,25 +6,37 @@ -- Portability : unknown -- -- Elliptic curve Diffie Hellman----module Crypto.PubKey.ECC.DH-    (-      Curve-    , PublicPoint-    , PrivateNumber-    , SharedKey(..)-    , generatePrivate-    , calculatePublic-    , getShared-    ) where+module Crypto.PubKey.ECC.DH (+    Curve,+    PublicPoint,+    PrivateNumber,+    SharedKey (..),+    generatePrivate,+    calculatePublic,+    getShared,+    tryGetShared,+) where +import Crypto.Error (+    CryptoError (..),+    CryptoFailable (..),+    throwCryptoError,+ ) import Crypto.Number.Generate (generateMax) import Crypto.Number.Serialize (i2ospOf_)-import Crypto.PubKey.ECC.Prim (pointMul)+import Crypto.PubKey.DH (SharedKey (..))+import Crypto.PubKey.ECC.Prim (isPointInSubgroup, isPointValid, pointMul)+import Crypto.PubKey.ECC.Types (+    Curve,+    Point (..),+    PrivateNumber,+    PublicPoint,+    common_curve,+    curveSizeBits,+    ecc_g,+    ecc_n,+ ) import Crypto.Random.Types-import Crypto.PubKey.DH (SharedKey(..))-import Crypto.PubKey.ECC.Types (PublicPoint, PrivateNumber, Curve, Point(..), curveSizeBits)-import Crypto.PubKey.ECC.Types (ecc_n, ecc_g, common_curve)  -- | Generating a private number d. generatePrivate :: MonadRandom m => Curve -> m PrivateNumber@@ -41,8 +53,37 @@  -- | Generating a shared key using our private number and --   the other party public point.+--+-- This raises the 'Crypto.Error.CryptoError' that 'tryGetShared' reports.  Use+-- 'tryGetShared' where the failure has to be handled. getShared :: Curve -> PrivateNumber -> PublicPoint -> SharedKey-getShared curve db qa = SharedKey $ i2ospOf_ ((nbBits + 7) `div` 8) x+getShared curve db qa = throwCryptoError $ tryGetShared curve db qa++-- | Generating a shared key using our private number and the other party+--   public point, reporting a rejected point instead of raising.+--+-- The public point comes from the other party, so it is checked before it is+-- multiplied.  A point that does not satisfy the curve equation is reported as+-- 'CryptoError_PointCoordinatesInvalid'.+--+-- Satisfying the equation is not by itself membership of the subgroup the base+-- point generates; the two coincide only when the cofactor is 1.  On a curve+-- whose cofactor is above 1 the other party can offer a point of small order,+-- and the value that comes back then depends on our private number only+-- through its residue modulo that order, which hands them those bits.  So the+-- point is also required to be in the subgroup, by 'isPointInSubgroup', and is+-- reported as 'CryptoError_PointSubgroupInvalid' when it is not.  That check+-- costs one further scalar multiplication, and is skipped where the cofactor+-- is 1 and it cannot fail.+--+-- An exchange that yields the point at infinity, and so has no x coordinate to+-- derive the key from, is reported as 'CryptoError_ScalarMultiplicationInvalid'.+tryGetShared :: Curve -> PrivateNumber -> PublicPoint -> CryptoFailable SharedKey+tryGetShared curve db qa+    | not (isPointValid curve qa) = CryptoFailed CryptoError_PointCoordinatesInvalid+    | not (isPointInSubgroup curve qa) = CryptoFailed CryptoError_PointSubgroupInvalid+    | otherwise = case pointMul curve db qa of+        Point x _ -> CryptoPassed $ SharedKey $ i2ospOf_ ((nbBits + 7) `div` 8) x+        PointO -> CryptoFailed CryptoError_ScalarMultiplicationInvalid   where-    Point x _ = pointMul curve db qa-    nbBits    = curveSizeBits curve+    nbBits = curveSizeBits curve
Crypto/PubKey/ECC/ECDSA.hs view
@@ -1,57 +1,126 @@--- | /WARNING:/ Signature operations may leak the private key. Signature verification--- should be safe. {-# LANGUAGE DeriveDataTypeable #-}-module Crypto.PubKey.ECC.ECDSA-    ( Signature(..)-    , PublicPoint-    , PublicKey(..)-    , PrivateNumber-    , PrivateKey(..)-    , KeyPair(..)-    , toPublicKey-    , toPrivateKey-    , signWith-    , signDigestWith-    , sign-    , signDigest-    , verify-    , verifyDigest-    ) where +-- | /WARNING:/ Signature operations may leak the private key.  The nonce is+-- inverted without a side channel on every curve, and on P-256 the scalar+-- multiplication is the constant-time C implementation, but what surrounds+-- them is 'Integer' arithmetic, whose cost follows the values it is given, and+-- on every other curve the multiplication follows the nonce as well.+-- Signature verification takes only public values and should be safe.+module Crypto.PubKey.ECC.ECDSA (+    Signature (..),+    ExtendedSignature (..),+    PublicPoint,+    PublicKey (..),+    PrivateNumber,+    PrivateKey (..),+    KeyPair (..),+    toPublicKey,+    toPrivateKey,+    signWith,+    signDigestWith,+    signExtendedDigestWith,+    sign,+    signDigest,+    signExtendedDigest,+    verify,+    verifyDigest,+    recover,+    recoverDigest,+    deterministicNonce,+) where++import Crypto.Debug (DebugShow (..)) import Control.Monad+import Data.Bits+import Data.ByteArray (ByteArrayAccess, ScrubbedBytes) import Data.Data  import Crypto.Hash-import Crypto.Internal.ByteArray (ByteArrayAccess)-import Crypto.Number.ModArithmetic (inverse)+import Crypto.Number.Basic import Crypto.Number.Generate-import Crypto.PubKey.ECC.Types+import Crypto.Number.ModArithmetic (inverse)+import Crypto.Number.Serialize import Crypto.PubKey.ECC.Prim+import Crypto.PubKey.ECC.Types import Crypto.PubKey.Internal (dsaTruncHashDigest)+import Crypto.Random.HmacDRG import Crypto.Random.Types  -- | Represent a ECDSA signature namely R and S. data Signature = Signature-    { sign_r :: Integer -- ^ ECDSA r-    , sign_s :: Integer -- ^ ECDSA s-    } deriving (Show,Read,Eq,Data)+    { sign_r :: Integer+    -- ^ ECDSA r+    , sign_s :: Integer+    -- ^ ECDSA s+    }+    deriving (Show, Read, Eq, Data) +-- | ECDSA signature with public key recovery information.+data ExtendedSignature = ExtendedSignature+    { index :: Integer+    -- ^ Index of the X coordinate+    , parity :: Bool+    -- ^ Parity of the Y coordinate+    , signature :: Signature+    -- ^ Inner signature+    }+    deriving (Show, Read, Eq, Data)+ -- | ECDSA Private Key. data PrivateKey = PrivateKey     { private_curve :: Curve-    , private_d     :: PrivateNumber-    } deriving (Show,Read,Eq,Data)+    , private_d :: PrivateNumber+    }+    deriving (Read, Eq, Data) +-- | The curve is shown; @private_d@ is not.  Use+-- 'Crypto.Debug.debugShow' to see it.+instance Show PrivateKey where+    showsPrec d k =+        showParen (d > 10) $+            showString "PrivateKey {private_curve = "+                . shows (private_curve k)+                . showString ", private_d = <secret>}"++instance DebugShow PrivateKey where+    debugShow k =+        showString "PrivateKey {private_curve = "+            . shows (private_curve k)+            . showString ", private_d = "+            . shows (private_d k)+            . showChar '}'+            $ ""+ -- | ECDSA Public Key. data PublicKey = PublicKey     { public_curve :: Curve-    , public_q     :: PublicPoint-    } deriving (Show,Read,Eq,Data)+    , public_q :: PublicPoint+    }+    deriving (Show, Read, Eq, Data)  -- | ECDSA Key Pair. data KeyPair = KeyPair Curve PublicPoint PrivateNumber-    deriving (Show,Read,Eq,Data)+    deriving (Read, Eq, Data) +instance Show KeyPair where+    showsPrec d (KeyPair c q _) =+        showParen (d > 10) $+            showString "KeyPair "+                . showsPrec 11 c+                . showChar ' '+                . showsPrec 11 q+                . showString " <secret>"++instance DebugShow KeyPair where+    debugShow (KeyPair c q x) =+        showString "KeyPair "+            . showsPrec 11 c+            . showChar ' '+            . showsPrec 11 q+            . showChar ' '+            . showsPrec 11 x+            $ ""+ -- | Public key of a ECDSA Key pair. toPublicKey :: KeyPair -> PublicKey toPublicKey (KeyPair curve pub _) = PublicKey curve pub@@ -63,71 +132,146 @@ -- | Sign digest using the private key and an explicit k number. -- -- /WARNING:/ Vulnerable to timing attacks.-signDigestWith :: HashAlgorithm hash-               => Integer     -- ^ k random number-               -> PrivateKey  -- ^ private key-               -> Digest hash -- ^ digest to sign-               -> Maybe Signature-signDigestWith k (PrivateKey curve d) digest = do+signExtendedDigestWith+    :: HashAlgorithm hash+    => Integer+    -- ^ k random number+    -> PrivateKey+    -- ^ private key+    -> Digest hash+    -- ^ digest to sign+    -> Maybe ExtendedSignature+signExtendedDigestWith k (PrivateKey curve d) digest = do     let z = dsaTruncHashDigest digest n         CurveCommon _ _ g n _ = common_curve curve-    let point = pointMul curve k g-    r <- case point of-              PointO    -> Nothing-              Point x _ -> return $ x `mod` n-    kInv <- inverse k n-    let s = kInv * (z + r * d) `mod` n+    (i, r, p) <- pointDecompose curve $ pointMul curve k g+    kInv <- scalarInverse curve k+    -- kInv and d are secret, so the arithmetic that mixes them goes through+    -- the curve's own, which on P-256 is the C implementation's+    let s = scalarMul curve kInv (scalarAdd curve z (scalarMul curve r d))     when (r == 0 || s == 0) Nothing-    return $ Signature r s+    return $+        if s <= n `unsafeShiftR` 1+            then ExtendedSignature i p $ Signature r s+            else ExtendedSignature i (not p) $ Signature r (n - s) +-- | Sign digest using the private key and an explicit k number.+--+-- /WARNING:/ Vulnerable to timing attacks.+signDigestWith+    :: HashAlgorithm hash+    => Integer+    -- ^ k random number+    -> PrivateKey+    -- ^ private key+    -> Digest hash+    -- ^ digest to sign+    -> Maybe Signature+signDigestWith k pk digest = signature <$> signExtendedDigestWith k pk digest+ -- | Sign message using the private key and an explicit k number. -- -- /WARNING:/ Vulnerable to timing attacks.-signWith :: (ByteArrayAccess msg, HashAlgorithm hash)-         => Integer    -- ^ k random number-         -> PrivateKey -- ^ private key-         -> hash       -- ^ hash function-         -> msg        -- ^ message to sign-         -> Maybe Signature+signWith+    :: (ByteArrayAccess msg, HashAlgorithm hash)+    => Integer+    -- ^ k random number+    -> PrivateKey+    -- ^ private key+    -> hash+    -- ^ hash function+    -> msg+    -- ^ message to sign+    -> Maybe Signature signWith k pk hashAlg msg = signDigestWith k pk (hashWith hashAlg msg)  -- | Sign digest using the private key. -- -- /WARNING:/ Vulnerable to timing attacks.-signDigest :: (HashAlgorithm hash, MonadRandom m)-           => PrivateKey -> Digest hash -> m Signature-signDigest pk digest = do+signExtendedDigest+    :: (HashAlgorithm hash, MonadRandom m)+    => PrivateKey -> Digest hash -> m ExtendedSignature+signExtendedDigest pk digest = do     k <- generateBetween 1 (n - 1)-    case signDigestWith k pk digest of-         Nothing  -> signDigest pk digest-         Just sig -> return sig-  where n = ecc_n . common_curve $ private_curve pk+    case signExtendedDigestWith k pk digest of+        Nothing -> signExtendedDigest pk digest+        Just sig -> return sig+  where+    n = ecc_n . common_curve $ private_curve pk +-- | Sign digest using the private key.+--+-- /WARNING:/ Vulnerable to timing attacks.+signDigest+    :: (HashAlgorithm hash, MonadRandom m)+    => PrivateKey -> Digest hash -> m Signature+signDigest pk digest = signature <$> signExtendedDigest pk digest+ -- | Sign message using the private key. -- -- /WARNING:/ Vulnerable to timing attacks.-sign :: (ByteArrayAccess msg, HashAlgorithm hash, MonadRandom m)-     => PrivateKey -> hash -> msg -> m Signature+sign+    :: (ByteArrayAccess msg, HashAlgorithm hash, MonadRandom m)+    => PrivateKey -> hash -> msg -> m Signature sign pk hashAlg msg = signDigest pk (hashWith hashAlg msg)  -- | Verify a digest using the public key.-verifyDigest :: HashAlgorithm hash => PublicKey -> Signature -> Digest hash -> Bool+verifyDigest+    :: HashAlgorithm hash => PublicKey -> Signature -> Digest hash -> Bool verifyDigest (PublicKey _ PointO) _ _ = False verifyDigest pk@(PublicKey curve q) (Signature r s) digest     | r < 1 || r >= n || s < 1 || s >= n = False     | otherwise = maybe False (r ==) $ do         w <- inverse s n-        let z  = dsaTruncHashDigest digest n+        let z = dsaTruncHashDigest digest n             u1 = z * w `mod` n             u2 = r * w `mod` n-            x  = pointAddTwoMuls curve u1 g u2 q+            x = pointAddTwoMuls curve u1 g u2 q         case x of-             PointO     -> Nothing-             Point x1 _ -> return $ x1 `mod` n-  where n = ecc_n cc-        g = ecc_g cc-        cc = common_curve $ public_curve pk+            PointO -> Nothing+            Point x1 _ -> return $ x1 `mod` n+  where+    n = ecc_n cc+    g = ecc_g cc+    cc = common_curve $ public_curve pk  -- | Verify a bytestring using the public key.-verify :: (ByteArrayAccess msg, HashAlgorithm hash) => hash -> PublicKey -> Signature -> msg -> Bool+verify+    :: (ByteArrayAccess msg, HashAlgorithm hash)+    => hash -> PublicKey -> Signature -> msg -> Bool verify hashAlg pk sig msg = verifyDigest pk sig (hashWith hashAlg msg)++-- | Recover the public key from an extended signature and a digest.+recoverDigest+    :: HashAlgorithm hash+    => Curve -> ExtendedSignature -> Digest hash -> Maybe PublicKey+recoverDigest curve (ExtendedSignature i p (Signature r s)) digest = do+    let CurveCommon _ _ g n _ = common_curve curve+    let z = dsaTruncHashDigest digest n+    w <- inverse r n+    c <- pointCompose curve i r p+    pure $ PublicKey curve $ pointAddTwoMuls curve (s * w) c (negate $ z * w) g++-- | Recover the public key from an extended signature and a message.+recover+    :: (ByteArrayAccess msg, HashAlgorithm hash)+    => hash -> Curve -> ExtendedSignature -> msg -> Maybe PublicKey+recover hashAlg curve sig msg = recoverDigest curve sig $ hashWith hashAlg msg++-- | Deterministic nonce generation according to RFC 6979.+-- Allows using different hash algorithms for the HMAC-based DRG and the message digest.+deterministicNonce+    :: (HashAlgorithm hashDRG, HashAlgorithm hashDigest)+    => hashDRG -> PrivateKey -> Digest hashDigest -> (Integer -> Maybe a) -> a+deterministicNonce alg (PrivateKey curve key) digest go = fst $ withDRG state run+  where+    state = update seed $ initial alg+      where+        seed = i2ospOf_ bytes key <> i2ospOf_ bytes message :: ScrubbedBytes+        message = dsaTruncHashDigest digest n `mod` n+    run = do+        k <- generatePrefix bits+        if 0 < k && k < n then maybe run pure $ go k else run+    bytes = (bits + 7) `unsafeShiftR` 3+    bits = numBits n+    n = ecc_n $ common_curve curve
Crypto/PubKey/ECC/Generate.hs view
@@ -1,30 +1,34 @@ -- | Signature generation. module Crypto.PubKey.ECC.Generate where -import Crypto.Random.Types-import Crypto.PubKey.ECC.Types-import Crypto.PubKey.ECC.ECDSA import Crypto.Number.Generate+import Crypto.PubKey.ECC.ECDSA import Crypto.PubKey.ECC.Prim+import Crypto.PubKey.ECC.Types+import Crypto.Random.Types  -- | Generate Q given d. -- -- /WARNING:/ Vulnerable to timing attacks.-generateQ :: Curve-          -> Integer-          -> Point+generateQ+    :: Curve+    -> Integer+    -> Point generateQ curve d = pointMul curve d g-  where g = ecc_g $ common_curve curve+  where+    g = ecc_g $ common_curve curve  -- | Generate a pair of (private, public) key. -- -- /WARNING:/ Vulnerable to timing attacks.-generate :: MonadRandom m-         => Curve -- ^ Elliptic Curve-         -> m (PublicKey, PrivateKey)+generate+    :: MonadRandom m+    => Curve+    -- ^ Elliptic Curve+    -> m (PublicKey, PrivateKey) generate curve = do     d <- generateBetween 1 (n - 1)     let q = generateQ curve d     return (PublicKey curve q, PrivateKey curve d)   where-        n = ecc_n $ common_curve curve+    n = ecc_n $ common_curve curve
Crypto/PubKey/ECC/P256.hs view
@@ -1,3 +1,7 @@+{-# LANGUAGE EmptyDataDecls #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# OPTIONS_GHC -fno-warn-unused-binds #-}+ -- | -- Module      : Crypto.PubKey.ECC.P256 -- License     : BSD-style@@ -6,67 +10,66 @@ -- Portability : unknown -- -- P256 support----{-# LANGUAGE GeneralizedNewtypeDeriving #-}-{-# LANGUAGE EmptyDataDecls #-}-{-# OPTIONS_GHC -fno-warn-unused-binds #-}-module Crypto.PubKey.ECC.P256-    ( Scalar-    , Point+module Crypto.PubKey.ECC.P256 (+    Scalar,+    Point,+     -- * Point arithmetic-    , pointBase-    , pointAdd-    , pointNegate-    , pointMul-    , pointDh-    , pointsMulVarTime-    , pointIsValid-    , pointIsAtInfinity-    , toPoint-    , pointX-    , pointToIntegers-    , pointFromIntegers-    , pointToBinary-    , pointFromBinary-    , unsafePointFromBinary+    pointBase,+    pointAdd,+    pointNegate,+    pointMul,+    pointDh,+    pointsMulVarTime,+    pointIsValid,+    pointIsAtInfinity,+    toPoint,+    pointX,+    pointToIntegers,+    pointFromIntegers,+    pointToBinary,+    pointFromBinary,+    unsafePointFromBinary,+     -- * Scalar arithmetic-    , scalarGenerate-    , scalarZero-    , scalarN-    , scalarIsZero-    , scalarAdd-    , scalarSub-    , scalarMul-    , scalarInv-    , scalarInvSafe-    , scalarCmp-    , scalarFromBinary-    , scalarToBinary-    , scalarFromInteger-    , scalarToInteger-    ) where+    scalarGenerate,+    scalarZero,+    scalarN,+    scalarIsZero,+    scalarReduce,+    scalarAdd,+    scalarSub,+    scalarMul,+    scalarInv,+    scalarInvSafe,+    scalarCmp,+    scalarFromBinary,+    scalarToBinary,+    scalarFromInteger,+    scalarToInteger,+) where -import           Data.Word-import           Foreign.Ptr-import           Foreign.C.Types+import Data.Word+import Foreign.C.Types+import Foreign.Ptr -import           Crypto.Internal.Compat-import           Crypto.Internal.Imports-import           Crypto.Internal.ByteArray+import Crypto.Error+import Crypto.Internal.ByteArray import qualified Crypto.Internal.ByteArray as B-import           Data.Memory.PtrMethods (memSet)-import           Crypto.Error-import           Crypto.Random-import           Crypto.Number.Serialize.Internal (os2ip, i2ospOf)-import qualified Crypto.Number.Serialize as S (os2ip, i2ospOf)+import Crypto.Internal.Compat+import Crypto.Internal.Imports+import qualified Crypto.Number.Serialize as S (i2ospOf, os2ip)+import Crypto.Number.Serialize.Internal (i2ospOf, os2ip)+import Crypto.Random+import Data.Memory.PtrMethods (memSet)  -- | A P256 scalar newtype Scalar = Scalar ScrubbedBytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData)  -- | A P256 point newtype Point = Point Bytes-    deriving (Show,Eq,NFData)+    deriving (Show, Eq, NFData)  scalarSize :: Int scalarSize = 32@@ -74,7 +77,7 @@ pointSize :: Int pointSize = 64 -type P256Digit  = Word32+type P256Digit = Word32  data P256Scalar data P256Y@@ -91,7 +94,7 @@ pointBase :: Point pointBase =     case scalarFromInteger 1 of-        CryptoPassed s  -> toPoint s+        CryptoPassed s -> toPoint s         CryptoFailed _ -> error "pointBase: assumption failed"  -- | Lift to curve a scalar@@ -99,19 +102,21 @@ -- Using the curve generator as base point compute: -- -- > scalar * G--- toPoint :: Scalar -> Point toPoint s     | scalarIsZero s = error "cannot create point from zero"-    | otherwise      =+    | otherwise =         withNewPoint $ \px py -> withScalar s $ \p ->             ccrypton_p256_basepoint_mul p px py  -- | Add a point to another point pointAdd :: Point -> Point -> Point-pointAdd a b = withNewPoint $ \dx dy ->-    withPoint a $ \ax ay -> withPoint b $ \bx by ->-        ccrypton_p256e_point_add ax ay bx by dx dy+pointAdd a b+    | pointIsAtInfinity a = b+    | pointIsAtInfinity b = a+    | otherwise = withNewPoint $ \dx dy ->+        withPoint a $ \ax ay -> withPoint b $ \bx by ->+            ccrypton_p256e_point_add ax ay bx by dx dy  -- | Negate a point pointNegate :: Point -> Point@@ -146,24 +151,24 @@     withScalar n1 $ \pn1 -> withScalar n2 $ \pn2 -> withPoint p $ \px py ->         ccrypton_p256_points_mul_vartime pn1 pn2 px py dx dy --- | Check if a 'Point' is valid+-- | Check if a t'Point' is valid pointIsValid :: Point -> Bool pointIsValid p = unsafeDoIO $ withPoint p $ \px py -> do     r <- ccrypton_p256_is_valid_point px py     return (r /= 0) --- | Check if a 'Point' is the point at infinity+-- | Check if a t'Point' is the point at infinity pointIsAtInfinity :: Point -> Bool pointIsAtInfinity (Point b) = constAllZero b --- | Return the x coordinate as a 'Scalar' if the point is not at infinity+-- | Return the x coordinate as a t'Scalar' if the point is not at infinity pointX :: Point -> Maybe Scalar pointX p     | pointIsAtInfinity p = Nothing-    | otherwise           = Just $-        withNewScalarFreeze $ \d    ->-        withPoint p         $ \px _ ->-            ccrypton_p256_mod ccrypton_SECP256r1_n (castPtr px) (castPtr d)+    | otherwise = Just $+        withNewScalarFreeze $ \d ->+            withPoint p $ \px _ ->+                ccrypton_p256_mod ccrypton_SECP256r1_n (castPtr px) (castPtr d)  -- | Convert a point to (x,y) Integers pointToIntegers :: Point -> (Integer, Integer)@@ -175,12 +180,14 @@         x <- os2ip temp scalarSize         ccrypton_p256_to_bin py temp         y <- os2ip temp scalarSize-        return (x,y)+        return (x, y)  -- | Convert from (x,y) Integers to a point pointFromIntegers :: (Integer, Integer) -> Point-pointFromIntegers (x,y) = withNewPoint $ \dx dy ->-    allocTemp scalarSize (\temp -> fill temp (castPtr dx) x >> fill temp (castPtr dy) y)+pointFromIntegers (x, y) = withNewPoint $ \dx dy ->+    allocTemp+        scalarSize+        (\temp -> fill temp (castPtr dx) x >> fill temp (castPtr dy) y)   where     -- put @n to @temp in big endian format, then from @temp to @dest in p256 scalar format     fill :: Ptr Word8 -> Ptr P256Scalar -> Integer -> IO ()@@ -207,15 +214,15 @@     validatePoint :: Point -> CryptoFailable Point     validatePoint p         | pointIsValid p = CryptoPassed p-        | otherwise      = CryptoFailed CryptoError_PointCoordinatesInvalid+        | otherwise = CryptoFailed CryptoError_PointCoordinatesInvalid  -- | Convert from binary to a point, possibly invalid unsafePointFromBinary :: ByteArrayAccess ba => ba -> CryptoFailable Point unsafePointFromBinary ba     | B.length ba /= pointSize = CryptoFailed CryptoError_PublicKeySizeInvalid-    | otherwise                =+    | otherwise =         CryptoPassed $ withNewPoint $ \px py -> B.withByteArray ba $ \src -> do-            ccrypton_p256_from_bin src                        (castPtr px)+            ccrypton_p256_from_bin src (castPtr px)             ccrypton_p256_from_bin (src `plusPtr` scalarSize) (castPtr py)  ------------------------------------------------------------------------@@ -245,6 +252,17 @@     result <- ccrypton_p256_is_zero d     return $ result /= 0 +-- | Bring a scalar below the order of the curve+--+-- 'scalarFromInteger' and 'scalarFromBinary' take any 256 bits, so a scalar+-- can arrive above the order; the arithmetic below wants it brought down+-- first.  Twice the order is more than 256 bits hold, so this is a single+-- subtraction, taken or not through a mask rather than a branch.+scalarReduce :: Scalar -> Scalar+scalarReduce a =+    withNewScalarFreeze $ \d -> withScalar a $ \pa ->+        ccrypton_p256_mod ccrypton_SECP256r1_n pa d+ -- | Perform addition between two scalars -- -- > a + b@@ -267,7 +285,7 @@ scalarMul :: Scalar -> Scalar -> Scalar scalarMul a b =     withNewScalarFreeze $ \d -> withScalar a $ \pa -> withScalar b $ \pb ->-         ccrypton_p256_modmul ccrypton_SECP256r1_n pa 0 pb d+        ccrypton_p256_modmul ccrypton_SECP256r1_n pa 0 pb d  -- | Give the inverse of the scalar --@@ -298,7 +316,7 @@ scalarFromBinary :: ByteArrayAccess ba => ba -> CryptoFailable Scalar scalarFromBinary ba     | B.length ba /= scalarSize = CryptoFailed CryptoError_SecretKeySizeInvalid-    | otherwise                 =+    | otherwise =         CryptoPassed $ withNewScalarFreeze $ \p -> B.withByteArray ba $ \b ->             ccrypton_p256_from_bin b p {-# NOINLINE scalarFromBinary #-}@@ -312,7 +330,10 @@ -- | Convert from an Integer to a P256 Scalar scalarFromInteger :: Integer -> CryptoFailable Scalar scalarFromInteger i =-    maybe (CryptoFailed CryptoError_SecretKeySizeInvalid) scalarFromBinary (S.i2ospOf 32 i :: Maybe Bytes)+    maybe+        (CryptoFailed CryptoError_SecretKeySizeInvalid)+        scalarFromBinary+        (S.i2ospOf 32 i :: Maybe Bytes)  -- | Convert from a P256 Scalar to an Integer scalarToInteger :: Scalar -> Integer@@ -370,53 +391,76 @@ foreign import ccall "crypton_p256_clear"     ccrypton_p256_clear :: Ptr P256Scalar -> IO () foreign import ccall "crypton_p256e_modadd"-    ccrypton_p256e_modadd :: Ptr P256Scalar -> Ptr P256Scalar -> Ptr P256Scalar -> Ptr P256Scalar -> IO ()+    ccrypton_p256e_modadd+        :: Ptr P256Scalar -> Ptr P256Scalar -> Ptr P256Scalar -> Ptr P256Scalar -> IO () foreign import ccall "crypton_p256_add_d"     ccrypton_p256_add_d :: Ptr P256Scalar -> P256Digit -> Ptr P256Scalar -> IO CInt foreign import ccall "crypton_p256e_modsub"-    ccrypton_p256e_modsub :: Ptr P256Scalar -> Ptr P256Scalar -> Ptr P256Scalar -> Ptr P256Scalar -> IO ()+    ccrypton_p256e_modsub+        :: Ptr P256Scalar -> Ptr P256Scalar -> Ptr P256Scalar -> Ptr P256Scalar -> IO () foreign import ccall "crypton_p256_cmp"     ccrypton_p256_cmp :: Ptr P256Scalar -> Ptr P256Scalar -> IO CInt foreign import ccall "crypton_p256_mod"     ccrypton_p256_mod :: Ptr P256Scalar -> Ptr P256Scalar -> Ptr P256Scalar -> IO () foreign import ccall "crypton_p256_modmul"-    ccrypton_p256_modmul :: Ptr P256Scalar -> Ptr P256Scalar -> P256Digit -> Ptr P256Scalar -> Ptr P256Scalar -> IO ()+    ccrypton_p256_modmul+        :: Ptr P256Scalar+        -> Ptr P256Scalar+        -> P256Digit+        -> Ptr P256Scalar+        -> Ptr P256Scalar+        -> IO () foreign import ccall "crypton_p256e_scalar_invert"     ccrypton_p256e_scalar_invert :: Ptr P256Scalar -> Ptr P256Scalar -> IO ()---foreign import ccall "crypton_p256_modinv"---    ccrypton_p256_modinv :: Ptr P256Scalar -> Ptr P256Scalar -> Ptr P256Scalar -> IO ()+ foreign import ccall "crypton_p256_modinv_vartime"-    ccrypton_p256_modinv_vartime :: Ptr P256Scalar -> Ptr P256Scalar -> Ptr P256Scalar -> IO ()+    ccrypton_p256_modinv_vartime+        :: Ptr P256Scalar -> Ptr P256Scalar -> Ptr P256Scalar -> IO () foreign import ccall "crypton_p256_base_point_mul"-    ccrypton_p256_basepoint_mul :: Ptr P256Scalar-                                   -> Ptr P256X -> Ptr P256Y-                                   -> IO ()+    ccrypton_p256_basepoint_mul+        :: Ptr P256Scalar+        -> Ptr P256X+        -> Ptr P256Y+        -> IO ()  foreign import ccall "crypton_p256e_point_add"-    ccrypton_p256e_point_add :: Ptr P256X -> Ptr P256Y-                                -> Ptr P256X -> Ptr P256Y-                                -> Ptr P256X -> Ptr P256Y-                                -> IO ()+    ccrypton_p256e_point_add+        :: Ptr P256X+        -> Ptr P256Y+        -> Ptr P256X+        -> Ptr P256Y+        -> Ptr P256X+        -> Ptr P256Y+        -> IO ()  foreign import ccall "crypton_p256e_point_negate"-    ccrypton_p256e_point_negate :: Ptr P256X -> Ptr P256Y-                                   -> Ptr P256X -> Ptr P256Y-                                   -> IO ()+    ccrypton_p256e_point_negate+        :: Ptr P256X+        -> Ptr P256Y+        -> Ptr P256X+        -> Ptr P256Y+        -> IO ()  -- compute (out_x,out_y) = n * (in_x,in_y) foreign import ccall "crypton_p256e_point_mul"-    ccrypton_p256e_point_mul :: Ptr P256Scalar -- n-                                -> Ptr P256X -> Ptr P256Y -- in_{x,y}-                                -> Ptr P256X -> Ptr P256Y -- out_{x,y}-                                -> IO ()+    ccrypton_p256e_point_mul+        :: Ptr P256Scalar -- n+        -> Ptr P256X+        -> Ptr P256Y -- in_{x,y}+        -> Ptr P256X+        -> Ptr P256Y -- out_{x,y}+        -> IO ()  -- compute (out_x,out,y) = n1 * G + n2 * (in_x,in_y) foreign import ccall "crypton_p256_points_mul_vartime"-    ccrypton_p256_points_mul_vartime :: Ptr P256Scalar -- n1-                                        -> Ptr P256Scalar -- n2-                                        -> Ptr P256X -> Ptr P256Y -- in_{x,y}-                                        -> Ptr P256X -> Ptr P256Y -- out_{x,y}-                                        -> IO ()+    ccrypton_p256_points_mul_vartime+        :: Ptr P256Scalar -- n1+        -> Ptr P256Scalar -- n2+        -> Ptr P256X+        -> Ptr P256Y -- in_{x,y}+        -> Ptr P256X+        -> Ptr P256Y -- out_{x,y}+        -> IO () foreign import ccall "crypton_p256_is_valid_point"     ccrypton_p256_is_valid_point :: Ptr P256X -> Ptr P256Y -> IO CInt
Crypto/PubKey/ECC/Prim.hs view
@@ -1,39 +1,164 @@+{-# LANGUAGE BangPatterns #-}+ -- | Elliptic Curve Arithmetic. ----- /WARNING:/ These functions are vulnerable to timing attacks.-module Crypto.PubKey.ECC.Prim-    ( scalarGenerate-    , pointAdd-    , pointNegate-    , pointDouble-    , pointBaseMul-    , pointMul-    , pointAddTwoMuls-    , isPointAtInfinity-    , isPointValid-    ) where+-- /WARNING:/ These functions are vulnerable to timing attacks, except on+-- P-256, whose multiplications go to the C implementation in+-- "Crypto.PubKey.ECC.P256".+module Crypto.PubKey.ECC.Prim (+    scalarGenerate,+    scalarInverse,+    scalarAdd,+    scalarMul,+    pointAdd,+    pointNegate,+    pointDouble,+    pointBaseMul,+    pointMul,+    pointAddTwoMuls,+    pointDecompose,+    pointCompose,+    isPointAtInfinity,+    isPointValid,+    isPointInSubgroup,+) where -import Data.Maybe-import Crypto.Number.ModArithmetic+import Crypto.Error (maybeCryptoError)+import Crypto.Internal.ECC (CurveField (..), MulResult (..), curveMul)+import Crypto.Number.Basic (numBits) import Crypto.Number.F2m import Crypto.Number.Generate (generateBetween)+import Crypto.Number.ModArithmetic+import qualified Crypto.PubKey.ECC.P256 as P256 import Crypto.PubKey.ECC.Types import Crypto.Random+import Data.Bits (shiftL, shiftR, testBit, (.&.))+import Data.Maybe +-- | P-256, the one curve here that has a C implementation: 'SEC_p256r1', also+-- known as NIST P-256 and prime256v1.+--+-- A 'Curve' carries its parameters rather than a name, so this compares the+-- parameters.  They are public, so the comparison tells an attacker nothing.+p256Curve :: Curve+p256Curve = getCurveByName SEC_p256r1+{-# NOINLINE p256Curve #-}++p256Order :: Integer+p256Order = ecc_n (common_curve p256Curve)++p256Base :: Point+p256Base = ecc_g (common_curve p256Curve)++-- | A point the C implementation will take: in range, on the curve, and not+-- the point at infinity, which it does not represent.  Anything else is left+-- to the generic code, which answers for points off the curve too.+toP256 :: Point -> Maybe P256.Point+toP256 PointO = Nothing+toP256 (Point x y)+    | x < 0 || y < 0 || x >= limit || y >= limit = Nothing+    | P256.pointIsValid p = Just p+    | otherwise = Nothing+  where+    limit = 1 `shiftL` 256+    p = P256.pointFromIntegers (x, y)++fromP256 :: P256.Point -> Point+fromP256 p+    | P256.pointIsAtInfinity p = PointO+    | otherwise = uncurry Point (P256.pointToIntegers p)++-- | Any 256-bit number as a scalar.+--+-- The arithmetic below takes them as they come: a 256-bit value is barely+-- over the order, and both the multiplication and the addition bring their+-- answer back under it.  'Nothing' is for what does not fit in 256 bits,+-- which no scalar anybody signs with does.+p256Scalar :: Integer -> Maybe P256.Scalar+p256Scalar n+    | n < 0 || n >= 1 `shiftL` 256 = Nothing+    | otherwise = maybeCryptoError (P256.scalarFromInteger n)++-- | The scalar reduced into the range the C implementation takes.+--+-- Every point it accepts has the curve's order, so reducing changes no+-- answer; 'Nothing' means the multiple is the point at infinity, which is the+-- generic code's business.+-- The reduction is a single masked subtraction, so a secret scalar does not+-- steer it, which taking the remainder would: dividing takes a number of+-- steps that follows the number being divided.  Anything wider than 256 bits+-- has to go through a division first, but a scalar that wide is not one+-- anybody signs with.+toP256Scalar :: Integer -> Maybe P256.Scalar+toP256Scalar n = case P256.scalarReduce <$> p256Scalar n of+    Nothing -> toP256Scalar (n `mod` p256Order) -- wider than 256 bits, or below zero+    Just s+        | P256.scalarIsZero s -> Nothing+        | otherwise -> Just s++-- | @n1 * p1 + n2 * p2@ through the C implementation, when one of the points+-- is the base point.  That is the shape signature verification uses.+p256AddTwoMuls :: Integer -> Point -> Integer -> Point -> Maybe Point+p256AddTwoMuls n1 p1 n2 p2+    | p1 == p256Base = withBase n1 n2 p2+    | p2 == p256Base = withBase n2 n1 p1+    | otherwise = Nothing+  where+    withBase a b q =+        fromP256+            <$> (P256.pointsMulVarTime <$> toP256Scalar a <*> toP256Scalar b <*> toP256 q)+ -- | Generate a valid scalar for a specific Curve scalarGenerate :: MonadRandom randomly => Curve -> randomly PrivateNumber scalarGenerate curve = generateBetween 1 (n - 1)   where-        n = ecc_n $ common_curve curve+    n = ecc_n $ common_curve curve ---TODO: Extract helper function for `fromMaybe PointO...`+-- | The inverse of a scalar modulo the order of the curve, without letting+-- the scalar steer how long the work takes.  This is what signing needs for+-- its nonce, which is as worth hiding as the private key itself: a handful of+-- signatures whose nonces are partly known give the key away.+--+-- On P-256 the C implementation does it; elsewhere it is 'inverseSafe'.+-- 'Nothing' means the scalar has no inverse, which for the curves in use here+-- means it was a multiple of the order.+scalarInverse :: Curve -> Integer -> Maybe Integer+scalarInverse c k+    | c == p256Curve+    , Just s <- toP256Scalar k =+        Just (P256.scalarToInteger (P256.scalarInvSafe s))+    | otherwise = inverseSafe k (ecc_n $ common_curve c) +-- | Addition modulo the order of the curve.+--+-- On P-256 this is the C implementation's arithmetic, which works in a fixed+-- width and so does not let the values steer it; elsewhere it is 'Integer'+-- arithmetic, whose cost follows the values.+scalarAdd :: Curve -> Integer -> Integer -> Integer+scalarAdd c a b+    | c == p256Curve+    , Just x <- p256Scalar a+    , Just y <- p256Scalar b =+        P256.scalarToInteger (P256.scalarAdd x y)+    | otherwise = (a + b) `mod` ecc_n (common_curve c)++-- | Multiplication modulo the order of the curve, as 'scalarAdd'.+scalarMul :: Curve -> Integer -> Integer -> Integer+scalarMul c a b+    | c == p256Curve+    , Just x <- p256Scalar a+    , Just y <- p256Scalar b =+        P256.scalarToInteger (P256.scalarMul x y)+    | otherwise = (a * b) `mod` ecc_n (common_curve c)++-- TODO: Extract helper function for `fromMaybe PointO...`+ -- | Elliptic Curve point negation: -- @pointNegate c p@ returns point @q@ such that @pointAdd c p q == PointO@. pointNegate :: Curve -> Point -> Point-pointNegate _            PointO     = PointO+pointNegate _ PointO = PointO pointNegate (CurveFP c) (Point x y) = Point x (ecc_p c - y)-pointNegate CurveF2m{}  (Point x y) = Point x (x `addF2m` y)+pointNegate CurveF2m{} (Point x y) = Point x (x `addF2m` y)  -- | Elliptic Curve point addition. --@@ -43,21 +168,22 @@ pointAdd _ PointO q = q pointAdd _ p PointO = p pointAdd c p q-  | p == q = pointDouble c p-  | p == pointNegate c q = PointO-pointAdd (CurveFP (CurvePrime pr _)) (Point xp yp) (Point xq yq)-    = fromMaybe PointO $ do+    | p == q = pointDouble c p+    | p == pointNegate c q = PointO+pointAdd (CurveFP (CurvePrime pr _)) (Point xp yp) (Point xq yq) =+    fromMaybe PointO $ do         s <- divmod (yp - yq) (xp - xq) pr-        let xr = (s ^ (2::Int) - xp - xq) `mod` pr+        let xr = (s ^ (2 :: Int) - xp - xq) `mod` pr             yr = (s * (xp - xr) - yp) `mod` pr         return $ Point xr yr-pointAdd (CurveF2m (CurveBinary fx cc)) (Point xp yp) (Point xq yq)-    = fromMaybe PointO $ do+pointAdd (CurveF2m (CurveBinary fx cc)) (Point xp yp) (Point xq yq) =+    fromMaybe PointO $ do         s <- divF2m fx (yp `addF2m` yq) (xp `addF2m` xq)         let xr = mulF2m fx s s `addF2m` s `addF2m` xp `addF2m` xq `addF2m` a             yr = mulF2m fx s (xp `addF2m` xr) `addF2m` xr `addF2m` yp         return $ Point xr yr-  where a = ecc_a cc+  where+    a = ecc_a cc  -- | Elliptic Curve point doubling. --@@ -74,73 +200,316 @@ -- >    s = xp + (yp / xp) -- >    xr = s ^ 2 + s + a -- >    yr = xp ^ 2 + (s+1) * xr--- pointDouble :: Curve -> Point -> Point pointDouble _ PointO = PointO pointDouble (CurveFP (CurvePrime pr cc)) (Point xp yp) = fromMaybe PointO $ do-    lambda <- divmod (3 * xp ^ (2::Int) + a) (2 * yp) pr-    let xr = (lambda ^ (2::Int) - 2 * xp) `mod` pr+    lambda <- divmod (3 * xp ^ (2 :: Int) + a) (2 * yp) pr+    let xr = (lambda ^ (2 :: Int) - 2 * xp) `mod` pr         yr = (lambda * (xp - xr) - yp) `mod` pr     return $ Point xr yr-  where a = ecc_a cc+  where+    a = ecc_a cc pointDouble (CurveF2m (CurveBinary fx cc)) (Point xp yp)-    | xp == 0   = PointO+    | xp == 0 = PointO     | otherwise = fromMaybe PointO $ do         s <- return . addF2m xp =<< divF2m fx yp xp         let xr = mulF2m fx s s `addF2m` s `addF2m` a             yr = mulF2m fx xp xp `addF2m` mulF2m fx xr (s `addF2m` 1)         return $ Point xr yr-  where a = ecc_a cc+  where+    a = ecc_a cc  -- | Elliptic curve point multiplication using the base ----- /WARNING:/ Vulnerable to timing attacks.+-- On P-256 this reaches the C implementation, which multiplies the base point+-- through a table of its own.+--+-- /WARNING:/ On every other curve, vulnerable to timing attacks. pointBaseMul :: Curve -> Integer -> Point pointBaseMul c n = pointMul c n (ecc_g $ common_curve c) --- | Elliptic curve point multiplication (double and add algorithm).+-- | Elliptic curve point multiplication. ----- /WARNING:/ Vulnerable to timing attacks.+-- Over a prime field this goes to C, four bits of scalar at a time, with the+-- multiple to add taken from a table read by touching every entry of it.+-- Over a binary field it also goes to C, as Montgomery's ladder: it carries+-- the x coordinates of two consecutive multiples -- their difference being+-- the point is what lets it carry no more than that -- and spends one+-- addition and one doubling on every bit whichever way the bit goes, with the+-- two exchanged by a mask rather than chosen by a branch.  Either way the work+-- follows the width of the curve's order and not the scalar.+--+-- What falls back on the 'Integer' arithmetic below is a point that is not on+-- the curve, the one point of a binary curve that has no x, and a prime the C+-- will not take.+--+-- Multiplying the base point of a curve over a prime field -- which is what+-- signing and making a key do, and nothing else does -- goes through a table+-- of its multiples, built when that curve is first asked for one and kept+-- afterwards.  The build is a few milliseconds and the table a few hundred+-- kilobytes, and a multiplication that uses it takes about a third of what+-- one without it takes.+--+-- On P-256 the multiplication goes to the C implementation in+-- "Crypto.PubKey.ECC.P256", which has a table for the base point.+--+-- /WARNING:/ What is left of the 'Integer' arithmetic below -- a point off+-- the curve, the one point of a binary curve with no x, a prime or a+-- polynomial the C will not take -- has uniform operation counts at best, and+-- uniform operation counts are not constant time: those operations cost what+-- the values they are given cost. pointMul :: Curve -> Integer -> Point -> Point pointMul _ _ PointO = PointO pointMul c n p-    | n <  0 = pointMul c (-n) (pointNegate c p)+    -- the base point has a table of its own in the C, which is what makes key+    -- generation and signing quicker than multiplying any other point+    | c == p256Curve+    , p == p256Base =+        maybe PointO (fromP256 . P256.toPoint) (toP256Scalar n)+    | c == p256Curve+    , Just q <- toP256 p =+        maybe PointO (\s -> fromP256 (P256.pointMul s q)) (toP256Scalar n)+    | n < 0 = pointMul c (-n) (pointNegate c p)     | n == 0 = PointO-    | n == 1 = p-    | odd n = pointAdd c p (pointMul c (n - 1) p)-    | otherwise = pointMul c (n `div` 2) (pointDouble c p)+    | otherwise =+        case c of+            CurveFP (CurvePrime pr cc) -> primeMul pr cc+            CurveF2m (CurveBinary fx cc) -> binaryMul fx cc+  where+    -- The C answers for a point on the curve; anything else keeps the+    -- answers it has always had from the code below.  Multiplying the base+    -- point, which is what signing and making a key do, goes through the+    -- table kept for it.+    primeMul pr cc = case p of+        Point px py+            | isPointValid c p ->+                answer slow $+                    curveMul+                        (Prime pr (ecc_a cc) (ecc_b cc))+                        (ecc_n cc)+                        n+                        px+                        py+                        (p == ecc_g cc)+        _ -> slow+      where+        slow =+            jacobianMul+                pr+                (ecc_a cc)+                (max (integerBits n) (integerBits (ecc_n cc)))+                n+                p --- | Elliptic curve double-scalar multiplication (uses Shamir's trick).+    -- The ladder answers for a point on the curve that has an x; the one+    -- point with no x, and anything off the curve, keep what they had.+    binaryMul fx cc = case p of+        Point px py+            | isPointValid c p ->+                answer (affineMul n p) $+                    curveMul (Binary fx (ecc_b cc)) (ecc_n cc) n px py False+        _ -> affineMul n p++    -- what the C could not take goes back to the code that was here before+    answer fallback r = case r of+        MulPoint x y -> Point x y+        MulInfinity -> PointO+        MulUnsupported -> fallback++    affineMul k q+        | k == 0 = PointO+        | k == 1 = q+        | odd k = pointAdd c q (affineMul (k - 1) q)+        | otherwise = affineMul (k `div` 2) (pointDouble c q)++-- | Number of bits needed to write n, for n > 0.+integerBits :: Integer -> Int+integerBits = go 0+  where+    go acc 0 = acc+    go acc k = go (acc + 1) (k `div` 2)++-- | A point in Jacobian coordinates: @(X, Y, Z)@ stands for the affine+-- @(X\/Z^2, Y\/Z^3)@, and @JPointO@ for the point at infinity.+data JPoint = JPointO | JPoint !Integer !Integer !Integer++-- | The field a prime curve works in, and how to reduce into it. --+-- Most curve primes are @2^k - c@ with @c@ far smaller than the prime.+-- Reducing is then a shift, a multiplication by @c@ and an addition, where+-- dividing a number twice the width costs about four times as much: 227ns+-- against 183 for a P-384 multiplication, and 226 against 89 for P-521, whose+-- @c@ is one.+-- | The prime, the width to fold at, and what to fold back in.  A @c@ of zero+-- says to divide instead, either because the prime has no such shape or+-- because it is too small for folding to pay: @c@ has to be under half the+-- width, or folding would not shrink the number, and below 256 bits the+-- handful of 'Integer' operations folding takes costs more than the division+-- it saves -- measured on P-192, where folding is 14% slower.+data Field = Field !Integer !Int !Integer++mkField :: Integer -> Field+mkField p+    | p > 0 && c > 0 && 2 * numBits c <= k && k >= 256 = Field p k c+    | otherwise = Field p 0 0+  where+    k = numBits p+    c = (1 `shiftL` k) - p++fieldPrime :: Field -> Integer+fieldPrime (Field p _ _) = p++fieldReduce :: Field -> Integer -> Integer+fieldReduce (Field p k c) x+    | c == 0 || x < 0 = x `mod` p+    | otherwise = trim (fold x)+  where+    mask = (1 `shiftL` k) - 1+    fold v+        | v > mask = fold ((v `shiftR` k) * c + (v .&. mask))+        | otherwise = v+    trim v+        | v >= p = trim (v - p)+        | otherwise = v+{-# INLINE fieldReduce #-}++-- | A point in affine coordinates: the second operand of every addition a+-- scalar multiplication makes, where knowing that z is one saves four+-- multiplications of the sixteen.+data Affine = AffineO | Affine !Integer !Integer++jacobianMul :: Integer -> Integer -> Int -> Integer -> Point -> Point+jacobianMul _ _ _ _ PointO = PointO+jacobianMul pr a bits n (Point px py) = fromJacobian f (go (bits - 1) JPointO)+  where+    f = mkField pr+    base = Affine px py++    -- The bangs are what make the addition happen at every bit.  Without+    -- them the one that is not taken stays a thunk and is never worked out,+    -- so the multiplication costs a step for every bit that is set rather+    -- than for every bit there is, and a single measurement tells an attacker+    -- how many bits of the scalar are set.+    go i acc+        | i < 0 = acc+        | otherwise =+            let !d = jDouble f a acc+                !s = jAddAffine f a d base+             in go (i - 1) (if testBit n i then s else d)++jDouble :: Field -> Integer -> JPoint -> JPoint+jDouble _ _ JPointO = JPointO+jDouble f a (JPoint x y z)+    | y == 0 = JPointO+    | otherwise = JPoint x3 y3 z3+  where+    red = fieldReduce f+    yy = red (y * y)+    delta = red (4 * x * yy)+    zz = red (z * z)+    m = red (3 * x * x + a * zz * zz)+    x3 = red (m * m - 2 * delta)+    y3 = red (m * (delta - x3) - 8 * yy * yy)+    z3 = red (2 * y * z)++-- | Add a point whose z is one, which is what a scalar multiplication always+-- adds: u1 is x1, s1 is y1, and z3 is one multiplication rather than two.+jAddAffine :: Field -> Integer -> JPoint -> Affine -> JPoint+jAddAffine _ _ p AffineO = p+jAddAffine _ _ JPointO (Affine x2 y2) = JPoint x2 y2 1+jAddAffine f a p@(JPoint x1 y1 z1) (Affine x2 y2)+    | h /= 0 = JPoint x3 y3 z3+    | r /= 0 = JPointO+    | otherwise = jDouble f a p+  where+    red = fieldReduce f+    z1s = red (z1 * z1)+    u2 = red (x2 * z1s)+    s2 = red (y2 * z1s * z1)+    h = red (u2 - x1)+    r = red (s2 - y1)+    h2 = red (h * h)+    h3 = red (h2 * h)+    x3 = red (r * r - h3 - 2 * x1 * h2)+    y3 = red (r * (x1 * h2 - x3) - y1 * h3)+    z3 = red (h * z1)++fromJacobian :: Field -> JPoint -> Point+fromJacobian _ JPointO = PointO+fromJacobian f (JPoint x y z) =+    case inverse z (fieldPrime f) of+        Nothing -> PointO+        Just zi ->+            let red = fieldReduce f+                zi2 = red (zi * zi)+             in Point (red (x * zi2)) (red (y * zi2 * zi))++-- | Elliptic curve double-scalar multiplication.+-- -- > pointAddTwoMuls c n1 p1 n2 p2 == pointAdd c (pointMul c n1 p1) -- >                                             (pointMul c n2 p2) --+-- which, apart from P-256, is how it is done: the two multiplications+-- separately, and then one addition.  P-256 has a double multiplication of+-- its own in C and takes it.+--+-- This used to be Shamir's trick, one pass over the bits of both scalars at+-- once, which shares the doublings between them and is the right thing to do+-- when the two multiplications would cost the same.  They no longer do.+-- 'pointMul' goes to C, and over a prime field it multiplies the base point+-- through a table of its multiples, which is a third of the price of an+-- ordinary multiplication -- and the base point is one of the two here, since+-- signature verification is what asks for this.  Sharing the doublings with a+-- pass in 'Integer' arithmetic gives that up and more: on P-384 it costs+-- twice what two multiplications in C cost, and on the curves over a binary+-- field, whose addition needs an inversion where the C has a ladder that+-- needs none, it costs two hundred times as much.+--+-- Both scalars are public wherever this is called from, so nothing here is+-- meant to hide them.+-- -- /WARNING:/ Vulnerable to timing attacks. pointAddTwoMuls :: Curve -> Integer -> Point -> Integer -> Point -> Point-pointAddTwoMuls _ _  PointO _  PointO = PointO-pointAddTwoMuls c _  PointO n2 p2     = pointMul c n2 p2-pointAddTwoMuls c n1 p1     _  PointO = pointMul c n1 p1-pointAddTwoMuls c n1 p1     n2 p2-    | n1 < 0    = pointAddTwoMuls c (-n1) (pointNegate c p1) n2 p2-    | n2 < 0    = pointAddTwoMuls c n1 p1 (-n2) (pointNegate c p2)-    | otherwise = go (n1, n2)+pointAddTwoMuls c n1 p1 n2 p2+    | c == p256Curve, Just r <- p256AddTwoMuls n1 p1 n2 p2 = r+    | otherwise = pointAdd c (pointMul c n1 p1) (pointMul c n2 p2) -  where-    p0 = pointAdd c p1 p2+-- | Decompose a point into index, residue, and parity.+--+-- Adapted from SEC 1: Elliptic Curve Cryptography, Version 2.0, section 2.3.3.+pointDecompose :: Curve -> Point -> Maybe (Integer, Integer, Bool)+pointDecompose _ PointO = Nothing+pointDecompose curve (Point x y) = do+    let CurveCommon _ _ _ n _ = common_curve curve+    let (index, residue) = x `divMod` n+    parity <- case curve of+        CurveFP _ -> pure $ odd y+        CurveF2m _ | x == 0 -> pure False+        CurveF2m (CurveBinary fx _) -> odd <$> divF2m fx y x+    pure (index, residue, parity) -    go (0,  0 ) = PointO-    go (k1, k2) =-        let q = pointDouble c $ go (k1 `div` 2, k2 `div` 2)-        in case (odd k1, odd k2) of-            (True  , True  ) -> pointAdd c p0 q-            (True  , False ) -> pointAdd c p1 q-            (False , True  ) -> pointAdd c p2 q-            (False , False ) -> q+-- | Compose a point from index, residue, and parity.+--+-- Adapted from SEC 1: Elliptic Curve Cryptography, Version 2.0, section 2.3.4.+pointCompose :: Curve -> Integer -> Integer -> Bool -> Maybe Point+pointCompose curve index residue parity = do+    let CurveCommon a b _ n _ = common_curve curve+    let x = residue + index * n+    y <- case curve of+        CurveFP (CurvePrime p _) -> do+            z <- squareRoot p $ x ^ (3 :: Int) + a * x + b+            pure $ if odd z == parity then z else p - z+        CurveF2m (CurveBinary fx _) | x == 0 -> pure $ sqrtF2m fx b+        CurveF2m (CurveBinary fx _) -> do+            c <- divF2m fx b $ squareF2m fx x+            z <- quadraticF2m fx $ addF2m x $ addF2m a c+            pure $ mulF2m fx x $ if odd z == parity then z else addF2m 1 z+    pure $ Point x y  -- | Check if a point is the point at infinity. isPointAtInfinity :: Point -> Bool isPointAtInfinity PointO = True-isPointAtInfinity _      = False+isPointAtInfinity _ = False  -- | check if a point is on specific curve --@@ -149,24 +518,78 @@ -- * x is not out of range -- * y is not out of range -- * the equation @y^2 = x^3 + a*x + b (mod p)@ holds+--+-- over a prime curve, and the corresponding checks over a binary curve: the+-- coordinates reduce to themselves in the field, and+-- @y^2 + x*y = x^3 + a*x^2 + b@ holds.+--+-- This is the check to make on a point that arrives from elsewhere, before+-- multiplying it by a private number.  Without it the multiplication is+-- carried out in whatever group the supplied point generates rather than the+-- curve group, and if that group is small the private number can be recovered+-- from the result.+--+-- Two things it does not establish:+--+-- * The point at infinity is reported as valid, since it is a member of the+--   curve group.  It is not a usable peer value: multiplying it by anything+--   yields the point at infinity again, which has no coordinates.  Reject it+--   separately where a peer is not allowed to send it.+--+-- * Being on the curve is not membership of the subgroup generated by the base+--   point.  The two coincide only when the cofactor is 1.  Of the curves in+--   'Crypto.PubKey.ECC.Types.CurveName' that holds for every prime curve+--   except @SEC_p112r2@ and @SEC_p128r2@, whose cofactor is 4, and for no+--   binary curve, whose cofactor is 2 or 4.  Where the cofactor is above 1 a+--   point on the curve may still generate a small subgroup, and ruling that+--   out needs a further check -- multiplying by the group order and requiring+--   the point at infinity, or clearing the cofactor -- that this function does+--   not make. isPointValid :: Curve -> Point -> Bool-isPointValid _                           PointO      = True+isPointValid _ PointO = True isPointValid (CurveFP (CurvePrime p cc)) (Point x y) =     isValid x && isValid y && (y ^ (2 :: Int)) `eqModP` (x ^ (3 :: Int) + a * x + b)-  where a  = ecc_a cc-        b  = ecc_b cc-        eqModP z1 z2 = (z1 `mod` p) == (z2 `mod` p)-        isValid e = e >= 0 && e < p+  where+    a = ecc_a cc+    b = ecc_b cc+    eqModP z1 z2 = (z1 `mod` p) == (z2 `mod` p)+    isValid e = e >= 0 && e < p isPointValid (CurveF2m (CurveBinary fx cc)) (Point x y) =-    and [ isValid x+    and+        [ isValid x         , isValid y         , ((((x `add` a) `mul` x `add` y) `mul` x) `add` b `add` (squareF2m fx y)) == 0         ]-  where a  = ecc_a cc-        b  = ecc_b cc-        add = addF2m-        mul = mulF2m fx-        isValid e = modF2m fx e == e+  where+    a = ecc_a cc+    b = ecc_b cc+    add = addF2m+    mul = mulF2m fx+    isValid e = modF2m fx e == e++-- | Check that a point is in the subgroup the base point generates, which is+-- the further check 'isPointValid' does not make.  A point that is on the+-- curve but outside that subgroup answers a multiplication modulo an order+-- smaller than the group's, so the multiplier -- a private number, where the+-- point came from a peer -- is revealed modulo that small order.+--+-- Where the cofactor is 1 the subgroup is the whole curve group and the+-- answer is 'True' for any point on the curve, at no cost.  Otherwise the+-- point is multiplied by the group order and the answer is whether that+-- reaches the point at infinity, which costs one scalar multiplication.  This+-- is the check OpenSSL's @EC_KEY_check_key@ makes.+--+-- The point at infinity is reported as in the subgroup, as 'isPointValid'+-- reports it valid; it is a member, and unusable for other reasons.+--+-- A point that is not on the curve at all has no meaningful answer here, so+-- check 'isPointValid' first.+isPointInSubgroup :: Curve -> Point -> Bool+isPointInSubgroup curve p+    | ecc_h cc == 1 = True+    | otherwise = pointMul curve (ecc_n cc) p == PointO+  where+    cc = common_curve curve  -- | div and mod divmod :: Integer -> Integer -> Integer -> Maybe Integer
Crypto/PubKey/ECC/Types.hs view
@@ -1,4 +1,5 @@ {-# LANGUAGE DeriveDataTypeable #-}+ -- | -- Module      : Crypto.PubKey.ECC.Types -- License     : BSD-style@@ -8,32 +9,35 @@ -- -- References: --   <https://tools.ietf.org/html/rfc5915>----module Crypto.PubKey.ECC.Types-    ( Curve(..)-    , Point(..)-    , PublicPoint-    , PrivateNumber-    , CurveBinary(..)-    , CurvePrime(..)-    , common_curve-    , curveSizeBits-    , ecc_fx-    , ecc_p-    , CurveCommon(..)+module Crypto.PubKey.ECC.Types (+    Curve (..),+    Point (..),+    PublicPoint,+    PrivateNumber,+    CurveBinary (..),+    CurvePrime (..),+    common_curve,+    curveSizeBits,+    ecc_fx,+    ecc_p,+    CurveCommon (..),+     -- * Recommended curves definition-    , CurveName(..)-    , getCurveByName-    ) where+    CurveName (..),+    getCurveByName,+) where -import           Data.Data-import           Crypto.Internal.Imports-import           Crypto.Number.Basic (numBits)+import Crypto.Internal.Imports+import Crypto.Number.Basic (numBits)+import Data.Data  -- | Define either a binary curve or a prime curve.-data Curve = CurveF2m CurveBinary -- ^ 𝔽(2^m)-           | CurveFP  CurvePrime  -- ^ 𝔽p-           deriving (Show,Read,Eq,Data)+data Curve+    = -- | 𝔽(2^m)+      CurveF2m CurveBinary+    | -- | 𝔽p+      CurveFP CurvePrime+    deriving (Show, Read, Eq, Data)  -- | ECC Public Point type PublicPoint = Point@@ -42,9 +46,11 @@ type PrivateNumber = Integer  -- | Define a point on a curve.-data Point = Point Integer Integer-           | PointO -- ^ Point at Infinity-           deriving (Show,Read,Eq,Data)+data Point+    = Point Integer Integer+    | -- | Point at Infinity+      PointO+    deriving (Show, Read, Eq, Data)  instance NFData Point where     rnf (Point x y) = x `seq` y `seq` ()@@ -53,7 +59,7 @@ -- | Define an elliptic curve in 𝔽(2^m). -- The firt parameter is the Integer representatioin of the irreducible polynomial f(x). data CurveBinary = CurveBinary Integer CurveCommon-    deriving (Show,Read,Eq,Data)+    deriving (Show, Read, Eq, Data)  instance NFData CurveBinary where     rnf (CurveBinary i cc) = i `seq` cc `seq` ()@@ -61,12 +67,12 @@ -- | Define an elliptic curve in 𝔽p. -- The first parameter is the Prime Number. data CurvePrime = CurvePrime Integer CurveCommon-    deriving (Show,Read,Eq,Data)+    deriving (Show, Read, Eq, Data)  -- | Parameters in common between binary and prime curves. common_curve :: Curve -> CurveCommon common_curve (CurveF2m (CurveBinary _ cc)) = cc-common_curve (CurveFP  (CurvePrime  _ cc)) = cc+common_curve (CurveFP (CurvePrime _ cc)) = cc  -- | Irreducible polynomial representing the characteristic of a CurveBinary. ecc_fx :: CurveBinary -> Integer@@ -79,16 +85,22 @@ -- | Define common parameters in a curve definition -- of the form: y^2 = x^3 + ax + b. data CurveCommon = CurveCommon-    { ecc_a :: Integer -- ^ curve parameter a-    , ecc_b :: Integer -- ^ curve parameter b-    , ecc_g :: Point   -- ^ base point-    , ecc_n :: Integer -- ^ order of G-    , ecc_h :: Integer -- ^ cofactor-    } deriving (Show,Read,Eq,Data)+    { ecc_a :: Integer+    -- ^ curve parameter a+    , ecc_b :: Integer+    -- ^ curve parameter b+    , ecc_g :: Point+    -- ^ base point+    , ecc_n :: Integer+    -- ^ order of G+    , ecc_h :: Integer+    -- ^ cofactor+    }+    deriving (Show, Read, Eq, Data)  -- | Define names for known recommended curves.-data CurveName =-      SEC_p112r1+data CurveName+    = SEC_p112r1     | SEC_p112r2     | SEC_p128r1     | SEC_p128r2@@ -121,8 +133,21 @@     | SEC_t409r1     | SEC_t571k1     | SEC_t571r1-    deriving (Show,Read,Eq,Ord,Enum,Bounded,Data)+    deriving (Show, Read, Eq, Ord, Enum, Bounded, Data) +{-# DEPRECATED+    SEC_t113r1, SEC_t113r2, SEC_t131r1, SEC_t131r2, SEC_t163k1, SEC_t163r1,+    SEC_t163r2, SEC_t193r1, SEC_t193r2, SEC_t233k1, SEC_t233r1, SEC_t239k1,+    SEC_t283k1, SEC_t283r1, SEC_t409k1, SEC_t409r1, SEC_t571k1, SEC_t571r1+    [ "This curve is over a binary field, and those are obsolete."+    , "They are also the curves whose cofactor is not 1, so a point from"+    , "a peer needs the subgroup check that costs a further scalar"+    , "multiplication; pyca/cryptography deprecated them for removal in"+    , "the release that fixed CVE-2026-26007.  This one will go in a"+    , "later major version of crypton.  Prefer a prime curve, or X25519."+    ]+    #-}+ {- curvesOIDs :: [ (CurveName, [Integer]) ] curvesOIDs =@@ -164,338 +189,527 @@  -- | get the size of the curve in bits curveSizeBits :: Curve -> Int-curveSizeBits (CurveFP  c) = numBits (ecc_p  c)+curveSizeBits (CurveFP c) = numBits (ecc_p c) curveSizeBits (CurveF2m c) = numBits (ecc_fx c) - 1  -- | Get the curve definition associated with a recommended known curve name. getCurveByName :: CurveName -> Curve-getCurveByName SEC_p112r1 = CurveFP  $ CurvePrime-    0xdb7c2abf62e35e668076bead208b-    (CurveCommon-        { ecc_a = 0xdb7c2abf62e35e668076bead2088-        , ecc_b = 0x659ef8ba043916eede8911702b22-        , ecc_g = Point 0x09487239995a5ee76b55f9c2f098+getCurveByName SEC_p112r1 =+    CurveFP $+        CurvePrime+            0xdb7c2abf62e35e668076bead208b+            ( CurveCommon+                { ecc_a = 0xdb7c2abf62e35e668076bead2088+                , ecc_b = 0x659ef8ba043916eede8911702b22+                , ecc_g =+                    Point+                        0x09487239995a5ee76b55f9c2f098                         0xa89ce5af8724c0a23e0e0ff77500-        , ecc_n = 0xdb7c2abf62e35e7628dfac6561c5-        , ecc_h = 1-        })-getCurveByName SEC_p112r2 = CurveFP  $ CurvePrime-    0xdb7c2abf62e35e668076bead208b-    (CurveCommon-        { ecc_a = 0x6127c24c05f38a0aaaf65c0ef02c-        , ecc_b = 0x51def1815db5ed74fcc34c85d709-        , ecc_g = Point 0x4ba30ab5e892b4e1649dd0928643+                , ecc_n = 0xdb7c2abf62e35e7628dfac6561c5+                , ecc_h = 1+                }+            )+getCurveByName SEC_p112r2 =+    CurveFP $+        CurvePrime+            0xdb7c2abf62e35e668076bead208b+            ( CurveCommon+                { ecc_a = 0x6127c24c05f38a0aaaf65c0ef02c+                , ecc_b = 0x51def1815db5ed74fcc34c85d709+                , ecc_g =+                    Point+                        0x4ba30ab5e892b4e1649dd0928643                         0xadcd46f5882e3747def36e956e97-        , ecc_n = 0x36df0aafd8b8d7597ca10520d04b-        , ecc_h = 4-        })-getCurveByName SEC_p128r1 = CurveFP  $ CurvePrime-    0xfffffffdffffffffffffffffffffffff-    (CurveCommon-        { ecc_a = 0xfffffffdfffffffffffffffffffffffc-        , ecc_b = 0xe87579c11079f43dd824993c2cee5ed3-        , ecc_g = Point 0x161ff7528b899b2d0c28607ca52c5b86+                , ecc_n = 0x36df0aafd8b8d7597ca10520d04b+                , ecc_h = 4+                }+            )+getCurveByName SEC_p128r1 =+    CurveFP $+        CurvePrime+            0xfffffffdffffffffffffffffffffffff+            ( CurveCommon+                { ecc_a = 0xfffffffdfffffffffffffffffffffffc+                , ecc_b = 0xe87579c11079f43dd824993c2cee5ed3+                , ecc_g =+                    Point+                        0x161ff7528b899b2d0c28607ca52c5b86                         0xcf5ac8395bafeb13c02da292dded7a83-        , ecc_n = 0xfffffffe0000000075a30d1b9038a115-        , ecc_h = 1-        })-getCurveByName SEC_p128r2 = CurveFP  $ CurvePrime-    0xfffffffdffffffffffffffffffffffff-    (CurveCommon-        { ecc_a = 0xd6031998d1b3bbfebf59cc9bbff9aee1-        , ecc_b = 0x5eeefca380d02919dc2c6558bb6d8a5d-        , ecc_g = Point 0x7b6aa5d85e572983e6fb32a7cdebc140+                , ecc_n = 0xfffffffe0000000075a30d1b9038a115+                , ecc_h = 1+                }+            )+getCurveByName SEC_p128r2 =+    CurveFP $+        CurvePrime+            0xfffffffdffffffffffffffffffffffff+            ( CurveCommon+                { ecc_a = 0xd6031998d1b3bbfebf59cc9bbff9aee1+                , ecc_b = 0x5eeefca380d02919dc2c6558bb6d8a5d+                , ecc_g =+                    Point+                        0x7b6aa5d85e572983e6fb32a7cdebc140                         0x27b6916a894d3aee7106fe805fc34b44-        , ecc_n = 0x3fffffff7fffffffbe0024720613b5a3-        , ecc_h = 4-        })-getCurveByName SEC_p160k1 = CurveFP  $ CurvePrime-    0x00fffffffffffffffffffffffffffffffeffffac73-    (CurveCommon-        { ecc_a = 0x000000000000000000000000000000000000000000-        , ecc_b = 0x000000000000000000000000000000000000000007-        , ecc_g = Point 0x003b4c382ce37aa192a4019e763036f4f5dd4d7ebb+                , ecc_n = 0x3fffffff7fffffffbe0024720613b5a3+                , ecc_h = 4+                }+            )+getCurveByName SEC_p160k1 =+    CurveFP $+        CurvePrime+            0x00fffffffffffffffffffffffffffffffeffffac73+            ( CurveCommon+                { ecc_a = 0x000000000000000000000000000000000000000000+                , ecc_b = 0x000000000000000000000000000000000000000007+                , ecc_g =+                    Point+                        0x003b4c382ce37aa192a4019e763036f4f5dd4d7ebb                         0x00938cf935318fdced6bc28286531733c3f03c4fee-        , ecc_n = 0x0100000000000000000001b8fa16dfab9aca16b6b3-        , ecc_h = 1-        })-getCurveByName SEC_p160r1 = CurveFP  $ CurvePrime-    0x00ffffffffffffffffffffffffffffffff7fffffff-    (CurveCommon-        { ecc_a = 0x00ffffffffffffffffffffffffffffffff7ffffffc-        , ecc_b = 0x001c97befc54bd7a8b65acf89f81d4d4adc565fa45-        , ecc_g = Point 0x004a96b5688ef573284664698968c38bb913cbfc82+                , ecc_n = 0x0100000000000000000001b8fa16dfab9aca16b6b3+                , ecc_h = 1+                }+            )+getCurveByName SEC_p160r1 =+    CurveFP $+        CurvePrime+            0x00ffffffffffffffffffffffffffffffff7fffffff+            ( CurveCommon+                { ecc_a = 0x00ffffffffffffffffffffffffffffffff7ffffffc+                , ecc_b = 0x001c97befc54bd7a8b65acf89f81d4d4adc565fa45+                , ecc_g =+                    Point+                        0x004a96b5688ef573284664698968c38bb913cbfc82                         0x0023a628553168947d59dcc912042351377ac5fb32-        , ecc_n = 0x0100000000000000000001f4c8f927aed3ca752257-        , ecc_h = 1-        })-getCurveByName SEC_p160r2 = CurveFP  $ CurvePrime-    0x00fffffffffffffffffffffffffffffffeffffac73-    (CurveCommon-        { ecc_a = 0x00fffffffffffffffffffffffffffffffeffffac70-        , ecc_b = 0x00b4e134d3fb59eb8bab57274904664d5af50388ba-        , ecc_g = Point 0x0052dcb034293a117e1f4ff11b30f7199d3144ce6d+                , ecc_n = 0x0100000000000000000001f4c8f927aed3ca752257+                , ecc_h = 1+                }+            )+getCurveByName SEC_p160r2 =+    CurveFP $+        CurvePrime+            0x00fffffffffffffffffffffffffffffffeffffac73+            ( CurveCommon+                { ecc_a = 0x00fffffffffffffffffffffffffffffffeffffac70+                , ecc_b = 0x00b4e134d3fb59eb8bab57274904664d5af50388ba+                , ecc_g =+                    Point+                        0x0052dcb034293a117e1f4ff11b30f7199d3144ce6d                         0x00feaffef2e331f296e071fa0df9982cfea7d43f2e-        , ecc_n = 0x0100000000000000000000351ee786a818f3a1a16b-        , ecc_h = 1-        })-getCurveByName SEC_p192k1 = CurveFP  $ CurvePrime-    0xfffffffffffffffffffffffffffffffffffffffeffffee37-    (CurveCommon-        { ecc_a = 0x000000000000000000000000000000000000000000000000-        , ecc_b = 0x000000000000000000000000000000000000000000000003-        , ecc_g = Point 0xdb4ff10ec057e9ae26b07d0280b7f4341da5d1b1eae06c7d+                , ecc_n = 0x0100000000000000000000351ee786a818f3a1a16b+                , ecc_h = 1+                }+            )+getCurveByName SEC_p192k1 =+    CurveFP $+        CurvePrime+            0xfffffffffffffffffffffffffffffffffffffffeffffee37+            ( CurveCommon+                { ecc_a = 0x000000000000000000000000000000000000000000000000+                , ecc_b = 0x000000000000000000000000000000000000000000000003+                , ecc_g =+                    Point+                        0xdb4ff10ec057e9ae26b07d0280b7f4341da5d1b1eae06c7d                         0x9b2f2f6d9c5628a7844163d015be86344082aa88d95e2f9d-        , ecc_n = 0xfffffffffffffffffffffffe26f2fc170f69466a74defd8d-        , ecc_h = 1-        })-getCurveByName SEC_p192r1 = CurveFP  $ CurvePrime-    0xfffffffffffffffffffffffffffffffeffffffffffffffff-    (CurveCommon-        { ecc_a = 0xfffffffffffffffffffffffffffffffefffffffffffffffc-        , ecc_b = 0x64210519e59c80e70fa7e9ab72243049feb8deecc146b9b1-        , ecc_g = Point 0x188da80eb03090f67cbf20eb43a18800f4ff0afd82ff1012+                , ecc_n = 0xfffffffffffffffffffffffe26f2fc170f69466a74defd8d+                , ecc_h = 1+                }+            )+getCurveByName SEC_p192r1 =+    CurveFP $+        CurvePrime+            0xfffffffffffffffffffffffffffffffeffffffffffffffff+            ( CurveCommon+                { ecc_a = 0xfffffffffffffffffffffffffffffffefffffffffffffffc+                , ecc_b = 0x64210519e59c80e70fa7e9ab72243049feb8deecc146b9b1+                , ecc_g =+                    Point+                        0x188da80eb03090f67cbf20eb43a18800f4ff0afd82ff1012                         0x07192b95ffc8da78631011ed6b24cdd573f977a11e794811-        , ecc_n = 0xffffffffffffffffffffffff99def836146bc9b1b4d22831-        , ecc_h = 1-        })-getCurveByName SEC_p224k1 = CurveFP  $ CurvePrime-    0x00fffffffffffffffffffffffffffffffffffffffffffffffeffffe56d-    (CurveCommon-        { ecc_a = 0x0000000000000000000000000000000000000000000000000000000000-        , ecc_b = 0x0000000000000000000000000000000000000000000000000000000005-        , ecc_g = Point 0x00a1455b334df099df30fc28a169a467e9e47075a90f7e650eb6b7a45c+                , ecc_n = 0xffffffffffffffffffffffff99def836146bc9b1b4d22831+                , ecc_h = 1+                }+            )+getCurveByName SEC_p224k1 =+    CurveFP $+        CurvePrime+            0x00fffffffffffffffffffffffffffffffffffffffffffffffeffffe56d+            ( CurveCommon+                { ecc_a = 0x0000000000000000000000000000000000000000000000000000000000+                , ecc_b = 0x0000000000000000000000000000000000000000000000000000000005+                , ecc_g =+                    Point+                        0x00a1455b334df099df30fc28a169a467e9e47075a90f7e650eb6b7a45c                         0x007e089fed7fba344282cafbd6f7e319f7c0b0bd59e2ca4bdb556d61a5-        , ecc_n = 0x010000000000000000000000000001dce8d2ec6184caf0a971769fb1f7-        , ecc_h = 1-        })-getCurveByName SEC_p224r1 = CurveFP  $ CurvePrime-    0xffffffffffffffffffffffffffffffff000000000000000000000001-    (CurveCommon-        { ecc_a = 0xfffffffffffffffffffffffffffffffefffffffffffffffffffffffe-        , ecc_b = 0xb4050a850c04b3abf54132565044b0b7d7bfd8ba270b39432355ffb4-        , ecc_g = Point 0xb70e0cbd6bb4bf7f321390b94a03c1d356c21122343280d6115c1d21+                , ecc_n = 0x010000000000000000000000000001dce8d2ec6184caf0a971769fb1f7+                , ecc_h = 1+                }+            )+getCurveByName SEC_p224r1 =+    CurveFP $+        CurvePrime+            0xffffffffffffffffffffffffffffffff000000000000000000000001+            ( CurveCommon+                { ecc_a = 0xfffffffffffffffffffffffffffffffefffffffffffffffffffffffe+                , ecc_b = 0xb4050a850c04b3abf54132565044b0b7d7bfd8ba270b39432355ffb4+                , ecc_g =+                    Point+                        0xb70e0cbd6bb4bf7f321390b94a03c1d356c21122343280d6115c1d21                         0xbd376388b5f723fb4c22dfe6cd4375a05a07476444d5819985007e34-        , ecc_n = 0xffffffffffffffffffffffffffff16a2e0b8f03e13dd29455c5c2a3d-        , ecc_h = 1-        })-getCurveByName SEC_p256k1 = CurveFP  $ CurvePrime-    0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f-    (CurveCommon-        { ecc_a = 0x0000000000000000000000000000000000000000000000000000000000000000-        , ecc_b = 0x0000000000000000000000000000000000000000000000000000000000000007-        , ecc_g = Point 0x79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798+                , ecc_n = 0xffffffffffffffffffffffffffff16a2e0b8f03e13dd29455c5c2a3d+                , ecc_h = 1+                }+            )+getCurveByName SEC_p256k1 =+    CurveFP $+        CurvePrime+            0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f+            ( CurveCommon+                { ecc_a = 0x0000000000000000000000000000000000000000000000000000000000000000+                , ecc_b = 0x0000000000000000000000000000000000000000000000000000000000000007+                , ecc_g =+                    Point+                        0x79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798                         0x483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8-        , ecc_n = 0xfffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141-        , ecc_h = 1-        })-getCurveByName SEC_p256r1 = CurveFP  $ CurvePrime-    0xffffffff00000001000000000000000000000000ffffffffffffffffffffffff-    (CurveCommon-        { ecc_a = 0xffffffff00000001000000000000000000000000fffffffffffffffffffffffc-        , ecc_b = 0x5ac635d8aa3a93e7b3ebbd55769886bc651d06b0cc53b0f63bce3c3e27d2604b-        , ecc_g = Point 0x6b17d1f2e12c4247f8bce6e563a440f277037d812deb33a0f4a13945d898c296+                , ecc_n = 0xfffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141+                , ecc_h = 1+                }+            )+getCurveByName SEC_p256r1 =+    CurveFP $+        CurvePrime+            0xffffffff00000001000000000000000000000000ffffffffffffffffffffffff+            ( CurveCommon+                { ecc_a = 0xffffffff00000001000000000000000000000000fffffffffffffffffffffffc+                , ecc_b = 0x5ac635d8aa3a93e7b3ebbd55769886bc651d06b0cc53b0f63bce3c3e27d2604b+                , ecc_g =+                    Point+                        0x6b17d1f2e12c4247f8bce6e563a440f277037d812deb33a0f4a13945d898c296                         0x4fe342e2fe1a7f9b8ee7eb4a7c0f9e162bce33576b315ececbb6406837bf51f5-        , ecc_n = 0xffffffff00000000ffffffffffffffffbce6faada7179e84f3b9cac2fc632551-        , ecc_h = 1-        })-getCurveByName SEC_p384r1 = CurveFP  $ CurvePrime-    0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffeffffffff0000000000000000ffffffff-    (CurveCommon-        { ecc_a = 0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffeffffffff0000000000000000fffffffc-        , ecc_b = 0xb3312fa7e23ee7e4988e056be3f82d19181d9c6efe8141120314088f5013875ac656398d8a2ed19d2a85c8edd3ec2aef-        , ecc_g = Point 0xaa87ca22be8b05378eb1c71ef320ad746e1d3b628ba79b9859f741e082542a385502f25dbf55296c3a545e3872760ab7+                , ecc_n = 0xffffffff00000000ffffffffffffffffbce6faada7179e84f3b9cac2fc632551+                , ecc_h = 1+                }+            )+getCurveByName SEC_p384r1 =+    CurveFP $+        CurvePrime+            0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffeffffffff0000000000000000ffffffff+            ( CurveCommon+                { ecc_a =+                    0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffeffffffff0000000000000000fffffffc+                , ecc_b =+                    0xb3312fa7e23ee7e4988e056be3f82d19181d9c6efe8141120314088f5013875ac656398d8a2ed19d2a85c8edd3ec2aef+                , ecc_g =+                    Point+                        0xaa87ca22be8b05378eb1c71ef320ad746e1d3b628ba79b9859f741e082542a385502f25dbf55296c3a545e3872760ab7                         0x3617de4a96262c6f5d9e98bf9292dc29f8f41dbd289a147ce9da3113b5f0b8c00a60b1ce1d7e819d7a431d7c90ea0e5f-        , ecc_n = 0xffffffffffffffffffffffffffffffffffffffffffffffffc7634d81f4372ddf581a0db248b0a77aecec196accc52973-        , ecc_h = 1-        })-getCurveByName SEC_p521r1 = CurveFP  $ CurvePrime-    0x01ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff-    (CurveCommon-        { ecc_a = 0x01fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc-        , ecc_b = 0x0051953eb9618e1c9a1f929a21a0b68540eea2da725b99b315f3b8b489918ef109e156193951ec7e937b1652c0bd3bb1bf073573df883d2c34f1ef451fd46b503f00-        , ecc_g = Point 0x00c6858e06b70404e9cd9e3ecb662395b4429c648139053fb521f828af606b4d3dbaa14b5e77efe75928fe1dc127a2ffa8de3348b3c1856a429bf97e7e31c2e5bd66+                , ecc_n =+                    0xffffffffffffffffffffffffffffffffffffffffffffffffc7634d81f4372ddf581a0db248b0a77aecec196accc52973+                , ecc_h = 1+                }+            )+getCurveByName SEC_p521r1 =+    CurveFP $+        CurvePrime+            0x01ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff+            ( CurveCommon+                { ecc_a =+                    0x01fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc+                , ecc_b =+                    0x0051953eb9618e1c9a1f929a21a0b68540eea2da725b99b315f3b8b489918ef109e156193951ec7e937b1652c0bd3bb1bf073573df883d2c34f1ef451fd46b503f00+                , ecc_g =+                    Point+                        0x00c6858e06b70404e9cd9e3ecb662395b4429c648139053fb521f828af606b4d3dbaa14b5e77efe75928fe1dc127a2ffa8de3348b3c1856a429bf97e7e31c2e5bd66                         0x011839296a789a3bc0045c8a5fb42c7d1bd998f54449579b446817afbd17273e662c97ee72995ef42640c550b9013fad0761353c7086a272c24088be94769fd16650-        , ecc_n = 0x01fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffa51868783bf2f966b7fcc0148f709a5d03bb5c9b8899c47aebb6fb71e91386409-        , ecc_h = 1-        })-getCurveByName SEC_t113r1 = CurveF2m $ CurveBinary-    0x020000000000000000000000000201-    (CurveCommon-        { ecc_a = 0x003088250ca6e7c7fe649ce85820f7-        , ecc_b = 0x00e8bee4d3e2260744188be0e9c723-        , ecc_g = Point 0x009d73616f35f4ab1407d73562c10f+                , ecc_n =+                    0x01fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffa51868783bf2f966b7fcc0148f709a5d03bb5c9b8899c47aebb6fb71e91386409+                , ecc_h = 1+                }+            )+getCurveByName SEC_t113r1 =+    CurveF2m $+        CurveBinary+            0x020000000000000000000000000201+            ( CurveCommon+                { ecc_a = 0x003088250ca6e7c7fe649ce85820f7+                , ecc_b = 0x00e8bee4d3e2260744188be0e9c723+                , ecc_g =+                    Point+                        0x009d73616f35f4ab1407d73562c10f                         0x00a52830277958ee84d1315ed31886-        , ecc_n = 0x0100000000000000d9ccec8a39e56f-        , ecc_h = 2-        })-getCurveByName SEC_t113r2 = CurveF2m $ CurveBinary-    0x020000000000000000000000000201-    (CurveCommon-        { ecc_a = 0x00689918dbec7e5a0dd6dfc0aa55c7-        , ecc_b = 0x0095e9a9ec9b297bd4bf36e059184f-        , ecc_g = Point 0x01a57a6a7b26ca5ef52fcdb8164797+                , ecc_n = 0x0100000000000000d9ccec8a39e56f+                , ecc_h = 2+                }+            )+getCurveByName SEC_t113r2 =+    CurveF2m $+        CurveBinary+            0x020000000000000000000000000201+            ( CurveCommon+                { ecc_a = 0x00689918dbec7e5a0dd6dfc0aa55c7+                , ecc_b = 0x0095e9a9ec9b297bd4bf36e059184f+                , ecc_g =+                    Point+                        0x01a57a6a7b26ca5ef52fcdb8164797                         0x00b3adc94ed1fe674c06e695baba1d-        , ecc_n = 0x010000000000000108789b2496af93-        , ecc_h = 2-        })-getCurveByName SEC_t131r1 = CurveF2m $ CurveBinary-    0x080000000000000000000000000000010d-    (CurveCommon-        { ecc_a = 0x07a11b09a76b562144418ff3ff8c2570b8-        , ecc_b = 0x0217c05610884b63b9c6c7291678f9d341-        , ecc_g = Point 0x0081baf91fdf9833c40f9c181343638399+                , ecc_n = 0x010000000000000108789b2496af93+                , ecc_h = 2+                }+            )+getCurveByName SEC_t131r1 =+    CurveF2m $+        CurveBinary+            0x080000000000000000000000000000010d+            ( CurveCommon+                { ecc_a = 0x07a11b09a76b562144418ff3ff8c2570b8+                , ecc_b = 0x0217c05610884b63b9c6c7291678f9d341+                , ecc_g =+                    Point+                        0x0081baf91fdf9833c40f9c181343638399                         0x078c6e7ea38c001f73c8134b1b4ef9e150-        , ecc_n = 0x0400000000000000023123953a9464b54d-        , ecc_h = 2-        })-getCurveByName SEC_t131r2 = CurveF2m $ CurveBinary-    0x080000000000000000000000000000010d-    (CurveCommon-        { ecc_a = 0x03e5a88919d7cafcbf415f07c2176573b2-        , ecc_b = 0x04b8266a46c55657ac734ce38f018f2192-        , ecc_g = Point 0x0356dcd8f2f95031ad652d23951bb366a8+                , ecc_n = 0x0400000000000000023123953a9464b54d+                , ecc_h = 2+                }+            )+getCurveByName SEC_t131r2 =+    CurveF2m $+        CurveBinary+            0x080000000000000000000000000000010d+            ( CurveCommon+                { ecc_a = 0x03e5a88919d7cafcbf415f07c2176573b2+                , ecc_b = 0x04b8266a46c55657ac734ce38f018f2192+                , ecc_g =+                    Point+                        0x0356dcd8f2f95031ad652d23951bb366a8                         0x0648f06d867940a5366d9e265de9eb240f-        , ecc_n = 0x0400000000000000016954a233049ba98f-        , ecc_h = 2-        })-getCurveByName SEC_t163k1 = CurveF2m $ CurveBinary-    0x0800000000000000000000000000000000000000c9-    (CurveCommon-        { ecc_a = 0x000000000000000000000000000000000000000001-        , ecc_b = 0x000000000000000000000000000000000000000001-        , ecc_g = Point 0x02fe13c0537bbc11acaa07d793de4e6d5e5c94eee8+                , ecc_n = 0x0400000000000000016954a233049ba98f+                , ecc_h = 2+                }+            )+getCurveByName SEC_t163k1 =+    CurveF2m $+        CurveBinary+            0x0800000000000000000000000000000000000000c9+            ( CurveCommon+                { ecc_a = 0x000000000000000000000000000000000000000001+                , ecc_b = 0x000000000000000000000000000000000000000001+                , ecc_g =+                    Point+                        0x02fe13c0537bbc11acaa07d793de4e6d5e5c94eee8                         0x0289070fb05d38ff58321f2e800536d538ccdaa3d9-        , ecc_n = 0x04000000000000000000020108a2e0cc0d99f8a5ef-        , ecc_h = 2-        })-getCurveByName SEC_t163r1 = CurveF2m $ CurveBinary-    0x0800000000000000000000000000000000000000c9-    (CurveCommon-        { ecc_a = 0x07b6882caaefa84f9554ff8428bd88e246d2782ae2-        , ecc_b = 0x0713612dcddcb40aab946bda29ca91f73af958afd9-        , ecc_g = Point 0x0369979697ab43897789566789567f787a7876a654+                , ecc_n = 0x04000000000000000000020108a2e0cc0d99f8a5ef+                , ecc_h = 2+                }+            )+getCurveByName SEC_t163r1 =+    CurveF2m $+        CurveBinary+            0x0800000000000000000000000000000000000000c9+            ( CurveCommon+                { ecc_a = 0x07b6882caaefa84f9554ff8428bd88e246d2782ae2+                , ecc_b = 0x0713612dcddcb40aab946bda29ca91f73af958afd9+                , ecc_g =+                    Point+                        0x0369979697ab43897789566789567f787a7876a654                         0x00435edb42efafb2989d51fefce3c80988f41ff883-        , ecc_n = 0x03ffffffffffffffffffff48aab689c29ca710279b-        , ecc_h = 2-        })-getCurveByName SEC_t163r2 = CurveF2m $ CurveBinary-    0x0800000000000000000000000000000000000000c9-    (CurveCommon-        { ecc_a = 0x000000000000000000000000000000000000000001-        , ecc_b = 0x020a601907b8c953ca1481eb10512f78744a3205fd-        , ecc_g = Point 0x03f0eba16286a2d57ea0991168d4994637e8343e36+                , ecc_n = 0x03ffffffffffffffffffff48aab689c29ca710279b+                , ecc_h = 2+                }+            )+getCurveByName SEC_t163r2 =+    CurveF2m $+        CurveBinary+            0x0800000000000000000000000000000000000000c9+            ( CurveCommon+                { ecc_a = 0x000000000000000000000000000000000000000001+                , ecc_b = 0x020a601907b8c953ca1481eb10512f78744a3205fd+                , ecc_g =+                    Point+                        0x03f0eba16286a2d57ea0991168d4994637e8343e36                         0x00d51fbc6c71a0094fa2cdd545b11c5c0c797324f1-        , ecc_n = 0x040000000000000000000292fe77e70c12a4234c33-        , ecc_h = 2-        })-getCurveByName SEC_t193r1 = CurveF2m $ CurveBinary-    0x02000000000000000000000000000000000000000000008001-    (CurveCommon-        { ecc_a = 0x0017858feb7a98975169e171f77b4087de098ac8a911df7b01-        , ecc_b = 0x00fdfb49bfe6c3a89facadaa7a1e5bbc7cc1c2e5d831478814-        , ecc_g = Point 0x01f481bc5f0ff84a74ad6cdf6fdef4bf6179625372d8c0c5e1+                , ecc_n = 0x040000000000000000000292fe77e70c12a4234c33+                , ecc_h = 2+                }+            )+getCurveByName SEC_t193r1 =+    CurveF2m $+        CurveBinary+            0x02000000000000000000000000000000000000000000008001+            ( CurveCommon+                { ecc_a = 0x0017858feb7a98975169e171f77b4087de098ac8a911df7b01+                , ecc_b = 0x00fdfb49bfe6c3a89facadaa7a1e5bbc7cc1c2e5d831478814+                , ecc_g =+                    Point+                        0x01f481bc5f0ff84a74ad6cdf6fdef4bf6179625372d8c0c5e1                         0x0025e399f2903712ccf3ea9e3a1ad17fb0b3201b6af7ce1b05-        , ecc_n = 0x01000000000000000000000000c7f34a778f443acc920eba49-        , ecc_h = 2-        })-getCurveByName SEC_t193r2 = CurveF2m $ CurveBinary-    0x02000000000000000000000000000000000000000000008001-    (CurveCommon-        { ecc_a = 0x0163f35a5137c2ce3ea6ed8667190b0bc43ecd69977702709b-        , ecc_b = 0x00c9bb9e8927d4d64c377e2ab2856a5b16e3efb7f61d4316ae-        , ecc_g = Point 0x00d9b67d192e0367c803f39e1a7e82ca14a651350aae617e8f+                , ecc_n = 0x01000000000000000000000000c7f34a778f443acc920eba49+                , ecc_h = 2+                }+            )+getCurveByName SEC_t193r2 =+    CurveF2m $+        CurveBinary+            0x02000000000000000000000000000000000000000000008001+            ( CurveCommon+                { ecc_a = 0x0163f35a5137c2ce3ea6ed8667190b0bc43ecd69977702709b+                , ecc_b = 0x00c9bb9e8927d4d64c377e2ab2856a5b16e3efb7f61d4316ae+                , ecc_g =+                    Point+                        0x00d9b67d192e0367c803f39e1a7e82ca14a651350aae617e8f                         0x01ce94335607c304ac29e7defbd9ca01f596f927224cdecf6c-        , ecc_n = 0x010000000000000000000000015aab561b005413ccd4ee99d5-        , ecc_h = 2-        })-getCurveByName SEC_t233k1 = CurveF2m $ CurveBinary-    0x020000000000000000000000000000000000000004000000000000000001-    (CurveCommon-        { ecc_a = 0x000000000000000000000000000000000000000000000000000000000000-        , ecc_b = 0x000000000000000000000000000000000000000000000000000000000001-        , ecc_g = Point 0x017232ba853a7e731af129f22ff4149563a419c26bf50a4c9d6eefad6126+                , ecc_n = 0x010000000000000000000000015aab561b005413ccd4ee99d5+                , ecc_h = 2+                }+            )+getCurveByName SEC_t233k1 =+    CurveF2m $+        CurveBinary+            0x020000000000000000000000000000000000000004000000000000000001+            ( CurveCommon+                { ecc_a = 0x000000000000000000000000000000000000000000000000000000000000+                , ecc_b = 0x000000000000000000000000000000000000000000000000000000000001+                , ecc_g =+                    Point+                        0x017232ba853a7e731af129f22ff4149563a419c26bf50a4c9d6eefad6126                         0x01db537dece819b7f70f555a67c427a8cd9bf18aeb9b56e0c11056fae6a3-        , ecc_n = 0x008000000000000000000000000000069d5bb915bcd46efb1ad5f173abdf-        , ecc_h = 4-        })-getCurveByName SEC_t233r1 = CurveF2m $ CurveBinary-    0x020000000000000000000000000000000000000004000000000000000001-    (CurveCommon-        { ecc_a = 0x000000000000000000000000000000000000000000000000000000000001-        , ecc_b = 0x0066647ede6c332c7f8c0923bb58213b333b20e9ce4281fe115f7d8f90ad-        , ecc_g = Point 0x00fac9dfcbac8313bb2139f1bb755fef65bc391f8b36f8f8eb7371fd558b+                , ecc_n = 0x008000000000000000000000000000069d5bb915bcd46efb1ad5f173abdf+                , ecc_h = 4+                }+            )+getCurveByName SEC_t233r1 =+    CurveF2m $+        CurveBinary+            0x020000000000000000000000000000000000000004000000000000000001+            ( CurveCommon+                { ecc_a = 0x000000000000000000000000000000000000000000000000000000000001+                , ecc_b = 0x0066647ede6c332c7f8c0923bb58213b333b20e9ce4281fe115f7d8f90ad+                , ecc_g =+                    Point+                        0x00fac9dfcbac8313bb2139f1bb755fef65bc391f8b36f8f8eb7371fd558b                         0x01006a08a41903350678e58528bebf8a0beff867a7ca36716f7e01f81052-        , ecc_n = 0x01000000000000000000000000000013e974e72f8a6922031d2603cfe0d7-        , ecc_h = 2-        })-getCurveByName SEC_t239k1 = CurveF2m $ CurveBinary-    0x800000000000000000004000000000000000000000000000000000000001-    (CurveCommon-        { ecc_a = 0x000000000000000000000000000000000000000000000000000000000000-        , ecc_b = 0x000000000000000000000000000000000000000000000000000000000001-        , ecc_g = Point 0x29a0b6a887a983e9730988a68727a8b2d126c44cc2cc7b2a6555193035dc+                , ecc_n = 0x01000000000000000000000000000013e974e72f8a6922031d2603cfe0d7+                , ecc_h = 2+                }+            )+getCurveByName SEC_t239k1 =+    CurveF2m $+        CurveBinary+            0x800000000000000000004000000000000000000000000000000000000001+            ( CurveCommon+                { ecc_a = 0x000000000000000000000000000000000000000000000000000000000000+                , ecc_b = 0x000000000000000000000000000000000000000000000000000000000001+                , ecc_g =+                    Point+                        0x29a0b6a887a983e9730988a68727a8b2d126c44cc2cc7b2a6555193035dc                         0x76310804f12e549bdb011c103089e73510acb275fc312a5dc6b76553f0ca-        , ecc_n = 0x2000000000000000000000000000005a79fec67cb6e91f1c1da800e478a5-        , ecc_h = 4-        })-getCurveByName SEC_t283k1 = CurveF2m $ CurveBinary-    0x0800000000000000000000000000000000000000000000000000000000000000000010a1-    (CurveCommon-        { ecc_a = 0x000000000000000000000000000000000000000000000000000000000000000000000000-        , ecc_b = 0x000000000000000000000000000000000000000000000000000000000000000000000001-        , ecc_g = Point 0x0503213f78ca44883f1a3b8162f188e553cd265f23c1567a16876913b0c2ac2458492836+                , ecc_n = 0x2000000000000000000000000000005a79fec67cb6e91f1c1da800e478a5+                , ecc_h = 4+                }+            )+getCurveByName SEC_t283k1 =+    CurveF2m $+        CurveBinary+            0x0800000000000000000000000000000000000000000000000000000000000000000010a1+            ( CurveCommon+                { ecc_a =+                    0x000000000000000000000000000000000000000000000000000000000000000000000000+                , ecc_b =+                    0x000000000000000000000000000000000000000000000000000000000000000000000001+                , ecc_g =+                    Point+                        0x0503213f78ca44883f1a3b8162f188e553cd265f23c1567a16876913b0c2ac2458492836                         0x01ccda380f1c9e318d90f95d07e5426fe87e45c0e8184698e45962364e34116177dd2259-        , ecc_n = 0x01ffffffffffffffffffffffffffffffffffe9ae2ed07577265dff7f94451e061e163c61-        , ecc_h = 4-        })-getCurveByName SEC_t283r1 = CurveF2m $ CurveBinary-    0x0800000000000000000000000000000000000000000000000000000000000000000010a1-    (CurveCommon-        { ecc_a = 0x000000000000000000000000000000000000000000000000000000000000000000000001-        , ecc_b = 0x027b680ac8b8596da5a4af8a19a0303fca97fd7645309fa2a581485af6263e313b79a2f5-        , ecc_g = Point 0x05f939258db7dd90e1934f8c70b0dfec2eed25b8557eac9c80e2e198f8cdbecd86b12053+                , ecc_n =+                    0x01ffffffffffffffffffffffffffffffffffe9ae2ed07577265dff7f94451e061e163c61+                , ecc_h = 4+                }+            )+getCurveByName SEC_t283r1 =+    CurveF2m $+        CurveBinary+            0x0800000000000000000000000000000000000000000000000000000000000000000010a1+            ( CurveCommon+                { ecc_a =+                    0x000000000000000000000000000000000000000000000000000000000000000000000001+                , ecc_b =+                    0x027b680ac8b8596da5a4af8a19a0303fca97fd7645309fa2a581485af6263e313b79a2f5+                , ecc_g =+                    Point+                        0x05f939258db7dd90e1934f8c70b0dfec2eed25b8557eac9c80e2e198f8cdbecd86b12053                         0x03676854fe24141cb98fe6d4b20d02b4516ff702350eddb0826779c813f0df45be8112f4-        , ecc_n = 0x03ffffffffffffffffffffffffffffffffffef90399660fc938a90165b042a7cefadb307-        , ecc_h = 2-        })-getCurveByName SEC_t409k1 = CurveF2m $ CurveBinary-    0x02000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000001-    (CurveCommon-        { ecc_a = 0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000-        , ecc_b = 0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001-        , ecc_g = Point 0x0060f05f658f49c1ad3ab1890f7184210efd0987e307c84c27accfb8f9f67cc2c460189eb5aaaa62ee222eb1b35540cfe9023746+                , ecc_n =+                    0x03ffffffffffffffffffffffffffffffffffef90399660fc938a90165b042a7cefadb307+                , ecc_h = 2+                }+            )+getCurveByName SEC_t409k1 =+    CurveF2m $+        CurveBinary+            0x02000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000001+            ( CurveCommon+                { ecc_a =+                    0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000+                , ecc_b =+                    0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001+                , ecc_g =+                    Point+                        0x0060f05f658f49c1ad3ab1890f7184210efd0987e307c84c27accfb8f9f67cc2c460189eb5aaaa62ee222eb1b35540cfe9023746                         0x01e369050b7c4e42acba1dacbf04299c3460782f918ea427e6325165e9ea10e3da5f6c42e9c55215aa9ca27a5863ec48d8e0286b-        , ecc_n = 0x007ffffffffffffffffffffffffffffffffffffffffffffffffffe5f83b2d4ea20400ec4557d5ed3e3e7ca5b4b5c83b8e01e5fcf-        , ecc_h = 4-        })-getCurveByName SEC_t409r1 = CurveF2m $ CurveBinary-    0x02000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000001-    (CurveCommon-        { ecc_a = 0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001-        , ecc_b = 0x0021a5c2c8ee9feb5c4b9a753b7b476b7fd6422ef1f3dd674761fa99d6ac27c8a9a197b272822f6cd57a55aa4f50ae317b13545f-        , ecc_g = Point 0x015d4860d088ddb3496b0c6064756260441cde4af1771d4db01ffe5b34e59703dc255a868a1180515603aeab60794e54bb7996a7+                , ecc_n =+                    0x007ffffffffffffffffffffffffffffffffffffffffffffffffffe5f83b2d4ea20400ec4557d5ed3e3e7ca5b4b5c83b8e01e5fcf+                , ecc_h = 4+                }+            )+getCurveByName SEC_t409r1 =+    CurveF2m $+        CurveBinary+            0x02000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000001+            ( CurveCommon+                { ecc_a =+                    0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001+                , ecc_b =+                    0x0021a5c2c8ee9feb5c4b9a753b7b476b7fd6422ef1f3dd674761fa99d6ac27c8a9a197b272822f6cd57a55aa4f50ae317b13545f+                , ecc_g =+                    Point+                        0x015d4860d088ddb3496b0c6064756260441cde4af1771d4db01ffe5b34e59703dc255a868a1180515603aeab60794e54bb7996a7                         0x0061b1cfab6be5f32bbfa78324ed106a7636b9c5a7bd198d0158aa4f5488d08f38514f1fdf4b4f40d2181b3681c364ba0273c706-        , ecc_n = 0x010000000000000000000000000000000000000000000000000001e2aad6a612f33307be5fa47c3c9e052f838164cd37d9a21173-        , ecc_h = 2-        })-getCurveByName SEC_t571k1 = CurveF2m $ CurveBinary-    0x080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000425-    (CurveCommon-        { ecc_a = 0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000-        , ecc_b = 0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001-        , ecc_g = Point 0x026eb7a859923fbc82189631f8103fe4ac9ca2970012d5d46024804801841ca44370958493b205e647da304db4ceb08cbbd1ba39494776fb988b47174dca88c7e2945283a01c8972+                , ecc_n =+                    0x010000000000000000000000000000000000000000000000000001e2aad6a612f33307be5fa47c3c9e052f838164cd37d9a21173+                , ecc_h = 2+                }+            )+getCurveByName SEC_t571k1 =+    CurveF2m $+        CurveBinary+            0x080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000425+            ( CurveCommon+                { ecc_a =+                    0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000+                , ecc_b =+                    0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001+                , ecc_g =+                    Point+                        0x026eb7a859923fbc82189631f8103fe4ac9ca2970012d5d46024804801841ca44370958493b205e647da304db4ceb08cbbd1ba39494776fb988b47174dca88c7e2945283a01c8972                         0x0349dc807f4fbf374f4aeade3bca95314dd58cec9f307a54ffc61efc006d8a2c9d4979c0ac44aea74fbebbb9f772aedcb620b01a7ba7af1b320430c8591984f601cd4c143ef1c7a3-        , ecc_n = 0x020000000000000000000000000000000000000000000000000000000000000000000000131850e1f19a63e4b391a8db917f4138b630d84be5d639381e91deb45cfe778f637c1001-        , ecc_h = 4-        })-getCurveByName SEC_t571r1 = CurveF2m $ CurveBinary-    0x080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000425-    (CurveCommon-        { ecc_a = 0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001-        , ecc_b = 0x02f40e7e2221f295de297117b7f3d62f5c6a97ffcb8ceff1cd6ba8ce4a9a18ad84ffabbd8efa59332be7ad6756a66e294afd185a78ff12aa520e4de739baca0c7ffeff7f2955727a-        , ecc_g = Point 0x0303001d34b856296c16c0d40d3cd7750a93d1d2955fa80aa5f40fc8db7b2abdbde53950f4c0d293cdd711a35b67fb1499ae60038614f1394abfa3b4c850d927e1e7769c8eec2d19+                , ecc_n =+                    0x020000000000000000000000000000000000000000000000000000000000000000000000131850e1f19a63e4b391a8db917f4138b630d84be5d639381e91deb45cfe778f637c1001+                , ecc_h = 4+                }+            )+getCurveByName SEC_t571r1 =+    CurveF2m $+        CurveBinary+            0x080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000425+            ( CurveCommon+                { ecc_a =+                    0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001+                , ecc_b =+                    0x02f40e7e2221f295de297117b7f3d62f5c6a97ffcb8ceff1cd6ba8ce4a9a18ad84ffabbd8efa59332be7ad6756a66e294afd185a78ff12aa520e4de739baca0c7ffeff7f2955727a+                , ecc_g =+                    Point+                        0x0303001d34b856296c16c0d40d3cd7750a93d1d2955fa80aa5f40fc8db7b2abdbde53950f4c0d293cdd711a35b67fb1499ae60038614f1394abfa3b4c850d927e1e7769c8eec2d19                         0x037bf27342da639b6dccfffeb73d69d78c6c27a6009cbbca1980f8533921e8a684423e43bab08a576291af8f461bb2a8b3531d2f0485c19b16e2f1516e23dd3c1a4827af1b8ac15b-        , ecc_n = 0x03ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe661ce18ff55987308059b186823851ec7dd9ca1161de93d5174d66e8382e9bb2fe84e47-        , ecc_h = 2-        })+                , ecc_n =+                    0x03ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe661ce18ff55987308059b186823851ec7dd9ca1161de93d5174d66e8382e9bb2fe84e47+                , ecc_h = 2+                }+            )
Crypto/PubKey/ECDSA.hs view
@@ -1,3 +1,10 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE StandaloneDeriving #-}+{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE UndecidableInstances #-}+ -- | -- Module      : Crypto.PubKey.ECDSA -- License     : BSD-style@@ -15,60 +22,66 @@ -- Signature operations with P-384 and P-521 may leak the private key. -- -- Signature verification should be safe for all curves.-{-# LANGUAGE BangPatterns #-}-{-# LANGUAGE FlexibleContexts #-}-{-# LANGUAGE ScopedTypeVariables #-}-{-# LANGUAGE StandaloneDeriving #-}-{-# LANGUAGE TypeFamilies #-}-{-# LANGUAGE UndecidableInstances #-}-module Crypto.PubKey.ECDSA-    ( EllipticCurveECDSA (..)+module Crypto.PubKey.ECDSA (+    EllipticCurveECDSA (..),+     -- * Public keys-    , PublicKey-    , encodePublic-    , decodePublic-    , toPublic+    PublicKey,+    encodePublic,+    decodePublic,+    toPublic,+     -- * Private keys-    , PrivateKey-    , encodePrivate-    , decodePrivate+    PrivateKey,+    encodePrivate,+    decodePrivate,+     -- * Signatures-    , Signature(..)-    , signatureFromIntegers-    , signatureToIntegers+    Signature (..),+    signatureFromIntegers,+    signatureToIntegers,+     -- * Generation and verification-    , signWith-    , signDigestWith-    , sign-    , signDigest-    , verify-    , verifyDigest-    ) where+    signWith,+    signDigestWith,+    sign,+    signDigest,+    verify,+    verifyDigest, -import           Control.Monad+    -- * Deterministic nonces+    deterministicNonce,+    signDeterministic,+    signDigestDeterministic,+) where -import           Crypto.ECC+import Control.Monad++import Crypto.ECC import qualified Crypto.ECC.Simple.Types as Simple-import           Crypto.Error-import           Crypto.Hash-import           Crypto.Hash.Types-import           Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess)-import           Crypto.Internal.Imports-import           Crypto.Number.ModArithmetic (inverseFermat)+import Crypto.Error+import Crypto.Hash+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess)+import Crypto.Internal.Imports+import Crypto.Number.Generate (generatePrefix)+import Crypto.Number.ModArithmetic (inverseSafe) import qualified Crypto.PubKey.ECC.P256 as P256-import           Crypto.Random.Types+import Crypto.Random.HmacDRG (initial, update)+import Crypto.Random.Types -import           Data.Bits+import Data.Bits import qualified Data.ByteArray as B-import           Data.Data+import Data.Data -import           Foreign.Ptr (Ptr)-import           Foreign.Storable (peekByteOff, pokeByteOff)+import Foreign.Ptr (Ptr)+import Foreign.Storable (peekByteOff, pokeByteOff)  -- | Represent a ECDSA signature namely R and S. data Signature curve = Signature-    { sign_r :: Scalar curve -- ^ ECDSA r-    , sign_s :: Scalar curve -- ^ ECDSA s+    { sign_r :: Scalar curve+    -- ^ ECDSA r+    , sign_s :: Scalar curve+    -- ^ ECDSA s     }  deriving instance Eq (Scalar curve) => Eq (Signature curve)@@ -99,15 +112,17 @@     pointX :: proxy curve -> Point curve -> Maybe (Scalar curve)  instance EllipticCurveECDSA Curve_P256R1 where-    scalarIsValid _ s = not (P256.scalarIsZero s)-                            && P256.scalarCmp s P256.scalarN == LT+    scalarIsValid _ s =+        not (P256.scalarIsZero s)+            && P256.scalarCmp s P256.scalarN == LT      scalarIsZero _ = P256.scalarIsZero -    scalarInv _ s = let inv = P256.scalarInvSafe s-                     in if P256.scalarIsZero inv then Nothing else Just inv+    scalarInv _ s =+        let inv = P256.scalarInvSafe s+         in if P256.scalarIsZero inv then Nothing else Just inv -    pointX _  = P256.pointX+    pointX _ = P256.pointX  instance EllipticCurveECDSA Curve_P384R1 where     scalarIsValid _ = ecScalarIsValid (Proxy :: Proxy Simple.SEC_p384r1)@@ -116,7 +131,7 @@      scalarInv _ = ecScalarInv (Proxy :: Proxy Simple.SEC_p384r1) -    pointX _  = ecPointX (Proxy :: Proxy Simple.SEC_p384r1)+    pointX _ = ecPointX (Proxy :: Proxy Simple.SEC_p384r1)  instance EllipticCurveECDSA Curve_P521R1 where     scalarIsValid _ = ecScalarIsValid (Proxy :: Proxy Simple.SEC_p521r1)@@ -125,12 +140,12 @@      scalarInv _ = ecScalarInv (Proxy :: Proxy Simple.SEC_p521r1) -    pointX _  = ecPointX (Proxy :: Proxy Simple.SEC_p521r1)-+    pointX _ = ecPointX (Proxy :: Proxy Simple.SEC_p521r1)  -- | Create a signature from integers (R, S).-signatureFromIntegers :: EllipticCurveECDSA curve-                      => proxy curve -> (Integer, Integer) -> CryptoFailable (Signature curve)+signatureFromIntegers+    :: EllipticCurveECDSA curve+    => proxy curve -> (Integer, Integer) -> CryptoFailable (Signature curve) signatureFromIntegers prx (r, s) =     liftA2 Signature (scalarFromInteger prx r) (scalarFromInteger prx s) @@ -138,41 +153,52 @@ -- -- The values can then be used to encode the signature to binary with -- ASN.1.-signatureToIntegers :: EllipticCurveECDSA curve-                    => proxy curve -> Signature curve -> (Integer, Integer)+signatureToIntegers+    :: EllipticCurveECDSA curve+    => proxy curve -> Signature curve -> (Integer, Integer) signatureToIntegers prx sig =     (scalarToInteger prx $ sign_r sig, scalarToInteger prx $ sign_s sig)  -- | Encode a public key into binary form, i.e. the uncompressed encoding -- referenced from <https://tools.ietf.org/html/rfc5480 RFC 5480> section 2.2.-encodePublic :: (EllipticCurve curve, ByteArray bs)-             => proxy curve -> PublicKey curve -> bs+encodePublic+    :: (EllipticCurve curve, ByteArray bs)+    => proxy curve -> PublicKey curve -> bs encodePublic = encodePoint  -- | Try to decode the binary form of a public key.-decodePublic :: (EllipticCurve curve, ByteArray bs)-             => proxy curve -> bs -> CryptoFailable (PublicKey curve)+decodePublic+    :: (EllipticCurve curve, ByteArray bs)+    => proxy curve -> bs -> CryptoFailable (PublicKey curve) decodePublic = decodePoint  -- | Encode a private key into binary form, i.e. the @privateKey@ field -- described in <https://tools.ietf.org/html/rfc5915 RFC 5915>.-encodePrivate :: (EllipticCurveECDSA curve, ByteArray bs)-              => proxy curve -> PrivateKey curve -> bs+encodePrivate+    :: (EllipticCurveECDSA curve, ByteArray bs)+    => proxy curve -> PrivateKey curve -> bs encodePrivate = encodeScalar  -- | Try to decode the binary form of a private key.-decodePrivate :: (EllipticCurveECDSA curve, ByteArray bs)-              => proxy curve -> bs -> CryptoFailable (PrivateKey curve)+decodePrivate+    :: (EllipticCurveECDSA curve, ByteArray bs)+    => proxy curve -> bs -> CryptoFailable (PrivateKey curve) decodePrivate = decodeScalar  -- | Create a public key from a private key.-toPublic :: EllipticCurveECDSA curve-         => proxy curve -> PrivateKey curve -> PublicKey curve+toPublic+    :: EllipticCurveECDSA curve+    => proxy curve -> PrivateKey curve -> PublicKey curve toPublic = pointBaseSmul  -- | Sign digest using the private key and an explicit k scalar.-signDigestWith :: (EllipticCurveECDSA curve, HashAlgorithm hash)-               => proxy curve -> Scalar curve -> PrivateKey curve -> Digest hash -> Maybe (Signature curve)+signDigestWith+    :: (EllipticCurveECDSA curve, HashAlgorithm hash)+    => proxy curve+    -> Scalar curve+    -> PrivateKey curve+    -> Digest hash+    -> Maybe (Signature curve) signDigestWith prx k d digest = do     let z = tHashDigest prx digest         point = pointBaseSmul prx k@@ -183,90 +209,184 @@     return $ Signature r s  -- | Sign message using the private key and an explicit k scalar.-signWith :: (EllipticCurveECDSA curve, ByteArrayAccess msg, HashAlgorithm hash)-         => proxy curve -> Scalar curve -> PrivateKey curve -> hash -> msg -> Maybe (Signature curve)+signWith+    :: (EllipticCurveECDSA curve, ByteArrayAccess msg, HashAlgorithm hash)+    => proxy curve+    -> Scalar curve+    -> PrivateKey curve+    -> hash+    -> msg+    -> Maybe (Signature curve) signWith prx k d hashAlg msg = signDigestWith prx k d (hashWith hashAlg msg)  -- | Sign a digest using hash and private key.-signDigest :: (EllipticCurveECDSA curve, MonadRandom m, HashAlgorithm hash)-           => proxy curve -> PrivateKey curve -> Digest hash -> m (Signature curve)+signDigest+    :: (EllipticCurveECDSA curve, MonadRandom m, HashAlgorithm hash)+    => proxy curve -> PrivateKey curve -> Digest hash -> m (Signature curve) signDigest prx pk digest = do     k <- curveGenerateScalar prx     case signDigestWith prx k pk digest of-        Nothing  -> signDigest prx pk digest+        Nothing -> signDigest prx pk digest         Just sig -> return sig  -- | Sign a message using hash and private key.-sign :: (EllipticCurveECDSA curve, MonadRandom m, ByteArrayAccess msg, HashAlgorithm hash)-     => proxy curve -> PrivateKey curve -> hash -> msg -> m (Signature curve)+sign+    :: ( EllipticCurveECDSA curve+       , MonadRandom m+       , ByteArrayAccess msg+       , HashAlgorithm hash+       )+    => proxy curve -> PrivateKey curve -> hash -> msg -> m (Signature curve) sign prx pk hashAlg msg = signDigest prx pk (hashWith hashAlg msg)  -- | Verify a digest using hash and public key.-verifyDigest :: (EllipticCurveECDSA curve, HashAlgorithm hash)-       => proxy curve -> PublicKey curve -> Signature curve -> Digest hash -> Bool+verifyDigest+    :: (EllipticCurveECDSA curve, HashAlgorithm hash)+    => proxy curve -> PublicKey curve -> Signature curve -> Digest hash -> Bool verifyDigest prx q (Signature r s) digest     | not (scalarIsValid prx r) = False     | not (scalarIsValid prx s) = False     | otherwise = maybe False (r ==) $ do         w <- scalarInv prx s-        let z  = tHashDigest prx digest+        let z = tHashDigest prx digest             u1 = scalarMul prx z w             u2 = scalarMul prx r w-            x  = pointsSmulVarTime prx u1 u2 q+            x = pointsSmulVarTime prx u1 u2 q         pointX prx x-    -- Note: precondition q /= PointO is not tested because we assume-    -- point decoding never decodes point at infinity. +-- Note: precondition q /= PointO is not tested because we assume+-- point decoding never decodes point at infinity.+ -- | Verify a signature using hash and public key.-verify :: (EllipticCurveECDSA curve, ByteArrayAccess msg, HashAlgorithm hash)-       => proxy curve -> hash -> PublicKey curve -> Signature curve -> msg -> Bool+verify+    :: (EllipticCurveECDSA curve, ByteArrayAccess msg, HashAlgorithm hash)+    => proxy curve -> hash -> PublicKey curve -> Signature curve -> msg -> Bool verify prx hashAlg q sig msg = verifyDigest prx q sig (hashWith hashAlg msg)  -- | Truncate a digest based on curve order size.-tHashDigest :: (EllipticCurveECDSA curve, HashAlgorithm hash)-            => proxy curve -> Digest hash -> Scalar curve-tHashDigest prx (Digest digest) = throwCryptoError $ decodeScalar prx encoded-  where m      = curveOrderBits prx-        d      = m - B.length digest * 8-        (n, r) = m `divMod` 8-        n'     = if r > 0 then succ n else n+-- | Deterministic nonce generation according to RFC 6979.+--+-- The nonce is derived from the private key and the message alone, so a+-- signature made this way needs no random number generator and cannot be the+-- one that repeats a nonce -- which, for ECDSA, hands over the private key.+--+-- The hash used to seed the generator is given separately from the one the+-- message was digested with, as RFC 6979 allows.+--+-- The last argument is what to do with a candidate nonce.  It may answer+-- 'Nothing', in which case another candidate is drawn, which is what+-- 'signDigestDeterministic' does for the r or s that comes out zero:+--+-- > deterministicNonce prx SHA256 priv digest (\k -> signDigestWith prx k priv digest)+deterministicNonce+    :: (EllipticCurveECDSA curve, HashAlgorithm hashDRG, HashAlgorithm hashDigest)+    => proxy curve+    -> hashDRG+    -> PrivateKey curve+    -> Digest hashDigest+    -> (Scalar curve -> Maybe a)+    -> a+deterministicNonce prx alg d digest go = fst $ withDRG state run+  where+    state = update seed $ initial alg+    -- RFC 6979 section 3.2 step d: int2octets(x) || bits2octets(h1).  The+    -- second is the truncated digest taken modulo the order, which is what+    -- scalarAdd with zero does, its contract being to reduce there.+    seed =+        B.append (encodeScalar prx d) (encodeScalar prx z)+            :: B.ScrubbedBytes+    z = scalarAdd prx (tHashDigest prx digest) zeroScalar+    zeroScalar = throwCryptoError $ scalarFromInteger prx 0+    run = do+        k <- generatePrefix (curveOrderBits prx)+        case scalarFromInteger prx k of+            CryptoPassed s+                | scalarIsValid prx s -> maybe run pure (go s)+            _ -> run -        encoded-            | d >  0    = B.zero (n' - B.length digest) `B.append` digest-            | d == 0    = digest-            | r == 0    = B.take n digest-            | otherwise = shiftBytes digest+-- | Sign a digest with a nonce derived from the private key and the digest,+-- as RFC 6979 says, rather than from a random number generator.+signDigestDeterministic+    :: (EllipticCurveECDSA curve, HashAlgorithm hashDRG, HashAlgorithm hashDigest)+    => proxy curve+    -> hashDRG+    -> PrivateKey curve+    -> Digest hashDigest+    -> Signature curve+signDigestDeterministic prx alg d digest =+    deterministicNonce prx alg d digest $ \k -> signDigestWith prx k d digest -        shiftBytes bs = B.allocAndFreeze n' $ \dst ->-            B.withByteArray bs $ \src -> go dst src 0 0+-- | Sign a message with a nonce derived from the private key and the message,+-- as RFC 6979 says, rather than from a random number generator.+signDeterministic+    :: ( EllipticCurveECDSA curve+       , HashAlgorithm hashDRG+       , HashAlgorithm hash+       , ByteArrayAccess msg+       )+    => proxy curve+    -> hashDRG+    -> PrivateKey curve+    -> hash+    -> msg+    -> Signature curve+signDeterministic prx alg d hashAlg msg =+    signDigestDeterministic prx alg d (hashWith hashAlg msg) -        go :: Ptr Word8 -> Ptr Word8 -> Word8 -> Int -> IO ()-        go dst src !a i-            | i >= n'   = return ()-            | otherwise = do-                b <- peekByteOff src i-                pokeByteOff dst i (unsafeShiftR b (8 - r) .|. unsafeShiftL a r)-                go dst src b (succ i)+tHashDigest+    :: (EllipticCurveECDSA curve, HashAlgorithm hash)+    => proxy curve -> Digest hash -> Scalar curve+tHashDigest prx dig = throwCryptoError $ decodeScalar prx encoded+  where+    digest = B.convert dig :: B.Bytes+    m = curveOrderBits prx+    d = m - B.length digest * 8+    (n, r) = m `divMod` 8+    n' = if r > 0 then succ n else n +    encoded+        | d > 0 = B.zero (n' - B.length digest) `B.append` digest+        | d == 0 = digest+        | r == 0 = B.take n digest+        | otherwise = shiftBytes digest +    shiftBytes bs = B.allocAndFreeze n' $ \dst ->+        B.withByteArray bs $ \src -> go dst src 0 0++    go :: Ptr Word8 -> Ptr Word8 -> Word8 -> Int -> IO ()+    go dst src !a i+        | i >= n' = return ()+        | otherwise = do+            b <- peekByteOff src i+            pokeByteOff dst i (unsafeShiftR b (8 - r) .|. unsafeShiftL a r)+            go dst src b (succ i)+ ecScalarIsValid :: Simple.Curve c => proxy c -> Simple.Scalar c -> Bool ecScalarIsValid prx (Simple.Scalar s) = s > 0 && s < n-  where n = Simple.curveEccN $ Simple.curveParameters prx+  where+    n = Simple.curveEccN $ Simple.curveParameters prx -ecScalarIsZero :: forall curve . Simple.Curve curve-               => Simple.Scalar curve -> Bool+ecScalarIsZero+    :: forall curve+     . Simple.Curve curve+    => Simple.Scalar curve -> Bool ecScalarIsZero (Simple.Scalar a) = a == 0 -ecScalarInv :: Simple.Curve c-            => proxy c -> Simple.Scalar c -> Maybe (Simple.Scalar c)-ecScalarInv prx (Simple.Scalar s)-    | i == 0    = Nothing-    | otherwise = Just $ Simple.Scalar i-  where n = Simple.curveEccN $ Simple.curveParameters prx-        i = inverseFermat s n+-- | 'inverseSafe' is the one that takes a fixed number of division steps+-- where the assembly for them is built, and checks whatever it gets by+-- multiplying out.  It answers 'Nothing' exactly where the exponentiation+-- this used to do answered zero.+ecScalarInv+    :: Simple.Curve c+    => proxy c -> Simple.Scalar c -> Maybe (Simple.Scalar c)+ecScalarInv prx (Simple.Scalar s) = Simple.Scalar <$> inverseSafe s n+  where+    n = Simple.curveEccN $ Simple.curveParameters prx -ecPointX :: Simple.Curve c-         => proxy c -> Simple.Point c -> Maybe (Simple.Scalar c)-ecPointX _   Simple.PointO      = Nothing+ecPointX+    :: Simple.Curve c+    => proxy c -> Simple.Point c -> Maybe (Simple.Scalar c)+ecPointX _ Simple.PointO = Nothing ecPointX prx (Simple.Point x _) = Just (Simple.Scalar $ x `mod` n)-  where n = Simple.curveEccN $ Simple.curveParameters prx+  where+    n = Simple.curveEccN $ Simple.curveParameters prx
Crypto/PubKey/ECIES.hs view
@@ -18,31 +18,37 @@ -- This module doesn't provide any symmetric data encryption capability or any mean to derive -- cryptographic key material for a symmetric key from the shared secret. -- this is left to the user for now.----module Crypto.PubKey.ECIES-    ( deriveEncrypt-    , deriveDecrypt-    ) where+module Crypto.PubKey.ECIES (+    deriveEncrypt,+    deriveDecrypt,+) where -import           Crypto.ECC-import           Crypto.Error-import           Crypto.Random+import Crypto.ECC+import Crypto.Error+import Crypto.Random  -- | Generate random a new Shared secret and the associated point -- to do a ECIES style encryption-deriveEncrypt :: (MonadRandom randomly, EllipticCurveDH curve)-              => proxy curve -- ^ representation of the curve-              -> Point curve -- ^ the public key of the receiver-              -> randomly (CryptoFailable (Point curve, SharedSecret))+deriveEncrypt+    :: (MonadRandom randomly, EllipticCurveDH curve)+    => proxy curve+    -- ^ representation of the curve+    -> Point curve+    -- ^ the public key of the receiver+    -> randomly (CryptoFailable (Point curve, SharedSecret)) deriveEncrypt proxy pub = do     (KeyPair rPoint rScalar) <- curveGenerateKeyPair proxy     return $ (\s -> (rPoint, s)) `fmap` ecdh proxy rScalar pub  -- | Derive the shared secret with the receiver key -- and the R point of the scheme.-deriveDecrypt :: EllipticCurveDH curve-              => proxy curve  -- ^ representation of the curve-              -> Point curve  -- ^ The received R (supposedly, randomly generated on the encrypt side)-              -> Scalar curve -- ^ The secret key of the receiver-              -> CryptoFailable SharedSecret+deriveDecrypt+    :: EllipticCurveDH curve+    => proxy curve+    -- ^ representation of the curve+    -> Point curve+    -- ^ The received R (supposedly, randomly generated on the encrypt side)+    -> Scalar curve+    -- ^ The secret key of the receiver+    -> CryptoFailable SharedSecret deriveDecrypt proxy point secret = ecdh proxy secret point
Crypto/PubKey/Ed25519.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.PubKey.Ed25519 -- License     : BSD-style@@ -6,51 +9,60 @@ -- Portability : unknown -- -- Ed25519 support----{-# LANGUAGE BangPatterns               #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.PubKey.Ed25519-    ( SecretKey-    , PublicKey-    , Signature+module Crypto.PubKey.Ed25519 (+    SecretKey,+    PublicKey,+    Signature,+     -- * Size constants-    , publicKeySize-    , secretKeySize-    , signatureSize+    publicKeySize,+    secretKeySize,+    signatureSize,+     -- * Smart constructors-    , signature-    , publicKey-    , secretKey+    signature,+    publicKey,+    secretKey,+     -- * Methods-    , toPublic-    , sign-    , verify-    , generateSecretKey-    ) where+    toPublic,+    sign,+    unsafeSign,+    verify,+    generateSecretKey,+) where -import           Data.Word-import           Foreign.C.Types-import           Foreign.Ptr+import Crypto.Debug (DebugShow (..), debugShowBytes)+import Data.Word+import Foreign.C.Types+import Foreign.Ptr -import           Crypto.Error-import           Crypto.Internal.ByteArray (ByteArrayAccess, Bytes,-                                            ScrubbedBytes, withByteArray)+import Crypto.Error+import Crypto.Internal.ByteArray (+    ByteArrayAccess,+    Bytes,+    ScrubbedBytes,+    withByteArray,+ ) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Internal.Compat-import           Crypto.Internal.Imports-import           Crypto.Random+import Crypto.Internal.Compat+import Crypto.Internal.Imports+import Crypto.Random  -- | An Ed25519 Secret key newtype SecretKey = SecretKey ScrubbedBytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData) +instance DebugShow SecretKey where+    debugShow = debugShowBytes "SecretKey"+ -- | An Ed25519 public key newtype PublicKey = PublicKey Bytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData)  -- | An Ed25519 signature newtype Signature = Signature Bytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData)  -- | Try to build a public key from a bytearray publicKey :: ByteArrayAccess ba => ba -> CryptoFailable PublicKey@@ -64,15 +76,16 @@ secretKey :: ByteArrayAccess ba => ba -> CryptoFailable SecretKey secretKey bs     | B.length bs == secretKeySize = unsafeDoIO $ withByteArray bs initialize-    | otherwise                    = CryptoFailed CryptoError_SecretKeyStructureInvalid+    | otherwise =+        CryptoFailed CryptoError_SecretKeyStructureInvalid   where-        initialize inp = do-            valid <- isValidPtr inp-            if valid-                then (CryptoPassed . SecretKey) <$> B.copy bs (\_ -> return ())-                else return $ CryptoFailed CryptoError_SecretKeyStructureInvalid-        isValidPtr _ =-            return True+    initialize inp = do+        valid <- isValidPtr inp+        if valid+            then (CryptoPassed . SecretKey) <$> B.copy bs (\_ -> return ())+            else return $ CryptoFailed CryptoError_SecretKeyStructureInvalid+    isValidPtr _ =+        return True {-# NOINLINE secretKey #-}  -- | Try to build a signature from a bytearray@@ -85,31 +98,51 @@  -- | Create a public key from a secret key toPublic :: SecretKey -> PublicKey-toPublic (SecretKey sec) = PublicKey <$>-    B.allocAndFreeze publicKeySize $ \result ->-    withByteArray sec              $ \psec   ->-        ccrypton_ed25519_publickey psec result+toPublic (SecretKey sec) = PublicKey+    <$> B.allocAndFreeze publicKeySize+    $ \result ->+        withByteArray sec $ \psec ->+            ccrypton_ed25519_publickey psec result {-# NOINLINE toPublic #-} --- | Sign a message using the key pair+-- | Sign a message using the key pair.+--   The public key parameter is ignored and its public key+--   is generated from the secret key parameter to prevent+--   Double Public Key Signing Function Oracle Attack. sign :: ByteArrayAccess ba => SecretKey -> PublicKey -> ba -> Signature-sign secret public message =+sign secret _public message =     Signature $ B.allocAndFreeze signatureSize $ \sig ->-        withByteArray secret  $ \sec ->-        withByteArray public  $ \pub ->-        withByteArray message $ \msg ->-             ccrypton_ed25519_sign msg (fromIntegral msgLen) sec pub sig+        withByteArray secret $ \sec ->+            withByteArray public $ \pub ->+                withByteArray message $ \msg ->+                    ccrypton_ed25519_sign msg (fromIntegral msgLen) sec pub sig   where     !msgLen = B.length message+    public = toPublic secret +-- | Sign a message using the key pair.  This is old `sign`, which is+-- vulnerable to private key compromise if the given public key does+-- not correspond to the secret key. This function is provided for+-- performance critical applications. To use it safely, applications+-- must verify or derive the public key.+unsafeSign :: ByteArrayAccess ba => SecretKey -> PublicKey -> ba -> Signature+unsafeSign secret public message =+    Signature $ B.allocAndFreeze signatureSize $ \sig ->+        withByteArray secret $ \sec ->+            withByteArray public $ \pub ->+                withByteArray message $ \msg ->+                    ccrypton_ed25519_sign msg (fromIntegral msgLen) sec pub sig+  where+    !msgLen = B.length message+ -- | Verify a message verify :: ByteArrayAccess ba => PublicKey -> ba -> Signature -> Bool verify public message signatureVal = unsafeDoIO $     withByteArray signatureVal $ \sig ->-    withByteArray public       $ \pub ->-    withByteArray message      $ \msg -> do-      r <- ccrypton_ed25519_sign_open msg (fromIntegral msgLen) pub sig-      return (r == 0)+        withByteArray public $ \pub ->+            withByteArray message $ \msg -> do+                r <- ccrypton_ed25519_sign_open msg (fromIntegral msgLen) pub sig+                return (r == 0)   where     !msgLen = B.length message @@ -130,21 +163,24 @@ signatureSize = 64  foreign import ccall "crypton_ed25519_publickey"-    ccrypton_ed25519_publickey :: Ptr SecretKey -- secret key-                                  -> Ptr PublicKey -- public key-                                  -> IO ()+    ccrypton_ed25519_publickey+        :: Ptr SecretKey -- secret key+        -> Ptr PublicKey -- public key+        -> IO ()  foreign import ccall "crypton_ed25519_sign_open"-    ccrypton_ed25519_sign_open :: Ptr Word8     -- message-                                  -> CSize         -- message len-                                  -> Ptr PublicKey -- public-                                  -> Ptr Signature -- signature-                                  -> IO CInt+    ccrypton_ed25519_sign_open+        :: Ptr Word8 -- message+        -> CSize -- message len+        -> Ptr PublicKey -- public+        -> Ptr Signature -- signature+        -> IO CInt  foreign import ccall "crypton_ed25519_sign"-    ccrypton_ed25519_sign :: Ptr Word8     -- message-                             -> CSize         -- message len-                             -> Ptr SecretKey -- secret-                             -> Ptr PublicKey -- public-                             -> Ptr Signature -- signature-                             -> IO ()+    ccrypton_ed25519_sign+        :: Ptr Word8 -- message+        -> CSize -- message len+        -> Ptr SecretKey -- secret+        -> Ptr PublicKey -- public+        -> Ptr Signature -- signature+        -> IO ()
Crypto/PubKey/Ed448.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.PubKey.Ed448 -- License     : BSD-style@@ -10,51 +13,60 @@ -- Internally uses Decaf point compression to omit the cofactor -- and implementation by Mike Hamburg.  Externally API and -- data types are compatible with the encoding specified in RFC 8032.----{-# LANGUAGE BangPatterns               #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.PubKey.Ed448-    ( SecretKey-    , PublicKey-    , Signature+module Crypto.PubKey.Ed448 (+    SecretKey,+    PublicKey,+    Signature,+     -- * Size constants-    , publicKeySize-    , secretKeySize-    , signatureSize+    publicKeySize,+    secretKeySize,+    signatureSize,+     -- * Smart constructors-    , signature-    , publicKey-    , secretKey+    signature,+    publicKey,+    secretKey,+     -- * Methods-    , toPublic-    , sign-    , verify-    , generateSecretKey-    ) where+    toPublic,+    sign,+    unsafeSign,+    verify,+    generateSecretKey,+) where -import           Data.Word-import           Foreign.C.Types-import           Foreign.Ptr+import Crypto.Debug (DebugShow (..), debugShowBytes)+import Data.Word+import Foreign.C.Types+import Foreign.Ptr -import           Crypto.Error-import           Crypto.Internal.ByteArray (ByteArrayAccess, Bytes,-                                            ScrubbedBytes, withByteArray)+import Crypto.Error+import Crypto.Internal.ByteArray (+    ByteArrayAccess,+    Bytes,+    ScrubbedBytes,+    withByteArray,+ ) import qualified Crypto.Internal.ByteArray as B-import           Crypto.Internal.Compat-import           Crypto.Internal.Imports-import           Crypto.Random+import Crypto.Internal.Compat+import Crypto.Internal.Imports+import Crypto.Random  -- | An Ed448 Secret key newtype SecretKey = SecretKey ScrubbedBytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData) +instance DebugShow SecretKey where+    debugShow = debugShowBytes "SecretKey"+ -- | An Ed448 public key newtype PublicKey = PublicKey Bytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData)  -- | An Ed448 signature newtype Signature = Signature Bytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData)  -- | Try to build a public key from a bytearray publicKey :: ByteArrayAccess ba => ba -> CryptoFailable PublicKey@@ -68,15 +80,16 @@ secretKey :: ByteArrayAccess ba => ba -> CryptoFailable SecretKey secretKey bs     | B.length bs == secretKeySize = unsafeDoIO $ withByteArray bs initialize-    | otherwise                    = CryptoFailed CryptoError_SecretKeyStructureInvalid+    | otherwise =+        CryptoFailed CryptoError_SecretKeyStructureInvalid   where-        initialize inp = do-            valid <- isValidPtr inp-            if valid-                then (CryptoPassed . SecretKey) <$> B.copy bs (\_ -> return ())-                else return $ CryptoFailed CryptoError_SecretKeyStructureInvalid-        isValidPtr _ =-            return True+    initialize inp = do+        valid <- isValidPtr inp+        if valid+            then (CryptoPassed . SecretKey) <$> B.copy bs (\_ -> return ())+            else return $ CryptoFailed CryptoError_SecretKeyStructureInvalid+    isValidPtr _ =+        return True {-# NOINLINE secretKey #-}  -- | Try to build a signature from a bytearray@@ -89,31 +102,51 @@  -- | Create a public key from a secret key toPublic :: SecretKey -> PublicKey-toPublic (SecretKey sec) = PublicKey <$>-    B.allocAndFreeze publicKeySize $ \result ->-    withByteArray sec              $ \psec   ->-        decaf_ed448_derive_public_key result psec+toPublic (SecretKey sec) = PublicKey+    <$> B.allocAndFreeze publicKeySize+    $ \result ->+        withByteArray sec $ \psec ->+            decaf_ed448_derive_public_key result psec {-# NOINLINE toPublic #-}  -- | Sign a message using the key pair+--   The public key parameter is ignored and its public key+--   is generated from the secret key parameter to prevent+--   Double Public Key Signing Function Oracle Attack. sign :: ByteArrayAccess ba => SecretKey -> PublicKey -> ba -> Signature-sign secret public message =+sign secret _public message =     Signature $ B.allocAndFreeze signatureSize $ \sig ->-        withByteArray secret  $ \sec ->-        withByteArray public  $ \pub ->-        withByteArray message $ \msg ->-             decaf_ed448_sign sig sec pub msg (fromIntegral msgLen) 0 no_context 0+        withByteArray secret $ \sec ->+            withByteArray public $ \pub ->+                withByteArray message $ \msg ->+                    decaf_ed448_sign sig sec pub msg (fromIntegral msgLen) 0 no_context 0   where     !msgLen = B.length message+    public = toPublic secret +-- | Sign a message using the key pair.  This is old `sign`, which is+-- vulnerable to private key compromise if the given public key does+-- not correspond to the secret key. This function is provided for+-- performance critical applications. To use it safely, applications+-- must verify or derive the public key.+unsafeSign :: ByteArrayAccess ba => SecretKey -> PublicKey -> ba -> Signature+unsafeSign secret public message =+    Signature $ B.allocAndFreeze signatureSize $ \sig ->+        withByteArray secret $ \sec ->+            withByteArray public $ \pub ->+                withByteArray message $ \msg ->+                    decaf_ed448_sign sig sec pub msg (fromIntegral msgLen) 0 no_context 0+  where+    !msgLen = B.length message+ -- | Verify a message verify :: ByteArrayAccess ba => PublicKey -> ba -> Signature -> Bool verify public message signatureVal = unsafeDoIO $     withByteArray signatureVal $ \sig ->-    withByteArray public       $ \pub ->-    withByteArray message      $ \msg -> do-      r <- decaf_ed448_verify sig pub msg (fromIntegral msgLen) 0 no_context 0-      return (r /= 0)+        withByteArray public $ \pub ->+            withByteArray message $ \msg -> do+                r <- decaf_ed448_verify sig pub msg (fromIntegral msgLen) 0 no_context 0+                return (r /= 0)   where     !msgLen = B.length message @@ -137,27 +170,30 @@ no_context = nullPtr -- not supported yet  foreign import ccall "crypton_decaf_ed448_derive_public_key"-    decaf_ed448_derive_public_key :: Ptr PublicKey -- public key-                                  -> Ptr SecretKey -- secret key-                                  -> IO ()+    decaf_ed448_derive_public_key+        :: Ptr PublicKey -- public key+        -> Ptr SecretKey -- secret key+        -> IO ()  foreign import ccall "crypton_decaf_ed448_sign"-    decaf_ed448_sign :: Ptr Signature -- signature-                     -> Ptr SecretKey -- secret-                     -> Ptr PublicKey -- public-                     -> Ptr Word8     -- message-                     -> CSize         -- message len-                     -> Word8         -- prehashed-                     -> Ptr Word8     -- context-                     -> Word8         -- context len-                     -> IO ()+    decaf_ed448_sign+        :: Ptr Signature -- signature+        -> Ptr SecretKey -- secret+        -> Ptr PublicKey -- public+        -> Ptr Word8 -- message+        -> CSize -- message len+        -> Word8 -- prehashed+        -> Ptr Word8 -- context+        -> Word8 -- context len+        -> IO ()  foreign import ccall "crypton_decaf_ed448_verify"-    decaf_ed448_verify :: Ptr Signature -- signature-                       -> Ptr PublicKey -- public-                       -> Ptr Word8     -- message-                       -> CSize         -- message len-                       -> Word8         -- prehashed-                       -> Ptr Word8     -- context-                       -> Word8         -- context len-                       -> IO CInt+    decaf_ed448_verify+        :: Ptr Signature -- signature+        -> Ptr PublicKey -- public+        -> Ptr Word8 -- message+        -> CSize -- message len+        -> Word8 -- prehashed+        -> Ptr Word8 -- context+        -> Word8 -- context len+        -> IO CInt
Crypto/PubKey/EdDSA.hs view
@@ -1,3 +1,12 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE RankNTypes #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE TypeOperators #-}+ -- | -- Module      : Crypto.PubKey.EdDSA -- License     : BSD-style@@ -15,80 +24,86 @@ -- This implementation is most useful when wanting to customize the hash -- algorithm.  See module "Crypto.PubKey.Ed25519" for faster Ed25519 with -- SHA-512.----{-# LANGUAGE DataKinds                  #-}-{-# LANGUAGE FlexibleContexts           #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-{-# LANGUAGE OverloadedStrings          #-}-{-# LANGUAGE RankNTypes                 #-}-{-# LANGUAGE ScopedTypeVariables        #-}-{-# LANGUAGE TypeFamilies               #-}-module Crypto.PubKey.EdDSA-    ( SecretKey-    , PublicKey-    , Signature+module Crypto.PubKey.EdDSA (+    SecretKey,+    PublicKey,+    Signature,+     -- * Curves with EdDSA implementation-    , EllipticCurveEdDSA(CurveDigestSize)-    , publicKeySize-    , secretKeySize-    , signatureSize+    EllipticCurveEdDSA (CurveDigestSize),+    publicKeySize,+    secretKeySize,+    signatureSize,+     -- * Smart constructors-    , signature-    , publicKey-    , secretKey+    signature,+    publicKey,+    secretKey,+     -- * Methods-    , toPublic-    , sign-    , signCtx-    , signPh-    , verify-    , verifyCtx-    , verifyPh-    , generateSecretKey-    ) where+    toPublic,+    sign,+    signCtx,+    signPh,+    verify,+    verifyCtx,+    verifyPh,+    generateSecretKey,+) where -import           Data.Bits-import           Data.ByteArray (ByteArray, ByteArrayAccess, Bytes, ScrubbedBytes, View)+import Crypto.Debug (DebugShow (..), debugShowBytes)+import Data.Bits+import Data.ByteArray (+    ByteArray,+    ByteArrayAccess,+    Bytes,+    ScrubbedBytes,+    View,+ ) import qualified Data.ByteArray as B-import           Data.ByteString (ByteString)-import           Data.Proxy+import Data.ByteString (ByteString)+import Data.Proxy -import           Crypto.ECC+import Crypto.ECC import qualified Crypto.ECC.Edwards25519 as Edwards25519-import           Crypto.Error-import           Crypto.Hash (Digest)-import           Crypto.Hash.IO-import           Crypto.Random--import           GHC.TypeLits (KnownNat, Nat)+import Crypto.Error+import Crypto.Hash (Digest)+import Crypto.Hash.IO+import Crypto.Random -import           Crypto.Internal.Builder-import           Crypto.Internal.Compat-import           Crypto.Internal.Imports-import           Crypto.Internal.Nat (integralNatVal)+import GHC.TypeLits (KnownNat, Nat) -import           Foreign.Storable+import Crypto.Internal.Builder+import Crypto.Internal.Compat+import Crypto.Internal.Imports+import Crypto.Internal.Nat (integralNatVal) +import Foreign.Storable  -- API  -- | An EdDSA Secret key newtype SecretKey curve = SecretKey ScrubbedBytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData) +instance DebugShow (SecretKey curve) where+    debugShow = debugShowBytes "SecretKey"+ -- | An EdDSA public key newtype PublicKey curve hash = PublicKey Bytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData)  -- | An EdDSA signature newtype Signature curve hash = Signature Bytes-    deriving (Show,Eq,ByteArrayAccess,NFData)+    deriving (Show, Eq, ByteArrayAccess, NFData)  -- | Elliptic curves with an implementation of EdDSA-class ( EllipticCurveBasepointArith curve-      , KnownNat (CurveDigestSize curve)-      ) => EllipticCurveEdDSA curve where-+class+    ( EllipticCurveBasepointArith curve+    , KnownNat (CurveDigestSize curve)+    ) =>+    EllipticCurveEdDSA curve+    where     -- | Size of the digest for this curve (in bytes)     type CurveDigestSize curve :: Nat @@ -96,43 +111,49 @@     secretKeySize :: proxy curve -> Int      -- hash with specified parameters-    hashWithDom :: (HashAlgorithm hash, ByteArrayAccess ctx, ByteArrayAccess msg)-                => proxy curve -> hash -> Bool -> ctx -> Builder -> msg -> Bytes+    hashWithDom+        :: (HashAlgorithm hash, ByteArrayAccess ctx, ByteArrayAccess msg)+        => proxy curve -> hash -> Bool -> ctx -> Builder -> msg -> Bytes      -- conversion between scalar, point and public key     pointPublic :: proxy curve -> Point curve -> PublicKey curve hash-    publicPoint :: proxy curve -> PublicKey curve hash -> CryptoFailable (Point curve)+    publicPoint+        :: proxy curve -> PublicKey curve hash -> CryptoFailable (Point curve)     encodeScalarLE :: ByteArray bs => proxy curve -> Scalar curve -> bs-    decodeScalarLE :: ByteArrayAccess bs => proxy curve -> bs -> CryptoFailable (Scalar curve)+    decodeScalarLE+        :: ByteArrayAccess bs => proxy curve -> bs -> CryptoFailable (Scalar curve)      -- how to use bits in a secret key-    scheduleSecret :: ( HashAlgorithm hash-                      , HashDigestSize hash ~ CurveDigestSize curve-                      )-                   => proxy curve-                   -> hash-                   -> SecretKey curve-                   -> (Scalar curve, View Bytes)+    scheduleSecret+        :: ( HashAlgorithm hash+           , HashDigestSize hash ~ CurveDigestSize curve+           )+        => proxy curve+        -> hash+        -> SecretKey curve+        -> (Scalar curve, View Bytes)  -- | Size of public keys for this curve (in bytes) publicKeySize :: EllipticCurveEdDSA curve => proxy curve -> Int publicKeySize prx = signatureSize prx `div` 2  -- | Size of signatures for this curve (in bytes)-signatureSize :: forall proxy curve . EllipticCurveEdDSA curve-              => proxy curve -> Int+signatureSize+    :: forall proxy curve+     . EllipticCurveEdDSA curve+    => proxy curve -> Int signatureSize _ = integralNatVal (Proxy :: Proxy (CurveDigestSize curve)) - -- Constructors  -- | Try to build a public key from a bytearray-publicKey :: ( EllipticCurveEdDSA curve-             , HashAlgorithm hash-             , HashDigestSize hash ~ CurveDigestSize curve-             , ByteArrayAccess ba-             )-          => proxy curve -> hash -> ba -> CryptoFailable (PublicKey curve hash)+publicKey+    :: ( EllipticCurveEdDSA curve+       , HashAlgorithm hash+       , HashDigestSize hash ~ CurveDigestSize curve+       , ByteArrayAccess ba+       )+    => proxy curve -> hash -> ba -> CryptoFailable (PublicKey curve hash) publicKey prx _ bs     | B.length bs == publicKeySize prx =         CryptoPassed (PublicKey $ B.convert bs)@@ -140,196 +161,256 @@         CryptoFailed CryptoError_PublicKeySizeInvalid  -- | Try to build a secret key from a bytearray-secretKey :: (EllipticCurveEdDSA curve, ByteArrayAccess ba)-          => proxy curve -> ba -> CryptoFailable (SecretKey curve)+secretKey+    :: (EllipticCurveEdDSA curve, ByteArrayAccess ba)+    => proxy curve -> ba -> CryptoFailable (SecretKey curve) secretKey prx bs     | B.length bs == secretKeySize prx =         CryptoPassed (SecretKey $ B.convert bs)-    | otherwise                        =+    | otherwise =         CryptoFailed CryptoError_SecretKeyStructureInvalid  -- | Try to build a signature from a bytearray-signature :: ( EllipticCurveEdDSA curve-             , HashAlgorithm hash-             , HashDigestSize hash ~ CurveDigestSize curve-             , ByteArrayAccess ba-             )-          => proxy curve -> hash -> ba -> CryptoFailable (Signature curve hash)+signature+    :: ( EllipticCurveEdDSA curve+       , HashAlgorithm hash+       , HashDigestSize hash ~ CurveDigestSize curve+       , ByteArrayAccess ba+       )+    => proxy curve -> hash -> ba -> CryptoFailable (Signature curve hash) signature prx _ bs     | B.length bs == signatureSize prx =         CryptoPassed (Signature $ B.convert bs)     | otherwise =         CryptoFailed CryptoError_SecretKeyStructureInvalid - -- Conversions  -- | Generate a secret key-generateSecretKey :: (EllipticCurveEdDSA curve, MonadRandom m)-                  => proxy curve -> m (SecretKey curve)+generateSecretKey+    :: (EllipticCurveEdDSA curve, MonadRandom m)+    => proxy curve -> m (SecretKey curve) generateSecretKey prx = SecretKey <$> getRandomBytes (secretKeySize prx)  -- | Create a public key from a secret key-toPublic :: ( EllipticCurveEdDSA curve-            , HashAlgorithm hash-            , HashDigestSize hash ~ CurveDigestSize curve-            )-         => proxy curve -> hash -> SecretKey curve -> PublicKey curve hash+toPublic+    :: ( EllipticCurveEdDSA curve+       , HashAlgorithm hash+       , HashDigestSize hash ~ CurveDigestSize curve+       )+    => proxy curve -> hash -> SecretKey curve -> PublicKey curve hash toPublic prx alg priv =     let p = pointBaseSmul prx (secretScalar prx alg priv)      in pointPublic prx p -secretScalar :: ( EllipticCurveEdDSA curve-                , HashAlgorithm hash-                , HashDigestSize hash ~ CurveDigestSize curve-                )-             => proxy curve -> hash -> SecretKey curve -> Scalar curve+secretScalar+    :: ( EllipticCurveEdDSA curve+       , HashAlgorithm hash+       , HashDigestSize hash ~ CurveDigestSize curve+       )+    => proxy curve -> hash -> SecretKey curve -> Scalar curve secretScalar prx alg priv = fst (scheduleSecret prx alg priv) - -- EdDSA signature generation & verification  -- | Sign a message using the key pair-sign :: ( EllipticCurveEdDSA curve-        , HashAlgorithm hash-        , HashDigestSize hash ~ CurveDigestSize curve-        , ByteArrayAccess msg-        )-     => proxy curve -> SecretKey curve -> PublicKey curve hash -> msg -> Signature curve hash+sign+    :: ( EllipticCurveEdDSA curve+       , HashAlgorithm hash+       , HashDigestSize hash ~ CurveDigestSize curve+       , ByteArrayAccess msg+       )+    => proxy curve+    -> SecretKey curve+    -> PublicKey curve hash+    -> msg+    -> Signature curve hash sign prx = signCtx prx emptyCtx  -- | Verify a message-verify :: ( EllipticCurveEdDSA curve-          , HashAlgorithm hash-          , HashDigestSize hash ~ CurveDigestSize curve-          , ByteArrayAccess msg-          )-       => proxy curve -> PublicKey curve hash -> msg -> Signature curve hash -> Bool+verify+    :: ( EllipticCurveEdDSA curve+       , HashAlgorithm hash+       , HashDigestSize hash ~ CurveDigestSize curve+       , ByteArrayAccess msg+       )+    => proxy curve -> PublicKey curve hash -> msg -> Signature curve hash -> Bool verify prx = verifyCtx prx emptyCtx  -- | Sign a message using the key pair under context @ctx@-signCtx :: ( EllipticCurveEdDSA curve-           , HashAlgorithm hash-           , HashDigestSize hash ~ CurveDigestSize curve-           , ByteArrayAccess ctx-           , ByteArrayAccess msg-           )-        => proxy curve -> ctx -> SecretKey curve -> PublicKey curve hash -> msg -> Signature curve hash+signCtx+    :: ( EllipticCurveEdDSA curve+       , HashAlgorithm hash+       , HashDigestSize hash ~ CurveDigestSize curve+       , ByteArrayAccess ctx+       , ByteArrayAccess msg+       )+    => proxy curve+    -> ctx+    -> SecretKey curve+    -> PublicKey curve hash+    -> msg+    -> Signature curve hash signCtx prx = signPhCtx prx False  -- | Verify a message under context @ctx@-verifyCtx :: ( EllipticCurveEdDSA curve-             , HashAlgorithm hash-             , HashDigestSize hash ~ CurveDigestSize curve-             , ByteArrayAccess ctx-             , ByteArrayAccess msg-             )-          => proxy curve -> ctx -> PublicKey curve hash -> msg -> Signature curve hash -> Bool+verifyCtx+    :: ( EllipticCurveEdDSA curve+       , HashAlgorithm hash+       , HashDigestSize hash ~ CurveDigestSize curve+       , ByteArrayAccess ctx+       , ByteArrayAccess msg+       )+    => proxy curve+    -> ctx+    -> PublicKey curve hash+    -> msg+    -> Signature curve hash+    -> Bool verifyCtx prx = verifyPhCtx prx False  -- | Sign a prehashed message using the key pair under context @ctx@-signPh :: ( EllipticCurveEdDSA curve-          , HashAlgorithm hash-          , HashDigestSize hash ~ CurveDigestSize curve-          , ByteArrayAccess ctx-          )-       => proxy curve -> ctx -> SecretKey curve -> PublicKey curve hash -> Digest prehash -> Signature curve hash+signPh+    :: ( EllipticCurveEdDSA curve+       , HashAlgorithm hash+       , HashDigestSize hash ~ CurveDigestSize curve+       , ByteArrayAccess ctx+       )+    => proxy curve+    -> ctx+    -> SecretKey curve+    -> PublicKey curve hash+    -> Digest prehash+    -> Signature curve hash signPh prx = signPhCtx prx True  -- | Verify a prehashed message under context @ctx@-verifyPh :: ( EllipticCurveEdDSA curve-            , HashAlgorithm hash-            , HashDigestSize hash ~ CurveDigestSize curve-            , ByteArrayAccess ctx-            )-         => proxy curve -> ctx -> PublicKey curve hash -> Digest prehash -> Signature curve hash -> Bool+verifyPh+    :: ( EllipticCurveEdDSA curve+       , HashAlgorithm hash+       , HashDigestSize hash ~ CurveDigestSize curve+       , ByteArrayAccess ctx+       )+    => proxy curve+    -> ctx+    -> PublicKey curve hash+    -> Digest prehash+    -> Signature curve hash+    -> Bool verifyPh prx = verifyPhCtx prx True -signPhCtx :: forall proxy curve hash ctx msg .-             ( EllipticCurveEdDSA curve-             , HashAlgorithm hash-             , HashDigestSize hash ~ CurveDigestSize curve-             , ByteArrayAccess ctx-             , ByteArrayAccess msg-             )-          => proxy curve -> Bool -> ctx -> SecretKey curve -> PublicKey curve hash -> msg -> Signature curve hash+signPhCtx+    :: forall proxy curve hash ctx msg+     . ( EllipticCurveEdDSA curve+       , HashAlgorithm hash+       , HashDigestSize hash ~ CurveDigestSize curve+       , ByteArrayAccess ctx+       , ByteArrayAccess msg+       )+    => proxy curve+    -> Bool+    -> ctx+    -> SecretKey curve+    -> PublicKey curve hash+    -> msg+    -> Signature curve hash signPhCtx prx ph ctx priv pub msg =-    let alg  = undefined :: hash+    let alg = undefined :: hash         (s, prefix) = scheduleSecret prx alg priv         digR = hashWithDom prx alg ph ctx (bytes prefix) msg-        r    = decodeScalarNoErr prx digR-        pR   = pointBaseSmul prx r-        bsR  = encodePoint prx pR-        sK   = getK prx ph ctx pub bsR msg-        sS   = scalarAdd prx r (scalarMul prx sK s)+        r = decodeScalarNoErr prx digR+        pR = pointBaseSmul prx r+        bsR = encodePoint prx pR+        sK = getK prx ph ctx pub bsR msg+        sS = scalarAdd prx r (scalarMul prx sK s)      in encodeSignature prx (bsR, pR, sS) -verifyPhCtx :: ( EllipticCurveEdDSA curve-               , HashAlgorithm hash-               , HashDigestSize hash ~ CurveDigestSize curve-               , ByteArrayAccess ctx-               , ByteArrayAccess msg-               )-            => proxy curve -> Bool -> ctx -> PublicKey curve hash -> msg -> Signature curve hash -> Bool+verifyPhCtx+    :: ( EllipticCurveEdDSA curve+       , HashAlgorithm hash+       , HashDigestSize hash ~ CurveDigestSize curve+       , ByteArrayAccess ctx+       , ByteArrayAccess msg+       )+    => proxy curve+    -> Bool+    -> ctx+    -> PublicKey curve hash+    -> msg+    -> Signature curve hash+    -> Bool verifyPhCtx prx ph ctx pub msg sig =     case doVerify of         CryptoPassed verified -> verified-        CryptoFailed _        -> False+        CryptoFailed _ -> False   where     doVerify = do         (bsR, pR, sS) <- decodeSignature prx sig         nPub <- pointNegate prx `fmap` publicPoint prx pub-        let sK  = getK prx ph ctx pub bsR msg+        let sK = getK prx ph ctx pub bsR msg             pR' = pointsSmulVarTime prx sS sK nPub         return (pR == pR')  emptyCtx :: Bytes emptyCtx = B.empty -getK :: forall proxy curve hash ctx msg .-        ( EllipticCurveEdDSA curve-        , HashAlgorithm hash-        , HashDigestSize hash ~ CurveDigestSize curve-        , ByteArrayAccess ctx-        , ByteArrayAccess msg-        )-     => proxy curve -> Bool -> ctx -> PublicKey curve hash -> Bytes -> msg -> Scalar curve+getK+    :: forall proxy curve hash ctx msg+     . ( EllipticCurveEdDSA curve+       , HashAlgorithm hash+       , HashDigestSize hash ~ CurveDigestSize curve+       , ByteArrayAccess ctx+       , ByteArrayAccess msg+       )+    => proxy curve+    -> Bool+    -> ctx+    -> PublicKey curve hash+    -> Bytes+    -> msg+    -> Scalar curve getK prx ph ctx (PublicKey pub) bsR msg =-    let alg  = undefined :: hash+    let alg = undefined :: hash         digK = hashWithDom prx alg ph ctx (bytes bsR <> bytes pub) msg      in decodeScalarNoErr prx digK -encodeSignature :: EllipticCurveEdDSA curve-                => proxy curve-                -> (Bytes, Point curve, Scalar curve)-                -> Signature curve hash-encodeSignature prx (bsR, _, sS) = Signature $ buildAndFreeze $-    bytes bsR <> bytes bsS <> zero len0+encodeSignature+    :: EllipticCurveEdDSA curve+    => proxy curve+    -> (Bytes, Point curve, Scalar curve)+    -> Signature curve hash+encodeSignature prx (bsR, _, sS) =+    Signature $+        buildAndFreeze $+            bytes bsR <> bytes bsS <> zero len0   where-    bsS  = encodeScalarLE prx sS :: Bytes+    bsS = encodeScalarLE prx sS :: Bytes     len0 = signatureSize prx - B.length bsR - B.length bsS -decodeSignature :: ( EllipticCurveEdDSA curve-                   , HashDigestSize hash ~ CurveDigestSize curve-                   )-                => proxy curve-                -> Signature curve hash-                -> CryptoFailable (Bytes, Point curve, Scalar curve)-decodeSignature prx (Signature bs) = do+decodeSignature+    :: ( EllipticCurveEdDSA curve+       , HashDigestSize hash ~ CurveDigestSize curve+       )+    => proxy curve+    -> Signature curve hash+    -> CryptoFailable (Bytes, Point curve, Scalar curve)+decodeSignature prx sig@(Signature bs) = do     let (bsR, bsS) = B.splitAt (publicKeySize prx) bs     pR <- decodePoint prx bsR     sS <- decodeScalarLE prx bsS-    return (bsR, pR, sS)+    if encodeSignature prx (encodePoint prx pR, pR, sS) == sig+        then return (bsR, pR, sS)+        else CryptoFailed CryptoError_PointFormatInvalid  -- implementations are supposed to decode any scalar up to the size of the digest-decodeScalarNoErr :: (EllipticCurveEdDSA curve, ByteArrayAccess bs)-                  => proxy curve -> bs -> Scalar curve+decodeScalarNoErr+    :: (EllipticCurveEdDSA curve, ByteArrayAccess bs)+    => proxy curve -> bs -> Scalar curve decodeScalarNoErr prx = unwrap "decodeScalarNoErr" . decodeScalarLE prx  unwrap :: String -> CryptoFailable a -> a unwrap name (CryptoFailed _) = error (name ++ ": assumption failed")-unwrap _    (CryptoPassed x) = x-+unwrap _ (CryptoPassed x) = x  -- Ed25519 implementation @@ -339,11 +420,13 @@      hashWithDom _ alg ph ctx bss         | not ph && B.null ctx = digestDomMsg alg bss-        | otherwise            = digestDomMsg alg (dom <> bss)-      where dom = bytes ("SigEd25519 no Ed25519 collisions" :: ByteString) <>-                  byte (if ph then 1 else 0) <>-                  byte (fromIntegral $ B.length ctx) <>-                  bytes ctx+        | otherwise = digestDomMsg alg (dom <> bss)+      where+        dom =+            bytes ("SigEd25519 no Ed25519 collisions" :: ByteString)+                <> byte (if ph then 1 else 0)+                <> byte (fromIntegral $ B.length ctx)+                <> bytes ctx      pointPublic _ = PublicKey . Edwards25519.pointEncode     publicPoint _ = Edwards25519.pointDecode@@ -353,15 +436,14 @@     scheduleSecret prx alg priv =         (decodeScalarNoErr prx clamped, B.dropView hashed 32)       where-        hashed  = digest alg $ \update -> update priv+        hashed = digest alg $ \update -> update priv          clamped :: Bytes         clamped = B.copyAndFreeze (B.takeView hashed 32) $ \p -> do-                      b0  <- peekElemOff p 0  :: IO Word8-                      b31 <- peekElemOff p 31 :: IO Word8-                      pokeElemOff p 31 ((b31 .&. 0x7F) .|. 0x40)-                      pokeElemOff p 0  (b0 .&. 0xF8)-+            b0 <- peekElemOff p 0 :: IO Word8+            b31 <- peekElemOff p 31 :: IO Word8+            pokeElemOff p 31 ((b31 .&. 0x7F) .|. 0x40)+            pokeElemOff p 0 (b0 .&. 0xF8)  {-   Optimize hashing by limiting the number of roundtrips between Haskell and C.@@ -375,15 +457,17 @@   pinned trampoline. -} -digestDomMsg :: (HashAlgorithm alg, ByteArrayAccess msg)-             => alg -> Builder -> msg -> Bytes+digestDomMsg+    :: (HashAlgorithm alg, ByteArrayAccess msg)+    => alg -> Builder -> msg -> Bytes digestDomMsg alg bss bs = digest alg $ \update ->     update (buildAndFreeze bss :: Bytes) >> update bs -digest :: HashAlgorithm alg-       => alg-       -> ((forall bs . ByteArrayAccess bs => bs -> IO ()) -> IO ())-       -> Bytes+digest+    :: HashAlgorithm alg+    => alg+    -> ((forall bs. ByteArrayAccess bs => bs -> IO ()) -> IO ())+    -> Bytes digest alg fn = B.convert $ unsafeDoIO $ do     mc <- hashMutableInitWith alg     fn (hashMutableUpdate mc)
Crypto/PubKey/ElGamal.hs view
@@ -1,3 +1,6 @@+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.PubKey.ElGamal -- License     : BSD-style@@ -5,143 +8,259 @@ -- Stability   : experimental -- Portability : Good ----- This module is a work in progress. do not use:--- it might eat your dog, your data or even both.+-- ElGamal encryption and signature over the multiplicative group of integers+-- modulo a prime, reusing the parameters of "Crypto.PubKey.DH". ----- TODO: provide a mapping between integer and ciphertext---       generate numbers correctly+-- /These are raw primitives, not a scheme./  The encryption here is textbook+-- ElGamal: it applies no padding, so it is malleable by construction --+-- multiplying a ciphertext's second component by @t@ multiplies the plaintext+-- by @t@ -- and it is not IND-CCA secure.  A message is an 'Integer' below the+-- modulus rather than a byte string, and nothing here maps one to the other.+-- Use it to build a scheme that adds those, or prefer+-- "Crypto.PubKey.RSA.OAEP" or "Crypto.PubKey.ECIES" where a scheme is what is+-- wanted. ---{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.PubKey.ElGamal-    ( Params-    , PublicNumber-    , PrivateNumber-    , EphemeralKey(..)-    , SharedKey-    , Signature+-- The signature primitive is likewise raw, and an ephemeral value must never+-- be reused between signatures: two signatures under the same @k@ reveal the+-- private key.+--+-- == What is kept from the clock, and what is not+--+-- Every exponentiation with a secret exponent is+-- 'Crypto.Number.ModArithmetic.expSafe'.  Decryption inverts the shared+-- secret by Fermat's little theorem rather than by the extended Euclidean+-- algorithm, whose steps follow the bits it is given.  'sign' cannot do that+-- -- @k@ is inverted modulo @p-1@, which is even -- so it blinds instead: the+-- algorithm is handed @k@ times a fresh random unit, and the blinder is+-- divided out afterwards.  'signWith', having no randomness of its own, hands+-- it @k@.+--+-- What is left is the 'Integer' arithmetic around all of that, whose cost+-- follows the size of the numbers.  See "Crypto.PubKey.DSA" for the same note+-- at more length.+module Crypto.PubKey.ElGamal (+    Params,+    PublicNumber,+    PrivateNumber,+    EphemeralKey (..),+    SharedKey,+    Signature (..),+     -- * Generation-    , generatePrivate-    , generatePublic+    generatePrivate,+    generatePublic,+     -- * Encryption and decryption with no scheme-    , encryptWith-    , encrypt-    , decrypt+    encryptWith,+    encrypt,+    decrypt,+     -- * Signature primitives-    , signWith-    , sign+    signWith,+    sign,+     -- * Verification primitives-    , verify-    ) where+    verify,+) where -import Data.Maybe (fromJust)-import Crypto.Internal.Imports+import Crypto.Error+import Crypto.Hash import Crypto.Internal.ByteArray (ByteArrayAccess)-import Crypto.Number.ModArithmetic (expSafe, expFast, inverse)-import Crypto.Number.Generate (generateMax)-import Crypto.Number.Serialize (os2ip)+import Crypto.Internal.Imports import Crypto.Number.Basic (gcde)+import Crypto.Number.Generate (generateBetween, generateMax)+import Crypto.Number.ModArithmetic (expFast, expSafe, inverseSafe)+import Crypto.Number.Serialize (os2ip)+import Crypto.PubKey.DH (+    Params (..),+    PrivateNumber (..),+    PublicNumber (..),+    SharedKey (..),+ ) import Crypto.Random.Types-import Crypto.PubKey.DH (PrivateNumber(..), PublicNumber(..), Params(..), SharedKey(..))-import Crypto.Hash+import Data.Data  -- | ElGamal Signature-data Signature = Signature (Integer, Integer)+data Signature = Signature+    { sign_r :: Integer+    -- ^ ElGamal r+    , sign_s :: Integer+    -- ^ ElGamal s+    }+    deriving (Show, Read, Eq, Data) +instance NFData Signature where+    rnf (Signature r s) = r `seq` s `seq` ()+ -- | ElGamal Ephemeral key. also called Temporary key. newtype EphemeralKey = EphemeralKey Integer     deriving (NFData) --- | generate a private number with no specific property--- this number is usually called a and need to be between--- 0 and q (order of the group G).---+-- | generate a private number, in @[1, q-1]@ where @q@ is the order of the+-- group.  Zero is excluded: it would make the public number 1 and the shared+-- value constant. generatePrivate :: MonadRandom m => Integer -> m PrivateNumber-generatePrivate q = PrivateNumber <$> generateMax q---- | generate an ephemeral key which is a number with no specific property,--- and need to be between 0 and q (order of the group G).----generateEphemeral :: MonadRandom m => Integer -> m EphemeralKey-generateEphemeral q = toEphemeral <$> generatePrivate q-    where toEphemeral (PrivateNumber n) = EphemeralKey n+generatePrivate q = PrivateNumber <$> generateBetween 1 (q - 1)  -- | generate a public number that is for the other party benefits. -- this number is usually called h=g^a generatePublic :: Params -> PrivateNumber -> PublicNumber generatePublic (Params p g _) (PrivateNumber a) = PublicNumber $ expSafe g a p +-- | Is the other party's public number usable?+--+-- @1@ and @p-1@ generate a group of one or two elements, so the value they+-- mask the message with is one of a handful of constants.+validPublic :: Integer -> Integer -> Bool+validPublic p h = h > 1 && h < p - 1+ -- | encrypt with a specified ephemeral key--- do not reuse ephemeral key.-encryptWith :: EphemeralKey -> Params -> PublicNumber -> Integer -> (Integer,Integer)-encryptWith (EphemeralKey b) (Params p g _) (PublicNumber h) m = (c1,c2)-    where s  = expSafe h b p-          c1 = expSafe g b p-          c2 = (s * m) `mod` p+--+-- The ephemeral key must lie in @[1, p-2]@ and must never be reused: zero+-- would leave the message unmasked, and a repeat lets anyone who learns one+-- plaintext recover the other.  A message must be below the modulus, or+-- decryption would return it reduced.+encryptWith+    :: EphemeralKey+    -> Params+    -> PublicNumber+    -> Integer+    -> CryptoFailable (Integer, Integer)+encryptWith (EphemeralKey b) (Params p g _) (PublicNumber h) m+    | b < 1 || b > p - 2 = CryptoFailed CryptoError_ParameterInvalid+    | not (validPublic p h) = CryptoFailed CryptoError_ParameterInvalid+    | m < 0 || m >= p = CryptoFailed CryptoError_ParameterInvalid+    | otherwise = CryptoPassed (c1, c2)+  where+    s = expSafe h b p+    c1 = expSafe g b p+    c2 = (s * m) `mod` p  -- | encrypt a message using params and public keys -- will generate b (called the ephemeral key)-encrypt :: MonadRandom m => Params -> PublicNumber -> Integer -> m (Integer,Integer)-encrypt params@(Params p _ _) public m = (\b -> encryptWith b params public m) <$> generateEphemeral q-    where q = p-1 -- p is prime, hence order of the group is p-1+encrypt+    :: MonadRandom m+    => Params+    -> PublicNumber+    -> Integer+    -> m (CryptoFailable (Integer, Integer))+encrypt params@(Params p _ _) public m+    | p < 5 = return (CryptoFailed CryptoError_ParameterInvalid)+    | otherwise = do+        b <- generateBetween 1 (p - 2)+        return $ encryptWith (EphemeralKey b) params public m  -- | decrypt message-decrypt :: Params -> PrivateNumber -> (Integer, Integer) -> Integer-decrypt (Params p _ _) (PrivateNumber a) (c1,c2) = (c2 * sm1) `mod` p-    where s   = expSafe c1 a p-          sm1 = fromJust $ inverse s p -- always inversible in Zp+--+-- @c1@ must be a unit modulo @p@; a ciphertext whose first component is zero+-- or out of range is rejected rather than raising.+decrypt+    :: Params -> PrivateNumber -> (Integer, Integer) -> CryptoFailable Integer+decrypt (Params p _ _) (PrivateNumber a) (c1, c2)+    | c1 <= 0 || c1 >= p = CryptoFailed CryptoError_ParameterInvalid+    | c2 < 0 || c2 >= p = CryptoFailed CryptoError_ParameterInvalid+    | otherwise = case inverseSafe s p of+        Nothing -> CryptoFailed CryptoError_ParameterInvalid+        Just sm1 -> CryptoPassed ((c2 * sm1) `mod` p)+  where+    -- the shared secret, which the extended Euclidean algorithm would take+    -- apart: its steps follow the bits of what it is given, and this one is+    -- worth the private number.  p is prime, so Fermat gives the inverse+    -- without reading it+    s = expSafe c1 a p --- | sign a message with an explicit k number+-- | sign a message with an explicit ephemeral value ----- if k is not appropriate, then no signature is returned.+-- @k@ has to lie in @[1, p-2]@ and be coprime with @p-1@.  'Nothing' says the+-- value handed in cannot be used: either it fails one of those two conditions,+-- or it is one of the few that produce a second component of zero.  Either way+-- the answer is to draw another @k@, which is what 'sign' does. ----- with some appropriate value of k, the signature generation can fail,--- and no signature is returned. User of this function need to retry--- with a different k value.-signWith :: (ByteArrayAccess msg, HashAlgorithm hash)-         => Integer         -- ^ random number k, between 0 and p-1 and gcd(k,p-1)=1-         -> Params          -- ^ DH params (p,g)-         -> PrivateNumber   -- ^ DH private key-         -> hash            -- ^ collision resistant hash algorithm-         -> msg             -- ^ message to sign-         -> Maybe Signature-signWith k (Params p g _) (PrivateNumber x) hashAlg msg-    | k >= p-1 || d > 1 = Nothing -- gcd(k,p-1) is not 1-    | s == 0            = Nothing-    | otherwise         = Just $ Signature (r,s)-    where r          = expSafe g k p-          h          = os2ip $ hashWith hashAlg msg-          s          = ((h - x*r) * kInv) `mod` (p-1)-          (kInv,_,d) = gcde k (p-1)+-- @k@ is an ephemeral private key.  It has to be drawn uniformly at random,+-- kept secret, and used for one signature only: the private number follows+-- from a signature and its @k@, and equally from two signatures made with the+-- same @k@.  None of that is visible to this function, which is why it takes+-- @k@ from the caller and checks only what it can.+signWith+    :: (ByteArrayAccess msg, HashAlgorithm hash)+    => Integer+    -- ^ ephemeral value k, in [1, p-2] and coprime with p-1+    -> Params+    -- ^ DH params (p,g)+    -> PrivateNumber+    -- ^ DH private key+    -> hash+    -- ^ collision resistant hash algorithm+    -> msg+    -- ^ message to sign+    -> Maybe Signature+signWith = signWithBlinder 1 --- | sign message+-- | The same with a blinder for the inversion of @k@. ----- This function will generate a random number, however--- as the signature might fail, the function will automatically retry--- until a proper signature has been created.+-- @k@ is inverted modulo @p-1@, which is even, so Fermat's little theorem+-- does not reach it the way it reaches DSA's @k@ modulo a prime order: the+-- extended Euclidean algorithm is the only way there, and its steps follow+-- the bits of what it is given.  What can be done instead is to hand it+-- something else: for a unit @b@, the inverse of @k*b@ times @b@ is the+-- inverse of @k@, and the steps then follow @k*b@, which is a fresh random+-- number.  A blinder of 1 is no blinding, which is what the exported+-- 'signWith' has to do, having no randomness of its own. ---sign :: (ByteArrayAccess msg, HashAlgorithm hash, MonadRandom m)-     => Params         -- ^ DH params (p,g)-     -> PrivateNumber  -- ^ DH private key-     -> hash           -- ^ collision resistant hash algorithm-     -> msg            -- ^ message to sign-     -> m Signature+-- When @b@ shares a factor with @p-1@ the algorithm reports it the same way+-- it reports one in @k@, and the answer is the same: draw again.+signWithBlinder+    :: (ByteArrayAccess msg, HashAlgorithm hash)+    => Integer -> Integer -> Params -> PrivateNumber -> hash -> msg -> Maybe Signature+signWithBlinder b k (Params p g _) (PrivateNumber x) hashAlg msg+    | k <= 0 || k >= p - 1 || b <= 0 || d > 1 = Nothing+    | s == 0 = Nothing+    | otherwise = Just $ Signature r s+  where+    r = expSafe g k p+    h = os2ip $ hashWith hashAlg msg+    s = ((h - x * r) * kInv) `mod` (p - 1)+    kInv = (kbInv * b) `mod` (p - 1)+    (kbInv, _, d) = gcde ((k * b) `mod` (p - 1)) (p - 1)++-- | sign message+--+-- This function draws the ephemeral value itself, and draws a fresh one on+-- each attempt until 'signWith' accepts it, so a caller who has no particular+-- @k@ in mind should use this rather than 'signWith'.+sign+    :: (ByteArrayAccess msg, HashAlgorithm hash, MonadRandom m)+    => Params+    -- ^ DH params (p,g)+    -> PrivateNumber+    -- ^ DH private key+    -> hash+    -- ^ collision resistant hash algorithm+    -> msg+    -- ^ message to sign+    -> m Signature sign params@(Params p _ _) priv hashAlg msg = do-    k <- generateMax (p-1)-    case signWith k params priv hashAlg msg of-        Nothing  -> sign params priv hashAlg msg+    k <- generateMax (p - 1)+    -- and a blinder for the inversion of k, which is the one step here that+    -- the extended Euclidean algorithm has to do+    b <- generateMax (p - 1)+    case signWithBlinder b k params priv hashAlg msg of+        Nothing -> sign params priv hashAlg msg         Just sig -> return sig  -- | verify a signature-verify :: (ByteArrayAccess msg, HashAlgorithm hash)-       => Params-       -> PublicNumber-       -> hash-       -> msg-       -> Signature-       -> Bool-verify (Params p g _) (PublicNumber y) hashAlg msg (Signature (r,s))-    | or [r <= 0,r >= p,s <= 0,s >= (p-1)] = False-    | otherwise                            = lhs == rhs-    where h   = os2ip $ hashWith hashAlg msg-          lhs = expFast g h p-          rhs = (expFast y r p * expFast r s p) `mod` p+verify+    :: (ByteArrayAccess msg, HashAlgorithm hash)+    => Params+    -> PublicNumber+    -> hash+    -> msg+    -> Signature+    -> Bool+verify (Params p g _) (PublicNumber y) hashAlg msg (Signature r s)+    | or [r <= 0, r >= p, s <= 0, s >= (p - 1)] = False+    | otherwise = lhs == rhs+  where+    h = os2ip $ hashWith hashAlg msg+    lhs = expFast g h p+    rhs = (expFast y r p * expFast r s p) `mod` p
Crypto/PubKey/Internal.hs view
@@ -4,16 +4,16 @@ -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----module Crypto.PubKey.Internal-    ( and'-    , (&&!)-    , dsaTruncHash-    , dsaTruncHashDigest-    ) where+module Crypto.PubKey.Internal (+    and',+    (&&!),+    dsaTruncHash,+    dsaTruncHashDigest,+) where  import Data.Bits (shiftR) import Data.List (foldl')+import Prelude hiding (foldl')  import Crypto.Hash import Crypto.Internal.ByteArray (ByteArrayAccess)@@ -26,13 +26,14 @@  -- | This is a strict version of &&. (&&!) :: Bool -> Bool -> Bool-True  &&! True  = True-True  &&! False = False-False &&! True  = False+True &&! True = True+True &&! False = False+False &&! True = False False &&! False = False  -- | Truncate and hash for DSA and ECDSA.-dsaTruncHash :: (ByteArrayAccess msg, HashAlgorithm hash) => hash -> msg -> Integer -> Integer+dsaTruncHash+    :: (ByteArrayAccess msg, HashAlgorithm hash) => hash -> msg -> Integer -> Integer dsaTruncHash hashAlg = dsaTruncHashDigest . hashWith hashAlg  -- | Truncate a digest for DSA and ECDSA.@@ -40,8 +41,9 @@ dsaTruncHashDigest digest n     | d > 0 = shiftR e d     | otherwise = e-  where e = os2ip digest-        d = hashDigestSize (getHashAlg digest) * 8 - numBits n+  where+    e = os2ip digest+    d = hashDigestSize (getHashAlg digest) * 8 - numBits n  getHashAlg :: Digest hash -> hash getHashAlg _ = undefined
Crypto/PubKey/MaskGenFunction.hs view
@@ -1,40 +1,45 @@+{-# LANGUAGE BangPatterns #-}+ -- | -- Module      : Crypto.PubKey.MaskGenFunction -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----{-# LANGUAGE BangPatterns #-}-module Crypto.PubKey.MaskGenFunction-    ( MaskGenAlgorithm-    , mgf1-    ) where+module Crypto.PubKey.MaskGenFunction (+    MaskGenAlgorithm,+    mgf1,+) where -import           Crypto.Number.Serialize (i2ospOf_)-import           Crypto.Hash-import           Crypto.Internal.ByteArray (ByteArrayAccess, ByteArray, Bytes)+import Crypto.Hash+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, Bytes) import qualified Crypto.Internal.ByteArray as B+import Crypto.Number.Serialize (i2ospOf_)  -- | Represent a mask generation algorithm type MaskGenAlgorithm seed output =-       seed   -- ^ seed-    -> Int    -- ^ length to generate+    seed+    -- ^ seed+    -> Int+    -- ^ length to generate     -> output  -- | Mask generation algorithm MGF1-mgf1 :: (ByteArrayAccess seed, ByteArray output, HashAlgorithm hashAlg)-     => hashAlg-     -> seed-     -> Int-     -> output+mgf1+    :: (ByteArrayAccess seed, ByteArray output, HashAlgorithm hashAlg)+    => hashAlg+    -> seed+    -> Int+    -> output mgf1 hashAlg seed len =     let !seededCtx = hashUpdate (hashInitWith hashAlg) seed-     in B.take len $ B.concat $ map (hashCounter seededCtx) [0..fromIntegral (maxCounter-1)]+     in B.take len $+            B.concat $+                map (hashCounter seededCtx) [0 .. fromIntegral (maxCounter - 1)]   where-    digestLen     = hashDigestSize hashAlg-    (chunks,left) = len `divMod` digestLen-    maxCounter    = if left > 0 then chunks + 1 else chunks+    digestLen = hashDigestSize hashAlg+    (chunks, left) = len `divMod` digestLen+    maxCounter = if left > 0 then chunks + 1 else chunks      hashCounter :: HashAlgorithm a => Context a -> Integer -> Digest a     hashCounter ctx counter = hashFinalize $ hashUpdate ctx (i2ospOf_ 4 counter :: Bytes)
Crypto/PubKey/RSA.hs view
@@ -1,26 +1,35 @@+{-# LANGUAGE ScopedTypeVariables #-}+ -- | -- Module      : Crypto.PubKey.RSA -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----module Crypto.PubKey.RSA-    ( Error(..)-    , PublicKey(..)-    , PrivateKey(..)-    , Blinder(..)+module Crypto.PubKey.RSA (+    Error (..),+    PublicKey (..),+    PrivateKey (..),+    Blinder (..),+     -- * Generation function-    , generateWith-    , generate-    , generateBlinder-    ) where+    generateWith,+    generate,+    generateBlinder,+) where -import Crypto.Random.Types-import Crypto.Number.ModArithmetic (inverse, inverseCoprimes)+import Crypto.Internal.ByteArray (ScrubbedBytes) import Crypto.Number.Generate (generateMax)+import Crypto.Number.ModArithmetic (+    expSafe,+    inverse,+    inverseCoprimes,+    inverseSafe,+ ) import Crypto.Number.Prime (generatePrime)+import Crypto.Number.Serialize (os2ip) import Crypto.PubKey.RSA.Types+import Crypto.Random.Types  {- -- some bad implementation will not serialize ASN.1 integer properly, leading@@ -52,39 +61,92 @@ -- -- * e=3 is popular as well, but proven to not be as secure for some cases. ---generateWith :: (Integer, Integer) -- ^ chosen distinct primes p and q-             -> Int                -- ^ size in bytes-             -> Integer            -- ^ RSA public exponent 'e'-             -> Maybe (PublicKey, PrivateKey)-generateWith (p,q) size e =-    case inverse e phi of+-- /WARNING:/ Making a key is not constant time, and cannot be: the search for+-- the two primes takes as long as it takes, and 'Crypto.Number.Prime' is not+-- constant time either.  What that leaks is about the search rather than+-- about the primes it settles on.  Of the arithmetic that does touch them,+-- the inverse of one prime modulo the other is worked out without a side+-- channel, and so is the private exponent, which is the inverse of @e@ modulo+-- @(p-1)*(q-1)@: @e@ being public lets that be worked out as a remainder, an+-- inverse modulo @e@ itself, and an exact division, none of which follows the+-- number being inverted.  An @e@ that is not prime keeps the extended+-- Euclidean algorithm, which for a public @e@ is one division by a small+-- number and then a few steps on numbers under it.+generateWith+    :: (Integer, Integer)+    -- ^ chosen distinct primes p and q+    -> Int+    -- ^ size in bytes+    -> Integer+    -- ^ RSA public exponent @e@+    -> Maybe (PublicKey, PrivateKey)+generateWith (p, q) size e =+    case privateExponent of         Nothing -> Nothing-        Just d  -> Just (pub,priv d)-  where n   = p*q-        phi = (p-1)*(q-1)-        -- q and p should be *distinct* *prime* numbers, hence always coprime-        qinv = inverseCoprimes q p-        pub = PublicKey { public_size = size-                        , public_n    = n-                        , public_e    = e-                        }-        priv d = PrivateKey { private_pub  = pub-                            , private_d    = d-                            , private_p    = p-                            , private_q    = q-                            , private_dP   = d `mod` (p-1)-                            , private_dQ   = d `mod` (q-1)-                            , private_qinv = qinv-                            }+        Just d -> Just (pub, priv d)+  where+    n = p * q+    phi = (p - 1) * (q - 1)+    -- The private exponent is the inverse of e modulo phi, and phi is the+    -- key.  The extended Euclidean algorithm would take a number of steps+    -- that follows it; e being public lets the work be about e instead.+    --+    -- Whatever d is, e * d = 1 + k * phi for some k under e, and reading that+    -- modulo e gives k = -phi^-1 mod e -- an inverse modulo a number of a+    -- handful of bits, which for a prime e is Fermat.  Then d is an exact+    -- division by e.  Nothing in that follows phi: the remainder and the+    -- division are one pass each over its limbs, and the rest is arithmetic+    -- the size of e.+    --+    -- Fermat wants a prime e, and rather than ask whether e is one -- which+    -- costs more than everything else here -- the k it gives is checked,+    -- which is arithmetic the size of e.  A composite e that fails the check+    -- keeps the algorithm it had.+    privateExponent+        | e <= 1 = Nothing+        | t == 0 = Nothing -- e divides phi, so there is no inverse+        | (k * t) `mod` e == e - 1 = Just ((1 + k * phi) `div` e)+        | otherwise = inverse e phi+      where+        t = phi `mod` e+        k = (e - expSafe t (e - 2) e) `mod` e+    -- q and p should be *distinct* *prime* numbers, hence always coprime.+    -- Both of them are the key itself, so the inverse is worked out through+    -- Fermat's little theorem rather than the extended Euclidean algorithm,+    -- whose steps follow the numbers it is given.  It falls back on the one+    -- that raises, which is what a p that is not prime deserves.+    qinv = case inverseSafe q p of+        Just i -> i+        Nothing -> inverseCoprimes q p+    pub =+        PublicKey+            { public_size = size+            , public_n = n+            , public_e = e+            }+    priv d =+        PrivateKey+            { private_pub = pub+            , private_d = d+            , private_p = p+            , private_q = q+            , private_dP = d `mod` (p - 1)+            , private_dQ = d `mod` (q - 1)+            , private_qinv = qinv+            }  -- | generate a pair of (private, public) key of size in bytes.-generate :: MonadRandom m-         => Int     -- ^ size in bytes-         -> Integer -- ^ RSA public exponent 'e'-         -> m (PublicKey, PrivateKey)+generate+    :: MonadRandom m+    => Int+    -- ^ size in bytes+    -> Integer+    -- ^ RSA public exponent @e@+    -> m (PublicKey, PrivateKey) generate size e = loop   where-    loop = do -- loop until we find a valid key pair given e+    loop = do+        -- loop until we find a valid key pair given e         pq <- generatePQ         case generateWith pq size e of             Nothing -> loop@@ -92,7 +154,7 @@     generatePQ = do         p <- generatePrime (8 * (size `div` 2))         q <- generateQ p-        return (p,q)+        return (p, q)     generateQ p = do         q <- generatePrime (8 * (size - (size `div` 2)))         if p == q then generateQ p else return q@@ -101,8 +163,30 @@ -- -- the unique parameter apart from the random number generator is the -- public key value N.-generateBlinder :: MonadRandom m-                => Integer -- ^ RSA public N parameter.-                -> m Blinder-generateBlinder n =-    (\r -> Blinder r (inverseCoprimes r n)) <$> generateMax n+--+-- The blinder holds a random number and its inverse.  N is composite, so+-- Fermat has no answer for the inverse and it goes through the extended+-- Euclidean algorithm, whose steps follow the number handed to it -- which+-- would be the number the blinding rests on.  So the algorithm is handed that+-- number multiplied by another random one instead, and its answer multiplied+-- by that number again, which leaves the inverse wanted and shows the+-- algorithm nothing that has anything to do with it.+generateBlinder+    :: forall m+     . MonadRandom m+    => Integer+    -- ^ RSA public N parameter.+    -> m Blinder+generateBlinder n = do+    r <- generateMax n+    -- The inverse goes through the extended Euclidean algorithm, whose steps+    -- follow the number handed to it, and r is what the blinding rests on.+    -- So another random number goes with it: the product is uniform and says+    -- nothing about r on its own, and multiplying its inverse by that number+    -- again leaves the inverse of r.  Sixteen bytes are enough to hide it and+    -- are under either prime, so the product is coprime with n whenever r is,+    -- as it was before.+    u <- os2ip <$> (getRandomBytes 16 :: m ScrubbedBytes)+    let v = (r * u) `mod` n+        rm1 = (inverseCoprimes v n * u) `mod` n+    return $ Blinder r rm1
Crypto/PubKey/RSA/OAEP.hs view
@@ -7,148 +7,222 @@ -- -- RSA OAEP mode -- <http://en.wikipedia.org/wiki/Optimal_asymmetric_encryption_padding>----module Crypto.PubKey.RSA.OAEP-    (-      OAEPParams(..)-    , defaultOAEPParams+module Crypto.PubKey.RSA.OAEP (+    OAEPParams (..),+    defaultOAEPParams,+     -- * OAEP encryption-    , encryptWithSeed-    , encrypt+    encryptWithSeed,+    encrypt,+     -- * OAEP decryption-    , decrypt-    , decryptSafer-    ) where+    decrypt,+    decryptSafer,+) where -import           Crypto.Hash-import           Crypto.Random.Types-import           Crypto.PubKey.RSA.Types-import           Crypto.PubKey.MaskGenFunction-import           Crypto.PubKey.RSA.Prim-import           Crypto.PubKey.RSA (generateBlinder)-import           Crypto.PubKey.Internal (and')-import           Data.ByteString (ByteString)+import Crypto.Hash+import Crypto.Number.Serialize (os2ip)+import Crypto.PubKey.Internal (and')+import Crypto.PubKey.MaskGenFunction+import Crypto.PubKey.RSA (generateBlinder)+import Crypto.PubKey.RSA.Prim+import Crypto.PubKey.RSA.Types+import Crypto.Random.Types+import Data.Bits (complement, shiftR, xor, (.&.), (.|.))+import Data.ByteString (ByteString) import qualified Data.ByteString as B-import           Data.Bits (xor)+import Data.List (foldl')+import Data.Word (Word32)+import Prelude hiding (foldl') -import           Crypto.Internal.ByteArray (ByteArrayAccess, ByteArray)-import qualified Crypto.Internal.ByteArray as B (convert)+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess)+import qualified Crypto.Internal.ByteArray as B (constEq, convert)  -- | Parameters for OAEP encryption/decryption data OAEPParams hash seed output = OAEPParams-    { oaepHash       :: hash                         -- ^ Hash function to use.-    , oaepMaskGenAlg :: MaskGenAlgorithm seed output -- ^ Mask Gen algorithm to use.-    , oaepLabel      :: Maybe ByteString             -- ^ Optional label prepended to message.+    { oaepHash :: hash+    -- ^ Hash function to use.+    , oaepMaskGenAlg :: MaskGenAlgorithm seed output+    -- ^ Mask Gen algorithm to use.+    , oaepLabel :: Maybe ByteString+    -- ^ Optional label prepended to message.     }  -- | Default Params with a specified hash function-defaultOAEPParams :: (ByteArrayAccess seed, ByteArray output, HashAlgorithm hash)-                  => hash-                  -> OAEPParams hash seed output+defaultOAEPParams+    :: (ByteArrayAccess seed, ByteArray output, HashAlgorithm hash)+    => hash+    -> OAEPParams hash seed output defaultOAEPParams hashAlg =-    OAEPParams { oaepHash         = hashAlg-               , oaepMaskGenAlg   = mgf1 hashAlg-               , oaepLabel        = Nothing-               }+    OAEPParams+        { oaepHash = hashAlg+        , oaepMaskGenAlg = mgf1 hashAlg+        , oaepLabel = Nothing+        }  -- | Encrypt a message using OAEP with a predefined seed.-encryptWithSeed :: HashAlgorithm hash-                => ByteString      -- ^ Seed-                -> OAEPParams hash ByteString ByteString -- ^ OAEP params to use for encryption-                -> PublicKey       -- ^ Public key.-                -> ByteString      -- ^ Message to encrypt-                -> Either Error ByteString+encryptWithSeed+    :: HashAlgorithm hash+    => ByteString+    -- ^ Seed+    -> OAEPParams hash ByteString ByteString+    -- ^ OAEP params to use for encryption+    -> PublicKey+    -- ^ Public key.+    -> ByteString+    -- ^ Message to encrypt+    -> Either Error ByteString encryptWithSeed seed oaep pk msg-    | k < 2*hashLen+2          = Left InvalidParameters+    | k < 2 * hashLen + 2 = Left InvalidParameters     | B.length seed /= hashLen = Left InvalidParameters-    | mLen > k - 2*hashLen-2   = Left MessageTooLong-    | otherwise                = Right $ ep pk em-    where -- parameters-          k          = public_size pk-          mLen       = B.length msg-          mgf        = oaepMaskGenAlg oaep-          labelHash  = hashWith (oaepHash oaep) (maybe B.empty id $ oaepLabel oaep)-          hashLen    = hashDigestSize (oaepHash oaep)+    | mLen > k - 2 * hashLen - 2 = Left MessageTooLong+    | otherwise = Right $ ep pk em+  where+    -- parameters+    k = public_size pk+    mLen = B.length msg+    mgf = oaepMaskGenAlg oaep+    labelHash = hashWith (oaepHash oaep) (maybe B.empty id $ oaepLabel oaep)+    hashLen = hashDigestSize (oaepHash oaep) -          -- put fields-          ps         = B.replicate (k - mLen - 2*hashLen - 2) 0-          db         = B.concat [B.convert labelHash, ps, B.singleton 0x1, msg]-          dbmask     = mgf seed (k - hashLen - 1)-          maskedDB   = B.pack $ B.zipWith xor db dbmask-          seedMask   = mgf maskedDB hashLen-          maskedSeed = B.pack $ B.zipWith xor seed seedMask-          em         = B.concat [B.singleton 0x0,maskedSeed,maskedDB]+    -- put fields+    ps = B.replicate (k - mLen - 2 * hashLen - 2) 0+    db = B.concat [B.convert labelHash, ps, B.singleton 0x1, msg]+    dbmask = mgf seed (k - hashLen - 1)+    maskedDB = B.pack $ B.zipWith xor db dbmask+    seedMask = mgf maskedDB hashLen+    maskedSeed = B.pack $ B.zipWith xor seed seedMask+    em = B.concat [B.singleton 0x0, maskedSeed, maskedDB]  -- | Encrypt a message using OAEP-encrypt :: (HashAlgorithm hash, MonadRandom m)-        => OAEPParams hash ByteString ByteString -- ^ OAEP params to use for encryption.-        -> PublicKey       -- ^ Public key.-        -> ByteString      -- ^ Message to encrypt-        -> m (Either Error ByteString)+encrypt+    :: (HashAlgorithm hash, MonadRandom m)+    => OAEPParams hash ByteString ByteString+    -- ^ OAEP params to use for encryption.+    -> PublicKey+    -- ^ Public key.+    -> ByteString+    -- ^ Message to encrypt+    -> m (Either Error ByteString) encrypt oaep pk msg = do     seed <- getRandomBytes hashLen     return (encryptWithSeed seed oaep pk msg)   where-    hashLen    = hashDigestSize (oaepHash oaep)+    hashLen = hashDigestSize (oaepHash oaep)  -- | un-pad a OAEP encoded message. -- -- It doesn't apply the RSA decryption primitive-unpad :: HashAlgorithm hash-      => OAEPParams hash ByteString ByteString -- ^ OAEP params to use-      -> Int             -- ^ size of the key in bytes-      -> ByteString      -- ^ encoded message (not encrypted)-      -> Either Error ByteString+--+-- The data block is scanned in full rather than up to the 01 octet separating+-- the padding from the message, and the label hash and the leading octet are+-- compared without an early exit, so neither the length of the padding nor+-- where a comparison first differs shows up in how long this takes.+--+-- What remains visible is the result itself: whether the block was well formed,+-- and the length of the message when it was.  That is the signal Manger's+-- attack needs, so a caller that decrypts attacker-supplied ciphertext must not+-- pass the distinction on.+unpad+    :: HashAlgorithm hash+    => OAEPParams hash ByteString ByteString+    -- ^ OAEP params to use+    -> Int+    -- ^ size of the key in bytes+    -> ByteString+    -- ^ encoded message (not encrypted)+    -> Either Error ByteString unpad oaep k em     | paddingSuccess = Right msg-    | otherwise      = Left MessageNotRecognized-    where -- parameters-          mgf        = oaepMaskGenAlg oaep-          labelHash  = B.convert $ hashWith (oaepHash oaep) (maybe B.empty id $ oaepLabel oaep)-          hashLen    = hashDigestSize (oaepHash oaep)-          -- getting em's fields-          (pb, em0)  = B.splitAt 1 em-          (maskedSeed,maskedDB) = B.splitAt hashLen em0-          seedMask   = mgf maskedDB hashLen-          seed       = B.pack $ B.zipWith xor maskedSeed seedMask-          dbmask     = mgf seed (k - hashLen - 1)-          db         = B.pack $ B.zipWith xor maskedDB dbmask-          -- getting db's fields-          (labelHash',db1) = B.splitAt hashLen db-          (_,db2)    = B.break (/= 0) db1-          (ps1,msg)  = B.splitAt 1 db2+    | otherwise = Left MessageNotRecognized+  where+    -- parameters+    mgf = oaepMaskGenAlg oaep+    labelHash =+        B.convert $ hashWith (oaepHash oaep) (maybe B.empty id $ oaepLabel oaep)+            :: ByteString+    hashLen = hashDigestSize (oaepHash oaep)+    -- getting em's fields+    (pb, em0) = B.splitAt 1 em+    (maskedSeed, maskedDB) = B.splitAt hashLen em0+    seedMask = mgf maskedDB hashLen+    seed = B.pack $ B.zipWith xor maskedSeed seedMask+    dbmask = mgf seed (k - hashLen - 1)+    db = B.pack $ B.zipWith xor maskedDB dbmask+    -- getting db's fields+    (labelHash', db1) = B.splitAt hashLen db -          paddingSuccess = and' [ labelHash' == labelHash -- no need for constant eq-                                , ps1        == B.replicate 1 0x1-                                , pb         == B.replicate 1 0x0-                                ]+    -- index of the first nonzero octet in db1, or its length when every octet+    -- is zero; all of them are looked at either way+    oneIndex =+        fst $+            foldl'+                step+                (fromIntegral (B.length db1) :: Word32, 1 :: Word32)+                (zip [0 ..] (B.unpack db1))+    step (idx, unseen) (i, b) = (select found i idx, unseen .&. complement found)+      where+        w = fromIntegral b :: Word32+        -- 0 when b is zero, 1 otherwise+        nonZero = (w .|. negate w) `shiftR` 31+        -- all ones at the first nonzero octet only+        found = negate (unseen .&. nonZero)+    select mask a b = (a .&. mask) .|. (b .&. complement mask) +    ps1 = B.take 1 $ B.drop (fromIntegral oneIndex) db1+    msg = B.drop (fromIntegral oneIndex + 1) db1++    paddingSuccess =+        and'+            [ labelHash' `B.constEq` labelHash+            , ps1 `B.constEq` B.replicate 1 0x1+            , pb `B.constEq` B.replicate 1 0x0+            ]+ -- | Decrypt a ciphertext using OAEP ----- When the signature is not in a context where an attacker could gain--- information from the timing of the operation, the blinder can be set to None.+-- The blinder is optional and 'Nothing' is accepted, but see t'Blinder' for+-- what it covers and when leaving it out is a decision rather than a default.+-- 'decryptSafer' generates one for you. ----- If unsure always set a blinder or use decryptSafer-decrypt :: HashAlgorithm hash-        => Maybe Blinder   -- ^ Optional blinder-        -> OAEPParams hash ByteString ByteString -- ^ OAEP params to use for decryption-        -> PrivateKey      -- ^ Private key-        -> ByteString      -- ^ Cipher text-        -> Either Error ByteString+-- Following RFC 8017, the ciphertext is rejected unless it is exactly as long+-- as the modulus (section 7.1.2, step 1) and its integer representative is+-- below the modulus (RSADP, section 5.1.2, step 1).  The decryption primitive+-- normalises any multiple of the modulus away, so without the second check+-- @c + n@ would decrypt to the same message as @c@, and a ciphertext would not+-- be unique to its plaintext.  Both checks are made on the ciphertext alone,+-- which is public, and report 'MessageSizeIncorrect'.+decrypt+    :: HashAlgorithm hash+    => Maybe Blinder+    -- ^ Optional blinder+    -> OAEPParams hash ByteString ByteString+    -- ^ OAEP params to use for decryption+    -> PrivateKey+    -- ^ Private key+    -> ByteString+    -- ^ Cipher text+    -> Either Error ByteString decrypt blinder oaep pk cipher     | B.length cipher /= k = Left MessageSizeIncorrect-    | k < 2*hashLen+2      = Left InvalidParameters-    | otherwise            = unpad oaep (private_size pk) $ dp blinder pk cipher-    where -- parameters-          k          = private_size pk-          hashLen    = hashDigestSize (oaepHash oaep)+    | os2ip cipher >= private_n pk = Left MessageSizeIncorrect+    | k < 2 * hashLen + 2 = Left InvalidParameters+    | otherwise = unpad oaep (private_size pk) $ dp blinder pk cipher+  where+    -- parameters+    k = private_size pk+    hashLen = hashDigestSize (oaepHash oaep)  -- | Decrypt a ciphertext using OAEP and by automatically generating a blinder.-decryptSafer :: (HashAlgorithm hash, MonadRandom m)-             => OAEPParams hash ByteString ByteString -- ^ OAEP params to use for decryption-             -> PrivateKey -- ^ Private key-             -> ByteString -- ^ Cipher text-             -> m (Either Error ByteString)+decryptSafer+    :: (HashAlgorithm hash, MonadRandom m)+    => OAEPParams hash ByteString ByteString+    -- ^ OAEP params to use for decryption+    -> PrivateKey+    -- ^ Private key+    -> ByteString+    -- ^ Cipher text+    -> m (Either Error ByteString) decryptSafer oaep pk cipher = do     blinder <- generateBlinder (private_n pk)     return (decrypt (Just blinder) oaep pk cipher)
Crypto/PubKey/RSA/PKCS15.hs view
@@ -4,37 +4,42 @@ -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----module Crypto.PubKey.RSA.PKCS15-    (+module Crypto.PubKey.RSA.PKCS15 (     -- * Padding and unpadding-      pad-    , padSignature-    , unpad+    pad,+    padSignature,+    unpad,+     -- * Private key operations-    , decrypt-    , decryptSafer-    , sign-    , signSafer+    decrypt,+    decryptSafer,+    sign,+    signSafer,+     -- * Public key operations-    , encrypt-    , verify+    encrypt,+    verify,+     -- * Hash ASN1 description-    , HashAlgorithmASN1-    ) where+    HashAlgorithmASN1,+) where -import           Crypto.Random.Types-import           Crypto.PubKey.Internal (and')-import           Crypto.PubKey.RSA.Types-import           Crypto.PubKey.RSA.Prim-import           Crypto.PubKey.RSA (generateBlinder)-import           Crypto.Hash+import Crypto.Hash+import Crypto.Number.Serialize (os2ip)+import Crypto.PubKey.Internal (and')+import Crypto.PubKey.RSA (generateBlinder)+import Crypto.PubKey.RSA.Prim+import Crypto.PubKey.RSA.Types+import Crypto.Random.Types -import           Data.ByteString (ByteString)-import           Data.Word+import Data.Bits (complement, shiftR, (.&.), (.|.))+import Data.ByteString (ByteString)+import Data.Word -import           Crypto.Internal.ByteArray (ByteArray, Bytes)+import Crypto.Internal.ByteArray (ByteArray, Bytes) import qualified Crypto.Internal.ByteArray as B+import Data.List (foldl')+import Prelude hiding (foldl')  -- | A specialized class for hash algorithm that can product -- a ASN1 wrapped description the algorithm plus the content@@ -46,28 +51,314 @@ -- http://uk.emc.com/emc-plus/rsa-labs/pkcs/files/h11300-wp-pkcs-1v2-2-rsa-cryptography-standard.pdf -- EMSA-PKCS1-v1_5 instance HashAlgorithmASN1 MD2 where-    hashDigestASN1 = addDigestPrefix [0x30,0x20,0x30,0x0c,0x06,0x08,0x2a,0x86,0x48,0x86,0xf7,0x0d,0x02,0x02,0x05,0x00,0x04,0x10]+    hashDigestASN1 =+        addDigestPrefix+            [ 0x30+            , 0x20+            , 0x30+            , 0x0c+            , 0x06+            , 0x08+            , 0x2a+            , 0x86+            , 0x48+            , 0x86+            , 0xf7+            , 0x0d+            , 0x02+            , 0x02+            , 0x05+            , 0x00+            , 0x04+            , 0x10+            ] instance HashAlgorithmASN1 MD5 where-    hashDigestASN1 = addDigestPrefix [0x30,0x20,0x30,0x0c,0x06,0x08,0x2a,0x86,0x48,0x86,0xf7,0x0d,0x02,0x05,0x05,0x00,0x04,0x10]+    hashDigestASN1 =+        addDigestPrefix+            [ 0x30+            , 0x20+            , 0x30+            , 0x0c+            , 0x06+            , 0x08+            , 0x2a+            , 0x86+            , 0x48+            , 0x86+            , 0xf7+            , 0x0d+            , 0x02+            , 0x05+            , 0x05+            , 0x00+            , 0x04+            , 0x10+            ] instance HashAlgorithmASN1 SHA1 where-    hashDigestASN1 = addDigestPrefix [0x30,0x21,0x30,0x09,0x06,0x05,0x2b,0x0e,0x03,0x02,0x1a,0x05,0x00,0x04,0x14]+    hashDigestASN1 =+        addDigestPrefix+            [ 0x30+            , 0x21+            , 0x30+            , 0x09+            , 0x06+            , 0x05+            , 0x2b+            , 0x0e+            , 0x03+            , 0x02+            , 0x1a+            , 0x05+            , 0x00+            , 0x04+            , 0x14+            ] instance HashAlgorithmASN1 SHA224 where-    hashDigestASN1 = addDigestPrefix [0x30,0x2d,0x30,0x0d,0x06,0x09,0x60,0x86,0x48,0x01,0x65,0x03,0x04,0x02,0x04,0x05,0x00,0x04,0x1c]+    hashDigestASN1 =+        addDigestPrefix+            [ 0x30+            , 0x2d+            , 0x30+            , 0x0d+            , 0x06+            , 0x09+            , 0x60+            , 0x86+            , 0x48+            , 0x01+            , 0x65+            , 0x03+            , 0x04+            , 0x02+            , 0x04+            , 0x05+            , 0x00+            , 0x04+            , 0x1c+            ] instance HashAlgorithmASN1 SHA256 where-    hashDigestASN1 = addDigestPrefix [0x30,0x31,0x30,0x0d,0x06,0x09,0x60,0x86,0x48,0x01,0x65,0x03,0x04,0x02,0x01,0x05,0x00,0x04,0x20]+    hashDigestASN1 =+        addDigestPrefix+            [ 0x30+            , 0x31+            , 0x30+            , 0x0d+            , 0x06+            , 0x09+            , 0x60+            , 0x86+            , 0x48+            , 0x01+            , 0x65+            , 0x03+            , 0x04+            , 0x02+            , 0x01+            , 0x05+            , 0x00+            , 0x04+            , 0x20+            ] instance HashAlgorithmASN1 SHA384 where-    hashDigestASN1 = addDigestPrefix [0x30,0x41,0x30,0x0d,0x06,0x09,0x60,0x86,0x48,0x01,0x65,0x03,0x04,0x02,0x02,0x05,0x00,0x04,0x30]+    hashDigestASN1 =+        addDigestPrefix+            [ 0x30+            , 0x41+            , 0x30+            , 0x0d+            , 0x06+            , 0x09+            , 0x60+            , 0x86+            , 0x48+            , 0x01+            , 0x65+            , 0x03+            , 0x04+            , 0x02+            , 0x02+            , 0x05+            , 0x00+            , 0x04+            , 0x30+            ] instance HashAlgorithmASN1 SHA512 where-    hashDigestASN1 = addDigestPrefix [0x30,0x51,0x30,0x0d,0x06,0x09,0x60,0x86,0x48,0x01,0x65,0x03,0x04,0x02,0x03,0x05,0x00,0x04,0x40]+    hashDigestASN1 =+        addDigestPrefix+            [ 0x30+            , 0x51+            , 0x30+            , 0x0d+            , 0x06+            , 0x09+            , 0x60+            , 0x86+            , 0x48+            , 0x01+            , 0x65+            , 0x03+            , 0x04+            , 0x02+            , 0x03+            , 0x05+            , 0x00+            , 0x04+            , 0x40+            ] instance HashAlgorithmASN1 SHA512t_224 where-    hashDigestASN1 = addDigestPrefix [0x30,0x2d,0x30,0x0d,0x06,0x09,0x60,0x86,0x48,0x01,0x65,0x03,0x04,0x02,0x05,0x05,0x00,0x04,0x1c]+    hashDigestASN1 =+        addDigestPrefix+            [ 0x30+            , 0x2d+            , 0x30+            , 0x0d+            , 0x06+            , 0x09+            , 0x60+            , 0x86+            , 0x48+            , 0x01+            , 0x65+            , 0x03+            , 0x04+            , 0x02+            , 0x05+            , 0x05+            , 0x00+            , 0x04+            , 0x1c+            ] instance HashAlgorithmASN1 SHA512t_256 where-    hashDigestASN1 = addDigestPrefix [0x30,0x31,0x30,0x0d,0x06,0x09,0x60,0x86,0x48,0x01,0x65,0x03,0x04,0x02,0x06,0x05,0x00,0x04,0x20]+    hashDigestASN1 =+        addDigestPrefix+            [ 0x30+            , 0x31+            , 0x30+            , 0x0d+            , 0x06+            , 0x09+            , 0x60+            , 0x86+            , 0x48+            , 0x01+            , 0x65+            , 0x03+            , 0x04+            , 0x02+            , 0x06+            , 0x05+            , 0x00+            , 0x04+            , 0x20+            ]+instance HashAlgorithmASN1 SHA3_224 where+    hashDigestASN1 =+        addDigestPrefix+            [ 0x30+            , 0x2b+            , 0x30+            , 0x0b+            , 0x06+            , 0x09+            , 0x60+            , 0x86+            , 0x48+            , 0x01+            , 0x65+            , 0x03+            , 0x04+            , 0x02+            , 0x07+            , 0x04+            , 0x1c+            ]+instance HashAlgorithmASN1 SHA3_256 where+    hashDigestASN1 =+        addDigestPrefix+            [ 0x30+            , 0x2f+            , 0x30+            , 0x0b+            , 0x06+            , 0x09+            , 0x60+            , 0x86+            , 0x48+            , 0x01+            , 0x65+            , 0x03+            , 0x04+            , 0x02+            , 0x08+            , 0x04+            , 0x20+            ]+instance HashAlgorithmASN1 SHA3_384 where+    hashDigestASN1 =+        addDigestPrefix+            [ 0x30+            , 0x3f+            , 0x30+            , 0x0b+            , 0x06+            , 0x09+            , 0x60+            , 0x86+            , 0x48+            , 0x01+            , 0x65+            , 0x03+            , 0x04+            , 0x02+            , 0x09+            , 0x04+            , 0x30+            ]+instance HashAlgorithmASN1 SHA3_512 where+    hashDigestASN1 =+        addDigestPrefix+            [ 0x30+            , 0x4f+            , 0x30+            , 0x0b+            , 0x06+            , 0x09+            , 0x60+            , 0x86+            , 0x48+            , 0x01+            , 0x65+            , 0x03+            , 0x04+            , 0x02+            , 0x0a+            , 0x04+            , 0x40+            ] instance HashAlgorithmASN1 RIPEMD160 where-    hashDigestASN1 = addDigestPrefix [0x30,0x21,0x30,0x09,0x06,0x05,0x2b,0x24,0x03,0x02,0x01,0x05,0x00,0x04,0x14]+    hashDigestASN1 =+        addDigestPrefix+            [ 0x30+            , 0x21+            , 0x30+            , 0x09+            , 0x06+            , 0x05+            , 0x2b+            , 0x24+            , 0x03+            , 0x02+            , 0x01+            , 0x05+            , 0x00+            , 0x04+            , 0x14+            ]  --+ -- ** Hack **+ -- -- this happens to not need a real ASN1 encoder, because -- thanks to the digest being a specific size AND@@ -80,7 +371,7 @@ --   Start Sequence --     ,Start Sequence --       ,OID oid---       ,Null+--       ,optional parameters (Null for SHA-2, absent for SHA-3) --     ,End Sequence --     ,OctetString digest --   ,End Sequence@@ -89,69 +380,115 @@     B.pack prefix `B.append` B.convert digest  -- | This produce a standard PKCS1.5 padding for encryption-pad :: (MonadRandom m, ByteArray message) => Int -> message -> m (Either Error message)+pad+    :: (MonadRandom m, ByteArray message) => Int -> message -> m (Either Error message) pad len m     | B.length m > len - 11 = return (Left MessageTooLong)-    | otherwise             = do+    | otherwise = do         padding <- getNonNullRandom (len - B.length m - 3)-        return $ Right $ B.concat [ B.pack [0,2], padding, B.pack [0], m ]-+        return $ Right $ B.concat [B.pack [0, 2], padding, B.pack [0], m]   where     -- get random non-null bytes     getNonNullRandom :: (ByteArray bytearray, MonadRandom m) => Int -> m bytearray     getNonNullRandom n = do         bs0 <- getRandomBytes n         let bytes = B.pack $ filter (/= 0) $ B.unpack (bs0 :: Bytes)-            left  = n - B.length bytes+            left = n - B.length bytes         if left == 0             then return bytes-            else do bend <- getNonNullRandom left-                    return (bytes `B.append` bend)+            else do+                bend <- getNonNullRandom left+                return (bytes `B.append` bend)  -- | Produce a standard PKCS1.5 padding for signature-padSignature :: ByteArray signature => Int -> signature -> Either Error signature+padSignature+    :: ByteArray signature => Int -> signature -> Either Error signature padSignature klen signature     | klen < siglen + 11 = Left SignatureTooLong-    | otherwise          = Right (B.pack padding `B.append` signature)+    | otherwise = Right (B.pack padding `B.append` signature)   where-        siglen    = B.length signature-        padding   = 0 : 1 : (replicate (klen - siglen - 3) 0xff ++ [0])+    siglen = B.length signature+    padding = 0 : 1 : (replicate (klen - siglen - 3) 0xff ++ [0])  -- | Try to remove a standard PKCS1.5 encryption padding.+--+-- The block is scanned in full rather than up to the octet ending the padding+-- string, so how long that string is does not show up in how long this takes.+--+-- What remains visible is the result itself: whether the padding was well+-- formed, and the length of the message when it was.  That is inherent to the+-- scheme, and it is the signal Bleichenbacher's attack needs, so a caller that+-- decrypts attacker-supplied ciphertext must not pass the distinction on --+-- TLS, for instance, continues with a random premaster secret and reports+-- nothing. unpad :: ByteArray bytearray => bytearray -> Either Error bytearray unpad packed     | paddingSuccess = Right m-    | otherwise      = Left MessageNotRecognized+    | otherwise = Left MessageNotRecognized   where-        (zt, ps0m)   = B.splitAt 2 packed-        (ps, zm)     = B.span (/= 0) ps0m-        (z, m)       = B.splitAt 1 zm-        paddingSuccess = and' [ zt `B.constEq` (B.pack [0,2] :: Bytes)-                              , z == B.zero 1-                              , B.length ps >= 8-                              ]+    len = B.length packed+    (zt, ps0m) = B.splitAt 2 packed +    -- index of the first zero octet in ps0m, counted from the start of packed,+    -- or len when there is none; every octet is looked at either way+    zeroIndex = fst $ foldl' step (fromIntegral len :: Word32, 1 :: Word32) indexed+    indexed = zip [2 ..] (B.unpack ps0m)+    step (idx, unseen) (i, b) = (select found i idx, unseen .&. complement found)+      where+        w = fromIntegral b :: Word32+        -- 0 when b is zero, 1 otherwise+        nonZero = (w .|. negate w) `shiftR` 31+        -- all ones at the first zero octet only+        found = negate (unseen .&. complement nonZero)+    select mask a b = (a .&. mask) .|. (b .&. complement mask)++    psLength = fromIntegral zeroIndex - 2 :: Int+    m = B.drop (fromIntegral zeroIndex + 1) packed+    paddingSuccess =+        and'+            [ zt `B.constEq` (B.pack [0, 2] :: Bytes)+            , fromIntegral zeroIndex < len+            , psLength >= 8+            ]+ -- | decrypt message using the private key. ----- When the decryption is not in a context where an attacker could gain--- information from the timing of the operation, the blinder can be set to None.------ If unsure always set a blinder or use decryptSafer+-- The blinder is optional and 'Nothing' is accepted, but see t'Blinder' for+-- what it covers and when leaving it out is a decision rather than a default.+-- 'decryptSafer' generates one for you. -- -- The message is returned un-padded.-decrypt :: Maybe Blinder -- ^ optional blinder-        -> PrivateKey    -- ^ RSA private key-        -> ByteString    -- ^ cipher text-        -> Either Error ByteString+--+-- Following RFC 8017, the ciphertext is rejected unless it is exactly as long+-- as the modulus (section 7.2.2, step 1) and its integer representative is+-- below the modulus (RSADP, section 5.1.2, step 1).  The decryption primitive+-- normalises any multiple of the modulus away, so without the second check+-- @c + n@ would decrypt to the same message as @c@, and a ciphertext would not+-- be unique to its plaintext.  Both checks are made on the ciphertext alone,+-- which is public, and report 'MessageSizeIncorrect'.+decrypt+    :: ByteArray ba+    => Maybe Blinder+    -- ^ optional blinder+    -> PrivateKey+    -- ^ RSA private key+    -> ByteString+    -- ^ cipher text+    -> Either Error ba decrypt blinder pk c     | B.length c /= (private_size pk) = Left MessageSizeIncorrect-    | otherwise                       = unpad $ dp blinder pk c+    | os2ip c >= private_n pk = Left MessageSizeIncorrect+    -- "convert" must be apply to "c".+    | otherwise = unpad $ dp blinder pk $ B.convert c  -- | decrypt message using the private key and by automatically generating a blinder.-decryptSafer :: MonadRandom m-             => PrivateKey -- ^ RSA private key-             -> ByteString -- ^ cipher text-             -> m (Either Error ByteString)+decryptSafer+    :: (MonadRandom m, ByteArray ba)+    => PrivateKey+    -- ^ RSA private key+    -> ByteString+    -- ^ cipher text+    -> m (Either Error ba) decryptSafer pk b = do     blinder <- generateBlinder (private_n pk)     return (decrypt (Just blinder) pk b)@@ -160,54 +497,79 @@ -- -- The message needs to be smaller than the key size - 11. -- The message should not be padded.-encrypt :: MonadRandom m => PublicKey -> ByteString -> m (Either Error ByteString)+encrypt+    :: (MonadRandom m, ByteArray ba) => PublicKey -> ba -> m (Either Error ByteString) encrypt pk m = do     r <- pad (public_size pk) m     case r of         Left err -> return $ Left err-        Right em -> return $ Right (ep pk em)+        Right em -> return $ Right (B.convert $ ep pk em)  -- | sign message using private key, a hash and its ASN1 description ----- When the signature is not in a context where an attacker could gain--- information from the timing of the operation, the blinder can be set to None.------ If unsure always set a blinder or use signSafer-sign :: HashAlgorithmASN1 hashAlg-     => Maybe Blinder -- ^ optional blinder-     -> Maybe hashAlg -- ^ hash algorithm-     -> PrivateKey    -- ^ private key-     -> ByteString    -- ^ message to sign-     -> Either Error ByteString+-- The blinder is optional and 'Nothing' is accepted, but see t'Blinder' for+-- what it covers and when leaving it out is a decision rather than a default.+-- 'signSafer' generates one for you.+sign+    :: HashAlgorithmASN1 hashAlg+    => Maybe Blinder+    -- ^ optional blinder+    -> Maybe hashAlg+    -- ^ hash algorithm+    -> PrivateKey+    -- ^ private key+    -> ByteString+    -- ^ message to sign+    -> Either Error ByteString sign blinder hashDescr pk m = dp blinder pk `fmap` makeSignature hashDescr (private_size pk) m  -- | sign message using the private key and by automatically generating a blinder.-signSafer :: (HashAlgorithmASN1 hashAlg, MonadRandom m)-          => Maybe hashAlg -- ^ Hash algorithm-          -> PrivateKey    -- ^ private key-          -> ByteString    -- ^ message to sign-          -> m (Either Error ByteString)+signSafer+    :: (HashAlgorithmASN1 hashAlg, MonadRandom m)+    => Maybe hashAlg+    -- ^ Hash algorithm+    -> PrivateKey+    -- ^ private key+    -> ByteString+    -- ^ message to sign+    -> m (Either Error ByteString) signSafer hashAlg pk m = do     blinder <- generateBlinder (private_n pk)     return (sign (Just blinder) hashAlg pk m)  -- | verify message with the signed message-verify :: HashAlgorithmASN1 hashAlg-       => Maybe hashAlg-       -> PublicKey-       -> ByteString-       -> ByteString-       -> Bool-verify hashAlg pk m sm =-    case makeSignature hashAlg (public_size pk) m of-        Left _  -> False-        Right s -> s == (ep pk sm)+--+-- Following RFC 8017, the signature is rejected unless it is exactly as long+-- as the modulus (section 8.2.2, step 1) and its integer representative is+-- below the modulus (section 5.2.2, step 1).  Verification works by+-- re-encoding the expected signature and comparing it with the result of the+-- public-key operation, and that operation normalises away both the length of+-- the encoding and any multiple of the modulus; without these checks a+-- zero-padded signature, or @s + n@, would verify just as well as @s@.+verify+    :: HashAlgorithmASN1 hashAlg+    => Maybe hashAlg+    -> PublicKey+    -> ByteString+    -- ^ Message+    -> ByteString+    -- ^ Signature+    -> Bool+verify hashAlg pk m sm+    | B.length sm /= public_size pk = False+    | os2ip sm >= public_n pk = False+    | otherwise =+        case makeSignature hashAlg (public_size pk) m of+            Left _ -> False+            Right s -> s == (ep pk sm)  -- | make signature digest, used in 'sign' and 'verify'-makeSignature :: HashAlgorithmASN1 hashAlg-              => Maybe hashAlg -- ^ optional hashing algorithm-              -> Int-              -> ByteString-              -> Either Error ByteString-makeSignature Nothing        klen m = padSignature klen m+makeSignature+    :: HashAlgorithmASN1 hashAlg+    => Maybe hashAlg+    -- ^ optional hashing algorithm+    -> Int+    -> ByteString+    -> Either Error ByteString+makeSignature Nothing klen m = padSignature klen m makeSignature (Just hashAlg) klen m = padSignature klen (hashDigestASN1 $ hashWith hashAlg m)
Crypto/PubKey/RSA/PSS.hs view
@@ -4,56 +4,63 @@ -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----module Crypto.PubKey.RSA.PSS-    ( PSSParams(..)-    , defaultPSSParams-    , defaultPSSParamsSHA1+module Crypto.PubKey.RSA.PSS (+    PSSParams (..),+    defaultPSSParams,+    defaultPSSParamsSHA1,+     -- * Sign and verify functions-    , signWithSalt-    , signDigestWithSalt-    , sign-    , signDigest-    , signSafer-    , signDigestSafer-    , verify-    , verifyDigest-    ) where+    signWithSalt,+    signDigestWithSalt,+    sign,+    signDigest,+    signSafer,+    signDigestSafer,+    verify,+    verifyDigest,+) where -import           Crypto.Random.Types-import           Crypto.PubKey.RSA.Types-import           Crypto.PubKey.RSA.Prim-import           Crypto.PubKey.RSA (generateBlinder)-import           Crypto.PubKey.MaskGenFunction-import           Crypto.Hash-import           Crypto.Number.Basic (numBits)-import           Data.Bits (xor, shiftR, (.&.))-import           Data.Word+import Crypto.Hash+import Crypto.Number.Basic (numBits)+import Crypto.Number.Serialize (os2ip)+import Crypto.PubKey.MaskGenFunction+import Crypto.PubKey.RSA (generateBlinder)+import Crypto.PubKey.RSA.Prim+import Crypto.PubKey.RSA.Types+import Crypto.Random.Types+import Data.Bits (complement, shiftR, xor, (.&.))+import Data.Word -import           Crypto.Internal.ByteArray (ByteArrayAccess, ByteArray)+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess) import qualified Crypto.Internal.ByteArray as B (convert, eq) -import           Data.ByteString (ByteString)+import Data.ByteString (ByteString) import qualified Data.ByteString as B  -- | Parameters for PSS signature/verification. data PSSParams hash seed output = PSSParams-    { pssHash         :: hash             -- ^ Hash function to use-    , pssMaskGenAlg   :: MaskGenAlgorithm seed output -- ^ Mask Gen algorithm to use-    , pssSaltLength   :: Int              -- ^ Length of salt. need to be <= to hLen.-    , pssTrailerField :: Word8            -- ^ Trailer field, usually 0xbc+    { pssHash :: hash+    -- ^ Hash function to use+    , pssMaskGenAlg :: MaskGenAlgorithm seed output+    -- ^ Mask Gen algorithm to use+    , pssSaltLength :: Int+    -- ^ Length of salt. need to be <= to hLen.+    , pssTrailerField :: Word8+    -- ^ Trailer field, usually 0xbc     }  -- | Default Params with a specified hash function-defaultPSSParams :: (ByteArrayAccess seed, ByteArray output, HashAlgorithm hash)-                 => hash-                 -> PSSParams hash seed output+defaultPSSParams+    :: (ByteArrayAccess seed, ByteArray output, HashAlgorithm hash)+    => hash+    -> PSSParams hash seed output defaultPSSParams hashAlg =-    PSSParams { pssHash         = hashAlg-              , pssMaskGenAlg   = mgf1 hashAlg-              , pssSaltLength   = hashDigestSize hashAlg-              , pssTrailerField = 0xbc-              }+    PSSParams+        { pssHash = hashAlg+        , pssMaskGenAlg = mgf1 hashAlg+        , pssSaltLength = hashDigestSize hashAlg+        , pssTrailerField = 0xbc+        }  -- | Default Params using SHA1 algorithm. defaultPSSParamsSHA1 :: PSSParams SHA1 ByteString ByteString@@ -62,138 +69,214 @@ -- | Sign using the PSS parameters and the salt explicitely passed as parameters. -- -- the function ignore SaltLength from the PSS Parameters-signDigestWithSalt :: HashAlgorithm hash-                   => ByteString    -- ^ Salt to use-                   -> Maybe Blinder -- ^ optional blinder to use-                   -> PSSParams hash ByteString ByteString -- ^ PSS Parameters to use-                   -> PrivateKey    -- ^ RSA Private Key-                   -> Digest hash   -- ^ Message digest-                   -> Either Error ByteString+--+-- See t'Blinder' for what the optional blinder covers and when leaving it out+-- is a decision rather than a default.  'signSafer' generates one for you.+signDigestWithSalt+    :: HashAlgorithm hash+    => ByteString+    -- ^ Salt to use+    -> Maybe Blinder+    -- ^ optional blinder to use+    -> PSSParams hash ByteString ByteString+    -- ^ PSS Parameters to use+    -> PrivateKey+    -- ^ RSA Private Key+    -> Digest hash+    -- ^ Message digest+    -> Either Error ByteString signDigestWithSalt salt blinder params pk digest     | emLen < hashLen + saltLen + 2 = Left InvalidParameters-    | otherwise                     = Right $ dp blinder pk em-    where k        = private_size pk-          emLen    = if emTruncate pubBits then k - 1 else k-          mHash    = B.convert digest-          dbLen    = emLen - hashLen - 1-          saltLen  = B.length salt-          hashLen  = hashDigestSize (pssHash params)-          pubBits  = numBits (private_n pk)-          m'       = B.concat [B.replicate 8 0,mHash,salt]-          h        = B.convert $ hashWith (pssHash params) m'-          db       = B.concat [B.replicate (dbLen - saltLen - 1) 0,B.singleton 1,salt]-          dbmask   = pssMaskGenAlg params h dbLen-          maskedDB = B.pack $ normalizeToKeySize pubBits $ B.zipWith xor db dbmask-          em       = B.concat [maskedDB, h, B.singleton (pssTrailerField params)]+    | otherwise = Right $ dp blinder pk em+  where+    k = private_size pk+    emLen = if emTruncate pubBits then k - 1 else k+    mHash = B.convert digest+    dbLen = emLen - hashLen - 1+    saltLen = B.length salt+    hashLen = hashDigestSize (pssHash params)+    pubBits = numBits (private_n pk)+    m' = B.concat [B.replicate 8 0, mHash, salt]+    h = B.convert $ hashWith (pssHash params) m'+    db = B.concat [B.replicate (dbLen - saltLen - 1) 0, B.singleton 1, salt]+    dbmask = pssMaskGenAlg params h dbLen+    maskedDB = B.pack $ normalizeToKeySize pubBits $ B.zipWith xor db dbmask+    em = B.concat [maskedDB, h, B.singleton (pssTrailerField params)]  -- | Sign using the PSS parameters and the salt explicitely passed as parameters. -- -- the function ignore SaltLength from the PSS Parameters-signWithSalt :: HashAlgorithm hash-             => ByteString    -- ^ Salt to use-             -> Maybe Blinder -- ^ optional blinder to use-             -> PSSParams hash ByteString ByteString -- ^ PSS Parameters to use-             -> PrivateKey    -- ^ RSA Private Key-             -> ByteString    -- ^ Message to sign-             -> Either Error ByteString+--+-- See t'Blinder' for what the optional blinder covers and when leaving it out+-- is a decision rather than a default.  'signSafer' generates one for you.+signWithSalt+    :: HashAlgorithm hash+    => ByteString+    -- ^ Salt to use+    -> Maybe Blinder+    -- ^ optional blinder to use+    -> PSSParams hash ByteString ByteString+    -- ^ PSS Parameters to use+    -> PrivateKey+    -- ^ RSA Private Key+    -> ByteString+    -- ^ Message to sign+    -> Either Error ByteString signWithSalt salt blinder params pk m = signDigestWithSalt salt blinder params pk mHash-    where mHash    = hashWith (pssHash params) m+  where+    mHash = hashWith (pssHash params) m  -- | Sign using the PSS Parameters-sign :: (HashAlgorithm hash, MonadRandom m)-     => Maybe Blinder   -- ^ optional blinder to use-     -> PSSParams hash ByteString ByteString -- ^ PSS Parameters to use-     -> PrivateKey      -- ^ RSA Private Key-     -> ByteString      -- ^ Message to sign-     -> m (Either Error ByteString)+--+-- See t'Blinder' for what the optional blinder covers and when leaving it out+-- is a decision rather than a default.  'signSafer' generates one for you.+sign+    :: (HashAlgorithm hash, MonadRandom m)+    => Maybe Blinder+    -- ^ optional blinder to use+    -> PSSParams hash ByteString ByteString+    -- ^ PSS Parameters to use+    -> PrivateKey+    -- ^ RSA Private Key+    -> ByteString+    -- ^ Message to sign+    -> m (Either Error ByteString) sign blinder params pk m = do     salt <- getRandomBytes (pssSaltLength params)     return (signWithSalt salt blinder params pk m)  -- | Sign using the PSS Parameters-signDigest :: (HashAlgorithm hash, MonadRandom m)-           => Maybe Blinder   -- ^ optional blinder to use-           -> PSSParams hash ByteString ByteString -- ^ PSS Parameters to use-           -> PrivateKey      -- ^ RSA Private Key-           -> Digest hash     -- ^ Message digest-           -> m (Either Error ByteString)+--+-- See t'Blinder' for what the optional blinder covers and when leaving it out+-- is a decision rather than a default.  'signSafer' generates one for you.+signDigest+    :: (HashAlgorithm hash, MonadRandom m)+    => Maybe Blinder+    -- ^ optional blinder to use+    -> PSSParams hash ByteString ByteString+    -- ^ PSS Parameters to use+    -> PrivateKey+    -- ^ RSA Private Key+    -> Digest hash+    -- ^ Message digest+    -> m (Either Error ByteString) signDigest blinder params pk digest = do     salt <- getRandomBytes (pssSaltLength params)     return (signDigestWithSalt salt blinder params pk digest)  -- | Sign using the PSS Parameters and an automatically generated blinder.-signSafer :: (HashAlgorithm hash, MonadRandom m)-          => PSSParams hash ByteString ByteString -- ^ PSS Parameters to use-          -> PrivateKey     -- ^ private key-          -> ByteString     -- ^ message to sign-          -> m (Either Error ByteString)+signSafer+    :: (HashAlgorithm hash, MonadRandom m)+    => PSSParams hash ByteString ByteString+    -- ^ PSS Parameters to use+    -> PrivateKey+    -- ^ private key+    -> ByteString+    -- ^ message to sign+    -> m (Either Error ByteString) signSafer params pk m = do     blinder <- generateBlinder (private_n pk)     sign (Just blinder) params pk m  -- | Sign using the PSS Parameters and an automatically generated blinder.-signDigestSafer :: (HashAlgorithm hash, MonadRandom m)-                => PSSParams hash ByteString ByteString -- ^ PSS Parameters to use-                -> PrivateKey     -- ^ private key-                -> Digest hash    -- ^ message digst-                -> m (Either Error ByteString)+signDigestSafer+    :: (HashAlgorithm hash, MonadRandom m)+    => PSSParams hash ByteString ByteString+    -- ^ PSS Parameters to use+    -> PrivateKey+    -- ^ private key+    -> Digest hash+    -- ^ message digst+    -> m (Either Error ByteString) signDigestSafer params pk digest = do     blinder <- generateBlinder (private_n pk)     signDigest (Just blinder) params pk digest  -- | Verify a signature using the PSS Parameters-verify :: HashAlgorithm hash-       => PSSParams hash ByteString ByteString-                     -- ^ PSS Parameters to use to verify,-                     --   this need to be identical to the parameters when signing-       -> PublicKey  -- ^ RSA Public Key-       -> ByteString -- ^ Message to verify-       -> ByteString -- ^ Signature-       -> Bool+verify+    :: HashAlgorithm hash+    => PSSParams hash ByteString ByteString+    -- ^ PSS Parameters to use to verify,+    --   this need to be identical to the parameters when signing+    -> PublicKey+    -- ^ RSA Public Key+    -> ByteString+    -- ^ Message to verify+    -> ByteString+    -- ^ Signature+    -> Bool verify params pk m = verifyDigest params pk mHash-  where mHash     = hashWith (pssHash params) m+  where+    mHash = hashWith (pssHash params) m  -- | Verify a signature using the PSS Parameters-verifyDigest :: HashAlgorithm hash-             => PSSParams hash ByteString ByteString-                            -- ^ PSS Parameters to use to verify,-                            --   this need to be identical to the parameters when signing-             -> PublicKey   -- ^ RSA Public Key-             -> Digest hash -- ^ Digest to verify-             -> ByteString  -- ^ Signature-             -> Bool+--+-- Following RFC 8017, the signature is rejected unless it is exactly as long+-- as the modulus (section 8.1.2, step 1) and its integer representative is+-- below the modulus (RSAVP1, section 5.2.2, step 1).  The public-key operation+-- normalises any multiple of the modulus away, so without the second check+-- @s + n@ would verify as readily as @s@, and a third party could turn one+-- valid signature into another without the private key.+verifyDigest+    :: HashAlgorithm hash+    => PSSParams hash ByteString ByteString+    -- ^ PSS Parameters to use to verify,+    --   this need to be identical to the parameters when signing+    -> PublicKey+    -- ^ RSA Public Key+    -> Digest hash+    -- ^ Digest to verify+    -> ByteString+    -- ^ Signature+    -> Bool verifyDigest params pk digest s-    | B.length s /= k                     = False-    | B.any (/= 0) pre                    = False+    | B.length s /= k = False+    | os2ip s >= public_n pk = False+    | B.any (/= 0) pre = False+    | B.any (\x -> x .&. topBits /= 0) (B.take 1 maskedDB) = False     | B.last em /= pssTrailerField params = False-    | B.any (/= 0) ps0                    = False-    | b1 /= B.singleton 1                 = False-    | otherwise                           = B.eq h h'-        where -- parameters-              hashLen   = hashDigestSize (pssHash params)-              mHash     = B.convert digest-              k         = public_size pk-              emLen     = if emTruncate pubBits then k - 1 else k-              dbLen     = emLen - hashLen - 1-              pubBits   = numBits (public_n pk)-              -- unmarshall fields-              (pre, em) = B.splitAt (k - emLen) (ep pk s) -- drop 0..1 byte-              maskedDB  = B.take dbLen em-              h         = B.take hashLen $ B.drop (B.length maskedDB) em-              dbmask    = pssMaskGenAlg params h dbLen-              db        = B.pack $ normalizeToKeySize pubBits $ B.zipWith xor maskedDB dbmask-              (ps0,z)   = B.break (== 1) db-              (b1,salt) = B.splitAt 1 z-              m'        = B.concat [B.replicate 8 0,mHash,salt]-              h'        = hashWith (pssHash params) m'+    | B.any (/= 0) ps0 = False+    | b1 /= B.singleton 1 = False+    | pssSaltLength params /= B.length salt = False+    | otherwise = B.eq h h'+  where+    -- parameters+    hashLen = hashDigestSize (pssHash params)+    mHash = B.convert digest+    k = public_size pk+    emLen = if emTruncate pubBits then k - 1 else k+    dbLen = emLen - hashLen - 1+    pubBits = numBits (public_n pk)+    -- RFC 8017 9.1.2 step 6: the leftmost 8*emLen - emBits bits of the+    -- leftmost octet of maskedDB have to be zero already.  Step 9 clears+    -- them in DB, which is what normalizeToKeySize does below, and clearing+    -- is not checking: without this an encoding with the top bit set -- one+    -- the standard calls inconsistent -- verifies as though it were sound,+    -- because the bit that made it wrong is thrown away before it is read.+    topBits = complement (normalizeMask pubBits)+    -- unmarshall fields+    (pre, em) = B.splitAt (k - emLen) (ep pk s) -- drop 0..1 byte+    maskedDB = B.take dbLen em+    h = B.take hashLen $ B.drop (B.length maskedDB) em+    dbmask = pssMaskGenAlg params h dbLen+    db = B.pack $ normalizeToKeySize pubBits $ B.zipWith xor maskedDB dbmask+    (ps0, z) = B.break (== 1) db+    (b1, salt) = B.splitAt 1 z+    m' = B.concat [B.replicate 8 0, mHash, salt]+    h' = hashWith (pssHash params) m'  -- When the modulus has bit length 1 modulo 8 we drop the first byte. emTruncate :: Int -> Bool-emTruncate bits = ((bits-1) .&. 0x7) == 0+emTruncate bits = ((bits - 1) .&. 0x7) == 0  normalizeToKeySize :: Int -> [Word8] -> [Word8]-normalizeToKeySize _    []     = [] -- very unlikely-normalizeToKeySize bits (x:xs) = x .&. mask : xs-    where mask = if sh > 0 then 0xff `shiftR` (8-sh) else 0xff-          sh   = (bits-1) .&. 0x7+normalizeToKeySize _ [] = [] -- very unlikely+normalizeToKeySize bits (x : xs) = x .&. normalizeMask bits : xs +-- | The bits of the leftmost octet that belong to the encoding: the low+-- @emBits `mod` 8@ of them, or all eight when that is zero.  Its complement+-- is the bits RFC 8017 requires to be zero.+normalizeMask :: Int -> Word8+normalizeMask bits = if sh > 0 then 0xff `shiftR` (8 - sh) else 0xff+  where+    sh = (bits - 1) .&. 0x7
Crypto/PubKey/RSA/Prim.hs view
@@ -4,19 +4,18 @@ -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----module Crypto.PubKey.RSA.Prim-    (+module Crypto.PubKey.RSA.Prim (     -- * Decrypt primitive-      dp+    dp,+     -- * Encrypt primitive-    , ep-    ) where+    ep,+) where -import           Crypto.PubKey.RSA.Types-import           Crypto.Number.ModArithmetic (expFast, expSafe)-import           Crypto.Number.Serialize (os2ip, i2ospOf_)-import           Crypto.Internal.ByteArray (ByteArray)+import Crypto.Internal.ByteArray (ByteArray)+import Crypto.Number.ModArithmetic (expFast, expSafe)+import Crypto.Number.Serialize (i2ospOf_, os2ip)+import Crypto.PubKey.RSA.Types  {- dpSlow computes the decrypted message not using any precomputed cache value.    only n and d need to valid. -}@@ -28,28 +27,32 @@    to compute than mod pq -} dpFast :: ByteArray ba => Blinder -> PrivateKey -> ba -> ba dpFast (Blinder r rm1) pk c =-    i2ospOf_ (private_size pk) (multiplication rm1 (m2 + h * (private_q pk)) (private_n pk))-    where-        re  = expFast r (public_e $ private_pub pk) (private_n pk)-        iC  = multiplication re (os2ip c) (private_n pk)-        m1  = expSafe iC (private_dP pk) (private_p pk)-        m2  = expSafe iC (private_dQ pk) (private_q pk)-        h   = ((private_qinv pk) * (m1 - m2)) `mod` (private_p pk)+    i2ospOf_+        (private_size pk)+        (multiplication rm1 (m2 + h * (private_q pk)) (private_n pk))+  where+    re = expFast r (public_e $ private_pub pk) (private_n pk)+    iC = multiplication re (os2ip c) (private_n pk)+    m1 = expSafe iC (private_dP pk) (private_p pk)+    m2 = expSafe iC (private_dQ pk) (private_q pk)+    h = ((private_qinv pk) * (m1 - m2)) `mod` (private_p pk)  dpFastNoBlinder :: ByteArray ba => PrivateKey -> ba -> ba dpFastNoBlinder pk c = i2ospOf_ (private_size pk) (m2 + h * (private_q pk))-     where iC = os2ip c-           m1 = expSafe iC (private_dP pk) (private_p pk)-           m2 = expSafe iC (private_dQ pk) (private_q pk)-           h  = ((private_qinv pk) * (m1 - m2)) `mod` (private_p pk)+  where+    iC = os2ip c+    m1 = expSafe iC (private_dP pk) (private_p pk)+    m2 = expSafe iC (private_dQ pk) (private_q pk)+    h = ((private_qinv pk) * (m1 - m2)) `mod` (private_p pk)  -- | Compute the RSA decrypt primitive. -- if the p and q numbers are available, then dpFast is used -- otherwise, we use dpSlow which only need d and n. dp :: ByteArray ba => Maybe Blinder -> PrivateKey -> ba -> ba dp blinder pk-    | private_p pk /= 0 && private_q pk /= 0 = maybe dpFastNoBlinder dpFast blinder $ pk-    | otherwise                              = dpSlow pk+    | private_p pk /= 0 && private_q pk /= 0 =+        maybe dpFastNoBlinder dpFast blinder $ pk+    | otherwise = dpSlow pk  -- | Compute the RSA encrypt primitive ep :: ByteArray ba => PublicKey -> ba -> ba
Crypto/PubKey/RSA/Types.hs view
@@ -1,54 +1,91 @@+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE DeriveGeneric #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.PubKey.RSA.Types -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.PubKey.RSA.Types-    ( Error(..)-    , Blinder(..)-    , PublicKey(..)-    , PrivateKey(..)-    , KeyPair(..)-    , toPublicKey-    , toPrivateKey-    , private_size-    , private_n-    , private_e-    ) where+module Crypto.PubKey.RSA.Types (+    Error (..),+    Blinder (..),+    PublicKey (..),+    PrivateKey (..),+    KeyPair (..),+    toPublicKey,+    toPrivateKey,+    private_size,+    private_n,+    private_e,+) where -import           Data.Data-import           Crypto.Internal.Imports+import Crypto.Debug (DebugShow (..))+import Crypto.Internal.Imports+import Data.Data --- | Blinder which is used to obfuscate the timing--- of the decryption primitive (used by decryption and signing).+import GHC.Generics++-- | A blinder, which keeps the timing of the private key operation from+-- saying anything about the number it was given.+--+-- The private exponent is not what is at risk.  'Crypto.Number.ModArithmetic.expSafe',+-- which the exponentiation goes through, keeps the /value/ of an exponent out+-- of the work it does.+--+-- What a blinder covers is the other side.  Without one, the operation runs+-- on the ciphertext as it arrived, so how long it takes depends on a number+-- an attacker may have chosen and can vary -- which is what a remote timing+-- attack on RSA needs.  With one, the input is multiplied by a random value+-- first and that value divided out afterwards, so the timing carries nothing+-- an attacker can steer.+--+-- Every private key operation here takes a @'Maybe' t'Blinder'@.  The+-- @Safer@ form of each -- 'Crypto.PubKey.RSA.PKCS15.decryptSafer',+-- 'Crypto.PubKey.RSA.PKCS15.signSafer' and their kind -- generates one and is+-- the one to reach for.  Pass 'Nothing' only where the input is not attacker+-- controlled and you have decided that it is not.+--+-- A blinder costs one more exponentiation, by the public exponent, which is+-- the cheap direction: measured on an Apple M4, PKCS#1 v1.5 signing goes from+-- about 601 to about 620 microseconds.+--+-- Use a blinder once.  'Crypto.PubKey.RSA.generateBlinder' makes a fresh one;+-- carrying one across operations is not what it is for. data Blinder = Blinder !Integer !Integer-             deriving (Show,Eq)+    deriving (Show, Eq)  -- | error possible during encryption, decryption or signing.-data Error =-      MessageSizeIncorrect -- ^ the message to decrypt is not of the correct size (need to be == private_size)-    | MessageTooLong       -- ^ the message to encrypt is too long-    | MessageNotRecognized -- ^ the message decrypted doesn't have a PKCS15 structure (0 2 .. 0 msg)-    | SignatureTooLong     -- ^ the message's digest is too long-    | InvalidParameters    -- ^ some parameters lead to breaking assumptions.-    deriving (Show,Eq)+data Error+    = -- | the message to decrypt is not of the correct size (need to be == private_size)+      MessageSizeIncorrect+    | -- | the message to encrypt is too long+      MessageTooLong+    | -- | the message decrypted doesn't have a PKCS15 structure (0 2 .. 0 msg)+      MessageNotRecognized+    | -- | the message's digest is too long+      SignatureTooLong+    | -- | some parameters lead to breaking assumptions.+      InvalidParameters+    deriving (Show, Eq)  -- | Represent a RSA public key data PublicKey = PublicKey-    { public_size :: Int      -- ^ size of key in bytes-    , public_n    :: Integer  -- ^ public p*q-    , public_e    :: Integer  -- ^ public exponent e-    } deriving (Show,Read,Eq,Data)+    { public_size :: Int+    -- ^ size of key in bytes+    , public_n :: Integer+    -- ^ public p*q+    , public_e :: Integer+    -- ^ public exponent e+    }+    deriving (Show, Read, Eq, Data, Generic)  instance NFData PublicKey where     rnf (PublicKey sz n e) = rnf n `seq` rnf e `seq` sz `seq` ()  -- | Represent a RSA private key.--- +-- -- Only the pub, d fields are mandatory to fill. -- -- p, q, dP, dQ, qinv are by-product during RSA generation,@@ -56,20 +93,65 @@ -- the decrypt and sign operation. -- -- implementations can leave optional fields to 0.--- data PrivateKey = PrivateKey-    { private_pub  :: PublicKey -- ^ public part of a private key (size, n and e)-    , private_d    :: Integer   -- ^ private exponent d-    , private_p    :: Integer   -- ^ p prime number-    , private_q    :: Integer   -- ^ q prime number-    , private_dP   :: Integer   -- ^ d mod (p-1)-    , private_dQ   :: Integer   -- ^ d mod (q-1)-    , private_qinv :: Integer   -- ^ q^(-1) mod p-    } deriving (Show,Read,Eq,Data)+    { private_pub :: PublicKey+    -- ^ public part of a private key (size, n and e)+    , private_d :: Integer+    -- ^ private exponent d+    , private_p :: Integer+    -- ^ p prime number+    , private_q :: Integer+    -- ^ q prime number+    , private_dP :: Integer+    -- ^ d mod (p-1)+    , private_dQ :: Integer+    -- ^ d mod (q-1)+    , private_qinv :: Integer+    -- ^ q^(-1) mod p+    }+    deriving (Read, Eq, Data, Generic) +-- | The public part is shown; the secret fields are not.  Use+-- 'Crypto.Debug.debugShow' to see them.+instance Show PrivateKey where+    showsPrec d k =+        showParen (d > 10) $+            showString "PrivateKey {private_pub = "+                . shows (private_pub k)+                . showString+                    ", private_d = <secret>, private_p = <secret>\+                    \, private_q = <secret>, private_dP = <secret>\+                    \, private_dQ = <secret>, private_qinv = <secret>}"++instance DebugShow PrivateKey where+    debugShow k =+        showString "PrivateKey {private_pub = "+            . shows (private_pub k)+            . showString ", private_d = "+            . shows (private_d k)+            . showString ", private_p = "+            . shows (private_p k)+            . showString ", private_q = "+            . shows (private_q k)+            . showString ", private_dP = "+            . shows (private_dP k)+            . showString ", private_dQ = "+            . shows (private_dQ k)+            . showString ", private_qinv = "+            . shows (private_qinv k)+            . showChar '}'+            $ ""+ instance NFData PrivateKey where     rnf (PrivateKey pub d p q dp dq qinv) =-        rnf pub `seq` rnf d `seq` rnf p `seq` rnf q `seq` rnf dp `seq` rnf dq `seq` qinv `seq` ()+        rnf pub `seq`+            rnf d `seq`+                rnf p `seq`+                    rnf q `seq`+                        rnf dp `seq`+                            rnf dq `seq`+                                qinv `seq`+                                    ()  -- | get the size in bytes from a private key private_size :: PrivateKey -> Int@@ -87,7 +169,14 @@ -- -- note the RSA private key contains already an instance of public key for efficiency newtype KeyPair = KeyPair PrivateKey-    deriving (Show,Read,Eq,Data,NFData)+    deriving (Read, Eq, Data, NFData)++instance Show KeyPair where+    showsPrec d (KeyPair k) =+        showParen (d > 10) $ showString "KeyPair " . showsPrec 11 k++instance DebugShow KeyPair where+    debugShow (KeyPair k) = "KeyPair (" ++ debugShow k ++ ")"  -- | Public key of a RSA KeyPair toPublicKey :: KeyPair -> PublicKey
Crypto/PubKey/Rabin/Basic.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE DeriveDataTypeable #-}+ -- | -- Module      : Crypto.PubKey.Rabin.Basic -- License     : BSD-style@@ -7,48 +9,94 @@ -- -- Rabin cryptosystem for public-key cryptography and digital signature. ---{-# LANGUAGE DeriveDataTypeable #-}-module Crypto.PubKey.Rabin.Basic-    ( PublicKey(..)-    , PrivateKey(..)-    , Signature(..)-    , generate-    , encrypt-    , encryptWithSeed-    , decrypt-    , sign-    , signWith-    , verify-    ) where+-- == What is kept from the clock, and what is not+--+-- The square roots modulo the secret primes are taken with+-- 'Crypto.Number.ModArithmetic.expSafe', which does not read the exponent it+-- is given.  Two things here do read what they are given.+--+-- Signing asks for the Jacobi symbol of the hash modulo each of the two+-- private primes, and the Jacobi symbol is computed by a sequence of+-- reductions whose number follows both of its arguments -- so the work done+-- per signature follows the primes.  Key generation runs the extended+-- Euclidean algorithm on the two primes for the same reason.  Neither has a+-- drop-in replacement here: a Jacobi symbol that does not read its arguments+-- is a different algorithm, not a different call.+--+-- Around all of that is 'Integer' arithmetic, whose cost follows the size of+-- the numbers; see "Crypto.PubKey.DSA" for that note at more length.+module Crypto.PubKey.Rabin.Basic (+    PublicKey (..),+    PrivateKey (..),+    Signature (..),+    generate,+    encrypt,+    encryptWithSeed,+    decrypt,+    sign,+    signWith,+    verify,+) where -import           Data.ByteString (ByteString)+import Crypto.Debug (DebugShow (..))+import Data.ByteString (ByteString) import qualified Data.ByteString as B-import           Data.Data-import           Data.Either (rights)+import Data.Data+import Data.Either (rights) -import           Crypto.Hash-import           Crypto.Number.Basic (gcde, numBytes)-import           Crypto.Number.ModArithmetic (expSafe, jacobi)-import           Crypto.Number.Serialize (i2osp, i2ospOf_, os2ip)-import           Crypto.PubKey.Rabin.OAEP -import           Crypto.PubKey.Rabin.Types-import           Crypto.Random (MonadRandom, getRandomBytes)+import Crypto.Hash+import Crypto.Number.Basic (gcde, numBytes)+import Crypto.Number.ModArithmetic (expSafe, jacobi)+import Crypto.Number.Serialize (i2osp, i2ospOf_, os2ip)+import Crypto.PubKey.Rabin.OAEP+import Crypto.PubKey.Rabin.Types+import Crypto.Random (MonadRandom, getRandomBytes)  -- | Represent a Rabin public key. data PublicKey = PublicKey-    { public_size :: Int      -- ^ size of key in bytes-    , public_n    :: Integer  -- ^ public p*q-    } deriving (Show, Read, Eq, Data)+    { public_size :: Int+    -- ^ size of key in bytes+    , public_n :: Integer+    -- ^ public p*q+    }+    deriving (Show, Read, Eq, Data)  -- | Represent a Rabin private key. data PrivateKey = PrivateKey     { private_pub :: PublicKey-    , private_p   :: Integer   -- ^ p prime number-    , private_q   :: Integer   -- ^ q prime number-    , private_a   :: Integer-    , private_b   :: Integer-    } deriving (Show, Read, Eq, Data)+    , private_p :: Integer+    -- ^ p prime number+    , private_q :: Integer+    -- ^ q prime number+    , private_a :: Integer+    , private_b :: Integer+    }+    deriving (Read, Eq, Data) +-- | The public part is shown; the secret fields are not.  Use+-- 'Crypto.Debug.debugShow' to see them.+instance Show PrivateKey where+    showsPrec d k =+        showParen (d > 10) $+            showString "PrivateKey {private_pub = "+                . shows (private_pub k)+                . showString ", private_p = <secret>, private_q = <secret>, private_a = <secret>, private_b = <secret>}"++instance DebugShow PrivateKey where+    debugShow k =+        showString "PrivateKey {private_pub = "+            . shows (private_pub k)+            . showString ", private_p = "+            . shows (private_p k)+            . showString ", private_q = "+            . shows (private_q k)+            . showString ", private_a = "+            . shows (private_a k)+            . showString ", private_b = "+            . shows (private_b k)+            . showChar '}'+            $ ""+ -- | Rabin Signature. data Signature = Signature (Integer, Integer) deriving (Show, Read, Eq, Data) @@ -56,175 +104,246 @@ -- Primes p and q are both congruent 3 mod 4. -- -- See algorithm 8.11 in "Handbook of Applied Cryptography" by Alfred J. Menezes et al.-generate :: MonadRandom m-         => Int-         -> m (PublicKey, PrivateKey)+generate+    :: MonadRandom m+    => Int+    -> m (PublicKey, PrivateKey) generate size = do     (p, q) <- generatePrimes size (\p -> p `mod` 4 == 3) (\q -> q `mod` 4 == 3)     return $ generateKeys p q-  where +  where     generateKeys p q =-        let n = p*q-            (a, b, _) = gcde p q -            publicKey = PublicKey { public_size = size-                                    , public_n    = n }-            privateKey = PrivateKey { private_pub = publicKey-                                    , private_p   = p-                                    , private_q   = q-                                    , private_a   = a-                                    , private_b   = b }-            in (publicKey, privateKey)+        let n = p * q+            (a, b, _) = gcde p q+            publicKey =+                PublicKey+                    { public_size = size+                    , public_n = n+                    }+            privateKey =+                PrivateKey+                    { private_pub = publicKey+                    , private_p = p+                    , private_q = q+                    , private_a = a+                    , private_b = b+                    }+         in (publicKey, privateKey)  -- | Encrypt plaintext using public key an a predefined OAEP seed. -- -- See algorithm 8.11 in "Handbook of Applied Cryptography" by Alfred J. Menezes et al.-encryptWithSeed :: HashAlgorithm hash-                => ByteString                               -- ^ Seed-                -> OAEPParams hash ByteString ByteString    -- ^ OAEP padding-                -> PublicKey                                -- ^ public key-                -> ByteString                               -- ^ plaintext-                -> Either Error ByteString+encryptWithSeed+    :: HashAlgorithm hash+    => ByteString+    -- ^ Seed+    -> OAEPParams hash ByteString ByteString+    -- ^ OAEP padding+    -> PublicKey+    -- ^ public key+    -> ByteString+    -- ^ plaintext+    -> Either Error ByteString encryptWithSeed seed oaep pk m =-    let n  = public_n pk-        k  = numBytes n+    let n = public_n pk+        k = numBytes n      in do-        m' <- pad seed oaep k m-        let m'' = os2ip m'-        return $ i2osp $ expSafe m'' 2 n+            m' <- pad seed oaep k m+            let m'' = os2ip m'+            return $ i2osp $ expSafe m'' 2 n  -- | Encrypt plaintext using public key.-encrypt :: (HashAlgorithm hash, MonadRandom m)-        => OAEPParams hash ByteString ByteString    -- ^ OAEP padding parameters-        -> PublicKey                                -- ^ public key-        -> ByteString                               -- ^ plaintext -        -> m (Either Error ByteString)+encrypt+    :: (HashAlgorithm hash, MonadRandom m)+    => OAEPParams hash ByteString ByteString+    -- ^ OAEP padding parameters+    -> PublicKey+    -- ^ public key+    -> ByteString+    -- ^ plaintext+    -> m (Either Error ByteString) encrypt oaep pk m = do     seed <- getRandomBytes hashLen     return $ encryptWithSeed seed oaep pk m   where-    hashLen = hashDigestSize (oaepHash oaep) +    hashLen = hashDigestSize (oaepHash oaep)  -- | Decrypt ciphertext using private key. --+-- The ciphertext has to be what 'encrypt' produces: the big-endian encoding,+-- with no leading zero octet, of a value below the modulus.  Squaring and the+-- square roots that undo it work modulo n, so without that condition @c@ and+-- @c + n@ -- and @c@ with a zero octet in front of it -- would all decrypt to+-- the same message, and a ciphertext would not be unique to its plaintext.+-- -- See algorithm 8.12 in "Handbook of Applied Cryptography" by Alfred J. Menezes et al.-decrypt :: HashAlgorithm hash-        => OAEPParams hash ByteString ByteString    -- ^ OAEP padding parameters-        -> PrivateKey                               -- ^ private key-        -> ByteString                               -- ^ ciphertext-        -> Maybe ByteString-decrypt oaep pk c =-    let p  = private_p pk -        q  = private_q pk     -        a  = private_a pk -        b  = private_b pk-        n  = public_n $ private_pub pk-        k  = numBytes n-        c' = os2ip c-        solutions = rights $ toList $ mapTuple (unpad oaep k . i2ospOf_ k) $ sqroot' c' p q a b n-     in if length solutions /= 1 then Nothing-        else Just $ head solutions-      where toList (w, x, y, z) = w:x:y:z:[]-            mapTuple f (w, x, y, z) = (f w, f x, f y, f z)+decrypt+    :: HashAlgorithm hash+    => OAEPParams hash ByteString ByteString+    -- ^ OAEP padding parameters+    -> PrivateKey+    -- ^ private key+    -> ByteString+    -- ^ ciphertext+    -> Maybe ByteString+decrypt oaep pk c+    | os2ip c >= public_n (private_pub pk) = Nothing+    | c /= (i2osp (os2ip c) :: ByteString) = Nothing+    | otherwise =+        let p = private_p pk+            q = private_q pk+            a = private_a pk+            b = private_b pk+            n = public_n $ private_pub pk+            k = numBytes n+            c' = os2ip c+            solutions = rights $ toList $ mapTuple (unpad oaep k . i2ospOf_ k) $ sqroot' c' p q a b n+         in case solutions of+                [x] -> Just x+                _ -> Nothing+  where+    toList (w, x, y, z) = w : x : y : z : []+    mapTuple f (w, x, y, z) = (f w, f x, f y, f z)  -- | Sign message using padding, hash algorithm and private key. -- -- See <https://en.wikipedia.org/wiki/Rabin_signature_algorithm>.-signWith :: HashAlgorithm hash-         => ByteString    -- ^ padding-         -> PrivateKey    -- ^ private key-         -> hash          -- ^ hash function-         -> ByteString    -- ^ message to sign-         -> Either Error Signature-signWith padding pk hashAlg m = do-    h <- calculateHash padding pk hashAlg m-    signature <- calculateSignature h-    return signature+signWith+    :: HashAlgorithm hash+    => ByteString+    -- ^ padding+    -> PrivateKey+    -- ^ private key+    -> hash+    -- ^ hash function+    -> ByteString+    -- ^ message to sign+    -> Either Error Signature+signWith padding pk hashAlg m+    -- the signature carries the padding as an integer, so a leading zero octet+    -- would not survive it: verify would hash one octet less than was signed+    | B.null padding || B.index padding 0 == 0 = Left InvalidParameters+    | otherwise = do+        h <- calculateHash padding pk hashAlg m+        signature <- calculateSignature h+        return signature   where     calculateSignature h =         let p = private_p pk-            q = private_q pk     -            a = private_a pk +            q = private_q pk+            a = private_a pk             b = private_b pk             n = public_n $ private_pub pk-         in if h >= n then Left MessageTooLong-            else let (r, _, _, _) = sqroot' h p q a b n-                  in Right $ Signature (os2ip padding, r)+         in if h >= n+                then Left MessageTooLong+                else+                    let (r, _, _, _) = sqroot' h p q a b n+                     in Right $ Signature (os2ip padding, r)  -- | Sign message using hash algorithm and private key. -- -- See <https://en.wikipedia.org/wiki/Rabin_signature_algorithm>.-sign :: (MonadRandom m, HashAlgorithm hash)-     => PrivateKey    -- ^ private key-     -> hash          -- ^ hash function-     -> ByteString    -- ^ message to sign-     -> m (Either Error Signature)+sign+    :: (MonadRandom m, HashAlgorithm hash)+    => PrivateKey+    -- ^ private key+    -> hash+    -- ^ hash function+    -> ByteString+    -- ^ message to sign+    -> m (Either Error Signature) sign pk hashAlg m = do     padding <- findPadding     return $ signWith padding pk hashAlg m-  where +  where     findPadding = do         padding <- getRandomBytes 8-        case calculateHash padding pk hashAlg m of-            Right _ -> return padding-            _       -> findPadding+        case (B.index padding 0, calculateHash padding pk hashAlg m) of+            -- a padding that starts with a zero octet is one signWith refuses+            (0, _) -> findPadding+            (_, Right _) -> return padding+            _ -> findPadding  -- | Calculate hash of message and padding. -- If the padding is valid, then the result of the hash operation is returned, otherwise an error.-calculateHash :: HashAlgorithm hash-              => ByteString    -- ^ padding-              -> PrivateKey    -- ^ private key-              -> hash          -- ^ hash function-              -> ByteString    -- ^ message to sign-              -> Either Error Integer-calculateHash padding pk hashAlg m = +calculateHash+    :: HashAlgorithm hash+    => ByteString+    -- ^ padding+    -> PrivateKey+    -- ^ private key+    -> hash+    -- ^ hash function+    -> ByteString+    -- ^ message to sign+    -> Either Error Integer+calculateHash padding pk hashAlg m =     let p = private_p pk         q = private_q pk         h = os2ip $ hashWith hashAlg $ B.append padding m      in case (jacobi (h `mod` p) p, jacobi (h `mod` q) q) of             (Just 1, Just 1) -> Right h-            _                -> Left InvalidParameters+            _ -> Left InvalidParameters  -- | Verify signature using hash algorithm and public key. -- -- See <https://en.wikipedia.org/wiki/Rabin_signature_algorithm>.-verify :: HashAlgorithm hash-       => PublicKey     -- ^ private key-       -> hash          -- ^ hash function-       -> ByteString    -- ^ message-       -> Signature     -- ^ signature-       -> Bool-verify pk hashAlg m (Signature (padding, s)) =-    let n  = public_n pk-        p  = i2osp padding-        h  = os2ip $ hashWith hashAlg $ B.append p m -        h' = expSafe s 2 n-     in h' == h+verify+    :: HashAlgorithm hash+    => PublicKey+    -- ^ private key+    -> hash+    -- ^ hash function+    -> ByteString+    -- ^ message+    -> Signature+    -- ^ signature+    -> Bool+verify pk hashAlg m (Signature (padding, s))+    -- squaring works modulo n, so s + n and -s would verify wherever s does+    | s < 0 || s >= n = False+    | padding < 0 = False+    | otherwise =+        let p = i2osp padding+            h = os2ip $ hashWith hashAlg $ B.append p m+            h' = expSafe s 2 n+         in h' == h+  where+    n = public_n pk  -- | Square roots modulo prime p where p is congruent 3 mod 4 -- Value a must be a quadratic residue modulo p (i.e. jacobi symbol (a/n) = 1). -- -- See algorithm 3.36 in "Handbook of Applied Cryptography" by Alfred J. Menezes et al.-sqroot :: Integer-       -> Integer   -- ^ prime p-       -> (Integer, Integer)+sqroot+    :: Integer+    -> Integer+    -- ^ prime p+    -> (Integer, Integer) sqroot a p =     let r = expSafe a ((p + 1) `div` 4) p      in (r, -r)  -- | Square roots modulo n given its prime factors p and q (both congruent 3 mod 4) -- Value a must be a quadratic residue of both modulo p and modulo q (i.e. jacobi symbols (a/p) = (a/q) = 1).--- +-- -- See algorithm 3.44 in "Handbook of Applied Cryptography" by Alfred J. Menezes et al.-sqroot' :: Integer -        -> Integer  -- ^ prime p-        -> Integer  -- ^ prime q-        -> Integer  -- ^ c such that c*p + d*q = 1-        -> Integer  -- ^ d such that c*p + d*q = 1-        -> Integer  -- ^ n = p*q-        -> (Integer, Integer, Integer, Integer)+sqroot'+    :: Integer+    -> Integer+    -- ^ prime p+    -> Integer+    -- ^ prime q+    -> Integer+    -- ^ c such that c*p + d*q = 1+    -> Integer+    -- ^ d such that c*p + d*q = 1+    -> Integer+    -- ^ n = p*q+    -> (Integer, Integer, Integer, Integer) sqroot' a p q c d n =     let (r, _) = sqroot a p         (s, _) = sqroot a q-        x      = (r*d*q + s*c*p) `mod` n-        y      = (r*d*q - s*c*p) `mod` n+        x = (r * d * q + s * c * p) `mod` n+        y = (r * d * q - s * c * p) `mod` n      in (x, (-x) `mod` n, y, (-y) `mod` n)
Crypto/PubKey/Rabin/Modified.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE DeriveDataTypeable #-}+ -- | -- Module      : Crypto.PubKey.Rabin.Modified -- License     : BSD-style@@ -7,95 +9,149 @@ -- -- Modified-Rabin public-key digital signature algorithm. -- See algorithm 11.30 in "Handbook of Applied Cryptography" by Alfred J. Menezes et al.----{-# LANGUAGE DeriveDataTypeable #-}-module Crypto.PubKey.Rabin.Modified-    ( PublicKey(..)-    , PrivateKey(..)-    , generate-    , sign-    , verify-    ) where+-- The Jacobi symbols here are taken modulo the public modulus, not the+-- private primes, so what "Crypto.PubKey.Rabin.Basic" says about that does+-- not apply; the note there about 'Integer' arithmetic does.+module Crypto.PubKey.Rabin.Modified (+    PublicKey (..),+    PrivateKey (..),+    generate,+    sign,+    verify,+) where -import           Data.ByteString-import           Data.Data+import Crypto.Debug (DebugShow (..))+import Data.ByteString+import Data.Data -import           Crypto.Hash-import           Crypto.Number.ModArithmetic (expSafe, jacobi)-import           Crypto.Number.Serialize (os2ip)-import           Crypto.PubKey.Rabin.Types-import           Crypto.Random.Types+import Crypto.Hash+import Crypto.Number.ModArithmetic (expSafe, jacobi)+import Crypto.Number.Serialize (os2ip)+import Crypto.PubKey.Rabin.Types+import Crypto.Random.Types  -- | Represent a Modified-Rabin public key. data PublicKey = PublicKey-    { public_size :: Int      -- ^ size of key in bytes-    , public_n    :: Integer  -- ^ public p*q-    } deriving (Show, Read, Eq, Data)+    { public_size :: Int+    -- ^ size of key in bytes+    , public_n :: Integer+    -- ^ public p*q+    }+    deriving (Show, Read, Eq, Data)  -- | Represent a Modified-Rabin private key. data PrivateKey = PrivateKey     { private_pub :: PublicKey-    , private_p   :: Integer   -- ^ p prime number-    , private_q   :: Integer   -- ^ q prime number-    , private_d   :: Integer-    } deriving (Show, Read, Eq, Data)+    , private_p :: Integer+    -- ^ p prime number+    , private_q :: Integer+    -- ^ q prime number+    , private_d :: Integer+    }+    deriving (Read, Eq, Data) +-- | The public part is shown; the secret fields are not.  Use+-- 'Crypto.Debug.debugShow' to see them.+instance Show PrivateKey where+    showsPrec d k =+        showParen (d > 10) $+            showString "PrivateKey {private_pub = "+                . shows (private_pub k)+                . showString ", private_p = <secret>, private_q = <secret>, private_d = <secret>}"++instance DebugShow PrivateKey where+    debugShow k =+        showString "PrivateKey {private_pub = "+            . shows (private_pub k)+            . showString ", private_p = "+            . shows (private_p k)+            . showString ", private_q = "+            . shows (private_q k)+            . showString ", private_d = "+            . shows (private_d k)+            . showChar '}'+            $ ""+ -- | Generate a pair of (private, public) key of size in bytes. -- Prime p is congruent 3 mod 8 and prime q is congruent 7 mod 8.-generate :: MonadRandom m-         => Int           -         -> m (PublicKey, PrivateKey)+generate+    :: MonadRandom m+    => Int+    -> m (PublicKey, PrivateKey) generate size = do     (p, q) <- generatePrimes size (\p -> p `mod` 8 == 3) (\q -> q `mod` 8 == 7)     return $ generateKeys p q-  where +  where     generateKeys p q =-        let n = p*q   +        let n = p * q             d = (n - p - q + 5) `div` 8-            publicKey = PublicKey { public_size = size-                                    , public_n    = n }-            privateKey = PrivateKey { private_pub = publicKey-                                    , private_p   = p-                                    , private_q   = q-                                    , private_d   = d }-            in (publicKey, privateKey)+            publicKey =+                PublicKey+                    { public_size = size+                    , public_n = n+                    }+            privateKey =+                PrivateKey+                    { private_pub = publicKey+                    , private_p = p+                    , private_q = q+                    , private_d = d+                    }+         in (publicKey, privateKey)  -- | Sign message using hash algorithm and private key.-sign :: HashAlgorithm hash-     => PrivateKey    -- ^ private key-     -> hash          -- ^ hash function-     -> ByteString    -- ^ message to sign-     -> Either Error Integer+sign+    :: HashAlgorithm hash+    => PrivateKey+    -- ^ private key+    -> hash+    -- ^ hash function+    -> ByteString+    -- ^ message to sign+    -> Either Error Integer sign pk hashAlg m =     let d = private_d pk         n = public_n $ private_pub pk         h = os2ip $ hashWith hashAlg m         limit = (n - 6) `div` 16-     in if h > limit then Left MessageTooLong-        else let h' = 16*h + 6-              in case jacobi h' n of-                    Just 1    -> Right $ expSafe h' d n-                    Just (-1) -> Right $ expSafe (h' `div` 2) d n-                    _         -> Left InvalidParameters+     in if h > limit+            then Left MessageTooLong+            else+                let h' = 16 * h + 6+                 in case jacobi h' n of+                        Just 1 -> Right $ expSafe h' d n+                        Just (-1) -> Right $ expSafe (h' `div` 2) d n+                        _ -> Left InvalidParameters  -- | Verify signature using hash algorithm and public key.-verify :: HashAlgorithm hash-       => PublicKey     -- ^ public key-       -> hash          -- ^ hash function-       -> ByteString    -- ^ message-       -> Integer       -- ^ signature-       -> Bool-verify pk hashAlg m s =-    let n   = public_n pk-        h   = os2ip $ hashWith hashAlg m-        s'  = expSafe s 2 n-        s'' = case s' `mod` 8 of-            6 -> s'-            3 -> 2*s'-            7 -> n - s'-            2 -> 2*(n - s')-            _ -> 0-     in case s'' `mod` 16 of-            6 -> let h' = (s'' - 6) `div` 16-                  in h' == h -            _ -> False+verify+    :: HashAlgorithm hash+    => PublicKey+    -- ^ public key+    -> hash+    -- ^ hash function+    -> ByteString+    -- ^ message+    -> Integer+    -- ^ signature+    -> Bool+verify pk hashAlg m s+    -- squaring works modulo n, so s + n and -s would verify wherever s does+    | s < 0 || s >= n = False+    | otherwise = go+  where+    n = public_n pk+    go =+        let h = os2ip $ hashWith hashAlg m+            s' = expSafe s 2 n+            s'' = case s' `mod` 8 of+                6 -> s'+                3 -> 2 * s'+                7 -> n - s'+                2 -> 2 * (n - s')+                _ -> 0+         in case s'' `mod` 16 of+                6 ->+                    let h' = (s'' - 6) `div` 16+                     in h' == h+                _ -> False
Crypto/PubKey/Rabin/OAEP.hs view
@@ -7,94 +7,145 @@ -- -- OAEP padding scheme. -- See <http://en.wikipedia.org/wiki/Optimal_asymmetric_encryption_padding>.----module Crypto.PubKey.Rabin.OAEP-    ( OAEPParams(..)-    , defaultOAEPParams-    , pad-    , unpad-    ) where-        -import           Data.ByteString (ByteString)+module Crypto.PubKey.Rabin.OAEP (+    OAEPParams (..),+    defaultOAEPParams,+    pad,+    unpad,+) where++import Data.Bits (complement, shiftR, xor, (.&.), (.|.))+import Data.ByteString (ByteString) import qualified Data.ByteString as B-import           Data.Bits (xor)+import Data.List (foldl')+import Data.Word (Word32)+import Prelude hiding (foldl') -import           Crypto.Hash-import           Crypto.Internal.ByteArray (ByteArrayAccess, ByteArray)-import qualified Crypto.Internal.ByteArray as B (convert)-import           Crypto.PubKey.MaskGenFunction-import           Crypto.PubKey.Internal (and')-import           Crypto.PubKey.Rabin.Types+import Crypto.Hash+import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess)+import qualified Crypto.Internal.ByteArray as B (constEq, convert)+import Crypto.PubKey.Internal (and')+import Crypto.PubKey.MaskGenFunction+import Crypto.PubKey.Rabin.Types  -- | Parameters for OAEP padding. data OAEPParams hash seed output = OAEPParams-    { oaepHash       :: hash                            -- ^ hash function to use-    , oaepMaskGenAlg :: MaskGenAlgorithm seed output    -- ^ mask Gen algorithm to use-    , oaepLabel      :: Maybe ByteString                -- ^ optional label prepended to message+    { oaepHash :: hash+    -- ^ hash function to use+    , oaepMaskGenAlg :: MaskGenAlgorithm seed output+    -- ^ mask Gen algorithm to use+    , oaepLabel :: Maybe ByteString+    -- ^ optional label prepended to message     }  -- | Default Params with a specified hash function.-defaultOAEPParams :: (ByteArrayAccess seed, ByteArray output, HashAlgorithm hash)-                  => hash-                  -> OAEPParams hash seed output+defaultOAEPParams+    :: (ByteArrayAccess seed, ByteArray output, HashAlgorithm hash)+    => hash+    -> OAEPParams hash seed output defaultOAEPParams hashAlg =-    OAEPParams { oaepHash       = hashAlg-               , oaepMaskGenAlg = mgf1 hashAlg-               , oaepLabel      = Nothing-               }+    OAEPParams+        { oaepHash = hashAlg+        , oaepMaskGenAlg = mgf1 hashAlg+        , oaepLabel = Nothing+        }  -- | Pad a message using OAEP.-pad :: HashAlgorithm hash-    => ByteString                               -- ^ Seed-    -> OAEPParams hash ByteString ByteString    -- ^ OAEP params to use-    -> Int                                      -- ^ size of public key in bytes-    -> ByteString                               -- ^ Message pad+pad+    :: HashAlgorithm hash+    => ByteString+    -- ^ Seed+    -> OAEPParams hash ByteString ByteString+    -- ^ OAEP params to use+    -> Int+    -- ^ size of public key in bytes+    -> ByteString+    -- ^ Message pad     -> Either Error ByteString pad seed oaep k msg-    | k < 2*hashLen+2          = Left InvalidParameters+    | k < 2 * hashLen + 2 = Left InvalidParameters     | B.length seed /= hashLen = Left InvalidParameters-    | mLen > k - 2*hashLen-2   = Left MessageTooLong-    | otherwise                = Right em-    where -- parameters-        mLen       = B.length msg-        mgf        = oaepMaskGenAlg oaep-        labelHash  = hashWith (oaepHash oaep) (maybe B.empty id $ oaepLabel oaep)-        hashLen    = hashDigestSize (oaepHash oaep)-        -- put fields-        ps         = B.replicate (k - mLen - 2*hashLen - 2) 0-        db         = B.concat [B.convert labelHash, ps, B.singleton 0x1, msg]-        dbmask     = mgf seed (k - hashLen - 1)-        maskedDB   = B.pack $ B.zipWith xor db dbmask-        seedMask   = mgf maskedDB hashLen-        maskedSeed = B.pack $ B.zipWith xor seed seedMask-        em         = B.concat [B.singleton 0x0, maskedSeed, maskedDB]+    | mLen > k - 2 * hashLen - 2 = Left MessageTooLong+    | otherwise = Right em+  where+    -- parameters+    mLen = B.length msg+    mgf = oaepMaskGenAlg oaep+    labelHash = hashWith (oaepHash oaep) (maybe B.empty id $ oaepLabel oaep)+    hashLen = hashDigestSize (oaepHash oaep)+    -- put fields+    ps = B.replicate (k - mLen - 2 * hashLen - 2) 0+    db = B.concat [B.convert labelHash, ps, B.singleton 0x1, msg]+    dbmask = mgf seed (k - hashLen - 1)+    maskedDB = B.pack $ B.zipWith xor db dbmask+    seedMask = mgf maskedDB hashLen+    maskedSeed = B.pack $ B.zipWith xor seed seedMask+    em = B.concat [B.singleton 0x0, maskedSeed, maskedDB]  -- | Un-pad a OAEP encoded message.-unpad :: HashAlgorithm hash-      => OAEPParams hash ByteString ByteString  -- ^ OAEP params to use-      -> Int                                    -- ^ size of public key in bytes-      -> ByteString                             -- ^ encoded message (not encrypted)-      -> Either Error ByteString+--+-- The data block is scanned in full rather than up to the 01 octet separating+-- the padding from the message, and the label hash and the leading octet are+-- compared without an early exit, so neither the length of the padding nor+-- where a comparison first differs shows up in how long this takes.  This is+-- what "Crypto.PubKey.RSA.OAEP" does with the same block.+--+-- What remains visible is the result itself: whether the block was well formed,+-- and the length of the message when it was.  That is the signal Manger's+-- attack needs, so a caller that decrypts attacker-supplied ciphertext must not+-- pass the distinction on.+unpad+    :: HashAlgorithm hash+    => OAEPParams hash ByteString ByteString+    -- ^ OAEP params to use+    -> Int+    -- ^ size of public key in bytes+    -> ByteString+    -- ^ encoded message (not encrypted)+    -> Either Error ByteString unpad oaep k em     | paddingSuccess = Right msg-    | otherwise      = Left MessageNotRecognized-    where -- parameters-        mgf        = oaepMaskGenAlg oaep-        labelHash  = B.convert $ hashWith (oaepHash oaep) (maybe B.empty id $ oaepLabel oaep)-        hashLen    = hashDigestSize (oaepHash oaep)-        -- getting em's fields-        (pb, em0)  = B.splitAt 1 em-        (maskedSeed, maskedDB) = B.splitAt hashLen em0-        seedMask   = mgf maskedDB hashLen-        seed       = B.pack $ B.zipWith xor maskedSeed seedMask-        dbmask     = mgf seed (k - hashLen - 1)-        db         = B.pack $ B.zipWith xor maskedDB dbmask-        -- getting db's fields-        (labelHash', db1) = B.splitAt hashLen db-        (_, db2)   = B.break (/= 0) db1-        (ps1, msg) = B.splitAt 1 db2+    | otherwise = Left MessageNotRecognized+  where+    -- parameters+    mgf = oaepMaskGenAlg oaep+    labelHash =+        B.convert $ hashWith (oaepHash oaep) (maybe B.empty id $ oaepLabel oaep)+            :: ByteString+    hashLen = hashDigestSize (oaepHash oaep)+    -- getting em's fields+    (pb, em0) = B.splitAt 1 em+    (maskedSeed, maskedDB) = B.splitAt hashLen em0+    seedMask = mgf maskedDB hashLen+    seed = B.pack $ B.zipWith xor maskedSeed seedMask+    dbmask = mgf seed (k - hashLen - 1)+    db = B.pack $ B.zipWith xor maskedDB dbmask+    -- getting db's fields+    (labelHash', db1) = B.splitAt hashLen db -        paddingSuccess = and' [ labelHash' == labelHash -- no need for constant eq-                              , ps1        == B.replicate 1 0x1-                              , pb         == B.replicate 1 0x0-                              ]+    -- index of the first nonzero octet in db1, or its length when every octet+    -- is zero; all of them are looked at either way+    oneIndex =+        fst $+            foldl'+                step+                (fromIntegral (B.length db1) :: Word32, 1 :: Word32)+                (zip [0 ..] (B.unpack db1))+    step (idx, unseen) (i, b) = (select found i idx, unseen .&. complement found)+      where+        w = fromIntegral b :: Word32+        -- 0 when b is zero, 1 otherwise+        nonZero = (w .|. negate w) `shiftR` 31+        -- all ones at the first nonzero octet only+        found = negate (unseen .&. nonZero)+    select mask a b = (a .&. mask) .|. (b .&. complement mask)++    ps1 = B.take 1 $ B.drop (fromIntegral oneIndex) db1+    msg = B.drop (fromIntegral oneIndex + 1) db1++    paddingSuccess =+        and'+            [ labelHash' `B.constEq` labelHash+            , ps1 `B.constEq` B.replicate 1 0x1+            , pb `B.constEq` B.replicate 1 0x0+            ]
Crypto/PubKey/Rabin/RW.hs view
@@ -1,3 +1,5 @@+{-# LANGUAGE DeriveDataTypeable #-}+ -- | -- Module      : Crypto.PubKey.Rabin.RW -- License     : BSD-style@@ -5,147 +7,219 @@ -- Stability   : experimental -- Portability : unknown ----- Rabin-Williams cryptosystem for public-key encryption and digital signature. +-- Rabin-Williams cryptosystem for public-key encryption and digital signature. -- See pages 323 - 324 in "Computational Number Theory and Modern Cryptography" by Song Y. Yan. -- Also inspired by https://github.com/vanilala/vncrypt/blob/master/vncrypt/vnrw_gmp.c.--- -{-# LANGUAGE DeriveDataTypeable #-}-module Crypto.PubKey.Rabin.RW-    ( PublicKey(..)-    , PrivateKey(..)-    , generate-    , encrypt-    , encryptWithSeed-    , decrypt-    , sign-    , verify-    ) where+-- The Jacobi symbols here are taken modulo the public modulus, not the+-- private primes, so what "Crypto.PubKey.Rabin.Basic" says about that does+-- not apply; the note there about 'Integer' arithmetic does.+module Crypto.PubKey.Rabin.RW (+    PublicKey (..),+    PrivateKey (..),+    generate,+    encrypt,+    encryptWithSeed,+    decrypt,+    sign,+    verify,+) where -import           Data.ByteString-import           Data.Data+import Crypto.Debug (DebugShow (..))+import Data.ByteString+import Data.Data -import           Crypto.Hash-import           Crypto.Number.Basic (numBytes)-import           Crypto.Number.ModArithmetic (expSafe, jacobi)-import           Crypto.Number.Serialize (i2osp, i2ospOf_, os2ip)-import           Crypto.PubKey.Rabin.OAEP-import           Crypto.PubKey.Rabin.Types-import           Crypto.Random.Types+import Crypto.Hash+import Crypto.Number.Basic (numBytes)+import Crypto.Number.ModArithmetic (expSafe, jacobi)+import Crypto.Number.Serialize (i2osp, i2ospOf_, os2ip)+import Crypto.PubKey.Rabin.OAEP+import Crypto.PubKey.Rabin.Types+import Crypto.Random.Types  -- | Represent a Rabin-Williams public key. data PublicKey = PublicKey-    { public_size :: Int      -- ^ size of key in bytes-    , public_n    :: Integer  -- ^ public p*q-    } deriving (Show, Read, Eq, Data)+    { public_size :: Int+    -- ^ size of key in bytes+    , public_n :: Integer+    -- ^ public p*q+    }+    deriving (Show, Read, Eq, Data)  -- | Represent a Rabin-Williams private key. data PrivateKey = PrivateKey     { private_pub :: PublicKey-    , private_p   :: Integer   -- ^ p prime number-    , private_q   :: Integer   -- ^ q prime number-    , private_d   :: Integer-    } deriving (Show, Read, Eq, Data)+    , private_p :: Integer+    -- ^ p prime number+    , private_q :: Integer+    -- ^ q prime number+    , private_d :: Integer+    }+    deriving (Read, Eq, Data) +-- | The public part is shown; the secret fields are not.  Use+-- 'Crypto.Debug.debugShow' to see them.+instance Show PrivateKey where+    showsPrec d k =+        showParen (d > 10) $+            showString "PrivateKey {private_pub = "+                . shows (private_pub k)+                . showString ", private_p = <secret>, private_q = <secret>, private_d = <secret>}"++instance DebugShow PrivateKey where+    debugShow k =+        showString "PrivateKey {private_pub = "+            . shows (private_pub k)+            . showString ", private_p = "+            . shows (private_p k)+            . showString ", private_q = "+            . shows (private_q k)+            . showString ", private_d = "+            . shows (private_d k)+            . showChar '}'+            $ ""+ -- | Generate a pair of (private, public) key of size in bytes. -- Prime p is congruent 3 mod 8 and prime q is congruent 7 mod 8.-generate :: MonadRandom m-         => Int           -         -> m (PublicKey, PrivateKey)+generate+    :: MonadRandom m+    => Int+    -> m (PublicKey, PrivateKey) generate size = do-    (p, q) <- generatePrimes size (\p -> p `mod` 8 == 3) (\q -> q `mod` 8 == 7) +    (p, q) <- generatePrimes size (\p -> p `mod` 8 == 3) (\q -> q `mod` 8 == 7)     return (generateKeys p q)-  where +  where     generateKeys p q =-        let n = p*q   -            d = ((p - 1)*(q - 1) `div` 4 + 1) `div` 2-            publicKey = PublicKey { public_size = size-                                    , public_n    = n }-            privateKey = PrivateKey { private_pub = publicKey-                                    , private_p   = p-                                    , private_q   = q-                                    , private_d   = d }-            in (publicKey, privateKey)+        let n = p * q+            d = ((p - 1) * (q - 1) `div` 4 + 1) `div` 2+            publicKey =+                PublicKey+                    { public_size = size+                    , public_n = n+                    }+            privateKey =+                PrivateKey+                    { private_pub = publicKey+                    , private_p = p+                    , private_q = q+                    , private_d = d+                    }+         in (publicKey, privateKey)  -- | Encrypt plaintext using public key an a predefined OAEP seed. -- -- See algorithm 8.11 in "Handbook of Applied Cryptography" by Alfred J. Menezes et al.-encryptWithSeed :: HashAlgorithm hash-                => ByteString                               -- ^ Seed-                -> OAEPParams hash ByteString ByteString    -- ^ OAEP padding-                -> PublicKey                                -- ^ public key-                -> ByteString                               -- ^ plaintext-                -> Either Error ByteString+encryptWithSeed+    :: HashAlgorithm hash+    => ByteString+    -- ^ Seed+    -> OAEPParams hash ByteString ByteString+    -- ^ OAEP padding+    -> PublicKey+    -- ^ public key+    -> ByteString+    -- ^ plaintext+    -> Either Error ByteString encryptWithSeed seed oaep pk m =     let n = public_n pk         k = numBytes n      in do-        m'  <- pad seed oaep k m-        m'' <- ep1 n $ os2ip m'-        return $ i2osp $ ep2 n m''+            m' <- pad seed oaep k m+            m'' <- ep1 n $ os2ip m'+            return $ i2osp $ ep2 n m''  -- | Encrypt plaintext using public key.-encrypt :: (HashAlgorithm hash, MonadRandom m)-        => OAEPParams hash ByteString ByteString    -- ^ OAEP padding parameters-        -> PublicKey                                -- ^ public key-        -> ByteString                               -- ^ plaintext -        -> m (Either Error ByteString)+encrypt+    :: (HashAlgorithm hash, MonadRandom m)+    => OAEPParams hash ByteString ByteString+    -- ^ OAEP padding parameters+    -> PublicKey+    -- ^ public key+    -> ByteString+    -- ^ plaintext+    -> m (Either Error ByteString) encrypt oaep pk m = do     seed <- getRandomBytes hashLen     return $ encryptWithSeed seed oaep pk m   where-    hashLen = hashDigestSize (oaepHash oaep)   +    hashLen = hashDigestSize (oaepHash oaep)  -- | Decrypt ciphertext using private key.-decrypt :: HashAlgorithm hash-        => OAEPParams hash ByteString ByteString    -- ^ OAEP padding parameters-        -> PrivateKey                               -- ^ private key-        -> ByteString                               -- ^ ciphertext-        -> Maybe ByteString-decrypt oaep pk c =-    let d  = private_d pk    -        n  = public_n $ private_pub pk-        k  = numBytes n-        c' = i2ospOf_ k $ dp2 n $ dp1 d n $ os2ip c-     in case unpad oaep k c' of-            Left _  -> Nothing-            Right p -> Just p   +--+-- The ciphertext has to be what 'encrypt' produces: the big-endian encoding,+-- with no leading zero octet, of a value below the modulus.  The primitives+-- work modulo n, so without that condition @c@ and @c + n@ -- and @c@ with a+-- zero octet in front of it -- would all decrypt to the same message, and a+-- ciphertext would not be unique to its plaintext.+decrypt+    :: HashAlgorithm hash+    => OAEPParams hash ByteString ByteString+    -- ^ OAEP padding parameters+    -> PrivateKey+    -- ^ private key+    -> ByteString+    -- ^ ciphertext+    -> Maybe ByteString+decrypt oaep pk c+    | os2ip c >= public_n (private_pub pk) = Nothing+    | c /= (i2osp (os2ip c) :: ByteString) = Nothing+    | otherwise =+        let d = private_d pk+            n = public_n $ private_pub pk+            k = numBytes n+            c' = i2ospOf_ k $ dp2 n $ dp1 d n $ os2ip c+         in case unpad oaep k c' of+                Left _ -> Nothing+                Right p -> Just p  -- | Sign message using hash algorithm and private key.-sign :: HashAlgorithm hash-     => PrivateKey  -- ^ private key-     -> hash        -- ^ hash function-     -> ByteString  -- ^ message to sign-     -> Either Error Integer+sign+    :: HashAlgorithm hash+    => PrivateKey+    -- ^ private key+    -> hash+    -- ^ hash function+    -> ByteString+    -- ^ message to sign+    -> Either Error Integer sign pk hashAlg m =     let d = private_d pk         n = public_n $ private_pub pk      in do-        m' <- ep1 n $ os2ip $ hashWith hashAlg m-        return $ dp1 d n m' +            m' <- ep1 n $ os2ip $ hashWith hashAlg m+            return $ dp1 d n m'  -- | Verify signature using hash algorithm and public key.-verify :: HashAlgorithm hash-       => PublicKey     -- ^ public key-       -> hash          -- ^ hash function-       -> ByteString    -- ^ message-       -> Integer       -- ^ signature-       -> Bool-verify pk hashAlg m s =-    let n  = public_n pk-        h  = os2ip $ hashWith hashAlg m-        h' = dp2 n $ ep2 n s-     in h' == h+verify+    :: HashAlgorithm hash+    => PublicKey+    -- ^ public key+    -> hash+    -- ^ hash function+    -> ByteString+    -- ^ message+    -> Integer+    -- ^ signature+    -> Bool+verify pk hashAlg m s+    -- squaring works modulo n, so s + n and -s would verify wherever s does+    | s < 0 || s >= n = False+    | otherwise =+        let h = os2ip $ hashWith hashAlg m+            h' = dp2 n $ ep2 n s+         in h' == h+  where+    n = public_n pk  -- | Encryption primitive 1 ep1 :: Integer -> Integer -> Either Error Integer ep1 n m =-    let m'   = 2*m + 1-        m''  = 2*m'-        m''' = 2*m''+    let m' = 2 * m + 1+        m'' = 2 * m'+        m''' = 2 * m''      in case jacobi m' n of             Just (-1) | m'' < n -> Right m''-            Just 1 | m''' < n   -> Right m'''-            _                   -> Left InvalidParameters+            Just 1 | m''' < n -> Right m'''+            _ -> Left InvalidParameters  -- | Encryption primitive 2 ep2 :: Integer -> Integer -> Integer@@ -157,10 +231,11 @@  -- | Decryption primitive 2 dp2 :: Integer -> Integer -> Integer-dp2 n c = let c'  = c `div` 2-              c'' = (n - c) `div` 2-           in case c `mod` 4 of-                0 -> ((c' `div` 2 - 1) `div` 2)-                1 -> ((c'' `div` 2 - 1) `div` 2)-                2 -> ((c' - 1) `div` 2)-                _ -> ((c'' - 1) `div` 2)+dp2 n c =+    let c' = c `div` 2+        c'' = (n - c) `div` 2+     in case c `mod` 4 of+            0 -> ((c' `div` 2 - 1) `div` 2)+            1 -> ((c'' `div` 2 - 1) `div` 2)+            2 -> ((c' - 1) `div` 2)+            _ -> ((c'' - 1) `div` 2)
Crypto/PubKey/Rabin/Types.hs view
@@ -4,40 +4,50 @@ -- Maintainer  : Carlos Rodriguez-Vega <crodveg@yahoo.es> -- Stability   : experimental -- Portability : unknown----module Crypto.PubKey.Rabin.Types-    ( Error(..)-    , generatePrimes-    ) where+module Crypto.PubKey.Rabin.Types (+    Error (..),+    PrimeCondition,+    generatePrimes,+) where  import Crypto.Number.Basic (numBits)-import Crypto.Number.Prime (generatePrime, findPrimeFromWith)+import Crypto.Number.Prime (findPrimeFromWith, generatePrime) import Crypto.Random.Types  type PrimeCondition = Integer -> Bool  -- | Error possible during encryption, decryption or signing.-data Error = MessageTooLong       -- ^ the message to encrypt is too long-           | MessageNotRecognized -- ^ the message decrypted doesn't have a OAEP structure-           | InvalidParameters    -- ^ some parameters lead to breaking assumptions-           deriving (Show, Eq)+data Error+    = -- | the message to encrypt is too long+      MessageTooLong+    | -- | the message decrypted doesn't have a OAEP structure+      MessageNotRecognized+    | -- | some parameters lead to breaking assumptions+      InvalidParameters+    deriving (Show, Eq)  -- | Generate primes p & q-generatePrimes :: MonadRandom m -               => Int                   -- ^ size in bytes          -               -> PrimeCondition        -- ^ condition prime p must satisfy-               -> PrimeCondition        -- ^ condition prime q must satisfy-               -> m (Integer, Integer)  -- ^ chosen distinct primes p and q+generatePrimes+    :: MonadRandom m+    => Int+    -- ^ size in bytes+    -> PrimeCondition+    -- ^ condition prime p must satisfy+    -> PrimeCondition+    -- ^ condition prime q must satisfy+    -> m (Integer, Integer)+    -- ^ chosen distinct primes p and q generatePrimes size pCond qCond =-    let pBits = (8*(size `div` 2))-        qBits = (8*(size - (size `div` 2)))+    let pBits = (8 * (size `div` 2))+        qBits = (8 * (size - (size `div` 2)))      in do-        p <- generatePrime' pBits pCond-        q <- generatePrime' qBits qCond-        return (p, q)-      where-        generatePrime' bits cond = do-            pr' <- generatePrime bits-            let pr = findPrimeFromWith cond pr'-            if numBits pr == bits then return pr+            p <- generatePrime' pBits pCond+            q <- generatePrime' qBits qCond+            return (p, q)+  where+    generatePrime' bits cond = do+        pr' <- generatePrime bits+        let pr = findPrimeFromWith cond pr'+        if numBits pr == bits+            then return pr             else generatePrime' bits cond
Crypto/Random.hs view
@@ -1,46 +1,55 @@+{-# LANGUAGE CPP #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.Random -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : stable -- Portability : good----{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.Random-    (+module Crypto.Random (     -- * Deterministic instances-      ChaChaDRG-    , SystemDRG-    , Seed+    ChaChaDRG,+    SystemDRG,+    Seed,+     -- * Seed-    , seedNew-    , seedFromInteger-    , seedToInteger-    , seedFromBinary+    seedNew,+    seedFromInteger,+    seedToInteger,+    seedFromBinary,+     -- * Deterministic Random class-    , getSystemDRG-    , drgNew-    , drgNewSeed-    , drgNewTest-    , withDRG-    , withRandomBytes-    , DRG(..)+    getSystemDRG,+    drgNew,+    drgNewSeed,+    drgNewTest,+    withDRG,+    withRandomBytes,+    DRG (..),+     -- * Random abstraction-    , MonadRandom(..)-    , MonadPseudoRandom-    ) where+    MonadRandom (..),+    MonadPseudoRandom,+) where  import Crypto.Error-import Crypto.Random.Types+import Crypto.Internal.Imports import Crypto.Random.ChaChaDRG import Crypto.Random.SystemDRG+import Crypto.Random.Types import Data.ByteArray (ByteArray, ByteArrayAccess, ScrubbedBytes) import qualified Data.ByteArray as B-import Crypto.Internal.Imports-import Crypto.Hash (Digest, SHA512, hash)  import qualified Crypto.Number.Serialize as Serialize +#ifdef INSECURE_ENTROPY+import Crypto.Hash (SHA512, Context)+import Crypto.Hash.IO+import Data.Memory.PtrMethods (memSet)+import Foreign.Ptr (Ptr, castPtr)+#endif+ newtype Seed = Seed ScrubbedBytes     deriving (ByteArrayAccess) @@ -50,27 +59,43 @@  -- | Create a new Seed from system entropy seedNew :: MonadRandom randomly => randomly Seed++#ifdef INSECURE_ENTROPY -- The degree of its randomness depends on the source, e.g. for iOS we -- have to compile with DoNotUseEntropy flag, as iOS doesn't allow -- using getentropy, and on some other systems it can be also -- potentially comprisable sources. Hashing of entropy before using -- it as a seed is a common mitigation for attacks via RNG/entropy -- source.-seedNew = (Seed . B.take seedLength . B.convert . (hash :: ScrubbedBytes -> Digest SHA512)) `fmap` getRandomBytes 64+seedNew = (Seed . scrubbedHash512) `fmap` getRandomBytes 64 +scrubbedHash512 :: ScrubbedBytes -> ScrubbedBytes+scrubbedHash512 = B.take seedLength . hash512+  where+    hash512 ba = B.unsafeCreate (hashDigestSize (undefined :: SHA512)) $ hashIO ba+    hashIO ba ptr = do+        ctx <- hashMutableInit+        hashMutableUpdate (ctx :: MutableContext SHA512) ba+        B.withByteArray ctx $ \pctx -> do+            hashInternalFinalize (castPtr pctx :: Ptr (Context SHA512)) ptr+            memSet pctx 0 $ hashInternalContextSize (undefined :: SHA512)+#else+seedNew = Seed `fmap` getRandomBytes seedLength+#endif+ -- | Convert a Seed to an integer seedToInteger :: Seed -> Integer seedToInteger (Seed b) = Serialize.os2ip b  -- | Convert an integer to a Seed seedFromInteger :: Integer -> Seed-seedFromInteger i = Seed $ Serialize.i2ospOf_ seedLength (i `mod` 2^(seedLength * 8))+seedFromInteger i = Seed $ Serialize.i2ospOf_ seedLength (i `mod` 2 ^ (seedLength * 8))  -- | Convert a binary to a seed seedFromBinary :: ByteArrayAccess b => b -> CryptoFailable Seed seedFromBinary b     | B.length b /= 40 = CryptoFailed (CryptoError_SeedSizeInvalid)-    | otherwise        = CryptoPassed $ Seed $ B.convert b+    | otherwise = CryptoPassed $ Seed $ B.convert b  -- | Create a new DRG from system entropy drgNew :: MonadRandom randomly => randomly ChaChaDRG@@ -102,4 +127,5 @@ -- This is equivalent to use Control.Arrow 'first' with 'randomBytesGenerate' withRandomBytes :: (ByteArray ba, DRG g) => g -> Int -> (ba -> a) -> (a, g) withRandomBytes rng len f = (f bs, rng')-  where (bs, rng') = randomBytesGenerate len rng+  where+    (bs, rng') = randomBytesGenerate len rng
Crypto/Random/ChaChaDRG.hs view
@@ -1,22 +1,26 @@+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+ -- | -- Module      : Crypto.Random.ChaChaDRG -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : stable -- Portability : good----{-# LANGUAGE GeneralizedNewtypeDeriving #-}-module Crypto.Random.ChaChaDRG-    ( ChaChaDRG-    , initialize-    , initializeWords-    ) where+module Crypto.Random.ChaChaDRG (+    ChaChaDRG,+    initialize,+    initializeWords,+) where -import           Crypto.Random.Types-import           Crypto.Internal.Imports-import           Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, ScrubbedBytes)+import Crypto.Internal.ByteArray (+    ByteArray,+    ByteArrayAccess,+    ScrubbedBytes,+ ) import qualified Crypto.Internal.ByteArray as B-import           Foreign.Storable (pokeElemOff)+import Crypto.Internal.Imports+import Crypto.Random.Types+import Foreign.Storable (pokeElemOff)  import qualified Crypto.Cipher.ChaCha as C @@ -29,18 +33,24 @@  -- | Initialize a new ChaCha context with the number of rounds, -- the key and the nonce associated.-initialize :: ByteArrayAccess seed-           => seed        -- ^ 40 bytes of seed-           -> ChaChaDRG   -- ^ the initial ChaCha state+initialize+    :: ByteArrayAccess seed+    => seed+    -- ^ 40 bytes of seed+    -> ChaChaDRG+    -- ^ the initial ChaCha state initialize seed = ChaChaDRG $ C.initializeSimple seed  -- | Initialize a new ChaCha context from 5-tuple of words64. -- This interface is useful when creating a RNG out of tests generators (e.g. QuickCheck). initializeWords :: (Word64, Word64, Word64, Word64, Word64) -> ChaChaDRG-initializeWords (a,b,c,d,e) = initialize (B.allocAndFreeze 40 fill :: ScrubbedBytes)-  where fill s = mapM_ (uncurry (pokeElemOff s)) [(0,a), (1,b), (2,c), (3,d), (4,e)]+initializeWords (a, b, c, d, e) = initialize (B.allocAndFreeze 40 fill :: ScrubbedBytes)+  where+    fill s = mapM_ (uncurry (pokeElemOff s)) [(0, a), (1, b), (2, c), (3, d), (4, e)]  generate :: ByteArray output => Int -> ChaChaDRG -> (output, ChaChaDRG) generate nbBytes st@(ChaChaDRG prevSt)     | nbBytes <= 0 = (B.empty, st)-    | otherwise    = let (output, newSt) = C.generateSimple prevSt nbBytes in (output, ChaChaDRG newSt)+    | otherwise =+        let (output, newSt) = C.generateSimple prevSt nbBytes+         in (output, ChaChaDRG newSt)
Crypto/Random/Entropy.hs view
@@ -4,16 +4,15 @@ -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----module Crypto.Random.Entropy-    ( getEntropy-    ) where+module Crypto.Random.Entropy (+    getEntropy,+) where -import           Data.Maybe (catMaybes)-import           Crypto.Internal.ByteArray (ByteArray)+import Crypto.Internal.ByteArray (ByteArray) import qualified Crypto.Internal.ByteArray as B+import Data.Maybe (catMaybes) -import           Crypto.Random.Entropy.Unsafe+import Crypto.Random.Entropy.Unsafe  -- | Get some entropy from the system source of entropy getEntropy :: ByteArray byteArray => Int -> IO byteArray
Crypto/Random/Entropy/RDRand.hs view
@@ -1,38 +1,39 @@+{-# LANGUAGE ForeignFunctionInterface #-}+ -- | -- Module      : Crypto.Random.Entropy.RDRand -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----{-# LANGUAGE ForeignFunctionInterface #-}-module Crypto.Random.Entropy.RDRand-    ( RDRand-    ) where+module Crypto.Random.Entropy.RDRand (+    RDRand,+) where -import Foreign.Ptr-import Foreign.C.Types-import Data.Word (Word8) import Crypto.Random.Entropy.Source+import Data.Word (Word8)+import Foreign.C.Types+import Foreign.Ptr  foreign import ccall unsafe "crypton_cpu_has_rdrand"-   c_cpu_has_rdrand :: IO CInt+    c_cpu_has_rdrand :: IO CInt  foreign import ccall unsafe "crypton_get_rand_bytes"-  c_get_rand_bytes :: Ptr Word8 -> CInt -> IO CInt+    c_get_rand_bytes :: Ptr Word8 -> CInt -> IO CInt  -- | Fake handle to Intel RDRand entropy CPU instruction data RDRand = RDRand  instance EntropySource RDRand where-    entropyOpen     = rdrandGrab+    entropyOpen = rdrandGrab     entropyGather _ = rdrandGetBytes-    entropyClose  _ = return ()+    entropyClose _ = return ()  rdrandGrab :: IO (Maybe RDRand) rdrandGrab = supported `fmap` c_cpu_has_rdrand-  where supported 0 = Nothing-        supported _ = Just RDRand+  where+    supported 0 = Nothing+    supported _ = Just RDRand  rdrandGetBytes :: Ptr Word8 -> Int -> IO Int rdrandGetBytes ptr sz = fromIntegral `fmap` c_get_rand_bytes ptr (fromIntegral sz)
Crypto/Random/Entropy/Source.hs view
@@ -4,19 +4,20 @@ -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good--- module Crypto.Random.Entropy.Source where -import Foreign.Ptr import Data.Word (Word8)+import Foreign.Ptr  -- | A handle to an entropy maker, either a system capability -- or a hardware generator. class EntropySource a where     -- | Try to open an handle for this source-    entropyOpen   :: IO (Maybe a)+    entropyOpen :: IO (Maybe a)+     -- | Try to gather a number of entropy bytes into a buffer.     -- Return the number of actual bytes gathered     entropyGather :: a -> Ptr Word8 -> Int -> IO Int+     -- | Close an open handle-    entropyClose  :: a -> IO ()+    entropyClose :: a -> IO ()
Crypto/Random/Entropy/Unix.hs view
@@ -1,29 +1,30 @@+{-# LANGUAGE ScopedTypeVariables #-}+ -- | -- Module      : Crypto.Random.Entropy.Unix -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----{-# LANGUAGE ScopedTypeVariables #-}-module Crypto.Random.Entropy.Unix-    ( DevRandom-    , DevURandom-    ) where+module Crypto.Random.Entropy.Unix (+    DevRandom,+    DevURandom,+) where -import Foreign.Ptr-import Data.Word (Word8)+import qualified Control.Exception as E import Crypto.Random.Entropy.Source-import Control.Exception as E+import Data.Word (Word8)+import Foreign.Ptr+import qualified System.IO.Error as E ---import System.Posix.Types (Fd)+-- import System.Posix.Types (Fd) import System.IO  type H = Handle type DeviceName = String  -- | Entropy device @/dev/random@ on unix system-newtype DevRandom  = DevRandom DeviceName+newtype DevRandom = DevRandom DeviceName  -- | Entropy device @/dev/urandom@ on unix system newtype DevURandom = DevURandom DeviceName@@ -32,43 +33,46 @@     entropyOpen = fmap DevRandom `fmap` testOpen "/dev/random"     entropyGather (DevRandom name) ptr n =         withDev name $ \h -> gatherDevEntropyNonBlock h ptr n-    entropyClose (DevRandom _)  = return ()+    entropyClose (DevRandom _) = return ()  instance EntropySource DevURandom where     entropyOpen = fmap DevURandom `fmap` testOpen "/dev/urandom"     entropyGather (DevURandom name) ptr n =         withDev name $ \h -> gatherDevEntropy h ptr n-    entropyClose (DevURandom _)  = return ()+    entropyClose (DevURandom _) = return ()  testOpen :: DeviceName -> IO (Maybe DeviceName) testOpen filepath = do     d <- openDev filepath     case d of         Nothing -> return Nothing-        Just h  -> closeDev h >> return (Just filepath)+        Just h -> closeDev h >> return (Just filepath)  openDev :: String -> IO (Maybe H)-openDev filepath = (Just `fmap` openAndNoBuffering) `E.catch` \(_ :: IOException) -> return Nothing-  where openAndNoBuffering = do-            h <- openBinaryFile filepath ReadMode-            hSetBuffering h NoBuffering-            return h+openDev filepath =+    (Just `fmap` openAndNoBuffering) `E.catchIOError` \_ -> return Nothing+  where+    openAndNoBuffering = do+        h <- openBinaryFile filepath ReadMode+        hSetBuffering h NoBuffering+        return h  withDev :: String -> (H -> IO a) -> IO a-withDev filepath f = openDev filepath >>= \h ->-    case h of-        Nothing -> error ("device " ++ filepath ++ " cannot be grabbed")-        Just fd -> f fd `E.finally` closeDev fd+withDev filepath f =+    openDev filepath >>= \h ->+        case h of+            Nothing -> error ("device " ++ filepath ++ " cannot be grabbed")+            Just fd -> f fd `E.finally` closeDev fd  closeDev :: H -> IO ()-closeDev h = hClose h `E.catch` \(_ :: IOException) -> return ()+closeDev h = hClose h `E.catchIOError` \_ -> return ()  gatherDevEntropy :: H -> Ptr Word8 -> Int -> IO Int gatherDevEntropy h ptr sz =-     (fromIntegral `fmap` hGetBufSome h ptr (fromIntegral sz))-    `E.catch` \(_ :: IOException) -> return 0+    (fromIntegral `fmap` hGetBufSome h ptr (fromIntegral sz))+        `E.catchIOError` \_ -> return 0  gatherDevEntropyNonBlock :: H -> Ptr Word8 -> Int -> IO Int gatherDevEntropyNonBlock h ptr sz =-     (fromIntegral `fmap` hGetBufNonBlocking h ptr (fromIntegral sz))-    `E.catch` \(_ :: IOException) -> return 0+    (fromIntegral `fmap` hGetBufNonBlocking h ptr (fromIntegral sz))+        `E.catchIOError` \_ -> return 0
Crypto/Random/Entropy/Unsafe.hs view
@@ -4,15 +4,14 @@ -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----module Crypto.Random.Entropy.Unsafe-    ( replenish-    , module Crypto.Random.Entropy.Backend-    ) where+module Crypto.Random.Entropy.Unsafe (+    replenish,+    module Crypto.Random.Entropy.Backend,+) where +import Crypto.Random.Entropy.Backend import Data.Word (Word8) import Foreign.Ptr (Ptr, plusPtr)-import Crypto.Random.Entropy.Backend  -- | Refill the entropy in a buffer --@@ -22,12 +21,14 @@ -- If the buffer cannot be refill after 3 loopings, this will raise -- an User Error exception replenish :: Int -> [EntropyBackend] -> Ptr Word8 -> IO ()-replenish _        []       _   = fail "crypton: random: cannot get any source of entropy on this system"+replenish _ [] _ = fail "crypton: random: cannot get any source of entropy on this system" replenish poolSize backends ptr = loop 0 backends ptr poolSize-  where loop :: Int -> [EntropyBackend] -> Ptr Word8 -> Int -> IO ()-        loop _     _  _ 0 = return ()-        loop retry [] p n | retry == 3 = error "crypton: random: cannot fully replenish"-                          | otherwise  = loop (retry+1) backends p n-        loop retry (b:bs) p n = do-            r <- gatherBackend b p n-            loop retry bs (p `plusPtr` r) (n - r)+  where+    loop :: Int -> [EntropyBackend] -> Ptr Word8 -> Int -> IO ()+    loop _ _ _ 0 = return ()+    loop retry [] p n+        | retry == 3 = error "crypton: random: cannot fully replenish"+        | otherwise = loop (retry + 1) backends p n+    loop retry (b : bs) p n = do+        r <- gatherBackend b p n+        loop retry bs (p `plusPtr` r) (n - r)
Crypto/Random/EntropyPool.hs view
@@ -4,22 +4,21 @@ -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----module Crypto.Random.EntropyPool-    ( EntropyPool-    , createEntropyPool-    , createEntropyPoolWith-    , getEntropyFrom-    ) where+module Crypto.Random.EntropyPool (+    EntropyPool,+    createEntropyPool,+    createEntropyPoolWith,+    getEntropyFrom,+) where -import           Control.Concurrent.MVar-import           Crypto.Random.Entropy.Unsafe-import           Crypto.Internal.ByteArray (ByteArray, ScrubbedBytes)+import Control.Concurrent.MVar+import Crypto.Internal.ByteArray (ByteArray, ScrubbedBytes) import qualified Crypto.Internal.ByteArray as B-import           Data.Word (Word8)-import           Data.Maybe (catMaybes)-import           Foreign.Marshal.Utils (copyBytes)-import           Foreign.Ptr (plusPtr, Ptr)+import Crypto.Random.Entropy.Unsafe+import Data.Maybe (catMaybes)+import Data.Word (Word8)+import Foreign.Marshal.Utils (copyBytes)+import Foreign.Ptr (Ptr, plusPtr)  -- | Pool of Entropy. Contains a self-mutating pool of entropy, -- that is always guaranteed to contain data.@@ -35,7 +34,7 @@ -- the pool can be shared between multiples RNGs. createEntropyPoolWith :: Int -> [EntropyBackend] -> IO EntropyPool createEntropyPoolWith poolSize backends = do-    m  <- newMVar 0+    m <- newMVar 0     sm <- B.alloc poolSize (replenish poolSize backends)     return $ EntropyPool backends m sm @@ -54,17 +53,18 @@     B.withByteArray sm $ \entropyPoolPtr ->         modifyMVar_ posM $ \pos ->             copyLoop outPtr entropyPoolPtr pos n-  where poolSize = B.length sm-        copyLoop d s pos left-            | left == 0 = return pos-            | otherwise = do-                wrappedPos <--                    if pos == poolSize-                        then replenish poolSize backends s >> return 0-                        else return pos-                let m = min (poolSize - wrappedPos) left-                copyBytes d (s `plusPtr` wrappedPos) m-                copyLoop (d `plusPtr` m) s (wrappedPos + m) (left - m)+  where+    poolSize = B.length sm+    copyLoop d s pos left+        | left == 0 = return pos+        | otherwise = do+            wrappedPos <-+                if pos == poolSize+                    then replenish poolSize backends s >> return 0+                    else return pos+            let m = min (poolSize - wrappedPos) left+            copyBytes d (s `plusPtr` wrappedPos) m+            copyLoop (d `plusPtr` m) s (wrappedPos + m) (left - m)  -- | Grab a chunk of entropy from the entropy pool. getEntropyFrom :: ByteArray byteArray => EntropyPool -> Int -> IO byteArray
+ Crypto/Random/HmacDRG.hs view
@@ -0,0 +1,51 @@+{-# LANGUAGE TypeApplications #-}++module Crypto.Random.HmacDRG (HmacDRG, initial, update) where++import Crypto.Hash+import Crypto.MAC.HMAC (HMAC (..), hmac)+import Crypto.Random.Types+import Data.ByteArray (ByteArrayAccess, Bytes, ScrubbedBytes)+import qualified Data.ByteArray as M+import Data.Maybe++-- | HMAC-based Deterministic Random Generator+--+-- Adapted from NIST Special Publication 800-90A Revision 1, Section 10.1.2+data HmacDRG hash = HmacDRG (Digest hash) (Digest hash)++-- | The initial DRG state. It should be seeded via 'update' before use.+initial :: HashAlgorithm hash => hash -> HmacDRG hash+initial algorithm = HmacDRG (constant 0x00) (constant 0x01)+  where+    constant =+        fromJust . digestFromByteString . M.replicate @Bytes (hashDigestSize algorithm)++-- | Update the DRG state with optional provided data.+update+    :: ByteArrayAccess input+    => HashAlgorithm hash+    => input -> HmacDRG hash -> HmacDRG hash+update input state0 = if M.null input then state1 else state2+  where+    state1 = step 0x00 state0+    state2 = step 0x01 state1+    step byte (HmacDRG key value) = HmacDRG keyNew valueNew+      where+        keyNew =+            hmacGetDigest $+                hmac key $+                    M.convert value <> M.singleton @ScrubbedBytes byte <> M.convert input+        valueNew = hmacGetDigest $ hmac keyNew value++instance HashAlgorithm hash => DRG (HmacDRG hash) where+    randomBytesGenerate count (HmacDRG key value) = (output, state)+      where+        output = M.take count result+        state = update @Bytes M.empty $ HmacDRG key new+        (result, new) = go M.empty value+        go buffer current+            | M.length buffer >= count = (buffer, current)+            | otherwise = go (buffer <> M.convert next) next+          where+            next = hmacGetDigest $ hmac key current
Crypto/Random/Probabilistic.hs view
@@ -4,25 +4,50 @@ -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----module Crypto.Random.Probabilistic-    ( probabilistic-    ) where+module Crypto.Random.Probabilistic (+    probabilisticFrom,+) where +import Crypto.Hash (SHA512 (..), hashWith)+import Crypto.Internal.ByteArray (ByteArrayAccess, ScrubbedBytes)+import qualified Crypto.Internal.ByteArray as B import Crypto.Internal.Compat-import Crypto.Random.Types import Crypto.Random+import Crypto.Random.ChaChaDRG (initialize) --- | This create a random number generator out of thin air with--- the system entropy; don't generally use as the IO is not exposed--- this can have unexpected random for.--- --- This is useful for probabilistic algorithm like Miller Rabin--- probably prime algorithm, given appropriate choice of the heuristic+-- | Run a probabilistic algorithm on a generator derived from the value it is+-- about to work on, and from a secret this process drew once. --+-- This is useful for a probabilistic algorithm like the Miller-Rabin primality+-- test, where the caller is a pure function and has to behave like one: the+-- same value has to give the same answer for as long as the process lives.+-- Deriving the generator from the value gives that much, and it keeps the+-- draws made for two different values independent of each other -- one+-- generator made once and shared by every call would make the witnesses drawn+-- for one value the witnesses for every value.+--+-- The process secret is what makes the derivation unpredictable.  The values+-- worked on may come from wherever the caller's input comes from, so the+-- generator must not be something that can be worked out from them.+--+-- The IO is not exposed and the result is not reproducible between processes. -- Generally, it's advised not to use this function.-probabilistic :: MonadPseudoRandom ChaChaDRG a -> a-probabilistic f = fst $ withDRG drg f-  where {-# NOINLINE drg #-}-        drg = unsafeDoIO drgNew-{-# NOINLINE probabilistic #-}+probabilisticFrom+    :: ByteArrayAccess seed+    => seed+    -- ^ the value being worked on, as bytes+    -> MonadPseudoRandom ChaChaDRG a+    -> a+probabilisticFrom material f = fst $ withDRG drg f+  where+    drg = initialize (B.take seedLength (B.convert digest :: ScrubbedBytes))+    digest = hashWith SHA512 (B.append secret (B.convert material) :: ScrubbedBytes)+    -- what Crypto.Random.ChaChaDRG.initialize wants, and no more than SHA-512+    -- produces+    seedLength = 40++-- | Drawn once, for the lifetime of the process: it is the only part of the+-- derivation above that an attacker supplying values cannot see.+secret :: ScrubbedBytes+secret = unsafeDoIO (getRandomBytes 32)+{-# NOINLINE secret #-}
Crypto/Random/SystemDRG.hs view
@@ -1,26 +1,26 @@+{-# LANGUAGE BangPatterns #-}+ -- | -- Module      : Crypto.Random.SystemDRG -- License     : BSD-style -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----{-# LANGUAGE BangPatterns #-}-module Crypto.Random.SystemDRG-    ( SystemDRG-    , getSystemDRG-    ) where+module Crypto.Random.SystemDRG (+    SystemDRG,+    getSystemDRG,+) where -import           Crypto.Random.Types-import           Crypto.Random.Entropy.Unsafe-import           Crypto.Internal.Compat-import           Data.ByteArray (ScrubbedBytes, ByteArray)-import           Data.Memory.PtrMethods as B (memCopy)-import           Data.Maybe (catMaybes)-import           Data.Tuple (swap)-import           Foreign.Ptr+import Crypto.Internal.Compat+import Crypto.Random.Entropy.Unsafe+import Crypto.Random.Types+import Data.ByteArray (ByteArray, ScrubbedBytes) import qualified Data.ByteArray as B-import           System.IO.Unsafe (unsafeInterleaveIO)+import Data.Maybe (catMaybes)+import Data.Memory.PtrMethods as B (memCopy)+import Data.Tuple (swap)+import Foreign.Ptr+import System.IO.Unsafe (unsafeInterleaveIO)  -- | A referentially transparent System representation of -- the random evaluated out of the system.@@ -43,21 +43,22 @@ getSystemDRG = do     backends <- catMaybes `fmap` sequence supportedBackends     let getNext = unsafeInterleaveIO $ do-            bs   <- B.alloc systemChunkSize (replenish systemChunkSize backends)+            bs <- B.alloc systemChunkSize (replenish systemChunkSize backends)             more <- getNext-            return (bs:more)+            return (bs : more)     SystemDRG 0 <$> getNext  generate :: ByteArray output => Int -> SystemDRG -> (output, SystemDRG) generate nbBytes (SystemDRG ofs sysChunks) = swap $ unsafeDoIO $ B.allocRet nbBytes $ loop ofs sysChunks nbBytes-  where loop currentOfs chunks 0 _ = return $! SystemDRG currentOfs chunks-        loop _          []     _ _ = error "SystemDRG: the impossible happened: empty chunk"-        loop currentOfs oChunks@(c:cs) n d = do-            let currentLeft = B.length c - currentOfs-                toCopy      = min n currentLeft-                nextOfs     = currentOfs + toCopy-                n'          = n - toCopy-            B.withByteArray c $ \src -> B.memCopy d (src `plusPtr` currentOfs) toCopy-            if nextOfs == B.length c-                then loop 0 cs n' (d `plusPtr` toCopy)-                else loop nextOfs oChunks n' (d `plusPtr` toCopy)+  where+    loop currentOfs chunks 0 _ = return $! SystemDRG currentOfs chunks+    loop _ [] _ _ = error "SystemDRG: the impossible happened: empty chunk"+    loop currentOfs oChunks@(c : cs) n d = do+        let currentLeft = B.length c - currentOfs+            toCopy = min n currentLeft+            nextOfs = currentOfs + toCopy+            n' = n - toCopy+        B.withByteArray c $ \src -> B.memCopy d (src `plusPtr` currentOfs) toCopy+        if nextOfs == B.length c+            then loop 0 cs n' (d `plusPtr` toCopy)+            else loop nextOfs oChunks n' (d `plusPtr` toCopy)
Crypto/Random/Types.hs view
@@ -4,17 +4,15 @@ -- Maintainer  : Vincent Hanquez <vincent@snarc.org> -- Stability   : experimental -- Portability : Good----module Crypto.Random.Types-    (-      MonadRandom(..)-    , MonadPseudoRandom-    , DRG(..)-    , withDRG-    ) where+module Crypto.Random.Types (+    MonadRandom (..),+    MonadPseudoRandom,+    DRG (..),+    withDRG,+) where -import Crypto.Random.Entropy import Crypto.Internal.ByteArray+import Crypto.Random.Entropy  -- | A monad constraint that allows to generate random bytes class Monad m => MonadRandom m where@@ -39,14 +37,14 @@         let (a, g2) = runPseudoRandom m g1 in (f a, g2)  instance DRG gen => Applicative (MonadPseudoRandom gen) where-    pure a     = MonadPseudoRandom $ \g -> (a, g)+    pure a = MonadPseudoRandom $ \g -> (a, g)     (<*>) fm m = MonadPseudoRandom $ \g1 ->         let (f, g2) = runPseudoRandom fm g1             (a, g3) = runPseudoRandom m g2          in (f a, g3)  instance DRG gen => Monad (MonadPseudoRandom gen) where-    return      = pure+    return = pure     (>>=) m1 m2 = MonadPseudoRandom $ \g1 ->         let (a, g2) = runPseudoRandom m1 g1          in runPseudoRandom (m2 a) g2@@ -55,6 +53,6 @@     getRandomBytes n = MonadPseudoRandom (randomBytesGenerate n)  -- | Run a pure computation with a Deterministic Random Generator--- in the 'MonadPseudoRandom'+-- in the t'MonadPseudoRandom' withDRG :: DRG gen => gen -> MonadPseudoRandom gen a -> (a, gen) withDRG gen m = runPseudoRandom m gen
Crypto/System/CPU.hs view
@@ -1,3 +1,7 @@+{-# LANGUAGE CPP #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE ForeignFunctionInterface #-}+ -- | -- Module      : Crypto.System.CPU -- License     : BSD-style@@ -6,14 +10,10 @@ -- Portability : unknown -- -- Gives information about crypton runtime environment.----{-# LANGUAGE CPP #-}-{-# LANGUAGE DeriveDataTypeable #-}-{-# LANGUAGE ForeignFunctionInterface #-}-module Crypto.System.CPU-    ( ProcessorOption (..)-    , processorOptions-    ) where+module Crypto.System.CPU (+    ProcessorOption (..),+    processorOptions,+) where  import Data.Data import Data.List (findIndices)@@ -33,10 +33,13 @@  -- | CPU options impacting cryptography implementation and library performance. data ProcessorOption-    = AESNI   -- ^ Support for AES instructions, with flag @support_aesni@-    | PCLMUL  -- ^ Support for CLMUL instructions, with flag @support_pclmuldq@-    | RDRAND  -- ^ Support for RDRAND instruction, with flag @support_rdrand@-    deriving (Show,Eq,Enum,Data)+    = -- | Support for AES instructions, with flag @support_aesni@+      AESNI+    | -- | Support for CLMUL instructions, with flag @support_pclmuldq@+      PCLMUL+    | -- | Support for RDRAND instruction, with flag @support_rdrand@+      RDRAND+    deriving (Show, Eq, Enum, Data)  -- | Options which have been enabled at compile time and are supported by the -- current CPU.@@ -44,11 +47,11 @@ processorOptions = unsafeDoIO $ do     p <- crypton_aes_cpu_init     options <- traverse (getOption p) aesOptions-    rdrand  <- hasRDRand-    return (decodeOptions options ++ [ RDRAND | rdrand ])+    rdrand <- hasRDRand+    return (decodeOptions options ++ [RDRAND | rdrand])   where-    aesOptions    = [ AESNI .. PCLMUL ]-    getOption p   = peekElemOff p . fromEnum+    aesOptions = [AESNI .. PCLMUL]+    getOption p = peekElemOff p . fromEnum     decodeOptions = map toEnum . findIndices (> 0) {-# NOINLINE processorOptions #-}
Crypto/Tutorial.hs view
@@ -1,22 +1,22 @@ -- | Examples of how to use @crypton@.-module Crypto.Tutorial-    ( -- * API design-      -- $api_design+module Crypto.Tutorial (+    -- * API design+    -- $api_design -      -- * Hash algorithms-      -- $hash_algorithms+    -- * Hash algorithms+    -- $hash_algorithms -      -- * Symmetric block ciphers-      -- $symmetric_block_ciphers+    -- * Symmetric block ciphers+    -- $symmetric_block_ciphers -      -- * Combining primitives-      -- $combining_primitives-    ) where+    -- * Combining primitives+    -- $combining_primitives+) where  -- $api_design -- -- APIs in crypton are often based on type classes from package--- <https://hackage.haskell.org/package/memory memory>, notably+-- <https://hackage.haskell.org/package/ram ram>, notably -- 'Data.ByteArray.ByteArrayAccess' and 'Data.ByteArray.ByteArray'. -- Module "Data.ByteArray" provides many primitives that are useful to -- work with crypton types.  For example function 'Data.ByteArray.convert'@@ -160,6 +160,7 @@ -- > import           Data.ByteString (ByteString) -- > import qualified Data.ByteString as B -- >+-- > import           Crypto.Error (throwCryptoError) -- > import qualified Crypto.Cipher.XSalsa as XSalsa -- > import qualified Crypto.MAC.Poly1305 as Poly1305 -- > import qualified Crypto.PubKey.Curve25519 as X25519@@ -175,7 +176,8 @@ -- >     state1       = XSalsa.derive state0 iv1 -- >     (rs, state2) = XSalsa.generate state1 32 -- >     (c, _)       = XSalsa.combine state2 content--- >     tag          = Poly1305.auth (rs :: ByteString) c+-- >     macKey       = throwCryptoError (Poly1305.key (rs :: ByteString))+-- >     tag          = Poly1305.auth macKey c -- > -- > -- | Try to open a @crypto_box@ packet and recover the content using the -- > -- 192-bit nonce, sender public key and receiver private key.@@ -192,4 +194,5 @@ -- >     state1       = XSalsa.derive state0 iv1 -- >     (rs, state2) = XSalsa.generate state1 32 -- >     (content, _) = XSalsa.combine state2 c--- >     tag          = Poly1305.auth (rs :: ByteString) c+-- >     macKey       = throwCryptoError (Poly1305.key (rs :: ByteString))+-- >     tag          = Poly1305.auth macKey c
LICENSE view
@@ -1,4 +1,5 @@ Copyright (c) 2006-2015 Vincent Hanquez <vincent@snarc.org>+Copyright (c) 2023-2026 Kazu Yamamoto <kazu@iij.ad.jp>  All rights reserved.
README.md view
@@ -3,91 +3,252 @@ crypton ========== -Crypton is a fork from cryptonite with the original author's permission.+`crypton` is a fork from `cryptonite` with the original author's permission. -Crypton is a haskell repository of cryptographic primitives. Each crypto-algorithm has specificities that are hard to wrap in common APIs and types,-so instead of trying to provide a common ground for algorithms, this package-provides a non-consistent low-level API. -If you have no idea what you're doing, please do not use this directly.-Instead, rely on higher level protocols or implementations.+`crypton` is a low-level cryptography library. To achieve high+performance, it utilizes C and assembly language to define FFI+bindings, structuring them in a way that makes them easy to use. -Documentation: [crypton on hackage](http://hackage.haskell.org/package/crypton) -Stability----------+Side channels+------------- -Crypton APIs are stable, and we only strive to add, not change or remove.-Note that because the API exposed is wide and also expose internals things (for-power users and flexibility), certains APIs can be revised in extreme cases-where we can't just add.+AES is where this matters most, and which implementation runs is decided at+runtime from what the processor has. -Versioning-----------+On x86-64 with AES-NI and carry-less multiply, and on AArch64 with the ARMv8+cryptographic extension, AES and GHASH are instructions rather than tables.+crypton's AES and AES-GCM then make no branch and no memory access that+depends on the key or on the data: the secrets stay in vector registers and+never reach one a branch can test, which the generated code is checked+against.  Every x86-64 part since about 2010 and every AArch64 part in+ordinary use has these. -Next version of `0.x` is `0.(x+1)`. There's no exceptions, or API related meaning-behind the numbers.+Where neither is present crypton falls back to a table-driven AES, which+indexes a 256-byte substitution table with data derived from the key and the+input.  **That is not constant time**, and on a machine where an attacker can+observe the cache it is open to a timing attack.  The fallback exists so that+the library builds and runs everywhere; it is not meant for a setting where+that matters. -Coding Style-------------+`Crypto.System.CPU.processorOptions` says which is in use.  `AESNI` in that+list means the instruction path, and `PCLMUL` that GHASH has its instruction+too; without `AESNI` it is the tables.  The list also reports `RDRAND`, which+is unrelated to this. -The coding style of this project mostly follows:-[haskell-style](https://github.com/tibbe/haskell-style-guide/blob/master/haskell-style.md)+    ghci> import Crypto.System.CPU+    ghci> processorOptions+    [AESNI,PCLMUL] -Support--------+RSA is the other place to know about, and there the choice is the caller's.+The private key operations in `Crypto.PubKey.RSA.PKCS15`, `.OAEP` and `.PSS`+take a `Maybe Blinder`, and `Nothing` is no harder to write than the safe+form: -See [Haskell packages guidelines](https://github.com/vincenthz/haskell-pkg-guidelines/blob/master/README.md#support)+    decrypt     :: Maybe Blinder -> PrivateKey -> ByteString -> ...+    decryptSafer :: MonadRandom m => PrivateKey -> ByteString -> m ... -Known Building Issues----------------------+The exponent itself is not what is at risk.  `expSafe` keeps the *value* of an+exponent out of the work it does, so the private exponent does not leak+through the exponentiation.  What a blinder covers is the other side: without+one, the operation runs on the ciphertext the caller was handed, so how long+it takes depends on a number an attacker may have chosen and can vary.  That+is what a remote timing attack on RSA needs.  With a blinder the input is+multiplied by a random value first and the result divided out afterwards, so+the timing carries nothing an attacker can steer. -On OSX <= 10.7, the system compiler doesn't understand the '-maes' option, and-with the lack of autodetection feature builtin in .cabal file, it is left on-the user to disable the aesni. See the [Disabling AESNI] section+`decryptSafer` and `signSafer` generate the blinder themselves and are the+ones to reach for.  Pass `Nothing` only where the input is not attacker+controlled and you have decided that it is not. -On CentOS 7 the default C compiler includes intrinsic header files incompatible-with per-function target options.  Solutions are to use GCC >= 4.9 or disable-flag *use_target_attributes* (see flag configuration examples below).+The RSA rows in the tables below are the unblinded path.  A blinder costs one+more exponentiation, by the public exponent, which is the cheap direction:+measured on the M4, signing goes from about 460 to about 476 microseconds,+under four per cent. -Disabling AESNI----------------+Performance+----------- -It may be useful to disable AESNI for building, testing or runtime purposes.-This is achieved with the *support_aesni* flag.+The algorithms a TLS connection uses, measured against the last release+before the rewrite and against OpenSSL on the same machine.  Throughput is+over 16 KiB messages; the public key operations are one operation each; every+figure is the best of several runs, and crypton and OpenSSL are run+alternately so that neither gets the quieter machine. -As part of configure of crypton:+Bulk encryption and hashing are measured through crypton's C layer, as+`openssl speed` measures OpenSSL's.  The public key operations are measured+through crypton's Haskell API, since that is where ECDSA and RSA live and it+is what a program actually calls; the Haskell layer adds well under a+microsecond, which the X25519 and ECDH P-256 rows confirm by agreeing with a+C-level measurement to within a percent.  Both releases of crypton are built+the same way -- `-optc-O3`, which is what each asks for -- and by+`cabal build`, since a copy of the sources compiled by hand does not measure+what a program linking the library gets, and leaves out whole implementations+without saying so.  Each column of a table comes from one run on the machine+named above it. -```-  cabal configure --flag='-support_aesni'-```+### x86-64 -or as part of an installation:+An AMD EPYC 7763, which has AES-NI, PCLMULQDQ, AVX2, ADX, VAES, VPCLMULQDQ+and the SHA extensions, against OpenSSL 4.0.3. -```-  cabal install --constraint="crypton -support_aesni"-```+Throughput in MB/s, **higher is better**: -For help with cabal flags, see: [stackoverflow : is there a way to define flags for cabal](http://stackoverflow.com/questions/23523869/is-there-any-way-to-define-flags-for-cabal-dependencies)+| | crypton 1.1.5 | crypton 2.1.5 | OpenSSL | 2.1.5 / OpenSSL |+| --- | ---: | ---: | ---: | ---: |+| AES-128-GCM | 1362 | **6038** | 4055 | 1.49 |+| AES-256-GCM | 1093 | **5462** | 3770 | 1.45 |+| ChaCha20-Poly1305 | 399 | 2211 | 2229 | 0.99 |+| SHA-1 | 727 | 1678 | 1673 | 1.00 |+| SHA-256 | 290 | 1585 | 1579 | 1.00 |+| SHA-512 | 463 | 804 | 751 | 1.07 |+| SHA3-256 | 109 | 424 | 425 | 1.00 | -Links------+Time per operation in microseconds, **lower is better**: -* [ChaCha](http://cr.yp.to/chacha.html)-* [ChaCha-test-vectors](https://github.com/secworks/chacha_testvectors.git)-* [Poly1305](http://cr.yp.to/mac.html)-* [Poly1305-test-vectors](http://tools.ietf.org/html/draft-nir-cfrg-chacha20-poly1305-06#page-12)-* [Salsa](http://cr.yp.to/snuffle.html)-* [Salsa128-test-vectors](https://github.com/alexwebr/salsa20/blob/master/test_vectors.128)-* [Salsa256-test-vectors](https://github.com/alexwebr/salsa20/blob/master/test_vectors.256)-* [XSalsa](https://cr.yp.to/snuffle/xsalsa-20081128.pdf)-* [PBKDF2](http://tools.ietf.org/html/rfc2898)-* [PBKDF2-test-vectors](http://www.ietf.org/rfc/rfc6070.txt)-* [Scrypt](http://www.tarsnap.com/scrypt.html)-* [Curve25519](http://cr.yp.to/ecdh.html)-* [Ed25519](http://ed25519.cr.yp.to/papers.html)-* [Ed448-Goldilocks](http://ed448goldilocks.sourceforge.net/)-* [EdDSA-test-vectors](http://www.ietf.org/rfc/rfc8032.txt)-* [AFIS](http://clemens.endorphin.org/cryptography)+| | crypton 1.1.5 | crypton 2.1.5 | OpenSSL | OpenSSL / 2.1.5 |+| --- | ---: | ---: | ---: | ---: |+| X25519 | 45.31 | 28.41 | 36.48 | 1.28 |+| ECDH P-256 | 165.4 | 51.14 | 51.65 | 1.01 |+| ECDH P-384 | 2278 | **165.0** | 847.5 | 5.13 |+| Ed25519 sign | 30.03 | 18.62 | 33.71 | 1.81 |+| Ed25519 verify | 48.05 | 47.66 | 110.6 | 2.32 |+| ECDSA P-256 sign | 81.70 | 18.96 | 21.87 | 1.15 |+| ECDSA P-256 verify | 233.3 | 70.47 | 67.52 | 0.96 |+| ECDSA P-384 sign | 2264 | **303.4** | 890.1 | 2.93 |+| ECDSA P-384 verify | 2676 | **471.4** | 721.5 | 1.53 |+| RSA-2048 sign/decrypt | 759.4 | 612.0 | 659.4 | 1.08 |+| RSA-2048 verify/encrypt | 33.56 | 30.21 | 18.86 | 0.62 | +### AArch64++An Apple M4, which has the AES, PMULL, SHA-1, SHA-2, SHA-512 and SHA-3+instructions, against OpenSSL 4.0.3.++Throughput in MB/s, **higher is better**:++| | crypton 1.1.5 | crypton 2.1.5 | OpenSSL | 2.1.5 / OpenSSL |+| --- | ---: | ---: | ---: | ---: |+| AES-128-GCM | 127 | **12422** | 10846 | 1.15 |+| AES-256-GCM | 98 | **9721** | 9197 | 1.06 |+| ChaCha20-Poly1305 | 771 | 2319 | 2250 | 1.03 |+| SHA-1 | 1209 | 3389 | 3361 | 1.01 |+| SHA-256 | 474 | 3400 | 3362 | 1.01 |+| SHA-512 | 730 | 1880 | 1883 | 1.00 |+| SHA3-256 | 550 | 1075 | 1065 | 1.01 |++Time per operation in microseconds, **lower is better**:++| | crypton 1.1.5 | crypton 2.1.5 | OpenSSL | OpenSSL / 2.1.5 |+| --- | ---: | ---: | ---: | ---: |+| X25519 | 18.27 | **12.22** | 15.53 | 1.27 |+| ECDH P-256 | 68.70 | **20.43** | 24.77 | 1.21 |+| ECDH P-384 | 3328 | **73.50** | 372.6 | 5.07 |+| Ed25519 sign | 13.58 | **7.75** | 13.23 | 1.71 |+| Ed25519 verify | 18.28 | 18.17 | 34.76 | 1.91 |+| ECDSA P-256 sign | 31.97 | **6.55** | 10.92 | 1.67 |+| ECDSA P-256 verify | 95.63 | **26.80** | 32.68 | 1.22 |+| ECDSA P-384 sign | 3219 | **124.1** | 394.2 | 3.18 |+| ECDSA P-384 verify | 3870 | **203.1** | 326.5 | 1.61 |+| RSA-2048 sign/decrypt | 447.9 | 460.1 | 319.9 | 0.70 |+| RSA-2048 verify/encrypt | 18.23 | 15.12 | 8.405 | 0.56 |++### What the numbers say++There are two changes behind the 1.1.5 column and the 2.1.5 one, not a+single steady improvement.++The first, in 2.0.0, was a rewrite: the bulk algorithms moved into C, the+curves other than P-256 moved out of Haskell `Integer` arithmetic, and+everything that touches a secret was made to take the same time whatever the+secret is.  1.1.5 had no AArch64 code of its own at all, which is why AES-GCM+there is close to a hundred times what it was, and on x86-64 it had AES-NI+and nothing else.++The second, from 2.1.0 onwards, is assembly, for the operations where C+cannot reach.  Which of the two a row owes its gain to is not the same+everywhere: ECDSA P-384 signing took nineteenfold from the rewrite and a+further fifth from the assembly, while X25519 waited for the assembly+entirely and ECDH P-384 is almost all of it.++Most of that assembly is not crypton's.  The prime curves, the inverse modulo+a group order, X25519, and RSA's Montgomery multiplication on x86-64 go+through [s2n-bignum](https://github.com/awslabs/s2n-bignum), vendored in+`cbits/s2n`.  Every routine in it carries a machine-checked proof in+HOL-Light that it computes what it says, and is written in a constant-time+style.  It is `Apache-2.0 OR ISC OR MIT-0`, and crypton takes it under ISC.++That licence is why any of this was possible.  The obvious assembly to reach+for is OpenSSL's and BoringSSL's `ecp_nistz256`, and it cannot be used here:+it is Apache-2.0 only, and Intel and CloudFlare hold copyright in it besides+OpenSSL, so nobody is in a position to relicense it.++Where crypton is behind, which is now the RSA rows on both architectures and+ECDSA P-256 verification on x86-64, there is one reason.  The AES-GCM rows+were the other half of this section until 2.1.5; they are ahead on both+machines now, and what the instructions do is still worth setting out.++*RSA.*  2.0.0 made signing slower than 1.1.5 on purpose: its modular+exponentiation stopped indexing a table with the bits of the exponent, and+hiding the exponent is what the difference bought.  On x86-64 that cost is+more than repaid -- s2n-bignum's Montgomery multiplication is twice the C's,+because the C cannot form the two carry chains `ADCX` and `ADOX` give, and+2.1.5 signs in less than 1.1.5 took while keeping what 2.0.0 gained.  On+AArch64 there is nothing to use: s2n-bignum has no generic routine for it,+and the same five that help on x86-64 measure level with the C there, so the+C stays and the gap with it.  No portable C closes that gap either -- the+measurements are in+[#275](https://github.com/kazu-yamamoto/crypton/issues/275).  Verification+does not move much either way: its exponent is 65537, seventeen bits, and+there is no exponentiation to speak of.++*The wide AES instructions.*  `VAES` and `VPCLMULQDQ` do two blocks where+`AES-NI` and `PCLMULQDQ` do one, and four in their 512-bit form.  crypton uses+the 256-bit form where the processor has it, which is Zen 3 and Ice Lake+onwards, and the 512-bit form where that is worth having, which is Ice Lake and+Zen 5 onwards.  There was nothing to borrow: the wide AES-GCM in OpenSSL,+BoringSSL and AWS-LC is Apache-2.0 and s2n-bignum has no GCM, so both files are+crypton's own.++Having the 256-bit one is where the 1.49 in the x86-64 table comes from, and+it is narrower than it sounds.  The EPYC 7763 is Zen 3: VAES and VPCLMULQDQ,+no AVX-512.  OpenSSL's x86-64 AES-GCM is `aesni-gcm-x86_64.pl`, which is+128-bit -- its `vaesenc`s are the VEX encoding of `AESENC` on `xmm`, and+there is not one `ymm` in the file -- or `aes-gcm-avx512.pl`, which wants+`AVX512VAES`.  There is no rung between them, so on this processor OpenSSL+takes a block at a time where crypton takes two.  The same idea as theirs,+one step further down the feature ladder; not a better one.++The 512-bit path arrived after 2.1.2, so it is in the 2.1.5 column -- but+neither machine in the tables above has AVX-512, so neither column shows it.+On the runners that do, measured over 16 KiB in MB/s: an EPYC 9V45 (Zen 5)+goes from 9616 to 14268 with it, a Xeon 6973P-C from 8095 to 9848, a Xeon+8573C from 6983 to 8447.  OpenSSL on those machines is ahead still -- 25760+on the first of them -- because it interleaves the GHASH with the AES where+crypton does them in turn.  Zen 4 keeps the 256-bit path: its 512-bit+instructions are two passes through a 256-bit datapath, so the wider encoding+buys nothing there and costs a little.++AArch64 has no counterpart to any of these: one AES block and one GHASH+multiplication at a time is all the instruction set offers.  Its AES-GCM+rows were 0.85 and 0.87 until 2.1.5, for that reason.  What closed it was+not width but the GHASH's representation -- H is twisted once at key setup+so that GCM's bit reflection is already undone, which turns a reduction of+some twenty-five shifts and XORs into two PMULL and six EOR and makes+Karatsuba worth taking.  The scheme is ARM's, from the BSD-3-Clause part of+[AArch64cryptolib](https://github.com/ARM-software/AArch64cryptolib),+written out in crypton's own intrinsics.  The AES there is ahead of+OpenSSL's and always was; it was the GHASH beside it that was behind.++One row wants a word of its own: crypton's `Ed25519.sign` derives the public+key from the secret key every time it signs, so that a caller who passes a+public key that does not match cannot be made to leak the private one.  That+costs a second scalar multiplication, which OpenSSL's signing does not pay --+and the row is still 1.71 on AArch64 and 1.81 on x86-64, so the safety is had+for nothing here rather than paid for.++SHA-1 is in the tables because a number of protocols and file formats still+ask for it, not because it is a good choice for anything new.  The algorithms+that nothing should ask for any more -- MD5, 3DES, RC4, CBC mode -- are left+out.
Setup.hs view
@@ -1,2 +1,3 @@ import Distribution.Simple+ main = defaultMain
benchs/Bench.hs view
@@ -1,43 +1,44 @@-{-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE ExistentialQuantification #-} {-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE TypeFamilies #-}+ module Main where -import Gauge.Main+import Test.Tasty.Bench -import           Crypto.Cipher.AES+import Crypto.Cipher.AES import qualified Crypto.Cipher.AESGCMSIV as AESGCMSIV-import           Crypto.Cipher.Blowfish-import           Crypto.Cipher.CAST5+import Crypto.Cipher.Blowfish+import Crypto.Cipher.CAST5 import qualified Crypto.Cipher.ChaChaPoly1305 as CP-import           Crypto.Cipher.DES-import           Crypto.Cipher.Twofish-import           Crypto.Cipher.Types-import           Crypto.ECC-import           Crypto.Error-import           Crypto.Hash+import Crypto.Cipher.DES+import Crypto.Cipher.Twofish+import Crypto.Cipher.Types+import Crypto.ECC+import Crypto.Error+import Crypto.Hash import qualified Crypto.KDF.BCrypt as BCrypt import qualified Crypto.KDF.PBKDF2 as PBKDF2-import           Crypto.Number.Basic (numBits)-import           Crypto.Number.Generate+import Crypto.Number.Basic (numBits)+import Crypto.Number.Generate import qualified Crypto.PubKey.DH as DH-import qualified Crypto.PubKey.ECC.Types as ECC import qualified Crypto.PubKey.ECC.Prim as ECC+import qualified Crypto.PubKey.ECC.Types as ECC import qualified Crypto.PubKey.ECDSA as ECDSA import qualified Crypto.PubKey.Ed25519 as Ed25519 import qualified Crypto.PubKey.EdDSA as EdDSA-import           Crypto.Random+import Crypto.Random -import           Control.DeepSeq (NFData)-import           Data.ByteArray (ByteArray, Bytes)+import Control.DeepSeq (NFData)+import Data.ByteArray (ByteArray, Bytes) import qualified Data.ByteString as B  import qualified Crypto.PubKey.ECC.P256 as P256  import Number.F2m -data HashAlg = forall alg . HashAlgorithm alg => HashAlg alg+data HashAlg = forall alg. HashAlgorithm alg => HashAlg alg  benchHash =     [ env oneKB $ \b -> bgroup "1KB" $ map (doHashBench b) hashAlgs@@ -65,13 +66,13 @@         , ("SHA512t_256", HashAlg SHA512t_256)         , ("RIPEMD160", HashAlg RIPEMD160)         , ("Tiger", HashAlg Tiger)-        --, ("Skein256-160", HashAlg Skein256_160)-        , ("Skein256-256", HashAlg Skein256_256)-        --, ("Skein512-160", HashAlg Skein512_160)-        , ("Skein512-384", HashAlg Skein512_384)+        , -- , ("Skein256-160", HashAlg Skein256_160)+          ("Skein256-256", HashAlg Skein256_256)+        , -- , ("Skein512-160", HashAlg Skein512_160)+          ("Skein512-384", HashAlg Skein512_384)         , ("Skein512-512", HashAlg Skein512_512)-        --, ("Skein512-896", HashAlg Skein512_896)-        , ("Whirlpool", HashAlg Whirlpool)+        , -- , ("Skein512-896", HashAlg Skein512_896)+          ("Whirlpool", HashAlg Whirlpool)         , ("Keccak-224", HashAlg Keccak_224)         , ("Keccak-256", HashAlg Keccak_256)         , ("Keccak-384", HashAlg Keccak_384)@@ -91,82 +92,99 @@         ]  benchPBKDF2 =-    [ bgroup "64"+    [ bgroup+        "64"         [ bench "cryptonite-PBKDF2-100-64" $ nf (pbkdf2 64) 100         , bench "cryptonite-PBKDF2-1000-64" $ nf (pbkdf2 64) 1000         , bench "cryptonite-PBKDF2-10000-64" $ nf (pbkdf2 64) 10000         ]-    , bgroup "128"+    , bgroup+        "128"         [ bench "cryptonite-PBKDF2-100-128" $ nf (pbkdf2 128) 100         , bench "cryptonite-PBKDF2-1000-128" $ nf (pbkdf2 128) 1000         , bench "cryptonite-PBKDF2-10000-128" $ nf (pbkdf2 128) 10000         ]     ]   where-        pbkdf2 :: Int -> Int -> B.ByteString-        pbkdf2 n iter = PBKDF2.generate (PBKDF2.prfHMAC SHA512) (params n iter) mypass mysalt+    pbkdf2 :: Int -> Int -> B.ByteString+    pbkdf2 n iter = PBKDF2.generate (PBKDF2.prfHMAC SHA512) (params n iter) mypass mysalt -        mypass, mysalt :: B.ByteString-        mypass = "password"-        mysalt = "salt"+    mypass, mysalt :: B.ByteString+    mypass = "password"+    mysalt = "salt" -        params n iter = PBKDF2.Parameters iter n+    params n iter = PBKDF2.Parameters iter n  benchBCrypt =-    [ bench "cryptonite-BCrypt-4"  $ nf bcrypt 4-    , bench "cryptonite-BCrypt-5"  $ nf bcrypt 5-    , bench "cryptonite-BCrypt-7"  $ nf bcrypt 7+    [ bench "cryptonite-BCrypt-4" $ nf bcrypt 4+    , bench "cryptonite-BCrypt-5" $ nf bcrypt 5+    , bench "cryptonite-BCrypt-7" $ nf bcrypt 7     , bench "cryptonite-BCrypt-11" $ nf bcrypt 11     ]   where-        bcrypt :: Int -> B.ByteString-        bcrypt cost = BCrypt.bcrypt cost mysalt mypass+    bcrypt :: Int -> B.ByteString+    bcrypt cost = BCrypt.bcrypt cost mysalt mypass -        mypass, mysalt :: B.ByteString-        mypass = "password"-        mysalt = "saltsaltsaltsalt"+    mypass, mysalt :: B.ByteString+    mypass = "password"+    mysalt = "saltsaltsaltsalt"  benchBlockCipher =     [ bgroup "ECB" benchECB     , bgroup "CBC" benchCBC     ]   where-        benchECB =-            [ bench "DES-input=1024" $ nf (run (undefined :: DES) cipherInit key8) input1024-            , bench "Blowfish128-input=1024" $ nf (run (undefined :: Blowfish128) cipherInit key16) input1024-            , bench "Twofish128-input=1024" $ nf (run (undefined :: Twofish128) cipherInit key16) input1024-            , bench "CAST5-128-input=1024" $ nf (run (undefined :: CAST5) cipherInit key16) input1024-            , bench "AES128-input=1024" $ nf (run (undefined :: AES128) cipherInit key16) input1024-            , bench "AES256-input=1024" $ nf (run (undefined :: AES256) cipherInit key32) input1024-            ]-          where run :: (ByteArray ba, ByteArray key, BlockCipher c)-                    => c -> (key -> CryptoFailable c) -> key -> ba -> ba-                run _witness initF key input =-                    (ecbEncrypt (throwCryptoError (initF key))) input+    benchECB =+        [ bench "DES-input=1024" $ nf (run (undefined :: DES) cipherInit key8) input1024+        , bench "Blowfish128-input=1024" $+            nf (run (undefined :: Blowfish128) cipherInit key16) input1024+        , bench "Twofish128-input=1024" $+            nf (run (undefined :: Twofish128) cipherInit key16) input1024+        , bench "CAST5-128-input=1024" $+            nf (run (undefined :: CAST5) cipherInit key16) input1024+        , bench "AES128-input=1024" $+            nf (run (undefined :: AES128) cipherInit key16) input1024+        , bench "AES256-input=1024" $+            nf (run (undefined :: AES256) cipherInit key32) input1024+        ]+      where+        run+            :: (ByteArray ba, ByteArray key, BlockCipher c)+            => c -> (key -> CryptoFailable c) -> key -> ba -> ba+        run _witness initF key input =+            (ecbEncrypt (throwCryptoError (initF key))) input -        benchCBC =-            [ bench "DES-input=1024" $ nf (run (undefined :: DES) cipherInit key8 iv8) input1024-            , bench "Blowfish128-input=1024" $ nf (run (undefined :: Blowfish128) cipherInit key16 iv8) input1024-            , bench "Twofish128-input=1024" $ nf (run (undefined :: Twofish128) cipherInit key16 iv16) input1024-            , bench "CAST5-128-input=1024" $ nf (run (undefined :: CAST5) cipherInit key16 iv8) input1024-            , bench "AES128-input=1024" $ nf (run (undefined :: AES128) cipherInit key16 iv16) input1024-            , bench "AES256-input=1024" $ nf (run (undefined :: AES256) cipherInit key32 iv16) input1024-            ]-          where run :: (ByteArray ba, ByteArray key, BlockCipher c)-                    => c -> (key -> CryptoFailable c) -> key -> IV c -> ba -> ba-                run _witness initF key iv input =-                    (cbcEncrypt (throwCryptoError (initF key))) iv input+    benchCBC =+        [ bench "DES-input=1024" $+            nf (run (undefined :: DES) cipherInit key8 iv8) input1024+        , bench "Blowfish128-input=1024" $+            nf (run (undefined :: Blowfish128) cipherInit key16 iv8) input1024+        , bench "Twofish128-input=1024" $+            nf (run (undefined :: Twofish128) cipherInit key16 iv16) input1024+        , bench "CAST5-128-input=1024" $+            nf (run (undefined :: CAST5) cipherInit key16 iv8) input1024+        , bench "AES128-input=1024" $+            nf (run (undefined :: AES128) cipherInit key16 iv16) input1024+        , bench "AES256-input=1024" $+            nf (run (undefined :: AES256) cipherInit key32 iv16) input1024+        ]+      where+        run+            :: (ByteArray ba, ByteArray key, BlockCipher c)+            => c -> (key -> CryptoFailable c) -> key -> IV c -> ba -> ba+        run _witness initF key iv input =+            (cbcEncrypt (throwCryptoError (initF key))) iv input -        key8  = B.replicate 8 0-        key16 = B.replicate 16 0-        key32 = B.replicate 32 0-        input1024 = B.replicate 1024 0+    key8 = B.replicate 8 0+    key16 = B.replicate 16 0+    key32 = B.replicate 32 0+    input1024 = B.replicate 1024 0 -        iv8 :: BlockCipher c => IV c-        iv8  = maybe (error "iv size 8") id  $ makeIV key8+    iv8 :: BlockCipher c => IV c+    iv8 = maybe (error "iv size 8") id $ makeIV key8 -        iv16 :: BlockCipher c => IV c-        iv16 = maybe (error "iv size 16") id $ makeIV key16+    iv16 :: BlockCipher c => IV c+    iv16 = maybe (error "iv size 16") id $ makeIV key16  benchAE =     [ bench "ChaChaPoly1305" $ nf (cp key32) (input64, input1024)@@ -174,81 +192,94 @@     , bench "AES-CCM" $ nf (ccm key32) (input64, input1024)     , bench "AES-GCM-SIV" $ nf (gcmsiv key32) (input64, input1024)     ]-  where cp k (ini, plain) =-            let iniState            = throwCryptoError $ CP.initialize k (throwCryptoError $ CP.nonce12 nonce12)-                afterAAD            = CP.finalizeAAD (CP.appendAAD ini iniState)-                (out, afterEncrypt) = CP.encrypt plain afterAAD-                outtag              = CP.finalize afterEncrypt-             in (outtag, out)+  where+    cp k (ini, plain) =+        let iniState =+                CP.initialize+                    (throwCryptoError $ CP.key k)+                    (throwCryptoError $ CP.nonce12 nonce12)+            afterAAD = CP.finalizeAAD (CP.appendAAD ini iniState)+            (out, afterEncrypt) = CP.encrypt plain afterAAD+            outtag = CP.finalize afterEncrypt+         in (outtag, out) -        gcm k (ini, plain) =-            let ctx = throwCryptoError (cipherInit k) :: AES256-                state = throwCryptoError $ aeadInit AEAD_GCM ctx nonce12-             in aeadSimpleEncrypt state ini plain 16+    gcm k (ini, plain) =+        let ctx = throwCryptoError (cipherInit k) :: AES256+            state = throwCryptoError $ aeadInit AEAD_GCM ctx nonce12+         in aeadSimpleEncrypt state ini plain 16 -        ccm k (ini, plain) =-            let ctx = throwCryptoError (cipherInit k) :: AES256-                mode = AEAD_CCM 1024 CCM_M16 CCM_L3-                state = throwCryptoError $ aeadInit mode ctx nonce12-             in aeadSimpleEncrypt state ini plain 16+    ccm k (ini, plain) =+        let ctx = throwCryptoError (cipherInit k) :: AES256+            mode = AEAD_CCM 1024 CCM_M16 CCM_L3+            state = throwCryptoError $ aeadInit mode ctx nonce12+         in aeadSimpleEncrypt state ini plain 16 -        gcmsiv k (ini, plain) =-            let ctx = throwCryptoError (cipherInit k) :: AES256-                iv = throwCryptoError (AESGCMSIV.nonce nonce12)-             in AESGCMSIV.encrypt ctx iv ini plain+    gcmsiv k (ini, plain) =+        let ctx = throwCryptoError (cipherInit k) :: AES256+            iv = throwCryptoError (AESGCMSIV.nonce nonce12)+         in AESGCMSIV.encrypt ctx iv ini plain -        input64 = B.replicate 64 0-        input1024 = B.replicate 1024 0+    input64 = B.replicate 64 0+    input1024 = B.replicate 1024 0 -        nonce12 :: B.ByteString-        nonce12 = B.replicate 12 0+    nonce12 :: B.ByteString+    nonce12 = B.replicate 12 0 -        key32 = B.replicate 32 0+    key32 = B.replicate 32 0  benchECC =-    [ bench "pointAddTwoMuls-baseline"  $ nf run_b (n1, p1, n2, p2)+    [ bench "pointAddTwoMuls-baseline" $ nf run_b (n1, p1, n2, p2)     , bench "pointAddTwoMuls-optimized" $ nf run_o (n1, p1, n2, p2)     , bench "pointAdd-ECC" $ nf run_c (p1, p2)     , bench "pointMul-ECC" $ nf run_d (n1, p2)     ]-  where run_b (n, p, k, q) = ECC.pointAdd c (ECC.pointMul c n p)-                                            (ECC.pointMul c k q)+  where+    run_b (n, p, k, q) =+        ECC.pointAdd+            c+            (ECC.pointMul c n p)+            (ECC.pointMul c k q) -        run_o (n, p, k, q) = ECC.pointAddTwoMuls c n p k q-        run_c (p, q) = ECC.pointAdd c p q-        run_d (n, p) = ECC.pointMul c n p+    run_o (n, p, k, q) = ECC.pointAddTwoMuls c n p k q+    run_c (p, q) = ECC.pointAdd c p q+    run_d (n, p) = ECC.pointMul c n p -        c  = ECC.getCurveByName ECC.SEC_p256r1-        p1 = ECC.pointBaseMul c n1-        p2 = ECC.pointBaseMul c n2-        n1 = 0x2ba9daf2363b2819e69b34a39cf496c2458a9b2a21505ea9e7b7cbca42dc7435-        n2 = 0xf054a7f60d10b8c2cf847ee90e9e029f8b0e971b09ca5f55c4d49921a11fadc1+    c = ECC.getCurveByName ECC.SEC_p256r1+    p1 = ECC.pointBaseMul c n1+    p2 = ECC.pointBaseMul c n2+    n1 = 0x2ba9daf2363b2819e69b34a39cf496c2458a9b2a21505ea9e7b7cbca42dc7435+    n2 = 0xf054a7f60d10b8c2cf847ee90e9e029f8b0e971b09ca5f55c4d49921a11fadc1  benchP256 =-    [ bench "pointAddTwoMuls-P256"  $ nf run_p (n1, p1, n2, p2)-    , bench "pointAdd-P256"  $ nf run_q (p1, p2)-    , bench "pointMul-P256"  $ nf run_t (n1, p1)+    [ bench "pointAddTwoMuls-P256" $ nf run_p (n1, p1, n2, p2)+    , bench "pointAdd-P256" $ nf run_q (p1, p2)+    , bench "pointMul-P256" $ nf run_t (n1, p1)     ]-  where run_p (n, p, k, q) = P256.pointAdd (P256.pointMul n p) (P256.pointMul k q)-        run_q (p, q) = P256.pointAdd p q-        run_t (n, p) = P256.pointMul n p--        xS = 0xde2444bebc8d36e682edd27e0f271508617519b3221a8fa0b77cab3989da97c9-        yS = 0xc093ae7ff36e5380fc01a5aad1e66659702de80f53cec576b6350b243042a256-        xT = 0x55a8b00f8da1d44e62f6b3b25316212e39540dc861c89575bb8cf92e35e0986b-        yT = 0x5421c3209c2d6c704835d82ac4c3dd90f61a8a52598b9e7ab656e9d8c8b24316-        p1 = P256.pointFromIntegers (xS, yS)-        p2 = P256.pointFromIntegers (xT, yT)-        n1 = throwCryptoError $ P256.scalarFromInteger 0x2ba9daf2363b2819e69b34a39cf496c2458a9b2a21505ea9e7b7cbca42dc7435-        n2 = throwCryptoError $ P256.scalarFromInteger 0xf054a7f60d10b8c2cf847ee90e9e029f8b0e971b09ca5f55c4d49921a11fadc1-+  where+    run_p (n, p, k, q) = P256.pointAdd (P256.pointMul n p) (P256.pointMul k q)+    run_q (p, q) = P256.pointAdd p q+    run_t (n, p) = P256.pointMul n p +    xS = 0xde2444bebc8d36e682edd27e0f271508617519b3221a8fa0b77cab3989da97c9+    yS = 0xc093ae7ff36e5380fc01a5aad1e66659702de80f53cec576b6350b243042a256+    xT = 0x55a8b00f8da1d44e62f6b3b25316212e39540dc861c89575bb8cf92e35e0986b+    yT = 0x5421c3209c2d6c704835d82ac4c3dd90f61a8a52598b9e7ab656e9d8c8b24316+    p1 = P256.pointFromIntegers (xS, yS)+    p2 = P256.pointFromIntegers (xT, yT)+    n1 =+        throwCryptoError $+            P256.scalarFromInteger+                0x2ba9daf2363b2819e69b34a39cf496c2458a9b2a21505ea9e7b7cbca42dc7435+    n2 =+        throwCryptoError $+            P256.scalarFromInteger+                0xf054a7f60d10b8c2cf847ee90e9e029f8b0e971b09ca5f55c4d49921a11fadc1  benchFFDH = map doFFDHBench primes   where     doFFDHBench (e, p) =         let bits = numBits p-            params = DH.Params { DH.params_p = p, DH.params_g = 2, DH.params_bits = bits }+            params = DH.Params{DH.params_p = p, DH.params_g = 2, DH.params_bits = bits}          in env (generate e params) $ bench (show bits) . nf (run params)      generate e params = do@@ -261,14 +292,31 @@     run params (priv, pub) = DH.getShared params priv pub      -- RFC 7919: prime p with minimal size of exponent-    primes = [ (225, 0xFFFFFFFFFFFFFFFFADF85458A2BB4A9AAFDC5620273D3CF1D8B9C583CE2D3695A9E13641146433FBCC939DCE249B3EF97D2FE363630C75D8F681B202AEC4617AD3DF1ED5D5FD65612433F51F5F066ED0856365553DED1AF3B557135E7F57C935984F0C70E0E68B77E2A689DAF3EFE8721DF158A136ADE73530ACCA4F483A797ABC0AB182B324FB61D108A94BB2C8E3FBB96ADAB760D7F4681D4F42A3DE394DF4AE56EDE76372BB190B07A7C8EE0A6D709E02FCE1CDF7E2ECC03404CD28342F619172FE9CE98583FF8E4F1232EEF28183C3FE3B1B4C6FAD733BB5FCBC2EC22005C58EF1837D1683B2C6F34A26C1B2EFFA886B423861285C97FFFFFFFFFFFFFFFF)-             , (275, 0xFFFFFFFFFFFFFFFFADF85458A2BB4A9AAFDC5620273D3CF1D8B9C583CE2D3695A9E13641146433FBCC939DCE249B3EF97D2FE363630C75D8F681B202AEC4617AD3DF1ED5D5FD65612433F51F5F066ED0856365553DED1AF3B557135E7F57C935984F0C70E0E68B77E2A689DAF3EFE8721DF158A136ADE73530ACCA4F483A797ABC0AB182B324FB61D108A94BB2C8E3FBB96ADAB760D7F4681D4F42A3DE394DF4AE56EDE76372BB190B07A7C8EE0A6D709E02FCE1CDF7E2ECC03404CD28342F619172FE9CE98583FF8E4F1232EEF28183C3FE3B1B4C6FAD733BB5FCBC2EC22005C58EF1837D1683B2C6F34A26C1B2EFFA886B4238611FCFDCDE355B3B6519035BBC34F4DEF99C023861B46FC9D6E6C9077AD91D2691F7F7EE598CB0FAC186D91CAEFE130985139270B4130C93BC437944F4FD4452E2D74DD364F2E21E71F54BFF5CAE82AB9C9DF69EE86D2BC522363A0DABC521979B0DEADA1DBF9A42D5C4484E0ABCD06BFA53DDEF3C1B20EE3FD59D7C25E41D2B66C62E37FFFFFFFFFFFFFFFF)-             , (325, 0xFFFFFFFFFFFFFFFFADF85458A2BB4A9AAFDC5620273D3CF1D8B9C583CE2D3695A9E13641146433FBCC939DCE249B3EF97D2FE363630C75D8F681B202AEC4617AD3DF1ED5D5FD65612433F51F5F066ED0856365553DED1AF3B557135E7F57C935984F0C70E0E68B77E2A689DAF3EFE8721DF158A136ADE73530ACCA4F483A797ABC0AB182B324FB61D108A94BB2C8E3FBB96ADAB760D7F4681D4F42A3DE394DF4AE56EDE76372BB190B07A7C8EE0A6D709E02FCE1CDF7E2ECC03404CD28342F619172FE9CE98583FF8E4F1232EEF28183C3FE3B1B4C6FAD733BB5FCBC2EC22005C58EF1837D1683B2C6F34A26C1B2EFFA886B4238611FCFDCDE355B3B6519035BBC34F4DEF99C023861B46FC9D6E6C9077AD91D2691F7F7EE598CB0FAC186D91CAEFE130985139270B4130C93BC437944F4FD4452E2D74DD364F2E21E71F54BFF5CAE82AB9C9DF69EE86D2BC522363A0DABC521979B0DEADA1DBF9A42D5C4484E0ABCD06BFA53DDEF3C1B20EE3FD59D7C25E41D2B669E1EF16E6F52C3164DF4FB7930E9E4E58857B6AC7D5F42D69F6D187763CF1D5503400487F55BA57E31CC7A7135C886EFB4318AED6A1E012D9E6832A907600A918130C46DC778F971AD0038092999A333CB8B7A1A1DB93D7140003C2A4ECEA9F98D0ACC0A8291CDCEC97DCF8EC9B55A7F88A46B4DB5A851F44182E1C68A007E5E655F6AFFFFFFFFFFFFFFFF)-             , (375, 0xFFFFFFFFFFFFFFFFADF85458A2BB4A9AAFDC5620273D3CF1D8B9C583CE2D3695A9E13641146433FBCC939DCE249B3EF97D2FE363630C75D8F681B202AEC4617AD3DF1ED5D5FD65612433F51F5F066ED0856365553DED1AF3B557135E7F57C935984F0C70E0E68B77E2A689DAF3EFE8721DF158A136ADE73530ACCA4F483A797ABC0AB182B324FB61D108A94BB2C8E3FBB96ADAB760D7F4681D4F42A3DE394DF4AE56EDE76372BB190B07A7C8EE0A6D709E02FCE1CDF7E2ECC03404CD28342F619172FE9CE98583FF8E4F1232EEF28183C3FE3B1B4C6FAD733BB5FCBC2EC22005C58EF1837D1683B2C6F34A26C1B2EFFA886B4238611FCFDCDE355B3B6519035BBC34F4DEF99C023861B46FC9D6E6C9077AD91D2691F7F7EE598CB0FAC186D91CAEFE130985139270B4130C93BC437944F4FD4452E2D74DD364F2E21E71F54BFF5CAE82AB9C9DF69EE86D2BC522363A0DABC521979B0DEADA1DBF9A42D5C4484E0ABCD06BFA53DDEF3C1B20EE3FD59D7C25E41D2B669E1EF16E6F52C3164DF4FB7930E9E4E58857B6AC7D5F42D69F6D187763CF1D5503400487F55BA57E31CC7A7135C886EFB4318AED6A1E012D9E6832A907600A918130C46DC778F971AD0038092999A333CB8B7A1A1DB93D7140003C2A4ECEA9F98D0ACC0A8291CDCEC97DCF8EC9B55A7F88A46B4DB5A851F44182E1C68A007E5E0DD9020BFD64B645036C7A4E677D2C38532A3A23BA4442CAF53EA63BB454329B7624C8917BDD64B1C0FD4CB38E8C334C701C3ACDAD0657FCCFEC719B1F5C3E4E46041F388147FB4CFDB477A52471F7A9A96910B855322EDB6340D8A00EF092350511E30ABEC1FFF9E3A26E7FB29F8C183023C3587E38DA0077D9B4763E4E4B94B2BBC194C6651E77CAF992EEAAC0232A281BF6B3A739C1226116820AE8DB5847A67CBEF9C9091B462D538CD72B03746AE77F5E62292C311562A846505DC82DB854338AE49F5235C95B91178CCF2DD5CACEF403EC9D1810C6272B045B3B71F9DC6B80D63FDD4A8E9ADB1E6962A69526D43161C1A41D570D7938DAD4A40E329CD0E40E65FFFFFFFFFFFFFFFF)-             , (400, 0xFFFFFFFFFFFFFFFFADF85458A2BB4A9AAFDC5620273D3CF1D8B9C583CE2D3695A9E13641146433FBCC939DCE249B3EF97D2FE363630C75D8F681B202AEC4617AD3DF1ED5D5FD65612433F51F5F066ED0856365553DED1AF3B557135E7F57C935984F0C70E0E68B77E2A689DAF3EFE8721DF158A136ADE73530ACCA4F483A797ABC0AB182B324FB61D108A94BB2C8E3FBB96ADAB760D7F4681D4F42A3DE394DF4AE56EDE76372BB190B07A7C8EE0A6D709E02FCE1CDF7E2ECC03404CD28342F619172FE9CE98583FF8E4F1232EEF28183C3FE3B1B4C6FAD733BB5FCBC2EC22005C58EF1837D1683B2C6F34A26C1B2EFFA886B4238611FCFDCDE355B3B6519035BBC34F4DEF99C023861B46FC9D6E6C9077AD91D2691F7F7EE598CB0FAC186D91CAEFE130985139270B4130C93BC437944F4FD4452E2D74DD364F2E21E71F54BFF5CAE82AB9C9DF69EE86D2BC522363A0DABC521979B0DEADA1DBF9A42D5C4484E0ABCD06BFA53DDEF3C1B20EE3FD59D7C25E41D2B669E1EF16E6F52C3164DF4FB7930E9E4E58857B6AC7D5F42D69F6D187763CF1D5503400487F55BA57E31CC7A7135C886EFB4318AED6A1E012D9E6832A907600A918130C46DC778F971AD0038092999A333CB8B7A1A1DB93D7140003C2A4ECEA9F98D0ACC0A8291CDCEC97DCF8EC9B55A7F88A46B4DB5A851F44182E1C68A007E5E0DD9020BFD64B645036C7A4E677D2C38532A3A23BA4442CAF53EA63BB454329B7624C8917BDD64B1C0FD4CB38E8C334C701C3ACDAD0657FCCFEC719B1F5C3E4E46041F388147FB4CFDB477A52471F7A9A96910B855322EDB6340D8A00EF092350511E30ABEC1FFF9E3A26E7FB29F8C183023C3587E38DA0077D9B4763E4E4B94B2BBC194C6651E77CAF992EEAAC0232A281BF6B3A739C1226116820AE8DB5847A67CBEF9C9091B462D538CD72B03746AE77F5E62292C311562A846505DC82DB854338AE49F5235C95B91178CCF2DD5CACEF403EC9D1810C6272B045B3B71F9DC6B80D63FDD4A8E9ADB1E6962A69526D43161C1A41D570D7938DAD4A40E329CCFF46AAA36AD004CF600C8381E425A31D951AE64FDB23FCEC9509D43687FEB69EDD1CC5E0B8CC3BDF64B10EF86B63142A3AB8829555B2F747C932665CB2C0F1CC01BD70229388839D2AF05E454504AC78B7582822846C0BA35C35F5C59160CC046FD8251541FC68C9C86B022BB7099876A460E7451A8A93109703FEE1C217E6C3826E52C51AA691E0E423CFC99E9E31650C1217B624816CDAD9A95F9D5B8019488D9C0A0A1FE3075A577E23183F81D4A3F2FA4571EFC8CE0BA8A4FE8B6855DFE72B0A66EDED2FBABFBE58A30FAFABE1C5D71A87E2F741EF8C1FE86FEA6BBFDE530677F0D97D11D49F7A8443D0822E506A9F4614E011E2A94838FF88CD68C8BB7C5C6424CFFFFFFFFFFFFFFFF)-             ]+    primes =+        [+            ( 225+            , 0xFFFFFFFFFFFFFFFFADF85458A2BB4A9AAFDC5620273D3CF1D8B9C583CE2D3695A9E13641146433FBCC939DCE249B3EF97D2FE363630C75D8F681B202AEC4617AD3DF1ED5D5FD65612433F51F5F066ED0856365553DED1AF3B557135E7F57C935984F0C70E0E68B77E2A689DAF3EFE8721DF158A136ADE73530ACCA4F483A797ABC0AB182B324FB61D108A94BB2C8E3FBB96ADAB760D7F4681D4F42A3DE394DF4AE56EDE76372BB190B07A7C8EE0A6D709E02FCE1CDF7E2ECC03404CD28342F619172FE9CE98583FF8E4F1232EEF28183C3FE3B1B4C6FAD733BB5FCBC2EC22005C58EF1837D1683B2C6F34A26C1B2EFFA886B423861285C97FFFFFFFFFFFFFFFF+            )+        ,+            ( 275+            , 0xFFFFFFFFFFFFFFFFADF85458A2BB4A9AAFDC5620273D3CF1D8B9C583CE2D3695A9E13641146433FBCC939DCE249B3EF97D2FE363630C75D8F681B202AEC4617AD3DF1ED5D5FD65612433F51F5F066ED0856365553DED1AF3B557135E7F57C935984F0C70E0E68B77E2A689DAF3EFE8721DF158A136ADE73530ACCA4F483A797ABC0AB182B324FB61D108A94BB2C8E3FBB96ADAB760D7F4681D4F42A3DE394DF4AE56EDE76372BB190B07A7C8EE0A6D709E02FCE1CDF7E2ECC03404CD28342F619172FE9CE98583FF8E4F1232EEF28183C3FE3B1B4C6FAD733BB5FCBC2EC22005C58EF1837D1683B2C6F34A26C1B2EFFA886B4238611FCFDCDE355B3B6519035BBC34F4DEF99C023861B46FC9D6E6C9077AD91D2691F7F7EE598CB0FAC186D91CAEFE130985139270B4130C93BC437944F4FD4452E2D74DD364F2E21E71F54BFF5CAE82AB9C9DF69EE86D2BC522363A0DABC521979B0DEADA1DBF9A42D5C4484E0ABCD06BFA53DDEF3C1B20EE3FD59D7C25E41D2B66C62E37FFFFFFFFFFFFFFFF+            )+        ,+            ( 325+            , 0xFFFFFFFFFFFFFFFFADF85458A2BB4A9AAFDC5620273D3CF1D8B9C583CE2D3695A9E13641146433FBCC939DCE249B3EF97D2FE363630C75D8F681B202AEC4617AD3DF1ED5D5FD65612433F51F5F066ED0856365553DED1AF3B557135E7F57C935984F0C70E0E68B77E2A689DAF3EFE8721DF158A136ADE73530ACCA4F483A797ABC0AB182B324FB61D108A94BB2C8E3FBB96ADAB760D7F4681D4F42A3DE394DF4AE56EDE76372BB190B07A7C8EE0A6D709E02FCE1CDF7E2ECC03404CD28342F619172FE9CE98583FF8E4F1232EEF28183C3FE3B1B4C6FAD733BB5FCBC2EC22005C58EF1837D1683B2C6F34A26C1B2EFFA886B4238611FCFDCDE355B3B6519035BBC34F4DEF99C023861B46FC9D6E6C9077AD91D2691F7F7EE598CB0FAC186D91CAEFE130985139270B4130C93BC437944F4FD4452E2D74DD364F2E21E71F54BFF5CAE82AB9C9DF69EE86D2BC522363A0DABC521979B0DEADA1DBF9A42D5C4484E0ABCD06BFA53DDEF3C1B20EE3FD59D7C25E41D2B669E1EF16E6F52C3164DF4FB7930E9E4E58857B6AC7D5F42D69F6D187763CF1D5503400487F55BA57E31CC7A7135C886EFB4318AED6A1E012D9E6832A907600A918130C46DC778F971AD0038092999A333CB8B7A1A1DB93D7140003C2A4ECEA9F98D0ACC0A8291CDCEC97DCF8EC9B55A7F88A46B4DB5A851F44182E1C68A007E5E655F6AFFFFFFFFFFFFFFFF+            )+        ,+            ( 375+            , 0xFFFFFFFFFFFFFFFFADF85458A2BB4A9AAFDC5620273D3CF1D8B9C583CE2D3695A9E13641146433FBCC939DCE249B3EF97D2FE363630C75D8F681B202AEC4617AD3DF1ED5D5FD65612433F51F5F066ED0856365553DED1AF3B557135E7F57C935984F0C70E0E68B77E2A689DAF3EFE8721DF158A136ADE73530ACCA4F483A797ABC0AB182B324FB61D108A94BB2C8E3FBB96ADAB760D7F4681D4F42A3DE394DF4AE56EDE76372BB190B07A7C8EE0A6D709E02FCE1CDF7E2ECC03404CD28342F619172FE9CE98583FF8E4F1232EEF28183C3FE3B1B4C6FAD733BB5FCBC2EC22005C58EF1837D1683B2C6F34A26C1B2EFFA886B4238611FCFDCDE355B3B6519035BBC34F4DEF99C023861B46FC9D6E6C9077AD91D2691F7F7EE598CB0FAC186D91CAEFE130985139270B4130C93BC437944F4FD4452E2D74DD364F2E21E71F54BFF5CAE82AB9C9DF69EE86D2BC522363A0DABC521979B0DEADA1DBF9A42D5C4484E0ABCD06BFA53DDEF3C1B20EE3FD59D7C25E41D2B669E1EF16E6F52C3164DF4FB7930E9E4E58857B6AC7D5F42D69F6D187763CF1D5503400487F55BA57E31CC7A7135C886EFB4318AED6A1E012D9E6832A907600A918130C46DC778F971AD0038092999A333CB8B7A1A1DB93D7140003C2A4ECEA9F98D0ACC0A8291CDCEC97DCF8EC9B55A7F88A46B4DB5A851F44182E1C68A007E5E0DD9020BFD64B645036C7A4E677D2C38532A3A23BA4442CAF53EA63BB454329B7624C8917BDD64B1C0FD4CB38E8C334C701C3ACDAD0657FCCFEC719B1F5C3E4E46041F388147FB4CFDB477A52471F7A9A96910B855322EDB6340D8A00EF092350511E30ABEC1FFF9E3A26E7FB29F8C183023C3587E38DA0077D9B4763E4E4B94B2BBC194C6651E77CAF992EEAAC0232A281BF6B3A739C1226116820AE8DB5847A67CBEF9C9091B462D538CD72B03746AE77F5E62292C311562A846505DC82DB854338AE49F5235C95B91178CCF2DD5CACEF403EC9D1810C6272B045B3B71F9DC6B80D63FDD4A8E9ADB1E6962A69526D43161C1A41D570D7938DAD4A40E329CD0E40E65FFFFFFFFFFFFFFFF+            )+        ,+            ( 400+            , 0xFFFFFFFFFFFFFFFFADF85458A2BB4A9AAFDC5620273D3CF1D8B9C583CE2D3695A9E13641146433FBCC939DCE249B3EF97D2FE363630C75D8F681B202AEC4617AD3DF1ED5D5FD65612433F51F5F066ED0856365553DED1AF3B557135E7F57C935984F0C70E0E68B77E2A689DAF3EFE8721DF158A136ADE73530ACCA4F483A797ABC0AB182B324FB61D108A94BB2C8E3FBB96ADAB760D7F4681D4F42A3DE394DF4AE56EDE76372BB190B07A7C8EE0A6D709E02FCE1CDF7E2ECC03404CD28342F619172FE9CE98583FF8E4F1232EEF28183C3FE3B1B4C6FAD733BB5FCBC2EC22005C58EF1837D1683B2C6F34A26C1B2EFFA886B4238611FCFDCDE355B3B6519035BBC34F4DEF99C023861B46FC9D6E6C9077AD91D2691F7F7EE598CB0FAC186D91CAEFE130985139270B4130C93BC437944F4FD4452E2D74DD364F2E21E71F54BFF5CAE82AB9C9DF69EE86D2BC522363A0DABC521979B0DEADA1DBF9A42D5C4484E0ABCD06BFA53DDEF3C1B20EE3FD59D7C25E41D2B669E1EF16E6F52C3164DF4FB7930E9E4E58857B6AC7D5F42D69F6D187763CF1D5503400487F55BA57E31CC7A7135C886EFB4318AED6A1E012D9E6832A907600A918130C46DC778F971AD0038092999A333CB8B7A1A1DB93D7140003C2A4ECEA9F98D0ACC0A8291CDCEC97DCF8EC9B55A7F88A46B4DB5A851F44182E1C68A007E5E0DD9020BFD64B645036C7A4E677D2C38532A3A23BA4442CAF53EA63BB454329B7624C8917BDD64B1C0FD4CB38E8C334C701C3ACDAD0657FCCFEC719B1F5C3E4E46041F388147FB4CFDB477A52471F7A9A96910B855322EDB6340D8A00EF092350511E30ABEC1FFF9E3A26E7FB29F8C183023C3587E38DA0077D9B4763E4E4B94B2BBC194C6651E77CAF992EEAAC0232A281BF6B3A739C1226116820AE8DB5847A67CBEF9C9091B462D538CD72B03746AE77F5E62292C311562A846505DC82DB854338AE49F5235C95B91178CCF2DD5CACEF403EC9D1810C6272B045B3B71F9DC6B80D63FDD4A8E9ADB1E6962A69526D43161C1A41D570D7938DAD4A40E329CCFF46AAA36AD004CF600C8381E425A31D951AE64FDB23FCEC9509D43687FEB69EDD1CC5E0B8CC3BDF64B10EF86B63142A3AB8829555B2F747C932665CB2C0F1CC01BD70229388839D2AF05E454504AC78B7582822846C0BA35C35F5C59160CC046FD8251541FC68C9C86B022BB7099876A460E7451A8A93109703FEE1C217E6C3826E52C51AA691E0E423CFC99E9E31650C1217B624816CDAD9A95F9D5B8019488D9C0A0A1FE3075A577E23183F81D4A3F2FA4571EFC8CE0BA8A4FE8B6855DFE72B0A66EDED2FBABFBE58A30FAFABE1C5D71A87E2F741EF8C1FE86FEA6BBFDE530677F0D97D11D49F7A8443D0822E506A9F4614E011E2A94838FF88CD68C8BB7C5C6424CFFFFFFFFFFFFFFFF+            )+        ] -data CurveDH = forall c . (EllipticCurveDH c, NFData (Scalar c), NFData (Point c)) => CurveDH c+data CurveDH+    = forall c. (EllipticCurveDH c, NFData (Scalar c), NFData (Point c)) => CurveDH c  benchECDH = map doECDHBench curves   where@@ -277,32 +325,38 @@          in env (generate proxy) $ bench name . nf (run proxy)      generate proxy = do-        KeyPair _      aScalar <- curveGenerateKeyPair proxy-        KeyPair bPoint _       <- curveGenerateKeyPair proxy+        KeyPair _ aScalar <- curveGenerateKeyPair proxy+        KeyPair bPoint _ <- curveGenerateKeyPair proxy         return (aScalar, bPoint)      run proxy (s, p) = throwCryptoError (ecdh proxy s p) -    curves = [ ("P256R1", CurveDH Curve_P256R1)-             , ("P384R1", CurveDH Curve_P384R1)-             , ("P521R1", CurveDH Curve_P521R1)-             , ("X25519", CurveDH Curve_X25519)-             , ("X448",   CurveDH Curve_X448)-             ]+    curves =+        [ ("P256R1", CurveDH Curve_P256R1)+        , ("P384R1", CurveDH Curve_P384R1)+        , ("P521R1", CurveDH Curve_P521R1)+        , ("X25519", CurveDH Curve_X25519)+        , ("X448", CurveDH Curve_X448)+        ] -data CurveHashECDSA =-    forall curve hashAlg . (ECDSA.EllipticCurveECDSA curve,-                            NFData (Scalar curve),-                            NFData (Point curve),-                            HashAlgorithm hashAlg) => CurveHashECDSA curve hashAlg+data CurveHashECDSA+    = forall curve hashAlg.+        ( ECDSA.EllipticCurveECDSA curve+        , NFData (Scalar curve)+        , NFData (Point curve)+        , HashAlgorithm hashAlg+        ) =>+      CurveHashECDSA curve hashAlg  benchECDSA = map doECDSABench curveHashes   where     doECDSABench (name, CurveHashECDSA c hashAlg) =         let proxy = Just c -- using Maybe as Proxy-         in bgroup name+         in bgroup+                name                 [ env (signGenerate proxy) $ bench "sign" . nfIO . signRun proxy hashAlg-                , env (verifyGenerate proxy hashAlg) $ bench "verify" . nf (verifyRun proxy hashAlg)+                , env (verifyGenerate proxy hashAlg) $+                    bench "verify" . nf (verifyRun proxy hashAlg)                 ]      signGenerate proxy = do@@ -323,30 +377,33 @@     tenKB :: IO Bytes     tenKB = getRandomBytes 10240 -    curveHashes = [ ("secp256r1_sha256", CurveHashECDSA Curve_P256R1 SHA256)-                  , ("secp384r1_sha384", CurveHashECDSA Curve_P384R1 SHA384)-                  , ("secp521r1_sha512", CurveHashECDSA Curve_P521R1 SHA512)-                  ]+    curveHashes =+        [ ("secp256r1_sha256", CurveHashECDSA Curve_P256R1 SHA256)+        , ("secp384r1_sha384", CurveHashECDSA Curve_P384R1 SHA384)+        , ("secp521r1_sha512", CurveHashECDSA Curve_P521R1 SHA512)+        ]  benchEdDSA =     [ bgroup "EdDSA-Ed25519" benchGenEd25519-    , bgroup "Ed25519"       benchEd25519+    , bgroup "Ed25519" benchEd25519     ]   where+    -- the environment is a key pair and a signature that the benchmarked+    -- operation only reads, so building it once outside the timed region is+    -- the same measurement gauge's perBatchEnv made     benchGen prx alg =-        [ bench "sign"   $ perBatchEnv (genEnv prx alg) (run_gen_sign   prx)-        , bench "verify" $ perBatchEnv (genEnv prx alg) (run_gen_verify prx)+        [ env (genEnv prx alg) $ bench "sign" . nfIO . run_gen_sign prx+        , env (genEnv prx alg) $ bench "verify" . nfIO . run_gen_verify prx         ]      benchGenEd25519 = benchGen (Just Curve_Edwards25519) SHA512-    benchEd25519    =-        [ bench "sign"   $ perBatchEnv ed25519Env run_ed25519_sign-        , bench "verify" $ perBatchEnv ed25519Env run_ed25519_verify+    benchEd25519 =+        [ env ed25519Env $ bench "sign" . nfIO . run_ed25519_sign+        , env ed25519Env $ bench "verify" . nfIO . run_ed25519_verify         ]      msg = B.empty -- empty message = worst-case scenario showing API overhead--    genEnv prx alg _ = do+    genEnv prx alg = do         sec <- EdDSA.generateSecretKey prx         let pub = EdDSA.toPublic prx alg sec             sig = EdDSA.sign prx sec pub msg@@ -356,7 +413,7 @@      run_gen_verify prx (_, pub, sig) = return (EdDSA.verify prx pub msg sig) -    ed25519Env _ = do+    ed25519Env = do         sec <- Ed25519.generateSecretKey         let pub = Ed25519.toPublic sec             sig = Ed25519.sign sec pub msg@@ -366,19 +423,21 @@      run_ed25519_verify (_, pub, sig) = return (Ed25519.verify pub msg sig) -main = defaultMain-    [ bgroup "hash" benchHash-    , bgroup "block-cipher" benchBlockCipher-    , bgroup "AE" benchAE-    , bgroup "pbkdf2" benchPBKDF2-    , bgroup "bcrypt" benchBCrypt-    , bgroup "ECC" benchECC-    , bgroup "P256" benchP256-    , bgroup "DH"-          [ bgroup "FFDH" benchFFDH-          , bgroup "ECDH" benchECDH-          ]-    , bgroup "ECDSA" benchECDSA-    , bgroup "EdDSA" benchEdDSA-    , bgroup "F2m" benchF2m-    ]+main =+    defaultMain+        [ bgroup "hash" benchHash+        , bgroup "block-cipher" benchBlockCipher+        , bgroup "AE" benchAE+        , bgroup "pbkdf2" benchPBKDF2+        , bgroup "bcrypt" benchBCrypt+        , bgroup "ECC" benchECC+        , bgroup "P256" benchP256+        , bgroup+            "DH"+            [ bgroup "FFDH" benchFFDH+            , bgroup "ECDH" benchECDH+            ]+        , bgroup "ECDSA" benchECDSA+        , bgroup "EdDSA" benchEdDSA+        , bgroup "F2m" benchF2m+        ]
benchs/Number/F2m.hs view
@@ -2,20 +2,23 @@  module Number.F2m (benchF2m) where -import Gauge.Main import System.Random+import Test.Tasty.Bench  import Crypto.Number.Basic (log2) import Crypto.Number.F2m  genInteger :: Int -> Int -> Integer-genInteger salt bits-    = head-    . dropWhile ((< bits) . log2)-    . scanl (\a r -> a * 2^(31 :: Int) + abs r) 0-    . randoms-    . mkStdGen-    $ salt + bits+genInteger salt bits = case candidates of+    x : _ -> x+    [] -> error "genInteger: the stream of candidates ran out"+  where+    candidates =+        dropWhile ((< bits) . log2)+            . scanl (\a r -> a * 2 ^ (31 :: Int) + abs r) 0+            . randoms+            . mkStdGen+            $ salt + bits  benchMod :: Int -> Benchmark benchMod bits = bench (show bits) $ nf (modF2m m) a@@ -46,8 +49,8 @@ bitsList = [64, 128, 256, 512, 1024, 2048]  benchF2m =-    [ bgroup    "modF2m" $ map benchMod    bitsList-    , bgroup    "mulF2m" $ map benchMul    bitsList+    [ bgroup "modF2m" $ map benchMod bitsList+    , bgroup "mulF2m" $ map benchMul bitsList     , bgroup "squareF2m" $ map benchSquare bitsList-    , bgroup    "invF2m" $ map benchInv    bitsList+    , bgroup "invF2m" $ map benchInv bitsList     ]
+ cbits/LICENSE.go view
@@ -0,0 +1,38 @@+The arrangement of the AArch64 multiply-accumulate loop in+cbits/crypton_bignum.h -- four limbs to an iteration, the low halves of the+products and the high halves accumulated in two chains -- follows+addMulVVWx in Go's crypto/internal/fips140/bigmod/nat_arm64.s, written out+in the assembler that file's compiler speaks.  Go is at+https://github.com/golang/go and carries the licence below.  That file's+own header reads "Copyright 2013 The Go Authors. All rights reserved.  Use+of this source code is governed by a BSD-style license that can be found in+the LICENSE file", and the LICENSE file it means is this one.+++Copyright 2009 The Go Authors.++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions are+met:++   * Redistributions of source code must retain the above copyright+notice, this list of conditions and the following disclaimer.+   * Redistributions in binary form must reproduce the above+copyright notice, this list of conditions and the following disclaimer+in the documentation and/or other materials provided with the+distribution.+   * Neither the name of Google LLC nor the names of its+contributors may be used to endorse or promote products derived from+this software without specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ cbits/aes/LICENSE.fusion view
@@ -0,0 +1,29 @@+Parts of cbits/aes/gcm_fused_x86.c follow the AES-GCM implementation in+picotls, lib/fusion.c, which is under the MIT license reproduced below.+The design is described by its author at++    http://blog.kazuhooku.com/2020/06/quicaes-gcm-12.html+    http://blog.kazuhooku.com/2020/06/quicaes-gcm-22.html++and the source is at https://github.com/h2o/picotls.+++Copyright (c) 2020-2022 Fastly, Kazuho Oku++Permission is hereby granted, free of charge, to any person obtaining a copy+of this software and associated documentation files (the "Software"), to+deal in the Software without restriction, including without limitation the+rights to use, copy, modify, merge, publish, distribute, sublicense, and/or+sell copies of the Software, and to permit persons to whom the Software is+furnished to do so, subject to the following conditions:++The above copyright notice and this permission notice shall be included in+all copies or substantial portions of the Software.++THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS+IN THE SOFTWARE.
+ cbits/aes/armv8.c view
@@ -0,0 +1,469 @@+/*+ * AES using the ARMv8-A Cryptographic Extensions.+ *+ * The generic code in aes/generic.c is S-box table driven, which on AArch64+ * was the only thing available: crypton_aes.c only ever swapped in the AES-NI+ * implementation, and that is gated on x86.  This provides the AArch64+ * equivalent.+ *+ * The key schedule is laid out exactly as x86ni.c lays it out, because+ * crypton_aes.c leaves some operations -- OCB and CCM -- pointing at the+ * generic implementation even once the accelerated table is installed, and+ * those read the forward schedule.  So: the forward round keys k[0..nbr]+ * first, in the order crypton_aes_generic_init writes them, then+ * InvMixColumns(k[nbr-1]) down to InvMixColumns(k[1]) for decryption.  The two+ * ends of the decryption schedule, k[nbr] and k[0], are read back out of the+ * forward half rather than stored twice, which is what makes AES-256 fit in+ * the 16*14*2 bytes of aes_key.data.+ */++#include <stdint.h>+#include <string.h>+#include <arm_neon.h>+#if defined(__linux__)+#include <sys/auxv.h>+#include <asm/hwcap.h>+#endif+#include "crypton_aes.h"+#include "crypton_bitfn.h"++/*+ * The AES and PMULL instructions are extensions, so a translation unit+ * compiled for baseline ARMv8-A may not use them.  Mark the functions that do,+ * the way cbits/aes/x86ni.h marks their x86 counterparts, rather than raising+ * -march for every file in the library: the flag use_target_attributes picks+ * between the two, and with it set -- which is the default -- nothing else+ * enables the extensions, so without these the file does not compile at all on+ * a toolchain whose baseline lacks them.  Apple's does not lack them, which is+ * why only Linux noticed.+ *+ * "+crypto" rather than "crypto": GCC rejects the latter.+ */+#include "crypton_armv8_target.h"++/* forward round keys: nbr + 1 of them, written by the generic key expansion */+#define FWD(key)  ((const uint8_t *) (key)->data)+/* InvMixColumns(k[nbr-1]) .. InvMixColumns(k[1]): nbr - 1 of them */+#define INV(key)  (((const uint8_t *) (key)->data) + 16 * ((key)->nbr + 1))++/*+ * The key schedule of FIPS 197 5.2, with the S-box the schedule needs coming+ * from the instructions rather than a table in memory.+ *+ * AArch64 has no counterpart to x86's AESKEYGENASSIST, but AESE is+ * AddRoundKey, SubBytes and ShiftRows together, so against a zero key it is+ * SubBytes and ShiftRows.  Give it a word in all four columns and ShiftRows+ * only moves identical bytes between them, which leaves every column holding+ * SubWord of that word.  RotWord is then a byte rotation, and on a register+ * whose four words are equal a rotation of the whole register by one byte+ * rotates each word.+ *+ * The words stay in vector registers throughout: a word moved to a general+ * register and back costs more than the instruction it is moved for.+ *+ * The exposure this removes is a small one -- sixteen lookups at addresses+ * derived from the key, once per key, against the per-block indexing the+ * instructions exist to remove -- but a key schedule is the one thing an+ * attacker most wants and it costs little to keep it out of the cache.+ */+CRYPTON_TARGET_ARMV8_CRYPTO+static uint32x4_t sub_word(uint32x4_t w)+{+	return vreinterpretq_u32_u8(+	    vaeseq_u8(vreinterpretq_u8_u32(w), vdupq_n_u8(0)));+}++CRYPTON_TARGET_ARMV8_CRYPTO+static uint32x4_t sub_rot_word(uint32x4_t w)+{+	const uint8x16_t s = vreinterpretq_u8_u32(sub_word(w));++	return vreinterpretq_u32_u8(vextq_u8(s, s, 1));+}++CRYPTON_TARGET_ARMV8_CRYPTO+void crypton_aes_armv8_init(aes_key *key, uint8_t *origkey, uint8_t size)+{+	/* 2^0 .. 2^9 in GF(2^8), which is as far as any key size reaches */+	static const uint32_t rcon[10] = {+		0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, 0x1b, 0x36,+	};+	uint32_t *w = (uint32_t *) key->data;+	uint8_t *inv;+	int nk, nw, i;++	switch (size) {+	case 16: key->nbr = 10; break;+	case 24: key->nbr = 12; break;+	case 32: key->nbr = 14; break;+	default: return;+	}+	nk = size / 4;                  /* words of key */+	nw = 4 * (key->nbr + 1);        /* words of schedule */++	memcpy(w, origkey, size);+	for (i = nk; i < nw; i++) {+		uint32x4_t t = vld1q_dup_u32(w + i - 1);++		if (i % nk == 0)+			t = veorq_u32(sub_rot_word(t),+			              vdupq_n_u32(rcon[i / nk - 1]));+		else if (nk > 6 && i % nk == 4)+			t = sub_word(t);+		vst1q_lane_u32(w + i, veorq_u32(t, vld1q_dup_u32(w + i - nk)), 0);+	}++	/* and the inverted round keys the decryption modes read */+	inv = ((uint8_t *) key->data) + 16 * (key->nbr + 1);+	for (i = 1; i < key->nbr; i++) {+		uint8x16_t rk =+		    vld1q_u8(((const uint8_t *) key->data) + 16 * (key->nbr - i));+		vst1q_u8(inv + 16 * (i - 1), vaesimcq_u8(rk));+	}+}++/*+ * Whether the extensions are actually present.+ *+ * They are mandatory on Apple silicon, and on other AArch64 systems the+ * kernel reports them through the auxiliary vector.  A system without them+ * keeps the generic implementation.+ */+int crypton_aes_armv8_available(void)+{+#if defined(__APPLE__)+	return 1;+#elif defined(__linux__)+	return (getauxval(AT_HWCAP) & HWCAP_AES) != 0;+#else+	return 0;+#endif+}++/*+ * GHASH using PMULL, the AArch64 counterpart to PCLMULQDQ.+ *+ * Not the transliteration of gfmul_pclmuldq in x86ni.c this used to be.  The+ * x86 formulation keeps H in the order GCM writes it and pays, at the end of+ * every batch, a reduction that first has to undo GCM's bit reflection: some+ * twenty-five shifts and XORs, and a batch of eight costs it once.+ *+ * Instead H is twisted once, at key setup, so that the reflection is already+ * undone and a reversed-polynomial multiply lands in the right place.  Two+ * things follow.  The reduction becomes two PMULL against 0xC2000..0 and six+ * EOR, a third of what it was.  And because nothing has to be byte-reversed+ * back and forth, Karatsuba pays: three PMULL a block rather than four, with+ * the middle terms accumulated in a third register and tidied up once per+ * batch.+ *+ * crypton tried Karatsuba in the old representation and measured it 1.6 per+ * cent slower -- the saved PMULL did not cover the extra EOR when the+ * reduction stayed as expensive as it was.  It is the pair that pays.+ * Measured over 16 KiB messages, this against the old code:+ *+ *                      Apple M4        Neoverse N2+ *     AES-128-GCM        1.30             1.25+ *     AES-192-GCM        1.22             1.25+ *     AES-256-GCM        1.19             1.24+ *+ * The scheme is ARM's, from the 'big' AES-GCM kernel of+ * https://github.com/ARM-software/AArch64cryptolib, which is BSD-3-Clause,+ * (c) 2018-2019 ARM Limited.  Their kernels under AArch64cryptolib_opt_bigger+ * are faster again and are NOT under that licence, whatever the repository's+ * LICENSE.md says; nothing here comes from those files.+ *+ * The table holds the twisted powers H^1 .. H^8 at htable[0 .. 7] and the+ * Karatsuba half of each -- its high 64 bits XOR its low -- in the first+ * eight bytes of htable[8 .. 15].  The running tag is kept the way GCM+ * writes it at every boundary, and swapped into the internal form on the way+ * in and out, which is two instructions.+ */++#define GHASH_MODC ((poly64_t) 0xC200000000000000ul)++/* the internal accumulator form, and back again: its own inverse */+CRYPTON_TARGET_ARMV8_CRYPTO+static inline uint8x16_t ghash_swap(uint8x16_t t)+{+	t = vrev64q_u8(t);+	return vextq_u8(t, t, 8);+}++/* the high and low halves XORed together, which is what Karatsuba wants */+CRYPTON_TARGET_ARMV8_CRYPTO+static inline poly64_t ghash_karat(poly64x2_t v)+{+	return (poly64_t) veor_u64(vget_high_u64(vreinterpretq_u64_p64(v)),+	                           vget_low_u64(vreinterpretq_u64_p64(v)));+}++/* the twisted power H^(i+1) */+#define GHASH_POW(ht, i)                                                      \+	vreinterpretq_p64_u8(vld1q_u8((const uint8_t *) &(ht)[i]))+/* and its Karatsuba half */+#define GHASH_KARAT(ht, i)                                                    \+	((poly64_t) vgetq_lane_u64(                                           \+	    vreinterpretq_u64_u8(vld1q_u8((const uint8_t *) &(ht)[8 + (i)])), 0))++/*+ * One block's three partial products, XORed into the accumulators.  b is+ * already in the internal form; hp and hk are the power it is to meet.+ */+#define GHASH_MUL(b, hp, hk, H, M, L)                                         \+	do {                                                                  \+		poly64x2_t b__ = (b);                                         \+		poly64x2_t hp__ = (hp);                                       \+		(H) = veorq_u64((H), vreinterpretq_u64_p128(                  \+		    vmull_high_p64(b__, hp__)));                              \+		(L) = veorq_u64((L), vreinterpretq_u64_p128(vmull_p64(        \+		    (poly64_t) vgetq_lane_u64(vreinterpretq_u64_p64(b__), 0), \+		    (poly64_t) vgetq_lane_u64(vreinterpretq_u64_p64(hp__), 0)))); \+		(M) = veorq_u64((M), vreinterpretq_u64_p128(                  \+		    vmull_p64(ghash_karat(b__), (hk))));                      \+	} while (0)++/*+ * Finish the Karatsuba -- the middle accumulator still holds only the+ * (ah^al)(bh^bl) terms and wants the other two taken out of it -- and reduce+ * the 256 bits modulo the GCM polynomial.  The result is in internal form.+ */+CRYPTON_TARGET_ARMV8_CRYPTO+static inline uint64x2_t ghash_reduce(uint64x2_t H, uint64x2_t M, uint64x2_t L)+{+	uint64x2_t t;++	M = veorq_u64(M, H);+	M = veorq_u64(M, L);++	t = vreinterpretq_u64_p128(vmull_p64(+	    (poly64_t) vgetq_lane_u64(H, 0), GHASH_MODC));+	H = vreinterpretq_u64_u8(vextq_u8(vreinterpretq_u8_u64(H),+	                                  vreinterpretq_u8_u64(H), 8));+	M = veorq_u64(M, t);+	M = veorq_u64(M, H);++	t = vreinterpretq_u64_p128(vmull_p64(+	    (poly64_t) vgetq_lane_u64(M, 0), GHASH_MODC));+	M = vreinterpretq_u64_u8(vextq_u8(vreinterpretq_u8_u64(M),+	                                  vreinterpretq_u8_u64(M), 8));+	L = veorq_u64(L, t);+	return veorq_u64(L, M);+}++/* a single block against H^1, accumulator in internal form */+CRYPTON_TARGET_ARMV8_CRYPTO+static inline uint64x2_t ghash_one(uint64x2_t acc, uint8x16_t blk,+                                   const block128 *ht)+{+	uint64x2_t H = vdupq_n_u64(0), M = H, L = H;+	poly64x2_t b;++	acc = vreinterpretq_u64_u8(vextq_u8(vreinterpretq_u8_u64(acc),+	                                    vreinterpretq_u8_u64(acc), 8));+	b = vreinterpretq_p64_u64(veorq_u64(+	    vreinterpretq_u64_u8(vrev64q_u8(blk)), acc));+	GHASH_MUL(b, GHASH_POW(ht, 0), GHASH_KARAT(ht, 0), H, M, L);+	return ghash_reduce(H, M, L);+}++/*+ * Twist H and raise it to the powers a batch needs.+ *+ * The twist is a shift left by one with 0xC2000..01 folded back in when a+ * bit falls off the top -- the same correction the old reduction applied to+ * every product, done once here instead.  Each further power is one multiply+ * in the twisted domain; the result comes out of ghash_reduce with its+ * halves swapped, which a batch undoes on the way in, so a stored power has+ * to be swapped back.+ */+CRYPTON_TARGET_ARMV8_CRYPTO+void crypton_aes_armv8_hinit_pmull(block128 *htable, const block128 *h)+{+	uint8x16_t hk = vrev64q_u8(vld1q_u8((const uint8_t *) h));+	uint64x2_t shl = vshlq_n_u64(vreinterpretq_u64_u8(hk), 1);+	uint64x2_t shr = vreinterpretq_u64_s64(+	    vshrq_n_s64(vreinterpretq_s64_u8(hk), 63));+	uint8x16_t mask = vextq_u8(vreinterpretq_u8_u64(shr),+	                           vreinterpretq_u8_u64(shr), 12);+	uint64x2_t tc = vdupq_n_u64(0);+	poly64x2_t base, p;+	int i;++	tc = vsetq_lane_u64(0xC200000000000001ul, tc, 0);+	tc = vsetq_lane_u64(1, tc, 1);+	tc = vandq_u64(vreinterpretq_u64_u8(mask), tc);+	base = vreinterpretq_p64_u64(veorq_u64(tc, shl));++	p = base;+	for (i = 0; i < 8; i++) {+		uint64x2_t H = vdupq_n_u64(0), M = H, L = H, r;++		vst1q_u8((uint8_t *) &htable[i],+		         vreinterpretq_u8_p64(p));+		vst1q_u8((uint8_t *) &htable[8 + i],+		         vreinterpretq_u8_u64(+		             vdupq_n_u64((uint64_t) ghash_karat(p))));++		GHASH_MUL(p, base, ghash_karat(base), H, M, L);+		r = ghash_reduce(H, M, L);+		p = vreinterpretq_p64_u8(vextq_u8(vreinterpretq_u8_u64(r),+		                                  vreinterpretq_u8_u64(r), 8));+	}+}++CRYPTON_TARGET_ARMV8_CRYPTO+void crypton_aes_armv8_gf_mul_pmull(block128 *a, const block128 *htable)+{+	uint64x2_t acc = vreinterpretq_u64_u8(+	    ghash_swap(vld1q_u8((const uint8_t *) a)));++	acc = ghash_one(acc, vdupq_n_u8(0), htable);+	vst1q_u8((uint8_t *) a, ghash_swap(vreinterpretq_u8_u64(acc)));+}++/*+ * Four GHASH steps -- ((((a^b0)H ^ b1)H ^ b2)H ^ b3)H -- with a single+ * reduction.  Expanded that is (a^b0)H^4 ^ b1*H^3 ^ b2*H^2 ^ b3*H, so the+ * four products can be summed first and reduced once, which is where the+ * time goes.  Aggregated reduction, from the Intel GCM paper.+ */+CRYPTON_TARGET_ARMV8_CRYPTO+void crypton_aes_armv8_gf_mul4_pmull(block128 *a, const block128 *blocks,+                                     const block128 *htable)+{+	uint64x2_t acc = vreinterpretq_u64_u8(+	    ghash_swap(vld1q_u8((const uint8_t *) a)));+	uint64x2_t H = vdupq_n_u64(0), M = H, L = H;+	poly64x2_t b;+	int i;++	acc = vreinterpretq_u64_u8(vextq_u8(vreinterpretq_u8_u64(acc),+	                                    vreinterpretq_u8_u64(acc), 8));+	b = vreinterpretq_p64_u64(veorq_u64(+	    vreinterpretq_u64_u8(vrev64q_u8(+	        vld1q_u8((const uint8_t *) &blocks[0]))), acc));+	GHASH_MUL(b, GHASH_POW(htable, 3), GHASH_KARAT(htable, 3), H, M, L);++	for (i = 1; i < 4; i++) {+		b = vreinterpretq_p64_u8(vrev64q_u8(+		    vld1q_u8((const uint8_t *) &blocks[i])));+		GHASH_MUL(b, GHASH_POW(htable, 3 - i),+		          GHASH_KARAT(htable, 3 - i), H, M, L);+	}++	acc = ghash_reduce(H, M, L);+	vst1q_u8((uint8_t *) a, ghash_swap(vreinterpretq_u8_u64(acc)));+}+++int crypton_aes_armv8_pmull_available(void)+{+#if defined(__APPLE__)+	return 1;+#elif defined(__linux__)+	return (getauxval(AT_HWCAP) & HWCAP_PMULL) != 0;+#else+	return 0;+#endif+}++/*+ * The XTS tweak advances by doubling in GF(2^128), which+ * crypton_aes_generic_gf_mulx does through memory.  Here it stays in a+ * register: shift both halves left by one, carry the low half's top bit into+ * the high half, and fold the bit that leaves the top back in as 0x87.  The+ * block is little-endian, so lane 0 is the low half.+ */+CRYPTON_TARGET_ARMV8_CRYPTO+static inline uint8x16_t gfmulx_neon(uint8x16_t v)+{+	const uint64x2_t x = vreinterpretq_u64_u8(v);+	const uint64x2_t zero = vdupq_n_u64(0);+	const uint64x2_t carry = vshrq_n_u64(x, 63);+	/* the low half's carry becomes the high half's bit 0 */+	const uint64x2_t into_hi = vextq_u64(zero, carry, 1);+	/* and the high half's becomes all ones, or nothing, in the low half */+	const uint64x2_t out = vsubq_u64(zero, vextq_u64(carry, zero, 1));+	const uint64x2_t poly = vsetq_lane_u64(0x87, zero, 0);++	return vreinterpretq_u8_u64(veorq_u64(+	    vorrq_u64(vshlq_n_u64(x, 1), into_hi), vandq_u64(out, poly)));+}++/*+ * The modes, generated once per key size.  See armv8_impl.c for why the+ * round count has to be a compile-time constant.+ */+#define SIZED(m) m##128+#define NBR 10+#include <aes/armv8_impl.c>+#undef SIZED+#undef NBR++#define SIZED(m) m##192+#define NBR 12+#include <aes/armv8_impl.c>+#undef SIZED+#undef NBR++#define SIZED(m) m##256+#define NBR 14+#include <aes/armv8_impl.c>+#undef SIZED+#undef NBR++/*+ * The fused entry point, over the three key sizes.  Each was generated with+ * its round count fixed, which is what lets the eight chains stay in+ * registers; the choice between them is made once per message here.+ */+CRYPTON_TARGET_ARMV8_CRYPTO+void crypton_aes_armv8_gcm_fused(uint8_t *out, const block128 *ht,+                                 aes_key *key, const uint8_t *nonce,+                                 const uint8_t *aad, uint32_t aadlen,+                                 const uint8_t *in, uint32_t inlen,+                                 uint32_t taglen, aes_key *hpkey,+                                 uint32_t sampleoff, uint8_t *mask)+{+	switch (key->strength) {+	case 0:+		crypton_aes_armv8_gcm_fused128(out, ht, key, nonce, aad, aadlen,+		                               in, inlen, taglen, hpkey,+		                               sampleoff, mask);+		break;+	case 1:+		crypton_aes_armv8_gcm_fused192(out, ht, key, nonce, aad, aadlen,+		                               in, inlen, taglen, hpkey,+		                               sampleoff, mask);+		break;+	default:+		crypton_aes_armv8_gcm_fused256(out, ht, key, nonce, aad, aadlen,+		                               in, inlen, taglen, hpkey,+		                               sampleoff, mask);+		break;+	}+}++CRYPTON_TARGET_ARMV8_CRYPTO+int crypton_aes_armv8_gcm_fused_dec(uint8_t *out, const block128 *ht,+                                    aes_key *key, const uint8_t *nonce,+                                    const uint8_t *aad, uint32_t aadlen,+                                    const uint8_t *in, uint32_t inlen,+                                    const uint8_t *tag, uint32_t taglen,+                                    uint8_t *outtag)+{+	switch (key->strength) {+	case 0:+		return crypton_aes_armv8_gcm_fused_dec128(out, ht, key, nonce,+		                                          aad, aadlen, in, inlen,+		                                          tag, taglen, outtag);+	case 1:+		return crypton_aes_armv8_gcm_fused_dec192(out, ht, key, nonce,+		                                          aad, aadlen, in, inlen,+		                                          tag, taglen, outtag);+	default:+		return crypton_aes_armv8_gcm_fused_dec256(out, ht, key, nonce,+		                                          aad, aadlen, in, inlen,+		                                          tag, taglen, outtag);+	}+}
+ cbits/aes/armv8_impl.c view
@@ -0,0 +1,824 @@+/*+ * Included from armv8.c once per key size, with NBR set to the number of+ * rounds and SIZED() naming the functions.  This mirrors x86ni_impl.c.+ *+ * Two things here want compile-time constants, and both are worth having.+ * With the round count fixed the compiler keeps the round keys scheduled+ * instead of reloading them against a count read out of the key.  With the+ * blocks in flight fixed it interleaves that many independent chains, which+ * is what covers the latency of AESE and AESMC -- one block at a time leaves+ * the pipeline waiting on itself.  On Apple silicon the two together are+ * worth about four times a loop that does one block with a round count from+ * memory.+ *+ * The blocks are named by constant index throughout, and every step is+ * written out one per block rather than left to a loop over s[i].  Such a+ * loop is only as good as the compiler's willingness to unroll it, and GCC+ * at -O2 declines: s[] then lives on the stack and each round turns into a+ * load and a store, which measured slower than the one-block code this+ * replaces.  Spelling the steps out costs nothing and leaves nothing to+ * decide.+ */++/* Eight chains is where the return flattens out on the cores measured. */+#define WAY 8++#define EACH1(m) m(0)+#define EACH8(m) m(0) m(1) m(2) m(3) m(4) m(5) m(6) m(7)++#define LOAD_IN(i)   s[i] = vld1q_u8((const uint8_t *) (input + (i)));+#define STORE_OUT(i) vst1q_u8((uint8_t *) (output + (i)), s[i]);++#define ENC_STEP(i)  s[i] = vaesmcq_u8(vaeseq_u8(s[i], k_));+#define ENC_LAST(i)  s[i] = veorq_u8(vaeseq_u8(s[i], k_), l_);+#define DEC_STEP(i)  s[i] = vaesimcq_u8(vaesdq_u8(s[i], k_));+#define DEC_LAST(i)  s[i] = veorq_u8(vaesdq_u8(s[i], k_), l_);++/* Encrypt the blocks EACH names, in place in s[].  rk must be in scope. */+#define ENC_ROUNDS(EACH)                                                     \+	do {                                                                 \+		int r_;                                                      \+		for (r_ = 0; r_ < NBR - 1; r_++) {                           \+			const uint8x16_t k_ = vld1q_u8(rk + 16 * r_);        \+			EACH(ENC_STEP)                                       \+		}                                                            \+		{                                                            \+			const uint8x16_t k_ = vld1q_u8(rk + 16 * (NBR - 1)); \+			const uint8x16_t l_ = vld1q_u8(rk + 16 * NBR);       \+			EACH(ENC_LAST)                                       \+		}                                                            \+	} while (0)++/*+ * Decrypt them.  fwd and inv must be in scope: the schedule is k[nbr],+ * imc(k[nbr-1]) .. imc(k[1]), k[0], so the two ends come from the forward+ * keys and the middle from the inverted ones.+ */+#define DEC_ROUNDS(EACH)                                                     \+	do {                                                                 \+		int r_;                                                      \+		{                                                            \+			const uint8x16_t k_ = vld1q_u8(fwd + 16 * NBR);      \+			EACH(DEC_STEP)                                       \+		}                                                            \+		for (r_ = 0; r_ < NBR - 2; r_++) {                           \+			const uint8x16_t k_ = vld1q_u8(inv + 16 * r_);       \+			EACH(DEC_STEP)                                       \+		}                                                            \+		{                                                            \+			const uint8x16_t k_ = vld1q_u8(inv + 16 * (NBR - 2));\+			const uint8x16_t l_ = vld1q_u8(fwd);                 \+			EACH(DEC_LAST)                                       \+		}                                                            \+	} while (0)++CRYPTON_TARGET_ARMV8_CRYPTO+void SIZED(crypton_aes_armv8_encrypt_block)(aes_block *output, aes_key *key, aes_block *input)+{+	const uint8_t *rk = FWD(key);+	uint8x16_t s[1];++	EACH1(LOAD_IN);+	ENC_ROUNDS(EACH1);+	EACH1(STORE_OUT);+}++CRYPTON_TARGET_ARMV8_CRYPTO+void SIZED(crypton_aes_armv8_decrypt_block)(aes_block *output, aes_key *key, aes_block *input)+{+	const uint8_t *fwd = FWD(key);+	const uint8_t *inv = INV(key);+	uint8x16_t s[1];++	EACH1(LOAD_IN);+	DEC_ROUNDS(EACH1);+	EACH1(STORE_OUT);+}++CRYPTON_TARGET_ARMV8_CRYPTO+void SIZED(crypton_aes_armv8_encrypt_ecb)(aes_block *output, aes_key *key, aes_block *input, uint32_t nb_blocks)+{+	const uint8_t *rk = FWD(key);+	uint8x16_t s[WAY];++	for (; nb_blocks >= WAY; nb_blocks -= WAY, input += WAY, output += WAY) {+		EACH8(LOAD_IN);+		ENC_ROUNDS(EACH8);+		EACH8(STORE_OUT);+	}+	for (; nb_blocks > 0; nb_blocks--, input++, output++) {+		EACH1(LOAD_IN);+		ENC_ROUNDS(EACH1);+		EACH1(STORE_OUT);+	}+}++CRYPTON_TARGET_ARMV8_CRYPTO+void SIZED(crypton_aes_armv8_decrypt_ecb)(aes_block *output, aes_key *key, aes_block *input, uint32_t nb_blocks)+{+	const uint8_t *fwd = FWD(key);+	const uint8_t *inv = INV(key);+	uint8x16_t s[WAY];++	for (; nb_blocks >= WAY; nb_blocks -= WAY, input += WAY, output += WAY) {+		EACH8(LOAD_IN);+		DEC_ROUNDS(EACH8);+		EACH8(STORE_OUT);+	}+	for (; nb_blocks > 0; nb_blocks--, input++, output++) {+		EACH1(LOAD_IN);+		DEC_ROUNDS(EACH1);+		EACH1(STORE_OUT);+	}+}++/* CBC encryption chains, so there is nothing to interleave.  It still gains+ * the round keys staying put. */+CRYPTON_TARGET_ARMV8_CRYPTO+void SIZED(crypton_aes_armv8_encrypt_cbc)(aes_block *output, aes_key *key, aes_block *_iv, aes_block *input, uint32_t nb_blocks)+{+	const uint8_t *rk = FWD(key);+	uint8x16_t iv = vld1q_u8((const uint8_t *) _iv);+	uint8x16_t s[1];++	for (; nb_blocks-- > 0; input++, output++) {+		s[0] = veorq_u8(iv, vld1q_u8((const uint8_t *) input));+		ENC_ROUNDS(EACH1);+		iv = s[0];+		EACH1(STORE_OUT);+	}+}++/* Decryption does not chain: each block is deciphered on its own and then+ * XORed with the ciphertext before it, so it interleaves like ECB. */+/* c[] holds the previous block at index 0 and this group's ciphertext after+ * it, so block i is XORed with c[i] and the next group starts from c[WAY]. */+#define CBC_KEEP(i)  c[(i) + 1] = s[i];+#define CBC_XOR(i)   vst1q_u8((uint8_t *) (output + (i)), veorq_u8(s[i], c[i]));++CRYPTON_TARGET_ARMV8_CRYPTO+void SIZED(crypton_aes_armv8_decrypt_cbc)(aes_block *output, aes_key *key, aes_block *_iv, aes_block *input, uint32_t nb_blocks)+{+	const uint8_t *fwd = FWD(key);+	const uint8_t *inv = INV(key);+	uint8x16_t iv = vld1q_u8((const uint8_t *) _iv);+	uint8x16_t s[WAY], c[WAY + 1];++	for (; nb_blocks >= WAY; nb_blocks -= WAY, input += WAY, output += WAY) {+		EACH8(LOAD_IN);+		c[0] = iv;+		EACH8(CBC_KEEP);+		DEC_ROUNDS(EACH8);+		EACH8(CBC_XOR);+		iv = c[WAY];+	}+	for (; nb_blocks > 0; nb_blocks--, input++, output++) {+		EACH1(LOAD_IN);+		c[1] = s[0];+		DEC_ROUNDS(EACH1);+		vst1q_u8((uint8_t *) output, veorq_u8(s[0], iv));+		iv = c[1];+	}+}++/*+ * CTR counts the whole 128 bits big-endian, with the carry crossing the+ * halves.  The arithmetic is kept identical to+ * crypton_aes_generic_encrypt_ctr, which also leaves the caller's IV alone.+ */+#define CTR_SET(i)  s[i] = vreinterpretq_u8_u64(vsetq_lane_u64(cpu_to_be64(lo + (i)), base, 1));+#define CTR_XOR(i)  vst1q_u8(output + 16 * (i), \+                             veorq_u8(s[i], vld1q_u8(input + 16 * (i))));++CRYPTON_TARGET_ARMV8_CRYPTO+void SIZED(crypton_aes_armv8_encrypt_ctr)(uint8_t *output, aes_key *key, aes_block *iv, uint8_t *input, uint32_t len)+{+	const uint8_t *rk = FWD(key);+	uint32_t nb_blocks = len / 16;+	uint32_t remaining = len % 16;+	aes_block ctr;+	uint8x16_t s[WAY];+	uint32_t i;++	block128_copy(&ctr, iv);++	/*+	 * The counter goes through memory only when its low half is about to+	 * wrap.  Otherwise it stays in registers: the top eight bytes do not+	 * change and the bottom eight are one add away.  That matters -- with+	 * a store and a reload for every block, CTR ran at the same speed for+	 * 128-bit and 256-bit keys, which is the giveaway that the cipher was+	 * not what it was waiting for.+	 */+	for (; nb_blocks >= WAY; nb_blocks -= WAY, input += 16 * WAY, output += 16 * WAY) {+		uint64_t lo = be64_to_cpu(ctr.q[1]);++		if (lo + (WAY - 1) < lo) {+			/* a block in this group carries into the top half;+			 * let the scalar increment deal with it */+			for (i = 0; i < WAY; i++, block128_inc_be(&ctr))+				s[i] = vld1q_u8((const uint8_t *) &ctr);+		} else {+			const uint64x2_t base =+			    vreinterpretq_u64_u8(vld1q_u8((const uint8_t *) &ctr));++			EACH8(CTR_SET);++			/* no block above needed a carry, but the counter left+			 * for the next group still can */+			ctr.q[1] = cpu_to_be64(lo + WAY);+			if (lo + WAY < lo)+				ctr.q[0] = cpu_to_be64(be64_to_cpu(ctr.q[0]) + 1);+		}+		ENC_ROUNDS(EACH8);+		EACH8(CTR_XOR);+	}+	for (; nb_blocks > 0; nb_blocks--, input += 16, output += 16) {+		s[0] = vld1q_u8((const uint8_t *) &ctr);+		block128_inc_be(&ctr);+		ENC_ROUNDS(EACH1);+		vst1q_u8(output, veorq_u8(s[0], vld1q_u8(input)));+	}+	if (remaining) {+		aes_block o;++		s[0] = vld1q_u8((const uint8_t *) &ctr);+		ENC_ROUNDS(EACH1);+		vst1q_u8((uint8_t *) &o, s[0]);+		for (i = 0; i < remaining; i++)+			output[i] = o.b[i] ^ input[i];+	}+}+++/*+ * GCM, rather than the generic loop calling the block function once per+ * block through the branch table.  Eight counter blocks go through the+ * rounds together, and their GHASH folds into a single reduction with+ * H^8 .. H^1, so a group costs one reduction instead of eight.  The tag+ * and the counter stay in registers across the whole run.+ *+ * GCM's counter is the low 32 bits only and wraps there, so unlike CTR+ * there is no carry to chase: the top twelve bytes never move.+ *+ * What this loop is short of is not overlap but instructions.  A group of+ * eight blocks compiles to 383 of them on an Apple M4, and only 152 are the+ * cipher -- eighty AESE and seventy-two AESMC.  The rest is the GHASH and+ * the counters.  Measured against OpenSSL on the same machine, 16 KiB+ * messages under AES-128:+ *+ *     crypton, this loop with the GHASH taken out     19883 MB/s+ *     OpenSSL, AES-128-CTR                            17176+ *     OpenSSL, AES-128-GCM                             9869+ *     crypton, AES-128-GCM                             8641+ *+ * The cipher here is ahead of OpenSSL's; all of the 0.87 is the GHASH.+ * Four ways of closing it were tried and every one measured worse, so they+ * are written down here rather than tried again.  Each was checked to give+ * the same ciphertext and tag as this code for three key sizes and+ * thirty-five lengths, encrypt and decrypt, before being timed:+ *+ *     the GHASH held back a group and spread through the next    -1.3%+ *       group's AES rounds, so that the two do not queue         to -3.6%+ *     Karatsuba: three multiplications for the two halves        -1.6%+ *       instead of four+ *     the same with every product on a lane the instruction       0.0%+ *       reaches, which does remove sixteen fmov a group+ *     the same again with the H powers' halves added together    -3%+ *       already, in the spare half of htable+ *+ * The first fails because there was nothing to gain: a group's AES depends+ * on nothing in the group before it, so a wide out-of-order core already+ * runs the two together, and holding a group back only adds copies.  The+ * rest fail for one reason -- none of them makes the loop shorter.+ * Karatsuba buys a PMULL for two EOR and an EXT, and the folded table turns+ * sixteen `dup` into sixteen `ld1r` and sixteen more address adds.+ *+ * That last sentence used to end by saying a count which does come down+ * wants the data laid out differently.  It does, and since the GHASH was+ * rewritten against a twisted H -- see cbits/aes/armv8.c -- it is laid out+ * differently, so the figures above are what the *previous* GHASH gave.+ * Karatsuba pays now that the reduction it has to carry is a third of what+ * it was; the other three are untried in the new representation and the+ * first of them has no more reason to work than it had.+ */+#define GCM_CTR(i)   s[i] = vreinterpretq_u8_u32(vsetq_lane_u32(cpu_to_be32(c + 1 + (i)), base, 3));+#define GCM_ENC(i)   { const uint8x16_t m_ = vld1q_u8(input + 16 * (i)); \+                       s[i] = veorq_u8(s[i], m_); \+                       vst1q_u8(output + 16 * (i), s[i]); }+#define GCM_DEC(i)   { const uint8x16_t m_ = vld1q_u8(input + 16 * (i)); \+                       vst1q_u8(output + 16 * (i), veorq_u8(s[i], m_)); \+                       s[i] = m_; }+#define GCM_GHASH(i)                                                          \+	{                                                                     \+		poly64x2_t b_ = vreinterpretq_p64_u8(vrev64q_u8(s[i]));       \+		if ((i) == 0)                                                 \+			b_ = vreinterpretq_p64_u64(veorq_u64(                 \+			    vreinterpretq_u64_p64(b_), acc));                 \+		GHASH_MUL(b_, GHASH_POW(ht, WAY - 1 - (i)),                   \+		          GHASH_KARAT(ht, WAY - 1 - (i)), gH, gM, gL);        \+	}++/* the eight blocks now in s[] are the ciphertext; fold them into the tag */+#define GCM_FOLD()                                                            \+	do {                                                                  \+		uint64x2_t gH = vdupq_n_u64(0), gM = gH, gL = gH;             \+		acc = vreinterpretq_u64_u8(                                   \+		    vextq_u8(vreinterpretq_u8_u64(acc),                       \+		             vreinterpretq_u8_u64(acc), 8));                  \+		EACH8(GCM_GHASH)                                              \+		acc = ghash_reduce(gH, gM, gL);                               \+	} while (0)++#define GCM_PROLOGUE                                                          \+	const uint8_t *rk = FWD(key);                                         \+	const block128 *ht = gcm->htable;                                     \+	uint8x16_t s[WAY];                                                    \+	uint64x2_t acc = vreinterpretq_u64_u8(                                \+	    ghash_swap(vld1q_u8((const uint8_t *) &gcm->tag)));               \+	uint32_t c = be32_to_cpu(gcm->civ.d[3]);                              \+	uint32x4_t base = vreinterpretq_u32_u8(vld1q_u8((const uint8_t *) &gcm->civ))++/* one block, for what is left after the last group of eight */+#define GCM_ONE(load_m, store_c, ghash_of)                                    \+	do {                                                                  \+		const uint8x16_t m_ = (load_m);                               \+		c++;                                                          \+		s[0] = vreinterpretq_u8_u32(vsetq_lane_u32(cpu_to_be32(c), base, 3)); \+		ENC_ROUNDS(EACH1);                                            \+		s[0] = veorq_u8(s[0], m_);                                    \+		(store_c);                                                    \+		acc = ghash_one(acc, (ghash_of), ht);                         \+	} while (0)++#define GCM_EPILOGUE                                                          \+	do {                                                                  \+		gcm->civ.d[3] = cpu_to_be32(c);                               \+		vst1q_u8((uint8_t *) &gcm->tag,                               \+		         ghash_swap(vreinterpretq_u8_u64(acc)));              \+	} while (0)++CRYPTON_TARGET_ARMV8_CRYPTO+void SIZED(crypton_aes_armv8_gcm_encrypt)(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length)+{+	GCM_PROLOGUE;+	uint32_t i;++	gcm->length_input += length;++	for (; length >= 16 * WAY; input += 16 * WAY, output += 16 * WAY, length -= 16 * WAY) {+		EACH8(GCM_CTR);+		c += WAY;+		ENC_ROUNDS(EACH8);+		EACH8(GCM_ENC);+		GCM_FOLD();+	}+	for (; length >= 16; input += 16, output += 16, length -= 16) {+		GCM_ONE(vld1q_u8(input), vst1q_u8(output, s[0]), s[0]);+	}+	if (length) {+		aes_block m, o;++		block128_zero(&m);+		block128_copy_bytes(&m, input, length);+		c++;+		s[0] = vreinterpretq_u8_u32(vsetq_lane_u32(cpu_to_be32(c), base, 3));+		ENC_ROUNDS(EACH1);+		s[0] = veorq_u8(s[0], vld1q_u8((const uint8_t *) &m));+		vst1q_u8((uint8_t *) &o, s[0]);+		block128_zero(&m);+		for (i = 0; i < length; i++)+			output[i] = m.b[i] = o.b[i];+		acc = ghash_one(acc, vld1q_u8((const uint8_t *) &m), ht);+	}+	GCM_EPILOGUE;+}++CRYPTON_TARGET_ARMV8_CRYPTO+void SIZED(crypton_aes_armv8_gcm_decrypt)(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length)+{+	GCM_PROLOGUE;+	uint32_t i;++	gcm->length_input += length;++	for (; length >= 16 * WAY; input += 16 * WAY, output += 16 * WAY, length -= 16 * WAY) {+		EACH8(GCM_CTR);+		c += WAY;+		ENC_ROUNDS(EACH8);+		EACH8(GCM_DEC);+		GCM_FOLD();+	}+	for (; length >= 16; input += 16, output += 16, length -= 16) {+		const uint8x16_t ct = vld1q_u8(input);++		GCM_ONE(ct, vst1q_u8(output, s[0]), ct);+	}+	if (length) {+		aes_block m, o;++		block128_zero(&m);+		block128_copy_bytes(&m, input, length);+		c++;+		s[0] = vreinterpretq_u8_u32(vsetq_lane_u32(cpu_to_be32(c), base, 3));+		ENC_ROUNDS(EACH1);+		s[0] = veorq_u8(s[0], vld1q_u8((const uint8_t *) &m));+		vst1q_u8((uint8_t *) &o, s[0]);+		for (i = 0; i < length; i++)+			output[i] = o.b[i];+		acc = ghash_one(acc, vld1q_u8((const uint8_t *) &m), ht);+	}+	GCM_EPILOGUE;+}+++/*+ * XTS.  The tweak for each block is the one before it doubled, so a group's+ * eight tweaks are a short chain that runs while the eight AES chains are in+ * flight.  The first tweak is the data unit number enciphered under the+ * second key; spoint skips that many blocks into the unit.+ */+#define XTS_IN(i)   s[i] = veorq_u8(vld1q_u8((const uint8_t *) (input + (i))), t[i]);+#define XTS_OUT(i)  vst1q_u8((uint8_t *) (output + (i)), veorq_u8(s[i], t[i]));+/*+ * The tweak is kept in general-purpose registers and moved into a vector+ * one per block.  Doubling it costs three integer operations, and the+ * integer units have nothing else to do here, where the vector ones are+ * busy with the rounds and the exclusive ors: done in vector registers,+ * which is what this did, the eight doublings of a group take about as+ * long as the eight blocks of AES they are for.+ */+#define XTS_TWEAK(i) do {                                                  \+	t[i] = vreinterpretq_u8_u64(                                       \+	    vcombine_u64(vcreate_u64(tlo), vcreate_u64(thi)));             \+	{                                                                  \+		const uint64_t _c = thi >> 63;                             \+		thi = (thi << 1) | (tlo >> 63);                            \+		tlo = (tlo << 1) ^ (_c ? 0x87 : 0);                        \+	}                                                                  \+} while (0);+/*+ * The group after this one's.  Doubling is a chain -- each tweak waits for+ * the one before it -- and eight of them in front of the rounds that want+ * them is time in which nothing else happens, which on a processor whose+ * AES is this fast is most of the block.  Worked out a group early they+ * have nothing to wait for and go through the rounds of the group before,+ * which do not want the same units.  There are registers enough here for+ * both groups at once.+ */+#define XTS_TWEAK_NEXT(i) do {                                             \+	tn[i] = vreinterpretq_u8_u64(                                      \+	    vcombine_u64(vcreate_u64(tlo), vcreate_u64(thi)));             \+	{                                                                  \+		const uint64_t _c = thi >> 63;                             \+		thi = (thi << 1) | (tlo >> 63);                            \+		tlo = (tlo << 1) ^ (_c ? 0x87 : 0);                        \+	}                                                                  \+} while (0);+#define XTS_TWEAK_ROLL(i) do { t[i] = tn[i]; } while (0);++CRYPTON_TARGET_ARMV8_CRYPTO+void SIZED(crypton_aes_armv8_encrypt_xts)(aes_block *output, aes_key *key, aes_key *key2, aes_block *dataunit, uint32_t spoint, aes_block *input, uint32_t nb_blocks)+{+	const uint8_t *rk = FWD(key);+	uint8x16_t s[WAY], t[WAY], tn[WAY];+	uint64_t tlo, thi;++	{+		aes_block first;++		SIZED(crypton_aes_armv8_encrypt_block)(&first, key2, dataunit);+		tlo = first.q[0];+		thi = first.q[1];+	}+	while (spoint-- > 0) {+		const uint64_t c = thi >> 63;++		thi = (thi << 1) | (tlo >> 63);+		tlo = (tlo << 1) ^ (c ? 0x87 : 0);+	}++	EACH8(XTS_TWEAK);+	for (; nb_blocks >= WAY; nb_blocks -= WAY, input += WAY, output += WAY) {+		EACH8(XTS_IN);+		EACH8(XTS_TWEAK_NEXT);+		ENC_ROUNDS(EACH8);+		EACH8(XTS_OUT);+		EACH8(XTS_TWEAK_ROLL);+	}+	/* the group that was made ready and not used */+	{+		const uint64x2_t back = vreinterpretq_u64_u8(t[0]);++		tlo = vgetq_lane_u64(back, 0);+		thi = vgetq_lane_u64(back, 1);+	}+	for (; nb_blocks > 0; nb_blocks--, input++, output++) {+		EACH1(XTS_TWEAK);+		EACH1(XTS_IN);+		ENC_ROUNDS(EACH1);+		EACH1(XTS_OUT);+	}+}++CRYPTON_TARGET_ARMV8_CRYPTO+void SIZED(crypton_aes_armv8_decrypt_xts)(aes_block *output, aes_key *key, aes_key *key2, aes_block *dataunit, uint32_t spoint, aes_block *input, uint32_t nb_blocks)+{+	const uint8_t *fwd = FWD(key);+	const uint8_t *inv = INV(key);+	uint8x16_t s[WAY], t[WAY], tn[WAY];+	uint64_t tlo, thi;++	{+		aes_block first;++		/* the tweak is always enciphered, whichever way the data goes */+		SIZED(crypton_aes_armv8_encrypt_block)(&first, key2, dataunit);+		tlo = first.q[0];+		thi = first.q[1];+	}+	while (spoint-- > 0) {+		const uint64_t c = thi >> 63;++		thi = (thi << 1) | (tlo >> 63);+		tlo = (tlo << 1) ^ (c ? 0x87 : 0);+	}++	EACH8(XTS_TWEAK);+	for (; nb_blocks >= WAY; nb_blocks -= WAY, input += WAY, output += WAY) {+		EACH8(XTS_IN);+		EACH8(XTS_TWEAK_NEXT);+		DEC_ROUNDS(EACH8);+		EACH8(XTS_OUT);+		EACH8(XTS_TWEAK_ROLL);+	}+	/* the group that was made ready and not used */+	{+		const uint64x2_t back = vreinterpretq_u64_u8(t[0]);++		tlo = vgetq_lane_u64(back, 0);+		thi = vgetq_lane_u64(back, 1);+	}+	for (; nb_blocks > 0; nb_blocks--, input++, output++) {+		EACH1(XTS_TWEAK);+		EACH1(XTS_IN);+		DEC_ROUNDS(EACH1);+		EACH1(XTS_OUT);+	}+}++/*+ * One message, one call: the additional data, the counter-mode encryption,+ * the tag and the QUIC header protection mask, with the running tag and the+ * counter kept in registers from end to end.+ *+ * What this saves over composing crypton_aes_gcm_aad, _encrypt and _finish is+ * not the arithmetic but the boundaries.  Each of those reaches its+ * primitives through a branch table, so the 128-bit state goes back to memory+ * at every step and a header of one block pays a reduction of its own.  On an+ * Apple M4 that framing was 0.07 of the 0.112 microseconds a 100-byte packet+ * cost -- more than the encryption of the packet itself.+ *+ * The GHASH is taken in batches of WAY against the powers of H the key+ * already holds, so a batch costs one reduction rather than one per block,+ * and the additional data and the length block ride in the same batches as+ * the ciphertext instead of being multiplied on their own.+ */++/* start a batch, or continue one; blen is how many blocks this batch holds */+#define FG_ABSORB(blk)                                                        \+	do {                                                                  \+		poly64x2_t b_ = vreinterpretq_p64_u8(vrev64q_u8(blk));        \+		if (bn == 0) {                                                \+			uint32_t left_ = gtotal - gidx;                       \+			blen = left_ < WAY ? left_ : WAY;                     \+			acc = vreinterpretq_u64_u8(                           \+			    vextq_u8(vreinterpretq_u8_u64(acc),               \+			             vreinterpretq_u8_u64(acc), 8));          \+			b_ = vreinterpretq_p64_u64(veorq_u64(                 \+			    vreinterpretq_u64_p64(b_), acc));                 \+			gH = vdupq_n_u64(0);                                  \+			gM = gH;                                              \+			gL = gH;                                              \+		}                                                             \+		GHASH_MUL(b_, GHASH_POW(ht, blen - bn - 1),                   \+		          GHASH_KARAT(ht, blen - bn - 1), gH, gM, gL);        \+		gidx++; bn++;                                                 \+		if (bn == blen) { acc = ghash_reduce(gH, gM, gL); bn = 0; }   \+	} while (0)++/* a block that is short, zero padded, as GHASH wants it */+#define FG_PARTIAL(p, n)                                                      \+	({ uint8_t buf_[16]; memset(buf_, 0, 16); memcpy(buf_, (p), (n));     \+	   vld1q_u8(buf_); })++CRYPTON_TARGET_ARMV8_CRYPTO+void SIZED(crypton_aes_armv8_gcm_fused)(uint8_t *out, const block128 *ht,+                                        aes_key *key, const uint8_t *nonce,+                                        const uint8_t *aad, uint32_t aadlen,+                                        const uint8_t *in, uint32_t inlen,+                                        uint32_t taglen, aes_key *hpkey,+                                        uint32_t sampleoff, uint8_t *mask)+{+	const uint8_t *rk = FWD(key);+	uint8x16_t s[WAY];+	uint8x16_t ek0;+	uint64x2_t acc = vdupq_n_u64(0), gH = acc, gM = acc, gL = acc;+	uint32x4_t base;+	uint32_t c = 1, bn = 0, blen = 0, gidx = 0;+	uint32_t gtotal = (aadlen + 15) / 16 + (inlen + 15) / 16 + 1;+	uint32_t i, done;+	uint8_t y0[16], lenb[16];+	uint64_t la, lc;++	memcpy(y0, nonce, 12);+	y0[12] = 0; y0[13] = 0; y0[14] = 0; y0[15] = 1;+	base = vreinterpretq_u32_u8(vld1q_u8(y0));++	s[0] = vld1q_u8(y0);+	ENC_ROUNDS(EACH1);+	ek0 = s[0];++	for (i = 0; i + 16 <= aadlen; i += 16)+		FG_ABSORB(vld1q_u8(aad + i));+	if (i < aadlen)+		FG_ABSORB(FG_PARTIAL(aad + i, aadlen - i));++	for (done = 0; done + 16 * WAY <= inlen; done += 16 * WAY) {+		const uint8_t *p = in + done;+		uint8_t *q = out + done;++		EACH8(GCM_CTR);+		c += WAY;+		ENC_ROUNDS(EACH8);+		{+			const uint8_t *input = p;+			uint8_t *output = q;+			EACH8(GCM_ENC);+		}+		FG_ABSORB(s[0]); FG_ABSORB(s[1]); FG_ABSORB(s[2]); FG_ABSORB(s[3]);+		FG_ABSORB(s[4]); FG_ABSORB(s[5]); FG_ABSORB(s[6]); FG_ABSORB(s[7]);+	}++	for (; done < inlen; done += 16) {+		uint32_t n = inlen - done < 16 ? inlen - done : 16;+		uint8x16_t m_ = n == 16 ? vld1q_u8(in + done)+		                        : FG_PARTIAL(in + done, n);+		c++;+		s[0] = vreinterpretq_u8_u32(vsetq_lane_u32(cpu_to_be32(c), base, 3));+		ENC_ROUNDS(EACH1);+		s[0] = veorq_u8(s[0], m_);+		if (n == 16) {+			vst1q_u8(out + done, s[0]);+		} else {+			uint8_t buf_[16];+			vst1q_u8(buf_, s[0]);+			memcpy(out + done, buf_, n);+			memset(buf_ + n, 0, 16 - n);+			s[0] = vld1q_u8(buf_);+		}+		FG_ABSORB(s[0]);+	}++	la = (uint64_t) aadlen << 3;+	lc = (uint64_t) inlen << 3;+	for (i = 0; i < 8; i++) lenb[i] = (uint8_t) (la >> (56 - 8 * i));+	for (i = 0; i < 8; i++) lenb[8 + i] = (uint8_t) (lc >> (56 - 8 * i));+	FG_ABSORB(vld1q_u8(lenb));++	{+		uint8_t tbuf[16];+		vst1q_u8(tbuf, veorq_u8(ghash_swap(vreinterpretq_u8_u64(acc)), ek0));+		memcpy(out + inlen, tbuf, taglen);+	}++	if (hpkey != 0 && mask != 0) {+		block128 sample, m;+		memcpy(&sample, out + sampleoff, 16);+		crypton_aes_encrypt_ecb(&m, hpkey, &sample, 1);+		memcpy(mask, &m, 16);+	}+}+++/*+ * The same for decryption.  GCM_DEC leaves the ciphertext in s[] once it has+ * written the plaintext out, which is what GHASH wants, so the only other+ * difference is the end: the tag is compared here rather than written, every+ * byte of it whichever way the answer goes.+ */+CRYPTON_TARGET_ARMV8_CRYPTO+int SIZED(crypton_aes_armv8_gcm_fused_dec)(uint8_t *out, const block128 *ht,+                                           aes_key *key, const uint8_t *nonce,+                                           const uint8_t *aad, uint32_t aadlen,+                                           const uint8_t *in, uint32_t inlen,+                                           const uint8_t *tagp, uint32_t taglen,+                                           uint8_t *outtag)+{+	const uint8_t *rk = FWD(key);+	uint8x16_t s[WAY];+	uint8x16_t ek0;+	uint64x2_t acc = vdupq_n_u64(0), gH = acc, gM = acc, gL = acc;+	uint32x4_t base;+	uint32_t c = 1, bn = 0, blen = 0, gidx = 0;+	uint32_t gtotal = (aadlen + 15) / 16 + (inlen + 15) / 16 + 1;+	uint32_t i, done;+	uint8_t y0[16], lenb[16], want[16];+	uint64_t la, lc;+	uint8_t diff = 0;++	memcpy(y0, nonce, 12);+	y0[12] = 0; y0[13] = 0; y0[14] = 0; y0[15] = 1;+	base = vreinterpretq_u32_u8(vld1q_u8(y0));++	s[0] = vld1q_u8(y0);+	ENC_ROUNDS(EACH1);+	ek0 = s[0];++	for (i = 0; i + 16 <= aadlen; i += 16)+		FG_ABSORB(vld1q_u8(aad + i));+	if (i < aadlen)+		FG_ABSORB(FG_PARTIAL(aad + i, aadlen - i));++	for (done = 0; done + 16 * WAY <= inlen; done += 16 * WAY) {+		const uint8_t *p = in + done;+		uint8_t *q = out + done;++		EACH8(GCM_CTR);+		c += WAY;+		ENC_ROUNDS(EACH8);+		{+			const uint8_t *input = p;+			uint8_t *output = q;+			EACH8(GCM_DEC);+		}+		FG_ABSORB(s[0]); FG_ABSORB(s[1]); FG_ABSORB(s[2]); FG_ABSORB(s[3]);+		FG_ABSORB(s[4]); FG_ABSORB(s[5]); FG_ABSORB(s[6]); FG_ABSORB(s[7]);+	}++	for (; done < inlen; done += 16) {+		uint32_t n = inlen - done < 16 ? inlen - done : 16;+		uint8x16_t m_ = n == 16 ? vld1q_u8(in + done)+		                        : FG_PARTIAL(in + done, n);+		c++;+		s[0] = vreinterpretq_u8_u32(vsetq_lane_u32(cpu_to_be32(c), base, 3));+		ENC_ROUNDS(EACH1);+		{+			uint8x16_t pl = veorq_u8(s[0], m_);+			if (n == 16) {+				vst1q_u8(out + done, pl);+			} else {+				uint8_t buf_[16];+				vst1q_u8(buf_, pl);+				memcpy(out + done, buf_, n);+			}+		}+		FG_ABSORB(m_);+	}++	la = (uint64_t) aadlen << 3;+	lc = (uint64_t) inlen << 3;+	for (i = 0; i < 8; i++) lenb[i] = (uint8_t) (la >> (56 - 8 * i));+	for (i = 0; i < 8; i++) lenb[8 + i] = (uint8_t) (lc >> (56 - 8 * i));+	FG_ABSORB(vld1q_u8(lenb));++	vst1q_u8(want, veorq_u8(ghash_swap(vreinterpretq_u8_u64(acc)), ek0));+	if (outtag) {+		/* The caller holds the expected tag and will compare it itself. */+		memcpy(outtag, want, taglen);+		return 1;+	}+	for (i = 0; i < taglen; i++)+		diff |= (uint8_t) (want[i] ^ tagp[i]);+	return diff == 0;+}++#undef FG_ABSORB+#undef FG_PARTIAL++#undef WAY+#undef EACH1+#undef EACH8+#undef LOAD_IN+#undef STORE_OUT+#undef ENC_STEP+#undef ENC_LAST+#undef DEC_STEP+#undef DEC_LAST+#undef ENC_ROUNDS+#undef DEC_ROUNDS+#undef CBC_KEEP+#undef CBC_XOR+#undef CTR_SET+#undef CTR_XOR+#undef XTS_IN+#undef XTS_OUT+#undef XTS_TWEAK+#undef GCM_CTR+#undef GCM_ENC+#undef GCM_DEC+#undef GCM_GHASH+#undef GCM_FOLD+#undef GCM_PROLOGUE+#undef GCM_ONE+#undef GCM_EPILOGUE
cbits/aes/block128.h view
@@ -34,7 +34,21 @@ #include <crypton_bitfn.h> #include <crypton_align.h> -typedef union {+/* Packed, so that the union asks nothing of the address it is at.+ *+ * Several callers here make one of these out of a pointer of their own -- a+ * ciphertext, a tag, a nonce -- and without this that cast produces a pointer+ * the standard says may not exist, and reading q[] out of it is a member+ * access at an address the type is not aligned for.  crypton_align.h used to+ * answer that with need_alignment, which is zero on i386 and x86-64: the two+ * places the access is architecturally fine and the standard still says+ * nothing about it.  UndefinedBehaviorSanitizer reported it.+ *+ * With the alignment declared to be one, the compiler is the one that knows+ * what the target can do: on i386, x86-64 and AArch64 it emits the same load+ * and store it emitted before, and where a target cannot read a word off an+ * odd address it emits what that target needs. */+typedef union __attribute__((packed)) {        uint64_t q[2];        uint32_t d[4];        uint16_t w[8];
+ cbits/aes/gcm_fused_x86.c view
@@ -0,0 +1,1013 @@+/*+ * A fused AES-GCM for x86-64, following the design Kazuho Oku sets out in+ * "QUICむけにAES-GCM実装を最適化した話" and implements in picotls's+ * lib/fusion.c: keep AES-NI issuing every clock and fit everything else --+ * the additional data, the tag, the QUIC header protection mask -- into the+ * gaps it leaves.  Written in C with intrinsics rather than assembly, for+ * the same reason he gives: the scheduling is what is complicated here, and+ * it has to stay readable to stay correct.+ *+ * Parts of this file follow fusion closely enough to say so: `loadn` and the+ * two tables it reads, `loadn_page_end` and `storen` are its `loadn128`,+ * `loadn_end_of_page` and `storen128` in another spelling, and the+ * reduction of a 256-bit product is the sequence fusion takes from Gueron's+ * "AES-GCM for Efficient Authenticated Encryption".  fusion is under the MIT+ * license, which is in cbits/aes/LICENSE.fusion beside this file.+ *+ * The powers of H are built once per key, so the additional data, the+ * ciphertext and the length block are absorbed against them in batches that+ * share one reduction, rather than each block paying for a reduction of its+ * own.  How many powers, and so how large a batch, is+ * CRYPTON_GCM_FUSED_POWERS in crypton_aes.h.+ *+ * Only messages shorter than CRYPTON_GCM_FUSED_MAX_MESSAGE come here.  Above+ * that the stitched assembly in cbits/asm is faster, and crypton_aes.c sends+ * them there instead; below it, that assembly will not start at all.+ */++#include <crypton_cpu.h>++#ifdef WITH_GCM_FUSED++#include <stdint.h>+#include <string.h>+#include <wmmintrin.h>+#include <smmintrin.h>+#include <tmmintrin.h>++#include <crypton_aes.h>+#include <aes/gcm_fused_x86.h>++/*+ * aes_key is a struct of bytes, so its round keys sit wherever the members+ * before them leave them -- eight bytes in, as it happens.  A __m128i *+ * pointed at that gets an aligned load and a fault, so each round key is+ * fetched with an unaligned load instead.  Copying them somewhere aligned+ * would cost a copy per call, which at these message lengths is a tenth of+ * the whole; the load is free from L1 and the round key is fetched once for+ * all six lanes.+ */+#define RK(p, i) \+    _mm_loadu_si128((const __m128i *) ((const uint8_t *) (p) + 16 * (size_t) (i)))++/* a full sixteen-byte reversal: mask bytes 15,14,...,0 */+static const __m128i BSWAP = {0x08090a0b0c0d0e0fLL, 0x0001020304050607LL};+++#define TGT __attribute__((target("aes,pclmul,sse4.1")))++/* the two halves of a value added together: the term Karatsuba needs, and it+ * does not depend on what the value is multiplied by */+TGT static inline __m128i fold(__m128i a)+{+    return _mm_xor_si128(a, _mm_unpackhi_epi64(a, a));+}++/* H multiplied by x in the field, which is what puts H and its powers one+ * bit up: GCM numbers the bits of a field element the other way round from+ * the way the carry-less multiply does, and pre-shifting is what saves the+ * correction after every multiply.+ *+ * Written from the definition.  The field is GF(2)[x] modulo x^128 + x^127 ++ * x^126 + x^121 + 1, so multiplying by x is a shift of one place, and the+ * term that leaves the top comes back as the other four. */+TGT static __m128i twist(__m128i h)+{+    /* x^127 + x^126 + x^121 + 1, the terms x^128 is congruent to */+    const __m128i poly = _mm_set_epi64x(0xc200000000000000ULL, 1);+    /* the top bit of each half */+    __m128i tops = _mm_srli_epi64(h, 63);+    /* doubling a polynomial is a shift by one: each half doubles, and the+     * low half's top bit becomes the high half's bottom bit */+    __m128i doubled = _mm_or_si128(_mm_add_epi64(h, h),+                                   _mm_slli_si128(tops, 8));+    /* bit 127 is the one that leaves the field; spread it to a mask by+     * subtracting it from zero, and it brings the four terms back */+    __m128i mask = _mm_sub_epi64(_mm_setzero_si128(),+                                 _mm_unpackhi_epi64(tops, tops));++    return _mm_xor_si128(doubled, _mm_and_si128(mask, poly));+}++/* one reduction of a 256-bit product back into the field */+TGT static __m128i reduce256(__m128i lo, __m128i hi)+{+    const __m128i poly = _mm_set_epi64x(0xc200000000000000ULL, 1);+    __m128i t;++    t = _mm_clmulepi64_si128(lo, poly, 0x10);+    lo = _mm_xor_si128(_mm_shuffle_epi32(lo, 0x4e), t);+    t = _mm_clmulepi64_si128(lo, poly, 0x10);+    lo = _mm_xor_si128(_mm_shuffle_epi32(lo, 0x4e), t);+    return _mm_xor_si128(hi, lo);+}++/*+ * One multiply in exactly the form the hot loop uses it: the left operand+ * plain, the right one already twisted.  Building the table with the same+ * multiply that consumes it is the only way the two conventions cannot drift+ * apart.+ */+TGT static __m128i mul_twisted(__m128i a, __m128i ht)+{+    __m128i lo = _mm_clmulepi64_si128(a, ht, 0x00);+    __m128i hi = _mm_clmulepi64_si128(a, ht, 0x11);+    __m128i mid = _mm_clmulepi64_si128(fold(a), fold(ht), 0x00);++    mid = _mm_xor_si128(mid, _mm_xor_si128(lo, hi));+    lo = _mm_xor_si128(lo, _mm_slli_si128(mid, 8));+    hi = _mm_xor_si128(hi, _mm_srli_si128(mid, 8));+    return reduce256(lo, hi);+}++TGT void crypton_gcm_fused_key_init(aes_gcm_fused *fk, const aes_key *key)+{+    const uint8_t *rk = key->data;+    const int rounds = key->nbr;+    __m128i h, p;+    int i;++    /* H = E_K(0) */+    h = RK(rk, 0);+    for (i = 1; i < rounds; i++) h = _mm_aesenc_si128(h, RK(rk, i));+    h = _mm_aesenclast_si128(h, RK(rk, rounds));+    h = _mm_shuffle_epi8(h, BSWAP);++    {+        __m128i ht = twist(h);+        p = h;+        for (i = 0; i < CRYPTON_GCM_FUSED_POWERS; i++) {+            __m128i t = twist(p);+            _mm_storeu_si128((__m128i *) &fk->p[i].h, t);+            _mm_storeu_si128((__m128i *) &fk->p[i].r, fold(t));+            p = mul_twisted(p, ht);+        }+    }+}++/*+ * The running product.  Three plain locals and a macro, not a struct behind+ * a pointer: taking the address of the accumulators is enough to keep them+ * out of registers, and then every multiply reloads and restores them.  That+ * is the same mistake as reaching a table through an index the compiler+ * cannot fold, and it costs more here because it is on the inner path.+ */+#define GHASH_DECL __m128i glo = _mm_setzero_si128(),                        \+                            ghi = _mm_setzero_si128(),                       \+                            gmid = _mm_setzero_si128(),                      \+                            gtag = _mm_setzero_si128();                      \+                   int gidx = 0, gblen = 0, gbpos = 0++/*+ * Absorb one block.  Blocks are taken in batches of at most CRYPTON_GCM_FUSED_POWERS: the+ * first of a batch carries in the value the batch before it reduced to, the+ * rest go in against descending powers, and the batch ends with the one+ * reduction they share.  With the batch as long as the message this is+ * picotls's single reduction; with it fixed, the state stays a fixed size.+ */+#define GHASH_ONE(blk, unused_power)                                         \+    do {                                                                     \+        __m128i _b = (blk);                                                  \+        __m128i _h, _r;                                                      \+        if (gbpos == 0) {                                                    \+            int _left = gtotal - gidx;                                       \+            gblen = _left < CRYPTON_GCM_FUSED_POWERS ? _left : CRYPTON_GCM_FUSED_POWERS;             \+            _b = _mm_xor_si128(_b, gtag);                                    \+            glo = ghi = gmid = _mm_setzero_si128();                          \+        }                                                                    \+        _h = _mm_loadu_si128((const __m128i *) &fk->p[gblen-gbpos-1].h);     \+        _r = _mm_loadu_si128((const __m128i *) &fk->p[gblen-gbpos-1].r);     \+        glo = _mm_xor_si128(glo, _mm_clmulepi64_si128(_b, _h, 0x00));        \+        ghi = _mm_xor_si128(ghi, _mm_clmulepi64_si128(_b, _h, 0x11));        \+        gmid = _mm_xor_si128(gmid,                                           \+                   _mm_clmulepi64_si128(fold(_b), _r, 0x00));                \+        gidx++; gbpos++;                                                     \+        if (gbpos == gblen) {                                                \+            gtag = ghash_reduce(glo, ghi, gmid);                             \+            gbpos = 0;                                                       \+        }                                                                    \+    } while (0)++TGT static __m128i ghash_reduce(__m128i glo, __m128i ghi, __m128i gmid)+{+    __m128i mid = _mm_xor_si128(gmid, _mm_xor_si128(glo, ghi));+    __m128i lo = _mm_xor_si128(glo, _mm_slli_si128(mid, 8));+    __m128i hi = _mm_xor_si128(ghi, _mm_srli_si128(mid, 8));++    return reduce256(lo, hi);+}++/* zero every byte from n onwards, so a partial block can be fed to GHASH+ * without being written out and read back */+TGT static __m128i clampn(__m128i v, size_t n)+{+    const __m128i idx = {0x0706050403020100LL, 0x0f0e0d0c0b0a0908LL};+    return _mm_and_si128(v, _mm_cmpgt_epi8(_mm_set1_epi8((char) n), idx));+}++/*+ * A short block, zero padded, without going through the stack.+ *+ * The obvious way -- zero sixteen bytes, copy n in, load them back -- is+ * three trips to memory with a store the load has to wait for, and at these+ * lengths that is a tenth of the whole call.  Reading the sixteen bytes and+ * masking off what is above n is two instructions, but it reads past the end+ * of what the caller gave, so it has to be sure those bytes exist.+ *+ * They do unless the block ends a page.  A load of sixteen bytes that starts+ * at least sixteen from the end of a page stays inside it; and if the n bytes+ * asked for themselves cross the boundary then the next page is there to be+ * read as well.  What is left is a block near the end of a page whose own+ * bytes stop short of it, and that is read aligned -- which cannot leave the+ * page -- and shuffled down.  This is how picotls's fusion does it.+ */++/* thirty-two bytes of ones, then thirty-one of zeros: sixteen loaded from+ * 32 - n give n bytes of ones and the rest zeros */+static const uint8_t loadn_mask[63] = {+    0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,+    0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,+    0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,+    0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff};++/* the first sixteen map to byte offsets, the rest to zero */+static const uint8_t loadn_shuffle[31] = {+    0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,+    0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,+    0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80,+    0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80};++#if defined(__has_feature)+#if __has_feature(address_sanitizer)+#define NO_ASAN __attribute__((no_sanitize_address))+#endif+#elif defined(__SANITIZE_ADDRESS__)+#define NO_ASAN __attribute__((no_sanitize_address))+#endif+#ifndef NO_ASAN+#define NO_ASAN+#endif++TGT NO_ASAN static __m128i loadn_page_end(const uint8_t *p, size_t n)+{+    uintptr_t shift = (uintptr_t) p & 15;+    __m128i pattern = _mm_loadu_si128((const __m128i *) (loadn_shuffle + shift));++    (void) n;+    return _mm_shuffle_epi8(+        _mm_load_si128((const __m128i *) ((uintptr_t) p - shift)), pattern);+}++TGT NO_ASAN static __m128i loadn(const uint8_t *p, size_t n)+{+    __m128i mask = _mm_loadu_si128((const __m128i *) (loadn_mask + 32 - n));+    uintptr_t mod4k = (uintptr_t) p % 4096;+    __m128i v;++    if (mod4k <= 4096 - 16 || mod4k + n > 4096)+        v = _mm_loadu_si128((const __m128i *) p);+    else+        v = loadn_page_end(p, n);+    return _mm_and_si128(v, mask);+}++TGT static void storen(uint8_t *p, __m128i v, size_t n)+{+    uint8_t buf[16];+    _mm_storeu_si128((__m128i *) buf, v);+    memcpy(p, buf, n);+}++/* One block.  The ten rounds of the common case are written out for the+ * same reason the six-wide group is: a loop over a round count that lives in+ * the key leaves every round key fetched through an index the compiler+ * cannot fold, and adds a branch to a chain that is already latency-bound. */+TGT static __m128i aes_one_block(const uint8_t *rk, int rounds, __m128i v)+{+    const uint8_t *k = rk;+    int i;++    if (rounds == 10) {+        v = _mm_xor_si128(v, RK(k, 0));+        v = _mm_aesenc_si128(v, RK(k, 1));+        v = _mm_aesenc_si128(v, RK(k, 2));+        v = _mm_aesenc_si128(v, RK(k, 3));+        v = _mm_aesenc_si128(v, RK(k, 4));+        v = _mm_aesenc_si128(v, RK(k, 5));+        v = _mm_aesenc_si128(v, RK(k, 6));+        v = _mm_aesenc_si128(v, RK(k, 7));+        v = _mm_aesenc_si128(v, RK(k, 8));+        v = _mm_aesenc_si128(v, RK(k, 9));+        return _mm_aesenclast_si128(v, RK(k, 10));+    }+    v = _mm_xor_si128(v, RK(k, 0));+    for (i = 1; i < rounds; i++) v = _mm_aesenc_si128(v, RK(k, i));+    return _mm_aesenclast_si128(v, RK(k, rounds));+}++/*+ * Six blocks at once, with lane 5 free to run a different key schedule from+ * the rest.  Which schedule that lane uses is chosen once, into a pointer,+ * rather than tested inside the rounds, and the six live in named variables+ * rather than an array -- an array indexed by a running variable goes to+ * memory, and then every round is a load and a store instead of a register+ * to register operation, which is the whole of what this is trying to avoid.+ *+ * Lanes beyond what the caller needs still run.  Six are in flight whatever+ * the message length, so the spare ones cost nothing, and that is exactly+ * why the header protection mask and E(K,Y0) are worth putting in them+ * instead of giving each a dependent chain of its own.+ */+#define WIDE6(alt)                                                           \+    do {                                                                     \+        const uint8_t *ak = (alt);                                           \+        int r;                                                               \+        t0 = _mm_xor_si128(t0, RK(rk, 0));                                   \+        t1 = _mm_xor_si128(t1, RK(rk, 0));                                   \+        t2 = _mm_xor_si128(t2, RK(rk, 0));                                   \+        t3 = _mm_xor_si128(t3, RK(rk, 0));                                   \+        t4 = _mm_xor_si128(t4, RK(rk, 0));                                   \+        t5 = _mm_xor_si128(t5, RK(ak, 0));                                       \+        for (r = 1; r < rounds; r++) {                                   \+            __m128i k = RK(rk, r);                                           \+            t0 = _mm_aesenc_si128(t0, k);                                    \+            t1 = _mm_aesenc_si128(t1, k);                                    \+            t2 = _mm_aesenc_si128(t2, k);                                    \+            t3 = _mm_aesenc_si128(t3, k);                                    \+            t4 = _mm_aesenc_si128(t4, k);                                    \+            t5 = _mm_aesenc_si128(t5, RK(ak, r));                                \+            GSTEP();                                                         \+        }                                                                    \+        {                                                                    \+            __m128i k = RK(rk, rounds);                                  \+            t0 = _mm_aesenclast_si128(t0, k);                                \+            t1 = _mm_aesenclast_si128(t1, k);                                \+            t2 = _mm_aesenclast_si128(t2, k);                                \+            t3 = _mm_aesenclast_si128(t3, k);                                \+            t4 = _mm_aesenclast_si128(t4, k);                                \+            t5 = _mm_aesenclast_si128(t5, RK(ak, rounds));                   \+        }                                                                    \+    } while (0)++/* the same with nothing queued to absorb: the decryption side takes its+ * GHASH straight from the input and has no queue to drain */+#define WIDE6_NOQ(alt)                                                           \+    do {                                                                     \+        const uint8_t *ak = (alt);                                           \+        int r;                                                               \+        t0 = _mm_xor_si128(t0, RK(rk, 0));                                   \+        t1 = _mm_xor_si128(t1, RK(rk, 0));                                   \+        t2 = _mm_xor_si128(t2, RK(rk, 0));                                   \+        t3 = _mm_xor_si128(t3, RK(rk, 0));                                   \+        t4 = _mm_xor_si128(t4, RK(rk, 0));                                   \+        t5 = _mm_xor_si128(t5, RK(ak, 0));                                       \+        for (r = 1; r < rounds; r++) {                                   \+            __m128i k = RK(rk, r);                                           \+            t0 = _mm_aesenc_si128(t0, k);                                    \+            t1 = _mm_aesenc_si128(t1, k);                                    \+            t2 = _mm_aesenc_si128(t2, k);                                    \+            t3 = _mm_aesenc_si128(t3, k);                                    \+            t4 = _mm_aesenc_si128(t4, k);                                    \+            t5 = _mm_aesenc_si128(t5, RK(ak, r));                                \+        }                                                                    \+        {                                                                    \+            __m128i k = RK(rk, rounds);                                  \+            t0 = _mm_aesenclast_si128(t0, k);                                \+            t1 = _mm_aesenclast_si128(t1, k);                                \+            t2 = _mm_aesenclast_si128(t2, k);                                \+            t3 = _mm_aesenclast_si128(t3, k);                                \+            t4 = _mm_aesenclast_si128(t4, k);                                \+            t5 = _mm_aesenclast_si128(t5, RK(ak, rounds));                   \+        }                                                                    \+    } while (0)++/*+ * The same written out for the ten rounds of AES-128, with a slot for one+ * queued multiply between each of the first six.  The loop above cannot take+ * them: the round count is a value in the key, so there is no place the+ * compiler knows is a round apart from the next, and a test before each+ * multiply would end the basic block the scheduler works inside.  This pass+ * runs with a group's multiplies still waiting, and on a short message that+ * is most of what it has to do.+ */+#define WROUND6(r, ak)                                                       \+    do {                                                                     \+        __m128i k = RK(rk, r);                                               \+        t0 = _mm_aesenc_si128(t0, k);                                        \+        t1 = _mm_aesenc_si128(t1, k);                                        \+        t2 = _mm_aesenc_si128(t2, k);                                        \+        t3 = _mm_aesenc_si128(t3, k);                                        \+        t4 = _mm_aesenc_si128(t4, k);                                        \+        t5 = _mm_aesenc_si128(t5, RK(ak, r));                                \+    } while (0)++#define WIDE6_10(alt)                                                        \+    do {                                                                     \+        const uint8_t *ak = (alt);                                           \+        t0 = _mm_xor_si128(t0, RK(rk, 0));                                   \+        t1 = _mm_xor_si128(t1, RK(rk, 0));                                   \+        t2 = _mm_xor_si128(t2, RK(rk, 0));                                   \+        t3 = _mm_xor_si128(t3, RK(rk, 0));                                   \+        t4 = _mm_xor_si128(t4, RK(rk, 0));                                   \+        t5 = _mm_xor_si128(t5, RK(ak, 0));                                   \+        WROUND6(1, ak); GAT(0);                                              \+        WROUND6(2, ak); GAT(1);                                              \+        WROUND6(3, ak); GAT(2);                                              \+        WROUND6(4, ak); GAT(3);                                              \+        WROUND6(5, ak); GAT(4);                                              \+        WROUND6(6, ak); GAT(5);                                              \+        WROUND6(7, ak);                                                      \+        WROUND6(8, ak);                                                      \+        WROUND6(9, ak);                                                      \+        {                                                                    \+            __m128i k = RK(rk, 10);                                          \+            t0 = _mm_aesenclast_si128(t0, k);                                \+            t1 = _mm_aesenclast_si128(t1, k);                                \+            t2 = _mm_aesenclast_si128(t2, k);                                \+            t3 = _mm_aesenclast_si128(t3, k);                                \+            t4 = _mm_aesenclast_si128(t4, k);                                \+            t5 = _mm_aesenclast_si128(t5, RK(ak, 10));                       \+        }                                                                    \+    } while (0)++/*+ * v2: six blocks of AES in flight at once, so the ten rounds of one block no+ * longer wait on each other -- AES-NI is pipelined and will take one+ * instruction a clock as long as the instructions in flight are independent.+ * The GHASH multiplies of the group just finished are issued between the+ * rounds of the group now running, which is the stitching: they do not want+ * the same execution port, so held against each other they cost about what+ * the rounds alone cost.+ */++/*+ * The counter block, built without leaving the vector registers.+ *+ * GCM counts in the low 32 bits of the block, big endian, and wraps there.+ * ctr holds the block with its bytes reversed, so those four bytes are the+ * low lane and _mm_add_epi32 steps them without carrying into the nonce+ * above -- which is the wrap GCM asks for.  A shuffle puts the bytes back.+ *+ * The obvious way -- increment a uint32_t, byte swap it, pinsrd it in --+ * costs a move from a general register to a vector one for every lane, six+ * to a group, and those do not come free.+ */+#define CTR6(j)                                                              \+    do {                                                                     \+        ctr = _mm_add_epi32(ctr, one32);                                     \+        b##j = _mm_xor_si128(_mm_shuffle_epi8(ctr, BSWAP), RK(rk, 0));       \+    } while (0)++#define ROUND6(r)                                                            \+    do {                                                                     \+        __m128i k = RK(rk, r);                                               \+        b0 = _mm_aesenc_si128(b0, k);                                        \+        b1 = _mm_aesenc_si128(b1, k);                                        \+        b2 = _mm_aesenc_si128(b2, k);                                        \+        b3 = _mm_aesenc_si128(b3, k);                                        \+        b4 = _mm_aesenc_si128(b4, k);                                        \+        b5 = _mm_aesenc_si128(b5, k);                                        \+    } while (0)++#define LAST6(r)                                                             \+    do {                                                                     \+        __m128i k = RK(rk, r);                                               \+        b0 = _mm_aesenclast_si128(b0, k);                                    \+        b1 = _mm_aesenclast_si128(b1, k);                                    \+        b2 = _mm_aesenclast_si128(b2, k);                                    \+        b3 = _mm_aesenclast_si128(b3, k);                                    \+        b4 = _mm_aesenclast_si128(b4, k);                                    \+        b5 = _mm_aesenclast_si128(b5, k);                                    \+    } while (0)++/* one GHASH multiply, taken from a queue of blocks waiting to be absorbed,+ * to be issued in the gaps between AES rounds */+/* A ring, so that a block queued while others are still waiting costs an+ * index and not a move: the queue is walked from both ends and never+ * compacted. */+#define GQ_MASK 15++#define GPUSH(v)                                                             \+    do { gq[gw] = (v); gw++; gn++; } while (0)++#define GSTEP()                                                              \+    do {                                                                     \+        if (gn > 0) {                                                        \+            GHASH_ONE(gq[gi], gp);                                   \+            gi++; gp--; gn--;                                                \+        }                                                                    \+    } while (0)++/* the same at a slot the compiler can see, for the unrolled group below */+/*+ * One queued block, at a slot the compiler can see and with nothing to test+ * before it.  A test here would end the basic block, and the scheduler works+ * inside one: six tests turn the group into twelve blocks and the multiplies+ * can no longer be moved up among the rounds, which is the whole point of+ * writing them there.  The group below is entered only when the queue is+ * full, so there is nothing to test.+ */+/*+ * The block a slot names, read from the output where the group before it+ * left the ciphertext rather than from a copy kept beside it.  The copy+ * cost six stores a group for bytes already in memory; picotls's fusion+ * points its GHASH at the output it has just written for the same reason.+ */+#define GAT(j) GHASH_ONE(_mm_shuffle_epi8(                                   \+        _mm_loadu_si128((const __m128i *) (prev + 16 * (j))), BSWAP), gp - (j))++TGT void crypton_gcm_fused_encrypt(uint8_t *out, const aes_gcm_fused *fk,+                                   const aes_key *key, const uint8_t *nonce,+                                   const uint8_t *aad, size_t aadlen,+                                   const uint8_t *in, size_t inlen, size_t taglen,+                                   const aes_key *hpkey, size_t sampleoff,+                                   uint8_t *mask)+{+    const uint8_t *rk = key->data;+    const uint8_t *hprk = hpkey != 0 ? hpkey->data : rk;+    const int rounds = key->nbr;+    const int hprounds = hpkey != 0 ? hpkey->nbr : rounds;+    GHASH_DECL;+    __m128i ctrbase, ctr, one32, ek0, tag, b0, b1, b2, b3, b4, b5;+    const int ntail_pre = (int) ((inlen % 96 + 15) / 16);+    int lane_ek0;+    __m128i gq[6];+    unsigned gi = 0, gw = 0;+    int gn = 0;+    size_t nblk = (aadlen + 15) / 16 + (inlen + 15) / 16 + 1;+    const int gtotal = (int) nblk;+    int gp = (int) nblk;+    size_t i;+    size_t done;+    int lane_mask = 0;++    /*+     * Y0: the twelve bytes of nonce and a counter of one.  loadn reads the+     * nonce where it lies and masks what is above it, so this is one load+     * rather than three of four bytes each and a set built from them.+     */+    ctrbase = _mm_insert_epi32(loadn(nonce, 12), (int) __builtin_bswap32(1), 3);+    ctr = _mm_shuffle_epi8(ctrbase, BSWAP);+    one32 = _mm_set_epi32(0, 0, 0, 1);++    lane_ek0 = ntail_pre > 0 && ntail_pre <= 4;+    if (!lane_ek0)+        ek0 = aes_one_block(rk, rounds, ctrbase);++    /* The additional data goes in first and takes the highest powers, but it+     * is only queued here: absorbing it takes multiplies, and the multiplies+     * belong in the gaps between the AES rounds below rather than in front+     * of them where nothing else is running. */+    /*+     * The additional data goes in first and takes the highest powers.  It is+     * absorbed here rather than queued: what the queue is for is giving the+     * rounds below something to interleave with, and a queue that sometimes+     * holds the header and sometimes does not forces a test before every+     * multiply -- which is what stopped the interleaving from happening at+     * all.  See the peeled first group below.+     */+    {+        size_t nfull = aadlen / 16;+        size_t rest = aadlen % 16;++        for (i = 0; i < nfull; i++)+            GHASH_ONE(_mm_shuffle_epi8(+                _mm_loadu_si128((const __m128i *) (aad + i * 16)), BSWAP), 0);+        if (rest)+            GHASH_ONE(_mm_shuffle_epi8(loadn(aad + nfull * 16, rest), BSWAP), 0);+    }++    /* Whole groups of six.  The rounds are written out rather than looped:+     * the number of them is a value in the key, so a loop over it leaves the+     * compiler fetching each round key through an index it cannot fold, and+     * the six lanes go to memory with them.  Written out, the whole group+     * stays in registers, and the six multiplies of the group before can be+     * placed between the rounds by hand -- which is the stitching: AES-NI+     * and PCLMULQDQ do not contend for the same port, so the multiplies are+     * very nearly free.+     */+    done = 0;+    if (rounds == 10) {+        if (done + 96 <= inlen) {+            const uint8_t *p = in + done;+            uint8_t *q = out + done;++            CTR6(0); CTR6(1); CTR6(2); CTR6(3); CTR6(4); CTR6(5);+            ROUND6(1);+            ROUND6(2);+            ROUND6(3);+            ROUND6(4);+            ROUND6(5);+            ROUND6(6);+            ROUND6(7);+            ROUND6(8);+            ROUND6(9);+            LAST6(10);+            gn = 0; gi = 0; gw = 0;++            b0 = _mm_xor_si128(b0, _mm_loadu_si128((const __m128i *) p));+            b1 = _mm_xor_si128(b1, _mm_loadu_si128((const __m128i *) (p + 16)));+            b2 = _mm_xor_si128(b2, _mm_loadu_si128((const __m128i *) (p + 32)));+            b3 = _mm_xor_si128(b3, _mm_loadu_si128((const __m128i *) (p + 48)));+            b4 = _mm_xor_si128(b4, _mm_loadu_si128((const __m128i *) (p + 64)));+            b5 = _mm_xor_si128(b5, _mm_loadu_si128((const __m128i *) (p + 80)));+            _mm_storeu_si128((__m128i *) q, b0);+            _mm_storeu_si128((__m128i *) (q + 16), b1);+            _mm_storeu_si128((__m128i *) (q + 32), b2);+            _mm_storeu_si128((__m128i *) (q + 48), b3);+            _mm_storeu_si128((__m128i *) (q + 64), b4);+            _mm_storeu_si128((__m128i *) (q + 80), b5);++            done += 96;+        }+        for (; done + 96 <= inlen; done += 96) {+            const uint8_t *p = in + done;+            uint8_t *q = out + done;+            const uint8_t *prev = out + done - 96;++            CTR6(0); CTR6(1); CTR6(2); CTR6(3); CTR6(4); CTR6(5);+            ROUND6(1); GAT(0);+            ROUND6(2); GAT(1);+            ROUND6(3); GAT(2);+            ROUND6(4); GAT(3);+            ROUND6(5); GAT(4);+            ROUND6(6); GAT(5);+            ROUND6(7);+            ROUND6(8);+            ROUND6(9);+            LAST6(10);+            gp -= 6;++            b0 = _mm_xor_si128(b0, _mm_loadu_si128((const __m128i *) p));+            b1 = _mm_xor_si128(b1, _mm_loadu_si128((const __m128i *) (p + 16)));+            b2 = _mm_xor_si128(b2, _mm_loadu_si128((const __m128i *) (p + 32)));+            b3 = _mm_xor_si128(b3, _mm_loadu_si128((const __m128i *) (p + 48)));+            b4 = _mm_xor_si128(b4, _mm_loadu_si128((const __m128i *) (p + 64)));+            b5 = _mm_xor_si128(b5, _mm_loadu_si128((const __m128i *) (p + 80)));+            _mm_storeu_si128((__m128i *) q, b0);+            _mm_storeu_si128((__m128i *) (q + 16), b1);+            _mm_storeu_si128((__m128i *) (q + 32), b2);+            _mm_storeu_si128((__m128i *) (q + 48), b3);+            _mm_storeu_si128((__m128i *) (q + 64), b4);+            _mm_storeu_si128((__m128i *) (q + 80), b5);++            /* straight from the registers the last round left them in: the+             * queue existed only to hold them until the next group's rounds+             * could hide the multiplies, and that is 192 bytes of store and+             * load per 96 bytes of payload */+        }++    } else {+        for (done = 0; done + 96 <= inlen; done += 96) {+                const uint8_t *p = in + done;+                uint8_t *q = out + done;+                int r;++                CTR6(0); CTR6(1); CTR6(2); CTR6(3); CTR6(4); CTR6(5);+                for (r = 1; r < rounds; r++) {+                    ROUND6(r);+                    GSTEP();+                }+                LAST6(rounds);++                b0 = _mm_xor_si128(b0, _mm_loadu_si128((const __m128i *) p));+                b1 = _mm_xor_si128(b1, _mm_loadu_si128((const __m128i *) (p + 16)));+                b2 = _mm_xor_si128(b2, _mm_loadu_si128((const __m128i *) (p + 32)));+                b3 = _mm_xor_si128(b3, _mm_loadu_si128((const __m128i *) (p + 48)));+                b4 = _mm_xor_si128(b4, _mm_loadu_si128((const __m128i *) (p + 64)));+                b5 = _mm_xor_si128(b5, _mm_loadu_si128((const __m128i *) (p + 80)));+                _mm_storeu_si128((__m128i *) q, b0);+                _mm_storeu_si128((__m128i *) (q + 16), b1);+                _mm_storeu_si128((__m128i *) (q + 32), b2);+                _mm_storeu_si128((__m128i *) (q + 48), b3);+                _mm_storeu_si128((__m128i *) (q + 64), b4);+                _mm_storeu_si128((__m128i *) (q + 80), b5);++                while (gn > 0) GSTEP();+                gi = 0; gw = 0;+                gq[0] = _mm_shuffle_epi8(b0, BSWAP);+                gq[1] = _mm_shuffle_epi8(b1, BSWAP);+                gq[2] = _mm_shuffle_epi8(b2, BSWAP);+                gq[3] = _mm_shuffle_epi8(b3, BSWAP);+                gq[4] = _mm_shuffle_epi8(b4, BSWAP);+                gq[5] = _mm_shuffle_epi8(b5, BSWAP);+                gn = 6;+        }+    }++    /* The tail.  The wide pass below runs only when there are blocks for it:+     * sixty AES instructions to fill one lane is not worth it, so a message+     * that ends on a group boundary leaves E(K,Y0) the chain it was given+     * above and takes the mask on one of its own.+     *+     * The spare lane carries the mask only when two things hold.  The sample+     * has to lie entirely in output the groups above have already written:+     * this pass reads it while it runs, and the blocks it is itself+     * computing are stored after it, so a sample reaching into them would be+     * read before it exists.  And the two key schedules have to have the+     * same number of rounds, because the lanes share the loop that counts+     * them and a shorter schedule would be read past its end.  TLS and QUIC+     * satisfy both; anything else gets the mask on a chain of its own, which+     * is what it would have had anyway. */+    {+        __m128i t0, t1, t2, t3, t4, t5;+        __m128i tv[6];+        size_t toff[6];+        int ntail = 0, j;++        if (done >= inlen) goto no_tail;++        for (i = done; i < inlen; i += 16) {+            toff[ntail] = i;+            ctr = _mm_add_epi32(ctr, one32);+            tv[ntail] = _mm_shuffle_epi8(ctr, BSWAP);+            ntail++;+        }++        lane_mask = ntail > 0 && ntail <= 5 && hpkey != 0 && mask != 0+                 && hprounds == rounds+                 && sampleoff + 16 <= done;++        if (ntail > 0) {+            t0 = tv[0];+            t1 = ntail > 1 ? tv[1] : ctrbase;+            t2 = ntail > 2 ? tv[2] : ctrbase;+            t3 = ntail > 3 ? tv[3] : ctrbase;+            t4 = lane_ek0 ? ctrbase : (ntail > 4 ? tv[4] : ctrbase);+            t5 = lane_mask+               ? _mm_loadu_si128((const __m128i *) (out + sampleoff))+               : (ntail > 5 ? tv[5] : ctrbase);+            /* A group leaves exactly six queued, which is what lets the+             * slots below be named at compile time.  Where no group ran+             * there is nothing to place and the plain pass will do. */+            if (rounds == 10 && done >= 96) {+                const uint8_t *prev = out + done - 96;+                WIDE6_10(lane_mask ? hprk : rk);+            } else {+                WIDE6(lane_mask ? hprk : rk);+            }+            if (lane_ek0)+                ek0 = t4;+            else+                tv[4] = t4;+            if (lane_mask)+                _mm_storeu_si128((__m128i *) mask, t5);+            else if (ntail > 5)+                tv[5] = t5;+        }+no_tail:+        while (gn > 0) GSTEP();++        /* The last group's ciphertext is absorbed by the pass above where+         * there is one to absorb it.  A message that ends on a group+         * boundary has no such pass, so it is taken here. */+        if (rounds == 10 && done >= 96 && ntail == 0) {+            const uint8_t *prev = out + done - 96;+            GAT(0); GAT(1); GAT(2); GAT(3); GAT(4); GAT(5);+        }++        /*+         * The tail blocks, from the registers the pass left them in.  They+         * were going through an array indexed by the loop variable, which+         * the compiler cannot see through and so keeps in memory: every+         * block then reloaded its own keystream.  Named one per block and+         * reached by a test on a count instead, they stay where they are.+         */+#define TAILBLK(j, reg)                                                      \+        do {                                                                 \+            size_t off = done + 16 * (j);                                    \+            size_t n = inlen - off < 16 ? inlen - off : 16;                  \+            __m128i c = _mm_xor_si128(reg, n == 16                           \+                ? _mm_loadu_si128((const __m128i *) (in + off))              \+                : loadn(in + off, n));                                       \+            if (n == 16) {                                                   \+                _mm_storeu_si128((__m128i *) (out + off), c);                \+            } else {                                                         \+                /* the tag goes in directly above, so those bytes are        \+                 * written over anyway where there are sixteen to spare */   \+                if (n + taglen >= 16)                                        \+                    _mm_storeu_si128((__m128i *) (out + off), c);            \+                else                                                         \+                    storen(out + off, c, n);                                 \+                c = clampn(c, n);                                            \+            }                                                                \+            GHASH_ONE(_mm_shuffle_epi8(c, BSWAP), gp);                       \+            gp--;                                                            \+        } while (0)++        if (ntail > 0) TAILBLK(0, t0);+        if (ntail > 1) TAILBLK(1, t1);+        if (ntail > 2) TAILBLK(2, t2);+        if (ntail > 3) TAILBLK(3, t3);+        if (ntail > 4) TAILBLK(4, tv[4]);+        if (ntail > 5) TAILBLK(5, tv[5]);+#undef TAILBLK+    }++    {+        /*+         * The length block: the additional data's bit count and the+         * message's, each big endian in a half, and then reversed like+         * every other block on its way to GHASH.+         *+         * Reversed, that block is the two counts as ordinary little endian+         * words with the message's in the low half -- which is one set, and+         * no shuffle.  Sixteen byte stores to the stack and a load back is+         * what it cost before.+         */+        GHASH_ONE(_mm_set_epi64x((long long) ((uint64_t) aadlen << 3),+                                 (long long) ((uint64_t) inlen << 3)), gp);+    }++    tag = _mm_shuffle_epi8(gtag, BSWAP);+    tag = _mm_xor_si128(tag, ek0);+    if (taglen == 16)+        _mm_storeu_si128((__m128i *) (out + inlen), tag);+    else+        storen(out + inlen, tag, taglen);++    /* A sample the pass above could not reach -- because it covered blocks+     * that pass was still computing, or the tag, which is written just now+     * -- is taken here instead, where everything it can cover exists. */+    if (!lane_mask && hpkey != 0 && mask != 0)+        _mm_storeu_si128((__m128i *) mask,+                         aes_one_block(hprk, hprounds, _mm_loadu_si128(+                             (const __m128i *) (out + sampleoff))));+}+++/*+ * The same for decryption, which is the simpler of the two.+ *+ * What GHASH absorbs here is the ciphertext, and the ciphertext is the+ * input: it is there before any of the AES has run.  So there is no queue --+ * the multiplies of a group go between the rounds of that same group rather+ * than waiting for the one after, and nothing is stored and loaded back to+ * carry them across.+ *+ * The tag is compared here, every byte of it whichever way the answer goes,+ * and the answer is 1 for a message whose tag matched.+ */++/* one ciphertext block straight from the input, at a slot named here */+#define DAT(j) GHASH_ONE(_mm_shuffle_epi8(                                   \+        _mm_loadu_si128((const __m128i *) (p + 16 * (j))), BSWAP), 0)++/* the next counter block, into a named register */+#define CTRT(t)                                                              \+    do { ctr = _mm_add_epi32(ctr, one32);                                    \+         t = _mm_shuffle_epi8(ctr, BSWAP); } while (0)++TGT int crypton_gcm_fused_decrypt(uint8_t *out, const aes_gcm_fused *fk,+                                  const aes_key *key, const uint8_t *nonce,+                                  const uint8_t *aad, size_t aadlen,+                                  const uint8_t *in, size_t inlen,+                                  const uint8_t *tag, size_t taglen,+                                  uint8_t *outtag)+{+    const uint8_t *rk = key->data;+    const int rounds = key->nbr;+    GHASH_DECL;+    __m128i ctrbase, ctr, one32, ek0, want, b0, b1, b2, b3, b4, b5;+    const int ntail_pre = (int) ((inlen % 96 + 15) / 16);+    int lane_ek0, gp = 0;+    const int gtotal = (int) ((aadlen + 15) / 16 + (inlen + 15) / 16 + 1);+    size_t i;+    size_t done;+    uint8_t diff = 0;++    ctrbase = _mm_insert_epi32(loadn(nonce, 12), (int) __builtin_bswap32(1), 3);+    ctr = _mm_shuffle_epi8(ctrbase, BSWAP);+    one32 = _mm_set_epi32(0, 0, 0, 1);++    lane_ek0 = ntail_pre > 0 && ntail_pre <= 5;+    if (!lane_ek0)+        ek0 = aes_one_block(rk, rounds, ctrbase);++    {+        size_t nfull = aadlen / 16;+        size_t rest = aadlen % 16;++        for (i = 0; i < nfull; i++)+            GHASH_ONE(_mm_shuffle_epi8(+                _mm_loadu_si128((const __m128i *) (aad + i * 16)), BSWAP), 0);+        if (rest)+            GHASH_ONE(_mm_shuffle_epi8(loadn(aad + nfull * 16, rest), BSWAP), 0);+    }++    done = 0;+    if (rounds == 10) {+        for (; done + 96 <= inlen; done += 96) {+            const uint8_t *p = in + done;+            uint8_t *q = out + done;++            CTR6(0); CTR6(1); CTR6(2); CTR6(3); CTR6(4); CTR6(5);+            ROUND6(1); DAT(0);+            ROUND6(2); DAT(1);+            ROUND6(3); DAT(2);+            ROUND6(4); DAT(3);+            ROUND6(5); DAT(4);+            ROUND6(6); DAT(5);+            ROUND6(7);+            ROUND6(8);+            ROUND6(9);+            LAST6(10);++            _mm_storeu_si128((__m128i *) q,+                _mm_xor_si128(b0, _mm_loadu_si128((const __m128i *) p)));+            _mm_storeu_si128((__m128i *) (q + 16),+                _mm_xor_si128(b1, _mm_loadu_si128((const __m128i *) (p + 16))));+            _mm_storeu_si128((__m128i *) (q + 32),+                _mm_xor_si128(b2, _mm_loadu_si128((const __m128i *) (p + 32))));+            _mm_storeu_si128((__m128i *) (q + 48),+                _mm_xor_si128(b3, _mm_loadu_si128((const __m128i *) (p + 48))));+            _mm_storeu_si128((__m128i *) (q + 64),+                _mm_xor_si128(b4, _mm_loadu_si128((const __m128i *) (p + 64))));+            _mm_storeu_si128((__m128i *) (q + 80),+                _mm_xor_si128(b5, _mm_loadu_si128((const __m128i *) (p + 80))));+        }+    } else {+        for (; done + 96 <= inlen; done += 96) {+            const uint8_t *p = in + done;+            uint8_t *q = out + done;+            int r;++            CTR6(0); CTR6(1); CTR6(2); CTR6(3); CTR6(4); CTR6(5);+            for (r = 1; r < rounds; r++) {+                ROUND6(r);+                if (r <= 6) DAT(r - 1);+            }+            LAST6(rounds);++            _mm_storeu_si128((__m128i *) q,+                _mm_xor_si128(b0, _mm_loadu_si128((const __m128i *) p)));+            _mm_storeu_si128((__m128i *) (q + 16),+                _mm_xor_si128(b1, _mm_loadu_si128((const __m128i *) (p + 16))));+            _mm_storeu_si128((__m128i *) (q + 32),+                _mm_xor_si128(b2, _mm_loadu_si128((const __m128i *) (p + 32))));+            _mm_storeu_si128((__m128i *) (q + 48),+                _mm_xor_si128(b3, _mm_loadu_si128((const __m128i *) (p + 48))));+            _mm_storeu_si128((__m128i *) (q + 64),+                _mm_xor_si128(b4, _mm_loadu_si128((const __m128i *) (p + 64))));+            _mm_storeu_si128((__m128i *) (q + 80),+                _mm_xor_si128(b5, _mm_loadu_si128((const __m128i *) (p + 80))));+        }+    }++    /* the tail, with E(K,Y0) in a lane the length leaves idle */+    {+        __m128i t0, t1, t2, t3, t4, t5;+        int ntail = (int) ((inlen - done + 15) / 16), j;++        /* the counter is where the groups left it */+        t0 = t1 = t2 = t3 = t4 = t5 = ctrbase;+        if (ntail > 0) CTRT(t0);+        if (ntail > 1) CTRT(t1);+        if (ntail > 2) CTRT(t2);+        if (ntail > 3) CTRT(t3);+        if (ntail > 4) CTRT(t4);+        if (ntail > 5) CTRT(t5);++        if (ntail > 0) {+            WIDE6_NOQ(rk);+            if (lane_ek0) ek0 = t5;+        }++        for (j = 0; j < ntail; j++) {+            size_t off = done + 16 * (size_t) j;+            size_t n = inlen - off < 16 ? inlen - off : 16;+            __m128i c = n == 16 ? _mm_loadu_si128((const __m128i *) (in + off))+                                : loadn(in + off, n);+            __m128i ks = j == 0 ? t0 : j == 1 ? t1 : j == 2 ? t2+                       : j == 3 ? t3 : j == 4 ? t4 : t5;++            GHASH_ONE(_mm_shuffle_epi8(c, BSWAP), 0);+            {+                __m128i pl = _mm_xor_si128(ks, c);+                if (n == 16)+                    _mm_storeu_si128((__m128i *) (out + off), pl);+                else+                    storen(out + off, pl, n);+            }+        }+    }++    GHASH_ONE(_mm_set_epi64x((long long) ((uint64_t) aadlen << 3),+                             (long long) ((uint64_t) inlen << 3)), gp);++    want = _mm_xor_si128(_mm_shuffle_epi8(gtag, BSWAP), ek0);+    {+        uint8_t got[16];+        _mm_storeu_si128((__m128i *) got, want);+        if (outtag) {+            /* The caller holds the expected tag and will compare it itself. */+            memcpy(outtag, got, taglen);+            return 1;+        }+        for (i = 0; i < taglen; i++)+            diff |= (uint8_t) (got[i] ^ tag[i]);+    }+    return diff == 0;+}++#endif /* WITH_GCM_FUSED */
+ cbits/aes/gcm_fused_x86.h view
@@ -0,0 +1,55 @@+/*+ * Copyright (c) 2026 Kazu Yamamoto <kazu@iij.ad.jp>+ *+ * All rights reserved.+ *+ * Redistribution and use in source and binary forms, with or without+ * modification, are permitted provided that the following conditions+ * are met:+ * 1. Redistributions of source code must retain the above copyright+ *    notice, this list of conditions and the following disclaimer.+ * 2. Redistributions in binary form must reproduce the above copyright+ *    notice, this list of conditions and the following disclaimer in the+ *    documentation and/or other materials provided with the distribution.+ * 3. Neither the name of the author nor the names of his contributors+ *    may be used to endorse or promote products derived from this software+ *    without specific prior written permission.+ *+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF+ * SUCH DAMAGE.+ */++#ifndef CRYPTON_GCM_FUSED_X86_H+#define CRYPTON_GCM_FUSED_X86_H++#include <stdint.h>+#include <stddef.h>+#include <crypton_aes.h>++void crypton_gcm_fused_key_init(aes_gcm_fused *fk, const aes_key *key);++void crypton_gcm_fused_encrypt(uint8_t *out, const aes_gcm_fused *fk,+                               const aes_key *key,+                               const uint8_t *nonce,+                               const uint8_t *aad, size_t aadlen,+                               const uint8_t *in, size_t inlen, size_t taglen,+                               const aes_key *hpkey, size_t sampleoff,+                               uint8_t *mask);++int crypton_gcm_fused_decrypt(uint8_t *out, const aes_gcm_fused *fk,+                              const aes_key *key, const uint8_t *nonce,+                              const uint8_t *aad, size_t aadlen,+                              const uint8_t *in, size_t inlen,+                              const uint8_t *tag, size_t taglen,+                              uint8_t *outtag);++#endif
+ cbits/aes/gcm_vaes512_x86.c view
@@ -0,0 +1,364 @@+/*+ * AES-GCM through VAES and VPCLMULQDQ in their 512-bit form, which takes+ * four blocks where the 256-bit form in cbits/aes/gcm_vaes_x86.c takes two+ * and the 128-bit one takes one.  The instruction rate is the same, so the+ * work per group halves again.+ *+ * This is that file widened and nothing else: the same group of sixteen+ * blocks, the same descending powers of H sharing one reduction, the same+ * round keys read from memory rather than held in registers.  Four blocks to+ * a register means the group fills four of them rather than eight, which is+ * what leaves room for the group's own ciphertext to be kept for the GHASH+ * when encrypting.+ *+ * Nothing here is borrowed.  OpenSSL's and BoringSSL's AVX-512 AES-GCM is+ * Apache-2.0 and s2n-bignum has no GCM at all.+ *+ * The reduction at the end is a copy of the one in gcm_vaes_x86.c rather+ * than a call to it: the two files are compiled for different instruction+ * sets, and a function compiled for one cannot be inlined into the other.+ */+#include "aes/gcm_vaes512_x86.h"++#ifdef WITH_GCM_VAES512++#include <string.h>+#include <immintrin.h>++#include <aes/gf.h>+#include <aes/block128.h>++#if defined(__clang__) || defined(__GNUC__)+#define V512_TARGET \+	__attribute__((target("avx512f,avx512bw,avx512vl,aes,pclmul,vaes,vpclmulqdq")))+#else+#define V512_TARGET+#endif++/*+ * Thirty-two blocks to a group, four to a register, so eight registers are+ * in flight.  The number of registers is what matters as much as the blocks+ * per instruction: AES-NI has a latency of four cycles against a throughput+ * of one, so it takes eight independent chains to keep two ports busy.  Four+ * registers of four blocks was written first and measured *slower* than the+ * 256-bit path -- the blocks per instruction had doubled and the chains had+ * halved.+ *+ * The table holds sixteen powers of H, so the GHASH of a group is two passes+ * of sixteen blocks, the second picking up the tag the first leaves.+ */+#define V512WIDE 8+#define V512HALF 4+#define V512BYTES 512++/*+ * The 128-bit multiply of cbits/aes/x86ni.c, done in all four lanes at once.+ * Every shuffle and shift here works inside its own 128-bit lane, so the+ * four products never mix: what comes out is four independent carry-less+ * products, accumulated by the caller and reduced together at the end.+ */+V512_TARGET+static inline void clmul512(__m512i a, __m512i b, __m512i *lo, __m512i *hi)+{+	const __m512i bswap = _mm512_set4_epi32(+		0x00010203, 0x04050607, 0x08090a0b, 0x0c0d0e0f);+	__m512i t3, t4, t5, t6;++	a = _mm512_shuffle_epi8(a, bswap);++	/* Karatsuba, as in the 128-bit one: three multiplies, not four */+	t3 = _mm512_clmulepi64_epi128(a, b, 0x00);+	t6 = _mm512_clmulepi64_epi128(a, b, 0x11);+	t4 = _mm512_clmulepi64_epi128(+		_mm512_xor_si512(a, _mm512_shuffle_epi32(a, _MM_PERM_BADC)),+		_mm512_xor_si512(b, _mm512_shuffle_epi32(b, _MM_PERM_BADC)),+		0x00);+	t4 = _mm512_xor_si512(t4, _mm512_xor_si512(t3, t6));++	t5 = _mm512_bslli_epi128(t4, 8);+	t4 = _mm512_bsrli_epi128(t4, 8);++	*lo = _mm512_xor_si512(t3, t5);+	*hi = _mm512_xor_si512(t6, t4);+}++/*+ * The reduction of cbits/aes/x86ni.c.  By the time it runs the four lanes+ * have been folded into one, so there is one 256-bit product to reduce.+ */+V512_TARGET+static inline __m128i gfred512(__m128i t3, __m128i t6)+{+	const __m128i bswap = _mm_set_epi8(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15);+	__m128i t2, t4, t5, t7, t8, t9;++	t7 = _mm_srli_epi32(t3, 31);+	t8 = _mm_srli_epi32(t6, 31);+	t3 = _mm_slli_epi32(t3, 1);+	t6 = _mm_slli_epi32(t6, 1);++	t9 = _mm_srli_si128(t7, 12);+	t8 = _mm_slli_si128(t8, 4);+	t7 = _mm_slli_si128(t7, 4);+	t3 = _mm_or_si128(t3, t7);+	t6 = _mm_or_si128(t6, t8);+	t6 = _mm_or_si128(t6, t9);++	t7 = _mm_slli_epi32(t3, 31);+	t8 = _mm_slli_epi32(t3, 30);+	t9 = _mm_slli_epi32(t3, 25);++	t7 = _mm_xor_si128(t7, t8);+	t7 = _mm_xor_si128(t7, t9);+	t8 = _mm_srli_si128(t7, 4);+	t7 = _mm_slli_si128(t7, 12);+	t3 = _mm_xor_si128(t3, t7);++	t2 = _mm_srli_epi32(t3, 1);+	t4 = _mm_srli_epi32(t3, 2);+	t5 = _mm_srli_epi32(t3, 7);+	t2 = _mm_xor_si128(t2, t4);+	t2 = _mm_xor_si128(t2, t5);+	t2 = _mm_xor_si128(t2, t8);+	t3 = _mm_xor_si128(t3, t2);+	t6 = _mm_xor_si128(t6, t3);++	return _mm_shuffle_epi8(t6, bswap);+}++/* the four 128-bit lanes of a register added together */+V512_TARGET+static inline __m128i fold512(__m512i v)+{+	__m256i h = _mm256_xor_si256(_mm512_castsi512_si256(v),+	                             _mm512_extracti64x4_epi64(v, 1));++	return _mm_xor_si128(_mm256_castsi256_si128(h),+	                     _mm256_extracti128_si256(h, 1));+}++/*+ * Sixteen blocks against H^16 .. H^1, one reduction.  v[j] holds blocks 4j+ * to 4j+3 in its four lanes, so the powers for it are H^(16-4j) down to+ * H^(13-4j) -- the table's own order the other way round, hence the four+ * 128-bit loads rather than one 512-bit one.+ */+V512_TARGET+static inline __m128i ghash16(__m128i tag, const table_4bit htable,+                              const __m512i *v, int fromwire)+{+	__m512i lo = _mm512_setzero_si512(), hi = _mm512_setzero_si512();+	__m512i l, h, b;+	int j;++	for (j = 0; j < V512HALF; j++) {+		const __m128i p0 =+			_mm_loadu_si128((const __m128i *) &htable[15 - 4 * j]);+		const __m128i p1 =+			_mm_loadu_si128((const __m128i *) &htable[14 - 4 * j]);+		const __m128i p2 =+			_mm_loadu_si128((const __m128i *) &htable[13 - 4 * j]);+		const __m128i p3 =+			_mm_loadu_si128((const __m128i *) &htable[12 - 4 * j]);+		__m512i hp = _mm512_castsi128_si512(p0);++		hp = _mm512_inserti32x4(hp, p1, 1);+		hp = _mm512_inserti32x4(hp, p2, 2);+		hp = _mm512_inserti32x4(hp, p3, 3);++		b = fromwire ? _mm512_loadu_si512(v + j) : v[j];+		if (j == 0) /* the running tag joins the first block */+			b = _mm512_xor_si512(+				b, _mm512_inserti32x4(+					_mm512_setzero_si512(), tag, 0));+		clmul512(b, hp, &l, &h);+		lo = _mm512_xor_si512(lo, l);+		hi = _mm512_xor_si512(hi, h);+	}++	/* the four lanes are independent products of the same sum: fold them */+	return gfred512(fold512(lo), fold512(hi));+}++#define KK512(r) _mm512_broadcast_i32x4(_mm_loadu_si128(k_ + (r)))++#define AESENC32(K)                                                         \+	do {                                                                \+		const __m512i rk = (K);                                     \+		v[0] = _mm512_aesenc_epi128(v[0], rk);                      \+		v[1] = _mm512_aesenc_epi128(v[1], rk);                      \+		v[2] = _mm512_aesenc_epi128(v[2], rk);                      \+		v[3] = _mm512_aesenc_epi128(v[3], rk);                      \+		v[4] = _mm512_aesenc_epi128(v[4], rk);                      \+		v[5] = _mm512_aesenc_epi128(v[5], rk);                      \+		v[6] = _mm512_aesenc_epi128(v[6], rk);                      \+		v[7] = _mm512_aesenc_epi128(v[7], rk);                      \+	} while (0)++#define AESLAST32(K)                                                        \+	do {                                                                \+		const __m512i rk = (K);                                     \+		v[0] = _mm512_aesenclast_epi128(v[0], rk);                  \+		v[1] = _mm512_aesenclast_epi128(v[1], rk);                  \+		v[2] = _mm512_aesenclast_epi128(v[2], rk);                  \+		v[3] = _mm512_aesenclast_epi128(v[3], rk);                  \+		v[4] = _mm512_aesenclast_epi128(v[4], rk);                  \+		v[5] = _mm512_aesenclast_epi128(v[5], rk);                  \+		v[6] = _mm512_aesenclast_epi128(v[6], rk);                  \+		v[7] = _mm512_aesenclast_epi128(v[7], rk);                  \+	} while (0)++#define XOR32(K)                                                            \+	do {                                                                \+		const __m512i rk = (K);                                     \+		v[0] = _mm512_xor_si512(v[0], rk);                          \+		v[1] = _mm512_xor_si512(v[1], rk);                          \+		v[2] = _mm512_xor_si512(v[2], rk);                          \+		v[3] = _mm512_xor_si512(v[3], rk);                          \+		v[4] = _mm512_xor_si512(v[4], rk);                          \+		v[5] = _mm512_xor_si512(v[5], rk);                          \+		v[6] = _mm512_xor_si512(v[6], rk);                          \+		v[7] = _mm512_xor_si512(v[7], rk);                          \+	} while (0)++/*+ * The rounds are written out rather than looped for the reason the 128-bit+ * loop gives: the count is a value in the key, and a loop over it leaves the+ * round key reached through an index the compiler cannot fold.+ */+V512_TARGET+static inline __attribute__((always_inline)) void+rounds32(__m512i *v, const uint8_t *k, const int nbr)+{+	const __m128i *k_ = (const __m128i *) k;++	XOR32(KK512(0));+	AESENC32(KK512(1)); AESENC32(KK512(2)); AESENC32(KK512(3));+	AESENC32(KK512(4)); AESENC32(KK512(5)); AESENC32(KK512(6));+	AESENC32(KK512(7)); AESENC32(KK512(8)); AESENC32(KK512(9));+	if (nbr > 10) {+		AESENC32(KK512(10)); AESENC32(KK512(11));+		if (nbr > 12) {+			AESENC32(KK512(12)); AESENC32(KK512(13));+		}+	}+	AESLAST32(_mm512_broadcast_i32x4(_mm_loadu_si128(k_ + nbr)));+}++/* sixteen consecutive counters, four to a register.  GCM counts in the low+ * thirty-two bits and wraps there, which is what _mm_add_epi32 does. */+V512_TARGET+static inline __m128i counters32(__m512i *v, __m128i iv, __m128i one,+                                 __m128i bswap)+{+	int j;++	for (j = 0; j < V512WIDE; j++) {+		__m128i c0, c1, c2, c3;+		__m512i c;++		iv = _mm_add_epi32(iv, one);+		c0 = _mm_shuffle_epi8(iv, bswap);+		iv = _mm_add_epi32(iv, one);+		c1 = _mm_shuffle_epi8(iv, bswap);+		iv = _mm_add_epi32(iv, one);+		c2 = _mm_shuffle_epi8(iv, bswap);+		iv = _mm_add_epi32(iv, one);+		c3 = _mm_shuffle_epi8(iv, bswap);++		c = _mm512_castsi128_si512(c0);+		c = _mm512_inserti32x4(c, c1, 1);+		c = _mm512_inserti32x4(c, c2, 2);+		c = _mm512_inserti32x4(c, c3, 3);+		v[j] = c;+	}+	return iv;+}++/*+ * Inlined into three callers with the round count a constant in each, which+ * folds away the tests inside the group loop -- the same reason the 256-bit+ * file gives, where without it AES-256 lost what AES-128 gained.+ */+V512_TARGET+static inline __attribute__((always_inline)) uint32_t+bulk_n(uint8_t *output, aes_gcm *gcm, const aes_key *key,+       const uint8_t *input, uint32_t length, int decrypt, const int nbr)+{+	const __m128i bswap = _mm_setr_epi8(7,6,5,4,3,2,1,0,15,14,13,12,11,10,9,8);+	const __m128i one = _mm_set_epi32(0, 1, 0, 0);+	__m512i v[V512WIDE];+	__m128i iv, tag;+	uint32_t groups = length / V512BYTES;+	uint32_t done = 0;+	uint32_t g;+	int j;++	if (groups == 0)+		return 0;++	iv = _mm_shuffle_epi8(_mm_loadu_si128((const __m128i *) &gcm->civ), bswap);+	tag = _mm_loadu_si128((const __m128i *) &gcm->tag);++	for (g = 0; g < groups; g++, input += V512BYTES, output += V512BYTES,+	     done += V512BYTES) {+		iv = counters32(v, iv, one, bswap);+		rounds32(v, key->data, nbr);++		for (j = 0; j < V512WIDE; j++) {+			const __m512i in =+				_mm512_loadu_si512((const __m512i *) (input + 64 * j));++			v[j] = _mm512_xor_si512(v[j], in);+			_mm512_storeu_si512((__m512i *) (output + 64 * j), v[j]);+		}+		/* sixteen blocks to a pass, since that is how many powers of+		 * H the table holds; the second picks up the tag the first+		 * leaves */+		tag = ghash16(tag, gcm->htable,+		              decrypt ? (const __m512i *) input : v,+		              decrypt);+		tag = ghash16(tag, gcm->htable,+		              decrypt ? (const __m512i *) (input + 256)+		                      : v + V512HALF,+		              decrypt);+	}++	_mm_storeu_si128((__m128i *) &gcm->civ, _mm_shuffle_epi8(iv, bswap));+	_mm_storeu_si128((__m128i *) &gcm->tag, tag);+	return done;+}++V512_TARGET+static uint32_t bulk(uint8_t *output, aes_gcm *gcm, const aes_key *key,+                     const uint8_t *input, uint32_t length, int decrypt)+{+	switch (key->nbr) {+	case 10:+		return bulk_n(output, gcm, key, input, length, decrypt, 10);+	case 12:+		return bulk_n(output, gcm, key, input, length, decrypt, 12);+	case 14:+		return bulk_n(output, gcm, key, input, length, decrypt, 14);+	default:+		return 0; /* not a key length AES has */+	}+}++uint32_t crypton_gcm_vaes512_bulk_encrypt(uint8_t *output, aes_gcm *gcm,+                                          const aes_key *key,+                                          const uint8_t *input,+                                          uint32_t length)+{+	return bulk(output, gcm, key, input, length, 0);+}++uint32_t crypton_gcm_vaes512_bulk_decrypt(uint8_t *output, aes_gcm *gcm,+                                          const aes_key *key,+                                          const uint8_t *input,+                                          uint32_t length)+{+	return bulk(output, gcm, key, input, length, 1);+}++#endif
+ cbits/aes/gcm_vaes512_x86.h view
@@ -0,0 +1,37 @@+/*+ * AES-GCM in the 512-bit form of the AES and carry-less multiply+ * instructions, which do four blocks where the 128-bit ones do one and the+ * 256-bit ones in cbits/aes/gcm_vaes_x86.c do two.+ */+#ifndef CRYPTON_GCM_VAES512_X86_H+#define CRYPTON_GCM_VAES512_X86_H++#include <crypton_cpu.h>++#if defined(ARCH_X86) && defined(__x86_64__) && defined(WITH_AESNI) \+    && defined(WITH_PCLMUL)+#define WITH_GCM_VAES512+#endif++#ifdef WITH_GCM_VAES512++#include <stdint.h>+#include <crypton_aes.h>++/* The same contract as the 256-bit pair: whole groups off the front, the+ * counter left in gcm->civ and the running tag in gcm->tag, and the number+ * of bytes taken returned, a multiple of 512 and possibly zero. */+uint32_t crypton_gcm_vaes512_bulk_encrypt(uint8_t *output, aes_gcm *gcm,+                                          const aes_key *key,+                                          const uint8_t *input,+                                          uint32_t length);+uint32_t crypton_gcm_vaes512_bulk_decrypt(uint8_t *output, aes_gcm *gcm,+                                          const aes_key *key,+                                          const uint8_t *input,+                                          uint32_t length);++/* Thirty-two blocks is the least it will start on. */+#define GCM_VAES512_MIN_BLOCKS 32++#endif+#endif
+ cbits/aes/gcm_vaes_x86.c view
@@ -0,0 +1,325 @@+/*+ * AES-GCM through VAES and VPCLMULQDQ: the same AES and carry-less multiply+ * instructions the rest of this directory uses, in their 256-bit form, which+ * takes two blocks where the 128-bit form takes one.  The instruction rate is+ * the same, so the throughput is twice -- measured at 2.00 on an EPYC 9V74,+ * for both halves, with nothing else in the loop.+ *+ * Nothing here is borrowed.  OpenSSL's and BoringSSL's wide AES-GCM is+ * Apache-2.0, s2n-bignum has no GCM at all, and the CRYPTOGAMS assembly in+ * cbits/asm is 128-bit throughout -- its `vaesenc` is the VEX encoding of+ * AESENC on XMM, not the VAES extension.  So this is the 128-bit loop in+ * cbits/aes/x86ni_impl.c widened, and it keeps that loop's shape: a group of+ * counters through the rounds together, the round keys read from memory+ * rather than held in registers, and the group's GHASH folded against+ * descending powers of H so that sixteen blocks share one reduction.+ *+ * The powers come from the table crypton_aesni_hinit_pclmul fills.  It has+ * sixteen slots and the 128-bit loop uses eight of them; this uses all+ * sixteen, which is why that function now fills them.+ */+#include "aes/gcm_vaes_x86.h"++#ifdef WITH_GCM_VAES++#include <string.h>+#include <immintrin.h>++#include <aes/gf.h>+#include <aes/block128.h>++#if defined(__clang__) || defined(__GNUC__)+#define VAES_TARGET __attribute__((target("avx2,aes,pclmul,vaes,vpclmulqdq")))+#else+#define VAES_TARGET+#endif++/* sixteen blocks to a group, two to a register */+#define VWIDE 8++/*+ * The 128-bit multiply of cbits/aes/x86ni.c, done in both lanes at once.+ * Every shuffle and shift here works inside its own 128-bit half, so the two+ * products never mix: what comes out is two independent carry-less products,+ * accumulated by the caller and reduced together at the end.+ */+VAES_TARGET+static inline void clmul256(__m256i a, __m256i b, __m256i *lo, __m256i *hi)+{+	const __m256i bswap = _mm256_setr_epi8(+		15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,0,+		15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,0);+	__m256i t3, t4, t5, t6;++	a = _mm256_shuffle_epi8(a, bswap);++	/* Karatsuba, as in the 128-bit one: three multiplies, not four */+	t3 = _mm256_clmulepi64_epi128(a, b, 0x00);+	t6 = _mm256_clmulepi64_epi128(a, b, 0x11);+	t4 = _mm256_clmulepi64_epi128(+		_mm256_xor_si256(a, _mm256_shuffle_epi32(a, 0x4e)),+		_mm256_xor_si256(b, _mm256_shuffle_epi32(b, 0x4e)), 0x00);+	t4 = _mm256_xor_si256(t4, _mm256_xor_si256(t3, t6));++	t5 = _mm256_slli_si256(t4, 8);+	t4 = _mm256_srli_si256(t4, 8);++	*lo = _mm256_xor_si256(t3, t5);+	*hi = _mm256_xor_si256(t6, t4);+}++/*+ * The reduction of cbits/aes/x86ni.c, unchanged: by the time it runs the two+ * lanes have been folded into one, so there is one 256-bit product to reduce+ * and no reason to do it twice.+ */+VAES_TARGET+static inline __m128i gfred(__m128i t3, __m128i t6)+{+	const __m128i bswap = _mm_set_epi8(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15);+	__m128i t2, t4, t5, t7, t8, t9;++	t7 = _mm_srli_epi32(t3, 31);+	t8 = _mm_srli_epi32(t6, 31);+	t3 = _mm_slli_epi32(t3, 1);+	t6 = _mm_slli_epi32(t6, 1);++	t9 = _mm_srli_si128(t7, 12);+	t8 = _mm_slli_si128(t8, 4);+	t7 = _mm_slli_si128(t7, 4);+	t3 = _mm_or_si128(t3, t7);+	t6 = _mm_or_si128(t6, t8);+	t6 = _mm_or_si128(t6, t9);++	t7 = _mm_slli_epi32(t3, 31);+	t8 = _mm_slli_epi32(t3, 30);+	t9 = _mm_slli_epi32(t3, 25);++	t7 = _mm_xor_si128(t7, t8);+	t7 = _mm_xor_si128(t7, t9);+	t8 = _mm_srli_si128(t7, 4);+	t7 = _mm_slli_si128(t7, 12);+	t3 = _mm_xor_si128(t3, t7);++	t2 = _mm_srli_epi32(t3, 1);+	t4 = _mm_srli_epi32(t3, 2);+	t5 = _mm_srli_epi32(t3, 7);+	t2 = _mm_xor_si128(t2, t4);+	t2 = _mm_xor_si128(t2, t5);+	t2 = _mm_xor_si128(t2, t8);+	t3 = _mm_xor_si128(t3, t2);+	t6 = _mm_xor_si128(t6, t3);++	return _mm_shuffle_epi8(t6, bswap);+}++/*+ * Sixteen blocks against H^16 .. H^1, one reduction.  v[j] holds blocks 2j+ * and 2j+1 in its low and high halves, so the powers for it are H^(16-2j)+ * low and H^(15-2j) high -- the table's own order the other way round, hence+ * the pair of 128-bit loads rather than one 256-bit one.+ */+VAES_TARGET+static inline __m128i ghash16(__m128i tag, const table_4bit htable,+                              const __m256i *v, int fromwire)+{+	__m256i lo = _mm256_setzero_si256(), hi = _mm256_setzero_si256();+	__m256i l, h, b;+	int j;++	for (j = 0; j < VWIDE; j++) {+		const __m256i hp = _mm256_set_m128i(+			_mm_loadu_si128((const __m128i *) &htable[14 - 2 * j]),+			_mm_loadu_si128((const __m128i *) &htable[15 - 2 * j]));++		b = fromwire ? _mm256_loadu_si256(v + j) : v[j];+		if (j == 0) /* the running tag joins the first block */+			b = _mm256_xor_si256(+				b, _mm256_inserti128_si256(+					_mm256_setzero_si256(), tag, 0));+		clmul256(b, hp, &l, &h);+		lo = _mm256_xor_si256(lo, l);+		hi = _mm256_xor_si256(hi, h);+	}++	/* the two lanes are independent products of the same sum: fold them */+	return gfred(_mm_xor_si128(_mm256_castsi256_si128(lo),+	                           _mm256_extracti128_si256(lo, 1)),+	             _mm_xor_si128(_mm256_castsi256_si128(hi),+	                           _mm256_extracti128_si256(hi, 1)));+}++#define KK(r) _mm256_broadcastsi128_si256(_mm_loadu_si128(k_ + (r)))++#define AESENC16(K)                                                          \+	do {                                                                 \+		const __m256i rk = (K);                                      \+		v[0] = _mm256_aesenc_epi128(v[0], rk);                       \+		v[1] = _mm256_aesenc_epi128(v[1], rk);                       \+		v[2] = _mm256_aesenc_epi128(v[2], rk);                       \+		v[3] = _mm256_aesenc_epi128(v[3], rk);                       \+		v[4] = _mm256_aesenc_epi128(v[4], rk);                       \+		v[5] = _mm256_aesenc_epi128(v[5], rk);                       \+		v[6] = _mm256_aesenc_epi128(v[6], rk);                       \+		v[7] = _mm256_aesenc_epi128(v[7], rk);                       \+	} while (0)++#define AESLAST16(K)                                                         \+	do {                                                                 \+		const __m256i rk = (K);                                      \+		v[0] = _mm256_aesenclast_epi128(v[0], rk);                   \+		v[1] = _mm256_aesenclast_epi128(v[1], rk);                   \+		v[2] = _mm256_aesenclast_epi128(v[2], rk);                   \+		v[3] = _mm256_aesenclast_epi128(v[3], rk);                   \+		v[4] = _mm256_aesenclast_epi128(v[4], rk);                   \+		v[5] = _mm256_aesenclast_epi128(v[5], rk);                   \+		v[6] = _mm256_aesenclast_epi128(v[6], rk);                   \+		v[7] = _mm256_aesenclast_epi128(v[7], rk);                   \+	} while (0)++#define XOR16(K)                                                             \+	do {                                                                 \+		const __m256i rk = (K);                                      \+		v[0] = _mm256_xor_si256(v[0], rk);                           \+		v[1] = _mm256_xor_si256(v[1], rk);                           \+		v[2] = _mm256_xor_si256(v[2], rk);                           \+		v[3] = _mm256_xor_si256(v[3], rk);                           \+		v[4] = _mm256_xor_si256(v[4], rk);                           \+		v[5] = _mm256_xor_si256(v[5], rk);                           \+		v[6] = _mm256_xor_si256(v[6], rk);                           \+		v[7] = _mm256_xor_si256(v[7], rk);                           \+	} while (0)++/*+ * The rounds are written out rather than looped for the reason the 128-bit+ * loop gives: the count is a value in the key, and a loop over it leaves the+ * round key reached through an index the compiler cannot fold.+ */+VAES_TARGET+static inline __attribute__((always_inline)) void+rounds16(__m256i *v, const uint8_t *k, const int nbr)+{+	const __m128i *k_ = (const __m128i *) k;++	XOR16(KK(0));+	AESENC16(KK(1)); AESENC16(KK(2)); AESENC16(KK(3));+	AESENC16(KK(4)); AESENC16(KK(5)); AESENC16(KK(6));+	AESENC16(KK(7)); AESENC16(KK(8)); AESENC16(KK(9));+	if (nbr > 10) {+		AESENC16(KK(10)); AESENC16(KK(11));+		if (nbr > 12) {+			AESENC16(KK(12)); AESENC16(KK(13));+		}+	}+	AESLAST16(_mm256_broadcastsi128_si256(_mm_loadu_si128(k_ + nbr)));+}++/* sixteen consecutive counters, two to a register.  GCM counts in the low+ * thirty-two bits and wraps there, which is what _mm_add_epi32 does. */+VAES_TARGET+static inline __m128i counters16(__m256i *v, __m128i iv, __m128i one,+                                 __m128i bswap)+{+	int j;++	for (j = 0; j < VWIDE; j++) {+		__m128i c0, c1;++		iv = _mm_add_epi32(iv, one);+		c0 = _mm_shuffle_epi8(iv, bswap);+		iv = _mm_add_epi32(iv, one);+		c1 = _mm_shuffle_epi8(iv, bswap);+		v[j] = _mm256_set_m128i(c1, c0);+	}+	return iv;+}++/*+ * The round count is a value in the key, and a test on it inside the group+ * loop is a branch the 128-bit path does not have: that one compiles a+ * separate function for each key length through the SIZED macro.  This does+ * the same thing by being inlined into three callers with the count a+ * constant in each, which folds the tests away.  Without it AES-256 lost+ * what AES-128 gained.+ */+VAES_TARGET+static inline __attribute__((always_inline)) uint32_t+bulk_n(uint8_t *output, aes_gcm *gcm, const aes_key *key,+       const uint8_t *input, uint32_t length, int decrypt, const int nbr)+{+	const __m128i bswap = _mm_setr_epi8(7,6,5,4,3,2,1,0,15,14,13,12,11,10,9,8);+	const __m128i one = _mm_set_epi32(0, 1, 0, 0);+	__m256i v[VWIDE];+	__m128i iv, tag;+	uint32_t groups = length / 256;+	uint32_t done = 0;+	uint32_t g;+	int j;++	if (groups == 0)+		return 0;++	iv = _mm_shuffle_epi8(_mm_loadu_si128((const __m128i *) &gcm->civ), bswap);+	tag = _mm_loadu_si128((const __m128i *) &gcm->tag);++	for (g = 0; g < groups; g++, input += 256, output += 256, done += 256) {+		iv = counters16(v, iv, one, bswap);+		rounds16(v, key->data, nbr);++		/*+		 * The ciphertext is what the tag is taken over, and after+		 * this exclusive or it is in v itself when encrypting.  When+		 * decrypting it is the input, which the GHASH below reads+		 * again rather than keeping: there are sixteen vector+		 * registers, the group fills eight of them, and a second+		 * eight held aside is what makes the compiler spill.  The+		 * input is in L1 from the load a moment ago.+		 */+		for (j = 0; j < VWIDE; j++) {+			const __m256i in =+				_mm256_loadu_si256((const __m256i *) (input + 32 * j));++			v[j] = _mm256_xor_si256(v[j], in);+			_mm256_storeu_si256((__m256i *) (output + 32 * j), v[j]);+		}+		tag = ghash16(tag, gcm->htable,+		              decrypt ? (const __m256i *) input : v,+		              decrypt);+	}++	_mm_storeu_si128((__m128i *) &gcm->civ, _mm_shuffle_epi8(iv, bswap));+	_mm_storeu_si128((__m128i *) &gcm->tag, tag);+	return done;+}++VAES_TARGET+static uint32_t bulk(uint8_t *output, aes_gcm *gcm, const aes_key *key,+                     const uint8_t *input, uint32_t length, int decrypt)+{+	switch (key->nbr) {+	case 10:+		return bulk_n(output, gcm, key, input, length, decrypt, 10);+	case 12:+		return bulk_n(output, gcm, key, input, length, decrypt, 12);+	case 14:+		return bulk_n(output, gcm, key, input, length, decrypt, 14);+	default:+		return 0; /* not a key length AES has */+	}+}++uint32_t crypton_gcm_vaes_bulk_encrypt(uint8_t *output, aes_gcm *gcm,+                                       const aes_key *key,+                                       const uint8_t *input, uint32_t length)+{+	return bulk(output, gcm, key, input, length, 0);+}++uint32_t crypton_gcm_vaes_bulk_decrypt(uint8_t *output, aes_gcm *gcm,+                                       const aes_key *key,+                                       const uint8_t *input, uint32_t length)+{+	return bulk(output, gcm, key, input, length, 1);+}++#endif
+ cbits/aes/gcm_vaes_x86.h view
@@ -0,0 +1,35 @@+/*+ * AES-GCM in the 256-bit form of the AES and carry-less multiply+ * instructions, which do two blocks where the 128-bit ones do one.+ */+#ifndef CRYPTON_GCM_VAES_X86_H+#define CRYPTON_GCM_VAES_X86_H++#include <crypton_cpu.h>++#if defined(ARCH_X86) && defined(__x86_64__) && defined(WITH_AESNI) \+    && defined(WITH_PCLMUL)+#define WITH_GCM_VAES+#endif++#ifdef WITH_GCM_VAES++#include <stdint.h>+#include <crypton_aes.h>++/* The bulk of a message in whole groups of sixteen blocks, leaving the+ * counter in gcm->civ and the running tag in gcm->tag where the caller's own+ * loop expects to find them.  Returns the number of bytes taken, which is a+ * multiple of 256 and may be zero. */+uint32_t crypton_gcm_vaes_bulk_encrypt(uint8_t *output, aes_gcm *gcm,+                                       const aes_key *key,+                                       const uint8_t *input, uint32_t length);+uint32_t crypton_gcm_vaes_bulk_decrypt(uint8_t *output, aes_gcm *gcm,+                                       const aes_key *key,+                                       const uint8_t *input, uint32_t length);++/* Sixteen blocks is the least it will start on. */+#define GCM_VAES_MIN_BLOCKS 16++#endif+#endif
+ cbits/aes/gcm_x86_asm.c view
@@ -0,0 +1,266 @@+/*+ * Copyright (c) 2026 Kazu Yamamoto <kazu@iij.ad.jp>+ *+ * All rights reserved.+ *+ * Redistribution and use in source and binary forms, with or without+ * modification, are permitted provided that the following conditions+ * are met:+ * 1. Redistributions of source code must retain the above copyright+ *    notice, this list of conditions and the following disclaimer.+ * 2. Redistributions in binary form must reproduce the above copyright+ *    notice, this list of conditions and the following disclaimer in the+ *    documentation and/or other materials provided with the distribution.+ * 3. Neither the name of the author nor the names of his contributors+ *    may be used to endorse or promote products derived from this software+ *    without specific prior written permission.+ *+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF+ * SUCH DAMAGE.+ *+ * What the stitched AES-GCM assembly in cbits/asm needs in order to be+ * called: the two pieces of state it reads are laid out the way OpenSSL+ * lays them out, which is not the way crypton does, and neither is worth+ * changing the rest of the library for.  Both are built here, per message,+ * from the key schedule and the H that crypton already has.+ */++#include "crypton_cpu.h"++#ifdef WITH_X86_GCM_ASM++#include <stddef.h>+#include <stdint.h>+#include <string.h>+#include <wmmintrin.h>+#include <crypton_aes.h>+#include <aes/gcm_x86_asm.h>++#ifdef WITH_TARGET_ATTRIBUTES+#define TARGET_PCLMUL __attribute__((target("sse4.1,pclmul")))+#else+#define TARGET_PCLMUL+#endif++#define ALIGNMENT(n) __attribute__((aligned(n)))++/*+ * cbits/asm/aesni-gcm-x86_64-*.S.  Both answer how many bytes they got+ * through, which is a multiple of six blocks and is zero if the message is+ * shorter than they are willing to start on.+ */+size_t crypton_gcm_asm_encrypt(const void *in, void *out, size_t len,+                               const void *key, uint8_t ivec[16], void *Xi);+size_t crypton_gcm_asm_decrypt(const void *in, void *out, size_t len,+                               const void *key, uint8_t ivec[16], void *Xi);++/*+ * The key schedule as the assembly reads it: the encryption round keys,+ * and at offset 240 the number of rounds less one, which is the count+ * OpenSSL's AES-NI key setup leaves there -- 9, 11 and 13 -- and what the+ * assembly compares against to tell the three key sizes apart.+ */+struct asm_key {+	uint8_t rd_key[240];+	uint32_t rounds;+};++/*+ * The assembly reads the running tag from the front of this and the powers+ * of H from 32 bytes in, which is where they sit in OpenSSL's GCM context+ * -- the 16 bytes between them hold H itself there and nothing here.+ * Powers up to the sixth are used, since the loop takes six blocks at a+ * time, and each pair of them is followed by the halves the Karatsuba+ * multiplication would otherwise have to add up again.+ */+struct asm_gcm {+	block128 xi;+	block128 unused;+	block128 htable[9];+};++/*+ * H, and every power of it, is kept shifted up by one bit: GCM numbers the+ * bits of a field element the other way round from the way the carry-less+ * multiply does, and pre-shifting the operand is what saves the correction+ * after each multiply.+ *+ * Written from the definition.  The field is GF(2)[x] modulo x^128 + x^127 ++ * x^126 + x^121 + 1, so multiplying by x is a shift of one place, and the+ * term that leaves the top comes back as the other four.+ */+TARGET_PCLMUL+static __m128i twist(__m128i h)+{+	/* x^127 + x^126 + x^121 + 1, the terms x^128 is congruent to */+	const __m128i poly = _mm_set_epi64x(0xc200000000000000ULL, 1);+	/* the top bit of each half */+	__m128i tops = _mm_srli_epi64(h, 63);+	/* doubling a polynomial is a shift by one: each half doubles, and the+	 * low half's top bit becomes the high half's bottom bit */+	__m128i doubled = _mm_or_si128(_mm_add_epi64(h, h),+	                               _mm_slli_si128(tops, 8));+	/* bit 127 is the one that leaves the field; spread it to a mask by+	 * subtracting it from zero, and it brings the four terms back */+	__m128i mask = _mm_sub_epi64(_mm_setzero_si128(),+	                             _mm_unpackhi_epi64(tops, tops));++	return _mm_xor_si128(doubled, _mm_and_si128(mask, poly));+}++/* the two halves of a value added together, which is the term Karatsuba+ * needs and which does not depend on what it is multiplied by */+TARGET_PCLMUL+static __m128i fold(__m128i a)+{+	return _mm_xor_si128(a, _mm_unpackhi_epi64(a, a));+}++/*+ * The table the assembly reads: the first six powers of H, each shifted up+ * by one, and after each pair the two halves of both of them added+ * together, which is the term the Karatsuba multiplication would otherwise+ * work out for itself every time.+ *+ * The powers are not computed here.  crypton's own table already holds+ * H^1 to H^8, in the byte order the multiply wants and unshifted, so+ * twisting each one is the whole of the work -- which is why this is worth+ * doing per message rather than keeping a second table in the context.+ */+TARGET_PCLMUL+static void init_htable(struct asm_gcm *st, const aes_gcm *gcm)+{+	int i;++	for (i = 0; i < 3; i++) {+		__m128i odd = twist(_mm_loadu_si128(+		    (const __m128i *) &gcm->htable[2 * i]));+		__m128i even = twist(_mm_loadu_si128(+		    (const __m128i *) &gcm->htable[2 * i + 1]));++		_mm_storeu_si128((__m128i *) &st->htable[3 * i + 0], odd);+		_mm_storeu_si128((__m128i *) &st->htable[3 * i + 1], even);+		_mm_storeu_si128((__m128i *) &st->htable[3 * i + 2],+		                 _mm_unpacklo_epi64(fold(odd), fold(even)));+	}+}++/*+ * The counter block, whose bottom 32 bits are what counts, as GCM has it.+ * crypton keeps the value it last used and the assembly wants the one it+ * is to use next, so this steps between the two conventions at each end.+ */+static void ctr32_bump(uint8_t ivec[16], uint32_t delta)+{+	uint32_t c = ((uint32_t) ivec[12] << 24) | ((uint32_t) ivec[13] << 16)+	           | ((uint32_t) ivec[14] << 8) | (uint32_t) ivec[15];++	c += delta;+	ivec[12] = (uint8_t) (c >> 24);+	ivec[13] = (uint8_t) (c >> 16);+	ivec[14] = (uint8_t) (c >> 8);+	ivec[15] = (uint8_t) c;+}++/*+ * How much of the message to hand over.  The assembly works in groups of+ * six blocks, and what it leaves behind goes to a loop that works in groups+ * of eight and then one at a time.  Handing over every group it could take+ * often leaves two or four blocks to go through one at a time, which at a+ * multiply apiece costs more than the three groups it takes to line the+ * remainder up on eight.  So the length is rounded down to whichever number+ * of six-block groups within reach leaves the least behind, modulo eight.+ */+static uint32_t handover(uint32_t blocks)+{+	uint32_t groups = blocks / 6;+	uint32_t best = groups;+	uint32_t least = (blocks - 6 * groups) % 8;+	uint32_t i;++	for (i = 1; i <= 3 && groups >= i; i++) {+		uint32_t left = (blocks - 6 * (groups - i)) % 8;++		if (left < least) {+			least = left;+			best = groups - i;+		}+	}+	return best * 6 * 16;+}++int crypton_gcm_asm_usable(void)+{+	static int resolved = 0;+	static int usable = 0;++	if (!resolved) {+		const uint32_t need = CRYPTON_X86_AVX | CRYPTON_X86_MOVBE+		                    | CRYPTON_X86_PCLMUL;++		usable = (crypton_x86_simd_features() & need) == need;+		resolved = 1;+	}+	return usable;+}++TARGET_PCLMUL+static uint32_t bulk(int encrypt, uint8_t *output, aes_gcm *gcm, aes_key *key,+                     const uint8_t *input, uint32_t length)+{+	struct asm_gcm st ALIGNMENT(16);+	struct asm_key k ALIGNMENT(16);+	uint8_t ivec[16] ALIGNMENT(16);+	uint32_t hand;+	size_t done;++	if (!crypton_gcm_asm_usable())+		return 0;++	/* below its own minimum the assembly does nothing, so in that case+	 * give it everything and let it decide */+	hand = handover(length / 16);+	if (hand < (encrypt ? 0x60 * 3 : 0x60))+		hand = length;++	memcpy(k.rd_key, key->data, 16 * (size_t) (key->nbr + 1));+	k.rounds = (uint32_t) key->nbr - 1;+	memcpy(&st.xi, &gcm->tag, 16);+	memcpy(ivec, &gcm->civ, 16);+	ctr32_bump(ivec, 1);+	init_htable(&st, gcm);++	done = encrypt+	     ? crypton_gcm_asm_encrypt(input, output, hand, &k, ivec, &st.xi)+	     : crypton_gcm_asm_decrypt(input, output, hand, &k, ivec, &st.xi);++	if (done > 0) {+		ctr32_bump(ivec, 0xffffffff);+		memcpy(&gcm->tag, &st.xi, 16);+		memcpy(&gcm->civ, ivec, 16);+	}+	return (uint32_t) done;+}++uint32_t crypton_gcm_asm_bulk_encrypt(uint8_t *output, aes_gcm *gcm, aes_key *key,+                                      const uint8_t *input, uint32_t length)+{+	return bulk(1, output, gcm, key, input, length);+}++uint32_t crypton_gcm_asm_bulk_decrypt(uint8_t *output, aes_gcm *gcm, aes_key *key,+                                      const uint8_t *input, uint32_t length)+{+	return bulk(0, output, gcm, key, input, length);+}++#endif
+ cbits/aes/gcm_x86_asm.h view
@@ -0,0 +1,72 @@+/*+ * Copyright (c) 2026 Kazu Yamamoto <kazu@iij.ad.jp>+ *+ * All rights reserved.+ *+ * Redistribution and use in source and binary forms, with or without+ * modification, are permitted provided that the following conditions+ * are met:+ * 1. Redistributions of source code must retain the above copyright+ *    notice, this list of conditions and the following disclaimer.+ * 2. Redistributions in binary form must reproduce the above copyright+ *    notice, this list of conditions and the following disclaimer in the+ *    documentation and/or other materials provided with the distribution.+ * 3. Neither the name of the author nor the names of his contributors+ *    may be used to endorse or promote products derived from this software+ *    without specific prior written permission.+ *+ * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF+ * SUCH DAMAGE.+ */+#ifndef CRYPTON_AES_GCM_X86_ASM_H+#define CRYPTON_AES_GCM_X86_ASM_H++#ifdef WITH_X86_GCM_ASM++#include <stdint.h>+#include <crypton_aes.h>++/*+ * How long a message has to be before it is handed over.  The assembly+ * needs the powers of H in a layout of its own, and what does not fill six+ * blocks is left to the loop that would otherwise have taken eight at a+ * time, so a short message pays for the setup and for a tail that goes+ * through one block at a time.  Encryption also spends its first twelve+ * blocks in plain counter mode before the stitched loop starts, which is+ * why it has to be given a good deal more before it comes out ahead.+ *+ * Measured on a Haswell-generation x86-64: decryption is ahead from 288+ * bytes up, by 5 to 20 per cent, and below that loses by about as much.+ * Encryption between 288 and 1024 bytes is a wash -- it swings either way+ * by up to ten per cent depending on how the length divides into groups --+ * and from 1152 bytes it is ahead by 9 per cent or more, reaching 25 to 40+ * per cent once the message is a few kilobytes.+ */+#define GCM_ASM_MIN_BLOCKS_ENC 72+#define GCM_ASM_MIN_BLOCKS_DEC 18++/* whether the processor has what cbits/asm/aesni-gcm-x86_64-*.S needs */+int crypton_gcm_asm_usable(void);++/*+ * Encrypt or decrypt from the front of the message, hashing as it goes, and+ * answer how much was done -- a multiple of 96 bytes, possibly none of it.+ * The counter and the running tag in *gcm are brought forward by that much.+ */+uint32_t crypton_gcm_asm_bulk_encrypt(uint8_t *output, aes_gcm *gcm, aes_key *key,+                                      const uint8_t *input, uint32_t length);+uint32_t crypton_gcm_asm_bulk_decrypt(uint8_t *output, aes_gcm *gcm, aes_key *key,+                                      const uint8_t *input, uint32_t length);++#endif++#endif
cbits/aes/gf.c view
@@ -144,3 +144,19 @@ 			block128_cpu_swap_be(a, &b); /* restore BE order when done */ 	} }++/*+ * Four GHASH steps at once.  The generic table-driven multiply has no cheaper+ * way to do this than one block at a time; the point of the entry is that the+ * PMULL and PCLMUL versions can fold the four products into one reduction, so+ * the GCM loops hand over four blocks whenever they have them.+ */+void crypton_aes_generic_gf_mul4(block128 *a, const block128 *blocks, const table_4bit htable)+{+	int i;++	for (i = 0; i < 4; i++) {+		block128_xor(a, &blocks[i]);+		crypton_aes_generic_gf_mul(a, htable);+	}+}
cbits/aes/gf.h view
@@ -38,5 +38,6 @@  void crypton_aes_generic_hinit(table_4bit htable, const block128 *h); void crypton_aes_generic_gf_mul(block128 *a, const table_4bit htable);+void crypton_aes_generic_gf_mul4(block128 *a, const block128 *blocks, const table_4bit htable);  #endif
cbits/aes/x86ni.c view
@@ -37,7 +37,10 @@ #include <crypton_cpu.h> #include <aes/gf.h> #include <aes/x86ni.h>+#include <aes/gcm_vaes_x86.h>+#include <aes/gcm_vaes512_x86.h> #include <aes/block128.h>+#include <aes/gcm_x86_asm.h>  #ifdef ARCH_X86 #define ALIGN_UP(addr, size) (((addr) + ((size) - 1)) & (~((size) - 1)))@@ -56,7 +59,23 @@ 	return _mm_xor_si128(key, keygened); } +/*+ * SubWord(RotWord(w)), which is the one part of a key schedule that would+ * otherwise want the S-box out of a table.  AESKEYGENASSIST computes it for+ * the words in lanes 1 and 3 and exclusive-ors the round constant into the+ * result; the constant is an immediate, so it is left at zero here and+ * applied by the caller, which keeps the 192-bit schedule a loop.+ */ TARGET_AESNI+static uint32_t key_sub_rot(uint32_t w)+{+	const __m128i t =+	    _mm_aeskeygenassist_si128(_mm_setr_epi32(0, (int) w, 0, 0), 0x00);++	return (uint32_t) _mm_cvtsi128_si32(_mm_srli_si128(t, 4));+}++TARGET_AESNI static __m128i aes_128_key_expansion_aa(__m128i key, __m128i keygened) { 	keygened = _mm_shuffle_epi32(keygened, 0xaa);@@ -105,6 +124,34 @@ 		for (i = 0; i < 20; i++) 			_mm_storeu_si128(((__m128i *) out) + i, k[i]); 		break;+	case 24: {+		/*+		 * The 192-bit schedule takes six words at a time where a round+		 * key is four, so it does not fall into 128-bit pieces the way+		 * the other two do; it is built a word at a time instead.+		 * Thirteen round keys, then the eleven inverted ones.+		 */+		static const uint32_t rcon[8] = {+			0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80,+		};+		uint32_t w[52];++		memcpy(w, ikey, 24);+		for (i = 6; i < 52; i++) {+			uint32_t t = w[i - 1];++			if (i % 6 == 0)+				t = key_sub_rot(t) ^ rcon[i / 6 - 1];+			w[i] = w[i - 6] ^ t;+		}+		memcpy(out, w, sizeof(w));++		for (i = 1; i < 12; i++)+			_mm_storeu_si128(((__m128i *) out) + 12 + i,+			    _mm_aesimc_si128(_mm_loadu_si128(+			        ((const __m128i *) w) + (12 - i))));+		break;+	} 	case 32: #define AES_256_key_exp_1(K1, K2, RCON) aes_128_key_expansion_ff(K1, _mm_aeskeygenassist_si128(K2, RCON)) #define AES_256_key_exp_2(K1, K2)       aes_128_key_expansion_aa(K1, _mm_aeskeygenassist_si128(K2, 0x00))@@ -162,46 +209,109 @@ 	return v; } +/* memcpy rather than a cast, as everything else that moves bytes between a+ * crypton structure and a word does since block128 was packed.  The cast+ * this replaces was written in 2014, when block128 was a plain union and+ * taking a __m128i * to one promised nothing the type did not already+ * offer.  Packing it dropped its alignment to one, and the promise with it:+ * gcc has reported the cast ever since, and it is right to -- the attribute+ * on the local below is what makes the promise true, and nothing obliges+ * the next edit to keep it.  Sixteen bytes of memcpy between a __m128i and+ * a sixteen-byte object is one movdqu, or nothing at all when both stay in+ * registers. */ TARGET_AESNI static __m128i gfmul_generic(__m128i tag, const table_4bit htable) {-	aes_block _t ALIGNMENT(16);-	_mm_store_si128((__m128i *) &_t, tag);+	aes_block _t;+	memcpy(&_t, &tag, sizeof _t); 	crypton_aes_generic_gf_mul(&_t, htable);-	tag = _mm_load_si128((__m128i *) &_t);+	memcpy(&tag, &_t, sizeof tag); 	return tag; } +/* Four or eight GHASH steps.  The table-driven multiply gains nothing from+ * seeing them together; the PCLMUL versions below fold them into one+ * reduction. */+TARGET_AESNI+static __m128i gfmul4_generic(__m128i tag, const table_4bit htable, const __m128i *m)+{+	int i;++	for (i = 0; i < 4; i++)+		tag = gfmul_generic(_mm_xor_si128(tag, m[i]), htable);+	return tag;+}++TARGET_AESNI+static __m128i gfmul8_generic(__m128i tag, const table_4bit htable, const __m128i *m)+{+	int i;++	for (i = 0; i < 8; i++)+		tag = gfmul_generic(_mm_xor_si128(tag, m[i]), htable);+	return tag;+}+ #ifdef WITH_PCLMUL  __m128i (*crypton_gfmul_branch_ptr)(__m128i a, const table_4bit t) = gfmul_generic; #define gfmul(a,t) ((*crypton_gfmul_branch_ptr)(a,t)) +__m128i (*crypton_gfmul4_branch_ptr)(__m128i a, const table_4bit t, const __m128i *m) = gfmul4_generic;+#define gfmul4(a,t,m) ((*crypton_gfmul4_branch_ptr)(a,t,m))++__m128i (*crypton_gfmul8_branch_ptr)(__m128i a, const table_4bit t, const __m128i *m) = gfmul8_generic;+#define gfmul8(a,t,m) ((*crypton_gfmul8_branch_ptr)(a,t,m))+ /* See Intel carry-less-multiplication-instruction-in-gcm-mode-paper.pdf  *  * Adapted from figure 5, with additional byte swapping so that interface  * is simimar to crypton_aes_generic_gf_mul.  */+/*+ * The 256-bit carry-less product, before the reflection fixup and the+ * reduction.  Split out from the reduction because both of those are linear+ * over XOR: several products can be added together and fixed up just once,+ * which is what gf_mul4 below does.+ */ TARGET_AESNI_PCLMUL-static __m128i gfmul_pclmuldq(__m128i a, const table_4bit htable)+static inline void clmul_pclmuldq(__m128i a, __m128i b, __m128i *lo, __m128i *hi) {-	__m128i b, tmp2, tmp3, tmp4, tmp5, tmp6, tmp7, tmp8, tmp9;+	__m128i tmp3, tmp4, tmp5, tmp6; 	__m128i bswap_mask = _mm_set_epi8(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15);  	a = _mm_shuffle_epi8(a, bswap_mask);-	b = _mm_loadu_si128((__m128i *) htable); +	/*+	 * Karatsuba: the middle term of the product is+	 * (a0^a1)(b0^b1) ^ a0b0 ^ a1b1, which is one carry-less multiply+	 * where the direct form needs two.  Three PCLMULQDQ rather than+	 * four, at the cost of a few shuffles and exclusive ors -- worth it+	 * wherever the multiply is the narrower port, which is every part+	 * this has been measured on.+	 */ 	tmp3 = _mm_clmulepi64_si128(a, b, 0x00);-	tmp4 = _mm_clmulepi64_si128(a, b, 0x10);-	tmp5 = _mm_clmulepi64_si128(a, b, 0x01); 	tmp6 = _mm_clmulepi64_si128(a, b, 0x11);+	tmp4 = _mm_clmulepi64_si128(_mm_xor_si128(a, _mm_shuffle_epi32(a, 0x4e)),+	                            _mm_xor_si128(b, _mm_shuffle_epi32(b, 0x4e)),+	                            0x00);+	tmp4 = _mm_xor_si128(tmp4, _mm_xor_si128(tmp3, tmp6)); -	tmp4 = _mm_xor_si128(tmp4, tmp5); 	tmp5 = _mm_slli_si128(tmp4, 8); 	tmp4 = _mm_srli_si128(tmp4, 8);-	tmp3 = _mm_xor_si128(tmp3, tmp5);-	tmp6 = _mm_xor_si128(tmp6, tmp4); +	*lo = _mm_xor_si128(tmp3, tmp5);+	*hi = _mm_xor_si128(tmp6, tmp4);+}++/* Shift the 256-bit product left by one to undo GCM's bit reflection, then+ * reduce modulo the GCM polynomial.  This is the expensive half. */+TARGET_AESNI_PCLMUL+static inline __m128i gfred_pclmuldq(__m128i tmp3, __m128i tmp6)+{+	__m128i tmp2, tmp4, tmp5, tmp7, tmp8, tmp9;+	__m128i bswap_mask = _mm_set_epi8(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15);+ 	tmp7 = _mm_srli_epi32(tmp3, 31); 	tmp8 = _mm_srli_epi32(tmp6, 31); 	tmp3 = _mm_slli_epi32(tmp3, 1);@@ -236,14 +346,41 @@ 	return _mm_shuffle_epi8(tmp6, bswap_mask); } +TARGET_AESNI_PCLMUL+static __m128i gfmul_pclmuldq(__m128i a, const table_4bit htable)+{+	__m128i lo, hi;++	clmul_pclmuldq(a, _mm_loadu_si128((__m128i *) htable), &lo, &hi);+	return gfred_pclmuldq(lo, hi);+}++TARGET_AESNI_PCLMUL void crypton_aesni_hinit_pclmul(table_4bit htable, const block128 *h) {+	__m128i bswap_mask = _mm_set_epi8(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15);+	__m128i p;+	int i;+ 	/* When pclmul is active we don't need to fill the table.  Instead we just 	 * store H at index 0.  It is written in reverse order, so function 	 * gfmul_pclmuldq will not byte-swap this value. 	 */-	htable->q[0] = bitfn_swap64(h->q[1]);-	htable->q[1] = bitfn_swap64(h->q[0]);+	htable[0].q[0] = bitfn_swap64(h->q[1]);+	htable[0].q[1] = bitfn_swap64(h->q[0]);++	/* Indices 1..15 get H^2 .. H^16, which is what lets a group of blocks+	 * fold into one reduction: gf_mul4 uses the first four, the 128-bit+	 * GCM loop eight, and the 256-bit one all sixteen.  The table has+	 * sixteen slots and now they are all used.  Filling the upper half+	 * costs eight multiplies once per key, which is nothing beside a+	 * message. */+	p = _mm_loadu_si128((const __m128i *) h);+	for (i = 1; i < 16; i++) {+		p = gfmul_pclmuldq(p, htable);+		_mm_storeu_si128((__m128i *) &htable[i],+		                 _mm_shuffle_epi8(p, bswap_mask));+	} }  TARGET_AESNI_PCLMUL@@ -255,13 +392,74 @@ 	_mm_storeu_si128((__m128i *) a, _b); } +/*+ * Four GHASH steps -- ((((a^b0)H ^ b1)H ^ b2)H ^ b3)H -- with a single+ * reduction.  Expanded that is (a^b0)H^4 ^ b1*H^3 ^ b2*H^2 ^ b3*H, so the+ * four products can be summed first and reduced once, which is where the+ * time goes.  Aggregated reduction, from the Intel GCM paper.+ */+TARGET_AESNI_PCLMUL+static __m128i gfmul4_pclmul(__m128i tag, const table_4bit htable, const __m128i *m)+{+	__m128i lo, hi, l, h;+	int i;++	clmul_pclmuldq(_mm_xor_si128(tag, m[0]),+	               _mm_loadu_si128((const __m128i *) &htable[3]), &lo, &hi);++	for (i = 1; i < 4; i++) {+		clmul_pclmuldq(m[i], _mm_loadu_si128((const __m128i *) &htable[3 - i]),+		               &l, &h);+		lo = _mm_xor_si128(lo, l);+		hi = _mm_xor_si128(hi, h);+	}++	return gfred_pclmuldq(lo, hi);+}++TARGET_AESNI_PCLMUL+static __m128i gfmul8_pclmul(__m128i tag, const table_4bit htable, const __m128i *m)+{+	__m128i lo, hi, l, h;+	int i;++	clmul_pclmuldq(_mm_xor_si128(tag, m[0]),+	               _mm_loadu_si128((const __m128i *) &htable[7]), &lo, &hi);++	for (i = 1; i < 8; i++) {+		clmul_pclmuldq(m[i], _mm_loadu_si128((const __m128i *) &htable[7 - i]),+		               &l, &h);+		lo = _mm_xor_si128(lo, l);+		hi = _mm_xor_si128(hi, h);+	}++	return gfred_pclmuldq(lo, hi);+}++TARGET_AESNI_PCLMUL+void crypton_aesni_gf_mul4_pclmul(block128 *a, const block128 *blocks, const table_4bit htable)+{+	__m128i m[4];+	int i;++	for (i = 0; i < 4; i++)+		m[i] = _mm_loadu_si128((const __m128i *) &blocks[i]);++	_mm_storeu_si128((__m128i *) a,+	                 gfmul4_pclmul(_mm_loadu_si128((const __m128i *) a), htable, m));+}+ void crypton_aesni_init_pclmul(void) { 	crypton_gfmul_branch_ptr = gfmul_pclmuldq;+	crypton_gfmul4_branch_ptr = gfmul4_pclmul;+	crypton_gfmul8_branch_ptr = gfmul8_pclmul; }  #else #define gfmul(a,t) (gfmul_generic(a,t))+#define gfmul4(a,t,m) (gfmul4_generic(a,t,m))+#define gfmul8(a,t,m) (gfmul8_generic(a,t,m)) #endif  TARGET_AESNI@@ -271,6 +469,164 @@ 	return gfmul(tag, htable); } +TARGET_AESNI+static inline __m128i ghash_add4(__m128i tag, const table_4bit htable, const __m128i *m)+{+	return gfmul4(tag, htable, m);+}++TARGET_AESNI+static inline __m128i ghash_add8(__m128i tag, const table_4bit htable, const __m128i *m)+{+	return gfmul8(tag, htable, m);+}++/*+ * Eight blocks through the rounds with the round keys read from memory rather+ * than held in registers.+ *+ * There are sixteen vector registers.  Eight blocks and eleven to fifteen+ * round keys do not fit in them, and when the GCM loop preloaded the keys the+ * compiler spilled: ninety-six stack accesses around a hundred AESENCs, which+ * cost more than half the loop's throughput.  AESENC takes a memory operand,+ * and the round keys are in L1 from one group to the next, so reading them+ * each round costs nothing and leaves the registers for the blocks.+ */+/*+ * Eight blocks through the rounds with the round keys read from memory rather+ * than held in registers.+ *+ * There are sixteen vector registers.  Eight blocks and eleven to fifteen+ * round keys do not fit in them, and when the GCM loop preloaded the keys the+ * compiler spilled: ninety-six stack accesses around a hundred AESENCs.+ * AESENC takes a memory operand and the round keys stay in L1 from one group+ * to the next, so reading them costs nothing and leaves the registers for the+ * blocks.+ *+ * The rounds are written out rather than looped: the loop cost a fifth of the+ * throughput, which is what -funroll-loops was recovering.+ */+#define K_(r) _mm_loadu_si128(k_ + (r))++/* the rounds beyond the tenth, which only a longer key has */+#define ROUNDS8_EXTRA_128+#define ROUNDS8_EXTRA_192 AESENC8(K_(10)) AESENC8(K_(11))+#define ROUNDS8_EXTRA_256 \+	AESENC8(K_(10)) AESENC8(K_(11)) AESENC8(K_(12)) AESENC8(K_(13))++#define DO_ENC_BLOCK8_MEM(m, k, nbr, EXTRA)                                  \+	do {                                                                 \+		const __m128i *k_ = (const __m128i *) (k);                   \+		XOR8(K_(0))                                                  \+		AESENC8(K_(1)) AESENC8(K_(2)) AESENC8(K_(3))                 \+		AESENC8(K_(4)) AESENC8(K_(5)) AESENC8(K_(6))                 \+		AESENC8(K_(7)) AESENC8(K_(8)) AESENC8(K_(9))                 \+		EXTRA                                                        \+		AESENCLAST8(K_(nbr))                                         \+	} while (0)++#define DO_ENC_BLOCK_MEM(m, k, nbr)                                          \+	do {                                                                 \+		const __m128i *k_ = (const __m128i *) (k);                   \+		int r_;                                                      \+		m = _mm_xor_si128(m, K_(0));                                 \+		for (r_ = 1; r_ < (nbr); r_++)                               \+			m = _mm_aesenc_si128(m, K_(r_));                     \+		m = _mm_aesenclast_si128(m, K_(nbr));                        \+	} while (0)++/*+ * GCM's GHASH, called directly rather than through the branch pointer the+ * other callers use: the pointer is a call the compiler cannot see through,+ * and these want to be scheduled against the rounds around them.  The cost is+ * that the GCM loops are compiled with the instruction and so may only be+ * installed where the processor has it, which crypton_aes.c sees to, as it+ * already does for the AArch64 ones.+ */+#ifdef WITH_PCLMUL++#define GCM_TARGET TARGET_AESNI_PCLMUL++TARGET_AESNI_PCLMUL+static inline __m128i gcm_ghash_add(__m128i tag, const table_4bit htable, __m128i m)+{+	return gfmul_pclmuldq(_mm_xor_si128(tag, m), htable);+}++TARGET_AESNI_PCLMUL+static inline __m128i gcm_ghash_add8(__m128i tag, const table_4bit htable, const __m128i *m)+{+	return gfmul8_pclmul(tag, htable, m);+}++/*+ * One block's carry-less multiply, accumulated rather than reduced, so that+ * the eight of a group can be spread between the rounds of the next group's+ * AES.+ */+TARGET_AESNI_PCLMUL+static inline void ghash_fold(__m128i *lo, __m128i *hi, __m128i b,+                              const table_4bit htable, int i)+{+	__m128i l, h;++	clmul_pclmuldq(b, _mm_loadu_si128((const __m128i *) &htable[i]), &l, &h);+	*lo = _mm_xor_si128(*lo, l);+	*hi = _mm_xor_si128(*hi, h);+}++#else++#define GCM_TARGET TARGET_AESNI+#define gcm_ghash_add(t, h, m)  ghash_add((t), (h), (m))+#define gcm_ghash_add8(t, h, m) ghash_add8((t), (h), (m))++#endif++/*+ * A group of eight encrypted, with the previous group's GHASH folded in+ * between the rounds where the build has the carry-less multiply: GH(j) after+ * round j + 1, and the reduction after round nine, which every key size+ * reaches.  The names are the ones the GCM loops use.+ */+#ifdef WITH_PCLMUL++#define GCM_GH(j)                                                            \+	ghash_fold(&glo_, &ghi_,                                             \+	           (j) == 0 ? _mm_xor_si128(tag, pending[0]) : pending[j],   \+	           gcm->htable, 7 - (j));++#define GCM_GHRED tag = gfred_pclmuldq(glo_, ghi_);++#define GCM_GROUP8(m, k, nbr, EXTRA)                                         \+	do {                                                                 \+		const __m128i *k_ = (const __m128i *) (k);                   \+		__m128i glo_ = _mm_setzero_si128();                          \+		__m128i ghi_ = _mm_setzero_si128();                          \+		XOR8(K_(0))                                                  \+		AESENC8(K_(1)) GCM_GH(0)                                     \+		AESENC8(K_(2)) GCM_GH(1)                                     \+		AESENC8(K_(3)) GCM_GH(2)                                     \+		AESENC8(K_(4)) GCM_GH(3)                                     \+		AESENC8(K_(5)) GCM_GH(4)                                     \+		AESENC8(K_(6)) GCM_GH(5)                                     \+		AESENC8(K_(7)) GCM_GH(6)                                     \+		AESENC8(K_(8)) GCM_GH(7)                                     \+		AESENC8(K_(9)) GCM_GHRED                                     \+		EXTRA                                                        \+		AESENCLAST8(K_(nbr))                                         \+	} while (0)++#else++#define GCM_GROUP8(m, k, nbr, EXTRA)                                         \+	do {                                                                 \+		DO_ENC_BLOCK8_MEM(m, k, nbr, EXTRA);                         \+		tag = ghash_add8(tag, gcm->htable, pending);                 \+	} while (0)++#endif+ #define PRELOAD_ENC_KEYS128(k) \ 	__m128i K0  = _mm_loadu_si128(((__m128i *) k)+0); \ 	__m128i K1  = _mm_loadu_si128(((__m128i *) k)+1); \@@ -284,6 +640,11 @@ 	__m128i K9  = _mm_loadu_si128(((__m128i *) k)+9); \ 	__m128i K10 = _mm_loadu_si128(((__m128i *) k)+10); +#define PRELOAD_ENC_KEYS192(k) \+	PRELOAD_ENC_KEYS128(k) \+	__m128i K11 = _mm_loadu_si128(((__m128i *) k)+11); \+	__m128i K12 = _mm_loadu_si128(((__m128i *) k)+12);+ #define PRELOAD_ENC_KEYS256(k) \ 	PRELOAD_ENC_KEYS128(k) \ 	__m128i K11 = _mm_loadu_si128(((__m128i *) k)+11); \@@ -304,6 +665,21 @@ 	m = _mm_aesenc_si128(m, K9); \ 	m = _mm_aesenclast_si128(m, K10); +#define DO_ENC_BLOCK192(m) \+	m = _mm_xor_si128(m, K0); \+	m = _mm_aesenc_si128(m, K1); \+	m = _mm_aesenc_si128(m, K2); \+	m = _mm_aesenc_si128(m, K3); \+	m = _mm_aesenc_si128(m, K4); \+	m = _mm_aesenc_si128(m, K5); \+	m = _mm_aesenc_si128(m, K6); \+	m = _mm_aesenc_si128(m, K7); \+	m = _mm_aesenc_si128(m, K8); \+	m = _mm_aesenc_si128(m, K9); \+	m = _mm_aesenc_si128(m, K10); \+	m = _mm_aesenc_si128(m, K11); \+	m = _mm_aesenclast_si128(m, K12);+ #define DO_ENC_BLOCK256(m) \ 	m = _mm_xor_si128(m, K0); \ 	m = _mm_aesenc_si128(m, K1); \@@ -334,10 +710,55 @@ 	__m128i K8  = _mm_loadu_si128(((__m128i *) k)+at+8); \ 	__m128i K9  = _mm_loadu_si128(((__m128i *) k)+at+9); \ +/*+ * Eight blocks through the rounds together, which is what covers the+ * latency of AESENC.  Written out one line per block rather than left to a+ * loop over m[i]: a loop is only as good as the compiler's willingness to+ * unroll it, and when it declines the blocks go to the stack and each+ * round becomes a load and a store.+ */+#define XOR8(KK) \+	m[0] = _mm_xor_si128(m[0], KK); m[1] = _mm_xor_si128(m[1], KK); \+	m[2] = _mm_xor_si128(m[2], KK); m[3] = _mm_xor_si128(m[3], KK); \+	m[4] = _mm_xor_si128(m[4], KK); m[5] = _mm_xor_si128(m[5], KK); \+	m[6] = _mm_xor_si128(m[6], KK); m[7] = _mm_xor_si128(m[7], KK);++#define AESENC8(KK) \+	m[0] = _mm_aesenc_si128(m[0], KK); m[1] = _mm_aesenc_si128(m[1], KK); \+	m[2] = _mm_aesenc_si128(m[2], KK); m[3] = _mm_aesenc_si128(m[3], KK); \+	m[4] = _mm_aesenc_si128(m[4], KK); m[5] = _mm_aesenc_si128(m[5], KK); \+	m[6] = _mm_aesenc_si128(m[6], KK); m[7] = _mm_aesenc_si128(m[7], KK);++#define AESENCLAST8(KK) \+	m[0] = _mm_aesenclast_si128(m[0], KK); m[1] = _mm_aesenclast_si128(m[1], KK); \+	m[2] = _mm_aesenclast_si128(m[2], KK); m[3] = _mm_aesenclast_si128(m[3], KK); \+	m[4] = _mm_aesenclast_si128(m[4], KK); m[5] = _mm_aesenclast_si128(m[5], KK); \+	m[6] = _mm_aesenclast_si128(m[6], KK); m[7] = _mm_aesenclast_si128(m[7], KK);++#define DO_ENC_BLOCK8_128(m) \+	XOR8(K0) AESENC8(K1) AESENC8(K2) AESENC8(K3) AESENC8(K4) AESENC8(K5) \+	AESENC8(K6) AESENC8(K7) AESENC8(K8) AESENC8(K9) AESENCLAST8(K10)++#define DO_ENC_BLOCK8_192(m) \+	XOR8(K0) AESENC8(K1) AESENC8(K2) AESENC8(K3) AESENC8(K4) AESENC8(K5) \+	AESENC8(K6) AESENC8(K7) AESENC8(K8) AESENC8(K9) AESENC8(K10) \+	AESENC8(K11) AESENCLAST8(K12)++#define DO_ENC_BLOCK8_256(m) \+	XOR8(K0) AESENC8(K1) AESENC8(K2) AESENC8(K3) AESENC8(K4) AESENC8(K5) \+	AESENC8(K6) AESENC8(K7) AESENC8(K8) AESENC8(K9) AESENC8(K10) \+	AESENC8(K11) AESENC8(K12) AESENC8(K13) AESENCLAST8(K14)+ #define PRELOAD_DEC_KEYS128(k) \ 	PRELOAD_DEC_KEYS_AT(k, 10) \ 	__m128i K10 = _mm_loadu_si128(((__m128i *) k)+0); +#define PRELOAD_DEC_KEYS192(k) \+	PRELOAD_DEC_KEYS_AT(k, 12) \+	__m128i K10 = _mm_loadu_si128(((__m128i *) k)+12+10); \+	__m128i K11 = _mm_loadu_si128(((__m128i *) k)+12+11); \+	__m128i K12 = _mm_loadu_si128(((__m128i *) k)+0);+ #define PRELOAD_DEC_KEYS256(k) \ 	PRELOAD_DEC_KEYS_AT(k, 14) \ 	__m128i K10 = _mm_loadu_si128(((__m128i *) k)+14+10); \@@ -346,6 +767,76 @@ 	__m128i K13 = _mm_loadu_si128(((__m128i *) k)+14+13); \ 	__m128i K14 = _mm_loadu_si128(((__m128i *) k)+0); +#define AESDEC8(KK) \+	m[0] = _mm_aesdec_si128(m[0], KK); m[1] = _mm_aesdec_si128(m[1], KK); \+	m[2] = _mm_aesdec_si128(m[2], KK); m[3] = _mm_aesdec_si128(m[3], KK); \+	m[4] = _mm_aesdec_si128(m[4], KK); m[5] = _mm_aesdec_si128(m[5], KK); \+	m[6] = _mm_aesdec_si128(m[6], KK); m[7] = _mm_aesdec_si128(m[7], KK);++#define AESDECLAST8(KK) \+	m[0] = _mm_aesdeclast_si128(m[0], KK); m[1] = _mm_aesdeclast_si128(m[1], KK); \+	m[2] = _mm_aesdeclast_si128(m[2], KK); m[3] = _mm_aesdeclast_si128(m[3], KK); \+	m[4] = _mm_aesdeclast_si128(m[4], KK); m[5] = _mm_aesdeclast_si128(m[5], KK); \+	m[6] = _mm_aesdeclast_si128(m[6], KK); m[7] = _mm_aesdeclast_si128(m[7], KK);++#define DO_DEC_BLOCK8_128(m) \+	XOR8(K0) AESDEC8(K1) AESDEC8(K2) AESDEC8(K3) AESDEC8(K4) AESDEC8(K5) \+	AESDEC8(K6) AESDEC8(K7) AESDEC8(K8) AESDEC8(K9) AESDECLAST8(K10)++#define DO_DEC_BLOCK8_192(m) \+	XOR8(K0) AESDEC8(K1) AESDEC8(K2) AESDEC8(K3) AESDEC8(K4) AESDEC8(K5) \+	AESDEC8(K6) AESDEC8(K7) AESDEC8(K8) AESDEC8(K9) AESDEC8(K10) \+	AESDEC8(K11) AESDECLAST8(K12)++#define DO_DEC_BLOCK8_256(m) \+	XOR8(K0) AESDEC8(K1) AESDEC8(K2) AESDEC8(K3) AESDEC8(K4) AESDEC8(K5) \+	AESDEC8(K6) AESDEC8(K7) AESDEC8(K8) AESDEC8(K9) AESDEC8(K10) \+	AESDEC8(K11) AESDEC8(K12) AESDEC8(K13) AESDECLAST8(K14)++/*+ * The XTS tweak advances by doubling in GF(2^128).  gfmulx above does that+ * through memory; this keeps it in a register, which matters once eight+ * tweaks are wanted per group.  The block is little-endian, so the low+ * 64-bit half is first.+ */+TARGET_AESNI+static inline __m128i gfmulx_sse(__m128i v)+{+	const __m128i poly = _mm_set_epi64x(0, 0x87);+	const __m128i carry = _mm_srli_epi64(v, 63);+	/* the low half's carry becomes the high half's bit 0 */+	const __m128i into_hi = _mm_slli_si128(carry, 8);+	/* and the high half's becomes all ones, or nothing, in the low half */+	const __m128i out = _mm_sub_epi64(_mm_setzero_si128(), _mm_srli_si128(carry, 8));++	return _mm_xor_si128(_mm_or_si128(_mm_slli_epi64(v, 1), into_hi),+	                     _mm_and_si128(out, poly));+}++/*+ * The tweak doubles in a pair of general-purpose registers and is moved+ * into a vector one per block.  The doubling is three integer operations,+ * and the integer units have nothing else to do here, where there are only+ * sixteen vector registers and the rounds want as many of them as they can+ * get: done in vector registers, which is what this did, the eight+ * doublings of a group both lengthen the critical path and push the round+ * keys out to memory.+ */+#define XTS_TWEAK_STEP(lo, hi) do {                                          \+	const uint64_t _c = (hi) >> 63;                                      \+	(hi) = ((hi) << 1) | ((lo) >> 63);                                   \+	(lo) = ((lo) << 1) ^ (_c ? 0x87 : 0);                                \+} while (0)++/* the eight tweaks a group needs, from the one it starts at */+#define XTS_TWEAKS8(dst, lo, hi) do {                                        \+	int _i;                                                              \+	for (_i = 0; _i < 8; _i++) {                                         \+		(dst)[_i] = _mm_set_epi64x((long long) (hi), (long long) (lo)); \+		XTS_TWEAK_STEP(lo, hi);                                      \+	}                                                                    \+} while (0)+ #define DO_DEC_BLOCK128(m) \ 	m = _mm_xor_si128(m, K0); \ 	m = _mm_aesdec_si128(m, K1); \@@ -359,6 +850,21 @@ 	m = _mm_aesdec_si128(m, K9); \ 	m = _mm_aesdeclast_si128(m, K10); +#define DO_DEC_BLOCK192(m) \+	m = _mm_xor_si128(m, K0); \+	m = _mm_aesdec_si128(m, K1); \+	m = _mm_aesdec_si128(m, K2); \+	m = _mm_aesdec_si128(m, K3); \+	m = _mm_aesdec_si128(m, K4); \+	m = _mm_aesdec_si128(m, K5); \+	m = _mm_aesdec_si128(m, K6); \+	m = _mm_aesdec_si128(m, K7); \+	m = _mm_aesdec_si128(m, K8); \+	m = _mm_aesdec_si128(m, K9); \+	m = _mm_aesdec_si128(m, K10); \+	m = _mm_aesdec_si128(m, K11); \+	m = _mm_aesdeclast_si128(m, K12);+ #define DO_DEC_BLOCK256(m) \ 	m = _mm_xor_si128(m, K0); \ 	m = _mm_aesdec_si128(m, K1); \@@ -377,34 +883,73 @@ 	m = _mm_aesdeclast_si128(m, K14);  #define SIZE 128+#define NBR 10+#define ROUNDS8_EXTRA ROUNDS8_EXTRA_128 #define SIZED(m) m##128 #define PRELOAD_ENC PRELOAD_ENC_KEYS128 #define DO_ENC_BLOCK DO_ENC_BLOCK128+#define DO_ENC_BLOCK8 DO_ENC_BLOCK8_128 #define PRELOAD_DEC PRELOAD_DEC_KEYS128 #define DO_DEC_BLOCK DO_DEC_BLOCK128+#define DO_DEC_BLOCK8 DO_DEC_BLOCK8_128 #include <aes/x86ni_impl.c>  #undef SIZE+#undef NBR+#undef ROUNDS8_EXTRA #undef SIZED #undef PRELOAD_ENC #undef PRELOAD_DEC #undef DO_ENC_BLOCK+#undef DO_ENC_BLOCK8 #undef DO_DEC_BLOCK+#undef DO_DEC_BLOCK8 +#define SIZED(m) m##192+#define SIZE 192+#define NBR 12+#define ROUNDS8_EXTRA ROUNDS8_EXTRA_192+#define PRELOAD_ENC PRELOAD_ENC_KEYS192+#define DO_ENC_BLOCK DO_ENC_BLOCK192+#define DO_ENC_BLOCK8 DO_ENC_BLOCK8_192+#define PRELOAD_DEC PRELOAD_DEC_KEYS192+#define DO_DEC_BLOCK DO_DEC_BLOCK192+#define DO_DEC_BLOCK8 DO_DEC_BLOCK8_192+#include <aes/x86ni_impl.c>++#undef SIZE+#undef NBR+#undef ROUNDS8_EXTRA+#undef SIZED+#undef PRELOAD_ENC+#undef PRELOAD_DEC+#undef DO_ENC_BLOCK+#undef DO_ENC_BLOCK8+#undef DO_DEC_BLOCK+#undef DO_DEC_BLOCK8+ #define SIZED(m) m##256 #define SIZE 256+#define NBR 14+#define ROUNDS8_EXTRA ROUNDS8_EXTRA_256 #define PRELOAD_ENC PRELOAD_ENC_KEYS256 #define DO_ENC_BLOCK DO_ENC_BLOCK256+#define DO_ENC_BLOCK8 DO_ENC_BLOCK8_256 #define PRELOAD_DEC PRELOAD_DEC_KEYS256 #define DO_DEC_BLOCK DO_DEC_BLOCK256+#define DO_DEC_BLOCK8 DO_DEC_BLOCK8_256 #include <aes/x86ni_impl.c>  #undef SIZE+#undef NBR+#undef ROUNDS8_EXTRA #undef SIZED #undef PRELOAD_ENC #undef PRELOAD_DEC #undef DO_ENC_BLOCK+#undef DO_ENC_BLOCK8 #undef DO_DEC_BLOCK+#undef DO_DEC_BLOCK8  #endif
cbits/aes/x86ni.h view
@@ -59,34 +59,30 @@ #endif  void crypton_aesni_init(aes_key *key, uint8_t *origkey, uint8_t size);-void crypton_aesni_encrypt_block128(aes_block *out, aes_key *key, aes_block *in);-void crypton_aesni_encrypt_block256(aes_block *out, aes_key *key, aes_block *in);-void crypton_aesni_decrypt_block128(aes_block *out, aes_key *key, aes_block *in);-void crypton_aesni_decrypt_block256(aes_block *out, aes_key *key, aes_block *in);-void crypton_aesni_encrypt_ecb128(aes_block *out, aes_key *key, aes_block *in, uint32_t blocks);-void crypton_aesni_encrypt_ecb256(aes_block *out, aes_key *key, aes_block *in, uint32_t blocks);-void crypton_aesni_decrypt_ecb128(aes_block *out, aes_key *key, aes_block *in, uint32_t blocks);-void crypton_aesni_decrypt_ecb256(aes_block *out, aes_key *key, aes_block *in, uint32_t blocks);-void crypton_aesni_encrypt_cbc128(aes_block *out, aes_key *key, aes_block *_iv, aes_block *in, uint32_t blocks);-void crypton_aesni_encrypt_cbc256(aes_block *out, aes_key *key, aes_block *_iv, aes_block *in, uint32_t blocks);-void crypton_aesni_decrypt_cbc128(aes_block *out, aes_key *key, aes_block *_iv, aes_block *in, uint32_t blocks);-void crypton_aesni_decrypt_cbc256(aes_block *out, aes_key *key, aes_block *_iv, aes_block *in, uint32_t blocks);-void crypton_aesni_encrypt_ctr128(uint8_t *out, aes_key *key, aes_block *_iv, uint8_t *in, uint32_t length);-void crypton_aesni_encrypt_ctr256(uint8_t *out, aes_key *key, aes_block *_iv, uint8_t *in, uint32_t length);-void crypton_aesni_encrypt_c32_128(uint8_t *out, aes_key *key, aes_block *_iv, uint8_t *in, uint32_t length);-void crypton_aesni_encrypt_c32_256(uint8_t *out, aes_key *key, aes_block *_iv, uint8_t *in, uint32_t length);-void crypton_aesni_encrypt_xts128(aes_block *out, aes_key *key1, aes_key *key2,-                           aes_block *_tweak, uint32_t spoint, aes_block *in, uint32_t blocks);-void crypton_aesni_encrypt_xts256(aes_block *out, aes_key *key1, aes_key *key2,-                           aes_block *_tweak, uint32_t spoint, aes_block *in, uint32_t blocks);--void crypton_aesni_gcm_encrypt128(uint8_t *out, aes_gcm *gcm, aes_key *key, uint8_t *in, uint32_t length);-void crypton_aesni_gcm_encrypt256(uint8_t *out, aes_gcm *gcm, aes_key *key, uint8_t *in, uint32_t length);+#define AESNI_DECLS(sz) \+	void crypton_aesni_encrypt_block##sz(aes_block *out, aes_key *key, aes_block *in); \+	void crypton_aesni_decrypt_block##sz(aes_block *out, aes_key *key, aes_block *in); \+	void crypton_aesni_encrypt_ecb##sz(aes_block *out, aes_key *key, aes_block *in, uint32_t blocks); \+	void crypton_aesni_decrypt_ecb##sz(aes_block *out, aes_key *key, aes_block *in, uint32_t blocks); \+	void crypton_aesni_encrypt_cbc##sz(aes_block *out, aes_key *key, aes_block *_iv, aes_block *in, uint32_t blocks); \+	void crypton_aesni_decrypt_cbc##sz(aes_block *out, aes_key *key, aes_block *_iv, aes_block *in, uint32_t blocks); \+	void crypton_aesni_encrypt_ctr##sz(uint8_t *out, aes_key *key, aes_block *_iv, uint8_t *in, uint32_t length); \+	void crypton_aesni_encrypt_c32_##sz(uint8_t *out, aes_key *key, aes_block *_iv, uint8_t *in, uint32_t length); \+	void crypton_aesni_encrypt_xts##sz(aes_block *out, aes_key *key1, aes_key *key2, \+	                           aes_block *_tweak, uint32_t spoint, aes_block *in, uint32_t blocks); \+	void crypton_aesni_decrypt_xts##sz(aes_block *out, aes_key *key1, aes_key *key2, \+	                           aes_block *_tweak, uint32_t spoint, aes_block *in, uint32_t blocks); \+	void crypton_aesni_gcm_encrypt##sz(uint8_t *out, aes_gcm *gcm, aes_key *key, uint8_t *in, uint32_t length); \+	void crypton_aesni_gcm_decrypt##sz(uint8_t *out, aes_gcm *gcm, aes_key *key, uint8_t *in, uint32_t length);+AESNI_DECLS(128)+AESNI_DECLS(192)+AESNI_DECLS(256)  #ifdef WITH_PCLMUL void crypton_aesni_init_pclmul(void); void crypton_aesni_hinit_pclmul(table_4bit htable, const block128 *h); void crypton_aesni_gf_mul_pclmul(block128 *a, const table_4bit htable);+void crypton_aesni_gf_mul4_pclmul(block128 *a, const block128 *blocks, const table_4bit htable); #endif  #endif
cbits/aes/x86ni_impl.c view
@@ -204,34 +204,124 @@ void SIZED(crypton_aesni_encrypt_xts)(aes_block *out, aes_key *key1, aes_key *key2,                                aes_block *_tweak, uint32_t spoint, aes_block *in, uint32_t blocks) {-	__m128i tweak = _mm_loadu_si128((__m128i *) _tweak);+	uint64_t tlo, thi;  	do { 		__m128i *k2 = (__m128i *) key2->data;+		__m128i tweak = _mm_loadu_si128((__m128i *) _tweak);+		aes_block first ALIGNMENT(16);+ 		PRELOAD_ENC(k2); 		DO_ENC_BLOCK(tweak);+		_mm_storeu_si128((__m128i *) &first, tweak);+		tlo = first.q[0];+		thi = first.q[1];  		while (spoint-- > 0)-			tweak = gfmulx(tweak);+			XTS_TWEAK_STEP(tlo, thi); 	} while (0) ;  	do { 		__m128i *k1 = (__m128i *) key1->data;-		PRELOAD_ENC(k1); -		for ( ; blocks-- > 0; in += 1, out += 1, tweak = gfmulx(tweak)) {+		/*+		 * Eight at a time.  The eight tweaks are kept from one group+		 * to the next and each is advanced by eight doublings at+		 * once, which is a single multiplication and does not wait+		 * for the other seven; doubling along the group instead,+		 * which is what this did, puts a chain of eight in front of+		 * every set of rounds, and on a processor whose AES is fast+		 * that chain is most of the block.+		 */+		for ( ; blocks >= 8; blocks -= 8, in += 8, out += 8) {+			__m128i m[8], t[8];+			int i;++			XTS_TWEAKS8(t, tlo, thi);+			for (i = 0; i < 8; i++)+				m[i] = _mm_xor_si128(+				    _mm_loadu_si128((__m128i *) (in + i)), t[i]);+			DO_ENC_BLOCK8_MEM(m, k1, NBR, ROUNDS8_EXTRA);+			for (i = 0; i < 8; i++)+				_mm_storeu_si128((__m128i *) (out + i),+				                 _mm_xor_si128(m[i], t[i]));+		}+		for ( ; blocks-- > 0; in += 1, out += 1) {+			const __m128i tweak =+			    _mm_set_epi64x((long long) thi, (long long) tlo); 			__m128i m = _mm_loadu_si128((__m128i *) in);  			m = _mm_xor_si128(m, tweak);-			DO_ENC_BLOCK(m);+			DO_ENC_BLOCK_MEM(m, k1, NBR); 			m = _mm_xor_si128(m, tweak);  			_mm_storeu_si128((__m128i *) out, m);+			XTS_TWEAK_STEP(tlo, thi); 		} 	} while (0); } +/*+ * XTS the other way, which until now fell to the generic loop -- and which+ * nothing reached at all, since crypton_aes_decrypt_xts called the generic+ * function directly rather than through the branch table.  The tweak is+ * enciphered whichever way the data goes; only the data is deciphered.+ */ TARGET_AESNI+void SIZED(crypton_aesni_decrypt_xts)(aes_block *out, aes_key *key1, aes_key *key2,+                               aes_block *_tweak, uint32_t spoint, aes_block *in, uint32_t blocks)+{+	uint64_t tlo, thi;++	do {+		__m128i *k2 = (__m128i *) key2->data;+		__m128i tweak = _mm_loadu_si128((__m128i *) _tweak);+		aes_block first ALIGNMENT(16);++		PRELOAD_ENC(k2);+		DO_ENC_BLOCK(tweak);+		_mm_storeu_si128((__m128i *) &first, tweak);+		tlo = first.q[0];+		thi = first.q[1];++		while (spoint-- > 0)+			XTS_TWEAK_STEP(tlo, thi);+	} while (0) ;++	do {+		__m128i *k1 = (__m128i *) key1->data;+		PRELOAD_DEC(k1);++		/* the tweaks kept and advanced, as encryption has them */+		for ( ; blocks >= 8; blocks -= 8, in += 8, out += 8) {+			__m128i m[8], t[8];+			int i;++			XTS_TWEAKS8(t, tlo, thi);+			for (i = 0; i < 8; i++)+				m[i] = _mm_xor_si128(+				    _mm_loadu_si128((__m128i *) (in + i)), t[i]);+			DO_DEC_BLOCK8(m);+			for (i = 0; i < 8; i++)+				_mm_storeu_si128((__m128i *) (out + i),+				                 _mm_xor_si128(m[i], t[i]));+		}+		for ( ; blocks-- > 0; in += 1, out += 1) {+			const __m128i tweak =+			    _mm_set_epi64x((long long) thi, (long long) tlo);+			__m128i m = _mm_loadu_si128((__m128i *) in);++			m = _mm_xor_si128(m, tweak);+			DO_DEC_BLOCK(m);+			m = _mm_xor_si128(m, tweak);++			_mm_storeu_si128((__m128i *) out, m);+			XTS_TWEAK_STEP(tlo, thi);+		}+	} while (0);+}++GCM_TARGET void SIZED(crypton_aesni_gcm_encrypt)(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length) { 	__m128i *k = (__m128i *) key->data;@@ -239,15 +329,107 @@ 	__m128i one        = _mm_set_epi32(0,1,0,0); 	uint32_t nb_blocks = length / 16; 	uint32_t part_block_len = length % 16;+	/* the group of ciphertext whose GHASH has not been taken yet */+	__m128i pending[8];+	int held = 0;  	gcm->length_input += length; +#ifdef WITH_GCM_VAES512+	/*+	 * The widest form the processor has, first: four blocks to an+	 * instruction where the 256-bit one below takes two and the assembly+	 * after that takes one.  Same contract throughout -- whole groups off+	 * the front, the counter and the tag left behind.+	 */+	if (nb_blocks >= GCM_VAES512_MIN_BLOCKS+	    && (crypton_x86_simd_features() & CRYPTON_X86_VAES512)) {+		uint32_t done = crypton_gcm_vaes512_bulk_encrypt(+			output, gcm, key, input, nb_blocks * 16);++		output += done;+		input += done;+		nb_blocks -= done / 16;+	}+#endif+#ifdef WITH_GCM_VAES+	/*+	 * The 256-bit instructions next: they take two blocks where the ones+	 * below take one, and the assembly that follows is 128-bit+	 * throughout.+	 */+	if (nb_blocks >= GCM_VAES_MIN_BLOCKS+	    && (crypton_x86_simd_features() & CRYPTON_X86_VAES)) {+		uint32_t done = crypton_gcm_vaes_bulk_encrypt(output, gcm, key,+		                                              input,+		                                              nb_blocks * 16);++		output += done;+		input += done;+		nb_blocks -= done / 16;+	}+#endif+#if defined(WITH_X86_GCM_ASM) && defined(WITH_PCLMUL)+	/*+	 * The stitched assembly next, which takes whole groups of six+	 * blocks off the front of the message and leaves the counter and the+	 * running tag where the loop below expects to find them.  It wants+	 * eighteen blocks before it will start, and answers with what it did.+	 */+	if (nb_blocks >= GCM_ASM_MIN_BLOCKS_ENC) {+		uint32_t done = crypton_gcm_asm_bulk_encrypt(output, gcm, key,+		                                             input, nb_blocks * 16);++		output += done;+		input += done;+		nb_blocks -= done / 16;+	}+#endif+ 	__m128i tag = _mm_loadu_si128((__m128i *) &gcm->tag); 	__m128i iv = _mm_loadu_si128((__m128i *) &gcm->civ); 	iv = _mm_shuffle_epi8(iv, bswap_mask); -	PRELOAD_ENC(k); +	/*+	 * Eight blocks at a time: the counters go through the rounds together+	 * so the pipeline has something to do while AESENC is in flight, and+	 * their GHASH folds into one reduction against H^8 .. H^1 rather than+	 * eight.+	 *+	 * The GHASH is of the group before, not this one.  Taken in step the+	 * two halves cannot overlap at all: the multiply of a block waits for+	 * the rounds that produced it, and on this processor they do not even+	 * want the same port -- AESENC and PCLMULQDQ issue to different ones,+	 * so held a group apart they run through each other.  It costs one+	 * group's worth of ciphertext kept aside and a last GHASH after the+	 * loop.+	 */+	for (; nb_blocks >= 8; nb_blocks -= 8, output += 128, input += 128) {+		__m128i m[8];+		int i;++		for (i = 0; i < 8; i++) {+			/* iv += 1, put back in big endian */+			iv = _mm_add_epi32(iv, one);+			m[i] = _mm_shuffle_epi8(iv, bswap_mask);+		}+		if (held)+			GCM_GROUP8(m, k, NBR, ROUNDS8_EXTRA);+		else+			DO_ENC_BLOCK8_MEM(m, k, NBR, ROUNDS8_EXTRA);++		for (i = 0; i < 8; i++) {+			m[i] = _mm_xor_si128(m[i],+			                     _mm_loadu_si128((__m128i *) (input + 16 * i)));+			_mm_storeu_si128((__m128i *) (output + 16 * i), m[i]);+		}+		for (i = 0; i < 8; i++)+			pending[i] = m[i];+		held = 1;+	}+	if (held)+		tag = gcm_ghash_add8(tag, gcm->htable, pending); 	for (; nb_blocks-- > 0; output += 16, input += 16) { 		/* iv += 1 */ 		iv = _mm_add_epi32(iv, one);@@ -255,11 +437,11 @@ 		/* put back iv in big endian, encrypt it, 		 * and xor it to input */ 		__m128i tmp = _mm_shuffle_epi8(iv, bswap_mask);-		DO_ENC_BLOCK(tmp);+		DO_ENC_BLOCK_MEM(tmp, k, NBR); 		__m128i m = _mm_loadu_si128((__m128i *) input); 		m = _mm_xor_si128(m, tmp); -		tag = ghash_add(tag, gcm->htable, m);+		tag = gcm_ghash_add(tag, gcm->htable, m);  		/* store it out */ 		_mm_storeu_si128((__m128i *) output, m);@@ -294,16 +476,145 @@  		/* put back iv in big endian mode, encrypt it and xor it with input */ 		__m128i tmp = _mm_shuffle_epi8(iv, bswap_mask);-		DO_ENC_BLOCK(tmp);+		DO_ENC_BLOCK_MEM(tmp, k, NBR);  		__m128i m = _mm_loadu_si128((__m128i *) &block); 		m = _mm_xor_si128(m, tmp); 		m = _mm_shuffle_epi8(m, mask); -		tag = ghash_add(tag, gcm->htable, m);+		tag = gcm_ghash_add(tag, gcm->htable, m);  		/* make output */ 		_mm_storeu_si128((__m128i *) &block.b, m);+		memcpy(output, &block.b, part_block_len);+	}+	/* store back IV & tag */+	__m128i tmp = _mm_shuffle_epi8(iv, bswap_mask);+	_mm_storeu_si128((__m128i *) &gcm->civ, tmp);+	_mm_storeu_si128((__m128i *) &gcm->tag, tag);+}++/*+ * GCM decryption, which until now fell to the generic loop: that advances+ * the counter and calls the block function once per block through the+ * branch table, and measured a quarter the speed of encryption on the same+ * machine.  The shape is the encryption loop with two differences -- the+ * tag is taken over the ciphertext, which is the input rather than the+ * output, and the ciphertext is read before anything is written, since+ * output may be input.+ */+GCM_TARGET+void SIZED(crypton_aesni_gcm_decrypt)(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length)+{+	__m128i *k = (__m128i *) key->data;+	__m128i bswap_mask = _mm_setr_epi8(7,6,5,4,3,2,1,0,15,14,13,12,11,10,9,8);+	__m128i one        = _mm_set_epi32(0,1,0,0);+	uint32_t nb_blocks = length / 16;+	uint32_t part_block_len = length % 16;+	/* the group of ciphertext whose GHASH has not been taken yet */+	__m128i pending[8];+	int held = 0;++	gcm->length_input += length;++#ifdef WITH_GCM_VAES512+	/* as in encryption; the tag is taken over the input here */+	if (nb_blocks >= GCM_VAES512_MIN_BLOCKS+	    && (crypton_x86_simd_features() & CRYPTON_X86_VAES512)) {+		uint32_t done = crypton_gcm_vaes512_bulk_decrypt(+			output, gcm, key, input, nb_blocks * 16);++		output += done;+		input += done;+		nb_blocks -= done / 16;+	}+#endif+#ifdef WITH_GCM_VAES+	/* as in encryption; the tag is taken over the input here */+	if (nb_blocks >= GCM_VAES_MIN_BLOCKS+	    && (crypton_x86_simd_features() & CRYPTON_X86_VAES)) {+		uint32_t done = crypton_gcm_vaes_bulk_decrypt(output, gcm, key,+		                                              input,+		                                              nb_blocks * 16);++		output += done;+		input += done;+		nb_blocks -= done / 16;+	}+#endif+#if defined(WITH_X86_GCM_ASM) && defined(WITH_PCLMUL)+	/* the same as encryption, except that decryption has nothing to+	 * hold back and so will start on six blocks */+	if (nb_blocks >= GCM_ASM_MIN_BLOCKS_DEC) {+		uint32_t done = crypton_gcm_asm_bulk_decrypt(output, gcm, key,+		                                             input, nb_blocks * 16);++		output += done;+		input += done;+		nb_blocks -= done / 16;+	}+#endif++	__m128i tag = _mm_loadu_si128((__m128i *) &gcm->tag);+	__m128i iv = _mm_loadu_si128((__m128i *) &gcm->civ);+	iv = _mm_shuffle_epi8(iv, bswap_mask);+++	/* the group before's GHASH, alongside this group's rounds, as+	 * encryption does it */+	for (; nb_blocks >= 8; nb_blocks -= 8, output += 128, input += 128) {+		__m128i m[8], c[8];+		int i;++		for (i = 0; i < 8; i++) {+			/* iv += 1, put back in big endian */+			iv = _mm_add_epi32(iv, one);+			m[i] = _mm_shuffle_epi8(iv, bswap_mask);+		}+		for (i = 0; i < 8; i++)+			c[i] = _mm_loadu_si128((__m128i *) (input + 16 * i));+		if (held)+			GCM_GROUP8(m, k, NBR, ROUNDS8_EXTRA);+		else+			DO_ENC_BLOCK8_MEM(m, k, NBR, ROUNDS8_EXTRA);++		for (i = 0; i < 8; i++)+			_mm_storeu_si128((__m128i *) (output + 16 * i),+			                 _mm_xor_si128(m[i], c[i]));+		for (i = 0; i < 8; i++)+			pending[i] = c[i];+		held = 1;+	}+	if (held)+		tag = gcm_ghash_add8(tag, gcm->htable, pending);+	for (; nb_blocks-- > 0; output += 16, input += 16) {+		__m128i c = _mm_loadu_si128((__m128i *) input);++		iv = _mm_add_epi32(iv, one);+		__m128i tmp = _mm_shuffle_epi8(iv, bswap_mask);+		DO_ENC_BLOCK_MEM(tmp, k, NBR);++		tag = gcm_ghash_add(tag, gcm->htable, c);+		_mm_storeu_si128((__m128i *) output, _mm_xor_si128(tmp, c));+	}+	if (part_block_len > 0) {+		aes_block block;++		/* the ciphertext padded with zeros is what the tag is taken+		 * over, so no mask is needed the way encryption needs one */+		block128_zero(&block);+		block128_copy_bytes(&block, input, part_block_len);+		__m128i c = _mm_loadu_si128((__m128i *) &block);++		/* iv += 1 */+		iv = _mm_add_epi32(iv, one);++		__m128i tmp = _mm_shuffle_epi8(iv, bswap_mask);+		DO_ENC_BLOCK_MEM(tmp, k, NBR);++		tag = gcm_ghash_add(tag, gcm->htable, c);++		_mm_storeu_si128((__m128i *) &block.b, _mm_xor_si128(tmp, c)); 		memcpy(output, &block.b, part_block_len); 	} 	/* store back IV & tag */
+ cbits/asm/LICENSE.cryptogams view
@@ -0,0 +1,36 @@+Copyright (c) 2006, CRYPTOGAMS by <appro@openssl.org>+All rights reserved.++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions+are met:++      *	Redistributions of source code must retain copyright notices,+	this list of conditions and the following disclaimer.++      *	Redistributions in binary form must reproduce the above+	copyright notice, this list of conditions and the following+	disclaimer in the documentation and/or other materials+	provided with the distribution.++      *	Neither the name of the CRYPTOGAMS nor the names of its+	copyright holder and contributors may be used to endorse or+	promote products derived from this software without specific+	prior written permission.++ALTERNATIVELY, provided that this notice is retained in full, this+product may be distributed under the terms of the GNU General Public+License (GPL), in which case the provisions of the GPL apply INSTEAD OF+those given above.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ cbits/asm/README.md view
@@ -0,0 +1,171 @@+# Vendored assembly++## What is here++Two modules from [CRYPTOGAMS](https://github.com/dot-asm/cryptogams), by Andy+Polyakov, checked in unmodified together with the translators they need:++| generator | what it is |+| --- | --- |+| `aesni-gcm-x86_64.pl` | AES-NI/PCLMULQDQ stitched AES-GCM for x86-64 |+| `chacha-x86_64.pl` | ChaCha20 for x86-64 |+| `poly1305-x86_64.pl` | Poly1305 for x86-64 |+| `sha512-x86_64.pl` | SHA-256 and SHA-512 for x86-64 (one generator, two outputs, as on AArch64) |+| `keccak1600-x86_64.pl` | Keccak for x86-64 |+| `chacha-armv8.pl` | ChaCha20 for AArch64 |+| `poly1305-armv8.pl` | Poly1305 for AArch64 |+| `sha1-armv8.pl` | SHA-1 for AArch64 |+| `sha512-armv8.pl` | SHA-256 for AArch64 (the generator emits SHA-512 or SHA-256 according to the name it is given, and only the latter is wanted) |+| `keccak1600-armv8.pl` | Keccak for AArch64 |++`x86_64-xlate.pl`, `arm-xlate.pl` and `arm_arch.h` are the machinery those+modules use.  `generate.sh` runs the generators to produce the `.S` files, which are+what crypton actually compiles -- one per object format, since the calling+convention and the assembler syntax differ.  The names of the entry points are+changed on the way through, and the ELF output is given the note that says the+code does not want an executable stack.++Keeping the generated files in the tree means building crypton needs no perl.++## Why++Both do something the compiler will not do with intrinsics.++**AES-GCM.** Counter-mode AES and GHASH do not compete for the same execution+ports, so a loop that interleaves them at instruction granularity runs both in+the time one of them would take.  Written in C the interleaving does not+survive: given the AES rounds and the multiplies of a group of blocks, GCC and+Clang schedule all the multiplies after all the rounds, which is the sum of the+two rather than the maximum.++**ChaCha20.** On AArch64 the vector registers hold four ChaCha states and there+is no room for a fifth, so further parallelism has to come from the integer+side: that module runs a fifth block through the general registers alongside+four in the vector ones, and above 512 bytes two alongside six.  Which register+holds which word is the whole trick, and that is not something C says.  The+x86-64 module is worth taking for a different reason -- it has vector code for+lengths the C here still takes a block at a time, so a 256-byte message more+than doubles -- and is a few per cent ahead in bulk besides.++**Poly1305.** One multiplication modulo 2^130 - 5 depends on the one before it,+so what there is to win is in how the multiplies and the carries are laid+against each other, and in keeping the accumulator in whichever base costs+less: these modules work in base 2^64 while the message is short and switch to+base 2^26 for the vector loop, which is a decision no compiler will make for+you.  On AArch64 that is twice the speed of the C here at 16 KiB and three and+a half times at 64 bytes; on x86-64, a quarter faster at 16 KiB and nearly+three times at 64.++The x86-64 module also has paths for AVX-512, which are **not** taken.  What+the generator emits is chosen from the version of the assembler it is told+about, and `generate.sh` tells it one that predates AVX-512: no machine here+can run those paths, an assembler old enough to be in use cannot always+assemble them, and a path nothing has executed is not worth the few per cent+it might be worth.++**SHA-1, SHA-256, SHA-512 and Keccak.** On AArch64 the instructions are the same ones the+intrinsics here already use.  What the module does is schedule them across a+whole run of blocks instead of one at a time, and keep the message schedule of+the next block moving while the rounds of this one are still going, which a+per-block C function cannot do at all.  A quarter faster, and it needs no+alignment and no copy since it reads the message as bytes.  The x86-64 module+is the same idea with more paths to choose from -- the SHA extensions, AVX2,+AVX, SSSE3 -- and is about a fifth faster than the C on a machine with AVX2+and no SHA extensions.  The AArch64 SHA-1 and Keccak modules are the same+story again -- the instructions are the ones the intrinsics here use, and what+the modules add is the arrangement: the schedule of the next four SHA-1 rounds+against the rounds of this one, and one Keccak round against the next.  The+x86-64 Keccak is there for a different reason: nothing on that side has+instructions for this permutation, and what the module has over the C is that+its twenty-five lanes stay in registers across a round, where a compiler given+the C spills them.  Two and a half times, and level with openssl.++## Interfaces++    size_t crypton_gcm_asm_encrypt(const void *in, void *out, size_t len,+                                   const void *key, unsigned char ivec[16],+                                   void *Xi);+    size_t crypton_gcm_asm_decrypt(... the same ...);++Both return the number of bytes processed, which is a multiple of 96 and may be+zero: encryption wants at least 288 bytes to start, decryption at least 96.+Whatever is left over is the caller's to finish.++`key` is the AES key schedule in the layout the OpenSSL assembly expects -- the+round keys, and at offset 240 one less than the number of rounds, which is what+OpenSSL's own AES-NI key setup puts there -- and `Xi` points at the running+GHASH state, with the table of powers of H, in the layout `gcm_init_avx` leaves+behind, 32 bytes past it.  `cbits/aes/gcm_x86_asm.c` builds both, and the+multiplication that fills that table is written there in C rather than taken+from `ghash-x86_64.pl`: it runs once per message, so it is not worth a second+vendored file, and the one in OpenSSL is under a licence this package does not+use.++The code needs AES-NI, PCLMULQDQ, AVX and MOVBE, which+`crypton_x86_simd_features()` is asked about before any of it is called.++    void crypton_chacha20_ctr32(unsigned char *out, const unsigned char *in,+                                size_t len, const unsigned int key[8],+                                const unsigned int counter[4]);++Twenty rounds, the constants that go with a 256-bit key, and a 32-bit counter+which it does not write back: the caller advances it by the number of blocks.+Any length is accepted; the AArch64 module's vector path starts at 192 bytes,+the x86-64 one's rather lower.  They ask `crypton_armcap_P` and+`crypton_ia32cap_P` respectively what the processor has, and+`cbits/crypton_chacha.c` calls them only for the states they fit -- twenty+rounds, a 256-bit key, and only as many blocks as the 32-bit counter has room+for.++    int  crypton_poly1305_asm_init(void *ctx, const unsigned char key[16],+                                   void *func[2]);+    void crypton_poly1305_asm_blocks(void *ctx, const unsigned char *inp,+                                     size_t len, unsigned int padbit);+    void crypton_poly1305_asm_emit(void *ctx, unsigned char mac[16],+                                   const unsigned int nonce[4]);++`ctx` is 192 bytes of state the module keeps for itself -- its accumulator, the+clamped key and the powers of it -- and `key` is the first half of the Poly1305+key, the second half being handed to `emit` as `nonce`.  `padbit` is the bit+above each block, set for the blocks of the message and clear for the padded+last one.  `len` is a whole number of blocks.++Initialisation hands back through `func` the pair of functions its own dispatch+would use, the vector entry point not being exported, and+`cbits/crypton_poly1305.c` calls those.  It reads `crypton_armcap_P` to choose+between them; `cbits/crypton_cpu.c` defines that.++The x86-64 Poly1305 module presents the same three functions, and reads+`crypton_ia32cap_P` -- cpuid's own words, in the order OpenSSL keeps them --+where the AArch64 one reads `crypton_armcap_P`.  `cbits/crypton_cpu.c` fills+it, with the bits for anything the operating system will not preserve cleared,+and the AVX-512 ones cleared whatever the processor says.++    void crypton_sha1_asm_block_data_order(unsigned int state[5],+                                           const void *data, size_t blocks);+    void crypton_sha256_asm_block_data_order(unsigned int state[8],+                                             const void *data, size_t blocks);+    size_t crypton_keccak_asm_absorb_cext(unsigned long long state[25],+                                          const void *inp, size_t len,+                                          size_t bsz);++The state is the words of the digest in host order and `blocks` whole blocks of+64 bytes.  Each entry point picks between the SHA-2 instructions, NEON and+plain integer code from `crypton_armcap_P`, whose SHA-1 and SHA-256 bits+`cbits/crypton_sha1.c` and `cbits/crypton_sha256.c` set once they have asked+the operating system whether the processor has them -- they are optional in+ARMv8.0.++Keccak's absorb takes the state as its twenty-five lanes, `bsz` as the rate in+bytes, and answers with what was left over.  On AArch64 the `_cext` entry point+is the one that uses the SHA-3 instructions, and `cbits/crypton_sha3.c` calls+it only where its own runtime check has found them; the x86-64 module asks+nothing of the processor beyond the baseline and is called wherever it is+compiled in.++## Licence++`LICENSE.cryptogams` is the licence the CRYPTOGAMS files are distributed under.+It is the three-clause BSD licence, with the GNU General Public Licence offered+as an alternative; crypton takes the former, which is the licence of the rest+of this package.
+ cbits/asm/aesni-gcm-x86_64-elf.S view
@@ -0,0 +1,814 @@+.text	++.type	_crypton_gcm_asm_ctr32_ghash_6x,@function+.align	32+_crypton_gcm_asm_ctr32_ghash_6x:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	vmovdqu	32(%r11),%xmm2+	subq	$6,%rdx+	vpxor	%xmm4,%xmm4,%xmm4+	vmovdqu	0-128(%rcx),%xmm15+	vpaddb	%xmm2,%xmm1,%xmm10+	vpaddb	%xmm2,%xmm10,%xmm11+	vpaddb	%xmm2,%xmm11,%xmm12+	vpaddb	%xmm2,%xmm12,%xmm13+	vpaddb	%xmm2,%xmm13,%xmm14+	vpxor	%xmm15,%xmm1,%xmm9+	vmovdqu	%xmm4,16+8(%rsp)+	jmp	.Loop6x++.align	32+.Loop6x:+	addl	$100663296,%ebx+	jc	.Lhandle_ctr32+	vmovdqu	0-32(%r9),%xmm3+	vpaddb	%xmm2,%xmm14,%xmm1+	vpxor	%xmm15,%xmm10,%xmm10+	vpxor	%xmm15,%xmm11,%xmm11++.Lresume_ctr32:+	vmovdqu	%xmm1,(%r8)+	vpclmulqdq	$0x10,%xmm3,%xmm7,%xmm5+	vpxor	%xmm15,%xmm12,%xmm12+	vmovups	16-128(%rcx),%xmm2+	vpclmulqdq	$0x01,%xmm3,%xmm7,%xmm6+	xorq	%r12,%r12+	cmpq	%r14,%r15++	vaesenc	%xmm2,%xmm9,%xmm9+	vmovdqu	48+8(%rsp),%xmm0+	vpxor	%xmm15,%xmm13,%xmm13+	vpclmulqdq	$0x00,%xmm3,%xmm7,%xmm1+	vaesenc	%xmm2,%xmm10,%xmm10+	vpxor	%xmm15,%xmm14,%xmm14+	setnc	%r12b+	vpclmulqdq	$0x11,%xmm3,%xmm7,%xmm7+	vaesenc	%xmm2,%xmm11,%xmm11+	vmovdqu	16-32(%r9),%xmm3+	negq	%r12+	vaesenc	%xmm2,%xmm12,%xmm12+	vpxor	%xmm5,%xmm6,%xmm6+	vpclmulqdq	$0x00,%xmm3,%xmm0,%xmm5+	vpxor	%xmm4,%xmm8,%xmm8+	vaesenc	%xmm2,%xmm13,%xmm13+	vpxor	%xmm5,%xmm1,%xmm4+	andq	$0x60,%r12+	vmovups	32-128(%rcx),%xmm15+	vpclmulqdq	$0x10,%xmm3,%xmm0,%xmm1+	vaesenc	%xmm2,%xmm14,%xmm14++	vpclmulqdq	$0x01,%xmm3,%xmm0,%xmm2+	leaq	(%r14,%r12,1),%r14+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	16+8(%rsp),%xmm8,%xmm8+	vpclmulqdq	$0x11,%xmm3,%xmm0,%xmm3+	vmovdqu	64+8(%rsp),%xmm0+	vaesenc	%xmm15,%xmm10,%xmm10+	movbeq	88(%r14),%r13+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	80(%r14),%r12+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r13,32+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	movq	%r12,40+8(%rsp)+	vmovdqu	48-32(%r9),%xmm5+	vaesenc	%xmm15,%xmm14,%xmm14++	vmovups	48-128(%rcx),%xmm15+	vpxor	%xmm1,%xmm6,%xmm6+	vpclmulqdq	$0x00,%xmm5,%xmm0,%xmm1+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm2,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm5,%xmm0,%xmm2+	vaesenc	%xmm15,%xmm10,%xmm10+	vpxor	%xmm3,%xmm7,%xmm7+	vpclmulqdq	$0x01,%xmm5,%xmm0,%xmm3+	vaesenc	%xmm15,%xmm11,%xmm11+	vpclmulqdq	$0x11,%xmm5,%xmm0,%xmm5+	vmovdqu	80+8(%rsp),%xmm0+	vaesenc	%xmm15,%xmm12,%xmm12+	vaesenc	%xmm15,%xmm13,%xmm13+	vpxor	%xmm1,%xmm4,%xmm4+	vmovdqu	64-32(%r9),%xmm1+	vaesenc	%xmm15,%xmm14,%xmm14++	vmovups	64-128(%rcx),%xmm15+	vpxor	%xmm2,%xmm6,%xmm6+	vpclmulqdq	$0x00,%xmm1,%xmm0,%xmm2+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm3,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm1,%xmm0,%xmm3+	vaesenc	%xmm15,%xmm10,%xmm10+	movbeq	72(%r14),%r13+	vpxor	%xmm5,%xmm7,%xmm7+	vpclmulqdq	$0x01,%xmm1,%xmm0,%xmm5+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	64(%r14),%r12+	vpclmulqdq	$0x11,%xmm1,%xmm0,%xmm1+	vmovdqu	96+8(%rsp),%xmm0+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r13,48+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	movq	%r12,56+8(%rsp)+	vpxor	%xmm2,%xmm4,%xmm4+	vmovdqu	96-32(%r9),%xmm2+	vaesenc	%xmm15,%xmm14,%xmm14++	vmovups	80-128(%rcx),%xmm15+	vpxor	%xmm3,%xmm6,%xmm6+	vpclmulqdq	$0x00,%xmm2,%xmm0,%xmm3+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm5,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm2,%xmm0,%xmm5+	vaesenc	%xmm15,%xmm10,%xmm10+	movbeq	56(%r14),%r13+	vpxor	%xmm1,%xmm7,%xmm7+	vpclmulqdq	$0x01,%xmm2,%xmm0,%xmm1+	vpxor	112+8(%rsp),%xmm8,%xmm8+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	48(%r14),%r12+	vpclmulqdq	$0x11,%xmm2,%xmm0,%xmm2+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r13,64+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	movq	%r12,72+8(%rsp)+	vpxor	%xmm3,%xmm4,%xmm4+	vmovdqu	112-32(%r9),%xmm3+	vaesenc	%xmm15,%xmm14,%xmm14++	vmovups	96-128(%rcx),%xmm15+	vpxor	%xmm5,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm3,%xmm8,%xmm5+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm1,%xmm6,%xmm6+	vpclmulqdq	$0x01,%xmm3,%xmm8,%xmm1+	vaesenc	%xmm15,%xmm10,%xmm10+	movbeq	40(%r14),%r13+	vpxor	%xmm2,%xmm7,%xmm7+	vpclmulqdq	$0x00,%xmm3,%xmm8,%xmm2+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	32(%r14),%r12+	vpclmulqdq	$0x11,%xmm3,%xmm8,%xmm8+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r13,80+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	movq	%r12,88+8(%rsp)+	vpxor	%xmm5,%xmm6,%xmm6+	vaesenc	%xmm15,%xmm14,%xmm14+	vpxor	%xmm1,%xmm6,%xmm6++	vmovups	112-128(%rcx),%xmm15+	vpslldq	$8,%xmm6,%xmm5+	vpxor	%xmm2,%xmm4,%xmm4+	vmovdqu	16(%r11),%xmm3++	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm8,%xmm7,%xmm7+	vaesenc	%xmm15,%xmm10,%xmm10+	vpxor	%xmm5,%xmm4,%xmm4+	movbeq	24(%r14),%r13+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	16(%r14),%r12+	vpalignr	$8,%xmm4,%xmm4,%xmm0+	vpclmulqdq	$0x10,%xmm3,%xmm4,%xmm4+	movq	%r13,96+8(%rsp)+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r12,104+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	vmovups	128-128(%rcx),%xmm1+	vaesenc	%xmm15,%xmm14,%xmm14++	vaesenc	%xmm1,%xmm9,%xmm9+	vmovups	144-128(%rcx),%xmm15+	vaesenc	%xmm1,%xmm10,%xmm10+	vpsrldq	$8,%xmm6,%xmm6+	vaesenc	%xmm1,%xmm11,%xmm11+	vpxor	%xmm6,%xmm7,%xmm7+	vaesenc	%xmm1,%xmm12,%xmm12+	vpxor	%xmm0,%xmm4,%xmm4+	movbeq	8(%r14),%r13+	vaesenc	%xmm1,%xmm13,%xmm13+	movbeq	0(%r14),%r12+	vaesenc	%xmm1,%xmm14,%xmm14+	vmovups	160-128(%rcx),%xmm1+	cmpl	$11,%r10d+	jb	.Lenc_tail++	vaesenc	%xmm15,%xmm9,%xmm9+	vaesenc	%xmm15,%xmm10,%xmm10+	vaesenc	%xmm15,%xmm11,%xmm11+	vaesenc	%xmm15,%xmm12,%xmm12+	vaesenc	%xmm15,%xmm13,%xmm13+	vaesenc	%xmm15,%xmm14,%xmm14++	vaesenc	%xmm1,%xmm9,%xmm9+	vaesenc	%xmm1,%xmm10,%xmm10+	vaesenc	%xmm1,%xmm11,%xmm11+	vaesenc	%xmm1,%xmm12,%xmm12+	vaesenc	%xmm1,%xmm13,%xmm13+	vmovups	176-128(%rcx),%xmm15+	vaesenc	%xmm1,%xmm14,%xmm14+	vmovups	192-128(%rcx),%xmm1+	je	.Lenc_tail++	vaesenc	%xmm15,%xmm9,%xmm9+	vaesenc	%xmm15,%xmm10,%xmm10+	vaesenc	%xmm15,%xmm11,%xmm11+	vaesenc	%xmm15,%xmm12,%xmm12+	vaesenc	%xmm15,%xmm13,%xmm13+	vaesenc	%xmm15,%xmm14,%xmm14++	vaesenc	%xmm1,%xmm9,%xmm9+	vaesenc	%xmm1,%xmm10,%xmm10+	vaesenc	%xmm1,%xmm11,%xmm11+	vaesenc	%xmm1,%xmm12,%xmm12+	vaesenc	%xmm1,%xmm13,%xmm13+	vmovups	208-128(%rcx),%xmm15+	vaesenc	%xmm1,%xmm14,%xmm14+	vmovups	224-128(%rcx),%xmm1+	jmp	.Lenc_tail++.align	32+.Lhandle_ctr32:+	vmovdqu	(%r11),%xmm0+	vpshufb	%xmm0,%xmm1,%xmm6+	vmovdqu	48(%r11),%xmm5+	vpaddd	64(%r11),%xmm6,%xmm10+	vpaddd	%xmm5,%xmm6,%xmm11+	vmovdqu	0-32(%r9),%xmm3+	vpaddd	%xmm5,%xmm10,%xmm12+	vpshufb	%xmm0,%xmm10,%xmm10+	vpaddd	%xmm5,%xmm11,%xmm13+	vpshufb	%xmm0,%xmm11,%xmm11+	vpxor	%xmm15,%xmm10,%xmm10+	vpaddd	%xmm5,%xmm12,%xmm14+	vpshufb	%xmm0,%xmm12,%xmm12+	vpxor	%xmm15,%xmm11,%xmm11+	vpaddd	%xmm5,%xmm13,%xmm1+	vpshufb	%xmm0,%xmm13,%xmm13+	vpshufb	%xmm0,%xmm14,%xmm14+	vpshufb	%xmm0,%xmm1,%xmm1+	jmp	.Lresume_ctr32++.align	32+.Lenc_tail:+	vaesenc	%xmm15,%xmm9,%xmm9+	vmovdqu	%xmm7,16+8(%rsp)+	vpalignr	$8,%xmm4,%xmm4,%xmm8+	vaesenc	%xmm15,%xmm10,%xmm10+	vpclmulqdq	$0x10,%xmm3,%xmm4,%xmm4+	vpxor	0(%rdi),%xmm1,%xmm2+	vaesenc	%xmm15,%xmm11,%xmm11+	vpxor	16(%rdi),%xmm1,%xmm0+	vaesenc	%xmm15,%xmm12,%xmm12+	vpxor	32(%rdi),%xmm1,%xmm5+	vaesenc	%xmm15,%xmm13,%xmm13+	vpxor	48(%rdi),%xmm1,%xmm6+	vaesenc	%xmm15,%xmm14,%xmm14+	vpxor	64(%rdi),%xmm1,%xmm7+	vpxor	80(%rdi),%xmm1,%xmm3+	vmovdqu	(%r8),%xmm1++	vaesenclast	%xmm2,%xmm9,%xmm9+	vmovdqu	32(%r11),%xmm2+	vaesenclast	%xmm0,%xmm10,%xmm10+	vpaddb	%xmm2,%xmm1,%xmm0+	movq	%r13,112+8(%rsp)+	leaq	96(%rdi),%rdi+	vaesenclast	%xmm5,%xmm11,%xmm11+	vpaddb	%xmm2,%xmm0,%xmm5+	movq	%r12,120+8(%rsp)+	leaq	96(%rsi),%rsi+	vmovdqu	0-128(%rcx),%xmm15+	vaesenclast	%xmm6,%xmm12,%xmm12+	vpaddb	%xmm2,%xmm5,%xmm6+	vaesenclast	%xmm7,%xmm13,%xmm13+	vpaddb	%xmm2,%xmm6,%xmm7+	vaesenclast	%xmm3,%xmm14,%xmm14+	vpaddb	%xmm2,%xmm7,%xmm3++	addq	$0x60,%rax+	subq	$0x6,%rdx+	jc	.L6x_done++	vmovups	%xmm9,-96(%rsi)+	vpxor	%xmm15,%xmm1,%xmm9+	vmovups	%xmm10,-80(%rsi)+	vmovdqa	%xmm0,%xmm10+	vmovups	%xmm11,-64(%rsi)+	vmovdqa	%xmm5,%xmm11+	vmovups	%xmm12,-48(%rsi)+	vmovdqa	%xmm6,%xmm12+	vmovups	%xmm13,-32(%rsi)+	vmovdqa	%xmm7,%xmm13+	vmovups	%xmm14,-16(%rsi)+	vmovdqa	%xmm3,%xmm14+	vmovdqu	32+8(%rsp),%xmm7+	jmp	.Loop6x++.L6x_done:+	vpxor	16+8(%rsp),%xmm8,%xmm8+	vpxor	%xmm4,%xmm8,%xmm8++	.byte	0xf3,0xc3+.cfi_endproc	+.size	_crypton_gcm_asm_ctr32_ghash_6x,.-_crypton_gcm_asm_ctr32_ghash_6x+.globl	crypton_gcm_asm_decrypt+.type	crypton_gcm_asm_decrypt,@function+.align	32+crypton_gcm_asm_decrypt:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+	xorq	%rax,%rax+	cmpq	$0x60,%rdx+	jb	.Lgcm_dec_abort++	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56++	vzeroupper++	vmovdqu	(%r8),%xmm1+	addq	$-128,%rsp+	movl	12(%r8),%ebx+	leaq	.Lbswap_mask(%rip),%r11+	leaq	-128(%rcx),%r14+	movq	$0xf80,%r15+	vmovdqu	(%r9),%xmm8+	andq	$-128,%rsp+	vmovdqu	(%r11),%xmm0+	leaq	128(%rcx),%rcx+	leaq	32+32(%r9),%r9+	movl	240-128(%rcx),%r10d+	vpshufb	%xmm0,%xmm8,%xmm8++	andq	%r15,%r14+	andq	%rsp,%r15+	subq	%r14,%r15+	jc	.Ldec_no_key_aliasing+	cmpq	$768,%r15+	jnc	.Ldec_no_key_aliasing+	subq	%r15,%rsp+.Ldec_no_key_aliasing:++	vmovdqu	80(%rdi),%xmm7+	leaq	(%rdi),%r14+	vmovdqu	64(%rdi),%xmm4+	leaq	-192(%rdi,%rdx,1),%r15+	vmovdqu	48(%rdi),%xmm5+	shrq	$4,%rdx+	xorq	%rax,%rax+	vmovdqu	32(%rdi),%xmm6+	vpshufb	%xmm0,%xmm7,%xmm7+	vmovdqu	16(%rdi),%xmm2+	vpshufb	%xmm0,%xmm4,%xmm4+	vmovdqu	(%rdi),%xmm3+	vpshufb	%xmm0,%xmm5,%xmm5+	vmovdqu	%xmm4,48(%rsp)+	vpshufb	%xmm0,%xmm6,%xmm6+	vmovdqu	%xmm5,64(%rsp)+	vpshufb	%xmm0,%xmm2,%xmm2+	vmovdqu	%xmm6,80(%rsp)+	vpshufb	%xmm0,%xmm3,%xmm3+	vmovdqu	%xmm2,96(%rsp)+	vmovdqu	%xmm3,112(%rsp)++	call	_crypton_gcm_asm_ctr32_ghash_6x++	vmovups	%xmm9,-96(%rsi)+	vmovups	%xmm10,-80(%rsi)+	vmovups	%xmm11,-64(%rsi)+	vmovups	%xmm12,-48(%rsi)+	vmovups	%xmm13,-32(%rsi)+	vmovups	%xmm14,-16(%rsi)++	vpshufb	(%r11),%xmm8,%xmm8+	vmovdqu	%xmm8,-64(%r9)++	vzeroupper+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+.Lgcm_dec_abort:+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_gcm_asm_decrypt,.-crypton_gcm_asm_decrypt+.type	_crypton_gcm_asm_ctr32_6x,@function+.align	32+_crypton_gcm_asm_ctr32_6x:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	vmovdqu	0-128(%rcx),%xmm4+	vmovdqu	32(%r11),%xmm2+	leaq	-1(%r10),%r13+	vmovups	16-128(%rcx),%xmm15+	leaq	32-128(%rcx),%r12+	vpxor	%xmm4,%xmm1,%xmm9+	addl	$100663296,%ebx+	jc	.Lhandle_ctr32_2+	vpaddb	%xmm2,%xmm1,%xmm10+	vpaddb	%xmm2,%xmm10,%xmm11+	vpxor	%xmm4,%xmm10,%xmm10+	vpaddb	%xmm2,%xmm11,%xmm12+	vpxor	%xmm4,%xmm11,%xmm11+	vpaddb	%xmm2,%xmm12,%xmm13+	vpxor	%xmm4,%xmm12,%xmm12+	vpaddb	%xmm2,%xmm13,%xmm14+	vpxor	%xmm4,%xmm13,%xmm13+	vpaddb	%xmm2,%xmm14,%xmm1+	vpxor	%xmm4,%xmm14,%xmm14+	jmp	.Loop_ctr32++.align	16+.Loop_ctr32:+	vaesenc	%xmm15,%xmm9,%xmm9+	vaesenc	%xmm15,%xmm10,%xmm10+	vaesenc	%xmm15,%xmm11,%xmm11+	vaesenc	%xmm15,%xmm12,%xmm12+	vaesenc	%xmm15,%xmm13,%xmm13+	vaesenc	%xmm15,%xmm14,%xmm14+	vmovups	(%r12),%xmm15+	leaq	16(%r12),%r12+	decl	%r13d+	jnz	.Loop_ctr32++	vmovdqu	(%r12),%xmm3+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	0(%rdi),%xmm3,%xmm4+	vaesenc	%xmm15,%xmm10,%xmm10+	vpxor	16(%rdi),%xmm3,%xmm5+	vaesenc	%xmm15,%xmm11,%xmm11+	vpxor	32(%rdi),%xmm3,%xmm6+	vaesenc	%xmm15,%xmm12,%xmm12+	vpxor	48(%rdi),%xmm3,%xmm8+	vaesenc	%xmm15,%xmm13,%xmm13+	vpxor	64(%rdi),%xmm3,%xmm2+	vaesenc	%xmm15,%xmm14,%xmm14+	vpxor	80(%rdi),%xmm3,%xmm3+	leaq	96(%rdi),%rdi++	vaesenclast	%xmm4,%xmm9,%xmm9+	vaesenclast	%xmm5,%xmm10,%xmm10+	vaesenclast	%xmm6,%xmm11,%xmm11+	vaesenclast	%xmm8,%xmm12,%xmm12+	vaesenclast	%xmm2,%xmm13,%xmm13+	vaesenclast	%xmm3,%xmm14,%xmm14+	vmovups	%xmm9,0(%rsi)+	vmovups	%xmm10,16(%rsi)+	vmovups	%xmm11,32(%rsi)+	vmovups	%xmm12,48(%rsi)+	vmovups	%xmm13,64(%rsi)+	vmovups	%xmm14,80(%rsi)+	leaq	96(%rsi),%rsi++	.byte	0xf3,0xc3+.align	32+.Lhandle_ctr32_2:+	vpshufb	%xmm0,%xmm1,%xmm6+	vmovdqu	48(%r11),%xmm5+	vpaddd	64(%r11),%xmm6,%xmm10+	vpaddd	%xmm5,%xmm6,%xmm11+	vpaddd	%xmm5,%xmm10,%xmm12+	vpshufb	%xmm0,%xmm10,%xmm10+	vpaddd	%xmm5,%xmm11,%xmm13+	vpshufb	%xmm0,%xmm11,%xmm11+	vpxor	%xmm4,%xmm10,%xmm10+	vpaddd	%xmm5,%xmm12,%xmm14+	vpshufb	%xmm0,%xmm12,%xmm12+	vpxor	%xmm4,%xmm11,%xmm11+	vpaddd	%xmm5,%xmm13,%xmm1+	vpshufb	%xmm0,%xmm13,%xmm13+	vpxor	%xmm4,%xmm12,%xmm12+	vpshufb	%xmm0,%xmm14,%xmm14+	vpxor	%xmm4,%xmm13,%xmm13+	vpshufb	%xmm0,%xmm1,%xmm1+	vpxor	%xmm4,%xmm14,%xmm14+	jmp	.Loop_ctr32+.cfi_endproc	+.size	_crypton_gcm_asm_ctr32_6x,.-_crypton_gcm_asm_ctr32_6x++.globl	crypton_gcm_asm_encrypt+.type	crypton_gcm_asm_encrypt,@function+.align	32+crypton_gcm_asm_encrypt:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+	xorq	%rax,%rax+	cmpq	$288,%rdx+	jb	.Lgcm_enc_abort++	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56++	vzeroupper++	vmovdqu	(%r8),%xmm1+	addq	$-128,%rsp+	movl	12(%r8),%ebx+	leaq	.Lbswap_mask(%rip),%r11+	leaq	-128(%rcx),%r14+	movq	$0xf80,%r15+	leaq	128(%rcx),%rcx+	vmovdqu	(%r11),%xmm0+	andq	$-128,%rsp+	movl	240-128(%rcx),%r10d++	andq	%r15,%r14+	andq	%rsp,%r15+	subq	%r14,%r15+	jc	.Lenc_no_key_aliasing+	cmpq	$768,%r15+	jnc	.Lenc_no_key_aliasing+	subq	%r15,%rsp+.Lenc_no_key_aliasing:++	leaq	(%rsi),%r14+	leaq	-192(%rsi,%rdx,1),%r15+	shrq	$4,%rdx++	call	_crypton_gcm_asm_ctr32_6x+	vpshufb	%xmm0,%xmm9,%xmm8+	vpshufb	%xmm0,%xmm10,%xmm2+	vmovdqu	%xmm8,112(%rsp)+	vpshufb	%xmm0,%xmm11,%xmm4+	vmovdqu	%xmm2,96(%rsp)+	vpshufb	%xmm0,%xmm12,%xmm5+	vmovdqu	%xmm4,80(%rsp)+	vpshufb	%xmm0,%xmm13,%xmm6+	vmovdqu	%xmm5,64(%rsp)+	vpshufb	%xmm0,%xmm14,%xmm7+	vmovdqu	%xmm6,48(%rsp)++	call	_crypton_gcm_asm_ctr32_6x++	vmovdqu	(%r9),%xmm8+	leaq	32+32(%r9),%r9+	subq	$12,%rdx+	movq	$192,%rax+	vpshufb	%xmm0,%xmm8,%xmm8++	call	_crypton_gcm_asm_ctr32_ghash_6x+	vmovdqu	32(%rsp),%xmm7+	vmovdqu	(%r11),%xmm0+	vmovdqu	0-32(%r9),%xmm3+	vpunpckhqdq	%xmm7,%xmm7,%xmm1+	vmovdqu	32-32(%r9),%xmm15+	vmovups	%xmm9,-96(%rsi)+	vpshufb	%xmm0,%xmm9,%xmm9+	vpxor	%xmm7,%xmm1,%xmm1+	vmovups	%xmm10,-80(%rsi)+	vpshufb	%xmm0,%xmm10,%xmm10+	vmovups	%xmm11,-64(%rsi)+	vpshufb	%xmm0,%xmm11,%xmm11+	vmovups	%xmm12,-48(%rsi)+	vpshufb	%xmm0,%xmm12,%xmm12+	vmovups	%xmm13,-32(%rsi)+	vpshufb	%xmm0,%xmm13,%xmm13+	vmovups	%xmm14,-16(%rsi)+	vpshufb	%xmm0,%xmm14,%xmm14+	vmovdqu	%xmm9,16(%rsp)+	vmovdqu	48(%rsp),%xmm6+	vmovdqu	16-32(%r9),%xmm0+	vpunpckhqdq	%xmm6,%xmm6,%xmm2+	vpclmulqdq	$0x00,%xmm3,%xmm7,%xmm5+	vpxor	%xmm6,%xmm2,%xmm2+	vpclmulqdq	$0x11,%xmm3,%xmm7,%xmm7+	vpclmulqdq	$0x00,%xmm15,%xmm1,%xmm1++	vmovdqu	64(%rsp),%xmm9+	vpclmulqdq	$0x00,%xmm0,%xmm6,%xmm4+	vmovdqu	48-32(%r9),%xmm3+	vpxor	%xmm5,%xmm4,%xmm4+	vpunpckhqdq	%xmm9,%xmm9,%xmm5+	vpclmulqdq	$0x11,%xmm0,%xmm6,%xmm6+	vpxor	%xmm9,%xmm5,%xmm5+	vpxor	%xmm7,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm15,%xmm2,%xmm2+	vmovdqu	80-32(%r9),%xmm15+	vpxor	%xmm1,%xmm2,%xmm2++	vmovdqu	80(%rsp),%xmm1+	vpclmulqdq	$0x00,%xmm3,%xmm9,%xmm7+	vmovdqu	64-32(%r9),%xmm0+	vpxor	%xmm4,%xmm7,%xmm7+	vpunpckhqdq	%xmm1,%xmm1,%xmm4+	vpclmulqdq	$0x11,%xmm3,%xmm9,%xmm9+	vpxor	%xmm1,%xmm4,%xmm4+	vpxor	%xmm6,%xmm9,%xmm9+	vpclmulqdq	$0x00,%xmm15,%xmm5,%xmm5+	vpxor	%xmm2,%xmm5,%xmm5++	vmovdqu	96(%rsp),%xmm2+	vpclmulqdq	$0x00,%xmm0,%xmm1,%xmm6+	vmovdqu	96-32(%r9),%xmm3+	vpxor	%xmm7,%xmm6,%xmm6+	vpunpckhqdq	%xmm2,%xmm2,%xmm7+	vpclmulqdq	$0x11,%xmm0,%xmm1,%xmm1+	vpxor	%xmm2,%xmm7,%xmm7+	vpxor	%xmm9,%xmm1,%xmm1+	vpclmulqdq	$0x10,%xmm15,%xmm4,%xmm4+	vmovdqu	128-32(%r9),%xmm15+	vpxor	%xmm5,%xmm4,%xmm4++	vpxor	112(%rsp),%xmm8,%xmm8+	vpclmulqdq	$0x00,%xmm3,%xmm2,%xmm5+	vmovdqu	112-32(%r9),%xmm0+	vpunpckhqdq	%xmm8,%xmm8,%xmm9+	vpxor	%xmm6,%xmm5,%xmm5+	vpclmulqdq	$0x11,%xmm3,%xmm2,%xmm2+	vpxor	%xmm8,%xmm9,%xmm9+	vpxor	%xmm1,%xmm2,%xmm2+	vpclmulqdq	$0x00,%xmm15,%xmm7,%xmm7+	vpxor	%xmm4,%xmm7,%xmm4++	vpclmulqdq	$0x00,%xmm0,%xmm8,%xmm6+	vmovdqu	0-32(%r9),%xmm3+	vpunpckhqdq	%xmm14,%xmm14,%xmm1+	vpclmulqdq	$0x11,%xmm0,%xmm8,%xmm8+	vpxor	%xmm14,%xmm1,%xmm1+	vpxor	%xmm5,%xmm6,%xmm5+	vpclmulqdq	$0x10,%xmm15,%xmm9,%xmm9+	vmovdqu	32-32(%r9),%xmm15+	vpxor	%xmm2,%xmm8,%xmm7+	vpxor	%xmm4,%xmm9,%xmm6++	vmovdqu	16-32(%r9),%xmm0+	vpxor	%xmm5,%xmm7,%xmm9+	vpclmulqdq	$0x00,%xmm3,%xmm14,%xmm4+	vpxor	%xmm9,%xmm6,%xmm6+	vpunpckhqdq	%xmm13,%xmm13,%xmm2+	vpclmulqdq	$0x11,%xmm3,%xmm14,%xmm14+	vpxor	%xmm13,%xmm2,%xmm2+	vpslldq	$8,%xmm6,%xmm9+	vpclmulqdq	$0x00,%xmm15,%xmm1,%xmm1+	vpxor	%xmm9,%xmm5,%xmm8+	vpsrldq	$8,%xmm6,%xmm6+	vpxor	%xmm6,%xmm7,%xmm7++	vpclmulqdq	$0x00,%xmm0,%xmm13,%xmm5+	vmovdqu	48-32(%r9),%xmm3+	vpxor	%xmm4,%xmm5,%xmm5+	vpunpckhqdq	%xmm12,%xmm12,%xmm9+	vpclmulqdq	$0x11,%xmm0,%xmm13,%xmm13+	vpxor	%xmm12,%xmm9,%xmm9+	vpxor	%xmm14,%xmm13,%xmm13+	vpalignr	$8,%xmm8,%xmm8,%xmm14+	vpclmulqdq	$0x10,%xmm15,%xmm2,%xmm2+	vmovdqu	80-32(%r9),%xmm15+	vpxor	%xmm1,%xmm2,%xmm2++	vpclmulqdq	$0x00,%xmm3,%xmm12,%xmm4+	vmovdqu	64-32(%r9),%xmm0+	vpxor	%xmm5,%xmm4,%xmm4+	vpunpckhqdq	%xmm11,%xmm11,%xmm1+	vpclmulqdq	$0x11,%xmm3,%xmm12,%xmm12+	vpxor	%xmm11,%xmm1,%xmm1+	vpxor	%xmm13,%xmm12,%xmm12+	vxorps	16(%rsp),%xmm7,%xmm7+	vpclmulqdq	$0x00,%xmm15,%xmm9,%xmm9+	vpxor	%xmm2,%xmm9,%xmm9++	vpclmulqdq	$0x10,16(%r11),%xmm8,%xmm8+	vxorps	%xmm14,%xmm8,%xmm8++	vpclmulqdq	$0x00,%xmm0,%xmm11,%xmm5+	vmovdqu	96-32(%r9),%xmm3+	vpxor	%xmm4,%xmm5,%xmm5+	vpunpckhqdq	%xmm10,%xmm10,%xmm2+	vpclmulqdq	$0x11,%xmm0,%xmm11,%xmm11+	vpxor	%xmm10,%xmm2,%xmm2+	vpalignr	$8,%xmm8,%xmm8,%xmm14+	vpxor	%xmm12,%xmm11,%xmm11+	vpclmulqdq	$0x10,%xmm15,%xmm1,%xmm1+	vmovdqu	128-32(%r9),%xmm15+	vpxor	%xmm9,%xmm1,%xmm1++	vxorps	%xmm7,%xmm14,%xmm14+	vpclmulqdq	$0x10,16(%r11),%xmm8,%xmm8+	vxorps	%xmm14,%xmm8,%xmm8++	vpclmulqdq	$0x00,%xmm3,%xmm10,%xmm4+	vmovdqu	112-32(%r9),%xmm0+	vpxor	%xmm5,%xmm4,%xmm4+	vpunpckhqdq	%xmm8,%xmm8,%xmm9+	vpclmulqdq	$0x11,%xmm3,%xmm10,%xmm10+	vpxor	%xmm8,%xmm9,%xmm9+	vpxor	%xmm11,%xmm10,%xmm10+	vpclmulqdq	$0x00,%xmm15,%xmm2,%xmm2+	vpxor	%xmm1,%xmm2,%xmm2++	vpclmulqdq	$0x00,%xmm0,%xmm8,%xmm5+	vpclmulqdq	$0x11,%xmm0,%xmm8,%xmm7+	vpxor	%xmm4,%xmm5,%xmm5+	vpclmulqdq	$0x10,%xmm15,%xmm9,%xmm6+	vpxor	%xmm10,%xmm7,%xmm7+	vpxor	%xmm2,%xmm6,%xmm6++	vpxor	%xmm5,%xmm7,%xmm4+	vpxor	%xmm4,%xmm6,%xmm6+	vpslldq	$8,%xmm6,%xmm1+	vmovdqu	16(%r11),%xmm3+	vpsrldq	$8,%xmm6,%xmm6+	vpxor	%xmm1,%xmm5,%xmm8+	vpxor	%xmm6,%xmm7,%xmm7++	vpalignr	$8,%xmm8,%xmm8,%xmm2+	vpclmulqdq	$0x10,%xmm3,%xmm8,%xmm8+	vpxor	%xmm2,%xmm8,%xmm8++	vpalignr	$8,%xmm8,%xmm8,%xmm2+	vpclmulqdq	$0x10,%xmm3,%xmm8,%xmm8+	vpxor	%xmm7,%xmm2,%xmm2+	vpxor	%xmm2,%xmm8,%xmm8+	vpshufb	(%r11),%xmm8,%xmm8+	vmovdqu	%xmm8,-64(%r9)++	vzeroupper+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+.Lgcm_enc_abort:+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_gcm_asm_encrypt,.-crypton_gcm_asm_encrypt+.align	64+.Lbswap_mask:+.byte	15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,0+.Lpoly:+.byte	0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0xc2+.Lone_msb:+.byte	0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1+.Ltwo_lsb:+.byte	2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0+.Lone_lsb:+.byte	1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0+.byte	65,69,83,45,78,73,32,71,67,77,32,109,111,100,117,108,101,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,60,97,112,112,114,111,64,111,112,101,110,115,115,108,46,111,114,103,62,0+.align	64++.section	.note.gnu.property,"a",@note+	.long	4,2f-1f,5+	.byte	0x47,0x4E,0x55,0+1:	.long	0xc0000002,4,3+.align	8+2:++.section	.note.GNU-stack,"",@progbits
+ cbits/asm/aesni-gcm-x86_64-macosx.S view
@@ -0,0 +1,805 @@+.text	+++.p2align	5+_crypton_gcm_asm_ctr32_ghash_6x:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	vmovdqu	32(%r11),%xmm2+	subq	$6,%rdx+	vpxor	%xmm4,%xmm4,%xmm4+	vmovdqu	0-128(%rcx),%xmm15+	vpaddb	%xmm2,%xmm1,%xmm10+	vpaddb	%xmm2,%xmm10,%xmm11+	vpaddb	%xmm2,%xmm11,%xmm12+	vpaddb	%xmm2,%xmm12,%xmm13+	vpaddb	%xmm2,%xmm13,%xmm14+	vpxor	%xmm15,%xmm1,%xmm9+	vmovdqu	%xmm4,16+8(%rsp)+	jmp	L$oop6x++.p2align	5+L$oop6x:+	addl	$100663296,%ebx+	jc	L$handle_ctr32+	vmovdqu	0-32(%r9),%xmm3+	vpaddb	%xmm2,%xmm14,%xmm1+	vpxor	%xmm15,%xmm10,%xmm10+	vpxor	%xmm15,%xmm11,%xmm11++L$resume_ctr32:+	vmovdqu	%xmm1,(%r8)+	vpclmulqdq	$0x10,%xmm3,%xmm7,%xmm5+	vpxor	%xmm15,%xmm12,%xmm12+	vmovups	16-128(%rcx),%xmm2+	vpclmulqdq	$0x01,%xmm3,%xmm7,%xmm6+	xorq	%r12,%r12+	cmpq	%r14,%r15++	vaesenc	%xmm2,%xmm9,%xmm9+	vmovdqu	48+8(%rsp),%xmm0+	vpxor	%xmm15,%xmm13,%xmm13+	vpclmulqdq	$0x00,%xmm3,%xmm7,%xmm1+	vaesenc	%xmm2,%xmm10,%xmm10+	vpxor	%xmm15,%xmm14,%xmm14+	setnc	%r12b+	vpclmulqdq	$0x11,%xmm3,%xmm7,%xmm7+	vaesenc	%xmm2,%xmm11,%xmm11+	vmovdqu	16-32(%r9),%xmm3+	negq	%r12+	vaesenc	%xmm2,%xmm12,%xmm12+	vpxor	%xmm5,%xmm6,%xmm6+	vpclmulqdq	$0x00,%xmm3,%xmm0,%xmm5+	vpxor	%xmm4,%xmm8,%xmm8+	vaesenc	%xmm2,%xmm13,%xmm13+	vpxor	%xmm5,%xmm1,%xmm4+	andq	$0x60,%r12+	vmovups	32-128(%rcx),%xmm15+	vpclmulqdq	$0x10,%xmm3,%xmm0,%xmm1+	vaesenc	%xmm2,%xmm14,%xmm14++	vpclmulqdq	$0x01,%xmm3,%xmm0,%xmm2+	leaq	(%r14,%r12,1),%r14+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	16+8(%rsp),%xmm8,%xmm8+	vpclmulqdq	$0x11,%xmm3,%xmm0,%xmm3+	vmovdqu	64+8(%rsp),%xmm0+	vaesenc	%xmm15,%xmm10,%xmm10+	movbeq	88(%r14),%r13+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	80(%r14),%r12+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r13,32+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	movq	%r12,40+8(%rsp)+	vmovdqu	48-32(%r9),%xmm5+	vaesenc	%xmm15,%xmm14,%xmm14++	vmovups	48-128(%rcx),%xmm15+	vpxor	%xmm1,%xmm6,%xmm6+	vpclmulqdq	$0x00,%xmm5,%xmm0,%xmm1+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm2,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm5,%xmm0,%xmm2+	vaesenc	%xmm15,%xmm10,%xmm10+	vpxor	%xmm3,%xmm7,%xmm7+	vpclmulqdq	$0x01,%xmm5,%xmm0,%xmm3+	vaesenc	%xmm15,%xmm11,%xmm11+	vpclmulqdq	$0x11,%xmm5,%xmm0,%xmm5+	vmovdqu	80+8(%rsp),%xmm0+	vaesenc	%xmm15,%xmm12,%xmm12+	vaesenc	%xmm15,%xmm13,%xmm13+	vpxor	%xmm1,%xmm4,%xmm4+	vmovdqu	64-32(%r9),%xmm1+	vaesenc	%xmm15,%xmm14,%xmm14++	vmovups	64-128(%rcx),%xmm15+	vpxor	%xmm2,%xmm6,%xmm6+	vpclmulqdq	$0x00,%xmm1,%xmm0,%xmm2+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm3,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm1,%xmm0,%xmm3+	vaesenc	%xmm15,%xmm10,%xmm10+	movbeq	72(%r14),%r13+	vpxor	%xmm5,%xmm7,%xmm7+	vpclmulqdq	$0x01,%xmm1,%xmm0,%xmm5+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	64(%r14),%r12+	vpclmulqdq	$0x11,%xmm1,%xmm0,%xmm1+	vmovdqu	96+8(%rsp),%xmm0+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r13,48+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	movq	%r12,56+8(%rsp)+	vpxor	%xmm2,%xmm4,%xmm4+	vmovdqu	96-32(%r9),%xmm2+	vaesenc	%xmm15,%xmm14,%xmm14++	vmovups	80-128(%rcx),%xmm15+	vpxor	%xmm3,%xmm6,%xmm6+	vpclmulqdq	$0x00,%xmm2,%xmm0,%xmm3+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm5,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm2,%xmm0,%xmm5+	vaesenc	%xmm15,%xmm10,%xmm10+	movbeq	56(%r14),%r13+	vpxor	%xmm1,%xmm7,%xmm7+	vpclmulqdq	$0x01,%xmm2,%xmm0,%xmm1+	vpxor	112+8(%rsp),%xmm8,%xmm8+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	48(%r14),%r12+	vpclmulqdq	$0x11,%xmm2,%xmm0,%xmm2+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r13,64+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	movq	%r12,72+8(%rsp)+	vpxor	%xmm3,%xmm4,%xmm4+	vmovdqu	112-32(%r9),%xmm3+	vaesenc	%xmm15,%xmm14,%xmm14++	vmovups	96-128(%rcx),%xmm15+	vpxor	%xmm5,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm3,%xmm8,%xmm5+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm1,%xmm6,%xmm6+	vpclmulqdq	$0x01,%xmm3,%xmm8,%xmm1+	vaesenc	%xmm15,%xmm10,%xmm10+	movbeq	40(%r14),%r13+	vpxor	%xmm2,%xmm7,%xmm7+	vpclmulqdq	$0x00,%xmm3,%xmm8,%xmm2+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	32(%r14),%r12+	vpclmulqdq	$0x11,%xmm3,%xmm8,%xmm8+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r13,80+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	movq	%r12,88+8(%rsp)+	vpxor	%xmm5,%xmm6,%xmm6+	vaesenc	%xmm15,%xmm14,%xmm14+	vpxor	%xmm1,%xmm6,%xmm6++	vmovups	112-128(%rcx),%xmm15+	vpslldq	$8,%xmm6,%xmm5+	vpxor	%xmm2,%xmm4,%xmm4+	vmovdqu	16(%r11),%xmm3++	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm8,%xmm7,%xmm7+	vaesenc	%xmm15,%xmm10,%xmm10+	vpxor	%xmm5,%xmm4,%xmm4+	movbeq	24(%r14),%r13+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	16(%r14),%r12+	vpalignr	$8,%xmm4,%xmm4,%xmm0+	vpclmulqdq	$0x10,%xmm3,%xmm4,%xmm4+	movq	%r13,96+8(%rsp)+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r12,104+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	vmovups	128-128(%rcx),%xmm1+	vaesenc	%xmm15,%xmm14,%xmm14++	vaesenc	%xmm1,%xmm9,%xmm9+	vmovups	144-128(%rcx),%xmm15+	vaesenc	%xmm1,%xmm10,%xmm10+	vpsrldq	$8,%xmm6,%xmm6+	vaesenc	%xmm1,%xmm11,%xmm11+	vpxor	%xmm6,%xmm7,%xmm7+	vaesenc	%xmm1,%xmm12,%xmm12+	vpxor	%xmm0,%xmm4,%xmm4+	movbeq	8(%r14),%r13+	vaesenc	%xmm1,%xmm13,%xmm13+	movbeq	0(%r14),%r12+	vaesenc	%xmm1,%xmm14,%xmm14+	vmovups	160-128(%rcx),%xmm1+	cmpl	$11,%r10d+	jb	L$enc_tail++	vaesenc	%xmm15,%xmm9,%xmm9+	vaesenc	%xmm15,%xmm10,%xmm10+	vaesenc	%xmm15,%xmm11,%xmm11+	vaesenc	%xmm15,%xmm12,%xmm12+	vaesenc	%xmm15,%xmm13,%xmm13+	vaesenc	%xmm15,%xmm14,%xmm14++	vaesenc	%xmm1,%xmm9,%xmm9+	vaesenc	%xmm1,%xmm10,%xmm10+	vaesenc	%xmm1,%xmm11,%xmm11+	vaesenc	%xmm1,%xmm12,%xmm12+	vaesenc	%xmm1,%xmm13,%xmm13+	vmovups	176-128(%rcx),%xmm15+	vaesenc	%xmm1,%xmm14,%xmm14+	vmovups	192-128(%rcx),%xmm1+	je	L$enc_tail++	vaesenc	%xmm15,%xmm9,%xmm9+	vaesenc	%xmm15,%xmm10,%xmm10+	vaesenc	%xmm15,%xmm11,%xmm11+	vaesenc	%xmm15,%xmm12,%xmm12+	vaesenc	%xmm15,%xmm13,%xmm13+	vaesenc	%xmm15,%xmm14,%xmm14++	vaesenc	%xmm1,%xmm9,%xmm9+	vaesenc	%xmm1,%xmm10,%xmm10+	vaesenc	%xmm1,%xmm11,%xmm11+	vaesenc	%xmm1,%xmm12,%xmm12+	vaesenc	%xmm1,%xmm13,%xmm13+	vmovups	208-128(%rcx),%xmm15+	vaesenc	%xmm1,%xmm14,%xmm14+	vmovups	224-128(%rcx),%xmm1+	jmp	L$enc_tail++.p2align	5+L$handle_ctr32:+	vmovdqu	(%r11),%xmm0+	vpshufb	%xmm0,%xmm1,%xmm6+	vmovdqu	48(%r11),%xmm5+	vpaddd	64(%r11),%xmm6,%xmm10+	vpaddd	%xmm5,%xmm6,%xmm11+	vmovdqu	0-32(%r9),%xmm3+	vpaddd	%xmm5,%xmm10,%xmm12+	vpshufb	%xmm0,%xmm10,%xmm10+	vpaddd	%xmm5,%xmm11,%xmm13+	vpshufb	%xmm0,%xmm11,%xmm11+	vpxor	%xmm15,%xmm10,%xmm10+	vpaddd	%xmm5,%xmm12,%xmm14+	vpshufb	%xmm0,%xmm12,%xmm12+	vpxor	%xmm15,%xmm11,%xmm11+	vpaddd	%xmm5,%xmm13,%xmm1+	vpshufb	%xmm0,%xmm13,%xmm13+	vpshufb	%xmm0,%xmm14,%xmm14+	vpshufb	%xmm0,%xmm1,%xmm1+	jmp	L$resume_ctr32++.p2align	5+L$enc_tail:+	vaesenc	%xmm15,%xmm9,%xmm9+	vmovdqu	%xmm7,16+8(%rsp)+	vpalignr	$8,%xmm4,%xmm4,%xmm8+	vaesenc	%xmm15,%xmm10,%xmm10+	vpclmulqdq	$0x10,%xmm3,%xmm4,%xmm4+	vpxor	0(%rdi),%xmm1,%xmm2+	vaesenc	%xmm15,%xmm11,%xmm11+	vpxor	16(%rdi),%xmm1,%xmm0+	vaesenc	%xmm15,%xmm12,%xmm12+	vpxor	32(%rdi),%xmm1,%xmm5+	vaesenc	%xmm15,%xmm13,%xmm13+	vpxor	48(%rdi),%xmm1,%xmm6+	vaesenc	%xmm15,%xmm14,%xmm14+	vpxor	64(%rdi),%xmm1,%xmm7+	vpxor	80(%rdi),%xmm1,%xmm3+	vmovdqu	(%r8),%xmm1++	vaesenclast	%xmm2,%xmm9,%xmm9+	vmovdqu	32(%r11),%xmm2+	vaesenclast	%xmm0,%xmm10,%xmm10+	vpaddb	%xmm2,%xmm1,%xmm0+	movq	%r13,112+8(%rsp)+	leaq	96(%rdi),%rdi+	vaesenclast	%xmm5,%xmm11,%xmm11+	vpaddb	%xmm2,%xmm0,%xmm5+	movq	%r12,120+8(%rsp)+	leaq	96(%rsi),%rsi+	vmovdqu	0-128(%rcx),%xmm15+	vaesenclast	%xmm6,%xmm12,%xmm12+	vpaddb	%xmm2,%xmm5,%xmm6+	vaesenclast	%xmm7,%xmm13,%xmm13+	vpaddb	%xmm2,%xmm6,%xmm7+	vaesenclast	%xmm3,%xmm14,%xmm14+	vpaddb	%xmm2,%xmm7,%xmm3++	addq	$0x60,%rax+	subq	$0x6,%rdx+	jc	L$6x_done++	vmovups	%xmm9,-96(%rsi)+	vpxor	%xmm15,%xmm1,%xmm9+	vmovups	%xmm10,-80(%rsi)+	vmovdqa	%xmm0,%xmm10+	vmovups	%xmm11,-64(%rsi)+	vmovdqa	%xmm5,%xmm11+	vmovups	%xmm12,-48(%rsi)+	vmovdqa	%xmm6,%xmm12+	vmovups	%xmm13,-32(%rsi)+	vmovdqa	%xmm7,%xmm13+	vmovups	%xmm14,-16(%rsi)+	vmovdqa	%xmm3,%xmm14+	vmovdqu	32+8(%rsp),%xmm7+	jmp	L$oop6x++L$6x_done:+	vpxor	16+8(%rsp),%xmm8,%xmm8+	vpxor	%xmm4,%xmm8,%xmm8++	.byte	0xf3,0xc3+.cfi_endproc	++.globl	_crypton_gcm_asm_decrypt++.p2align	5+_crypton_gcm_asm_decrypt:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+	xorq	%rax,%rax+	cmpq	$0x60,%rdx+	jb	L$gcm_dec_abort++	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56++	vzeroupper++	vmovdqu	(%r8),%xmm1+	addq	$-128,%rsp+	movl	12(%r8),%ebx+	leaq	L$bswap_mask(%rip),%r11+	leaq	-128(%rcx),%r14+	movq	$0xf80,%r15+	vmovdqu	(%r9),%xmm8+	andq	$-128,%rsp+	vmovdqu	(%r11),%xmm0+	leaq	128(%rcx),%rcx+	leaq	32+32(%r9),%r9+	movl	240-128(%rcx),%r10d+	vpshufb	%xmm0,%xmm8,%xmm8++	andq	%r15,%r14+	andq	%rsp,%r15+	subq	%r14,%r15+	jc	L$dec_no_key_aliasing+	cmpq	$768,%r15+	jnc	L$dec_no_key_aliasing+	subq	%r15,%rsp+L$dec_no_key_aliasing:++	vmovdqu	80(%rdi),%xmm7+	leaq	(%rdi),%r14+	vmovdqu	64(%rdi),%xmm4+	leaq	-192(%rdi,%rdx,1),%r15+	vmovdqu	48(%rdi),%xmm5+	shrq	$4,%rdx+	xorq	%rax,%rax+	vmovdqu	32(%rdi),%xmm6+	vpshufb	%xmm0,%xmm7,%xmm7+	vmovdqu	16(%rdi),%xmm2+	vpshufb	%xmm0,%xmm4,%xmm4+	vmovdqu	(%rdi),%xmm3+	vpshufb	%xmm0,%xmm5,%xmm5+	vmovdqu	%xmm4,48(%rsp)+	vpshufb	%xmm0,%xmm6,%xmm6+	vmovdqu	%xmm5,64(%rsp)+	vpshufb	%xmm0,%xmm2,%xmm2+	vmovdqu	%xmm6,80(%rsp)+	vpshufb	%xmm0,%xmm3,%xmm3+	vmovdqu	%xmm2,96(%rsp)+	vmovdqu	%xmm3,112(%rsp)++	call	_crypton_gcm_asm_ctr32_ghash_6x++	vmovups	%xmm9,-96(%rsi)+	vmovups	%xmm10,-80(%rsi)+	vmovups	%xmm11,-64(%rsi)+	vmovups	%xmm12,-48(%rsi)+	vmovups	%xmm13,-32(%rsi)+	vmovups	%xmm14,-16(%rsi)++	vpshufb	(%r11),%xmm8,%xmm8+	vmovdqu	%xmm8,-64(%r9)++	vzeroupper+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+L$gcm_dec_abort:+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+++.p2align	5+_crypton_gcm_asm_ctr32_6x:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	vmovdqu	0-128(%rcx),%xmm4+	vmovdqu	32(%r11),%xmm2+	leaq	-1(%r10),%r13+	vmovups	16-128(%rcx),%xmm15+	leaq	32-128(%rcx),%r12+	vpxor	%xmm4,%xmm1,%xmm9+	addl	$100663296,%ebx+	jc	L$handle_ctr32_2+	vpaddb	%xmm2,%xmm1,%xmm10+	vpaddb	%xmm2,%xmm10,%xmm11+	vpxor	%xmm4,%xmm10,%xmm10+	vpaddb	%xmm2,%xmm11,%xmm12+	vpxor	%xmm4,%xmm11,%xmm11+	vpaddb	%xmm2,%xmm12,%xmm13+	vpxor	%xmm4,%xmm12,%xmm12+	vpaddb	%xmm2,%xmm13,%xmm14+	vpxor	%xmm4,%xmm13,%xmm13+	vpaddb	%xmm2,%xmm14,%xmm1+	vpxor	%xmm4,%xmm14,%xmm14+	jmp	L$oop_ctr32++.p2align	4+L$oop_ctr32:+	vaesenc	%xmm15,%xmm9,%xmm9+	vaesenc	%xmm15,%xmm10,%xmm10+	vaesenc	%xmm15,%xmm11,%xmm11+	vaesenc	%xmm15,%xmm12,%xmm12+	vaesenc	%xmm15,%xmm13,%xmm13+	vaesenc	%xmm15,%xmm14,%xmm14+	vmovups	(%r12),%xmm15+	leaq	16(%r12),%r12+	decl	%r13d+	jnz	L$oop_ctr32++	vmovdqu	(%r12),%xmm3+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	0(%rdi),%xmm3,%xmm4+	vaesenc	%xmm15,%xmm10,%xmm10+	vpxor	16(%rdi),%xmm3,%xmm5+	vaesenc	%xmm15,%xmm11,%xmm11+	vpxor	32(%rdi),%xmm3,%xmm6+	vaesenc	%xmm15,%xmm12,%xmm12+	vpxor	48(%rdi),%xmm3,%xmm8+	vaesenc	%xmm15,%xmm13,%xmm13+	vpxor	64(%rdi),%xmm3,%xmm2+	vaesenc	%xmm15,%xmm14,%xmm14+	vpxor	80(%rdi),%xmm3,%xmm3+	leaq	96(%rdi),%rdi++	vaesenclast	%xmm4,%xmm9,%xmm9+	vaesenclast	%xmm5,%xmm10,%xmm10+	vaesenclast	%xmm6,%xmm11,%xmm11+	vaesenclast	%xmm8,%xmm12,%xmm12+	vaesenclast	%xmm2,%xmm13,%xmm13+	vaesenclast	%xmm3,%xmm14,%xmm14+	vmovups	%xmm9,0(%rsi)+	vmovups	%xmm10,16(%rsi)+	vmovups	%xmm11,32(%rsi)+	vmovups	%xmm12,48(%rsi)+	vmovups	%xmm13,64(%rsi)+	vmovups	%xmm14,80(%rsi)+	leaq	96(%rsi),%rsi++	.byte	0xf3,0xc3+.p2align	5+L$handle_ctr32_2:+	vpshufb	%xmm0,%xmm1,%xmm6+	vmovdqu	48(%r11),%xmm5+	vpaddd	64(%r11),%xmm6,%xmm10+	vpaddd	%xmm5,%xmm6,%xmm11+	vpaddd	%xmm5,%xmm10,%xmm12+	vpshufb	%xmm0,%xmm10,%xmm10+	vpaddd	%xmm5,%xmm11,%xmm13+	vpshufb	%xmm0,%xmm11,%xmm11+	vpxor	%xmm4,%xmm10,%xmm10+	vpaddd	%xmm5,%xmm12,%xmm14+	vpshufb	%xmm0,%xmm12,%xmm12+	vpxor	%xmm4,%xmm11,%xmm11+	vpaddd	%xmm5,%xmm13,%xmm1+	vpshufb	%xmm0,%xmm13,%xmm13+	vpxor	%xmm4,%xmm12,%xmm12+	vpshufb	%xmm0,%xmm14,%xmm14+	vpxor	%xmm4,%xmm13,%xmm13+	vpshufb	%xmm0,%xmm1,%xmm1+	vpxor	%xmm4,%xmm14,%xmm14+	jmp	L$oop_ctr32+.cfi_endproc	+++.globl	_crypton_gcm_asm_encrypt++.p2align	5+_crypton_gcm_asm_encrypt:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+	xorq	%rax,%rax+	cmpq	$288,%rdx+	jb	L$gcm_enc_abort++	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56++	vzeroupper++	vmovdqu	(%r8),%xmm1+	addq	$-128,%rsp+	movl	12(%r8),%ebx+	leaq	L$bswap_mask(%rip),%r11+	leaq	-128(%rcx),%r14+	movq	$0xf80,%r15+	leaq	128(%rcx),%rcx+	vmovdqu	(%r11),%xmm0+	andq	$-128,%rsp+	movl	240-128(%rcx),%r10d++	andq	%r15,%r14+	andq	%rsp,%r15+	subq	%r14,%r15+	jc	L$enc_no_key_aliasing+	cmpq	$768,%r15+	jnc	L$enc_no_key_aliasing+	subq	%r15,%rsp+L$enc_no_key_aliasing:++	leaq	(%rsi),%r14+	leaq	-192(%rsi,%rdx,1),%r15+	shrq	$4,%rdx++	call	_crypton_gcm_asm_ctr32_6x+	vpshufb	%xmm0,%xmm9,%xmm8+	vpshufb	%xmm0,%xmm10,%xmm2+	vmovdqu	%xmm8,112(%rsp)+	vpshufb	%xmm0,%xmm11,%xmm4+	vmovdqu	%xmm2,96(%rsp)+	vpshufb	%xmm0,%xmm12,%xmm5+	vmovdqu	%xmm4,80(%rsp)+	vpshufb	%xmm0,%xmm13,%xmm6+	vmovdqu	%xmm5,64(%rsp)+	vpshufb	%xmm0,%xmm14,%xmm7+	vmovdqu	%xmm6,48(%rsp)++	call	_crypton_gcm_asm_ctr32_6x++	vmovdqu	(%r9),%xmm8+	leaq	32+32(%r9),%r9+	subq	$12,%rdx+	movq	$192,%rax+	vpshufb	%xmm0,%xmm8,%xmm8++	call	_crypton_gcm_asm_ctr32_ghash_6x+	vmovdqu	32(%rsp),%xmm7+	vmovdqu	(%r11),%xmm0+	vmovdqu	0-32(%r9),%xmm3+	vpunpckhqdq	%xmm7,%xmm7,%xmm1+	vmovdqu	32-32(%r9),%xmm15+	vmovups	%xmm9,-96(%rsi)+	vpshufb	%xmm0,%xmm9,%xmm9+	vpxor	%xmm7,%xmm1,%xmm1+	vmovups	%xmm10,-80(%rsi)+	vpshufb	%xmm0,%xmm10,%xmm10+	vmovups	%xmm11,-64(%rsi)+	vpshufb	%xmm0,%xmm11,%xmm11+	vmovups	%xmm12,-48(%rsi)+	vpshufb	%xmm0,%xmm12,%xmm12+	vmovups	%xmm13,-32(%rsi)+	vpshufb	%xmm0,%xmm13,%xmm13+	vmovups	%xmm14,-16(%rsi)+	vpshufb	%xmm0,%xmm14,%xmm14+	vmovdqu	%xmm9,16(%rsp)+	vmovdqu	48(%rsp),%xmm6+	vmovdqu	16-32(%r9),%xmm0+	vpunpckhqdq	%xmm6,%xmm6,%xmm2+	vpclmulqdq	$0x00,%xmm3,%xmm7,%xmm5+	vpxor	%xmm6,%xmm2,%xmm2+	vpclmulqdq	$0x11,%xmm3,%xmm7,%xmm7+	vpclmulqdq	$0x00,%xmm15,%xmm1,%xmm1++	vmovdqu	64(%rsp),%xmm9+	vpclmulqdq	$0x00,%xmm0,%xmm6,%xmm4+	vmovdqu	48-32(%r9),%xmm3+	vpxor	%xmm5,%xmm4,%xmm4+	vpunpckhqdq	%xmm9,%xmm9,%xmm5+	vpclmulqdq	$0x11,%xmm0,%xmm6,%xmm6+	vpxor	%xmm9,%xmm5,%xmm5+	vpxor	%xmm7,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm15,%xmm2,%xmm2+	vmovdqu	80-32(%r9),%xmm15+	vpxor	%xmm1,%xmm2,%xmm2++	vmovdqu	80(%rsp),%xmm1+	vpclmulqdq	$0x00,%xmm3,%xmm9,%xmm7+	vmovdqu	64-32(%r9),%xmm0+	vpxor	%xmm4,%xmm7,%xmm7+	vpunpckhqdq	%xmm1,%xmm1,%xmm4+	vpclmulqdq	$0x11,%xmm3,%xmm9,%xmm9+	vpxor	%xmm1,%xmm4,%xmm4+	vpxor	%xmm6,%xmm9,%xmm9+	vpclmulqdq	$0x00,%xmm15,%xmm5,%xmm5+	vpxor	%xmm2,%xmm5,%xmm5++	vmovdqu	96(%rsp),%xmm2+	vpclmulqdq	$0x00,%xmm0,%xmm1,%xmm6+	vmovdqu	96-32(%r9),%xmm3+	vpxor	%xmm7,%xmm6,%xmm6+	vpunpckhqdq	%xmm2,%xmm2,%xmm7+	vpclmulqdq	$0x11,%xmm0,%xmm1,%xmm1+	vpxor	%xmm2,%xmm7,%xmm7+	vpxor	%xmm9,%xmm1,%xmm1+	vpclmulqdq	$0x10,%xmm15,%xmm4,%xmm4+	vmovdqu	128-32(%r9),%xmm15+	vpxor	%xmm5,%xmm4,%xmm4++	vpxor	112(%rsp),%xmm8,%xmm8+	vpclmulqdq	$0x00,%xmm3,%xmm2,%xmm5+	vmovdqu	112-32(%r9),%xmm0+	vpunpckhqdq	%xmm8,%xmm8,%xmm9+	vpxor	%xmm6,%xmm5,%xmm5+	vpclmulqdq	$0x11,%xmm3,%xmm2,%xmm2+	vpxor	%xmm8,%xmm9,%xmm9+	vpxor	%xmm1,%xmm2,%xmm2+	vpclmulqdq	$0x00,%xmm15,%xmm7,%xmm7+	vpxor	%xmm4,%xmm7,%xmm4++	vpclmulqdq	$0x00,%xmm0,%xmm8,%xmm6+	vmovdqu	0-32(%r9),%xmm3+	vpunpckhqdq	%xmm14,%xmm14,%xmm1+	vpclmulqdq	$0x11,%xmm0,%xmm8,%xmm8+	vpxor	%xmm14,%xmm1,%xmm1+	vpxor	%xmm5,%xmm6,%xmm5+	vpclmulqdq	$0x10,%xmm15,%xmm9,%xmm9+	vmovdqu	32-32(%r9),%xmm15+	vpxor	%xmm2,%xmm8,%xmm7+	vpxor	%xmm4,%xmm9,%xmm6++	vmovdqu	16-32(%r9),%xmm0+	vpxor	%xmm5,%xmm7,%xmm9+	vpclmulqdq	$0x00,%xmm3,%xmm14,%xmm4+	vpxor	%xmm9,%xmm6,%xmm6+	vpunpckhqdq	%xmm13,%xmm13,%xmm2+	vpclmulqdq	$0x11,%xmm3,%xmm14,%xmm14+	vpxor	%xmm13,%xmm2,%xmm2+	vpslldq	$8,%xmm6,%xmm9+	vpclmulqdq	$0x00,%xmm15,%xmm1,%xmm1+	vpxor	%xmm9,%xmm5,%xmm8+	vpsrldq	$8,%xmm6,%xmm6+	vpxor	%xmm6,%xmm7,%xmm7++	vpclmulqdq	$0x00,%xmm0,%xmm13,%xmm5+	vmovdqu	48-32(%r9),%xmm3+	vpxor	%xmm4,%xmm5,%xmm5+	vpunpckhqdq	%xmm12,%xmm12,%xmm9+	vpclmulqdq	$0x11,%xmm0,%xmm13,%xmm13+	vpxor	%xmm12,%xmm9,%xmm9+	vpxor	%xmm14,%xmm13,%xmm13+	vpalignr	$8,%xmm8,%xmm8,%xmm14+	vpclmulqdq	$0x10,%xmm15,%xmm2,%xmm2+	vmovdqu	80-32(%r9),%xmm15+	vpxor	%xmm1,%xmm2,%xmm2++	vpclmulqdq	$0x00,%xmm3,%xmm12,%xmm4+	vmovdqu	64-32(%r9),%xmm0+	vpxor	%xmm5,%xmm4,%xmm4+	vpunpckhqdq	%xmm11,%xmm11,%xmm1+	vpclmulqdq	$0x11,%xmm3,%xmm12,%xmm12+	vpxor	%xmm11,%xmm1,%xmm1+	vpxor	%xmm13,%xmm12,%xmm12+	vxorps	16(%rsp),%xmm7,%xmm7+	vpclmulqdq	$0x00,%xmm15,%xmm9,%xmm9+	vpxor	%xmm2,%xmm9,%xmm9++	vpclmulqdq	$0x10,16(%r11),%xmm8,%xmm8+	vxorps	%xmm14,%xmm8,%xmm8++	vpclmulqdq	$0x00,%xmm0,%xmm11,%xmm5+	vmovdqu	96-32(%r9),%xmm3+	vpxor	%xmm4,%xmm5,%xmm5+	vpunpckhqdq	%xmm10,%xmm10,%xmm2+	vpclmulqdq	$0x11,%xmm0,%xmm11,%xmm11+	vpxor	%xmm10,%xmm2,%xmm2+	vpalignr	$8,%xmm8,%xmm8,%xmm14+	vpxor	%xmm12,%xmm11,%xmm11+	vpclmulqdq	$0x10,%xmm15,%xmm1,%xmm1+	vmovdqu	128-32(%r9),%xmm15+	vpxor	%xmm9,%xmm1,%xmm1++	vxorps	%xmm7,%xmm14,%xmm14+	vpclmulqdq	$0x10,16(%r11),%xmm8,%xmm8+	vxorps	%xmm14,%xmm8,%xmm8++	vpclmulqdq	$0x00,%xmm3,%xmm10,%xmm4+	vmovdqu	112-32(%r9),%xmm0+	vpxor	%xmm5,%xmm4,%xmm4+	vpunpckhqdq	%xmm8,%xmm8,%xmm9+	vpclmulqdq	$0x11,%xmm3,%xmm10,%xmm10+	vpxor	%xmm8,%xmm9,%xmm9+	vpxor	%xmm11,%xmm10,%xmm10+	vpclmulqdq	$0x00,%xmm15,%xmm2,%xmm2+	vpxor	%xmm1,%xmm2,%xmm2++	vpclmulqdq	$0x00,%xmm0,%xmm8,%xmm5+	vpclmulqdq	$0x11,%xmm0,%xmm8,%xmm7+	vpxor	%xmm4,%xmm5,%xmm5+	vpclmulqdq	$0x10,%xmm15,%xmm9,%xmm6+	vpxor	%xmm10,%xmm7,%xmm7+	vpxor	%xmm2,%xmm6,%xmm6++	vpxor	%xmm5,%xmm7,%xmm4+	vpxor	%xmm4,%xmm6,%xmm6+	vpslldq	$8,%xmm6,%xmm1+	vmovdqu	16(%r11),%xmm3+	vpsrldq	$8,%xmm6,%xmm6+	vpxor	%xmm1,%xmm5,%xmm8+	vpxor	%xmm6,%xmm7,%xmm7++	vpalignr	$8,%xmm8,%xmm8,%xmm2+	vpclmulqdq	$0x10,%xmm3,%xmm8,%xmm8+	vpxor	%xmm2,%xmm8,%xmm8++	vpalignr	$8,%xmm8,%xmm8,%xmm2+	vpclmulqdq	$0x10,%xmm3,%xmm8,%xmm8+	vpxor	%xmm7,%xmm2,%xmm2+	vpxor	%xmm2,%xmm8,%xmm8+	vpshufb	(%r11),%xmm8,%xmm8+	vmovdqu	%xmm8,-64(%r9)++	vzeroupper+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+L$gcm_enc_abort:+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	++.p2align	6+L$bswap_mask:+.byte	15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,0+L$poly:+.byte	0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0xc2+L$one_msb:+.byte	0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1+L$two_lsb:+.byte	2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0+L$one_lsb:+.byte	1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0+.byte	65,69,83,45,78,73,32,71,67,77,32,109,111,100,117,108,101,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,60,97,112,112,114,111,64,111,112,101,110,115,115,108,46,111,114,103,62,0+.p2align	6
+ cbits/asm/aesni-gcm-x86_64-mingw64.S view
@@ -0,0 +1,965 @@+.text	++.def	_crypton_gcm_asm_ctr32_ghash_6x;	.scl 3;	.type 32;	.endef+.p2align	5+_crypton_gcm_asm_ctr32_ghash_6x:+	.byte	0xf3,0x0f,0x1e,0xfa+++	vmovdqu	32(%r11),%xmm2+	subq	$6,%rdx+	vpxor	%xmm4,%xmm4,%xmm4+	vmovdqu	0-128(%rcx),%xmm15+	vpaddb	%xmm2,%xmm1,%xmm10+	vpaddb	%xmm2,%xmm10,%xmm11+	vpaddb	%xmm2,%xmm11,%xmm12+	vpaddb	%xmm2,%xmm12,%xmm13+	vpaddb	%xmm2,%xmm13,%xmm14+	vpxor	%xmm15,%xmm1,%xmm9+	vmovdqu	%xmm4,16+8(%rsp)+	jmp	.Loop6x++.p2align	5+.Loop6x:+	addl	$100663296,%ebx+	jc	.Lhandle_ctr32+	vmovdqu	0-32(%r9),%xmm3+	vpaddb	%xmm2,%xmm14,%xmm1+	vpxor	%xmm15,%xmm10,%xmm10+	vpxor	%xmm15,%xmm11,%xmm11++.Lresume_ctr32:+	vmovdqu	%xmm1,(%r8)+	vpclmulqdq	$0x10,%xmm3,%xmm7,%xmm5+	vpxor	%xmm15,%xmm12,%xmm12+	vmovups	16-128(%rcx),%xmm2+	vpclmulqdq	$0x01,%xmm3,%xmm7,%xmm6+	xorq	%r12,%r12+	cmpq	%r14,%r15++	vaesenc	%xmm2,%xmm9,%xmm9+	vmovdqu	48+8(%rsp),%xmm0+	vpxor	%xmm15,%xmm13,%xmm13+	vpclmulqdq	$0x00,%xmm3,%xmm7,%xmm1+	vaesenc	%xmm2,%xmm10,%xmm10+	vpxor	%xmm15,%xmm14,%xmm14+	setnc	%r12b+	vpclmulqdq	$0x11,%xmm3,%xmm7,%xmm7+	vaesenc	%xmm2,%xmm11,%xmm11+	vmovdqu	16-32(%r9),%xmm3+	negq	%r12+	vaesenc	%xmm2,%xmm12,%xmm12+	vpxor	%xmm5,%xmm6,%xmm6+	vpclmulqdq	$0x00,%xmm3,%xmm0,%xmm5+	vpxor	%xmm4,%xmm8,%xmm8+	vaesenc	%xmm2,%xmm13,%xmm13+	vpxor	%xmm5,%xmm1,%xmm4+	andq	$0x60,%r12+	vmovups	32-128(%rcx),%xmm15+	vpclmulqdq	$0x10,%xmm3,%xmm0,%xmm1+	vaesenc	%xmm2,%xmm14,%xmm14++	vpclmulqdq	$0x01,%xmm3,%xmm0,%xmm2+	leaq	(%r14,%r12,1),%r14+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	16+8(%rsp),%xmm8,%xmm8+	vpclmulqdq	$0x11,%xmm3,%xmm0,%xmm3+	vmovdqu	64+8(%rsp),%xmm0+	vaesenc	%xmm15,%xmm10,%xmm10+	movbeq	88(%r14),%r13+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	80(%r14),%r12+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r13,32+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	movq	%r12,40+8(%rsp)+	vmovdqu	48-32(%r9),%xmm5+	vaesenc	%xmm15,%xmm14,%xmm14++	vmovups	48-128(%rcx),%xmm15+	vpxor	%xmm1,%xmm6,%xmm6+	vpclmulqdq	$0x00,%xmm5,%xmm0,%xmm1+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm2,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm5,%xmm0,%xmm2+	vaesenc	%xmm15,%xmm10,%xmm10+	vpxor	%xmm3,%xmm7,%xmm7+	vpclmulqdq	$0x01,%xmm5,%xmm0,%xmm3+	vaesenc	%xmm15,%xmm11,%xmm11+	vpclmulqdq	$0x11,%xmm5,%xmm0,%xmm5+	vmovdqu	80+8(%rsp),%xmm0+	vaesenc	%xmm15,%xmm12,%xmm12+	vaesenc	%xmm15,%xmm13,%xmm13+	vpxor	%xmm1,%xmm4,%xmm4+	vmovdqu	64-32(%r9),%xmm1+	vaesenc	%xmm15,%xmm14,%xmm14++	vmovups	64-128(%rcx),%xmm15+	vpxor	%xmm2,%xmm6,%xmm6+	vpclmulqdq	$0x00,%xmm1,%xmm0,%xmm2+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm3,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm1,%xmm0,%xmm3+	vaesenc	%xmm15,%xmm10,%xmm10+	movbeq	72(%r14),%r13+	vpxor	%xmm5,%xmm7,%xmm7+	vpclmulqdq	$0x01,%xmm1,%xmm0,%xmm5+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	64(%r14),%r12+	vpclmulqdq	$0x11,%xmm1,%xmm0,%xmm1+	vmovdqu	96+8(%rsp),%xmm0+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r13,48+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	movq	%r12,56+8(%rsp)+	vpxor	%xmm2,%xmm4,%xmm4+	vmovdqu	96-32(%r9),%xmm2+	vaesenc	%xmm15,%xmm14,%xmm14++	vmovups	80-128(%rcx),%xmm15+	vpxor	%xmm3,%xmm6,%xmm6+	vpclmulqdq	$0x00,%xmm2,%xmm0,%xmm3+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm5,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm2,%xmm0,%xmm5+	vaesenc	%xmm15,%xmm10,%xmm10+	movbeq	56(%r14),%r13+	vpxor	%xmm1,%xmm7,%xmm7+	vpclmulqdq	$0x01,%xmm2,%xmm0,%xmm1+	vpxor	112+8(%rsp),%xmm8,%xmm8+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	48(%r14),%r12+	vpclmulqdq	$0x11,%xmm2,%xmm0,%xmm2+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r13,64+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	movq	%r12,72+8(%rsp)+	vpxor	%xmm3,%xmm4,%xmm4+	vmovdqu	112-32(%r9),%xmm3+	vaesenc	%xmm15,%xmm14,%xmm14++	vmovups	96-128(%rcx),%xmm15+	vpxor	%xmm5,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm3,%xmm8,%xmm5+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm1,%xmm6,%xmm6+	vpclmulqdq	$0x01,%xmm3,%xmm8,%xmm1+	vaesenc	%xmm15,%xmm10,%xmm10+	movbeq	40(%r14),%r13+	vpxor	%xmm2,%xmm7,%xmm7+	vpclmulqdq	$0x00,%xmm3,%xmm8,%xmm2+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	32(%r14),%r12+	vpclmulqdq	$0x11,%xmm3,%xmm8,%xmm8+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r13,80+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	movq	%r12,88+8(%rsp)+	vpxor	%xmm5,%xmm6,%xmm6+	vaesenc	%xmm15,%xmm14,%xmm14+	vpxor	%xmm1,%xmm6,%xmm6++	vmovups	112-128(%rcx),%xmm15+	vpslldq	$8,%xmm6,%xmm5+	vpxor	%xmm2,%xmm4,%xmm4+	vmovdqu	16(%r11),%xmm3++	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	%xmm8,%xmm7,%xmm7+	vaesenc	%xmm15,%xmm10,%xmm10+	vpxor	%xmm5,%xmm4,%xmm4+	movbeq	24(%r14),%r13+	vaesenc	%xmm15,%xmm11,%xmm11+	movbeq	16(%r14),%r12+	vpalignr	$8,%xmm4,%xmm4,%xmm0+	vpclmulqdq	$0x10,%xmm3,%xmm4,%xmm4+	movq	%r13,96+8(%rsp)+	vaesenc	%xmm15,%xmm12,%xmm12+	movq	%r12,104+8(%rsp)+	vaesenc	%xmm15,%xmm13,%xmm13+	vmovups	128-128(%rcx),%xmm1+	vaesenc	%xmm15,%xmm14,%xmm14++	vaesenc	%xmm1,%xmm9,%xmm9+	vmovups	144-128(%rcx),%xmm15+	vaesenc	%xmm1,%xmm10,%xmm10+	vpsrldq	$8,%xmm6,%xmm6+	vaesenc	%xmm1,%xmm11,%xmm11+	vpxor	%xmm6,%xmm7,%xmm7+	vaesenc	%xmm1,%xmm12,%xmm12+	vpxor	%xmm0,%xmm4,%xmm4+	movbeq	8(%r14),%r13+	vaesenc	%xmm1,%xmm13,%xmm13+	movbeq	0(%r14),%r12+	vaesenc	%xmm1,%xmm14,%xmm14+	vmovups	160-128(%rcx),%xmm1+	cmpl	$11,%r10d+	jb	.Lenc_tail++	vaesenc	%xmm15,%xmm9,%xmm9+	vaesenc	%xmm15,%xmm10,%xmm10+	vaesenc	%xmm15,%xmm11,%xmm11+	vaesenc	%xmm15,%xmm12,%xmm12+	vaesenc	%xmm15,%xmm13,%xmm13+	vaesenc	%xmm15,%xmm14,%xmm14++	vaesenc	%xmm1,%xmm9,%xmm9+	vaesenc	%xmm1,%xmm10,%xmm10+	vaesenc	%xmm1,%xmm11,%xmm11+	vaesenc	%xmm1,%xmm12,%xmm12+	vaesenc	%xmm1,%xmm13,%xmm13+	vmovups	176-128(%rcx),%xmm15+	vaesenc	%xmm1,%xmm14,%xmm14+	vmovups	192-128(%rcx),%xmm1+	je	.Lenc_tail++	vaesenc	%xmm15,%xmm9,%xmm9+	vaesenc	%xmm15,%xmm10,%xmm10+	vaesenc	%xmm15,%xmm11,%xmm11+	vaesenc	%xmm15,%xmm12,%xmm12+	vaesenc	%xmm15,%xmm13,%xmm13+	vaesenc	%xmm15,%xmm14,%xmm14++	vaesenc	%xmm1,%xmm9,%xmm9+	vaesenc	%xmm1,%xmm10,%xmm10+	vaesenc	%xmm1,%xmm11,%xmm11+	vaesenc	%xmm1,%xmm12,%xmm12+	vaesenc	%xmm1,%xmm13,%xmm13+	vmovups	208-128(%rcx),%xmm15+	vaesenc	%xmm1,%xmm14,%xmm14+	vmovups	224-128(%rcx),%xmm1+	jmp	.Lenc_tail++.p2align	5+.Lhandle_ctr32:+	vmovdqu	(%r11),%xmm0+	vpshufb	%xmm0,%xmm1,%xmm6+	vmovdqu	48(%r11),%xmm5+	vpaddd	64(%r11),%xmm6,%xmm10+	vpaddd	%xmm5,%xmm6,%xmm11+	vmovdqu	0-32(%r9),%xmm3+	vpaddd	%xmm5,%xmm10,%xmm12+	vpshufb	%xmm0,%xmm10,%xmm10+	vpaddd	%xmm5,%xmm11,%xmm13+	vpshufb	%xmm0,%xmm11,%xmm11+	vpxor	%xmm15,%xmm10,%xmm10+	vpaddd	%xmm5,%xmm12,%xmm14+	vpshufb	%xmm0,%xmm12,%xmm12+	vpxor	%xmm15,%xmm11,%xmm11+	vpaddd	%xmm5,%xmm13,%xmm1+	vpshufb	%xmm0,%xmm13,%xmm13+	vpshufb	%xmm0,%xmm14,%xmm14+	vpshufb	%xmm0,%xmm1,%xmm1+	jmp	.Lresume_ctr32++.p2align	5+.Lenc_tail:+	vaesenc	%xmm15,%xmm9,%xmm9+	vmovdqu	%xmm7,16+8(%rsp)+	vpalignr	$8,%xmm4,%xmm4,%xmm8+	vaesenc	%xmm15,%xmm10,%xmm10+	vpclmulqdq	$0x10,%xmm3,%xmm4,%xmm4+	vpxor	0(%rdi),%xmm1,%xmm2+	vaesenc	%xmm15,%xmm11,%xmm11+	vpxor	16(%rdi),%xmm1,%xmm0+	vaesenc	%xmm15,%xmm12,%xmm12+	vpxor	32(%rdi),%xmm1,%xmm5+	vaesenc	%xmm15,%xmm13,%xmm13+	vpxor	48(%rdi),%xmm1,%xmm6+	vaesenc	%xmm15,%xmm14,%xmm14+	vpxor	64(%rdi),%xmm1,%xmm7+	vpxor	80(%rdi),%xmm1,%xmm3+	vmovdqu	(%r8),%xmm1++	vaesenclast	%xmm2,%xmm9,%xmm9+	vmovdqu	32(%r11),%xmm2+	vaesenclast	%xmm0,%xmm10,%xmm10+	vpaddb	%xmm2,%xmm1,%xmm0+	movq	%r13,112+8(%rsp)+	leaq	96(%rdi),%rdi+	vaesenclast	%xmm5,%xmm11,%xmm11+	vpaddb	%xmm2,%xmm0,%xmm5+	movq	%r12,120+8(%rsp)+	leaq	96(%rsi),%rsi+	vmovdqu	0-128(%rcx),%xmm15+	vaesenclast	%xmm6,%xmm12,%xmm12+	vpaddb	%xmm2,%xmm5,%xmm6+	vaesenclast	%xmm7,%xmm13,%xmm13+	vpaddb	%xmm2,%xmm6,%xmm7+	vaesenclast	%xmm3,%xmm14,%xmm14+	vpaddb	%xmm2,%xmm7,%xmm3++	addq	$0x60,%rax+	subq	$0x6,%rdx+	jc	.L6x_done++	vmovups	%xmm9,-96(%rsi)+	vpxor	%xmm15,%xmm1,%xmm9+	vmovups	%xmm10,-80(%rsi)+	vmovdqa	%xmm0,%xmm10+	vmovups	%xmm11,-64(%rsi)+	vmovdqa	%xmm5,%xmm11+	vmovups	%xmm12,-48(%rsi)+	vmovdqa	%xmm6,%xmm12+	vmovups	%xmm13,-32(%rsi)+	vmovdqa	%xmm7,%xmm13+	vmovups	%xmm14,-16(%rsi)+	vmovdqa	%xmm3,%xmm14+	vmovdqu	32+8(%rsp),%xmm7+	jmp	.Loop6x++.L6x_done:+	vpxor	16+8(%rsp),%xmm8,%xmm8+	vpxor	%xmm4,%xmm8,%xmm8++	.byte	0xf3,0xc3+++.globl	crypton_gcm_asm_decrypt+.def	crypton_gcm_asm_decrypt;	.scl 2;	.type 32;	.endef+.p2align	5+crypton_gcm_asm_decrypt:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_gcm_asm_decrypt:+++	pushq	%rbp++	movq	%rsp,%rbp++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	movq	%r9,%rcx+	movq	48(%rsp),%r8+	movq	56(%rsp),%r9+	xorq	%rax,%rax+	cmpq	$0x60,%rdx+	jb	.Lgcm_dec_abort++	pushq	%rbx++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	leaq	-168(%rsp),%rsp++	movaps	%xmm6,-208(%rbp)+	movaps	%xmm7,-192(%rbp)+	movaps	%xmm8,-176(%rbp)+	movaps	%xmm9,-160(%rbp)+	movaps	%xmm10,-144(%rbp)+	movaps	%xmm11,-128(%rbp)+	movaps	%xmm12,-112(%rbp)+	movaps	%xmm13,-96(%rbp)+	movaps	%xmm14,-80(%rbp)+	movaps	%xmm15,-64(%rbp)++.LSEH_body_crypton_gcm_asm_decrypt:++	vzeroupper++	vmovdqu	(%r8),%xmm1+	addq	$-128,%rsp+	movl	12(%r8),%ebx+	leaq	.Lbswap_mask(%rip),%r11+	leaq	-128(%rcx),%r14+	movq	$0xf80,%r15+	vmovdqu	(%r9),%xmm8+	andq	$-128,%rsp+	vmovdqu	(%r11),%xmm0+	leaq	128(%rcx),%rcx+	leaq	32+32(%r9),%r9+	movl	240-128(%rcx),%r10d+	vpshufb	%xmm0,%xmm8,%xmm8++	andq	%r15,%r14+	andq	%rsp,%r15+	subq	%r14,%r15+	jc	.Ldec_no_key_aliasing+	cmpq	$768,%r15+	jnc	.Ldec_no_key_aliasing+	subq	%r15,%rsp+.Ldec_no_key_aliasing:++	vmovdqu	80(%rdi),%xmm7+	leaq	(%rdi),%r14+	vmovdqu	64(%rdi),%xmm4+	leaq	-192(%rdi,%rdx,1),%r15+	vmovdqu	48(%rdi),%xmm5+	shrq	$4,%rdx+	xorq	%rax,%rax+	vmovdqu	32(%rdi),%xmm6+	vpshufb	%xmm0,%xmm7,%xmm7+	vmovdqu	16(%rdi),%xmm2+	vpshufb	%xmm0,%xmm4,%xmm4+	vmovdqu	(%rdi),%xmm3+	vpshufb	%xmm0,%xmm5,%xmm5+	vmovdqu	%xmm4,48(%rsp)+	vpshufb	%xmm0,%xmm6,%xmm6+	vmovdqu	%xmm5,64(%rsp)+	vpshufb	%xmm0,%xmm2,%xmm2+	vmovdqu	%xmm6,80(%rsp)+	vpshufb	%xmm0,%xmm3,%xmm3+	vmovdqu	%xmm2,96(%rsp)+	vmovdqu	%xmm3,112(%rsp)++	call	_crypton_gcm_asm_ctr32_ghash_6x++	vmovups	%xmm9,-96(%rsi)+	vmovups	%xmm10,-80(%rsi)+	vmovups	%xmm11,-64(%rsi)+	vmovups	%xmm12,-48(%rsi)+	vmovups	%xmm13,-32(%rsi)+	vmovups	%xmm14,-16(%rsi)++	vpshufb	(%r11),%xmm8,%xmm8+	vmovdqu	%xmm8,-64(%r9)++	vzeroupper+	movaps	-208(%rbp),%xmm6+	movaps	-192(%rbp),%xmm7+	movaps	-176(%rbp),%xmm8+	movaps	-160(%rbp),%xmm9+	movaps	-144(%rbp),%xmm10+	movaps	-128(%rbp),%xmm11+	movaps	-112(%rbp),%xmm12+	movaps	-96(%rbp),%xmm13+	movaps	-80(%rbp),%xmm14+	movaps	-64(%rbp),%xmm15+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp++.Lgcm_dec_abort:+	popq	%rbp++.LSEH_epilogue_crypton_gcm_asm_decrypt:+	mov	8(%rsp),%rdi+	mov	16(%rsp),%rsi++	.byte	0xf3,0xc3++.LSEH_end_crypton_gcm_asm_decrypt:+.def	_crypton_gcm_asm_ctr32_6x;	.scl 3;	.type 32;	.endef+.p2align	5+_crypton_gcm_asm_ctr32_6x:+	.byte	0xf3,0x0f,0x1e,0xfa+++	vmovdqu	0-128(%rcx),%xmm4+	vmovdqu	32(%r11),%xmm2+	leaq	-1(%r10),%r13+	vmovups	16-128(%rcx),%xmm15+	leaq	32-128(%rcx),%r12+	vpxor	%xmm4,%xmm1,%xmm9+	addl	$100663296,%ebx+	jc	.Lhandle_ctr32_2+	vpaddb	%xmm2,%xmm1,%xmm10+	vpaddb	%xmm2,%xmm10,%xmm11+	vpxor	%xmm4,%xmm10,%xmm10+	vpaddb	%xmm2,%xmm11,%xmm12+	vpxor	%xmm4,%xmm11,%xmm11+	vpaddb	%xmm2,%xmm12,%xmm13+	vpxor	%xmm4,%xmm12,%xmm12+	vpaddb	%xmm2,%xmm13,%xmm14+	vpxor	%xmm4,%xmm13,%xmm13+	vpaddb	%xmm2,%xmm14,%xmm1+	vpxor	%xmm4,%xmm14,%xmm14+	jmp	.Loop_ctr32++.p2align	4+.Loop_ctr32:+	vaesenc	%xmm15,%xmm9,%xmm9+	vaesenc	%xmm15,%xmm10,%xmm10+	vaesenc	%xmm15,%xmm11,%xmm11+	vaesenc	%xmm15,%xmm12,%xmm12+	vaesenc	%xmm15,%xmm13,%xmm13+	vaesenc	%xmm15,%xmm14,%xmm14+	vmovups	(%r12),%xmm15+	leaq	16(%r12),%r12+	decl	%r13d+	jnz	.Loop_ctr32++	vmovdqu	(%r12),%xmm3+	vaesenc	%xmm15,%xmm9,%xmm9+	vpxor	0(%rdi),%xmm3,%xmm4+	vaesenc	%xmm15,%xmm10,%xmm10+	vpxor	16(%rdi),%xmm3,%xmm5+	vaesenc	%xmm15,%xmm11,%xmm11+	vpxor	32(%rdi),%xmm3,%xmm6+	vaesenc	%xmm15,%xmm12,%xmm12+	vpxor	48(%rdi),%xmm3,%xmm8+	vaesenc	%xmm15,%xmm13,%xmm13+	vpxor	64(%rdi),%xmm3,%xmm2+	vaesenc	%xmm15,%xmm14,%xmm14+	vpxor	80(%rdi),%xmm3,%xmm3+	leaq	96(%rdi),%rdi++	vaesenclast	%xmm4,%xmm9,%xmm9+	vaesenclast	%xmm5,%xmm10,%xmm10+	vaesenclast	%xmm6,%xmm11,%xmm11+	vaesenclast	%xmm8,%xmm12,%xmm12+	vaesenclast	%xmm2,%xmm13,%xmm13+	vaesenclast	%xmm3,%xmm14,%xmm14+	vmovups	%xmm9,0(%rsi)+	vmovups	%xmm10,16(%rsi)+	vmovups	%xmm11,32(%rsi)+	vmovups	%xmm12,48(%rsi)+	vmovups	%xmm13,64(%rsi)+	vmovups	%xmm14,80(%rsi)+	leaq	96(%rsi),%rsi++	.byte	0xf3,0xc3+.p2align	5+.Lhandle_ctr32_2:+	vpshufb	%xmm0,%xmm1,%xmm6+	vmovdqu	48(%r11),%xmm5+	vpaddd	64(%r11),%xmm6,%xmm10+	vpaddd	%xmm5,%xmm6,%xmm11+	vpaddd	%xmm5,%xmm10,%xmm12+	vpshufb	%xmm0,%xmm10,%xmm10+	vpaddd	%xmm5,%xmm11,%xmm13+	vpshufb	%xmm0,%xmm11,%xmm11+	vpxor	%xmm4,%xmm10,%xmm10+	vpaddd	%xmm5,%xmm12,%xmm14+	vpshufb	%xmm0,%xmm12,%xmm12+	vpxor	%xmm4,%xmm11,%xmm11+	vpaddd	%xmm5,%xmm13,%xmm1+	vpshufb	%xmm0,%xmm13,%xmm13+	vpxor	%xmm4,%xmm12,%xmm12+	vpshufb	%xmm0,%xmm14,%xmm14+	vpxor	%xmm4,%xmm13,%xmm13+	vpshufb	%xmm0,%xmm1,%xmm1+	vpxor	%xmm4,%xmm14,%xmm14+	jmp	.Loop_ctr32++++.globl	crypton_gcm_asm_encrypt+.def	crypton_gcm_asm_encrypt;	.scl 2;	.type 32;	.endef+.p2align	5+crypton_gcm_asm_encrypt:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_gcm_asm_encrypt:+++	pushq	%rbp++	movq	%rsp,%rbp++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	movq	%r9,%rcx+	movq	48(%rsp),%r8+	movq	56(%rsp),%r9+	xorq	%rax,%rax+	cmpq	$288,%rdx+	jb	.Lgcm_enc_abort++	pushq	%rbx++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	leaq	-168(%rsp),%rsp++	movaps	%xmm6,-208(%rbp)+	movaps	%xmm7,-192(%rbp)+	movaps	%xmm8,-176(%rbp)+	movaps	%xmm9,-160(%rbp)+	movaps	%xmm10,-144(%rbp)+	movaps	%xmm11,-128(%rbp)+	movaps	%xmm12,-112(%rbp)+	movaps	%xmm13,-96(%rbp)+	movaps	%xmm14,-80(%rbp)+	movaps	%xmm15,-64(%rbp)++.LSEH_body_crypton_gcm_asm_encrypt:++	vzeroupper++	vmovdqu	(%r8),%xmm1+	addq	$-128,%rsp+	movl	12(%r8),%ebx+	leaq	.Lbswap_mask(%rip),%r11+	leaq	-128(%rcx),%r14+	movq	$0xf80,%r15+	leaq	128(%rcx),%rcx+	vmovdqu	(%r11),%xmm0+	andq	$-128,%rsp+	movl	240-128(%rcx),%r10d++	andq	%r15,%r14+	andq	%rsp,%r15+	subq	%r14,%r15+	jc	.Lenc_no_key_aliasing+	cmpq	$768,%r15+	jnc	.Lenc_no_key_aliasing+	subq	%r15,%rsp+.Lenc_no_key_aliasing:++	leaq	(%rsi),%r14+	leaq	-192(%rsi,%rdx,1),%r15+	shrq	$4,%rdx++	call	_crypton_gcm_asm_ctr32_6x+	vpshufb	%xmm0,%xmm9,%xmm8+	vpshufb	%xmm0,%xmm10,%xmm2+	vmovdqu	%xmm8,112(%rsp)+	vpshufb	%xmm0,%xmm11,%xmm4+	vmovdqu	%xmm2,96(%rsp)+	vpshufb	%xmm0,%xmm12,%xmm5+	vmovdqu	%xmm4,80(%rsp)+	vpshufb	%xmm0,%xmm13,%xmm6+	vmovdqu	%xmm5,64(%rsp)+	vpshufb	%xmm0,%xmm14,%xmm7+	vmovdqu	%xmm6,48(%rsp)++	call	_crypton_gcm_asm_ctr32_6x++	vmovdqu	(%r9),%xmm8+	leaq	32+32(%r9),%r9+	subq	$12,%rdx+	movq	$192,%rax+	vpshufb	%xmm0,%xmm8,%xmm8++	call	_crypton_gcm_asm_ctr32_ghash_6x+	vmovdqu	32(%rsp),%xmm7+	vmovdqu	(%r11),%xmm0+	vmovdqu	0-32(%r9),%xmm3+	vpunpckhqdq	%xmm7,%xmm7,%xmm1+	vmovdqu	32-32(%r9),%xmm15+	vmovups	%xmm9,-96(%rsi)+	vpshufb	%xmm0,%xmm9,%xmm9+	vpxor	%xmm7,%xmm1,%xmm1+	vmovups	%xmm10,-80(%rsi)+	vpshufb	%xmm0,%xmm10,%xmm10+	vmovups	%xmm11,-64(%rsi)+	vpshufb	%xmm0,%xmm11,%xmm11+	vmovups	%xmm12,-48(%rsi)+	vpshufb	%xmm0,%xmm12,%xmm12+	vmovups	%xmm13,-32(%rsi)+	vpshufb	%xmm0,%xmm13,%xmm13+	vmovups	%xmm14,-16(%rsi)+	vpshufb	%xmm0,%xmm14,%xmm14+	vmovdqu	%xmm9,16(%rsp)+	vmovdqu	48(%rsp),%xmm6+	vmovdqu	16-32(%r9),%xmm0+	vpunpckhqdq	%xmm6,%xmm6,%xmm2+	vpclmulqdq	$0x00,%xmm3,%xmm7,%xmm5+	vpxor	%xmm6,%xmm2,%xmm2+	vpclmulqdq	$0x11,%xmm3,%xmm7,%xmm7+	vpclmulqdq	$0x00,%xmm15,%xmm1,%xmm1++	vmovdqu	64(%rsp),%xmm9+	vpclmulqdq	$0x00,%xmm0,%xmm6,%xmm4+	vmovdqu	48-32(%r9),%xmm3+	vpxor	%xmm5,%xmm4,%xmm4+	vpunpckhqdq	%xmm9,%xmm9,%xmm5+	vpclmulqdq	$0x11,%xmm0,%xmm6,%xmm6+	vpxor	%xmm9,%xmm5,%xmm5+	vpxor	%xmm7,%xmm6,%xmm6+	vpclmulqdq	$0x10,%xmm15,%xmm2,%xmm2+	vmovdqu	80-32(%r9),%xmm15+	vpxor	%xmm1,%xmm2,%xmm2++	vmovdqu	80(%rsp),%xmm1+	vpclmulqdq	$0x00,%xmm3,%xmm9,%xmm7+	vmovdqu	64-32(%r9),%xmm0+	vpxor	%xmm4,%xmm7,%xmm7+	vpunpckhqdq	%xmm1,%xmm1,%xmm4+	vpclmulqdq	$0x11,%xmm3,%xmm9,%xmm9+	vpxor	%xmm1,%xmm4,%xmm4+	vpxor	%xmm6,%xmm9,%xmm9+	vpclmulqdq	$0x00,%xmm15,%xmm5,%xmm5+	vpxor	%xmm2,%xmm5,%xmm5++	vmovdqu	96(%rsp),%xmm2+	vpclmulqdq	$0x00,%xmm0,%xmm1,%xmm6+	vmovdqu	96-32(%r9),%xmm3+	vpxor	%xmm7,%xmm6,%xmm6+	vpunpckhqdq	%xmm2,%xmm2,%xmm7+	vpclmulqdq	$0x11,%xmm0,%xmm1,%xmm1+	vpxor	%xmm2,%xmm7,%xmm7+	vpxor	%xmm9,%xmm1,%xmm1+	vpclmulqdq	$0x10,%xmm15,%xmm4,%xmm4+	vmovdqu	128-32(%r9),%xmm15+	vpxor	%xmm5,%xmm4,%xmm4++	vpxor	112(%rsp),%xmm8,%xmm8+	vpclmulqdq	$0x00,%xmm3,%xmm2,%xmm5+	vmovdqu	112-32(%r9),%xmm0+	vpunpckhqdq	%xmm8,%xmm8,%xmm9+	vpxor	%xmm6,%xmm5,%xmm5+	vpclmulqdq	$0x11,%xmm3,%xmm2,%xmm2+	vpxor	%xmm8,%xmm9,%xmm9+	vpxor	%xmm1,%xmm2,%xmm2+	vpclmulqdq	$0x00,%xmm15,%xmm7,%xmm7+	vpxor	%xmm4,%xmm7,%xmm4++	vpclmulqdq	$0x00,%xmm0,%xmm8,%xmm6+	vmovdqu	0-32(%r9),%xmm3+	vpunpckhqdq	%xmm14,%xmm14,%xmm1+	vpclmulqdq	$0x11,%xmm0,%xmm8,%xmm8+	vpxor	%xmm14,%xmm1,%xmm1+	vpxor	%xmm5,%xmm6,%xmm5+	vpclmulqdq	$0x10,%xmm15,%xmm9,%xmm9+	vmovdqu	32-32(%r9),%xmm15+	vpxor	%xmm2,%xmm8,%xmm7+	vpxor	%xmm4,%xmm9,%xmm6++	vmovdqu	16-32(%r9),%xmm0+	vpxor	%xmm5,%xmm7,%xmm9+	vpclmulqdq	$0x00,%xmm3,%xmm14,%xmm4+	vpxor	%xmm9,%xmm6,%xmm6+	vpunpckhqdq	%xmm13,%xmm13,%xmm2+	vpclmulqdq	$0x11,%xmm3,%xmm14,%xmm14+	vpxor	%xmm13,%xmm2,%xmm2+	vpslldq	$8,%xmm6,%xmm9+	vpclmulqdq	$0x00,%xmm15,%xmm1,%xmm1+	vpxor	%xmm9,%xmm5,%xmm8+	vpsrldq	$8,%xmm6,%xmm6+	vpxor	%xmm6,%xmm7,%xmm7++	vpclmulqdq	$0x00,%xmm0,%xmm13,%xmm5+	vmovdqu	48-32(%r9),%xmm3+	vpxor	%xmm4,%xmm5,%xmm5+	vpunpckhqdq	%xmm12,%xmm12,%xmm9+	vpclmulqdq	$0x11,%xmm0,%xmm13,%xmm13+	vpxor	%xmm12,%xmm9,%xmm9+	vpxor	%xmm14,%xmm13,%xmm13+	vpalignr	$8,%xmm8,%xmm8,%xmm14+	vpclmulqdq	$0x10,%xmm15,%xmm2,%xmm2+	vmovdqu	80-32(%r9),%xmm15+	vpxor	%xmm1,%xmm2,%xmm2++	vpclmulqdq	$0x00,%xmm3,%xmm12,%xmm4+	vmovdqu	64-32(%r9),%xmm0+	vpxor	%xmm5,%xmm4,%xmm4+	vpunpckhqdq	%xmm11,%xmm11,%xmm1+	vpclmulqdq	$0x11,%xmm3,%xmm12,%xmm12+	vpxor	%xmm11,%xmm1,%xmm1+	vpxor	%xmm13,%xmm12,%xmm12+	vxorps	16(%rsp),%xmm7,%xmm7+	vpclmulqdq	$0x00,%xmm15,%xmm9,%xmm9+	vpxor	%xmm2,%xmm9,%xmm9++	vpclmulqdq	$0x10,16(%r11),%xmm8,%xmm8+	vxorps	%xmm14,%xmm8,%xmm8++	vpclmulqdq	$0x00,%xmm0,%xmm11,%xmm5+	vmovdqu	96-32(%r9),%xmm3+	vpxor	%xmm4,%xmm5,%xmm5+	vpunpckhqdq	%xmm10,%xmm10,%xmm2+	vpclmulqdq	$0x11,%xmm0,%xmm11,%xmm11+	vpxor	%xmm10,%xmm2,%xmm2+	vpalignr	$8,%xmm8,%xmm8,%xmm14+	vpxor	%xmm12,%xmm11,%xmm11+	vpclmulqdq	$0x10,%xmm15,%xmm1,%xmm1+	vmovdqu	128-32(%r9),%xmm15+	vpxor	%xmm9,%xmm1,%xmm1++	vxorps	%xmm7,%xmm14,%xmm14+	vpclmulqdq	$0x10,16(%r11),%xmm8,%xmm8+	vxorps	%xmm14,%xmm8,%xmm8++	vpclmulqdq	$0x00,%xmm3,%xmm10,%xmm4+	vmovdqu	112-32(%r9),%xmm0+	vpxor	%xmm5,%xmm4,%xmm4+	vpunpckhqdq	%xmm8,%xmm8,%xmm9+	vpclmulqdq	$0x11,%xmm3,%xmm10,%xmm10+	vpxor	%xmm8,%xmm9,%xmm9+	vpxor	%xmm11,%xmm10,%xmm10+	vpclmulqdq	$0x00,%xmm15,%xmm2,%xmm2+	vpxor	%xmm1,%xmm2,%xmm2++	vpclmulqdq	$0x00,%xmm0,%xmm8,%xmm5+	vpclmulqdq	$0x11,%xmm0,%xmm8,%xmm7+	vpxor	%xmm4,%xmm5,%xmm5+	vpclmulqdq	$0x10,%xmm15,%xmm9,%xmm6+	vpxor	%xmm10,%xmm7,%xmm7+	vpxor	%xmm2,%xmm6,%xmm6++	vpxor	%xmm5,%xmm7,%xmm4+	vpxor	%xmm4,%xmm6,%xmm6+	vpslldq	$8,%xmm6,%xmm1+	vmovdqu	16(%r11),%xmm3+	vpsrldq	$8,%xmm6,%xmm6+	vpxor	%xmm1,%xmm5,%xmm8+	vpxor	%xmm6,%xmm7,%xmm7++	vpalignr	$8,%xmm8,%xmm8,%xmm2+	vpclmulqdq	$0x10,%xmm3,%xmm8,%xmm8+	vpxor	%xmm2,%xmm8,%xmm8++	vpalignr	$8,%xmm8,%xmm8,%xmm2+	vpclmulqdq	$0x10,%xmm3,%xmm8,%xmm8+	vpxor	%xmm7,%xmm2,%xmm2+	vpxor	%xmm2,%xmm8,%xmm8+	vpshufb	(%r11),%xmm8,%xmm8+	vmovdqu	%xmm8,-64(%r9)++	vzeroupper+	movaps	-208(%rbp),%xmm6+	movaps	-192(%rbp),%xmm7+	movaps	-176(%rbp),%xmm8+	movaps	-160(%rbp),%xmm9+	movaps	-144(%rbp),%xmm10+	movaps	-128(%rbp),%xmm11+	movaps	-112(%rbp),%xmm12+	movaps	-96(%rbp),%xmm13+	movaps	-80(%rbp),%xmm14+	movaps	-64(%rbp),%xmm15+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp++.Lgcm_enc_abort:+	popq	%rbp++.LSEH_epilogue_crypton_gcm_asm_encrypt:+	mov	8(%rsp),%rdi+	mov	16(%rsp),%rsi++	.byte	0xf3,0xc3++.LSEH_end_crypton_gcm_asm_encrypt:+.p2align	6+.Lbswap_mask:+.byte	15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,0+.Lpoly:+.byte	0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0xc2+.Lone_msb:+.byte	0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1+.Ltwo_lsb:+.byte	2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0+.Lone_lsb:+.byte	1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0+.byte	65,69,83,45,78,73,32,71,67,77,32,109,111,100,117,108,101,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,60,97,112,112,114,111,64,111,112,101,110,115,115,108,46,111,114,103,62,0+.p2align	6+.section	.pdata+.p2align	2+.rva	.LSEH_begin_crypton_gcm_asm_decrypt+.rva	.LSEH_body_crypton_gcm_asm_decrypt+.rva	.LSEH_info_crypton_gcm_asm_decrypt_prologue++.rva	.LSEH_body_crypton_gcm_asm_decrypt+.rva	.LSEH_epilogue_crypton_gcm_asm_decrypt+.rva	.LSEH_info_crypton_gcm_asm_decrypt_body++.rva	.LSEH_epilogue_crypton_gcm_asm_decrypt+.rva	.LSEH_end_crypton_gcm_asm_decrypt+.rva	.LSEH_info_crypton_gcm_asm_decrypt_epilogue++.rva	.LSEH_begin_crypton_gcm_asm_encrypt+.rva	.LSEH_body_crypton_gcm_asm_encrypt+.rva	.LSEH_info_crypton_gcm_asm_encrypt_prologue++.rva	.LSEH_body_crypton_gcm_asm_encrypt+.rva	.LSEH_epilogue_crypton_gcm_asm_encrypt+.rva	.LSEH_info_crypton_gcm_asm_encrypt_body++.rva	.LSEH_epilogue_crypton_gcm_asm_encrypt+.rva	.LSEH_end_crypton_gcm_asm_encrypt+.rva	.LSEH_info_crypton_gcm_asm_encrypt_epilogue++.section	.xdata+.p2align	3+.LSEH_info_crypton_gcm_asm_decrypt_prologue:+.byte	1,4,6,0x05+.byte	4,0x74,2,0+.byte	4,0x64,3,0+.byte	4,0x53+.byte	1,0x50+.long	0,0+.LSEH_info_crypton_gcm_asm_decrypt_body:+.byte	1,0,38,213+.byte	0x00,0x68,0x00,0x00+.byte	0x00,0x78,0x01,0x00+.byte	0x00,0x88,0x02,0x00+.byte	0x00,0x98,0x03,0x00+.byte	0x00,0xa8,0x04,0x00+.byte	0x00,0xb8,0x05,0x00+.byte	0x00,0xc8,0x06,0x00+.byte	0x00,0xd8,0x07,0x00+.byte	0x00,0xe8,0x08,0x00+.byte	0x00,0xf8,0x09,0x00+.byte	0x00,0xf4,0x15,0x00+.byte	0x00,0xe4,0x16,0x00+.byte	0x00,0xd4,0x17,0x00+.byte	0x00,0xc4,0x18,0x00+.byte	0x00,0x34,0x19,0x00+.byte	0x00,0x74,0x1c,0x00+.byte	0x00,0x64,0x1d,0x00+.byte	0x00,0x53+.byte	0x00,0x01,0x1a,0x00+.byte	0x00,0x50+.byte	0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00+.LSEH_info_crypton_gcm_asm_decrypt_epilogue:+.byte	1,0,4,0+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0x00,0x00,0x00++.LSEH_info_crypton_gcm_asm_encrypt_prologue:+.byte	1,4,6,0x05+.byte	4,0x74,2,0+.byte	4,0x64,3,0+.byte	4,0x53+.byte	1,0x50+.long	0,0+.LSEH_info_crypton_gcm_asm_encrypt_body:+.byte	1,0,38,213+.byte	0x00,0x68,0x00,0x00+.byte	0x00,0x78,0x01,0x00+.byte	0x00,0x88,0x02,0x00+.byte	0x00,0x98,0x03,0x00+.byte	0x00,0xa8,0x04,0x00+.byte	0x00,0xb8,0x05,0x00+.byte	0x00,0xc8,0x06,0x00+.byte	0x00,0xd8,0x07,0x00+.byte	0x00,0xe8,0x08,0x00+.byte	0x00,0xf8,0x09,0x00+.byte	0x00,0xf4,0x15,0x00+.byte	0x00,0xe4,0x16,0x00+.byte	0x00,0xd4,0x17,0x00+.byte	0x00,0xc4,0x18,0x00+.byte	0x00,0x34,0x19,0x00+.byte	0x00,0x74,0x1c,0x00+.byte	0x00,0x64,0x1d,0x00+.byte	0x00,0x53+.byte	0x00,0x01,0x1a,0x00+.byte	0x00,0x50+.byte	0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00+.LSEH_info_crypton_gcm_asm_encrypt_epilogue:+.byte	1,0,4,0+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0x00,0x00,0x00+
+ cbits/asm/aesni-gcm-x86_64.pl view
@@ -0,0 +1,974 @@+#! /usr/bin/env perl+#+# ====================================================================+# Written by Andy Polyakov <appro@openssl.org> for the OpenSSL+# project. The module is, however, dual licensed under OpenSSL and+# CRYPTOGAMS licenses depending on where you obtain it. For further+# details see http://www.openssl.org/~appro/cryptogams/.+# ====================================================================+#+#+# AES-NI-CTR+GHASH stitch.+#+# February 2013+#+# OpenSSL GCM implementation is organized in such way that its+# performance is rather close to the sum of its streamed components,+# in the context parallelized AES-NI CTR and modulo-scheduled+# PCLMULQDQ-enabled GHASH. Unfortunately, as no stitch implementation+# was observed to perform significantly better than the sum of the+# components on contemporary CPUs, the effort was deemed impossible to+# justify. This module is based on combination of Intel submissions,+# [1] and [2], with MOVBE twist suggested by Ilya Albrekht and Max+# Locktyukhin of Intel Corp. who verified that it reduces shuffles+# pressure with notable relative improvement, achieving 1.0 cycle per+# byte processed with 128-bit key on Haswell processor, 0.74 - on+# Broadwell, 0.63 - on Skylake... [Mentioned results are raw profiled+# measurements for favourable packet size, one divisible by 96.+# Applications using the EVP interface will observe a few percent+# worse performance.]+#+# Knights Landing processes 1 byte in 1.25 cycles (measured with EVP).+#+# [1] http://rt.openssl.org/Ticket/Display.html?id=2900&user=guest&pass=guest+# [2] http://www.intel.com/content/dam/www/public/us/en/documents/software-support/enabling-high-performance-gcm.pdf++$flavour = shift;+$output  = shift;+if ($flavour =~ /\./) { $output = $flavour; undef $flavour; }++$win64=0; $win64=1 if ($flavour =~ /[nm]asm|mingw64/ || $output =~ /\.asm$/);++$0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1;+( $xlate="${dir}x86_64-xlate.pl" and -f $xlate ) or+( $xlate="${dir}../../perlasm/x86_64-xlate.pl" and -f $xlate) or+die "can't locate x86_64-xlate.pl";++$ENV{CC} //= "cc";+if (`$ENV{CC} -Wa,-v -c -o /dev/null -x assembler /dev/null 2>&1`+		=~ /GNU assembler version ([2-9]\.[0-9]+)/) {+	$avx = ($1>=2.20) + ($1>=2.22);+}++if (!$avx && $win64 && ($flavour =~ /nasm/ || $ENV{ASM} =~ /nasm/) &&+	    `nasm -v 2>&1` =~ /NASM version ([2-9]\.[0-9]+)/) {+	$avx = ($1>=2.09) + ($1>=2.10);+}++if (!$avx && $win64 && ($flavour =~ /masm/ || $ENV{ASM} =~ /ml64/) &&+	    `ml64 2>&1` =~ /Version ([0-9]+)\./) {+	$avx = ($1>=10) + ($1>=11);+}++if (!$avx && `$ENV{CC} -v 2>&1` =~ /((?:clang|LLVM) version|.*based on LLVM) ([0-9]+\.[0-9]+)/) {+	$avx = ($2>=3.0) + ($2>3.0);+}++open OUT,"| \"$^X\" \"$xlate\" $flavour \"$output\"";+*STDOUT=*OUT;++if ($avx>1) {{{++($inp,$out,$len,$key,$ivp,$Xip)=("%rdi","%rsi","%rdx","%rcx","%r8","%r9");++($Ii,$T1,$T2,$Hkey,+ $Z0,$Z1,$Z2,$Z3,$Xi) = map("%xmm$_",(0..8));++($inout0,$inout1,$inout2,$inout3,$inout4,$inout5,$rndkey) = map("%xmm$_",(9..15));++($counter,$rounds,$ret,$const,$in0,$end0)=("%ebx","%r10d","%rax","%r11","%r14","%r15");++$code=<<___;+.text++.type	_aesni_ctr32_ghash_6x,\@abi-omnipotent+.align	32+_aesni_ctr32_ghash_6x:+.cfi_startproc+	vmovdqu		0x20($const),$T2	# borrow $T2, .Lone_msb+	sub		\$6,$len+	vpxor		$Z0,$Z0,$Z0		# $Z0   = 0+	vmovdqu		0x00-0x80($key),$rndkey+	vpaddb		$T2,$T1,$inout1+	vpaddb		$T2,$inout1,$inout2+	vpaddb		$T2,$inout2,$inout3+	vpaddb		$T2,$inout3,$inout4+	vpaddb		$T2,$inout4,$inout5+	vpxor		$rndkey,$T1,$inout0+	vmovdqu		$Z0,16+8(%rsp)		# "$Z3" = 0+	jmp		.Loop6x++.align	32+.Loop6x:+	add		\$`6<<24`,$counter+	jc		.Lhandle_ctr32		# discard $inout[1-5]?+	vmovdqu		0x00-0x20($Xip),$Hkey	# $Hkey^1+	  vpaddb	$T2,$inout5,$T1		# next counter value+	  vpxor		$rndkey,$inout1,$inout1+	  vpxor		$rndkey,$inout2,$inout2++.Lresume_ctr32:+	vmovdqu		$T1,($ivp)		# save next counter value+	vpclmulqdq	\$0x10,$Hkey,$Z3,$Z1+	  vpxor		$rndkey,$inout3,$inout3+	  vmovups	0x10-0x80($key),$T2	# borrow $T2 for $rndkey+	vpclmulqdq	\$0x01,$Hkey,$Z3,$Z2+	xor		%r12,%r12+	cmp		$in0,$end0++	  vaesenc	$T2,$inout0,$inout0+	vmovdqu		0x30+8(%rsp),$Ii	# I[4]+	  vpxor		$rndkey,$inout4,$inout4+	vpclmulqdq	\$0x00,$Hkey,$Z3,$T1+	  vaesenc	$T2,$inout1,$inout1+	  vpxor		$rndkey,$inout5,$inout5+	setnc		%r12b+	vpclmulqdq	\$0x11,$Hkey,$Z3,$Z3+	  vaesenc	$T2,$inout2,$inout2+	vmovdqu		0x10-0x20($Xip),$Hkey	# $Hkey^2+	neg		%r12+	  vaesenc	$T2,$inout3,$inout3+	 vpxor		$Z1,$Z2,$Z2+	vpclmulqdq	\$0x00,$Hkey,$Ii,$Z1+	 vpxor		$Z0,$Xi,$Xi		# modulo-scheduled+	  vaesenc	$T2,$inout4,$inout4+	 vpxor		$Z1,$T1,$Z0+	and		\$0x60,%r12+	  vmovups	0x20-0x80($key),$rndkey+	vpclmulqdq	\$0x10,$Hkey,$Ii,$T1+	  vaesenc	$T2,$inout5,$inout5++	vpclmulqdq	\$0x01,$Hkey,$Ii,$T2+	lea		($in0,%r12),$in0+	  vaesenc	$rndkey,$inout0,$inout0+	 vpxor		16+8(%rsp),$Xi,$Xi	# modulo-scheduled [vpxor $Z3,$Xi,$Xi]+	vpclmulqdq	\$0x11,$Hkey,$Ii,$Hkey+	 vmovdqu	0x40+8(%rsp),$Ii	# I[3]+	  vaesenc	$rndkey,$inout1,$inout1+	movbe		0x58($in0),%r13+	  vaesenc	$rndkey,$inout2,$inout2+	movbe		0x50($in0),%r12+	  vaesenc	$rndkey,$inout3,$inout3+	mov		%r13,0x20+8(%rsp)+	  vaesenc	$rndkey,$inout4,$inout4+	mov		%r12,0x28+8(%rsp)+	vmovdqu		0x30-0x20($Xip),$Z1	# borrow $Z1 for $Hkey^3+	  vaesenc	$rndkey,$inout5,$inout5++	  vmovups	0x30-0x80($key),$rndkey+	 vpxor		$T1,$Z2,$Z2+	vpclmulqdq	\$0x00,$Z1,$Ii,$T1+	  vaesenc	$rndkey,$inout0,$inout0+	 vpxor		$T2,$Z2,$Z2+	vpclmulqdq	\$0x10,$Z1,$Ii,$T2+	  vaesenc	$rndkey,$inout1,$inout1+	 vpxor		$Hkey,$Z3,$Z3+	vpclmulqdq	\$0x01,$Z1,$Ii,$Hkey+	  vaesenc	$rndkey,$inout2,$inout2+	vpclmulqdq	\$0x11,$Z1,$Ii,$Z1+	 vmovdqu	0x50+8(%rsp),$Ii	# I[2]+	  vaesenc	$rndkey,$inout3,$inout3+	  vaesenc	$rndkey,$inout4,$inout4+	 vpxor		$T1,$Z0,$Z0+	vmovdqu		0x40-0x20($Xip),$T1	# borrow $T1 for $Hkey^4+	  vaesenc	$rndkey,$inout5,$inout5++	  vmovups	0x40-0x80($key),$rndkey+	 vpxor		$T2,$Z2,$Z2+	vpclmulqdq	\$0x00,$T1,$Ii,$T2+	  vaesenc	$rndkey,$inout0,$inout0+	 vpxor		$Hkey,$Z2,$Z2+	vpclmulqdq	\$0x10,$T1,$Ii,$Hkey+	  vaesenc	$rndkey,$inout1,$inout1+	movbe		0x48($in0),%r13+	 vpxor		$Z1,$Z3,$Z3+	vpclmulqdq	\$0x01,$T1,$Ii,$Z1+	  vaesenc	$rndkey,$inout2,$inout2+	movbe		0x40($in0),%r12+	vpclmulqdq	\$0x11,$T1,$Ii,$T1+	 vmovdqu	0x60+8(%rsp),$Ii	# I[1]+	  vaesenc	$rndkey,$inout3,$inout3+	mov		%r13,0x30+8(%rsp)+	  vaesenc	$rndkey,$inout4,$inout4+	mov		%r12,0x38+8(%rsp)+	 vpxor		$T2,$Z0,$Z0+	vmovdqu		0x60-0x20($Xip),$T2	# borrow $T2 for $Hkey^5+	  vaesenc	$rndkey,$inout5,$inout5++	  vmovups	0x50-0x80($key),$rndkey+	 vpxor		$Hkey,$Z2,$Z2+	vpclmulqdq	\$0x00,$T2,$Ii,$Hkey+	  vaesenc	$rndkey,$inout0,$inout0+	 vpxor		$Z1,$Z2,$Z2+	vpclmulqdq	\$0x10,$T2,$Ii,$Z1+	  vaesenc	$rndkey,$inout1,$inout1+	movbe		0x38($in0),%r13+	 vpxor		$T1,$Z3,$Z3+	vpclmulqdq	\$0x01,$T2,$Ii,$T1+	 vpxor		0x70+8(%rsp),$Xi,$Xi	# accumulate I[0]+	  vaesenc	$rndkey,$inout2,$inout2+	movbe		0x30($in0),%r12+	vpclmulqdq	\$0x11,$T2,$Ii,$T2+	  vaesenc	$rndkey,$inout3,$inout3+	mov		%r13,0x40+8(%rsp)+	  vaesenc	$rndkey,$inout4,$inout4+	mov		%r12,0x48+8(%rsp)+	 vpxor		$Hkey,$Z0,$Z0+	 vmovdqu	0x70-0x20($Xip),$Hkey	# $Hkey^6+	  vaesenc	$rndkey,$inout5,$inout5++	  vmovups	0x60-0x80($key),$rndkey+	 vpxor		$Z1,$Z2,$Z2+	vpclmulqdq	\$0x10,$Hkey,$Xi,$Z1+	  vaesenc	$rndkey,$inout0,$inout0+	 vpxor		$T1,$Z2,$Z2+	vpclmulqdq	\$0x01,$Hkey,$Xi,$T1+	  vaesenc	$rndkey,$inout1,$inout1+	movbe		0x28($in0),%r13+	 vpxor		$T2,$Z3,$Z3+	vpclmulqdq	\$0x00,$Hkey,$Xi,$T2+	  vaesenc	$rndkey,$inout2,$inout2+	movbe		0x20($in0),%r12+	vpclmulqdq	\$0x11,$Hkey,$Xi,$Xi+	  vaesenc	$rndkey,$inout3,$inout3+	mov		%r13,0x50+8(%rsp)+	  vaesenc	$rndkey,$inout4,$inout4+	mov		%r12,0x58+8(%rsp)+	vpxor		$Z1,$Z2,$Z2+	  vaesenc	$rndkey,$inout5,$inout5+	vpxor		$T1,$Z2,$Z2++	  vmovups	0x70-0x80($key),$rndkey+	vpslldq		\$8,$Z2,$Z1+	vpxor		$T2,$Z0,$Z0+	vmovdqu		0x10($const),$Hkey	# .Lpoly++	  vaesenc	$rndkey,$inout0,$inout0+	vpxor		$Xi,$Z3,$Z3+	  vaesenc	$rndkey,$inout1,$inout1+	vpxor		$Z1,$Z0,$Z0+	movbe		0x18($in0),%r13+	  vaesenc	$rndkey,$inout2,$inout2+	movbe		0x10($in0),%r12+	vpalignr	\$8,$Z0,$Z0,$Ii		# 1st phase+	vpclmulqdq	\$0x10,$Hkey,$Z0,$Z0+	mov		%r13,0x60+8(%rsp)+	  vaesenc	$rndkey,$inout3,$inout3+	mov		%r12,0x68+8(%rsp)+	  vaesenc	$rndkey,$inout4,$inout4+	  vmovups	0x80-0x80($key),$T1	# borrow $T1 for $rndkey+	  vaesenc	$rndkey,$inout5,$inout5++	  vaesenc	$T1,$inout0,$inout0+	  vmovups	0x90-0x80($key),$rndkey+	  vaesenc	$T1,$inout1,$inout1+	vpsrldq		\$8,$Z2,$Z2+	  vaesenc	$T1,$inout2,$inout2+	vpxor		$Z2,$Z3,$Z3+	  vaesenc	$T1,$inout3,$inout3+	vpxor		$Ii,$Z0,$Z0+	movbe		0x08($in0),%r13+	  vaesenc	$T1,$inout4,$inout4+	movbe		0x00($in0),%r12+	  vaesenc	$T1,$inout5,$inout5+	  vmovups	0xa0-0x80($key),$T1+	  cmp		\$11,$rounds+	  jb		.Lenc_tail		# 128-bit key++	  vaesenc	$rndkey,$inout0,$inout0+	  vaesenc	$rndkey,$inout1,$inout1+	  vaesenc	$rndkey,$inout2,$inout2+	  vaesenc	$rndkey,$inout3,$inout3+	  vaesenc	$rndkey,$inout4,$inout4+	  vaesenc	$rndkey,$inout5,$inout5++	  vaesenc	$T1,$inout0,$inout0+	  vaesenc	$T1,$inout1,$inout1+	  vaesenc	$T1,$inout2,$inout2+	  vaesenc	$T1,$inout3,$inout3+	  vaesenc	$T1,$inout4,$inout4+	  vmovups	0xb0-0x80($key),$rndkey+	  vaesenc	$T1,$inout5,$inout5+	  vmovups	0xc0-0x80($key),$T1+	  je		.Lenc_tail		# 192-bit key++	  vaesenc	$rndkey,$inout0,$inout0+	  vaesenc	$rndkey,$inout1,$inout1+	  vaesenc	$rndkey,$inout2,$inout2+	  vaesenc	$rndkey,$inout3,$inout3+	  vaesenc	$rndkey,$inout4,$inout4+	  vaesenc	$rndkey,$inout5,$inout5++	  vaesenc	$T1,$inout0,$inout0+	  vaesenc	$T1,$inout1,$inout1+	  vaesenc	$T1,$inout2,$inout2+	  vaesenc	$T1,$inout3,$inout3+	  vaesenc	$T1,$inout4,$inout4+	  vmovups	0xd0-0x80($key),$rndkey+	  vaesenc	$T1,$inout5,$inout5+	  vmovups	0xe0-0x80($key),$T1+	  jmp		.Lenc_tail		# 256-bit key++.align	32+.Lhandle_ctr32:+	vmovdqu		($const),$Ii		# borrow $Ii for .Lbswap_mask+	  vpshufb	$Ii,$T1,$Z2		# byte-swap counter+	  vmovdqu	0x30($const),$Z1	# borrow $Z1, .Ltwo_lsb+	  vpaddd	0x40($const),$Z2,$inout1	# .Lone_lsb+	  vpaddd	$Z1,$Z2,$inout2+	vmovdqu		0x00-0x20($Xip),$Hkey	# $Hkey^1+	  vpaddd	$Z1,$inout1,$inout3+	  vpshufb	$Ii,$inout1,$inout1+	  vpaddd	$Z1,$inout2,$inout4+	  vpshufb	$Ii,$inout2,$inout2+	  vpxor		$rndkey,$inout1,$inout1+	  vpaddd	$Z1,$inout3,$inout5+	  vpshufb	$Ii,$inout3,$inout3+	  vpxor		$rndkey,$inout2,$inout2+	  vpaddd	$Z1,$inout4,$T1		# byte-swapped next counter value+	  vpshufb	$Ii,$inout4,$inout4+	  vpshufb	$Ii,$inout5,$inout5+	  vpshufb	$Ii,$T1,$T1		# next counter value+	jmp		.Lresume_ctr32++.align	32+.Lenc_tail:+	  vaesenc	$rndkey,$inout0,$inout0+	vmovdqu		$Z3,16+8(%rsp)		# postpone vpxor $Z3,$Xi,$Xi+	vpalignr	\$8,$Z0,$Z0,$Xi		# 2nd phase+	  vaesenc	$rndkey,$inout1,$inout1+	vpclmulqdq	\$0x10,$Hkey,$Z0,$Z0+	  vpxor		0x00($inp),$T1,$T2+	  vaesenc	$rndkey,$inout2,$inout2+	  vpxor		0x10($inp),$T1,$Ii+	  vaesenc	$rndkey,$inout3,$inout3+	  vpxor		0x20($inp),$T1,$Z1+	  vaesenc	$rndkey,$inout4,$inout4+	  vpxor		0x30($inp),$T1,$Z2+	  vaesenc	$rndkey,$inout5,$inout5+	  vpxor		0x40($inp),$T1,$Z3+	  vpxor		0x50($inp),$T1,$Hkey+	  vmovdqu	($ivp),$T1		# load next counter value++	  vaesenclast	$T2,$inout0,$inout0+	  vmovdqu	0x20($const),$T2	# borrow $T2, .Lone_msb+	  vaesenclast	$Ii,$inout1,$inout1+	 vpaddb		$T2,$T1,$Ii+	mov		%r13,0x70+8(%rsp)+	lea		0x60($inp),$inp+	  vaesenclast	$Z1,$inout2,$inout2+	 vpaddb		$T2,$Ii,$Z1+	mov		%r12,0x78+8(%rsp)+	lea		0x60($out),$out+	  vmovdqu	0x00-0x80($key),$rndkey+	  vaesenclast	$Z2,$inout3,$inout3+	 vpaddb		$T2,$Z1,$Z2+	  vaesenclast	$Z3, $inout4,$inout4+	 vpaddb		$T2,$Z2,$Z3+	  vaesenclast	$Hkey,$inout5,$inout5+	 vpaddb		$T2,$Z3,$Hkey++	add		\$0x60,$ret+	sub		\$0x6,$len+	jc		.L6x_done++	  vmovups	$inout0,-0x60($out)	# save output+	 vpxor		$rndkey,$T1,$inout0+	  vmovups	$inout1,-0x50($out)+	 vmovdqa	$Ii,$inout1		# 0 latency+	  vmovups	$inout2,-0x40($out)+	 vmovdqa	$Z1,$inout2		# 0 latency+	  vmovups	$inout3,-0x30($out)+	 vmovdqa	$Z2,$inout3		# 0 latency+	  vmovups	$inout4,-0x20($out)+	 vmovdqa	$Z3,$inout4		# 0 latency+	  vmovups	$inout5,-0x10($out)+	 vmovdqa	$Hkey,$inout5		# 0 latency+	vmovdqu		0x20+8(%rsp),$Z3	# I[5]+	jmp		.Loop6x++.L6x_done:+	vpxor		16+8(%rsp),$Xi,$Xi	# modulo-scheduled+	vpxor		$Z0,$Xi,$Xi		# modulo-scheduled++	ret+.cfi_endproc+.size	_aesni_ctr32_ghash_6x,.-_aesni_ctr32_ghash_6x+___+######################################################################+#+# size_t aesni_gcm_[en|de]crypt(const void *inp, void *out, size_t len,+#		const AES_KEY *key, unsigned char iv[16],+#		struct { u128 Xi,H,Htbl[9]; } *Xip);+$code.=<<___;+.globl	aesni_gcm_decrypt+.type	aesni_gcm_decrypt,\@function,6,"unwind"+.align	32+aesni_gcm_decrypt:+.cfi_startproc+	push	%rbp+.cfi_push	%rbp+	mov	%rsp,%rbp+.cfi_def_cfa_register	%rbp+	xor	$ret,$ret+	cmp	\$0x60,$len			# minimal accepted length+	jb	.Lgcm_dec_abort++	push	%rbx+.cfi_push	%rbx+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15+___+$code.=<<___ if ($win64);+	lea	-0xa8(%rsp),%rsp+.cfi_alloca	0xa8+	movaps	%xmm6,-0xd0(%rbp)+	movaps	%xmm7,-0xc0(%rbp)+	movaps	%xmm8,-0xb0(%rbp)+	movaps	%xmm9,-0xa0(%rbp)+	movaps	%xmm10,-0x90(%rbp)+	movaps	%xmm11,-0x80(%rbp)+	movaps	%xmm12,-0x70(%rbp)+	movaps	%xmm13,-0x60(%rbp)+	movaps	%xmm14,-0x50(%rbp)+	movaps	%xmm15,-0x40(%rbp)+.cfi_offset	%xmm6-%xmm15,-0xe0+___+$code.=<<___;+.cfi_end_prologue+	vzeroupper++	vmovdqu		($ivp),$T1		# input counter value+	add		\$-128,%rsp+	mov		12($ivp),$counter+	lea		.Lbswap_mask(%rip),$const+	lea		-0x80($key),$in0	# borrow $in0+	mov		\$0xf80,$end0		# borrow $end0+	vmovdqu		($Xip),$Xi		# load Xi+	and		\$-128,%rsp		# ensure stack alignment+	vmovdqu		($const),$Ii		# borrow $Ii for .Lbswap_mask+	lea		0x80($key),$key		# size optimization+	lea		0x20+0x20($Xip),$Xip	# size optimization+	mov		0xf0-0x80($key),$rounds+	vpshufb		$Ii,$Xi,$Xi++	and		$end0,$in0+	and		%rsp,$end0+	sub		$in0,$end0+	jc		.Ldec_no_key_aliasing+	cmp		\$768,$end0+	jnc		.Ldec_no_key_aliasing+	sub		$end0,%rsp		# avoid aliasing with key+.Ldec_no_key_aliasing:++	vmovdqu		0x50($inp),$Z3		# I[5]+	lea		($inp),$in0+	vmovdqu		0x40($inp),$Z0+	lea		-0xc0($inp,$len),$end0+	vmovdqu		0x30($inp),$Z1+	shr		\$4,$len+	xor		$ret,$ret+	vmovdqu		0x20($inp),$Z2+	 vpshufb	$Ii,$Z3,$Z3		# passed to _aesni_ctr32_ghash_6x+	vmovdqu		0x10($inp),$T2+	 vpshufb	$Ii,$Z0,$Z0+	vmovdqu		($inp),$Hkey+	 vpshufb	$Ii,$Z1,$Z1+	vmovdqu		$Z0,0x30(%rsp)+	 vpshufb	$Ii,$Z2,$Z2+	vmovdqu		$Z1,0x40(%rsp)+	 vpshufb	$Ii,$T2,$T2+	vmovdqu		$Z2,0x50(%rsp)+	 vpshufb	$Ii,$Hkey,$Hkey+	vmovdqu		$T2,0x60(%rsp)+	vmovdqu		$Hkey,0x70(%rsp)++	call		_aesni_ctr32_ghash_6x++	vmovups		$inout0,-0x60($out)	# save output+	vmovups		$inout1,-0x50($out)+	vmovups		$inout2,-0x40($out)+	vmovups		$inout3,-0x30($out)+	vmovups		$inout4,-0x20($out)+	vmovups		$inout5,-0x10($out)++	vpshufb		($const),$Xi,$Xi	# .Lbswap_mask+	vmovdqu		$Xi,-0x40($Xip)		# output Xi++	vzeroupper+___+$code.=<<___ if ($win64);+	movaps	-0xd0(%rbp),%xmm6+	movaps	-0xc0(%rbp),%xmm7+	movaps	-0xb0(%rbp),%xmm8+	movaps	-0xa0(%rbp),%xmm9+	movaps	-0x90(%rbp),%xmm10+	movaps	-0x80(%rbp),%xmm11+	movaps	-0x70(%rbp),%xmm12+	movaps	-0x60(%rbp),%xmm13+	movaps	-0x50(%rbp),%xmm14+	movaps	-0x40(%rbp),%xmm15+___+$code.=<<___;+	mov	-0x28(%rbp),%r15+	mov	-0x20(%rbp),%r14+	mov	-0x18(%rbp),%r13+	mov	-0x10(%rbp),%r12+	mov	-0x08(%rbp),%rbx+	mov	%rbp,%rsp			# restore %rsp+.cfi_def_cfa_register	%rsp+.Lgcm_dec_abort:+	pop	%rbp+.cfi_pop	%rbp+.cfi_epilogue+	ret+.cfi_endproc+.size	aesni_gcm_decrypt,.-aesni_gcm_decrypt+___++$code.=<<___;+.type	_aesni_ctr32_6x,\@abi-omnipotent+.align	32+_aesni_ctr32_6x:+.cfi_startproc+	vmovdqu		0x00-0x80($key),$Z0	# borrow $Z0 for $rndkey+	vmovdqu		0x20($const),$T2	# borrow $T2, .Lone_msb+	lea		-1($rounds),%r13+	vmovups		0x10-0x80($key),$rndkey+	lea		0x20-0x80($key),%r12+	vpxor		$Z0,$T1,$inout0+	add		\$`6<<24`,$counter+	jc		.Lhandle_ctr32_2+	vpaddb		$T2,$T1,$inout1+	vpaddb		$T2,$inout1,$inout2+	vpxor		$Z0,$inout1,$inout1+	vpaddb		$T2,$inout2,$inout3+	vpxor		$Z0,$inout2,$inout2+	vpaddb		$T2,$inout3,$inout4+	vpxor		$Z0,$inout3,$inout3+	vpaddb		$T2,$inout4,$inout5+	vpxor		$Z0,$inout4,$inout4+	vpaddb		$T2,$inout5,$T1+	vpxor		$Z0,$inout5,$inout5+	jmp		.Loop_ctr32++.align	16+.Loop_ctr32:+	vaesenc		$rndkey,$inout0,$inout0+	vaesenc		$rndkey,$inout1,$inout1+	vaesenc		$rndkey,$inout2,$inout2+	vaesenc		$rndkey,$inout3,$inout3+	vaesenc		$rndkey,$inout4,$inout4+	vaesenc		$rndkey,$inout5,$inout5+	vmovups		(%r12),$rndkey+	lea		0x10(%r12),%r12+	dec		%r13d+	jnz		.Loop_ctr32++	vmovdqu		(%r12),$Hkey		# last round key+	vaesenc		$rndkey,$inout0,$inout0+	vpxor		0x00($inp),$Hkey,$Z0+	vaesenc		$rndkey,$inout1,$inout1+	vpxor		0x10($inp),$Hkey,$Z1+	vaesenc		$rndkey,$inout2,$inout2+	vpxor		0x20($inp),$Hkey,$Z2+	vaesenc		$rndkey,$inout3,$inout3+	vpxor		0x30($inp),$Hkey,$Xi+	vaesenc		$rndkey,$inout4,$inout4+	vpxor		0x40($inp),$Hkey,$T2+	vaesenc		$rndkey,$inout5,$inout5+	vpxor		0x50($inp),$Hkey,$Hkey+	lea		0x60($inp),$inp++	vaesenclast	$Z0,$inout0,$inout0+	vaesenclast	$Z1,$inout1,$inout1+	vaesenclast	$Z2,$inout2,$inout2+	vaesenclast	$Xi,$inout3,$inout3+	vaesenclast	$T2,$inout4,$inout4+	vaesenclast	$Hkey,$inout5,$inout5+	vmovups		$inout0,0x00($out)+	vmovups		$inout1,0x10($out)+	vmovups		$inout2,0x20($out)+	vmovups		$inout3,0x30($out)+	vmovups		$inout4,0x40($out)+	vmovups		$inout5,0x50($out)+	lea		0x60($out),$out++	ret+.align	32+.Lhandle_ctr32_2:+	vpshufb		$Ii,$T1,$Z2		# byte-swap counter+	vmovdqu		0x30($const),$Z1	# borrow $Z1, .Ltwo_lsb+	vpaddd		0x40($const),$Z2,$inout1	# .Lone_lsb+	vpaddd		$Z1,$Z2,$inout2+	vpaddd		$Z1,$inout1,$inout3+	vpshufb		$Ii,$inout1,$inout1+	vpaddd		$Z1,$inout2,$inout4+	vpshufb		$Ii,$inout2,$inout2+	vpxor		$Z0,$inout1,$inout1+	vpaddd		$Z1,$inout3,$inout5+	vpshufb		$Ii,$inout3,$inout3+	vpxor		$Z0,$inout2,$inout2+	vpaddd		$Z1,$inout4,$T1		# byte-swapped next counter value+	vpshufb		$Ii,$inout4,$inout4+	vpxor		$Z0,$inout3,$inout3+	vpshufb		$Ii,$inout5,$inout5+	vpxor		$Z0,$inout4,$inout4+	vpshufb		$Ii,$T1,$T1		# next counter value+	vpxor		$Z0,$inout5,$inout5+	jmp	.Loop_ctr32+.cfi_endproc+.size	_aesni_ctr32_6x,.-_aesni_ctr32_6x++.globl	aesni_gcm_encrypt+.type	aesni_gcm_encrypt,\@function,6,"unwind"+.align	32+aesni_gcm_encrypt:+.cfi_startproc+	push	%rbp+.cfi_push	%rbp+	mov	%rsp,%rbp+.cfi_def_cfa_register	%rbp+	xor	$ret,$ret+	cmp	\$0x60*3,$len			# minimal accepted length+	jb	.Lgcm_enc_abort++	push	%rbx+.cfi_push	%rbx+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15+___+$code.=<<___ if ($win64);+	lea	-0xa8(%rsp),%rsp+.cfi_alloca	0xa8+	movaps	%xmm6,-0xd0(%rbp)+	movaps	%xmm7,-0xc0(%rbp)+	movaps	%xmm8,-0xb0(%rbp)+	movaps	%xmm9,-0xa0(%rbp)+	movaps	%xmm10,-0x90(%rbp)+	movaps	%xmm11,-0x80(%rbp)+	movaps	%xmm12,-0x70(%rbp)+	movaps	%xmm13,-0x60(%rbp)+	movaps	%xmm14,-0x50(%rbp)+	movaps	%xmm15,-0x40(%rbp)+.cfi_offset	%xmm6-%xmm15,-0xe0+___+$code.=<<___;+.cfi_end_prologue+	vzeroupper++	vmovdqu		($ivp),$T1		# input counter value+	add		\$-128,%rsp+	mov		12($ivp),$counter+	lea		.Lbswap_mask(%rip),$const+	lea		-0x80($key),$in0	# borrow $in0+	mov		\$0xf80,$end0		# borrow $end0+	lea		0x80($key),$key		# size optimization+	vmovdqu		($const),$Ii		# borrow $Ii for .Lbswap_mask+	and		\$-128,%rsp		# ensure stack alignment+	mov		0xf0-0x80($key),$rounds++	and		$end0,$in0+	and		%rsp,$end0+	sub		$in0,$end0+	jc		.Lenc_no_key_aliasing+	cmp		\$768,$end0+	jnc		.Lenc_no_key_aliasing+	sub		$end0,%rsp		# avoid aliasing with key+.Lenc_no_key_aliasing:++	lea		($out),$in0+	lea		-0xc0($out,$len),$end0+	shr		\$4,$len++	call		_aesni_ctr32_6x+	vpshufb		$Ii,$inout0,$Xi		# save bswapped output on stack+	vpshufb		$Ii,$inout1,$T2+	vmovdqu		$Xi,0x70(%rsp)+	vpshufb		$Ii,$inout2,$Z0+	vmovdqu		$T2,0x60(%rsp)+	vpshufb		$Ii,$inout3,$Z1+	vmovdqu		$Z0,0x50(%rsp)+	vpshufb		$Ii,$inout4,$Z2+	vmovdqu		$Z1,0x40(%rsp)+	vpshufb		$Ii,$inout5,$Z3		# passed to _aesni_ctr32_ghash_6x+	vmovdqu		$Z2,0x30(%rsp)++	call		_aesni_ctr32_6x++	vmovdqu		($Xip),$Xi		# load Xi+	lea		0x20+0x20($Xip),$Xip	# size optimization+	sub		\$12,$len+	mov		\$0x60*2,$ret+	vpshufb		$Ii,$Xi,$Xi++	call		_aesni_ctr32_ghash_6x+	vmovdqu		0x20(%rsp),$Z3		# I[5]+	 vmovdqu	($const),$Ii		# borrow $Ii for .Lbswap_mask+	vmovdqu		0x00-0x20($Xip),$Hkey	# $Hkey^1+	vpunpckhqdq	$Z3,$Z3,$T1+	vmovdqu		0x20-0x20($Xip),$rndkey	# borrow $rndkey for $HK+	 vmovups	$inout0,-0x60($out)	# save output+	 vpshufb	$Ii,$inout0,$inout0	# but keep bswapped copy+	vpxor		$Z3,$T1,$T1+	 vmovups	$inout1,-0x50($out)+	 vpshufb	$Ii,$inout1,$inout1+	 vmovups	$inout2,-0x40($out)+	 vpshufb	$Ii,$inout2,$inout2+	 vmovups	$inout3,-0x30($out)+	 vpshufb	$Ii,$inout3,$inout3+	 vmovups	$inout4,-0x20($out)+	 vpshufb	$Ii,$inout4,$inout4+	 vmovups	$inout5,-0x10($out)+	 vpshufb	$Ii,$inout5,$inout5+	 vmovdqu	$inout0,0x10(%rsp)	# free $inout0+___+{ my ($HK,$T3)=($rndkey,$inout0);++$code.=<<___;+	 vmovdqu	0x30(%rsp),$Z2		# I[4]+	 vmovdqu	0x10-0x20($Xip),$Ii	# borrow $Ii for $Hkey^2+	 vpunpckhqdq	$Z2,$Z2,$T2+	vpclmulqdq	\$0x00,$Hkey,$Z3,$Z1+	 vpxor		$Z2,$T2,$T2+	vpclmulqdq	\$0x11,$Hkey,$Z3,$Z3+	vpclmulqdq	\$0x00,$HK,$T1,$T1++	 vmovdqu	0x40(%rsp),$T3		# I[3]+	vpclmulqdq	\$0x00,$Ii,$Z2,$Z0+	 vmovdqu	0x30-0x20($Xip),$Hkey	# $Hkey^3+	vpxor		$Z1,$Z0,$Z0+	 vpunpckhqdq	$T3,$T3,$Z1+	vpclmulqdq	\$0x11,$Ii,$Z2,$Z2+	 vpxor		$T3,$Z1,$Z1+	vpxor		$Z3,$Z2,$Z2+	vpclmulqdq	\$0x10,$HK,$T2,$T2+	 vmovdqu	0x50-0x20($Xip),$HK+	vpxor		$T1,$T2,$T2++	 vmovdqu	0x50(%rsp),$T1		# I[2]+	vpclmulqdq	\$0x00,$Hkey,$T3,$Z3+	 vmovdqu	0x40-0x20($Xip),$Ii	# borrow $Ii for $Hkey^4+	vpxor		$Z0,$Z3,$Z3+	 vpunpckhqdq	$T1,$T1,$Z0+	vpclmulqdq	\$0x11,$Hkey,$T3,$T3+	 vpxor		$T1,$Z0,$Z0+	vpxor		$Z2,$T3,$T3+	vpclmulqdq	\$0x00,$HK,$Z1,$Z1+	vpxor		$T2,$Z1,$Z1++	 vmovdqu	0x60(%rsp),$T2		# I[1]+	vpclmulqdq	\$0x00,$Ii,$T1,$Z2+	 vmovdqu	0x60-0x20($Xip),$Hkey	# $Hkey^5+	vpxor		$Z3,$Z2,$Z2+	 vpunpckhqdq	$T2,$T2,$Z3+	vpclmulqdq	\$0x11,$Ii,$T1,$T1+	 vpxor		$T2,$Z3,$Z3+	vpxor		$T3,$T1,$T1+	vpclmulqdq	\$0x10,$HK,$Z0,$Z0+	 vmovdqu	0x80-0x20($Xip),$HK+	vpxor		$Z1,$Z0,$Z0++	 vpxor		0x70(%rsp),$Xi,$Xi	# accumulate I[0]+	vpclmulqdq	\$0x00,$Hkey,$T2,$Z1+	 vmovdqu	0x70-0x20($Xip),$Ii	# borrow $Ii for $Hkey^6+	 vpunpckhqdq	$Xi,$Xi,$T3+	vpxor		$Z2,$Z1,$Z1+	vpclmulqdq	\$0x11,$Hkey,$T2,$T2+	 vpxor		$Xi,$T3,$T3+	vpxor		$T1,$T2,$T2+	vpclmulqdq	\$0x00,$HK,$Z3,$Z3+	vpxor		$Z0,$Z3,$Z0++	vpclmulqdq	\$0x00,$Ii,$Xi,$Z2+	 vmovdqu	0x00-0x20($Xip),$Hkey	# $Hkey^1+	 vpunpckhqdq	$inout5,$inout5,$T1+	vpclmulqdq	\$0x11,$Ii,$Xi,$Xi+	 vpxor		$inout5,$T1,$T1+	vpxor		$Z1,$Z2,$Z1+	vpclmulqdq	\$0x10,$HK,$T3,$T3+	 vmovdqu	0x20-0x20($Xip),$HK+	vpxor		$T2,$Xi,$Z3+	vpxor		$Z0,$T3,$Z2++	 vmovdqu	0x10-0x20($Xip),$Ii	# borrow $Ii for $Hkey^2+	  vpxor		$Z1,$Z3,$T3		# aggregated Karatsuba post-processing+	vpclmulqdq	\$0x00,$Hkey,$inout5,$Z0+	  vpxor		$T3,$Z2,$Z2+	 vpunpckhqdq	$inout4,$inout4,$T2+	vpclmulqdq	\$0x11,$Hkey,$inout5,$inout5+	 vpxor		$inout4,$T2,$T2+	  vpslldq	\$8,$Z2,$T3+	vpclmulqdq	\$0x00,$HK,$T1,$T1+	  vpxor		$T3,$Z1,$Xi+	  vpsrldq	\$8,$Z2,$Z2+	  vpxor		$Z2,$Z3,$Z3++	vpclmulqdq	\$0x00,$Ii,$inout4,$Z1+	 vmovdqu	0x30-0x20($Xip),$Hkey	# $Hkey^3+	vpxor		$Z0,$Z1,$Z1+	 vpunpckhqdq	$inout3,$inout3,$T3+	vpclmulqdq	\$0x11,$Ii,$inout4,$inout4+	 vpxor		$inout3,$T3,$T3+	vpxor		$inout5,$inout4,$inout4+	  vpalignr	\$8,$Xi,$Xi,$inout5	# 1st phase+	vpclmulqdq	\$0x10,$HK,$T2,$T2+	 vmovdqu	0x50-0x20($Xip),$HK+	vpxor		$T1,$T2,$T2++	vpclmulqdq	\$0x00,$Hkey,$inout3,$Z0+	 vmovdqu	0x40-0x20($Xip),$Ii	# borrow $Ii for $Hkey^4+	vpxor		$Z1,$Z0,$Z0+	 vpunpckhqdq	$inout2,$inout2,$T1+	vpclmulqdq	\$0x11,$Hkey,$inout3,$inout3+	 vpxor		$inout2,$T1,$T1+	vpxor		$inout4,$inout3,$inout3+	  vxorps	0x10(%rsp),$Z3,$Z3	# accumulate $inout0+	vpclmulqdq	\$0x00,$HK,$T3,$T3+	vpxor		$T2,$T3,$T3++	  vpclmulqdq	\$0x10,0x10($const),$Xi,$Xi+	  vxorps	$inout5,$Xi,$Xi++	vpclmulqdq	\$0x00,$Ii,$inout2,$Z1+	 vmovdqu	0x60-0x20($Xip),$Hkey	# $Hkey^5+	vpxor		$Z0,$Z1,$Z1+	 vpunpckhqdq	$inout1,$inout1,$T2+	vpclmulqdq	\$0x11,$Ii,$inout2,$inout2+	 vpxor		$inout1,$T2,$T2+	  vpalignr	\$8,$Xi,$Xi,$inout5	# 2nd phase+	vpxor		$inout3,$inout2,$inout2+	vpclmulqdq	\$0x10,$HK,$T1,$T1+	 vmovdqu	0x80-0x20($Xip),$HK+	vpxor		$T3,$T1,$T1++	  vxorps	$Z3,$inout5,$inout5+	  vpclmulqdq	\$0x10,0x10($const),$Xi,$Xi+	  vxorps	$inout5,$Xi,$Xi++	vpclmulqdq	\$0x00,$Hkey,$inout1,$Z0+	 vmovdqu	0x70-0x20($Xip),$Ii	# borrow $Ii for $Hkey^6+	vpxor		$Z1,$Z0,$Z0+	 vpunpckhqdq	$Xi,$Xi,$T3+	vpclmulqdq	\$0x11,$Hkey,$inout1,$inout1+	 vpxor		$Xi,$T3,$T3+	vpxor		$inout2,$inout1,$inout1+	vpclmulqdq	\$0x00,$HK,$T2,$T2+	vpxor		$T1,$T2,$T2++	vpclmulqdq	\$0x00,$Ii,$Xi,$Z1+	vpclmulqdq	\$0x11,$Ii,$Xi,$Z3+	vpxor		$Z0,$Z1,$Z1+	vpclmulqdq	\$0x10,$HK,$T3,$Z2+	vpxor		$inout1,$Z3,$Z3+	vpxor		$T2,$Z2,$Z2++	vpxor		$Z1,$Z3,$Z0		# aggregated Karatsuba post-processing+	vpxor		$Z0,$Z2,$Z2+	vpslldq		\$8,$Z2,$T1+	vmovdqu		0x10($const),$Hkey	# .Lpoly+	vpsrldq		\$8,$Z2,$Z2+	vpxor		$T1,$Z1,$Xi+	vpxor		$Z2,$Z3,$Z3++	vpalignr	\$8,$Xi,$Xi,$T2		# 1st phase+	vpclmulqdq	\$0x10,$Hkey,$Xi,$Xi+	vpxor		$T2,$Xi,$Xi++	vpalignr	\$8,$Xi,$Xi,$T2		# 2nd phase+	vpclmulqdq	\$0x10,$Hkey,$Xi,$Xi+	vpxor		$Z3,$T2,$T2+	vpxor		$T2,$Xi,$Xi+___+}+$code.=<<___;+	vpshufb		($const),$Xi,$Xi	# .Lbswap_mask+	vmovdqu		$Xi,-0x40($Xip)		# output Xi++	vzeroupper+___+$code.=<<___ if ($win64);+	movaps	-0xd0(%rbp),%xmm6+	movaps	-0xc0(%rbp),%xmm7+	movaps	-0xb0(%rbp),%xmm8+	movaps	-0xa0(%rbp),%xmm9+	movaps	-0x90(%rbp),%xmm10+	movaps	-0x80(%rbp),%xmm11+	movaps	-0x70(%rbp),%xmm12+	movaps	-0x60(%rbp),%xmm13+	movaps	-0x50(%rbp),%xmm14+	movaps	-0x40(%rbp),%xmm15+___+$code.=<<___;+	mov	-0x28(%rbp),%r15+	mov	-0x20(%rbp),%r14+	mov	-0x18(%rbp),%r13+	mov	-0x10(%rbp),%r12+	mov	-0x08(%rbp),%rbx+	mov	%rbp,%rsp			# restore %rsp+.cfi_def_cfa_register	%rsp+.Lgcm_enc_abort:+	pop	%rbp+.cfi_pop	%rbp+.cfi_epilogue+	ret+.cfi_endproc+.size	aesni_gcm_encrypt,.-aesni_gcm_encrypt+___++$code.=<<___;+.align	64+.Lbswap_mask:+	.byte	15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,0+.Lpoly:+	.byte	0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0xc2+.Lone_msb:+	.byte	0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1+.Ltwo_lsb:+	.byte	2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0+.Lone_lsb:+	.byte	1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0+.asciz	"AES-NI GCM module for x86_64, CRYPTOGAMS by <appro\@openssl.org>"+.align	64+___+}}} else {{{+$code=<<___;	# assembler is too old+.text++.globl	aesni_gcm_encrypt+.type	aesni_gcm_encrypt,\@abi-omnipotent+aesni_gcm_encrypt:+.cfi_startproc+	xor	%eax,%eax+	ret+.cfi_endproc+.size	aesni_gcm_encrypt,.-aesni_gcm_encrypt++.globl	aesni_gcm_decrypt+.type	aesni_gcm_decrypt,\@abi-omnipotent+aesni_gcm_decrypt:+.cfi_startproc+	xor	%eax,%eax+	ret+.cfi_endproc+.size	aesni_gcm_decrypt,.-aesni_gcm_decrypt+___+}}}++$code =~ s/\`([^\`]*)\`/eval($1)/gem;++print $code;++close STDOUT or die "error closing STDOUT: $!";
+ cbits/asm/arm-xlate.pl view
@@ -0,0 +1,467 @@+#! /usr/bin/env perl+#+# ARM assembler distiller/adapter by \@dot-asm.++use strict;++################################################################+# Recognized "flavour"-s are:+#+# linux[32|64]	GNU assembler, effectively pass-through+# ios[32|64]	global symbols' decorations, PIC tweaks, etc.+# win[32|64]	Visual Studio armasm-specific directives+# coff[32|64]	e.g. clang --target=arm-windows ...+# cheri64	L64P128 platform+#+my $flavour = shift;+   $flavour = "linux" if (!$flavour or $flavour eq "void");++my $output = shift;+open STDOUT,">$output" || die "can't open $output: $!";++my %GLOBALS;+my $dotinlocallabels = ($flavour !~ /ios/) ? 1 : 0;+my $in_proc;	# used with 'windows' flavour++################################################################+# directives which need special treatment on different platforms+################################################################+my $arch = sub { } if ($flavour !~ /linux|coff64/);# omit .arch+my $fpu  = sub { } if ($flavour !~ /linux/);       # omit .fpu++my $rodata = sub {+    SWITCH: for ($flavour) {+	/linux|cheri/	&& return ".section\t.rodata";+	/ios/		&& return ".section\t__TEXT,__const";+	/coff/		&& return ".section\t.rdata,\"dr\"";+	/win/		&& return "\tAREA\t|.rdata|,DATA,READONLY,ALIGN=8";+	last;+    }+};++my $hidden = sub {+    if ($flavour =~ /ios/)	{ ".private_extern\t".join(',',@_); }+} if ($flavour !~ /linux|cheri/);++my $comm = sub {+    my @args = split(/,\s*/,shift);+    my $name = @args[0];+    my $global = \$GLOBALS{$name};+    my $ret;++    if ($flavour =~ /ios32/)	{+	$ret = ".comm\t_$name,@args[1]\n";+	$ret .= ".non_lazy_symbol_pointer\n";+	$ret .= "$name:\n";+	$ret .= ".indirect_symbol\t_$name\n";+	$ret .= ".long\t0\n";+	$ret .= ".previous";+	$name = "_$name";+    } elsif ($flavour =~ /ios64/) {+	$name = "_$name";+	$ret = ".comm\t$name,@args[1]";+    } elsif ($flavour =~ /win/) {+	$ret = "\tCOMMON\t|$name|,@args[1]";+    } elsif ($flavour =~ /coff/) {+	$ret = ".comm\t$name,@args[1]";+    } else {+	$ret = ".comm\t".join(',',@args);+    }++    $$global = $name;+    $ret;+};++my $globl = sub {+    my $name = shift;+    my $global = \$GLOBALS{$name};+    my $ret;++    SWITCH: for ($flavour) {+	/ios/		&& do { $name = "_$name"; last; };+	/win/		&& do { $ret = ""; last; };+    }++    $ret = ".globl	$name" if (!defined($ret));+    $$global = $name;+    $ret;+};+my $global = $globl;++my $extern = sub {+    &$globl(@_);+    if ($flavour =~ /win/) {+	return "\tEXTERN\t@_";+    }+    return;	# return nothing+};++my $type = sub {+    my $arg = join(',',@_);+    my $ret;++    SWITCH: for ($flavour) {+	/ios32/		&& do { if ($arg =~ /(\w+),\s*%function/) {+				    $ret = "#ifdef __thumb2__\n" .+					   ".thumb_func	$1\n" .+					   "#endif";+				}+				last;+			      };+	/win/		&& do { if ($arg =~ /(\w+),\s*%(function|object)/) {+				    my $type = "[DATA]";+				    if ($2 eq "function") {+					$in_proc = $1;+					$type = "[FUNC]";+				    }+				    $ret = $GLOBALS{$1} ? "\tEXPORT\t|$1|$type"+							: "";+				}+				last;+			      };+	/coff/		&& do { if ($arg =~ /(\w+),\s*%function/) {+				    $ret = ".def	$1;\n".+					   ".type	32;\n".+					   ".endef";+				}+				last;+			      };+    }+    return $ret;+} if ($flavour !~ /linux|cheri/);++my $size = sub {+    if ($in_proc && $flavour =~ /win/) {+	$in_proc = undef;+	return "\tENDP";+    }+} if ($flavour !~ /linux|cheri/);++my $inst = sub {+    if ($flavour =~ /win/)	{ "\tDCDU\t".join(',',@_); }+    else			{ ".long\t".join(',',@_);  }+} if ($flavour !~ /linux|cheri/);++my $asciz = sub {+    my $line = join(",",@_);+    if ($line =~ /^"(.*)"$/)+    {	if ($flavour =~ /win/) {+	    "\tDCB\t$line,0\n\tALIGN\t4";+	} else {+	    ".byte	" . join(",",unpack("C*",$1),0) . "\n.align	2";+	}+    } else {	"";	}+};++my $align = sub {+    "\tALIGN\t".2**@_[0];+} if ($flavour =~ /win/);+   $align = sub {+    ".p2align\t".@_[0];+} if ($flavour =~ /coff/);++my $byte = sub {+    "\tDCB\t".join(',',@_);+} if ($flavour =~ /win/);++my $short = sub {+    "\tDCWU\t".join(',',@_);+} if ($flavour =~ /win/);++my $word = sub {+    "\tDCDU\t".join(',',@_);+} if ($flavour =~ /win/);++my $long = $word if ($flavour =~ /win/);++my $quad = sub {+    "\tDCQU\t".join(',',@_);+} if ($flavour =~ /win/);++my $skip = sub {+    "\tSPACE\t".shift;+} if ($flavour =~ /win/);++my $code = sub {+    "\tCODE@_[0]";+} if ($flavour =~ /win/);++my $thumb = sub {	# .thumb should appear prior .text in source+    "# define ARM THUMB\n" .+    "\tTHUMB";+} if ($flavour =~ /win/);++my $text = sub {+    "\tAREA\t|.text|,CODE,ALIGN=8,".($flavour =~ /64/ ? "ARM64" : "ARM");+} if ($flavour =~ /win/);++my $syntax = sub {} if ($flavour =~ /win/);	# omit .syntax++my $rva = sub {+    # .rva directive comes in handy only on 32-bit Windows, i.e. it can+    # be used only in '#if defined(_WIN32) && !defined(_WIN64)' sections.+    # However! Corresponding compilers don't seem to bet on PIC, which+    # raises the question why would assembler programmer have to jump+    # through the hoops? But just in case, it would go as following:+    #+    #	ldr	r1,.LOPENSSL_armcap+    #	ldr	r2,.LOPENSSL_armcap+4+    #	adr	r0,.LOPENSSL_armcap+    #	bic	r1,r1,#1		; de-thumb-ify link.exe's ideas+    #	sub	r0,r0,r1		; r0 is image base now+    #	ldr	r0,[r0,r2]+    #	...+    #.LOPENSSL_armcap:+    #	.rva	.LOPENSSL_armcap	; self-reference+    #	.rva	OPENSSL_armcap_P	; real target+    #+    # Non-position-independent [and ISA-neutral] alternative is so much+    # simpler:+    #+    #	ldr	r0,.LOPENSSL_armcap+    #	ldr	r0,[r0]+    #	...+    #.LOPENSSL_armcap:+    #	.long	OPENSSL_armcap_P+    #+    "\tDCDU\t@_[0]\n\tRELOC\t2"+} if ($flavour =~ /win(?!64)/);++################################################################+# some broken instructions in Visual Studio armasm[64]...++my $it = sub {} if ($flavour =~ /win32/);	# omit 'it'++my $ext = sub {+    "\text8\t".join(',',@_);+} if ($flavour =~ /win64/);++my $csel = sub {+    my ($args,$comment) = split(m|\s*//|,shift);+    my @regs = split(m|,\s*|,$args);+    my $cond = pop(@regs);++    "\tcsel$cond\t".join(',',@regs);+} if ($flavour =~ /win64/);++my $csetm = sub {+    my ($args,$comment) = split(m|\s*//|,shift);+    my @regs = split(m|,\s*|,$args);+    my $cond = pop(@regs);++    "\tcsetm$cond\t".join(',',@regs);+} if ($flavour =~ /win64/);++# ... then conditional branch instructions are also broken, but+# maintaining all the variants is tedious, so I kludge-fix it+# elsewhere...++################################################################+# CHERI-specific synthetic instructions+my $scvalue = sub {+    my ($args,$comment) = split(m|\s*//|,shift);+    $args =~ s/\b(?:x([0-9]+)|(sp))\b/c$1$2/g;+    my @regs = split(m|,\s*|,$args);+    @regs[2] =~ s/\bc([0-9])\b/x$1/;++    "\tscvalue\t".join(',',@regs);+};++my $cadd = sub {+    my ($args,$comment) = split(m|\s*//|,shift);+    if ($flavour =~ /cheri/) {+	$args =~ s/\b(?:x([0-9]+)|(sp))\b/c$1$2/g;+    } else {+	$args =~ s/\bc([0-9]+)\b/x$1/g;+    }+    my @regs = split(m|,\s*|,$args);+    @regs[2] =~ s/c([0-9])/x$1/;++    "\tadd\t".join(',',@regs);+};++my $csub = sub {+    my ($args,$comment) = split(m|\s*//|,shift);+    if ($flavour =~ /cheri/) {+	$args =~ s/\b(?:x([0-9]+)|(sp))\b/c$1$2/g;+    } else {+	$args =~ s/\bc([0-9]+)\b/x$1/g;+    }+    my @regs = split(m|,\s*|,$args);+    @regs[2] =~ s/c([0-9])/x$1/;++    "\tsub\t".join(',',@regs);+};++my $cmov = sub {+    my $args = shift;+    if ($flavour =~ /cheri/) {+	$args =~ s/\b(?:x([0-9]+)|(sp))\b/c$1$2/g;+    } else {+	$args =~ s/\bc([0-9]+)\b/x$1/g;+    }++    "\tmov\t".$args;+};++my $adr = sub {+    my $args = shift;+    $args =~ s/\bx([0-9]+)\b/c$1/g;++    "\tadr\t".$args;+} if ($flavour =~ /cheri/);++################################################################+my $adrp = sub {+    my ($args,$comment) = split(m|\s*//|,shift);+    "\tadrp\t$args\@PAGE";+} if ($flavour =~ /ios64/);++my $paciasp = sub {+    ($flavour =~ /linux|cheri/) ? "\t.inst\t0xd503233f"+                                : &$inst(0xd503233f);+};++my $autiasp = sub {+    ($flavour =~ /linux|cheri/) ? "\t.inst\t0xd50323bf"+                                : &$inst(0xd50323bf);+};++sub range {+  my ($r,$sfx,$start,$end) = @_;++    join(",",map("$r$_$sfx",($start..$end)));+}++sub expand_line {+  my $line = shift;+  my @ret = ();++    pos($line)=0;++    while ($line =~ m/\G[^@\/\{\"]*/g) {+	if ($line =~ m/\G(@|\/\/|$)/gc) {+	    last;+	}+	elsif ($line =~ m/\G\{/gc) {+	    my $saved_pos = pos($line);+	    $line =~ s/\G([rdqv])([0-9]+)([^\-]*)\-\1([0-9]+)\3/range($1,$3,$2,$4)/e;+	    pos($line) = $saved_pos;+	    $line =~ m/\G[^\}]*\}/g;+	}+	elsif ($line =~ m/\G\"/gc) {+	    $line =~ m/\G[^\"]*\"/g;+	}+    }++    $line =~ s/\b(\w+)/$GLOBALS{$1} or $1/ge;++    if ($flavour =~ /cheri/) {+	$line =~ s/\[\s*(?:x([0-9]+)|(sp))\s*(,?.*)\]/[c$1$2$3]/;+    } else {+	$line =~ s/\bc((?:[0-9]+|zr))\b/x$1/g;+	$line =~ s/\bcsp\b/sp/g;+    }++    if ($flavour =~ /win/) {+	# adjust alignment hints, "[rN,:32]" -> "[rN@32]"+	$line =~ s/(\[\s*(?:r[0-9]+|sp))\s*,?\s*:([0-9]+\s*\])/$1\@$2/;+	# adjust local labels, ".Lwhatever" -> "|$Lwhatever|"+	$line =~ s/\.(L\w{2,})/|\$$1|/g;+	# omit "#:lo12:" on win64+	$line =~ s/#:lo12://;+    } elsif ($flavour =~ /coff(?!64)/) {+	$line =~ s/\.L(\w{2,})/(\$ML$1)/g;+    } elsif ($flavour =~ /ios64/) {+	$line =~ s/#:lo12:(\w+)/$1\@PAGEOFF/;+    }++    if ($flavour =~ /64/) {+	# "vX.Md[N]" -> "vX.d[N]+	$line =~ s/\b(v[0-9]+)\.[1-9]+([bhsd]\[[0-9]+\])/$1.$2/;+    }++    return $line;+}++if ($flavour =~ /win(32|64)/) {+    print<<___;+ GBLA __SIZEOF_POINTER__+__SIZEOF_POINTER__ SETA $1/8+___+}++while(my $line=<>) {++    if ($flavour =~ /win/) {+	if ($line =~ m/^#\s*(ifdef|ifndef|else|endif)\b(.*)/) {+	    my ($op, $arg) = ($1, $2);+	    $op = "if :def:"		if ($op eq "ifdef");+	    $op = "if :lnot::def:"	if ($op eq "ifndef");+	    print " ".$op.$arg."\n";+	    next;+	}+	$line =~ s|//.*||;+    }++    # fix up assembler-specific commentary delimiter+    $line =~ s/@(?=[\s@])/\;/g if ($flavour =~ /win|coff/);++    if ($line =~ m/^\s*(#|@|;|\/\/)/)	{ print $line; next; }++    $line =~ s|/\*.*\*/||;	# get rid of C-style comments...+    $line =~ s|^\s+||;		# ... and skip white spaces in beginning...+    $line =~ s|\s+$||;		# ... and at the end++    {+	$line =~ s|[\b\.]L(\w{2,})|L$1|g;	# common denominator for Locallabel+	$line =~ s|\bL(\w{2,})|\.L$1|g	if ($dotinlocallabels);+    }++    {+	$line =~ s|(^[\.\w]+)\:\s*||;+	my $label = $1;+	if ($label) {+	    $label = ($GLOBALS{$label} or $label);+	    if ($flavour =~ /win/) {+		$label =~ s|^\.L(?=\w)|\$L|;+		printf "|%s|%s", $label, ($label eq $in_proc ? " PROC" : "");+	    } else {+		$label =~ s|^\.L(?=\w)|\$ML| if ($flavour =~ /coff(?!64)/);+		printf "%s:", $label;+	    }+	}+    }++    if ($line !~ m/^[#@;]/) {+	$line =~ s|^\s*(\.?)(\S+)\s*||;+	my $c = $1; $c = "\t" if ($c eq "");+	my $mnemonic = $2;+	my $opcode;+	if ($mnemonic =~ m/([^\.]+)\.([^\.]+)/) {+	    $opcode = eval("\$$1_$2");+	} else {+	    $opcode = eval("\$$mnemonic");+	}++	my $arg=expand_line($line);++	if (ref($opcode) eq 'CODE') {+	    $line = &$opcode($arg);+	} elsif ($mnemonic)         {+	    if ($flavour =~ /win64/) {+		# "b.cond" -> "bcond", kludge-fix:-(+		$mnemonic =~ s/^b\.([a-z]{2}$)/b$1/;+	    }+	    $line = $c.$mnemonic;+	    $line.= "\t$arg" if ($arg ne "");+	}+    }++    print $line if ($line);+    print "\n";+}++print "\tEND\n" if ($flavour =~ /win/);++close STDOUT;
+ cbits/asm/arm_arch.h view
@@ -0,0 +1,101 @@+#ifndef __ARM_ARCH_H__+#define __ARM_ARCH_H__++#if !defined(__ARM_ARCH__)+# if defined(__CC_ARM)+#  if __TARGET_ARCH_THUMB+#   define __thumb__+#   if __TARGET_ARCH_THUMB >= 4+#    define __thumb2__+#   endif+#  endif+#  if __TARGET_ARCH_ARM+#   define __ARM_ARCH__ __TARGET_ARCH_ARM+#  else+#   define __ARM_ARCH__ (__TARGET_ARCH_THUMB + 3)+#  endif+#  if defined(__BIG_ENDIAN)+#   define __ARMEB__+#  else+#   define __ARMEL__+#  endif+# elif defined(__GNUC__) || defined(__clang__)+#  if	defined(__aarch64__)+#   define __ARM_ARCH__ 8+#   ifdef __AARCH64EB__+#    define __ARMEB__+#   else+#    define __ARMEL__+#   endif+#  elif defined(__ARM_ARCH)+#   define __ARM_ARCH__ __ARM_ARCH+ /*+  * Why didn't gcc define __ARM_ARCH from start? Instead it defined+  * bunch of below macros. See all_architectures[] table in+  * gcc/config/arm/arm.c. On a side note it defines+  * __ARMEL__/__ARMEB__ for little-/big-endian.+  */+#  elif defined(__ARM_ARCH_8A__)+#   define __ARM_ARCH__ 8+#  elif	defined(__ARM_ARCH_7__) || defined(__ARM_ARCH_7A__)	|| \+	defined(__ARM_ARCH_7R__)|| defined(__ARM_ARCH_7M__)	|| \+	defined(__ARM_ARCH_7EM__)+#   define __ARM_ARCH__ 7+#  elif	defined(__ARM_ARCH_6__)	|| defined(__ARM_ARCH_6J__)	|| \+	defined(__ARM_ARCH_6K__)|| defined(__ARM_ARCH_6M__)	|| \+	defined(__ARM_ARCH_6Z__)|| defined(__ARM_ARCH_6ZK__)	|| \+	defined(__ARM_ARCH_6T2__)+#   define __ARM_ARCH__ 6+#  elif	defined(__ARM_ARCH_5__)	|| defined(__ARM_ARCH_5T__)	|| \+	defined(__ARM_ARCH_5E__)|| defined(__ARM_ARCH_5TE__)	|| \+	defined(__ARM_ARCH_5TEJ__)+#   define __ARM_ARCH__ 5+#  elif	defined(__ARM_ARCH_4__)	|| defined(__ARM_ARCH_4T__)+#   define __ARM_ARCH__ 4+#  else+#   error "unsupported ARM architecture"+#  endif+# elif defined(_MSC_VER)+#  define __ARMEL__+#  if defined(_M_ARM)+#   define __ARM_ARCH__ _M_ARM+#   if defined(_M_THUMB)+#    define __thumb__+#    if _M_THUMB >= 7+#     define __thumb2__+#    endif+#   endif+#  elif defined(_M_ARM64)+#   define __AARCH64EL__+#   define __ARM_ARCH__ 8+#  else+#   error "unsupported ARM architecture"+#  endif+# endif+#endif++#if !defined(__ARM_MAX_ARCH__)+# define __ARM_MAX_ARCH__ __ARM_ARCH__+#endif++#if __ARM_MAX_ARCH__<__ARM_ARCH__+# error "__ARM_MAX_ARCH__ can't be less than __ARM_ARCH__"+#elif __ARM_MAX_ARCH__!=__ARM_ARCH__+# if __ARM_ARCH__<7 && __ARM_MAX_ARCH__>=7 && defined(__ARMEB__)+#  error "can't build universal big-endian binary"+# endif+#endif++#ifndef __ASSEMBLER__+extern unsigned int OPENSSL_armcap_P;+#endif++#define ARMV7_NEON	(1<<0)+#define ARMV7_TICK	(1<<1)+#define ARMV8_AES	(1<<2)+#define ARMV8_SHA1	(1<<3)+#define ARMV8_SHA256	(1<<4)+#define ARMV8_PMULL	(1<<5)+#define ARMV8_SHA512	(1<<6)++#endif
+ cbits/asm/chacha-armv8-ios64.S view
@@ -0,0 +1,2053 @@+#ifndef	__KERNEL__+# include "arm_arch.h"++#endif++.text++.align	5+Lsigma:+.quad	0x3320646e61707865,0x6b20657479622d32		// endian-neutral+Lone:+.long	1,2,3,4+Lrot24:+.long	0x02010003,0x06050407,0x0a09080b,0x0e0d0c0f+.byte	67,104,97,67,104,97,50,48,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.align	2++.globl	_crypton_chacha20_asm_ctr32++.align	5+_crypton_chacha20_asm_ctr32:+	cbz	x2,Labort+	cmp	x2,#192+	b.lo	Lshort++#ifndef	__KERNEL__+	adrp	x17,_crypton_armcap_P@PAGE+	ldr	w17,[x17,_crypton_armcap_P@PAGEOFF]+	tst	w17,#ARMV7_NEON+	b.ne	Lcrypton_chacha20_asm_neon+#endif++Lshort:+.long	0xd503233f			// paciasp+	stp	x29,x30,[sp,#-12*__SIZEOF_POINTER__]!+	add	x29,sp,#0++	adr	x5,Lsigma+	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	stp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	stp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	stp	x27,x28,[sp,#10*__SIZEOF_POINTER__]+	sub	sp,sp,#64++	ldp	x22,x23,[x5]		// load sigma+	ldp	x24,x25,[x3]		// load key+	ldp	x26,x27,[x3,#16]+	ldp	x28,x30,[x4]		// load counter+#ifdef	__AARCH64EB__+	ror	x24,x24,#32+	ror	x25,x25,#32+	ror	x26,x26,#32+	ror	x27,x27,#32+	ror	x28,x28,#32+	ror	x30,x30,#32+#endif++Loop_outer:+	mov	w5,w22			// unpack key block+	lsr	x6,x22,#32+	mov	w7,w23+	lsr	x8,x23,#32+	mov	w9,w24+	lsr	x10,x24,#32+	mov	w11,w25+	lsr	x12,x25,#32+	mov	w13,w26+	lsr	x14,x26,#32+	mov	w15,w27+	lsr	x16,x27,#32+	mov	w17,w28+	lsr	x19,x28,#32+	mov	w20,w30+	lsr	x21,x30,#32++	mov	x4,#10+	subs	x2,x2,#64+Loop:+	sub	x4,x4,#1+	add	w5,w5,w9+	add	w6,w6,w10+	add	w7,w7,w11+	add	w8,w8,w12+	eor	w17,w17,w5+	eor	w19,w19,w6+	eor	w20,w20,w7+	eor	w21,w21,w8+	ror	w17,w17,#16+	ror	w19,w19,#16+	ror	w20,w20,#16+	ror	w21,w21,#16+	add	w13,w13,w17+	add	w14,w14,w19+	add	w15,w15,w20+	add	w16,w16,w21+	eor	w9,w9,w13+	eor	w10,w10,w14+	eor	w11,w11,w15+	eor	w12,w12,w16+	ror	w9,w9,#20+	ror	w10,w10,#20+	ror	w11,w11,#20+	ror	w12,w12,#20+	add	w5,w5,w9+	add	w6,w6,w10+	add	w7,w7,w11+	add	w8,w8,w12+	eor	w17,w17,w5+	eor	w19,w19,w6+	eor	w20,w20,w7+	eor	w21,w21,w8+	ror	w17,w17,#24+	ror	w19,w19,#24+	ror	w20,w20,#24+	ror	w21,w21,#24+	add	w13,w13,w17+	add	w14,w14,w19+	add	w15,w15,w20+	add	w16,w16,w21+	eor	w9,w9,w13+	eor	w10,w10,w14+	eor	w11,w11,w15+	eor	w12,w12,w16+	ror	w9,w9,#25+	ror	w10,w10,#25+	ror	w11,w11,#25+	ror	w12,w12,#25+	add	w5,w5,w10+	add	w6,w6,w11+	add	w7,w7,w12+	add	w8,w8,w9+	eor	w21,w21,w5+	eor	w17,w17,w6+	eor	w19,w19,w7+	eor	w20,w20,w8+	ror	w21,w21,#16+	ror	w17,w17,#16+	ror	w19,w19,#16+	ror	w20,w20,#16+	add	w15,w15,w21+	add	w16,w16,w17+	add	w13,w13,w19+	add	w14,w14,w20+	eor	w10,w10,w15+	eor	w11,w11,w16+	eor	w12,w12,w13+	eor	w9,w9,w14+	ror	w10,w10,#20+	ror	w11,w11,#20+	ror	w12,w12,#20+	ror	w9,w9,#20+	add	w5,w5,w10+	add	w6,w6,w11+	add	w7,w7,w12+	add	w8,w8,w9+	eor	w21,w21,w5+	eor	w17,w17,w6+	eor	w19,w19,w7+	eor	w20,w20,w8+	ror	w21,w21,#24+	ror	w17,w17,#24+	ror	w19,w19,#24+	ror	w20,w20,#24+	add	w15,w15,w21+	add	w16,w16,w17+	add	w13,w13,w19+	add	w14,w14,w20+	eor	w10,w10,w15+	eor	w11,w11,w16+	eor	w12,w12,w13+	eor	w9,w9,w14+	ror	w10,w10,#25+	ror	w11,w11,#25+	ror	w12,w12,#25+	ror	w9,w9,#25+	cbnz	x4,Loop++	add	w5,w5,w22		// accumulate key block+	add	x6,x6,x22,lsr#32+	add	w7,w7,w23+	add	x8,x8,x23,lsr#32+	add	w9,w9,w24+	add	x10,x10,x24,lsr#32+	add	w11,w11,w25+	add	x12,x12,x25,lsr#32+	add	w13,w13,w26+	add	x14,x14,x26,lsr#32+	add	w15,w15,w27+	add	x16,x16,x27,lsr#32+	add	w17,w17,w28+	add	x19,x19,x28,lsr#32+	add	w20,w20,w30+	add	x21,x21,x30,lsr#32++	b.lo	Ltail++	add	x5,x5,x6,lsl#32	// pack+	add	x7,x7,x8,lsl#32+	ldp	x6,x8,[x1,#0]		// load input+	add	x9,x9,x10,lsl#32+	add	x11,x11,x12,lsl#32+	ldp	x10,x12,[x1,#16]+	add	x13,x13,x14,lsl#32+	add	x15,x15,x16,lsl#32+	ldp	x14,x16,[x1,#32]+	add	x17,x17,x19,lsl#32+	add	x20,x20,x21,lsl#32+	ldp	x19,x21,[x1,#48]+	add	x1,x1,#64+#ifdef	__AARCH64EB__+	rev	x5,x5+	rev	x7,x7+	rev	x9,x9+	rev	x11,x11+	rev	x13,x13+	rev	x15,x15+	rev	x17,x17+	rev	x20,x20+#endif+	eor	x5,x5,x6+	eor	x7,x7,x8+	eor	x9,x9,x10+	eor	x11,x11,x12+	eor	x13,x13,x14+	eor	x15,x15,x16+	eor	x17,x17,x19+	eor	x20,x20,x21++	stp	x5,x7,[x0,#0]		// store output+	add	x28,x28,#1			// increment counter+	stp	x9,x11,[x0,#16]+	stp	x13,x15,[x0,#32]+	stp	x17,x20,[x0,#48]+	add	x0,x0,#64++	b.hi	Loop_outer++	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#64+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#12*__SIZEOF_POINTER__+.long	0xd50323bf			// autiasp+Labort:+	ret++.align	4+Ltail:+	add	x2,x2,#64+Less_than_64:+	sub	x0,x0,#1+	add	x1,x1,x2+	add	x0,x0,x2+	add	x4,sp,x2+	neg	x2,x2++	add	x5,x5,x6,lsl#32	// pack+	add	x7,x7,x8,lsl#32+	add	x9,x9,x10,lsl#32+	add	x11,x11,x12,lsl#32+	add	x13,x13,x14,lsl#32+	add	x15,x15,x16,lsl#32+	add	x17,x17,x19,lsl#32+	add	x20,x20,x21,lsl#32+#ifdef	__AARCH64EB__+	rev	x5,x5+	rev	x7,x7+	rev	x9,x9+	rev	x11,x11+	rev	x13,x13+	rev	x15,x15+	rev	x17,x17+	rev	x20,x20+#endif+	stp	x5,x7,[sp,#0]		// off-load complete block+	stp	x9,x11,[sp,#16]+	stp	x13,x15,[sp,#32]+	stp	x17,x20,[sp,#48]++Loop_tail:+	ldrb	w10,[x1,x2]+	ldrb	w11,[x4,x2]+	add	x2,x2,#1+	eor	w10,w10,w11+	strb	w10,[x0,x2]+	cbnz	x2,Loop_tail++	stp	xzr,xzr,[sp,#0]			// wipe off-load area+	stp	xzr,xzr,[sp,#16]+	stp	xzr,xzr,[sp,#32]+	stp	xzr,xzr,[sp,#48]++	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#64+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#12*__SIZEOF_POINTER__+.long	0xd50323bf			// autiasp+	ret+++#ifdef	__KERNEL__+.globl	_crypton_chacha20_asm_neon+#endif++.align	5+_crypton_chacha20_asm_neon:+Lcrypton_chacha20_asm_neon:+.long	0xd503233f			// paciasp+	stp	x29,x30,[sp,#-12*__SIZEOF_POINTER__]!+	add	x29,sp,#0++	adr	x5,Lsigma+	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	stp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	stp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	stp	x27,x28,[sp,#10*__SIZEOF_POINTER__]+	cmp	x2,#512+	b.hs	L512_or_more_neon++	sub	sp,sp,#64++	ldp	x22,x23,[x5]		// load sigma+	ld1	{v0.4s},[x5],#16+	ldp	x24,x25,[x3]		// load key+	ldp	x26,x27,[x3,#16]+	ld1	{v1.4s,v2.4s},[x3]+	ldp	x28,x30,[x4]		// load counter+	ld1	{v3.4s},[x4]+	stp	d8,d9,[sp]			// meet ABI requirements+	ld1	{v8.4s,v9.4s},[x5]+#ifdef	__AARCH64EB__+	rev64	v0.4s,v0.4s+	ror	x24,x24,#32+	ror	x25,x25,#32+	ror	x26,x26,#32+	ror	x27,x27,#32+	ror	x28,x28,#32+	ror	x30,x30,#32+#endif++Loop_outer_neon:+	dup	v16.4s,v0.s[0]			// unpack key block+	mov	w5,w22+	dup	v20.4s,v0.s[1]+	lsr	x6,x22,#32+	dup	v24.4s,v0.s[2]+	mov	w7,w23+	dup	v28.4s,v0.s[3]+	lsr	x8,x23,#32+	dup	v17.4s,v1.s[0]+	mov	w9,w24+	dup	v21.4s,v1.s[1]+	lsr	x10,x24,#32+	dup	v25.4s,v1.s[2]+	mov	w11,w25+	dup	v29.4s,v1.s[3]+	lsr	x12,x25,#32+	dup	v19.4s,v3.s[0]+	mov	w13,w26+	dup	v23.4s,v3.s[1]+	lsr	x14,x26,#32+	dup	v27.4s,v3.s[2]+	mov	w15,w27+	dup	v31.4s,v3.s[3]+	lsr	x16,x27,#32+	add	v19.4s,v19.4s,v8.4s+	mov	w17,w28+	dup	v18.4s,v2.s[0]+	lsr	x19,x28,#32+	dup	v22.4s,v2.s[1]+	mov	w20,w30+	dup	v26.4s,v2.s[2]+	lsr	x21,x30,#32+	dup	v30.4s,v2.s[3]++	mov	x4,#10+	subs	x2,x2,#320+Loop_neon:+	sub	x4,x4,#1+	add	v16.4s,v16.4s,v17.4s+	add	v20.4s,v20.4s,v21.4s+	add	v24.4s,v24.4s,v25.4s+	add	v28.4s,v28.4s,v29.4s+	eor	v19.16b,v19.16b,v16.16b+	eor	v23.16b,v23.16b,v20.16b+	eor	v27.16b,v27.16b,v24.16b+	eor	v31.16b,v31.16b,v28.16b+	add	w5,w5,w9+	rev32	v19.8h,v19.8h+	add	w6,w6,w10+	rev32	v23.8h,v23.8h+	add	w7,w7,w11+	rev32	v27.8h,v27.8h+	add	w8,w8,w12+	rev32	v31.8h,v31.8h+	eor	w17,w17,w5+	add	v18.4s,v18.4s,v19.4s+	eor	w19,w19,w6+	add	v22.4s,v22.4s,v23.4s+	eor	w20,w20,w7+	add	v26.4s,v26.4s,v27.4s+	eor	w21,w21,w8+	add	v30.4s,v30.4s,v31.4s+	ror	w17,w17,#16+	eor	v4.16b,v17.16b,v18.16b+	ror	w19,w19,#16+	eor	v5.16b,v21.16b,v22.16b+	ror	w20,w20,#16+	eor	v6.16b,v25.16b,v26.16b+	ror	w21,w21,#16+	eor	v7.16b,v29.16b,v30.16b+	add	w13,w13,w17+	ushr	v17.4s,v4.4s,#20+	add	w14,w14,w19+	ushr	v21.4s,v5.4s,#20+	add	w15,w15,w20+	ushr	v25.4s,v6.4s,#20+	add	w16,w16,w21+	ushr	v29.4s,v7.4s,#20+	eor	w9,w9,w13+	sli	v17.4s,v4.4s,#12+	eor	w10,w10,w14+	sli	v21.4s,v5.4s,#12+	eor	w11,w11,w15+	sli	v25.4s,v6.4s,#12+	eor	w12,w12,w16+	sli	v29.4s,v7.4s,#12+	ror	w9,w9,#20+	add	v16.4s,v16.4s,v17.4s+	ror	w10,w10,#20+	add	v20.4s,v20.4s,v21.4s+	ror	w11,w11,#20+	add	v24.4s,v24.4s,v25.4s+	ror	w12,w12,#20+	add	v28.4s,v28.4s,v29.4s+	add	w5,w5,w9+	eor	v4.16b,v19.16b,v16.16b+	add	w6,w6,w10+	eor	v5.16b,v23.16b,v20.16b+	add	w7,w7,w11+	eor	v6.16b,v27.16b,v24.16b+	add	w8,w8,w12+	eor	v7.16b,v31.16b,v28.16b+	eor	w17,w17,w5+	tbl	v19.16b,{v4.16b},v9.16b+	eor	w19,w19,w6+	tbl	v23.16b,{v5.16b},v9.16b+	eor	w20,w20,w7+	tbl	v27.16b,{v6.16b},v9.16b+	eor	w21,w21,w8+	tbl	v31.16b,{v7.16b},v9.16b+	ror	w17,w17,#24+	add	v18.4s,v18.4s,v19.4s+	ror	w19,w19,#24+	add	v22.4s,v22.4s,v23.4s+	ror	w20,w20,#24+	add	v26.4s,v26.4s,v27.4s+	ror	w21,w21,#24+	add	v30.4s,v30.4s,v31.4s+	add	w13,w13,w17+	eor	v4.16b,v17.16b,v18.16b+	add	w14,w14,w19+	eor	v5.16b,v21.16b,v22.16b+	add	w15,w15,w20+	eor	v6.16b,v25.16b,v26.16b+	add	w16,w16,w21+	eor	v7.16b,v29.16b,v30.16b+	eor	w9,w9,w13+	ushr	v17.4s,v4.4s,#25+	eor	w10,w10,w14+	ushr	v21.4s,v5.4s,#25+	eor	w11,w11,w15+	ushr	v25.4s,v6.4s,#25+	eor	w12,w12,w16+	ushr	v29.4s,v7.4s,#25+	ror	w9,w9,#25+	sli	v17.4s,v4.4s,#7+	ror	w10,w10,#25+	sli	v21.4s,v5.4s,#7+	ror	w11,w11,#25+	sli	v25.4s,v6.4s,#7+	ror	w12,w12,#25+	sli	v29.4s,v7.4s,#7+	add	v16.4s,v16.4s,v21.4s+	add	v20.4s,v20.4s,v25.4s+	add	v24.4s,v24.4s,v29.4s+	add	v28.4s,v28.4s,v17.4s+	eor	v31.16b,v31.16b,v16.16b+	eor	v19.16b,v19.16b,v20.16b+	eor	v23.16b,v23.16b,v24.16b+	eor	v27.16b,v27.16b,v28.16b+	add	w5,w5,w10+	rev32	v31.8h,v31.8h+	add	w6,w6,w11+	rev32	v19.8h,v19.8h+	add	w7,w7,w12+	rev32	v23.8h,v23.8h+	add	w8,w8,w9+	rev32	v27.8h,v27.8h+	eor	w21,w21,w5+	add	v26.4s,v26.4s,v31.4s+	eor	w17,w17,w6+	add	v30.4s,v30.4s,v19.4s+	eor	w19,w19,w7+	add	v18.4s,v18.4s,v23.4s+	eor	w20,w20,w8+	add	v22.4s,v22.4s,v27.4s+	ror	w21,w21,#16+	eor	v4.16b,v21.16b,v26.16b+	ror	w17,w17,#16+	eor	v5.16b,v25.16b,v30.16b+	ror	w19,w19,#16+	eor	v6.16b,v29.16b,v18.16b+	ror	w20,w20,#16+	eor	v7.16b,v17.16b,v22.16b+	add	w15,w15,w21+	ushr	v21.4s,v4.4s,#20+	add	w16,w16,w17+	ushr	v25.4s,v5.4s,#20+	add	w13,w13,w19+	ushr	v29.4s,v6.4s,#20+	add	w14,w14,w20+	ushr	v17.4s,v7.4s,#20+	eor	w10,w10,w15+	sli	v21.4s,v4.4s,#12+	eor	w11,w11,w16+	sli	v25.4s,v5.4s,#12+	eor	w12,w12,w13+	sli	v29.4s,v6.4s,#12+	eor	w9,w9,w14+	sli	v17.4s,v7.4s,#12+	ror	w10,w10,#20+	add	v16.4s,v16.4s,v21.4s+	ror	w11,w11,#20+	add	v20.4s,v20.4s,v25.4s+	ror	w12,w12,#20+	add	v24.4s,v24.4s,v29.4s+	ror	w9,w9,#20+	add	v28.4s,v28.4s,v17.4s+	add	w5,w5,w10+	eor	v4.16b,v31.16b,v16.16b+	add	w6,w6,w11+	eor	v5.16b,v19.16b,v20.16b+	add	w7,w7,w12+	eor	v6.16b,v23.16b,v24.16b+	add	w8,w8,w9+	eor	v7.16b,v27.16b,v28.16b+	eor	w21,w21,w5+	tbl	v31.16b,{v4.16b},v9.16b+	eor	w17,w17,w6+	tbl	v19.16b,{v5.16b},v9.16b+	eor	w19,w19,w7+	tbl	v23.16b,{v6.16b},v9.16b+	eor	w20,w20,w8+	tbl	v27.16b,{v7.16b},v9.16b+	ror	w21,w21,#24+	add	v26.4s,v26.4s,v31.4s+	ror	w17,w17,#24+	add	v30.4s,v30.4s,v19.4s+	ror	w19,w19,#24+	add	v18.4s,v18.4s,v23.4s+	ror	w20,w20,#24+	add	v22.4s,v22.4s,v27.4s+	add	w15,w15,w21+	eor	v4.16b,v21.16b,v26.16b+	add	w16,w16,w17+	eor	v5.16b,v25.16b,v30.16b+	add	w13,w13,w19+	eor	v6.16b,v29.16b,v18.16b+	add	w14,w14,w20+	eor	v7.16b,v17.16b,v22.16b+	eor	w10,w10,w15+	ushr	v21.4s,v4.4s,#25+	eor	w11,w11,w16+	ushr	v25.4s,v5.4s,#25+	eor	w12,w12,w13+	ushr	v29.4s,v6.4s,#25+	eor	w9,w9,w14+	ushr	v17.4s,v7.4s,#25+	ror	w10,w10,#25+	sli	v21.4s,v4.4s,#7+	ror	w11,w11,#25+	sli	v25.4s,v5.4s,#7+	ror	w12,w12,#25+	sli	v29.4s,v6.4s,#7+	ror	w9,w9,#25+	sli	v17.4s,v7.4s,#7+	cbnz	x4,Loop_neon++	add	v19.4s,v19.4s,v8.4s++	zip1	v4.4s,v16.4s,v20.4s			// transpose data+	zip1	v5.4s,v24.4s,v28.4s+	zip2	v6.4s,v16.4s,v20.4s+	zip2	v7.4s,v24.4s,v28.4s+	zip1	v16.2d,v4.2d,v5.2d+	zip2	v20.2d,v4.2d,v5.2d+	zip1	v24.2d,v6.2d,v7.2d+	zip2	v28.2d,v6.2d,v7.2d++	zip1	v4.4s,v17.4s,v21.4s+	zip1	v5.4s,v25.4s,v29.4s+	zip2	v6.4s,v17.4s,v21.4s+	zip2	v7.4s,v25.4s,v29.4s+	zip1	v17.2d,v4.2d,v5.2d+	zip2	v21.2d,v4.2d,v5.2d+	zip1	v25.2d,v6.2d,v7.2d+	zip2	v29.2d,v6.2d,v7.2d++	zip1	v4.4s,v18.4s,v22.4s+	add	w5,w5,w22		// accumulate key block+	zip1	v5.4s,v26.4s,v30.4s+	add	x6,x6,x22,lsr#32+	zip2	v6.4s,v18.4s,v22.4s+	add	w7,w7,w23+	zip2	v7.4s,v26.4s,v30.4s+	add	x8,x8,x23,lsr#32+	zip1	v18.2d,v4.2d,v5.2d+	add	w9,w9,w24+	zip2	v22.2d,v4.2d,v5.2d+	add	x10,x10,x24,lsr#32+	zip1	v26.2d,v6.2d,v7.2d+	add	w11,w11,w25+	zip2	v30.2d,v6.2d,v7.2d+	add	x12,x12,x25,lsr#32++	zip1	v4.4s,v19.4s,v23.4s+	add	w13,w13,w26+	zip1	v5.4s,v27.4s,v31.4s+	add	x14,x14,x26,lsr#32+	zip2	v6.4s,v19.4s,v23.4s+	add	w15,w15,w27+	zip2	v7.4s,v27.4s,v31.4s+	add	x16,x16,x27,lsr#32+	zip1	v19.2d,v4.2d,v5.2d+	add	w17,w17,w28+	zip2	v23.2d,v4.2d,v5.2d+	add	x19,x19,x28,lsr#32+	zip1	v27.2d,v6.2d,v7.2d+	add	w20,w20,w30+	zip2	v31.2d,v6.2d,v7.2d+	add	x21,x21,x30,lsr#32++	b.lo	Ltail_neon++	add	x5,x5,x6,lsl#32	// pack+	add	x7,x7,x8,lsl#32+	ldp	x6,x8,[x1,#0]		// load input+	add	v16.4s,v16.4s,v0.4s			// accumulate key block+	add	x9,x9,x10,lsl#32+	add	x11,x11,x12,lsl#32+	ldp	x10,x12,[x1,#16]+	add	v17.4s,v17.4s,v1.4s+	add	x13,x13,x14,lsl#32+	add	x15,x15,x16,lsl#32+	ldp	x14,x16,[x1,#32]+	add	v18.4s,v18.4s,v2.4s+	add	x17,x17,x19,lsl#32+	add	x20,x20,x21,lsl#32+	ldp	x19,x21,[x1,#48]+	add	v19.4s,v19.4s,v3.4s+	add	x1,x1,#64+#ifdef	__AARCH64EB__+	rev	x5,x5+	rev	x7,x7+	rev	x9,x9+	rev	x11,x11+	rev	x13,x13+	rev	x15,x15+	rev	x17,x17+	rev	x20,x20+#endif+	ld1	{v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64+	eor	x5,x5,x6+	add	v20.4s,v20.4s,v0.4s+	eor	x7,x7,x8+	add	v21.4s,v21.4s,v1.4s+	eor	x9,x9,x10+	add	v22.4s,v22.4s,v2.4s+	eor	x11,x11,x12+	add	v23.4s,v23.4s,v3.4s+	eor	x13,x13,x14+	eor	v16.16b,v16.16b,v4.16b+	movi	v4.4s,#5+	eor	x15,x15,x16+	eor	v17.16b,v17.16b,v5.16b+	eor	x17,x17,x19+	eor	v18.16b,v18.16b,v6.16b+	eor	x20,x20,x21+	eor	v19.16b,v19.16b,v7.16b+	add	v8.4s,v8.4s,v4.4s			// += 5+	ld1	{v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64++	stp	x5,x7,[x0,#0]		// store output+	add	x28,x28,#5			// increment counter+	stp	x9,x11,[x0,#16]+	stp	x13,x15,[x0,#32]+	stp	x17,x20,[x0,#48]+	add	x0,x0,#64++	st1	{v16.16b,v17.16b,v18.16b,v19.16b},[x0],#64+	add	v24.4s,v24.4s,v0.4s+	add	v25.4s,v25.4s,v1.4s+	add	v26.4s,v26.4s,v2.4s+	add	v27.4s,v27.4s,v3.4s+	ld1	{v16.16b,v17.16b,v18.16b,v19.16b},[x1],#64++	eor	v20.16b,v20.16b,v4.16b+	eor	v21.16b,v21.16b,v5.16b+	eor	v22.16b,v22.16b,v6.16b+	eor	v23.16b,v23.16b,v7.16b+	st1	{v20.16b,v21.16b,v22.16b,v23.16b},[x0],#64+	add	v28.4s,v28.4s,v0.4s+	add	v29.4s,v29.4s,v1.4s+	add	v30.4s,v30.4s,v2.4s+	add	v31.4s,v31.4s,v3.4s+	ld1	{v20.16b,v21.16b,v22.16b,v23.16b},[x1],#64++	eor	v24.16b,v24.16b,v16.16b+	eor	v25.16b,v25.16b,v17.16b+	eor	v26.16b,v26.16b,v18.16b+	eor	v27.16b,v27.16b,v19.16b+	st1	{v24.16b,v25.16b,v26.16b,v27.16b},[x0],#64++	eor	v28.16b,v28.16b,v20.16b+	eor	v29.16b,v29.16b,v21.16b+	eor	v30.16b,v30.16b,v22.16b+	eor	v31.16b,v31.16b,v23.16b+	st1	{v28.16b,v29.16b,v30.16b,v31.16b},[x0],#64++	b.hi	Loop_outer_neon++	ldp	d8,d9,[sp]			// meet ABI requirements+	eor	v1.16b,v1.16b,v1.16b		// cleanse key and nonce+	eor	v2.16b,v2.16b,v2.16b+	eor	v3.16b,v3.16b,v3.16b++	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#64+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#12*__SIZEOF_POINTER__+.long	0xd50323bf			// autiasp+	ret++.align	4+Ltail_neon:+	add	x2,x2,#320+	ldp	d8,d9,[sp]			// meet ABI requirements+	cmp	x2,#64+	b.lo	Less_than_64_neon++	add	x5,x5,x6,lsl#32	// pack+	add	x7,x7,x8,lsl#32+	ldp	x6,x8,[x1,#0]		// load input+	add	x9,x9,x10,lsl#32+	add	x11,x11,x12,lsl#32+	ldp	x10,x12,[x1,#16]+	add	x13,x13,x14,lsl#32+	add	x15,x15,x16,lsl#32+	ldp	x14,x16,[x1,#32]+	add	x17,x17,x19,lsl#32+	add	x20,x20,x21,lsl#32+	ldp	x19,x21,[x1,#48]+	add	x1,x1,#64+#ifdef	__AARCH64EB__+	rev	x5,x5+	rev	x7,x7+	rev	x9,x9+	rev	x11,x11+	rev	x13,x13+	rev	x15,x15+	rev	x17,x17+	rev	x20,x20+#endif+	eor	x5,x5,x6+	eor	x7,x7,x8+	eor	x9,x9,x10+	eor	x11,x11,x12+	eor	x13,x13,x14+	eor	x15,x15,x16+	eor	x17,x17,x19+	eor	x20,x20,x21++	stp	x5,x7,[x0,#0]		// store output+	add	v16.4s,v16.4s,v0.4s			// accumulate key block+	stp	x9,x11,[x0,#16]+	add	v17.4s,v17.4s,v1.4s+	stp	x13,x15,[x0,#32]+	add	v18.4s,v18.4s,v2.4s+	stp	x17,x20,[x0,#48]+	add	v19.4s,v19.4s,v3.4s+	add	x0,x0,#64+	b.eq	Ldone_neon+	sub	x2,x2,#64+	cmp	x2,#64+	b.lo	Last_neon++	ld1	{v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64+	eor	v16.16b,v16.16b,v4.16b+	eor	v17.16b,v17.16b,v5.16b+	eor	v18.16b,v18.16b,v6.16b+	eor	v19.16b,v19.16b,v7.16b+	st1	{v16.16b,v17.16b,v18.16b,v19.16b},[x0],#64+	b.eq	Ldone_neon++	add	v16.4s,v20.4s,v0.4s+	add	v17.4s,v21.4s,v1.4s+	sub	x2,x2,#64+	add	v18.4s,v22.4s,v2.4s+	cmp	x2,#64+	add	v19.4s,v23.4s,v3.4s+	b.lo	Last_neon++	ld1	{v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64+	eor	v20.16b,v16.16b,v4.16b+	eor	v21.16b,v17.16b,v5.16b+	eor	v22.16b,v18.16b,v6.16b+	eor	v23.16b,v19.16b,v7.16b+	st1	{v20.16b,v21.16b,v22.16b,v23.16b},[x0],#64+	b.eq	Ldone_neon++	add	v16.4s,v24.4s,v0.4s+	add	v17.4s,v25.4s,v1.4s+	sub	x2,x2,#64+	add	v18.4s,v26.4s,v2.4s+	cmp	x2,#64+	add	v19.4s,v27.4s,v3.4s+	b.lo	Last_neon++	ld1	{v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64+	eor	v24.16b,v16.16b,v4.16b+	eor	v25.16b,v17.16b,v5.16b+	eor	v26.16b,v18.16b,v6.16b+	eor	v27.16b,v19.16b,v7.16b+	st1	{v24.16b,v25.16b,v26.16b,v27.16b},[x0],#64+	b.eq	Ldone_neon++	add	v16.4s,v28.4s,v0.4s+	add	v17.4s,v29.4s,v1.4s+	add	v18.4s,v30.4s,v2.4s+	add	v19.4s,v31.4s,v3.4s+	sub	x2,x2,#64++Last_neon:+	st1	{v16.16b,v17.16b,v18.16b,v19.16b},[sp]		// off-load complete block++	sub	x0,x0,#1+	add	x1,x1,x2+	add	x0,x0,x2+	add	x4,sp,x2+	neg	x2,x2++Loop_tail_neon:+	ldrb	w10,[x1,x2]+	ldrb	w11,[x4,x2]+	add	x2,x2,#1+	eor	w10,w10,w11+	strb	w10,[x0,x2]+	cbnz	x2,Loop_tail_neon++	stp	q0,q0,[sp,#0]		// wipe off-load area+	stp	q0,q0,[sp,#32]		// [with known constant]++Ldone_neon:+	eor	v1.16b,v1.16b,v1.16b		// cleanse key and nonce+	eor	v2.16b,v2.16b,v2.16b+	eor	v3.16b,v3.16b,v3.16b++	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#64+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#12*__SIZEOF_POINTER__+.long	0xd50323bf			// autiasp+	ret++.align	4+Less_than_64_neon:+	eor	v1.16b,v1.16b,v1.16b		// cleanse key and nonce+	eor	v2.16b,v2.16b,v2.16b+	eor	v3.16b,v3.16b,v3.16b+	b	Less_than_64+++.align	5+crypton_chacha20_asm_512_neon:+.long	0xd503233f			// paciasp+	stp	x29,x30,[sp,#-12*__SIZEOF_POINTER__]!+	add	x29,sp,#0++	adr	x5,Lsigma+	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	stp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	stp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	stp	x27,x28,[sp,#10*__SIZEOF_POINTER__]++L512_or_more_neon:+	sub	sp,sp,#128+64++	eor	v7.16b,v7.16b,v7.16b+	ldp	x22,x23,[x5]		// load sigma+	ld1	{v0.4s},[x5],#16+	ldp	x24,x25,[x3]		// load key+	ldp	x26,x27,[x3,#16]+	ld1	{v1.4s,v2.4s},[x3]+	ldp	x28,x30,[x4]		// load counter+	ld1	{v3.4s},[x4]+	ld1	{v7.s}[0],[x5]+	add	x3,x5,#16+#ifdef	__AARCH64EB__+	rev64	v0.4s,v0.4s+	ror	x24,x24,#32+	ror	x25,x25,#32+	ror	x26,x26,#32+	ror	x27,x27,#32+	ror	x28,x28,#32+	ror	x30,x30,#32+#endif+	add	v3.4s,v3.4s,v7.4s		// += 1+	stp	q0,q1,[sp,#0]		// off-load key block, invariant part+	add	v3.4s,v3.4s,v7.4s		// not typo+	str	q2,[sp,#32]+	add	v4.4s,v3.4s,v7.4s+	add	v5.4s,v4.4s,v7.4s+	add	v6.4s,v5.4s,v7.4s+	shl	v7.4s,v7.4s,#2			// 1 -> 4++	stp	d8,d9,[sp,#128+0]		// meet ABI requirements+	stp	d10,d11,[sp,#128+16]+	stp	d12,d13,[sp,#128+32]+	stp	d14,d15,[sp,#128+48]++	sub	x2,x2,#512			// not typo++Loop_outer_512_neon:+	mov	v8.16b,v0.16b+	mov	v12.16b,v0.16b+	mov	v16.16b,v0.16b+	mov	v20.16b,v0.16b+	mov	v24.16b,v0.16b+	mov	v28.16b,v0.16b+	mov	v9.16b,v1.16b+	mov	w5,w22			// unpack key block+	mov	v13.16b,v1.16b+	lsr	x6,x22,#32+	mov	v17.16b,v1.16b+	mov	w7,w23+	mov	v21.16b,v1.16b+	lsr	x8,x23,#32+	mov	v25.16b,v1.16b+	mov	w9,w24+	mov	v29.16b,v1.16b+	lsr	x10,x24,#32+	mov	v11.16b,v3.16b+	mov	w11,w25+	mov	v15.16b,v4.16b+	lsr	x12,x25,#32+	mov	v19.16b,v5.16b+	mov	w13,w26+	mov	v23.16b,v6.16b+	lsr	x14,x26,#32+	mov	v10.16b,v2.16b+	mov	w15,w27+	mov	v14.16b,v2.16b+	lsr	x16,x27,#32+	add	v27.4s,v11.4s,v7.4s			// +4+	mov	w17,w28+	add	v31.4s,v15.4s,v7.4s			// +4+	lsr	x19,x28,#32+	mov	v18.16b,v2.16b+	mov	w20,w30+	mov	v22.16b,v2.16b+	lsr	x21,x30,#32+	mov	v26.16b,v2.16b+	stp	q3,q4,[sp,#48]		// off-load key block, variable part+	mov	v30.16b,v2.16b+	stp	q5,q6,[sp,#80]++	mov	x4,#5+	ld1	{v6.4s},[x3]+	subs	x2,x2,#512+Loop_upper_neon:+	sub	x4,x4,#1+	add	v8.4s,v8.4s,v9.4s+	add	w5,w5,w9+	add	v12.4s,v12.4s,v13.4s+	add	w6,w6,w10+	add	v16.4s,v16.4s,v17.4s+	add	w7,w7,w11+	add	v20.4s,v20.4s,v21.4s+	add	w8,w8,w12+	add	v24.4s,v24.4s,v25.4s+	eor	w17,w17,w5+	add	v28.4s,v28.4s,v29.4s+	eor	w19,w19,w6+	eor	v11.16b,v11.16b,v8.16b+	eor	w20,w20,w7+	eor	v15.16b,v15.16b,v12.16b+	eor	w21,w21,w8+	eor	v19.16b,v19.16b,v16.16b+	ror	w17,w17,#16+	eor	v23.16b,v23.16b,v20.16b+	ror	w19,w19,#16+	eor	v27.16b,v27.16b,v24.16b+	ror	w20,w20,#16+	eor	v31.16b,v31.16b,v28.16b+	ror	w21,w21,#16+	rev32	v11.8h,v11.8h+	add	w13,w13,w17+	rev32	v15.8h,v15.8h+	add	w14,w14,w19+	rev32	v19.8h,v19.8h+	add	w15,w15,w20+	rev32	v23.8h,v23.8h+	add	w16,w16,w21+	rev32	v27.8h,v27.8h+	eor	w9,w9,w13+	rev32	v31.8h,v31.8h+	eor	w10,w10,w14+	add	v10.4s,v10.4s,v11.4s+	eor	w11,w11,w15+	add	v14.4s,v14.4s,v15.4s+	eor	w12,w12,w16+	add	v18.4s,v18.4s,v19.4s+	ror	w9,w9,#20+	add	v22.4s,v22.4s,v23.4s+	ror	w10,w10,#20+	add	v26.4s,v26.4s,v27.4s+	ror	w11,w11,#20+	add	v30.4s,v30.4s,v31.4s+	ror	w12,w12,#20+	eor	v0.16b,v9.16b,v10.16b+	add	w5,w5,w9+	eor	v1.16b,v13.16b,v14.16b+	add	w6,w6,w10+	eor	v2.16b,v17.16b,v18.16b+	add	w7,w7,w11+	eor	v3.16b,v21.16b,v22.16b+	add	w8,w8,w12+	eor	v4.16b,v25.16b,v26.16b+	eor	w17,w17,w5+	eor	v5.16b,v29.16b,v30.16b+	eor	w19,w19,w6+	ushr	v9.4s,v0.4s,#20+	eor	w20,w20,w7+	ushr	v13.4s,v1.4s,#20+	eor	w21,w21,w8+	ushr	v17.4s,v2.4s,#20+	ror	w17,w17,#24+	ushr	v21.4s,v3.4s,#20+	ror	w19,w19,#24+	ushr	v25.4s,v4.4s,#20+	ror	w20,w20,#24+	ushr	v29.4s,v5.4s,#20+	ror	w21,w21,#24+	sli	v9.4s,v0.4s,#12+	add	w13,w13,w17+	sli	v13.4s,v1.4s,#12+	add	w14,w14,w19+	sli	v17.4s,v2.4s,#12+	add	w15,w15,w20+	sli	v21.4s,v3.4s,#12+	add	w16,w16,w21+	sli	v25.4s,v4.4s,#12+	eor	w9,w9,w13+	sli	v29.4s,v5.4s,#12+	eor	w10,w10,w14+	add	v8.4s,v8.4s,v9.4s+	eor	w11,w11,w15+	add	v12.4s,v12.4s,v13.4s+	eor	w12,w12,w16+	add	v16.4s,v16.4s,v17.4s+	ror	w9,w9,#25+	add	v20.4s,v20.4s,v21.4s+	ror	w10,w10,#25+	add	v24.4s,v24.4s,v25.4s+	ror	w11,w11,#25+	add	v28.4s,v28.4s,v29.4s+	ror	w12,w12,#25+	eor	v11.16b,v11.16b,v8.16b+	add	w5,w5,w10+	eor	v15.16b,v15.16b,v12.16b+	add	w6,w6,w11+	eor	v19.16b,v19.16b,v16.16b+	add	w7,w7,w12+	eor	v23.16b,v23.16b,v20.16b+	add	w8,w8,w9+	eor	v27.16b,v27.16b,v24.16b+	eor	w21,w21,w5+	eor	v31.16b,v31.16b,v28.16b+	eor	w17,w17,w6+	tbl	v11.16b,{v11.16b},v6.16b+	eor	w19,w19,w7+	tbl	v15.16b,{v15.16b},v6.16b+	eor	w20,w20,w8+	tbl	v19.16b,{v19.16b},v6.16b+	ror	w21,w21,#16+	tbl	v23.16b,{v23.16b},v6.16b+	ror	w17,w17,#16+	tbl	v27.16b,{v27.16b},v6.16b+	ror	w19,w19,#16+	tbl	v31.16b,{v31.16b},v6.16b+	ror	w20,w20,#16+	add	v10.4s,v10.4s,v11.4s+	add	w15,w15,w21+	add	v14.4s,v14.4s,v15.4s+	add	w16,w16,w17+	add	v18.4s,v18.4s,v19.4s+	add	w13,w13,w19+	add	v22.4s,v22.4s,v23.4s+	add	w14,w14,w20+	add	v26.4s,v26.4s,v27.4s+	eor	w10,w10,w15+	add	v30.4s,v30.4s,v31.4s+	eor	w11,w11,w16+	eor	v0.16b,v9.16b,v10.16b+	eor	w12,w12,w13+	eor	v1.16b,v13.16b,v14.16b+	eor	w9,w9,w14+	eor	v2.16b,v17.16b,v18.16b+	ror	w10,w10,#20+	eor	v3.16b,v21.16b,v22.16b+	ror	w11,w11,#20+	eor	v4.16b,v25.16b,v26.16b+	ror	w12,w12,#20+	eor	v5.16b,v29.16b,v30.16b+	ror	w9,w9,#20+	ushr	v9.4s,v0.4s,#25+	add	w5,w5,w10+	ushr	v13.4s,v1.4s,#25+	add	w6,w6,w11+	ushr	v17.4s,v2.4s,#25+	add	w7,w7,w12+	ushr	v21.4s,v3.4s,#25+	add	w8,w8,w9+	ushr	v25.4s,v4.4s,#25+	eor	w21,w21,w5+	ushr	v29.4s,v5.4s,#25+	eor	w17,w17,w6+	sli	v9.4s,v0.4s,#7+	eor	w19,w19,w7+	sli	v13.4s,v1.4s,#7+	eor	w20,w20,w8+	sli	v17.4s,v2.4s,#7+	ror	w21,w21,#24+	sli	v21.4s,v3.4s,#7+	ror	w17,w17,#24+	sli	v25.4s,v4.4s,#7+	ror	w19,w19,#24+	sli	v29.4s,v5.4s,#7+	ror	w20,w20,#24+	ext	v10.16b,v10.16b,v10.16b,#8+	add	w15,w15,w21+	ext	v14.16b,v14.16b,v14.16b,#8+	add	w16,w16,w17+	ext	v18.16b,v18.16b,v18.16b,#8+	add	w13,w13,w19+	ext	v22.16b,v22.16b,v22.16b,#8+	add	w14,w14,w20+	ext	v26.16b,v26.16b,v26.16b,#8+	eor	w10,w10,w15+	ext	v30.16b,v30.16b,v30.16b,#8+	eor	w11,w11,w16+	ext	v11.16b,v11.16b,v11.16b,#12+	eor	w12,w12,w13+	ext	v15.16b,v15.16b,v15.16b,#12+	eor	w9,w9,w14+	ext	v19.16b,v19.16b,v19.16b,#12+	ror	w10,w10,#25+	ext	v23.16b,v23.16b,v23.16b,#12+	ror	w11,w11,#25+	ext	v27.16b,v27.16b,v27.16b,#12+	ror	w12,w12,#25+	ext	v31.16b,v31.16b,v31.16b,#12+	ror	w9,w9,#25+	ext	v9.16b,v9.16b,v9.16b,#4+	ext	v13.16b,v13.16b,v13.16b,#4+	ext	v17.16b,v17.16b,v17.16b,#4+	ext	v21.16b,v21.16b,v21.16b,#4+	ext	v25.16b,v25.16b,v25.16b,#4+	ext	v29.16b,v29.16b,v29.16b,#4+	add	v8.4s,v8.4s,v9.4s+	add	w5,w5,w9+	add	v12.4s,v12.4s,v13.4s+	add	w6,w6,w10+	add	v16.4s,v16.4s,v17.4s+	add	w7,w7,w11+	add	v20.4s,v20.4s,v21.4s+	add	w8,w8,w12+	add	v24.4s,v24.4s,v25.4s+	eor	w17,w17,w5+	add	v28.4s,v28.4s,v29.4s+	eor	w19,w19,w6+	eor	v11.16b,v11.16b,v8.16b+	eor	w20,w20,w7+	eor	v15.16b,v15.16b,v12.16b+	eor	w21,w21,w8+	eor	v19.16b,v19.16b,v16.16b+	ror	w17,w17,#16+	eor	v23.16b,v23.16b,v20.16b+	ror	w19,w19,#16+	eor	v27.16b,v27.16b,v24.16b+	ror	w20,w20,#16+	eor	v31.16b,v31.16b,v28.16b+	ror	w21,w21,#16+	rev32	v11.8h,v11.8h+	add	w13,w13,w17+	rev32	v15.8h,v15.8h+	add	w14,w14,w19+	rev32	v19.8h,v19.8h+	add	w15,w15,w20+	rev32	v23.8h,v23.8h+	add	w16,w16,w21+	rev32	v27.8h,v27.8h+	eor	w9,w9,w13+	rev32	v31.8h,v31.8h+	eor	w10,w10,w14+	add	v10.4s,v10.4s,v11.4s+	eor	w11,w11,w15+	add	v14.4s,v14.4s,v15.4s+	eor	w12,w12,w16+	add	v18.4s,v18.4s,v19.4s+	ror	w9,w9,#20+	add	v22.4s,v22.4s,v23.4s+	ror	w10,w10,#20+	add	v26.4s,v26.4s,v27.4s+	ror	w11,w11,#20+	add	v30.4s,v30.4s,v31.4s+	ror	w12,w12,#20+	eor	v0.16b,v9.16b,v10.16b+	add	w5,w5,w9+	eor	v1.16b,v13.16b,v14.16b+	add	w6,w6,w10+	eor	v2.16b,v17.16b,v18.16b+	add	w7,w7,w11+	eor	v3.16b,v21.16b,v22.16b+	add	w8,w8,w12+	eor	v4.16b,v25.16b,v26.16b+	eor	w17,w17,w5+	eor	v5.16b,v29.16b,v30.16b+	eor	w19,w19,w6+	ushr	v9.4s,v0.4s,#20+	eor	w20,w20,w7+	ushr	v13.4s,v1.4s,#20+	eor	w21,w21,w8+	ushr	v17.4s,v2.4s,#20+	ror	w17,w17,#24+	ushr	v21.4s,v3.4s,#20+	ror	w19,w19,#24+	ushr	v25.4s,v4.4s,#20+	ror	w20,w20,#24+	ushr	v29.4s,v5.4s,#20+	ror	w21,w21,#24+	sli	v9.4s,v0.4s,#12+	add	w13,w13,w17+	sli	v13.4s,v1.4s,#12+	add	w14,w14,w19+	sli	v17.4s,v2.4s,#12+	add	w15,w15,w20+	sli	v21.4s,v3.4s,#12+	add	w16,w16,w21+	sli	v25.4s,v4.4s,#12+	eor	w9,w9,w13+	sli	v29.4s,v5.4s,#12+	eor	w10,w10,w14+	add	v8.4s,v8.4s,v9.4s+	eor	w11,w11,w15+	add	v12.4s,v12.4s,v13.4s+	eor	w12,w12,w16+	add	v16.4s,v16.4s,v17.4s+	ror	w9,w9,#25+	add	v20.4s,v20.4s,v21.4s+	ror	w10,w10,#25+	add	v24.4s,v24.4s,v25.4s+	ror	w11,w11,#25+	add	v28.4s,v28.4s,v29.4s+	ror	w12,w12,#25+	eor	v11.16b,v11.16b,v8.16b+	add	w5,w5,w10+	eor	v15.16b,v15.16b,v12.16b+	add	w6,w6,w11+	eor	v19.16b,v19.16b,v16.16b+	add	w7,w7,w12+	eor	v23.16b,v23.16b,v20.16b+	add	w8,w8,w9+	eor	v27.16b,v27.16b,v24.16b+	eor	w21,w21,w5+	eor	v31.16b,v31.16b,v28.16b+	eor	w17,w17,w6+	tbl	v11.16b,{v11.16b},v6.16b+	eor	w19,w19,w7+	tbl	v15.16b,{v15.16b},v6.16b+	eor	w20,w20,w8+	tbl	v19.16b,{v19.16b},v6.16b+	ror	w21,w21,#16+	tbl	v23.16b,{v23.16b},v6.16b+	ror	w17,w17,#16+	tbl	v27.16b,{v27.16b},v6.16b+	ror	w19,w19,#16+	tbl	v31.16b,{v31.16b},v6.16b+	ror	w20,w20,#16+	add	v10.4s,v10.4s,v11.4s+	add	w15,w15,w21+	add	v14.4s,v14.4s,v15.4s+	add	w16,w16,w17+	add	v18.4s,v18.4s,v19.4s+	add	w13,w13,w19+	add	v22.4s,v22.4s,v23.4s+	add	w14,w14,w20+	add	v26.4s,v26.4s,v27.4s+	eor	w10,w10,w15+	add	v30.4s,v30.4s,v31.4s+	eor	w11,w11,w16+	eor	v0.16b,v9.16b,v10.16b+	eor	w12,w12,w13+	eor	v1.16b,v13.16b,v14.16b+	eor	w9,w9,w14+	eor	v2.16b,v17.16b,v18.16b+	ror	w10,w10,#20+	eor	v3.16b,v21.16b,v22.16b+	ror	w11,w11,#20+	eor	v4.16b,v25.16b,v26.16b+	ror	w12,w12,#20+	eor	v5.16b,v29.16b,v30.16b+	ror	w9,w9,#20+	ushr	v9.4s,v0.4s,#25+	add	w5,w5,w10+	ushr	v13.4s,v1.4s,#25+	add	w6,w6,w11+	ushr	v17.4s,v2.4s,#25+	add	w7,w7,w12+	ushr	v21.4s,v3.4s,#25+	add	w8,w8,w9+	ushr	v25.4s,v4.4s,#25+	eor	w21,w21,w5+	ushr	v29.4s,v5.4s,#25+	eor	w17,w17,w6+	sli	v9.4s,v0.4s,#7+	eor	w19,w19,w7+	sli	v13.4s,v1.4s,#7+	eor	w20,w20,w8+	sli	v17.4s,v2.4s,#7+	ror	w21,w21,#24+	sli	v21.4s,v3.4s,#7+	ror	w17,w17,#24+	sli	v25.4s,v4.4s,#7+	ror	w19,w19,#24+	sli	v29.4s,v5.4s,#7+	ror	w20,w20,#24+	ext	v10.16b,v10.16b,v10.16b,#8+	add	w15,w15,w21+	ext	v14.16b,v14.16b,v14.16b,#8+	add	w16,w16,w17+	ext	v18.16b,v18.16b,v18.16b,#8+	add	w13,w13,w19+	ext	v22.16b,v22.16b,v22.16b,#8+	add	w14,w14,w20+	ext	v26.16b,v26.16b,v26.16b,#8+	eor	w10,w10,w15+	ext	v30.16b,v30.16b,v30.16b,#8+	eor	w11,w11,w16+	ext	v11.16b,v11.16b,v11.16b,#4+	eor	w12,w12,w13+	ext	v15.16b,v15.16b,v15.16b,#4+	eor	w9,w9,w14+	ext	v19.16b,v19.16b,v19.16b,#4+	ror	w10,w10,#25+	ext	v23.16b,v23.16b,v23.16b,#4+	ror	w11,w11,#25+	ext	v27.16b,v27.16b,v27.16b,#4+	ror	w12,w12,#25+	ext	v31.16b,v31.16b,v31.16b,#4+	ror	w9,w9,#25+	ext	v9.16b,v9.16b,v9.16b,#12+	ext	v13.16b,v13.16b,v13.16b,#12+	ext	v17.16b,v17.16b,v17.16b,#12+	ext	v21.16b,v21.16b,v21.16b,#12+	ext	v25.16b,v25.16b,v25.16b,#12+	ext	v29.16b,v29.16b,v29.16b,#12+	cbnz	x4,Loop_upper_neon++	add	w5,w5,w22		// accumulate key block+	add	x6,x6,x22,lsr#32+	add	w7,w7,w23+	add	x8,x8,x23,lsr#32+	add	w9,w9,w24+	add	x10,x10,x24,lsr#32+	add	w11,w11,w25+	add	x12,x12,x25,lsr#32+	add	w13,w13,w26+	add	x14,x14,x26,lsr#32+	add	w15,w15,w27+	add	x16,x16,x27,lsr#32+	add	w17,w17,w28+	add	x19,x19,x28,lsr#32+	add	w20,w20,w30+	add	x21,x21,x30,lsr#32++	add	x5,x5,x6,lsl#32	// pack+	add	x7,x7,x8,lsl#32+	ldp	x6,x8,[x1,#0]		// load input+	add	x9,x9,x10,lsl#32+	add	x11,x11,x12,lsl#32+	ldp	x10,x12,[x1,#16]+	add	x13,x13,x14,lsl#32+	add	x15,x15,x16,lsl#32+	ldp	x14,x16,[x1,#32]+	add	x17,x17,x19,lsl#32+	add	x20,x20,x21,lsl#32+	ldp	x19,x21,[x1,#48]+	add	x1,x1,#64+#ifdef	__AARCH64EB__+	rev	x5,x5+	rev	x7,x7+	rev	x9,x9+	rev	x11,x11+	rev	x13,x13+	rev	x15,x15+	rev	x17,x17+	rev	x20,x20+#endif+	eor	x5,x5,x6+	eor	x7,x7,x8+	eor	x9,x9,x10+	eor	x11,x11,x12+	eor	x13,x13,x14+	eor	x15,x15,x16+	eor	x17,x17,x19+	eor	x20,x20,x21++	stp	x5,x7,[x0,#0]		// store output+	add	x28,x28,#1			// increment counter+	mov	w5,w22			// unpack key block+	lsr	x6,x22,#32+	stp	x9,x11,[x0,#16]+	mov	w7,w23+	lsr	x8,x23,#32+	stp	x13,x15,[x0,#32]+	mov	w9,w24+	lsr	x10,x24,#32+	stp	x17,x20,[x0,#48]+	add	x0,x0,#64+	mov	w11,w25+	lsr	x12,x25,#32+	mov	w13,w26+	lsr	x14,x26,#32+	mov	w15,w27+	lsr	x16,x27,#32+	mov	w17,w28+	lsr	x19,x28,#32+	mov	w20,w30+	lsr	x21,x30,#32++	mov	x4,#5+Loop_lower_neon:+	sub	x4,x4,#1+	add	v8.4s,v8.4s,v9.4s+	add	w5,w5,w9+	add	v12.4s,v12.4s,v13.4s+	add	w6,w6,w10+	add	v16.4s,v16.4s,v17.4s+	add	w7,w7,w11+	add	v20.4s,v20.4s,v21.4s+	add	w8,w8,w12+	add	v24.4s,v24.4s,v25.4s+	eor	w17,w17,w5+	add	v28.4s,v28.4s,v29.4s+	eor	w19,w19,w6+	eor	v11.16b,v11.16b,v8.16b+	eor	w20,w20,w7+	eor	v15.16b,v15.16b,v12.16b+	eor	w21,w21,w8+	eor	v19.16b,v19.16b,v16.16b+	ror	w17,w17,#16+	eor	v23.16b,v23.16b,v20.16b+	ror	w19,w19,#16+	eor	v27.16b,v27.16b,v24.16b+	ror	w20,w20,#16+	eor	v31.16b,v31.16b,v28.16b+	ror	w21,w21,#16+	rev32	v11.8h,v11.8h+	add	w13,w13,w17+	rev32	v15.8h,v15.8h+	add	w14,w14,w19+	rev32	v19.8h,v19.8h+	add	w15,w15,w20+	rev32	v23.8h,v23.8h+	add	w16,w16,w21+	rev32	v27.8h,v27.8h+	eor	w9,w9,w13+	rev32	v31.8h,v31.8h+	eor	w10,w10,w14+	add	v10.4s,v10.4s,v11.4s+	eor	w11,w11,w15+	add	v14.4s,v14.4s,v15.4s+	eor	w12,w12,w16+	add	v18.4s,v18.4s,v19.4s+	ror	w9,w9,#20+	add	v22.4s,v22.4s,v23.4s+	ror	w10,w10,#20+	add	v26.4s,v26.4s,v27.4s+	ror	w11,w11,#20+	add	v30.4s,v30.4s,v31.4s+	ror	w12,w12,#20+	eor	v0.16b,v9.16b,v10.16b+	add	w5,w5,w9+	eor	v1.16b,v13.16b,v14.16b+	add	w6,w6,w10+	eor	v2.16b,v17.16b,v18.16b+	add	w7,w7,w11+	eor	v3.16b,v21.16b,v22.16b+	add	w8,w8,w12+	eor	v4.16b,v25.16b,v26.16b+	eor	w17,w17,w5+	eor	v5.16b,v29.16b,v30.16b+	eor	w19,w19,w6+	ushr	v9.4s,v0.4s,#20+	eor	w20,w20,w7+	ushr	v13.4s,v1.4s,#20+	eor	w21,w21,w8+	ushr	v17.4s,v2.4s,#20+	ror	w17,w17,#24+	ushr	v21.4s,v3.4s,#20+	ror	w19,w19,#24+	ushr	v25.4s,v4.4s,#20+	ror	w20,w20,#24+	ushr	v29.4s,v5.4s,#20+	ror	w21,w21,#24+	sli	v9.4s,v0.4s,#12+	add	w13,w13,w17+	sli	v13.4s,v1.4s,#12+	add	w14,w14,w19+	sli	v17.4s,v2.4s,#12+	add	w15,w15,w20+	sli	v21.4s,v3.4s,#12+	add	w16,w16,w21+	sli	v25.4s,v4.4s,#12+	eor	w9,w9,w13+	sli	v29.4s,v5.4s,#12+	eor	w10,w10,w14+	add	v8.4s,v8.4s,v9.4s+	eor	w11,w11,w15+	add	v12.4s,v12.4s,v13.4s+	eor	w12,w12,w16+	add	v16.4s,v16.4s,v17.4s+	ror	w9,w9,#25+	add	v20.4s,v20.4s,v21.4s+	ror	w10,w10,#25+	add	v24.4s,v24.4s,v25.4s+	ror	w11,w11,#25+	add	v28.4s,v28.4s,v29.4s+	ror	w12,w12,#25+	eor	v11.16b,v11.16b,v8.16b+	add	w5,w5,w10+	eor	v15.16b,v15.16b,v12.16b+	add	w6,w6,w11+	eor	v19.16b,v19.16b,v16.16b+	add	w7,w7,w12+	eor	v23.16b,v23.16b,v20.16b+	add	w8,w8,w9+	eor	v27.16b,v27.16b,v24.16b+	eor	w21,w21,w5+	eor	v31.16b,v31.16b,v28.16b+	eor	w17,w17,w6+	tbl	v11.16b,{v11.16b},v6.16b+	eor	w19,w19,w7+	tbl	v15.16b,{v15.16b},v6.16b+	eor	w20,w20,w8+	tbl	v19.16b,{v19.16b},v6.16b+	ror	w21,w21,#16+	tbl	v23.16b,{v23.16b},v6.16b+	ror	w17,w17,#16+	tbl	v27.16b,{v27.16b},v6.16b+	ror	w19,w19,#16+	tbl	v31.16b,{v31.16b},v6.16b+	ror	w20,w20,#16+	add	v10.4s,v10.4s,v11.4s+	add	w15,w15,w21+	add	v14.4s,v14.4s,v15.4s+	add	w16,w16,w17+	add	v18.4s,v18.4s,v19.4s+	add	w13,w13,w19+	add	v22.4s,v22.4s,v23.4s+	add	w14,w14,w20+	add	v26.4s,v26.4s,v27.4s+	eor	w10,w10,w15+	add	v30.4s,v30.4s,v31.4s+	eor	w11,w11,w16+	eor	v0.16b,v9.16b,v10.16b+	eor	w12,w12,w13+	eor	v1.16b,v13.16b,v14.16b+	eor	w9,w9,w14+	eor	v2.16b,v17.16b,v18.16b+	ror	w10,w10,#20+	eor	v3.16b,v21.16b,v22.16b+	ror	w11,w11,#20+	eor	v4.16b,v25.16b,v26.16b+	ror	w12,w12,#20+	eor	v5.16b,v29.16b,v30.16b+	ror	w9,w9,#20+	ushr	v9.4s,v0.4s,#25+	add	w5,w5,w10+	ushr	v13.4s,v1.4s,#25+	add	w6,w6,w11+	ushr	v17.4s,v2.4s,#25+	add	w7,w7,w12+	ushr	v21.4s,v3.4s,#25+	add	w8,w8,w9+	ushr	v25.4s,v4.4s,#25+	eor	w21,w21,w5+	ushr	v29.4s,v5.4s,#25+	eor	w17,w17,w6+	sli	v9.4s,v0.4s,#7+	eor	w19,w19,w7+	sli	v13.4s,v1.4s,#7+	eor	w20,w20,w8+	sli	v17.4s,v2.4s,#7+	ror	w21,w21,#24+	sli	v21.4s,v3.4s,#7+	ror	w17,w17,#24+	sli	v25.4s,v4.4s,#7+	ror	w19,w19,#24+	sli	v29.4s,v5.4s,#7+	ror	w20,w20,#24+	ext	v10.16b,v10.16b,v10.16b,#8+	add	w15,w15,w21+	ext	v14.16b,v14.16b,v14.16b,#8+	add	w16,w16,w17+	ext	v18.16b,v18.16b,v18.16b,#8+	add	w13,w13,w19+	ext	v22.16b,v22.16b,v22.16b,#8+	add	w14,w14,w20+	ext	v26.16b,v26.16b,v26.16b,#8+	eor	w10,w10,w15+	ext	v30.16b,v30.16b,v30.16b,#8+	eor	w11,w11,w16+	ext	v11.16b,v11.16b,v11.16b,#12+	eor	w12,w12,w13+	ext	v15.16b,v15.16b,v15.16b,#12+	eor	w9,w9,w14+	ext	v19.16b,v19.16b,v19.16b,#12+	ror	w10,w10,#25+	ext	v23.16b,v23.16b,v23.16b,#12+	ror	w11,w11,#25+	ext	v27.16b,v27.16b,v27.16b,#12+	ror	w12,w12,#25+	ext	v31.16b,v31.16b,v31.16b,#12+	ror	w9,w9,#25+	ext	v9.16b,v9.16b,v9.16b,#4+	ext	v13.16b,v13.16b,v13.16b,#4+	ext	v17.16b,v17.16b,v17.16b,#4+	ext	v21.16b,v21.16b,v21.16b,#4+	ext	v25.16b,v25.16b,v25.16b,#4+	ext	v29.16b,v29.16b,v29.16b,#4+	add	v8.4s,v8.4s,v9.4s+	add	w5,w5,w9+	add	v12.4s,v12.4s,v13.4s+	add	w6,w6,w10+	add	v16.4s,v16.4s,v17.4s+	add	w7,w7,w11+	add	v20.4s,v20.4s,v21.4s+	add	w8,w8,w12+	add	v24.4s,v24.4s,v25.4s+	eor	w17,w17,w5+	add	v28.4s,v28.4s,v29.4s+	eor	w19,w19,w6+	eor	v11.16b,v11.16b,v8.16b+	eor	w20,w20,w7+	eor	v15.16b,v15.16b,v12.16b+	eor	w21,w21,w8+	eor	v19.16b,v19.16b,v16.16b+	ror	w17,w17,#16+	eor	v23.16b,v23.16b,v20.16b+	ror	w19,w19,#16+	eor	v27.16b,v27.16b,v24.16b+	ror	w20,w20,#16+	eor	v31.16b,v31.16b,v28.16b+	ror	w21,w21,#16+	rev32	v11.8h,v11.8h+	add	w13,w13,w17+	rev32	v15.8h,v15.8h+	add	w14,w14,w19+	rev32	v19.8h,v19.8h+	add	w15,w15,w20+	rev32	v23.8h,v23.8h+	add	w16,w16,w21+	rev32	v27.8h,v27.8h+	eor	w9,w9,w13+	rev32	v31.8h,v31.8h+	eor	w10,w10,w14+	add	v10.4s,v10.4s,v11.4s+	eor	w11,w11,w15+	add	v14.4s,v14.4s,v15.4s+	eor	w12,w12,w16+	add	v18.4s,v18.4s,v19.4s+	ror	w9,w9,#20+	add	v22.4s,v22.4s,v23.4s+	ror	w10,w10,#20+	add	v26.4s,v26.4s,v27.4s+	ror	w11,w11,#20+	add	v30.4s,v30.4s,v31.4s+	ror	w12,w12,#20+	eor	v0.16b,v9.16b,v10.16b+	add	w5,w5,w9+	eor	v1.16b,v13.16b,v14.16b+	add	w6,w6,w10+	eor	v2.16b,v17.16b,v18.16b+	add	w7,w7,w11+	eor	v3.16b,v21.16b,v22.16b+	add	w8,w8,w12+	eor	v4.16b,v25.16b,v26.16b+	eor	w17,w17,w5+	eor	v5.16b,v29.16b,v30.16b+	eor	w19,w19,w6+	ushr	v9.4s,v0.4s,#20+	eor	w20,w20,w7+	ushr	v13.4s,v1.4s,#20+	eor	w21,w21,w8+	ushr	v17.4s,v2.4s,#20+	ror	w17,w17,#24+	ushr	v21.4s,v3.4s,#20+	ror	w19,w19,#24+	ushr	v25.4s,v4.4s,#20+	ror	w20,w20,#24+	ushr	v29.4s,v5.4s,#20+	ror	w21,w21,#24+	sli	v9.4s,v0.4s,#12+	add	w13,w13,w17+	sli	v13.4s,v1.4s,#12+	add	w14,w14,w19+	sli	v17.4s,v2.4s,#12+	add	w15,w15,w20+	sli	v21.4s,v3.4s,#12+	add	w16,w16,w21+	sli	v25.4s,v4.4s,#12+	eor	w9,w9,w13+	sli	v29.4s,v5.4s,#12+	eor	w10,w10,w14+	add	v8.4s,v8.4s,v9.4s+	eor	w11,w11,w15+	add	v12.4s,v12.4s,v13.4s+	eor	w12,w12,w16+	add	v16.4s,v16.4s,v17.4s+	ror	w9,w9,#25+	add	v20.4s,v20.4s,v21.4s+	ror	w10,w10,#25+	add	v24.4s,v24.4s,v25.4s+	ror	w11,w11,#25+	add	v28.4s,v28.4s,v29.4s+	ror	w12,w12,#25+	eor	v11.16b,v11.16b,v8.16b+	add	w5,w5,w10+	eor	v15.16b,v15.16b,v12.16b+	add	w6,w6,w11+	eor	v19.16b,v19.16b,v16.16b+	add	w7,w7,w12+	eor	v23.16b,v23.16b,v20.16b+	add	w8,w8,w9+	eor	v27.16b,v27.16b,v24.16b+	eor	w21,w21,w5+	eor	v31.16b,v31.16b,v28.16b+	eor	w17,w17,w6+	tbl	v11.16b,{v11.16b},v6.16b+	eor	w19,w19,w7+	tbl	v15.16b,{v15.16b},v6.16b+	eor	w20,w20,w8+	tbl	v19.16b,{v19.16b},v6.16b+	ror	w21,w21,#16+	tbl	v23.16b,{v23.16b},v6.16b+	ror	w17,w17,#16+	tbl	v27.16b,{v27.16b},v6.16b+	ror	w19,w19,#16+	tbl	v31.16b,{v31.16b},v6.16b+	ror	w20,w20,#16+	add	v10.4s,v10.4s,v11.4s+	add	w15,w15,w21+	add	v14.4s,v14.4s,v15.4s+	add	w16,w16,w17+	add	v18.4s,v18.4s,v19.4s+	add	w13,w13,w19+	add	v22.4s,v22.4s,v23.4s+	add	w14,w14,w20+	add	v26.4s,v26.4s,v27.4s+	eor	w10,w10,w15+	add	v30.4s,v30.4s,v31.4s+	eor	w11,w11,w16+	eor	v0.16b,v9.16b,v10.16b+	eor	w12,w12,w13+	eor	v1.16b,v13.16b,v14.16b+	eor	w9,w9,w14+	eor	v2.16b,v17.16b,v18.16b+	ror	w10,w10,#20+	eor	v3.16b,v21.16b,v22.16b+	ror	w11,w11,#20+	eor	v4.16b,v25.16b,v26.16b+	ror	w12,w12,#20+	eor	v5.16b,v29.16b,v30.16b+	ror	w9,w9,#20+	ushr	v9.4s,v0.4s,#25+	add	w5,w5,w10+	ushr	v13.4s,v1.4s,#25+	add	w6,w6,w11+	ushr	v17.4s,v2.4s,#25+	add	w7,w7,w12+	ushr	v21.4s,v3.4s,#25+	add	w8,w8,w9+	ushr	v25.4s,v4.4s,#25+	eor	w21,w21,w5+	ushr	v29.4s,v5.4s,#25+	eor	w17,w17,w6+	sli	v9.4s,v0.4s,#7+	eor	w19,w19,w7+	sli	v13.4s,v1.4s,#7+	eor	w20,w20,w8+	sli	v17.4s,v2.4s,#7+	ror	w21,w21,#24+	sli	v21.4s,v3.4s,#7+	ror	w17,w17,#24+	sli	v25.4s,v4.4s,#7+	ror	w19,w19,#24+	sli	v29.4s,v5.4s,#7+	ror	w20,w20,#24+	ext	v10.16b,v10.16b,v10.16b,#8+	add	w15,w15,w21+	ext	v14.16b,v14.16b,v14.16b,#8+	add	w16,w16,w17+	ext	v18.16b,v18.16b,v18.16b,#8+	add	w13,w13,w19+	ext	v22.16b,v22.16b,v22.16b,#8+	add	w14,w14,w20+	ext	v26.16b,v26.16b,v26.16b,#8+	eor	w10,w10,w15+	ext	v30.16b,v30.16b,v30.16b,#8+	eor	w11,w11,w16+	ext	v11.16b,v11.16b,v11.16b,#4+	eor	w12,w12,w13+	ext	v15.16b,v15.16b,v15.16b,#4+	eor	w9,w9,w14+	ext	v19.16b,v19.16b,v19.16b,#4+	ror	w10,w10,#25+	ext	v23.16b,v23.16b,v23.16b,#4+	ror	w11,w11,#25+	ext	v27.16b,v27.16b,v27.16b,#4+	ror	w12,w12,#25+	ext	v31.16b,v31.16b,v31.16b,#4+	ror	w9,w9,#25+	ext	v9.16b,v9.16b,v9.16b,#12+	ext	v13.16b,v13.16b,v13.16b,#12+	ext	v17.16b,v17.16b,v17.16b,#12+	ext	v21.16b,v21.16b,v21.16b,#12+	ext	v25.16b,v25.16b,v25.16b,#12+	ext	v29.16b,v29.16b,v29.16b,#12+	cbnz	x4,Loop_lower_neon++	add	w5,w5,w22		// accumulate key block+	ldp	q0,q1,[sp,#0]+	add	x6,x6,x22,lsr#32+	ldp	q2,q3,[sp,#32]+	add	w7,w7,w23+	ldp	q4,q5,[sp,#64]+	add	x8,x8,x23,lsr#32+	ldr	q6,[sp,#96]+	add	v8.4s,v8.4s,v0.4s+	add	w9,w9,w24+	add	v12.4s,v12.4s,v0.4s+	add	x10,x10,x24,lsr#32+	add	v16.4s,v16.4s,v0.4s+	add	w11,w11,w25+	add	v20.4s,v20.4s,v0.4s+	add	x12,x12,x25,lsr#32+	add	v24.4s,v24.4s,v0.4s+	add	w13,w13,w26+	add	v28.4s,v28.4s,v0.4s+	add	x14,x14,x26,lsr#32+	add	v10.4s,v10.4s,v2.4s+	add	w15,w15,w27+	add	v14.4s,v14.4s,v2.4s+	add	x16,x16,x27,lsr#32+	add	v18.4s,v18.4s,v2.4s+	add	w17,w17,w28+	add	v22.4s,v22.4s,v2.4s+	add	x19,x19,x28,lsr#32+	add	v26.4s,v26.4s,v2.4s+	add	w20,w20,w30+	add	v30.4s,v30.4s,v2.4s+	add	x21,x21,x30,lsr#32+	add	v27.4s,v27.4s,v7.4s			// +4+	add	x5,x5,x6,lsl#32	// pack+	add	v31.4s,v31.4s,v7.4s			// +4+	add	x7,x7,x8,lsl#32+	add	v11.4s,v11.4s,v3.4s+	ldp	x6,x8,[x1,#0]		// load input+	add	v15.4s,v15.4s,v4.4s+	add	x9,x9,x10,lsl#32+	add	v19.4s,v19.4s,v5.4s+	add	x11,x11,x12,lsl#32+	add	v23.4s,v23.4s,v6.4s+	ldp	x10,x12,[x1,#16]+	add	v27.4s,v27.4s,v3.4s+	add	x13,x13,x14,lsl#32+	add	v31.4s,v31.4s,v4.4s+	add	x15,x15,x16,lsl#32+	add	v9.4s,v9.4s,v1.4s+	ldp	x14,x16,[x1,#32]+	add	v13.4s,v13.4s,v1.4s+	add	x17,x17,x19,lsl#32+	add	v17.4s,v17.4s,v1.4s+	add	x20,x20,x21,lsl#32+	add	v21.4s,v21.4s,v1.4s+	ldp	x19,x21,[x1,#48]+	add	v25.4s,v25.4s,v1.4s+	add	x1,x1,#64+	add	v29.4s,v29.4s,v1.4s++#ifdef	__AARCH64EB__+	rev	x5,x5+	rev	x7,x7+	rev	x9,x9+	rev	x11,x11+	rev	x13,x13+	rev	x15,x15+	rev	x17,x17+	rev	x20,x20+#endif+	ld1	{v0.16b,v1.16b,v2.16b,v3.16b},[x1],#64+	eor	x5,x5,x6+	eor	x7,x7,x8+	eor	x9,x9,x10+	eor	x11,x11,x12+	eor	x13,x13,x14+	eor	v8.16b,v8.16b,v0.16b+	eor	x15,x15,x16+	eor	v9.16b,v9.16b,v1.16b+	eor	x17,x17,x19+	eor	v10.16b,v10.16b,v2.16b+	eor	x20,x20,x21+	eor	v11.16b,v11.16b,v3.16b+	ld1	{v0.16b,v1.16b,v2.16b,v3.16b},[x1],#64++	stp	x5,x7,[x0,#0]		// store output+	add	x28,x28,#7			// increment counter+	stp	x9,x11,[x0,#16]+	stp	x13,x15,[x0,#32]+	stp	x17,x20,[x0,#48]+	add	x0,x0,#64+	st1	{v8.16b,v9.16b,v10.16b,v11.16b},[x0],#64++	ld1	{v8.16b,v9.16b,v10.16b,v11.16b},[x1],#64+	eor	v12.16b,v12.16b,v0.16b+	eor	v13.16b,v13.16b,v1.16b+	eor	v14.16b,v14.16b,v2.16b+	eor	v15.16b,v15.16b,v3.16b+	st1	{v12.16b,v13.16b,v14.16b,v15.16b},[x0],#64++	ld1	{v12.16b,v13.16b,v14.16b,v15.16b},[x1],#64+	eor	v16.16b,v16.16b,v8.16b+	ldp	q0,q1,[sp,#0]+	eor	v17.16b,v17.16b,v9.16b+	ldp	q2,q3,[sp,#32]+	eor	v18.16b,v18.16b,v10.16b+	eor	v19.16b,v19.16b,v11.16b+	st1	{v16.16b,v17.16b,v18.16b,v19.16b},[x0],#64++	ld1	{v16.16b,v17.16b,v18.16b,v19.16b},[x1],#64+	eor	v20.16b,v20.16b,v12.16b+	eor	v21.16b,v21.16b,v13.16b+	eor	v22.16b,v22.16b,v14.16b+	eor	v23.16b,v23.16b,v15.16b+	st1	{v20.16b,v21.16b,v22.16b,v23.16b},[x0],#64++	ld1	{v20.16b,v21.16b,v22.16b,v23.16b},[x1],#64+	eor	v24.16b,v24.16b,v16.16b+	eor	v25.16b,v25.16b,v17.16b+	eor	v26.16b,v26.16b,v18.16b+	eor	v27.16b,v27.16b,v19.16b+	st1	{v24.16b,v25.16b,v26.16b,v27.16b},[x0],#64++	shl	v8.4s,v7.4s,#1			// 4 -> 8+	eor	v28.16b,v28.16b,v20.16b+	eor	v29.16b,v29.16b,v21.16b+	eor	v30.16b,v30.16b,v22.16b+	eor	v31.16b,v31.16b,v23.16b+	st1	{v28.16b,v29.16b,v30.16b,v31.16b},[x0],#64++	add	v3.4s,v3.4s,v8.4s			// += 8+	add	v4.4s,v4.4s,v8.4s+	add	v5.4s,v5.4s,v8.4s+	add	v6.4s,v6.4s,v8.4s++	b.hs	Loop_outer_512_neon++	adds	x2,x2,#512+	ushr	v7.4s,v7.4s,#1			// 4 -> 2++	ldp	d10,d11,[sp,#128+16]		// meet ABI requirements+	ldp	d12,d13,[sp,#128+32]+	ldp	d14,d15,[sp,#128+48]++	stp	q0,q0,[sp,#16]		// wipe key off-load area+	stp	q0,q0,[sp,#48]		// [with known constant]+	stp	q0,q0,[sp,#80]++	b.eq	Ldone_512_neon++	// we have <512 bytes tail, harmonize state with other contexts+	sub	x3,x3,#16+	cmp	x2,#192+	add	sp,sp,#128+	sub	v3.4s,v3.4s,v7.4s		// -= 2+	ld1	{v8.4s,v9.4s},[x3]+	b.hs	Loop_outer_neon++	ldp	d8,d9,[sp,#0]			// meet ABI requirements+	eor	v1.16b,v1.16b,v1.16b		// cleanse key and nonce+	eor	v2.16b,v2.16b,v2.16b+	eor	v3.16b,v3.16b,v3.16b+	eor	v4.16b,v4.16b,v4.16b+	eor	v5.16b,v5.16b,v5.16b+	eor	v6.16b,v6.16b,v6.16b+	b	Loop_outer++Ldone_512_neon:+	ldp	d8,d9,[sp,#128+0]		// meet ABI requirements+	eor	v1.16b,v1.16b,v1.16b		// cleanse key and nonce+	eor	v2.16b,v2.16b,v2.16b+	eor	v3.16b,v3.16b,v3.16b+	eor	v4.16b,v4.16b,v4.16b+	eor	v5.16b,v5.16b,v5.16b+	eor	v6.16b,v6.16b,v6.16b++	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#128+64+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#12*__SIZEOF_POINTER__+.long	0xd50323bf			// autiasp+	ret+
+ cbits/asm/chacha-armv8-linux64.S view
@@ -0,0 +1,2055 @@+#ifndef	__KERNEL__+# include "arm_arch.h"++#endif++.text++.align	5+.Lsigma:+.quad	0x3320646e61707865,0x6b20657479622d32		// endian-neutral+.Lone:+.long	1,2,3,4+.Lrot24:+.long	0x02010003,0x06050407,0x0a09080b,0x0e0d0c0f+.byte	67,104,97,67,104,97,50,48,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.align	2++.globl	crypton_chacha20_asm_ctr32+.type	crypton_chacha20_asm_ctr32,%function+.align	5+crypton_chacha20_asm_ctr32:+	cbz	x2,.Labort+	cmp	x2,#192+	b.lo	.Lshort++#ifndef	__KERNEL__+	adrp	x17,crypton_armcap_P+	ldr	w17,[x17,#:lo12:crypton_armcap_P]+	tst	w17,#ARMV7_NEON+	b.ne	.Lcrypton_chacha20_asm_neon+#endif++.Lshort:+.inst	0xd503233f			// paciasp+	stp	x29,x30,[sp,#-12*__SIZEOF_POINTER__]!+	add	x29,sp,#0++	adr	x5,.Lsigma+	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	stp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	stp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	stp	x27,x28,[sp,#10*__SIZEOF_POINTER__]+	sub	sp,sp,#64++	ldp	x22,x23,[x5]		// load sigma+	ldp	x24,x25,[x3]		// load key+	ldp	x26,x27,[x3,#16]+	ldp	x28,x30,[x4]		// load counter+#ifdef	__AARCH64EB__+	ror	x24,x24,#32+	ror	x25,x25,#32+	ror	x26,x26,#32+	ror	x27,x27,#32+	ror	x28,x28,#32+	ror	x30,x30,#32+#endif++.Loop_outer:+	mov	w5,w22			// unpack key block+	lsr	x6,x22,#32+	mov	w7,w23+	lsr	x8,x23,#32+	mov	w9,w24+	lsr	x10,x24,#32+	mov	w11,w25+	lsr	x12,x25,#32+	mov	w13,w26+	lsr	x14,x26,#32+	mov	w15,w27+	lsr	x16,x27,#32+	mov	w17,w28+	lsr	x19,x28,#32+	mov	w20,w30+	lsr	x21,x30,#32++	mov	x4,#10+	subs	x2,x2,#64+.Loop:+	sub	x4,x4,#1+	add	w5,w5,w9+	add	w6,w6,w10+	add	w7,w7,w11+	add	w8,w8,w12+	eor	w17,w17,w5+	eor	w19,w19,w6+	eor	w20,w20,w7+	eor	w21,w21,w8+	ror	w17,w17,#16+	ror	w19,w19,#16+	ror	w20,w20,#16+	ror	w21,w21,#16+	add	w13,w13,w17+	add	w14,w14,w19+	add	w15,w15,w20+	add	w16,w16,w21+	eor	w9,w9,w13+	eor	w10,w10,w14+	eor	w11,w11,w15+	eor	w12,w12,w16+	ror	w9,w9,#20+	ror	w10,w10,#20+	ror	w11,w11,#20+	ror	w12,w12,#20+	add	w5,w5,w9+	add	w6,w6,w10+	add	w7,w7,w11+	add	w8,w8,w12+	eor	w17,w17,w5+	eor	w19,w19,w6+	eor	w20,w20,w7+	eor	w21,w21,w8+	ror	w17,w17,#24+	ror	w19,w19,#24+	ror	w20,w20,#24+	ror	w21,w21,#24+	add	w13,w13,w17+	add	w14,w14,w19+	add	w15,w15,w20+	add	w16,w16,w21+	eor	w9,w9,w13+	eor	w10,w10,w14+	eor	w11,w11,w15+	eor	w12,w12,w16+	ror	w9,w9,#25+	ror	w10,w10,#25+	ror	w11,w11,#25+	ror	w12,w12,#25+	add	w5,w5,w10+	add	w6,w6,w11+	add	w7,w7,w12+	add	w8,w8,w9+	eor	w21,w21,w5+	eor	w17,w17,w6+	eor	w19,w19,w7+	eor	w20,w20,w8+	ror	w21,w21,#16+	ror	w17,w17,#16+	ror	w19,w19,#16+	ror	w20,w20,#16+	add	w15,w15,w21+	add	w16,w16,w17+	add	w13,w13,w19+	add	w14,w14,w20+	eor	w10,w10,w15+	eor	w11,w11,w16+	eor	w12,w12,w13+	eor	w9,w9,w14+	ror	w10,w10,#20+	ror	w11,w11,#20+	ror	w12,w12,#20+	ror	w9,w9,#20+	add	w5,w5,w10+	add	w6,w6,w11+	add	w7,w7,w12+	add	w8,w8,w9+	eor	w21,w21,w5+	eor	w17,w17,w6+	eor	w19,w19,w7+	eor	w20,w20,w8+	ror	w21,w21,#24+	ror	w17,w17,#24+	ror	w19,w19,#24+	ror	w20,w20,#24+	add	w15,w15,w21+	add	w16,w16,w17+	add	w13,w13,w19+	add	w14,w14,w20+	eor	w10,w10,w15+	eor	w11,w11,w16+	eor	w12,w12,w13+	eor	w9,w9,w14+	ror	w10,w10,#25+	ror	w11,w11,#25+	ror	w12,w12,#25+	ror	w9,w9,#25+	cbnz	x4,.Loop++	add	w5,w5,w22		// accumulate key block+	add	x6,x6,x22,lsr#32+	add	w7,w7,w23+	add	x8,x8,x23,lsr#32+	add	w9,w9,w24+	add	x10,x10,x24,lsr#32+	add	w11,w11,w25+	add	x12,x12,x25,lsr#32+	add	w13,w13,w26+	add	x14,x14,x26,lsr#32+	add	w15,w15,w27+	add	x16,x16,x27,lsr#32+	add	w17,w17,w28+	add	x19,x19,x28,lsr#32+	add	w20,w20,w30+	add	x21,x21,x30,lsr#32++	b.lo	.Ltail++	add	x5,x5,x6,lsl#32	// pack+	add	x7,x7,x8,lsl#32+	ldp	x6,x8,[x1,#0]		// load input+	add	x9,x9,x10,lsl#32+	add	x11,x11,x12,lsl#32+	ldp	x10,x12,[x1,#16]+	add	x13,x13,x14,lsl#32+	add	x15,x15,x16,lsl#32+	ldp	x14,x16,[x1,#32]+	add	x17,x17,x19,lsl#32+	add	x20,x20,x21,lsl#32+	ldp	x19,x21,[x1,#48]+	add	x1,x1,#64+#ifdef	__AARCH64EB__+	rev	x5,x5+	rev	x7,x7+	rev	x9,x9+	rev	x11,x11+	rev	x13,x13+	rev	x15,x15+	rev	x17,x17+	rev	x20,x20+#endif+	eor	x5,x5,x6+	eor	x7,x7,x8+	eor	x9,x9,x10+	eor	x11,x11,x12+	eor	x13,x13,x14+	eor	x15,x15,x16+	eor	x17,x17,x19+	eor	x20,x20,x21++	stp	x5,x7,[x0,#0]		// store output+	add	x28,x28,#1			// increment counter+	stp	x9,x11,[x0,#16]+	stp	x13,x15,[x0,#32]+	stp	x17,x20,[x0,#48]+	add	x0,x0,#64++	b.hi	.Loop_outer++	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#64+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#12*__SIZEOF_POINTER__+.inst	0xd50323bf			// autiasp+.Labort:+	ret++.align	4+.Ltail:+	add	x2,x2,#64+.Less_than_64:+	sub	x0,x0,#1+	add	x1,x1,x2+	add	x0,x0,x2+	add	x4,sp,x2+	neg	x2,x2++	add	x5,x5,x6,lsl#32	// pack+	add	x7,x7,x8,lsl#32+	add	x9,x9,x10,lsl#32+	add	x11,x11,x12,lsl#32+	add	x13,x13,x14,lsl#32+	add	x15,x15,x16,lsl#32+	add	x17,x17,x19,lsl#32+	add	x20,x20,x21,lsl#32+#ifdef	__AARCH64EB__+	rev	x5,x5+	rev	x7,x7+	rev	x9,x9+	rev	x11,x11+	rev	x13,x13+	rev	x15,x15+	rev	x17,x17+	rev	x20,x20+#endif+	stp	x5,x7,[sp,#0]		// off-load complete block+	stp	x9,x11,[sp,#16]+	stp	x13,x15,[sp,#32]+	stp	x17,x20,[sp,#48]++.Loop_tail:+	ldrb	w10,[x1,x2]+	ldrb	w11,[x4,x2]+	add	x2,x2,#1+	eor	w10,w10,w11+	strb	w10,[x0,x2]+	cbnz	x2,.Loop_tail++	stp	xzr,xzr,[sp,#0]			// wipe off-load area+	stp	xzr,xzr,[sp,#16]+	stp	xzr,xzr,[sp,#32]+	stp	xzr,xzr,[sp,#48]++	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#64+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#12*__SIZEOF_POINTER__+.inst	0xd50323bf			// autiasp+	ret+.size	crypton_chacha20_asm_ctr32,.-crypton_chacha20_asm_ctr32++#ifdef	__KERNEL__+.globl	crypton_chacha20_asm_neon+#endif+.type	crypton_chacha20_asm_neon,%function+.align	5+crypton_chacha20_asm_neon:+.Lcrypton_chacha20_asm_neon:+.inst	0xd503233f			// paciasp+	stp	x29,x30,[sp,#-12*__SIZEOF_POINTER__]!+	add	x29,sp,#0++	adr	x5,.Lsigma+	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	stp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	stp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	stp	x27,x28,[sp,#10*__SIZEOF_POINTER__]+	cmp	x2,#512+	b.hs	.L512_or_more_neon++	sub	sp,sp,#64++	ldp	x22,x23,[x5]		// load sigma+	ld1	{v0.4s},[x5],#16+	ldp	x24,x25,[x3]		// load key+	ldp	x26,x27,[x3,#16]+	ld1	{v1.4s,v2.4s},[x3]+	ldp	x28,x30,[x4]		// load counter+	ld1	{v3.4s},[x4]+	stp	d8,d9,[sp]			// meet ABI requirements+	ld1	{v8.4s,v9.4s},[x5]+#ifdef	__AARCH64EB__+	rev64	v0.4s,v0.4s+	ror	x24,x24,#32+	ror	x25,x25,#32+	ror	x26,x26,#32+	ror	x27,x27,#32+	ror	x28,x28,#32+	ror	x30,x30,#32+#endif++.Loop_outer_neon:+	dup	v16.4s,v0.s[0]			// unpack key block+	mov	w5,w22+	dup	v20.4s,v0.s[1]+	lsr	x6,x22,#32+	dup	v24.4s,v0.s[2]+	mov	w7,w23+	dup	v28.4s,v0.s[3]+	lsr	x8,x23,#32+	dup	v17.4s,v1.s[0]+	mov	w9,w24+	dup	v21.4s,v1.s[1]+	lsr	x10,x24,#32+	dup	v25.4s,v1.s[2]+	mov	w11,w25+	dup	v29.4s,v1.s[3]+	lsr	x12,x25,#32+	dup	v19.4s,v3.s[0]+	mov	w13,w26+	dup	v23.4s,v3.s[1]+	lsr	x14,x26,#32+	dup	v27.4s,v3.s[2]+	mov	w15,w27+	dup	v31.4s,v3.s[3]+	lsr	x16,x27,#32+	add	v19.4s,v19.4s,v8.4s+	mov	w17,w28+	dup	v18.4s,v2.s[0]+	lsr	x19,x28,#32+	dup	v22.4s,v2.s[1]+	mov	w20,w30+	dup	v26.4s,v2.s[2]+	lsr	x21,x30,#32+	dup	v30.4s,v2.s[3]++	mov	x4,#10+	subs	x2,x2,#320+.Loop_neon:+	sub	x4,x4,#1+	add	v16.4s,v16.4s,v17.4s+	add	v20.4s,v20.4s,v21.4s+	add	v24.4s,v24.4s,v25.4s+	add	v28.4s,v28.4s,v29.4s+	eor	v19.16b,v19.16b,v16.16b+	eor	v23.16b,v23.16b,v20.16b+	eor	v27.16b,v27.16b,v24.16b+	eor	v31.16b,v31.16b,v28.16b+	add	w5,w5,w9+	rev32	v19.8h,v19.8h+	add	w6,w6,w10+	rev32	v23.8h,v23.8h+	add	w7,w7,w11+	rev32	v27.8h,v27.8h+	add	w8,w8,w12+	rev32	v31.8h,v31.8h+	eor	w17,w17,w5+	add	v18.4s,v18.4s,v19.4s+	eor	w19,w19,w6+	add	v22.4s,v22.4s,v23.4s+	eor	w20,w20,w7+	add	v26.4s,v26.4s,v27.4s+	eor	w21,w21,w8+	add	v30.4s,v30.4s,v31.4s+	ror	w17,w17,#16+	eor	v4.16b,v17.16b,v18.16b+	ror	w19,w19,#16+	eor	v5.16b,v21.16b,v22.16b+	ror	w20,w20,#16+	eor	v6.16b,v25.16b,v26.16b+	ror	w21,w21,#16+	eor	v7.16b,v29.16b,v30.16b+	add	w13,w13,w17+	ushr	v17.4s,v4.4s,#20+	add	w14,w14,w19+	ushr	v21.4s,v5.4s,#20+	add	w15,w15,w20+	ushr	v25.4s,v6.4s,#20+	add	w16,w16,w21+	ushr	v29.4s,v7.4s,#20+	eor	w9,w9,w13+	sli	v17.4s,v4.4s,#12+	eor	w10,w10,w14+	sli	v21.4s,v5.4s,#12+	eor	w11,w11,w15+	sli	v25.4s,v6.4s,#12+	eor	w12,w12,w16+	sli	v29.4s,v7.4s,#12+	ror	w9,w9,#20+	add	v16.4s,v16.4s,v17.4s+	ror	w10,w10,#20+	add	v20.4s,v20.4s,v21.4s+	ror	w11,w11,#20+	add	v24.4s,v24.4s,v25.4s+	ror	w12,w12,#20+	add	v28.4s,v28.4s,v29.4s+	add	w5,w5,w9+	eor	v4.16b,v19.16b,v16.16b+	add	w6,w6,w10+	eor	v5.16b,v23.16b,v20.16b+	add	w7,w7,w11+	eor	v6.16b,v27.16b,v24.16b+	add	w8,w8,w12+	eor	v7.16b,v31.16b,v28.16b+	eor	w17,w17,w5+	tbl	v19.16b,{v4.16b},v9.16b+	eor	w19,w19,w6+	tbl	v23.16b,{v5.16b},v9.16b+	eor	w20,w20,w7+	tbl	v27.16b,{v6.16b},v9.16b+	eor	w21,w21,w8+	tbl	v31.16b,{v7.16b},v9.16b+	ror	w17,w17,#24+	add	v18.4s,v18.4s,v19.4s+	ror	w19,w19,#24+	add	v22.4s,v22.4s,v23.4s+	ror	w20,w20,#24+	add	v26.4s,v26.4s,v27.4s+	ror	w21,w21,#24+	add	v30.4s,v30.4s,v31.4s+	add	w13,w13,w17+	eor	v4.16b,v17.16b,v18.16b+	add	w14,w14,w19+	eor	v5.16b,v21.16b,v22.16b+	add	w15,w15,w20+	eor	v6.16b,v25.16b,v26.16b+	add	w16,w16,w21+	eor	v7.16b,v29.16b,v30.16b+	eor	w9,w9,w13+	ushr	v17.4s,v4.4s,#25+	eor	w10,w10,w14+	ushr	v21.4s,v5.4s,#25+	eor	w11,w11,w15+	ushr	v25.4s,v6.4s,#25+	eor	w12,w12,w16+	ushr	v29.4s,v7.4s,#25+	ror	w9,w9,#25+	sli	v17.4s,v4.4s,#7+	ror	w10,w10,#25+	sli	v21.4s,v5.4s,#7+	ror	w11,w11,#25+	sli	v25.4s,v6.4s,#7+	ror	w12,w12,#25+	sli	v29.4s,v7.4s,#7+	add	v16.4s,v16.4s,v21.4s+	add	v20.4s,v20.4s,v25.4s+	add	v24.4s,v24.4s,v29.4s+	add	v28.4s,v28.4s,v17.4s+	eor	v31.16b,v31.16b,v16.16b+	eor	v19.16b,v19.16b,v20.16b+	eor	v23.16b,v23.16b,v24.16b+	eor	v27.16b,v27.16b,v28.16b+	add	w5,w5,w10+	rev32	v31.8h,v31.8h+	add	w6,w6,w11+	rev32	v19.8h,v19.8h+	add	w7,w7,w12+	rev32	v23.8h,v23.8h+	add	w8,w8,w9+	rev32	v27.8h,v27.8h+	eor	w21,w21,w5+	add	v26.4s,v26.4s,v31.4s+	eor	w17,w17,w6+	add	v30.4s,v30.4s,v19.4s+	eor	w19,w19,w7+	add	v18.4s,v18.4s,v23.4s+	eor	w20,w20,w8+	add	v22.4s,v22.4s,v27.4s+	ror	w21,w21,#16+	eor	v4.16b,v21.16b,v26.16b+	ror	w17,w17,#16+	eor	v5.16b,v25.16b,v30.16b+	ror	w19,w19,#16+	eor	v6.16b,v29.16b,v18.16b+	ror	w20,w20,#16+	eor	v7.16b,v17.16b,v22.16b+	add	w15,w15,w21+	ushr	v21.4s,v4.4s,#20+	add	w16,w16,w17+	ushr	v25.4s,v5.4s,#20+	add	w13,w13,w19+	ushr	v29.4s,v6.4s,#20+	add	w14,w14,w20+	ushr	v17.4s,v7.4s,#20+	eor	w10,w10,w15+	sli	v21.4s,v4.4s,#12+	eor	w11,w11,w16+	sli	v25.4s,v5.4s,#12+	eor	w12,w12,w13+	sli	v29.4s,v6.4s,#12+	eor	w9,w9,w14+	sli	v17.4s,v7.4s,#12+	ror	w10,w10,#20+	add	v16.4s,v16.4s,v21.4s+	ror	w11,w11,#20+	add	v20.4s,v20.4s,v25.4s+	ror	w12,w12,#20+	add	v24.4s,v24.4s,v29.4s+	ror	w9,w9,#20+	add	v28.4s,v28.4s,v17.4s+	add	w5,w5,w10+	eor	v4.16b,v31.16b,v16.16b+	add	w6,w6,w11+	eor	v5.16b,v19.16b,v20.16b+	add	w7,w7,w12+	eor	v6.16b,v23.16b,v24.16b+	add	w8,w8,w9+	eor	v7.16b,v27.16b,v28.16b+	eor	w21,w21,w5+	tbl	v31.16b,{v4.16b},v9.16b+	eor	w17,w17,w6+	tbl	v19.16b,{v5.16b},v9.16b+	eor	w19,w19,w7+	tbl	v23.16b,{v6.16b},v9.16b+	eor	w20,w20,w8+	tbl	v27.16b,{v7.16b},v9.16b+	ror	w21,w21,#24+	add	v26.4s,v26.4s,v31.4s+	ror	w17,w17,#24+	add	v30.4s,v30.4s,v19.4s+	ror	w19,w19,#24+	add	v18.4s,v18.4s,v23.4s+	ror	w20,w20,#24+	add	v22.4s,v22.4s,v27.4s+	add	w15,w15,w21+	eor	v4.16b,v21.16b,v26.16b+	add	w16,w16,w17+	eor	v5.16b,v25.16b,v30.16b+	add	w13,w13,w19+	eor	v6.16b,v29.16b,v18.16b+	add	w14,w14,w20+	eor	v7.16b,v17.16b,v22.16b+	eor	w10,w10,w15+	ushr	v21.4s,v4.4s,#25+	eor	w11,w11,w16+	ushr	v25.4s,v5.4s,#25+	eor	w12,w12,w13+	ushr	v29.4s,v6.4s,#25+	eor	w9,w9,w14+	ushr	v17.4s,v7.4s,#25+	ror	w10,w10,#25+	sli	v21.4s,v4.4s,#7+	ror	w11,w11,#25+	sli	v25.4s,v5.4s,#7+	ror	w12,w12,#25+	sli	v29.4s,v6.4s,#7+	ror	w9,w9,#25+	sli	v17.4s,v7.4s,#7+	cbnz	x4,.Loop_neon++	add	v19.4s,v19.4s,v8.4s++	zip1	v4.4s,v16.4s,v20.4s			// transpose data+	zip1	v5.4s,v24.4s,v28.4s+	zip2	v6.4s,v16.4s,v20.4s+	zip2	v7.4s,v24.4s,v28.4s+	zip1	v16.2d,v4.2d,v5.2d+	zip2	v20.2d,v4.2d,v5.2d+	zip1	v24.2d,v6.2d,v7.2d+	zip2	v28.2d,v6.2d,v7.2d++	zip1	v4.4s,v17.4s,v21.4s+	zip1	v5.4s,v25.4s,v29.4s+	zip2	v6.4s,v17.4s,v21.4s+	zip2	v7.4s,v25.4s,v29.4s+	zip1	v17.2d,v4.2d,v5.2d+	zip2	v21.2d,v4.2d,v5.2d+	zip1	v25.2d,v6.2d,v7.2d+	zip2	v29.2d,v6.2d,v7.2d++	zip1	v4.4s,v18.4s,v22.4s+	add	w5,w5,w22		// accumulate key block+	zip1	v5.4s,v26.4s,v30.4s+	add	x6,x6,x22,lsr#32+	zip2	v6.4s,v18.4s,v22.4s+	add	w7,w7,w23+	zip2	v7.4s,v26.4s,v30.4s+	add	x8,x8,x23,lsr#32+	zip1	v18.2d,v4.2d,v5.2d+	add	w9,w9,w24+	zip2	v22.2d,v4.2d,v5.2d+	add	x10,x10,x24,lsr#32+	zip1	v26.2d,v6.2d,v7.2d+	add	w11,w11,w25+	zip2	v30.2d,v6.2d,v7.2d+	add	x12,x12,x25,lsr#32++	zip1	v4.4s,v19.4s,v23.4s+	add	w13,w13,w26+	zip1	v5.4s,v27.4s,v31.4s+	add	x14,x14,x26,lsr#32+	zip2	v6.4s,v19.4s,v23.4s+	add	w15,w15,w27+	zip2	v7.4s,v27.4s,v31.4s+	add	x16,x16,x27,lsr#32+	zip1	v19.2d,v4.2d,v5.2d+	add	w17,w17,w28+	zip2	v23.2d,v4.2d,v5.2d+	add	x19,x19,x28,lsr#32+	zip1	v27.2d,v6.2d,v7.2d+	add	w20,w20,w30+	zip2	v31.2d,v6.2d,v7.2d+	add	x21,x21,x30,lsr#32++	b.lo	.Ltail_neon++	add	x5,x5,x6,lsl#32	// pack+	add	x7,x7,x8,lsl#32+	ldp	x6,x8,[x1,#0]		// load input+	add	v16.4s,v16.4s,v0.4s			// accumulate key block+	add	x9,x9,x10,lsl#32+	add	x11,x11,x12,lsl#32+	ldp	x10,x12,[x1,#16]+	add	v17.4s,v17.4s,v1.4s+	add	x13,x13,x14,lsl#32+	add	x15,x15,x16,lsl#32+	ldp	x14,x16,[x1,#32]+	add	v18.4s,v18.4s,v2.4s+	add	x17,x17,x19,lsl#32+	add	x20,x20,x21,lsl#32+	ldp	x19,x21,[x1,#48]+	add	v19.4s,v19.4s,v3.4s+	add	x1,x1,#64+#ifdef	__AARCH64EB__+	rev	x5,x5+	rev	x7,x7+	rev	x9,x9+	rev	x11,x11+	rev	x13,x13+	rev	x15,x15+	rev	x17,x17+	rev	x20,x20+#endif+	ld1	{v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64+	eor	x5,x5,x6+	add	v20.4s,v20.4s,v0.4s+	eor	x7,x7,x8+	add	v21.4s,v21.4s,v1.4s+	eor	x9,x9,x10+	add	v22.4s,v22.4s,v2.4s+	eor	x11,x11,x12+	add	v23.4s,v23.4s,v3.4s+	eor	x13,x13,x14+	eor	v16.16b,v16.16b,v4.16b+	movi	v4.4s,#5+	eor	x15,x15,x16+	eor	v17.16b,v17.16b,v5.16b+	eor	x17,x17,x19+	eor	v18.16b,v18.16b,v6.16b+	eor	x20,x20,x21+	eor	v19.16b,v19.16b,v7.16b+	add	v8.4s,v8.4s,v4.4s			// += 5+	ld1	{v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64++	stp	x5,x7,[x0,#0]		// store output+	add	x28,x28,#5			// increment counter+	stp	x9,x11,[x0,#16]+	stp	x13,x15,[x0,#32]+	stp	x17,x20,[x0,#48]+	add	x0,x0,#64++	st1	{v16.16b,v17.16b,v18.16b,v19.16b},[x0],#64+	add	v24.4s,v24.4s,v0.4s+	add	v25.4s,v25.4s,v1.4s+	add	v26.4s,v26.4s,v2.4s+	add	v27.4s,v27.4s,v3.4s+	ld1	{v16.16b,v17.16b,v18.16b,v19.16b},[x1],#64++	eor	v20.16b,v20.16b,v4.16b+	eor	v21.16b,v21.16b,v5.16b+	eor	v22.16b,v22.16b,v6.16b+	eor	v23.16b,v23.16b,v7.16b+	st1	{v20.16b,v21.16b,v22.16b,v23.16b},[x0],#64+	add	v28.4s,v28.4s,v0.4s+	add	v29.4s,v29.4s,v1.4s+	add	v30.4s,v30.4s,v2.4s+	add	v31.4s,v31.4s,v3.4s+	ld1	{v20.16b,v21.16b,v22.16b,v23.16b},[x1],#64++	eor	v24.16b,v24.16b,v16.16b+	eor	v25.16b,v25.16b,v17.16b+	eor	v26.16b,v26.16b,v18.16b+	eor	v27.16b,v27.16b,v19.16b+	st1	{v24.16b,v25.16b,v26.16b,v27.16b},[x0],#64++	eor	v28.16b,v28.16b,v20.16b+	eor	v29.16b,v29.16b,v21.16b+	eor	v30.16b,v30.16b,v22.16b+	eor	v31.16b,v31.16b,v23.16b+	st1	{v28.16b,v29.16b,v30.16b,v31.16b},[x0],#64++	b.hi	.Loop_outer_neon++	ldp	d8,d9,[sp]			// meet ABI requirements+	eor	v1.16b,v1.16b,v1.16b		// cleanse key and nonce+	eor	v2.16b,v2.16b,v2.16b+	eor	v3.16b,v3.16b,v3.16b++	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#64+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#12*__SIZEOF_POINTER__+.inst	0xd50323bf			// autiasp+	ret++.align	4+.Ltail_neon:+	add	x2,x2,#320+	ldp	d8,d9,[sp]			// meet ABI requirements+	cmp	x2,#64+	b.lo	.Less_than_64_neon++	add	x5,x5,x6,lsl#32	// pack+	add	x7,x7,x8,lsl#32+	ldp	x6,x8,[x1,#0]		// load input+	add	x9,x9,x10,lsl#32+	add	x11,x11,x12,lsl#32+	ldp	x10,x12,[x1,#16]+	add	x13,x13,x14,lsl#32+	add	x15,x15,x16,lsl#32+	ldp	x14,x16,[x1,#32]+	add	x17,x17,x19,lsl#32+	add	x20,x20,x21,lsl#32+	ldp	x19,x21,[x1,#48]+	add	x1,x1,#64+#ifdef	__AARCH64EB__+	rev	x5,x5+	rev	x7,x7+	rev	x9,x9+	rev	x11,x11+	rev	x13,x13+	rev	x15,x15+	rev	x17,x17+	rev	x20,x20+#endif+	eor	x5,x5,x6+	eor	x7,x7,x8+	eor	x9,x9,x10+	eor	x11,x11,x12+	eor	x13,x13,x14+	eor	x15,x15,x16+	eor	x17,x17,x19+	eor	x20,x20,x21++	stp	x5,x7,[x0,#0]		// store output+	add	v16.4s,v16.4s,v0.4s			// accumulate key block+	stp	x9,x11,[x0,#16]+	add	v17.4s,v17.4s,v1.4s+	stp	x13,x15,[x0,#32]+	add	v18.4s,v18.4s,v2.4s+	stp	x17,x20,[x0,#48]+	add	v19.4s,v19.4s,v3.4s+	add	x0,x0,#64+	b.eq	.Ldone_neon+	sub	x2,x2,#64+	cmp	x2,#64+	b.lo	.Last_neon++	ld1	{v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64+	eor	v16.16b,v16.16b,v4.16b+	eor	v17.16b,v17.16b,v5.16b+	eor	v18.16b,v18.16b,v6.16b+	eor	v19.16b,v19.16b,v7.16b+	st1	{v16.16b,v17.16b,v18.16b,v19.16b},[x0],#64+	b.eq	.Ldone_neon++	add	v16.4s,v20.4s,v0.4s+	add	v17.4s,v21.4s,v1.4s+	sub	x2,x2,#64+	add	v18.4s,v22.4s,v2.4s+	cmp	x2,#64+	add	v19.4s,v23.4s,v3.4s+	b.lo	.Last_neon++	ld1	{v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64+	eor	v20.16b,v16.16b,v4.16b+	eor	v21.16b,v17.16b,v5.16b+	eor	v22.16b,v18.16b,v6.16b+	eor	v23.16b,v19.16b,v7.16b+	st1	{v20.16b,v21.16b,v22.16b,v23.16b},[x0],#64+	b.eq	.Ldone_neon++	add	v16.4s,v24.4s,v0.4s+	add	v17.4s,v25.4s,v1.4s+	sub	x2,x2,#64+	add	v18.4s,v26.4s,v2.4s+	cmp	x2,#64+	add	v19.4s,v27.4s,v3.4s+	b.lo	.Last_neon++	ld1	{v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64+	eor	v24.16b,v16.16b,v4.16b+	eor	v25.16b,v17.16b,v5.16b+	eor	v26.16b,v18.16b,v6.16b+	eor	v27.16b,v19.16b,v7.16b+	st1	{v24.16b,v25.16b,v26.16b,v27.16b},[x0],#64+	b.eq	.Ldone_neon++	add	v16.4s,v28.4s,v0.4s+	add	v17.4s,v29.4s,v1.4s+	add	v18.4s,v30.4s,v2.4s+	add	v19.4s,v31.4s,v3.4s+	sub	x2,x2,#64++.Last_neon:+	st1	{v16.16b,v17.16b,v18.16b,v19.16b},[sp]		// off-load complete block++	sub	x0,x0,#1+	add	x1,x1,x2+	add	x0,x0,x2+	add	x4,sp,x2+	neg	x2,x2++.Loop_tail_neon:+	ldrb	w10,[x1,x2]+	ldrb	w11,[x4,x2]+	add	x2,x2,#1+	eor	w10,w10,w11+	strb	w10,[x0,x2]+	cbnz	x2,.Loop_tail_neon++	stp	q0,q0,[sp,#0]		// wipe off-load area+	stp	q0,q0,[sp,#32]		// [with known constant]++.Ldone_neon:+	eor	v1.16b,v1.16b,v1.16b		// cleanse key and nonce+	eor	v2.16b,v2.16b,v2.16b+	eor	v3.16b,v3.16b,v3.16b++	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#64+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#12*__SIZEOF_POINTER__+.inst	0xd50323bf			// autiasp+	ret++.align	4+.Less_than_64_neon:+	eor	v1.16b,v1.16b,v1.16b		// cleanse key and nonce+	eor	v2.16b,v2.16b,v2.16b+	eor	v3.16b,v3.16b,v3.16b+	b	.Less_than_64+.size	crypton_chacha20_asm_neon,.-crypton_chacha20_asm_neon+.type	crypton_chacha20_asm_512_neon,%function+.align	5+crypton_chacha20_asm_512_neon:+.inst	0xd503233f			// paciasp+	stp	x29,x30,[sp,#-12*__SIZEOF_POINTER__]!+	add	x29,sp,#0++	adr	x5,.Lsigma+	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	stp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	stp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	stp	x27,x28,[sp,#10*__SIZEOF_POINTER__]++.L512_or_more_neon:+	sub	sp,sp,#128+64++	eor	v7.16b,v7.16b,v7.16b+	ldp	x22,x23,[x5]		// load sigma+	ld1	{v0.4s},[x5],#16+	ldp	x24,x25,[x3]		// load key+	ldp	x26,x27,[x3,#16]+	ld1	{v1.4s,v2.4s},[x3]+	ldp	x28,x30,[x4]		// load counter+	ld1	{v3.4s},[x4]+	ld1	{v7.s}[0],[x5]+	add	x3,x5,#16+#ifdef	__AARCH64EB__+	rev64	v0.4s,v0.4s+	ror	x24,x24,#32+	ror	x25,x25,#32+	ror	x26,x26,#32+	ror	x27,x27,#32+	ror	x28,x28,#32+	ror	x30,x30,#32+#endif+	add	v3.4s,v3.4s,v7.4s		// += 1+	stp	q0,q1,[sp,#0]		// off-load key block, invariant part+	add	v3.4s,v3.4s,v7.4s		// not typo+	str	q2,[sp,#32]+	add	v4.4s,v3.4s,v7.4s+	add	v5.4s,v4.4s,v7.4s+	add	v6.4s,v5.4s,v7.4s+	shl	v7.4s,v7.4s,#2			// 1 -> 4++	stp	d8,d9,[sp,#128+0]		// meet ABI requirements+	stp	d10,d11,[sp,#128+16]+	stp	d12,d13,[sp,#128+32]+	stp	d14,d15,[sp,#128+48]++	sub	x2,x2,#512			// not typo++.Loop_outer_512_neon:+	mov	v8.16b,v0.16b+	mov	v12.16b,v0.16b+	mov	v16.16b,v0.16b+	mov	v20.16b,v0.16b+	mov	v24.16b,v0.16b+	mov	v28.16b,v0.16b+	mov	v9.16b,v1.16b+	mov	w5,w22			// unpack key block+	mov	v13.16b,v1.16b+	lsr	x6,x22,#32+	mov	v17.16b,v1.16b+	mov	w7,w23+	mov	v21.16b,v1.16b+	lsr	x8,x23,#32+	mov	v25.16b,v1.16b+	mov	w9,w24+	mov	v29.16b,v1.16b+	lsr	x10,x24,#32+	mov	v11.16b,v3.16b+	mov	w11,w25+	mov	v15.16b,v4.16b+	lsr	x12,x25,#32+	mov	v19.16b,v5.16b+	mov	w13,w26+	mov	v23.16b,v6.16b+	lsr	x14,x26,#32+	mov	v10.16b,v2.16b+	mov	w15,w27+	mov	v14.16b,v2.16b+	lsr	x16,x27,#32+	add	v27.4s,v11.4s,v7.4s			// +4+	mov	w17,w28+	add	v31.4s,v15.4s,v7.4s			// +4+	lsr	x19,x28,#32+	mov	v18.16b,v2.16b+	mov	w20,w30+	mov	v22.16b,v2.16b+	lsr	x21,x30,#32+	mov	v26.16b,v2.16b+	stp	q3,q4,[sp,#48]		// off-load key block, variable part+	mov	v30.16b,v2.16b+	stp	q5,q6,[sp,#80]++	mov	x4,#5+	ld1	{v6.4s},[x3]+	subs	x2,x2,#512+.Loop_upper_neon:+	sub	x4,x4,#1+	add	v8.4s,v8.4s,v9.4s+	add	w5,w5,w9+	add	v12.4s,v12.4s,v13.4s+	add	w6,w6,w10+	add	v16.4s,v16.4s,v17.4s+	add	w7,w7,w11+	add	v20.4s,v20.4s,v21.4s+	add	w8,w8,w12+	add	v24.4s,v24.4s,v25.4s+	eor	w17,w17,w5+	add	v28.4s,v28.4s,v29.4s+	eor	w19,w19,w6+	eor	v11.16b,v11.16b,v8.16b+	eor	w20,w20,w7+	eor	v15.16b,v15.16b,v12.16b+	eor	w21,w21,w8+	eor	v19.16b,v19.16b,v16.16b+	ror	w17,w17,#16+	eor	v23.16b,v23.16b,v20.16b+	ror	w19,w19,#16+	eor	v27.16b,v27.16b,v24.16b+	ror	w20,w20,#16+	eor	v31.16b,v31.16b,v28.16b+	ror	w21,w21,#16+	rev32	v11.8h,v11.8h+	add	w13,w13,w17+	rev32	v15.8h,v15.8h+	add	w14,w14,w19+	rev32	v19.8h,v19.8h+	add	w15,w15,w20+	rev32	v23.8h,v23.8h+	add	w16,w16,w21+	rev32	v27.8h,v27.8h+	eor	w9,w9,w13+	rev32	v31.8h,v31.8h+	eor	w10,w10,w14+	add	v10.4s,v10.4s,v11.4s+	eor	w11,w11,w15+	add	v14.4s,v14.4s,v15.4s+	eor	w12,w12,w16+	add	v18.4s,v18.4s,v19.4s+	ror	w9,w9,#20+	add	v22.4s,v22.4s,v23.4s+	ror	w10,w10,#20+	add	v26.4s,v26.4s,v27.4s+	ror	w11,w11,#20+	add	v30.4s,v30.4s,v31.4s+	ror	w12,w12,#20+	eor	v0.16b,v9.16b,v10.16b+	add	w5,w5,w9+	eor	v1.16b,v13.16b,v14.16b+	add	w6,w6,w10+	eor	v2.16b,v17.16b,v18.16b+	add	w7,w7,w11+	eor	v3.16b,v21.16b,v22.16b+	add	w8,w8,w12+	eor	v4.16b,v25.16b,v26.16b+	eor	w17,w17,w5+	eor	v5.16b,v29.16b,v30.16b+	eor	w19,w19,w6+	ushr	v9.4s,v0.4s,#20+	eor	w20,w20,w7+	ushr	v13.4s,v1.4s,#20+	eor	w21,w21,w8+	ushr	v17.4s,v2.4s,#20+	ror	w17,w17,#24+	ushr	v21.4s,v3.4s,#20+	ror	w19,w19,#24+	ushr	v25.4s,v4.4s,#20+	ror	w20,w20,#24+	ushr	v29.4s,v5.4s,#20+	ror	w21,w21,#24+	sli	v9.4s,v0.4s,#12+	add	w13,w13,w17+	sli	v13.4s,v1.4s,#12+	add	w14,w14,w19+	sli	v17.4s,v2.4s,#12+	add	w15,w15,w20+	sli	v21.4s,v3.4s,#12+	add	w16,w16,w21+	sli	v25.4s,v4.4s,#12+	eor	w9,w9,w13+	sli	v29.4s,v5.4s,#12+	eor	w10,w10,w14+	add	v8.4s,v8.4s,v9.4s+	eor	w11,w11,w15+	add	v12.4s,v12.4s,v13.4s+	eor	w12,w12,w16+	add	v16.4s,v16.4s,v17.4s+	ror	w9,w9,#25+	add	v20.4s,v20.4s,v21.4s+	ror	w10,w10,#25+	add	v24.4s,v24.4s,v25.4s+	ror	w11,w11,#25+	add	v28.4s,v28.4s,v29.4s+	ror	w12,w12,#25+	eor	v11.16b,v11.16b,v8.16b+	add	w5,w5,w10+	eor	v15.16b,v15.16b,v12.16b+	add	w6,w6,w11+	eor	v19.16b,v19.16b,v16.16b+	add	w7,w7,w12+	eor	v23.16b,v23.16b,v20.16b+	add	w8,w8,w9+	eor	v27.16b,v27.16b,v24.16b+	eor	w21,w21,w5+	eor	v31.16b,v31.16b,v28.16b+	eor	w17,w17,w6+	tbl	v11.16b,{v11.16b},v6.16b+	eor	w19,w19,w7+	tbl	v15.16b,{v15.16b},v6.16b+	eor	w20,w20,w8+	tbl	v19.16b,{v19.16b},v6.16b+	ror	w21,w21,#16+	tbl	v23.16b,{v23.16b},v6.16b+	ror	w17,w17,#16+	tbl	v27.16b,{v27.16b},v6.16b+	ror	w19,w19,#16+	tbl	v31.16b,{v31.16b},v6.16b+	ror	w20,w20,#16+	add	v10.4s,v10.4s,v11.4s+	add	w15,w15,w21+	add	v14.4s,v14.4s,v15.4s+	add	w16,w16,w17+	add	v18.4s,v18.4s,v19.4s+	add	w13,w13,w19+	add	v22.4s,v22.4s,v23.4s+	add	w14,w14,w20+	add	v26.4s,v26.4s,v27.4s+	eor	w10,w10,w15+	add	v30.4s,v30.4s,v31.4s+	eor	w11,w11,w16+	eor	v0.16b,v9.16b,v10.16b+	eor	w12,w12,w13+	eor	v1.16b,v13.16b,v14.16b+	eor	w9,w9,w14+	eor	v2.16b,v17.16b,v18.16b+	ror	w10,w10,#20+	eor	v3.16b,v21.16b,v22.16b+	ror	w11,w11,#20+	eor	v4.16b,v25.16b,v26.16b+	ror	w12,w12,#20+	eor	v5.16b,v29.16b,v30.16b+	ror	w9,w9,#20+	ushr	v9.4s,v0.4s,#25+	add	w5,w5,w10+	ushr	v13.4s,v1.4s,#25+	add	w6,w6,w11+	ushr	v17.4s,v2.4s,#25+	add	w7,w7,w12+	ushr	v21.4s,v3.4s,#25+	add	w8,w8,w9+	ushr	v25.4s,v4.4s,#25+	eor	w21,w21,w5+	ushr	v29.4s,v5.4s,#25+	eor	w17,w17,w6+	sli	v9.4s,v0.4s,#7+	eor	w19,w19,w7+	sli	v13.4s,v1.4s,#7+	eor	w20,w20,w8+	sli	v17.4s,v2.4s,#7+	ror	w21,w21,#24+	sli	v21.4s,v3.4s,#7+	ror	w17,w17,#24+	sli	v25.4s,v4.4s,#7+	ror	w19,w19,#24+	sli	v29.4s,v5.4s,#7+	ror	w20,w20,#24+	ext	v10.16b,v10.16b,v10.16b,#8+	add	w15,w15,w21+	ext	v14.16b,v14.16b,v14.16b,#8+	add	w16,w16,w17+	ext	v18.16b,v18.16b,v18.16b,#8+	add	w13,w13,w19+	ext	v22.16b,v22.16b,v22.16b,#8+	add	w14,w14,w20+	ext	v26.16b,v26.16b,v26.16b,#8+	eor	w10,w10,w15+	ext	v30.16b,v30.16b,v30.16b,#8+	eor	w11,w11,w16+	ext	v11.16b,v11.16b,v11.16b,#12+	eor	w12,w12,w13+	ext	v15.16b,v15.16b,v15.16b,#12+	eor	w9,w9,w14+	ext	v19.16b,v19.16b,v19.16b,#12+	ror	w10,w10,#25+	ext	v23.16b,v23.16b,v23.16b,#12+	ror	w11,w11,#25+	ext	v27.16b,v27.16b,v27.16b,#12+	ror	w12,w12,#25+	ext	v31.16b,v31.16b,v31.16b,#12+	ror	w9,w9,#25+	ext	v9.16b,v9.16b,v9.16b,#4+	ext	v13.16b,v13.16b,v13.16b,#4+	ext	v17.16b,v17.16b,v17.16b,#4+	ext	v21.16b,v21.16b,v21.16b,#4+	ext	v25.16b,v25.16b,v25.16b,#4+	ext	v29.16b,v29.16b,v29.16b,#4+	add	v8.4s,v8.4s,v9.4s+	add	w5,w5,w9+	add	v12.4s,v12.4s,v13.4s+	add	w6,w6,w10+	add	v16.4s,v16.4s,v17.4s+	add	w7,w7,w11+	add	v20.4s,v20.4s,v21.4s+	add	w8,w8,w12+	add	v24.4s,v24.4s,v25.4s+	eor	w17,w17,w5+	add	v28.4s,v28.4s,v29.4s+	eor	w19,w19,w6+	eor	v11.16b,v11.16b,v8.16b+	eor	w20,w20,w7+	eor	v15.16b,v15.16b,v12.16b+	eor	w21,w21,w8+	eor	v19.16b,v19.16b,v16.16b+	ror	w17,w17,#16+	eor	v23.16b,v23.16b,v20.16b+	ror	w19,w19,#16+	eor	v27.16b,v27.16b,v24.16b+	ror	w20,w20,#16+	eor	v31.16b,v31.16b,v28.16b+	ror	w21,w21,#16+	rev32	v11.8h,v11.8h+	add	w13,w13,w17+	rev32	v15.8h,v15.8h+	add	w14,w14,w19+	rev32	v19.8h,v19.8h+	add	w15,w15,w20+	rev32	v23.8h,v23.8h+	add	w16,w16,w21+	rev32	v27.8h,v27.8h+	eor	w9,w9,w13+	rev32	v31.8h,v31.8h+	eor	w10,w10,w14+	add	v10.4s,v10.4s,v11.4s+	eor	w11,w11,w15+	add	v14.4s,v14.4s,v15.4s+	eor	w12,w12,w16+	add	v18.4s,v18.4s,v19.4s+	ror	w9,w9,#20+	add	v22.4s,v22.4s,v23.4s+	ror	w10,w10,#20+	add	v26.4s,v26.4s,v27.4s+	ror	w11,w11,#20+	add	v30.4s,v30.4s,v31.4s+	ror	w12,w12,#20+	eor	v0.16b,v9.16b,v10.16b+	add	w5,w5,w9+	eor	v1.16b,v13.16b,v14.16b+	add	w6,w6,w10+	eor	v2.16b,v17.16b,v18.16b+	add	w7,w7,w11+	eor	v3.16b,v21.16b,v22.16b+	add	w8,w8,w12+	eor	v4.16b,v25.16b,v26.16b+	eor	w17,w17,w5+	eor	v5.16b,v29.16b,v30.16b+	eor	w19,w19,w6+	ushr	v9.4s,v0.4s,#20+	eor	w20,w20,w7+	ushr	v13.4s,v1.4s,#20+	eor	w21,w21,w8+	ushr	v17.4s,v2.4s,#20+	ror	w17,w17,#24+	ushr	v21.4s,v3.4s,#20+	ror	w19,w19,#24+	ushr	v25.4s,v4.4s,#20+	ror	w20,w20,#24+	ushr	v29.4s,v5.4s,#20+	ror	w21,w21,#24+	sli	v9.4s,v0.4s,#12+	add	w13,w13,w17+	sli	v13.4s,v1.4s,#12+	add	w14,w14,w19+	sli	v17.4s,v2.4s,#12+	add	w15,w15,w20+	sli	v21.4s,v3.4s,#12+	add	w16,w16,w21+	sli	v25.4s,v4.4s,#12+	eor	w9,w9,w13+	sli	v29.4s,v5.4s,#12+	eor	w10,w10,w14+	add	v8.4s,v8.4s,v9.4s+	eor	w11,w11,w15+	add	v12.4s,v12.4s,v13.4s+	eor	w12,w12,w16+	add	v16.4s,v16.4s,v17.4s+	ror	w9,w9,#25+	add	v20.4s,v20.4s,v21.4s+	ror	w10,w10,#25+	add	v24.4s,v24.4s,v25.4s+	ror	w11,w11,#25+	add	v28.4s,v28.4s,v29.4s+	ror	w12,w12,#25+	eor	v11.16b,v11.16b,v8.16b+	add	w5,w5,w10+	eor	v15.16b,v15.16b,v12.16b+	add	w6,w6,w11+	eor	v19.16b,v19.16b,v16.16b+	add	w7,w7,w12+	eor	v23.16b,v23.16b,v20.16b+	add	w8,w8,w9+	eor	v27.16b,v27.16b,v24.16b+	eor	w21,w21,w5+	eor	v31.16b,v31.16b,v28.16b+	eor	w17,w17,w6+	tbl	v11.16b,{v11.16b},v6.16b+	eor	w19,w19,w7+	tbl	v15.16b,{v15.16b},v6.16b+	eor	w20,w20,w8+	tbl	v19.16b,{v19.16b},v6.16b+	ror	w21,w21,#16+	tbl	v23.16b,{v23.16b},v6.16b+	ror	w17,w17,#16+	tbl	v27.16b,{v27.16b},v6.16b+	ror	w19,w19,#16+	tbl	v31.16b,{v31.16b},v6.16b+	ror	w20,w20,#16+	add	v10.4s,v10.4s,v11.4s+	add	w15,w15,w21+	add	v14.4s,v14.4s,v15.4s+	add	w16,w16,w17+	add	v18.4s,v18.4s,v19.4s+	add	w13,w13,w19+	add	v22.4s,v22.4s,v23.4s+	add	w14,w14,w20+	add	v26.4s,v26.4s,v27.4s+	eor	w10,w10,w15+	add	v30.4s,v30.4s,v31.4s+	eor	w11,w11,w16+	eor	v0.16b,v9.16b,v10.16b+	eor	w12,w12,w13+	eor	v1.16b,v13.16b,v14.16b+	eor	w9,w9,w14+	eor	v2.16b,v17.16b,v18.16b+	ror	w10,w10,#20+	eor	v3.16b,v21.16b,v22.16b+	ror	w11,w11,#20+	eor	v4.16b,v25.16b,v26.16b+	ror	w12,w12,#20+	eor	v5.16b,v29.16b,v30.16b+	ror	w9,w9,#20+	ushr	v9.4s,v0.4s,#25+	add	w5,w5,w10+	ushr	v13.4s,v1.4s,#25+	add	w6,w6,w11+	ushr	v17.4s,v2.4s,#25+	add	w7,w7,w12+	ushr	v21.4s,v3.4s,#25+	add	w8,w8,w9+	ushr	v25.4s,v4.4s,#25+	eor	w21,w21,w5+	ushr	v29.4s,v5.4s,#25+	eor	w17,w17,w6+	sli	v9.4s,v0.4s,#7+	eor	w19,w19,w7+	sli	v13.4s,v1.4s,#7+	eor	w20,w20,w8+	sli	v17.4s,v2.4s,#7+	ror	w21,w21,#24+	sli	v21.4s,v3.4s,#7+	ror	w17,w17,#24+	sli	v25.4s,v4.4s,#7+	ror	w19,w19,#24+	sli	v29.4s,v5.4s,#7+	ror	w20,w20,#24+	ext	v10.16b,v10.16b,v10.16b,#8+	add	w15,w15,w21+	ext	v14.16b,v14.16b,v14.16b,#8+	add	w16,w16,w17+	ext	v18.16b,v18.16b,v18.16b,#8+	add	w13,w13,w19+	ext	v22.16b,v22.16b,v22.16b,#8+	add	w14,w14,w20+	ext	v26.16b,v26.16b,v26.16b,#8+	eor	w10,w10,w15+	ext	v30.16b,v30.16b,v30.16b,#8+	eor	w11,w11,w16+	ext	v11.16b,v11.16b,v11.16b,#4+	eor	w12,w12,w13+	ext	v15.16b,v15.16b,v15.16b,#4+	eor	w9,w9,w14+	ext	v19.16b,v19.16b,v19.16b,#4+	ror	w10,w10,#25+	ext	v23.16b,v23.16b,v23.16b,#4+	ror	w11,w11,#25+	ext	v27.16b,v27.16b,v27.16b,#4+	ror	w12,w12,#25+	ext	v31.16b,v31.16b,v31.16b,#4+	ror	w9,w9,#25+	ext	v9.16b,v9.16b,v9.16b,#12+	ext	v13.16b,v13.16b,v13.16b,#12+	ext	v17.16b,v17.16b,v17.16b,#12+	ext	v21.16b,v21.16b,v21.16b,#12+	ext	v25.16b,v25.16b,v25.16b,#12+	ext	v29.16b,v29.16b,v29.16b,#12+	cbnz	x4,.Loop_upper_neon++	add	w5,w5,w22		// accumulate key block+	add	x6,x6,x22,lsr#32+	add	w7,w7,w23+	add	x8,x8,x23,lsr#32+	add	w9,w9,w24+	add	x10,x10,x24,lsr#32+	add	w11,w11,w25+	add	x12,x12,x25,lsr#32+	add	w13,w13,w26+	add	x14,x14,x26,lsr#32+	add	w15,w15,w27+	add	x16,x16,x27,lsr#32+	add	w17,w17,w28+	add	x19,x19,x28,lsr#32+	add	w20,w20,w30+	add	x21,x21,x30,lsr#32++	add	x5,x5,x6,lsl#32	// pack+	add	x7,x7,x8,lsl#32+	ldp	x6,x8,[x1,#0]		// load input+	add	x9,x9,x10,lsl#32+	add	x11,x11,x12,lsl#32+	ldp	x10,x12,[x1,#16]+	add	x13,x13,x14,lsl#32+	add	x15,x15,x16,lsl#32+	ldp	x14,x16,[x1,#32]+	add	x17,x17,x19,lsl#32+	add	x20,x20,x21,lsl#32+	ldp	x19,x21,[x1,#48]+	add	x1,x1,#64+#ifdef	__AARCH64EB__+	rev	x5,x5+	rev	x7,x7+	rev	x9,x9+	rev	x11,x11+	rev	x13,x13+	rev	x15,x15+	rev	x17,x17+	rev	x20,x20+#endif+	eor	x5,x5,x6+	eor	x7,x7,x8+	eor	x9,x9,x10+	eor	x11,x11,x12+	eor	x13,x13,x14+	eor	x15,x15,x16+	eor	x17,x17,x19+	eor	x20,x20,x21++	stp	x5,x7,[x0,#0]		// store output+	add	x28,x28,#1			// increment counter+	mov	w5,w22			// unpack key block+	lsr	x6,x22,#32+	stp	x9,x11,[x0,#16]+	mov	w7,w23+	lsr	x8,x23,#32+	stp	x13,x15,[x0,#32]+	mov	w9,w24+	lsr	x10,x24,#32+	stp	x17,x20,[x0,#48]+	add	x0,x0,#64+	mov	w11,w25+	lsr	x12,x25,#32+	mov	w13,w26+	lsr	x14,x26,#32+	mov	w15,w27+	lsr	x16,x27,#32+	mov	w17,w28+	lsr	x19,x28,#32+	mov	w20,w30+	lsr	x21,x30,#32++	mov	x4,#5+.Loop_lower_neon:+	sub	x4,x4,#1+	add	v8.4s,v8.4s,v9.4s+	add	w5,w5,w9+	add	v12.4s,v12.4s,v13.4s+	add	w6,w6,w10+	add	v16.4s,v16.4s,v17.4s+	add	w7,w7,w11+	add	v20.4s,v20.4s,v21.4s+	add	w8,w8,w12+	add	v24.4s,v24.4s,v25.4s+	eor	w17,w17,w5+	add	v28.4s,v28.4s,v29.4s+	eor	w19,w19,w6+	eor	v11.16b,v11.16b,v8.16b+	eor	w20,w20,w7+	eor	v15.16b,v15.16b,v12.16b+	eor	w21,w21,w8+	eor	v19.16b,v19.16b,v16.16b+	ror	w17,w17,#16+	eor	v23.16b,v23.16b,v20.16b+	ror	w19,w19,#16+	eor	v27.16b,v27.16b,v24.16b+	ror	w20,w20,#16+	eor	v31.16b,v31.16b,v28.16b+	ror	w21,w21,#16+	rev32	v11.8h,v11.8h+	add	w13,w13,w17+	rev32	v15.8h,v15.8h+	add	w14,w14,w19+	rev32	v19.8h,v19.8h+	add	w15,w15,w20+	rev32	v23.8h,v23.8h+	add	w16,w16,w21+	rev32	v27.8h,v27.8h+	eor	w9,w9,w13+	rev32	v31.8h,v31.8h+	eor	w10,w10,w14+	add	v10.4s,v10.4s,v11.4s+	eor	w11,w11,w15+	add	v14.4s,v14.4s,v15.4s+	eor	w12,w12,w16+	add	v18.4s,v18.4s,v19.4s+	ror	w9,w9,#20+	add	v22.4s,v22.4s,v23.4s+	ror	w10,w10,#20+	add	v26.4s,v26.4s,v27.4s+	ror	w11,w11,#20+	add	v30.4s,v30.4s,v31.4s+	ror	w12,w12,#20+	eor	v0.16b,v9.16b,v10.16b+	add	w5,w5,w9+	eor	v1.16b,v13.16b,v14.16b+	add	w6,w6,w10+	eor	v2.16b,v17.16b,v18.16b+	add	w7,w7,w11+	eor	v3.16b,v21.16b,v22.16b+	add	w8,w8,w12+	eor	v4.16b,v25.16b,v26.16b+	eor	w17,w17,w5+	eor	v5.16b,v29.16b,v30.16b+	eor	w19,w19,w6+	ushr	v9.4s,v0.4s,#20+	eor	w20,w20,w7+	ushr	v13.4s,v1.4s,#20+	eor	w21,w21,w8+	ushr	v17.4s,v2.4s,#20+	ror	w17,w17,#24+	ushr	v21.4s,v3.4s,#20+	ror	w19,w19,#24+	ushr	v25.4s,v4.4s,#20+	ror	w20,w20,#24+	ushr	v29.4s,v5.4s,#20+	ror	w21,w21,#24+	sli	v9.4s,v0.4s,#12+	add	w13,w13,w17+	sli	v13.4s,v1.4s,#12+	add	w14,w14,w19+	sli	v17.4s,v2.4s,#12+	add	w15,w15,w20+	sli	v21.4s,v3.4s,#12+	add	w16,w16,w21+	sli	v25.4s,v4.4s,#12+	eor	w9,w9,w13+	sli	v29.4s,v5.4s,#12+	eor	w10,w10,w14+	add	v8.4s,v8.4s,v9.4s+	eor	w11,w11,w15+	add	v12.4s,v12.4s,v13.4s+	eor	w12,w12,w16+	add	v16.4s,v16.4s,v17.4s+	ror	w9,w9,#25+	add	v20.4s,v20.4s,v21.4s+	ror	w10,w10,#25+	add	v24.4s,v24.4s,v25.4s+	ror	w11,w11,#25+	add	v28.4s,v28.4s,v29.4s+	ror	w12,w12,#25+	eor	v11.16b,v11.16b,v8.16b+	add	w5,w5,w10+	eor	v15.16b,v15.16b,v12.16b+	add	w6,w6,w11+	eor	v19.16b,v19.16b,v16.16b+	add	w7,w7,w12+	eor	v23.16b,v23.16b,v20.16b+	add	w8,w8,w9+	eor	v27.16b,v27.16b,v24.16b+	eor	w21,w21,w5+	eor	v31.16b,v31.16b,v28.16b+	eor	w17,w17,w6+	tbl	v11.16b,{v11.16b},v6.16b+	eor	w19,w19,w7+	tbl	v15.16b,{v15.16b},v6.16b+	eor	w20,w20,w8+	tbl	v19.16b,{v19.16b},v6.16b+	ror	w21,w21,#16+	tbl	v23.16b,{v23.16b},v6.16b+	ror	w17,w17,#16+	tbl	v27.16b,{v27.16b},v6.16b+	ror	w19,w19,#16+	tbl	v31.16b,{v31.16b},v6.16b+	ror	w20,w20,#16+	add	v10.4s,v10.4s,v11.4s+	add	w15,w15,w21+	add	v14.4s,v14.4s,v15.4s+	add	w16,w16,w17+	add	v18.4s,v18.4s,v19.4s+	add	w13,w13,w19+	add	v22.4s,v22.4s,v23.4s+	add	w14,w14,w20+	add	v26.4s,v26.4s,v27.4s+	eor	w10,w10,w15+	add	v30.4s,v30.4s,v31.4s+	eor	w11,w11,w16+	eor	v0.16b,v9.16b,v10.16b+	eor	w12,w12,w13+	eor	v1.16b,v13.16b,v14.16b+	eor	w9,w9,w14+	eor	v2.16b,v17.16b,v18.16b+	ror	w10,w10,#20+	eor	v3.16b,v21.16b,v22.16b+	ror	w11,w11,#20+	eor	v4.16b,v25.16b,v26.16b+	ror	w12,w12,#20+	eor	v5.16b,v29.16b,v30.16b+	ror	w9,w9,#20+	ushr	v9.4s,v0.4s,#25+	add	w5,w5,w10+	ushr	v13.4s,v1.4s,#25+	add	w6,w6,w11+	ushr	v17.4s,v2.4s,#25+	add	w7,w7,w12+	ushr	v21.4s,v3.4s,#25+	add	w8,w8,w9+	ushr	v25.4s,v4.4s,#25+	eor	w21,w21,w5+	ushr	v29.4s,v5.4s,#25+	eor	w17,w17,w6+	sli	v9.4s,v0.4s,#7+	eor	w19,w19,w7+	sli	v13.4s,v1.4s,#7+	eor	w20,w20,w8+	sli	v17.4s,v2.4s,#7+	ror	w21,w21,#24+	sli	v21.4s,v3.4s,#7+	ror	w17,w17,#24+	sli	v25.4s,v4.4s,#7+	ror	w19,w19,#24+	sli	v29.4s,v5.4s,#7+	ror	w20,w20,#24+	ext	v10.16b,v10.16b,v10.16b,#8+	add	w15,w15,w21+	ext	v14.16b,v14.16b,v14.16b,#8+	add	w16,w16,w17+	ext	v18.16b,v18.16b,v18.16b,#8+	add	w13,w13,w19+	ext	v22.16b,v22.16b,v22.16b,#8+	add	w14,w14,w20+	ext	v26.16b,v26.16b,v26.16b,#8+	eor	w10,w10,w15+	ext	v30.16b,v30.16b,v30.16b,#8+	eor	w11,w11,w16+	ext	v11.16b,v11.16b,v11.16b,#12+	eor	w12,w12,w13+	ext	v15.16b,v15.16b,v15.16b,#12+	eor	w9,w9,w14+	ext	v19.16b,v19.16b,v19.16b,#12+	ror	w10,w10,#25+	ext	v23.16b,v23.16b,v23.16b,#12+	ror	w11,w11,#25+	ext	v27.16b,v27.16b,v27.16b,#12+	ror	w12,w12,#25+	ext	v31.16b,v31.16b,v31.16b,#12+	ror	w9,w9,#25+	ext	v9.16b,v9.16b,v9.16b,#4+	ext	v13.16b,v13.16b,v13.16b,#4+	ext	v17.16b,v17.16b,v17.16b,#4+	ext	v21.16b,v21.16b,v21.16b,#4+	ext	v25.16b,v25.16b,v25.16b,#4+	ext	v29.16b,v29.16b,v29.16b,#4+	add	v8.4s,v8.4s,v9.4s+	add	w5,w5,w9+	add	v12.4s,v12.4s,v13.4s+	add	w6,w6,w10+	add	v16.4s,v16.4s,v17.4s+	add	w7,w7,w11+	add	v20.4s,v20.4s,v21.4s+	add	w8,w8,w12+	add	v24.4s,v24.4s,v25.4s+	eor	w17,w17,w5+	add	v28.4s,v28.4s,v29.4s+	eor	w19,w19,w6+	eor	v11.16b,v11.16b,v8.16b+	eor	w20,w20,w7+	eor	v15.16b,v15.16b,v12.16b+	eor	w21,w21,w8+	eor	v19.16b,v19.16b,v16.16b+	ror	w17,w17,#16+	eor	v23.16b,v23.16b,v20.16b+	ror	w19,w19,#16+	eor	v27.16b,v27.16b,v24.16b+	ror	w20,w20,#16+	eor	v31.16b,v31.16b,v28.16b+	ror	w21,w21,#16+	rev32	v11.8h,v11.8h+	add	w13,w13,w17+	rev32	v15.8h,v15.8h+	add	w14,w14,w19+	rev32	v19.8h,v19.8h+	add	w15,w15,w20+	rev32	v23.8h,v23.8h+	add	w16,w16,w21+	rev32	v27.8h,v27.8h+	eor	w9,w9,w13+	rev32	v31.8h,v31.8h+	eor	w10,w10,w14+	add	v10.4s,v10.4s,v11.4s+	eor	w11,w11,w15+	add	v14.4s,v14.4s,v15.4s+	eor	w12,w12,w16+	add	v18.4s,v18.4s,v19.4s+	ror	w9,w9,#20+	add	v22.4s,v22.4s,v23.4s+	ror	w10,w10,#20+	add	v26.4s,v26.4s,v27.4s+	ror	w11,w11,#20+	add	v30.4s,v30.4s,v31.4s+	ror	w12,w12,#20+	eor	v0.16b,v9.16b,v10.16b+	add	w5,w5,w9+	eor	v1.16b,v13.16b,v14.16b+	add	w6,w6,w10+	eor	v2.16b,v17.16b,v18.16b+	add	w7,w7,w11+	eor	v3.16b,v21.16b,v22.16b+	add	w8,w8,w12+	eor	v4.16b,v25.16b,v26.16b+	eor	w17,w17,w5+	eor	v5.16b,v29.16b,v30.16b+	eor	w19,w19,w6+	ushr	v9.4s,v0.4s,#20+	eor	w20,w20,w7+	ushr	v13.4s,v1.4s,#20+	eor	w21,w21,w8+	ushr	v17.4s,v2.4s,#20+	ror	w17,w17,#24+	ushr	v21.4s,v3.4s,#20+	ror	w19,w19,#24+	ushr	v25.4s,v4.4s,#20+	ror	w20,w20,#24+	ushr	v29.4s,v5.4s,#20+	ror	w21,w21,#24+	sli	v9.4s,v0.4s,#12+	add	w13,w13,w17+	sli	v13.4s,v1.4s,#12+	add	w14,w14,w19+	sli	v17.4s,v2.4s,#12+	add	w15,w15,w20+	sli	v21.4s,v3.4s,#12+	add	w16,w16,w21+	sli	v25.4s,v4.4s,#12+	eor	w9,w9,w13+	sli	v29.4s,v5.4s,#12+	eor	w10,w10,w14+	add	v8.4s,v8.4s,v9.4s+	eor	w11,w11,w15+	add	v12.4s,v12.4s,v13.4s+	eor	w12,w12,w16+	add	v16.4s,v16.4s,v17.4s+	ror	w9,w9,#25+	add	v20.4s,v20.4s,v21.4s+	ror	w10,w10,#25+	add	v24.4s,v24.4s,v25.4s+	ror	w11,w11,#25+	add	v28.4s,v28.4s,v29.4s+	ror	w12,w12,#25+	eor	v11.16b,v11.16b,v8.16b+	add	w5,w5,w10+	eor	v15.16b,v15.16b,v12.16b+	add	w6,w6,w11+	eor	v19.16b,v19.16b,v16.16b+	add	w7,w7,w12+	eor	v23.16b,v23.16b,v20.16b+	add	w8,w8,w9+	eor	v27.16b,v27.16b,v24.16b+	eor	w21,w21,w5+	eor	v31.16b,v31.16b,v28.16b+	eor	w17,w17,w6+	tbl	v11.16b,{v11.16b},v6.16b+	eor	w19,w19,w7+	tbl	v15.16b,{v15.16b},v6.16b+	eor	w20,w20,w8+	tbl	v19.16b,{v19.16b},v6.16b+	ror	w21,w21,#16+	tbl	v23.16b,{v23.16b},v6.16b+	ror	w17,w17,#16+	tbl	v27.16b,{v27.16b},v6.16b+	ror	w19,w19,#16+	tbl	v31.16b,{v31.16b},v6.16b+	ror	w20,w20,#16+	add	v10.4s,v10.4s,v11.4s+	add	w15,w15,w21+	add	v14.4s,v14.4s,v15.4s+	add	w16,w16,w17+	add	v18.4s,v18.4s,v19.4s+	add	w13,w13,w19+	add	v22.4s,v22.4s,v23.4s+	add	w14,w14,w20+	add	v26.4s,v26.4s,v27.4s+	eor	w10,w10,w15+	add	v30.4s,v30.4s,v31.4s+	eor	w11,w11,w16+	eor	v0.16b,v9.16b,v10.16b+	eor	w12,w12,w13+	eor	v1.16b,v13.16b,v14.16b+	eor	w9,w9,w14+	eor	v2.16b,v17.16b,v18.16b+	ror	w10,w10,#20+	eor	v3.16b,v21.16b,v22.16b+	ror	w11,w11,#20+	eor	v4.16b,v25.16b,v26.16b+	ror	w12,w12,#20+	eor	v5.16b,v29.16b,v30.16b+	ror	w9,w9,#20+	ushr	v9.4s,v0.4s,#25+	add	w5,w5,w10+	ushr	v13.4s,v1.4s,#25+	add	w6,w6,w11+	ushr	v17.4s,v2.4s,#25+	add	w7,w7,w12+	ushr	v21.4s,v3.4s,#25+	add	w8,w8,w9+	ushr	v25.4s,v4.4s,#25+	eor	w21,w21,w5+	ushr	v29.4s,v5.4s,#25+	eor	w17,w17,w6+	sli	v9.4s,v0.4s,#7+	eor	w19,w19,w7+	sli	v13.4s,v1.4s,#7+	eor	w20,w20,w8+	sli	v17.4s,v2.4s,#7+	ror	w21,w21,#24+	sli	v21.4s,v3.4s,#7+	ror	w17,w17,#24+	sli	v25.4s,v4.4s,#7+	ror	w19,w19,#24+	sli	v29.4s,v5.4s,#7+	ror	w20,w20,#24+	ext	v10.16b,v10.16b,v10.16b,#8+	add	w15,w15,w21+	ext	v14.16b,v14.16b,v14.16b,#8+	add	w16,w16,w17+	ext	v18.16b,v18.16b,v18.16b,#8+	add	w13,w13,w19+	ext	v22.16b,v22.16b,v22.16b,#8+	add	w14,w14,w20+	ext	v26.16b,v26.16b,v26.16b,#8+	eor	w10,w10,w15+	ext	v30.16b,v30.16b,v30.16b,#8+	eor	w11,w11,w16+	ext	v11.16b,v11.16b,v11.16b,#4+	eor	w12,w12,w13+	ext	v15.16b,v15.16b,v15.16b,#4+	eor	w9,w9,w14+	ext	v19.16b,v19.16b,v19.16b,#4+	ror	w10,w10,#25+	ext	v23.16b,v23.16b,v23.16b,#4+	ror	w11,w11,#25+	ext	v27.16b,v27.16b,v27.16b,#4+	ror	w12,w12,#25+	ext	v31.16b,v31.16b,v31.16b,#4+	ror	w9,w9,#25+	ext	v9.16b,v9.16b,v9.16b,#12+	ext	v13.16b,v13.16b,v13.16b,#12+	ext	v17.16b,v17.16b,v17.16b,#12+	ext	v21.16b,v21.16b,v21.16b,#12+	ext	v25.16b,v25.16b,v25.16b,#12+	ext	v29.16b,v29.16b,v29.16b,#12+	cbnz	x4,.Loop_lower_neon++	add	w5,w5,w22		// accumulate key block+	ldp	q0,q1,[sp,#0]+	add	x6,x6,x22,lsr#32+	ldp	q2,q3,[sp,#32]+	add	w7,w7,w23+	ldp	q4,q5,[sp,#64]+	add	x8,x8,x23,lsr#32+	ldr	q6,[sp,#96]+	add	v8.4s,v8.4s,v0.4s+	add	w9,w9,w24+	add	v12.4s,v12.4s,v0.4s+	add	x10,x10,x24,lsr#32+	add	v16.4s,v16.4s,v0.4s+	add	w11,w11,w25+	add	v20.4s,v20.4s,v0.4s+	add	x12,x12,x25,lsr#32+	add	v24.4s,v24.4s,v0.4s+	add	w13,w13,w26+	add	v28.4s,v28.4s,v0.4s+	add	x14,x14,x26,lsr#32+	add	v10.4s,v10.4s,v2.4s+	add	w15,w15,w27+	add	v14.4s,v14.4s,v2.4s+	add	x16,x16,x27,lsr#32+	add	v18.4s,v18.4s,v2.4s+	add	w17,w17,w28+	add	v22.4s,v22.4s,v2.4s+	add	x19,x19,x28,lsr#32+	add	v26.4s,v26.4s,v2.4s+	add	w20,w20,w30+	add	v30.4s,v30.4s,v2.4s+	add	x21,x21,x30,lsr#32+	add	v27.4s,v27.4s,v7.4s			// +4+	add	x5,x5,x6,lsl#32	// pack+	add	v31.4s,v31.4s,v7.4s			// +4+	add	x7,x7,x8,lsl#32+	add	v11.4s,v11.4s,v3.4s+	ldp	x6,x8,[x1,#0]		// load input+	add	v15.4s,v15.4s,v4.4s+	add	x9,x9,x10,lsl#32+	add	v19.4s,v19.4s,v5.4s+	add	x11,x11,x12,lsl#32+	add	v23.4s,v23.4s,v6.4s+	ldp	x10,x12,[x1,#16]+	add	v27.4s,v27.4s,v3.4s+	add	x13,x13,x14,lsl#32+	add	v31.4s,v31.4s,v4.4s+	add	x15,x15,x16,lsl#32+	add	v9.4s,v9.4s,v1.4s+	ldp	x14,x16,[x1,#32]+	add	v13.4s,v13.4s,v1.4s+	add	x17,x17,x19,lsl#32+	add	v17.4s,v17.4s,v1.4s+	add	x20,x20,x21,lsl#32+	add	v21.4s,v21.4s,v1.4s+	ldp	x19,x21,[x1,#48]+	add	v25.4s,v25.4s,v1.4s+	add	x1,x1,#64+	add	v29.4s,v29.4s,v1.4s++#ifdef	__AARCH64EB__+	rev	x5,x5+	rev	x7,x7+	rev	x9,x9+	rev	x11,x11+	rev	x13,x13+	rev	x15,x15+	rev	x17,x17+	rev	x20,x20+#endif+	ld1	{v0.16b,v1.16b,v2.16b,v3.16b},[x1],#64+	eor	x5,x5,x6+	eor	x7,x7,x8+	eor	x9,x9,x10+	eor	x11,x11,x12+	eor	x13,x13,x14+	eor	v8.16b,v8.16b,v0.16b+	eor	x15,x15,x16+	eor	v9.16b,v9.16b,v1.16b+	eor	x17,x17,x19+	eor	v10.16b,v10.16b,v2.16b+	eor	x20,x20,x21+	eor	v11.16b,v11.16b,v3.16b+	ld1	{v0.16b,v1.16b,v2.16b,v3.16b},[x1],#64++	stp	x5,x7,[x0,#0]		// store output+	add	x28,x28,#7			// increment counter+	stp	x9,x11,[x0,#16]+	stp	x13,x15,[x0,#32]+	stp	x17,x20,[x0,#48]+	add	x0,x0,#64+	st1	{v8.16b,v9.16b,v10.16b,v11.16b},[x0],#64++	ld1	{v8.16b,v9.16b,v10.16b,v11.16b},[x1],#64+	eor	v12.16b,v12.16b,v0.16b+	eor	v13.16b,v13.16b,v1.16b+	eor	v14.16b,v14.16b,v2.16b+	eor	v15.16b,v15.16b,v3.16b+	st1	{v12.16b,v13.16b,v14.16b,v15.16b},[x0],#64++	ld1	{v12.16b,v13.16b,v14.16b,v15.16b},[x1],#64+	eor	v16.16b,v16.16b,v8.16b+	ldp	q0,q1,[sp,#0]+	eor	v17.16b,v17.16b,v9.16b+	ldp	q2,q3,[sp,#32]+	eor	v18.16b,v18.16b,v10.16b+	eor	v19.16b,v19.16b,v11.16b+	st1	{v16.16b,v17.16b,v18.16b,v19.16b},[x0],#64++	ld1	{v16.16b,v17.16b,v18.16b,v19.16b},[x1],#64+	eor	v20.16b,v20.16b,v12.16b+	eor	v21.16b,v21.16b,v13.16b+	eor	v22.16b,v22.16b,v14.16b+	eor	v23.16b,v23.16b,v15.16b+	st1	{v20.16b,v21.16b,v22.16b,v23.16b},[x0],#64++	ld1	{v20.16b,v21.16b,v22.16b,v23.16b},[x1],#64+	eor	v24.16b,v24.16b,v16.16b+	eor	v25.16b,v25.16b,v17.16b+	eor	v26.16b,v26.16b,v18.16b+	eor	v27.16b,v27.16b,v19.16b+	st1	{v24.16b,v25.16b,v26.16b,v27.16b},[x0],#64++	shl	v8.4s,v7.4s,#1			// 4 -> 8+	eor	v28.16b,v28.16b,v20.16b+	eor	v29.16b,v29.16b,v21.16b+	eor	v30.16b,v30.16b,v22.16b+	eor	v31.16b,v31.16b,v23.16b+	st1	{v28.16b,v29.16b,v30.16b,v31.16b},[x0],#64++	add	v3.4s,v3.4s,v8.4s			// += 8+	add	v4.4s,v4.4s,v8.4s+	add	v5.4s,v5.4s,v8.4s+	add	v6.4s,v6.4s,v8.4s++	b.hs	.Loop_outer_512_neon++	adds	x2,x2,#512+	ushr	v7.4s,v7.4s,#1			// 4 -> 2++	ldp	d10,d11,[sp,#128+16]		// meet ABI requirements+	ldp	d12,d13,[sp,#128+32]+	ldp	d14,d15,[sp,#128+48]++	stp	q0,q0,[sp,#16]		// wipe key off-load area+	stp	q0,q0,[sp,#48]		// [with known constant]+	stp	q0,q0,[sp,#80]++	b.eq	.Ldone_512_neon++	// we have <512 bytes tail, harmonize state with other contexts+	sub	x3,x3,#16+	cmp	x2,#192+	add	sp,sp,#128+	sub	v3.4s,v3.4s,v7.4s		// -= 2+	ld1	{v8.4s,v9.4s},[x3]+	b.hs	.Loop_outer_neon++	ldp	d8,d9,[sp,#0]			// meet ABI requirements+	eor	v1.16b,v1.16b,v1.16b		// cleanse key and nonce+	eor	v2.16b,v2.16b,v2.16b+	eor	v3.16b,v3.16b,v3.16b+	eor	v4.16b,v4.16b,v4.16b+	eor	v5.16b,v5.16b,v5.16b+	eor	v6.16b,v6.16b,v6.16b+	b	.Loop_outer++.Ldone_512_neon:+	ldp	d8,d9,[sp,#128+0]		// meet ABI requirements+	eor	v1.16b,v1.16b,v1.16b		// cleanse key and nonce+	eor	v2.16b,v2.16b,v2.16b+	eor	v3.16b,v3.16b,v3.16b+	eor	v4.16b,v4.16b,v4.16b+	eor	v5.16b,v5.16b,v5.16b+	eor	v6.16b,v6.16b,v6.16b++	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#128+64+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#12*__SIZEOF_POINTER__+.inst	0xd50323bf			// autiasp+	ret+.size	crypton_chacha20_asm_512_neon,.-crypton_chacha20_asm_512_neon++.section	.note.GNU-stack,"",%progbits
+ cbits/asm/chacha-armv8.pl view
@@ -0,0 +1,1328 @@+#!/usr/bin/env perl+#+# ====================================================================+# Written by Andy Polyakov, @dot-asm, initially for the OpenSSL+# project.+# ====================================================================+#+# June 2015+#+# ChaCha20 for ARMv8.+#+# April 2019+#+# Replace 3xNEON+1xIALU code path with 4+1. 4+1 is actually fastest+# option on most(*), but not all, processors, yet 6+2 is retained.+# This is because penalties are considered tolerable in comparison to+# improvement on processors where 6+2 helps. Most notably +37% on+# ThunderX2. It's server-oriented processor which will have to serve+# as many requests as possible. While others are mostly clients, when+# performance doesn't have to be absolute top-notch, just fast enough,+# as majority of time is spent "entertaining" relatively slow human.+#+# Performance in cycles per byte out of large buffer.+#+#			IALU/gcc-4.9	4xNEON+1xIALU	6xNEON+2xIALU+#+# Apple A7		5.50/+49%	2.72		1.60+# Apple A14/M1		4.50/+27%	1.84		1.27+# Cortex-A53		8.40/+80%	4.06		4.45(*)+# Cortex-A57		8.06/+43%	4.08		4.40(*)+# Cortex-A76		5.52		2.90		2.40+# Cortex-X2		4.35		2.53		1.62+# Cortex-X925		3.94		1.79		1.30+# Denver		4.50/+82%	2.30		2.70(*)+# X-Gene		9.50/+46%	8.20		8.90(*)+# Mongoose		8.00/+44%	2.74		3.12(*)+# Kryo			8.17/+50%	4.47		4.65(*)+# ThunderX2		7.22/+48%	5.64		4.10+# Snapdragon X		3.90		1.79		1.25+#+# (*)	slower than 4+1:-(++$flavour=shift;+$output=shift;++if ($flavour && $flavour ne "void") {+    $0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1;+    ( $xlate="${dir}arm-xlate.pl" and -f $xlate ) or+    ( $xlate="${dir}../../perlasm/arm-xlate.pl" and -f $xlate) or+    die "can't locate arm-xlate.pl";++    open STDOUT,"| \"$^X\" $xlate $flavour $output";+} else {+    open STDOUT,">$output";+}++sub AUTOLOAD()		# thunk [simplified] x86-style perlasm+{ my $opcode = $AUTOLOAD; $opcode =~ s/.*:://; $opcode =~ s/_/\./;+  my $arg = pop;+    $arg = "#$arg" if ($arg*1 eq $arg);+    $code .= "\t$opcode\t".join(',',@_,$arg)."\n";+}++my ($out,$inp,$len,$key,$ctr) = map("x$_",(0..4));++my @x=map("x$_",(5..17,19..21));+my @d=map("x$_",(22..28,30));++sub ROUND {+my ($a0,$b0,$c0,$d0)=@_;+my ($a1,$b1,$c1,$d1)=map(($_&~3)+(($_+1)&3),($a0,$b0,$c0,$d0));+my ($a2,$b2,$c2,$d2)=map(($_&~3)+(($_+1)&3),($a1,$b1,$c1,$d1));+my ($a3,$b3,$c3,$d3)=map(($_&~3)+(($_+1)&3),($a2,$b2,$c2,$d2));++    (+	"&add_32	(@x[$a0],@x[$a0],@x[$b0])",+	 "&add_32	(@x[$a1],@x[$a1],@x[$b1])",+	  "&add_32	(@x[$a2],@x[$a2],@x[$b2])",+	   "&add_32	(@x[$a3],@x[$a3],@x[$b3])",+	"&eor_32	(@x[$d0],@x[$d0],@x[$a0])",+	 "&eor_32	(@x[$d1],@x[$d1],@x[$a1])",+	  "&eor_32	(@x[$d2],@x[$d2],@x[$a2])",+	   "&eor_32	(@x[$d3],@x[$d3],@x[$a3])",+	"&ror_32	(@x[$d0],@x[$d0],16)",+	 "&ror_32	(@x[$d1],@x[$d1],16)",+	  "&ror_32	(@x[$d2],@x[$d2],16)",+	   "&ror_32	(@x[$d3],@x[$d3],16)",++	"&add_32	(@x[$c0],@x[$c0],@x[$d0])",+	 "&add_32	(@x[$c1],@x[$c1],@x[$d1])",+	  "&add_32	(@x[$c2],@x[$c2],@x[$d2])",+	   "&add_32	(@x[$c3],@x[$c3],@x[$d3])",+	"&eor_32	(@x[$b0],@x[$b0],@x[$c0])",+	 "&eor_32	(@x[$b1],@x[$b1],@x[$c1])",+	  "&eor_32	(@x[$b2],@x[$b2],@x[$c2])",+	   "&eor_32	(@x[$b3],@x[$b3],@x[$c3])",+	"&ror_32	(@x[$b0],@x[$b0],20)",+	 "&ror_32	(@x[$b1],@x[$b1],20)",+	  "&ror_32	(@x[$b2],@x[$b2],20)",+	   "&ror_32	(@x[$b3],@x[$b3],20)",++	"&add_32	(@x[$a0],@x[$a0],@x[$b0])",+	 "&add_32	(@x[$a1],@x[$a1],@x[$b1])",+	  "&add_32	(@x[$a2],@x[$a2],@x[$b2])",+	   "&add_32	(@x[$a3],@x[$a3],@x[$b3])",+	"&eor_32	(@x[$d0],@x[$d0],@x[$a0])",+	 "&eor_32	(@x[$d1],@x[$d1],@x[$a1])",+	  "&eor_32	(@x[$d2],@x[$d2],@x[$a2])",+	   "&eor_32	(@x[$d3],@x[$d3],@x[$a3])",+	"&ror_32	(@x[$d0],@x[$d0],24)",+	 "&ror_32	(@x[$d1],@x[$d1],24)",+	  "&ror_32	(@x[$d2],@x[$d2],24)",+	   "&ror_32	(@x[$d3],@x[$d3],24)",++	"&add_32	(@x[$c0],@x[$c0],@x[$d0])",+	 "&add_32	(@x[$c1],@x[$c1],@x[$d1])",+	  "&add_32	(@x[$c2],@x[$c2],@x[$d2])",+	   "&add_32	(@x[$c3],@x[$c3],@x[$d3])",+	"&eor_32	(@x[$b0],@x[$b0],@x[$c0])",+	 "&eor_32	(@x[$b1],@x[$b1],@x[$c1])",+	  "&eor_32	(@x[$b2],@x[$b2],@x[$c2])",+	   "&eor_32	(@x[$b3],@x[$b3],@x[$c3])",+	"&ror_32	(@x[$b0],@x[$b0],25)",+	 "&ror_32	(@x[$b1],@x[$b1],25)",+	  "&ror_32	(@x[$b2],@x[$b2],25)",+	   "&ror_32	(@x[$b3],@x[$b3],25)"+    );+}++$code.=<<___;+#ifndef	__KERNEL__+# include "arm_arch.h"+.extern	OPENSSL_armcap_P+#endif++.text++.align	5+.Lsigma:+.quad	0x3320646e61707865,0x6b20657479622d32		// endian-neutral+.Lone:+.long	1,2,3,4+.Lrot24:+.long	0x02010003,0x06050407,0x0a09080b,0x0e0d0c0f+.asciz	"ChaCha20 for ARMv8, CRYPTOGAMS by \@dot-asm"++.globl	ChaCha20_ctr32+.type	ChaCha20_ctr32,%function+.align	5+ChaCha20_ctr32:+	cbz	$len,.Labort+	cmp	$len,#192+	b.lo	.Lshort++#ifndef	__KERNEL__+	adrp	c17,OPENSSL_armcap_P+	ldr	w17,[c17,#:lo12:OPENSSL_armcap_P]+	tst	w17,#ARMV7_NEON+	b.ne	.LChaCha20_neon+#endif++.Lshort:+	.inst	0xd503233f			// paciasp+	stp	c29,c30,[sp,#-12*__SIZEOF_POINTER__]!+	add	c29,csp,#0++	adr	@x[0],.Lsigma+	stp	c19,c20,[csp,#2*__SIZEOF_POINTER__]+	stp	c21,c22,[csp,#4*__SIZEOF_POINTER__]+	stp	c23,c24,[csp,#6*__SIZEOF_POINTER__]+	stp	c25,c26,[csp,#8*__SIZEOF_POINTER__]+	stp	c27,c28,[csp,#10*__SIZEOF_POINTER__]+	sub	csp,csp,#64++	ldp	@d[0],@d[1],[@x[0]]		// load sigma+	ldp	@d[2],@d[3],[$key]		// load key+	ldp	@d[4],@d[5],[$key,#16]+	ldp	@d[6],@d[7],[$ctr]		// load counter+#ifdef	__AARCH64EB__+	ror	@d[2],@d[2],#32+	ror	@d[3],@d[3],#32+	ror	@d[4],@d[4],#32+	ror	@d[5],@d[5],#32+	ror	@d[6],@d[6],#32+	ror	@d[7],@d[7],#32+#endif++.Loop_outer:+	mov.32	@x[0],@d[0]			// unpack key block+	lsr	@x[1],@d[0],#32+	mov.32	@x[2],@d[1]+	lsr	@x[3],@d[1],#32+	mov.32	@x[4],@d[2]+	lsr	@x[5],@d[2],#32+	mov.32	@x[6],@d[3]+	lsr	@x[7],@d[3],#32+	mov.32	@x[8],@d[4]+	lsr	@x[9],@d[4],#32+	mov.32	@x[10],@d[5]+	lsr	@x[11],@d[5],#32+	mov.32	@x[12],@d[6]+	lsr	@x[13],@d[6],#32+	mov.32	@x[14],@d[7]+	lsr	@x[15],@d[7],#32++	mov	$ctr,#10+	subs	$len,$len,#64+.Loop:+	sub	$ctr,$ctr,#1+___+	foreach (&ROUND(0, 4, 8,12)) { eval; }+	foreach (&ROUND(0, 5,10,15)) { eval; }+$code.=<<___;+	cbnz	$ctr,.Loop++	add.32	@x[0],@x[0],@d[0]		// accumulate key block+	add	@x[1],@x[1],@d[0],lsr#32+	add.32	@x[2],@x[2],@d[1]+	add	@x[3],@x[3],@d[1],lsr#32+	add.32	@x[4],@x[4],@d[2]+	add	@x[5],@x[5],@d[2],lsr#32+	add.32	@x[6],@x[6],@d[3]+	add	@x[7],@x[7],@d[3],lsr#32+	add.32	@x[8],@x[8],@d[4]+	add	@x[9],@x[9],@d[4],lsr#32+	add.32	@x[10],@x[10],@d[5]+	add	@x[11],@x[11],@d[5],lsr#32+	add.32	@x[12],@x[12],@d[6]+	add	@x[13],@x[13],@d[6],lsr#32+	add.32	@x[14],@x[14],@d[7]+	add	@x[15],@x[15],@d[7],lsr#32++	b.lo	.Ltail++	add	@x[0],@x[0],@x[1],lsl#32	// pack+	add	@x[2],@x[2],@x[3],lsl#32+	ldp	@x[1],@x[3],[$inp,#0]		// load input+	add	@x[4],@x[4],@x[5],lsl#32+	add	@x[6],@x[6],@x[7],lsl#32+	ldp	@x[5],@x[7],[$inp,#16]+	add	@x[8],@x[8],@x[9],lsl#32+	add	@x[10],@x[10],@x[11],lsl#32+	ldp	@x[9],@x[11],[$inp,#32]+	add	@x[12],@x[12],@x[13],lsl#32+	add	@x[14],@x[14],@x[15],lsl#32+	ldp	@x[13],@x[15],[$inp,#48]+	cadd	$inp,$inp,#64+#ifdef	__AARCH64EB__+	rev	@x[0],@x[0]+	rev	@x[2],@x[2]+	rev	@x[4],@x[4]+	rev	@x[6],@x[6]+	rev	@x[8],@x[8]+	rev	@x[10],@x[10]+	rev	@x[12],@x[12]+	rev	@x[14],@x[14]+#endif+	eor	@x[0],@x[0],@x[1]+	eor	@x[2],@x[2],@x[3]+	eor	@x[4],@x[4],@x[5]+	eor	@x[6],@x[6],@x[7]+	eor	@x[8],@x[8],@x[9]+	eor	@x[10],@x[10],@x[11]+	eor	@x[12],@x[12],@x[13]+	eor	@x[14],@x[14],@x[15]++	stp	@x[0],@x[2],[$out,#0]		// store output+	 add	@d[6],@d[6],#1			// increment counter+	stp	@x[4],@x[6],[$out,#16]+	stp	@x[8],@x[10],[$out,#32]+	stp	@x[12],@x[14],[$out,#48]+	cadd	$out,$out,#64++	b.hi	.Loop_outer++	ldp	c19,c20,[c29,#2*__SIZEOF_POINTER__]+	add	csp,csp,#64+	ldp	c21,c22,[c29,#4*__SIZEOF_POINTER__]+	ldp	c23,c24,[c29,#6*__SIZEOF_POINTER__]+	ldp	c25,c26,[c29,#8*__SIZEOF_POINTER__]+	ldp	c27,c28,[c29,#10*__SIZEOF_POINTER__]+	ldp	c29,c30,[csp],#12*__SIZEOF_POINTER__+	.inst	0xd50323bf			// autiasp+.Labort:+	ret++.align	4+.Ltail:+	add	$len,$len,#64+.Less_than_64:+	csub	$out,$out,#1+	cadd	$inp,$inp,$len+	cadd	$out,$out,$len+	cadd	$ctr,sp,$len+	neg	$len,$len++	add	@x[0],@x[0],@x[1],lsl#32	// pack+	add	@x[2],@x[2],@x[3],lsl#32+	add	@x[4],@x[4],@x[5],lsl#32+	add	@x[6],@x[6],@x[7],lsl#32+	add	@x[8],@x[8],@x[9],lsl#32+	add	@x[10],@x[10],@x[11],lsl#32+	add	@x[12],@x[12],@x[13],lsl#32+	add	@x[14],@x[14],@x[15],lsl#32+#ifdef	__AARCH64EB__+	rev	@x[0],@x[0]+	rev	@x[2],@x[2]+	rev	@x[4],@x[4]+	rev	@x[6],@x[6]+	rev	@x[8],@x[8]+	rev	@x[10],@x[10]+	rev	@x[12],@x[12]+	rev	@x[14],@x[14]+#endif+	stp	@x[0],@x[2],[sp,#0]		// off-load complete block+	stp	@x[4],@x[6],[sp,#16]+	stp	@x[8],@x[10],[sp,#32]+	stp	@x[12],@x[14],[sp,#48]++.Loop_tail:+	ldrb	w10,[$inp,$len]+	ldrb	w11,[$ctr,$len]+	add	$len,$len,#1+	eor	w10,w10,w11+	strb	w10,[$out,$len]+	cbnz	$len,.Loop_tail++	stp	xzr,xzr,[sp,#0]			// wipe off-load area+	stp	xzr,xzr,[sp,#16]+	stp	xzr,xzr,[sp,#32]+	stp	xzr,xzr,[sp,#48]++	ldp	c19,c20,[c29,#2*__SIZEOF_POINTER__]+	add	csp,csp,#64+	ldp	c21,c22,[c29,#4*__SIZEOF_POINTER__]+	ldp	c23,c24,[c29,#6*__SIZEOF_POINTER__]+	ldp	c25,c26,[c29,#8*__SIZEOF_POINTER__]+	ldp	c27,c28,[c29,#10*__SIZEOF_POINTER__]+	ldp	c29,c30,[csp],#12*__SIZEOF_POINTER__+	.inst	0xd50323bf			// autiasp+	ret+.size	ChaCha20_ctr32,.-ChaCha20_ctr32+___++{{{+########################################################################+# 4x"vertical" layout reduces *total* amount of instructions by trading+# 60 "horizontal" permutations in inner loop for 32-instruction diagonal+# transposition at the loop exit. And since NEON instruction issue rate+# is customarily limited, it's possible to process one additional block+# with scalar instructions at no additional cost. Hence the "4+1"+# description...++my @K = map("v$_.4s",(0..3));+my ($xt0,$xt1,$xt2,$xt3, $CTR,$ROT24) = map("v$_.4s",(4..9));+my @X = map("v$_.4s",(16,20,24,28, 17,21,25,29, 18,22,26,30, 19,23,27,31));+my ($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3,+    $xc0,$xc1,$xc2,$xc3, $xd0,$xd1,$xd2,$xd3) = @X;++sub NEON_lane_ROUND {+my ($a0,$b0,$c0,$d0)=@_;+my ($a1,$b1,$c1,$d1)=map(($_&~3)+(($_+1)&3),($a0,$b0,$c0,$d0));+my ($a2,$b2,$c2,$d2)=map(($_&~3)+(($_+1)&3),($a1,$b1,$c1,$d1));+my ($a3,$b3,$c3,$d3)=map(($_&~3)+(($_+1)&3),($a2,$b2,$c2,$d2));+my @x=map("'$_'",@X);++	(+	"&add		(@x[$a0],@x[$a0],@x[$b0])",	# Q1+	 "&add		(@x[$a1],@x[$a1],@x[$b1])",	# Q2+	  "&add		(@x[$a2],@x[$a2],@x[$b2])",	# Q3+	   "&add	(@x[$a3],@x[$a3],@x[$b3])",	# Q4+	"&eor		(@x[$d0],@x[$d0],@x[$a0])",+	 "&eor		(@x[$d1],@x[$d1],@x[$a1])",+	  "&eor		(@x[$d2],@x[$d2],@x[$a2])",+	   "&eor	(@x[$d3],@x[$d3],@x[$a3])",+	"&rev32_16	(@x[$d0],@x[$d0])",+	 "&rev32_16	(@x[$d1],@x[$d1])",+	  "&rev32_16	(@x[$d2],@x[$d2])",+	   "&rev32_16	(@x[$d3],@x[$d3])",++	"&add		(@x[$c0],@x[$c0],@x[$d0])",+	 "&add		(@x[$c1],@x[$c1],@x[$d1])",+	  "&add		(@x[$c2],@x[$c2],@x[$d2])",+	   "&add	(@x[$c3],@x[$c3],@x[$d3])",+	"&eor		('$xt0',@x[$b0],@x[$c0])",+	 "&eor		('$xt1',@x[$b1],@x[$c1])",+	  "&eor		('$xt2',@x[$b2],@x[$c2])",+	   "&eor	('$xt3',@x[$b3],@x[$c3])",+	"&ushr		(@x[$b0],'$xt0',20)",+	 "&ushr		(@x[$b1],'$xt1',20)",+	  "&ushr	(@x[$b2],'$xt2',20)",+	   "&ushr	(@x[$b3],'$xt3',20)",+	"&sli		(@x[$b0],'$xt0',12)",+	 "&sli		(@x[$b1],'$xt1',12)",+	  "&sli		(@x[$b2],'$xt2',12)",+	   "&sli	(@x[$b3],'$xt3',12)",++	"&add		(@x[$a0],@x[$a0],@x[$b0])",+	 "&add		(@x[$a1],@x[$a1],@x[$b1])",+	  "&add		(@x[$a2],@x[$a2],@x[$b2])",+	   "&add	(@x[$a3],@x[$a3],@x[$b3])",+	"&eor		('$xt0',@x[$d0],@x[$a0])",+	 "&eor		('$xt1',@x[$d1],@x[$a1])",+	  "&eor		('$xt2',@x[$d2],@x[$a2])",+	   "&eor	('$xt3',@x[$d3],@x[$a3])",+	"&tbl		(@x[$d0],'{$xt0}','$ROT24')",+	 "&tbl		(@x[$d1],'{$xt1}','$ROT24')",+	  "&tbl		(@x[$d2],'{$xt2}','$ROT24')",+	   "&tbl	(@x[$d3],'{$xt3}','$ROT24')",++	"&add		(@x[$c0],@x[$c0],@x[$d0])",+	 "&add		(@x[$c1],@x[$c1],@x[$d1])",+	  "&add		(@x[$c2],@x[$c2],@x[$d2])",+	   "&add	(@x[$c3],@x[$c3],@x[$d3])",+	"&eor		('$xt0',@x[$b0],@x[$c0])",+	 "&eor		('$xt1',@x[$b1],@x[$c1])",+	  "&eor		('$xt2',@x[$b2],@x[$c2])",+	   "&eor	('$xt3',@x[$b3],@x[$c3])",+	"&ushr		(@x[$b0],'$xt0',25)",+	 "&ushr		(@x[$b1],'$xt1',25)",+	  "&ushr	(@x[$b2],'$xt2',25)",+	   "&ushr	(@x[$b3],'$xt3',25)",+	"&sli		(@x[$b0],'$xt0',7)",+	 "&sli		(@x[$b1],'$xt1',7)",+	  "&sli		(@x[$b2],'$xt2',7)",+	   "&sli	(@x[$b3],'$xt3',7)"+	);+}++$code.=<<___;++#ifdef	__KERNEL__+.globl	ChaCha20_neon+#endif+.type	ChaCha20_neon,%function+.align	5+ChaCha20_neon:+.LChaCha20_neon:+	.inst	0xd503233f			// paciasp+	stp	c29,c30,[csp,#-12*__SIZEOF_POINTER__]!+	add	c29,csp,#0++	adr	@x[0],.Lsigma+	stp	c19,c20,[csp,#2*__SIZEOF_POINTER__]+	stp	c21,c22,[csp,#4*__SIZEOF_POINTER__]+	stp	c23,c24,[csp,#6*__SIZEOF_POINTER__]+	stp	c25,c26,[csp,#8*__SIZEOF_POINTER__]+	stp	c27,c28,[csp,#10*__SIZEOF_POINTER__]+	cmp	$len,#512+	b.hs	.L512_or_more_neon++	sub	csp,csp,#64++	ldp	@d[0],@d[1],[@x[0]]		// load sigma+	ld1	{@K[0]},[@x[0]],#16+	ldp	@d[2],@d[3],[$key]		// load key+	ldp	@d[4],@d[5],[$key,#16]+	ld1	{@K[1],@K[2]},[$key]+	ldp	@d[6],@d[7],[$ctr]		// load counter+	ld1	{@K[3]},[$ctr]+	stp	d8,d9,[sp]			// meet ABI requirements+	ld1	{$CTR,$ROT24},[@x[0]]+#ifdef	__AARCH64EB__+	rev64	@K[0],@K[0]+	ror	@d[2],@d[2],#32+	ror	@d[3],@d[3],#32+	ror	@d[4],@d[4],#32+	ror	@d[5],@d[5],#32+	ror	@d[6],@d[6],#32+	ror	@d[7],@d[7],#32+#endif++.Loop_outer_neon:+	dup	$xa0,@{K[0]}[0]			// unpack key block+	 mov.32	@x[0],@d[0]+	dup	$xa1,@{K[0]}[1]+	 lsr	@x[1],@d[0],#32+	dup	$xa2,@{K[0]}[2]+	 mov.32	@x[2],@d[1]+	dup	$xa3,@{K[0]}[3]+	 lsr	@x[3],@d[1],#32+	dup	$xb0,@{K[1]}[0]+	 mov.32	@x[4],@d[2]+	dup	$xb1,@{K[1]}[1]+	 lsr	@x[5],@d[2],#32+	dup	$xb2,@{K[1]}[2]+	 mov.32	@x[6],@d[3]+	dup	$xb3,@{K[1]}[3]+	 lsr	@x[7],@d[3],#32+	dup	$xd0,@{K[3]}[0]+	 mov.32	@x[8],@d[4]+	dup	$xd1,@{K[3]}[1]+	 lsr	@x[9],@d[4],#32+	dup	$xd2,@{K[3]}[2]+	 mov.32	@x[10],@d[5]+	dup	$xd3,@{K[3]}[3]+	 lsr	@x[11],@d[5],#32+	add	$xd0,$xd0,$CTR+	 mov.32	@x[12],@d[6]+	dup	$xc0,@{K[2]}[0]+	 lsr	@x[13],@d[6],#32+	dup	$xc1,@{K[2]}[1]+	 mov.32	@x[14],@d[7]+	dup	$xc2,@{K[2]}[2]+	 lsr	@x[15],@d[7],#32+	dup	$xc3,@{K[2]}[3]++	mov	$ctr,#10+	subs	$len,$len,#320+.Loop_neon:+	sub	$ctr,$ctr,#1+___+	my @plus_one=&ROUND(0,4,8,12);	my $i=0;+	foreach (&NEON_lane_ROUND(0,4,8,12))  { eval; eval(shift(@plus_one)) if ($i++ > 6); }+	foreach (@plus_one) { eval; }++	@plus_one=&ROUND(0,5,10,15);	$i=0;+	foreach (&NEON_lane_ROUND(0,5,10,15)) { eval; eval(shift(@plus_one)) if ($i++ > 6); }+	foreach (@plus_one) { eval; }+$code.=<<___;+	cbnz	$ctr,.Loop_neon++	add	$xd0,$xd0,$CTR++	zip1	$xt0,$xa0,$xa1			// transpose data+	zip1	$xt1,$xa2,$xa3+	zip2	$xt2,$xa0,$xa1+	zip2	$xt3,$xa2,$xa3+	zip1.64	$xa0,$xt0,$xt1+	zip2.64	$xa1,$xt0,$xt1+	zip1.64	$xa2,$xt2,$xt3+	zip2.64	$xa3,$xt2,$xt3++	zip1	$xt0,$xb0,$xb1+	zip1	$xt1,$xb2,$xb3+	zip2	$xt2,$xb0,$xb1+	zip2	$xt3,$xb2,$xb3+	zip1.64	$xb0,$xt0,$xt1+	zip2.64	$xb1,$xt0,$xt1+	zip1.64	$xb2,$xt2,$xt3+	zip2.64	$xb3,$xt2,$xt3++	zip1	$xt0,$xc0,$xc1+	 add.32	@x[0],@x[0],@d[0]		// accumulate key block+	zip1	$xt1,$xc2,$xc3+	 add	@x[1],@x[1],@d[0],lsr#32+	zip2	$xt2,$xc0,$xc1+	 add.32	@x[2],@x[2],@d[1]+	zip2	$xt3,$xc2,$xc3+	 add	@x[3],@x[3],@d[1],lsr#32+	zip1.64	$xc0,$xt0,$xt1+	 add.32	@x[4],@x[4],@d[2]+	zip2.64	$xc1,$xt0,$xt1+	 add	@x[5],@x[5],@d[2],lsr#32+	zip1.64	$xc2,$xt2,$xt3+	 add.32	@x[6],@x[6],@d[3]+	zip2.64	$xc3,$xt2,$xt3+	 add	@x[7],@x[7],@d[3],lsr#32++	zip1	$xt0,$xd0,$xd1+	 add.32	@x[8],@x[8],@d[4]+	zip1	$xt1,$xd2,$xd3+	 add	@x[9],@x[9],@d[4],lsr#32+	zip2	$xt2,$xd0,$xd1+	 add.32	@x[10],@x[10],@d[5]+	zip2	$xt3,$xd2,$xd3+	 add	@x[11],@x[11],@d[5],lsr#32+	zip1.64	$xd0,$xt0,$xt1+	 add.32	@x[12],@x[12],@d[6]+	zip2.64	$xd1,$xt0,$xt1+	 add	@x[13],@x[13],@d[6],lsr#32+	zip1.64	$xd2,$xt2,$xt3+	 add.32	@x[14],@x[14],@d[7]+	zip2.64	$xd3,$xt2,$xt3+	 add	@x[15],@x[15],@d[7],lsr#32++	b.lo	.Ltail_neon++	add	@x[0],@x[0],@x[1],lsl#32	// pack+	add	@x[2],@x[2],@x[3],lsl#32+	ldp	@x[1],@x[3],[$inp,#0]		// load input+	 add	$xa0,$xa0,@K[0]			// accumulate key block+	add	@x[4],@x[4],@x[5],lsl#32+	add	@x[6],@x[6],@x[7],lsl#32+	ldp	@x[5],@x[7],[$inp,#16]+	 add	$xb0,$xb0,@K[1]+	add	@x[8],@x[8],@x[9],lsl#32+	add	@x[10],@x[10],@x[11],lsl#32+	ldp	@x[9],@x[11],[$inp,#32]+	 add	$xc0,$xc0,@K[2]+	add	@x[12],@x[12],@x[13],lsl#32+	add	@x[14],@x[14],@x[15],lsl#32+	ldp	@x[13],@x[15],[$inp,#48]+	 add	$xd0,$xd0,@K[3]+	cadd	$inp,$inp,#64+#ifdef	__AARCH64EB__+	rev	@x[0],@x[0]+	rev	@x[2],@x[2]+	rev	@x[4],@x[4]+	rev	@x[6],@x[6]+	rev	@x[8],@x[8]+	rev	@x[10],@x[10]+	rev	@x[12],@x[12]+	rev	@x[14],@x[14]+#endif+	ld1.8	{$xt0-$xt3},[$inp],#64+	eor	@x[0],@x[0],@x[1]+	 add	$xa1,$xa1,@K[0]+	eor	@x[2],@x[2],@x[3]+	 add	$xb1,$xb1,@K[1]+	eor	@x[4],@x[4],@x[5]+	 add	$xc1,$xc1,@K[2]+	eor	@x[6],@x[6],@x[7]+	 add	$xd1,$xd1,@K[3]+	eor	@x[8],@x[8],@x[9]+	 eor	$xa0,$xa0,$xt0+	 movi	$xt0,#5+	eor	@x[10],@x[10],@x[11]+	 eor	$xb0,$xb0,$xt1+	eor	@x[12],@x[12],@x[13]+	 eor	$xc0,$xc0,$xt2+	eor	@x[14],@x[14],@x[15]+	 eor	$xd0,$xd0,$xt3+	 add	$CTR,$CTR,$xt0			// += 5+	 ld1.8	{$xt0-$xt3},[$inp],#64++	stp	@x[0],@x[2],[$out,#0]		// store output+	 add	@d[6],@d[6],#5			// increment counter+	stp	@x[4],@x[6],[$out,#16]+	stp	@x[8],@x[10],[$out,#32]+	stp	@x[12],@x[14],[$out,#48]+	cadd	$out,$out,#64++	st1.8	{$xa0-$xd0},[$out],#64+	 add	$xa2,$xa2,@K[0]+	 add	$xb2,$xb2,@K[1]+	 add	$xc2,$xc2,@K[2]+	 add	$xd2,$xd2,@K[3]+	ld1.8	{$xa0-$xd0},[$inp],#64++	eor	$xa1,$xa1,$xt0+	eor	$xb1,$xb1,$xt1+	eor	$xc1,$xc1,$xt2+	eor	$xd1,$xd1,$xt3+	st1.8	{$xa1-$xd1},[$out],#64+	 add	$xa3,$xa3,@K[0]+	 add	$xb3,$xb3,@K[1]+	 add	$xc3,$xc3,@K[2]+	 add	$xd3,$xd3,@K[3]+	ld1.8	{$xa1-$xd1},[$inp],#64++	eor	$xa2,$xa2,$xa0+	eor	$xb2,$xb2,$xb0+	eor	$xc2,$xc2,$xc0+	eor	$xd2,$xd2,$xd0+	st1.8	{$xa2-$xd2},[$out],#64++	eor	$xa3,$xa3,$xa1+	eor	$xb3,$xb3,$xb1+	eor	$xc3,$xc3,$xc1+	eor	$xd3,$xd3,$xd1+	st1.8	{$xa3-$xd3},[$out],#64++	b.hi	.Loop_outer_neon++	ldp	d8,d9,[sp]			// meet ABI requirements+	eor	@K[1],@K[1],@K[1]		// cleanse key and nonce+	eor	@K[2],@K[2],@K[2]+	eor	@K[3],@K[3],@K[3]++	ldp	c19,c20,[c29,#2*__SIZEOF_POINTER__]+	add	csp,csp,#64+	ldp	c21,c22,[c29,#4*__SIZEOF_POINTER__]+	ldp	c23,c24,[c29,#6*__SIZEOF_POINTER__]+	ldp	c25,c26,[c29,#8*__SIZEOF_POINTER__]+	ldp	c27,c28,[c29,#10*__SIZEOF_POINTER__]+	ldp	c29,c30,[csp],#12*__SIZEOF_POINTER__+	.inst	0xd50323bf			// autiasp+	ret++.align	4+.Ltail_neon:+	add	$len,$len,#320+	ldp	d8,d9,[sp]			// meet ABI requirements+	cmp	$len,#64+	b.lo	.Less_than_64_neon++	add	@x[0],@x[0],@x[1],lsl#32	// pack+	add	@x[2],@x[2],@x[3],lsl#32+	ldp	@x[1],@x[3],[$inp,#0]		// load input+	add	@x[4],@x[4],@x[5],lsl#32+	add	@x[6],@x[6],@x[7],lsl#32+	ldp	@x[5],@x[7],[$inp,#16]+	add	@x[8],@x[8],@x[9],lsl#32+	add	@x[10],@x[10],@x[11],lsl#32+	ldp	@x[9],@x[11],[$inp,#32]+	add	@x[12],@x[12],@x[13],lsl#32+	add	@x[14],@x[14],@x[15],lsl#32+	ldp	@x[13],@x[15],[$inp,#48]+	cadd	$inp,$inp,#64+#ifdef	__AARCH64EB__+	rev	@x[0],@x[0]+	rev	@x[2],@x[2]+	rev	@x[4],@x[4]+	rev	@x[6],@x[6]+	rev	@x[8],@x[8]+	rev	@x[10],@x[10]+	rev	@x[12],@x[12]+	rev	@x[14],@x[14]+#endif+	eor	@x[0],@x[0],@x[1]+	eor	@x[2],@x[2],@x[3]+	eor	@x[4],@x[4],@x[5]+	eor	@x[6],@x[6],@x[7]+	eor	@x[8],@x[8],@x[9]+	eor	@x[10],@x[10],@x[11]+	eor	@x[12],@x[12],@x[13]+	eor	@x[14],@x[14],@x[15]++	stp	@x[0],@x[2],[$out,#0]		// store output+	 add	$xa0,$xa0,@K[0]			// accumulate key block+	stp	@x[4],@x[6],[$out,#16]+	 add	$xb0,$xb0,@K[1]+	stp	@x[8],@x[10],[$out,#32]+	 add	$xc0,$xc0,@K[2]+	stp	@x[12],@x[14],[$out,#48]+	 add	$xd0,$xd0,@K[3]+	cadd	$out,$out,#64+	b.eq	.Ldone_neon+	sub	$len,$len,#64+	cmp	$len,#64+	b.lo	.Last_neon++	ld1.8	{$xt0-$xt3},[$inp],#64+	eor	$xa0,$xa0,$xt0+	eor	$xb0,$xb0,$xt1+	eor	$xc0,$xc0,$xt2+	eor	$xd0,$xd0,$xt3+	st1.8	{$xa0-$xd0},[$out],#64+	b.eq	.Ldone_neon++	add	$xa0,$xa1,@K[0]+	add	$xb0,$xb1,@K[1]+	sub	$len,$len,#64+	add	$xc0,$xc1,@K[2]+	cmp	$len,#64+	add	$xd0,$xd1,@K[3]+	b.lo	.Last_neon++	ld1.8	{$xt0-$xt3},[$inp],#64+	eor	$xa1,$xa0,$xt0+	eor	$xb1,$xb0,$xt1+	eor	$xc1,$xc0,$xt2+	eor	$xd1,$xd0,$xt3+	st1.8	{$xa1-$xd1},[$out],#64+	b.eq	.Ldone_neon++	add	$xa0,$xa2,@K[0]+	add	$xb0,$xb2,@K[1]+	sub	$len,$len,#64+	add	$xc0,$xc2,@K[2]+	cmp	$len,#64+	add	$xd0,$xd2,@K[3]+	b.lo	.Last_neon++	ld1.8	{$xt0-$xt3},[$inp],#64+	eor	$xa2,$xa0,$xt0+	eor	$xb2,$xb0,$xt1+	eor	$xc2,$xc0,$xt2+	eor	$xd2,$xd0,$xt3+	st1.8	{$xa2-$xd2},[$out],#64+	b.eq	.Ldone_neon++	add	$xa0,$xa3,@K[0]+	add	$xb0,$xb3,@K[1]+	add	$xc0,$xc3,@K[2]+	add	$xd0,$xd3,@K[3]+	sub	$len,$len,#64++.Last_neon:+	st1.8	{$xa0-$xd0},[sp]		// off-load complete block++	csub	$out,$out,#1+	cadd	$inp,$inp,$len+	cadd	$out,$out,$len+	cadd	$ctr,sp,$len+	neg	$len,$len++.Loop_tail_neon:+	ldrb	w10,[$inp,$len]+	ldrb	w11,[$ctr,$len]+	add	$len,$len,#1+	eor	w10,w10,w11+	strb	w10,[$out,$len]+	cbnz	$len,.Loop_tail_neon++	stp	@K[0],@K[0],[sp,#0]		// wipe off-load area+	stp	@K[0],@K[0],[sp,#32]		// [with known constant]++.Ldone_neon:+	eor	@K[1],@K[1],@K[1]		// cleanse key and nonce+	eor	@K[2],@K[2],@K[2]+	eor	@K[3],@K[3],@K[3]++	ldp	c19,c20,[c29,#2*__SIZEOF_POINTER__]+	add	csp,csp,#64+	ldp	c21,c22,[c29,#4*__SIZEOF_POINTER__]+	ldp	c23,c24,[c29,#6*__SIZEOF_POINTER__]+	ldp	c25,c26,[c29,#8*__SIZEOF_POINTER__]+	ldp	c27,c28,[c29,#10*__SIZEOF_POINTER__]+	ldp	c29,c30,[csp],#12*__SIZEOF_POINTER__+	.inst	0xd50323bf			// autiasp+	ret++.align	4+.Less_than_64_neon:+	eor	@K[1],@K[1],@K[1]		// cleanse key and nonce+	eor	@K[2],@K[2],@K[2]+	eor	@K[3],@K[3],@K[3]+	b	.Less_than_64+.size	ChaCha20_neon,.-ChaCha20_neon+___+{+########################################################################+# While "vertical" layout minimizes total amount of instructions, number+# of blocks processed in parallel is limited to 4x. And trouble is that+# if NEON instructions are high-latency enough, algorithmic dependencies+# will manifest themselves as idle/wasted cycles. 6x"horizontal" avoids+# these gaps and achieves better performance. Since NEON instruction+# sequence is >2x longer, it's possible to slip in two additional blocks+# processed with scalar code path at no additional cost. Hence the "6+2"+# description...++my @K = map("v$_.4s",(0..6));+my ($T0,$T1,$T2,$T3,$T4,$T5)=@K;+my ($A0,$B0,$C0,$D0,$A1,$B1,$C1,$D1,$A2,$B2,$C2,$D2,+    $A3,$B3,$C3,$D3,$A4,$B4,$C4,$D4,$A5,$B5,$C5,$D5) = map("v$_.4s",(8..31));+my $rot24 = @K[6];+my $ONE = "v7.4s";++sub NEONROUND {+my $odd = pop;+my ($a,$b,$c,$d,$t)=@_;++	(+	"&add		('$a','$a','$b')",+	"&eor		('$d','$d','$a')",+	"&rev32_16	('$d','$d')",		# vrot ($d,16)++	"&add		('$c','$c','$d')",+	"&eor		('$t','$b','$c')",+	"&ushr		('$b','$t',20)",+	"&sli		('$b','$t',12)",++	"&add		('$a','$a','$b')",+	"&eor		('$d','$d','$a')",+	"&tbl		('$d','{$d}','$rot24')",++	"&add		('$c','$c','$d')",+	"&eor		('$t','$b','$c')",+	"&ushr		('$b','$t',25)",+	"&sli		('$b','$t',7)",++	"&ext		('$c','$c','$c',8)",+	"&ext		('$d','$d','$d',$odd?4:12)",+	"&ext		('$b','$b','$b',$odd?12:4)"+	);+}++$code.=<<___;+.type	ChaCha20_512_neon,%function+.align	5+ChaCha20_512_neon:+	.inst	0xd503233f			// paciasp+	stp	c29,c30,[csp,#-12*__SIZEOF_POINTER__]!+	add	c29,csp,#0++	adr	@x[0],.Lsigma+	stp	c19,c20,[csp,#2*__SIZEOF_POINTER__]+	stp	c21,c22,[csp,#4*__SIZEOF_POINTER__]+	stp	c23,c24,[csp,#6*__SIZEOF_POINTER__]+	stp	c25,c26,[csp,#8*__SIZEOF_POINTER__]+	stp	c27,c28,[csp,#10*__SIZEOF_POINTER__]++.L512_or_more_neon:+	sub	csp,csp,#128+64++	eor	$ONE,$ONE,$ONE+	ldp	@d[0],@d[1],[@x[0]]		// load sigma+	ld1	{@K[0]},[@x[0]],#16+	ldp	@d[2],@d[3],[$key]		// load key+	ldp	@d[4],@d[5],[$key,#16]+	ld1	{@K[1],@K[2]},[$key]+	ldp	@d[6],@d[7],[$ctr]		// load counter+	ld1	{@K[3]},[$ctr]+	ld1	{$ONE}[0],[@x[0]]+	cadd	$key,@x[0],#16			// .Lrot24+#ifdef	__AARCH64EB__+	rev64	@K[0],@K[0]+	ror	@d[2],@d[2],#32+	ror	@d[3],@d[3],#32+	ror	@d[4],@d[4],#32+	ror	@d[5],@d[5],#32+	ror	@d[6],@d[6],#32+	ror	@d[7],@d[7],#32+#endif+	add	@K[3],@K[3],$ONE		// += 1+	stp	@K[0],@K[1],[sp,#0]		// off-load key block, invariant part+	add	@K[3],@K[3],$ONE		// not typo+	str	@K[2],[sp,#32]+	add	@K[4],@K[3],$ONE+	add	@K[5],@K[4],$ONE+	add	@K[6],@K[5],$ONE+	shl	$ONE,$ONE,#2			// 1 -> 4++	stp	d8,d9,[sp,#128+0]		// meet ABI requirements+	stp	d10,d11,[sp,#128+16]+	stp	d12,d13,[sp,#128+32]+	stp	d14,d15,[sp,#128+48]++	sub	$len,$len,#512			// not typo++.Loop_outer_512_neon:+	 mov	$A0,@K[0]+	 mov	$A1,@K[0]+	 mov	$A2,@K[0]+	 mov	$A3,@K[0]+	 mov	$A4,@K[0]+	 mov	$A5,@K[0]+	 mov	$B0,@K[1]+	mov.32	@x[0],@d[0]			// unpack key block+	 mov	$B1,@K[1]+	lsr	@x[1],@d[0],#32+	 mov	$B2,@K[1]+	mov.32	@x[2],@d[1]+	 mov	$B3,@K[1]+	lsr	@x[3],@d[1],#32+	 mov	$B4,@K[1]+	mov.32	@x[4],@d[2]+	 mov	$B5,@K[1]+	lsr	@x[5],@d[2],#32+	 mov	$D0,@K[3]+	mov.32	@x[6],@d[3]+	 mov	$D1,@K[4]+	lsr	@x[7],@d[3],#32+	 mov	$D2,@K[5]+	mov.32	@x[8],@d[4]+	 mov	$D3,@K[6]+	lsr	@x[9],@d[4],#32+	 mov	$C0,@K[2]+	mov.32	@x[10],@d[5]+	 mov	$C1,@K[2]+	lsr	@x[11],@d[5],#32+	 add	$D4,$D0,$ONE			// +4+	mov.32	@x[12],@d[6]+	 add	$D5,$D1,$ONE			// +4+	lsr	@x[13],@d[6],#32+	 mov	$C2,@K[2]+	mov.32	@x[14],@d[7]+	 mov	$C3,@K[2]+	lsr	@x[15],@d[7],#32+	 mov	$C4,@K[2]+	 stp	@K[3],@K[4],[sp,#48]		// off-load key block, variable part+	 mov	$C5,@K[2]+	 stp	@K[5],@K[6],[sp,#80]++	mov	$ctr,#5+	ld1	{$rot24},[$key]+	subs	$len,$len,#512+.Loop_upper_neon:+	sub	$ctr,$ctr,#1+___+	my @thread0=&NEONROUND($A0,$B0,$C0,$D0,$T0,0);+	my @thread1=&NEONROUND($A1,$B1,$C1,$D1,$T1,0);+	my @thread2=&NEONROUND($A2,$B2,$C2,$D2,$T2,0);+	my @thread3=&NEONROUND($A3,$B3,$C3,$D3,$T3,0);+	my @thread4=&NEONROUND($A4,$B4,$C4,$D4,$T4,0);+	my @thread5=&NEONROUND($A5,$B5,$C5,$D5,$T5,0);+	my @thread67=(&ROUND(0,4,8,12),&ROUND(0,5,10,15));+	my $diff = ($#thread0+1)*6 - $#thread67 - 1;+	my $i = 0;++	foreach (@thread0) {+		eval;			eval(shift(@thread67));+		eval(shift(@thread1));	eval(shift(@thread67));+		eval(shift(@thread2));	eval(shift(@thread67));+		eval(shift(@thread3));	eval(shift(@thread67));+		eval(shift(@thread4));	eval(shift(@thread67));+		eval(shift(@thread5));	eval(shift(@thread67));+	}++	@thread0=&NEONROUND($A0,$B0,$C0,$D0,$T0,1);+	@thread1=&NEONROUND($A1,$B1,$C1,$D1,$T1,1);+	@thread2=&NEONROUND($A2,$B2,$C2,$D2,$T2,1);+	@thread3=&NEONROUND($A3,$B3,$C3,$D3,$T3,1);+	@thread4=&NEONROUND($A4,$B4,$C4,$D4,$T4,1);+	@thread5=&NEONROUND($A5,$B5,$C5,$D5,$T5,1);+	@thread67=(&ROUND(0,4,8,12),&ROUND(0,5,10,15));++	foreach (@thread0) {+		eval;			eval(shift(@thread67));+		eval(shift(@thread1));	eval(shift(@thread67));+		eval(shift(@thread2));	eval(shift(@thread67));+		eval(shift(@thread3));	eval(shift(@thread67));+		eval(shift(@thread4));	eval(shift(@thread67));+		eval(shift(@thread5));	eval(shift(@thread67));+	}+$code.=<<___;+	cbnz	$ctr,.Loop_upper_neon++	add.32	@x[0],@x[0],@d[0]		// accumulate key block+	add	@x[1],@x[1],@d[0],lsr#32+	add.32	@x[2],@x[2],@d[1]+	add	@x[3],@x[3],@d[1],lsr#32+	add.32	@x[4],@x[4],@d[2]+	add	@x[5],@x[5],@d[2],lsr#32+	add.32	@x[6],@x[6],@d[3]+	add	@x[7],@x[7],@d[3],lsr#32+	add.32	@x[8],@x[8],@d[4]+	add	@x[9],@x[9],@d[4],lsr#32+	add.32	@x[10],@x[10],@d[5]+	add	@x[11],@x[11],@d[5],lsr#32+	add.32	@x[12],@x[12],@d[6]+	add	@x[13],@x[13],@d[6],lsr#32+	add.32	@x[14],@x[14],@d[7]+	add	@x[15],@x[15],@d[7],lsr#32++	add	@x[0],@x[0],@x[1],lsl#32	// pack+	add	@x[2],@x[2],@x[3],lsl#32+	ldp	@x[1],@x[3],[$inp,#0]		// load input+	add	@x[4],@x[4],@x[5],lsl#32+	add	@x[6],@x[6],@x[7],lsl#32+	ldp	@x[5],@x[7],[$inp,#16]+	add	@x[8],@x[8],@x[9],lsl#32+	add	@x[10],@x[10],@x[11],lsl#32+	ldp	@x[9],@x[11],[$inp,#32]+	add	@x[12],@x[12],@x[13],lsl#32+	add	@x[14],@x[14],@x[15],lsl#32+	ldp	@x[13],@x[15],[$inp,#48]+	cadd	$inp,$inp,#64+#ifdef	__AARCH64EB__+	rev	@x[0],@x[0]+	rev	@x[2],@x[2]+	rev	@x[4],@x[4]+	rev	@x[6],@x[6]+	rev	@x[8],@x[8]+	rev	@x[10],@x[10]+	rev	@x[12],@x[12]+	rev	@x[14],@x[14]+#endif+	eor	@x[0],@x[0],@x[1]+	eor	@x[2],@x[2],@x[3]+	eor	@x[4],@x[4],@x[5]+	eor	@x[6],@x[6],@x[7]+	eor	@x[8],@x[8],@x[9]+	eor	@x[10],@x[10],@x[11]+	eor	@x[12],@x[12],@x[13]+	eor	@x[14],@x[14],@x[15]++	 stp	@x[0],@x[2],[$out,#0]		// store output+	 add	@d[6],@d[6],#1			// increment counter+	mov.32	@x[0],@d[0]			// unpack key block+	lsr	@x[1],@d[0],#32+	 stp	@x[4],@x[6],[$out,#16]+	mov.32	@x[2],@d[1]+	lsr	@x[3],@d[1],#32+	 stp	@x[8],@x[10],[$out,#32]+	mov.32	@x[4],@d[2]+	lsr	@x[5],@d[2],#32+	 stp	@x[12],@x[14],[$out,#48]+	 cadd	$out,$out,#64+	mov.32	@x[6],@d[3]+	lsr	@x[7],@d[3],#32+	mov.32	@x[8],@d[4]+	lsr	@x[9],@d[4],#32+	mov.32	@x[10],@d[5]+	lsr	@x[11],@d[5],#32+	mov.32	@x[12],@d[6]+	lsr	@x[13],@d[6],#32+	mov.32	@x[14],@d[7]+	lsr	@x[15],@d[7],#32++	mov	$ctr,#5+.Loop_lower_neon:+	sub	$ctr,$ctr,#1+___+	@thread0=&NEONROUND($A0,$B0,$C0,$D0,$T0,0);+	@thread1=&NEONROUND($A1,$B1,$C1,$D1,$T1,0);+	@thread2=&NEONROUND($A2,$B2,$C2,$D2,$T2,0);+	@thread3=&NEONROUND($A3,$B3,$C3,$D3,$T3,0);+	@thread4=&NEONROUND($A4,$B4,$C4,$D4,$T4,0);+	@thread5=&NEONROUND($A5,$B5,$C5,$D5,$T5,0);+	@thread67=(&ROUND(0,4,8,12),&ROUND(0,5,10,15));++	foreach (@thread0) {+		eval;			eval(shift(@thread67));+		eval(shift(@thread1));	eval(shift(@thread67));+		eval(shift(@thread2));	eval(shift(@thread67));+		eval(shift(@thread3));	eval(shift(@thread67));+		eval(shift(@thread4));	eval(shift(@thread67));+		eval(shift(@thread5));	eval(shift(@thread67));+	}++	@thread0=&NEONROUND($A0,$B0,$C0,$D0,$T0,1);+	@thread1=&NEONROUND($A1,$B1,$C1,$D1,$T1,1);+	@thread2=&NEONROUND($A2,$B2,$C2,$D2,$T2,1);+	@thread3=&NEONROUND($A3,$B3,$C3,$D3,$T3,1);+	@thread4=&NEONROUND($A4,$B4,$C4,$D4,$T4,1);+	@thread5=&NEONROUND($A5,$B5,$C5,$D5,$T5,1);+	@thread67=(&ROUND(0,4,8,12),&ROUND(0,5,10,15));++	foreach (@thread0) {+		eval;			eval(shift(@thread67));+		eval(shift(@thread1));	eval(shift(@thread67));+		eval(shift(@thread2));	eval(shift(@thread67));+		eval(shift(@thread3));	eval(shift(@thread67));+		eval(shift(@thread4));	eval(shift(@thread67));+		eval(shift(@thread5));	eval(shift(@thread67));+	}+$code.=<<___;+	cbnz	$ctr,.Loop_lower_neon++	add.32	@x[0],@x[0],@d[0]		// accumulate key block+	 ldp	@K[0],@K[1],[sp,#0]+	add	@x[1],@x[1],@d[0],lsr#32+	 ldp	@K[2],@K[3],[sp,#32]+	add.32	@x[2],@x[2],@d[1]+	 ldp	@K[4],@K[5],[sp,#64]+	add	@x[3],@x[3],@d[1],lsr#32+	 ldr	@K[6],[sp,#96]+	 add	$A0,$A0,@K[0]+	add.32	@x[4],@x[4],@d[2]+	 add	$A1,$A1,@K[0]+	add	@x[5],@x[5],@d[2],lsr#32+	 add	$A2,$A2,@K[0]+	add.32	@x[6],@x[6],@d[3]+	 add	$A3,$A3,@K[0]+	add	@x[7],@x[7],@d[3],lsr#32+	 add	$A4,$A4,@K[0]+	add.32	@x[8],@x[8],@d[4]+	 add	$A5,$A5,@K[0]+	add	@x[9],@x[9],@d[4],lsr#32+	 add	$C0,$C0,@K[2]+	add.32	@x[10],@x[10],@d[5]+	 add	$C1,$C1,@K[2]+	add	@x[11],@x[11],@d[5],lsr#32+	 add	$C2,$C2,@K[2]+	add.32	@x[12],@x[12],@d[6]+	 add	$C3,$C3,@K[2]+	add	@x[13],@x[13],@d[6],lsr#32+	 add	$C4,$C4,@K[2]+	add.32	@x[14],@x[14],@d[7]+	 add	$C5,$C5,@K[2]+	add	@x[15],@x[15],@d[7],lsr#32+	 add	$D4,$D4,$ONE			// +4+	add	@x[0],@x[0],@x[1],lsl#32	// pack+	 add	$D5,$D5,$ONE			// +4+	add	@x[2],@x[2],@x[3],lsl#32+	 add	$D0,$D0,@K[3]+	ldp	@x[1],@x[3],[$inp,#0]		// load input+	 add	$D1,$D1,@K[4]+	add	@x[4],@x[4],@x[5],lsl#32+	 add	$D2,$D2,@K[5]+	add	@x[6],@x[6],@x[7],lsl#32+	 add	$D3,$D3,@K[6]+	ldp	@x[5],@x[7],[$inp,#16]+	 add	$D4,$D4,@K[3]+	add	@x[8],@x[8],@x[9],lsl#32+	 add	$D5,$D5,@K[4]+	add	@x[10],@x[10],@x[11],lsl#32+	 add	$B0,$B0,@K[1]+	ldp	@x[9],@x[11],[$inp,#32]+	 add	$B1,$B1,@K[1]+	add	@x[12],@x[12],@x[13],lsl#32+	 add	$B2,$B2,@K[1]+	add	@x[14],@x[14],@x[15],lsl#32+	 add	$B3,$B3,@K[1]+	ldp	@x[13],@x[15],[$inp,#48]+	 add	$B4,$B4,@K[1]+	cadd	$inp,$inp,#64+	 add	$B5,$B5,@K[1]++#ifdef	__AARCH64EB__+	rev	@x[0],@x[0]+	rev	@x[2],@x[2]+	rev	@x[4],@x[4]+	rev	@x[6],@x[6]+	rev	@x[8],@x[8]+	rev	@x[10],@x[10]+	rev	@x[12],@x[12]+	rev	@x[14],@x[14]+#endif+	ld1.8	{$T0-$T3},[$inp],#64+	eor	@x[0],@x[0],@x[1]+	eor	@x[2],@x[2],@x[3]+	eor	@x[4],@x[4],@x[5]+	eor	@x[6],@x[6],@x[7]+	eor	@x[8],@x[8],@x[9]+	 eor	$A0,$A0,$T0+	eor	@x[10],@x[10],@x[11]+	 eor	$B0,$B0,$T1+	eor	@x[12],@x[12],@x[13]+	 eor	$C0,$C0,$T2+	eor	@x[14],@x[14],@x[15]+	 eor	$D0,$D0,$T3+	 ld1.8	{$T0-$T3},[$inp],#64++	stp	@x[0],@x[2],[$out,#0]		// store output+	 add	@d[6],@d[6],#7			// increment counter+	stp	@x[4],@x[6],[$out,#16]+	stp	@x[8],@x[10],[$out,#32]+	stp	@x[12],@x[14],[$out,#48]+	cadd	$out,$out,#64+	st1.8	{$A0-$D0},[$out],#64++	ld1.8	{$A0-$D0},[$inp],#64+	eor	$A1,$A1,$T0+	eor	$B1,$B1,$T1+	eor	$C1,$C1,$T2+	eor	$D1,$D1,$T3+	st1.8	{$A1-$D1},[$out],#64++	ld1.8	{$A1-$D1},[$inp],#64+	eor	$A2,$A2,$A0+	 ldp	@K[0],@K[1],[sp,#0]+	eor	$B2,$B2,$B0+	 ldp	@K[2],@K[3],[sp,#32]+	eor	$C2,$C2,$C0+	eor	$D2,$D2,$D0+	st1.8	{$A2-$D2},[$out],#64++	ld1.8	{$A2-$D2},[$inp],#64+	eor	$A3,$A3,$A1+	eor	$B3,$B3,$B1+	eor	$C3,$C3,$C1+	eor	$D3,$D3,$D1+	st1.8	{$A3-$D3},[$out],#64++	ld1.8	{$A3-$D3},[$inp],#64+	eor	$A4,$A4,$A2+	eor	$B4,$B4,$B2+	eor	$C4,$C4,$C2+	eor	$D4,$D4,$D2+	st1.8	{$A4-$D4},[$out],#64++	shl	$A0,$ONE,#1			// 4 -> 8+	eor	$A5,$A5,$A3+	eor	$B5,$B5,$B3+	eor	$C5,$C5,$C3+	eor	$D5,$D5,$D3+	st1.8	{$A5-$D5},[$out],#64++	add	@K[3],@K[3],$A0			// += 8+	add	@K[4],@K[4],$A0+	add	@K[5],@K[5],$A0+	add	@K[6],@K[6],$A0++	b.hs	.Loop_outer_512_neon++	adds	$len,$len,#512+	ushr	$ONE,$ONE,#1			// 4 -> 2++	ldp	d10,d11,[sp,#128+16]		// meet ABI requirements+	ldp	d12,d13,[sp,#128+32]+	ldp	d14,d15,[sp,#128+48]++	stp	@K[0],@K[0],[sp,#16]		// wipe key off-load area+	stp	@K[0],@K[0],[sp,#48]		// [with known constant]+	stp	@K[0],@K[0],[sp,#80]++	b.eq	.Ldone_512_neon++	// we have <512 bytes tail, harmonize state with other contexts+	csub	$key,$key,#16			// .Lone+	cmp	$len,#192+	cadd	sp,sp,#128+	sub	@K[3],@K[3],$ONE		// -= 2+	ld1	{$CTR,$ROT24},[$key]+	b.hs	.Loop_outer_neon++	ldp	d8,d9,[sp,#0]			// meet ABI requirements+	eor	@K[1],@K[1],@K[1]		// cleanse key and nonce+	eor	@K[2],@K[2],@K[2]+	eor	@K[3],@K[3],@K[3]+	eor	@K[4],@K[4],@K[4]+	eor	@K[5],@K[5],@K[5]+	eor	@K[6],@K[6],@K[6]+	b	.Loop_outer++.Ldone_512_neon:+	ldp	d8,d9,[sp,#128+0]		// meet ABI requirements+	eor	@K[1],@K[1],@K[1]		// cleanse key and nonce+	eor	@K[2],@K[2],@K[2]+	eor	@K[3],@K[3],@K[3]+	eor	@K[4],@K[4],@K[4]+	eor	@K[5],@K[5],@K[5]+	eor	@K[6],@K[6],@K[6]++	ldp	c19,c20,[c29,#2*__SIZEOF_POINTER__]+	add	csp,csp,#128+64+	ldp	c21,c22,[c29,#4*__SIZEOF_POINTER__]+	ldp	c23,c24,[c29,#6*__SIZEOF_POINTER__]+	ldp	c25,c26,[c29,#8*__SIZEOF_POINTER__]+	ldp	c27,c28,[c29,#10*__SIZEOF_POINTER__]+	ldp	c29,c30,[csp],#12*__SIZEOF_POINTER__+	.inst	0xd50323bf			// autiasp+	ret+.size	ChaCha20_512_neon,.-ChaCha20_512_neon+___+}+}}}++foreach (split("\n",$code)) {+	s/\`([^\`]*)\`/eval $1/geo;++	(s/\b([a-z]+)\.32\b/$1/ and (s/x([0-9]+)/w$1/g or 1))	or+	(m/\b(eor|ext|mov|tbl)\b/ and (s/\.4s/\.16b/g or 1))	or+	(s/\b((?:ld|st)1)\.8\b/$1/ and (s/\.4s/\.16b/g or 1))	or+	(m/\b(ld|st)[rp]\b/ and (s/v([0-9]+)\.4s/q$1/g or 1))	or+	(m/\b(dup|ld1)\b/ and (s/\.4(s}?\[[0-3]\])/.$1/g or 1))	or+	(s/\b(zip[12])\.64\b/$1/ and (s/\.4s/\.2d/g or 1))	or+	(s/\brev32\.16\b/rev32/ and (s/\.4s/\.8h/g or 1));++	#s/\bq([0-9]+)#(lo|hi)/sprintf "d%d",2*$1+($2 eq "hi")/geo;++	print $_,"\n";+}+close STDOUT;	# flush
+ cbits/asm/chacha-x86_64-elf.S view
@@ -0,0 +1,2241 @@+.text	++++.align	64+.Lzero:+.long	0,0,0,0+.Lone:+.long	1,0,0,0+.Linc:+.long	0,1,2,3+.Lfour:+.long	4,4,4,4+.Lincy:+.long	0,2,4,6,1,3,5,7+.Leight:+.long	8,8,8,8,8,8,8,8+.Lrot16:+.byte	0x2,0x3,0x0,0x1, 0x6,0x7,0x4,0x5, 0xa,0xb,0x8,0x9, 0xe,0xf,0xc,0xd+.Lrot24:+.byte	0x3,0x0,0x1,0x2, 0x7,0x4,0x5,0x6, 0xb,0x8,0x9,0xa, 0xf,0xc,0xd,0xe+.Ltwoy:+.long	2,0,0,0, 2,0,0,0+.align	64+.Lzeroz:+.long	0,0,0,0, 1,0,0,0, 2,0,0,0, 3,0,0,0+.Lfourz:+.long	4,0,0,0, 4,0,0,0, 4,0,0,0, 4,0,0,0+.Lincz:+.long	0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15+.Lsixteen:+.long	16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16+.Lsigma:+.byte	101,120,112,97,110,100,32,51,50,45,98,121,116,101,32,107,0+.byte	67,104,97,67,104,97,50,48,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.globl	crypton_chacha20_asm_ctr32+.type	crypton_chacha20_asm_ctr32,@function+.align	64+crypton_chacha20_asm_ctr32:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	cmpq	$0,%rdx+	je	.Lno_data+	movq	crypton_ia32cap_P+4(%rip),%r9+	testl	$512,%r9d+	jnz	.Lcrypton_chacha20_asm_ssse3+	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56+	subq	$64+24,%rsp+.cfi_adjust_cfa_offset	88+.Lctr32_body:++	movq	%rdx,%rbp++	movq	0(%rcx),%r12+	movq	8(%rcx),%r13+	movq	16(%rcx),%r14+	movq	24(%rcx),%r15+	movq	0(%r8),%rax+	movq	8(%r8),%rdx+	movq	%r12,16(%rsp)+	movq	%r13,24(%rsp)+	movq	%r14,0(%rsp)+	movq	%r15,8(%rsp)+	movq	%rax,48(%rsp)+	movq	%rdx,56(%rsp)+	jmp	.Loop_outer++.align	32+.Loop_outer:+	movl	$0x61707865,%eax+	movl	$0x3320646e,%ebx+	movl	$0x79622d32,%ecx+	movl	$0x6b206574,%edx+	movl	16(%rsp),%r8d+	movl	20(%rsp),%r9d+	movl	24(%rsp),%r10d+	movl	28(%rsp),%r11d+	movl	48(%rsp),%r12d+	movl	52(%rsp),%r13d+	movl	56(%rsp),%r14d+	movq	%r15,40(%rsp)+	movl	60(%rsp),%r15d++	movq	%rbp,64+0(%rsp)+	movq	%rsi,64+8(%rsp)+	movl	0(%rsp),%esi+	movq	%rdi,64+16(%rsp)+	movl	4(%rsp),%edi+	movl	$10,%ebp+	jmp	.Loop++.align	32+.Loop:+	addl	%r8d,%eax+	xorl	%eax,%r12d+	roll	$16,%r12d+	addl	%r9d,%ebx+	xorl	%ebx,%r13d+	roll	$16,%r13d+	addl	%r12d,%esi+	xorl	%esi,%r8d+	roll	$12,%r8d+	addl	%r13d,%edi+	xorl	%edi,%r9d+	roll	$12,%r9d+	addl	%r8d,%eax+	xorl	%eax,%r12d+	roll	$8,%r12d+	addl	%r9d,%ebx+	xorl	%ebx,%r13d+	roll	$8,%r13d+	addl	%r12d,%esi+	xorl	%esi,%r8d+	roll	$7,%r8d+	addl	%r13d,%edi+	xorl	%edi,%r9d+	roll	$7,%r9d+	movl	%esi,32(%rsp)+	movl	%edi,36(%rsp)+	movl	40(%rsp),%esi+	movl	44(%rsp),%edi+	addl	%r10d,%ecx+	xorl	%ecx,%r14d+	roll	$16,%r14d+	addl	%r11d,%edx+	xorl	%edx,%r15d+	roll	$16,%r15d+	addl	%r14d,%esi+	xorl	%esi,%r10d+	roll	$12,%r10d+	addl	%r15d,%edi+	xorl	%edi,%r11d+	roll	$12,%r11d+	addl	%r10d,%ecx+	xorl	%ecx,%r14d+	roll	$8,%r14d+	addl	%r11d,%edx+	xorl	%edx,%r15d+	roll	$8,%r15d+	addl	%r14d,%esi+	xorl	%esi,%r10d+	roll	$7,%r10d+	addl	%r15d,%edi+	xorl	%edi,%r11d+	roll	$7,%r11d+	addl	%r9d,%eax+	xorl	%eax,%r15d+	roll	$16,%r15d+	addl	%r10d,%ebx+	xorl	%ebx,%r12d+	roll	$16,%r12d+	addl	%r15d,%esi+	xorl	%esi,%r9d+	roll	$12,%r9d+	addl	%r12d,%edi+	xorl	%edi,%r10d+	roll	$12,%r10d+	addl	%r9d,%eax+	xorl	%eax,%r15d+	roll	$8,%r15d+	addl	%r10d,%ebx+	xorl	%ebx,%r12d+	roll	$8,%r12d+	addl	%r15d,%esi+	xorl	%esi,%r9d+	roll	$7,%r9d+	addl	%r12d,%edi+	xorl	%edi,%r10d+	roll	$7,%r10d+	movl	%esi,40(%rsp)+	movl	%edi,44(%rsp)+	movl	32(%rsp),%esi+	movl	36(%rsp),%edi+	addl	%r11d,%ecx+	xorl	%ecx,%r13d+	roll	$16,%r13d+	addl	%r8d,%edx+	xorl	%edx,%r14d+	roll	$16,%r14d+	addl	%r13d,%esi+	xorl	%esi,%r11d+	roll	$12,%r11d+	addl	%r14d,%edi+	xorl	%edi,%r8d+	roll	$12,%r8d+	addl	%r11d,%ecx+	xorl	%ecx,%r13d+	roll	$8,%r13d+	addl	%r8d,%edx+	xorl	%edx,%r14d+	roll	$8,%r14d+	addl	%r13d,%esi+	xorl	%esi,%r11d+	roll	$7,%r11d+	addl	%r14d,%edi+	xorl	%edi,%r8d+	roll	$7,%r8d+	decl	%ebp+	jnz	.Loop+	addl	0(%rsp),%esi+	addl	4(%rsp),%edi+	movq	64(%rsp),%rbp+	movl	%esi,32(%rsp)+	movq	64+8(%rsp),%rsi+	movl	%edi,36(%rsp)+	movq	64+16(%rsp),%rdi++	addl	$0x61707865,%eax+	addl	$0x3320646e,%ebx+	addl	$0x79622d32,%ecx+	addl	$0x6b206574,%edx+	addl	16(%rsp),%r8d+	addl	20(%rsp),%r9d+	addl	24(%rsp),%r10d+	addl	28(%rsp),%r11d+	addl	48(%rsp),%r12d+	addl	52(%rsp),%r13d+	addl	56(%rsp),%r14d+	addl	60(%rsp),%r15d++	cmpq	$64,%rbp+	jb	.Ltail++	xorl	0(%rsi),%eax+	xorl	4(%rsi),%ebx+	xorl	8(%rsi),%ecx+	xorl	12(%rsi),%edx+	movl	%eax,0(%rdi)+	movl	32(%rsp),%eax+	movl	%ebx,4(%rdi)+	movl	36(%rsp),%ebx+	movl	%ecx,8(%rdi)+	movl	40(%rsp),%ecx+	movl	%edx,12(%rdi)+	movl	44(%rsp),%edx+	xorl	16(%rsi),%r8d+	addl	8(%rsp),%ecx+	xorl	20(%rsi),%r9d+	addl	12(%rsp),%edx+	xorl	24(%rsi),%r10d+	xorl	28(%rsi),%r11d+	xorl	32(%rsi),%eax+	xorl	36(%rsi),%ebx+	xorl	40(%rsi),%ecx+	xorl	44(%rsi),%edx+	xorl	48(%rsi),%r12d+	xorl	52(%rsi),%r13d+	xorl	56(%rsi),%r14d+	xorl	60(%rsi),%r15d+	leaq	64(%rsi),%rsi++	addl	$1,48(%rsp)++	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)+	movl	%eax,32(%rdi)+	movl	%ebx,36(%rdi)+	movl	%ecx,40(%rdi)+	movl	%edx,44(%rdi)+	movl	%r12d,48(%rdi)+	movl	%r13d,52(%rdi)+	movl	%r14d,56(%rdi)+	movl	%r15d,60(%rdi)+	leaq	64(%rdi),%rdi+	movq	8(%rsp),%r15++	subq	$64,%rbp+	jnz	.Loop_outer++	jmp	.Ldone++.align	16+.Ltail:+	movl	%eax,0(%rsp)+	movl	8(%rsp),%eax+	movl	%ebx,4(%rsp)+	movl	12(%rsp),%ebx+	movl	%ecx,8(%rsp)+	addl	40(%rsp),%eax+	movl	%edx,12(%rsp)+	addl	44(%rsp),%ebx+	movl	%r8d,16(%rsp)+	movl	%r9d,20(%rsp)+	movl	%r10d,24(%rsp)+	movl	%r11d,28(%rsp)+	movl	%eax,40(%rsp)+	movl	%ebx,44(%rsp)+	xorq	%rbx,%rbx+	movl	%r12d,48(%rsp)+	movl	%r13d,52(%rsp)+	movl	%r14d,56(%rsp)+	movl	%r15d,60(%rsp)++.Loop_tail:+	movzbl	(%rsi,%rbx,1),%eax+	movzbl	(%rsp,%rbx,1),%edx+	leaq	1(%rbx),%rbx+	xorl	%edx,%eax+	movb	%al,-1(%rdi,%rbx,1)+	decq	%rbp+	jnz	.Loop_tail++.Ldone:+	leaq	64+24+48(%rsp),%rsi+.cfi_def_cfa	%rsi,8+	movq	-48(%rsi),%r15+.cfi_restore	%r15+	movq	-40(%rsi),%r14+.cfi_restore	%r14+	movq	-32(%rsi),%r13+.cfi_restore	%r13+	movq	-24(%rsi),%r12+.cfi_restore	%r12+	movq	-16(%rsi),%rbp+.cfi_restore	%rbp+	movq	-8(%rsi),%rbx+.cfi_restore	%rbx+	leaq	(%rsi),%rsp+.cfi_def_cfa_register	%rsp+.Lno_data:+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_chacha20_asm_ctr32,.-crypton_chacha20_asm_ctr32+.type	crypton_chacha20_asm_ssse3,@function+.align	32+crypton_chacha20_asm_ssse3:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++.Lcrypton_chacha20_asm_ssse3:+	movq	%rsp,%r10+.cfi_def_cfa_register	%r10+	testl	$2048,%r9d+	jnz	.Lcrypton_chacha20_asm_4xop+	cmpq	$128,%rdx+	je	.Lcrypton_chacha20_asm_128+	ja	.Lcrypton_chacha20_asm_4x++.Ldo_sse3_after_all:+	subq	$64+8,%rsp+	andq	$-16,%rsp+	movdqa	.Lsigma(%rip),%xmm0+	movdqu	(%rcx),%xmm1+	movdqu	16(%rcx),%xmm2+	movdqu	(%r8),%xmm3+	movdqa	.Lrot16(%rip),%xmm6+	movdqa	.Lrot24(%rip),%xmm7++	movdqa	%xmm0,0(%rsp)+	movdqa	%xmm1,16(%rsp)+	movdqa	%xmm2,32(%rsp)+	movdqa	%xmm3,48(%rsp)+	movq	$10,%r8+	jmp	.Loop_ssse3++.align	32+.Loop_outer_ssse3:+	movdqa	.Lone(%rip),%xmm3+	movdqa	0(%rsp),%xmm0+	movdqa	16(%rsp),%xmm1+	movdqa	32(%rsp),%xmm2+	paddd	48(%rsp),%xmm3+	movq	$10,%r8+	movdqa	%xmm3,48(%rsp)+	jmp	.Loop_ssse3++.align	32+.Loop_ssse3:+	paddd	%xmm1,%xmm0+	pxor	%xmm0,%xmm3+.byte	102,15,56,0,222+	paddd	%xmm3,%xmm2+	pxor	%xmm2,%xmm1+	movdqa	%xmm1,%xmm4+	psrld	$20,%xmm1+	pslld	$12,%xmm4+	por	%xmm4,%xmm1+	paddd	%xmm1,%xmm0+	pxor	%xmm0,%xmm3+.byte	102,15,56,0,223+	paddd	%xmm3,%xmm2+	pxor	%xmm2,%xmm1+	movdqa	%xmm1,%xmm4+	psrld	$25,%xmm1+	pslld	$7,%xmm4+	por	%xmm4,%xmm1+	pshufd	$78,%xmm2,%xmm2+	pshufd	$57,%xmm1,%xmm1+	pshufd	$147,%xmm3,%xmm3+	nop+	paddd	%xmm1,%xmm0+	pxor	%xmm0,%xmm3+.byte	102,15,56,0,222+	paddd	%xmm3,%xmm2+	pxor	%xmm2,%xmm1+	movdqa	%xmm1,%xmm4+	psrld	$20,%xmm1+	pslld	$12,%xmm4+	por	%xmm4,%xmm1+	paddd	%xmm1,%xmm0+	pxor	%xmm0,%xmm3+.byte	102,15,56,0,223+	paddd	%xmm3,%xmm2+	pxor	%xmm2,%xmm1+	movdqa	%xmm1,%xmm4+	psrld	$25,%xmm1+	pslld	$7,%xmm4+	por	%xmm4,%xmm1+	pshufd	$78,%xmm2,%xmm2+	pshufd	$147,%xmm1,%xmm1+	pshufd	$57,%xmm3,%xmm3+	decq	%r8+	jnz	.Loop_ssse3+	paddd	0(%rsp),%xmm0+	paddd	16(%rsp),%xmm1+	paddd	32(%rsp),%xmm2+	paddd	48(%rsp),%xmm3++	cmpq	$64,%rdx+	jb	.Ltail_ssse3++	movdqu	0(%rsi),%xmm4+	movdqu	16(%rsi),%xmm5+	pxor	%xmm4,%xmm0+	movdqu	32(%rsi),%xmm4+	pxor	%xmm5,%xmm1+	movdqu	48(%rsi),%xmm5+	leaq	64(%rsi),%rsi+	pxor	%xmm4,%xmm2+	pxor	%xmm5,%xmm3++	movdqu	%xmm0,0(%rdi)+	movdqu	%xmm1,16(%rdi)+	movdqu	%xmm2,32(%rdi)+	movdqu	%xmm3,48(%rdi)+	leaq	64(%rdi),%rdi++	subq	$64,%rdx+	jnz	.Loop_outer_ssse3++	jmp	.Ldone_ssse3++.align	16+.Ltail_ssse3:+	movdqa	%xmm0,0(%rsp)+	movdqa	%xmm1,16(%rsp)+	movdqa	%xmm2,32(%rsp)+	movdqa	%xmm3,48(%rsp)+	xorq	%r8,%r8++.Loop_tail_ssse3:+	movzbl	(%rsi,%r8,1),%eax+	movzbl	(%rsp,%r8,1),%ecx+	leaq	1(%r8),%r8+	xorl	%ecx,%eax+	movb	%al,-1(%rdi,%r8,1)+	decq	%rdx+	jnz	.Loop_tail_ssse3++.Ldone_ssse3:+	leaq	(%r10),%rsp+.cfi_def_cfa_register	%rsp+.Lssse3_epilogue:+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_chacha20_asm_ssse3,.-crypton_chacha20_asm_ssse3+.type	crypton_chacha20_asm_128,@function+.align	32+crypton_chacha20_asm_128:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++.Lcrypton_chacha20_asm_128:+	movq	%rsp,%r10+.cfi_def_cfa_register	%r10+	subq	$64+8,%rsp+	andq	$-16,%rsp+	movdqa	.Lsigma(%rip),%xmm8+	movdqu	(%rcx),%xmm9+	movdqu	16(%rcx),%xmm2+	movdqu	(%r8),%xmm3+	movdqa	.Lone(%rip),%xmm1+	movdqa	.Lrot16(%rip),%xmm6+	movdqa	.Lrot24(%rip),%xmm7++	movdqa	%xmm8,%xmm10+	movdqa	%xmm8,0(%rsp)+	movdqa	%xmm9,%xmm11+	movdqa	%xmm9,16(%rsp)+	movdqa	%xmm2,%xmm0+	movdqa	%xmm2,32(%rsp)+	paddd	%xmm3,%xmm1+	movdqa	%xmm3,48(%rsp)+	movq	$10,%r8+	jmp	.Loop_128++.align	32+.Loop_128:+	paddd	%xmm9,%xmm8+	pxor	%xmm8,%xmm3+	paddd	%xmm11,%xmm10+	pxor	%xmm10,%xmm1+.byte	102,15,56,0,222+.byte	102,15,56,0,206+	paddd	%xmm3,%xmm2+	paddd	%xmm1,%xmm0+	pxor	%xmm2,%xmm9+	pxor	%xmm0,%xmm11+	movdqa	%xmm9,%xmm4+	psrld	$20,%xmm9+	movdqa	%xmm11,%xmm5+	pslld	$12,%xmm4+	psrld	$20,%xmm11+	por	%xmm4,%xmm9+	pslld	$12,%xmm5+	por	%xmm5,%xmm11+	paddd	%xmm9,%xmm8+	pxor	%xmm8,%xmm3+	paddd	%xmm11,%xmm10+	pxor	%xmm10,%xmm1+.byte	102,15,56,0,223+.byte	102,15,56,0,207+	paddd	%xmm3,%xmm2+	paddd	%xmm1,%xmm0+	pxor	%xmm2,%xmm9+	pxor	%xmm0,%xmm11+	movdqa	%xmm9,%xmm4+	psrld	$25,%xmm9+	movdqa	%xmm11,%xmm5+	pslld	$7,%xmm4+	psrld	$25,%xmm11+	por	%xmm4,%xmm9+	pslld	$7,%xmm5+	por	%xmm5,%xmm11+	pshufd	$78,%xmm2,%xmm2+	pshufd	$57,%xmm9,%xmm9+	pshufd	$147,%xmm3,%xmm3+	pshufd	$78,%xmm0,%xmm0+	pshufd	$57,%xmm11,%xmm11+	pshufd	$147,%xmm1,%xmm1+	paddd	%xmm9,%xmm8+	pxor	%xmm8,%xmm3+	paddd	%xmm11,%xmm10+	pxor	%xmm10,%xmm1+.byte	102,15,56,0,222+.byte	102,15,56,0,206+	paddd	%xmm3,%xmm2+	paddd	%xmm1,%xmm0+	pxor	%xmm2,%xmm9+	pxor	%xmm0,%xmm11+	movdqa	%xmm9,%xmm4+	psrld	$20,%xmm9+	movdqa	%xmm11,%xmm5+	pslld	$12,%xmm4+	psrld	$20,%xmm11+	por	%xmm4,%xmm9+	pslld	$12,%xmm5+	por	%xmm5,%xmm11+	paddd	%xmm9,%xmm8+	pxor	%xmm8,%xmm3+	paddd	%xmm11,%xmm10+	pxor	%xmm10,%xmm1+.byte	102,15,56,0,223+.byte	102,15,56,0,207+	paddd	%xmm3,%xmm2+	paddd	%xmm1,%xmm0+	pxor	%xmm2,%xmm9+	pxor	%xmm0,%xmm11+	movdqa	%xmm9,%xmm4+	psrld	$25,%xmm9+	movdqa	%xmm11,%xmm5+	pslld	$7,%xmm4+	psrld	$25,%xmm11+	por	%xmm4,%xmm9+	pslld	$7,%xmm5+	por	%xmm5,%xmm11+	pshufd	$78,%xmm2,%xmm2+	pshufd	$147,%xmm9,%xmm9+	pshufd	$57,%xmm3,%xmm3+	pshufd	$78,%xmm0,%xmm0+	pshufd	$147,%xmm11,%xmm11+	pshufd	$57,%xmm1,%xmm1+	decq	%r8+	jnz	.Loop_128+	paddd	0(%rsp),%xmm8+	paddd	16(%rsp),%xmm9+	paddd	32(%rsp),%xmm2+	paddd	48(%rsp),%xmm3+	paddd	.Lone(%rip),%xmm1+	paddd	0(%rsp),%xmm10+	paddd	16(%rsp),%xmm11+	paddd	32(%rsp),%xmm0+	paddd	48(%rsp),%xmm1++	movdqu	0(%rsi),%xmm4+	movdqu	16(%rsi),%xmm5+	pxor	%xmm4,%xmm8+	movdqu	32(%rsi),%xmm4+	pxor	%xmm5,%xmm9+	movdqu	48(%rsi),%xmm5+	pxor	%xmm4,%xmm2+	movdqu	64(%rsi),%xmm4+	pxor	%xmm5,%xmm3+	movdqu	80(%rsi),%xmm5+	pxor	%xmm4,%xmm10+	movdqu	96(%rsi),%xmm4+	pxor	%xmm5,%xmm11+	movdqu	112(%rsi),%xmm5+	pxor	%xmm4,%xmm0+	pxor	%xmm5,%xmm1++	movdqu	%xmm8,0(%rdi)+	movdqu	%xmm9,16(%rdi)+	movdqu	%xmm2,32(%rdi)+	movdqu	%xmm3,48(%rdi)+	movdqu	%xmm10,64(%rdi)+	movdqu	%xmm11,80(%rdi)+	movdqu	%xmm0,96(%rdi)+	movdqu	%xmm1,112(%rdi)+	leaq	(%r10),%rsp+.cfi_def_cfa_register	%rsp+.L128_epilogue:+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_chacha20_asm_128,.-crypton_chacha20_asm_128+.type	crypton_chacha20_asm_4x,@function+.align	32+crypton_chacha20_asm_4x:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++.Lcrypton_chacha20_asm_4x:+	movq	%rsp,%r10+.cfi_def_cfa_register	%r10+	movq	%r9,%r11+	shrq	$32,%r9+	testq	$32,%r9+	jnz	.Lcrypton_chacha20_asm_8x+	cmpq	$192,%rdx+	ja	.Lproceed4x++	andq	$71303168,%r11+	cmpq	$4194304,%r11+	je	.Ldo_sse3_after_all++.Lproceed4x:+	subq	$0x140+8,%rsp+	andq	$-16,%rsp+	movdqa	.Lsigma(%rip),%xmm11+	movdqu	(%rcx),%xmm15+	movdqu	16(%rcx),%xmm7+	movdqu	(%r8),%xmm3+	leaq	256(%rsp),%rcx+	leaq	.Lrot16(%rip),%r9+	leaq	.Lrot24(%rip),%r11++	pshufd	$0x00,%xmm11,%xmm8+	pshufd	$0x55,%xmm11,%xmm9+	movdqa	%xmm8,64(%rsp)+	pshufd	$0xaa,%xmm11,%xmm10+	movdqa	%xmm9,80(%rsp)+	pshufd	$0xff,%xmm11,%xmm11+	movdqa	%xmm10,96(%rsp)+	movdqa	%xmm11,112(%rsp)++	pshufd	$0x00,%xmm15,%xmm12+	pshufd	$0x55,%xmm15,%xmm13+	movdqa	%xmm12,128-256(%rcx)+	pshufd	$0xaa,%xmm15,%xmm14+	movdqa	%xmm13,144-256(%rcx)+	pshufd	$0xff,%xmm15,%xmm15+	movdqa	%xmm14,160-256(%rcx)+	movdqa	%xmm15,176-256(%rcx)++	pshufd	$0x00,%xmm7,%xmm4+	pshufd	$0x55,%xmm7,%xmm5+	movdqa	%xmm4,192-256(%rcx)+	pshufd	$0xaa,%xmm7,%xmm6+	movdqa	%xmm5,208-256(%rcx)+	pshufd	$0xff,%xmm7,%xmm7+	movdqa	%xmm6,224-256(%rcx)+	movdqa	%xmm7,240-256(%rcx)++	pshufd	$0x00,%xmm3,%xmm0+	pshufd	$0x55,%xmm3,%xmm1+	paddd	.Linc(%rip),%xmm0+	pshufd	$0xaa,%xmm3,%xmm2+	movdqa	%xmm1,272-256(%rcx)+	pshufd	$0xff,%xmm3,%xmm3+	movdqa	%xmm2,288-256(%rcx)+	movdqa	%xmm3,304-256(%rcx)++	jmp	.Loop_enter4x++.align	32+.Loop_outer4x:+	movdqa	64(%rsp),%xmm8+	movdqa	80(%rsp),%xmm9+	movdqa	96(%rsp),%xmm10+	movdqa	112(%rsp),%xmm11+	movdqa	128-256(%rcx),%xmm12+	movdqa	144-256(%rcx),%xmm13+	movdqa	160-256(%rcx),%xmm14+	movdqa	176-256(%rcx),%xmm15+	movdqa	192-256(%rcx),%xmm4+	movdqa	208-256(%rcx),%xmm5+	movdqa	224-256(%rcx),%xmm6+	movdqa	240-256(%rcx),%xmm7+	movdqa	256-256(%rcx),%xmm0+	movdqa	272-256(%rcx),%xmm1+	movdqa	288-256(%rcx),%xmm2+	movdqa	304-256(%rcx),%xmm3+	paddd	.Lfour(%rip),%xmm0++.Loop_enter4x:+	movdqa	%xmm6,32(%rsp)+	movdqa	%xmm7,48(%rsp)+	movdqa	(%r9),%xmm7+	movl	$10,%eax+	movdqa	%xmm0,256-256(%rcx)+	jmp	.Loop4x++.align	32+.Loop4x:+	paddd	%xmm12,%xmm8+	paddd	%xmm13,%xmm9+	pxor	%xmm8,%xmm0+	pxor	%xmm9,%xmm1+.byte	102,15,56,0,199+.byte	102,15,56,0,207+	paddd	%xmm0,%xmm4+	paddd	%xmm1,%xmm5+	pxor	%xmm4,%xmm12+	pxor	%xmm5,%xmm13+	movdqa	%xmm12,%xmm6+	pslld	$12,%xmm12+	psrld	$20,%xmm6+	movdqa	%xmm13,%xmm7+	pslld	$12,%xmm13+	por	%xmm6,%xmm12+	psrld	$20,%xmm7+	movdqa	(%r11),%xmm6+	por	%xmm7,%xmm13+	paddd	%xmm12,%xmm8+	paddd	%xmm13,%xmm9+	pxor	%xmm8,%xmm0+	pxor	%xmm9,%xmm1+.byte	102,15,56,0,198+.byte	102,15,56,0,206+	paddd	%xmm0,%xmm4+	paddd	%xmm1,%xmm5+	pxor	%xmm4,%xmm12+	pxor	%xmm5,%xmm13+	movdqa	%xmm12,%xmm7+	pslld	$7,%xmm12+	psrld	$25,%xmm7+	movdqa	%xmm13,%xmm6+	pslld	$7,%xmm13+	por	%xmm7,%xmm12+	psrld	$25,%xmm6+	movdqa	(%r9),%xmm7+	por	%xmm6,%xmm13+	movdqa	%xmm4,0(%rsp)+	movdqa	%xmm5,16(%rsp)+	movdqa	32(%rsp),%xmm4+	movdqa	48(%rsp),%xmm5+	paddd	%xmm14,%xmm10+	paddd	%xmm15,%xmm11+	pxor	%xmm10,%xmm2+	pxor	%xmm11,%xmm3+.byte	102,15,56,0,215+.byte	102,15,56,0,223+	paddd	%xmm2,%xmm4+	paddd	%xmm3,%xmm5+	pxor	%xmm4,%xmm14+	pxor	%xmm5,%xmm15+	movdqa	%xmm14,%xmm6+	pslld	$12,%xmm14+	psrld	$20,%xmm6+	movdqa	%xmm15,%xmm7+	pslld	$12,%xmm15+	por	%xmm6,%xmm14+	psrld	$20,%xmm7+	movdqa	(%r11),%xmm6+	por	%xmm7,%xmm15+	paddd	%xmm14,%xmm10+	paddd	%xmm15,%xmm11+	pxor	%xmm10,%xmm2+	pxor	%xmm11,%xmm3+.byte	102,15,56,0,214+.byte	102,15,56,0,222+	paddd	%xmm2,%xmm4+	paddd	%xmm3,%xmm5+	pxor	%xmm4,%xmm14+	pxor	%xmm5,%xmm15+	movdqa	%xmm14,%xmm7+	pslld	$7,%xmm14+	psrld	$25,%xmm7+	movdqa	%xmm15,%xmm6+	pslld	$7,%xmm15+	por	%xmm7,%xmm14+	psrld	$25,%xmm6+	movdqa	(%r9),%xmm7+	por	%xmm6,%xmm15+	paddd	%xmm13,%xmm8+	paddd	%xmm14,%xmm9+	pxor	%xmm8,%xmm3+	pxor	%xmm9,%xmm0+.byte	102,15,56,0,223+.byte	102,15,56,0,199+	paddd	%xmm3,%xmm4+	paddd	%xmm0,%xmm5+	pxor	%xmm4,%xmm13+	pxor	%xmm5,%xmm14+	movdqa	%xmm13,%xmm6+	pslld	$12,%xmm13+	psrld	$20,%xmm6+	movdqa	%xmm14,%xmm7+	pslld	$12,%xmm14+	por	%xmm6,%xmm13+	psrld	$20,%xmm7+	movdqa	(%r11),%xmm6+	por	%xmm7,%xmm14+	paddd	%xmm13,%xmm8+	paddd	%xmm14,%xmm9+	pxor	%xmm8,%xmm3+	pxor	%xmm9,%xmm0+.byte	102,15,56,0,222+.byte	102,15,56,0,198+	paddd	%xmm3,%xmm4+	paddd	%xmm0,%xmm5+	pxor	%xmm4,%xmm13+	pxor	%xmm5,%xmm14+	movdqa	%xmm13,%xmm7+	pslld	$7,%xmm13+	psrld	$25,%xmm7+	movdqa	%xmm14,%xmm6+	pslld	$7,%xmm14+	por	%xmm7,%xmm13+	psrld	$25,%xmm6+	movdqa	(%r9),%xmm7+	por	%xmm6,%xmm14+	movdqa	%xmm4,32(%rsp)+	movdqa	%xmm5,48(%rsp)+	movdqa	0(%rsp),%xmm4+	movdqa	16(%rsp),%xmm5+	paddd	%xmm15,%xmm10+	paddd	%xmm12,%xmm11+	pxor	%xmm10,%xmm1+	pxor	%xmm11,%xmm2+.byte	102,15,56,0,207+.byte	102,15,56,0,215+	paddd	%xmm1,%xmm4+	paddd	%xmm2,%xmm5+	pxor	%xmm4,%xmm15+	pxor	%xmm5,%xmm12+	movdqa	%xmm15,%xmm6+	pslld	$12,%xmm15+	psrld	$20,%xmm6+	movdqa	%xmm12,%xmm7+	pslld	$12,%xmm12+	por	%xmm6,%xmm15+	psrld	$20,%xmm7+	movdqa	(%r11),%xmm6+	por	%xmm7,%xmm12+	paddd	%xmm15,%xmm10+	paddd	%xmm12,%xmm11+	pxor	%xmm10,%xmm1+	pxor	%xmm11,%xmm2+.byte	102,15,56,0,206+.byte	102,15,56,0,214+	paddd	%xmm1,%xmm4+	paddd	%xmm2,%xmm5+	pxor	%xmm4,%xmm15+	pxor	%xmm5,%xmm12+	movdqa	%xmm15,%xmm7+	pslld	$7,%xmm15+	psrld	$25,%xmm7+	movdqa	%xmm12,%xmm6+	pslld	$7,%xmm12+	por	%xmm7,%xmm15+	psrld	$25,%xmm6+	movdqa	(%r9),%xmm7+	por	%xmm6,%xmm12+	decl	%eax+	jnz	.Loop4x++	paddd	64(%rsp),%xmm8+	paddd	80(%rsp),%xmm9+	paddd	96(%rsp),%xmm10+	paddd	112(%rsp),%xmm11++	movdqa	%xmm8,%xmm6+	punpckldq	%xmm9,%xmm8+	movdqa	%xmm10,%xmm7+	punpckldq	%xmm11,%xmm10+	punpckhdq	%xmm9,%xmm6+	punpckhdq	%xmm11,%xmm7+	movdqa	%xmm8,%xmm9+	punpcklqdq	%xmm10,%xmm8+	movdqa	%xmm6,%xmm11+	punpcklqdq	%xmm7,%xmm6+	punpckhqdq	%xmm10,%xmm9+	punpckhqdq	%xmm7,%xmm11+	paddd	128-256(%rcx),%xmm12+	paddd	144-256(%rcx),%xmm13+	paddd	160-256(%rcx),%xmm14+	paddd	176-256(%rcx),%xmm15++	movdqa	%xmm8,0(%rsp)+	movdqa	%xmm9,16(%rsp)+	movdqa	32(%rsp),%xmm8+	movdqa	48(%rsp),%xmm9++	movdqa	%xmm12,%xmm10+	punpckldq	%xmm13,%xmm12+	movdqa	%xmm14,%xmm7+	punpckldq	%xmm15,%xmm14+	punpckhdq	%xmm13,%xmm10+	punpckhdq	%xmm15,%xmm7+	movdqa	%xmm12,%xmm13+	punpcklqdq	%xmm14,%xmm12+	movdqa	%xmm10,%xmm15+	punpcklqdq	%xmm7,%xmm10+	punpckhqdq	%xmm14,%xmm13+	punpckhqdq	%xmm7,%xmm15+	paddd	192-256(%rcx),%xmm4+	paddd	208-256(%rcx),%xmm5+	paddd	224-256(%rcx),%xmm8+	paddd	240-256(%rcx),%xmm9++	movdqa	%xmm6,32(%rsp)+	movdqa	%xmm11,48(%rsp)++	movdqa	%xmm4,%xmm14+	punpckldq	%xmm5,%xmm4+	movdqa	%xmm8,%xmm7+	punpckldq	%xmm9,%xmm8+	punpckhdq	%xmm5,%xmm14+	punpckhdq	%xmm9,%xmm7+	movdqa	%xmm4,%xmm5+	punpcklqdq	%xmm8,%xmm4+	movdqa	%xmm14,%xmm9+	punpcklqdq	%xmm7,%xmm14+	punpckhqdq	%xmm8,%xmm5+	punpckhqdq	%xmm7,%xmm9+	paddd	256-256(%rcx),%xmm0+	paddd	272-256(%rcx),%xmm1+	paddd	288-256(%rcx),%xmm2+	paddd	304-256(%rcx),%xmm3++	movdqa	%xmm0,%xmm8+	punpckldq	%xmm1,%xmm0+	movdqa	%xmm2,%xmm7+	punpckldq	%xmm3,%xmm2+	punpckhdq	%xmm1,%xmm8+	punpckhdq	%xmm3,%xmm7+	movdqa	%xmm0,%xmm1+	punpcklqdq	%xmm2,%xmm0+	movdqa	%xmm8,%xmm3+	punpcklqdq	%xmm7,%xmm8+	punpckhqdq	%xmm2,%xmm1+	punpckhqdq	%xmm7,%xmm3+	cmpq	$256,%rdx+	jb	.Ltail4x++	movdqu	0(%rsi),%xmm6+	movdqu	16(%rsi),%xmm11+	movdqu	32(%rsi),%xmm2+	movdqu	48(%rsi),%xmm7+	pxor	0(%rsp),%xmm6+	pxor	%xmm12,%xmm11+	pxor	%xmm4,%xmm2+	pxor	%xmm0,%xmm7++	movdqu	%xmm6,0(%rdi)+	movdqu	64(%rsi),%xmm6+	movdqu	%xmm11,16(%rdi)+	movdqu	80(%rsi),%xmm11+	movdqu	%xmm2,32(%rdi)+	movdqu	96(%rsi),%xmm2+	movdqu	%xmm7,48(%rdi)+	movdqu	112(%rsi),%xmm7+	leaq	128(%rsi),%rsi+	pxor	16(%rsp),%xmm6+	pxor	%xmm13,%xmm11+	pxor	%xmm5,%xmm2+	pxor	%xmm1,%xmm7++	movdqu	%xmm6,64(%rdi)+	movdqu	0(%rsi),%xmm6+	movdqu	%xmm11,80(%rdi)+	movdqu	16(%rsi),%xmm11+	movdqu	%xmm2,96(%rdi)+	movdqu	32(%rsi),%xmm2+	movdqu	%xmm7,112(%rdi)+	leaq	128(%rdi),%rdi+	movdqu	48(%rsi),%xmm7+	pxor	32(%rsp),%xmm6+	pxor	%xmm10,%xmm11+	pxor	%xmm14,%xmm2+	pxor	%xmm8,%xmm7++	movdqu	%xmm6,0(%rdi)+	movdqu	64(%rsi),%xmm6+	movdqu	%xmm11,16(%rdi)+	movdqu	80(%rsi),%xmm11+	movdqu	%xmm2,32(%rdi)+	movdqu	96(%rsi),%xmm2+	movdqu	%xmm7,48(%rdi)+	movdqu	112(%rsi),%xmm7+	leaq	128(%rsi),%rsi+	pxor	48(%rsp),%xmm6+	pxor	%xmm15,%xmm11+	pxor	%xmm9,%xmm2+	pxor	%xmm3,%xmm7+	movdqu	%xmm6,64(%rdi)+	movdqu	%xmm11,80(%rdi)+	movdqu	%xmm2,96(%rdi)+	movdqu	%xmm7,112(%rdi)+	leaq	128(%rdi),%rdi++	subq	$256,%rdx+	jnz	.Loop_outer4x++	jmp	.Ldone4x++.Ltail4x:+	cmpq	$192,%rdx+	jae	.L192_or_more4x+	cmpq	$128,%rdx+	jae	.L128_or_more4x+	cmpq	$64,%rdx+	jae	.L64_or_more4x+++	xorq	%r9,%r9++	movdqa	%xmm12,16(%rsp)+	movdqa	%xmm4,32(%rsp)+	movdqa	%xmm0,48(%rsp)+	jmp	.Loop_tail4x++.align	32+.L64_or_more4x:+	movdqu	0(%rsi),%xmm6+	movdqu	16(%rsi),%xmm11+	movdqu	32(%rsi),%xmm2+	movdqu	48(%rsi),%xmm7+	pxor	0(%rsp),%xmm6+	pxor	%xmm12,%xmm11+	pxor	%xmm4,%xmm2+	pxor	%xmm0,%xmm7+	movdqu	%xmm6,0(%rdi)+	movdqu	%xmm11,16(%rdi)+	movdqu	%xmm2,32(%rdi)+	movdqu	%xmm7,48(%rdi)+	je	.Ldone4x++	movdqa	16(%rsp),%xmm6+	leaq	64(%rsi),%rsi+	xorq	%r9,%r9+	movdqa	%xmm6,0(%rsp)+	movdqa	%xmm13,16(%rsp)+	leaq	64(%rdi),%rdi+	movdqa	%xmm5,32(%rsp)+	subq	$64,%rdx+	movdqa	%xmm1,48(%rsp)+	jmp	.Loop_tail4x++.align	32+.L128_or_more4x:+	movdqu	0(%rsi),%xmm6+	movdqu	16(%rsi),%xmm11+	movdqu	32(%rsi),%xmm2+	movdqu	48(%rsi),%xmm7+	pxor	0(%rsp),%xmm6+	pxor	%xmm12,%xmm11+	pxor	%xmm4,%xmm2+	pxor	%xmm0,%xmm7++	movdqu	%xmm6,0(%rdi)+	movdqu	64(%rsi),%xmm6+	movdqu	%xmm11,16(%rdi)+	movdqu	80(%rsi),%xmm11+	movdqu	%xmm2,32(%rdi)+	movdqu	96(%rsi),%xmm2+	movdqu	%xmm7,48(%rdi)+	movdqu	112(%rsi),%xmm7+	pxor	16(%rsp),%xmm6+	pxor	%xmm13,%xmm11+	pxor	%xmm5,%xmm2+	pxor	%xmm1,%xmm7+	movdqu	%xmm6,64(%rdi)+	movdqu	%xmm11,80(%rdi)+	movdqu	%xmm2,96(%rdi)+	movdqu	%xmm7,112(%rdi)+	je	.Ldone4x++	movdqa	32(%rsp),%xmm6+	leaq	128(%rsi),%rsi+	xorq	%r9,%r9+	movdqa	%xmm6,0(%rsp)+	movdqa	%xmm10,16(%rsp)+	leaq	128(%rdi),%rdi+	movdqa	%xmm14,32(%rsp)+	subq	$128,%rdx+	movdqa	%xmm8,48(%rsp)+	jmp	.Loop_tail4x++.align	32+.L192_or_more4x:+	movdqu	0(%rsi),%xmm6+	movdqu	16(%rsi),%xmm11+	movdqu	32(%rsi),%xmm2+	movdqu	48(%rsi),%xmm7+	pxor	0(%rsp),%xmm6+	pxor	%xmm12,%xmm11+	pxor	%xmm4,%xmm2+	pxor	%xmm0,%xmm7++	movdqu	%xmm6,0(%rdi)+	movdqu	64(%rsi),%xmm6+	movdqu	%xmm11,16(%rdi)+	movdqu	80(%rsi),%xmm11+	movdqu	%xmm2,32(%rdi)+	movdqu	96(%rsi),%xmm2+	movdqu	%xmm7,48(%rdi)+	movdqu	112(%rsi),%xmm7+	leaq	128(%rsi),%rsi+	pxor	16(%rsp),%xmm6+	pxor	%xmm13,%xmm11+	pxor	%xmm5,%xmm2+	pxor	%xmm1,%xmm7++	movdqu	%xmm6,64(%rdi)+	movdqu	0(%rsi),%xmm6+	movdqu	%xmm11,80(%rdi)+	movdqu	16(%rsi),%xmm11+	movdqu	%xmm2,96(%rdi)+	movdqu	32(%rsi),%xmm2+	movdqu	%xmm7,112(%rdi)+	leaq	128(%rdi),%rdi+	movdqu	48(%rsi),%xmm7+	pxor	32(%rsp),%xmm6+	pxor	%xmm10,%xmm11+	pxor	%xmm14,%xmm2+	pxor	%xmm8,%xmm7+	movdqu	%xmm6,0(%rdi)+	movdqu	%xmm11,16(%rdi)+	movdqu	%xmm2,32(%rdi)+	movdqu	%xmm7,48(%rdi)+	je	.Ldone4x++	movdqa	48(%rsp),%xmm6+	leaq	64(%rsi),%rsi+	xorq	%r9,%r9+	movdqa	%xmm6,0(%rsp)+	movdqa	%xmm15,16(%rsp)+	leaq	64(%rdi),%rdi+	movdqa	%xmm9,32(%rsp)+	subq	$192,%rdx+	movdqa	%xmm3,48(%rsp)++.Loop_tail4x:+	movzbl	(%rsi,%r9,1),%eax+	movzbl	(%rsp,%r9,1),%ecx+	leaq	1(%r9),%r9+	xorl	%ecx,%eax+	movb	%al,-1(%rdi,%r9,1)+	decq	%rdx+	jnz	.Loop_tail4x++.Ldone4x:+	leaq	(%r10),%rsp+.cfi_def_cfa_register	%rsp+.L4x_epilogue:+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_chacha20_asm_4x,.-crypton_chacha20_asm_4x+.type	crypton_chacha20_asm_4xop,@function+.align	32+crypton_chacha20_asm_4xop:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++.Lcrypton_chacha20_asm_4xop:+	movq	%rsp,%r10+.cfi_def_cfa_register	%r10+	subq	$0x140+8,%rsp+	andq	$-16,%rsp+	vzeroupper++	vmovdqa	.Lsigma(%rip),%xmm11+	vmovdqu	(%rcx),%xmm3+	vmovdqu	16(%rcx),%xmm15+	vmovdqu	(%r8),%xmm7+	leaq	256(%rsp),%rcx++	vpshufd	$0x00,%xmm11,%xmm8+	vpshufd	$0x55,%xmm11,%xmm9+	vmovdqa	%xmm8,64(%rsp)+	vpshufd	$0xaa,%xmm11,%xmm10+	vmovdqa	%xmm9,80(%rsp)+	vpshufd	$0xff,%xmm11,%xmm11+	vmovdqa	%xmm10,96(%rsp)+	vmovdqa	%xmm11,112(%rsp)++	vpshufd	$0x00,%xmm3,%xmm0+	vpshufd	$0x55,%xmm3,%xmm1+	vmovdqa	%xmm0,128-256(%rcx)+	vpshufd	$0xaa,%xmm3,%xmm2+	vmovdqa	%xmm1,144-256(%rcx)+	vpshufd	$0xff,%xmm3,%xmm3+	vmovdqa	%xmm2,160-256(%rcx)+	vmovdqa	%xmm3,176-256(%rcx)++	vpshufd	$0x00,%xmm15,%xmm12+	vpshufd	$0x55,%xmm15,%xmm13+	vmovdqa	%xmm12,192-256(%rcx)+	vpshufd	$0xaa,%xmm15,%xmm14+	vmovdqa	%xmm13,208-256(%rcx)+	vpshufd	$0xff,%xmm15,%xmm15+	vmovdqa	%xmm14,224-256(%rcx)+	vmovdqa	%xmm15,240-256(%rcx)++	vpshufd	$0x00,%xmm7,%xmm4+	vpshufd	$0x55,%xmm7,%xmm5+	vpaddd	.Linc(%rip),%xmm4,%xmm4+	vpshufd	$0xaa,%xmm7,%xmm6+	vmovdqa	%xmm5,272-256(%rcx)+	vpshufd	$0xff,%xmm7,%xmm7+	vmovdqa	%xmm6,288-256(%rcx)+	vmovdqa	%xmm7,304-256(%rcx)++	jmp	.Loop_enter4xop++.align	32+.Loop_outer4xop:+	vmovdqa	64(%rsp),%xmm8+	vmovdqa	80(%rsp),%xmm9+	vmovdqa	96(%rsp),%xmm10+	vmovdqa	112(%rsp),%xmm11+	vmovdqa	128-256(%rcx),%xmm0+	vmovdqa	144-256(%rcx),%xmm1+	vmovdqa	160-256(%rcx),%xmm2+	vmovdqa	176-256(%rcx),%xmm3+	vmovdqa	192-256(%rcx),%xmm12+	vmovdqa	208-256(%rcx),%xmm13+	vmovdqa	224-256(%rcx),%xmm14+	vmovdqa	240-256(%rcx),%xmm15+	vmovdqa	256-256(%rcx),%xmm4+	vmovdqa	272-256(%rcx),%xmm5+	vmovdqa	288-256(%rcx),%xmm6+	vmovdqa	304-256(%rcx),%xmm7+	vpaddd	.Lfour(%rip),%xmm4,%xmm4++.Loop_enter4xop:+	movl	$10,%eax+	vmovdqa	%xmm4,256-256(%rcx)+	jmp	.Loop4xop++.align	32+.Loop4xop:+	vpaddd	%xmm0,%xmm8,%xmm8+	vpaddd	%xmm1,%xmm9,%xmm9+	vpaddd	%xmm2,%xmm10,%xmm10+	vpaddd	%xmm3,%xmm11,%xmm11+	vpxor	%xmm4,%xmm8,%xmm4+	vpxor	%xmm5,%xmm9,%xmm5+	vpxor	%xmm6,%xmm10,%xmm6+	vpxor	%xmm7,%xmm11,%xmm7+.byte	143,232,120,194,228,16+.byte	143,232,120,194,237,16+.byte	143,232,120,194,246,16+.byte	143,232,120,194,255,16+	vpaddd	%xmm4,%xmm12,%xmm12+	vpaddd	%xmm5,%xmm13,%xmm13+	vpaddd	%xmm6,%xmm14,%xmm14+	vpaddd	%xmm7,%xmm15,%xmm15+	vpxor	%xmm0,%xmm12,%xmm0+	vpxor	%xmm1,%xmm13,%xmm1+	vpxor	%xmm14,%xmm2,%xmm2+	vpxor	%xmm15,%xmm3,%xmm3+.byte	143,232,120,194,192,12+.byte	143,232,120,194,201,12+.byte	143,232,120,194,210,12+.byte	143,232,120,194,219,12+	vpaddd	%xmm8,%xmm0,%xmm8+	vpaddd	%xmm9,%xmm1,%xmm9+	vpaddd	%xmm2,%xmm10,%xmm10+	vpaddd	%xmm3,%xmm11,%xmm11+	vpxor	%xmm4,%xmm8,%xmm4+	vpxor	%xmm5,%xmm9,%xmm5+	vpxor	%xmm6,%xmm10,%xmm6+	vpxor	%xmm7,%xmm11,%xmm7+.byte	143,232,120,194,228,8+.byte	143,232,120,194,237,8+.byte	143,232,120,194,246,8+.byte	143,232,120,194,255,8+	vpaddd	%xmm4,%xmm12,%xmm12+	vpaddd	%xmm5,%xmm13,%xmm13+	vpaddd	%xmm6,%xmm14,%xmm14+	vpaddd	%xmm7,%xmm15,%xmm15+	vpxor	%xmm0,%xmm12,%xmm0+	vpxor	%xmm1,%xmm13,%xmm1+	vpxor	%xmm14,%xmm2,%xmm2+	vpxor	%xmm15,%xmm3,%xmm3+.byte	143,232,120,194,192,7+.byte	143,232,120,194,201,7+.byte	143,232,120,194,210,7+.byte	143,232,120,194,219,7+	vpaddd	%xmm1,%xmm8,%xmm8+	vpaddd	%xmm2,%xmm9,%xmm9+	vpaddd	%xmm3,%xmm10,%xmm10+	vpaddd	%xmm0,%xmm11,%xmm11+	vpxor	%xmm7,%xmm8,%xmm7+	vpxor	%xmm4,%xmm9,%xmm4+	vpxor	%xmm5,%xmm10,%xmm5+	vpxor	%xmm6,%xmm11,%xmm6+.byte	143,232,120,194,255,16+.byte	143,232,120,194,228,16+.byte	143,232,120,194,237,16+.byte	143,232,120,194,246,16+	vpaddd	%xmm7,%xmm14,%xmm14+	vpaddd	%xmm4,%xmm15,%xmm15+	vpaddd	%xmm5,%xmm12,%xmm12+	vpaddd	%xmm6,%xmm13,%xmm13+	vpxor	%xmm1,%xmm14,%xmm1+	vpxor	%xmm2,%xmm15,%xmm2+	vpxor	%xmm12,%xmm3,%xmm3+	vpxor	%xmm13,%xmm0,%xmm0+.byte	143,232,120,194,201,12+.byte	143,232,120,194,210,12+.byte	143,232,120,194,219,12+.byte	143,232,120,194,192,12+	vpaddd	%xmm8,%xmm1,%xmm8+	vpaddd	%xmm9,%xmm2,%xmm9+	vpaddd	%xmm3,%xmm10,%xmm10+	vpaddd	%xmm0,%xmm11,%xmm11+	vpxor	%xmm7,%xmm8,%xmm7+	vpxor	%xmm4,%xmm9,%xmm4+	vpxor	%xmm5,%xmm10,%xmm5+	vpxor	%xmm6,%xmm11,%xmm6+.byte	143,232,120,194,255,8+.byte	143,232,120,194,228,8+.byte	143,232,120,194,237,8+.byte	143,232,120,194,246,8+	vpaddd	%xmm7,%xmm14,%xmm14+	vpaddd	%xmm4,%xmm15,%xmm15+	vpaddd	%xmm5,%xmm12,%xmm12+	vpaddd	%xmm6,%xmm13,%xmm13+	vpxor	%xmm1,%xmm14,%xmm1+	vpxor	%xmm2,%xmm15,%xmm2+	vpxor	%xmm12,%xmm3,%xmm3+	vpxor	%xmm13,%xmm0,%xmm0+.byte	143,232,120,194,201,7+.byte	143,232,120,194,210,7+.byte	143,232,120,194,219,7+.byte	143,232,120,194,192,7+	decl	%eax+	jnz	.Loop4xop++	vpaddd	64(%rsp),%xmm8,%xmm8+	vpaddd	80(%rsp),%xmm9,%xmm9+	vpaddd	96(%rsp),%xmm10,%xmm10+	vpaddd	112(%rsp),%xmm11,%xmm11++	vmovdqa	%xmm14,32(%rsp)+	vmovdqa	%xmm15,48(%rsp)++	vpunpckldq	%xmm9,%xmm8,%xmm14+	vpunpckldq	%xmm11,%xmm10,%xmm15+	vpunpckhdq	%xmm9,%xmm8,%xmm8+	vpunpckhdq	%xmm11,%xmm10,%xmm10+	vpunpcklqdq	%xmm15,%xmm14,%xmm9+	vpunpckhqdq	%xmm15,%xmm14,%xmm14+	vpunpcklqdq	%xmm10,%xmm8,%xmm11+	vpunpckhqdq	%xmm10,%xmm8,%xmm8+	vpaddd	128-256(%rcx),%xmm0,%xmm0+	vpaddd	144-256(%rcx),%xmm1,%xmm1+	vpaddd	160-256(%rcx),%xmm2,%xmm2+	vpaddd	176-256(%rcx),%xmm3,%xmm3++	vmovdqa	%xmm9,0(%rsp)+	vmovdqa	%xmm14,16(%rsp)+	vmovdqa	32(%rsp),%xmm9+	vmovdqa	48(%rsp),%xmm14++	vpunpckldq	%xmm1,%xmm0,%xmm10+	vpunpckldq	%xmm3,%xmm2,%xmm15+	vpunpckhdq	%xmm1,%xmm0,%xmm0+	vpunpckhdq	%xmm3,%xmm2,%xmm2+	vpunpcklqdq	%xmm15,%xmm10,%xmm1+	vpunpckhqdq	%xmm15,%xmm10,%xmm10+	vpunpcklqdq	%xmm2,%xmm0,%xmm3+	vpunpckhqdq	%xmm2,%xmm0,%xmm0+	vpaddd	192-256(%rcx),%xmm12,%xmm12+	vpaddd	208-256(%rcx),%xmm13,%xmm13+	vpaddd	224-256(%rcx),%xmm9,%xmm9+	vpaddd	240-256(%rcx),%xmm14,%xmm14++	vpunpckldq	%xmm13,%xmm12,%xmm2+	vpunpckldq	%xmm14,%xmm9,%xmm15+	vpunpckhdq	%xmm13,%xmm12,%xmm12+	vpunpckhdq	%xmm14,%xmm9,%xmm9+	vpunpcklqdq	%xmm15,%xmm2,%xmm13+	vpunpckhqdq	%xmm15,%xmm2,%xmm2+	vpunpcklqdq	%xmm9,%xmm12,%xmm14+	vpunpckhqdq	%xmm9,%xmm12,%xmm12+	vpaddd	256-256(%rcx),%xmm4,%xmm4+	vpaddd	272-256(%rcx),%xmm5,%xmm5+	vpaddd	288-256(%rcx),%xmm6,%xmm6+	vpaddd	304-256(%rcx),%xmm7,%xmm7++	vpunpckldq	%xmm5,%xmm4,%xmm9+	vpunpckldq	%xmm7,%xmm6,%xmm15+	vpunpckhdq	%xmm5,%xmm4,%xmm4+	vpunpckhdq	%xmm7,%xmm6,%xmm6+	vpunpcklqdq	%xmm15,%xmm9,%xmm5+	vpunpckhqdq	%xmm15,%xmm9,%xmm9+	vpunpcklqdq	%xmm6,%xmm4,%xmm7+	vpunpckhqdq	%xmm6,%xmm4,%xmm4+	vmovdqa	0(%rsp),%xmm6+	vmovdqa	16(%rsp),%xmm15++	cmpq	$256,%rdx+	jb	.Ltail4xop++	vpxor	0(%rsi),%xmm6,%xmm6+	vpxor	16(%rsi),%xmm1,%xmm1+	vpxor	32(%rsi),%xmm13,%xmm13+	vpxor	48(%rsi),%xmm5,%xmm5+	vpxor	64(%rsi),%xmm15,%xmm15+	vpxor	80(%rsi),%xmm10,%xmm10+	vpxor	96(%rsi),%xmm2,%xmm2+	vpxor	112(%rsi),%xmm9,%xmm9+	leaq	128(%rsi),%rsi+	vpxor	0(%rsi),%xmm11,%xmm11+	vpxor	16(%rsi),%xmm3,%xmm3+	vpxor	32(%rsi),%xmm14,%xmm14+	vpxor	48(%rsi),%xmm7,%xmm7+	vpxor	64(%rsi),%xmm8,%xmm8+	vpxor	80(%rsi),%xmm0,%xmm0+	vpxor	96(%rsi),%xmm12,%xmm12+	vpxor	112(%rsi),%xmm4,%xmm4+	leaq	128(%rsi),%rsi++	vmovdqu	%xmm6,0(%rdi)+	vmovdqu	%xmm1,16(%rdi)+	vmovdqu	%xmm13,32(%rdi)+	vmovdqu	%xmm5,48(%rdi)+	vmovdqu	%xmm15,64(%rdi)+	vmovdqu	%xmm10,80(%rdi)+	vmovdqu	%xmm2,96(%rdi)+	vmovdqu	%xmm9,112(%rdi)+	leaq	128(%rdi),%rdi+	vmovdqu	%xmm11,0(%rdi)+	vmovdqu	%xmm3,16(%rdi)+	vmovdqu	%xmm14,32(%rdi)+	vmovdqu	%xmm7,48(%rdi)+	vmovdqu	%xmm8,64(%rdi)+	vmovdqu	%xmm0,80(%rdi)+	vmovdqu	%xmm12,96(%rdi)+	vmovdqu	%xmm4,112(%rdi)+	leaq	128(%rdi),%rdi++	subq	$256,%rdx+	jnz	.Loop_outer4xop++	jmp	.Ldone4xop++.align	32+.Ltail4xop:+	cmpq	$192,%rdx+	jae	.L192_or_more4xop+	cmpq	$128,%rdx+	jae	.L128_or_more4xop+	cmpq	$64,%rdx+	jae	.L64_or_more4xop++	xorq	%r9,%r9+	vmovdqa	%xmm6,0(%rsp)+	vmovdqa	%xmm1,16(%rsp)+	vmovdqa	%xmm13,32(%rsp)+	vmovdqa	%xmm5,48(%rsp)+	jmp	.Loop_tail4xop++.align	32+.L64_or_more4xop:+	vpxor	0(%rsi),%xmm6,%xmm6+	vpxor	16(%rsi),%xmm1,%xmm1+	vpxor	32(%rsi),%xmm13,%xmm13+	vpxor	48(%rsi),%xmm5,%xmm5+	vmovdqu	%xmm6,0(%rdi)+	vmovdqu	%xmm1,16(%rdi)+	vmovdqu	%xmm13,32(%rdi)+	vmovdqu	%xmm5,48(%rdi)+	je	.Ldone4xop++	leaq	64(%rsi),%rsi+	vmovdqa	%xmm15,0(%rsp)+	xorq	%r9,%r9+	vmovdqa	%xmm10,16(%rsp)+	leaq	64(%rdi),%rdi+	vmovdqa	%xmm2,32(%rsp)+	subq	$64,%rdx+	vmovdqa	%xmm9,48(%rsp)+	jmp	.Loop_tail4xop++.align	32+.L128_or_more4xop:+	vpxor	0(%rsi),%xmm6,%xmm6+	vpxor	16(%rsi),%xmm1,%xmm1+	vpxor	32(%rsi),%xmm13,%xmm13+	vpxor	48(%rsi),%xmm5,%xmm5+	vpxor	64(%rsi),%xmm15,%xmm15+	vpxor	80(%rsi),%xmm10,%xmm10+	vpxor	96(%rsi),%xmm2,%xmm2+	vpxor	112(%rsi),%xmm9,%xmm9++	vmovdqu	%xmm6,0(%rdi)+	vmovdqu	%xmm1,16(%rdi)+	vmovdqu	%xmm13,32(%rdi)+	vmovdqu	%xmm5,48(%rdi)+	vmovdqu	%xmm15,64(%rdi)+	vmovdqu	%xmm10,80(%rdi)+	vmovdqu	%xmm2,96(%rdi)+	vmovdqu	%xmm9,112(%rdi)+	je	.Ldone4xop++	leaq	128(%rsi),%rsi+	vmovdqa	%xmm11,0(%rsp)+	xorq	%r9,%r9+	vmovdqa	%xmm3,16(%rsp)+	leaq	128(%rdi),%rdi+	vmovdqa	%xmm14,32(%rsp)+	subq	$128,%rdx+	vmovdqa	%xmm7,48(%rsp)+	jmp	.Loop_tail4xop++.align	32+.L192_or_more4xop:+	vpxor	0(%rsi),%xmm6,%xmm6+	vpxor	16(%rsi),%xmm1,%xmm1+	vpxor	32(%rsi),%xmm13,%xmm13+	vpxor	48(%rsi),%xmm5,%xmm5+	vpxor	64(%rsi),%xmm15,%xmm15+	vpxor	80(%rsi),%xmm10,%xmm10+	vpxor	96(%rsi),%xmm2,%xmm2+	vpxor	112(%rsi),%xmm9,%xmm9+	leaq	128(%rsi),%rsi+	vpxor	0(%rsi),%xmm11,%xmm11+	vpxor	16(%rsi),%xmm3,%xmm3+	vpxor	32(%rsi),%xmm14,%xmm14+	vpxor	48(%rsi),%xmm7,%xmm7++	vmovdqu	%xmm6,0(%rdi)+	vmovdqu	%xmm1,16(%rdi)+	vmovdqu	%xmm13,32(%rdi)+	vmovdqu	%xmm5,48(%rdi)+	vmovdqu	%xmm15,64(%rdi)+	vmovdqu	%xmm10,80(%rdi)+	vmovdqu	%xmm2,96(%rdi)+	vmovdqu	%xmm9,112(%rdi)+	leaq	128(%rdi),%rdi+	vmovdqu	%xmm11,0(%rdi)+	vmovdqu	%xmm3,16(%rdi)+	vmovdqu	%xmm14,32(%rdi)+	vmovdqu	%xmm7,48(%rdi)+	je	.Ldone4xop++	leaq	64(%rsi),%rsi+	vmovdqa	%xmm8,0(%rsp)+	xorq	%r9,%r9+	vmovdqa	%xmm0,16(%rsp)+	leaq	64(%rdi),%rdi+	vmovdqa	%xmm12,32(%rsp)+	subq	$192,%rdx+	vmovdqa	%xmm4,48(%rsp)++.Loop_tail4xop:+	movzbl	(%rsi,%r9,1),%eax+	movzbl	(%rsp,%r9,1),%ecx+	leaq	1(%r9),%r9+	xorl	%ecx,%eax+	movb	%al,-1(%rdi,%r9,1)+	decq	%rdx+	jnz	.Loop_tail4xop++.Ldone4xop:+	vzeroupper+	leaq	(%r10),%rsp+.cfi_def_cfa_register	%rsp+.L4xop_epilogue:+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_chacha20_asm_4xop,.-crypton_chacha20_asm_4xop+.type	crypton_chacha20_asm_avx2,@function+.align	32+crypton_chacha20_asm_avx2:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++.Lcrypton_chacha20_asm_8x:+	movq	%rsp,%r10+.cfi_def_cfa_register	%r10+	subq	$0x280+8,%rsp+	andq	$-32,%rsp+	vzeroupper+++++++++++	vbroadcasti128	.Lsigma(%rip),%ymm11+	vbroadcasti128	(%rcx),%ymm3+	vbroadcasti128	16(%rcx),%ymm15+	vbroadcasti128	(%r8),%ymm7+	leaq	256(%rsp),%rcx+	leaq	512(%rsp),%rax+	leaq	.Lrot16(%rip),%r9+	leaq	.Lrot24(%rip),%r11++	vpshufd	$0x00,%ymm11,%ymm8+	vpshufd	$0x55,%ymm11,%ymm9+	vmovdqa	%ymm8,128-256(%rcx)+	vpshufd	$0xaa,%ymm11,%ymm10+	vmovdqa	%ymm9,160-256(%rcx)+	vpshufd	$0xff,%ymm11,%ymm11+	vmovdqa	%ymm10,192-256(%rcx)+	vmovdqa	%ymm11,224-256(%rcx)++	vpshufd	$0x00,%ymm3,%ymm0+	vpshufd	$0x55,%ymm3,%ymm1+	vmovdqa	%ymm0,256-256(%rcx)+	vpshufd	$0xaa,%ymm3,%ymm2+	vmovdqa	%ymm1,288-256(%rcx)+	vpshufd	$0xff,%ymm3,%ymm3+	vmovdqa	%ymm2,320-256(%rcx)+	vmovdqa	%ymm3,352-256(%rcx)++	vpshufd	$0x00,%ymm15,%ymm12+	vpshufd	$0x55,%ymm15,%ymm13+	vmovdqa	%ymm12,384-512(%rax)+	vpshufd	$0xaa,%ymm15,%ymm14+	vmovdqa	%ymm13,416-512(%rax)+	vpshufd	$0xff,%ymm15,%ymm15+	vmovdqa	%ymm14,448-512(%rax)+	vmovdqa	%ymm15,480-512(%rax)++	vpshufd	$0x00,%ymm7,%ymm4+	vpshufd	$0x55,%ymm7,%ymm5+	vpaddd	.Lincy(%rip),%ymm4,%ymm4+	vpshufd	$0xaa,%ymm7,%ymm6+	vmovdqa	%ymm5,544-512(%rax)+	vpshufd	$0xff,%ymm7,%ymm7+	vmovdqa	%ymm6,576-512(%rax)+	vmovdqa	%ymm7,608-512(%rax)++	jmp	.Loop_enter8x++.align	32+.Loop_outer8x:+	vmovdqa	128-256(%rcx),%ymm8+	vmovdqa	160-256(%rcx),%ymm9+	vmovdqa	192-256(%rcx),%ymm10+	vmovdqa	224-256(%rcx),%ymm11+	vmovdqa	256-256(%rcx),%ymm0+	vmovdqa	288-256(%rcx),%ymm1+	vmovdqa	320-256(%rcx),%ymm2+	vmovdqa	352-256(%rcx),%ymm3+	vmovdqa	384-512(%rax),%ymm12+	vmovdqa	416-512(%rax),%ymm13+	vmovdqa	448-512(%rax),%ymm14+	vmovdqa	480-512(%rax),%ymm15+	vmovdqa	512-512(%rax),%ymm4+	vmovdqa	544-512(%rax),%ymm5+	vmovdqa	576-512(%rax),%ymm6+	vmovdqa	608-512(%rax),%ymm7+	vpaddd	.Leight(%rip),%ymm4,%ymm4++.Loop_enter8x:+	vmovdqa	%ymm14,64(%rsp)+	vmovdqa	%ymm15,96(%rsp)+	vbroadcasti128	(%r9),%ymm15+	vmovdqa	%ymm4,512-512(%rax)+	movl	$10,%eax+	jmp	.Loop8x++.align	32+.Loop8x:+	vpaddd	%ymm0,%ymm8,%ymm8+	vpxor	%ymm4,%ymm8,%ymm4+	vpshufb	%ymm15,%ymm4,%ymm4+	vpaddd	%ymm1,%ymm9,%ymm9+	vpxor	%ymm5,%ymm9,%ymm5+	vpshufb	%ymm15,%ymm5,%ymm5+	vpaddd	%ymm4,%ymm12,%ymm12+	vpxor	%ymm0,%ymm12,%ymm0+	vpslld	$12,%ymm0,%ymm14+	vpsrld	$20,%ymm0,%ymm0+	vpor	%ymm0,%ymm14,%ymm0+	vbroadcasti128	(%r11),%ymm14+	vpaddd	%ymm5,%ymm13,%ymm13+	vpxor	%ymm1,%ymm13,%ymm1+	vpslld	$12,%ymm1,%ymm15+	vpsrld	$20,%ymm1,%ymm1+	vpor	%ymm1,%ymm15,%ymm1+	vpaddd	%ymm0,%ymm8,%ymm8+	vpxor	%ymm4,%ymm8,%ymm4+	vpshufb	%ymm14,%ymm4,%ymm4+	vpaddd	%ymm1,%ymm9,%ymm9+	vpxor	%ymm5,%ymm9,%ymm5+	vpshufb	%ymm14,%ymm5,%ymm5+	vpaddd	%ymm4,%ymm12,%ymm12+	vpxor	%ymm0,%ymm12,%ymm0+	vpslld	$7,%ymm0,%ymm15+	vpsrld	$25,%ymm0,%ymm0+	vpor	%ymm0,%ymm15,%ymm0+	vbroadcasti128	(%r9),%ymm15+	vpaddd	%ymm5,%ymm13,%ymm13+	vpxor	%ymm1,%ymm13,%ymm1+	vpslld	$7,%ymm1,%ymm14+	vpsrld	$25,%ymm1,%ymm1+	vpor	%ymm1,%ymm14,%ymm1+	vmovdqa	%ymm12,0(%rsp)+	vmovdqa	%ymm13,32(%rsp)+	vmovdqa	64(%rsp),%ymm12+	vmovdqa	96(%rsp),%ymm13+	vpaddd	%ymm2,%ymm10,%ymm10+	vpxor	%ymm6,%ymm10,%ymm6+	vpshufb	%ymm15,%ymm6,%ymm6+	vpaddd	%ymm3,%ymm11,%ymm11+	vpxor	%ymm7,%ymm11,%ymm7+	vpshufb	%ymm15,%ymm7,%ymm7+	vpaddd	%ymm6,%ymm12,%ymm12+	vpxor	%ymm2,%ymm12,%ymm2+	vpslld	$12,%ymm2,%ymm14+	vpsrld	$20,%ymm2,%ymm2+	vpor	%ymm2,%ymm14,%ymm2+	vbroadcasti128	(%r11),%ymm14+	vpaddd	%ymm7,%ymm13,%ymm13+	vpxor	%ymm3,%ymm13,%ymm3+	vpslld	$12,%ymm3,%ymm15+	vpsrld	$20,%ymm3,%ymm3+	vpor	%ymm3,%ymm15,%ymm3+	vpaddd	%ymm2,%ymm10,%ymm10+	vpxor	%ymm6,%ymm10,%ymm6+	vpshufb	%ymm14,%ymm6,%ymm6+	vpaddd	%ymm3,%ymm11,%ymm11+	vpxor	%ymm7,%ymm11,%ymm7+	vpshufb	%ymm14,%ymm7,%ymm7+	vpaddd	%ymm6,%ymm12,%ymm12+	vpxor	%ymm2,%ymm12,%ymm2+	vpslld	$7,%ymm2,%ymm15+	vpsrld	$25,%ymm2,%ymm2+	vpor	%ymm2,%ymm15,%ymm2+	vbroadcasti128	(%r9),%ymm15+	vpaddd	%ymm7,%ymm13,%ymm13+	vpxor	%ymm3,%ymm13,%ymm3+	vpslld	$7,%ymm3,%ymm14+	vpsrld	$25,%ymm3,%ymm3+	vpor	%ymm3,%ymm14,%ymm3+	vpaddd	%ymm1,%ymm8,%ymm8+	vpxor	%ymm7,%ymm8,%ymm7+	vpshufb	%ymm15,%ymm7,%ymm7+	vpaddd	%ymm2,%ymm9,%ymm9+	vpxor	%ymm4,%ymm9,%ymm4+	vpshufb	%ymm15,%ymm4,%ymm4+	vpaddd	%ymm7,%ymm12,%ymm12+	vpxor	%ymm1,%ymm12,%ymm1+	vpslld	$12,%ymm1,%ymm14+	vpsrld	$20,%ymm1,%ymm1+	vpor	%ymm1,%ymm14,%ymm1+	vbroadcasti128	(%r11),%ymm14+	vpaddd	%ymm4,%ymm13,%ymm13+	vpxor	%ymm2,%ymm13,%ymm2+	vpslld	$12,%ymm2,%ymm15+	vpsrld	$20,%ymm2,%ymm2+	vpor	%ymm2,%ymm15,%ymm2+	vpaddd	%ymm1,%ymm8,%ymm8+	vpxor	%ymm7,%ymm8,%ymm7+	vpshufb	%ymm14,%ymm7,%ymm7+	vpaddd	%ymm2,%ymm9,%ymm9+	vpxor	%ymm4,%ymm9,%ymm4+	vpshufb	%ymm14,%ymm4,%ymm4+	vpaddd	%ymm7,%ymm12,%ymm12+	vpxor	%ymm1,%ymm12,%ymm1+	vpslld	$7,%ymm1,%ymm15+	vpsrld	$25,%ymm1,%ymm1+	vpor	%ymm1,%ymm15,%ymm1+	vbroadcasti128	(%r9),%ymm15+	vpaddd	%ymm4,%ymm13,%ymm13+	vpxor	%ymm2,%ymm13,%ymm2+	vpslld	$7,%ymm2,%ymm14+	vpsrld	$25,%ymm2,%ymm2+	vpor	%ymm2,%ymm14,%ymm2+	vmovdqa	%ymm12,64(%rsp)+	vmovdqa	%ymm13,96(%rsp)+	vmovdqa	0(%rsp),%ymm12+	vmovdqa	32(%rsp),%ymm13+	vpaddd	%ymm3,%ymm10,%ymm10+	vpxor	%ymm5,%ymm10,%ymm5+	vpshufb	%ymm15,%ymm5,%ymm5+	vpaddd	%ymm0,%ymm11,%ymm11+	vpxor	%ymm6,%ymm11,%ymm6+	vpshufb	%ymm15,%ymm6,%ymm6+	vpaddd	%ymm5,%ymm12,%ymm12+	vpxor	%ymm3,%ymm12,%ymm3+	vpslld	$12,%ymm3,%ymm14+	vpsrld	$20,%ymm3,%ymm3+	vpor	%ymm3,%ymm14,%ymm3+	vbroadcasti128	(%r11),%ymm14+	vpaddd	%ymm6,%ymm13,%ymm13+	vpxor	%ymm0,%ymm13,%ymm0+	vpslld	$12,%ymm0,%ymm15+	vpsrld	$20,%ymm0,%ymm0+	vpor	%ymm0,%ymm15,%ymm0+	vpaddd	%ymm3,%ymm10,%ymm10+	vpxor	%ymm5,%ymm10,%ymm5+	vpshufb	%ymm14,%ymm5,%ymm5+	vpaddd	%ymm0,%ymm11,%ymm11+	vpxor	%ymm6,%ymm11,%ymm6+	vpshufb	%ymm14,%ymm6,%ymm6+	vpaddd	%ymm5,%ymm12,%ymm12+	vpxor	%ymm3,%ymm12,%ymm3+	vpslld	$7,%ymm3,%ymm15+	vpsrld	$25,%ymm3,%ymm3+	vpor	%ymm3,%ymm15,%ymm3+	vbroadcasti128	(%r9),%ymm15+	vpaddd	%ymm6,%ymm13,%ymm13+	vpxor	%ymm0,%ymm13,%ymm0+	vpslld	$7,%ymm0,%ymm14+	vpsrld	$25,%ymm0,%ymm0+	vpor	%ymm0,%ymm14,%ymm0+	decl	%eax+	jnz	.Loop8x++	leaq	512(%rsp),%rax+	vpaddd	128-256(%rcx),%ymm8,%ymm8+	vpaddd	160-256(%rcx),%ymm9,%ymm9+	vpaddd	192-256(%rcx),%ymm10,%ymm10+	vpaddd	224-256(%rcx),%ymm11,%ymm11++	vpunpckldq	%ymm9,%ymm8,%ymm14+	vpunpckldq	%ymm11,%ymm10,%ymm15+	vpunpckhdq	%ymm9,%ymm8,%ymm8+	vpunpckhdq	%ymm11,%ymm10,%ymm10+	vpunpcklqdq	%ymm15,%ymm14,%ymm9+	vpunpckhqdq	%ymm15,%ymm14,%ymm14+	vpunpcklqdq	%ymm10,%ymm8,%ymm11+	vpunpckhqdq	%ymm10,%ymm8,%ymm8+	vpaddd	256-256(%rcx),%ymm0,%ymm0+	vpaddd	288-256(%rcx),%ymm1,%ymm1+	vpaddd	320-256(%rcx),%ymm2,%ymm2+	vpaddd	352-256(%rcx),%ymm3,%ymm3++	vpunpckldq	%ymm1,%ymm0,%ymm10+	vpunpckldq	%ymm3,%ymm2,%ymm15+	vpunpckhdq	%ymm1,%ymm0,%ymm0+	vpunpckhdq	%ymm3,%ymm2,%ymm2+	vpunpcklqdq	%ymm15,%ymm10,%ymm1+	vpunpckhqdq	%ymm15,%ymm10,%ymm10+	vpunpcklqdq	%ymm2,%ymm0,%ymm3+	vpunpckhqdq	%ymm2,%ymm0,%ymm0+	vperm2i128	$0x20,%ymm1,%ymm9,%ymm15+	vperm2i128	$0x31,%ymm1,%ymm9,%ymm1+	vperm2i128	$0x20,%ymm10,%ymm14,%ymm9+	vperm2i128	$0x31,%ymm10,%ymm14,%ymm10+	vperm2i128	$0x20,%ymm3,%ymm11,%ymm14+	vperm2i128	$0x31,%ymm3,%ymm11,%ymm3+	vperm2i128	$0x20,%ymm0,%ymm8,%ymm11+	vperm2i128	$0x31,%ymm0,%ymm8,%ymm0+	vmovdqa	%ymm15,0(%rsp)+	vmovdqa	%ymm9,32(%rsp)+	vmovdqa	64(%rsp),%ymm15+	vmovdqa	96(%rsp),%ymm9++	vpaddd	384-512(%rax),%ymm12,%ymm12+	vpaddd	416-512(%rax),%ymm13,%ymm13+	vpaddd	448-512(%rax),%ymm15,%ymm15+	vpaddd	480-512(%rax),%ymm9,%ymm9++	vpunpckldq	%ymm13,%ymm12,%ymm2+	vpunpckldq	%ymm9,%ymm15,%ymm8+	vpunpckhdq	%ymm13,%ymm12,%ymm12+	vpunpckhdq	%ymm9,%ymm15,%ymm15+	vpunpcklqdq	%ymm8,%ymm2,%ymm13+	vpunpckhqdq	%ymm8,%ymm2,%ymm2+	vpunpcklqdq	%ymm15,%ymm12,%ymm9+	vpunpckhqdq	%ymm15,%ymm12,%ymm12+	vpaddd	512-512(%rax),%ymm4,%ymm4+	vpaddd	544-512(%rax),%ymm5,%ymm5+	vpaddd	576-512(%rax),%ymm6,%ymm6+	vpaddd	608-512(%rax),%ymm7,%ymm7++	vpunpckldq	%ymm5,%ymm4,%ymm15+	vpunpckldq	%ymm7,%ymm6,%ymm8+	vpunpckhdq	%ymm5,%ymm4,%ymm4+	vpunpckhdq	%ymm7,%ymm6,%ymm6+	vpunpcklqdq	%ymm8,%ymm15,%ymm5+	vpunpckhqdq	%ymm8,%ymm15,%ymm15+	vpunpcklqdq	%ymm6,%ymm4,%ymm7+	vpunpckhqdq	%ymm6,%ymm4,%ymm4+	vperm2i128	$0x20,%ymm5,%ymm13,%ymm8+	vperm2i128	$0x31,%ymm5,%ymm13,%ymm5+	vperm2i128	$0x20,%ymm15,%ymm2,%ymm13+	vperm2i128	$0x31,%ymm15,%ymm2,%ymm15+	vperm2i128	$0x20,%ymm7,%ymm9,%ymm2+	vperm2i128	$0x31,%ymm7,%ymm9,%ymm7+	vperm2i128	$0x20,%ymm4,%ymm12,%ymm9+	vperm2i128	$0x31,%ymm4,%ymm12,%ymm4+	vmovdqa	0(%rsp),%ymm6+	vmovdqa	32(%rsp),%ymm12++	cmpq	$512,%rdx+	jb	.Ltail8x++	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	leaq	128(%rsi),%rsi+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	leaq	128(%rdi),%rdi++	vpxor	0(%rsi),%ymm12,%ymm12+	vpxor	32(%rsi),%ymm13,%ymm13+	vpxor	64(%rsi),%ymm10,%ymm10+	vpxor	96(%rsi),%ymm15,%ymm15+	leaq	128(%rsi),%rsi+	vmovdqu	%ymm12,0(%rdi)+	vmovdqu	%ymm13,32(%rdi)+	vmovdqu	%ymm10,64(%rdi)+	vmovdqu	%ymm15,96(%rdi)+	leaq	128(%rdi),%rdi++	vpxor	0(%rsi),%ymm14,%ymm14+	vpxor	32(%rsi),%ymm2,%ymm2+	vpxor	64(%rsi),%ymm3,%ymm3+	vpxor	96(%rsi),%ymm7,%ymm7+	leaq	128(%rsi),%rsi+	vmovdqu	%ymm14,0(%rdi)+	vmovdqu	%ymm2,32(%rdi)+	vmovdqu	%ymm3,64(%rdi)+	vmovdqu	%ymm7,96(%rdi)+	leaq	128(%rdi),%rdi++	vpxor	0(%rsi),%ymm11,%ymm11+	vpxor	32(%rsi),%ymm9,%ymm9+	vpxor	64(%rsi),%ymm0,%ymm0+	vpxor	96(%rsi),%ymm4,%ymm4+	leaq	128(%rsi),%rsi+	vmovdqu	%ymm11,0(%rdi)+	vmovdqu	%ymm9,32(%rdi)+	vmovdqu	%ymm0,64(%rdi)+	vmovdqu	%ymm4,96(%rdi)+	leaq	128(%rdi),%rdi++	subq	$512,%rdx+	jnz	.Loop_outer8x++	jmp	.Ldone8x++.Ltail8x:+	cmpq	$448,%rdx+	jae	.L448_or_more8x+	cmpq	$384,%rdx+	jae	.L384_or_more8x+	cmpq	$320,%rdx+	jae	.L320_or_more8x+	cmpq	$256,%rdx+	jae	.L256_or_more8x+	cmpq	$192,%rdx+	jae	.L192_or_more8x+	cmpq	$128,%rdx+	jae	.L128_or_more8x+	cmpq	$64,%rdx+	jae	.L64_or_more8x++	xorq	%r9,%r9+	vmovdqa	%ymm6,0(%rsp)+	vmovdqa	%ymm8,32(%rsp)+	jmp	.Loop_tail8x++.align	32+.L64_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	je	.Ldone8x++	leaq	64(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm1,0(%rsp)+	leaq	64(%rdi),%rdi+	subq	$64,%rdx+	vmovdqa	%ymm5,32(%rsp)+	jmp	.Loop_tail8x++.align	32+.L128_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	je	.Ldone8x++	leaq	128(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm12,0(%rsp)+	leaq	128(%rdi),%rdi+	subq	$128,%rdx+	vmovdqa	%ymm13,32(%rsp)+	jmp	.Loop_tail8x++.align	32+.L192_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	je	.Ldone8x++	leaq	192(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm10,0(%rsp)+	leaq	192(%rdi),%rdi+	subq	$192,%rdx+	vmovdqa	%ymm15,32(%rsp)+	jmp	.Loop_tail8x++.align	32+.L256_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vpxor	192(%rsi),%ymm10,%ymm10+	vpxor	224(%rsi),%ymm15,%ymm15+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	vmovdqu	%ymm10,192(%rdi)+	vmovdqu	%ymm15,224(%rdi)+	je	.Ldone8x++	leaq	256(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm14,0(%rsp)+	leaq	256(%rdi),%rdi+	subq	$256,%rdx+	vmovdqa	%ymm2,32(%rsp)+	jmp	.Loop_tail8x++.align	32+.L320_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vpxor	192(%rsi),%ymm10,%ymm10+	vpxor	224(%rsi),%ymm15,%ymm15+	vpxor	256(%rsi),%ymm14,%ymm14+	vpxor	288(%rsi),%ymm2,%ymm2+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	vmovdqu	%ymm10,192(%rdi)+	vmovdqu	%ymm15,224(%rdi)+	vmovdqu	%ymm14,256(%rdi)+	vmovdqu	%ymm2,288(%rdi)+	je	.Ldone8x++	leaq	320(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm3,0(%rsp)+	leaq	320(%rdi),%rdi+	subq	$320,%rdx+	vmovdqa	%ymm7,32(%rsp)+	jmp	.Loop_tail8x++.align	32+.L384_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vpxor	192(%rsi),%ymm10,%ymm10+	vpxor	224(%rsi),%ymm15,%ymm15+	vpxor	256(%rsi),%ymm14,%ymm14+	vpxor	288(%rsi),%ymm2,%ymm2+	vpxor	320(%rsi),%ymm3,%ymm3+	vpxor	352(%rsi),%ymm7,%ymm7+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	vmovdqu	%ymm10,192(%rdi)+	vmovdqu	%ymm15,224(%rdi)+	vmovdqu	%ymm14,256(%rdi)+	vmovdqu	%ymm2,288(%rdi)+	vmovdqu	%ymm3,320(%rdi)+	vmovdqu	%ymm7,352(%rdi)+	je	.Ldone8x++	leaq	384(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm11,0(%rsp)+	leaq	384(%rdi),%rdi+	subq	$384,%rdx+	vmovdqa	%ymm9,32(%rsp)+	jmp	.Loop_tail8x++.align	32+.L448_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vpxor	192(%rsi),%ymm10,%ymm10+	vpxor	224(%rsi),%ymm15,%ymm15+	vpxor	256(%rsi),%ymm14,%ymm14+	vpxor	288(%rsi),%ymm2,%ymm2+	vpxor	320(%rsi),%ymm3,%ymm3+	vpxor	352(%rsi),%ymm7,%ymm7+	vpxor	384(%rsi),%ymm11,%ymm11+	vpxor	416(%rsi),%ymm9,%ymm9+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	vmovdqu	%ymm10,192(%rdi)+	vmovdqu	%ymm15,224(%rdi)+	vmovdqu	%ymm14,256(%rdi)+	vmovdqu	%ymm2,288(%rdi)+	vmovdqu	%ymm3,320(%rdi)+	vmovdqu	%ymm7,352(%rdi)+	vmovdqu	%ymm11,384(%rdi)+	vmovdqu	%ymm9,416(%rdi)+	je	.Ldone8x++	leaq	448(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm0,0(%rsp)+	leaq	448(%rdi),%rdi+	subq	$448,%rdx+	vmovdqa	%ymm4,32(%rsp)++.Loop_tail8x:+	movzbl	(%rsi,%r9,1),%eax+	movzbl	(%rsp,%r9,1),%ecx+	leaq	1(%r9),%r9+	xorl	%ecx,%eax+	movb	%al,-1(%rdi,%r9,1)+	decq	%rdx+	jnz	.Loop_tail8x++.Ldone8x:+	vzeroall+	leaq	(%r10),%rsp+.cfi_def_cfa_register	%rsp+.Lavx2_epilogue:+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_chacha20_asm_avx2,.-crypton_chacha20_asm_avx2++.section	.note.gnu.property,"a",@note+	.long	4,2f-1f,5+	.byte	0x47,0x4E,0x55,0+1:	.long	0xc0000002,4,3+.align	8+2:++.section	.note.GNU-stack,"",@progbits
+ cbits/asm/chacha-x86_64-macosx.S view
@@ -0,0 +1,2232 @@+.text	++++.p2align	6+L$zero:+.long	0,0,0,0+L$one:+.long	1,0,0,0+L$inc:+.long	0,1,2,3+L$four:+.long	4,4,4,4+L$incy:+.long	0,2,4,6,1,3,5,7+L$eight:+.long	8,8,8,8,8,8,8,8+L$rot16:+.byte	0x2,0x3,0x0,0x1, 0x6,0x7,0x4,0x5, 0xa,0xb,0x8,0x9, 0xe,0xf,0xc,0xd+L$rot24:+.byte	0x3,0x0,0x1,0x2, 0x7,0x4,0x5,0x6, 0xb,0x8,0x9,0xa, 0xf,0xc,0xd,0xe+L$twoy:+.long	2,0,0,0, 2,0,0,0+.p2align	6+L$zeroz:+.long	0,0,0,0, 1,0,0,0, 2,0,0,0, 3,0,0,0+L$fourz:+.long	4,0,0,0, 4,0,0,0, 4,0,0,0, 4,0,0,0+L$incz:+.long	0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15+L$sixteen:+.long	16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16+L$sigma:+.byte	101,120,112,97,110,100,32,51,50,45,98,121,116,101,32,107,0+.byte	67,104,97,67,104,97,50,48,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.globl	_crypton_chacha20_asm_ctr32++.p2align	6+_crypton_chacha20_asm_ctr32:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	cmpq	$0,%rdx+	je	L$no_data+	movq	_crypton_ia32cap_P+4(%rip),%r9+	testl	$512,%r9d+	jnz	L$crypton_chacha20_asm_ssse3+	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56+	subq	$64+24,%rsp+.cfi_adjust_cfa_offset	88+L$ctr32_body:++	movq	%rdx,%rbp++	movq	0(%rcx),%r12+	movq	8(%rcx),%r13+	movq	16(%rcx),%r14+	movq	24(%rcx),%r15+	movq	0(%r8),%rax+	movq	8(%r8),%rdx+	movq	%r12,16(%rsp)+	movq	%r13,24(%rsp)+	movq	%r14,0(%rsp)+	movq	%r15,8(%rsp)+	movq	%rax,48(%rsp)+	movq	%rdx,56(%rsp)+	jmp	L$oop_outer++.p2align	5+L$oop_outer:+	movl	$0x61707865,%eax+	movl	$0x3320646e,%ebx+	movl	$0x79622d32,%ecx+	movl	$0x6b206574,%edx+	movl	16(%rsp),%r8d+	movl	20(%rsp),%r9d+	movl	24(%rsp),%r10d+	movl	28(%rsp),%r11d+	movl	48(%rsp),%r12d+	movl	52(%rsp),%r13d+	movl	56(%rsp),%r14d+	movq	%r15,40(%rsp)+	movl	60(%rsp),%r15d++	movq	%rbp,64+0(%rsp)+	movq	%rsi,64+8(%rsp)+	movl	0(%rsp),%esi+	movq	%rdi,64+16(%rsp)+	movl	4(%rsp),%edi+	movl	$10,%ebp+	jmp	L$oop++.p2align	5+L$oop:+	addl	%r8d,%eax+	xorl	%eax,%r12d+	roll	$16,%r12d+	addl	%r9d,%ebx+	xorl	%ebx,%r13d+	roll	$16,%r13d+	addl	%r12d,%esi+	xorl	%esi,%r8d+	roll	$12,%r8d+	addl	%r13d,%edi+	xorl	%edi,%r9d+	roll	$12,%r9d+	addl	%r8d,%eax+	xorl	%eax,%r12d+	roll	$8,%r12d+	addl	%r9d,%ebx+	xorl	%ebx,%r13d+	roll	$8,%r13d+	addl	%r12d,%esi+	xorl	%esi,%r8d+	roll	$7,%r8d+	addl	%r13d,%edi+	xorl	%edi,%r9d+	roll	$7,%r9d+	movl	%esi,32(%rsp)+	movl	%edi,36(%rsp)+	movl	40(%rsp),%esi+	movl	44(%rsp),%edi+	addl	%r10d,%ecx+	xorl	%ecx,%r14d+	roll	$16,%r14d+	addl	%r11d,%edx+	xorl	%edx,%r15d+	roll	$16,%r15d+	addl	%r14d,%esi+	xorl	%esi,%r10d+	roll	$12,%r10d+	addl	%r15d,%edi+	xorl	%edi,%r11d+	roll	$12,%r11d+	addl	%r10d,%ecx+	xorl	%ecx,%r14d+	roll	$8,%r14d+	addl	%r11d,%edx+	xorl	%edx,%r15d+	roll	$8,%r15d+	addl	%r14d,%esi+	xorl	%esi,%r10d+	roll	$7,%r10d+	addl	%r15d,%edi+	xorl	%edi,%r11d+	roll	$7,%r11d+	addl	%r9d,%eax+	xorl	%eax,%r15d+	roll	$16,%r15d+	addl	%r10d,%ebx+	xorl	%ebx,%r12d+	roll	$16,%r12d+	addl	%r15d,%esi+	xorl	%esi,%r9d+	roll	$12,%r9d+	addl	%r12d,%edi+	xorl	%edi,%r10d+	roll	$12,%r10d+	addl	%r9d,%eax+	xorl	%eax,%r15d+	roll	$8,%r15d+	addl	%r10d,%ebx+	xorl	%ebx,%r12d+	roll	$8,%r12d+	addl	%r15d,%esi+	xorl	%esi,%r9d+	roll	$7,%r9d+	addl	%r12d,%edi+	xorl	%edi,%r10d+	roll	$7,%r10d+	movl	%esi,40(%rsp)+	movl	%edi,44(%rsp)+	movl	32(%rsp),%esi+	movl	36(%rsp),%edi+	addl	%r11d,%ecx+	xorl	%ecx,%r13d+	roll	$16,%r13d+	addl	%r8d,%edx+	xorl	%edx,%r14d+	roll	$16,%r14d+	addl	%r13d,%esi+	xorl	%esi,%r11d+	roll	$12,%r11d+	addl	%r14d,%edi+	xorl	%edi,%r8d+	roll	$12,%r8d+	addl	%r11d,%ecx+	xorl	%ecx,%r13d+	roll	$8,%r13d+	addl	%r8d,%edx+	xorl	%edx,%r14d+	roll	$8,%r14d+	addl	%r13d,%esi+	xorl	%esi,%r11d+	roll	$7,%r11d+	addl	%r14d,%edi+	xorl	%edi,%r8d+	roll	$7,%r8d+	decl	%ebp+	jnz	L$oop+	addl	0(%rsp),%esi+	addl	4(%rsp),%edi+	movq	64(%rsp),%rbp+	movl	%esi,32(%rsp)+	movq	64+8(%rsp),%rsi+	movl	%edi,36(%rsp)+	movq	64+16(%rsp),%rdi++	addl	$0x61707865,%eax+	addl	$0x3320646e,%ebx+	addl	$0x79622d32,%ecx+	addl	$0x6b206574,%edx+	addl	16(%rsp),%r8d+	addl	20(%rsp),%r9d+	addl	24(%rsp),%r10d+	addl	28(%rsp),%r11d+	addl	48(%rsp),%r12d+	addl	52(%rsp),%r13d+	addl	56(%rsp),%r14d+	addl	60(%rsp),%r15d++	cmpq	$64,%rbp+	jb	L$tail++	xorl	0(%rsi),%eax+	xorl	4(%rsi),%ebx+	xorl	8(%rsi),%ecx+	xorl	12(%rsi),%edx+	movl	%eax,0(%rdi)+	movl	32(%rsp),%eax+	movl	%ebx,4(%rdi)+	movl	36(%rsp),%ebx+	movl	%ecx,8(%rdi)+	movl	40(%rsp),%ecx+	movl	%edx,12(%rdi)+	movl	44(%rsp),%edx+	xorl	16(%rsi),%r8d+	addl	8(%rsp),%ecx+	xorl	20(%rsi),%r9d+	addl	12(%rsp),%edx+	xorl	24(%rsi),%r10d+	xorl	28(%rsi),%r11d+	xorl	32(%rsi),%eax+	xorl	36(%rsi),%ebx+	xorl	40(%rsi),%ecx+	xorl	44(%rsi),%edx+	xorl	48(%rsi),%r12d+	xorl	52(%rsi),%r13d+	xorl	56(%rsi),%r14d+	xorl	60(%rsi),%r15d+	leaq	64(%rsi),%rsi++	addl	$1,48(%rsp)++	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)+	movl	%eax,32(%rdi)+	movl	%ebx,36(%rdi)+	movl	%ecx,40(%rdi)+	movl	%edx,44(%rdi)+	movl	%r12d,48(%rdi)+	movl	%r13d,52(%rdi)+	movl	%r14d,56(%rdi)+	movl	%r15d,60(%rdi)+	leaq	64(%rdi),%rdi+	movq	8(%rsp),%r15++	subq	$64,%rbp+	jnz	L$oop_outer++	jmp	L$done++.p2align	4+L$tail:+	movl	%eax,0(%rsp)+	movl	8(%rsp),%eax+	movl	%ebx,4(%rsp)+	movl	12(%rsp),%ebx+	movl	%ecx,8(%rsp)+	addl	40(%rsp),%eax+	movl	%edx,12(%rsp)+	addl	44(%rsp),%ebx+	movl	%r8d,16(%rsp)+	movl	%r9d,20(%rsp)+	movl	%r10d,24(%rsp)+	movl	%r11d,28(%rsp)+	movl	%eax,40(%rsp)+	movl	%ebx,44(%rsp)+	xorq	%rbx,%rbx+	movl	%r12d,48(%rsp)+	movl	%r13d,52(%rsp)+	movl	%r14d,56(%rsp)+	movl	%r15d,60(%rsp)++L$oop_tail:+	movzbl	(%rsi,%rbx,1),%eax+	movzbl	(%rsp,%rbx,1),%edx+	leaq	1(%rbx),%rbx+	xorl	%edx,%eax+	movb	%al,-1(%rdi,%rbx,1)+	decq	%rbp+	jnz	L$oop_tail++L$done:+	leaq	64+24+48(%rsp),%rsi+.cfi_def_cfa	%rsi,8+	movq	-48(%rsi),%r15+.cfi_restore	%r15+	movq	-40(%rsi),%r14+.cfi_restore	%r14+	movq	-32(%rsi),%r13+.cfi_restore	%r13+	movq	-24(%rsi),%r12+.cfi_restore	%r12+	movq	-16(%rsi),%rbp+.cfi_restore	%rbp+	movq	-8(%rsi),%rbx+.cfi_restore	%rbx+	leaq	(%rsi),%rsp+.cfi_def_cfa_register	%rsp+L$no_data:+	.byte	0xf3,0xc3+.cfi_endproc	+++.p2align	5+crypton_chacha20_asm_ssse3:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++L$crypton_chacha20_asm_ssse3:+	movq	%rsp,%r10+.cfi_def_cfa_register	%r10+	testl	$2048,%r9d+	jnz	L$crypton_chacha20_asm_4xop+	cmpq	$128,%rdx+	je	L$crypton_chacha20_asm_128+	ja	L$crypton_chacha20_asm_4x++L$do_sse3_after_all:+	subq	$64+8,%rsp+	andq	$-16,%rsp+	movdqa	L$sigma(%rip),%xmm0+	movdqu	(%rcx),%xmm1+	movdqu	16(%rcx),%xmm2+	movdqu	(%r8),%xmm3+	movdqa	L$rot16(%rip),%xmm6+	movdqa	L$rot24(%rip),%xmm7++	movdqa	%xmm0,0(%rsp)+	movdqa	%xmm1,16(%rsp)+	movdqa	%xmm2,32(%rsp)+	movdqa	%xmm3,48(%rsp)+	movq	$10,%r8+	jmp	L$oop_ssse3++.p2align	5+L$oop_outer_ssse3:+	movdqa	L$one(%rip),%xmm3+	movdqa	0(%rsp),%xmm0+	movdqa	16(%rsp),%xmm1+	movdqa	32(%rsp),%xmm2+	paddd	48(%rsp),%xmm3+	movq	$10,%r8+	movdqa	%xmm3,48(%rsp)+	jmp	L$oop_ssse3++.p2align	5+L$oop_ssse3:+	paddd	%xmm1,%xmm0+	pxor	%xmm0,%xmm3+.byte	102,15,56,0,222+	paddd	%xmm3,%xmm2+	pxor	%xmm2,%xmm1+	movdqa	%xmm1,%xmm4+	psrld	$20,%xmm1+	pslld	$12,%xmm4+	por	%xmm4,%xmm1+	paddd	%xmm1,%xmm0+	pxor	%xmm0,%xmm3+.byte	102,15,56,0,223+	paddd	%xmm3,%xmm2+	pxor	%xmm2,%xmm1+	movdqa	%xmm1,%xmm4+	psrld	$25,%xmm1+	pslld	$7,%xmm4+	por	%xmm4,%xmm1+	pshufd	$78,%xmm2,%xmm2+	pshufd	$57,%xmm1,%xmm1+	pshufd	$147,%xmm3,%xmm3+	nop+	paddd	%xmm1,%xmm0+	pxor	%xmm0,%xmm3+.byte	102,15,56,0,222+	paddd	%xmm3,%xmm2+	pxor	%xmm2,%xmm1+	movdqa	%xmm1,%xmm4+	psrld	$20,%xmm1+	pslld	$12,%xmm4+	por	%xmm4,%xmm1+	paddd	%xmm1,%xmm0+	pxor	%xmm0,%xmm3+.byte	102,15,56,0,223+	paddd	%xmm3,%xmm2+	pxor	%xmm2,%xmm1+	movdqa	%xmm1,%xmm4+	psrld	$25,%xmm1+	pslld	$7,%xmm4+	por	%xmm4,%xmm1+	pshufd	$78,%xmm2,%xmm2+	pshufd	$147,%xmm1,%xmm1+	pshufd	$57,%xmm3,%xmm3+	decq	%r8+	jnz	L$oop_ssse3+	paddd	0(%rsp),%xmm0+	paddd	16(%rsp),%xmm1+	paddd	32(%rsp),%xmm2+	paddd	48(%rsp),%xmm3++	cmpq	$64,%rdx+	jb	L$tail_ssse3++	movdqu	0(%rsi),%xmm4+	movdqu	16(%rsi),%xmm5+	pxor	%xmm4,%xmm0+	movdqu	32(%rsi),%xmm4+	pxor	%xmm5,%xmm1+	movdqu	48(%rsi),%xmm5+	leaq	64(%rsi),%rsi+	pxor	%xmm4,%xmm2+	pxor	%xmm5,%xmm3++	movdqu	%xmm0,0(%rdi)+	movdqu	%xmm1,16(%rdi)+	movdqu	%xmm2,32(%rdi)+	movdqu	%xmm3,48(%rdi)+	leaq	64(%rdi),%rdi++	subq	$64,%rdx+	jnz	L$oop_outer_ssse3++	jmp	L$done_ssse3++.p2align	4+L$tail_ssse3:+	movdqa	%xmm0,0(%rsp)+	movdqa	%xmm1,16(%rsp)+	movdqa	%xmm2,32(%rsp)+	movdqa	%xmm3,48(%rsp)+	xorq	%r8,%r8++L$oop_tail_ssse3:+	movzbl	(%rsi,%r8,1),%eax+	movzbl	(%rsp,%r8,1),%ecx+	leaq	1(%r8),%r8+	xorl	%ecx,%eax+	movb	%al,-1(%rdi,%r8,1)+	decq	%rdx+	jnz	L$oop_tail_ssse3++L$done_ssse3:+	leaq	(%r10),%rsp+.cfi_def_cfa_register	%rsp+L$ssse3_epilogue:+	.byte	0xf3,0xc3+.cfi_endproc	+++.p2align	5+crypton_chacha20_asm_128:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++L$crypton_chacha20_asm_128:+	movq	%rsp,%r10+.cfi_def_cfa_register	%r10+	subq	$64+8,%rsp+	andq	$-16,%rsp+	movdqa	L$sigma(%rip),%xmm8+	movdqu	(%rcx),%xmm9+	movdqu	16(%rcx),%xmm2+	movdqu	(%r8),%xmm3+	movdqa	L$one(%rip),%xmm1+	movdqa	L$rot16(%rip),%xmm6+	movdqa	L$rot24(%rip),%xmm7++	movdqa	%xmm8,%xmm10+	movdqa	%xmm8,0(%rsp)+	movdqa	%xmm9,%xmm11+	movdqa	%xmm9,16(%rsp)+	movdqa	%xmm2,%xmm0+	movdqa	%xmm2,32(%rsp)+	paddd	%xmm3,%xmm1+	movdqa	%xmm3,48(%rsp)+	movq	$10,%r8+	jmp	L$oop_128++.p2align	5+L$oop_128:+	paddd	%xmm9,%xmm8+	pxor	%xmm8,%xmm3+	paddd	%xmm11,%xmm10+	pxor	%xmm10,%xmm1+.byte	102,15,56,0,222+.byte	102,15,56,0,206+	paddd	%xmm3,%xmm2+	paddd	%xmm1,%xmm0+	pxor	%xmm2,%xmm9+	pxor	%xmm0,%xmm11+	movdqa	%xmm9,%xmm4+	psrld	$20,%xmm9+	movdqa	%xmm11,%xmm5+	pslld	$12,%xmm4+	psrld	$20,%xmm11+	por	%xmm4,%xmm9+	pslld	$12,%xmm5+	por	%xmm5,%xmm11+	paddd	%xmm9,%xmm8+	pxor	%xmm8,%xmm3+	paddd	%xmm11,%xmm10+	pxor	%xmm10,%xmm1+.byte	102,15,56,0,223+.byte	102,15,56,0,207+	paddd	%xmm3,%xmm2+	paddd	%xmm1,%xmm0+	pxor	%xmm2,%xmm9+	pxor	%xmm0,%xmm11+	movdqa	%xmm9,%xmm4+	psrld	$25,%xmm9+	movdqa	%xmm11,%xmm5+	pslld	$7,%xmm4+	psrld	$25,%xmm11+	por	%xmm4,%xmm9+	pslld	$7,%xmm5+	por	%xmm5,%xmm11+	pshufd	$78,%xmm2,%xmm2+	pshufd	$57,%xmm9,%xmm9+	pshufd	$147,%xmm3,%xmm3+	pshufd	$78,%xmm0,%xmm0+	pshufd	$57,%xmm11,%xmm11+	pshufd	$147,%xmm1,%xmm1+	paddd	%xmm9,%xmm8+	pxor	%xmm8,%xmm3+	paddd	%xmm11,%xmm10+	pxor	%xmm10,%xmm1+.byte	102,15,56,0,222+.byte	102,15,56,0,206+	paddd	%xmm3,%xmm2+	paddd	%xmm1,%xmm0+	pxor	%xmm2,%xmm9+	pxor	%xmm0,%xmm11+	movdqa	%xmm9,%xmm4+	psrld	$20,%xmm9+	movdqa	%xmm11,%xmm5+	pslld	$12,%xmm4+	psrld	$20,%xmm11+	por	%xmm4,%xmm9+	pslld	$12,%xmm5+	por	%xmm5,%xmm11+	paddd	%xmm9,%xmm8+	pxor	%xmm8,%xmm3+	paddd	%xmm11,%xmm10+	pxor	%xmm10,%xmm1+.byte	102,15,56,0,223+.byte	102,15,56,0,207+	paddd	%xmm3,%xmm2+	paddd	%xmm1,%xmm0+	pxor	%xmm2,%xmm9+	pxor	%xmm0,%xmm11+	movdqa	%xmm9,%xmm4+	psrld	$25,%xmm9+	movdqa	%xmm11,%xmm5+	pslld	$7,%xmm4+	psrld	$25,%xmm11+	por	%xmm4,%xmm9+	pslld	$7,%xmm5+	por	%xmm5,%xmm11+	pshufd	$78,%xmm2,%xmm2+	pshufd	$147,%xmm9,%xmm9+	pshufd	$57,%xmm3,%xmm3+	pshufd	$78,%xmm0,%xmm0+	pshufd	$147,%xmm11,%xmm11+	pshufd	$57,%xmm1,%xmm1+	decq	%r8+	jnz	L$oop_128+	paddd	0(%rsp),%xmm8+	paddd	16(%rsp),%xmm9+	paddd	32(%rsp),%xmm2+	paddd	48(%rsp),%xmm3+	paddd	L$one(%rip),%xmm1+	paddd	0(%rsp),%xmm10+	paddd	16(%rsp),%xmm11+	paddd	32(%rsp),%xmm0+	paddd	48(%rsp),%xmm1++	movdqu	0(%rsi),%xmm4+	movdqu	16(%rsi),%xmm5+	pxor	%xmm4,%xmm8+	movdqu	32(%rsi),%xmm4+	pxor	%xmm5,%xmm9+	movdqu	48(%rsi),%xmm5+	pxor	%xmm4,%xmm2+	movdqu	64(%rsi),%xmm4+	pxor	%xmm5,%xmm3+	movdqu	80(%rsi),%xmm5+	pxor	%xmm4,%xmm10+	movdqu	96(%rsi),%xmm4+	pxor	%xmm5,%xmm11+	movdqu	112(%rsi),%xmm5+	pxor	%xmm4,%xmm0+	pxor	%xmm5,%xmm1++	movdqu	%xmm8,0(%rdi)+	movdqu	%xmm9,16(%rdi)+	movdqu	%xmm2,32(%rdi)+	movdqu	%xmm3,48(%rdi)+	movdqu	%xmm10,64(%rdi)+	movdqu	%xmm11,80(%rdi)+	movdqu	%xmm0,96(%rdi)+	movdqu	%xmm1,112(%rdi)+	leaq	(%r10),%rsp+.cfi_def_cfa_register	%rsp+L$128_epilogue:+	.byte	0xf3,0xc3+.cfi_endproc	+++.p2align	5+crypton_chacha20_asm_4x:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++L$crypton_chacha20_asm_4x:+	movq	%rsp,%r10+.cfi_def_cfa_register	%r10+	movq	%r9,%r11+	shrq	$32,%r9+	testq	$32,%r9+	jnz	L$crypton_chacha20_asm_8x+	cmpq	$192,%rdx+	ja	L$proceed4x++	andq	$71303168,%r11+	cmpq	$4194304,%r11+	je	L$do_sse3_after_all++L$proceed4x:+	subq	$0x140+8,%rsp+	andq	$-16,%rsp+	movdqa	L$sigma(%rip),%xmm11+	movdqu	(%rcx),%xmm15+	movdqu	16(%rcx),%xmm7+	movdqu	(%r8),%xmm3+	leaq	256(%rsp),%rcx+	leaq	L$rot16(%rip),%r9+	leaq	L$rot24(%rip),%r11++	pshufd	$0x00,%xmm11,%xmm8+	pshufd	$0x55,%xmm11,%xmm9+	movdqa	%xmm8,64(%rsp)+	pshufd	$0xaa,%xmm11,%xmm10+	movdqa	%xmm9,80(%rsp)+	pshufd	$0xff,%xmm11,%xmm11+	movdqa	%xmm10,96(%rsp)+	movdqa	%xmm11,112(%rsp)++	pshufd	$0x00,%xmm15,%xmm12+	pshufd	$0x55,%xmm15,%xmm13+	movdqa	%xmm12,128-256(%rcx)+	pshufd	$0xaa,%xmm15,%xmm14+	movdqa	%xmm13,144-256(%rcx)+	pshufd	$0xff,%xmm15,%xmm15+	movdqa	%xmm14,160-256(%rcx)+	movdqa	%xmm15,176-256(%rcx)++	pshufd	$0x00,%xmm7,%xmm4+	pshufd	$0x55,%xmm7,%xmm5+	movdqa	%xmm4,192-256(%rcx)+	pshufd	$0xaa,%xmm7,%xmm6+	movdqa	%xmm5,208-256(%rcx)+	pshufd	$0xff,%xmm7,%xmm7+	movdqa	%xmm6,224-256(%rcx)+	movdqa	%xmm7,240-256(%rcx)++	pshufd	$0x00,%xmm3,%xmm0+	pshufd	$0x55,%xmm3,%xmm1+	paddd	L$inc(%rip),%xmm0+	pshufd	$0xaa,%xmm3,%xmm2+	movdqa	%xmm1,272-256(%rcx)+	pshufd	$0xff,%xmm3,%xmm3+	movdqa	%xmm2,288-256(%rcx)+	movdqa	%xmm3,304-256(%rcx)++	jmp	L$oop_enter4x++.p2align	5+L$oop_outer4x:+	movdqa	64(%rsp),%xmm8+	movdqa	80(%rsp),%xmm9+	movdqa	96(%rsp),%xmm10+	movdqa	112(%rsp),%xmm11+	movdqa	128-256(%rcx),%xmm12+	movdqa	144-256(%rcx),%xmm13+	movdqa	160-256(%rcx),%xmm14+	movdqa	176-256(%rcx),%xmm15+	movdqa	192-256(%rcx),%xmm4+	movdqa	208-256(%rcx),%xmm5+	movdqa	224-256(%rcx),%xmm6+	movdqa	240-256(%rcx),%xmm7+	movdqa	256-256(%rcx),%xmm0+	movdqa	272-256(%rcx),%xmm1+	movdqa	288-256(%rcx),%xmm2+	movdqa	304-256(%rcx),%xmm3+	paddd	L$four(%rip),%xmm0++L$oop_enter4x:+	movdqa	%xmm6,32(%rsp)+	movdqa	%xmm7,48(%rsp)+	movdqa	(%r9),%xmm7+	movl	$10,%eax+	movdqa	%xmm0,256-256(%rcx)+	jmp	L$oop4x++.p2align	5+L$oop4x:+	paddd	%xmm12,%xmm8+	paddd	%xmm13,%xmm9+	pxor	%xmm8,%xmm0+	pxor	%xmm9,%xmm1+.byte	102,15,56,0,199+.byte	102,15,56,0,207+	paddd	%xmm0,%xmm4+	paddd	%xmm1,%xmm5+	pxor	%xmm4,%xmm12+	pxor	%xmm5,%xmm13+	movdqa	%xmm12,%xmm6+	pslld	$12,%xmm12+	psrld	$20,%xmm6+	movdqa	%xmm13,%xmm7+	pslld	$12,%xmm13+	por	%xmm6,%xmm12+	psrld	$20,%xmm7+	movdqa	(%r11),%xmm6+	por	%xmm7,%xmm13+	paddd	%xmm12,%xmm8+	paddd	%xmm13,%xmm9+	pxor	%xmm8,%xmm0+	pxor	%xmm9,%xmm1+.byte	102,15,56,0,198+.byte	102,15,56,0,206+	paddd	%xmm0,%xmm4+	paddd	%xmm1,%xmm5+	pxor	%xmm4,%xmm12+	pxor	%xmm5,%xmm13+	movdqa	%xmm12,%xmm7+	pslld	$7,%xmm12+	psrld	$25,%xmm7+	movdqa	%xmm13,%xmm6+	pslld	$7,%xmm13+	por	%xmm7,%xmm12+	psrld	$25,%xmm6+	movdqa	(%r9),%xmm7+	por	%xmm6,%xmm13+	movdqa	%xmm4,0(%rsp)+	movdqa	%xmm5,16(%rsp)+	movdqa	32(%rsp),%xmm4+	movdqa	48(%rsp),%xmm5+	paddd	%xmm14,%xmm10+	paddd	%xmm15,%xmm11+	pxor	%xmm10,%xmm2+	pxor	%xmm11,%xmm3+.byte	102,15,56,0,215+.byte	102,15,56,0,223+	paddd	%xmm2,%xmm4+	paddd	%xmm3,%xmm5+	pxor	%xmm4,%xmm14+	pxor	%xmm5,%xmm15+	movdqa	%xmm14,%xmm6+	pslld	$12,%xmm14+	psrld	$20,%xmm6+	movdqa	%xmm15,%xmm7+	pslld	$12,%xmm15+	por	%xmm6,%xmm14+	psrld	$20,%xmm7+	movdqa	(%r11),%xmm6+	por	%xmm7,%xmm15+	paddd	%xmm14,%xmm10+	paddd	%xmm15,%xmm11+	pxor	%xmm10,%xmm2+	pxor	%xmm11,%xmm3+.byte	102,15,56,0,214+.byte	102,15,56,0,222+	paddd	%xmm2,%xmm4+	paddd	%xmm3,%xmm5+	pxor	%xmm4,%xmm14+	pxor	%xmm5,%xmm15+	movdqa	%xmm14,%xmm7+	pslld	$7,%xmm14+	psrld	$25,%xmm7+	movdqa	%xmm15,%xmm6+	pslld	$7,%xmm15+	por	%xmm7,%xmm14+	psrld	$25,%xmm6+	movdqa	(%r9),%xmm7+	por	%xmm6,%xmm15+	paddd	%xmm13,%xmm8+	paddd	%xmm14,%xmm9+	pxor	%xmm8,%xmm3+	pxor	%xmm9,%xmm0+.byte	102,15,56,0,223+.byte	102,15,56,0,199+	paddd	%xmm3,%xmm4+	paddd	%xmm0,%xmm5+	pxor	%xmm4,%xmm13+	pxor	%xmm5,%xmm14+	movdqa	%xmm13,%xmm6+	pslld	$12,%xmm13+	psrld	$20,%xmm6+	movdqa	%xmm14,%xmm7+	pslld	$12,%xmm14+	por	%xmm6,%xmm13+	psrld	$20,%xmm7+	movdqa	(%r11),%xmm6+	por	%xmm7,%xmm14+	paddd	%xmm13,%xmm8+	paddd	%xmm14,%xmm9+	pxor	%xmm8,%xmm3+	pxor	%xmm9,%xmm0+.byte	102,15,56,0,222+.byte	102,15,56,0,198+	paddd	%xmm3,%xmm4+	paddd	%xmm0,%xmm5+	pxor	%xmm4,%xmm13+	pxor	%xmm5,%xmm14+	movdqa	%xmm13,%xmm7+	pslld	$7,%xmm13+	psrld	$25,%xmm7+	movdqa	%xmm14,%xmm6+	pslld	$7,%xmm14+	por	%xmm7,%xmm13+	psrld	$25,%xmm6+	movdqa	(%r9),%xmm7+	por	%xmm6,%xmm14+	movdqa	%xmm4,32(%rsp)+	movdqa	%xmm5,48(%rsp)+	movdqa	0(%rsp),%xmm4+	movdqa	16(%rsp),%xmm5+	paddd	%xmm15,%xmm10+	paddd	%xmm12,%xmm11+	pxor	%xmm10,%xmm1+	pxor	%xmm11,%xmm2+.byte	102,15,56,0,207+.byte	102,15,56,0,215+	paddd	%xmm1,%xmm4+	paddd	%xmm2,%xmm5+	pxor	%xmm4,%xmm15+	pxor	%xmm5,%xmm12+	movdqa	%xmm15,%xmm6+	pslld	$12,%xmm15+	psrld	$20,%xmm6+	movdqa	%xmm12,%xmm7+	pslld	$12,%xmm12+	por	%xmm6,%xmm15+	psrld	$20,%xmm7+	movdqa	(%r11),%xmm6+	por	%xmm7,%xmm12+	paddd	%xmm15,%xmm10+	paddd	%xmm12,%xmm11+	pxor	%xmm10,%xmm1+	pxor	%xmm11,%xmm2+.byte	102,15,56,0,206+.byte	102,15,56,0,214+	paddd	%xmm1,%xmm4+	paddd	%xmm2,%xmm5+	pxor	%xmm4,%xmm15+	pxor	%xmm5,%xmm12+	movdqa	%xmm15,%xmm7+	pslld	$7,%xmm15+	psrld	$25,%xmm7+	movdqa	%xmm12,%xmm6+	pslld	$7,%xmm12+	por	%xmm7,%xmm15+	psrld	$25,%xmm6+	movdqa	(%r9),%xmm7+	por	%xmm6,%xmm12+	decl	%eax+	jnz	L$oop4x++	paddd	64(%rsp),%xmm8+	paddd	80(%rsp),%xmm9+	paddd	96(%rsp),%xmm10+	paddd	112(%rsp),%xmm11++	movdqa	%xmm8,%xmm6+	punpckldq	%xmm9,%xmm8+	movdqa	%xmm10,%xmm7+	punpckldq	%xmm11,%xmm10+	punpckhdq	%xmm9,%xmm6+	punpckhdq	%xmm11,%xmm7+	movdqa	%xmm8,%xmm9+	punpcklqdq	%xmm10,%xmm8+	movdqa	%xmm6,%xmm11+	punpcklqdq	%xmm7,%xmm6+	punpckhqdq	%xmm10,%xmm9+	punpckhqdq	%xmm7,%xmm11+	paddd	128-256(%rcx),%xmm12+	paddd	144-256(%rcx),%xmm13+	paddd	160-256(%rcx),%xmm14+	paddd	176-256(%rcx),%xmm15++	movdqa	%xmm8,0(%rsp)+	movdqa	%xmm9,16(%rsp)+	movdqa	32(%rsp),%xmm8+	movdqa	48(%rsp),%xmm9++	movdqa	%xmm12,%xmm10+	punpckldq	%xmm13,%xmm12+	movdqa	%xmm14,%xmm7+	punpckldq	%xmm15,%xmm14+	punpckhdq	%xmm13,%xmm10+	punpckhdq	%xmm15,%xmm7+	movdqa	%xmm12,%xmm13+	punpcklqdq	%xmm14,%xmm12+	movdqa	%xmm10,%xmm15+	punpcklqdq	%xmm7,%xmm10+	punpckhqdq	%xmm14,%xmm13+	punpckhqdq	%xmm7,%xmm15+	paddd	192-256(%rcx),%xmm4+	paddd	208-256(%rcx),%xmm5+	paddd	224-256(%rcx),%xmm8+	paddd	240-256(%rcx),%xmm9++	movdqa	%xmm6,32(%rsp)+	movdqa	%xmm11,48(%rsp)++	movdqa	%xmm4,%xmm14+	punpckldq	%xmm5,%xmm4+	movdqa	%xmm8,%xmm7+	punpckldq	%xmm9,%xmm8+	punpckhdq	%xmm5,%xmm14+	punpckhdq	%xmm9,%xmm7+	movdqa	%xmm4,%xmm5+	punpcklqdq	%xmm8,%xmm4+	movdqa	%xmm14,%xmm9+	punpcklqdq	%xmm7,%xmm14+	punpckhqdq	%xmm8,%xmm5+	punpckhqdq	%xmm7,%xmm9+	paddd	256-256(%rcx),%xmm0+	paddd	272-256(%rcx),%xmm1+	paddd	288-256(%rcx),%xmm2+	paddd	304-256(%rcx),%xmm3++	movdqa	%xmm0,%xmm8+	punpckldq	%xmm1,%xmm0+	movdqa	%xmm2,%xmm7+	punpckldq	%xmm3,%xmm2+	punpckhdq	%xmm1,%xmm8+	punpckhdq	%xmm3,%xmm7+	movdqa	%xmm0,%xmm1+	punpcklqdq	%xmm2,%xmm0+	movdqa	%xmm8,%xmm3+	punpcklqdq	%xmm7,%xmm8+	punpckhqdq	%xmm2,%xmm1+	punpckhqdq	%xmm7,%xmm3+	cmpq	$256,%rdx+	jb	L$tail4x++	movdqu	0(%rsi),%xmm6+	movdqu	16(%rsi),%xmm11+	movdqu	32(%rsi),%xmm2+	movdqu	48(%rsi),%xmm7+	pxor	0(%rsp),%xmm6+	pxor	%xmm12,%xmm11+	pxor	%xmm4,%xmm2+	pxor	%xmm0,%xmm7++	movdqu	%xmm6,0(%rdi)+	movdqu	64(%rsi),%xmm6+	movdqu	%xmm11,16(%rdi)+	movdqu	80(%rsi),%xmm11+	movdqu	%xmm2,32(%rdi)+	movdqu	96(%rsi),%xmm2+	movdqu	%xmm7,48(%rdi)+	movdqu	112(%rsi),%xmm7+	leaq	128(%rsi),%rsi+	pxor	16(%rsp),%xmm6+	pxor	%xmm13,%xmm11+	pxor	%xmm5,%xmm2+	pxor	%xmm1,%xmm7++	movdqu	%xmm6,64(%rdi)+	movdqu	0(%rsi),%xmm6+	movdqu	%xmm11,80(%rdi)+	movdqu	16(%rsi),%xmm11+	movdqu	%xmm2,96(%rdi)+	movdqu	32(%rsi),%xmm2+	movdqu	%xmm7,112(%rdi)+	leaq	128(%rdi),%rdi+	movdqu	48(%rsi),%xmm7+	pxor	32(%rsp),%xmm6+	pxor	%xmm10,%xmm11+	pxor	%xmm14,%xmm2+	pxor	%xmm8,%xmm7++	movdqu	%xmm6,0(%rdi)+	movdqu	64(%rsi),%xmm6+	movdqu	%xmm11,16(%rdi)+	movdqu	80(%rsi),%xmm11+	movdqu	%xmm2,32(%rdi)+	movdqu	96(%rsi),%xmm2+	movdqu	%xmm7,48(%rdi)+	movdqu	112(%rsi),%xmm7+	leaq	128(%rsi),%rsi+	pxor	48(%rsp),%xmm6+	pxor	%xmm15,%xmm11+	pxor	%xmm9,%xmm2+	pxor	%xmm3,%xmm7+	movdqu	%xmm6,64(%rdi)+	movdqu	%xmm11,80(%rdi)+	movdqu	%xmm2,96(%rdi)+	movdqu	%xmm7,112(%rdi)+	leaq	128(%rdi),%rdi++	subq	$256,%rdx+	jnz	L$oop_outer4x++	jmp	L$done4x++L$tail4x:+	cmpq	$192,%rdx+	jae	L$192_or_more4x+	cmpq	$128,%rdx+	jae	L$128_or_more4x+	cmpq	$64,%rdx+	jae	L$64_or_more4x+++	xorq	%r9,%r9++	movdqa	%xmm12,16(%rsp)+	movdqa	%xmm4,32(%rsp)+	movdqa	%xmm0,48(%rsp)+	jmp	L$oop_tail4x++.p2align	5+L$64_or_more4x:+	movdqu	0(%rsi),%xmm6+	movdqu	16(%rsi),%xmm11+	movdqu	32(%rsi),%xmm2+	movdqu	48(%rsi),%xmm7+	pxor	0(%rsp),%xmm6+	pxor	%xmm12,%xmm11+	pxor	%xmm4,%xmm2+	pxor	%xmm0,%xmm7+	movdqu	%xmm6,0(%rdi)+	movdqu	%xmm11,16(%rdi)+	movdqu	%xmm2,32(%rdi)+	movdqu	%xmm7,48(%rdi)+	je	L$done4x++	movdqa	16(%rsp),%xmm6+	leaq	64(%rsi),%rsi+	xorq	%r9,%r9+	movdqa	%xmm6,0(%rsp)+	movdqa	%xmm13,16(%rsp)+	leaq	64(%rdi),%rdi+	movdqa	%xmm5,32(%rsp)+	subq	$64,%rdx+	movdqa	%xmm1,48(%rsp)+	jmp	L$oop_tail4x++.p2align	5+L$128_or_more4x:+	movdqu	0(%rsi),%xmm6+	movdqu	16(%rsi),%xmm11+	movdqu	32(%rsi),%xmm2+	movdqu	48(%rsi),%xmm7+	pxor	0(%rsp),%xmm6+	pxor	%xmm12,%xmm11+	pxor	%xmm4,%xmm2+	pxor	%xmm0,%xmm7++	movdqu	%xmm6,0(%rdi)+	movdqu	64(%rsi),%xmm6+	movdqu	%xmm11,16(%rdi)+	movdqu	80(%rsi),%xmm11+	movdqu	%xmm2,32(%rdi)+	movdqu	96(%rsi),%xmm2+	movdqu	%xmm7,48(%rdi)+	movdqu	112(%rsi),%xmm7+	pxor	16(%rsp),%xmm6+	pxor	%xmm13,%xmm11+	pxor	%xmm5,%xmm2+	pxor	%xmm1,%xmm7+	movdqu	%xmm6,64(%rdi)+	movdqu	%xmm11,80(%rdi)+	movdqu	%xmm2,96(%rdi)+	movdqu	%xmm7,112(%rdi)+	je	L$done4x++	movdqa	32(%rsp),%xmm6+	leaq	128(%rsi),%rsi+	xorq	%r9,%r9+	movdqa	%xmm6,0(%rsp)+	movdqa	%xmm10,16(%rsp)+	leaq	128(%rdi),%rdi+	movdqa	%xmm14,32(%rsp)+	subq	$128,%rdx+	movdqa	%xmm8,48(%rsp)+	jmp	L$oop_tail4x++.p2align	5+L$192_or_more4x:+	movdqu	0(%rsi),%xmm6+	movdqu	16(%rsi),%xmm11+	movdqu	32(%rsi),%xmm2+	movdqu	48(%rsi),%xmm7+	pxor	0(%rsp),%xmm6+	pxor	%xmm12,%xmm11+	pxor	%xmm4,%xmm2+	pxor	%xmm0,%xmm7++	movdqu	%xmm6,0(%rdi)+	movdqu	64(%rsi),%xmm6+	movdqu	%xmm11,16(%rdi)+	movdqu	80(%rsi),%xmm11+	movdqu	%xmm2,32(%rdi)+	movdqu	96(%rsi),%xmm2+	movdqu	%xmm7,48(%rdi)+	movdqu	112(%rsi),%xmm7+	leaq	128(%rsi),%rsi+	pxor	16(%rsp),%xmm6+	pxor	%xmm13,%xmm11+	pxor	%xmm5,%xmm2+	pxor	%xmm1,%xmm7++	movdqu	%xmm6,64(%rdi)+	movdqu	0(%rsi),%xmm6+	movdqu	%xmm11,80(%rdi)+	movdqu	16(%rsi),%xmm11+	movdqu	%xmm2,96(%rdi)+	movdqu	32(%rsi),%xmm2+	movdqu	%xmm7,112(%rdi)+	leaq	128(%rdi),%rdi+	movdqu	48(%rsi),%xmm7+	pxor	32(%rsp),%xmm6+	pxor	%xmm10,%xmm11+	pxor	%xmm14,%xmm2+	pxor	%xmm8,%xmm7+	movdqu	%xmm6,0(%rdi)+	movdqu	%xmm11,16(%rdi)+	movdqu	%xmm2,32(%rdi)+	movdqu	%xmm7,48(%rdi)+	je	L$done4x++	movdqa	48(%rsp),%xmm6+	leaq	64(%rsi),%rsi+	xorq	%r9,%r9+	movdqa	%xmm6,0(%rsp)+	movdqa	%xmm15,16(%rsp)+	leaq	64(%rdi),%rdi+	movdqa	%xmm9,32(%rsp)+	subq	$192,%rdx+	movdqa	%xmm3,48(%rsp)++L$oop_tail4x:+	movzbl	(%rsi,%r9,1),%eax+	movzbl	(%rsp,%r9,1),%ecx+	leaq	1(%r9),%r9+	xorl	%ecx,%eax+	movb	%al,-1(%rdi,%r9,1)+	decq	%rdx+	jnz	L$oop_tail4x++L$done4x:+	leaq	(%r10),%rsp+.cfi_def_cfa_register	%rsp+L$4x_epilogue:+	.byte	0xf3,0xc3+.cfi_endproc	+++.p2align	5+crypton_chacha20_asm_4xop:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++L$crypton_chacha20_asm_4xop:+	movq	%rsp,%r10+.cfi_def_cfa_register	%r10+	subq	$0x140+8,%rsp+	andq	$-16,%rsp+	vzeroupper++	vmovdqa	L$sigma(%rip),%xmm11+	vmovdqu	(%rcx),%xmm3+	vmovdqu	16(%rcx),%xmm15+	vmovdqu	(%r8),%xmm7+	leaq	256(%rsp),%rcx++	vpshufd	$0x00,%xmm11,%xmm8+	vpshufd	$0x55,%xmm11,%xmm9+	vmovdqa	%xmm8,64(%rsp)+	vpshufd	$0xaa,%xmm11,%xmm10+	vmovdqa	%xmm9,80(%rsp)+	vpshufd	$0xff,%xmm11,%xmm11+	vmovdqa	%xmm10,96(%rsp)+	vmovdqa	%xmm11,112(%rsp)++	vpshufd	$0x00,%xmm3,%xmm0+	vpshufd	$0x55,%xmm3,%xmm1+	vmovdqa	%xmm0,128-256(%rcx)+	vpshufd	$0xaa,%xmm3,%xmm2+	vmovdqa	%xmm1,144-256(%rcx)+	vpshufd	$0xff,%xmm3,%xmm3+	vmovdqa	%xmm2,160-256(%rcx)+	vmovdqa	%xmm3,176-256(%rcx)++	vpshufd	$0x00,%xmm15,%xmm12+	vpshufd	$0x55,%xmm15,%xmm13+	vmovdqa	%xmm12,192-256(%rcx)+	vpshufd	$0xaa,%xmm15,%xmm14+	vmovdqa	%xmm13,208-256(%rcx)+	vpshufd	$0xff,%xmm15,%xmm15+	vmovdqa	%xmm14,224-256(%rcx)+	vmovdqa	%xmm15,240-256(%rcx)++	vpshufd	$0x00,%xmm7,%xmm4+	vpshufd	$0x55,%xmm7,%xmm5+	vpaddd	L$inc(%rip),%xmm4,%xmm4+	vpshufd	$0xaa,%xmm7,%xmm6+	vmovdqa	%xmm5,272-256(%rcx)+	vpshufd	$0xff,%xmm7,%xmm7+	vmovdqa	%xmm6,288-256(%rcx)+	vmovdqa	%xmm7,304-256(%rcx)++	jmp	L$oop_enter4xop++.p2align	5+L$oop_outer4xop:+	vmovdqa	64(%rsp),%xmm8+	vmovdqa	80(%rsp),%xmm9+	vmovdqa	96(%rsp),%xmm10+	vmovdqa	112(%rsp),%xmm11+	vmovdqa	128-256(%rcx),%xmm0+	vmovdqa	144-256(%rcx),%xmm1+	vmovdqa	160-256(%rcx),%xmm2+	vmovdqa	176-256(%rcx),%xmm3+	vmovdqa	192-256(%rcx),%xmm12+	vmovdqa	208-256(%rcx),%xmm13+	vmovdqa	224-256(%rcx),%xmm14+	vmovdqa	240-256(%rcx),%xmm15+	vmovdqa	256-256(%rcx),%xmm4+	vmovdqa	272-256(%rcx),%xmm5+	vmovdqa	288-256(%rcx),%xmm6+	vmovdqa	304-256(%rcx),%xmm7+	vpaddd	L$four(%rip),%xmm4,%xmm4++L$oop_enter4xop:+	movl	$10,%eax+	vmovdqa	%xmm4,256-256(%rcx)+	jmp	L$oop4xop++.p2align	5+L$oop4xop:+	vpaddd	%xmm0,%xmm8,%xmm8+	vpaddd	%xmm1,%xmm9,%xmm9+	vpaddd	%xmm2,%xmm10,%xmm10+	vpaddd	%xmm3,%xmm11,%xmm11+	vpxor	%xmm4,%xmm8,%xmm4+	vpxor	%xmm5,%xmm9,%xmm5+	vpxor	%xmm6,%xmm10,%xmm6+	vpxor	%xmm7,%xmm11,%xmm7+.byte	143,232,120,194,228,16+.byte	143,232,120,194,237,16+.byte	143,232,120,194,246,16+.byte	143,232,120,194,255,16+	vpaddd	%xmm4,%xmm12,%xmm12+	vpaddd	%xmm5,%xmm13,%xmm13+	vpaddd	%xmm6,%xmm14,%xmm14+	vpaddd	%xmm7,%xmm15,%xmm15+	vpxor	%xmm0,%xmm12,%xmm0+	vpxor	%xmm1,%xmm13,%xmm1+	vpxor	%xmm14,%xmm2,%xmm2+	vpxor	%xmm15,%xmm3,%xmm3+.byte	143,232,120,194,192,12+.byte	143,232,120,194,201,12+.byte	143,232,120,194,210,12+.byte	143,232,120,194,219,12+	vpaddd	%xmm8,%xmm0,%xmm8+	vpaddd	%xmm9,%xmm1,%xmm9+	vpaddd	%xmm2,%xmm10,%xmm10+	vpaddd	%xmm3,%xmm11,%xmm11+	vpxor	%xmm4,%xmm8,%xmm4+	vpxor	%xmm5,%xmm9,%xmm5+	vpxor	%xmm6,%xmm10,%xmm6+	vpxor	%xmm7,%xmm11,%xmm7+.byte	143,232,120,194,228,8+.byte	143,232,120,194,237,8+.byte	143,232,120,194,246,8+.byte	143,232,120,194,255,8+	vpaddd	%xmm4,%xmm12,%xmm12+	vpaddd	%xmm5,%xmm13,%xmm13+	vpaddd	%xmm6,%xmm14,%xmm14+	vpaddd	%xmm7,%xmm15,%xmm15+	vpxor	%xmm0,%xmm12,%xmm0+	vpxor	%xmm1,%xmm13,%xmm1+	vpxor	%xmm14,%xmm2,%xmm2+	vpxor	%xmm15,%xmm3,%xmm3+.byte	143,232,120,194,192,7+.byte	143,232,120,194,201,7+.byte	143,232,120,194,210,7+.byte	143,232,120,194,219,7+	vpaddd	%xmm1,%xmm8,%xmm8+	vpaddd	%xmm2,%xmm9,%xmm9+	vpaddd	%xmm3,%xmm10,%xmm10+	vpaddd	%xmm0,%xmm11,%xmm11+	vpxor	%xmm7,%xmm8,%xmm7+	vpxor	%xmm4,%xmm9,%xmm4+	vpxor	%xmm5,%xmm10,%xmm5+	vpxor	%xmm6,%xmm11,%xmm6+.byte	143,232,120,194,255,16+.byte	143,232,120,194,228,16+.byte	143,232,120,194,237,16+.byte	143,232,120,194,246,16+	vpaddd	%xmm7,%xmm14,%xmm14+	vpaddd	%xmm4,%xmm15,%xmm15+	vpaddd	%xmm5,%xmm12,%xmm12+	vpaddd	%xmm6,%xmm13,%xmm13+	vpxor	%xmm1,%xmm14,%xmm1+	vpxor	%xmm2,%xmm15,%xmm2+	vpxor	%xmm12,%xmm3,%xmm3+	vpxor	%xmm13,%xmm0,%xmm0+.byte	143,232,120,194,201,12+.byte	143,232,120,194,210,12+.byte	143,232,120,194,219,12+.byte	143,232,120,194,192,12+	vpaddd	%xmm8,%xmm1,%xmm8+	vpaddd	%xmm9,%xmm2,%xmm9+	vpaddd	%xmm3,%xmm10,%xmm10+	vpaddd	%xmm0,%xmm11,%xmm11+	vpxor	%xmm7,%xmm8,%xmm7+	vpxor	%xmm4,%xmm9,%xmm4+	vpxor	%xmm5,%xmm10,%xmm5+	vpxor	%xmm6,%xmm11,%xmm6+.byte	143,232,120,194,255,8+.byte	143,232,120,194,228,8+.byte	143,232,120,194,237,8+.byte	143,232,120,194,246,8+	vpaddd	%xmm7,%xmm14,%xmm14+	vpaddd	%xmm4,%xmm15,%xmm15+	vpaddd	%xmm5,%xmm12,%xmm12+	vpaddd	%xmm6,%xmm13,%xmm13+	vpxor	%xmm1,%xmm14,%xmm1+	vpxor	%xmm2,%xmm15,%xmm2+	vpxor	%xmm12,%xmm3,%xmm3+	vpxor	%xmm13,%xmm0,%xmm0+.byte	143,232,120,194,201,7+.byte	143,232,120,194,210,7+.byte	143,232,120,194,219,7+.byte	143,232,120,194,192,7+	decl	%eax+	jnz	L$oop4xop++	vpaddd	64(%rsp),%xmm8,%xmm8+	vpaddd	80(%rsp),%xmm9,%xmm9+	vpaddd	96(%rsp),%xmm10,%xmm10+	vpaddd	112(%rsp),%xmm11,%xmm11++	vmovdqa	%xmm14,32(%rsp)+	vmovdqa	%xmm15,48(%rsp)++	vpunpckldq	%xmm9,%xmm8,%xmm14+	vpunpckldq	%xmm11,%xmm10,%xmm15+	vpunpckhdq	%xmm9,%xmm8,%xmm8+	vpunpckhdq	%xmm11,%xmm10,%xmm10+	vpunpcklqdq	%xmm15,%xmm14,%xmm9+	vpunpckhqdq	%xmm15,%xmm14,%xmm14+	vpunpcklqdq	%xmm10,%xmm8,%xmm11+	vpunpckhqdq	%xmm10,%xmm8,%xmm8+	vpaddd	128-256(%rcx),%xmm0,%xmm0+	vpaddd	144-256(%rcx),%xmm1,%xmm1+	vpaddd	160-256(%rcx),%xmm2,%xmm2+	vpaddd	176-256(%rcx),%xmm3,%xmm3++	vmovdqa	%xmm9,0(%rsp)+	vmovdqa	%xmm14,16(%rsp)+	vmovdqa	32(%rsp),%xmm9+	vmovdqa	48(%rsp),%xmm14++	vpunpckldq	%xmm1,%xmm0,%xmm10+	vpunpckldq	%xmm3,%xmm2,%xmm15+	vpunpckhdq	%xmm1,%xmm0,%xmm0+	vpunpckhdq	%xmm3,%xmm2,%xmm2+	vpunpcklqdq	%xmm15,%xmm10,%xmm1+	vpunpckhqdq	%xmm15,%xmm10,%xmm10+	vpunpcklqdq	%xmm2,%xmm0,%xmm3+	vpunpckhqdq	%xmm2,%xmm0,%xmm0+	vpaddd	192-256(%rcx),%xmm12,%xmm12+	vpaddd	208-256(%rcx),%xmm13,%xmm13+	vpaddd	224-256(%rcx),%xmm9,%xmm9+	vpaddd	240-256(%rcx),%xmm14,%xmm14++	vpunpckldq	%xmm13,%xmm12,%xmm2+	vpunpckldq	%xmm14,%xmm9,%xmm15+	vpunpckhdq	%xmm13,%xmm12,%xmm12+	vpunpckhdq	%xmm14,%xmm9,%xmm9+	vpunpcklqdq	%xmm15,%xmm2,%xmm13+	vpunpckhqdq	%xmm15,%xmm2,%xmm2+	vpunpcklqdq	%xmm9,%xmm12,%xmm14+	vpunpckhqdq	%xmm9,%xmm12,%xmm12+	vpaddd	256-256(%rcx),%xmm4,%xmm4+	vpaddd	272-256(%rcx),%xmm5,%xmm5+	vpaddd	288-256(%rcx),%xmm6,%xmm6+	vpaddd	304-256(%rcx),%xmm7,%xmm7++	vpunpckldq	%xmm5,%xmm4,%xmm9+	vpunpckldq	%xmm7,%xmm6,%xmm15+	vpunpckhdq	%xmm5,%xmm4,%xmm4+	vpunpckhdq	%xmm7,%xmm6,%xmm6+	vpunpcklqdq	%xmm15,%xmm9,%xmm5+	vpunpckhqdq	%xmm15,%xmm9,%xmm9+	vpunpcklqdq	%xmm6,%xmm4,%xmm7+	vpunpckhqdq	%xmm6,%xmm4,%xmm4+	vmovdqa	0(%rsp),%xmm6+	vmovdqa	16(%rsp),%xmm15++	cmpq	$256,%rdx+	jb	L$tail4xop++	vpxor	0(%rsi),%xmm6,%xmm6+	vpxor	16(%rsi),%xmm1,%xmm1+	vpxor	32(%rsi),%xmm13,%xmm13+	vpxor	48(%rsi),%xmm5,%xmm5+	vpxor	64(%rsi),%xmm15,%xmm15+	vpxor	80(%rsi),%xmm10,%xmm10+	vpxor	96(%rsi),%xmm2,%xmm2+	vpxor	112(%rsi),%xmm9,%xmm9+	leaq	128(%rsi),%rsi+	vpxor	0(%rsi),%xmm11,%xmm11+	vpxor	16(%rsi),%xmm3,%xmm3+	vpxor	32(%rsi),%xmm14,%xmm14+	vpxor	48(%rsi),%xmm7,%xmm7+	vpxor	64(%rsi),%xmm8,%xmm8+	vpxor	80(%rsi),%xmm0,%xmm0+	vpxor	96(%rsi),%xmm12,%xmm12+	vpxor	112(%rsi),%xmm4,%xmm4+	leaq	128(%rsi),%rsi++	vmovdqu	%xmm6,0(%rdi)+	vmovdqu	%xmm1,16(%rdi)+	vmovdqu	%xmm13,32(%rdi)+	vmovdqu	%xmm5,48(%rdi)+	vmovdqu	%xmm15,64(%rdi)+	vmovdqu	%xmm10,80(%rdi)+	vmovdqu	%xmm2,96(%rdi)+	vmovdqu	%xmm9,112(%rdi)+	leaq	128(%rdi),%rdi+	vmovdqu	%xmm11,0(%rdi)+	vmovdqu	%xmm3,16(%rdi)+	vmovdqu	%xmm14,32(%rdi)+	vmovdqu	%xmm7,48(%rdi)+	vmovdqu	%xmm8,64(%rdi)+	vmovdqu	%xmm0,80(%rdi)+	vmovdqu	%xmm12,96(%rdi)+	vmovdqu	%xmm4,112(%rdi)+	leaq	128(%rdi),%rdi++	subq	$256,%rdx+	jnz	L$oop_outer4xop++	jmp	L$done4xop++.p2align	5+L$tail4xop:+	cmpq	$192,%rdx+	jae	L$192_or_more4xop+	cmpq	$128,%rdx+	jae	L$128_or_more4xop+	cmpq	$64,%rdx+	jae	L$64_or_more4xop++	xorq	%r9,%r9+	vmovdqa	%xmm6,0(%rsp)+	vmovdqa	%xmm1,16(%rsp)+	vmovdqa	%xmm13,32(%rsp)+	vmovdqa	%xmm5,48(%rsp)+	jmp	L$oop_tail4xop++.p2align	5+L$64_or_more4xop:+	vpxor	0(%rsi),%xmm6,%xmm6+	vpxor	16(%rsi),%xmm1,%xmm1+	vpxor	32(%rsi),%xmm13,%xmm13+	vpxor	48(%rsi),%xmm5,%xmm5+	vmovdqu	%xmm6,0(%rdi)+	vmovdqu	%xmm1,16(%rdi)+	vmovdqu	%xmm13,32(%rdi)+	vmovdqu	%xmm5,48(%rdi)+	je	L$done4xop++	leaq	64(%rsi),%rsi+	vmovdqa	%xmm15,0(%rsp)+	xorq	%r9,%r9+	vmovdqa	%xmm10,16(%rsp)+	leaq	64(%rdi),%rdi+	vmovdqa	%xmm2,32(%rsp)+	subq	$64,%rdx+	vmovdqa	%xmm9,48(%rsp)+	jmp	L$oop_tail4xop++.p2align	5+L$128_or_more4xop:+	vpxor	0(%rsi),%xmm6,%xmm6+	vpxor	16(%rsi),%xmm1,%xmm1+	vpxor	32(%rsi),%xmm13,%xmm13+	vpxor	48(%rsi),%xmm5,%xmm5+	vpxor	64(%rsi),%xmm15,%xmm15+	vpxor	80(%rsi),%xmm10,%xmm10+	vpxor	96(%rsi),%xmm2,%xmm2+	vpxor	112(%rsi),%xmm9,%xmm9++	vmovdqu	%xmm6,0(%rdi)+	vmovdqu	%xmm1,16(%rdi)+	vmovdqu	%xmm13,32(%rdi)+	vmovdqu	%xmm5,48(%rdi)+	vmovdqu	%xmm15,64(%rdi)+	vmovdqu	%xmm10,80(%rdi)+	vmovdqu	%xmm2,96(%rdi)+	vmovdqu	%xmm9,112(%rdi)+	je	L$done4xop++	leaq	128(%rsi),%rsi+	vmovdqa	%xmm11,0(%rsp)+	xorq	%r9,%r9+	vmovdqa	%xmm3,16(%rsp)+	leaq	128(%rdi),%rdi+	vmovdqa	%xmm14,32(%rsp)+	subq	$128,%rdx+	vmovdqa	%xmm7,48(%rsp)+	jmp	L$oop_tail4xop++.p2align	5+L$192_or_more4xop:+	vpxor	0(%rsi),%xmm6,%xmm6+	vpxor	16(%rsi),%xmm1,%xmm1+	vpxor	32(%rsi),%xmm13,%xmm13+	vpxor	48(%rsi),%xmm5,%xmm5+	vpxor	64(%rsi),%xmm15,%xmm15+	vpxor	80(%rsi),%xmm10,%xmm10+	vpxor	96(%rsi),%xmm2,%xmm2+	vpxor	112(%rsi),%xmm9,%xmm9+	leaq	128(%rsi),%rsi+	vpxor	0(%rsi),%xmm11,%xmm11+	vpxor	16(%rsi),%xmm3,%xmm3+	vpxor	32(%rsi),%xmm14,%xmm14+	vpxor	48(%rsi),%xmm7,%xmm7++	vmovdqu	%xmm6,0(%rdi)+	vmovdqu	%xmm1,16(%rdi)+	vmovdqu	%xmm13,32(%rdi)+	vmovdqu	%xmm5,48(%rdi)+	vmovdqu	%xmm15,64(%rdi)+	vmovdqu	%xmm10,80(%rdi)+	vmovdqu	%xmm2,96(%rdi)+	vmovdqu	%xmm9,112(%rdi)+	leaq	128(%rdi),%rdi+	vmovdqu	%xmm11,0(%rdi)+	vmovdqu	%xmm3,16(%rdi)+	vmovdqu	%xmm14,32(%rdi)+	vmovdqu	%xmm7,48(%rdi)+	je	L$done4xop++	leaq	64(%rsi),%rsi+	vmovdqa	%xmm8,0(%rsp)+	xorq	%r9,%r9+	vmovdqa	%xmm0,16(%rsp)+	leaq	64(%rdi),%rdi+	vmovdqa	%xmm12,32(%rsp)+	subq	$192,%rdx+	vmovdqa	%xmm4,48(%rsp)++L$oop_tail4xop:+	movzbl	(%rsi,%r9,1),%eax+	movzbl	(%rsp,%r9,1),%ecx+	leaq	1(%r9),%r9+	xorl	%ecx,%eax+	movb	%al,-1(%rdi,%r9,1)+	decq	%rdx+	jnz	L$oop_tail4xop++L$done4xop:+	vzeroupper+	leaq	(%r10),%rsp+.cfi_def_cfa_register	%rsp+L$4xop_epilogue:+	.byte	0xf3,0xc3+.cfi_endproc	+++.p2align	5+crypton_chacha20_asm_avx2:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++L$crypton_chacha20_asm_8x:+	movq	%rsp,%r10+.cfi_def_cfa_register	%r10+	subq	$0x280+8,%rsp+	andq	$-32,%rsp+	vzeroupper+++++++++++	vbroadcasti128	L$sigma(%rip),%ymm11+	vbroadcasti128	(%rcx),%ymm3+	vbroadcasti128	16(%rcx),%ymm15+	vbroadcasti128	(%r8),%ymm7+	leaq	256(%rsp),%rcx+	leaq	512(%rsp),%rax+	leaq	L$rot16(%rip),%r9+	leaq	L$rot24(%rip),%r11++	vpshufd	$0x00,%ymm11,%ymm8+	vpshufd	$0x55,%ymm11,%ymm9+	vmovdqa	%ymm8,128-256(%rcx)+	vpshufd	$0xaa,%ymm11,%ymm10+	vmovdqa	%ymm9,160-256(%rcx)+	vpshufd	$0xff,%ymm11,%ymm11+	vmovdqa	%ymm10,192-256(%rcx)+	vmovdqa	%ymm11,224-256(%rcx)++	vpshufd	$0x00,%ymm3,%ymm0+	vpshufd	$0x55,%ymm3,%ymm1+	vmovdqa	%ymm0,256-256(%rcx)+	vpshufd	$0xaa,%ymm3,%ymm2+	vmovdqa	%ymm1,288-256(%rcx)+	vpshufd	$0xff,%ymm3,%ymm3+	vmovdqa	%ymm2,320-256(%rcx)+	vmovdqa	%ymm3,352-256(%rcx)++	vpshufd	$0x00,%ymm15,%ymm12+	vpshufd	$0x55,%ymm15,%ymm13+	vmovdqa	%ymm12,384-512(%rax)+	vpshufd	$0xaa,%ymm15,%ymm14+	vmovdqa	%ymm13,416-512(%rax)+	vpshufd	$0xff,%ymm15,%ymm15+	vmovdqa	%ymm14,448-512(%rax)+	vmovdqa	%ymm15,480-512(%rax)++	vpshufd	$0x00,%ymm7,%ymm4+	vpshufd	$0x55,%ymm7,%ymm5+	vpaddd	L$incy(%rip),%ymm4,%ymm4+	vpshufd	$0xaa,%ymm7,%ymm6+	vmovdqa	%ymm5,544-512(%rax)+	vpshufd	$0xff,%ymm7,%ymm7+	vmovdqa	%ymm6,576-512(%rax)+	vmovdqa	%ymm7,608-512(%rax)++	jmp	L$oop_enter8x++.p2align	5+L$oop_outer8x:+	vmovdqa	128-256(%rcx),%ymm8+	vmovdqa	160-256(%rcx),%ymm9+	vmovdqa	192-256(%rcx),%ymm10+	vmovdqa	224-256(%rcx),%ymm11+	vmovdqa	256-256(%rcx),%ymm0+	vmovdqa	288-256(%rcx),%ymm1+	vmovdqa	320-256(%rcx),%ymm2+	vmovdqa	352-256(%rcx),%ymm3+	vmovdqa	384-512(%rax),%ymm12+	vmovdqa	416-512(%rax),%ymm13+	vmovdqa	448-512(%rax),%ymm14+	vmovdqa	480-512(%rax),%ymm15+	vmovdqa	512-512(%rax),%ymm4+	vmovdqa	544-512(%rax),%ymm5+	vmovdqa	576-512(%rax),%ymm6+	vmovdqa	608-512(%rax),%ymm7+	vpaddd	L$eight(%rip),%ymm4,%ymm4++L$oop_enter8x:+	vmovdqa	%ymm14,64(%rsp)+	vmovdqa	%ymm15,96(%rsp)+	vbroadcasti128	(%r9),%ymm15+	vmovdqa	%ymm4,512-512(%rax)+	movl	$10,%eax+	jmp	L$oop8x++.p2align	5+L$oop8x:+	vpaddd	%ymm0,%ymm8,%ymm8+	vpxor	%ymm4,%ymm8,%ymm4+	vpshufb	%ymm15,%ymm4,%ymm4+	vpaddd	%ymm1,%ymm9,%ymm9+	vpxor	%ymm5,%ymm9,%ymm5+	vpshufb	%ymm15,%ymm5,%ymm5+	vpaddd	%ymm4,%ymm12,%ymm12+	vpxor	%ymm0,%ymm12,%ymm0+	vpslld	$12,%ymm0,%ymm14+	vpsrld	$20,%ymm0,%ymm0+	vpor	%ymm0,%ymm14,%ymm0+	vbroadcasti128	(%r11),%ymm14+	vpaddd	%ymm5,%ymm13,%ymm13+	vpxor	%ymm1,%ymm13,%ymm1+	vpslld	$12,%ymm1,%ymm15+	vpsrld	$20,%ymm1,%ymm1+	vpor	%ymm1,%ymm15,%ymm1+	vpaddd	%ymm0,%ymm8,%ymm8+	vpxor	%ymm4,%ymm8,%ymm4+	vpshufb	%ymm14,%ymm4,%ymm4+	vpaddd	%ymm1,%ymm9,%ymm9+	vpxor	%ymm5,%ymm9,%ymm5+	vpshufb	%ymm14,%ymm5,%ymm5+	vpaddd	%ymm4,%ymm12,%ymm12+	vpxor	%ymm0,%ymm12,%ymm0+	vpslld	$7,%ymm0,%ymm15+	vpsrld	$25,%ymm0,%ymm0+	vpor	%ymm0,%ymm15,%ymm0+	vbroadcasti128	(%r9),%ymm15+	vpaddd	%ymm5,%ymm13,%ymm13+	vpxor	%ymm1,%ymm13,%ymm1+	vpslld	$7,%ymm1,%ymm14+	vpsrld	$25,%ymm1,%ymm1+	vpor	%ymm1,%ymm14,%ymm1+	vmovdqa	%ymm12,0(%rsp)+	vmovdqa	%ymm13,32(%rsp)+	vmovdqa	64(%rsp),%ymm12+	vmovdqa	96(%rsp),%ymm13+	vpaddd	%ymm2,%ymm10,%ymm10+	vpxor	%ymm6,%ymm10,%ymm6+	vpshufb	%ymm15,%ymm6,%ymm6+	vpaddd	%ymm3,%ymm11,%ymm11+	vpxor	%ymm7,%ymm11,%ymm7+	vpshufb	%ymm15,%ymm7,%ymm7+	vpaddd	%ymm6,%ymm12,%ymm12+	vpxor	%ymm2,%ymm12,%ymm2+	vpslld	$12,%ymm2,%ymm14+	vpsrld	$20,%ymm2,%ymm2+	vpor	%ymm2,%ymm14,%ymm2+	vbroadcasti128	(%r11),%ymm14+	vpaddd	%ymm7,%ymm13,%ymm13+	vpxor	%ymm3,%ymm13,%ymm3+	vpslld	$12,%ymm3,%ymm15+	vpsrld	$20,%ymm3,%ymm3+	vpor	%ymm3,%ymm15,%ymm3+	vpaddd	%ymm2,%ymm10,%ymm10+	vpxor	%ymm6,%ymm10,%ymm6+	vpshufb	%ymm14,%ymm6,%ymm6+	vpaddd	%ymm3,%ymm11,%ymm11+	vpxor	%ymm7,%ymm11,%ymm7+	vpshufb	%ymm14,%ymm7,%ymm7+	vpaddd	%ymm6,%ymm12,%ymm12+	vpxor	%ymm2,%ymm12,%ymm2+	vpslld	$7,%ymm2,%ymm15+	vpsrld	$25,%ymm2,%ymm2+	vpor	%ymm2,%ymm15,%ymm2+	vbroadcasti128	(%r9),%ymm15+	vpaddd	%ymm7,%ymm13,%ymm13+	vpxor	%ymm3,%ymm13,%ymm3+	vpslld	$7,%ymm3,%ymm14+	vpsrld	$25,%ymm3,%ymm3+	vpor	%ymm3,%ymm14,%ymm3+	vpaddd	%ymm1,%ymm8,%ymm8+	vpxor	%ymm7,%ymm8,%ymm7+	vpshufb	%ymm15,%ymm7,%ymm7+	vpaddd	%ymm2,%ymm9,%ymm9+	vpxor	%ymm4,%ymm9,%ymm4+	vpshufb	%ymm15,%ymm4,%ymm4+	vpaddd	%ymm7,%ymm12,%ymm12+	vpxor	%ymm1,%ymm12,%ymm1+	vpslld	$12,%ymm1,%ymm14+	vpsrld	$20,%ymm1,%ymm1+	vpor	%ymm1,%ymm14,%ymm1+	vbroadcasti128	(%r11),%ymm14+	vpaddd	%ymm4,%ymm13,%ymm13+	vpxor	%ymm2,%ymm13,%ymm2+	vpslld	$12,%ymm2,%ymm15+	vpsrld	$20,%ymm2,%ymm2+	vpor	%ymm2,%ymm15,%ymm2+	vpaddd	%ymm1,%ymm8,%ymm8+	vpxor	%ymm7,%ymm8,%ymm7+	vpshufb	%ymm14,%ymm7,%ymm7+	vpaddd	%ymm2,%ymm9,%ymm9+	vpxor	%ymm4,%ymm9,%ymm4+	vpshufb	%ymm14,%ymm4,%ymm4+	vpaddd	%ymm7,%ymm12,%ymm12+	vpxor	%ymm1,%ymm12,%ymm1+	vpslld	$7,%ymm1,%ymm15+	vpsrld	$25,%ymm1,%ymm1+	vpor	%ymm1,%ymm15,%ymm1+	vbroadcasti128	(%r9),%ymm15+	vpaddd	%ymm4,%ymm13,%ymm13+	vpxor	%ymm2,%ymm13,%ymm2+	vpslld	$7,%ymm2,%ymm14+	vpsrld	$25,%ymm2,%ymm2+	vpor	%ymm2,%ymm14,%ymm2+	vmovdqa	%ymm12,64(%rsp)+	vmovdqa	%ymm13,96(%rsp)+	vmovdqa	0(%rsp),%ymm12+	vmovdqa	32(%rsp),%ymm13+	vpaddd	%ymm3,%ymm10,%ymm10+	vpxor	%ymm5,%ymm10,%ymm5+	vpshufb	%ymm15,%ymm5,%ymm5+	vpaddd	%ymm0,%ymm11,%ymm11+	vpxor	%ymm6,%ymm11,%ymm6+	vpshufb	%ymm15,%ymm6,%ymm6+	vpaddd	%ymm5,%ymm12,%ymm12+	vpxor	%ymm3,%ymm12,%ymm3+	vpslld	$12,%ymm3,%ymm14+	vpsrld	$20,%ymm3,%ymm3+	vpor	%ymm3,%ymm14,%ymm3+	vbroadcasti128	(%r11),%ymm14+	vpaddd	%ymm6,%ymm13,%ymm13+	vpxor	%ymm0,%ymm13,%ymm0+	vpslld	$12,%ymm0,%ymm15+	vpsrld	$20,%ymm0,%ymm0+	vpor	%ymm0,%ymm15,%ymm0+	vpaddd	%ymm3,%ymm10,%ymm10+	vpxor	%ymm5,%ymm10,%ymm5+	vpshufb	%ymm14,%ymm5,%ymm5+	vpaddd	%ymm0,%ymm11,%ymm11+	vpxor	%ymm6,%ymm11,%ymm6+	vpshufb	%ymm14,%ymm6,%ymm6+	vpaddd	%ymm5,%ymm12,%ymm12+	vpxor	%ymm3,%ymm12,%ymm3+	vpslld	$7,%ymm3,%ymm15+	vpsrld	$25,%ymm3,%ymm3+	vpor	%ymm3,%ymm15,%ymm3+	vbroadcasti128	(%r9),%ymm15+	vpaddd	%ymm6,%ymm13,%ymm13+	vpxor	%ymm0,%ymm13,%ymm0+	vpslld	$7,%ymm0,%ymm14+	vpsrld	$25,%ymm0,%ymm0+	vpor	%ymm0,%ymm14,%ymm0+	decl	%eax+	jnz	L$oop8x++	leaq	512(%rsp),%rax+	vpaddd	128-256(%rcx),%ymm8,%ymm8+	vpaddd	160-256(%rcx),%ymm9,%ymm9+	vpaddd	192-256(%rcx),%ymm10,%ymm10+	vpaddd	224-256(%rcx),%ymm11,%ymm11++	vpunpckldq	%ymm9,%ymm8,%ymm14+	vpunpckldq	%ymm11,%ymm10,%ymm15+	vpunpckhdq	%ymm9,%ymm8,%ymm8+	vpunpckhdq	%ymm11,%ymm10,%ymm10+	vpunpcklqdq	%ymm15,%ymm14,%ymm9+	vpunpckhqdq	%ymm15,%ymm14,%ymm14+	vpunpcklqdq	%ymm10,%ymm8,%ymm11+	vpunpckhqdq	%ymm10,%ymm8,%ymm8+	vpaddd	256-256(%rcx),%ymm0,%ymm0+	vpaddd	288-256(%rcx),%ymm1,%ymm1+	vpaddd	320-256(%rcx),%ymm2,%ymm2+	vpaddd	352-256(%rcx),%ymm3,%ymm3++	vpunpckldq	%ymm1,%ymm0,%ymm10+	vpunpckldq	%ymm3,%ymm2,%ymm15+	vpunpckhdq	%ymm1,%ymm0,%ymm0+	vpunpckhdq	%ymm3,%ymm2,%ymm2+	vpunpcklqdq	%ymm15,%ymm10,%ymm1+	vpunpckhqdq	%ymm15,%ymm10,%ymm10+	vpunpcklqdq	%ymm2,%ymm0,%ymm3+	vpunpckhqdq	%ymm2,%ymm0,%ymm0+	vperm2i128	$0x20,%ymm1,%ymm9,%ymm15+	vperm2i128	$0x31,%ymm1,%ymm9,%ymm1+	vperm2i128	$0x20,%ymm10,%ymm14,%ymm9+	vperm2i128	$0x31,%ymm10,%ymm14,%ymm10+	vperm2i128	$0x20,%ymm3,%ymm11,%ymm14+	vperm2i128	$0x31,%ymm3,%ymm11,%ymm3+	vperm2i128	$0x20,%ymm0,%ymm8,%ymm11+	vperm2i128	$0x31,%ymm0,%ymm8,%ymm0+	vmovdqa	%ymm15,0(%rsp)+	vmovdqa	%ymm9,32(%rsp)+	vmovdqa	64(%rsp),%ymm15+	vmovdqa	96(%rsp),%ymm9++	vpaddd	384-512(%rax),%ymm12,%ymm12+	vpaddd	416-512(%rax),%ymm13,%ymm13+	vpaddd	448-512(%rax),%ymm15,%ymm15+	vpaddd	480-512(%rax),%ymm9,%ymm9++	vpunpckldq	%ymm13,%ymm12,%ymm2+	vpunpckldq	%ymm9,%ymm15,%ymm8+	vpunpckhdq	%ymm13,%ymm12,%ymm12+	vpunpckhdq	%ymm9,%ymm15,%ymm15+	vpunpcklqdq	%ymm8,%ymm2,%ymm13+	vpunpckhqdq	%ymm8,%ymm2,%ymm2+	vpunpcklqdq	%ymm15,%ymm12,%ymm9+	vpunpckhqdq	%ymm15,%ymm12,%ymm12+	vpaddd	512-512(%rax),%ymm4,%ymm4+	vpaddd	544-512(%rax),%ymm5,%ymm5+	vpaddd	576-512(%rax),%ymm6,%ymm6+	vpaddd	608-512(%rax),%ymm7,%ymm7++	vpunpckldq	%ymm5,%ymm4,%ymm15+	vpunpckldq	%ymm7,%ymm6,%ymm8+	vpunpckhdq	%ymm5,%ymm4,%ymm4+	vpunpckhdq	%ymm7,%ymm6,%ymm6+	vpunpcklqdq	%ymm8,%ymm15,%ymm5+	vpunpckhqdq	%ymm8,%ymm15,%ymm15+	vpunpcklqdq	%ymm6,%ymm4,%ymm7+	vpunpckhqdq	%ymm6,%ymm4,%ymm4+	vperm2i128	$0x20,%ymm5,%ymm13,%ymm8+	vperm2i128	$0x31,%ymm5,%ymm13,%ymm5+	vperm2i128	$0x20,%ymm15,%ymm2,%ymm13+	vperm2i128	$0x31,%ymm15,%ymm2,%ymm15+	vperm2i128	$0x20,%ymm7,%ymm9,%ymm2+	vperm2i128	$0x31,%ymm7,%ymm9,%ymm7+	vperm2i128	$0x20,%ymm4,%ymm12,%ymm9+	vperm2i128	$0x31,%ymm4,%ymm12,%ymm4+	vmovdqa	0(%rsp),%ymm6+	vmovdqa	32(%rsp),%ymm12++	cmpq	$512,%rdx+	jb	L$tail8x++	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	leaq	128(%rsi),%rsi+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	leaq	128(%rdi),%rdi++	vpxor	0(%rsi),%ymm12,%ymm12+	vpxor	32(%rsi),%ymm13,%ymm13+	vpxor	64(%rsi),%ymm10,%ymm10+	vpxor	96(%rsi),%ymm15,%ymm15+	leaq	128(%rsi),%rsi+	vmovdqu	%ymm12,0(%rdi)+	vmovdqu	%ymm13,32(%rdi)+	vmovdqu	%ymm10,64(%rdi)+	vmovdqu	%ymm15,96(%rdi)+	leaq	128(%rdi),%rdi++	vpxor	0(%rsi),%ymm14,%ymm14+	vpxor	32(%rsi),%ymm2,%ymm2+	vpxor	64(%rsi),%ymm3,%ymm3+	vpxor	96(%rsi),%ymm7,%ymm7+	leaq	128(%rsi),%rsi+	vmovdqu	%ymm14,0(%rdi)+	vmovdqu	%ymm2,32(%rdi)+	vmovdqu	%ymm3,64(%rdi)+	vmovdqu	%ymm7,96(%rdi)+	leaq	128(%rdi),%rdi++	vpxor	0(%rsi),%ymm11,%ymm11+	vpxor	32(%rsi),%ymm9,%ymm9+	vpxor	64(%rsi),%ymm0,%ymm0+	vpxor	96(%rsi),%ymm4,%ymm4+	leaq	128(%rsi),%rsi+	vmovdqu	%ymm11,0(%rdi)+	vmovdqu	%ymm9,32(%rdi)+	vmovdqu	%ymm0,64(%rdi)+	vmovdqu	%ymm4,96(%rdi)+	leaq	128(%rdi),%rdi++	subq	$512,%rdx+	jnz	L$oop_outer8x++	jmp	L$done8x++L$tail8x:+	cmpq	$448,%rdx+	jae	L$448_or_more8x+	cmpq	$384,%rdx+	jae	L$384_or_more8x+	cmpq	$320,%rdx+	jae	L$320_or_more8x+	cmpq	$256,%rdx+	jae	L$256_or_more8x+	cmpq	$192,%rdx+	jae	L$192_or_more8x+	cmpq	$128,%rdx+	jae	L$128_or_more8x+	cmpq	$64,%rdx+	jae	L$64_or_more8x++	xorq	%r9,%r9+	vmovdqa	%ymm6,0(%rsp)+	vmovdqa	%ymm8,32(%rsp)+	jmp	L$oop_tail8x++.p2align	5+L$64_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	je	L$done8x++	leaq	64(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm1,0(%rsp)+	leaq	64(%rdi),%rdi+	subq	$64,%rdx+	vmovdqa	%ymm5,32(%rsp)+	jmp	L$oop_tail8x++.p2align	5+L$128_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	je	L$done8x++	leaq	128(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm12,0(%rsp)+	leaq	128(%rdi),%rdi+	subq	$128,%rdx+	vmovdqa	%ymm13,32(%rsp)+	jmp	L$oop_tail8x++.p2align	5+L$192_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	je	L$done8x++	leaq	192(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm10,0(%rsp)+	leaq	192(%rdi),%rdi+	subq	$192,%rdx+	vmovdqa	%ymm15,32(%rsp)+	jmp	L$oop_tail8x++.p2align	5+L$256_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vpxor	192(%rsi),%ymm10,%ymm10+	vpxor	224(%rsi),%ymm15,%ymm15+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	vmovdqu	%ymm10,192(%rdi)+	vmovdqu	%ymm15,224(%rdi)+	je	L$done8x++	leaq	256(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm14,0(%rsp)+	leaq	256(%rdi),%rdi+	subq	$256,%rdx+	vmovdqa	%ymm2,32(%rsp)+	jmp	L$oop_tail8x++.p2align	5+L$320_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vpxor	192(%rsi),%ymm10,%ymm10+	vpxor	224(%rsi),%ymm15,%ymm15+	vpxor	256(%rsi),%ymm14,%ymm14+	vpxor	288(%rsi),%ymm2,%ymm2+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	vmovdqu	%ymm10,192(%rdi)+	vmovdqu	%ymm15,224(%rdi)+	vmovdqu	%ymm14,256(%rdi)+	vmovdqu	%ymm2,288(%rdi)+	je	L$done8x++	leaq	320(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm3,0(%rsp)+	leaq	320(%rdi),%rdi+	subq	$320,%rdx+	vmovdqa	%ymm7,32(%rsp)+	jmp	L$oop_tail8x++.p2align	5+L$384_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vpxor	192(%rsi),%ymm10,%ymm10+	vpxor	224(%rsi),%ymm15,%ymm15+	vpxor	256(%rsi),%ymm14,%ymm14+	vpxor	288(%rsi),%ymm2,%ymm2+	vpxor	320(%rsi),%ymm3,%ymm3+	vpxor	352(%rsi),%ymm7,%ymm7+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	vmovdqu	%ymm10,192(%rdi)+	vmovdqu	%ymm15,224(%rdi)+	vmovdqu	%ymm14,256(%rdi)+	vmovdqu	%ymm2,288(%rdi)+	vmovdqu	%ymm3,320(%rdi)+	vmovdqu	%ymm7,352(%rdi)+	je	L$done8x++	leaq	384(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm11,0(%rsp)+	leaq	384(%rdi),%rdi+	subq	$384,%rdx+	vmovdqa	%ymm9,32(%rsp)+	jmp	L$oop_tail8x++.p2align	5+L$448_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vpxor	192(%rsi),%ymm10,%ymm10+	vpxor	224(%rsi),%ymm15,%ymm15+	vpxor	256(%rsi),%ymm14,%ymm14+	vpxor	288(%rsi),%ymm2,%ymm2+	vpxor	320(%rsi),%ymm3,%ymm3+	vpxor	352(%rsi),%ymm7,%ymm7+	vpxor	384(%rsi),%ymm11,%ymm11+	vpxor	416(%rsi),%ymm9,%ymm9+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	vmovdqu	%ymm10,192(%rdi)+	vmovdqu	%ymm15,224(%rdi)+	vmovdqu	%ymm14,256(%rdi)+	vmovdqu	%ymm2,288(%rdi)+	vmovdqu	%ymm3,320(%rdi)+	vmovdqu	%ymm7,352(%rdi)+	vmovdqu	%ymm11,384(%rdi)+	vmovdqu	%ymm9,416(%rdi)+	je	L$done8x++	leaq	448(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm0,0(%rsp)+	leaq	448(%rdi),%rdi+	subq	$448,%rdx+	vmovdqa	%ymm4,32(%rsp)++L$oop_tail8x:+	movzbl	(%rsi,%r9,1),%eax+	movzbl	(%rsp,%r9,1),%ecx+	leaq	1(%r9),%r9+	xorl	%ecx,%eax+	movb	%al,-1(%rdi,%r9,1)+	decq	%rdx+	jnz	L$oop_tail8x++L$done8x:+	vzeroall+	leaq	(%r10),%rsp+.cfi_def_cfa_register	%rsp+L$avx2_epilogue:+	.byte	0xf3,0xc3+.cfi_endproc	+
+ cbits/asm/chacha-x86_64-mingw64.S view
@@ -0,0 +1,2556 @@+.text	++++.p2align	6+.Lzero:+.long	0,0,0,0+.Lone:+.long	1,0,0,0+.Linc:+.long	0,1,2,3+.Lfour:+.long	4,4,4,4+.Lincy:+.long	0,2,4,6,1,3,5,7+.Leight:+.long	8,8,8,8,8,8,8,8+.Lrot16:+.byte	0x2,0x3,0x0,0x1, 0x6,0x7,0x4,0x5, 0xa,0xb,0x8,0x9, 0xe,0xf,0xc,0xd+.Lrot24:+.byte	0x3,0x0,0x1,0x2, 0x7,0x4,0x5,0x6, 0xb,0x8,0x9,0xa, 0xf,0xc,0xd,0xe+.Ltwoy:+.long	2,0,0,0, 2,0,0,0+.p2align	6+.Lzeroz:+.long	0,0,0,0, 1,0,0,0, 2,0,0,0, 3,0,0,0+.Lfourz:+.long	4,0,0,0, 4,0,0,0, 4,0,0,0, 4,0,0,0+.Lincz:+.long	0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15+.Lsixteen:+.long	16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16+.Lsigma:+.byte	101,120,112,97,110,100,32,51,50,45,98,121,116,101,32,107,0+.byte	67,104,97,67,104,97,50,48,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.globl	crypton_chacha20_asm_ctr32+.def	crypton_chacha20_asm_ctr32;	.scl 2;	.type 32;	.endef+.p2align	6+crypton_chacha20_asm_ctr32:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%rax+.LSEH_begin_crypton_chacha20_asm_ctr32:+++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	movq	%r9,%rcx+	movq	40(%rsp),%r8+	cmpq	$0,%rdx+	je	.Lno_data+	movq	crypton_ia32cap_P+4(%rip),%r9+	testl	$512,%r9d+	jnz	.Lcrypton_chacha20_asm_ssse3+	pushq	%rbx++	pushq	%rbp++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	subq	$64+24,%rsp++.Lctr32_body:++	movq	%rdx,%rbp++	movq	0(%rcx),%r12+	movq	8(%rcx),%r13+	movq	16(%rcx),%r14+	movq	24(%rcx),%r15+	movq	0(%r8),%rax+	movq	8(%r8),%rdx+	movq	%r12,16(%rsp)+	movq	%r13,24(%rsp)+	movq	%r14,0(%rsp)+	movq	%r15,8(%rsp)+	movq	%rax,48(%rsp)+	movq	%rdx,56(%rsp)+	jmp	.Loop_outer++.p2align	5+.Loop_outer:+	movl	$0x61707865,%eax+	movl	$0x3320646e,%ebx+	movl	$0x79622d32,%ecx+	movl	$0x6b206574,%edx+	movl	16(%rsp),%r8d+	movl	20(%rsp),%r9d+	movl	24(%rsp),%r10d+	movl	28(%rsp),%r11d+	movl	48(%rsp),%r12d+	movl	52(%rsp),%r13d+	movl	56(%rsp),%r14d+	movq	%r15,40(%rsp)+	movl	60(%rsp),%r15d++	movq	%rbp,64+0(%rsp)+	movq	%rsi,64+8(%rsp)+	movl	0(%rsp),%esi+	movq	%rdi,64+16(%rsp)+	movl	4(%rsp),%edi+	movl	$10,%ebp+	jmp	.Loop++.p2align	5+.Loop:+	addl	%r8d,%eax+	xorl	%eax,%r12d+	roll	$16,%r12d+	addl	%r9d,%ebx+	xorl	%ebx,%r13d+	roll	$16,%r13d+	addl	%r12d,%esi+	xorl	%esi,%r8d+	roll	$12,%r8d+	addl	%r13d,%edi+	xorl	%edi,%r9d+	roll	$12,%r9d+	addl	%r8d,%eax+	xorl	%eax,%r12d+	roll	$8,%r12d+	addl	%r9d,%ebx+	xorl	%ebx,%r13d+	roll	$8,%r13d+	addl	%r12d,%esi+	xorl	%esi,%r8d+	roll	$7,%r8d+	addl	%r13d,%edi+	xorl	%edi,%r9d+	roll	$7,%r9d+	movl	%esi,32(%rsp)+	movl	%edi,36(%rsp)+	movl	40(%rsp),%esi+	movl	44(%rsp),%edi+	addl	%r10d,%ecx+	xorl	%ecx,%r14d+	roll	$16,%r14d+	addl	%r11d,%edx+	xorl	%edx,%r15d+	roll	$16,%r15d+	addl	%r14d,%esi+	xorl	%esi,%r10d+	roll	$12,%r10d+	addl	%r15d,%edi+	xorl	%edi,%r11d+	roll	$12,%r11d+	addl	%r10d,%ecx+	xorl	%ecx,%r14d+	roll	$8,%r14d+	addl	%r11d,%edx+	xorl	%edx,%r15d+	roll	$8,%r15d+	addl	%r14d,%esi+	xorl	%esi,%r10d+	roll	$7,%r10d+	addl	%r15d,%edi+	xorl	%edi,%r11d+	roll	$7,%r11d+	addl	%r9d,%eax+	xorl	%eax,%r15d+	roll	$16,%r15d+	addl	%r10d,%ebx+	xorl	%ebx,%r12d+	roll	$16,%r12d+	addl	%r15d,%esi+	xorl	%esi,%r9d+	roll	$12,%r9d+	addl	%r12d,%edi+	xorl	%edi,%r10d+	roll	$12,%r10d+	addl	%r9d,%eax+	xorl	%eax,%r15d+	roll	$8,%r15d+	addl	%r10d,%ebx+	xorl	%ebx,%r12d+	roll	$8,%r12d+	addl	%r15d,%esi+	xorl	%esi,%r9d+	roll	$7,%r9d+	addl	%r12d,%edi+	xorl	%edi,%r10d+	roll	$7,%r10d+	movl	%esi,40(%rsp)+	movl	%edi,44(%rsp)+	movl	32(%rsp),%esi+	movl	36(%rsp),%edi+	addl	%r11d,%ecx+	xorl	%ecx,%r13d+	roll	$16,%r13d+	addl	%r8d,%edx+	xorl	%edx,%r14d+	roll	$16,%r14d+	addl	%r13d,%esi+	xorl	%esi,%r11d+	roll	$12,%r11d+	addl	%r14d,%edi+	xorl	%edi,%r8d+	roll	$12,%r8d+	addl	%r11d,%ecx+	xorl	%ecx,%r13d+	roll	$8,%r13d+	addl	%r8d,%edx+	xorl	%edx,%r14d+	roll	$8,%r14d+	addl	%r13d,%esi+	xorl	%esi,%r11d+	roll	$7,%r11d+	addl	%r14d,%edi+	xorl	%edi,%r8d+	roll	$7,%r8d+	decl	%ebp+	jnz	.Loop+	addl	0(%rsp),%esi+	addl	4(%rsp),%edi+	movq	64(%rsp),%rbp+	movl	%esi,32(%rsp)+	movq	64+8(%rsp),%rsi+	movl	%edi,36(%rsp)+	movq	64+16(%rsp),%rdi++	addl	$0x61707865,%eax+	addl	$0x3320646e,%ebx+	addl	$0x79622d32,%ecx+	addl	$0x6b206574,%edx+	addl	16(%rsp),%r8d+	addl	20(%rsp),%r9d+	addl	24(%rsp),%r10d+	addl	28(%rsp),%r11d+	addl	48(%rsp),%r12d+	addl	52(%rsp),%r13d+	addl	56(%rsp),%r14d+	addl	60(%rsp),%r15d++	cmpq	$64,%rbp+	jb	.Ltail++	xorl	0(%rsi),%eax+	xorl	4(%rsi),%ebx+	xorl	8(%rsi),%ecx+	xorl	12(%rsi),%edx+	movl	%eax,0(%rdi)+	movl	32(%rsp),%eax+	movl	%ebx,4(%rdi)+	movl	36(%rsp),%ebx+	movl	%ecx,8(%rdi)+	movl	40(%rsp),%ecx+	movl	%edx,12(%rdi)+	movl	44(%rsp),%edx+	xorl	16(%rsi),%r8d+	addl	8(%rsp),%ecx+	xorl	20(%rsi),%r9d+	addl	12(%rsp),%edx+	xorl	24(%rsi),%r10d+	xorl	28(%rsi),%r11d+	xorl	32(%rsi),%eax+	xorl	36(%rsi),%ebx+	xorl	40(%rsi),%ecx+	xorl	44(%rsi),%edx+	xorl	48(%rsi),%r12d+	xorl	52(%rsi),%r13d+	xorl	56(%rsi),%r14d+	xorl	60(%rsi),%r15d+	leaq	64(%rsi),%rsi++	addl	$1,48(%rsp)++	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)+	movl	%eax,32(%rdi)+	movl	%ebx,36(%rdi)+	movl	%ecx,40(%rdi)+	movl	%edx,44(%rdi)+	movl	%r12d,48(%rdi)+	movl	%r13d,52(%rdi)+	movl	%r14d,56(%rdi)+	movl	%r15d,60(%rdi)+	leaq	64(%rdi),%rdi+	movq	8(%rsp),%r15++	subq	$64,%rbp+	jnz	.Loop_outer++	jmp	.Ldone++.p2align	4+.Ltail:+	movl	%eax,0(%rsp)+	movl	8(%rsp),%eax+	movl	%ebx,4(%rsp)+	movl	12(%rsp),%ebx+	movl	%ecx,8(%rsp)+	addl	40(%rsp),%eax+	movl	%edx,12(%rsp)+	addl	44(%rsp),%ebx+	movl	%r8d,16(%rsp)+	movl	%r9d,20(%rsp)+	movl	%r10d,24(%rsp)+	movl	%r11d,28(%rsp)+	movl	%eax,40(%rsp)+	movl	%ebx,44(%rsp)+	xorq	%rbx,%rbx+	movl	%r12d,48(%rsp)+	movl	%r13d,52(%rsp)+	movl	%r14d,56(%rsp)+	movl	%r15d,60(%rsp)++.Loop_tail:+	movzbl	(%rsi,%rbx,1),%eax+	movzbl	(%rsp,%rbx,1),%edx+	leaq	1(%rbx),%rbx+	xorl	%edx,%eax+	movb	%al,-1(%rdi,%rbx,1)+	decq	%rbp+	jnz	.Loop_tail++.Ldone:+	leaq	64+24+48(%rsp),%rsi++	movq	-48(%rsi),%r15++	movq	-40(%rsi),%r14++	movq	-32(%rsi),%r13++	movq	-24(%rsi),%r12++	movq	-16(%rsi),%rbp++	movq	-8(%rsi),%rbx++	leaq	(%rsi),%rsp++.Lno_data:+	movq	8(%rsp),%rdi+	movq	16(%rsp),%rsi+	.byte	0xf3,0xc3++.LSEH_end_crypton_chacha20_asm_ctr32:+.def	crypton_chacha20_asm_ssse3;	.scl 3;	.type 32;	.endef+.p2align	5+crypton_chacha20_asm_ssse3:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%rax+.LSEH_begin_crypton_chacha20_asm_ssse3:+++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	movq	%r9,%rcx+	movq	40(%rsp),%r8+.Lcrypton_chacha20_asm_ssse3:+	movq	%rsp,%r10++	testl	$2048,%r9d+	jnz	.Lcrypton_chacha20_asm_4xop+	cmpq	$128,%rdx+	je	.Lcrypton_chacha20_asm_128+	ja	.Lcrypton_chacha20_asm_4x++.Ldo_sse3_after_all:+	subq	$64+40,%rsp+	andq	$-16,%rsp+	movaps	%xmm6,-40(%r10)+	movaps	%xmm7,-24(%r10)+.Lssse3_body:+	movdqa	.Lsigma(%rip),%xmm0+	movdqu	(%rcx),%xmm1+	movdqu	16(%rcx),%xmm2+	movdqu	(%r8),%xmm3+	movdqa	.Lrot16(%rip),%xmm6+	movdqa	.Lrot24(%rip),%xmm7++	movdqa	%xmm0,0(%rsp)+	movdqa	%xmm1,16(%rsp)+	movdqa	%xmm2,32(%rsp)+	movdqa	%xmm3,48(%rsp)+	movq	$10,%r8+	jmp	.Loop_ssse3++.p2align	5+.Loop_outer_ssse3:+	movdqa	.Lone(%rip),%xmm3+	movdqa	0(%rsp),%xmm0+	movdqa	16(%rsp),%xmm1+	movdqa	32(%rsp),%xmm2+	paddd	48(%rsp),%xmm3+	movq	$10,%r8+	movdqa	%xmm3,48(%rsp)+	jmp	.Loop_ssse3++.p2align	5+.Loop_ssse3:+	paddd	%xmm1,%xmm0+	pxor	%xmm0,%xmm3+.byte	102,15,56,0,222+	paddd	%xmm3,%xmm2+	pxor	%xmm2,%xmm1+	movdqa	%xmm1,%xmm4+	psrld	$20,%xmm1+	pslld	$12,%xmm4+	por	%xmm4,%xmm1+	paddd	%xmm1,%xmm0+	pxor	%xmm0,%xmm3+.byte	102,15,56,0,223+	paddd	%xmm3,%xmm2+	pxor	%xmm2,%xmm1+	movdqa	%xmm1,%xmm4+	psrld	$25,%xmm1+	pslld	$7,%xmm4+	por	%xmm4,%xmm1+	pshufd	$78,%xmm2,%xmm2+	pshufd	$57,%xmm1,%xmm1+	pshufd	$147,%xmm3,%xmm3+	nop+	paddd	%xmm1,%xmm0+	pxor	%xmm0,%xmm3+.byte	102,15,56,0,222+	paddd	%xmm3,%xmm2+	pxor	%xmm2,%xmm1+	movdqa	%xmm1,%xmm4+	psrld	$20,%xmm1+	pslld	$12,%xmm4+	por	%xmm4,%xmm1+	paddd	%xmm1,%xmm0+	pxor	%xmm0,%xmm3+.byte	102,15,56,0,223+	paddd	%xmm3,%xmm2+	pxor	%xmm2,%xmm1+	movdqa	%xmm1,%xmm4+	psrld	$25,%xmm1+	pslld	$7,%xmm4+	por	%xmm4,%xmm1+	pshufd	$78,%xmm2,%xmm2+	pshufd	$147,%xmm1,%xmm1+	pshufd	$57,%xmm3,%xmm3+	decq	%r8+	jnz	.Loop_ssse3+	paddd	0(%rsp),%xmm0+	paddd	16(%rsp),%xmm1+	paddd	32(%rsp),%xmm2+	paddd	48(%rsp),%xmm3++	cmpq	$64,%rdx+	jb	.Ltail_ssse3++	movdqu	0(%rsi),%xmm4+	movdqu	16(%rsi),%xmm5+	pxor	%xmm4,%xmm0+	movdqu	32(%rsi),%xmm4+	pxor	%xmm5,%xmm1+	movdqu	48(%rsi),%xmm5+	leaq	64(%rsi),%rsi+	pxor	%xmm4,%xmm2+	pxor	%xmm5,%xmm3++	movdqu	%xmm0,0(%rdi)+	movdqu	%xmm1,16(%rdi)+	movdqu	%xmm2,32(%rdi)+	movdqu	%xmm3,48(%rdi)+	leaq	64(%rdi),%rdi++	subq	$64,%rdx+	jnz	.Loop_outer_ssse3++	jmp	.Ldone_ssse3++.p2align	4+.Ltail_ssse3:+	movdqa	%xmm0,0(%rsp)+	movdqa	%xmm1,16(%rsp)+	movdqa	%xmm2,32(%rsp)+	movdqa	%xmm3,48(%rsp)+	xorq	%r8,%r8++.Loop_tail_ssse3:+	movzbl	(%rsi,%r8,1),%eax+	movzbl	(%rsp,%r8,1),%ecx+	leaq	1(%r8),%r8+	xorl	%ecx,%eax+	movb	%al,-1(%rdi,%r8,1)+	decq	%rdx+	jnz	.Loop_tail_ssse3++.Ldone_ssse3:+	movaps	-40(%r10),%xmm6+	movaps	-24(%r10),%xmm7+	leaq	(%r10),%rsp++.Lssse3_epilogue:+	movq	8(%rsp),%rdi+	movq	16(%rsp),%rsi+	.byte	0xf3,0xc3++.LSEH_end_crypton_chacha20_asm_ssse3:+.def	crypton_chacha20_asm_128;	.scl 3;	.type 32;	.endef+.p2align	5+crypton_chacha20_asm_128:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%rax+.LSEH_begin_crypton_chacha20_asm_128:+++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	movq	%r9,%rcx+	movq	40(%rsp),%r8+.Lcrypton_chacha20_asm_128:+	movq	%rsp,%r10++	subq	$64+104,%rsp+	andq	$-16,%rsp+	movaps	%xmm6,-104(%r10)+	movaps	%xmm7,-88(%r10)+	movaps	%xmm8,-72(%r10)+	movaps	%xmm9,-56(%r10)+	movaps	%xmm10,-40(%r10)+	movaps	%xmm11,-24(%r10)+.L128_body:+	movdqa	.Lsigma(%rip),%xmm8+	movdqu	(%rcx),%xmm9+	movdqu	16(%rcx),%xmm2+	movdqu	(%r8),%xmm3+	movdqa	.Lone(%rip),%xmm1+	movdqa	.Lrot16(%rip),%xmm6+	movdqa	.Lrot24(%rip),%xmm7++	movdqa	%xmm8,%xmm10+	movdqa	%xmm8,0(%rsp)+	movdqa	%xmm9,%xmm11+	movdqa	%xmm9,16(%rsp)+	movdqa	%xmm2,%xmm0+	movdqa	%xmm2,32(%rsp)+	paddd	%xmm3,%xmm1+	movdqa	%xmm3,48(%rsp)+	movq	$10,%r8+	jmp	.Loop_128++.p2align	5+.Loop_128:+	paddd	%xmm9,%xmm8+	pxor	%xmm8,%xmm3+	paddd	%xmm11,%xmm10+	pxor	%xmm10,%xmm1+.byte	102,15,56,0,222+.byte	102,15,56,0,206+	paddd	%xmm3,%xmm2+	paddd	%xmm1,%xmm0+	pxor	%xmm2,%xmm9+	pxor	%xmm0,%xmm11+	movdqa	%xmm9,%xmm4+	psrld	$20,%xmm9+	movdqa	%xmm11,%xmm5+	pslld	$12,%xmm4+	psrld	$20,%xmm11+	por	%xmm4,%xmm9+	pslld	$12,%xmm5+	por	%xmm5,%xmm11+	paddd	%xmm9,%xmm8+	pxor	%xmm8,%xmm3+	paddd	%xmm11,%xmm10+	pxor	%xmm10,%xmm1+.byte	102,15,56,0,223+.byte	102,15,56,0,207+	paddd	%xmm3,%xmm2+	paddd	%xmm1,%xmm0+	pxor	%xmm2,%xmm9+	pxor	%xmm0,%xmm11+	movdqa	%xmm9,%xmm4+	psrld	$25,%xmm9+	movdqa	%xmm11,%xmm5+	pslld	$7,%xmm4+	psrld	$25,%xmm11+	por	%xmm4,%xmm9+	pslld	$7,%xmm5+	por	%xmm5,%xmm11+	pshufd	$78,%xmm2,%xmm2+	pshufd	$57,%xmm9,%xmm9+	pshufd	$147,%xmm3,%xmm3+	pshufd	$78,%xmm0,%xmm0+	pshufd	$57,%xmm11,%xmm11+	pshufd	$147,%xmm1,%xmm1+	paddd	%xmm9,%xmm8+	pxor	%xmm8,%xmm3+	paddd	%xmm11,%xmm10+	pxor	%xmm10,%xmm1+.byte	102,15,56,0,222+.byte	102,15,56,0,206+	paddd	%xmm3,%xmm2+	paddd	%xmm1,%xmm0+	pxor	%xmm2,%xmm9+	pxor	%xmm0,%xmm11+	movdqa	%xmm9,%xmm4+	psrld	$20,%xmm9+	movdqa	%xmm11,%xmm5+	pslld	$12,%xmm4+	psrld	$20,%xmm11+	por	%xmm4,%xmm9+	pslld	$12,%xmm5+	por	%xmm5,%xmm11+	paddd	%xmm9,%xmm8+	pxor	%xmm8,%xmm3+	paddd	%xmm11,%xmm10+	pxor	%xmm10,%xmm1+.byte	102,15,56,0,223+.byte	102,15,56,0,207+	paddd	%xmm3,%xmm2+	paddd	%xmm1,%xmm0+	pxor	%xmm2,%xmm9+	pxor	%xmm0,%xmm11+	movdqa	%xmm9,%xmm4+	psrld	$25,%xmm9+	movdqa	%xmm11,%xmm5+	pslld	$7,%xmm4+	psrld	$25,%xmm11+	por	%xmm4,%xmm9+	pslld	$7,%xmm5+	por	%xmm5,%xmm11+	pshufd	$78,%xmm2,%xmm2+	pshufd	$147,%xmm9,%xmm9+	pshufd	$57,%xmm3,%xmm3+	pshufd	$78,%xmm0,%xmm0+	pshufd	$147,%xmm11,%xmm11+	pshufd	$57,%xmm1,%xmm1+	decq	%r8+	jnz	.Loop_128+	paddd	0(%rsp),%xmm8+	paddd	16(%rsp),%xmm9+	paddd	32(%rsp),%xmm2+	paddd	48(%rsp),%xmm3+	paddd	.Lone(%rip),%xmm1+	paddd	0(%rsp),%xmm10+	paddd	16(%rsp),%xmm11+	paddd	32(%rsp),%xmm0+	paddd	48(%rsp),%xmm1++	movdqu	0(%rsi),%xmm4+	movdqu	16(%rsi),%xmm5+	pxor	%xmm4,%xmm8+	movdqu	32(%rsi),%xmm4+	pxor	%xmm5,%xmm9+	movdqu	48(%rsi),%xmm5+	pxor	%xmm4,%xmm2+	movdqu	64(%rsi),%xmm4+	pxor	%xmm5,%xmm3+	movdqu	80(%rsi),%xmm5+	pxor	%xmm4,%xmm10+	movdqu	96(%rsi),%xmm4+	pxor	%xmm5,%xmm11+	movdqu	112(%rsi),%xmm5+	pxor	%xmm4,%xmm0+	pxor	%xmm5,%xmm1++	movdqu	%xmm8,0(%rdi)+	movdqu	%xmm9,16(%rdi)+	movdqu	%xmm2,32(%rdi)+	movdqu	%xmm3,48(%rdi)+	movdqu	%xmm10,64(%rdi)+	movdqu	%xmm11,80(%rdi)+	movdqu	%xmm0,96(%rdi)+	movdqu	%xmm1,112(%rdi)+	movaps	-104(%r10),%xmm6+	movaps	-88(%r10),%xmm7+	movaps	-72(%r10),%xmm8+	movaps	-56(%r10),%xmm9+	movaps	-40(%r10),%xmm10+	movaps	-24(%r10),%xmm11+	leaq	(%r10),%rsp++.L128_epilogue:+	movq	8(%rsp),%rdi+	movq	16(%rsp),%rsi+	.byte	0xf3,0xc3++.LSEH_end_crypton_chacha20_asm_128:+.def	crypton_chacha20_asm_4x;	.scl 3;	.type 32;	.endef+.p2align	5+crypton_chacha20_asm_4x:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%rax+.LSEH_begin_crypton_chacha20_asm_4x:+++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	movq	%r9,%rcx+	movq	40(%rsp),%r8+.Lcrypton_chacha20_asm_4x:+	movq	%rsp,%r10++	movq	%r9,%r11+	shrq	$32,%r9+	testq	$32,%r9+	jnz	.Lcrypton_chacha20_asm_8x+	cmpq	$192,%rdx+	ja	.Lproceed4x++	andq	$71303168,%r11+	cmpq	$4194304,%r11+	je	.Ldo_sse3_after_all++.Lproceed4x:+	subq	$0x140+168,%rsp+	andq	$-16,%rsp+	movaps	%xmm6,-168(%r10)+	movaps	%xmm7,-152(%r10)+	movaps	%xmm8,-136(%r10)+	movaps	%xmm9,-120(%r10)+	movaps	%xmm10,-104(%r10)+	movaps	%xmm11,-88(%r10)+	movaps	%xmm12,-72(%r10)+	movaps	%xmm13,-56(%r10)+	movaps	%xmm14,-40(%r10)+	movaps	%xmm15,-24(%r10)+.L4x_body:+	movdqa	.Lsigma(%rip),%xmm11+	movdqu	(%rcx),%xmm15+	movdqu	16(%rcx),%xmm7+	movdqu	(%r8),%xmm3+	leaq	256(%rsp),%rcx+	leaq	.Lrot16(%rip),%r9+	leaq	.Lrot24(%rip),%r11++	pshufd	$0x00,%xmm11,%xmm8+	pshufd	$0x55,%xmm11,%xmm9+	movdqa	%xmm8,64(%rsp)+	pshufd	$0xaa,%xmm11,%xmm10+	movdqa	%xmm9,80(%rsp)+	pshufd	$0xff,%xmm11,%xmm11+	movdqa	%xmm10,96(%rsp)+	movdqa	%xmm11,112(%rsp)++	pshufd	$0x00,%xmm15,%xmm12+	pshufd	$0x55,%xmm15,%xmm13+	movdqa	%xmm12,128-256(%rcx)+	pshufd	$0xaa,%xmm15,%xmm14+	movdqa	%xmm13,144-256(%rcx)+	pshufd	$0xff,%xmm15,%xmm15+	movdqa	%xmm14,160-256(%rcx)+	movdqa	%xmm15,176-256(%rcx)++	pshufd	$0x00,%xmm7,%xmm4+	pshufd	$0x55,%xmm7,%xmm5+	movdqa	%xmm4,192-256(%rcx)+	pshufd	$0xaa,%xmm7,%xmm6+	movdqa	%xmm5,208-256(%rcx)+	pshufd	$0xff,%xmm7,%xmm7+	movdqa	%xmm6,224-256(%rcx)+	movdqa	%xmm7,240-256(%rcx)++	pshufd	$0x00,%xmm3,%xmm0+	pshufd	$0x55,%xmm3,%xmm1+	paddd	.Linc(%rip),%xmm0+	pshufd	$0xaa,%xmm3,%xmm2+	movdqa	%xmm1,272-256(%rcx)+	pshufd	$0xff,%xmm3,%xmm3+	movdqa	%xmm2,288-256(%rcx)+	movdqa	%xmm3,304-256(%rcx)++	jmp	.Loop_enter4x++.p2align	5+.Loop_outer4x:+	movdqa	64(%rsp),%xmm8+	movdqa	80(%rsp),%xmm9+	movdqa	96(%rsp),%xmm10+	movdqa	112(%rsp),%xmm11+	movdqa	128-256(%rcx),%xmm12+	movdqa	144-256(%rcx),%xmm13+	movdqa	160-256(%rcx),%xmm14+	movdqa	176-256(%rcx),%xmm15+	movdqa	192-256(%rcx),%xmm4+	movdqa	208-256(%rcx),%xmm5+	movdqa	224-256(%rcx),%xmm6+	movdqa	240-256(%rcx),%xmm7+	movdqa	256-256(%rcx),%xmm0+	movdqa	272-256(%rcx),%xmm1+	movdqa	288-256(%rcx),%xmm2+	movdqa	304-256(%rcx),%xmm3+	paddd	.Lfour(%rip),%xmm0++.Loop_enter4x:+	movdqa	%xmm6,32(%rsp)+	movdqa	%xmm7,48(%rsp)+	movdqa	(%r9),%xmm7+	movl	$10,%eax+	movdqa	%xmm0,256-256(%rcx)+	jmp	.Loop4x++.p2align	5+.Loop4x:+	paddd	%xmm12,%xmm8+	paddd	%xmm13,%xmm9+	pxor	%xmm8,%xmm0+	pxor	%xmm9,%xmm1+.byte	102,15,56,0,199+.byte	102,15,56,0,207+	paddd	%xmm0,%xmm4+	paddd	%xmm1,%xmm5+	pxor	%xmm4,%xmm12+	pxor	%xmm5,%xmm13+	movdqa	%xmm12,%xmm6+	pslld	$12,%xmm12+	psrld	$20,%xmm6+	movdqa	%xmm13,%xmm7+	pslld	$12,%xmm13+	por	%xmm6,%xmm12+	psrld	$20,%xmm7+	movdqa	(%r11),%xmm6+	por	%xmm7,%xmm13+	paddd	%xmm12,%xmm8+	paddd	%xmm13,%xmm9+	pxor	%xmm8,%xmm0+	pxor	%xmm9,%xmm1+.byte	102,15,56,0,198+.byte	102,15,56,0,206+	paddd	%xmm0,%xmm4+	paddd	%xmm1,%xmm5+	pxor	%xmm4,%xmm12+	pxor	%xmm5,%xmm13+	movdqa	%xmm12,%xmm7+	pslld	$7,%xmm12+	psrld	$25,%xmm7+	movdqa	%xmm13,%xmm6+	pslld	$7,%xmm13+	por	%xmm7,%xmm12+	psrld	$25,%xmm6+	movdqa	(%r9),%xmm7+	por	%xmm6,%xmm13+	movdqa	%xmm4,0(%rsp)+	movdqa	%xmm5,16(%rsp)+	movdqa	32(%rsp),%xmm4+	movdqa	48(%rsp),%xmm5+	paddd	%xmm14,%xmm10+	paddd	%xmm15,%xmm11+	pxor	%xmm10,%xmm2+	pxor	%xmm11,%xmm3+.byte	102,15,56,0,215+.byte	102,15,56,0,223+	paddd	%xmm2,%xmm4+	paddd	%xmm3,%xmm5+	pxor	%xmm4,%xmm14+	pxor	%xmm5,%xmm15+	movdqa	%xmm14,%xmm6+	pslld	$12,%xmm14+	psrld	$20,%xmm6+	movdqa	%xmm15,%xmm7+	pslld	$12,%xmm15+	por	%xmm6,%xmm14+	psrld	$20,%xmm7+	movdqa	(%r11),%xmm6+	por	%xmm7,%xmm15+	paddd	%xmm14,%xmm10+	paddd	%xmm15,%xmm11+	pxor	%xmm10,%xmm2+	pxor	%xmm11,%xmm3+.byte	102,15,56,0,214+.byte	102,15,56,0,222+	paddd	%xmm2,%xmm4+	paddd	%xmm3,%xmm5+	pxor	%xmm4,%xmm14+	pxor	%xmm5,%xmm15+	movdqa	%xmm14,%xmm7+	pslld	$7,%xmm14+	psrld	$25,%xmm7+	movdqa	%xmm15,%xmm6+	pslld	$7,%xmm15+	por	%xmm7,%xmm14+	psrld	$25,%xmm6+	movdqa	(%r9),%xmm7+	por	%xmm6,%xmm15+	paddd	%xmm13,%xmm8+	paddd	%xmm14,%xmm9+	pxor	%xmm8,%xmm3+	pxor	%xmm9,%xmm0+.byte	102,15,56,0,223+.byte	102,15,56,0,199+	paddd	%xmm3,%xmm4+	paddd	%xmm0,%xmm5+	pxor	%xmm4,%xmm13+	pxor	%xmm5,%xmm14+	movdqa	%xmm13,%xmm6+	pslld	$12,%xmm13+	psrld	$20,%xmm6+	movdqa	%xmm14,%xmm7+	pslld	$12,%xmm14+	por	%xmm6,%xmm13+	psrld	$20,%xmm7+	movdqa	(%r11),%xmm6+	por	%xmm7,%xmm14+	paddd	%xmm13,%xmm8+	paddd	%xmm14,%xmm9+	pxor	%xmm8,%xmm3+	pxor	%xmm9,%xmm0+.byte	102,15,56,0,222+.byte	102,15,56,0,198+	paddd	%xmm3,%xmm4+	paddd	%xmm0,%xmm5+	pxor	%xmm4,%xmm13+	pxor	%xmm5,%xmm14+	movdqa	%xmm13,%xmm7+	pslld	$7,%xmm13+	psrld	$25,%xmm7+	movdqa	%xmm14,%xmm6+	pslld	$7,%xmm14+	por	%xmm7,%xmm13+	psrld	$25,%xmm6+	movdqa	(%r9),%xmm7+	por	%xmm6,%xmm14+	movdqa	%xmm4,32(%rsp)+	movdqa	%xmm5,48(%rsp)+	movdqa	0(%rsp),%xmm4+	movdqa	16(%rsp),%xmm5+	paddd	%xmm15,%xmm10+	paddd	%xmm12,%xmm11+	pxor	%xmm10,%xmm1+	pxor	%xmm11,%xmm2+.byte	102,15,56,0,207+.byte	102,15,56,0,215+	paddd	%xmm1,%xmm4+	paddd	%xmm2,%xmm5+	pxor	%xmm4,%xmm15+	pxor	%xmm5,%xmm12+	movdqa	%xmm15,%xmm6+	pslld	$12,%xmm15+	psrld	$20,%xmm6+	movdqa	%xmm12,%xmm7+	pslld	$12,%xmm12+	por	%xmm6,%xmm15+	psrld	$20,%xmm7+	movdqa	(%r11),%xmm6+	por	%xmm7,%xmm12+	paddd	%xmm15,%xmm10+	paddd	%xmm12,%xmm11+	pxor	%xmm10,%xmm1+	pxor	%xmm11,%xmm2+.byte	102,15,56,0,206+.byte	102,15,56,0,214+	paddd	%xmm1,%xmm4+	paddd	%xmm2,%xmm5+	pxor	%xmm4,%xmm15+	pxor	%xmm5,%xmm12+	movdqa	%xmm15,%xmm7+	pslld	$7,%xmm15+	psrld	$25,%xmm7+	movdqa	%xmm12,%xmm6+	pslld	$7,%xmm12+	por	%xmm7,%xmm15+	psrld	$25,%xmm6+	movdqa	(%r9),%xmm7+	por	%xmm6,%xmm12+	decl	%eax+	jnz	.Loop4x++	paddd	64(%rsp),%xmm8+	paddd	80(%rsp),%xmm9+	paddd	96(%rsp),%xmm10+	paddd	112(%rsp),%xmm11++	movdqa	%xmm8,%xmm6+	punpckldq	%xmm9,%xmm8+	movdqa	%xmm10,%xmm7+	punpckldq	%xmm11,%xmm10+	punpckhdq	%xmm9,%xmm6+	punpckhdq	%xmm11,%xmm7+	movdqa	%xmm8,%xmm9+	punpcklqdq	%xmm10,%xmm8+	movdqa	%xmm6,%xmm11+	punpcklqdq	%xmm7,%xmm6+	punpckhqdq	%xmm10,%xmm9+	punpckhqdq	%xmm7,%xmm11+	paddd	128-256(%rcx),%xmm12+	paddd	144-256(%rcx),%xmm13+	paddd	160-256(%rcx),%xmm14+	paddd	176-256(%rcx),%xmm15++	movdqa	%xmm8,0(%rsp)+	movdqa	%xmm9,16(%rsp)+	movdqa	32(%rsp),%xmm8+	movdqa	48(%rsp),%xmm9++	movdqa	%xmm12,%xmm10+	punpckldq	%xmm13,%xmm12+	movdqa	%xmm14,%xmm7+	punpckldq	%xmm15,%xmm14+	punpckhdq	%xmm13,%xmm10+	punpckhdq	%xmm15,%xmm7+	movdqa	%xmm12,%xmm13+	punpcklqdq	%xmm14,%xmm12+	movdqa	%xmm10,%xmm15+	punpcklqdq	%xmm7,%xmm10+	punpckhqdq	%xmm14,%xmm13+	punpckhqdq	%xmm7,%xmm15+	paddd	192-256(%rcx),%xmm4+	paddd	208-256(%rcx),%xmm5+	paddd	224-256(%rcx),%xmm8+	paddd	240-256(%rcx),%xmm9++	movdqa	%xmm6,32(%rsp)+	movdqa	%xmm11,48(%rsp)++	movdqa	%xmm4,%xmm14+	punpckldq	%xmm5,%xmm4+	movdqa	%xmm8,%xmm7+	punpckldq	%xmm9,%xmm8+	punpckhdq	%xmm5,%xmm14+	punpckhdq	%xmm9,%xmm7+	movdqa	%xmm4,%xmm5+	punpcklqdq	%xmm8,%xmm4+	movdqa	%xmm14,%xmm9+	punpcklqdq	%xmm7,%xmm14+	punpckhqdq	%xmm8,%xmm5+	punpckhqdq	%xmm7,%xmm9+	paddd	256-256(%rcx),%xmm0+	paddd	272-256(%rcx),%xmm1+	paddd	288-256(%rcx),%xmm2+	paddd	304-256(%rcx),%xmm3++	movdqa	%xmm0,%xmm8+	punpckldq	%xmm1,%xmm0+	movdqa	%xmm2,%xmm7+	punpckldq	%xmm3,%xmm2+	punpckhdq	%xmm1,%xmm8+	punpckhdq	%xmm3,%xmm7+	movdqa	%xmm0,%xmm1+	punpcklqdq	%xmm2,%xmm0+	movdqa	%xmm8,%xmm3+	punpcklqdq	%xmm7,%xmm8+	punpckhqdq	%xmm2,%xmm1+	punpckhqdq	%xmm7,%xmm3+	cmpq	$256,%rdx+	jb	.Ltail4x++	movdqu	0(%rsi),%xmm6+	movdqu	16(%rsi),%xmm11+	movdqu	32(%rsi),%xmm2+	movdqu	48(%rsi),%xmm7+	pxor	0(%rsp),%xmm6+	pxor	%xmm12,%xmm11+	pxor	%xmm4,%xmm2+	pxor	%xmm0,%xmm7++	movdqu	%xmm6,0(%rdi)+	movdqu	64(%rsi),%xmm6+	movdqu	%xmm11,16(%rdi)+	movdqu	80(%rsi),%xmm11+	movdqu	%xmm2,32(%rdi)+	movdqu	96(%rsi),%xmm2+	movdqu	%xmm7,48(%rdi)+	movdqu	112(%rsi),%xmm7+	leaq	128(%rsi),%rsi+	pxor	16(%rsp),%xmm6+	pxor	%xmm13,%xmm11+	pxor	%xmm5,%xmm2+	pxor	%xmm1,%xmm7++	movdqu	%xmm6,64(%rdi)+	movdqu	0(%rsi),%xmm6+	movdqu	%xmm11,80(%rdi)+	movdqu	16(%rsi),%xmm11+	movdqu	%xmm2,96(%rdi)+	movdqu	32(%rsi),%xmm2+	movdqu	%xmm7,112(%rdi)+	leaq	128(%rdi),%rdi+	movdqu	48(%rsi),%xmm7+	pxor	32(%rsp),%xmm6+	pxor	%xmm10,%xmm11+	pxor	%xmm14,%xmm2+	pxor	%xmm8,%xmm7++	movdqu	%xmm6,0(%rdi)+	movdqu	64(%rsi),%xmm6+	movdqu	%xmm11,16(%rdi)+	movdqu	80(%rsi),%xmm11+	movdqu	%xmm2,32(%rdi)+	movdqu	96(%rsi),%xmm2+	movdqu	%xmm7,48(%rdi)+	movdqu	112(%rsi),%xmm7+	leaq	128(%rsi),%rsi+	pxor	48(%rsp),%xmm6+	pxor	%xmm15,%xmm11+	pxor	%xmm9,%xmm2+	pxor	%xmm3,%xmm7+	movdqu	%xmm6,64(%rdi)+	movdqu	%xmm11,80(%rdi)+	movdqu	%xmm2,96(%rdi)+	movdqu	%xmm7,112(%rdi)+	leaq	128(%rdi),%rdi++	subq	$256,%rdx+	jnz	.Loop_outer4x++	jmp	.Ldone4x++.Ltail4x:+	cmpq	$192,%rdx+	jae	.L192_or_more4x+	cmpq	$128,%rdx+	jae	.L128_or_more4x+	cmpq	$64,%rdx+	jae	.L64_or_more4x+++	xorq	%r9,%r9++	movdqa	%xmm12,16(%rsp)+	movdqa	%xmm4,32(%rsp)+	movdqa	%xmm0,48(%rsp)+	jmp	.Loop_tail4x++.p2align	5+.L64_or_more4x:+	movdqu	0(%rsi),%xmm6+	movdqu	16(%rsi),%xmm11+	movdqu	32(%rsi),%xmm2+	movdqu	48(%rsi),%xmm7+	pxor	0(%rsp),%xmm6+	pxor	%xmm12,%xmm11+	pxor	%xmm4,%xmm2+	pxor	%xmm0,%xmm7+	movdqu	%xmm6,0(%rdi)+	movdqu	%xmm11,16(%rdi)+	movdqu	%xmm2,32(%rdi)+	movdqu	%xmm7,48(%rdi)+	je	.Ldone4x++	movdqa	16(%rsp),%xmm6+	leaq	64(%rsi),%rsi+	xorq	%r9,%r9+	movdqa	%xmm6,0(%rsp)+	movdqa	%xmm13,16(%rsp)+	leaq	64(%rdi),%rdi+	movdqa	%xmm5,32(%rsp)+	subq	$64,%rdx+	movdqa	%xmm1,48(%rsp)+	jmp	.Loop_tail4x++.p2align	5+.L128_or_more4x:+	movdqu	0(%rsi),%xmm6+	movdqu	16(%rsi),%xmm11+	movdqu	32(%rsi),%xmm2+	movdqu	48(%rsi),%xmm7+	pxor	0(%rsp),%xmm6+	pxor	%xmm12,%xmm11+	pxor	%xmm4,%xmm2+	pxor	%xmm0,%xmm7++	movdqu	%xmm6,0(%rdi)+	movdqu	64(%rsi),%xmm6+	movdqu	%xmm11,16(%rdi)+	movdqu	80(%rsi),%xmm11+	movdqu	%xmm2,32(%rdi)+	movdqu	96(%rsi),%xmm2+	movdqu	%xmm7,48(%rdi)+	movdqu	112(%rsi),%xmm7+	pxor	16(%rsp),%xmm6+	pxor	%xmm13,%xmm11+	pxor	%xmm5,%xmm2+	pxor	%xmm1,%xmm7+	movdqu	%xmm6,64(%rdi)+	movdqu	%xmm11,80(%rdi)+	movdqu	%xmm2,96(%rdi)+	movdqu	%xmm7,112(%rdi)+	je	.Ldone4x++	movdqa	32(%rsp),%xmm6+	leaq	128(%rsi),%rsi+	xorq	%r9,%r9+	movdqa	%xmm6,0(%rsp)+	movdqa	%xmm10,16(%rsp)+	leaq	128(%rdi),%rdi+	movdqa	%xmm14,32(%rsp)+	subq	$128,%rdx+	movdqa	%xmm8,48(%rsp)+	jmp	.Loop_tail4x++.p2align	5+.L192_or_more4x:+	movdqu	0(%rsi),%xmm6+	movdqu	16(%rsi),%xmm11+	movdqu	32(%rsi),%xmm2+	movdqu	48(%rsi),%xmm7+	pxor	0(%rsp),%xmm6+	pxor	%xmm12,%xmm11+	pxor	%xmm4,%xmm2+	pxor	%xmm0,%xmm7++	movdqu	%xmm6,0(%rdi)+	movdqu	64(%rsi),%xmm6+	movdqu	%xmm11,16(%rdi)+	movdqu	80(%rsi),%xmm11+	movdqu	%xmm2,32(%rdi)+	movdqu	96(%rsi),%xmm2+	movdqu	%xmm7,48(%rdi)+	movdqu	112(%rsi),%xmm7+	leaq	128(%rsi),%rsi+	pxor	16(%rsp),%xmm6+	pxor	%xmm13,%xmm11+	pxor	%xmm5,%xmm2+	pxor	%xmm1,%xmm7++	movdqu	%xmm6,64(%rdi)+	movdqu	0(%rsi),%xmm6+	movdqu	%xmm11,80(%rdi)+	movdqu	16(%rsi),%xmm11+	movdqu	%xmm2,96(%rdi)+	movdqu	32(%rsi),%xmm2+	movdqu	%xmm7,112(%rdi)+	leaq	128(%rdi),%rdi+	movdqu	48(%rsi),%xmm7+	pxor	32(%rsp),%xmm6+	pxor	%xmm10,%xmm11+	pxor	%xmm14,%xmm2+	pxor	%xmm8,%xmm7+	movdqu	%xmm6,0(%rdi)+	movdqu	%xmm11,16(%rdi)+	movdqu	%xmm2,32(%rdi)+	movdqu	%xmm7,48(%rdi)+	je	.Ldone4x++	movdqa	48(%rsp),%xmm6+	leaq	64(%rsi),%rsi+	xorq	%r9,%r9+	movdqa	%xmm6,0(%rsp)+	movdqa	%xmm15,16(%rsp)+	leaq	64(%rdi),%rdi+	movdqa	%xmm9,32(%rsp)+	subq	$192,%rdx+	movdqa	%xmm3,48(%rsp)++.Loop_tail4x:+	movzbl	(%rsi,%r9,1),%eax+	movzbl	(%rsp,%r9,1),%ecx+	leaq	1(%r9),%r9+	xorl	%ecx,%eax+	movb	%al,-1(%rdi,%r9,1)+	decq	%rdx+	jnz	.Loop_tail4x++.Ldone4x:+	movaps	-168(%r10),%xmm6+	movaps	-152(%r10),%xmm7+	movaps	-136(%r10),%xmm8+	movaps	-120(%r10),%xmm9+	movaps	-104(%r10),%xmm10+	movaps	-88(%r10),%xmm11+	movaps	-72(%r10),%xmm12+	movaps	-56(%r10),%xmm13+	movaps	-40(%r10),%xmm14+	movaps	-24(%r10),%xmm15+	leaq	(%r10),%rsp++.L4x_epilogue:+	movq	8(%rsp),%rdi+	movq	16(%rsp),%rsi+	.byte	0xf3,0xc3++.LSEH_end_crypton_chacha20_asm_4x:+.def	crypton_chacha20_asm_4xop;	.scl 3;	.type 32;	.endef+.p2align	5+crypton_chacha20_asm_4xop:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%rax+.LSEH_begin_crypton_chacha20_asm_4xop:+++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	movq	%r9,%rcx+	movq	40(%rsp),%r8+.Lcrypton_chacha20_asm_4xop:+	movq	%rsp,%r10++	subq	$0x140+168,%rsp+	andq	$-16,%rsp+	movaps	%xmm6,-168(%r10)+	movaps	%xmm7,-152(%r10)+	movaps	%xmm8,-136(%r10)+	movaps	%xmm9,-120(%r10)+	movaps	%xmm10,-104(%r10)+	movaps	%xmm11,-88(%r10)+	movaps	%xmm12,-72(%r10)+	movaps	%xmm13,-56(%r10)+	movaps	%xmm14,-40(%r10)+	movaps	%xmm15,-24(%r10)+.L4xop_body:+	vzeroupper++	vmovdqa	.Lsigma(%rip),%xmm11+	vmovdqu	(%rcx),%xmm3+	vmovdqu	16(%rcx),%xmm15+	vmovdqu	(%r8),%xmm7+	leaq	256(%rsp),%rcx++	vpshufd	$0x00,%xmm11,%xmm8+	vpshufd	$0x55,%xmm11,%xmm9+	vmovdqa	%xmm8,64(%rsp)+	vpshufd	$0xaa,%xmm11,%xmm10+	vmovdqa	%xmm9,80(%rsp)+	vpshufd	$0xff,%xmm11,%xmm11+	vmovdqa	%xmm10,96(%rsp)+	vmovdqa	%xmm11,112(%rsp)++	vpshufd	$0x00,%xmm3,%xmm0+	vpshufd	$0x55,%xmm3,%xmm1+	vmovdqa	%xmm0,128-256(%rcx)+	vpshufd	$0xaa,%xmm3,%xmm2+	vmovdqa	%xmm1,144-256(%rcx)+	vpshufd	$0xff,%xmm3,%xmm3+	vmovdqa	%xmm2,160-256(%rcx)+	vmovdqa	%xmm3,176-256(%rcx)++	vpshufd	$0x00,%xmm15,%xmm12+	vpshufd	$0x55,%xmm15,%xmm13+	vmovdqa	%xmm12,192-256(%rcx)+	vpshufd	$0xaa,%xmm15,%xmm14+	vmovdqa	%xmm13,208-256(%rcx)+	vpshufd	$0xff,%xmm15,%xmm15+	vmovdqa	%xmm14,224-256(%rcx)+	vmovdqa	%xmm15,240-256(%rcx)++	vpshufd	$0x00,%xmm7,%xmm4+	vpshufd	$0x55,%xmm7,%xmm5+	vpaddd	.Linc(%rip),%xmm4,%xmm4+	vpshufd	$0xaa,%xmm7,%xmm6+	vmovdqa	%xmm5,272-256(%rcx)+	vpshufd	$0xff,%xmm7,%xmm7+	vmovdqa	%xmm6,288-256(%rcx)+	vmovdqa	%xmm7,304-256(%rcx)++	jmp	.Loop_enter4xop++.p2align	5+.Loop_outer4xop:+	vmovdqa	64(%rsp),%xmm8+	vmovdqa	80(%rsp),%xmm9+	vmovdqa	96(%rsp),%xmm10+	vmovdqa	112(%rsp),%xmm11+	vmovdqa	128-256(%rcx),%xmm0+	vmovdqa	144-256(%rcx),%xmm1+	vmovdqa	160-256(%rcx),%xmm2+	vmovdqa	176-256(%rcx),%xmm3+	vmovdqa	192-256(%rcx),%xmm12+	vmovdqa	208-256(%rcx),%xmm13+	vmovdqa	224-256(%rcx),%xmm14+	vmovdqa	240-256(%rcx),%xmm15+	vmovdqa	256-256(%rcx),%xmm4+	vmovdqa	272-256(%rcx),%xmm5+	vmovdqa	288-256(%rcx),%xmm6+	vmovdqa	304-256(%rcx),%xmm7+	vpaddd	.Lfour(%rip),%xmm4,%xmm4++.Loop_enter4xop:+	movl	$10,%eax+	vmovdqa	%xmm4,256-256(%rcx)+	jmp	.Loop4xop++.p2align	5+.Loop4xop:+	vpaddd	%xmm0,%xmm8,%xmm8+	vpaddd	%xmm1,%xmm9,%xmm9+	vpaddd	%xmm2,%xmm10,%xmm10+	vpaddd	%xmm3,%xmm11,%xmm11+	vpxor	%xmm4,%xmm8,%xmm4+	vpxor	%xmm5,%xmm9,%xmm5+	vpxor	%xmm6,%xmm10,%xmm6+	vpxor	%xmm7,%xmm11,%xmm7+.byte	143,232,120,194,228,16+.byte	143,232,120,194,237,16+.byte	143,232,120,194,246,16+.byte	143,232,120,194,255,16+	vpaddd	%xmm4,%xmm12,%xmm12+	vpaddd	%xmm5,%xmm13,%xmm13+	vpaddd	%xmm6,%xmm14,%xmm14+	vpaddd	%xmm7,%xmm15,%xmm15+	vpxor	%xmm0,%xmm12,%xmm0+	vpxor	%xmm1,%xmm13,%xmm1+	vpxor	%xmm14,%xmm2,%xmm2+	vpxor	%xmm15,%xmm3,%xmm3+.byte	143,232,120,194,192,12+.byte	143,232,120,194,201,12+.byte	143,232,120,194,210,12+.byte	143,232,120,194,219,12+	vpaddd	%xmm8,%xmm0,%xmm8+	vpaddd	%xmm9,%xmm1,%xmm9+	vpaddd	%xmm2,%xmm10,%xmm10+	vpaddd	%xmm3,%xmm11,%xmm11+	vpxor	%xmm4,%xmm8,%xmm4+	vpxor	%xmm5,%xmm9,%xmm5+	vpxor	%xmm6,%xmm10,%xmm6+	vpxor	%xmm7,%xmm11,%xmm7+.byte	143,232,120,194,228,8+.byte	143,232,120,194,237,8+.byte	143,232,120,194,246,8+.byte	143,232,120,194,255,8+	vpaddd	%xmm4,%xmm12,%xmm12+	vpaddd	%xmm5,%xmm13,%xmm13+	vpaddd	%xmm6,%xmm14,%xmm14+	vpaddd	%xmm7,%xmm15,%xmm15+	vpxor	%xmm0,%xmm12,%xmm0+	vpxor	%xmm1,%xmm13,%xmm1+	vpxor	%xmm14,%xmm2,%xmm2+	vpxor	%xmm15,%xmm3,%xmm3+.byte	143,232,120,194,192,7+.byte	143,232,120,194,201,7+.byte	143,232,120,194,210,7+.byte	143,232,120,194,219,7+	vpaddd	%xmm1,%xmm8,%xmm8+	vpaddd	%xmm2,%xmm9,%xmm9+	vpaddd	%xmm3,%xmm10,%xmm10+	vpaddd	%xmm0,%xmm11,%xmm11+	vpxor	%xmm7,%xmm8,%xmm7+	vpxor	%xmm4,%xmm9,%xmm4+	vpxor	%xmm5,%xmm10,%xmm5+	vpxor	%xmm6,%xmm11,%xmm6+.byte	143,232,120,194,255,16+.byte	143,232,120,194,228,16+.byte	143,232,120,194,237,16+.byte	143,232,120,194,246,16+	vpaddd	%xmm7,%xmm14,%xmm14+	vpaddd	%xmm4,%xmm15,%xmm15+	vpaddd	%xmm5,%xmm12,%xmm12+	vpaddd	%xmm6,%xmm13,%xmm13+	vpxor	%xmm1,%xmm14,%xmm1+	vpxor	%xmm2,%xmm15,%xmm2+	vpxor	%xmm12,%xmm3,%xmm3+	vpxor	%xmm13,%xmm0,%xmm0+.byte	143,232,120,194,201,12+.byte	143,232,120,194,210,12+.byte	143,232,120,194,219,12+.byte	143,232,120,194,192,12+	vpaddd	%xmm8,%xmm1,%xmm8+	vpaddd	%xmm9,%xmm2,%xmm9+	vpaddd	%xmm3,%xmm10,%xmm10+	vpaddd	%xmm0,%xmm11,%xmm11+	vpxor	%xmm7,%xmm8,%xmm7+	vpxor	%xmm4,%xmm9,%xmm4+	vpxor	%xmm5,%xmm10,%xmm5+	vpxor	%xmm6,%xmm11,%xmm6+.byte	143,232,120,194,255,8+.byte	143,232,120,194,228,8+.byte	143,232,120,194,237,8+.byte	143,232,120,194,246,8+	vpaddd	%xmm7,%xmm14,%xmm14+	vpaddd	%xmm4,%xmm15,%xmm15+	vpaddd	%xmm5,%xmm12,%xmm12+	vpaddd	%xmm6,%xmm13,%xmm13+	vpxor	%xmm1,%xmm14,%xmm1+	vpxor	%xmm2,%xmm15,%xmm2+	vpxor	%xmm12,%xmm3,%xmm3+	vpxor	%xmm13,%xmm0,%xmm0+.byte	143,232,120,194,201,7+.byte	143,232,120,194,210,7+.byte	143,232,120,194,219,7+.byte	143,232,120,194,192,7+	decl	%eax+	jnz	.Loop4xop++	vpaddd	64(%rsp),%xmm8,%xmm8+	vpaddd	80(%rsp),%xmm9,%xmm9+	vpaddd	96(%rsp),%xmm10,%xmm10+	vpaddd	112(%rsp),%xmm11,%xmm11++	vmovdqa	%xmm14,32(%rsp)+	vmovdqa	%xmm15,48(%rsp)++	vpunpckldq	%xmm9,%xmm8,%xmm14+	vpunpckldq	%xmm11,%xmm10,%xmm15+	vpunpckhdq	%xmm9,%xmm8,%xmm8+	vpunpckhdq	%xmm11,%xmm10,%xmm10+	vpunpcklqdq	%xmm15,%xmm14,%xmm9+	vpunpckhqdq	%xmm15,%xmm14,%xmm14+	vpunpcklqdq	%xmm10,%xmm8,%xmm11+	vpunpckhqdq	%xmm10,%xmm8,%xmm8+	vpaddd	128-256(%rcx),%xmm0,%xmm0+	vpaddd	144-256(%rcx),%xmm1,%xmm1+	vpaddd	160-256(%rcx),%xmm2,%xmm2+	vpaddd	176-256(%rcx),%xmm3,%xmm3++	vmovdqa	%xmm9,0(%rsp)+	vmovdqa	%xmm14,16(%rsp)+	vmovdqa	32(%rsp),%xmm9+	vmovdqa	48(%rsp),%xmm14++	vpunpckldq	%xmm1,%xmm0,%xmm10+	vpunpckldq	%xmm3,%xmm2,%xmm15+	vpunpckhdq	%xmm1,%xmm0,%xmm0+	vpunpckhdq	%xmm3,%xmm2,%xmm2+	vpunpcklqdq	%xmm15,%xmm10,%xmm1+	vpunpckhqdq	%xmm15,%xmm10,%xmm10+	vpunpcklqdq	%xmm2,%xmm0,%xmm3+	vpunpckhqdq	%xmm2,%xmm0,%xmm0+	vpaddd	192-256(%rcx),%xmm12,%xmm12+	vpaddd	208-256(%rcx),%xmm13,%xmm13+	vpaddd	224-256(%rcx),%xmm9,%xmm9+	vpaddd	240-256(%rcx),%xmm14,%xmm14++	vpunpckldq	%xmm13,%xmm12,%xmm2+	vpunpckldq	%xmm14,%xmm9,%xmm15+	vpunpckhdq	%xmm13,%xmm12,%xmm12+	vpunpckhdq	%xmm14,%xmm9,%xmm9+	vpunpcklqdq	%xmm15,%xmm2,%xmm13+	vpunpckhqdq	%xmm15,%xmm2,%xmm2+	vpunpcklqdq	%xmm9,%xmm12,%xmm14+	vpunpckhqdq	%xmm9,%xmm12,%xmm12+	vpaddd	256-256(%rcx),%xmm4,%xmm4+	vpaddd	272-256(%rcx),%xmm5,%xmm5+	vpaddd	288-256(%rcx),%xmm6,%xmm6+	vpaddd	304-256(%rcx),%xmm7,%xmm7++	vpunpckldq	%xmm5,%xmm4,%xmm9+	vpunpckldq	%xmm7,%xmm6,%xmm15+	vpunpckhdq	%xmm5,%xmm4,%xmm4+	vpunpckhdq	%xmm7,%xmm6,%xmm6+	vpunpcklqdq	%xmm15,%xmm9,%xmm5+	vpunpckhqdq	%xmm15,%xmm9,%xmm9+	vpunpcklqdq	%xmm6,%xmm4,%xmm7+	vpunpckhqdq	%xmm6,%xmm4,%xmm4+	vmovdqa	0(%rsp),%xmm6+	vmovdqa	16(%rsp),%xmm15++	cmpq	$256,%rdx+	jb	.Ltail4xop++	vpxor	0(%rsi),%xmm6,%xmm6+	vpxor	16(%rsi),%xmm1,%xmm1+	vpxor	32(%rsi),%xmm13,%xmm13+	vpxor	48(%rsi),%xmm5,%xmm5+	vpxor	64(%rsi),%xmm15,%xmm15+	vpxor	80(%rsi),%xmm10,%xmm10+	vpxor	96(%rsi),%xmm2,%xmm2+	vpxor	112(%rsi),%xmm9,%xmm9+	leaq	128(%rsi),%rsi+	vpxor	0(%rsi),%xmm11,%xmm11+	vpxor	16(%rsi),%xmm3,%xmm3+	vpxor	32(%rsi),%xmm14,%xmm14+	vpxor	48(%rsi),%xmm7,%xmm7+	vpxor	64(%rsi),%xmm8,%xmm8+	vpxor	80(%rsi),%xmm0,%xmm0+	vpxor	96(%rsi),%xmm12,%xmm12+	vpxor	112(%rsi),%xmm4,%xmm4+	leaq	128(%rsi),%rsi++	vmovdqu	%xmm6,0(%rdi)+	vmovdqu	%xmm1,16(%rdi)+	vmovdqu	%xmm13,32(%rdi)+	vmovdqu	%xmm5,48(%rdi)+	vmovdqu	%xmm15,64(%rdi)+	vmovdqu	%xmm10,80(%rdi)+	vmovdqu	%xmm2,96(%rdi)+	vmovdqu	%xmm9,112(%rdi)+	leaq	128(%rdi),%rdi+	vmovdqu	%xmm11,0(%rdi)+	vmovdqu	%xmm3,16(%rdi)+	vmovdqu	%xmm14,32(%rdi)+	vmovdqu	%xmm7,48(%rdi)+	vmovdqu	%xmm8,64(%rdi)+	vmovdqu	%xmm0,80(%rdi)+	vmovdqu	%xmm12,96(%rdi)+	vmovdqu	%xmm4,112(%rdi)+	leaq	128(%rdi),%rdi++	subq	$256,%rdx+	jnz	.Loop_outer4xop++	jmp	.Ldone4xop++.p2align	5+.Ltail4xop:+	cmpq	$192,%rdx+	jae	.L192_or_more4xop+	cmpq	$128,%rdx+	jae	.L128_or_more4xop+	cmpq	$64,%rdx+	jae	.L64_or_more4xop++	xorq	%r9,%r9+	vmovdqa	%xmm6,0(%rsp)+	vmovdqa	%xmm1,16(%rsp)+	vmovdqa	%xmm13,32(%rsp)+	vmovdqa	%xmm5,48(%rsp)+	jmp	.Loop_tail4xop++.p2align	5+.L64_or_more4xop:+	vpxor	0(%rsi),%xmm6,%xmm6+	vpxor	16(%rsi),%xmm1,%xmm1+	vpxor	32(%rsi),%xmm13,%xmm13+	vpxor	48(%rsi),%xmm5,%xmm5+	vmovdqu	%xmm6,0(%rdi)+	vmovdqu	%xmm1,16(%rdi)+	vmovdqu	%xmm13,32(%rdi)+	vmovdqu	%xmm5,48(%rdi)+	je	.Ldone4xop++	leaq	64(%rsi),%rsi+	vmovdqa	%xmm15,0(%rsp)+	xorq	%r9,%r9+	vmovdqa	%xmm10,16(%rsp)+	leaq	64(%rdi),%rdi+	vmovdqa	%xmm2,32(%rsp)+	subq	$64,%rdx+	vmovdqa	%xmm9,48(%rsp)+	jmp	.Loop_tail4xop++.p2align	5+.L128_or_more4xop:+	vpxor	0(%rsi),%xmm6,%xmm6+	vpxor	16(%rsi),%xmm1,%xmm1+	vpxor	32(%rsi),%xmm13,%xmm13+	vpxor	48(%rsi),%xmm5,%xmm5+	vpxor	64(%rsi),%xmm15,%xmm15+	vpxor	80(%rsi),%xmm10,%xmm10+	vpxor	96(%rsi),%xmm2,%xmm2+	vpxor	112(%rsi),%xmm9,%xmm9++	vmovdqu	%xmm6,0(%rdi)+	vmovdqu	%xmm1,16(%rdi)+	vmovdqu	%xmm13,32(%rdi)+	vmovdqu	%xmm5,48(%rdi)+	vmovdqu	%xmm15,64(%rdi)+	vmovdqu	%xmm10,80(%rdi)+	vmovdqu	%xmm2,96(%rdi)+	vmovdqu	%xmm9,112(%rdi)+	je	.Ldone4xop++	leaq	128(%rsi),%rsi+	vmovdqa	%xmm11,0(%rsp)+	xorq	%r9,%r9+	vmovdqa	%xmm3,16(%rsp)+	leaq	128(%rdi),%rdi+	vmovdqa	%xmm14,32(%rsp)+	subq	$128,%rdx+	vmovdqa	%xmm7,48(%rsp)+	jmp	.Loop_tail4xop++.p2align	5+.L192_or_more4xop:+	vpxor	0(%rsi),%xmm6,%xmm6+	vpxor	16(%rsi),%xmm1,%xmm1+	vpxor	32(%rsi),%xmm13,%xmm13+	vpxor	48(%rsi),%xmm5,%xmm5+	vpxor	64(%rsi),%xmm15,%xmm15+	vpxor	80(%rsi),%xmm10,%xmm10+	vpxor	96(%rsi),%xmm2,%xmm2+	vpxor	112(%rsi),%xmm9,%xmm9+	leaq	128(%rsi),%rsi+	vpxor	0(%rsi),%xmm11,%xmm11+	vpxor	16(%rsi),%xmm3,%xmm3+	vpxor	32(%rsi),%xmm14,%xmm14+	vpxor	48(%rsi),%xmm7,%xmm7++	vmovdqu	%xmm6,0(%rdi)+	vmovdqu	%xmm1,16(%rdi)+	vmovdqu	%xmm13,32(%rdi)+	vmovdqu	%xmm5,48(%rdi)+	vmovdqu	%xmm15,64(%rdi)+	vmovdqu	%xmm10,80(%rdi)+	vmovdqu	%xmm2,96(%rdi)+	vmovdqu	%xmm9,112(%rdi)+	leaq	128(%rdi),%rdi+	vmovdqu	%xmm11,0(%rdi)+	vmovdqu	%xmm3,16(%rdi)+	vmovdqu	%xmm14,32(%rdi)+	vmovdqu	%xmm7,48(%rdi)+	je	.Ldone4xop++	leaq	64(%rsi),%rsi+	vmovdqa	%xmm8,0(%rsp)+	xorq	%r9,%r9+	vmovdqa	%xmm0,16(%rsp)+	leaq	64(%rdi),%rdi+	vmovdqa	%xmm12,32(%rsp)+	subq	$192,%rdx+	vmovdqa	%xmm4,48(%rsp)++.Loop_tail4xop:+	movzbl	(%rsi,%r9,1),%eax+	movzbl	(%rsp,%r9,1),%ecx+	leaq	1(%r9),%r9+	xorl	%ecx,%eax+	movb	%al,-1(%rdi,%r9,1)+	decq	%rdx+	jnz	.Loop_tail4xop++.Ldone4xop:+	vzeroupper+	movaps	-168(%r10),%xmm6+	movaps	-152(%r10),%xmm7+	movaps	-136(%r10),%xmm8+	movaps	-120(%r10),%xmm9+	movaps	-104(%r10),%xmm10+	movaps	-88(%r10),%xmm11+	movaps	-72(%r10),%xmm12+	movaps	-56(%r10),%xmm13+	movaps	-40(%r10),%xmm14+	movaps	-24(%r10),%xmm15+	leaq	(%r10),%rsp++.L4xop_epilogue:+	movq	8(%rsp),%rdi+	movq	16(%rsp),%rsi+	.byte	0xf3,0xc3++.LSEH_end_crypton_chacha20_asm_4xop:+.def	crypton_chacha20_asm_avx2;	.scl 3;	.type 32;	.endef+.p2align	5+crypton_chacha20_asm_avx2:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%rax+.LSEH_begin_crypton_chacha20_asm_avx2:+++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	movq	%r9,%rcx+	movq	40(%rsp),%r8+.Lcrypton_chacha20_asm_8x:+	movq	%rsp,%r10++	subq	$0x280+168,%rsp+	andq	$-32,%rsp+	movaps	%xmm6,-168(%r10)+	movaps	%xmm7,-152(%r10)+	movaps	%xmm8,-136(%r10)+	movaps	%xmm9,-120(%r10)+	movaps	%xmm10,-104(%r10)+	movaps	%xmm11,-88(%r10)+	movaps	%xmm12,-72(%r10)+	movaps	%xmm13,-56(%r10)+	movaps	%xmm14,-40(%r10)+	movaps	%xmm15,-24(%r10)+.Lavx2_body:+	vzeroupper+++++++++++	vbroadcasti128	.Lsigma(%rip),%ymm11+	vbroadcasti128	(%rcx),%ymm3+	vbroadcasti128	16(%rcx),%ymm15+	vbroadcasti128	(%r8),%ymm7+	leaq	256(%rsp),%rcx+	leaq	512(%rsp),%rax+	leaq	.Lrot16(%rip),%r9+	leaq	.Lrot24(%rip),%r11++	vpshufd	$0x00,%ymm11,%ymm8+	vpshufd	$0x55,%ymm11,%ymm9+	vmovdqa	%ymm8,128-256(%rcx)+	vpshufd	$0xaa,%ymm11,%ymm10+	vmovdqa	%ymm9,160-256(%rcx)+	vpshufd	$0xff,%ymm11,%ymm11+	vmovdqa	%ymm10,192-256(%rcx)+	vmovdqa	%ymm11,224-256(%rcx)++	vpshufd	$0x00,%ymm3,%ymm0+	vpshufd	$0x55,%ymm3,%ymm1+	vmovdqa	%ymm0,256-256(%rcx)+	vpshufd	$0xaa,%ymm3,%ymm2+	vmovdqa	%ymm1,288-256(%rcx)+	vpshufd	$0xff,%ymm3,%ymm3+	vmovdqa	%ymm2,320-256(%rcx)+	vmovdqa	%ymm3,352-256(%rcx)++	vpshufd	$0x00,%ymm15,%ymm12+	vpshufd	$0x55,%ymm15,%ymm13+	vmovdqa	%ymm12,384-512(%rax)+	vpshufd	$0xaa,%ymm15,%ymm14+	vmovdqa	%ymm13,416-512(%rax)+	vpshufd	$0xff,%ymm15,%ymm15+	vmovdqa	%ymm14,448-512(%rax)+	vmovdqa	%ymm15,480-512(%rax)++	vpshufd	$0x00,%ymm7,%ymm4+	vpshufd	$0x55,%ymm7,%ymm5+	vpaddd	.Lincy(%rip),%ymm4,%ymm4+	vpshufd	$0xaa,%ymm7,%ymm6+	vmovdqa	%ymm5,544-512(%rax)+	vpshufd	$0xff,%ymm7,%ymm7+	vmovdqa	%ymm6,576-512(%rax)+	vmovdqa	%ymm7,608-512(%rax)++	jmp	.Loop_enter8x++.p2align	5+.Loop_outer8x:+	vmovdqa	128-256(%rcx),%ymm8+	vmovdqa	160-256(%rcx),%ymm9+	vmovdqa	192-256(%rcx),%ymm10+	vmovdqa	224-256(%rcx),%ymm11+	vmovdqa	256-256(%rcx),%ymm0+	vmovdqa	288-256(%rcx),%ymm1+	vmovdqa	320-256(%rcx),%ymm2+	vmovdqa	352-256(%rcx),%ymm3+	vmovdqa	384-512(%rax),%ymm12+	vmovdqa	416-512(%rax),%ymm13+	vmovdqa	448-512(%rax),%ymm14+	vmovdqa	480-512(%rax),%ymm15+	vmovdqa	512-512(%rax),%ymm4+	vmovdqa	544-512(%rax),%ymm5+	vmovdqa	576-512(%rax),%ymm6+	vmovdqa	608-512(%rax),%ymm7+	vpaddd	.Leight(%rip),%ymm4,%ymm4++.Loop_enter8x:+	vmovdqa	%ymm14,64(%rsp)+	vmovdqa	%ymm15,96(%rsp)+	vbroadcasti128	(%r9),%ymm15+	vmovdqa	%ymm4,512-512(%rax)+	movl	$10,%eax+	jmp	.Loop8x++.p2align	5+.Loop8x:+	vpaddd	%ymm0,%ymm8,%ymm8+	vpxor	%ymm4,%ymm8,%ymm4+	vpshufb	%ymm15,%ymm4,%ymm4+	vpaddd	%ymm1,%ymm9,%ymm9+	vpxor	%ymm5,%ymm9,%ymm5+	vpshufb	%ymm15,%ymm5,%ymm5+	vpaddd	%ymm4,%ymm12,%ymm12+	vpxor	%ymm0,%ymm12,%ymm0+	vpslld	$12,%ymm0,%ymm14+	vpsrld	$20,%ymm0,%ymm0+	vpor	%ymm0,%ymm14,%ymm0+	vbroadcasti128	(%r11),%ymm14+	vpaddd	%ymm5,%ymm13,%ymm13+	vpxor	%ymm1,%ymm13,%ymm1+	vpslld	$12,%ymm1,%ymm15+	vpsrld	$20,%ymm1,%ymm1+	vpor	%ymm1,%ymm15,%ymm1+	vpaddd	%ymm0,%ymm8,%ymm8+	vpxor	%ymm4,%ymm8,%ymm4+	vpshufb	%ymm14,%ymm4,%ymm4+	vpaddd	%ymm1,%ymm9,%ymm9+	vpxor	%ymm5,%ymm9,%ymm5+	vpshufb	%ymm14,%ymm5,%ymm5+	vpaddd	%ymm4,%ymm12,%ymm12+	vpxor	%ymm0,%ymm12,%ymm0+	vpslld	$7,%ymm0,%ymm15+	vpsrld	$25,%ymm0,%ymm0+	vpor	%ymm0,%ymm15,%ymm0+	vbroadcasti128	(%r9),%ymm15+	vpaddd	%ymm5,%ymm13,%ymm13+	vpxor	%ymm1,%ymm13,%ymm1+	vpslld	$7,%ymm1,%ymm14+	vpsrld	$25,%ymm1,%ymm1+	vpor	%ymm1,%ymm14,%ymm1+	vmovdqa	%ymm12,0(%rsp)+	vmovdqa	%ymm13,32(%rsp)+	vmovdqa	64(%rsp),%ymm12+	vmovdqa	96(%rsp),%ymm13+	vpaddd	%ymm2,%ymm10,%ymm10+	vpxor	%ymm6,%ymm10,%ymm6+	vpshufb	%ymm15,%ymm6,%ymm6+	vpaddd	%ymm3,%ymm11,%ymm11+	vpxor	%ymm7,%ymm11,%ymm7+	vpshufb	%ymm15,%ymm7,%ymm7+	vpaddd	%ymm6,%ymm12,%ymm12+	vpxor	%ymm2,%ymm12,%ymm2+	vpslld	$12,%ymm2,%ymm14+	vpsrld	$20,%ymm2,%ymm2+	vpor	%ymm2,%ymm14,%ymm2+	vbroadcasti128	(%r11),%ymm14+	vpaddd	%ymm7,%ymm13,%ymm13+	vpxor	%ymm3,%ymm13,%ymm3+	vpslld	$12,%ymm3,%ymm15+	vpsrld	$20,%ymm3,%ymm3+	vpor	%ymm3,%ymm15,%ymm3+	vpaddd	%ymm2,%ymm10,%ymm10+	vpxor	%ymm6,%ymm10,%ymm6+	vpshufb	%ymm14,%ymm6,%ymm6+	vpaddd	%ymm3,%ymm11,%ymm11+	vpxor	%ymm7,%ymm11,%ymm7+	vpshufb	%ymm14,%ymm7,%ymm7+	vpaddd	%ymm6,%ymm12,%ymm12+	vpxor	%ymm2,%ymm12,%ymm2+	vpslld	$7,%ymm2,%ymm15+	vpsrld	$25,%ymm2,%ymm2+	vpor	%ymm2,%ymm15,%ymm2+	vbroadcasti128	(%r9),%ymm15+	vpaddd	%ymm7,%ymm13,%ymm13+	vpxor	%ymm3,%ymm13,%ymm3+	vpslld	$7,%ymm3,%ymm14+	vpsrld	$25,%ymm3,%ymm3+	vpor	%ymm3,%ymm14,%ymm3+	vpaddd	%ymm1,%ymm8,%ymm8+	vpxor	%ymm7,%ymm8,%ymm7+	vpshufb	%ymm15,%ymm7,%ymm7+	vpaddd	%ymm2,%ymm9,%ymm9+	vpxor	%ymm4,%ymm9,%ymm4+	vpshufb	%ymm15,%ymm4,%ymm4+	vpaddd	%ymm7,%ymm12,%ymm12+	vpxor	%ymm1,%ymm12,%ymm1+	vpslld	$12,%ymm1,%ymm14+	vpsrld	$20,%ymm1,%ymm1+	vpor	%ymm1,%ymm14,%ymm1+	vbroadcasti128	(%r11),%ymm14+	vpaddd	%ymm4,%ymm13,%ymm13+	vpxor	%ymm2,%ymm13,%ymm2+	vpslld	$12,%ymm2,%ymm15+	vpsrld	$20,%ymm2,%ymm2+	vpor	%ymm2,%ymm15,%ymm2+	vpaddd	%ymm1,%ymm8,%ymm8+	vpxor	%ymm7,%ymm8,%ymm7+	vpshufb	%ymm14,%ymm7,%ymm7+	vpaddd	%ymm2,%ymm9,%ymm9+	vpxor	%ymm4,%ymm9,%ymm4+	vpshufb	%ymm14,%ymm4,%ymm4+	vpaddd	%ymm7,%ymm12,%ymm12+	vpxor	%ymm1,%ymm12,%ymm1+	vpslld	$7,%ymm1,%ymm15+	vpsrld	$25,%ymm1,%ymm1+	vpor	%ymm1,%ymm15,%ymm1+	vbroadcasti128	(%r9),%ymm15+	vpaddd	%ymm4,%ymm13,%ymm13+	vpxor	%ymm2,%ymm13,%ymm2+	vpslld	$7,%ymm2,%ymm14+	vpsrld	$25,%ymm2,%ymm2+	vpor	%ymm2,%ymm14,%ymm2+	vmovdqa	%ymm12,64(%rsp)+	vmovdqa	%ymm13,96(%rsp)+	vmovdqa	0(%rsp),%ymm12+	vmovdqa	32(%rsp),%ymm13+	vpaddd	%ymm3,%ymm10,%ymm10+	vpxor	%ymm5,%ymm10,%ymm5+	vpshufb	%ymm15,%ymm5,%ymm5+	vpaddd	%ymm0,%ymm11,%ymm11+	vpxor	%ymm6,%ymm11,%ymm6+	vpshufb	%ymm15,%ymm6,%ymm6+	vpaddd	%ymm5,%ymm12,%ymm12+	vpxor	%ymm3,%ymm12,%ymm3+	vpslld	$12,%ymm3,%ymm14+	vpsrld	$20,%ymm3,%ymm3+	vpor	%ymm3,%ymm14,%ymm3+	vbroadcasti128	(%r11),%ymm14+	vpaddd	%ymm6,%ymm13,%ymm13+	vpxor	%ymm0,%ymm13,%ymm0+	vpslld	$12,%ymm0,%ymm15+	vpsrld	$20,%ymm0,%ymm0+	vpor	%ymm0,%ymm15,%ymm0+	vpaddd	%ymm3,%ymm10,%ymm10+	vpxor	%ymm5,%ymm10,%ymm5+	vpshufb	%ymm14,%ymm5,%ymm5+	vpaddd	%ymm0,%ymm11,%ymm11+	vpxor	%ymm6,%ymm11,%ymm6+	vpshufb	%ymm14,%ymm6,%ymm6+	vpaddd	%ymm5,%ymm12,%ymm12+	vpxor	%ymm3,%ymm12,%ymm3+	vpslld	$7,%ymm3,%ymm15+	vpsrld	$25,%ymm3,%ymm3+	vpor	%ymm3,%ymm15,%ymm3+	vbroadcasti128	(%r9),%ymm15+	vpaddd	%ymm6,%ymm13,%ymm13+	vpxor	%ymm0,%ymm13,%ymm0+	vpslld	$7,%ymm0,%ymm14+	vpsrld	$25,%ymm0,%ymm0+	vpor	%ymm0,%ymm14,%ymm0+	decl	%eax+	jnz	.Loop8x++	leaq	512(%rsp),%rax+	vpaddd	128-256(%rcx),%ymm8,%ymm8+	vpaddd	160-256(%rcx),%ymm9,%ymm9+	vpaddd	192-256(%rcx),%ymm10,%ymm10+	vpaddd	224-256(%rcx),%ymm11,%ymm11++	vpunpckldq	%ymm9,%ymm8,%ymm14+	vpunpckldq	%ymm11,%ymm10,%ymm15+	vpunpckhdq	%ymm9,%ymm8,%ymm8+	vpunpckhdq	%ymm11,%ymm10,%ymm10+	vpunpcklqdq	%ymm15,%ymm14,%ymm9+	vpunpckhqdq	%ymm15,%ymm14,%ymm14+	vpunpcklqdq	%ymm10,%ymm8,%ymm11+	vpunpckhqdq	%ymm10,%ymm8,%ymm8+	vpaddd	256-256(%rcx),%ymm0,%ymm0+	vpaddd	288-256(%rcx),%ymm1,%ymm1+	vpaddd	320-256(%rcx),%ymm2,%ymm2+	vpaddd	352-256(%rcx),%ymm3,%ymm3++	vpunpckldq	%ymm1,%ymm0,%ymm10+	vpunpckldq	%ymm3,%ymm2,%ymm15+	vpunpckhdq	%ymm1,%ymm0,%ymm0+	vpunpckhdq	%ymm3,%ymm2,%ymm2+	vpunpcklqdq	%ymm15,%ymm10,%ymm1+	vpunpckhqdq	%ymm15,%ymm10,%ymm10+	vpunpcklqdq	%ymm2,%ymm0,%ymm3+	vpunpckhqdq	%ymm2,%ymm0,%ymm0+	vperm2i128	$0x20,%ymm1,%ymm9,%ymm15+	vperm2i128	$0x31,%ymm1,%ymm9,%ymm1+	vperm2i128	$0x20,%ymm10,%ymm14,%ymm9+	vperm2i128	$0x31,%ymm10,%ymm14,%ymm10+	vperm2i128	$0x20,%ymm3,%ymm11,%ymm14+	vperm2i128	$0x31,%ymm3,%ymm11,%ymm3+	vperm2i128	$0x20,%ymm0,%ymm8,%ymm11+	vperm2i128	$0x31,%ymm0,%ymm8,%ymm0+	vmovdqa	%ymm15,0(%rsp)+	vmovdqa	%ymm9,32(%rsp)+	vmovdqa	64(%rsp),%ymm15+	vmovdqa	96(%rsp),%ymm9++	vpaddd	384-512(%rax),%ymm12,%ymm12+	vpaddd	416-512(%rax),%ymm13,%ymm13+	vpaddd	448-512(%rax),%ymm15,%ymm15+	vpaddd	480-512(%rax),%ymm9,%ymm9++	vpunpckldq	%ymm13,%ymm12,%ymm2+	vpunpckldq	%ymm9,%ymm15,%ymm8+	vpunpckhdq	%ymm13,%ymm12,%ymm12+	vpunpckhdq	%ymm9,%ymm15,%ymm15+	vpunpcklqdq	%ymm8,%ymm2,%ymm13+	vpunpckhqdq	%ymm8,%ymm2,%ymm2+	vpunpcklqdq	%ymm15,%ymm12,%ymm9+	vpunpckhqdq	%ymm15,%ymm12,%ymm12+	vpaddd	512-512(%rax),%ymm4,%ymm4+	vpaddd	544-512(%rax),%ymm5,%ymm5+	vpaddd	576-512(%rax),%ymm6,%ymm6+	vpaddd	608-512(%rax),%ymm7,%ymm7++	vpunpckldq	%ymm5,%ymm4,%ymm15+	vpunpckldq	%ymm7,%ymm6,%ymm8+	vpunpckhdq	%ymm5,%ymm4,%ymm4+	vpunpckhdq	%ymm7,%ymm6,%ymm6+	vpunpcklqdq	%ymm8,%ymm15,%ymm5+	vpunpckhqdq	%ymm8,%ymm15,%ymm15+	vpunpcklqdq	%ymm6,%ymm4,%ymm7+	vpunpckhqdq	%ymm6,%ymm4,%ymm4+	vperm2i128	$0x20,%ymm5,%ymm13,%ymm8+	vperm2i128	$0x31,%ymm5,%ymm13,%ymm5+	vperm2i128	$0x20,%ymm15,%ymm2,%ymm13+	vperm2i128	$0x31,%ymm15,%ymm2,%ymm15+	vperm2i128	$0x20,%ymm7,%ymm9,%ymm2+	vperm2i128	$0x31,%ymm7,%ymm9,%ymm7+	vperm2i128	$0x20,%ymm4,%ymm12,%ymm9+	vperm2i128	$0x31,%ymm4,%ymm12,%ymm4+	vmovdqa	0(%rsp),%ymm6+	vmovdqa	32(%rsp),%ymm12++	cmpq	$512,%rdx+	jb	.Ltail8x++	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	leaq	128(%rsi),%rsi+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	leaq	128(%rdi),%rdi++	vpxor	0(%rsi),%ymm12,%ymm12+	vpxor	32(%rsi),%ymm13,%ymm13+	vpxor	64(%rsi),%ymm10,%ymm10+	vpxor	96(%rsi),%ymm15,%ymm15+	leaq	128(%rsi),%rsi+	vmovdqu	%ymm12,0(%rdi)+	vmovdqu	%ymm13,32(%rdi)+	vmovdqu	%ymm10,64(%rdi)+	vmovdqu	%ymm15,96(%rdi)+	leaq	128(%rdi),%rdi++	vpxor	0(%rsi),%ymm14,%ymm14+	vpxor	32(%rsi),%ymm2,%ymm2+	vpxor	64(%rsi),%ymm3,%ymm3+	vpxor	96(%rsi),%ymm7,%ymm7+	leaq	128(%rsi),%rsi+	vmovdqu	%ymm14,0(%rdi)+	vmovdqu	%ymm2,32(%rdi)+	vmovdqu	%ymm3,64(%rdi)+	vmovdqu	%ymm7,96(%rdi)+	leaq	128(%rdi),%rdi++	vpxor	0(%rsi),%ymm11,%ymm11+	vpxor	32(%rsi),%ymm9,%ymm9+	vpxor	64(%rsi),%ymm0,%ymm0+	vpxor	96(%rsi),%ymm4,%ymm4+	leaq	128(%rsi),%rsi+	vmovdqu	%ymm11,0(%rdi)+	vmovdqu	%ymm9,32(%rdi)+	vmovdqu	%ymm0,64(%rdi)+	vmovdqu	%ymm4,96(%rdi)+	leaq	128(%rdi),%rdi++	subq	$512,%rdx+	jnz	.Loop_outer8x++	jmp	.Ldone8x++.Ltail8x:+	cmpq	$448,%rdx+	jae	.L448_or_more8x+	cmpq	$384,%rdx+	jae	.L384_or_more8x+	cmpq	$320,%rdx+	jae	.L320_or_more8x+	cmpq	$256,%rdx+	jae	.L256_or_more8x+	cmpq	$192,%rdx+	jae	.L192_or_more8x+	cmpq	$128,%rdx+	jae	.L128_or_more8x+	cmpq	$64,%rdx+	jae	.L64_or_more8x++	xorq	%r9,%r9+	vmovdqa	%ymm6,0(%rsp)+	vmovdqa	%ymm8,32(%rsp)+	jmp	.Loop_tail8x++.p2align	5+.L64_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	je	.Ldone8x++	leaq	64(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm1,0(%rsp)+	leaq	64(%rdi),%rdi+	subq	$64,%rdx+	vmovdqa	%ymm5,32(%rsp)+	jmp	.Loop_tail8x++.p2align	5+.L128_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	je	.Ldone8x++	leaq	128(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm12,0(%rsp)+	leaq	128(%rdi),%rdi+	subq	$128,%rdx+	vmovdqa	%ymm13,32(%rsp)+	jmp	.Loop_tail8x++.p2align	5+.L192_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	je	.Ldone8x++	leaq	192(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm10,0(%rsp)+	leaq	192(%rdi),%rdi+	subq	$192,%rdx+	vmovdqa	%ymm15,32(%rsp)+	jmp	.Loop_tail8x++.p2align	5+.L256_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vpxor	192(%rsi),%ymm10,%ymm10+	vpxor	224(%rsi),%ymm15,%ymm15+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	vmovdqu	%ymm10,192(%rdi)+	vmovdqu	%ymm15,224(%rdi)+	je	.Ldone8x++	leaq	256(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm14,0(%rsp)+	leaq	256(%rdi),%rdi+	subq	$256,%rdx+	vmovdqa	%ymm2,32(%rsp)+	jmp	.Loop_tail8x++.p2align	5+.L320_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vpxor	192(%rsi),%ymm10,%ymm10+	vpxor	224(%rsi),%ymm15,%ymm15+	vpxor	256(%rsi),%ymm14,%ymm14+	vpxor	288(%rsi),%ymm2,%ymm2+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	vmovdqu	%ymm10,192(%rdi)+	vmovdqu	%ymm15,224(%rdi)+	vmovdqu	%ymm14,256(%rdi)+	vmovdqu	%ymm2,288(%rdi)+	je	.Ldone8x++	leaq	320(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm3,0(%rsp)+	leaq	320(%rdi),%rdi+	subq	$320,%rdx+	vmovdqa	%ymm7,32(%rsp)+	jmp	.Loop_tail8x++.p2align	5+.L384_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vpxor	192(%rsi),%ymm10,%ymm10+	vpxor	224(%rsi),%ymm15,%ymm15+	vpxor	256(%rsi),%ymm14,%ymm14+	vpxor	288(%rsi),%ymm2,%ymm2+	vpxor	320(%rsi),%ymm3,%ymm3+	vpxor	352(%rsi),%ymm7,%ymm7+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	vmovdqu	%ymm10,192(%rdi)+	vmovdqu	%ymm15,224(%rdi)+	vmovdqu	%ymm14,256(%rdi)+	vmovdqu	%ymm2,288(%rdi)+	vmovdqu	%ymm3,320(%rdi)+	vmovdqu	%ymm7,352(%rdi)+	je	.Ldone8x++	leaq	384(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm11,0(%rsp)+	leaq	384(%rdi),%rdi+	subq	$384,%rdx+	vmovdqa	%ymm9,32(%rsp)+	jmp	.Loop_tail8x++.p2align	5+.L448_or_more8x:+	vpxor	0(%rsi),%ymm6,%ymm6+	vpxor	32(%rsi),%ymm8,%ymm8+	vpxor	64(%rsi),%ymm1,%ymm1+	vpxor	96(%rsi),%ymm5,%ymm5+	vpxor	128(%rsi),%ymm12,%ymm12+	vpxor	160(%rsi),%ymm13,%ymm13+	vpxor	192(%rsi),%ymm10,%ymm10+	vpxor	224(%rsi),%ymm15,%ymm15+	vpxor	256(%rsi),%ymm14,%ymm14+	vpxor	288(%rsi),%ymm2,%ymm2+	vpxor	320(%rsi),%ymm3,%ymm3+	vpxor	352(%rsi),%ymm7,%ymm7+	vpxor	384(%rsi),%ymm11,%ymm11+	vpxor	416(%rsi),%ymm9,%ymm9+	vmovdqu	%ymm6,0(%rdi)+	vmovdqu	%ymm8,32(%rdi)+	vmovdqu	%ymm1,64(%rdi)+	vmovdqu	%ymm5,96(%rdi)+	vmovdqu	%ymm12,128(%rdi)+	vmovdqu	%ymm13,160(%rdi)+	vmovdqu	%ymm10,192(%rdi)+	vmovdqu	%ymm15,224(%rdi)+	vmovdqu	%ymm14,256(%rdi)+	vmovdqu	%ymm2,288(%rdi)+	vmovdqu	%ymm3,320(%rdi)+	vmovdqu	%ymm7,352(%rdi)+	vmovdqu	%ymm11,384(%rdi)+	vmovdqu	%ymm9,416(%rdi)+	je	.Ldone8x++	leaq	448(%rsi),%rsi+	xorq	%r9,%r9+	vmovdqa	%ymm0,0(%rsp)+	leaq	448(%rdi),%rdi+	subq	$448,%rdx+	vmovdqa	%ymm4,32(%rsp)++.Loop_tail8x:+	movzbl	(%rsi,%r9,1),%eax+	movzbl	(%rsp,%r9,1),%ecx+	leaq	1(%r9),%r9+	xorl	%ecx,%eax+	movb	%al,-1(%rdi,%r9,1)+	decq	%rdx+	jnz	.Loop_tail8x++.Ldone8x:+	vzeroall+	movaps	-168(%r10),%xmm6+	movaps	-152(%r10),%xmm7+	movaps	-136(%r10),%xmm8+	movaps	-120(%r10),%xmm9+	movaps	-104(%r10),%xmm10+	movaps	-88(%r10),%xmm11+	movaps	-72(%r10),%xmm12+	movaps	-56(%r10),%xmm13+	movaps	-40(%r10),%xmm14+	movaps	-24(%r10),%xmm15+	leaq	(%r10),%rsp++.Lavx2_epilogue:+	movq	8(%rsp),%rdi+	movq	16(%rsp),%rsi+	.byte	0xf3,0xc3++.LSEH_end_crypton_chacha20_asm_avx2:++.def	se_handler;	.scl 3;	.type 32;	.endef+.p2align	4+se_handler:+	.byte	0xf3,0x0f,0x1e,0xfa++	pushq	%rsi+	pushq	%rdi+	pushq	%rbx+	pushq	%rbp+	pushq	%r12+	pushq	%r13+	pushq	%r14+	pushq	%r15+	pushfq+	subq	$64,%rsp++	movq	120(%r8),%rax+	movq	248(%r8),%rbx++	movq	8(%r9),%rsi+	movq	56(%r9),%r11++	leaq	.Lctr32_body(%rip),%r10+	cmpq	%r10,%rbx+	jb	.Lcommon_seh_tail++	movq	152(%r8),%rax++	leaq	.Lno_data(%rip),%r10+	cmpq	%r10,%rbx+	jae	.Lcommon_seh_tail++	leaq	64+24+48(%rax),%rax++	movq	-8(%rax),%rbx+	movq	-16(%rax),%rbp+	movq	-24(%rax),%r12+	movq	-32(%rax),%r13+	movq	-40(%rax),%r14+	movq	-48(%rax),%r15+	movq	%rbx,144(%r8)+	movq	%rbp,160(%r8)+	movq	%r12,216(%r8)+	movq	%r13,224(%r8)+	movq	%r14,232(%r8)+	movq	%r15,240(%r8)++.Lcommon_seh_tail:+	movq	8(%rax),%rdi+	movq	16(%rax),%rsi+	movq	%rax,152(%r8)+	movq	%rsi,168(%r8)+	movq	%rdi,176(%r8)++	movq	40(%r9),%rdi+	movq	%r8,%rsi+	movl	$154,%ecx+.long	0xa548f3fc++	movq	%r9,%rsi+	xorq	%rcx,%rcx+	movq	8(%rsi),%rdx+	movq	0(%rsi),%r8+	movq	16(%rsi),%r9+	movq	40(%rsi),%r10+	leaq	56(%rsi),%r11+	leaq	24(%rsi),%r12+	movq	%r10,32(%rsp)+	movq	%r11,40(%rsp)+	movq	%r12,48(%rsp)+	movq	%rcx,56(%rsp)+	call	*__imp_RtlVirtualUnwind(%rip)++	movl	$1,%eax+	addq	$64,%rsp+	popfq+	popq	%r15+	popq	%r14+	popq	%r13+	popq	%r12+	popq	%rbp+	popq	%rbx+	popq	%rdi+	popq	%rsi+	.byte	0xf3,0xc3+++.def	simd_handler;	.scl 3;	.type 32;	.endef+.p2align	4+simd_handler:+	.byte	0xf3,0x0f,0x1e,0xfa++	pushq	%rsi+	pushq	%rdi+	pushq	%rbx+	pushq	%rbp+	pushq	%r12+	pushq	%r13+	pushq	%r14+	pushq	%r15+	pushfq+	subq	$64,%rsp++	movq	120(%r8),%rax+	movq	248(%r8),%rbx++	movq	8(%r9),%rsi+	movq	56(%r9),%r11++	movl	0(%r11),%r10d+	leaq	(%rsi,%r10,1),%r10+	cmpq	%r10,%rbx+	jb	.Lcommon_seh_tail++	movq	200(%r8),%rax++	movl	4(%r11),%r10d+	movl	8(%r11),%ecx+	leaq	(%rsi,%r10,1),%r10+	cmpq	%r10,%rbx+	jae	.Lcommon_seh_tail++	negq	%rcx+	leaq	-8(%rax,%rcx,1),%rsi+	leaq	512(%r8),%rdi+	negl	%ecx+	shrl	$3,%ecx+.long	0xa548f3fc++	jmp	.Lcommon_seh_tail+++.section	.pdata+.p2align	2+.rva	.LSEH_begin_crypton_chacha20_asm_ctr32+.rva	.LSEH_end_crypton_chacha20_asm_ctr32+.rva	.LSEH_info_crypton_chacha20_asm_ctr32++.rva	.LSEH_begin_crypton_chacha20_asm_ssse3+.rva	.LSEH_end_crypton_chacha20_asm_ssse3+.rva	.LSEH_info_crypton_chacha20_asm_ssse3++.rva	.LSEH_begin_crypton_chacha20_asm_128+.rva	.LSEH_end_crypton_chacha20_asm_128+.rva	.LSEH_info_crypton_chacha20_asm_128++.rva	.LSEH_begin_crypton_chacha20_asm_4x+.rva	.LSEH_end_crypton_chacha20_asm_4x+.rva	.LSEH_info_crypton_chacha20_asm_4x+.rva	.LSEH_begin_crypton_chacha20_asm_4xop+.rva	.LSEH_end_crypton_chacha20_asm_4xop+.rva	.LSEH_info_crypton_chacha20_asm_4xop+.rva	.LSEH_begin_crypton_chacha20_asm_avx2+.rva	.LSEH_end_crypton_chacha20_asm_avx2+.rva	.LSEH_info_crypton_chacha20_asm_avx2+.section	.xdata+.p2align	3+.LSEH_info_crypton_chacha20_asm_ctr32:+.byte	9,0,0,0+.rva	se_handler++.LSEH_info_crypton_chacha20_asm_ssse3:+.byte	9,0,0,0+.rva	simd_handler+.rva	.Lssse3_body,.Lssse3_epilogue+.long	0x20,0++.LSEH_info_crypton_chacha20_asm_128:+.byte	9,0,0,0+.rva	simd_handler+.rva	.L128_body,.L128_epilogue+.long	0x60,0++.LSEH_info_crypton_chacha20_asm_4x:+.byte	9,0,0,0+.rva	simd_handler+.rva	.L4x_body,.L4x_epilogue+.long	0xa0,0+.LSEH_info_crypton_chacha20_asm_4xop:+.byte	9,0,0,0+.rva	simd_handler+.rva	.L4xop_body,.L4xop_epilogue+.long	0xa0,0+.LSEH_info_crypton_chacha20_asm_avx2:+.byte	9,0,0,0+.rva	simd_handler+.rva	.Lavx2_body,.Lavx2_epilogue+.long	0xa0,0
+ cbits/asm/chacha-x86_64.pl view
@@ -0,0 +1,4044 @@+#!/usr/bin/env perl+#+# ====================================================================+# Written by Andy Polyakov, @dot-asm, initially  for the OpenSSL+# project.+# ====================================================================+#+# November 2014+#+# ChaCha20 for x86_64.+#+# December 2016+#+# Add AVX512F code path.+#+# December 2017+#+# Add AVX512VL code path.+#+# Performance in cycles per byte out of large buffer.+#+#		IALU/gcc 4.8(i)	1x/2xSSSE3(ii)	4xSSSE3	    NxAVX(v)+#+# P4		9.48/+99%	-		-+# Core2		7.83/+55%	7.90/5.76	4.35+# Westmere	7.19/+50%	5.60/4.50	3.00+# Sandy Bridge	8.31/+42%	5.45/4.00	2.72+# Ivy Bridge	6.71/+46%	5.40/?		2.41+# Haswell	5.92/+43%	5.20/3.45	2.42        1.23+# Skylake[-X]	5.87/+39%	4.70/3.22	2.31        1.19[0.80(vi)]+# Cannon Lake	5.87/+39%	4.60/3.20	2.26        0.80(vi)+# Rocket Lake	5.86/+39%	?		2.30	    0.58+# Silvermont	12.0/+33%	7.75/6.90	7.03(iii)+# Knights L	11.7/-		?		9.60(iii)   0.80+# Goldmont	10.6/+17%	5.10/3.52	3.28+# Sledgehammer	7.28/+52%	-		-+# Bulldozer	9.66/+28%	9.85/5.35(iv)	3.06(iv)+# Ryzen		5.96/+50%	5.19/3.00	2.40        2.09+# VIA Nano	10.5/+46%	6.72/6.88	6.05+#+# (i)	compared to older gcc 3.x one can observe >2x improvement on+#	most platforms;+# (ii)	2xSSSE3 is code path optimized specifically for 128 bytes used+#	by chacha20_poly1305_tls_cipher, results are EVP-free;+# (iii)	this is not optimal result for Atom because of MSROM+#	limitations, SSE2 can do better, but gain is considered too+#	low to justify the [maintenance] effort;+# (iv)	Bulldozer actually executes 4xXOP code path that delivers 2.20+#	and 4.85 for 128-byte inputs;+# (v)	8xAVX2, 8xAVX512VL or 16xAVX512F, whichever best applicable;+# (vi)	even though Skylake-X can execute AVX512F code and deliver 0.57+#	cpb in single thread, the corresponding capability is suppressed;++$flavour = shift;+$output  = shift;+if ($flavour =~ /\./) { $output = $flavour; undef $flavour; }++$win64=0; $win64=1 if ($flavour =~ /[nm]asm|mingw64/ || $output =~ /\.asm$/);++$0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1;+( $xlate="${dir}x86_64-xlate.pl" and -f $xlate ) or+( $xlate="${dir}../../perlasm/x86_64-xlate.pl" and -f $xlate) or+die "can't locate x86_64-xlate.pl";++$avx=undef;++if (!defined($avx) && $win64 && ($flavour =~ /nasm/ || $ENV{ASM} =~ /nasm/) &&+	   ($ENV{ASM} //= "nasm") &&+	   `"$ENV{ASM}" -v 2>&1` =~ /NASM version ([0-9]+\.[0-9]+)(?:\.([0-9]+))?/) {+	$avx = ($1>=2.09) + ($1>=2.10) + ($1>=2.12);+	$avx += 1 if ($1==2.11 && $2>=8);+}++if (!defined($avx) && $win64 && ($flavour =~ /masm/ || $ENV{ASM} =~ /ml64/) &&+	   ($ENV{ASM} //= "ml64") &&+	   `"$ENV{ASM}" 2>&1` =~ /Version ([0-9]+)\./) {+	$avx = ($1>=10) + ($1>=11) + ($1>=14);+}++$ENV{CC} //= "cc";+if (!defined($avx) && `$ENV{CC} -Wa,-v -c -o /dev/zero -x assembler /dev/null 2>&1`+		=~ /GNU assembler version ([0-9]+)\.([0-9]+)/) {+	my $ver = $1 + $2/100.0;	# 3.1->3.01, 3.10->3.10+	$avx = ($ver>=2.19) + ($ver>=2.22) + ($ver>=2.25);+}++if (!defined($avx) && `$ENV{CC} -v 2>&1`+		=~ /((?:^clang|LLVM) version|.*based on LLVM) ([0-9]+)\.([0-9]+)/) {+	my $ver = $2 + $3/100.0;	# 3.1->3.01, 3.10->3.10+	$avx = ($ver>=3.0) + ($ver>3.0);+	$avx += ($ver>=7.0) if ($1 =~ /^clang/);+}++open OUT,"| \"$^X\" \"$xlate\" $flavour \"$output\"";+*STDOUT=*OUT;++# input parameter block+($out,$inp,$len,$key,$counter)=("%rdi","%rsi","%rdx","%rcx","%r8");++$code.=<<___;+.text++.extern OPENSSL_ia32cap_P++.align	64+.Lzero:+.long	0,0,0,0+.Lone:+.long	1,0,0,0+.Linc:+.long	0,1,2,3+.Lfour:+.long	4,4,4,4+.Lincy:+.long	0,2,4,6,1,3,5,7+.Leight:+.long	8,8,8,8,8,8,8,8+.Lrot16:+.byte	0x2,0x3,0x0,0x1, 0x6,0x7,0x4,0x5, 0xa,0xb,0x8,0x9, 0xe,0xf,0xc,0xd+.Lrot24:+.byte	0x3,0x0,0x1,0x2, 0x7,0x4,0x5,0x6, 0xb,0x8,0x9,0xa, 0xf,0xc,0xd,0xe+.Ltwoy:+.long	2,0,0,0, 2,0,0,0+.align	64+.Lzeroz:+.long	0,0,0,0, 1,0,0,0, 2,0,0,0, 3,0,0,0+.Lfourz:+.long	4,0,0,0, 4,0,0,0, 4,0,0,0, 4,0,0,0+.Lincz:+.long	0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15+.Lsixteen:+.long	16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16+.Lsigma:+.asciz	"expand 32-byte k"+.asciz	"ChaCha20 for x86_64, CRYPTOGAMS by \@dot-asm"+___++sub AUTOLOAD()          # thunk [simplified] 32-bit style perlasm+{ my $opcode = $AUTOLOAD; $opcode =~ s/.*:://;+  my $arg = pop;+    $arg = "\$$arg" if ($arg*1 eq $arg);+    $code .= "\t$opcode\t".join(',',$arg,reverse @_)."\n";+}++@x=("%eax","%ebx","%ecx","%edx",map("%r${_}d",(8..11)),+    "%nox","%nox","%nox","%nox",map("%r${_}d",(12..15)));+@t=("%esi","%edi");++sub ROUND {			# critical path is 24 cycles per round+my ($a0,$b0,$c0,$d0)=@_;+my ($a1,$b1,$c1,$d1)=map(($_&~3)+(($_+1)&3),($a0,$b0,$c0,$d0));+my ($a2,$b2,$c2,$d2)=map(($_&~3)+(($_+1)&3),($a1,$b1,$c1,$d1));+my ($a3,$b3,$c3,$d3)=map(($_&~3)+(($_+1)&3),($a2,$b2,$c2,$d2));+my ($xc,$xc_)=map("\"$_\"",@t);+my @x=map("\"$_\"",@x);++	# Consider order in which variables are addressed by their+	# index:+	#+	#	a   b   c   d+	#+	#	0   4   8  12 < even round+	#	1   5   9  13+	#	2   6  10  14+	#	3   7  11  15+	#	0   5  10  15 < odd round+	#	1   6  11  12+	#	2   7   8  13+	#	3   4   9  14+	#+	# 'a', 'b' and 'd's are permanently allocated in registers,+	# @x[0..7,12..15], while 'c's are maintained in memory. If+	# you observe 'c' column, you'll notice that pair of 'c's is+	# invariant between rounds. This means that we have to reload+	# them once per round, in the middle. This is why you'll see+	# bunch of 'c' stores and loads in the middle, but none in+	# the beginning or end.++	# Normally instructions would be interleaved to favour in-order+	# execution. Generally out-of-order cores manage it gracefully,+	# but not this time for some reason. As in-order execution+	# cores are dying breed, old Atom is the only one around,+	# instructions are left uninterleaved. Besides, Atom is better+	# off executing 1xSSSE3 code anyway...++	(+	"&add	(@x[$a0],@x[$b0])",	# Q1+	"&xor	(@x[$d0],@x[$a0])",+	"&rol	(@x[$d0],16)",+	 "&add	(@x[$a1],@x[$b1])",	# Q2+	 "&xor	(@x[$d1],@x[$a1])",+	 "&rol	(@x[$d1],16)",++	"&add	($xc,@x[$d0])",+	"&xor	(@x[$b0],$xc)",+	"&rol	(@x[$b0],12)",+	 "&add	($xc_,@x[$d1])",+	 "&xor	(@x[$b1],$xc_)",+	 "&rol	(@x[$b1],12)",++	"&add	(@x[$a0],@x[$b0])",+	"&xor	(@x[$d0],@x[$a0])",+	"&rol	(@x[$d0],8)",+	 "&add	(@x[$a1],@x[$b1])",+	 "&xor	(@x[$d1],@x[$a1])",+	 "&rol	(@x[$d1],8)",++	"&add	($xc,@x[$d0])",+	"&xor	(@x[$b0],$xc)",+	"&rol	(@x[$b0],7)",+	 "&add	($xc_,@x[$d1])",+	 "&xor	(@x[$b1],$xc_)",+	 "&rol	(@x[$b1],7)",++	"&mov	(\"4*$c0(%rsp)\",$xc)",	# reload pair of 'c's+	 "&mov	(\"4*$c1(%rsp)\",$xc_)",+	"&mov	($xc,\"4*$c2(%rsp)\")",+	 "&mov	($xc_,\"4*$c3(%rsp)\")",++	"&add	(@x[$a2],@x[$b2])",	# Q3+	"&xor	(@x[$d2],@x[$a2])",+	"&rol	(@x[$d2],16)",+	 "&add	(@x[$a3],@x[$b3])",	# Q4+	 "&xor	(@x[$d3],@x[$a3])",+	 "&rol	(@x[$d3],16)",++	"&add	($xc,@x[$d2])",+	"&xor	(@x[$b2],$xc)",+	"&rol	(@x[$b2],12)",+	 "&add	($xc_,@x[$d3])",+	 "&xor	(@x[$b3],$xc_)",+	 "&rol	(@x[$b3],12)",++	"&add	(@x[$a2],@x[$b2])",+	"&xor	(@x[$d2],@x[$a2])",+	"&rol	(@x[$d2],8)",+	 "&add	(@x[$a3],@x[$b3])",+	 "&xor	(@x[$d3],@x[$a3])",+	 "&rol	(@x[$d3],8)",++	"&add	($xc,@x[$d2])",+	"&xor	(@x[$b2],$xc)",+	"&rol	(@x[$b2],7)",+	 "&add	($xc_,@x[$d3])",+	 "&xor	(@x[$b3],$xc_)",+	 "&rol	(@x[$b3],7)"+	);+}++########################################################################+# Generic code path that handles all lengths on pre-SSSE3 processors.+$code.=<<___;+.globl	ChaCha20_ctr32+.type	ChaCha20_ctr32,\@function,5+.align	64+ChaCha20_ctr32:+.cfi_startproc+	cmp	\$0,$len+	je	.Lno_data+___+					if ($flavour !~ /kernel/) {+$code.=<<___;+	mov	OPENSSL_ia32cap_P+4(%rip),%r9+___+$code.=<<___	if ($avx>2);+	bt	\$48,%r9		# check for AVX512F+	jc	.LChaCha20_avx512+	test	%r9,%r9			# check for AVX512VL+	js	.LChaCha20_avx512vl+___+$code.=<<___;+	test	\$`1<<(41-32)`,%r9d+	jnz	.LChaCha20_ssse3+___+					}+$code.=<<___;+	push	%rbx+.cfi_push	%rbx+	push	%rbp+.cfi_push	%rbp+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15+	sub	\$64+24,%rsp+.cfi_adjust_cfa_offset	64+24+.Lctr32_body:++	mov	$len,%rbp		# reassign $len++	mov	0($key),%r12		# copy key and counter to stack+	mov	8($key),%r13+	mov	16($key),%r14+	mov	24($key),%r15+	mov	0($counter),%rax+	mov	8($counter),%rdx+	mov	%r12,4*4(%rsp)+	mov	%r13,4*6(%rsp)+	mov	%r14,4*0(%rsp)+	mov	%r15,4*2(%rsp)+	mov	%rax,4*12(%rsp)+	mov	%rdx,4*14(%rsp)+	jmp	.Loop_outer++.align	32+.Loop_outer:+	mov	\$0x61707865,@x[0]      # 'expa'+	mov	\$0x3320646e,@x[1]      # 'nd 3'+	mov	\$0x79622d32,@x[2]      # '2-by'+	mov	\$0x6b206574,@x[3]      # 'te k'+	mov	4*4(%rsp),@x[4]+	mov	4*5(%rsp),@x[5]+	mov	4*6(%rsp),@x[6]+	mov	4*7(%rsp),@x[7]+	mov	4*12(%rsp),@x[12]+	mov	4*13(%rsp),@x[13]+	mov	4*14(%rsp),@x[14]+	mov	%r15,4*10(%rsp)		# "@x[10]:@x[11]"+	mov	4*15(%rsp),@x[15]++	mov	%rbp,64+0(%rsp)		# save len+	mov	$inp,64+8(%rsp)		# save inp+	mov	0(%rsp),@t[0]		# "@x[8]"+	mov	$out,64+16(%rsp)	# save out+	mov	4(%rsp),@t[1]		# "@x[9]"+	mov	\$10,%ebp+	jmp	.Loop++.align	32+.Loop:+___+	foreach (&ROUND (0, 4, 8,12)) { eval; }+	foreach (&ROUND	(0, 5,10,15)) { eval; }+	&dec	("%ebp");+	&jnz	(".Loop");++$code.=<<___;+	add	4*0(%rsp),@t[0]		# modulo-scheduled+	add	4*1(%rsp),@t[1]+	mov	64(%rsp),%rbp		# load len+	mov	@t[0],4*8(%rsp)+	mov	64+8(%rsp),$inp		# load inp+	mov	@t[1],4*9(%rsp)+	mov	64+16(%rsp),$out	# load out++	add	\$0x61707865,@x[0]      # 'expa'+	add	\$0x3320646e,@x[1]      # 'nd 3'+	add	\$0x79622d32,@x[2]      # '2-by'+	add	\$0x6b206574,@x[3]      # 'te k'+	add	4*4(%rsp),@x[4]+	add	4*5(%rsp),@x[5]+	add	4*6(%rsp),@x[6]+	add	4*7(%rsp),@x[7]+	add	4*12(%rsp),@x[12]+	add	4*13(%rsp),@x[13]+	add	4*14(%rsp),@x[14]+	add	4*15(%rsp),@x[15]++	cmp	\$64,%rbp+	jb	.Ltail++	xor	4*0($inp),@x[0]		# xor with input+	xor	4*1($inp),@x[1]+	xor	4*2($inp),@x[2]+	xor	4*3($inp),@x[3]+	mov	@x[0],4*0($out)		# write output+	 mov	4*8(%rsp),@x[0]		# load @x[8]-@x[11]+	mov	@x[1],4*1($out)+	 mov	4*9(%rsp),@x[1]+	mov	@x[2],4*2($out)+	 mov	4*10(%rsp),@x[2]+	mov	@x[3],4*3($out)+	 mov	4*11(%rsp),@x[3]+	xor	4*4($inp),@x[4]+	 add	4*2(%rsp),@x[2]+	xor	4*5($inp),@x[5]+	 add	4*3(%rsp),@x[3]+	xor	4*6($inp),@x[6]+	xor	4*7($inp),@x[7]+	xor	4*8($inp),@x[0]+	xor	4*9($inp),@x[1]+	xor	4*10($inp),@x[2]+	xor	4*11($inp),@x[3]+	xor	4*12($inp),@x[12]+	xor	4*13($inp),@x[13]+	xor	4*14($inp),@x[14]+	xor	4*15($inp),@x[15]+	lea	4*16($inp),$inp		# inp+=64++	addl	\$1,4*12(%rsp)		# increment counter++	mov	@x[4],4*4($out)+	mov	@x[5],4*5($out)+	mov	@x[6],4*6($out)+	mov	@x[7],4*7($out)+	mov	@x[0],4*8($out)+	mov	@x[1],4*9($out)+	mov	@x[2],4*10($out)+	mov	@x[3],4*11($out)+	mov	@x[12],4*12($out)+	mov	@x[13],4*13($out)+	mov	@x[14],4*14($out)+	mov	@x[15],4*15($out)+	lea	4*16($out),$out		# out+=64+	mov	4*2(%rsp),%r15++	sub	\$64,%rbp+	jnz	.Loop_outer++	jmp	.Ldone++.align	16+.Ltail:+	mov	@x[0],4*0(%rsp)+	 mov	4*2(%rsp),@x[0]+	mov	@x[1],4*1(%rsp)+	 mov	4*3(%rsp),@x[1]+	mov	@x[2],4*2(%rsp)+	 add	4*10(%rsp),@x[0]+	mov	@x[3],4*3(%rsp)+	 add	4*11(%rsp),@x[1]+	mov	@x[4],4*4(%rsp)+	mov	@x[5],4*5(%rsp)+	mov	@x[6],4*6(%rsp)+	mov	@x[7],4*7(%rsp)+	mov	@x[0],4*10(%rsp)+	mov	@x[1],4*11(%rsp)+	xor	%rbx,%rbx+	mov	@x[12],4*12(%rsp)+	mov	@x[13],4*13(%rsp)+	mov	@x[14],4*14(%rsp)+	mov	@x[15],4*15(%rsp)++.Loop_tail:+	movzb	($inp,%rbx),%eax+	movzb	(%rsp,%rbx),%edx+	lea	1(%rbx),%rbx+	xor	%edx,%eax+	mov	%al,-1($out,%rbx)+	dec	%rbp+	jnz	.Loop_tail++.Ldone:+	lea	64+24+48(%rsp),%rsi+.cfi_def_cfa	%rsi,8+	mov	-48(%rsi),%r15+.cfi_restore	%r15+	mov	-40(%rsi),%r14+.cfi_restore	%r14+	mov	-32(%rsi),%r13+.cfi_restore	%r13+	mov	-24(%rsi),%r12+.cfi_restore	%r12+	mov	-16(%rsi),%rbp+.cfi_restore	%rbp+	mov	-8(%rsi),%rbx+.cfi_restore	%rbx+	lea	(%rsi),%rsp+.cfi_def_cfa_register	%rsp+.Lno_data:+	ret+.cfi_endproc+.size	ChaCha20_ctr32,.-ChaCha20_ctr32+___++########################################################################+# SSSE3 code path that handles shorter lengths+{+my ($a,$b,$c,$d,$t,$t1,$rot16,$rot24)=map("%xmm$_",(0..7));++sub SSSE3ROUND {	# critical path is 20 "SIMD ticks" per round+	&paddd	($a,$b);+	&pxor	($d,$a);+	&pshufb	($d,$rot16);++	&paddd	($c,$d);+	&pxor	($b,$c);+	&movdqa	($t,$b);+	&psrld	($b,20);+	&pslld	($t,12);+	&por	($b,$t);++	&paddd	($a,$b);+	&pxor	($d,$a);+	&pshufb	($d,$rot24);++	&paddd	($c,$d);+	&pxor	($b,$c);+	&movdqa	($t,$b);+	&psrld	($b,25);+	&pslld	($t,7);+	&por	($b,$t);+}++my $xframe = $win64 ? 32+8 : 8;++$code.=<<___	if ($flavour =~ /kernel/);+.globl	ChaCha20_ssse3+___+$code.=<<___;+.type	ChaCha20_ssse3,\@function,5+.align	32+ChaCha20_ssse3:+.cfi_startproc+.LChaCha20_ssse3:+	mov	%rsp,%r10		# frame pointer+.cfi_def_cfa_register	%r10+___+$code.=<<___	if ($avx && $flavour !~ /kernel/);+	test	\$`1<<(43-32)`,%r9d+	jnz	.LChaCha20_4xop		# XOP is fastest even if we use 1/4+___+$code.=<<___;+	cmp	\$128,$len		# we might throw away some data,+	je	.LChaCha20_128+	ja	.LChaCha20_4x		# but overall it won't be slower++.Ldo_sse3_after_all:+	sub	\$64+$xframe,%rsp+	and	\$-16,%rsp+___+$code.=<<___	if ($win64);+	movaps	%xmm6,-0x28(%r10)+	movaps	%xmm7,-0x18(%r10)+.Lssse3_body:+___+$code.=<<___;+	movdqa	.Lsigma(%rip),$a+	movdqu	($key),$b+	movdqu	16($key),$c+	movdqu	($counter),$d+	movdqa	.Lrot16(%rip),$rot16+	movdqa	.Lrot24(%rip),$rot24++	movdqa	$a,0x00(%rsp)+	movdqa	$b,0x10(%rsp)+	movdqa	$c,0x20(%rsp)+	movdqa	$d,0x30(%rsp)+	mov	\$10,$counter		# reuse $counter+	jmp	.Loop_ssse3++.align	32+.Loop_outer_ssse3:+	movdqa	.Lone(%rip),$d+	movdqa	0x00(%rsp),$a+	movdqa	0x10(%rsp),$b+	movdqa	0x20(%rsp),$c+	paddd	0x30(%rsp),$d+	mov	\$10,$counter+	movdqa	$d,0x30(%rsp)+	jmp	.Loop_ssse3++.align	32+.Loop_ssse3:+___+	&SSSE3ROUND();+	&pshufd	($c,$c,0b01001110);+	&pshufd	($b,$b,0b00111001);+	&pshufd	($d,$d,0b10010011);+	&nop	();++	&SSSE3ROUND();+	&pshufd	($c,$c,0b01001110);+	&pshufd	($b,$b,0b10010011);+	&pshufd	($d,$d,0b00111001);++	&dec	($counter);+	&jnz	(".Loop_ssse3");++$code.=<<___;+	paddd	0x00(%rsp),$a+	paddd	0x10(%rsp),$b+	paddd	0x20(%rsp),$c+	paddd	0x30(%rsp),$d++	cmp	\$64,$len+	jb	.Ltail_ssse3++	movdqu	0x00($inp),$t+	movdqu	0x10($inp),$t1+	pxor	$t,$a			# xor with input+	movdqu	0x20($inp),$t+	pxor	$t1,$b+	movdqu	0x30($inp),$t1+	lea	0x40($inp),$inp		# inp+=64+	pxor	$t,$c+	pxor	$t1,$d++	movdqu	$a,0x00($out)		# write output+	movdqu	$b,0x10($out)+	movdqu	$c,0x20($out)+	movdqu	$d,0x30($out)+	lea	0x40($out),$out		# out+=64++	sub	\$64,$len+	jnz	.Loop_outer_ssse3++	jmp	.Ldone_ssse3++.align	16+.Ltail_ssse3:+	movdqa	$a,0x00(%rsp)+	movdqa	$b,0x10(%rsp)+	movdqa	$c,0x20(%rsp)+	movdqa	$d,0x30(%rsp)+	xor	$counter,$counter++.Loop_tail_ssse3:+	movzb	($inp,$counter),%eax+	movzb	(%rsp,$counter),%ecx+	lea	1($counter),$counter+	xor	%ecx,%eax+	mov	%al,-1($out,$counter)+	dec	$len+	jnz	.Loop_tail_ssse3++.Ldone_ssse3:+___+$code.=<<___	if ($win64);+	movaps	-0x28(%r10),%xmm6+	movaps	-0x18(%r10),%xmm7+___+$code.=<<___;+	lea	(%r10),%rsp+.cfi_def_cfa_register	%rsp+.Lssse3_epilogue:+	ret+.cfi_endproc+.size	ChaCha20_ssse3,.-ChaCha20_ssse3+___+}++########################################################################+# SSSE3 code path that handles 128-byte inputs+{+my ($a,$b,$c,$d,$t,$t1,$rot16,$rot24)=map("%xmm$_",(8,9,2..7));+my ($a1,$b1,$c1,$d1)=map("%xmm$_",(10,11,0,1));++sub SSSE3ROUND_2x {+	&paddd	($a,$b);+	&pxor	($d,$a);+	 &paddd	($a1,$b1);+	 &pxor	($d1,$a1);+	&pshufb	($d,$rot16);+	 &pshufb($d1,$rot16);++	&paddd	($c,$d);+	 &paddd	($c1,$d1);+	&pxor	($b,$c);+	 &pxor	($b1,$c1);+	&movdqa	($t,$b);+	&psrld	($b,20);+	 &movdqa($t1,$b1);+	&pslld	($t,12);+	 &psrld	($b1,20);+	&por	($b,$t);+	 &pslld	($t1,12);+	 &por	($b1,$t1);++	&paddd	($a,$b);+	&pxor	($d,$a);+	 &paddd	($a1,$b1);+	 &pxor	($d1,$a1);+	&pshufb	($d,$rot24);+	 &pshufb($d1,$rot24);++	&paddd	($c,$d);+	 &paddd	($c1,$d1);+	&pxor	($b,$c);+	 &pxor	($b1,$c1);+	&movdqa	($t,$b);+	&psrld	($b,25);+	 &movdqa($t1,$b1);+	&pslld	($t,7);+	 &psrld	($b1,25);+	&por	($b,$t);+	 &pslld	($t1,7);+	 &por	($b1,$t1);+}++my $xframe = $win64 ? 0x68 : 8;++$code.=<<___;+.type	ChaCha20_128,\@function,5+.align	32+ChaCha20_128:+.cfi_startproc+.LChaCha20_128:+	mov	%rsp,%r10		# frame pointer+.cfi_def_cfa_register	%r10+	sub	\$64+$xframe,%rsp+	and	\$-16,%rsp+___+$code.=<<___	if ($win64);+	movaps	%xmm6,-0x68(%r10)+	movaps	%xmm7,-0x58(%r10)+	movaps	%xmm8,-0x48(%r10)+	movaps	%xmm9,-0x38(%r10)+	movaps	%xmm10,-0x28(%r10)+	movaps	%xmm11,-0x18(%r10)+.L128_body:+___+$code.=<<___;+	movdqa	.Lsigma(%rip),$a+	movdqu	($key),$b+	movdqu	16($key),$c+	movdqu	($counter),$d+	movdqa	.Lone(%rip),$d1+	movdqa	.Lrot16(%rip),$rot16+	movdqa	.Lrot24(%rip),$rot24++	movdqa	$a,$a1+	movdqa	$a,0x00(%rsp)+	movdqa	$b,$b1+	movdqa	$b,0x10(%rsp)+	movdqa	$c,$c1+	movdqa	$c,0x20(%rsp)+	paddd	$d,$d1+	movdqa	$d,0x30(%rsp)+	mov	\$10,$counter		# reuse $counter+	jmp	.Loop_128++.align	32+.Loop_128:+___+	&SSSE3ROUND_2x();+	&pshufd	($c,$c,0b01001110);+	&pshufd	($b,$b,0b00111001);+	&pshufd	($d,$d,0b10010011);+	&pshufd	($c1,$c1,0b01001110);+	&pshufd	($b1,$b1,0b00111001);+	&pshufd	($d1,$d1,0b10010011);++	&SSSE3ROUND_2x();+	&pshufd	($c,$c,0b01001110);+	&pshufd	($b,$b,0b10010011);+	&pshufd	($d,$d,0b00111001);+	&pshufd	($c1,$c1,0b01001110);+	&pshufd	($b1,$b1,0b10010011);+	&pshufd	($d1,$d1,0b00111001);++	&dec	($counter);+	&jnz	(".Loop_128");++$code.=<<___;+	paddd	0x00(%rsp),$a+	paddd	0x10(%rsp),$b+	paddd	0x20(%rsp),$c+	paddd	0x30(%rsp),$d+	paddd	.Lone(%rip),$d1+	paddd	0x00(%rsp),$a1+	paddd	0x10(%rsp),$b1+	paddd	0x20(%rsp),$c1+	paddd	0x30(%rsp),$d1++	movdqu	0x00($inp),$t+	movdqu	0x10($inp),$t1+	pxor	$t,$a			# xor with input+	movdqu	0x20($inp),$t+	pxor	$t1,$b+	movdqu	0x30($inp),$t1+	pxor	$t,$c+	movdqu	0x40($inp),$t+	pxor	$t1,$d+	movdqu	0x50($inp),$t1+	pxor	$t,$a1+	movdqu	0x60($inp),$t+	pxor	$t1,$b1+	movdqu	0x70($inp),$t1+	pxor	$t,$c1+	pxor	$t1,$d1++	movdqu	$a,0x00($out)		# write output+	movdqu	$b,0x10($out)+	movdqu	$c,0x20($out)+	movdqu	$d,0x30($out)+	movdqu	$a1,0x40($out)+	movdqu	$b1,0x50($out)+	movdqu	$c1,0x60($out)+	movdqu	$d1,0x70($out)+___+$code.=<<___	if ($win64);+	movaps	-0x68(%r10),%xmm6+	movaps	-0x58(%r10),%xmm7+	movaps	-0x48(%r10),%xmm8+	movaps	-0x38(%r10),%xmm9+	movaps	-0x28(%r10),%xmm10+	movaps	-0x18(%r10),%xmm11+___+$code.=<<___;+	lea	(%r10),%rsp+.cfi_def_cfa_register	%rsp+.L128_epilogue:+	ret+.cfi_endproc+.size	ChaCha20_128,.-ChaCha20_128+___+}++########################################################################+# SSSE3 code path that handles longer messages.+{+# assign variables to favor Atom front-end+my ($xd0,$xd1,$xd2,$xd3, $xt0,$xt1,$xt2,$xt3,+    $xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3)=map("%xmm$_",(0..15));+my  @xx=($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3,+	"%nox","%nox","%nox","%nox", $xd0,$xd1,$xd2,$xd3);++sub SSSE3_lane_ROUND {+my ($a0,$b0,$c0,$d0)=@_;+my ($a1,$b1,$c1,$d1)=map(($_&~3)+(($_+1)&3),($a0,$b0,$c0,$d0));+my ($a2,$b2,$c2,$d2)=map(($_&~3)+(($_+1)&3),($a1,$b1,$c1,$d1));+my ($a3,$b3,$c3,$d3)=map(($_&~3)+(($_+1)&3),($a2,$b2,$c2,$d2));+my ($xc,$xc_,$t0,$t1)=map("\"$_\"",$xt0,$xt1,$xt2,$xt3);+my @x=map("\"$_\"",@xx);++	# Consider order in which variables are addressed by their+	# index:+	#+	#	a   b   c   d+	#+	#	0   4   8  12 < even round+	#	1   5   9  13+	#	2   6  10  14+	#	3   7  11  15+	#	0   5  10  15 < odd round+	#	1   6  11  12+	#	2   7   8  13+	#	3   4   9  14+	#+	# 'a', 'b' and 'd's are permanently allocated in registers,+	# @x[0..7,12..15], while 'c's are maintained in memory. If+	# you observe 'c' column, you'll notice that pair of 'c's is+	# invariant between rounds. This means that we have to reload+	# them once per round, in the middle. This is why you'll see+	# bunch of 'c' stores and loads in the middle, but none in+	# the beginning or end.++	(+	"&paddd		(@x[$a0],@x[$b0])",	# Q1+	 "&paddd	(@x[$a1],@x[$b1])",	# Q2+	"&pxor		(@x[$d0],@x[$a0])",+	 "&pxor		(@x[$d1],@x[$a1])",+	"&pshufb	(@x[$d0],$t1)",+	 "&pshufb	(@x[$d1],$t1)",++	"&paddd		($xc,@x[$d0])",+	 "&paddd	($xc_,@x[$d1])",+	"&pxor		(@x[$b0],$xc)",+	 "&pxor		(@x[$b1],$xc_)",+	"&movdqa	($t0,@x[$b0])",+	"&pslld		(@x[$b0],12)",+	"&psrld		($t0,20)",+	 "&movdqa	($t1,@x[$b1])",+	 "&pslld	(@x[$b1],12)",+	"&por		(@x[$b0],$t0)",+	 "&psrld	($t1,20)",+	"&movdqa	($t0,'(%r11)')",	# .Lrot24(%rip)+	 "&por		(@x[$b1],$t1)",++	"&paddd		(@x[$a0],@x[$b0])",+	 "&paddd	(@x[$a1],@x[$b1])",+	"&pxor		(@x[$d0],@x[$a0])",+	 "&pxor		(@x[$d1],@x[$a1])",+	"&pshufb	(@x[$d0],$t0)",+	 "&pshufb	(@x[$d1],$t0)",++	"&paddd		($xc,@x[$d0])",+	 "&paddd	($xc_,@x[$d1])",+	"&pxor		(@x[$b0],$xc)",+	 "&pxor		(@x[$b1],$xc_)",+	"&movdqa	($t1,@x[$b0])",+	"&pslld		(@x[$b0],7)",+	"&psrld		($t1,25)",+	 "&movdqa	($t0,@x[$b1])",+	 "&pslld	(@x[$b1],7)",+	"&por		(@x[$b0],$t1)",+	 "&psrld	($t0,25)",+	"&movdqa	($t1,'(%r9)')",		# .Lrot16(%rip)+	 "&por		(@x[$b1],$t0)",++	"&movdqa	(\"`16*($c0-8)`(%rsp)\",$xc)",	# reload pair of 'c's+	 "&movdqa	(\"`16*($c1-8)`(%rsp)\",$xc_)",+	"&movdqa	($xc,\"`16*($c2-8)`(%rsp)\")",+	 "&movdqa	($xc_,\"`16*($c3-8)`(%rsp)\")",++	"&paddd		(@x[$a2],@x[$b2])",	# Q3+	 "&paddd	(@x[$a3],@x[$b3])",	# Q4+	"&pxor		(@x[$d2],@x[$a2])",+	 "&pxor		(@x[$d3],@x[$a3])",+	"&pshufb	(@x[$d2],$t1)",+	 "&pshufb	(@x[$d3],$t1)",++	"&paddd		($xc,@x[$d2])",+	 "&paddd	($xc_,@x[$d3])",+	"&pxor		(@x[$b2],$xc)",+	 "&pxor		(@x[$b3],$xc_)",+	"&movdqa	($t0,@x[$b2])",+	"&pslld		(@x[$b2],12)",+	"&psrld		($t0,20)",+	 "&movdqa	($t1,@x[$b3])",+	 "&pslld	(@x[$b3],12)",+	"&por		(@x[$b2],$t0)",+	 "&psrld	($t1,20)",+	"&movdqa	($t0,'(%r11)')",	# .Lrot24(%rip)+	 "&por		(@x[$b3],$t1)",++	"&paddd		(@x[$a2],@x[$b2])",+	 "&paddd	(@x[$a3],@x[$b3])",+	"&pxor		(@x[$d2],@x[$a2])",+	 "&pxor		(@x[$d3],@x[$a3])",+	"&pshufb	(@x[$d2],$t0)",+	 "&pshufb	(@x[$d3],$t0)",++	"&paddd		($xc,@x[$d2])",+	 "&paddd	($xc_,@x[$d3])",+	"&pxor		(@x[$b2],$xc)",+	 "&pxor		(@x[$b3],$xc_)",+	"&movdqa	($t1,@x[$b2])",+	"&pslld		(@x[$b2],7)",+	"&psrld		($t1,25)",+	 "&movdqa	($t0,@x[$b3])",+	 "&pslld	(@x[$b3],7)",+	"&por		(@x[$b2],$t1)",+	 "&psrld	($t0,25)",+	"&movdqa	($t1,'(%r9)')",		# .Lrot16(%rip)+	 "&por		(@x[$b3],$t0)"+	);+}++my $xframe = $win64 ? 0xa8 : 8;++$code.=<<___;+.type	ChaCha20_4x,\@function,5+.align	32+ChaCha20_4x:+.cfi_startproc+.LChaCha20_4x:+	mov		%rsp,%r10		# frame pointer+.cfi_def_cfa_register	%r10+	mov		%r9,%r11+___+$code.=<<___	if ($avx>1 && $flavour !~ /kernel/);+	shr		\$32,%r9		# OPENSSL_ia32cap_P+8+	test		\$`1<<5`,%r9		# test AVX2+	jnz		.LChaCha20_8x+___+$code.=<<___;+	cmp		\$192,$len+	ja		.Lproceed4x++	and		\$`1<<26|1<<22`,%r11	# isolate XSAVE+MOVBE+	cmp		\$`1<<22`,%r11		# check for MOVBE without XSAVE+	je		.Ldo_sse3_after_all	# to detect Atom++.Lproceed4x:+	sub		\$0x140+$xframe,%rsp+	and		\$-16,%rsp+___+	################ stack layout+	# +0x00		SIMD equivalent of @x[8-12]+	# ...+	# +0x40		constant copy of key[0-2] smashed by lanes+	# ...+	# +0x100	SIMD counters (with nonce smashed by lanes)+	# ...+	# +0x140+$code.=<<___	if ($win64);+	movaps		%xmm6,-0xa8(%r10)+	movaps		%xmm7,-0x98(%r10)+	movaps		%xmm8,-0x88(%r10)+	movaps		%xmm9,-0x78(%r10)+	movaps		%xmm10,-0x68(%r10)+	movaps		%xmm11,-0x58(%r10)+	movaps		%xmm12,-0x48(%r10)+	movaps		%xmm13,-0x38(%r10)+	movaps		%xmm14,-0x28(%r10)+	movaps		%xmm15,-0x18(%r10)+.L4x_body:+___+$code.=<<___;+	movdqa		.Lsigma(%rip),$xa3	# key[0]+	movdqu		($key),$xb3		# key[1]+	movdqu		16($key),$xt3		# key[2]+	movdqu		($counter),$xd3		# key[3]+	lea		0x100(%rsp),%rcx	# size optimization+	lea		.Lrot16(%rip),%r9+	lea		.Lrot24(%rip),%r11++	pshufd		\$0x00,$xa3,$xa0	# smash key by lanes...+	pshufd		\$0x55,$xa3,$xa1+	movdqa		$xa0,0x40(%rsp)		# ... and offload+	pshufd		\$0xaa,$xa3,$xa2+	movdqa		$xa1,0x50(%rsp)+	pshufd		\$0xff,$xa3,$xa3+	movdqa		$xa2,0x60(%rsp)+	movdqa		$xa3,0x70(%rsp)++	pshufd		\$0x00,$xb3,$xb0+	pshufd		\$0x55,$xb3,$xb1+	movdqa		$xb0,0x80-0x100(%rcx)+	pshufd		\$0xaa,$xb3,$xb2+	movdqa		$xb1,0x90-0x100(%rcx)+	pshufd		\$0xff,$xb3,$xb3+	movdqa		$xb2,0xa0-0x100(%rcx)+	movdqa		$xb3,0xb0-0x100(%rcx)++	pshufd		\$0x00,$xt3,$xt0	# "$xc0"+	pshufd		\$0x55,$xt3,$xt1	# "$xc1"+	movdqa		$xt0,0xc0-0x100(%rcx)+	pshufd		\$0xaa,$xt3,$xt2	# "$xc2"+	movdqa		$xt1,0xd0-0x100(%rcx)+	pshufd		\$0xff,$xt3,$xt3	# "$xc3"+	movdqa		$xt2,0xe0-0x100(%rcx)+	movdqa		$xt3,0xf0-0x100(%rcx)++	pshufd		\$0x00,$xd3,$xd0+	pshufd		\$0x55,$xd3,$xd1+	paddd		.Linc(%rip),$xd0	# don't save counters yet+	pshufd		\$0xaa,$xd3,$xd2+	movdqa		$xd1,0x110-0x100(%rcx)+	pshufd		\$0xff,$xd3,$xd3+	movdqa		$xd2,0x120-0x100(%rcx)+	movdqa		$xd3,0x130-0x100(%rcx)++	jmp		.Loop_enter4x++.align	32+.Loop_outer4x:+	movdqa		0x40(%rsp),$xa0		# re-load smashed key+	movdqa		0x50(%rsp),$xa1+	movdqa		0x60(%rsp),$xa2+	movdqa		0x70(%rsp),$xa3+	movdqa		0x80-0x100(%rcx),$xb0+	movdqa		0x90-0x100(%rcx),$xb1+	movdqa		0xa0-0x100(%rcx),$xb2+	movdqa		0xb0-0x100(%rcx),$xb3+	movdqa		0xc0-0x100(%rcx),$xt0	# "$xc0"+	movdqa		0xd0-0x100(%rcx),$xt1	# "$xc1"+	movdqa		0xe0-0x100(%rcx),$xt2	# "$xc2"+	movdqa		0xf0-0x100(%rcx),$xt3	# "$xc3"+	movdqa		0x100-0x100(%rcx),$xd0+	movdqa		0x110-0x100(%rcx),$xd1+	movdqa		0x120-0x100(%rcx),$xd2+	movdqa		0x130-0x100(%rcx),$xd3+	paddd		.Lfour(%rip),$xd0	# next SIMD counters++.Loop_enter4x:+	movdqa		$xt2,0x20(%rsp)		# SIMD equivalent of "@x[10]"+	movdqa		$xt3,0x30(%rsp)		# SIMD equivalent of "@x[11]"+	movdqa		(%r9),$xt3		# .Lrot16(%rip)+	mov		\$10,%eax+	movdqa		$xd0,0x100-0x100(%rcx)	# save SIMD counters+	jmp		.Loop4x++.align	32+.Loop4x:+___+	foreach (&SSSE3_lane_ROUND(0, 4, 8,12)) { eval; }+	foreach (&SSSE3_lane_ROUND(0, 5,10,15)) { eval; }+$code.=<<___;+	dec		%eax+	jnz		.Loop4x++	paddd		0x40(%rsp),$xa0		# accumulate key material+	paddd		0x50(%rsp),$xa1+	paddd		0x60(%rsp),$xa2+	paddd		0x70(%rsp),$xa3++	movdqa		$xa0,$xt2		# "de-interlace" data+	punpckldq	$xa1,$xa0+	movdqa		$xa2,$xt3+	punpckldq	$xa3,$xa2+	punpckhdq	$xa1,$xt2+	punpckhdq	$xa3,$xt3+	movdqa		$xa0,$xa1+	punpcklqdq	$xa2,$xa0		# "a0"+	movdqa		$xt2,$xa3+	punpcklqdq	$xt3,$xt2		# "a2"+	punpckhqdq	$xa2,$xa1		# "a1"+	punpckhqdq	$xt3,$xa3		# "a3"+___+	($xa2,$xt2)=($xt2,$xa2);+$code.=<<___;+	paddd		0x80-0x100(%rcx),$xb0+	paddd		0x90-0x100(%rcx),$xb1+	paddd		0xa0-0x100(%rcx),$xb2+	paddd		0xb0-0x100(%rcx),$xb3++	movdqa		$xa0,0x00(%rsp)		# offload $xaN+	movdqa		$xa1,0x10(%rsp)+	movdqa		0x20(%rsp),$xa0		# "xc2"+	movdqa		0x30(%rsp),$xa1		# "xc3"++	movdqa		$xb0,$xt2+	punpckldq	$xb1,$xb0+	movdqa		$xb2,$xt3+	punpckldq	$xb3,$xb2+	punpckhdq	$xb1,$xt2+	punpckhdq	$xb3,$xt3+	movdqa		$xb0,$xb1+	punpcklqdq	$xb2,$xb0		# "b0"+	movdqa		$xt2,$xb3+	punpcklqdq	$xt3,$xt2		# "b2"+	punpckhqdq	$xb2,$xb1		# "b1"+	punpckhqdq	$xt3,$xb3		# "b3"+___+	($xb2,$xt2)=($xt2,$xb2);+	my ($xc0,$xc1,$xc2,$xc3)=($xt0,$xt1,$xa0,$xa1);+$code.=<<___;+	paddd		0xc0-0x100(%rcx),$xc0+	paddd		0xd0-0x100(%rcx),$xc1+	paddd		0xe0-0x100(%rcx),$xc2+	paddd		0xf0-0x100(%rcx),$xc3++	movdqa		$xa2,0x20(%rsp)		# keep offloading $xaN+	movdqa		$xa3,0x30(%rsp)++	movdqa		$xc0,$xt2+	punpckldq	$xc1,$xc0+	movdqa		$xc2,$xt3+	punpckldq	$xc3,$xc2+	punpckhdq	$xc1,$xt2+	punpckhdq	$xc3,$xt3+	movdqa		$xc0,$xc1+	punpcklqdq	$xc2,$xc0		# "c0"+	movdqa		$xt2,$xc3+	punpcklqdq	$xt3,$xt2		# "c2"+	punpckhqdq	$xc2,$xc1		# "c1"+	punpckhqdq	$xt3,$xc3		# "c3"+___+	($xc2,$xt2)=($xt2,$xc2);+	($xt0,$xt1)=($xa2,$xa3);		# use $xaN as temporary+$code.=<<___;+	paddd		0x100-0x100(%rcx),$xd0+	paddd		0x110-0x100(%rcx),$xd1+	paddd		0x120-0x100(%rcx),$xd2+	paddd		0x130-0x100(%rcx),$xd3++	movdqa		$xd0,$xt2+	punpckldq	$xd1,$xd0+	movdqa		$xd2,$xt3+	punpckldq	$xd3,$xd2+	punpckhdq	$xd1,$xt2+	punpckhdq	$xd3,$xt3+	movdqa		$xd0,$xd1+	punpcklqdq	$xd2,$xd0		# "d0"+	movdqa		$xt2,$xd3+	punpcklqdq	$xt3,$xt2		# "d2"+	punpckhqdq	$xd2,$xd1		# "d1"+	punpckhqdq	$xt3,$xd3		# "d3"+___+	($xd2,$xt2)=($xt2,$xd2);+$code.=<<___;+	cmp		\$64*4,$len+	jb		.Ltail4x++	movdqu		0x00($inp),$xt0		# xor with input+	movdqu		0x10($inp),$xt1+	movdqu		0x20($inp),$xt2+	movdqu		0x30($inp),$xt3+	pxor		0x00(%rsp),$xt0		# $xaN is offloaded, remember?+	pxor		$xb0,$xt1+	pxor		$xc0,$xt2+	pxor		$xd0,$xt3++	 movdqu		$xt0,0x00($out)+	movdqu		0x40($inp),$xt0+	 movdqu		$xt1,0x10($out)+	movdqu		0x50($inp),$xt1+	 movdqu		$xt2,0x20($out)+	movdqu		0x60($inp),$xt2+	 movdqu		$xt3,0x30($out)+	movdqu		0x70($inp),$xt3+	lea		0x80($inp),$inp		# size optimization+	pxor		0x10(%rsp),$xt0+	pxor		$xb1,$xt1+	pxor		$xc1,$xt2+	pxor		$xd1,$xt3++	 movdqu		$xt0,0x40($out)+	movdqu		0x00($inp),$xt0+	 movdqu		$xt1,0x50($out)+	movdqu		0x10($inp),$xt1+	 movdqu		$xt2,0x60($out)+	movdqu		0x20($inp),$xt2+	 movdqu		$xt3,0x70($out)+	 lea		0x80($out),$out		# size optimization+	movdqu		0x30($inp),$xt3+	pxor		0x20(%rsp),$xt0+	pxor		$xb2,$xt1+	pxor		$xc2,$xt2+	pxor		$xd2,$xt3++	 movdqu		$xt0,0x00($out)+	movdqu		0x40($inp),$xt0+	 movdqu		$xt1,0x10($out)+	movdqu		0x50($inp),$xt1+	 movdqu		$xt2,0x20($out)+	movdqu		0x60($inp),$xt2+	 movdqu		$xt3,0x30($out)+	movdqu		0x70($inp),$xt3+	lea		0x80($inp),$inp		# inp+=64*4+	pxor		0x30(%rsp),$xt0+	pxor		$xb3,$xt1+	pxor		$xc3,$xt2+	pxor		$xd3,$xt3+	movdqu		$xt0,0x40($out)+	movdqu		$xt1,0x50($out)+	movdqu		$xt2,0x60($out)+	movdqu		$xt3,0x70($out)+	lea		0x80($out),$out		# out+=64*4++	sub		\$64*4,$len+	jnz		.Loop_outer4x++	jmp		.Ldone4x++.Ltail4x:+	cmp		\$192,$len+	jae		.L192_or_more4x+	cmp		\$128,$len+	jae		.L128_or_more4x+	cmp		\$64,$len+	jae		.L64_or_more4x++	#movdqa		0x00(%rsp),$xt0		# $xaN is offloaded, remember?+	xor		%r9,%r9+	#movdqa		$xt0,0x00(%rsp)+	movdqa		$xb0,0x10(%rsp)+	movdqa		$xc0,0x20(%rsp)+	movdqa		$xd0,0x30(%rsp)+	jmp		.Loop_tail4x++.align	32+.L64_or_more4x:+	movdqu		0x00($inp),$xt0		# xor with input+	movdqu		0x10($inp),$xt1+	movdqu		0x20($inp),$xt2+	movdqu		0x30($inp),$xt3+	pxor		0x00(%rsp),$xt0		# $xaxN is offloaded, remember?+	pxor		$xb0,$xt1+	pxor		$xc0,$xt2+	pxor		$xd0,$xt3+	movdqu		$xt0,0x00($out)+	movdqu		$xt1,0x10($out)+	movdqu		$xt2,0x20($out)+	movdqu		$xt3,0x30($out)+	je		.Ldone4x++	movdqa		0x10(%rsp),$xt0		# $xaN is offloaded, remember?+	lea		0x40($inp),$inp		# inp+=64*1+	xor		%r9,%r9+	movdqa		$xt0,0x00(%rsp)+	movdqa		$xb1,0x10(%rsp)+	lea		0x40($out),$out		# out+=64*1+	movdqa		$xc1,0x20(%rsp)+	sub		\$64,$len		# len-=64*1+	movdqa		$xd1,0x30(%rsp)+	jmp		.Loop_tail4x++.align	32+.L128_or_more4x:+	movdqu		0x00($inp),$xt0		# xor with input+	movdqu		0x10($inp),$xt1+	movdqu		0x20($inp),$xt2+	movdqu		0x30($inp),$xt3+	pxor		0x00(%rsp),$xt0		# $xaN is offloaded, remember?+	pxor		$xb0,$xt1+	pxor		$xc0,$xt2+	pxor		$xd0,$xt3++	 movdqu		$xt0,0x00($out)+	movdqu		0x40($inp),$xt0+	 movdqu		$xt1,0x10($out)+	movdqu		0x50($inp),$xt1+	 movdqu		$xt2,0x20($out)+	movdqu		0x60($inp),$xt2+	 movdqu		$xt3,0x30($out)+	movdqu		0x70($inp),$xt3+	pxor		0x10(%rsp),$xt0+	pxor		$xb1,$xt1+	pxor		$xc1,$xt2+	pxor		$xd1,$xt3+	movdqu		$xt0,0x40($out)+	movdqu		$xt1,0x50($out)+	movdqu		$xt2,0x60($out)+	movdqu		$xt3,0x70($out)+	je		.Ldone4x++	movdqa		0x20(%rsp),$xt0		# $xaN is offloaded, remember?+	lea		0x80($inp),$inp		# inp+=64*2+	xor		%r9,%r9+	movdqa		$xt0,0x00(%rsp)+	movdqa		$xb2,0x10(%rsp)+	lea		0x80($out),$out		# out+=64*2+	movdqa		$xc2,0x20(%rsp)+	sub		\$128,$len		# len-=64*2+	movdqa		$xd2,0x30(%rsp)+	jmp		.Loop_tail4x++.align	32+.L192_or_more4x:+	movdqu		0x00($inp),$xt0		# xor with input+	movdqu		0x10($inp),$xt1+	movdqu		0x20($inp),$xt2+	movdqu		0x30($inp),$xt3+	pxor		0x00(%rsp),$xt0		# $xaN is offloaded, remember?+	pxor		$xb0,$xt1+	pxor		$xc0,$xt2+	pxor		$xd0,$xt3++	 movdqu		$xt0,0x00($out)+	movdqu		0x40($inp),$xt0+	 movdqu		$xt1,0x10($out)+	movdqu		0x50($inp),$xt1+	 movdqu		$xt2,0x20($out)+	movdqu		0x60($inp),$xt2+	 movdqu		$xt3,0x30($out)+	movdqu		0x70($inp),$xt3+	lea		0x80($inp),$inp		# size optimization+	pxor		0x10(%rsp),$xt0+	pxor		$xb1,$xt1+	pxor		$xc1,$xt2+	pxor		$xd1,$xt3++	 movdqu		$xt0,0x40($out)+	movdqu		0x00($inp),$xt0+	 movdqu		$xt1,0x50($out)+	movdqu		0x10($inp),$xt1+	 movdqu		$xt2,0x60($out)+	movdqu		0x20($inp),$xt2+	 movdqu		$xt3,0x70($out)+	 lea		0x80($out),$out		# size optimization+	movdqu		0x30($inp),$xt3+	pxor		0x20(%rsp),$xt0+	pxor		$xb2,$xt1+	pxor		$xc2,$xt2+	pxor		$xd2,$xt3+	movdqu		$xt0,0x00($out)+	movdqu		$xt1,0x10($out)+	movdqu		$xt2,0x20($out)+	movdqu		$xt3,0x30($out)+	je		.Ldone4x++	movdqa		0x30(%rsp),$xt0		# $xaN is offloaded, remember?+	lea		0x40($inp),$inp		# inp+=64*3+	xor		%r9,%r9+	movdqa		$xt0,0x00(%rsp)+	movdqa		$xb3,0x10(%rsp)+	lea		0x40($out),$out		# out+=64*3+	movdqa		$xc3,0x20(%rsp)+	sub		\$192,$len		# len-=64*3+	movdqa		$xd3,0x30(%rsp)++.Loop_tail4x:+	movzb		($inp,%r9),%eax+	movzb		(%rsp,%r9),%ecx+	lea		1(%r9),%r9+	xor		%ecx,%eax+	mov		%al,-1($out,%r9)+	dec		$len+	jnz		.Loop_tail4x++.Ldone4x:+___+$code.=<<___	if ($win64);+	movaps		-0xa8(%r10),%xmm6+	movaps		-0x98(%r10),%xmm7+	movaps		-0x88(%r10),%xmm8+	movaps		-0x78(%r10),%xmm9+	movaps		-0x68(%r10),%xmm10+	movaps		-0x58(%r10),%xmm11+	movaps		-0x48(%r10),%xmm12+	movaps		-0x38(%r10),%xmm13+	movaps		-0x28(%r10),%xmm14+	movaps		-0x18(%r10),%xmm15+___+$code.=<<___;+	lea		(%r10),%rsp+.cfi_def_cfa_register	%rsp+.L4x_epilogue:+	ret+.cfi_endproc+.size	ChaCha20_4x,.-ChaCha20_4x+___+}++########################################################################+# XOP code path that handles all lengths.+if ($avx) {+# There is some "anomaly" observed depending on instructions' size or+# alignment. If you look closely at below code you'll notice that+# sometimes argument order varies. The order affects instruction+# encoding by making it larger, and such fiddling gives 5% performance+# improvement. This is on FX-4100...++my ($xb0,$xb1,$xb2,$xb3, $xd0,$xd1,$xd2,$xd3,+    $xa0,$xa1,$xa2,$xa3, $xt0,$xt1,$xt2,$xt3)=map("%xmm$_",(0..15));+my  @xx=($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3,+	 $xt0,$xt1,$xt2,$xt3, $xd0,$xd1,$xd2,$xd3);++sub XOP_lane_ROUND {+my ($a0,$b0,$c0,$d0)=@_;+my ($a1,$b1,$c1,$d1)=map(($_&~3)+(($_+1)&3),($a0,$b0,$c0,$d0));+my ($a2,$b2,$c2,$d2)=map(($_&~3)+(($_+1)&3),($a1,$b1,$c1,$d1));+my ($a3,$b3,$c3,$d3)=map(($_&~3)+(($_+1)&3),($a2,$b2,$c2,$d2));+my @x=map("\"$_\"",@xx);++	(+	"&vpaddd	(@x[$a0],@x[$a0],@x[$b0])",	# Q1+	 "&vpaddd	(@x[$a1],@x[$a1],@x[$b1])",	# Q2+	  "&vpaddd	(@x[$a2],@x[$a2],@x[$b2])",	# Q3+	   "&vpaddd	(@x[$a3],@x[$a3],@x[$b3])",	# Q4+	"&vpxor		(@x[$d0],@x[$a0],@x[$d0])",+	 "&vpxor	(@x[$d1],@x[$a1],@x[$d1])",+	  "&vpxor	(@x[$d2],@x[$a2],@x[$d2])",+	   "&vpxor	(@x[$d3],@x[$a3],@x[$d3])",+	"&vprotd	(@x[$d0],@x[$d0],16)",+	 "&vprotd	(@x[$d1],@x[$d1],16)",+	  "&vprotd	(@x[$d2],@x[$d2],16)",+	   "&vprotd	(@x[$d3],@x[$d3],16)",++	"&vpaddd	(@x[$c0],@x[$c0],@x[$d0])",+	 "&vpaddd	(@x[$c1],@x[$c1],@x[$d1])",+	  "&vpaddd	(@x[$c2],@x[$c2],@x[$d2])",+	   "&vpaddd	(@x[$c3],@x[$c3],@x[$d3])",+	"&vpxor		(@x[$b0],@x[$c0],@x[$b0])",+	 "&vpxor	(@x[$b1],@x[$c1],@x[$b1])",+	  "&vpxor	(@x[$b2],@x[$b2],@x[$c2])",	# flip+	   "&vpxor	(@x[$b3],@x[$b3],@x[$c3])",	# flip+	"&vprotd	(@x[$b0],@x[$b0],12)",+	 "&vprotd	(@x[$b1],@x[$b1],12)",+	  "&vprotd	(@x[$b2],@x[$b2],12)",+	   "&vprotd	(@x[$b3],@x[$b3],12)",++	"&vpaddd	(@x[$a0],@x[$b0],@x[$a0])",	# flip+	 "&vpaddd	(@x[$a1],@x[$b1],@x[$a1])",	# flip+	  "&vpaddd	(@x[$a2],@x[$a2],@x[$b2])",+	   "&vpaddd	(@x[$a3],@x[$a3],@x[$b3])",+	"&vpxor		(@x[$d0],@x[$a0],@x[$d0])",+	 "&vpxor	(@x[$d1],@x[$a1],@x[$d1])",+	  "&vpxor	(@x[$d2],@x[$a2],@x[$d2])",+	   "&vpxor	(@x[$d3],@x[$a3],@x[$d3])",+	"&vprotd	(@x[$d0],@x[$d0],8)",+	 "&vprotd	(@x[$d1],@x[$d1],8)",+	  "&vprotd	(@x[$d2],@x[$d2],8)",+	   "&vprotd	(@x[$d3],@x[$d3],8)",++	"&vpaddd	(@x[$c0],@x[$c0],@x[$d0])",+	 "&vpaddd	(@x[$c1],@x[$c1],@x[$d1])",+	  "&vpaddd	(@x[$c2],@x[$c2],@x[$d2])",+	   "&vpaddd	(@x[$c3],@x[$c3],@x[$d3])",+	"&vpxor		(@x[$b0],@x[$c0],@x[$b0])",+	 "&vpxor	(@x[$b1],@x[$c1],@x[$b1])",+	  "&vpxor	(@x[$b2],@x[$b2],@x[$c2])",	# flip+	   "&vpxor	(@x[$b3],@x[$b3],@x[$c3])",	# flip+	"&vprotd	(@x[$b0],@x[$b0],7)",+	 "&vprotd	(@x[$b1],@x[$b1],7)",+	  "&vprotd	(@x[$b2],@x[$b2],7)",+	   "&vprotd	(@x[$b3],@x[$b3],7)"+	);+}++my $xframe = $win64 ? 0xa8 : 8;++$code.=<<___	if ($flavour =~ /kernel/);+.globl	ChaCha20_4xop+___+$code.=<<___;+.type	ChaCha20_4xop,\@function,5+.align	32+ChaCha20_4xop:+.cfi_startproc+.LChaCha20_4xop:+	mov		%rsp,%r10		# frame pointer+.cfi_def_cfa_register	%r10+	sub		\$0x140+$xframe,%rsp+	and		\$-16,%rsp+___+	################ stack layout+	# +0x00		SIMD equivalent of @x[8-12]+	# ...+	# +0x40		constant copy of key[0-2] smashed by lanes+	# ...+	# +0x100	SIMD counters (with nonce smashed by lanes)+	# ...+	# +0x140+$code.=<<___	if ($win64);+	movaps		%xmm6,-0xa8(%r10)+	movaps		%xmm7,-0x98(%r10)+	movaps		%xmm8,-0x88(%r10)+	movaps		%xmm9,-0x78(%r10)+	movaps		%xmm10,-0x68(%r10)+	movaps		%xmm11,-0x58(%r10)+	movaps		%xmm12,-0x48(%r10)+	movaps		%xmm13,-0x38(%r10)+	movaps		%xmm14,-0x28(%r10)+	movaps		%xmm15,-0x18(%r10)+.L4xop_body:+___+$code.=<<___;+	vzeroupper++	vmovdqa		.Lsigma(%rip),$xa3	# key[0]+	vmovdqu		($key),$xb3		# key[1]+	vmovdqu		16($key),$xt3		# key[2]+	vmovdqu		($counter),$xd3		# key[3]+	lea		0x100(%rsp),%rcx	# size optimization++	vpshufd		\$0x00,$xa3,$xa0	# smash key by lanes...+	vpshufd		\$0x55,$xa3,$xa1+	vmovdqa		$xa0,0x40(%rsp)		# ... and offload+	vpshufd		\$0xaa,$xa3,$xa2+	vmovdqa		$xa1,0x50(%rsp)+	vpshufd		\$0xff,$xa3,$xa3+	vmovdqa		$xa2,0x60(%rsp)+	vmovdqa		$xa3,0x70(%rsp)++	vpshufd		\$0x00,$xb3,$xb0+	vpshufd		\$0x55,$xb3,$xb1+	vmovdqa		$xb0,0x80-0x100(%rcx)+	vpshufd		\$0xaa,$xb3,$xb2+	vmovdqa		$xb1,0x90-0x100(%rcx)+	vpshufd		\$0xff,$xb3,$xb3+	vmovdqa		$xb2,0xa0-0x100(%rcx)+	vmovdqa		$xb3,0xb0-0x100(%rcx)++	vpshufd		\$0x00,$xt3,$xt0	# "$xc0"+	vpshufd		\$0x55,$xt3,$xt1	# "$xc1"+	vmovdqa		$xt0,0xc0-0x100(%rcx)+	vpshufd		\$0xaa,$xt3,$xt2	# "$xc2"+	vmovdqa		$xt1,0xd0-0x100(%rcx)+	vpshufd		\$0xff,$xt3,$xt3	# "$xc3"+	vmovdqa		$xt2,0xe0-0x100(%rcx)+	vmovdqa		$xt3,0xf0-0x100(%rcx)++	vpshufd		\$0x00,$xd3,$xd0+	vpshufd		\$0x55,$xd3,$xd1+	vpaddd		.Linc(%rip),$xd0,$xd0	# don't save counters yet+	vpshufd		\$0xaa,$xd3,$xd2+	vmovdqa		$xd1,0x110-0x100(%rcx)+	vpshufd		\$0xff,$xd3,$xd3+	vmovdqa		$xd2,0x120-0x100(%rcx)+	vmovdqa		$xd3,0x130-0x100(%rcx)++	jmp		.Loop_enter4xop++.align	32+.Loop_outer4xop:+	vmovdqa		0x40(%rsp),$xa0		# re-load smashed key+	vmovdqa		0x50(%rsp),$xa1+	vmovdqa		0x60(%rsp),$xa2+	vmovdqa		0x70(%rsp),$xa3+	vmovdqa		0x80-0x100(%rcx),$xb0+	vmovdqa		0x90-0x100(%rcx),$xb1+	vmovdqa		0xa0-0x100(%rcx),$xb2+	vmovdqa		0xb0-0x100(%rcx),$xb3+	vmovdqa		0xc0-0x100(%rcx),$xt0	# "$xc0"+	vmovdqa		0xd0-0x100(%rcx),$xt1	# "$xc1"+	vmovdqa		0xe0-0x100(%rcx),$xt2	# "$xc2"+	vmovdqa		0xf0-0x100(%rcx),$xt3	# "$xc3"+	vmovdqa		0x100-0x100(%rcx),$xd0+	vmovdqa		0x110-0x100(%rcx),$xd1+	vmovdqa		0x120-0x100(%rcx),$xd2+	vmovdqa		0x130-0x100(%rcx),$xd3+	vpaddd		.Lfour(%rip),$xd0,$xd0	# next SIMD counters++.Loop_enter4xop:+	mov		\$10,%eax+	vmovdqa		$xd0,0x100-0x100(%rcx)	# save SIMD counters+	jmp		.Loop4xop++.align	32+.Loop4xop:+___+	foreach (&XOP_lane_ROUND(0, 4, 8,12)) { eval; }+	foreach (&XOP_lane_ROUND(0, 5,10,15)) { eval; }+$code.=<<___;+	dec		%eax+	jnz		.Loop4xop++	vpaddd		0x40(%rsp),$xa0,$xa0	# accumulate key material+	vpaddd		0x50(%rsp),$xa1,$xa1+	vpaddd		0x60(%rsp),$xa2,$xa2+	vpaddd		0x70(%rsp),$xa3,$xa3++	vmovdqa		$xt2,0x20(%rsp)		# offload $xc2,3+	vmovdqa		$xt3,0x30(%rsp)++	vpunpckldq	$xa1,$xa0,$xt2		# "de-interlace" data+	vpunpckldq	$xa3,$xa2,$xt3+	vpunpckhdq	$xa1,$xa0,$xa0+	vpunpckhdq	$xa3,$xa2,$xa2+	vpunpcklqdq	$xt3,$xt2,$xa1		# "a0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "a1"+	vpunpcklqdq	$xa2,$xa0,$xa3		# "a2"+	vpunpckhqdq	$xa2,$xa0,$xa0		# "a3"+___+        ($xa0,$xa1,$xa2,$xa3,$xt2)=($xa1,$xt2,$xa3,$xa0,$xa2);+$code.=<<___;+	vpaddd		0x80-0x100(%rcx),$xb0,$xb0+	vpaddd		0x90-0x100(%rcx),$xb1,$xb1+	vpaddd		0xa0-0x100(%rcx),$xb2,$xb2+	vpaddd		0xb0-0x100(%rcx),$xb3,$xb3++	vmovdqa		$xa0,0x00(%rsp)		# offload $xa0,1+	vmovdqa		$xa1,0x10(%rsp)+	vmovdqa		0x20(%rsp),$xa0		# "xc2"+	vmovdqa		0x30(%rsp),$xa1		# "xc3"++	vpunpckldq	$xb1,$xb0,$xt2+	vpunpckldq	$xb3,$xb2,$xt3+	vpunpckhdq	$xb1,$xb0,$xb0+	vpunpckhdq	$xb3,$xb2,$xb2+	vpunpcklqdq	$xt3,$xt2,$xb1		# "b0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "b1"+	vpunpcklqdq	$xb2,$xb0,$xb3		# "b2"+	vpunpckhqdq	$xb2,$xb0,$xb0		# "b3"+___+	($xb0,$xb1,$xb2,$xb3,$xt2)=($xb1,$xt2,$xb3,$xb0,$xb2);+	my ($xc0,$xc1,$xc2,$xc3)=($xt0,$xt1,$xa0,$xa1);+$code.=<<___;+	vpaddd		0xc0-0x100(%rcx),$xc0,$xc0+	vpaddd		0xd0-0x100(%rcx),$xc1,$xc1+	vpaddd		0xe0-0x100(%rcx),$xc2,$xc2+	vpaddd		0xf0-0x100(%rcx),$xc3,$xc3++	vpunpckldq	$xc1,$xc0,$xt2+	vpunpckldq	$xc3,$xc2,$xt3+	vpunpckhdq	$xc1,$xc0,$xc0+	vpunpckhdq	$xc3,$xc2,$xc2+	vpunpcklqdq	$xt3,$xt2,$xc1		# "c0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "c1"+	vpunpcklqdq	$xc2,$xc0,$xc3		# "c2"+	vpunpckhqdq	$xc2,$xc0,$xc0		# "c3"+___+	($xc0,$xc1,$xc2,$xc3,$xt2)=($xc1,$xt2,$xc3,$xc0,$xc2);+$code.=<<___;+	vpaddd		0x100-0x100(%rcx),$xd0,$xd0+	vpaddd		0x110-0x100(%rcx),$xd1,$xd1+	vpaddd		0x120-0x100(%rcx),$xd2,$xd2+	vpaddd		0x130-0x100(%rcx),$xd3,$xd3++	vpunpckldq	$xd1,$xd0,$xt2+	vpunpckldq	$xd3,$xd2,$xt3+	vpunpckhdq	$xd1,$xd0,$xd0+	vpunpckhdq	$xd3,$xd2,$xd2+	vpunpcklqdq	$xt3,$xt2,$xd1		# "d0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "d1"+	vpunpcklqdq	$xd2,$xd0,$xd3		# "d2"+	vpunpckhqdq	$xd2,$xd0,$xd0		# "d3"+___+	($xd0,$xd1,$xd2,$xd3,$xt2)=($xd1,$xt2,$xd3,$xd0,$xd2);+	($xa0,$xa1)=($xt2,$xt3);+$code.=<<___;+	vmovdqa		0x00(%rsp),$xa0		# restore $xa0,1+	vmovdqa		0x10(%rsp),$xa1++	cmp		\$64*4,$len+	jb		.Ltail4xop++	vpxor		0x00($inp),$xa0,$xa0	# xor with input+	vpxor		0x10($inp),$xb0,$xb0+	vpxor		0x20($inp),$xc0,$xc0+	vpxor		0x30($inp),$xd0,$xd0+	vpxor		0x40($inp),$xa1,$xa1+	vpxor		0x50($inp),$xb1,$xb1+	vpxor		0x60($inp),$xc1,$xc1+	vpxor		0x70($inp),$xd1,$xd1+	lea		0x80($inp),$inp		# size optimization+	vpxor		0x00($inp),$xa2,$xa2+	vpxor		0x10($inp),$xb2,$xb2+	vpxor		0x20($inp),$xc2,$xc2+	vpxor		0x30($inp),$xd2,$xd2+	vpxor		0x40($inp),$xa3,$xa3+	vpxor		0x50($inp),$xb3,$xb3+	vpxor		0x60($inp),$xc3,$xc3+	vpxor		0x70($inp),$xd3,$xd3+	lea		0x80($inp),$inp		# inp+=64*4++	vmovdqu		$xa0,0x00($out)+	vmovdqu		$xb0,0x10($out)+	vmovdqu		$xc0,0x20($out)+	vmovdqu		$xd0,0x30($out)+	vmovdqu		$xa1,0x40($out)+	vmovdqu		$xb1,0x50($out)+	vmovdqu		$xc1,0x60($out)+	vmovdqu		$xd1,0x70($out)+	lea		0x80($out),$out		# size optimization+	vmovdqu		$xa2,0x00($out)+	vmovdqu		$xb2,0x10($out)+	vmovdqu		$xc2,0x20($out)+	vmovdqu		$xd2,0x30($out)+	vmovdqu		$xa3,0x40($out)+	vmovdqu		$xb3,0x50($out)+	vmovdqu		$xc3,0x60($out)+	vmovdqu		$xd3,0x70($out)+	lea		0x80($out),$out		# out+=64*4++	sub		\$64*4,$len+	jnz		.Loop_outer4xop++	jmp		.Ldone4xop++.align	32+.Ltail4xop:+	cmp		\$192,$len+	jae		.L192_or_more4xop+	cmp		\$128,$len+	jae		.L128_or_more4xop+	cmp		\$64,$len+	jae		.L64_or_more4xop++	xor		%r9,%r9+	vmovdqa		$xa0,0x00(%rsp)+	vmovdqa		$xb0,0x10(%rsp)+	vmovdqa		$xc0,0x20(%rsp)+	vmovdqa		$xd0,0x30(%rsp)+	jmp		.Loop_tail4xop++.align	32+.L64_or_more4xop:+	vpxor		0x00($inp),$xa0,$xa0	# xor with input+	vpxor		0x10($inp),$xb0,$xb0+	vpxor		0x20($inp),$xc0,$xc0+	vpxor		0x30($inp),$xd0,$xd0+	vmovdqu		$xa0,0x00($out)+	vmovdqu		$xb0,0x10($out)+	vmovdqu		$xc0,0x20($out)+	vmovdqu		$xd0,0x30($out)+	je		.Ldone4xop++	lea		0x40($inp),$inp		# inp+=64*1+	vmovdqa		$xa1,0x00(%rsp)+	xor		%r9,%r9+	vmovdqa		$xb1,0x10(%rsp)+	lea		0x40($out),$out		# out+=64*1+	vmovdqa		$xc1,0x20(%rsp)+	sub		\$64,$len		# len-=64*1+	vmovdqa		$xd1,0x30(%rsp)+	jmp		.Loop_tail4xop++.align	32+.L128_or_more4xop:+	vpxor		0x00($inp),$xa0,$xa0	# xor with input+	vpxor		0x10($inp),$xb0,$xb0+	vpxor		0x20($inp),$xc0,$xc0+	vpxor		0x30($inp),$xd0,$xd0+	vpxor		0x40($inp),$xa1,$xa1+	vpxor		0x50($inp),$xb1,$xb1+	vpxor		0x60($inp),$xc1,$xc1+	vpxor		0x70($inp),$xd1,$xd1++	vmovdqu		$xa0,0x00($out)+	vmovdqu		$xb0,0x10($out)+	vmovdqu		$xc0,0x20($out)+	vmovdqu		$xd0,0x30($out)+	vmovdqu		$xa1,0x40($out)+	vmovdqu		$xb1,0x50($out)+	vmovdqu		$xc1,0x60($out)+	vmovdqu		$xd1,0x70($out)+	je		.Ldone4xop++	lea		0x80($inp),$inp		# inp+=64*2+	vmovdqa		$xa2,0x00(%rsp)+	xor		%r9,%r9+	vmovdqa		$xb2,0x10(%rsp)+	lea		0x80($out),$out		# out+=64*2+	vmovdqa		$xc2,0x20(%rsp)+	sub		\$128,$len		# len-=64*2+	vmovdqa		$xd2,0x30(%rsp)+	jmp		.Loop_tail4xop++.align	32+.L192_or_more4xop:+	vpxor		0x00($inp),$xa0,$xa0	# xor with input+	vpxor		0x10($inp),$xb0,$xb0+	vpxor		0x20($inp),$xc0,$xc0+	vpxor		0x30($inp),$xd0,$xd0+	vpxor		0x40($inp),$xa1,$xa1+	vpxor		0x50($inp),$xb1,$xb1+	vpxor		0x60($inp),$xc1,$xc1+	vpxor		0x70($inp),$xd1,$xd1+	lea		0x80($inp),$inp		# size optimization+	vpxor		0x00($inp),$xa2,$xa2+	vpxor		0x10($inp),$xb2,$xb2+	vpxor		0x20($inp),$xc2,$xc2+	vpxor		0x30($inp),$xd2,$xd2++	vmovdqu		$xa0,0x00($out)+	vmovdqu		$xb0,0x10($out)+	vmovdqu		$xc0,0x20($out)+	vmovdqu		$xd0,0x30($out)+	vmovdqu		$xa1,0x40($out)+	vmovdqu		$xb1,0x50($out)+	vmovdqu		$xc1,0x60($out)+	vmovdqu		$xd1,0x70($out)+	lea		0x80($out),$out		# size optimization+	vmovdqu		$xa2,0x00($out)+	vmovdqu		$xb2,0x10($out)+	vmovdqu		$xc2,0x20($out)+	vmovdqu		$xd2,0x30($out)+	je		.Ldone4xop++	lea		0x40($inp),$inp		# inp+=64*3+	vmovdqa		$xa3,0x00(%rsp)+	xor		%r9,%r9+	vmovdqa		$xb3,0x10(%rsp)+	lea		0x40($out),$out		# out+=64*3+	vmovdqa		$xc3,0x20(%rsp)+	sub		\$192,$len		# len-=64*3+	vmovdqa		$xd3,0x30(%rsp)++.Loop_tail4xop:+	movzb		($inp,%r9),%eax+	movzb		(%rsp,%r9),%ecx+	lea		1(%r9),%r9+	xor		%ecx,%eax+	mov		%al,-1($out,%r9)+	dec		$len+	jnz		.Loop_tail4xop++.Ldone4xop:+	vzeroupper+___+$code.=<<___	if ($win64);+	movaps		-0xa8(%r10),%xmm6+	movaps		-0x98(%r10),%xmm7+	movaps		-0x88(%r10),%xmm8+	movaps		-0x78(%r10),%xmm9+	movaps		-0x68(%r10),%xmm10+	movaps		-0x58(%r10),%xmm11+	movaps		-0x48(%r10),%xmm12+	movaps		-0x38(%r10),%xmm13+	movaps		-0x28(%r10),%xmm14+	movaps		-0x18(%r10),%xmm15+___+$code.=<<___;+	lea		(%r10),%rsp+.cfi_def_cfa_register	%rsp+.L4xop_epilogue:+	ret+.cfi_endproc+.size	ChaCha20_4xop,.-ChaCha20_4xop+___+}++########################################################################+# AVX2 code path+if ($avx>1) {+my ($xb0,$xb1,$xb2,$xb3, $xd0,$xd1,$xd2,$xd3,+    $xa0,$xa1,$xa2,$xa3, $xt0,$xt1,$xt2,$xt3)=map("%ymm$_",(0..15));+my @xx=($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3,+	"%nox","%nox","%nox","%nox", $xd0,$xd1,$xd2,$xd3);++sub AVX2_lane_ROUND {+my ($a0,$b0,$c0,$d0)=@_;+my ($a1,$b1,$c1,$d1)=map(($_&~3)+(($_+1)&3),($a0,$b0,$c0,$d0));+my ($a2,$b2,$c2,$d2)=map(($_&~3)+(($_+1)&3),($a1,$b1,$c1,$d1));+my ($a3,$b3,$c3,$d3)=map(($_&~3)+(($_+1)&3),($a2,$b2,$c2,$d2));+my ($xc,$xc_,$t0,$t1)=map("\"$_\"",$xt0,$xt1,$xt2,$xt3);+my @x=map("\"$_\"",@xx);++	# Consider order in which variables are addressed by their+	# index:+	#+	#	a   b   c   d+	#+	#	0   4   8  12 < even round+	#	1   5   9  13+	#	2   6  10  14+	#	3   7  11  15+	#	0   5  10  15 < odd round+	#	1   6  11  12+	#	2   7   8  13+	#	3   4   9  14+	#+	# 'a', 'b' and 'd's are permanently allocated in registers,+	# @x[0..7,12..15], while 'c's are maintained in memory. If+	# you observe 'c' column, you'll notice that pair of 'c's is+	# invariant between rounds. This means that we have to reload+	# them once per round, in the middle. This is why you'll see+	# bunch of 'c' stores and loads in the middle, but none in+	# the beginning or end.++	(+	"&vpaddd	(@x[$a0],@x[$a0],@x[$b0])",	# Q1+	"&vpxor		(@x[$d0],@x[$a0],@x[$d0])",+	"&vpshufb	(@x[$d0],@x[$d0],$t1)",+	 "&vpaddd	(@x[$a1],@x[$a1],@x[$b1])",	# Q2+	 "&vpxor	(@x[$d1],@x[$a1],@x[$d1])",+	 "&vpshufb	(@x[$d1],@x[$d1],$t1)",++	"&vpaddd	($xc,$xc,@x[$d0])",+	"&vpxor		(@x[$b0],$xc,@x[$b0])",+	"&vpslld	($t0,@x[$b0],12)",+	"&vpsrld	(@x[$b0],@x[$b0],20)",+	"&vpor		(@x[$b0],$t0,@x[$b0])",+	"&vbroadcasti128($t0,'(%r11)')",		# .Lrot24(%rip)+	 "&vpaddd	($xc_,$xc_,@x[$d1])",+	 "&vpxor	(@x[$b1],$xc_,@x[$b1])",+	 "&vpslld	($t1,@x[$b1],12)",+	 "&vpsrld	(@x[$b1],@x[$b1],20)",+	 "&vpor		(@x[$b1],$t1,@x[$b1])",++	"&vpaddd	(@x[$a0],@x[$a0],@x[$b0])",+	"&vpxor		(@x[$d0],@x[$a0],@x[$d0])",+	"&vpshufb	(@x[$d0],@x[$d0],$t0)",+	 "&vpaddd	(@x[$a1],@x[$a1],@x[$b1])",+	 "&vpxor	(@x[$d1],@x[$a1],@x[$d1])",+	 "&vpshufb	(@x[$d1],@x[$d1],$t0)",++	"&vpaddd	($xc,$xc,@x[$d0])",+	"&vpxor		(@x[$b0],$xc,@x[$b0])",+	"&vpslld	($t1,@x[$b0],7)",+	"&vpsrld	(@x[$b0],@x[$b0],25)",+	"&vpor		(@x[$b0],$t1,@x[$b0])",+	"&vbroadcasti128($t1,'(%r9)')",			# .Lrot16(%rip)+	 "&vpaddd	($xc_,$xc_,@x[$d1])",+	 "&vpxor	(@x[$b1],$xc_,@x[$b1])",+	 "&vpslld	($t0,@x[$b1],7)",+	 "&vpsrld	(@x[$b1],@x[$b1],25)",+	 "&vpor		(@x[$b1],$t0,@x[$b1])",++	"&vmovdqa	(\"`32*($c0-8)`(%rsp)\",$xc)",	# reload pair of 'c's+	 "&vmovdqa	(\"`32*($c1-8)`(%rsp)\",$xc_)",+	"&vmovdqa	($xc,\"`32*($c2-8)`(%rsp)\")",+	 "&vmovdqa	($xc_,\"`32*($c3-8)`(%rsp)\")",++	"&vpaddd	(@x[$a2],@x[$a2],@x[$b2])",	# Q3+	"&vpxor		(@x[$d2],@x[$a2],@x[$d2])",+	"&vpshufb	(@x[$d2],@x[$d2],$t1)",+	 "&vpaddd	(@x[$a3],@x[$a3],@x[$b3])",	# Q4+	 "&vpxor	(@x[$d3],@x[$a3],@x[$d3])",+	 "&vpshufb	(@x[$d3],@x[$d3],$t1)",++	"&vpaddd	($xc,$xc,@x[$d2])",+	"&vpxor		(@x[$b2],$xc,@x[$b2])",+	"&vpslld	($t0,@x[$b2],12)",+	"&vpsrld	(@x[$b2],@x[$b2],20)",+	"&vpor		(@x[$b2],$t0,@x[$b2])",+	"&vbroadcasti128($t0,'(%r11)')",		# .Lrot24(%rip)+	 "&vpaddd	($xc_,$xc_,@x[$d3])",+	 "&vpxor	(@x[$b3],$xc_,@x[$b3])",+	 "&vpslld	($t1,@x[$b3],12)",+	 "&vpsrld	(@x[$b3],@x[$b3],20)",+	 "&vpor		(@x[$b3],$t1,@x[$b3])",++	"&vpaddd	(@x[$a2],@x[$a2],@x[$b2])",+	"&vpxor		(@x[$d2],@x[$a2],@x[$d2])",+	"&vpshufb	(@x[$d2],@x[$d2],$t0)",+	 "&vpaddd	(@x[$a3],@x[$a3],@x[$b3])",+	 "&vpxor	(@x[$d3],@x[$a3],@x[$d3])",+	 "&vpshufb	(@x[$d3],@x[$d3],$t0)",++	"&vpaddd	($xc,$xc,@x[$d2])",+	"&vpxor		(@x[$b2],$xc,@x[$b2])",+	"&vpslld	($t1,@x[$b2],7)",+	"&vpsrld	(@x[$b2],@x[$b2],25)",+	"&vpor		(@x[$b2],$t1,@x[$b2])",+	"&vbroadcasti128($t1,'(%r9)')",			# .Lrot16(%rip)+	 "&vpaddd	($xc_,$xc_,@x[$d3])",+	 "&vpxor	(@x[$b3],$xc_,@x[$b3])",+	 "&vpslld	($t0,@x[$b3],7)",+	 "&vpsrld	(@x[$b3],@x[$b3],25)",+	 "&vpor		(@x[$b3],$t0,@x[$b3])"+	);+}++my $xframe = $win64 ? 0xa8 : 8;++$code.=<<___	if ($flavour =~ /kernel/);+.globl	ChaCha20_avx2+___+$code.=<<___;+.type	ChaCha20_avx2,\@function,5+.align	32+ChaCha20_avx2:+.cfi_startproc+.LChaCha20_8x:+	mov		%rsp,%r10		# frame register+.cfi_def_cfa_register	%r10+	sub		\$0x280+$xframe,%rsp+	and		\$-32,%rsp+___+$code.=<<___	if ($win64);+	movaps		%xmm6,-0xa8(%r10)+	movaps		%xmm7,-0x98(%r10)+	movaps		%xmm8,-0x88(%r10)+	movaps		%xmm9,-0x78(%r10)+	movaps		%xmm10,-0x68(%r10)+	movaps		%xmm11,-0x58(%r10)+	movaps		%xmm12,-0x48(%r10)+	movaps		%xmm13,-0x38(%r10)+	movaps		%xmm14,-0x28(%r10)+	movaps		%xmm15,-0x18(%r10)+.Lavx2_body:+___+$code.=<<___;+	vzeroupper++	################ stack layout+	# +0x00		SIMD equivalent of @x[8-12]+	# ...+	# +0x80		constant copy of key[0-2] smashed by lanes+	# ...+	# +0x200	SIMD counters (with nonce smashed by lanes)+	# ...+	# +0x280++	vbroadcasti128	.Lsigma(%rip),$xa3	# key[0]+	vbroadcasti128	($key),$xb3		# key[1]+	vbroadcasti128	16($key),$xt3		# key[2]+	vbroadcasti128	($counter),$xd3		# key[3]+	lea		0x100(%rsp),%rcx	# size optimization+	lea		0x200(%rsp),%rax	# size optimization+	lea		.Lrot16(%rip),%r9+	lea		.Lrot24(%rip),%r11++	vpshufd		\$0x00,$xa3,$xa0	# smash key by lanes...+	vpshufd		\$0x55,$xa3,$xa1+	vmovdqa		$xa0,0x80-0x100(%rcx)	# ... and offload+	vpshufd		\$0xaa,$xa3,$xa2+	vmovdqa		$xa1,0xa0-0x100(%rcx)+	vpshufd		\$0xff,$xa3,$xa3+	vmovdqa		$xa2,0xc0-0x100(%rcx)+	vmovdqa		$xa3,0xe0-0x100(%rcx)++	vpshufd		\$0x00,$xb3,$xb0+	vpshufd		\$0x55,$xb3,$xb1+	vmovdqa		$xb0,0x100-0x100(%rcx)+	vpshufd		\$0xaa,$xb3,$xb2+	vmovdqa		$xb1,0x120-0x100(%rcx)+	vpshufd		\$0xff,$xb3,$xb3+	vmovdqa		$xb2,0x140-0x100(%rcx)+	vmovdqa		$xb3,0x160-0x100(%rcx)++	vpshufd		\$0x00,$xt3,$xt0	# "xc0"+	vpshufd		\$0x55,$xt3,$xt1	# "xc1"+	vmovdqa		$xt0,0x180-0x200(%rax)+	vpshufd		\$0xaa,$xt3,$xt2	# "xc2"+	vmovdqa		$xt1,0x1a0-0x200(%rax)+	vpshufd		\$0xff,$xt3,$xt3	# "xc3"+	vmovdqa		$xt2,0x1c0-0x200(%rax)+	vmovdqa		$xt3,0x1e0-0x200(%rax)++	vpshufd		\$0x00,$xd3,$xd0+	vpshufd		\$0x55,$xd3,$xd1+	vpaddd		.Lincy(%rip),$xd0,$xd0	# don't save counters yet+	vpshufd		\$0xaa,$xd3,$xd2+	vmovdqa		$xd1,0x220-0x200(%rax)+	vpshufd		\$0xff,$xd3,$xd3+	vmovdqa		$xd2,0x240-0x200(%rax)+	vmovdqa		$xd3,0x260-0x200(%rax)++	jmp		.Loop_enter8x++.align	32+.Loop_outer8x:+	vmovdqa		0x80-0x100(%rcx),$xa0	# re-load smashed key+	vmovdqa		0xa0-0x100(%rcx),$xa1+	vmovdqa		0xc0-0x100(%rcx),$xa2+	vmovdqa		0xe0-0x100(%rcx),$xa3+	vmovdqa		0x100-0x100(%rcx),$xb0+	vmovdqa		0x120-0x100(%rcx),$xb1+	vmovdqa		0x140-0x100(%rcx),$xb2+	vmovdqa		0x160-0x100(%rcx),$xb3+	vmovdqa		0x180-0x200(%rax),$xt0	# "xc0"+	vmovdqa		0x1a0-0x200(%rax),$xt1	# "xc1"+	vmovdqa		0x1c0-0x200(%rax),$xt2	# "xc2"+	vmovdqa		0x1e0-0x200(%rax),$xt3	# "xc3"+	vmovdqa		0x200-0x200(%rax),$xd0+	vmovdqa		0x220-0x200(%rax),$xd1+	vmovdqa		0x240-0x200(%rax),$xd2+	vmovdqa		0x260-0x200(%rax),$xd3+	vpaddd		.Leight(%rip),$xd0,$xd0	# next SIMD counters++.Loop_enter8x:+	vmovdqa		$xt2,0x40(%rsp)		# SIMD equivalent of "@x[10]"+	vmovdqa		$xt3,0x60(%rsp)		# SIMD equivalent of "@x[11]"+	vbroadcasti128	(%r9),$xt3+	vmovdqa		$xd0,0x200-0x200(%rax)	# save SIMD counters+	mov		\$10,%eax+	jmp		.Loop8x++.align	32+.Loop8x:+___+	foreach (&AVX2_lane_ROUND(0, 4, 8,12)) { eval; }+	foreach (&AVX2_lane_ROUND(0, 5,10,15)) { eval; }+$code.=<<___;+	dec		%eax+	jnz		.Loop8x++	lea		0x200(%rsp),%rax	# size optimization+	vpaddd		0x80-0x100(%rcx),$xa0,$xa0	# accumulate key+	vpaddd		0xa0-0x100(%rcx),$xa1,$xa1+	vpaddd		0xc0-0x100(%rcx),$xa2,$xa2+	vpaddd		0xe0-0x100(%rcx),$xa3,$xa3++	vpunpckldq	$xa1,$xa0,$xt2		# "de-interlace" data+	vpunpckldq	$xa3,$xa2,$xt3+	vpunpckhdq	$xa1,$xa0,$xa0+	vpunpckhdq	$xa3,$xa2,$xa2+	vpunpcklqdq	$xt3,$xt2,$xa1		# "a0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "a1"+	vpunpcklqdq	$xa2,$xa0,$xa3		# "a2"+	vpunpckhqdq	$xa2,$xa0,$xa0		# "a3"+___+	($xa0,$xa1,$xa2,$xa3,$xt2)=($xa1,$xt2,$xa3,$xa0,$xa2);+$code.=<<___;+	vpaddd		0x100-0x100(%rcx),$xb0,$xb0+	vpaddd		0x120-0x100(%rcx),$xb1,$xb1+	vpaddd		0x140-0x100(%rcx),$xb2,$xb2+	vpaddd		0x160-0x100(%rcx),$xb3,$xb3++	vpunpckldq	$xb1,$xb0,$xt2+	vpunpckldq	$xb3,$xb2,$xt3+	vpunpckhdq	$xb1,$xb0,$xb0+	vpunpckhdq	$xb3,$xb2,$xb2+	vpunpcklqdq	$xt3,$xt2,$xb1		# "b0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "b1"+	vpunpcklqdq	$xb2,$xb0,$xb3		# "b2"+	vpunpckhqdq	$xb2,$xb0,$xb0		# "b3"+___+	($xb0,$xb1,$xb2,$xb3,$xt2)=($xb1,$xt2,$xb3,$xb0,$xb2);+$code.=<<___;+	vperm2i128	\$0x20,$xb0,$xa0,$xt3	# "de-interlace" further+	vperm2i128	\$0x31,$xb0,$xa0,$xb0+	vperm2i128	\$0x20,$xb1,$xa1,$xa0+	vperm2i128	\$0x31,$xb1,$xa1,$xb1+	vperm2i128	\$0x20,$xb2,$xa2,$xa1+	vperm2i128	\$0x31,$xb2,$xa2,$xb2+	vperm2i128	\$0x20,$xb3,$xa3,$xa2+	vperm2i128	\$0x31,$xb3,$xa3,$xb3+___+	($xa0,$xa1,$xa2,$xa3,$xt3)=($xt3,$xa0,$xa1,$xa2,$xa3);+	my ($xc0,$xc1,$xc2,$xc3)=($xt0,$xt1,$xa0,$xa1);+$code.=<<___;+	vmovdqa		$xa0,0x00(%rsp)		# offload $xaN+	vmovdqa		$xa1,0x20(%rsp)+	vmovdqa		0x40(%rsp),$xc2		# $xa0+	vmovdqa		0x60(%rsp),$xc3		# $xa1++	vpaddd		0x180-0x200(%rax),$xc0,$xc0+	vpaddd		0x1a0-0x200(%rax),$xc1,$xc1+	vpaddd		0x1c0-0x200(%rax),$xc2,$xc2+	vpaddd		0x1e0-0x200(%rax),$xc3,$xc3++	vpunpckldq	$xc1,$xc0,$xt2+	vpunpckldq	$xc3,$xc2,$xt3+	vpunpckhdq	$xc1,$xc0,$xc0+	vpunpckhdq	$xc3,$xc2,$xc2+	vpunpcklqdq	$xt3,$xt2,$xc1		# "c0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "c1"+	vpunpcklqdq	$xc2,$xc0,$xc3		# "c2"+	vpunpckhqdq	$xc2,$xc0,$xc0		# "c3"+___+	($xc0,$xc1,$xc2,$xc3,$xt2)=($xc1,$xt2,$xc3,$xc0,$xc2);+$code.=<<___;+	vpaddd		0x200-0x200(%rax),$xd0,$xd0+	vpaddd		0x220-0x200(%rax),$xd1,$xd1+	vpaddd		0x240-0x200(%rax),$xd2,$xd2+	vpaddd		0x260-0x200(%rax),$xd3,$xd3++	vpunpckldq	$xd1,$xd0,$xt2+	vpunpckldq	$xd3,$xd2,$xt3+	vpunpckhdq	$xd1,$xd0,$xd0+	vpunpckhdq	$xd3,$xd2,$xd2+	vpunpcklqdq	$xt3,$xt2,$xd1		# "d0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "d1"+	vpunpcklqdq	$xd2,$xd0,$xd3		# "d2"+	vpunpckhqdq	$xd2,$xd0,$xd0		# "d3"+___+	($xd0,$xd1,$xd2,$xd3,$xt2)=($xd1,$xt2,$xd3,$xd0,$xd2);+$code.=<<___;+	vperm2i128	\$0x20,$xd0,$xc0,$xt3	# "de-interlace" further+	vperm2i128	\$0x31,$xd0,$xc0,$xd0+	vperm2i128	\$0x20,$xd1,$xc1,$xc0+	vperm2i128	\$0x31,$xd1,$xc1,$xd1+	vperm2i128	\$0x20,$xd2,$xc2,$xc1+	vperm2i128	\$0x31,$xd2,$xc2,$xd2+	vperm2i128	\$0x20,$xd3,$xc3,$xc2+	vperm2i128	\$0x31,$xd3,$xc3,$xd3+___+	($xc0,$xc1,$xc2,$xc3,$xt3)=($xt3,$xc0,$xc1,$xc2,$xc3);+	($xb0,$xb1,$xb2,$xb3,$xc0,$xc1,$xc2,$xc3)=+	($xc0,$xc1,$xc2,$xc3,$xb0,$xb1,$xb2,$xb3);+	($xa0,$xa1)=($xt2,$xt3);+$code.=<<___;+	vmovdqa		0x00(%rsp),$xa0		# $xaN was offloaded, remember?+	vmovdqa		0x20(%rsp),$xa1++	cmp		\$64*8,$len+	jb		.Ltail8x++	vpxor		0x00($inp),$xa0,$xa0	# xor with input+	vpxor		0x20($inp),$xb0,$xb0+	vpxor		0x40($inp),$xc0,$xc0+	vpxor		0x60($inp),$xd0,$xd0+	lea		0x80($inp),$inp		# size optimization+	vmovdqu		$xa0,0x00($out)+	vmovdqu		$xb0,0x20($out)+	vmovdqu		$xc0,0x40($out)+	vmovdqu		$xd0,0x60($out)+	lea		0x80($out),$out		# size optimization++	vpxor		0x00($inp),$xa1,$xa1+	vpxor		0x20($inp),$xb1,$xb1+	vpxor		0x40($inp),$xc1,$xc1+	vpxor		0x60($inp),$xd1,$xd1+	lea		0x80($inp),$inp		# size optimization+	vmovdqu		$xa1,0x00($out)+	vmovdqu		$xb1,0x20($out)+	vmovdqu		$xc1,0x40($out)+	vmovdqu		$xd1,0x60($out)+	lea		0x80($out),$out		# size optimization++	vpxor		0x00($inp),$xa2,$xa2+	vpxor		0x20($inp),$xb2,$xb2+	vpxor		0x40($inp),$xc2,$xc2+	vpxor		0x60($inp),$xd2,$xd2+	lea		0x80($inp),$inp		# size optimization+	vmovdqu		$xa2,0x00($out)+	vmovdqu		$xb2,0x20($out)+	vmovdqu		$xc2,0x40($out)+	vmovdqu		$xd2,0x60($out)+	lea		0x80($out),$out		# size optimization++	vpxor		0x00($inp),$xa3,$xa3+	vpxor		0x20($inp),$xb3,$xb3+	vpxor		0x40($inp),$xc3,$xc3+	vpxor		0x60($inp),$xd3,$xd3+	lea		0x80($inp),$inp		# size optimization+	vmovdqu		$xa3,0x00($out)+	vmovdqu		$xb3,0x20($out)+	vmovdqu		$xc3,0x40($out)+	vmovdqu		$xd3,0x60($out)+	lea		0x80($out),$out		# size optimization++	sub		\$64*8,$len+	jnz		.Loop_outer8x++	jmp		.Ldone8x++.Ltail8x:+	cmp		\$448,$len+	jae		.L448_or_more8x+	cmp		\$384,$len+	jae		.L384_or_more8x+	cmp		\$320,$len+	jae		.L320_or_more8x+	cmp		\$256,$len+	jae		.L256_or_more8x+	cmp		\$192,$len+	jae		.L192_or_more8x+	cmp		\$128,$len+	jae		.L128_or_more8x+	cmp		\$64,$len+	jae		.L64_or_more8x++	xor		%r9,%r9+	vmovdqa		$xa0,0x00(%rsp)+	vmovdqa		$xb0,0x20(%rsp)+	jmp		.Loop_tail8x++.align	32+.L64_or_more8x:+	vpxor		0x00($inp),$xa0,$xa0	# xor with input+	vpxor		0x20($inp),$xb0,$xb0+	vmovdqu		$xa0,0x00($out)+	vmovdqu		$xb0,0x20($out)+	je		.Ldone8x++	lea		0x40($inp),$inp		# inp+=64*1+	xor		%r9,%r9+	vmovdqa		$xc0,0x00(%rsp)+	lea		0x40($out),$out		# out+=64*1+	sub		\$64,$len		# len-=64*1+	vmovdqa		$xd0,0x20(%rsp)+	jmp		.Loop_tail8x++.align	32+.L128_or_more8x:+	vpxor		0x00($inp),$xa0,$xa0	# xor with input+	vpxor		0x20($inp),$xb0,$xb0+	vpxor		0x40($inp),$xc0,$xc0+	vpxor		0x60($inp),$xd0,$xd0+	vmovdqu		$xa0,0x00($out)+	vmovdqu		$xb0,0x20($out)+	vmovdqu		$xc0,0x40($out)+	vmovdqu		$xd0,0x60($out)+	je		.Ldone8x++	lea		0x80($inp),$inp		# inp+=64*2+	xor		%r9,%r9+	vmovdqa		$xa1,0x00(%rsp)+	lea		0x80($out),$out		# out+=64*2+	sub		\$128,$len		# len-=64*2+	vmovdqa		$xb1,0x20(%rsp)+	jmp		.Loop_tail8x++.align	32+.L192_or_more8x:+	vpxor		0x00($inp),$xa0,$xa0	# xor with input+	vpxor		0x20($inp),$xb0,$xb0+	vpxor		0x40($inp),$xc0,$xc0+	vpxor		0x60($inp),$xd0,$xd0+	vpxor		0x80($inp),$xa1,$xa1+	vpxor		0xa0($inp),$xb1,$xb1+	vmovdqu		$xa0,0x00($out)+	vmovdqu		$xb0,0x20($out)+	vmovdqu		$xc0,0x40($out)+	vmovdqu		$xd0,0x60($out)+	vmovdqu		$xa1,0x80($out)+	vmovdqu		$xb1,0xa0($out)+	je		.Ldone8x++	lea		0xc0($inp),$inp		# inp+=64*3+	xor		%r9,%r9+	vmovdqa		$xc1,0x00(%rsp)+	lea		0xc0($out),$out		# out+=64*3+	sub		\$192,$len		# len-=64*3+	vmovdqa		$xd1,0x20(%rsp)+	jmp		.Loop_tail8x++.align	32+.L256_or_more8x:+	vpxor		0x00($inp),$xa0,$xa0	# xor with input+	vpxor		0x20($inp),$xb0,$xb0+	vpxor		0x40($inp),$xc0,$xc0+	vpxor		0x60($inp),$xd0,$xd0+	vpxor		0x80($inp),$xa1,$xa1+	vpxor		0xa0($inp),$xb1,$xb1+	vpxor		0xc0($inp),$xc1,$xc1+	vpxor		0xe0($inp),$xd1,$xd1+	vmovdqu		$xa0,0x00($out)+	vmovdqu		$xb0,0x20($out)+	vmovdqu		$xc0,0x40($out)+	vmovdqu		$xd0,0x60($out)+	vmovdqu		$xa1,0x80($out)+	vmovdqu		$xb1,0xa0($out)+	vmovdqu		$xc1,0xc0($out)+	vmovdqu		$xd1,0xe0($out)+	je		.Ldone8x++	lea		0x100($inp),$inp	# inp+=64*4+	xor		%r9,%r9+	vmovdqa		$xa2,0x00(%rsp)+	lea		0x100($out),$out	# out+=64*4+	sub		\$256,$len		# len-=64*4+	vmovdqa		$xb2,0x20(%rsp)+	jmp		.Loop_tail8x++.align	32+.L320_or_more8x:+	vpxor		0x00($inp),$xa0,$xa0	# xor with input+	vpxor		0x20($inp),$xb0,$xb0+	vpxor		0x40($inp),$xc0,$xc0+	vpxor		0x60($inp),$xd0,$xd0+	vpxor		0x80($inp),$xa1,$xa1+	vpxor		0xa0($inp),$xb1,$xb1+	vpxor		0xc0($inp),$xc1,$xc1+	vpxor		0xe0($inp),$xd1,$xd1+	vpxor		0x100($inp),$xa2,$xa2+	vpxor		0x120($inp),$xb2,$xb2+	vmovdqu		$xa0,0x00($out)+	vmovdqu		$xb0,0x20($out)+	vmovdqu		$xc0,0x40($out)+	vmovdqu		$xd0,0x60($out)+	vmovdqu		$xa1,0x80($out)+	vmovdqu		$xb1,0xa0($out)+	vmovdqu		$xc1,0xc0($out)+	vmovdqu		$xd1,0xe0($out)+	vmovdqu		$xa2,0x100($out)+	vmovdqu		$xb2,0x120($out)+	je		.Ldone8x++	lea		0x140($inp),$inp	# inp+=64*5+	xor		%r9,%r9+	vmovdqa		$xc2,0x00(%rsp)+	lea		0x140($out),$out	# out+=64*5+	sub		\$320,$len		# len-=64*5+	vmovdqa		$xd2,0x20(%rsp)+	jmp		.Loop_tail8x++.align	32+.L384_or_more8x:+	vpxor		0x00($inp),$xa0,$xa0	# xor with input+	vpxor		0x20($inp),$xb0,$xb0+	vpxor		0x40($inp),$xc0,$xc0+	vpxor		0x60($inp),$xd0,$xd0+	vpxor		0x80($inp),$xa1,$xa1+	vpxor		0xa0($inp),$xb1,$xb1+	vpxor		0xc0($inp),$xc1,$xc1+	vpxor		0xe0($inp),$xd1,$xd1+	vpxor		0x100($inp),$xa2,$xa2+	vpxor		0x120($inp),$xb2,$xb2+	vpxor		0x140($inp),$xc2,$xc2+	vpxor		0x160($inp),$xd2,$xd2+	vmovdqu		$xa0,0x00($out)+	vmovdqu		$xb0,0x20($out)+	vmovdqu		$xc0,0x40($out)+	vmovdqu		$xd0,0x60($out)+	vmovdqu		$xa1,0x80($out)+	vmovdqu		$xb1,0xa0($out)+	vmovdqu		$xc1,0xc0($out)+	vmovdqu		$xd1,0xe0($out)+	vmovdqu		$xa2,0x100($out)+	vmovdqu		$xb2,0x120($out)+	vmovdqu		$xc2,0x140($out)+	vmovdqu		$xd2,0x160($out)+	je		.Ldone8x++	lea		0x180($inp),$inp	# inp+=64*6+	xor		%r9,%r9+	vmovdqa		$xa3,0x00(%rsp)+	lea		0x180($out),$out	# out+=64*6+	sub		\$384,$len		# len-=64*6+	vmovdqa		$xb3,0x20(%rsp)+	jmp		.Loop_tail8x++.align	32+.L448_or_more8x:+	vpxor		0x00($inp),$xa0,$xa0	# xor with input+	vpxor		0x20($inp),$xb0,$xb0+	vpxor		0x40($inp),$xc0,$xc0+	vpxor		0x60($inp),$xd0,$xd0+	vpxor		0x80($inp),$xa1,$xa1+	vpxor		0xa0($inp),$xb1,$xb1+	vpxor		0xc0($inp),$xc1,$xc1+	vpxor		0xe0($inp),$xd1,$xd1+	vpxor		0x100($inp),$xa2,$xa2+	vpxor		0x120($inp),$xb2,$xb2+	vpxor		0x140($inp),$xc2,$xc2+	vpxor		0x160($inp),$xd2,$xd2+	vpxor		0x180($inp),$xa3,$xa3+	vpxor		0x1a0($inp),$xb3,$xb3+	vmovdqu		$xa0,0x00($out)+	vmovdqu		$xb0,0x20($out)+	vmovdqu		$xc0,0x40($out)+	vmovdqu		$xd0,0x60($out)+	vmovdqu		$xa1,0x80($out)+	vmovdqu		$xb1,0xa0($out)+	vmovdqu		$xc1,0xc0($out)+	vmovdqu		$xd1,0xe0($out)+	vmovdqu		$xa2,0x100($out)+	vmovdqu		$xb2,0x120($out)+	vmovdqu		$xc2,0x140($out)+	vmovdqu		$xd2,0x160($out)+	vmovdqu		$xa3,0x180($out)+	vmovdqu		$xb3,0x1a0($out)+	je		.Ldone8x++	lea		0x1c0($inp),$inp	# inp+=64*7+	xor		%r9,%r9+	vmovdqa		$xc3,0x00(%rsp)+	lea		0x1c0($out),$out	# out+=64*7+	sub		\$448,$len		# len-=64*7+	vmovdqa		$xd3,0x20(%rsp)++.Loop_tail8x:+	movzb		($inp,%r9),%eax+	movzb		(%rsp,%r9),%ecx+	lea		1(%r9),%r9+	xor		%ecx,%eax+	mov		%al,-1($out,%r9)+	dec		$len+	jnz		.Loop_tail8x++.Ldone8x:+	vzeroall+___+$code.=<<___	if ($win64);+	movaps		-0xa8(%r10),%xmm6+	movaps		-0x98(%r10),%xmm7+	movaps		-0x88(%r10),%xmm8+	movaps		-0x78(%r10),%xmm9+	movaps		-0x68(%r10),%xmm10+	movaps		-0x58(%r10),%xmm11+	movaps		-0x48(%r10),%xmm12+	movaps		-0x38(%r10),%xmm13+	movaps		-0x28(%r10),%xmm14+	movaps		-0x18(%r10),%xmm15+___+$code.=<<___;+	lea		(%r10),%rsp+.cfi_def_cfa_register	%rsp+.Lavx2_epilogue:+	ret+.cfi_endproc+.size	ChaCha20_avx2,.-ChaCha20_avx2+___+}++########################################################################+# AVX512 code paths+if ($avx>2) {+# This one handles shorter inputs...++my ($a,$b,$c,$d, $a_,$b_,$c_,$d_,$fourz) = map("%zmm$_",(0..3,16..20));+my ($t0,$t1,$t2,$t3) = map("%xmm$_",(4..7));++sub vpxord()		# size optimization+{ my $opcode = "vpxor";	# adhere to vpxor when possible++    foreach (@_) {+	if (/%([zy])mm([0-9]+)/ && ($1 eq "z" || $2>=16)) {+	    $opcode = "vpxord";+	    last;+	}+    }++    $code .= "\t$opcode\t".join(',',reverse @_)."\n";+}++sub AVX512ROUND {	# critical path is 14 "SIMD ticks" per round+	&vpaddd	($a,$a,$b);+	&vpxord	($d,$d,$a);+	&vprold	($d,$d,16);++	&vpaddd	($c,$c,$d);+	&vpxord	($b,$b,$c);+	&vprold	($b,$b,12);++	&vpaddd	($a,$a,$b);+	&vpxord	($d,$d,$a);+	&vprold	($d,$d,8);++	&vpaddd	($c,$c,$d);+	&vpxord	($b,$b,$c);+	&vprold	($b,$b,7);+}++my $xframe = $win64 ? 32+8 : 8;++$code.=<<___	if ($flavour =~ /kernel/);+.globl	ChaCha20_avx512+___+$code.=<<___;+.type	ChaCha20_avx512,\@function,5+.align	32+ChaCha20_avx512:+.cfi_startproc+.LChaCha20_avx512:+	mov	%rsp,%r10		# frame pointer+.cfi_def_cfa_register	%r10+	cmp	\$512,$len+	ja	.LChaCha20_16x++	sub	\$64+$xframe,%rsp+	and	\$-16,%rsp+___+$code.=<<___	if ($win64);+	movaps	%xmm6,-0x28(%r10)+	movaps	%xmm7,-0x18(%r10)+.Lavx512_body:+___+$code.=<<___;+	vbroadcasti32x4	.Lsigma(%rip),$a+	vbroadcasti32x4	($key),$b_+	vbroadcasti32x4	16($key),$c_+	vbroadcasti32x4	($counter),$d_++	vmovdqa32	$a,$a_+	vmovdqa32	.Lfourz(%rip),$fourz+	vpaddd		.Lzeroz(%rip),$d_,$d+	jmp		.Loop_outer_avx512++.align	32+.Loop_outer_avx512:+	vmovdqa32	$b_,$b+	vmovdqa32	$c_,$c+	vmovdqa32	$d,$d_+	mov		\$10,$counter		# reuse $counter+	jmp		.Loop_avx512++.align	32+.Loop_avx512:+___+	&AVX512ROUND();+	&vpshufd	($c,$c,0b01001110);+	&vpshufd	($b,$b,0b00111001);+	&vpshufd	($d,$d,0b10010011);++	&AVX512ROUND();+	&vpshufd	($c,$c,0b01001110);+	&vpshufd	($b,$b,0b10010011);+	&vpshufd	($d,$d,0b00111001);++	&dec		($counter);+	&jnz		(".Loop_avx512");++$code.=<<___;+	vpaddd		$a_,$a,$a+	vpaddd		$b_,$b,$b+	vpaddd		$c_,$c,$c+	vpaddd		$d_,$d,$d++	sub		\$64,$len+	jb		.Ltail64_avx512++	vpxor		0x00($inp),%x#$a,$t0	# xor with input+	vpxor		0x10($inp),%x#$b,$t1+	vpxor		0x20($inp),%x#$c,$t2+	vpxor		0x30($inp),%x#$d,$t3+	lea		0x40($inp),$inp		# inp+=64++	vmovdqu		$t0,0x00($out)		# write output+	vmovdqu		$t1,0x10($out)+	vmovdqu		$t2,0x20($out)+	vmovdqu		$t3,0x30($out)+	lea		0x40($out),$out		# out+=64++	jz		.Ldone_avx512++	vextracti32x4	\$1,$a,$t0+	vextracti32x4	\$1,$b,$t1+	vextracti32x4	\$1,$c,$t2+	vextracti32x4	\$1,$d,$t3++	sub		\$64,$len+	jb		.Ltail_avx512++	vpxor		0x00($inp),$t0,$t0	# xor with input+	vpxor		0x10($inp),$t1,$t1+	vpxor		0x20($inp),$t2,$t2+	vpxor		0x30($inp),$t3,$t3+	lea		0x40($inp),$inp		# inp+=64++	vmovdqu		$t0,0x00($out)		# write output+	vmovdqu		$t1,0x10($out)+	vmovdqu		$t2,0x20($out)+	vmovdqu		$t3,0x30($out)+	lea		0x40($out),$out		# out+=64++	jz		.Ldone_avx512++	vextracti32x4	\$2,$a,$t0+	vextracti32x4	\$2,$b,$t1+	vextracti32x4	\$2,$c,$t2+	vextracti32x4	\$2,$d,$t3++	sub		\$64,$len+	jb		.Ltail_avx512++	vpxor		0x00($inp),$t0,$t0	# xor with input+	vpxor		0x10($inp),$t1,$t1+	vpxor		0x20($inp),$t2,$t2+	vpxor		0x30($inp),$t3,$t3+	lea		0x40($inp),$inp		# inp+=64++	vmovdqu		$t0,0x00($out)		# write output+	vmovdqu		$t1,0x10($out)+	vmovdqu		$t2,0x20($out)+	vmovdqu		$t3,0x30($out)+	lea		0x40($out),$out		# out+=64++	jz		.Ldone_avx512++	vextracti32x4	\$3,$a,$t0+	vextracti32x4	\$3,$b,$t1+	vextracti32x4	\$3,$c,$t2+	vextracti32x4	\$3,$d,$t3++	sub		\$64,$len+	jb		.Ltail_avx512++	vmovdqa32	$a_,$a+	vpaddd		$fourz,$d_,$d++	vpxor		0x00($inp),$t0,$t0	# xor with input+	vpxor		0x10($inp),$t1,$t1+	vpxor		0x20($inp),$t2,$t2+	vpxor		0x30($inp),$t3,$t3+	lea		0x40($inp),$inp		# inp+=64++	vmovdqu		$t0,0x00($out)		# write output+	vmovdqu		$t1,0x10($out)+	vmovdqu		$t2,0x20($out)+	vmovdqu		$t3,0x30($out)+	lea		0x40($out),$out		# out+=64++	jnz		.Loop_outer_avx512++	jmp		.Ldone_avx512++.align	16+.Ltail64_avx512:+	vmovdqa		%x#$a,0x00(%rsp)+	vmovdqa		%x#$b,0x10(%rsp)+	vmovdqa		%x#$c,0x20(%rsp)+	vmovdqa		%x#$d,0x30(%rsp)+	add		\$64,$len+	jmp		.Loop_tail_avx512++.align	16+.Ltail_avx512:+	vmovdqa		$t0,0x00(%rsp)+	vmovdqa		$t1,0x10(%rsp)+	vmovdqa		$t2,0x20(%rsp)+	vmovdqa		$t3,0x30(%rsp)+	add		\$64,$len++.Loop_tail_avx512:+	movzb		($inp,$counter),%eax+	movzb		(%rsp,$counter),%ecx+	lea		1($counter),$counter+	xor		%ecx,%eax+	mov		%al,-1($out,$counter)+	dec		$len+	jnz		.Loop_tail_avx512++	vmovdqu32	$a_,0x00(%rsp)++.Ldone_avx512:+	vzeroall+___+$code.=<<___	if ($win64);+	movaps	-0x28(%r10),%xmm6+	movaps	-0x18(%r10),%xmm7+___+$code.=<<___;+	lea	(%r10),%rsp+.cfi_def_cfa_register	%rsp+.Lavx512_epilogue:+	ret+.cfi_endproc+.size	ChaCha20_avx512,.-ChaCha20_avx512+___++map(s/%z/%y/, $a,$b,$c,$d, $a_,$b_,$c_,$d_,$fourz);++$code.=<<___	if ($flavour =~ /kernel/);+.globl	ChaCha20_avx512vl+___+$code.=<<___;+.type	ChaCha20_avx512vl,\@function,5+.align	32+ChaCha20_avx512vl:+.cfi_startproc+.LChaCha20_avx512vl:+	mov	%rsp,%r10		# frame pointer+.cfi_def_cfa_register	%r10+	cmp	\$128,$len+	ja	.LChaCha20_8xvl++	sub	\$64+$xframe,%rsp+___+$code.=<<___	if ($win64);+	movaps	%xmm6,-0x28(%r10)+	movaps	%xmm7,-0x18(%r10)+.Lavx512vl_body:+___+$code.=<<___;+	vbroadcasti32x4	.Lsigma(%rip),$a+	vbroadcasti32x4	($key),$b_+	vbroadcasti32x4	16($key),$c_+	vbroadcasti32x4	($counter),$d_++	vmovdqa32	$a,$a_+	vmovdqa32	.Ltwoy(%rip),$fourz+	vpaddd		.Lzeroz(%rip),$d_,$d+	jmp		.Loop_outer_avx512vl++.align	32+.Loop_outer_avx512vl:+	vmovdqa32	$b_,$b+	vmovdqa32	$c_,$c+	vmovdqa32	$d,$d_+	mov		\$10,$counter		# reuse $counter+	jmp		.Loop_avx512vl++.align	32+.Loop_avx512vl:+___+	&AVX512ROUND();+	&vpshufd	($c,$c,0b01001110);+	&vpshufd	($b,$b,0b00111001);+	&vpshufd	($d,$d,0b10010011);++	&AVX512ROUND();+	&vpshufd	($c,$c,0b01001110);+	&vpshufd	($b,$b,0b10010011);+	&vpshufd	($d,$d,0b00111001);++	&sub		($counter,1);+	&jnz		(".Loop_avx512vl");++$code.=<<___;+	vpaddd		$a_,$a,$a+	vpaddd		$b_,$b,$b+	vpaddd		$c_,$c,$c+	vpaddd		$d_,$d,$d++	sub		\$64,$len+	jb		.Ltail64_avx512vl++	vpxor		0x00($inp),%x#$a,$t0	# xor with input+	vpxor		0x10($inp),%x#$b,$t1+	vpxor		0x20($inp),%x#$c,$t2+	vpxor		0x30($inp),%x#$d,$t3+	lea		0x40($inp),$inp		# inp+=64++	vmovdqu		$t0,0x00($out)		# write output+	vmovdqu		$t1,0x10($out)+	vmovdqu		$t2,0x20($out)+	vmovdqu		$t3,0x30($out)+	lea		0x40($out),$out		# out+=64++	jz		.Ldone_avx512vl++	vextracti128	\$1,$a,$t0+	vextracti128	\$1,$b,$t1+	vextracti128	\$1,$c,$t2+	vextracti128	\$1,$d,$t3++	sub		\$64,$len+	jb		.Ltail_avx512vl++	vmovdqa32	$a_,$a+	vpaddd		$fourz,$d_,$d++	vpxor		0x00($inp),$t0,$t0	# xor with input+	vpxor		0x10($inp),$t1,$t1+	vpxor		0x20($inp),$t2,$t2+	vpxor		0x30($inp),$t3,$t3+	lea		0x40($inp),$inp		# inp+=64++	vmovdqu		$t0,0x00($out)		# write output+	vmovdqu		$t1,0x10($out)+	vmovdqu		$t2,0x20($out)+	vmovdqu		$t3,0x30($out)+	lea		0x40($out),$out		# out+=64++	jnz		.Loop_outer_avx512vl++	jmp		.Ldone_avx512vl++.align	16+.Ltail64_avx512vl:+	vmovdqa		%x#$a,0x00(%rsp)+	vmovdqa		%x#$b,0x10(%rsp)+	vmovdqa		%x#$c,0x20(%rsp)+	vmovdqa		%x#$d,0x30(%rsp)+	add		\$64,$len+	jmp		.Loop_tail_avx512vl++.align	16+.Ltail_avx512vl:+	vmovdqa		$t0,0x00(%rsp)+	vmovdqa		$t1,0x10(%rsp)+	vmovdqa		$t2,0x20(%rsp)+	vmovdqa		$t3,0x30(%rsp)+	add		\$64,$len++.Loop_tail_avx512vl:+	movzb		($inp,$counter),%eax+	movzb		(%rsp,$counter),%ecx+	lea		1($counter),$counter+	xor		%ecx,%eax+	mov		%al,-1($out,$counter)+	dec		$len+	jnz		.Loop_tail_avx512vl++	vmovdqu32	$a_,0x00(%rsp)+	vmovdqu32	$a_,0x20(%rsp)++.Ldone_avx512vl:+	vzeroall+___+$code.=<<___	if ($win64);+	movaps	-0x28(%r10),%xmm6+	movaps	-0x18(%r10),%xmm7+___+$code.=<<___;+	lea	(%r10),%rsp+.cfi_def_cfa_register	%rsp+.Lavx512vl_epilogue:+	ret+.cfi_endproc+.size	ChaCha20_avx512vl,.-ChaCha20_avx512vl+___+}+if ($avx>2) {+# This one handles longer inputs...++my ($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3,+    $xc0,$xc1,$xc2,$xc3, $xd0,$xd1,$xd2,$xd3)=map("%zmm$_",(0..15));+my  @xx=($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3,+	 $xc0,$xc1,$xc2,$xc3, $xd0,$xd1,$xd2,$xd3);+my @key=map("%zmm$_",(16..31));+my ($xt0,$xt1,$xt2,$xt3)=@key[0..3];++sub AVX512_lane_ROUND {+my ($a0,$b0,$c0,$d0)=@_;+my ($a1,$b1,$c1,$d1)=map(($_&~3)+(($_+1)&3),($a0,$b0,$c0,$d0));+my ($a2,$b2,$c2,$d2)=map(($_&~3)+(($_+1)&3),($a1,$b1,$c1,$d1));+my ($a3,$b3,$c3,$d3)=map(($_&~3)+(($_+1)&3),($a2,$b2,$c2,$d2));+my @x=map("\"$_\"",@xx);++	(+	"&vpaddd	(@x[$a0],@x[$a0],@x[$b0])",	# Q1+	 "&vpaddd	(@x[$a1],@x[$a1],@x[$b1])",	# Q2+	  "&vpaddd	(@x[$a2],@x[$a2],@x[$b2])",	# Q3+	   "&vpaddd	(@x[$a3],@x[$a3],@x[$b3])",	# Q4+	"&vpxord	(@x[$d0],@x[$d0],@x[$a0])",+	 "&vpxord	(@x[$d1],@x[$d1],@x[$a1])",+	  "&vpxord	(@x[$d2],@x[$d2],@x[$a2])",+	   "&vpxord	(@x[$d3],@x[$d3],@x[$a3])",+	"&vprold	(@x[$d0],@x[$d0],16)",+	 "&vprold	(@x[$d1],@x[$d1],16)",+	  "&vprold	(@x[$d2],@x[$d2],16)",+	   "&vprold	(@x[$d3],@x[$d3],16)",++	"&vpaddd	(@x[$c0],@x[$c0],@x[$d0])",+	 "&vpaddd	(@x[$c1],@x[$c1],@x[$d1])",+	  "&vpaddd	(@x[$c2],@x[$c2],@x[$d2])",+	   "&vpaddd	(@x[$c3],@x[$c3],@x[$d3])",+	"&vpxord	(@x[$b0],@x[$b0],@x[$c0])",+	 "&vpxord	(@x[$b1],@x[$b1],@x[$c1])",+	  "&vpxord	(@x[$b2],@x[$b2],@x[$c2])",+	   "&vpxord	(@x[$b3],@x[$b3],@x[$c3])",+	"&vprold	(@x[$b0],@x[$b0],12)",+	 "&vprold	(@x[$b1],@x[$b1],12)",+	  "&vprold	(@x[$b2],@x[$b2],12)",+	   "&vprold	(@x[$b3],@x[$b3],12)",++	"&vpaddd	(@x[$a0],@x[$a0],@x[$b0])",+	 "&vpaddd	(@x[$a1],@x[$a1],@x[$b1])",+	  "&vpaddd	(@x[$a2],@x[$a2],@x[$b2])",+	   "&vpaddd	(@x[$a3],@x[$a3],@x[$b3])",+	"&vpxord	(@x[$d0],@x[$d0],@x[$a0])",+	 "&vpxord	(@x[$d1],@x[$d1],@x[$a1])",+	  "&vpxord	(@x[$d2],@x[$d2],@x[$a2])",+	   "&vpxord	(@x[$d3],@x[$d3],@x[$a3])",+	"&vprold	(@x[$d0],@x[$d0],8)",+	 "&vprold	(@x[$d1],@x[$d1],8)",+	  "&vprold	(@x[$d2],@x[$d2],8)",+	   "&vprold	(@x[$d3],@x[$d3],8)",++	"&vpaddd	(@x[$c0],@x[$c0],@x[$d0])",+	 "&vpaddd	(@x[$c1],@x[$c1],@x[$d1])",+	  "&vpaddd	(@x[$c2],@x[$c2],@x[$d2])",+	   "&vpaddd	(@x[$c3],@x[$c3],@x[$d3])",+	"&vpxord	(@x[$b0],@x[$b0],@x[$c0])",+	 "&vpxord	(@x[$b1],@x[$b1],@x[$c1])",+	  "&vpxord	(@x[$b2],@x[$b2],@x[$c2])",+	   "&vpxord	(@x[$b3],@x[$b3],@x[$c3])",+	"&vprold	(@x[$b0],@x[$b0],7)",+	 "&vprold	(@x[$b1],@x[$b1],7)",+	  "&vprold	(@x[$b2],@x[$b2],7)",+	   "&vprold	(@x[$b3],@x[$b3],7)"+	);+}++my $xframe = $win64 ? 0xa8 : 8;++$code.=<<___;+.type	ChaCha20_16x,\@function,5+.align	32+ChaCha20_16x:+.cfi_startproc+.LChaCha20_16x:+	mov		%rsp,%r10		# frame register+.cfi_def_cfa_register	%r10+	sub		\$64+$xframe,%rsp+	and		\$-64,%rsp+___+$code.=<<___	if ($win64);+	movaps		%xmm6,-0xa8(%r10)+	movaps		%xmm7,-0x98(%r10)+	movaps		%xmm8,-0x88(%r10)+	movaps		%xmm9,-0x78(%r10)+	movaps		%xmm10,-0x68(%r10)+	movaps		%xmm11,-0x58(%r10)+	movaps		%xmm12,-0x48(%r10)+	movaps		%xmm13,-0x38(%r10)+	movaps		%xmm14,-0x28(%r10)+	movaps		%xmm15,-0x18(%r10)+.L16x_body:+___+$code.=<<___;+	vzeroupper++	lea		.Lsigma(%rip),%r9+	vbroadcasti32x4	(%r9),$xa3		# key[0]+	vbroadcasti32x4	($key),$xb3		# key[1]+	vbroadcasti32x4	16($key),$xc3		# key[2]+	vbroadcasti32x4	($counter),$xd3		# key[3]++	vpshufd		\$0x00,$xa3,$xa0	# smash key by lanes...+	vpshufd		\$0x55,$xa3,$xa1+	vpshufd		\$0xaa,$xa3,$xa2+	vpshufd		\$0xff,$xa3,$xa3+	vmovdqa64	$xa0,@key[0]+	vmovdqa64	$xa1,@key[1]+	vmovdqa64	$xa2,@key[2]+	vmovdqa64	$xa3,@key[3]++	vpshufd		\$0x00,$xb3,$xb0+	vpshufd		\$0x55,$xb3,$xb1+	vpshufd		\$0xaa,$xb3,$xb2+	vpshufd		\$0xff,$xb3,$xb3+	vmovdqa64	$xb0,@key[4]+	vmovdqa64	$xb1,@key[5]+	vmovdqa64	$xb2,@key[6]+	vmovdqa64	$xb3,@key[7]++	vpshufd		\$0x00,$xc3,$xc0+	vpshufd		\$0x55,$xc3,$xc1+	vpshufd		\$0xaa,$xc3,$xc2+	vpshufd		\$0xff,$xc3,$xc3+	vmovdqa64	$xc0,@key[8]+	vmovdqa64	$xc1,@key[9]+	vmovdqa64	$xc2,@key[10]+	vmovdqa64	$xc3,@key[11]++	vpshufd		\$0x00,$xd3,$xd0+	vpshufd		\$0x55,$xd3,$xd1+	vpshufd		\$0xaa,$xd3,$xd2+	vpshufd		\$0xff,$xd3,$xd3+	vpaddd		.Lincz(%rip),$xd0,$xd0	# don't save counters yet+	vmovdqa64	$xd0,@key[12]+	vmovdqa64	$xd1,@key[13]+	vmovdqa64	$xd2,@key[14]+	vmovdqa64	$xd3,@key[15]++	mov		\$10,%eax+	jmp		.Loop16x++.align	32+.Loop_outer16x:+	vpbroadcastd	0(%r9),$xa0		# reload key+	vpbroadcastd	4(%r9),$xa1+	vpbroadcastd	8(%r9),$xa2+	vpbroadcastd	12(%r9),$xa3+	vpaddd		.Lsixteen(%rip),@key[12],@key[12]	# next SIMD counters+	vmovdqa64	@key[4],$xb0+	vmovdqa64	@key[5],$xb1+	vmovdqa64	@key[6],$xb2+	vmovdqa64	@key[7],$xb3+	vmovdqa64	@key[8],$xc0+	vmovdqa64	@key[9],$xc1+	vmovdqa64	@key[10],$xc2+	vmovdqa64	@key[11],$xc3+	vmovdqa64	@key[12],$xd0+	vmovdqa64	@key[13],$xd1+	vmovdqa64	@key[14],$xd2+	vmovdqa64	@key[15],$xd3++	vmovdqa64	$xa0,@key[0]+	vmovdqa64	$xa1,@key[1]+	vmovdqa64	$xa2,@key[2]+	vmovdqa64	$xa3,@key[3]++	mov		\$10,%eax+	jmp		.Loop16x++.align	32+.Loop16x:+___+	foreach (&AVX512_lane_ROUND(0, 4, 8,12)) { eval; }+	foreach (&AVX512_lane_ROUND(0, 5,10,15)) { eval; }+$code.=<<___;+	dec		%eax+	jnz		.Loop16x++	vpaddd		@key[0],$xa0,$xa0	# accumulate key+	vpaddd		@key[1],$xa1,$xa1+	vpaddd		@key[2],$xa2,$xa2+	vpaddd		@key[3],$xa3,$xa3++	vpunpckldq	$xa1,$xa0,$xt2		# "de-interlace" data+	vpunpckldq	$xa3,$xa2,$xt3+	vpunpckhdq	$xa1,$xa0,$xa0+	vpunpckhdq	$xa3,$xa2,$xa2+	vpunpcklqdq	$xt3,$xt2,$xa1		# "a0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "a1"+	vpunpcklqdq	$xa2,$xa0,$xa3		# "a2"+	vpunpckhqdq	$xa2,$xa0,$xa0		# "a3"+___+	($xa0,$xa1,$xa2,$xa3,$xt2)=($xa1,$xt2,$xa3,$xa0,$xa2);+$code.=<<___;+	vpaddd		@key[4],$xb0,$xb0+	vpaddd		@key[5],$xb1,$xb1+	vpaddd		@key[6],$xb2,$xb2+	vpaddd		@key[7],$xb3,$xb3++	vpunpckldq	$xb1,$xb0,$xt2+	vpunpckldq	$xb3,$xb2,$xt3+	vpunpckhdq	$xb1,$xb0,$xb0+	vpunpckhdq	$xb3,$xb2,$xb2+	vpunpcklqdq	$xt3,$xt2,$xb1		# "b0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "b1"+	vpunpcklqdq	$xb2,$xb0,$xb3		# "b2"+	vpunpckhqdq	$xb2,$xb0,$xb0		# "b3"+___+	($xb0,$xb1,$xb2,$xb3,$xt2)=($xb1,$xt2,$xb3,$xb0,$xb2);+$code.=<<___;+	vshufi32x4	\$0x44,$xb0,$xa0,$xt3	# "de-interlace" further+	vshufi32x4	\$0xee,$xb0,$xa0,$xb0+	vshufi32x4	\$0x44,$xb1,$xa1,$xa0+	vshufi32x4	\$0xee,$xb1,$xa1,$xb1+	vshufi32x4	\$0x44,$xb2,$xa2,$xa1+	vshufi32x4	\$0xee,$xb2,$xa2,$xb2+	vshufi32x4	\$0x44,$xb3,$xa3,$xa2+	vshufi32x4	\$0xee,$xb3,$xa3,$xb3+___+	($xa0,$xa1,$xa2,$xa3,$xt3)=($xt3,$xa0,$xa1,$xa2,$xa3);+$code.=<<___;+	vpaddd		@key[8],$xc0,$xc0+	vpaddd		@key[9],$xc1,$xc1+	vpaddd		@key[10],$xc2,$xc2+	vpaddd		@key[11],$xc3,$xc3++	vpunpckldq	$xc1,$xc0,$xt2+	vpunpckldq	$xc3,$xc2,$xt3+	vpunpckhdq	$xc1,$xc0,$xc0+	vpunpckhdq	$xc3,$xc2,$xc2+	vpunpcklqdq	$xt3,$xt2,$xc1		# "c0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "c1"+	vpunpcklqdq	$xc2,$xc0,$xc3		# "c2"+	vpunpckhqdq	$xc2,$xc0,$xc0		# "c3"+___+	($xc0,$xc1,$xc2,$xc3,$xt2)=($xc1,$xt2,$xc3,$xc0,$xc2);+$code.=<<___;+	vpaddd		@key[12],$xd0,$xd0+	vpaddd		@key[13],$xd1,$xd1+	vpaddd		@key[14],$xd2,$xd2+	vpaddd		@key[15],$xd3,$xd3++	vpunpckldq	$xd1,$xd0,$xt2+	vpunpckldq	$xd3,$xd2,$xt3+	vpunpckhdq	$xd1,$xd0,$xd0+	vpunpckhdq	$xd3,$xd2,$xd2+	vpunpcklqdq	$xt3,$xt2,$xd1		# "d0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "d1"+	vpunpcklqdq	$xd2,$xd0,$xd3		# "d2"+	vpunpckhqdq	$xd2,$xd0,$xd0		# "d3"+___+	($xd0,$xd1,$xd2,$xd3,$xt2)=($xd1,$xt2,$xd3,$xd0,$xd2);+$code.=<<___;+	vshufi32x4	\$0x44,$xd0,$xc0,$xt3	# "de-interlace" further+	vshufi32x4	\$0xee,$xd0,$xc0,$xd0+	vshufi32x4	\$0x44,$xd1,$xc1,$xc0+	vshufi32x4	\$0xee,$xd1,$xc1,$xd1+	vshufi32x4	\$0x44,$xd2,$xc2,$xc1+	vshufi32x4	\$0xee,$xd2,$xc2,$xd2+	vshufi32x4	\$0x44,$xd3,$xc3,$xc2+	vshufi32x4	\$0xee,$xd3,$xc3,$xd3+___+	($xc0,$xc1,$xc2,$xc3,$xt3)=($xt3,$xc0,$xc1,$xc2,$xc3);+$code.=<<___;+	vshufi32x4	\$0x88,$xc0,$xa0,$xt0	# "de-interlace" further+	vshufi32x4	\$0xdd,$xc0,$xa0,$xa0+	 vshufi32x4	\$0x88,$xd0,$xb0,$xc0+	 vshufi32x4	\$0xdd,$xd0,$xb0,$xd0+	vshufi32x4	\$0x88,$xc1,$xa1,$xt1+	vshufi32x4	\$0xdd,$xc1,$xa1,$xa1+	 vshufi32x4	\$0x88,$xd1,$xb1,$xc1+	 vshufi32x4	\$0xdd,$xd1,$xb1,$xd1+	vshufi32x4	\$0x88,$xc2,$xa2,$xt2+	vshufi32x4	\$0xdd,$xc2,$xa2,$xa2+	 vshufi32x4	\$0x88,$xd2,$xb2,$xc2+	 vshufi32x4	\$0xdd,$xd2,$xb2,$xd2+	vshufi32x4	\$0x88,$xc3,$xa3,$xt3+	vshufi32x4	\$0xdd,$xc3,$xa3,$xa3+	 vshufi32x4	\$0x88,$xd3,$xb3,$xc3+	 vshufi32x4	\$0xdd,$xd3,$xb3,$xd3+___+	($xa0,$xa1,$xa2,$xa3,$xb0,$xb1,$xb2,$xb3)=+	($xt0,$xt1,$xt2,$xt3,$xa0,$xa1,$xa2,$xa3);++	($xa0,$xb0,$xc0,$xd0, $xa1,$xb1,$xc1,$xd1,+	 $xa2,$xb2,$xc2,$xd2, $xa3,$xb3,$xc3,$xd3) =+	($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3,+	 $xc0,$xc1,$xc2,$xc3, $xd0,$xd1,$xd2,$xd3);+$code.=<<___;+	cmp		\$64*16,$len+	jb		.Ltail16x++	vpxord		0x00($inp),$xa0,$xa0	# xor with input+	vpxord		0x40($inp),$xb0,$xb0+	vpxord		0x80($inp),$xc0,$xc0+	vpxord		0xc0($inp),$xd0,$xd0+	vmovdqu32	$xa0,0x00($out)+	vmovdqu32	$xb0,0x40($out)+	vmovdqu32	$xc0,0x80($out)+	vmovdqu32	$xd0,0xc0($out)++	vpxord		0x100($inp),$xa1,$xa1+	vpxord		0x140($inp),$xb1,$xb1+	vpxord		0x180($inp),$xc1,$xc1+	vpxord		0x1c0($inp),$xd1,$xd1+	vmovdqu32	$xa1,0x100($out)+	vmovdqu32	$xb1,0x140($out)+	vmovdqu32	$xc1,0x180($out)+	vmovdqu32	$xd1,0x1c0($out)++	vpxord		0x200($inp),$xa2,$xa2+	vpxord		0x240($inp),$xb2,$xb2+	vpxord		0x280($inp),$xc2,$xc2+	vpxord		0x2c0($inp),$xd2,$xd2+	vmovdqu32	$xa2,0x200($out)+	vmovdqu32	$xb2,0x240($out)+	vmovdqu32	$xc2,0x280($out)+	vmovdqu32	$xd2,0x2c0($out)++	vpxord		0x300($inp),$xa3,$xa3+	vpxord		0x340($inp),$xb3,$xb3+	vpxord		0x380($inp),$xc3,$xc3+	vpxord		0x3c0($inp),$xd3,$xd3+	lea		0x400($inp),$inp+	vmovdqu32	$xa3,0x300($out)+	vmovdqu32	$xb3,0x340($out)+	vmovdqu32	$xc3,0x380($out)+	vmovdqu32	$xd3,0x3c0($out)+	lea		0x400($out),$out++	sub		\$64*16,$len+	jnz		.Loop_outer16x++	jmp		.Ldone16x++.align	32+.Ltail16x:+	xor		%r9,%r9+	sub		$inp,$out+	cmp		\$64*1,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xa0,$xa0	# xor with input+	vmovdqu32	$xa0,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xb0,$xa0+	lea		64($inp),$inp++	cmp		\$64*2,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xb0,$xb0+	vmovdqu32	$xb0,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xc0,$xa0+	lea		64($inp),$inp++	cmp		\$64*3,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xc0,$xc0+	vmovdqu32	$xc0,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xd0,$xa0+	lea		64($inp),$inp++	cmp		\$64*4,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xd0,$xd0+	vmovdqu32	$xd0,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xa1,$xa0+	lea		64($inp),$inp++	cmp		\$64*5,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xa1,$xa1+	vmovdqu32	$xa1,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xb1,$xa0+	lea		64($inp),$inp++	cmp		\$64*6,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xb1,$xb1+	vmovdqu32	$xb1,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xc1,$xa0+	lea		64($inp),$inp++	cmp		\$64*7,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xc1,$xc1+	vmovdqu32	$xc1,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xd1,$xa0+	lea		64($inp),$inp++	cmp		\$64*8,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xd1,$xd1+	vmovdqu32	$xd1,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xa2,$xa0+	lea		64($inp),$inp++	cmp		\$64*9,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xa2,$xa2+	vmovdqu32	$xa2,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xb2,$xa0+	lea		64($inp),$inp++	cmp		\$64*10,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xb2,$xb2+	vmovdqu32	$xb2,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xc2,$xa0+	lea		64($inp),$inp++	cmp		\$64*11,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xc2,$xc2+	vmovdqu32	$xc2,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xd2,$xa0+	lea		64($inp),$inp++	cmp		\$64*12,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xd2,$xd2+	vmovdqu32	$xd2,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xa3,$xa0+	lea		64($inp),$inp++	cmp		\$64*13,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xa3,$xa3+	vmovdqu32	$xa3,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xb3,$xa0+	lea		64($inp),$inp++	cmp		\$64*14,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xb3,$xb3+	vmovdqu32	$xb3,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xc3,$xa0+	lea		64($inp),$inp++	cmp		\$64*15,$len+	jb		.Less_than_64_16x+	vpxord		($inp),$xc3,$xc3+	vmovdqu32	$xc3,($out,$inp)+	je		.Ldone16x+	vmovdqa32	$xd3,$xa0+	lea		64($inp),$inp++.Less_than_64_16x:+	vmovdqa32	$xa0,0x00(%rsp)+	lea		($out,$inp),$out+	and		\$63,$len++.Loop_tail16x:+	movzb		($inp,%r9),%eax+	movzb		(%rsp,%r9),%ecx+	lea		1(%r9),%r9+	xor		%ecx,%eax+	mov		%al,-1($out,%r9)+	dec		$len+	jnz		.Loop_tail16x++	vpxord		$xa0,$xa0,$xa0+	vmovdqa32	$xa0,0(%rsp)++.Ldone16x:+	vzeroall+___+$code.=<<___	if ($win64);+	movaps		-0xa8(%r10),%xmm6+	movaps		-0x98(%r10),%xmm7+	movaps		-0x88(%r10),%xmm8+	movaps		-0x78(%r10),%xmm9+	movaps		-0x68(%r10),%xmm10+	movaps		-0x58(%r10),%xmm11+	movaps		-0x48(%r10),%xmm12+	movaps		-0x38(%r10),%xmm13+	movaps		-0x28(%r10),%xmm14+	movaps		-0x18(%r10),%xmm15+___+$code.=<<___;+	lea		(%r10),%rsp+.cfi_def_cfa_register	%rsp+.L16x_epilogue:+	ret+.cfi_endproc+.size	ChaCha20_16x,.-ChaCha20_16x+___++# switch to %ymm domain+($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3,+ $xc0,$xc1,$xc2,$xc3, $xd0,$xd1,$xd2,$xd3)=map("%ymm$_",(0..15));+@xx=($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3,+     $xc0,$xc1,$xc2,$xc3, $xd0,$xd1,$xd2,$xd3);+@key=map("%ymm$_",(16..31));+($xt0,$xt1,$xt2,$xt3)=@key[0..3];++$code.=<<___;+.type	ChaCha20_8xvl,\@function,5+.align	32+ChaCha20_8xvl:+.cfi_startproc+.LChaCha20_8xvl:+	mov		%rsp,%r10		# frame register+.cfi_def_cfa_register	%r10+	sub		\$64+$xframe,%rsp+	and		\$-64,%rsp+___+$code.=<<___	if ($win64);+	movaps		%xmm6,-0xa8(%r10)+	movaps		%xmm7,-0x98(%r10)+	movaps		%xmm8,-0x88(%r10)+	movaps		%xmm9,-0x78(%r10)+	movaps		%xmm10,-0x68(%r10)+	movaps		%xmm11,-0x58(%r10)+	movaps		%xmm12,-0x48(%r10)+	movaps		%xmm13,-0x38(%r10)+	movaps		%xmm14,-0x28(%r10)+	movaps		%xmm15,-0x18(%r10)+.L8xvl_body:+___+$code.=<<___;+	vzeroupper++	lea		.Lsigma(%rip),%r9+	vbroadcasti128	(%r9),$xa3		# key[0]+	vbroadcasti128	($key),$xb3		# key[1]+	vbroadcasti128	16($key),$xc3		# key[2]+	vbroadcasti128	($counter),$xd3		# key[3]++	vpshufd		\$0x00,$xa3,$xa0	# smash key by lanes...+	vpshufd		\$0x55,$xa3,$xa1+	vpshufd		\$0xaa,$xa3,$xa2+	vpshufd		\$0xff,$xa3,$xa3+	vmovdqa64	$xa0,@key[0]+	vmovdqa64	$xa1,@key[1]+	vmovdqa64	$xa2,@key[2]+	vmovdqa64	$xa3,@key[3]++	vpshufd		\$0x00,$xb3,$xb0+	vpshufd		\$0x55,$xb3,$xb1+	vpshufd		\$0xaa,$xb3,$xb2+	vpshufd		\$0xff,$xb3,$xb3+	vmovdqa64	$xb0,@key[4]+	vmovdqa64	$xb1,@key[5]+	vmovdqa64	$xb2,@key[6]+	vmovdqa64	$xb3,@key[7]++	vpshufd		\$0x00,$xc3,$xc0+	vpshufd		\$0x55,$xc3,$xc1+	vpshufd		\$0xaa,$xc3,$xc2+	vpshufd		\$0xff,$xc3,$xc3+	vmovdqa64	$xc0,@key[8]+	vmovdqa64	$xc1,@key[9]+	vmovdqa64	$xc2,@key[10]+	vmovdqa64	$xc3,@key[11]++	vpshufd		\$0x00,$xd3,$xd0+	vpshufd		\$0x55,$xd3,$xd1+	vpshufd		\$0xaa,$xd3,$xd2+	vpshufd		\$0xff,$xd3,$xd3+	vpaddd		.Lincy(%rip),$xd0,$xd0	# don't save counters yet+	vmovdqa64	$xd0,@key[12]+	vmovdqa64	$xd1,@key[13]+	vmovdqa64	$xd2,@key[14]+	vmovdqa64	$xd3,@key[15]++	mov		\$10,%eax+	jmp		.Loop8xvl++.align	32+.Loop_outer8xvl:+	#vpbroadcastd	0(%r9),$xa0		# reload key+	#vpbroadcastd	4(%r9),$xa1+	vpbroadcastd	8(%r9),$xa2+	vpbroadcastd	12(%r9),$xa3+	vpaddd		.Leight(%rip),@key[12],@key[12]	# next SIMD counters+	vmovdqa64	@key[4],$xb0+	vmovdqa64	@key[5],$xb1+	vmovdqa64	@key[6],$xb2+	vmovdqa64	@key[7],$xb3+	vmovdqa64	@key[8],$xc0+	vmovdqa64	@key[9],$xc1+	vmovdqa64	@key[10],$xc2+	vmovdqa64	@key[11],$xc3+	vmovdqa64	@key[12],$xd0+	vmovdqa64	@key[13],$xd1+	vmovdqa64	@key[14],$xd2+	vmovdqa64	@key[15],$xd3++	vmovdqa64	$xa0,@key[0]+	vmovdqa64	$xa1,@key[1]+	vmovdqa64	$xa2,@key[2]+	vmovdqa64	$xa3,@key[3]++	mov		\$10,%eax+	jmp		.Loop8xvl++.align	32+.Loop8xvl:+___+	foreach (&AVX512_lane_ROUND(0, 4, 8,12)) { eval; }+	foreach (&AVX512_lane_ROUND(0, 5,10,15)) { eval; }+$code.=<<___;+	dec		%eax+	jnz		.Loop8xvl++	vpaddd		@key[0],$xa0,$xa0	# accumulate key+	vpaddd		@key[1],$xa1,$xa1+	vpaddd		@key[2],$xa2,$xa2+	vpaddd		@key[3],$xa3,$xa3++	vpunpckldq	$xa1,$xa0,$xt2		# "de-interlace" data+	vpunpckldq	$xa3,$xa2,$xt3+	vpunpckhdq	$xa1,$xa0,$xa0+	vpunpckhdq	$xa3,$xa2,$xa2+	vpunpcklqdq	$xt3,$xt2,$xa1		# "a0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "a1"+	vpunpcklqdq	$xa2,$xa0,$xa3		# "a2"+	vpunpckhqdq	$xa2,$xa0,$xa0		# "a3"+___+	($xa0,$xa1,$xa2,$xa3,$xt2)=($xa1,$xt2,$xa3,$xa0,$xa2);+$code.=<<___;+	vpaddd		@key[4],$xb0,$xb0+	vpaddd		@key[5],$xb1,$xb1+	vpaddd		@key[6],$xb2,$xb2+	vpaddd		@key[7],$xb3,$xb3++	vpunpckldq	$xb1,$xb0,$xt2+	vpunpckldq	$xb3,$xb2,$xt3+	vpunpckhdq	$xb1,$xb0,$xb0+	vpunpckhdq	$xb3,$xb2,$xb2+	vpunpcklqdq	$xt3,$xt2,$xb1		# "b0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "b1"+	vpunpcklqdq	$xb2,$xb0,$xb3		# "b2"+	vpunpckhqdq	$xb2,$xb0,$xb0		# "b3"+___+	($xb0,$xb1,$xb2,$xb3,$xt2)=($xb1,$xt2,$xb3,$xb0,$xb2);+$code.=<<___;+	vshufi32x4	\$0,$xb0,$xa0,$xt3	# "de-interlace" further+	vshufi32x4	\$3,$xb0,$xa0,$xb0+	vshufi32x4	\$0,$xb1,$xa1,$xa0+	vshufi32x4	\$3,$xb1,$xa1,$xb1+	vshufi32x4	\$0,$xb2,$xa2,$xa1+	vshufi32x4	\$3,$xb2,$xa2,$xb2+	vshufi32x4	\$0,$xb3,$xa3,$xa2+	vshufi32x4	\$3,$xb3,$xa3,$xb3+___+	($xa0,$xa1,$xa2,$xa3,$xt3)=($xt3,$xa0,$xa1,$xa2,$xa3);+$code.=<<___;+	vpaddd		@key[8],$xc0,$xc0+	vpaddd		@key[9],$xc1,$xc1+	vpaddd		@key[10],$xc2,$xc2+	vpaddd		@key[11],$xc3,$xc3++	vpunpckldq	$xc1,$xc0,$xt2+	vpunpckldq	$xc3,$xc2,$xt3+	vpunpckhdq	$xc1,$xc0,$xc0+	vpunpckhdq	$xc3,$xc2,$xc2+	vpunpcklqdq	$xt3,$xt2,$xc1		# "c0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "c1"+	vpunpcklqdq	$xc2,$xc0,$xc3		# "c2"+	vpunpckhqdq	$xc2,$xc0,$xc0		# "c3"+___+	($xc0,$xc1,$xc2,$xc3,$xt2)=($xc1,$xt2,$xc3,$xc0,$xc2);+$code.=<<___;+	vpaddd		@key[12],$xd0,$xd0+	vpaddd		@key[13],$xd1,$xd1+	vpaddd		@key[14],$xd2,$xd2+	vpaddd		@key[15],$xd3,$xd3++	vpunpckldq	$xd1,$xd0,$xt2+	vpunpckldq	$xd3,$xd2,$xt3+	vpunpckhdq	$xd1,$xd0,$xd0+	vpunpckhdq	$xd3,$xd2,$xd2+	vpunpcklqdq	$xt3,$xt2,$xd1		# "d0"+	vpunpckhqdq	$xt3,$xt2,$xt2		# "d1"+	vpunpcklqdq	$xd2,$xd0,$xd3		# "d2"+	vpunpckhqdq	$xd2,$xd0,$xd0		# "d3"+___+	($xd0,$xd1,$xd2,$xd3,$xt2)=($xd1,$xt2,$xd3,$xd0,$xd2);+$code.=<<___;+	vperm2i128	\$0x20,$xd0,$xc0,$xt3	# "de-interlace" further+	vperm2i128	\$0x31,$xd0,$xc0,$xd0+	vperm2i128	\$0x20,$xd1,$xc1,$xc0+	vperm2i128	\$0x31,$xd1,$xc1,$xd1+	vperm2i128	\$0x20,$xd2,$xc2,$xc1+	vperm2i128	\$0x31,$xd2,$xc2,$xd2+	vperm2i128	\$0x20,$xd3,$xc3,$xc2+	vperm2i128	\$0x31,$xd3,$xc3,$xd3+___+	($xc0,$xc1,$xc2,$xc3,$xt3)=($xt3,$xc0,$xc1,$xc2,$xc3);+	($xb0,$xb1,$xb2,$xb3,$xc0,$xc1,$xc2,$xc3)=+	($xc0,$xc1,$xc2,$xc3,$xb0,$xb1,$xb2,$xb3);+$code.=<<___;+	cmp		\$64*8,$len+	jb		.Ltail8xvl++	mov		\$0x80,%eax		# size optimization+	vpxord		0x00($inp),$xa0,$xa0	# xor with input+	vpxor		0x20($inp),$xb0,$xb0+	vpxor		0x40($inp),$xc0,$xc0+	vpxor		0x60($inp),$xd0,$xd0+	lea		($inp,%rax),$inp	# size optimization+	vmovdqu32	$xa0,0x00($out)+	vmovdqu		$xb0,0x20($out)+	vmovdqu		$xc0,0x40($out)+	vmovdqu		$xd0,0x60($out)+	lea		($out,%rax),$out	# size optimization++	vpxor		0x00($inp),$xa1,$xa1+	vpxor		0x20($inp),$xb1,$xb1+	vpxor		0x40($inp),$xc1,$xc1+	vpxor		0x60($inp),$xd1,$xd1+	lea		($inp,%rax),$inp	# size optimization+	vmovdqu		$xa1,0x00($out)+	vmovdqu		$xb1,0x20($out)+	vmovdqu		$xc1,0x40($out)+	vmovdqu		$xd1,0x60($out)+	lea		($out,%rax),$out	# size optimization++	vpxord		0x00($inp),$xa2,$xa2+	vpxor		0x20($inp),$xb2,$xb2+	vpxor		0x40($inp),$xc2,$xc2+	vpxor		0x60($inp),$xd2,$xd2+	lea		($inp,%rax),$inp	# size optimization+	vmovdqu32	$xa2,0x00($out)+	vmovdqu		$xb2,0x20($out)+	vmovdqu		$xc2,0x40($out)+	vmovdqu		$xd2,0x60($out)+	lea		($out,%rax),$out	# size optimization++	vpxor		0x00($inp),$xa3,$xa3+	vpxor		0x20($inp),$xb3,$xb3+	vpxor		0x40($inp),$xc3,$xc3+	vpxor		0x60($inp),$xd3,$xd3+	lea		($inp,%rax),$inp	# size optimization+	vmovdqu		$xa3,0x00($out)+	vmovdqu		$xb3,0x20($out)+	vmovdqu		$xc3,0x40($out)+	vmovdqu		$xd3,0x60($out)+	lea		($out,%rax),$out	# size optimization++	vpbroadcastd	0(%r9),%ymm0		# reload key+	vpbroadcastd	4(%r9),%ymm1++	sub		\$64*8,$len+	jnz		.Loop_outer8xvl++	jmp		.Ldone8xvl++.align	32+.Ltail8xvl:+	vmovdqa64	$xa0,%ymm8		# size optimization+___+$xa0 = "%ymm8";+$code.=<<___;+	xor		%r9,%r9+	sub		$inp,$out+	cmp		\$64*1,$len+	jb		.Less_than_64_8xvl+	vpxor		0x00($inp),$xa0,$xa0	# xor with input+	vpxor		0x20($inp),$xb0,$xb0+	vmovdqu		$xa0,0x00($out,$inp)+	vmovdqu		$xb0,0x20($out,$inp)+	je		.Ldone8xvl+	vmovdqa		$xc0,$xa0+	vmovdqa		$xd0,$xb0+	lea		64($inp),$inp++	cmp		\$64*2,$len+	jb		.Less_than_64_8xvl+	vpxor		0x00($inp),$xc0,$xc0+	vpxor		0x20($inp),$xd0,$xd0+	vmovdqu		$xc0,0x00($out,$inp)+	vmovdqu		$xd0,0x20($out,$inp)+	je		.Ldone8xvl+	vmovdqa		$xa1,$xa0+	vmovdqa		$xb1,$xb0+	lea		64($inp),$inp++	cmp		\$64*3,$len+	jb		.Less_than_64_8xvl+	vpxor		0x00($inp),$xa1,$xa1+	vpxor		0x20($inp),$xb1,$xb1+	vmovdqu		$xa1,0x00($out,$inp)+	vmovdqu		$xb1,0x20($out,$inp)+	je		.Ldone8xvl+	vmovdqa		$xc1,$xa0+	vmovdqa		$xd1,$xb0+	lea		64($inp),$inp++	cmp		\$64*4,$len+	jb		.Less_than_64_8xvl+	vpxor		0x00($inp),$xc1,$xc1+	vpxor		0x20($inp),$xd1,$xd1+	vmovdqu		$xc1,0x00($out,$inp)+	vmovdqu		$xd1,0x20($out,$inp)+	je		.Ldone8xvl+	vmovdqa32	$xa2,$xa0+	vmovdqa		$xb2,$xb0+	lea		64($inp),$inp++	cmp		\$64*5,$len+	jb		.Less_than_64_8xvl+	vpxord		0x00($inp),$xa2,$xa2+	vpxor		0x20($inp),$xb2,$xb2+	vmovdqu32	$xa2,0x00($out,$inp)+	vmovdqu		$xb2,0x20($out,$inp)+	je		.Ldone8xvl+	vmovdqa		$xc2,$xa0+	vmovdqa		$xd2,$xb0+	lea		64($inp),$inp++	cmp		\$64*6,$len+	jb		.Less_than_64_8xvl+	vpxor		0x00($inp),$xc2,$xc2+	vpxor		0x20($inp),$xd2,$xd2+	vmovdqu		$xc2,0x00($out,$inp)+	vmovdqu		$xd2,0x20($out,$inp)+	je		.Ldone8xvl+	vmovdqa		$xa3,$xa0+	vmovdqa		$xb3,$xb0+	lea		64($inp),$inp++	cmp		\$64*7,$len+	jb		.Less_than_64_8xvl+	vpxor		0x00($inp),$xa3,$xa3+	vpxor		0x20($inp),$xb3,$xb3+	vmovdqu		$xa3,0x00($out,$inp)+	vmovdqu		$xb3,0x20($out,$inp)+	je		.Ldone8xvl+	vmovdqa		$xc3,$xa0+	vmovdqa		$xd3,$xb0+	lea		64($inp),$inp++.Less_than_64_8xvl:+	vmovdqa		$xa0,0x00(%rsp)+	vmovdqa		$xb0,0x20(%rsp)+	lea		($out,$inp),$out+	and		\$63,$len++.Loop_tail8xvl:+	movzb		($inp,%r9),%eax+	movzb		(%rsp,%r9),%ecx+	lea		1(%r9),%r9+	xor		%ecx,%eax+	mov		%al,-1($out,%r9)+	dec		$len+	jnz		.Loop_tail8xvl++	vpxor		$xa0,$xa0,$xa0+	vmovdqa		$xa0,0x00(%rsp)+	vmovdqa		$xa0,0x20(%rsp)++.Ldone8xvl:+	vzeroall+___+$code.=<<___	if ($win64);+	movaps		-0xa8(%r10),%xmm6+	movaps		-0x98(%r10),%xmm7+	movaps		-0x88(%r10),%xmm8+	movaps		-0x78(%r10),%xmm9+	movaps		-0x68(%r10),%xmm10+	movaps		-0x58(%r10),%xmm11+	movaps		-0x48(%r10),%xmm12+	movaps		-0x38(%r10),%xmm13+	movaps		-0x28(%r10),%xmm14+	movaps		-0x18(%r10),%xmm15+___+$code.=<<___;+	lea		(%r10),%rsp+.cfi_def_cfa_register	%rsp+.L8xvl_epilogue:+	ret+.cfi_endproc+.size	ChaCha20_8xvl,.-ChaCha20_8xvl+___+}++# EXCEPTION_DISPOSITION handler (EXCEPTION_RECORD *rec,ULONG64 frame,+#		CONTEXT *context,DISPATCHER_CONTEXT *disp)+if ($win64) {+$rec="%rcx";+$frame="%rdx";+$context="%r8";+$disp="%r9";++$code.=<<___;+.extern	__imp_RtlVirtualUnwind+.type	se_handler,\@abi-omnipotent+.align	16+se_handler:+	push	%rsi+	push	%rdi+	push	%rbx+	push	%rbp+	push	%r12+	push	%r13+	push	%r14+	push	%r15+	pushfq+	sub	\$64,%rsp++	mov	120($context),%rax	# pull context->Rax+	mov	248($context),%rbx	# pull context->Rip++	mov	8($disp),%rsi		# disp->ImageBase+	mov	56($disp),%r11		# disp->HandlerData++	lea	.Lctr32_body(%rip),%r10+	cmp	%r10,%rbx		# context->Rip<.Lprologue+	jb	.Lcommon_seh_tail++	mov	152($context),%rax	# pull context->Rsp++	lea	.Lno_data(%rip),%r10	# epilogue label+	cmp	%r10,%rbx		# context->Rip>=.Lepilogue+	jae	.Lcommon_seh_tail++	lea	64+24+48(%rax),%rax++	mov	-8(%rax),%rbx+	mov	-16(%rax),%rbp+	mov	-24(%rax),%r12+	mov	-32(%rax),%r13+	mov	-40(%rax),%r14+	mov	-48(%rax),%r15+	mov	%rbx,144($context)	# restore context->Rbx+	mov	%rbp,160($context)	# restore context->Rbp+	mov	%r12,216($context)	# restore context->R12+	mov	%r13,224($context)	# restore context->R13+	mov	%r14,232($context)	# restore context->R14+	mov	%r15,240($context)	# restore context->R14++.Lcommon_seh_tail:+	mov	8(%rax),%rdi+	mov	16(%rax),%rsi+	mov	%rax,152($context)	# restore context->Rsp+	mov	%rsi,168($context)	# restore context->Rsi+	mov	%rdi,176($context)	# restore context->Rdi++	mov	40($disp),%rdi		# disp->ContextRecord+	mov	$context,%rsi		# context+	mov	\$154,%ecx		# sizeof(CONTEXT)+	.long	0xa548f3fc		# cld; rep movsq++	mov	$disp,%rsi+	xor	%rcx,%rcx		# arg1, UNW_FLAG_NHANDLER+	mov	8(%rsi),%rdx		# arg2, disp->ImageBase+	mov	0(%rsi),%r8		# arg3, disp->ControlPc+	mov	16(%rsi),%r9		# arg4, disp->FunctionEntry+	mov	40(%rsi),%r10		# disp->ContextRecord+	lea	56(%rsi),%r11		# &disp->HandlerData+	lea	24(%rsi),%r12		# &disp->EstablisherFrame+	mov	%r10,32(%rsp)		# arg5+	mov	%r11,40(%rsp)		# arg6+	mov	%r12,48(%rsp)		# arg7+	mov	%rcx,56(%rsp)		# arg8, (NULL)+	call	*__imp_RtlVirtualUnwind(%rip)++	mov	\$1,%eax		# ExceptionContinueSearch+	add	\$64,%rsp+	popfq+	pop	%r15+	pop	%r14+	pop	%r13+	pop	%r12+	pop	%rbp+	pop	%rbx+	pop	%rdi+	pop	%rsi+	ret+.size	se_handler,.-se_handler++.type	simd_handler,\@abi-omnipotent+.align	16+simd_handler:+	push	%rsi+	push	%rdi+	push	%rbx+	push	%rbp+	push	%r12+	push	%r13+	push	%r14+	push	%r15+	pushfq+	sub	\$64,%rsp++	mov	120($context),%rax	# pull context->Rax+	mov	248($context),%rbx	# pull context->Rip++	mov	8($disp),%rsi		# disp->ImageBase+	mov	56($disp),%r11		# disp->HandlerData++	mov	0(%r11),%r10d		# HandlerData[0]+	lea	(%rsi,%r10),%r10	# prologue label+	cmp	%r10,%rbx		# context->Rip<prologue label+	jb	.Lcommon_seh_tail++	mov	200($context),%rax	# pull context->R10++	mov	4(%r11),%r10d		# HandlerData[1]+	mov	8(%r11),%ecx		# HandlerData[2]+	lea	(%rsi,%r10),%r10	# epilogue label+	cmp	%r10,%rbx		# context->Rip>=epilogue label+	jae	.Lcommon_seh_tail++	neg	%rcx+	lea	-8(%rax,%rcx),%rsi+	lea	512($context),%rdi	# &context.Xmm6+	neg	%ecx+	shr	\$3,%ecx+	.long	0xa548f3fc		# cld; rep movsq++	jmp	.Lcommon_seh_tail+.size	simd_handler,.-simd_handler++.section	.pdata+.align	4+	.rva	.LSEH_begin_ChaCha20_ctr32+	.rva	.LSEH_end_ChaCha20_ctr32+	.rva	.LSEH_info_ChaCha20_ctr32++	.rva	.LSEH_begin_ChaCha20_ssse3+	.rva	.LSEH_end_ChaCha20_ssse3+	.rva	.LSEH_info_ChaCha20_ssse3++	.rva	.LSEH_begin_ChaCha20_128+	.rva	.LSEH_end_ChaCha20_128+	.rva	.LSEH_info_ChaCha20_128++	.rva	.LSEH_begin_ChaCha20_4x+	.rva	.LSEH_end_ChaCha20_4x+	.rva	.LSEH_info_ChaCha20_4x+___+$code.=<<___ if ($avx);+	.rva	.LSEH_begin_ChaCha20_4xop+	.rva	.LSEH_end_ChaCha20_4xop+	.rva	.LSEH_info_ChaCha20_4xop+___+$code.=<<___ if ($avx>1);+	.rva	.LSEH_begin_ChaCha20_avx2+	.rva	.LSEH_end_ChaCha20_avx2+	.rva	.LSEH_info_ChaCha20_avx2+___+$code.=<<___ if ($avx>2);+	.rva	.LSEH_begin_ChaCha20_avx512+	.rva	.LSEH_end_ChaCha20_avx512+	.rva	.LSEH_info_ChaCha20_avx512++	.rva	.LSEH_begin_ChaCha20_avx512vl+	.rva	.LSEH_end_ChaCha20_avx512vl+	.rva	.LSEH_info_ChaCha20_avx512vl++	.rva	.LSEH_begin_ChaCha20_16x+	.rva	.LSEH_end_ChaCha20_16x+	.rva	.LSEH_info_ChaCha20_16x++	.rva	.LSEH_begin_ChaCha20_8xvl+	.rva	.LSEH_end_ChaCha20_8xvl+	.rva	.LSEH_info_ChaCha20_8xvl+___+$code.=<<___;+.section	.xdata+.align	8+.LSEH_info_ChaCha20_ctr32:+	.byte	9,0,0,0+	.rva	se_handler++.LSEH_info_ChaCha20_ssse3:+	.byte	9,0,0,0+	.rva	simd_handler+	.rva	.Lssse3_body,.Lssse3_epilogue+	.long	0x20,0++.LSEH_info_ChaCha20_128:+	.byte	9,0,0,0+	.rva	simd_handler+	.rva	.L128_body,.L128_epilogue+	.long	0x60,0++.LSEH_info_ChaCha20_4x:+	.byte	9,0,0,0+	.rva	simd_handler+	.rva	.L4x_body,.L4x_epilogue+	.long	0xa0,0+___+$code.=<<___ if ($avx);+.LSEH_info_ChaCha20_4xop:+	.byte	9,0,0,0+	.rva	simd_handler+	.rva	.L4xop_body,.L4xop_epilogue		# HandlerData[]+	.long	0xa0,0+___+$code.=<<___ if ($avx>1);+.LSEH_info_ChaCha20_avx2:+	.byte	9,0,0,0+	.rva	simd_handler+	.rva	.Lavx2_body,.Lavx2_epilogue		# HandlerData[]+	.long	0xa0,0+___+$code.=<<___ if ($avx>2);+.LSEH_info_ChaCha20_avx512:+	.byte	9,0,0,0+	.rva	simd_handler+	.rva	.Lavx512_body,.Lavx512_epilogue		# HandlerData[]+	.long	0x20,0++.LSEH_info_ChaCha20_avx512vl:+	.byte	9,0,0,0+	.rva	simd_handler+	.rva	.Lavx512vl_body,.Lavx512vl_epilogue	# HandlerData[]+	.long	0x20,0++.LSEH_info_ChaCha20_16x:+	.byte	9,0,0,0+	.rva	simd_handler+	.rva	.L16x_body,.L16x_epilogue		# HandlerData[]+	.long	0xa0,0++.LSEH_info_ChaCha20_8xvl:+	.byte	9,0,0,0+	.rva	simd_handler+	.rva	.L8xvl_body,.L8xvl_epilogue		# HandlerData[]+	.long	0xa0,0+___+}++foreach (split("\n",$code)) {+	s/\`([^\`]*)\`/eval $1/ge;++	s/%x#%[yz]/%x/g;	# "down-shift"++	print $_,"\n";+}++close STDOUT;
+ cbits/asm/generate.sh view
@@ -0,0 +1,153 @@+#!/bin/sh+#+# Regenerate the assembly checked in beside this script.+#+# The .pl files come from the CRYPTOGAMS distribution, unmodified:+#+#   https://github.com/dot-asm/cryptogams+#     x86_64/aesni-gcm-x86_64.pl	x86_64/chacha-x86_64.pl+#     x86_64/poly1305-x86_64.pl	x86_64/sha512-x86_64.pl+#     x86_64/keccak1600-x86_64.pl	x86_64/x86_64-xlate.pl+#     arm/chacha-armv8.pl		arm/poly1305-armv8.pl+#     arm/sha1-armv8.pl		arm/sha512-armv8.pl+#     arm/keccak1600-armv8.pl	arm/arm-xlate.pl+#     arm/arm_arch.h+#+# The .pl files are the generator, not the product: each one emits+# assembly for a given "flavour", which is the calling convention and the+# object format together.  The output is checked in so that building+# crypton needs no perl.+#+# Two things are done to the output here.  The entry points are renamed:+# a program that links both crypton and OpenSSL would otherwise have two+# definitions of, say, aesni_gcm_encrypt, and the linker is entitled to+# refuse that.  The same goes for OPENSSL_armcap_P, which the ChaCha+# module reads to find out whether the processor has NEON, and which+# crypton defines for itself in cbits/crypton_chacha.c.  And the ELF+# output of the x86-64 module is given the note that says the code does+# not want an executable stack, which the generator leaves to the caller.+#+# Run this on a GNU/Linux host.  The mingw64 flavour asks the compiler+# what __USER_LABEL_PREFIX__ is for its target, and a compiler for a+# platform that decorates symbols -- Apple's, for one -- answers for+# itself rather than for Windows, which would leave every entry point in+# that file with a leading underscore that nothing looks for.+#+# Usage: cd cbits/asm && ./generate.sh++set -e++# The x86-64 generators choose what to emit from the version of the+# assembler they are told about, so they are told one, rather than left to+# ask whatever compiler happens to be here: the checked-in files should not+# depend on the host that produced them.  2.24 predates AVX-512, which is+# the point -- the Poly1305 module has paths for it, and this does not take+# them, no machine here being able to run them, and a path nothing has+# executed not being worth the few per cent it might be worth.  It leaves+# both modules with everything through AVX2.+cat > tmp-cc <<'SHIM'+#!/bin/sh+case "$*" in+*-Wa,-v*) echo "GNU assembler version 2.24" ;;+esac+exit 0+SHIM+chmod +x tmp-cc+CC=./tmp-cc+export CC++for flavour in elf macosx mingw64; do+	perl aesni-gcm-x86_64.pl $flavour tmp-$flavour.S+	sed -e 's/aesni_gcm_/crypton_gcm_asm_/g' \+	    -e 's/aesni_ctr32_/crypton_gcm_asm_ctr32_/g' \+	    tmp-$flavour.S > aesni-gcm-x86_64-$flavour.S++	perl poly1305-x86_64.pl $flavour tmp-$flavour.S+	sed -e 's/poly1305_/crypton_poly1305_asm_/g' \+	    -e 's/xor128_/crypton_xor128_/g' \+	    -e 's/OPENSSL_ia32cap_P/crypton_ia32cap_P/g' \+	    tmp-$flavour.S > poly1305-x86_64-$flavour.S++	perl chacha-x86_64.pl $flavour tmp-$flavour.S+	sed -e 's/ChaCha20_/crypton_chacha20_asm_/g' \+	    -e 's/OPENSSL_ia32cap_P/crypton_ia32cap_P/g' \+	    tmp-$flavour.S > chacha-x86_64-$flavour.S++	# as on AArch64, this generator emits SHA-512 or SHA-256 according+	# to the name it is given, and both are wanted here+	perl sha512-x86_64.pl $flavour tmp-$flavour.S+	sed -e 's/sha256_block_/crypton_sha256_asm_block_/g' \+	    -e 's/OPENSSL_ia32cap_P/crypton_ia32cap_P/g' \+	    tmp-$flavour.S > sha256-x86_64-$flavour.S++	perl keccak1600-x86_64.pl $flavour tmp-k-$flavour.S+	sed -e 's/SHA3_absorb/crypton_keccak_asm_absorb/g' \+	    -e 's/SHA3_squeeze/crypton_keccak_asm_squeeze/g' \+	    -e 's/KeccakF1600/crypton_keccak_asm_f1600/g' \+	    tmp-k-$flavour.S > keccak1600-x86_64-$flavour.S++	perl sha512-x86_64.pl $flavour tmp-512-$flavour.S+	sed -e 's/sha512_block_/crypton_sha512_asm_block_/g' \+	    -e 's/OPENSSL_ia32cap_P/crypton_ia32cap_P/g' \+	    tmp-512-$flavour.S > sha512-x86_64-$flavour.S+	rm -f tmp-$flavour.S tmp-512-$flavour.S tmp-k-$flavour.S+done++for f in aesni-gcm-x86_64-elf.S poly1305-x86_64-elf.S chacha-x86_64-elf.S \+	 sha256-x86_64-elf.S sha512-x86_64-elf.S keccak1600-x86_64-elf.S; do+	cat >> $f <<-NOTE++	.section	.note.GNU-stack,"",@progbits+	NOTE+done++unset CC+rm -f tmp-cc++for flavour in linux64 ios64; do+	perl chacha-armv8.pl $flavour tmp-$flavour.S+	sed -e 's/ChaCha20_/crypton_chacha20_asm_/g' \+	    -e 's/OPENSSL_armcap_P/crypton_armcap_P/g' \+	    tmp-$flavour.S > chacha-armv8-$flavour.S++	perl poly1305-armv8.pl $flavour tmp-$flavour.S+	sed -e 's/poly1305_/crypton_poly1305_asm_/g' \+	    -e 's/OPENSSL_armcap_P/crypton_armcap_P/g' \+	    tmp-$flavour.S > poly1305-armv8-$flavour.S++	# the same generator emits SHA-512 or SHA-256 according to the name+	# it is given, and only the SHA-256 one is wanted here+	perl sha512-armv8.pl $flavour tmp-$flavour.S+	sed -e 's/sha256_block_/crypton_sha256_asm_block_/g' \+	    -e 's/OPENSSL_armcap_P/crypton_armcap_P/g' \+	    tmp-$flavour.S > sha256-armv8-$flavour.S++	perl sha1-armv8.pl $flavour tmp-$flavour.S+	sed -e 's/sha1_block_/crypton_sha1_asm_block_/g' \+	    -e 's/OPENSSL_armcap_P/crypton_armcap_P/g' \+	    tmp-$flavour.S > sha1-armv8-$flavour.S++	perl keccak1600-armv8.pl $flavour tmp-$flavour.S+	sed -e 's/SHA3_absorb/crypton_keccak_asm_absorb/g' \+	    -e 's/SHA3_squeeze/crypton_keccak_asm_squeeze/g' \+	    tmp-$flavour.S > keccak1600-armv8-$flavour.S+	rm -f tmp-$flavour.S+done++cat >> chacha-armv8-linux64.S <<'NOTE'++.section	.note.GNU-stack,"",%progbits+NOTE++cat >> poly1305-armv8-linux64.S <<'NOTE'++.section	.note.GNU-stack,"",%progbits+NOTE++for f in sha1-armv8-linux64.S sha256-armv8-linux64.S \+	 keccak1600-armv8-linux64.S; do+	cat >> $f <<-NOTE++	.section	.note.GNU-stack,"",%progbits+	NOTE+done
+ cbits/asm/keccak1600-armv8-ios64.S view
@@ -0,0 +1,841 @@+.text++.align	8	// strategic alignment and padding that allows to use+		// address value as loop termination condition...+.quad	0,0,0,0,0,0,0,0++iotas:+.quad	0x0000000000000001+.quad	0x0000000000008082+.quad	0x800000000000808a+.quad	0x8000000080008000+.quad	0x000000000000808b+.quad	0x0000000080000001+.quad	0x8000000080008081+.quad	0x8000000000008009+.quad	0x000000000000008a+.quad	0x0000000000000088+.quad	0x0000000080008009+.quad	0x000000008000000a+Liotas12:+.quad	0x000000008000808b+.quad	0x800000000000008b+.quad	0x8000000000008089+.quad	0x8000000000008003+.quad	0x8000000000008002+.quad	0x8000000000000080+.quad	0x000000000000800a+.quad	0x800000008000000a+.quad	0x8000000080008081+.quad	0x8000000000008080+.quad	0x0000000080000001+.quad	0x8000000080008008+++.align	5+KeccakF1600_int:+.long	0xd503233f			// paciasp+	stp	x28,x30,[sp,#16]		// stack is pre-allocated+	b	Loop+.align	4+Loop:+	////////////////////////////////////////// Theta+	eor	x26,x0,x5+	stp	x4,x9,[sp,#0]	// offload pair...+	eor	x27,x1,x6+	eor	x28,x2,x7+	eor	x30,x3,x8+	eor	x4,x4,x9+	eor	x26,x26,x10+	eor	x27,x27,x11+	eor	x28,x28,x12+	eor	x30,x30,x13+	eor	x4,x4,x14+	eor	x26,x26,x15+	eor	x27,x27,x16+	eor	x28,x28,x17+	eor	x30,x30,x25+	eor	x4,x4,x19+	eor	x26,x26,x20+	eor	x28,x28,x22+	eor	x27,x27,x21+	eor	x30,x30,x23+	eor	x4,x4,x24++	eor	x9,x26,x28,ror#63++	eor	x1,x1,x9+	eor	x6,x6,x9+	eor	x11,x11,x9+	eor	x16,x16,x9+	eor	x21,x21,x9++	eor	x9,x27,x30,ror#63+	eor	x28,x28,x4,ror#63+	eor	x30,x30,x26,ror#63+	eor	x4,x4,x27,ror#63++	eor	x27,   x2,x9		// mov	x27,x2+	eor	x7,x7,x9+	eor	x12,x12,x9+	eor	x17,x17,x9+	eor	x22,x22,x9++	eor	x0,x0,x4+	eor	x5,x5,x4+	eor	x10,x10,x4+	eor	x15,x15,x4+	eor	x20,x20,x4+	ldp	x4,x9,[sp,#0]	// re-load offloaded data+	eor	x26,   x3,x28		// mov	x26,x3+	eor	x8,x8,x28+	eor	x13,x13,x28+	eor	x25,x25,x28+	eor	x23,x23,x28++	eor	x28,   x4,x30		// mov	x28,x4+	eor	x9,x9,x30+	eor	x14,x14,x30+	eor	x19,x19,x30+	eor	x24,x24,x30++	////////////////////////////////////////// Rho+Pi+	mov	x30,x1+	ror	x1,x6,#64-44+	//mov	x27,x2+	ror	x2,x12,#64-43+	//mov	x26,x3+	ror	x3,x25,#64-21	// ?+	//mov	x28,x4+	ror	x4,x24,#64-14	// ?++	ror	x6,x9,#64-20	// ?+	ror	x12,x13,#64-25	// ?+	ror	x25,x17,#64-15+	ror	x24,x21,#64-2	// ?++	ror	x9,x22,#64-61+	ror	x13,x19,#64-8+	ror	x17,x11,#64-10+	ror	x21,x8,#64-55++	ror	x22,x14,#64-39+	ror	x19,x23,#64-56+	ror	x11,x7,#64-6	// ?+	ror	x8,x16,#64-45++	ror	x14,x20,#64-18+	ror	x23,x15,#64-41+	ror	x7,x10,#64-3+	ror	x16,x5,#64-36	// ?++	ror	x5,x26,#64-28	// ?+	ror	x10,x30,#64-1+	ror	x15,x28,#64-27	// ?+	ror	x20,x27,#64-62	// ?++	////////////////////////////////////////// Chi+Iota+	bic	x26,x2,x1+	bic	x27,x3,x2+	bic	x28,x0,x4+	bic	x30,x1,x0+	eor	x0,x0,x26+	bic	x26,x4,x3+	eor	x1,x1,x27+	ldr	x27,[sp,#16]+	eor	x3,x3,x28+	eor	x4,x4,x30+	eor	x2,x2,x26+	ldr	x30,[x27],#8		// Iota[i++]++	bic	x26,x7,x6+	tst	x27,#255			// are we done?+	str	x27,[sp,#16]+	bic	x27,x8,x7+	bic	x28,x5,x9+	eor	x0,x0,x30		// A[0][0] ^= Iota+	bic	x30,x6,x5+	eor	x5,x5,x26+	bic	x26,x9,x8+	eor	x6,x6,x27+	eor	x8,x8,x28+	eor	x9,x9,x30+	eor	x7,x7,x26++	bic	x26,x12,x11+	bic	x27,x13,x12+	bic	x28,x10,x14+	bic	x30,x11,x10+	eor	x10,x10,x26+	bic	x26,x14,x13+	eor	x11,x11,x27+	eor	x13,x13,x28+	eor	x14,x14,x30+	eor	x12,x12,x26++	bic	x26,x17,x16+	bic	x27,x25,x17+	bic	x28,x15,x19+	bic	x30,x16,x15+	eor	x15,x15,x26+	bic	x26,x19,x25+	eor	x16,x16,x27+	eor	x25,x25,x28+	eor	x19,x19,x30+	eor	x17,x17,x26++	bic	x26,x22,x21+	bic	x27,x23,x22+	bic	x28,x20,x24+	bic	x30,x21,x20+	eor	x20,x20,x26+	bic	x26,x24,x23+	eor	x21,x21,x27+	eor	x23,x23,x28+	eor	x24,x24,x30+	eor	x22,x22,x26++	bne	Loop++	ldr	x30,[sp,#16+__SIZEOF_POINTER__]+.long	0xd50323bf			// autiasp+	ret++++.align	5+KeccakF1600:+.long	0xd503233f			// paciasp+	stp	x29,x30,[sp,#-16*__SIZEOF_POINTER__]!+	add	x29,sp,#0+	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	stp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	stp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	stp	x27,x28,[sp,#10*__SIZEOF_POINTER__]+	sub	sp,sp,#16+4*__SIZEOF_POINTER__++	str	x0,[sp,#16+2*__SIZEOF_POINTER__]	// offload argument+	mov	x26,x0+	ldp	x0,x1,[x0,#16*0]+	ldp	x2,x3,[x26,#16*1]+	ldp	x4,x5,[x26,#16*2]+	ldp	x6,x7,[x26,#16*3]+	ldp	x8,x9,[x26,#16*4]+	ldp	x10,x11,[x26,#16*5]+	ldp	x12,x13,[x26,#16*6]+	ldp	x14,x15,[x26,#16*7]+	ldp	x16,x17,[x26,#16*8]+	ldp	x25,x19,[x26,#16*9]+	ldp	x20,x21,[x26,#16*10]+	ldp	x22,x23,[x26,#16*11]+	ldr	x24,[x26,#16*12]++	adr	x28,iotas+	bl	KeccakF1600_int++	ldr	x26,[sp,#16+2*__SIZEOF_POINTER__]+	stp	x0,x1,[x26,#16*0]+	stp	x2,x3,[x26,#16*1]+	stp	x4,x5,[x26,#16*2]+	stp	x6,x7,[x26,#16*3]+	stp	x8,x9,[x26,#16*4]+	stp	x10,x11,[x26,#16*5]+	stp	x12,x13,[x26,#16*6]+	stp	x14,x15,[x26,#16*7]+	stp	x16,x17,[x26,#16*8]+	stp	x25,x19,[x26,#16*9]+	stp	x20,x21,[x26,#16*10]+	stp	x22,x23,[x26,#16*11]+	str	x24,[x26,#16*12]++	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#16+4*__SIZEOF_POINTER__+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#16*__SIZEOF_POINTER__+.long	0xd50323bf			// autiasp+	ret+++.globl	_crypton_keccak_asm_absorb++.align	5+_crypton_keccak_asm_absorb:+.long	0xd503233f			// paciasp+	stp	x29,x30,[sp,#-16*__SIZEOF_POINTER__]!+	add	x29,sp,#0+	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	stp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	stp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	stp	x27,x28,[sp,#10*__SIZEOF_POINTER__]+	sub	sp,sp,#16+4*__SIZEOF_POINTER__+16++	stp	x0,x1,[sp,#16+2*__SIZEOF_POINTER__]	// offload arguments+	stp	x2,x3,[sp,#16+4*__SIZEOF_POINTER__]++	mov	x26,x0			// uint64_t A[5][5]+	mov	x27,x1			// const void *inp+	mov	x28,x2			// size_t len+	mov	x30,x3			// size_t bsz+	ldp	x0,x1,[x26,#16*0]+	ldp	x2,x3,[x26,#16*1]+	ldp	x4,x5,[x26,#16*2]+	ldp	x6,x7,[x26,#16*3]+	ldp	x8,x9,[x26,#16*4]+	ldp	x10,x11,[x26,#16*5]+	ldp	x12,x13,[x26,#16*6]+	ldp	x14,x15,[x26,#16*7]+	ldp	x16,x17,[x26,#16*8]+	ldp	x25,x19,[x26,#16*9]+	ldp	x20,x21,[x26,#16*10]+	ldp	x22,x23,[x26,#16*11]+	ldr	x24,[x26,#16*12]+	b	Loop_absorb++.align	4+Loop_absorb:+	subs	x26,x28,x30		// len - bsz+	blo	Labsorbed++	str	x26,[sp,#16+4*__SIZEOF_POINTER__]	// save len - bsz+	cmp	x30,#104+	ldr	x26,[x27,#0]		// A[0][0] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x0,x0,x26+	ldr	x26,[x27,#8]		// A[0][1] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x1,x1,x26+	ldr	x26,[x27,#16]		// A[0][2] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x2,x2,x26+	ldr	x26,[x27,#24]		// A[0][3] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x3,x3,x26+	ldr	x26,[x27,#32]		// A[0][4] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x4,x4,x26+	ldr	x26,[x27,#40]		// A[1][0] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x5,x5,x26+	ldr	x26,[x27,#48]		// A[1][1] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x6,x6,x26+	ldr	x26,[x27,#56]		// A[1][2] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x7,x7,x26+	ldr	x26,[x27,#64]		// A[1][3] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x8,x8,x26+	blo	Lprocess_block++	ldr	x26,[x27,#72]		// A[1][4] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x9,x9,x26+	ldr	x26,[x27,#80]		// A[2][0] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x10,x10,x26+	ldr	x26,[x27,#88]		// A[2][1] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x11,x11,x26+	ldr	x26,[x27,#96]		// A[2][2] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x12,x12,x26+	beq	Lprocess_block++	cmp	x30,#144+	ldr	x26,[x27,#104]		// A[2][3] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x13,x13,x26+	ldr	x26,[x27,#112]		// A[2][4] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x14,x14,x26+	ldr	x26,[x27,#120]		// A[3][0] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x15,x15,x26+	ldr	x26,[x27,#128]		// A[3][1] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x16,x16,x26+	blo	Lprocess_block++	ldr	x26,[x27,#136]		// A[3][2] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x17,x17,x26+	beq	Lprocess_block++	ldr	x26,[x27,#144]		// A[3][3] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x25,x25,x26+	ldr	x26,[x27,#152]		// A[3][4] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x19,x19,x26+	ldr	x26,[x27,#160]		// A[4][0] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x20,x20,x26++Lprocess_block:+	add	x27,x27,x30+	str	x27,[sp,#16+3*__SIZEOF_POINTER__]	// save inp++	adr	x28,iotas+	bl	KeccakF1600_int++	ldr	x27,[sp,#16+3*__SIZEOF_POINTER__]	// restore arguments+	ldp	x28,x30,[sp,#16+4*__SIZEOF_POINTER__]+	b	Loop_absorb++.align	4+Labsorbed:+	ldr	x27,[sp,#16+2*__SIZEOF_POINTER__]+	stp	x0,x1,[x27,#16*0]+	stp	x2,x3,[x27,#16*1]+	stp	x4,x5,[x27,#16*2]+	stp	x6,x7,[x27,#16*3]+	stp	x8,x9,[x27,#16*4]+	stp	x10,x11,[x27,#16*5]+	stp	x12,x13,[x27,#16*6]+	stp	x14,x15,[x27,#16*7]+	stp	x16,x17,[x27,#16*8]+	stp	x25,x19,[x27,#16*9]+	stp	x20,x21,[x27,#16*10]+	stp	x22,x23,[x27,#16*11]+	str	x24,[x27,#16*12]++	mov	x0,x28			// return value+	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#16+4*__SIZEOF_POINTER__+16+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#16*__SIZEOF_POINTER__+.long	0xd50323bf			// autiasp+	ret++.globl	_crypton_keccak_asm_squeeze++.align	5+_crypton_keccak_asm_squeeze:+.long	0xd503233f			// paciasp+	stp	x29,x30,[sp,#-6*__SIZEOF_POINTER__]!+	add	x29,sp,#0+	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]++	mov	x19,x0			// put aside arguments+	mov	x20,x1+	mov	x21,x2+	mov	x22,x3++Loop_squeeze:+	ldr	x4,[x0],#8+	cmp	x21,#8+	blo	Lsqueeze_tail+#ifdef	__AARCH64EB__+	rev	x4,x4+#endif+	str	x4,[x20],#8+	subs	x21,x21,#8+	beq	Lsqueeze_done++	subs	x3,x3,#8+	bhi	Loop_squeeze++	mov	x0,x19+	bl	KeccakF1600+	mov	x0,x19+	mov	x3,x22+	b	Loop_squeeze++.align	4+Lsqueeze_tail:+	strb	w4,[x20],#1+	lsr	x4,x4,#8+	subs	x21,x21,#1+	beq	Lsqueeze_done+	strb	w4,[x20],#1+	lsr	x4,x4,#8+	subs	x21,x21,#1+	beq	Lsqueeze_done+	strb	w4,[x20],#1+	lsr	x4,x4,#8+	subs	x21,x21,#1+	beq	Lsqueeze_done+	strb	w4,[x20],#1+	lsr	x4,x4,#8+	subs	x21,x21,#1+	beq	Lsqueeze_done+	strb	w4,[x20],#1+	lsr	x4,x4,#8+	subs	x21,x21,#1+	beq	Lsqueeze_done+	strb	w4,[x20],#1+	lsr	x4,x4,#8+	subs	x21,x21,#1+	beq	Lsqueeze_done+	strb	w4,[x20],#1++Lsqueeze_done:+	ldp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	ldp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#6*__SIZEOF_POINTER__+.long	0xd50323bf			// autiasp+	ret+++.align	5+KeccakF1600_ce:+Loop_ce:+	////////////////////////////////////////////////// Theta+.long	0xce0f2a99	//eor3 v25.16b,v20.16b,v15.16b,v10.16b+.long	0xce102eba	//eor3 v26.16b,v21.16b,v16.16b,v11.16b+.long	0xce1132db	//eor3 v27.16b,v22.16b,v17.16b,v12.16b+.long	0xce1236fc	//eor3 v28.16b,v23.16b,v18.16b,v13.16b+.long	0xce133b1d	//eor3 v29.16b,v24.16b,v19.16b,v14.16b+.long	0xce050339	//eor3 v25.16b,v25.16b,   v5.16b,v0.16b+.long	0xce06075a	//eor3 v26.16b,v26.16b,   v6.16b,v1.16b+.long	0xce070b7b	//eor3 v27.16b,v27.16b,   v7.16b,v2.16b+.long	0xce080f9c	//eor3 v28.16b,v28.16b,   v8.16b,v3.16b+.long	0xce0913bd	//eor3 v29.16b,v29.16b,   v9.16b,v4.16b++.long	0xce7b8f3e	//rax1 v30.2d,v25.2d,v27.2d			// D[1]+.long	0xce7c8f5f	//rax1 v31.2d,v26.2d,v28.2d			// D[2]+.long	0xce7d8f7b	//rax1 v27.2d,v27.2d,v29.2d			// D[3]+.long	0xce798f9c	//rax1 v28.2d,v28.2d,v25.2d			// D[4]+.long	0xce7a8fbd	//rax1 v29.2d,v29.2d,v26.2d			// D[0]++	////////////////////////////////////////////////// Theta+Rho+Pi+.long	0xce9efc39	//xar v25.2d,   v1.2d,v30.2d,#64-1 // C[0]=A[2][0]++.long	0xce9e50c1	//xar v1.2d,v6.2d,v30.2d,#64-44+.long	0xce9cb126	//xar v6.2d,v9.2d,v28.2d,#64-20+.long	0xce9f0ec9	//xar v9.2d,v22.2d,v31.2d,#64-61+.long	0xce9c65d6	//xar v22.2d,v14.2d,v28.2d,#64-39+.long	0xce9dba8e	//xar v14.2d,v20.2d,v29.2d,#64-18++.long	0xce9f085a	//xar v26.2d,   v2.2d,v31.2d,#64-62 // C[1]=A[4][0]++.long	0xce9f5582	//xar v2.2d,v12.2d,v31.2d,#64-43+.long	0xce9b9dac	//xar v12.2d,v13.2d,v27.2d,#64-25+.long	0xce9ce26d	//xar v13.2d,v19.2d,v28.2d,#64-8+.long	0xce9b22f3	//xar v19.2d,v23.2d,v27.2d,#64-56+.long	0xce9d5df7	//xar v23.2d,v15.2d,v29.2d,#64-41++.long	0xce9c948f	//xar v15.2d,v4.2d,v28.2d,#64-27++.long	0xce9ccb1c	//xar v28.2d,   v24.2d,v28.2d,#64-14 // D[4]=A[0][4]+.long	0xce9efab8	//xar v24.2d,v21.2d,v30.2d,#64-2+.long	0xce9b2508	//xar v8.2d,v8.2d,v27.2d,#64-55 // A[1][3]=A[4][1]+.long	0xce9e4e04	//xar v4.2d,v16.2d,v30.2d,#64-45 // A[0][4]=A[1][3]+.long	0xce9d70b0	//xar v16.2d,v5.2d,v29.2d,#64-36++.long	0xce9b9065	//xar v5.2d,v3.2d,v27.2d,#64-28++	eor	v0.16b,v0.16b,v29.16b++.long	0xce9bae5b	//xar v27.2d,   v18.2d,v27.2d,#64-21 // D[3]=A[0][3]+.long	0xce9fc623	//xar v3.2d,v17.2d,v31.2d,#64-15 // A[0][3]=A[3][3]+.long	0xce9ed97e	//xar v30.2d,   v11.2d,v30.2d,#64-10 // D[1]=A[3][2]+.long	0xce9fe8ff	//xar v31.2d,   v7.2d,v31.2d,#64-6 // D[2]=A[2][1]+.long	0xce9df55d	//xar v29.2d,   v10.2d,v29.2d,#64-3 // D[0]=A[1][2]++	////////////////////////////////////////////////// Chi+Iota+.long	0xce362354	//bcax v20.16b,v26.16b,   v22.16b,v8.16b	// A[1][3]=A[4][1]+.long	0xce375915	//bcax v21.16b,v8.16b,v23.16b,v22.16b	// A[1][3]=A[4][1]+.long	0xce385ed6	//bcax v22.16b,v22.16b,v24.16b,v23.16b+.long	0xce3a62f7	//bcax v23.16b,v23.16b,v26.16b,   v24.16b+.long	0xce286b18	//bcax v24.16b,v24.16b,v8.16b,v26.16b	// A[1][3]=A[4][1]++	ld1r	{v26.2d},[x10],#8++.long	0xce330fd1	//bcax v17.16b,v30.16b,   v19.16b,v3.16b	// A[0][3]=A[3][3]+.long	0xce2f4c72	//bcax v18.16b,v3.16b,v15.16b,v19.16b	// A[0][3]=A[3][3]+.long	0xce303e73	//bcax v19.16b,v19.16b,v16.16b,v15.16b+.long	0xce3e41ef	//bcax v15.16b,v15.16b,v30.16b,   v16.16b+.long	0xce237a10	//bcax v16.16b,v16.16b,v3.16b,v30.16b	// A[0][3]=A[3][3]++.long	0xce2c7f2a	//bcax v10.16b,v25.16b,   v12.16b,v31.16b+.long	0xce2d33eb	//bcax v11.16b,v31.16b,   v13.16b,v12.16b+.long	0xce2e358c	//bcax v12.16b,v12.16b,v14.16b,v13.16b+.long	0xce3939ad	//bcax v13.16b,v13.16b,v25.16b,   v14.16b+.long	0xce3f65ce	//bcax v14.16b,v14.16b,v31.16b,   v25.16b++.long	0xce2913a7	//bcax v7.16b,v29.16b,   v9.16b,v4.16b	// A[0][4]=A[1][3]+.long	0xce252488	//bcax v8.16b,v4.16b,v5.16b,v9.16b	// A[0][4]=A[1][3]+.long	0xce261529	//bcax v9.16b,v9.16b,v6.16b,v5.16b+.long	0xce3d18a5	//bcax v5.16b,v5.16b,v29.16b,   v6.16b+.long	0xce2474c6	//bcax v6.16b,v6.16b,v4.16b,v29.16b	// A[0][4]=A[1][3]++.long	0xce207363	//bcax v3.16b,v27.16b,   v0.16b,v28.16b+.long	0xce210384	//bcax v4.16b,v28.16b,   v1.16b,v0.16b+.long	0xce220400	//bcax v0.16b,v0.16b,v2.16b,v1.16b+.long	0xce3b0821	//bcax v1.16b,v1.16b,v27.16b,   v2.16b+.long	0xce3c6c42	//bcax v2.16b,v2.16b,v28.16b,   v27.16b++	eor	v0.16b,v0.16b,v26.16b++	tst	x10,#255+	bne	Loop_ce++	ret++++.align	5+KeccakF1600_cext:+.long	0xd503233f		// paciasp+	stp	x29,x30,[sp,#-2*__SIZEOF_POINTER__-64]!+	add	x29,sp,#0+	stp	d8,d9,[sp,#2*__SIZEOF_POINTER__+0]	// per ABI requirement+	stp	d10,d11,[sp,#2*__SIZEOF_POINTER__+16]+	stp	d12,d13,[sp,#2*__SIZEOF_POINTER__+32]+	stp	d14,d15,[sp,#2*__SIZEOF_POINTER__+48]+	ldp	d0,d1,[x0,#8*0]+	ldp	d2,d3,[x0,#8*2]+	ldp	d4,d5,[x0,#8*4]+	ldp	d6,d7,[x0,#8*6]+	ldp	d8,d9,[x0,#8*8]+	ldp	d10,d11,[x0,#8*10]+	ldp	d12,d13,[x0,#8*12]+	ldp	d14,d15,[x0,#8*14]+	ldp	d16,d17,[x0,#8*16]+	ldp	d18,d19,[x0,#8*18]+	ldp	d20,d21,[x0,#8*20]+	ldp	d22,d23,[x0,#8*22]+	ldr	d24,[x0,#8*24]+	adr	x10,iotas+	bl	KeccakF1600_ce+	ldr	x30,[sp,#__SIZEOF_POINTER__]+	stp	d0,d1,[x0,#8*0]+	stp	d2,d3,[x0,#8*2]+	stp	d4,d5,[x0,#8*4]+	stp	d6,d7,[x0,#8*6]+	stp	d8,d9,[x0,#8*8]+	stp	d10,d11,[x0,#8*10]+	stp	d12,d13,[x0,#8*12]+	stp	d14,d15,[x0,#8*14]+	stp	d16,d17,[x0,#8*16]+	stp	d18,d19,[x0,#8*18]+	stp	d20,d21,[x0,#8*20]+	stp	d22,d23,[x0,#8*22]+	str	d24,[x0,#8*24]++	ldp	d8,d9,[sp,#2*__SIZEOF_POINTER__+0]+	ldp	d10,d11,[sp,#2*__SIZEOF_POINTER__+16]+	ldp	d12,d13,[sp,#2*__SIZEOF_POINTER__+32]+	ldp	d14,d15,[sp,#2*__SIZEOF_POINTER__+48]+	ldr	x29,[sp],#2*__SIZEOF_POINTER__+64+.long	0xd50323bf		// autiasp+	ret++.globl	_crypton_keccak_asm_absorb_cext++.align	5+_crypton_keccak_asm_absorb_cext:+.long	0xd503233f		// paciasp+	stp	x29,x30,[sp,#-2*__SIZEOF_POINTER__-64]!+	add	x29,sp,#0+	stp	d8,d9,[sp,#2*__SIZEOF_POINTER__+0]	// per ABI requirement+	stp	d10,d11,[sp,#2*__SIZEOF_POINTER__+16]+	stp	d12,d13,[sp,#2*__SIZEOF_POINTER__+32]+	stp	d14,d15,[sp,#2*__SIZEOF_POINTER__+48]+	ldp	d0,d1,[x0,#8*0]+	ldp	d2,d3,[x0,#8*2]+	ldp	d4,d5,[x0,#8*4]+	ldp	d6,d7,[x0,#8*6]+	ldp	d8,d9,[x0,#8*8]+	ldp	d10,d11,[x0,#8*10]+	ldp	d12,d13,[x0,#8*12]+	ldp	d14,d15,[x0,#8*14]+	ldp	d16,d17,[x0,#8*16]+	ldp	d18,d19,[x0,#8*18]+	ldp	d20,d21,[x0,#8*20]+	ldp	d22,d23,[x0,#8*22]+	ldr	d24,[x0,#8*24]+	b	Loop_absorb_ce++.align	4+Loop_absorb_ce:+	subs	x2,x2,x3		// len - bsz+	blo	Labsorbed_ce++	cmp	x3,#104+	ld1	{v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32+	eor	v0.16b,v0.16b,v27.16b+	eor	v1.16b,v1.16b,v28.16b+	eor	v2.16b,v2.16b,v29.16b+	eor	v3.16b,v3.16b,v30.16b+	ld1	{v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32+	eor	v4.16b,v4.16b,v27.16b+	eor	v5.16b,v5.16b,v28.16b+	eor	v6.16b,v6.16b,v29.16b+	eor	v7.16b,v7.16b,v30.16b+	ld1	{v31.8b},[x1],#8	// A[1][4] ^= *inp+++	eor	v8.16b,v8.16b,v31.16b+	blo	Lprocess_block_ce++	ld1	{v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32+	eor	v9.16b,v9.16b,v27.16b+	eor	v10.16b,v10.16b,v28.16b+	eor	v11.16b,v11.16b,v29.16b+	eor	v12.16b,v12.16b,v30.16b+	beq	Lprocess_block_ce++	cmp	x3,#144+	ld1	{v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32+	eor	v13.16b,v13.16b,v27.16b+	eor	v14.16b,v14.16b,v28.16b+	eor	v15.16b,v15.16b,v29.16b+	eor	v16.16b,v16.16b,v30.16b+	blo	Lprocess_block_ce++	ld1	{v31.8b},[x1],#8	// A[3][3] ^= *inp+++	eor	v17.16b,v17.16b,v31.16b+	beq	Lprocess_block_ce++	ld1	{v28.8b,v29.8b,v30.8b},[x1],#24+	eor	v18.16b,v18.16b,v28.16b+	eor	v19.16b,v19.16b,v29.16b+	eor	v20.16b,v20.16b,v30.16b++Lprocess_block_ce:+	adr	x10,iotas+	bl	KeccakF1600_ce++	b	Loop_absorb_ce++.align	4+Labsorbed_ce:+	stp	d0,d1,[x0,#8*0]+	stp	d2,d3,[x0,#8*2]+	stp	d4,d5,[x0,#8*4]+	stp	d6,d7,[x0,#8*6]+	stp	d8,d9,[x0,#8*8]+	stp	d10,d11,[x0,#8*10]+	stp	d12,d13,[x0,#8*12]+	stp	d14,d15,[x0,#8*14]+	stp	d16,d17,[x0,#8*16]+	stp	d18,d19,[x0,#8*18]+	stp	d20,d21,[x0,#8*20]+	stp	d22,d23,[x0,#8*22]+	str	d24,[x0,#8*24]+	add	x0,x2,x3		// return value++	ldp	d8,d9,[sp,#2*__SIZEOF_POINTER__+0]+	ldp	d10,d11,[sp,#2*__SIZEOF_POINTER__+16]+	ldp	d12,d13,[sp,#2*__SIZEOF_POINTER__+32]+	ldp	d14,d15,[sp,#2*__SIZEOF_POINTER__+48]+	ldp	x29,x30,[sp],#2*__SIZEOF_POINTER__+64+.long	0xd50323bf		// autiasp+	ret++.globl	_crypton_keccak_asm_squeeze_cext++.align	5+_crypton_keccak_asm_squeeze_cext:+.long	0xd503233f		// paciasp+	stp	x29,x30,[sp,#-2*__SIZEOF_POINTER__]!+	add	x29,sp,#0+	mov	x9,x0+	mov	x10,x3++Loop_squeeze_ce:+	ldr	x4,[x9],#8+	cmp	x2,#8+	blo	Lsqueeze_tail_ce+#ifdef	__AARCH64EB__+	rev	x4,x4+#endif+	str	x4,[x1],#8+	beq	Lsqueeze_done_ce++	sub	x2,x2,#8+	subs	x10,x10,#8+	bhi	Loop_squeeze_ce++	bl	KeccakF1600_cext+	ldr	x30,[sp,#__SIZEOF_POINTER__]+	mov	x9,x0+	mov	x10,x3+	b	Loop_squeeze_ce++.align	4+Lsqueeze_tail_ce:+	strb	w4,[x1],#1+	lsr	x4,x4,#8+	subs	x2,x2,#1+	beq	Lsqueeze_done_ce+	strb	w4,[x1],#1+	lsr	x4,x4,#8+	subs	x2,x2,#1+	beq	Lsqueeze_done_ce+	strb	w4,[x1],#1+	lsr	x4,x4,#8+	subs	x2,x2,#1+	beq	Lsqueeze_done_ce+	strb	w4,[x1],#1+	lsr	x4,x4,#8+	subs	x2,x2,#1+	beq	Lsqueeze_done_ce+	strb	w4,[x1],#1+	lsr	x4,x4,#8+	subs	x2,x2,#1+	beq	Lsqueeze_done_ce+	strb	w4,[x1],#1+	lsr	x4,x4,#8+	subs	x2,x2,#1+	beq	Lsqueeze_done_ce+	strb	w4,[x1],#1++Lsqueeze_done_ce:+	ldr	x29,[sp],#2*__SIZEOF_POINTER__+.long	0xd50323bf		// autiasp+	ret++.byte	75,101,99,99,97,107,45,49,54,48,48,32,97,98,115,111,114,98,32,97,110,100,32,115,113,117,101,101,122,101,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.align	2
+ cbits/asm/keccak1600-armv8-linux64.S view
@@ -0,0 +1,843 @@+.text++.align	8	// strategic alignment and padding that allows to use+		// address value as loop termination condition...+.quad	0,0,0,0,0,0,0,0+.type	iotas,%object+iotas:+.quad	0x0000000000000001+.quad	0x0000000000008082+.quad	0x800000000000808a+.quad	0x8000000080008000+.quad	0x000000000000808b+.quad	0x0000000080000001+.quad	0x8000000080008081+.quad	0x8000000000008009+.quad	0x000000000000008a+.quad	0x0000000000000088+.quad	0x0000000080008009+.quad	0x000000008000000a+.Liotas12:+.quad	0x000000008000808b+.quad	0x800000000000008b+.quad	0x8000000000008089+.quad	0x8000000000008003+.quad	0x8000000000008002+.quad	0x8000000000000080+.quad	0x000000000000800a+.quad	0x800000008000000a+.quad	0x8000000080008081+.quad	0x8000000000008080+.quad	0x0000000080000001+.quad	0x8000000080008008+.size	iotas,.-iotas+.type	KeccakF1600_int,%function+.align	5+KeccakF1600_int:+.inst	0xd503233f			// paciasp+	stp	x28,x30,[sp,#16]		// stack is pre-allocated+	b	.Loop+.align	4+.Loop:+	////////////////////////////////////////// Theta+	eor	x26,x0,x5+	stp	x4,x9,[sp,#0]	// offload pair...+	eor	x27,x1,x6+	eor	x28,x2,x7+	eor	x30,x3,x8+	eor	x4,x4,x9+	eor	x26,x26,x10+	eor	x27,x27,x11+	eor	x28,x28,x12+	eor	x30,x30,x13+	eor	x4,x4,x14+	eor	x26,x26,x15+	eor	x27,x27,x16+	eor	x28,x28,x17+	eor	x30,x30,x25+	eor	x4,x4,x19+	eor	x26,x26,x20+	eor	x28,x28,x22+	eor	x27,x27,x21+	eor	x30,x30,x23+	eor	x4,x4,x24++	eor	x9,x26,x28,ror#63++	eor	x1,x1,x9+	eor	x6,x6,x9+	eor	x11,x11,x9+	eor	x16,x16,x9+	eor	x21,x21,x9++	eor	x9,x27,x30,ror#63+	eor	x28,x28,x4,ror#63+	eor	x30,x30,x26,ror#63+	eor	x4,x4,x27,ror#63++	eor	x27,   x2,x9		// mov	x27,x2+	eor	x7,x7,x9+	eor	x12,x12,x9+	eor	x17,x17,x9+	eor	x22,x22,x9++	eor	x0,x0,x4+	eor	x5,x5,x4+	eor	x10,x10,x4+	eor	x15,x15,x4+	eor	x20,x20,x4+	ldp	x4,x9,[sp,#0]	// re-load offloaded data+	eor	x26,   x3,x28		// mov	x26,x3+	eor	x8,x8,x28+	eor	x13,x13,x28+	eor	x25,x25,x28+	eor	x23,x23,x28++	eor	x28,   x4,x30		// mov	x28,x4+	eor	x9,x9,x30+	eor	x14,x14,x30+	eor	x19,x19,x30+	eor	x24,x24,x30++	////////////////////////////////////////// Rho+Pi+	mov	x30,x1+	ror	x1,x6,#64-44+	//mov	x27,x2+	ror	x2,x12,#64-43+	//mov	x26,x3+	ror	x3,x25,#64-21	// ?+	//mov	x28,x4+	ror	x4,x24,#64-14	// ?++	ror	x6,x9,#64-20	// ?+	ror	x12,x13,#64-25	// ?+	ror	x25,x17,#64-15+	ror	x24,x21,#64-2	// ?++	ror	x9,x22,#64-61+	ror	x13,x19,#64-8+	ror	x17,x11,#64-10+	ror	x21,x8,#64-55++	ror	x22,x14,#64-39+	ror	x19,x23,#64-56+	ror	x11,x7,#64-6	// ?+	ror	x8,x16,#64-45++	ror	x14,x20,#64-18+	ror	x23,x15,#64-41+	ror	x7,x10,#64-3+	ror	x16,x5,#64-36	// ?++	ror	x5,x26,#64-28	// ?+	ror	x10,x30,#64-1+	ror	x15,x28,#64-27	// ?+	ror	x20,x27,#64-62	// ?++	////////////////////////////////////////// Chi+Iota+	bic	x26,x2,x1+	bic	x27,x3,x2+	bic	x28,x0,x4+	bic	x30,x1,x0+	eor	x0,x0,x26+	bic	x26,x4,x3+	eor	x1,x1,x27+	ldr	x27,[sp,#16]+	eor	x3,x3,x28+	eor	x4,x4,x30+	eor	x2,x2,x26+	ldr	x30,[x27],#8		// Iota[i++]++	bic	x26,x7,x6+	tst	x27,#255			// are we done?+	str	x27,[sp,#16]+	bic	x27,x8,x7+	bic	x28,x5,x9+	eor	x0,x0,x30		// A[0][0] ^= Iota+	bic	x30,x6,x5+	eor	x5,x5,x26+	bic	x26,x9,x8+	eor	x6,x6,x27+	eor	x8,x8,x28+	eor	x9,x9,x30+	eor	x7,x7,x26++	bic	x26,x12,x11+	bic	x27,x13,x12+	bic	x28,x10,x14+	bic	x30,x11,x10+	eor	x10,x10,x26+	bic	x26,x14,x13+	eor	x11,x11,x27+	eor	x13,x13,x28+	eor	x14,x14,x30+	eor	x12,x12,x26++	bic	x26,x17,x16+	bic	x27,x25,x17+	bic	x28,x15,x19+	bic	x30,x16,x15+	eor	x15,x15,x26+	bic	x26,x19,x25+	eor	x16,x16,x27+	eor	x25,x25,x28+	eor	x19,x19,x30+	eor	x17,x17,x26++	bic	x26,x22,x21+	bic	x27,x23,x22+	bic	x28,x20,x24+	bic	x30,x21,x20+	eor	x20,x20,x26+	bic	x26,x24,x23+	eor	x21,x21,x27+	eor	x23,x23,x28+	eor	x24,x24,x30+	eor	x22,x22,x26++	bne	.Loop++	ldr	x30,[sp,#16+__SIZEOF_POINTER__]+.inst	0xd50323bf			// autiasp+	ret+.size	KeccakF1600_int,.-KeccakF1600_int++.type	KeccakF1600,%function+.align	5+KeccakF1600:+.inst	0xd503233f			// paciasp+	stp	x29,x30,[sp,#-16*__SIZEOF_POINTER__]!+	add	x29,sp,#0+	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	stp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	stp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	stp	x27,x28,[sp,#10*__SIZEOF_POINTER__]+	sub	sp,sp,#16+4*__SIZEOF_POINTER__++	str	x0,[sp,#16+2*__SIZEOF_POINTER__]	// offload argument+	mov	x26,x0+	ldp	x0,x1,[x0,#16*0]+	ldp	x2,x3,[x26,#16*1]+	ldp	x4,x5,[x26,#16*2]+	ldp	x6,x7,[x26,#16*3]+	ldp	x8,x9,[x26,#16*4]+	ldp	x10,x11,[x26,#16*5]+	ldp	x12,x13,[x26,#16*6]+	ldp	x14,x15,[x26,#16*7]+	ldp	x16,x17,[x26,#16*8]+	ldp	x25,x19,[x26,#16*9]+	ldp	x20,x21,[x26,#16*10]+	ldp	x22,x23,[x26,#16*11]+	ldr	x24,[x26,#16*12]++	adr	x28,iotas+	bl	KeccakF1600_int++	ldr	x26,[sp,#16+2*__SIZEOF_POINTER__]+	stp	x0,x1,[x26,#16*0]+	stp	x2,x3,[x26,#16*1]+	stp	x4,x5,[x26,#16*2]+	stp	x6,x7,[x26,#16*3]+	stp	x8,x9,[x26,#16*4]+	stp	x10,x11,[x26,#16*5]+	stp	x12,x13,[x26,#16*6]+	stp	x14,x15,[x26,#16*7]+	stp	x16,x17,[x26,#16*8]+	stp	x25,x19,[x26,#16*9]+	stp	x20,x21,[x26,#16*10]+	stp	x22,x23,[x26,#16*11]+	str	x24,[x26,#16*12]++	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#16+4*__SIZEOF_POINTER__+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#16*__SIZEOF_POINTER__+.inst	0xd50323bf			// autiasp+	ret+.size	KeccakF1600,.-KeccakF1600++.globl	crypton_keccak_asm_absorb+.type	crypton_keccak_asm_absorb,%function+.align	5+crypton_keccak_asm_absorb:+.inst	0xd503233f			// paciasp+	stp	x29,x30,[sp,#-16*__SIZEOF_POINTER__]!+	add	x29,sp,#0+	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	stp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	stp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	stp	x27,x28,[sp,#10*__SIZEOF_POINTER__]+	sub	sp,sp,#16+4*__SIZEOF_POINTER__+16++	stp	x0,x1,[sp,#16+2*__SIZEOF_POINTER__]	// offload arguments+	stp	x2,x3,[sp,#16+4*__SIZEOF_POINTER__]++	mov	x26,x0			// uint64_t A[5][5]+	mov	x27,x1			// const void *inp+	mov	x28,x2			// size_t len+	mov	x30,x3			// size_t bsz+	ldp	x0,x1,[x26,#16*0]+	ldp	x2,x3,[x26,#16*1]+	ldp	x4,x5,[x26,#16*2]+	ldp	x6,x7,[x26,#16*3]+	ldp	x8,x9,[x26,#16*4]+	ldp	x10,x11,[x26,#16*5]+	ldp	x12,x13,[x26,#16*6]+	ldp	x14,x15,[x26,#16*7]+	ldp	x16,x17,[x26,#16*8]+	ldp	x25,x19,[x26,#16*9]+	ldp	x20,x21,[x26,#16*10]+	ldp	x22,x23,[x26,#16*11]+	ldr	x24,[x26,#16*12]+	b	.Loop_absorb++.align	4+.Loop_absorb:+	subs	x26,x28,x30		// len - bsz+	blo	.Labsorbed++	str	x26,[sp,#16+4*__SIZEOF_POINTER__]	// save len - bsz+	cmp	x30,#104+	ldr	x26,[x27,#0]		// A[0][0] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x0,x0,x26+	ldr	x26,[x27,#8]		// A[0][1] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x1,x1,x26+	ldr	x26,[x27,#16]		// A[0][2] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x2,x2,x26+	ldr	x26,[x27,#24]		// A[0][3] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x3,x3,x26+	ldr	x26,[x27,#32]		// A[0][4] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x4,x4,x26+	ldr	x26,[x27,#40]		// A[1][0] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x5,x5,x26+	ldr	x26,[x27,#48]		// A[1][1] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x6,x6,x26+	ldr	x26,[x27,#56]		// A[1][2] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x7,x7,x26+	ldr	x26,[x27,#64]		// A[1][3] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x8,x8,x26+	blo	.Lprocess_block++	ldr	x26,[x27,#72]		// A[1][4] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x9,x9,x26+	ldr	x26,[x27,#80]		// A[2][0] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x10,x10,x26+	ldr	x26,[x27,#88]		// A[2][1] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x11,x11,x26+	ldr	x26,[x27,#96]		// A[2][2] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x12,x12,x26+	beq	.Lprocess_block++	cmp	x30,#144+	ldr	x26,[x27,#104]		// A[2][3] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x13,x13,x26+	ldr	x26,[x27,#112]		// A[2][4] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x14,x14,x26+	ldr	x26,[x27,#120]		// A[3][0] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x15,x15,x26+	ldr	x26,[x27,#128]		// A[3][1] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x16,x16,x26+	blo	.Lprocess_block++	ldr	x26,[x27,#136]		// A[3][2] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x17,x17,x26+	beq	.Lprocess_block++	ldr	x26,[x27,#144]		// A[3][3] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x25,x25,x26+	ldr	x26,[x27,#152]		// A[3][4] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x19,x19,x26+	ldr	x26,[x27,#160]		// A[4][0] ^= *inp+++#ifdef	__AARCH64EB__+	rev	x26,x26+#endif+	eor	x20,x20,x26++.Lprocess_block:+	add	x27,x27,x30+	str	x27,[sp,#16+3*__SIZEOF_POINTER__]	// save inp++	adr	x28,iotas+	bl	KeccakF1600_int++	ldr	x27,[sp,#16+3*__SIZEOF_POINTER__]	// restore arguments+	ldp	x28,x30,[sp,#16+4*__SIZEOF_POINTER__]+	b	.Loop_absorb++.align	4+.Labsorbed:+	ldr	x27,[sp,#16+2*__SIZEOF_POINTER__]+	stp	x0,x1,[x27,#16*0]+	stp	x2,x3,[x27,#16*1]+	stp	x4,x5,[x27,#16*2]+	stp	x6,x7,[x27,#16*3]+	stp	x8,x9,[x27,#16*4]+	stp	x10,x11,[x27,#16*5]+	stp	x12,x13,[x27,#16*6]+	stp	x14,x15,[x27,#16*7]+	stp	x16,x17,[x27,#16*8]+	stp	x25,x19,[x27,#16*9]+	stp	x20,x21,[x27,#16*10]+	stp	x22,x23,[x27,#16*11]+	str	x24,[x27,#16*12]++	mov	x0,x28			// return value+	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#16+4*__SIZEOF_POINTER__+16+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#16*__SIZEOF_POINTER__+.inst	0xd50323bf			// autiasp+	ret+.size	crypton_keccak_asm_absorb,.-crypton_keccak_asm_absorb+.globl	crypton_keccak_asm_squeeze+.type	crypton_keccak_asm_squeeze,%function+.align	5+crypton_keccak_asm_squeeze:+.inst	0xd503233f			// paciasp+	stp	x29,x30,[sp,#-6*__SIZEOF_POINTER__]!+	add	x29,sp,#0+	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]++	mov	x19,x0			// put aside arguments+	mov	x20,x1+	mov	x21,x2+	mov	x22,x3++.Loop_squeeze:+	ldr	x4,[x0],#8+	cmp	x21,#8+	blo	.Lsqueeze_tail+#ifdef	__AARCH64EB__+	rev	x4,x4+#endif+	str	x4,[x20],#8+	subs	x21,x21,#8+	beq	.Lsqueeze_done++	subs	x3,x3,#8+	bhi	.Loop_squeeze++	mov	x0,x19+	bl	KeccakF1600+	mov	x0,x19+	mov	x3,x22+	b	.Loop_squeeze++.align	4+.Lsqueeze_tail:+	strb	w4,[x20],#1+	lsr	x4,x4,#8+	subs	x21,x21,#1+	beq	.Lsqueeze_done+	strb	w4,[x20],#1+	lsr	x4,x4,#8+	subs	x21,x21,#1+	beq	.Lsqueeze_done+	strb	w4,[x20],#1+	lsr	x4,x4,#8+	subs	x21,x21,#1+	beq	.Lsqueeze_done+	strb	w4,[x20],#1+	lsr	x4,x4,#8+	subs	x21,x21,#1+	beq	.Lsqueeze_done+	strb	w4,[x20],#1+	lsr	x4,x4,#8+	subs	x21,x21,#1+	beq	.Lsqueeze_done+	strb	w4,[x20],#1+	lsr	x4,x4,#8+	subs	x21,x21,#1+	beq	.Lsqueeze_done+	strb	w4,[x20],#1++.Lsqueeze_done:+	ldp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	ldp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#6*__SIZEOF_POINTER__+.inst	0xd50323bf			// autiasp+	ret+.size	crypton_keccak_asm_squeeze,.-crypton_keccak_asm_squeeze+.type	KeccakF1600_ce,%function+.align	5+KeccakF1600_ce:+.Loop_ce:+	////////////////////////////////////////////////// Theta+.inst	0xce0f2a99	//eor3 v25.16b,v20.16b,v15.16b,v10.16b+.inst	0xce102eba	//eor3 v26.16b,v21.16b,v16.16b,v11.16b+.inst	0xce1132db	//eor3 v27.16b,v22.16b,v17.16b,v12.16b+.inst	0xce1236fc	//eor3 v28.16b,v23.16b,v18.16b,v13.16b+.inst	0xce133b1d	//eor3 v29.16b,v24.16b,v19.16b,v14.16b+.inst	0xce050339	//eor3 v25.16b,v25.16b,   v5.16b,v0.16b+.inst	0xce06075a	//eor3 v26.16b,v26.16b,   v6.16b,v1.16b+.inst	0xce070b7b	//eor3 v27.16b,v27.16b,   v7.16b,v2.16b+.inst	0xce080f9c	//eor3 v28.16b,v28.16b,   v8.16b,v3.16b+.inst	0xce0913bd	//eor3 v29.16b,v29.16b,   v9.16b,v4.16b++.inst	0xce7b8f3e	//rax1 v30.2d,v25.2d,v27.2d			// D[1]+.inst	0xce7c8f5f	//rax1 v31.2d,v26.2d,v28.2d			// D[2]+.inst	0xce7d8f7b	//rax1 v27.2d,v27.2d,v29.2d			// D[3]+.inst	0xce798f9c	//rax1 v28.2d,v28.2d,v25.2d			// D[4]+.inst	0xce7a8fbd	//rax1 v29.2d,v29.2d,v26.2d			// D[0]++	////////////////////////////////////////////////// Theta+Rho+Pi+.inst	0xce9efc39	//xar v25.2d,   v1.2d,v30.2d,#64-1 // C[0]=A[2][0]++.inst	0xce9e50c1	//xar v1.2d,v6.2d,v30.2d,#64-44+.inst	0xce9cb126	//xar v6.2d,v9.2d,v28.2d,#64-20+.inst	0xce9f0ec9	//xar v9.2d,v22.2d,v31.2d,#64-61+.inst	0xce9c65d6	//xar v22.2d,v14.2d,v28.2d,#64-39+.inst	0xce9dba8e	//xar v14.2d,v20.2d,v29.2d,#64-18++.inst	0xce9f085a	//xar v26.2d,   v2.2d,v31.2d,#64-62 // C[1]=A[4][0]++.inst	0xce9f5582	//xar v2.2d,v12.2d,v31.2d,#64-43+.inst	0xce9b9dac	//xar v12.2d,v13.2d,v27.2d,#64-25+.inst	0xce9ce26d	//xar v13.2d,v19.2d,v28.2d,#64-8+.inst	0xce9b22f3	//xar v19.2d,v23.2d,v27.2d,#64-56+.inst	0xce9d5df7	//xar v23.2d,v15.2d,v29.2d,#64-41++.inst	0xce9c948f	//xar v15.2d,v4.2d,v28.2d,#64-27++.inst	0xce9ccb1c	//xar v28.2d,   v24.2d,v28.2d,#64-14 // D[4]=A[0][4]+.inst	0xce9efab8	//xar v24.2d,v21.2d,v30.2d,#64-2+.inst	0xce9b2508	//xar v8.2d,v8.2d,v27.2d,#64-55 // A[1][3]=A[4][1]+.inst	0xce9e4e04	//xar v4.2d,v16.2d,v30.2d,#64-45 // A[0][4]=A[1][3]+.inst	0xce9d70b0	//xar v16.2d,v5.2d,v29.2d,#64-36++.inst	0xce9b9065	//xar v5.2d,v3.2d,v27.2d,#64-28++	eor	v0.16b,v0.16b,v29.16b++.inst	0xce9bae5b	//xar v27.2d,   v18.2d,v27.2d,#64-21 // D[3]=A[0][3]+.inst	0xce9fc623	//xar v3.2d,v17.2d,v31.2d,#64-15 // A[0][3]=A[3][3]+.inst	0xce9ed97e	//xar v30.2d,   v11.2d,v30.2d,#64-10 // D[1]=A[3][2]+.inst	0xce9fe8ff	//xar v31.2d,   v7.2d,v31.2d,#64-6 // D[2]=A[2][1]+.inst	0xce9df55d	//xar v29.2d,   v10.2d,v29.2d,#64-3 // D[0]=A[1][2]++	////////////////////////////////////////////////// Chi+Iota+.inst	0xce362354	//bcax v20.16b,v26.16b,   v22.16b,v8.16b	// A[1][3]=A[4][1]+.inst	0xce375915	//bcax v21.16b,v8.16b,v23.16b,v22.16b	// A[1][3]=A[4][1]+.inst	0xce385ed6	//bcax v22.16b,v22.16b,v24.16b,v23.16b+.inst	0xce3a62f7	//bcax v23.16b,v23.16b,v26.16b,   v24.16b+.inst	0xce286b18	//bcax v24.16b,v24.16b,v8.16b,v26.16b	// A[1][3]=A[4][1]++	ld1r	{v26.2d},[x10],#8++.inst	0xce330fd1	//bcax v17.16b,v30.16b,   v19.16b,v3.16b	// A[0][3]=A[3][3]+.inst	0xce2f4c72	//bcax v18.16b,v3.16b,v15.16b,v19.16b	// A[0][3]=A[3][3]+.inst	0xce303e73	//bcax v19.16b,v19.16b,v16.16b,v15.16b+.inst	0xce3e41ef	//bcax v15.16b,v15.16b,v30.16b,   v16.16b+.inst	0xce237a10	//bcax v16.16b,v16.16b,v3.16b,v30.16b	// A[0][3]=A[3][3]++.inst	0xce2c7f2a	//bcax v10.16b,v25.16b,   v12.16b,v31.16b+.inst	0xce2d33eb	//bcax v11.16b,v31.16b,   v13.16b,v12.16b+.inst	0xce2e358c	//bcax v12.16b,v12.16b,v14.16b,v13.16b+.inst	0xce3939ad	//bcax v13.16b,v13.16b,v25.16b,   v14.16b+.inst	0xce3f65ce	//bcax v14.16b,v14.16b,v31.16b,   v25.16b++.inst	0xce2913a7	//bcax v7.16b,v29.16b,   v9.16b,v4.16b	// A[0][4]=A[1][3]+.inst	0xce252488	//bcax v8.16b,v4.16b,v5.16b,v9.16b	// A[0][4]=A[1][3]+.inst	0xce261529	//bcax v9.16b,v9.16b,v6.16b,v5.16b+.inst	0xce3d18a5	//bcax v5.16b,v5.16b,v29.16b,   v6.16b+.inst	0xce2474c6	//bcax v6.16b,v6.16b,v4.16b,v29.16b	// A[0][4]=A[1][3]++.inst	0xce207363	//bcax v3.16b,v27.16b,   v0.16b,v28.16b+.inst	0xce210384	//bcax v4.16b,v28.16b,   v1.16b,v0.16b+.inst	0xce220400	//bcax v0.16b,v0.16b,v2.16b,v1.16b+.inst	0xce3b0821	//bcax v1.16b,v1.16b,v27.16b,   v2.16b+.inst	0xce3c6c42	//bcax v2.16b,v2.16b,v28.16b,   v27.16b++	eor	v0.16b,v0.16b,v26.16b++	tst	x10,#255+	bne	.Loop_ce++	ret+.size	KeccakF1600_ce,.-KeccakF1600_ce++.type	KeccakF1600_cext,%function+.align	5+KeccakF1600_cext:+.inst	0xd503233f		// paciasp+	stp	x29,x30,[sp,#-2*__SIZEOF_POINTER__-64]!+	add	x29,sp,#0+	stp	d8,d9,[sp,#2*__SIZEOF_POINTER__+0]	// per ABI requirement+	stp	d10,d11,[sp,#2*__SIZEOF_POINTER__+16]+	stp	d12,d13,[sp,#2*__SIZEOF_POINTER__+32]+	stp	d14,d15,[sp,#2*__SIZEOF_POINTER__+48]+	ldp	d0,d1,[x0,#8*0]+	ldp	d2,d3,[x0,#8*2]+	ldp	d4,d5,[x0,#8*4]+	ldp	d6,d7,[x0,#8*6]+	ldp	d8,d9,[x0,#8*8]+	ldp	d10,d11,[x0,#8*10]+	ldp	d12,d13,[x0,#8*12]+	ldp	d14,d15,[x0,#8*14]+	ldp	d16,d17,[x0,#8*16]+	ldp	d18,d19,[x0,#8*18]+	ldp	d20,d21,[x0,#8*20]+	ldp	d22,d23,[x0,#8*22]+	ldr	d24,[x0,#8*24]+	adr	x10,iotas+	bl	KeccakF1600_ce+	ldr	x30,[sp,#__SIZEOF_POINTER__]+	stp	d0,d1,[x0,#8*0]+	stp	d2,d3,[x0,#8*2]+	stp	d4,d5,[x0,#8*4]+	stp	d6,d7,[x0,#8*6]+	stp	d8,d9,[x0,#8*8]+	stp	d10,d11,[x0,#8*10]+	stp	d12,d13,[x0,#8*12]+	stp	d14,d15,[x0,#8*14]+	stp	d16,d17,[x0,#8*16]+	stp	d18,d19,[x0,#8*18]+	stp	d20,d21,[x0,#8*20]+	stp	d22,d23,[x0,#8*22]+	str	d24,[x0,#8*24]++	ldp	d8,d9,[sp,#2*__SIZEOF_POINTER__+0]+	ldp	d10,d11,[sp,#2*__SIZEOF_POINTER__+16]+	ldp	d12,d13,[sp,#2*__SIZEOF_POINTER__+32]+	ldp	d14,d15,[sp,#2*__SIZEOF_POINTER__+48]+	ldr	x29,[sp],#2*__SIZEOF_POINTER__+64+.inst	0xd50323bf		// autiasp+	ret+.size	KeccakF1600_cext,.-KeccakF1600_cext+.globl	crypton_keccak_asm_absorb_cext+.type	crypton_keccak_asm_absorb_cext,%function+.align	5+crypton_keccak_asm_absorb_cext:+.inst	0xd503233f		// paciasp+	stp	x29,x30,[sp,#-2*__SIZEOF_POINTER__-64]!+	add	x29,sp,#0+	stp	d8,d9,[sp,#2*__SIZEOF_POINTER__+0]	// per ABI requirement+	stp	d10,d11,[sp,#2*__SIZEOF_POINTER__+16]+	stp	d12,d13,[sp,#2*__SIZEOF_POINTER__+32]+	stp	d14,d15,[sp,#2*__SIZEOF_POINTER__+48]+	ldp	d0,d1,[x0,#8*0]+	ldp	d2,d3,[x0,#8*2]+	ldp	d4,d5,[x0,#8*4]+	ldp	d6,d7,[x0,#8*6]+	ldp	d8,d9,[x0,#8*8]+	ldp	d10,d11,[x0,#8*10]+	ldp	d12,d13,[x0,#8*12]+	ldp	d14,d15,[x0,#8*14]+	ldp	d16,d17,[x0,#8*16]+	ldp	d18,d19,[x0,#8*18]+	ldp	d20,d21,[x0,#8*20]+	ldp	d22,d23,[x0,#8*22]+	ldr	d24,[x0,#8*24]+	b	.Loop_absorb_ce++.align	4+.Loop_absorb_ce:+	subs	x2,x2,x3		// len - bsz+	blo	.Labsorbed_ce++	cmp	x3,#104+	ld1	{v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32+	eor	v0.16b,v0.16b,v27.16b+	eor	v1.16b,v1.16b,v28.16b+	eor	v2.16b,v2.16b,v29.16b+	eor	v3.16b,v3.16b,v30.16b+	ld1	{v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32+	eor	v4.16b,v4.16b,v27.16b+	eor	v5.16b,v5.16b,v28.16b+	eor	v6.16b,v6.16b,v29.16b+	eor	v7.16b,v7.16b,v30.16b+	ld1	{v31.8b},[x1],#8	// A[1][4] ^= *inp+++	eor	v8.16b,v8.16b,v31.16b+	blo	.Lprocess_block_ce++	ld1	{v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32+	eor	v9.16b,v9.16b,v27.16b+	eor	v10.16b,v10.16b,v28.16b+	eor	v11.16b,v11.16b,v29.16b+	eor	v12.16b,v12.16b,v30.16b+	beq	.Lprocess_block_ce++	cmp	x3,#144+	ld1	{v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32+	eor	v13.16b,v13.16b,v27.16b+	eor	v14.16b,v14.16b,v28.16b+	eor	v15.16b,v15.16b,v29.16b+	eor	v16.16b,v16.16b,v30.16b+	blo	.Lprocess_block_ce++	ld1	{v31.8b},[x1],#8	// A[3][3] ^= *inp+++	eor	v17.16b,v17.16b,v31.16b+	beq	.Lprocess_block_ce++	ld1	{v28.8b,v29.8b,v30.8b},[x1],#24+	eor	v18.16b,v18.16b,v28.16b+	eor	v19.16b,v19.16b,v29.16b+	eor	v20.16b,v20.16b,v30.16b++.Lprocess_block_ce:+	adr	x10,iotas+	bl	KeccakF1600_ce++	b	.Loop_absorb_ce++.align	4+.Labsorbed_ce:+	stp	d0,d1,[x0,#8*0]+	stp	d2,d3,[x0,#8*2]+	stp	d4,d5,[x0,#8*4]+	stp	d6,d7,[x0,#8*6]+	stp	d8,d9,[x0,#8*8]+	stp	d10,d11,[x0,#8*10]+	stp	d12,d13,[x0,#8*12]+	stp	d14,d15,[x0,#8*14]+	stp	d16,d17,[x0,#8*16]+	stp	d18,d19,[x0,#8*18]+	stp	d20,d21,[x0,#8*20]+	stp	d22,d23,[x0,#8*22]+	str	d24,[x0,#8*24]+	add	x0,x2,x3		// return value++	ldp	d8,d9,[sp,#2*__SIZEOF_POINTER__+0]+	ldp	d10,d11,[sp,#2*__SIZEOF_POINTER__+16]+	ldp	d12,d13,[sp,#2*__SIZEOF_POINTER__+32]+	ldp	d14,d15,[sp,#2*__SIZEOF_POINTER__+48]+	ldp	x29,x30,[sp],#2*__SIZEOF_POINTER__+64+.inst	0xd50323bf		// autiasp+	ret+.size	crypton_keccak_asm_absorb_cext,.-crypton_keccak_asm_absorb_cext+.globl	crypton_keccak_asm_squeeze_cext+.type	crypton_keccak_asm_squeeze_cext,%function+.align	5+crypton_keccak_asm_squeeze_cext:+.inst	0xd503233f		// paciasp+	stp	x29,x30,[sp,#-2*__SIZEOF_POINTER__]!+	add	x29,sp,#0+	mov	x9,x0+	mov	x10,x3++.Loop_squeeze_ce:+	ldr	x4,[x9],#8+	cmp	x2,#8+	blo	.Lsqueeze_tail_ce+#ifdef	__AARCH64EB__+	rev	x4,x4+#endif+	str	x4,[x1],#8+	beq	.Lsqueeze_done_ce++	sub	x2,x2,#8+	subs	x10,x10,#8+	bhi	.Loop_squeeze_ce++	bl	KeccakF1600_cext+	ldr	x30,[sp,#__SIZEOF_POINTER__]+	mov	x9,x0+	mov	x10,x3+	b	.Loop_squeeze_ce++.align	4+.Lsqueeze_tail_ce:+	strb	w4,[x1],#1+	lsr	x4,x4,#8+	subs	x2,x2,#1+	beq	.Lsqueeze_done_ce+	strb	w4,[x1],#1+	lsr	x4,x4,#8+	subs	x2,x2,#1+	beq	.Lsqueeze_done_ce+	strb	w4,[x1],#1+	lsr	x4,x4,#8+	subs	x2,x2,#1+	beq	.Lsqueeze_done_ce+	strb	w4,[x1],#1+	lsr	x4,x4,#8+	subs	x2,x2,#1+	beq	.Lsqueeze_done_ce+	strb	w4,[x1],#1+	lsr	x4,x4,#8+	subs	x2,x2,#1+	beq	.Lsqueeze_done_ce+	strb	w4,[x1],#1+	lsr	x4,x4,#8+	subs	x2,x2,#1+	beq	.Lsqueeze_done_ce+	strb	w4,[x1],#1++.Lsqueeze_done_ce:+	ldr	x29,[sp],#2*__SIZEOF_POINTER__+.inst	0xd50323bf		// autiasp+	ret+.size	crypton_keccak_asm_squeeze_cext,.-crypton_keccak_asm_squeeze_cext+.byte	75,101,99,99,97,107,45,49,54,48,48,32,97,98,115,111,114,98,32,97,110,100,32,115,113,117,101,101,122,101,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.align	2++.section	.note.GNU-stack,"",%progbits
+ cbits/asm/keccak1600-armv8.pl view
@@ -0,0 +1,932 @@+#!/usr/bin/env perl+#+# ====================================================================+# Written by Andy Polyakov, @dot-asm, initially for the OpenSSL+# project. The module is, however, dual licensed under OpenSSL and+# CRYPTOGAMS licenses depending on where you obtain it. For further+# details see http://www.openssl.org/~appro/cryptogams/.+# ====================================================================+#+# Keccak-1600 for ARMv8.+#+# June 2017.+#+# This is straightforward KECCAK_1X_ALT implementation. It makes no+# sense to attempt SIMD/NEON implementation for following reason.+# 64-bit lanes of vector registers can't be addressed as easily as in+# 32-bit mode. This means that 64-bit NEON is bound to be slower than+# 32-bit NEON, and this implementation is faster than 32-bit NEON on+# same processor. Even though it takes more scalar xor's and andn's,+# it gets compensated by availability of rotate. Not to forget that+# most processors achieve higher issue rate with scalar instructions.+#+# February 2018.+#+# Add hardware-assisted ARMv8.2 implementation. It's KECCAK_1X_ALT+# variant with register permutation/rotation twist that allows to+# eliminate copies to temporary registers. If you look closely you'll+# notice that it uses only one lane of vector registers. The new+# instructions effectively facilitate parallel hashing, which we don't+# support [yet?]. But lowest-level core procedure is prepared for it.+# The inner round is 67 [vector] instructions, so it's not actually+# obvious that it will provide performance improvement [in serial+# hash] as long as vector instructions issue rate is limited to 1 per+# cycle...+#+######################################################################+# Numbers are cycles per processed byte.+#+#		r=1088(*)+#+# Cortex-A53	13+# Cortex-A57	12+# Cortex-A76	7.9+# Cortex-X2	6.1 (***)+# Cortex-X925	3.0 (**)+# X-Gene	14+# Mongoose	10+# Kryo		12+# Snapdragon X	3.8 (**)+# Denver	7.8+# Apple A7	7.2+# Apple A10	6.1+# Apple A12	4.4+# Apple A14/M1	3.5 (**)+# ThunderX2	9.7+#+# (*)	Corresponds to SHA3-256. No improvement coefficients are listed+#	because they vary too much from compiler to compiler. Newer+#	compiler does much better and improvement varies from 5% on+#	Cortex-A57 to 25% on Cortex-A53. While in comparison to older+#	compiler this code is at least 2x faster...+# (**)	The result is for hardware-assisted implementation below.+# (***)	Hardware-assisted code is significantly slower, 11.3,+#	apparently because the processor can issue just one SHA3+#	instruction per cycle.++$flavour = shift;+$output  = shift;++if ($flavour && $flavour ne "void") {+    $0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1;+    ( $xlate="${dir}arm-xlate.pl" and -f $xlate ) or+    ( $xlate="${dir}../../perlasm/arm-xlate.pl" and -f $xlate) or+    die "can't locate arm-xlate.pl";++    open STDOUT,"| \"$^X\" $xlate $flavour $output";+} else {+    open STDOUT,">$output";+}++my @rhotates = ([  0,  1, 62, 28, 27 ],+                [ 36, 44,  6, 55, 20 ],+                [  3, 10, 43, 25, 39 ],+                [ 41, 45, 15, 21,  8 ],+                [ 18,  2, 61, 56, 14 ]);++my $sha3ops = ($flavour =~ /\+sha3/);++$code.=<<___	if ($sha3ops);+.arch	armv8.2-a+sha3+___+$code.=<<___;+.text++.align 8	// strategic alignment and padding that allows to use+		// address value as loop termination condition...+	.quad	0,0,0,0,0,0,0,0+.type	iotas,%object+iotas:+	.quad	0x0000000000000001+	.quad	0x0000000000008082+	.quad	0x800000000000808a+	.quad	0x8000000080008000+	.quad	0x000000000000808b+	.quad	0x0000000080000001+	.quad	0x8000000080008081+	.quad	0x8000000000008009+	.quad	0x000000000000008a+	.quad	0x0000000000000088+	.quad	0x0000000080008009+	.quad	0x000000008000000a+.Liotas12:+	.quad	0x000000008000808b+	.quad	0x800000000000008b+	.quad	0x8000000000008089+	.quad	0x8000000000008003+	.quad	0x8000000000008002+	.quad	0x8000000000000080+	.quad	0x000000000000800a+	.quad	0x800000008000000a+	.quad	0x8000000080008081+	.quad	0x8000000000008080+	.quad	0x0000000080000001+	.quad	0x8000000080008008+.size	iotas,.-iotas+___+								{{{+my @A = map([ "x$_", "x".($_+1), "x".($_+2), "x".($_+3), "x".($_+4) ],+            (0, 5, 10, 15, 20));+   $A[3][3] = "x25"; # x18 is reserved++my @C = map("x$_", (26,27,28,30));++$code.=<<___;+.type	KeccakF1600_int,%function+.align	5+KeccakF1600_int:+	.inst	0xd503233f			// paciasp+	stp	c#$C[2],c30,[csp,#16]		// stack is pre-allocated+	b	.Loop+.align	4+.Loop:+	////////////////////////////////////////// Theta+	eor	$C[0],$A[0][0],$A[1][0]+	stp	$A[0][4],$A[1][4],[sp,#0]	// offload pair...+	eor	$C[1],$A[0][1],$A[1][1]+	eor	$C[2],$A[0][2],$A[1][2]+	eor	$C[3],$A[0][3],$A[1][3]+___+	$C[4]=$A[0][4];+	$C[5]=$A[1][4];+$code.=<<___;+	eor	$C[4],$A[0][4],$A[1][4]+	eor	$C[0],$C[0],$A[2][0]+	eor	$C[1],$C[1],$A[2][1]+	eor	$C[2],$C[2],$A[2][2]+	eor	$C[3],$C[3],$A[2][3]+	eor	$C[4],$C[4],$A[2][4]+	eor	$C[0],$C[0],$A[3][0]+	eor	$C[1],$C[1],$A[3][1]+	eor	$C[2],$C[2],$A[3][2]+	eor	$C[3],$C[3],$A[3][3]+	eor	$C[4],$C[4],$A[3][4]+	eor	$C[0],$C[0],$A[4][0]+	eor	$C[2],$C[2],$A[4][2]+	eor	$C[1],$C[1],$A[4][1]+	eor	$C[3],$C[3],$A[4][3]+	eor	$C[4],$C[4],$A[4][4]++	eor	$C[5],$C[0],$C[2],ror#63++	eor	$A[0][1],$A[0][1],$C[5]+	eor	$A[1][1],$A[1][1],$C[5]+	eor	$A[2][1],$A[2][1],$C[5]+	eor	$A[3][1],$A[3][1],$C[5]+	eor	$A[4][1],$A[4][1],$C[5]++	eor	$C[5],$C[1],$C[3],ror#63+	eor	$C[2],$C[2],$C[4],ror#63+	eor	$C[3],$C[3],$C[0],ror#63+	eor	$C[4],$C[4],$C[1],ror#63++	eor	$C[1],   $A[0][2],$C[5]		// mov	$C[1],$A[0][2]+	eor	$A[1][2],$A[1][2],$C[5]+	eor	$A[2][2],$A[2][2],$C[5]+	eor	$A[3][2],$A[3][2],$C[5]+	eor	$A[4][2],$A[4][2],$C[5]++	eor	$A[0][0],$A[0][0],$C[4]+	eor	$A[1][0],$A[1][0],$C[4]+	eor	$A[2][0],$A[2][0],$C[4]+	eor	$A[3][0],$A[3][0],$C[4]+	eor	$A[4][0],$A[4][0],$C[4]+___+	$C[4]=undef;+	$C[5]=undef;+$code.=<<___;+	ldp	$A[0][4],$A[1][4],[sp,#0]	// re-load offloaded data+	eor	$C[0],   $A[0][3],$C[2]		// mov	$C[0],$A[0][3]+	eor	$A[1][3],$A[1][3],$C[2]+	eor	$A[2][3],$A[2][3],$C[2]+	eor	$A[3][3],$A[3][3],$C[2]+	eor	$A[4][3],$A[4][3],$C[2]++	eor	$C[2],   $A[0][4],$C[3]		// mov	$C[2],$A[0][4]+	eor	$A[1][4],$A[1][4],$C[3]+	eor	$A[2][4],$A[2][4],$C[3]+	eor	$A[3][4],$A[3][4],$C[3]+	eor	$A[4][4],$A[4][4],$C[3]++	////////////////////////////////////////// Rho+Pi+	mov	$C[3],$A[0][1]+	ror	$A[0][1],$A[1][1],#64-$rhotates[1][1]+	//mov	$C[1],$A[0][2]+	ror	$A[0][2],$A[2][2],#64-$rhotates[2][2]+	//mov	$C[0],$A[0][3]+	ror	$A[0][3],$A[3][3],#64-$rhotates[3][3]	// ?+	//mov	$C[2],$A[0][4]+	ror	$A[0][4],$A[4][4],#64-$rhotates[4][4]	// ?++	ror	$A[1][1],$A[1][4],#64-$rhotates[1][4]	// ?+	ror	$A[2][2],$A[2][3],#64-$rhotates[2][3]	// ?+	ror	$A[3][3],$A[3][2],#64-$rhotates[3][2]+	ror	$A[4][4],$A[4][1],#64-$rhotates[4][1]	// ?++	ror	$A[1][4],$A[4][2],#64-$rhotates[4][2]+	ror	$A[2][3],$A[3][4],#64-$rhotates[3][4]+	ror	$A[3][2],$A[2][1],#64-$rhotates[2][1]+	ror	$A[4][1],$A[1][3],#64-$rhotates[1][3]++	ror	$A[4][2],$A[2][4],#64-$rhotates[2][4]+	ror	$A[3][4],$A[4][3],#64-$rhotates[4][3]+	ror	$A[2][1],$A[1][2],#64-$rhotates[1][2]	// ?+	ror	$A[1][3],$A[3][1],#64-$rhotates[3][1]++	ror	$A[2][4],$A[4][0],#64-$rhotates[4][0]+	ror	$A[4][3],$A[3][0],#64-$rhotates[3][0]+	ror	$A[1][2],$A[2][0],#64-$rhotates[2][0]+	ror	$A[3][1],$A[1][0],#64-$rhotates[1][0]	// ?++	ror	$A[1][0],$C[0],#64-$rhotates[0][3]	// ?+	ror	$A[2][0],$C[3],#64-$rhotates[0][1]+	ror	$A[3][0],$C[2],#64-$rhotates[0][4]	// ?+	ror	$A[4][0],$C[1],#64-$rhotates[0][2]	// ?++	////////////////////////////////////////// Chi+Iota+	bic	$C[0],$A[0][2],$A[0][1]+	bic	$C[1],$A[0][3],$A[0][2]+	bic	$C[2],$A[0][0],$A[0][4]+	bic	$C[3],$A[0][1],$A[0][0]+	eor	$A[0][0],$A[0][0],$C[0]+	bic	$C[0],$A[0][4],$A[0][3]+	eor	$A[0][1],$A[0][1],$C[1]+	 ldr	c#$C[1],[csp,#16]+	eor	$A[0][3],$A[0][3],$C[2]+	eor	$A[0][4],$A[0][4],$C[3]+	eor	$A[0][2],$A[0][2],$C[0]+	 ldr	$C[3],[$C[1]],#8		// Iota[i++]++	bic	$C[0],$A[1][2],$A[1][1]+	 tst	$C[1],#255			// are we done?+	 str	c#$C[1],[csp,#16]+	bic	$C[1],$A[1][3],$A[1][2]+	bic	$C[2],$A[1][0],$A[1][4]+	 eor	$A[0][0],$A[0][0],$C[3]		// A[0][0] ^= Iota+	bic	$C[3],$A[1][1],$A[1][0]+	eor	$A[1][0],$A[1][0],$C[0]+	bic	$C[0],$A[1][4],$A[1][3]+	eor	$A[1][1],$A[1][1],$C[1]+	eor	$A[1][3],$A[1][3],$C[2]+	eor	$A[1][4],$A[1][4],$C[3]+	eor	$A[1][2],$A[1][2],$C[0]++	bic	$C[0],$A[2][2],$A[2][1]+	bic	$C[1],$A[2][3],$A[2][2]+	bic	$C[2],$A[2][0],$A[2][4]+	bic	$C[3],$A[2][1],$A[2][0]+	eor	$A[2][0],$A[2][0],$C[0]+	bic	$C[0],$A[2][4],$A[2][3]+	eor	$A[2][1],$A[2][1],$C[1]+	eor	$A[2][3],$A[2][3],$C[2]+	eor	$A[2][4],$A[2][4],$C[3]+	eor	$A[2][2],$A[2][2],$C[0]++	bic	$C[0],$A[3][2],$A[3][1]+	bic	$C[1],$A[3][3],$A[3][2]+	bic	$C[2],$A[3][0],$A[3][4]+	bic	$C[3],$A[3][1],$A[3][0]+	eor	$A[3][0],$A[3][0],$C[0]+	bic	$C[0],$A[3][4],$A[3][3]+	eor	$A[3][1],$A[3][1],$C[1]+	eor	$A[3][3],$A[3][3],$C[2]+	eor	$A[3][4],$A[3][4],$C[3]+	eor	$A[3][2],$A[3][2],$C[0]++	bic	$C[0],$A[4][2],$A[4][1]+	bic	$C[1],$A[4][3],$A[4][2]+	bic	$C[2],$A[4][0],$A[4][4]+	bic	$C[3],$A[4][1],$A[4][0]+	eor	$A[4][0],$A[4][0],$C[0]+	bic	$C[0],$A[4][4],$A[4][3]+	eor	$A[4][1],$A[4][1],$C[1]+	eor	$A[4][3],$A[4][3],$C[2]+	eor	$A[4][4],$A[4][4],$C[3]+	eor	$A[4][2],$A[4][2],$C[0]++	bne	.Loop++	ldr	c30,[csp,#16+__SIZEOF_POINTER__]+	.inst	0xd50323bf			// autiasp+	ret+.size	KeccakF1600_int,.-KeccakF1600_int++.type	KeccakF1600,%function+.align	5+KeccakF1600:+	.inst	0xd503233f			// paciasp+	stp	c29,c30,[csp,#-16*__SIZEOF_POINTER__]!+	add	c29,csp,#0+	stp	c19,c20,[csp,#2*__SIZEOF_POINTER__]+	stp	c21,c22,[csp,#4*__SIZEOF_POINTER__]+	stp	c23,c24,[csp,#6*__SIZEOF_POINTER__]+	stp	c25,c26,[csp,#8*__SIZEOF_POINTER__]+	stp	c27,c28,[csp,#10*__SIZEOF_POINTER__]+	sub	csp,csp,#16+4*__SIZEOF_POINTER__++	str	c0,[csp,#16+2*__SIZEOF_POINTER__]	// offload argument+	mov	c#$C[0],c0+	ldp	$A[0][0],$A[0][1],[x0,#16*0]+	ldp	$A[0][2],$A[0][3],[$C[0],#16*1]+	ldp	$A[0][4],$A[1][0],[$C[0],#16*2]+	ldp	$A[1][1],$A[1][2],[$C[0],#16*3]+	ldp	$A[1][3],$A[1][4],[$C[0],#16*4]+	ldp	$A[2][0],$A[2][1],[$C[0],#16*5]+	ldp	$A[2][2],$A[2][3],[$C[0],#16*6]+	ldp	$A[2][4],$A[3][0],[$C[0],#16*7]+	ldp	$A[3][1],$A[3][2],[$C[0],#16*8]+	ldp	$A[3][3],$A[3][4],[$C[0],#16*9]+	ldp	$A[4][0],$A[4][1],[$C[0],#16*10]+	ldp	$A[4][2],$A[4][3],[$C[0],#16*11]+	ldr	$A[4][4],[$C[0],#16*12]++	adr	$C[2],iotas+	bl	KeccakF1600_int++	ldr	c#$C[0],[csp,#16+2*__SIZEOF_POINTER__]+	stp	$A[0][0],$A[0][1],[$C[0],#16*0]+	stp	$A[0][2],$A[0][3],[$C[0],#16*1]+	stp	$A[0][4],$A[1][0],[$C[0],#16*2]+	stp	$A[1][1],$A[1][2],[$C[0],#16*3]+	stp	$A[1][3],$A[1][4],[$C[0],#16*4]+	stp	$A[2][0],$A[2][1],[$C[0],#16*5]+	stp	$A[2][2],$A[2][3],[$C[0],#16*6]+	stp	$A[2][4],$A[3][0],[$C[0],#16*7]+	stp	$A[3][1],$A[3][2],[$C[0],#16*8]+	stp	$A[3][3],$A[3][4],[$C[0],#16*9]+	stp	$A[4][0],$A[4][1],[$C[0],#16*10]+	stp	$A[4][2],$A[4][3],[$C[0],#16*11]+	str	$A[4][4],[$C[0],#16*12]++	ldp	c19,c20,[c29,#2*__SIZEOF_POINTER__]+	add	csp,csp,#16+4*__SIZEOF_POINTER__+	ldp	c21,c22,[c29,#4*__SIZEOF_POINTER__]+	ldp	c23,c24,[c29,#6*__SIZEOF_POINTER__]+	ldp	c25,c26,[c29,#8*__SIZEOF_POINTER__]+	ldp	c27,c28,[c29,#10*__SIZEOF_POINTER__]+	ldp	c29,c30,[csp],#16*__SIZEOF_POINTER__+	.inst	0xd50323bf			// autiasp+	ret+.size	KeccakF1600,.-KeccakF1600++.globl	SHA3_absorb+.type	SHA3_absorb,%function+.align	5+SHA3_absorb:+	.inst	0xd503233f			// paciasp+	stp	c29,c30,[csp,#-16*__SIZEOF_POINTER__]!+	add	c29,csp,#0+	stp	c19,c20,[csp,#2*__SIZEOF_POINTER__]+	stp	c21,c22,[csp,#4*__SIZEOF_POINTER__]+	stp	c23,c24,[csp,#6*__SIZEOF_POINTER__]+	stp	c25,c26,[csp,#8*__SIZEOF_POINTER__]+	stp	c27,c28,[csp,#10*__SIZEOF_POINTER__]+	sub	csp,csp,#16+4*__SIZEOF_POINTER__+16++	stp	c0,c1,[csp,#16+2*__SIZEOF_POINTER__]	// offload arguments+	stp	x2,x3,[csp,#16+4*__SIZEOF_POINTER__]++	mov	c#$C[0],c0			// uint64_t A[5][5]+	mov	c#$C[1],c1			// const void *inp+	mov	$C[2],x2			// size_t len+	mov	$C[3],x3			// size_t bsz+	ldp	$A[0][0],$A[0][1],[$C[0],#16*0]+	ldp	$A[0][2],$A[0][3],[$C[0],#16*1]+	ldp	$A[0][4],$A[1][0],[$C[0],#16*2]+	ldp	$A[1][1],$A[1][2],[$C[0],#16*3]+	ldp	$A[1][3],$A[1][4],[$C[0],#16*4]+	ldp	$A[2][0],$A[2][1],[$C[0],#16*5]+	ldp	$A[2][2],$A[2][3],[$C[0],#16*6]+	ldp	$A[2][4],$A[3][0],[$C[0],#16*7]+	ldp	$A[3][1],$A[3][2],[$C[0],#16*8]+	ldp	$A[3][3],$A[3][4],[$C[0],#16*9]+	ldp	$A[4][0],$A[4][1],[$C[0],#16*10]+	ldp	$A[4][2],$A[4][3],[$C[0],#16*11]+	ldr	$A[4][4],[$C[0],#16*12]+	b	.Loop_absorb++.align	4+.Loop_absorb:+	subs	$C[0],$C[2],$C[3]		// len - bsz+	blo	.Labsorbed++	str	$C[0],[csp,#16+4*__SIZEOF_POINTER__]	// save len - bsz+	cmp	$C[3],#104+___+sub load_n_xor {+    my ($from,$to) = @_;++    for (my $i=$from; $i<=$to; $i++) {+$code.=<<___;+	ldr	$C[0],[$C[1],#`8*$i`]		// A[`$i/5`][`$i%5`] ^= *inp+++#ifdef	__AARCH64EB__+	rev	$C[0],$C[0]+#endif+	eor	$A[$i/5][$i%5],$A[$i/5][$i%5],$C[0]+___+    }+}+load_n_xor(0,8);+$code.=<<___;+	blo	.Lprocess_block++___+load_n_xor(9,12);+$code.=<<___;+	beq	.Lprocess_block++	cmp	$C[3],#144+___+load_n_xor(13,16);+$code.=<<___;+	blo	.Lprocess_block++___+load_n_xor(17,17);+$code.=<<___;+	beq	.Lprocess_block++___+load_n_xor(18,20);+$code.=<<___;++.Lprocess_block:+	add	c#$C[1],c#@C[1],@C[3]+	str	c#$C[1],[csp,#16+3*__SIZEOF_POINTER__]	// save inp++	adr	$C[2],iotas+	bl	KeccakF1600_int++	ldr	c#$C[1],[csp,#16+3*__SIZEOF_POINTER__]	// restore arguments+	ldp	$C[2],$C[3],[csp,#16+4*__SIZEOF_POINTER__]+	b	.Loop_absorb++.align	4+.Labsorbed:+	ldr	c#$C[1],[sp,#16+2*__SIZEOF_POINTER__]+	stp	$A[0][0],$A[0][1],[$C[1],#16*0]+	stp	$A[0][2],$A[0][3],[$C[1],#16*1]+	stp	$A[0][4],$A[1][0],[$C[1],#16*2]+	stp	$A[1][1],$A[1][2],[$C[1],#16*3]+	stp	$A[1][3],$A[1][4],[$C[1],#16*4]+	stp	$A[2][0],$A[2][1],[$C[1],#16*5]+	stp	$A[2][2],$A[2][3],[$C[1],#16*6]+	stp	$A[2][4],$A[3][0],[$C[1],#16*7]+	stp	$A[3][1],$A[3][2],[$C[1],#16*8]+	stp	$A[3][3],$A[3][4],[$C[1],#16*9]+	stp	$A[4][0],$A[4][1],[$C[1],#16*10]+	stp	$A[4][2],$A[4][3],[$C[1],#16*11]+	str	$A[4][4],[$C[1],#16*12]++	mov	x0,$C[2]			// return value+	ldp	c19,c20,[c29,#2*__SIZEOF_POINTER__]+	add	csp,csp,#16+4*__SIZEOF_POINTER__+16+	ldp	c21,c22,[c29,#4*__SIZEOF_POINTER__]+	ldp	c23,c24,[c29,#6*__SIZEOF_POINTER__]+	ldp	c25,c26,[c29,#8*__SIZEOF_POINTER__]+	ldp	c27,c28,[c29,#10*__SIZEOF_POINTER__]+	ldp	c29,c30,[csp],#16*__SIZEOF_POINTER__+	.inst	0xd50323bf			// autiasp+	ret+.size	SHA3_absorb,.-SHA3_absorb+___+{+my ($A_flat,$out,$len,$bsz) = map("x$_",(19..22));+$code.=<<___;+.globl	SHA3_squeeze+.type	SHA3_squeeze,%function+.align	5+SHA3_squeeze:+	.inst	0xd503233f			// paciasp+	stp	c29,c30,[csp,#-6*__SIZEOF_POINTER__]!+	add	c29,csp,#0+	stp	c19,c20,[csp,#2*__SIZEOF_POINTER__]+	stp	c21,c22,[csp,#4*__SIZEOF_POINTER__]++	cmov	$A_flat,x0			// put aside arguments+	cmov	$out,x1+	mov	$len,x2+	mov	$bsz,x3++.Loop_squeeze:+	ldr	x4,[x0],#8+	cmp	$len,#8+	blo	.Lsqueeze_tail+#ifdef	__AARCH64EB__+	rev	x4,x4+#endif+	str	x4,[$out],#8+	subs	$len,$len,#8+	beq	.Lsqueeze_done++	subs	x3,x3,#8+	bhi	.Loop_squeeze++	cmov	x0,$A_flat+	bl	KeccakF1600+	cmov	x0,$A_flat+	mov	x3,$bsz+	b	.Loop_squeeze++.align	4+.Lsqueeze_tail:+	strb	w4,[$out],#1+	lsr	x4,x4,#8+	subs	$len,$len,#1+	beq	.Lsqueeze_done+	strb	w4,[$out],#1+	lsr	x4,x4,#8+	subs	$len,$len,#1+	beq	.Lsqueeze_done+	strb	w4,[$out],#1+	lsr	x4,x4,#8+	subs	$len,$len,#1+	beq	.Lsqueeze_done+	strb	w4,[$out],#1+	lsr	x4,x4,#8+	subs	$len,$len,#1+	beq	.Lsqueeze_done+	strb	w4,[$out],#1+	lsr	x4,x4,#8+	subs	$len,$len,#1+	beq	.Lsqueeze_done+	strb	w4,[$out],#1+	lsr	x4,x4,#8+	subs	$len,$len,#1+	beq	.Lsqueeze_done+	strb	w4,[$out],#1++.Lsqueeze_done:+	ldp	c19,c20,[csp,#2*__SIZEOF_POINTER__]+	ldp	c21,c22,[csp,#4*__SIZEOF_POINTER__]+	ldp	c29,c30,[csp],#6*__SIZEOF_POINTER__+	.inst	0xd50323bf			// autiasp+	ret+.size	SHA3_squeeze,.-SHA3_squeeze+___+}								}}}+								{{{+my @A = map([ "v".$_.".16b", "v".($_+1).".16b", "v".($_+2).".16b",+                             "v".($_+3).".16b", "v".($_+4).".16b" ],+            (0, 5, 10, 15, 20));++my @C = map("v$_.16b", (25..31));+my @D = @C[4,5,6,2,3];++$code.=<<___;+.type	KeccakF1600_ce,%function+.align	5+KeccakF1600_ce:+.Loop_ce:+	////////////////////////////////////////////////// Theta+	eor3	$C[0],$A[4][0],$A[3][0],$A[2][0]+	eor3	$C[1],$A[4][1],$A[3][1],$A[2][1]+	eor3	$C[2],$A[4][2],$A[3][2],$A[2][2]+	eor3	$C[3],$A[4][3],$A[3][3],$A[2][3]+	eor3	$C[4],$A[4][4],$A[3][4],$A[2][4]+	eor3	$C[0],$C[0],   $A[1][0],$A[0][0]+	eor3	$C[1],$C[1],   $A[1][1],$A[0][1]+	eor3	$C[2],$C[2],   $A[1][2],$A[0][2]+	eor3	$C[3],$C[3],   $A[1][3],$A[0][3]+	eor3	$C[4],$C[4],   $A[1][4],$A[0][4]++	rax1	$C[5],$C[0],$C[2]			// D[1]+	rax1	$C[6],$C[1],$C[3]			// D[2]+	rax1	$C[2],$C[2],$C[4]			// D[3]+	rax1	$C[3],$C[3],$C[0]			// D[4]+	rax1	$C[4],$C[4],$C[1]			// D[0]++	////////////////////////////////////////////////// Theta+Rho+Pi+	xar	$C[0],   $A[0][1],$D[1],#64-$rhotates[0][1] // C[0]=A[2][0]++	xar	$A[0][1],$A[1][1],$D[1],#64-$rhotates[1][1]+	xar	$A[1][1],$A[1][4],$D[4],#64-$rhotates[1][4]+	xar	$A[1][4],$A[4][2],$D[2],#64-$rhotates[4][2]+	xar	$A[4][2],$A[2][4],$D[4],#64-$rhotates[2][4]+	xar	$A[2][4],$A[4][0],$D[0],#64-$rhotates[4][0]++	xar	$C[1],   $A[0][2],$D[2],#64-$rhotates[0][2] // C[1]=A[4][0]++	xar	$A[0][2],$A[2][2],$D[2],#64-$rhotates[2][2]+	xar	$A[2][2],$A[2][3],$D[3],#64-$rhotates[2][3]+	xar	$A[2][3],$A[3][4],$D[4],#64-$rhotates[3][4]+	xar	$A[3][4],$A[4][3],$D[3],#64-$rhotates[4][3]+	xar	$A[4][3],$A[3][0],$D[0],#64-$rhotates[3][0]++	xar	$A[3][0],$A[0][4],$D[4],#64-$rhotates[0][4]++	xar	$D[4],   $A[4][4],$D[4],#64-$rhotates[4][4] // D[4]=A[0][4]+	xar	$A[4][4],$A[4][1],$D[1],#64-$rhotates[4][1]+	xar	$A[1][3],$A[1][3],$D[3],#64-$rhotates[1][3] // A[1][3]=A[4][1]+	xar	$A[0][4],$A[3][1],$D[1],#64-$rhotates[3][1] // A[0][4]=A[1][3]+	xar	$A[3][1],$A[1][0],$D[0],#64-$rhotates[1][0]++	xar	$A[1][0],$A[0][3],$D[3],#64-$rhotates[0][3]++	eor	$A[0][0],$A[0][0],$D[0]++	xar	$D[3],   $A[3][3],$D[3],#64-$rhotates[3][3] // D[3]=A[0][3]+	xar	$A[0][3],$A[3][2],$D[2],#64-$rhotates[3][2] // A[0][3]=A[3][3]+	xar	$D[1],   $A[2][1],$D[1],#64-$rhotates[2][1] // D[1]=A[3][2]+	xar	$D[2],   $A[1][2],$D[2],#64-$rhotates[1][2] // D[2]=A[2][1]+	xar	$D[0],   $A[2][0],$D[0],#64-$rhotates[2][0] // D[0]=A[1][2]++	////////////////////////////////////////////////// Chi+Iota+	bcax	$A[4][0],$C[1],   $A[4][2],$A[1][3]	// A[1][3]=A[4][1]+	bcax	$A[4][1],$A[1][3],$A[4][3],$A[4][2]	// A[1][3]=A[4][1]+	bcax	$A[4][2],$A[4][2],$A[4][4],$A[4][3]+	bcax	$A[4][3],$A[4][3],$C[1],   $A[4][4]+	bcax	$A[4][4],$A[4][4],$A[1][3],$C[1]	// A[1][3]=A[4][1]++	ld1r	{$C[1]},[x10],#8++	bcax	$A[3][2],$D[1],   $A[3][4],$A[0][3]	// A[0][3]=A[3][3]+	bcax	$A[3][3],$A[0][3],$A[3][0],$A[3][4]	// A[0][3]=A[3][3]+	bcax	$A[3][4],$A[3][4],$A[3][1],$A[3][0]+	bcax	$A[3][0],$A[3][0],$D[1],   $A[3][1]+	bcax	$A[3][1],$A[3][1],$A[0][3],$D[1]	// A[0][3]=A[3][3]++	bcax	$A[2][0],$C[0],   $A[2][2],$D[2]+	bcax	$A[2][1],$D[2],   $A[2][3],$A[2][2]+	bcax	$A[2][2],$A[2][2],$A[2][4],$A[2][3]+	bcax	$A[2][3],$A[2][3],$C[0],   $A[2][4]+	bcax	$A[2][4],$A[2][4],$D[2],   $C[0]++	bcax	$A[1][2],$D[0],   $A[1][4],$A[0][4]	// A[0][4]=A[1][3]+	bcax	$A[1][3],$A[0][4],$A[1][0],$A[1][4]	// A[0][4]=A[1][3]+	bcax	$A[1][4],$A[1][4],$A[1][1],$A[1][0]+	bcax	$A[1][0],$A[1][0],$D[0],   $A[1][1]+	bcax	$A[1][1],$A[1][1],$A[0][4],$D[0]	// A[0][4]=A[1][3]++	bcax	$A[0][3],$D[3],   $A[0][0],$D[4]+	bcax	$A[0][4],$D[4],   $A[0][1],$A[0][0]+	bcax	$A[0][0],$A[0][0],$A[0][2],$A[0][1]+	bcax	$A[0][1],$A[0][1],$D[3],   $A[0][2]+	bcax	$A[0][2],$A[0][2],$D[4],   $D[3]++	eor	$A[0][0],$A[0][0],$C[1]++	tst	x10,#255+	bne	.Loop_ce++	ret+.size	KeccakF1600_ce,.-KeccakF1600_ce++.type	KeccakF1600_cext,%function+.align	5+KeccakF1600_cext:+	.inst	0xd503233f		// paciasp+	stp	c29,c30,[csp,#-2*__SIZEOF_POINTER__-64]!+	add	c29,csp,#0+	stp	d8,d9,[csp,#2*__SIZEOF_POINTER__+0]	// per ABI requirement+	stp	d10,d11,[csp,#2*__SIZEOF_POINTER__+16]+	stp	d12,d13,[csp,#2*__SIZEOF_POINTER__+32]+	stp	d14,d15,[csp,#2*__SIZEOF_POINTER__+48]+___+for($i=0; $i<24; $i+=2) {		# load A[5][5]+my $j=$i+1;+$code.=<<___;+	ldp	d$i,d$j,[x0,#8*$i]+___+}+$code.=<<___;+	ldr	d24,[x0,#8*$i]+	adr	x10,iotas+	bl	KeccakF1600_ce+	ldr	c30,[csp,#__SIZEOF_POINTER__]+___+for($i=0; $i<24; $i+=2) {		# store A[5][5]+my $j=$i+1;+$code.=<<___;+	stp	d$i,d$j,[x0,#8*$i]+___+}+$code.=<<___;+	str	d24,[x0,#8*$i]++	ldp	d8,d9,[csp,#2*__SIZEOF_POINTER__+0]+	ldp	d10,d11,[csp,#2*__SIZEOF_POINTER__+16]+	ldp	d12,d13,[csp,#2*__SIZEOF_POINTER__+32]+	ldp	d14,d15,[csp,#2*__SIZEOF_POINTER__+48]+	ldr	c29,[csp],#2*__SIZEOF_POINTER__+64+	.inst	0xd50323bf		// autiasp+	ret+.size	KeccakF1600_cext,.-KeccakF1600_cext+___++{+my ($ctx,$inp,$len,$bsz) = map("x$_",(0..3));++$code.=<<___;+.globl	SHA3_absorb_cext+.type	SHA3_absorb_cext,%function+.align	5+SHA3_absorb_cext:+	.inst	0xd503233f		// paciasp+	stp	c29,c30,[csp,#-2*__SIZEOF_POINTER__-64]!+	add	c29,csp,#0+	stp	d8,d9,[csp,#2*__SIZEOF_POINTER__+0]	// per ABI requirement+	stp	d10,d11,[csp,#2*__SIZEOF_POINTER__+16]+	stp	d12,d13,[csp,#2*__SIZEOF_POINTER__+32]+	stp	d14,d15,[csp,#2*__SIZEOF_POINTER__+48]+___+for($i=0; $i<24; $i+=2) {		# load A[5][5]+my $j=$i+1;+$code.=<<___;+	ldp	d$i,d$j,[x0,#8*$i]+___+}+$code.=<<___;+	ldr	d24,[x0,#8*$i]+	b	.Loop_absorb_ce++.align	4+.Loop_absorb_ce:+	subs	$len,$len,$bsz		// len - bsz+	blo	.Labsorbed_ce++	cmp	$bsz,#104+___+sub load_n_xor_ce {+    my ($from,$to) = @_;+    my $range = $to-$from+1;++    while ($range>=4) {+$code.=<<___;+	ld1	{v27.8b-v30.8b},[$inp],#32+	eor 	$A[$from/5][$from%5],$A[$from/5][$from++%5],v27.16b+	eor 	$A[$from/5][$from%5],$A[$from/5][$from++%5],v28.16b+	eor 	$A[$from/5][$from%5],$A[$from/5][$from++%5],v29.16b+	eor 	$A[$from/5][$from%5],$A[$from/5][$from++%5],v30.16b+___+	$range-=4;+    }+    while ($range>=3) {+$code.=<<___;+	ld1	{v28.8b-v30.8b},[$inp],#24+	eor 	$A[$from/5][$from%5],$A[$from/5][$from++%5],v28.16b+	eor 	$A[$from/5][$from%5],$A[$from/5][$from++%5],v29.16b+	eor 	$A[$from/5][$from%5],$A[$from/5][$from++%5],v30.16b+___+	$range-=3;+    }+    while ($from<=$to) {+$code.=<<___;+	ld1	{v31.8b},[$inp],#8	// A[`$from/5`][`$from%5`] ^= *inp+++	eor	$A[$from/5][$from%5],$A[$from/5][$from++%5],v31.16b+___+    }+}+load_n_xor_ce(0,8);+$code.=<<___;+	blo	.Lprocess_block_ce++___+load_n_xor_ce(9,12);+$code.=<<___;+	beq	.Lprocess_block_ce++	cmp	$bsz,#144+___+load_n_xor_ce(13,16);+$code.=<<___;+	blo	.Lprocess_block_ce++___+load_n_xor_ce(17,17);+$code.=<<___;+	beq	.Lprocess_block_ce++___+load_n_xor_ce(18,20);+$code.=<<___;++.Lprocess_block_ce:+	adr	x10,iotas+	bl	KeccakF1600_ce++	b	.Loop_absorb_ce++.align	4+.Labsorbed_ce:+___+for($i=0; $i<24; $i+=2) {		# store A[5][5]+my $j=$i+1;+$code.=<<___;+	stp	d$i,d$j,[x0,#8*$i]+___+}+$code.=<<___;+	str	d24,[x0,#8*$i]+	add	x0,$len,$bsz		// return value++	ldp	d8,d9,[csp,#2*__SIZEOF_POINTER__+0]+	ldp	d10,d11,[csp,#2*__SIZEOF_POINTER__+16]+	ldp	d12,d13,[csp,#2*__SIZEOF_POINTER__+32]+	ldp	d14,d15,[csp,#2*__SIZEOF_POINTER__+48]+	ldp	c29,c30,[csp],#2*__SIZEOF_POINTER__+64+	.inst	0xd50323bf		// autiasp+	ret+.size	SHA3_absorb_cext,.-SHA3_absorb_cext+___+}+{+my ($ctx,$out,$len,$bsz) = map("x$_",(0..3));+$code.=<<___;+.globl	SHA3_squeeze_cext+.type	SHA3_squeeze_cext,%function+.align	5+SHA3_squeeze_cext:+	.inst	0xd503233f		// paciasp+	stp	c29,c30,[csp,#-2*__SIZEOF_POINTER__]!+	add	c29,csp,#0+	cmov	x9,$ctx+	mov	x10,$bsz++.Loop_squeeze_ce:+	ldr	x4,[x9],#8+	cmp	$len,#8+	blo	.Lsqueeze_tail_ce+#ifdef	__AARCH64EB__+	rev	x4,x4+#endif+	str	x4,[$out],#8+	beq	.Lsqueeze_done_ce++	sub	$len,$len,#8+	subs	x10,x10,#8+	bhi	.Loop_squeeze_ce++	bl	KeccakF1600_cext+	ldr	c30,[csp,#__SIZEOF_POINTER__]+	cmov	x9,$ctx+	mov	x10,$bsz+	b	.Loop_squeeze_ce++.align	4+.Lsqueeze_tail_ce:+	strb	w4,[$out],#1+	lsr	x4,x4,#8+	subs	$len,$len,#1+	beq	.Lsqueeze_done_ce+	strb	w4,[$out],#1+	lsr	x4,x4,#8+	subs	$len,$len,#1+	beq	.Lsqueeze_done_ce+	strb	w4,[$out],#1+	lsr	x4,x4,#8+	subs	$len,$len,#1+	beq	.Lsqueeze_done_ce+	strb	w4,[$out],#1+	lsr	x4,x4,#8+	subs	$len,$len,#1+	beq	.Lsqueeze_done_ce+	strb	w4,[$out],#1+	lsr	x4,x4,#8+	subs	$len,$len,#1+	beq	.Lsqueeze_done_ce+	strb	w4,[$out],#1+	lsr	x4,x4,#8+	subs	$len,$len,#1+	beq	.Lsqueeze_done_ce+	strb	w4,[$out],#1++.Lsqueeze_done_ce:+	ldr	c29,[csp],#2*__SIZEOF_POINTER__+	.inst	0xd50323bf		// autiasp+	ret+.size	SHA3_squeeze_cext,.-SHA3_squeeze_cext+___+}								}}}+$code.=<<___;+.asciz	"Keccak-1600 absorb and squeeze for ARMv8, CRYPTOGAMS by \@dot-asm"+___++{   my  %opcode = (+	"rax1"	=> 0xce608c00,	"eor3"	=> 0xce000000,+	"bcax"	=> 0xce200000,	"xar"	=> 0xce800000	);++    sub unsha3 {+	my ($mnemonic,$arg)=@_;++	$arg =~ m/[qv]([0-9]+)[^,]*,\s*[qv]([0-9]+)[^,]*(?:,\s*[qv]([0-9]+)[^,]*(?:,\s*[qv#]([0-9\-]+))?)?/+	&&+	sprintf ".inst\t0x%08x\t//%s %s",+			$opcode{$mnemonic}|$1|($2<<5)|($3<<16)|(eval($4)<<10),+			$mnemonic,$arg;+    }+}++foreach(split("\n",$code)) {+	use integer;++	s/\`([^\`]*)\`/eval($1)/ge;++	m/\b(ld1r|rax1|xar)\b/ and s/\.16b/.2d/g;+	$sha3ops or s/\b(eor3|rax1|xar|bcax)\s+(v.*)/unsha3($1,$2)/ge;+	s/([cw])#x([0-9]+)/$1$2/g;++	print $_,"\n";+}++close STDOUT;
+ cbits/asm/keccak1600-x86_64-elf.S view
@@ -0,0 +1,538 @@+.text	++.type	__crypton_keccak_asm_f1600,@function+.align	32+__crypton_keccak_asm_f1600:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa++	movq	60(%rdi),%rax+	movq	68(%rdi),%rbx+	movq	76(%rdi),%rcx+	movq	84(%rdi),%rdx+	movq	92(%rdi),%rbp+	jmp	.Loop++.align	32+.Loop:+	movq	-100(%rdi),%r8+	movq	-52(%rdi),%r9+	movq	-4(%rdi),%r10+	movq	44(%rdi),%r11++	xorq	-84(%rdi),%rcx+	xorq	-76(%rdi),%rdx+	xorq	%r8,%rax+	xorq	-92(%rdi),%rbx+	xorq	-44(%rdi),%rcx+	xorq	-60(%rdi),%rax+	movq	%rbp,%r12+	xorq	-68(%rdi),%rbp++	xorq	%r10,%rcx+	xorq	-20(%rdi),%rax+	xorq	-36(%rdi),%rdx+	xorq	%r9,%rbx+	xorq	-28(%rdi),%rbp++	xorq	36(%rdi),%rcx+	xorq	20(%rdi),%rax+	xorq	4(%rdi),%rdx+	xorq	-12(%rdi),%rbx+	xorq	12(%rdi),%rbp++	movq	%rcx,%r13+	rolq	$1,%rcx+	xorq	%rax,%rcx+	xorq	%r11,%rdx++	rolq	$1,%rax+	xorq	%rdx,%rax+	xorq	28(%rdi),%rbx++	rolq	$1,%rdx+	xorq	%rbx,%rdx+	xorq	52(%rdi),%rbp++	rolq	$1,%rbx+	xorq	%rbp,%rbx++	rolq	$1,%rbp+	xorq	%r13,%rbp+	xorq	%rcx,%r9+	xorq	%rdx,%r10+	rolq	$44,%r9+	xorq	%rbp,%r11+	xorq	%rax,%r12+	rolq	$43,%r10+	xorq	%rbx,%r8+	movq	%r9,%r13+	rolq	$21,%r11+	orq	%r10,%r9+	xorq	%r8,%r9+	rolq	$14,%r12++	xorq	(%r15),%r9+	leaq	8(%r15),%r15++	movq	%r12,%r14+	andq	%r11,%r12+	movq	%r9,-100(%rsi)+	xorq	%r10,%r12+	notq	%r10+	movq	%r12,-84(%rsi)++	orq	%r11,%r10+	movq	76(%rdi),%r12+	xorq	%r13,%r10+	movq	%r10,-92(%rsi)++	andq	%r8,%r13+	movq	-28(%rdi),%r9+	xorq	%r14,%r13+	movq	-20(%rdi),%r10+	movq	%r13,-68(%rsi)++	orq	%r8,%r14+	movq	-76(%rdi),%r8+	xorq	%r11,%r14+	movq	28(%rdi),%r11+	movq	%r14,-76(%rsi)+++	xorq	%rbp,%r8+	xorq	%rdx,%r12+	rolq	$28,%r8+	xorq	%rcx,%r11+	xorq	%rax,%r9+	rolq	$61,%r12+	rolq	$45,%r11+	xorq	%rbx,%r10+	rolq	$20,%r9+	movq	%r8,%r13+	orq	%r12,%r8+	rolq	$3,%r10++	xorq	%r11,%r8+	movq	%r8,-36(%rsi)++	movq	%r9,%r14+	andq	%r13,%r9+	movq	-92(%rdi),%r8+	xorq	%r12,%r9+	notq	%r12+	movq	%r9,-28(%rsi)++	orq	%r11,%r12+	movq	-44(%rdi),%r9+	xorq	%r10,%r12+	movq	%r12,-44(%rsi)++	andq	%r10,%r11+	movq	60(%rdi),%r12+	xorq	%r14,%r11+	movq	%r11,-52(%rsi)++	orq	%r10,%r14+	movq	4(%rdi),%r10+	xorq	%r13,%r14+	movq	52(%rdi),%r11+	movq	%r14,-60(%rsi)+++	xorq	%rbp,%r10+	xorq	%rax,%r11+	rolq	$25,%r10+	xorq	%rdx,%r9+	rolq	$8,%r11+	xorq	%rbx,%r12+	rolq	$6,%r9+	xorq	%rcx,%r8+	rolq	$18,%r12+	movq	%r10,%r13+	andq	%r11,%r10+	rolq	$1,%r8++	notq	%r11+	xorq	%r9,%r10+	movq	%r10,-12(%rsi)++	movq	%r12,%r14+	andq	%r11,%r12+	movq	-12(%rdi),%r10+	xorq	%r13,%r12+	movq	%r12,-4(%rsi)++	orq	%r9,%r13+	movq	84(%rdi),%r12+	xorq	%r8,%r13+	movq	%r13,-20(%rsi)++	andq	%r8,%r9+	xorq	%r14,%r9+	movq	%r9,12(%rsi)++	orq	%r8,%r14+	movq	-60(%rdi),%r9+	xorq	%r11,%r14+	movq	36(%rdi),%r11+	movq	%r14,4(%rsi)+++	movq	-68(%rdi),%r8++	xorq	%rcx,%r10+	xorq	%rdx,%r11+	rolq	$10,%r10+	xorq	%rbx,%r9+	rolq	$15,%r11+	xorq	%rbp,%r12+	rolq	$36,%r9+	xorq	%rax,%r8+	rolq	$56,%r12+	movq	%r10,%r13+	orq	%r11,%r10+	rolq	$27,%r8++	notq	%r11+	xorq	%r9,%r10+	movq	%r10,28(%rsi)++	movq	%r12,%r14+	orq	%r11,%r12+	xorq	%r13,%r12+	movq	%r12,36(%rsi)++	andq	%r9,%r13+	xorq	%r8,%r13+	movq	%r13,20(%rsi)++	orq	%r8,%r9+	xorq	%r14,%r9+	movq	%r9,52(%rsi)++	andq	%r14,%r8+	xorq	%r11,%r8+	movq	%r8,44(%rsi)+++	xorq	-84(%rdi),%rdx+	xorq	-36(%rdi),%rbp+	rolq	$62,%rdx+	xorq	68(%rdi),%rcx+	rolq	$55,%rbp+	xorq	12(%rdi),%rax+	rolq	$2,%rcx+	xorq	20(%rdi),%rbx+	xchgq	%rsi,%rdi+	rolq	$39,%rax+	rolq	$41,%rbx+	movq	%rdx,%r13+	andq	%rbp,%rdx+	notq	%rbp+	xorq	%rcx,%rdx+	movq	%rdx,92(%rdi)++	movq	%rax,%r14+	andq	%rbp,%rax+	xorq	%r13,%rax+	movq	%rax,60(%rdi)++	orq	%rcx,%r13+	xorq	%rbx,%r13+	movq	%r13,84(%rdi)++	andq	%rbx,%rcx+	xorq	%r14,%rcx+	movq	%rcx,76(%rdi)++	orq	%r14,%rbx+	xorq	%rbp,%rbx+	movq	%rbx,68(%rdi)++	movq	%rdx,%rbp+	movq	%r13,%rdx++	testq	$255,%r15+	jnz	.Loop++	leaq	-192(%r15),%r15+	.byte	0xf3,0xc3+.cfi_endproc+.size	__crypton_keccak_asm_f1600,.-__crypton_keccak_asm_f1600++.globl	crypton_keccak_asm_f1600+.type	crypton_keccak_asm_f1600,@function+.align	32+crypton_keccak_asm_f1600:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56++	leaq	100(%rdi),%rdi+	subq	$200,%rsp+.cfi_adjust_cfa_offset	200+++	notq	-92(%rdi)+	notq	-84(%rdi)+	notq	-36(%rdi)+	notq	-4(%rdi)+	notq	36(%rdi)+	notq	60(%rdi)++	leaq	iotas(%rip),%r15+	leaq	100(%rsp),%rsi++	call	__crypton_keccak_asm_f1600++	notq	-92(%rdi)+	notq	-84(%rdi)+	notq	-36(%rdi)+	notq	-4(%rdi)+	notq	36(%rdi)+	notq	60(%rdi)+	leaq	-100(%rdi),%rdi++	leaq	248(%rsp),%r11+.cfi_def_cfa	%r11,8+	movq	-48(%r11),%r15+	movq	-40(%r11),%r14+	movq	-32(%r11),%r13+	movq	-24(%r11),%r12+	movq	-16(%r11),%rbp+	movq	-8(%r11),%rbx+	leaq	(%r11),%rsp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbp+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_keccak_asm_f1600,.-crypton_keccak_asm_f1600+.globl	crypton_keccak_asm_absorb+.type	crypton_keccak_asm_absorb,@function+.align	32+crypton_keccak_asm_absorb:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56++	leaq	100(%rdi),%rdi+	subq	$232,%rsp+.cfi_adjust_cfa_offset	232+++	movq	%rsi,%r9+	leaq	100(%rsp),%rsi++	notq	-92(%rdi)+	notq	-84(%rdi)+	notq	-36(%rdi)+	notq	-4(%rdi)+	notq	36(%rdi)+	notq	60(%rdi)+	leaq	iotas(%rip),%r15++	movq	%rcx,216-100(%rsi)++.Loop_absorb:+	cmpq	%rcx,%rdx+	jc	.Ldone_absorb++	shrq	$3,%rcx+	leaq	-100(%rdi),%r8++.Lblock_absorb:+	movq	(%r9),%rax+	leaq	8(%r9),%r9+	xorq	(%r8),%rax+	leaq	8(%r8),%r8+	subq	$8,%rdx+	movq	%rax,-8(%r8)+	subq	$1,%rcx+	jnz	.Lblock_absorb++	movq	%r9,200-100(%rsi)+	movq	%rdx,208-100(%rsi)+	call	__crypton_keccak_asm_f1600+	movq	200-100(%rsi),%r9+	movq	208-100(%rsi),%rdx+	movq	216-100(%rsi),%rcx+	jmp	.Loop_absorb++.align	32+.Ldone_absorb:+	movq	%rdx,%rax++	notq	-92(%rdi)+	notq	-84(%rdi)+	notq	-36(%rdi)+	notq	-4(%rdi)+	notq	36(%rdi)+	notq	60(%rdi)++	leaq	280(%rsp),%r11+.cfi_def_cfa	%r11,8+	movq	-48(%r11),%r15+	movq	-40(%r11),%r14+	movq	-32(%r11),%r13+	movq	-24(%r11),%r12+	movq	-16(%r11),%rbp+	movq	-8(%r11),%rbx+	leaq	(%r11),%rsp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbp+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_keccak_asm_absorb,.-crypton_keccak_asm_absorb+.globl	crypton_keccak_asm_squeeze+.type	crypton_keccak_asm_squeeze,@function+.align	32+crypton_keccak_asm_squeeze:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-16+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-24+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-32+	subq	$32,%rsp+.cfi_adjust_cfa_offset	32+++	shrq	$3,%rcx+	movq	%rdi,%r8+	movq	%rsi,%r12+	movq	%rdx,%r13+	movq	%rcx,%r14+	jmp	.Loop_squeeze++.align	32+.Loop_squeeze:+	cmpq	$8,%r13+	jb	.Ltail_squeeze++	movq	(%r8),%rax+	leaq	8(%r8),%r8+	movq	%rax,(%r12)+	leaq	8(%r12),%r12+	subq	$8,%r13+	jz	.Ldone_squeeze++	subq	$1,%rcx+	jnz	.Loop_squeeze++	movq	%rdi,%rcx+	call	crypton_keccak_asm_f1600+	movq	%rdi,%r8+	movq	%r14,%rcx+	jmp	.Loop_squeeze++.Ltail_squeeze:+	movq	%r8,%rsi+	movq	%r12,%rdi+	movq	%r13,%rcx+.byte	0xf3,0xa4++.Ldone_squeeze:+	movq	32(%rsp),%r14+	movq	40(%rsp),%r13+	movq	48(%rsp),%r12+	addq	$56,%rsp+.cfi_adjust_cfa_offset	-56+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_keccak_asm_squeeze,.-crypton_keccak_asm_squeeze+.align	256+.quad	0,0,0,0,0,0,0,0+.type	iotas,@object+iotas:+.quad	0x0000000000000001+.quad	0x0000000000008082+.quad	0x800000000000808a+.quad	0x8000000080008000+.quad	0x000000000000808b+.quad	0x0000000080000001+.quad	0x8000000080008081+.quad	0x8000000000008009+.quad	0x000000000000008a+.quad	0x0000000000000088+.quad	0x0000000080008009+.quad	0x000000008000000a+.quad	0x000000008000808b+.quad	0x800000000000008b+.quad	0x8000000000008089+.quad	0x8000000000008003+.quad	0x8000000000008002+.quad	0x8000000000000080+.quad	0x000000000000800a+.quad	0x800000008000000a+.quad	0x8000000080008081+.quad	0x8000000000008080+.quad	0x0000000080000001+.quad	0x8000000080008008+.size	iotas,.-iotas+.byte	75,101,99,99,97,107,45,49,54,48,48,32,97,98,115,111,114,98,32,97,110,100,32,115,113,117,101,101,122,101,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,60,97,112,112,114,111,64,111,112,101,110,115,115,108,46,111,114,103,62,0++.section	.note.gnu.property,"a",@note+	.long	4,2f-1f,5+	.byte	0x47,0x4E,0x55,0+1:	.long	0xc0000002,4,3+.align	8+2:++.section	.note.GNU-stack,"",@progbits
+ cbits/asm/keccak1600-x86_64-macosx.S view
@@ -0,0 +1,529 @@+.text	+++.p2align	5+__crypton_keccak_asm_f1600:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa++	movq	60(%rdi),%rax+	movq	68(%rdi),%rbx+	movq	76(%rdi),%rcx+	movq	84(%rdi),%rdx+	movq	92(%rdi),%rbp+	jmp	L$oop++.p2align	5+L$oop:+	movq	-100(%rdi),%r8+	movq	-52(%rdi),%r9+	movq	-4(%rdi),%r10+	movq	44(%rdi),%r11++	xorq	-84(%rdi),%rcx+	xorq	-76(%rdi),%rdx+	xorq	%r8,%rax+	xorq	-92(%rdi),%rbx+	xorq	-44(%rdi),%rcx+	xorq	-60(%rdi),%rax+	movq	%rbp,%r12+	xorq	-68(%rdi),%rbp++	xorq	%r10,%rcx+	xorq	-20(%rdi),%rax+	xorq	-36(%rdi),%rdx+	xorq	%r9,%rbx+	xorq	-28(%rdi),%rbp++	xorq	36(%rdi),%rcx+	xorq	20(%rdi),%rax+	xorq	4(%rdi),%rdx+	xorq	-12(%rdi),%rbx+	xorq	12(%rdi),%rbp++	movq	%rcx,%r13+	rolq	$1,%rcx+	xorq	%rax,%rcx+	xorq	%r11,%rdx++	rolq	$1,%rax+	xorq	%rdx,%rax+	xorq	28(%rdi),%rbx++	rolq	$1,%rdx+	xorq	%rbx,%rdx+	xorq	52(%rdi),%rbp++	rolq	$1,%rbx+	xorq	%rbp,%rbx++	rolq	$1,%rbp+	xorq	%r13,%rbp+	xorq	%rcx,%r9+	xorq	%rdx,%r10+	rolq	$44,%r9+	xorq	%rbp,%r11+	xorq	%rax,%r12+	rolq	$43,%r10+	xorq	%rbx,%r8+	movq	%r9,%r13+	rolq	$21,%r11+	orq	%r10,%r9+	xorq	%r8,%r9+	rolq	$14,%r12++	xorq	(%r15),%r9+	leaq	8(%r15),%r15++	movq	%r12,%r14+	andq	%r11,%r12+	movq	%r9,-100(%rsi)+	xorq	%r10,%r12+	notq	%r10+	movq	%r12,-84(%rsi)++	orq	%r11,%r10+	movq	76(%rdi),%r12+	xorq	%r13,%r10+	movq	%r10,-92(%rsi)++	andq	%r8,%r13+	movq	-28(%rdi),%r9+	xorq	%r14,%r13+	movq	-20(%rdi),%r10+	movq	%r13,-68(%rsi)++	orq	%r8,%r14+	movq	-76(%rdi),%r8+	xorq	%r11,%r14+	movq	28(%rdi),%r11+	movq	%r14,-76(%rsi)+++	xorq	%rbp,%r8+	xorq	%rdx,%r12+	rolq	$28,%r8+	xorq	%rcx,%r11+	xorq	%rax,%r9+	rolq	$61,%r12+	rolq	$45,%r11+	xorq	%rbx,%r10+	rolq	$20,%r9+	movq	%r8,%r13+	orq	%r12,%r8+	rolq	$3,%r10++	xorq	%r11,%r8+	movq	%r8,-36(%rsi)++	movq	%r9,%r14+	andq	%r13,%r9+	movq	-92(%rdi),%r8+	xorq	%r12,%r9+	notq	%r12+	movq	%r9,-28(%rsi)++	orq	%r11,%r12+	movq	-44(%rdi),%r9+	xorq	%r10,%r12+	movq	%r12,-44(%rsi)++	andq	%r10,%r11+	movq	60(%rdi),%r12+	xorq	%r14,%r11+	movq	%r11,-52(%rsi)++	orq	%r10,%r14+	movq	4(%rdi),%r10+	xorq	%r13,%r14+	movq	52(%rdi),%r11+	movq	%r14,-60(%rsi)+++	xorq	%rbp,%r10+	xorq	%rax,%r11+	rolq	$25,%r10+	xorq	%rdx,%r9+	rolq	$8,%r11+	xorq	%rbx,%r12+	rolq	$6,%r9+	xorq	%rcx,%r8+	rolq	$18,%r12+	movq	%r10,%r13+	andq	%r11,%r10+	rolq	$1,%r8++	notq	%r11+	xorq	%r9,%r10+	movq	%r10,-12(%rsi)++	movq	%r12,%r14+	andq	%r11,%r12+	movq	-12(%rdi),%r10+	xorq	%r13,%r12+	movq	%r12,-4(%rsi)++	orq	%r9,%r13+	movq	84(%rdi),%r12+	xorq	%r8,%r13+	movq	%r13,-20(%rsi)++	andq	%r8,%r9+	xorq	%r14,%r9+	movq	%r9,12(%rsi)++	orq	%r8,%r14+	movq	-60(%rdi),%r9+	xorq	%r11,%r14+	movq	36(%rdi),%r11+	movq	%r14,4(%rsi)+++	movq	-68(%rdi),%r8++	xorq	%rcx,%r10+	xorq	%rdx,%r11+	rolq	$10,%r10+	xorq	%rbx,%r9+	rolq	$15,%r11+	xorq	%rbp,%r12+	rolq	$36,%r9+	xorq	%rax,%r8+	rolq	$56,%r12+	movq	%r10,%r13+	orq	%r11,%r10+	rolq	$27,%r8++	notq	%r11+	xorq	%r9,%r10+	movq	%r10,28(%rsi)++	movq	%r12,%r14+	orq	%r11,%r12+	xorq	%r13,%r12+	movq	%r12,36(%rsi)++	andq	%r9,%r13+	xorq	%r8,%r13+	movq	%r13,20(%rsi)++	orq	%r8,%r9+	xorq	%r14,%r9+	movq	%r9,52(%rsi)++	andq	%r14,%r8+	xorq	%r11,%r8+	movq	%r8,44(%rsi)+++	xorq	-84(%rdi),%rdx+	xorq	-36(%rdi),%rbp+	rolq	$62,%rdx+	xorq	68(%rdi),%rcx+	rolq	$55,%rbp+	xorq	12(%rdi),%rax+	rolq	$2,%rcx+	xorq	20(%rdi),%rbx+	xchgq	%rsi,%rdi+	rolq	$39,%rax+	rolq	$41,%rbx+	movq	%rdx,%r13+	andq	%rbp,%rdx+	notq	%rbp+	xorq	%rcx,%rdx+	movq	%rdx,92(%rdi)++	movq	%rax,%r14+	andq	%rbp,%rax+	xorq	%r13,%rax+	movq	%rax,60(%rdi)++	orq	%rcx,%r13+	xorq	%rbx,%r13+	movq	%r13,84(%rdi)++	andq	%rbx,%rcx+	xorq	%r14,%rcx+	movq	%rcx,76(%rdi)++	orq	%r14,%rbx+	xorq	%rbp,%rbx+	movq	%rbx,68(%rdi)++	movq	%rdx,%rbp+	movq	%r13,%rdx++	testq	$255,%r15+	jnz	L$oop++	leaq	-192(%r15),%r15+	.byte	0xf3,0xc3+.cfi_endproc+++.globl	_crypton_keccak_asm_f1600++.p2align	5+_crypton_keccak_asm_f1600:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56++	leaq	100(%rdi),%rdi+	subq	$200,%rsp+.cfi_adjust_cfa_offset	200+++	notq	-92(%rdi)+	notq	-84(%rdi)+	notq	-36(%rdi)+	notq	-4(%rdi)+	notq	36(%rdi)+	notq	60(%rdi)++	leaq	iotas(%rip),%r15+	leaq	100(%rsp),%rsi++	call	__crypton_keccak_asm_f1600++	notq	-92(%rdi)+	notq	-84(%rdi)+	notq	-36(%rdi)+	notq	-4(%rdi)+	notq	36(%rdi)+	notq	60(%rdi)+	leaq	-100(%rdi),%rdi++	leaq	248(%rsp),%r11+.cfi_def_cfa	%r11,8+	movq	-48(%r11),%r15+	movq	-40(%r11),%r14+	movq	-32(%r11),%r13+	movq	-24(%r11),%r12+	movq	-16(%r11),%rbp+	movq	-8(%r11),%rbx+	leaq	(%r11),%rsp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbp+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	++.globl	_crypton_keccak_asm_absorb++.p2align	5+_crypton_keccak_asm_absorb:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56++	leaq	100(%rdi),%rdi+	subq	$232,%rsp+.cfi_adjust_cfa_offset	232+++	movq	%rsi,%r9+	leaq	100(%rsp),%rsi++	notq	-92(%rdi)+	notq	-84(%rdi)+	notq	-36(%rdi)+	notq	-4(%rdi)+	notq	36(%rdi)+	notq	60(%rdi)+	leaq	iotas(%rip),%r15++	movq	%rcx,216-100(%rsi)++L$oop_absorb:+	cmpq	%rcx,%rdx+	jc	L$done_absorb++	shrq	$3,%rcx+	leaq	-100(%rdi),%r8++L$block_absorb:+	movq	(%r9),%rax+	leaq	8(%r9),%r9+	xorq	(%r8),%rax+	leaq	8(%r8),%r8+	subq	$8,%rdx+	movq	%rax,-8(%r8)+	subq	$1,%rcx+	jnz	L$block_absorb++	movq	%r9,200-100(%rsi)+	movq	%rdx,208-100(%rsi)+	call	__crypton_keccak_asm_f1600+	movq	200-100(%rsi),%r9+	movq	208-100(%rsi),%rdx+	movq	216-100(%rsi),%rcx+	jmp	L$oop_absorb++.p2align	5+L$done_absorb:+	movq	%rdx,%rax++	notq	-92(%rdi)+	notq	-84(%rdi)+	notq	-36(%rdi)+	notq	-4(%rdi)+	notq	36(%rdi)+	notq	60(%rdi)++	leaq	280(%rsp),%r11+.cfi_def_cfa	%r11,8+	movq	-48(%r11),%r15+	movq	-40(%r11),%r14+	movq	-32(%r11),%r13+	movq	-24(%r11),%r12+	movq	-16(%r11),%rbp+	movq	-8(%r11),%rbx+	leaq	(%r11),%rsp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbp+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	++.globl	_crypton_keccak_asm_squeeze++.p2align	5+_crypton_keccak_asm_squeeze:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-16+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-24+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-32+	subq	$32,%rsp+.cfi_adjust_cfa_offset	32+++	shrq	$3,%rcx+	movq	%rdi,%r8+	movq	%rsi,%r12+	movq	%rdx,%r13+	movq	%rcx,%r14+	jmp	L$oop_squeeze++.p2align	5+L$oop_squeeze:+	cmpq	$8,%r13+	jb	L$tail_squeeze++	movq	(%r8),%rax+	leaq	8(%r8),%r8+	movq	%rax,(%r12)+	leaq	8(%r12),%r12+	subq	$8,%r13+	jz	L$done_squeeze++	subq	$1,%rcx+	jnz	L$oop_squeeze++	movq	%rdi,%rcx+	call	_crypton_keccak_asm_f1600+	movq	%rdi,%r8+	movq	%r14,%rcx+	jmp	L$oop_squeeze++L$tail_squeeze:+	movq	%r8,%rsi+	movq	%r12,%rdi+	movq	%r13,%rcx+.byte	0xf3,0xa4++L$done_squeeze:+	movq	32(%rsp),%r14+	movq	40(%rsp),%r13+	movq	48(%rsp),%r12+	addq	$56,%rsp+.cfi_adjust_cfa_offset	-56+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+	.byte	0xf3,0xc3+.cfi_endproc	++.p2align	8+.quad	0,0,0,0,0,0,0,0++iotas:+.quad	0x0000000000000001+.quad	0x0000000000008082+.quad	0x800000000000808a+.quad	0x8000000080008000+.quad	0x000000000000808b+.quad	0x0000000080000001+.quad	0x8000000080008081+.quad	0x8000000000008009+.quad	0x000000000000008a+.quad	0x0000000000000088+.quad	0x0000000080008009+.quad	0x000000008000000a+.quad	0x000000008000808b+.quad	0x800000000000008b+.quad	0x8000000000008089+.quad	0x8000000000008003+.quad	0x8000000000008002+.quad	0x8000000000000080+.quad	0x000000000000800a+.quad	0x800000008000000a+.quad	0x8000000080008081+.quad	0x8000000000008080+.quad	0x0000000080000001+.quad	0x8000000080008008++.byte	75,101,99,99,97,107,45,49,54,48,48,32,97,98,115,111,114,98,32,97,110,100,32,115,113,117,101,101,122,101,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,60,97,112,112,114,111,64,111,112,101,110,115,115,108,46,111,114,103,62,0
+ cbits/asm/keccak1600-x86_64-mingw64.S view
@@ -0,0 +1,648 @@+.text	++.def	__crypton_keccak_asm_f1600;	.scl 3;	.type 32;	.endef+.p2align	5+__crypton_keccak_asm_f1600:+	.byte	0xf3,0x0f,0x1e,0xfa++	movq	60(%rdi),%rax+	movq	68(%rdi),%rbx+	movq	76(%rdi),%rcx+	movq	84(%rdi),%rdx+	movq	92(%rdi),%rbp+	jmp	.Loop++.p2align	5+.Loop:+	movq	-100(%rdi),%r8+	movq	-52(%rdi),%r9+	movq	-4(%rdi),%r10+	movq	44(%rdi),%r11++	xorq	-84(%rdi),%rcx+	xorq	-76(%rdi),%rdx+	xorq	%r8,%rax+	xorq	-92(%rdi),%rbx+	xorq	-44(%rdi),%rcx+	xorq	-60(%rdi),%rax+	movq	%rbp,%r12+	xorq	-68(%rdi),%rbp++	xorq	%r10,%rcx+	xorq	-20(%rdi),%rax+	xorq	-36(%rdi),%rdx+	xorq	%r9,%rbx+	xorq	-28(%rdi),%rbp++	xorq	36(%rdi),%rcx+	xorq	20(%rdi),%rax+	xorq	4(%rdi),%rdx+	xorq	-12(%rdi),%rbx+	xorq	12(%rdi),%rbp++	movq	%rcx,%r13+	rolq	$1,%rcx+	xorq	%rax,%rcx+	xorq	%r11,%rdx++	rolq	$1,%rax+	xorq	%rdx,%rax+	xorq	28(%rdi),%rbx++	rolq	$1,%rdx+	xorq	%rbx,%rdx+	xorq	52(%rdi),%rbp++	rolq	$1,%rbx+	xorq	%rbp,%rbx++	rolq	$1,%rbp+	xorq	%r13,%rbp+	xorq	%rcx,%r9+	xorq	%rdx,%r10+	rolq	$44,%r9+	xorq	%rbp,%r11+	xorq	%rax,%r12+	rolq	$43,%r10+	xorq	%rbx,%r8+	movq	%r9,%r13+	rolq	$21,%r11+	orq	%r10,%r9+	xorq	%r8,%r9+	rolq	$14,%r12++	xorq	(%r15),%r9+	leaq	8(%r15),%r15++	movq	%r12,%r14+	andq	%r11,%r12+	movq	%r9,-100(%rsi)+	xorq	%r10,%r12+	notq	%r10+	movq	%r12,-84(%rsi)++	orq	%r11,%r10+	movq	76(%rdi),%r12+	xorq	%r13,%r10+	movq	%r10,-92(%rsi)++	andq	%r8,%r13+	movq	-28(%rdi),%r9+	xorq	%r14,%r13+	movq	-20(%rdi),%r10+	movq	%r13,-68(%rsi)++	orq	%r8,%r14+	movq	-76(%rdi),%r8+	xorq	%r11,%r14+	movq	28(%rdi),%r11+	movq	%r14,-76(%rsi)+++	xorq	%rbp,%r8+	xorq	%rdx,%r12+	rolq	$28,%r8+	xorq	%rcx,%r11+	xorq	%rax,%r9+	rolq	$61,%r12+	rolq	$45,%r11+	xorq	%rbx,%r10+	rolq	$20,%r9+	movq	%r8,%r13+	orq	%r12,%r8+	rolq	$3,%r10++	xorq	%r11,%r8+	movq	%r8,-36(%rsi)++	movq	%r9,%r14+	andq	%r13,%r9+	movq	-92(%rdi),%r8+	xorq	%r12,%r9+	notq	%r12+	movq	%r9,-28(%rsi)++	orq	%r11,%r12+	movq	-44(%rdi),%r9+	xorq	%r10,%r12+	movq	%r12,-44(%rsi)++	andq	%r10,%r11+	movq	60(%rdi),%r12+	xorq	%r14,%r11+	movq	%r11,-52(%rsi)++	orq	%r10,%r14+	movq	4(%rdi),%r10+	xorq	%r13,%r14+	movq	52(%rdi),%r11+	movq	%r14,-60(%rsi)+++	xorq	%rbp,%r10+	xorq	%rax,%r11+	rolq	$25,%r10+	xorq	%rdx,%r9+	rolq	$8,%r11+	xorq	%rbx,%r12+	rolq	$6,%r9+	xorq	%rcx,%r8+	rolq	$18,%r12+	movq	%r10,%r13+	andq	%r11,%r10+	rolq	$1,%r8++	notq	%r11+	xorq	%r9,%r10+	movq	%r10,-12(%rsi)++	movq	%r12,%r14+	andq	%r11,%r12+	movq	-12(%rdi),%r10+	xorq	%r13,%r12+	movq	%r12,-4(%rsi)++	orq	%r9,%r13+	movq	84(%rdi),%r12+	xorq	%r8,%r13+	movq	%r13,-20(%rsi)++	andq	%r8,%r9+	xorq	%r14,%r9+	movq	%r9,12(%rsi)++	orq	%r8,%r14+	movq	-60(%rdi),%r9+	xorq	%r11,%r14+	movq	36(%rdi),%r11+	movq	%r14,4(%rsi)+++	movq	-68(%rdi),%r8++	xorq	%rcx,%r10+	xorq	%rdx,%r11+	rolq	$10,%r10+	xorq	%rbx,%r9+	rolq	$15,%r11+	xorq	%rbp,%r12+	rolq	$36,%r9+	xorq	%rax,%r8+	rolq	$56,%r12+	movq	%r10,%r13+	orq	%r11,%r10+	rolq	$27,%r8++	notq	%r11+	xorq	%r9,%r10+	movq	%r10,28(%rsi)++	movq	%r12,%r14+	orq	%r11,%r12+	xorq	%r13,%r12+	movq	%r12,36(%rsi)++	andq	%r9,%r13+	xorq	%r8,%r13+	movq	%r13,20(%rsi)++	orq	%r8,%r9+	xorq	%r14,%r9+	movq	%r9,52(%rsi)++	andq	%r14,%r8+	xorq	%r11,%r8+	movq	%r8,44(%rsi)+++	xorq	-84(%rdi),%rdx+	xorq	-36(%rdi),%rbp+	rolq	$62,%rdx+	xorq	68(%rdi),%rcx+	rolq	$55,%rbp+	xorq	12(%rdi),%rax+	rolq	$2,%rcx+	xorq	20(%rdi),%rbx+	xchgq	%rsi,%rdi+	rolq	$39,%rax+	rolq	$41,%rbx+	movq	%rdx,%r13+	andq	%rbp,%rdx+	notq	%rbp+	xorq	%rcx,%rdx+	movq	%rdx,92(%rdi)++	movq	%rax,%r14+	andq	%rbp,%rax+	xorq	%r13,%rax+	movq	%rax,60(%rdi)++	orq	%rcx,%r13+	xorq	%rbx,%r13+	movq	%r13,84(%rdi)++	andq	%rbx,%rcx+	xorq	%r14,%rcx+	movq	%rcx,76(%rdi)++	orq	%r14,%rbx+	xorq	%rbp,%rbx+	movq	%rbx,68(%rdi)++	movq	%rdx,%rbp+	movq	%r13,%rdx++	testq	$255,%r15+	jnz	.Loop++	leaq	-192(%r15),%r15+	.byte	0xf3,0xc3+++.globl	crypton_keccak_asm_f1600+.def	crypton_keccak_asm_f1600;	.scl 2;	.type 32;	.endef+.p2align	5+crypton_keccak_asm_f1600:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_keccak_asm_f1600:+++	movq	%rcx,%rdi+	pushq	%rbx++	pushq	%rbp++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15+++	leaq	100(%rdi),%rdi+	subq	$200,%rsp++.LSEH_body_crypton_keccak_asm_f1600:+++	notq	-92(%rdi)+	notq	-84(%rdi)+	notq	-36(%rdi)+	notq	-4(%rdi)+	notq	36(%rdi)+	notq	60(%rdi)++	leaq	iotas(%rip),%r15+	leaq	100(%rsp),%rsi++	call	__crypton_keccak_asm_f1600++	notq	-92(%rdi)+	notq	-84(%rdi)+	notq	-36(%rdi)+	notq	-4(%rdi)+	notq	36(%rdi)+	notq	60(%rdi)+	leaq	-100(%rdi),%rdi++	leaq	248(%rsp),%r11++	movq	-48(%r11),%r15+	movq	-40(%r11),%r14+	movq	-32(%r11),%r13+	movq	-24(%r11),%r12+	movq	-16(%r11),%rbp+	movq	-8(%r11),%rbx+	leaq	(%r11),%rsp+.LSEH_epilogue_crypton_keccak_asm_f1600:+	mov	8(%r11),%rdi+	mov	16(%r11),%rsi++	.byte	0xf3,0xc3++.LSEH_end_crypton_keccak_asm_f1600:+.globl	crypton_keccak_asm_absorb+.def	crypton_keccak_asm_absorb;	.scl 2;	.type 32;	.endef+.p2align	5+crypton_keccak_asm_absorb:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_keccak_asm_absorb:+++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	movq	%r9,%rcx+	pushq	%rbx++	pushq	%rbp++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15+++	leaq	100(%rdi),%rdi+	subq	$232,%rsp++.LSEH_body_crypton_keccak_asm_absorb:+++	movq	%rsi,%r9+	leaq	100(%rsp),%rsi++	notq	-92(%rdi)+	notq	-84(%rdi)+	notq	-36(%rdi)+	notq	-4(%rdi)+	notq	36(%rdi)+	notq	60(%rdi)+	leaq	iotas(%rip),%r15++	movq	%rcx,216-100(%rsi)++.Loop_absorb:+	cmpq	%rcx,%rdx+	jc	.Ldone_absorb++	shrq	$3,%rcx+	leaq	-100(%rdi),%r8++.Lblock_absorb:+	movq	(%r9),%rax+	leaq	8(%r9),%r9+	xorq	(%r8),%rax+	leaq	8(%r8),%r8+	subq	$8,%rdx+	movq	%rax,-8(%r8)+	subq	$1,%rcx+	jnz	.Lblock_absorb++	movq	%r9,200-100(%rsi)+	movq	%rdx,208-100(%rsi)+	call	__crypton_keccak_asm_f1600+	movq	200-100(%rsi),%r9+	movq	208-100(%rsi),%rdx+	movq	216-100(%rsi),%rcx+	jmp	.Loop_absorb++.p2align	5+.Ldone_absorb:+	movq	%rdx,%rax++	notq	-92(%rdi)+	notq	-84(%rdi)+	notq	-36(%rdi)+	notq	-4(%rdi)+	notq	36(%rdi)+	notq	60(%rdi)++	leaq	280(%rsp),%r11++	movq	-48(%r11),%r15+	movq	-40(%r11),%r14+	movq	-32(%r11),%r13+	movq	-24(%r11),%r12+	movq	-16(%r11),%rbp+	movq	-8(%r11),%rbx+	leaq	(%r11),%rsp+.LSEH_epilogue_crypton_keccak_asm_absorb:+	mov	8(%r11),%rdi+	mov	16(%r11),%rsi++	.byte	0xf3,0xc3++.LSEH_end_crypton_keccak_asm_absorb:+.globl	crypton_keccak_asm_squeeze+.def	crypton_keccak_asm_squeeze;	.scl 2;	.type 32;	.endef+.p2align	5+crypton_keccak_asm_squeeze:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_keccak_asm_squeeze:+++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	movq	%r9,%rcx+	pushq	%r12++	pushq	%r13++	pushq	%r14++	subq	$32,%rsp++.LSEH_body_crypton_keccak_asm_squeeze:+++	shrq	$3,%rcx+	movq	%rdi,%r8+	movq	%rsi,%r12+	movq	%rdx,%r13+	movq	%rcx,%r14+	jmp	.Loop_squeeze++.p2align	5+.Loop_squeeze:+	cmpq	$8,%r13+	jb	.Ltail_squeeze++	movq	(%r8),%rax+	leaq	8(%r8),%r8+	movq	%rax,(%r12)+	leaq	8(%r12),%r12+	subq	$8,%r13+	jz	.Ldone_squeeze++	subq	$1,%rcx+	jnz	.Loop_squeeze++	movq	%rdi,%rcx+	call	crypton_keccak_asm_f1600+	movq	%rdi,%r8+	movq	%r14,%rcx+	jmp	.Loop_squeeze++.Ltail_squeeze:+	movq	%r8,%rsi+	movq	%r12,%rdi+	movq	%r13,%rcx+.byte	0xf3,0xa4++.Ldone_squeeze:+	movq	32(%rsp),%r14+	movq	40(%rsp),%r13+	movq	48(%rsp),%r12+	addq	$56,%rsp++.LSEH_epilogue_crypton_keccak_asm_squeeze:+	mov	8(%rsp),%rdi+	mov	16(%rsp),%rsi++	.byte	0xf3,0xc3++.LSEH_end_crypton_keccak_asm_squeeze:+.p2align	8+.quad	0,0,0,0,0,0,0,0++iotas:+.quad	0x0000000000000001+.quad	0x0000000000008082+.quad	0x800000000000808a+.quad	0x8000000080008000+.quad	0x000000000000808b+.quad	0x0000000080000001+.quad	0x8000000080008081+.quad	0x8000000000008009+.quad	0x000000000000008a+.quad	0x0000000000000088+.quad	0x0000000080008009+.quad	0x000000008000000a+.quad	0x000000008000808b+.quad	0x800000000000008b+.quad	0x8000000000008089+.quad	0x8000000000008003+.quad	0x8000000000008002+.quad	0x8000000000000080+.quad	0x000000000000800a+.quad	0x800000008000000a+.quad	0x8000000080008081+.quad	0x8000000000008080+.quad	0x0000000080000001+.quad	0x8000000080008008++.byte	75,101,99,99,97,107,45,49,54,48,48,32,97,98,115,111,114,98,32,97,110,100,32,115,113,117,101,101,122,101,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,60,97,112,112,114,111,64,111,112,101,110,115,115,108,46,111,114,103,62,0+.section	.pdata+.p2align	2+.rva	.LSEH_begin_crypton_keccak_asm_f1600+.rva	.LSEH_body_crypton_keccak_asm_f1600+.rva	.LSEH_info_crypton_keccak_asm_f1600_prologue++.rva	.LSEH_body_crypton_keccak_asm_f1600+.rva	.LSEH_epilogue_crypton_keccak_asm_f1600+.rva	.LSEH_info_crypton_keccak_asm_f1600_body++.rva	.LSEH_epilogue_crypton_keccak_asm_f1600+.rva	.LSEH_end_crypton_keccak_asm_f1600+.rva	.LSEH_info_crypton_keccak_asm_f1600_epilogue++.rva	.LSEH_begin_crypton_keccak_asm_absorb+.rva	.LSEH_body_crypton_keccak_asm_absorb+.rva	.LSEH_info_crypton_keccak_asm_absorb_prologue++.rva	.LSEH_body_crypton_keccak_asm_absorb+.rva	.LSEH_epilogue_crypton_keccak_asm_absorb+.rva	.LSEH_info_crypton_keccak_asm_absorb_body++.rva	.LSEH_epilogue_crypton_keccak_asm_absorb+.rva	.LSEH_end_crypton_keccak_asm_absorb+.rva	.LSEH_info_crypton_keccak_asm_absorb_epilogue++.rva	.LSEH_begin_crypton_keccak_asm_squeeze+.rva	.LSEH_body_crypton_keccak_asm_squeeze+.rva	.LSEH_info_crypton_keccak_asm_squeeze_prologue++.rva	.LSEH_body_crypton_keccak_asm_squeeze+.rva	.LSEH_epilogue_crypton_keccak_asm_squeeze+.rva	.LSEH_info_crypton_keccak_asm_squeeze_body++.rva	.LSEH_epilogue_crypton_keccak_asm_squeeze+.rva	.LSEH_end_crypton_keccak_asm_squeeze+.rva	.LSEH_info_crypton_keccak_asm_squeeze_epilogue++.section	.xdata+.p2align	3+.LSEH_info_crypton_keccak_asm_f1600_prologue:+.byte	1,0,5,0x0b+.byte	0,0x74,1,0+.byte	0,0x64,2,0+.byte	0,0xb3+.byte	0,0+.long	0,0+.LSEH_info_crypton_keccak_asm_f1600_body:+.byte	1,0,18,0+.byte	0x00,0xf4,0x19,0x00+.byte	0x00,0xe4,0x1a,0x00+.byte	0x00,0xd4,0x1b,0x00+.byte	0x00,0xc4,0x1c,0x00+.byte	0x00,0x54,0x1d,0x00+.byte	0x00,0x34,0x1e,0x00+.byte	0x00,0x74,0x20,0x00+.byte	0x00,0x64,0x21,0x00+.byte	0x00,0x01,0x1f,0x00+.byte	0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00+.LSEH_info_crypton_keccak_asm_f1600_epilogue:+.byte	1,0,5,11+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0xb3+.byte	0x00,0x00,0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00++.LSEH_info_crypton_keccak_asm_absorb_prologue:+.byte	1,0,5,0x0b+.byte	0,0x74,1,0+.byte	0,0x64,2,0+.byte	0,0xb3+.byte	0,0+.long	0,0+.LSEH_info_crypton_keccak_asm_absorb_body:+.byte	1,0,18,0+.byte	0x00,0xf4,0x1d,0x00+.byte	0x00,0xe4,0x1e,0x00+.byte	0x00,0xd4,0x1f,0x00+.byte	0x00,0xc4,0x20,0x00+.byte	0x00,0x54,0x21,0x00+.byte	0x00,0x34,0x22,0x00+.byte	0x00,0x74,0x24,0x00+.byte	0x00,0x64,0x25,0x00+.byte	0x00,0x01,0x23,0x00+.byte	0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00+.LSEH_info_crypton_keccak_asm_absorb_epilogue:+.byte	1,0,5,11+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0xb3+.byte	0x00,0x00,0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00++.LSEH_info_crypton_keccak_asm_squeeze_prologue:+.byte	1,0,5,0x0b+.byte	0,0x74,1,0+.byte	0,0x64,2,0+.byte	0,0xb3+.byte	0,0+.long	0,0+.LSEH_info_crypton_keccak_asm_squeeze_body:+.byte	1,0,11,0+.byte	0x00,0xe4,0x04,0x00+.byte	0x00,0xd4,0x05,0x00+.byte	0x00,0xc4,0x06,0x00+.byte	0x00,0x74,0x08,0x00+.byte	0x00,0x64,0x09,0x00+.byte	0x00,0x62+.byte	0x00,0x00,0x00,0x00,0x00,0x00+.LSEH_info_crypton_keccak_asm_squeeze_epilogue:+.byte	1,0,4,0+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0x00,0x00,0x00+
+ cbits/asm/keccak1600-x86_64.pl view
@@ -0,0 +1,601 @@+#!/usr/bin/env perl+#+# ====================================================================+# Written by Andy Polyakov <appro@openssl.org> for the OpenSSL+# project. The module is, however, dual licensed under OpenSSL and+# CRYPTOGAMS licenses depending on where you obtain it. For further+# details see http://www.openssl.org/~appro/cryptogams/.+# ====================================================================+#+# Keccak-1600 for x86_64.+#+# June 2017.+#+# Below code is [lane complementing] KECCAK_2X implementation (see+# sha/keccak1600.c) with C[5] and D[5] held in register bank. Though+# instead of actually unrolling the loop pair-wise I simply flip+# pointers to T[][] and A[][] at the end of round. Since number of+# rounds is even, last round writes to A[][] and everything works out.+# How does it compare to x86_64 assembly module in Keccak Code Package?+# Depending on processor it's either as fast or faster by up to 15%...+#+########################################################################+# Numbers are cycles per processed byte out of large message.+#+#			r=1088(*)+#+# P4			25.8+# Core 2		12.9+# Westmere		13.7+# Sandy Bridge		12.9(**)+# Haswell		9.6+# Skylake		9.4+# Ice Lake		8.6+# Silvermont		22.8+# Goldmont		15.8+# VIA Nano		17.3+# Sledgehammer		13.3+# Bulldozer		16.5+# Ryzen			8.8+# Zen 4			7.6+#+# (*)	Corresponds to SHA3-256. Improvement over compiler-generate+#	varies a lot, most commont coefficient is 15% in comparison to+#	gcc-5.x, 50% for gcc-4.x, 90% for gcc-3.x.+# (**)	Sandy Bridge has broken rotate instruction. Performance can be+#	improved by 14% by replacing rotates with double-precision+#	shift with same register as source and destination.++$flavour = shift;+$output  = shift;+if ($flavour =~ /\./) { $output = $flavour; undef $flavour; }++$win64=0; $win64=1 if ($flavour =~ /[nm]asm|mingw64/ || $output =~ /\.asm$/);++$0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1;+( $xlate="${dir}x86_64-xlate.pl" and -f $xlate ) or+( $xlate="${dir}../../perlasm/x86_64-xlate.pl" and -f $xlate) or+die "can't locate x86_64-xlate.pl";++open OUT,"| \"$^X\" \"$xlate\" $flavour \"$output\"";+*STDOUT=*OUT;++my @A = map([ 8*$_-100, 8*($_+1)-100, 8*($_+2)-100,+              8*($_+3)-100, 8*($_+4)-100 ], (0,5,10,15,20));++my @C = ("%rax","%rbx","%rcx","%rdx","%rbp");+my @D = map("%r$_",(8..12));+my @T = map("%r$_",(13..14));+my $iotas = "%r15";++my @rhotates = ([  0,  1, 62, 28, 27 ],+                [ 36, 44,  6, 55, 20 ],+                [  3, 10, 43, 25, 39 ],+                [ 41, 45, 15, 21,  8 ],+                [ 18,  2, 61, 56, 14 ]);++$code.=<<___;+.text++.type	__KeccakF1600,\@abi-omnipotent+.align	32+__KeccakF1600:+	mov	$A[4][0](%rdi),@C[0]+	mov	$A[4][1](%rdi),@C[1]+	mov	$A[4][2](%rdi),@C[2]+	mov	$A[4][3](%rdi),@C[3]+	mov	$A[4][4](%rdi),@C[4]+	jmp	.Loop++.align	32+.Loop:+	mov	$A[0][0](%rdi),@D[0]+	mov	$A[1][1](%rdi),@D[1]+	mov	$A[2][2](%rdi),@D[2]+	mov	$A[3][3](%rdi),@D[3]++	xor	$A[0][2](%rdi),@C[2]+	xor	$A[0][3](%rdi),@C[3]+	xor	@D[0],         @C[0]+	xor	$A[0][1](%rdi),@C[1]+	 xor	$A[1][2](%rdi),@C[2]+	 xor	$A[1][0](%rdi),@C[0]+	mov	@C[4],@D[4]+	xor	$A[0][4](%rdi),@C[4]++	xor	@D[2],         @C[2]+	xor	$A[2][0](%rdi),@C[0]+	 xor	$A[1][3](%rdi),@C[3]+	 xor	@D[1],         @C[1]+	 xor	$A[1][4](%rdi),@C[4]++	xor	$A[3][2](%rdi),@C[2]+	xor	$A[3][0](%rdi),@C[0]+	 xor	$A[2][3](%rdi),@C[3]+	 xor	$A[2][1](%rdi),@C[1]+	 xor	$A[2][4](%rdi),@C[4]++	mov	@C[2],@T[0]+	rol	\$1,@C[2]+	xor	@C[0],@C[2]		# D[1] = ROL64(C[2], 1) ^ C[0]+	 xor	@D[3],         @C[3]++	rol	\$1,@C[0]+	xor	@C[3],@C[0]		# D[4] = ROL64(C[0], 1) ^ C[3]+	 xor	$A[3][1](%rdi),@C[1]++	rol	\$1,@C[3]+	xor	@C[1],@C[3]		# D[2] = ROL64(C[3], 1) ^ C[1]+	 xor	$A[3][4](%rdi),@C[4]++	rol	\$1,@C[1]+	xor	@C[4],@C[1]		# D[0] = ROL64(C[1], 1) ^ C[4]++	rol	\$1,@C[4]+	xor	@T[0],@C[4]		# D[3] = ROL64(C[4], 1) ^ C[2]+___+	(@D[0..4], @C) = (@C[1..4,0], @D);+$code.=<<___;+	xor	@D[1],@C[1]+	xor	@D[2],@C[2]+	rol	\$$rhotates[1][1],@C[1]+	xor	@D[3],@C[3]+	xor	@D[4],@C[4]+	rol	\$$rhotates[2][2],@C[2]+	xor	@D[0],@C[0]+	 mov	@C[1],@T[0]+	rol	\$$rhotates[3][3],@C[3]+	 or	@C[2],@C[1]+	 xor	@C[0],@C[1]		#           C[0] ^ ( C[1] | C[2])+	rol	\$$rhotates[4][4],@C[4]++	 xor	($iotas),@C[1]+	 lea	8($iotas),$iotas++	mov	@C[4],@T[1]+	and	@C[3],@C[4]+	 mov	@C[1],$A[0][0](%rsi)	# R[0][0] = C[0] ^ ( C[1] | C[2]) ^ iotas[i]+	xor	@C[2],@C[4]		#           C[2] ^ ( C[4] & C[3])+	not	@C[2]+	mov	@C[4],$A[0][2](%rsi)	# R[0][2] = C[2] ^ ( C[4] & C[3])++	or	@C[3],@C[2]+	  mov	$A[4][2](%rdi),@C[4]+	xor	@T[0],@C[2]		#           C[1] ^ (~C[2] | C[3])+	mov	@C[2],$A[0][1](%rsi)	# R[0][1] = C[1] ^ (~C[2] | C[3])++	and	@C[0],@T[0]+	  mov	$A[1][4](%rdi),@C[1]+	xor	@T[1],@T[0]		#           C[4] ^ ( C[1] & C[0])+	  mov	$A[2][0](%rdi),@C[2]+	mov	@T[0],$A[0][4](%rsi)	# R[0][4] = C[4] ^ ( C[1] & C[0])++	or	@C[0],@T[1]+	  mov	$A[0][3](%rdi),@C[0]+	xor	@C[3],@T[1]		#           C[3] ^ ( C[4] | C[0])+	  mov	$A[3][1](%rdi),@C[3]+	mov	@T[1],$A[0][3](%rsi)	# R[0][3] = C[3] ^ ( C[4] | C[0])+++	xor	@D[3],@C[0]+	xor	@D[2],@C[4]+	rol	\$$rhotates[0][3],@C[0]+	xor	@D[1],@C[3]+	xor	@D[4],@C[1]+	rol	\$$rhotates[4][2],@C[4]+	rol	\$$rhotates[3][1],@C[3]+	xor	@D[0],@C[2]+	rol	\$$rhotates[1][4],@C[1]+	 mov	@C[0],@T[0]+	 or	@C[4],@C[0]+	rol	\$$rhotates[2][0],@C[2]++	xor	@C[3],@C[0]		#           C[3] ^ (C[0] |  C[4])+	mov	@C[0],$A[1][3](%rsi)	# R[1][3] = C[3] ^ (C[0] |  C[4])++	mov	@C[1],@T[1]+	and	@T[0],@C[1]+	  mov	$A[0][1](%rdi),@C[0]+	xor	@C[4],@C[1]		#           C[4] ^ (C[1] &  C[0])+	not	@C[4]+	mov	@C[1],$A[1][4](%rsi)	# R[1][4] = C[4] ^ (C[1] &  C[0])++	or	@C[3],@C[4]+	  mov	$A[1][2](%rdi),@C[1]+	xor	@C[2],@C[4]		#           C[2] ^ (~C[4] | C[3])+	mov	@C[4],$A[1][2](%rsi)	# R[1][2] = C[2] ^ (~C[4] | C[3])++	and	@C[2],@C[3]+	  mov	$A[4][0](%rdi),@C[4]+	xor	@T[1],@C[3]		#           C[1] ^ (C[3] &  C[2])+	mov	@C[3],$A[1][1](%rsi)	# R[1][1] = C[1] ^ (C[3] &  C[2])++	or	@C[2],@T[1]+	  mov	$A[2][3](%rdi),@C[2]+	xor	@T[0],@T[1]		#           C[0] ^ (C[1] |  C[2])+	  mov	$A[3][4](%rdi),@C[3]+	mov	@T[1],$A[1][0](%rsi)	# R[1][0] = C[0] ^ (C[1] |  C[2])+++	xor	@D[3],@C[2]+	xor	@D[4],@C[3]+	rol	\$$rhotates[2][3],@C[2]+	xor	@D[2],@C[1]+	rol	\$$rhotates[3][4],@C[3]+	xor	@D[0],@C[4]+	rol	\$$rhotates[1][2],@C[1]+	xor	@D[1],@C[0]+	rol	\$$rhotates[4][0],@C[4]+	 mov	@C[2],@T[0]+	 and	@C[3],@C[2]+	rol	\$$rhotates[0][1],@C[0]++	not	@C[3]+	xor	@C[1],@C[2]		#            C[1] ^ ( C[2] & C[3])+	mov	@C[2],$A[2][1](%rsi)	# R[2][1] =  C[1] ^ ( C[2] & C[3])++	mov	@C[4],@T[1]+	and	@C[3],@C[4]+	  mov	$A[2][1](%rdi),@C[2]+	xor	@T[0],@C[4]		#            C[2] ^ ( C[4] & ~C[3])+	mov	@C[4],$A[2][2](%rsi)	# R[2][2] =  C[2] ^ ( C[4] & ~C[3])++	or	@C[1],@T[0]+	  mov	$A[4][3](%rdi),@C[4]+	xor	@C[0],@T[0]		#            C[0] ^ ( C[2] | C[1])+	mov	@T[0],$A[2][0](%rsi)	# R[2][0] =  C[0] ^ ( C[2] | C[1])++	and	@C[0],@C[1]+	xor	@T[1],@C[1]		#            C[4] ^ ( C[1] & C[0])+	mov	@C[1],$A[2][4](%rsi)	# R[2][4] =  C[4] ^ ( C[1] & C[0])++	or	@C[0],@T[1]+	  mov	$A[1][0](%rdi),@C[1]+	xor	@C[3],@T[1]		#           ~C[3] ^ ( C[0] | C[4])+	  mov	$A[3][2](%rdi),@C[3]+	mov	@T[1],$A[2][3](%rsi)	# R[2][3] = ~C[3] ^ ( C[0] | C[4])+++	mov	$A[0][4](%rdi),@C[0]++	xor	@D[1],@C[2]+	xor	@D[2],@C[3]+	rol	\$$rhotates[2][1],@C[2]+	xor	@D[0],@C[1]+	rol	\$$rhotates[3][2],@C[3]+	xor	@D[3],@C[4]+	rol	\$$rhotates[1][0],@C[1]+	xor	@D[4],@C[0]+	rol	\$$rhotates[4][3],@C[4]+	 mov	@C[2],@T[0]+	 or	@C[3],@C[2]+	rol	\$$rhotates[0][4],@C[0]++	not	@C[3]+	xor	@C[1],@C[2]		#            C[1] ^ ( C[2] | C[3])+	mov	@C[2],$A[3][1](%rsi)	# R[3][1] =  C[1] ^ ( C[2] | C[3])++	mov	@C[4],@T[1]+	or	@C[3],@C[4]+	xor	@T[0],@C[4]		#            C[2] ^ ( C[4] | ~C[3])+	mov	@C[4],$A[3][2](%rsi)	# R[3][2] =  C[2] ^ ( C[4] | ~C[3])++	and	@C[1],@T[0]+	xor	@C[0],@T[0]		#            C[0] ^ ( C[2] & C[1])+	mov	@T[0],$A[3][0](%rsi)	# R[3][0] =  C[0] ^ ( C[2] & C[1])++	or	@C[0],@C[1]+	xor	@T[1],@C[1]		#            C[4] ^ ( C[1] | C[0])+	mov	@C[1],$A[3][4](%rsi)	# R[3][4] =  C[4] ^ ( C[1] | C[0])++	and	@T[1],@C[0]+	xor	@C[3],@C[0]		#           ~C[3] ^ ( C[0] & C[4])+	mov	@C[0],$A[3][3](%rsi)	# R[3][3] = ~C[3] ^ ( C[0] & C[4])+++	xor	$A[0][2](%rdi),@D[2]+	xor	$A[1][3](%rdi),@D[3]+	rol	\$$rhotates[0][2],@D[2]+	xor	$A[4][1](%rdi),@D[1]+	rol	\$$rhotates[1][3],@D[3]+	xor	$A[2][4](%rdi),@D[4]+	rol	\$$rhotates[4][1],@D[1]+	xor	$A[3][0](%rdi),@D[0]+	xchg	%rsi,%rdi+	rol	\$$rhotates[2][4],@D[4]+	rol	\$$rhotates[3][0],@D[0]+___+	@C = @D[2..4,0,1];+$code.=<<___;+	mov	@C[0],@T[0]+	and	@C[1],@C[0]+	not	@C[1]+	xor	@C[4],@C[0]		#            C[4] ^ ( C[0] & C[1])+	mov	@C[0],$A[4][4](%rdi)	# R[4][4] =  C[4] ^ ( C[0] & C[1])++	mov	@C[2],@T[1]+	and	@C[1],@C[2]+	xor	@T[0],@C[2]		#            C[0] ^ ( C[2] & ~C[1])+	mov	@C[2],$A[4][0](%rdi)	# R[4][0] =  C[0] ^ ( C[2] & ~C[1])++	or	@C[4],@T[0]+	xor	@C[3],@T[0]		#            C[3] ^ ( C[0] | C[4])+	mov	@T[0],$A[4][3](%rdi)	# R[4][3] =  C[3] ^ ( C[0] | C[4])++	and	@C[3],@C[4]+	xor	@T[1],@C[4]		#            C[2] ^ ( C[4] & C[3])+	mov	@C[4],$A[4][2](%rdi)	# R[4][2] =  C[2] ^ ( C[4] & C[3])++	or	@T[1],@C[3]+	xor	@C[1],@C[3]		#           ~C[1] ^ ( C[2] | C[3])+	mov	@C[3],$A[4][1](%rdi)	# R[4][1] = ~C[1] ^ ( C[2] | C[3])++	mov	@C[0],@C[1]		# harmonize with the loop top+	mov	@T[0],@C[0]++	test	\$255,$iotas+	jnz	.Loop++	lea	-192($iotas),$iotas	# rewind iotas+	ret+.size	__KeccakF1600,.-__KeccakF1600++.globl	KeccakF1600+.type	KeccakF1600,\@function,1,"unwind"+.align	32+KeccakF1600:+.cfi_startproc+	push	%rbx+.cfi_push	%rbx+	push	%rbp+.cfi_push	%rbp+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15++	lea	100(%rdi),%rdi		# size optimization+	sub	\$200,%rsp+.cfi_alloca	200+.cfi_end_prologue++	notq	$A[0][1](%rdi)+	notq	$A[0][2](%rdi)+	notq	$A[1][3](%rdi)+	notq	$A[2][2](%rdi)+	notq	$A[3][2](%rdi)+	notq	$A[4][0](%rdi)++	lea	iotas(%rip),$iotas+	lea	100(%rsp),%rsi		# size optimization++	call	__KeccakF1600++	notq	$A[0][1](%rdi)+	notq	$A[0][2](%rdi)+	notq	$A[1][3](%rdi)+	notq	$A[2][2](%rdi)+	notq	$A[3][2](%rdi)+	notq	$A[4][0](%rdi)+	lea	-100(%rdi),%rdi		# preserve A[][]++	lea	248(%rsp),%r11+.cfi_def_cfa	%r11,8+	mov	-48(%r11),%r15+	mov	-40(%r11),%r14+	mov	-32(%r11),%r13+	mov	-24(%r11),%r12+	mov	-16(%r11),%rbp+	mov	-8(%r11),%rbx+	lea	(%r11),%rsp+.cfi_epilogue+	ret+.cfi_endproc+.size	KeccakF1600,.-KeccakF1600+___++{ my ($A_flat,$inp,$len,$bsz) = ("%rdi","%rsi","%rdx","%rcx");+     ($A_flat,$inp) = ("%r8","%r9");+$code.=<<___;+.globl	SHA3_absorb+.type	SHA3_absorb,\@function,4,"unwind"+.align	32+SHA3_absorb:+.cfi_startproc+	push	%rbx+.cfi_push	%rbx+	push	%rbp+.cfi_push	%rbp+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15++	lea	100(%rdi),%rdi		# size optimization+	sub	\$232,%rsp+.cfi_alloca	232+.cfi_end_prologue++	mov	%rsi,$inp+	lea	100(%rsp),%rsi		# size optimization++	notq	$A[0][1](%rdi)+	notq	$A[0][2](%rdi)+	notq	$A[1][3](%rdi)+	notq	$A[2][2](%rdi)+	notq	$A[3][2](%rdi)+	notq	$A[4][0](%rdi)+	lea	iotas(%rip),$iotas++	mov	$bsz,216-100(%rsi)	# save bsz++.Loop_absorb:+	cmp	$bsz,$len+	jc	.Ldone_absorb++	shr	\$3,$bsz+	lea	-100(%rdi),$A_flat++.Lblock_absorb:+	mov	($inp),%rax+	lea	8($inp),$inp+	xor	($A_flat),%rax+	lea	8($A_flat),$A_flat+	sub	\$8,$len+	mov	%rax,-8($A_flat)+	sub	\$1,$bsz+	jnz	.Lblock_absorb++	mov	$inp,200-100(%rsi)	# save inp+	mov	$len,208-100(%rsi)	# save len+	call	__KeccakF1600+	mov	200-100(%rsi),$inp	# pull inp+	mov	208-100(%rsi),$len	# pull len+	mov	216-100(%rsi),$bsz	# pull bsz+	jmp	.Loop_absorb++.align	32+.Ldone_absorb:+	mov	$len,%rax		# return value++	notq	$A[0][1](%rdi)+	notq	$A[0][2](%rdi)+	notq	$A[1][3](%rdi)+	notq	$A[2][2](%rdi)+	notq	$A[3][2](%rdi)+	notq	$A[4][0](%rdi)++	lea	280(%rsp),%r11+.cfi_def_cfa	%r11,8+	mov	-48(%r11),%r15+	mov	-40(%r11),%r14+	mov	-32(%r11),%r13+	mov	-24(%r11),%r12+	mov	-16(%r11),%rbp+	mov	-8(%r11),%rbx+	lea	(%r11),%rsp+.cfi_epilogue+	ret+.cfi_endproc+.size	SHA3_absorb,.-SHA3_absorb+___+}+{ my ($A_flat,$out,$len,$bsz) = ("%rdi","%rsi","%rdx","%rcx");+     ($out,$len,$bsz) = ("%r12","%r13","%r14");++$code.=<<___;+.globl	SHA3_squeeze+.type	SHA3_squeeze,\@function,4,"unwind"+.align	32+SHA3_squeeze:+.cfi_startproc+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	sub	\$32,%rsp		# Windows thing+.cfi_alloca	32+.cfi_end_prologue++	shr	\$3,%rcx+	mov	$A_flat,%r8+	mov	%rsi,$out+	mov	%rdx,$len+	mov	%rcx,$bsz+	jmp	.Loop_squeeze++.align	32+.Loop_squeeze:+	cmp	\$8,$len+	jb	.Ltail_squeeze++	mov	(%r8),%rax+	lea	8(%r8),%r8+	mov	%rax,($out)+	lea	8($out),$out+	sub	\$8,$len		# len -= 8+	jz	.Ldone_squeeze++	sub	\$1,%rcx		# bsz--+	jnz	.Loop_squeeze++	mov	%rdi,%rcx		# Windows thing+	call	KeccakF1600+	mov	$A_flat,%r8+	mov	$bsz,%rcx+	jmp	.Loop_squeeze++.Ltail_squeeze:+	mov	%r8, %rsi+	mov	$out,%rdi+	mov	$len,%rcx+	.byte	0xf3,0xa4		# rep	movsb++.Ldone_squeeze:+	mov	32(%rsp),%r14+	mov	40(%rsp),%r13+	mov	48(%rsp),%r12+	add	\$56,%rsp+.cfi_alloca	-56+.cfi_epilogue+	ret+.cfi_endproc+.size	SHA3_squeeze,.-SHA3_squeeze+___+}+$code.=<<___;+.align	256+	.quad	0,0,0,0,0,0,0,0+.type	iotas,\@object+iotas:+	.quad	0x0000000000000001+	.quad	0x0000000000008082+	.quad	0x800000000000808a+	.quad	0x8000000080008000+	.quad	0x000000000000808b+	.quad	0x0000000080000001+	.quad	0x8000000080008081+	.quad	0x8000000000008009+	.quad	0x000000000000008a+	.quad	0x0000000000000088+	.quad	0x0000000080008009+	.quad	0x000000008000000a+	.quad	0x000000008000808b+	.quad	0x800000000000008b+	.quad	0x8000000000008089+	.quad	0x8000000000008003+	.quad	0x8000000000008002+	.quad	0x8000000000000080+	.quad	0x000000000000800a+	.quad	0x800000008000000a+	.quad	0x8000000080008081+	.quad	0x8000000000008080+	.quad	0x0000000080000001+	.quad	0x8000000080008008+.size	iotas,.-iotas+.asciz	"Keccak-1600 absorb and squeeze for x86_64, CRYPTOGAMS by <appro\@openssl.org>"+___++foreach (split("\n",$code)) {+	# Below replacement results in 11.2 on Sandy Bridge, 9.4 on+	# Haswell, but it hurts other processors by up to 2-3-4x...+	#s/rol\s+(\$[0-9]+),(%[a-z][a-z0-9]+)/shld\t$1,$2,$2/;++	# Below replacement results in 9.3 on Haswell [as well as+	# on Ryzen, i.e. it *hurts* Ryzen]...+	#s/rol\s+\$([0-9]+),(%[a-z][a-z0-9]+)/rorx\t\$64-$1,$2,$2/;++	print $_, "\n";+}++close STDOUT;
+ cbits/asm/poly1305-armv8-ios64.S view
@@ -0,0 +1,844 @@+#ifndef __KERNEL__+# include "arm_arch.h"++#endif++.text++// forward "declarations" are required for Apple+.globl	_crypton_poly1305_asm_blocks+.globl	_crypton_poly1305_asm_emit++.globl	_crypton_poly1305_asm_init++.align	5+_crypton_poly1305_asm_init:+	cmp	x1,xzr+	stp	xzr,xzr,[x0]		// zero hash value+	stp	xzr,xzr,[x0,#16]	// [along with is_base2_26]++	csel	x0,xzr,x0,eq+	b.eq	Lno_key++#ifndef	__KERNEL__+	adrp	x17,_crypton_armcap_P@PAGE+	ldr	w17,[x17,_crypton_armcap_P@PAGEOFF]+#endif++	ldp	x7,x8,[x1]		// load key+	mov	x9,#0xfffffffc0fffffff+	movk	x9,#0x0fff,lsl#48+#ifdef	__AARCH64EB__+	rev	x7,x7			// flip bytes+	rev	x8,x8+#endif+	and	x7,x7,x9		// &=0ffffffc0fffffff+	and	x9,x9,#-4+	and	x8,x8,x9		// &=0ffffffc0ffffffc+	mov	w9,#-1+	stp	x7,x8,[x0,#32]	// save key value+	str	w9,[x0,#48]	// impossible key power value++#ifndef	__KERNEL__+	tst	w17,#ARMV7_NEON++	adr	x13,Lcrypton_poly1305_asm_blocks+	adr	x15,Lcrypton_poly1305_asm_blocks_neon+	adr	x14,Lcrypton_poly1305_asm_emit++	csel	x13,x13,x15,eq+# ifdef	__CHERI_PURE_CAPABILITY__+	add	x13, x13, #1+	add	x14, x14, #1+	seal	x13, x13, rb+	seal	x14, x14, rb+# endif++# ifdef	__ILP32__+	stp	w13,w14,[x2]+# else+	stp	x13,x14,[x2]+# endif+	mov	x0,#1+#else+	mov	x0,#0+#endif+Lno_key:+	ret++++.align	5+_crypton_poly1305_asm_blocks:+Lcrypton_poly1305_asm_blocks:+	ands	x2,x2,#-16+	b.eq	Lno_data++	ldp	x4,x5,[x0]		// load hash value+	ldp	x6,x17,[x0,#16]	// [along with is_base2_26]+	ldp	x7,x8,[x0,#32]	// load key value++#ifdef	__AARCH64EB__+	lsr	x12,x4,#32+	mov	w13,w4+	lsr	x14,x5,#32+	mov	w15,w5+	lsr	x16,x6,#32+#else+	mov	w12,w4+	lsr	x13,x4,#32+	mov	w14,w5+	lsr	x15,x5,#32+	mov	w16,w6+#endif++	add	x12,x12,x13,lsl#26	// base 2^26 -> base 2^64+	lsr	x13,x14,#12+	adds	x12,x12,x14,lsl#52+	add	x13,x13,x15,lsl#14+	adc	x13,x13,xzr+	lsr	x14,x16,#24+	adds	x13,x13,x16,lsl#40+	adc	x14,x14,xzr++	cmp	x17,#0			// is_base2_26?+	add	x9,x8,x8,lsr#2	// s1 = r1 + (r1 >> 2)+	csel	x4,x4,x12,eq		// choose between radixes+	csel	x5,x5,x13,eq+	csel	x6,x6,x14,eq++Loop:+	ldp	x10,x11,[x1],#16	// load input+	sub	x2,x2,#16+#ifdef	__AARCH64EB__+	rev	x10,x10+	rev	x11,x11+#endif+	adds	x4,x4,x10		// accumulate input+	adcs	x5,x5,x11++	mul	x12,x4,x7		// h0*r0+	adc	x6,x6,x3+	umulh	x13,x4,x7++	mul	x10,x5,x9		// h1*5*r1+	umulh	x11,x5,x9++	adds	x12,x12,x10+	mul	x10,x4,x8		// h0*r1+	adc	x13,x13,x11+	umulh	x14,x4,x8++	adds	x13,x13,x10+	mul	x10,x5,x7		// h1*r0+	adc	x14,x14,xzr+	umulh	x11,x5,x7++	adds	x13,x13,x10+	mul	x10,x6,x9		// h2*5*r1+	adc	x14,x14,x11+	mul	x11,x6,x7		// h2*r0++	adds	x13,x13,x10+	adc	x14,x14,x11++	and	x10,x14,#-4		// final reduction+	and	x6,x14,#3+	add	x10,x10,x14,lsr#2+	adds	x4,x12,x10+	adcs	x5,x13,xzr+	adc	x6,x6,xzr++	cbnz	x2,Loop++	stp	x4,x5,[x0]		// store hash value+	stp	x6,xzr,[x0,#16]	// [and clear is_base2_26]++Lno_data:+	ret++++.align	5+_crypton_poly1305_asm_emit:+Lcrypton_poly1305_asm_emit:+	ldp	x4,x5,[x0]		// load hash base 2^64+	ldp	x6,x7,[x0,#16]	// [along with is_base2_26]+	ldp	x10,x11,[x2]	// load nonce++#ifdef	__AARCH64EB__+	lsr	x12,x4,#32+	mov	w13,w4+	lsr	x14,x5,#32+	mov	w15,w5+	lsr	x16,x6,#32+#else+	mov	w12,w4+	lsr	x13,x4,#32+	mov	w14,w5+	lsr	x15,x5,#32+	mov	w16,w6+#endif++	add	x12,x12,x13,lsl#26	// base 2^26 -> base 2^64+	lsr	x13,x14,#12+	adds	x12,x12,x14,lsl#52+	add	x13,x13,x15,lsl#14+	adc	x13,x13,xzr+	lsr	x14,x16,#24+	adds	x13,x13,x16,lsl#40+	adc	x14,x14,xzr++	cmp	x7,#0			// is_base2_26?+	csel	x4,x4,x12,eq		// choose between radixes+	csel	x5,x5,x13,eq+	csel	x6,x6,x14,eq++	adds	x12,x4,#5		// compare to modulus+	adcs	x13,x5,xzr+	adc	x14,x6,xzr++	tst	x14,#-4			// see if it's carried/borrowed++	csel	x4,x4,x12,eq+	csel	x5,x5,x13,eq++#ifdef	__AARCH64EB__+	ror	x10,x10,#32		// flip nonce words+	ror	x11,x11,#32+#endif+	adds	x4,x4,x10		// accumulate nonce+	adc	x5,x5,x11+#ifdef	__AARCH64EB__+	rev	x4,x4			// flip output bytes+	rev	x5,x5+#endif+	stp	x4,x5,[x1]		// write result++	ret+++.align	5+crypton_poly1305_asm_mult:+	mul	x12,x4,x7		// h0*r0+	umulh	x13,x4,x7++	mul	x10,x5,x9		// h1*5*r1+	umulh	x11,x5,x9++	adds	x12,x12,x10+	mul	x10,x4,x8		// h0*r1+	adc	x13,x13,x11+	umulh	x14,x4,x8++	adds	x13,x13,x10+	mul	x10,x5,x7		// h1*r0+	adc	x14,x14,xzr+	umulh	x11,x5,x7++	adds	x13,x13,x10+	mul	x10,x6,x9		// h2*5*r1+	adc	x14,x14,x11+	mul	x11,x6,x7		// h2*r0++	adds	x13,x13,x10+	adc	x14,x14,x11++	and	x10,x14,#-4		// final reduction+	and	x6,x14,#3+	add	x10,x10,x14,lsr#2+	adds	x4,x12,x10+	adcs	x5,x13,xzr+	adc	x6,x6,xzr++	ret++++.align	4+crypton_poly1305_asm_splat:+	and	x12,x4,#0x03ffffff	// base 2^64 -> base 2^26+	ubfx	x13,x4,#26,#26+	extr	x14,x5,x4,#52+	and	x14,x14,#0x03ffffff+	ubfx	x15,x5,#14,#26+	extr	x16,x6,x5,#40++	str	w12,[x0,#16*0]	// r0+	add	w12,w13,w13,lsl#2	// r1*5+	str	w13,[x0,#16*1]	// r1+	add	w13,w14,w14,lsl#2	// r2*5+	str	w12,[x0,#16*2]	// s1+	str	w14,[x0,#16*3]	// r2+	add	w14,w15,w15,lsl#2	// r3*5+	str	w13,[x0,#16*4]	// s2+	str	w15,[x0,#16*5]	// r3+	add	w15,w16,w16,lsl#2	// r4*5+	str	w14,[x0,#16*6]	// s3+	str	w16,[x0,#16*7]	// r4+	str	w15,[x0,#16*8]	// s4++	ret+++#ifdef	__KERNEL__+.globl	_crypton_poly1305_asm_blocks_neon+#endif++.align	5+_crypton_poly1305_asm_blocks_neon:+Lcrypton_poly1305_asm_blocks_neon:+	ldr	x17,[x0,#24]+	cmp	x2,#128+	b.lo	Lcrypton_poly1305_asm_blocks++.long	0xd503233f		// paciasp+	stp	x29,x30,[sp,#-2*__SIZEOF_POINTER__-64]!+	add	x29,sp,#0++	stp	d8,d9,[sp,#2*__SIZEOF_POINTER__+0]	// meet ABI requirements+	stp	d10,d11,[sp,#2*__SIZEOF_POINTER__+16]+	stp	d12,d13,[sp,#2*__SIZEOF_POINTER__+32]+	stp	d14,d15,[sp,#2*__SIZEOF_POINTER__+48]++	cbz	x17,Lbase2_64_neon++	ldp	w10,w11,[x0]		// load hash value base 2^26+	ldp	w12,w13,[x0,#8]+	ldr	w14,[x0,#16]++	tst	x2,#31+	b.eq	Leven_neon++	ldp	x7,x8,[x0,#32]	// load key value++	add	x4,x10,x11,lsl#26	// base 2^26 -> base 2^64+	lsr	x5,x12,#12+	adds	x4,x4,x12,lsl#52+	add	x5,x5,x13,lsl#14+	adc	x5,x5,xzr+	lsr	x6,x14,#24+	adds	x5,x5,x14,lsl#40+	adc	x14,x6,xzr		// can be partially reduced...++	ldp	x12,x13,[x1],#16	// load input+	sub	x2,x2,#16+	add	x9,x8,x8,lsr#2	// s1 = r1 + (r1 >> 2)++#ifdef	__AARCH64EB__+	rev	x12,x12+	rev	x13,x13+#endif+	adds	x4,x4,x12		// accumulate input+	adcs	x5,x5,x13+	adc	x6,x6,x3++	bl	crypton_poly1305_asm_mult++	and	x10,x4,#0x03ffffff	// base 2^64 -> base 2^26+	ubfx	x11,x4,#26,#26+	extr	x12,x5,x4,#52+	and	x12,x12,#0x03ffffff+	ubfx	x13,x5,#14,#26+	extr	x14,x6,x5,#40++	b	Leven_neon++.align	4+Lbase2_64_neon:+	ldp	x7,x8,[x0,#32]	// load key value++	ldp	x4,x5,[x0]		// load hash value base 2^64+	ldr	x6,[x0,#16]++	tst	x2,#31+	b.eq	Linit_neon++	ldp	x12,x13,[x1],#16	// load input+	sub	x2,x2,#16+	add	x9,x8,x8,lsr#2	// s1 = r1 + (r1 >> 2)+#ifdef	__AARCH64EB__+	rev	x12,x12+	rev	x13,x13+#endif+	adds	x4,x4,x12		// accumulate input+	adcs	x5,x5,x13+	adc	x6,x6,x3++	bl	crypton_poly1305_asm_mult++Linit_neon:+	ldr	w17,[x0,#48]		// first table element+	and	x10,x4,#0x03ffffff	// base 2^64 -> base 2^26+	ubfx	x11,x4,#26,#26+	extr	x12,x5,x4,#52+	and	x12,x12,#0x03ffffff+	ubfx	x13,x5,#14,#26+	extr	x14,x6,x5,#40++	cmp	w17,#-1			// is value impossible?+	b.ne	Leven_neon++	fmov	d24,x10+	fmov	d25,x11+	fmov	d26,x12+	fmov	d27,x13+	fmov	d28,x14++	////////////////////////////////// initialize r^n table+	mov	x4,x7			// r^1+	add	x9,x8,x8,lsr#2	// s1 = r1 + (r1 >> 2)+	mov	x5,x8+	mov	x6,xzr+	add	x0,x0,#48+12+	bl	crypton_poly1305_asm_splat++	bl	crypton_poly1305_asm_mult		// r^2+	sub	x0,x0,#4+	bl	crypton_poly1305_asm_splat++	bl	crypton_poly1305_asm_mult		// r^3+	sub	x0,x0,#4+	bl	crypton_poly1305_asm_splat++	bl	crypton_poly1305_asm_mult		// r^4+	sub	x0,x0,#4+	bl	crypton_poly1305_asm_splat+	sub	x0,x0,#48+	b	Ldo_neon++.align	4+Leven_neon:+	fmov	d24,x10+	fmov	d25,x11+	fmov	d26,x12+	fmov	d27,x13+	fmov	d28,x14++Ldo_neon:+	ldp	x8,x12,[x1,#32]	// inp[2:3]+	subs	x2,x2,#64+	ldp	x9,x13,[x1,#48]+	add	x16,x1,#96+	adr	x17,Lzeros++	lsl	x3,x3,#24+	add	x15,x0,#48++#ifdef	__AARCH64EB__+	rev	x8,x8+	rev	x12,x12+	rev	x9,x9+	rev	x13,x13+#endif+	and	x4,x8,#0x03ffffff	// base 2^64 -> base 2^26+	and	x5,x9,#0x03ffffff+	ubfx	x6,x8,#26,#26+	ubfx	x7,x9,#26,#26+	add	x4,x4,x5,lsl#32		// bfi	x4,x5,#32,#32+	extr	x8,x12,x8,#52+	extr	x9,x13,x9,#52+	add	x6,x6,x7,lsl#32		// bfi	x6,x7,#32,#32+	fmov	d14,x4+	and	x8,x8,#0x03ffffff+	and	x9,x9,#0x03ffffff+	ubfx	x10,x12,#14,#26+	ubfx	x11,x13,#14,#26+	add	x12,x3,x12,lsr#40+	add	x13,x3,x13,lsr#40+	add	x8,x8,x9,lsl#32		// bfi	x8,x9,#32,#32+	fmov	d15,x6+	add	x10,x10,x11,lsl#32	// bfi	x10,x11,#32,#32+	add	x12,x12,x13,lsl#32	// bfi	x12,x13,#32,#32+	fmov	d16,x8+	fmov	d17,x10+	fmov	d18,x12++	ldp	x8,x12,[x1],#16	// inp[0:1]+	ldp	x9,x13,[x1],#48++	ld1	{v0.4s,v1.4s,v2.4s,v3.4s},[x15],#64+	ld1	{v4.4s,v5.4s,v6.4s,v7.4s},[x15],#64+	ld1	{v8.4s},[x15]++#ifdef	__AARCH64EB__+	rev	x8,x8+	rev	x12,x12+	rev	x9,x9+	rev	x13,x13+#endif+	and	x4,x8,#0x03ffffff	// base 2^64 -> base 2^26+	and	x5,x9,#0x03ffffff+	ubfx	x6,x8,#26,#26+	ubfx	x7,x9,#26,#26+	add	x4,x4,x5,lsl#32		// bfi	x4,x5,#32,#32+	extr	x8,x12,x8,#52+	extr	x9,x13,x9,#52+	add	x6,x6,x7,lsl#32		// bfi	x6,x7,#32,#32+	fmov	d9,x4+	and	x8,x8,#0x03ffffff+	and	x9,x9,#0x03ffffff+	ubfx	x10,x12,#14,#26+	ubfx	x11,x13,#14,#26+	add	x12,x3,x12,lsr#40+	add	x13,x3,x13,lsr#40+	add	x8,x8,x9,lsl#32		// bfi	x8,x9,#32,#32+	fmov	d10,x6+	add	x10,x10,x11,lsl#32	// bfi	x10,x11,#32,#32+	add	x12,x12,x13,lsl#32	// bfi	x12,x13,#32,#32+	movi	v31.2d,#-1+	fmov	d11,x8+	fmov	d12,x10+	fmov	d13,x12+	ushr	v31.2d,v31.2d,#38++	b.ls	Lskip_loop++.align	4+Loop_neon:+	////////////////////////////////////////////////////////////////+	// ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2+	// ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^3+inp[7]*r+	//   ___________________/+	// ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2+inp[8])*r^2+	// ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^4+inp[7]*r^2+inp[9])*r+	//   ___________________/ ____________________/+	//+	// Note that we start with inp[2:3]*r^2. This is because it+	// doesn't depend on reduction in previous iteration.+	////////////////////////////////////////////////////////////////+	// d4 = h0*r4 + h1*r3   + h2*r2   + h3*r1   + h4*r0+	// d3 = h0*r3 + h1*r2   + h2*r1   + h3*r0   + h4*5*r4+	// d2 = h0*r2 + h1*r1   + h2*r0   + h3*5*r4 + h4*5*r3+	// d1 = h0*r1 + h1*r0   + h2*5*r4 + h3*5*r3 + h4*5*r2+	// d0 = h0*r0 + h1*5*r4 + h2*5*r3 + h3*5*r2 + h4*5*r1++	subs	x2,x2,#64+	umull	v23.2d,v14.2s,v7.s[2]+	csel	x16,x17,x16,lo+	umull	v22.2d,v14.2s,v5.s[2]+	umull	v21.2d,v14.2s,v3.s[2]+	ldp	x8,x12,[x16],#16	// inp[2:3] (or zero)+	umull	v20.2d,v14.2s,v1.s[2]+	ldp	x9,x13,[x16],#48+	umull	v19.2d,v14.2s,v0.s[2]+#ifdef	__AARCH64EB__+	rev	x8,x8+	rev	x12,x12+	rev	x9,x9+	rev	x13,x13+#endif++	umlal	v23.2d,v15.2s,v5.s[2]+	and	x4,x8,#0x03ffffff	// base 2^64 -> base 2^26+	umlal	v22.2d,v15.2s,v3.s[2]+	and	x5,x9,#0x03ffffff+	umlal	v21.2d,v15.2s,v1.s[2]+	ubfx	x6,x8,#26,#26+	umlal	v20.2d,v15.2s,v0.s[2]+	ubfx	x7,x9,#26,#26+	umlal	v19.2d,v15.2s,v8.s[2]+	add	x4,x4,x5,lsl#32		// bfi	x4,x5,#32,#32++	umlal	v23.2d,v16.2s,v3.s[2]+	extr	x8,x12,x8,#52+	umlal	v22.2d,v16.2s,v1.s[2]+	extr	x9,x13,x9,#52+	umlal	v21.2d,v16.2s,v0.s[2]+	add	x6,x6,x7,lsl#32		// bfi	x6,x7,#32,#32+	umlal	v20.2d,v16.2s,v8.s[2]+	fmov	d14,x4+	umlal	v19.2d,v16.2s,v6.s[2]+	and	x8,x8,#0x03ffffff++	umlal	v23.2d,v17.2s,v1.s[2]+	and	x9,x9,#0x03ffffff+	umlal	v22.2d,v17.2s,v0.s[2]+	ubfx	x10,x12,#14,#26+	umlal	v21.2d,v17.2s,v8.s[2]+	ubfx	x11,x13,#14,#26+	umlal	v20.2d,v17.2s,v6.s[2]+	add	x8,x8,x9,lsl#32		// bfi	x8,x9,#32,#32+	umlal	v19.2d,v17.2s,v4.s[2]+	fmov	d15,x6++	add	v11.2s,v11.2s,v26.2s+	add	x12,x3,x12,lsr#40+	umlal	v23.2d,v18.2s,v0.s[2]+	add	x13,x3,x13,lsr#40+	umlal	v22.2d,v18.2s,v8.s[2]+	add	x10,x10,x11,lsl#32	// bfi	x10,x11,#32,#32+	umlal	v21.2d,v18.2s,v6.s[2]+	add	x12,x12,x13,lsl#32	// bfi	x12,x13,#32,#32+	umlal	v20.2d,v18.2s,v4.s[2]+	fmov	d16,x8+	umlal	v19.2d,v18.2s,v2.s[2]+	fmov	d17,x10++	////////////////////////////////////////////////////////////////+	// (hash+inp[0:1])*r^4 and accumulate++	add	v9.2s,v9.2s,v24.2s+	fmov	d18,x12+	umlal	v22.2d,v11.2s,v1.s[0]+	ldp	x8,x12,[x1],#16	// inp[0:1]+	umlal	v19.2d,v11.2s,v6.s[0]+	ldp	x9,x13,[x1],#48+	umlal	v23.2d,v11.2s,v3.s[0]+	umlal	v20.2d,v11.2s,v8.s[0]+	umlal	v21.2d,v11.2s,v0.s[0]+#ifdef	__AARCH64EB__+	rev	x8,x8+	rev	x12,x12+	rev	x9,x9+	rev	x13,x13+#endif++	add	v10.2s,v10.2s,v25.2s+	umlal	v22.2d,v9.2s,v5.s[0]+	umlal	v23.2d,v9.2s,v7.s[0]+	and	x4,x8,#0x03ffffff	// base 2^64 -> base 2^26+	umlal	v21.2d,v9.2s,v3.s[0]+	and	x5,x9,#0x03ffffff+	umlal	v19.2d,v9.2s,v0.s[0]+	ubfx	x6,x8,#26,#26+	umlal	v20.2d,v9.2s,v1.s[0]+	ubfx	x7,x9,#26,#26++	add	v12.2s,v12.2s,v27.2s+	add	x4,x4,x5,lsl#32		// bfi	x4,x5,#32,#32+	umlal	v22.2d,v10.2s,v3.s[0]+	extr	x8,x12,x8,#52+	umlal	v23.2d,v10.2s,v5.s[0]+	extr	x9,x13,x9,#52+	umlal	v19.2d,v10.2s,v8.s[0]+	add	x6,x6,x7,lsl#32		// bfi	x6,x7,#32,#32+	umlal	v21.2d,v10.2s,v1.s[0]+	fmov	d9,x4+	umlal	v20.2d,v10.2s,v0.s[0]+	and	x8,x8,#0x03ffffff++	add	v13.2s,v13.2s,v28.2s+	and	x9,x9,#0x03ffffff+	umlal	v22.2d,v12.2s,v0.s[0]+	ubfx	x10,x12,#14,#26+	umlal	v19.2d,v12.2s,v4.s[0]+	ubfx	x11,x13,#14,#26+	umlal	v23.2d,v12.2s,v1.s[0]+	add	x8,x8,x9,lsl#32		// bfi	x8,x9,#32,#32+	umlal	v20.2d,v12.2s,v6.s[0]+	fmov	d10,x6+	umlal	v21.2d,v12.2s,v8.s[0]+	add	x12,x3,x12,lsr#40++	umlal	v22.2d,v13.2s,v8.s[0]+	add	x13,x3,x13,lsr#40+	umlal	v19.2d,v13.2s,v2.s[0]+	add	x10,x10,x11,lsl#32	// bfi	x10,x11,#32,#32+	umlal	v23.2d,v13.2s,v0.s[0]+	add	x12,x12,x13,lsl#32	// bfi	x12,x13,#32,#32+	umlal	v20.2d,v13.2s,v4.s[0]+	fmov	d11,x8+	umlal	v21.2d,v13.2s,v6.s[0]+	fmov	d12,x10+	fmov	d13,x12++	/////////////////////////////////////////////////////////////////+	// lazy reduction as discussed in "NEON crypto" by D.J. Bernstein+	// and P. Schwabe+	//+	// [see discussion in poly1305-armv4 module]++	ushr	v29.2d,v22.2d,#26+	xtn	v27.2s,v22.2d+	ushr	v30.2d,v19.2d,#26+	and	v19.16b,v19.16b,v31.16b+	add	v23.2d,v23.2d,v29.2d	// h3 -> h4+	bic	v27.2s,#0xfc,lsl#24	// &=0x03ffffff+	add	v20.2d,v20.2d,v30.2d	// h0 -> h1++	ushr	v29.2d,v23.2d,#26+	xtn	v28.2s,v23.2d+	ushr	v30.2d,v20.2d,#26+	xtn	v25.2s,v20.2d+	bic	v28.2s,#0xfc,lsl#24+	add	v21.2d,v21.2d,v30.2d	// h1 -> h2++	add	v19.2d,v19.2d,v29.2d+	shl	v29.2d,v29.2d,#2+	shrn	v30.2s,v21.2d,#26+	xtn	v26.2s,v21.2d+	add	v19.2d,v19.2d,v29.2d	// h4 -> h0+	bic	v25.2s,#0xfc,lsl#24+	add	v27.2s,v27.2s,v30.2s		// h2 -> h3+	bic	v26.2s,#0xfc,lsl#24++	shrn	v29.2s,v19.2d,#26+	xtn	v24.2s,v19.2d+	ushr	v30.2s,v27.2s,#26+	bic	v27.2s,#0xfc,lsl#24+	bic	v24.2s,#0xfc,lsl#24+	add	v25.2s,v25.2s,v29.2s		// h0 -> h1+	add	v28.2s,v28.2s,v30.2s		// h3 -> h4++	b.hi	Loop_neon++Lskip_loop:+	dup	v16.2d,v16.d[0]+	add	v11.2s,v11.2s,v26.2s++	////////////////////////////////////////////////////////////////+	// multiply (inp[0:1]+hash) or inp[2:3] by r^2:r^1++	adds	x2,x2,#32+	b.ne	Long_tail++	dup	v16.2d,v11.d[0]+	add	v14.2s,v9.2s,v24.2s+	add	v17.2s,v12.2s,v27.2s+	add	v15.2s,v10.2s,v25.2s+	add	v18.2s,v13.2s,v28.2s++Long_tail:+	dup	v14.2d,v14.d[0]+	umull2	v19.2d,v16.4s,v6.4s+	umull2	v22.2d,v16.4s,v1.4s+	umull2	v23.2d,v16.4s,v3.4s+	umull2	v21.2d,v16.4s,v0.4s+	umull2	v20.2d,v16.4s,v8.4s++	dup	v15.2d,v15.d[0]+	umlal2	v19.2d,v14.4s,v0.4s+	umlal2	v21.2d,v14.4s,v3.4s+	umlal2	v22.2d,v14.4s,v5.4s+	umlal2	v23.2d,v14.4s,v7.4s+	umlal2	v20.2d,v14.4s,v1.4s++	dup	v17.2d,v17.d[0]+	umlal2	v19.2d,v15.4s,v8.4s+	umlal2	v22.2d,v15.4s,v3.4s+	umlal2	v21.2d,v15.4s,v1.4s+	umlal2	v23.2d,v15.4s,v5.4s+	umlal2	v20.2d,v15.4s,v0.4s++	dup	v18.2d,v18.d[0]+	umlal2	v22.2d,v17.4s,v0.4s+	umlal2	v23.2d,v17.4s,v1.4s+	umlal2	v19.2d,v17.4s,v4.4s+	umlal2	v20.2d,v17.4s,v6.4s+	umlal2	v21.2d,v17.4s,v8.4s++	umlal2	v22.2d,v18.4s,v8.4s+	umlal2	v19.2d,v18.4s,v2.4s+	umlal2	v23.2d,v18.4s,v0.4s+	umlal2	v20.2d,v18.4s,v4.4s+	umlal2	v21.2d,v18.4s,v6.4s++	b.eq	Lshort_tail++	////////////////////////////////////////////////////////////////+	// (hash+inp[0:1])*r^4:r^3 and accumulate++	add	v9.2s,v9.2s,v24.2s+	umlal	v22.2d,v11.2s,v1.2s+	umlal	v19.2d,v11.2s,v6.2s+	umlal	v23.2d,v11.2s,v3.2s+	umlal	v20.2d,v11.2s,v8.2s+	umlal	v21.2d,v11.2s,v0.2s++	add	v10.2s,v10.2s,v25.2s+	umlal	v22.2d,v9.2s,v5.2s+	umlal	v19.2d,v9.2s,v0.2s+	umlal	v23.2d,v9.2s,v7.2s+	umlal	v20.2d,v9.2s,v1.2s+	umlal	v21.2d,v9.2s,v3.2s++	add	v12.2s,v12.2s,v27.2s+	umlal	v22.2d,v10.2s,v3.2s+	umlal	v19.2d,v10.2s,v8.2s+	umlal	v23.2d,v10.2s,v5.2s+	umlal	v20.2d,v10.2s,v0.2s+	umlal	v21.2d,v10.2s,v1.2s++	add	v13.2s,v13.2s,v28.2s+	umlal	v22.2d,v12.2s,v0.2s+	umlal	v19.2d,v12.2s,v4.2s+	umlal	v23.2d,v12.2s,v1.2s+	umlal	v20.2d,v12.2s,v6.2s+	umlal	v21.2d,v12.2s,v8.2s++	umlal	v22.2d,v13.2s,v8.2s+	umlal	v19.2d,v13.2s,v2.2s+	umlal	v23.2d,v13.2s,v0.2s+	umlal	v20.2d,v13.2s,v4.2s+	umlal	v21.2d,v13.2s,v6.2s++Lshort_tail:+	////////////////////////////////////////////////////////////////+	// horizontal add++	addp	v22.2d,v22.2d,v22.2d+	ldp	d8,d9,[sp,#2*__SIZEOF_POINTER__+0]	// meet ABI requirements+	addp	v19.2d,v19.2d,v19.2d+	ldp	d10,d11,[sp,#2*__SIZEOF_POINTER__+16]+	addp	v23.2d,v23.2d,v23.2d+	ldp	d12,d13,[sp,#2*__SIZEOF_POINTER__+32]+	addp	v20.2d,v20.2d,v20.2d+	ldp	d14,d15,[sp,#2*__SIZEOF_POINTER__+48]+	addp	v21.2d,v21.2d,v21.2d+	ldr	x30,[sp,#__SIZEOF_POINTER__]++	////////////////////////////////////////////////////////////////+	// lazy reduction, but without narrowing++	ushr	v29.2d,v22.2d,#26+	and	v22.16b,v22.16b,v31.16b+	ushr	v30.2d,v19.2d,#26+	and	v19.16b,v19.16b,v31.16b++	add	v23.2d,v23.2d,v29.2d	// h3 -> h4+	add	v20.2d,v20.2d,v30.2d	// h0 -> h1++	ushr	v29.2d,v23.2d,#26+	and	v23.16b,v23.16b,v31.16b+	ushr	v30.2d,v20.2d,#26+	and	v20.16b,v20.16b,v31.16b+	add	v21.2d,v21.2d,v30.2d	// h1 -> h2++	add	v19.2d,v19.2d,v29.2d+	shl	v29.2d,v29.2d,#2+	ushr	v30.2d,v21.2d,#26+	and	v21.16b,v21.16b,v31.16b+	add	v19.2d,v19.2d,v29.2d	// h4 -> h0+	add	v22.2d,v22.2d,v30.2d	// h2 -> h3++	ushr	v29.2d,v19.2d,#26+	and	v19.16b,v19.16b,v31.16b+	ushr	v30.2d,v22.2d,#26+	and	v22.16b,v22.16b,v31.16b+	add	v20.2d,v20.2d,v29.2d	// h0 -> h1+	add	v23.2d,v23.2d,v30.2d	// h3 -> h4++	////////////////////////////////////////////////////////////////+	// write the result, can be partially reduced++	st4	{v19.s,v20.s,v21.s,v22.s}[0],[x0],#16+	mov	x4,#1+	st1	{v23.s}[0],[x0]+	str	x4,[x0,#8]		// set is_base2_26++	ldr	x29,[sp],#2*__SIZEOF_POINTER__+64+.long	0xd50323bf		// autiasp+	ret+++.align	5+Lzeros:+.long	0,0,0,0,0,0,0,0+.byte	80,111,108,121,49,51,48,53,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.align	2+.align	2+#if !defined(__KERNEL__) && !defined(_WIN64)+.comm	__crypton_armcap_P,4+.private_extern	_crypton_armcap_P+#endif
+ cbits/asm/poly1305-armv8-linux64.S view
@@ -0,0 +1,846 @@+#ifndef __KERNEL__+# include "arm_arch.h"++#endif++.text++// forward "declarations" are required for Apple+.globl	crypton_poly1305_asm_blocks+.globl	crypton_poly1305_asm_emit++.globl	crypton_poly1305_asm_init+.type	crypton_poly1305_asm_init,%function+.align	5+crypton_poly1305_asm_init:+	cmp	x1,xzr+	stp	xzr,xzr,[x0]		// zero hash value+	stp	xzr,xzr,[x0,#16]	// [along with is_base2_26]++	csel	x0,xzr,x0,eq+	b.eq	.Lno_key++#ifndef	__KERNEL__+	adrp	x17,crypton_armcap_P+	ldr	w17,[x17,#:lo12:crypton_armcap_P]+#endif++	ldp	x7,x8,[x1]		// load key+	mov	x9,#0xfffffffc0fffffff+	movk	x9,#0x0fff,lsl#48+#ifdef	__AARCH64EB__+	rev	x7,x7			// flip bytes+	rev	x8,x8+#endif+	and	x7,x7,x9		// &=0ffffffc0fffffff+	and	x9,x9,#-4+	and	x8,x8,x9		// &=0ffffffc0ffffffc+	mov	w9,#-1+	stp	x7,x8,[x0,#32]	// save key value+	str	w9,[x0,#48]	// impossible key power value++#ifndef	__KERNEL__+	tst	w17,#ARMV7_NEON++	adr	x13,.Lcrypton_poly1305_asm_blocks+	adr	x15,.Lcrypton_poly1305_asm_blocks_neon+	adr	x14,.Lcrypton_poly1305_asm_emit++	csel	x13,x13,x15,eq+# ifdef	__CHERI_PURE_CAPABILITY__+	add	x13, x13, #1+	add	x14, x14, #1+	seal	x13, x13, rb+	seal	x14, x14, rb+# endif++# ifdef	__ILP32__+	stp	w13,w14,[x2]+# else+	stp	x13,x14,[x2]+# endif+	mov	x0,#1+#else+	mov	x0,#0+#endif+.Lno_key:+	ret+.size	crypton_poly1305_asm_init,.-crypton_poly1305_asm_init++.type	crypton_poly1305_asm_blocks,%function+.align	5+crypton_poly1305_asm_blocks:+.Lcrypton_poly1305_asm_blocks:+	ands	x2,x2,#-16+	b.eq	.Lno_data++	ldp	x4,x5,[x0]		// load hash value+	ldp	x6,x17,[x0,#16]	// [along with is_base2_26]+	ldp	x7,x8,[x0,#32]	// load key value++#ifdef	__AARCH64EB__+	lsr	x12,x4,#32+	mov	w13,w4+	lsr	x14,x5,#32+	mov	w15,w5+	lsr	x16,x6,#32+#else+	mov	w12,w4+	lsr	x13,x4,#32+	mov	w14,w5+	lsr	x15,x5,#32+	mov	w16,w6+#endif++	add	x12,x12,x13,lsl#26	// base 2^26 -> base 2^64+	lsr	x13,x14,#12+	adds	x12,x12,x14,lsl#52+	add	x13,x13,x15,lsl#14+	adc	x13,x13,xzr+	lsr	x14,x16,#24+	adds	x13,x13,x16,lsl#40+	adc	x14,x14,xzr++	cmp	x17,#0			// is_base2_26?+	add	x9,x8,x8,lsr#2	// s1 = r1 + (r1 >> 2)+	csel	x4,x4,x12,eq		// choose between radixes+	csel	x5,x5,x13,eq+	csel	x6,x6,x14,eq++.Loop:+	ldp	x10,x11,[x1],#16	// load input+	sub	x2,x2,#16+#ifdef	__AARCH64EB__+	rev	x10,x10+	rev	x11,x11+#endif+	adds	x4,x4,x10		// accumulate input+	adcs	x5,x5,x11++	mul	x12,x4,x7		// h0*r0+	adc	x6,x6,x3+	umulh	x13,x4,x7++	mul	x10,x5,x9		// h1*5*r1+	umulh	x11,x5,x9++	adds	x12,x12,x10+	mul	x10,x4,x8		// h0*r1+	adc	x13,x13,x11+	umulh	x14,x4,x8++	adds	x13,x13,x10+	mul	x10,x5,x7		// h1*r0+	adc	x14,x14,xzr+	umulh	x11,x5,x7++	adds	x13,x13,x10+	mul	x10,x6,x9		// h2*5*r1+	adc	x14,x14,x11+	mul	x11,x6,x7		// h2*r0++	adds	x13,x13,x10+	adc	x14,x14,x11++	and	x10,x14,#-4		// final reduction+	and	x6,x14,#3+	add	x10,x10,x14,lsr#2+	adds	x4,x12,x10+	adcs	x5,x13,xzr+	adc	x6,x6,xzr++	cbnz	x2,.Loop++	stp	x4,x5,[x0]		// store hash value+	stp	x6,xzr,[x0,#16]	// [and clear is_base2_26]++.Lno_data:+	ret+.size	crypton_poly1305_asm_blocks,.-crypton_poly1305_asm_blocks++.type	crypton_poly1305_asm_emit,%function+.align	5+crypton_poly1305_asm_emit:+.Lcrypton_poly1305_asm_emit:+	ldp	x4,x5,[x0]		// load hash base 2^64+	ldp	x6,x7,[x0,#16]	// [along with is_base2_26]+	ldp	x10,x11,[x2]	// load nonce++#ifdef	__AARCH64EB__+	lsr	x12,x4,#32+	mov	w13,w4+	lsr	x14,x5,#32+	mov	w15,w5+	lsr	x16,x6,#32+#else+	mov	w12,w4+	lsr	x13,x4,#32+	mov	w14,w5+	lsr	x15,x5,#32+	mov	w16,w6+#endif++	add	x12,x12,x13,lsl#26	// base 2^26 -> base 2^64+	lsr	x13,x14,#12+	adds	x12,x12,x14,lsl#52+	add	x13,x13,x15,lsl#14+	adc	x13,x13,xzr+	lsr	x14,x16,#24+	adds	x13,x13,x16,lsl#40+	adc	x14,x14,xzr++	cmp	x7,#0			// is_base2_26?+	csel	x4,x4,x12,eq		// choose between radixes+	csel	x5,x5,x13,eq+	csel	x6,x6,x14,eq++	adds	x12,x4,#5		// compare to modulus+	adcs	x13,x5,xzr+	adc	x14,x6,xzr++	tst	x14,#-4			// see if it's carried/borrowed++	csel	x4,x4,x12,eq+	csel	x5,x5,x13,eq++#ifdef	__AARCH64EB__+	ror	x10,x10,#32		// flip nonce words+	ror	x11,x11,#32+#endif+	adds	x4,x4,x10		// accumulate nonce+	adc	x5,x5,x11+#ifdef	__AARCH64EB__+	rev	x4,x4			// flip output bytes+	rev	x5,x5+#endif+	stp	x4,x5,[x1]		// write result++	ret+.size	crypton_poly1305_asm_emit,.-crypton_poly1305_asm_emit+.type	crypton_poly1305_asm_mult,%function+.align	5+crypton_poly1305_asm_mult:+	mul	x12,x4,x7		// h0*r0+	umulh	x13,x4,x7++	mul	x10,x5,x9		// h1*5*r1+	umulh	x11,x5,x9++	adds	x12,x12,x10+	mul	x10,x4,x8		// h0*r1+	adc	x13,x13,x11+	umulh	x14,x4,x8++	adds	x13,x13,x10+	mul	x10,x5,x7		// h1*r0+	adc	x14,x14,xzr+	umulh	x11,x5,x7++	adds	x13,x13,x10+	mul	x10,x6,x9		// h2*5*r1+	adc	x14,x14,x11+	mul	x11,x6,x7		// h2*r0++	adds	x13,x13,x10+	adc	x14,x14,x11++	and	x10,x14,#-4		// final reduction+	and	x6,x14,#3+	add	x10,x10,x14,lsr#2+	adds	x4,x12,x10+	adcs	x5,x13,xzr+	adc	x6,x6,xzr++	ret+.size	crypton_poly1305_asm_mult,.-crypton_poly1305_asm_mult++.type	crypton_poly1305_asm_splat,%function+.align	4+crypton_poly1305_asm_splat:+	and	x12,x4,#0x03ffffff	// base 2^64 -> base 2^26+	ubfx	x13,x4,#26,#26+	extr	x14,x5,x4,#52+	and	x14,x14,#0x03ffffff+	ubfx	x15,x5,#14,#26+	extr	x16,x6,x5,#40++	str	w12,[x0,#16*0]	// r0+	add	w12,w13,w13,lsl#2	// r1*5+	str	w13,[x0,#16*1]	// r1+	add	w13,w14,w14,lsl#2	// r2*5+	str	w12,[x0,#16*2]	// s1+	str	w14,[x0,#16*3]	// r2+	add	w14,w15,w15,lsl#2	// r3*5+	str	w13,[x0,#16*4]	// s2+	str	w15,[x0,#16*5]	// r3+	add	w15,w16,w16,lsl#2	// r4*5+	str	w14,[x0,#16*6]	// s3+	str	w16,[x0,#16*7]	// r4+	str	w15,[x0,#16*8]	// s4++	ret+.size	crypton_poly1305_asm_splat,.-crypton_poly1305_asm_splat++#ifdef	__KERNEL__+.globl	crypton_poly1305_asm_blocks_neon+#endif+.type	crypton_poly1305_asm_blocks_neon,%function+.align	5+crypton_poly1305_asm_blocks_neon:+.Lcrypton_poly1305_asm_blocks_neon:+	ldr	x17,[x0,#24]+	cmp	x2,#128+	b.lo	.Lcrypton_poly1305_asm_blocks++.inst	0xd503233f		// paciasp+	stp	x29,x30,[sp,#-2*__SIZEOF_POINTER__-64]!+	add	x29,sp,#0++	stp	d8,d9,[sp,#2*__SIZEOF_POINTER__+0]	// meet ABI requirements+	stp	d10,d11,[sp,#2*__SIZEOF_POINTER__+16]+	stp	d12,d13,[sp,#2*__SIZEOF_POINTER__+32]+	stp	d14,d15,[sp,#2*__SIZEOF_POINTER__+48]++	cbz	x17,.Lbase2_64_neon++	ldp	w10,w11,[x0]		// load hash value base 2^26+	ldp	w12,w13,[x0,#8]+	ldr	w14,[x0,#16]++	tst	x2,#31+	b.eq	.Leven_neon++	ldp	x7,x8,[x0,#32]	// load key value++	add	x4,x10,x11,lsl#26	// base 2^26 -> base 2^64+	lsr	x5,x12,#12+	adds	x4,x4,x12,lsl#52+	add	x5,x5,x13,lsl#14+	adc	x5,x5,xzr+	lsr	x6,x14,#24+	adds	x5,x5,x14,lsl#40+	adc	x14,x6,xzr		// can be partially reduced...++	ldp	x12,x13,[x1],#16	// load input+	sub	x2,x2,#16+	add	x9,x8,x8,lsr#2	// s1 = r1 + (r1 >> 2)++#ifdef	__AARCH64EB__+	rev	x12,x12+	rev	x13,x13+#endif+	adds	x4,x4,x12		// accumulate input+	adcs	x5,x5,x13+	adc	x6,x6,x3++	bl	crypton_poly1305_asm_mult++	and	x10,x4,#0x03ffffff	// base 2^64 -> base 2^26+	ubfx	x11,x4,#26,#26+	extr	x12,x5,x4,#52+	and	x12,x12,#0x03ffffff+	ubfx	x13,x5,#14,#26+	extr	x14,x6,x5,#40++	b	.Leven_neon++.align	4+.Lbase2_64_neon:+	ldp	x7,x8,[x0,#32]	// load key value++	ldp	x4,x5,[x0]		// load hash value base 2^64+	ldr	x6,[x0,#16]++	tst	x2,#31+	b.eq	.Linit_neon++	ldp	x12,x13,[x1],#16	// load input+	sub	x2,x2,#16+	add	x9,x8,x8,lsr#2	// s1 = r1 + (r1 >> 2)+#ifdef	__AARCH64EB__+	rev	x12,x12+	rev	x13,x13+#endif+	adds	x4,x4,x12		// accumulate input+	adcs	x5,x5,x13+	adc	x6,x6,x3++	bl	crypton_poly1305_asm_mult++.Linit_neon:+	ldr	w17,[x0,#48]		// first table element+	and	x10,x4,#0x03ffffff	// base 2^64 -> base 2^26+	ubfx	x11,x4,#26,#26+	extr	x12,x5,x4,#52+	and	x12,x12,#0x03ffffff+	ubfx	x13,x5,#14,#26+	extr	x14,x6,x5,#40++	cmp	w17,#-1			// is value impossible?+	b.ne	.Leven_neon++	fmov	d24,x10+	fmov	d25,x11+	fmov	d26,x12+	fmov	d27,x13+	fmov	d28,x14++	////////////////////////////////// initialize r^n table+	mov	x4,x7			// r^1+	add	x9,x8,x8,lsr#2	// s1 = r1 + (r1 >> 2)+	mov	x5,x8+	mov	x6,xzr+	add	x0,x0,#48+12+	bl	crypton_poly1305_asm_splat++	bl	crypton_poly1305_asm_mult		// r^2+	sub	x0,x0,#4+	bl	crypton_poly1305_asm_splat++	bl	crypton_poly1305_asm_mult		// r^3+	sub	x0,x0,#4+	bl	crypton_poly1305_asm_splat++	bl	crypton_poly1305_asm_mult		// r^4+	sub	x0,x0,#4+	bl	crypton_poly1305_asm_splat+	sub	x0,x0,#48+	b	.Ldo_neon++.align	4+.Leven_neon:+	fmov	d24,x10+	fmov	d25,x11+	fmov	d26,x12+	fmov	d27,x13+	fmov	d28,x14++.Ldo_neon:+	ldp	x8,x12,[x1,#32]	// inp[2:3]+	subs	x2,x2,#64+	ldp	x9,x13,[x1,#48]+	add	x16,x1,#96+	adr	x17,.Lzeros++	lsl	x3,x3,#24+	add	x15,x0,#48++#ifdef	__AARCH64EB__+	rev	x8,x8+	rev	x12,x12+	rev	x9,x9+	rev	x13,x13+#endif+	and	x4,x8,#0x03ffffff	// base 2^64 -> base 2^26+	and	x5,x9,#0x03ffffff+	ubfx	x6,x8,#26,#26+	ubfx	x7,x9,#26,#26+	add	x4,x4,x5,lsl#32		// bfi	x4,x5,#32,#32+	extr	x8,x12,x8,#52+	extr	x9,x13,x9,#52+	add	x6,x6,x7,lsl#32		// bfi	x6,x7,#32,#32+	fmov	d14,x4+	and	x8,x8,#0x03ffffff+	and	x9,x9,#0x03ffffff+	ubfx	x10,x12,#14,#26+	ubfx	x11,x13,#14,#26+	add	x12,x3,x12,lsr#40+	add	x13,x3,x13,lsr#40+	add	x8,x8,x9,lsl#32		// bfi	x8,x9,#32,#32+	fmov	d15,x6+	add	x10,x10,x11,lsl#32	// bfi	x10,x11,#32,#32+	add	x12,x12,x13,lsl#32	// bfi	x12,x13,#32,#32+	fmov	d16,x8+	fmov	d17,x10+	fmov	d18,x12++	ldp	x8,x12,[x1],#16	// inp[0:1]+	ldp	x9,x13,[x1],#48++	ld1	{v0.4s,v1.4s,v2.4s,v3.4s},[x15],#64+	ld1	{v4.4s,v5.4s,v6.4s,v7.4s},[x15],#64+	ld1	{v8.4s},[x15]++#ifdef	__AARCH64EB__+	rev	x8,x8+	rev	x12,x12+	rev	x9,x9+	rev	x13,x13+#endif+	and	x4,x8,#0x03ffffff	// base 2^64 -> base 2^26+	and	x5,x9,#0x03ffffff+	ubfx	x6,x8,#26,#26+	ubfx	x7,x9,#26,#26+	add	x4,x4,x5,lsl#32		// bfi	x4,x5,#32,#32+	extr	x8,x12,x8,#52+	extr	x9,x13,x9,#52+	add	x6,x6,x7,lsl#32		// bfi	x6,x7,#32,#32+	fmov	d9,x4+	and	x8,x8,#0x03ffffff+	and	x9,x9,#0x03ffffff+	ubfx	x10,x12,#14,#26+	ubfx	x11,x13,#14,#26+	add	x12,x3,x12,lsr#40+	add	x13,x3,x13,lsr#40+	add	x8,x8,x9,lsl#32		// bfi	x8,x9,#32,#32+	fmov	d10,x6+	add	x10,x10,x11,lsl#32	// bfi	x10,x11,#32,#32+	add	x12,x12,x13,lsl#32	// bfi	x12,x13,#32,#32+	movi	v31.2d,#-1+	fmov	d11,x8+	fmov	d12,x10+	fmov	d13,x12+	ushr	v31.2d,v31.2d,#38++	b.ls	.Lskip_loop++.align	4+.Loop_neon:+	////////////////////////////////////////////////////////////////+	// ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2+	// ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^3+inp[7]*r+	//   ___________________/+	// ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2+inp[8])*r^2+	// ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^4+inp[7]*r^2+inp[9])*r+	//   ___________________/ ____________________/+	//+	// Note that we start with inp[2:3]*r^2. This is because it+	// doesn't depend on reduction in previous iteration.+	////////////////////////////////////////////////////////////////+	// d4 = h0*r4 + h1*r3   + h2*r2   + h3*r1   + h4*r0+	// d3 = h0*r3 + h1*r2   + h2*r1   + h3*r0   + h4*5*r4+	// d2 = h0*r2 + h1*r1   + h2*r0   + h3*5*r4 + h4*5*r3+	// d1 = h0*r1 + h1*r0   + h2*5*r4 + h3*5*r3 + h4*5*r2+	// d0 = h0*r0 + h1*5*r4 + h2*5*r3 + h3*5*r2 + h4*5*r1++	subs	x2,x2,#64+	umull	v23.2d,v14.2s,v7.s[2]+	csel	x16,x17,x16,lo+	umull	v22.2d,v14.2s,v5.s[2]+	umull	v21.2d,v14.2s,v3.s[2]+	ldp	x8,x12,[x16],#16	// inp[2:3] (or zero)+	umull	v20.2d,v14.2s,v1.s[2]+	ldp	x9,x13,[x16],#48+	umull	v19.2d,v14.2s,v0.s[2]+#ifdef	__AARCH64EB__+	rev	x8,x8+	rev	x12,x12+	rev	x9,x9+	rev	x13,x13+#endif++	umlal	v23.2d,v15.2s,v5.s[2]+	and	x4,x8,#0x03ffffff	// base 2^64 -> base 2^26+	umlal	v22.2d,v15.2s,v3.s[2]+	and	x5,x9,#0x03ffffff+	umlal	v21.2d,v15.2s,v1.s[2]+	ubfx	x6,x8,#26,#26+	umlal	v20.2d,v15.2s,v0.s[2]+	ubfx	x7,x9,#26,#26+	umlal	v19.2d,v15.2s,v8.s[2]+	add	x4,x4,x5,lsl#32		// bfi	x4,x5,#32,#32++	umlal	v23.2d,v16.2s,v3.s[2]+	extr	x8,x12,x8,#52+	umlal	v22.2d,v16.2s,v1.s[2]+	extr	x9,x13,x9,#52+	umlal	v21.2d,v16.2s,v0.s[2]+	add	x6,x6,x7,lsl#32		// bfi	x6,x7,#32,#32+	umlal	v20.2d,v16.2s,v8.s[2]+	fmov	d14,x4+	umlal	v19.2d,v16.2s,v6.s[2]+	and	x8,x8,#0x03ffffff++	umlal	v23.2d,v17.2s,v1.s[2]+	and	x9,x9,#0x03ffffff+	umlal	v22.2d,v17.2s,v0.s[2]+	ubfx	x10,x12,#14,#26+	umlal	v21.2d,v17.2s,v8.s[2]+	ubfx	x11,x13,#14,#26+	umlal	v20.2d,v17.2s,v6.s[2]+	add	x8,x8,x9,lsl#32		// bfi	x8,x9,#32,#32+	umlal	v19.2d,v17.2s,v4.s[2]+	fmov	d15,x6++	add	v11.2s,v11.2s,v26.2s+	add	x12,x3,x12,lsr#40+	umlal	v23.2d,v18.2s,v0.s[2]+	add	x13,x3,x13,lsr#40+	umlal	v22.2d,v18.2s,v8.s[2]+	add	x10,x10,x11,lsl#32	// bfi	x10,x11,#32,#32+	umlal	v21.2d,v18.2s,v6.s[2]+	add	x12,x12,x13,lsl#32	// bfi	x12,x13,#32,#32+	umlal	v20.2d,v18.2s,v4.s[2]+	fmov	d16,x8+	umlal	v19.2d,v18.2s,v2.s[2]+	fmov	d17,x10++	////////////////////////////////////////////////////////////////+	// (hash+inp[0:1])*r^4 and accumulate++	add	v9.2s,v9.2s,v24.2s+	fmov	d18,x12+	umlal	v22.2d,v11.2s,v1.s[0]+	ldp	x8,x12,[x1],#16	// inp[0:1]+	umlal	v19.2d,v11.2s,v6.s[0]+	ldp	x9,x13,[x1],#48+	umlal	v23.2d,v11.2s,v3.s[0]+	umlal	v20.2d,v11.2s,v8.s[0]+	umlal	v21.2d,v11.2s,v0.s[0]+#ifdef	__AARCH64EB__+	rev	x8,x8+	rev	x12,x12+	rev	x9,x9+	rev	x13,x13+#endif++	add	v10.2s,v10.2s,v25.2s+	umlal	v22.2d,v9.2s,v5.s[0]+	umlal	v23.2d,v9.2s,v7.s[0]+	and	x4,x8,#0x03ffffff	// base 2^64 -> base 2^26+	umlal	v21.2d,v9.2s,v3.s[0]+	and	x5,x9,#0x03ffffff+	umlal	v19.2d,v9.2s,v0.s[0]+	ubfx	x6,x8,#26,#26+	umlal	v20.2d,v9.2s,v1.s[0]+	ubfx	x7,x9,#26,#26++	add	v12.2s,v12.2s,v27.2s+	add	x4,x4,x5,lsl#32		// bfi	x4,x5,#32,#32+	umlal	v22.2d,v10.2s,v3.s[0]+	extr	x8,x12,x8,#52+	umlal	v23.2d,v10.2s,v5.s[0]+	extr	x9,x13,x9,#52+	umlal	v19.2d,v10.2s,v8.s[0]+	add	x6,x6,x7,lsl#32		// bfi	x6,x7,#32,#32+	umlal	v21.2d,v10.2s,v1.s[0]+	fmov	d9,x4+	umlal	v20.2d,v10.2s,v0.s[0]+	and	x8,x8,#0x03ffffff++	add	v13.2s,v13.2s,v28.2s+	and	x9,x9,#0x03ffffff+	umlal	v22.2d,v12.2s,v0.s[0]+	ubfx	x10,x12,#14,#26+	umlal	v19.2d,v12.2s,v4.s[0]+	ubfx	x11,x13,#14,#26+	umlal	v23.2d,v12.2s,v1.s[0]+	add	x8,x8,x9,lsl#32		// bfi	x8,x9,#32,#32+	umlal	v20.2d,v12.2s,v6.s[0]+	fmov	d10,x6+	umlal	v21.2d,v12.2s,v8.s[0]+	add	x12,x3,x12,lsr#40++	umlal	v22.2d,v13.2s,v8.s[0]+	add	x13,x3,x13,lsr#40+	umlal	v19.2d,v13.2s,v2.s[0]+	add	x10,x10,x11,lsl#32	// bfi	x10,x11,#32,#32+	umlal	v23.2d,v13.2s,v0.s[0]+	add	x12,x12,x13,lsl#32	// bfi	x12,x13,#32,#32+	umlal	v20.2d,v13.2s,v4.s[0]+	fmov	d11,x8+	umlal	v21.2d,v13.2s,v6.s[0]+	fmov	d12,x10+	fmov	d13,x12++	/////////////////////////////////////////////////////////////////+	// lazy reduction as discussed in "NEON crypto" by D.J. Bernstein+	// and P. Schwabe+	//+	// [see discussion in poly1305-armv4 module]++	ushr	v29.2d,v22.2d,#26+	xtn	v27.2s,v22.2d+	ushr	v30.2d,v19.2d,#26+	and	v19.16b,v19.16b,v31.16b+	add	v23.2d,v23.2d,v29.2d	// h3 -> h4+	bic	v27.2s,#0xfc,lsl#24	// &=0x03ffffff+	add	v20.2d,v20.2d,v30.2d	// h0 -> h1++	ushr	v29.2d,v23.2d,#26+	xtn	v28.2s,v23.2d+	ushr	v30.2d,v20.2d,#26+	xtn	v25.2s,v20.2d+	bic	v28.2s,#0xfc,lsl#24+	add	v21.2d,v21.2d,v30.2d	// h1 -> h2++	add	v19.2d,v19.2d,v29.2d+	shl	v29.2d,v29.2d,#2+	shrn	v30.2s,v21.2d,#26+	xtn	v26.2s,v21.2d+	add	v19.2d,v19.2d,v29.2d	// h4 -> h0+	bic	v25.2s,#0xfc,lsl#24+	add	v27.2s,v27.2s,v30.2s		// h2 -> h3+	bic	v26.2s,#0xfc,lsl#24++	shrn	v29.2s,v19.2d,#26+	xtn	v24.2s,v19.2d+	ushr	v30.2s,v27.2s,#26+	bic	v27.2s,#0xfc,lsl#24+	bic	v24.2s,#0xfc,lsl#24+	add	v25.2s,v25.2s,v29.2s		// h0 -> h1+	add	v28.2s,v28.2s,v30.2s		// h3 -> h4++	b.hi	.Loop_neon++.Lskip_loop:+	dup	v16.2d,v16.d[0]+	add	v11.2s,v11.2s,v26.2s++	////////////////////////////////////////////////////////////////+	// multiply (inp[0:1]+hash) or inp[2:3] by r^2:r^1++	adds	x2,x2,#32+	b.ne	.Long_tail++	dup	v16.2d,v11.d[0]+	add	v14.2s,v9.2s,v24.2s+	add	v17.2s,v12.2s,v27.2s+	add	v15.2s,v10.2s,v25.2s+	add	v18.2s,v13.2s,v28.2s++.Long_tail:+	dup	v14.2d,v14.d[0]+	umull2	v19.2d,v16.4s,v6.4s+	umull2	v22.2d,v16.4s,v1.4s+	umull2	v23.2d,v16.4s,v3.4s+	umull2	v21.2d,v16.4s,v0.4s+	umull2	v20.2d,v16.4s,v8.4s++	dup	v15.2d,v15.d[0]+	umlal2	v19.2d,v14.4s,v0.4s+	umlal2	v21.2d,v14.4s,v3.4s+	umlal2	v22.2d,v14.4s,v5.4s+	umlal2	v23.2d,v14.4s,v7.4s+	umlal2	v20.2d,v14.4s,v1.4s++	dup	v17.2d,v17.d[0]+	umlal2	v19.2d,v15.4s,v8.4s+	umlal2	v22.2d,v15.4s,v3.4s+	umlal2	v21.2d,v15.4s,v1.4s+	umlal2	v23.2d,v15.4s,v5.4s+	umlal2	v20.2d,v15.4s,v0.4s++	dup	v18.2d,v18.d[0]+	umlal2	v22.2d,v17.4s,v0.4s+	umlal2	v23.2d,v17.4s,v1.4s+	umlal2	v19.2d,v17.4s,v4.4s+	umlal2	v20.2d,v17.4s,v6.4s+	umlal2	v21.2d,v17.4s,v8.4s++	umlal2	v22.2d,v18.4s,v8.4s+	umlal2	v19.2d,v18.4s,v2.4s+	umlal2	v23.2d,v18.4s,v0.4s+	umlal2	v20.2d,v18.4s,v4.4s+	umlal2	v21.2d,v18.4s,v6.4s++	b.eq	.Lshort_tail++	////////////////////////////////////////////////////////////////+	// (hash+inp[0:1])*r^4:r^3 and accumulate++	add	v9.2s,v9.2s,v24.2s+	umlal	v22.2d,v11.2s,v1.2s+	umlal	v19.2d,v11.2s,v6.2s+	umlal	v23.2d,v11.2s,v3.2s+	umlal	v20.2d,v11.2s,v8.2s+	umlal	v21.2d,v11.2s,v0.2s++	add	v10.2s,v10.2s,v25.2s+	umlal	v22.2d,v9.2s,v5.2s+	umlal	v19.2d,v9.2s,v0.2s+	umlal	v23.2d,v9.2s,v7.2s+	umlal	v20.2d,v9.2s,v1.2s+	umlal	v21.2d,v9.2s,v3.2s++	add	v12.2s,v12.2s,v27.2s+	umlal	v22.2d,v10.2s,v3.2s+	umlal	v19.2d,v10.2s,v8.2s+	umlal	v23.2d,v10.2s,v5.2s+	umlal	v20.2d,v10.2s,v0.2s+	umlal	v21.2d,v10.2s,v1.2s++	add	v13.2s,v13.2s,v28.2s+	umlal	v22.2d,v12.2s,v0.2s+	umlal	v19.2d,v12.2s,v4.2s+	umlal	v23.2d,v12.2s,v1.2s+	umlal	v20.2d,v12.2s,v6.2s+	umlal	v21.2d,v12.2s,v8.2s++	umlal	v22.2d,v13.2s,v8.2s+	umlal	v19.2d,v13.2s,v2.2s+	umlal	v23.2d,v13.2s,v0.2s+	umlal	v20.2d,v13.2s,v4.2s+	umlal	v21.2d,v13.2s,v6.2s++.Lshort_tail:+	////////////////////////////////////////////////////////////////+	// horizontal add++	addp	v22.2d,v22.2d,v22.2d+	ldp	d8,d9,[sp,#2*__SIZEOF_POINTER__+0]	// meet ABI requirements+	addp	v19.2d,v19.2d,v19.2d+	ldp	d10,d11,[sp,#2*__SIZEOF_POINTER__+16]+	addp	v23.2d,v23.2d,v23.2d+	ldp	d12,d13,[sp,#2*__SIZEOF_POINTER__+32]+	addp	v20.2d,v20.2d,v20.2d+	ldp	d14,d15,[sp,#2*__SIZEOF_POINTER__+48]+	addp	v21.2d,v21.2d,v21.2d+	ldr	x30,[sp,#__SIZEOF_POINTER__]++	////////////////////////////////////////////////////////////////+	// lazy reduction, but without narrowing++	ushr	v29.2d,v22.2d,#26+	and	v22.16b,v22.16b,v31.16b+	ushr	v30.2d,v19.2d,#26+	and	v19.16b,v19.16b,v31.16b++	add	v23.2d,v23.2d,v29.2d	// h3 -> h4+	add	v20.2d,v20.2d,v30.2d	// h0 -> h1++	ushr	v29.2d,v23.2d,#26+	and	v23.16b,v23.16b,v31.16b+	ushr	v30.2d,v20.2d,#26+	and	v20.16b,v20.16b,v31.16b+	add	v21.2d,v21.2d,v30.2d	// h1 -> h2++	add	v19.2d,v19.2d,v29.2d+	shl	v29.2d,v29.2d,#2+	ushr	v30.2d,v21.2d,#26+	and	v21.16b,v21.16b,v31.16b+	add	v19.2d,v19.2d,v29.2d	// h4 -> h0+	add	v22.2d,v22.2d,v30.2d	// h2 -> h3++	ushr	v29.2d,v19.2d,#26+	and	v19.16b,v19.16b,v31.16b+	ushr	v30.2d,v22.2d,#26+	and	v22.16b,v22.16b,v31.16b+	add	v20.2d,v20.2d,v29.2d	// h0 -> h1+	add	v23.2d,v23.2d,v30.2d	// h3 -> h4++	////////////////////////////////////////////////////////////////+	// write the result, can be partially reduced++	st4	{v19.s,v20.s,v21.s,v22.s}[0],[x0],#16+	mov	x4,#1+	st1	{v23.s}[0],[x0]+	str	x4,[x0,#8]		// set is_base2_26++	ldr	x29,[sp],#2*__SIZEOF_POINTER__+64+.inst	0xd50323bf		// autiasp+	ret+.size	crypton_poly1305_asm_blocks_neon,.-crypton_poly1305_asm_blocks_neon++.align	5+.Lzeros:+.long	0,0,0,0,0,0,0,0+.byte	80,111,108,121,49,51,48,53,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.align	2+.align	2+#if !defined(__KERNEL__) && !defined(_WIN64)+.comm	crypton_armcap_P,4,4+.hidden	crypton_armcap_P+#endif++.section	.note.GNU-stack,"",%progbits
+ cbits/asm/poly1305-armv8.pl view
@@ -0,0 +1,927 @@+#!/usr/bin/env perl+# SPDX-License-Identifier: GPL-1.0+ OR BSD-3-Clause+#+# ====================================================================+# Written by Andy Polyakov, @dot-asm, initially for the OpenSSL+# project.+# ====================================================================+#+# This module implements Poly1305 hash for ARMv8.+#+# June 2015+#+# Numbers are cycles per processed byte with poly1305_blocks alone.+#+#		IALU/gcc-4.9	NEON+#+# Apple A7	1.86/+5%	0.72+# Apple A10			0.71+# Apple A14/M1	0.97/+63%	0.48+# Cortex-A53	2.69/+58%	1.47+# Cortex-A57	2.70/+7%	1.14+# Cortex-A76	2.60		1.00+# Cortex-X2	1.00		0.66+# Cortex-X925	1.00		0.53+# Denver	1.64/+50%	1.18(*)+# X-Gene	2.13/+68%	2.27+# Mongoose	1.77/+75%	1.12+# Kryo		2.70/+55%	1.13+# ThunderX2	1.17/+95%	1.36+# Snapdragon X	0.95		0.48+#+# (*)	estimate based on resources availability is less than 1.0,+#	i.e. measured result is worse than expected, presumably binary+#	translator is not almighty;++$flavour=shift;+$output=shift;++if ($flavour && $flavour ne "void") {+    $0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1;+    ( $xlate="${dir}arm-xlate.pl" and -f $xlate ) or+    ( $xlate="${dir}../../perlasm/arm-xlate.pl" and -f $xlate) or+    die "can't locate arm-xlate.pl";++    open STDOUT,"| \"$^X\" $xlate $flavour $output";+} else {+    open STDOUT,">$output";+}++my ($ctx,$inp,$len,$padbit) = map("x$_",(0..3));+my ($mac,$nonce)=($inp,$len);++my ($h0,$h1,$h2,$r0,$r1,$s1,$t0,$t1,$d0,$d1,$d2) = map("x$_",(4..14));++$code.=<<___;+#ifndef __KERNEL__+# include "arm_arch.h"+.extern	OPENSSL_armcap_P+#endif++.text++// forward "declarations" are required for Apple+.globl	poly1305_blocks+.globl	poly1305_emit++.globl	poly1305_init+.type	poly1305_init,%function+.align	5+poly1305_init:+	cmp	$inp,xzr+	stp	xzr,xzr,[$ctx]		// zero hash value+	stp	xzr,xzr,[$ctx,#16]	// [along with is_base2_26]++	csel	c0,czr,c0,eq+	b.eq	.Lno_key++#ifndef	__KERNEL__+	adrp	c17,OPENSSL_armcap_P+	ldr	w17,[c17,#:lo12:OPENSSL_armcap_P]+#endif++	ldp	$r0,$r1,[$inp]		// load key+	mov	$s1,#0xfffffffc0fffffff+	movk	$s1,#0x0fff,lsl#48+#ifdef	__AARCH64EB__+	rev	$r0,$r0			// flip bytes+	rev	$r1,$r1+#endif+	and	$r0,$r0,$s1		// &=0ffffffc0fffffff+	and	$s1,$s1,#-4+	and	$r1,$r1,$s1		// &=0ffffffc0ffffffc+	mov	w#$s1,#-1+	stp	$r0,$r1,[$ctx,#32]	// save key value+	str	w#$s1,[$ctx,#48]	// impossible key power value++#ifndef	__KERNEL__+	tst	w17,#ARMV7_NEON++	adr	c13,.Lpoly1305_blocks+	adr	c15,.Lpoly1305_blocks_neon+	adr	c14,.Lpoly1305_emit++	csel	c13,c13,c15,eq+# ifdef	__CHERI_PURE_CAPABILITY__+	add	c13, c13, #1+	add	c14, c14, #1+	seal	c13, c13, rb+	seal	c14, c14, rb+# endif++# ifdef	__ILP32__+	stp	w13,w14,[$len]+# else+	stp	c13,c14,[$len]+# endif+	mov	x0,#1+#else+	mov	x0,#0+#endif+.Lno_key:+	ret+.size	poly1305_init,.-poly1305_init++.type	poly1305_blocks,%function+.align	5+poly1305_blocks:+.Lpoly1305_blocks:+	ands	$len,$len,#-16+	b.eq	.Lno_data++	ldp	$h0,$h1,[$ctx]		// load hash value+	ldp	$h2,x17,[$ctx,#16]	// [along with is_base2_26]+	ldp	$r0,$r1,[$ctx,#32]	// load key value++#ifdef	__AARCH64EB__+	lsr	$d0,$h0,#32+	mov	w#$d1,w#$h0+	lsr	$d2,$h1,#32+	mov	w15,w#$h1+	lsr	x16,$h2,#32+#else+	mov	w#$d0,w#$h0+	lsr	$d1,$h0,#32+	mov	w#$d2,w#$h1+	lsr	x15,$h1,#32+	mov	w16,w#$h2+#endif++	add	$d0,$d0,$d1,lsl#26	// base 2^26 -> base 2^64+	lsr	$d1,$d2,#12+	adds	$d0,$d0,$d2,lsl#52+	add	$d1,$d1,x15,lsl#14+	adc	$d1,$d1,xzr+	lsr	$d2,x16,#24+	adds	$d1,$d1,x16,lsl#40+	adc	$d2,$d2,xzr++	cmp	x17,#0			// is_base2_26?+	add	$s1,$r1,$r1,lsr#2	// s1 = r1 + (r1 >> 2)+	csel	$h0,$h0,$d0,eq		// choose between radixes+	csel	$h1,$h1,$d1,eq+	csel	$h2,$h2,$d2,eq++.Loop:+	ldp	$t0,$t1,[$inp],#16	// load input+	sub	$len,$len,#16+#ifdef	__AARCH64EB__+	rev	$t0,$t0+	rev	$t1,$t1+#endif+	adds	$h0,$h0,$t0		// accumulate input+	adcs	$h1,$h1,$t1++	mul	$d0,$h0,$r0		// h0*r0+	adc	$h2,$h2,$padbit+	umulh	$d1,$h0,$r0++	mul	$t0,$h1,$s1		// h1*5*r1+	umulh	$t1,$h1,$s1++	adds	$d0,$d0,$t0+	mul	$t0,$h0,$r1		// h0*r1+	adc	$d1,$d1,$t1+	umulh	$d2,$h0,$r1++	adds	$d1,$d1,$t0+	mul	$t0,$h1,$r0		// h1*r0+	adc	$d2,$d2,xzr+	umulh	$t1,$h1,$r0++	adds	$d1,$d1,$t0+	mul	$t0,$h2,$s1		// h2*5*r1+	adc	$d2,$d2,$t1+	mul	$t1,$h2,$r0		// h2*r0++	adds	$d1,$d1,$t0+	adc	$d2,$d2,$t1++	and	$t0,$d2,#-4		// final reduction+	and	$h2,$d2,#3+	add	$t0,$t0,$d2,lsr#2+	adds	$h0,$d0,$t0+	adcs	$h1,$d1,xzr+	adc	$h2,$h2,xzr++	cbnz	$len,.Loop++	stp	$h0,$h1,[$ctx]		// store hash value+	stp	$h2,xzr,[$ctx,#16]	// [and clear is_base2_26]++.Lno_data:+	ret+.size	poly1305_blocks,.-poly1305_blocks++.type	poly1305_emit,%function+.align	5+poly1305_emit:+.Lpoly1305_emit:+	ldp	$h0,$h1,[$ctx]		// load hash base 2^64+	ldp	$h2,$r0,[$ctx,#16]	// [along with is_base2_26]+	ldp	$t0,$t1,[$nonce]	// load nonce++#ifdef	__AARCH64EB__+	lsr	$d0,$h0,#32+	mov	w#$d1,w#$h0+	lsr	$d2,$h1,#32+	mov	w15,w#$h1+	lsr	x16,$h2,#32+#else+	mov	w#$d0,w#$h0+	lsr	$d1,$h0,#32+	mov	w#$d2,w#$h1+	lsr	x15,$h1,#32+	mov	w16,w#$h2+#endif++	add	$d0,$d0,$d1,lsl#26	// base 2^26 -> base 2^64+	lsr	$d1,$d2,#12+	adds	$d0,$d0,$d2,lsl#52+	add	$d1,$d1,x15,lsl#14+	adc	$d1,$d1,xzr+	lsr	$d2,x16,#24+	adds	$d1,$d1,x16,lsl#40+	adc	$d2,$d2,xzr++	cmp	$r0,#0			// is_base2_26?+	csel	$h0,$h0,$d0,eq		// choose between radixes+	csel	$h1,$h1,$d1,eq+	csel	$h2,$h2,$d2,eq++	adds	$d0,$h0,#5		// compare to modulus+	adcs	$d1,$h1,xzr+	adc	$d2,$h2,xzr++	tst	$d2,#-4			// see if it's carried/borrowed++	csel	$h0,$h0,$d0,eq+	csel	$h1,$h1,$d1,eq++#ifdef	__AARCH64EB__+	ror	$t0,$t0,#32		// flip nonce words+	ror	$t1,$t1,#32+#endif+	adds	$h0,$h0,$t0		// accumulate nonce+	adc	$h1,$h1,$t1+#ifdef	__AARCH64EB__+	rev	$h0,$h0			// flip output bytes+	rev	$h1,$h1+#endif+	stp	$h0,$h1,[$mac]		// write result++	ret+.size	poly1305_emit,.-poly1305_emit+___+my ($R0,$R1,$S1,$R2,$S2,$R3,$S3,$R4,$S4) = map("v$_.4s",(0..8));+my ($IN01_0,$IN01_1,$IN01_2,$IN01_3,$IN01_4) = map("v$_.2s",(9..13));+my ($IN23_0,$IN23_1,$IN23_2,$IN23_3,$IN23_4) = map("v$_.2s",(14..18));+my ($ACC0,$ACC1,$ACC2,$ACC3,$ACC4) = map("v$_.2d",(19..23));+my ($H0,$H1,$H2,$H3,$H4) = map("v$_.2s",(24..28));+my ($T0,$T1,$MASK) = map("v$_",(29..31));++my ($in2,$zeros)=("x16","x17");+my $is_base2_26 = $zeros;		# borrow++$code.=<<___;+.type	poly1305_mult,%function+.align	5+poly1305_mult:+	mul	$d0,$h0,$r0		// h0*r0+	umulh	$d1,$h0,$r0++	mul	$t0,$h1,$s1		// h1*5*r1+	umulh	$t1,$h1,$s1++	adds	$d0,$d0,$t0+	mul	$t0,$h0,$r1		// h0*r1+	adc	$d1,$d1,$t1+	umulh	$d2,$h0,$r1++	adds	$d1,$d1,$t0+	mul	$t0,$h1,$r0		// h1*r0+	adc	$d2,$d2,xzr+	umulh	$t1,$h1,$r0++	adds	$d1,$d1,$t0+	mul	$t0,$h2,$s1		// h2*5*r1+	adc	$d2,$d2,$t1+	mul	$t1,$h2,$r0		// h2*r0++	adds	$d1,$d1,$t0+	adc	$d2,$d2,$t1++	and	$t0,$d2,#-4		// final reduction+	and	$h2,$d2,#3+	add	$t0,$t0,$d2,lsr#2+	adds	$h0,$d0,$t0+	adcs	$h1,$d1,xzr+	adc	$h2,$h2,xzr++	ret+.size	poly1305_mult,.-poly1305_mult++.type	poly1305_splat,%function+.align	4+poly1305_splat:+	and	x12,$h0,#0x03ffffff	// base 2^64 -> base 2^26+	ubfx	x13,$h0,#26,#26+	extr	x14,$h1,$h0,#52+	and	x14,x14,#0x03ffffff+	ubfx	x15,$h1,#14,#26+	extr	x16,$h2,$h1,#40++	str	w12,[$ctx,#16*0]	// r0+	add	w12,w13,w13,lsl#2	// r1*5+	str	w13,[$ctx,#16*1]	// r1+	add	w13,w14,w14,lsl#2	// r2*5+	str	w12,[$ctx,#16*2]	// s1+	str	w14,[$ctx,#16*3]	// r2+	add	w14,w15,w15,lsl#2	// r3*5+	str	w13,[$ctx,#16*4]	// s2+	str	w15,[$ctx,#16*5]	// r3+	add	w15,w16,w16,lsl#2	// r4*5+	str	w14,[$ctx,#16*6]	// s3+	str	w16,[$ctx,#16*7]	// r4+	str	w15,[$ctx,#16*8]	// s4++	ret+.size	poly1305_splat,.-poly1305_splat++#ifdef	__KERNEL__+.globl	poly1305_blocks_neon+#endif+.type	poly1305_blocks_neon,%function+.align	5+poly1305_blocks_neon:+.Lpoly1305_blocks_neon:+	ldr	$is_base2_26,[$ctx,#24]+	cmp	$len,#128+	b.lo	.Lpoly1305_blocks++	.inst	0xd503233f		// paciasp+	stp	c29,c30,[csp,#-2*__SIZEOF_POINTER__-64]!+	add	c29,csp,#0++	stp	d8,d9,[csp,#2*__SIZEOF_POINTER__+0]	// meet ABI requirements+	stp	d10,d11,[csp,#2*__SIZEOF_POINTER__+16]+	stp	d12,d13,[csp,#2*__SIZEOF_POINTER__+32]+	stp	d14,d15,[csp,#2*__SIZEOF_POINTER__+48]++	cbz	$is_base2_26,.Lbase2_64_neon++	ldp	w10,w11,[$ctx]		// load hash value base 2^26+	ldp	w12,w13,[$ctx,#8]+	ldr	w14,[$ctx,#16]++	tst	$len,#31+	b.eq	.Leven_neon++	ldp	$r0,$r1,[$ctx,#32]	// load key value++	add	$h0,x10,x11,lsl#26	// base 2^26 -> base 2^64+	lsr	$h1,x12,#12+	adds	$h0,$h0,x12,lsl#52+	add	$h1,$h1,x13,lsl#14+	adc	$h1,$h1,xzr+	lsr	$h2,x14,#24+	adds	$h1,$h1,x14,lsl#40+	adc	$d2,$h2,xzr		// can be partially reduced...++	ldp	$d0,$d1,[$inp],#16	// load input+	sub	$len,$len,#16+	add	$s1,$r1,$r1,lsr#2	// s1 = r1 + (r1 >> 2)++#ifdef	__AARCH64EB__+	rev	$d0,$d0+	rev	$d1,$d1+#endif+	adds	$h0,$h0,$d0		// accumulate input+	adcs	$h1,$h1,$d1+	adc	$h2,$h2,$padbit++	bl	poly1305_mult++	and	x10,$h0,#0x03ffffff	// base 2^64 -> base 2^26+	ubfx	x11,$h0,#26,#26+	extr	x12,$h1,$h0,#52+	and	x12,x12,#0x03ffffff+	ubfx	x13,$h1,#14,#26+	extr	x14,$h2,$h1,#40++	b	.Leven_neon++.align	4+.Lbase2_64_neon:+	ldp	$r0,$r1,[$ctx,#32]	// load key value++	ldp	$h0,$h1,[$ctx]		// load hash value base 2^64+	ldr	$h2,[$ctx,#16]++	tst	$len,#31+	b.eq	.Linit_neon++	ldp	$d0,$d1,[$inp],#16	// load input+	sub	$len,$len,#16+	add	$s1,$r1,$r1,lsr#2	// s1 = r1 + (r1 >> 2)+#ifdef	__AARCH64EB__+	rev	$d0,$d0+	rev	$d1,$d1+#endif+	adds	$h0,$h0,$d0		// accumulate input+	adcs	$h1,$h1,$d1+	adc	$h2,$h2,$padbit++	bl	poly1305_mult++.Linit_neon:+	ldr	w17,[$ctx,#48]		// first table element+	and	x10,$h0,#0x03ffffff	// base 2^64 -> base 2^26+	ubfx	x11,$h0,#26,#26+	extr	x12,$h1,$h0,#52+	and	x12,x12,#0x03ffffff+	ubfx	x13,$h1,#14,#26+	extr	x14,$h2,$h1,#40++	cmp	w17,#-1			// is value impossible?+	b.ne	.Leven_neon++	fmov	${H0},x10+	fmov	${H1},x11+	fmov	${H2},x12+	fmov	${H3},x13+	fmov	${H4},x14++	////////////////////////////////// initialize r^n table+	mov	$h0,$r0			// r^1+	add	$s1,$r1,$r1,lsr#2	// s1 = r1 + (r1 >> 2)+	mov	$h1,$r1+	mov	$h2,xzr+	cadd	$ctx,$ctx,#48+12+	bl	poly1305_splat++	bl	poly1305_mult		// r^2+	csub	$ctx,$ctx,#4+	bl	poly1305_splat++	bl	poly1305_mult		// r^3+	csub	$ctx,$ctx,#4+	bl	poly1305_splat++	bl	poly1305_mult		// r^4+	csub	$ctx,$ctx,#4+	bl	poly1305_splat+	csub	$ctx,$ctx,#48		// restore original $ctx+	b	.Ldo_neon++.align	4+.Leven_neon:+	fmov	${H0},x10+	fmov	${H1},x11+	fmov	${H2},x12+	fmov	${H3},x13+	fmov	${H4},x14++.Ldo_neon:+	ldp	x8,x12,[$inp,#32]	// inp[2:3]+	subs	$len,$len,#64+	ldp	x9,x13,[$inp,#48]+	cadd	$in2,$inp,#96+	adr	$zeros,.Lzeros++	lsl	$padbit,$padbit,#24+	cadd	x15,$ctx,#48++#ifdef	__AARCH64EB__+	rev	x8,x8+	rev	x12,x12+	rev	x9,x9+	rev	x13,x13+#endif+	and	x4,x8,#0x03ffffff	// base 2^64 -> base 2^26+	and	x5,x9,#0x03ffffff+	ubfx	x6,x8,#26,#26+	ubfx	x7,x9,#26,#26+	add	x4,x4,x5,lsl#32		// bfi	x4,x5,#32,#32+	extr	x8,x12,x8,#52+	extr	x9,x13,x9,#52+	add	x6,x6,x7,lsl#32		// bfi	x6,x7,#32,#32+	fmov	$IN23_0,x4+	and	x8,x8,#0x03ffffff+	and	x9,x9,#0x03ffffff+	ubfx	x10,x12,#14,#26+	ubfx	x11,x13,#14,#26+	add	x12,$padbit,x12,lsr#40+	add	x13,$padbit,x13,lsr#40+	add	x8,x8,x9,lsl#32		// bfi	x8,x9,#32,#32+	fmov	$IN23_1,x6+	add	x10,x10,x11,lsl#32	// bfi	x10,x11,#32,#32+	add	x12,x12,x13,lsl#32	// bfi	x12,x13,#32,#32+	fmov	$IN23_2,x8+	fmov	$IN23_3,x10+	fmov	$IN23_4,x12++	ldp	x8,x12,[$inp],#16	// inp[0:1]+	ldp	x9,x13,[$inp],#48++	ld1	{$R0,$R1,$S1,$R2},[x15],#64+	ld1	{$S2,$R3,$S3,$R4},[x15],#64+	ld1	{$S4},[x15]++#ifdef	__AARCH64EB__+	rev	x8,x8+	rev	x12,x12+	rev	x9,x9+	rev	x13,x13+#endif+	and	x4,x8,#0x03ffffff	// base 2^64 -> base 2^26+	and	x5,x9,#0x03ffffff+	ubfx	x6,x8,#26,#26+	ubfx	x7,x9,#26,#26+	add	x4,x4,x5,lsl#32		// bfi	x4,x5,#32,#32+	extr	x8,x12,x8,#52+	extr	x9,x13,x9,#52+	add	x6,x6,x7,lsl#32		// bfi	x6,x7,#32,#32+	fmov	$IN01_0,x4+	and	x8,x8,#0x03ffffff+	and	x9,x9,#0x03ffffff+	ubfx	x10,x12,#14,#26+	ubfx	x11,x13,#14,#26+	add	x12,$padbit,x12,lsr#40+	add	x13,$padbit,x13,lsr#40+	add	x8,x8,x9,lsl#32		// bfi	x8,x9,#32,#32+	fmov	$IN01_1,x6+	add	x10,x10,x11,lsl#32	// bfi	x10,x11,#32,#32+	add	x12,x12,x13,lsl#32	// bfi	x12,x13,#32,#32+	movi	$MASK.2d,#-1+	fmov	$IN01_2,x8+	fmov	$IN01_3,x10+	fmov	$IN01_4,x12+	ushr	$MASK.2d,$MASK.2d,#38++	b.ls	.Lskip_loop++.align	4+.Loop_neon:+	////////////////////////////////////////////////////////////////+	// ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2+	// ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^3+inp[7]*r+	//   \___________________/+	// ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2+inp[8])*r^2+	// ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^4+inp[7]*r^2+inp[9])*r+	//   \___________________/ \____________________/+	//+	// Note that we start with inp[2:3]*r^2. This is because it+	// doesn't depend on reduction in previous iteration.+	////////////////////////////////////////////////////////////////+	// d4 = h0*r4 + h1*r3   + h2*r2   + h3*r1   + h4*r0+	// d3 = h0*r3 + h1*r2   + h2*r1   + h3*r0   + h4*5*r4+	// d2 = h0*r2 + h1*r1   + h2*r0   + h3*5*r4 + h4*5*r3+	// d1 = h0*r1 + h1*r0   + h2*5*r4 + h3*5*r3 + h4*5*r2+	// d0 = h0*r0 + h1*5*r4 + h2*5*r3 + h3*5*r2 + h4*5*r1++	subs	$len,$len,#64+	umull	$ACC4,$IN23_0,${R4}[2]+	csel	c#$in2,c#$zeros,c#$in2,lo+	umull	$ACC3,$IN23_0,${R3}[2]+	umull	$ACC2,$IN23_0,${R2}[2]+	 ldp	x8,x12,[$in2],#16	// inp[2:3] (or zero)+	umull	$ACC1,$IN23_0,${R1}[2]+	 ldp	x9,x13,[$in2],#48+	umull	$ACC0,$IN23_0,${R0}[2]+#ifdef	__AARCH64EB__+	 rev	x8,x8+	 rev	x12,x12+	 rev	x9,x9+	 rev	x13,x13+#endif++	umlal	$ACC4,$IN23_1,${R3}[2]+	 and	x4,x8,#0x03ffffff	// base 2^64 -> base 2^26+	umlal	$ACC3,$IN23_1,${R2}[2]+	 and	x5,x9,#0x03ffffff+	umlal	$ACC2,$IN23_1,${R1}[2]+	 ubfx	x6,x8,#26,#26+	umlal	$ACC1,$IN23_1,${R0}[2]+	 ubfx	x7,x9,#26,#26+	umlal	$ACC0,$IN23_1,${S4}[2]+	 add	x4,x4,x5,lsl#32		// bfi	x4,x5,#32,#32++	umlal	$ACC4,$IN23_2,${R2}[2]+	 extr	x8,x12,x8,#52+	umlal	$ACC3,$IN23_2,${R1}[2]+	 extr	x9,x13,x9,#52+	umlal	$ACC2,$IN23_2,${R0}[2]+	 add	x6,x6,x7,lsl#32		// bfi	x6,x7,#32,#32+	umlal	$ACC1,$IN23_2,${S4}[2]+	 fmov	$IN23_0,x4+	umlal	$ACC0,$IN23_2,${S3}[2]+	 and	x8,x8,#0x03ffffff++	umlal	$ACC4,$IN23_3,${R1}[2]+	 and	x9,x9,#0x03ffffff+	umlal	$ACC3,$IN23_3,${R0}[2]+	 ubfx	x10,x12,#14,#26+	umlal	$ACC2,$IN23_3,${S4}[2]+	 ubfx	x11,x13,#14,#26+	umlal	$ACC1,$IN23_3,${S3}[2]+	 add	x8,x8,x9,lsl#32		// bfi	x8,x9,#32,#32+	umlal	$ACC0,$IN23_3,${S2}[2]+	 fmov	$IN23_1,x6++	add	$IN01_2,$IN01_2,$H2+	 add	x12,$padbit,x12,lsr#40+	umlal	$ACC4,$IN23_4,${R0}[2]+	 add	x13,$padbit,x13,lsr#40+	umlal	$ACC3,$IN23_4,${S4}[2]+	 add	x10,x10,x11,lsl#32	// bfi	x10,x11,#32,#32+	umlal	$ACC2,$IN23_4,${S3}[2]+	 add	x12,x12,x13,lsl#32	// bfi	x12,x13,#32,#32+	umlal	$ACC1,$IN23_4,${S2}[2]+	 fmov	$IN23_2,x8+	umlal	$ACC0,$IN23_4,${S1}[2]+	 fmov	$IN23_3,x10++	////////////////////////////////////////////////////////////////+	// (hash+inp[0:1])*r^4 and accumulate++	add	$IN01_0,$IN01_0,$H0+	 fmov	$IN23_4,x12+	umlal	$ACC3,$IN01_2,${R1}[0]+	 ldp	x8,x12,[$inp],#16	// inp[0:1]+	umlal	$ACC0,$IN01_2,${S3}[0]+	 ldp	x9,x13,[$inp],#48+	umlal	$ACC4,$IN01_2,${R2}[0]+	umlal	$ACC1,$IN01_2,${S4}[0]+	umlal	$ACC2,$IN01_2,${R0}[0]+#ifdef	__AARCH64EB__+	 rev	x8,x8+	 rev	x12,x12+	 rev	x9,x9+	 rev	x13,x13+#endif++	add	$IN01_1,$IN01_1,$H1+	umlal	$ACC3,$IN01_0,${R3}[0]+	umlal	$ACC4,$IN01_0,${R4}[0]+	 and	x4,x8,#0x03ffffff	// base 2^64 -> base 2^26+	umlal	$ACC2,$IN01_0,${R2}[0]+	 and	x5,x9,#0x03ffffff+	umlal	$ACC0,$IN01_0,${R0}[0]+	 ubfx	x6,x8,#26,#26+	umlal	$ACC1,$IN01_0,${R1}[0]+	 ubfx	x7,x9,#26,#26++	add	$IN01_3,$IN01_3,$H3+	 add	x4,x4,x5,lsl#32		// bfi	x4,x5,#32,#32+	umlal	$ACC3,$IN01_1,${R2}[0]+	 extr	x8,x12,x8,#52+	umlal	$ACC4,$IN01_1,${R3}[0]+	 extr	x9,x13,x9,#52+	umlal	$ACC0,$IN01_1,${S4}[0]+	 add	x6,x6,x7,lsl#32		// bfi	x6,x7,#32,#32+	umlal	$ACC2,$IN01_1,${R1}[0]+	 fmov	$IN01_0,x4+	umlal	$ACC1,$IN01_1,${R0}[0]+	 and	x8,x8,#0x03ffffff++	add	$IN01_4,$IN01_4,$H4+	 and	x9,x9,#0x03ffffff+	umlal	$ACC3,$IN01_3,${R0}[0]+	 ubfx	x10,x12,#14,#26+	umlal	$ACC0,$IN01_3,${S2}[0]+	 ubfx	x11,x13,#14,#26+	umlal	$ACC4,$IN01_3,${R1}[0]+	 add	x8,x8,x9,lsl#32		// bfi	x8,x9,#32,#32+	umlal	$ACC1,$IN01_3,${S3}[0]+	 fmov	$IN01_1,x6+	umlal	$ACC2,$IN01_3,${S4}[0]+	 add	x12,$padbit,x12,lsr#40++	umlal	$ACC3,$IN01_4,${S4}[0]+	 add	x13,$padbit,x13,lsr#40+	umlal	$ACC0,$IN01_4,${S1}[0]+	 add	x10,x10,x11,lsl#32	// bfi	x10,x11,#32,#32+	umlal	$ACC4,$IN01_4,${R0}[0]+	 add	x12,x12,x13,lsl#32	// bfi	x12,x13,#32,#32+	umlal	$ACC1,$IN01_4,${S2}[0]+	 fmov	$IN01_2,x8+	umlal	$ACC2,$IN01_4,${S3}[0]+	 fmov	$IN01_3,x10+	 fmov	$IN01_4,x12++	/////////////////////////////////////////////////////////////////+	// lazy reduction as discussed in "NEON crypto" by D.J. Bernstein+	// and P. Schwabe+	//+	// [see discussion in poly1305-armv4 module]++	ushr	$T0.2d,$ACC3,#26+	xtn	$H3,$ACC3+	 ushr	$T1.2d,$ACC0,#26+	 and	$ACC0,$ACC0,$MASK.2d+	add	$ACC4,$ACC4,$T0.2d	// h3 -> h4+	bic	$H3,#0xfc,lsl#24	// &=0x03ffffff+	 add	$ACC1,$ACC1,$T1.2d	// h0 -> h1++	ushr	$T0.2d,$ACC4,#26+	xtn	$H4,$ACC4+	 ushr	$T1.2d,$ACC1,#26+	 xtn	$H1,$ACC1+	bic	$H4,#0xfc,lsl#24+	 add	$ACC2,$ACC2,$T1.2d	// h1 -> h2++	add	$ACC0,$ACC0,$T0.2d+	shl	$T0.2d,$T0.2d,#2+	 shrn	$T1.2s,$ACC2,#26+	 xtn	$H2,$ACC2+	add	$ACC0,$ACC0,$T0.2d	// h4 -> h0+	 bic	$H1,#0xfc,lsl#24+	 add	$H3,$H3,$T1.2s		// h2 -> h3+	 bic	$H2,#0xfc,lsl#24++	shrn	$T0.2s,$ACC0,#26+	xtn	$H0,$ACC0+	 ushr	$T1.2s,$H3,#26+	 bic	$H3,#0xfc,lsl#24+	 bic	$H0,#0xfc,lsl#24+	add	$H1,$H1,$T0.2s		// h0 -> h1+	 add	$H4,$H4,$T1.2s		// h3 -> h4++	b.hi	.Loop_neon++.Lskip_loop:+	dup	$IN23_2,${IN23_2}[0]+	add	$IN01_2,$IN01_2,$H2++	////////////////////////////////////////////////////////////////+	// multiply (inp[0:1]+hash) or inp[2:3] by r^2:r^1++	adds	$len,$len,#32+	b.ne	.Long_tail++	dup	$IN23_2,${IN01_2}[0]+	add	$IN23_0,$IN01_0,$H0+	add	$IN23_3,$IN01_3,$H3+	add	$IN23_1,$IN01_1,$H1+	add	$IN23_4,$IN01_4,$H4++.Long_tail:+	dup	$IN23_0,${IN23_0}[0]+	umull2	$ACC0,$IN23_2,${S3}+	umull2	$ACC3,$IN23_2,${R1}+	umull2	$ACC4,$IN23_2,${R2}+	umull2	$ACC2,$IN23_2,${R0}+	umull2	$ACC1,$IN23_2,${S4}++	dup	$IN23_1,${IN23_1}[0]+	umlal2	$ACC0,$IN23_0,${R0}+	umlal2	$ACC2,$IN23_0,${R2}+	umlal2	$ACC3,$IN23_0,${R3}+	umlal2	$ACC4,$IN23_0,${R4}+	umlal2	$ACC1,$IN23_0,${R1}++	dup	$IN23_3,${IN23_3}[0]+	umlal2	$ACC0,$IN23_1,${S4}+	umlal2	$ACC3,$IN23_1,${R2}+	umlal2	$ACC2,$IN23_1,${R1}+	umlal2	$ACC4,$IN23_1,${R3}+	umlal2	$ACC1,$IN23_1,${R0}++	dup	$IN23_4,${IN23_4}[0]+	umlal2	$ACC3,$IN23_3,${R0}+	umlal2	$ACC4,$IN23_3,${R1}+	umlal2	$ACC0,$IN23_3,${S2}+	umlal2	$ACC1,$IN23_3,${S3}+	umlal2	$ACC2,$IN23_3,${S4}++	umlal2	$ACC3,$IN23_4,${S4}+	umlal2	$ACC0,$IN23_4,${S1}+	umlal2	$ACC4,$IN23_4,${R0}+	umlal2	$ACC1,$IN23_4,${S2}+	umlal2	$ACC2,$IN23_4,${S3}++	b.eq	.Lshort_tail++	////////////////////////////////////////////////////////////////+	// (hash+inp[0:1])*r^4:r^3 and accumulate++	add	$IN01_0,$IN01_0,$H0+	umlal	$ACC3,$IN01_2,${R1}+	umlal	$ACC0,$IN01_2,${S3}+	umlal	$ACC4,$IN01_2,${R2}+	umlal	$ACC1,$IN01_2,${S4}+	umlal	$ACC2,$IN01_2,${R0}++	add	$IN01_1,$IN01_1,$H1+	umlal	$ACC3,$IN01_0,${R3}+	umlal	$ACC0,$IN01_0,${R0}+	umlal	$ACC4,$IN01_0,${R4}+	umlal	$ACC1,$IN01_0,${R1}+	umlal	$ACC2,$IN01_0,${R2}++	add	$IN01_3,$IN01_3,$H3+	umlal	$ACC3,$IN01_1,${R2}+	umlal	$ACC0,$IN01_1,${S4}+	umlal	$ACC4,$IN01_1,${R3}+	umlal	$ACC1,$IN01_1,${R0}+	umlal	$ACC2,$IN01_1,${R1}++	add	$IN01_4,$IN01_4,$H4+	umlal	$ACC3,$IN01_3,${R0}+	umlal	$ACC0,$IN01_3,${S2}+	umlal	$ACC4,$IN01_3,${R1}+	umlal	$ACC1,$IN01_3,${S3}+	umlal	$ACC2,$IN01_3,${S4}++	umlal	$ACC3,$IN01_4,${S4}+	umlal	$ACC0,$IN01_4,${S1}+	umlal	$ACC4,$IN01_4,${R0}+	umlal	$ACC1,$IN01_4,${S2}+	umlal	$ACC2,$IN01_4,${S3}++.Lshort_tail:+	////////////////////////////////////////////////////////////////+	// horizontal add++	addp	$ACC3,$ACC3,$ACC3+	 ldp	d8,d9,[sp,#2*__SIZEOF_POINTER__+0]	// meet ABI requirements+	addp	$ACC0,$ACC0,$ACC0+	 ldp	d10,d11,[sp,#2*__SIZEOF_POINTER__+16]+	addp	$ACC4,$ACC4,$ACC4+	 ldp	d12,d13,[sp,#2*__SIZEOF_POINTER__+32]+	addp	$ACC1,$ACC1,$ACC1+	 ldp	d14,d15,[sp,#2*__SIZEOF_POINTER__+48]+	addp	$ACC2,$ACC2,$ACC2+	 ldr	c30,[csp,#__SIZEOF_POINTER__]++	////////////////////////////////////////////////////////////////+	// lazy reduction, but without narrowing++	ushr	$T0.2d,$ACC3,#26+	and	$ACC3,$ACC3,$MASK.2d+	 ushr	$T1.2d,$ACC0,#26+	 and	$ACC0,$ACC0,$MASK.2d++	add	$ACC4,$ACC4,$T0.2d	// h3 -> h4+	 add	$ACC1,$ACC1,$T1.2d	// h0 -> h1++	ushr	$T0.2d,$ACC4,#26+	and	$ACC4,$ACC4,$MASK.2d+	 ushr	$T1.2d,$ACC1,#26+	 and	$ACC1,$ACC1,$MASK.2d+	 add	$ACC2,$ACC2,$T1.2d	// h1 -> h2++	add	$ACC0,$ACC0,$T0.2d+	shl	$T0.2d,$T0.2d,#2+	 ushr	$T1.2d,$ACC2,#26+	 and	$ACC2,$ACC2,$MASK.2d+	add	$ACC0,$ACC0,$T0.2d	// h4 -> h0+	 add	$ACC3,$ACC3,$T1.2d	// h2 -> h3++	ushr	$T0.2d,$ACC0,#26+	and	$ACC0,$ACC0,$MASK.2d+	 ushr	$T1.2d,$ACC3,#26+	 and	$ACC3,$ACC3,$MASK.2d+	add	$ACC1,$ACC1,$T0.2d	// h0 -> h1+	 add	$ACC4,$ACC4,$T1.2d	// h3 -> h4++	////////////////////////////////////////////////////////////////+	// write the result, can be partially reduced++	st4	{$ACC0,$ACC1,$ACC2,$ACC3}[0],[$ctx],#16+	mov	x4,#1+	st1	{$ACC4}[0],[$ctx]+	str	x4,[$ctx,#8]		// set is_base2_26++	ldr	c29,[csp],#2*__SIZEOF_POINTER__+64+	 .inst	0xd50323bf		// autiasp+	ret+.size	poly1305_blocks_neon,.-poly1305_blocks_neon++.align	5+.Lzeros:+.long	0,0,0,0,0,0,0,0+.asciz	"Poly1305 for ARMv8, CRYPTOGAMS by \@dot-asm"+.align	2+#if !defined(__KERNEL__) && !defined(_WIN64)+.comm	OPENSSL_armcap_P,4,4+.hidden	OPENSSL_armcap_P+#endif+___++foreach (split("\n",$code)) {+	s/\b(shrn\s+v[0-9]+)\.[24]d/$1.2s/			or+	s/\b(fmov\s+)v([0-9]+)[^,]*,\s*x([0-9]+)/$1d$2,x$3/	or+	(m/\bdup\b/ and (s/\.[24]s/.2d/g or 1))			or+	(m/\b(eor|and)/ and (s/\.[248][sdh]/.16b/g or 1))	or+	(m/\bum(ul|la)l\b/ and (s/\.4s/.2s/g or 1))		or+	(m/\bum(ul|la)l2\b/ and (s/\.2s/.4s/g or 1))		or+	(m/\bst[1-4]\s+{[^}]+}\[/ and (s/\.[24]d/.s/g or 1));++	s/\.[124]([sd])\[/.$1\[/;+	s/([cw])#x([0-9]+)/$1$2/g;++	print $_,"\n";+}+close STDOUT;
+ cbits/asm/poly1305-x86_64-elf.S view
@@ -0,0 +1,2033 @@+.text	++++.globl	crypton_poly1305_asm_init+.hidden	crypton_poly1305_asm_init+.globl	crypton_poly1305_asm_blocks+.hidden	crypton_poly1305_asm_blocks+.globl	crypton_poly1305_asm_emit+.hidden	crypton_poly1305_asm_emit++.type	crypton_poly1305_asm_init,@function+.align	32+crypton_poly1305_asm_init:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa++	xorq	%rax,%rax+	movq	%rax,0(%rdi)+	movq	%rax,8(%rdi)+	movq	%rax,16(%rdi)++	cmpq	$0,%rsi+	je	.Lno_key++	movq	$0x0ffffffc0fffffff,%rax+	leaq	-3(%rax),%rcx+	andq	0(%rsi),%rax+	andq	8(%rsi),%rcx+	movq	%rax,24(%rdi)+	movq	%rcx,32(%rdi)+	movl	$-1,48(%rdi)+	leaq	crypton_poly1305_asm_blocks(%rip),%r10+	leaq	crypton_poly1305_asm_emit(%rip),%r11+	movq	crypton_ia32cap_P+4(%rip),%r9+	leaq	crypton_poly1305_asm_blocks_avx(%rip),%rax+	btq	$28,%r9+	cmovcq	%rax,%r10+	leaq	crypton_poly1305_asm_blocks_avx2(%rip),%rax+	btq	$37,%r9+	cmovcq	%rax,%r10+	movq	%r10,0(%rdx)+	movq	%r11,8(%rdx)+	movl	$1,%eax+.Lno_key:+	.byte	0xf3,0xc3+.cfi_endproc+.size	crypton_poly1305_asm_init,.-crypton_poly1305_asm_init++.type	crypton_poly1305_asm_blocks,@function+.align	32+crypton_poly1305_asm_blocks:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++.Lblocks:+	shrq	$4,%rdx+	jz	.Lno_data++	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56+	leaq	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+.Lblocks_body:++	movq	%rdx,%r15++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13++	movq	0(%rdi),%r14+	movq	8(%rdi),%rbx+	movq	16(%rdi),%rbp++	movl	%r14d,%eax+	movl	4(%rdi),%edx+	movl	%ebx,%r8d+	movl	12(%rdi),%r10d+	movl	%ebp,%r12d++	shlq	$26,%rdx+	movq	%r8,%r9+	shlq	$52,%r8+	addq	%rdx,%rax+	shrq	$12,%r9+	addq	%rax,%r8+	adcq	$0,%r9++	shlq	$14,%r10+	movq	%r12,%rax+	shrq	$24,%r12+	addq	%r10,%r9+	shlq	$40,%rax+	addq	%rax,%r9+	adcq	$0,%r12++	cmpq	$4,%rbp++	cmovaq	%r8,%r14+	cmovaq	%r9,%rbx+	cmovaq	%r12,%rbp++	movq	%r13,%r12+	shrq	$2,%r13+	movq	%r12,%rax+	addq	%r12,%r13+	jmp	.Loop++.align	32+.Loop:+	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp+	mulq	%r14+	movq	%rax,%r9+	movq	%r11,%rax+	movq	%rdx,%r10++	mulq	%r14+	movq	%rax,%r14+	movq	%r11,%rax+	movq	%rdx,%r8++	mulq	%rbx+	addq	%rax,%r9+	movq	%r13,%rax+	adcq	%rdx,%r10++	mulq	%rbx+	movq	%rbp,%rbx+	addq	%rax,%r14+	adcq	%rdx,%r8++	imulq	%r13,%rbx+	addq	%rbx,%r9+	movq	%r8,%rbx+	adcq	$0,%r10++	imulq	%r11,%rbp+	addq	%r9,%rbx+	movq	$-4,%rax+	adcq	%rbp,%r10++	andq	%r10,%rax+	movq	%r10,%rbp+	shrq	$2,%r10+	andq	$3,%rbp+	addq	%r10,%rax+	addq	%rax,%r14+	adcq	$0,%rbx+	adcq	$0,%rbp+	movq	%r12,%rax+	decq	%r15+	jnz	.Loop++	movq	%r14,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rbp,16(%rdi)++	movq	8(%rsp),%r15+.cfi_restore	%r15+	movq	16(%rsp),%r14+.cfi_restore	%r14+	movq	24(%rsp),%r13+.cfi_restore	%r13+	movq	32(%rsp),%r12+.cfi_restore	%r12+	movq	40(%rsp),%rbp+.cfi_restore	%rbp+	movq	48(%rsp),%rbx+.cfi_restore	%rbx+	leaq	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+.Lno_data:+.Lblocks_epilogue:+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_poly1305_asm_blocks,.-crypton_poly1305_asm_blocks++.type	crypton_poly1305_asm_emit,@function+.align	32+crypton_poly1305_asm_emit:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa++	movl	0(%rdi),%eax+	movl	4(%rdi),%ecx+	movl	8(%rdi),%r8d+	movl	12(%rdi),%r11d+	movl	16(%rdi),%r10d++	shlq	$26,%rcx+	movq	%r8,%r9+	shlq	$52,%r8+	addq	%rcx,%rax+	shrq	$12,%r9+	addq	%rax,%r8+	adcq	$0,%r9++	shlq	$14,%r11+	movq	%r10,%rax+	shrq	$24,%r10+	addq	%r11,%r9+	movq	0(%rdi),%rcx+	shlq	$40,%rax+	movq	8(%rdi),%r11+	addq	%rax,%r9+	movq	16(%rdi),%rax+	adcq	$0,%r10++	cmpq	$4,%rax++	cmovbeq	%rcx,%r8+	cmovbeq	%r11,%r9+	cmovbeq	%rax,%r10++	movq	%r8,%rax+	addq	$5,%r8+	movq	%r9,%rcx+	adcq	$0,%r9+	adcq	$0,%r10+	shrq	$2,%r10+	cmovnzq	%r8,%rax+	cmovnzq	%r9,%rcx++	addq	0(%rdx),%rax+	adcq	8(%rdx),%rcx+	movq	%rax,0(%rsi)+	movq	%rcx,8(%rsi)++	.byte	0xf3,0xc3+.cfi_endproc+.size	crypton_poly1305_asm_emit,.-crypton_poly1305_asm_emit+.type	__crypton_poly1305_asm_block,@function+.align	32+__crypton_poly1305_asm_block:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa++	mulq	%r14+	movq	%rax,%r9+	movq	%r11,%rax+	movq	%rdx,%r10++	mulq	%r14+	movq	%rax,%r14+	movq	%r11,%rax+	movq	%rdx,%r8++	mulq	%rbx+	addq	%rax,%r9+	movq	%r13,%rax+	adcq	%rdx,%r10++	mulq	%rbx+	movq	%rbp,%rbx+	addq	%rax,%r14+	adcq	%rdx,%r8++	imulq	%r13,%rbx+	addq	%rbx,%r9+	movq	%r8,%rbx+	adcq	$0,%r10++	imulq	%r11,%rbp+	addq	%r9,%rbx+	movq	$-4,%rax+	adcq	%rbp,%r10++	andq	%r10,%rax+	movq	%r10,%rbp+	shrq	$2,%r10+	andq	$3,%rbp+	addq	%r10,%rax+	addq	%rax,%r14+	adcq	$0,%rbx+	adcq	$0,%rbp+	.byte	0xf3,0xc3+.cfi_endproc+.size	__crypton_poly1305_asm_block,.-__crypton_poly1305_asm_block++.type	__crypton_poly1305_asm_init_avx,@function+.align	32+__crypton_poly1305_asm_init_avx:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa++	cmpl	$-1,48(%rdi)+	jne	.Ldone_init_avx++	movq	%r11,%r14+	movq	%r12,%rbx+	xorq	%rbp,%rbp++	leaq	48+64(%rdi),%rdi++	movq	%r12,%rax+	call	__crypton_poly1305_asm_block++	movl	$0x3ffffff,%eax+	movl	$0x3ffffff,%edx+	movq	%r14,%r8+	andl	%r14d,%eax+	movq	%r11,%r9+	andl	%r11d,%edx+	movl	%eax,-64(%rdi)+	shrq	$26,%r8+	movl	%edx,-60(%rdi)+	shrq	$26,%r9++	movl	$0x3ffffff,%eax+	movl	$0x3ffffff,%edx+	andl	%r8d,%eax+	andl	%r9d,%edx+	movl	%eax,-48(%rdi)+	leal	(%rax,%rax,4),%eax+	movl	%edx,-44(%rdi)+	leal	(%rdx,%rdx,4),%edx+	movl	%eax,-32(%rdi)+	shrq	$26,%r8+	movl	%edx,-28(%rdi)+	shrq	$26,%r9++	movq	%rbx,%rax+	movq	%r12,%rdx+	shlq	$12,%rax+	shlq	$12,%rdx+	orq	%r8,%rax+	orq	%r9,%rdx+	andl	$0x3ffffff,%eax+	andl	$0x3ffffff,%edx+	movl	%eax,-16(%rdi)+	leal	(%rax,%rax,4),%eax+	movl	%edx,-12(%rdi)+	leal	(%rdx,%rdx,4),%edx+	movl	%eax,0(%rdi)+	movq	%rbx,%r8+	movl	%edx,4(%rdi)+	movq	%r12,%r9++	movl	$0x3ffffff,%eax+	movl	$0x3ffffff,%edx+	shrq	$14,%r8+	shrq	$14,%r9+	andl	%r8d,%eax+	andl	%r9d,%edx+	movl	%eax,16(%rdi)+	leal	(%rax,%rax,4),%eax+	movl	%edx,20(%rdi)+	leal	(%rdx,%rdx,4),%edx+	movl	%eax,32(%rdi)+	shrq	$26,%r8+	movl	%edx,36(%rdi)+	shrq	$26,%r9++	movq	%rbp,%rax+	shlq	$24,%rax+	orq	%rax,%r8+	movl	%r8d,48(%rdi)+	leaq	(%r8,%r8,4),%r8+	movl	%r9d,52(%rdi)+	leaq	(%r9,%r9,4),%r9+	movl	%r8d,64(%rdi)+	movl	%r9d,68(%rdi)++	movq	%r12,%rax+	call	__crypton_poly1305_asm_block++	movl	$0x3ffffff,%eax+	movq	%r14,%r8+	andl	%r14d,%eax+	shrq	$26,%r8+	movl	%eax,-52(%rdi)++	movl	$0x3ffffff,%edx+	andl	%r8d,%edx+	movl	%edx,-36(%rdi)+	leal	(%rdx,%rdx,4),%edx+	shrq	$26,%r8+	movl	%edx,-20(%rdi)++	movq	%rbx,%rax+	shlq	$12,%rax+	orq	%r8,%rax+	andl	$0x3ffffff,%eax+	movl	%eax,-4(%rdi)+	leal	(%rax,%rax,4),%eax+	movq	%rbx,%r8+	movl	%eax,12(%rdi)++	movl	$0x3ffffff,%edx+	shrq	$14,%r8+	andl	%r8d,%edx+	movl	%edx,28(%rdi)+	leal	(%rdx,%rdx,4),%edx+	shrq	$26,%r8+	movl	%edx,44(%rdi)++	movq	%rbp,%rax+	shlq	$24,%rax+	orq	%rax,%r8+	movl	%r8d,60(%rdi)+	leaq	(%r8,%r8,4),%r8+	movl	%r8d,76(%rdi)++	movq	%r12,%rax+	call	__crypton_poly1305_asm_block++	movl	$0x3ffffff,%eax+	movq	%r14,%r8+	andl	%r14d,%eax+	shrq	$26,%r8+	movl	%eax,-56(%rdi)++	movl	$0x3ffffff,%edx+	andl	%r8d,%edx+	movl	%edx,-40(%rdi)+	leal	(%rdx,%rdx,4),%edx+	shrq	$26,%r8+	movl	%edx,-24(%rdi)++	movq	%rbx,%rax+	shlq	$12,%rax+	orq	%r8,%rax+	andl	$0x3ffffff,%eax+	movl	%eax,-8(%rdi)+	leal	(%rax,%rax,4),%eax+	movq	%rbx,%r8+	movl	%eax,8(%rdi)++	movl	$0x3ffffff,%edx+	shrq	$14,%r8+	andl	%r8d,%edx+	movl	%edx,24(%rdi)+	leal	(%rdx,%rdx,4),%edx+	shrq	$26,%r8+	movl	%edx,40(%rdi)++	movq	%rbp,%rax+	shlq	$24,%rax+	orq	%rax,%r8+	movl	%r8d,56(%rdi)+	leaq	(%r8,%r8,4),%r8+	movl	%r8d,72(%rdi)++	leaq	-48-64(%rdi),%rdi+.Ldone_init_avx:+	.byte	0xf3,0xc3+.cfi_endproc+.size	__crypton_poly1305_asm_init_avx,.-__crypton_poly1305_asm_init_avx++.type	crypton_poly1305_asm_blocks_avx,@function+.align	32+crypton_poly1305_asm_blocks_avx:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	movl	20(%rdi),%r8d+	cmpq	$128,%rdx+	jb	.Lblocks++	andq	$-16,%rdx++	vzeroupper++	testl	%r8d,%r8d+	jz	.Lbase2_64_avx++	testq	$31,%rdx+	jz	.Leven_avx++	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56+	leaq	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+.Lblocks_avx_body:++	movq	%rdx,%r15++	movq	0(%rdi),%r8+	movq	8(%rdi),%r9+	movl	16(%rdi),%ebp++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13+++	movl	%r8d,%r14d+	andq	$-2147483648,%r8+	movq	%r9,%r12+	movl	%r9d,%ebx+	andq	$-2147483648,%r9++	shrq	$6,%r8+	shlq	$52,%r12+	addq	%r8,%r14+	shrq	$12,%rbx+	shrq	$18,%r9+	addq	%r12,%r14+	adcq	%r9,%rbx++	movq	%rbp,%r8+	shlq	$40,%r8+	shrq	$24,%rbp+	addq	%r8,%rbx+	adcq	$0,%rbp++	movq	%r13,%r12+	movq	%r13,%rax+	shrq	$2,%r13+	addq	%r12,%r13++	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp++	call	__crypton_poly1305_asm_block+++	movq	%r14,%rax+	movq	%r14,%rdx+	shrq	$52,%r14+	movq	%rbx,%r11+	movq	%rbx,%r12+	shrq	$26,%rdx+	andq	$0x3ffffff,%rax+	shlq	$12,%r11+	andq	$0x3ffffff,%rdx+	shrq	$14,%rbx+	orq	%r11,%r14+	shlq	$24,%rbp+	andq	$0x3ffffff,%r14+	shrq	$40,%r12+	andq	$0x3ffffff,%rbx+	orq	%r12,%rbp++	vmovd	%eax,%xmm0+	vmovd	%edx,%xmm1+	vmovd	%r14d,%xmm2+	vmovd	%ebx,%xmm3+	vmovd	%ebp,%xmm4++	leaq	-16(%r15),%rdx++	movq	8(%rsp),%r15+.cfi_restore	%r15+	movq	16(%rsp),%r14+.cfi_restore	%r14+	movq	24(%rsp),%r13+.cfi_restore	%r13+	movq	32(%rsp),%r12+.cfi_restore	%r12+	movq	40(%rsp),%rbp+.cfi_restore	%rbp+	movq	48(%rsp),%rbx+.cfi_restore	%rbx+	leaq	56(%rsp),%rax+	leaq	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+.Lblocks_avx_epilogue:+	jmp	.Ldo_avx+.cfi_endproc	++.align	32+.Lbase2_64_avx:+.cfi_startproc	+	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56+	leaq	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+.Lbase2_64_avx_body:++	movq	%rdx,%r15++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13++	movq	0(%rdi),%r14+	movq	8(%rdi),%rbx+	movl	16(%rdi),%ebp++	movq	%r13,%r12+	movq	%r13,%rax+	shrq	$2,%r13+	addq	%r12,%r13++	testq	$31,%rdx+	jz	.Linit_avx++	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp+	subq	$16,%r15++	call	__crypton_poly1305_asm_block++.Linit_avx:++	movq	%r14,%rax+	movq	%r14,%rdx+	shrq	$52,%r14+	movq	%rbx,%r8+	movq	%rbx,%r9+	shrq	$26,%rdx+	andq	$0x3ffffff,%rax+	shlq	$12,%r8+	andq	$0x3ffffff,%rdx+	shrq	$14,%rbx+	orq	%r8,%r14+	shlq	$24,%rbp+	andq	$0x3ffffff,%r14+	shrq	$40,%r9+	andq	$0x3ffffff,%rbx+	orq	%r9,%rbp++	vmovd	%eax,%xmm0+	vmovd	%edx,%xmm1+	vmovd	%r14d,%xmm2+	vmovd	%ebx,%xmm3+	vmovd	%ebp,%xmm4+	movl	$1,20(%rdi)++	call	__crypton_poly1305_asm_init_avx++	movq	%r15,%rdx++	movq	8(%rsp),%r15+.cfi_restore	%r15+	movq	16(%rsp),%r14+.cfi_restore	%r14+	movq	24(%rsp),%r13+.cfi_restore	%r13+	movq	32(%rsp),%r12+.cfi_restore	%r12+	movq	40(%rsp),%rbp+.cfi_restore	%rbp+	movq	48(%rsp),%rbx+.cfi_restore	%rbx+	leaq	56(%rsp),%rax+	leaq	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+.Lbase2_64_avx_epilogue:+	jmp	.Ldo_avx+.cfi_endproc	++.align	32+.Leven_avx:+.cfi_startproc	+	vmovd	0(%rdi),%xmm0+	vmovd	4(%rdi),%xmm1+	vmovd	8(%rdi),%xmm2+	vmovd	12(%rdi),%xmm3+	vmovd	16(%rdi),%xmm4++.Ldo_avx:+	leaq	-88(%rsp),%r11+.cfi_def_cfa	%r11,0x60+	subq	$0x178,%rsp+	subq	$64,%rdx+	leaq	-32(%rsi),%rax+	cmovcq	%rax,%rsi++	vmovdqu	48(%rdi),%xmm14+	leaq	112(%rdi),%rdi+	leaq	.Lconst(%rip),%rcx++++	vmovdqu	32(%rsi),%xmm5+	vmovdqu	48(%rsi),%xmm6+	vmovdqa	64(%rcx),%xmm15++	vpsrldq	$6,%xmm5,%xmm7+	vpsrldq	$6,%xmm6,%xmm8+	vpunpckhqdq	%xmm6,%xmm5,%xmm9+	vpunpcklqdq	%xmm6,%xmm5,%xmm5+	vpunpcklqdq	%xmm8,%xmm7,%xmm8++	vpsrlq	$40,%xmm9,%xmm9+	vpsrlq	$26,%xmm5,%xmm6+	vpand	%xmm15,%xmm5,%xmm5+	vpsrlq	$4,%xmm8,%xmm7+	vpand	%xmm15,%xmm6,%xmm6+	vpsrlq	$30,%xmm8,%xmm8+	vpand	%xmm15,%xmm7,%xmm7+	vpand	%xmm15,%xmm8,%xmm8+	vpor	32(%rcx),%xmm9,%xmm9++	jbe	.Lskip_loop_avx+++	vmovdqu	-48(%rdi),%xmm11+	vmovdqu	-32(%rdi),%xmm12+	vpshufd	$0xEE,%xmm14,%xmm13+	vpshufd	$0x44,%xmm14,%xmm10+	vmovdqa	%xmm13,-144(%r11)+	vmovdqa	%xmm10,0(%rsp)+	vpshufd	$0xEE,%xmm11,%xmm14+	vmovdqu	-16(%rdi),%xmm10+	vpshufd	$0x44,%xmm11,%xmm11+	vmovdqa	%xmm14,-128(%r11)+	vmovdqa	%xmm11,16(%rsp)+	vpshufd	$0xEE,%xmm12,%xmm13+	vmovdqu	0(%rdi),%xmm11+	vpshufd	$0x44,%xmm12,%xmm12+	vmovdqa	%xmm13,-112(%r11)+	vmovdqa	%xmm12,32(%rsp)+	vpshufd	$0xEE,%xmm10,%xmm14+	vmovdqu	16(%rdi),%xmm12+	vpshufd	$0x44,%xmm10,%xmm10+	vmovdqa	%xmm14,-96(%r11)+	vmovdqa	%xmm10,48(%rsp)+	vpshufd	$0xEE,%xmm11,%xmm13+	vmovdqu	32(%rdi),%xmm10+	vpshufd	$0x44,%xmm11,%xmm11+	vmovdqa	%xmm13,-80(%r11)+	vmovdqa	%xmm11,64(%rsp)+	vpshufd	$0xEE,%xmm12,%xmm14+	vmovdqu	48(%rdi),%xmm11+	vpshufd	$0x44,%xmm12,%xmm12+	vmovdqa	%xmm14,-64(%r11)+	vmovdqa	%xmm12,80(%rsp)+	vpshufd	$0xEE,%xmm10,%xmm13+	vmovdqu	64(%rdi),%xmm12+	vpshufd	$0x44,%xmm10,%xmm10+	vmovdqa	%xmm13,-48(%r11)+	vmovdqa	%xmm10,96(%rsp)+	vpshufd	$0xEE,%xmm11,%xmm14+	vpshufd	$0x44,%xmm11,%xmm11+	vmovdqa	%xmm14,-32(%r11)+	vmovdqa	%xmm11,112(%rsp)+	vpshufd	$0xEE,%xmm12,%xmm13+	vmovdqa	0(%rsp),%xmm14+	vpshufd	$0x44,%xmm12,%xmm12+	vmovdqa	%xmm13,-16(%r11)+	vmovdqa	%xmm12,128(%rsp)++	jmp	.Loop_avx++.align	32+.Loop_avx:+++++++++++++++++++++	vpmuludq	%xmm5,%xmm14,%xmm10+	vpmuludq	%xmm6,%xmm14,%xmm11+	vmovdqa	%xmm2,32(%r11)+	vpmuludq	%xmm7,%xmm14,%xmm12+	vmovdqa	16(%rsp),%xmm2+	vpmuludq	%xmm8,%xmm14,%xmm13+	vpmuludq	%xmm9,%xmm14,%xmm14++	vmovdqa	%xmm0,0(%r11)+	vpmuludq	32(%rsp),%xmm9,%xmm0+	vmovdqa	%xmm1,16(%r11)+	vpmuludq	%xmm8,%xmm2,%xmm1+	vpaddq	%xmm0,%xmm10,%xmm10+	vpaddq	%xmm1,%xmm14,%xmm14+	vmovdqa	%xmm3,48(%r11)+	vpmuludq	%xmm7,%xmm2,%xmm0+	vpmuludq	%xmm6,%xmm2,%xmm1+	vpaddq	%xmm0,%xmm13,%xmm13+	vmovdqa	48(%rsp),%xmm3+	vpaddq	%xmm1,%xmm12,%xmm12+	vmovdqa	%xmm4,64(%r11)+	vpmuludq	%xmm5,%xmm2,%xmm2+	vpmuludq	%xmm7,%xmm3,%xmm0+	vpaddq	%xmm2,%xmm11,%xmm11++	vmovdqa	64(%rsp),%xmm4+	vpaddq	%xmm0,%xmm14,%xmm14+	vpmuludq	%xmm6,%xmm3,%xmm1+	vpmuludq	%xmm5,%xmm3,%xmm3+	vpaddq	%xmm1,%xmm13,%xmm13+	vmovdqa	80(%rsp),%xmm2+	vpaddq	%xmm3,%xmm12,%xmm12+	vpmuludq	%xmm9,%xmm4,%xmm0+	vpmuludq	%xmm8,%xmm4,%xmm4+	vpaddq	%xmm0,%xmm11,%xmm11+	vmovdqa	96(%rsp),%xmm3+	vpaddq	%xmm4,%xmm10,%xmm10++	vmovdqa	128(%rsp),%xmm4+	vpmuludq	%xmm6,%xmm2,%xmm1+	vpmuludq	%xmm5,%xmm2,%xmm2+	vpaddq	%xmm1,%xmm14,%xmm14+	vpaddq	%xmm2,%xmm13,%xmm13+	vpmuludq	%xmm9,%xmm3,%xmm0+	vpmuludq	%xmm8,%xmm3,%xmm1+	vpaddq	%xmm0,%xmm12,%xmm12+	vmovdqu	0(%rsi),%xmm0+	vpaddq	%xmm1,%xmm11,%xmm11+	vpmuludq	%xmm7,%xmm3,%xmm3+	vpmuludq	%xmm7,%xmm4,%xmm7+	vpaddq	%xmm3,%xmm10,%xmm10++	vmovdqu	16(%rsi),%xmm1+	vpaddq	%xmm7,%xmm11,%xmm11+	vpmuludq	%xmm8,%xmm4,%xmm8+	vpmuludq	%xmm9,%xmm4,%xmm9+	vpsrldq	$6,%xmm0,%xmm2+	vpaddq	%xmm8,%xmm12,%xmm12+	vpaddq	%xmm9,%xmm13,%xmm13+	vpsrldq	$6,%xmm1,%xmm3+	vpmuludq	112(%rsp),%xmm5,%xmm9+	vpmuludq	%xmm6,%xmm4,%xmm5+	vpunpckhqdq	%xmm1,%xmm0,%xmm4+	vpaddq	%xmm9,%xmm14,%xmm14+	vmovdqa	-144(%r11),%xmm9+	vpaddq	%xmm5,%xmm10,%xmm10++	vpunpcklqdq	%xmm1,%xmm0,%xmm0+	vpunpcklqdq	%xmm3,%xmm2,%xmm3+++	vpsrldq	$5,%xmm4,%xmm4+	vpsrlq	$26,%xmm0,%xmm1+	vpand	%xmm15,%xmm0,%xmm0+	vpsrlq	$4,%xmm3,%xmm2+	vpand	%xmm15,%xmm1,%xmm1+	vpand	0(%rcx),%xmm4,%xmm4+	vpsrlq	$30,%xmm3,%xmm3+	vpand	%xmm15,%xmm2,%xmm2+	vpand	%xmm15,%xmm3,%xmm3+	vpor	32(%rcx),%xmm4,%xmm4++	vpaddq	0(%r11),%xmm0,%xmm0+	vpaddq	16(%r11),%xmm1,%xmm1+	vpaddq	32(%r11),%xmm2,%xmm2+	vpaddq	48(%r11),%xmm3,%xmm3+	vpaddq	64(%r11),%xmm4,%xmm4++	leaq	32(%rsi),%rax+	leaq	64(%rsi),%rsi+	subq	$64,%rdx+	cmovcq	%rax,%rsi+++++++++++	vpmuludq	%xmm0,%xmm9,%xmm5+	vpmuludq	%xmm1,%xmm9,%xmm6+	vpaddq	%xmm5,%xmm10,%xmm10+	vpaddq	%xmm6,%xmm11,%xmm11+	vmovdqa	-128(%r11),%xmm7+	vpmuludq	%xmm2,%xmm9,%xmm5+	vpmuludq	%xmm3,%xmm9,%xmm6+	vpaddq	%xmm5,%xmm12,%xmm12+	vpaddq	%xmm6,%xmm13,%xmm13+	vpmuludq	%xmm4,%xmm9,%xmm9+	vpmuludq	-112(%r11),%xmm4,%xmm5+	vpaddq	%xmm9,%xmm14,%xmm14++	vpaddq	%xmm5,%xmm10,%xmm10+	vpmuludq	%xmm2,%xmm7,%xmm6+	vpmuludq	%xmm3,%xmm7,%xmm5+	vpaddq	%xmm6,%xmm13,%xmm13+	vmovdqa	-96(%r11),%xmm8+	vpaddq	%xmm5,%xmm14,%xmm14+	vpmuludq	%xmm1,%xmm7,%xmm6+	vpmuludq	%xmm0,%xmm7,%xmm7+	vpaddq	%xmm6,%xmm12,%xmm12+	vpaddq	%xmm7,%xmm11,%xmm11++	vmovdqa	-80(%r11),%xmm9+	vpmuludq	%xmm2,%xmm8,%xmm5+	vpmuludq	%xmm1,%xmm8,%xmm6+	vpaddq	%xmm5,%xmm14,%xmm14+	vpaddq	%xmm6,%xmm13,%xmm13+	vmovdqa	-64(%r11),%xmm7+	vpmuludq	%xmm0,%xmm8,%xmm8+	vpmuludq	%xmm4,%xmm9,%xmm5+	vpaddq	%xmm8,%xmm12,%xmm12+	vpaddq	%xmm5,%xmm11,%xmm11+	vmovdqa	-48(%r11),%xmm8+	vpmuludq	%xmm3,%xmm9,%xmm9+	vpmuludq	%xmm1,%xmm7,%xmm6+	vpaddq	%xmm9,%xmm10,%xmm10++	vmovdqa	-16(%r11),%xmm9+	vpaddq	%xmm6,%xmm14,%xmm14+	vpmuludq	%xmm0,%xmm7,%xmm7+	vpmuludq	%xmm4,%xmm8,%xmm5+	vpaddq	%xmm7,%xmm13,%xmm13+	vpaddq	%xmm5,%xmm12,%xmm12+	vmovdqu	32(%rsi),%xmm5+	vpmuludq	%xmm3,%xmm8,%xmm7+	vpmuludq	%xmm2,%xmm8,%xmm8+	vpaddq	%xmm7,%xmm11,%xmm11+	vmovdqu	48(%rsi),%xmm6+	vpaddq	%xmm8,%xmm10,%xmm10++	vpmuludq	%xmm2,%xmm9,%xmm2+	vpmuludq	%xmm3,%xmm9,%xmm3+	vpsrldq	$6,%xmm5,%xmm7+	vpaddq	%xmm2,%xmm11,%xmm11+	vpmuludq	%xmm4,%xmm9,%xmm4+	vpsrldq	$6,%xmm6,%xmm8+	vpaddq	%xmm3,%xmm12,%xmm2+	vpaddq	%xmm4,%xmm13,%xmm3+	vpmuludq	-32(%r11),%xmm0,%xmm4+	vpmuludq	%xmm1,%xmm9,%xmm0+	vpunpckhqdq	%xmm6,%xmm5,%xmm9+	vpaddq	%xmm4,%xmm14,%xmm4+	vpaddq	%xmm0,%xmm10,%xmm0++	vpunpcklqdq	%xmm6,%xmm5,%xmm5+	vpunpcklqdq	%xmm8,%xmm7,%xmm8+++	vpsrldq	$5,%xmm9,%xmm9+	vpsrlq	$26,%xmm5,%xmm6+	vmovdqa	0(%rsp),%xmm14+	vpand	%xmm15,%xmm5,%xmm5+	vpsrlq	$4,%xmm8,%xmm7+	vpand	%xmm15,%xmm6,%xmm6+	vpand	0(%rcx),%xmm9,%xmm9+	vpsrlq	$30,%xmm8,%xmm8+	vpand	%xmm15,%xmm7,%xmm7+	vpand	%xmm15,%xmm8,%xmm8+	vpor	32(%rcx),%xmm9,%xmm9++++++	vpsrlq	$26,%xmm3,%xmm13+	vpand	%xmm15,%xmm3,%xmm3+	vpaddq	%xmm13,%xmm4,%xmm4++	vpsrlq	$26,%xmm0,%xmm10+	vpand	%xmm15,%xmm0,%xmm0+	vpaddq	%xmm10,%xmm11,%xmm1++	vpsrlq	$26,%xmm4,%xmm10+	vpand	%xmm15,%xmm4,%xmm4++	vpsrlq	$26,%xmm1,%xmm11+	vpand	%xmm15,%xmm1,%xmm1+	vpaddq	%xmm11,%xmm2,%xmm2++	vpaddq	%xmm10,%xmm0,%xmm0+	vpsllq	$2,%xmm10,%xmm10+	vpaddq	%xmm10,%xmm0,%xmm0++	vpsrlq	$26,%xmm2,%xmm12+	vpand	%xmm15,%xmm2,%xmm2+	vpaddq	%xmm12,%xmm3,%xmm3++	vpsrlq	$26,%xmm0,%xmm10+	vpand	%xmm15,%xmm0,%xmm0+	vpaddq	%xmm10,%xmm1,%xmm1++	vpsrlq	$26,%xmm3,%xmm13+	vpand	%xmm15,%xmm3,%xmm3+	vpaddq	%xmm13,%xmm4,%xmm4++	ja	.Loop_avx++.Lskip_loop_avx:++++	vpshufd	$0x10,%xmm14,%xmm14+	addq	$32,%rdx+	jnz	.Long_tail_avx++	vpaddq	%xmm2,%xmm7,%xmm7+	vpaddq	%xmm0,%xmm5,%xmm5+	vpaddq	%xmm1,%xmm6,%xmm6+	vpaddq	%xmm3,%xmm8,%xmm8+	vpaddq	%xmm4,%xmm9,%xmm9++.Long_tail_avx:+	vmovdqa	%xmm2,32(%r11)+	vmovdqa	%xmm0,0(%r11)+	vmovdqa	%xmm1,16(%r11)+	vmovdqa	%xmm3,48(%r11)+	vmovdqa	%xmm4,64(%r11)++++++++	vpmuludq	%xmm7,%xmm14,%xmm12+	vpmuludq	%xmm5,%xmm14,%xmm10+	vpshufd	$0x10,-48(%rdi),%xmm2+	vpmuludq	%xmm6,%xmm14,%xmm11+	vpmuludq	%xmm8,%xmm14,%xmm13+	vpmuludq	%xmm9,%xmm14,%xmm14++	vpmuludq	%xmm8,%xmm2,%xmm0+	vpaddq	%xmm0,%xmm14,%xmm14+	vpshufd	$0x10,-32(%rdi),%xmm3+	vpmuludq	%xmm7,%xmm2,%xmm1+	vpaddq	%xmm1,%xmm13,%xmm13+	vpshufd	$0x10,-16(%rdi),%xmm4+	vpmuludq	%xmm6,%xmm2,%xmm0+	vpaddq	%xmm0,%xmm12,%xmm12+	vpmuludq	%xmm5,%xmm2,%xmm2+	vpaddq	%xmm2,%xmm11,%xmm11+	vpmuludq	%xmm9,%xmm3,%xmm3+	vpaddq	%xmm3,%xmm10,%xmm10++	vpshufd	$0x10,0(%rdi),%xmm2+	vpmuludq	%xmm7,%xmm4,%xmm1+	vpaddq	%xmm1,%xmm14,%xmm14+	vpmuludq	%xmm6,%xmm4,%xmm0+	vpaddq	%xmm0,%xmm13,%xmm13+	vpshufd	$0x10,16(%rdi),%xmm3+	vpmuludq	%xmm5,%xmm4,%xmm4+	vpaddq	%xmm4,%xmm12,%xmm12+	vpmuludq	%xmm9,%xmm2,%xmm1+	vpaddq	%xmm1,%xmm11,%xmm11+	vpshufd	$0x10,32(%rdi),%xmm4+	vpmuludq	%xmm8,%xmm2,%xmm2+	vpaddq	%xmm2,%xmm10,%xmm10++	vpmuludq	%xmm6,%xmm3,%xmm0+	vpaddq	%xmm0,%xmm14,%xmm14+	vpmuludq	%xmm5,%xmm3,%xmm3+	vpaddq	%xmm3,%xmm13,%xmm13+	vpshufd	$0x10,48(%rdi),%xmm2+	vpmuludq	%xmm9,%xmm4,%xmm1+	vpaddq	%xmm1,%xmm12,%xmm12+	vpshufd	$0x10,64(%rdi),%xmm3+	vpmuludq	%xmm8,%xmm4,%xmm0+	vpaddq	%xmm0,%xmm11,%xmm11+	vpmuludq	%xmm7,%xmm4,%xmm4+	vpaddq	%xmm4,%xmm10,%xmm10++	vpmuludq	%xmm5,%xmm2,%xmm2+	vpaddq	%xmm2,%xmm14,%xmm14+	vpmuludq	%xmm9,%xmm3,%xmm1+	vpaddq	%xmm1,%xmm13,%xmm13+	vpmuludq	%xmm8,%xmm3,%xmm0+	vpaddq	%xmm0,%xmm12,%xmm12+	vpmuludq	%xmm7,%xmm3,%xmm1+	vpaddq	%xmm1,%xmm11,%xmm11+	vpmuludq	%xmm6,%xmm3,%xmm3+	vpaddq	%xmm3,%xmm10,%xmm10++	jz	.Lshort_tail_avx++	vmovdqu	0(%rsi),%xmm0+	vmovdqu	16(%rsi),%xmm1++	vpsrldq	$6,%xmm0,%xmm2+	vpsrldq	$6,%xmm1,%xmm3+	vpunpckhqdq	%xmm1,%xmm0,%xmm4+	vpunpcklqdq	%xmm1,%xmm0,%xmm0+	vpunpcklqdq	%xmm3,%xmm2,%xmm3++	vpsrlq	$40,%xmm4,%xmm4+	vpsrlq	$26,%xmm0,%xmm1+	vpand	%xmm15,%xmm0,%xmm0+	vpsrlq	$4,%xmm3,%xmm2+	vpand	%xmm15,%xmm1,%xmm1+	vpsrlq	$30,%xmm3,%xmm3+	vpand	%xmm15,%xmm2,%xmm2+	vpand	%xmm15,%xmm3,%xmm3+	vpor	32(%rcx),%xmm4,%xmm4++	vpshufd	$0x32,-64(%rdi),%xmm9+	vpaddq	0(%r11),%xmm0,%xmm0+	vpaddq	16(%r11),%xmm1,%xmm1+	vpaddq	32(%r11),%xmm2,%xmm2+	vpaddq	48(%r11),%xmm3,%xmm3+	vpaddq	64(%r11),%xmm4,%xmm4+++++	vpmuludq	%xmm0,%xmm9,%xmm5+	vpaddq	%xmm5,%xmm10,%xmm10+	vpmuludq	%xmm1,%xmm9,%xmm6+	vpaddq	%xmm6,%xmm11,%xmm11+	vpmuludq	%xmm2,%xmm9,%xmm5+	vpaddq	%xmm5,%xmm12,%xmm12+	vpshufd	$0x32,-48(%rdi),%xmm7+	vpmuludq	%xmm3,%xmm9,%xmm6+	vpaddq	%xmm6,%xmm13,%xmm13+	vpmuludq	%xmm4,%xmm9,%xmm9+	vpaddq	%xmm9,%xmm14,%xmm14++	vpmuludq	%xmm3,%xmm7,%xmm5+	vpaddq	%xmm5,%xmm14,%xmm14+	vpshufd	$0x32,-32(%rdi),%xmm8+	vpmuludq	%xmm2,%xmm7,%xmm6+	vpaddq	%xmm6,%xmm13,%xmm13+	vpshufd	$0x32,-16(%rdi),%xmm9+	vpmuludq	%xmm1,%xmm7,%xmm5+	vpaddq	%xmm5,%xmm12,%xmm12+	vpmuludq	%xmm0,%xmm7,%xmm7+	vpaddq	%xmm7,%xmm11,%xmm11+	vpmuludq	%xmm4,%xmm8,%xmm8+	vpaddq	%xmm8,%xmm10,%xmm10++	vpshufd	$0x32,0(%rdi),%xmm7+	vpmuludq	%xmm2,%xmm9,%xmm6+	vpaddq	%xmm6,%xmm14,%xmm14+	vpmuludq	%xmm1,%xmm9,%xmm5+	vpaddq	%xmm5,%xmm13,%xmm13+	vpshufd	$0x32,16(%rdi),%xmm8+	vpmuludq	%xmm0,%xmm9,%xmm9+	vpaddq	%xmm9,%xmm12,%xmm12+	vpmuludq	%xmm4,%xmm7,%xmm6+	vpaddq	%xmm6,%xmm11,%xmm11+	vpshufd	$0x32,32(%rdi),%xmm9+	vpmuludq	%xmm3,%xmm7,%xmm7+	vpaddq	%xmm7,%xmm10,%xmm10++	vpmuludq	%xmm1,%xmm8,%xmm5+	vpaddq	%xmm5,%xmm14,%xmm14+	vpmuludq	%xmm0,%xmm8,%xmm8+	vpaddq	%xmm8,%xmm13,%xmm13+	vpshufd	$0x32,48(%rdi),%xmm7+	vpmuludq	%xmm4,%xmm9,%xmm6+	vpaddq	%xmm6,%xmm12,%xmm12+	vpshufd	$0x32,64(%rdi),%xmm8+	vpmuludq	%xmm3,%xmm9,%xmm5+	vpaddq	%xmm5,%xmm11,%xmm11+	vpmuludq	%xmm2,%xmm9,%xmm9+	vpaddq	%xmm9,%xmm10,%xmm10++	vpmuludq	%xmm0,%xmm7,%xmm7+	vpaddq	%xmm7,%xmm14,%xmm14+	vpmuludq	%xmm4,%xmm8,%xmm6+	vpaddq	%xmm6,%xmm13,%xmm13+	vpmuludq	%xmm3,%xmm8,%xmm5+	vpaddq	%xmm5,%xmm12,%xmm12+	vpmuludq	%xmm2,%xmm8,%xmm6+	vpaddq	%xmm6,%xmm11,%xmm11+	vpmuludq	%xmm1,%xmm8,%xmm8+	vpaddq	%xmm8,%xmm10,%xmm10++.Lshort_tail_avx:++++	vpsrldq	$8,%xmm14,%xmm9+	vpsrldq	$8,%xmm13,%xmm8+	vpsrldq	$8,%xmm11,%xmm6+	vpsrldq	$8,%xmm10,%xmm5+	vpsrldq	$8,%xmm12,%xmm7+	vpaddq	%xmm8,%xmm13,%xmm13+	vpaddq	%xmm9,%xmm14,%xmm14+	vpaddq	%xmm5,%xmm10,%xmm10+	vpaddq	%xmm6,%xmm11,%xmm11+	vpaddq	%xmm7,%xmm12,%xmm12+++++	vpsrlq	$26,%xmm13,%xmm3+	vpand	%xmm15,%xmm13,%xmm13+	vpaddq	%xmm3,%xmm14,%xmm14++	vpsrlq	$26,%xmm10,%xmm0+	vpand	%xmm15,%xmm10,%xmm10+	vpaddq	%xmm0,%xmm11,%xmm11++	vpsrlq	$26,%xmm14,%xmm4+	vpand	%xmm15,%xmm14,%xmm14++	vpsrlq	$26,%xmm11,%xmm1+	vpand	%xmm15,%xmm11,%xmm11+	vpaddq	%xmm1,%xmm12,%xmm12++	vpaddq	%xmm4,%xmm10,%xmm10+	vpsllq	$2,%xmm4,%xmm4+	vpaddq	%xmm4,%xmm10,%xmm10++	vpsrlq	$26,%xmm12,%xmm2+	vpand	%xmm15,%xmm12,%xmm12+	vpaddq	%xmm2,%xmm13,%xmm13++	vpsrlq	$26,%xmm10,%xmm0+	vpand	%xmm15,%xmm10,%xmm10+	vpaddq	%xmm0,%xmm11,%xmm11++	vpsrlq	$26,%xmm13,%xmm3+	vpand	%xmm15,%xmm13,%xmm13+	vpaddq	%xmm3,%xmm14,%xmm14++	vmovd	%xmm10,-112(%rdi)+	vmovd	%xmm11,-108(%rdi)+	vmovd	%xmm12,-104(%rdi)+	vmovd	%xmm13,-100(%rdi)+	vmovd	%xmm14,-96(%rdi)+	leaq	88(%r11),%rsp+.cfi_def_cfa	%rsp,8+	vzeroupper+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_poly1305_asm_blocks_avx,.-crypton_poly1305_asm_blocks_avx+.type	crypton_poly1305_asm_blocks_avx2,@function+.align	32+crypton_poly1305_asm_blocks_avx2:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	movl	20(%rdi),%r8d+	cmpq	$128,%rdx+	jb	.Lblocks++	andq	$-16,%rdx++	vzeroupper++	testl	%r8d,%r8d+	jz	.Lbase2_64_avx2++	testq	$63,%rdx+	jz	.Leven_avx2++	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56+	leaq	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+.Lblocks_avx2_body:++	movq	%rdx,%r15++	movq	0(%rdi),%r8+	movq	8(%rdi),%r9+	movl	16(%rdi),%ebp++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13+++	movl	%r8d,%r14d+	andq	$-2147483648,%r8+	movq	%r9,%r12+	movl	%r9d,%ebx+	andq	$-2147483648,%r9++	shrq	$6,%r8+	shlq	$52,%r12+	addq	%r8,%r14+	shrq	$12,%rbx+	shrq	$18,%r9+	addq	%r12,%r14+	adcq	%r9,%rbx++	movq	%rbp,%r8+	shlq	$40,%r8+	shrq	$24,%rbp+	addq	%r8,%rbx+	adcq	$0,%rbp++	movq	%r13,%r12+	movq	%r13,%rax+	shrq	$2,%r13+	addq	%r12,%r13++.Lbase2_26_pre_avx2:+	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp+	subq	$16,%r15++	call	__crypton_poly1305_asm_block+	movq	%r12,%rax++	testq	$63,%r15+	jnz	.Lbase2_26_pre_avx2+++	movq	%r14,%rax+	movq	%r14,%rdx+	shrq	$52,%r14+	movq	%rbx,%r11+	movq	%rbx,%r12+	shrq	$26,%rdx+	andq	$0x3ffffff,%rax+	shlq	$12,%r11+	andq	$0x3ffffff,%rdx+	shrq	$14,%rbx+	orq	%r11,%r14+	shlq	$24,%rbp+	andq	$0x3ffffff,%r14+	shrq	$40,%r12+	andq	$0x3ffffff,%rbx+	orq	%r12,%rbp++	vmovd	%eax,%xmm0+	vmovd	%edx,%xmm1+	vmovd	%r14d,%xmm2+	vmovd	%ebx,%xmm3+	vmovd	%ebp,%xmm4++	movq	%r15,%rdx++	movq	8(%rsp),%r15+.cfi_restore	%r15+	movq	16(%rsp),%r14+.cfi_restore	%r14+	movq	24(%rsp),%r13+.cfi_restore	%r13+	movq	32(%rsp),%r12+.cfi_restore	%r12+	movq	40(%rsp),%rbp+.cfi_restore	%rbp+	movq	48(%rsp),%rbx+.cfi_restore	%rbx+	leaq	56(%rsp),%rax+	leaq	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+.Lblocks_avx2_epilogue:+	jmp	.Ldo_avx2+.cfi_endproc	++.align	32+.Lbase2_64_avx2:+.cfi_startproc	+	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56+	leaq	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+.Lbase2_64_avx2_body:++	movq	%rdx,%r15++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13++	movq	0(%rdi),%r14+	movq	8(%rdi),%rbx+	movl	16(%rdi),%ebp++	movq	%r13,%r12+	movq	%r13,%rax+	shrq	$2,%r13+	addq	%r12,%r13++	testq	$63,%rdx+	jz	.Linit_avx2++.Lbase2_64_pre_avx2:+	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp+	subq	$16,%r15++	call	__crypton_poly1305_asm_block+	movq	%r12,%rax++	testq	$63,%r15+	jnz	.Lbase2_64_pre_avx2++.Linit_avx2:++	movq	%r14,%rax+	movq	%r14,%rdx+	shrq	$52,%r14+	movq	%rbx,%r8+	movq	%rbx,%r9+	shrq	$26,%rdx+	andq	$0x3ffffff,%rax+	shlq	$12,%r8+	andq	$0x3ffffff,%rdx+	shrq	$14,%rbx+	orq	%r8,%r14+	shlq	$24,%rbp+	andq	$0x3ffffff,%r14+	shrq	$40,%r9+	andq	$0x3ffffff,%rbx+	orq	%r9,%rbp++	vmovd	%eax,%xmm0+	vmovd	%edx,%xmm1+	vmovd	%r14d,%xmm2+	vmovd	%ebx,%xmm3+	vmovd	%ebp,%xmm4+	movl	$1,20(%rdi)++	call	__crypton_poly1305_asm_init_avx++	movq	%r15,%rdx++	movq	8(%rsp),%r15+.cfi_restore	%r15+	movq	16(%rsp),%r14+.cfi_restore	%r14+	movq	24(%rsp),%r13+.cfi_restore	%r13+	movq	32(%rsp),%r12+.cfi_restore	%r12+	movq	40(%rsp),%rbp+.cfi_restore	%rbp+	movq	48(%rsp),%rbx+.cfi_restore	%rbx+	leaq	56(%rsp),%rax+	leaq	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+.Lbase2_64_avx2_epilogue:+	jmp	.Ldo_avx2+.cfi_endproc	++.align	32+.Leven_avx2:+.cfi_startproc	+	vmovd	0(%rdi),%xmm0+	vmovd	4(%rdi),%xmm1+	vmovd	8(%rdi),%xmm2+	vmovd	12(%rdi),%xmm3+	vmovd	16(%rdi),%xmm4++.Ldo_avx2:+	leaq	-8(%rsp),%r11+.cfi_def_cfa	%r11,16+	subq	$0x128,%rsp+	leaq	.Lconst(%rip),%rcx+	leaq	48+64(%rdi),%rdi+	vmovdqa	96(%rcx),%ymm7+++	vmovdqu	-64(%rdi),%xmm9+	andq	$-512,%rsp+	vmovdqu	-48(%rdi),%xmm10+	vmovdqu	-32(%rdi),%xmm6+	vmovdqu	-16(%rdi),%xmm11+	vmovdqu	0(%rdi),%xmm12+	vmovdqu	16(%rdi),%xmm13+	leaq	144(%rsp),%rax+	vmovdqu	32(%rdi),%xmm14+	vpermd	%ymm9,%ymm7,%ymm9+	vmovdqu	48(%rdi),%xmm15+	vpermd	%ymm10,%ymm7,%ymm10+	vmovdqu	64(%rdi),%xmm5+	vpermd	%ymm6,%ymm7,%ymm6+	vmovdqa	%ymm9,0(%rsp)+	vpermd	%ymm11,%ymm7,%ymm11+	vmovdqa	%ymm10,32-144(%rax)+	vpermd	%ymm12,%ymm7,%ymm12+	vmovdqa	%ymm6,64-144(%rax)+	vpermd	%ymm13,%ymm7,%ymm13+	vmovdqa	%ymm11,96-144(%rax)+	vpermd	%ymm14,%ymm7,%ymm14+	vmovdqa	%ymm12,128-144(%rax)+	vpermd	%ymm15,%ymm7,%ymm15+	vmovdqa	%ymm13,160-144(%rax)+	vpermd	%ymm5,%ymm7,%ymm5+	vmovdqa	%ymm14,192-144(%rax)+	vmovdqa	%ymm15,224-144(%rax)+	vmovdqa	%ymm5,256-144(%rax)+	vmovdqa	64(%rcx),%ymm5++++	vmovdqu	0(%rsi),%xmm7+	vmovdqu	16(%rsi),%xmm8+	vinserti128	$1,32(%rsi),%ymm7,%ymm7+	vinserti128	$1,48(%rsi),%ymm8,%ymm8+	leaq	64(%rsi),%rsi++	vpsrldq	$6,%ymm7,%ymm9+	vpsrldq	$6,%ymm8,%ymm10+	vpunpckhqdq	%ymm8,%ymm7,%ymm6+	vpunpcklqdq	%ymm10,%ymm9,%ymm9+	vpunpcklqdq	%ymm8,%ymm7,%ymm7++	vpsrlq	$30,%ymm9,%ymm10+	vpsrlq	$4,%ymm9,%ymm9+	vpsrlq	$26,%ymm7,%ymm8+	vpsrlq	$40,%ymm6,%ymm6+	vpand	%ymm5,%ymm9,%ymm9+	vpand	%ymm5,%ymm7,%ymm7+	vpand	%ymm5,%ymm8,%ymm8+	vpand	%ymm5,%ymm10,%ymm10+	vpor	32(%rcx),%ymm6,%ymm6++	vpaddq	%ymm2,%ymm9,%ymm2+	subq	$64,%rdx+	jz	.Ltail_avx2+	jmp	.Loop_avx2++.align	32+.Loop_avx2:+++++++++	vpaddq	%ymm0,%ymm7,%ymm0+	vmovdqa	0(%rsp),%ymm7+	vpaddq	%ymm1,%ymm8,%ymm1+	vmovdqa	32(%rsp),%ymm8+	vpaddq	%ymm3,%ymm10,%ymm3+	vmovdqa	96(%rsp),%ymm9+	vpaddq	%ymm4,%ymm6,%ymm4+	vmovdqa	48(%rax),%ymm10+	vmovdqa	112(%rax),%ymm5+++++++++++++++++	vpmuludq	%ymm2,%ymm7,%ymm13+	vpmuludq	%ymm2,%ymm8,%ymm14+	vpmuludq	%ymm2,%ymm9,%ymm15+	vpmuludq	%ymm2,%ymm10,%ymm11+	vpmuludq	%ymm2,%ymm5,%ymm12++	vpmuludq	%ymm0,%ymm8,%ymm6+	vpmuludq	%ymm1,%ymm8,%ymm2+	vpaddq	%ymm6,%ymm12,%ymm12+	vpaddq	%ymm2,%ymm13,%ymm13+	vpmuludq	%ymm3,%ymm8,%ymm6+	vpmuludq	64(%rsp),%ymm4,%ymm2+	vpaddq	%ymm6,%ymm15,%ymm15+	vpaddq	%ymm2,%ymm11,%ymm11+	vmovdqa	-16(%rax),%ymm8++	vpmuludq	%ymm0,%ymm7,%ymm6+	vpmuludq	%ymm1,%ymm7,%ymm2+	vpaddq	%ymm6,%ymm11,%ymm11+	vpaddq	%ymm2,%ymm12,%ymm12+	vpmuludq	%ymm3,%ymm7,%ymm6+	vpmuludq	%ymm4,%ymm7,%ymm2+	vmovdqu	0(%rsi),%xmm7+	vpaddq	%ymm6,%ymm14,%ymm14+	vpaddq	%ymm2,%ymm15,%ymm15+	vinserti128	$1,32(%rsi),%ymm7,%ymm7++	vpmuludq	%ymm3,%ymm8,%ymm6+	vpmuludq	%ymm4,%ymm8,%ymm2+	vmovdqu	16(%rsi),%xmm8+	vpaddq	%ymm6,%ymm11,%ymm11+	vpaddq	%ymm2,%ymm12,%ymm12+	vmovdqa	16(%rax),%ymm2+	vpmuludq	%ymm1,%ymm9,%ymm6+	vpmuludq	%ymm0,%ymm9,%ymm9+	vpaddq	%ymm6,%ymm14,%ymm14+	vpaddq	%ymm9,%ymm13,%ymm13+	vinserti128	$1,48(%rsi),%ymm8,%ymm8+	leaq	64(%rsi),%rsi++	vpmuludq	%ymm1,%ymm2,%ymm6+	vpmuludq	%ymm0,%ymm2,%ymm2+	vpsrldq	$6,%ymm7,%ymm9+	vpaddq	%ymm6,%ymm15,%ymm15+	vpaddq	%ymm2,%ymm14,%ymm14+	vpmuludq	%ymm3,%ymm10,%ymm6+	vpmuludq	%ymm4,%ymm10,%ymm2+	vpsrldq	$6,%ymm8,%ymm10+	vpaddq	%ymm6,%ymm12,%ymm12+	vpaddq	%ymm2,%ymm13,%ymm13+	vpunpckhqdq	%ymm8,%ymm7,%ymm6++	vpmuludq	%ymm3,%ymm5,%ymm3+	vpmuludq	%ymm4,%ymm5,%ymm4+	vpunpcklqdq	%ymm8,%ymm7,%ymm7+	vpaddq	%ymm3,%ymm13,%ymm2+	vpaddq	%ymm4,%ymm14,%ymm3+	vpunpcklqdq	%ymm10,%ymm9,%ymm10+	vpmuludq	80(%rax),%ymm0,%ymm4+	vpmuludq	%ymm1,%ymm5,%ymm0+	vmovdqa	64(%rcx),%ymm5+	vpaddq	%ymm4,%ymm15,%ymm4+	vpaddq	%ymm0,%ymm11,%ymm0+++++	vpsrlq	$26,%ymm3,%ymm14+	vpand	%ymm5,%ymm3,%ymm3+	vpaddq	%ymm14,%ymm4,%ymm4++	vpsrlq	$26,%ymm0,%ymm11+	vpand	%ymm5,%ymm0,%ymm0+	vpaddq	%ymm11,%ymm12,%ymm1++	vpsrlq	$26,%ymm4,%ymm15+	vpand	%ymm5,%ymm4,%ymm4++	vpsrlq	$4,%ymm10,%ymm9++	vpsrlq	$26,%ymm1,%ymm12+	vpand	%ymm5,%ymm1,%ymm1+	vpaddq	%ymm12,%ymm2,%ymm2++	vpaddq	%ymm15,%ymm0,%ymm0+	vpsllq	$2,%ymm15,%ymm15+	vpaddq	%ymm15,%ymm0,%ymm0++	vpand	%ymm5,%ymm9,%ymm9+	vpsrlq	$26,%ymm7,%ymm8++	vpsrlq	$26,%ymm2,%ymm13+	vpand	%ymm5,%ymm2,%ymm2+	vpaddq	%ymm13,%ymm3,%ymm3++	vpaddq	%ymm9,%ymm2,%ymm2+	vpsrlq	$30,%ymm10,%ymm10++	vpsrlq	$26,%ymm0,%ymm11+	vpand	%ymm5,%ymm0,%ymm0+	vpaddq	%ymm11,%ymm1,%ymm1++	vpsrlq	$40,%ymm6,%ymm6++	vpsrlq	$26,%ymm3,%ymm14+	vpand	%ymm5,%ymm3,%ymm3+	vpaddq	%ymm14,%ymm4,%ymm4++	vpand	%ymm5,%ymm7,%ymm7+	vpand	%ymm5,%ymm8,%ymm8+	vpand	%ymm5,%ymm10,%ymm10+	vpor	32(%rcx),%ymm6,%ymm6++	subq	$64,%rdx+	jnz	.Loop_avx2++.byte	0x66,0x90+.Ltail_avx2:++++++++	vpaddq	%ymm0,%ymm7,%ymm0+	vmovdqu	4(%rsp),%ymm7+	vpaddq	%ymm1,%ymm8,%ymm1+	vmovdqu	36(%rsp),%ymm8+	vpaddq	%ymm3,%ymm10,%ymm3+	vmovdqu	100(%rsp),%ymm9+	vpaddq	%ymm4,%ymm6,%ymm4+	vmovdqu	52(%rax),%ymm10+	vmovdqu	116(%rax),%ymm5++	vpmuludq	%ymm2,%ymm7,%ymm13+	vpmuludq	%ymm2,%ymm8,%ymm14+	vpmuludq	%ymm2,%ymm9,%ymm15+	vpmuludq	%ymm2,%ymm10,%ymm11+	vpmuludq	%ymm2,%ymm5,%ymm12++	vpmuludq	%ymm0,%ymm8,%ymm6+	vpmuludq	%ymm1,%ymm8,%ymm2+	vpaddq	%ymm6,%ymm12,%ymm12+	vpaddq	%ymm2,%ymm13,%ymm13+	vpmuludq	%ymm3,%ymm8,%ymm6+	vpmuludq	68(%rsp),%ymm4,%ymm2+	vpaddq	%ymm6,%ymm15,%ymm15+	vpaddq	%ymm2,%ymm11,%ymm11++	vpmuludq	%ymm0,%ymm7,%ymm6+	vpmuludq	%ymm1,%ymm7,%ymm2+	vpaddq	%ymm6,%ymm11,%ymm11+	vmovdqu	-12(%rax),%ymm8+	vpaddq	%ymm2,%ymm12,%ymm12+	vpmuludq	%ymm3,%ymm7,%ymm6+	vpmuludq	%ymm4,%ymm7,%ymm2+	vpaddq	%ymm6,%ymm14,%ymm14+	vpaddq	%ymm2,%ymm15,%ymm15++	vpmuludq	%ymm3,%ymm8,%ymm6+	vpmuludq	%ymm4,%ymm8,%ymm2+	vpaddq	%ymm6,%ymm11,%ymm11+	vpaddq	%ymm2,%ymm12,%ymm12+	vmovdqu	20(%rax),%ymm2+	vpmuludq	%ymm1,%ymm9,%ymm6+	vpmuludq	%ymm0,%ymm9,%ymm9+	vpaddq	%ymm6,%ymm14,%ymm14+	vpaddq	%ymm9,%ymm13,%ymm13++	vpmuludq	%ymm1,%ymm2,%ymm6+	vpmuludq	%ymm0,%ymm2,%ymm2+	vpaddq	%ymm6,%ymm15,%ymm15+	vpaddq	%ymm2,%ymm14,%ymm14+	vpmuludq	%ymm3,%ymm10,%ymm6+	vpmuludq	%ymm4,%ymm10,%ymm2+	vpaddq	%ymm6,%ymm12,%ymm12+	vpaddq	%ymm2,%ymm13,%ymm13++	vpmuludq	%ymm3,%ymm5,%ymm3+	vpmuludq	%ymm4,%ymm5,%ymm4+	vpaddq	%ymm3,%ymm13,%ymm2+	vpaddq	%ymm4,%ymm14,%ymm3+	vpmuludq	84(%rax),%ymm0,%ymm4+	vpmuludq	%ymm1,%ymm5,%ymm0+	vmovdqa	64(%rcx),%ymm5+	vpaddq	%ymm4,%ymm15,%ymm4+	vpaddq	%ymm0,%ymm11,%ymm0+++++	vpsrldq	$8,%ymm12,%ymm8+	vpsrldq	$8,%ymm2,%ymm9+	vpsrldq	$8,%ymm3,%ymm10+	vpsrldq	$8,%ymm4,%ymm6+	vpsrldq	$8,%ymm0,%ymm7+	vpaddq	%ymm8,%ymm12,%ymm12+	vpaddq	%ymm9,%ymm2,%ymm2+	vpaddq	%ymm10,%ymm3,%ymm3+	vpaddq	%ymm6,%ymm4,%ymm4+	vpaddq	%ymm7,%ymm0,%ymm0++	vpermq	$0x2,%ymm3,%ymm10+	vpermq	$0x2,%ymm4,%ymm6+	vpermq	$0x2,%ymm0,%ymm7+	vpermq	$0x2,%ymm12,%ymm8+	vpermq	$0x2,%ymm2,%ymm9+	vpaddq	%ymm10,%ymm3,%ymm3+	vpaddq	%ymm6,%ymm4,%ymm4+	vpaddq	%ymm7,%ymm0,%ymm0+	vpaddq	%ymm8,%ymm12,%ymm12+	vpaddq	%ymm9,%ymm2,%ymm2+++++	vpsrlq	$26,%ymm3,%ymm14+	vpand	%ymm5,%ymm3,%ymm3+	vpaddq	%ymm14,%ymm4,%ymm4++	vpsrlq	$26,%ymm0,%ymm11+	vpand	%ymm5,%ymm0,%ymm0+	vpaddq	%ymm11,%ymm12,%ymm1++	vpsrlq	$26,%ymm4,%ymm15+	vpand	%ymm5,%ymm4,%ymm4++	vpsrlq	$26,%ymm1,%ymm12+	vpand	%ymm5,%ymm1,%ymm1+	vpaddq	%ymm12,%ymm2,%ymm2++	vpaddq	%ymm15,%ymm0,%ymm0+	vpsllq	$2,%ymm15,%ymm15+	vpaddq	%ymm15,%ymm0,%ymm0++	vpsrlq	$26,%ymm2,%ymm13+	vpand	%ymm5,%ymm2,%ymm2+	vpaddq	%ymm13,%ymm3,%ymm3++	vpsrlq	$26,%ymm0,%ymm11+	vpand	%ymm5,%ymm0,%ymm0+	vpaddq	%ymm11,%ymm1,%ymm1++	vpsrlq	$26,%ymm3,%ymm14+	vpand	%ymm5,%ymm3,%ymm3+	vpaddq	%ymm14,%ymm4,%ymm4++	vmovd	%xmm0,-112(%rdi)+	vmovd	%xmm1,-108(%rdi)+	vmovd	%xmm2,-104(%rdi)+	vmovd	%xmm3,-100(%rdi)+	vmovd	%xmm4,-96(%rdi)+	leaq	8(%r11),%rsp+.cfi_def_cfa	%rsp,8+	vzeroupper+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_poly1305_asm_blocks_avx2,.-crypton_poly1305_asm_blocks_avx2+.align	64+.Lconst:+.Lmask24:+.long	0x0ffffff,0,0x0ffffff,0,0x0ffffff,0,0x0ffffff,0+.L129:+.long	16777216,0,16777216,0,16777216,0,16777216,0+.Lmask26:+.long	0x3ffffff,0,0x3ffffff,0,0x3ffffff,0,0x3ffffff,0+.Lpermd_avx2:+.long	2,2,2,3,2,0,2,1+.Lpermd_avx512:+.long	0,0,0,1, 0,2,0,3, 0,4,0,5, 0,6,0,7++.L2_44_inp_permd:+.long	0,1,1,2,2,3,7,7+.L2_44_inp_shift:+.quad	0,12,24,64+.L2_44_mask:+.quad	0xfffffffffff,0xfffffffffff,0x3ffffffffff,0xffffffffffffffff+.L2_44_shift_rgt:+.quad	44,44,42,64+.L2_44_shift_lft:+.quad	8,8,10,64++.align	64+.Lx_mask44:+.quad	0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff+.quad	0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff+.Lx_mask42:+.quad	0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff+.quad	0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff+.byte	80,111,108,121,49,51,48,53,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.align	16+.globl	crypton_xor128_encrypt_n_pad+.type	crypton_xor128_encrypt_n_pad,@function+.align	16+crypton_xor128_encrypt_n_pad:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa++	subq	%rdx,%rsi+	subq	%rdx,%rdi+	movq	%rcx,%r10+	shrq	$4,%rcx+	jz	.Ltail_enc+	nop+.Loop_enc_xmm:+	movdqu	(%rsi,%rdx,1),%xmm0+	pxor	(%rdx),%xmm0+	movdqu	%xmm0,(%rdi,%rdx,1)+	movdqa	%xmm0,(%rdx)+	leaq	16(%rdx),%rdx+	decq	%rcx+	jnz	.Loop_enc_xmm++	andq	$15,%r10+	jz	.Ldone_enc++.Ltail_enc:+	movq	$16,%rcx+	subq	%r10,%rcx+	xorl	%eax,%eax+.Loop_enc_byte:+	movb	(%rsi,%rdx,1),%al+	xorb	(%rdx),%al+	movb	%al,(%rdi,%rdx,1)+	movb	%al,(%rdx)+	leaq	1(%rdx),%rdx+	decq	%r10+	jnz	.Loop_enc_byte++	xorl	%eax,%eax+.Loop_enc_pad:+	movb	%al,(%rdx)+	leaq	1(%rdx),%rdx+	decq	%rcx+	jnz	.Loop_enc_pad++.Ldone_enc:+	movq	%rdx,%rax+	.byte	0xf3,0xc3+.cfi_endproc+.size	crypton_xor128_encrypt_n_pad,.-crypton_xor128_encrypt_n_pad++.globl	crypton_xor128_decrypt_n_pad+.type	crypton_xor128_decrypt_n_pad,@function+.align	16+crypton_xor128_decrypt_n_pad:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa++	subq	%rdx,%rsi+	subq	%rdx,%rdi+	movq	%rcx,%r10+	shrq	$4,%rcx+	jz	.Ltail_dec+	nop+.Loop_dec_xmm:+	movdqu	(%rsi,%rdx,1),%xmm0+	movdqa	(%rdx),%xmm1+	pxor	%xmm0,%xmm1+	movdqu	%xmm1,(%rdi,%rdx,1)+	movdqa	%xmm0,(%rdx)+	leaq	16(%rdx),%rdx+	decq	%rcx+	jnz	.Loop_dec_xmm++	pxor	%xmm1,%xmm1+	andq	$15,%r10+	jz	.Ldone_dec++.Ltail_dec:+	movq	$16,%rcx+	subq	%r10,%rcx+	xorl	%eax,%eax+	xorq	%r11,%r11+.Loop_dec_byte:+	movb	(%rsi,%rdx,1),%r11b+	movb	(%rdx),%al+	xorb	%r11b,%al+	movb	%al,(%rdi,%rdx,1)+	movb	%r11b,(%rdx)+	leaq	1(%rdx),%rdx+	decq	%r10+	jnz	.Loop_dec_byte++	xorl	%eax,%eax+.Loop_dec_pad:+	movb	%al,(%rdx)+	leaq	1(%rdx),%rdx+	decq	%rcx+	jnz	.Loop_dec_pad++.Ldone_dec:+	movq	%rdx,%rax+	.byte	0xf3,0xc3+.cfi_endproc+.size	crypton_xor128_decrypt_n_pad,.-crypton_xor128_decrypt_n_pad++.section	.note.gnu.property,"a",@note+	.long	4,2f-1f,5+	.byte	0x47,0x4E,0x55,0+1:	.long	0xc0000002,4,3+.align	8+2:++.section	.note.GNU-stack,"",@progbits
+ cbits/asm/poly1305-x86_64-macosx.S view
@@ -0,0 +1,2024 @@+.text	++++.globl	_crypton_poly1305_asm_init+.private_extern	_crypton_poly1305_asm_init+.globl	_crypton_poly1305_asm_blocks+.private_extern	_crypton_poly1305_asm_blocks+.globl	_crypton_poly1305_asm_emit+.private_extern	_crypton_poly1305_asm_emit+++.p2align	5+_crypton_poly1305_asm_init:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa++	xorq	%rax,%rax+	movq	%rax,0(%rdi)+	movq	%rax,8(%rdi)+	movq	%rax,16(%rdi)++	cmpq	$0,%rsi+	je	L$no_key++	movq	$0x0ffffffc0fffffff,%rax+	leaq	-3(%rax),%rcx+	andq	0(%rsi),%rax+	andq	8(%rsi),%rcx+	movq	%rax,24(%rdi)+	movq	%rcx,32(%rdi)+	movl	$-1,48(%rdi)+	leaq	_crypton_poly1305_asm_blocks(%rip),%r10+	leaq	_crypton_poly1305_asm_emit(%rip),%r11+	movq	_crypton_ia32cap_P+4(%rip),%r9+	leaq	crypton_poly1305_asm_blocks_avx(%rip),%rax+	btq	$28,%r9+	cmovcq	%rax,%r10+	leaq	crypton_poly1305_asm_blocks_avx2(%rip),%rax+	btq	$37,%r9+	cmovcq	%rax,%r10+	movq	%r10,0(%rdx)+	movq	%r11,8(%rdx)+	movl	$1,%eax+L$no_key:+	.byte	0xf3,0xc3+.cfi_endproc++++.p2align	5+_crypton_poly1305_asm_blocks:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++L$blocks:+	shrq	$4,%rdx+	jz	L$no_data++	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56+	leaq	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+L$blocks_body:++	movq	%rdx,%r15++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13++	movq	0(%rdi),%r14+	movq	8(%rdi),%rbx+	movq	16(%rdi),%rbp++	movl	%r14d,%eax+	movl	4(%rdi),%edx+	movl	%ebx,%r8d+	movl	12(%rdi),%r10d+	movl	%ebp,%r12d++	shlq	$26,%rdx+	movq	%r8,%r9+	shlq	$52,%r8+	addq	%rdx,%rax+	shrq	$12,%r9+	addq	%rax,%r8+	adcq	$0,%r9++	shlq	$14,%r10+	movq	%r12,%rax+	shrq	$24,%r12+	addq	%r10,%r9+	shlq	$40,%rax+	addq	%rax,%r9+	adcq	$0,%r12++	cmpq	$4,%rbp++	cmovaq	%r8,%r14+	cmovaq	%r9,%rbx+	cmovaq	%r12,%rbp++	movq	%r13,%r12+	shrq	$2,%r13+	movq	%r12,%rax+	addq	%r12,%r13+	jmp	L$oop++.p2align	5+L$oop:+	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp+	mulq	%r14+	movq	%rax,%r9+	movq	%r11,%rax+	movq	%rdx,%r10++	mulq	%r14+	movq	%rax,%r14+	movq	%r11,%rax+	movq	%rdx,%r8++	mulq	%rbx+	addq	%rax,%r9+	movq	%r13,%rax+	adcq	%rdx,%r10++	mulq	%rbx+	movq	%rbp,%rbx+	addq	%rax,%r14+	adcq	%rdx,%r8++	imulq	%r13,%rbx+	addq	%rbx,%r9+	movq	%r8,%rbx+	adcq	$0,%r10++	imulq	%r11,%rbp+	addq	%r9,%rbx+	movq	$-4,%rax+	adcq	%rbp,%r10++	andq	%r10,%rax+	movq	%r10,%rbp+	shrq	$2,%r10+	andq	$3,%rbp+	addq	%r10,%rax+	addq	%rax,%r14+	adcq	$0,%rbx+	adcq	$0,%rbp+	movq	%r12,%rax+	decq	%r15+	jnz	L$oop++	movq	%r14,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rbp,16(%rdi)++	movq	8(%rsp),%r15+.cfi_restore	%r15+	movq	16(%rsp),%r14+.cfi_restore	%r14+	movq	24(%rsp),%r13+.cfi_restore	%r13+	movq	32(%rsp),%r12+.cfi_restore	%r12+	movq	40(%rsp),%rbp+.cfi_restore	%rbp+	movq	48(%rsp),%rbx+.cfi_restore	%rbx+	leaq	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+L$no_data:+L$blocks_epilogue:+	.byte	0xf3,0xc3+.cfi_endproc	++++.p2align	5+_crypton_poly1305_asm_emit:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa++	movl	0(%rdi),%eax+	movl	4(%rdi),%ecx+	movl	8(%rdi),%r8d+	movl	12(%rdi),%r11d+	movl	16(%rdi),%r10d++	shlq	$26,%rcx+	movq	%r8,%r9+	shlq	$52,%r8+	addq	%rcx,%rax+	shrq	$12,%r9+	addq	%rax,%r8+	adcq	$0,%r9++	shlq	$14,%r11+	movq	%r10,%rax+	shrq	$24,%r10+	addq	%r11,%r9+	movq	0(%rdi),%rcx+	shlq	$40,%rax+	movq	8(%rdi),%r11+	addq	%rax,%r9+	movq	16(%rdi),%rax+	adcq	$0,%r10++	cmpq	$4,%rax++	cmovbeq	%rcx,%r8+	cmovbeq	%r11,%r9+	cmovbeq	%rax,%r10++	movq	%r8,%rax+	addq	$5,%r8+	movq	%r9,%rcx+	adcq	$0,%r9+	adcq	$0,%r10+	shrq	$2,%r10+	cmovnzq	%r8,%rax+	cmovnzq	%r9,%rcx++	addq	0(%rdx),%rax+	adcq	8(%rdx),%rcx+	movq	%rax,0(%rsi)+	movq	%rcx,8(%rsi)++	.byte	0xf3,0xc3+.cfi_endproc+++.p2align	5+__crypton_poly1305_asm_block:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa++	mulq	%r14+	movq	%rax,%r9+	movq	%r11,%rax+	movq	%rdx,%r10++	mulq	%r14+	movq	%rax,%r14+	movq	%r11,%rax+	movq	%rdx,%r8++	mulq	%rbx+	addq	%rax,%r9+	movq	%r13,%rax+	adcq	%rdx,%r10++	mulq	%rbx+	movq	%rbp,%rbx+	addq	%rax,%r14+	adcq	%rdx,%r8++	imulq	%r13,%rbx+	addq	%rbx,%r9+	movq	%r8,%rbx+	adcq	$0,%r10++	imulq	%r11,%rbp+	addq	%r9,%rbx+	movq	$-4,%rax+	adcq	%rbp,%r10++	andq	%r10,%rax+	movq	%r10,%rbp+	shrq	$2,%r10+	andq	$3,%rbp+	addq	%r10,%rax+	addq	%rax,%r14+	adcq	$0,%rbx+	adcq	$0,%rbp+	.byte	0xf3,0xc3+.cfi_endproc++++.p2align	5+__crypton_poly1305_asm_init_avx:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa++	cmpl	$-1,48(%rdi)+	jne	L$done_init_avx++	movq	%r11,%r14+	movq	%r12,%rbx+	xorq	%rbp,%rbp++	leaq	48+64(%rdi),%rdi++	movq	%r12,%rax+	call	__crypton_poly1305_asm_block++	movl	$0x3ffffff,%eax+	movl	$0x3ffffff,%edx+	movq	%r14,%r8+	andl	%r14d,%eax+	movq	%r11,%r9+	andl	%r11d,%edx+	movl	%eax,-64(%rdi)+	shrq	$26,%r8+	movl	%edx,-60(%rdi)+	shrq	$26,%r9++	movl	$0x3ffffff,%eax+	movl	$0x3ffffff,%edx+	andl	%r8d,%eax+	andl	%r9d,%edx+	movl	%eax,-48(%rdi)+	leal	(%rax,%rax,4),%eax+	movl	%edx,-44(%rdi)+	leal	(%rdx,%rdx,4),%edx+	movl	%eax,-32(%rdi)+	shrq	$26,%r8+	movl	%edx,-28(%rdi)+	shrq	$26,%r9++	movq	%rbx,%rax+	movq	%r12,%rdx+	shlq	$12,%rax+	shlq	$12,%rdx+	orq	%r8,%rax+	orq	%r9,%rdx+	andl	$0x3ffffff,%eax+	andl	$0x3ffffff,%edx+	movl	%eax,-16(%rdi)+	leal	(%rax,%rax,4),%eax+	movl	%edx,-12(%rdi)+	leal	(%rdx,%rdx,4),%edx+	movl	%eax,0(%rdi)+	movq	%rbx,%r8+	movl	%edx,4(%rdi)+	movq	%r12,%r9++	movl	$0x3ffffff,%eax+	movl	$0x3ffffff,%edx+	shrq	$14,%r8+	shrq	$14,%r9+	andl	%r8d,%eax+	andl	%r9d,%edx+	movl	%eax,16(%rdi)+	leal	(%rax,%rax,4),%eax+	movl	%edx,20(%rdi)+	leal	(%rdx,%rdx,4),%edx+	movl	%eax,32(%rdi)+	shrq	$26,%r8+	movl	%edx,36(%rdi)+	shrq	$26,%r9++	movq	%rbp,%rax+	shlq	$24,%rax+	orq	%rax,%r8+	movl	%r8d,48(%rdi)+	leaq	(%r8,%r8,4),%r8+	movl	%r9d,52(%rdi)+	leaq	(%r9,%r9,4),%r9+	movl	%r8d,64(%rdi)+	movl	%r9d,68(%rdi)++	movq	%r12,%rax+	call	__crypton_poly1305_asm_block++	movl	$0x3ffffff,%eax+	movq	%r14,%r8+	andl	%r14d,%eax+	shrq	$26,%r8+	movl	%eax,-52(%rdi)++	movl	$0x3ffffff,%edx+	andl	%r8d,%edx+	movl	%edx,-36(%rdi)+	leal	(%rdx,%rdx,4),%edx+	shrq	$26,%r8+	movl	%edx,-20(%rdi)++	movq	%rbx,%rax+	shlq	$12,%rax+	orq	%r8,%rax+	andl	$0x3ffffff,%eax+	movl	%eax,-4(%rdi)+	leal	(%rax,%rax,4),%eax+	movq	%rbx,%r8+	movl	%eax,12(%rdi)++	movl	$0x3ffffff,%edx+	shrq	$14,%r8+	andl	%r8d,%edx+	movl	%edx,28(%rdi)+	leal	(%rdx,%rdx,4),%edx+	shrq	$26,%r8+	movl	%edx,44(%rdi)++	movq	%rbp,%rax+	shlq	$24,%rax+	orq	%rax,%r8+	movl	%r8d,60(%rdi)+	leaq	(%r8,%r8,4),%r8+	movl	%r8d,76(%rdi)++	movq	%r12,%rax+	call	__crypton_poly1305_asm_block++	movl	$0x3ffffff,%eax+	movq	%r14,%r8+	andl	%r14d,%eax+	shrq	$26,%r8+	movl	%eax,-56(%rdi)++	movl	$0x3ffffff,%edx+	andl	%r8d,%edx+	movl	%edx,-40(%rdi)+	leal	(%rdx,%rdx,4),%edx+	shrq	$26,%r8+	movl	%edx,-24(%rdi)++	movq	%rbx,%rax+	shlq	$12,%rax+	orq	%r8,%rax+	andl	$0x3ffffff,%eax+	movl	%eax,-8(%rdi)+	leal	(%rax,%rax,4),%eax+	movq	%rbx,%r8+	movl	%eax,8(%rdi)++	movl	$0x3ffffff,%edx+	shrq	$14,%r8+	andl	%r8d,%edx+	movl	%edx,24(%rdi)+	leal	(%rdx,%rdx,4),%edx+	shrq	$26,%r8+	movl	%edx,40(%rdi)++	movq	%rbp,%rax+	shlq	$24,%rax+	orq	%rax,%r8+	movl	%r8d,56(%rdi)+	leaq	(%r8,%r8,4),%r8+	movl	%r8d,72(%rdi)++	leaq	-48-64(%rdi),%rdi+L$done_init_avx:+	.byte	0xf3,0xc3+.cfi_endproc++++.p2align	5+crypton_poly1305_asm_blocks_avx:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	movl	20(%rdi),%r8d+	cmpq	$128,%rdx+	jb	L$blocks++	andq	$-16,%rdx++	vzeroupper++	testl	%r8d,%r8d+	jz	L$base2_64_avx++	testq	$31,%rdx+	jz	L$even_avx++	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56+	leaq	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+L$blocks_avx_body:++	movq	%rdx,%r15++	movq	0(%rdi),%r8+	movq	8(%rdi),%r9+	movl	16(%rdi),%ebp++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13+++	movl	%r8d,%r14d+	andq	$-2147483648,%r8+	movq	%r9,%r12+	movl	%r9d,%ebx+	andq	$-2147483648,%r9++	shrq	$6,%r8+	shlq	$52,%r12+	addq	%r8,%r14+	shrq	$12,%rbx+	shrq	$18,%r9+	addq	%r12,%r14+	adcq	%r9,%rbx++	movq	%rbp,%r8+	shlq	$40,%r8+	shrq	$24,%rbp+	addq	%r8,%rbx+	adcq	$0,%rbp++	movq	%r13,%r12+	movq	%r13,%rax+	shrq	$2,%r13+	addq	%r12,%r13++	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp++	call	__crypton_poly1305_asm_block+++	movq	%r14,%rax+	movq	%r14,%rdx+	shrq	$52,%r14+	movq	%rbx,%r11+	movq	%rbx,%r12+	shrq	$26,%rdx+	andq	$0x3ffffff,%rax+	shlq	$12,%r11+	andq	$0x3ffffff,%rdx+	shrq	$14,%rbx+	orq	%r11,%r14+	shlq	$24,%rbp+	andq	$0x3ffffff,%r14+	shrq	$40,%r12+	andq	$0x3ffffff,%rbx+	orq	%r12,%rbp++	vmovd	%eax,%xmm0+	vmovd	%edx,%xmm1+	vmovd	%r14d,%xmm2+	vmovd	%ebx,%xmm3+	vmovd	%ebp,%xmm4++	leaq	-16(%r15),%rdx++	movq	8(%rsp),%r15+.cfi_restore	%r15+	movq	16(%rsp),%r14+.cfi_restore	%r14+	movq	24(%rsp),%r13+.cfi_restore	%r13+	movq	32(%rsp),%r12+.cfi_restore	%r12+	movq	40(%rsp),%rbp+.cfi_restore	%rbp+	movq	48(%rsp),%rbx+.cfi_restore	%rbx+	leaq	56(%rsp),%rax+	leaq	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+L$blocks_avx_epilogue:+	jmp	L$do_avx+.cfi_endproc	++.p2align	5+L$base2_64_avx:+.cfi_startproc	+	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56+	leaq	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+L$base2_64_avx_body:++	movq	%rdx,%r15++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13++	movq	0(%rdi),%r14+	movq	8(%rdi),%rbx+	movl	16(%rdi),%ebp++	movq	%r13,%r12+	movq	%r13,%rax+	shrq	$2,%r13+	addq	%r12,%r13++	testq	$31,%rdx+	jz	L$init_avx++	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp+	subq	$16,%r15++	call	__crypton_poly1305_asm_block++L$init_avx:++	movq	%r14,%rax+	movq	%r14,%rdx+	shrq	$52,%r14+	movq	%rbx,%r8+	movq	%rbx,%r9+	shrq	$26,%rdx+	andq	$0x3ffffff,%rax+	shlq	$12,%r8+	andq	$0x3ffffff,%rdx+	shrq	$14,%rbx+	orq	%r8,%r14+	shlq	$24,%rbp+	andq	$0x3ffffff,%r14+	shrq	$40,%r9+	andq	$0x3ffffff,%rbx+	orq	%r9,%rbp++	vmovd	%eax,%xmm0+	vmovd	%edx,%xmm1+	vmovd	%r14d,%xmm2+	vmovd	%ebx,%xmm3+	vmovd	%ebp,%xmm4+	movl	$1,20(%rdi)++	call	__crypton_poly1305_asm_init_avx++	movq	%r15,%rdx++	movq	8(%rsp),%r15+.cfi_restore	%r15+	movq	16(%rsp),%r14+.cfi_restore	%r14+	movq	24(%rsp),%r13+.cfi_restore	%r13+	movq	32(%rsp),%r12+.cfi_restore	%r12+	movq	40(%rsp),%rbp+.cfi_restore	%rbp+	movq	48(%rsp),%rbx+.cfi_restore	%rbx+	leaq	56(%rsp),%rax+	leaq	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+L$base2_64_avx_epilogue:+	jmp	L$do_avx+.cfi_endproc	++.p2align	5+L$even_avx:+.cfi_startproc	+	vmovd	0(%rdi),%xmm0+	vmovd	4(%rdi),%xmm1+	vmovd	8(%rdi),%xmm2+	vmovd	12(%rdi),%xmm3+	vmovd	16(%rdi),%xmm4++L$do_avx:+	leaq	-88(%rsp),%r11+.cfi_def_cfa	%r11,0x60+	subq	$0x178,%rsp+	subq	$64,%rdx+	leaq	-32(%rsi),%rax+	cmovcq	%rax,%rsi++	vmovdqu	48(%rdi),%xmm14+	leaq	112(%rdi),%rdi+	leaq	L$const(%rip),%rcx++++	vmovdqu	32(%rsi),%xmm5+	vmovdqu	48(%rsi),%xmm6+	vmovdqa	64(%rcx),%xmm15++	vpsrldq	$6,%xmm5,%xmm7+	vpsrldq	$6,%xmm6,%xmm8+	vpunpckhqdq	%xmm6,%xmm5,%xmm9+	vpunpcklqdq	%xmm6,%xmm5,%xmm5+	vpunpcklqdq	%xmm8,%xmm7,%xmm8++	vpsrlq	$40,%xmm9,%xmm9+	vpsrlq	$26,%xmm5,%xmm6+	vpand	%xmm15,%xmm5,%xmm5+	vpsrlq	$4,%xmm8,%xmm7+	vpand	%xmm15,%xmm6,%xmm6+	vpsrlq	$30,%xmm8,%xmm8+	vpand	%xmm15,%xmm7,%xmm7+	vpand	%xmm15,%xmm8,%xmm8+	vpor	32(%rcx),%xmm9,%xmm9++	jbe	L$skip_loop_avx+++	vmovdqu	-48(%rdi),%xmm11+	vmovdqu	-32(%rdi),%xmm12+	vpshufd	$0xEE,%xmm14,%xmm13+	vpshufd	$0x44,%xmm14,%xmm10+	vmovdqa	%xmm13,-144(%r11)+	vmovdqa	%xmm10,0(%rsp)+	vpshufd	$0xEE,%xmm11,%xmm14+	vmovdqu	-16(%rdi),%xmm10+	vpshufd	$0x44,%xmm11,%xmm11+	vmovdqa	%xmm14,-128(%r11)+	vmovdqa	%xmm11,16(%rsp)+	vpshufd	$0xEE,%xmm12,%xmm13+	vmovdqu	0(%rdi),%xmm11+	vpshufd	$0x44,%xmm12,%xmm12+	vmovdqa	%xmm13,-112(%r11)+	vmovdqa	%xmm12,32(%rsp)+	vpshufd	$0xEE,%xmm10,%xmm14+	vmovdqu	16(%rdi),%xmm12+	vpshufd	$0x44,%xmm10,%xmm10+	vmovdqa	%xmm14,-96(%r11)+	vmovdqa	%xmm10,48(%rsp)+	vpshufd	$0xEE,%xmm11,%xmm13+	vmovdqu	32(%rdi),%xmm10+	vpshufd	$0x44,%xmm11,%xmm11+	vmovdqa	%xmm13,-80(%r11)+	vmovdqa	%xmm11,64(%rsp)+	vpshufd	$0xEE,%xmm12,%xmm14+	vmovdqu	48(%rdi),%xmm11+	vpshufd	$0x44,%xmm12,%xmm12+	vmovdqa	%xmm14,-64(%r11)+	vmovdqa	%xmm12,80(%rsp)+	vpshufd	$0xEE,%xmm10,%xmm13+	vmovdqu	64(%rdi),%xmm12+	vpshufd	$0x44,%xmm10,%xmm10+	vmovdqa	%xmm13,-48(%r11)+	vmovdqa	%xmm10,96(%rsp)+	vpshufd	$0xEE,%xmm11,%xmm14+	vpshufd	$0x44,%xmm11,%xmm11+	vmovdqa	%xmm14,-32(%r11)+	vmovdqa	%xmm11,112(%rsp)+	vpshufd	$0xEE,%xmm12,%xmm13+	vmovdqa	0(%rsp),%xmm14+	vpshufd	$0x44,%xmm12,%xmm12+	vmovdqa	%xmm13,-16(%r11)+	vmovdqa	%xmm12,128(%rsp)++	jmp	L$oop_avx++.p2align	5+L$oop_avx:+++++++++++++++++++++	vpmuludq	%xmm5,%xmm14,%xmm10+	vpmuludq	%xmm6,%xmm14,%xmm11+	vmovdqa	%xmm2,32(%r11)+	vpmuludq	%xmm7,%xmm14,%xmm12+	vmovdqa	16(%rsp),%xmm2+	vpmuludq	%xmm8,%xmm14,%xmm13+	vpmuludq	%xmm9,%xmm14,%xmm14++	vmovdqa	%xmm0,0(%r11)+	vpmuludq	32(%rsp),%xmm9,%xmm0+	vmovdqa	%xmm1,16(%r11)+	vpmuludq	%xmm8,%xmm2,%xmm1+	vpaddq	%xmm0,%xmm10,%xmm10+	vpaddq	%xmm1,%xmm14,%xmm14+	vmovdqa	%xmm3,48(%r11)+	vpmuludq	%xmm7,%xmm2,%xmm0+	vpmuludq	%xmm6,%xmm2,%xmm1+	vpaddq	%xmm0,%xmm13,%xmm13+	vmovdqa	48(%rsp),%xmm3+	vpaddq	%xmm1,%xmm12,%xmm12+	vmovdqa	%xmm4,64(%r11)+	vpmuludq	%xmm5,%xmm2,%xmm2+	vpmuludq	%xmm7,%xmm3,%xmm0+	vpaddq	%xmm2,%xmm11,%xmm11++	vmovdqa	64(%rsp),%xmm4+	vpaddq	%xmm0,%xmm14,%xmm14+	vpmuludq	%xmm6,%xmm3,%xmm1+	vpmuludq	%xmm5,%xmm3,%xmm3+	vpaddq	%xmm1,%xmm13,%xmm13+	vmovdqa	80(%rsp),%xmm2+	vpaddq	%xmm3,%xmm12,%xmm12+	vpmuludq	%xmm9,%xmm4,%xmm0+	vpmuludq	%xmm8,%xmm4,%xmm4+	vpaddq	%xmm0,%xmm11,%xmm11+	vmovdqa	96(%rsp),%xmm3+	vpaddq	%xmm4,%xmm10,%xmm10++	vmovdqa	128(%rsp),%xmm4+	vpmuludq	%xmm6,%xmm2,%xmm1+	vpmuludq	%xmm5,%xmm2,%xmm2+	vpaddq	%xmm1,%xmm14,%xmm14+	vpaddq	%xmm2,%xmm13,%xmm13+	vpmuludq	%xmm9,%xmm3,%xmm0+	vpmuludq	%xmm8,%xmm3,%xmm1+	vpaddq	%xmm0,%xmm12,%xmm12+	vmovdqu	0(%rsi),%xmm0+	vpaddq	%xmm1,%xmm11,%xmm11+	vpmuludq	%xmm7,%xmm3,%xmm3+	vpmuludq	%xmm7,%xmm4,%xmm7+	vpaddq	%xmm3,%xmm10,%xmm10++	vmovdqu	16(%rsi),%xmm1+	vpaddq	%xmm7,%xmm11,%xmm11+	vpmuludq	%xmm8,%xmm4,%xmm8+	vpmuludq	%xmm9,%xmm4,%xmm9+	vpsrldq	$6,%xmm0,%xmm2+	vpaddq	%xmm8,%xmm12,%xmm12+	vpaddq	%xmm9,%xmm13,%xmm13+	vpsrldq	$6,%xmm1,%xmm3+	vpmuludq	112(%rsp),%xmm5,%xmm9+	vpmuludq	%xmm6,%xmm4,%xmm5+	vpunpckhqdq	%xmm1,%xmm0,%xmm4+	vpaddq	%xmm9,%xmm14,%xmm14+	vmovdqa	-144(%r11),%xmm9+	vpaddq	%xmm5,%xmm10,%xmm10++	vpunpcklqdq	%xmm1,%xmm0,%xmm0+	vpunpcklqdq	%xmm3,%xmm2,%xmm3+++	vpsrldq	$5,%xmm4,%xmm4+	vpsrlq	$26,%xmm0,%xmm1+	vpand	%xmm15,%xmm0,%xmm0+	vpsrlq	$4,%xmm3,%xmm2+	vpand	%xmm15,%xmm1,%xmm1+	vpand	0(%rcx),%xmm4,%xmm4+	vpsrlq	$30,%xmm3,%xmm3+	vpand	%xmm15,%xmm2,%xmm2+	vpand	%xmm15,%xmm3,%xmm3+	vpor	32(%rcx),%xmm4,%xmm4++	vpaddq	0(%r11),%xmm0,%xmm0+	vpaddq	16(%r11),%xmm1,%xmm1+	vpaddq	32(%r11),%xmm2,%xmm2+	vpaddq	48(%r11),%xmm3,%xmm3+	vpaddq	64(%r11),%xmm4,%xmm4++	leaq	32(%rsi),%rax+	leaq	64(%rsi),%rsi+	subq	$64,%rdx+	cmovcq	%rax,%rsi+++++++++++	vpmuludq	%xmm0,%xmm9,%xmm5+	vpmuludq	%xmm1,%xmm9,%xmm6+	vpaddq	%xmm5,%xmm10,%xmm10+	vpaddq	%xmm6,%xmm11,%xmm11+	vmovdqa	-128(%r11),%xmm7+	vpmuludq	%xmm2,%xmm9,%xmm5+	vpmuludq	%xmm3,%xmm9,%xmm6+	vpaddq	%xmm5,%xmm12,%xmm12+	vpaddq	%xmm6,%xmm13,%xmm13+	vpmuludq	%xmm4,%xmm9,%xmm9+	vpmuludq	-112(%r11),%xmm4,%xmm5+	vpaddq	%xmm9,%xmm14,%xmm14++	vpaddq	%xmm5,%xmm10,%xmm10+	vpmuludq	%xmm2,%xmm7,%xmm6+	vpmuludq	%xmm3,%xmm7,%xmm5+	vpaddq	%xmm6,%xmm13,%xmm13+	vmovdqa	-96(%r11),%xmm8+	vpaddq	%xmm5,%xmm14,%xmm14+	vpmuludq	%xmm1,%xmm7,%xmm6+	vpmuludq	%xmm0,%xmm7,%xmm7+	vpaddq	%xmm6,%xmm12,%xmm12+	vpaddq	%xmm7,%xmm11,%xmm11++	vmovdqa	-80(%r11),%xmm9+	vpmuludq	%xmm2,%xmm8,%xmm5+	vpmuludq	%xmm1,%xmm8,%xmm6+	vpaddq	%xmm5,%xmm14,%xmm14+	vpaddq	%xmm6,%xmm13,%xmm13+	vmovdqa	-64(%r11),%xmm7+	vpmuludq	%xmm0,%xmm8,%xmm8+	vpmuludq	%xmm4,%xmm9,%xmm5+	vpaddq	%xmm8,%xmm12,%xmm12+	vpaddq	%xmm5,%xmm11,%xmm11+	vmovdqa	-48(%r11),%xmm8+	vpmuludq	%xmm3,%xmm9,%xmm9+	vpmuludq	%xmm1,%xmm7,%xmm6+	vpaddq	%xmm9,%xmm10,%xmm10++	vmovdqa	-16(%r11),%xmm9+	vpaddq	%xmm6,%xmm14,%xmm14+	vpmuludq	%xmm0,%xmm7,%xmm7+	vpmuludq	%xmm4,%xmm8,%xmm5+	vpaddq	%xmm7,%xmm13,%xmm13+	vpaddq	%xmm5,%xmm12,%xmm12+	vmovdqu	32(%rsi),%xmm5+	vpmuludq	%xmm3,%xmm8,%xmm7+	vpmuludq	%xmm2,%xmm8,%xmm8+	vpaddq	%xmm7,%xmm11,%xmm11+	vmovdqu	48(%rsi),%xmm6+	vpaddq	%xmm8,%xmm10,%xmm10++	vpmuludq	%xmm2,%xmm9,%xmm2+	vpmuludq	%xmm3,%xmm9,%xmm3+	vpsrldq	$6,%xmm5,%xmm7+	vpaddq	%xmm2,%xmm11,%xmm11+	vpmuludq	%xmm4,%xmm9,%xmm4+	vpsrldq	$6,%xmm6,%xmm8+	vpaddq	%xmm3,%xmm12,%xmm2+	vpaddq	%xmm4,%xmm13,%xmm3+	vpmuludq	-32(%r11),%xmm0,%xmm4+	vpmuludq	%xmm1,%xmm9,%xmm0+	vpunpckhqdq	%xmm6,%xmm5,%xmm9+	vpaddq	%xmm4,%xmm14,%xmm4+	vpaddq	%xmm0,%xmm10,%xmm0++	vpunpcklqdq	%xmm6,%xmm5,%xmm5+	vpunpcklqdq	%xmm8,%xmm7,%xmm8+++	vpsrldq	$5,%xmm9,%xmm9+	vpsrlq	$26,%xmm5,%xmm6+	vmovdqa	0(%rsp),%xmm14+	vpand	%xmm15,%xmm5,%xmm5+	vpsrlq	$4,%xmm8,%xmm7+	vpand	%xmm15,%xmm6,%xmm6+	vpand	0(%rcx),%xmm9,%xmm9+	vpsrlq	$30,%xmm8,%xmm8+	vpand	%xmm15,%xmm7,%xmm7+	vpand	%xmm15,%xmm8,%xmm8+	vpor	32(%rcx),%xmm9,%xmm9++++++	vpsrlq	$26,%xmm3,%xmm13+	vpand	%xmm15,%xmm3,%xmm3+	vpaddq	%xmm13,%xmm4,%xmm4++	vpsrlq	$26,%xmm0,%xmm10+	vpand	%xmm15,%xmm0,%xmm0+	vpaddq	%xmm10,%xmm11,%xmm1++	vpsrlq	$26,%xmm4,%xmm10+	vpand	%xmm15,%xmm4,%xmm4++	vpsrlq	$26,%xmm1,%xmm11+	vpand	%xmm15,%xmm1,%xmm1+	vpaddq	%xmm11,%xmm2,%xmm2++	vpaddq	%xmm10,%xmm0,%xmm0+	vpsllq	$2,%xmm10,%xmm10+	vpaddq	%xmm10,%xmm0,%xmm0++	vpsrlq	$26,%xmm2,%xmm12+	vpand	%xmm15,%xmm2,%xmm2+	vpaddq	%xmm12,%xmm3,%xmm3++	vpsrlq	$26,%xmm0,%xmm10+	vpand	%xmm15,%xmm0,%xmm0+	vpaddq	%xmm10,%xmm1,%xmm1++	vpsrlq	$26,%xmm3,%xmm13+	vpand	%xmm15,%xmm3,%xmm3+	vpaddq	%xmm13,%xmm4,%xmm4++	ja	L$oop_avx++L$skip_loop_avx:++++	vpshufd	$0x10,%xmm14,%xmm14+	addq	$32,%rdx+	jnz	L$ong_tail_avx++	vpaddq	%xmm2,%xmm7,%xmm7+	vpaddq	%xmm0,%xmm5,%xmm5+	vpaddq	%xmm1,%xmm6,%xmm6+	vpaddq	%xmm3,%xmm8,%xmm8+	vpaddq	%xmm4,%xmm9,%xmm9++L$ong_tail_avx:+	vmovdqa	%xmm2,32(%r11)+	vmovdqa	%xmm0,0(%r11)+	vmovdqa	%xmm1,16(%r11)+	vmovdqa	%xmm3,48(%r11)+	vmovdqa	%xmm4,64(%r11)++++++++	vpmuludq	%xmm7,%xmm14,%xmm12+	vpmuludq	%xmm5,%xmm14,%xmm10+	vpshufd	$0x10,-48(%rdi),%xmm2+	vpmuludq	%xmm6,%xmm14,%xmm11+	vpmuludq	%xmm8,%xmm14,%xmm13+	vpmuludq	%xmm9,%xmm14,%xmm14++	vpmuludq	%xmm8,%xmm2,%xmm0+	vpaddq	%xmm0,%xmm14,%xmm14+	vpshufd	$0x10,-32(%rdi),%xmm3+	vpmuludq	%xmm7,%xmm2,%xmm1+	vpaddq	%xmm1,%xmm13,%xmm13+	vpshufd	$0x10,-16(%rdi),%xmm4+	vpmuludq	%xmm6,%xmm2,%xmm0+	vpaddq	%xmm0,%xmm12,%xmm12+	vpmuludq	%xmm5,%xmm2,%xmm2+	vpaddq	%xmm2,%xmm11,%xmm11+	vpmuludq	%xmm9,%xmm3,%xmm3+	vpaddq	%xmm3,%xmm10,%xmm10++	vpshufd	$0x10,0(%rdi),%xmm2+	vpmuludq	%xmm7,%xmm4,%xmm1+	vpaddq	%xmm1,%xmm14,%xmm14+	vpmuludq	%xmm6,%xmm4,%xmm0+	vpaddq	%xmm0,%xmm13,%xmm13+	vpshufd	$0x10,16(%rdi),%xmm3+	vpmuludq	%xmm5,%xmm4,%xmm4+	vpaddq	%xmm4,%xmm12,%xmm12+	vpmuludq	%xmm9,%xmm2,%xmm1+	vpaddq	%xmm1,%xmm11,%xmm11+	vpshufd	$0x10,32(%rdi),%xmm4+	vpmuludq	%xmm8,%xmm2,%xmm2+	vpaddq	%xmm2,%xmm10,%xmm10++	vpmuludq	%xmm6,%xmm3,%xmm0+	vpaddq	%xmm0,%xmm14,%xmm14+	vpmuludq	%xmm5,%xmm3,%xmm3+	vpaddq	%xmm3,%xmm13,%xmm13+	vpshufd	$0x10,48(%rdi),%xmm2+	vpmuludq	%xmm9,%xmm4,%xmm1+	vpaddq	%xmm1,%xmm12,%xmm12+	vpshufd	$0x10,64(%rdi),%xmm3+	vpmuludq	%xmm8,%xmm4,%xmm0+	vpaddq	%xmm0,%xmm11,%xmm11+	vpmuludq	%xmm7,%xmm4,%xmm4+	vpaddq	%xmm4,%xmm10,%xmm10++	vpmuludq	%xmm5,%xmm2,%xmm2+	vpaddq	%xmm2,%xmm14,%xmm14+	vpmuludq	%xmm9,%xmm3,%xmm1+	vpaddq	%xmm1,%xmm13,%xmm13+	vpmuludq	%xmm8,%xmm3,%xmm0+	vpaddq	%xmm0,%xmm12,%xmm12+	vpmuludq	%xmm7,%xmm3,%xmm1+	vpaddq	%xmm1,%xmm11,%xmm11+	vpmuludq	%xmm6,%xmm3,%xmm3+	vpaddq	%xmm3,%xmm10,%xmm10++	jz	L$short_tail_avx++	vmovdqu	0(%rsi),%xmm0+	vmovdqu	16(%rsi),%xmm1++	vpsrldq	$6,%xmm0,%xmm2+	vpsrldq	$6,%xmm1,%xmm3+	vpunpckhqdq	%xmm1,%xmm0,%xmm4+	vpunpcklqdq	%xmm1,%xmm0,%xmm0+	vpunpcklqdq	%xmm3,%xmm2,%xmm3++	vpsrlq	$40,%xmm4,%xmm4+	vpsrlq	$26,%xmm0,%xmm1+	vpand	%xmm15,%xmm0,%xmm0+	vpsrlq	$4,%xmm3,%xmm2+	vpand	%xmm15,%xmm1,%xmm1+	vpsrlq	$30,%xmm3,%xmm3+	vpand	%xmm15,%xmm2,%xmm2+	vpand	%xmm15,%xmm3,%xmm3+	vpor	32(%rcx),%xmm4,%xmm4++	vpshufd	$0x32,-64(%rdi),%xmm9+	vpaddq	0(%r11),%xmm0,%xmm0+	vpaddq	16(%r11),%xmm1,%xmm1+	vpaddq	32(%r11),%xmm2,%xmm2+	vpaddq	48(%r11),%xmm3,%xmm3+	vpaddq	64(%r11),%xmm4,%xmm4+++++	vpmuludq	%xmm0,%xmm9,%xmm5+	vpaddq	%xmm5,%xmm10,%xmm10+	vpmuludq	%xmm1,%xmm9,%xmm6+	vpaddq	%xmm6,%xmm11,%xmm11+	vpmuludq	%xmm2,%xmm9,%xmm5+	vpaddq	%xmm5,%xmm12,%xmm12+	vpshufd	$0x32,-48(%rdi),%xmm7+	vpmuludq	%xmm3,%xmm9,%xmm6+	vpaddq	%xmm6,%xmm13,%xmm13+	vpmuludq	%xmm4,%xmm9,%xmm9+	vpaddq	%xmm9,%xmm14,%xmm14++	vpmuludq	%xmm3,%xmm7,%xmm5+	vpaddq	%xmm5,%xmm14,%xmm14+	vpshufd	$0x32,-32(%rdi),%xmm8+	vpmuludq	%xmm2,%xmm7,%xmm6+	vpaddq	%xmm6,%xmm13,%xmm13+	vpshufd	$0x32,-16(%rdi),%xmm9+	vpmuludq	%xmm1,%xmm7,%xmm5+	vpaddq	%xmm5,%xmm12,%xmm12+	vpmuludq	%xmm0,%xmm7,%xmm7+	vpaddq	%xmm7,%xmm11,%xmm11+	vpmuludq	%xmm4,%xmm8,%xmm8+	vpaddq	%xmm8,%xmm10,%xmm10++	vpshufd	$0x32,0(%rdi),%xmm7+	vpmuludq	%xmm2,%xmm9,%xmm6+	vpaddq	%xmm6,%xmm14,%xmm14+	vpmuludq	%xmm1,%xmm9,%xmm5+	vpaddq	%xmm5,%xmm13,%xmm13+	vpshufd	$0x32,16(%rdi),%xmm8+	vpmuludq	%xmm0,%xmm9,%xmm9+	vpaddq	%xmm9,%xmm12,%xmm12+	vpmuludq	%xmm4,%xmm7,%xmm6+	vpaddq	%xmm6,%xmm11,%xmm11+	vpshufd	$0x32,32(%rdi),%xmm9+	vpmuludq	%xmm3,%xmm7,%xmm7+	vpaddq	%xmm7,%xmm10,%xmm10++	vpmuludq	%xmm1,%xmm8,%xmm5+	vpaddq	%xmm5,%xmm14,%xmm14+	vpmuludq	%xmm0,%xmm8,%xmm8+	vpaddq	%xmm8,%xmm13,%xmm13+	vpshufd	$0x32,48(%rdi),%xmm7+	vpmuludq	%xmm4,%xmm9,%xmm6+	vpaddq	%xmm6,%xmm12,%xmm12+	vpshufd	$0x32,64(%rdi),%xmm8+	vpmuludq	%xmm3,%xmm9,%xmm5+	vpaddq	%xmm5,%xmm11,%xmm11+	vpmuludq	%xmm2,%xmm9,%xmm9+	vpaddq	%xmm9,%xmm10,%xmm10++	vpmuludq	%xmm0,%xmm7,%xmm7+	vpaddq	%xmm7,%xmm14,%xmm14+	vpmuludq	%xmm4,%xmm8,%xmm6+	vpaddq	%xmm6,%xmm13,%xmm13+	vpmuludq	%xmm3,%xmm8,%xmm5+	vpaddq	%xmm5,%xmm12,%xmm12+	vpmuludq	%xmm2,%xmm8,%xmm6+	vpaddq	%xmm6,%xmm11,%xmm11+	vpmuludq	%xmm1,%xmm8,%xmm8+	vpaddq	%xmm8,%xmm10,%xmm10++L$short_tail_avx:++++	vpsrldq	$8,%xmm14,%xmm9+	vpsrldq	$8,%xmm13,%xmm8+	vpsrldq	$8,%xmm11,%xmm6+	vpsrldq	$8,%xmm10,%xmm5+	vpsrldq	$8,%xmm12,%xmm7+	vpaddq	%xmm8,%xmm13,%xmm13+	vpaddq	%xmm9,%xmm14,%xmm14+	vpaddq	%xmm5,%xmm10,%xmm10+	vpaddq	%xmm6,%xmm11,%xmm11+	vpaddq	%xmm7,%xmm12,%xmm12+++++	vpsrlq	$26,%xmm13,%xmm3+	vpand	%xmm15,%xmm13,%xmm13+	vpaddq	%xmm3,%xmm14,%xmm14++	vpsrlq	$26,%xmm10,%xmm0+	vpand	%xmm15,%xmm10,%xmm10+	vpaddq	%xmm0,%xmm11,%xmm11++	vpsrlq	$26,%xmm14,%xmm4+	vpand	%xmm15,%xmm14,%xmm14++	vpsrlq	$26,%xmm11,%xmm1+	vpand	%xmm15,%xmm11,%xmm11+	vpaddq	%xmm1,%xmm12,%xmm12++	vpaddq	%xmm4,%xmm10,%xmm10+	vpsllq	$2,%xmm4,%xmm4+	vpaddq	%xmm4,%xmm10,%xmm10++	vpsrlq	$26,%xmm12,%xmm2+	vpand	%xmm15,%xmm12,%xmm12+	vpaddq	%xmm2,%xmm13,%xmm13++	vpsrlq	$26,%xmm10,%xmm0+	vpand	%xmm15,%xmm10,%xmm10+	vpaddq	%xmm0,%xmm11,%xmm11++	vpsrlq	$26,%xmm13,%xmm3+	vpand	%xmm15,%xmm13,%xmm13+	vpaddq	%xmm3,%xmm14,%xmm14++	vmovd	%xmm10,-112(%rdi)+	vmovd	%xmm11,-108(%rdi)+	vmovd	%xmm12,-104(%rdi)+	vmovd	%xmm13,-100(%rdi)+	vmovd	%xmm14,-96(%rdi)+	leaq	88(%r11),%rsp+.cfi_def_cfa	%rsp,8+	vzeroupper+	.byte	0xf3,0xc3+.cfi_endproc	+++.p2align	5+crypton_poly1305_asm_blocks_avx2:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	movl	20(%rdi),%r8d+	cmpq	$128,%rdx+	jb	L$blocks++	andq	$-16,%rdx++	vzeroupper++	testl	%r8d,%r8d+	jz	L$base2_64_avx2++	testq	$63,%rdx+	jz	L$even_avx2++	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56+	leaq	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+L$blocks_avx2_body:++	movq	%rdx,%r15++	movq	0(%rdi),%r8+	movq	8(%rdi),%r9+	movl	16(%rdi),%ebp++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13+++	movl	%r8d,%r14d+	andq	$-2147483648,%r8+	movq	%r9,%r12+	movl	%r9d,%ebx+	andq	$-2147483648,%r9++	shrq	$6,%r8+	shlq	$52,%r12+	addq	%r8,%r14+	shrq	$12,%rbx+	shrq	$18,%r9+	addq	%r12,%r14+	adcq	%r9,%rbx++	movq	%rbp,%r8+	shlq	$40,%r8+	shrq	$24,%rbp+	addq	%r8,%rbx+	adcq	$0,%rbp++	movq	%r13,%r12+	movq	%r13,%rax+	shrq	$2,%r13+	addq	%r12,%r13++L$base2_26_pre_avx2:+	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp+	subq	$16,%r15++	call	__crypton_poly1305_asm_block+	movq	%r12,%rax++	testq	$63,%r15+	jnz	L$base2_26_pre_avx2+++	movq	%r14,%rax+	movq	%r14,%rdx+	shrq	$52,%r14+	movq	%rbx,%r11+	movq	%rbx,%r12+	shrq	$26,%rdx+	andq	$0x3ffffff,%rax+	shlq	$12,%r11+	andq	$0x3ffffff,%rdx+	shrq	$14,%rbx+	orq	%r11,%r14+	shlq	$24,%rbp+	andq	$0x3ffffff,%r14+	shrq	$40,%r12+	andq	$0x3ffffff,%rbx+	orq	%r12,%rbp++	vmovd	%eax,%xmm0+	vmovd	%edx,%xmm1+	vmovd	%r14d,%xmm2+	vmovd	%ebx,%xmm3+	vmovd	%ebp,%xmm4++	movq	%r15,%rdx++	movq	8(%rsp),%r15+.cfi_restore	%r15+	movq	16(%rsp),%r14+.cfi_restore	%r14+	movq	24(%rsp),%r13+.cfi_restore	%r13+	movq	32(%rsp),%r12+.cfi_restore	%r12+	movq	40(%rsp),%rbp+.cfi_restore	%rbp+	movq	48(%rsp),%rbx+.cfi_restore	%rbx+	leaq	56(%rsp),%rax+	leaq	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+L$blocks_avx2_epilogue:+	jmp	L$do_avx2+.cfi_endproc	++.p2align	5+L$base2_64_avx2:+.cfi_startproc	+	pushq	%rbx+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbx,-16+	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-24+	pushq	%r12+.cfi_adjust_cfa_offset	8+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_adjust_cfa_offset	8+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_adjust_cfa_offset	8+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_adjust_cfa_offset	8+.cfi_offset	%r15,-56+	leaq	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+L$base2_64_avx2_body:++	movq	%rdx,%r15++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13++	movq	0(%rdi),%r14+	movq	8(%rdi),%rbx+	movl	16(%rdi),%ebp++	movq	%r13,%r12+	movq	%r13,%rax+	shrq	$2,%r13+	addq	%r12,%r13++	testq	$63,%rdx+	jz	L$init_avx2++L$base2_64_pre_avx2:+	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp+	subq	$16,%r15++	call	__crypton_poly1305_asm_block+	movq	%r12,%rax++	testq	$63,%r15+	jnz	L$base2_64_pre_avx2++L$init_avx2:++	movq	%r14,%rax+	movq	%r14,%rdx+	shrq	$52,%r14+	movq	%rbx,%r8+	movq	%rbx,%r9+	shrq	$26,%rdx+	andq	$0x3ffffff,%rax+	shlq	$12,%r8+	andq	$0x3ffffff,%rdx+	shrq	$14,%rbx+	orq	%r8,%r14+	shlq	$24,%rbp+	andq	$0x3ffffff,%r14+	shrq	$40,%r9+	andq	$0x3ffffff,%rbx+	orq	%r9,%rbp++	vmovd	%eax,%xmm0+	vmovd	%edx,%xmm1+	vmovd	%r14d,%xmm2+	vmovd	%ebx,%xmm3+	vmovd	%ebp,%xmm4+	movl	$1,20(%rdi)++	call	__crypton_poly1305_asm_init_avx++	movq	%r15,%rdx++	movq	8(%rsp),%r15+.cfi_restore	%r15+	movq	16(%rsp),%r14+.cfi_restore	%r14+	movq	24(%rsp),%r13+.cfi_restore	%r13+	movq	32(%rsp),%r12+.cfi_restore	%r12+	movq	40(%rsp),%rbp+.cfi_restore	%rbp+	movq	48(%rsp),%rbx+.cfi_restore	%rbx+	leaq	56(%rsp),%rax+	leaq	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+L$base2_64_avx2_epilogue:+	jmp	L$do_avx2+.cfi_endproc	++.p2align	5+L$even_avx2:+.cfi_startproc	+	vmovd	0(%rdi),%xmm0+	vmovd	4(%rdi),%xmm1+	vmovd	8(%rdi),%xmm2+	vmovd	12(%rdi),%xmm3+	vmovd	16(%rdi),%xmm4++L$do_avx2:+	leaq	-8(%rsp),%r11+.cfi_def_cfa	%r11,16+	subq	$0x128,%rsp+	leaq	L$const(%rip),%rcx+	leaq	48+64(%rdi),%rdi+	vmovdqa	96(%rcx),%ymm7+++	vmovdqu	-64(%rdi),%xmm9+	andq	$-512,%rsp+	vmovdqu	-48(%rdi),%xmm10+	vmovdqu	-32(%rdi),%xmm6+	vmovdqu	-16(%rdi),%xmm11+	vmovdqu	0(%rdi),%xmm12+	vmovdqu	16(%rdi),%xmm13+	leaq	144(%rsp),%rax+	vmovdqu	32(%rdi),%xmm14+	vpermd	%ymm9,%ymm7,%ymm9+	vmovdqu	48(%rdi),%xmm15+	vpermd	%ymm10,%ymm7,%ymm10+	vmovdqu	64(%rdi),%xmm5+	vpermd	%ymm6,%ymm7,%ymm6+	vmovdqa	%ymm9,0(%rsp)+	vpermd	%ymm11,%ymm7,%ymm11+	vmovdqa	%ymm10,32-144(%rax)+	vpermd	%ymm12,%ymm7,%ymm12+	vmovdqa	%ymm6,64-144(%rax)+	vpermd	%ymm13,%ymm7,%ymm13+	vmovdqa	%ymm11,96-144(%rax)+	vpermd	%ymm14,%ymm7,%ymm14+	vmovdqa	%ymm12,128-144(%rax)+	vpermd	%ymm15,%ymm7,%ymm15+	vmovdqa	%ymm13,160-144(%rax)+	vpermd	%ymm5,%ymm7,%ymm5+	vmovdqa	%ymm14,192-144(%rax)+	vmovdqa	%ymm15,224-144(%rax)+	vmovdqa	%ymm5,256-144(%rax)+	vmovdqa	64(%rcx),%ymm5++++	vmovdqu	0(%rsi),%xmm7+	vmovdqu	16(%rsi),%xmm8+	vinserti128	$1,32(%rsi),%ymm7,%ymm7+	vinserti128	$1,48(%rsi),%ymm8,%ymm8+	leaq	64(%rsi),%rsi++	vpsrldq	$6,%ymm7,%ymm9+	vpsrldq	$6,%ymm8,%ymm10+	vpunpckhqdq	%ymm8,%ymm7,%ymm6+	vpunpcklqdq	%ymm10,%ymm9,%ymm9+	vpunpcklqdq	%ymm8,%ymm7,%ymm7++	vpsrlq	$30,%ymm9,%ymm10+	vpsrlq	$4,%ymm9,%ymm9+	vpsrlq	$26,%ymm7,%ymm8+	vpsrlq	$40,%ymm6,%ymm6+	vpand	%ymm5,%ymm9,%ymm9+	vpand	%ymm5,%ymm7,%ymm7+	vpand	%ymm5,%ymm8,%ymm8+	vpand	%ymm5,%ymm10,%ymm10+	vpor	32(%rcx),%ymm6,%ymm6++	vpaddq	%ymm2,%ymm9,%ymm2+	subq	$64,%rdx+	jz	L$tail_avx2+	jmp	L$oop_avx2++.p2align	5+L$oop_avx2:+++++++++	vpaddq	%ymm0,%ymm7,%ymm0+	vmovdqa	0(%rsp),%ymm7+	vpaddq	%ymm1,%ymm8,%ymm1+	vmovdqa	32(%rsp),%ymm8+	vpaddq	%ymm3,%ymm10,%ymm3+	vmovdqa	96(%rsp),%ymm9+	vpaddq	%ymm4,%ymm6,%ymm4+	vmovdqa	48(%rax),%ymm10+	vmovdqa	112(%rax),%ymm5+++++++++++++++++	vpmuludq	%ymm2,%ymm7,%ymm13+	vpmuludq	%ymm2,%ymm8,%ymm14+	vpmuludq	%ymm2,%ymm9,%ymm15+	vpmuludq	%ymm2,%ymm10,%ymm11+	vpmuludq	%ymm2,%ymm5,%ymm12++	vpmuludq	%ymm0,%ymm8,%ymm6+	vpmuludq	%ymm1,%ymm8,%ymm2+	vpaddq	%ymm6,%ymm12,%ymm12+	vpaddq	%ymm2,%ymm13,%ymm13+	vpmuludq	%ymm3,%ymm8,%ymm6+	vpmuludq	64(%rsp),%ymm4,%ymm2+	vpaddq	%ymm6,%ymm15,%ymm15+	vpaddq	%ymm2,%ymm11,%ymm11+	vmovdqa	-16(%rax),%ymm8++	vpmuludq	%ymm0,%ymm7,%ymm6+	vpmuludq	%ymm1,%ymm7,%ymm2+	vpaddq	%ymm6,%ymm11,%ymm11+	vpaddq	%ymm2,%ymm12,%ymm12+	vpmuludq	%ymm3,%ymm7,%ymm6+	vpmuludq	%ymm4,%ymm7,%ymm2+	vmovdqu	0(%rsi),%xmm7+	vpaddq	%ymm6,%ymm14,%ymm14+	vpaddq	%ymm2,%ymm15,%ymm15+	vinserti128	$1,32(%rsi),%ymm7,%ymm7++	vpmuludq	%ymm3,%ymm8,%ymm6+	vpmuludq	%ymm4,%ymm8,%ymm2+	vmovdqu	16(%rsi),%xmm8+	vpaddq	%ymm6,%ymm11,%ymm11+	vpaddq	%ymm2,%ymm12,%ymm12+	vmovdqa	16(%rax),%ymm2+	vpmuludq	%ymm1,%ymm9,%ymm6+	vpmuludq	%ymm0,%ymm9,%ymm9+	vpaddq	%ymm6,%ymm14,%ymm14+	vpaddq	%ymm9,%ymm13,%ymm13+	vinserti128	$1,48(%rsi),%ymm8,%ymm8+	leaq	64(%rsi),%rsi++	vpmuludq	%ymm1,%ymm2,%ymm6+	vpmuludq	%ymm0,%ymm2,%ymm2+	vpsrldq	$6,%ymm7,%ymm9+	vpaddq	%ymm6,%ymm15,%ymm15+	vpaddq	%ymm2,%ymm14,%ymm14+	vpmuludq	%ymm3,%ymm10,%ymm6+	vpmuludq	%ymm4,%ymm10,%ymm2+	vpsrldq	$6,%ymm8,%ymm10+	vpaddq	%ymm6,%ymm12,%ymm12+	vpaddq	%ymm2,%ymm13,%ymm13+	vpunpckhqdq	%ymm8,%ymm7,%ymm6++	vpmuludq	%ymm3,%ymm5,%ymm3+	vpmuludq	%ymm4,%ymm5,%ymm4+	vpunpcklqdq	%ymm8,%ymm7,%ymm7+	vpaddq	%ymm3,%ymm13,%ymm2+	vpaddq	%ymm4,%ymm14,%ymm3+	vpunpcklqdq	%ymm10,%ymm9,%ymm10+	vpmuludq	80(%rax),%ymm0,%ymm4+	vpmuludq	%ymm1,%ymm5,%ymm0+	vmovdqa	64(%rcx),%ymm5+	vpaddq	%ymm4,%ymm15,%ymm4+	vpaddq	%ymm0,%ymm11,%ymm0+++++	vpsrlq	$26,%ymm3,%ymm14+	vpand	%ymm5,%ymm3,%ymm3+	vpaddq	%ymm14,%ymm4,%ymm4++	vpsrlq	$26,%ymm0,%ymm11+	vpand	%ymm5,%ymm0,%ymm0+	vpaddq	%ymm11,%ymm12,%ymm1++	vpsrlq	$26,%ymm4,%ymm15+	vpand	%ymm5,%ymm4,%ymm4++	vpsrlq	$4,%ymm10,%ymm9++	vpsrlq	$26,%ymm1,%ymm12+	vpand	%ymm5,%ymm1,%ymm1+	vpaddq	%ymm12,%ymm2,%ymm2++	vpaddq	%ymm15,%ymm0,%ymm0+	vpsllq	$2,%ymm15,%ymm15+	vpaddq	%ymm15,%ymm0,%ymm0++	vpand	%ymm5,%ymm9,%ymm9+	vpsrlq	$26,%ymm7,%ymm8++	vpsrlq	$26,%ymm2,%ymm13+	vpand	%ymm5,%ymm2,%ymm2+	vpaddq	%ymm13,%ymm3,%ymm3++	vpaddq	%ymm9,%ymm2,%ymm2+	vpsrlq	$30,%ymm10,%ymm10++	vpsrlq	$26,%ymm0,%ymm11+	vpand	%ymm5,%ymm0,%ymm0+	vpaddq	%ymm11,%ymm1,%ymm1++	vpsrlq	$40,%ymm6,%ymm6++	vpsrlq	$26,%ymm3,%ymm14+	vpand	%ymm5,%ymm3,%ymm3+	vpaddq	%ymm14,%ymm4,%ymm4++	vpand	%ymm5,%ymm7,%ymm7+	vpand	%ymm5,%ymm8,%ymm8+	vpand	%ymm5,%ymm10,%ymm10+	vpor	32(%rcx),%ymm6,%ymm6++	subq	$64,%rdx+	jnz	L$oop_avx2++.byte	0x66,0x90+L$tail_avx2:++++++++	vpaddq	%ymm0,%ymm7,%ymm0+	vmovdqu	4(%rsp),%ymm7+	vpaddq	%ymm1,%ymm8,%ymm1+	vmovdqu	36(%rsp),%ymm8+	vpaddq	%ymm3,%ymm10,%ymm3+	vmovdqu	100(%rsp),%ymm9+	vpaddq	%ymm4,%ymm6,%ymm4+	vmovdqu	52(%rax),%ymm10+	vmovdqu	116(%rax),%ymm5++	vpmuludq	%ymm2,%ymm7,%ymm13+	vpmuludq	%ymm2,%ymm8,%ymm14+	vpmuludq	%ymm2,%ymm9,%ymm15+	vpmuludq	%ymm2,%ymm10,%ymm11+	vpmuludq	%ymm2,%ymm5,%ymm12++	vpmuludq	%ymm0,%ymm8,%ymm6+	vpmuludq	%ymm1,%ymm8,%ymm2+	vpaddq	%ymm6,%ymm12,%ymm12+	vpaddq	%ymm2,%ymm13,%ymm13+	vpmuludq	%ymm3,%ymm8,%ymm6+	vpmuludq	68(%rsp),%ymm4,%ymm2+	vpaddq	%ymm6,%ymm15,%ymm15+	vpaddq	%ymm2,%ymm11,%ymm11++	vpmuludq	%ymm0,%ymm7,%ymm6+	vpmuludq	%ymm1,%ymm7,%ymm2+	vpaddq	%ymm6,%ymm11,%ymm11+	vmovdqu	-12(%rax),%ymm8+	vpaddq	%ymm2,%ymm12,%ymm12+	vpmuludq	%ymm3,%ymm7,%ymm6+	vpmuludq	%ymm4,%ymm7,%ymm2+	vpaddq	%ymm6,%ymm14,%ymm14+	vpaddq	%ymm2,%ymm15,%ymm15++	vpmuludq	%ymm3,%ymm8,%ymm6+	vpmuludq	%ymm4,%ymm8,%ymm2+	vpaddq	%ymm6,%ymm11,%ymm11+	vpaddq	%ymm2,%ymm12,%ymm12+	vmovdqu	20(%rax),%ymm2+	vpmuludq	%ymm1,%ymm9,%ymm6+	vpmuludq	%ymm0,%ymm9,%ymm9+	vpaddq	%ymm6,%ymm14,%ymm14+	vpaddq	%ymm9,%ymm13,%ymm13++	vpmuludq	%ymm1,%ymm2,%ymm6+	vpmuludq	%ymm0,%ymm2,%ymm2+	vpaddq	%ymm6,%ymm15,%ymm15+	vpaddq	%ymm2,%ymm14,%ymm14+	vpmuludq	%ymm3,%ymm10,%ymm6+	vpmuludq	%ymm4,%ymm10,%ymm2+	vpaddq	%ymm6,%ymm12,%ymm12+	vpaddq	%ymm2,%ymm13,%ymm13++	vpmuludq	%ymm3,%ymm5,%ymm3+	vpmuludq	%ymm4,%ymm5,%ymm4+	vpaddq	%ymm3,%ymm13,%ymm2+	vpaddq	%ymm4,%ymm14,%ymm3+	vpmuludq	84(%rax),%ymm0,%ymm4+	vpmuludq	%ymm1,%ymm5,%ymm0+	vmovdqa	64(%rcx),%ymm5+	vpaddq	%ymm4,%ymm15,%ymm4+	vpaddq	%ymm0,%ymm11,%ymm0+++++	vpsrldq	$8,%ymm12,%ymm8+	vpsrldq	$8,%ymm2,%ymm9+	vpsrldq	$8,%ymm3,%ymm10+	vpsrldq	$8,%ymm4,%ymm6+	vpsrldq	$8,%ymm0,%ymm7+	vpaddq	%ymm8,%ymm12,%ymm12+	vpaddq	%ymm9,%ymm2,%ymm2+	vpaddq	%ymm10,%ymm3,%ymm3+	vpaddq	%ymm6,%ymm4,%ymm4+	vpaddq	%ymm7,%ymm0,%ymm0++	vpermq	$0x2,%ymm3,%ymm10+	vpermq	$0x2,%ymm4,%ymm6+	vpermq	$0x2,%ymm0,%ymm7+	vpermq	$0x2,%ymm12,%ymm8+	vpermq	$0x2,%ymm2,%ymm9+	vpaddq	%ymm10,%ymm3,%ymm3+	vpaddq	%ymm6,%ymm4,%ymm4+	vpaddq	%ymm7,%ymm0,%ymm0+	vpaddq	%ymm8,%ymm12,%ymm12+	vpaddq	%ymm9,%ymm2,%ymm2+++++	vpsrlq	$26,%ymm3,%ymm14+	vpand	%ymm5,%ymm3,%ymm3+	vpaddq	%ymm14,%ymm4,%ymm4++	vpsrlq	$26,%ymm0,%ymm11+	vpand	%ymm5,%ymm0,%ymm0+	vpaddq	%ymm11,%ymm12,%ymm1++	vpsrlq	$26,%ymm4,%ymm15+	vpand	%ymm5,%ymm4,%ymm4++	vpsrlq	$26,%ymm1,%ymm12+	vpand	%ymm5,%ymm1,%ymm1+	vpaddq	%ymm12,%ymm2,%ymm2++	vpaddq	%ymm15,%ymm0,%ymm0+	vpsllq	$2,%ymm15,%ymm15+	vpaddq	%ymm15,%ymm0,%ymm0++	vpsrlq	$26,%ymm2,%ymm13+	vpand	%ymm5,%ymm2,%ymm2+	vpaddq	%ymm13,%ymm3,%ymm3++	vpsrlq	$26,%ymm0,%ymm11+	vpand	%ymm5,%ymm0,%ymm0+	vpaddq	%ymm11,%ymm1,%ymm1++	vpsrlq	$26,%ymm3,%ymm14+	vpand	%ymm5,%ymm3,%ymm3+	vpaddq	%ymm14,%ymm4,%ymm4++	vmovd	%xmm0,-112(%rdi)+	vmovd	%xmm1,-108(%rdi)+	vmovd	%xmm2,-104(%rdi)+	vmovd	%xmm3,-100(%rdi)+	vmovd	%xmm4,-96(%rdi)+	leaq	8(%r11),%rsp+.cfi_def_cfa	%rsp,8+	vzeroupper+	.byte	0xf3,0xc3+.cfi_endproc	++.p2align	6+L$const:+L$mask24:+.long	0x0ffffff,0,0x0ffffff,0,0x0ffffff,0,0x0ffffff,0+L$129:+.long	16777216,0,16777216,0,16777216,0,16777216,0+L$mask26:+.long	0x3ffffff,0,0x3ffffff,0,0x3ffffff,0,0x3ffffff,0+L$permd_avx2:+.long	2,2,2,3,2,0,2,1+L$permd_avx512:+.long	0,0,0,1, 0,2,0,3, 0,4,0,5, 0,6,0,7++L$2_44_inp_permd:+.long	0,1,1,2,2,3,7,7+L$2_44_inp_shift:+.quad	0,12,24,64+L$2_44_mask:+.quad	0xfffffffffff,0xfffffffffff,0x3ffffffffff,0xffffffffffffffff+L$2_44_shift_rgt:+.quad	44,44,42,64+L$2_44_shift_lft:+.quad	8,8,10,64++.p2align	6+L$x_mask44:+.quad	0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff+.quad	0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff+L$x_mask42:+.quad	0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff+.quad	0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff+.byte	80,111,108,121,49,51,48,53,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.p2align	4+.globl	_crypton_xor128_encrypt_n_pad++.p2align	4+_crypton_xor128_encrypt_n_pad:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa++	subq	%rdx,%rsi+	subq	%rdx,%rdi+	movq	%rcx,%r10+	shrq	$4,%rcx+	jz	L$tail_enc+	nop+L$oop_enc_xmm:+	movdqu	(%rsi,%rdx,1),%xmm0+	pxor	(%rdx),%xmm0+	movdqu	%xmm0,(%rdi,%rdx,1)+	movdqa	%xmm0,(%rdx)+	leaq	16(%rdx),%rdx+	decq	%rcx+	jnz	L$oop_enc_xmm++	andq	$15,%r10+	jz	L$done_enc++L$tail_enc:+	movq	$16,%rcx+	subq	%r10,%rcx+	xorl	%eax,%eax+L$oop_enc_byte:+	movb	(%rsi,%rdx,1),%al+	xorb	(%rdx),%al+	movb	%al,(%rdi,%rdx,1)+	movb	%al,(%rdx)+	leaq	1(%rdx),%rdx+	decq	%r10+	jnz	L$oop_enc_byte++	xorl	%eax,%eax+L$oop_enc_pad:+	movb	%al,(%rdx)+	leaq	1(%rdx),%rdx+	decq	%rcx+	jnz	L$oop_enc_pad++L$done_enc:+	movq	%rdx,%rax+	.byte	0xf3,0xc3+.cfi_endproc+++.globl	_crypton_xor128_decrypt_n_pad++.p2align	4+_crypton_xor128_decrypt_n_pad:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa++	subq	%rdx,%rsi+	subq	%rdx,%rdi+	movq	%rcx,%r10+	shrq	$4,%rcx+	jz	L$tail_dec+	nop+L$oop_dec_xmm:+	movdqu	(%rsi,%rdx,1),%xmm0+	movdqa	(%rdx),%xmm1+	pxor	%xmm0,%xmm1+	movdqu	%xmm1,(%rdi,%rdx,1)+	movdqa	%xmm0,(%rdx)+	leaq	16(%rdx),%rdx+	decq	%rcx+	jnz	L$oop_dec_xmm++	pxor	%xmm1,%xmm1+	andq	$15,%r10+	jz	L$done_dec++L$tail_dec:+	movq	$16,%rcx+	subq	%r10,%rcx+	xorl	%eax,%eax+	xorq	%r11,%r11+L$oop_dec_byte:+	movb	(%rsi,%rdx,1),%r11b+	movb	(%rdx),%al+	xorb	%r11b,%al+	movb	%al,(%rdi,%rdx,1)+	movb	%r11b,(%rdx)+	leaq	1(%rdx),%rdx+	decq	%r10+	jnz	L$oop_dec_byte++	xorl	%eax,%eax+L$oop_dec_pad:+	movb	%al,(%rdx)+	leaq	1(%rdx),%rdx+	decq	%rcx+	jnz	L$oop_dec_pad++L$done_dec:+	movq	%rdx,%rax+	.byte	0xf3,0xc3+.cfi_endproc+
+ cbits/asm/poly1305-x86_64-mingw64.S view
@@ -0,0 +1,2281 @@+.text	++++.globl	crypton_poly1305_asm_init++.globl	crypton_poly1305_asm_blocks++.globl	crypton_poly1305_asm_emit+++.def	crypton_poly1305_asm_init;	.scl 2;	.type 32;	.endef+.p2align	5+crypton_poly1305_asm_init:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%rax+.LSEH_begin_crypton_poly1305_asm_init:++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	xorq	%rax,%rax+	movq	%rax,0(%rdi)+	movq	%rax,8(%rdi)+	movq	%rax,16(%rdi)++	cmpq	$0,%rsi+	je	.Lno_key++	movq	$0x0ffffffc0fffffff,%rax+	leaq	-3(%rax),%rcx+	andq	0(%rsi),%rax+	andq	8(%rsi),%rcx+	movq	%rax,24(%rdi)+	movq	%rcx,32(%rdi)+	movl	$-1,48(%rdi)+	leaq	crypton_poly1305_asm_blocks(%rip),%r10+	leaq	crypton_poly1305_asm_emit(%rip),%r11+	movq	crypton_ia32cap_P+4(%rip),%r9+	leaq	crypton_poly1305_asm_blocks_avx(%rip),%rax+	btq	$28,%r9+	cmovcq	%rax,%r10+	leaq	crypton_poly1305_asm_blocks_avx2(%rip),%rax+	btq	$37,%r9+	cmovcq	%rax,%r10+	movq	%r10,0(%rdx)+	movq	%r11,8(%rdx)+	movl	$1,%eax+.Lno_key:+	movq	8(%rsp),%rdi+	movq	16(%rsp),%rsi+	.byte	0xf3,0xc3+.LSEH_end_crypton_poly1305_asm_init:++.def	crypton_poly1305_asm_blocks;	.scl 2;	.type 32;	.endef+.p2align	5+crypton_poly1305_asm_blocks:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%rax+.LSEH_begin_crypton_poly1305_asm_blocks:+++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	movq	%r9,%rcx+.Lblocks:+	shrq	$4,%rdx+	jz	.Lno_data++	pushq	%rbx++	pushq	%rbp++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	leaq	-8(%rsp),%rsp++.Lblocks_body:++	movq	%rdx,%r15++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13++	movq	0(%rdi),%r14+	movq	8(%rdi),%rbx+	movq	16(%rdi),%rbp++	movl	%r14d,%eax+	movl	4(%rdi),%edx+	movl	%ebx,%r8d+	movl	12(%rdi),%r10d+	movl	%ebp,%r12d++	shlq	$26,%rdx+	movq	%r8,%r9+	shlq	$52,%r8+	addq	%rdx,%rax+	shrq	$12,%r9+	addq	%rax,%r8+	adcq	$0,%r9++	shlq	$14,%r10+	movq	%r12,%rax+	shrq	$24,%r12+	addq	%r10,%r9+	shlq	$40,%rax+	addq	%rax,%r9+	adcq	$0,%r12++	cmpq	$4,%rbp++	cmovaq	%r8,%r14+	cmovaq	%r9,%rbx+	cmovaq	%r12,%rbp++	movq	%r13,%r12+	shrq	$2,%r13+	movq	%r12,%rax+	addq	%r12,%r13+	jmp	.Loop++.p2align	5+.Loop:+	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp+	mulq	%r14+	movq	%rax,%r9+	movq	%r11,%rax+	movq	%rdx,%r10++	mulq	%r14+	movq	%rax,%r14+	movq	%r11,%rax+	movq	%rdx,%r8++	mulq	%rbx+	addq	%rax,%r9+	movq	%r13,%rax+	adcq	%rdx,%r10++	mulq	%rbx+	movq	%rbp,%rbx+	addq	%rax,%r14+	adcq	%rdx,%r8++	imulq	%r13,%rbx+	addq	%rbx,%r9+	movq	%r8,%rbx+	adcq	$0,%r10++	imulq	%r11,%rbp+	addq	%r9,%rbx+	movq	$-4,%rax+	adcq	%rbp,%r10++	andq	%r10,%rax+	movq	%r10,%rbp+	shrq	$2,%r10+	andq	$3,%rbp+	addq	%r10,%rax+	addq	%rax,%r14+	adcq	$0,%rbx+	adcq	$0,%rbp+	movq	%r12,%rax+	decq	%r15+	jnz	.Loop++	movq	%r14,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rbp,16(%rdi)++	movq	8(%rsp),%r15++	movq	16(%rsp),%r14++	movq	24(%rsp),%r13++	movq	32(%rsp),%r12++	movq	40(%rsp),%rbp++	movq	48(%rsp),%rbx++	leaq	56(%rsp),%rsp++.Lno_data:+.Lblocks_epilogue:+	movq	8(%rsp),%rdi+	movq	16(%rsp),%rsi+	.byte	0xf3,0xc3++.LSEH_end_crypton_poly1305_asm_blocks:++.def	crypton_poly1305_asm_emit;	.scl 2;	.type 32;	.endef+.p2align	5+crypton_poly1305_asm_emit:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%rax+.LSEH_begin_crypton_poly1305_asm_emit:++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	movl	0(%rdi),%eax+	movl	4(%rdi),%ecx+	movl	8(%rdi),%r8d+	movl	12(%rdi),%r11d+	movl	16(%rdi),%r10d++	shlq	$26,%rcx+	movq	%r8,%r9+	shlq	$52,%r8+	addq	%rcx,%rax+	shrq	$12,%r9+	addq	%rax,%r8+	adcq	$0,%r9++	shlq	$14,%r11+	movq	%r10,%rax+	shrq	$24,%r10+	addq	%r11,%r9+	movq	0(%rdi),%rcx+	shlq	$40,%rax+	movq	8(%rdi),%r11+	addq	%rax,%r9+	movq	16(%rdi),%rax+	adcq	$0,%r10++	cmpq	$4,%rax++	cmovbeq	%rcx,%r8+	cmovbeq	%r11,%r9+	cmovbeq	%rax,%r10++	movq	%r8,%rax+	addq	$5,%r8+	movq	%r9,%rcx+	adcq	$0,%r9+	adcq	$0,%r10+	shrq	$2,%r10+	cmovnzq	%r8,%rax+	cmovnzq	%r9,%rcx++	addq	0(%rdx),%rax+	adcq	8(%rdx),%rcx+	movq	%rax,0(%rsi)+	movq	%rcx,8(%rsi)++	movq	8(%rsp),%rdi+	movq	16(%rsp),%rsi+	.byte	0xf3,0xc3+.LSEH_end_crypton_poly1305_asm_emit:+.def	__crypton_poly1305_asm_block;	.scl 3;	.type 32;	.endef+.p2align	5+__crypton_poly1305_asm_block:+	.byte	0xf3,0x0f,0x1e,0xfa++	mulq	%r14+	movq	%rax,%r9+	movq	%r11,%rax+	movq	%rdx,%r10++	mulq	%r14+	movq	%rax,%r14+	movq	%r11,%rax+	movq	%rdx,%r8++	mulq	%rbx+	addq	%rax,%r9+	movq	%r13,%rax+	adcq	%rdx,%r10++	mulq	%rbx+	movq	%rbp,%rbx+	addq	%rax,%r14+	adcq	%rdx,%r8++	imulq	%r13,%rbx+	addq	%rbx,%r9+	movq	%r8,%rbx+	adcq	$0,%r10++	imulq	%r11,%rbp+	addq	%r9,%rbx+	movq	$-4,%rax+	adcq	%rbp,%r10++	andq	%r10,%rax+	movq	%r10,%rbp+	shrq	$2,%r10+	andq	$3,%rbp+	addq	%r10,%rax+	addq	%rax,%r14+	adcq	$0,%rbx+	adcq	$0,%rbp+	.byte	0xf3,0xc3+++.def	__crypton_poly1305_asm_init_avx;	.scl 3;	.type 32;	.endef+.p2align	5+__crypton_poly1305_asm_init_avx:+	.byte	0xf3,0x0f,0x1e,0xfa++	cmpl	$-1,48(%rdi)+	jne	.Ldone_init_avx++	movq	%r11,%r14+	movq	%r12,%rbx+	xorq	%rbp,%rbp++	leaq	48+64(%rdi),%rdi++	movq	%r12,%rax+	call	__crypton_poly1305_asm_block++	movl	$0x3ffffff,%eax+	movl	$0x3ffffff,%edx+	movq	%r14,%r8+	andl	%r14d,%eax+	movq	%r11,%r9+	andl	%r11d,%edx+	movl	%eax,-64(%rdi)+	shrq	$26,%r8+	movl	%edx,-60(%rdi)+	shrq	$26,%r9++	movl	$0x3ffffff,%eax+	movl	$0x3ffffff,%edx+	andl	%r8d,%eax+	andl	%r9d,%edx+	movl	%eax,-48(%rdi)+	leal	(%rax,%rax,4),%eax+	movl	%edx,-44(%rdi)+	leal	(%rdx,%rdx,4),%edx+	movl	%eax,-32(%rdi)+	shrq	$26,%r8+	movl	%edx,-28(%rdi)+	shrq	$26,%r9++	movq	%rbx,%rax+	movq	%r12,%rdx+	shlq	$12,%rax+	shlq	$12,%rdx+	orq	%r8,%rax+	orq	%r9,%rdx+	andl	$0x3ffffff,%eax+	andl	$0x3ffffff,%edx+	movl	%eax,-16(%rdi)+	leal	(%rax,%rax,4),%eax+	movl	%edx,-12(%rdi)+	leal	(%rdx,%rdx,4),%edx+	movl	%eax,0(%rdi)+	movq	%rbx,%r8+	movl	%edx,4(%rdi)+	movq	%r12,%r9++	movl	$0x3ffffff,%eax+	movl	$0x3ffffff,%edx+	shrq	$14,%r8+	shrq	$14,%r9+	andl	%r8d,%eax+	andl	%r9d,%edx+	movl	%eax,16(%rdi)+	leal	(%rax,%rax,4),%eax+	movl	%edx,20(%rdi)+	leal	(%rdx,%rdx,4),%edx+	movl	%eax,32(%rdi)+	shrq	$26,%r8+	movl	%edx,36(%rdi)+	shrq	$26,%r9++	movq	%rbp,%rax+	shlq	$24,%rax+	orq	%rax,%r8+	movl	%r8d,48(%rdi)+	leaq	(%r8,%r8,4),%r8+	movl	%r9d,52(%rdi)+	leaq	(%r9,%r9,4),%r9+	movl	%r8d,64(%rdi)+	movl	%r9d,68(%rdi)++	movq	%r12,%rax+	call	__crypton_poly1305_asm_block++	movl	$0x3ffffff,%eax+	movq	%r14,%r8+	andl	%r14d,%eax+	shrq	$26,%r8+	movl	%eax,-52(%rdi)++	movl	$0x3ffffff,%edx+	andl	%r8d,%edx+	movl	%edx,-36(%rdi)+	leal	(%rdx,%rdx,4),%edx+	shrq	$26,%r8+	movl	%edx,-20(%rdi)++	movq	%rbx,%rax+	shlq	$12,%rax+	orq	%r8,%rax+	andl	$0x3ffffff,%eax+	movl	%eax,-4(%rdi)+	leal	(%rax,%rax,4),%eax+	movq	%rbx,%r8+	movl	%eax,12(%rdi)++	movl	$0x3ffffff,%edx+	shrq	$14,%r8+	andl	%r8d,%edx+	movl	%edx,28(%rdi)+	leal	(%rdx,%rdx,4),%edx+	shrq	$26,%r8+	movl	%edx,44(%rdi)++	movq	%rbp,%rax+	shlq	$24,%rax+	orq	%rax,%r8+	movl	%r8d,60(%rdi)+	leaq	(%r8,%r8,4),%r8+	movl	%r8d,76(%rdi)++	movq	%r12,%rax+	call	__crypton_poly1305_asm_block++	movl	$0x3ffffff,%eax+	movq	%r14,%r8+	andl	%r14d,%eax+	shrq	$26,%r8+	movl	%eax,-56(%rdi)++	movl	$0x3ffffff,%edx+	andl	%r8d,%edx+	movl	%edx,-40(%rdi)+	leal	(%rdx,%rdx,4),%edx+	shrq	$26,%r8+	movl	%edx,-24(%rdi)++	movq	%rbx,%rax+	shlq	$12,%rax+	orq	%r8,%rax+	andl	$0x3ffffff,%eax+	movl	%eax,-8(%rdi)+	leal	(%rax,%rax,4),%eax+	movq	%rbx,%r8+	movl	%eax,8(%rdi)++	movl	$0x3ffffff,%edx+	shrq	$14,%r8+	andl	%r8d,%edx+	movl	%edx,24(%rdi)+	leal	(%rdx,%rdx,4),%edx+	shrq	$26,%r8+	movl	%edx,40(%rdi)++	movq	%rbp,%rax+	shlq	$24,%rax+	orq	%rax,%r8+	movl	%r8d,56(%rdi)+	leaq	(%r8,%r8,4),%r8+	movl	%r8d,72(%rdi)++	leaq	-48-64(%rdi),%rdi+.Ldone_init_avx:+	.byte	0xf3,0xc3+++.def	crypton_poly1305_asm_blocks_avx;	.scl 3;	.type 32;	.endef+.p2align	5+crypton_poly1305_asm_blocks_avx:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%rax+.LSEH_begin_crypton_poly1305_asm_blocks_avx:+++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	movq	%r9,%rcx+	movl	20(%rdi),%r8d+	cmpq	$128,%rdx+	jb	.Lblocks++	andq	$-16,%rdx++	vzeroupper++	testl	%r8d,%r8d+	jz	.Lbase2_64_avx++	testq	$31,%rdx+	jz	.Leven_avx++	pushq	%rbx++	pushq	%rbp++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	leaq	-8(%rsp),%rsp++.Lblocks_avx_body:++	movq	%rdx,%r15++	movq	0(%rdi),%r8+	movq	8(%rdi),%r9+	movl	16(%rdi),%ebp++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13+++	movl	%r8d,%r14d+	andq	$-2147483648,%r8+	movq	%r9,%r12+	movl	%r9d,%ebx+	andq	$-2147483648,%r9++	shrq	$6,%r8+	shlq	$52,%r12+	addq	%r8,%r14+	shrq	$12,%rbx+	shrq	$18,%r9+	addq	%r12,%r14+	adcq	%r9,%rbx++	movq	%rbp,%r8+	shlq	$40,%r8+	shrq	$24,%rbp+	addq	%r8,%rbx+	adcq	$0,%rbp++	movq	%r13,%r12+	movq	%r13,%rax+	shrq	$2,%r13+	addq	%r12,%r13++	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp++	call	__crypton_poly1305_asm_block+++	movq	%r14,%rax+	movq	%r14,%rdx+	shrq	$52,%r14+	movq	%rbx,%r11+	movq	%rbx,%r12+	shrq	$26,%rdx+	andq	$0x3ffffff,%rax+	shlq	$12,%r11+	andq	$0x3ffffff,%rdx+	shrq	$14,%rbx+	orq	%r11,%r14+	shlq	$24,%rbp+	andq	$0x3ffffff,%r14+	shrq	$40,%r12+	andq	$0x3ffffff,%rbx+	orq	%r12,%rbp++	vmovd	%eax,%xmm0+	vmovd	%edx,%xmm1+	vmovd	%r14d,%xmm2+	vmovd	%ebx,%xmm3+	vmovd	%ebp,%xmm4++	leaq	-16(%r15),%rdx++	movq	8(%rsp),%r15++	movq	16(%rsp),%r14++	movq	24(%rsp),%r13++	movq	32(%rsp),%r12++	movq	40(%rsp),%rbp++	movq	48(%rsp),%rbx++	leaq	56(%rsp),%rax+	leaq	56(%rsp),%rsp++.Lblocks_avx_epilogue:+	jmp	.Ldo_avx+++.p2align	5+.Lbase2_64_avx:++	pushq	%rbx++	pushq	%rbp++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	leaq	-8(%rsp),%rsp++.Lbase2_64_avx_body:++	movq	%rdx,%r15++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13++	movq	0(%rdi),%r14+	movq	8(%rdi),%rbx+	movl	16(%rdi),%ebp++	movq	%r13,%r12+	movq	%r13,%rax+	shrq	$2,%r13+	addq	%r12,%r13++	testq	$31,%rdx+	jz	.Linit_avx++	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp+	subq	$16,%r15++	call	__crypton_poly1305_asm_block++.Linit_avx:++	movq	%r14,%rax+	movq	%r14,%rdx+	shrq	$52,%r14+	movq	%rbx,%r8+	movq	%rbx,%r9+	shrq	$26,%rdx+	andq	$0x3ffffff,%rax+	shlq	$12,%r8+	andq	$0x3ffffff,%rdx+	shrq	$14,%rbx+	orq	%r8,%r14+	shlq	$24,%rbp+	andq	$0x3ffffff,%r14+	shrq	$40,%r9+	andq	$0x3ffffff,%rbx+	orq	%r9,%rbp++	vmovd	%eax,%xmm0+	vmovd	%edx,%xmm1+	vmovd	%r14d,%xmm2+	vmovd	%ebx,%xmm3+	vmovd	%ebp,%xmm4+	movl	$1,20(%rdi)++	call	__crypton_poly1305_asm_init_avx++	movq	%r15,%rdx++	movq	8(%rsp),%r15++	movq	16(%rsp),%r14++	movq	24(%rsp),%r13++	movq	32(%rsp),%r12++	movq	40(%rsp),%rbp++	movq	48(%rsp),%rbx++	leaq	56(%rsp),%rax+	leaq	56(%rsp),%rsp++.Lbase2_64_avx_epilogue:+	jmp	.Ldo_avx+++.p2align	5+.Leven_avx:++	vmovd	0(%rdi),%xmm0+	vmovd	4(%rdi),%xmm1+	vmovd	8(%rdi),%xmm2+	vmovd	12(%rdi),%xmm3+	vmovd	16(%rdi),%xmm4++.Ldo_avx:+	leaq	-248(%rsp),%r11+	subq	$0x218,%rsp+	vmovdqa	%xmm6,80(%r11)+	vmovdqa	%xmm7,96(%r11)+	vmovdqa	%xmm8,112(%r11)+	vmovdqa	%xmm9,128(%r11)+	vmovdqa	%xmm10,144(%r11)+	vmovdqa	%xmm11,160(%r11)+	vmovdqa	%xmm12,176(%r11)+	vmovdqa	%xmm13,192(%r11)+	vmovdqa	%xmm14,208(%r11)+	vmovdqa	%xmm15,224(%r11)+.Ldo_avx_body:+	subq	$64,%rdx+	leaq	-32(%rsi),%rax+	cmovcq	%rax,%rsi++	vmovdqu	48(%rdi),%xmm14+	leaq	112(%rdi),%rdi+	leaq	.Lconst(%rip),%rcx++++	vmovdqu	32(%rsi),%xmm5+	vmovdqu	48(%rsi),%xmm6+	vmovdqa	64(%rcx),%xmm15++	vpsrldq	$6,%xmm5,%xmm7+	vpsrldq	$6,%xmm6,%xmm8+	vpunpckhqdq	%xmm6,%xmm5,%xmm9+	vpunpcklqdq	%xmm6,%xmm5,%xmm5+	vpunpcklqdq	%xmm8,%xmm7,%xmm8++	vpsrlq	$40,%xmm9,%xmm9+	vpsrlq	$26,%xmm5,%xmm6+	vpand	%xmm15,%xmm5,%xmm5+	vpsrlq	$4,%xmm8,%xmm7+	vpand	%xmm15,%xmm6,%xmm6+	vpsrlq	$30,%xmm8,%xmm8+	vpand	%xmm15,%xmm7,%xmm7+	vpand	%xmm15,%xmm8,%xmm8+	vpor	32(%rcx),%xmm9,%xmm9++	jbe	.Lskip_loop_avx+++	vmovdqu	-48(%rdi),%xmm11+	vmovdqu	-32(%rdi),%xmm12+	vpshufd	$0xEE,%xmm14,%xmm13+	vpshufd	$0x44,%xmm14,%xmm10+	vmovdqa	%xmm13,-144(%r11)+	vmovdqa	%xmm10,0(%rsp)+	vpshufd	$0xEE,%xmm11,%xmm14+	vmovdqu	-16(%rdi),%xmm10+	vpshufd	$0x44,%xmm11,%xmm11+	vmovdqa	%xmm14,-128(%r11)+	vmovdqa	%xmm11,16(%rsp)+	vpshufd	$0xEE,%xmm12,%xmm13+	vmovdqu	0(%rdi),%xmm11+	vpshufd	$0x44,%xmm12,%xmm12+	vmovdqa	%xmm13,-112(%r11)+	vmovdqa	%xmm12,32(%rsp)+	vpshufd	$0xEE,%xmm10,%xmm14+	vmovdqu	16(%rdi),%xmm12+	vpshufd	$0x44,%xmm10,%xmm10+	vmovdqa	%xmm14,-96(%r11)+	vmovdqa	%xmm10,48(%rsp)+	vpshufd	$0xEE,%xmm11,%xmm13+	vmovdqu	32(%rdi),%xmm10+	vpshufd	$0x44,%xmm11,%xmm11+	vmovdqa	%xmm13,-80(%r11)+	vmovdqa	%xmm11,64(%rsp)+	vpshufd	$0xEE,%xmm12,%xmm14+	vmovdqu	48(%rdi),%xmm11+	vpshufd	$0x44,%xmm12,%xmm12+	vmovdqa	%xmm14,-64(%r11)+	vmovdqa	%xmm12,80(%rsp)+	vpshufd	$0xEE,%xmm10,%xmm13+	vmovdqu	64(%rdi),%xmm12+	vpshufd	$0x44,%xmm10,%xmm10+	vmovdqa	%xmm13,-48(%r11)+	vmovdqa	%xmm10,96(%rsp)+	vpshufd	$0xEE,%xmm11,%xmm14+	vpshufd	$0x44,%xmm11,%xmm11+	vmovdqa	%xmm14,-32(%r11)+	vmovdqa	%xmm11,112(%rsp)+	vpshufd	$0xEE,%xmm12,%xmm13+	vmovdqa	0(%rsp),%xmm14+	vpshufd	$0x44,%xmm12,%xmm12+	vmovdqa	%xmm13,-16(%r11)+	vmovdqa	%xmm12,128(%rsp)++	jmp	.Loop_avx++.p2align	5+.Loop_avx:+++++++++++++++++++++	vpmuludq	%xmm5,%xmm14,%xmm10+	vpmuludq	%xmm6,%xmm14,%xmm11+	vmovdqa	%xmm2,32(%r11)+	vpmuludq	%xmm7,%xmm14,%xmm12+	vmovdqa	16(%rsp),%xmm2+	vpmuludq	%xmm8,%xmm14,%xmm13+	vpmuludq	%xmm9,%xmm14,%xmm14++	vmovdqa	%xmm0,0(%r11)+	vpmuludq	32(%rsp),%xmm9,%xmm0+	vmovdqa	%xmm1,16(%r11)+	vpmuludq	%xmm8,%xmm2,%xmm1+	vpaddq	%xmm0,%xmm10,%xmm10+	vpaddq	%xmm1,%xmm14,%xmm14+	vmovdqa	%xmm3,48(%r11)+	vpmuludq	%xmm7,%xmm2,%xmm0+	vpmuludq	%xmm6,%xmm2,%xmm1+	vpaddq	%xmm0,%xmm13,%xmm13+	vmovdqa	48(%rsp),%xmm3+	vpaddq	%xmm1,%xmm12,%xmm12+	vmovdqa	%xmm4,64(%r11)+	vpmuludq	%xmm5,%xmm2,%xmm2+	vpmuludq	%xmm7,%xmm3,%xmm0+	vpaddq	%xmm2,%xmm11,%xmm11++	vmovdqa	64(%rsp),%xmm4+	vpaddq	%xmm0,%xmm14,%xmm14+	vpmuludq	%xmm6,%xmm3,%xmm1+	vpmuludq	%xmm5,%xmm3,%xmm3+	vpaddq	%xmm1,%xmm13,%xmm13+	vmovdqa	80(%rsp),%xmm2+	vpaddq	%xmm3,%xmm12,%xmm12+	vpmuludq	%xmm9,%xmm4,%xmm0+	vpmuludq	%xmm8,%xmm4,%xmm4+	vpaddq	%xmm0,%xmm11,%xmm11+	vmovdqa	96(%rsp),%xmm3+	vpaddq	%xmm4,%xmm10,%xmm10++	vmovdqa	128(%rsp),%xmm4+	vpmuludq	%xmm6,%xmm2,%xmm1+	vpmuludq	%xmm5,%xmm2,%xmm2+	vpaddq	%xmm1,%xmm14,%xmm14+	vpaddq	%xmm2,%xmm13,%xmm13+	vpmuludq	%xmm9,%xmm3,%xmm0+	vpmuludq	%xmm8,%xmm3,%xmm1+	vpaddq	%xmm0,%xmm12,%xmm12+	vmovdqu	0(%rsi),%xmm0+	vpaddq	%xmm1,%xmm11,%xmm11+	vpmuludq	%xmm7,%xmm3,%xmm3+	vpmuludq	%xmm7,%xmm4,%xmm7+	vpaddq	%xmm3,%xmm10,%xmm10++	vmovdqu	16(%rsi),%xmm1+	vpaddq	%xmm7,%xmm11,%xmm11+	vpmuludq	%xmm8,%xmm4,%xmm8+	vpmuludq	%xmm9,%xmm4,%xmm9+	vpsrldq	$6,%xmm0,%xmm2+	vpaddq	%xmm8,%xmm12,%xmm12+	vpaddq	%xmm9,%xmm13,%xmm13+	vpsrldq	$6,%xmm1,%xmm3+	vpmuludq	112(%rsp),%xmm5,%xmm9+	vpmuludq	%xmm6,%xmm4,%xmm5+	vpunpckhqdq	%xmm1,%xmm0,%xmm4+	vpaddq	%xmm9,%xmm14,%xmm14+	vmovdqa	-144(%r11),%xmm9+	vpaddq	%xmm5,%xmm10,%xmm10++	vpunpcklqdq	%xmm1,%xmm0,%xmm0+	vpunpcklqdq	%xmm3,%xmm2,%xmm3+++	vpsrldq	$5,%xmm4,%xmm4+	vpsrlq	$26,%xmm0,%xmm1+	vpand	%xmm15,%xmm0,%xmm0+	vpsrlq	$4,%xmm3,%xmm2+	vpand	%xmm15,%xmm1,%xmm1+	vpand	0(%rcx),%xmm4,%xmm4+	vpsrlq	$30,%xmm3,%xmm3+	vpand	%xmm15,%xmm2,%xmm2+	vpand	%xmm15,%xmm3,%xmm3+	vpor	32(%rcx),%xmm4,%xmm4++	vpaddq	0(%r11),%xmm0,%xmm0+	vpaddq	16(%r11),%xmm1,%xmm1+	vpaddq	32(%r11),%xmm2,%xmm2+	vpaddq	48(%r11),%xmm3,%xmm3+	vpaddq	64(%r11),%xmm4,%xmm4++	leaq	32(%rsi),%rax+	leaq	64(%rsi),%rsi+	subq	$64,%rdx+	cmovcq	%rax,%rsi+++++++++++	vpmuludq	%xmm0,%xmm9,%xmm5+	vpmuludq	%xmm1,%xmm9,%xmm6+	vpaddq	%xmm5,%xmm10,%xmm10+	vpaddq	%xmm6,%xmm11,%xmm11+	vmovdqa	-128(%r11),%xmm7+	vpmuludq	%xmm2,%xmm9,%xmm5+	vpmuludq	%xmm3,%xmm9,%xmm6+	vpaddq	%xmm5,%xmm12,%xmm12+	vpaddq	%xmm6,%xmm13,%xmm13+	vpmuludq	%xmm4,%xmm9,%xmm9+	vpmuludq	-112(%r11),%xmm4,%xmm5+	vpaddq	%xmm9,%xmm14,%xmm14++	vpaddq	%xmm5,%xmm10,%xmm10+	vpmuludq	%xmm2,%xmm7,%xmm6+	vpmuludq	%xmm3,%xmm7,%xmm5+	vpaddq	%xmm6,%xmm13,%xmm13+	vmovdqa	-96(%r11),%xmm8+	vpaddq	%xmm5,%xmm14,%xmm14+	vpmuludq	%xmm1,%xmm7,%xmm6+	vpmuludq	%xmm0,%xmm7,%xmm7+	vpaddq	%xmm6,%xmm12,%xmm12+	vpaddq	%xmm7,%xmm11,%xmm11++	vmovdqa	-80(%r11),%xmm9+	vpmuludq	%xmm2,%xmm8,%xmm5+	vpmuludq	%xmm1,%xmm8,%xmm6+	vpaddq	%xmm5,%xmm14,%xmm14+	vpaddq	%xmm6,%xmm13,%xmm13+	vmovdqa	-64(%r11),%xmm7+	vpmuludq	%xmm0,%xmm8,%xmm8+	vpmuludq	%xmm4,%xmm9,%xmm5+	vpaddq	%xmm8,%xmm12,%xmm12+	vpaddq	%xmm5,%xmm11,%xmm11+	vmovdqa	-48(%r11),%xmm8+	vpmuludq	%xmm3,%xmm9,%xmm9+	vpmuludq	%xmm1,%xmm7,%xmm6+	vpaddq	%xmm9,%xmm10,%xmm10++	vmovdqa	-16(%r11),%xmm9+	vpaddq	%xmm6,%xmm14,%xmm14+	vpmuludq	%xmm0,%xmm7,%xmm7+	vpmuludq	%xmm4,%xmm8,%xmm5+	vpaddq	%xmm7,%xmm13,%xmm13+	vpaddq	%xmm5,%xmm12,%xmm12+	vmovdqu	32(%rsi),%xmm5+	vpmuludq	%xmm3,%xmm8,%xmm7+	vpmuludq	%xmm2,%xmm8,%xmm8+	vpaddq	%xmm7,%xmm11,%xmm11+	vmovdqu	48(%rsi),%xmm6+	vpaddq	%xmm8,%xmm10,%xmm10++	vpmuludq	%xmm2,%xmm9,%xmm2+	vpmuludq	%xmm3,%xmm9,%xmm3+	vpsrldq	$6,%xmm5,%xmm7+	vpaddq	%xmm2,%xmm11,%xmm11+	vpmuludq	%xmm4,%xmm9,%xmm4+	vpsrldq	$6,%xmm6,%xmm8+	vpaddq	%xmm3,%xmm12,%xmm2+	vpaddq	%xmm4,%xmm13,%xmm3+	vpmuludq	-32(%r11),%xmm0,%xmm4+	vpmuludq	%xmm1,%xmm9,%xmm0+	vpunpckhqdq	%xmm6,%xmm5,%xmm9+	vpaddq	%xmm4,%xmm14,%xmm4+	vpaddq	%xmm0,%xmm10,%xmm0++	vpunpcklqdq	%xmm6,%xmm5,%xmm5+	vpunpcklqdq	%xmm8,%xmm7,%xmm8+++	vpsrldq	$5,%xmm9,%xmm9+	vpsrlq	$26,%xmm5,%xmm6+	vmovdqa	0(%rsp),%xmm14+	vpand	%xmm15,%xmm5,%xmm5+	vpsrlq	$4,%xmm8,%xmm7+	vpand	%xmm15,%xmm6,%xmm6+	vpand	0(%rcx),%xmm9,%xmm9+	vpsrlq	$30,%xmm8,%xmm8+	vpand	%xmm15,%xmm7,%xmm7+	vpand	%xmm15,%xmm8,%xmm8+	vpor	32(%rcx),%xmm9,%xmm9++++++	vpsrlq	$26,%xmm3,%xmm13+	vpand	%xmm15,%xmm3,%xmm3+	vpaddq	%xmm13,%xmm4,%xmm4++	vpsrlq	$26,%xmm0,%xmm10+	vpand	%xmm15,%xmm0,%xmm0+	vpaddq	%xmm10,%xmm11,%xmm1++	vpsrlq	$26,%xmm4,%xmm10+	vpand	%xmm15,%xmm4,%xmm4++	vpsrlq	$26,%xmm1,%xmm11+	vpand	%xmm15,%xmm1,%xmm1+	vpaddq	%xmm11,%xmm2,%xmm2++	vpaddq	%xmm10,%xmm0,%xmm0+	vpsllq	$2,%xmm10,%xmm10+	vpaddq	%xmm10,%xmm0,%xmm0++	vpsrlq	$26,%xmm2,%xmm12+	vpand	%xmm15,%xmm2,%xmm2+	vpaddq	%xmm12,%xmm3,%xmm3++	vpsrlq	$26,%xmm0,%xmm10+	vpand	%xmm15,%xmm0,%xmm0+	vpaddq	%xmm10,%xmm1,%xmm1++	vpsrlq	$26,%xmm3,%xmm13+	vpand	%xmm15,%xmm3,%xmm3+	vpaddq	%xmm13,%xmm4,%xmm4++	ja	.Loop_avx++.Lskip_loop_avx:++++	vpshufd	$0x10,%xmm14,%xmm14+	addq	$32,%rdx+	jnz	.Long_tail_avx++	vpaddq	%xmm2,%xmm7,%xmm7+	vpaddq	%xmm0,%xmm5,%xmm5+	vpaddq	%xmm1,%xmm6,%xmm6+	vpaddq	%xmm3,%xmm8,%xmm8+	vpaddq	%xmm4,%xmm9,%xmm9++.Long_tail_avx:+	vmovdqa	%xmm2,32(%r11)+	vmovdqa	%xmm0,0(%r11)+	vmovdqa	%xmm1,16(%r11)+	vmovdqa	%xmm3,48(%r11)+	vmovdqa	%xmm4,64(%r11)++++++++	vpmuludq	%xmm7,%xmm14,%xmm12+	vpmuludq	%xmm5,%xmm14,%xmm10+	vpshufd	$0x10,-48(%rdi),%xmm2+	vpmuludq	%xmm6,%xmm14,%xmm11+	vpmuludq	%xmm8,%xmm14,%xmm13+	vpmuludq	%xmm9,%xmm14,%xmm14++	vpmuludq	%xmm8,%xmm2,%xmm0+	vpaddq	%xmm0,%xmm14,%xmm14+	vpshufd	$0x10,-32(%rdi),%xmm3+	vpmuludq	%xmm7,%xmm2,%xmm1+	vpaddq	%xmm1,%xmm13,%xmm13+	vpshufd	$0x10,-16(%rdi),%xmm4+	vpmuludq	%xmm6,%xmm2,%xmm0+	vpaddq	%xmm0,%xmm12,%xmm12+	vpmuludq	%xmm5,%xmm2,%xmm2+	vpaddq	%xmm2,%xmm11,%xmm11+	vpmuludq	%xmm9,%xmm3,%xmm3+	vpaddq	%xmm3,%xmm10,%xmm10++	vpshufd	$0x10,0(%rdi),%xmm2+	vpmuludq	%xmm7,%xmm4,%xmm1+	vpaddq	%xmm1,%xmm14,%xmm14+	vpmuludq	%xmm6,%xmm4,%xmm0+	vpaddq	%xmm0,%xmm13,%xmm13+	vpshufd	$0x10,16(%rdi),%xmm3+	vpmuludq	%xmm5,%xmm4,%xmm4+	vpaddq	%xmm4,%xmm12,%xmm12+	vpmuludq	%xmm9,%xmm2,%xmm1+	vpaddq	%xmm1,%xmm11,%xmm11+	vpshufd	$0x10,32(%rdi),%xmm4+	vpmuludq	%xmm8,%xmm2,%xmm2+	vpaddq	%xmm2,%xmm10,%xmm10++	vpmuludq	%xmm6,%xmm3,%xmm0+	vpaddq	%xmm0,%xmm14,%xmm14+	vpmuludq	%xmm5,%xmm3,%xmm3+	vpaddq	%xmm3,%xmm13,%xmm13+	vpshufd	$0x10,48(%rdi),%xmm2+	vpmuludq	%xmm9,%xmm4,%xmm1+	vpaddq	%xmm1,%xmm12,%xmm12+	vpshufd	$0x10,64(%rdi),%xmm3+	vpmuludq	%xmm8,%xmm4,%xmm0+	vpaddq	%xmm0,%xmm11,%xmm11+	vpmuludq	%xmm7,%xmm4,%xmm4+	vpaddq	%xmm4,%xmm10,%xmm10++	vpmuludq	%xmm5,%xmm2,%xmm2+	vpaddq	%xmm2,%xmm14,%xmm14+	vpmuludq	%xmm9,%xmm3,%xmm1+	vpaddq	%xmm1,%xmm13,%xmm13+	vpmuludq	%xmm8,%xmm3,%xmm0+	vpaddq	%xmm0,%xmm12,%xmm12+	vpmuludq	%xmm7,%xmm3,%xmm1+	vpaddq	%xmm1,%xmm11,%xmm11+	vpmuludq	%xmm6,%xmm3,%xmm3+	vpaddq	%xmm3,%xmm10,%xmm10++	jz	.Lshort_tail_avx++	vmovdqu	0(%rsi),%xmm0+	vmovdqu	16(%rsi),%xmm1++	vpsrldq	$6,%xmm0,%xmm2+	vpsrldq	$6,%xmm1,%xmm3+	vpunpckhqdq	%xmm1,%xmm0,%xmm4+	vpunpcklqdq	%xmm1,%xmm0,%xmm0+	vpunpcklqdq	%xmm3,%xmm2,%xmm3++	vpsrlq	$40,%xmm4,%xmm4+	vpsrlq	$26,%xmm0,%xmm1+	vpand	%xmm15,%xmm0,%xmm0+	vpsrlq	$4,%xmm3,%xmm2+	vpand	%xmm15,%xmm1,%xmm1+	vpsrlq	$30,%xmm3,%xmm3+	vpand	%xmm15,%xmm2,%xmm2+	vpand	%xmm15,%xmm3,%xmm3+	vpor	32(%rcx),%xmm4,%xmm4++	vpshufd	$0x32,-64(%rdi),%xmm9+	vpaddq	0(%r11),%xmm0,%xmm0+	vpaddq	16(%r11),%xmm1,%xmm1+	vpaddq	32(%r11),%xmm2,%xmm2+	vpaddq	48(%r11),%xmm3,%xmm3+	vpaddq	64(%r11),%xmm4,%xmm4+++++	vpmuludq	%xmm0,%xmm9,%xmm5+	vpaddq	%xmm5,%xmm10,%xmm10+	vpmuludq	%xmm1,%xmm9,%xmm6+	vpaddq	%xmm6,%xmm11,%xmm11+	vpmuludq	%xmm2,%xmm9,%xmm5+	vpaddq	%xmm5,%xmm12,%xmm12+	vpshufd	$0x32,-48(%rdi),%xmm7+	vpmuludq	%xmm3,%xmm9,%xmm6+	vpaddq	%xmm6,%xmm13,%xmm13+	vpmuludq	%xmm4,%xmm9,%xmm9+	vpaddq	%xmm9,%xmm14,%xmm14++	vpmuludq	%xmm3,%xmm7,%xmm5+	vpaddq	%xmm5,%xmm14,%xmm14+	vpshufd	$0x32,-32(%rdi),%xmm8+	vpmuludq	%xmm2,%xmm7,%xmm6+	vpaddq	%xmm6,%xmm13,%xmm13+	vpshufd	$0x32,-16(%rdi),%xmm9+	vpmuludq	%xmm1,%xmm7,%xmm5+	vpaddq	%xmm5,%xmm12,%xmm12+	vpmuludq	%xmm0,%xmm7,%xmm7+	vpaddq	%xmm7,%xmm11,%xmm11+	vpmuludq	%xmm4,%xmm8,%xmm8+	vpaddq	%xmm8,%xmm10,%xmm10++	vpshufd	$0x32,0(%rdi),%xmm7+	vpmuludq	%xmm2,%xmm9,%xmm6+	vpaddq	%xmm6,%xmm14,%xmm14+	vpmuludq	%xmm1,%xmm9,%xmm5+	vpaddq	%xmm5,%xmm13,%xmm13+	vpshufd	$0x32,16(%rdi),%xmm8+	vpmuludq	%xmm0,%xmm9,%xmm9+	vpaddq	%xmm9,%xmm12,%xmm12+	vpmuludq	%xmm4,%xmm7,%xmm6+	vpaddq	%xmm6,%xmm11,%xmm11+	vpshufd	$0x32,32(%rdi),%xmm9+	vpmuludq	%xmm3,%xmm7,%xmm7+	vpaddq	%xmm7,%xmm10,%xmm10++	vpmuludq	%xmm1,%xmm8,%xmm5+	vpaddq	%xmm5,%xmm14,%xmm14+	vpmuludq	%xmm0,%xmm8,%xmm8+	vpaddq	%xmm8,%xmm13,%xmm13+	vpshufd	$0x32,48(%rdi),%xmm7+	vpmuludq	%xmm4,%xmm9,%xmm6+	vpaddq	%xmm6,%xmm12,%xmm12+	vpshufd	$0x32,64(%rdi),%xmm8+	vpmuludq	%xmm3,%xmm9,%xmm5+	vpaddq	%xmm5,%xmm11,%xmm11+	vpmuludq	%xmm2,%xmm9,%xmm9+	vpaddq	%xmm9,%xmm10,%xmm10++	vpmuludq	%xmm0,%xmm7,%xmm7+	vpaddq	%xmm7,%xmm14,%xmm14+	vpmuludq	%xmm4,%xmm8,%xmm6+	vpaddq	%xmm6,%xmm13,%xmm13+	vpmuludq	%xmm3,%xmm8,%xmm5+	vpaddq	%xmm5,%xmm12,%xmm12+	vpmuludq	%xmm2,%xmm8,%xmm6+	vpaddq	%xmm6,%xmm11,%xmm11+	vpmuludq	%xmm1,%xmm8,%xmm8+	vpaddq	%xmm8,%xmm10,%xmm10++.Lshort_tail_avx:++++	vpsrldq	$8,%xmm14,%xmm9+	vpsrldq	$8,%xmm13,%xmm8+	vpsrldq	$8,%xmm11,%xmm6+	vpsrldq	$8,%xmm10,%xmm5+	vpsrldq	$8,%xmm12,%xmm7+	vpaddq	%xmm8,%xmm13,%xmm13+	vpaddq	%xmm9,%xmm14,%xmm14+	vpaddq	%xmm5,%xmm10,%xmm10+	vpaddq	%xmm6,%xmm11,%xmm11+	vpaddq	%xmm7,%xmm12,%xmm12+++++	vpsrlq	$26,%xmm13,%xmm3+	vpand	%xmm15,%xmm13,%xmm13+	vpaddq	%xmm3,%xmm14,%xmm14++	vpsrlq	$26,%xmm10,%xmm0+	vpand	%xmm15,%xmm10,%xmm10+	vpaddq	%xmm0,%xmm11,%xmm11++	vpsrlq	$26,%xmm14,%xmm4+	vpand	%xmm15,%xmm14,%xmm14++	vpsrlq	$26,%xmm11,%xmm1+	vpand	%xmm15,%xmm11,%xmm11+	vpaddq	%xmm1,%xmm12,%xmm12++	vpaddq	%xmm4,%xmm10,%xmm10+	vpsllq	$2,%xmm4,%xmm4+	vpaddq	%xmm4,%xmm10,%xmm10++	vpsrlq	$26,%xmm12,%xmm2+	vpand	%xmm15,%xmm12,%xmm12+	vpaddq	%xmm2,%xmm13,%xmm13++	vpsrlq	$26,%xmm10,%xmm0+	vpand	%xmm15,%xmm10,%xmm10+	vpaddq	%xmm0,%xmm11,%xmm11++	vpsrlq	$26,%xmm13,%xmm3+	vpand	%xmm15,%xmm13,%xmm13+	vpaddq	%xmm3,%xmm14,%xmm14++	vmovd	%xmm10,-112(%rdi)+	vmovd	%xmm11,-108(%rdi)+	vmovd	%xmm12,-104(%rdi)+	vmovd	%xmm13,-100(%rdi)+	vmovd	%xmm14,-96(%rdi)+	vmovdqa	80(%r11),%xmm6+	vmovdqa	96(%r11),%xmm7+	vmovdqa	112(%r11),%xmm8+	vmovdqa	128(%r11),%xmm9+	vmovdqa	144(%r11),%xmm10+	vmovdqa	160(%r11),%xmm11+	vmovdqa	176(%r11),%xmm12+	vmovdqa	192(%r11),%xmm13+	vmovdqa	208(%r11),%xmm14+	vmovdqa	224(%r11),%xmm15+	leaq	248(%r11),%rsp+.Ldo_avx_epilogue:+	vzeroupper+	movq	8(%rsp),%rdi+	movq	16(%rsp),%rsi+	.byte	0xf3,0xc3++.LSEH_end_crypton_poly1305_asm_blocks_avx:+.def	crypton_poly1305_asm_blocks_avx2;	.scl 3;	.type 32;	.endef+.p2align	5+crypton_poly1305_asm_blocks_avx2:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%rax+.LSEH_begin_crypton_poly1305_asm_blocks_avx2:+++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	movq	%r9,%rcx+	movl	20(%rdi),%r8d+	cmpq	$128,%rdx+	jb	.Lblocks++	andq	$-16,%rdx++	vzeroupper++	testl	%r8d,%r8d+	jz	.Lbase2_64_avx2++	testq	$63,%rdx+	jz	.Leven_avx2++	pushq	%rbx++	pushq	%rbp++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	leaq	-8(%rsp),%rsp++.Lblocks_avx2_body:++	movq	%rdx,%r15++	movq	0(%rdi),%r8+	movq	8(%rdi),%r9+	movl	16(%rdi),%ebp++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13+++	movl	%r8d,%r14d+	andq	$-2147483648,%r8+	movq	%r9,%r12+	movl	%r9d,%ebx+	andq	$-2147483648,%r9++	shrq	$6,%r8+	shlq	$52,%r12+	addq	%r8,%r14+	shrq	$12,%rbx+	shrq	$18,%r9+	addq	%r12,%r14+	adcq	%r9,%rbx++	movq	%rbp,%r8+	shlq	$40,%r8+	shrq	$24,%rbp+	addq	%r8,%rbx+	adcq	$0,%rbp++	movq	%r13,%r12+	movq	%r13,%rax+	shrq	$2,%r13+	addq	%r12,%r13++.Lbase2_26_pre_avx2:+	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp+	subq	$16,%r15++	call	__crypton_poly1305_asm_block+	movq	%r12,%rax++	testq	$63,%r15+	jnz	.Lbase2_26_pre_avx2+++	movq	%r14,%rax+	movq	%r14,%rdx+	shrq	$52,%r14+	movq	%rbx,%r11+	movq	%rbx,%r12+	shrq	$26,%rdx+	andq	$0x3ffffff,%rax+	shlq	$12,%r11+	andq	$0x3ffffff,%rdx+	shrq	$14,%rbx+	orq	%r11,%r14+	shlq	$24,%rbp+	andq	$0x3ffffff,%r14+	shrq	$40,%r12+	andq	$0x3ffffff,%rbx+	orq	%r12,%rbp++	vmovd	%eax,%xmm0+	vmovd	%edx,%xmm1+	vmovd	%r14d,%xmm2+	vmovd	%ebx,%xmm3+	vmovd	%ebp,%xmm4++	movq	%r15,%rdx++	movq	8(%rsp),%r15++	movq	16(%rsp),%r14++	movq	24(%rsp),%r13++	movq	32(%rsp),%r12++	movq	40(%rsp),%rbp++	movq	48(%rsp),%rbx++	leaq	56(%rsp),%rax+	leaq	56(%rsp),%rsp++.Lblocks_avx2_epilogue:+	jmp	.Ldo_avx2+++.p2align	5+.Lbase2_64_avx2:++	pushq	%rbx++	pushq	%rbp++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	leaq	-8(%rsp),%rsp++.Lbase2_64_avx2_body:++	movq	%rdx,%r15++	movq	24(%rdi),%r11+	movq	32(%rdi),%r13++	movq	0(%rdi),%r14+	movq	8(%rdi),%rbx+	movl	16(%rdi),%ebp++	movq	%r13,%r12+	movq	%r13,%rax+	shrq	$2,%r13+	addq	%r12,%r13++	testq	$63,%rdx+	jz	.Linit_avx2++.Lbase2_64_pre_avx2:+	addq	0(%rsi),%r14+	adcq	8(%rsi),%rbx+	leaq	16(%rsi),%rsi+	adcq	%rcx,%rbp+	subq	$16,%r15++	call	__crypton_poly1305_asm_block+	movq	%r12,%rax++	testq	$63,%r15+	jnz	.Lbase2_64_pre_avx2++.Linit_avx2:++	movq	%r14,%rax+	movq	%r14,%rdx+	shrq	$52,%r14+	movq	%rbx,%r8+	movq	%rbx,%r9+	shrq	$26,%rdx+	andq	$0x3ffffff,%rax+	shlq	$12,%r8+	andq	$0x3ffffff,%rdx+	shrq	$14,%rbx+	orq	%r8,%r14+	shlq	$24,%rbp+	andq	$0x3ffffff,%r14+	shrq	$40,%r9+	andq	$0x3ffffff,%rbx+	orq	%r9,%rbp++	vmovd	%eax,%xmm0+	vmovd	%edx,%xmm1+	vmovd	%r14d,%xmm2+	vmovd	%ebx,%xmm3+	vmovd	%ebp,%xmm4+	movl	$1,20(%rdi)++	call	__crypton_poly1305_asm_init_avx++	movq	%r15,%rdx++	movq	8(%rsp),%r15++	movq	16(%rsp),%r14++	movq	24(%rsp),%r13++	movq	32(%rsp),%r12++	movq	40(%rsp),%rbp++	movq	48(%rsp),%rbx++	leaq	56(%rsp),%rax+	leaq	56(%rsp),%rsp++.Lbase2_64_avx2_epilogue:+	jmp	.Ldo_avx2+++.p2align	5+.Leven_avx2:++	vmovd	0(%rdi),%xmm0+	vmovd	4(%rdi),%xmm1+	vmovd	8(%rdi),%xmm2+	vmovd	12(%rdi),%xmm3+	vmovd	16(%rdi),%xmm4++.Ldo_avx2:+	leaq	-248(%rsp),%r11+	subq	$0x1c8,%rsp+	vmovdqa	%xmm6,80(%r11)+	vmovdqa	%xmm7,96(%r11)+	vmovdqa	%xmm8,112(%r11)+	vmovdqa	%xmm9,128(%r11)+	vmovdqa	%xmm10,144(%r11)+	vmovdqa	%xmm11,160(%r11)+	vmovdqa	%xmm12,176(%r11)+	vmovdqa	%xmm13,192(%r11)+	vmovdqa	%xmm14,208(%r11)+	vmovdqa	%xmm15,224(%r11)+.Ldo_avx2_body:+	leaq	.Lconst(%rip),%rcx+	leaq	48+64(%rdi),%rdi+	vmovdqa	96(%rcx),%ymm7+++	vmovdqu	-64(%rdi),%xmm9+	andq	$-512,%rsp+	vmovdqu	-48(%rdi),%xmm10+	vmovdqu	-32(%rdi),%xmm6+	vmovdqu	-16(%rdi),%xmm11+	vmovdqu	0(%rdi),%xmm12+	vmovdqu	16(%rdi),%xmm13+	leaq	144(%rsp),%rax+	vmovdqu	32(%rdi),%xmm14+	vpermd	%ymm9,%ymm7,%ymm9+	vmovdqu	48(%rdi),%xmm15+	vpermd	%ymm10,%ymm7,%ymm10+	vmovdqu	64(%rdi),%xmm5+	vpermd	%ymm6,%ymm7,%ymm6+	vmovdqa	%ymm9,0(%rsp)+	vpermd	%ymm11,%ymm7,%ymm11+	vmovdqa	%ymm10,32-144(%rax)+	vpermd	%ymm12,%ymm7,%ymm12+	vmovdqa	%ymm6,64-144(%rax)+	vpermd	%ymm13,%ymm7,%ymm13+	vmovdqa	%ymm11,96-144(%rax)+	vpermd	%ymm14,%ymm7,%ymm14+	vmovdqa	%ymm12,128-144(%rax)+	vpermd	%ymm15,%ymm7,%ymm15+	vmovdqa	%ymm13,160-144(%rax)+	vpermd	%ymm5,%ymm7,%ymm5+	vmovdqa	%ymm14,192-144(%rax)+	vmovdqa	%ymm15,224-144(%rax)+	vmovdqa	%ymm5,256-144(%rax)+	vmovdqa	64(%rcx),%ymm5++++	vmovdqu	0(%rsi),%xmm7+	vmovdqu	16(%rsi),%xmm8+	vinserti128	$1,32(%rsi),%ymm7,%ymm7+	vinserti128	$1,48(%rsi),%ymm8,%ymm8+	leaq	64(%rsi),%rsi++	vpsrldq	$6,%ymm7,%ymm9+	vpsrldq	$6,%ymm8,%ymm10+	vpunpckhqdq	%ymm8,%ymm7,%ymm6+	vpunpcklqdq	%ymm10,%ymm9,%ymm9+	vpunpcklqdq	%ymm8,%ymm7,%ymm7++	vpsrlq	$30,%ymm9,%ymm10+	vpsrlq	$4,%ymm9,%ymm9+	vpsrlq	$26,%ymm7,%ymm8+	vpsrlq	$40,%ymm6,%ymm6+	vpand	%ymm5,%ymm9,%ymm9+	vpand	%ymm5,%ymm7,%ymm7+	vpand	%ymm5,%ymm8,%ymm8+	vpand	%ymm5,%ymm10,%ymm10+	vpor	32(%rcx),%ymm6,%ymm6++	vpaddq	%ymm2,%ymm9,%ymm2+	subq	$64,%rdx+	jz	.Ltail_avx2+	jmp	.Loop_avx2++.p2align	5+.Loop_avx2:+++++++++	vpaddq	%ymm0,%ymm7,%ymm0+	vmovdqa	0(%rsp),%ymm7+	vpaddq	%ymm1,%ymm8,%ymm1+	vmovdqa	32(%rsp),%ymm8+	vpaddq	%ymm3,%ymm10,%ymm3+	vmovdqa	96(%rsp),%ymm9+	vpaddq	%ymm4,%ymm6,%ymm4+	vmovdqa	48(%rax),%ymm10+	vmovdqa	112(%rax),%ymm5+++++++++++++++++	vpmuludq	%ymm2,%ymm7,%ymm13+	vpmuludq	%ymm2,%ymm8,%ymm14+	vpmuludq	%ymm2,%ymm9,%ymm15+	vpmuludq	%ymm2,%ymm10,%ymm11+	vpmuludq	%ymm2,%ymm5,%ymm12++	vpmuludq	%ymm0,%ymm8,%ymm6+	vpmuludq	%ymm1,%ymm8,%ymm2+	vpaddq	%ymm6,%ymm12,%ymm12+	vpaddq	%ymm2,%ymm13,%ymm13+	vpmuludq	%ymm3,%ymm8,%ymm6+	vpmuludq	64(%rsp),%ymm4,%ymm2+	vpaddq	%ymm6,%ymm15,%ymm15+	vpaddq	%ymm2,%ymm11,%ymm11+	vmovdqa	-16(%rax),%ymm8++	vpmuludq	%ymm0,%ymm7,%ymm6+	vpmuludq	%ymm1,%ymm7,%ymm2+	vpaddq	%ymm6,%ymm11,%ymm11+	vpaddq	%ymm2,%ymm12,%ymm12+	vpmuludq	%ymm3,%ymm7,%ymm6+	vpmuludq	%ymm4,%ymm7,%ymm2+	vmovdqu	0(%rsi),%xmm7+	vpaddq	%ymm6,%ymm14,%ymm14+	vpaddq	%ymm2,%ymm15,%ymm15+	vinserti128	$1,32(%rsi),%ymm7,%ymm7++	vpmuludq	%ymm3,%ymm8,%ymm6+	vpmuludq	%ymm4,%ymm8,%ymm2+	vmovdqu	16(%rsi),%xmm8+	vpaddq	%ymm6,%ymm11,%ymm11+	vpaddq	%ymm2,%ymm12,%ymm12+	vmovdqa	16(%rax),%ymm2+	vpmuludq	%ymm1,%ymm9,%ymm6+	vpmuludq	%ymm0,%ymm9,%ymm9+	vpaddq	%ymm6,%ymm14,%ymm14+	vpaddq	%ymm9,%ymm13,%ymm13+	vinserti128	$1,48(%rsi),%ymm8,%ymm8+	leaq	64(%rsi),%rsi++	vpmuludq	%ymm1,%ymm2,%ymm6+	vpmuludq	%ymm0,%ymm2,%ymm2+	vpsrldq	$6,%ymm7,%ymm9+	vpaddq	%ymm6,%ymm15,%ymm15+	vpaddq	%ymm2,%ymm14,%ymm14+	vpmuludq	%ymm3,%ymm10,%ymm6+	vpmuludq	%ymm4,%ymm10,%ymm2+	vpsrldq	$6,%ymm8,%ymm10+	vpaddq	%ymm6,%ymm12,%ymm12+	vpaddq	%ymm2,%ymm13,%ymm13+	vpunpckhqdq	%ymm8,%ymm7,%ymm6++	vpmuludq	%ymm3,%ymm5,%ymm3+	vpmuludq	%ymm4,%ymm5,%ymm4+	vpunpcklqdq	%ymm8,%ymm7,%ymm7+	vpaddq	%ymm3,%ymm13,%ymm2+	vpaddq	%ymm4,%ymm14,%ymm3+	vpunpcklqdq	%ymm10,%ymm9,%ymm10+	vpmuludq	80(%rax),%ymm0,%ymm4+	vpmuludq	%ymm1,%ymm5,%ymm0+	vmovdqa	64(%rcx),%ymm5+	vpaddq	%ymm4,%ymm15,%ymm4+	vpaddq	%ymm0,%ymm11,%ymm0+++++	vpsrlq	$26,%ymm3,%ymm14+	vpand	%ymm5,%ymm3,%ymm3+	vpaddq	%ymm14,%ymm4,%ymm4++	vpsrlq	$26,%ymm0,%ymm11+	vpand	%ymm5,%ymm0,%ymm0+	vpaddq	%ymm11,%ymm12,%ymm1++	vpsrlq	$26,%ymm4,%ymm15+	vpand	%ymm5,%ymm4,%ymm4++	vpsrlq	$4,%ymm10,%ymm9++	vpsrlq	$26,%ymm1,%ymm12+	vpand	%ymm5,%ymm1,%ymm1+	vpaddq	%ymm12,%ymm2,%ymm2++	vpaddq	%ymm15,%ymm0,%ymm0+	vpsllq	$2,%ymm15,%ymm15+	vpaddq	%ymm15,%ymm0,%ymm0++	vpand	%ymm5,%ymm9,%ymm9+	vpsrlq	$26,%ymm7,%ymm8++	vpsrlq	$26,%ymm2,%ymm13+	vpand	%ymm5,%ymm2,%ymm2+	vpaddq	%ymm13,%ymm3,%ymm3++	vpaddq	%ymm9,%ymm2,%ymm2+	vpsrlq	$30,%ymm10,%ymm10++	vpsrlq	$26,%ymm0,%ymm11+	vpand	%ymm5,%ymm0,%ymm0+	vpaddq	%ymm11,%ymm1,%ymm1++	vpsrlq	$40,%ymm6,%ymm6++	vpsrlq	$26,%ymm3,%ymm14+	vpand	%ymm5,%ymm3,%ymm3+	vpaddq	%ymm14,%ymm4,%ymm4++	vpand	%ymm5,%ymm7,%ymm7+	vpand	%ymm5,%ymm8,%ymm8+	vpand	%ymm5,%ymm10,%ymm10+	vpor	32(%rcx),%ymm6,%ymm6++	subq	$64,%rdx+	jnz	.Loop_avx2++.byte	0x66,0x90+.Ltail_avx2:++++++++	vpaddq	%ymm0,%ymm7,%ymm0+	vmovdqu	4(%rsp),%ymm7+	vpaddq	%ymm1,%ymm8,%ymm1+	vmovdqu	36(%rsp),%ymm8+	vpaddq	%ymm3,%ymm10,%ymm3+	vmovdqu	100(%rsp),%ymm9+	vpaddq	%ymm4,%ymm6,%ymm4+	vmovdqu	52(%rax),%ymm10+	vmovdqu	116(%rax),%ymm5++	vpmuludq	%ymm2,%ymm7,%ymm13+	vpmuludq	%ymm2,%ymm8,%ymm14+	vpmuludq	%ymm2,%ymm9,%ymm15+	vpmuludq	%ymm2,%ymm10,%ymm11+	vpmuludq	%ymm2,%ymm5,%ymm12++	vpmuludq	%ymm0,%ymm8,%ymm6+	vpmuludq	%ymm1,%ymm8,%ymm2+	vpaddq	%ymm6,%ymm12,%ymm12+	vpaddq	%ymm2,%ymm13,%ymm13+	vpmuludq	%ymm3,%ymm8,%ymm6+	vpmuludq	68(%rsp),%ymm4,%ymm2+	vpaddq	%ymm6,%ymm15,%ymm15+	vpaddq	%ymm2,%ymm11,%ymm11++	vpmuludq	%ymm0,%ymm7,%ymm6+	vpmuludq	%ymm1,%ymm7,%ymm2+	vpaddq	%ymm6,%ymm11,%ymm11+	vmovdqu	-12(%rax),%ymm8+	vpaddq	%ymm2,%ymm12,%ymm12+	vpmuludq	%ymm3,%ymm7,%ymm6+	vpmuludq	%ymm4,%ymm7,%ymm2+	vpaddq	%ymm6,%ymm14,%ymm14+	vpaddq	%ymm2,%ymm15,%ymm15++	vpmuludq	%ymm3,%ymm8,%ymm6+	vpmuludq	%ymm4,%ymm8,%ymm2+	vpaddq	%ymm6,%ymm11,%ymm11+	vpaddq	%ymm2,%ymm12,%ymm12+	vmovdqu	20(%rax),%ymm2+	vpmuludq	%ymm1,%ymm9,%ymm6+	vpmuludq	%ymm0,%ymm9,%ymm9+	vpaddq	%ymm6,%ymm14,%ymm14+	vpaddq	%ymm9,%ymm13,%ymm13++	vpmuludq	%ymm1,%ymm2,%ymm6+	vpmuludq	%ymm0,%ymm2,%ymm2+	vpaddq	%ymm6,%ymm15,%ymm15+	vpaddq	%ymm2,%ymm14,%ymm14+	vpmuludq	%ymm3,%ymm10,%ymm6+	vpmuludq	%ymm4,%ymm10,%ymm2+	vpaddq	%ymm6,%ymm12,%ymm12+	vpaddq	%ymm2,%ymm13,%ymm13++	vpmuludq	%ymm3,%ymm5,%ymm3+	vpmuludq	%ymm4,%ymm5,%ymm4+	vpaddq	%ymm3,%ymm13,%ymm2+	vpaddq	%ymm4,%ymm14,%ymm3+	vpmuludq	84(%rax),%ymm0,%ymm4+	vpmuludq	%ymm1,%ymm5,%ymm0+	vmovdqa	64(%rcx),%ymm5+	vpaddq	%ymm4,%ymm15,%ymm4+	vpaddq	%ymm0,%ymm11,%ymm0+++++	vpsrldq	$8,%ymm12,%ymm8+	vpsrldq	$8,%ymm2,%ymm9+	vpsrldq	$8,%ymm3,%ymm10+	vpsrldq	$8,%ymm4,%ymm6+	vpsrldq	$8,%ymm0,%ymm7+	vpaddq	%ymm8,%ymm12,%ymm12+	vpaddq	%ymm9,%ymm2,%ymm2+	vpaddq	%ymm10,%ymm3,%ymm3+	vpaddq	%ymm6,%ymm4,%ymm4+	vpaddq	%ymm7,%ymm0,%ymm0++	vpermq	$0x2,%ymm3,%ymm10+	vpermq	$0x2,%ymm4,%ymm6+	vpermq	$0x2,%ymm0,%ymm7+	vpermq	$0x2,%ymm12,%ymm8+	vpermq	$0x2,%ymm2,%ymm9+	vpaddq	%ymm10,%ymm3,%ymm3+	vpaddq	%ymm6,%ymm4,%ymm4+	vpaddq	%ymm7,%ymm0,%ymm0+	vpaddq	%ymm8,%ymm12,%ymm12+	vpaddq	%ymm9,%ymm2,%ymm2+++++	vpsrlq	$26,%ymm3,%ymm14+	vpand	%ymm5,%ymm3,%ymm3+	vpaddq	%ymm14,%ymm4,%ymm4++	vpsrlq	$26,%ymm0,%ymm11+	vpand	%ymm5,%ymm0,%ymm0+	vpaddq	%ymm11,%ymm12,%ymm1++	vpsrlq	$26,%ymm4,%ymm15+	vpand	%ymm5,%ymm4,%ymm4++	vpsrlq	$26,%ymm1,%ymm12+	vpand	%ymm5,%ymm1,%ymm1+	vpaddq	%ymm12,%ymm2,%ymm2++	vpaddq	%ymm15,%ymm0,%ymm0+	vpsllq	$2,%ymm15,%ymm15+	vpaddq	%ymm15,%ymm0,%ymm0++	vpsrlq	$26,%ymm2,%ymm13+	vpand	%ymm5,%ymm2,%ymm2+	vpaddq	%ymm13,%ymm3,%ymm3++	vpsrlq	$26,%ymm0,%ymm11+	vpand	%ymm5,%ymm0,%ymm0+	vpaddq	%ymm11,%ymm1,%ymm1++	vpsrlq	$26,%ymm3,%ymm14+	vpand	%ymm5,%ymm3,%ymm3+	vpaddq	%ymm14,%ymm4,%ymm4++	vmovd	%xmm0,-112(%rdi)+	vmovd	%xmm1,-108(%rdi)+	vmovd	%xmm2,-104(%rdi)+	vmovd	%xmm3,-100(%rdi)+	vmovd	%xmm4,-96(%rdi)+	vmovdqa	80(%r11),%xmm6+	vmovdqa	96(%r11),%xmm7+	vmovdqa	112(%r11),%xmm8+	vmovdqa	128(%r11),%xmm9+	vmovdqa	144(%r11),%xmm10+	vmovdqa	160(%r11),%xmm11+	vmovdqa	176(%r11),%xmm12+	vmovdqa	192(%r11),%xmm13+	vmovdqa	208(%r11),%xmm14+	vmovdqa	224(%r11),%xmm15+	leaq	248(%r11),%rsp+.Ldo_avx2_epilogue:+	vzeroupper+	movq	8(%rsp),%rdi+	movq	16(%rsp),%rsi+	.byte	0xf3,0xc3++.LSEH_end_crypton_poly1305_asm_blocks_avx2:+.p2align	6+.Lconst:+.Lmask24:+.long	0x0ffffff,0,0x0ffffff,0,0x0ffffff,0,0x0ffffff,0+.L129:+.long	16777216,0,16777216,0,16777216,0,16777216,0+.Lmask26:+.long	0x3ffffff,0,0x3ffffff,0,0x3ffffff,0,0x3ffffff,0+.Lpermd_avx2:+.long	2,2,2,3,2,0,2,1+.Lpermd_avx512:+.long	0,0,0,1, 0,2,0,3, 0,4,0,5, 0,6,0,7++.L2_44_inp_permd:+.long	0,1,1,2,2,3,7,7+.L2_44_inp_shift:+.quad	0,12,24,64+.L2_44_mask:+.quad	0xfffffffffff,0xfffffffffff,0x3ffffffffff,0xffffffffffffffff+.L2_44_shift_rgt:+.quad	44,44,42,64+.L2_44_shift_lft:+.quad	8,8,10,64++.p2align	6+.Lx_mask44:+.quad	0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff+.quad	0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff+.Lx_mask42:+.quad	0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff+.quad	0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff+.byte	80,111,108,121,49,51,48,53,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.p2align	4+.globl	crypton_xor128_encrypt_n_pad+.def	crypton_xor128_encrypt_n_pad;	.scl 2;	.type 32;	.endef+.p2align	4+crypton_xor128_encrypt_n_pad:+	.byte	0xf3,0x0f,0x1e,0xfa++	subq	%r8,%rdx+	subq	%r8,%rcx+	movq	%r9,%r10+	shrq	$4,%r9+	jz	.Ltail_enc+	nop+.Loop_enc_xmm:+	movdqu	(%rdx,%r8,1),%xmm0+	pxor	(%r8),%xmm0+	movdqu	%xmm0,(%rcx,%r8,1)+	movdqa	%xmm0,(%r8)+	leaq	16(%r8),%r8+	decq	%r9+	jnz	.Loop_enc_xmm++	andq	$15,%r10+	jz	.Ldone_enc++.Ltail_enc:+	movq	$16,%r9+	subq	%r10,%r9+	xorl	%eax,%eax+.Loop_enc_byte:+	movb	(%rdx,%r8,1),%al+	xorb	(%r8),%al+	movb	%al,(%rcx,%r8,1)+	movb	%al,(%r8)+	leaq	1(%r8),%r8+	decq	%r10+	jnz	.Loop_enc_byte++	xorl	%eax,%eax+.Loop_enc_pad:+	movb	%al,(%r8)+	leaq	1(%r8),%r8+	decq	%r9+	jnz	.Loop_enc_pad++.Ldone_enc:+	movq	%r8,%rax+	.byte	0xf3,0xc3+++.globl	crypton_xor128_decrypt_n_pad+.def	crypton_xor128_decrypt_n_pad;	.scl 2;	.type 32;	.endef+.p2align	4+crypton_xor128_decrypt_n_pad:+	.byte	0xf3,0x0f,0x1e,0xfa++	subq	%r8,%rdx+	subq	%r8,%rcx+	movq	%r9,%r10+	shrq	$4,%r9+	jz	.Ltail_dec+	nop+.Loop_dec_xmm:+	movdqu	(%rdx,%r8,1),%xmm0+	movdqa	(%r8),%xmm1+	pxor	%xmm0,%xmm1+	movdqu	%xmm1,(%rcx,%r8,1)+	movdqa	%xmm0,(%r8)+	leaq	16(%r8),%r8+	decq	%r9+	jnz	.Loop_dec_xmm++	pxor	%xmm1,%xmm1+	andq	$15,%r10+	jz	.Ldone_dec++.Ltail_dec:+	movq	$16,%r9+	subq	%r10,%r9+	xorl	%eax,%eax+	xorq	%r11,%r11+.Loop_dec_byte:+	movb	(%rdx,%r8,1),%r11b+	movb	(%r8),%al+	xorb	%r11b,%al+	movb	%al,(%rcx,%r8,1)+	movb	%r11b,(%r8)+	leaq	1(%r8),%r8+	decq	%r10+	jnz	.Loop_dec_byte++	xorl	%eax,%eax+.Loop_dec_pad:+	movb	%al,(%r8)+	leaq	1(%r8),%r8+	decq	%r9+	jnz	.Loop_dec_pad++.Ldone_dec:+	movq	%r8,%rax+	.byte	0xf3,0xc3+++.def	se_handler;	.scl 3;	.type 32;	.endef+.p2align	4+se_handler:+	.byte	0xf3,0x0f,0x1e,0xfa++	pushq	%rsi+	pushq	%rdi+	pushq	%rbx+	pushq	%rbp+	pushq	%r12+	pushq	%r13+	pushq	%r14+	pushq	%r15+	pushfq+	subq	$64,%rsp++	movq	120(%r8),%rax+	movq	248(%r8),%rbx++	movq	8(%r9),%rsi+	movq	56(%r9),%r11++	movl	0(%r11),%r10d+	leaq	(%rsi,%r10,1),%r10+	cmpq	%r10,%rbx+	jb	.Lcommon_seh_tail++	movq	152(%r8),%rax++	movl	4(%r11),%r10d+	leaq	(%rsi,%r10,1),%r10+	cmpq	%r10,%rbx+	jae	.Lcommon_seh_tail++	leaq	56(%rax),%rax++	movq	-8(%rax),%rbx+	movq	-16(%rax),%rbp+	movq	-24(%rax),%r12+	movq	-32(%rax),%r13+	movq	-40(%rax),%r14+	movq	-48(%rax),%r15+	movq	%rbx,144(%r8)+	movq	%rbp,160(%r8)+	movq	%r12,216(%r8)+	movq	%r13,224(%r8)+	movq	%r14,232(%r8)+	movq	%r15,240(%r8)++	jmp	.Lcommon_seh_tail+++.def	avx_handler;	.scl 3;	.type 32;	.endef+.p2align	4+avx_handler:+	.byte	0xf3,0x0f,0x1e,0xfa++	pushq	%rsi+	pushq	%rdi+	pushq	%rbx+	pushq	%rbp+	pushq	%r12+	pushq	%r13+	pushq	%r14+	pushq	%r15+	pushfq+	subq	$64,%rsp++	movq	120(%r8),%rax+	movq	248(%r8),%rbx++	movq	8(%r9),%rsi+	movq	56(%r9),%r11++	movl	0(%r11),%r10d+	leaq	(%rsi,%r10,1),%r10+	cmpq	%r10,%rbx+	jb	.Lcommon_seh_tail++	movq	152(%r8),%rax++	movl	4(%r11),%r10d+	leaq	(%rsi,%r10,1),%r10+	cmpq	%r10,%rbx+	jae	.Lcommon_seh_tail++	movq	208(%r8),%rax++	leaq	80(%rax),%rsi+	leaq	248(%rax),%rax+	leaq	512(%r8),%rdi+	movl	$20,%ecx+.long	0xa548f3fc++.Lcommon_seh_tail:+	movq	8(%rax),%rdi+	movq	16(%rax),%rsi+	movq	%rax,152(%r8)+	movq	%rsi,168(%r8)+	movq	%rdi,176(%r8)++	movq	40(%r9),%rdi+	movq	%r8,%rsi+	movl	$154,%ecx+.long	0xa548f3fc++	movq	%r9,%rsi+	xorq	%rcx,%rcx+	movq	8(%rsi),%rdx+	movq	0(%rsi),%r8+	movq	16(%rsi),%r9+	movq	40(%rsi),%r10+	leaq	56(%rsi),%r11+	leaq	24(%rsi),%r12+	movq	%r10,32(%rsp)+	movq	%r11,40(%rsp)+	movq	%r12,48(%rsp)+	movq	%rcx,56(%rsp)+	call	*__imp_RtlVirtualUnwind(%rip)++	movl	$1,%eax+	addq	$64,%rsp+	popfq+	popq	%r15+	popq	%r14+	popq	%r13+	popq	%r12+	popq	%rbp+	popq	%rbx+	popq	%rdi+	popq	%rsi+	.byte	0xf3,0xc3+++.section	.pdata+.p2align	2+.rva	.LSEH_begin_crypton_poly1305_asm_init+.rva	.LSEH_end_crypton_poly1305_asm_init+.rva	.LSEH_info_crypton_poly1305_asm_init++.rva	.LSEH_begin_crypton_poly1305_asm_blocks+.rva	.LSEH_end_crypton_poly1305_asm_blocks+.rva	.LSEH_info_crypton_poly1305_asm_blocks++.rva	.LSEH_begin_crypton_poly1305_asm_emit+.rva	.LSEH_end_crypton_poly1305_asm_emit+.rva	.LSEH_info_crypton_poly1305_asm_emit+.rva	.LSEH_begin_crypton_poly1305_asm_blocks_avx+.rva	.Lbase2_64_avx+.rva	.LSEH_info_crypton_poly1305_asm_blocks_avx_1++.rva	.Lbase2_64_avx+.rva	.Leven_avx+.rva	.LSEH_info_crypton_poly1305_asm_blocks_avx_2++.rva	.Leven_avx+.rva	.LSEH_end_crypton_poly1305_asm_blocks_avx+.rva	.LSEH_info_crypton_poly1305_asm_blocks_avx_3+.rva	.LSEH_begin_crypton_poly1305_asm_blocks_avx2+.rva	.Lbase2_64_avx2+.rva	.LSEH_info_crypton_poly1305_asm_blocks_avx2_1++.rva	.Lbase2_64_avx2+.rva	.Leven_avx2+.rva	.LSEH_info_crypton_poly1305_asm_blocks_avx2_2++.rva	.Leven_avx2+.rva	.LSEH_end_crypton_poly1305_asm_blocks_avx2+.rva	.LSEH_info_crypton_poly1305_asm_blocks_avx2_3+.section	.xdata+.p2align	3+.LSEH_info_crypton_poly1305_asm_init:+.byte	9,0,0,0+.rva	se_handler+.long	0,0++.LSEH_info_crypton_poly1305_asm_blocks:+.byte	9,0,0,0+.rva	se_handler+.rva	.Lblocks_body,.Lblocks_epilogue++.LSEH_info_crypton_poly1305_asm_emit:+.byte	9,0,0,0+.rva	se_handler+.long	0,0+.LSEH_info_crypton_poly1305_asm_blocks_avx_1:+.byte	9,0,0,0+.rva	se_handler+.rva	.Lblocks_avx_body,.Lblocks_avx_epilogue++.LSEH_info_crypton_poly1305_asm_blocks_avx_2:+.byte	9,0,0,0+.rva	se_handler+.rva	.Lbase2_64_avx_body,.Lbase2_64_avx_epilogue++.LSEH_info_crypton_poly1305_asm_blocks_avx_3:+.byte	9,0,0,0+.rva	avx_handler+.rva	.Ldo_avx_body,.Ldo_avx_epilogue+.LSEH_info_crypton_poly1305_asm_blocks_avx2_1:+.byte	9,0,0,0+.rva	se_handler+.rva	.Lblocks_avx2_body,.Lblocks_avx2_epilogue++.LSEH_info_crypton_poly1305_asm_blocks_avx2_2:+.byte	9,0,0,0+.rva	se_handler+.rva	.Lbase2_64_avx2_body,.Lbase2_64_avx2_epilogue++.LSEH_info_crypton_poly1305_asm_blocks_avx2_3:+.byte	9,0,0,0+.rva	avx_handler+.rva	.Ldo_avx2_body,.Ldo_avx2_epilogue
+ cbits/asm/poly1305-x86_64.pl view
@@ -0,0 +1,4333 @@+#!/usr/bin/env perl+#+# ====================================================================+# Written by Andy Polyakov, @dot-asm, initially  for the OpenSSL+# project.+# ====================================================================+#+# This module implements Poly1305 hash for x86_64.+#+# March 2015+#+# Initial release.+#+# December 2016+#+# Add AVX512F+VL+BW code path.+#+# November 2017+#+# Convert AVX512F+VL+BW code path to pure AVX512F, so that it can be+# executed even on Knights Landing. Trigger for modification was+# observation that AVX512 code paths can negatively affect overall+# Skylake-X system performance. Since we are likely to suppress+# AVX512F capability flag [at least on Skylake-X], conversion serves+# as kind of "investment protection". Note that next *lake processor,+# Cannonlake, has AVX512IFMA code path to execute...+#+# Numbers are cycles per processed byte with poly1305_blocks alone,+# most are measured with rdtsc at fixed clock frequency.+#+#		IALU/gcc-4.8(i)	AVX(ii)		AVX2	AVX-512+# P4		4.46/+120%	-+# Core 2	2.41/+90%	-+# Westmere	1.88/+120%	-+# Sandy Bridge	1.39/+140%	1.10+# Haswell	1.14/+175%	1.11		0.65+# Skylake[-X]	1.13/+120%	0.96		0.51	[0.35]+# Cannon Lake	1.13/+120%	0.93		0.38(iv)0.24(iv)+# Rocket Lake	1.13/+120%	0.84		0.43(iv)0.24(iv)+# Silvermont	2.83/+95%	-+# Knights L	3.60/?		1.65		1.10	0.41(iii)+# Goldmont	1.70/+180%	-+# VIA Nano	1.82/+150%	-+# Sledgehammer	1.38/+160%	-+# Bulldozer	2.30/+130%	0.97+# Ryzen		1.15/+200%	1.08		1.18+#+# (i)	improvement coefficients relative to clang are more modest and+#	are ~50% on most processors, in both cases we are comparing to+#	__int128 code;+# (ii)	SSE2 implementation was attempted, but among non-AVX processors+#	it was faster than integer-only code only on older Intel P4 and+#	Core processors, 50-30%, less newer processor is, but slower on+#	contemporary ones, for example almost 2x slower on Atom, and as+#	former are naturally disappearing, SSE2 is deemed unnecessary;+# (iii)	strangely enough performance seems to vary from core to core,+#	listed result is best case;+# (iv)	these are IFMA results, which in addition means that first IALU+#	column does not reflect short-input performance;++$flavour = shift;+$output  = shift;+if ($flavour =~ /\./) { $output = $flavour; undef $flavour; }++$win64=0; $win64=1 if ($flavour =~ /[nm]asm|mingw64/ || $output =~ /\.asm$/);++$0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1;+( $xlate="${dir}x86_64-xlate.pl" and -f $xlate ) or+( $xlate="${dir}../../perlasm/x86_64-xlate.pl" and -f $xlate) or+die "can't locate x86_64-xlate.pl";++$avx=undef;++if (!defined($avx) && $win64 && ($flavour =~ /nasm/ || $ENV{ASM} =~ /nasm/) &&+	   ($ENV{ASM} //= "nasm") &&+	   `"$ENV{ASM}" -v 2>&1` =~ /NASM version ([0-9]+\.[0-9]+)(?:\.([0-9]+))?/) {+	$avx = ($1>=2.09) + ($1>=2.10) + 2 * ($1>=2.12);+	$avx += 2 if ($1==2.11 && $2>=8);+}++if (!defined($avx) && $win64 && ($flavour =~ /masm/ || $ENV{ASM} =~ /ml64/) &&+	   ($ENV{ASM} //= "ml64") &&+	   `"$ENV{ASM}" 2>&1` =~ /Version ([0-9]+)\./) {+	$avx = ($1>=10) + ($1>=12) + 2 * ($1>=14);+}++$ENV{CC} //= "cc";+if (!defined($avx) && `$ENV{CC} -Wa,-v -c -o /dev/zero -x assembler /dev/null 2>&1`+		=~ /GNU assembler version ([0-9]+)\.([0-9]+)/) {+	my $ver = $1 + $2/100.0;	# 3.1->3.01, 3.10->3.10+	$avx = ($ver>=2.19) + ($ver>=2.22) + ($ver>=2.25) + ($ver>=2.26);+}++if (!defined($avx) && `$ENV{CC} -v 2>&1`+		=~ /((?:^clang|LLVM) version|.*based on LLVM) ([0-9]+)\.([0-9]+)/) {+	my $ver = $2 + $3/100.0;	# 3.1->3.01, 3.10->3.10+	$avx = ($ver>=3.0) + ($ver>3.0);+	$avx += 2*($ver>=7.0) if ($1 =~ /^clang/);+}++open OUT,"| \"$^X\" \"$xlate\" $flavour \"$output\"";+*STDOUT=*OUT;++my ($ctx,$inp,$len,$padbit)=("%rdi","%rsi","%rdx","%rcx");+my ($mac,$nonce)=($inp,$len);	# *_emit arguments+my ($d1,$d2,$d3, $r0,$r1,$s1)=map("%r$_",(8..13));+my ($h0,$h1,$h2)=("%r14","%rbx","%rbp");++sub poly1305_iteration {+# input:	copy of $r1 in %rax, $h0-$h2, $r0-$r1+# output:	$h0-$h2 *= $r0-$r1+$code.=<<___;+	mulq	$h0			# h0*r1+	mov	%rax,$d2+	 mov	$r0,%rax+	mov	%rdx,$d3++	mulq	$h0			# h0*r0+	mov	%rax,$h0		# future $h0+	 mov	$r0,%rax+	mov	%rdx,$d1++	mulq	$h1			# h1*r0+	add	%rax,$d2+	 mov	$s1,%rax+	adc	%rdx,$d3++	mulq	$h1			# h1*s1+	 mov	$h2,$h1			# borrow $h1+	add	%rax,$h0+	adc	%rdx,$d1++	imulq	$s1,$h1			# h2*s1+	add	$h1,$d2+	 mov	$d1,$h1+	adc	\$0,$d3++	imulq	$r0,$h2			# h2*r0+	add	$d2,$h1+	mov	\$-4,%rax		# mask value+	adc	$h2,$d3++	and	$d3,%rax		# last reduction step+	mov	$d3,$h2+	shr	\$2,$d3+	and	\$3,$h2+	add	$d3,%rax+	add	%rax,$h0+	adc	\$0,$h1+	adc	\$0,$h2+___+}++########################################################################+# Layout of opaque area is following.+#+#	unsigned __int64 h[3];		# current hash value base 2^64+#	unsigned __int64 r[2];		# key value base 2^64++if ($flavour =~ /kernel/) {+$code.=<<___	if ($avx);+.globl	poly1305_blocks_avx+___+$code.=<<___	if ($avx>1);+.globl	poly1305_blocks_avx2+___+$code.=<<___	if ($avx>3);+.globl	poly1305_init_base2_44+.globl	poly1305_blocks_base2_44+.globl	poly1305_emit_base2_44+.globl	poly1305_blocks_vpmadd52+___+}+$code.=<<___;+.text++.extern	OPENSSL_ia32cap_P++.globl	poly1305_init+.hidden	poly1305_init+.globl	poly1305_blocks+.hidden	poly1305_blocks+.globl	poly1305_emit+.hidden	poly1305_emit++.type	poly1305_init,\@function,3+.align	32+poly1305_init:+	xor	%rax,%rax+	mov	%rax,0($ctx)		# initialize hash value+	mov	%rax,8($ctx)+	mov	%rax,16($ctx)		# [along with is_base2_26]++	cmp	\$0,$inp+	je	.Lno_key++	mov	\$0x0ffffffc0fffffff,%rax+	lea	-3(%rax),%rcx		# $0x0ffffffc0ffffffc+	and	0($inp),%rax+	and	8($inp),%rcx+	mov	%rax,24($ctx)+	mov	%rcx,32($ctx)+___+$code.=<<___	if ($avx);+	movl	\$-1,48($ctx)		# write impossible value+___+					if ($flavour !~ /kernel/) {+$code.=<<___;+	lea	poly1305_blocks(%rip),%r10+	lea	poly1305_emit(%rip),%r11+___+$code.=<<___	if ($avx);+	mov	OPENSSL_ia32cap_P+4(%rip),%r9+	lea	poly1305_blocks_avx(%rip),%rax+	bt	\$`60-32`,%r9		# AVX?+	cmovc	%rax,%r10+___+$code.=<<___	if ($avx>1);+	lea	poly1305_blocks_avx2(%rip),%rax+	bt	\$`5+32`,%r9		# AVX2?+	cmovc	%rax,%r10+___+$code.=<<___	if ($avx>3);+	mov	\$`(1<<31|1<<21)`,%rax	# AVX512VL|AVX512IFMA+	shr	\$32,%r9+	and	%rax,%r9+	cmp	%rax,%r9+	je	.Linit_base2_44+___+$code.=<<___	if ($flavour !~ /elf32/);+	mov	%r10,0(%rdx)+	mov	%r11,8(%rdx)+___+$code.=<<___	if ($flavour =~ /elf32/);+	mov	%r10d,0(%rdx)+	mov	%r11d,4(%rdx)+___+					}+$code.=<<___;+	mov	\$1,%eax+.Lno_key:+	ret+.size	poly1305_init,.-poly1305_init++.type	poly1305_blocks,\@function,4+.align	32+poly1305_blocks:+.cfi_startproc+.Lblocks:+	shr	\$4,$len+	jz	.Lno_data		# too short++	push	%rbx+.cfi_push	%rbx+	push	%rbp+.cfi_push	%rbp+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15+	lea	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+.Lblocks_body:++	mov	$len,%r15		# reassign $len++	mov	24($ctx),$r0		# load r+	mov	32($ctx),$s1++	mov	0($ctx),$h0		# load hash value base 2^64+	mov	8($ctx),$h1+	mov	16($ctx),$h2		# [along with is_base2_26]++	mov	$h0#d,%eax		# load hash value base 2^26+	mov	4($ctx),%edx+	mov	$h1#d,%r8d+	mov	12($ctx),%r10d+	mov	$h2#d,%r12d++	shl	\$26,%rdx		# base 2^26 -> base 2^64+	mov	%r8,%r9+	shl	\$52,%r8+	add	%rdx,%rax+	shr	\$12,%r9+	add	%rax,%r8		# h0+	adc	\$0,%r9++	shl	\$14,%r10+	mov	%r12,%rax+	shr	\$24,%r12+	add	%r10,%r9+	shl	\$40,%rax+	add	%rax,%r9		# h1+	adc	\$0,%r12		# h2++	cmp	\$4,$h2			# is_base2_26? [4 is as good as 2^32-1]++	cmova	%r8,$h0			# choose between radixes+	cmova	%r9,$h1+	cmova	%r12,$h2++	mov	$s1,$r1+	shr	\$2,$s1+	mov	$r1,%rax+	add	$r1,$s1			# s1 = r1 + (r1 >> 2)+	jmp	.Loop++.align	32+.Loop:+	add	0($inp),$h0		# accumulate input+	adc	8($inp),$h1+	lea	16($inp),$inp+	adc	$padbit,$h2+___+	&poly1305_iteration();+$code.=<<___;+	mov	$r1,%rax+	dec	%r15			# len-=16+	jnz	.Loop++	mov	$h0,0($ctx)		# store hash value+	mov	$h1,8($ctx)+	mov	$h2,16($ctx)++	mov	8(%rsp),%r15+.cfi_restore	%r15+	mov	16(%rsp),%r14+.cfi_restore	%r14+	mov	24(%rsp),%r13+.cfi_restore	%r13+	mov	32(%rsp),%r12+.cfi_restore	%r12+	mov	40(%rsp),%rbp+.cfi_restore	%rbp+	mov	48(%rsp),%rbx+.cfi_restore	%rbx+	lea	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+.Lno_data:+.Lblocks_epilogue:+	ret+.cfi_endproc+.size	poly1305_blocks,.-poly1305_blocks++.type	poly1305_emit,\@function,3+.align	32+poly1305_emit:+	mov	0($ctx),%eax	# load hash value base 2^26+	mov	4($ctx),%ecx+	mov	8($ctx),%r8d+	mov	12($ctx),%r11d+	mov	16($ctx),%r10d++	shl	\$26,%rcx	# base 2^26 -> base 2^64+	mov	%r8,%r9+	shl	\$52,%r8+	add	%rcx,%rax+	shr	\$12,%r9+	add	%rax,%r8	# h0+	adc	\$0,%r9++	shl	\$14,%r11+	mov	%r10,%rax+	shr	\$24,%r10+	add	%r11,%r9+	 mov	0($ctx),%rcx	# load hash value base 2^64+	shl	\$40,%rax+	 mov	8($ctx),%r11+	add	%rax,%r9	# h1+	 mov	16($ctx),%rax	# [along with is_base2_26]+	adc	\$0,%r10	# h2++	cmp	\$4,%rax	# is_base2_26? [4 is as good as 2^32-1]++	cmovbe	%rcx,%r8	# choose between radixes+	cmovbe	%r11,%r9+	cmovbe	%rax,%r10++	mov	%r8,%rax+	add	\$5,%r8		# compare to modulus+	mov	%r9,%rcx+	adc	\$0,%r9+	adc	\$0,%r10+	shr	\$2,%r10	# did 130-bit value overflow?+	cmovnz	%r8,%rax+	cmovnz	%r9,%rcx++	add	0($nonce),%rax	# accumulate nonce+	adc	8($nonce),%rcx+	mov	%rax,0($mac)	# write result+	mov	%rcx,8($mac)++	ret+.size	poly1305_emit,.-poly1305_emit+___+if ($avx) {++########################################################################+# Layout of opaque area is following.+#+#	unsigned __int32 h[5];		# current hash value base 2^26+#	unsigned __int32 is_base2_26;+#	unsigned __int64 r[2];		# key value base 2^64+#	unsigned __int64 pad;+#	struct { unsigned __int32 r^2, r^1, r^4, r^3; } r[9];+#+# where r^n are base 2^26 digits of degrees of multiplier key. There are+# 5 digits, but last four are interleaved with multiples of 5, totalling+# in 9 elements: r0, r1, 5*r1, r2, 5*r2, r3, 5*r3, r4, 5*r4.++my ($H0,$H1,$H2,$H3,$H4, $T0,$T1,$T2,$T3,$T4, $D0,$D1,$D2,$D3,$D4, $MASK) =+    map("%xmm$_",(0..15));++$code.=<<___;+.type	__poly1305_block,\@abi-omnipotent+.align	32+__poly1305_block:+___+	&poly1305_iteration();+$code.=<<___;+	ret+.size	__poly1305_block,.-__poly1305_block++.type	__poly1305_init_avx,\@abi-omnipotent+.align	32+__poly1305_init_avx:+	cmpl	\$-1,48($ctx)+	jne	.Ldone_init_avx++	mov	$r0,$h0+	mov	$r1,$h1+	xor	$h2,$h2++	lea	48+64($ctx),$ctx	# size optimization++	mov	$r1,%rax+	call	__poly1305_block	# r^2++	mov	\$0x3ffffff,%eax	# save interleaved r^2 and r base 2^26+	mov	\$0x3ffffff,%edx+	mov	$h0,$d1+	and	$h0#d,%eax+	mov	$r0,$d2+	and	$r0#d,%edx+	mov	%eax,`16*0+0-64`($ctx)+	shr	\$26,$d1+	mov	%edx,`16*0+4-64`($ctx)+	shr	\$26,$d2++	mov	\$0x3ffffff,%eax+	mov	\$0x3ffffff,%edx+	and	$d1#d,%eax+	and	$d2#d,%edx+	mov	%eax,`16*1+0-64`($ctx)+	lea	(%rax,%rax,4),%eax	# *5+	mov	%edx,`16*1+4-64`($ctx)+	lea	(%rdx,%rdx,4),%edx	# *5+	mov	%eax,`16*2+0-64`($ctx)+	shr	\$26,$d1+	mov	%edx,`16*2+4-64`($ctx)+	shr	\$26,$d2++	mov	$h1,%rax+	mov	$r1,%rdx+	shl	\$12,%rax+	shl	\$12,%rdx+	or	$d1,%rax+	or	$d2,%rdx+	and	\$0x3ffffff,%eax+	and	\$0x3ffffff,%edx+	mov	%eax,`16*3+0-64`($ctx)+	lea	(%rax,%rax,4),%eax	# *5+	mov	%edx,`16*3+4-64`($ctx)+	lea	(%rdx,%rdx,4),%edx	# *5+	mov	%eax,`16*4+0-64`($ctx)+	mov	$h1,$d1+	mov	%edx,`16*4+4-64`($ctx)+	mov	$r1,$d2++	mov	\$0x3ffffff,%eax+	mov	\$0x3ffffff,%edx+	shr	\$14,$d1+	shr	\$14,$d2+	and	$d1#d,%eax+	and	$d2#d,%edx+	mov	%eax,`16*5+0-64`($ctx)+	lea	(%rax,%rax,4),%eax	# *5+	mov	%edx,`16*5+4-64`($ctx)+	lea	(%rdx,%rdx,4),%edx	# *5+	mov	%eax,`16*6+0-64`($ctx)+	shr	\$26,$d1+	mov	%edx,`16*6+4-64`($ctx)+	shr	\$26,$d2++	mov	$h2,%rax+	shl	\$24,%rax+	or	%rax,$d1+	mov	$d1#d,`16*7+0-64`($ctx)+	lea	($d1,$d1,4),$d1		# *5+	mov	$d2#d,`16*7+4-64`($ctx)+	lea	($d2,$d2,4),$d2		# *5+	mov	$d1#d,`16*8+0-64`($ctx)+	mov	$d2#d,`16*8+4-64`($ctx)++	mov	$r1,%rax+	call	__poly1305_block	# r^3++	mov	\$0x3ffffff,%eax	# save r^3 base 2^26+	mov	$h0,$d1+	and	$h0#d,%eax+	shr	\$26,$d1+	mov	%eax,`16*0+12-64`($ctx)++	mov	\$0x3ffffff,%edx+	and	$d1#d,%edx+	mov	%edx,`16*1+12-64`($ctx)+	lea	(%rdx,%rdx,4),%edx	# *5+	shr	\$26,$d1+	mov	%edx,`16*2+12-64`($ctx)++	mov	$h1,%rax+	shl	\$12,%rax+	or	$d1,%rax+	and	\$0x3ffffff,%eax+	mov	%eax,`16*3+12-64`($ctx)+	lea	(%rax,%rax,4),%eax	# *5+	mov	$h1,$d1+	mov	%eax,`16*4+12-64`($ctx)++	mov	\$0x3ffffff,%edx+	shr	\$14,$d1+	and	$d1#d,%edx+	mov	%edx,`16*5+12-64`($ctx)+	lea	(%rdx,%rdx,4),%edx	# *5+	shr	\$26,$d1+	mov	%edx,`16*6+12-64`($ctx)++	mov	$h2,%rax+	shl	\$24,%rax+	or	%rax,$d1+	mov	$d1#d,`16*7+12-64`($ctx)+	lea	($d1,$d1,4),$d1		# *5+	mov	$d1#d,`16*8+12-64`($ctx)++	mov	$r1,%rax+	call	__poly1305_block	# r^4++	mov	\$0x3ffffff,%eax	# save r^4 base 2^26+	mov	$h0,$d1+	and	$h0#d,%eax+	shr	\$26,$d1+	mov	%eax,`16*0+8-64`($ctx)++	mov	\$0x3ffffff,%edx+	and	$d1#d,%edx+	mov	%edx,`16*1+8-64`($ctx)+	lea	(%rdx,%rdx,4),%edx	# *5+	shr	\$26,$d1+	mov	%edx,`16*2+8-64`($ctx)++	mov	$h1,%rax+	shl	\$12,%rax+	or	$d1,%rax+	and	\$0x3ffffff,%eax+	mov	%eax,`16*3+8-64`($ctx)+	lea	(%rax,%rax,4),%eax	# *5+	mov	$h1,$d1+	mov	%eax,`16*4+8-64`($ctx)++	mov	\$0x3ffffff,%edx+	shr	\$14,$d1+	and	$d1#d,%edx+	mov	%edx,`16*5+8-64`($ctx)+	lea	(%rdx,%rdx,4),%edx	# *5+	shr	\$26,$d1+	mov	%edx,`16*6+8-64`($ctx)++	mov	$h2,%rax+	shl	\$24,%rax+	or	%rax,$d1+	mov	$d1#d,`16*7+8-64`($ctx)+	lea	($d1,$d1,4),$d1		# *5+	mov	$d1#d,`16*8+8-64`($ctx)++	lea	-48-64($ctx),$ctx	# size [de-]optimization+.Ldone_init_avx:+	ret+.size	__poly1305_init_avx,.-__poly1305_init_avx++.type	poly1305_blocks_avx,\@function,4+.align	32+poly1305_blocks_avx:+.cfi_startproc+	mov	20($ctx),%r8d		# load is_base2_26+	cmp	\$128,$len+	jb	.Lblocks++	and	\$-16,$len++	vzeroupper++	test	%r8d,%r8d		# is_base2_26?+	jz	.Lbase2_64_avx++	test	\$31,$len+	jz	.Leven_avx++	push	%rbx+.cfi_push	%rbx+	push	%rbp+.cfi_push	%rbp+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15+	lea	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+.Lblocks_avx_body:++	mov	$len,%r15		# reassign $len++	mov	0($ctx),$d1		# load hash value+	mov	8($ctx),$d2+	mov	16($ctx),$h2#d++	mov	24($ctx),$r0		# load r+	mov	32($ctx),$s1++	################################# base 2^26 -> base 2^64+	mov	$d1#d,$h0#d+	and	\$`-1*(1<<31)`,$d1+	mov	$d2,$r1			# borrow $r1+	mov	$d2#d,$h1#d+	and	\$`-1*(1<<31)`,$d2++	shr	\$6,$d1+	shl	\$52,$r1+	add	$d1,$h0+	shr	\$12,$h1+	shr	\$18,$d2+	add	$r1,$h0+	adc	$d2,$h1++	mov	$h2,$d1+	shl	\$40,$d1+	shr	\$24,$h2+	add	$d1,$h1+	adc	\$0,$h2			# can be partially reduced...++	mov	$s1,$r1+	mov	$s1,%rax+	shr	\$2,$s1+	add	$r1,$s1			# s1 = r1 + (r1 >> 2)++	add	0($inp),$h0		# accumulate input+	adc	8($inp),$h1+	lea	16($inp),$inp+	adc	$padbit,$h2++	call	__poly1305_block++	################################# base 2^64 -> base 2^26+	mov	$h0,%rax+	mov	$h0,%rdx+	shr	\$52,$h0+	mov	$h1,$r0+	mov	$h1,$r1+	shr	\$26,%rdx+	and	\$0x3ffffff,%rax	# h[0]+	shl	\$12,$r0+	and	\$0x3ffffff,%rdx	# h[1]+	shr	\$14,$h1+	or	$r0,$h0+	shl	\$24,$h2+	and	\$0x3ffffff,$h0		# h[2]+	shr	\$40,$r1+	and	\$0x3ffffff,$h1		# h[3]+	or	$r1,$h2			# h[4]++	vmovd	%rax#d,$H0+	vmovd	%rdx#d,$H1+	vmovd	$h0#d,$H2+	vmovd	$h1#d,$H3+	vmovd	$h2#d,$H4++	lea	-16(%r15),$len++	mov	8(%rsp),%r15+.cfi_restore	%r15+	mov	16(%rsp),%r14+.cfi_restore	%r14+	mov	24(%rsp),%r13+.cfi_restore	%r13+	mov	32(%rsp),%r12+.cfi_restore	%r12+	mov	40(%rsp),%rbp+.cfi_restore	%rbp+	mov	48(%rsp),%rbx+.cfi_restore	%rbx+	lea	56(%rsp),%rax		# for win64+	lea	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+.Lblocks_avx_epilogue:+	jmp	.Ldo_avx+.cfi_endproc++.align	32+.Lbase2_64_avx:+.cfi_startproc+	push	%rbx+.cfi_push	%rbx+	push	%rbp+.cfi_push	%rbp+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15+	lea	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+.Lbase2_64_avx_body:++	mov	$len,%r15		# reassign $len++	mov	24($ctx),$r0		# load r+	mov	32($ctx),$s1++	mov	0($ctx),$h0		# load hash value+	mov	8($ctx),$h1+	mov	16($ctx),$h2#d++	mov	$s1,$r1+	mov	$s1,%rax+	shr	\$2,$s1+	add	$r1,$s1			# s1 = r1 + (r1 >> 2)++	test	\$31,$len+	jz	.Linit_avx++	add	0($inp),$h0		# accumulate input+	adc	8($inp),$h1+	lea	16($inp),$inp+	adc	$padbit,$h2+	sub	\$16,%r15++	call	__poly1305_block++.Linit_avx:+	################################# base 2^64 -> base 2^26+	mov	$h0,%rax+	mov	$h0,%rdx+	shr	\$52,$h0+	mov	$h1,$d1+	mov	$h1,$d2+	shr	\$26,%rdx+	and	\$0x3ffffff,%rax	# h[0]+	shl	\$12,$d1+	and	\$0x3ffffff,%rdx	# h[1]+	shr	\$14,$h1+	or	$d1,$h0+	shl	\$24,$h2+	and	\$0x3ffffff,$h0		# h[2]+	shr	\$40,$d2+	and	\$0x3ffffff,$h1		# h[3]+	or	$d2,$h2			# h[4]++	vmovd	%rax#d,$H0+	vmovd	%rdx#d,$H1+	vmovd	$h0#d,$H2+	vmovd	$h1#d,$H3+	vmovd	$h2#d,$H4+	movl	\$1,20($ctx)		# set is_base2_26++	call	__poly1305_init_avx++	mov	%r15,$len++	mov	8(%rsp),%r15+.cfi_restore	%r15+	mov	16(%rsp),%r14+.cfi_restore	%r14+	mov	24(%rsp),%r13+.cfi_restore	%r13+	mov	32(%rsp),%r12+.cfi_restore	%r12+	mov	40(%rsp),%rbp+.cfi_restore	%rbp+	mov	48(%rsp),%rbx+.cfi_restore	%rbx+	lea	56(%rsp),%rax		# for win64+	lea	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+.Lbase2_64_avx_epilogue:+	jmp	.Ldo_avx+.cfi_endproc++.align	32+.Leven_avx:+.cfi_startproc+	vmovd		4*0($ctx),$H0		# load hash value+	vmovd		4*1($ctx),$H1+	vmovd		4*2($ctx),$H2+	vmovd		4*3($ctx),$H3+	vmovd		4*4($ctx),$H4++.Ldo_avx:+___+$code.=<<___	if (!$win64);+	lea		-0x58(%rsp),%r11+.cfi_def_cfa		%r11,0x60+	sub		\$0x178,%rsp+___+$code.=<<___	if ($win64);+	lea		-0xf8(%rsp),%r11+	sub		\$0x218,%rsp+	vmovdqa		%xmm6,0x50(%r11)+	vmovdqa		%xmm7,0x60(%r11)+	vmovdqa		%xmm8,0x70(%r11)+	vmovdqa		%xmm9,0x80(%r11)+	vmovdqa		%xmm10,0x90(%r11)+	vmovdqa		%xmm11,0xa0(%r11)+	vmovdqa		%xmm12,0xb0(%r11)+	vmovdqa		%xmm13,0xc0(%r11)+	vmovdqa		%xmm14,0xd0(%r11)+	vmovdqa		%xmm15,0xe0(%r11)+.Ldo_avx_body:+___+$code.=<<___;+	sub		\$64,$len+	lea		-32($inp),%rax+	cmovc		%rax,$inp++	vmovdqu		`16*3`($ctx),$D4	# preload r0^2+	lea		`16*3+64`($ctx),$ctx	# size optimization+	lea		.Lconst(%rip),%rcx++	################################################################+	# load input+	vmovdqu		16*2($inp),$T0+	vmovdqu		16*3($inp),$T1+	vmovdqa		64(%rcx),$MASK		# .Lmask26++	vpsrldq		\$6,$T0,$T2		# splat input+	vpsrldq		\$6,$T1,$T3+	vpunpckhqdq	$T1,$T0,$T4		# 4+	vpunpcklqdq	$T1,$T0,$T0		# 0:1+	vpunpcklqdq	$T3,$T2,$T3		# 2:3++	vpsrlq		\$40,$T4,$T4		# 4+	vpsrlq		\$26,$T0,$T1+	vpand		$MASK,$T0,$T0		# 0+	vpsrlq		\$4,$T3,$T2+	vpand		$MASK,$T1,$T1		# 1+	vpsrlq		\$30,$T3,$T3+	vpand		$MASK,$T2,$T2		# 2+	vpand		$MASK,$T3,$T3		# 3+	vpor		32(%rcx),$T4,$T4	# padbit, yes, always++	jbe		.Lskip_loop_avx++	# expand and copy pre-calculated table to stack+	vmovdqu		`16*1-64`($ctx),$D1+	vmovdqu		`16*2-64`($ctx),$D2+	vpshufd		\$0xEE,$D4,$D3		# 34xx -> 3434+	vpshufd		\$0x44,$D4,$D0		# xx12 -> 1212+	vmovdqa		$D3,-0x90(%r11)+	vmovdqa		$D0,0x00(%rsp)+	vpshufd		\$0xEE,$D1,$D4+	vmovdqu		`16*3-64`($ctx),$D0+	vpshufd		\$0x44,$D1,$D1+	vmovdqa		$D4,-0x80(%r11)+	vmovdqa		$D1,0x10(%rsp)+	vpshufd		\$0xEE,$D2,$D3+	vmovdqu		`16*4-64`($ctx),$D1+	vpshufd		\$0x44,$D2,$D2+	vmovdqa		$D3,-0x70(%r11)+	vmovdqa		$D2,0x20(%rsp)+	vpshufd		\$0xEE,$D0,$D4+	vmovdqu		`16*5-64`($ctx),$D2+	vpshufd		\$0x44,$D0,$D0+	vmovdqa		$D4,-0x60(%r11)+	vmovdqa		$D0,0x30(%rsp)+	vpshufd		\$0xEE,$D1,$D3+	vmovdqu		`16*6-64`($ctx),$D0+	vpshufd		\$0x44,$D1,$D1+	vmovdqa		$D3,-0x50(%r11)+	vmovdqa		$D1,0x40(%rsp)+	vpshufd		\$0xEE,$D2,$D4+	vmovdqu		`16*7-64`($ctx),$D1+	vpshufd		\$0x44,$D2,$D2+	vmovdqa		$D4,-0x40(%r11)+	vmovdqa		$D2,0x50(%rsp)+	vpshufd		\$0xEE,$D0,$D3+	vmovdqu		`16*8-64`($ctx),$D2+	vpshufd		\$0x44,$D0,$D0+	vmovdqa		$D3,-0x30(%r11)+	vmovdqa		$D0,0x60(%rsp)+	vpshufd		\$0xEE,$D1,$D4+	vpshufd		\$0x44,$D1,$D1+	vmovdqa		$D4,-0x20(%r11)+	vmovdqa		$D1,0x70(%rsp)+	vpshufd		\$0xEE,$D2,$D3+	 vmovdqa	0x00(%rsp),$D4		# preload r0^2+	vpshufd		\$0x44,$D2,$D2+	vmovdqa		$D3,-0x10(%r11)+	vmovdqa		$D2,0x80(%rsp)++	jmp		.Loop_avx++.align	32+.Loop_avx:+	################################################################+	# ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2+	# ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^3+inp[7]*r+	#   \___________________/+	# ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2+inp[8])*r^2+	# ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^4+inp[7]*r^2+inp[9])*r+	#   \___________________/ \____________________/+	#+	# Note that we start with inp[2:3]*r^2. This is because it+	# doesn't depend on reduction in previous iteration.+	################################################################+	# d4 = h4*r0 + h3*r1   + h2*r2   + h1*r3   + h0*r4+	# d3 = h3*r0 + h2*r1   + h1*r2   + h0*r3   + h4*5*r4+	# d2 = h2*r0 + h1*r1   + h0*r2   + h4*5*r3 + h3*5*r4+	# d1 = h1*r0 + h0*r1   + h4*5*r2 + h3*5*r3 + h2*5*r4+	# d0 = h0*r0 + h4*5*r1 + h3*5*r2 + h2*5*r3 + h1*5*r4+	#+	# though note that $Tx and $Hx are "reversed" in this section,+	# and $D4 is preloaded with r0^2...++	vpmuludq	$T0,$D4,$D0		# d0 = h0*r0+	vpmuludq	$T1,$D4,$D1		# d1 = h1*r0+	  vmovdqa	$H2,0x20(%r11)				# offload hash+	vpmuludq	$T2,$D4,$D2		# d3 = h2*r0+	 vmovdqa	0x10(%rsp),$H2		# r1^2+	vpmuludq	$T3,$D4,$D3		# d3 = h3*r0+	vpmuludq	$T4,$D4,$D4		# d4 = h4*r0++	  vmovdqa	$H0,0x00(%r11)				#+	vpmuludq	0x20(%rsp),$T4,$H0	# h4*s1+	  vmovdqa	$H1,0x10(%r11)				#+	vpmuludq	$T3,$H2,$H1		# h3*r1+	vpaddq		$H0,$D0,$D0		# d0 += h4*s1+	vpaddq		$H1,$D4,$D4		# d4 += h3*r1+	  vmovdqa	$H3,0x30(%r11)				#+	vpmuludq	$T2,$H2,$H0		# h2*r1+	vpmuludq	$T1,$H2,$H1		# h1*r1+	vpaddq		$H0,$D3,$D3		# d3 += h2*r1+	 vmovdqa	0x30(%rsp),$H3		# r2^2+	vpaddq		$H1,$D2,$D2		# d2 += h1*r1+	  vmovdqa	$H4,0x40(%r11)				#+	vpmuludq	$T0,$H2,$H2		# h0*r1+	 vpmuludq	$T2,$H3,$H0		# h2*r2+	vpaddq		$H2,$D1,$D1		# d1 += h0*r1++	 vmovdqa	0x40(%rsp),$H4		# s2^2+	vpaddq		$H0,$D4,$D4		# d4 += h2*r2+	vpmuludq	$T1,$H3,$H1		# h1*r2+	vpmuludq	$T0,$H3,$H3		# h0*r2+	vpaddq		$H1,$D3,$D3		# d3 += h1*r2+	 vmovdqa	0x50(%rsp),$H2		# r3^2+	vpaddq		$H3,$D2,$D2		# d2 += h0*r2+	vpmuludq	$T4,$H4,$H0		# h4*s2+	vpmuludq	$T3,$H4,$H4		# h3*s2+	vpaddq		$H0,$D1,$D1		# d1 += h4*s2+	 vmovdqa	0x60(%rsp),$H3		# s3^2+	vpaddq		$H4,$D0,$D0		# d0 += h3*s2++	 vmovdqa	0x80(%rsp),$H4		# s4^2+	vpmuludq	$T1,$H2,$H1		# h1*r3+	vpmuludq	$T0,$H2,$H2		# h0*r3+	vpaddq		$H1,$D4,$D4		# d4 += h1*r3+	vpaddq		$H2,$D3,$D3		# d3 += h0*r3+	vpmuludq	$T4,$H3,$H0		# h4*s3+	vpmuludq	$T3,$H3,$H1		# h3*s3+	vpaddq		$H0,$D2,$D2		# d2 += h4*s3+	 vmovdqu	16*0($inp),$H0				# load input+	vpaddq		$H1,$D1,$D1		# d1 += h3*s3+	vpmuludq	$T2,$H3,$H3		# h2*s3+	 vpmuludq	$T2,$H4,$T2		# h2*s4+	vpaddq		$H3,$D0,$D0		# d0 += h2*s3++	 vmovdqu	16*1($inp),$H1				#+	vpaddq		$T2,$D1,$D1		# d1 += h2*s4+	vpmuludq	$T3,$H4,$T3		# h3*s4+	vpmuludq	$T4,$H4,$T4		# h4*s4+	 vpsrldq	\$6,$H0,$H2				# splat input+	vpaddq		$T3,$D2,$D2		# d2 += h3*s4+	vpaddq		$T4,$D3,$D3		# d3 += h4*s4+	 vpsrldq	\$6,$H1,$H3				#+	vpmuludq	0x70(%rsp),$T0,$T4	# h0*r4+	vpmuludq	$T1,$H4,$T0		# h1*s4+	 vpunpckhqdq	$H1,$H0,$H4		# 4+	vpaddq		$T4,$D4,$D4		# d4 += h0*r4+	 vmovdqa	-0x90(%r11),$T4		# r0^4+	vpaddq		$T0,$D0,$D0		# d0 += h1*s4++	vpunpcklqdq	$H1,$H0,$H0		# 0:1+	vpunpcklqdq	$H3,$H2,$H3		# 2:3++	#vpsrlq		\$40,$H4,$H4		# 4+	vpsrldq		\$`40/8`,$H4,$H4	# 4+	vpsrlq		\$26,$H0,$H1+	vpand		$MASK,$H0,$H0		# 0+	vpsrlq		\$4,$H3,$H2+	vpand		$MASK,$H1,$H1		# 1+	vpand		0(%rcx),$H4,$H4		# .Lmask24+	vpsrlq		\$30,$H3,$H3+	vpand		$MASK,$H2,$H2		# 2+	vpand		$MASK,$H3,$H3		# 3+	vpor		32(%rcx),$H4,$H4	# padbit, yes, always++	vpaddq		0x00(%r11),$H0,$H0	# add hash value+	vpaddq		0x10(%r11),$H1,$H1+	vpaddq		0x20(%r11),$H2,$H2+	vpaddq		0x30(%r11),$H3,$H3+	vpaddq		0x40(%r11),$H4,$H4++	lea		16*2($inp),%rax+	lea		16*4($inp),$inp+	sub		\$64,$len+	cmovc		%rax,$inp++	################################################################+	# Now we accumulate (inp[0:1]+hash)*r^4+	################################################################+	# d4 = h4*r0 + h3*r1   + h2*r2   + h1*r3   + h0*r4+	# d3 = h3*r0 + h2*r1   + h1*r2   + h0*r3   + h4*5*r4+	# d2 = h2*r0 + h1*r1   + h0*r2   + h4*5*r3 + h3*5*r4+	# d1 = h1*r0 + h0*r1   + h4*5*r2 + h3*5*r3 + h2*5*r4+	# d0 = h0*r0 + h4*5*r1 + h3*5*r2 + h2*5*r3 + h1*5*r4++	vpmuludq	$H0,$T4,$T0		# h0*r0+	vpmuludq	$H1,$T4,$T1		# h1*r0+	vpaddq		$T0,$D0,$D0+	vpaddq		$T1,$D1,$D1+	 vmovdqa	-0x80(%r11),$T2		# r1^4+	vpmuludq	$H2,$T4,$T0		# h2*r0+	vpmuludq	$H3,$T4,$T1		# h3*r0+	vpaddq		$T0,$D2,$D2+	vpaddq		$T1,$D3,$D3+	vpmuludq	$H4,$T4,$T4		# h4*r0+	 vpmuludq	-0x70(%r11),$H4,$T0	# h4*s1+	vpaddq		$T4,$D4,$D4++	vpaddq		$T0,$D0,$D0		# d0 += h4*s1+	vpmuludq	$H2,$T2,$T1		# h2*r1+	vpmuludq	$H3,$T2,$T0		# h3*r1+	vpaddq		$T1,$D3,$D3		# d3 += h2*r1+	 vmovdqa	-0x60(%r11),$T3		# r2^4+	vpaddq		$T0,$D4,$D4		# d4 += h3*r1+	vpmuludq	$H1,$T2,$T1		# h1*r1+	vpmuludq	$H0,$T2,$T2		# h0*r1+	vpaddq		$T1,$D2,$D2		# d2 += h1*r1+	vpaddq		$T2,$D1,$D1		# d1 += h0*r1++	 vmovdqa	-0x50(%r11),$T4		# s2^4+	vpmuludq	$H2,$T3,$T0		# h2*r2+	vpmuludq	$H1,$T3,$T1		# h1*r2+	vpaddq		$T0,$D4,$D4		# d4 += h2*r2+	vpaddq		$T1,$D3,$D3		# d3 += h1*r2+	 vmovdqa	-0x40(%r11),$T2		# r3^4+	vpmuludq	$H0,$T3,$T3		# h0*r2+	vpmuludq	$H4,$T4,$T0		# h4*s2+	vpaddq		$T3,$D2,$D2		# d2 += h0*r2+	vpaddq		$T0,$D1,$D1		# d1 += h4*s2+	 vmovdqa	-0x30(%r11),$T3		# s3^4+	vpmuludq	$H3,$T4,$T4		# h3*s2+	 vpmuludq	$H1,$T2,$T1		# h1*r3+	vpaddq		$T4,$D0,$D0		# d0 += h3*s2++	 vmovdqa	-0x10(%r11),$T4		# s4^4+	vpaddq		$T1,$D4,$D4		# d4 += h1*r3+	vpmuludq	$H0,$T2,$T2		# h0*r3+	vpmuludq	$H4,$T3,$T0		# h4*s3+	vpaddq		$T2,$D3,$D3		# d3 += h0*r3+	vpaddq		$T0,$D2,$D2		# d2 += h4*s3+	 vmovdqu	16*2($inp),$T0				# load input+	vpmuludq	$H3,$T3,$T2		# h3*s3+	vpmuludq	$H2,$T3,$T3		# h2*s3+	vpaddq		$T2,$D1,$D1		# d1 += h3*s3+	 vmovdqu	16*3($inp),$T1				#+	vpaddq		$T3,$D0,$D0		# d0 += h2*s3++	vpmuludq	$H2,$T4,$H2		# h2*s4+	vpmuludq	$H3,$T4,$H3		# h3*s4+	 vpsrldq	\$6,$T0,$T2				# splat input+	vpaddq		$H2,$D1,$D1		# d1 += h2*s4+	vpmuludq	$H4,$T4,$H4		# h4*s4+	 vpsrldq	\$6,$T1,$T3				#+	vpaddq		$H3,$D2,$H2		# h2 = d2 + h3*s4+	vpaddq		$H4,$D3,$H3		# h3 = d3 + h4*s4+	vpmuludq	-0x20(%r11),$H0,$H4	# h0*r4+	vpmuludq	$H1,$T4,$H0+	 vpunpckhqdq	$T1,$T0,$T4		# 4+	vpaddq		$H4,$D4,$H4		# h4 = d4 + h0*r4+	vpaddq		$H0,$D0,$H0		# h0 = d0 + h1*s4++	vpunpcklqdq	$T1,$T0,$T0		# 0:1+	vpunpcklqdq	$T3,$T2,$T3		# 2:3++	#vpsrlq		\$40,$T4,$T4		# 4+	vpsrldq		\$`40/8`,$T4,$T4	# 4+	vpsrlq		\$26,$T0,$T1+	 vmovdqa	0x00(%rsp),$D4		# preload r0^2+	vpand		$MASK,$T0,$T0		# 0+	vpsrlq		\$4,$T3,$T2+	vpand		$MASK,$T1,$T1		# 1+	vpand		0(%rcx),$T4,$T4		# .Lmask24+	vpsrlq		\$30,$T3,$T3+	vpand		$MASK,$T2,$T2		# 2+	vpand		$MASK,$T3,$T3		# 3+	vpor		32(%rcx),$T4,$T4	# padbit, yes, always++	################################################################+	# lazy reduction as discussed in "NEON crypto" by D.J. Bernstein+	# and P. Schwabe++	vpsrlq		\$26,$H3,$D3+	vpand		$MASK,$H3,$H3+	vpaddq		$D3,$H4,$H4		# h3 -> h4++	vpsrlq		\$26,$H0,$D0+	vpand		$MASK,$H0,$H0+	vpaddq		$D0,$D1,$H1		# h0 -> h1++	vpsrlq		\$26,$H4,$D0+	vpand		$MASK,$H4,$H4++	vpsrlq		\$26,$H1,$D1+	vpand		$MASK,$H1,$H1+	vpaddq		$D1,$H2,$H2		# h1 -> h2++	vpaddq		$D0,$H0,$H0+	vpsllq		\$2,$D0,$D0+	vpaddq		$D0,$H0,$H0		# h4 -> h0++	vpsrlq		\$26,$H2,$D2+	vpand		$MASK,$H2,$H2+	vpaddq		$D2,$H3,$H3		# h2 -> h3++	vpsrlq		\$26,$H0,$D0+	vpand		$MASK,$H0,$H0+	vpaddq		$D0,$H1,$H1		# h0 -> h1++	vpsrlq		\$26,$H3,$D3+	vpand		$MASK,$H3,$H3+	vpaddq		$D3,$H4,$H4		# h3 -> h4++	ja		.Loop_avx++.Lskip_loop_avx:+	################################################################+	# multiply (inp[0:1]+hash) or inp[2:3] by r^2:r^1++	vpshufd		\$0x10,$D4,$D4		# r0^n, xx12 -> x1x2+	add		\$32,$len+	jnz		.Long_tail_avx++	vpaddq		$H2,$T2,$T2+	vpaddq		$H0,$T0,$T0+	vpaddq		$H1,$T1,$T1+	vpaddq		$H3,$T3,$T3+	vpaddq		$H4,$T4,$T4++.Long_tail_avx:+	vmovdqa		$H2,0x20(%r11)+	vmovdqa		$H0,0x00(%r11)+	vmovdqa		$H1,0x10(%r11)+	vmovdqa		$H3,0x30(%r11)+	vmovdqa		$H4,0x40(%r11)++	# d4 = h4*r0 + h3*r1   + h2*r2   + h1*r3   + h0*r4+	# d3 = h3*r0 + h2*r1   + h1*r2   + h0*r3   + h4*5*r4+	# d2 = h2*r0 + h1*r1   + h0*r2   + h4*5*r3 + h3*5*r4+	# d1 = h1*r0 + h0*r1   + h4*5*r2 + h3*5*r3 + h2*5*r4+	# d0 = h0*r0 + h4*5*r1 + h3*5*r2 + h2*5*r3 + h1*5*r4++	vpmuludq	$T2,$D4,$D2		# d2 = h2*r0+	vpmuludq	$T0,$D4,$D0		# d0 = h0*r0+	 vpshufd	\$0x10,`16*1-64`($ctx),$H2		# r1^n+	vpmuludq	$T1,$D4,$D1		# d1 = h1*r0+	vpmuludq	$T3,$D4,$D3		# d3 = h3*r0+	vpmuludq	$T4,$D4,$D4		# d4 = h4*r0++	vpmuludq	$T3,$H2,$H0		# h3*r1+	vpaddq		$H0,$D4,$D4		# d4 += h3*r1+	 vpshufd	\$0x10,`16*2-64`($ctx),$H3		# s1^n+	vpmuludq	$T2,$H2,$H1		# h2*r1+	vpaddq		$H1,$D3,$D3		# d3 += h2*r1+	 vpshufd	\$0x10,`16*3-64`($ctx),$H4		# r2^n+	vpmuludq	$T1,$H2,$H0		# h1*r1+	vpaddq		$H0,$D2,$D2		# d2 += h1*r1+	vpmuludq	$T0,$H2,$H2		# h0*r1+	vpaddq		$H2,$D1,$D1		# d1 += h0*r1+	vpmuludq	$T4,$H3,$H3		# h4*s1+	vpaddq		$H3,$D0,$D0		# d0 += h4*s1++	 vpshufd	\$0x10,`16*4-64`($ctx),$H2		# s2^n+	vpmuludq	$T2,$H4,$H1		# h2*r2+	vpaddq		$H1,$D4,$D4		# d4 += h2*r2+	vpmuludq	$T1,$H4,$H0		# h1*r2+	vpaddq		$H0,$D3,$D3		# d3 += h1*r2+	 vpshufd	\$0x10,`16*5-64`($ctx),$H3		# r3^n+	vpmuludq	$T0,$H4,$H4		# h0*r2+	vpaddq		$H4,$D2,$D2		# d2 += h0*r2+	vpmuludq	$T4,$H2,$H1		# h4*s2+	vpaddq		$H1,$D1,$D1		# d1 += h4*s2+	 vpshufd	\$0x10,`16*6-64`($ctx),$H4		# s3^n+	vpmuludq	$T3,$H2,$H2		# h3*s2+	vpaddq		$H2,$D0,$D0		# d0 += h3*s2++	vpmuludq	$T1,$H3,$H0		# h1*r3+	vpaddq		$H0,$D4,$D4		# d4 += h1*r3+	vpmuludq	$T0,$H3,$H3		# h0*r3+	vpaddq		$H3,$D3,$D3		# d3 += h0*r3+	 vpshufd	\$0x10,`16*7-64`($ctx),$H2		# r4^n+	vpmuludq	$T4,$H4,$H1		# h4*s3+	vpaddq		$H1,$D2,$D2		# d2 += h4*s3+	 vpshufd	\$0x10,`16*8-64`($ctx),$H3		# s4^n+	vpmuludq	$T3,$H4,$H0		# h3*s3+	vpaddq		$H0,$D1,$D1		# d1 += h3*s3+	vpmuludq	$T2,$H4,$H4		# h2*s3+	vpaddq		$H4,$D0,$D0		# d0 += h2*s3++	vpmuludq	$T0,$H2,$H2		# h0*r4+	vpaddq		$H2,$D4,$D4		# h4 = d4 + h0*r4+	vpmuludq	$T4,$H3,$H1		# h4*s4+	vpaddq		$H1,$D3,$D3		# h3 = d3 + h4*s4+	vpmuludq	$T3,$H3,$H0		# h3*s4+	vpaddq		$H0,$D2,$D2		# h2 = d2 + h3*s4+	vpmuludq	$T2,$H3,$H1		# h2*s4+	vpaddq		$H1,$D1,$D1		# h1 = d1 + h2*s4+	vpmuludq	$T1,$H3,$H3		# h1*s4+	vpaddq		$H3,$D0,$D0		# h0 = d0 + h1*s4++	jz		.Lshort_tail_avx++	vmovdqu		16*0($inp),$H0		# load input+	vmovdqu		16*1($inp),$H1++	vpsrldq		\$6,$H0,$H2		# splat input+	vpsrldq		\$6,$H1,$H3+	vpunpckhqdq	$H1,$H0,$H4		# 4+	vpunpcklqdq	$H1,$H0,$H0		# 0:1+	vpunpcklqdq	$H3,$H2,$H3		# 2:3++	vpsrlq		\$40,$H4,$H4		# 4+	vpsrlq		\$26,$H0,$H1+	vpand		$MASK,$H0,$H0		# 0+	vpsrlq		\$4,$H3,$H2+	vpand		$MASK,$H1,$H1		# 1+	vpsrlq		\$30,$H3,$H3+	vpand		$MASK,$H2,$H2		# 2+	vpand		$MASK,$H3,$H3		# 3+	vpor		32(%rcx),$H4,$H4	# padbit, yes, always++	vpshufd		\$0x32,`16*0-64`($ctx),$T4	# r0^n, 34xx -> x3x4+	vpaddq		0x00(%r11),$H0,$H0+	vpaddq		0x10(%r11),$H1,$H1+	vpaddq		0x20(%r11),$H2,$H2+	vpaddq		0x30(%r11),$H3,$H3+	vpaddq		0x40(%r11),$H4,$H4++	################################################################+	# multiply (inp[0:1]+hash) by r^4:r^3 and accumulate++	vpmuludq	$H0,$T4,$T0		# h0*r0+	vpaddq		$T0,$D0,$D0		# d0 += h0*r0+	vpmuludq	$H1,$T4,$T1		# h1*r0+	vpaddq		$T1,$D1,$D1		# d1 += h1*r0+	vpmuludq	$H2,$T4,$T0		# h2*r0+	vpaddq		$T0,$D2,$D2		# d2 += h2*r0+	 vpshufd	\$0x32,`16*1-64`($ctx),$T2		# r1^n+	vpmuludq	$H3,$T4,$T1		# h3*r0+	vpaddq		$T1,$D3,$D3		# d3 += h3*r0+	vpmuludq	$H4,$T4,$T4		# h4*r0+	vpaddq		$T4,$D4,$D4		# d4 += h4*r0++	vpmuludq	$H3,$T2,$T0		# h3*r1+	vpaddq		$T0,$D4,$D4		# d4 += h3*r1+	 vpshufd	\$0x32,`16*2-64`($ctx),$T3		# s1+	vpmuludq	$H2,$T2,$T1		# h2*r1+	vpaddq		$T1,$D3,$D3		# d3 += h2*r1+	 vpshufd	\$0x32,`16*3-64`($ctx),$T4		# r2+	vpmuludq	$H1,$T2,$T0		# h1*r1+	vpaddq		$T0,$D2,$D2		# d2 += h1*r1+	vpmuludq	$H0,$T2,$T2		# h0*r1+	vpaddq		$T2,$D1,$D1		# d1 += h0*r1+	vpmuludq	$H4,$T3,$T3		# h4*s1+	vpaddq		$T3,$D0,$D0		# d0 += h4*s1++	 vpshufd	\$0x32,`16*4-64`($ctx),$T2		# s2+	vpmuludq	$H2,$T4,$T1		# h2*r2+	vpaddq		$T1,$D4,$D4		# d4 += h2*r2+	vpmuludq	$H1,$T4,$T0		# h1*r2+	vpaddq		$T0,$D3,$D3		# d3 += h1*r2+	 vpshufd	\$0x32,`16*5-64`($ctx),$T3		# r3+	vpmuludq	$H0,$T4,$T4		# h0*r2+	vpaddq		$T4,$D2,$D2		# d2 += h0*r2+	vpmuludq	$H4,$T2,$T1		# h4*s2+	vpaddq		$T1,$D1,$D1		# d1 += h4*s2+	 vpshufd	\$0x32,`16*6-64`($ctx),$T4		# s3+	vpmuludq	$H3,$T2,$T2		# h3*s2+	vpaddq		$T2,$D0,$D0		# d0 += h3*s2++	vpmuludq	$H1,$T3,$T0		# h1*r3+	vpaddq		$T0,$D4,$D4		# d4 += h1*r3+	vpmuludq	$H0,$T3,$T3		# h0*r3+	vpaddq		$T3,$D3,$D3		# d3 += h0*r3+	 vpshufd	\$0x32,`16*7-64`($ctx),$T2		# r4+	vpmuludq	$H4,$T4,$T1		# h4*s3+	vpaddq		$T1,$D2,$D2		# d2 += h4*s3+	 vpshufd	\$0x32,`16*8-64`($ctx),$T3		# s4+	vpmuludq	$H3,$T4,$T0		# h3*s3+	vpaddq		$T0,$D1,$D1		# d1 += h3*s3+	vpmuludq	$H2,$T4,$T4		# h2*s3+	vpaddq		$T4,$D0,$D0		# d0 += h2*s3++	vpmuludq	$H0,$T2,$T2		# h0*r4+	vpaddq		$T2,$D4,$D4		# d4 += h0*r4+	vpmuludq	$H4,$T3,$T1		# h4*s4+	vpaddq		$T1,$D3,$D3		# d3 += h4*s4+	vpmuludq	$H3,$T3,$T0		# h3*s4+	vpaddq		$T0,$D2,$D2		# d2 += h3*s4+	vpmuludq	$H2,$T3,$T1		# h2*s4+	vpaddq		$T1,$D1,$D1		# d1 += h2*s4+	vpmuludq	$H1,$T3,$T3		# h1*s4+	vpaddq		$T3,$D0,$D0		# d0 += h1*s4++.Lshort_tail_avx:+	################################################################+	# horizontal addition++	vpsrldq		\$8,$D4,$T4+	vpsrldq		\$8,$D3,$T3+	vpsrldq		\$8,$D1,$T1+	vpsrldq		\$8,$D0,$T0+	vpsrldq		\$8,$D2,$T2+	vpaddq		$T3,$D3,$D3+	vpaddq		$T4,$D4,$D4+	vpaddq		$T0,$D0,$D0+	vpaddq		$T1,$D1,$D1+	vpaddq		$T2,$D2,$D2++	################################################################+	# lazy reduction++	vpsrlq		\$26,$D3,$H3+	vpand		$MASK,$D3,$D3+	vpaddq		$H3,$D4,$D4		# h3 -> h4++	vpsrlq		\$26,$D0,$H0+	vpand		$MASK,$D0,$D0+	vpaddq		$H0,$D1,$D1		# h0 -> h1++	vpsrlq		\$26,$D4,$H4+	vpand		$MASK,$D4,$D4++	vpsrlq		\$26,$D1,$H1+	vpand		$MASK,$D1,$D1+	vpaddq		$H1,$D2,$D2		# h1 -> h2++	vpaddq		$H4,$D0,$D0+	vpsllq		\$2,$H4,$H4+	vpaddq		$H4,$D0,$D0		# h4 -> h0++	vpsrlq		\$26,$D2,$H2+	vpand		$MASK,$D2,$D2+	vpaddq		$H2,$D3,$D3		# h2 -> h3++	vpsrlq		\$26,$D0,$H0+	vpand		$MASK,$D0,$D0+	vpaddq		$H0,$D1,$D1		# h0 -> h1++	vpsrlq		\$26,$D3,$H3+	vpand		$MASK,$D3,$D3+	vpaddq		$H3,$D4,$D4		# h3 -> h4++	vmovd		$D0,`4*0-48-64`($ctx)	# save partially reduced+	vmovd		$D1,`4*1-48-64`($ctx)+	vmovd		$D2,`4*2-48-64`($ctx)+	vmovd		$D3,`4*3-48-64`($ctx)+	vmovd		$D4,`4*4-48-64`($ctx)+___+$code.=<<___	if ($win64);+	vmovdqa		0x50(%r11),%xmm6+	vmovdqa		0x60(%r11),%xmm7+	vmovdqa		0x70(%r11),%xmm8+	vmovdqa		0x80(%r11),%xmm9+	vmovdqa		0x90(%r11),%xmm10+	vmovdqa		0xa0(%r11),%xmm11+	vmovdqa		0xb0(%r11),%xmm12+	vmovdqa		0xc0(%r11),%xmm13+	vmovdqa		0xd0(%r11),%xmm14+	vmovdqa		0xe0(%r11),%xmm15+	lea		0xf8(%r11),%rsp+.Ldo_avx_epilogue:+___+$code.=<<___	if (!$win64);+	lea		0x58(%r11),%rsp+.cfi_def_cfa		%rsp,8+___+$code.=<<___;+	vzeroupper+	ret+.cfi_endproc+.size	poly1305_blocks_avx,.-poly1305_blocks_avx+___++if ($avx>1) {+my ($H0,$H1,$H2,$H3,$H4, $MASK, $T4,$T0,$T1,$T2,$T3, $D0,$D1,$D2,$D3,$D4) =+    map("%ymm$_",(0..15));+my $S4=$MASK;++$code.=<<___;+.type	poly1305_blocks_avx2,\@function,4+.align	32+poly1305_blocks_avx2:+.cfi_startproc+	mov	20($ctx),%r8d		# load is_base2_26+	cmp	\$128,$len+	jb	.Lblocks++	and	\$-16,$len++	vzeroupper++	test	%r8d,%r8d		# is_base2_26?+	jz	.Lbase2_64_avx2++	test	\$63,$len+	jz	.Leven_avx2++	push	%rbx+.cfi_push	%rbx+	push	%rbp+.cfi_push	%rbp+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15+	lea	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+.Lblocks_avx2_body:++	mov	$len,%r15		# reassign $len++	mov	0($ctx),$d1		# load hash value+	mov	8($ctx),$d2+	mov	16($ctx),$h2#d++	mov	24($ctx),$r0		# load r+	mov	32($ctx),$s1++	################################# base 2^26 -> base 2^64+	mov	$d1#d,$h0#d+	and	\$`-1*(1<<31)`,$d1+	mov	$d2,$r1			# borrow $r1+	mov	$d2#d,$h1#d+	and	\$`-1*(1<<31)`,$d2++	shr	\$6,$d1+	shl	\$52,$r1+	add	$d1,$h0+	shr	\$12,$h1+	shr	\$18,$d2+	add	$r1,$h0+	adc	$d2,$h1++	mov	$h2,$d1+	shl	\$40,$d1+	shr	\$24,$h2+	add	$d1,$h1+	adc	\$0,$h2			# can be partially reduced...++	mov	$s1,$r1+	mov	$s1,%rax+	shr	\$2,$s1+	add	$r1,$s1			# s1 = r1 + (r1 >> 2)++.Lbase2_26_pre_avx2:+	add	0($inp),$h0		# accumulate input+	adc	8($inp),$h1+	lea	16($inp),$inp+	adc	$padbit,$h2+	sub	\$16,%r15++	call	__poly1305_block+	mov	$r1,%rax++	test	\$63,%r15+	jnz	.Lbase2_26_pre_avx2++	################################# base 2^64 -> base 2^26+	mov	$h0,%rax+	mov	$h0,%rdx+	shr	\$52,$h0+	mov	$h1,$r0+	mov	$h1,$r1+	shr	\$26,%rdx+	and	\$0x3ffffff,%rax	# h[0]+	shl	\$12,$r0+	and	\$0x3ffffff,%rdx	# h[1]+	shr	\$14,$h1+	or	$r0,$h0+	shl	\$24,$h2+	and	\$0x3ffffff,$h0		# h[2]+	shr	\$40,$r1+	and	\$0x3ffffff,$h1		# h[3]+	or	$r1,$h2			# h[4]++	vmovd	%rax#d,%x#$H0+	vmovd	%rdx#d,%x#$H1+	vmovd	$h0#d,%x#$H2+	vmovd	$h1#d,%x#$H3+	vmovd	$h2#d,%x#$H4++	mov	%r15,$len			# restore $len++	mov	8(%rsp),%r15+.cfi_restore	%r15+	mov	16(%rsp),%r14+.cfi_restore	%r14+	mov	24(%rsp),%r13+.cfi_restore	%r13+	mov	32(%rsp),%r12+.cfi_restore	%r12+	mov	40(%rsp),%rbp+.cfi_restore	%rbp+	mov	48(%rsp),%rbx+.cfi_restore	%rbx+	lea	56(%rsp),%rax			# for win64+	lea	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+.Lblocks_avx2_epilogue:+	jmp	.Ldo_avx2+.cfi_endproc++.align	32+.Lbase2_64_avx2:+.cfi_startproc+	push	%rbx+.cfi_push	%rbx+	push	%rbp+.cfi_push	%rbp+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15+	lea	-8(%rsp),%rsp+.cfi_adjust_cfa_offset	8+.Lbase2_64_avx2_body:++	mov	$len,%r15		# reassign $len++	mov	24($ctx),$r0		# load r+	mov	32($ctx),$s1++	mov	0($ctx),$h0		# load hash value+	mov	8($ctx),$h1+	mov	16($ctx),$h2#d++	mov	$s1,$r1+	mov	$s1,%rax+	shr	\$2,$s1+	add	$r1,$s1			# s1 = r1 + (r1 >> 2)++	test	\$63,$len+	jz	.Linit_avx2++.Lbase2_64_pre_avx2:+	add	0($inp),$h0		# accumulate input+	adc	8($inp),$h1+	lea	16($inp),$inp+	adc	$padbit,$h2+	sub	\$16,%r15++	call	__poly1305_block+	mov	$r1,%rax++	test	\$63,%r15+	jnz	.Lbase2_64_pre_avx2++.Linit_avx2:+	################################# base 2^64 -> base 2^26+	mov	$h0,%rax+	mov	$h0,%rdx+	shr	\$52,$h0+	mov	$h1,$d1+	mov	$h1,$d2+	shr	\$26,%rdx+	and	\$0x3ffffff,%rax	# h[0]+	shl	\$12,$d1+	and	\$0x3ffffff,%rdx	# h[1]+	shr	\$14,$h1+	or	$d1,$h0+	shl	\$24,$h2+	and	\$0x3ffffff,$h0		# h[2]+	shr	\$40,$d2+	and	\$0x3ffffff,$h1		# h[3]+	or	$d2,$h2			# h[4]++	vmovd	%rax#d,%x#$H0+	vmovd	%rdx#d,%x#$H1+	vmovd	$h0#d,%x#$H2+	vmovd	$h1#d,%x#$H3+	vmovd	$h2#d,%x#$H4+	movl	\$1,20($ctx)		# set is_base2_26++	call	__poly1305_init_avx++	mov	%r15,$len			# restore $len++	mov	8(%rsp),%r15+.cfi_restore	%r15+	mov	16(%rsp),%r14+.cfi_restore	%r14+	mov	24(%rsp),%r13+.cfi_restore	%r13+	mov	32(%rsp),%r12+.cfi_restore	%r12+	mov	40(%rsp),%rbp+.cfi_restore	%rbp+	mov	48(%rsp),%rbx+.cfi_restore	%rbx+	lea	56(%rsp),%rax			# for inw64+	lea	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+.Lbase2_64_avx2_epilogue:+	jmp	.Ldo_avx2+.cfi_endproc++.align	32+.Leven_avx2:+.cfi_startproc+	vmovd		4*0($ctx),%x#$H0	# load hash value base 2^26+	vmovd		4*1($ctx),%x#$H1+	vmovd		4*2($ctx),%x#$H2+	vmovd		4*3($ctx),%x#$H3+	vmovd		4*4($ctx),%x#$H4++.Ldo_avx2:+___+$code.=<<___		if ($avx>2 && $flavour !~ /kernel/);+	mov		OPENSSL_ia32cap_P+8(%rip),%r10d+	cmp		\$512,$len+	jb		.Lskip_avx512+	test		\$`1<<16`,%r10d		# check for AVX512F+	jnz		.Lblocks_avx512+.Lskip_avx512:+___+$code.=<<___	if (!$win64);+	lea		-8(%rsp),%r11+.cfi_def_cfa		%r11,16+	sub		\$0x128,%rsp+___+$code.=<<___	if ($win64);+	lea		-0xf8(%rsp),%r11+	sub		\$0x1c8,%rsp+	vmovdqa		%xmm6,0x50(%r11)+	vmovdqa		%xmm7,0x60(%r11)+	vmovdqa		%xmm8,0x70(%r11)+	vmovdqa		%xmm9,0x80(%r11)+	vmovdqa		%xmm10,0x90(%r11)+	vmovdqa		%xmm11,0xa0(%r11)+	vmovdqa		%xmm12,0xb0(%r11)+	vmovdqa		%xmm13,0xc0(%r11)+	vmovdqa		%xmm14,0xd0(%r11)+	vmovdqa		%xmm15,0xe0(%r11)+.Ldo_avx2_body:+___+$code.=<<___;+	lea		.Lconst(%rip),%rcx+	lea		48+64($ctx),$ctx	# size optimization+	vmovdqa		96(%rcx),$T0		# .Lpermd_avx2++	# expand and copy pre-calculated table to stack+	vmovdqu		`16*0-64`($ctx),%x#$T2+	and		\$-512,%rsp+	vmovdqu		`16*1-64`($ctx),%x#$T3+	vmovdqu		`16*2-64`($ctx),%x#$T4+	vmovdqu		`16*3-64`($ctx),%x#$D0+	vmovdqu		`16*4-64`($ctx),%x#$D1+	vmovdqu		`16*5-64`($ctx),%x#$D2+	lea		0x90(%rsp),%rax		# size optimization+	vmovdqu		`16*6-64`($ctx),%x#$D3+	vpermd		$T2,$T0,$T2		# 00003412 -> 14243444+	vmovdqu		`16*7-64`($ctx),%x#$D4+	vpermd		$T3,$T0,$T3+	vmovdqu		`16*8-64`($ctx),%x#$MASK+	vpermd		$T4,$T0,$T4+	vmovdqa		$T2,0x00(%rsp)+	vpermd		$D0,$T0,$D0+	vmovdqa		$T3,0x20-0x90(%rax)+	vpermd		$D1,$T0,$D1+	vmovdqa		$T4,0x40-0x90(%rax)+	vpermd		$D2,$T0,$D2+	vmovdqa		$D0,0x60-0x90(%rax)+	vpermd		$D3,$T0,$D3+	vmovdqa		$D1,0x80-0x90(%rax)+	vpermd		$D4,$T0,$D4+	vmovdqa		$D2,0xa0-0x90(%rax)+	vpermd		$MASK,$T0,$MASK+	vmovdqa		$D3,0xc0-0x90(%rax)+	vmovdqa		$D4,0xe0-0x90(%rax)+	vmovdqa		$MASK,0x100-0x90(%rax)+	vmovdqa		64(%rcx),$MASK		# .Lmask26++	################################################################+	# load input+	vmovdqu		16*0($inp),%x#$T0+	vmovdqu		16*1($inp),%x#$T1+	vinserti128	\$1,16*2($inp),$T0,$T0+	vinserti128	\$1,16*3($inp),$T1,$T1+	lea		16*4($inp),$inp++	vpsrldq		\$6,$T0,$T2		# splat input+	vpsrldq		\$6,$T1,$T3+	vpunpckhqdq	$T1,$T0,$T4		# 4+	vpunpcklqdq	$T3,$T2,$T2		# 2:3+	vpunpcklqdq	$T1,$T0,$T0		# 0:1++	vpsrlq		\$30,$T2,$T3+	vpsrlq		\$4,$T2,$T2+	vpsrlq		\$26,$T0,$T1+	vpsrlq		\$40,$T4,$T4		# 4+	vpand		$MASK,$T2,$T2		# 2+	vpand		$MASK,$T0,$T0		# 0+	vpand		$MASK,$T1,$T1		# 1+	vpand		$MASK,$T3,$T3		# 3+	vpor		32(%rcx),$T4,$T4	# padbit, yes, always++	vpaddq		$H2,$T2,$H2		# accumulate input+	sub		\$64,$len+	jz		.Ltail_avx2+	jmp		.Loop_avx2++.align	32+.Loop_avx2:+	################################################################+	# ((inp[0]*r^4+inp[4])*r^4+inp[ 8])*r^4+	# ((inp[1]*r^4+inp[5])*r^4+inp[ 9])*r^3+	# ((inp[2]*r^4+inp[6])*r^4+inp[10])*r^2+	# ((inp[3]*r^4+inp[7])*r^4+inp[11])*r^1+	#   \________/\__________/+	################################################################+	#vpaddq		$H2,$T2,$H2		# accumulate input+	vpaddq		$H0,$T0,$H0+	vmovdqa		`32*0`(%rsp),$T0	# r0^4+	vpaddq		$H1,$T1,$H1+	vmovdqa		`32*1`(%rsp),$T1	# r1^4+	vpaddq		$H3,$T3,$H3+	vmovdqa		`32*3`(%rsp),$T2	# r2^4+	vpaddq		$H4,$T4,$H4+	vmovdqa		`32*6-0x90`(%rax),$T3	# s3^4+	vmovdqa		`32*8-0x90`(%rax),$S4	# s4^4++	# d4 = h4*r0 + h3*r1   + h2*r2   + h1*r3   + h0*r4+	# d3 = h3*r0 + h2*r1   + h1*r2   + h0*r3   + h4*5*r4+	# d2 = h2*r0 + h1*r1   + h0*r2   + h4*5*r3 + h3*5*r4+	# d1 = h1*r0 + h0*r1   + h4*5*r2 + h3*5*r3 + h2*5*r4+	# d0 = h0*r0 + h4*5*r1 + h3*5*r2 + h2*5*r3 + h1*5*r4+	#+	# however, as h2 is "chronologically" first one available pull+	# corresponding operations up, so it's+	#+	# d4 = h2*r2   + h4*r0 + h3*r1             + h1*r3   + h0*r4+	# d3 = h2*r1   + h3*r0           + h1*r2   + h0*r3   + h4*5*r4+	# d2 = h2*r0           + h1*r1   + h0*r2   + h4*5*r3 + h3*5*r4+	# d1 = h2*5*r4 + h1*r0 + h0*r1   + h4*5*r2 + h3*5*r3+	# d0 = h2*5*r3 + h0*r0 + h4*5*r1 + h3*5*r2           + h1*5*r4++	vpmuludq	$H2,$T0,$D2		# d2 = h2*r0+	vpmuludq	$H2,$T1,$D3		# d3 = h2*r1+	vpmuludq	$H2,$T2,$D4		# d4 = h2*r2+	vpmuludq	$H2,$T3,$D0		# d0 = h2*s3+	vpmuludq	$H2,$S4,$D1		# d1 = h2*s4++	vpmuludq	$H0,$T1,$T4		# h0*r1+	vpmuludq	$H1,$T1,$H2		# h1*r1, borrow $H2 as temp+	vpaddq		$T4,$D1,$D1		# d1 += h0*r1+	vpaddq		$H2,$D2,$D2		# d2 += h1*r1+	vpmuludq	$H3,$T1,$T4		# h3*r1+	vpmuludq	`32*2`(%rsp),$H4,$H2	# h4*s1+	vpaddq		$T4,$D4,$D4		# d4 += h3*r1+	vpaddq		$H2,$D0,$D0		# d0 += h4*s1+	 vmovdqa	`32*4-0x90`(%rax),$T1	# s2++	vpmuludq	$H0,$T0,$T4		# h0*r0+	vpmuludq	$H1,$T0,$H2		# h1*r0+	vpaddq		$T4,$D0,$D0		# d0 += h0*r0+	vpaddq		$H2,$D1,$D1		# d1 += h1*r0+	vpmuludq	$H3,$T0,$T4		# h3*r0+	vpmuludq	$H4,$T0,$H2		# h4*r0+	 vmovdqu	16*0($inp),%x#$T0	# load input+	vpaddq		$T4,$D3,$D3		# d3 += h3*r0+	vpaddq		$H2,$D4,$D4		# d4 += h4*r0+	 vinserti128	\$1,16*2($inp),$T0,$T0++	vpmuludq	$H3,$T1,$T4		# h3*s2+	vpmuludq	$H4,$T1,$H2		# h4*s2+	 vmovdqu	16*1($inp),%x#$T1+	vpaddq		$T4,$D0,$D0		# d0 += h3*s2+	vpaddq		$H2,$D1,$D1		# d1 += h4*s2+	 vmovdqa	`32*5-0x90`(%rax),$H2	# r3+	vpmuludq	$H1,$T2,$T4		# h1*r2+	vpmuludq	$H0,$T2,$T2		# h0*r2+	vpaddq		$T4,$D3,$D3		# d3 += h1*r2+	vpaddq		$T2,$D2,$D2		# d2 += h0*r2+	 vinserti128	\$1,16*3($inp),$T1,$T1+	 lea		16*4($inp),$inp++	vpmuludq	$H1,$H2,$T4		# h1*r3+	vpmuludq	$H0,$H2,$H2		# h0*r3+	 vpsrldq	\$6,$T0,$T2		# splat input+	vpaddq		$T4,$D4,$D4		# d4 += h1*r3+	vpaddq		$H2,$D3,$D3		# d3 += h0*r3+	vpmuludq	$H3,$T3,$T4		# h3*s3+	vpmuludq	$H4,$T3,$H2		# h4*s3+	 vpsrldq	\$6,$T1,$T3+	vpaddq		$T4,$D1,$D1		# d1 += h3*s3+	vpaddq		$H2,$D2,$D2		# d2 += h4*s3+	 vpunpckhqdq	$T1,$T0,$T4		# 4++	vpmuludq	$H3,$S4,$H3		# h3*s4+	vpmuludq	$H4,$S4,$H4		# h4*s4+	 vpunpcklqdq	$T1,$T0,$T0		# 0:1+	vpaddq		$H3,$D2,$H2		# h2 = d2 + h3*r4+	vpaddq		$H4,$D3,$H3		# h3 = d3 + h4*r4+	 vpunpcklqdq	$T3,$T2,$T3		# 2:3+	vpmuludq	`32*7-0x90`(%rax),$H0,$H4	# h0*r4+	vpmuludq	$H1,$S4,$H0		# h1*s4+	vmovdqa		64(%rcx),$MASK		# .Lmask26+	vpaddq		$H4,$D4,$H4		# h4 = d4 + h0*r4+	vpaddq		$H0,$D0,$H0		# h0 = d0 + h1*s4++	################################################################+	# lazy reduction (interleaved with tail of input splat)++	vpsrlq		\$26,$H3,$D3+	vpand		$MASK,$H3,$H3+	vpaddq		$D3,$H4,$H4		# h3 -> h4++	vpsrlq		\$26,$H0,$D0+	vpand		$MASK,$H0,$H0+	vpaddq		$D0,$D1,$H1		# h0 -> h1++	vpsrlq		\$26,$H4,$D4+	vpand		$MASK,$H4,$H4++	 vpsrlq		\$4,$T3,$T2++	vpsrlq		\$26,$H1,$D1+	vpand		$MASK,$H1,$H1+	vpaddq		$D1,$H2,$H2		# h1 -> h2++	vpaddq		$D4,$H0,$H0+	vpsllq		\$2,$D4,$D4+	vpaddq		$D4,$H0,$H0		# h4 -> h0++	 vpand		$MASK,$T2,$T2		# 2+	 vpsrlq		\$26,$T0,$T1++	vpsrlq		\$26,$H2,$D2+	vpand		$MASK,$H2,$H2+	vpaddq		$D2,$H3,$H3		# h2 -> h3++	 vpaddq		$T2,$H2,$H2		# modulo-scheduled+	 vpsrlq		\$30,$T3,$T3++	vpsrlq		\$26,$H0,$D0+	vpand		$MASK,$H0,$H0+	vpaddq		$D0,$H1,$H1		# h0 -> h1++	 vpsrlq		\$40,$T4,$T4		# 4++	vpsrlq		\$26,$H3,$D3+	vpand		$MASK,$H3,$H3+	vpaddq		$D3,$H4,$H4		# h3 -> h4++	 vpand		$MASK,$T0,$T0		# 0+	 vpand		$MASK,$T1,$T1		# 1+	 vpand		$MASK,$T3,$T3		# 3+	 vpor		32(%rcx),$T4,$T4	# padbit, yes, always++	sub		\$64,$len+	jnz		.Loop_avx2++	.byte		0x66,0x90+.Ltail_avx2:+	################################################################+	# while above multiplications were by r^4 in all lanes, in last+	# iteration we multiply least significant lane by r^4 and most+	# significant one by r, so copy of above except that references+	# to the precomputed table are displaced by 4...++	#vpaddq		$H2,$T2,$H2		# accumulate input+	vpaddq		$H0,$T0,$H0+	vmovdqu		`32*0+4`(%rsp),$T0	# r0^4+	vpaddq		$H1,$T1,$H1+	vmovdqu		`32*1+4`(%rsp),$T1	# r1^4+	vpaddq		$H3,$T3,$H3+	vmovdqu		`32*3+4`(%rsp),$T2	# r2^4+	vpaddq		$H4,$T4,$H4+	vmovdqu		`32*6+4-0x90`(%rax),$T3	# s3^4+	vmovdqu		`32*8+4-0x90`(%rax),$S4	# s4^4++	vpmuludq	$H2,$T0,$D2		# d2 = h2*r0+	vpmuludq	$H2,$T1,$D3		# d3 = h2*r1+	vpmuludq	$H2,$T2,$D4		# d4 = h2*r2+	vpmuludq	$H2,$T3,$D0		# d0 = h2*s3+	vpmuludq	$H2,$S4,$D1		# d1 = h2*s4++	vpmuludq	$H0,$T1,$T4		# h0*r1+	vpmuludq	$H1,$T1,$H2		# h1*r1+	vpaddq		$T4,$D1,$D1		# d1 += h0*r1+	vpaddq		$H2,$D2,$D2		# d2 += h1*r1+	vpmuludq	$H3,$T1,$T4		# h3*r1+	vpmuludq	`32*2+4`(%rsp),$H4,$H2	# h4*s1+	vpaddq		$T4,$D4,$D4		# d4 += h3*r1+	vpaddq		$H2,$D0,$D0		# d0 += h4*s1++	vpmuludq	$H0,$T0,$T4		# h0*r0+	vpmuludq	$H1,$T0,$H2		# h1*r0+	vpaddq		$T4,$D0,$D0		# d0 += h0*r0+	 vmovdqu	`32*4+4-0x90`(%rax),$T1	# s2+	vpaddq		$H2,$D1,$D1		# d1 += h1*r0+	vpmuludq	$H3,$T0,$T4		# h3*r0+	vpmuludq	$H4,$T0,$H2		# h4*r0+	vpaddq		$T4,$D3,$D3		# d3 += h3*r0+	vpaddq		$H2,$D4,$D4		# d4 += h4*r0++	vpmuludq	$H3,$T1,$T4		# h3*s2+	vpmuludq	$H4,$T1,$H2		# h4*s2+	vpaddq		$T4,$D0,$D0		# d0 += h3*s2+	vpaddq		$H2,$D1,$D1		# d1 += h4*s2+	 vmovdqu	`32*5+4-0x90`(%rax),$H2	# r3+	vpmuludq	$H1,$T2,$T4		# h1*r2+	vpmuludq	$H0,$T2,$T2		# h0*r2+	vpaddq		$T4,$D3,$D3		# d3 += h1*r2+	vpaddq		$T2,$D2,$D2		# d2 += h0*r2++	vpmuludq	$H1,$H2,$T4		# h1*r3+	vpmuludq	$H0,$H2,$H2		# h0*r3+	vpaddq		$T4,$D4,$D4		# d4 += h1*r3+	vpaddq		$H2,$D3,$D3		# d3 += h0*r3+	vpmuludq	$H3,$T3,$T4		# h3*s3+	vpmuludq	$H4,$T3,$H2		# h4*s3+	vpaddq		$T4,$D1,$D1		# d1 += h3*s3+	vpaddq		$H2,$D2,$D2		# d2 += h4*s3++	vpmuludq	$H3,$S4,$H3		# h3*s4+	vpmuludq	$H4,$S4,$H4		# h4*s4+	vpaddq		$H3,$D2,$H2		# h2 = d2 + h3*r4+	vpaddq		$H4,$D3,$H3		# h3 = d3 + h4*r4+	vpmuludq	`32*7+4-0x90`(%rax),$H0,$H4		# h0*r4+	vpmuludq	$H1,$S4,$H0		# h1*s4+	vmovdqa		64(%rcx),$MASK		# .Lmask26+	vpaddq		$H4,$D4,$H4		# h4 = d4 + h0*r4+	vpaddq		$H0,$D0,$H0		# h0 = d0 + h1*s4++	################################################################+	# horizontal addition++	vpsrldq		\$8,$D1,$T1+	vpsrldq		\$8,$H2,$T2+	vpsrldq		\$8,$H3,$T3+	vpsrldq		\$8,$H4,$T4+	vpsrldq		\$8,$H0,$T0+	vpaddq		$T1,$D1,$D1+	vpaddq		$T2,$H2,$H2+	vpaddq		$T3,$H3,$H3+	vpaddq		$T4,$H4,$H4+	vpaddq		$T0,$H0,$H0++	vpermq		\$0x2,$H3,$T3+	vpermq		\$0x2,$H4,$T4+	vpermq		\$0x2,$H0,$T0+	vpermq		\$0x2,$D1,$T1+	vpermq		\$0x2,$H2,$T2+	vpaddq		$T3,$H3,$H3+	vpaddq		$T4,$H4,$H4+	vpaddq		$T0,$H0,$H0+	vpaddq		$T1,$D1,$D1+	vpaddq		$T2,$H2,$H2++	################################################################+	# lazy reduction++	vpsrlq		\$26,$H3,$D3+	vpand		$MASK,$H3,$H3+	vpaddq		$D3,$H4,$H4		# h3 -> h4++	vpsrlq		\$26,$H0,$D0+	vpand		$MASK,$H0,$H0+	vpaddq		$D0,$D1,$H1		# h0 -> h1++	vpsrlq		\$26,$H4,$D4+	vpand		$MASK,$H4,$H4++	vpsrlq		\$26,$H1,$D1+	vpand		$MASK,$H1,$H1+	vpaddq		$D1,$H2,$H2		# h1 -> h2++	vpaddq		$D4,$H0,$H0+	vpsllq		\$2,$D4,$D4+	vpaddq		$D4,$H0,$H0		# h4 -> h0++	vpsrlq		\$26,$H2,$D2+	vpand		$MASK,$H2,$H2+	vpaddq		$D2,$H3,$H3		# h2 -> h3++	vpsrlq		\$26,$H0,$D0+	vpand		$MASK,$H0,$H0+	vpaddq		$D0,$H1,$H1		# h0 -> h1++	vpsrlq		\$26,$H3,$D3+	vpand		$MASK,$H3,$H3+	vpaddq		$D3,$H4,$H4		# h3 -> h4++	vmovd		%x#$H0,`4*0-48-64`($ctx)# save partially reduced+	vmovd		%x#$H1,`4*1-48-64`($ctx)+	vmovd		%x#$H2,`4*2-48-64`($ctx)+	vmovd		%x#$H3,`4*3-48-64`($ctx)+	vmovd		%x#$H4,`4*4-48-64`($ctx)+___+$code.=<<___	if ($win64);+	vmovdqa		0x50(%r11),%xmm6+	vmovdqa		0x60(%r11),%xmm7+	vmovdqa		0x70(%r11),%xmm8+	vmovdqa		0x80(%r11),%xmm9+	vmovdqa		0x90(%r11),%xmm10+	vmovdqa		0xa0(%r11),%xmm11+	vmovdqa		0xb0(%r11),%xmm12+	vmovdqa		0xc0(%r11),%xmm13+	vmovdqa		0xd0(%r11),%xmm14+	vmovdqa		0xe0(%r11),%xmm15+	lea		0xf8(%r11),%rsp+.Ldo_avx2_epilogue:+___+$code.=<<___	if (!$win64);+	lea		8(%r11),%rsp+.cfi_def_cfa		%rsp,8+___+$code.=<<___;+	vzeroupper+	ret+.cfi_endproc+.size	poly1305_blocks_avx2,.-poly1305_blocks_avx2+___+#######################################################################+if ($avx>2 && $flavour !~ /kernel/) {+# On entry we have input length divisible by 64. But since inner loop+# processes 128 bytes per iteration, cases when length is not divisible+# by 128 are handled by passing tail 64 bytes to .Ltail_avx2. For this+# reason stack layout is kept identical to poly1305_blocks_avx2. If not+# for this tail, we wouldn't have to even allocate stack frame...++my ($R0,$R1,$R2,$R3,$R4, $S1,$S2,$S3,$S4) = map("%zmm$_",(16..24));+my ($M0,$M1,$M2,$M3,$M4) = map("%zmm$_",(25..29));+my $PADBIT="%zmm30";++map(s/%y/%z/,($T4,$T0,$T1,$T2,$T3));		# switch to %zmm domain+map(s/%y/%z/,($D0,$D1,$D2,$D3,$D4));+map(s/%y/%z/,($H0,$H1,$H2,$H3,$H4));+map(s/%y/%z/,($MASK));++$code.=<<___;+.type	poly1305_blocks_avx512,\@function,4+.align	32+poly1305_blocks_avx512:+.cfi_startproc+.Lblocks_avx512:+	mov		\$15,%eax+	kmovw		%eax,%k2+___+$code.=<<___	if (!$win64);+	lea		-8(%rsp),%r11+.cfi_def_cfa		%r11,16+	sub		\$0x128,%rsp+___+$code.=<<___	if ($win64);+	lea		-0xf8(%rsp),%r11+	sub		\$0x1c8,%rsp+	vmovdqa		%xmm6,0x50(%r11)+	vmovdqa		%xmm7,0x60(%r11)+	vmovdqa		%xmm8,0x70(%r11)+	vmovdqa		%xmm9,0x80(%r11)+	vmovdqa		%xmm10,0x90(%r11)+	vmovdqa		%xmm11,0xa0(%r11)+	vmovdqa		%xmm12,0xb0(%r11)+	vmovdqa		%xmm13,0xc0(%r11)+	vmovdqa		%xmm14,0xd0(%r11)+	vmovdqa		%xmm15,0xe0(%r11)+.Ldo_avx512_body:+___+$code.=<<___;+	lea		.Lconst(%rip),%rcx+	lea		48+64($ctx),$ctx	# size optimization+	vmovdqa		96(%rcx),%y#$T2		# .Lpermd_avx2++	# expand pre-calculated table+	vmovdqu		`16*0-64`($ctx),%x#$D0	# will become expanded ${R0}+	and		\$-512,%rsp+	vmovdqu		`16*1-64`($ctx),%x#$D1	# will become ... ${R1}+	mov		\$0x20,%rax+	vmovdqu		`16*2-64`($ctx),%x#$T0	# ... ${S1}+	vmovdqu		`16*3-64`($ctx),%x#$D2	# ... ${R2}+	vmovdqu		`16*4-64`($ctx),%x#$T1	# ... ${S2}+	vmovdqu		`16*5-64`($ctx),%x#$D3	# ... ${R3}+	vmovdqu		`16*6-64`($ctx),%x#$T3	# ... ${S3}+	vmovdqu		`16*7-64`($ctx),%x#$D4	# ... ${R4}+	vmovdqu		`16*8-64`($ctx),%x#$T4	# ... ${S4}+	vpermd		$D0,$T2,$R0		# 00003412 -> 14243444+	vpbroadcastq	64(%rcx),$MASK		# .Lmask26+	vpermd		$D1,$T2,$R1+	vpermd		$T0,$T2,$S1+	vpermd		$D2,$T2,$R2+	vmovdqa64	$R0,0x00(%rsp){%k2}	# save in case $len%128 != 0+	 vpsrlq		\$32,$R0,$T0		# 14243444 -> 01020304+	vpermd		$T1,$T2,$S2+	vmovdqu64	$R1,0x00(%rsp,%rax){%k2}+	 vpsrlq		\$32,$R1,$T1+	vpermd		$D3,$T2,$R3+	vmovdqa64	$S1,0x40(%rsp){%k2}+	vpermd		$T3,$T2,$S3+	vpermd		$D4,$T2,$R4+	vmovdqu64	$R2,0x40(%rsp,%rax){%k2}+	vpermd		$T4,$T2,$S4+	vmovdqa64	$S2,0x80(%rsp){%k2}+	vmovdqu64	$R3,0x80(%rsp,%rax){%k2}+	vmovdqa64	$S3,0xc0(%rsp){%k2}+	vmovdqu64	$R4,0xc0(%rsp,%rax){%k2}+	vmovdqa64	$S4,0x100(%rsp){%k2}++	################################################################+	# calculate 5th through 8th powers of the key+	#+	# d0 = r0'*r0 + r1'*5*r4 + r2'*5*r3 + r3'*5*r2 + r4'*5*r1+	# d1 = r0'*r1 + r1'*r0   + r2'*5*r4 + r3'*5*r3 + r4'*5*r2+	# d2 = r0'*r2 + r1'*r1   + r2'*r0   + r3'*5*r4 + r4'*5*r3+	# d3 = r0'*r3 + r1'*r2   + r2'*r1   + r3'*r0   + r4'*5*r4+	# d4 = r0'*r4 + r1'*r3   + r2'*r2   + r3'*r1   + r4'*r0++	vpmuludq	$T0,$R0,$D0		# d0 = r0'*r0+	vpmuludq	$T0,$R1,$D1		# d1 = r0'*r1+	vpmuludq	$T0,$R2,$D2		# d2 = r0'*r2+	vpmuludq	$T0,$R3,$D3		# d3 = r0'*r3+	vpmuludq	$T0,$R4,$D4		# d4 = r0'*r4+	 vpsrlq		\$32,$R2,$T2++	vpmuludq	$T1,$S4,$M0+	vpmuludq	$T1,$R0,$M1+	vpmuludq	$T1,$R1,$M2+	vpmuludq	$T1,$R2,$M3+	vpmuludq	$T1,$R3,$M4+	 vpsrlq		\$32,$R3,$T3+	vpaddq		$M0,$D0,$D0		# d0 += r1'*5*r4+	vpaddq		$M1,$D1,$D1		# d1 += r1'*r0+	vpaddq		$M2,$D2,$D2		# d2 += r1'*r1+	vpaddq		$M3,$D3,$D3		# d3 += r1'*r2+	vpaddq		$M4,$D4,$D4		# d4 += r1'*r3++	vpmuludq	$T2,$S3,$M0+	vpmuludq	$T2,$S4,$M1+	vpmuludq	$T2,$R1,$M3+	vpmuludq	$T2,$R2,$M4+	vpmuludq	$T2,$R0,$M2+	 vpsrlq		\$32,$R4,$T4+	vpaddq		$M0,$D0,$D0		# d0 += r2'*5*r3+	vpaddq		$M1,$D1,$D1		# d1 += r2'*5*r4+	vpaddq		$M3,$D3,$D3		# d3 += r2'*r1+	vpaddq		$M4,$D4,$D4		# d4 += r2'*r2+	vpaddq		$M2,$D2,$D2		# d2 += r2'*r0++	vpmuludq	$T3,$S2,$M0+	vpmuludq	$T3,$R0,$M3+	vpmuludq	$T3,$R1,$M4+	vpmuludq	$T3,$S3,$M1+	vpmuludq	$T3,$S4,$M2+	vpaddq		$M0,$D0,$D0		# d0 += r3'*5*r2+	vpaddq		$M3,$D3,$D3		# d3 += r3'*r0+	vpaddq		$M4,$D4,$D4		# d4 += r3'*r1+	vpaddq		$M1,$D1,$D1		# d1 += r3'*5*r3+	vpaddq		$M2,$D2,$D2		# d2 += r3'*5*r4++	vpmuludq	$T4,$S4,$M3+	vpmuludq	$T4,$R0,$M4+	vpmuludq	$T4,$S1,$M0+	vpmuludq	$T4,$S2,$M1+	vpmuludq	$T4,$S3,$M2+	vpaddq		$M3,$D3,$D3		# d3 += r2'*5*r4+	vpaddq		$M4,$D4,$D4		# d4 += r2'*r0+	vpaddq		$M0,$D0,$D0		# d0 += r2'*5*r1+	vpaddq		$M1,$D1,$D1		# d1 += r2'*5*r2+	vpaddq		$M2,$D2,$D2		# d2 += r2'*5*r3++	################################################################+	# load input+	vmovdqu64	16*0($inp),%z#$T3+	vmovdqu64	16*4($inp),%z#$T4+	lea		16*8($inp),$inp++	################################################################+	# lazy reduction++	vpsrlq		\$26,$D3,$M3+	vpandq		$MASK,$D3,$D3+	vpaddq		$M3,$D4,$D4		# d3 -> d4++	vpsrlq		\$26,$D0,$M0+	vpandq		$MASK,$D0,$D0+	vpaddq		$M0,$D1,$D1		# d0 -> d1++	vpsrlq		\$26,$D4,$M4+	vpandq		$MASK,$D4,$D4++	vpsrlq		\$26,$D1,$M1+	vpandq		$MASK,$D1,$D1+	vpaddq		$M1,$D2,$D2		# d1 -> d2++	vpaddq		$M4,$D0,$D0+	vpsllq		\$2,$M4,$M4+	vpaddq		$M4,$D0,$D0		# d4 -> d0++	vpsrlq		\$26,$D2,$M2+	vpandq		$MASK,$D2,$D2+	vpaddq		$M2,$D3,$D3		# d2 -> d3++	vpsrlq		\$26,$D0,$M0+	vpandq		$MASK,$D0,$D0+	vpaddq		$M0,$D1,$D1		# d0 -> d1++	vpsrlq		\$26,$D3,$M3+	vpandq		$MASK,$D3,$D3+	vpaddq		$M3,$D4,$D4		# d3 -> d4++	################################################################+	# at this point we have 14243444 in $R0-$S4 and 05060708 in+	# $D0-$D4, ...++	vpunpcklqdq	$T4,$T3,$T0	# transpose input+	vpunpckhqdq	$T4,$T3,$T4++	# ... since input 64-bit lanes are ordered as 73625140, we could+	# "vperm" it to 76543210 (here and in each loop iteration), *or*+	# we could just flow along, hence the goal for $R0-$S4 is+	# 1858286838784888 ...++	vmovdqa32	128(%rcx),$M0		# .Lpermd_avx512:+	mov		\$0x7777,%eax+	kmovw		%eax,%k1++	vpermd		$R0,$M0,$R0		# 14243444 -> 1---2---3---4---+	vpermd		$R1,$M0,$R1+	vpermd		$R2,$M0,$R2+	vpermd		$R3,$M0,$R3+	vpermd		$R4,$M0,$R4++	vpermd		$D0,$M0,${R0}{%k1}	# 05060708 -> 1858286838784888+	vpermd		$D1,$M0,${R1}{%k1}+	vpermd		$D2,$M0,${R2}{%k1}+	vpermd		$D3,$M0,${R3}{%k1}+	vpermd		$D4,$M0,${R4}{%k1}++	vpslld		\$2,$R1,$S1		# *5+	vpslld		\$2,$R2,$S2+	vpslld		\$2,$R3,$S3+	vpslld		\$2,$R4,$S4+	vpaddd		$R1,$S1,$S1+	vpaddd		$R2,$S2,$S2+	vpaddd		$R3,$S3,$S3+	vpaddd		$R4,$S4,$S4++	vpbroadcastq	32(%rcx),$PADBIT	# .L129++	vpsrlq		\$52,$T0,$T2		# splat input+	vpsllq		\$12,$T4,$T3+	vporq		$T3,$T2,$T2+	vpsrlq		\$26,$T0,$T1+	vpsrlq		\$14,$T4,$T3+	vpsrlq		\$40,$T4,$T4		# 4+	vpandq		$MASK,$T2,$T2		# 2+	vpandq		$MASK,$T0,$T0		# 0+	#vpandq		$MASK,$T1,$T1		# 1+	#vpandq		$MASK,$T3,$T3		# 3+	#vporq		$PADBIT,$T4,$T4		# padbit, yes, always++	vpaddq		$H2,$T2,$H2		# accumulate input+	sub		\$192,$len+	jbe		.Ltail_avx512+	jmp		.Loop_avx512++.align	32+.Loop_avx512:+	################################################################+	# ((inp[0]*r^8+inp[ 8])*r^8+inp[16])*r^8+	# ((inp[1]*r^8+inp[ 9])*r^8+inp[17])*r^7+	# ((inp[2]*r^8+inp[10])*r^8+inp[18])*r^6+	# ((inp[3]*r^8+inp[11])*r^8+inp[19])*r^5+	# ((inp[4]*r^8+inp[12])*r^8+inp[20])*r^4+	# ((inp[5]*r^8+inp[13])*r^8+inp[21])*r^3+	# ((inp[6]*r^8+inp[14])*r^8+inp[22])*r^2+	# ((inp[7]*r^8+inp[15])*r^8+inp[23])*r^1+	#   \________/\___________/+	################################################################+	#vpaddq		$H2,$T2,$H2		# accumulate input++	# d4 = h4*r0 + h3*r1   + h2*r2   + h1*r3   + h0*r4+	# d3 = h3*r0 + h2*r1   + h1*r2   + h0*r3   + h4*5*r4+	# d2 = h2*r0 + h1*r1   + h0*r2   + h4*5*r3 + h3*5*r4+	# d1 = h1*r0 + h0*r1   + h4*5*r2 + h3*5*r3 + h2*5*r4+	# d0 = h0*r0 + h4*5*r1 + h3*5*r2 + h2*5*r3 + h1*5*r4+	#+	# however, as h2 is "chronologically" first one available pull+	# corresponding operations up, so it's+	#+	# d3 = h2*r1   + h0*r3 + h1*r2   + h3*r0 + h4*5*r4+	# d4 = h2*r2   + h0*r4 + h1*r3   + h3*r1 + h4*r0+	# d0 = h2*5*r3 + h0*r0 + h1*5*r4         + h3*5*r2 + h4*5*r1+	# d1 = h2*5*r4 + h0*r1           + h1*r0 + h3*5*r3 + h4*5*r2+	# d2 = h2*r0           + h0*r2   + h1*r1 + h3*5*r4 + h4*5*r3++	vpmuludq	$H2,$R1,$D3		# d3 = h2*r1+	 vpaddq		$H0,$T0,$H0+	vpmuludq	$H2,$R2,$D4		# d4 = h2*r2+	 vpandq		$MASK,$T1,$T1		# 1+	vpmuludq	$H2,$S3,$D0		# d0 = h2*s3+	 vpandq		$MASK,$T3,$T3		# 3+	vpmuludq	$H2,$S4,$D1		# d1 = h2*s4+	 vporq		$PADBIT,$T4,$T4		# padbit, yes, always+	vpmuludq	$H2,$R0,$D2		# d2 = h2*r0+	 vpaddq		$H1,$T1,$H1		# accumulate input+	 vpaddq		$H3,$T3,$H3+	 vpaddq		$H4,$T4,$H4++	  vmovdqu64	16*0($inp),$T3		# load input+	  vmovdqu64	16*4($inp),$T4+	  lea		16*8($inp),$inp+	vpmuludq	$H0,$R3,$M3+	vpmuludq	$H0,$R4,$M4+	vpmuludq	$H0,$R0,$M0+	vpmuludq	$H0,$R1,$M1+	vpaddq		$M3,$D3,$D3		# d3 += h0*r3+	vpaddq		$M4,$D4,$D4		# d4 += h0*r4+	vpaddq		$M0,$D0,$D0		# d0 += h0*r0+	vpaddq		$M1,$D1,$D1		# d1 += h0*r1++	vpmuludq	$H1,$R2,$M3+	vpmuludq	$H1,$R3,$M4+	vpmuludq	$H1,$S4,$M0+	vpmuludq	$H0,$R2,$M2+	vpaddq		$M3,$D3,$D3		# d3 += h1*r2+	vpaddq		$M4,$D4,$D4		# d4 += h1*r3+	vpaddq		$M0,$D0,$D0		# d0 += h1*s4+	vpaddq		$M2,$D2,$D2		# d2 += h0*r2++	  vpunpcklqdq	$T4,$T3,$T0		# transpose input+	  vpunpckhqdq	$T4,$T3,$T4++	vpmuludq	$H3,$R0,$M3+	vpmuludq	$H3,$R1,$M4+	vpmuludq	$H1,$R0,$M1+	vpmuludq	$H1,$R1,$M2+	vpaddq		$M3,$D3,$D3		# d3 += h3*r0+	vpaddq		$M4,$D4,$D4		# d4 += h3*r1+	vpaddq		$M1,$D1,$D1		# d1 += h1*r0+	vpaddq		$M2,$D2,$D2		# d2 += h1*r1++	vpmuludq	$H4,$S4,$M3+	vpmuludq	$H4,$R0,$M4+	vpmuludq	$H3,$S2,$M0+	vpmuludq	$H3,$S3,$M1+	vpaddq		$M3,$D3,$D3		# d3 += h4*s4+	vpmuludq	$H3,$S4,$M2+	vpaddq		$M4,$D4,$D4		# d4 += h4*r0+	vpaddq		$M0,$D0,$D0		# d0 += h3*s2+	vpaddq		$M1,$D1,$D1		# d1 += h3*s3+	vpaddq		$M2,$D2,$D2		# d2 += h3*s4++	vpmuludq	$H4,$S1,$M0+	vpmuludq	$H4,$S2,$M1+	vpmuludq	$H4,$S3,$M2+	vpaddq		$M0,$D0,$H0		# h0 = d0 + h4*s1+	vpaddq		$M1,$D1,$H1		# h1 = d2 + h4*s2+	vpaddq		$M2,$D2,$H2		# h2 = d3 + h4*s3++	################################################################+	# lazy reduction (interleaved with input splat)++	 vpsrlq		\$52,$T0,$T2		# splat input+	 vpsllq		\$12,$T4,$T3++	vpsrlq		\$26,$D3,$H3+	vpandq		$MASK,$D3,$D3+	vpaddq		$H3,$D4,$H4		# h3 -> h4++	 vporq		$T3,$T2,$T2++	vpsrlq		\$26,$H0,$D0+	vpandq		$MASK,$H0,$H0+	vpaddq		$D0,$H1,$H1		# h0 -> h1++	 vpandq		$MASK,$T2,$T2		# 2++	vpsrlq		\$26,$H4,$D4+	vpandq		$MASK,$H4,$H4++	vpsrlq		\$26,$H1,$D1+	vpandq		$MASK,$H1,$H1+	vpaddq		$D1,$H2,$H2		# h1 -> h2++	vpaddq		$D4,$H0,$H0+	vpsllq		\$2,$D4,$D4+	vpaddq		$D4,$H0,$H0		# h4 -> h0++	 vpaddq		$T2,$H2,$H2		# modulo-scheduled+	 vpsrlq		\$26,$T0,$T1++	vpsrlq		\$26,$H2,$D2+	vpandq		$MASK,$H2,$H2+	vpaddq		$D2,$D3,$H3		# h2 -> h3++	 vpsrlq		\$14,$T4,$T3++	vpsrlq		\$26,$H0,$D0+	vpandq		$MASK,$H0,$H0+	vpaddq		$D0,$H1,$H1		# h0 -> h1++	 vpsrlq		\$40,$T4,$T4		# 4++	vpsrlq		\$26,$H3,$D3+	vpandq		$MASK,$H3,$H3+	vpaddq		$D3,$H4,$H4		# h3 -> h4++	 vpandq		$MASK,$T0,$T0		# 0+	 #vpandq	$MASK,$T1,$T1		# 1+	 #vpandq	$MASK,$T3,$T3		# 3+	 #vporq		$PADBIT,$T4,$T4		# padbit, yes, always++	sub		\$128,$len+	ja		.Loop_avx512++.Ltail_avx512:+	################################################################+	# while above multiplications were by r^8 in all lanes, in last+	# iteration we multiply least significant lane by r^8 and most+	# significant one by r, that's why table gets shifted...++	vpsrlq		\$32,$R0,$R0		# 0105020603070408+	vpsrlq		\$32,$R1,$R1+	vpsrlq		\$32,$R2,$R2+	vpsrlq		\$32,$S3,$S3+	vpsrlq		\$32,$S4,$S4+	vpsrlq		\$32,$R3,$R3+	vpsrlq		\$32,$R4,$R4+	vpsrlq		\$32,$S1,$S1+	vpsrlq		\$32,$S2,$S2++	################################################################+	# load either next or last 64 byte of input+	lea		($inp,$len),$inp++	#vpaddq		$H2,$T2,$H2		# accumulate input+	vpaddq		$H0,$T0,$H0++	vpmuludq	$H2,$R1,$D3		# d3 = h2*r1+	vpmuludq	$H2,$R2,$D4		# d4 = h2*r2+	vpmuludq	$H2,$S3,$D0		# d0 = h2*s3+	 vpandq		$MASK,$T1,$T1		# 1+	vpmuludq	$H2,$S4,$D1		# d1 = h2*s4+	 vpandq		$MASK,$T3,$T3		# 3+	vpmuludq	$H2,$R0,$D2		# d2 = h2*r0+	 vporq		$PADBIT,$T4,$T4		# padbit, yes, always+	 vpaddq		$H1,$T1,$H1		# accumulate input+	 vpaddq		$H3,$T3,$H3+	 vpaddq		$H4,$T4,$H4++	  vmovdqu	16*0($inp),%x#$T0+	vpmuludq	$H0,$R3,$M3+	vpmuludq	$H0,$R4,$M4+	vpmuludq	$H0,$R0,$M0+	vpmuludq	$H0,$R1,$M1+	vpaddq		$M3,$D3,$D3		# d3 += h0*r3+	vpaddq		$M4,$D4,$D4		# d4 += h0*r4+	vpaddq		$M0,$D0,$D0		# d0 += h0*r0+	vpaddq		$M1,$D1,$D1		# d1 += h0*r1++	  vmovdqu	16*1($inp),%x#$T1+	vpmuludq	$H1,$R2,$M3+	vpmuludq	$H1,$R3,$M4+	vpmuludq	$H1,$S4,$M0+	vpmuludq	$H0,$R2,$M2+	vpaddq		$M3,$D3,$D3		# d3 += h1*r2+	vpaddq		$M4,$D4,$D4		# d4 += h1*r3+	vpaddq		$M0,$D0,$D0		# d0 += h1*s4+	vpaddq		$M2,$D2,$D2		# d2 += h0*r2++	  vinserti128	\$1,16*2($inp),%y#$T0,%y#$T0+	vpmuludq	$H3,$R0,$M3+	vpmuludq	$H3,$R1,$M4+	vpmuludq	$H1,$R0,$M1+	vpmuludq	$H1,$R1,$M2+	vpaddq		$M3,$D3,$D3		# d3 += h3*r0+	vpaddq		$M4,$D4,$D4		# d4 += h3*r1+	vpaddq		$M1,$D1,$D1		# d1 += h1*r0+	vpaddq		$M2,$D2,$D2		# d2 += h1*r1++	  vinserti128	\$1,16*3($inp),%y#$T1,%y#$T1+	vpmuludq	$H4,$S4,$M3+	vpmuludq	$H4,$R0,$M4+	vpmuludq	$H3,$S2,$M0+	vpmuludq	$H3,$S3,$M1+	vpmuludq	$H3,$S4,$M2+	vpaddq		$M3,$D3,$H3		# h3 = d3 + h4*s4+	vpaddq		$M4,$D4,$D4		# d4 += h4*r0+	vpaddq		$M0,$D0,$D0		# d0 += h3*s2+	vpaddq		$M1,$D1,$D1		# d1 += h3*s3+	vpaddq		$M2,$D2,$D2		# d2 += h3*s4++	vpmuludq	$H4,$S1,$M0+	vpmuludq	$H4,$S2,$M1+	vpmuludq	$H4,$S3,$M2+	vpaddq		$M0,$D0,$H0		# h0 = d0 + h4*s1+	vpaddq		$M1,$D1,$H1		# h1 = d2 + h4*s2+	vpaddq		$M2,$D2,$H2		# h2 = d3 + h4*s3++	################################################################+	# horizontal addition++	mov		\$1,%eax+	vpermq		\$0xb1,$H3,$D3+	vpermq		\$0xb1,$D4,$H4+	vpermq		\$0xb1,$H0,$D0+	vpermq		\$0xb1,$H1,$D1+	vpermq		\$0xb1,$H2,$D2+	vpaddq		$D3,$H3,$H3+	vpaddq		$D4,$H4,$H4+	vpaddq		$D0,$H0,$H0+	vpaddq		$D1,$H1,$H1+	vpaddq		$D2,$H2,$H2++	kmovw		%eax,%k3+	vpermq		\$0x2,$H3,$D3+	vpermq		\$0x2,$H4,$D4+	vpermq		\$0x2,$H0,$D0+	vpermq		\$0x2,$H1,$D1+	vpermq		\$0x2,$H2,$D2+	vpaddq		$D3,$H3,$H3+	vpaddq		$D4,$H4,$H4+	vpaddq		$D0,$H0,$H0+	vpaddq		$D1,$H1,$H1+	vpaddq		$D2,$H2,$H2++	vextracti64x4	\$0x1,$H3,%y#$D3+	vextracti64x4	\$0x1,$H4,%y#$D4+	vextracti64x4	\$0x1,$H0,%y#$D0+	vextracti64x4	\$0x1,$H1,%y#$D1+	vextracti64x4	\$0x1,$H2,%y#$D2+	vpaddq		$D3,$H3,${H3}{%k3}{z}	# keep single qword in case+	vpaddq		$D4,$H4,${H4}{%k3}{z}	# it's passed to .Ltail_avx2+	vpaddq		$D0,$H0,${H0}{%k3}{z}+	vpaddq		$D1,$H1,${H1}{%k3}{z}+	vpaddq		$D2,$H2,${H2}{%k3}{z}+___+map(s/%z/%y/,($T0,$T1,$T2,$T3,$T4, $PADBIT));+map(s/%z/%y/,($H0,$H1,$H2,$H3,$H4, $D0,$D1,$D2,$D3,$D4, $MASK));+$code.=<<___;+	################################################################+	# lazy reduction (interleaved with input splat)++	vpsrlq		\$26,$H3,$D3+	vpand		$MASK,$H3,$H3+	 vpsrldq	\$6,$T0,$T2		# splat input+	 vpsrldq	\$6,$T1,$T3+	 vpunpckhqdq	$T1,$T0,$T4		# 4+	vpaddq		$D3,$H4,$H4		# h3 -> h4++	vpsrlq		\$26,$H0,$D0+	vpand		$MASK,$H0,$H0+	 vpunpcklqdq	$T3,$T2,$T2		# 2:3+	 vpunpcklqdq	$T1,$T0,$T0		# 0:1+	vpaddq		$D0,$H1,$H1		# h0 -> h1++	vpsrlq		\$26,$H4,$D4+	vpand		$MASK,$H4,$H4++	vpsrlq		\$26,$H1,$D1+	vpand		$MASK,$H1,$H1+	 vpsrlq		\$30,$T2,$T3+	 vpsrlq		\$4,$T2,$T2+	vpaddq		$D1,$H2,$H2		# h1 -> h2++	vpaddq		$D4,$H0,$H0+	vpsllq		\$2,$D4,$D4+	 vpsrlq		\$26,$T0,$T1+	 vpsrlq		\$40,$T4,$T4		# 4+	vpaddq		$D4,$H0,$H0		# h4 -> h0++	vpsrlq		\$26,$H2,$D2+	vpand		$MASK,$H2,$H2+	 vpand		$MASK,$T2,$T2		# 2+	 vpand		$MASK,$T0,$T0		# 0+	vpaddq		$D2,$H3,$H3		# h2 -> h3++	vpsrlq		\$26,$H0,$D0+	vpand		$MASK,$H0,$H0+	 vpaddq		$H2,$T2,$H2		# accumulate input for .Ltail_avx2+	 vpand		$MASK,$T1,$T1		# 1+	vpaddq		$D0,$H1,$H1		# h0 -> h1++	vpsrlq		\$26,$H3,$D3+	vpand		$MASK,$H3,$H3+	 vpand		$MASK,$T3,$T3		# 3+	 vpor		32(%rcx),$T4,$T4	# padbit, yes, always+	vpaddq		$D3,$H4,$H4		# h3 -> h4++	lea		0x90(%rsp),%rax		# size optimization for .Ltail_avx2+	add		\$64,$len+	jnz		.Ltail_avx2++	vpsubq		$T2,$H2,$H2		# undo input accumulation+	vmovd		%x#$H0,`4*0-48-64`($ctx)# save partially reduced+	vmovd		%x#$H1,`4*1-48-64`($ctx)+	vmovd		%x#$H2,`4*2-48-64`($ctx)+	vmovd		%x#$H3,`4*3-48-64`($ctx)+	vmovd		%x#$H4,`4*4-48-64`($ctx)+	vzeroall+___+$code.=<<___	if ($win64);+	movdqa		0x50(%r11),%xmm6+	movdqa		0x60(%r11),%xmm7+	movdqa		0x70(%r11),%xmm8+	movdqa		0x80(%r11),%xmm9+	movdqa		0x90(%r11),%xmm10+	movdqa		0xa0(%r11),%xmm11+	movdqa		0xb0(%r11),%xmm12+	movdqa		0xc0(%r11),%xmm13+	movdqa		0xd0(%r11),%xmm14+	movdqa		0xe0(%r11),%xmm15+	lea		0xf8(%r11),%rsp+.Ldo_avx512_epilogue:+___+$code.=<<___	if (!$win64);+	lea		8(%r11),%rsp+.cfi_def_cfa		%rsp,8+___+$code.=<<___;+	ret+.cfi_endproc+.size	poly1305_blocks_avx512,.-poly1305_blocks_avx512+___+}+if ($avx>3) {+########################################################################+# VPMADD52 version using 2^44 radix.+#+# One can argue that base 2^52 would be more natural. Well, even though+# some operations would be more natural, one has to recognize couple of+# things. Base 2^52 doesn't provide advantage over base 2^44 if you look+# at amount of multiply-n-accumulate operations. Secondly, it makes it+# impossible to pre-compute multiples of 5 [referred to as s[]/sN in+# reference implementations], which means that more such operations+# would have to be performed in inner loop, which in turn makes critical+# path longer. In other words, even though base 2^44 reduction might+# look less elegant, overall critical path is actually shorter...++########################################################################+# Layout of opaque area is following.+#+#	unsigned __int64 h[3];		# current hash value base 2^44+#	unsigned __int64 s[2];		# key value*20 base 2^44+#	unsigned __int64 r[3];		# key value base 2^44+#	struct { unsigned __int64 r^1, r^3, r^2, r^4; } R[4];+#					# r^n positions reflect+#					# placement in register, not+#					# memory, R[3] is R[1]*20++$code.=<<___;+.type	poly1305_init_base2_44,\@function,3+.align	32+poly1305_init_base2_44:+	xor	%rax,%rax+	mov	%rax,0($ctx)		# initialize hash value+	mov	%rax,8($ctx)+	mov	%rax,16($ctx)++	cmp	\$0,$inp+	je	.Lno_key_base2_44++.Linit_base2_44:+	mov	\$0x0ffffffc0fffffff,%rax+	mov	\$0x0ffffffc0ffffffc,%rcx+	and	0($inp),%rax+	mov	\$0x00000fffffffffff,%r8+	and	8($inp),%rcx+	mov	\$0x00000fffffffffff,%r9+	and	%rax,%r8		# base 2^64 -> base 2^44+	shrd	\$44,%rcx,%rax+	mov	%r8,40($ctx)		# r0+	and	%r9,%rax+	shr	\$24,%rcx+	mov	%rax,48($ctx)		# r1+	lea	(%rax,%rax,4),%rax	# *5+	mov	%rcx,56($ctx)		# r2+	shl	\$2,%rax		# magic <<2+	lea	(%rcx,%rcx,4),%rcx	# *5+	shl	\$2,%rcx		# magic <<2+	mov	%rax,24($ctx)		# s1+	mov	%rcx,32($ctx)		# s2+	movq	\$-1,64($ctx)		# write impossible value+___+					if ($flavour !~ /kernel/) {+$code.=<<___;+	lea	poly1305_blocks_vpmadd52(%rip),%r10+	lea	poly1305_emit_base2_44(%rip),%r11+___+$code.=<<___	if ($flavour !~ /elf32/);+	mov	%r10,0(%rdx)+	mov	%r11,8(%rdx)+___+$code.=<<___	if ($flavour =~ /elf32/);+	mov	%r10d,0(%rdx)+	mov	%r11d,4(%rdx)+___+					}+$code.=<<___;+	mov	\$1,%eax+.Lno_key_base2_44:+	ret+.size	poly1305_init_base2_44,.-poly1305_init_base2_44+___+{+my ($h0,$h1,$h2, $d1,$d2,$d3, $r0,$r1,$s2) = map("%r$_",("dx",8..15));+$code.=<<___;+.type	poly1305_blocks_base2_44,\@function,4+.align	32+poly1305_blocks_base2_44:+.cfi_startproc+.Lblocks_base2_44:+	push	%rbx+.cfi_push	%rbx+	push	%rbp+.cfi_push	%rbp+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15++	and	\$-16,$len+	add	$inp,$len		# end of buffer+	shl	\$40,$padbit+	push	$len+.cfi_adjust_cfa_offset	8+.Lblocks_base2_44_body:++	mov	0($ctx),$h0		# load hash value+	mov	8($ctx),$h1+	mov	16($ctx),$h2++	mov	40($ctx),$r0		# load key+	mov	48($ctx),$r1+	mov	32($ctx),$s2+	mov	\$0xfffff00000000000,%rax+	jmp	.Loop_base2_44+	ud2++.align	32+.Loop_base2_44:+	mov	0($inp),$d2		# load input+	mov	8($inp),$d3+	lea	16($inp),$inp++	andn	$d2,%rax,$d1		# base 2^64 -> base 2^44+	shrd	\$44,$d3,$d2+	add	$d1,$h0			# accumulate input+	shr	\$24,$d3+	andn	$d2,%rax,$d2+	add	$padbit,$h2++	add	$d2,$h1+	add	$d3,$h2++	#mov	$h0,%rdx		# h0 is %rdx+	mulx	$r0,$d1,%rbx		# h0*r0+	mulx	$r1,$d2,%rcx		# h0*r1+	mulx	56($ctx),$d3,%rbp	# h0*r2++	mov	$h1,%rdx+	mulx	$s2,%rax,$h1		# h1*s2+	add	%rax,$d1+	adc	%rbx,$h1+	mulx	$r0,%rax,%rbx		# h1*r0+	add	%rax,$d2+	adc	%rbx,%rcx+	mulx	$r1,%rax,%rbx		# h1*r1+	mov	$h2,%rdx+	add	%rax,$d3+	adc	%rbx,%rbp++	mulx	24($ctx),%rax,%rbx	# h2*s1+	add	%rax,$d1+	adc	%rbx,$h1+	mulx	$s2,%rax,$h2		# h2*s2+	add	%rax,$d2+	adc	%rcx,$h2+	mulx	$r0,%rax,%rbx		# h2*r0+	add	%rax,$d3+	adc	%rbx,%rbp++	mov	\$0xfffff00000000000,%rax+	andn	$d1,%rax,$h0+	shrd	\$44,$h1,$d1+	add	$d1,$d2+	adc	\$0,$h2+	andn	$d2,%rax,$h1+	shrd	\$44,$h2,$d2+	mov	\$0x03ffffffffff,$h2+	add	$d2,$d3+	adc	\$0,%rbp+	and	$d3,$h2+	shrd	\$42,%rbp,$d3++	mov	\$0x10000000000,$padbit+	lea	($d3,$d3,4),$d3		# *=5+	add	$d3,$h0++	cmp	0(%rsp),$inp+	jb	.Loop_base2_44++	mov	$h0,0($ctx)		# store hash value+	mov	$h1,8($ctx)+	mov	$h2,16($ctx)++	mov	8(%rsp),%r15+.cfi_restore	%r15+	mov	16(%rsp),%r14+.cfi_restore	%r14+	mov	24(%rsp),%r13+.cfi_restore	%r13+	mov	32(%rsp),%r12+.cfi_restore	%r12+	mov	40(%rsp),%rbp+.cfi_restore	%rbp+	mov	48(%rsp),%rbx+.cfi_restore	%rbx+	lea	56(%rsp),%rsp+.cfi_adjust_cfa_offset	-56+.Lblocks_base2_44_epilogue:+	ret+.cfi_endproc+.size	poly1305_blocks_base2_44,.-poly1305_blocks_base2_44+___+}+{+my ($H0,$H1,$H2,$r2r1r0,$r1r0s2,$r0s2s1,$Dlo,$Dhi) = map("%ymm$_",(0..5,16,17));+my ($T0,$inp_permd,$inp_shift,$PAD) = map("%ymm$_",(18..21));+my ($reduc_mask,$reduc_rght,$reduc_left) = map("%ymm$_",(22..25));+my ($T1,$T2,$T3) = map("%ymm$_",(26..28));++$code.=<<___;+.type	poly1305_blocks_vpmadd52,\@function,4+.align	32+poly1305_blocks_vpmadd52:+	and	\$-16,$len+	jz	.Lno_data_vpmadd52		# too short++	mov	64($ctx),%r8			# peek on power of the key++	# if powers of the key are not calculated yet, process up to 3+	# blocks with scalar single-block subroutine above, otherwise+	# ensure that input length is divisible by 2 blocks and pass+	# the rest down to next subroutine...++	mov	\$0x30,%r9+	mov	\$0x10,%r10+	cmp	\$0x40,$len			# is input long+	cmovae	%r10,%r9+	test	%r8,%r8				# is power value impossible?+	cmovns	%r10,%r9++	and	$len,%r9			# is input of favourable length?+	jz	.Lblocks_vpmadd52_4x++	sub	%r9,$len+	cmovz	%r9,$len+	jz	.Lblocks_base2_44++	#########################################+	mov		\$7,%r10d+	mov		\$1,%r11d+	shl		\$40,$padbit+	kmovw		%r10d,%k7+	lea		.L2_44_inp_permd(%rip),%r10+	kmovw		%r11d,%k1++	vmovq		$padbit,%x#$PAD+	shr		\$40,$padbit		# restore original value+	vmovdqa64	0(%r10),$inp_permd	# .L2_44_inp_permd+	vmovdqa64	32(%r10),$inp_shift	# .L2_44_inp_shift+	vpermq		\$0xcf,$PAD,$PAD+	vmovdqa64	64(%r10),$reduc_mask	# .L2_44_mask++	vmovdqu64	0($ctx),${Dlo}{%k7}{z}		# load hash value+	vmovdqu64	40($ctx),${r2r1r0}{%k7}{z}	# load keys+	vmovdqu64	32($ctx),${r1r0s2}{%k7}{z}+	vmovdqu64	24($ctx),${r0s2s1}{%k7}{z}++	vmovdqa64	96(%r10),$reduc_rght	# .L2_44_shift_rgt+	vmovdqa64	128(%r10),$reduc_left	# .L2_44_shift_lft++	vmovdqu32	0($inp),%x#$T0		# load input as ----3210+	lea		16($inp),$inp++	vpermd		$T0,$inp_permd,$T0	# ----3210 -> --322110+	vpsrlvq		$inp_shift,$T0,$T0+	vpandq		$reduc_mask,$T0,$T0+	vporq		$PAD,$T0,$T0++	vpaddq		$T0,$Dlo,$Dlo		# accumulate input+	vpxord		$T2,$T2,$T2+	vpxord		$T3,$T3,$T3++	vpermq		\$0,$Dlo,${H0}{%k7}{z}	# smash hash value+	vpermq		\$0b01010101,$Dlo,${H1}{%k7}{z}+	vpermq		\$0b10101010,$Dlo,${H2}{%k7}{z}++	vpxord		$T0,$T0,$T0+	vpxord		$T1,$T1,$T1+	vpmadd52luq	$r2r1r0,$H0,$T2+	vpmadd52huq	$r2r1r0,$H0,$T3++	vpxord		$Dlo,$Dlo,$Dlo+	vpxord		$Dhi,$Dhi,$Dhi+	vpmadd52luq	$r1r0s2,$H1,$T0+	vpmadd52huq	$r1r0s2,$H1,$T1++	vpmadd52luq	$r0s2s1,$H2,$Dlo+	vpmadd52huq	$r0s2s1,$H2,$Dhi++	vpaddq		$T0,$T2,$T2+	vpaddq		$T1,$T3,$T3+	vpaddq		$T2,$Dlo,$Dlo+	vpaddq		$T3,$Dhi,$Dhi++	vpsrlvq		$reduc_rght,$Dlo,$T0	# 0 in topmost qword+	vpsllvq		$reduc_left,$Dhi,$Dhi	# 0 in topmost qword+	vpandq		$reduc_mask,$Dlo,$Dlo++	vpaddq		$T0,$Dhi,$Dhi++	vpermq		\$0b10010011,$Dhi,$Dhi	# 0 in lowest qword++	vpaddq		$Dhi,$Dlo,$Dlo		# note topmost qword :-)++	vpsrlvq		$reduc_rght,$Dlo,$T0	# 0 in topmost word+	vpandq		$reduc_mask,$Dlo,$Dlo++	vpermq		\$0b10010011,$T0,$T0++	vpaddq		$T0,$Dlo,$Dlo++	vpermq		\$0b10010011,$Dlo,${T0}{%k1}{z}++	vpaddq		$T0,$Dlo,$Dlo+	vpsllq		\$2,$T0,$T0++	vpaddq		$T0,$Dlo,$Dlo++	vmovdqu64	$Dlo,0($ctx){%k7}	# store hash value++	jmp		.Lblocks_vpmadd52_4x++.Lno_data_vpmadd52:+	ret+.size	poly1305_blocks_vpmadd52,.-poly1305_blocks_vpmadd52+___+}+{+########################################################################+# As implied by its name 4x subroutine processes 4 blocks in parallel+# (but handles even 4*n+2 blocks lengths). It takes up to 4th key power+# and is handled in 256-bit %ymm registers.++my ($H0,$H1,$H2,$R0,$R1,$R2,$S1,$S2) = map("%ymm$_",(0..5,16,17));+my ($D0lo,$D0hi,$D1lo,$D1hi,$D2lo,$D2hi) = map("%ymm$_",(18..23));+my ($T0,$T1,$T2,$T3,$T4,$tmp,$mask44,$PAD) = map("%ymm$_",(24..31));++$code.=<<___;+.type	poly1305_blocks_vpmadd52_4x,\@function,4+.align	32+poly1305_blocks_vpmadd52_4x:+	and	\$-16,$len+	jz	.Lno_data_vpmadd52_4x		# too short++	mov	64($ctx),%r8			# peek on power of the key++.Lblocks_vpmadd52_4x:+	shl		\$40,$padbit+	shr		\$4,$len+	vpbroadcastq	$padbit,$PAD++	vmovdqa64	.Lx_mask44(%rip),$mask44+	mov		\$5,%eax+	kmovw		%eax,%k1		# used in 2x path++	test		%r8,%r8			# is power value impossible?+	js		.Linit_vpmadd52		# if it is, then init R[4]++	vmovq		0($ctx),%x#$H0		# load current hash value+	vmovq		8($ctx),%x#$H1+	vmovq		16($ctx),%x#$H2++	test		\$3,$len		# is length 4*n+2?+	jnz		.Lblocks_vpmadd52_2x_do++.Lblocks_vpmadd52_4x_do:+	vpbroadcastq	64($ctx),$R0		# load 4th power of the key+	vpbroadcastq	96($ctx),$R1+	vpbroadcastq	128($ctx),$R2+	vpbroadcastq	160($ctx),$S1++.Lblocks_vpmadd52_4x_key_loaded:+	vpsllq		\$2,$R2,$S2		# S2 = R2*5*4+	vpaddq		$R2,$S2,$S2+	vpsllq		\$2,$S2,$S2++	#test		\$7,$len		# is len 8*n?+	#jz		.Lblocks_vpmadd52_8x++	vmovdqu64	16*0($inp),$T2		# load data+	vmovdqu64	16*2($inp),$T3+	lea		16*4($inp),$inp++	vpunpcklqdq	$T3,$T2,$T1		# transpose data+	vpunpckhqdq	$T3,$T2,$T3++	# at this point 64-bit lanes are ordered as 3-1-2-0++	vpsrlq		\$24,$T3,$T2		# splat the data+	vporq		$PAD,$T2,$T2+	 vpaddq		$T2,$H2,$H2		# accumulate input+	vpandq		$mask44,$T1,$T0+	vpsrlq		\$44,$T1,$T1+	vpsllq		\$20,$T3,$T3+	vporq		$T3,$T1,$T1+	vpandq		$mask44,$T1,$T1++	sub		\$4,$len+	jz		.Ltail_vpmadd52_4x+	jmp		.Loop_vpmadd52_4x+	ud2++.align	32+.Linit_vpmadd52:+	vmovq		24($ctx),%x#$S1		# load key+	vmovq		56($ctx),%x#$H2+	vmovq		32($ctx),%x#$S2+	vmovq		40($ctx),%x#$R0+	vmovq		48($ctx),%x#$R1++	vmovdqa		$R0,$H0+	vmovdqa		$R1,$H1+	vmovdqa		$H2,$R2++	mov		\$2,%eax++.Lmul_init_vpmadd52:+	vpxorq		$D0lo,$D0lo,$D0lo+	vpxorq		$D0hi,$D0hi,$D0hi+	vpxorq		$D1lo,$D1lo,$D1lo+	vpxorq		$D1hi,$D1hi,$D1hi+	vpxorq		$D2lo,$D2lo,$D2lo+	vpxorq		$D2hi,$D2hi,$D2hi+	vpmadd52luq	$H2,$S1,$D0lo+	vpxorq		$T0,$T0,$T0+	vpxorq		$T1,$T1,$T1+	vpmadd52huq	$H2,$S1,$D0hi+	vpxorq		$T2,$T2,$T2+	vpxorq		$T3,$T3,$T3+	vpmadd52luq	$H2,$S2,$D1lo+	vpxorq		$T4,$T4,$T4+	vpxorq		$tmp,$tmp,$tmp+	vpmadd52huq	$H2,$S2,$D1hi+	vpmadd52luq	$H2,$R0,$D2lo+	vpmadd52huq	$H2,$R0,$D2hi++	vpmadd52luq	$H0,$R0,$T0+	vpmadd52huq	$H0,$R0,$T1+	vpmadd52luq	$H0,$R1,$T2+	vpmadd52huq	$H0,$R1,$T3+	vpmadd52luq	$H0,$R2,$T4+	vpmadd52huq	$H0,$R2,$tmp++	vpmadd52luq	$H1,$S2,$D0lo+	vpmadd52huq	$H1,$S2,$D0hi+	vpmadd52luq	$H1,$R0,$D1lo+	vpmadd52huq	$H1,$R0,$D1hi+	vpaddq		$T0,$D0lo,$D0lo+	vpaddq		$T1,$D0hi,$D0hi+	vpmadd52luq	$H1,$R1,$D2lo+	vpaddq		$T2,$D1lo,$D1lo+	vpaddq		$T3,$D1hi,$D1hi+	vpmadd52huq	$H1,$R1,$D2hi+	vpaddq		$T4,$D2lo,$D2lo+	vpaddq		$tmp,$D2hi,$D2hi++	################################################################+	# partial reduction+	vpsrlq		\$44,$D0lo,$tmp+	vpsllq		\$8,$D0hi,$D0hi+	vpandq		$mask44,$D0lo,$H0+	vpaddq		$tmp,$D0hi,$D0hi++	vpaddq		$D0hi,$D1lo,$D1lo++	vpsrlq		\$44,$D1lo,$tmp+	vpsllq		\$8,$D1hi,$D1hi+	vpandq		$mask44,$D1lo,$H1+	vpaddq		$tmp,$D1hi,$D1hi++	vpaddq		$D1hi,$D2lo,$D2lo++	vpsrlq		\$42,$D2lo,$tmp+	vpsllq		\$10,$D2hi,$D2hi+	vpandq		.Lx_mask42(%rip),$D2lo,$H2+	vpaddq		$tmp,$D2hi,$D2hi++	vpaddq		$D2hi,$H0,$H0+	vpsllq		\$2,$D2hi,$D2hi++	vpaddq		$D2hi,$H0,$H0++	vpsrlq		\$44,$H0,$tmp		# additional step+	vpandq		$mask44,$H0,$H0++	vpaddq		$tmp,$H1,$H1++	dec		%eax+	jz		.Ldone_init_vpmadd52++	vpunpcklqdq	$R1,$H1,$R1		# 1,2+	vpbroadcastq	%x#$H1,%x#$H1		# 2,2+	vpunpcklqdq	$R2,$H2,$R2+	vpbroadcastq	%x#$H2,%x#$H2+	vpunpcklqdq	$R0,$H0,$R0+	vpbroadcastq	%x#$H0,%x#$H0++	vpsllq		\$2,$R1,$S1		# S1 = R1*5*4+	vpsllq		\$2,$R2,$S2		# S2 = R2*5*4+	vpaddq		$R1,$S1,$S1+	vpaddq		$R2,$S2,$S2+	vpsllq		\$2,$S1,$S1+	vpsllq		\$2,$S2,$S2++	jmp		.Lmul_init_vpmadd52+	ud2++.align	32+.Ldone_init_vpmadd52:+	vinserti128	\$1,%x#$R1,$H1,$R1	# 1,2,3,4+	vinserti128	\$1,%x#$R2,$H2,$R2+	vinserti128	\$1,%x#$R0,$H0,$R0++	vpermq		\$0b11011000,$R1,$R1	# 1,3,2,4+	vpermq		\$0b11011000,$R2,$R2+	vpermq		\$0b11011000,$R0,$R0++	vpsllq		\$2,$R1,$S1		# S1 = R1*5*4+	vpaddq		$R1,$S1,$S1+	vpsllq		\$2,$S1,$S1++	vmovq		0($ctx),%x#$H0		# load current hash value+	vmovq		8($ctx),%x#$H1+	vmovq		16($ctx),%x#$H2++	test		\$3,$len		# is length 4*n+2?+	jnz		.Ldone_init_vpmadd52_2x++	vmovdqu64	$R0,64($ctx)		# save key powers+	vpbroadcastq	%x#$R0,$R0		# broadcast 4th power+	vmovdqu64	$R1,96($ctx)+	vpbroadcastq	%x#$R1,$R1+	vmovdqu64	$R2,128($ctx)+	vpbroadcastq	%x#$R2,$R2+	vmovdqu64	$S1,160($ctx)+	vpbroadcastq	%x#$S1,$S1++	jmp		.Lblocks_vpmadd52_4x_key_loaded+	ud2++.align	32+.Ldone_init_vpmadd52_2x:+	vmovdqu64	$R0,64($ctx)		# save key powers+	vpsrldq		\$8,$R0,$R0		# 0-1-0-2+	vmovdqu64	$R1,96($ctx)+	vpsrldq		\$8,$R1,$R1+	vmovdqu64	$R2,128($ctx)+	vpsrldq		\$8,$R2,$R2+	vmovdqu64	$S1,160($ctx)+	vpsrldq		\$8,$S1,$S1+	jmp		.Lblocks_vpmadd52_2x_key_loaded+	ud2++.align	32+.Lblocks_vpmadd52_2x_do:+	vmovdqu64	128+8($ctx),${R2}{%k1}{z}# load 2nd and 1st key powers+	vmovdqu64	160+8($ctx),${S1}{%k1}{z}+	vmovdqu64	64+8($ctx),${R0}{%k1}{z}+	vmovdqu64	96+8($ctx),${R1}{%k1}{z}++.Lblocks_vpmadd52_2x_key_loaded:+	vmovdqu64	16*0($inp),$T2		# load data+	vpxorq		$T3,$T3,$T3+	lea		16*2($inp),$inp++	vpunpcklqdq	$T3,$T2,$T1		# transpose data+	vpunpckhqdq	$T3,$T2,$T3++	# at this point 64-bit lanes are ordered as x-1-x-0++	vpsrlq		\$24,$T3,$T2		# splat the data+	vporq		$PAD,$T2,$T2+	 vpaddq		$T2,$H2,$H2		# accumulate input+	vpandq		$mask44,$T1,$T0+	vpsrlq		\$44,$T1,$T1+	vpsllq		\$20,$T3,$T3+	vporq		$T3,$T1,$T1+	vpandq		$mask44,$T1,$T1++	jmp		.Ltail_vpmadd52_2x+	ud2++.align	32+.Loop_vpmadd52_4x:+	#vpaddq		$T2,$H2,$H2		# accumulate input+	vpaddq		$T0,$H0,$H0+	vpaddq		$T1,$H1,$H1++	vpxorq		$D0lo,$D0lo,$D0lo+	vpxorq		$D0hi,$D0hi,$D0hi+	vpxorq		$D1lo,$D1lo,$D1lo+	vpxorq		$D1hi,$D1hi,$D1hi+	vpxorq		$D2lo,$D2lo,$D2lo+	vpxorq		$D2hi,$D2hi,$D2hi+	vpmadd52luq	$H2,$S1,$D0lo+	vpxorq		$T0,$T0,$T0+	vpxorq		$T1,$T1,$T1+	vpmadd52huq	$H2,$S1,$D0hi+	vpxorq		$T2,$T2,$T2+	vpxorq		$T3,$T3,$T3+	vpmadd52luq	$H2,$S2,$D1lo+	vpxorq		$T4,$T4,$T4+	vpxorq		$tmp,$tmp,$tmp+	vpmadd52huq	$H2,$S2,$D1hi+	vpmadd52luq	$H2,$R0,$D2lo+	vpmadd52huq	$H2,$R0,$D2hi++	vpmadd52luq	$H0,$R0,$T0+	vpmadd52huq	$H0,$R0,$T1+	vpmadd52luq	$H0,$R1,$T2+	vpmadd52huq	$H0,$R1,$T3+	vpmadd52luq	$H0,$R2,$T4+	vpmadd52huq	$H0,$R2,$tmp++	vpmadd52luq	$H1,$S2,$D0lo+	vpmadd52huq	$H1,$S2,$D0hi+	vpmadd52luq	$H1,$R0,$D1lo+	vpmadd52huq	$H1,$R0,$D1hi+	vpaddq		$T0,$D0lo,$D0lo+	vpaddq		$T1,$D0hi,$D0hi+	vpmadd52luq	$H1,$R1,$D2lo+	vpaddq		$T2,$D1lo,$D1lo+	vpaddq		$T3,$D1hi,$D1hi+	vpmadd52huq	$H1,$R1,$D2hi+	vpaddq		$T4,$D2lo,$D2lo+	vpaddq		$tmp,$D2hi,$D2hi++	 vmovdqu64	16*0($inp),$T2		# load data+	 vmovdqu64	16*2($inp),$T3+	 lea		16*4($inp),$inp+	 vpunpcklqdq	$T3,$T2,$T1		# transpose data+	 vpunpckhqdq	$T3,$T2,$T3++	################################################################+	# partial reduction (interleaved with data splat)+	vpsrlq		\$44,$D0lo,$tmp+	vpsllq		\$8,$D0hi,$D0hi+	vpandq		$mask44,$D0lo,$H0+	vpaddq		$tmp,$D0hi,$D0hi++	 vpsrlq		\$24,$T3,$T2+	 vporq		$PAD,$T2,$T2+	vpaddq		$D0hi,$D1lo,$D1lo++	vpsrlq		\$44,$D1lo,$tmp+	vpsllq		\$8,$D1hi,$D1hi+	vpandq		$mask44,$D1lo,$H1+	vpaddq		$tmp,$D1hi,$D1hi++	 vpandq		$mask44,$T1,$T0+	 vpsrlq		\$44,$T1,$T1+	 vpsllq		\$20,$T3,$T3+	vpaddq		$D1hi,$D2lo,$D2lo++	vpsrlq		\$42,$D2lo,$tmp+	vpsllq		\$10,$D2hi,$D2hi+	vpandq		.Lx_mask42(%rip),$D2lo,$H2+	vpaddq		$tmp,$D2hi,$D2hi++	  vpaddq	$T2,$H2,$H2		# accumulate input+	vpaddq		$D2hi,$H0,$H0+	vpsllq		\$2,$D2hi,$D2hi++	vpaddq		$D2hi,$H0,$H0+	 vporq		$T3,$T1,$T1+	 vpandq		$mask44,$T1,$T1++	vpsrlq		\$44,$H0,$tmp		# additional step+	vpandq		$mask44,$H0,$H0++	vpaddq		$tmp,$H1,$H1++	sub		\$4,$len		# len-=64+	jnz		.Loop_vpmadd52_4x++.Ltail_vpmadd52_4x:+	vmovdqu64	128($ctx),$R2		# load all key powers+	vmovdqu64	160($ctx),$S1+	vmovdqu64	64($ctx),$R0+	vmovdqu64	96($ctx),$R1++.Ltail_vpmadd52_2x:+	vpsllq		\$2,$R2,$S2		# S2 = R2*5*4+	vpaddq		$R2,$S2,$S2+	vpsllq		\$2,$S2,$S2++	#vpaddq		$T2,$H2,$H2		# accumulate input+	vpaddq		$T0,$H0,$H0+	vpaddq		$T1,$H1,$H1++	vpxorq		$D0lo,$D0lo,$D0lo+	vpxorq		$D0hi,$D0hi,$D0hi+	vpxorq		$D1lo,$D1lo,$D1lo+	vpxorq		$D1hi,$D1hi,$D1hi+	vpxorq		$D2lo,$D2lo,$D2lo+	vpxorq		$D2hi,$D2hi,$D2hi+	vpmadd52luq	$H2,$S1,$D0lo+	vpxorq		$T0,$T0,$T0+	vpxorq		$T1,$T1,$T1+	vpmadd52huq	$H2,$S1,$D0hi+	vpxorq		$T2,$T2,$T2+	vpxorq		$T3,$T3,$T3+	vpmadd52luq	$H2,$S2,$D1lo+	vpxorq		$T4,$T4,$T4+	vpxorq		$tmp,$tmp,$tmp+	vpmadd52huq	$H2,$S2,$D1hi+	vpmadd52luq	$H2,$R0,$D2lo+	vpmadd52huq	$H2,$R0,$D2hi++	vpmadd52luq	$H0,$R0,$T0+	vpmadd52huq	$H0,$R0,$T1+	vpmadd52luq	$H0,$R1,$T2+	vpmadd52huq	$H0,$R1,$T3+	vpmadd52luq	$H0,$R2,$T4+	vpmadd52huq	$H0,$R2,$tmp++	vpmadd52luq	$H1,$S2,$D0lo+	vpmadd52huq	$H1,$S2,$D0hi+	vpmadd52luq	$H1,$R0,$D1lo+	vpmadd52huq	$H1,$R0,$D1hi+	vpaddq		$T0,$D0lo,$D0lo+	vpaddq		$T1,$D0hi,$D0hi+	vpmadd52luq	$H1,$R1,$D2lo+	vpaddq		$T2,$D1lo,$D1lo+	vpaddq		$T3,$D1hi,$D1hi+	vpmadd52huq	$H1,$R1,$D2hi+	vpaddq		$T4,$D2lo,$D2lo+	vpaddq		$tmp,$D2hi,$D2hi++	################################################################+	# horizontal addition++	mov		\$1,%eax+	kmovw		%eax,%k1+	vpsrldq		\$8,$D0lo,$T0+	vpsrldq		\$8,$D0hi,$H0+	vpsrldq		\$8,$D1lo,$T1+	vpsrldq		\$8,$D1hi,$H1+	vpaddq		$T0,$D0lo,$D0lo+	vpaddq		$H0,$D0hi,$D0hi+	vpsrldq		\$8,$D2lo,$T2+	vpsrldq		\$8,$D2hi,$H2+	vpaddq		$T1,$D1lo,$D1lo+	vpaddq		$H1,$D1hi,$D1hi+	 vpermq		\$0x2,$D0lo,$T0+	 vpermq		\$0x2,$D0hi,$H0+	vpaddq		$T2,$D2lo,$D2lo+	vpaddq		$H2,$D2hi,$D2hi++	vpermq		\$0x2,$D1lo,$T1+	vpermq		\$0x2,$D1hi,$H1+	vpaddq		$T0,$D0lo,${D0lo}{%k1}{z}+	vpaddq		$H0,$D0hi,${D0hi}{%k1}{z}+	vpermq		\$0x2,$D2lo,$T2+	vpermq		\$0x2,$D2hi,$H2+	vpaddq		$T1,$D1lo,${D1lo}{%k1}{z}+	vpaddq		$H1,$D1hi,${D1hi}{%k1}{z}+	vpaddq		$T2,$D2lo,${D2lo}{%k1}{z}+	vpaddq		$H2,$D2hi,${D2hi}{%k1}{z}++	################################################################+	# partial reduction+	vpsrlq		\$44,$D0lo,$tmp+	vpsllq		\$8,$D0hi,$D0hi+	vpandq		$mask44,$D0lo,$H0+	vpaddq		$tmp,$D0hi,$D0hi++	vpaddq		$D0hi,$D1lo,$D1lo++	vpsrlq		\$44,$D1lo,$tmp+	vpsllq		\$8,$D1hi,$D1hi+	vpandq		$mask44,$D1lo,$H1+	vpaddq		$tmp,$D1hi,$D1hi++	vpaddq		$D1hi,$D2lo,$D2lo++	vpsrlq		\$42,$D2lo,$tmp+	vpsllq		\$10,$D2hi,$D2hi+	vpandq		.Lx_mask42(%rip),$D2lo,$H2+	vpaddq		$tmp,$D2hi,$D2hi++	vpaddq		$D2hi,$H0,$H0+	vpsllq		\$2,$D2hi,$D2hi++	vpaddq		$D2hi,$H0,$H0++	vpsrlq		\$44,$H0,$tmp		# additional step+	vpandq		$mask44,$H0,$H0++	vpaddq		$tmp,$H1,$H1+						# at this point $len is+						# either 4*n+2 or 0...+	sub		\$2,$len		# len-=32+	ja		.Lblocks_vpmadd52_4x_do++	vmovq		%x#$H0,0($ctx)+	vmovq		%x#$H1,8($ctx)+	vmovq		%x#$H2,16($ctx)+	vzeroall++.Lno_data_vpmadd52_4x:+	ret+.size	poly1305_blocks_vpmadd52_4x,.-poly1305_blocks_vpmadd52_4x+___+}+if (0) {+########################################################################+# As implied by its name 8x subroutine processes 8 blocks in parallel...+# This is intermediate version, as it's used only in cases when input+# length is either 8*n, 8*n+1 or 8*n+2...++my ($H0,$H1,$H2,$R0,$R1,$R2,$S1,$S2) = map("%ymm$_",(0..5,16,17));+my ($D0lo,$D0hi,$D1lo,$D1hi,$D2lo,$D2hi) = map("%ymm$_",(18..23));+my ($T0,$T1,$T2,$T3,$mask44,$mask42,$tmp,$PAD) = map("%ymm$_",(24..31));+my ($RR0,$RR1,$RR2,$SS1,$SS2) = map("%ymm$_",(6..10));++$code.=<<___;+.type	poly1305_blocks_vpmadd52_8x,\@function,4+.align	32+poly1305_blocks_vpmadd52_8x:+	shr	\$4,$len+	jz	.Lno_data_vpmadd52_8x		# too short++	shl	\$40,$padbit+	mov	64($ctx),%r8			# peek on power of the key++	vmovdqa64	.Lx_mask44(%rip),$mask44+	vmovdqa64	.Lx_mask42(%rip),$mask42++	test	%r8,%r8				# is power value impossible?+	js	.Linit_vpmadd52			# if it is, then init R[4]++	vmovq	0($ctx),%x#$H0			# load current hash value+	vmovq	8($ctx),%x#$H1+	vmovq	16($ctx),%x#$H2++.Lblocks_vpmadd52_8x:+	################################################################+	# fist we calculate more key powers++	vmovdqu64	128($ctx),$R2		# load 1-3-2-4 powers+	vmovdqu64	160($ctx),$S1+	vmovdqu64	64($ctx),$R0+	vmovdqu64	96($ctx),$R1++	vpsllq		\$2,$R2,$S2		# S2 = R2*5*4+	vpaddq		$R2,$S2,$S2+	vpsllq		\$2,$S2,$S2++	vpbroadcastq	%x#$R2,$RR2		# broadcast 4th power+	vpbroadcastq	%x#$R0,$RR0+	vpbroadcastq	%x#$R1,$RR1++	vpxorq		$D0lo,$D0lo,$D0lo+	vpmadd52luq	$RR2,$S1,$D0lo+	vpxorq		$D0hi,$D0hi,$D0hi+	vpmadd52huq	$RR2,$S1,$D0hi+	vpxorq		$D1lo,$D1lo,$D1lo+	vpmadd52luq	$RR2,$S2,$D1lo+	vpxorq		$D1hi,$D1hi,$D1hi+	vpmadd52huq	$RR2,$S2,$D1hi+	vpxorq		$D2lo,$D2lo,$D2lo+	vpmadd52luq	$RR2,$R0,$D2lo+	vpxorq		$D2hi,$D2hi,$D2hi+	vpmadd52huq	$RR2,$R0,$D2hi++	vpmadd52luq	$RR0,$R0,$D0lo+	vpmadd52huq	$RR0,$R0,$D0hi+	vpmadd52luq	$RR0,$R1,$D1lo+	vpmadd52huq	$RR0,$R1,$D1hi+	vpmadd52luq	$RR0,$R2,$D2lo+	vpmadd52huq	$RR0,$R2,$D2hi++	vpmadd52luq	$RR1,$S2,$D0lo+	vpmadd52huq	$RR1,$S2,$D0hi+	vpmadd52luq	$RR1,$R0,$D1lo+	vpmadd52huq	$RR1,$R0,$D1hi+	vpmadd52luq	$RR1,$R1,$D2lo+	vpmadd52huq	$RR1,$R1,$D2hi++	################################################################+	# partial reduction+	vpsrlq		\$44,$D0lo,$tmp+	vpsllq		\$8,$D0hi,$D0hi+	vpandq		$mask44,$D0lo,$RR0+	vpaddq		$tmp,$D0hi,$D0hi++	vpaddq		$D0hi,$D1lo,$D1lo++	vpsrlq		\$44,$D1lo,$tmp+	vpsllq		\$8,$D1hi,$D1hi+	vpandq		$mask44,$D1lo,$RR1+	vpaddq		$tmp,$D1hi,$D1hi++	vpaddq		$D1hi,$D2lo,$D2lo++	vpsrlq		\$42,$D2lo,$tmp+	vpsllq		\$10,$D2hi,$D2hi+	vpandq		$mask42,$D2lo,$RR2+	vpaddq		$tmp,$D2hi,$D2hi++	vpaddq		$D2hi,$RR0,$RR0+	vpsllq		\$2,$D2hi,$D2hi++	vpaddq		$D2hi,$RR0,$RR0++	vpsrlq		\$44,$RR0,$tmp		# additional step+	vpandq		$mask44,$RR0,$RR0++	vpaddq		$tmp,$RR1,$RR1++	################################################################+	# At this point Rx holds 1324 powers, RRx - 5768, and the goal+	# is 15263748, which reflects how data is loaded...++	vpunpcklqdq	$R2,$RR2,$T2		# 3748+	vpunpckhqdq	$R2,$RR2,$R2		# 1526+	vpunpcklqdq	$R0,$RR0,$T0+	vpunpckhqdq	$R0,$RR0,$R0+	vpunpcklqdq	$R1,$RR1,$T1+	vpunpckhqdq	$R1,$RR1,$R1+___+######## switch to %zmm+map(s/%y/%z/, $H0,$H1,$H2,$R0,$R1,$R2,$S1,$S2);+map(s/%y/%z/, $D0lo,$D0hi,$D1lo,$D1hi,$D2lo,$D2hi);+map(s/%y/%z/, $T0,$T1,$T2,$T3,$mask44,$mask42,$tmp,$PAD);+map(s/%y/%z/, $RR0,$RR1,$RR2,$SS1,$SS2);++$code.=<<___;+	vshufi64x2	\$0x44,$R2,$T2,$RR2	# 15263748+	vshufi64x2	\$0x44,$R0,$T0,$RR0+	vshufi64x2	\$0x44,$R1,$T1,$RR1++	vmovdqu64	16*0($inp),$T2		# load data+	vmovdqu64	16*4($inp),$T3+	lea		16*8($inp),$inp++	vpsllq		\$2,$RR2,$SS2		# S2 = R2*5*4+	vpsllq		\$2,$RR1,$SS1		# S1 = R1*5*4+	vpaddq		$RR2,$SS2,$SS2+	vpaddq		$RR1,$SS1,$SS1+	vpsllq		\$2,$SS2,$SS2+	vpsllq		\$2,$SS1,$SS1++	vpbroadcastq	$padbit,$PAD+	vpbroadcastq	%x#$mask44,$mask44+	vpbroadcastq	%x#$mask42,$mask42++	vpbroadcastq	%x#$SS1,$S1		# broadcast 8th power+	vpbroadcastq	%x#$SS2,$S2+	vpbroadcastq	%x#$RR0,$R0+	vpbroadcastq	%x#$RR1,$R1+	vpbroadcastq	%x#$RR2,$R2++	vpunpcklqdq	$T3,$T2,$T1		# transpose data+	vpunpckhqdq	$T3,$T2,$T3++	# at this point 64-bit lanes are ordered as 73625140++	vpsrlq		\$24,$T3,$T2		# splat the data+	vporq		$PAD,$T2,$T2+	 vpaddq		$T2,$H2,$H2		# accumulate input+	vpandq		$mask44,$T1,$T0+	vpsrlq		\$44,$T1,$T1+	vpsllq		\$20,$T3,$T3+	vporq		$T3,$T1,$T1+	vpandq		$mask44,$T1,$T1++	sub		\$8,$len+	jz		.Ltail_vpmadd52_8x+	jmp		.Loop_vpmadd52_8x++.align	32+.Loop_vpmadd52_8x:+	#vpaddq		$T2,$H2,$H2		# accumulate input+	vpaddq		$T0,$H0,$H0+	vpaddq		$T1,$H1,$H1++	vpxorq		$D0lo,$D0lo,$D0lo+	vpmadd52luq	$H2,$S1,$D0lo+	vpxorq		$D0hi,$D0hi,$D0hi+	vpmadd52huq	$H2,$S1,$D0hi+	vpxorq		$D1lo,$D1lo,$D1lo+	vpmadd52luq	$H2,$S2,$D1lo+	vpxorq		$D1hi,$D1hi,$D1hi+	vpmadd52huq	$H2,$S2,$D1hi+	vpxorq		$D2lo,$D2lo,$D2lo+	vpmadd52luq	$H2,$R0,$D2lo+	vpxorq		$D2hi,$D2hi,$D2hi+	vpmadd52huq	$H2,$R0,$D2hi++	 vmovdqu64	16*0($inp),$T2		# load data+	 vmovdqu64	16*4($inp),$T3+	 lea		16*8($inp),$inp+	vpmadd52luq	$H0,$R0,$D0lo+	vpmadd52huq	$H0,$R0,$D0hi+	vpmadd52luq	$H0,$R1,$D1lo+	vpmadd52huq	$H0,$R1,$D1hi+	vpmadd52luq	$H0,$R2,$D2lo+	vpmadd52huq	$H0,$R2,$D2hi++	 vpunpcklqdq	$T3,$T2,$T1		# transpose data+	 vpunpckhqdq	$T3,$T2,$T3+	vpmadd52luq	$H1,$S2,$D0lo+	vpmadd52huq	$H1,$S2,$D0hi+	vpmadd52luq	$H1,$R0,$D1lo+	vpmadd52huq	$H1,$R0,$D1hi+	vpmadd52luq	$H1,$R1,$D2lo+	vpmadd52huq	$H1,$R1,$D2hi++	################################################################+	# partial reduction (interleaved with data splat)+	vpsrlq		\$44,$D0lo,$tmp+	vpsllq		\$8,$D0hi,$D0hi+	vpandq		$mask44,$D0lo,$H0+	vpaddq		$tmp,$D0hi,$D0hi++	 vpsrlq		\$24,$T3,$T2+	 vporq		$PAD,$T2,$T2+	vpaddq		$D0hi,$D1lo,$D1lo++	vpsrlq		\$44,$D1lo,$tmp+	vpsllq		\$8,$D1hi,$D1hi+	vpandq		$mask44,$D1lo,$H1+	vpaddq		$tmp,$D1hi,$D1hi++	 vpandq		$mask44,$T1,$T0+	 vpsrlq		\$44,$T1,$T1+	 vpsllq		\$20,$T3,$T3+	vpaddq		$D1hi,$D2lo,$D2lo++	vpsrlq		\$42,$D2lo,$tmp+	vpsllq		\$10,$D2hi,$D2hi+	vpandq		$mask42,$D2lo,$H2+	vpaddq		$tmp,$D2hi,$D2hi++	  vpaddq	$T2,$H2,$H2		# accumulate input+	vpaddq		$D2hi,$H0,$H0+	vpsllq		\$2,$D2hi,$D2hi++	vpaddq		$D2hi,$H0,$H0+	 vporq		$T3,$T1,$T1+	 vpandq		$mask44,$T1,$T1++	vpsrlq		\$44,$H0,$tmp		# additional step+	vpandq		$mask44,$H0,$H0++	vpaddq		$tmp,$H1,$H1++	sub		\$8,$len		# len-=128+	jnz		.Loop_vpmadd52_8x++.Ltail_vpmadd52_8x:+	#vpaddq		$T2,$H2,$H2		# accumulate input+	vpaddq		$T0,$H0,$H0+	vpaddq		$T1,$H1,$H1++	vpxorq		$D0lo,$D0lo,$D0lo+	vpmadd52luq	$H2,$SS1,$D0lo+	vpxorq		$D0hi,$D0hi,$D0hi+	vpmadd52huq	$H2,$SS1,$D0hi+	vpxorq		$D1lo,$D1lo,$D1lo+	vpmadd52luq	$H2,$SS2,$D1lo+	vpxorq		$D1hi,$D1hi,$D1hi+	vpmadd52huq	$H2,$SS2,$D1hi+	vpxorq		$D2lo,$D2lo,$D2lo+	vpmadd52luq	$H2,$RR0,$D2lo+	vpxorq		$D2hi,$D2hi,$D2hi+	vpmadd52huq	$H2,$RR0,$D2hi++	vpmadd52luq	$H0,$RR0,$D0lo+	vpmadd52huq	$H0,$RR0,$D0hi+	vpmadd52luq	$H0,$RR1,$D1lo+	vpmadd52huq	$H0,$RR1,$D1hi+	vpmadd52luq	$H0,$RR2,$D2lo+	vpmadd52huq	$H0,$RR2,$D2hi++	vpmadd52luq	$H1,$SS2,$D0lo+	vpmadd52huq	$H1,$SS2,$D0hi+	vpmadd52luq	$H1,$RR0,$D1lo+	vpmadd52huq	$H1,$RR0,$D1hi+	vpmadd52luq	$H1,$RR1,$D2lo+	vpmadd52huq	$H1,$RR1,$D2hi++	################################################################+	# horizontal addition++	mov		\$1,%eax+	kmovw		%eax,%k1+	vpsrldq		\$8,$D0lo,$T0+	vpsrldq		\$8,$D0hi,$H0+	vpsrldq		\$8,$D1lo,$T1+	vpsrldq		\$8,$D1hi,$H1+	vpaddq		$T0,$D0lo,$D0lo+	vpaddq		$H0,$D0hi,$D0hi+	vpsrldq		\$8,$D2lo,$T2+	vpsrldq		\$8,$D2hi,$H2+	vpaddq		$T1,$D1lo,$D1lo+	vpaddq		$H1,$D1hi,$D1hi+	 vpermq		\$0x2,$D0lo,$T0+	 vpermq		\$0x2,$D0hi,$H0+	vpaddq		$T2,$D2lo,$D2lo+	vpaddq		$H2,$D2hi,$D2hi++	vpermq		\$0x2,$D1lo,$T1+	vpermq		\$0x2,$D1hi,$H1+	vpaddq		$T0,$D0lo,$D0lo+	vpaddq		$H0,$D0hi,$D0hi+	vpermq		\$0x2,$D2lo,$T2+	vpermq		\$0x2,$D2hi,$H2+	vpaddq		$T1,$D1lo,$D1lo+	vpaddq		$H1,$D1hi,$D1hi+	 vextracti64x4	\$1,$D0lo,%y#$T0+	 vextracti64x4	\$1,$D0hi,%y#$H0+	vpaddq		$T2,$D2lo,$D2lo+	vpaddq		$H2,$D2hi,$D2hi++	vextracti64x4	\$1,$D1lo,%y#$T1+	vextracti64x4	\$1,$D1hi,%y#$H1+	vextracti64x4	\$1,$D2lo,%y#$T2+	vextracti64x4	\$1,$D2hi,%y#$H2+___+######## switch back to %ymm+map(s/%z/%y/, $H0,$H1,$H2,$R0,$R1,$R2,$S1,$S2);+map(s/%z/%y/, $D0lo,$D0hi,$D1lo,$D1hi,$D2lo,$D2hi);+map(s/%z/%y/, $T0,$T1,$T2,$T3,$mask44,$mask42,$tmp,$PAD);++$code.=<<___;+	vpaddq		$T0,$D0lo,${D0lo}{%k1}{z}+	vpaddq		$H0,$D0hi,${D0hi}{%k1}{z}+	vpaddq		$T1,$D1lo,${D1lo}{%k1}{z}+	vpaddq		$H1,$D1hi,${D1hi}{%k1}{z}+	vpaddq		$T2,$D2lo,${D2lo}{%k1}{z}+	vpaddq		$H2,$D2hi,${D2hi}{%k1}{z}++	################################################################+	# partial reduction+	vpsrlq		\$44,$D0lo,$tmp+	vpsllq		\$8,$D0hi,$D0hi+	vpandq		$mask44,$D0lo,$H0+	vpaddq		$tmp,$D0hi,$D0hi++	vpaddq		$D0hi,$D1lo,$D1lo++	vpsrlq		\$44,$D1lo,$tmp+	vpsllq		\$8,$D1hi,$D1hi+	vpandq		$mask44,$D1lo,$H1+	vpaddq		$tmp,$D1hi,$D1hi++	vpaddq		$D1hi,$D2lo,$D2lo++	vpsrlq		\$42,$D2lo,$tmp+	vpsllq		\$10,$D2hi,$D2hi+	vpandq		$mask42,$D2lo,$H2+	vpaddq		$tmp,$D2hi,$D2hi++	vpaddq		$D2hi,$H0,$H0+	vpsllq		\$2,$D2hi,$D2hi++	vpaddq		$D2hi,$H0,$H0++	vpsrlq		\$44,$H0,$tmp		# additional step+	vpandq		$mask44,$H0,$H0++	vpaddq		$tmp,$H1,$H1++	################################################################++	vmovq		%x#$H0,0($ctx)+	vmovq		%x#$H1,8($ctx)+	vmovq		%x#$H2,16($ctx)+	vzeroall++.Lno_data_vpmadd52_8x:+	ret+.size	poly1305_blocks_vpmadd52_8x,.-poly1305_blocks_vpmadd52_8x+___+}+$code.=<<___;+.type	poly1305_emit_base2_44,\@function,3+.align	32+poly1305_emit_base2_44:+	mov	0($ctx),%r8	# load hash value+	mov	8($ctx),%r9+	mov	16($ctx),%r10++	mov	%r9,%rax	# base 2^44 -> base 2^64+	shr	\$20,%r9+	shl	\$44,%rax+	mov	%r10,%rcx+	shr	\$40,%r10+	shl	\$24,%rcx++	add	%rax,%r8+	adc	%rcx,%r9+	adc	\$0,%r10++	mov	%r8,%rax+	add	\$5,%r8		# compare to modulus+	mov	%r9,%rcx+	adc	\$0,%r9+	adc	\$0,%r10+	shr	\$2,%r10	# did 130-bit value overflow?+	cmovnz	%r8,%rax+	cmovnz	%r9,%rcx++	add	0($nonce),%rax	# accumulate nonce+	adc	8($nonce),%rcx+	mov	%rax,0($mac)	# write result+	mov	%rcx,8($mac)++	ret+.size	poly1305_emit_base2_44,.-poly1305_emit_base2_44+___+}	}+$code.=<<___;+.align	64+.Lconst:+.Lmask24:+.long	0x0ffffff,0,0x0ffffff,0,0x0ffffff,0,0x0ffffff,0+.L129:+.long	`1<<24`,0,`1<<24`,0,`1<<24`,0,`1<<24`,0+.Lmask26:+.long	0x3ffffff,0,0x3ffffff,0,0x3ffffff,0,0x3ffffff,0+.Lpermd_avx2:+.long	2,2,2,3,2,0,2,1+.Lpermd_avx512:+.long	0,0,0,1, 0,2,0,3, 0,4,0,5, 0,6,0,7++.L2_44_inp_permd:+.long	0,1,1,2,2,3,7,7+.L2_44_inp_shift:+.quad	0,12,24,64+.L2_44_mask:+.quad	0xfffffffffff,0xfffffffffff,0x3ffffffffff,0xffffffffffffffff+.L2_44_shift_rgt:+.quad	44,44,42,64+.L2_44_shift_lft:+.quad	8,8,10,64++.align	64+.Lx_mask44:+.quad	0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff+.quad	0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff+.Lx_mask42:+.quad	0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff+.quad	0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff+___+}+$code.=<<___;+.asciz	"Poly1305 for x86_64, CRYPTOGAMS by \@dot-asm"+.align	16+___++{	# chacha20-poly1305 helpers+my ($out,$inp,$otp,$len)=$win64 ? ("%rcx","%rdx","%r8", "%r9") :  # Win64 order+                                  ("%rdi","%rsi","%rdx","%rcx");  # Unix order+$code.=<<___;+.globl	xor128_encrypt_n_pad+.type	xor128_encrypt_n_pad,\@abi-omnipotent+.align	16+xor128_encrypt_n_pad:+	sub	$otp,$inp+	sub	$otp,$out+	mov	$len,%r10		# put len aside+	shr	\$4,$len		# len / 16+	jz	.Ltail_enc+	nop+.Loop_enc_xmm:+	movdqu	($inp,$otp),%xmm0+	pxor	($otp),%xmm0+	movdqu	%xmm0,($out,$otp)+	movdqa	%xmm0,($otp)+	lea	16($otp),$otp+	dec	$len+	jnz	.Loop_enc_xmm++	and	\$15,%r10		# len % 16+	jz	.Ldone_enc++.Ltail_enc:+	mov	\$16,$len+	sub	%r10,$len+	xor	%eax,%eax+.Loop_enc_byte:+	mov	($inp,$otp),%al+	xor	($otp),%al+	mov	%al,($out,$otp)+	mov	%al,($otp)+	lea	1($otp),$otp+	dec	%r10+	jnz	.Loop_enc_byte++	xor	%eax,%eax+.Loop_enc_pad:+	mov	%al,($otp)+	lea	1($otp),$otp+	dec	$len+	jnz	.Loop_enc_pad++.Ldone_enc:+	mov	$otp,%rax+	ret+.size	xor128_encrypt_n_pad,.-xor128_encrypt_n_pad++.globl	xor128_decrypt_n_pad+.type	xor128_decrypt_n_pad,\@abi-omnipotent+.align	16+xor128_decrypt_n_pad:+	sub	$otp,$inp+	sub	$otp,$out+	mov	$len,%r10		# put len aside+	shr	\$4,$len		# len / 16+	jz	.Ltail_dec+	nop+.Loop_dec_xmm:+	movdqu	($inp,$otp),%xmm0+	movdqa	($otp),%xmm1+	pxor	%xmm0,%xmm1+	movdqu	%xmm1,($out,$otp)+	movdqa	%xmm0,($otp)+	lea	16($otp),$otp+	dec	$len+	jnz	.Loop_dec_xmm++	pxor	%xmm1,%xmm1+	and	\$15,%r10		# len % 16+	jz	.Ldone_dec++.Ltail_dec:+	mov	\$16,$len+	sub	%r10,$len+	xor	%eax,%eax+	xor	%r11,%r11+.Loop_dec_byte:+	mov	($inp,$otp),%r11b+	mov	($otp),%al+	xor	%r11b,%al+	mov	%al,($out,$otp)+	mov	%r11b,($otp)+	lea	1($otp),$otp+	dec	%r10+	jnz	.Loop_dec_byte++	xor	%eax,%eax+.Loop_dec_pad:+	mov	%al,($otp)+	lea	1($otp),$otp+	dec	$len+	jnz	.Loop_dec_pad++.Ldone_dec:+	mov	$otp,%rax+	ret+.size	xor128_decrypt_n_pad,.-xor128_decrypt_n_pad+___+}++# EXCEPTION_DISPOSITION handler (EXCEPTION_RECORD *rec,ULONG64 frame,+#		CONTEXT *context,DISPATCHER_CONTEXT *disp)+if ($win64) {+$rec="%rcx";+$frame="%rdx";+$context="%r8";+$disp="%r9";++$code.=<<___;+.extern	__imp_RtlVirtualUnwind+.type	se_handler,\@abi-omnipotent+.align	16+se_handler:+	push	%rsi+	push	%rdi+	push	%rbx+	push	%rbp+	push	%r12+	push	%r13+	push	%r14+	push	%r15+	pushfq+	sub	\$64,%rsp++	mov	120($context),%rax	# pull context->Rax+	mov	248($context),%rbx	# pull context->Rip++	mov	8($disp),%rsi		# disp->ImageBase+	mov	56($disp),%r11		# disp->HandlerData++	mov	0(%r11),%r10d		# HandlerData[0]+	lea	(%rsi,%r10),%r10	# prologue label+	cmp	%r10,%rbx		# context->Rip<.Lprologue+	jb	.Lcommon_seh_tail++	mov	152($context),%rax	# pull context->Rsp++	mov	4(%r11),%r10d		# HandlerData[1]+	lea	(%rsi,%r10),%r10	# epilogue label+	cmp	%r10,%rbx		# context->Rip>=.Lepilogue+	jae	.Lcommon_seh_tail++	lea	56(%rax),%rax++	mov	-8(%rax),%rbx+	mov	-16(%rax),%rbp+	mov	-24(%rax),%r12+	mov	-32(%rax),%r13+	mov	-40(%rax),%r14+	mov	-48(%rax),%r15+	mov	%rbx,144($context)	# restore context->Rbx+	mov	%rbp,160($context)	# restore context->Rbp+	mov	%r12,216($context)	# restore context->R12+	mov	%r13,224($context)	# restore context->R13+	mov	%r14,232($context)	# restore context->R14+	mov	%r15,240($context)	# restore context->R14++	jmp	.Lcommon_seh_tail+.size	se_handler,.-se_handler++.type	avx_handler,\@abi-omnipotent+.align	16+avx_handler:+	push	%rsi+	push	%rdi+	push	%rbx+	push	%rbp+	push	%r12+	push	%r13+	push	%r14+	push	%r15+	pushfq+	sub	\$64,%rsp++	mov	120($context),%rax	# pull context->Rax+	mov	248($context),%rbx	# pull context->Rip++	mov	8($disp),%rsi		# disp->ImageBase+	mov	56($disp),%r11		# disp->HandlerData++	mov	0(%r11),%r10d		# HandlerData[0]+	lea	(%rsi,%r10),%r10	# prologue label+	cmp	%r10,%rbx		# context->Rip<prologue label+	jb	.Lcommon_seh_tail++	mov	152($context),%rax	# pull context->Rsp++	mov	4(%r11),%r10d		# HandlerData[1]+	lea	(%rsi,%r10),%r10	# epilogue label+	cmp	%r10,%rbx		# context->Rip>=epilogue label+	jae	.Lcommon_seh_tail++	mov	208($context),%rax	# pull context->R11++	lea	0x50(%rax),%rsi+	lea	0xf8(%rax),%rax+	lea	512($context),%rdi	# &context.Xmm6+	mov	\$20,%ecx+	.long	0xa548f3fc		# cld; rep movsq++.Lcommon_seh_tail:+	mov	8(%rax),%rdi+	mov	16(%rax),%rsi+	mov	%rax,152($context)	# restore context->Rsp+	mov	%rsi,168($context)	# restore context->Rsi+	mov	%rdi,176($context)	# restore context->Rdi++	mov	40($disp),%rdi		# disp->ContextRecord+	mov	$context,%rsi		# context+	mov	\$154,%ecx		# sizeof(CONTEXT)+	.long	0xa548f3fc		# cld; rep movsq++	mov	$disp,%rsi+	xor	%rcx,%rcx		# arg1, UNW_FLAG_NHANDLER+	mov	8(%rsi),%rdx		# arg2, disp->ImageBase+	mov	0(%rsi),%r8		# arg3, disp->ControlPc+	mov	16(%rsi),%r9		# arg4, disp->FunctionEntry+	mov	40(%rsi),%r10		# disp->ContextRecord+	lea	56(%rsi),%r11		# &disp->HandlerData+	lea	24(%rsi),%r12		# &disp->EstablisherFrame+	mov	%r10,32(%rsp)		# arg5+	mov	%r11,40(%rsp)		# arg6+	mov	%r12,48(%rsp)		# arg7+	mov	%rcx,56(%rsp)		# arg8, (NULL)+	call	*__imp_RtlVirtualUnwind(%rip)++	mov	\$1,%eax		# ExceptionContinueSearch+	add	\$64,%rsp+	popfq+	pop	%r15+	pop	%r14+	pop	%r13+	pop	%r12+	pop	%rbp+	pop	%rbx+	pop	%rdi+	pop	%rsi+	ret+.size	avx_handler,.-avx_handler++.section	.pdata+.align	4+	.rva	.LSEH_begin_poly1305_init+	.rva	.LSEH_end_poly1305_init+	.rva	.LSEH_info_poly1305_init++	.rva	.LSEH_begin_poly1305_blocks+	.rva	.LSEH_end_poly1305_blocks+	.rva	.LSEH_info_poly1305_blocks++	.rva	.LSEH_begin_poly1305_emit+	.rva	.LSEH_end_poly1305_emit+	.rva	.LSEH_info_poly1305_emit+___+$code.=<<___ if ($avx);+	.rva	.LSEH_begin_poly1305_blocks_avx+	.rva	.Lbase2_64_avx+	.rva	.LSEH_info_poly1305_blocks_avx_1++	.rva	.Lbase2_64_avx+	.rva	.Leven_avx+	.rva	.LSEH_info_poly1305_blocks_avx_2++	.rva	.Leven_avx+	.rva	.LSEH_end_poly1305_blocks_avx+	.rva	.LSEH_info_poly1305_blocks_avx_3+___+$code.=<<___ if ($avx>1);+	.rva	.LSEH_begin_poly1305_blocks_avx2+	.rva	.Lbase2_64_avx2+	.rva	.LSEH_info_poly1305_blocks_avx2_1++	.rva	.Lbase2_64_avx2+	.rva	.Leven_avx2+	.rva	.LSEH_info_poly1305_blocks_avx2_2++	.rva	.Leven_avx2+	.rva	.LSEH_end_poly1305_blocks_avx2+	.rva	.LSEH_info_poly1305_blocks_avx2_3+___+$code.=<<___ if ($avx>2);+	.rva	.LSEH_begin_poly1305_blocks_avx512+	.rva	.LSEH_end_poly1305_blocks_avx512+	.rva	.LSEH_info_poly1305_blocks_avx512+___+$code.=<<___ if ($avx>3);+	.rva	.LSEH_begin_poly1305_init_base2_44+	.rva	.LSEH_end_poly1305_init_base2_44+	.rva	.LSEH_info_poly1305_init_base2_44++	.rva	.LSEH_begin_poly1305_blocks_base2_44+	.rva	.LSEH_end_poly1305_blocks_base2_44+	.rva	.LSEH_info_poly1305_blocks_base2_44++	.rva	.LSEH_begin_poly1305_blocks_vpmadd52+	.rva	.LSEH_end_poly1305_blocks_vpmadd52+	.rva	.LSEH_info_poly1305_blocks_vpmadd52++	.rva	.LSEH_begin_poly1305_blocks_vpmadd52_4x+	.rva	.LSEH_end_poly1305_blocks_vpmadd52_4x+	.rva	.LSEH_info_poly1305_blocks_vpmadd52_4x++	.rva	.LSEH_begin_poly1305_emit_base2_44+	.rva	.LSEH_end_poly1305_emit_base2_44+	.rva	.LSEH_info_poly1305_emit_base2_44+___+$code.=<<___;+.section	.xdata+.align	8+.LSEH_info_poly1305_init:+	.byte	9,0,0,0+	.rva	se_handler+	.long	0,0			# 0,0 means "no stack frame allocated"++.LSEH_info_poly1305_blocks:+	.byte	9,0,0,0+	.rva	se_handler+	.rva	.Lblocks_body,.Lblocks_epilogue++.LSEH_info_poly1305_emit:+	.byte	9,0,0,0+	.rva	se_handler+	.long	0,0+___+$code.=<<___ if ($avx);+.LSEH_info_poly1305_blocks_avx_1:+	.byte	9,0,0,0+	.rva	se_handler+	.rva	.Lblocks_avx_body,.Lblocks_avx_epilogue		# HandlerData[]++.LSEH_info_poly1305_blocks_avx_2:+	.byte	9,0,0,0+	.rva	se_handler+	.rva	.Lbase2_64_avx_body,.Lbase2_64_avx_epilogue	# HandlerData[]++.LSEH_info_poly1305_blocks_avx_3:+	.byte	9,0,0,0+	.rva	avx_handler+	.rva	.Ldo_avx_body,.Ldo_avx_epilogue			# HandlerData[]+___+$code.=<<___ if ($avx>1);+.LSEH_info_poly1305_blocks_avx2_1:+	.byte	9,0,0,0+	.rva	se_handler+	.rva	.Lblocks_avx2_body,.Lblocks_avx2_epilogue	# HandlerData[]++.LSEH_info_poly1305_blocks_avx2_2:+	.byte	9,0,0,0+	.rva	se_handler+	.rva	.Lbase2_64_avx2_body,.Lbase2_64_avx2_epilogue	# HandlerData[]++.LSEH_info_poly1305_blocks_avx2_3:+	.byte	9,0,0,0+	.rva	avx_handler+	.rva	.Ldo_avx2_body,.Ldo_avx2_epilogue		# HandlerData[]+___+$code.=<<___ if ($avx>2);+.LSEH_info_poly1305_blocks_avx512:+	.byte	9,0,0,0+	.rva	avx_handler+	.rva	.Ldo_avx512_body,.Ldo_avx512_epilogue		# HandlerData[]+___+$code.=<<___ if ($avx>3);+.LSEH_info_poly1305_init_base2_44:+	.byte	9,0,0,0+	.rva	se_handler+	.long	0,0++.LSEH_info_poly1305_blocks_base2_44:+	.byte	9,0,0,0+	.rva	se_handler+	.rva	.Lblocks_base2_44_body,.Lblocks_base2_44_epilogue++.LSEH_info_poly1305_blocks_vpmadd52:+	.byte	9,0,0,0+	.rva	se_handler+	.long	0,0++.LSEH_info_poly1305_blocks_vpmadd52_4x:+	.byte	9,0,0,0+	.rva	se_handler+	.long	0,0++.LSEH_info_poly1305_emit_base2_44:+	.byte	9,0,0,0+	.rva	se_handler+	.long	0,0+___+}++foreach (split('\n',$code)) {+	s/\`([^\`]*)\`/eval($1)/ge;+	s/%r([a-z]+)#d/%e$1/g;+	s/%r([0-9]+)#d/%r$1d/g;+	s/%x#%[yz]/%x/g or s/%y#%z/%y/g or s/%z#%[yz]/%z/g;++	print $_,"\n";+}+close STDOUT;
+ cbits/asm/sha1-armv8-ios64.S view
@@ -0,0 +1,1216 @@+#ifndef	__KERNEL__+# include "arm_arch.h"++#else+.globl	_crypton_sha1_asm_block_armv8+#endif++.text++.globl	_crypton_sha1_asm_block_data_order++.align	6+_crypton_sha1_asm_block_data_order:+	adrp	x16,_crypton_armcap_P@PAGE+	ldr	w16,[x16,_crypton_armcap_P@PAGEOFF]+	tst	w16,#ARMV8_SHA1+	b.ne	Lv8_entry++	stp	x29,x30,[sp,#-12*__SIZEOF_POINTER__]!+	add	x29,sp,#0+	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	stp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	stp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	stp	x27,x28,[sp,#10*__SIZEOF_POINTER__]++	ldp	w20,w21,[x0]+	ldp	w22,w23,[x0,#8]+	ldr	w24,[x0,#16]++Loop:+	ldr	x3,[x1],#64+	movz	w28,#0x7999+	sub	x2,x2,#1+	movk	w28,#0x5a82,lsl#16+#ifdef	__AARCH64EB__+	ror	x3,x3,#32+#else+	rev32	x3,x3+#endif+	add	w24,w24,w28		// warm it up+	add	w24,w24,w3+	lsr	x4,x3,#32+	ldur	x5,[x1,#-56]+	bic	w25,w23,w21+	and	w26,w22,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	orr	w25,w25,w26+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	add	w23,w23,w4	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+#ifdef	__AARCH64EB__+	ror	x5,x5,#32+#else+	rev32	x5,x5+#endif+	bic	w25,w22,w20+	and	w26,w21,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	orr	w25,w25,w26+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	add	w22,w22,w5	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	lsr	x6,x5,#32+	ldur	x7,[x1,#-48]+	bic	w25,w21,w24+	and	w26,w20,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	orr	w25,w25,w26+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	add	w21,w21,w6	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+#ifdef	__AARCH64EB__+	ror	x7,x7,#32+#else+	rev32	x7,x7+#endif+	bic	w25,w20,w23+	and	w26,w24,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	orr	w25,w25,w26+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	add	w20,w20,w7	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	lsr	x8,x7,#32+	ldur	x9,[x1,#-40]+	bic	w25,w24,w22+	and	w26,w23,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	orr	w25,w25,w26+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	add	w24,w24,w8	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+#ifdef	__AARCH64EB__+	ror	x9,x9,#32+#else+	rev32	x9,x9+#endif+	bic	w25,w23,w21+	and	w26,w22,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	orr	w25,w25,w26+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	add	w23,w23,w9	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	lsr	x10,x9,#32+	ldur	x11,[x1,#-32]+	bic	w25,w22,w20+	and	w26,w21,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	orr	w25,w25,w26+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	add	w22,w22,w10	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+#ifdef	__AARCH64EB__+	ror	x11,x11,#32+#else+	rev32	x11,x11+#endif+	bic	w25,w21,w24+	and	w26,w20,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	orr	w25,w25,w26+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	add	w21,w21,w11	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	lsr	x12,x11,#32+	ldur	x13,[x1,#-24]+	bic	w25,w20,w23+	and	w26,w24,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	orr	w25,w25,w26+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	add	w20,w20,w12	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+#ifdef	__AARCH64EB__+	ror	x13,x13,#32+#else+	rev32	x13,x13+#endif+	bic	w25,w24,w22+	and	w26,w23,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	orr	w25,w25,w26+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	add	w24,w24,w13	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	lsr	x14,x13,#32+	ldur	x15,[x1,#-16]+	bic	w25,w23,w21+	and	w26,w22,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	orr	w25,w25,w26+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	add	w23,w23,w14	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+#ifdef	__AARCH64EB__+	ror	x15,x15,#32+#else+	rev32	x15,x15+#endif+	bic	w25,w22,w20+	and	w26,w21,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	orr	w25,w25,w26+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	add	w22,w22,w15	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	lsr	x16,x15,#32+	ldur	x17,[x1,#-8]+	bic	w25,w21,w24+	and	w26,w20,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	orr	w25,w25,w26+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	add	w21,w21,w16	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+#ifdef	__AARCH64EB__+	ror	x17,x17,#32+#else+	rev32	x17,x17+#endif+	bic	w25,w20,w23+	and	w26,w24,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	orr	w25,w25,w26+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	add	w20,w20,w17	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	lsr	x19,x17,#32+	eor	w3,w3,w5+	bic	w25,w24,w22+	and	w26,w23,w22+	ror	w27,w21,#27+	eor	w3,w3,w11+	add	w24,w24,w28		// future e+=K+	orr	w25,w25,w26+	add	w20,w20,w27		// e+=rot(a,5)+	eor	w3,w3,w16+	ror	w22,w22,#2+	add	w24,w24,w19	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w3,w3,#31+	eor	w4,w4,w6+	bic	w25,w23,w21+	and	w26,w22,w21+	ror	w27,w20,#27+	eor	w4,w4,w12+	add	w23,w23,w28		// future e+=K+	orr	w25,w25,w26+	add	w24,w24,w27		// e+=rot(a,5)+	eor	w4,w4,w17+	ror	w21,w21,#2+	add	w23,w23,w3	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w4,w4,#31+	eor	w5,w5,w7+	bic	w25,w22,w20+	and	w26,w21,w20+	ror	w27,w24,#27+	eor	w5,w5,w13+	add	w22,w22,w28		// future e+=K+	orr	w25,w25,w26+	add	w23,w23,w27		// e+=rot(a,5)+	eor	w5,w5,w19+	ror	w20,w20,#2+	add	w22,w22,w4	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w5,w5,#31+	eor	w6,w6,w8+	bic	w25,w21,w24+	and	w26,w20,w24+	ror	w27,w23,#27+	eor	w6,w6,w14+	add	w21,w21,w28		// future e+=K+	orr	w25,w25,w26+	add	w22,w22,w27		// e+=rot(a,5)+	eor	w6,w6,w3+	ror	w24,w24,#2+	add	w21,w21,w5	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w6,w6,#31+	eor	w7,w7,w9+	bic	w25,w20,w23+	and	w26,w24,w23+	ror	w27,w22,#27+	eor	w7,w7,w15+	add	w20,w20,w28		// future e+=K+	orr	w25,w25,w26+	add	w21,w21,w27		// e+=rot(a,5)+	eor	w7,w7,w4+	ror	w23,w23,#2+	add	w20,w20,w6	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w7,w7,#31+	movz	w28,#0xeba1+	movk	w28,#0x6ed9,lsl#16+	eor	w8,w8,w10+	bic	w25,w24,w22+	and	w26,w23,w22+	ror	w27,w21,#27+	eor	w8,w8,w16+	add	w24,w24,w28		// future e+=K+	orr	w25,w25,w26+	add	w20,w20,w27		// e+=rot(a,5)+	eor	w8,w8,w5+	ror	w22,w22,#2+	add	w24,w24,w7	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w8,w8,#31+	eor	w9,w9,w11+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w9,w9,w17+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w9,w9,w6+	add	w23,w23,w8	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w9,w9,#31+	eor	w10,w10,w12+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w10,w10,w19+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w10,w10,w7+	add	w22,w22,w9	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w10,w10,#31+	eor	w11,w11,w13+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w11,w11,w3+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w11,w11,w8+	add	w21,w21,w10	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w11,w11,#31+	eor	w12,w12,w14+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w12,w12,w4+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	eor	w12,w12,w9+	add	w20,w20,w11	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w12,w12,#31+	eor	w13,w13,w15+	eor	w25,w24,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	eor	w13,w13,w5+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	eor	w13,w13,w10+	add	w24,w24,w12	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w13,w13,#31+	eor	w14,w14,w16+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w14,w14,w6+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w14,w14,w11+	add	w23,w23,w13	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w14,w14,#31+	eor	w15,w15,w17+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w15,w15,w7+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w15,w15,w12+	add	w22,w22,w14	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w15,w15,#31+	eor	w16,w16,w19+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w16,w16,w8+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w16,w16,w13+	add	w21,w21,w15	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w16,w16,#31+	eor	w17,w17,w3+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w17,w17,w9+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	eor	w17,w17,w14+	add	w20,w20,w16	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w17,w17,#31+	eor	w19,w19,w4+	eor	w25,w24,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	eor	w19,w19,w10+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	eor	w19,w19,w15+	add	w24,w24,w17	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w19,w19,#31+	eor	w3,w3,w5+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w3,w3,w11+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w3,w3,w16+	add	w23,w23,w19	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w3,w3,#31+	eor	w4,w4,w6+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w4,w4,w12+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w4,w4,w17+	add	w22,w22,w3	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w4,w4,#31+	eor	w5,w5,w7+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w5,w5,w13+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w5,w5,w19+	add	w21,w21,w4	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w5,w5,#31+	eor	w6,w6,w8+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w6,w6,w14+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	eor	w6,w6,w3+	add	w20,w20,w5	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w6,w6,#31+	eor	w7,w7,w9+	eor	w25,w24,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	eor	w7,w7,w15+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	eor	w7,w7,w4+	add	w24,w24,w6	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w7,w7,#31+	eor	w8,w8,w10+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w8,w8,w16+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w8,w8,w5+	add	w23,w23,w7	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w8,w8,#31+	eor	w9,w9,w11+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w9,w9,w17+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w9,w9,w6+	add	w22,w22,w8	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w9,w9,#31+	eor	w10,w10,w12+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w10,w10,w19+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w10,w10,w7+	add	w21,w21,w9	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w10,w10,#31+	eor	w11,w11,w13+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w11,w11,w3+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	eor	w11,w11,w8+	add	w20,w20,w10	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w11,w11,#31+	movz	w28,#0xbcdc+	movk	w28,#0x8f1b,lsl#16+	eor	w12,w12,w14+	eor	w25,w24,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	eor	w12,w12,w4+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	eor	w12,w12,w9+	add	w24,w24,w11	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w12,w12,#31+	orr	w25,w21,w22+	and	w26,w21,w22+	eor	w13,w13,w15+	ror	w27,w20,#27+	and	w25,w25,w23+	add	w23,w23,w28		// future e+=K+	eor	w13,w13,w5+	add	w24,w24,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w21,w21,#2+	eor	w13,w13,w10+	add	w23,w23,w12	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w13,w13,#31+	orr	w25,w20,w21+	and	w26,w20,w21+	eor	w14,w14,w16+	ror	w27,w24,#27+	and	w25,w25,w22+	add	w22,w22,w28		// future e+=K+	eor	w14,w14,w6+	add	w23,w23,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w20,w20,#2+	eor	w14,w14,w11+	add	w22,w22,w13	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w14,w14,#31+	orr	w25,w24,w20+	and	w26,w24,w20+	eor	w15,w15,w17+	ror	w27,w23,#27+	and	w25,w25,w21+	add	w21,w21,w28		// future e+=K+	eor	w15,w15,w7+	add	w22,w22,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w24,w24,#2+	eor	w15,w15,w12+	add	w21,w21,w14	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w15,w15,#31+	orr	w25,w23,w24+	and	w26,w23,w24+	eor	w16,w16,w19+	ror	w27,w22,#27+	and	w25,w25,w20+	add	w20,w20,w28		// future e+=K+	eor	w16,w16,w8+	add	w21,w21,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w23,w23,#2+	eor	w16,w16,w13+	add	w20,w20,w15	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w16,w16,#31+	orr	w25,w22,w23+	and	w26,w22,w23+	eor	w17,w17,w3+	ror	w27,w21,#27+	and	w25,w25,w24+	add	w24,w24,w28		// future e+=K+	eor	w17,w17,w9+	add	w20,w20,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w22,w22,#2+	eor	w17,w17,w14+	add	w24,w24,w16	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w17,w17,#31+	orr	w25,w21,w22+	and	w26,w21,w22+	eor	w19,w19,w4+	ror	w27,w20,#27+	and	w25,w25,w23+	add	w23,w23,w28		// future e+=K+	eor	w19,w19,w10+	add	w24,w24,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w21,w21,#2+	eor	w19,w19,w15+	add	w23,w23,w17	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w19,w19,#31+	orr	w25,w20,w21+	and	w26,w20,w21+	eor	w3,w3,w5+	ror	w27,w24,#27+	and	w25,w25,w22+	add	w22,w22,w28		// future e+=K+	eor	w3,w3,w11+	add	w23,w23,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w20,w20,#2+	eor	w3,w3,w16+	add	w22,w22,w19	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w3,w3,#31+	orr	w25,w24,w20+	and	w26,w24,w20+	eor	w4,w4,w6+	ror	w27,w23,#27+	and	w25,w25,w21+	add	w21,w21,w28		// future e+=K+	eor	w4,w4,w12+	add	w22,w22,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w24,w24,#2+	eor	w4,w4,w17+	add	w21,w21,w3	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w4,w4,#31+	orr	w25,w23,w24+	and	w26,w23,w24+	eor	w5,w5,w7+	ror	w27,w22,#27+	and	w25,w25,w20+	add	w20,w20,w28		// future e+=K+	eor	w5,w5,w13+	add	w21,w21,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w23,w23,#2+	eor	w5,w5,w19+	add	w20,w20,w4	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w5,w5,#31+	orr	w25,w22,w23+	and	w26,w22,w23+	eor	w6,w6,w8+	ror	w27,w21,#27+	and	w25,w25,w24+	add	w24,w24,w28		// future e+=K+	eor	w6,w6,w14+	add	w20,w20,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w22,w22,#2+	eor	w6,w6,w3+	add	w24,w24,w5	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w6,w6,#31+	orr	w25,w21,w22+	and	w26,w21,w22+	eor	w7,w7,w9+	ror	w27,w20,#27+	and	w25,w25,w23+	add	w23,w23,w28		// future e+=K+	eor	w7,w7,w15+	add	w24,w24,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w21,w21,#2+	eor	w7,w7,w4+	add	w23,w23,w6	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w7,w7,#31+	orr	w25,w20,w21+	and	w26,w20,w21+	eor	w8,w8,w10+	ror	w27,w24,#27+	and	w25,w25,w22+	add	w22,w22,w28		// future e+=K+	eor	w8,w8,w16+	add	w23,w23,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w20,w20,#2+	eor	w8,w8,w5+	add	w22,w22,w7	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w8,w8,#31+	orr	w25,w24,w20+	and	w26,w24,w20+	eor	w9,w9,w11+	ror	w27,w23,#27+	and	w25,w25,w21+	add	w21,w21,w28		// future e+=K+	eor	w9,w9,w17+	add	w22,w22,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w24,w24,#2+	eor	w9,w9,w6+	add	w21,w21,w8	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w9,w9,#31+	orr	w25,w23,w24+	and	w26,w23,w24+	eor	w10,w10,w12+	ror	w27,w22,#27+	and	w25,w25,w20+	add	w20,w20,w28		// future e+=K+	eor	w10,w10,w19+	add	w21,w21,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w23,w23,#2+	eor	w10,w10,w7+	add	w20,w20,w9	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w10,w10,#31+	orr	w25,w22,w23+	and	w26,w22,w23+	eor	w11,w11,w13+	ror	w27,w21,#27+	and	w25,w25,w24+	add	w24,w24,w28		// future e+=K+	eor	w11,w11,w3+	add	w20,w20,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w22,w22,#2+	eor	w11,w11,w8+	add	w24,w24,w10	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w11,w11,#31+	orr	w25,w21,w22+	and	w26,w21,w22+	eor	w12,w12,w14+	ror	w27,w20,#27+	and	w25,w25,w23+	add	w23,w23,w28		// future e+=K+	eor	w12,w12,w4+	add	w24,w24,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w21,w21,#2+	eor	w12,w12,w9+	add	w23,w23,w11	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w12,w12,#31+	orr	w25,w20,w21+	and	w26,w20,w21+	eor	w13,w13,w15+	ror	w27,w24,#27+	and	w25,w25,w22+	add	w22,w22,w28		// future e+=K+	eor	w13,w13,w5+	add	w23,w23,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w20,w20,#2+	eor	w13,w13,w10+	add	w22,w22,w12	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w13,w13,#31+	orr	w25,w24,w20+	and	w26,w24,w20+	eor	w14,w14,w16+	ror	w27,w23,#27+	and	w25,w25,w21+	add	w21,w21,w28		// future e+=K+	eor	w14,w14,w6+	add	w22,w22,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w24,w24,#2+	eor	w14,w14,w11+	add	w21,w21,w13	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w14,w14,#31+	orr	w25,w23,w24+	and	w26,w23,w24+	eor	w15,w15,w17+	ror	w27,w22,#27+	and	w25,w25,w20+	add	w20,w20,w28		// future e+=K+	eor	w15,w15,w7+	add	w21,w21,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w23,w23,#2+	eor	w15,w15,w12+	add	w20,w20,w14	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w15,w15,#31+	movz	w28,#0xc1d6+	movk	w28,#0xca62,lsl#16+	orr	w25,w22,w23+	and	w26,w22,w23+	eor	w16,w16,w19+	ror	w27,w21,#27+	and	w25,w25,w24+	add	w24,w24,w28		// future e+=K+	eor	w16,w16,w8+	add	w20,w20,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w22,w22,#2+	eor	w16,w16,w13+	add	w24,w24,w15	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w16,w16,#31+	eor	w17,w17,w3+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w17,w17,w9+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w17,w17,w14+	add	w23,w23,w16	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w17,w17,#31+	eor	w19,w19,w4+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w19,w19,w10+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w19,w19,w15+	add	w22,w22,w17	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w19,w19,#31+	eor	w3,w3,w5+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w3,w3,w11+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w3,w3,w16+	add	w21,w21,w19	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w3,w3,#31+	eor	w4,w4,w6+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w4,w4,w12+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	eor	w4,w4,w17+	add	w20,w20,w3	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w4,w4,#31+	eor	w5,w5,w7+	eor	w25,w24,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	eor	w5,w5,w13+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	eor	w5,w5,w19+	add	w24,w24,w4	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w5,w5,#31+	eor	w6,w6,w8+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w6,w6,w14+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w6,w6,w3+	add	w23,w23,w5	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w6,w6,#31+	eor	w7,w7,w9+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w7,w7,w15+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w7,w7,w4+	add	w22,w22,w6	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w7,w7,#31+	eor	w8,w8,w10+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w8,w8,w16+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w8,w8,w5+	add	w21,w21,w7	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w8,w8,#31+	eor	w9,w9,w11+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w9,w9,w17+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	eor	w9,w9,w6+	add	w20,w20,w8	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w9,w9,#31+	eor	w10,w10,w12+	eor	w25,w24,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	eor	w10,w10,w19+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	eor	w10,w10,w7+	add	w24,w24,w9	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w10,w10,#31+	eor	w11,w11,w13+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w11,w11,w3+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w11,w11,w8+	add	w23,w23,w10	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w11,w11,#31+	eor	w12,w12,w14+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w12,w12,w4+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w12,w12,w9+	add	w22,w22,w11	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w12,w12,#31+	eor	w13,w13,w15+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w13,w13,w5+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w13,w13,w10+	add	w21,w21,w12	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w13,w13,#31+	eor	w14,w14,w16+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w14,w14,w6+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	eor	w14,w14,w11+	add	w20,w20,w13	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w14,w14,#31+	eor	w15,w15,w17+	eor	w25,w24,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	eor	w15,w15,w7+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	eor	w15,w15,w12+	add	w24,w24,w14	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w15,w15,#31+	eor	w16,w16,w19+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w16,w16,w8+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w16,w16,w13+	add	w23,w23,w15	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w16,w16,#31+	eor	w17,w17,w3+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w17,w17,w9+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w17,w17,w14+	add	w22,w22,w16	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w17,w17,#31+	eor	w19,w19,w4+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w19,w19,w10+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w19,w19,w15+	add	w21,w21,w17	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w19,w19,#31+	ldp	w4,w5,[x0]+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	add	w20,w20,w19	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ldp	w6,w7,[x0,#8]+	eor	w25,w24,w22+	ror	w27,w21,#27+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	ldr	w8,[x0,#16]+	add	w20,w20,w25		// e+=F(b,c,d)+	add	w21,w21,w5+	add	w22,w22,w6+	add	w20,w20,w4+	add	w23,w23,w7+	add	w24,w24,w8+	stp	w20,w21,[x0]+	stp	w22,w23,[x0,#8]+	str	w24,[x0,#16]+	cbnz	x2,Loop++	ldp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	ldp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[sp,#10*__SIZEOF_POINTER__]+	ldr	x29,[sp],#12*__SIZEOF_POINTER__+	ret+++.align	6+_crypton_sha1_asm_block_armv8:+Lv8_entry:+	stp	x29,x30,[sp,#-16]!+	add	x29,sp,#0++	adr	x4,Lconst+	eor	v1.16b,v1.16b,v1.16b+	ld1	{v0.4s},[x0],#16+	ld1	{v1.s}[0],[x0]+	sub	x0,x0,#16+	ld1	{v16.4s,v17.4s,v18.4s,v19.4s},[x4]++Loop_hw:+	ld1	{v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64+	sub	x2,x2,#1+	rev32	v4.16b,v4.16b+	rev32	v5.16b,v5.16b++	add	v20.4s,v16.4s,v4.4s+	rev32	v6.16b,v6.16b+	orr	v22.16b,v0.16b,v0.16b	// offload++	add	v21.4s,v16.4s,v5.4s+	rev32	v7.16b,v7.16b+.long	0x5e280803	//sha1h v3.16b,v0.16b+.long	0x5e140020	//sha1c v0.16b,v1.16b,v20.4s		// 0+	add	v20.4s,v16.4s,v6.4s+.long	0x5e0630a4	//sha1su0 v4.16b,v5.16b,v6.16b+.long	0x5e280802	//sha1h v2.16b,v0.16b		// 1+.long	0x5e150060	//sha1c v0.16b,v3.16b,v21.4s+	add	v21.4s,v16.4s,v7.4s+.long	0x5e2818e4	//sha1su1 v4.16b,v7.16b+.long	0x5e0730c5	//sha1su0 v5.16b,v6.16b,v7.16b+.long	0x5e280803	//sha1h v3.16b,v0.16b		// 2+.long	0x5e140040	//sha1c v0.16b,v2.16b,v20.4s+	add	v20.4s,v16.4s,v4.4s+.long	0x5e281885	//sha1su1 v5.16b,v4.16b+.long	0x5e0430e6	//sha1su0 v6.16b,v7.16b,v4.16b+.long	0x5e280802	//sha1h v2.16b,v0.16b		// 3+.long	0x5e150060	//sha1c v0.16b,v3.16b,v21.4s+	add	v21.4s,v17.4s,v5.4s+.long	0x5e2818a6	//sha1su1 v6.16b,v5.16b+.long	0x5e053087	//sha1su0 v7.16b,v4.16b,v5.16b+.long	0x5e280803	//sha1h v3.16b,v0.16b		// 4+.long	0x5e140040	//sha1c v0.16b,v2.16b,v20.4s+	add	v20.4s,v17.4s,v6.4s+.long	0x5e2818c7	//sha1su1 v7.16b,v6.16b+.long	0x5e0630a4	//sha1su0 v4.16b,v5.16b,v6.16b+.long	0x5e280802	//sha1h v2.16b,v0.16b		// 5+.long	0x5e151060	//sha1p v0.16b,v3.16b,v21.4s+	add	v21.4s,v17.4s,v7.4s+.long	0x5e2818e4	//sha1su1 v4.16b,v7.16b+.long	0x5e0730c5	//sha1su0 v5.16b,v6.16b,v7.16b+.long	0x5e280803	//sha1h v3.16b,v0.16b		// 6+.long	0x5e141040	//sha1p v0.16b,v2.16b,v20.4s+	add	v20.4s,v17.4s,v4.4s+.long	0x5e281885	//sha1su1 v5.16b,v4.16b+.long	0x5e0430e6	//sha1su0 v6.16b,v7.16b,v4.16b+.long	0x5e280802	//sha1h v2.16b,v0.16b		// 7+.long	0x5e151060	//sha1p v0.16b,v3.16b,v21.4s+	add	v21.4s,v17.4s,v5.4s+.long	0x5e2818a6	//sha1su1 v6.16b,v5.16b+.long	0x5e053087	//sha1su0 v7.16b,v4.16b,v5.16b+.long	0x5e280803	//sha1h v3.16b,v0.16b		// 8+.long	0x5e141040	//sha1p v0.16b,v2.16b,v20.4s+	add	v20.4s,v18.4s,v6.4s+.long	0x5e2818c7	//sha1su1 v7.16b,v6.16b+.long	0x5e0630a4	//sha1su0 v4.16b,v5.16b,v6.16b+.long	0x5e280802	//sha1h v2.16b,v0.16b		// 9+.long	0x5e151060	//sha1p v0.16b,v3.16b,v21.4s+	add	v21.4s,v18.4s,v7.4s+.long	0x5e2818e4	//sha1su1 v4.16b,v7.16b+.long	0x5e0730c5	//sha1su0 v5.16b,v6.16b,v7.16b+.long	0x5e280803	//sha1h v3.16b,v0.16b		// 10+.long	0x5e142040	//sha1m v0.16b,v2.16b,v20.4s+	add	v20.4s,v18.4s,v4.4s+.long	0x5e281885	//sha1su1 v5.16b,v4.16b+.long	0x5e0430e6	//sha1su0 v6.16b,v7.16b,v4.16b+.long	0x5e280802	//sha1h v2.16b,v0.16b		// 11+.long	0x5e152060	//sha1m v0.16b,v3.16b,v21.4s+	add	v21.4s,v18.4s,v5.4s+.long	0x5e2818a6	//sha1su1 v6.16b,v5.16b+.long	0x5e053087	//sha1su0 v7.16b,v4.16b,v5.16b+.long	0x5e280803	//sha1h v3.16b,v0.16b		// 12+.long	0x5e142040	//sha1m v0.16b,v2.16b,v20.4s+	add	v20.4s,v18.4s,v6.4s+.long	0x5e2818c7	//sha1su1 v7.16b,v6.16b+.long	0x5e0630a4	//sha1su0 v4.16b,v5.16b,v6.16b+.long	0x5e280802	//sha1h v2.16b,v0.16b		// 13+.long	0x5e152060	//sha1m v0.16b,v3.16b,v21.4s+	add	v21.4s,v19.4s,v7.4s+.long	0x5e2818e4	//sha1su1 v4.16b,v7.16b+.long	0x5e0730c5	//sha1su0 v5.16b,v6.16b,v7.16b+.long	0x5e280803	//sha1h v3.16b,v0.16b		// 14+.long	0x5e142040	//sha1m v0.16b,v2.16b,v20.4s+	add	v20.4s,v19.4s,v4.4s+.long	0x5e281885	//sha1su1 v5.16b,v4.16b+.long	0x5e0430e6	//sha1su0 v6.16b,v7.16b,v4.16b+.long	0x5e280802	//sha1h v2.16b,v0.16b		// 15+.long	0x5e151060	//sha1p v0.16b,v3.16b,v21.4s+	add	v21.4s,v19.4s,v5.4s+.long	0x5e2818a6	//sha1su1 v6.16b,v5.16b+.long	0x5e053087	//sha1su0 v7.16b,v4.16b,v5.16b+.long	0x5e280803	//sha1h v3.16b,v0.16b		// 16+.long	0x5e141040	//sha1p v0.16b,v2.16b,v20.4s+	add	v20.4s,v19.4s,v6.4s+.long	0x5e2818c7	//sha1su1 v7.16b,v6.16b+.long	0x5e280802	//sha1h v2.16b,v0.16b		// 17+.long	0x5e151060	//sha1p v0.16b,v3.16b,v21.4s+	add	v21.4s,v19.4s,v7.4s++.long	0x5e280803	//sha1h v3.16b,v0.16b		// 18+.long	0x5e141040	//sha1p v0.16b,v2.16b,v20.4s++.long	0x5e280802	//sha1h v2.16b,v0.16b		// 19+.long	0x5e151060	//sha1p v0.16b,v3.16b,v21.4s++	add	v1.4s,v1.4s,v2.4s+	add	v0.4s,v0.4s,v22.4s++	cbnz	x2,Loop_hw++	st1	{v0.4s},[x0],#16+	st1	{v1.s}[0],[x0]++	ldr	x29,[sp],#16+	ret++.align	6+Lconst:+.long	0x5a827999,0x5a827999,0x5a827999,0x5a827999	//K_00_19+.long	0x6ed9eba1,0x6ed9eba1,0x6ed9eba1,0x6ed9eba1	//K_20_39+.long	0x8f1bbcdc,0x8f1bbcdc,0x8f1bbcdc,0x8f1bbcdc	//K_40_59+.long	0xca62c1d6,0xca62c1d6,0xca62c1d6,0xca62c1d6	//K_60_79+.byte	83,72,65,49,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.align	2+.align	2+#if !defined(__KERNELL__) && !defined(_WIN64)+.comm	__crypton_armcap_P,4+.private_extern	_crypton_armcap_P+#endif
+ cbits/asm/sha1-armv8-linux64.S view
@@ -0,0 +1,1218 @@+#ifndef	__KERNEL__+# include "arm_arch.h"++#else+.globl	crypton_sha1_asm_block_armv8+#endif++.text++.globl	crypton_sha1_asm_block_data_order+.type	crypton_sha1_asm_block_data_order,%function+.align	6+crypton_sha1_asm_block_data_order:+	adrp	x16,crypton_armcap_P+	ldr	w16,[x16,#:lo12:crypton_armcap_P]+	tst	w16,#ARMV8_SHA1+	b.ne	.Lv8_entry++	stp	x29,x30,[sp,#-12*__SIZEOF_POINTER__]!+	add	x29,sp,#0+	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	stp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	stp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	stp	x27,x28,[sp,#10*__SIZEOF_POINTER__]++	ldp	w20,w21,[x0]+	ldp	w22,w23,[x0,#8]+	ldr	w24,[x0,#16]++.Loop:+	ldr	x3,[x1],#64+	movz	w28,#0x7999+	sub	x2,x2,#1+	movk	w28,#0x5a82,lsl#16+#ifdef	__AARCH64EB__+	ror	x3,x3,#32+#else+	rev32	x3,x3+#endif+	add	w24,w24,w28		// warm it up+	add	w24,w24,w3+	lsr	x4,x3,#32+	ldur	x5,[x1,#-56]+	bic	w25,w23,w21+	and	w26,w22,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	orr	w25,w25,w26+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	add	w23,w23,w4	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+#ifdef	__AARCH64EB__+	ror	x5,x5,#32+#else+	rev32	x5,x5+#endif+	bic	w25,w22,w20+	and	w26,w21,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	orr	w25,w25,w26+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	add	w22,w22,w5	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	lsr	x6,x5,#32+	ldur	x7,[x1,#-48]+	bic	w25,w21,w24+	and	w26,w20,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	orr	w25,w25,w26+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	add	w21,w21,w6	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+#ifdef	__AARCH64EB__+	ror	x7,x7,#32+#else+	rev32	x7,x7+#endif+	bic	w25,w20,w23+	and	w26,w24,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	orr	w25,w25,w26+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	add	w20,w20,w7	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	lsr	x8,x7,#32+	ldur	x9,[x1,#-40]+	bic	w25,w24,w22+	and	w26,w23,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	orr	w25,w25,w26+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	add	w24,w24,w8	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+#ifdef	__AARCH64EB__+	ror	x9,x9,#32+#else+	rev32	x9,x9+#endif+	bic	w25,w23,w21+	and	w26,w22,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	orr	w25,w25,w26+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	add	w23,w23,w9	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	lsr	x10,x9,#32+	ldur	x11,[x1,#-32]+	bic	w25,w22,w20+	and	w26,w21,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	orr	w25,w25,w26+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	add	w22,w22,w10	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+#ifdef	__AARCH64EB__+	ror	x11,x11,#32+#else+	rev32	x11,x11+#endif+	bic	w25,w21,w24+	and	w26,w20,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	orr	w25,w25,w26+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	add	w21,w21,w11	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	lsr	x12,x11,#32+	ldur	x13,[x1,#-24]+	bic	w25,w20,w23+	and	w26,w24,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	orr	w25,w25,w26+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	add	w20,w20,w12	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+#ifdef	__AARCH64EB__+	ror	x13,x13,#32+#else+	rev32	x13,x13+#endif+	bic	w25,w24,w22+	and	w26,w23,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	orr	w25,w25,w26+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	add	w24,w24,w13	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	lsr	x14,x13,#32+	ldur	x15,[x1,#-16]+	bic	w25,w23,w21+	and	w26,w22,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	orr	w25,w25,w26+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	add	w23,w23,w14	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+#ifdef	__AARCH64EB__+	ror	x15,x15,#32+#else+	rev32	x15,x15+#endif+	bic	w25,w22,w20+	and	w26,w21,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	orr	w25,w25,w26+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	add	w22,w22,w15	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	lsr	x16,x15,#32+	ldur	x17,[x1,#-8]+	bic	w25,w21,w24+	and	w26,w20,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	orr	w25,w25,w26+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	add	w21,w21,w16	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+#ifdef	__AARCH64EB__+	ror	x17,x17,#32+#else+	rev32	x17,x17+#endif+	bic	w25,w20,w23+	and	w26,w24,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	orr	w25,w25,w26+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	add	w20,w20,w17	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	lsr	x19,x17,#32+	eor	w3,w3,w5+	bic	w25,w24,w22+	and	w26,w23,w22+	ror	w27,w21,#27+	eor	w3,w3,w11+	add	w24,w24,w28		// future e+=K+	orr	w25,w25,w26+	add	w20,w20,w27		// e+=rot(a,5)+	eor	w3,w3,w16+	ror	w22,w22,#2+	add	w24,w24,w19	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w3,w3,#31+	eor	w4,w4,w6+	bic	w25,w23,w21+	and	w26,w22,w21+	ror	w27,w20,#27+	eor	w4,w4,w12+	add	w23,w23,w28		// future e+=K+	orr	w25,w25,w26+	add	w24,w24,w27		// e+=rot(a,5)+	eor	w4,w4,w17+	ror	w21,w21,#2+	add	w23,w23,w3	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w4,w4,#31+	eor	w5,w5,w7+	bic	w25,w22,w20+	and	w26,w21,w20+	ror	w27,w24,#27+	eor	w5,w5,w13+	add	w22,w22,w28		// future e+=K+	orr	w25,w25,w26+	add	w23,w23,w27		// e+=rot(a,5)+	eor	w5,w5,w19+	ror	w20,w20,#2+	add	w22,w22,w4	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w5,w5,#31+	eor	w6,w6,w8+	bic	w25,w21,w24+	and	w26,w20,w24+	ror	w27,w23,#27+	eor	w6,w6,w14+	add	w21,w21,w28		// future e+=K+	orr	w25,w25,w26+	add	w22,w22,w27		// e+=rot(a,5)+	eor	w6,w6,w3+	ror	w24,w24,#2+	add	w21,w21,w5	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w6,w6,#31+	eor	w7,w7,w9+	bic	w25,w20,w23+	and	w26,w24,w23+	ror	w27,w22,#27+	eor	w7,w7,w15+	add	w20,w20,w28		// future e+=K+	orr	w25,w25,w26+	add	w21,w21,w27		// e+=rot(a,5)+	eor	w7,w7,w4+	ror	w23,w23,#2+	add	w20,w20,w6	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w7,w7,#31+	movz	w28,#0xeba1+	movk	w28,#0x6ed9,lsl#16+	eor	w8,w8,w10+	bic	w25,w24,w22+	and	w26,w23,w22+	ror	w27,w21,#27+	eor	w8,w8,w16+	add	w24,w24,w28		// future e+=K+	orr	w25,w25,w26+	add	w20,w20,w27		// e+=rot(a,5)+	eor	w8,w8,w5+	ror	w22,w22,#2+	add	w24,w24,w7	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w8,w8,#31+	eor	w9,w9,w11+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w9,w9,w17+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w9,w9,w6+	add	w23,w23,w8	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w9,w9,#31+	eor	w10,w10,w12+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w10,w10,w19+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w10,w10,w7+	add	w22,w22,w9	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w10,w10,#31+	eor	w11,w11,w13+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w11,w11,w3+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w11,w11,w8+	add	w21,w21,w10	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w11,w11,#31+	eor	w12,w12,w14+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w12,w12,w4+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	eor	w12,w12,w9+	add	w20,w20,w11	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w12,w12,#31+	eor	w13,w13,w15+	eor	w25,w24,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	eor	w13,w13,w5+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	eor	w13,w13,w10+	add	w24,w24,w12	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w13,w13,#31+	eor	w14,w14,w16+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w14,w14,w6+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w14,w14,w11+	add	w23,w23,w13	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w14,w14,#31+	eor	w15,w15,w17+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w15,w15,w7+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w15,w15,w12+	add	w22,w22,w14	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w15,w15,#31+	eor	w16,w16,w19+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w16,w16,w8+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w16,w16,w13+	add	w21,w21,w15	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w16,w16,#31+	eor	w17,w17,w3+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w17,w17,w9+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	eor	w17,w17,w14+	add	w20,w20,w16	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w17,w17,#31+	eor	w19,w19,w4+	eor	w25,w24,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	eor	w19,w19,w10+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	eor	w19,w19,w15+	add	w24,w24,w17	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w19,w19,#31+	eor	w3,w3,w5+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w3,w3,w11+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w3,w3,w16+	add	w23,w23,w19	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w3,w3,#31+	eor	w4,w4,w6+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w4,w4,w12+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w4,w4,w17+	add	w22,w22,w3	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w4,w4,#31+	eor	w5,w5,w7+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w5,w5,w13+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w5,w5,w19+	add	w21,w21,w4	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w5,w5,#31+	eor	w6,w6,w8+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w6,w6,w14+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	eor	w6,w6,w3+	add	w20,w20,w5	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w6,w6,#31+	eor	w7,w7,w9+	eor	w25,w24,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	eor	w7,w7,w15+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	eor	w7,w7,w4+	add	w24,w24,w6	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w7,w7,#31+	eor	w8,w8,w10+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w8,w8,w16+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w8,w8,w5+	add	w23,w23,w7	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w8,w8,#31+	eor	w9,w9,w11+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w9,w9,w17+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w9,w9,w6+	add	w22,w22,w8	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w9,w9,#31+	eor	w10,w10,w12+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w10,w10,w19+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w10,w10,w7+	add	w21,w21,w9	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w10,w10,#31+	eor	w11,w11,w13+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w11,w11,w3+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	eor	w11,w11,w8+	add	w20,w20,w10	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w11,w11,#31+	movz	w28,#0xbcdc+	movk	w28,#0x8f1b,lsl#16+	eor	w12,w12,w14+	eor	w25,w24,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	eor	w12,w12,w4+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	eor	w12,w12,w9+	add	w24,w24,w11	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w12,w12,#31+	orr	w25,w21,w22+	and	w26,w21,w22+	eor	w13,w13,w15+	ror	w27,w20,#27+	and	w25,w25,w23+	add	w23,w23,w28		// future e+=K+	eor	w13,w13,w5+	add	w24,w24,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w21,w21,#2+	eor	w13,w13,w10+	add	w23,w23,w12	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w13,w13,#31+	orr	w25,w20,w21+	and	w26,w20,w21+	eor	w14,w14,w16+	ror	w27,w24,#27+	and	w25,w25,w22+	add	w22,w22,w28		// future e+=K+	eor	w14,w14,w6+	add	w23,w23,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w20,w20,#2+	eor	w14,w14,w11+	add	w22,w22,w13	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w14,w14,#31+	orr	w25,w24,w20+	and	w26,w24,w20+	eor	w15,w15,w17+	ror	w27,w23,#27+	and	w25,w25,w21+	add	w21,w21,w28		// future e+=K+	eor	w15,w15,w7+	add	w22,w22,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w24,w24,#2+	eor	w15,w15,w12+	add	w21,w21,w14	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w15,w15,#31+	orr	w25,w23,w24+	and	w26,w23,w24+	eor	w16,w16,w19+	ror	w27,w22,#27+	and	w25,w25,w20+	add	w20,w20,w28		// future e+=K+	eor	w16,w16,w8+	add	w21,w21,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w23,w23,#2+	eor	w16,w16,w13+	add	w20,w20,w15	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w16,w16,#31+	orr	w25,w22,w23+	and	w26,w22,w23+	eor	w17,w17,w3+	ror	w27,w21,#27+	and	w25,w25,w24+	add	w24,w24,w28		// future e+=K+	eor	w17,w17,w9+	add	w20,w20,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w22,w22,#2+	eor	w17,w17,w14+	add	w24,w24,w16	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w17,w17,#31+	orr	w25,w21,w22+	and	w26,w21,w22+	eor	w19,w19,w4+	ror	w27,w20,#27+	and	w25,w25,w23+	add	w23,w23,w28		// future e+=K+	eor	w19,w19,w10+	add	w24,w24,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w21,w21,#2+	eor	w19,w19,w15+	add	w23,w23,w17	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w19,w19,#31+	orr	w25,w20,w21+	and	w26,w20,w21+	eor	w3,w3,w5+	ror	w27,w24,#27+	and	w25,w25,w22+	add	w22,w22,w28		// future e+=K+	eor	w3,w3,w11+	add	w23,w23,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w20,w20,#2+	eor	w3,w3,w16+	add	w22,w22,w19	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w3,w3,#31+	orr	w25,w24,w20+	and	w26,w24,w20+	eor	w4,w4,w6+	ror	w27,w23,#27+	and	w25,w25,w21+	add	w21,w21,w28		// future e+=K+	eor	w4,w4,w12+	add	w22,w22,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w24,w24,#2+	eor	w4,w4,w17+	add	w21,w21,w3	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w4,w4,#31+	orr	w25,w23,w24+	and	w26,w23,w24+	eor	w5,w5,w7+	ror	w27,w22,#27+	and	w25,w25,w20+	add	w20,w20,w28		// future e+=K+	eor	w5,w5,w13+	add	w21,w21,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w23,w23,#2+	eor	w5,w5,w19+	add	w20,w20,w4	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w5,w5,#31+	orr	w25,w22,w23+	and	w26,w22,w23+	eor	w6,w6,w8+	ror	w27,w21,#27+	and	w25,w25,w24+	add	w24,w24,w28		// future e+=K+	eor	w6,w6,w14+	add	w20,w20,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w22,w22,#2+	eor	w6,w6,w3+	add	w24,w24,w5	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w6,w6,#31+	orr	w25,w21,w22+	and	w26,w21,w22+	eor	w7,w7,w9+	ror	w27,w20,#27+	and	w25,w25,w23+	add	w23,w23,w28		// future e+=K+	eor	w7,w7,w15+	add	w24,w24,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w21,w21,#2+	eor	w7,w7,w4+	add	w23,w23,w6	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w7,w7,#31+	orr	w25,w20,w21+	and	w26,w20,w21+	eor	w8,w8,w10+	ror	w27,w24,#27+	and	w25,w25,w22+	add	w22,w22,w28		// future e+=K+	eor	w8,w8,w16+	add	w23,w23,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w20,w20,#2+	eor	w8,w8,w5+	add	w22,w22,w7	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w8,w8,#31+	orr	w25,w24,w20+	and	w26,w24,w20+	eor	w9,w9,w11+	ror	w27,w23,#27+	and	w25,w25,w21+	add	w21,w21,w28		// future e+=K+	eor	w9,w9,w17+	add	w22,w22,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w24,w24,#2+	eor	w9,w9,w6+	add	w21,w21,w8	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w9,w9,#31+	orr	w25,w23,w24+	and	w26,w23,w24+	eor	w10,w10,w12+	ror	w27,w22,#27+	and	w25,w25,w20+	add	w20,w20,w28		// future e+=K+	eor	w10,w10,w19+	add	w21,w21,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w23,w23,#2+	eor	w10,w10,w7+	add	w20,w20,w9	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w10,w10,#31+	orr	w25,w22,w23+	and	w26,w22,w23+	eor	w11,w11,w13+	ror	w27,w21,#27+	and	w25,w25,w24+	add	w24,w24,w28		// future e+=K+	eor	w11,w11,w3+	add	w20,w20,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w22,w22,#2+	eor	w11,w11,w8+	add	w24,w24,w10	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w11,w11,#31+	orr	w25,w21,w22+	and	w26,w21,w22+	eor	w12,w12,w14+	ror	w27,w20,#27+	and	w25,w25,w23+	add	w23,w23,w28		// future e+=K+	eor	w12,w12,w4+	add	w24,w24,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w21,w21,#2+	eor	w12,w12,w9+	add	w23,w23,w11	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w12,w12,#31+	orr	w25,w20,w21+	and	w26,w20,w21+	eor	w13,w13,w15+	ror	w27,w24,#27+	and	w25,w25,w22+	add	w22,w22,w28		// future e+=K+	eor	w13,w13,w5+	add	w23,w23,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w20,w20,#2+	eor	w13,w13,w10+	add	w22,w22,w12	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w13,w13,#31+	orr	w25,w24,w20+	and	w26,w24,w20+	eor	w14,w14,w16+	ror	w27,w23,#27+	and	w25,w25,w21+	add	w21,w21,w28		// future e+=K+	eor	w14,w14,w6+	add	w22,w22,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w24,w24,#2+	eor	w14,w14,w11+	add	w21,w21,w13	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w14,w14,#31+	orr	w25,w23,w24+	and	w26,w23,w24+	eor	w15,w15,w17+	ror	w27,w22,#27+	and	w25,w25,w20+	add	w20,w20,w28		// future e+=K+	eor	w15,w15,w7+	add	w21,w21,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w23,w23,#2+	eor	w15,w15,w12+	add	w20,w20,w14	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w15,w15,#31+	movz	w28,#0xc1d6+	movk	w28,#0xca62,lsl#16+	orr	w25,w22,w23+	and	w26,w22,w23+	eor	w16,w16,w19+	ror	w27,w21,#27+	and	w25,w25,w24+	add	w24,w24,w28		// future e+=K+	eor	w16,w16,w8+	add	w20,w20,w27		// e+=rot(a,5)+	orr	w25,w25,w26+	ror	w22,w22,#2+	eor	w16,w16,w13+	add	w24,w24,w15	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w16,w16,#31+	eor	w17,w17,w3+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w17,w17,w9+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w17,w17,w14+	add	w23,w23,w16	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w17,w17,#31+	eor	w19,w19,w4+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w19,w19,w10+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w19,w19,w15+	add	w22,w22,w17	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w19,w19,#31+	eor	w3,w3,w5+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w3,w3,w11+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w3,w3,w16+	add	w21,w21,w19	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w3,w3,#31+	eor	w4,w4,w6+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w4,w4,w12+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	eor	w4,w4,w17+	add	w20,w20,w3	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w4,w4,#31+	eor	w5,w5,w7+	eor	w25,w24,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	eor	w5,w5,w13+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	eor	w5,w5,w19+	add	w24,w24,w4	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w5,w5,#31+	eor	w6,w6,w8+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w6,w6,w14+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w6,w6,w3+	add	w23,w23,w5	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w6,w6,#31+	eor	w7,w7,w9+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w7,w7,w15+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w7,w7,w4+	add	w22,w22,w6	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w7,w7,#31+	eor	w8,w8,w10+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w8,w8,w16+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w8,w8,w5+	add	w21,w21,w7	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w8,w8,#31+	eor	w9,w9,w11+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w9,w9,w17+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	eor	w9,w9,w6+	add	w20,w20,w8	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w9,w9,#31+	eor	w10,w10,w12+	eor	w25,w24,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	eor	w10,w10,w19+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	eor	w10,w10,w7+	add	w24,w24,w9	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w10,w10,#31+	eor	w11,w11,w13+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w11,w11,w3+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w11,w11,w8+	add	w23,w23,w10	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w11,w11,#31+	eor	w12,w12,w14+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w12,w12,w4+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w12,w12,w9+	add	w22,w22,w11	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w12,w12,#31+	eor	w13,w13,w15+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w13,w13,w5+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w13,w13,w10+	add	w21,w21,w12	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w13,w13,#31+	eor	w14,w14,w16+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w14,w14,w6+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	eor	w14,w14,w11+	add	w20,w20,w13	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ror	w14,w14,#31+	eor	w15,w15,w17+	eor	w25,w24,w22+	ror	w27,w21,#27+	add	w24,w24,w28		// future e+=K+	eor	w15,w15,w7+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	eor	w15,w15,w12+	add	w24,w24,w14	// future e+=X[i]+	add	w20,w20,w25		// e+=F(b,c,d)+	ror	w15,w15,#31+	eor	w16,w16,w19+	eor	w25,w23,w21+	ror	w27,w20,#27+	add	w23,w23,w28		// future e+=K+	eor	w16,w16,w8+	eor	w25,w25,w22+	add	w24,w24,w27		// e+=rot(a,5)+	ror	w21,w21,#2+	eor	w16,w16,w13+	add	w23,w23,w15	// future e+=X[i]+	add	w24,w24,w25		// e+=F(b,c,d)+	ror	w16,w16,#31+	eor	w17,w17,w3+	eor	w25,w22,w20+	ror	w27,w24,#27+	add	w22,w22,w28		// future e+=K+	eor	w17,w17,w9+	eor	w25,w25,w21+	add	w23,w23,w27		// e+=rot(a,5)+	ror	w20,w20,#2+	eor	w17,w17,w14+	add	w22,w22,w16	// future e+=X[i]+	add	w23,w23,w25		// e+=F(b,c,d)+	ror	w17,w17,#31+	eor	w19,w19,w4+	eor	w25,w21,w24+	ror	w27,w23,#27+	add	w21,w21,w28		// future e+=K+	eor	w19,w19,w10+	eor	w25,w25,w20+	add	w22,w22,w27		// e+=rot(a,5)+	ror	w24,w24,#2+	eor	w19,w19,w15+	add	w21,w21,w17	// future e+=X[i]+	add	w22,w22,w25		// e+=F(b,c,d)+	ror	w19,w19,#31+	ldp	w4,w5,[x0]+	eor	w25,w20,w23+	ror	w27,w22,#27+	add	w20,w20,w28		// future e+=K+	eor	w25,w25,w24+	add	w21,w21,w27		// e+=rot(a,5)+	ror	w23,w23,#2+	add	w20,w20,w19	// future e+=X[i]+	add	w21,w21,w25		// e+=F(b,c,d)+	ldp	w6,w7,[x0,#8]+	eor	w25,w24,w22+	ror	w27,w21,#27+	eor	w25,w25,w23+	add	w20,w20,w27		// e+=rot(a,5)+	ror	w22,w22,#2+	ldr	w8,[x0,#16]+	add	w20,w20,w25		// e+=F(b,c,d)+	add	w21,w21,w5+	add	w22,w22,w6+	add	w20,w20,w4+	add	w23,w23,w7+	add	w24,w24,w8+	stp	w20,w21,[x0]+	stp	w22,w23,[x0,#8]+	str	w24,[x0,#16]+	cbnz	x2,.Loop++	ldp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	ldp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[sp,#10*__SIZEOF_POINTER__]+	ldr	x29,[sp],#12*__SIZEOF_POINTER__+	ret+.size	crypton_sha1_asm_block_data_order,.-crypton_sha1_asm_block_data_order+.type	crypton_sha1_asm_block_armv8,%function+.align	6+crypton_sha1_asm_block_armv8:+.Lv8_entry:+	stp	x29,x30,[sp,#-16]!+	add	x29,sp,#0++	adr	x4,.Lconst+	eor	v1.16b,v1.16b,v1.16b+	ld1	{v0.4s},[x0],#16+	ld1	{v1.s}[0],[x0]+	sub	x0,x0,#16+	ld1	{v16.4s,v17.4s,v18.4s,v19.4s},[x4]++.Loop_hw:+	ld1	{v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64+	sub	x2,x2,#1+	rev32	v4.16b,v4.16b+	rev32	v5.16b,v5.16b++	add	v20.4s,v16.4s,v4.4s+	rev32	v6.16b,v6.16b+	orr	v22.16b,v0.16b,v0.16b	// offload++	add	v21.4s,v16.4s,v5.4s+	rev32	v7.16b,v7.16b+.inst	0x5e280803	//sha1h v3.16b,v0.16b+.inst	0x5e140020	//sha1c v0.16b,v1.16b,v20.4s		// 0+	add	v20.4s,v16.4s,v6.4s+.inst	0x5e0630a4	//sha1su0 v4.16b,v5.16b,v6.16b+.inst	0x5e280802	//sha1h v2.16b,v0.16b		// 1+.inst	0x5e150060	//sha1c v0.16b,v3.16b,v21.4s+	add	v21.4s,v16.4s,v7.4s+.inst	0x5e2818e4	//sha1su1 v4.16b,v7.16b+.inst	0x5e0730c5	//sha1su0 v5.16b,v6.16b,v7.16b+.inst	0x5e280803	//sha1h v3.16b,v0.16b		// 2+.inst	0x5e140040	//sha1c v0.16b,v2.16b,v20.4s+	add	v20.4s,v16.4s,v4.4s+.inst	0x5e281885	//sha1su1 v5.16b,v4.16b+.inst	0x5e0430e6	//sha1su0 v6.16b,v7.16b,v4.16b+.inst	0x5e280802	//sha1h v2.16b,v0.16b		// 3+.inst	0x5e150060	//sha1c v0.16b,v3.16b,v21.4s+	add	v21.4s,v17.4s,v5.4s+.inst	0x5e2818a6	//sha1su1 v6.16b,v5.16b+.inst	0x5e053087	//sha1su0 v7.16b,v4.16b,v5.16b+.inst	0x5e280803	//sha1h v3.16b,v0.16b		// 4+.inst	0x5e140040	//sha1c v0.16b,v2.16b,v20.4s+	add	v20.4s,v17.4s,v6.4s+.inst	0x5e2818c7	//sha1su1 v7.16b,v6.16b+.inst	0x5e0630a4	//sha1su0 v4.16b,v5.16b,v6.16b+.inst	0x5e280802	//sha1h v2.16b,v0.16b		// 5+.inst	0x5e151060	//sha1p v0.16b,v3.16b,v21.4s+	add	v21.4s,v17.4s,v7.4s+.inst	0x5e2818e4	//sha1su1 v4.16b,v7.16b+.inst	0x5e0730c5	//sha1su0 v5.16b,v6.16b,v7.16b+.inst	0x5e280803	//sha1h v3.16b,v0.16b		// 6+.inst	0x5e141040	//sha1p v0.16b,v2.16b,v20.4s+	add	v20.4s,v17.4s,v4.4s+.inst	0x5e281885	//sha1su1 v5.16b,v4.16b+.inst	0x5e0430e6	//sha1su0 v6.16b,v7.16b,v4.16b+.inst	0x5e280802	//sha1h v2.16b,v0.16b		// 7+.inst	0x5e151060	//sha1p v0.16b,v3.16b,v21.4s+	add	v21.4s,v17.4s,v5.4s+.inst	0x5e2818a6	//sha1su1 v6.16b,v5.16b+.inst	0x5e053087	//sha1su0 v7.16b,v4.16b,v5.16b+.inst	0x5e280803	//sha1h v3.16b,v0.16b		// 8+.inst	0x5e141040	//sha1p v0.16b,v2.16b,v20.4s+	add	v20.4s,v18.4s,v6.4s+.inst	0x5e2818c7	//sha1su1 v7.16b,v6.16b+.inst	0x5e0630a4	//sha1su0 v4.16b,v5.16b,v6.16b+.inst	0x5e280802	//sha1h v2.16b,v0.16b		// 9+.inst	0x5e151060	//sha1p v0.16b,v3.16b,v21.4s+	add	v21.4s,v18.4s,v7.4s+.inst	0x5e2818e4	//sha1su1 v4.16b,v7.16b+.inst	0x5e0730c5	//sha1su0 v5.16b,v6.16b,v7.16b+.inst	0x5e280803	//sha1h v3.16b,v0.16b		// 10+.inst	0x5e142040	//sha1m v0.16b,v2.16b,v20.4s+	add	v20.4s,v18.4s,v4.4s+.inst	0x5e281885	//sha1su1 v5.16b,v4.16b+.inst	0x5e0430e6	//sha1su0 v6.16b,v7.16b,v4.16b+.inst	0x5e280802	//sha1h v2.16b,v0.16b		// 11+.inst	0x5e152060	//sha1m v0.16b,v3.16b,v21.4s+	add	v21.4s,v18.4s,v5.4s+.inst	0x5e2818a6	//sha1su1 v6.16b,v5.16b+.inst	0x5e053087	//sha1su0 v7.16b,v4.16b,v5.16b+.inst	0x5e280803	//sha1h v3.16b,v0.16b		// 12+.inst	0x5e142040	//sha1m v0.16b,v2.16b,v20.4s+	add	v20.4s,v18.4s,v6.4s+.inst	0x5e2818c7	//sha1su1 v7.16b,v6.16b+.inst	0x5e0630a4	//sha1su0 v4.16b,v5.16b,v6.16b+.inst	0x5e280802	//sha1h v2.16b,v0.16b		// 13+.inst	0x5e152060	//sha1m v0.16b,v3.16b,v21.4s+	add	v21.4s,v19.4s,v7.4s+.inst	0x5e2818e4	//sha1su1 v4.16b,v7.16b+.inst	0x5e0730c5	//sha1su0 v5.16b,v6.16b,v7.16b+.inst	0x5e280803	//sha1h v3.16b,v0.16b		// 14+.inst	0x5e142040	//sha1m v0.16b,v2.16b,v20.4s+	add	v20.4s,v19.4s,v4.4s+.inst	0x5e281885	//sha1su1 v5.16b,v4.16b+.inst	0x5e0430e6	//sha1su0 v6.16b,v7.16b,v4.16b+.inst	0x5e280802	//sha1h v2.16b,v0.16b		// 15+.inst	0x5e151060	//sha1p v0.16b,v3.16b,v21.4s+	add	v21.4s,v19.4s,v5.4s+.inst	0x5e2818a6	//sha1su1 v6.16b,v5.16b+.inst	0x5e053087	//sha1su0 v7.16b,v4.16b,v5.16b+.inst	0x5e280803	//sha1h v3.16b,v0.16b		// 16+.inst	0x5e141040	//sha1p v0.16b,v2.16b,v20.4s+	add	v20.4s,v19.4s,v6.4s+.inst	0x5e2818c7	//sha1su1 v7.16b,v6.16b+.inst	0x5e280802	//sha1h v2.16b,v0.16b		// 17+.inst	0x5e151060	//sha1p v0.16b,v3.16b,v21.4s+	add	v21.4s,v19.4s,v7.4s++.inst	0x5e280803	//sha1h v3.16b,v0.16b		// 18+.inst	0x5e141040	//sha1p v0.16b,v2.16b,v20.4s++.inst	0x5e280802	//sha1h v2.16b,v0.16b		// 19+.inst	0x5e151060	//sha1p v0.16b,v3.16b,v21.4s++	add	v1.4s,v1.4s,v2.4s+	add	v0.4s,v0.4s,v22.4s++	cbnz	x2,.Loop_hw++	st1	{v0.4s},[x0],#16+	st1	{v1.s}[0],[x0]++	ldr	x29,[sp],#16+	ret+.size	crypton_sha1_asm_block_armv8,.-crypton_sha1_asm_block_armv8+.align	6+.Lconst:+.long	0x5a827999,0x5a827999,0x5a827999,0x5a827999	//K_00_19+.long	0x6ed9eba1,0x6ed9eba1,0x6ed9eba1,0x6ed9eba1	//K_20_39+.long	0x8f1bbcdc,0x8f1bbcdc,0x8f1bbcdc,0x8f1bbcdc	//K_40_59+.long	0xca62c1d6,0xca62c1d6,0xca62c1d6,0xca62c1d6	//K_60_79+.byte	83,72,65,49,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.align	2+.align	2+#if !defined(__KERNELL__) && !defined(_WIN64)+.comm	crypton_armcap_P,4,4+.hidden	crypton_armcap_P+#endif++.section	.note.GNU-stack,"",%progbits
+ cbits/asm/sha1-armv8.pl view
@@ -0,0 +1,362 @@+#!/usr/bin/env perl+#+# ====================================================================+# Written by Andy Polyakov, @dot-asm, initially for the OpenSSL+# project.+# ====================================================================+#+# SHA1 for ARMv8.+#+# Performance in cycles per processed byte and improvement coefficient+# over code generated with "default" compiler:+#+#		hardware-assisted	software(*)+# Apple A7	2.31			4.13 (+14%)+# Apple A10	1.61+# Apple A14/M1	1.32			3.82 (-13%)(***)+# Cortex-A53	2.24			8.03 (+97%)+# Cortex-A57	2.35			7.88 (+74%)+# Cortex-A76	1.64			5.20+# Cortex-X2	1.63			4.07+# Cortex-X925	1.64			3.81+# Denver	2.13			3.97 (+0%)(**)+# X-Gene				8.80 (+200%)+# Mongoose	2.05			6.50 (+160%)+# Kryo		1.88			8.00 (+90%)+# ThunderX2	2.64			6.36 (+150%)+# Snapdraon X	1.48			3.82+#+# (*)	Software results are presented mostly for reference purposes.+# (**)	Keep in mind that Denver relies on binary translation, which+#	optimizes compiler output at run-time.+# (***)	There is some room for improvement on "extra-wide" processor+#	such as A14/M1. Nothing is done, because it's not used anyway.++$flavour = shift;+$output  = shift;++if ($flavour && $flavour ne "void") {+    $0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1;+    ( $xlate="${dir}arm-xlate.pl" and -f $xlate ) or+    ( $xlate="${dir}../../perlasm/arm-xlate.pl" and -f $xlate) or+    die "can't locate arm-xlate.pl";++    open STDOUT,"| \"$^X\" $xlate $flavour $output";+} else {+    open STDOUT,">$output";+}++($ctx,$inp,$num)=("x0","x1","x2");+@Xw=map("w$_",(3..17,19));+@Xx=map("x$_",(3..17,19));+@V=($A,$B,$C,$D,$E)=map("w$_",(20..24));+($t0,$t1,$t2,$K)=map("w$_",(25..28));+++sub BODY_00_19 {+my ($i,$a,$b,$c,$d,$e)=@_;+my $j=($i+2)&15;++$code.=<<___ if ($i<15 && !($i&1));+	lsr	@Xx[$i+1],@Xx[$i],#32+___+$code.=<<___ if ($i<14 && !($i&1));+	ldur	@Xx[$i+2],[$inp,#`($i+2)*4-64`]+___+$code.=<<___ if ($i<14 && ($i&1));+#ifdef	__AARCH64EB__+	ror	@Xx[$i+1],@Xx[$i+1],#32+#else+	rev32	@Xx[$i+1],@Xx[$i+1]+#endif+___+$code.=<<___ if ($i<14);+	bic	$t0,$d,$b+	and	$t1,$c,$b+	ror	$t2,$a,#27+	add	$d,$d,$K		// future e+=K+	orr	$t0,$t0,$t1+	add	$e,$e,$t2		// e+=rot(a,5)+	ror	$b,$b,#2+	add	$d,$d,@Xw[($i+1)&15]	// future e+=X[i]+	add	$e,$e,$t0		// e+=F(b,c,d)+___+$code.=<<___ if ($i==19);+	movz	$K,#0xeba1+	movk	$K,#0x6ed9,lsl#16+___+$code.=<<___ if ($i>=14);+	 eor	@Xw[$j],@Xw[$j],@Xw[($j+2)&15]+	bic	$t0,$d,$b+	and	$t1,$c,$b+	ror	$t2,$a,#27+	 eor	@Xw[$j],@Xw[$j],@Xw[($j+8)&15]+	add	$d,$d,$K		// future e+=K+	orr	$t0,$t0,$t1+	add	$e,$e,$t2		// e+=rot(a,5)+	 eor	@Xw[$j],@Xw[$j],@Xw[($j+13)&15]+	ror	$b,$b,#2+	add	$d,$d,@Xw[($i+1)&15]	// future e+=X[i]+	add	$e,$e,$t0		// e+=F(b,c,d)+	 ror	@Xw[$j],@Xw[$j],#31+___+}++sub BODY_40_59 {+my ($i,$a,$b,$c,$d,$e)=@_;+my $j=($i+2)&15;++$code.=<<___ if ($i==59);+	movz	$K,#0xc1d6+	movk	$K,#0xca62,lsl#16+___+$code.=<<___;+	orr	$t0,$b,$c+	and	$t1,$b,$c+	 eor	@Xw[$j],@Xw[$j],@Xw[($j+2)&15]+	ror	$t2,$a,#27+	and	$t0,$t0,$d+	add	$d,$d,$K		// future e+=K+	 eor	@Xw[$j],@Xw[$j],@Xw[($j+8)&15]+	add	$e,$e,$t2		// e+=rot(a,5)+	orr	$t0,$t0,$t1+	ror	$b,$b,#2+	 eor	@Xw[$j],@Xw[$j],@Xw[($j+13)&15]+	add	$d,$d,@Xw[($i+1)&15]	// future e+=X[i]+	add	$e,$e,$t0		// e+=F(b,c,d)+	 ror	@Xw[$j],@Xw[$j],#31+___+}++sub BODY_20_39 {+my ($i,$a,$b,$c,$d,$e)=@_;+my $j=($i+2)&15;++$code.=<<___ if ($i==39);+	movz	$K,#0xbcdc+	movk	$K,#0x8f1b,lsl#16+___+$code.=<<___ if ($i<78);+	 eor	@Xw[$j],@Xw[$j],@Xw[($j+2)&15]+	eor	$t0,$d,$b+	ror	$t2,$a,#27+	add	$d,$d,$K		// future e+=K+	 eor	@Xw[$j],@Xw[$j],@Xw[($j+8)&15]+	eor	$t0,$t0,$c+	add	$e,$e,$t2		// e+=rot(a,5)+	ror	$b,$b,#2+	 eor	@Xw[$j],@Xw[$j],@Xw[($j+13)&15]+	add	$d,$d,@Xw[($i+1)&15]	// future e+=X[i]+	add	$e,$e,$t0		// e+=F(b,c,d)+	 ror	@Xw[$j],@Xw[$j],#31+___+$code.=<<___ if ($i==78);+	ldp	@Xw[1],@Xw[2],[$ctx]+	eor	$t0,$d,$b+	ror	$t2,$a,#27+	add	$d,$d,$K		// future e+=K+	eor	$t0,$t0,$c+	add	$e,$e,$t2		// e+=rot(a,5)+	ror	$b,$b,#2+	add	$d,$d,@Xw[($i+1)&15]	// future e+=X[i]+	add	$e,$e,$t0		// e+=F(b,c,d)+___+$code.=<<___ if ($i==79);+	ldp	@Xw[3],@Xw[4],[$ctx,#8]+	eor	$t0,$d,$b+	ror	$t2,$a,#27+	eor	$t0,$t0,$c+	add	$e,$e,$t2		// e+=rot(a,5)+	ror	$b,$b,#2+	ldr	@Xw[5],[$ctx,#16]+	add	$e,$e,$t0		// e+=F(b,c,d)+___+}++$code.=<<___;+#ifndef	__KERNEL__+# include "arm_arch.h"+.extern OPENSSL_armcap_P+#else+.globl	sha1_block_armv8+#endif++.text++.globl	sha1_block_data_order+.type	sha1_block_data_order,%function+.align	6+sha1_block_data_order:+	adrp	c16,OPENSSL_armcap_P+	ldr	w16,[c16,#:lo12:OPENSSL_armcap_P]+	tst	w16,#ARMV8_SHA1+	b.ne	.Lv8_entry++	stp	c29,c30,[csp,#-12*__SIZEOF_POINTER__]!+	add	c29,csp,#0+	stp	c19,c20,[csp,#2*__SIZEOF_POINTER__]+	stp	c21,c22,[csp,#4*__SIZEOF_POINTER__]+	stp	c23,c24,[csp,#6*__SIZEOF_POINTER__]+	stp	c25,c26,[csp,#8*__SIZEOF_POINTER__]+	stp	c27,c28,[csp,#10*__SIZEOF_POINTER__]++	ldp	$A,$B,[$ctx]+	ldp	$C,$D,[$ctx,#8]+	ldr	$E,[$ctx,#16]++.Loop:+	ldr	@Xx[0],[$inp],#64+	movz	$K,#0x7999+	sub	$num,$num,#1+	movk	$K,#0x5a82,lsl#16+#ifdef	__AARCH64EB__+	ror	$Xx[0],@Xx[0],#32+#else+	rev32	@Xx[0],@Xx[0]+#endif+	add	$E,$E,$K		// warm it up+	add	$E,$E,@Xw[0]+___+for($i=0;$i<20;$i++)	{ &BODY_00_19($i,@V); unshift(@V,pop(@V)); }+for(;$i<40;$i++)	{ &BODY_20_39($i,@V); unshift(@V,pop(@V)); }+for(;$i<60;$i++)	{ &BODY_40_59($i,@V); unshift(@V,pop(@V)); }+for(;$i<80;$i++)	{ &BODY_20_39($i,@V); unshift(@V,pop(@V)); }+$code.=<<___;+	add	$B,$B,@Xw[2]+	add	$C,$C,@Xw[3]+	add	$A,$A,@Xw[1]+	add	$D,$D,@Xw[4]+	add	$E,$E,@Xw[5]+	stp	$A,$B,[$ctx]+	stp	$C,$D,[$ctx,#8]+	str	$E,[$ctx,#16]+	cbnz	$num,.Loop++	ldp	c19,c20,[csp,#2*__SIZEOF_POINTER__]+	ldp	c21,c22,[csp,#4*__SIZEOF_POINTER__]+	ldp	c23,c24,[csp,#6*__SIZEOF_POINTER__]+	ldp	c25,c26,[csp,#8*__SIZEOF_POINTER__]+	ldp	c27,c28,[csp,#10*__SIZEOF_POINTER__]+	ldr	c29,[csp],#12*__SIZEOF_POINTER__+	ret+.size	sha1_block_data_order,.-sha1_block_data_order+___+{{{+my ($ABCD,$E,$E0,$E1)=map("v$_.16b",(0..3));+my @MSG=map("v$_.16b",(4..7));+my @Kxx=map("v$_.4s",(16..19));+my ($W0,$W1)=("v20.4s","v21.4s");+my $ABCD_SAVE="v22.16b";++$code.=<<___;+.type	sha1_block_armv8,%function+.align	6+sha1_block_armv8:+.Lv8_entry:+	stp	x29,x30,[sp,#-16]!+	add	x29,sp,#0++	adr	x4,.Lconst+	eor	$E,$E,$E+	ld1.32	{$ABCD},[$ctx],#16+	ld1.32	{$E}[0],[$ctx]+	csub	$ctx,$ctx,#16+	ld1.32	{@Kxx[0]-@Kxx[3]},[x4]++.Loop_hw:+	ld1	{@MSG[0]-@MSG[3]},[$inp],#64+	sub	$num,$num,#1+	rev32	@MSG[0],@MSG[0]+	rev32	@MSG[1],@MSG[1]++	add.i32	$W0,@Kxx[0],@MSG[0]+	rev32	@MSG[2],@MSG[2]+	orr	$ABCD_SAVE,$ABCD,$ABCD	// offload++	add.i32	$W1,@Kxx[0],@MSG[1]+	rev32	@MSG[3],@MSG[3]+	sha1h	$E1,$ABCD+	sha1c	$ABCD,$E,$W0		// 0+	add.i32	$W0,@Kxx[$j],@MSG[2]+	sha1su0	@MSG[0],@MSG[1],@MSG[2]+___+for ($j=0,$i=1;$i<20-3;$i++) {+my $f=("c","p","m","p")[$i/5];+$code.=<<___;+	sha1h	$E0,$ABCD		// $i+	sha1$f	$ABCD,$E1,$W1+	add.i32	$W1,@Kxx[$j],@MSG[3]+	sha1su1	@MSG[0],@MSG[3]+___+$code.=<<___ if ($i<20-4);+	sha1su0	@MSG[1],@MSG[2],@MSG[3]+___+	($E0,$E1)=($E1,$E0);		($W0,$W1)=($W1,$W0);+	push(@MSG,shift(@MSG));		$j++ if ((($i+3)%5)==0);+}+$code.=<<___;+	sha1h	$E0,$ABCD		// $i+	sha1p	$ABCD,$E1,$W1+	add.i32	$W1,@Kxx[$j],@MSG[3]++	sha1h	$E1,$ABCD		// 18+	sha1p	$ABCD,$E0,$W0++	sha1h	$E0,$ABCD		// 19+	sha1p	$ABCD,$E1,$W1++	add.i32	$E,$E,$E0+	add.i32	$ABCD,$ABCD,$ABCD_SAVE++	cbnz	$num,.Loop_hw++	st1.32	{$ABCD},[$ctx],#16+	st1.32	{$E}[0],[$ctx]++	ldr	x29,[sp],#16+	ret+.size	sha1_block_armv8,.-sha1_block_armv8+.align	6+.Lconst:+.long	0x5a827999,0x5a827999,0x5a827999,0x5a827999	//K_00_19+.long	0x6ed9eba1,0x6ed9eba1,0x6ed9eba1,0x6ed9eba1	//K_20_39+.long	0x8f1bbcdc,0x8f1bbcdc,0x8f1bbcdc,0x8f1bbcdc	//K_40_59+.long	0xca62c1d6,0xca62c1d6,0xca62c1d6,0xca62c1d6	//K_60_79+.asciz	"SHA1 block transform for ARMv8, CRYPTOGAMS by \@dot-asm"+.align	2+#if !defined(__KERNELL__) && !defined(_WIN64)+.comm	OPENSSL_armcap_P,4,4+.hidden	OPENSSL_armcap_P+#endif+___+}}}++{   my	%opcode = (+	"sha1c"		=> 0x5e000000,	"sha1p"		=> 0x5e001000,+	"sha1m"		=> 0x5e002000,	"sha1su0"	=> 0x5e003000,+	"sha1h"		=> 0x5e280800,	"sha1su1"	=> 0x5e281800	);++    sub unsha1 {+	my ($mnemonic,$arg)=@_;++	$arg =~ m/[qv]([0-9]+)[^,]*,\s*[qv]([0-9]+)[^,]*(?:,\s*[qv]([0-9]+))?/o+	&&+	sprintf ".inst\t0x%08x\t//%s %s",+			$opcode{$mnemonic}|$1|($2<<5)|($3<<16),+			$mnemonic,$arg;+    }+}++foreach(split("\n",$code)) {++	s/\`([^\`]*)\`/eval($1)/geo;++	s/\b(sha1\w+)\s+([qv].*)/unsha1($1,$2)/geo;++	s/\.\w?32\b//o		and s/\.16b/\.4s/go;+	m/(ld|st)1[^\[]+\[0\]/o	and s/\.4s/\.s/go;++	print $_,"\n";+}++close STDOUT;
+ cbits/asm/sha256-armv8-ios64.S view
@@ -0,0 +1,2051 @@+// SPDX-License-Identifier: GPL-1.0+ OR BSD-3-Clause+//+// ====================================================================+// Written by Andy Polyakov, @dot-asm, initially for the OpenSSL+// project.+// ====================================================================+//+// SHA256/512 for ARMv8.+//+// Performance in cycles per processed byte and improvement coefficient+// over code generated with "default" compiler:+//+//		SHA256-hw	SHA256(*)	SHA512+// Apple A7	1.97		10.5 (+33%)	6.73 (-1%(**))+// Apple A10	1.30				5.81+// Apple A12	1.31				5.06+// Apple A14/M1	1.30		8.19 (+14%)	2.24 (hw)+// Cortex-A53	2.38		15.5 (+115%)	10.0 (+150%(***))+// Cortex-A57	2.31		11.6 (+86%)	7.51 (+260%(***))+// Cortex-A76	1.60		9.5		6.05+// Cortex-X2	1.60		7.3		2.60 (hw)+// Cortex-X925	1.57		5.97		2.55 (hw)+// Denver	2.01		10.5 (+26%)	6.70 (+8%)+// X-Gene			20.0 (+100%)	12.8 (+300%(***))+// Mongoose	2.36		13.0 (+50%)	8.36 (+33%)+// Kryo		1.92		17.4 (+30%)	11.2 (+8%)+// ThunderX2	2.54		13.2 (+40%)	8.40 (+18%)+// Shapdragon X	1.40		7.43		2.23 (hw)+//+// (*)	Software SHA256 results are of lesser relevance, presented+//	mostly for informational purposes.+// (**)	The result is a trade-off: it's possible to improve it by+//	10% (or by 1 cycle per round), but at the cost of 20% loss+//	on Cortex-A53 (or by 4 cycles per round).+// (***)	Super-impressive coefficients over gcc-generated code are+//	indication of some compiler "pathology", most notably code+//	generated with -mgeneral-regs-only is significantly faster+//	and the gap is only 40-90%.+//+// October 2016.+//+// Originally it was reckoned that it makes no sense to implement NEON+// version of SHA256 for 64-bit processors. This is because performance+// improvement on most wide-spread Cortex-A5x processors was observed+// to be marginal, same on Cortex-A53 and ~10% on A57. But then it was+// observed that 32-bit NEON SHA256 performs significantly better than+// 64-bit scalar version on *some* of the more recent processors. As+// result 64-bit NEON version of SHA256 was added to provide best+// all-round performance. For example it executes ~30% faster on X-Gene+// and Mongoose. [For reference, NEON version of SHA512 is bound to+// deliver much less improvement, likely *negative* on Cortex-A5x.+// Which is why NEON support is limited to SHA256.]++#ifndef	__KERNEL__+# include "arm_arch.h"++#endif++.text++.globl	_crypton_sha256_asm_block_data_order++.align	6+_crypton_sha256_asm_block_data_order:+#ifndef	__KERNEL__+	adrp	x16,_crypton_armcap_P@PAGE+	ldr	w16,[x16,_crypton_armcap_P@PAGEOFF]+	tst	w16,#ARMV8_SHA256+	b.ne	Lv8_entry+	tst	w16,#ARMV7_NEON+	b.ne	Lneon_entry+#endif+.long	0xd503233f				// paciasp+	stp	x29,x30,[sp,#-16*__SIZEOF_POINTER__]!+	add	x29,sp,#0++	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	stp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	stp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	stp	x27,x28,[sp,#10*__SIZEOF_POINTER__]+	sub	sp,sp,#4*4++	ldp	w20,w21,[x0]				// load context+	ldp	w22,w23,[x0,#2*4]+	lsl	x2,x2,#6+	ldp	w24,w25,[x0,#4*4]+	add	x2,x1,x2+	ldp	w26,w27,[x0,#6*4]+	adr	x30,LK256+	stp	x0,x2,[x29,#12*__SIZEOF_POINTER__]++Loop:+	ldp	w3,w4,[x1],#2*4+	ldr	w19,[x30],#4			// *K+++	eor	w28,w21,w22				// magic seed+	str	x1,[x29,#14*__SIZEOF_POINTER__]+#ifndef	__AARCH64EB__+	rev	w3,w3			// 0+#endif+	ror	w16,w24,#6+	add	w27,w27,w19			// h+=K[i]+	eor	w6,w24,w24,ror#14+	and	w17,w25,w24+	bic	w19,w26,w24+	add	w27,w27,w3			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w20,w21			// a^b, b^c in next round+	eor	w16,w16,w6,ror#11	// Sigma1(e)+	ror	w6,w20,#2+	add	w27,w27,w17			// h+=Ch(e,f,g)+	eor	w17,w20,w20,ror#9+	add	w27,w27,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w23,w23,w27			// d+=h+	eor	w28,w28,w21			// Maj(a,b,c)+	eor	w17,w6,w17,ror#13	// Sigma0(a)+	add	w27,w27,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w27,w27,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w4,w4			// 1+#endif+	ldp	w5,w6,[x1],#2*4+	add	w27,w27,w17			// h+=Sigma0(a)+	ror	w16,w23,#6+	add	w26,w26,w28			// h+=K[i]+	eor	w7,w23,w23,ror#14+	and	w17,w24,w23+	bic	w28,w25,w23+	add	w26,w26,w4			// h+=X[i]+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w27,w20			// a^b, b^c in next round+	eor	w16,w16,w7,ror#11	// Sigma1(e)+	ror	w7,w27,#2+	add	w26,w26,w17			// h+=Ch(e,f,g)+	eor	w17,w27,w27,ror#9+	add	w26,w26,w16			// h+=Sigma1(e)+	and	w19,w19,w28			// (b^c)&=(a^b)+	add	w22,w22,w26			// d+=h+	eor	w19,w19,w20			// Maj(a,b,c)+	eor	w17,w7,w17,ror#13	// Sigma0(a)+	add	w26,w26,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	//add	w26,w26,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w5,w5			// 2+#endif+	add	w26,w26,w17			// h+=Sigma0(a)+	ror	w16,w22,#6+	add	w25,w25,w19			// h+=K[i]+	eor	w8,w22,w22,ror#14+	and	w17,w23,w22+	bic	w19,w24,w22+	add	w25,w25,w5			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w26,w27			// a^b, b^c in next round+	eor	w16,w16,w8,ror#11	// Sigma1(e)+	ror	w8,w26,#2+	add	w25,w25,w17			// h+=Ch(e,f,g)+	eor	w17,w26,w26,ror#9+	add	w25,w25,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w21,w21,w25			// d+=h+	eor	w28,w28,w27			// Maj(a,b,c)+	eor	w17,w8,w17,ror#13	// Sigma0(a)+	add	w25,w25,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w25,w25,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w6,w6			// 3+#endif+	ldp	w7,w8,[x1],#2*4+	add	w25,w25,w17			// h+=Sigma0(a)+	ror	w16,w21,#6+	add	w24,w24,w28			// h+=K[i]+	eor	w9,w21,w21,ror#14+	and	w17,w22,w21+	bic	w28,w23,w21+	add	w24,w24,w6			// h+=X[i]+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w25,w26			// a^b, b^c in next round+	eor	w16,w16,w9,ror#11	// Sigma1(e)+	ror	w9,w25,#2+	add	w24,w24,w17			// h+=Ch(e,f,g)+	eor	w17,w25,w25,ror#9+	add	w24,w24,w16			// h+=Sigma1(e)+	and	w19,w19,w28			// (b^c)&=(a^b)+	add	w20,w20,w24			// d+=h+	eor	w19,w19,w26			// Maj(a,b,c)+	eor	w17,w9,w17,ror#13	// Sigma0(a)+	add	w24,w24,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	//add	w24,w24,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w7,w7			// 4+#endif+	add	w24,w24,w17			// h+=Sigma0(a)+	ror	w16,w20,#6+	add	w23,w23,w19			// h+=K[i]+	eor	w10,w20,w20,ror#14+	and	w17,w21,w20+	bic	w19,w22,w20+	add	w23,w23,w7			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w24,w25			// a^b, b^c in next round+	eor	w16,w16,w10,ror#11	// Sigma1(e)+	ror	w10,w24,#2+	add	w23,w23,w17			// h+=Ch(e,f,g)+	eor	w17,w24,w24,ror#9+	add	w23,w23,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w27,w27,w23			// d+=h+	eor	w28,w28,w25			// Maj(a,b,c)+	eor	w17,w10,w17,ror#13	// Sigma0(a)+	add	w23,w23,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w23,w23,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w8,w8			// 5+#endif+	ldp	w9,w10,[x1],#2*4+	add	w23,w23,w17			// h+=Sigma0(a)+	ror	w16,w27,#6+	add	w22,w22,w28			// h+=K[i]+	eor	w11,w27,w27,ror#14+	and	w17,w20,w27+	bic	w28,w21,w27+	add	w22,w22,w8			// h+=X[i]+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w23,w24			// a^b, b^c in next round+	eor	w16,w16,w11,ror#11	// Sigma1(e)+	ror	w11,w23,#2+	add	w22,w22,w17			// h+=Ch(e,f,g)+	eor	w17,w23,w23,ror#9+	add	w22,w22,w16			// h+=Sigma1(e)+	and	w19,w19,w28			// (b^c)&=(a^b)+	add	w26,w26,w22			// d+=h+	eor	w19,w19,w24			// Maj(a,b,c)+	eor	w17,w11,w17,ror#13	// Sigma0(a)+	add	w22,w22,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	//add	w22,w22,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w9,w9			// 6+#endif+	add	w22,w22,w17			// h+=Sigma0(a)+	ror	w16,w26,#6+	add	w21,w21,w19			// h+=K[i]+	eor	w12,w26,w26,ror#14+	and	w17,w27,w26+	bic	w19,w20,w26+	add	w21,w21,w9			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w22,w23			// a^b, b^c in next round+	eor	w16,w16,w12,ror#11	// Sigma1(e)+	ror	w12,w22,#2+	add	w21,w21,w17			// h+=Ch(e,f,g)+	eor	w17,w22,w22,ror#9+	add	w21,w21,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w25,w25,w21			// d+=h+	eor	w28,w28,w23			// Maj(a,b,c)+	eor	w17,w12,w17,ror#13	// Sigma0(a)+	add	w21,w21,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w21,w21,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w10,w10			// 7+#endif+	ldp	w11,w12,[x1],#2*4+	add	w21,w21,w17			// h+=Sigma0(a)+	ror	w16,w25,#6+	add	w20,w20,w28			// h+=K[i]+	eor	w13,w25,w25,ror#14+	and	w17,w26,w25+	bic	w28,w27,w25+	add	w20,w20,w10			// h+=X[i]+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w21,w22			// a^b, b^c in next round+	eor	w16,w16,w13,ror#11	// Sigma1(e)+	ror	w13,w21,#2+	add	w20,w20,w17			// h+=Ch(e,f,g)+	eor	w17,w21,w21,ror#9+	add	w20,w20,w16			// h+=Sigma1(e)+	and	w19,w19,w28			// (b^c)&=(a^b)+	add	w24,w24,w20			// d+=h+	eor	w19,w19,w22			// Maj(a,b,c)+	eor	w17,w13,w17,ror#13	// Sigma0(a)+	add	w20,w20,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	//add	w20,w20,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w11,w11			// 8+#endif+	add	w20,w20,w17			// h+=Sigma0(a)+	ror	w16,w24,#6+	add	w27,w27,w19			// h+=K[i]+	eor	w14,w24,w24,ror#14+	and	w17,w25,w24+	bic	w19,w26,w24+	add	w27,w27,w11			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w20,w21			// a^b, b^c in next round+	eor	w16,w16,w14,ror#11	// Sigma1(e)+	ror	w14,w20,#2+	add	w27,w27,w17			// h+=Ch(e,f,g)+	eor	w17,w20,w20,ror#9+	add	w27,w27,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w23,w23,w27			// d+=h+	eor	w28,w28,w21			// Maj(a,b,c)+	eor	w17,w14,w17,ror#13	// Sigma0(a)+	add	w27,w27,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w27,w27,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w12,w12			// 9+#endif+	ldp	w13,w14,[x1],#2*4+	add	w27,w27,w17			// h+=Sigma0(a)+	ror	w16,w23,#6+	add	w26,w26,w28			// h+=K[i]+	eor	w15,w23,w23,ror#14+	and	w17,w24,w23+	bic	w28,w25,w23+	add	w26,w26,w12			// h+=X[i]+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w27,w20			// a^b, b^c in next round+	eor	w16,w16,w15,ror#11	// Sigma1(e)+	ror	w15,w27,#2+	add	w26,w26,w17			// h+=Ch(e,f,g)+	eor	w17,w27,w27,ror#9+	add	w26,w26,w16			// h+=Sigma1(e)+	and	w19,w19,w28			// (b^c)&=(a^b)+	add	w22,w22,w26			// d+=h+	eor	w19,w19,w20			// Maj(a,b,c)+	eor	w17,w15,w17,ror#13	// Sigma0(a)+	add	w26,w26,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	//add	w26,w26,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w13,w13			// 10+#endif+	add	w26,w26,w17			// h+=Sigma0(a)+	ror	w16,w22,#6+	add	w25,w25,w19			// h+=K[i]+	eor	w0,w22,w22,ror#14+	and	w17,w23,w22+	bic	w19,w24,w22+	add	w25,w25,w13			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w26,w27			// a^b, b^c in next round+	eor	w16,w16,w0,ror#11	// Sigma1(e)+	ror	w0,w26,#2+	add	w25,w25,w17			// h+=Ch(e,f,g)+	eor	w17,w26,w26,ror#9+	add	w25,w25,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w21,w21,w25			// d+=h+	eor	w28,w28,w27			// Maj(a,b,c)+	eor	w17,w0,w17,ror#13	// Sigma0(a)+	add	w25,w25,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w25,w25,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w14,w14			// 11+#endif+	ldp	w15,w0,[x1],#2*4+	add	w25,w25,w17			// h+=Sigma0(a)+	str	w6,[sp,#12]+	ror	w16,w21,#6+	add	w24,w24,w28			// h+=K[i]+	eor	w6,w21,w21,ror#14+	and	w17,w22,w21+	bic	w28,w23,w21+	add	w24,w24,w14			// h+=X[i]+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w25,w26			// a^b, b^c in next round+	eor	w16,w16,w6,ror#11	// Sigma1(e)+	ror	w6,w25,#2+	add	w24,w24,w17			// h+=Ch(e,f,g)+	eor	w17,w25,w25,ror#9+	add	w24,w24,w16			// h+=Sigma1(e)+	and	w19,w19,w28			// (b^c)&=(a^b)+	add	w20,w20,w24			// d+=h+	eor	w19,w19,w26			// Maj(a,b,c)+	eor	w17,w6,w17,ror#13	// Sigma0(a)+	add	w24,w24,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	//add	w24,w24,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w15,w15			// 12+#endif+	add	w24,w24,w17			// h+=Sigma0(a)+	str	w7,[sp,#0]+	ror	w16,w20,#6+	add	w23,w23,w19			// h+=K[i]+	eor	w7,w20,w20,ror#14+	and	w17,w21,w20+	bic	w19,w22,w20+	add	w23,w23,w15			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w24,w25			// a^b, b^c in next round+	eor	w16,w16,w7,ror#11	// Sigma1(e)+	ror	w7,w24,#2+	add	w23,w23,w17			// h+=Ch(e,f,g)+	eor	w17,w24,w24,ror#9+	add	w23,w23,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w27,w27,w23			// d+=h+	eor	w28,w28,w25			// Maj(a,b,c)+	eor	w17,w7,w17,ror#13	// Sigma0(a)+	add	w23,w23,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w23,w23,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w0,w0			// 13+#endif+	ldp	w1,w2,[x1]+	add	w23,w23,w17			// h+=Sigma0(a)+	str	w8,[sp,#4]+	ror	w16,w27,#6+	add	w22,w22,w28			// h+=K[i]+	eor	w8,w27,w27,ror#14+	and	w17,w20,w27+	bic	w28,w21,w27+	add	w22,w22,w0			// h+=X[i]+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w23,w24			// a^b, b^c in next round+	eor	w16,w16,w8,ror#11	// Sigma1(e)+	ror	w8,w23,#2+	add	w22,w22,w17			// h+=Ch(e,f,g)+	eor	w17,w23,w23,ror#9+	add	w22,w22,w16			// h+=Sigma1(e)+	and	w19,w19,w28			// (b^c)&=(a^b)+	add	w26,w26,w22			// d+=h+	eor	w19,w19,w24			// Maj(a,b,c)+	eor	w17,w8,w17,ror#13	// Sigma0(a)+	add	w22,w22,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	//add	w22,w22,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w1,w1			// 14+#endif+	ldr	w6,[sp,#12]+	add	w22,w22,w17			// h+=Sigma0(a)+	str	w9,[sp,#8]+	ror	w16,w26,#6+	add	w21,w21,w19			// h+=K[i]+	eor	w9,w26,w26,ror#14+	and	w17,w27,w26+	bic	w19,w20,w26+	add	w21,w21,w1			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w22,w23			// a^b, b^c in next round+	eor	w16,w16,w9,ror#11	// Sigma1(e)+	ror	w9,w22,#2+	add	w21,w21,w17			// h+=Ch(e,f,g)+	eor	w17,w22,w22,ror#9+	add	w21,w21,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w25,w25,w21			// d+=h+	eor	w28,w28,w23			// Maj(a,b,c)+	eor	w17,w9,w17,ror#13	// Sigma0(a)+	add	w21,w21,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w21,w21,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w2,w2			// 15+#endif+	ldr	w7,[sp,#0]+	add	w21,w21,w17			// h+=Sigma0(a)+	str	w10,[sp,#12]+	ror	w16,w25,#6+	add	w20,w20,w28			// h+=K[i]+	ror	w9,w4,#7+	and	w17,w26,w25+	ror	w8,w1,#17+	bic	w28,w27,w25+	ror	w10,w21,#2+	add	w20,w20,w2			// h+=X[i]+	eor	w16,w16,w25,ror#11+	eor	w9,w9,w4,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w21,w22			// a^b, b^c in next round+	eor	w16,w16,w25,ror#25	// Sigma1(e)+	eor	w10,w10,w21,ror#13+	add	w20,w20,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w8,w8,w1,ror#19+	eor	w9,w9,w4,lsr#3	// sigma0(X[i+1])+	add	w20,w20,w16			// h+=Sigma1(e)+	eor	w19,w19,w22			// Maj(a,b,c)+	eor	w17,w10,w21,ror#22	// Sigma0(a)+	eor	w8,w8,w1,lsr#10	// sigma1(X[i+14])+	add	w3,w3,w12+	add	w24,w24,w20			// d+=h+	add	w20,w20,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w3,w3,w9+	add	w20,w20,w17			// h+=Sigma0(a)+	add	w3,w3,w8+Loop_16_xx:+	ldr	w8,[sp,#4]+	str	w11,[sp,#0]+	ror	w16,w24,#6+	add	w27,w27,w19			// h+=K[i]+	ror	w10,w5,#7+	and	w17,w25,w24+	ror	w9,w2,#17+	bic	w19,w26,w24+	ror	w11,w20,#2+	add	w27,w27,w3			// h+=X[i]+	eor	w16,w16,w24,ror#11+	eor	w10,w10,w5,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w20,w21			// a^b, b^c in next round+	eor	w16,w16,w24,ror#25	// Sigma1(e)+	eor	w11,w11,w20,ror#13+	add	w27,w27,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w9,w9,w2,ror#19+	eor	w10,w10,w5,lsr#3	// sigma0(X[i+1])+	add	w27,w27,w16			// h+=Sigma1(e)+	eor	w28,w28,w21			// Maj(a,b,c)+	eor	w17,w11,w20,ror#22	// Sigma0(a)+	eor	w9,w9,w2,lsr#10	// sigma1(X[i+14])+	add	w4,w4,w13+	add	w23,w23,w27			// d+=h+	add	w27,w27,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w4,w4,w10+	add	w27,w27,w17			// h+=Sigma0(a)+	add	w4,w4,w9+	ldr	w9,[sp,#8]+	str	w12,[sp,#4]+	ror	w16,w23,#6+	add	w26,w26,w28			// h+=K[i]+	ror	w11,w6,#7+	and	w17,w24,w23+	ror	w10,w3,#17+	bic	w28,w25,w23+	ror	w12,w27,#2+	add	w26,w26,w4			// h+=X[i]+	eor	w16,w16,w23,ror#11+	eor	w11,w11,w6,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w27,w20			// a^b, b^c in next round+	eor	w16,w16,w23,ror#25	// Sigma1(e)+	eor	w12,w12,w27,ror#13+	add	w26,w26,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w10,w10,w3,ror#19+	eor	w11,w11,w6,lsr#3	// sigma0(X[i+1])+	add	w26,w26,w16			// h+=Sigma1(e)+	eor	w19,w19,w20			// Maj(a,b,c)+	eor	w17,w12,w27,ror#22	// Sigma0(a)+	eor	w10,w10,w3,lsr#10	// sigma1(X[i+14])+	add	w5,w5,w14+	add	w22,w22,w26			// d+=h+	add	w26,w26,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w5,w5,w11+	add	w26,w26,w17			// h+=Sigma0(a)+	add	w5,w5,w10+	ldr	w10,[sp,#12]+	str	w13,[sp,#8]+	ror	w16,w22,#6+	add	w25,w25,w19			// h+=K[i]+	ror	w12,w7,#7+	and	w17,w23,w22+	ror	w11,w4,#17+	bic	w19,w24,w22+	ror	w13,w26,#2+	add	w25,w25,w5			// h+=X[i]+	eor	w16,w16,w22,ror#11+	eor	w12,w12,w7,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w26,w27			// a^b, b^c in next round+	eor	w16,w16,w22,ror#25	// Sigma1(e)+	eor	w13,w13,w26,ror#13+	add	w25,w25,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w11,w11,w4,ror#19+	eor	w12,w12,w7,lsr#3	// sigma0(X[i+1])+	add	w25,w25,w16			// h+=Sigma1(e)+	eor	w28,w28,w27			// Maj(a,b,c)+	eor	w17,w13,w26,ror#22	// Sigma0(a)+	eor	w11,w11,w4,lsr#10	// sigma1(X[i+14])+	add	w6,w6,w15+	add	w21,w21,w25			// d+=h+	add	w25,w25,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w6,w6,w12+	add	w25,w25,w17			// h+=Sigma0(a)+	add	w6,w6,w11+	ldr	w11,[sp,#0]+	str	w14,[sp,#12]+	ror	w16,w21,#6+	add	w24,w24,w28			// h+=K[i]+	ror	w13,w8,#7+	and	w17,w22,w21+	ror	w12,w5,#17+	bic	w28,w23,w21+	ror	w14,w25,#2+	add	w24,w24,w6			// h+=X[i]+	eor	w16,w16,w21,ror#11+	eor	w13,w13,w8,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w25,w26			// a^b, b^c in next round+	eor	w16,w16,w21,ror#25	// Sigma1(e)+	eor	w14,w14,w25,ror#13+	add	w24,w24,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w12,w12,w5,ror#19+	eor	w13,w13,w8,lsr#3	// sigma0(X[i+1])+	add	w24,w24,w16			// h+=Sigma1(e)+	eor	w19,w19,w26			// Maj(a,b,c)+	eor	w17,w14,w25,ror#22	// Sigma0(a)+	eor	w12,w12,w5,lsr#10	// sigma1(X[i+14])+	add	w7,w7,w0+	add	w20,w20,w24			// d+=h+	add	w24,w24,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w7,w7,w13+	add	w24,w24,w17			// h+=Sigma0(a)+	add	w7,w7,w12+	ldr	w12,[sp,#4]+	str	w15,[sp,#0]+	ror	w16,w20,#6+	add	w23,w23,w19			// h+=K[i]+	ror	w14,w9,#7+	and	w17,w21,w20+	ror	w13,w6,#17+	bic	w19,w22,w20+	ror	w15,w24,#2+	add	w23,w23,w7			// h+=X[i]+	eor	w16,w16,w20,ror#11+	eor	w14,w14,w9,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w24,w25			// a^b, b^c in next round+	eor	w16,w16,w20,ror#25	// Sigma1(e)+	eor	w15,w15,w24,ror#13+	add	w23,w23,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w13,w13,w6,ror#19+	eor	w14,w14,w9,lsr#3	// sigma0(X[i+1])+	add	w23,w23,w16			// h+=Sigma1(e)+	eor	w28,w28,w25			// Maj(a,b,c)+	eor	w17,w15,w24,ror#22	// Sigma0(a)+	eor	w13,w13,w6,lsr#10	// sigma1(X[i+14])+	add	w8,w8,w1+	add	w27,w27,w23			// d+=h+	add	w23,w23,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w8,w8,w14+	add	w23,w23,w17			// h+=Sigma0(a)+	add	w8,w8,w13+	ldr	w13,[sp,#8]+	str	w0,[sp,#4]+	ror	w16,w27,#6+	add	w22,w22,w28			// h+=K[i]+	ror	w15,w10,#7+	and	w17,w20,w27+	ror	w14,w7,#17+	bic	w28,w21,w27+	ror	w0,w23,#2+	add	w22,w22,w8			// h+=X[i]+	eor	w16,w16,w27,ror#11+	eor	w15,w15,w10,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w23,w24			// a^b, b^c in next round+	eor	w16,w16,w27,ror#25	// Sigma1(e)+	eor	w0,w0,w23,ror#13+	add	w22,w22,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w14,w14,w7,ror#19+	eor	w15,w15,w10,lsr#3	// sigma0(X[i+1])+	add	w22,w22,w16			// h+=Sigma1(e)+	eor	w19,w19,w24			// Maj(a,b,c)+	eor	w17,w0,w23,ror#22	// Sigma0(a)+	eor	w14,w14,w7,lsr#10	// sigma1(X[i+14])+	add	w9,w9,w2+	add	w26,w26,w22			// d+=h+	add	w22,w22,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w9,w9,w15+	add	w22,w22,w17			// h+=Sigma0(a)+	add	w9,w9,w14+	ldr	w14,[sp,#12]+	str	w1,[sp,#8]+	ror	w16,w26,#6+	add	w21,w21,w19			// h+=K[i]+	ror	w0,w11,#7+	and	w17,w27,w26+	ror	w15,w8,#17+	bic	w19,w20,w26+	ror	w1,w22,#2+	add	w21,w21,w9			// h+=X[i]+	eor	w16,w16,w26,ror#11+	eor	w0,w0,w11,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w22,w23			// a^b, b^c in next round+	eor	w16,w16,w26,ror#25	// Sigma1(e)+	eor	w1,w1,w22,ror#13+	add	w21,w21,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w15,w15,w8,ror#19+	eor	w0,w0,w11,lsr#3	// sigma0(X[i+1])+	add	w21,w21,w16			// h+=Sigma1(e)+	eor	w28,w28,w23			// Maj(a,b,c)+	eor	w17,w1,w22,ror#22	// Sigma0(a)+	eor	w15,w15,w8,lsr#10	// sigma1(X[i+14])+	add	w10,w10,w3+	add	w25,w25,w21			// d+=h+	add	w21,w21,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w10,w10,w0+	add	w21,w21,w17			// h+=Sigma0(a)+	add	w10,w10,w15+	ldr	w15,[sp,#0]+	str	w2,[sp,#12]+	ror	w16,w25,#6+	add	w20,w20,w28			// h+=K[i]+	ror	w1,w12,#7+	and	w17,w26,w25+	ror	w0,w9,#17+	bic	w28,w27,w25+	ror	w2,w21,#2+	add	w20,w20,w10			// h+=X[i]+	eor	w16,w16,w25,ror#11+	eor	w1,w1,w12,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w21,w22			// a^b, b^c in next round+	eor	w16,w16,w25,ror#25	// Sigma1(e)+	eor	w2,w2,w21,ror#13+	add	w20,w20,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w0,w0,w9,ror#19+	eor	w1,w1,w12,lsr#3	// sigma0(X[i+1])+	add	w20,w20,w16			// h+=Sigma1(e)+	eor	w19,w19,w22			// Maj(a,b,c)+	eor	w17,w2,w21,ror#22	// Sigma0(a)+	eor	w0,w0,w9,lsr#10	// sigma1(X[i+14])+	add	w11,w11,w4+	add	w24,w24,w20			// d+=h+	add	w20,w20,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w11,w11,w1+	add	w20,w20,w17			// h+=Sigma0(a)+	add	w11,w11,w0+	ldr	w0,[sp,#4]+	str	w3,[sp,#0]+	ror	w16,w24,#6+	add	w27,w27,w19			// h+=K[i]+	ror	w2,w13,#7+	and	w17,w25,w24+	ror	w1,w10,#17+	bic	w19,w26,w24+	ror	w3,w20,#2+	add	w27,w27,w11			// h+=X[i]+	eor	w16,w16,w24,ror#11+	eor	w2,w2,w13,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w20,w21			// a^b, b^c in next round+	eor	w16,w16,w24,ror#25	// Sigma1(e)+	eor	w3,w3,w20,ror#13+	add	w27,w27,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w1,w1,w10,ror#19+	eor	w2,w2,w13,lsr#3	// sigma0(X[i+1])+	add	w27,w27,w16			// h+=Sigma1(e)+	eor	w28,w28,w21			// Maj(a,b,c)+	eor	w17,w3,w20,ror#22	// Sigma0(a)+	eor	w1,w1,w10,lsr#10	// sigma1(X[i+14])+	add	w12,w12,w5+	add	w23,w23,w27			// d+=h+	add	w27,w27,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w12,w12,w2+	add	w27,w27,w17			// h+=Sigma0(a)+	add	w12,w12,w1+	ldr	w1,[sp,#8]+	str	w4,[sp,#4]+	ror	w16,w23,#6+	add	w26,w26,w28			// h+=K[i]+	ror	w3,w14,#7+	and	w17,w24,w23+	ror	w2,w11,#17+	bic	w28,w25,w23+	ror	w4,w27,#2+	add	w26,w26,w12			// h+=X[i]+	eor	w16,w16,w23,ror#11+	eor	w3,w3,w14,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w27,w20			// a^b, b^c in next round+	eor	w16,w16,w23,ror#25	// Sigma1(e)+	eor	w4,w4,w27,ror#13+	add	w26,w26,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w2,w2,w11,ror#19+	eor	w3,w3,w14,lsr#3	// sigma0(X[i+1])+	add	w26,w26,w16			// h+=Sigma1(e)+	eor	w19,w19,w20			// Maj(a,b,c)+	eor	w17,w4,w27,ror#22	// Sigma0(a)+	eor	w2,w2,w11,lsr#10	// sigma1(X[i+14])+	add	w13,w13,w6+	add	w22,w22,w26			// d+=h+	add	w26,w26,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w13,w13,w3+	add	w26,w26,w17			// h+=Sigma0(a)+	add	w13,w13,w2+	ldr	w2,[sp,#12]+	str	w5,[sp,#8]+	ror	w16,w22,#6+	add	w25,w25,w19			// h+=K[i]+	ror	w4,w15,#7+	and	w17,w23,w22+	ror	w3,w12,#17+	bic	w19,w24,w22+	ror	w5,w26,#2+	add	w25,w25,w13			// h+=X[i]+	eor	w16,w16,w22,ror#11+	eor	w4,w4,w15,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w26,w27			// a^b, b^c in next round+	eor	w16,w16,w22,ror#25	// Sigma1(e)+	eor	w5,w5,w26,ror#13+	add	w25,w25,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w3,w3,w12,ror#19+	eor	w4,w4,w15,lsr#3	// sigma0(X[i+1])+	add	w25,w25,w16			// h+=Sigma1(e)+	eor	w28,w28,w27			// Maj(a,b,c)+	eor	w17,w5,w26,ror#22	// Sigma0(a)+	eor	w3,w3,w12,lsr#10	// sigma1(X[i+14])+	add	w14,w14,w7+	add	w21,w21,w25			// d+=h+	add	w25,w25,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w14,w14,w4+	add	w25,w25,w17			// h+=Sigma0(a)+	add	w14,w14,w3+	ldr	w3,[sp,#0]+	str	w6,[sp,#12]+	ror	w16,w21,#6+	add	w24,w24,w28			// h+=K[i]+	ror	w5,w0,#7+	and	w17,w22,w21+	ror	w4,w13,#17+	bic	w28,w23,w21+	ror	w6,w25,#2+	add	w24,w24,w14			// h+=X[i]+	eor	w16,w16,w21,ror#11+	eor	w5,w5,w0,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w25,w26			// a^b, b^c in next round+	eor	w16,w16,w21,ror#25	// Sigma1(e)+	eor	w6,w6,w25,ror#13+	add	w24,w24,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w4,w4,w13,ror#19+	eor	w5,w5,w0,lsr#3	// sigma0(X[i+1])+	add	w24,w24,w16			// h+=Sigma1(e)+	eor	w19,w19,w26			// Maj(a,b,c)+	eor	w17,w6,w25,ror#22	// Sigma0(a)+	eor	w4,w4,w13,lsr#10	// sigma1(X[i+14])+	add	w15,w15,w8+	add	w20,w20,w24			// d+=h+	add	w24,w24,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w15,w15,w5+	add	w24,w24,w17			// h+=Sigma0(a)+	add	w15,w15,w4+	ldr	w4,[sp,#4]+	str	w7,[sp,#0]+	ror	w16,w20,#6+	add	w23,w23,w19			// h+=K[i]+	ror	w6,w1,#7+	and	w17,w21,w20+	ror	w5,w14,#17+	bic	w19,w22,w20+	ror	w7,w24,#2+	add	w23,w23,w15			// h+=X[i]+	eor	w16,w16,w20,ror#11+	eor	w6,w6,w1,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w24,w25			// a^b, b^c in next round+	eor	w16,w16,w20,ror#25	// Sigma1(e)+	eor	w7,w7,w24,ror#13+	add	w23,w23,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w5,w5,w14,ror#19+	eor	w6,w6,w1,lsr#3	// sigma0(X[i+1])+	add	w23,w23,w16			// h+=Sigma1(e)+	eor	w28,w28,w25			// Maj(a,b,c)+	eor	w17,w7,w24,ror#22	// Sigma0(a)+	eor	w5,w5,w14,lsr#10	// sigma1(X[i+14])+	add	w0,w0,w9+	add	w27,w27,w23			// d+=h+	add	w23,w23,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w0,w0,w6+	add	w23,w23,w17			// h+=Sigma0(a)+	add	w0,w0,w5+	ldr	w5,[sp,#8]+	str	w8,[sp,#4]+	ror	w16,w27,#6+	add	w22,w22,w28			// h+=K[i]+	ror	w7,w2,#7+	and	w17,w20,w27+	ror	w6,w15,#17+	bic	w28,w21,w27+	ror	w8,w23,#2+	add	w22,w22,w0			// h+=X[i]+	eor	w16,w16,w27,ror#11+	eor	w7,w7,w2,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w23,w24			// a^b, b^c in next round+	eor	w16,w16,w27,ror#25	// Sigma1(e)+	eor	w8,w8,w23,ror#13+	add	w22,w22,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w6,w6,w15,ror#19+	eor	w7,w7,w2,lsr#3	// sigma0(X[i+1])+	add	w22,w22,w16			// h+=Sigma1(e)+	eor	w19,w19,w24			// Maj(a,b,c)+	eor	w17,w8,w23,ror#22	// Sigma0(a)+	eor	w6,w6,w15,lsr#10	// sigma1(X[i+14])+	add	w1,w1,w10+	add	w26,w26,w22			// d+=h+	add	w22,w22,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w1,w1,w7+	add	w22,w22,w17			// h+=Sigma0(a)+	add	w1,w1,w6+	ldr	w6,[sp,#12]+	str	w9,[sp,#8]+	ror	w16,w26,#6+	add	w21,w21,w19			// h+=K[i]+	ror	w8,w3,#7+	and	w17,w27,w26+	ror	w7,w0,#17+	bic	w19,w20,w26+	ror	w9,w22,#2+	add	w21,w21,w1			// h+=X[i]+	eor	w16,w16,w26,ror#11+	eor	w8,w8,w3,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w22,w23			// a^b, b^c in next round+	eor	w16,w16,w26,ror#25	// Sigma1(e)+	eor	w9,w9,w22,ror#13+	add	w21,w21,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w7,w7,w0,ror#19+	eor	w8,w8,w3,lsr#3	// sigma0(X[i+1])+	add	w21,w21,w16			// h+=Sigma1(e)+	eor	w28,w28,w23			// Maj(a,b,c)+	eor	w17,w9,w22,ror#22	// Sigma0(a)+	eor	w7,w7,w0,lsr#10	// sigma1(X[i+14])+	add	w2,w2,w11+	add	w25,w25,w21			// d+=h+	add	w21,w21,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w2,w2,w8+	add	w21,w21,w17			// h+=Sigma0(a)+	add	w2,w2,w7+	ldr	w7,[sp,#0]+	str	w10,[sp,#12]+	ror	w16,w25,#6+	add	w20,w20,w28			// h+=K[i]+	ror	w9,w4,#7+	and	w17,w26,w25+	ror	w8,w1,#17+	bic	w28,w27,w25+	ror	w10,w21,#2+	add	w20,w20,w2			// h+=X[i]+	eor	w16,w16,w25,ror#11+	eor	w9,w9,w4,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w21,w22			// a^b, b^c in next round+	eor	w16,w16,w25,ror#25	// Sigma1(e)+	eor	w10,w10,w21,ror#13+	add	w20,w20,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w8,w8,w1,ror#19+	eor	w9,w9,w4,lsr#3	// sigma0(X[i+1])+	add	w20,w20,w16			// h+=Sigma1(e)+	eor	w19,w19,w22			// Maj(a,b,c)+	eor	w17,w10,w21,ror#22	// Sigma0(a)+	eor	w8,w8,w1,lsr#10	// sigma1(X[i+14])+	add	w3,w3,w12+	add	w24,w24,w20			// d+=h+	add	w20,w20,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w3,w3,w9+	add	w20,w20,w17			// h+=Sigma0(a)+	add	w3,w3,w8+	cbnz	w19,Loop_16_xx++	ldp	x0,x2,[x29,#12*__SIZEOF_POINTER__]+	ldr	x1,[x29,#14*__SIZEOF_POINTER__]+	sub	x30,x30,#260++	ldp	w3,w4,[x0]+	ldp	w5,w6,[x0,#2*4]+	add	x1,x1,#14*4+	ldp	w7,w8,[x0,#4*4]+	add	w20,w20,w3+	ldp	w9,w10,[x0,#6*4]+	add	w21,w21,w4+	add	w22,w22,w5+	add	w23,w23,w6+	stp	w20,w21,[x0]+	add	w24,w24,w7+	add	w25,w25,w8+	stp	w22,w23,[x0,#2*4]+	add	w26,w26,w9+	add	w27,w27,w10+	cmp	x1,x2+	stp	w24,w25,[x0,#4*4]+	stp	w26,w27,[x0,#6*4]+	b.ne	Loop++	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#4*4+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#16*__SIZEOF_POINTER__+.long	0xd50323bf				// autiasp+	ret+++.align	6++LK256:+.long	0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5+.long	0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5+.long	0xd807aa98,0x12835b01,0x243185be,0x550c7dc3+.long	0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174+.long	0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc+.long	0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da+.long	0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7+.long	0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967+.long	0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13+.long	0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85+.long	0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3+.long	0xd192e819,0xd6990624,0xf40e3585,0x106aa070+.long	0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5+.long	0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3+.long	0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208+.long	0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2+.long	0	//terminator++.byte	83,72,65,50,53,54,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.align	2+.align	2+#ifndef	__KERNEL__++.align	6+crypton_sha256_asm_block_armv8:+Lv8_entry:+	stp	x29,x30,[sp,#-2*__SIZEOF_POINTER__]!+	add	x29,sp,#0++	ld1	{v0.4s,v1.4s},[x0]+	adr	x3,LK256++Loop_hw:+	ld1	{v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64+	sub	x2,x2,#1+	ld1	{v16.4s},[x3],#16+	rev32	v4.16b,v4.16b+	rev32	v5.16b,v5.16b+	rev32	v6.16b,v6.16b+	rev32	v7.16b,v7.16b+	orr	v18.16b,v0.16b,v0.16b		// offload+	orr	v19.16b,v1.16b,v1.16b+	ld1	{v17.4s},[x3],#16+	add	v16.4s,v16.4s,v4.4s+.long	0x5e2828a4	//sha256su0 v4.16b,v5.16b+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.long	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s+.long	0x5e0760c4	//sha256su1 v4.16b,v6.16b,v7.16b+	ld1	{v16.4s},[x3],#16+	add	v17.4s,v17.4s,v5.4s+.long	0x5e2828c5	//sha256su0 v5.16b,v6.16b+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.long	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s+.long	0x5e0460e5	//sha256su1 v5.16b,v7.16b,v4.16b+	ld1	{v17.4s},[x3],#16+	add	v16.4s,v16.4s,v6.4s+.long	0x5e2828e6	//sha256su0 v6.16b,v7.16b+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.long	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s+.long	0x5e056086	//sha256su1 v6.16b,v4.16b,v5.16b+	ld1	{v16.4s},[x3],#16+	add	v17.4s,v17.4s,v7.4s+.long	0x5e282887	//sha256su0 v7.16b,v4.16b+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.long	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s+.long	0x5e0660a7	//sha256su1 v7.16b,v5.16b,v6.16b+	ld1	{v17.4s},[x3],#16+	add	v16.4s,v16.4s,v4.4s+.long	0x5e2828a4	//sha256su0 v4.16b,v5.16b+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.long	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s+.long	0x5e0760c4	//sha256su1 v4.16b,v6.16b,v7.16b+	ld1	{v16.4s},[x3],#16+	add	v17.4s,v17.4s,v5.4s+.long	0x5e2828c5	//sha256su0 v5.16b,v6.16b+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.long	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s+.long	0x5e0460e5	//sha256su1 v5.16b,v7.16b,v4.16b+	ld1	{v17.4s},[x3],#16+	add	v16.4s,v16.4s,v6.4s+.long	0x5e2828e6	//sha256su0 v6.16b,v7.16b+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.long	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s+.long	0x5e056086	//sha256su1 v6.16b,v4.16b,v5.16b+	ld1	{v16.4s},[x3],#16+	add	v17.4s,v17.4s,v7.4s+.long	0x5e282887	//sha256su0 v7.16b,v4.16b+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.long	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s+.long	0x5e0660a7	//sha256su1 v7.16b,v5.16b,v6.16b+	ld1	{v17.4s},[x3],#16+	add	v16.4s,v16.4s,v4.4s+.long	0x5e2828a4	//sha256su0 v4.16b,v5.16b+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.long	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s+.long	0x5e0760c4	//sha256su1 v4.16b,v6.16b,v7.16b+	ld1	{v16.4s},[x3],#16+	add	v17.4s,v17.4s,v5.4s+.long	0x5e2828c5	//sha256su0 v5.16b,v6.16b+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.long	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s+.long	0x5e0460e5	//sha256su1 v5.16b,v7.16b,v4.16b+	ld1	{v17.4s},[x3],#16+	add	v16.4s,v16.4s,v6.4s+.long	0x5e2828e6	//sha256su0 v6.16b,v7.16b+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.long	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s+.long	0x5e056086	//sha256su1 v6.16b,v4.16b,v5.16b+	ld1	{v16.4s},[x3],#16+	add	v17.4s,v17.4s,v7.4s+.long	0x5e282887	//sha256su0 v7.16b,v4.16b+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.long	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s+.long	0x5e0660a7	//sha256su1 v7.16b,v5.16b,v6.16b+	ld1	{v17.4s},[x3],#16+	add	v16.4s,v16.4s,v4.4s+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.long	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s++	ld1	{v16.4s},[x3],#16+	add	v17.4s,v17.4s,v5.4s+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.long	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s++	ld1	{v17.4s},[x3]+	add	v16.4s,v16.4s,v6.4s+	sub	x3,x3,#64*4-16+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.long	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s++	add	v17.4s,v17.4s,v7.4s+	orr	v2.16b,v0.16b,v0.16b+.long	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.long	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s++	add	v0.4s,v0.4s,v18.4s+	add	v1.4s,v1.4s,v19.4s++	cbnz	x2,Loop_hw++	st1	{v0.4s,v1.4s},[x0]++	ldr	x29,[sp],#2*__SIZEOF_POINTER__+	ret++#endif+#ifdef	__KERNEL__+.globl	_crypton_sha256_asm_block_neon+#endif++.align	4+_crypton_sha256_asm_block_neon:+Lneon_entry:+	stp	x29, x30, [sp, #-2*__SIZEOF_POINTER__]!+	mov	x29, sp+	sub	sp,sp,#16*4++	adr	x16,LK256+	add	x2,x1,x2,lsl#6	// len to point at the end of inp++	ld1	{v0.16b},[x1], #16+	ld1	{v1.16b},[x1], #16+	ld1	{v2.16b},[x1], #16+	ld1	{v3.16b},[x1], #16+	ld1	{v4.4s},[x16], #16+	ld1	{v5.4s},[x16], #16+	ld1	{v6.4s},[x16], #16+	ld1	{v7.4s},[x16], #16+	rev32	v0.16b,v0.16b		// yes, even on+	rev32	v1.16b,v1.16b		// big-endian+	rev32	v2.16b,v2.16b+	rev32	v3.16b,v3.16b+	mov	x17,sp+	add	v4.4s,v4.4s,v0.4s+	add	v5.4s,v5.4s,v1.4s+	add	v6.4s,v6.4s,v2.4s+	st1	{v4.4s,v5.4s},[x17], #32+	add	v7.4s,v7.4s,v3.4s+	st1	{v6.4s,v7.4s},[x17]+	sub	x17,x17,#32++	ldp	w3,w4,[x0]+	ldp	w5,w6,[x0,#8]+	ldp	w7,w8,[x0,#16]+	ldp	w9,w10,[x0,#24]+	ldr	w12,[sp,#0]+	mov	w13,wzr+	eor	w14,w4,w5+	mov	w15,wzr+	b	L_00_48++.align	4+L_00_48:+	ext	v4.16b,v0.16b,v1.16b,#4+	add	w10,w10,w12+	add	w3,w3,w15+	and	w12,w8,w7+	bic	w15,w9,w7+	ext	v7.16b,v2.16b,v3.16b,#4+	eor	w11,w7,w7,ror#5+	add	w3,w3,w13+	mov	d19,v3.d[1]+	orr	w12,w12,w15+	eor	w11,w11,w7,ror#19+	ushr	v6.4s,v4.4s,#7+	eor	w15,w3,w3,ror#11+	ushr	v5.4s,v4.4s,#3+	add	w10,w10,w12+	add	v0.4s,v0.4s,v7.4s+	ror	w11,w11,#6+	sli	v6.4s,v4.4s,#25+	eor	w13,w3,w4+	eor	w15,w15,w3,ror#20+	ushr	v7.4s,v4.4s,#18+	add	w10,w10,w11+	ldr	w12,[sp,#4]+	and	w14,w14,w13+	eor	v5.16b,v5.16b,v6.16b+	ror	w15,w15,#2+	add	w6,w6,w10+	sli	v7.4s,v4.4s,#14+	eor	w14,w14,w4+	ushr	v16.4s,v19.4s,#17+	add	w9,w9,w12+	add	w10,w10,w15+	and	w12,w7,w6+	eor	v5.16b,v5.16b,v7.16b+	bic	w15,w8,w6+	eor	w11,w6,w6,ror#5+	sli	v16.4s,v19.4s,#15+	add	w10,w10,w14+	orr	w12,w12,w15+	ushr	v17.4s,v19.4s,#10+	eor	w11,w11,w6,ror#19+	eor	w15,w10,w10,ror#11+	ushr	v7.4s,v19.4s,#19+	add	w9,w9,w12+	ror	w11,w11,#6+	add	v0.4s,v0.4s,v5.4s+	eor	w14,w10,w3+	eor	w15,w15,w10,ror#20+	sli	v7.4s,v19.4s,#13+	add	w9,w9,w11+	ldr	w12,[sp,#8]+	and	w13,w13,w14+	eor	v17.16b,v17.16b,v16.16b+	ror	w15,w15,#2+	add	w5,w5,w9+	eor	w13,w13,w3+	eor	v17.16b,v17.16b,v7.16b+	add	w8,w8,w12+	add	w9,w9,w15+	and	w12,w6,w5+	add	v0.4s,v0.4s,v17.4s+	bic	w15,w7,w5+	eor	w11,w5,w5,ror#5+	add	w9,w9,w13+	ushr	v18.4s,v0.4s,#17+	orr	w12,w12,w15+	ushr	v19.4s,v0.4s,#10+	eor	w11,w11,w5,ror#19+	eor	w15,w9,w9,ror#11+	sli	v18.4s,v0.4s,#15+	add	w8,w8,w12+	ushr	v17.4s,v0.4s,#19+	ror	w11,w11,#6+	eor	w13,w9,w10+	eor	v19.16b,v19.16b,v18.16b+	eor	w15,w15,w9,ror#20+	add	w8,w8,w11+	sli	v17.4s,v0.4s,#13+	ldr	w12,[sp,#12]+	and	w14,w14,w13+	ror	w15,w15,#2+	ld1	{v4.4s},[x16], #16+	add	w4,w4,w8+	eor	v19.16b,v19.16b,v17.16b+	eor	w14,w14,w10+	eor	v17.16b,v17.16b,v17.16b+	add	w7,w7,w12+	add	w8,w8,w15+	and	w12,w5,w4+	mov	v17.d[1],v19.d[0]+	bic	w15,w6,w4+	eor	w11,w4,w4,ror#5+	add	w8,w8,w14+	add	v0.4s,v0.4s,v17.4s+	orr	w12,w12,w15+	eor	w11,w11,w4,ror#19+	eor	w15,w8,w8,ror#11+	add	v4.4s,v4.4s,v0.4s+	add	w7,w7,w12+	ror	w11,w11,#6+	eor	w14,w8,w9+	eor	w15,w15,w8,ror#20+	add	w7,w7,w11+	ldr	w12,[sp,#16]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w3,w3,w7+	eor	w13,w13,w9+	st1	{v4.4s},[x17], #16+	ext	v4.16b,v1.16b,v2.16b,#4+	add	w6,w6,w12+	add	w7,w7,w15+	and	w12,w4,w3+	bic	w15,w5,w3+	ext	v7.16b,v3.16b,v0.16b,#4+	eor	w11,w3,w3,ror#5+	add	w7,w7,w13+	mov	d19,v0.d[1]+	orr	w12,w12,w15+	eor	w11,w11,w3,ror#19+	ushr	v6.4s,v4.4s,#7+	eor	w15,w7,w7,ror#11+	ushr	v5.4s,v4.4s,#3+	add	w6,w6,w12+	add	v1.4s,v1.4s,v7.4s+	ror	w11,w11,#6+	sli	v6.4s,v4.4s,#25+	eor	w13,w7,w8+	eor	w15,w15,w7,ror#20+	ushr	v7.4s,v4.4s,#18+	add	w6,w6,w11+	ldr	w12,[sp,#20]+	and	w14,w14,w13+	eor	v5.16b,v5.16b,v6.16b+	ror	w15,w15,#2+	add	w10,w10,w6+	sli	v7.4s,v4.4s,#14+	eor	w14,w14,w8+	ushr	v16.4s,v19.4s,#17+	add	w5,w5,w12+	add	w6,w6,w15+	and	w12,w3,w10+	eor	v5.16b,v5.16b,v7.16b+	bic	w15,w4,w10+	eor	w11,w10,w10,ror#5+	sli	v16.4s,v19.4s,#15+	add	w6,w6,w14+	orr	w12,w12,w15+	ushr	v17.4s,v19.4s,#10+	eor	w11,w11,w10,ror#19+	eor	w15,w6,w6,ror#11+	ushr	v7.4s,v19.4s,#19+	add	w5,w5,w12+	ror	w11,w11,#6+	add	v1.4s,v1.4s,v5.4s+	eor	w14,w6,w7+	eor	w15,w15,w6,ror#20+	sli	v7.4s,v19.4s,#13+	add	w5,w5,w11+	ldr	w12,[sp,#24]+	and	w13,w13,w14+	eor	v17.16b,v17.16b,v16.16b+	ror	w15,w15,#2+	add	w9,w9,w5+	eor	w13,w13,w7+	eor	v17.16b,v17.16b,v7.16b+	add	w4,w4,w12+	add	w5,w5,w15+	and	w12,w10,w9+	add	v1.4s,v1.4s,v17.4s+	bic	w15,w3,w9+	eor	w11,w9,w9,ror#5+	add	w5,w5,w13+	ushr	v18.4s,v1.4s,#17+	orr	w12,w12,w15+	ushr	v19.4s,v1.4s,#10+	eor	w11,w11,w9,ror#19+	eor	w15,w5,w5,ror#11+	sli	v18.4s,v1.4s,#15+	add	w4,w4,w12+	ushr	v17.4s,v1.4s,#19+	ror	w11,w11,#6+	eor	w13,w5,w6+	eor	v19.16b,v19.16b,v18.16b+	eor	w15,w15,w5,ror#20+	add	w4,w4,w11+	sli	v17.4s,v1.4s,#13+	ldr	w12,[sp,#28]+	and	w14,w14,w13+	ror	w15,w15,#2+	ld1	{v4.4s},[x16], #16+	add	w8,w8,w4+	eor	v19.16b,v19.16b,v17.16b+	eor	w14,w14,w6+	eor	v17.16b,v17.16b,v17.16b+	add	w3,w3,w12+	add	w4,w4,w15+	and	w12,w9,w8+	mov	v17.d[1],v19.d[0]+	bic	w15,w10,w8+	eor	w11,w8,w8,ror#5+	add	w4,w4,w14+	add	v1.4s,v1.4s,v17.4s+	orr	w12,w12,w15+	eor	w11,w11,w8,ror#19+	eor	w15,w4,w4,ror#11+	add	v4.4s,v4.4s,v1.4s+	add	w3,w3,w12+	ror	w11,w11,#6+	eor	w14,w4,w5+	eor	w15,w15,w4,ror#20+	add	w3,w3,w11+	ldr	w12,[sp,#32]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w7,w7,w3+	eor	w13,w13,w5+	st1	{v4.4s},[x17], #16+	ext	v4.16b,v2.16b,v3.16b,#4+	add	w10,w10,w12+	add	w3,w3,w15+	and	w12,w8,w7+	bic	w15,w9,w7+	ext	v7.16b,v0.16b,v1.16b,#4+	eor	w11,w7,w7,ror#5+	add	w3,w3,w13+	mov	d19,v1.d[1]+	orr	w12,w12,w15+	eor	w11,w11,w7,ror#19+	ushr	v6.4s,v4.4s,#7+	eor	w15,w3,w3,ror#11+	ushr	v5.4s,v4.4s,#3+	add	w10,w10,w12+	add	v2.4s,v2.4s,v7.4s+	ror	w11,w11,#6+	sli	v6.4s,v4.4s,#25+	eor	w13,w3,w4+	eor	w15,w15,w3,ror#20+	ushr	v7.4s,v4.4s,#18+	add	w10,w10,w11+	ldr	w12,[sp,#36]+	and	w14,w14,w13+	eor	v5.16b,v5.16b,v6.16b+	ror	w15,w15,#2+	add	w6,w6,w10+	sli	v7.4s,v4.4s,#14+	eor	w14,w14,w4+	ushr	v16.4s,v19.4s,#17+	add	w9,w9,w12+	add	w10,w10,w15+	and	w12,w7,w6+	eor	v5.16b,v5.16b,v7.16b+	bic	w15,w8,w6+	eor	w11,w6,w6,ror#5+	sli	v16.4s,v19.4s,#15+	add	w10,w10,w14+	orr	w12,w12,w15+	ushr	v17.4s,v19.4s,#10+	eor	w11,w11,w6,ror#19+	eor	w15,w10,w10,ror#11+	ushr	v7.4s,v19.4s,#19+	add	w9,w9,w12+	ror	w11,w11,#6+	add	v2.4s,v2.4s,v5.4s+	eor	w14,w10,w3+	eor	w15,w15,w10,ror#20+	sli	v7.4s,v19.4s,#13+	add	w9,w9,w11+	ldr	w12,[sp,#40]+	and	w13,w13,w14+	eor	v17.16b,v17.16b,v16.16b+	ror	w15,w15,#2+	add	w5,w5,w9+	eor	w13,w13,w3+	eor	v17.16b,v17.16b,v7.16b+	add	w8,w8,w12+	add	w9,w9,w15+	and	w12,w6,w5+	add	v2.4s,v2.4s,v17.4s+	bic	w15,w7,w5+	eor	w11,w5,w5,ror#5+	add	w9,w9,w13+	ushr	v18.4s,v2.4s,#17+	orr	w12,w12,w15+	ushr	v19.4s,v2.4s,#10+	eor	w11,w11,w5,ror#19+	eor	w15,w9,w9,ror#11+	sli	v18.4s,v2.4s,#15+	add	w8,w8,w12+	ushr	v17.4s,v2.4s,#19+	ror	w11,w11,#6+	eor	w13,w9,w10+	eor	v19.16b,v19.16b,v18.16b+	eor	w15,w15,w9,ror#20+	add	w8,w8,w11+	sli	v17.4s,v2.4s,#13+	ldr	w12,[sp,#44]+	and	w14,w14,w13+	ror	w15,w15,#2+	ld1	{v4.4s},[x16], #16+	add	w4,w4,w8+	eor	v19.16b,v19.16b,v17.16b+	eor	w14,w14,w10+	eor	v17.16b,v17.16b,v17.16b+	add	w7,w7,w12+	add	w8,w8,w15+	and	w12,w5,w4+	mov	v17.d[1],v19.d[0]+	bic	w15,w6,w4+	eor	w11,w4,w4,ror#5+	add	w8,w8,w14+	add	v2.4s,v2.4s,v17.4s+	orr	w12,w12,w15+	eor	w11,w11,w4,ror#19+	eor	w15,w8,w8,ror#11+	add	v4.4s,v4.4s,v2.4s+	add	w7,w7,w12+	ror	w11,w11,#6+	eor	w14,w8,w9+	eor	w15,w15,w8,ror#20+	add	w7,w7,w11+	ldr	w12,[sp,#48]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w3,w3,w7+	eor	w13,w13,w9+	st1	{v4.4s},[x17], #16+	ext	v4.16b,v3.16b,v0.16b,#4+	add	w6,w6,w12+	add	w7,w7,w15+	and	w12,w4,w3+	bic	w15,w5,w3+	ext	v7.16b,v1.16b,v2.16b,#4+	eor	w11,w3,w3,ror#5+	add	w7,w7,w13+	mov	d19,v2.d[1]+	orr	w12,w12,w15+	eor	w11,w11,w3,ror#19+	ushr	v6.4s,v4.4s,#7+	eor	w15,w7,w7,ror#11+	ushr	v5.4s,v4.4s,#3+	add	w6,w6,w12+	add	v3.4s,v3.4s,v7.4s+	ror	w11,w11,#6+	sli	v6.4s,v4.4s,#25+	eor	w13,w7,w8+	eor	w15,w15,w7,ror#20+	ushr	v7.4s,v4.4s,#18+	add	w6,w6,w11+	ldr	w12,[sp,#52]+	and	w14,w14,w13+	eor	v5.16b,v5.16b,v6.16b+	ror	w15,w15,#2+	add	w10,w10,w6+	sli	v7.4s,v4.4s,#14+	eor	w14,w14,w8+	ushr	v16.4s,v19.4s,#17+	add	w5,w5,w12+	add	w6,w6,w15+	and	w12,w3,w10+	eor	v5.16b,v5.16b,v7.16b+	bic	w15,w4,w10+	eor	w11,w10,w10,ror#5+	sli	v16.4s,v19.4s,#15+	add	w6,w6,w14+	orr	w12,w12,w15+	ushr	v17.4s,v19.4s,#10+	eor	w11,w11,w10,ror#19+	eor	w15,w6,w6,ror#11+	ushr	v7.4s,v19.4s,#19+	add	w5,w5,w12+	ror	w11,w11,#6+	add	v3.4s,v3.4s,v5.4s+	eor	w14,w6,w7+	eor	w15,w15,w6,ror#20+	sli	v7.4s,v19.4s,#13+	add	w5,w5,w11+	ldr	w12,[sp,#56]+	and	w13,w13,w14+	eor	v17.16b,v17.16b,v16.16b+	ror	w15,w15,#2+	add	w9,w9,w5+	eor	w13,w13,w7+	eor	v17.16b,v17.16b,v7.16b+	add	w4,w4,w12+	add	w5,w5,w15+	and	w12,w10,w9+	add	v3.4s,v3.4s,v17.4s+	bic	w15,w3,w9+	eor	w11,w9,w9,ror#5+	add	w5,w5,w13+	ushr	v18.4s,v3.4s,#17+	orr	w12,w12,w15+	ushr	v19.4s,v3.4s,#10+	eor	w11,w11,w9,ror#19+	eor	w15,w5,w5,ror#11+	sli	v18.4s,v3.4s,#15+	add	w4,w4,w12+	ushr	v17.4s,v3.4s,#19+	ror	w11,w11,#6+	eor	w13,w5,w6+	eor	v19.16b,v19.16b,v18.16b+	eor	w15,w15,w5,ror#20+	add	w4,w4,w11+	sli	v17.4s,v3.4s,#13+	ldr	w12,[sp,#60]+	and	w14,w14,w13+	ror	w15,w15,#2+	ld1	{v4.4s},[x16], #16+	add	w8,w8,w4+	eor	v19.16b,v19.16b,v17.16b+	eor	w14,w14,w6+	eor	v17.16b,v17.16b,v17.16b+	add	w3,w3,w12+	add	w4,w4,w15+	and	w12,w9,w8+	mov	v17.d[1],v19.d[0]+	bic	w15,w10,w8+	eor	w11,w8,w8,ror#5+	add	w4,w4,w14+	add	v3.4s,v3.4s,v17.4s+	orr	w12,w12,w15+	eor	w11,w11,w8,ror#19+	eor	w15,w4,w4,ror#11+	add	v4.4s,v4.4s,v3.4s+	add	w3,w3,w12+	ror	w11,w11,#6+	eor	w14,w4,w5+	eor	w15,w15,w4,ror#20+	add	w3,w3,w11+	ldr	w12,[x16]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w7,w7,w3+	eor	w13,w13,w5+	st1	{v4.4s},[x17], #16+	cmp	w12,#0				// check for K256 terminator+	ldr	w12,[sp,#0]+	sub	x17,x17,#64+	bne	L_00_48++	sub	x16,x16,#256+	cmp	x1,x2+	mov	x17, #-64+	csel	x17, x17, xzr, eq+	add	x1,x1,x17+	mov	x17,sp+	add	w10,w10,w12+	add	w3,w3,w15+	and	w12,w8,w7+	ld1	{v0.16b},[x1],#16+	bic	w15,w9,w7+	eor	w11,w7,w7,ror#5+	ld1	{v4.4s},[x16],#16+	add	w3,w3,w13+	orr	w12,w12,w15+	eor	w11,w11,w7,ror#19+	eor	w15,w3,w3,ror#11+	rev32	v0.16b,v0.16b+	add	w10,w10,w12+	ror	w11,w11,#6+	eor	w13,w3,w4+	eor	w15,w15,w3,ror#20+	add	v4.4s,v4.4s,v0.4s+	add	w10,w10,w11+	ldr	w12,[sp,#4]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w6,w6,w10+	eor	w14,w14,w4+	add	w9,w9,w12+	add	w10,w10,w15+	and	w12,w7,w6+	bic	w15,w8,w6+	eor	w11,w6,w6,ror#5+	add	w10,w10,w14+	orr	w12,w12,w15+	eor	w11,w11,w6,ror#19+	eor	w15,w10,w10,ror#11+	add	w9,w9,w12+	ror	w11,w11,#6+	eor	w14,w10,w3+	eor	w15,w15,w10,ror#20+	add	w9,w9,w11+	ldr	w12,[sp,#8]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w5,w5,w9+	eor	w13,w13,w3+	add	w8,w8,w12+	add	w9,w9,w15+	and	w12,w6,w5+	bic	w15,w7,w5+	eor	w11,w5,w5,ror#5+	add	w9,w9,w13+	orr	w12,w12,w15+	eor	w11,w11,w5,ror#19+	eor	w15,w9,w9,ror#11+	add	w8,w8,w12+	ror	w11,w11,#6+	eor	w13,w9,w10+	eor	w15,w15,w9,ror#20+	add	w8,w8,w11+	ldr	w12,[sp,#12]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w4,w4,w8+	eor	w14,w14,w10+	add	w7,w7,w12+	add	w8,w8,w15+	and	w12,w5,w4+	bic	w15,w6,w4+	eor	w11,w4,w4,ror#5+	add	w8,w8,w14+	orr	w12,w12,w15+	eor	w11,w11,w4,ror#19+	eor	w15,w8,w8,ror#11+	add	w7,w7,w12+	ror	w11,w11,#6+	eor	w14,w8,w9+	eor	w15,w15,w8,ror#20+	add	w7,w7,w11+	ldr	w12,[sp,#16]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w3,w3,w7+	eor	w13,w13,w9+	st1	{v4.4s},[x17], #16+	add	w6,w6,w12+	add	w7,w7,w15+	and	w12,w4,w3+	ld1	{v1.16b},[x1],#16+	bic	w15,w5,w3+	eor	w11,w3,w3,ror#5+	ld1	{v4.4s},[x16],#16+	add	w7,w7,w13+	orr	w12,w12,w15+	eor	w11,w11,w3,ror#19+	eor	w15,w7,w7,ror#11+	rev32	v1.16b,v1.16b+	add	w6,w6,w12+	ror	w11,w11,#6+	eor	w13,w7,w8+	eor	w15,w15,w7,ror#20+	add	v4.4s,v4.4s,v1.4s+	add	w6,w6,w11+	ldr	w12,[sp,#20]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w10,w10,w6+	eor	w14,w14,w8+	add	w5,w5,w12+	add	w6,w6,w15+	and	w12,w3,w10+	bic	w15,w4,w10+	eor	w11,w10,w10,ror#5+	add	w6,w6,w14+	orr	w12,w12,w15+	eor	w11,w11,w10,ror#19+	eor	w15,w6,w6,ror#11+	add	w5,w5,w12+	ror	w11,w11,#6+	eor	w14,w6,w7+	eor	w15,w15,w6,ror#20+	add	w5,w5,w11+	ldr	w12,[sp,#24]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w9,w9,w5+	eor	w13,w13,w7+	add	w4,w4,w12+	add	w5,w5,w15+	and	w12,w10,w9+	bic	w15,w3,w9+	eor	w11,w9,w9,ror#5+	add	w5,w5,w13+	orr	w12,w12,w15+	eor	w11,w11,w9,ror#19+	eor	w15,w5,w5,ror#11+	add	w4,w4,w12+	ror	w11,w11,#6+	eor	w13,w5,w6+	eor	w15,w15,w5,ror#20+	add	w4,w4,w11+	ldr	w12,[sp,#28]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w8,w8,w4+	eor	w14,w14,w6+	add	w3,w3,w12+	add	w4,w4,w15+	and	w12,w9,w8+	bic	w15,w10,w8+	eor	w11,w8,w8,ror#5+	add	w4,w4,w14+	orr	w12,w12,w15+	eor	w11,w11,w8,ror#19+	eor	w15,w4,w4,ror#11+	add	w3,w3,w12+	ror	w11,w11,#6+	eor	w14,w4,w5+	eor	w15,w15,w4,ror#20+	add	w3,w3,w11+	ldr	w12,[sp,#32]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w7,w7,w3+	eor	w13,w13,w5+	st1	{v4.4s},[x17], #16+	add	w10,w10,w12+	add	w3,w3,w15+	and	w12,w8,w7+	ld1	{v2.16b},[x1],#16+	bic	w15,w9,w7+	eor	w11,w7,w7,ror#5+	ld1	{v4.4s},[x16],#16+	add	w3,w3,w13+	orr	w12,w12,w15+	eor	w11,w11,w7,ror#19+	eor	w15,w3,w3,ror#11+	rev32	v2.16b,v2.16b+	add	w10,w10,w12+	ror	w11,w11,#6+	eor	w13,w3,w4+	eor	w15,w15,w3,ror#20+	add	v4.4s,v4.4s,v2.4s+	add	w10,w10,w11+	ldr	w12,[sp,#36]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w6,w6,w10+	eor	w14,w14,w4+	add	w9,w9,w12+	add	w10,w10,w15+	and	w12,w7,w6+	bic	w15,w8,w6+	eor	w11,w6,w6,ror#5+	add	w10,w10,w14+	orr	w12,w12,w15+	eor	w11,w11,w6,ror#19+	eor	w15,w10,w10,ror#11+	add	w9,w9,w12+	ror	w11,w11,#6+	eor	w14,w10,w3+	eor	w15,w15,w10,ror#20+	add	w9,w9,w11+	ldr	w12,[sp,#40]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w5,w5,w9+	eor	w13,w13,w3+	add	w8,w8,w12+	add	w9,w9,w15+	and	w12,w6,w5+	bic	w15,w7,w5+	eor	w11,w5,w5,ror#5+	add	w9,w9,w13+	orr	w12,w12,w15+	eor	w11,w11,w5,ror#19+	eor	w15,w9,w9,ror#11+	add	w8,w8,w12+	ror	w11,w11,#6+	eor	w13,w9,w10+	eor	w15,w15,w9,ror#20+	add	w8,w8,w11+	ldr	w12,[sp,#44]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w4,w4,w8+	eor	w14,w14,w10+	add	w7,w7,w12+	add	w8,w8,w15+	and	w12,w5,w4+	bic	w15,w6,w4+	eor	w11,w4,w4,ror#5+	add	w8,w8,w14+	orr	w12,w12,w15+	eor	w11,w11,w4,ror#19+	eor	w15,w8,w8,ror#11+	add	w7,w7,w12+	ror	w11,w11,#6+	eor	w14,w8,w9+	eor	w15,w15,w8,ror#20+	add	w7,w7,w11+	ldr	w12,[sp,#48]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w3,w3,w7+	eor	w13,w13,w9+	st1	{v4.4s},[x17], #16+	add	w6,w6,w12+	add	w7,w7,w15+	and	w12,w4,w3+	ld1	{v3.16b},[x1],#16+	bic	w15,w5,w3+	eor	w11,w3,w3,ror#5+	ld1	{v4.4s},[x16],#16+	add	w7,w7,w13+	orr	w12,w12,w15+	eor	w11,w11,w3,ror#19+	eor	w15,w7,w7,ror#11+	rev32	v3.16b,v3.16b+	add	w6,w6,w12+	ror	w11,w11,#6+	eor	w13,w7,w8+	eor	w15,w15,w7,ror#20+	add	v4.4s,v4.4s,v3.4s+	add	w6,w6,w11+	ldr	w12,[sp,#52]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w10,w10,w6+	eor	w14,w14,w8+	add	w5,w5,w12+	add	w6,w6,w15+	and	w12,w3,w10+	bic	w15,w4,w10+	eor	w11,w10,w10,ror#5+	add	w6,w6,w14+	orr	w12,w12,w15+	eor	w11,w11,w10,ror#19+	eor	w15,w6,w6,ror#11+	add	w5,w5,w12+	ror	w11,w11,#6+	eor	w14,w6,w7+	eor	w15,w15,w6,ror#20+	add	w5,w5,w11+	ldr	w12,[sp,#56]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w9,w9,w5+	eor	w13,w13,w7+	add	w4,w4,w12+	add	w5,w5,w15+	and	w12,w10,w9+	bic	w15,w3,w9+	eor	w11,w9,w9,ror#5+	add	w5,w5,w13+	orr	w12,w12,w15+	eor	w11,w11,w9,ror#19+	eor	w15,w5,w5,ror#11+	add	w4,w4,w12+	ror	w11,w11,#6+	eor	w13,w5,w6+	eor	w15,w15,w5,ror#20+	add	w4,w4,w11+	ldr	w12,[sp,#60]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w8,w8,w4+	eor	w14,w14,w6+	add	w3,w3,w12+	add	w4,w4,w15+	and	w12,w9,w8+	bic	w15,w10,w8+	eor	w11,w8,w8,ror#5+	add	w4,w4,w14+	orr	w12,w12,w15+	eor	w11,w11,w8,ror#19+	eor	w15,w4,w4,ror#11+	add	w3,w3,w12+	ror	w11,w11,#6+	eor	w14,w4,w5+	eor	w15,w15,w4,ror#20+	add	w3,w3,w11+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w7,w7,w3+	eor	w13,w13,w5+	st1	{v4.4s},[x17], #16+	add	w3,w3,w15			// h+=Sigma0(a) from the past+	ldp	w11,w12,[x0,#0]+	add	w3,w3,w13			// h+=Maj(a,b,c) from the past+	ldp	w13,w14,[x0,#8]+	add	w3,w3,w11			// accumulate+	add	w4,w4,w12+	ldp	w11,w12,[x0,#16]+	add	w5,w5,w13+	add	w6,w6,w14+	ldp	w13,w14,[x0,#24]+	add	w7,w7,w11+	add	w8,w8,w12+	ldr	w12,[sp,#0]+	stp	w3,w4,[x0,#0]+	add	w9,w9,w13+	mov	w13,wzr+	stp	w5,w6,[x0,#8]+	add	w10,w10,w14+	stp	w7,w8,[x0,#16]+	eor	w14,w4,w5+	stp	w9,w10,[x0,#24]+	mov	w15,wzr+	mov	x17,sp+	b.ne	L_00_48++	ldr	x29,[x29]+	add	sp,sp,#16*4+2*__SIZEOF_POINTER__+	ret++#if !defined(__KERNEL__) && !defined(_WIN64)+.comm	__crypton_armcap_P,4+.private_extern	_crypton_armcap_P+#endif
+ cbits/asm/sha256-armv8-linux64.S view
@@ -0,0 +1,2053 @@+// SPDX-License-Identifier: GPL-1.0+ OR BSD-3-Clause+//+// ====================================================================+// Written by Andy Polyakov, @dot-asm, initially for the OpenSSL+// project.+// ====================================================================+//+// SHA256/512 for ARMv8.+//+// Performance in cycles per processed byte and improvement coefficient+// over code generated with "default" compiler:+//+//		SHA256-hw	SHA256(*)	SHA512+// Apple A7	1.97		10.5 (+33%)	6.73 (-1%(**))+// Apple A10	1.30				5.81+// Apple A12	1.31				5.06+// Apple A14/M1	1.30		8.19 (+14%)	2.24 (hw)+// Cortex-A53	2.38		15.5 (+115%)	10.0 (+150%(***))+// Cortex-A57	2.31		11.6 (+86%)	7.51 (+260%(***))+// Cortex-A76	1.60		9.5		6.05+// Cortex-X2	1.60		7.3		2.60 (hw)+// Cortex-X925	1.57		5.97		2.55 (hw)+// Denver	2.01		10.5 (+26%)	6.70 (+8%)+// X-Gene			20.0 (+100%)	12.8 (+300%(***))+// Mongoose	2.36		13.0 (+50%)	8.36 (+33%)+// Kryo		1.92		17.4 (+30%)	11.2 (+8%)+// ThunderX2	2.54		13.2 (+40%)	8.40 (+18%)+// Shapdragon X	1.40		7.43		2.23 (hw)+//+// (*)	Software SHA256 results are of lesser relevance, presented+//	mostly for informational purposes.+// (**)	The result is a trade-off: it's possible to improve it by+//	10% (or by 1 cycle per round), but at the cost of 20% loss+//	on Cortex-A53 (or by 4 cycles per round).+// (***)	Super-impressive coefficients over gcc-generated code are+//	indication of some compiler "pathology", most notably code+//	generated with -mgeneral-regs-only is significantly faster+//	and the gap is only 40-90%.+//+// October 2016.+//+// Originally it was reckoned that it makes no sense to implement NEON+// version of SHA256 for 64-bit processors. This is because performance+// improvement on most wide-spread Cortex-A5x processors was observed+// to be marginal, same on Cortex-A53 and ~10% on A57. But then it was+// observed that 32-bit NEON SHA256 performs significantly better than+// 64-bit scalar version on *some* of the more recent processors. As+// result 64-bit NEON version of SHA256 was added to provide best+// all-round performance. For example it executes ~30% faster on X-Gene+// and Mongoose. [For reference, NEON version of SHA512 is bound to+// deliver much less improvement, likely *negative* on Cortex-A5x.+// Which is why NEON support is limited to SHA256.]++#ifndef	__KERNEL__+# include "arm_arch.h"++#endif++.text++.globl	crypton_sha256_asm_block_data_order+.type	crypton_sha256_asm_block_data_order,%function+.align	6+crypton_sha256_asm_block_data_order:+#ifndef	__KERNEL__+	adrp	x16,crypton_armcap_P+	ldr	w16,[x16,#:lo12:crypton_armcap_P]+	tst	w16,#ARMV8_SHA256+	b.ne	.Lv8_entry+	tst	w16,#ARMV7_NEON+	b.ne	.Lneon_entry+#endif+.inst	0xd503233f				// paciasp+	stp	x29,x30,[sp,#-16*__SIZEOF_POINTER__]!+	add	x29,sp,#0++	stp	x19,x20,[sp,#2*__SIZEOF_POINTER__]+	stp	x21,x22,[sp,#4*__SIZEOF_POINTER__]+	stp	x23,x24,[sp,#6*__SIZEOF_POINTER__]+	stp	x25,x26,[sp,#8*__SIZEOF_POINTER__]+	stp	x27,x28,[sp,#10*__SIZEOF_POINTER__]+	sub	sp,sp,#4*4++	ldp	w20,w21,[x0]				// load context+	ldp	w22,w23,[x0,#2*4]+	lsl	x2,x2,#6+	ldp	w24,w25,[x0,#4*4]+	add	x2,x1,x2+	ldp	w26,w27,[x0,#6*4]+	adr	x30,.LK256+	stp	x0,x2,[x29,#12*__SIZEOF_POINTER__]++.Loop:+	ldp	w3,w4,[x1],#2*4+	ldr	w19,[x30],#4			// *K+++	eor	w28,w21,w22				// magic seed+	str	x1,[x29,#14*__SIZEOF_POINTER__]+#ifndef	__AARCH64EB__+	rev	w3,w3			// 0+#endif+	ror	w16,w24,#6+	add	w27,w27,w19			// h+=K[i]+	eor	w6,w24,w24,ror#14+	and	w17,w25,w24+	bic	w19,w26,w24+	add	w27,w27,w3			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w20,w21			// a^b, b^c in next round+	eor	w16,w16,w6,ror#11	// Sigma1(e)+	ror	w6,w20,#2+	add	w27,w27,w17			// h+=Ch(e,f,g)+	eor	w17,w20,w20,ror#9+	add	w27,w27,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w23,w23,w27			// d+=h+	eor	w28,w28,w21			// Maj(a,b,c)+	eor	w17,w6,w17,ror#13	// Sigma0(a)+	add	w27,w27,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w27,w27,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w4,w4			// 1+#endif+	ldp	w5,w6,[x1],#2*4+	add	w27,w27,w17			// h+=Sigma0(a)+	ror	w16,w23,#6+	add	w26,w26,w28			// h+=K[i]+	eor	w7,w23,w23,ror#14+	and	w17,w24,w23+	bic	w28,w25,w23+	add	w26,w26,w4			// h+=X[i]+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w27,w20			// a^b, b^c in next round+	eor	w16,w16,w7,ror#11	// Sigma1(e)+	ror	w7,w27,#2+	add	w26,w26,w17			// h+=Ch(e,f,g)+	eor	w17,w27,w27,ror#9+	add	w26,w26,w16			// h+=Sigma1(e)+	and	w19,w19,w28			// (b^c)&=(a^b)+	add	w22,w22,w26			// d+=h+	eor	w19,w19,w20			// Maj(a,b,c)+	eor	w17,w7,w17,ror#13	// Sigma0(a)+	add	w26,w26,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	//add	w26,w26,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w5,w5			// 2+#endif+	add	w26,w26,w17			// h+=Sigma0(a)+	ror	w16,w22,#6+	add	w25,w25,w19			// h+=K[i]+	eor	w8,w22,w22,ror#14+	and	w17,w23,w22+	bic	w19,w24,w22+	add	w25,w25,w5			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w26,w27			// a^b, b^c in next round+	eor	w16,w16,w8,ror#11	// Sigma1(e)+	ror	w8,w26,#2+	add	w25,w25,w17			// h+=Ch(e,f,g)+	eor	w17,w26,w26,ror#9+	add	w25,w25,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w21,w21,w25			// d+=h+	eor	w28,w28,w27			// Maj(a,b,c)+	eor	w17,w8,w17,ror#13	// Sigma0(a)+	add	w25,w25,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w25,w25,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w6,w6			// 3+#endif+	ldp	w7,w8,[x1],#2*4+	add	w25,w25,w17			// h+=Sigma0(a)+	ror	w16,w21,#6+	add	w24,w24,w28			// h+=K[i]+	eor	w9,w21,w21,ror#14+	and	w17,w22,w21+	bic	w28,w23,w21+	add	w24,w24,w6			// h+=X[i]+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w25,w26			// a^b, b^c in next round+	eor	w16,w16,w9,ror#11	// Sigma1(e)+	ror	w9,w25,#2+	add	w24,w24,w17			// h+=Ch(e,f,g)+	eor	w17,w25,w25,ror#9+	add	w24,w24,w16			// h+=Sigma1(e)+	and	w19,w19,w28			// (b^c)&=(a^b)+	add	w20,w20,w24			// d+=h+	eor	w19,w19,w26			// Maj(a,b,c)+	eor	w17,w9,w17,ror#13	// Sigma0(a)+	add	w24,w24,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	//add	w24,w24,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w7,w7			// 4+#endif+	add	w24,w24,w17			// h+=Sigma0(a)+	ror	w16,w20,#6+	add	w23,w23,w19			// h+=K[i]+	eor	w10,w20,w20,ror#14+	and	w17,w21,w20+	bic	w19,w22,w20+	add	w23,w23,w7			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w24,w25			// a^b, b^c in next round+	eor	w16,w16,w10,ror#11	// Sigma1(e)+	ror	w10,w24,#2+	add	w23,w23,w17			// h+=Ch(e,f,g)+	eor	w17,w24,w24,ror#9+	add	w23,w23,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w27,w27,w23			// d+=h+	eor	w28,w28,w25			// Maj(a,b,c)+	eor	w17,w10,w17,ror#13	// Sigma0(a)+	add	w23,w23,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w23,w23,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w8,w8			// 5+#endif+	ldp	w9,w10,[x1],#2*4+	add	w23,w23,w17			// h+=Sigma0(a)+	ror	w16,w27,#6+	add	w22,w22,w28			// h+=K[i]+	eor	w11,w27,w27,ror#14+	and	w17,w20,w27+	bic	w28,w21,w27+	add	w22,w22,w8			// h+=X[i]+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w23,w24			// a^b, b^c in next round+	eor	w16,w16,w11,ror#11	// Sigma1(e)+	ror	w11,w23,#2+	add	w22,w22,w17			// h+=Ch(e,f,g)+	eor	w17,w23,w23,ror#9+	add	w22,w22,w16			// h+=Sigma1(e)+	and	w19,w19,w28			// (b^c)&=(a^b)+	add	w26,w26,w22			// d+=h+	eor	w19,w19,w24			// Maj(a,b,c)+	eor	w17,w11,w17,ror#13	// Sigma0(a)+	add	w22,w22,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	//add	w22,w22,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w9,w9			// 6+#endif+	add	w22,w22,w17			// h+=Sigma0(a)+	ror	w16,w26,#6+	add	w21,w21,w19			// h+=K[i]+	eor	w12,w26,w26,ror#14+	and	w17,w27,w26+	bic	w19,w20,w26+	add	w21,w21,w9			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w22,w23			// a^b, b^c in next round+	eor	w16,w16,w12,ror#11	// Sigma1(e)+	ror	w12,w22,#2+	add	w21,w21,w17			// h+=Ch(e,f,g)+	eor	w17,w22,w22,ror#9+	add	w21,w21,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w25,w25,w21			// d+=h+	eor	w28,w28,w23			// Maj(a,b,c)+	eor	w17,w12,w17,ror#13	// Sigma0(a)+	add	w21,w21,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w21,w21,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w10,w10			// 7+#endif+	ldp	w11,w12,[x1],#2*4+	add	w21,w21,w17			// h+=Sigma0(a)+	ror	w16,w25,#6+	add	w20,w20,w28			// h+=K[i]+	eor	w13,w25,w25,ror#14+	and	w17,w26,w25+	bic	w28,w27,w25+	add	w20,w20,w10			// h+=X[i]+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w21,w22			// a^b, b^c in next round+	eor	w16,w16,w13,ror#11	// Sigma1(e)+	ror	w13,w21,#2+	add	w20,w20,w17			// h+=Ch(e,f,g)+	eor	w17,w21,w21,ror#9+	add	w20,w20,w16			// h+=Sigma1(e)+	and	w19,w19,w28			// (b^c)&=(a^b)+	add	w24,w24,w20			// d+=h+	eor	w19,w19,w22			// Maj(a,b,c)+	eor	w17,w13,w17,ror#13	// Sigma0(a)+	add	w20,w20,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	//add	w20,w20,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w11,w11			// 8+#endif+	add	w20,w20,w17			// h+=Sigma0(a)+	ror	w16,w24,#6+	add	w27,w27,w19			// h+=K[i]+	eor	w14,w24,w24,ror#14+	and	w17,w25,w24+	bic	w19,w26,w24+	add	w27,w27,w11			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w20,w21			// a^b, b^c in next round+	eor	w16,w16,w14,ror#11	// Sigma1(e)+	ror	w14,w20,#2+	add	w27,w27,w17			// h+=Ch(e,f,g)+	eor	w17,w20,w20,ror#9+	add	w27,w27,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w23,w23,w27			// d+=h+	eor	w28,w28,w21			// Maj(a,b,c)+	eor	w17,w14,w17,ror#13	// Sigma0(a)+	add	w27,w27,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w27,w27,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w12,w12			// 9+#endif+	ldp	w13,w14,[x1],#2*4+	add	w27,w27,w17			// h+=Sigma0(a)+	ror	w16,w23,#6+	add	w26,w26,w28			// h+=K[i]+	eor	w15,w23,w23,ror#14+	and	w17,w24,w23+	bic	w28,w25,w23+	add	w26,w26,w12			// h+=X[i]+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w27,w20			// a^b, b^c in next round+	eor	w16,w16,w15,ror#11	// Sigma1(e)+	ror	w15,w27,#2+	add	w26,w26,w17			// h+=Ch(e,f,g)+	eor	w17,w27,w27,ror#9+	add	w26,w26,w16			// h+=Sigma1(e)+	and	w19,w19,w28			// (b^c)&=(a^b)+	add	w22,w22,w26			// d+=h+	eor	w19,w19,w20			// Maj(a,b,c)+	eor	w17,w15,w17,ror#13	// Sigma0(a)+	add	w26,w26,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	//add	w26,w26,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w13,w13			// 10+#endif+	add	w26,w26,w17			// h+=Sigma0(a)+	ror	w16,w22,#6+	add	w25,w25,w19			// h+=K[i]+	eor	w0,w22,w22,ror#14+	and	w17,w23,w22+	bic	w19,w24,w22+	add	w25,w25,w13			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w26,w27			// a^b, b^c in next round+	eor	w16,w16,w0,ror#11	// Sigma1(e)+	ror	w0,w26,#2+	add	w25,w25,w17			// h+=Ch(e,f,g)+	eor	w17,w26,w26,ror#9+	add	w25,w25,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w21,w21,w25			// d+=h+	eor	w28,w28,w27			// Maj(a,b,c)+	eor	w17,w0,w17,ror#13	// Sigma0(a)+	add	w25,w25,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w25,w25,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w14,w14			// 11+#endif+	ldp	w15,w0,[x1],#2*4+	add	w25,w25,w17			// h+=Sigma0(a)+	str	w6,[sp,#12]+	ror	w16,w21,#6+	add	w24,w24,w28			// h+=K[i]+	eor	w6,w21,w21,ror#14+	and	w17,w22,w21+	bic	w28,w23,w21+	add	w24,w24,w14			// h+=X[i]+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w25,w26			// a^b, b^c in next round+	eor	w16,w16,w6,ror#11	// Sigma1(e)+	ror	w6,w25,#2+	add	w24,w24,w17			// h+=Ch(e,f,g)+	eor	w17,w25,w25,ror#9+	add	w24,w24,w16			// h+=Sigma1(e)+	and	w19,w19,w28			// (b^c)&=(a^b)+	add	w20,w20,w24			// d+=h+	eor	w19,w19,w26			// Maj(a,b,c)+	eor	w17,w6,w17,ror#13	// Sigma0(a)+	add	w24,w24,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	//add	w24,w24,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w15,w15			// 12+#endif+	add	w24,w24,w17			// h+=Sigma0(a)+	str	w7,[sp,#0]+	ror	w16,w20,#6+	add	w23,w23,w19			// h+=K[i]+	eor	w7,w20,w20,ror#14+	and	w17,w21,w20+	bic	w19,w22,w20+	add	w23,w23,w15			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w24,w25			// a^b, b^c in next round+	eor	w16,w16,w7,ror#11	// Sigma1(e)+	ror	w7,w24,#2+	add	w23,w23,w17			// h+=Ch(e,f,g)+	eor	w17,w24,w24,ror#9+	add	w23,w23,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w27,w27,w23			// d+=h+	eor	w28,w28,w25			// Maj(a,b,c)+	eor	w17,w7,w17,ror#13	// Sigma0(a)+	add	w23,w23,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w23,w23,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w0,w0			// 13+#endif+	ldp	w1,w2,[x1]+	add	w23,w23,w17			// h+=Sigma0(a)+	str	w8,[sp,#4]+	ror	w16,w27,#6+	add	w22,w22,w28			// h+=K[i]+	eor	w8,w27,w27,ror#14+	and	w17,w20,w27+	bic	w28,w21,w27+	add	w22,w22,w0			// h+=X[i]+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w23,w24			// a^b, b^c in next round+	eor	w16,w16,w8,ror#11	// Sigma1(e)+	ror	w8,w23,#2+	add	w22,w22,w17			// h+=Ch(e,f,g)+	eor	w17,w23,w23,ror#9+	add	w22,w22,w16			// h+=Sigma1(e)+	and	w19,w19,w28			// (b^c)&=(a^b)+	add	w26,w26,w22			// d+=h+	eor	w19,w19,w24			// Maj(a,b,c)+	eor	w17,w8,w17,ror#13	// Sigma0(a)+	add	w22,w22,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	//add	w22,w22,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w1,w1			// 14+#endif+	ldr	w6,[sp,#12]+	add	w22,w22,w17			// h+=Sigma0(a)+	str	w9,[sp,#8]+	ror	w16,w26,#6+	add	w21,w21,w19			// h+=K[i]+	eor	w9,w26,w26,ror#14+	and	w17,w27,w26+	bic	w19,w20,w26+	add	w21,w21,w1			// h+=X[i]+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w22,w23			// a^b, b^c in next round+	eor	w16,w16,w9,ror#11	// Sigma1(e)+	ror	w9,w22,#2+	add	w21,w21,w17			// h+=Ch(e,f,g)+	eor	w17,w22,w22,ror#9+	add	w21,w21,w16			// h+=Sigma1(e)+	and	w28,w28,w19			// (b^c)&=(a^b)+	add	w25,w25,w21			// d+=h+	eor	w28,w28,w23			// Maj(a,b,c)+	eor	w17,w9,w17,ror#13	// Sigma0(a)+	add	w21,w21,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	//add	w21,w21,w17			// h+=Sigma0(a)+#ifndef	__AARCH64EB__+	rev	w2,w2			// 15+#endif+	ldr	w7,[sp,#0]+	add	w21,w21,w17			// h+=Sigma0(a)+	str	w10,[sp,#12]+	ror	w16,w25,#6+	add	w20,w20,w28			// h+=K[i]+	ror	w9,w4,#7+	and	w17,w26,w25+	ror	w8,w1,#17+	bic	w28,w27,w25+	ror	w10,w21,#2+	add	w20,w20,w2			// h+=X[i]+	eor	w16,w16,w25,ror#11+	eor	w9,w9,w4,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w21,w22			// a^b, b^c in next round+	eor	w16,w16,w25,ror#25	// Sigma1(e)+	eor	w10,w10,w21,ror#13+	add	w20,w20,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w8,w8,w1,ror#19+	eor	w9,w9,w4,lsr#3	// sigma0(X[i+1])+	add	w20,w20,w16			// h+=Sigma1(e)+	eor	w19,w19,w22			// Maj(a,b,c)+	eor	w17,w10,w21,ror#22	// Sigma0(a)+	eor	w8,w8,w1,lsr#10	// sigma1(X[i+14])+	add	w3,w3,w12+	add	w24,w24,w20			// d+=h+	add	w20,w20,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w3,w3,w9+	add	w20,w20,w17			// h+=Sigma0(a)+	add	w3,w3,w8+.Loop_16_xx:+	ldr	w8,[sp,#4]+	str	w11,[sp,#0]+	ror	w16,w24,#6+	add	w27,w27,w19			// h+=K[i]+	ror	w10,w5,#7+	and	w17,w25,w24+	ror	w9,w2,#17+	bic	w19,w26,w24+	ror	w11,w20,#2+	add	w27,w27,w3			// h+=X[i]+	eor	w16,w16,w24,ror#11+	eor	w10,w10,w5,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w20,w21			// a^b, b^c in next round+	eor	w16,w16,w24,ror#25	// Sigma1(e)+	eor	w11,w11,w20,ror#13+	add	w27,w27,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w9,w9,w2,ror#19+	eor	w10,w10,w5,lsr#3	// sigma0(X[i+1])+	add	w27,w27,w16			// h+=Sigma1(e)+	eor	w28,w28,w21			// Maj(a,b,c)+	eor	w17,w11,w20,ror#22	// Sigma0(a)+	eor	w9,w9,w2,lsr#10	// sigma1(X[i+14])+	add	w4,w4,w13+	add	w23,w23,w27			// d+=h+	add	w27,w27,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w4,w4,w10+	add	w27,w27,w17			// h+=Sigma0(a)+	add	w4,w4,w9+	ldr	w9,[sp,#8]+	str	w12,[sp,#4]+	ror	w16,w23,#6+	add	w26,w26,w28			// h+=K[i]+	ror	w11,w6,#7+	and	w17,w24,w23+	ror	w10,w3,#17+	bic	w28,w25,w23+	ror	w12,w27,#2+	add	w26,w26,w4			// h+=X[i]+	eor	w16,w16,w23,ror#11+	eor	w11,w11,w6,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w27,w20			// a^b, b^c in next round+	eor	w16,w16,w23,ror#25	// Sigma1(e)+	eor	w12,w12,w27,ror#13+	add	w26,w26,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w10,w10,w3,ror#19+	eor	w11,w11,w6,lsr#3	// sigma0(X[i+1])+	add	w26,w26,w16			// h+=Sigma1(e)+	eor	w19,w19,w20			// Maj(a,b,c)+	eor	w17,w12,w27,ror#22	// Sigma0(a)+	eor	w10,w10,w3,lsr#10	// sigma1(X[i+14])+	add	w5,w5,w14+	add	w22,w22,w26			// d+=h+	add	w26,w26,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w5,w5,w11+	add	w26,w26,w17			// h+=Sigma0(a)+	add	w5,w5,w10+	ldr	w10,[sp,#12]+	str	w13,[sp,#8]+	ror	w16,w22,#6+	add	w25,w25,w19			// h+=K[i]+	ror	w12,w7,#7+	and	w17,w23,w22+	ror	w11,w4,#17+	bic	w19,w24,w22+	ror	w13,w26,#2+	add	w25,w25,w5			// h+=X[i]+	eor	w16,w16,w22,ror#11+	eor	w12,w12,w7,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w26,w27			// a^b, b^c in next round+	eor	w16,w16,w22,ror#25	// Sigma1(e)+	eor	w13,w13,w26,ror#13+	add	w25,w25,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w11,w11,w4,ror#19+	eor	w12,w12,w7,lsr#3	// sigma0(X[i+1])+	add	w25,w25,w16			// h+=Sigma1(e)+	eor	w28,w28,w27			// Maj(a,b,c)+	eor	w17,w13,w26,ror#22	// Sigma0(a)+	eor	w11,w11,w4,lsr#10	// sigma1(X[i+14])+	add	w6,w6,w15+	add	w21,w21,w25			// d+=h+	add	w25,w25,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w6,w6,w12+	add	w25,w25,w17			// h+=Sigma0(a)+	add	w6,w6,w11+	ldr	w11,[sp,#0]+	str	w14,[sp,#12]+	ror	w16,w21,#6+	add	w24,w24,w28			// h+=K[i]+	ror	w13,w8,#7+	and	w17,w22,w21+	ror	w12,w5,#17+	bic	w28,w23,w21+	ror	w14,w25,#2+	add	w24,w24,w6			// h+=X[i]+	eor	w16,w16,w21,ror#11+	eor	w13,w13,w8,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w25,w26			// a^b, b^c in next round+	eor	w16,w16,w21,ror#25	// Sigma1(e)+	eor	w14,w14,w25,ror#13+	add	w24,w24,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w12,w12,w5,ror#19+	eor	w13,w13,w8,lsr#3	// sigma0(X[i+1])+	add	w24,w24,w16			// h+=Sigma1(e)+	eor	w19,w19,w26			// Maj(a,b,c)+	eor	w17,w14,w25,ror#22	// Sigma0(a)+	eor	w12,w12,w5,lsr#10	// sigma1(X[i+14])+	add	w7,w7,w0+	add	w20,w20,w24			// d+=h+	add	w24,w24,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w7,w7,w13+	add	w24,w24,w17			// h+=Sigma0(a)+	add	w7,w7,w12+	ldr	w12,[sp,#4]+	str	w15,[sp,#0]+	ror	w16,w20,#6+	add	w23,w23,w19			// h+=K[i]+	ror	w14,w9,#7+	and	w17,w21,w20+	ror	w13,w6,#17+	bic	w19,w22,w20+	ror	w15,w24,#2+	add	w23,w23,w7			// h+=X[i]+	eor	w16,w16,w20,ror#11+	eor	w14,w14,w9,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w24,w25			// a^b, b^c in next round+	eor	w16,w16,w20,ror#25	// Sigma1(e)+	eor	w15,w15,w24,ror#13+	add	w23,w23,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w13,w13,w6,ror#19+	eor	w14,w14,w9,lsr#3	// sigma0(X[i+1])+	add	w23,w23,w16			// h+=Sigma1(e)+	eor	w28,w28,w25			// Maj(a,b,c)+	eor	w17,w15,w24,ror#22	// Sigma0(a)+	eor	w13,w13,w6,lsr#10	// sigma1(X[i+14])+	add	w8,w8,w1+	add	w27,w27,w23			// d+=h+	add	w23,w23,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w8,w8,w14+	add	w23,w23,w17			// h+=Sigma0(a)+	add	w8,w8,w13+	ldr	w13,[sp,#8]+	str	w0,[sp,#4]+	ror	w16,w27,#6+	add	w22,w22,w28			// h+=K[i]+	ror	w15,w10,#7+	and	w17,w20,w27+	ror	w14,w7,#17+	bic	w28,w21,w27+	ror	w0,w23,#2+	add	w22,w22,w8			// h+=X[i]+	eor	w16,w16,w27,ror#11+	eor	w15,w15,w10,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w23,w24			// a^b, b^c in next round+	eor	w16,w16,w27,ror#25	// Sigma1(e)+	eor	w0,w0,w23,ror#13+	add	w22,w22,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w14,w14,w7,ror#19+	eor	w15,w15,w10,lsr#3	// sigma0(X[i+1])+	add	w22,w22,w16			// h+=Sigma1(e)+	eor	w19,w19,w24			// Maj(a,b,c)+	eor	w17,w0,w23,ror#22	// Sigma0(a)+	eor	w14,w14,w7,lsr#10	// sigma1(X[i+14])+	add	w9,w9,w2+	add	w26,w26,w22			// d+=h+	add	w22,w22,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w9,w9,w15+	add	w22,w22,w17			// h+=Sigma0(a)+	add	w9,w9,w14+	ldr	w14,[sp,#12]+	str	w1,[sp,#8]+	ror	w16,w26,#6+	add	w21,w21,w19			// h+=K[i]+	ror	w0,w11,#7+	and	w17,w27,w26+	ror	w15,w8,#17+	bic	w19,w20,w26+	ror	w1,w22,#2+	add	w21,w21,w9			// h+=X[i]+	eor	w16,w16,w26,ror#11+	eor	w0,w0,w11,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w22,w23			// a^b, b^c in next round+	eor	w16,w16,w26,ror#25	// Sigma1(e)+	eor	w1,w1,w22,ror#13+	add	w21,w21,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w15,w15,w8,ror#19+	eor	w0,w0,w11,lsr#3	// sigma0(X[i+1])+	add	w21,w21,w16			// h+=Sigma1(e)+	eor	w28,w28,w23			// Maj(a,b,c)+	eor	w17,w1,w22,ror#22	// Sigma0(a)+	eor	w15,w15,w8,lsr#10	// sigma1(X[i+14])+	add	w10,w10,w3+	add	w25,w25,w21			// d+=h+	add	w21,w21,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w10,w10,w0+	add	w21,w21,w17			// h+=Sigma0(a)+	add	w10,w10,w15+	ldr	w15,[sp,#0]+	str	w2,[sp,#12]+	ror	w16,w25,#6+	add	w20,w20,w28			// h+=K[i]+	ror	w1,w12,#7+	and	w17,w26,w25+	ror	w0,w9,#17+	bic	w28,w27,w25+	ror	w2,w21,#2+	add	w20,w20,w10			// h+=X[i]+	eor	w16,w16,w25,ror#11+	eor	w1,w1,w12,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w21,w22			// a^b, b^c in next round+	eor	w16,w16,w25,ror#25	// Sigma1(e)+	eor	w2,w2,w21,ror#13+	add	w20,w20,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w0,w0,w9,ror#19+	eor	w1,w1,w12,lsr#3	// sigma0(X[i+1])+	add	w20,w20,w16			// h+=Sigma1(e)+	eor	w19,w19,w22			// Maj(a,b,c)+	eor	w17,w2,w21,ror#22	// Sigma0(a)+	eor	w0,w0,w9,lsr#10	// sigma1(X[i+14])+	add	w11,w11,w4+	add	w24,w24,w20			// d+=h+	add	w20,w20,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w11,w11,w1+	add	w20,w20,w17			// h+=Sigma0(a)+	add	w11,w11,w0+	ldr	w0,[sp,#4]+	str	w3,[sp,#0]+	ror	w16,w24,#6+	add	w27,w27,w19			// h+=K[i]+	ror	w2,w13,#7+	and	w17,w25,w24+	ror	w1,w10,#17+	bic	w19,w26,w24+	ror	w3,w20,#2+	add	w27,w27,w11			// h+=X[i]+	eor	w16,w16,w24,ror#11+	eor	w2,w2,w13,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w20,w21			// a^b, b^c in next round+	eor	w16,w16,w24,ror#25	// Sigma1(e)+	eor	w3,w3,w20,ror#13+	add	w27,w27,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w1,w1,w10,ror#19+	eor	w2,w2,w13,lsr#3	// sigma0(X[i+1])+	add	w27,w27,w16			// h+=Sigma1(e)+	eor	w28,w28,w21			// Maj(a,b,c)+	eor	w17,w3,w20,ror#22	// Sigma0(a)+	eor	w1,w1,w10,lsr#10	// sigma1(X[i+14])+	add	w12,w12,w5+	add	w23,w23,w27			// d+=h+	add	w27,w27,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w12,w12,w2+	add	w27,w27,w17			// h+=Sigma0(a)+	add	w12,w12,w1+	ldr	w1,[sp,#8]+	str	w4,[sp,#4]+	ror	w16,w23,#6+	add	w26,w26,w28			// h+=K[i]+	ror	w3,w14,#7+	and	w17,w24,w23+	ror	w2,w11,#17+	bic	w28,w25,w23+	ror	w4,w27,#2+	add	w26,w26,w12			// h+=X[i]+	eor	w16,w16,w23,ror#11+	eor	w3,w3,w14,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w27,w20			// a^b, b^c in next round+	eor	w16,w16,w23,ror#25	// Sigma1(e)+	eor	w4,w4,w27,ror#13+	add	w26,w26,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w2,w2,w11,ror#19+	eor	w3,w3,w14,lsr#3	// sigma0(X[i+1])+	add	w26,w26,w16			// h+=Sigma1(e)+	eor	w19,w19,w20			// Maj(a,b,c)+	eor	w17,w4,w27,ror#22	// Sigma0(a)+	eor	w2,w2,w11,lsr#10	// sigma1(X[i+14])+	add	w13,w13,w6+	add	w22,w22,w26			// d+=h+	add	w26,w26,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w13,w13,w3+	add	w26,w26,w17			// h+=Sigma0(a)+	add	w13,w13,w2+	ldr	w2,[sp,#12]+	str	w5,[sp,#8]+	ror	w16,w22,#6+	add	w25,w25,w19			// h+=K[i]+	ror	w4,w15,#7+	and	w17,w23,w22+	ror	w3,w12,#17+	bic	w19,w24,w22+	ror	w5,w26,#2+	add	w25,w25,w13			// h+=X[i]+	eor	w16,w16,w22,ror#11+	eor	w4,w4,w15,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w26,w27			// a^b, b^c in next round+	eor	w16,w16,w22,ror#25	// Sigma1(e)+	eor	w5,w5,w26,ror#13+	add	w25,w25,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w3,w3,w12,ror#19+	eor	w4,w4,w15,lsr#3	// sigma0(X[i+1])+	add	w25,w25,w16			// h+=Sigma1(e)+	eor	w28,w28,w27			// Maj(a,b,c)+	eor	w17,w5,w26,ror#22	// Sigma0(a)+	eor	w3,w3,w12,lsr#10	// sigma1(X[i+14])+	add	w14,w14,w7+	add	w21,w21,w25			// d+=h+	add	w25,w25,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w14,w14,w4+	add	w25,w25,w17			// h+=Sigma0(a)+	add	w14,w14,w3+	ldr	w3,[sp,#0]+	str	w6,[sp,#12]+	ror	w16,w21,#6+	add	w24,w24,w28			// h+=K[i]+	ror	w5,w0,#7+	and	w17,w22,w21+	ror	w4,w13,#17+	bic	w28,w23,w21+	ror	w6,w25,#2+	add	w24,w24,w14			// h+=X[i]+	eor	w16,w16,w21,ror#11+	eor	w5,w5,w0,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w25,w26			// a^b, b^c in next round+	eor	w16,w16,w21,ror#25	// Sigma1(e)+	eor	w6,w6,w25,ror#13+	add	w24,w24,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w4,w4,w13,ror#19+	eor	w5,w5,w0,lsr#3	// sigma0(X[i+1])+	add	w24,w24,w16			// h+=Sigma1(e)+	eor	w19,w19,w26			// Maj(a,b,c)+	eor	w17,w6,w25,ror#22	// Sigma0(a)+	eor	w4,w4,w13,lsr#10	// sigma1(X[i+14])+	add	w15,w15,w8+	add	w20,w20,w24			// d+=h+	add	w24,w24,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w15,w15,w5+	add	w24,w24,w17			// h+=Sigma0(a)+	add	w15,w15,w4+	ldr	w4,[sp,#4]+	str	w7,[sp,#0]+	ror	w16,w20,#6+	add	w23,w23,w19			// h+=K[i]+	ror	w6,w1,#7+	and	w17,w21,w20+	ror	w5,w14,#17+	bic	w19,w22,w20+	ror	w7,w24,#2+	add	w23,w23,w15			// h+=X[i]+	eor	w16,w16,w20,ror#11+	eor	w6,w6,w1,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w24,w25			// a^b, b^c in next round+	eor	w16,w16,w20,ror#25	// Sigma1(e)+	eor	w7,w7,w24,ror#13+	add	w23,w23,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w5,w5,w14,ror#19+	eor	w6,w6,w1,lsr#3	// sigma0(X[i+1])+	add	w23,w23,w16			// h+=Sigma1(e)+	eor	w28,w28,w25			// Maj(a,b,c)+	eor	w17,w7,w24,ror#22	// Sigma0(a)+	eor	w5,w5,w14,lsr#10	// sigma1(X[i+14])+	add	w0,w0,w9+	add	w27,w27,w23			// d+=h+	add	w23,w23,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w0,w0,w6+	add	w23,w23,w17			// h+=Sigma0(a)+	add	w0,w0,w5+	ldr	w5,[sp,#8]+	str	w8,[sp,#4]+	ror	w16,w27,#6+	add	w22,w22,w28			// h+=K[i]+	ror	w7,w2,#7+	and	w17,w20,w27+	ror	w6,w15,#17+	bic	w28,w21,w27+	ror	w8,w23,#2+	add	w22,w22,w0			// h+=X[i]+	eor	w16,w16,w27,ror#11+	eor	w7,w7,w2,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w23,w24			// a^b, b^c in next round+	eor	w16,w16,w27,ror#25	// Sigma1(e)+	eor	w8,w8,w23,ror#13+	add	w22,w22,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w6,w6,w15,ror#19+	eor	w7,w7,w2,lsr#3	// sigma0(X[i+1])+	add	w22,w22,w16			// h+=Sigma1(e)+	eor	w19,w19,w24			// Maj(a,b,c)+	eor	w17,w8,w23,ror#22	// Sigma0(a)+	eor	w6,w6,w15,lsr#10	// sigma1(X[i+14])+	add	w1,w1,w10+	add	w26,w26,w22			// d+=h+	add	w22,w22,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w1,w1,w7+	add	w22,w22,w17			// h+=Sigma0(a)+	add	w1,w1,w6+	ldr	w6,[sp,#12]+	str	w9,[sp,#8]+	ror	w16,w26,#6+	add	w21,w21,w19			// h+=K[i]+	ror	w8,w3,#7+	and	w17,w27,w26+	ror	w7,w0,#17+	bic	w19,w20,w26+	ror	w9,w22,#2+	add	w21,w21,w1			// h+=X[i]+	eor	w16,w16,w26,ror#11+	eor	w8,w8,w3,ror#18+	orr	w17,w17,w19			// Ch(e,f,g)+	eor	w19,w22,w23			// a^b, b^c in next round+	eor	w16,w16,w26,ror#25	// Sigma1(e)+	eor	w9,w9,w22,ror#13+	add	w21,w21,w17			// h+=Ch(e,f,g)+	and	w28,w28,w19			// (b^c)&=(a^b)+	eor	w7,w7,w0,ror#19+	eor	w8,w8,w3,lsr#3	// sigma0(X[i+1])+	add	w21,w21,w16			// h+=Sigma1(e)+	eor	w28,w28,w23			// Maj(a,b,c)+	eor	w17,w9,w22,ror#22	// Sigma0(a)+	eor	w7,w7,w0,lsr#10	// sigma1(X[i+14])+	add	w2,w2,w11+	add	w25,w25,w21			// d+=h+	add	w21,w21,w28			// h+=Maj(a,b,c)+	ldr	w28,[x30],#4		// *K++, w19 in next round+	add	w2,w2,w8+	add	w21,w21,w17			// h+=Sigma0(a)+	add	w2,w2,w7+	ldr	w7,[sp,#0]+	str	w10,[sp,#12]+	ror	w16,w25,#6+	add	w20,w20,w28			// h+=K[i]+	ror	w9,w4,#7+	and	w17,w26,w25+	ror	w8,w1,#17+	bic	w28,w27,w25+	ror	w10,w21,#2+	add	w20,w20,w2			// h+=X[i]+	eor	w16,w16,w25,ror#11+	eor	w9,w9,w4,ror#18+	orr	w17,w17,w28			// Ch(e,f,g)+	eor	w28,w21,w22			// a^b, b^c in next round+	eor	w16,w16,w25,ror#25	// Sigma1(e)+	eor	w10,w10,w21,ror#13+	add	w20,w20,w17			// h+=Ch(e,f,g)+	and	w19,w19,w28			// (b^c)&=(a^b)+	eor	w8,w8,w1,ror#19+	eor	w9,w9,w4,lsr#3	// sigma0(X[i+1])+	add	w20,w20,w16			// h+=Sigma1(e)+	eor	w19,w19,w22			// Maj(a,b,c)+	eor	w17,w10,w21,ror#22	// Sigma0(a)+	eor	w8,w8,w1,lsr#10	// sigma1(X[i+14])+	add	w3,w3,w12+	add	w24,w24,w20			// d+=h+	add	w20,w20,w19			// h+=Maj(a,b,c)+	ldr	w19,[x30],#4		// *K++, w28 in next round+	add	w3,w3,w9+	add	w20,w20,w17			// h+=Sigma0(a)+	add	w3,w3,w8+	cbnz	w19,.Loop_16_xx++	ldp	x0,x2,[x29,#12*__SIZEOF_POINTER__]+	ldr	x1,[x29,#14*__SIZEOF_POINTER__]+	sub	x30,x30,#260++	ldp	w3,w4,[x0]+	ldp	w5,w6,[x0,#2*4]+	add	x1,x1,#14*4+	ldp	w7,w8,[x0,#4*4]+	add	w20,w20,w3+	ldp	w9,w10,[x0,#6*4]+	add	w21,w21,w4+	add	w22,w22,w5+	add	w23,w23,w6+	stp	w20,w21,[x0]+	add	w24,w24,w7+	add	w25,w25,w8+	stp	w22,w23,[x0,#2*4]+	add	w26,w26,w9+	add	w27,w27,w10+	cmp	x1,x2+	stp	w24,w25,[x0,#4*4]+	stp	w26,w27,[x0,#6*4]+	b.ne	.Loop++	ldp	x19,x20,[x29,#2*__SIZEOF_POINTER__]+	add	sp,sp,#4*4+	ldp	x21,x22,[x29,#4*__SIZEOF_POINTER__]+	ldp	x23,x24,[x29,#6*__SIZEOF_POINTER__]+	ldp	x25,x26,[x29,#8*__SIZEOF_POINTER__]+	ldp	x27,x28,[x29,#10*__SIZEOF_POINTER__]+	ldp	x29,x30,[sp],#16*__SIZEOF_POINTER__+.inst	0xd50323bf				// autiasp+	ret+.size	crypton_sha256_asm_block_data_order,.-crypton_sha256_asm_block_data_order++.align	6+.type	.LK256,%object+.LK256:+.long	0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5+.long	0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5+.long	0xd807aa98,0x12835b01,0x243185be,0x550c7dc3+.long	0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174+.long	0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc+.long	0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da+.long	0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7+.long	0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967+.long	0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13+.long	0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85+.long	0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3+.long	0xd192e819,0xd6990624,0xf40e3585,0x106aa070+.long	0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5+.long	0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3+.long	0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208+.long	0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2+.long	0	//terminator+.size	.LK256,.-.LK256+.byte	83,72,65,50,53,54,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.align	2+.align	2+#ifndef	__KERNEL__+.type	crypton_sha256_asm_block_armv8,%function+.align	6+crypton_sha256_asm_block_armv8:+.Lv8_entry:+	stp	x29,x30,[sp,#-2*__SIZEOF_POINTER__]!+	add	x29,sp,#0++	ld1	{v0.4s,v1.4s},[x0]+	adr	x3,.LK256++.Loop_hw:+	ld1	{v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64+	sub	x2,x2,#1+	ld1	{v16.4s},[x3],#16+	rev32	v4.16b,v4.16b+	rev32	v5.16b,v5.16b+	rev32	v6.16b,v6.16b+	rev32	v7.16b,v7.16b+	orr	v18.16b,v0.16b,v0.16b		// offload+	orr	v19.16b,v1.16b,v1.16b+	ld1	{v17.4s},[x3],#16+	add	v16.4s,v16.4s,v4.4s+.inst	0x5e2828a4	//sha256su0 v4.16b,v5.16b+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.inst	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s+.inst	0x5e0760c4	//sha256su1 v4.16b,v6.16b,v7.16b+	ld1	{v16.4s},[x3],#16+	add	v17.4s,v17.4s,v5.4s+.inst	0x5e2828c5	//sha256su0 v5.16b,v6.16b+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.inst	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s+.inst	0x5e0460e5	//sha256su1 v5.16b,v7.16b,v4.16b+	ld1	{v17.4s},[x3],#16+	add	v16.4s,v16.4s,v6.4s+.inst	0x5e2828e6	//sha256su0 v6.16b,v7.16b+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.inst	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s+.inst	0x5e056086	//sha256su1 v6.16b,v4.16b,v5.16b+	ld1	{v16.4s},[x3],#16+	add	v17.4s,v17.4s,v7.4s+.inst	0x5e282887	//sha256su0 v7.16b,v4.16b+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.inst	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s+.inst	0x5e0660a7	//sha256su1 v7.16b,v5.16b,v6.16b+	ld1	{v17.4s},[x3],#16+	add	v16.4s,v16.4s,v4.4s+.inst	0x5e2828a4	//sha256su0 v4.16b,v5.16b+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.inst	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s+.inst	0x5e0760c4	//sha256su1 v4.16b,v6.16b,v7.16b+	ld1	{v16.4s},[x3],#16+	add	v17.4s,v17.4s,v5.4s+.inst	0x5e2828c5	//sha256su0 v5.16b,v6.16b+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.inst	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s+.inst	0x5e0460e5	//sha256su1 v5.16b,v7.16b,v4.16b+	ld1	{v17.4s},[x3],#16+	add	v16.4s,v16.4s,v6.4s+.inst	0x5e2828e6	//sha256su0 v6.16b,v7.16b+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.inst	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s+.inst	0x5e056086	//sha256su1 v6.16b,v4.16b,v5.16b+	ld1	{v16.4s},[x3],#16+	add	v17.4s,v17.4s,v7.4s+.inst	0x5e282887	//sha256su0 v7.16b,v4.16b+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.inst	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s+.inst	0x5e0660a7	//sha256su1 v7.16b,v5.16b,v6.16b+	ld1	{v17.4s},[x3],#16+	add	v16.4s,v16.4s,v4.4s+.inst	0x5e2828a4	//sha256su0 v4.16b,v5.16b+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.inst	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s+.inst	0x5e0760c4	//sha256su1 v4.16b,v6.16b,v7.16b+	ld1	{v16.4s},[x3],#16+	add	v17.4s,v17.4s,v5.4s+.inst	0x5e2828c5	//sha256su0 v5.16b,v6.16b+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.inst	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s+.inst	0x5e0460e5	//sha256su1 v5.16b,v7.16b,v4.16b+	ld1	{v17.4s},[x3],#16+	add	v16.4s,v16.4s,v6.4s+.inst	0x5e2828e6	//sha256su0 v6.16b,v7.16b+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.inst	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s+.inst	0x5e056086	//sha256su1 v6.16b,v4.16b,v5.16b+	ld1	{v16.4s},[x3],#16+	add	v17.4s,v17.4s,v7.4s+.inst	0x5e282887	//sha256su0 v7.16b,v4.16b+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.inst	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s+.inst	0x5e0660a7	//sha256su1 v7.16b,v5.16b,v6.16b+	ld1	{v17.4s},[x3],#16+	add	v16.4s,v16.4s,v4.4s+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.inst	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s++	ld1	{v16.4s},[x3],#16+	add	v17.4s,v17.4s,v5.4s+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.inst	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s++	ld1	{v17.4s},[x3]+	add	v16.4s,v16.4s,v6.4s+	sub	x3,x3,#64*4-16+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e104020	//sha256h v0.16b,v1.16b,v16.4s+.inst	0x5e105041	//sha256h2 v1.16b,v2.16b,v16.4s++	add	v17.4s,v17.4s,v7.4s+	orr	v2.16b,v0.16b,v0.16b+.inst	0x5e114020	//sha256h v0.16b,v1.16b,v17.4s+.inst	0x5e115041	//sha256h2 v1.16b,v2.16b,v17.4s++	add	v0.4s,v0.4s,v18.4s+	add	v1.4s,v1.4s,v19.4s++	cbnz	x2,.Loop_hw++	st1	{v0.4s,v1.4s},[x0]++	ldr	x29,[sp],#2*__SIZEOF_POINTER__+	ret+.size	crypton_sha256_asm_block_armv8,.-crypton_sha256_asm_block_armv8+#endif+#ifdef	__KERNEL__+.globl	crypton_sha256_asm_block_neon+#endif+.type	crypton_sha256_asm_block_neon,%function+.align	4+crypton_sha256_asm_block_neon:+.Lneon_entry:+	stp	x29, x30, [sp, #-2*__SIZEOF_POINTER__]!+	mov	x29, sp+	sub	sp,sp,#16*4++	adr	x16,.LK256+	add	x2,x1,x2,lsl#6	// len to point at the end of inp++	ld1	{v0.16b},[x1], #16+	ld1	{v1.16b},[x1], #16+	ld1	{v2.16b},[x1], #16+	ld1	{v3.16b},[x1], #16+	ld1	{v4.4s},[x16], #16+	ld1	{v5.4s},[x16], #16+	ld1	{v6.4s},[x16], #16+	ld1	{v7.4s},[x16], #16+	rev32	v0.16b,v0.16b		// yes, even on+	rev32	v1.16b,v1.16b		// big-endian+	rev32	v2.16b,v2.16b+	rev32	v3.16b,v3.16b+	mov	x17,sp+	add	v4.4s,v4.4s,v0.4s+	add	v5.4s,v5.4s,v1.4s+	add	v6.4s,v6.4s,v2.4s+	st1	{v4.4s,v5.4s},[x17], #32+	add	v7.4s,v7.4s,v3.4s+	st1	{v6.4s,v7.4s},[x17]+	sub	x17,x17,#32++	ldp	w3,w4,[x0]+	ldp	w5,w6,[x0,#8]+	ldp	w7,w8,[x0,#16]+	ldp	w9,w10,[x0,#24]+	ldr	w12,[sp,#0]+	mov	w13,wzr+	eor	w14,w4,w5+	mov	w15,wzr+	b	.L_00_48++.align	4+.L_00_48:+	ext	v4.16b,v0.16b,v1.16b,#4+	add	w10,w10,w12+	add	w3,w3,w15+	and	w12,w8,w7+	bic	w15,w9,w7+	ext	v7.16b,v2.16b,v3.16b,#4+	eor	w11,w7,w7,ror#5+	add	w3,w3,w13+	mov	d19,v3.d[1]+	orr	w12,w12,w15+	eor	w11,w11,w7,ror#19+	ushr	v6.4s,v4.4s,#7+	eor	w15,w3,w3,ror#11+	ushr	v5.4s,v4.4s,#3+	add	w10,w10,w12+	add	v0.4s,v0.4s,v7.4s+	ror	w11,w11,#6+	sli	v6.4s,v4.4s,#25+	eor	w13,w3,w4+	eor	w15,w15,w3,ror#20+	ushr	v7.4s,v4.4s,#18+	add	w10,w10,w11+	ldr	w12,[sp,#4]+	and	w14,w14,w13+	eor	v5.16b,v5.16b,v6.16b+	ror	w15,w15,#2+	add	w6,w6,w10+	sli	v7.4s,v4.4s,#14+	eor	w14,w14,w4+	ushr	v16.4s,v19.4s,#17+	add	w9,w9,w12+	add	w10,w10,w15+	and	w12,w7,w6+	eor	v5.16b,v5.16b,v7.16b+	bic	w15,w8,w6+	eor	w11,w6,w6,ror#5+	sli	v16.4s,v19.4s,#15+	add	w10,w10,w14+	orr	w12,w12,w15+	ushr	v17.4s,v19.4s,#10+	eor	w11,w11,w6,ror#19+	eor	w15,w10,w10,ror#11+	ushr	v7.4s,v19.4s,#19+	add	w9,w9,w12+	ror	w11,w11,#6+	add	v0.4s,v0.4s,v5.4s+	eor	w14,w10,w3+	eor	w15,w15,w10,ror#20+	sli	v7.4s,v19.4s,#13+	add	w9,w9,w11+	ldr	w12,[sp,#8]+	and	w13,w13,w14+	eor	v17.16b,v17.16b,v16.16b+	ror	w15,w15,#2+	add	w5,w5,w9+	eor	w13,w13,w3+	eor	v17.16b,v17.16b,v7.16b+	add	w8,w8,w12+	add	w9,w9,w15+	and	w12,w6,w5+	add	v0.4s,v0.4s,v17.4s+	bic	w15,w7,w5+	eor	w11,w5,w5,ror#5+	add	w9,w9,w13+	ushr	v18.4s,v0.4s,#17+	orr	w12,w12,w15+	ushr	v19.4s,v0.4s,#10+	eor	w11,w11,w5,ror#19+	eor	w15,w9,w9,ror#11+	sli	v18.4s,v0.4s,#15+	add	w8,w8,w12+	ushr	v17.4s,v0.4s,#19+	ror	w11,w11,#6+	eor	w13,w9,w10+	eor	v19.16b,v19.16b,v18.16b+	eor	w15,w15,w9,ror#20+	add	w8,w8,w11+	sli	v17.4s,v0.4s,#13+	ldr	w12,[sp,#12]+	and	w14,w14,w13+	ror	w15,w15,#2+	ld1	{v4.4s},[x16], #16+	add	w4,w4,w8+	eor	v19.16b,v19.16b,v17.16b+	eor	w14,w14,w10+	eor	v17.16b,v17.16b,v17.16b+	add	w7,w7,w12+	add	w8,w8,w15+	and	w12,w5,w4+	mov	v17.d[1],v19.d[0]+	bic	w15,w6,w4+	eor	w11,w4,w4,ror#5+	add	w8,w8,w14+	add	v0.4s,v0.4s,v17.4s+	orr	w12,w12,w15+	eor	w11,w11,w4,ror#19+	eor	w15,w8,w8,ror#11+	add	v4.4s,v4.4s,v0.4s+	add	w7,w7,w12+	ror	w11,w11,#6+	eor	w14,w8,w9+	eor	w15,w15,w8,ror#20+	add	w7,w7,w11+	ldr	w12,[sp,#16]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w3,w3,w7+	eor	w13,w13,w9+	st1	{v4.4s},[x17], #16+	ext	v4.16b,v1.16b,v2.16b,#4+	add	w6,w6,w12+	add	w7,w7,w15+	and	w12,w4,w3+	bic	w15,w5,w3+	ext	v7.16b,v3.16b,v0.16b,#4+	eor	w11,w3,w3,ror#5+	add	w7,w7,w13+	mov	d19,v0.d[1]+	orr	w12,w12,w15+	eor	w11,w11,w3,ror#19+	ushr	v6.4s,v4.4s,#7+	eor	w15,w7,w7,ror#11+	ushr	v5.4s,v4.4s,#3+	add	w6,w6,w12+	add	v1.4s,v1.4s,v7.4s+	ror	w11,w11,#6+	sli	v6.4s,v4.4s,#25+	eor	w13,w7,w8+	eor	w15,w15,w7,ror#20+	ushr	v7.4s,v4.4s,#18+	add	w6,w6,w11+	ldr	w12,[sp,#20]+	and	w14,w14,w13+	eor	v5.16b,v5.16b,v6.16b+	ror	w15,w15,#2+	add	w10,w10,w6+	sli	v7.4s,v4.4s,#14+	eor	w14,w14,w8+	ushr	v16.4s,v19.4s,#17+	add	w5,w5,w12+	add	w6,w6,w15+	and	w12,w3,w10+	eor	v5.16b,v5.16b,v7.16b+	bic	w15,w4,w10+	eor	w11,w10,w10,ror#5+	sli	v16.4s,v19.4s,#15+	add	w6,w6,w14+	orr	w12,w12,w15+	ushr	v17.4s,v19.4s,#10+	eor	w11,w11,w10,ror#19+	eor	w15,w6,w6,ror#11+	ushr	v7.4s,v19.4s,#19+	add	w5,w5,w12+	ror	w11,w11,#6+	add	v1.4s,v1.4s,v5.4s+	eor	w14,w6,w7+	eor	w15,w15,w6,ror#20+	sli	v7.4s,v19.4s,#13+	add	w5,w5,w11+	ldr	w12,[sp,#24]+	and	w13,w13,w14+	eor	v17.16b,v17.16b,v16.16b+	ror	w15,w15,#2+	add	w9,w9,w5+	eor	w13,w13,w7+	eor	v17.16b,v17.16b,v7.16b+	add	w4,w4,w12+	add	w5,w5,w15+	and	w12,w10,w9+	add	v1.4s,v1.4s,v17.4s+	bic	w15,w3,w9+	eor	w11,w9,w9,ror#5+	add	w5,w5,w13+	ushr	v18.4s,v1.4s,#17+	orr	w12,w12,w15+	ushr	v19.4s,v1.4s,#10+	eor	w11,w11,w9,ror#19+	eor	w15,w5,w5,ror#11+	sli	v18.4s,v1.4s,#15+	add	w4,w4,w12+	ushr	v17.4s,v1.4s,#19+	ror	w11,w11,#6+	eor	w13,w5,w6+	eor	v19.16b,v19.16b,v18.16b+	eor	w15,w15,w5,ror#20+	add	w4,w4,w11+	sli	v17.4s,v1.4s,#13+	ldr	w12,[sp,#28]+	and	w14,w14,w13+	ror	w15,w15,#2+	ld1	{v4.4s},[x16], #16+	add	w8,w8,w4+	eor	v19.16b,v19.16b,v17.16b+	eor	w14,w14,w6+	eor	v17.16b,v17.16b,v17.16b+	add	w3,w3,w12+	add	w4,w4,w15+	and	w12,w9,w8+	mov	v17.d[1],v19.d[0]+	bic	w15,w10,w8+	eor	w11,w8,w8,ror#5+	add	w4,w4,w14+	add	v1.4s,v1.4s,v17.4s+	orr	w12,w12,w15+	eor	w11,w11,w8,ror#19+	eor	w15,w4,w4,ror#11+	add	v4.4s,v4.4s,v1.4s+	add	w3,w3,w12+	ror	w11,w11,#6+	eor	w14,w4,w5+	eor	w15,w15,w4,ror#20+	add	w3,w3,w11+	ldr	w12,[sp,#32]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w7,w7,w3+	eor	w13,w13,w5+	st1	{v4.4s},[x17], #16+	ext	v4.16b,v2.16b,v3.16b,#4+	add	w10,w10,w12+	add	w3,w3,w15+	and	w12,w8,w7+	bic	w15,w9,w7+	ext	v7.16b,v0.16b,v1.16b,#4+	eor	w11,w7,w7,ror#5+	add	w3,w3,w13+	mov	d19,v1.d[1]+	orr	w12,w12,w15+	eor	w11,w11,w7,ror#19+	ushr	v6.4s,v4.4s,#7+	eor	w15,w3,w3,ror#11+	ushr	v5.4s,v4.4s,#3+	add	w10,w10,w12+	add	v2.4s,v2.4s,v7.4s+	ror	w11,w11,#6+	sli	v6.4s,v4.4s,#25+	eor	w13,w3,w4+	eor	w15,w15,w3,ror#20+	ushr	v7.4s,v4.4s,#18+	add	w10,w10,w11+	ldr	w12,[sp,#36]+	and	w14,w14,w13+	eor	v5.16b,v5.16b,v6.16b+	ror	w15,w15,#2+	add	w6,w6,w10+	sli	v7.4s,v4.4s,#14+	eor	w14,w14,w4+	ushr	v16.4s,v19.4s,#17+	add	w9,w9,w12+	add	w10,w10,w15+	and	w12,w7,w6+	eor	v5.16b,v5.16b,v7.16b+	bic	w15,w8,w6+	eor	w11,w6,w6,ror#5+	sli	v16.4s,v19.4s,#15+	add	w10,w10,w14+	orr	w12,w12,w15+	ushr	v17.4s,v19.4s,#10+	eor	w11,w11,w6,ror#19+	eor	w15,w10,w10,ror#11+	ushr	v7.4s,v19.4s,#19+	add	w9,w9,w12+	ror	w11,w11,#6+	add	v2.4s,v2.4s,v5.4s+	eor	w14,w10,w3+	eor	w15,w15,w10,ror#20+	sli	v7.4s,v19.4s,#13+	add	w9,w9,w11+	ldr	w12,[sp,#40]+	and	w13,w13,w14+	eor	v17.16b,v17.16b,v16.16b+	ror	w15,w15,#2+	add	w5,w5,w9+	eor	w13,w13,w3+	eor	v17.16b,v17.16b,v7.16b+	add	w8,w8,w12+	add	w9,w9,w15+	and	w12,w6,w5+	add	v2.4s,v2.4s,v17.4s+	bic	w15,w7,w5+	eor	w11,w5,w5,ror#5+	add	w9,w9,w13+	ushr	v18.4s,v2.4s,#17+	orr	w12,w12,w15+	ushr	v19.4s,v2.4s,#10+	eor	w11,w11,w5,ror#19+	eor	w15,w9,w9,ror#11+	sli	v18.4s,v2.4s,#15+	add	w8,w8,w12+	ushr	v17.4s,v2.4s,#19+	ror	w11,w11,#6+	eor	w13,w9,w10+	eor	v19.16b,v19.16b,v18.16b+	eor	w15,w15,w9,ror#20+	add	w8,w8,w11+	sli	v17.4s,v2.4s,#13+	ldr	w12,[sp,#44]+	and	w14,w14,w13+	ror	w15,w15,#2+	ld1	{v4.4s},[x16], #16+	add	w4,w4,w8+	eor	v19.16b,v19.16b,v17.16b+	eor	w14,w14,w10+	eor	v17.16b,v17.16b,v17.16b+	add	w7,w7,w12+	add	w8,w8,w15+	and	w12,w5,w4+	mov	v17.d[1],v19.d[0]+	bic	w15,w6,w4+	eor	w11,w4,w4,ror#5+	add	w8,w8,w14+	add	v2.4s,v2.4s,v17.4s+	orr	w12,w12,w15+	eor	w11,w11,w4,ror#19+	eor	w15,w8,w8,ror#11+	add	v4.4s,v4.4s,v2.4s+	add	w7,w7,w12+	ror	w11,w11,#6+	eor	w14,w8,w9+	eor	w15,w15,w8,ror#20+	add	w7,w7,w11+	ldr	w12,[sp,#48]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w3,w3,w7+	eor	w13,w13,w9+	st1	{v4.4s},[x17], #16+	ext	v4.16b,v3.16b,v0.16b,#4+	add	w6,w6,w12+	add	w7,w7,w15+	and	w12,w4,w3+	bic	w15,w5,w3+	ext	v7.16b,v1.16b,v2.16b,#4+	eor	w11,w3,w3,ror#5+	add	w7,w7,w13+	mov	d19,v2.d[1]+	orr	w12,w12,w15+	eor	w11,w11,w3,ror#19+	ushr	v6.4s,v4.4s,#7+	eor	w15,w7,w7,ror#11+	ushr	v5.4s,v4.4s,#3+	add	w6,w6,w12+	add	v3.4s,v3.4s,v7.4s+	ror	w11,w11,#6+	sli	v6.4s,v4.4s,#25+	eor	w13,w7,w8+	eor	w15,w15,w7,ror#20+	ushr	v7.4s,v4.4s,#18+	add	w6,w6,w11+	ldr	w12,[sp,#52]+	and	w14,w14,w13+	eor	v5.16b,v5.16b,v6.16b+	ror	w15,w15,#2+	add	w10,w10,w6+	sli	v7.4s,v4.4s,#14+	eor	w14,w14,w8+	ushr	v16.4s,v19.4s,#17+	add	w5,w5,w12+	add	w6,w6,w15+	and	w12,w3,w10+	eor	v5.16b,v5.16b,v7.16b+	bic	w15,w4,w10+	eor	w11,w10,w10,ror#5+	sli	v16.4s,v19.4s,#15+	add	w6,w6,w14+	orr	w12,w12,w15+	ushr	v17.4s,v19.4s,#10+	eor	w11,w11,w10,ror#19+	eor	w15,w6,w6,ror#11+	ushr	v7.4s,v19.4s,#19+	add	w5,w5,w12+	ror	w11,w11,#6+	add	v3.4s,v3.4s,v5.4s+	eor	w14,w6,w7+	eor	w15,w15,w6,ror#20+	sli	v7.4s,v19.4s,#13+	add	w5,w5,w11+	ldr	w12,[sp,#56]+	and	w13,w13,w14+	eor	v17.16b,v17.16b,v16.16b+	ror	w15,w15,#2+	add	w9,w9,w5+	eor	w13,w13,w7+	eor	v17.16b,v17.16b,v7.16b+	add	w4,w4,w12+	add	w5,w5,w15+	and	w12,w10,w9+	add	v3.4s,v3.4s,v17.4s+	bic	w15,w3,w9+	eor	w11,w9,w9,ror#5+	add	w5,w5,w13+	ushr	v18.4s,v3.4s,#17+	orr	w12,w12,w15+	ushr	v19.4s,v3.4s,#10+	eor	w11,w11,w9,ror#19+	eor	w15,w5,w5,ror#11+	sli	v18.4s,v3.4s,#15+	add	w4,w4,w12+	ushr	v17.4s,v3.4s,#19+	ror	w11,w11,#6+	eor	w13,w5,w6+	eor	v19.16b,v19.16b,v18.16b+	eor	w15,w15,w5,ror#20+	add	w4,w4,w11+	sli	v17.4s,v3.4s,#13+	ldr	w12,[sp,#60]+	and	w14,w14,w13+	ror	w15,w15,#2+	ld1	{v4.4s},[x16], #16+	add	w8,w8,w4+	eor	v19.16b,v19.16b,v17.16b+	eor	w14,w14,w6+	eor	v17.16b,v17.16b,v17.16b+	add	w3,w3,w12+	add	w4,w4,w15+	and	w12,w9,w8+	mov	v17.d[1],v19.d[0]+	bic	w15,w10,w8+	eor	w11,w8,w8,ror#5+	add	w4,w4,w14+	add	v3.4s,v3.4s,v17.4s+	orr	w12,w12,w15+	eor	w11,w11,w8,ror#19+	eor	w15,w4,w4,ror#11+	add	v4.4s,v4.4s,v3.4s+	add	w3,w3,w12+	ror	w11,w11,#6+	eor	w14,w4,w5+	eor	w15,w15,w4,ror#20+	add	w3,w3,w11+	ldr	w12,[x16]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w7,w7,w3+	eor	w13,w13,w5+	st1	{v4.4s},[x17], #16+	cmp	w12,#0				// check for K256 terminator+	ldr	w12,[sp,#0]+	sub	x17,x17,#64+	bne	.L_00_48++	sub	x16,x16,#256+	cmp	x1,x2+	mov	x17, #-64+	csel	x17, x17, xzr, eq+	add	x1,x1,x17+	mov	x17,sp+	add	w10,w10,w12+	add	w3,w3,w15+	and	w12,w8,w7+	ld1	{v0.16b},[x1],#16+	bic	w15,w9,w7+	eor	w11,w7,w7,ror#5+	ld1	{v4.4s},[x16],#16+	add	w3,w3,w13+	orr	w12,w12,w15+	eor	w11,w11,w7,ror#19+	eor	w15,w3,w3,ror#11+	rev32	v0.16b,v0.16b+	add	w10,w10,w12+	ror	w11,w11,#6+	eor	w13,w3,w4+	eor	w15,w15,w3,ror#20+	add	v4.4s,v4.4s,v0.4s+	add	w10,w10,w11+	ldr	w12,[sp,#4]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w6,w6,w10+	eor	w14,w14,w4+	add	w9,w9,w12+	add	w10,w10,w15+	and	w12,w7,w6+	bic	w15,w8,w6+	eor	w11,w6,w6,ror#5+	add	w10,w10,w14+	orr	w12,w12,w15+	eor	w11,w11,w6,ror#19+	eor	w15,w10,w10,ror#11+	add	w9,w9,w12+	ror	w11,w11,#6+	eor	w14,w10,w3+	eor	w15,w15,w10,ror#20+	add	w9,w9,w11+	ldr	w12,[sp,#8]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w5,w5,w9+	eor	w13,w13,w3+	add	w8,w8,w12+	add	w9,w9,w15+	and	w12,w6,w5+	bic	w15,w7,w5+	eor	w11,w5,w5,ror#5+	add	w9,w9,w13+	orr	w12,w12,w15+	eor	w11,w11,w5,ror#19+	eor	w15,w9,w9,ror#11+	add	w8,w8,w12+	ror	w11,w11,#6+	eor	w13,w9,w10+	eor	w15,w15,w9,ror#20+	add	w8,w8,w11+	ldr	w12,[sp,#12]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w4,w4,w8+	eor	w14,w14,w10+	add	w7,w7,w12+	add	w8,w8,w15+	and	w12,w5,w4+	bic	w15,w6,w4+	eor	w11,w4,w4,ror#5+	add	w8,w8,w14+	orr	w12,w12,w15+	eor	w11,w11,w4,ror#19+	eor	w15,w8,w8,ror#11+	add	w7,w7,w12+	ror	w11,w11,#6+	eor	w14,w8,w9+	eor	w15,w15,w8,ror#20+	add	w7,w7,w11+	ldr	w12,[sp,#16]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w3,w3,w7+	eor	w13,w13,w9+	st1	{v4.4s},[x17], #16+	add	w6,w6,w12+	add	w7,w7,w15+	and	w12,w4,w3+	ld1	{v1.16b},[x1],#16+	bic	w15,w5,w3+	eor	w11,w3,w3,ror#5+	ld1	{v4.4s},[x16],#16+	add	w7,w7,w13+	orr	w12,w12,w15+	eor	w11,w11,w3,ror#19+	eor	w15,w7,w7,ror#11+	rev32	v1.16b,v1.16b+	add	w6,w6,w12+	ror	w11,w11,#6+	eor	w13,w7,w8+	eor	w15,w15,w7,ror#20+	add	v4.4s,v4.4s,v1.4s+	add	w6,w6,w11+	ldr	w12,[sp,#20]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w10,w10,w6+	eor	w14,w14,w8+	add	w5,w5,w12+	add	w6,w6,w15+	and	w12,w3,w10+	bic	w15,w4,w10+	eor	w11,w10,w10,ror#5+	add	w6,w6,w14+	orr	w12,w12,w15+	eor	w11,w11,w10,ror#19+	eor	w15,w6,w6,ror#11+	add	w5,w5,w12+	ror	w11,w11,#6+	eor	w14,w6,w7+	eor	w15,w15,w6,ror#20+	add	w5,w5,w11+	ldr	w12,[sp,#24]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w9,w9,w5+	eor	w13,w13,w7+	add	w4,w4,w12+	add	w5,w5,w15+	and	w12,w10,w9+	bic	w15,w3,w9+	eor	w11,w9,w9,ror#5+	add	w5,w5,w13+	orr	w12,w12,w15+	eor	w11,w11,w9,ror#19+	eor	w15,w5,w5,ror#11+	add	w4,w4,w12+	ror	w11,w11,#6+	eor	w13,w5,w6+	eor	w15,w15,w5,ror#20+	add	w4,w4,w11+	ldr	w12,[sp,#28]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w8,w8,w4+	eor	w14,w14,w6+	add	w3,w3,w12+	add	w4,w4,w15+	and	w12,w9,w8+	bic	w15,w10,w8+	eor	w11,w8,w8,ror#5+	add	w4,w4,w14+	orr	w12,w12,w15+	eor	w11,w11,w8,ror#19+	eor	w15,w4,w4,ror#11+	add	w3,w3,w12+	ror	w11,w11,#6+	eor	w14,w4,w5+	eor	w15,w15,w4,ror#20+	add	w3,w3,w11+	ldr	w12,[sp,#32]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w7,w7,w3+	eor	w13,w13,w5+	st1	{v4.4s},[x17], #16+	add	w10,w10,w12+	add	w3,w3,w15+	and	w12,w8,w7+	ld1	{v2.16b},[x1],#16+	bic	w15,w9,w7+	eor	w11,w7,w7,ror#5+	ld1	{v4.4s},[x16],#16+	add	w3,w3,w13+	orr	w12,w12,w15+	eor	w11,w11,w7,ror#19+	eor	w15,w3,w3,ror#11+	rev32	v2.16b,v2.16b+	add	w10,w10,w12+	ror	w11,w11,#6+	eor	w13,w3,w4+	eor	w15,w15,w3,ror#20+	add	v4.4s,v4.4s,v2.4s+	add	w10,w10,w11+	ldr	w12,[sp,#36]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w6,w6,w10+	eor	w14,w14,w4+	add	w9,w9,w12+	add	w10,w10,w15+	and	w12,w7,w6+	bic	w15,w8,w6+	eor	w11,w6,w6,ror#5+	add	w10,w10,w14+	orr	w12,w12,w15+	eor	w11,w11,w6,ror#19+	eor	w15,w10,w10,ror#11+	add	w9,w9,w12+	ror	w11,w11,#6+	eor	w14,w10,w3+	eor	w15,w15,w10,ror#20+	add	w9,w9,w11+	ldr	w12,[sp,#40]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w5,w5,w9+	eor	w13,w13,w3+	add	w8,w8,w12+	add	w9,w9,w15+	and	w12,w6,w5+	bic	w15,w7,w5+	eor	w11,w5,w5,ror#5+	add	w9,w9,w13+	orr	w12,w12,w15+	eor	w11,w11,w5,ror#19+	eor	w15,w9,w9,ror#11+	add	w8,w8,w12+	ror	w11,w11,#6+	eor	w13,w9,w10+	eor	w15,w15,w9,ror#20+	add	w8,w8,w11+	ldr	w12,[sp,#44]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w4,w4,w8+	eor	w14,w14,w10+	add	w7,w7,w12+	add	w8,w8,w15+	and	w12,w5,w4+	bic	w15,w6,w4+	eor	w11,w4,w4,ror#5+	add	w8,w8,w14+	orr	w12,w12,w15+	eor	w11,w11,w4,ror#19+	eor	w15,w8,w8,ror#11+	add	w7,w7,w12+	ror	w11,w11,#6+	eor	w14,w8,w9+	eor	w15,w15,w8,ror#20+	add	w7,w7,w11+	ldr	w12,[sp,#48]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w3,w3,w7+	eor	w13,w13,w9+	st1	{v4.4s},[x17], #16+	add	w6,w6,w12+	add	w7,w7,w15+	and	w12,w4,w3+	ld1	{v3.16b},[x1],#16+	bic	w15,w5,w3+	eor	w11,w3,w3,ror#5+	ld1	{v4.4s},[x16],#16+	add	w7,w7,w13+	orr	w12,w12,w15+	eor	w11,w11,w3,ror#19+	eor	w15,w7,w7,ror#11+	rev32	v3.16b,v3.16b+	add	w6,w6,w12+	ror	w11,w11,#6+	eor	w13,w7,w8+	eor	w15,w15,w7,ror#20+	add	v4.4s,v4.4s,v3.4s+	add	w6,w6,w11+	ldr	w12,[sp,#52]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w10,w10,w6+	eor	w14,w14,w8+	add	w5,w5,w12+	add	w6,w6,w15+	and	w12,w3,w10+	bic	w15,w4,w10+	eor	w11,w10,w10,ror#5+	add	w6,w6,w14+	orr	w12,w12,w15+	eor	w11,w11,w10,ror#19+	eor	w15,w6,w6,ror#11+	add	w5,w5,w12+	ror	w11,w11,#6+	eor	w14,w6,w7+	eor	w15,w15,w6,ror#20+	add	w5,w5,w11+	ldr	w12,[sp,#56]+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w9,w9,w5+	eor	w13,w13,w7+	add	w4,w4,w12+	add	w5,w5,w15+	and	w12,w10,w9+	bic	w15,w3,w9+	eor	w11,w9,w9,ror#5+	add	w5,w5,w13+	orr	w12,w12,w15+	eor	w11,w11,w9,ror#19+	eor	w15,w5,w5,ror#11+	add	w4,w4,w12+	ror	w11,w11,#6+	eor	w13,w5,w6+	eor	w15,w15,w5,ror#20+	add	w4,w4,w11+	ldr	w12,[sp,#60]+	and	w14,w14,w13+	ror	w15,w15,#2+	add	w8,w8,w4+	eor	w14,w14,w6+	add	w3,w3,w12+	add	w4,w4,w15+	and	w12,w9,w8+	bic	w15,w10,w8+	eor	w11,w8,w8,ror#5+	add	w4,w4,w14+	orr	w12,w12,w15+	eor	w11,w11,w8,ror#19+	eor	w15,w4,w4,ror#11+	add	w3,w3,w12+	ror	w11,w11,#6+	eor	w14,w4,w5+	eor	w15,w15,w4,ror#20+	add	w3,w3,w11+	and	w13,w13,w14+	ror	w15,w15,#2+	add	w7,w7,w3+	eor	w13,w13,w5+	st1	{v4.4s},[x17], #16+	add	w3,w3,w15			// h+=Sigma0(a) from the past+	ldp	w11,w12,[x0,#0]+	add	w3,w3,w13			// h+=Maj(a,b,c) from the past+	ldp	w13,w14,[x0,#8]+	add	w3,w3,w11			// accumulate+	add	w4,w4,w12+	ldp	w11,w12,[x0,#16]+	add	w5,w5,w13+	add	w6,w6,w14+	ldp	w13,w14,[x0,#24]+	add	w7,w7,w11+	add	w8,w8,w12+	ldr	w12,[sp,#0]+	stp	w3,w4,[x0,#0]+	add	w9,w9,w13+	mov	w13,wzr+	stp	w5,w6,[x0,#8]+	add	w10,w10,w14+	stp	w7,w8,[x0,#16]+	eor	w14,w4,w5+	stp	w9,w10,[x0,#24]+	mov	w15,wzr+	mov	x17,sp+	b.ne	.L_00_48++	ldr	x29,[x29]+	add	sp,sp,#16*4+2*__SIZEOF_POINTER__+	ret+.size	crypton_sha256_asm_block_neon,.-crypton_sha256_asm_block_neon+#if !defined(__KERNEL__) && !defined(_WIN64)+.comm	crypton_armcap_P,4,4+.hidden	crypton_armcap_P+#endif++.section	.note.GNU-stack,"",%progbits
+ cbits/asm/sha256-x86_64-elf.S view
@@ -0,0 +1,5463 @@+.text	+++.globl	crypton_sha256_asm_block_data_order+.type	crypton_sha256_asm_block_data_order,@function+.align	16+crypton_sha256_asm_block_data_order:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+	leaq	crypton_ia32cap_P(%rip),%rax+	movl	0(%rax),%r9d+	movl	4(%rax),%r10d+	movl	8(%rax),%eax+	testl	$536870912,%eax+	jnz	.Lshaext_shortcut+	andl	$296,%eax+	cmpl	$296,%eax+	je	.Lavx2_shortcut+	andl	$1073741824,%r9d+	andl	$268435968,%r10d+	orl	%r9d,%r10d+	cmpl	$1342177792,%r10d+	je	.Lavx_shortcut+	testl	$512,%r10d+	jnz	.Lssse3_shortcut+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$64+24,%rsp++.cfi_def_cfa	%rsp,144++	leaq	(%rsi,%rdx,4),%rdx+	movq	%rdi,64+0(%rsp)+	movq	%rsi,64+8(%rsp)+	movq	%rdx,64+16(%rsp)++	movl	0(%rdi),%eax+	movl	4(%rdi),%ebx+	movl	8(%rdi),%ecx+	movl	12(%rdi),%edx+	movl	16(%rdi),%r8d+	movl	20(%rdi),%r9d+	movl	24(%rdi),%r10d+	movl	28(%rdi),%r11d+	jmp	.Lloop++.align	16+.Lloop:+	movl	%ebx,%edi+	leaq	K256(%rip),%rbp+	xorl	%ecx,%edi+	movl	0(%rsi),%r12d+	movl	%r8d,%r13d+	movl	%eax,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r9d,%r15d++	xorl	%r8d,%r13d+	rorl	$9,%r14d+	xorl	%r10d,%r15d++	movl	%r12d,0(%rsp)+	xorl	%eax,%r14d+	andl	%r8d,%r15d++	rorl	$5,%r13d+	addl	%r11d,%r12d+	xorl	%r10d,%r15d++	rorl	$11,%r14d+	xorl	%r8d,%r13d+	addl	%r15d,%r12d++	movl	%eax,%r15d+	addl	(%rbp),%r12d+	xorl	%eax,%r14d++	xorl	%ebx,%r15d+	rorl	$6,%r13d+	movl	%ebx,%r11d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r11d+	addl	%r12d,%edx+	addl	%r12d,%r11d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r11d+	movl	4(%rsi),%r12d+	movl	%edx,%r13d+	movl	%r11d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r8d,%edi++	xorl	%edx,%r13d+	rorl	$9,%r14d+	xorl	%r9d,%edi++	movl	%r12d,4(%rsp)+	xorl	%r11d,%r14d+	andl	%edx,%edi++	rorl	$5,%r13d+	addl	%r10d,%r12d+	xorl	%r9d,%edi++	rorl	$11,%r14d+	xorl	%edx,%r13d+	addl	%edi,%r12d++	movl	%r11d,%edi+	addl	(%rbp),%r12d+	xorl	%r11d,%r14d++	xorl	%eax,%edi+	rorl	$6,%r13d+	movl	%eax,%r10d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r10d+	addl	%r12d,%ecx+	addl	%r12d,%r10d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r10d+	movl	8(%rsi),%r12d+	movl	%ecx,%r13d+	movl	%r10d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%edx,%r15d++	xorl	%ecx,%r13d+	rorl	$9,%r14d+	xorl	%r8d,%r15d++	movl	%r12d,8(%rsp)+	xorl	%r10d,%r14d+	andl	%ecx,%r15d++	rorl	$5,%r13d+	addl	%r9d,%r12d+	xorl	%r8d,%r15d++	rorl	$11,%r14d+	xorl	%ecx,%r13d+	addl	%r15d,%r12d++	movl	%r10d,%r15d+	addl	(%rbp),%r12d+	xorl	%r10d,%r14d++	xorl	%r11d,%r15d+	rorl	$6,%r13d+	movl	%r11d,%r9d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r9d+	addl	%r12d,%ebx+	addl	%r12d,%r9d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r9d+	movl	12(%rsi),%r12d+	movl	%ebx,%r13d+	movl	%r9d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%ecx,%edi++	xorl	%ebx,%r13d+	rorl	$9,%r14d+	xorl	%edx,%edi++	movl	%r12d,12(%rsp)+	xorl	%r9d,%r14d+	andl	%ebx,%edi++	rorl	$5,%r13d+	addl	%r8d,%r12d+	xorl	%edx,%edi++	rorl	$11,%r14d+	xorl	%ebx,%r13d+	addl	%edi,%r12d++	movl	%r9d,%edi+	addl	(%rbp),%r12d+	xorl	%r9d,%r14d++	xorl	%r10d,%edi+	rorl	$6,%r13d+	movl	%r10d,%r8d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r8d+	addl	%r12d,%eax+	addl	%r12d,%r8d++	leaq	20(%rbp),%rbp+	addl	%r14d,%r8d+	movl	16(%rsi),%r12d+	movl	%eax,%r13d+	movl	%r8d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%ebx,%r15d++	xorl	%eax,%r13d+	rorl	$9,%r14d+	xorl	%ecx,%r15d++	movl	%r12d,16(%rsp)+	xorl	%r8d,%r14d+	andl	%eax,%r15d++	rorl	$5,%r13d+	addl	%edx,%r12d+	xorl	%ecx,%r15d++	rorl	$11,%r14d+	xorl	%eax,%r13d+	addl	%r15d,%r12d++	movl	%r8d,%r15d+	addl	(%rbp),%r12d+	xorl	%r8d,%r14d++	xorl	%r9d,%r15d+	rorl	$6,%r13d+	movl	%r9d,%edx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%edx+	addl	%r12d,%r11d+	addl	%r12d,%edx++	leaq	4(%rbp),%rbp+	addl	%r14d,%edx+	movl	20(%rsi),%r12d+	movl	%r11d,%r13d+	movl	%edx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%eax,%edi++	xorl	%r11d,%r13d+	rorl	$9,%r14d+	xorl	%ebx,%edi++	movl	%r12d,20(%rsp)+	xorl	%edx,%r14d+	andl	%r11d,%edi++	rorl	$5,%r13d+	addl	%ecx,%r12d+	xorl	%ebx,%edi++	rorl	$11,%r14d+	xorl	%r11d,%r13d+	addl	%edi,%r12d++	movl	%edx,%edi+	addl	(%rbp),%r12d+	xorl	%edx,%r14d++	xorl	%r8d,%edi+	rorl	$6,%r13d+	movl	%r8d,%ecx++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%ecx+	addl	%r12d,%r10d+	addl	%r12d,%ecx++	leaq	4(%rbp),%rbp+	addl	%r14d,%ecx+	movl	24(%rsi),%r12d+	movl	%r10d,%r13d+	movl	%ecx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r11d,%r15d++	xorl	%r10d,%r13d+	rorl	$9,%r14d+	xorl	%eax,%r15d++	movl	%r12d,24(%rsp)+	xorl	%ecx,%r14d+	andl	%r10d,%r15d++	rorl	$5,%r13d+	addl	%ebx,%r12d+	xorl	%eax,%r15d++	rorl	$11,%r14d+	xorl	%r10d,%r13d+	addl	%r15d,%r12d++	movl	%ecx,%r15d+	addl	(%rbp),%r12d+	xorl	%ecx,%r14d++	xorl	%edx,%r15d+	rorl	$6,%r13d+	movl	%edx,%ebx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%ebx+	addl	%r12d,%r9d+	addl	%r12d,%ebx++	leaq	4(%rbp),%rbp+	addl	%r14d,%ebx+	movl	28(%rsi),%r12d+	movl	%r9d,%r13d+	movl	%ebx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r10d,%edi++	xorl	%r9d,%r13d+	rorl	$9,%r14d+	xorl	%r11d,%edi++	movl	%r12d,28(%rsp)+	xorl	%ebx,%r14d+	andl	%r9d,%edi++	rorl	$5,%r13d+	addl	%eax,%r12d+	xorl	%r11d,%edi++	rorl	$11,%r14d+	xorl	%r9d,%r13d+	addl	%edi,%r12d++	movl	%ebx,%edi+	addl	(%rbp),%r12d+	xorl	%ebx,%r14d++	xorl	%ecx,%edi+	rorl	$6,%r13d+	movl	%ecx,%eax++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%eax+	addl	%r12d,%r8d+	addl	%r12d,%eax++	leaq	20(%rbp),%rbp+	addl	%r14d,%eax+	movl	32(%rsi),%r12d+	movl	%r8d,%r13d+	movl	%eax,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r9d,%r15d++	xorl	%r8d,%r13d+	rorl	$9,%r14d+	xorl	%r10d,%r15d++	movl	%r12d,32(%rsp)+	xorl	%eax,%r14d+	andl	%r8d,%r15d++	rorl	$5,%r13d+	addl	%r11d,%r12d+	xorl	%r10d,%r15d++	rorl	$11,%r14d+	xorl	%r8d,%r13d+	addl	%r15d,%r12d++	movl	%eax,%r15d+	addl	(%rbp),%r12d+	xorl	%eax,%r14d++	xorl	%ebx,%r15d+	rorl	$6,%r13d+	movl	%ebx,%r11d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r11d+	addl	%r12d,%edx+	addl	%r12d,%r11d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r11d+	movl	36(%rsi),%r12d+	movl	%edx,%r13d+	movl	%r11d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r8d,%edi++	xorl	%edx,%r13d+	rorl	$9,%r14d+	xorl	%r9d,%edi++	movl	%r12d,36(%rsp)+	xorl	%r11d,%r14d+	andl	%edx,%edi++	rorl	$5,%r13d+	addl	%r10d,%r12d+	xorl	%r9d,%edi++	rorl	$11,%r14d+	xorl	%edx,%r13d+	addl	%edi,%r12d++	movl	%r11d,%edi+	addl	(%rbp),%r12d+	xorl	%r11d,%r14d++	xorl	%eax,%edi+	rorl	$6,%r13d+	movl	%eax,%r10d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r10d+	addl	%r12d,%ecx+	addl	%r12d,%r10d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r10d+	movl	40(%rsi),%r12d+	movl	%ecx,%r13d+	movl	%r10d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%edx,%r15d++	xorl	%ecx,%r13d+	rorl	$9,%r14d+	xorl	%r8d,%r15d++	movl	%r12d,40(%rsp)+	xorl	%r10d,%r14d+	andl	%ecx,%r15d++	rorl	$5,%r13d+	addl	%r9d,%r12d+	xorl	%r8d,%r15d++	rorl	$11,%r14d+	xorl	%ecx,%r13d+	addl	%r15d,%r12d++	movl	%r10d,%r15d+	addl	(%rbp),%r12d+	xorl	%r10d,%r14d++	xorl	%r11d,%r15d+	rorl	$6,%r13d+	movl	%r11d,%r9d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r9d+	addl	%r12d,%ebx+	addl	%r12d,%r9d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r9d+	movl	44(%rsi),%r12d+	movl	%ebx,%r13d+	movl	%r9d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%ecx,%edi++	xorl	%ebx,%r13d+	rorl	$9,%r14d+	xorl	%edx,%edi++	movl	%r12d,44(%rsp)+	xorl	%r9d,%r14d+	andl	%ebx,%edi++	rorl	$5,%r13d+	addl	%r8d,%r12d+	xorl	%edx,%edi++	rorl	$11,%r14d+	xorl	%ebx,%r13d+	addl	%edi,%r12d++	movl	%r9d,%edi+	addl	(%rbp),%r12d+	xorl	%r9d,%r14d++	xorl	%r10d,%edi+	rorl	$6,%r13d+	movl	%r10d,%r8d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r8d+	addl	%r12d,%eax+	addl	%r12d,%r8d++	leaq	20(%rbp),%rbp+	addl	%r14d,%r8d+	movl	48(%rsi),%r12d+	movl	%eax,%r13d+	movl	%r8d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%ebx,%r15d++	xorl	%eax,%r13d+	rorl	$9,%r14d+	xorl	%ecx,%r15d++	movl	%r12d,48(%rsp)+	xorl	%r8d,%r14d+	andl	%eax,%r15d++	rorl	$5,%r13d+	addl	%edx,%r12d+	xorl	%ecx,%r15d++	rorl	$11,%r14d+	xorl	%eax,%r13d+	addl	%r15d,%r12d++	movl	%r8d,%r15d+	addl	(%rbp),%r12d+	xorl	%r8d,%r14d++	xorl	%r9d,%r15d+	rorl	$6,%r13d+	movl	%r9d,%edx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%edx+	addl	%r12d,%r11d+	addl	%r12d,%edx++	leaq	4(%rbp),%rbp+	addl	%r14d,%edx+	movl	52(%rsi),%r12d+	movl	%r11d,%r13d+	movl	%edx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%eax,%edi++	xorl	%r11d,%r13d+	rorl	$9,%r14d+	xorl	%ebx,%edi++	movl	%r12d,52(%rsp)+	xorl	%edx,%r14d+	andl	%r11d,%edi++	rorl	$5,%r13d+	addl	%ecx,%r12d+	xorl	%ebx,%edi++	rorl	$11,%r14d+	xorl	%r11d,%r13d+	addl	%edi,%r12d++	movl	%edx,%edi+	addl	(%rbp),%r12d+	xorl	%edx,%r14d++	xorl	%r8d,%edi+	rorl	$6,%r13d+	movl	%r8d,%ecx++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%ecx+	addl	%r12d,%r10d+	addl	%r12d,%ecx++	leaq	4(%rbp),%rbp+	addl	%r14d,%ecx+	movl	56(%rsi),%r12d+	movl	%r10d,%r13d+	movl	%ecx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r11d,%r15d++	xorl	%r10d,%r13d+	rorl	$9,%r14d+	xorl	%eax,%r15d++	movl	%r12d,56(%rsp)+	xorl	%ecx,%r14d+	andl	%r10d,%r15d++	rorl	$5,%r13d+	addl	%ebx,%r12d+	xorl	%eax,%r15d++	rorl	$11,%r14d+	xorl	%r10d,%r13d+	addl	%r15d,%r12d++	movl	%ecx,%r15d+	addl	(%rbp),%r12d+	xorl	%ecx,%r14d++	xorl	%edx,%r15d+	rorl	$6,%r13d+	movl	%edx,%ebx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%ebx+	addl	%r12d,%r9d+	addl	%r12d,%ebx++	leaq	4(%rbp),%rbp+	addl	%r14d,%ebx+	movl	60(%rsi),%r12d+	movl	%r9d,%r13d+	movl	%ebx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r10d,%edi++	xorl	%r9d,%r13d+	rorl	$9,%r14d+	xorl	%r11d,%edi++	movl	%r12d,60(%rsp)+	xorl	%ebx,%r14d+	andl	%r9d,%edi++	rorl	$5,%r13d+	addl	%eax,%r12d+	xorl	%r11d,%edi++	rorl	$11,%r14d+	xorl	%r9d,%r13d+	addl	%edi,%r12d++	movl	%ebx,%edi+	addl	(%rbp),%r12d+	xorl	%ebx,%r14d++	xorl	%ecx,%edi+	rorl	$6,%r13d+	movl	%ecx,%eax++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%eax+	addl	%r12d,%r8d+	addl	%r12d,%eax++	leaq	20(%rbp),%rbp+	jmp	.Lrounds_16_xx+.align	16+.Lrounds_16_xx:+	movl	4(%rsp),%r13d+	movl	56(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%eax+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	36(%rsp),%r12d++	addl	0(%rsp),%r12d+	movl	%r8d,%r13d+	addl	%r15d,%r12d+	movl	%eax,%r14d+	rorl	$14,%r13d+	movl	%r9d,%r15d++	xorl	%r8d,%r13d+	rorl	$9,%r14d+	xorl	%r10d,%r15d++	movl	%r12d,0(%rsp)+	xorl	%eax,%r14d+	andl	%r8d,%r15d++	rorl	$5,%r13d+	addl	%r11d,%r12d+	xorl	%r10d,%r15d++	rorl	$11,%r14d+	xorl	%r8d,%r13d+	addl	%r15d,%r12d++	movl	%eax,%r15d+	addl	(%rbp),%r12d+	xorl	%eax,%r14d++	xorl	%ebx,%r15d+	rorl	$6,%r13d+	movl	%ebx,%r11d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r11d+	addl	%r12d,%edx+	addl	%r12d,%r11d++	leaq	4(%rbp),%rbp+	movl	8(%rsp),%r13d+	movl	60(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r11d+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	40(%rsp),%r12d++	addl	4(%rsp),%r12d+	movl	%edx,%r13d+	addl	%edi,%r12d+	movl	%r11d,%r14d+	rorl	$14,%r13d+	movl	%r8d,%edi++	xorl	%edx,%r13d+	rorl	$9,%r14d+	xorl	%r9d,%edi++	movl	%r12d,4(%rsp)+	xorl	%r11d,%r14d+	andl	%edx,%edi++	rorl	$5,%r13d+	addl	%r10d,%r12d+	xorl	%r9d,%edi++	rorl	$11,%r14d+	xorl	%edx,%r13d+	addl	%edi,%r12d++	movl	%r11d,%edi+	addl	(%rbp),%r12d+	xorl	%r11d,%r14d++	xorl	%eax,%edi+	rorl	$6,%r13d+	movl	%eax,%r10d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r10d+	addl	%r12d,%ecx+	addl	%r12d,%r10d++	leaq	4(%rbp),%rbp+	movl	12(%rsp),%r13d+	movl	0(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r10d+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	44(%rsp),%r12d++	addl	8(%rsp),%r12d+	movl	%ecx,%r13d+	addl	%r15d,%r12d+	movl	%r10d,%r14d+	rorl	$14,%r13d+	movl	%edx,%r15d++	xorl	%ecx,%r13d+	rorl	$9,%r14d+	xorl	%r8d,%r15d++	movl	%r12d,8(%rsp)+	xorl	%r10d,%r14d+	andl	%ecx,%r15d++	rorl	$5,%r13d+	addl	%r9d,%r12d+	xorl	%r8d,%r15d++	rorl	$11,%r14d+	xorl	%ecx,%r13d+	addl	%r15d,%r12d++	movl	%r10d,%r15d+	addl	(%rbp),%r12d+	xorl	%r10d,%r14d++	xorl	%r11d,%r15d+	rorl	$6,%r13d+	movl	%r11d,%r9d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r9d+	addl	%r12d,%ebx+	addl	%r12d,%r9d++	leaq	4(%rbp),%rbp+	movl	16(%rsp),%r13d+	movl	4(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r9d+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	48(%rsp),%r12d++	addl	12(%rsp),%r12d+	movl	%ebx,%r13d+	addl	%edi,%r12d+	movl	%r9d,%r14d+	rorl	$14,%r13d+	movl	%ecx,%edi++	xorl	%ebx,%r13d+	rorl	$9,%r14d+	xorl	%edx,%edi++	movl	%r12d,12(%rsp)+	xorl	%r9d,%r14d+	andl	%ebx,%edi++	rorl	$5,%r13d+	addl	%r8d,%r12d+	xorl	%edx,%edi++	rorl	$11,%r14d+	xorl	%ebx,%r13d+	addl	%edi,%r12d++	movl	%r9d,%edi+	addl	(%rbp),%r12d+	xorl	%r9d,%r14d++	xorl	%r10d,%edi+	rorl	$6,%r13d+	movl	%r10d,%r8d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r8d+	addl	%r12d,%eax+	addl	%r12d,%r8d++	leaq	20(%rbp),%rbp+	movl	20(%rsp),%r13d+	movl	8(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r8d+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	52(%rsp),%r12d++	addl	16(%rsp),%r12d+	movl	%eax,%r13d+	addl	%r15d,%r12d+	movl	%r8d,%r14d+	rorl	$14,%r13d+	movl	%ebx,%r15d++	xorl	%eax,%r13d+	rorl	$9,%r14d+	xorl	%ecx,%r15d++	movl	%r12d,16(%rsp)+	xorl	%r8d,%r14d+	andl	%eax,%r15d++	rorl	$5,%r13d+	addl	%edx,%r12d+	xorl	%ecx,%r15d++	rorl	$11,%r14d+	xorl	%eax,%r13d+	addl	%r15d,%r12d++	movl	%r8d,%r15d+	addl	(%rbp),%r12d+	xorl	%r8d,%r14d++	xorl	%r9d,%r15d+	rorl	$6,%r13d+	movl	%r9d,%edx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%edx+	addl	%r12d,%r11d+	addl	%r12d,%edx++	leaq	4(%rbp),%rbp+	movl	24(%rsp),%r13d+	movl	12(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%edx+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	56(%rsp),%r12d++	addl	20(%rsp),%r12d+	movl	%r11d,%r13d+	addl	%edi,%r12d+	movl	%edx,%r14d+	rorl	$14,%r13d+	movl	%eax,%edi++	xorl	%r11d,%r13d+	rorl	$9,%r14d+	xorl	%ebx,%edi++	movl	%r12d,20(%rsp)+	xorl	%edx,%r14d+	andl	%r11d,%edi++	rorl	$5,%r13d+	addl	%ecx,%r12d+	xorl	%ebx,%edi++	rorl	$11,%r14d+	xorl	%r11d,%r13d+	addl	%edi,%r12d++	movl	%edx,%edi+	addl	(%rbp),%r12d+	xorl	%edx,%r14d++	xorl	%r8d,%edi+	rorl	$6,%r13d+	movl	%r8d,%ecx++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%ecx+	addl	%r12d,%r10d+	addl	%r12d,%ecx++	leaq	4(%rbp),%rbp+	movl	28(%rsp),%r13d+	movl	16(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%ecx+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	60(%rsp),%r12d++	addl	24(%rsp),%r12d+	movl	%r10d,%r13d+	addl	%r15d,%r12d+	movl	%ecx,%r14d+	rorl	$14,%r13d+	movl	%r11d,%r15d++	xorl	%r10d,%r13d+	rorl	$9,%r14d+	xorl	%eax,%r15d++	movl	%r12d,24(%rsp)+	xorl	%ecx,%r14d+	andl	%r10d,%r15d++	rorl	$5,%r13d+	addl	%ebx,%r12d+	xorl	%eax,%r15d++	rorl	$11,%r14d+	xorl	%r10d,%r13d+	addl	%r15d,%r12d++	movl	%ecx,%r15d+	addl	(%rbp),%r12d+	xorl	%ecx,%r14d++	xorl	%edx,%r15d+	rorl	$6,%r13d+	movl	%edx,%ebx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%ebx+	addl	%r12d,%r9d+	addl	%r12d,%ebx++	leaq	4(%rbp),%rbp+	movl	32(%rsp),%r13d+	movl	20(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%ebx+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	0(%rsp),%r12d++	addl	28(%rsp),%r12d+	movl	%r9d,%r13d+	addl	%edi,%r12d+	movl	%ebx,%r14d+	rorl	$14,%r13d+	movl	%r10d,%edi++	xorl	%r9d,%r13d+	rorl	$9,%r14d+	xorl	%r11d,%edi++	movl	%r12d,28(%rsp)+	xorl	%ebx,%r14d+	andl	%r9d,%edi++	rorl	$5,%r13d+	addl	%eax,%r12d+	xorl	%r11d,%edi++	rorl	$11,%r14d+	xorl	%r9d,%r13d+	addl	%edi,%r12d++	movl	%ebx,%edi+	addl	(%rbp),%r12d+	xorl	%ebx,%r14d++	xorl	%ecx,%edi+	rorl	$6,%r13d+	movl	%ecx,%eax++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%eax+	addl	%r12d,%r8d+	addl	%r12d,%eax++	leaq	20(%rbp),%rbp+	movl	36(%rsp),%r13d+	movl	24(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%eax+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	4(%rsp),%r12d++	addl	32(%rsp),%r12d+	movl	%r8d,%r13d+	addl	%r15d,%r12d+	movl	%eax,%r14d+	rorl	$14,%r13d+	movl	%r9d,%r15d++	xorl	%r8d,%r13d+	rorl	$9,%r14d+	xorl	%r10d,%r15d++	movl	%r12d,32(%rsp)+	xorl	%eax,%r14d+	andl	%r8d,%r15d++	rorl	$5,%r13d+	addl	%r11d,%r12d+	xorl	%r10d,%r15d++	rorl	$11,%r14d+	xorl	%r8d,%r13d+	addl	%r15d,%r12d++	movl	%eax,%r15d+	addl	(%rbp),%r12d+	xorl	%eax,%r14d++	xorl	%ebx,%r15d+	rorl	$6,%r13d+	movl	%ebx,%r11d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r11d+	addl	%r12d,%edx+	addl	%r12d,%r11d++	leaq	4(%rbp),%rbp+	movl	40(%rsp),%r13d+	movl	28(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r11d+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	8(%rsp),%r12d++	addl	36(%rsp),%r12d+	movl	%edx,%r13d+	addl	%edi,%r12d+	movl	%r11d,%r14d+	rorl	$14,%r13d+	movl	%r8d,%edi++	xorl	%edx,%r13d+	rorl	$9,%r14d+	xorl	%r9d,%edi++	movl	%r12d,36(%rsp)+	xorl	%r11d,%r14d+	andl	%edx,%edi++	rorl	$5,%r13d+	addl	%r10d,%r12d+	xorl	%r9d,%edi++	rorl	$11,%r14d+	xorl	%edx,%r13d+	addl	%edi,%r12d++	movl	%r11d,%edi+	addl	(%rbp),%r12d+	xorl	%r11d,%r14d++	xorl	%eax,%edi+	rorl	$6,%r13d+	movl	%eax,%r10d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r10d+	addl	%r12d,%ecx+	addl	%r12d,%r10d++	leaq	4(%rbp),%rbp+	movl	44(%rsp),%r13d+	movl	32(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r10d+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	12(%rsp),%r12d++	addl	40(%rsp),%r12d+	movl	%ecx,%r13d+	addl	%r15d,%r12d+	movl	%r10d,%r14d+	rorl	$14,%r13d+	movl	%edx,%r15d++	xorl	%ecx,%r13d+	rorl	$9,%r14d+	xorl	%r8d,%r15d++	movl	%r12d,40(%rsp)+	xorl	%r10d,%r14d+	andl	%ecx,%r15d++	rorl	$5,%r13d+	addl	%r9d,%r12d+	xorl	%r8d,%r15d++	rorl	$11,%r14d+	xorl	%ecx,%r13d+	addl	%r15d,%r12d++	movl	%r10d,%r15d+	addl	(%rbp),%r12d+	xorl	%r10d,%r14d++	xorl	%r11d,%r15d+	rorl	$6,%r13d+	movl	%r11d,%r9d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r9d+	addl	%r12d,%ebx+	addl	%r12d,%r9d++	leaq	4(%rbp),%rbp+	movl	48(%rsp),%r13d+	movl	36(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r9d+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	16(%rsp),%r12d++	addl	44(%rsp),%r12d+	movl	%ebx,%r13d+	addl	%edi,%r12d+	movl	%r9d,%r14d+	rorl	$14,%r13d+	movl	%ecx,%edi++	xorl	%ebx,%r13d+	rorl	$9,%r14d+	xorl	%edx,%edi++	movl	%r12d,44(%rsp)+	xorl	%r9d,%r14d+	andl	%ebx,%edi++	rorl	$5,%r13d+	addl	%r8d,%r12d+	xorl	%edx,%edi++	rorl	$11,%r14d+	xorl	%ebx,%r13d+	addl	%edi,%r12d++	movl	%r9d,%edi+	addl	(%rbp),%r12d+	xorl	%r9d,%r14d++	xorl	%r10d,%edi+	rorl	$6,%r13d+	movl	%r10d,%r8d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r8d+	addl	%r12d,%eax+	addl	%r12d,%r8d++	leaq	20(%rbp),%rbp+	movl	52(%rsp),%r13d+	movl	40(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r8d+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	20(%rsp),%r12d++	addl	48(%rsp),%r12d+	movl	%eax,%r13d+	addl	%r15d,%r12d+	movl	%r8d,%r14d+	rorl	$14,%r13d+	movl	%ebx,%r15d++	xorl	%eax,%r13d+	rorl	$9,%r14d+	xorl	%ecx,%r15d++	movl	%r12d,48(%rsp)+	xorl	%r8d,%r14d+	andl	%eax,%r15d++	rorl	$5,%r13d+	addl	%edx,%r12d+	xorl	%ecx,%r15d++	rorl	$11,%r14d+	xorl	%eax,%r13d+	addl	%r15d,%r12d++	movl	%r8d,%r15d+	addl	(%rbp),%r12d+	xorl	%r8d,%r14d++	xorl	%r9d,%r15d+	rorl	$6,%r13d+	movl	%r9d,%edx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%edx+	addl	%r12d,%r11d+	addl	%r12d,%edx++	leaq	4(%rbp),%rbp+	movl	56(%rsp),%r13d+	movl	44(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%edx+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	24(%rsp),%r12d++	addl	52(%rsp),%r12d+	movl	%r11d,%r13d+	addl	%edi,%r12d+	movl	%edx,%r14d+	rorl	$14,%r13d+	movl	%eax,%edi++	xorl	%r11d,%r13d+	rorl	$9,%r14d+	xorl	%ebx,%edi++	movl	%r12d,52(%rsp)+	xorl	%edx,%r14d+	andl	%r11d,%edi++	rorl	$5,%r13d+	addl	%ecx,%r12d+	xorl	%ebx,%edi++	rorl	$11,%r14d+	xorl	%r11d,%r13d+	addl	%edi,%r12d++	movl	%edx,%edi+	addl	(%rbp),%r12d+	xorl	%edx,%r14d++	xorl	%r8d,%edi+	rorl	$6,%r13d+	movl	%r8d,%ecx++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%ecx+	addl	%r12d,%r10d+	addl	%r12d,%ecx++	leaq	4(%rbp),%rbp+	movl	60(%rsp),%r13d+	movl	48(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%ecx+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	28(%rsp),%r12d++	addl	56(%rsp),%r12d+	movl	%r10d,%r13d+	addl	%r15d,%r12d+	movl	%ecx,%r14d+	rorl	$14,%r13d+	movl	%r11d,%r15d++	xorl	%r10d,%r13d+	rorl	$9,%r14d+	xorl	%eax,%r15d++	movl	%r12d,56(%rsp)+	xorl	%ecx,%r14d+	andl	%r10d,%r15d++	rorl	$5,%r13d+	addl	%ebx,%r12d+	xorl	%eax,%r15d++	rorl	$11,%r14d+	xorl	%r10d,%r13d+	addl	%r15d,%r12d++	movl	%ecx,%r15d+	addl	(%rbp),%r12d+	xorl	%ecx,%r14d++	xorl	%edx,%r15d+	rorl	$6,%r13d+	movl	%edx,%ebx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%ebx+	addl	%r12d,%r9d+	addl	%r12d,%ebx++	leaq	4(%rbp),%rbp+	movl	0(%rsp),%r13d+	movl	52(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%ebx+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	32(%rsp),%r12d++	addl	60(%rsp),%r12d+	movl	%r9d,%r13d+	addl	%edi,%r12d+	movl	%ebx,%r14d+	rorl	$14,%r13d+	movl	%r10d,%edi++	xorl	%r9d,%r13d+	rorl	$9,%r14d+	xorl	%r11d,%edi++	movl	%r12d,60(%rsp)+	xorl	%ebx,%r14d+	andl	%r9d,%edi++	rorl	$5,%r13d+	addl	%eax,%r12d+	xorl	%r11d,%edi++	rorl	$11,%r14d+	xorl	%r9d,%r13d+	addl	%edi,%r12d++	movl	%ebx,%edi+	addl	(%rbp),%r12d+	xorl	%ebx,%r14d++	xorl	%ecx,%edi+	rorl	$6,%r13d+	movl	%ecx,%eax++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%eax+	addl	%r12d,%r8d+	addl	%r12d,%eax++	leaq	20(%rbp),%rbp+	cmpb	$0,3(%rbp)+	jnz	.Lrounds_16_xx++	movq	64+0(%rsp),%rdi+	addl	%r14d,%eax+	leaq	64(%rsi),%rsi++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	addl	24(%rdi),%r10d+	addl	28(%rdi),%r11d++	cmpq	64+16(%rsp),%rsi++	movl	%eax,0(%rdi)+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)+	jb	.Lloop++	leaq	64+24+48(%rsp),%r11+.cfi_def_cfa	%r11,8+	movq	64+24(%rsp),%r15+	movq	-40(%r11),%r14+	movq	-32(%r11),%r13+	movq	-24(%r11),%r12+	movq	-16(%r11),%rbx+	movq	-8(%r11),%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbp+.cfi_restore	%rbx+	leaq	(%r11),%rsp+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_sha256_asm_block_data_order,.-crypton_sha256_asm_block_data_order+.align	64+.type	K256,@object+K256:+.long	0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5+.long	0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5+.long	0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5+.long	0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5+.long	0xd807aa98,0x12835b01,0x243185be,0x550c7dc3+.long	0xd807aa98,0x12835b01,0x243185be,0x550c7dc3+.long	0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174+.long	0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174+.long	0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc+.long	0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc+.long	0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da+.long	0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da+.long	0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7+.long	0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7+.long	0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967+.long	0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967+.long	0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13+.long	0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13+.long	0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85+.long	0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85+.long	0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3+.long	0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3+.long	0xd192e819,0xd6990624,0xf40e3585,0x106aa070+.long	0xd192e819,0xd6990624,0xf40e3585,0x106aa070+.long	0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5+.long	0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5+.long	0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3+.long	0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3+.long	0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208+.long	0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208+.long	0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2+.long	0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2++.long	0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f+.long	0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f+.long	0x03020100,0x0b0a0908,0xffffffff,0xffffffff+.long	0x03020100,0x0b0a0908,0xffffffff,0xffffffff+.long	0xffffffff,0xffffffff,0x03020100,0x0b0a0908+.long	0xffffffff,0xffffffff,0x03020100,0x0b0a0908+.byte	83,72,65,50,53,54,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.type	crypton_sha256_asm_block_data_order_shaext,@function+.align	64+crypton_sha256_asm_block_data_order_shaext:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+.Lshaext_shortcut:++	leaq	K256+128(%rip),%rcx+	movdqu	(%rdi),%xmm1+	movdqu	16(%rdi),%xmm2+	movdqa	512-128(%rcx),%xmm7++	pshufd	$0x1b,%xmm1,%xmm0+	pshufd	$0xb1,%xmm1,%xmm1+	pshufd	$0x1b,%xmm2,%xmm2+	movdqa	%xmm7,%xmm8+.byte	102,15,58,15,202,8+	punpcklqdq	%xmm0,%xmm2+	jmp	.Loop_shaext++.align	16+.Loop_shaext:+	movdqu	(%rsi),%xmm3+	movdqu	16(%rsi),%xmm4+	movdqu	32(%rsi),%xmm5+.byte	102,15,56,0,223+	movdqu	48(%rsi),%xmm6++	movdqa	0-128(%rcx),%xmm0+	paddd	%xmm3,%xmm0+.byte	102,15,56,0,231+	movdqa	%xmm2,%xmm10+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	nop+	movdqa	%xmm1,%xmm9+.byte	15,56,203,202++	movdqa	32-128(%rcx),%xmm0+	paddd	%xmm4,%xmm0+.byte	102,15,56,0,239+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	leaq	64(%rsi),%rsi+.byte	15,56,204,220+.byte	15,56,203,202++	movdqa	64-128(%rcx),%xmm0+	paddd	%xmm5,%xmm0+.byte	102,15,56,0,247+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm6,%xmm7+.byte	102,15,58,15,253,4+	nop+	paddd	%xmm7,%xmm3+.byte	15,56,204,229+.byte	15,56,203,202++	movdqa	96-128(%rcx),%xmm0+	paddd	%xmm6,%xmm0+.byte	15,56,205,222+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm3,%xmm7+.byte	102,15,58,15,254,4+	nop+	paddd	%xmm7,%xmm4+.byte	15,56,204,238+.byte	15,56,203,202+	movdqa	128-128(%rcx),%xmm0+	paddd	%xmm3,%xmm0+.byte	15,56,205,227+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm4,%xmm7+.byte	102,15,58,15,251,4+	nop+	paddd	%xmm7,%xmm5+.byte	15,56,204,243+.byte	15,56,203,202+	movdqa	160-128(%rcx),%xmm0+	paddd	%xmm4,%xmm0+.byte	15,56,205,236+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm5,%xmm7+.byte	102,15,58,15,252,4+	nop+	paddd	%xmm7,%xmm6+.byte	15,56,204,220+.byte	15,56,203,202+	movdqa	192-128(%rcx),%xmm0+	paddd	%xmm5,%xmm0+.byte	15,56,205,245+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm6,%xmm7+.byte	102,15,58,15,253,4+	nop+	paddd	%xmm7,%xmm3+.byte	15,56,204,229+.byte	15,56,203,202+	movdqa	224-128(%rcx),%xmm0+	paddd	%xmm6,%xmm0+.byte	15,56,205,222+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm3,%xmm7+.byte	102,15,58,15,254,4+	nop+	paddd	%xmm7,%xmm4+.byte	15,56,204,238+.byte	15,56,203,202+	movdqa	256-128(%rcx),%xmm0+	paddd	%xmm3,%xmm0+.byte	15,56,205,227+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm4,%xmm7+.byte	102,15,58,15,251,4+	nop+	paddd	%xmm7,%xmm5+.byte	15,56,204,243+.byte	15,56,203,202+	movdqa	288-128(%rcx),%xmm0+	paddd	%xmm4,%xmm0+.byte	15,56,205,236+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm5,%xmm7+.byte	102,15,58,15,252,4+	nop+	paddd	%xmm7,%xmm6+.byte	15,56,204,220+.byte	15,56,203,202+	movdqa	320-128(%rcx),%xmm0+	paddd	%xmm5,%xmm0+.byte	15,56,205,245+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm6,%xmm7+.byte	102,15,58,15,253,4+	nop+	paddd	%xmm7,%xmm3+.byte	15,56,204,229+.byte	15,56,203,202+	movdqa	352-128(%rcx),%xmm0+	paddd	%xmm6,%xmm0+.byte	15,56,205,222+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm3,%xmm7+.byte	102,15,58,15,254,4+	nop+	paddd	%xmm7,%xmm4+.byte	15,56,204,238+.byte	15,56,203,202+	movdqa	384-128(%rcx),%xmm0+	paddd	%xmm3,%xmm0+.byte	15,56,205,227+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm4,%xmm7+.byte	102,15,58,15,251,4+	nop+	paddd	%xmm7,%xmm5+.byte	15,56,204,243+.byte	15,56,203,202+	movdqa	416-128(%rcx),%xmm0+	paddd	%xmm4,%xmm0+.byte	15,56,205,236+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm5,%xmm7+.byte	102,15,58,15,252,4+.byte	15,56,203,202+	paddd	%xmm7,%xmm6++	movdqa	448-128(%rcx),%xmm0+	paddd	%xmm5,%xmm0+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+.byte	15,56,205,245+	movdqa	%xmm8,%xmm7+.byte	15,56,203,202++	movdqa	480-128(%rcx),%xmm0+	paddd	%xmm6,%xmm0+	nop+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	decq	%rdx+	nop+.byte	15,56,203,202++	paddd	%xmm10,%xmm2+	paddd	%xmm9,%xmm1+	jnz	.Loop_shaext++	pshufd	$0xb1,%xmm2,%xmm2+	pshufd	$0x1b,%xmm1,%xmm7+	pshufd	$0xb1,%xmm1,%xmm1+	punpckhqdq	%xmm2,%xmm1+.byte	102,15,58,15,215,8++	movdqu	%xmm1,(%rdi)+	movdqu	%xmm2,16(%rdi)+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp++	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_sha256_asm_block_data_order_shaext,.-crypton_sha256_asm_block_data_order_shaext+.type	crypton_sha256_asm_block_data_order_ssse3,@function+.align	64+crypton_sha256_asm_block_data_order_ssse3:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+.Lssse3_shortcut:+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$24,%rsp++	leaq	(%rsi,%rdx,4),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+++	leaq	-64(%rsp),%rsp+	movl	0(%rdi),%eax+	andq	$-64,%rsp+	movl	4(%rdi),%ebx+	movl	8(%rdi),%ecx+	movl	12(%rdi),%edx+	movl	16(%rdi),%r8d+	movl	20(%rdi),%r9d+	movl	24(%rdi),%r10d+	movl	28(%rdi),%r11d+++	jmp	.Lloop_ssse3+.align	16+.Lloop_ssse3:+	movdqa	K256+512(%rip),%xmm7+	movq	%rsi,-56(%rbp)+	movdqu	0(%rsi),%xmm0+	movdqu	16(%rsi),%xmm1+	movdqu	32(%rsi),%xmm2+.byte	102,15,56,0,199+	movdqu	48(%rsi),%xmm3+	leaq	K256(%rip),%rsi+.byte	102,15,56,0,207+	movdqa	0(%rsi),%xmm4+	movdqa	32(%rsi),%xmm5+.byte	102,15,56,0,215+	paddd	%xmm0,%xmm4+	movdqa	64(%rsi),%xmm6+.byte	102,15,56,0,223+	movdqa	96(%rsi),%xmm7+	paddd	%xmm1,%xmm5+	paddd	%xmm2,%xmm6+	paddd	%xmm3,%xmm7+	movdqa	%xmm4,0(%rsp)+	movl	%eax,%r14d+	movdqa	%xmm5,16(%rsp)+	movl	%ebx,%edi+	movdqa	%xmm6,32(%rsp)+	xorl	%ecx,%edi+	movdqa	%xmm7,48(%rsp)+	movl	%r8d,%r13d+	jmp	.Lssse3_00_47++.align	16+.Lssse3_00_47:+	subq	$-128,%rsi+	rorl	$14,%r13d+	movdqa	%xmm1,%xmm4+	movl	%r14d,%eax+	movl	%r9d,%r12d+	movdqa	%xmm3,%xmm7+	rorl	$9,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	rorl	$5,%r13d+	xorl	%eax,%r14d+.byte	102,15,58,15,224,4+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+.byte	102,15,58,15,250,4+	addl	0(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	rorl	$11,%r14d+	movdqa	%xmm4,%xmm5+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	movdqa	%xmm4,%xmm6+	rorl	$6,%r13d+	andl	%r15d,%edi+	psrld	$3,%xmm4+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	paddd	%xmm7,%xmm0+	rorl	$2,%r14d+	addl	%r11d,%edx+	psrld	$7,%xmm6+	addl	%edi,%r11d+	movl	%edx,%r13d+	pshufd	$250,%xmm3,%xmm7+	addl	%r11d,%r14d+	rorl	$14,%r13d+	pslld	$14,%xmm5+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	pxor	%xmm6,%xmm4+	rorl	$9,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	rorl	$5,%r13d+	psrld	$11,%xmm6+	xorl	%r11d,%r14d+	pxor	%xmm5,%xmm4+	andl	%edx,%r12d+	xorl	%edx,%r13d+	pslld	$11,%xmm5+	addl	4(%rsp),%r10d+	movl	%r11d,%edi+	pxor	%xmm6,%xmm4+	xorl	%r9d,%r12d+	rorl	$11,%r14d+	movdqa	%xmm7,%xmm6+	xorl	%eax,%edi+	addl	%r12d,%r10d+	pxor	%xmm5,%xmm4+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	psrld	$10,%xmm7+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	paddd	%xmm4,%xmm0+	rorl	$2,%r14d+	addl	%r10d,%ecx+	psrlq	$17,%xmm6+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	pxor	%xmm6,%xmm7+	rorl	$14,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	rorl	$9,%r14d+	psrlq	$2,%xmm6+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	pxor	%xmm6,%xmm7+	rorl	$5,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	pshufd	$128,%xmm7,%xmm7+	xorl	%ecx,%r13d+	addl	8(%rsp),%r9d+	movl	%r10d,%r15d+	psrldq	$8,%xmm7+	xorl	%r8d,%r12d+	rorl	$11,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	rorl	$6,%r13d+	paddd	%xmm7,%xmm0+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	pshufd	$80,%xmm0,%xmm7+	xorl	%r11d,%edi+	rorl	$2,%r14d+	addl	%r9d,%ebx+	movdqa	%xmm7,%xmm6+	addl	%edi,%r9d+	movl	%ebx,%r13d+	psrld	$10,%xmm7+	addl	%r9d,%r14d+	rorl	$14,%r13d+	psrlq	$17,%xmm6+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	pxor	%xmm6,%xmm7+	rorl	$9,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	rorl	$5,%r13d+	xorl	%r9d,%r14d+	psrlq	$2,%xmm6+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	12(%rsp),%r8d+	pxor	%xmm6,%xmm7+	movl	%r9d,%edi+	xorl	%edx,%r12d+	rorl	$11,%r14d+	pshufd	$8,%xmm7,%xmm7+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	movdqa	0(%rsi),%xmm6+	rorl	$6,%r13d+	andl	%edi,%r15d+	pslldq	$8,%xmm7+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	paddd	%xmm7,%xmm0+	rorl	$2,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	paddd	%xmm0,%xmm6+	movl	%eax,%r13d+	addl	%r8d,%r14d+	movdqa	%xmm6,0(%rsp)+	rorl	$14,%r13d+	movdqa	%xmm2,%xmm4+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	movdqa	%xmm0,%xmm7+	rorl	$9,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	rorl	$5,%r13d+	xorl	%r8d,%r14d+.byte	102,15,58,15,225,4+	andl	%eax,%r12d+	xorl	%eax,%r13d+.byte	102,15,58,15,251,4+	addl	16(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	rorl	$11,%r14d+	movdqa	%xmm4,%xmm5+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	movdqa	%xmm4,%xmm6+	rorl	$6,%r13d+	andl	%r15d,%edi+	psrld	$3,%xmm4+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	paddd	%xmm7,%xmm1+	rorl	$2,%r14d+	addl	%edx,%r11d+	psrld	$7,%xmm6+	addl	%edi,%edx+	movl	%r11d,%r13d+	pshufd	$250,%xmm0,%xmm7+	addl	%edx,%r14d+	rorl	$14,%r13d+	pslld	$14,%xmm5+	movl	%r14d,%edx+	movl	%eax,%r12d+	pxor	%xmm6,%xmm4+	rorl	$9,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	rorl	$5,%r13d+	psrld	$11,%xmm6+	xorl	%edx,%r14d+	pxor	%xmm5,%xmm4+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	pslld	$11,%xmm5+	addl	20(%rsp),%ecx+	movl	%edx,%edi+	pxor	%xmm6,%xmm4+	xorl	%ebx,%r12d+	rorl	$11,%r14d+	movdqa	%xmm7,%xmm6+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	pxor	%xmm5,%xmm4+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	psrld	$10,%xmm7+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	paddd	%xmm4,%xmm1+	rorl	$2,%r14d+	addl	%ecx,%r10d+	psrlq	$17,%xmm6+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	pxor	%xmm6,%xmm7+	rorl	$14,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	rorl	$9,%r14d+	psrlq	$2,%xmm6+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	pxor	%xmm6,%xmm7+	rorl	$5,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	pshufd	$128,%xmm7,%xmm7+	xorl	%r10d,%r13d+	addl	24(%rsp),%ebx+	movl	%ecx,%r15d+	psrldq	$8,%xmm7+	xorl	%eax,%r12d+	rorl	$11,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	rorl	$6,%r13d+	paddd	%xmm7,%xmm1+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	pshufd	$80,%xmm1,%xmm7+	xorl	%edx,%edi+	rorl	$2,%r14d+	addl	%ebx,%r9d+	movdqa	%xmm7,%xmm6+	addl	%edi,%ebx+	movl	%r9d,%r13d+	psrld	$10,%xmm7+	addl	%ebx,%r14d+	rorl	$14,%r13d+	psrlq	$17,%xmm6+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	pxor	%xmm6,%xmm7+	rorl	$9,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	rorl	$5,%r13d+	xorl	%ebx,%r14d+	psrlq	$2,%xmm6+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	28(%rsp),%eax+	pxor	%xmm6,%xmm7+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	rorl	$11,%r14d+	pshufd	$8,%xmm7,%xmm7+	xorl	%ecx,%edi+	addl	%r12d,%eax+	movdqa	32(%rsi),%xmm6+	rorl	$6,%r13d+	andl	%edi,%r15d+	pslldq	$8,%xmm7+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	paddd	%xmm7,%xmm1+	rorl	$2,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	paddd	%xmm1,%xmm6+	movl	%r8d,%r13d+	addl	%eax,%r14d+	movdqa	%xmm6,16(%rsp)+	rorl	$14,%r13d+	movdqa	%xmm3,%xmm4+	movl	%r14d,%eax+	movl	%r9d,%r12d+	movdqa	%xmm1,%xmm7+	rorl	$9,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	rorl	$5,%r13d+	xorl	%eax,%r14d+.byte	102,15,58,15,226,4+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+.byte	102,15,58,15,248,4+	addl	32(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	rorl	$11,%r14d+	movdqa	%xmm4,%xmm5+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	movdqa	%xmm4,%xmm6+	rorl	$6,%r13d+	andl	%r15d,%edi+	psrld	$3,%xmm4+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	paddd	%xmm7,%xmm2+	rorl	$2,%r14d+	addl	%r11d,%edx+	psrld	$7,%xmm6+	addl	%edi,%r11d+	movl	%edx,%r13d+	pshufd	$250,%xmm1,%xmm7+	addl	%r11d,%r14d+	rorl	$14,%r13d+	pslld	$14,%xmm5+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	pxor	%xmm6,%xmm4+	rorl	$9,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	rorl	$5,%r13d+	psrld	$11,%xmm6+	xorl	%r11d,%r14d+	pxor	%xmm5,%xmm4+	andl	%edx,%r12d+	xorl	%edx,%r13d+	pslld	$11,%xmm5+	addl	36(%rsp),%r10d+	movl	%r11d,%edi+	pxor	%xmm6,%xmm4+	xorl	%r9d,%r12d+	rorl	$11,%r14d+	movdqa	%xmm7,%xmm6+	xorl	%eax,%edi+	addl	%r12d,%r10d+	pxor	%xmm5,%xmm4+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	psrld	$10,%xmm7+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	paddd	%xmm4,%xmm2+	rorl	$2,%r14d+	addl	%r10d,%ecx+	psrlq	$17,%xmm6+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	pxor	%xmm6,%xmm7+	rorl	$14,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	rorl	$9,%r14d+	psrlq	$2,%xmm6+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	pxor	%xmm6,%xmm7+	rorl	$5,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	pshufd	$128,%xmm7,%xmm7+	xorl	%ecx,%r13d+	addl	40(%rsp),%r9d+	movl	%r10d,%r15d+	psrldq	$8,%xmm7+	xorl	%r8d,%r12d+	rorl	$11,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	rorl	$6,%r13d+	paddd	%xmm7,%xmm2+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	pshufd	$80,%xmm2,%xmm7+	xorl	%r11d,%edi+	rorl	$2,%r14d+	addl	%r9d,%ebx+	movdqa	%xmm7,%xmm6+	addl	%edi,%r9d+	movl	%ebx,%r13d+	psrld	$10,%xmm7+	addl	%r9d,%r14d+	rorl	$14,%r13d+	psrlq	$17,%xmm6+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	pxor	%xmm6,%xmm7+	rorl	$9,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	rorl	$5,%r13d+	xorl	%r9d,%r14d+	psrlq	$2,%xmm6+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	44(%rsp),%r8d+	pxor	%xmm6,%xmm7+	movl	%r9d,%edi+	xorl	%edx,%r12d+	rorl	$11,%r14d+	pshufd	$8,%xmm7,%xmm7+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	movdqa	64(%rsi),%xmm6+	rorl	$6,%r13d+	andl	%edi,%r15d+	pslldq	$8,%xmm7+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	paddd	%xmm7,%xmm2+	rorl	$2,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	paddd	%xmm2,%xmm6+	movl	%eax,%r13d+	addl	%r8d,%r14d+	movdqa	%xmm6,32(%rsp)+	rorl	$14,%r13d+	movdqa	%xmm0,%xmm4+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	movdqa	%xmm2,%xmm7+	rorl	$9,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	rorl	$5,%r13d+	xorl	%r8d,%r14d+.byte	102,15,58,15,227,4+	andl	%eax,%r12d+	xorl	%eax,%r13d+.byte	102,15,58,15,249,4+	addl	48(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	rorl	$11,%r14d+	movdqa	%xmm4,%xmm5+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	movdqa	%xmm4,%xmm6+	rorl	$6,%r13d+	andl	%r15d,%edi+	psrld	$3,%xmm4+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	paddd	%xmm7,%xmm3+	rorl	$2,%r14d+	addl	%edx,%r11d+	psrld	$7,%xmm6+	addl	%edi,%edx+	movl	%r11d,%r13d+	pshufd	$250,%xmm2,%xmm7+	addl	%edx,%r14d+	rorl	$14,%r13d+	pslld	$14,%xmm5+	movl	%r14d,%edx+	movl	%eax,%r12d+	pxor	%xmm6,%xmm4+	rorl	$9,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	rorl	$5,%r13d+	psrld	$11,%xmm6+	xorl	%edx,%r14d+	pxor	%xmm5,%xmm4+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	pslld	$11,%xmm5+	addl	52(%rsp),%ecx+	movl	%edx,%edi+	pxor	%xmm6,%xmm4+	xorl	%ebx,%r12d+	rorl	$11,%r14d+	movdqa	%xmm7,%xmm6+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	pxor	%xmm5,%xmm4+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	psrld	$10,%xmm7+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	paddd	%xmm4,%xmm3+	rorl	$2,%r14d+	addl	%ecx,%r10d+	psrlq	$17,%xmm6+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	pxor	%xmm6,%xmm7+	rorl	$14,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	rorl	$9,%r14d+	psrlq	$2,%xmm6+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	pxor	%xmm6,%xmm7+	rorl	$5,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	pshufd	$128,%xmm7,%xmm7+	xorl	%r10d,%r13d+	addl	56(%rsp),%ebx+	movl	%ecx,%r15d+	psrldq	$8,%xmm7+	xorl	%eax,%r12d+	rorl	$11,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	rorl	$6,%r13d+	paddd	%xmm7,%xmm3+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	pshufd	$80,%xmm3,%xmm7+	xorl	%edx,%edi+	rorl	$2,%r14d+	addl	%ebx,%r9d+	movdqa	%xmm7,%xmm6+	addl	%edi,%ebx+	movl	%r9d,%r13d+	psrld	$10,%xmm7+	addl	%ebx,%r14d+	rorl	$14,%r13d+	psrlq	$17,%xmm6+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	pxor	%xmm6,%xmm7+	rorl	$9,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	rorl	$5,%r13d+	xorl	%ebx,%r14d+	psrlq	$2,%xmm6+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	60(%rsp),%eax+	pxor	%xmm6,%xmm7+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	rorl	$11,%r14d+	pshufd	$8,%xmm7,%xmm7+	xorl	%ecx,%edi+	addl	%r12d,%eax+	movdqa	96(%rsi),%xmm6+	rorl	$6,%r13d+	andl	%edi,%r15d+	pslldq	$8,%xmm7+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	paddd	%xmm7,%xmm3+	rorl	$2,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	paddd	%xmm3,%xmm6+	movl	%r8d,%r13d+	addl	%eax,%r14d+	movdqa	%xmm6,48(%rsp)+	cmpb	$0,131(%rsi)+	jne	.Lssse3_00_47+	rorl	$14,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	rorl	$9,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	rorl	$5,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+	addl	0(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	rorl	$11,%r14d+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	rorl	$2,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	movl	%edx,%r13d+	addl	%r11d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	rorl	$9,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	rorl	$5,%r13d+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	addl	4(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	rorl	$11,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	rorl	$2,%r14d+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	rorl	$9,%r14d+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	rorl	$5,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	8(%rsp),%r9d+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	rorl	$11,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	xorl	%r11d,%edi+	rorl	$2,%r14d+	addl	%r9d,%ebx+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	rorl	$9,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	rorl	$5,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	12(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	rorl	$11,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	rorl	$2,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	rorl	$9,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	rorl	$5,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	xorl	%eax,%r13d+	addl	16(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	rorl	$11,%r14d+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	rorl	$2,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	movl	%r11d,%r13d+	addl	%edx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%edx+	movl	%eax,%r12d+	rorl	$9,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	rorl	$5,%r13d+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	addl	20(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	rorl	$11,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	rorl	$2,%r14d+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	rorl	$9,%r14d+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	rorl	$5,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	24(%rsp),%ebx+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	rorl	$11,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	xorl	%edx,%edi+	rorl	$2,%r14d+	addl	%ebx,%r9d+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	rorl	$9,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	rorl	$5,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	28(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	rorl	$11,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	rorl	$2,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	rorl	$14,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	rorl	$9,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	rorl	$5,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+	addl	32(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	rorl	$11,%r14d+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	rorl	$2,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	movl	%edx,%r13d+	addl	%r11d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	rorl	$9,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	rorl	$5,%r13d+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	addl	36(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	rorl	$11,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	rorl	$2,%r14d+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	rorl	$9,%r14d+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	rorl	$5,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	40(%rsp),%r9d+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	rorl	$11,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	xorl	%r11d,%edi+	rorl	$2,%r14d+	addl	%r9d,%ebx+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	rorl	$9,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	rorl	$5,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	44(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	rorl	$11,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	rorl	$2,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	rorl	$9,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	rorl	$5,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	xorl	%eax,%r13d+	addl	48(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	rorl	$11,%r14d+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	rorl	$2,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	movl	%r11d,%r13d+	addl	%edx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%edx+	movl	%eax,%r12d+	rorl	$9,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	rorl	$5,%r13d+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	addl	52(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	rorl	$11,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	rorl	$2,%r14d+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	rorl	$9,%r14d+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	rorl	$5,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	56(%rsp),%ebx+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	rorl	$11,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	xorl	%edx,%edi+	rorl	$2,%r14d+	addl	%ebx,%r9d+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	rorl	$9,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	rorl	$5,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	60(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	rorl	$11,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	rorl	$2,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	movq	-64(%rbp),%rdi+	movl	%r14d,%eax+	movq	-56(%rbp),%rsi++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	addl	24(%rdi),%r10d+	addl	28(%rdi),%r11d++	leaq	64(%rsi),%rsi+	cmpq	-48(%rbp),%rsi++	movl	%eax,0(%rdi)+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)+	jb	.Lloop_ssse3++	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_sha256_asm_block_data_order_ssse3,.-crypton_sha256_asm_block_data_order_ssse3+.type	crypton_sha256_asm_block_data_order_avx,@function+.align	64+crypton_sha256_asm_block_data_order_avx:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+.Lavx_shortcut:+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$24,%rsp++	leaq	(%rsi,%rdx,4),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+++	leaq	-64(%rsp),%rsp+	vzeroupper+	andq	$-64,%rsp+	movl	0(%rdi),%eax+	movl	4(%rdi),%ebx+	movl	8(%rdi),%ecx+	movl	12(%rdi),%edx+	movl	16(%rdi),%r8d+	movl	20(%rdi),%r9d+	movl	24(%rdi),%r10d+	movl	28(%rdi),%r11d+	vmovdqa	K256+512+32(%rip),%xmm8+	vmovdqa	K256+512+64(%rip),%xmm9+	jmp	.Lloop_avx+.align	16+.Lloop_avx:+	vmovdqa	K256+512(%rip),%xmm7+	movq	%rsi,-56(%rbp)+	vmovdqu	0(%rsi),%xmm0+	vmovdqu	16(%rsi),%xmm1+	vmovdqu	32(%rsi),%xmm2+	vmovdqu	48(%rsi),%xmm3+	vpshufb	%xmm7,%xmm0,%xmm0+	leaq	K256(%rip),%rsi+	vpshufb	%xmm7,%xmm1,%xmm1+	vpshufb	%xmm7,%xmm2,%xmm2+	vpaddd	0(%rsi),%xmm0,%xmm4+	vpshufb	%xmm7,%xmm3,%xmm3+	vpaddd	32(%rsi),%xmm1,%xmm5+	vpaddd	64(%rsi),%xmm2,%xmm6+	vpaddd	96(%rsi),%xmm3,%xmm7+	vmovdqa	%xmm4,0(%rsp)+	movl	%eax,%r14d+	vmovdqa	%xmm5,16(%rsp)+	movl	%ebx,%edi+	vmovdqa	%xmm6,32(%rsp)+	xorl	%ecx,%edi+	vmovdqa	%xmm7,48(%rsp)+	movl	%r8d,%r13d+	jmp	.Lavx_00_47++.align	16+.Lavx_00_47:+	subq	$-128,%rsi+	vpalignr	$4,%xmm0,%xmm1,%xmm4+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	vpalignr	$4,%xmm2,%xmm3,%xmm7+	shrdl	$9,%r14d,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	vpsrld	$7,%xmm4,%xmm6+	shrdl	$5,%r13d,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	vpaddd	%xmm7,%xmm0,%xmm0+	xorl	%r8d,%r13d+	addl	0(%rsp),%r11d+	movl	%eax,%r15d+	vpsrld	$3,%xmm4,%xmm7+	xorl	%r10d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ebx,%r15d+	vpslld	$14,%xmm4,%xmm5+	addl	%r12d,%r11d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	vpxor	%xmm6,%xmm7,%xmm4+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	vpshufd	$250,%xmm3,%xmm7+	shrdl	$2,%r14d,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	vpsrld	$11,%xmm6,%xmm6+	movl	%edx,%r13d+	addl	%r11d,%r14d+	shrdl	$14,%r13d,%r13d+	vpxor	%xmm5,%xmm4,%xmm4+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	shrdl	$9,%r14d,%r14d+	vpslld	$11,%xmm5,%xmm5+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	shrdl	$5,%r13d,%r13d+	vpxor	%xmm6,%xmm4,%xmm4+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	vpsrld	$10,%xmm7,%xmm6+	addl	4(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	vpxor	%xmm5,%xmm4,%xmm4+	shrdl	$11,%r14d,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	vpsrlq	$17,%xmm7,%xmm7+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	vpaddd	%xmm4,%xmm0,%xmm0+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	shrdl	$2,%r14d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%r10d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r10d+	vpxor	%xmm7,%xmm6,%xmm6+	movl	%edx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ecx,%r13d+	vpshufb	%xmm8,%xmm6,%xmm6+	xorl	%r8d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r10d,%r14d+	vpaddd	%xmm6,%xmm0,%xmm0+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	8(%rsp),%r9d+	vpshufd	$80,%xmm0,%xmm7+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	shrdl	$11,%r14d,%r14d+	vpsrld	$10,%xmm7,%xmm6+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	shrdl	$6,%r13d,%r13d+	vpsrlq	$17,%xmm7,%xmm7+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	vpxor	%xmm7,%xmm6,%xmm6+	xorl	%r11d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r9d,%ebx+	vpsrlq	$2,%xmm7,%xmm7+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	vpshufb	%xmm9,%xmm6,%xmm6+	shrdl	$9,%r14d,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	vpaddd	%xmm6,%xmm0,%xmm0+	shrdl	$5,%r13d,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	vpaddd	0(%rsi),%xmm0,%xmm6+	xorl	%ebx,%r13d+	addl	12(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	vmovdqa	%xmm6,0(%rsp)+	vpalignr	$4,%xmm1,%xmm2,%xmm4+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	vpalignr	$4,%xmm3,%xmm0,%xmm7+	shrdl	$9,%r14d,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	vpsrld	$7,%xmm4,%xmm6+	shrdl	$5,%r13d,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	vpaddd	%xmm7,%xmm1,%xmm1+	xorl	%eax,%r13d+	addl	16(%rsp),%edx+	movl	%r8d,%r15d+	vpsrld	$3,%xmm4,%xmm7+	xorl	%ecx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r9d,%r15d+	vpslld	$14,%xmm4,%xmm5+	addl	%r12d,%edx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	vpxor	%xmm6,%xmm7,%xmm4+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	vpshufd	$250,%xmm0,%xmm7+	shrdl	$2,%r14d,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	vpsrld	$11,%xmm6,%xmm6+	movl	%r11d,%r13d+	addl	%edx,%r14d+	shrdl	$14,%r13d,%r13d+	vpxor	%xmm5,%xmm4,%xmm4+	movl	%r14d,%edx+	movl	%eax,%r12d+	shrdl	$9,%r14d,%r14d+	vpslld	$11,%xmm5,%xmm5+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	shrdl	$5,%r13d,%r13d+	vpxor	%xmm6,%xmm4,%xmm4+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	vpsrld	$10,%xmm7,%xmm6+	addl	20(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	vpxor	%xmm5,%xmm4,%xmm4+	shrdl	$11,%r14d,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	vpsrlq	$17,%xmm7,%xmm7+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	vpaddd	%xmm4,%xmm1,%xmm1+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	shrdl	$2,%r14d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%ecx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ecx+	vpxor	%xmm7,%xmm6,%xmm6+	movl	%r11d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r10d,%r13d+	vpshufb	%xmm8,%xmm6,%xmm6+	xorl	%eax,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ecx,%r14d+	vpaddd	%xmm6,%xmm1,%xmm1+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	24(%rsp),%ebx+	vpshufd	$80,%xmm1,%xmm7+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	shrdl	$11,%r14d,%r14d+	vpsrld	$10,%xmm7,%xmm6+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	shrdl	$6,%r13d,%r13d+	vpsrlq	$17,%xmm7,%xmm7+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	vpxor	%xmm7,%xmm6,%xmm6+	xorl	%edx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%ebx,%r9d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	vpshufb	%xmm9,%xmm6,%xmm6+	shrdl	$9,%r14d,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	vpaddd	%xmm6,%xmm1,%xmm1+	shrdl	$5,%r13d,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	vpaddd	32(%rsi),%xmm1,%xmm6+	xorl	%r9d,%r13d+	addl	28(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	vmovdqa	%xmm6,16(%rsp)+	vpalignr	$4,%xmm2,%xmm3,%xmm4+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	vpalignr	$4,%xmm0,%xmm1,%xmm7+	shrdl	$9,%r14d,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	vpsrld	$7,%xmm4,%xmm6+	shrdl	$5,%r13d,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	vpaddd	%xmm7,%xmm2,%xmm2+	xorl	%r8d,%r13d+	addl	32(%rsp),%r11d+	movl	%eax,%r15d+	vpsrld	$3,%xmm4,%xmm7+	xorl	%r10d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ebx,%r15d+	vpslld	$14,%xmm4,%xmm5+	addl	%r12d,%r11d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	vpxor	%xmm6,%xmm7,%xmm4+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	vpshufd	$250,%xmm1,%xmm7+	shrdl	$2,%r14d,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	vpsrld	$11,%xmm6,%xmm6+	movl	%edx,%r13d+	addl	%r11d,%r14d+	shrdl	$14,%r13d,%r13d+	vpxor	%xmm5,%xmm4,%xmm4+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	shrdl	$9,%r14d,%r14d+	vpslld	$11,%xmm5,%xmm5+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	shrdl	$5,%r13d,%r13d+	vpxor	%xmm6,%xmm4,%xmm4+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	vpsrld	$10,%xmm7,%xmm6+	addl	36(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	vpxor	%xmm5,%xmm4,%xmm4+	shrdl	$11,%r14d,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	vpsrlq	$17,%xmm7,%xmm7+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	vpaddd	%xmm4,%xmm2,%xmm2+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	shrdl	$2,%r14d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%r10d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r10d+	vpxor	%xmm7,%xmm6,%xmm6+	movl	%edx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ecx,%r13d+	vpshufb	%xmm8,%xmm6,%xmm6+	xorl	%r8d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r10d,%r14d+	vpaddd	%xmm6,%xmm2,%xmm2+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	40(%rsp),%r9d+	vpshufd	$80,%xmm2,%xmm7+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	shrdl	$11,%r14d,%r14d+	vpsrld	$10,%xmm7,%xmm6+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	shrdl	$6,%r13d,%r13d+	vpsrlq	$17,%xmm7,%xmm7+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	vpxor	%xmm7,%xmm6,%xmm6+	xorl	%r11d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r9d,%ebx+	vpsrlq	$2,%xmm7,%xmm7+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	vpshufb	%xmm9,%xmm6,%xmm6+	shrdl	$9,%r14d,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	vpaddd	%xmm6,%xmm2,%xmm2+	shrdl	$5,%r13d,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	vpaddd	64(%rsi),%xmm2,%xmm6+	xorl	%ebx,%r13d+	addl	44(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	vmovdqa	%xmm6,32(%rsp)+	vpalignr	$4,%xmm3,%xmm0,%xmm4+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	vpalignr	$4,%xmm1,%xmm2,%xmm7+	shrdl	$9,%r14d,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	vpsrld	$7,%xmm4,%xmm6+	shrdl	$5,%r13d,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	vpaddd	%xmm7,%xmm3,%xmm3+	xorl	%eax,%r13d+	addl	48(%rsp),%edx+	movl	%r8d,%r15d+	vpsrld	$3,%xmm4,%xmm7+	xorl	%ecx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r9d,%r15d+	vpslld	$14,%xmm4,%xmm5+	addl	%r12d,%edx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	vpxor	%xmm6,%xmm7,%xmm4+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	vpshufd	$250,%xmm2,%xmm7+	shrdl	$2,%r14d,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	vpsrld	$11,%xmm6,%xmm6+	movl	%r11d,%r13d+	addl	%edx,%r14d+	shrdl	$14,%r13d,%r13d+	vpxor	%xmm5,%xmm4,%xmm4+	movl	%r14d,%edx+	movl	%eax,%r12d+	shrdl	$9,%r14d,%r14d+	vpslld	$11,%xmm5,%xmm5+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	shrdl	$5,%r13d,%r13d+	vpxor	%xmm6,%xmm4,%xmm4+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	vpsrld	$10,%xmm7,%xmm6+	addl	52(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	vpxor	%xmm5,%xmm4,%xmm4+	shrdl	$11,%r14d,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	vpsrlq	$17,%xmm7,%xmm7+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	vpaddd	%xmm4,%xmm3,%xmm3+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	shrdl	$2,%r14d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%ecx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ecx+	vpxor	%xmm7,%xmm6,%xmm6+	movl	%r11d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r10d,%r13d+	vpshufb	%xmm8,%xmm6,%xmm6+	xorl	%eax,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ecx,%r14d+	vpaddd	%xmm6,%xmm3,%xmm3+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	56(%rsp),%ebx+	vpshufd	$80,%xmm3,%xmm7+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	shrdl	$11,%r14d,%r14d+	vpsrld	$10,%xmm7,%xmm6+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	shrdl	$6,%r13d,%r13d+	vpsrlq	$17,%xmm7,%xmm7+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	vpxor	%xmm7,%xmm6,%xmm6+	xorl	%edx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%ebx,%r9d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	vpshufb	%xmm9,%xmm6,%xmm6+	shrdl	$9,%r14d,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	vpaddd	%xmm6,%xmm3,%xmm3+	shrdl	$5,%r13d,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	vpaddd	96(%rsi),%xmm3,%xmm6+	xorl	%r9d,%r13d+	addl	60(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	vmovdqa	%xmm6,48(%rsp)+	cmpb	$0,131(%rsi)+	jne	.Lavx_00_47+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+	addl	0(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	movl	%edx,%r13d+	addl	%r11d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	addl	4(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	8(%rsp),%r9d+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	xorl	%r11d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r9d,%ebx+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	12(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	xorl	%eax,%r13d+	addl	16(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	movl	%r11d,%r13d+	addl	%edx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%edx+	movl	%eax,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	addl	20(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	24(%rsp),%ebx+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	xorl	%edx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%ebx,%r9d+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	28(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+	addl	32(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	movl	%edx,%r13d+	addl	%r11d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	addl	36(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	40(%rsp),%r9d+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	xorl	%r11d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r9d,%ebx+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	44(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	xorl	%eax,%r13d+	addl	48(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	movl	%r11d,%r13d+	addl	%edx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%edx+	movl	%eax,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	addl	52(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	56(%rsp),%ebx+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	xorl	%edx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%ebx,%r9d+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	60(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	movq	-64(%rbp),%rdi+	movl	%r14d,%eax+	movq	-56(%rbp),%rsi++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	addl	24(%rdi),%r10d+	addl	28(%rdi),%r11d++	leaq	64(%rsi),%rsi+	cmpq	-48(%rbp),%rsi++	movl	%eax,0(%rdi)+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)+	jb	.Lloop_avx++	vzeroupper+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_sha256_asm_block_data_order_avx,.-crypton_sha256_asm_block_data_order_avx+.type	crypton_sha256_asm_block_data_order_avx2,@function+.align	64+crypton_sha256_asm_block_data_order_avx2:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+.Lavx2_shortcut:+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$24,%rsp++	leaq	(%rsi,%rdx,4),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+++	leaq	-64(%rsp),%rsp+	vzeroupper+	andq	$-64,%rsp+	subq	$-64,%rsi+	movl	0(%rdi),%eax+	movq	%rsi,%r12+	movl	4(%rdi),%ebx+	cmpq	%rdx,%rsi+	movl	8(%rdi),%ecx+	cmoveq	%rsp,%r12+	movl	12(%rdi),%edx+	movl	16(%rdi),%r8d+	movl	20(%rdi),%r9d+	movl	24(%rdi),%r10d+	movl	28(%rdi),%r11d+	vmovdqa	K256+512+32(%rip),%ymm8+	vmovdqa	K256+512+64(%rip),%ymm9+	jmp	.Loop_avx2+.align	16+.Loop_avx2:+	vmovdqa	K256+512(%rip),%ymm7+	movq	%rsi,-56(%rbp)+	vmovdqu	-64+0(%rsi),%xmm0+	vmovdqu	-64+16(%rsi),%xmm1+	vmovdqu	-64+32(%rsi),%xmm2+	vmovdqu	-64+48(%rsi),%xmm3+	leaq	K256(%rip),%rsi+	vinserti128	$1,(%r12),%ymm0,%ymm0+	vinserti128	$1,16(%r12),%ymm1,%ymm1+	vpshufb	%ymm7,%ymm0,%ymm0+	vinserti128	$1,32(%r12),%ymm2,%ymm2+	vpshufb	%ymm7,%ymm1,%ymm1+	vinserti128	$1,48(%r12),%ymm3,%ymm3++	vpshufb	%ymm7,%ymm2,%ymm2+	vpaddd	0(%rsi),%ymm0,%ymm4+	vpshufb	%ymm7,%ymm3,%ymm3+	vpaddd	32(%rsi),%ymm1,%ymm5+	vpaddd	64(%rsi),%ymm2,%ymm6+	vpaddd	96(%rsi),%ymm3,%ymm7+	vmovdqa	%ymm4,0(%rsp)+	xorl	%r14d,%r14d+	vmovdqa	%ymm5,32(%rsp)+	leaq	-64(%rsp),%rsp+	movl	%ebx,%edi+	vmovdqa	%ymm6,0(%rsp)+	xorl	%ecx,%edi+	vmovdqa	%ymm7,32(%rsp)+	movl	%r9d,%r12d+	subq	$-32*4,%rsi+	jmp	.Lavx2_00_47++.align	16+.Lavx2_00_47:+	leaq	-64(%rsp),%rsp+	vpalignr	$4,%ymm0,%ymm1,%ymm4+	addl	0+128(%rsp),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	vpalignr	$4,%ymm2,%ymm3,%ymm7+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	vpsrld	$7,%ymm4,%ymm6+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	vpaddd	%ymm7,%ymm0,%ymm0+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	vpsrld	$3,%ymm4,%ymm7+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	vpslld	$14,%ymm4,%ymm5+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	vpxor	%ymm6,%ymm7,%ymm4+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	vpshufd	$250,%ymm3,%ymm7+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	vpsrld	$11,%ymm6,%ymm6+	addl	4+128(%rsp),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	vpslld	$11,%ymm5,%ymm5+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	vpxor	%ymm6,%ymm4,%ymm4+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	vpsrld	$10,%ymm7,%ymm6+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	vpsrlq	$17,%ymm7,%ymm7+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	vpaddd	%ymm4,%ymm0,%ymm0+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	8+128(%rsp),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	vpshufb	%ymm8,%ymm6,%ymm6+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	vpaddd	%ymm6,%ymm0,%ymm0+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	vpshufd	$80,%ymm0,%ymm7+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	vpsrld	$10,%ymm7,%ymm6+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	vpsrlq	$17,%ymm7,%ymm7+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	12+128(%rsp),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	vpshufb	%ymm9,%ymm6,%ymm6+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	vpaddd	%ymm6,%ymm0,%ymm0+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	vpaddd	0(%rsi),%ymm0,%ymm6+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	vmovdqa	%ymm6,0(%rsp)+	vpalignr	$4,%ymm1,%ymm2,%ymm4+	addl	32+128(%rsp),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	vpalignr	$4,%ymm3,%ymm0,%ymm7+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	vpsrld	$7,%ymm4,%ymm6+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	vpaddd	%ymm7,%ymm1,%ymm1+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	vpsrld	$3,%ymm4,%ymm7+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	vpslld	$14,%ymm4,%ymm5+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	vpxor	%ymm6,%ymm7,%ymm4+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	vpshufd	$250,%ymm0,%ymm7+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	vpsrld	$11,%ymm6,%ymm6+	addl	36+128(%rsp),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	vpslld	$11,%ymm5,%ymm5+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	vpxor	%ymm6,%ymm4,%ymm4+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	vpsrld	$10,%ymm7,%ymm6+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	vpsrlq	$17,%ymm7,%ymm7+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	vpaddd	%ymm4,%ymm1,%ymm1+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	40+128(%rsp),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	vpshufb	%ymm8,%ymm6,%ymm6+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	vpaddd	%ymm6,%ymm1,%ymm1+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	vpshufd	$80,%ymm1,%ymm7+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	vpsrld	$10,%ymm7,%ymm6+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	vpsrlq	$17,%ymm7,%ymm7+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	44+128(%rsp),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	vpshufb	%ymm9,%ymm6,%ymm6+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	vpaddd	%ymm6,%ymm1,%ymm1+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	vpaddd	32(%rsi),%ymm1,%ymm6+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	vmovdqa	%ymm6,32(%rsp)+	leaq	-64(%rsp),%rsp+	vpalignr	$4,%ymm2,%ymm3,%ymm4+	addl	0+128(%rsp),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	vpalignr	$4,%ymm0,%ymm1,%ymm7+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	vpsrld	$7,%ymm4,%ymm6+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	vpaddd	%ymm7,%ymm2,%ymm2+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	vpsrld	$3,%ymm4,%ymm7+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	vpslld	$14,%ymm4,%ymm5+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	vpxor	%ymm6,%ymm7,%ymm4+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	vpshufd	$250,%ymm1,%ymm7+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	vpsrld	$11,%ymm6,%ymm6+	addl	4+128(%rsp),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	vpslld	$11,%ymm5,%ymm5+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	vpxor	%ymm6,%ymm4,%ymm4+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	vpsrld	$10,%ymm7,%ymm6+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	vpsrlq	$17,%ymm7,%ymm7+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	vpaddd	%ymm4,%ymm2,%ymm2+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	8+128(%rsp),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	vpshufb	%ymm8,%ymm6,%ymm6+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	vpaddd	%ymm6,%ymm2,%ymm2+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	vpshufd	$80,%ymm2,%ymm7+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	vpsrld	$10,%ymm7,%ymm6+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	vpsrlq	$17,%ymm7,%ymm7+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	12+128(%rsp),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	vpshufb	%ymm9,%ymm6,%ymm6+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	vpaddd	%ymm6,%ymm2,%ymm2+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	vpaddd	64(%rsi),%ymm2,%ymm6+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	vmovdqa	%ymm6,0(%rsp)+	vpalignr	$4,%ymm3,%ymm0,%ymm4+	addl	32+128(%rsp),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	vpalignr	$4,%ymm1,%ymm2,%ymm7+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	vpsrld	$7,%ymm4,%ymm6+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	vpaddd	%ymm7,%ymm3,%ymm3+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	vpsrld	$3,%ymm4,%ymm7+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	vpslld	$14,%ymm4,%ymm5+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	vpxor	%ymm6,%ymm7,%ymm4+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	vpshufd	$250,%ymm2,%ymm7+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	vpsrld	$11,%ymm6,%ymm6+	addl	36+128(%rsp),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	vpslld	$11,%ymm5,%ymm5+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	vpxor	%ymm6,%ymm4,%ymm4+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	vpsrld	$10,%ymm7,%ymm6+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	vpsrlq	$17,%ymm7,%ymm7+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	vpaddd	%ymm4,%ymm3,%ymm3+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	40+128(%rsp),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	vpshufb	%ymm8,%ymm6,%ymm6+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	vpaddd	%ymm6,%ymm3,%ymm3+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	vpshufd	$80,%ymm3,%ymm7+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	vpsrld	$10,%ymm7,%ymm6+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	vpsrlq	$17,%ymm7,%ymm7+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	44+128(%rsp),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	vpshufb	%ymm9,%ymm6,%ymm6+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	vpaddd	%ymm6,%ymm3,%ymm3+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	vpaddd	96(%rsi),%ymm3,%ymm6+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	vmovdqa	%ymm6,32(%rsp)+	leaq	128(%rsi),%rsi+	cmpb	$0,3(%rsi)+	jne	.Lavx2_00_47+	addl	0+64(%rsp),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	addl	4+64(%rsp),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	addl	8+64(%rsp),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	addl	12+64(%rsp),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	addl	32+64(%rsp),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	addl	36+64(%rsp),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	addl	40+64(%rsp),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	addl	44+64(%rsp),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	addl	0(%rsp),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	addl	4(%rsp),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	addl	8(%rsp),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	addl	12(%rsp),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	addl	32(%rsp),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	addl	36(%rsp),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	addl	40(%rsp),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	addl	44(%rsp),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	movq	-64(%rbp),%rdi+	addl	%r14d,%eax+	movl	-56(%rbp),%r12d++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	addl	24(%rdi),%r10d+	addl	28(%rdi),%r11d++	movl	%eax,0(%rdi)+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)++	cmpl	-48(%rbp),%r12d+	je	.Ldone_avx2++	leaq	448(%rsp),%rsi+	xorl	%r14d,%r14d+	movl	%ebx,%edi+	xorl	%ecx,%edi+	movl	%r9d,%r12d+	jmp	.Lower_avx2+.align	16+.Lower_avx2:+	addl	0+16(%rsi),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	addl	4+16(%rsi),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	addl	8+16(%rsi),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	addl	12+16(%rsi),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	addl	32+16(%rsi),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	addl	36+16(%rsi),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	addl	40+16(%rsi),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	addl	44+16(%rsi),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	leaq	-64(%rsi),%rsi+	cmpq	%rsp,%rsi+	jae	.Lower_avx2++	movq	-64(%rbp),%rdi+	addl	%r14d,%eax+	movq	-56(%rbp),%rsi+	leaq	448(%rsp),%rsp++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	leaq	128(%rsi),%rsi+	addl	24(%rdi),%r10d+	movq	%rsi,%r12+	addl	28(%rdi),%r11d+	cmpq	-48(%rbp),%rsi++	movl	%eax,0(%rdi)+	cmoveq	%rsp,%r12+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)++	jbe	.Loop_avx2++.Ldone_avx2:+	vzeroupper+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_sha256_asm_block_data_order_avx2,.-crypton_sha256_asm_block_data_order_avx2++.section	.note.gnu.property,"a",@note+	.long	4,2f-1f,5+	.byte	0x47,0x4E,0x55,0+1:	.long	0xc0000002,4,3+.align	8+2:++.section	.note.GNU-stack,"",@progbits
+ cbits/asm/sha256-x86_64-macosx.S view
@@ -0,0 +1,5454 @@+.text	+++.globl	_crypton_sha256_asm_block_data_order++.p2align	4+_crypton_sha256_asm_block_data_order:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+	leaq	_crypton_ia32cap_P(%rip),%rax+	movl	0(%rax),%r9d+	movl	4(%rax),%r10d+	movl	8(%rax),%eax+	testl	$536870912,%eax+	jnz	L$shaext_shortcut+	andl	$296,%eax+	cmpl	$296,%eax+	je	L$avx2_shortcut+	andl	$1073741824,%r9d+	andl	$268435968,%r10d+	orl	%r9d,%r10d+	cmpl	$1342177792,%r10d+	je	L$avx_shortcut+	testl	$512,%r10d+	jnz	L$ssse3_shortcut+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$64+24,%rsp++.cfi_def_cfa	%rsp,144++	leaq	(%rsi,%rdx,4),%rdx+	movq	%rdi,64+0(%rsp)+	movq	%rsi,64+8(%rsp)+	movq	%rdx,64+16(%rsp)++	movl	0(%rdi),%eax+	movl	4(%rdi),%ebx+	movl	8(%rdi),%ecx+	movl	12(%rdi),%edx+	movl	16(%rdi),%r8d+	movl	20(%rdi),%r9d+	movl	24(%rdi),%r10d+	movl	28(%rdi),%r11d+	jmp	L$loop++.p2align	4+L$loop:+	movl	%ebx,%edi+	leaq	K256(%rip),%rbp+	xorl	%ecx,%edi+	movl	0(%rsi),%r12d+	movl	%r8d,%r13d+	movl	%eax,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r9d,%r15d++	xorl	%r8d,%r13d+	rorl	$9,%r14d+	xorl	%r10d,%r15d++	movl	%r12d,0(%rsp)+	xorl	%eax,%r14d+	andl	%r8d,%r15d++	rorl	$5,%r13d+	addl	%r11d,%r12d+	xorl	%r10d,%r15d++	rorl	$11,%r14d+	xorl	%r8d,%r13d+	addl	%r15d,%r12d++	movl	%eax,%r15d+	addl	(%rbp),%r12d+	xorl	%eax,%r14d++	xorl	%ebx,%r15d+	rorl	$6,%r13d+	movl	%ebx,%r11d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r11d+	addl	%r12d,%edx+	addl	%r12d,%r11d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r11d+	movl	4(%rsi),%r12d+	movl	%edx,%r13d+	movl	%r11d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r8d,%edi++	xorl	%edx,%r13d+	rorl	$9,%r14d+	xorl	%r9d,%edi++	movl	%r12d,4(%rsp)+	xorl	%r11d,%r14d+	andl	%edx,%edi++	rorl	$5,%r13d+	addl	%r10d,%r12d+	xorl	%r9d,%edi++	rorl	$11,%r14d+	xorl	%edx,%r13d+	addl	%edi,%r12d++	movl	%r11d,%edi+	addl	(%rbp),%r12d+	xorl	%r11d,%r14d++	xorl	%eax,%edi+	rorl	$6,%r13d+	movl	%eax,%r10d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r10d+	addl	%r12d,%ecx+	addl	%r12d,%r10d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r10d+	movl	8(%rsi),%r12d+	movl	%ecx,%r13d+	movl	%r10d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%edx,%r15d++	xorl	%ecx,%r13d+	rorl	$9,%r14d+	xorl	%r8d,%r15d++	movl	%r12d,8(%rsp)+	xorl	%r10d,%r14d+	andl	%ecx,%r15d++	rorl	$5,%r13d+	addl	%r9d,%r12d+	xorl	%r8d,%r15d++	rorl	$11,%r14d+	xorl	%ecx,%r13d+	addl	%r15d,%r12d++	movl	%r10d,%r15d+	addl	(%rbp),%r12d+	xorl	%r10d,%r14d++	xorl	%r11d,%r15d+	rorl	$6,%r13d+	movl	%r11d,%r9d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r9d+	addl	%r12d,%ebx+	addl	%r12d,%r9d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r9d+	movl	12(%rsi),%r12d+	movl	%ebx,%r13d+	movl	%r9d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%ecx,%edi++	xorl	%ebx,%r13d+	rorl	$9,%r14d+	xorl	%edx,%edi++	movl	%r12d,12(%rsp)+	xorl	%r9d,%r14d+	andl	%ebx,%edi++	rorl	$5,%r13d+	addl	%r8d,%r12d+	xorl	%edx,%edi++	rorl	$11,%r14d+	xorl	%ebx,%r13d+	addl	%edi,%r12d++	movl	%r9d,%edi+	addl	(%rbp),%r12d+	xorl	%r9d,%r14d++	xorl	%r10d,%edi+	rorl	$6,%r13d+	movl	%r10d,%r8d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r8d+	addl	%r12d,%eax+	addl	%r12d,%r8d++	leaq	20(%rbp),%rbp+	addl	%r14d,%r8d+	movl	16(%rsi),%r12d+	movl	%eax,%r13d+	movl	%r8d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%ebx,%r15d++	xorl	%eax,%r13d+	rorl	$9,%r14d+	xorl	%ecx,%r15d++	movl	%r12d,16(%rsp)+	xorl	%r8d,%r14d+	andl	%eax,%r15d++	rorl	$5,%r13d+	addl	%edx,%r12d+	xorl	%ecx,%r15d++	rorl	$11,%r14d+	xorl	%eax,%r13d+	addl	%r15d,%r12d++	movl	%r8d,%r15d+	addl	(%rbp),%r12d+	xorl	%r8d,%r14d++	xorl	%r9d,%r15d+	rorl	$6,%r13d+	movl	%r9d,%edx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%edx+	addl	%r12d,%r11d+	addl	%r12d,%edx++	leaq	4(%rbp),%rbp+	addl	%r14d,%edx+	movl	20(%rsi),%r12d+	movl	%r11d,%r13d+	movl	%edx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%eax,%edi++	xorl	%r11d,%r13d+	rorl	$9,%r14d+	xorl	%ebx,%edi++	movl	%r12d,20(%rsp)+	xorl	%edx,%r14d+	andl	%r11d,%edi++	rorl	$5,%r13d+	addl	%ecx,%r12d+	xorl	%ebx,%edi++	rorl	$11,%r14d+	xorl	%r11d,%r13d+	addl	%edi,%r12d++	movl	%edx,%edi+	addl	(%rbp),%r12d+	xorl	%edx,%r14d++	xorl	%r8d,%edi+	rorl	$6,%r13d+	movl	%r8d,%ecx++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%ecx+	addl	%r12d,%r10d+	addl	%r12d,%ecx++	leaq	4(%rbp),%rbp+	addl	%r14d,%ecx+	movl	24(%rsi),%r12d+	movl	%r10d,%r13d+	movl	%ecx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r11d,%r15d++	xorl	%r10d,%r13d+	rorl	$9,%r14d+	xorl	%eax,%r15d++	movl	%r12d,24(%rsp)+	xorl	%ecx,%r14d+	andl	%r10d,%r15d++	rorl	$5,%r13d+	addl	%ebx,%r12d+	xorl	%eax,%r15d++	rorl	$11,%r14d+	xorl	%r10d,%r13d+	addl	%r15d,%r12d++	movl	%ecx,%r15d+	addl	(%rbp),%r12d+	xorl	%ecx,%r14d++	xorl	%edx,%r15d+	rorl	$6,%r13d+	movl	%edx,%ebx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%ebx+	addl	%r12d,%r9d+	addl	%r12d,%ebx++	leaq	4(%rbp),%rbp+	addl	%r14d,%ebx+	movl	28(%rsi),%r12d+	movl	%r9d,%r13d+	movl	%ebx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r10d,%edi++	xorl	%r9d,%r13d+	rorl	$9,%r14d+	xorl	%r11d,%edi++	movl	%r12d,28(%rsp)+	xorl	%ebx,%r14d+	andl	%r9d,%edi++	rorl	$5,%r13d+	addl	%eax,%r12d+	xorl	%r11d,%edi++	rorl	$11,%r14d+	xorl	%r9d,%r13d+	addl	%edi,%r12d++	movl	%ebx,%edi+	addl	(%rbp),%r12d+	xorl	%ebx,%r14d++	xorl	%ecx,%edi+	rorl	$6,%r13d+	movl	%ecx,%eax++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%eax+	addl	%r12d,%r8d+	addl	%r12d,%eax++	leaq	20(%rbp),%rbp+	addl	%r14d,%eax+	movl	32(%rsi),%r12d+	movl	%r8d,%r13d+	movl	%eax,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r9d,%r15d++	xorl	%r8d,%r13d+	rorl	$9,%r14d+	xorl	%r10d,%r15d++	movl	%r12d,32(%rsp)+	xorl	%eax,%r14d+	andl	%r8d,%r15d++	rorl	$5,%r13d+	addl	%r11d,%r12d+	xorl	%r10d,%r15d++	rorl	$11,%r14d+	xorl	%r8d,%r13d+	addl	%r15d,%r12d++	movl	%eax,%r15d+	addl	(%rbp),%r12d+	xorl	%eax,%r14d++	xorl	%ebx,%r15d+	rorl	$6,%r13d+	movl	%ebx,%r11d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r11d+	addl	%r12d,%edx+	addl	%r12d,%r11d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r11d+	movl	36(%rsi),%r12d+	movl	%edx,%r13d+	movl	%r11d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r8d,%edi++	xorl	%edx,%r13d+	rorl	$9,%r14d+	xorl	%r9d,%edi++	movl	%r12d,36(%rsp)+	xorl	%r11d,%r14d+	andl	%edx,%edi++	rorl	$5,%r13d+	addl	%r10d,%r12d+	xorl	%r9d,%edi++	rorl	$11,%r14d+	xorl	%edx,%r13d+	addl	%edi,%r12d++	movl	%r11d,%edi+	addl	(%rbp),%r12d+	xorl	%r11d,%r14d++	xorl	%eax,%edi+	rorl	$6,%r13d+	movl	%eax,%r10d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r10d+	addl	%r12d,%ecx+	addl	%r12d,%r10d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r10d+	movl	40(%rsi),%r12d+	movl	%ecx,%r13d+	movl	%r10d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%edx,%r15d++	xorl	%ecx,%r13d+	rorl	$9,%r14d+	xorl	%r8d,%r15d++	movl	%r12d,40(%rsp)+	xorl	%r10d,%r14d+	andl	%ecx,%r15d++	rorl	$5,%r13d+	addl	%r9d,%r12d+	xorl	%r8d,%r15d++	rorl	$11,%r14d+	xorl	%ecx,%r13d+	addl	%r15d,%r12d++	movl	%r10d,%r15d+	addl	(%rbp),%r12d+	xorl	%r10d,%r14d++	xorl	%r11d,%r15d+	rorl	$6,%r13d+	movl	%r11d,%r9d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r9d+	addl	%r12d,%ebx+	addl	%r12d,%r9d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r9d+	movl	44(%rsi),%r12d+	movl	%ebx,%r13d+	movl	%r9d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%ecx,%edi++	xorl	%ebx,%r13d+	rorl	$9,%r14d+	xorl	%edx,%edi++	movl	%r12d,44(%rsp)+	xorl	%r9d,%r14d+	andl	%ebx,%edi++	rorl	$5,%r13d+	addl	%r8d,%r12d+	xorl	%edx,%edi++	rorl	$11,%r14d+	xorl	%ebx,%r13d+	addl	%edi,%r12d++	movl	%r9d,%edi+	addl	(%rbp),%r12d+	xorl	%r9d,%r14d++	xorl	%r10d,%edi+	rorl	$6,%r13d+	movl	%r10d,%r8d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r8d+	addl	%r12d,%eax+	addl	%r12d,%r8d++	leaq	20(%rbp),%rbp+	addl	%r14d,%r8d+	movl	48(%rsi),%r12d+	movl	%eax,%r13d+	movl	%r8d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%ebx,%r15d++	xorl	%eax,%r13d+	rorl	$9,%r14d+	xorl	%ecx,%r15d++	movl	%r12d,48(%rsp)+	xorl	%r8d,%r14d+	andl	%eax,%r15d++	rorl	$5,%r13d+	addl	%edx,%r12d+	xorl	%ecx,%r15d++	rorl	$11,%r14d+	xorl	%eax,%r13d+	addl	%r15d,%r12d++	movl	%r8d,%r15d+	addl	(%rbp),%r12d+	xorl	%r8d,%r14d++	xorl	%r9d,%r15d+	rorl	$6,%r13d+	movl	%r9d,%edx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%edx+	addl	%r12d,%r11d+	addl	%r12d,%edx++	leaq	4(%rbp),%rbp+	addl	%r14d,%edx+	movl	52(%rsi),%r12d+	movl	%r11d,%r13d+	movl	%edx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%eax,%edi++	xorl	%r11d,%r13d+	rorl	$9,%r14d+	xorl	%ebx,%edi++	movl	%r12d,52(%rsp)+	xorl	%edx,%r14d+	andl	%r11d,%edi++	rorl	$5,%r13d+	addl	%ecx,%r12d+	xorl	%ebx,%edi++	rorl	$11,%r14d+	xorl	%r11d,%r13d+	addl	%edi,%r12d++	movl	%edx,%edi+	addl	(%rbp),%r12d+	xorl	%edx,%r14d++	xorl	%r8d,%edi+	rorl	$6,%r13d+	movl	%r8d,%ecx++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%ecx+	addl	%r12d,%r10d+	addl	%r12d,%ecx++	leaq	4(%rbp),%rbp+	addl	%r14d,%ecx+	movl	56(%rsi),%r12d+	movl	%r10d,%r13d+	movl	%ecx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r11d,%r15d++	xorl	%r10d,%r13d+	rorl	$9,%r14d+	xorl	%eax,%r15d++	movl	%r12d,56(%rsp)+	xorl	%ecx,%r14d+	andl	%r10d,%r15d++	rorl	$5,%r13d+	addl	%ebx,%r12d+	xorl	%eax,%r15d++	rorl	$11,%r14d+	xorl	%r10d,%r13d+	addl	%r15d,%r12d++	movl	%ecx,%r15d+	addl	(%rbp),%r12d+	xorl	%ecx,%r14d++	xorl	%edx,%r15d+	rorl	$6,%r13d+	movl	%edx,%ebx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%ebx+	addl	%r12d,%r9d+	addl	%r12d,%ebx++	leaq	4(%rbp),%rbp+	addl	%r14d,%ebx+	movl	60(%rsi),%r12d+	movl	%r9d,%r13d+	movl	%ebx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r10d,%edi++	xorl	%r9d,%r13d+	rorl	$9,%r14d+	xorl	%r11d,%edi++	movl	%r12d,60(%rsp)+	xorl	%ebx,%r14d+	andl	%r9d,%edi++	rorl	$5,%r13d+	addl	%eax,%r12d+	xorl	%r11d,%edi++	rorl	$11,%r14d+	xorl	%r9d,%r13d+	addl	%edi,%r12d++	movl	%ebx,%edi+	addl	(%rbp),%r12d+	xorl	%ebx,%r14d++	xorl	%ecx,%edi+	rorl	$6,%r13d+	movl	%ecx,%eax++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%eax+	addl	%r12d,%r8d+	addl	%r12d,%eax++	leaq	20(%rbp),%rbp+	jmp	L$rounds_16_xx+.p2align	4+L$rounds_16_xx:+	movl	4(%rsp),%r13d+	movl	56(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%eax+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	36(%rsp),%r12d++	addl	0(%rsp),%r12d+	movl	%r8d,%r13d+	addl	%r15d,%r12d+	movl	%eax,%r14d+	rorl	$14,%r13d+	movl	%r9d,%r15d++	xorl	%r8d,%r13d+	rorl	$9,%r14d+	xorl	%r10d,%r15d++	movl	%r12d,0(%rsp)+	xorl	%eax,%r14d+	andl	%r8d,%r15d++	rorl	$5,%r13d+	addl	%r11d,%r12d+	xorl	%r10d,%r15d++	rorl	$11,%r14d+	xorl	%r8d,%r13d+	addl	%r15d,%r12d++	movl	%eax,%r15d+	addl	(%rbp),%r12d+	xorl	%eax,%r14d++	xorl	%ebx,%r15d+	rorl	$6,%r13d+	movl	%ebx,%r11d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r11d+	addl	%r12d,%edx+	addl	%r12d,%r11d++	leaq	4(%rbp),%rbp+	movl	8(%rsp),%r13d+	movl	60(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r11d+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	40(%rsp),%r12d++	addl	4(%rsp),%r12d+	movl	%edx,%r13d+	addl	%edi,%r12d+	movl	%r11d,%r14d+	rorl	$14,%r13d+	movl	%r8d,%edi++	xorl	%edx,%r13d+	rorl	$9,%r14d+	xorl	%r9d,%edi++	movl	%r12d,4(%rsp)+	xorl	%r11d,%r14d+	andl	%edx,%edi++	rorl	$5,%r13d+	addl	%r10d,%r12d+	xorl	%r9d,%edi++	rorl	$11,%r14d+	xorl	%edx,%r13d+	addl	%edi,%r12d++	movl	%r11d,%edi+	addl	(%rbp),%r12d+	xorl	%r11d,%r14d++	xorl	%eax,%edi+	rorl	$6,%r13d+	movl	%eax,%r10d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r10d+	addl	%r12d,%ecx+	addl	%r12d,%r10d++	leaq	4(%rbp),%rbp+	movl	12(%rsp),%r13d+	movl	0(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r10d+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	44(%rsp),%r12d++	addl	8(%rsp),%r12d+	movl	%ecx,%r13d+	addl	%r15d,%r12d+	movl	%r10d,%r14d+	rorl	$14,%r13d+	movl	%edx,%r15d++	xorl	%ecx,%r13d+	rorl	$9,%r14d+	xorl	%r8d,%r15d++	movl	%r12d,8(%rsp)+	xorl	%r10d,%r14d+	andl	%ecx,%r15d++	rorl	$5,%r13d+	addl	%r9d,%r12d+	xorl	%r8d,%r15d++	rorl	$11,%r14d+	xorl	%ecx,%r13d+	addl	%r15d,%r12d++	movl	%r10d,%r15d+	addl	(%rbp),%r12d+	xorl	%r10d,%r14d++	xorl	%r11d,%r15d+	rorl	$6,%r13d+	movl	%r11d,%r9d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r9d+	addl	%r12d,%ebx+	addl	%r12d,%r9d++	leaq	4(%rbp),%rbp+	movl	16(%rsp),%r13d+	movl	4(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r9d+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	48(%rsp),%r12d++	addl	12(%rsp),%r12d+	movl	%ebx,%r13d+	addl	%edi,%r12d+	movl	%r9d,%r14d+	rorl	$14,%r13d+	movl	%ecx,%edi++	xorl	%ebx,%r13d+	rorl	$9,%r14d+	xorl	%edx,%edi++	movl	%r12d,12(%rsp)+	xorl	%r9d,%r14d+	andl	%ebx,%edi++	rorl	$5,%r13d+	addl	%r8d,%r12d+	xorl	%edx,%edi++	rorl	$11,%r14d+	xorl	%ebx,%r13d+	addl	%edi,%r12d++	movl	%r9d,%edi+	addl	(%rbp),%r12d+	xorl	%r9d,%r14d++	xorl	%r10d,%edi+	rorl	$6,%r13d+	movl	%r10d,%r8d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r8d+	addl	%r12d,%eax+	addl	%r12d,%r8d++	leaq	20(%rbp),%rbp+	movl	20(%rsp),%r13d+	movl	8(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r8d+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	52(%rsp),%r12d++	addl	16(%rsp),%r12d+	movl	%eax,%r13d+	addl	%r15d,%r12d+	movl	%r8d,%r14d+	rorl	$14,%r13d+	movl	%ebx,%r15d++	xorl	%eax,%r13d+	rorl	$9,%r14d+	xorl	%ecx,%r15d++	movl	%r12d,16(%rsp)+	xorl	%r8d,%r14d+	andl	%eax,%r15d++	rorl	$5,%r13d+	addl	%edx,%r12d+	xorl	%ecx,%r15d++	rorl	$11,%r14d+	xorl	%eax,%r13d+	addl	%r15d,%r12d++	movl	%r8d,%r15d+	addl	(%rbp),%r12d+	xorl	%r8d,%r14d++	xorl	%r9d,%r15d+	rorl	$6,%r13d+	movl	%r9d,%edx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%edx+	addl	%r12d,%r11d+	addl	%r12d,%edx++	leaq	4(%rbp),%rbp+	movl	24(%rsp),%r13d+	movl	12(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%edx+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	56(%rsp),%r12d++	addl	20(%rsp),%r12d+	movl	%r11d,%r13d+	addl	%edi,%r12d+	movl	%edx,%r14d+	rorl	$14,%r13d+	movl	%eax,%edi++	xorl	%r11d,%r13d+	rorl	$9,%r14d+	xorl	%ebx,%edi++	movl	%r12d,20(%rsp)+	xorl	%edx,%r14d+	andl	%r11d,%edi++	rorl	$5,%r13d+	addl	%ecx,%r12d+	xorl	%ebx,%edi++	rorl	$11,%r14d+	xorl	%r11d,%r13d+	addl	%edi,%r12d++	movl	%edx,%edi+	addl	(%rbp),%r12d+	xorl	%edx,%r14d++	xorl	%r8d,%edi+	rorl	$6,%r13d+	movl	%r8d,%ecx++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%ecx+	addl	%r12d,%r10d+	addl	%r12d,%ecx++	leaq	4(%rbp),%rbp+	movl	28(%rsp),%r13d+	movl	16(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%ecx+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	60(%rsp),%r12d++	addl	24(%rsp),%r12d+	movl	%r10d,%r13d+	addl	%r15d,%r12d+	movl	%ecx,%r14d+	rorl	$14,%r13d+	movl	%r11d,%r15d++	xorl	%r10d,%r13d+	rorl	$9,%r14d+	xorl	%eax,%r15d++	movl	%r12d,24(%rsp)+	xorl	%ecx,%r14d+	andl	%r10d,%r15d++	rorl	$5,%r13d+	addl	%ebx,%r12d+	xorl	%eax,%r15d++	rorl	$11,%r14d+	xorl	%r10d,%r13d+	addl	%r15d,%r12d++	movl	%ecx,%r15d+	addl	(%rbp),%r12d+	xorl	%ecx,%r14d++	xorl	%edx,%r15d+	rorl	$6,%r13d+	movl	%edx,%ebx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%ebx+	addl	%r12d,%r9d+	addl	%r12d,%ebx++	leaq	4(%rbp),%rbp+	movl	32(%rsp),%r13d+	movl	20(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%ebx+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	0(%rsp),%r12d++	addl	28(%rsp),%r12d+	movl	%r9d,%r13d+	addl	%edi,%r12d+	movl	%ebx,%r14d+	rorl	$14,%r13d+	movl	%r10d,%edi++	xorl	%r9d,%r13d+	rorl	$9,%r14d+	xorl	%r11d,%edi++	movl	%r12d,28(%rsp)+	xorl	%ebx,%r14d+	andl	%r9d,%edi++	rorl	$5,%r13d+	addl	%eax,%r12d+	xorl	%r11d,%edi++	rorl	$11,%r14d+	xorl	%r9d,%r13d+	addl	%edi,%r12d++	movl	%ebx,%edi+	addl	(%rbp),%r12d+	xorl	%ebx,%r14d++	xorl	%ecx,%edi+	rorl	$6,%r13d+	movl	%ecx,%eax++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%eax+	addl	%r12d,%r8d+	addl	%r12d,%eax++	leaq	20(%rbp),%rbp+	movl	36(%rsp),%r13d+	movl	24(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%eax+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	4(%rsp),%r12d++	addl	32(%rsp),%r12d+	movl	%r8d,%r13d+	addl	%r15d,%r12d+	movl	%eax,%r14d+	rorl	$14,%r13d+	movl	%r9d,%r15d++	xorl	%r8d,%r13d+	rorl	$9,%r14d+	xorl	%r10d,%r15d++	movl	%r12d,32(%rsp)+	xorl	%eax,%r14d+	andl	%r8d,%r15d++	rorl	$5,%r13d+	addl	%r11d,%r12d+	xorl	%r10d,%r15d++	rorl	$11,%r14d+	xorl	%r8d,%r13d+	addl	%r15d,%r12d++	movl	%eax,%r15d+	addl	(%rbp),%r12d+	xorl	%eax,%r14d++	xorl	%ebx,%r15d+	rorl	$6,%r13d+	movl	%ebx,%r11d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r11d+	addl	%r12d,%edx+	addl	%r12d,%r11d++	leaq	4(%rbp),%rbp+	movl	40(%rsp),%r13d+	movl	28(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r11d+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	8(%rsp),%r12d++	addl	36(%rsp),%r12d+	movl	%edx,%r13d+	addl	%edi,%r12d+	movl	%r11d,%r14d+	rorl	$14,%r13d+	movl	%r8d,%edi++	xorl	%edx,%r13d+	rorl	$9,%r14d+	xorl	%r9d,%edi++	movl	%r12d,36(%rsp)+	xorl	%r11d,%r14d+	andl	%edx,%edi++	rorl	$5,%r13d+	addl	%r10d,%r12d+	xorl	%r9d,%edi++	rorl	$11,%r14d+	xorl	%edx,%r13d+	addl	%edi,%r12d++	movl	%r11d,%edi+	addl	(%rbp),%r12d+	xorl	%r11d,%r14d++	xorl	%eax,%edi+	rorl	$6,%r13d+	movl	%eax,%r10d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r10d+	addl	%r12d,%ecx+	addl	%r12d,%r10d++	leaq	4(%rbp),%rbp+	movl	44(%rsp),%r13d+	movl	32(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r10d+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	12(%rsp),%r12d++	addl	40(%rsp),%r12d+	movl	%ecx,%r13d+	addl	%r15d,%r12d+	movl	%r10d,%r14d+	rorl	$14,%r13d+	movl	%edx,%r15d++	xorl	%ecx,%r13d+	rorl	$9,%r14d+	xorl	%r8d,%r15d++	movl	%r12d,40(%rsp)+	xorl	%r10d,%r14d+	andl	%ecx,%r15d++	rorl	$5,%r13d+	addl	%r9d,%r12d+	xorl	%r8d,%r15d++	rorl	$11,%r14d+	xorl	%ecx,%r13d+	addl	%r15d,%r12d++	movl	%r10d,%r15d+	addl	(%rbp),%r12d+	xorl	%r10d,%r14d++	xorl	%r11d,%r15d+	rorl	$6,%r13d+	movl	%r11d,%r9d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r9d+	addl	%r12d,%ebx+	addl	%r12d,%r9d++	leaq	4(%rbp),%rbp+	movl	48(%rsp),%r13d+	movl	36(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r9d+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	16(%rsp),%r12d++	addl	44(%rsp),%r12d+	movl	%ebx,%r13d+	addl	%edi,%r12d+	movl	%r9d,%r14d+	rorl	$14,%r13d+	movl	%ecx,%edi++	xorl	%ebx,%r13d+	rorl	$9,%r14d+	xorl	%edx,%edi++	movl	%r12d,44(%rsp)+	xorl	%r9d,%r14d+	andl	%ebx,%edi++	rorl	$5,%r13d+	addl	%r8d,%r12d+	xorl	%edx,%edi++	rorl	$11,%r14d+	xorl	%ebx,%r13d+	addl	%edi,%r12d++	movl	%r9d,%edi+	addl	(%rbp),%r12d+	xorl	%r9d,%r14d++	xorl	%r10d,%edi+	rorl	$6,%r13d+	movl	%r10d,%r8d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r8d+	addl	%r12d,%eax+	addl	%r12d,%r8d++	leaq	20(%rbp),%rbp+	movl	52(%rsp),%r13d+	movl	40(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r8d+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	20(%rsp),%r12d++	addl	48(%rsp),%r12d+	movl	%eax,%r13d+	addl	%r15d,%r12d+	movl	%r8d,%r14d+	rorl	$14,%r13d+	movl	%ebx,%r15d++	xorl	%eax,%r13d+	rorl	$9,%r14d+	xorl	%ecx,%r15d++	movl	%r12d,48(%rsp)+	xorl	%r8d,%r14d+	andl	%eax,%r15d++	rorl	$5,%r13d+	addl	%edx,%r12d+	xorl	%ecx,%r15d++	rorl	$11,%r14d+	xorl	%eax,%r13d+	addl	%r15d,%r12d++	movl	%r8d,%r15d+	addl	(%rbp),%r12d+	xorl	%r8d,%r14d++	xorl	%r9d,%r15d+	rorl	$6,%r13d+	movl	%r9d,%edx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%edx+	addl	%r12d,%r11d+	addl	%r12d,%edx++	leaq	4(%rbp),%rbp+	movl	56(%rsp),%r13d+	movl	44(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%edx+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	24(%rsp),%r12d++	addl	52(%rsp),%r12d+	movl	%r11d,%r13d+	addl	%edi,%r12d+	movl	%edx,%r14d+	rorl	$14,%r13d+	movl	%eax,%edi++	xorl	%r11d,%r13d+	rorl	$9,%r14d+	xorl	%ebx,%edi++	movl	%r12d,52(%rsp)+	xorl	%edx,%r14d+	andl	%r11d,%edi++	rorl	$5,%r13d+	addl	%ecx,%r12d+	xorl	%ebx,%edi++	rorl	$11,%r14d+	xorl	%r11d,%r13d+	addl	%edi,%r12d++	movl	%edx,%edi+	addl	(%rbp),%r12d+	xorl	%edx,%r14d++	xorl	%r8d,%edi+	rorl	$6,%r13d+	movl	%r8d,%ecx++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%ecx+	addl	%r12d,%r10d+	addl	%r12d,%ecx++	leaq	4(%rbp),%rbp+	movl	60(%rsp),%r13d+	movl	48(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%ecx+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	28(%rsp),%r12d++	addl	56(%rsp),%r12d+	movl	%r10d,%r13d+	addl	%r15d,%r12d+	movl	%ecx,%r14d+	rorl	$14,%r13d+	movl	%r11d,%r15d++	xorl	%r10d,%r13d+	rorl	$9,%r14d+	xorl	%eax,%r15d++	movl	%r12d,56(%rsp)+	xorl	%ecx,%r14d+	andl	%r10d,%r15d++	rorl	$5,%r13d+	addl	%ebx,%r12d+	xorl	%eax,%r15d++	rorl	$11,%r14d+	xorl	%r10d,%r13d+	addl	%r15d,%r12d++	movl	%ecx,%r15d+	addl	(%rbp),%r12d+	xorl	%ecx,%r14d++	xorl	%edx,%r15d+	rorl	$6,%r13d+	movl	%edx,%ebx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%ebx+	addl	%r12d,%r9d+	addl	%r12d,%ebx++	leaq	4(%rbp),%rbp+	movl	0(%rsp),%r13d+	movl	52(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%ebx+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	32(%rsp),%r12d++	addl	60(%rsp),%r12d+	movl	%r9d,%r13d+	addl	%edi,%r12d+	movl	%ebx,%r14d+	rorl	$14,%r13d+	movl	%r10d,%edi++	xorl	%r9d,%r13d+	rorl	$9,%r14d+	xorl	%r11d,%edi++	movl	%r12d,60(%rsp)+	xorl	%ebx,%r14d+	andl	%r9d,%edi++	rorl	$5,%r13d+	addl	%eax,%r12d+	xorl	%r11d,%edi++	rorl	$11,%r14d+	xorl	%r9d,%r13d+	addl	%edi,%r12d++	movl	%ebx,%edi+	addl	(%rbp),%r12d+	xorl	%ebx,%r14d++	xorl	%ecx,%edi+	rorl	$6,%r13d+	movl	%ecx,%eax++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%eax+	addl	%r12d,%r8d+	addl	%r12d,%eax++	leaq	20(%rbp),%rbp+	cmpb	$0,3(%rbp)+	jnz	L$rounds_16_xx++	movq	64+0(%rsp),%rdi+	addl	%r14d,%eax+	leaq	64(%rsi),%rsi++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	addl	24(%rdi),%r10d+	addl	28(%rdi),%r11d++	cmpq	64+16(%rsp),%rsi++	movl	%eax,0(%rdi)+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)+	jb	L$loop++	leaq	64+24+48(%rsp),%r11+.cfi_def_cfa	%r11,8+	movq	64+24(%rsp),%r15+	movq	-40(%r11),%r14+	movq	-32(%r11),%r13+	movq	-24(%r11),%r12+	movq	-16(%r11),%rbx+	movq	-8(%r11),%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbp+.cfi_restore	%rbx+	leaq	(%r11),%rsp+	.byte	0xf3,0xc3+.cfi_endproc	++.p2align	6++K256:+.long	0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5+.long	0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5+.long	0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5+.long	0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5+.long	0xd807aa98,0x12835b01,0x243185be,0x550c7dc3+.long	0xd807aa98,0x12835b01,0x243185be,0x550c7dc3+.long	0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174+.long	0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174+.long	0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc+.long	0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc+.long	0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da+.long	0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da+.long	0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7+.long	0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7+.long	0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967+.long	0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967+.long	0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13+.long	0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13+.long	0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85+.long	0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85+.long	0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3+.long	0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3+.long	0xd192e819,0xd6990624,0xf40e3585,0x106aa070+.long	0xd192e819,0xd6990624,0xf40e3585,0x106aa070+.long	0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5+.long	0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5+.long	0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3+.long	0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3+.long	0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208+.long	0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208+.long	0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2+.long	0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2++.long	0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f+.long	0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f+.long	0x03020100,0x0b0a0908,0xffffffff,0xffffffff+.long	0x03020100,0x0b0a0908,0xffffffff,0xffffffff+.long	0xffffffff,0xffffffff,0x03020100,0x0b0a0908+.long	0xffffffff,0xffffffff,0x03020100,0x0b0a0908+.byte	83,72,65,50,53,54,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0++.p2align	6+crypton_sha256_asm_block_data_order_shaext:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+L$shaext_shortcut:++	leaq	K256+128(%rip),%rcx+	movdqu	(%rdi),%xmm1+	movdqu	16(%rdi),%xmm2+	movdqa	512-128(%rcx),%xmm7++	pshufd	$0x1b,%xmm1,%xmm0+	pshufd	$0xb1,%xmm1,%xmm1+	pshufd	$0x1b,%xmm2,%xmm2+	movdqa	%xmm7,%xmm8+.byte	102,15,58,15,202,8+	punpcklqdq	%xmm0,%xmm2+	jmp	L$oop_shaext++.p2align	4+L$oop_shaext:+	movdqu	(%rsi),%xmm3+	movdqu	16(%rsi),%xmm4+	movdqu	32(%rsi),%xmm5+.byte	102,15,56,0,223+	movdqu	48(%rsi),%xmm6++	movdqa	0-128(%rcx),%xmm0+	paddd	%xmm3,%xmm0+.byte	102,15,56,0,231+	movdqa	%xmm2,%xmm10+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	nop+	movdqa	%xmm1,%xmm9+.byte	15,56,203,202++	movdqa	32-128(%rcx),%xmm0+	paddd	%xmm4,%xmm0+.byte	102,15,56,0,239+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	leaq	64(%rsi),%rsi+.byte	15,56,204,220+.byte	15,56,203,202++	movdqa	64-128(%rcx),%xmm0+	paddd	%xmm5,%xmm0+.byte	102,15,56,0,247+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm6,%xmm7+.byte	102,15,58,15,253,4+	nop+	paddd	%xmm7,%xmm3+.byte	15,56,204,229+.byte	15,56,203,202++	movdqa	96-128(%rcx),%xmm0+	paddd	%xmm6,%xmm0+.byte	15,56,205,222+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm3,%xmm7+.byte	102,15,58,15,254,4+	nop+	paddd	%xmm7,%xmm4+.byte	15,56,204,238+.byte	15,56,203,202+	movdqa	128-128(%rcx),%xmm0+	paddd	%xmm3,%xmm0+.byte	15,56,205,227+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm4,%xmm7+.byte	102,15,58,15,251,4+	nop+	paddd	%xmm7,%xmm5+.byte	15,56,204,243+.byte	15,56,203,202+	movdqa	160-128(%rcx),%xmm0+	paddd	%xmm4,%xmm0+.byte	15,56,205,236+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm5,%xmm7+.byte	102,15,58,15,252,4+	nop+	paddd	%xmm7,%xmm6+.byte	15,56,204,220+.byte	15,56,203,202+	movdqa	192-128(%rcx),%xmm0+	paddd	%xmm5,%xmm0+.byte	15,56,205,245+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm6,%xmm7+.byte	102,15,58,15,253,4+	nop+	paddd	%xmm7,%xmm3+.byte	15,56,204,229+.byte	15,56,203,202+	movdqa	224-128(%rcx),%xmm0+	paddd	%xmm6,%xmm0+.byte	15,56,205,222+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm3,%xmm7+.byte	102,15,58,15,254,4+	nop+	paddd	%xmm7,%xmm4+.byte	15,56,204,238+.byte	15,56,203,202+	movdqa	256-128(%rcx),%xmm0+	paddd	%xmm3,%xmm0+.byte	15,56,205,227+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm4,%xmm7+.byte	102,15,58,15,251,4+	nop+	paddd	%xmm7,%xmm5+.byte	15,56,204,243+.byte	15,56,203,202+	movdqa	288-128(%rcx),%xmm0+	paddd	%xmm4,%xmm0+.byte	15,56,205,236+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm5,%xmm7+.byte	102,15,58,15,252,4+	nop+	paddd	%xmm7,%xmm6+.byte	15,56,204,220+.byte	15,56,203,202+	movdqa	320-128(%rcx),%xmm0+	paddd	%xmm5,%xmm0+.byte	15,56,205,245+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm6,%xmm7+.byte	102,15,58,15,253,4+	nop+	paddd	%xmm7,%xmm3+.byte	15,56,204,229+.byte	15,56,203,202+	movdqa	352-128(%rcx),%xmm0+	paddd	%xmm6,%xmm0+.byte	15,56,205,222+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm3,%xmm7+.byte	102,15,58,15,254,4+	nop+	paddd	%xmm7,%xmm4+.byte	15,56,204,238+.byte	15,56,203,202+	movdqa	384-128(%rcx),%xmm0+	paddd	%xmm3,%xmm0+.byte	15,56,205,227+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm4,%xmm7+.byte	102,15,58,15,251,4+	nop+	paddd	%xmm7,%xmm5+.byte	15,56,204,243+.byte	15,56,203,202+	movdqa	416-128(%rcx),%xmm0+	paddd	%xmm4,%xmm0+.byte	15,56,205,236+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm5,%xmm7+.byte	102,15,58,15,252,4+.byte	15,56,203,202+	paddd	%xmm7,%xmm6++	movdqa	448-128(%rcx),%xmm0+	paddd	%xmm5,%xmm0+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+.byte	15,56,205,245+	movdqa	%xmm8,%xmm7+.byte	15,56,203,202++	movdqa	480-128(%rcx),%xmm0+	paddd	%xmm6,%xmm0+	nop+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	decq	%rdx+	nop+.byte	15,56,203,202++	paddd	%xmm10,%xmm2+	paddd	%xmm9,%xmm1+	jnz	L$oop_shaext++	pshufd	$0xb1,%xmm2,%xmm2+	pshufd	$0x1b,%xmm1,%xmm7+	pshufd	$0xb1,%xmm1,%xmm1+	punpckhqdq	%xmm2,%xmm1+.byte	102,15,58,15,215,8++	movdqu	%xmm1,(%rdi)+	movdqu	%xmm2,16(%rdi)+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp++	.byte	0xf3,0xc3+.cfi_endproc	+++.p2align	6+crypton_sha256_asm_block_data_order_ssse3:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+L$ssse3_shortcut:+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$24,%rsp++	leaq	(%rsi,%rdx,4),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+++	leaq	-64(%rsp),%rsp+	movl	0(%rdi),%eax+	andq	$-64,%rsp+	movl	4(%rdi),%ebx+	movl	8(%rdi),%ecx+	movl	12(%rdi),%edx+	movl	16(%rdi),%r8d+	movl	20(%rdi),%r9d+	movl	24(%rdi),%r10d+	movl	28(%rdi),%r11d+++	jmp	L$loop_ssse3+.p2align	4+L$loop_ssse3:+	movdqa	K256+512(%rip),%xmm7+	movq	%rsi,-56(%rbp)+	movdqu	0(%rsi),%xmm0+	movdqu	16(%rsi),%xmm1+	movdqu	32(%rsi),%xmm2+.byte	102,15,56,0,199+	movdqu	48(%rsi),%xmm3+	leaq	K256(%rip),%rsi+.byte	102,15,56,0,207+	movdqa	0(%rsi),%xmm4+	movdqa	32(%rsi),%xmm5+.byte	102,15,56,0,215+	paddd	%xmm0,%xmm4+	movdqa	64(%rsi),%xmm6+.byte	102,15,56,0,223+	movdqa	96(%rsi),%xmm7+	paddd	%xmm1,%xmm5+	paddd	%xmm2,%xmm6+	paddd	%xmm3,%xmm7+	movdqa	%xmm4,0(%rsp)+	movl	%eax,%r14d+	movdqa	%xmm5,16(%rsp)+	movl	%ebx,%edi+	movdqa	%xmm6,32(%rsp)+	xorl	%ecx,%edi+	movdqa	%xmm7,48(%rsp)+	movl	%r8d,%r13d+	jmp	L$ssse3_00_47++.p2align	4+L$ssse3_00_47:+	subq	$-128,%rsi+	rorl	$14,%r13d+	movdqa	%xmm1,%xmm4+	movl	%r14d,%eax+	movl	%r9d,%r12d+	movdqa	%xmm3,%xmm7+	rorl	$9,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	rorl	$5,%r13d+	xorl	%eax,%r14d+.byte	102,15,58,15,224,4+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+.byte	102,15,58,15,250,4+	addl	0(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	rorl	$11,%r14d+	movdqa	%xmm4,%xmm5+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	movdqa	%xmm4,%xmm6+	rorl	$6,%r13d+	andl	%r15d,%edi+	psrld	$3,%xmm4+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	paddd	%xmm7,%xmm0+	rorl	$2,%r14d+	addl	%r11d,%edx+	psrld	$7,%xmm6+	addl	%edi,%r11d+	movl	%edx,%r13d+	pshufd	$250,%xmm3,%xmm7+	addl	%r11d,%r14d+	rorl	$14,%r13d+	pslld	$14,%xmm5+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	pxor	%xmm6,%xmm4+	rorl	$9,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	rorl	$5,%r13d+	psrld	$11,%xmm6+	xorl	%r11d,%r14d+	pxor	%xmm5,%xmm4+	andl	%edx,%r12d+	xorl	%edx,%r13d+	pslld	$11,%xmm5+	addl	4(%rsp),%r10d+	movl	%r11d,%edi+	pxor	%xmm6,%xmm4+	xorl	%r9d,%r12d+	rorl	$11,%r14d+	movdqa	%xmm7,%xmm6+	xorl	%eax,%edi+	addl	%r12d,%r10d+	pxor	%xmm5,%xmm4+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	psrld	$10,%xmm7+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	paddd	%xmm4,%xmm0+	rorl	$2,%r14d+	addl	%r10d,%ecx+	psrlq	$17,%xmm6+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	pxor	%xmm6,%xmm7+	rorl	$14,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	rorl	$9,%r14d+	psrlq	$2,%xmm6+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	pxor	%xmm6,%xmm7+	rorl	$5,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	pshufd	$128,%xmm7,%xmm7+	xorl	%ecx,%r13d+	addl	8(%rsp),%r9d+	movl	%r10d,%r15d+	psrldq	$8,%xmm7+	xorl	%r8d,%r12d+	rorl	$11,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	rorl	$6,%r13d+	paddd	%xmm7,%xmm0+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	pshufd	$80,%xmm0,%xmm7+	xorl	%r11d,%edi+	rorl	$2,%r14d+	addl	%r9d,%ebx+	movdqa	%xmm7,%xmm6+	addl	%edi,%r9d+	movl	%ebx,%r13d+	psrld	$10,%xmm7+	addl	%r9d,%r14d+	rorl	$14,%r13d+	psrlq	$17,%xmm6+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	pxor	%xmm6,%xmm7+	rorl	$9,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	rorl	$5,%r13d+	xorl	%r9d,%r14d+	psrlq	$2,%xmm6+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	12(%rsp),%r8d+	pxor	%xmm6,%xmm7+	movl	%r9d,%edi+	xorl	%edx,%r12d+	rorl	$11,%r14d+	pshufd	$8,%xmm7,%xmm7+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	movdqa	0(%rsi),%xmm6+	rorl	$6,%r13d+	andl	%edi,%r15d+	pslldq	$8,%xmm7+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	paddd	%xmm7,%xmm0+	rorl	$2,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	paddd	%xmm0,%xmm6+	movl	%eax,%r13d+	addl	%r8d,%r14d+	movdqa	%xmm6,0(%rsp)+	rorl	$14,%r13d+	movdqa	%xmm2,%xmm4+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	movdqa	%xmm0,%xmm7+	rorl	$9,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	rorl	$5,%r13d+	xorl	%r8d,%r14d+.byte	102,15,58,15,225,4+	andl	%eax,%r12d+	xorl	%eax,%r13d+.byte	102,15,58,15,251,4+	addl	16(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	rorl	$11,%r14d+	movdqa	%xmm4,%xmm5+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	movdqa	%xmm4,%xmm6+	rorl	$6,%r13d+	andl	%r15d,%edi+	psrld	$3,%xmm4+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	paddd	%xmm7,%xmm1+	rorl	$2,%r14d+	addl	%edx,%r11d+	psrld	$7,%xmm6+	addl	%edi,%edx+	movl	%r11d,%r13d+	pshufd	$250,%xmm0,%xmm7+	addl	%edx,%r14d+	rorl	$14,%r13d+	pslld	$14,%xmm5+	movl	%r14d,%edx+	movl	%eax,%r12d+	pxor	%xmm6,%xmm4+	rorl	$9,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	rorl	$5,%r13d+	psrld	$11,%xmm6+	xorl	%edx,%r14d+	pxor	%xmm5,%xmm4+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	pslld	$11,%xmm5+	addl	20(%rsp),%ecx+	movl	%edx,%edi+	pxor	%xmm6,%xmm4+	xorl	%ebx,%r12d+	rorl	$11,%r14d+	movdqa	%xmm7,%xmm6+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	pxor	%xmm5,%xmm4+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	psrld	$10,%xmm7+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	paddd	%xmm4,%xmm1+	rorl	$2,%r14d+	addl	%ecx,%r10d+	psrlq	$17,%xmm6+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	pxor	%xmm6,%xmm7+	rorl	$14,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	rorl	$9,%r14d+	psrlq	$2,%xmm6+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	pxor	%xmm6,%xmm7+	rorl	$5,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	pshufd	$128,%xmm7,%xmm7+	xorl	%r10d,%r13d+	addl	24(%rsp),%ebx+	movl	%ecx,%r15d+	psrldq	$8,%xmm7+	xorl	%eax,%r12d+	rorl	$11,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	rorl	$6,%r13d+	paddd	%xmm7,%xmm1+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	pshufd	$80,%xmm1,%xmm7+	xorl	%edx,%edi+	rorl	$2,%r14d+	addl	%ebx,%r9d+	movdqa	%xmm7,%xmm6+	addl	%edi,%ebx+	movl	%r9d,%r13d+	psrld	$10,%xmm7+	addl	%ebx,%r14d+	rorl	$14,%r13d+	psrlq	$17,%xmm6+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	pxor	%xmm6,%xmm7+	rorl	$9,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	rorl	$5,%r13d+	xorl	%ebx,%r14d+	psrlq	$2,%xmm6+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	28(%rsp),%eax+	pxor	%xmm6,%xmm7+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	rorl	$11,%r14d+	pshufd	$8,%xmm7,%xmm7+	xorl	%ecx,%edi+	addl	%r12d,%eax+	movdqa	32(%rsi),%xmm6+	rorl	$6,%r13d+	andl	%edi,%r15d+	pslldq	$8,%xmm7+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	paddd	%xmm7,%xmm1+	rorl	$2,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	paddd	%xmm1,%xmm6+	movl	%r8d,%r13d+	addl	%eax,%r14d+	movdqa	%xmm6,16(%rsp)+	rorl	$14,%r13d+	movdqa	%xmm3,%xmm4+	movl	%r14d,%eax+	movl	%r9d,%r12d+	movdqa	%xmm1,%xmm7+	rorl	$9,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	rorl	$5,%r13d+	xorl	%eax,%r14d+.byte	102,15,58,15,226,4+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+.byte	102,15,58,15,248,4+	addl	32(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	rorl	$11,%r14d+	movdqa	%xmm4,%xmm5+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	movdqa	%xmm4,%xmm6+	rorl	$6,%r13d+	andl	%r15d,%edi+	psrld	$3,%xmm4+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	paddd	%xmm7,%xmm2+	rorl	$2,%r14d+	addl	%r11d,%edx+	psrld	$7,%xmm6+	addl	%edi,%r11d+	movl	%edx,%r13d+	pshufd	$250,%xmm1,%xmm7+	addl	%r11d,%r14d+	rorl	$14,%r13d+	pslld	$14,%xmm5+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	pxor	%xmm6,%xmm4+	rorl	$9,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	rorl	$5,%r13d+	psrld	$11,%xmm6+	xorl	%r11d,%r14d+	pxor	%xmm5,%xmm4+	andl	%edx,%r12d+	xorl	%edx,%r13d+	pslld	$11,%xmm5+	addl	36(%rsp),%r10d+	movl	%r11d,%edi+	pxor	%xmm6,%xmm4+	xorl	%r9d,%r12d+	rorl	$11,%r14d+	movdqa	%xmm7,%xmm6+	xorl	%eax,%edi+	addl	%r12d,%r10d+	pxor	%xmm5,%xmm4+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	psrld	$10,%xmm7+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	paddd	%xmm4,%xmm2+	rorl	$2,%r14d+	addl	%r10d,%ecx+	psrlq	$17,%xmm6+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	pxor	%xmm6,%xmm7+	rorl	$14,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	rorl	$9,%r14d+	psrlq	$2,%xmm6+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	pxor	%xmm6,%xmm7+	rorl	$5,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	pshufd	$128,%xmm7,%xmm7+	xorl	%ecx,%r13d+	addl	40(%rsp),%r9d+	movl	%r10d,%r15d+	psrldq	$8,%xmm7+	xorl	%r8d,%r12d+	rorl	$11,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	rorl	$6,%r13d+	paddd	%xmm7,%xmm2+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	pshufd	$80,%xmm2,%xmm7+	xorl	%r11d,%edi+	rorl	$2,%r14d+	addl	%r9d,%ebx+	movdqa	%xmm7,%xmm6+	addl	%edi,%r9d+	movl	%ebx,%r13d+	psrld	$10,%xmm7+	addl	%r9d,%r14d+	rorl	$14,%r13d+	psrlq	$17,%xmm6+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	pxor	%xmm6,%xmm7+	rorl	$9,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	rorl	$5,%r13d+	xorl	%r9d,%r14d+	psrlq	$2,%xmm6+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	44(%rsp),%r8d+	pxor	%xmm6,%xmm7+	movl	%r9d,%edi+	xorl	%edx,%r12d+	rorl	$11,%r14d+	pshufd	$8,%xmm7,%xmm7+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	movdqa	64(%rsi),%xmm6+	rorl	$6,%r13d+	andl	%edi,%r15d+	pslldq	$8,%xmm7+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	paddd	%xmm7,%xmm2+	rorl	$2,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	paddd	%xmm2,%xmm6+	movl	%eax,%r13d+	addl	%r8d,%r14d+	movdqa	%xmm6,32(%rsp)+	rorl	$14,%r13d+	movdqa	%xmm0,%xmm4+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	movdqa	%xmm2,%xmm7+	rorl	$9,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	rorl	$5,%r13d+	xorl	%r8d,%r14d+.byte	102,15,58,15,227,4+	andl	%eax,%r12d+	xorl	%eax,%r13d+.byte	102,15,58,15,249,4+	addl	48(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	rorl	$11,%r14d+	movdqa	%xmm4,%xmm5+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	movdqa	%xmm4,%xmm6+	rorl	$6,%r13d+	andl	%r15d,%edi+	psrld	$3,%xmm4+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	paddd	%xmm7,%xmm3+	rorl	$2,%r14d+	addl	%edx,%r11d+	psrld	$7,%xmm6+	addl	%edi,%edx+	movl	%r11d,%r13d+	pshufd	$250,%xmm2,%xmm7+	addl	%edx,%r14d+	rorl	$14,%r13d+	pslld	$14,%xmm5+	movl	%r14d,%edx+	movl	%eax,%r12d+	pxor	%xmm6,%xmm4+	rorl	$9,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	rorl	$5,%r13d+	psrld	$11,%xmm6+	xorl	%edx,%r14d+	pxor	%xmm5,%xmm4+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	pslld	$11,%xmm5+	addl	52(%rsp),%ecx+	movl	%edx,%edi+	pxor	%xmm6,%xmm4+	xorl	%ebx,%r12d+	rorl	$11,%r14d+	movdqa	%xmm7,%xmm6+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	pxor	%xmm5,%xmm4+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	psrld	$10,%xmm7+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	paddd	%xmm4,%xmm3+	rorl	$2,%r14d+	addl	%ecx,%r10d+	psrlq	$17,%xmm6+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	pxor	%xmm6,%xmm7+	rorl	$14,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	rorl	$9,%r14d+	psrlq	$2,%xmm6+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	pxor	%xmm6,%xmm7+	rorl	$5,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	pshufd	$128,%xmm7,%xmm7+	xorl	%r10d,%r13d+	addl	56(%rsp),%ebx+	movl	%ecx,%r15d+	psrldq	$8,%xmm7+	xorl	%eax,%r12d+	rorl	$11,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	rorl	$6,%r13d+	paddd	%xmm7,%xmm3+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	pshufd	$80,%xmm3,%xmm7+	xorl	%edx,%edi+	rorl	$2,%r14d+	addl	%ebx,%r9d+	movdqa	%xmm7,%xmm6+	addl	%edi,%ebx+	movl	%r9d,%r13d+	psrld	$10,%xmm7+	addl	%ebx,%r14d+	rorl	$14,%r13d+	psrlq	$17,%xmm6+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	pxor	%xmm6,%xmm7+	rorl	$9,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	rorl	$5,%r13d+	xorl	%ebx,%r14d+	psrlq	$2,%xmm6+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	60(%rsp),%eax+	pxor	%xmm6,%xmm7+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	rorl	$11,%r14d+	pshufd	$8,%xmm7,%xmm7+	xorl	%ecx,%edi+	addl	%r12d,%eax+	movdqa	96(%rsi),%xmm6+	rorl	$6,%r13d+	andl	%edi,%r15d+	pslldq	$8,%xmm7+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	paddd	%xmm7,%xmm3+	rorl	$2,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	paddd	%xmm3,%xmm6+	movl	%r8d,%r13d+	addl	%eax,%r14d+	movdqa	%xmm6,48(%rsp)+	cmpb	$0,131(%rsi)+	jne	L$ssse3_00_47+	rorl	$14,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	rorl	$9,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	rorl	$5,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+	addl	0(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	rorl	$11,%r14d+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	rorl	$2,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	movl	%edx,%r13d+	addl	%r11d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	rorl	$9,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	rorl	$5,%r13d+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	addl	4(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	rorl	$11,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	rorl	$2,%r14d+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	rorl	$9,%r14d+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	rorl	$5,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	8(%rsp),%r9d+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	rorl	$11,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	xorl	%r11d,%edi+	rorl	$2,%r14d+	addl	%r9d,%ebx+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	rorl	$9,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	rorl	$5,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	12(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	rorl	$11,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	rorl	$2,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	rorl	$9,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	rorl	$5,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	xorl	%eax,%r13d+	addl	16(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	rorl	$11,%r14d+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	rorl	$2,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	movl	%r11d,%r13d+	addl	%edx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%edx+	movl	%eax,%r12d+	rorl	$9,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	rorl	$5,%r13d+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	addl	20(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	rorl	$11,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	rorl	$2,%r14d+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	rorl	$9,%r14d+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	rorl	$5,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	24(%rsp),%ebx+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	rorl	$11,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	xorl	%edx,%edi+	rorl	$2,%r14d+	addl	%ebx,%r9d+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	rorl	$9,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	rorl	$5,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	28(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	rorl	$11,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	rorl	$2,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	rorl	$14,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	rorl	$9,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	rorl	$5,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+	addl	32(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	rorl	$11,%r14d+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	rorl	$2,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	movl	%edx,%r13d+	addl	%r11d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	rorl	$9,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	rorl	$5,%r13d+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	addl	36(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	rorl	$11,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	rorl	$2,%r14d+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	rorl	$9,%r14d+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	rorl	$5,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	40(%rsp),%r9d+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	rorl	$11,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	xorl	%r11d,%edi+	rorl	$2,%r14d+	addl	%r9d,%ebx+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	rorl	$9,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	rorl	$5,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	44(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	rorl	$11,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	rorl	$2,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	rorl	$9,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	rorl	$5,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	xorl	%eax,%r13d+	addl	48(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	rorl	$11,%r14d+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	rorl	$2,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	movl	%r11d,%r13d+	addl	%edx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%edx+	movl	%eax,%r12d+	rorl	$9,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	rorl	$5,%r13d+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	addl	52(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	rorl	$11,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	rorl	$2,%r14d+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	rorl	$9,%r14d+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	rorl	$5,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	56(%rsp),%ebx+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	rorl	$11,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	xorl	%edx,%edi+	rorl	$2,%r14d+	addl	%ebx,%r9d+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	rorl	$9,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	rorl	$5,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	60(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	rorl	$11,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	rorl	$2,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	movq	-64(%rbp),%rdi+	movl	%r14d,%eax+	movq	-56(%rbp),%rsi++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	addl	24(%rdi),%r10d+	addl	28(%rdi),%r11d++	leaq	64(%rsi),%rsi+	cmpq	-48(%rbp),%rsi++	movl	%eax,0(%rdi)+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)+	jb	L$loop_ssse3++	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+++.p2align	6+crypton_sha256_asm_block_data_order_avx:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+L$avx_shortcut:+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$24,%rsp++	leaq	(%rsi,%rdx,4),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+++	leaq	-64(%rsp),%rsp+	vzeroupper+	andq	$-64,%rsp+	movl	0(%rdi),%eax+	movl	4(%rdi),%ebx+	movl	8(%rdi),%ecx+	movl	12(%rdi),%edx+	movl	16(%rdi),%r8d+	movl	20(%rdi),%r9d+	movl	24(%rdi),%r10d+	movl	28(%rdi),%r11d+	vmovdqa	K256+512+32(%rip),%xmm8+	vmovdqa	K256+512+64(%rip),%xmm9+	jmp	L$loop_avx+.p2align	4+L$loop_avx:+	vmovdqa	K256+512(%rip),%xmm7+	movq	%rsi,-56(%rbp)+	vmovdqu	0(%rsi),%xmm0+	vmovdqu	16(%rsi),%xmm1+	vmovdqu	32(%rsi),%xmm2+	vmovdqu	48(%rsi),%xmm3+	vpshufb	%xmm7,%xmm0,%xmm0+	leaq	K256(%rip),%rsi+	vpshufb	%xmm7,%xmm1,%xmm1+	vpshufb	%xmm7,%xmm2,%xmm2+	vpaddd	0(%rsi),%xmm0,%xmm4+	vpshufb	%xmm7,%xmm3,%xmm3+	vpaddd	32(%rsi),%xmm1,%xmm5+	vpaddd	64(%rsi),%xmm2,%xmm6+	vpaddd	96(%rsi),%xmm3,%xmm7+	vmovdqa	%xmm4,0(%rsp)+	movl	%eax,%r14d+	vmovdqa	%xmm5,16(%rsp)+	movl	%ebx,%edi+	vmovdqa	%xmm6,32(%rsp)+	xorl	%ecx,%edi+	vmovdqa	%xmm7,48(%rsp)+	movl	%r8d,%r13d+	jmp	L$avx_00_47++.p2align	4+L$avx_00_47:+	subq	$-128,%rsi+	vpalignr	$4,%xmm0,%xmm1,%xmm4+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	vpalignr	$4,%xmm2,%xmm3,%xmm7+	shrdl	$9,%r14d,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	vpsrld	$7,%xmm4,%xmm6+	shrdl	$5,%r13d,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	vpaddd	%xmm7,%xmm0,%xmm0+	xorl	%r8d,%r13d+	addl	0(%rsp),%r11d+	movl	%eax,%r15d+	vpsrld	$3,%xmm4,%xmm7+	xorl	%r10d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ebx,%r15d+	vpslld	$14,%xmm4,%xmm5+	addl	%r12d,%r11d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	vpxor	%xmm6,%xmm7,%xmm4+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	vpshufd	$250,%xmm3,%xmm7+	shrdl	$2,%r14d,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	vpsrld	$11,%xmm6,%xmm6+	movl	%edx,%r13d+	addl	%r11d,%r14d+	shrdl	$14,%r13d,%r13d+	vpxor	%xmm5,%xmm4,%xmm4+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	shrdl	$9,%r14d,%r14d+	vpslld	$11,%xmm5,%xmm5+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	shrdl	$5,%r13d,%r13d+	vpxor	%xmm6,%xmm4,%xmm4+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	vpsrld	$10,%xmm7,%xmm6+	addl	4(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	vpxor	%xmm5,%xmm4,%xmm4+	shrdl	$11,%r14d,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	vpsrlq	$17,%xmm7,%xmm7+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	vpaddd	%xmm4,%xmm0,%xmm0+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	shrdl	$2,%r14d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%r10d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r10d+	vpxor	%xmm7,%xmm6,%xmm6+	movl	%edx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ecx,%r13d+	vpshufb	%xmm8,%xmm6,%xmm6+	xorl	%r8d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r10d,%r14d+	vpaddd	%xmm6,%xmm0,%xmm0+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	8(%rsp),%r9d+	vpshufd	$80,%xmm0,%xmm7+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	shrdl	$11,%r14d,%r14d+	vpsrld	$10,%xmm7,%xmm6+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	shrdl	$6,%r13d,%r13d+	vpsrlq	$17,%xmm7,%xmm7+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	vpxor	%xmm7,%xmm6,%xmm6+	xorl	%r11d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r9d,%ebx+	vpsrlq	$2,%xmm7,%xmm7+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	vpshufb	%xmm9,%xmm6,%xmm6+	shrdl	$9,%r14d,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	vpaddd	%xmm6,%xmm0,%xmm0+	shrdl	$5,%r13d,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	vpaddd	0(%rsi),%xmm0,%xmm6+	xorl	%ebx,%r13d+	addl	12(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	vmovdqa	%xmm6,0(%rsp)+	vpalignr	$4,%xmm1,%xmm2,%xmm4+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	vpalignr	$4,%xmm3,%xmm0,%xmm7+	shrdl	$9,%r14d,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	vpsrld	$7,%xmm4,%xmm6+	shrdl	$5,%r13d,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	vpaddd	%xmm7,%xmm1,%xmm1+	xorl	%eax,%r13d+	addl	16(%rsp),%edx+	movl	%r8d,%r15d+	vpsrld	$3,%xmm4,%xmm7+	xorl	%ecx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r9d,%r15d+	vpslld	$14,%xmm4,%xmm5+	addl	%r12d,%edx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	vpxor	%xmm6,%xmm7,%xmm4+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	vpshufd	$250,%xmm0,%xmm7+	shrdl	$2,%r14d,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	vpsrld	$11,%xmm6,%xmm6+	movl	%r11d,%r13d+	addl	%edx,%r14d+	shrdl	$14,%r13d,%r13d+	vpxor	%xmm5,%xmm4,%xmm4+	movl	%r14d,%edx+	movl	%eax,%r12d+	shrdl	$9,%r14d,%r14d+	vpslld	$11,%xmm5,%xmm5+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	shrdl	$5,%r13d,%r13d+	vpxor	%xmm6,%xmm4,%xmm4+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	vpsrld	$10,%xmm7,%xmm6+	addl	20(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	vpxor	%xmm5,%xmm4,%xmm4+	shrdl	$11,%r14d,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	vpsrlq	$17,%xmm7,%xmm7+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	vpaddd	%xmm4,%xmm1,%xmm1+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	shrdl	$2,%r14d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%ecx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ecx+	vpxor	%xmm7,%xmm6,%xmm6+	movl	%r11d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r10d,%r13d+	vpshufb	%xmm8,%xmm6,%xmm6+	xorl	%eax,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ecx,%r14d+	vpaddd	%xmm6,%xmm1,%xmm1+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	24(%rsp),%ebx+	vpshufd	$80,%xmm1,%xmm7+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	shrdl	$11,%r14d,%r14d+	vpsrld	$10,%xmm7,%xmm6+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	shrdl	$6,%r13d,%r13d+	vpsrlq	$17,%xmm7,%xmm7+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	vpxor	%xmm7,%xmm6,%xmm6+	xorl	%edx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%ebx,%r9d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	vpshufb	%xmm9,%xmm6,%xmm6+	shrdl	$9,%r14d,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	vpaddd	%xmm6,%xmm1,%xmm1+	shrdl	$5,%r13d,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	vpaddd	32(%rsi),%xmm1,%xmm6+	xorl	%r9d,%r13d+	addl	28(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	vmovdqa	%xmm6,16(%rsp)+	vpalignr	$4,%xmm2,%xmm3,%xmm4+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	vpalignr	$4,%xmm0,%xmm1,%xmm7+	shrdl	$9,%r14d,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	vpsrld	$7,%xmm4,%xmm6+	shrdl	$5,%r13d,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	vpaddd	%xmm7,%xmm2,%xmm2+	xorl	%r8d,%r13d+	addl	32(%rsp),%r11d+	movl	%eax,%r15d+	vpsrld	$3,%xmm4,%xmm7+	xorl	%r10d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ebx,%r15d+	vpslld	$14,%xmm4,%xmm5+	addl	%r12d,%r11d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	vpxor	%xmm6,%xmm7,%xmm4+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	vpshufd	$250,%xmm1,%xmm7+	shrdl	$2,%r14d,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	vpsrld	$11,%xmm6,%xmm6+	movl	%edx,%r13d+	addl	%r11d,%r14d+	shrdl	$14,%r13d,%r13d+	vpxor	%xmm5,%xmm4,%xmm4+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	shrdl	$9,%r14d,%r14d+	vpslld	$11,%xmm5,%xmm5+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	shrdl	$5,%r13d,%r13d+	vpxor	%xmm6,%xmm4,%xmm4+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	vpsrld	$10,%xmm7,%xmm6+	addl	36(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	vpxor	%xmm5,%xmm4,%xmm4+	shrdl	$11,%r14d,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	vpsrlq	$17,%xmm7,%xmm7+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	vpaddd	%xmm4,%xmm2,%xmm2+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	shrdl	$2,%r14d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%r10d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r10d+	vpxor	%xmm7,%xmm6,%xmm6+	movl	%edx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ecx,%r13d+	vpshufb	%xmm8,%xmm6,%xmm6+	xorl	%r8d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r10d,%r14d+	vpaddd	%xmm6,%xmm2,%xmm2+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	40(%rsp),%r9d+	vpshufd	$80,%xmm2,%xmm7+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	shrdl	$11,%r14d,%r14d+	vpsrld	$10,%xmm7,%xmm6+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	shrdl	$6,%r13d,%r13d+	vpsrlq	$17,%xmm7,%xmm7+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	vpxor	%xmm7,%xmm6,%xmm6+	xorl	%r11d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r9d,%ebx+	vpsrlq	$2,%xmm7,%xmm7+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	vpshufb	%xmm9,%xmm6,%xmm6+	shrdl	$9,%r14d,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	vpaddd	%xmm6,%xmm2,%xmm2+	shrdl	$5,%r13d,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	vpaddd	64(%rsi),%xmm2,%xmm6+	xorl	%ebx,%r13d+	addl	44(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	vmovdqa	%xmm6,32(%rsp)+	vpalignr	$4,%xmm3,%xmm0,%xmm4+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	vpalignr	$4,%xmm1,%xmm2,%xmm7+	shrdl	$9,%r14d,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	vpsrld	$7,%xmm4,%xmm6+	shrdl	$5,%r13d,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	vpaddd	%xmm7,%xmm3,%xmm3+	xorl	%eax,%r13d+	addl	48(%rsp),%edx+	movl	%r8d,%r15d+	vpsrld	$3,%xmm4,%xmm7+	xorl	%ecx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r9d,%r15d+	vpslld	$14,%xmm4,%xmm5+	addl	%r12d,%edx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	vpxor	%xmm6,%xmm7,%xmm4+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	vpshufd	$250,%xmm2,%xmm7+	shrdl	$2,%r14d,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	vpsrld	$11,%xmm6,%xmm6+	movl	%r11d,%r13d+	addl	%edx,%r14d+	shrdl	$14,%r13d,%r13d+	vpxor	%xmm5,%xmm4,%xmm4+	movl	%r14d,%edx+	movl	%eax,%r12d+	shrdl	$9,%r14d,%r14d+	vpslld	$11,%xmm5,%xmm5+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	shrdl	$5,%r13d,%r13d+	vpxor	%xmm6,%xmm4,%xmm4+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	vpsrld	$10,%xmm7,%xmm6+	addl	52(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	vpxor	%xmm5,%xmm4,%xmm4+	shrdl	$11,%r14d,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	vpsrlq	$17,%xmm7,%xmm7+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	vpaddd	%xmm4,%xmm3,%xmm3+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	shrdl	$2,%r14d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%ecx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ecx+	vpxor	%xmm7,%xmm6,%xmm6+	movl	%r11d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r10d,%r13d+	vpshufb	%xmm8,%xmm6,%xmm6+	xorl	%eax,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ecx,%r14d+	vpaddd	%xmm6,%xmm3,%xmm3+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	56(%rsp),%ebx+	vpshufd	$80,%xmm3,%xmm7+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	shrdl	$11,%r14d,%r14d+	vpsrld	$10,%xmm7,%xmm6+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	shrdl	$6,%r13d,%r13d+	vpsrlq	$17,%xmm7,%xmm7+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	vpxor	%xmm7,%xmm6,%xmm6+	xorl	%edx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%ebx,%r9d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	vpshufb	%xmm9,%xmm6,%xmm6+	shrdl	$9,%r14d,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	vpaddd	%xmm6,%xmm3,%xmm3+	shrdl	$5,%r13d,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	vpaddd	96(%rsi),%xmm3,%xmm6+	xorl	%r9d,%r13d+	addl	60(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	vmovdqa	%xmm6,48(%rsp)+	cmpb	$0,131(%rsi)+	jne	L$avx_00_47+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+	addl	0(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	movl	%edx,%r13d+	addl	%r11d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	addl	4(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	8(%rsp),%r9d+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	xorl	%r11d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r9d,%ebx+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	12(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	xorl	%eax,%r13d+	addl	16(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	movl	%r11d,%r13d+	addl	%edx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%edx+	movl	%eax,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	addl	20(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	24(%rsp),%ebx+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	xorl	%edx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%ebx,%r9d+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	28(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+	addl	32(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	movl	%edx,%r13d+	addl	%r11d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	addl	36(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	40(%rsp),%r9d+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	xorl	%r11d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r9d,%ebx+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	44(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	xorl	%eax,%r13d+	addl	48(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	movl	%r11d,%r13d+	addl	%edx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%edx+	movl	%eax,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	addl	52(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	56(%rsp),%ebx+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	xorl	%edx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%ebx,%r9d+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	60(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	movq	-64(%rbp),%rdi+	movl	%r14d,%eax+	movq	-56(%rbp),%rsi++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	addl	24(%rdi),%r10d+	addl	28(%rdi),%r11d++	leaq	64(%rsi),%rsi+	cmpq	-48(%rbp),%rsi++	movl	%eax,0(%rdi)+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)+	jb	L$loop_avx++	vzeroupper+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+++.p2align	6+crypton_sha256_asm_block_data_order_avx2:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+L$avx2_shortcut:+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$24,%rsp++	leaq	(%rsi,%rdx,4),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+++	leaq	-64(%rsp),%rsp+	vzeroupper+	andq	$-64,%rsp+	subq	$-64,%rsi+	movl	0(%rdi),%eax+	movq	%rsi,%r12+	movl	4(%rdi),%ebx+	cmpq	%rdx,%rsi+	movl	8(%rdi),%ecx+	cmoveq	%rsp,%r12+	movl	12(%rdi),%edx+	movl	16(%rdi),%r8d+	movl	20(%rdi),%r9d+	movl	24(%rdi),%r10d+	movl	28(%rdi),%r11d+	vmovdqa	K256+512+32(%rip),%ymm8+	vmovdqa	K256+512+64(%rip),%ymm9+	jmp	L$oop_avx2+.p2align	4+L$oop_avx2:+	vmovdqa	K256+512(%rip),%ymm7+	movq	%rsi,-56(%rbp)+	vmovdqu	-64+0(%rsi),%xmm0+	vmovdqu	-64+16(%rsi),%xmm1+	vmovdqu	-64+32(%rsi),%xmm2+	vmovdqu	-64+48(%rsi),%xmm3+	leaq	K256(%rip),%rsi+	vinserti128	$1,(%r12),%ymm0,%ymm0+	vinserti128	$1,16(%r12),%ymm1,%ymm1+	vpshufb	%ymm7,%ymm0,%ymm0+	vinserti128	$1,32(%r12),%ymm2,%ymm2+	vpshufb	%ymm7,%ymm1,%ymm1+	vinserti128	$1,48(%r12),%ymm3,%ymm3++	vpshufb	%ymm7,%ymm2,%ymm2+	vpaddd	0(%rsi),%ymm0,%ymm4+	vpshufb	%ymm7,%ymm3,%ymm3+	vpaddd	32(%rsi),%ymm1,%ymm5+	vpaddd	64(%rsi),%ymm2,%ymm6+	vpaddd	96(%rsi),%ymm3,%ymm7+	vmovdqa	%ymm4,0(%rsp)+	xorl	%r14d,%r14d+	vmovdqa	%ymm5,32(%rsp)+	leaq	-64(%rsp),%rsp+	movl	%ebx,%edi+	vmovdqa	%ymm6,0(%rsp)+	xorl	%ecx,%edi+	vmovdqa	%ymm7,32(%rsp)+	movl	%r9d,%r12d+	subq	$-32*4,%rsi+	jmp	L$avx2_00_47++.p2align	4+L$avx2_00_47:+	leaq	-64(%rsp),%rsp+	vpalignr	$4,%ymm0,%ymm1,%ymm4+	addl	0+128(%rsp),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	vpalignr	$4,%ymm2,%ymm3,%ymm7+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	vpsrld	$7,%ymm4,%ymm6+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	vpaddd	%ymm7,%ymm0,%ymm0+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	vpsrld	$3,%ymm4,%ymm7+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	vpslld	$14,%ymm4,%ymm5+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	vpxor	%ymm6,%ymm7,%ymm4+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	vpshufd	$250,%ymm3,%ymm7+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	vpsrld	$11,%ymm6,%ymm6+	addl	4+128(%rsp),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	vpslld	$11,%ymm5,%ymm5+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	vpxor	%ymm6,%ymm4,%ymm4+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	vpsrld	$10,%ymm7,%ymm6+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	vpsrlq	$17,%ymm7,%ymm7+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	vpaddd	%ymm4,%ymm0,%ymm0+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	8+128(%rsp),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	vpshufb	%ymm8,%ymm6,%ymm6+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	vpaddd	%ymm6,%ymm0,%ymm0+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	vpshufd	$80,%ymm0,%ymm7+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	vpsrld	$10,%ymm7,%ymm6+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	vpsrlq	$17,%ymm7,%ymm7+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	12+128(%rsp),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	vpshufb	%ymm9,%ymm6,%ymm6+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	vpaddd	%ymm6,%ymm0,%ymm0+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	vpaddd	0(%rsi),%ymm0,%ymm6+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	vmovdqa	%ymm6,0(%rsp)+	vpalignr	$4,%ymm1,%ymm2,%ymm4+	addl	32+128(%rsp),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	vpalignr	$4,%ymm3,%ymm0,%ymm7+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	vpsrld	$7,%ymm4,%ymm6+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	vpaddd	%ymm7,%ymm1,%ymm1+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	vpsrld	$3,%ymm4,%ymm7+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	vpslld	$14,%ymm4,%ymm5+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	vpxor	%ymm6,%ymm7,%ymm4+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	vpshufd	$250,%ymm0,%ymm7+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	vpsrld	$11,%ymm6,%ymm6+	addl	36+128(%rsp),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	vpslld	$11,%ymm5,%ymm5+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	vpxor	%ymm6,%ymm4,%ymm4+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	vpsrld	$10,%ymm7,%ymm6+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	vpsrlq	$17,%ymm7,%ymm7+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	vpaddd	%ymm4,%ymm1,%ymm1+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	40+128(%rsp),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	vpshufb	%ymm8,%ymm6,%ymm6+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	vpaddd	%ymm6,%ymm1,%ymm1+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	vpshufd	$80,%ymm1,%ymm7+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	vpsrld	$10,%ymm7,%ymm6+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	vpsrlq	$17,%ymm7,%ymm7+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	44+128(%rsp),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	vpshufb	%ymm9,%ymm6,%ymm6+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	vpaddd	%ymm6,%ymm1,%ymm1+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	vpaddd	32(%rsi),%ymm1,%ymm6+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	vmovdqa	%ymm6,32(%rsp)+	leaq	-64(%rsp),%rsp+	vpalignr	$4,%ymm2,%ymm3,%ymm4+	addl	0+128(%rsp),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	vpalignr	$4,%ymm0,%ymm1,%ymm7+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	vpsrld	$7,%ymm4,%ymm6+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	vpaddd	%ymm7,%ymm2,%ymm2+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	vpsrld	$3,%ymm4,%ymm7+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	vpslld	$14,%ymm4,%ymm5+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	vpxor	%ymm6,%ymm7,%ymm4+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	vpshufd	$250,%ymm1,%ymm7+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	vpsrld	$11,%ymm6,%ymm6+	addl	4+128(%rsp),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	vpslld	$11,%ymm5,%ymm5+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	vpxor	%ymm6,%ymm4,%ymm4+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	vpsrld	$10,%ymm7,%ymm6+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	vpsrlq	$17,%ymm7,%ymm7+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	vpaddd	%ymm4,%ymm2,%ymm2+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	8+128(%rsp),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	vpshufb	%ymm8,%ymm6,%ymm6+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	vpaddd	%ymm6,%ymm2,%ymm2+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	vpshufd	$80,%ymm2,%ymm7+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	vpsrld	$10,%ymm7,%ymm6+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	vpsrlq	$17,%ymm7,%ymm7+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	12+128(%rsp),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	vpshufb	%ymm9,%ymm6,%ymm6+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	vpaddd	%ymm6,%ymm2,%ymm2+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	vpaddd	64(%rsi),%ymm2,%ymm6+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	vmovdqa	%ymm6,0(%rsp)+	vpalignr	$4,%ymm3,%ymm0,%ymm4+	addl	32+128(%rsp),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	vpalignr	$4,%ymm1,%ymm2,%ymm7+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	vpsrld	$7,%ymm4,%ymm6+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	vpaddd	%ymm7,%ymm3,%ymm3+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	vpsrld	$3,%ymm4,%ymm7+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	vpslld	$14,%ymm4,%ymm5+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	vpxor	%ymm6,%ymm7,%ymm4+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	vpshufd	$250,%ymm2,%ymm7+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	vpsrld	$11,%ymm6,%ymm6+	addl	36+128(%rsp),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	vpslld	$11,%ymm5,%ymm5+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	vpxor	%ymm6,%ymm4,%ymm4+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	vpsrld	$10,%ymm7,%ymm6+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	vpsrlq	$17,%ymm7,%ymm7+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	vpaddd	%ymm4,%ymm3,%ymm3+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	40+128(%rsp),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	vpshufb	%ymm8,%ymm6,%ymm6+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	vpaddd	%ymm6,%ymm3,%ymm3+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	vpshufd	$80,%ymm3,%ymm7+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	vpsrld	$10,%ymm7,%ymm6+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	vpsrlq	$17,%ymm7,%ymm7+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	44+128(%rsp),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	vpshufb	%ymm9,%ymm6,%ymm6+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	vpaddd	%ymm6,%ymm3,%ymm3+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	vpaddd	96(%rsi),%ymm3,%ymm6+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	vmovdqa	%ymm6,32(%rsp)+	leaq	128(%rsi),%rsi+	cmpb	$0,3(%rsi)+	jne	L$avx2_00_47+	addl	0+64(%rsp),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	addl	4+64(%rsp),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	addl	8+64(%rsp),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	addl	12+64(%rsp),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	addl	32+64(%rsp),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	addl	36+64(%rsp),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	addl	40+64(%rsp),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	addl	44+64(%rsp),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	addl	0(%rsp),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	addl	4(%rsp),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	addl	8(%rsp),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	addl	12(%rsp),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	addl	32(%rsp),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	addl	36(%rsp),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	addl	40(%rsp),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	addl	44(%rsp),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	movq	-64(%rbp),%rdi+	addl	%r14d,%eax+	movl	-56(%rbp),%r12d++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	addl	24(%rdi),%r10d+	addl	28(%rdi),%r11d++	movl	%eax,0(%rdi)+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)++	cmpl	-48(%rbp),%r12d+	je	L$done_avx2++	leaq	448(%rsp),%rsi+	xorl	%r14d,%r14d+	movl	%ebx,%edi+	xorl	%ecx,%edi+	movl	%r9d,%r12d+	jmp	L$ower_avx2+.p2align	4+L$ower_avx2:+	addl	0+16(%rsi),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	addl	4+16(%rsi),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	addl	8+16(%rsi),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	addl	12+16(%rsi),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	addl	32+16(%rsi),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	addl	36+16(%rsi),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	addl	40+16(%rsi),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	addl	44+16(%rsi),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	leaq	-64(%rsi),%rsi+	cmpq	%rsp,%rsi+	jae	L$ower_avx2++	movq	-64(%rbp),%rdi+	addl	%r14d,%eax+	movq	-56(%rbp),%rsi+	leaq	448(%rsp),%rsp++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	leaq	128(%rsi),%rsi+	addl	24(%rdi),%r10d+	movq	%rsi,%r12+	addl	28(%rdi),%r11d+	cmpq	-48(%rbp),%rsi++	movl	%eax,0(%rdi)+	cmoveq	%rsp,%r12+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)++	jbe	L$oop_avx2++L$done_avx2:+	vzeroupper+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+
+ cbits/asm/sha256-x86_64-mingw64.S view
@@ -0,0 +1,5731 @@+.text	+++.globl	crypton_sha256_asm_block_data_order+.def	crypton_sha256_asm_block_data_order;	.scl 2;	.type 32;	.endef+.p2align	4+crypton_sha256_asm_block_data_order:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_sha256_asm_block_data_order:+++	pushq	%rbp++	movq	%rsp,%rbp++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	leaq	crypton_ia32cap_P(%rip),%rax+	movl	0(%rax),%r9d+	movl	4(%rax),%r10d+	movl	8(%rax),%eax+	testl	$536870912,%eax+	jnz	.Lshaext_shortcut+	andl	$296,%eax+	cmpl	$296,%eax+	je	.Lavx2_shortcut+	andl	$1073741824,%r9d+	andl	$268435968,%r10d+	orl	%r9d,%r10d+	cmpl	$1342177792,%r10d+	je	.Lavx_shortcut+	testl	$512,%r10d+	jnz	.Lssse3_shortcut+	pushq	%rbx++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	shlq	$4,%rdx+	subq	$64+24,%rsp+++.LSEH_body_crypton_sha256_asm_block_data_order:++	leaq	(%rsi,%rdx,4),%rdx+	movq	%rdi,64+0(%rsp)+	movq	%rsi,64+8(%rsp)+	movq	%rdx,64+16(%rsp)++	movl	0(%rdi),%eax+	movl	4(%rdi),%ebx+	movl	8(%rdi),%ecx+	movl	12(%rdi),%edx+	movl	16(%rdi),%r8d+	movl	20(%rdi),%r9d+	movl	24(%rdi),%r10d+	movl	28(%rdi),%r11d+	jmp	.Lloop++.p2align	4+.Lloop:+	movl	%ebx,%edi+	leaq	K256(%rip),%rbp+	xorl	%ecx,%edi+	movl	0(%rsi),%r12d+	movl	%r8d,%r13d+	movl	%eax,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r9d,%r15d++	xorl	%r8d,%r13d+	rorl	$9,%r14d+	xorl	%r10d,%r15d++	movl	%r12d,0(%rsp)+	xorl	%eax,%r14d+	andl	%r8d,%r15d++	rorl	$5,%r13d+	addl	%r11d,%r12d+	xorl	%r10d,%r15d++	rorl	$11,%r14d+	xorl	%r8d,%r13d+	addl	%r15d,%r12d++	movl	%eax,%r15d+	addl	(%rbp),%r12d+	xorl	%eax,%r14d++	xorl	%ebx,%r15d+	rorl	$6,%r13d+	movl	%ebx,%r11d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r11d+	addl	%r12d,%edx+	addl	%r12d,%r11d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r11d+	movl	4(%rsi),%r12d+	movl	%edx,%r13d+	movl	%r11d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r8d,%edi++	xorl	%edx,%r13d+	rorl	$9,%r14d+	xorl	%r9d,%edi++	movl	%r12d,4(%rsp)+	xorl	%r11d,%r14d+	andl	%edx,%edi++	rorl	$5,%r13d+	addl	%r10d,%r12d+	xorl	%r9d,%edi++	rorl	$11,%r14d+	xorl	%edx,%r13d+	addl	%edi,%r12d++	movl	%r11d,%edi+	addl	(%rbp),%r12d+	xorl	%r11d,%r14d++	xorl	%eax,%edi+	rorl	$6,%r13d+	movl	%eax,%r10d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r10d+	addl	%r12d,%ecx+	addl	%r12d,%r10d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r10d+	movl	8(%rsi),%r12d+	movl	%ecx,%r13d+	movl	%r10d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%edx,%r15d++	xorl	%ecx,%r13d+	rorl	$9,%r14d+	xorl	%r8d,%r15d++	movl	%r12d,8(%rsp)+	xorl	%r10d,%r14d+	andl	%ecx,%r15d++	rorl	$5,%r13d+	addl	%r9d,%r12d+	xorl	%r8d,%r15d++	rorl	$11,%r14d+	xorl	%ecx,%r13d+	addl	%r15d,%r12d++	movl	%r10d,%r15d+	addl	(%rbp),%r12d+	xorl	%r10d,%r14d++	xorl	%r11d,%r15d+	rorl	$6,%r13d+	movl	%r11d,%r9d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r9d+	addl	%r12d,%ebx+	addl	%r12d,%r9d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r9d+	movl	12(%rsi),%r12d+	movl	%ebx,%r13d+	movl	%r9d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%ecx,%edi++	xorl	%ebx,%r13d+	rorl	$9,%r14d+	xorl	%edx,%edi++	movl	%r12d,12(%rsp)+	xorl	%r9d,%r14d+	andl	%ebx,%edi++	rorl	$5,%r13d+	addl	%r8d,%r12d+	xorl	%edx,%edi++	rorl	$11,%r14d+	xorl	%ebx,%r13d+	addl	%edi,%r12d++	movl	%r9d,%edi+	addl	(%rbp),%r12d+	xorl	%r9d,%r14d++	xorl	%r10d,%edi+	rorl	$6,%r13d+	movl	%r10d,%r8d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r8d+	addl	%r12d,%eax+	addl	%r12d,%r8d++	leaq	20(%rbp),%rbp+	addl	%r14d,%r8d+	movl	16(%rsi),%r12d+	movl	%eax,%r13d+	movl	%r8d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%ebx,%r15d++	xorl	%eax,%r13d+	rorl	$9,%r14d+	xorl	%ecx,%r15d++	movl	%r12d,16(%rsp)+	xorl	%r8d,%r14d+	andl	%eax,%r15d++	rorl	$5,%r13d+	addl	%edx,%r12d+	xorl	%ecx,%r15d++	rorl	$11,%r14d+	xorl	%eax,%r13d+	addl	%r15d,%r12d++	movl	%r8d,%r15d+	addl	(%rbp),%r12d+	xorl	%r8d,%r14d++	xorl	%r9d,%r15d+	rorl	$6,%r13d+	movl	%r9d,%edx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%edx+	addl	%r12d,%r11d+	addl	%r12d,%edx++	leaq	4(%rbp),%rbp+	addl	%r14d,%edx+	movl	20(%rsi),%r12d+	movl	%r11d,%r13d+	movl	%edx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%eax,%edi++	xorl	%r11d,%r13d+	rorl	$9,%r14d+	xorl	%ebx,%edi++	movl	%r12d,20(%rsp)+	xorl	%edx,%r14d+	andl	%r11d,%edi++	rorl	$5,%r13d+	addl	%ecx,%r12d+	xorl	%ebx,%edi++	rorl	$11,%r14d+	xorl	%r11d,%r13d+	addl	%edi,%r12d++	movl	%edx,%edi+	addl	(%rbp),%r12d+	xorl	%edx,%r14d++	xorl	%r8d,%edi+	rorl	$6,%r13d+	movl	%r8d,%ecx++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%ecx+	addl	%r12d,%r10d+	addl	%r12d,%ecx++	leaq	4(%rbp),%rbp+	addl	%r14d,%ecx+	movl	24(%rsi),%r12d+	movl	%r10d,%r13d+	movl	%ecx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r11d,%r15d++	xorl	%r10d,%r13d+	rorl	$9,%r14d+	xorl	%eax,%r15d++	movl	%r12d,24(%rsp)+	xorl	%ecx,%r14d+	andl	%r10d,%r15d++	rorl	$5,%r13d+	addl	%ebx,%r12d+	xorl	%eax,%r15d++	rorl	$11,%r14d+	xorl	%r10d,%r13d+	addl	%r15d,%r12d++	movl	%ecx,%r15d+	addl	(%rbp),%r12d+	xorl	%ecx,%r14d++	xorl	%edx,%r15d+	rorl	$6,%r13d+	movl	%edx,%ebx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%ebx+	addl	%r12d,%r9d+	addl	%r12d,%ebx++	leaq	4(%rbp),%rbp+	addl	%r14d,%ebx+	movl	28(%rsi),%r12d+	movl	%r9d,%r13d+	movl	%ebx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r10d,%edi++	xorl	%r9d,%r13d+	rorl	$9,%r14d+	xorl	%r11d,%edi++	movl	%r12d,28(%rsp)+	xorl	%ebx,%r14d+	andl	%r9d,%edi++	rorl	$5,%r13d+	addl	%eax,%r12d+	xorl	%r11d,%edi++	rorl	$11,%r14d+	xorl	%r9d,%r13d+	addl	%edi,%r12d++	movl	%ebx,%edi+	addl	(%rbp),%r12d+	xorl	%ebx,%r14d++	xorl	%ecx,%edi+	rorl	$6,%r13d+	movl	%ecx,%eax++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%eax+	addl	%r12d,%r8d+	addl	%r12d,%eax++	leaq	20(%rbp),%rbp+	addl	%r14d,%eax+	movl	32(%rsi),%r12d+	movl	%r8d,%r13d+	movl	%eax,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r9d,%r15d++	xorl	%r8d,%r13d+	rorl	$9,%r14d+	xorl	%r10d,%r15d++	movl	%r12d,32(%rsp)+	xorl	%eax,%r14d+	andl	%r8d,%r15d++	rorl	$5,%r13d+	addl	%r11d,%r12d+	xorl	%r10d,%r15d++	rorl	$11,%r14d+	xorl	%r8d,%r13d+	addl	%r15d,%r12d++	movl	%eax,%r15d+	addl	(%rbp),%r12d+	xorl	%eax,%r14d++	xorl	%ebx,%r15d+	rorl	$6,%r13d+	movl	%ebx,%r11d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r11d+	addl	%r12d,%edx+	addl	%r12d,%r11d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r11d+	movl	36(%rsi),%r12d+	movl	%edx,%r13d+	movl	%r11d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r8d,%edi++	xorl	%edx,%r13d+	rorl	$9,%r14d+	xorl	%r9d,%edi++	movl	%r12d,36(%rsp)+	xorl	%r11d,%r14d+	andl	%edx,%edi++	rorl	$5,%r13d+	addl	%r10d,%r12d+	xorl	%r9d,%edi++	rorl	$11,%r14d+	xorl	%edx,%r13d+	addl	%edi,%r12d++	movl	%r11d,%edi+	addl	(%rbp),%r12d+	xorl	%r11d,%r14d++	xorl	%eax,%edi+	rorl	$6,%r13d+	movl	%eax,%r10d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r10d+	addl	%r12d,%ecx+	addl	%r12d,%r10d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r10d+	movl	40(%rsi),%r12d+	movl	%ecx,%r13d+	movl	%r10d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%edx,%r15d++	xorl	%ecx,%r13d+	rorl	$9,%r14d+	xorl	%r8d,%r15d++	movl	%r12d,40(%rsp)+	xorl	%r10d,%r14d+	andl	%ecx,%r15d++	rorl	$5,%r13d+	addl	%r9d,%r12d+	xorl	%r8d,%r15d++	rorl	$11,%r14d+	xorl	%ecx,%r13d+	addl	%r15d,%r12d++	movl	%r10d,%r15d+	addl	(%rbp),%r12d+	xorl	%r10d,%r14d++	xorl	%r11d,%r15d+	rorl	$6,%r13d+	movl	%r11d,%r9d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r9d+	addl	%r12d,%ebx+	addl	%r12d,%r9d++	leaq	4(%rbp),%rbp+	addl	%r14d,%r9d+	movl	44(%rsi),%r12d+	movl	%ebx,%r13d+	movl	%r9d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%ecx,%edi++	xorl	%ebx,%r13d+	rorl	$9,%r14d+	xorl	%edx,%edi++	movl	%r12d,44(%rsp)+	xorl	%r9d,%r14d+	andl	%ebx,%edi++	rorl	$5,%r13d+	addl	%r8d,%r12d+	xorl	%edx,%edi++	rorl	$11,%r14d+	xorl	%ebx,%r13d+	addl	%edi,%r12d++	movl	%r9d,%edi+	addl	(%rbp),%r12d+	xorl	%r9d,%r14d++	xorl	%r10d,%edi+	rorl	$6,%r13d+	movl	%r10d,%r8d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r8d+	addl	%r12d,%eax+	addl	%r12d,%r8d++	leaq	20(%rbp),%rbp+	addl	%r14d,%r8d+	movl	48(%rsi),%r12d+	movl	%eax,%r13d+	movl	%r8d,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%ebx,%r15d++	xorl	%eax,%r13d+	rorl	$9,%r14d+	xorl	%ecx,%r15d++	movl	%r12d,48(%rsp)+	xorl	%r8d,%r14d+	andl	%eax,%r15d++	rorl	$5,%r13d+	addl	%edx,%r12d+	xorl	%ecx,%r15d++	rorl	$11,%r14d+	xorl	%eax,%r13d+	addl	%r15d,%r12d++	movl	%r8d,%r15d+	addl	(%rbp),%r12d+	xorl	%r8d,%r14d++	xorl	%r9d,%r15d+	rorl	$6,%r13d+	movl	%r9d,%edx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%edx+	addl	%r12d,%r11d+	addl	%r12d,%edx++	leaq	4(%rbp),%rbp+	addl	%r14d,%edx+	movl	52(%rsi),%r12d+	movl	%r11d,%r13d+	movl	%edx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%eax,%edi++	xorl	%r11d,%r13d+	rorl	$9,%r14d+	xorl	%ebx,%edi++	movl	%r12d,52(%rsp)+	xorl	%edx,%r14d+	andl	%r11d,%edi++	rorl	$5,%r13d+	addl	%ecx,%r12d+	xorl	%ebx,%edi++	rorl	$11,%r14d+	xorl	%r11d,%r13d+	addl	%edi,%r12d++	movl	%edx,%edi+	addl	(%rbp),%r12d+	xorl	%edx,%r14d++	xorl	%r8d,%edi+	rorl	$6,%r13d+	movl	%r8d,%ecx++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%ecx+	addl	%r12d,%r10d+	addl	%r12d,%ecx++	leaq	4(%rbp),%rbp+	addl	%r14d,%ecx+	movl	56(%rsi),%r12d+	movl	%r10d,%r13d+	movl	%ecx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r11d,%r15d++	xorl	%r10d,%r13d+	rorl	$9,%r14d+	xorl	%eax,%r15d++	movl	%r12d,56(%rsp)+	xorl	%ecx,%r14d+	andl	%r10d,%r15d++	rorl	$5,%r13d+	addl	%ebx,%r12d+	xorl	%eax,%r15d++	rorl	$11,%r14d+	xorl	%r10d,%r13d+	addl	%r15d,%r12d++	movl	%ecx,%r15d+	addl	(%rbp),%r12d+	xorl	%ecx,%r14d++	xorl	%edx,%r15d+	rorl	$6,%r13d+	movl	%edx,%ebx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%ebx+	addl	%r12d,%r9d+	addl	%r12d,%ebx++	leaq	4(%rbp),%rbp+	addl	%r14d,%ebx+	movl	60(%rsi),%r12d+	movl	%r9d,%r13d+	movl	%ebx,%r14d+	bswapl	%r12d+	rorl	$14,%r13d+	movl	%r10d,%edi++	xorl	%r9d,%r13d+	rorl	$9,%r14d+	xorl	%r11d,%edi++	movl	%r12d,60(%rsp)+	xorl	%ebx,%r14d+	andl	%r9d,%edi++	rorl	$5,%r13d+	addl	%eax,%r12d+	xorl	%r11d,%edi++	rorl	$11,%r14d+	xorl	%r9d,%r13d+	addl	%edi,%r12d++	movl	%ebx,%edi+	addl	(%rbp),%r12d+	xorl	%ebx,%r14d++	xorl	%ecx,%edi+	rorl	$6,%r13d+	movl	%ecx,%eax++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%eax+	addl	%r12d,%r8d+	addl	%r12d,%eax++	leaq	20(%rbp),%rbp+	jmp	.Lrounds_16_xx+.p2align	4+.Lrounds_16_xx:+	movl	4(%rsp),%r13d+	movl	56(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%eax+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	36(%rsp),%r12d++	addl	0(%rsp),%r12d+	movl	%r8d,%r13d+	addl	%r15d,%r12d+	movl	%eax,%r14d+	rorl	$14,%r13d+	movl	%r9d,%r15d++	xorl	%r8d,%r13d+	rorl	$9,%r14d+	xorl	%r10d,%r15d++	movl	%r12d,0(%rsp)+	xorl	%eax,%r14d+	andl	%r8d,%r15d++	rorl	$5,%r13d+	addl	%r11d,%r12d+	xorl	%r10d,%r15d++	rorl	$11,%r14d+	xorl	%r8d,%r13d+	addl	%r15d,%r12d++	movl	%eax,%r15d+	addl	(%rbp),%r12d+	xorl	%eax,%r14d++	xorl	%ebx,%r15d+	rorl	$6,%r13d+	movl	%ebx,%r11d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r11d+	addl	%r12d,%edx+	addl	%r12d,%r11d++	leaq	4(%rbp),%rbp+	movl	8(%rsp),%r13d+	movl	60(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r11d+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	40(%rsp),%r12d++	addl	4(%rsp),%r12d+	movl	%edx,%r13d+	addl	%edi,%r12d+	movl	%r11d,%r14d+	rorl	$14,%r13d+	movl	%r8d,%edi++	xorl	%edx,%r13d+	rorl	$9,%r14d+	xorl	%r9d,%edi++	movl	%r12d,4(%rsp)+	xorl	%r11d,%r14d+	andl	%edx,%edi++	rorl	$5,%r13d+	addl	%r10d,%r12d+	xorl	%r9d,%edi++	rorl	$11,%r14d+	xorl	%edx,%r13d+	addl	%edi,%r12d++	movl	%r11d,%edi+	addl	(%rbp),%r12d+	xorl	%r11d,%r14d++	xorl	%eax,%edi+	rorl	$6,%r13d+	movl	%eax,%r10d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r10d+	addl	%r12d,%ecx+	addl	%r12d,%r10d++	leaq	4(%rbp),%rbp+	movl	12(%rsp),%r13d+	movl	0(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r10d+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	44(%rsp),%r12d++	addl	8(%rsp),%r12d+	movl	%ecx,%r13d+	addl	%r15d,%r12d+	movl	%r10d,%r14d+	rorl	$14,%r13d+	movl	%edx,%r15d++	xorl	%ecx,%r13d+	rorl	$9,%r14d+	xorl	%r8d,%r15d++	movl	%r12d,8(%rsp)+	xorl	%r10d,%r14d+	andl	%ecx,%r15d++	rorl	$5,%r13d+	addl	%r9d,%r12d+	xorl	%r8d,%r15d++	rorl	$11,%r14d+	xorl	%ecx,%r13d+	addl	%r15d,%r12d++	movl	%r10d,%r15d+	addl	(%rbp),%r12d+	xorl	%r10d,%r14d++	xorl	%r11d,%r15d+	rorl	$6,%r13d+	movl	%r11d,%r9d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r9d+	addl	%r12d,%ebx+	addl	%r12d,%r9d++	leaq	4(%rbp),%rbp+	movl	16(%rsp),%r13d+	movl	4(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r9d+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	48(%rsp),%r12d++	addl	12(%rsp),%r12d+	movl	%ebx,%r13d+	addl	%edi,%r12d+	movl	%r9d,%r14d+	rorl	$14,%r13d+	movl	%ecx,%edi++	xorl	%ebx,%r13d+	rorl	$9,%r14d+	xorl	%edx,%edi++	movl	%r12d,12(%rsp)+	xorl	%r9d,%r14d+	andl	%ebx,%edi++	rorl	$5,%r13d+	addl	%r8d,%r12d+	xorl	%edx,%edi++	rorl	$11,%r14d+	xorl	%ebx,%r13d+	addl	%edi,%r12d++	movl	%r9d,%edi+	addl	(%rbp),%r12d+	xorl	%r9d,%r14d++	xorl	%r10d,%edi+	rorl	$6,%r13d+	movl	%r10d,%r8d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r8d+	addl	%r12d,%eax+	addl	%r12d,%r8d++	leaq	20(%rbp),%rbp+	movl	20(%rsp),%r13d+	movl	8(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r8d+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	52(%rsp),%r12d++	addl	16(%rsp),%r12d+	movl	%eax,%r13d+	addl	%r15d,%r12d+	movl	%r8d,%r14d+	rorl	$14,%r13d+	movl	%ebx,%r15d++	xorl	%eax,%r13d+	rorl	$9,%r14d+	xorl	%ecx,%r15d++	movl	%r12d,16(%rsp)+	xorl	%r8d,%r14d+	andl	%eax,%r15d++	rorl	$5,%r13d+	addl	%edx,%r12d+	xorl	%ecx,%r15d++	rorl	$11,%r14d+	xorl	%eax,%r13d+	addl	%r15d,%r12d++	movl	%r8d,%r15d+	addl	(%rbp),%r12d+	xorl	%r8d,%r14d++	xorl	%r9d,%r15d+	rorl	$6,%r13d+	movl	%r9d,%edx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%edx+	addl	%r12d,%r11d+	addl	%r12d,%edx++	leaq	4(%rbp),%rbp+	movl	24(%rsp),%r13d+	movl	12(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%edx+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	56(%rsp),%r12d++	addl	20(%rsp),%r12d+	movl	%r11d,%r13d+	addl	%edi,%r12d+	movl	%edx,%r14d+	rorl	$14,%r13d+	movl	%eax,%edi++	xorl	%r11d,%r13d+	rorl	$9,%r14d+	xorl	%ebx,%edi++	movl	%r12d,20(%rsp)+	xorl	%edx,%r14d+	andl	%r11d,%edi++	rorl	$5,%r13d+	addl	%ecx,%r12d+	xorl	%ebx,%edi++	rorl	$11,%r14d+	xorl	%r11d,%r13d+	addl	%edi,%r12d++	movl	%edx,%edi+	addl	(%rbp),%r12d+	xorl	%edx,%r14d++	xorl	%r8d,%edi+	rorl	$6,%r13d+	movl	%r8d,%ecx++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%ecx+	addl	%r12d,%r10d+	addl	%r12d,%ecx++	leaq	4(%rbp),%rbp+	movl	28(%rsp),%r13d+	movl	16(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%ecx+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	60(%rsp),%r12d++	addl	24(%rsp),%r12d+	movl	%r10d,%r13d+	addl	%r15d,%r12d+	movl	%ecx,%r14d+	rorl	$14,%r13d+	movl	%r11d,%r15d++	xorl	%r10d,%r13d+	rorl	$9,%r14d+	xorl	%eax,%r15d++	movl	%r12d,24(%rsp)+	xorl	%ecx,%r14d+	andl	%r10d,%r15d++	rorl	$5,%r13d+	addl	%ebx,%r12d+	xorl	%eax,%r15d++	rorl	$11,%r14d+	xorl	%r10d,%r13d+	addl	%r15d,%r12d++	movl	%ecx,%r15d+	addl	(%rbp),%r12d+	xorl	%ecx,%r14d++	xorl	%edx,%r15d+	rorl	$6,%r13d+	movl	%edx,%ebx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%ebx+	addl	%r12d,%r9d+	addl	%r12d,%ebx++	leaq	4(%rbp),%rbp+	movl	32(%rsp),%r13d+	movl	20(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%ebx+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	0(%rsp),%r12d++	addl	28(%rsp),%r12d+	movl	%r9d,%r13d+	addl	%edi,%r12d+	movl	%ebx,%r14d+	rorl	$14,%r13d+	movl	%r10d,%edi++	xorl	%r9d,%r13d+	rorl	$9,%r14d+	xorl	%r11d,%edi++	movl	%r12d,28(%rsp)+	xorl	%ebx,%r14d+	andl	%r9d,%edi++	rorl	$5,%r13d+	addl	%eax,%r12d+	xorl	%r11d,%edi++	rorl	$11,%r14d+	xorl	%r9d,%r13d+	addl	%edi,%r12d++	movl	%ebx,%edi+	addl	(%rbp),%r12d+	xorl	%ebx,%r14d++	xorl	%ecx,%edi+	rorl	$6,%r13d+	movl	%ecx,%eax++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%eax+	addl	%r12d,%r8d+	addl	%r12d,%eax++	leaq	20(%rbp),%rbp+	movl	36(%rsp),%r13d+	movl	24(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%eax+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	4(%rsp),%r12d++	addl	32(%rsp),%r12d+	movl	%r8d,%r13d+	addl	%r15d,%r12d+	movl	%eax,%r14d+	rorl	$14,%r13d+	movl	%r9d,%r15d++	xorl	%r8d,%r13d+	rorl	$9,%r14d+	xorl	%r10d,%r15d++	movl	%r12d,32(%rsp)+	xorl	%eax,%r14d+	andl	%r8d,%r15d++	rorl	$5,%r13d+	addl	%r11d,%r12d+	xorl	%r10d,%r15d++	rorl	$11,%r14d+	xorl	%r8d,%r13d+	addl	%r15d,%r12d++	movl	%eax,%r15d+	addl	(%rbp),%r12d+	xorl	%eax,%r14d++	xorl	%ebx,%r15d+	rorl	$6,%r13d+	movl	%ebx,%r11d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r11d+	addl	%r12d,%edx+	addl	%r12d,%r11d++	leaq	4(%rbp),%rbp+	movl	40(%rsp),%r13d+	movl	28(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r11d+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	8(%rsp),%r12d++	addl	36(%rsp),%r12d+	movl	%edx,%r13d+	addl	%edi,%r12d+	movl	%r11d,%r14d+	rorl	$14,%r13d+	movl	%r8d,%edi++	xorl	%edx,%r13d+	rorl	$9,%r14d+	xorl	%r9d,%edi++	movl	%r12d,36(%rsp)+	xorl	%r11d,%r14d+	andl	%edx,%edi++	rorl	$5,%r13d+	addl	%r10d,%r12d+	xorl	%r9d,%edi++	rorl	$11,%r14d+	xorl	%edx,%r13d+	addl	%edi,%r12d++	movl	%r11d,%edi+	addl	(%rbp),%r12d+	xorl	%r11d,%r14d++	xorl	%eax,%edi+	rorl	$6,%r13d+	movl	%eax,%r10d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r10d+	addl	%r12d,%ecx+	addl	%r12d,%r10d++	leaq	4(%rbp),%rbp+	movl	44(%rsp),%r13d+	movl	32(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r10d+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	12(%rsp),%r12d++	addl	40(%rsp),%r12d+	movl	%ecx,%r13d+	addl	%r15d,%r12d+	movl	%r10d,%r14d+	rorl	$14,%r13d+	movl	%edx,%r15d++	xorl	%ecx,%r13d+	rorl	$9,%r14d+	xorl	%r8d,%r15d++	movl	%r12d,40(%rsp)+	xorl	%r10d,%r14d+	andl	%ecx,%r15d++	rorl	$5,%r13d+	addl	%r9d,%r12d+	xorl	%r8d,%r15d++	rorl	$11,%r14d+	xorl	%ecx,%r13d+	addl	%r15d,%r12d++	movl	%r10d,%r15d+	addl	(%rbp),%r12d+	xorl	%r10d,%r14d++	xorl	%r11d,%r15d+	rorl	$6,%r13d+	movl	%r11d,%r9d++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%r9d+	addl	%r12d,%ebx+	addl	%r12d,%r9d++	leaq	4(%rbp),%rbp+	movl	48(%rsp),%r13d+	movl	36(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r9d+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	16(%rsp),%r12d++	addl	44(%rsp),%r12d+	movl	%ebx,%r13d+	addl	%edi,%r12d+	movl	%r9d,%r14d+	rorl	$14,%r13d+	movl	%ecx,%edi++	xorl	%ebx,%r13d+	rorl	$9,%r14d+	xorl	%edx,%edi++	movl	%r12d,44(%rsp)+	xorl	%r9d,%r14d+	andl	%ebx,%edi++	rorl	$5,%r13d+	addl	%r8d,%r12d+	xorl	%edx,%edi++	rorl	$11,%r14d+	xorl	%ebx,%r13d+	addl	%edi,%r12d++	movl	%r9d,%edi+	addl	(%rbp),%r12d+	xorl	%r9d,%r14d++	xorl	%r10d,%edi+	rorl	$6,%r13d+	movl	%r10d,%r8d++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%r8d+	addl	%r12d,%eax+	addl	%r12d,%r8d++	leaq	20(%rbp),%rbp+	movl	52(%rsp),%r13d+	movl	40(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%r8d+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	20(%rsp),%r12d++	addl	48(%rsp),%r12d+	movl	%eax,%r13d+	addl	%r15d,%r12d+	movl	%r8d,%r14d+	rorl	$14,%r13d+	movl	%ebx,%r15d++	xorl	%eax,%r13d+	rorl	$9,%r14d+	xorl	%ecx,%r15d++	movl	%r12d,48(%rsp)+	xorl	%r8d,%r14d+	andl	%eax,%r15d++	rorl	$5,%r13d+	addl	%edx,%r12d+	xorl	%ecx,%r15d++	rorl	$11,%r14d+	xorl	%eax,%r13d+	addl	%r15d,%r12d++	movl	%r8d,%r15d+	addl	(%rbp),%r12d+	xorl	%r8d,%r14d++	xorl	%r9d,%r15d+	rorl	$6,%r13d+	movl	%r9d,%edx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%edx+	addl	%r12d,%r11d+	addl	%r12d,%edx++	leaq	4(%rbp),%rbp+	movl	56(%rsp),%r13d+	movl	44(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%edx+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	24(%rsp),%r12d++	addl	52(%rsp),%r12d+	movl	%r11d,%r13d+	addl	%edi,%r12d+	movl	%edx,%r14d+	rorl	$14,%r13d+	movl	%eax,%edi++	xorl	%r11d,%r13d+	rorl	$9,%r14d+	xorl	%ebx,%edi++	movl	%r12d,52(%rsp)+	xorl	%edx,%r14d+	andl	%r11d,%edi++	rorl	$5,%r13d+	addl	%ecx,%r12d+	xorl	%ebx,%edi++	rorl	$11,%r14d+	xorl	%r11d,%r13d+	addl	%edi,%r12d++	movl	%edx,%edi+	addl	(%rbp),%r12d+	xorl	%edx,%r14d++	xorl	%r8d,%edi+	rorl	$6,%r13d+	movl	%r8d,%ecx++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%ecx+	addl	%r12d,%r10d+	addl	%r12d,%ecx++	leaq	4(%rbp),%rbp+	movl	60(%rsp),%r13d+	movl	48(%rsp),%r15d++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%ecx+	movl	%r15d,%r14d+	rorl	$2,%r15d++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%r15d+	shrl	$10,%r14d++	rorl	$17,%r15d+	xorl	%r13d,%r12d+	xorl	%r14d,%r15d+	addl	28(%rsp),%r12d++	addl	56(%rsp),%r12d+	movl	%r10d,%r13d+	addl	%r15d,%r12d+	movl	%ecx,%r14d+	rorl	$14,%r13d+	movl	%r11d,%r15d++	xorl	%r10d,%r13d+	rorl	$9,%r14d+	xorl	%eax,%r15d++	movl	%r12d,56(%rsp)+	xorl	%ecx,%r14d+	andl	%r10d,%r15d++	rorl	$5,%r13d+	addl	%ebx,%r12d+	xorl	%eax,%r15d++	rorl	$11,%r14d+	xorl	%r10d,%r13d+	addl	%r15d,%r12d++	movl	%ecx,%r15d+	addl	(%rbp),%r12d+	xorl	%ecx,%r14d++	xorl	%edx,%r15d+	rorl	$6,%r13d+	movl	%edx,%ebx++	andl	%r15d,%edi+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%edi,%ebx+	addl	%r12d,%r9d+	addl	%r12d,%ebx++	leaq	4(%rbp),%rbp+	movl	0(%rsp),%r13d+	movl	52(%rsp),%edi++	movl	%r13d,%r12d+	rorl	$11,%r13d+	addl	%r14d,%ebx+	movl	%edi,%r14d+	rorl	$2,%edi++	xorl	%r12d,%r13d+	shrl	$3,%r12d+	rorl	$7,%r13d+	xorl	%r14d,%edi+	shrl	$10,%r14d++	rorl	$17,%edi+	xorl	%r13d,%r12d+	xorl	%r14d,%edi+	addl	32(%rsp),%r12d++	addl	60(%rsp),%r12d+	movl	%r9d,%r13d+	addl	%edi,%r12d+	movl	%ebx,%r14d+	rorl	$14,%r13d+	movl	%r10d,%edi++	xorl	%r9d,%r13d+	rorl	$9,%r14d+	xorl	%r11d,%edi++	movl	%r12d,60(%rsp)+	xorl	%ebx,%r14d+	andl	%r9d,%edi++	rorl	$5,%r13d+	addl	%eax,%r12d+	xorl	%r11d,%edi++	rorl	$11,%r14d+	xorl	%r9d,%r13d+	addl	%edi,%r12d++	movl	%ebx,%edi+	addl	(%rbp),%r12d+	xorl	%ebx,%r14d++	xorl	%ecx,%edi+	rorl	$6,%r13d+	movl	%ecx,%eax++	andl	%edi,%r15d+	rorl	$2,%r14d+	addl	%r13d,%r12d++	xorl	%r15d,%eax+	addl	%r12d,%r8d+	addl	%r12d,%eax++	leaq	20(%rbp),%rbp+	cmpb	$0,3(%rbp)+	jnz	.Lrounds_16_xx++	movq	64+0(%rsp),%rdi+	addl	%r14d,%eax+	leaq	64(%rsi),%rsi++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	addl	24(%rdi),%r10d+	addl	28(%rdi),%r11d++	cmpq	64+16(%rsp),%rsi++	movl	%eax,0(%rdi)+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)+	jb	.Lloop++	leaq	64+24+48(%rsp),%r11++	movq	64+24(%rsp),%r15+	movq	-40(%r11),%r14+	movq	-32(%r11),%r13+	movq	-24(%r11),%r12+	movq	-16(%r11),%rbx+	movq	-8(%r11),%rbp+.LSEH_epilogue_crypton_sha256_asm_block_data_order:+	mov	8(%r11),%rdi+	mov	16(%r11),%rsi++	leaq	(%r11),%rsp+	.byte	0xf3,0xc3++.LSEH_end_crypton_sha256_asm_block_data_order:+.p2align	6++K256:+.long	0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5+.long	0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5+.long	0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5+.long	0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5+.long	0xd807aa98,0x12835b01,0x243185be,0x550c7dc3+.long	0xd807aa98,0x12835b01,0x243185be,0x550c7dc3+.long	0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174+.long	0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174+.long	0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc+.long	0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc+.long	0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da+.long	0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da+.long	0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7+.long	0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7+.long	0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967+.long	0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967+.long	0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13+.long	0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13+.long	0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85+.long	0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85+.long	0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3+.long	0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3+.long	0xd192e819,0xd6990624,0xf40e3585,0x106aa070+.long	0xd192e819,0xd6990624,0xf40e3585,0x106aa070+.long	0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5+.long	0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5+.long	0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3+.long	0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3+.long	0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208+.long	0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208+.long	0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2+.long	0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2++.long	0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f+.long	0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f+.long	0x03020100,0x0b0a0908,0xffffffff,0xffffffff+.long	0x03020100,0x0b0a0908,0xffffffff,0xffffffff+.long	0xffffffff,0xffffffff,0x03020100,0x0b0a0908+.long	0xffffffff,0xffffffff,0x03020100,0x0b0a0908+.byte	83,72,65,50,53,54,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.def	crypton_sha256_asm_block_data_order_shaext;	.scl 3;	.type 32;	.endef+.p2align	6+crypton_sha256_asm_block_data_order_shaext:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_sha256_asm_block_data_order_shaext:+++	pushq	%rbp++	movq	%rsp,%rbp++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+.Lshaext_shortcut:+	subq	$0x50,%rsp++	movaps	%xmm6,-80(%rbp)+	movaps	%xmm7,-64(%rbp)+	movaps	%xmm8,-48(%rbp)+	movaps	%xmm9,-32(%rbp)+	movaps	%xmm10,-16(%rbp)++.LSEH_body_crypton_sha256_asm_block_data_order_shaext:++	leaq	K256+128(%rip),%rcx+	movdqu	(%rdi),%xmm1+	movdqu	16(%rdi),%xmm2+	movdqa	512-128(%rcx),%xmm7++	pshufd	$0x1b,%xmm1,%xmm0+	pshufd	$0xb1,%xmm1,%xmm1+	pshufd	$0x1b,%xmm2,%xmm2+	movdqa	%xmm7,%xmm8+.byte	102,15,58,15,202,8+	punpcklqdq	%xmm0,%xmm2+	jmp	.Loop_shaext++.p2align	4+.Loop_shaext:+	movdqu	(%rsi),%xmm3+	movdqu	16(%rsi),%xmm4+	movdqu	32(%rsi),%xmm5+.byte	102,15,56,0,223+	movdqu	48(%rsi),%xmm6++	movdqa	0-128(%rcx),%xmm0+	paddd	%xmm3,%xmm0+.byte	102,15,56,0,231+	movdqa	%xmm2,%xmm10+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	nop+	movdqa	%xmm1,%xmm9+.byte	15,56,203,202++	movdqa	32-128(%rcx),%xmm0+	paddd	%xmm4,%xmm0+.byte	102,15,56,0,239+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	leaq	64(%rsi),%rsi+.byte	15,56,204,220+.byte	15,56,203,202++	movdqa	64-128(%rcx),%xmm0+	paddd	%xmm5,%xmm0+.byte	102,15,56,0,247+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm6,%xmm7+.byte	102,15,58,15,253,4+	nop+	paddd	%xmm7,%xmm3+.byte	15,56,204,229+.byte	15,56,203,202++	movdqa	96-128(%rcx),%xmm0+	paddd	%xmm6,%xmm0+.byte	15,56,205,222+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm3,%xmm7+.byte	102,15,58,15,254,4+	nop+	paddd	%xmm7,%xmm4+.byte	15,56,204,238+.byte	15,56,203,202+	movdqa	128-128(%rcx),%xmm0+	paddd	%xmm3,%xmm0+.byte	15,56,205,227+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm4,%xmm7+.byte	102,15,58,15,251,4+	nop+	paddd	%xmm7,%xmm5+.byte	15,56,204,243+.byte	15,56,203,202+	movdqa	160-128(%rcx),%xmm0+	paddd	%xmm4,%xmm0+.byte	15,56,205,236+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm5,%xmm7+.byte	102,15,58,15,252,4+	nop+	paddd	%xmm7,%xmm6+.byte	15,56,204,220+.byte	15,56,203,202+	movdqa	192-128(%rcx),%xmm0+	paddd	%xmm5,%xmm0+.byte	15,56,205,245+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm6,%xmm7+.byte	102,15,58,15,253,4+	nop+	paddd	%xmm7,%xmm3+.byte	15,56,204,229+.byte	15,56,203,202+	movdqa	224-128(%rcx),%xmm0+	paddd	%xmm6,%xmm0+.byte	15,56,205,222+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm3,%xmm7+.byte	102,15,58,15,254,4+	nop+	paddd	%xmm7,%xmm4+.byte	15,56,204,238+.byte	15,56,203,202+	movdqa	256-128(%rcx),%xmm0+	paddd	%xmm3,%xmm0+.byte	15,56,205,227+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm4,%xmm7+.byte	102,15,58,15,251,4+	nop+	paddd	%xmm7,%xmm5+.byte	15,56,204,243+.byte	15,56,203,202+	movdqa	288-128(%rcx),%xmm0+	paddd	%xmm4,%xmm0+.byte	15,56,205,236+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm5,%xmm7+.byte	102,15,58,15,252,4+	nop+	paddd	%xmm7,%xmm6+.byte	15,56,204,220+.byte	15,56,203,202+	movdqa	320-128(%rcx),%xmm0+	paddd	%xmm5,%xmm0+.byte	15,56,205,245+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm6,%xmm7+.byte	102,15,58,15,253,4+	nop+	paddd	%xmm7,%xmm3+.byte	15,56,204,229+.byte	15,56,203,202+	movdqa	352-128(%rcx),%xmm0+	paddd	%xmm6,%xmm0+.byte	15,56,205,222+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm3,%xmm7+.byte	102,15,58,15,254,4+	nop+	paddd	%xmm7,%xmm4+.byte	15,56,204,238+.byte	15,56,203,202+	movdqa	384-128(%rcx),%xmm0+	paddd	%xmm3,%xmm0+.byte	15,56,205,227+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm4,%xmm7+.byte	102,15,58,15,251,4+	nop+	paddd	%xmm7,%xmm5+.byte	15,56,204,243+.byte	15,56,203,202+	movdqa	416-128(%rcx),%xmm0+	paddd	%xmm4,%xmm0+.byte	15,56,205,236+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	movdqa	%xmm5,%xmm7+.byte	102,15,58,15,252,4+.byte	15,56,203,202+	paddd	%xmm7,%xmm6++	movdqa	448-128(%rcx),%xmm0+	paddd	%xmm5,%xmm0+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+.byte	15,56,205,245+	movdqa	%xmm8,%xmm7+.byte	15,56,203,202++	movdqa	480-128(%rcx),%xmm0+	paddd	%xmm6,%xmm0+	nop+.byte	15,56,203,209+	pshufd	$0x0e,%xmm0,%xmm0+	decq	%rdx+	nop+.byte	15,56,203,202++	paddd	%xmm10,%xmm2+	paddd	%xmm9,%xmm1+	jnz	.Loop_shaext++	pshufd	$0xb1,%xmm2,%xmm2+	pshufd	$0x1b,%xmm1,%xmm7+	pshufd	$0xb1,%xmm1,%xmm1+	punpckhqdq	%xmm2,%xmm1+.byte	102,15,58,15,215,8++	movdqu	%xmm1,(%rdi)+	movdqu	%xmm2,16(%rdi)+	movaps	-80(%rbp),%xmm6+	movaps	-64(%rbp),%xmm7+	movaps	-48(%rbp),%xmm8+	movaps	-32(%rbp),%xmm9+	movaps	-16(%rbp),%xmm10+	movq	%rbp,%rsp++	popq	%rbp++.LSEH_epilogue_crypton_sha256_asm_block_data_order_shaext:+	mov	8(%rsp),%rdi+	mov	16(%rsp),%rsi++	.byte	0xf3,0xc3++.LSEH_end_crypton_sha256_asm_block_data_order_shaext:+.def	crypton_sha256_asm_block_data_order_ssse3;	.scl 3;	.type 32;	.endef+.p2align	6+crypton_sha256_asm_block_data_order_ssse3:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_sha256_asm_block_data_order_ssse3:+++	pushq	%rbp++	movq	%rsp,%rbp++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+.Lssse3_shortcut:+	pushq	%rbx++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	shlq	$4,%rdx+	subq	$88,%rsp++	leaq	(%rsi,%rdx,4),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+	movaps	%xmm6,-128(%rbp)+	movaps	%xmm7,-112(%rbp)+	movaps	%xmm8,-96(%rbp)+	movaps	%xmm9,-80(%rbp)++.LSEH_body_crypton_sha256_asm_block_data_order_ssse3:+++	leaq	-64(%rsp),%rsp+	movl	0(%rdi),%eax+	andq	$-64,%rsp+	movl	4(%rdi),%ebx+	movl	8(%rdi),%ecx+	movl	12(%rdi),%edx+	movl	16(%rdi),%r8d+	movl	20(%rdi),%r9d+	movl	24(%rdi),%r10d+	movl	28(%rdi),%r11d+++	jmp	.Lloop_ssse3+.p2align	4+.Lloop_ssse3:+	movdqa	K256+512(%rip),%xmm7+	movq	%rsi,-56(%rbp)+	movdqu	0(%rsi),%xmm0+	movdqu	16(%rsi),%xmm1+	movdqu	32(%rsi),%xmm2+.byte	102,15,56,0,199+	movdqu	48(%rsi),%xmm3+	leaq	K256(%rip),%rsi+.byte	102,15,56,0,207+	movdqa	0(%rsi),%xmm4+	movdqa	32(%rsi),%xmm5+.byte	102,15,56,0,215+	paddd	%xmm0,%xmm4+	movdqa	64(%rsi),%xmm6+.byte	102,15,56,0,223+	movdqa	96(%rsi),%xmm7+	paddd	%xmm1,%xmm5+	paddd	%xmm2,%xmm6+	paddd	%xmm3,%xmm7+	movdqa	%xmm4,0(%rsp)+	movl	%eax,%r14d+	movdqa	%xmm5,16(%rsp)+	movl	%ebx,%edi+	movdqa	%xmm6,32(%rsp)+	xorl	%ecx,%edi+	movdqa	%xmm7,48(%rsp)+	movl	%r8d,%r13d+	jmp	.Lssse3_00_47++.p2align	4+.Lssse3_00_47:+	subq	$-128,%rsi+	rorl	$14,%r13d+	movdqa	%xmm1,%xmm4+	movl	%r14d,%eax+	movl	%r9d,%r12d+	movdqa	%xmm3,%xmm7+	rorl	$9,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	rorl	$5,%r13d+	xorl	%eax,%r14d+.byte	102,15,58,15,224,4+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+.byte	102,15,58,15,250,4+	addl	0(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	rorl	$11,%r14d+	movdqa	%xmm4,%xmm5+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	movdqa	%xmm4,%xmm6+	rorl	$6,%r13d+	andl	%r15d,%edi+	psrld	$3,%xmm4+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	paddd	%xmm7,%xmm0+	rorl	$2,%r14d+	addl	%r11d,%edx+	psrld	$7,%xmm6+	addl	%edi,%r11d+	movl	%edx,%r13d+	pshufd	$250,%xmm3,%xmm7+	addl	%r11d,%r14d+	rorl	$14,%r13d+	pslld	$14,%xmm5+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	pxor	%xmm6,%xmm4+	rorl	$9,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	rorl	$5,%r13d+	psrld	$11,%xmm6+	xorl	%r11d,%r14d+	pxor	%xmm5,%xmm4+	andl	%edx,%r12d+	xorl	%edx,%r13d+	pslld	$11,%xmm5+	addl	4(%rsp),%r10d+	movl	%r11d,%edi+	pxor	%xmm6,%xmm4+	xorl	%r9d,%r12d+	rorl	$11,%r14d+	movdqa	%xmm7,%xmm6+	xorl	%eax,%edi+	addl	%r12d,%r10d+	pxor	%xmm5,%xmm4+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	psrld	$10,%xmm7+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	paddd	%xmm4,%xmm0+	rorl	$2,%r14d+	addl	%r10d,%ecx+	psrlq	$17,%xmm6+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	pxor	%xmm6,%xmm7+	rorl	$14,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	rorl	$9,%r14d+	psrlq	$2,%xmm6+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	pxor	%xmm6,%xmm7+	rorl	$5,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	pshufd	$128,%xmm7,%xmm7+	xorl	%ecx,%r13d+	addl	8(%rsp),%r9d+	movl	%r10d,%r15d+	psrldq	$8,%xmm7+	xorl	%r8d,%r12d+	rorl	$11,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	rorl	$6,%r13d+	paddd	%xmm7,%xmm0+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	pshufd	$80,%xmm0,%xmm7+	xorl	%r11d,%edi+	rorl	$2,%r14d+	addl	%r9d,%ebx+	movdqa	%xmm7,%xmm6+	addl	%edi,%r9d+	movl	%ebx,%r13d+	psrld	$10,%xmm7+	addl	%r9d,%r14d+	rorl	$14,%r13d+	psrlq	$17,%xmm6+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	pxor	%xmm6,%xmm7+	rorl	$9,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	rorl	$5,%r13d+	xorl	%r9d,%r14d+	psrlq	$2,%xmm6+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	12(%rsp),%r8d+	pxor	%xmm6,%xmm7+	movl	%r9d,%edi+	xorl	%edx,%r12d+	rorl	$11,%r14d+	pshufd	$8,%xmm7,%xmm7+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	movdqa	0(%rsi),%xmm6+	rorl	$6,%r13d+	andl	%edi,%r15d+	pslldq	$8,%xmm7+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	paddd	%xmm7,%xmm0+	rorl	$2,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	paddd	%xmm0,%xmm6+	movl	%eax,%r13d+	addl	%r8d,%r14d+	movdqa	%xmm6,0(%rsp)+	rorl	$14,%r13d+	movdqa	%xmm2,%xmm4+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	movdqa	%xmm0,%xmm7+	rorl	$9,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	rorl	$5,%r13d+	xorl	%r8d,%r14d+.byte	102,15,58,15,225,4+	andl	%eax,%r12d+	xorl	%eax,%r13d+.byte	102,15,58,15,251,4+	addl	16(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	rorl	$11,%r14d+	movdqa	%xmm4,%xmm5+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	movdqa	%xmm4,%xmm6+	rorl	$6,%r13d+	andl	%r15d,%edi+	psrld	$3,%xmm4+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	paddd	%xmm7,%xmm1+	rorl	$2,%r14d+	addl	%edx,%r11d+	psrld	$7,%xmm6+	addl	%edi,%edx+	movl	%r11d,%r13d+	pshufd	$250,%xmm0,%xmm7+	addl	%edx,%r14d+	rorl	$14,%r13d+	pslld	$14,%xmm5+	movl	%r14d,%edx+	movl	%eax,%r12d+	pxor	%xmm6,%xmm4+	rorl	$9,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	rorl	$5,%r13d+	psrld	$11,%xmm6+	xorl	%edx,%r14d+	pxor	%xmm5,%xmm4+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	pslld	$11,%xmm5+	addl	20(%rsp),%ecx+	movl	%edx,%edi+	pxor	%xmm6,%xmm4+	xorl	%ebx,%r12d+	rorl	$11,%r14d+	movdqa	%xmm7,%xmm6+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	pxor	%xmm5,%xmm4+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	psrld	$10,%xmm7+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	paddd	%xmm4,%xmm1+	rorl	$2,%r14d+	addl	%ecx,%r10d+	psrlq	$17,%xmm6+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	pxor	%xmm6,%xmm7+	rorl	$14,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	rorl	$9,%r14d+	psrlq	$2,%xmm6+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	pxor	%xmm6,%xmm7+	rorl	$5,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	pshufd	$128,%xmm7,%xmm7+	xorl	%r10d,%r13d+	addl	24(%rsp),%ebx+	movl	%ecx,%r15d+	psrldq	$8,%xmm7+	xorl	%eax,%r12d+	rorl	$11,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	rorl	$6,%r13d+	paddd	%xmm7,%xmm1+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	pshufd	$80,%xmm1,%xmm7+	xorl	%edx,%edi+	rorl	$2,%r14d+	addl	%ebx,%r9d+	movdqa	%xmm7,%xmm6+	addl	%edi,%ebx+	movl	%r9d,%r13d+	psrld	$10,%xmm7+	addl	%ebx,%r14d+	rorl	$14,%r13d+	psrlq	$17,%xmm6+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	pxor	%xmm6,%xmm7+	rorl	$9,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	rorl	$5,%r13d+	xorl	%ebx,%r14d+	psrlq	$2,%xmm6+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	28(%rsp),%eax+	pxor	%xmm6,%xmm7+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	rorl	$11,%r14d+	pshufd	$8,%xmm7,%xmm7+	xorl	%ecx,%edi+	addl	%r12d,%eax+	movdqa	32(%rsi),%xmm6+	rorl	$6,%r13d+	andl	%edi,%r15d+	pslldq	$8,%xmm7+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	paddd	%xmm7,%xmm1+	rorl	$2,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	paddd	%xmm1,%xmm6+	movl	%r8d,%r13d+	addl	%eax,%r14d+	movdqa	%xmm6,16(%rsp)+	rorl	$14,%r13d+	movdqa	%xmm3,%xmm4+	movl	%r14d,%eax+	movl	%r9d,%r12d+	movdqa	%xmm1,%xmm7+	rorl	$9,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	rorl	$5,%r13d+	xorl	%eax,%r14d+.byte	102,15,58,15,226,4+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+.byte	102,15,58,15,248,4+	addl	32(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	rorl	$11,%r14d+	movdqa	%xmm4,%xmm5+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	movdqa	%xmm4,%xmm6+	rorl	$6,%r13d+	andl	%r15d,%edi+	psrld	$3,%xmm4+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	paddd	%xmm7,%xmm2+	rorl	$2,%r14d+	addl	%r11d,%edx+	psrld	$7,%xmm6+	addl	%edi,%r11d+	movl	%edx,%r13d+	pshufd	$250,%xmm1,%xmm7+	addl	%r11d,%r14d+	rorl	$14,%r13d+	pslld	$14,%xmm5+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	pxor	%xmm6,%xmm4+	rorl	$9,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	rorl	$5,%r13d+	psrld	$11,%xmm6+	xorl	%r11d,%r14d+	pxor	%xmm5,%xmm4+	andl	%edx,%r12d+	xorl	%edx,%r13d+	pslld	$11,%xmm5+	addl	36(%rsp),%r10d+	movl	%r11d,%edi+	pxor	%xmm6,%xmm4+	xorl	%r9d,%r12d+	rorl	$11,%r14d+	movdqa	%xmm7,%xmm6+	xorl	%eax,%edi+	addl	%r12d,%r10d+	pxor	%xmm5,%xmm4+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	psrld	$10,%xmm7+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	paddd	%xmm4,%xmm2+	rorl	$2,%r14d+	addl	%r10d,%ecx+	psrlq	$17,%xmm6+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	pxor	%xmm6,%xmm7+	rorl	$14,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	rorl	$9,%r14d+	psrlq	$2,%xmm6+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	pxor	%xmm6,%xmm7+	rorl	$5,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	pshufd	$128,%xmm7,%xmm7+	xorl	%ecx,%r13d+	addl	40(%rsp),%r9d+	movl	%r10d,%r15d+	psrldq	$8,%xmm7+	xorl	%r8d,%r12d+	rorl	$11,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	rorl	$6,%r13d+	paddd	%xmm7,%xmm2+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	pshufd	$80,%xmm2,%xmm7+	xorl	%r11d,%edi+	rorl	$2,%r14d+	addl	%r9d,%ebx+	movdqa	%xmm7,%xmm6+	addl	%edi,%r9d+	movl	%ebx,%r13d+	psrld	$10,%xmm7+	addl	%r9d,%r14d+	rorl	$14,%r13d+	psrlq	$17,%xmm6+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	pxor	%xmm6,%xmm7+	rorl	$9,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	rorl	$5,%r13d+	xorl	%r9d,%r14d+	psrlq	$2,%xmm6+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	44(%rsp),%r8d+	pxor	%xmm6,%xmm7+	movl	%r9d,%edi+	xorl	%edx,%r12d+	rorl	$11,%r14d+	pshufd	$8,%xmm7,%xmm7+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	movdqa	64(%rsi),%xmm6+	rorl	$6,%r13d+	andl	%edi,%r15d+	pslldq	$8,%xmm7+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	paddd	%xmm7,%xmm2+	rorl	$2,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	paddd	%xmm2,%xmm6+	movl	%eax,%r13d+	addl	%r8d,%r14d+	movdqa	%xmm6,32(%rsp)+	rorl	$14,%r13d+	movdqa	%xmm0,%xmm4+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	movdqa	%xmm2,%xmm7+	rorl	$9,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	rorl	$5,%r13d+	xorl	%r8d,%r14d+.byte	102,15,58,15,227,4+	andl	%eax,%r12d+	xorl	%eax,%r13d+.byte	102,15,58,15,249,4+	addl	48(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	rorl	$11,%r14d+	movdqa	%xmm4,%xmm5+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	movdqa	%xmm4,%xmm6+	rorl	$6,%r13d+	andl	%r15d,%edi+	psrld	$3,%xmm4+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	paddd	%xmm7,%xmm3+	rorl	$2,%r14d+	addl	%edx,%r11d+	psrld	$7,%xmm6+	addl	%edi,%edx+	movl	%r11d,%r13d+	pshufd	$250,%xmm2,%xmm7+	addl	%edx,%r14d+	rorl	$14,%r13d+	pslld	$14,%xmm5+	movl	%r14d,%edx+	movl	%eax,%r12d+	pxor	%xmm6,%xmm4+	rorl	$9,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	rorl	$5,%r13d+	psrld	$11,%xmm6+	xorl	%edx,%r14d+	pxor	%xmm5,%xmm4+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	pslld	$11,%xmm5+	addl	52(%rsp),%ecx+	movl	%edx,%edi+	pxor	%xmm6,%xmm4+	xorl	%ebx,%r12d+	rorl	$11,%r14d+	movdqa	%xmm7,%xmm6+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	pxor	%xmm5,%xmm4+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	psrld	$10,%xmm7+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	paddd	%xmm4,%xmm3+	rorl	$2,%r14d+	addl	%ecx,%r10d+	psrlq	$17,%xmm6+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	pxor	%xmm6,%xmm7+	rorl	$14,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	rorl	$9,%r14d+	psrlq	$2,%xmm6+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	pxor	%xmm6,%xmm7+	rorl	$5,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	pshufd	$128,%xmm7,%xmm7+	xorl	%r10d,%r13d+	addl	56(%rsp),%ebx+	movl	%ecx,%r15d+	psrldq	$8,%xmm7+	xorl	%eax,%r12d+	rorl	$11,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	rorl	$6,%r13d+	paddd	%xmm7,%xmm3+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	pshufd	$80,%xmm3,%xmm7+	xorl	%edx,%edi+	rorl	$2,%r14d+	addl	%ebx,%r9d+	movdqa	%xmm7,%xmm6+	addl	%edi,%ebx+	movl	%r9d,%r13d+	psrld	$10,%xmm7+	addl	%ebx,%r14d+	rorl	$14,%r13d+	psrlq	$17,%xmm6+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	pxor	%xmm6,%xmm7+	rorl	$9,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	rorl	$5,%r13d+	xorl	%ebx,%r14d+	psrlq	$2,%xmm6+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	60(%rsp),%eax+	pxor	%xmm6,%xmm7+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	rorl	$11,%r14d+	pshufd	$8,%xmm7,%xmm7+	xorl	%ecx,%edi+	addl	%r12d,%eax+	movdqa	96(%rsi),%xmm6+	rorl	$6,%r13d+	andl	%edi,%r15d+	pslldq	$8,%xmm7+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	paddd	%xmm7,%xmm3+	rorl	$2,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	paddd	%xmm3,%xmm6+	movl	%r8d,%r13d+	addl	%eax,%r14d+	movdqa	%xmm6,48(%rsp)+	cmpb	$0,131(%rsi)+	jne	.Lssse3_00_47+	rorl	$14,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	rorl	$9,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	rorl	$5,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+	addl	0(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	rorl	$11,%r14d+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	rorl	$2,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	movl	%edx,%r13d+	addl	%r11d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	rorl	$9,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	rorl	$5,%r13d+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	addl	4(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	rorl	$11,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	rorl	$2,%r14d+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	rorl	$9,%r14d+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	rorl	$5,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	8(%rsp),%r9d+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	rorl	$11,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	xorl	%r11d,%edi+	rorl	$2,%r14d+	addl	%r9d,%ebx+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	rorl	$9,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	rorl	$5,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	12(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	rorl	$11,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	rorl	$2,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	rorl	$9,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	rorl	$5,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	xorl	%eax,%r13d+	addl	16(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	rorl	$11,%r14d+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	rorl	$2,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	movl	%r11d,%r13d+	addl	%edx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%edx+	movl	%eax,%r12d+	rorl	$9,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	rorl	$5,%r13d+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	addl	20(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	rorl	$11,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	rorl	$2,%r14d+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	rorl	$9,%r14d+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	rorl	$5,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	24(%rsp),%ebx+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	rorl	$11,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	xorl	%edx,%edi+	rorl	$2,%r14d+	addl	%ebx,%r9d+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	rorl	$9,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	rorl	$5,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	28(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	rorl	$11,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	rorl	$2,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	rorl	$14,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	rorl	$9,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	rorl	$5,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+	addl	32(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	rorl	$11,%r14d+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	rorl	$2,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	movl	%edx,%r13d+	addl	%r11d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	rorl	$9,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	rorl	$5,%r13d+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	addl	36(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	rorl	$11,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	rorl	$2,%r14d+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	rorl	$9,%r14d+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	rorl	$5,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	40(%rsp),%r9d+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	rorl	$11,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	xorl	%r11d,%edi+	rorl	$2,%r14d+	addl	%r9d,%ebx+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	rorl	$9,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	rorl	$5,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	44(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	rorl	$11,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	rorl	$2,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	rorl	$14,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	rorl	$9,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	rorl	$5,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	xorl	%eax,%r13d+	addl	48(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	rorl	$11,%r14d+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	rorl	$2,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	movl	%r11d,%r13d+	addl	%edx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%edx+	movl	%eax,%r12d+	rorl	$9,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	rorl	$5,%r13d+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	addl	52(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	rorl	$11,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	rorl	$2,%r14d+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	rorl	$9,%r14d+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	rorl	$5,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	56(%rsp),%ebx+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	rorl	$11,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	rorl	$6,%r13d+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	xorl	%edx,%edi+	rorl	$2,%r14d+	addl	%ebx,%r9d+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	rorl	$14,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	rorl	$9,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	rorl	$5,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	60(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	rorl	$11,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	rorl	$6,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	rorl	$2,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	movq	-64(%rbp),%rdi+	movl	%r14d,%eax+	movq	-56(%rbp),%rsi++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	addl	24(%rdi),%r10d+	addl	28(%rdi),%r11d++	leaq	64(%rsi),%rsi+	cmpq	-48(%rbp),%rsi++	movl	%eax,0(%rdi)+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)+	jb	.Lloop_ssse3++	movaps	-128(%rbp),%xmm6+	movaps	-112(%rbp),%xmm7+	movaps	-96(%rbp),%xmm8+	movaps	-80(%rbp),%xmm9+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp++	popq	%rbp++.LSEH_epilogue_crypton_sha256_asm_block_data_order_ssse3:+	mov	8(%rsp),%rdi+	mov	16(%rsp),%rsi++	.byte	0xf3,0xc3++.LSEH_end_crypton_sha256_asm_block_data_order_ssse3:+.def	crypton_sha256_asm_block_data_order_avx;	.scl 3;	.type 32;	.endef+.p2align	6+crypton_sha256_asm_block_data_order_avx:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_sha256_asm_block_data_order_avx:+++	pushq	%rbp++	movq	%rsp,%rbp++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+.Lavx_shortcut:+	pushq	%rbx++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	shlq	$4,%rdx+	subq	$120,%rsp++	leaq	(%rsi,%rdx,4),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+	movaps	%xmm6,-160(%rbp)+	movaps	%xmm7,-144(%rbp)+	movaps	%xmm8,-128(%rbp)+	movaps	%xmm9,-112(%rbp)++.LSEH_body_crypton_sha256_asm_block_data_order_avx:+++	leaq	-64(%rsp),%rsp+	vzeroupper+	andq	$-64,%rsp+	movl	0(%rdi),%eax+	movl	4(%rdi),%ebx+	movl	8(%rdi),%ecx+	movl	12(%rdi),%edx+	movl	16(%rdi),%r8d+	movl	20(%rdi),%r9d+	movl	24(%rdi),%r10d+	movl	28(%rdi),%r11d+	vmovdqa	K256+512+32(%rip),%xmm8+	vmovdqa	K256+512+64(%rip),%xmm9+	jmp	.Lloop_avx+.p2align	4+.Lloop_avx:+	vmovdqa	K256+512(%rip),%xmm7+	movq	%rsi,-56(%rbp)+	vmovdqu	0(%rsi),%xmm0+	vmovdqu	16(%rsi),%xmm1+	vmovdqu	32(%rsi),%xmm2+	vmovdqu	48(%rsi),%xmm3+	vpshufb	%xmm7,%xmm0,%xmm0+	leaq	K256(%rip),%rsi+	vpshufb	%xmm7,%xmm1,%xmm1+	vpshufb	%xmm7,%xmm2,%xmm2+	vpaddd	0(%rsi),%xmm0,%xmm4+	vpshufb	%xmm7,%xmm3,%xmm3+	vpaddd	32(%rsi),%xmm1,%xmm5+	vpaddd	64(%rsi),%xmm2,%xmm6+	vpaddd	96(%rsi),%xmm3,%xmm7+	vmovdqa	%xmm4,0(%rsp)+	movl	%eax,%r14d+	vmovdqa	%xmm5,16(%rsp)+	movl	%ebx,%edi+	vmovdqa	%xmm6,32(%rsp)+	xorl	%ecx,%edi+	vmovdqa	%xmm7,48(%rsp)+	movl	%r8d,%r13d+	jmp	.Lavx_00_47++.p2align	4+.Lavx_00_47:+	subq	$-128,%rsi+	vpalignr	$4,%xmm0,%xmm1,%xmm4+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	vpalignr	$4,%xmm2,%xmm3,%xmm7+	shrdl	$9,%r14d,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	vpsrld	$7,%xmm4,%xmm6+	shrdl	$5,%r13d,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	vpaddd	%xmm7,%xmm0,%xmm0+	xorl	%r8d,%r13d+	addl	0(%rsp),%r11d+	movl	%eax,%r15d+	vpsrld	$3,%xmm4,%xmm7+	xorl	%r10d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ebx,%r15d+	vpslld	$14,%xmm4,%xmm5+	addl	%r12d,%r11d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	vpxor	%xmm6,%xmm7,%xmm4+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	vpshufd	$250,%xmm3,%xmm7+	shrdl	$2,%r14d,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	vpsrld	$11,%xmm6,%xmm6+	movl	%edx,%r13d+	addl	%r11d,%r14d+	shrdl	$14,%r13d,%r13d+	vpxor	%xmm5,%xmm4,%xmm4+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	shrdl	$9,%r14d,%r14d+	vpslld	$11,%xmm5,%xmm5+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	shrdl	$5,%r13d,%r13d+	vpxor	%xmm6,%xmm4,%xmm4+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	vpsrld	$10,%xmm7,%xmm6+	addl	4(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	vpxor	%xmm5,%xmm4,%xmm4+	shrdl	$11,%r14d,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	vpsrlq	$17,%xmm7,%xmm7+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	vpaddd	%xmm4,%xmm0,%xmm0+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	shrdl	$2,%r14d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%r10d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r10d+	vpxor	%xmm7,%xmm6,%xmm6+	movl	%edx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ecx,%r13d+	vpshufb	%xmm8,%xmm6,%xmm6+	xorl	%r8d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r10d,%r14d+	vpaddd	%xmm6,%xmm0,%xmm0+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	8(%rsp),%r9d+	vpshufd	$80,%xmm0,%xmm7+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	shrdl	$11,%r14d,%r14d+	vpsrld	$10,%xmm7,%xmm6+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	shrdl	$6,%r13d,%r13d+	vpsrlq	$17,%xmm7,%xmm7+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	vpxor	%xmm7,%xmm6,%xmm6+	xorl	%r11d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r9d,%ebx+	vpsrlq	$2,%xmm7,%xmm7+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	vpshufb	%xmm9,%xmm6,%xmm6+	shrdl	$9,%r14d,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	vpaddd	%xmm6,%xmm0,%xmm0+	shrdl	$5,%r13d,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	vpaddd	0(%rsi),%xmm0,%xmm6+	xorl	%ebx,%r13d+	addl	12(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	vmovdqa	%xmm6,0(%rsp)+	vpalignr	$4,%xmm1,%xmm2,%xmm4+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	vpalignr	$4,%xmm3,%xmm0,%xmm7+	shrdl	$9,%r14d,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	vpsrld	$7,%xmm4,%xmm6+	shrdl	$5,%r13d,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	vpaddd	%xmm7,%xmm1,%xmm1+	xorl	%eax,%r13d+	addl	16(%rsp),%edx+	movl	%r8d,%r15d+	vpsrld	$3,%xmm4,%xmm7+	xorl	%ecx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r9d,%r15d+	vpslld	$14,%xmm4,%xmm5+	addl	%r12d,%edx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	vpxor	%xmm6,%xmm7,%xmm4+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	vpshufd	$250,%xmm0,%xmm7+	shrdl	$2,%r14d,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	vpsrld	$11,%xmm6,%xmm6+	movl	%r11d,%r13d+	addl	%edx,%r14d+	shrdl	$14,%r13d,%r13d+	vpxor	%xmm5,%xmm4,%xmm4+	movl	%r14d,%edx+	movl	%eax,%r12d+	shrdl	$9,%r14d,%r14d+	vpslld	$11,%xmm5,%xmm5+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	shrdl	$5,%r13d,%r13d+	vpxor	%xmm6,%xmm4,%xmm4+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	vpsrld	$10,%xmm7,%xmm6+	addl	20(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	vpxor	%xmm5,%xmm4,%xmm4+	shrdl	$11,%r14d,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	vpsrlq	$17,%xmm7,%xmm7+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	vpaddd	%xmm4,%xmm1,%xmm1+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	shrdl	$2,%r14d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%ecx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ecx+	vpxor	%xmm7,%xmm6,%xmm6+	movl	%r11d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r10d,%r13d+	vpshufb	%xmm8,%xmm6,%xmm6+	xorl	%eax,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ecx,%r14d+	vpaddd	%xmm6,%xmm1,%xmm1+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	24(%rsp),%ebx+	vpshufd	$80,%xmm1,%xmm7+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	shrdl	$11,%r14d,%r14d+	vpsrld	$10,%xmm7,%xmm6+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	shrdl	$6,%r13d,%r13d+	vpsrlq	$17,%xmm7,%xmm7+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	vpxor	%xmm7,%xmm6,%xmm6+	xorl	%edx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%ebx,%r9d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	vpshufb	%xmm9,%xmm6,%xmm6+	shrdl	$9,%r14d,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	vpaddd	%xmm6,%xmm1,%xmm1+	shrdl	$5,%r13d,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	vpaddd	32(%rsi),%xmm1,%xmm6+	xorl	%r9d,%r13d+	addl	28(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	vmovdqa	%xmm6,16(%rsp)+	vpalignr	$4,%xmm2,%xmm3,%xmm4+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	vpalignr	$4,%xmm0,%xmm1,%xmm7+	shrdl	$9,%r14d,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	vpsrld	$7,%xmm4,%xmm6+	shrdl	$5,%r13d,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	vpaddd	%xmm7,%xmm2,%xmm2+	xorl	%r8d,%r13d+	addl	32(%rsp),%r11d+	movl	%eax,%r15d+	vpsrld	$3,%xmm4,%xmm7+	xorl	%r10d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ebx,%r15d+	vpslld	$14,%xmm4,%xmm5+	addl	%r12d,%r11d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	vpxor	%xmm6,%xmm7,%xmm4+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	vpshufd	$250,%xmm1,%xmm7+	shrdl	$2,%r14d,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	vpsrld	$11,%xmm6,%xmm6+	movl	%edx,%r13d+	addl	%r11d,%r14d+	shrdl	$14,%r13d,%r13d+	vpxor	%xmm5,%xmm4,%xmm4+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	shrdl	$9,%r14d,%r14d+	vpslld	$11,%xmm5,%xmm5+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	shrdl	$5,%r13d,%r13d+	vpxor	%xmm6,%xmm4,%xmm4+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	vpsrld	$10,%xmm7,%xmm6+	addl	36(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	vpxor	%xmm5,%xmm4,%xmm4+	shrdl	$11,%r14d,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	vpsrlq	$17,%xmm7,%xmm7+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	vpaddd	%xmm4,%xmm2,%xmm2+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	shrdl	$2,%r14d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%r10d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r10d+	vpxor	%xmm7,%xmm6,%xmm6+	movl	%edx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ecx,%r13d+	vpshufb	%xmm8,%xmm6,%xmm6+	xorl	%r8d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r10d,%r14d+	vpaddd	%xmm6,%xmm2,%xmm2+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	40(%rsp),%r9d+	vpshufd	$80,%xmm2,%xmm7+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	shrdl	$11,%r14d,%r14d+	vpsrld	$10,%xmm7,%xmm6+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	shrdl	$6,%r13d,%r13d+	vpsrlq	$17,%xmm7,%xmm7+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	vpxor	%xmm7,%xmm6,%xmm6+	xorl	%r11d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r9d,%ebx+	vpsrlq	$2,%xmm7,%xmm7+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	vpshufb	%xmm9,%xmm6,%xmm6+	shrdl	$9,%r14d,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	vpaddd	%xmm6,%xmm2,%xmm2+	shrdl	$5,%r13d,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	vpaddd	64(%rsi),%xmm2,%xmm6+	xorl	%ebx,%r13d+	addl	44(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	vmovdqa	%xmm6,32(%rsp)+	vpalignr	$4,%xmm3,%xmm0,%xmm4+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	vpalignr	$4,%xmm1,%xmm2,%xmm7+	shrdl	$9,%r14d,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	vpsrld	$7,%xmm4,%xmm6+	shrdl	$5,%r13d,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	vpaddd	%xmm7,%xmm3,%xmm3+	xorl	%eax,%r13d+	addl	48(%rsp),%edx+	movl	%r8d,%r15d+	vpsrld	$3,%xmm4,%xmm7+	xorl	%ecx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r9d,%r15d+	vpslld	$14,%xmm4,%xmm5+	addl	%r12d,%edx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	vpxor	%xmm6,%xmm7,%xmm4+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	vpshufd	$250,%xmm2,%xmm7+	shrdl	$2,%r14d,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	vpsrld	$11,%xmm6,%xmm6+	movl	%r11d,%r13d+	addl	%edx,%r14d+	shrdl	$14,%r13d,%r13d+	vpxor	%xmm5,%xmm4,%xmm4+	movl	%r14d,%edx+	movl	%eax,%r12d+	shrdl	$9,%r14d,%r14d+	vpslld	$11,%xmm5,%xmm5+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	shrdl	$5,%r13d,%r13d+	vpxor	%xmm6,%xmm4,%xmm4+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	vpsrld	$10,%xmm7,%xmm6+	addl	52(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	vpxor	%xmm5,%xmm4,%xmm4+	shrdl	$11,%r14d,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	vpsrlq	$17,%xmm7,%xmm7+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	vpaddd	%xmm4,%xmm3,%xmm3+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	shrdl	$2,%r14d,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%ecx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ecx+	vpxor	%xmm7,%xmm6,%xmm6+	movl	%r11d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r10d,%r13d+	vpshufb	%xmm8,%xmm6,%xmm6+	xorl	%eax,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ecx,%r14d+	vpaddd	%xmm6,%xmm3,%xmm3+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	56(%rsp),%ebx+	vpshufd	$80,%xmm3,%xmm7+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	shrdl	$11,%r14d,%r14d+	vpsrld	$10,%xmm7,%xmm6+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	shrdl	$6,%r13d,%r13d+	vpsrlq	$17,%xmm7,%xmm7+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	vpxor	%xmm7,%xmm6,%xmm6+	xorl	%edx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%ebx,%r9d+	vpsrlq	$2,%xmm7,%xmm7+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	vpxor	%xmm7,%xmm6,%xmm6+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	vpshufb	%xmm9,%xmm6,%xmm6+	shrdl	$9,%r14d,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	vpaddd	%xmm6,%xmm3,%xmm3+	shrdl	$5,%r13d,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	vpaddd	96(%rsi),%xmm3,%xmm6+	xorl	%r9d,%r13d+	addl	60(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	vmovdqa	%xmm6,48(%rsp)+	cmpb	$0,131(%rsi)+	jne	.Lavx_00_47+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+	addl	0(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	movl	%edx,%r13d+	addl	%r11d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	addl	4(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	8(%rsp),%r9d+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	xorl	%r11d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r9d,%ebx+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	12(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	xorl	%eax,%r13d+	addl	16(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	movl	%r11d,%r13d+	addl	%edx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%edx+	movl	%eax,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	addl	20(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	24(%rsp),%ebx+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	xorl	%edx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%ebx,%r9d+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	28(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%eax+	movl	%r9d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r8d,%r13d+	xorl	%r10d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%eax,%r14d+	andl	%r8d,%r12d+	xorl	%r8d,%r13d+	addl	32(%rsp),%r11d+	movl	%eax,%r15d+	xorl	%r10d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ebx,%r15d+	addl	%r12d,%r11d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%eax,%r14d+	addl	%r13d,%r11d+	xorl	%ebx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r11d,%edx+	addl	%edi,%r11d+	movl	%edx,%r13d+	addl	%r11d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r11d+	movl	%r8d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%edx,%r13d+	xorl	%r9d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r11d,%r14d+	andl	%edx,%r12d+	xorl	%edx,%r13d+	addl	36(%rsp),%r10d+	movl	%r11d,%edi+	xorl	%r9d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%eax,%edi+	addl	%r12d,%r10d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r11d,%r14d+	addl	%r13d,%r10d+	xorl	%eax,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r10d,%ecx+	addl	%r15d,%r10d+	movl	%ecx,%r13d+	addl	%r10d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r10d+	movl	%edx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ecx,%r13d+	xorl	%r8d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r10d,%r14d+	andl	%ecx,%r12d+	xorl	%ecx,%r13d+	addl	40(%rsp),%r9d+	movl	%r10d,%r15d+	xorl	%r8d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r11d,%r15d+	addl	%r12d,%r9d+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%r10d,%r14d+	addl	%r13d,%r9d+	xorl	%r11d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%r9d,%ebx+	addl	%edi,%r9d+	movl	%ebx,%r13d+	addl	%r9d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r9d+	movl	%ecx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%ebx,%r13d+	xorl	%edx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r9d,%r14d+	andl	%ebx,%r12d+	xorl	%ebx,%r13d+	addl	44(%rsp),%r8d+	movl	%r9d,%edi+	xorl	%edx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r10d,%edi+	addl	%r12d,%r8d+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%r9d,%r14d+	addl	%r13d,%r8d+	xorl	%r10d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%r8d,%eax+	addl	%r15d,%r8d+	movl	%eax,%r13d+	addl	%r8d,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%r8d+	movl	%ebx,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%eax,%r13d+	xorl	%ecx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%r8d,%r14d+	andl	%eax,%r12d+	xorl	%eax,%r13d+	addl	48(%rsp),%edx+	movl	%r8d,%r15d+	xorl	%ecx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r9d,%r15d+	addl	%r12d,%edx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%r8d,%r14d+	addl	%r13d,%edx+	xorl	%r9d,%edi+	shrdl	$2,%r14d,%r14d+	addl	%edx,%r11d+	addl	%edi,%edx+	movl	%r11d,%r13d+	addl	%edx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%edx+	movl	%eax,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r11d,%r13d+	xorl	%ebx,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%edx,%r14d+	andl	%r11d,%r12d+	xorl	%r11d,%r13d+	addl	52(%rsp),%ecx+	movl	%edx,%edi+	xorl	%ebx,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%r8d,%edi+	addl	%r12d,%ecx+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%edx,%r14d+	addl	%r13d,%ecx+	xorl	%r8d,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%ecx,%r10d+	addl	%r15d,%ecx+	movl	%r10d,%r13d+	addl	%ecx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ecx+	movl	%r11d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r10d,%r13d+	xorl	%eax,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ecx,%r14d+	andl	%r10d,%r12d+	xorl	%r10d,%r13d+	addl	56(%rsp),%ebx+	movl	%ecx,%r15d+	xorl	%eax,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%edx,%r15d+	addl	%r12d,%ebx+	shrdl	$6,%r13d,%r13d+	andl	%r15d,%edi+	xorl	%ecx,%r14d+	addl	%r13d,%ebx+	xorl	%edx,%edi+	shrdl	$2,%r14d,%r14d+	addl	%ebx,%r9d+	addl	%edi,%ebx+	movl	%r9d,%r13d+	addl	%ebx,%r14d+	shrdl	$14,%r13d,%r13d+	movl	%r14d,%ebx+	movl	%r10d,%r12d+	shrdl	$9,%r14d,%r14d+	xorl	%r9d,%r13d+	xorl	%r11d,%r12d+	shrdl	$5,%r13d,%r13d+	xorl	%ebx,%r14d+	andl	%r9d,%r12d+	xorl	%r9d,%r13d+	addl	60(%rsp),%eax+	movl	%ebx,%edi+	xorl	%r11d,%r12d+	shrdl	$11,%r14d,%r14d+	xorl	%ecx,%edi+	addl	%r12d,%eax+	shrdl	$6,%r13d,%r13d+	andl	%edi,%r15d+	xorl	%ebx,%r14d+	addl	%r13d,%eax+	xorl	%ecx,%r15d+	shrdl	$2,%r14d,%r14d+	addl	%eax,%r8d+	addl	%r15d,%eax+	movl	%r8d,%r13d+	addl	%eax,%r14d+	movq	-64(%rbp),%rdi+	movl	%r14d,%eax+	movq	-56(%rbp),%rsi++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	addl	24(%rdi),%r10d+	addl	28(%rdi),%r11d++	leaq	64(%rsi),%rsi+	cmpq	-48(%rbp),%rsi++	movl	%eax,0(%rdi)+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)+	jb	.Lloop_avx++	vzeroupper+	movaps	-160(%rbp),%xmm6+	movaps	-144(%rbp),%xmm7+	movaps	-128(%rbp),%xmm8+	movaps	-112(%rbp),%xmm9+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp++	popq	%rbp++.LSEH_epilogue_crypton_sha256_asm_block_data_order_avx:+	mov	8(%rsp),%rdi+	mov	16(%rsp),%rsi++	.byte	0xf3,0xc3++.LSEH_end_crypton_sha256_asm_block_data_order_avx:+.def	crypton_sha256_asm_block_data_order_avx2;	.scl 3;	.type 32;	.endef+.p2align	6+crypton_sha256_asm_block_data_order_avx2:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_sha256_asm_block_data_order_avx2:+++	pushq	%rbp++	movq	%rsp,%rbp++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+.Lavx2_shortcut:+	pushq	%rbx++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	shlq	$4,%rdx+	subq	$120,%rsp++	leaq	(%rsi,%rdx,4),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+	movaps	%xmm6,-160(%rbp)+	movaps	%xmm7,-144(%rbp)+	movaps	%xmm8,-128(%rbp)+	movaps	%xmm9,-112(%rbp)++.LSEH_body_crypton_sha256_asm_block_data_order_avx2:+++	leaq	-64(%rsp),%rsp+	vzeroupper+	andq	$-64,%rsp+	subq	$-64,%rsi+	movl	0(%rdi),%eax+	movq	%rsi,%r12+	movl	4(%rdi),%ebx+	cmpq	%rdx,%rsi+	movl	8(%rdi),%ecx+	cmoveq	%rsp,%r12+	movl	12(%rdi),%edx+	movl	16(%rdi),%r8d+	movl	20(%rdi),%r9d+	movl	24(%rdi),%r10d+	movl	28(%rdi),%r11d+	vmovdqa	K256+512+32(%rip),%ymm8+	vmovdqa	K256+512+64(%rip),%ymm9+	jmp	.Loop_avx2+.p2align	4+.Loop_avx2:+	vmovdqa	K256+512(%rip),%ymm7+	movq	%rsi,-56(%rbp)+	vmovdqu	-64+0(%rsi),%xmm0+	vmovdqu	-64+16(%rsi),%xmm1+	vmovdqu	-64+32(%rsi),%xmm2+	vmovdqu	-64+48(%rsi),%xmm3+	leaq	K256(%rip),%rsi+	vinserti128	$1,(%r12),%ymm0,%ymm0+	vinserti128	$1,16(%r12),%ymm1,%ymm1+	vpshufb	%ymm7,%ymm0,%ymm0+	vinserti128	$1,32(%r12),%ymm2,%ymm2+	vpshufb	%ymm7,%ymm1,%ymm1+	vinserti128	$1,48(%r12),%ymm3,%ymm3++	vpshufb	%ymm7,%ymm2,%ymm2+	vpaddd	0(%rsi),%ymm0,%ymm4+	vpshufb	%ymm7,%ymm3,%ymm3+	vpaddd	32(%rsi),%ymm1,%ymm5+	vpaddd	64(%rsi),%ymm2,%ymm6+	vpaddd	96(%rsi),%ymm3,%ymm7+	vmovdqa	%ymm4,0(%rsp)+	xorl	%r14d,%r14d+	vmovdqa	%ymm5,32(%rsp)+	leaq	-64(%rsp),%rsp+	movl	%ebx,%edi+	vmovdqa	%ymm6,0(%rsp)+	xorl	%ecx,%edi+	vmovdqa	%ymm7,32(%rsp)+	movl	%r9d,%r12d+	subq	$-32*4,%rsi+	jmp	.Lavx2_00_47++.p2align	4+.Lavx2_00_47:+	leaq	-64(%rsp),%rsp+	vpalignr	$4,%ymm0,%ymm1,%ymm4+	addl	0+128(%rsp),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	vpalignr	$4,%ymm2,%ymm3,%ymm7+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	vpsrld	$7,%ymm4,%ymm6+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	vpaddd	%ymm7,%ymm0,%ymm0+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	vpsrld	$3,%ymm4,%ymm7+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	vpslld	$14,%ymm4,%ymm5+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	vpxor	%ymm6,%ymm7,%ymm4+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	vpshufd	$250,%ymm3,%ymm7+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	vpsrld	$11,%ymm6,%ymm6+	addl	4+128(%rsp),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	vpslld	$11,%ymm5,%ymm5+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	vpxor	%ymm6,%ymm4,%ymm4+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	vpsrld	$10,%ymm7,%ymm6+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	vpsrlq	$17,%ymm7,%ymm7+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	vpaddd	%ymm4,%ymm0,%ymm0+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	8+128(%rsp),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	vpshufb	%ymm8,%ymm6,%ymm6+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	vpaddd	%ymm6,%ymm0,%ymm0+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	vpshufd	$80,%ymm0,%ymm7+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	vpsrld	$10,%ymm7,%ymm6+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	vpsrlq	$17,%ymm7,%ymm7+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	12+128(%rsp),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	vpshufb	%ymm9,%ymm6,%ymm6+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	vpaddd	%ymm6,%ymm0,%ymm0+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	vpaddd	0(%rsi),%ymm0,%ymm6+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	vmovdqa	%ymm6,0(%rsp)+	vpalignr	$4,%ymm1,%ymm2,%ymm4+	addl	32+128(%rsp),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	vpalignr	$4,%ymm3,%ymm0,%ymm7+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	vpsrld	$7,%ymm4,%ymm6+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	vpaddd	%ymm7,%ymm1,%ymm1+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	vpsrld	$3,%ymm4,%ymm7+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	vpslld	$14,%ymm4,%ymm5+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	vpxor	%ymm6,%ymm7,%ymm4+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	vpshufd	$250,%ymm0,%ymm7+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	vpsrld	$11,%ymm6,%ymm6+	addl	36+128(%rsp),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	vpslld	$11,%ymm5,%ymm5+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	vpxor	%ymm6,%ymm4,%ymm4+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	vpsrld	$10,%ymm7,%ymm6+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	vpsrlq	$17,%ymm7,%ymm7+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	vpaddd	%ymm4,%ymm1,%ymm1+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	40+128(%rsp),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	vpshufb	%ymm8,%ymm6,%ymm6+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	vpaddd	%ymm6,%ymm1,%ymm1+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	vpshufd	$80,%ymm1,%ymm7+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	vpsrld	$10,%ymm7,%ymm6+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	vpsrlq	$17,%ymm7,%ymm7+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	44+128(%rsp),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	vpshufb	%ymm9,%ymm6,%ymm6+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	vpaddd	%ymm6,%ymm1,%ymm1+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	vpaddd	32(%rsi),%ymm1,%ymm6+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	vmovdqa	%ymm6,32(%rsp)+	leaq	-64(%rsp),%rsp+	vpalignr	$4,%ymm2,%ymm3,%ymm4+	addl	0+128(%rsp),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	vpalignr	$4,%ymm0,%ymm1,%ymm7+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	vpsrld	$7,%ymm4,%ymm6+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	vpaddd	%ymm7,%ymm2,%ymm2+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	vpsrld	$3,%ymm4,%ymm7+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	vpslld	$14,%ymm4,%ymm5+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	vpxor	%ymm6,%ymm7,%ymm4+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	vpshufd	$250,%ymm1,%ymm7+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	vpsrld	$11,%ymm6,%ymm6+	addl	4+128(%rsp),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	vpslld	$11,%ymm5,%ymm5+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	vpxor	%ymm6,%ymm4,%ymm4+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	vpsrld	$10,%ymm7,%ymm6+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	vpsrlq	$17,%ymm7,%ymm7+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	vpaddd	%ymm4,%ymm2,%ymm2+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	8+128(%rsp),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	vpshufb	%ymm8,%ymm6,%ymm6+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	vpaddd	%ymm6,%ymm2,%ymm2+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	vpshufd	$80,%ymm2,%ymm7+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	vpsrld	$10,%ymm7,%ymm6+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	vpsrlq	$17,%ymm7,%ymm7+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	12+128(%rsp),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	vpshufb	%ymm9,%ymm6,%ymm6+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	vpaddd	%ymm6,%ymm2,%ymm2+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	vpaddd	64(%rsi),%ymm2,%ymm6+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	vmovdqa	%ymm6,0(%rsp)+	vpalignr	$4,%ymm3,%ymm0,%ymm4+	addl	32+128(%rsp),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	vpalignr	$4,%ymm1,%ymm2,%ymm7+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	vpsrld	$7,%ymm4,%ymm6+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	vpaddd	%ymm7,%ymm3,%ymm3+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	vpsrld	$3,%ymm4,%ymm7+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	vpslld	$14,%ymm4,%ymm5+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	vpxor	%ymm6,%ymm7,%ymm4+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	vpshufd	$250,%ymm2,%ymm7+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	vpsrld	$11,%ymm6,%ymm6+	addl	36+128(%rsp),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	vpslld	$11,%ymm5,%ymm5+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	vpxor	%ymm6,%ymm4,%ymm4+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	vpsrld	$10,%ymm7,%ymm6+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	vpxor	%ymm5,%ymm4,%ymm4+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	vpsrlq	$17,%ymm7,%ymm7+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	vpaddd	%ymm4,%ymm3,%ymm3+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	40+128(%rsp),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	vpshufb	%ymm8,%ymm6,%ymm6+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	vpaddd	%ymm6,%ymm3,%ymm3+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	vpshufd	$80,%ymm3,%ymm7+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	vpsrld	$10,%ymm7,%ymm6+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	vpsrlq	$17,%ymm7,%ymm7+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	vpxor	%ymm7,%ymm6,%ymm6+	addl	44+128(%rsp),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	vpsrlq	$2,%ymm7,%ymm7+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	vpxor	%ymm7,%ymm6,%ymm6+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	vpshufb	%ymm9,%ymm6,%ymm6+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	vpaddd	%ymm6,%ymm3,%ymm3+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	vpaddd	96(%rsi),%ymm3,%ymm6+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	vmovdqa	%ymm6,32(%rsp)+	leaq	128(%rsi),%rsi+	cmpb	$0,3(%rsi)+	jne	.Lavx2_00_47+	addl	0+64(%rsp),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	addl	4+64(%rsp),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	addl	8+64(%rsp),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	addl	12+64(%rsp),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	addl	32+64(%rsp),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	addl	36+64(%rsp),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	addl	40+64(%rsp),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	addl	44+64(%rsp),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	addl	0(%rsp),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	addl	4(%rsp),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	addl	8(%rsp),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	addl	12(%rsp),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	addl	32(%rsp),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	addl	36(%rsp),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	addl	40(%rsp),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	addl	44(%rsp),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	movq	-64(%rbp),%rdi+	addl	%r14d,%eax+	movl	-56(%rbp),%r12d++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	addl	24(%rdi),%r10d+	addl	28(%rdi),%r11d++	movl	%eax,0(%rdi)+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)++	cmpl	-48(%rbp),%r12d+	je	.Ldone_avx2++	leaq	448(%rsp),%rsi+	xorl	%r14d,%r14d+	movl	%ebx,%edi+	xorl	%ecx,%edi+	movl	%r9d,%r12d+	jmp	.Lower_avx2+.p2align	4+.Lower_avx2:+	addl	0+16(%rsi),%r11d+	andl	%r8d,%r12d+	rorxl	$25,%r8d,%r13d+	rorxl	$11,%r8d,%r15d+	leal	(%rax,%r14,1),%eax+	leal	(%r11,%r12,1),%r11d+	andnl	%r10d,%r8d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r8d,%r14d+	leal	(%r11,%r12,1),%r11d+	xorl	%r14d,%r13d+	movl	%eax,%r15d+	rorxl	$22,%eax,%r12d+	leal	(%r11,%r13,1),%r11d+	xorl	%ebx,%r15d+	rorxl	$13,%eax,%r14d+	rorxl	$2,%eax,%r13d+	leal	(%rdx,%r11,1),%edx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%ebx,%edi+	xorl	%r13d,%r14d+	leal	(%r11,%rdi,1),%r11d+	movl	%r8d,%r12d+	addl	4+16(%rsi),%r10d+	andl	%edx,%r12d+	rorxl	$25,%edx,%r13d+	rorxl	$11,%edx,%edi+	leal	(%r11,%r14,1),%r11d+	leal	(%r10,%r12,1),%r10d+	andnl	%r9d,%edx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%edx,%r14d+	leal	(%r10,%r12,1),%r10d+	xorl	%r14d,%r13d+	movl	%r11d,%edi+	rorxl	$22,%r11d,%r12d+	leal	(%r10,%r13,1),%r10d+	xorl	%eax,%edi+	rorxl	$13,%r11d,%r14d+	rorxl	$2,%r11d,%r13d+	leal	(%rcx,%r10,1),%ecx+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%eax,%r15d+	xorl	%r13d,%r14d+	leal	(%r10,%r15,1),%r10d+	movl	%edx,%r12d+	addl	8+16(%rsi),%r9d+	andl	%ecx,%r12d+	rorxl	$25,%ecx,%r13d+	rorxl	$11,%ecx,%r15d+	leal	(%r10,%r14,1),%r10d+	leal	(%r9,%r12,1),%r9d+	andnl	%r8d,%ecx,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%ecx,%r14d+	leal	(%r9,%r12,1),%r9d+	xorl	%r14d,%r13d+	movl	%r10d,%r15d+	rorxl	$22,%r10d,%r12d+	leal	(%r9,%r13,1),%r9d+	xorl	%r11d,%r15d+	rorxl	$13,%r10d,%r14d+	rorxl	$2,%r10d,%r13d+	leal	(%rbx,%r9,1),%ebx+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r11d,%edi+	xorl	%r13d,%r14d+	leal	(%r9,%rdi,1),%r9d+	movl	%ecx,%r12d+	addl	12+16(%rsi),%r8d+	andl	%ebx,%r12d+	rorxl	$25,%ebx,%r13d+	rorxl	$11,%ebx,%edi+	leal	(%r9,%r14,1),%r9d+	leal	(%r8,%r12,1),%r8d+	andnl	%edx,%ebx,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%ebx,%r14d+	leal	(%r8,%r12,1),%r8d+	xorl	%r14d,%r13d+	movl	%r9d,%edi+	rorxl	$22,%r9d,%r12d+	leal	(%r8,%r13,1),%r8d+	xorl	%r10d,%edi+	rorxl	$13,%r9d,%r14d+	rorxl	$2,%r9d,%r13d+	leal	(%rax,%r8,1),%eax+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r10d,%r15d+	xorl	%r13d,%r14d+	leal	(%r8,%r15,1),%r8d+	movl	%ebx,%r12d+	addl	32+16(%rsi),%edx+	andl	%eax,%r12d+	rorxl	$25,%eax,%r13d+	rorxl	$11,%eax,%r15d+	leal	(%r8,%r14,1),%r8d+	leal	(%rdx,%r12,1),%edx+	andnl	%ecx,%eax,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%eax,%r14d+	leal	(%rdx,%r12,1),%edx+	xorl	%r14d,%r13d+	movl	%r8d,%r15d+	rorxl	$22,%r8d,%r12d+	leal	(%rdx,%r13,1),%edx+	xorl	%r9d,%r15d+	rorxl	$13,%r8d,%r14d+	rorxl	$2,%r8d,%r13d+	leal	(%r11,%rdx,1),%r11d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%r9d,%edi+	xorl	%r13d,%r14d+	leal	(%rdx,%rdi,1),%edx+	movl	%eax,%r12d+	addl	36+16(%rsi),%ecx+	andl	%r11d,%r12d+	rorxl	$25,%r11d,%r13d+	rorxl	$11,%r11d,%edi+	leal	(%rdx,%r14,1),%edx+	leal	(%rcx,%r12,1),%ecx+	andnl	%ebx,%r11d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r11d,%r14d+	leal	(%rcx,%r12,1),%ecx+	xorl	%r14d,%r13d+	movl	%edx,%edi+	rorxl	$22,%edx,%r12d+	leal	(%rcx,%r13,1),%ecx+	xorl	%r8d,%edi+	rorxl	$13,%edx,%r14d+	rorxl	$2,%edx,%r13d+	leal	(%r10,%rcx,1),%r10d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%r8d,%r15d+	xorl	%r13d,%r14d+	leal	(%rcx,%r15,1),%ecx+	movl	%r11d,%r12d+	addl	40+16(%rsi),%ebx+	andl	%r10d,%r12d+	rorxl	$25,%r10d,%r13d+	rorxl	$11,%r10d,%r15d+	leal	(%rcx,%r14,1),%ecx+	leal	(%rbx,%r12,1),%ebx+	andnl	%eax,%r10d,%r12d+	xorl	%r15d,%r13d+	rorxl	$6,%r10d,%r14d+	leal	(%rbx,%r12,1),%ebx+	xorl	%r14d,%r13d+	movl	%ecx,%r15d+	rorxl	$22,%ecx,%r12d+	leal	(%rbx,%r13,1),%ebx+	xorl	%edx,%r15d+	rorxl	$13,%ecx,%r14d+	rorxl	$2,%ecx,%r13d+	leal	(%r9,%rbx,1),%r9d+	andl	%r15d,%edi+	xorl	%r12d,%r14d+	xorl	%edx,%edi+	xorl	%r13d,%r14d+	leal	(%rbx,%rdi,1),%ebx+	movl	%r10d,%r12d+	addl	44+16(%rsi),%eax+	andl	%r9d,%r12d+	rorxl	$25,%r9d,%r13d+	rorxl	$11,%r9d,%edi+	leal	(%rbx,%r14,1),%ebx+	leal	(%rax,%r12,1),%eax+	andnl	%r11d,%r9d,%r12d+	xorl	%edi,%r13d+	rorxl	$6,%r9d,%r14d+	leal	(%rax,%r12,1),%eax+	xorl	%r14d,%r13d+	movl	%ebx,%edi+	rorxl	$22,%ebx,%r12d+	leal	(%rax,%r13,1),%eax+	xorl	%ecx,%edi+	rorxl	$13,%ebx,%r14d+	rorxl	$2,%ebx,%r13d+	leal	(%r8,%rax,1),%r8d+	andl	%edi,%r15d+	xorl	%r12d,%r14d+	xorl	%ecx,%r15d+	xorl	%r13d,%r14d+	leal	(%rax,%r15,1),%eax+	movl	%r9d,%r12d+	leaq	-64(%rsi),%rsi+	cmpq	%rsp,%rsi+	jae	.Lower_avx2++	movq	-64(%rbp),%rdi+	addl	%r14d,%eax+	movq	-56(%rbp),%rsi+	leaq	448(%rsp),%rsp++	addl	0(%rdi),%eax+	addl	4(%rdi),%ebx+	addl	8(%rdi),%ecx+	addl	12(%rdi),%edx+	addl	16(%rdi),%r8d+	addl	20(%rdi),%r9d+	leaq	128(%rsi),%rsi+	addl	24(%rdi),%r10d+	movq	%rsi,%r12+	addl	28(%rdi),%r11d+	cmpq	-48(%rbp),%rsi++	movl	%eax,0(%rdi)+	cmoveq	%rsp,%r12+	movl	%ebx,4(%rdi)+	movl	%ecx,8(%rdi)+	movl	%edx,12(%rdi)+	movl	%r8d,16(%rdi)+	movl	%r9d,20(%rdi)+	movl	%r10d,24(%rdi)+	movl	%r11d,28(%rdi)++	jbe	.Loop_avx2++.Ldone_avx2:+	vzeroupper+	movaps	-160(%rbp),%xmm6+	movaps	-144(%rbp),%xmm7+	movaps	-128(%rbp),%xmm8+	movaps	-112(%rbp),%xmm9+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp++	popq	%rbp++.LSEH_epilogue_crypton_sha256_asm_block_data_order_avx2:+	mov	8(%rsp),%rdi+	mov	16(%rsp),%rsi++	.byte	0xf3,0xc3++.LSEH_end_crypton_sha256_asm_block_data_order_avx2:+.section	.pdata+.p2align	2+.rva	.LSEH_begin_crypton_sha256_asm_block_data_order+.rva	.LSEH_body_crypton_sha256_asm_block_data_order+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_prologue++.rva	.LSEH_body_crypton_sha256_asm_block_data_order+.rva	.LSEH_epilogue_crypton_sha256_asm_block_data_order+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_body++.rva	.LSEH_epilogue_crypton_sha256_asm_block_data_order+.rva	.LSEH_end_crypton_sha256_asm_block_data_order+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_epilogue++.rva	.LSEH_begin_crypton_sha256_asm_block_data_order_shaext+.rva	.LSEH_body_crypton_sha256_asm_block_data_order_shaext+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_shaext_prologue++.rva	.LSEH_body_crypton_sha256_asm_block_data_order_shaext+.rva	.LSEH_epilogue_crypton_sha256_asm_block_data_order_shaext+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_shaext_body++.rva	.LSEH_epilogue_crypton_sha256_asm_block_data_order_shaext+.rva	.LSEH_end_crypton_sha256_asm_block_data_order_shaext+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_shaext_epilogue++.rva	.LSEH_begin_crypton_sha256_asm_block_data_order_ssse3+.rva	.LSEH_body_crypton_sha256_asm_block_data_order_ssse3+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_ssse3_prologue++.rva	.LSEH_body_crypton_sha256_asm_block_data_order_ssse3+.rva	.LSEH_epilogue_crypton_sha256_asm_block_data_order_ssse3+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_ssse3_body++.rva	.LSEH_epilogue_crypton_sha256_asm_block_data_order_ssse3+.rva	.LSEH_end_crypton_sha256_asm_block_data_order_ssse3+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_ssse3_epilogue++.rva	.LSEH_begin_crypton_sha256_asm_block_data_order_avx+.rva	.LSEH_body_crypton_sha256_asm_block_data_order_avx+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_avx_prologue++.rva	.LSEH_body_crypton_sha256_asm_block_data_order_avx+.rva	.LSEH_epilogue_crypton_sha256_asm_block_data_order_avx+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_avx_body++.rva	.LSEH_epilogue_crypton_sha256_asm_block_data_order_avx+.rva	.LSEH_end_crypton_sha256_asm_block_data_order_avx+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_avx_epilogue++.rva	.LSEH_begin_crypton_sha256_asm_block_data_order_avx2+.rva	.LSEH_body_crypton_sha256_asm_block_data_order_avx2+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_avx2_prologue++.rva	.LSEH_body_crypton_sha256_asm_block_data_order_avx2+.rva	.LSEH_epilogue_crypton_sha256_asm_block_data_order_avx2+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_avx2_body++.rva	.LSEH_epilogue_crypton_sha256_asm_block_data_order_avx2+.rva	.LSEH_end_crypton_sha256_asm_block_data_order_avx2+.rva	.LSEH_info_crypton_sha256_asm_block_data_order_avx2_epilogue++.section	.xdata+.p2align	3+.LSEH_info_crypton_sha256_asm_block_data_order_prologue:+.byte	1,4,6,0x05+.byte	4,0x74,2,0+.byte	4,0x64,3,0+.byte	4,0x53+.byte	1,0x50+.long	0,0+.LSEH_info_crypton_sha256_asm_block_data_order_body:+.byte	1,0,18,0+.byte	0x00,0xf4,0x0b,0x00+.byte	0x00,0xe4,0x0c,0x00+.byte	0x00,0xd4,0x0d,0x00+.byte	0x00,0xc4,0x0e,0x00+.byte	0x00,0x34,0x0f,0x00+.byte	0x00,0x54,0x10,0x00+.byte	0x00,0x74,0x12,0x00+.byte	0x00,0x64,0x13,0x00+.byte	0x00,0x01,0x11,0x00+.byte	0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00+.LSEH_info_crypton_sha256_asm_block_data_order_epilogue:+.byte	1,0,5,11+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0xb3+.byte	0x00,0x00,0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00++.LSEH_info_crypton_sha256_asm_block_data_order_shaext_prologue:+.byte	1,4,6,0x05+.byte	4,0x74,2,0+.byte	4,0x64,3,0+.byte	4,0x53+.byte	1,0x50+.long	0,0+.LSEH_info_crypton_sha256_asm_block_data_order_shaext_body:+.byte	1,0,17,85+.byte	0x00,0x68,0x00,0x00+.byte	0x00,0x78,0x01,0x00+.byte	0x00,0x88,0x02,0x00+.byte	0x00,0x98,0x03,0x00+.byte	0x00,0xa8,0x04,0x00+.byte	0x00,0x74,0x0c,0x00+.byte	0x00,0x64,0x0d,0x00+.byte	0x00,0x53+.byte	0x00,0x92+.byte	0x00,0x50+.byte	0x00,0x00,0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00+.LSEH_info_crypton_sha256_asm_block_data_order_shaext_epilogue:+.byte	1,0,4,0+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0x00,0x00,0x00++.LSEH_info_crypton_sha256_asm_block_data_order_ssse3_prologue:+.byte	1,4,6,0x05+.byte	4,0x74,2,0+.byte	4,0x64,3,0+.byte	4,0x53+.byte	1,0x50+.long	0,0+.LSEH_info_crypton_sha256_asm_block_data_order_ssse3_body:+.byte	1,0,25,133+.byte	0x00,0x68,0x00,0x00+.byte	0x00,0x78,0x01,0x00+.byte	0x00,0x88,0x02,0x00+.byte	0x00,0x98,0x03,0x00+.byte	0x00,0xf4,0x0b,0x00+.byte	0x00,0xe4,0x0c,0x00+.byte	0x00,0xd4,0x0d,0x00+.byte	0x00,0xc4,0x0e,0x00+.byte	0x00,0x34,0x0f,0x00+.byte	0x00,0x74,0x12,0x00+.byte	0x00,0x64,0x13,0x00+.byte	0x00,0x53+.byte	0x00,0xf2+.byte	0x00,0x50+.byte	0x00,0x00,0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00+.LSEH_info_crypton_sha256_asm_block_data_order_ssse3_epilogue:+.byte	1,0,4,0+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0x00,0x00,0x00++.LSEH_info_crypton_sha256_asm_block_data_order_avx_prologue:+.byte	1,4,6,0x05+.byte	4,0x74,2,0+.byte	4,0x64,3,0+.byte	4,0x53+.byte	1,0x50+.long	0,0+.LSEH_info_crypton_sha256_asm_block_data_order_avx_body:+.byte	1,0,26,165+.byte	0x00,0x68,0x00,0x00+.byte	0x00,0x78,0x01,0x00+.byte	0x00,0x88,0x02,0x00+.byte	0x00,0x98,0x03,0x00+.byte	0x00,0xf4,0x0f,0x00+.byte	0x00,0xe4,0x10,0x00+.byte	0x00,0xd4,0x11,0x00+.byte	0x00,0xc4,0x12,0x00+.byte	0x00,0x34,0x13,0x00+.byte	0x00,0x74,0x16,0x00+.byte	0x00,0x64,0x17,0x00+.byte	0x00,0x53+.byte	0x00,0x01,0x14,0x00+.byte	0x00,0x50+.byte	0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00+.LSEH_info_crypton_sha256_asm_block_data_order_avx_epilogue:+.byte	1,0,4,0+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0x00,0x00,0x00++.LSEH_info_crypton_sha256_asm_block_data_order_avx2_prologue:+.byte	1,4,6,0x05+.byte	4,0x74,2,0+.byte	4,0x64,3,0+.byte	4,0x53+.byte	1,0x50+.long	0,0+.LSEH_info_crypton_sha256_asm_block_data_order_avx2_body:+.byte	1,0,26,165+.byte	0x00,0x68,0x00,0x00+.byte	0x00,0x78,0x01,0x00+.byte	0x00,0x88,0x02,0x00+.byte	0x00,0x98,0x03,0x00+.byte	0x00,0xf4,0x0f,0x00+.byte	0x00,0xe4,0x10,0x00+.byte	0x00,0xd4,0x11,0x00+.byte	0x00,0xc4,0x12,0x00+.byte	0x00,0x34,0x13,0x00+.byte	0x00,0x74,0x16,0x00+.byte	0x00,0x64,0x17,0x00+.byte	0x00,0x53+.byte	0x00,0x01,0x14,0x00+.byte	0x00,0x50+.byte	0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00+.LSEH_info_crypton_sha256_asm_block_data_order_avx2_epilogue:+.byte	1,0,4,0+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0x00,0x00,0x00+
+ cbits/asm/sha512-armv8.pl view
@@ -0,0 +1,892 @@+#!/usr/bin/env perl+# SPDX-License-Identifier: GPL-1.0+ OR BSD-3-Clause+#+# ====================================================================+# Written by Andy Polyakov, @dot-asm, initially for the OpenSSL+# project.+# ====================================================================+#+# SHA256/512 for ARMv8.+#+# Performance in cycles per processed byte and improvement coefficient+# over code generated with "default" compiler:+#+#		SHA256-hw	SHA256(*)	SHA512+# Apple A7	1.97		10.5 (+33%)	6.73 (-1%(**))+# Apple A10	1.30				5.81+# Apple A12	1.31				5.06+# Apple A14/M1	1.30		8.19 (+14%)	2.24 (hw)+# Cortex-A53	2.38		15.5 (+115%)	10.0 (+150%(***))+# Cortex-A57	2.31		11.6 (+86%)	7.51 (+260%(***))+# Cortex-A76	1.60		9.5		6.05+# Cortex-X2	1.60		7.3		2.60 (hw)+# Cortex-X925	1.57		5.97		2.55 (hw)+# Denver	2.01		10.5 (+26%)	6.70 (+8%)+# X-Gene			20.0 (+100%)	12.8 (+300%(***))+# Mongoose	2.36		13.0 (+50%)	8.36 (+33%)+# Kryo		1.92		17.4 (+30%)	11.2 (+8%)+# ThunderX2	2.54		13.2 (+40%)	8.40 (+18%)+# Shapdragon X	1.40		7.43		2.23 (hw)+#+# (*)	Software SHA256 results are of lesser relevance, presented+#	mostly for informational purposes.+# (**)	The result is a trade-off: it's possible to improve it by+#	10% (or by 1 cycle per round), but at the cost of 20% loss+#	on Cortex-A53 (or by 4 cycles per round).+# (***)	Super-impressive coefficients over gcc-generated code are+#	indication of some compiler "pathology", most notably code+#	generated with -mgeneral-regs-only is significantly faster+#	and the gap is only 40-90%.+#+# October 2016.+#+# Originally it was reckoned that it makes no sense to implement NEON+# version of SHA256 for 64-bit processors. This is because performance+# improvement on most wide-spread Cortex-A5x processors was observed+# to be marginal, same on Cortex-A53 and ~10% on A57. But then it was+# observed that 32-bit NEON SHA256 performs significantly better than+# 64-bit scalar version on *some* of the more recent processors. As+# result 64-bit NEON version of SHA256 was added to provide best+# all-round performance. For example it executes ~30% faster on X-Gene+# and Mongoose. [For reference, NEON version of SHA512 is bound to+# deliver much less improvement, likely *negative* on Cortex-A5x.+# Which is why NEON support is limited to SHA256.]++$flavour = shift;+$output  = shift;++if ($flavour && $flavour ne "void") {+    $0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1;+    ( $xlate="${dir}arm-xlate.pl" and -f $xlate ) or+    ( $xlate="${dir}../../perlasm/arm-xlate.pl" and -f $xlate) or+    die "can't locate arm-xlate.pl";++    open STDOUT,"| \"$^X\" $xlate $flavour $output";+} else {+    open STDOUT,">$output";+}++if ($output =~ /512/) {+	$BITS=512;+	$SZ=8;+	@Sigma0=(28,34,39);+	@Sigma1=(14,18,41);+	@sigma0=(1,  8, 7);+	@sigma1=(19,61, 6);+	$rounds=80;+	$reg_t="x";+} else {+	$BITS=256;+	$SZ=4;+	@Sigma0=( 2,13,22);+	@Sigma1=( 6,11,25);+	@sigma0=( 7,18, 3);+	@sigma1=(17,19,10);+	$rounds=64;+	$reg_t="w";+}++$func="sha${BITS}_block_data_order";++($ctx,$inp,$num,$Ktbl)=map("x$_",(0..2,30));++@X=map("$reg_t$_",(3..15,0..2));+@V=($A,$B,$C,$D,$E,$F,$G,$H)=map("$reg_t$_",(20..27));+($t0,$t1,$t2,$t3)=map("$reg_t$_",(16,17,19,28));++sub BODY_00_xx {+my ($i,$a,$b,$c,$d,$e,$f,$g,$h)=@_;+my $j=($i+1)&15;+my ($T0,$T1,$T2)=(@X[($i-8)&15],@X[($i-9)&15],@X[($i-10)&15]);+   $T0=@X[$i+3] if ($i<11);++$code.=<<___	if ($i<16);+#ifndef	__AARCH64EB__+	rev	@X[$i],@X[$i]			// $i+#endif+___+$code.=<<___	if ($i<13 && ($i&1));+	ldp	@X[$i+1],@X[$i+2],[$inp],#2*$SZ+___+$code.=<<___	if ($i==13);+	ldp	@X[14],@X[15],[$inp]+___+$code.=<<___	if ($i>=14);+	ldr	@X[($i-11)&15],[sp,#`$SZ*(($i-11)%4)`]+___+$code.=<<___	if ($i>0 && $i<16);+	add	$a,$a,$t1			// h+=Sigma0(a)+___+$code.=<<___	if ($i>=11);+	str	@X[($i-8)&15],[sp,#`$SZ*(($i-8)%4)`]+___+# While ARMv8 specifies merged rotate-n-logical operation such as+# 'eor x,y,z,ror#n', it was found to negatively affect performance+# on Apple A7. The reason seems to be that it requires even 'y' to+# be available earlier. This means that such merged instruction is+# not necessarily best choice on critical path... On the other hand+# Cortex-A5x handles merged instructions much better than disjoint+# rotate and logical... See (**) footnote above.+$code.=<<___	if ($i<15);+	ror	$t0,$e,#$Sigma1[0]+	add	$h,$h,$t2			// h+=K[i]+	eor	$T0,$e,$e,ror#`$Sigma1[2]-$Sigma1[1]`+	and	$t1,$f,$e+	bic	$t2,$g,$e+	add	$h,$h,@X[$i&15]			// h+=X[i]+	orr	$t1,$t1,$t2			// Ch(e,f,g)+	eor	$t2,$a,$b			// a^b, b^c in next round+	eor	$t0,$t0,$T0,ror#$Sigma1[1]	// Sigma1(e)+	ror	$T0,$a,#$Sigma0[0]+	add	$h,$h,$t1			// h+=Ch(e,f,g)+	eor	$t1,$a,$a,ror#`$Sigma0[2]-$Sigma0[1]`+	add	$h,$h,$t0			// h+=Sigma1(e)+	and	$t3,$t3,$t2			// (b^c)&=(a^b)+	add	$d,$d,$h			// d+=h+	eor	$t3,$t3,$b			// Maj(a,b,c)+	eor	$t1,$T0,$t1,ror#$Sigma0[1]	// Sigma0(a)+	add	$h,$h,$t3			// h+=Maj(a,b,c)+	ldr	$t3,[$Ktbl],#$SZ		// *K++, $t2 in next round+	//add	$h,$h,$t1			// h+=Sigma0(a)+___+$code.=<<___	if ($i>=15);+	ror	$t0,$e,#$Sigma1[0]+	add	$h,$h,$t2			// h+=K[i]+	ror	$T1,@X[($j+1)&15],#$sigma0[0]+	and	$t1,$f,$e+	ror	$T2,@X[($j+14)&15],#$sigma1[0]+	bic	$t2,$g,$e+	ror	$T0,$a,#$Sigma0[0]+	add	$h,$h,@X[$i&15]			// h+=X[i]+	eor	$t0,$t0,$e,ror#$Sigma1[1]+	eor	$T1,$T1,@X[($j+1)&15],ror#$sigma0[1]+	orr	$t1,$t1,$t2			// Ch(e,f,g)+	eor	$t2,$a,$b			// a^b, b^c in next round+	eor	$t0,$t0,$e,ror#$Sigma1[2]	// Sigma1(e)+	eor	$T0,$T0,$a,ror#$Sigma0[1]+	add	$h,$h,$t1			// h+=Ch(e,f,g)+	and	$t3,$t3,$t2			// (b^c)&=(a^b)+	eor	$T2,$T2,@X[($j+14)&15],ror#$sigma1[1]+	eor	$T1,$T1,@X[($j+1)&15],lsr#$sigma0[2]	// sigma0(X[i+1])+	add	$h,$h,$t0			// h+=Sigma1(e)+	eor	$t3,$t3,$b			// Maj(a,b,c)+	eor	$t1,$T0,$a,ror#$Sigma0[2]	// Sigma0(a)+	eor	$T2,$T2,@X[($j+14)&15],lsr#$sigma1[2]	// sigma1(X[i+14])+	add	@X[$j],@X[$j],@X[($j+9)&15]+	add	$d,$d,$h			// d+=h+	add	$h,$h,$t3			// h+=Maj(a,b,c)+	ldr	$t3,[$Ktbl],#$SZ		// *K++, $t2 in next round+	add	@X[$j],@X[$j],$T1+	add	$h,$h,$t1			// h+=Sigma0(a)+	add	@X[$j],@X[$j],$T2+___+	($t2,$t3)=($t3,$t2);+}++$code.=<<___;+#ifndef	__KERNEL__+# include "arm_arch.h"+.extern	OPENSSL_armcap_P+#endif++.text++.globl	$func+.type	$func,%function+.align	6+$func:+#ifndef	__KERNEL__+	adrp	c16,OPENSSL_armcap_P+	ldr	w16,[c16,#:lo12:OPENSSL_armcap_P]+___+$code.=<<___	if ($SZ==4);+	tst	w16,#ARMV8_SHA256+	b.ne	.Lv8_entry+	tst	w16,#ARMV7_NEON+	b.ne	.Lneon_entry+___+$code.=<<___	if ($SZ==8);+	tst	w16,#ARMV8_SHA512+	b.ne	.Lv8_entry+___+$code.=<<___;+#endif+	.inst	0xd503233f				// paciasp+	stp	c29,c30,[csp,#-16*__SIZEOF_POINTER__]!+	add	c29,csp,#0++	stp	c19,c20,[csp,#2*__SIZEOF_POINTER__]+	stp	c21,c22,[csp,#4*__SIZEOF_POINTER__]+	stp	c23,c24,[csp,#6*__SIZEOF_POINTER__]+	stp	c25,c26,[csp,#8*__SIZEOF_POINTER__]+	stp	c27,c28,[csp,#10*__SIZEOF_POINTER__]+	sub	csp,csp,#4*$SZ++	ldp	$A,$B,[$ctx]				// load context+	ldp	$C,$D,[$ctx,#2*$SZ]+	lsl	$num,$num,#`log(16*$SZ)/log(2)`+	ldp	$E,$F,[$ctx,#4*$SZ]+	cadd	$num,$inp,$num				// end of input+	ldp	$G,$H,[$ctx,#6*$SZ]+	adr	$Ktbl,.LK$BITS+	stp	c#$ctx,c#$num,[c29,#12*__SIZEOF_POINTER__]++.Loop:+	ldp	@X[0],@X[1],[$inp],#2*$SZ+	ldr	$t2,[$Ktbl],#$SZ			// *K+++	eor	$t3,$B,$C				// magic seed+	str	c#$inp,[c29,#14*__SIZEOF_POINTER__]+___+for ($i=0;$i<16;$i++)	{ &BODY_00_xx($i,@V); unshift(@V,pop(@V)); }+$code.=".Loop_16_xx:\n";+for (;$i<32;$i++)	{ &BODY_00_xx($i,@V); unshift(@V,pop(@V)); }+$code.=<<___;+	cbnz	$t2,.Loop_16_xx++	ldp	c#$ctx,c#$num,[c29,#12*__SIZEOF_POINTER__]+	ldr	c#$inp,[c29,#14*__SIZEOF_POINTER__]+	csub	$Ktbl,$Ktbl,#`$SZ*($rounds+1)`		// rewind++	ldp	@X[0],@X[1],[$ctx]+	ldp	@X[2],@X[3],[$ctx,#2*$SZ]+	cadd	$inp,$inp,#14*$SZ			// advance input pointer+	ldp	@X[4],@X[5],[$ctx,#4*$SZ]+	add	$A,$A,@X[0]+	ldp	@X[6],@X[7],[$ctx,#6*$SZ]+	add	$B,$B,@X[1]+	add	$C,$C,@X[2]+	add	$D,$D,@X[3]+	stp	$A,$B,[$ctx]+	add	$E,$E,@X[4]+	add	$F,$F,@X[5]+	stp	$C,$D,[$ctx,#2*$SZ]+	add	$G,$G,@X[6]+	add	$H,$H,@X[7]+	cmp	$inp,$num+	stp	$E,$F,[$ctx,#4*$SZ]+	stp	$G,$H,[$ctx,#6*$SZ]+	b.ne	.Loop++	ldp	c19,c20,[c29,#2*__SIZEOF_POINTER__]+	add	csp,csp,#4*$SZ+	ldp	c21,c22,[c29,#4*__SIZEOF_POINTER__]+	ldp	c23,c24,[c29,#6*__SIZEOF_POINTER__]+	ldp	c25,c26,[c29,#8*__SIZEOF_POINTER__]+	ldp	c27,c28,[c29,#10*__SIZEOF_POINTER__]+	ldp	c29,c30,[csp],#16*__SIZEOF_POINTER__+	.inst	0xd50323bf				// autiasp+	ret+.size	$func,.-$func++.align	6+.type	.LK$BITS,%object+.LK$BITS:+___+$code.=<<___ if ($SZ==8);+	.quad	0x428a2f98d728ae22,0x7137449123ef65cd+	.quad	0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc+	.quad	0x3956c25bf348b538,0x59f111f1b605d019+	.quad	0x923f82a4af194f9b,0xab1c5ed5da6d8118+	.quad	0xd807aa98a3030242,0x12835b0145706fbe+	.quad	0x243185be4ee4b28c,0x550c7dc3d5ffb4e2+	.quad	0x72be5d74f27b896f,0x80deb1fe3b1696b1+	.quad	0x9bdc06a725c71235,0xc19bf174cf692694+	.quad	0xe49b69c19ef14ad2,0xefbe4786384f25e3+	.quad	0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65+	.quad	0x2de92c6f592b0275,0x4a7484aa6ea6e483+	.quad	0x5cb0a9dcbd41fbd4,0x76f988da831153b5+	.quad	0x983e5152ee66dfab,0xa831c66d2db43210+	.quad	0xb00327c898fb213f,0xbf597fc7beef0ee4+	.quad	0xc6e00bf33da88fc2,0xd5a79147930aa725+	.quad	0x06ca6351e003826f,0x142929670a0e6e70+	.quad	0x27b70a8546d22ffc,0x2e1b21385c26c926+	.quad	0x4d2c6dfc5ac42aed,0x53380d139d95b3df+	.quad	0x650a73548baf63de,0x766a0abb3c77b2a8+	.quad	0x81c2c92e47edaee6,0x92722c851482353b+	.quad	0xa2bfe8a14cf10364,0xa81a664bbc423001+	.quad	0xc24b8b70d0f89791,0xc76c51a30654be30+	.quad	0xd192e819d6ef5218,0xd69906245565a910+	.quad	0xf40e35855771202a,0x106aa07032bbd1b8+	.quad	0x19a4c116b8d2d0c8,0x1e376c085141ab53+	.quad	0x2748774cdf8eeb99,0x34b0bcb5e19b48a8+	.quad	0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb+	.quad	0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3+	.quad	0x748f82ee5defb2fc,0x78a5636f43172f60+	.quad	0x84c87814a1f0ab72,0x8cc702081a6439ec+	.quad	0x90befffa23631e28,0xa4506cebde82bde9+	.quad	0xbef9a3f7b2c67915,0xc67178f2e372532b+	.quad	0xca273eceea26619c,0xd186b8c721c0c207+	.quad	0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178+	.quad	0x06f067aa72176fba,0x0a637dc5a2c898a6+	.quad	0x113f9804bef90dae,0x1b710b35131c471b+	.quad	0x28db77f523047d84,0x32caab7b40c72493+	.quad	0x3c9ebe0a15c9bebc,0x431d67c49c100d4c+	.quad	0x4cc5d4becb3e42b6,0x597f299cfc657e2a+	.quad	0x5fcb6fab3ad6faec,0x6c44198c4a475817+	.quad	0	// terminator+___+$code.=<<___ if ($SZ==4);+	.long	0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5+	.long	0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5+	.long	0xd807aa98,0x12835b01,0x243185be,0x550c7dc3+	.long	0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174+	.long	0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc+	.long	0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da+	.long	0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7+	.long	0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967+	.long	0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13+	.long	0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85+	.long	0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3+	.long	0xd192e819,0xd6990624,0xf40e3585,0x106aa070+	.long	0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5+	.long	0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3+	.long	0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208+	.long	0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2+	.long	0	//terminator+___+$code.=<<___;+.size	.LK$BITS,.-.LK$BITS+.asciz	"SHA$BITS block transform for ARMv8, CRYPTOGAMS by \@dot-asm"+.align	2+___++if ($SZ==4) {+my $Ktbl="x3";++my ($ABCD,$EFGH,$abcd)=map("v$_.16b",(0..2));+my @MSG=map("v$_.16b",(4..7));+my ($W0,$W1)=("v16.4s","v17.4s");+my ($ABCD_SAVE,$EFGH_SAVE)=("v18.16b","v19.16b");++$code.=<<___;+#ifndef	__KERNEL__+.type	sha256_block_armv8,%function+.align	6+sha256_block_armv8:+.Lv8_entry:+	stp		c29,c30,[csp,#-2*__SIZEOF_POINTER__]!+	add		c29,csp,#0++	ld1.32		{$ABCD,$EFGH},[$ctx]+	adr		$Ktbl,.LK256++.Loop_hw:+	ld1		{@MSG[0]-@MSG[3]},[$inp],#64+	sub		$num,$num,#1+	ld1.32		{$W0},[$Ktbl],#16+	rev32		@MSG[0],@MSG[0]+	rev32		@MSG[1],@MSG[1]+	rev32		@MSG[2],@MSG[2]+	rev32		@MSG[3],@MSG[3]+	orr		$ABCD_SAVE,$ABCD,$ABCD		// offload+	orr		$EFGH_SAVE,$EFGH,$EFGH+___+for($i=0;$i<12;$i++) {+$code.=<<___;+	ld1.32		{$W1},[$Ktbl],#16+	add.i32		$W0,$W0,@MSG[0]+	sha256su0	@MSG[0],@MSG[1]+	orr		$abcd,$ABCD,$ABCD+	sha256h		$ABCD,$EFGH,$W0+	sha256h2	$EFGH,$abcd,$W0+	sha256su1	@MSG[0],@MSG[2],@MSG[3]+___+	($W0,$W1)=($W1,$W0);	push(@MSG,shift(@MSG));+}+$code.=<<___;+	ld1.32		{$W1},[$Ktbl],#16+	add.i32		$W0,$W0,@MSG[0]+	orr		$abcd,$ABCD,$ABCD+	sha256h		$ABCD,$EFGH,$W0+	sha256h2	$EFGH,$abcd,$W0++	ld1.32		{$W0},[$Ktbl],#16+	add.i32		$W1,$W1,@MSG[1]+	orr		$abcd,$ABCD,$ABCD+	sha256h		$ABCD,$EFGH,$W1+	sha256h2	$EFGH,$abcd,$W1++	ld1.32		{$W1},[$Ktbl]+	add.i32		$W0,$W0,@MSG[2]+	csub		$Ktbl,$Ktbl,#$rounds*$SZ-16	// rewind+	orr		$abcd,$ABCD,$ABCD+	sha256h		$ABCD,$EFGH,$W0+	sha256h2	$EFGH,$abcd,$W0++	add.i32		$W1,$W1,@MSG[3]+	orr		$abcd,$ABCD,$ABCD+	sha256h		$ABCD,$EFGH,$W1+	sha256h2	$EFGH,$abcd,$W1++	add.i32		$ABCD,$ABCD,$ABCD_SAVE+	add.i32		$EFGH,$EFGH,$EFGH_SAVE++	cbnz		$num,.Loop_hw++	st1.32		{$ABCD,$EFGH},[$ctx]++	ldr		c29,[csp],#2*__SIZEOF_POINTER__+	ret+.size	sha256_block_armv8,.-sha256_block_armv8+#endif+___+}++if ($SZ==4) {	######################################### NEON stuff #+# You'll surely note a lot of similarities with sha256-armv4 module,+# and of course it's not a coincidence. sha256-armv4 was used as+# initial template, but was adapted for ARMv8 instruction set and+# extensively re-tuned for all-round performance.++my @V = ($A,$B,$C,$D,$E,$F,$G,$H) = map("w$_",(3..10));+my ($t0,$t1,$t2,$t3,$t4) = map("w$_",(11..15));+my $Ktbl="x16";+my $Xfer="x17";+my @X = map("q$_",(0..3));+my ($T0,$T1,$T2,$T3,$T4,$T5,$T6,$T7) = map("q$_",(4..7,16..19));+my $j=0;++sub AUTOLOAD()          # thunk [simplified] x86-style perlasm+{ my $opcode = $AUTOLOAD; $opcode =~ s/.*:://; $opcode =~ s/_/\./;+  my $arg = pop;+    $arg = "#$arg" if ($arg*1 eq $arg);+    $code .= "\t$opcode\t".join(',',@_,$arg)."\n";+}++sub Dscalar { shift =~ m|[qv]([0-9]+)|?"d$1":""; }+sub Dlo     { shift =~ m|[qv]([0-9]+)|?"v$1.d[0]":""; }+sub Dhi     { shift =~ m|[qv]([0-9]+)|?"v$1.d[1]":""; }++sub Xupdate()+{ use integer;+  my $body = shift;+  my @insns = (&$body,&$body,&$body,&$body);+  my ($a,$b,$c,$d,$e,$f,$g,$h);++	&ext_8		($T0,@X[0],@X[1],4);	# X[1..4]+	 eval(shift(@insns));+	 eval(shift(@insns));+	 eval(shift(@insns));+	&ext_8		($T3,@X[2],@X[3],4);	# X[9..12]+	 eval(shift(@insns));+	 eval(shift(@insns));+	&mov		(&Dscalar($T7),&Dhi(@X[3]));	# X[14..15]+	 eval(shift(@insns));+	 eval(shift(@insns));+	&ushr_32	($T2,$T0,$sigma0[0]);+	 eval(shift(@insns));+	&ushr_32	($T1,$T0,$sigma0[2]);+	 eval(shift(@insns));+	&add_32 	(@X[0],@X[0],$T3);	# X[0..3] += X[9..12]+	 eval(shift(@insns));+	&sli_32		($T2,$T0,32-$sigma0[0]);+	 eval(shift(@insns));+	 eval(shift(@insns));+	&ushr_32	($T3,$T0,$sigma0[1]);+	 eval(shift(@insns));+	 eval(shift(@insns));+	&eor_8		($T1,$T1,$T2);+	 eval(shift(@insns));+	 eval(shift(@insns));+	&sli_32		($T3,$T0,32-$sigma0[1]);+	 eval(shift(@insns));+	 eval(shift(@insns));+	  &ushr_32	($T4,$T7,$sigma1[0]);+	 eval(shift(@insns));+	 eval(shift(@insns));+	&eor_8		($T1,$T1,$T3);		# sigma0(X[1..4])+	 eval(shift(@insns));+	 eval(shift(@insns));+	  &sli_32	($T4,$T7,32-$sigma1[0]);+	 eval(shift(@insns));+	 eval(shift(@insns));+	  &ushr_32	($T5,$T7,$sigma1[2]);+	 eval(shift(@insns));+	 eval(shift(@insns));+	  &ushr_32	($T3,$T7,$sigma1[1]);+	 eval(shift(@insns));+	 eval(shift(@insns));+	&add_32		(@X[0],@X[0],$T1);	# X[0..3] += sigma0(X[1..4])+	 eval(shift(@insns));+	 eval(shift(@insns));+	  &sli_u32	($T3,$T7,32-$sigma1[1]);+	 eval(shift(@insns));+	 eval(shift(@insns));+	  &eor_8	($T5,$T5,$T4);+	 eval(shift(@insns));+	 eval(shift(@insns));+	 eval(shift(@insns));+	  &eor_8	($T5,$T5,$T3);		# sigma1(X[14..15])+	 eval(shift(@insns));+	 eval(shift(@insns));+	 eval(shift(@insns));+	&add_32		(@X[0],@X[0],$T5);	# X[0..1] += sigma1(X[14..15])+	 eval(shift(@insns));+	 eval(shift(@insns));+	 eval(shift(@insns));+	  &ushr_32	($T6,@X[0],$sigma1[0]);+	 eval(shift(@insns));+	  &ushr_32	($T7,@X[0],$sigma1[2]);+	 eval(shift(@insns));+	 eval(shift(@insns));+	  &sli_32	($T6,@X[0],32-$sigma1[0]);+	 eval(shift(@insns));+	  &ushr_32	($T5,@X[0],$sigma1[1]);+	 eval(shift(@insns));+	 eval(shift(@insns));+	  &eor_8	($T7,$T7,$T6);+	 eval(shift(@insns));+	 eval(shift(@insns));+	  &sli_32	($T5,@X[0],32-$sigma1[1]);+	 eval(shift(@insns));+	 eval(shift(@insns));+	&ld1_32		("{$T0}","[$Ktbl], #16");+	 eval(shift(@insns));+	  &eor_8	($T7,$T7,$T5);		# sigma1(X[16..17])+	 eval(shift(@insns));+	 eval(shift(@insns));+	&eor_8		($T5,$T5,$T5);+	 eval(shift(@insns));+	 eval(shift(@insns));+	&mov		(&Dhi($T5), &Dlo($T7));+	 eval(shift(@insns));+	 eval(shift(@insns));+	 eval(shift(@insns));+	&add_32		(@X[0],@X[0],$T5);	# X[2..3] += sigma1(X[16..17])+	 eval(shift(@insns));+	 eval(shift(@insns));+	 eval(shift(@insns));+	&add_32		($T0,$T0,@X[0]);+	 while($#insns>=1) { eval(shift(@insns)); }+	&st1_32		("{$T0}","[$Xfer], #16");+	 eval(shift(@insns));++	push(@X,shift(@X));		# "rotate" X[]+}++sub Xpreload()+{ use integer;+  my $body = shift;+  my @insns = (&$body,&$body,&$body,&$body);+  my ($a,$b,$c,$d,$e,$f,$g,$h);++	 eval(shift(@insns));+	 eval(shift(@insns));+	&ld1_8		("{@X[0]}","[$inp],#16");+	 eval(shift(@insns));+	 eval(shift(@insns));+	&ld1_32		("{$T0}","[$Ktbl],#16");+	 eval(shift(@insns));+	 eval(shift(@insns));+	 eval(shift(@insns));+	 eval(shift(@insns));+	&rev32		(@X[0],@X[0]);+	 eval(shift(@insns));+	 eval(shift(@insns));+	 eval(shift(@insns));+	 eval(shift(@insns));+	&add_32		($T0,$T0,@X[0]);+	 foreach (@insns) { eval; }	# remaining instructions+	&st1_32		("{$T0}","[$Xfer], #16");++	push(@X,shift(@X));		# "rotate" X[]+}++sub body_00_15 () {+	(+	'($a,$b,$c,$d,$e,$f,$g,$h)=@V;'.+	'&add	($h,$h,$t1)',			# h+=X[i]+K[i]+	'&add	($a,$a,$t4);'.			# h+=Sigma0(a) from the past+	'&and	($t1,$f,$e)',+	'&bic	($t4,$g,$e)',+	'&eor	($t0,$e,$e,"ror#".($Sigma1[1]-$Sigma1[0]))',+	'&add	($a,$a,$t2)',			# h+=Maj(a,b,c) from the past+	'&orr	($t1,$t1,$t4)',			# Ch(e,f,g)+	'&eor	($t0,$t0,$e,"ror#".($Sigma1[2]-$Sigma1[0]))',	# Sigma1(e)+	'&eor	($t4,$a,$a,"ror#".($Sigma0[1]-$Sigma0[0]))',+	'&add	($h,$h,$t1)',			# h+=Ch(e,f,g)+	'&ror	($t0,$t0,"#$Sigma1[0]")',+	'&eor	($t2,$a,$b)',			# a^b, b^c in next round+	'&eor	($t4,$t4,$a,"ror#".($Sigma0[2]-$Sigma0[0]))',	# Sigma0(a)+	'&add	($h,$h,$t0)',			# h+=Sigma1(e)+	'&ldr	($t1,sprintf "[sp,#%d]",4*(($j+1)&15))	if (($j&15)!=15);'.+	'&ldr	($t1,"[$Ktbl]")				if ($j==15);'.+	'&and	($t3,$t3,$t2)',			# (b^c)&=(a^b)+	'&ror	($t4,$t4,"#$Sigma0[0]")',+	'&add	($d,$d,$h)',			# d+=h+	'&eor	($t3,$t3,$b)',			# Maj(a,b,c)+	'$j++;	unshift(@V,pop(@V)); ($t2,$t3)=($t3,$t2);'+	)+}++$code.=<<___;+#ifdef	__KERNEL__+.globl	sha256_block_neon+#endif+.type	sha256_block_neon,%function+.align	4+sha256_block_neon:+.Lneon_entry:+	stp	c29, c30, [csp, #-2*__SIZEOF_POINTER__]!+	mov	c29, csp+	sub	csp,csp,#16*4++	adr	$Ktbl,.LK256+	add	$num,$inp,$num,lsl#6	// len to point at the end of inp++	ld1.8	{@X[0]},[$inp], #16+	ld1.8	{@X[1]},[$inp], #16+	ld1.8	{@X[2]},[$inp], #16+	ld1.8	{@X[3]},[$inp], #16+	ld1.32	{$T0},[$Ktbl], #16+	ld1.32	{$T1},[$Ktbl], #16+	ld1.32	{$T2},[$Ktbl], #16+	ld1.32	{$T3},[$Ktbl], #16+	rev32	@X[0],@X[0]		// yes, even on+	rev32	@X[1],@X[1]		// big-endian+	rev32	@X[2],@X[2]+	rev32	@X[3],@X[3]+	cmov	$Xfer,sp+	add.32	$T0,$T0,@X[0]+	add.32	$T1,$T1,@X[1]+	add.32	$T2,$T2,@X[2]+	st1.32	{$T0-$T1},[$Xfer], #32+	add.32	$T3,$T3,@X[3]+	st1.32	{$T2-$T3},[$Xfer]+	csub	$Xfer,$Xfer,#32++	ldp	$A,$B,[$ctx]+	ldp	$C,$D,[$ctx,#8]+	ldp	$E,$F,[$ctx,#16]+	ldp	$G,$H,[$ctx,#24]+	ldr	$t1,[sp,#0]+	mov	$t2,wzr+	eor	$t3,$B,$C+	mov	$t4,wzr+	b	.L_00_48++.align	4+.L_00_48:+___+	&Xupdate(\&body_00_15);+	&Xupdate(\&body_00_15);+	&Xupdate(\&body_00_15);+	&Xupdate(\&body_00_15);+$code.=<<___;+	cmp	$t1,#0				// check for K256 terminator+	ldr	$t1,[sp,#0]+	csub	$Xfer,$Xfer,#64+	bne	.L_00_48++	csub	$Ktbl,$Ktbl,#256		// rewind $Ktbl+	cmp	$inp,$num+	mov	$Xfer, #-64+	csel	$Xfer, $Xfer, xzr, eq+	cadd	$inp,$inp,$Xfer			// avoid SEGV+	cmov	$Xfer,sp+___+	&Xpreload(\&body_00_15);+	&Xpreload(\&body_00_15);+	&Xpreload(\&body_00_15);+	&Xpreload(\&body_00_15);+$code.=<<___;+	add	$A,$A,$t4			// h+=Sigma0(a) from the past+	ldp	$t0,$t1,[$ctx,#0]+	add	$A,$A,$t2			// h+=Maj(a,b,c) from the past+	ldp	$t2,$t3,[$ctx,#8]+	add	$A,$A,$t0			// accumulate+	add	$B,$B,$t1+	ldp	$t0,$t1,[$ctx,#16]+	add	$C,$C,$t2+	add	$D,$D,$t3+	ldp	$t2,$t3,[$ctx,#24]+	add	$E,$E,$t0+	add	$F,$F,$t1+	 ldr	$t1,[sp,#0]+	stp	$A,$B,[$ctx,#0]+	add	$G,$G,$t2+	 mov	$t2,wzr+	stp	$C,$D,[$ctx,#8]+	add	$H,$H,$t3+	stp	$E,$F,[$ctx,#16]+	 eor	$t3,$B,$C+	stp	$G,$H,[$ctx,#24]+	 mov	$t4,wzr+	 cmov	$Xfer,sp+	b.ne	.L_00_48++	ldr	c29,[c29]+	add	csp,csp,#16*4+2*__SIZEOF_POINTER__+	ret+.size	sha256_block_neon,.-sha256_block_neon+___+}++if ($SZ==8) {+my $Ktbl="x3";++my @H = map("v$_.16b",(0..4));+my ($fg,$de,$m9_10)=map("v$_.16b",(5..7));+my @MSG=map("v$_.16b",(16..23));+my ($W0,$W1)=("v24.2d","v25.2d");+my ($AB,$CD,$EF,$GH)=map("v$_.16b",(26..29));++$code.=<<___;+#ifndef	__KERNEL__+.type	sha512_block_armv8,%function+.align	6+sha512_block_armv8:+.Lv8_entry:+	stp		c29,c30,[csp,#-2*__SIZEOF_POINTER__]!+	add		c29,csp,#0++	ld1		{@MSG[0]-@MSG[3]},[$inp],#64	// load input+	ld1		{@MSG[4]-@MSG[7]},[$inp],#64++	ld1.64		{@H[0]-@H[3]},[$ctx]		// load context+	adr		$Ktbl,.LK512++	rev64		@MSG[0],@MSG[0]+	rev64		@MSG[1],@MSG[1]+	rev64		@MSG[2],@MSG[2]+	rev64		@MSG[3],@MSG[3]+	rev64		@MSG[4],@MSG[4]+	rev64		@MSG[5],@MSG[5]+	rev64		@MSG[6],@MSG[6]+	rev64		@MSG[7],@MSG[7]+	b		.Loop_hw++.align	4+.Loop_hw:+	ld1.64		{$W0},[$Ktbl],#16+	subs		$num,$num,#1+	sub		c4,c#$inp,#128+	orr		$AB,@H[0],@H[0]			// offload+	orr		$CD,@H[1],@H[1]+	orr		$EF,@H[2],@H[2]+	orr		$GH,@H[3],@H[3]+	csel		c#$inp,c#$inp,c4,ne		// conditional rewind+___+for($i=0;$i<32;$i++) {+$code.=<<___;+	add.i64		$W0,$W0,@MSG[0]+	ld1.64		{$W1},[$Ktbl],#16+	ext		$W0,$W0,$W0,#8+	ext		$fg,@H[2],@H[3],#8+	ext		$de,@H[1],@H[2],#8+	add.i64		@H[3],@H[3],$W0			// "T1 + H + K512[i]"+	 sha512su0	@MSG[0],@MSG[1]+	 ext		$m9_10,@MSG[4],@MSG[5],#8+	sha512h		@H[3],$fg,$de+	 sha512su1	@MSG[0],@MSG[7],$m9_10+	add.i64		@H[4],@H[1],@H[3]		// "D + T1"+	sha512h2	@H[3],$H[1],@H[0]+___+	($W0,$W1)=($W1,$W0);	push(@MSG,shift(@MSG));+	@H = (@H[3],@H[0],@H[4],@H[2],@H[1]);+}+for(;$i<40;$i++) {+$code.=<<___	if ($i<39);+	ld1.64		{$W1},[$Ktbl],#16+___+$code.=<<___	if ($i==39);+	csub		$Ktbl,$Ktbl,#$rounds*$SZ	// rewind+___+$code.=<<___;+	add.i64		$W0,$W0,@MSG[0]+	 ld1		{@MSG[0]},[$inp],#16		// load next input+	ext		$W0,$W0,$W0,#8+	ext		$fg,@H[2],@H[3],#8+	ext		$de,@H[1],@H[2],#8+	add.i64		@H[3],@H[3],$W0			// "T1 + H + K512[i]"+	sha512h		@H[3],$fg,$de+	 rev64		@MSG[0],@MSG[0]+	add.i64		@H[4],@H[1],@H[3]		// "D + T1"+	sha512h2	@H[3],$H[1],@H[0]+___+	($W0,$W1)=($W1,$W0);	push(@MSG,shift(@MSG));+	@H = (@H[3],@H[0],@H[4],@H[2],@H[1]);+}+$code.=<<___;+	add.i64		@H[0],@H[0],$AB			// accumulate+	add.i64		@H[1],@H[1],$CD+	add.i64		@H[2],@H[2],$EF+	add.i64		@H[3],@H[3],$GH++	cbnz		$num,.Loop_hw++	st1.64		{@H[0]-@H[3]},[$ctx]		// store context++	ldr		c29,[csp],#2*__SIZEOF_POINTER__+	ret+.size	sha512_block_armv8,.-sha512_block_armv8+#endif+___+}++$code.=<<___;+#if !defined(__KERNEL__) && !defined(_WIN64)+.comm	OPENSSL_armcap_P,4,4+.hidden	OPENSSL_armcap_P+#endif+___++{   my  %opcode = (+	"sha256h"	=> 0x5e004000,	"sha256h2"	=> 0x5e005000,+	"sha256su0"	=> 0x5e282800,	"sha256su1"	=> 0x5e006000	);++    sub unsha256 {+	my ($mnemonic,$arg)=@_;++	$arg =~ m/[qv]([0-9]+)[^,]*,\s*[qv]([0-9]+)[^,]*(?:,\s*[qv]([0-9]+))?/o+	&&+	sprintf ".inst\t0x%08x\t//%s %s",+			$opcode{$mnemonic}|$1|($2<<5)|($3<<16),+			$mnemonic,$arg;+    }+}++{   my  %opcode = (+	"sha512h"	=> 0xce608000,	"sha512h2"	=> 0xce608400,+	"sha512su0"	=> 0xcec08000,	"sha512su1"	=> 0xce608800	);++    sub unsha512 {+	my ($mnemonic,$arg)=@_;++	$arg =~ m/[qv]([0-9]+)[^,]*,\s*[qv]([0-9]+)[^,]*(?:,\s*[qv]([0-9]+))?/o+	&&+	sprintf ".inst\t0x%08x\t//%s %s",+			$opcode{$mnemonic}|$1|($2<<5)|($3<<16),+			$mnemonic,$arg;+    }+}++open SELF,$0;+while(<SELF>) {+        next if (/^#!/);+        last if (!s/^#/\/\// and !/^$/);+        print;+}+close SELF;++foreach(split("\n",$code)) {++	s/\`([^\`]*)\`/eval($1)/ge;++	s/\b(sha512\w+)\s+([qv].*)/unsha512($1,$2)/ge	or+	s/\b(sha256\w+)\s+([qv].*)/unsha256($1,$2)/ge;++	s/\bq([0-9]+)\b/v$1.16b/g;		# old->new registers++	s/\.[ui]?8(\s)/$1/;+	s/\.\w?64\b//		and s/\.16b/\.2d/g	or+	s/\.\w?32\b//		and s/\.16b/\.4s/g;+	m/\bext\b/		and s/\.2d/\.16b/g	or+	m/(ld|st)1[^\[]+\[0\]/	and s/\.4s/\.s/g;++	s/([cw])#x([0-9]+)/$1$2/g;++	print $_,"\n";+}++close STDOUT;
+ cbits/asm/sha512-x86_64-elf.S view
@@ -0,0 +1,5727 @@+.text	+++.globl	crypton_sha512_asm_block_data_order+.type	crypton_sha512_asm_block_data_order,@function+.align	16+crypton_sha512_asm_block_data_order:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+	leaq	crypton_ia32cap_P(%rip),%rax+	movl	0(%rax),%r9d+	movl	4(%rax),%r10d+	movl	8(%rax),%eax+	testl	$2048,%r10d+	jnz	.Lxop_shortcut+	andl	$296,%eax+	cmpl	$296,%eax+	je	.Lavx2_shortcut+	andl	$1073741824,%r9d+	andl	$268435968,%r10d+	orl	%r9d,%r10d+	cmpl	$1342177792,%r10d+	je	.Lavx_shortcut+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$128+24,%rsp++.cfi_def_cfa	%rsp,208++	leaq	(%rsi,%rdx,8),%rdx+	movq	%rdi,128+0(%rsp)+	movq	%rsi,128+8(%rsp)+	movq	%rdx,128+16(%rsp)++	movq	0(%rdi),%rax+	movq	8(%rdi),%rbx+	movq	16(%rdi),%rcx+	movq	24(%rdi),%rdx+	movq	32(%rdi),%r8+	movq	40(%rdi),%r9+	movq	48(%rdi),%r10+	movq	56(%rdi),%r11+	jmp	.Lloop++.align	16+.Lloop:+	movq	%rbx,%rdi+	leaq	K512(%rip),%rbp+	xorq	%rcx,%rdi+	movq	0(%rsi),%r12+	movq	%r8,%r13+	movq	%rax,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r9,%r15++	xorq	%r8,%r13+	rorq	$5,%r14+	xorq	%r10,%r15++	movq	%r12,0(%rsp)+	xorq	%rax,%r14+	andq	%r8,%r15++	rorq	$4,%r13+	addq	%r11,%r12+	xorq	%r10,%r15++	rorq	$6,%r14+	xorq	%r8,%r13+	addq	%r15,%r12++	movq	%rax,%r15+	addq	(%rbp),%r12+	xorq	%rax,%r14++	xorq	%rbx,%r15+	rorq	$14,%r13+	movq	%rbx,%r11++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r11+	addq	%r12,%rdx+	addq	%r12,%r11++	leaq	8(%rbp),%rbp+	addq	%r14,%r11+	movq	8(%rsi),%r12+	movq	%rdx,%r13+	movq	%r11,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r8,%rdi++	xorq	%rdx,%r13+	rorq	$5,%r14+	xorq	%r9,%rdi++	movq	%r12,8(%rsp)+	xorq	%r11,%r14+	andq	%rdx,%rdi++	rorq	$4,%r13+	addq	%r10,%r12+	xorq	%r9,%rdi++	rorq	$6,%r14+	xorq	%rdx,%r13+	addq	%rdi,%r12++	movq	%r11,%rdi+	addq	(%rbp),%r12+	xorq	%r11,%r14++	xorq	%rax,%rdi+	rorq	$14,%r13+	movq	%rax,%r10++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r10+	addq	%r12,%rcx+	addq	%r12,%r10++	leaq	24(%rbp),%rbp+	addq	%r14,%r10+	movq	16(%rsi),%r12+	movq	%rcx,%r13+	movq	%r10,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rdx,%r15++	xorq	%rcx,%r13+	rorq	$5,%r14+	xorq	%r8,%r15++	movq	%r12,16(%rsp)+	xorq	%r10,%r14+	andq	%rcx,%r15++	rorq	$4,%r13+	addq	%r9,%r12+	xorq	%r8,%r15++	rorq	$6,%r14+	xorq	%rcx,%r13+	addq	%r15,%r12++	movq	%r10,%r15+	addq	(%rbp),%r12+	xorq	%r10,%r14++	xorq	%r11,%r15+	rorq	$14,%r13+	movq	%r11,%r9++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r9+	addq	%r12,%rbx+	addq	%r12,%r9++	leaq	8(%rbp),%rbp+	addq	%r14,%r9+	movq	24(%rsi),%r12+	movq	%rbx,%r13+	movq	%r9,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rcx,%rdi++	xorq	%rbx,%r13+	rorq	$5,%r14+	xorq	%rdx,%rdi++	movq	%r12,24(%rsp)+	xorq	%r9,%r14+	andq	%rbx,%rdi++	rorq	$4,%r13+	addq	%r8,%r12+	xorq	%rdx,%rdi++	rorq	$6,%r14+	xorq	%rbx,%r13+	addq	%rdi,%r12++	movq	%r9,%rdi+	addq	(%rbp),%r12+	xorq	%r9,%r14++	xorq	%r10,%rdi+	rorq	$14,%r13+	movq	%r10,%r8++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r8+	addq	%r12,%rax+	addq	%r12,%r8++	leaq	24(%rbp),%rbp+	addq	%r14,%r8+	movq	32(%rsi),%r12+	movq	%rax,%r13+	movq	%r8,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rbx,%r15++	xorq	%rax,%r13+	rorq	$5,%r14+	xorq	%rcx,%r15++	movq	%r12,32(%rsp)+	xorq	%r8,%r14+	andq	%rax,%r15++	rorq	$4,%r13+	addq	%rdx,%r12+	xorq	%rcx,%r15++	rorq	$6,%r14+	xorq	%rax,%r13+	addq	%r15,%r12++	movq	%r8,%r15+	addq	(%rbp),%r12+	xorq	%r8,%r14++	xorq	%r9,%r15+	rorq	$14,%r13+	movq	%r9,%rdx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rdx+	addq	%r12,%r11+	addq	%r12,%rdx++	leaq	8(%rbp),%rbp+	addq	%r14,%rdx+	movq	40(%rsi),%r12+	movq	%r11,%r13+	movq	%rdx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rax,%rdi++	xorq	%r11,%r13+	rorq	$5,%r14+	xorq	%rbx,%rdi++	movq	%r12,40(%rsp)+	xorq	%rdx,%r14+	andq	%r11,%rdi++	rorq	$4,%r13+	addq	%rcx,%r12+	xorq	%rbx,%rdi++	rorq	$6,%r14+	xorq	%r11,%r13+	addq	%rdi,%r12++	movq	%rdx,%rdi+	addq	(%rbp),%r12+	xorq	%rdx,%r14++	xorq	%r8,%rdi+	rorq	$14,%r13+	movq	%r8,%rcx++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rcx+	addq	%r12,%r10+	addq	%r12,%rcx++	leaq	24(%rbp),%rbp+	addq	%r14,%rcx+	movq	48(%rsi),%r12+	movq	%r10,%r13+	movq	%rcx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r11,%r15++	xorq	%r10,%r13+	rorq	$5,%r14+	xorq	%rax,%r15++	movq	%r12,48(%rsp)+	xorq	%rcx,%r14+	andq	%r10,%r15++	rorq	$4,%r13+	addq	%rbx,%r12+	xorq	%rax,%r15++	rorq	$6,%r14+	xorq	%r10,%r13+	addq	%r15,%r12++	movq	%rcx,%r15+	addq	(%rbp),%r12+	xorq	%rcx,%r14++	xorq	%rdx,%r15+	rorq	$14,%r13+	movq	%rdx,%rbx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rbx+	addq	%r12,%r9+	addq	%r12,%rbx++	leaq	8(%rbp),%rbp+	addq	%r14,%rbx+	movq	56(%rsi),%r12+	movq	%r9,%r13+	movq	%rbx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r10,%rdi++	xorq	%r9,%r13+	rorq	$5,%r14+	xorq	%r11,%rdi++	movq	%r12,56(%rsp)+	xorq	%rbx,%r14+	andq	%r9,%rdi++	rorq	$4,%r13+	addq	%rax,%r12+	xorq	%r11,%rdi++	rorq	$6,%r14+	xorq	%r9,%r13+	addq	%rdi,%r12++	movq	%rbx,%rdi+	addq	(%rbp),%r12+	xorq	%rbx,%r14++	xorq	%rcx,%rdi+	rorq	$14,%r13+	movq	%rcx,%rax++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rax+	addq	%r12,%r8+	addq	%r12,%rax++	leaq	24(%rbp),%rbp+	addq	%r14,%rax+	movq	64(%rsi),%r12+	movq	%r8,%r13+	movq	%rax,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r9,%r15++	xorq	%r8,%r13+	rorq	$5,%r14+	xorq	%r10,%r15++	movq	%r12,64(%rsp)+	xorq	%rax,%r14+	andq	%r8,%r15++	rorq	$4,%r13+	addq	%r11,%r12+	xorq	%r10,%r15++	rorq	$6,%r14+	xorq	%r8,%r13+	addq	%r15,%r12++	movq	%rax,%r15+	addq	(%rbp),%r12+	xorq	%rax,%r14++	xorq	%rbx,%r15+	rorq	$14,%r13+	movq	%rbx,%r11++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r11+	addq	%r12,%rdx+	addq	%r12,%r11++	leaq	8(%rbp),%rbp+	addq	%r14,%r11+	movq	72(%rsi),%r12+	movq	%rdx,%r13+	movq	%r11,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r8,%rdi++	xorq	%rdx,%r13+	rorq	$5,%r14+	xorq	%r9,%rdi++	movq	%r12,72(%rsp)+	xorq	%r11,%r14+	andq	%rdx,%rdi++	rorq	$4,%r13+	addq	%r10,%r12+	xorq	%r9,%rdi++	rorq	$6,%r14+	xorq	%rdx,%r13+	addq	%rdi,%r12++	movq	%r11,%rdi+	addq	(%rbp),%r12+	xorq	%r11,%r14++	xorq	%rax,%rdi+	rorq	$14,%r13+	movq	%rax,%r10++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r10+	addq	%r12,%rcx+	addq	%r12,%r10++	leaq	24(%rbp),%rbp+	addq	%r14,%r10+	movq	80(%rsi),%r12+	movq	%rcx,%r13+	movq	%r10,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rdx,%r15++	xorq	%rcx,%r13+	rorq	$5,%r14+	xorq	%r8,%r15++	movq	%r12,80(%rsp)+	xorq	%r10,%r14+	andq	%rcx,%r15++	rorq	$4,%r13+	addq	%r9,%r12+	xorq	%r8,%r15++	rorq	$6,%r14+	xorq	%rcx,%r13+	addq	%r15,%r12++	movq	%r10,%r15+	addq	(%rbp),%r12+	xorq	%r10,%r14++	xorq	%r11,%r15+	rorq	$14,%r13+	movq	%r11,%r9++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r9+	addq	%r12,%rbx+	addq	%r12,%r9++	leaq	8(%rbp),%rbp+	addq	%r14,%r9+	movq	88(%rsi),%r12+	movq	%rbx,%r13+	movq	%r9,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rcx,%rdi++	xorq	%rbx,%r13+	rorq	$5,%r14+	xorq	%rdx,%rdi++	movq	%r12,88(%rsp)+	xorq	%r9,%r14+	andq	%rbx,%rdi++	rorq	$4,%r13+	addq	%r8,%r12+	xorq	%rdx,%rdi++	rorq	$6,%r14+	xorq	%rbx,%r13+	addq	%rdi,%r12++	movq	%r9,%rdi+	addq	(%rbp),%r12+	xorq	%r9,%r14++	xorq	%r10,%rdi+	rorq	$14,%r13+	movq	%r10,%r8++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r8+	addq	%r12,%rax+	addq	%r12,%r8++	leaq	24(%rbp),%rbp+	addq	%r14,%r8+	movq	96(%rsi),%r12+	movq	%rax,%r13+	movq	%r8,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rbx,%r15++	xorq	%rax,%r13+	rorq	$5,%r14+	xorq	%rcx,%r15++	movq	%r12,96(%rsp)+	xorq	%r8,%r14+	andq	%rax,%r15++	rorq	$4,%r13+	addq	%rdx,%r12+	xorq	%rcx,%r15++	rorq	$6,%r14+	xorq	%rax,%r13+	addq	%r15,%r12++	movq	%r8,%r15+	addq	(%rbp),%r12+	xorq	%r8,%r14++	xorq	%r9,%r15+	rorq	$14,%r13+	movq	%r9,%rdx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rdx+	addq	%r12,%r11+	addq	%r12,%rdx++	leaq	8(%rbp),%rbp+	addq	%r14,%rdx+	movq	104(%rsi),%r12+	movq	%r11,%r13+	movq	%rdx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rax,%rdi++	xorq	%r11,%r13+	rorq	$5,%r14+	xorq	%rbx,%rdi++	movq	%r12,104(%rsp)+	xorq	%rdx,%r14+	andq	%r11,%rdi++	rorq	$4,%r13+	addq	%rcx,%r12+	xorq	%rbx,%rdi++	rorq	$6,%r14+	xorq	%r11,%r13+	addq	%rdi,%r12++	movq	%rdx,%rdi+	addq	(%rbp),%r12+	xorq	%rdx,%r14++	xorq	%r8,%rdi+	rorq	$14,%r13+	movq	%r8,%rcx++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rcx+	addq	%r12,%r10+	addq	%r12,%rcx++	leaq	24(%rbp),%rbp+	addq	%r14,%rcx+	movq	112(%rsi),%r12+	movq	%r10,%r13+	movq	%rcx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r11,%r15++	xorq	%r10,%r13+	rorq	$5,%r14+	xorq	%rax,%r15++	movq	%r12,112(%rsp)+	xorq	%rcx,%r14+	andq	%r10,%r15++	rorq	$4,%r13+	addq	%rbx,%r12+	xorq	%rax,%r15++	rorq	$6,%r14+	xorq	%r10,%r13+	addq	%r15,%r12++	movq	%rcx,%r15+	addq	(%rbp),%r12+	xorq	%rcx,%r14++	xorq	%rdx,%r15+	rorq	$14,%r13+	movq	%rdx,%rbx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rbx+	addq	%r12,%r9+	addq	%r12,%rbx++	leaq	8(%rbp),%rbp+	addq	%r14,%rbx+	movq	120(%rsi),%r12+	movq	%r9,%r13+	movq	%rbx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r10,%rdi++	xorq	%r9,%r13+	rorq	$5,%r14+	xorq	%r11,%rdi++	movq	%r12,120(%rsp)+	xorq	%rbx,%r14+	andq	%r9,%rdi++	rorq	$4,%r13+	addq	%rax,%r12+	xorq	%r11,%rdi++	rorq	$6,%r14+	xorq	%r9,%r13+	addq	%rdi,%r12++	movq	%rbx,%rdi+	addq	(%rbp),%r12+	xorq	%rbx,%r14++	xorq	%rcx,%rdi+	rorq	$14,%r13+	movq	%rcx,%rax++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rax+	addq	%r12,%r8+	addq	%r12,%rax++	leaq	24(%rbp),%rbp+	jmp	.Lrounds_16_xx+.align	16+.Lrounds_16_xx:+	movq	8(%rsp),%r13+	movq	112(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rax+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	72(%rsp),%r12++	addq	0(%rsp),%r12+	movq	%r8,%r13+	addq	%r15,%r12+	movq	%rax,%r14+	rorq	$23,%r13+	movq	%r9,%r15++	xorq	%r8,%r13+	rorq	$5,%r14+	xorq	%r10,%r15++	movq	%r12,0(%rsp)+	xorq	%rax,%r14+	andq	%r8,%r15++	rorq	$4,%r13+	addq	%r11,%r12+	xorq	%r10,%r15++	rorq	$6,%r14+	xorq	%r8,%r13+	addq	%r15,%r12++	movq	%rax,%r15+	addq	(%rbp),%r12+	xorq	%rax,%r14++	xorq	%rbx,%r15+	rorq	$14,%r13+	movq	%rbx,%r11++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r11+	addq	%r12,%rdx+	addq	%r12,%r11++	leaq	8(%rbp),%rbp+	movq	16(%rsp),%r13+	movq	120(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r11+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	80(%rsp),%r12++	addq	8(%rsp),%r12+	movq	%rdx,%r13+	addq	%rdi,%r12+	movq	%r11,%r14+	rorq	$23,%r13+	movq	%r8,%rdi++	xorq	%rdx,%r13+	rorq	$5,%r14+	xorq	%r9,%rdi++	movq	%r12,8(%rsp)+	xorq	%r11,%r14+	andq	%rdx,%rdi++	rorq	$4,%r13+	addq	%r10,%r12+	xorq	%r9,%rdi++	rorq	$6,%r14+	xorq	%rdx,%r13+	addq	%rdi,%r12++	movq	%r11,%rdi+	addq	(%rbp),%r12+	xorq	%r11,%r14++	xorq	%rax,%rdi+	rorq	$14,%r13+	movq	%rax,%r10++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r10+	addq	%r12,%rcx+	addq	%r12,%r10++	leaq	24(%rbp),%rbp+	movq	24(%rsp),%r13+	movq	0(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r10+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	88(%rsp),%r12++	addq	16(%rsp),%r12+	movq	%rcx,%r13+	addq	%r15,%r12+	movq	%r10,%r14+	rorq	$23,%r13+	movq	%rdx,%r15++	xorq	%rcx,%r13+	rorq	$5,%r14+	xorq	%r8,%r15++	movq	%r12,16(%rsp)+	xorq	%r10,%r14+	andq	%rcx,%r15++	rorq	$4,%r13+	addq	%r9,%r12+	xorq	%r8,%r15++	rorq	$6,%r14+	xorq	%rcx,%r13+	addq	%r15,%r12++	movq	%r10,%r15+	addq	(%rbp),%r12+	xorq	%r10,%r14++	xorq	%r11,%r15+	rorq	$14,%r13+	movq	%r11,%r9++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r9+	addq	%r12,%rbx+	addq	%r12,%r9++	leaq	8(%rbp),%rbp+	movq	32(%rsp),%r13+	movq	8(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r9+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	96(%rsp),%r12++	addq	24(%rsp),%r12+	movq	%rbx,%r13+	addq	%rdi,%r12+	movq	%r9,%r14+	rorq	$23,%r13+	movq	%rcx,%rdi++	xorq	%rbx,%r13+	rorq	$5,%r14+	xorq	%rdx,%rdi++	movq	%r12,24(%rsp)+	xorq	%r9,%r14+	andq	%rbx,%rdi++	rorq	$4,%r13+	addq	%r8,%r12+	xorq	%rdx,%rdi++	rorq	$6,%r14+	xorq	%rbx,%r13+	addq	%rdi,%r12++	movq	%r9,%rdi+	addq	(%rbp),%r12+	xorq	%r9,%r14++	xorq	%r10,%rdi+	rorq	$14,%r13+	movq	%r10,%r8++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r8+	addq	%r12,%rax+	addq	%r12,%r8++	leaq	24(%rbp),%rbp+	movq	40(%rsp),%r13+	movq	16(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r8+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	104(%rsp),%r12++	addq	32(%rsp),%r12+	movq	%rax,%r13+	addq	%r15,%r12+	movq	%r8,%r14+	rorq	$23,%r13+	movq	%rbx,%r15++	xorq	%rax,%r13+	rorq	$5,%r14+	xorq	%rcx,%r15++	movq	%r12,32(%rsp)+	xorq	%r8,%r14+	andq	%rax,%r15++	rorq	$4,%r13+	addq	%rdx,%r12+	xorq	%rcx,%r15++	rorq	$6,%r14+	xorq	%rax,%r13+	addq	%r15,%r12++	movq	%r8,%r15+	addq	(%rbp),%r12+	xorq	%r8,%r14++	xorq	%r9,%r15+	rorq	$14,%r13+	movq	%r9,%rdx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rdx+	addq	%r12,%r11+	addq	%r12,%rdx++	leaq	8(%rbp),%rbp+	movq	48(%rsp),%r13+	movq	24(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rdx+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	112(%rsp),%r12++	addq	40(%rsp),%r12+	movq	%r11,%r13+	addq	%rdi,%r12+	movq	%rdx,%r14+	rorq	$23,%r13+	movq	%rax,%rdi++	xorq	%r11,%r13+	rorq	$5,%r14+	xorq	%rbx,%rdi++	movq	%r12,40(%rsp)+	xorq	%rdx,%r14+	andq	%r11,%rdi++	rorq	$4,%r13+	addq	%rcx,%r12+	xorq	%rbx,%rdi++	rorq	$6,%r14+	xorq	%r11,%r13+	addq	%rdi,%r12++	movq	%rdx,%rdi+	addq	(%rbp),%r12+	xorq	%rdx,%r14++	xorq	%r8,%rdi+	rorq	$14,%r13+	movq	%r8,%rcx++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rcx+	addq	%r12,%r10+	addq	%r12,%rcx++	leaq	24(%rbp),%rbp+	movq	56(%rsp),%r13+	movq	32(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rcx+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	120(%rsp),%r12++	addq	48(%rsp),%r12+	movq	%r10,%r13+	addq	%r15,%r12+	movq	%rcx,%r14+	rorq	$23,%r13+	movq	%r11,%r15++	xorq	%r10,%r13+	rorq	$5,%r14+	xorq	%rax,%r15++	movq	%r12,48(%rsp)+	xorq	%rcx,%r14+	andq	%r10,%r15++	rorq	$4,%r13+	addq	%rbx,%r12+	xorq	%rax,%r15++	rorq	$6,%r14+	xorq	%r10,%r13+	addq	%r15,%r12++	movq	%rcx,%r15+	addq	(%rbp),%r12+	xorq	%rcx,%r14++	xorq	%rdx,%r15+	rorq	$14,%r13+	movq	%rdx,%rbx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rbx+	addq	%r12,%r9+	addq	%r12,%rbx++	leaq	8(%rbp),%rbp+	movq	64(%rsp),%r13+	movq	40(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rbx+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	0(%rsp),%r12++	addq	56(%rsp),%r12+	movq	%r9,%r13+	addq	%rdi,%r12+	movq	%rbx,%r14+	rorq	$23,%r13+	movq	%r10,%rdi++	xorq	%r9,%r13+	rorq	$5,%r14+	xorq	%r11,%rdi++	movq	%r12,56(%rsp)+	xorq	%rbx,%r14+	andq	%r9,%rdi++	rorq	$4,%r13+	addq	%rax,%r12+	xorq	%r11,%rdi++	rorq	$6,%r14+	xorq	%r9,%r13+	addq	%rdi,%r12++	movq	%rbx,%rdi+	addq	(%rbp),%r12+	xorq	%rbx,%r14++	xorq	%rcx,%rdi+	rorq	$14,%r13+	movq	%rcx,%rax++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rax+	addq	%r12,%r8+	addq	%r12,%rax++	leaq	24(%rbp),%rbp+	movq	72(%rsp),%r13+	movq	48(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rax+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	8(%rsp),%r12++	addq	64(%rsp),%r12+	movq	%r8,%r13+	addq	%r15,%r12+	movq	%rax,%r14+	rorq	$23,%r13+	movq	%r9,%r15++	xorq	%r8,%r13+	rorq	$5,%r14+	xorq	%r10,%r15++	movq	%r12,64(%rsp)+	xorq	%rax,%r14+	andq	%r8,%r15++	rorq	$4,%r13+	addq	%r11,%r12+	xorq	%r10,%r15++	rorq	$6,%r14+	xorq	%r8,%r13+	addq	%r15,%r12++	movq	%rax,%r15+	addq	(%rbp),%r12+	xorq	%rax,%r14++	xorq	%rbx,%r15+	rorq	$14,%r13+	movq	%rbx,%r11++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r11+	addq	%r12,%rdx+	addq	%r12,%r11++	leaq	8(%rbp),%rbp+	movq	80(%rsp),%r13+	movq	56(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r11+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	16(%rsp),%r12++	addq	72(%rsp),%r12+	movq	%rdx,%r13+	addq	%rdi,%r12+	movq	%r11,%r14+	rorq	$23,%r13+	movq	%r8,%rdi++	xorq	%rdx,%r13+	rorq	$5,%r14+	xorq	%r9,%rdi++	movq	%r12,72(%rsp)+	xorq	%r11,%r14+	andq	%rdx,%rdi++	rorq	$4,%r13+	addq	%r10,%r12+	xorq	%r9,%rdi++	rorq	$6,%r14+	xorq	%rdx,%r13+	addq	%rdi,%r12++	movq	%r11,%rdi+	addq	(%rbp),%r12+	xorq	%r11,%r14++	xorq	%rax,%rdi+	rorq	$14,%r13+	movq	%rax,%r10++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r10+	addq	%r12,%rcx+	addq	%r12,%r10++	leaq	24(%rbp),%rbp+	movq	88(%rsp),%r13+	movq	64(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r10+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	24(%rsp),%r12++	addq	80(%rsp),%r12+	movq	%rcx,%r13+	addq	%r15,%r12+	movq	%r10,%r14+	rorq	$23,%r13+	movq	%rdx,%r15++	xorq	%rcx,%r13+	rorq	$5,%r14+	xorq	%r8,%r15++	movq	%r12,80(%rsp)+	xorq	%r10,%r14+	andq	%rcx,%r15++	rorq	$4,%r13+	addq	%r9,%r12+	xorq	%r8,%r15++	rorq	$6,%r14+	xorq	%rcx,%r13+	addq	%r15,%r12++	movq	%r10,%r15+	addq	(%rbp),%r12+	xorq	%r10,%r14++	xorq	%r11,%r15+	rorq	$14,%r13+	movq	%r11,%r9++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r9+	addq	%r12,%rbx+	addq	%r12,%r9++	leaq	8(%rbp),%rbp+	movq	96(%rsp),%r13+	movq	72(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r9+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	32(%rsp),%r12++	addq	88(%rsp),%r12+	movq	%rbx,%r13+	addq	%rdi,%r12+	movq	%r9,%r14+	rorq	$23,%r13+	movq	%rcx,%rdi++	xorq	%rbx,%r13+	rorq	$5,%r14+	xorq	%rdx,%rdi++	movq	%r12,88(%rsp)+	xorq	%r9,%r14+	andq	%rbx,%rdi++	rorq	$4,%r13+	addq	%r8,%r12+	xorq	%rdx,%rdi++	rorq	$6,%r14+	xorq	%rbx,%r13+	addq	%rdi,%r12++	movq	%r9,%rdi+	addq	(%rbp),%r12+	xorq	%r9,%r14++	xorq	%r10,%rdi+	rorq	$14,%r13+	movq	%r10,%r8++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r8+	addq	%r12,%rax+	addq	%r12,%r8++	leaq	24(%rbp),%rbp+	movq	104(%rsp),%r13+	movq	80(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r8+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	40(%rsp),%r12++	addq	96(%rsp),%r12+	movq	%rax,%r13+	addq	%r15,%r12+	movq	%r8,%r14+	rorq	$23,%r13+	movq	%rbx,%r15++	xorq	%rax,%r13+	rorq	$5,%r14+	xorq	%rcx,%r15++	movq	%r12,96(%rsp)+	xorq	%r8,%r14+	andq	%rax,%r15++	rorq	$4,%r13+	addq	%rdx,%r12+	xorq	%rcx,%r15++	rorq	$6,%r14+	xorq	%rax,%r13+	addq	%r15,%r12++	movq	%r8,%r15+	addq	(%rbp),%r12+	xorq	%r8,%r14++	xorq	%r9,%r15+	rorq	$14,%r13+	movq	%r9,%rdx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rdx+	addq	%r12,%r11+	addq	%r12,%rdx++	leaq	8(%rbp),%rbp+	movq	112(%rsp),%r13+	movq	88(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rdx+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	48(%rsp),%r12++	addq	104(%rsp),%r12+	movq	%r11,%r13+	addq	%rdi,%r12+	movq	%rdx,%r14+	rorq	$23,%r13+	movq	%rax,%rdi++	xorq	%r11,%r13+	rorq	$5,%r14+	xorq	%rbx,%rdi++	movq	%r12,104(%rsp)+	xorq	%rdx,%r14+	andq	%r11,%rdi++	rorq	$4,%r13+	addq	%rcx,%r12+	xorq	%rbx,%rdi++	rorq	$6,%r14+	xorq	%r11,%r13+	addq	%rdi,%r12++	movq	%rdx,%rdi+	addq	(%rbp),%r12+	xorq	%rdx,%r14++	xorq	%r8,%rdi+	rorq	$14,%r13+	movq	%r8,%rcx++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rcx+	addq	%r12,%r10+	addq	%r12,%rcx++	leaq	24(%rbp),%rbp+	movq	120(%rsp),%r13+	movq	96(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rcx+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	56(%rsp),%r12++	addq	112(%rsp),%r12+	movq	%r10,%r13+	addq	%r15,%r12+	movq	%rcx,%r14+	rorq	$23,%r13+	movq	%r11,%r15++	xorq	%r10,%r13+	rorq	$5,%r14+	xorq	%rax,%r15++	movq	%r12,112(%rsp)+	xorq	%rcx,%r14+	andq	%r10,%r15++	rorq	$4,%r13+	addq	%rbx,%r12+	xorq	%rax,%r15++	rorq	$6,%r14+	xorq	%r10,%r13+	addq	%r15,%r12++	movq	%rcx,%r15+	addq	(%rbp),%r12+	xorq	%rcx,%r14++	xorq	%rdx,%r15+	rorq	$14,%r13+	movq	%rdx,%rbx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rbx+	addq	%r12,%r9+	addq	%r12,%rbx++	leaq	8(%rbp),%rbp+	movq	0(%rsp),%r13+	movq	104(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rbx+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	64(%rsp),%r12++	addq	120(%rsp),%r12+	movq	%r9,%r13+	addq	%rdi,%r12+	movq	%rbx,%r14+	rorq	$23,%r13+	movq	%r10,%rdi++	xorq	%r9,%r13+	rorq	$5,%r14+	xorq	%r11,%rdi++	movq	%r12,120(%rsp)+	xorq	%rbx,%r14+	andq	%r9,%rdi++	rorq	$4,%r13+	addq	%rax,%r12+	xorq	%r11,%rdi++	rorq	$6,%r14+	xorq	%r9,%r13+	addq	%rdi,%r12++	movq	%rbx,%rdi+	addq	(%rbp),%r12+	xorq	%rbx,%r14++	xorq	%rcx,%rdi+	rorq	$14,%r13+	movq	%rcx,%rax++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rax+	addq	%r12,%r8+	addq	%r12,%rax++	leaq	24(%rbp),%rbp+	cmpb	$0,7(%rbp)+	jnz	.Lrounds_16_xx++	movq	128+0(%rsp),%rdi+	addq	%r14,%rax+	leaq	128(%rsi),%rsi++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	addq	48(%rdi),%r10+	addq	56(%rdi),%r11++	cmpq	128+16(%rsp),%rsi++	movq	%rax,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)+	jb	.Lloop++	leaq	128+24+48(%rsp),%r11+.cfi_def_cfa	%r11,8+	movq	128+24(%rsp),%r15+	movq	-40(%r11),%r14+	movq	-32(%r11),%r13+	movq	-24(%r11),%r12+	movq	-16(%r11),%rbx+	movq	-8(%r11),%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbp+.cfi_restore	%rbx+	leaq	(%r11),%rsp+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_sha512_asm_block_data_order,.-crypton_sha512_asm_block_data_order+.align	64+.type	K512,@object+K512:+.quad	0x428a2f98d728ae22,0x7137449123ef65cd+.quad	0x428a2f98d728ae22,0x7137449123ef65cd+.quad	0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc+.quad	0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc+.quad	0x3956c25bf348b538,0x59f111f1b605d019+.quad	0x3956c25bf348b538,0x59f111f1b605d019+.quad	0x923f82a4af194f9b,0xab1c5ed5da6d8118+.quad	0x923f82a4af194f9b,0xab1c5ed5da6d8118+.quad	0xd807aa98a3030242,0x12835b0145706fbe+.quad	0xd807aa98a3030242,0x12835b0145706fbe+.quad	0x243185be4ee4b28c,0x550c7dc3d5ffb4e2+.quad	0x243185be4ee4b28c,0x550c7dc3d5ffb4e2+.quad	0x72be5d74f27b896f,0x80deb1fe3b1696b1+.quad	0x72be5d74f27b896f,0x80deb1fe3b1696b1+.quad	0x9bdc06a725c71235,0xc19bf174cf692694+.quad	0x9bdc06a725c71235,0xc19bf174cf692694+.quad	0xe49b69c19ef14ad2,0xefbe4786384f25e3+.quad	0xe49b69c19ef14ad2,0xefbe4786384f25e3+.quad	0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65+.quad	0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65+.quad	0x2de92c6f592b0275,0x4a7484aa6ea6e483+.quad	0x2de92c6f592b0275,0x4a7484aa6ea6e483+.quad	0x5cb0a9dcbd41fbd4,0x76f988da831153b5+.quad	0x5cb0a9dcbd41fbd4,0x76f988da831153b5+.quad	0x983e5152ee66dfab,0xa831c66d2db43210+.quad	0x983e5152ee66dfab,0xa831c66d2db43210+.quad	0xb00327c898fb213f,0xbf597fc7beef0ee4+.quad	0xb00327c898fb213f,0xbf597fc7beef0ee4+.quad	0xc6e00bf33da88fc2,0xd5a79147930aa725+.quad	0xc6e00bf33da88fc2,0xd5a79147930aa725+.quad	0x06ca6351e003826f,0x142929670a0e6e70+.quad	0x06ca6351e003826f,0x142929670a0e6e70+.quad	0x27b70a8546d22ffc,0x2e1b21385c26c926+.quad	0x27b70a8546d22ffc,0x2e1b21385c26c926+.quad	0x4d2c6dfc5ac42aed,0x53380d139d95b3df+.quad	0x4d2c6dfc5ac42aed,0x53380d139d95b3df+.quad	0x650a73548baf63de,0x766a0abb3c77b2a8+.quad	0x650a73548baf63de,0x766a0abb3c77b2a8+.quad	0x81c2c92e47edaee6,0x92722c851482353b+.quad	0x81c2c92e47edaee6,0x92722c851482353b+.quad	0xa2bfe8a14cf10364,0xa81a664bbc423001+.quad	0xa2bfe8a14cf10364,0xa81a664bbc423001+.quad	0xc24b8b70d0f89791,0xc76c51a30654be30+.quad	0xc24b8b70d0f89791,0xc76c51a30654be30+.quad	0xd192e819d6ef5218,0xd69906245565a910+.quad	0xd192e819d6ef5218,0xd69906245565a910+.quad	0xf40e35855771202a,0x106aa07032bbd1b8+.quad	0xf40e35855771202a,0x106aa07032bbd1b8+.quad	0x19a4c116b8d2d0c8,0x1e376c085141ab53+.quad	0x19a4c116b8d2d0c8,0x1e376c085141ab53+.quad	0x2748774cdf8eeb99,0x34b0bcb5e19b48a8+.quad	0x2748774cdf8eeb99,0x34b0bcb5e19b48a8+.quad	0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb+.quad	0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb+.quad	0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3+.quad	0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3+.quad	0x748f82ee5defb2fc,0x78a5636f43172f60+.quad	0x748f82ee5defb2fc,0x78a5636f43172f60+.quad	0x84c87814a1f0ab72,0x8cc702081a6439ec+.quad	0x84c87814a1f0ab72,0x8cc702081a6439ec+.quad	0x90befffa23631e28,0xa4506cebde82bde9+.quad	0x90befffa23631e28,0xa4506cebde82bde9+.quad	0xbef9a3f7b2c67915,0xc67178f2e372532b+.quad	0xbef9a3f7b2c67915,0xc67178f2e372532b+.quad	0xca273eceea26619c,0xd186b8c721c0c207+.quad	0xca273eceea26619c,0xd186b8c721c0c207+.quad	0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178+.quad	0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178+.quad	0x06f067aa72176fba,0x0a637dc5a2c898a6+.quad	0x06f067aa72176fba,0x0a637dc5a2c898a6+.quad	0x113f9804bef90dae,0x1b710b35131c471b+.quad	0x113f9804bef90dae,0x1b710b35131c471b+.quad	0x28db77f523047d84,0x32caab7b40c72493+.quad	0x28db77f523047d84,0x32caab7b40c72493+.quad	0x3c9ebe0a15c9bebc,0x431d67c49c100d4c+.quad	0x3c9ebe0a15c9bebc,0x431d67c49c100d4c+.quad	0x4cc5d4becb3e42b6,0x597f299cfc657e2a+.quad	0x4cc5d4becb3e42b6,0x597f299cfc657e2a+.quad	0x5fcb6fab3ad6faec,0x6c44198c4a475817+.quad	0x5fcb6fab3ad6faec,0x6c44198c4a475817++.quad	0x0001020304050607,0x08090a0b0c0d0e0f+.quad	0x0001020304050607,0x08090a0b0c0d0e0f++K512_nodup:+.quad	0x428a2f98d728ae22,0x7137449123ef65cd+.quad	0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc+.quad	0x3956c25bf348b538,0x59f111f1b605d019+.quad	0x923f82a4af194f9b,0xab1c5ed5da6d8118+.quad	0xd807aa98a3030242,0x12835b0145706fbe+.quad	0x243185be4ee4b28c,0x550c7dc3d5ffb4e2+.quad	0x72be5d74f27b896f,0x80deb1fe3b1696b1+.quad	0x9bdc06a725c71235,0xc19bf174cf692694+.quad	0xe49b69c19ef14ad2,0xefbe4786384f25e3+.quad	0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65+.quad	0x2de92c6f592b0275,0x4a7484aa6ea6e483+.quad	0x5cb0a9dcbd41fbd4,0x76f988da831153b5+.quad	0x983e5152ee66dfab,0xa831c66d2db43210+.quad	0xb00327c898fb213f,0xbf597fc7beef0ee4+.quad	0xc6e00bf33da88fc2,0xd5a79147930aa725+.quad	0x06ca6351e003826f,0x142929670a0e6e70+.quad	0x27b70a8546d22ffc,0x2e1b21385c26c926+.quad	0x4d2c6dfc5ac42aed,0x53380d139d95b3df+.quad	0x650a73548baf63de,0x766a0abb3c77b2a8+.quad	0x81c2c92e47edaee6,0x92722c851482353b+.quad	0xa2bfe8a14cf10364,0xa81a664bbc423001+.quad	0xc24b8b70d0f89791,0xc76c51a30654be30+.quad	0xd192e819d6ef5218,0xd69906245565a910+.quad	0xf40e35855771202a,0x106aa07032bbd1b8+.quad	0x19a4c116b8d2d0c8,0x1e376c085141ab53+.quad	0x2748774cdf8eeb99,0x34b0bcb5e19b48a8+.quad	0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb+.quad	0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3+.quad	0x748f82ee5defb2fc,0x78a5636f43172f60+.quad	0x84c87814a1f0ab72,0x8cc702081a6439ec+.quad	0x90befffa23631e28,0xa4506cebde82bde9+.quad	0xbef9a3f7b2c67915,0xc67178f2e372532b+.quad	0xca273eceea26619c,0xd186b8c721c0c207+.quad	0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178+.quad	0x06f067aa72176fba,0x0a637dc5a2c898a6+.quad	0x113f9804bef90dae,0x1b710b35131c471b+.quad	0x28db77f523047d84,0x32caab7b40c72493+.quad	0x3c9ebe0a15c9bebc,0x431d67c49c100d4c+.quad	0x4cc5d4becb3e42b6,0x597f299cfc657e2a+.quad	0x5fcb6fab3ad6faec,0x6c44198c4a475817++.byte	83,72,65,53,49,50,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.globl	crypton_sha512_asm_block_data_order_shaext+.type	crypton_sha512_asm_block_data_order_shaext,@function+.align	64+crypton_sha512_asm_block_data_order_shaext:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+.Lshaext_shortcut:++	leaq	K512_nodup+128(%rip),%rcx+	vmovdqu	(%rdi),%ymm0+	vmovdqu	32(%rdi),%ymm1+	vmovdqa	-160(%rcx),%ymm8++	vpermq	$27,%ymm0,%ymm0+	vpblendd	$15,%ymm1,%ymm0,%ymm5+	vpblendd	$15,%ymm0,%ymm1,%ymm6+	vpermq	$225,%ymm5,%ymm5+	vpermq	$75,%ymm6,%ymm6+	jmp	.Loop_shaext++.align	16+.Loop_shaext:+	vmovdqu	(%rsi),%ymm0+	vmovdqu	32(%rsi),%ymm1+	vmovdqu	64(%rsi),%ymm2+	vpshufb	%ymm8,%ymm0,%ymm0+	vmovdqu	96(%rsi),%ymm3++	vpaddq	0-128(%rcx),%ymm0,%ymm4+	vpshufb	%ymm8,%ymm1,%ymm1+	vmovdqa	%ymm6,%ymm10+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vmovdqa	%ymm5,%ymm9+.byte	196,226,79,203,236++	vpaddq	32-128(%rcx),%ymm1,%ymm4+	vpshufb	%ymm8,%ymm2,%ymm2+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	leaq	128(%rsi),%rsi+.byte	196,226,127,204,193+.byte	196,226,79,203,236++	vpaddq	64-128(%rcx),%ymm2,%ymm4+	vpshufb	%ymm8,%ymm3,%ymm3+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm3,%ymm2,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm0,%ymm0+.byte	196,226,127,204,202+.byte	196,226,79,203,236++	vpaddq	96-128(%rcx),%ymm3,%ymm4+.byte	196,226,127,205,195+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm0,%ymm3,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm1,%ymm1+.byte	196,226,127,204,211+.byte	196,226,79,203,236+	vpaddq	128-128(%rcx),%ymm0,%ymm4+.byte	196,226,127,205,200+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm1,%ymm0,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm2,%ymm2+.byte	196,226,127,204,216+.byte	196,226,79,203,236+	vpaddq	160-128(%rcx),%ymm1,%ymm4+.byte	196,226,127,205,209+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm2,%ymm1,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm3,%ymm3+.byte	196,226,127,204,193+.byte	196,226,79,203,236+	vpaddq	192-128(%rcx),%ymm2,%ymm4+.byte	196,226,127,205,218+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm3,%ymm2,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm0,%ymm0+.byte	196,226,127,204,202+.byte	196,226,79,203,236+	vpaddq	224-128(%rcx),%ymm3,%ymm4+.byte	196,226,127,205,195+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm0,%ymm3,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm1,%ymm1+.byte	196,226,127,204,211+.byte	196,226,79,203,236+	vpaddq	256-128(%rcx),%ymm0,%ymm4+.byte	196,226,127,205,200+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm1,%ymm0,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm2,%ymm2+.byte	196,226,127,204,216+.byte	196,226,79,203,236+	vpaddq	288-128(%rcx),%ymm1,%ymm4+.byte	196,226,127,205,209+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm2,%ymm1,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm3,%ymm3+.byte	196,226,127,204,193+.byte	196,226,79,203,236+	vpaddq	320-128(%rcx),%ymm2,%ymm4+.byte	196,226,127,205,218+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm3,%ymm2,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm0,%ymm0+.byte	196,226,127,204,202+.byte	196,226,79,203,236+	vpaddq	352-128(%rcx),%ymm3,%ymm4+.byte	196,226,127,205,195+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm0,%ymm3,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm1,%ymm1+.byte	196,226,127,204,211+.byte	196,226,79,203,236+	vpaddq	384-128(%rcx),%ymm0,%ymm4+.byte	196,226,127,205,200+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm1,%ymm0,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm2,%ymm2+.byte	196,226,127,204,216+.byte	196,226,79,203,236+	vpaddq	416-128(%rcx),%ymm1,%ymm4+.byte	196,226,127,205,209+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm2,%ymm1,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm3,%ymm3+.byte	196,226,127,204,193+.byte	196,226,79,203,236+	vpaddq	448-128(%rcx),%ymm2,%ymm4+.byte	196,226,127,205,218+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm3,%ymm2,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm0,%ymm0+.byte	196,226,127,204,202+.byte	196,226,79,203,236+	vpaddq	480-128(%rcx),%ymm3,%ymm4+.byte	196,226,127,205,195+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm0,%ymm3,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm1,%ymm1+.byte	196,226,127,204,211+.byte	196,226,79,203,236+	vpaddq	512-128(%rcx),%ymm0,%ymm4+.byte	196,226,127,205,200+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm1,%ymm0,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm2,%ymm2+.byte	196,226,127,204,216+.byte	196,226,79,203,236+	vpaddq	544-128(%rcx),%ymm1,%ymm4+.byte	196,226,127,205,209+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm2,%ymm1,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+.byte	196,226,79,203,236+	vpaddq	%ymm7,%ymm3,%ymm3++	vpaddq	576-128(%rcx),%ymm2,%ymm4+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+.byte	196,226,127,205,218+.byte	196,226,79,203,236++	vpaddq	608-128(%rcx),%ymm3,%ymm4+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	decq	%rdx+.byte	196,226,79,203,236++	vpaddq	%ymm10,%ymm6,%ymm6+	vpaddq	%ymm9,%ymm5,%ymm5+	jnz	.Loop_shaext++	vpermq	$75,%ymm5,%ymm5+	vpblendd	$240,%ymm6,%ymm5,%ymm1+	vpblendd	$240,%ymm5,%ymm6,%ymm2+	vpermq	$180,%ymm1,%ymm1+	vpermq	$27,%ymm2,%ymm2++	vmovdqu	%ymm1,(%rdi)+	vmovdqu	%ymm2,32(%rdi)++	vzeroupper+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp++	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_sha512_asm_block_data_order_shaext,.-crypton_sha512_asm_block_data_order_shaext+.type	crypton_sha512_asm_block_data_order_xop,@function+.align	64+crypton_sha512_asm_block_data_order_xop:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+.Lxop_shortcut:+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$24,%rsp++	leaq	(%rsi,%rdx,8),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+++	leaq	-128(%rsp),%rsp+	vzeroupper+	andq	$-64,%rsp+	movq	0(%rdi),%rax+	movq	8(%rdi),%rbx+	movq	16(%rdi),%rcx+	movq	24(%rdi),%rdx+	movq	32(%rdi),%r8+	movq	40(%rdi),%r9+	movq	48(%rdi),%r10+	movq	56(%rdi),%r11+	jmp	.Lloop_xop+.align	16+.Lloop_xop:+	vmovdqa	K512+1280(%rip),%xmm11+	movq	%rsi,-56(%rbp)+	vmovdqu	0(%rsi),%xmm0+	vmovdqu	16(%rsi),%xmm1+	vmovdqu	32(%rsi),%xmm2+	vpshufb	%xmm11,%xmm0,%xmm0+	vmovdqu	48(%rsi),%xmm3+	vpshufb	%xmm11,%xmm1,%xmm1+	vmovdqu	64(%rsi),%xmm4+	vpshufb	%xmm11,%xmm2,%xmm2+	vmovdqu	80(%rsi),%xmm5+	vpshufb	%xmm11,%xmm3,%xmm3+	vmovdqu	96(%rsi),%xmm6+	vpshufb	%xmm11,%xmm4,%xmm4+	vmovdqu	112(%rsi),%xmm7+	leaq	K512+128(%rip),%rsi+	vpshufb	%xmm11,%xmm5,%xmm5+	vpaddq	-128(%rsi),%xmm0,%xmm8+	vpshufb	%xmm11,%xmm6,%xmm6+	vpaddq	-96(%rsi),%xmm1,%xmm9+	vpshufb	%xmm11,%xmm7,%xmm7+	vpaddq	-64(%rsi),%xmm2,%xmm10+	vpaddq	-32(%rsi),%xmm3,%xmm11+	vmovdqa	%xmm8,0(%rsp)+	vpaddq	0(%rsi),%xmm4,%xmm8+	vmovdqa	%xmm9,16(%rsp)+	vpaddq	32(%rsi),%xmm5,%xmm9+	vmovdqa	%xmm10,32(%rsp)+	vpaddq	64(%rsi),%xmm6,%xmm10+	vmovdqa	%xmm11,48(%rsp)+	vpaddq	96(%rsi),%xmm7,%xmm11+	vmovdqa	%xmm8,64(%rsp)+	movq	%rax,%r14+	vmovdqa	%xmm9,80(%rsp)+	movq	%rbx,%rdi+	vmovdqa	%xmm10,96(%rsp)+	xorq	%rcx,%rdi+	vmovdqa	%xmm11,112(%rsp)+	movq	%r8,%r13+	jmp	.Lxop_00_47++.align	16+.Lxop_00_47:+	addq	$256,%rsi+	vpalignr	$8,%xmm0,%xmm1,%xmm8+	rorq	$23,%r13+	movq	%r14,%rax+	vpalignr	$8,%xmm4,%xmm5,%xmm11+	movq	%r9,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%r8,%r13+	xorq	%r10,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%rax,%r14+	vpaddq	%xmm11,%xmm0,%xmm0+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	0(%rsp),%r11+	movq	%rax,%r15+.byte	143,72,120,195,209,7+	xorq	%r10,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%rbx,%r15+	addq	%r12,%r11+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,223,3+	xorq	%rax,%r14+	addq	%r13,%r11+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rbx,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm7,%xmm10+	addq	%r11,%rdx+	addq	%rdi,%r11+	vpaddq	%xmm8,%xmm0,%xmm0+	movq	%rdx,%r13+	addq	%r11,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%r11+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%r8,%r12+	rorq	$5,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	vpaddq	%xmm11,%xmm0,%xmm0+	addq	8(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	rorq	$6,%r14+	vpaddq	-128(%rsi),%xmm0,%xmm10+	xorq	%rax,%rdi+	addq	%r12,%r10+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	rorq	$28,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	vmovdqa	%xmm10,0(%rsp)+	vpalignr	$8,%xmm1,%xmm2,%xmm8+	rorq	$23,%r13+	movq	%r14,%r10+	vpalignr	$8,%xmm5,%xmm6,%xmm11+	movq	%rdx,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%rcx,%r13+	xorq	%r8,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%r10,%r14+	vpaddq	%xmm11,%xmm1,%xmm1+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	16(%rsp),%r9+	movq	%r10,%r15+.byte	143,72,120,195,209,7+	xorq	%r8,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%r11,%r15+	addq	%r12,%r9+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,216,3+	xorq	%r10,%r14+	addq	%r13,%r9+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r11,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm0,%xmm10+	addq	%r9,%rbx+	addq	%rdi,%r9+	vpaddq	%xmm8,%xmm1,%xmm1+	movq	%rbx,%r13+	addq	%r9,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%r9+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%rcx,%r12+	rorq	$5,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	vpaddq	%xmm11,%xmm1,%xmm1+	addq	24(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	rorq	$6,%r14+	vpaddq	-96(%rsi),%xmm1,%xmm10+	xorq	%r10,%rdi+	addq	%r12,%r8+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	rorq	$28,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	vmovdqa	%xmm10,16(%rsp)+	vpalignr	$8,%xmm2,%xmm3,%xmm8+	rorq	$23,%r13+	movq	%r14,%r8+	vpalignr	$8,%xmm6,%xmm7,%xmm11+	movq	%rbx,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%rax,%r13+	xorq	%rcx,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%r8,%r14+	vpaddq	%xmm11,%xmm2,%xmm2+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	32(%rsp),%rdx+	movq	%r8,%r15+.byte	143,72,120,195,209,7+	xorq	%rcx,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%r9,%r15+	addq	%r12,%rdx+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,217,3+	xorq	%r8,%r14+	addq	%r13,%rdx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r9,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm1,%xmm10+	addq	%rdx,%r11+	addq	%rdi,%rdx+	vpaddq	%xmm8,%xmm2,%xmm2+	movq	%r11,%r13+	addq	%rdx,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%rdx+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%rax,%r12+	rorq	$5,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	vpaddq	%xmm11,%xmm2,%xmm2+	addq	40(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	rorq	$6,%r14+	vpaddq	-64(%rsi),%xmm2,%xmm10+	xorq	%r8,%rdi+	addq	%r12,%rcx+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	rorq	$28,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	vmovdqa	%xmm10,32(%rsp)+	vpalignr	$8,%xmm3,%xmm4,%xmm8+	rorq	$23,%r13+	movq	%r14,%rcx+	vpalignr	$8,%xmm7,%xmm0,%xmm11+	movq	%r11,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%r10,%r13+	xorq	%rax,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%rcx,%r14+	vpaddq	%xmm11,%xmm3,%xmm3+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	48(%rsp),%rbx+	movq	%rcx,%r15+.byte	143,72,120,195,209,7+	xorq	%rax,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%rdx,%r15+	addq	%r12,%rbx+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,218,3+	xorq	%rcx,%r14+	addq	%r13,%rbx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rdx,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm2,%xmm10+	addq	%rbx,%r9+	addq	%rdi,%rbx+	vpaddq	%xmm8,%xmm3,%xmm3+	movq	%r9,%r13+	addq	%rbx,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%rbx+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%r10,%r12+	rorq	$5,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	vpaddq	%xmm11,%xmm3,%xmm3+	addq	56(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	rorq	$6,%r14+	vpaddq	-32(%rsi),%xmm3,%xmm10+	xorq	%rcx,%rdi+	addq	%r12,%rax+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	rorq	$28,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	vmovdqa	%xmm10,48(%rsp)+	vpalignr	$8,%xmm4,%xmm5,%xmm8+	rorq	$23,%r13+	movq	%r14,%rax+	vpalignr	$8,%xmm0,%xmm1,%xmm11+	movq	%r9,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%r8,%r13+	xorq	%r10,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%rax,%r14+	vpaddq	%xmm11,%xmm4,%xmm4+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	64(%rsp),%r11+	movq	%rax,%r15+.byte	143,72,120,195,209,7+	xorq	%r10,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%rbx,%r15+	addq	%r12,%r11+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,219,3+	xorq	%rax,%r14+	addq	%r13,%r11+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rbx,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm3,%xmm10+	addq	%r11,%rdx+	addq	%rdi,%r11+	vpaddq	%xmm8,%xmm4,%xmm4+	movq	%rdx,%r13+	addq	%r11,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%r11+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%r8,%r12+	rorq	$5,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	vpaddq	%xmm11,%xmm4,%xmm4+	addq	72(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	rorq	$6,%r14+	vpaddq	0(%rsi),%xmm4,%xmm10+	xorq	%rax,%rdi+	addq	%r12,%r10+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	rorq	$28,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	vmovdqa	%xmm10,64(%rsp)+	vpalignr	$8,%xmm5,%xmm6,%xmm8+	rorq	$23,%r13+	movq	%r14,%r10+	vpalignr	$8,%xmm1,%xmm2,%xmm11+	movq	%rdx,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%rcx,%r13+	xorq	%r8,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%r10,%r14+	vpaddq	%xmm11,%xmm5,%xmm5+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	80(%rsp),%r9+	movq	%r10,%r15+.byte	143,72,120,195,209,7+	xorq	%r8,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%r11,%r15+	addq	%r12,%r9+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,220,3+	xorq	%r10,%r14+	addq	%r13,%r9+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r11,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm4,%xmm10+	addq	%r9,%rbx+	addq	%rdi,%r9+	vpaddq	%xmm8,%xmm5,%xmm5+	movq	%rbx,%r13+	addq	%r9,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%r9+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%rcx,%r12+	rorq	$5,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	vpaddq	%xmm11,%xmm5,%xmm5+	addq	88(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	rorq	$6,%r14+	vpaddq	32(%rsi),%xmm5,%xmm10+	xorq	%r10,%rdi+	addq	%r12,%r8+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	rorq	$28,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	vmovdqa	%xmm10,80(%rsp)+	vpalignr	$8,%xmm6,%xmm7,%xmm8+	rorq	$23,%r13+	movq	%r14,%r8+	vpalignr	$8,%xmm2,%xmm3,%xmm11+	movq	%rbx,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%rax,%r13+	xorq	%rcx,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%r8,%r14+	vpaddq	%xmm11,%xmm6,%xmm6+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	96(%rsp),%rdx+	movq	%r8,%r15+.byte	143,72,120,195,209,7+	xorq	%rcx,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%r9,%r15+	addq	%r12,%rdx+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,221,3+	xorq	%r8,%r14+	addq	%r13,%rdx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r9,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm5,%xmm10+	addq	%rdx,%r11+	addq	%rdi,%rdx+	vpaddq	%xmm8,%xmm6,%xmm6+	movq	%r11,%r13+	addq	%rdx,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%rdx+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%rax,%r12+	rorq	$5,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	vpaddq	%xmm11,%xmm6,%xmm6+	addq	104(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	rorq	$6,%r14+	vpaddq	64(%rsi),%xmm6,%xmm10+	xorq	%r8,%rdi+	addq	%r12,%rcx+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	rorq	$28,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	vmovdqa	%xmm10,96(%rsp)+	vpalignr	$8,%xmm7,%xmm0,%xmm8+	rorq	$23,%r13+	movq	%r14,%rcx+	vpalignr	$8,%xmm3,%xmm4,%xmm11+	movq	%r11,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%r10,%r13+	xorq	%rax,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%rcx,%r14+	vpaddq	%xmm11,%xmm7,%xmm7+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	112(%rsp),%rbx+	movq	%rcx,%r15+.byte	143,72,120,195,209,7+	xorq	%rax,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%rdx,%r15+	addq	%r12,%rbx+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,222,3+	xorq	%rcx,%r14+	addq	%r13,%rbx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rdx,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm6,%xmm10+	addq	%rbx,%r9+	addq	%rdi,%rbx+	vpaddq	%xmm8,%xmm7,%xmm7+	movq	%r9,%r13+	addq	%rbx,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%rbx+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%r10,%r12+	rorq	$5,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	vpaddq	%xmm11,%xmm7,%xmm7+	addq	120(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	rorq	$6,%r14+	vpaddq	96(%rsi),%xmm7,%xmm10+	xorq	%rcx,%rdi+	addq	%r12,%rax+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	rorq	$28,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	vmovdqa	%xmm10,112(%rsp)+	cmpb	$0,135(%rsi)+	jne	.Lxop_00_47+	rorq	$23,%r13+	movq	%r14,%rax+	movq	%r9,%r12+	rorq	$5,%r14+	xorq	%r8,%r13+	xorq	%r10,%r12+	rorq	$4,%r13+	xorq	%rax,%r14+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	0(%rsp),%r11+	movq	%rax,%r15+	xorq	%r10,%r12+	rorq	$6,%r14+	xorq	%rbx,%r15+	addq	%r12,%r11+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%rax,%r14+	addq	%r13,%r11+	xorq	%rbx,%rdi+	rorq	$28,%r14+	addq	%r11,%rdx+	addq	%rdi,%r11+	movq	%rdx,%r13+	addq	%r11,%r14+	rorq	$23,%r13+	movq	%r14,%r11+	movq	%r8,%r12+	rorq	$5,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	rorq	$4,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	addq	8(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	rorq	$6,%r14+	xorq	%rax,%rdi+	addq	%r12,%r10+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	rorq	$28,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	rorq	$23,%r13+	movq	%r14,%r10+	movq	%rdx,%r12+	rorq	$5,%r14+	xorq	%rcx,%r13+	xorq	%r8,%r12+	rorq	$4,%r13+	xorq	%r10,%r14+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	16(%rsp),%r9+	movq	%r10,%r15+	xorq	%r8,%r12+	rorq	$6,%r14+	xorq	%r11,%r15+	addq	%r12,%r9+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%r10,%r14+	addq	%r13,%r9+	xorq	%r11,%rdi+	rorq	$28,%r14+	addq	%r9,%rbx+	addq	%rdi,%r9+	movq	%rbx,%r13+	addq	%r9,%r14+	rorq	$23,%r13+	movq	%r14,%r9+	movq	%rcx,%r12+	rorq	$5,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	rorq	$4,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	addq	24(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	rorq	$6,%r14+	xorq	%r10,%rdi+	addq	%r12,%r8+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	rorq	$28,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	rorq	$23,%r13+	movq	%r14,%r8+	movq	%rbx,%r12+	rorq	$5,%r14+	xorq	%rax,%r13+	xorq	%rcx,%r12+	rorq	$4,%r13+	xorq	%r8,%r14+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	32(%rsp),%rdx+	movq	%r8,%r15+	xorq	%rcx,%r12+	rorq	$6,%r14+	xorq	%r9,%r15+	addq	%r12,%rdx+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%r8,%r14+	addq	%r13,%rdx+	xorq	%r9,%rdi+	rorq	$28,%r14+	addq	%rdx,%r11+	addq	%rdi,%rdx+	movq	%r11,%r13+	addq	%rdx,%r14+	rorq	$23,%r13+	movq	%r14,%rdx+	movq	%rax,%r12+	rorq	$5,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	rorq	$4,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	addq	40(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	rorq	$6,%r14+	xorq	%r8,%rdi+	addq	%r12,%rcx+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	rorq	$28,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	rorq	$23,%r13+	movq	%r14,%rcx+	movq	%r11,%r12+	rorq	$5,%r14+	xorq	%r10,%r13+	xorq	%rax,%r12+	rorq	$4,%r13+	xorq	%rcx,%r14+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	48(%rsp),%rbx+	movq	%rcx,%r15+	xorq	%rax,%r12+	rorq	$6,%r14+	xorq	%rdx,%r15+	addq	%r12,%rbx+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%rcx,%r14+	addq	%r13,%rbx+	xorq	%rdx,%rdi+	rorq	$28,%r14+	addq	%rbx,%r9+	addq	%rdi,%rbx+	movq	%r9,%r13+	addq	%rbx,%r14+	rorq	$23,%r13+	movq	%r14,%rbx+	movq	%r10,%r12+	rorq	$5,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	rorq	$4,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	addq	56(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	rorq	$6,%r14+	xorq	%rcx,%rdi+	addq	%r12,%rax+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	rorq	$28,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	rorq	$23,%r13+	movq	%r14,%rax+	movq	%r9,%r12+	rorq	$5,%r14+	xorq	%r8,%r13+	xorq	%r10,%r12+	rorq	$4,%r13+	xorq	%rax,%r14+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	64(%rsp),%r11+	movq	%rax,%r15+	xorq	%r10,%r12+	rorq	$6,%r14+	xorq	%rbx,%r15+	addq	%r12,%r11+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%rax,%r14+	addq	%r13,%r11+	xorq	%rbx,%rdi+	rorq	$28,%r14+	addq	%r11,%rdx+	addq	%rdi,%r11+	movq	%rdx,%r13+	addq	%r11,%r14+	rorq	$23,%r13+	movq	%r14,%r11+	movq	%r8,%r12+	rorq	$5,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	rorq	$4,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	addq	72(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	rorq	$6,%r14+	xorq	%rax,%rdi+	addq	%r12,%r10+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	rorq	$28,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	rorq	$23,%r13+	movq	%r14,%r10+	movq	%rdx,%r12+	rorq	$5,%r14+	xorq	%rcx,%r13+	xorq	%r8,%r12+	rorq	$4,%r13+	xorq	%r10,%r14+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	80(%rsp),%r9+	movq	%r10,%r15+	xorq	%r8,%r12+	rorq	$6,%r14+	xorq	%r11,%r15+	addq	%r12,%r9+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%r10,%r14+	addq	%r13,%r9+	xorq	%r11,%rdi+	rorq	$28,%r14+	addq	%r9,%rbx+	addq	%rdi,%r9+	movq	%rbx,%r13+	addq	%r9,%r14+	rorq	$23,%r13+	movq	%r14,%r9+	movq	%rcx,%r12+	rorq	$5,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	rorq	$4,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	addq	88(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	rorq	$6,%r14+	xorq	%r10,%rdi+	addq	%r12,%r8+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	rorq	$28,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	rorq	$23,%r13+	movq	%r14,%r8+	movq	%rbx,%r12+	rorq	$5,%r14+	xorq	%rax,%r13+	xorq	%rcx,%r12+	rorq	$4,%r13+	xorq	%r8,%r14+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	96(%rsp),%rdx+	movq	%r8,%r15+	xorq	%rcx,%r12+	rorq	$6,%r14+	xorq	%r9,%r15+	addq	%r12,%rdx+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%r8,%r14+	addq	%r13,%rdx+	xorq	%r9,%rdi+	rorq	$28,%r14+	addq	%rdx,%r11+	addq	%rdi,%rdx+	movq	%r11,%r13+	addq	%rdx,%r14+	rorq	$23,%r13+	movq	%r14,%rdx+	movq	%rax,%r12+	rorq	$5,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	rorq	$4,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	addq	104(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	rorq	$6,%r14+	xorq	%r8,%rdi+	addq	%r12,%rcx+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	rorq	$28,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	rorq	$23,%r13+	movq	%r14,%rcx+	movq	%r11,%r12+	rorq	$5,%r14+	xorq	%r10,%r13+	xorq	%rax,%r12+	rorq	$4,%r13+	xorq	%rcx,%r14+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	112(%rsp),%rbx+	movq	%rcx,%r15+	xorq	%rax,%r12+	rorq	$6,%r14+	xorq	%rdx,%r15+	addq	%r12,%rbx+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%rcx,%r14+	addq	%r13,%rbx+	xorq	%rdx,%rdi+	rorq	$28,%r14+	addq	%rbx,%r9+	addq	%rdi,%rbx+	movq	%r9,%r13+	addq	%rbx,%r14+	rorq	$23,%r13+	movq	%r14,%rbx+	movq	%r10,%r12+	rorq	$5,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	rorq	$4,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	addq	120(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	rorq	$6,%r14+	xorq	%rcx,%rdi+	addq	%r12,%rax+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	rorq	$28,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	movq	-64(%rbp),%rdi+	movq	%r14,%rax+	movq	-56(%rbp),%rsi++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	addq	48(%rdi),%r10+	addq	56(%rdi),%r11++	leaq	128(%rsi),%rsi+	cmpq	-48(%rbp),%rsi++	movq	%rax,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)+	jb	.Lloop_xop++	vzeroupper+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_sha512_asm_block_data_order_xop,.-crypton_sha512_asm_block_data_order_xop+.type	crypton_sha512_asm_block_data_order_avx,@function+.align	64+crypton_sha512_asm_block_data_order_avx:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+.Lavx_shortcut:+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$24,%rsp++	leaq	(%rsi,%rdx,8),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+++	leaq	-128(%rsp),%rsp+	vzeroupper+	andq	$-64,%rsp+	movq	0(%rdi),%rax+	movq	8(%rdi),%rbx+	movq	16(%rdi),%rcx+	movq	24(%rdi),%rdx+	movq	32(%rdi),%r8+	movq	40(%rdi),%r9+	movq	48(%rdi),%r10+	movq	56(%rdi),%r11+	jmp	.Lloop_avx+.align	16+.Lloop_avx:+	vmovdqa	K512+1280(%rip),%xmm11+	movq	%rsi,-56(%rbp)+	vmovdqu	0(%rsi),%xmm0+	vmovdqu	16(%rsi),%xmm1+	vmovdqu	32(%rsi),%xmm2+	vpshufb	%xmm11,%xmm0,%xmm0+	vmovdqu	48(%rsi),%xmm3+	vpshufb	%xmm11,%xmm1,%xmm1+	vmovdqu	64(%rsi),%xmm4+	vpshufb	%xmm11,%xmm2,%xmm2+	vmovdqu	80(%rsi),%xmm5+	vpshufb	%xmm11,%xmm3,%xmm3+	vmovdqu	96(%rsi),%xmm6+	vpshufb	%xmm11,%xmm4,%xmm4+	vmovdqu	112(%rsi),%xmm7+	leaq	K512+128(%rip),%rsi+	vpshufb	%xmm11,%xmm5,%xmm5+	vpaddq	-128(%rsi),%xmm0,%xmm8+	vpshufb	%xmm11,%xmm6,%xmm6+	vpaddq	-96(%rsi),%xmm1,%xmm9+	vpshufb	%xmm11,%xmm7,%xmm7+	vpaddq	-64(%rsi),%xmm2,%xmm10+	vpaddq	-32(%rsi),%xmm3,%xmm11+	vmovdqa	%xmm8,0(%rsp)+	vpaddq	0(%rsi),%xmm4,%xmm8+	vmovdqa	%xmm9,16(%rsp)+	vpaddq	32(%rsi),%xmm5,%xmm9+	vmovdqa	%xmm10,32(%rsp)+	vpaddq	64(%rsi),%xmm6,%xmm10+	vmovdqa	%xmm11,48(%rsp)+	vpaddq	96(%rsi),%xmm7,%xmm11+	vmovdqa	%xmm8,64(%rsp)+	movq	%rax,%r14+	vmovdqa	%xmm9,80(%rsp)+	movq	%rbx,%rdi+	vmovdqa	%xmm10,96(%rsp)+	xorq	%rcx,%rdi+	vmovdqa	%xmm11,112(%rsp)+	movq	%r8,%r13+	jmp	.Lavx_00_47++.align	16+.Lavx_00_47:+	addq	$256,%rsi+	vpalignr	$8,%xmm0,%xmm1,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%rax+	vpalignr	$8,%xmm4,%xmm5,%xmm11+	movq	%r9,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%r8,%r13+	xorq	%r10,%r12+	vpaddq	%xmm11,%xmm0,%xmm0+	shrdq	$4,%r13,%r13+	xorq	%rax,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%r8,%r12+	xorq	%r8,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	0(%rsp),%r11+	movq	%rax,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%r10,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%rbx,%r15+	addq	%r12,%r11+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%rax,%r14+	addq	%r13,%r11+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rbx,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm7,%xmm11+	addq	%r11,%rdx+	addq	%rdi,%r11+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%rdx,%r13+	addq	%r11,%r14+	vpsllq	$3,%xmm7,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%r11+	vpaddq	%xmm8,%xmm0,%xmm0+	movq	%r8,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm7,%xmm9+	xorq	%rdx,%r13+	xorq	%r9,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%r11,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%rdx,%r12+	xorq	%rdx,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	8(%rsp),%r10+	movq	%r11,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%r9,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%rax,%rdi+	addq	%r12,%r10+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm0,%xmm0+	xorq	%r11,%r14+	addq	%r13,%r10+	vpaddq	-128(%rsi),%xmm0,%xmm10+	xorq	%rax,%r15+	shrdq	$28,%r14,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	vmovdqa	%xmm10,0(%rsp)+	vpalignr	$8,%xmm1,%xmm2,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%r10+	vpalignr	$8,%xmm5,%xmm6,%xmm11+	movq	%rdx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%rcx,%r13+	xorq	%r8,%r12+	vpaddq	%xmm11,%xmm1,%xmm1+	shrdq	$4,%r13,%r13+	xorq	%r10,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%rcx,%r12+	xorq	%rcx,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	16(%rsp),%r9+	movq	%r10,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%r8,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%r11,%r15+	addq	%r12,%r9+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%r10,%r14+	addq	%r13,%r9+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r11,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm0,%xmm11+	addq	%r9,%rbx+	addq	%rdi,%r9+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%rbx,%r13+	addq	%r9,%r14+	vpsllq	$3,%xmm0,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%r9+	vpaddq	%xmm8,%xmm1,%xmm1+	movq	%rcx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm0,%xmm9+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%r9,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%rbx,%r12+	xorq	%rbx,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	24(%rsp),%r8+	movq	%r9,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%rdx,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%r10,%rdi+	addq	%r12,%r8+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm1,%xmm1+	xorq	%r9,%r14+	addq	%r13,%r8+	vpaddq	-96(%rsi),%xmm1,%xmm10+	xorq	%r10,%r15+	shrdq	$28,%r14,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	vmovdqa	%xmm10,16(%rsp)+	vpalignr	$8,%xmm2,%xmm3,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%r8+	vpalignr	$8,%xmm6,%xmm7,%xmm11+	movq	%rbx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%rax,%r13+	xorq	%rcx,%r12+	vpaddq	%xmm11,%xmm2,%xmm2+	shrdq	$4,%r13,%r13+	xorq	%r8,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%rax,%r12+	xorq	%rax,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	32(%rsp),%rdx+	movq	%r8,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%rcx,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%r9,%r15+	addq	%r12,%rdx+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%r8,%r14+	addq	%r13,%rdx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r9,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm1,%xmm11+	addq	%rdx,%r11+	addq	%rdi,%rdx+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%r11,%r13+	addq	%rdx,%r14+	vpsllq	$3,%xmm1,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%rdx+	vpaddq	%xmm8,%xmm2,%xmm2+	movq	%rax,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm1,%xmm9+	xorq	%r11,%r13+	xorq	%rbx,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%rdx,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%r11,%r12+	xorq	%r11,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	40(%rsp),%rcx+	movq	%rdx,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%rbx,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%r8,%rdi+	addq	%r12,%rcx+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm2,%xmm2+	xorq	%rdx,%r14+	addq	%r13,%rcx+	vpaddq	-64(%rsi),%xmm2,%xmm10+	xorq	%r8,%r15+	shrdq	$28,%r14,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	vmovdqa	%xmm10,32(%rsp)+	vpalignr	$8,%xmm3,%xmm4,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%rcx+	vpalignr	$8,%xmm7,%xmm0,%xmm11+	movq	%r11,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%r10,%r13+	xorq	%rax,%r12+	vpaddq	%xmm11,%xmm3,%xmm3+	shrdq	$4,%r13,%r13+	xorq	%rcx,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%r10,%r12+	xorq	%r10,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	48(%rsp),%rbx+	movq	%rcx,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%rax,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%rdx,%r15+	addq	%r12,%rbx+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%rcx,%r14+	addq	%r13,%rbx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rdx,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm2,%xmm11+	addq	%rbx,%r9+	addq	%rdi,%rbx+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%r9,%r13+	addq	%rbx,%r14+	vpsllq	$3,%xmm2,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%rbx+	vpaddq	%xmm8,%xmm3,%xmm3+	movq	%r10,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm2,%xmm9+	xorq	%r9,%r13+	xorq	%r11,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%rbx,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%r9,%r12+	xorq	%r9,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	56(%rsp),%rax+	movq	%rbx,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%r11,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%rcx,%rdi+	addq	%r12,%rax+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm3,%xmm3+	xorq	%rbx,%r14+	addq	%r13,%rax+	vpaddq	-32(%rsi),%xmm3,%xmm10+	xorq	%rcx,%r15+	shrdq	$28,%r14,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	vmovdqa	%xmm10,48(%rsp)+	vpalignr	$8,%xmm4,%xmm5,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%rax+	vpalignr	$8,%xmm0,%xmm1,%xmm11+	movq	%r9,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%r8,%r13+	xorq	%r10,%r12+	vpaddq	%xmm11,%xmm4,%xmm4+	shrdq	$4,%r13,%r13+	xorq	%rax,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%r8,%r12+	xorq	%r8,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	64(%rsp),%r11+	movq	%rax,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%r10,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%rbx,%r15+	addq	%r12,%r11+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%rax,%r14+	addq	%r13,%r11+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rbx,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm3,%xmm11+	addq	%r11,%rdx+	addq	%rdi,%r11+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%rdx,%r13+	addq	%r11,%r14+	vpsllq	$3,%xmm3,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%r11+	vpaddq	%xmm8,%xmm4,%xmm4+	movq	%r8,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm3,%xmm9+	xorq	%rdx,%r13+	xorq	%r9,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%r11,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%rdx,%r12+	xorq	%rdx,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	72(%rsp),%r10+	movq	%r11,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%r9,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%rax,%rdi+	addq	%r12,%r10+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm4,%xmm4+	xorq	%r11,%r14+	addq	%r13,%r10+	vpaddq	0(%rsi),%xmm4,%xmm10+	xorq	%rax,%r15+	shrdq	$28,%r14,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	vmovdqa	%xmm10,64(%rsp)+	vpalignr	$8,%xmm5,%xmm6,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%r10+	vpalignr	$8,%xmm1,%xmm2,%xmm11+	movq	%rdx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%rcx,%r13+	xorq	%r8,%r12+	vpaddq	%xmm11,%xmm5,%xmm5+	shrdq	$4,%r13,%r13+	xorq	%r10,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%rcx,%r12+	xorq	%rcx,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	80(%rsp),%r9+	movq	%r10,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%r8,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%r11,%r15+	addq	%r12,%r9+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%r10,%r14+	addq	%r13,%r9+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r11,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm4,%xmm11+	addq	%r9,%rbx+	addq	%rdi,%r9+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%rbx,%r13+	addq	%r9,%r14+	vpsllq	$3,%xmm4,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%r9+	vpaddq	%xmm8,%xmm5,%xmm5+	movq	%rcx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm4,%xmm9+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%r9,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%rbx,%r12+	xorq	%rbx,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	88(%rsp),%r8+	movq	%r9,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%rdx,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%r10,%rdi+	addq	%r12,%r8+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm5,%xmm5+	xorq	%r9,%r14+	addq	%r13,%r8+	vpaddq	32(%rsi),%xmm5,%xmm10+	xorq	%r10,%r15+	shrdq	$28,%r14,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	vmovdqa	%xmm10,80(%rsp)+	vpalignr	$8,%xmm6,%xmm7,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%r8+	vpalignr	$8,%xmm2,%xmm3,%xmm11+	movq	%rbx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%rax,%r13+	xorq	%rcx,%r12+	vpaddq	%xmm11,%xmm6,%xmm6+	shrdq	$4,%r13,%r13+	xorq	%r8,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%rax,%r12+	xorq	%rax,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	96(%rsp),%rdx+	movq	%r8,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%rcx,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%r9,%r15+	addq	%r12,%rdx+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%r8,%r14+	addq	%r13,%rdx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r9,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm5,%xmm11+	addq	%rdx,%r11+	addq	%rdi,%rdx+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%r11,%r13+	addq	%rdx,%r14+	vpsllq	$3,%xmm5,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%rdx+	vpaddq	%xmm8,%xmm6,%xmm6+	movq	%rax,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm5,%xmm9+	xorq	%r11,%r13+	xorq	%rbx,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%rdx,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%r11,%r12+	xorq	%r11,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	104(%rsp),%rcx+	movq	%rdx,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%rbx,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%r8,%rdi+	addq	%r12,%rcx+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm6,%xmm6+	xorq	%rdx,%r14+	addq	%r13,%rcx+	vpaddq	64(%rsi),%xmm6,%xmm10+	xorq	%r8,%r15+	shrdq	$28,%r14,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	vmovdqa	%xmm10,96(%rsp)+	vpalignr	$8,%xmm7,%xmm0,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%rcx+	vpalignr	$8,%xmm3,%xmm4,%xmm11+	movq	%r11,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%r10,%r13+	xorq	%rax,%r12+	vpaddq	%xmm11,%xmm7,%xmm7+	shrdq	$4,%r13,%r13+	xorq	%rcx,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%r10,%r12+	xorq	%r10,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	112(%rsp),%rbx+	movq	%rcx,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%rax,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%rdx,%r15+	addq	%r12,%rbx+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%rcx,%r14+	addq	%r13,%rbx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rdx,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm6,%xmm11+	addq	%rbx,%r9+	addq	%rdi,%rbx+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%r9,%r13+	addq	%rbx,%r14+	vpsllq	$3,%xmm6,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%rbx+	vpaddq	%xmm8,%xmm7,%xmm7+	movq	%r10,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm6,%xmm9+	xorq	%r9,%r13+	xorq	%r11,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%rbx,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%r9,%r12+	xorq	%r9,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	120(%rsp),%rax+	movq	%rbx,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%r11,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%rcx,%rdi+	addq	%r12,%rax+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm7,%xmm7+	xorq	%rbx,%r14+	addq	%r13,%rax+	vpaddq	96(%rsi),%xmm7,%xmm10+	xorq	%rcx,%r15+	shrdq	$28,%r14,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	vmovdqa	%xmm10,112(%rsp)+	cmpb	$0,135(%rsi)+	jne	.Lavx_00_47+	shrdq	$23,%r13,%r13+	movq	%r14,%rax+	movq	%r9,%r12+	shrdq	$5,%r14,%r14+	xorq	%r8,%r13+	xorq	%r10,%r12+	shrdq	$4,%r13,%r13+	xorq	%rax,%r14+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	0(%rsp),%r11+	movq	%rax,%r15+	xorq	%r10,%r12+	shrdq	$6,%r14,%r14+	xorq	%rbx,%r15+	addq	%r12,%r11+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%rax,%r14+	addq	%r13,%r11+	xorq	%rbx,%rdi+	shrdq	$28,%r14,%r14+	addq	%r11,%rdx+	addq	%rdi,%r11+	movq	%rdx,%r13+	addq	%r11,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r11+	movq	%r8,%r12+	shrdq	$5,%r14,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	shrdq	$4,%r13,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	addq	8(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	shrdq	$6,%r14,%r14+	xorq	%rax,%rdi+	addq	%r12,%r10+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	shrdq	$28,%r14,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r10+	movq	%rdx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rcx,%r13+	xorq	%r8,%r12+	shrdq	$4,%r13,%r13+	xorq	%r10,%r14+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	16(%rsp),%r9+	movq	%r10,%r15+	xorq	%r8,%r12+	shrdq	$6,%r14,%r14+	xorq	%r11,%r15+	addq	%r12,%r9+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%r10,%r14+	addq	%r13,%r9+	xorq	%r11,%rdi+	shrdq	$28,%r14,%r14+	addq	%r9,%rbx+	addq	%rdi,%r9+	movq	%rbx,%r13+	addq	%r9,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r9+	movq	%rcx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	shrdq	$4,%r13,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	addq	24(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r10,%rdi+	addq	%r12,%r8+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	shrdq	$28,%r14,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r8+	movq	%rbx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rax,%r13+	xorq	%rcx,%r12+	shrdq	$4,%r13,%r13+	xorq	%r8,%r14+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	32(%rsp),%rdx+	movq	%r8,%r15+	xorq	%rcx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r9,%r15+	addq	%r12,%rdx+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%r8,%r14+	addq	%r13,%rdx+	xorq	%r9,%rdi+	shrdq	$28,%r14,%r14+	addq	%rdx,%r11+	addq	%rdi,%rdx+	movq	%r11,%r13+	addq	%rdx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rdx+	movq	%rax,%r12+	shrdq	$5,%r14,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	shrdq	$4,%r13,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	addq	40(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r8,%rdi+	addq	%r12,%rcx+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	shrdq	$28,%r14,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rcx+	movq	%r11,%r12+	shrdq	$5,%r14,%r14+	xorq	%r10,%r13+	xorq	%rax,%r12+	shrdq	$4,%r13,%r13+	xorq	%rcx,%r14+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	48(%rsp),%rbx+	movq	%rcx,%r15+	xorq	%rax,%r12+	shrdq	$6,%r14,%r14+	xorq	%rdx,%r15+	addq	%r12,%rbx+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%rcx,%r14+	addq	%r13,%rbx+	xorq	%rdx,%rdi+	shrdq	$28,%r14,%r14+	addq	%rbx,%r9+	addq	%rdi,%rbx+	movq	%r9,%r13+	addq	%rbx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rbx+	movq	%r10,%r12+	shrdq	$5,%r14,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	shrdq	$4,%r13,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	addq	56(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	shrdq	$6,%r14,%r14+	xorq	%rcx,%rdi+	addq	%r12,%rax+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	shrdq	$28,%r14,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rax+	movq	%r9,%r12+	shrdq	$5,%r14,%r14+	xorq	%r8,%r13+	xorq	%r10,%r12+	shrdq	$4,%r13,%r13+	xorq	%rax,%r14+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	64(%rsp),%r11+	movq	%rax,%r15+	xorq	%r10,%r12+	shrdq	$6,%r14,%r14+	xorq	%rbx,%r15+	addq	%r12,%r11+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%rax,%r14+	addq	%r13,%r11+	xorq	%rbx,%rdi+	shrdq	$28,%r14,%r14+	addq	%r11,%rdx+	addq	%rdi,%r11+	movq	%rdx,%r13+	addq	%r11,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r11+	movq	%r8,%r12+	shrdq	$5,%r14,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	shrdq	$4,%r13,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	addq	72(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	shrdq	$6,%r14,%r14+	xorq	%rax,%rdi+	addq	%r12,%r10+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	shrdq	$28,%r14,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r10+	movq	%rdx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rcx,%r13+	xorq	%r8,%r12+	shrdq	$4,%r13,%r13+	xorq	%r10,%r14+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	80(%rsp),%r9+	movq	%r10,%r15+	xorq	%r8,%r12+	shrdq	$6,%r14,%r14+	xorq	%r11,%r15+	addq	%r12,%r9+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%r10,%r14+	addq	%r13,%r9+	xorq	%r11,%rdi+	shrdq	$28,%r14,%r14+	addq	%r9,%rbx+	addq	%rdi,%r9+	movq	%rbx,%r13+	addq	%r9,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r9+	movq	%rcx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	shrdq	$4,%r13,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	addq	88(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r10,%rdi+	addq	%r12,%r8+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	shrdq	$28,%r14,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r8+	movq	%rbx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rax,%r13+	xorq	%rcx,%r12+	shrdq	$4,%r13,%r13+	xorq	%r8,%r14+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	96(%rsp),%rdx+	movq	%r8,%r15+	xorq	%rcx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r9,%r15+	addq	%r12,%rdx+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%r8,%r14+	addq	%r13,%rdx+	xorq	%r9,%rdi+	shrdq	$28,%r14,%r14+	addq	%rdx,%r11+	addq	%rdi,%rdx+	movq	%r11,%r13+	addq	%rdx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rdx+	movq	%rax,%r12+	shrdq	$5,%r14,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	shrdq	$4,%r13,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	addq	104(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r8,%rdi+	addq	%r12,%rcx+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	shrdq	$28,%r14,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rcx+	movq	%r11,%r12+	shrdq	$5,%r14,%r14+	xorq	%r10,%r13+	xorq	%rax,%r12+	shrdq	$4,%r13,%r13+	xorq	%rcx,%r14+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	112(%rsp),%rbx+	movq	%rcx,%r15+	xorq	%rax,%r12+	shrdq	$6,%r14,%r14+	xorq	%rdx,%r15+	addq	%r12,%rbx+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%rcx,%r14+	addq	%r13,%rbx+	xorq	%rdx,%rdi+	shrdq	$28,%r14,%r14+	addq	%rbx,%r9+	addq	%rdi,%rbx+	movq	%r9,%r13+	addq	%rbx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rbx+	movq	%r10,%r12+	shrdq	$5,%r14,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	shrdq	$4,%r13,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	addq	120(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	shrdq	$6,%r14,%r14+	xorq	%rcx,%rdi+	addq	%r12,%rax+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	shrdq	$28,%r14,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	movq	-64(%rbp),%rdi+	movq	%r14,%rax+	movq	-56(%rbp),%rsi++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	addq	48(%rdi),%r10+	addq	56(%rdi),%r11++	leaq	128(%rsi),%rsi+	cmpq	-48(%rbp),%rsi++	movq	%rax,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)+	jb	.Lloop_avx++	vzeroupper+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_sha512_asm_block_data_order_avx,.-crypton_sha512_asm_block_data_order_avx+.type	crypton_sha512_asm_block_data_order_avx2,@function+.align	64+crypton_sha512_asm_block_data_order_avx2:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+.Lavx2_shortcut:+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$24,%rsp++	leaq	(%rsi,%rdx,8),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+++	leaq	-128(%rsp),%rsp+	vzeroupper+	andq	$-128,%rsp+	subq	$-128,%rsi+	movq	0(%rdi),%rax+	movq	%rsi,%r12+	movq	8(%rdi),%rbx+	cmpq	%rdx,%rsi+	movq	16(%rdi),%rcx+	cmoveq	%rsp,%r12+	movq	24(%rdi),%rdx+	movq	32(%rdi),%r8+	movq	40(%rdi),%r9+	movq	48(%rdi),%r10+	movq	56(%rdi),%r11+	jmp	.Loop_avx2+.align	16+.Loop_avx2:+	vmovdqa	K512+1280(%rip),%ymm10+	movq	%rsi,-56(%rbp)+	vmovdqu	-128(%rsi),%xmm0+	vmovdqu	-128+16(%rsi),%xmm1+	vmovdqu	-128+32(%rsi),%xmm2+	vmovdqu	-128+48(%rsi),%xmm3+	vmovdqu	-128+64(%rsi),%xmm4+	vmovdqu	-128+80(%rsi),%xmm5+	vmovdqu	-128+96(%rsi),%xmm6+	vmovdqu	-128+112(%rsi),%xmm7+	leaq	K512+128(%rip),%rsi+	vinserti128	$1,(%r12),%ymm0,%ymm0+	vinserti128	$1,16(%r12),%ymm1,%ymm1+	vpshufb	%ymm10,%ymm0,%ymm0+	vinserti128	$1,32(%r12),%ymm2,%ymm2+	vpshufb	%ymm10,%ymm1,%ymm1+	vinserti128	$1,48(%r12),%ymm3,%ymm3+	vpshufb	%ymm10,%ymm2,%ymm2+	vinserti128	$1,64(%r12),%ymm4,%ymm4+	vpshufb	%ymm10,%ymm3,%ymm3+	vinserti128	$1,80(%r12),%ymm5,%ymm5+	vpshufb	%ymm10,%ymm4,%ymm4+	vinserti128	$1,96(%r12),%ymm6,%ymm6+	vpshufb	%ymm10,%ymm5,%ymm5+	vinserti128	$1,112(%r12),%ymm7,%ymm7++	vpaddq	-128(%rsi),%ymm0,%ymm8+	vpshufb	%ymm10,%ymm6,%ymm6+	vpaddq	-96(%rsi),%ymm1,%ymm9+	vpshufb	%ymm10,%ymm7,%ymm7+	vpaddq	-64(%rsi),%ymm2,%ymm10+	vpaddq	-32(%rsi),%ymm3,%ymm11+	vmovdqa	%ymm8,0(%rsp)+	vpaddq	0(%rsi),%ymm4,%ymm8+	vmovdqa	%ymm9,32(%rsp)+	vpaddq	32(%rsi),%ymm5,%ymm9+	vmovdqa	%ymm10,64(%rsp)+	vpaddq	64(%rsi),%ymm6,%ymm10+	vmovdqa	%ymm11,96(%rsp)+	leaq	-128(%rsp),%rsp+	vpaddq	96(%rsi),%ymm7,%ymm11+	vmovdqa	%ymm8,0(%rsp)+	xorq	%r14,%r14+	vmovdqa	%ymm9,32(%rsp)+	movq	%rbx,%rdi+	vmovdqa	%ymm10,64(%rsp)+	xorq	%rcx,%rdi+	vmovdqa	%ymm11,96(%rsp)+	movq	%r9,%r12+	addq	$32*8,%rsi+	jmp	.Lavx2_00_47++.align	16+.Lavx2_00_47:+	leaq	-128(%rsp),%rsp+	vpalignr	$8,%ymm0,%ymm1,%ymm8+	addq	0+256(%rsp),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	vpalignr	$8,%ymm4,%ymm5,%ymm11+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	vpaddq	%ymm11,%ymm0,%ymm0+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	vpsrlq	$6,%ymm7,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	vpsllq	$3,%ymm7,%ymm10+	vpaddq	%ymm8,%ymm0,%ymm0+	addq	8+256(%rsp),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	vpsrlq	$19,%ymm7,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	vpaddq	%ymm11,%ymm0,%ymm0+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	vpaddq	-128(%rsi),%ymm0,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	vmovdqa	%ymm10,0(%rsp)+	vpalignr	$8,%ymm1,%ymm2,%ymm8+	addq	32+256(%rsp),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	vpalignr	$8,%ymm5,%ymm6,%ymm11+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	vpaddq	%ymm11,%ymm1,%ymm1+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	vpsrlq	$6,%ymm0,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	vpsllq	$3,%ymm0,%ymm10+	vpaddq	%ymm8,%ymm1,%ymm1+	addq	40+256(%rsp),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	vpsrlq	$19,%ymm0,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	vpaddq	%ymm11,%ymm1,%ymm1+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	vpaddq	-96(%rsi),%ymm1,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	vmovdqa	%ymm10,32(%rsp)+	vpalignr	$8,%ymm2,%ymm3,%ymm8+	addq	64+256(%rsp),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	vpalignr	$8,%ymm6,%ymm7,%ymm11+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	vpaddq	%ymm11,%ymm2,%ymm2+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	vpsrlq	$6,%ymm1,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	vpsllq	$3,%ymm1,%ymm10+	vpaddq	%ymm8,%ymm2,%ymm2+	addq	72+256(%rsp),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	vpsrlq	$19,%ymm1,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	vpaddq	%ymm11,%ymm2,%ymm2+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	vpaddq	-64(%rsi),%ymm2,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	vmovdqa	%ymm10,64(%rsp)+	vpalignr	$8,%ymm3,%ymm4,%ymm8+	addq	96+256(%rsp),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	vpalignr	$8,%ymm7,%ymm0,%ymm11+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	vpaddq	%ymm11,%ymm3,%ymm3+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	vpsrlq	$6,%ymm2,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	vpsllq	$3,%ymm2,%ymm10+	vpaddq	%ymm8,%ymm3,%ymm3+	addq	104+256(%rsp),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	vpsrlq	$19,%ymm2,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	vpaddq	%ymm11,%ymm3,%ymm3+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	vpaddq	-32(%rsi),%ymm3,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	vmovdqa	%ymm10,96(%rsp)+	leaq	-128(%rsp),%rsp+	vpalignr	$8,%ymm4,%ymm5,%ymm8+	addq	0+256(%rsp),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	vpalignr	$8,%ymm0,%ymm1,%ymm11+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	vpaddq	%ymm11,%ymm4,%ymm4+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	vpsrlq	$6,%ymm3,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	vpsllq	$3,%ymm3,%ymm10+	vpaddq	%ymm8,%ymm4,%ymm4+	addq	8+256(%rsp),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	vpsrlq	$19,%ymm3,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	vpaddq	%ymm11,%ymm4,%ymm4+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	vpaddq	0(%rsi),%ymm4,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	vmovdqa	%ymm10,0(%rsp)+	vpalignr	$8,%ymm5,%ymm6,%ymm8+	addq	32+256(%rsp),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	vpalignr	$8,%ymm1,%ymm2,%ymm11+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	vpaddq	%ymm11,%ymm5,%ymm5+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	vpsrlq	$6,%ymm4,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	vpsllq	$3,%ymm4,%ymm10+	vpaddq	%ymm8,%ymm5,%ymm5+	addq	40+256(%rsp),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	vpsrlq	$19,%ymm4,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	vpaddq	%ymm11,%ymm5,%ymm5+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	vpaddq	32(%rsi),%ymm5,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	vmovdqa	%ymm10,32(%rsp)+	vpalignr	$8,%ymm6,%ymm7,%ymm8+	addq	64+256(%rsp),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	vpalignr	$8,%ymm2,%ymm3,%ymm11+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	vpaddq	%ymm11,%ymm6,%ymm6+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	vpsrlq	$6,%ymm5,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	vpsllq	$3,%ymm5,%ymm10+	vpaddq	%ymm8,%ymm6,%ymm6+	addq	72+256(%rsp),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	vpsrlq	$19,%ymm5,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	vpaddq	%ymm11,%ymm6,%ymm6+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	vpaddq	64(%rsi),%ymm6,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	vmovdqa	%ymm10,64(%rsp)+	vpalignr	$8,%ymm7,%ymm0,%ymm8+	addq	96+256(%rsp),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	vpalignr	$8,%ymm3,%ymm4,%ymm11+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	vpaddq	%ymm11,%ymm7,%ymm7+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	vpsrlq	$6,%ymm6,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	vpsllq	$3,%ymm6,%ymm10+	vpaddq	%ymm8,%ymm7,%ymm7+	addq	104+256(%rsp),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	vpsrlq	$19,%ymm6,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	vpaddq	%ymm11,%ymm7,%ymm7+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	vpaddq	96(%rsi),%ymm7,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	vmovdqa	%ymm10,96(%rsp)+	leaq	256(%rsi),%rsi+	cmpb	$0,-121(%rsi)+	jne	.Lavx2_00_47+	addq	0+128(%rsp),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	addq	8+128(%rsp),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	addq	32+128(%rsp),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	addq	40+128(%rsp),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	addq	64+128(%rsp),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	addq	72+128(%rsp),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	addq	96+128(%rsp),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	addq	104+128(%rsp),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	addq	0(%rsp),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	addq	8(%rsp),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	addq	32(%rsp),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	addq	40(%rsp),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	addq	64(%rsp),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	addq	72(%rsp),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	addq	96(%rsp),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	addq	104(%rsp),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	movq	-64(%rbp),%rdi+	addq	%r14,%rax+	movq	-56(%rbp),%r12++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	addq	48(%rdi),%r10+	addq	56(%rdi),%r11++	movq	%rax,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)++	cmpq	-48(%rbp),%r12+	je	.Ldone_avx2++	leaq	1152(%rsp),%rsi+	xorq	%r14,%r14+	movq	%rbx,%rdi+	xorq	%rcx,%rdi+	movq	%r9,%r12+	jmp	.Lower_avx2+.align	16+.Lower_avx2:+	addq	0+16(%rsi),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	addq	8+16(%rsi),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	addq	32+16(%rsi),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	addq	40+16(%rsi),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	addq	64+16(%rsi),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	addq	72+16(%rsi),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	addq	96+16(%rsi),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	addq	104+16(%rsi),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	leaq	-128(%rsi),%rsi+	cmpq	%rsp,%rsi+	jae	.Lower_avx2++	movq	-64(%rbp),%rdi+	addq	%r14,%rax+	movq	-56(%rbp),%rsi+	leaq	1152(%rsp),%rsp++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	leaq	256(%rsi),%rsi+	addq	48(%rdi),%r10+	movq	%rsi,%r12+	addq	56(%rdi),%r11+	cmpq	-48(%rbp),%rsi++	movq	%rax,0(%rdi)+	cmoveq	%rsp,%r12+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)++	jbe	.Loop_avx2++.Ldone_avx2:+	vzeroupper+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+.size	crypton_sha512_asm_block_data_order_avx2,.-crypton_sha512_asm_block_data_order_avx2++.section	.note.gnu.property,"a",@note+	.long	4,2f-1f,5+	.byte	0x47,0x4E,0x55,0+1:	.long	0xc0000002,4,3+.align	8+2:++.section	.note.GNU-stack,"",@progbits
+ cbits/asm/sha512-x86_64-macosx.S view
@@ -0,0 +1,5718 @@+.text	+++.globl	_crypton_sha512_asm_block_data_order++.p2align	4+_crypton_sha512_asm_block_data_order:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+	leaq	_crypton_ia32cap_P(%rip),%rax+	movl	0(%rax),%r9d+	movl	4(%rax),%r10d+	movl	8(%rax),%eax+	testl	$2048,%r10d+	jnz	L$xop_shortcut+	andl	$296,%eax+	cmpl	$296,%eax+	je	L$avx2_shortcut+	andl	$1073741824,%r9d+	andl	$268435968,%r10d+	orl	%r9d,%r10d+	cmpl	$1342177792,%r10d+	je	L$avx_shortcut+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$128+24,%rsp++.cfi_def_cfa	%rsp,208++	leaq	(%rsi,%rdx,8),%rdx+	movq	%rdi,128+0(%rsp)+	movq	%rsi,128+8(%rsp)+	movq	%rdx,128+16(%rsp)++	movq	0(%rdi),%rax+	movq	8(%rdi),%rbx+	movq	16(%rdi),%rcx+	movq	24(%rdi),%rdx+	movq	32(%rdi),%r8+	movq	40(%rdi),%r9+	movq	48(%rdi),%r10+	movq	56(%rdi),%r11+	jmp	L$loop++.p2align	4+L$loop:+	movq	%rbx,%rdi+	leaq	K512(%rip),%rbp+	xorq	%rcx,%rdi+	movq	0(%rsi),%r12+	movq	%r8,%r13+	movq	%rax,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r9,%r15++	xorq	%r8,%r13+	rorq	$5,%r14+	xorq	%r10,%r15++	movq	%r12,0(%rsp)+	xorq	%rax,%r14+	andq	%r8,%r15++	rorq	$4,%r13+	addq	%r11,%r12+	xorq	%r10,%r15++	rorq	$6,%r14+	xorq	%r8,%r13+	addq	%r15,%r12++	movq	%rax,%r15+	addq	(%rbp),%r12+	xorq	%rax,%r14++	xorq	%rbx,%r15+	rorq	$14,%r13+	movq	%rbx,%r11++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r11+	addq	%r12,%rdx+	addq	%r12,%r11++	leaq	8(%rbp),%rbp+	addq	%r14,%r11+	movq	8(%rsi),%r12+	movq	%rdx,%r13+	movq	%r11,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r8,%rdi++	xorq	%rdx,%r13+	rorq	$5,%r14+	xorq	%r9,%rdi++	movq	%r12,8(%rsp)+	xorq	%r11,%r14+	andq	%rdx,%rdi++	rorq	$4,%r13+	addq	%r10,%r12+	xorq	%r9,%rdi++	rorq	$6,%r14+	xorq	%rdx,%r13+	addq	%rdi,%r12++	movq	%r11,%rdi+	addq	(%rbp),%r12+	xorq	%r11,%r14++	xorq	%rax,%rdi+	rorq	$14,%r13+	movq	%rax,%r10++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r10+	addq	%r12,%rcx+	addq	%r12,%r10++	leaq	24(%rbp),%rbp+	addq	%r14,%r10+	movq	16(%rsi),%r12+	movq	%rcx,%r13+	movq	%r10,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rdx,%r15++	xorq	%rcx,%r13+	rorq	$5,%r14+	xorq	%r8,%r15++	movq	%r12,16(%rsp)+	xorq	%r10,%r14+	andq	%rcx,%r15++	rorq	$4,%r13+	addq	%r9,%r12+	xorq	%r8,%r15++	rorq	$6,%r14+	xorq	%rcx,%r13+	addq	%r15,%r12++	movq	%r10,%r15+	addq	(%rbp),%r12+	xorq	%r10,%r14++	xorq	%r11,%r15+	rorq	$14,%r13+	movq	%r11,%r9++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r9+	addq	%r12,%rbx+	addq	%r12,%r9++	leaq	8(%rbp),%rbp+	addq	%r14,%r9+	movq	24(%rsi),%r12+	movq	%rbx,%r13+	movq	%r9,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rcx,%rdi++	xorq	%rbx,%r13+	rorq	$5,%r14+	xorq	%rdx,%rdi++	movq	%r12,24(%rsp)+	xorq	%r9,%r14+	andq	%rbx,%rdi++	rorq	$4,%r13+	addq	%r8,%r12+	xorq	%rdx,%rdi++	rorq	$6,%r14+	xorq	%rbx,%r13+	addq	%rdi,%r12++	movq	%r9,%rdi+	addq	(%rbp),%r12+	xorq	%r9,%r14++	xorq	%r10,%rdi+	rorq	$14,%r13+	movq	%r10,%r8++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r8+	addq	%r12,%rax+	addq	%r12,%r8++	leaq	24(%rbp),%rbp+	addq	%r14,%r8+	movq	32(%rsi),%r12+	movq	%rax,%r13+	movq	%r8,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rbx,%r15++	xorq	%rax,%r13+	rorq	$5,%r14+	xorq	%rcx,%r15++	movq	%r12,32(%rsp)+	xorq	%r8,%r14+	andq	%rax,%r15++	rorq	$4,%r13+	addq	%rdx,%r12+	xorq	%rcx,%r15++	rorq	$6,%r14+	xorq	%rax,%r13+	addq	%r15,%r12++	movq	%r8,%r15+	addq	(%rbp),%r12+	xorq	%r8,%r14++	xorq	%r9,%r15+	rorq	$14,%r13+	movq	%r9,%rdx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rdx+	addq	%r12,%r11+	addq	%r12,%rdx++	leaq	8(%rbp),%rbp+	addq	%r14,%rdx+	movq	40(%rsi),%r12+	movq	%r11,%r13+	movq	%rdx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rax,%rdi++	xorq	%r11,%r13+	rorq	$5,%r14+	xorq	%rbx,%rdi++	movq	%r12,40(%rsp)+	xorq	%rdx,%r14+	andq	%r11,%rdi++	rorq	$4,%r13+	addq	%rcx,%r12+	xorq	%rbx,%rdi++	rorq	$6,%r14+	xorq	%r11,%r13+	addq	%rdi,%r12++	movq	%rdx,%rdi+	addq	(%rbp),%r12+	xorq	%rdx,%r14++	xorq	%r8,%rdi+	rorq	$14,%r13+	movq	%r8,%rcx++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rcx+	addq	%r12,%r10+	addq	%r12,%rcx++	leaq	24(%rbp),%rbp+	addq	%r14,%rcx+	movq	48(%rsi),%r12+	movq	%r10,%r13+	movq	%rcx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r11,%r15++	xorq	%r10,%r13+	rorq	$5,%r14+	xorq	%rax,%r15++	movq	%r12,48(%rsp)+	xorq	%rcx,%r14+	andq	%r10,%r15++	rorq	$4,%r13+	addq	%rbx,%r12+	xorq	%rax,%r15++	rorq	$6,%r14+	xorq	%r10,%r13+	addq	%r15,%r12++	movq	%rcx,%r15+	addq	(%rbp),%r12+	xorq	%rcx,%r14++	xorq	%rdx,%r15+	rorq	$14,%r13+	movq	%rdx,%rbx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rbx+	addq	%r12,%r9+	addq	%r12,%rbx++	leaq	8(%rbp),%rbp+	addq	%r14,%rbx+	movq	56(%rsi),%r12+	movq	%r9,%r13+	movq	%rbx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r10,%rdi++	xorq	%r9,%r13+	rorq	$5,%r14+	xorq	%r11,%rdi++	movq	%r12,56(%rsp)+	xorq	%rbx,%r14+	andq	%r9,%rdi++	rorq	$4,%r13+	addq	%rax,%r12+	xorq	%r11,%rdi++	rorq	$6,%r14+	xorq	%r9,%r13+	addq	%rdi,%r12++	movq	%rbx,%rdi+	addq	(%rbp),%r12+	xorq	%rbx,%r14++	xorq	%rcx,%rdi+	rorq	$14,%r13+	movq	%rcx,%rax++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rax+	addq	%r12,%r8+	addq	%r12,%rax++	leaq	24(%rbp),%rbp+	addq	%r14,%rax+	movq	64(%rsi),%r12+	movq	%r8,%r13+	movq	%rax,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r9,%r15++	xorq	%r8,%r13+	rorq	$5,%r14+	xorq	%r10,%r15++	movq	%r12,64(%rsp)+	xorq	%rax,%r14+	andq	%r8,%r15++	rorq	$4,%r13+	addq	%r11,%r12+	xorq	%r10,%r15++	rorq	$6,%r14+	xorq	%r8,%r13+	addq	%r15,%r12++	movq	%rax,%r15+	addq	(%rbp),%r12+	xorq	%rax,%r14++	xorq	%rbx,%r15+	rorq	$14,%r13+	movq	%rbx,%r11++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r11+	addq	%r12,%rdx+	addq	%r12,%r11++	leaq	8(%rbp),%rbp+	addq	%r14,%r11+	movq	72(%rsi),%r12+	movq	%rdx,%r13+	movq	%r11,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r8,%rdi++	xorq	%rdx,%r13+	rorq	$5,%r14+	xorq	%r9,%rdi++	movq	%r12,72(%rsp)+	xorq	%r11,%r14+	andq	%rdx,%rdi++	rorq	$4,%r13+	addq	%r10,%r12+	xorq	%r9,%rdi++	rorq	$6,%r14+	xorq	%rdx,%r13+	addq	%rdi,%r12++	movq	%r11,%rdi+	addq	(%rbp),%r12+	xorq	%r11,%r14++	xorq	%rax,%rdi+	rorq	$14,%r13+	movq	%rax,%r10++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r10+	addq	%r12,%rcx+	addq	%r12,%r10++	leaq	24(%rbp),%rbp+	addq	%r14,%r10+	movq	80(%rsi),%r12+	movq	%rcx,%r13+	movq	%r10,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rdx,%r15++	xorq	%rcx,%r13+	rorq	$5,%r14+	xorq	%r8,%r15++	movq	%r12,80(%rsp)+	xorq	%r10,%r14+	andq	%rcx,%r15++	rorq	$4,%r13+	addq	%r9,%r12+	xorq	%r8,%r15++	rorq	$6,%r14+	xorq	%rcx,%r13+	addq	%r15,%r12++	movq	%r10,%r15+	addq	(%rbp),%r12+	xorq	%r10,%r14++	xorq	%r11,%r15+	rorq	$14,%r13+	movq	%r11,%r9++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r9+	addq	%r12,%rbx+	addq	%r12,%r9++	leaq	8(%rbp),%rbp+	addq	%r14,%r9+	movq	88(%rsi),%r12+	movq	%rbx,%r13+	movq	%r9,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rcx,%rdi++	xorq	%rbx,%r13+	rorq	$5,%r14+	xorq	%rdx,%rdi++	movq	%r12,88(%rsp)+	xorq	%r9,%r14+	andq	%rbx,%rdi++	rorq	$4,%r13+	addq	%r8,%r12+	xorq	%rdx,%rdi++	rorq	$6,%r14+	xorq	%rbx,%r13+	addq	%rdi,%r12++	movq	%r9,%rdi+	addq	(%rbp),%r12+	xorq	%r9,%r14++	xorq	%r10,%rdi+	rorq	$14,%r13+	movq	%r10,%r8++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r8+	addq	%r12,%rax+	addq	%r12,%r8++	leaq	24(%rbp),%rbp+	addq	%r14,%r8+	movq	96(%rsi),%r12+	movq	%rax,%r13+	movq	%r8,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rbx,%r15++	xorq	%rax,%r13+	rorq	$5,%r14+	xorq	%rcx,%r15++	movq	%r12,96(%rsp)+	xorq	%r8,%r14+	andq	%rax,%r15++	rorq	$4,%r13+	addq	%rdx,%r12+	xorq	%rcx,%r15++	rorq	$6,%r14+	xorq	%rax,%r13+	addq	%r15,%r12++	movq	%r8,%r15+	addq	(%rbp),%r12+	xorq	%r8,%r14++	xorq	%r9,%r15+	rorq	$14,%r13+	movq	%r9,%rdx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rdx+	addq	%r12,%r11+	addq	%r12,%rdx++	leaq	8(%rbp),%rbp+	addq	%r14,%rdx+	movq	104(%rsi),%r12+	movq	%r11,%r13+	movq	%rdx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rax,%rdi++	xorq	%r11,%r13+	rorq	$5,%r14+	xorq	%rbx,%rdi++	movq	%r12,104(%rsp)+	xorq	%rdx,%r14+	andq	%r11,%rdi++	rorq	$4,%r13+	addq	%rcx,%r12+	xorq	%rbx,%rdi++	rorq	$6,%r14+	xorq	%r11,%r13+	addq	%rdi,%r12++	movq	%rdx,%rdi+	addq	(%rbp),%r12+	xorq	%rdx,%r14++	xorq	%r8,%rdi+	rorq	$14,%r13+	movq	%r8,%rcx++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rcx+	addq	%r12,%r10+	addq	%r12,%rcx++	leaq	24(%rbp),%rbp+	addq	%r14,%rcx+	movq	112(%rsi),%r12+	movq	%r10,%r13+	movq	%rcx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r11,%r15++	xorq	%r10,%r13+	rorq	$5,%r14+	xorq	%rax,%r15++	movq	%r12,112(%rsp)+	xorq	%rcx,%r14+	andq	%r10,%r15++	rorq	$4,%r13+	addq	%rbx,%r12+	xorq	%rax,%r15++	rorq	$6,%r14+	xorq	%r10,%r13+	addq	%r15,%r12++	movq	%rcx,%r15+	addq	(%rbp),%r12+	xorq	%rcx,%r14++	xorq	%rdx,%r15+	rorq	$14,%r13+	movq	%rdx,%rbx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rbx+	addq	%r12,%r9+	addq	%r12,%rbx++	leaq	8(%rbp),%rbp+	addq	%r14,%rbx+	movq	120(%rsi),%r12+	movq	%r9,%r13+	movq	%rbx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r10,%rdi++	xorq	%r9,%r13+	rorq	$5,%r14+	xorq	%r11,%rdi++	movq	%r12,120(%rsp)+	xorq	%rbx,%r14+	andq	%r9,%rdi++	rorq	$4,%r13+	addq	%rax,%r12+	xorq	%r11,%rdi++	rorq	$6,%r14+	xorq	%r9,%r13+	addq	%rdi,%r12++	movq	%rbx,%rdi+	addq	(%rbp),%r12+	xorq	%rbx,%r14++	xorq	%rcx,%rdi+	rorq	$14,%r13+	movq	%rcx,%rax++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rax+	addq	%r12,%r8+	addq	%r12,%rax++	leaq	24(%rbp),%rbp+	jmp	L$rounds_16_xx+.p2align	4+L$rounds_16_xx:+	movq	8(%rsp),%r13+	movq	112(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rax+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	72(%rsp),%r12++	addq	0(%rsp),%r12+	movq	%r8,%r13+	addq	%r15,%r12+	movq	%rax,%r14+	rorq	$23,%r13+	movq	%r9,%r15++	xorq	%r8,%r13+	rorq	$5,%r14+	xorq	%r10,%r15++	movq	%r12,0(%rsp)+	xorq	%rax,%r14+	andq	%r8,%r15++	rorq	$4,%r13+	addq	%r11,%r12+	xorq	%r10,%r15++	rorq	$6,%r14+	xorq	%r8,%r13+	addq	%r15,%r12++	movq	%rax,%r15+	addq	(%rbp),%r12+	xorq	%rax,%r14++	xorq	%rbx,%r15+	rorq	$14,%r13+	movq	%rbx,%r11++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r11+	addq	%r12,%rdx+	addq	%r12,%r11++	leaq	8(%rbp),%rbp+	movq	16(%rsp),%r13+	movq	120(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r11+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	80(%rsp),%r12++	addq	8(%rsp),%r12+	movq	%rdx,%r13+	addq	%rdi,%r12+	movq	%r11,%r14+	rorq	$23,%r13+	movq	%r8,%rdi++	xorq	%rdx,%r13+	rorq	$5,%r14+	xorq	%r9,%rdi++	movq	%r12,8(%rsp)+	xorq	%r11,%r14+	andq	%rdx,%rdi++	rorq	$4,%r13+	addq	%r10,%r12+	xorq	%r9,%rdi++	rorq	$6,%r14+	xorq	%rdx,%r13+	addq	%rdi,%r12++	movq	%r11,%rdi+	addq	(%rbp),%r12+	xorq	%r11,%r14++	xorq	%rax,%rdi+	rorq	$14,%r13+	movq	%rax,%r10++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r10+	addq	%r12,%rcx+	addq	%r12,%r10++	leaq	24(%rbp),%rbp+	movq	24(%rsp),%r13+	movq	0(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r10+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	88(%rsp),%r12++	addq	16(%rsp),%r12+	movq	%rcx,%r13+	addq	%r15,%r12+	movq	%r10,%r14+	rorq	$23,%r13+	movq	%rdx,%r15++	xorq	%rcx,%r13+	rorq	$5,%r14+	xorq	%r8,%r15++	movq	%r12,16(%rsp)+	xorq	%r10,%r14+	andq	%rcx,%r15++	rorq	$4,%r13+	addq	%r9,%r12+	xorq	%r8,%r15++	rorq	$6,%r14+	xorq	%rcx,%r13+	addq	%r15,%r12++	movq	%r10,%r15+	addq	(%rbp),%r12+	xorq	%r10,%r14++	xorq	%r11,%r15+	rorq	$14,%r13+	movq	%r11,%r9++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r9+	addq	%r12,%rbx+	addq	%r12,%r9++	leaq	8(%rbp),%rbp+	movq	32(%rsp),%r13+	movq	8(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r9+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	96(%rsp),%r12++	addq	24(%rsp),%r12+	movq	%rbx,%r13+	addq	%rdi,%r12+	movq	%r9,%r14+	rorq	$23,%r13+	movq	%rcx,%rdi++	xorq	%rbx,%r13+	rorq	$5,%r14+	xorq	%rdx,%rdi++	movq	%r12,24(%rsp)+	xorq	%r9,%r14+	andq	%rbx,%rdi++	rorq	$4,%r13+	addq	%r8,%r12+	xorq	%rdx,%rdi++	rorq	$6,%r14+	xorq	%rbx,%r13+	addq	%rdi,%r12++	movq	%r9,%rdi+	addq	(%rbp),%r12+	xorq	%r9,%r14++	xorq	%r10,%rdi+	rorq	$14,%r13+	movq	%r10,%r8++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r8+	addq	%r12,%rax+	addq	%r12,%r8++	leaq	24(%rbp),%rbp+	movq	40(%rsp),%r13+	movq	16(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r8+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	104(%rsp),%r12++	addq	32(%rsp),%r12+	movq	%rax,%r13+	addq	%r15,%r12+	movq	%r8,%r14+	rorq	$23,%r13+	movq	%rbx,%r15++	xorq	%rax,%r13+	rorq	$5,%r14+	xorq	%rcx,%r15++	movq	%r12,32(%rsp)+	xorq	%r8,%r14+	andq	%rax,%r15++	rorq	$4,%r13+	addq	%rdx,%r12+	xorq	%rcx,%r15++	rorq	$6,%r14+	xorq	%rax,%r13+	addq	%r15,%r12++	movq	%r8,%r15+	addq	(%rbp),%r12+	xorq	%r8,%r14++	xorq	%r9,%r15+	rorq	$14,%r13+	movq	%r9,%rdx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rdx+	addq	%r12,%r11+	addq	%r12,%rdx++	leaq	8(%rbp),%rbp+	movq	48(%rsp),%r13+	movq	24(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rdx+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	112(%rsp),%r12++	addq	40(%rsp),%r12+	movq	%r11,%r13+	addq	%rdi,%r12+	movq	%rdx,%r14+	rorq	$23,%r13+	movq	%rax,%rdi++	xorq	%r11,%r13+	rorq	$5,%r14+	xorq	%rbx,%rdi++	movq	%r12,40(%rsp)+	xorq	%rdx,%r14+	andq	%r11,%rdi++	rorq	$4,%r13+	addq	%rcx,%r12+	xorq	%rbx,%rdi++	rorq	$6,%r14+	xorq	%r11,%r13+	addq	%rdi,%r12++	movq	%rdx,%rdi+	addq	(%rbp),%r12+	xorq	%rdx,%r14++	xorq	%r8,%rdi+	rorq	$14,%r13+	movq	%r8,%rcx++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rcx+	addq	%r12,%r10+	addq	%r12,%rcx++	leaq	24(%rbp),%rbp+	movq	56(%rsp),%r13+	movq	32(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rcx+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	120(%rsp),%r12++	addq	48(%rsp),%r12+	movq	%r10,%r13+	addq	%r15,%r12+	movq	%rcx,%r14+	rorq	$23,%r13+	movq	%r11,%r15++	xorq	%r10,%r13+	rorq	$5,%r14+	xorq	%rax,%r15++	movq	%r12,48(%rsp)+	xorq	%rcx,%r14+	andq	%r10,%r15++	rorq	$4,%r13+	addq	%rbx,%r12+	xorq	%rax,%r15++	rorq	$6,%r14+	xorq	%r10,%r13+	addq	%r15,%r12++	movq	%rcx,%r15+	addq	(%rbp),%r12+	xorq	%rcx,%r14++	xorq	%rdx,%r15+	rorq	$14,%r13+	movq	%rdx,%rbx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rbx+	addq	%r12,%r9+	addq	%r12,%rbx++	leaq	8(%rbp),%rbp+	movq	64(%rsp),%r13+	movq	40(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rbx+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	0(%rsp),%r12++	addq	56(%rsp),%r12+	movq	%r9,%r13+	addq	%rdi,%r12+	movq	%rbx,%r14+	rorq	$23,%r13+	movq	%r10,%rdi++	xorq	%r9,%r13+	rorq	$5,%r14+	xorq	%r11,%rdi++	movq	%r12,56(%rsp)+	xorq	%rbx,%r14+	andq	%r9,%rdi++	rorq	$4,%r13+	addq	%rax,%r12+	xorq	%r11,%rdi++	rorq	$6,%r14+	xorq	%r9,%r13+	addq	%rdi,%r12++	movq	%rbx,%rdi+	addq	(%rbp),%r12+	xorq	%rbx,%r14++	xorq	%rcx,%rdi+	rorq	$14,%r13+	movq	%rcx,%rax++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rax+	addq	%r12,%r8+	addq	%r12,%rax++	leaq	24(%rbp),%rbp+	movq	72(%rsp),%r13+	movq	48(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rax+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	8(%rsp),%r12++	addq	64(%rsp),%r12+	movq	%r8,%r13+	addq	%r15,%r12+	movq	%rax,%r14+	rorq	$23,%r13+	movq	%r9,%r15++	xorq	%r8,%r13+	rorq	$5,%r14+	xorq	%r10,%r15++	movq	%r12,64(%rsp)+	xorq	%rax,%r14+	andq	%r8,%r15++	rorq	$4,%r13+	addq	%r11,%r12+	xorq	%r10,%r15++	rorq	$6,%r14+	xorq	%r8,%r13+	addq	%r15,%r12++	movq	%rax,%r15+	addq	(%rbp),%r12+	xorq	%rax,%r14++	xorq	%rbx,%r15+	rorq	$14,%r13+	movq	%rbx,%r11++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r11+	addq	%r12,%rdx+	addq	%r12,%r11++	leaq	8(%rbp),%rbp+	movq	80(%rsp),%r13+	movq	56(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r11+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	16(%rsp),%r12++	addq	72(%rsp),%r12+	movq	%rdx,%r13+	addq	%rdi,%r12+	movq	%r11,%r14+	rorq	$23,%r13+	movq	%r8,%rdi++	xorq	%rdx,%r13+	rorq	$5,%r14+	xorq	%r9,%rdi++	movq	%r12,72(%rsp)+	xorq	%r11,%r14+	andq	%rdx,%rdi++	rorq	$4,%r13+	addq	%r10,%r12+	xorq	%r9,%rdi++	rorq	$6,%r14+	xorq	%rdx,%r13+	addq	%rdi,%r12++	movq	%r11,%rdi+	addq	(%rbp),%r12+	xorq	%r11,%r14++	xorq	%rax,%rdi+	rorq	$14,%r13+	movq	%rax,%r10++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r10+	addq	%r12,%rcx+	addq	%r12,%r10++	leaq	24(%rbp),%rbp+	movq	88(%rsp),%r13+	movq	64(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r10+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	24(%rsp),%r12++	addq	80(%rsp),%r12+	movq	%rcx,%r13+	addq	%r15,%r12+	movq	%r10,%r14+	rorq	$23,%r13+	movq	%rdx,%r15++	xorq	%rcx,%r13+	rorq	$5,%r14+	xorq	%r8,%r15++	movq	%r12,80(%rsp)+	xorq	%r10,%r14+	andq	%rcx,%r15++	rorq	$4,%r13+	addq	%r9,%r12+	xorq	%r8,%r15++	rorq	$6,%r14+	xorq	%rcx,%r13+	addq	%r15,%r12++	movq	%r10,%r15+	addq	(%rbp),%r12+	xorq	%r10,%r14++	xorq	%r11,%r15+	rorq	$14,%r13+	movq	%r11,%r9++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r9+	addq	%r12,%rbx+	addq	%r12,%r9++	leaq	8(%rbp),%rbp+	movq	96(%rsp),%r13+	movq	72(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r9+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	32(%rsp),%r12++	addq	88(%rsp),%r12+	movq	%rbx,%r13+	addq	%rdi,%r12+	movq	%r9,%r14+	rorq	$23,%r13+	movq	%rcx,%rdi++	xorq	%rbx,%r13+	rorq	$5,%r14+	xorq	%rdx,%rdi++	movq	%r12,88(%rsp)+	xorq	%r9,%r14+	andq	%rbx,%rdi++	rorq	$4,%r13+	addq	%r8,%r12+	xorq	%rdx,%rdi++	rorq	$6,%r14+	xorq	%rbx,%r13+	addq	%rdi,%r12++	movq	%r9,%rdi+	addq	(%rbp),%r12+	xorq	%r9,%r14++	xorq	%r10,%rdi+	rorq	$14,%r13+	movq	%r10,%r8++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r8+	addq	%r12,%rax+	addq	%r12,%r8++	leaq	24(%rbp),%rbp+	movq	104(%rsp),%r13+	movq	80(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r8+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	40(%rsp),%r12++	addq	96(%rsp),%r12+	movq	%rax,%r13+	addq	%r15,%r12+	movq	%r8,%r14+	rorq	$23,%r13+	movq	%rbx,%r15++	xorq	%rax,%r13+	rorq	$5,%r14+	xorq	%rcx,%r15++	movq	%r12,96(%rsp)+	xorq	%r8,%r14+	andq	%rax,%r15++	rorq	$4,%r13+	addq	%rdx,%r12+	xorq	%rcx,%r15++	rorq	$6,%r14+	xorq	%rax,%r13+	addq	%r15,%r12++	movq	%r8,%r15+	addq	(%rbp),%r12+	xorq	%r8,%r14++	xorq	%r9,%r15+	rorq	$14,%r13+	movq	%r9,%rdx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rdx+	addq	%r12,%r11+	addq	%r12,%rdx++	leaq	8(%rbp),%rbp+	movq	112(%rsp),%r13+	movq	88(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rdx+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	48(%rsp),%r12++	addq	104(%rsp),%r12+	movq	%r11,%r13+	addq	%rdi,%r12+	movq	%rdx,%r14+	rorq	$23,%r13+	movq	%rax,%rdi++	xorq	%r11,%r13+	rorq	$5,%r14+	xorq	%rbx,%rdi++	movq	%r12,104(%rsp)+	xorq	%rdx,%r14+	andq	%r11,%rdi++	rorq	$4,%r13+	addq	%rcx,%r12+	xorq	%rbx,%rdi++	rorq	$6,%r14+	xorq	%r11,%r13+	addq	%rdi,%r12++	movq	%rdx,%rdi+	addq	(%rbp),%r12+	xorq	%rdx,%r14++	xorq	%r8,%rdi+	rorq	$14,%r13+	movq	%r8,%rcx++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rcx+	addq	%r12,%r10+	addq	%r12,%rcx++	leaq	24(%rbp),%rbp+	movq	120(%rsp),%r13+	movq	96(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rcx+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	56(%rsp),%r12++	addq	112(%rsp),%r12+	movq	%r10,%r13+	addq	%r15,%r12+	movq	%rcx,%r14+	rorq	$23,%r13+	movq	%r11,%r15++	xorq	%r10,%r13+	rorq	$5,%r14+	xorq	%rax,%r15++	movq	%r12,112(%rsp)+	xorq	%rcx,%r14+	andq	%r10,%r15++	rorq	$4,%r13+	addq	%rbx,%r12+	xorq	%rax,%r15++	rorq	$6,%r14+	xorq	%r10,%r13+	addq	%r15,%r12++	movq	%rcx,%r15+	addq	(%rbp),%r12+	xorq	%rcx,%r14++	xorq	%rdx,%r15+	rorq	$14,%r13+	movq	%rdx,%rbx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rbx+	addq	%r12,%r9+	addq	%r12,%rbx++	leaq	8(%rbp),%rbp+	movq	0(%rsp),%r13+	movq	104(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rbx+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	64(%rsp),%r12++	addq	120(%rsp),%r12+	movq	%r9,%r13+	addq	%rdi,%r12+	movq	%rbx,%r14+	rorq	$23,%r13+	movq	%r10,%rdi++	xorq	%r9,%r13+	rorq	$5,%r14+	xorq	%r11,%rdi++	movq	%r12,120(%rsp)+	xorq	%rbx,%r14+	andq	%r9,%rdi++	rorq	$4,%r13+	addq	%rax,%r12+	xorq	%r11,%rdi++	rorq	$6,%r14+	xorq	%r9,%r13+	addq	%rdi,%r12++	movq	%rbx,%rdi+	addq	(%rbp),%r12+	xorq	%rbx,%r14++	xorq	%rcx,%rdi+	rorq	$14,%r13+	movq	%rcx,%rax++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rax+	addq	%r12,%r8+	addq	%r12,%rax++	leaq	24(%rbp),%rbp+	cmpb	$0,7(%rbp)+	jnz	L$rounds_16_xx++	movq	128+0(%rsp),%rdi+	addq	%r14,%rax+	leaq	128(%rsi),%rsi++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	addq	48(%rdi),%r10+	addq	56(%rdi),%r11++	cmpq	128+16(%rsp),%rsi++	movq	%rax,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)+	jb	L$loop++	leaq	128+24+48(%rsp),%r11+.cfi_def_cfa	%r11,8+	movq	128+24(%rsp),%r15+	movq	-40(%r11),%r14+	movq	-32(%r11),%r13+	movq	-24(%r11),%r12+	movq	-16(%r11),%rbx+	movq	-8(%r11),%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbp+.cfi_restore	%rbx+	leaq	(%r11),%rsp+	.byte	0xf3,0xc3+.cfi_endproc	++.p2align	6++K512:+.quad	0x428a2f98d728ae22,0x7137449123ef65cd+.quad	0x428a2f98d728ae22,0x7137449123ef65cd+.quad	0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc+.quad	0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc+.quad	0x3956c25bf348b538,0x59f111f1b605d019+.quad	0x3956c25bf348b538,0x59f111f1b605d019+.quad	0x923f82a4af194f9b,0xab1c5ed5da6d8118+.quad	0x923f82a4af194f9b,0xab1c5ed5da6d8118+.quad	0xd807aa98a3030242,0x12835b0145706fbe+.quad	0xd807aa98a3030242,0x12835b0145706fbe+.quad	0x243185be4ee4b28c,0x550c7dc3d5ffb4e2+.quad	0x243185be4ee4b28c,0x550c7dc3d5ffb4e2+.quad	0x72be5d74f27b896f,0x80deb1fe3b1696b1+.quad	0x72be5d74f27b896f,0x80deb1fe3b1696b1+.quad	0x9bdc06a725c71235,0xc19bf174cf692694+.quad	0x9bdc06a725c71235,0xc19bf174cf692694+.quad	0xe49b69c19ef14ad2,0xefbe4786384f25e3+.quad	0xe49b69c19ef14ad2,0xefbe4786384f25e3+.quad	0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65+.quad	0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65+.quad	0x2de92c6f592b0275,0x4a7484aa6ea6e483+.quad	0x2de92c6f592b0275,0x4a7484aa6ea6e483+.quad	0x5cb0a9dcbd41fbd4,0x76f988da831153b5+.quad	0x5cb0a9dcbd41fbd4,0x76f988da831153b5+.quad	0x983e5152ee66dfab,0xa831c66d2db43210+.quad	0x983e5152ee66dfab,0xa831c66d2db43210+.quad	0xb00327c898fb213f,0xbf597fc7beef0ee4+.quad	0xb00327c898fb213f,0xbf597fc7beef0ee4+.quad	0xc6e00bf33da88fc2,0xd5a79147930aa725+.quad	0xc6e00bf33da88fc2,0xd5a79147930aa725+.quad	0x06ca6351e003826f,0x142929670a0e6e70+.quad	0x06ca6351e003826f,0x142929670a0e6e70+.quad	0x27b70a8546d22ffc,0x2e1b21385c26c926+.quad	0x27b70a8546d22ffc,0x2e1b21385c26c926+.quad	0x4d2c6dfc5ac42aed,0x53380d139d95b3df+.quad	0x4d2c6dfc5ac42aed,0x53380d139d95b3df+.quad	0x650a73548baf63de,0x766a0abb3c77b2a8+.quad	0x650a73548baf63de,0x766a0abb3c77b2a8+.quad	0x81c2c92e47edaee6,0x92722c851482353b+.quad	0x81c2c92e47edaee6,0x92722c851482353b+.quad	0xa2bfe8a14cf10364,0xa81a664bbc423001+.quad	0xa2bfe8a14cf10364,0xa81a664bbc423001+.quad	0xc24b8b70d0f89791,0xc76c51a30654be30+.quad	0xc24b8b70d0f89791,0xc76c51a30654be30+.quad	0xd192e819d6ef5218,0xd69906245565a910+.quad	0xd192e819d6ef5218,0xd69906245565a910+.quad	0xf40e35855771202a,0x106aa07032bbd1b8+.quad	0xf40e35855771202a,0x106aa07032bbd1b8+.quad	0x19a4c116b8d2d0c8,0x1e376c085141ab53+.quad	0x19a4c116b8d2d0c8,0x1e376c085141ab53+.quad	0x2748774cdf8eeb99,0x34b0bcb5e19b48a8+.quad	0x2748774cdf8eeb99,0x34b0bcb5e19b48a8+.quad	0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb+.quad	0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb+.quad	0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3+.quad	0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3+.quad	0x748f82ee5defb2fc,0x78a5636f43172f60+.quad	0x748f82ee5defb2fc,0x78a5636f43172f60+.quad	0x84c87814a1f0ab72,0x8cc702081a6439ec+.quad	0x84c87814a1f0ab72,0x8cc702081a6439ec+.quad	0x90befffa23631e28,0xa4506cebde82bde9+.quad	0x90befffa23631e28,0xa4506cebde82bde9+.quad	0xbef9a3f7b2c67915,0xc67178f2e372532b+.quad	0xbef9a3f7b2c67915,0xc67178f2e372532b+.quad	0xca273eceea26619c,0xd186b8c721c0c207+.quad	0xca273eceea26619c,0xd186b8c721c0c207+.quad	0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178+.quad	0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178+.quad	0x06f067aa72176fba,0x0a637dc5a2c898a6+.quad	0x06f067aa72176fba,0x0a637dc5a2c898a6+.quad	0x113f9804bef90dae,0x1b710b35131c471b+.quad	0x113f9804bef90dae,0x1b710b35131c471b+.quad	0x28db77f523047d84,0x32caab7b40c72493+.quad	0x28db77f523047d84,0x32caab7b40c72493+.quad	0x3c9ebe0a15c9bebc,0x431d67c49c100d4c+.quad	0x3c9ebe0a15c9bebc,0x431d67c49c100d4c+.quad	0x4cc5d4becb3e42b6,0x597f299cfc657e2a+.quad	0x4cc5d4becb3e42b6,0x597f299cfc657e2a+.quad	0x5fcb6fab3ad6faec,0x6c44198c4a475817+.quad	0x5fcb6fab3ad6faec,0x6c44198c4a475817++.quad	0x0001020304050607,0x08090a0b0c0d0e0f+.quad	0x0001020304050607,0x08090a0b0c0d0e0f++K512_nodup:+.quad	0x428a2f98d728ae22,0x7137449123ef65cd+.quad	0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc+.quad	0x3956c25bf348b538,0x59f111f1b605d019+.quad	0x923f82a4af194f9b,0xab1c5ed5da6d8118+.quad	0xd807aa98a3030242,0x12835b0145706fbe+.quad	0x243185be4ee4b28c,0x550c7dc3d5ffb4e2+.quad	0x72be5d74f27b896f,0x80deb1fe3b1696b1+.quad	0x9bdc06a725c71235,0xc19bf174cf692694+.quad	0xe49b69c19ef14ad2,0xefbe4786384f25e3+.quad	0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65+.quad	0x2de92c6f592b0275,0x4a7484aa6ea6e483+.quad	0x5cb0a9dcbd41fbd4,0x76f988da831153b5+.quad	0x983e5152ee66dfab,0xa831c66d2db43210+.quad	0xb00327c898fb213f,0xbf597fc7beef0ee4+.quad	0xc6e00bf33da88fc2,0xd5a79147930aa725+.quad	0x06ca6351e003826f,0x142929670a0e6e70+.quad	0x27b70a8546d22ffc,0x2e1b21385c26c926+.quad	0x4d2c6dfc5ac42aed,0x53380d139d95b3df+.quad	0x650a73548baf63de,0x766a0abb3c77b2a8+.quad	0x81c2c92e47edaee6,0x92722c851482353b+.quad	0xa2bfe8a14cf10364,0xa81a664bbc423001+.quad	0xc24b8b70d0f89791,0xc76c51a30654be30+.quad	0xd192e819d6ef5218,0xd69906245565a910+.quad	0xf40e35855771202a,0x106aa07032bbd1b8+.quad	0x19a4c116b8d2d0c8,0x1e376c085141ab53+.quad	0x2748774cdf8eeb99,0x34b0bcb5e19b48a8+.quad	0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb+.quad	0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3+.quad	0x748f82ee5defb2fc,0x78a5636f43172f60+.quad	0x84c87814a1f0ab72,0x8cc702081a6439ec+.quad	0x90befffa23631e28,0xa4506cebde82bde9+.quad	0xbef9a3f7b2c67915,0xc67178f2e372532b+.quad	0xca273eceea26619c,0xd186b8c721c0c207+.quad	0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178+.quad	0x06f067aa72176fba,0x0a637dc5a2c898a6+.quad	0x113f9804bef90dae,0x1b710b35131c471b+.quad	0x28db77f523047d84,0x32caab7b40c72493+.quad	0x3c9ebe0a15c9bebc,0x431d67c49c100d4c+.quad	0x4cc5d4becb3e42b6,0x597f299cfc657e2a+.quad	0x5fcb6fab3ad6faec,0x6c44198c4a475817++.byte	83,72,65,53,49,50,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.globl	_crypton_sha512_asm_block_data_order_shaext++.p2align	6+_crypton_sha512_asm_block_data_order_shaext:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+L$shaext_shortcut:++	leaq	K512_nodup+128(%rip),%rcx+	vmovdqu	(%rdi),%ymm0+	vmovdqu	32(%rdi),%ymm1+	vmovdqa	-160(%rcx),%ymm8++	vpermq	$27,%ymm0,%ymm0+	vpblendd	$15,%ymm1,%ymm0,%ymm5+	vpblendd	$15,%ymm0,%ymm1,%ymm6+	vpermq	$225,%ymm5,%ymm5+	vpermq	$75,%ymm6,%ymm6+	jmp	L$oop_shaext++.p2align	4+L$oop_shaext:+	vmovdqu	(%rsi),%ymm0+	vmovdqu	32(%rsi),%ymm1+	vmovdqu	64(%rsi),%ymm2+	vpshufb	%ymm8,%ymm0,%ymm0+	vmovdqu	96(%rsi),%ymm3++	vpaddq	0-128(%rcx),%ymm0,%ymm4+	vpshufb	%ymm8,%ymm1,%ymm1+	vmovdqa	%ymm6,%ymm10+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vmovdqa	%ymm5,%ymm9+.byte	196,226,79,203,236++	vpaddq	32-128(%rcx),%ymm1,%ymm4+	vpshufb	%ymm8,%ymm2,%ymm2+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	leaq	128(%rsi),%rsi+.byte	196,226,127,204,193+.byte	196,226,79,203,236++	vpaddq	64-128(%rcx),%ymm2,%ymm4+	vpshufb	%ymm8,%ymm3,%ymm3+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm3,%ymm2,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm0,%ymm0+.byte	196,226,127,204,202+.byte	196,226,79,203,236++	vpaddq	96-128(%rcx),%ymm3,%ymm4+.byte	196,226,127,205,195+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm0,%ymm3,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm1,%ymm1+.byte	196,226,127,204,211+.byte	196,226,79,203,236+	vpaddq	128-128(%rcx),%ymm0,%ymm4+.byte	196,226,127,205,200+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm1,%ymm0,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm2,%ymm2+.byte	196,226,127,204,216+.byte	196,226,79,203,236+	vpaddq	160-128(%rcx),%ymm1,%ymm4+.byte	196,226,127,205,209+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm2,%ymm1,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm3,%ymm3+.byte	196,226,127,204,193+.byte	196,226,79,203,236+	vpaddq	192-128(%rcx),%ymm2,%ymm4+.byte	196,226,127,205,218+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm3,%ymm2,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm0,%ymm0+.byte	196,226,127,204,202+.byte	196,226,79,203,236+	vpaddq	224-128(%rcx),%ymm3,%ymm4+.byte	196,226,127,205,195+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm0,%ymm3,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm1,%ymm1+.byte	196,226,127,204,211+.byte	196,226,79,203,236+	vpaddq	256-128(%rcx),%ymm0,%ymm4+.byte	196,226,127,205,200+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm1,%ymm0,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm2,%ymm2+.byte	196,226,127,204,216+.byte	196,226,79,203,236+	vpaddq	288-128(%rcx),%ymm1,%ymm4+.byte	196,226,127,205,209+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm2,%ymm1,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm3,%ymm3+.byte	196,226,127,204,193+.byte	196,226,79,203,236+	vpaddq	320-128(%rcx),%ymm2,%ymm4+.byte	196,226,127,205,218+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm3,%ymm2,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm0,%ymm0+.byte	196,226,127,204,202+.byte	196,226,79,203,236+	vpaddq	352-128(%rcx),%ymm3,%ymm4+.byte	196,226,127,205,195+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm0,%ymm3,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm1,%ymm1+.byte	196,226,127,204,211+.byte	196,226,79,203,236+	vpaddq	384-128(%rcx),%ymm0,%ymm4+.byte	196,226,127,205,200+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm1,%ymm0,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm2,%ymm2+.byte	196,226,127,204,216+.byte	196,226,79,203,236+	vpaddq	416-128(%rcx),%ymm1,%ymm4+.byte	196,226,127,205,209+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm2,%ymm1,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm3,%ymm3+.byte	196,226,127,204,193+.byte	196,226,79,203,236+	vpaddq	448-128(%rcx),%ymm2,%ymm4+.byte	196,226,127,205,218+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm3,%ymm2,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm0,%ymm0+.byte	196,226,127,204,202+.byte	196,226,79,203,236+	vpaddq	480-128(%rcx),%ymm3,%ymm4+.byte	196,226,127,205,195+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm0,%ymm3,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm1,%ymm1+.byte	196,226,127,204,211+.byte	196,226,79,203,236+	vpaddq	512-128(%rcx),%ymm0,%ymm4+.byte	196,226,127,205,200+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm1,%ymm0,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm2,%ymm2+.byte	196,226,127,204,216+.byte	196,226,79,203,236+	vpaddq	544-128(%rcx),%ymm1,%ymm4+.byte	196,226,127,205,209+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm2,%ymm1,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+.byte	196,226,79,203,236+	vpaddq	%ymm7,%ymm3,%ymm3++	vpaddq	576-128(%rcx),%ymm2,%ymm4+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+.byte	196,226,127,205,218+.byte	196,226,79,203,236++	vpaddq	608-128(%rcx),%ymm3,%ymm4+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	decq	%rdx+.byte	196,226,79,203,236++	vpaddq	%ymm10,%ymm6,%ymm6+	vpaddq	%ymm9,%ymm5,%ymm5+	jnz	L$oop_shaext++	vpermq	$75,%ymm5,%ymm5+	vpblendd	$240,%ymm6,%ymm5,%ymm1+	vpblendd	$240,%ymm5,%ymm6,%ymm2+	vpermq	$180,%ymm1,%ymm1+	vpermq	$27,%ymm2,%ymm2++	vmovdqu	%ymm1,(%rdi)+	vmovdqu	%ymm2,32(%rdi)++	vzeroupper+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp++	.byte	0xf3,0xc3+.cfi_endproc	+++.p2align	6+crypton_sha512_asm_block_data_order_xop:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+L$xop_shortcut:+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$24,%rsp++	leaq	(%rsi,%rdx,8),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+++	leaq	-128(%rsp),%rsp+	vzeroupper+	andq	$-64,%rsp+	movq	0(%rdi),%rax+	movq	8(%rdi),%rbx+	movq	16(%rdi),%rcx+	movq	24(%rdi),%rdx+	movq	32(%rdi),%r8+	movq	40(%rdi),%r9+	movq	48(%rdi),%r10+	movq	56(%rdi),%r11+	jmp	L$loop_xop+.p2align	4+L$loop_xop:+	vmovdqa	K512+1280(%rip),%xmm11+	movq	%rsi,-56(%rbp)+	vmovdqu	0(%rsi),%xmm0+	vmovdqu	16(%rsi),%xmm1+	vmovdqu	32(%rsi),%xmm2+	vpshufb	%xmm11,%xmm0,%xmm0+	vmovdqu	48(%rsi),%xmm3+	vpshufb	%xmm11,%xmm1,%xmm1+	vmovdqu	64(%rsi),%xmm4+	vpshufb	%xmm11,%xmm2,%xmm2+	vmovdqu	80(%rsi),%xmm5+	vpshufb	%xmm11,%xmm3,%xmm3+	vmovdqu	96(%rsi),%xmm6+	vpshufb	%xmm11,%xmm4,%xmm4+	vmovdqu	112(%rsi),%xmm7+	leaq	K512+128(%rip),%rsi+	vpshufb	%xmm11,%xmm5,%xmm5+	vpaddq	-128(%rsi),%xmm0,%xmm8+	vpshufb	%xmm11,%xmm6,%xmm6+	vpaddq	-96(%rsi),%xmm1,%xmm9+	vpshufb	%xmm11,%xmm7,%xmm7+	vpaddq	-64(%rsi),%xmm2,%xmm10+	vpaddq	-32(%rsi),%xmm3,%xmm11+	vmovdqa	%xmm8,0(%rsp)+	vpaddq	0(%rsi),%xmm4,%xmm8+	vmovdqa	%xmm9,16(%rsp)+	vpaddq	32(%rsi),%xmm5,%xmm9+	vmovdqa	%xmm10,32(%rsp)+	vpaddq	64(%rsi),%xmm6,%xmm10+	vmovdqa	%xmm11,48(%rsp)+	vpaddq	96(%rsi),%xmm7,%xmm11+	vmovdqa	%xmm8,64(%rsp)+	movq	%rax,%r14+	vmovdqa	%xmm9,80(%rsp)+	movq	%rbx,%rdi+	vmovdqa	%xmm10,96(%rsp)+	xorq	%rcx,%rdi+	vmovdqa	%xmm11,112(%rsp)+	movq	%r8,%r13+	jmp	L$xop_00_47++.p2align	4+L$xop_00_47:+	addq	$256,%rsi+	vpalignr	$8,%xmm0,%xmm1,%xmm8+	rorq	$23,%r13+	movq	%r14,%rax+	vpalignr	$8,%xmm4,%xmm5,%xmm11+	movq	%r9,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%r8,%r13+	xorq	%r10,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%rax,%r14+	vpaddq	%xmm11,%xmm0,%xmm0+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	0(%rsp),%r11+	movq	%rax,%r15+.byte	143,72,120,195,209,7+	xorq	%r10,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%rbx,%r15+	addq	%r12,%r11+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,223,3+	xorq	%rax,%r14+	addq	%r13,%r11+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rbx,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm7,%xmm10+	addq	%r11,%rdx+	addq	%rdi,%r11+	vpaddq	%xmm8,%xmm0,%xmm0+	movq	%rdx,%r13+	addq	%r11,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%r11+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%r8,%r12+	rorq	$5,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	vpaddq	%xmm11,%xmm0,%xmm0+	addq	8(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	rorq	$6,%r14+	vpaddq	-128(%rsi),%xmm0,%xmm10+	xorq	%rax,%rdi+	addq	%r12,%r10+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	rorq	$28,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	vmovdqa	%xmm10,0(%rsp)+	vpalignr	$8,%xmm1,%xmm2,%xmm8+	rorq	$23,%r13+	movq	%r14,%r10+	vpalignr	$8,%xmm5,%xmm6,%xmm11+	movq	%rdx,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%rcx,%r13+	xorq	%r8,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%r10,%r14+	vpaddq	%xmm11,%xmm1,%xmm1+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	16(%rsp),%r9+	movq	%r10,%r15+.byte	143,72,120,195,209,7+	xorq	%r8,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%r11,%r15+	addq	%r12,%r9+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,216,3+	xorq	%r10,%r14+	addq	%r13,%r9+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r11,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm0,%xmm10+	addq	%r9,%rbx+	addq	%rdi,%r9+	vpaddq	%xmm8,%xmm1,%xmm1+	movq	%rbx,%r13+	addq	%r9,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%r9+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%rcx,%r12+	rorq	$5,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	vpaddq	%xmm11,%xmm1,%xmm1+	addq	24(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	rorq	$6,%r14+	vpaddq	-96(%rsi),%xmm1,%xmm10+	xorq	%r10,%rdi+	addq	%r12,%r8+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	rorq	$28,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	vmovdqa	%xmm10,16(%rsp)+	vpalignr	$8,%xmm2,%xmm3,%xmm8+	rorq	$23,%r13+	movq	%r14,%r8+	vpalignr	$8,%xmm6,%xmm7,%xmm11+	movq	%rbx,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%rax,%r13+	xorq	%rcx,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%r8,%r14+	vpaddq	%xmm11,%xmm2,%xmm2+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	32(%rsp),%rdx+	movq	%r8,%r15+.byte	143,72,120,195,209,7+	xorq	%rcx,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%r9,%r15+	addq	%r12,%rdx+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,217,3+	xorq	%r8,%r14+	addq	%r13,%rdx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r9,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm1,%xmm10+	addq	%rdx,%r11+	addq	%rdi,%rdx+	vpaddq	%xmm8,%xmm2,%xmm2+	movq	%r11,%r13+	addq	%rdx,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%rdx+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%rax,%r12+	rorq	$5,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	vpaddq	%xmm11,%xmm2,%xmm2+	addq	40(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	rorq	$6,%r14+	vpaddq	-64(%rsi),%xmm2,%xmm10+	xorq	%r8,%rdi+	addq	%r12,%rcx+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	rorq	$28,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	vmovdqa	%xmm10,32(%rsp)+	vpalignr	$8,%xmm3,%xmm4,%xmm8+	rorq	$23,%r13+	movq	%r14,%rcx+	vpalignr	$8,%xmm7,%xmm0,%xmm11+	movq	%r11,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%r10,%r13+	xorq	%rax,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%rcx,%r14+	vpaddq	%xmm11,%xmm3,%xmm3+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	48(%rsp),%rbx+	movq	%rcx,%r15+.byte	143,72,120,195,209,7+	xorq	%rax,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%rdx,%r15+	addq	%r12,%rbx+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,218,3+	xorq	%rcx,%r14+	addq	%r13,%rbx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rdx,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm2,%xmm10+	addq	%rbx,%r9+	addq	%rdi,%rbx+	vpaddq	%xmm8,%xmm3,%xmm3+	movq	%r9,%r13+	addq	%rbx,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%rbx+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%r10,%r12+	rorq	$5,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	vpaddq	%xmm11,%xmm3,%xmm3+	addq	56(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	rorq	$6,%r14+	vpaddq	-32(%rsi),%xmm3,%xmm10+	xorq	%rcx,%rdi+	addq	%r12,%rax+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	rorq	$28,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	vmovdqa	%xmm10,48(%rsp)+	vpalignr	$8,%xmm4,%xmm5,%xmm8+	rorq	$23,%r13+	movq	%r14,%rax+	vpalignr	$8,%xmm0,%xmm1,%xmm11+	movq	%r9,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%r8,%r13+	xorq	%r10,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%rax,%r14+	vpaddq	%xmm11,%xmm4,%xmm4+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	64(%rsp),%r11+	movq	%rax,%r15+.byte	143,72,120,195,209,7+	xorq	%r10,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%rbx,%r15+	addq	%r12,%r11+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,219,3+	xorq	%rax,%r14+	addq	%r13,%r11+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rbx,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm3,%xmm10+	addq	%r11,%rdx+	addq	%rdi,%r11+	vpaddq	%xmm8,%xmm4,%xmm4+	movq	%rdx,%r13+	addq	%r11,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%r11+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%r8,%r12+	rorq	$5,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	vpaddq	%xmm11,%xmm4,%xmm4+	addq	72(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	rorq	$6,%r14+	vpaddq	0(%rsi),%xmm4,%xmm10+	xorq	%rax,%rdi+	addq	%r12,%r10+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	rorq	$28,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	vmovdqa	%xmm10,64(%rsp)+	vpalignr	$8,%xmm5,%xmm6,%xmm8+	rorq	$23,%r13+	movq	%r14,%r10+	vpalignr	$8,%xmm1,%xmm2,%xmm11+	movq	%rdx,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%rcx,%r13+	xorq	%r8,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%r10,%r14+	vpaddq	%xmm11,%xmm5,%xmm5+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	80(%rsp),%r9+	movq	%r10,%r15+.byte	143,72,120,195,209,7+	xorq	%r8,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%r11,%r15+	addq	%r12,%r9+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,220,3+	xorq	%r10,%r14+	addq	%r13,%r9+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r11,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm4,%xmm10+	addq	%r9,%rbx+	addq	%rdi,%r9+	vpaddq	%xmm8,%xmm5,%xmm5+	movq	%rbx,%r13+	addq	%r9,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%r9+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%rcx,%r12+	rorq	$5,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	vpaddq	%xmm11,%xmm5,%xmm5+	addq	88(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	rorq	$6,%r14+	vpaddq	32(%rsi),%xmm5,%xmm10+	xorq	%r10,%rdi+	addq	%r12,%r8+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	rorq	$28,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	vmovdqa	%xmm10,80(%rsp)+	vpalignr	$8,%xmm6,%xmm7,%xmm8+	rorq	$23,%r13+	movq	%r14,%r8+	vpalignr	$8,%xmm2,%xmm3,%xmm11+	movq	%rbx,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%rax,%r13+	xorq	%rcx,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%r8,%r14+	vpaddq	%xmm11,%xmm6,%xmm6+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	96(%rsp),%rdx+	movq	%r8,%r15+.byte	143,72,120,195,209,7+	xorq	%rcx,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%r9,%r15+	addq	%r12,%rdx+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,221,3+	xorq	%r8,%r14+	addq	%r13,%rdx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r9,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm5,%xmm10+	addq	%rdx,%r11+	addq	%rdi,%rdx+	vpaddq	%xmm8,%xmm6,%xmm6+	movq	%r11,%r13+	addq	%rdx,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%rdx+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%rax,%r12+	rorq	$5,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	vpaddq	%xmm11,%xmm6,%xmm6+	addq	104(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	rorq	$6,%r14+	vpaddq	64(%rsi),%xmm6,%xmm10+	xorq	%r8,%rdi+	addq	%r12,%rcx+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	rorq	$28,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	vmovdqa	%xmm10,96(%rsp)+	vpalignr	$8,%xmm7,%xmm0,%xmm8+	rorq	$23,%r13+	movq	%r14,%rcx+	vpalignr	$8,%xmm3,%xmm4,%xmm11+	movq	%r11,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%r10,%r13+	xorq	%rax,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%rcx,%r14+	vpaddq	%xmm11,%xmm7,%xmm7+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	112(%rsp),%rbx+	movq	%rcx,%r15+.byte	143,72,120,195,209,7+	xorq	%rax,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%rdx,%r15+	addq	%r12,%rbx+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,222,3+	xorq	%rcx,%r14+	addq	%r13,%rbx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rdx,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm6,%xmm10+	addq	%rbx,%r9+	addq	%rdi,%rbx+	vpaddq	%xmm8,%xmm7,%xmm7+	movq	%r9,%r13+	addq	%rbx,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%rbx+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%r10,%r12+	rorq	$5,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	vpaddq	%xmm11,%xmm7,%xmm7+	addq	120(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	rorq	$6,%r14+	vpaddq	96(%rsi),%xmm7,%xmm10+	xorq	%rcx,%rdi+	addq	%r12,%rax+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	rorq	$28,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	vmovdqa	%xmm10,112(%rsp)+	cmpb	$0,135(%rsi)+	jne	L$xop_00_47+	rorq	$23,%r13+	movq	%r14,%rax+	movq	%r9,%r12+	rorq	$5,%r14+	xorq	%r8,%r13+	xorq	%r10,%r12+	rorq	$4,%r13+	xorq	%rax,%r14+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	0(%rsp),%r11+	movq	%rax,%r15+	xorq	%r10,%r12+	rorq	$6,%r14+	xorq	%rbx,%r15+	addq	%r12,%r11+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%rax,%r14+	addq	%r13,%r11+	xorq	%rbx,%rdi+	rorq	$28,%r14+	addq	%r11,%rdx+	addq	%rdi,%r11+	movq	%rdx,%r13+	addq	%r11,%r14+	rorq	$23,%r13+	movq	%r14,%r11+	movq	%r8,%r12+	rorq	$5,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	rorq	$4,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	addq	8(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	rorq	$6,%r14+	xorq	%rax,%rdi+	addq	%r12,%r10+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	rorq	$28,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	rorq	$23,%r13+	movq	%r14,%r10+	movq	%rdx,%r12+	rorq	$5,%r14+	xorq	%rcx,%r13+	xorq	%r8,%r12+	rorq	$4,%r13+	xorq	%r10,%r14+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	16(%rsp),%r9+	movq	%r10,%r15+	xorq	%r8,%r12+	rorq	$6,%r14+	xorq	%r11,%r15+	addq	%r12,%r9+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%r10,%r14+	addq	%r13,%r9+	xorq	%r11,%rdi+	rorq	$28,%r14+	addq	%r9,%rbx+	addq	%rdi,%r9+	movq	%rbx,%r13+	addq	%r9,%r14+	rorq	$23,%r13+	movq	%r14,%r9+	movq	%rcx,%r12+	rorq	$5,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	rorq	$4,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	addq	24(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	rorq	$6,%r14+	xorq	%r10,%rdi+	addq	%r12,%r8+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	rorq	$28,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	rorq	$23,%r13+	movq	%r14,%r8+	movq	%rbx,%r12+	rorq	$5,%r14+	xorq	%rax,%r13+	xorq	%rcx,%r12+	rorq	$4,%r13+	xorq	%r8,%r14+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	32(%rsp),%rdx+	movq	%r8,%r15+	xorq	%rcx,%r12+	rorq	$6,%r14+	xorq	%r9,%r15+	addq	%r12,%rdx+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%r8,%r14+	addq	%r13,%rdx+	xorq	%r9,%rdi+	rorq	$28,%r14+	addq	%rdx,%r11+	addq	%rdi,%rdx+	movq	%r11,%r13+	addq	%rdx,%r14+	rorq	$23,%r13+	movq	%r14,%rdx+	movq	%rax,%r12+	rorq	$5,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	rorq	$4,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	addq	40(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	rorq	$6,%r14+	xorq	%r8,%rdi+	addq	%r12,%rcx+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	rorq	$28,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	rorq	$23,%r13+	movq	%r14,%rcx+	movq	%r11,%r12+	rorq	$5,%r14+	xorq	%r10,%r13+	xorq	%rax,%r12+	rorq	$4,%r13+	xorq	%rcx,%r14+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	48(%rsp),%rbx+	movq	%rcx,%r15+	xorq	%rax,%r12+	rorq	$6,%r14+	xorq	%rdx,%r15+	addq	%r12,%rbx+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%rcx,%r14+	addq	%r13,%rbx+	xorq	%rdx,%rdi+	rorq	$28,%r14+	addq	%rbx,%r9+	addq	%rdi,%rbx+	movq	%r9,%r13+	addq	%rbx,%r14+	rorq	$23,%r13+	movq	%r14,%rbx+	movq	%r10,%r12+	rorq	$5,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	rorq	$4,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	addq	56(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	rorq	$6,%r14+	xorq	%rcx,%rdi+	addq	%r12,%rax+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	rorq	$28,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	rorq	$23,%r13+	movq	%r14,%rax+	movq	%r9,%r12+	rorq	$5,%r14+	xorq	%r8,%r13+	xorq	%r10,%r12+	rorq	$4,%r13+	xorq	%rax,%r14+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	64(%rsp),%r11+	movq	%rax,%r15+	xorq	%r10,%r12+	rorq	$6,%r14+	xorq	%rbx,%r15+	addq	%r12,%r11+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%rax,%r14+	addq	%r13,%r11+	xorq	%rbx,%rdi+	rorq	$28,%r14+	addq	%r11,%rdx+	addq	%rdi,%r11+	movq	%rdx,%r13+	addq	%r11,%r14+	rorq	$23,%r13+	movq	%r14,%r11+	movq	%r8,%r12+	rorq	$5,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	rorq	$4,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	addq	72(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	rorq	$6,%r14+	xorq	%rax,%rdi+	addq	%r12,%r10+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	rorq	$28,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	rorq	$23,%r13+	movq	%r14,%r10+	movq	%rdx,%r12+	rorq	$5,%r14+	xorq	%rcx,%r13+	xorq	%r8,%r12+	rorq	$4,%r13+	xorq	%r10,%r14+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	80(%rsp),%r9+	movq	%r10,%r15+	xorq	%r8,%r12+	rorq	$6,%r14+	xorq	%r11,%r15+	addq	%r12,%r9+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%r10,%r14+	addq	%r13,%r9+	xorq	%r11,%rdi+	rorq	$28,%r14+	addq	%r9,%rbx+	addq	%rdi,%r9+	movq	%rbx,%r13+	addq	%r9,%r14+	rorq	$23,%r13+	movq	%r14,%r9+	movq	%rcx,%r12+	rorq	$5,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	rorq	$4,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	addq	88(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	rorq	$6,%r14+	xorq	%r10,%rdi+	addq	%r12,%r8+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	rorq	$28,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	rorq	$23,%r13+	movq	%r14,%r8+	movq	%rbx,%r12+	rorq	$5,%r14+	xorq	%rax,%r13+	xorq	%rcx,%r12+	rorq	$4,%r13+	xorq	%r8,%r14+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	96(%rsp),%rdx+	movq	%r8,%r15+	xorq	%rcx,%r12+	rorq	$6,%r14+	xorq	%r9,%r15+	addq	%r12,%rdx+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%r8,%r14+	addq	%r13,%rdx+	xorq	%r9,%rdi+	rorq	$28,%r14+	addq	%rdx,%r11+	addq	%rdi,%rdx+	movq	%r11,%r13+	addq	%rdx,%r14+	rorq	$23,%r13+	movq	%r14,%rdx+	movq	%rax,%r12+	rorq	$5,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	rorq	$4,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	addq	104(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	rorq	$6,%r14+	xorq	%r8,%rdi+	addq	%r12,%rcx+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	rorq	$28,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	rorq	$23,%r13+	movq	%r14,%rcx+	movq	%r11,%r12+	rorq	$5,%r14+	xorq	%r10,%r13+	xorq	%rax,%r12+	rorq	$4,%r13+	xorq	%rcx,%r14+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	112(%rsp),%rbx+	movq	%rcx,%r15+	xorq	%rax,%r12+	rorq	$6,%r14+	xorq	%rdx,%r15+	addq	%r12,%rbx+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%rcx,%r14+	addq	%r13,%rbx+	xorq	%rdx,%rdi+	rorq	$28,%r14+	addq	%rbx,%r9+	addq	%rdi,%rbx+	movq	%r9,%r13+	addq	%rbx,%r14+	rorq	$23,%r13+	movq	%r14,%rbx+	movq	%r10,%r12+	rorq	$5,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	rorq	$4,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	addq	120(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	rorq	$6,%r14+	xorq	%rcx,%rdi+	addq	%r12,%rax+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	rorq	$28,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	movq	-64(%rbp),%rdi+	movq	%r14,%rax+	movq	-56(%rbp),%rsi++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	addq	48(%rdi),%r10+	addq	56(%rdi),%r11++	leaq	128(%rsi),%rsi+	cmpq	-48(%rbp),%rsi++	movq	%rax,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)+	jb	L$loop_xop++	vzeroupper+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+++.p2align	6+crypton_sha512_asm_block_data_order_avx:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+L$avx_shortcut:+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$24,%rsp++	leaq	(%rsi,%rdx,8),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+++	leaq	-128(%rsp),%rsp+	vzeroupper+	andq	$-64,%rsp+	movq	0(%rdi),%rax+	movq	8(%rdi),%rbx+	movq	16(%rdi),%rcx+	movq	24(%rdi),%rdx+	movq	32(%rdi),%r8+	movq	40(%rdi),%r9+	movq	48(%rdi),%r10+	movq	56(%rdi),%r11+	jmp	L$loop_avx+.p2align	4+L$loop_avx:+	vmovdqa	K512+1280(%rip),%xmm11+	movq	%rsi,-56(%rbp)+	vmovdqu	0(%rsi),%xmm0+	vmovdqu	16(%rsi),%xmm1+	vmovdqu	32(%rsi),%xmm2+	vpshufb	%xmm11,%xmm0,%xmm0+	vmovdqu	48(%rsi),%xmm3+	vpshufb	%xmm11,%xmm1,%xmm1+	vmovdqu	64(%rsi),%xmm4+	vpshufb	%xmm11,%xmm2,%xmm2+	vmovdqu	80(%rsi),%xmm5+	vpshufb	%xmm11,%xmm3,%xmm3+	vmovdqu	96(%rsi),%xmm6+	vpshufb	%xmm11,%xmm4,%xmm4+	vmovdqu	112(%rsi),%xmm7+	leaq	K512+128(%rip),%rsi+	vpshufb	%xmm11,%xmm5,%xmm5+	vpaddq	-128(%rsi),%xmm0,%xmm8+	vpshufb	%xmm11,%xmm6,%xmm6+	vpaddq	-96(%rsi),%xmm1,%xmm9+	vpshufb	%xmm11,%xmm7,%xmm7+	vpaddq	-64(%rsi),%xmm2,%xmm10+	vpaddq	-32(%rsi),%xmm3,%xmm11+	vmovdqa	%xmm8,0(%rsp)+	vpaddq	0(%rsi),%xmm4,%xmm8+	vmovdqa	%xmm9,16(%rsp)+	vpaddq	32(%rsi),%xmm5,%xmm9+	vmovdqa	%xmm10,32(%rsp)+	vpaddq	64(%rsi),%xmm6,%xmm10+	vmovdqa	%xmm11,48(%rsp)+	vpaddq	96(%rsi),%xmm7,%xmm11+	vmovdqa	%xmm8,64(%rsp)+	movq	%rax,%r14+	vmovdqa	%xmm9,80(%rsp)+	movq	%rbx,%rdi+	vmovdqa	%xmm10,96(%rsp)+	xorq	%rcx,%rdi+	vmovdqa	%xmm11,112(%rsp)+	movq	%r8,%r13+	jmp	L$avx_00_47++.p2align	4+L$avx_00_47:+	addq	$256,%rsi+	vpalignr	$8,%xmm0,%xmm1,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%rax+	vpalignr	$8,%xmm4,%xmm5,%xmm11+	movq	%r9,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%r8,%r13+	xorq	%r10,%r12+	vpaddq	%xmm11,%xmm0,%xmm0+	shrdq	$4,%r13,%r13+	xorq	%rax,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%r8,%r12+	xorq	%r8,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	0(%rsp),%r11+	movq	%rax,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%r10,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%rbx,%r15+	addq	%r12,%r11+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%rax,%r14+	addq	%r13,%r11+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rbx,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm7,%xmm11+	addq	%r11,%rdx+	addq	%rdi,%r11+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%rdx,%r13+	addq	%r11,%r14+	vpsllq	$3,%xmm7,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%r11+	vpaddq	%xmm8,%xmm0,%xmm0+	movq	%r8,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm7,%xmm9+	xorq	%rdx,%r13+	xorq	%r9,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%r11,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%rdx,%r12+	xorq	%rdx,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	8(%rsp),%r10+	movq	%r11,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%r9,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%rax,%rdi+	addq	%r12,%r10+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm0,%xmm0+	xorq	%r11,%r14+	addq	%r13,%r10+	vpaddq	-128(%rsi),%xmm0,%xmm10+	xorq	%rax,%r15+	shrdq	$28,%r14,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	vmovdqa	%xmm10,0(%rsp)+	vpalignr	$8,%xmm1,%xmm2,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%r10+	vpalignr	$8,%xmm5,%xmm6,%xmm11+	movq	%rdx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%rcx,%r13+	xorq	%r8,%r12+	vpaddq	%xmm11,%xmm1,%xmm1+	shrdq	$4,%r13,%r13+	xorq	%r10,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%rcx,%r12+	xorq	%rcx,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	16(%rsp),%r9+	movq	%r10,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%r8,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%r11,%r15+	addq	%r12,%r9+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%r10,%r14+	addq	%r13,%r9+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r11,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm0,%xmm11+	addq	%r9,%rbx+	addq	%rdi,%r9+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%rbx,%r13+	addq	%r9,%r14+	vpsllq	$3,%xmm0,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%r9+	vpaddq	%xmm8,%xmm1,%xmm1+	movq	%rcx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm0,%xmm9+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%r9,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%rbx,%r12+	xorq	%rbx,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	24(%rsp),%r8+	movq	%r9,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%rdx,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%r10,%rdi+	addq	%r12,%r8+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm1,%xmm1+	xorq	%r9,%r14+	addq	%r13,%r8+	vpaddq	-96(%rsi),%xmm1,%xmm10+	xorq	%r10,%r15+	shrdq	$28,%r14,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	vmovdqa	%xmm10,16(%rsp)+	vpalignr	$8,%xmm2,%xmm3,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%r8+	vpalignr	$8,%xmm6,%xmm7,%xmm11+	movq	%rbx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%rax,%r13+	xorq	%rcx,%r12+	vpaddq	%xmm11,%xmm2,%xmm2+	shrdq	$4,%r13,%r13+	xorq	%r8,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%rax,%r12+	xorq	%rax,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	32(%rsp),%rdx+	movq	%r8,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%rcx,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%r9,%r15+	addq	%r12,%rdx+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%r8,%r14+	addq	%r13,%rdx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r9,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm1,%xmm11+	addq	%rdx,%r11+	addq	%rdi,%rdx+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%r11,%r13+	addq	%rdx,%r14+	vpsllq	$3,%xmm1,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%rdx+	vpaddq	%xmm8,%xmm2,%xmm2+	movq	%rax,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm1,%xmm9+	xorq	%r11,%r13+	xorq	%rbx,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%rdx,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%r11,%r12+	xorq	%r11,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	40(%rsp),%rcx+	movq	%rdx,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%rbx,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%r8,%rdi+	addq	%r12,%rcx+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm2,%xmm2+	xorq	%rdx,%r14+	addq	%r13,%rcx+	vpaddq	-64(%rsi),%xmm2,%xmm10+	xorq	%r8,%r15+	shrdq	$28,%r14,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	vmovdqa	%xmm10,32(%rsp)+	vpalignr	$8,%xmm3,%xmm4,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%rcx+	vpalignr	$8,%xmm7,%xmm0,%xmm11+	movq	%r11,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%r10,%r13+	xorq	%rax,%r12+	vpaddq	%xmm11,%xmm3,%xmm3+	shrdq	$4,%r13,%r13+	xorq	%rcx,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%r10,%r12+	xorq	%r10,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	48(%rsp),%rbx+	movq	%rcx,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%rax,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%rdx,%r15+	addq	%r12,%rbx+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%rcx,%r14+	addq	%r13,%rbx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rdx,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm2,%xmm11+	addq	%rbx,%r9+	addq	%rdi,%rbx+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%r9,%r13+	addq	%rbx,%r14+	vpsllq	$3,%xmm2,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%rbx+	vpaddq	%xmm8,%xmm3,%xmm3+	movq	%r10,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm2,%xmm9+	xorq	%r9,%r13+	xorq	%r11,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%rbx,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%r9,%r12+	xorq	%r9,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	56(%rsp),%rax+	movq	%rbx,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%r11,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%rcx,%rdi+	addq	%r12,%rax+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm3,%xmm3+	xorq	%rbx,%r14+	addq	%r13,%rax+	vpaddq	-32(%rsi),%xmm3,%xmm10+	xorq	%rcx,%r15+	shrdq	$28,%r14,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	vmovdqa	%xmm10,48(%rsp)+	vpalignr	$8,%xmm4,%xmm5,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%rax+	vpalignr	$8,%xmm0,%xmm1,%xmm11+	movq	%r9,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%r8,%r13+	xorq	%r10,%r12+	vpaddq	%xmm11,%xmm4,%xmm4+	shrdq	$4,%r13,%r13+	xorq	%rax,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%r8,%r12+	xorq	%r8,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	64(%rsp),%r11+	movq	%rax,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%r10,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%rbx,%r15+	addq	%r12,%r11+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%rax,%r14+	addq	%r13,%r11+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rbx,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm3,%xmm11+	addq	%r11,%rdx+	addq	%rdi,%r11+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%rdx,%r13+	addq	%r11,%r14+	vpsllq	$3,%xmm3,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%r11+	vpaddq	%xmm8,%xmm4,%xmm4+	movq	%r8,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm3,%xmm9+	xorq	%rdx,%r13+	xorq	%r9,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%r11,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%rdx,%r12+	xorq	%rdx,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	72(%rsp),%r10+	movq	%r11,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%r9,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%rax,%rdi+	addq	%r12,%r10+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm4,%xmm4+	xorq	%r11,%r14+	addq	%r13,%r10+	vpaddq	0(%rsi),%xmm4,%xmm10+	xorq	%rax,%r15+	shrdq	$28,%r14,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	vmovdqa	%xmm10,64(%rsp)+	vpalignr	$8,%xmm5,%xmm6,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%r10+	vpalignr	$8,%xmm1,%xmm2,%xmm11+	movq	%rdx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%rcx,%r13+	xorq	%r8,%r12+	vpaddq	%xmm11,%xmm5,%xmm5+	shrdq	$4,%r13,%r13+	xorq	%r10,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%rcx,%r12+	xorq	%rcx,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	80(%rsp),%r9+	movq	%r10,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%r8,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%r11,%r15+	addq	%r12,%r9+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%r10,%r14+	addq	%r13,%r9+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r11,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm4,%xmm11+	addq	%r9,%rbx+	addq	%rdi,%r9+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%rbx,%r13+	addq	%r9,%r14+	vpsllq	$3,%xmm4,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%r9+	vpaddq	%xmm8,%xmm5,%xmm5+	movq	%rcx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm4,%xmm9+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%r9,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%rbx,%r12+	xorq	%rbx,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	88(%rsp),%r8+	movq	%r9,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%rdx,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%r10,%rdi+	addq	%r12,%r8+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm5,%xmm5+	xorq	%r9,%r14+	addq	%r13,%r8+	vpaddq	32(%rsi),%xmm5,%xmm10+	xorq	%r10,%r15+	shrdq	$28,%r14,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	vmovdqa	%xmm10,80(%rsp)+	vpalignr	$8,%xmm6,%xmm7,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%r8+	vpalignr	$8,%xmm2,%xmm3,%xmm11+	movq	%rbx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%rax,%r13+	xorq	%rcx,%r12+	vpaddq	%xmm11,%xmm6,%xmm6+	shrdq	$4,%r13,%r13+	xorq	%r8,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%rax,%r12+	xorq	%rax,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	96(%rsp),%rdx+	movq	%r8,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%rcx,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%r9,%r15+	addq	%r12,%rdx+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%r8,%r14+	addq	%r13,%rdx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r9,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm5,%xmm11+	addq	%rdx,%r11+	addq	%rdi,%rdx+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%r11,%r13+	addq	%rdx,%r14+	vpsllq	$3,%xmm5,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%rdx+	vpaddq	%xmm8,%xmm6,%xmm6+	movq	%rax,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm5,%xmm9+	xorq	%r11,%r13+	xorq	%rbx,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%rdx,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%r11,%r12+	xorq	%r11,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	104(%rsp),%rcx+	movq	%rdx,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%rbx,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%r8,%rdi+	addq	%r12,%rcx+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm6,%xmm6+	xorq	%rdx,%r14+	addq	%r13,%rcx+	vpaddq	64(%rsi),%xmm6,%xmm10+	xorq	%r8,%r15+	shrdq	$28,%r14,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	vmovdqa	%xmm10,96(%rsp)+	vpalignr	$8,%xmm7,%xmm0,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%rcx+	vpalignr	$8,%xmm3,%xmm4,%xmm11+	movq	%r11,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%r10,%r13+	xorq	%rax,%r12+	vpaddq	%xmm11,%xmm7,%xmm7+	shrdq	$4,%r13,%r13+	xorq	%rcx,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%r10,%r12+	xorq	%r10,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	112(%rsp),%rbx+	movq	%rcx,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%rax,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%rdx,%r15+	addq	%r12,%rbx+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%rcx,%r14+	addq	%r13,%rbx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rdx,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm6,%xmm11+	addq	%rbx,%r9+	addq	%rdi,%rbx+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%r9,%r13+	addq	%rbx,%r14+	vpsllq	$3,%xmm6,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%rbx+	vpaddq	%xmm8,%xmm7,%xmm7+	movq	%r10,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm6,%xmm9+	xorq	%r9,%r13+	xorq	%r11,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%rbx,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%r9,%r12+	xorq	%r9,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	120(%rsp),%rax+	movq	%rbx,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%r11,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%rcx,%rdi+	addq	%r12,%rax+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm7,%xmm7+	xorq	%rbx,%r14+	addq	%r13,%rax+	vpaddq	96(%rsi),%xmm7,%xmm10+	xorq	%rcx,%r15+	shrdq	$28,%r14,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	vmovdqa	%xmm10,112(%rsp)+	cmpb	$0,135(%rsi)+	jne	L$avx_00_47+	shrdq	$23,%r13,%r13+	movq	%r14,%rax+	movq	%r9,%r12+	shrdq	$5,%r14,%r14+	xorq	%r8,%r13+	xorq	%r10,%r12+	shrdq	$4,%r13,%r13+	xorq	%rax,%r14+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	0(%rsp),%r11+	movq	%rax,%r15+	xorq	%r10,%r12+	shrdq	$6,%r14,%r14+	xorq	%rbx,%r15+	addq	%r12,%r11+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%rax,%r14+	addq	%r13,%r11+	xorq	%rbx,%rdi+	shrdq	$28,%r14,%r14+	addq	%r11,%rdx+	addq	%rdi,%r11+	movq	%rdx,%r13+	addq	%r11,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r11+	movq	%r8,%r12+	shrdq	$5,%r14,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	shrdq	$4,%r13,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	addq	8(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	shrdq	$6,%r14,%r14+	xorq	%rax,%rdi+	addq	%r12,%r10+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	shrdq	$28,%r14,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r10+	movq	%rdx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rcx,%r13+	xorq	%r8,%r12+	shrdq	$4,%r13,%r13+	xorq	%r10,%r14+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	16(%rsp),%r9+	movq	%r10,%r15+	xorq	%r8,%r12+	shrdq	$6,%r14,%r14+	xorq	%r11,%r15+	addq	%r12,%r9+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%r10,%r14+	addq	%r13,%r9+	xorq	%r11,%rdi+	shrdq	$28,%r14,%r14+	addq	%r9,%rbx+	addq	%rdi,%r9+	movq	%rbx,%r13+	addq	%r9,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r9+	movq	%rcx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	shrdq	$4,%r13,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	addq	24(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r10,%rdi+	addq	%r12,%r8+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	shrdq	$28,%r14,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r8+	movq	%rbx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rax,%r13+	xorq	%rcx,%r12+	shrdq	$4,%r13,%r13+	xorq	%r8,%r14+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	32(%rsp),%rdx+	movq	%r8,%r15+	xorq	%rcx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r9,%r15+	addq	%r12,%rdx+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%r8,%r14+	addq	%r13,%rdx+	xorq	%r9,%rdi+	shrdq	$28,%r14,%r14+	addq	%rdx,%r11+	addq	%rdi,%rdx+	movq	%r11,%r13+	addq	%rdx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rdx+	movq	%rax,%r12+	shrdq	$5,%r14,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	shrdq	$4,%r13,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	addq	40(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r8,%rdi+	addq	%r12,%rcx+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	shrdq	$28,%r14,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rcx+	movq	%r11,%r12+	shrdq	$5,%r14,%r14+	xorq	%r10,%r13+	xorq	%rax,%r12+	shrdq	$4,%r13,%r13+	xorq	%rcx,%r14+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	48(%rsp),%rbx+	movq	%rcx,%r15+	xorq	%rax,%r12+	shrdq	$6,%r14,%r14+	xorq	%rdx,%r15+	addq	%r12,%rbx+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%rcx,%r14+	addq	%r13,%rbx+	xorq	%rdx,%rdi+	shrdq	$28,%r14,%r14+	addq	%rbx,%r9+	addq	%rdi,%rbx+	movq	%r9,%r13+	addq	%rbx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rbx+	movq	%r10,%r12+	shrdq	$5,%r14,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	shrdq	$4,%r13,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	addq	56(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	shrdq	$6,%r14,%r14+	xorq	%rcx,%rdi+	addq	%r12,%rax+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	shrdq	$28,%r14,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rax+	movq	%r9,%r12+	shrdq	$5,%r14,%r14+	xorq	%r8,%r13+	xorq	%r10,%r12+	shrdq	$4,%r13,%r13+	xorq	%rax,%r14+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	64(%rsp),%r11+	movq	%rax,%r15+	xorq	%r10,%r12+	shrdq	$6,%r14,%r14+	xorq	%rbx,%r15+	addq	%r12,%r11+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%rax,%r14+	addq	%r13,%r11+	xorq	%rbx,%rdi+	shrdq	$28,%r14,%r14+	addq	%r11,%rdx+	addq	%rdi,%r11+	movq	%rdx,%r13+	addq	%r11,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r11+	movq	%r8,%r12+	shrdq	$5,%r14,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	shrdq	$4,%r13,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	addq	72(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	shrdq	$6,%r14,%r14+	xorq	%rax,%rdi+	addq	%r12,%r10+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	shrdq	$28,%r14,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r10+	movq	%rdx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rcx,%r13+	xorq	%r8,%r12+	shrdq	$4,%r13,%r13+	xorq	%r10,%r14+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	80(%rsp),%r9+	movq	%r10,%r15+	xorq	%r8,%r12+	shrdq	$6,%r14,%r14+	xorq	%r11,%r15+	addq	%r12,%r9+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%r10,%r14+	addq	%r13,%r9+	xorq	%r11,%rdi+	shrdq	$28,%r14,%r14+	addq	%r9,%rbx+	addq	%rdi,%r9+	movq	%rbx,%r13+	addq	%r9,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r9+	movq	%rcx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	shrdq	$4,%r13,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	addq	88(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r10,%rdi+	addq	%r12,%r8+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	shrdq	$28,%r14,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r8+	movq	%rbx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rax,%r13+	xorq	%rcx,%r12+	shrdq	$4,%r13,%r13+	xorq	%r8,%r14+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	96(%rsp),%rdx+	movq	%r8,%r15+	xorq	%rcx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r9,%r15+	addq	%r12,%rdx+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%r8,%r14+	addq	%r13,%rdx+	xorq	%r9,%rdi+	shrdq	$28,%r14,%r14+	addq	%rdx,%r11+	addq	%rdi,%rdx+	movq	%r11,%r13+	addq	%rdx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rdx+	movq	%rax,%r12+	shrdq	$5,%r14,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	shrdq	$4,%r13,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	addq	104(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r8,%rdi+	addq	%r12,%rcx+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	shrdq	$28,%r14,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rcx+	movq	%r11,%r12+	shrdq	$5,%r14,%r14+	xorq	%r10,%r13+	xorq	%rax,%r12+	shrdq	$4,%r13,%r13+	xorq	%rcx,%r14+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	112(%rsp),%rbx+	movq	%rcx,%r15+	xorq	%rax,%r12+	shrdq	$6,%r14,%r14+	xorq	%rdx,%r15+	addq	%r12,%rbx+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%rcx,%r14+	addq	%r13,%rbx+	xorq	%rdx,%rdi+	shrdq	$28,%r14,%r14+	addq	%rbx,%r9+	addq	%rdi,%rbx+	movq	%r9,%r13+	addq	%rbx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rbx+	movq	%r10,%r12+	shrdq	$5,%r14,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	shrdq	$4,%r13,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	addq	120(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	shrdq	$6,%r14,%r14+	xorq	%rcx,%rdi+	addq	%r12,%rax+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	shrdq	$28,%r14,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	movq	-64(%rbp),%rdi+	movq	%r14,%rax+	movq	-56(%rbp),%rsi++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	addq	48(%rdi),%r10+	addq	56(%rdi),%r11++	leaq	128(%rsi),%rsi+	cmpq	-48(%rbp),%rsi++	movq	%rax,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)+	jb	L$loop_avx++	vzeroupper+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+++.p2align	6+crypton_sha512_asm_block_data_order_avx2:+.cfi_startproc+	.byte	0xf3,0x0f,0x1e,0xfa+++	pushq	%rbp+.cfi_adjust_cfa_offset	8+.cfi_offset	%rbp,-16+	movq	%rsp,%rbp+.cfi_def_cfa_register	%rbp+L$avx2_shortcut:+	pushq	%rbx+.cfi_offset	%rbx,-24+	pushq	%r12+.cfi_offset	%r12,-32+	pushq	%r13+.cfi_offset	%r13,-40+	pushq	%r14+.cfi_offset	%r14,-48+	pushq	%r15+.cfi_offset	%r15,-56+	shlq	$4,%rdx+	subq	$24,%rsp++	leaq	(%rsi,%rdx,8),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+++	leaq	-128(%rsp),%rsp+	vzeroupper+	andq	$-128,%rsp+	subq	$-128,%rsi+	movq	0(%rdi),%rax+	movq	%rsi,%r12+	movq	8(%rdi),%rbx+	cmpq	%rdx,%rsi+	movq	16(%rdi),%rcx+	cmoveq	%rsp,%r12+	movq	24(%rdi),%rdx+	movq	32(%rdi),%r8+	movq	40(%rdi),%r9+	movq	48(%rdi),%r10+	movq	56(%rdi),%r11+	jmp	L$oop_avx2+.p2align	4+L$oop_avx2:+	vmovdqa	K512+1280(%rip),%ymm10+	movq	%rsi,-56(%rbp)+	vmovdqu	-128(%rsi),%xmm0+	vmovdqu	-128+16(%rsi),%xmm1+	vmovdqu	-128+32(%rsi),%xmm2+	vmovdqu	-128+48(%rsi),%xmm3+	vmovdqu	-128+64(%rsi),%xmm4+	vmovdqu	-128+80(%rsi),%xmm5+	vmovdqu	-128+96(%rsi),%xmm6+	vmovdqu	-128+112(%rsi),%xmm7+	leaq	K512+128(%rip),%rsi+	vinserti128	$1,(%r12),%ymm0,%ymm0+	vinserti128	$1,16(%r12),%ymm1,%ymm1+	vpshufb	%ymm10,%ymm0,%ymm0+	vinserti128	$1,32(%r12),%ymm2,%ymm2+	vpshufb	%ymm10,%ymm1,%ymm1+	vinserti128	$1,48(%r12),%ymm3,%ymm3+	vpshufb	%ymm10,%ymm2,%ymm2+	vinserti128	$1,64(%r12),%ymm4,%ymm4+	vpshufb	%ymm10,%ymm3,%ymm3+	vinserti128	$1,80(%r12),%ymm5,%ymm5+	vpshufb	%ymm10,%ymm4,%ymm4+	vinserti128	$1,96(%r12),%ymm6,%ymm6+	vpshufb	%ymm10,%ymm5,%ymm5+	vinserti128	$1,112(%r12),%ymm7,%ymm7++	vpaddq	-128(%rsi),%ymm0,%ymm8+	vpshufb	%ymm10,%ymm6,%ymm6+	vpaddq	-96(%rsi),%ymm1,%ymm9+	vpshufb	%ymm10,%ymm7,%ymm7+	vpaddq	-64(%rsi),%ymm2,%ymm10+	vpaddq	-32(%rsi),%ymm3,%ymm11+	vmovdqa	%ymm8,0(%rsp)+	vpaddq	0(%rsi),%ymm4,%ymm8+	vmovdqa	%ymm9,32(%rsp)+	vpaddq	32(%rsi),%ymm5,%ymm9+	vmovdqa	%ymm10,64(%rsp)+	vpaddq	64(%rsi),%ymm6,%ymm10+	vmovdqa	%ymm11,96(%rsp)+	leaq	-128(%rsp),%rsp+	vpaddq	96(%rsi),%ymm7,%ymm11+	vmovdqa	%ymm8,0(%rsp)+	xorq	%r14,%r14+	vmovdqa	%ymm9,32(%rsp)+	movq	%rbx,%rdi+	vmovdqa	%ymm10,64(%rsp)+	xorq	%rcx,%rdi+	vmovdqa	%ymm11,96(%rsp)+	movq	%r9,%r12+	addq	$32*8,%rsi+	jmp	L$avx2_00_47++.p2align	4+L$avx2_00_47:+	leaq	-128(%rsp),%rsp+	vpalignr	$8,%ymm0,%ymm1,%ymm8+	addq	0+256(%rsp),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	vpalignr	$8,%ymm4,%ymm5,%ymm11+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	vpaddq	%ymm11,%ymm0,%ymm0+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	vpsrlq	$6,%ymm7,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	vpsllq	$3,%ymm7,%ymm10+	vpaddq	%ymm8,%ymm0,%ymm0+	addq	8+256(%rsp),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	vpsrlq	$19,%ymm7,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	vpaddq	%ymm11,%ymm0,%ymm0+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	vpaddq	-128(%rsi),%ymm0,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	vmovdqa	%ymm10,0(%rsp)+	vpalignr	$8,%ymm1,%ymm2,%ymm8+	addq	32+256(%rsp),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	vpalignr	$8,%ymm5,%ymm6,%ymm11+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	vpaddq	%ymm11,%ymm1,%ymm1+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	vpsrlq	$6,%ymm0,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	vpsllq	$3,%ymm0,%ymm10+	vpaddq	%ymm8,%ymm1,%ymm1+	addq	40+256(%rsp),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	vpsrlq	$19,%ymm0,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	vpaddq	%ymm11,%ymm1,%ymm1+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	vpaddq	-96(%rsi),%ymm1,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	vmovdqa	%ymm10,32(%rsp)+	vpalignr	$8,%ymm2,%ymm3,%ymm8+	addq	64+256(%rsp),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	vpalignr	$8,%ymm6,%ymm7,%ymm11+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	vpaddq	%ymm11,%ymm2,%ymm2+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	vpsrlq	$6,%ymm1,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	vpsllq	$3,%ymm1,%ymm10+	vpaddq	%ymm8,%ymm2,%ymm2+	addq	72+256(%rsp),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	vpsrlq	$19,%ymm1,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	vpaddq	%ymm11,%ymm2,%ymm2+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	vpaddq	-64(%rsi),%ymm2,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	vmovdqa	%ymm10,64(%rsp)+	vpalignr	$8,%ymm3,%ymm4,%ymm8+	addq	96+256(%rsp),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	vpalignr	$8,%ymm7,%ymm0,%ymm11+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	vpaddq	%ymm11,%ymm3,%ymm3+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	vpsrlq	$6,%ymm2,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	vpsllq	$3,%ymm2,%ymm10+	vpaddq	%ymm8,%ymm3,%ymm3+	addq	104+256(%rsp),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	vpsrlq	$19,%ymm2,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	vpaddq	%ymm11,%ymm3,%ymm3+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	vpaddq	-32(%rsi),%ymm3,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	vmovdqa	%ymm10,96(%rsp)+	leaq	-128(%rsp),%rsp+	vpalignr	$8,%ymm4,%ymm5,%ymm8+	addq	0+256(%rsp),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	vpalignr	$8,%ymm0,%ymm1,%ymm11+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	vpaddq	%ymm11,%ymm4,%ymm4+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	vpsrlq	$6,%ymm3,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	vpsllq	$3,%ymm3,%ymm10+	vpaddq	%ymm8,%ymm4,%ymm4+	addq	8+256(%rsp),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	vpsrlq	$19,%ymm3,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	vpaddq	%ymm11,%ymm4,%ymm4+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	vpaddq	0(%rsi),%ymm4,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	vmovdqa	%ymm10,0(%rsp)+	vpalignr	$8,%ymm5,%ymm6,%ymm8+	addq	32+256(%rsp),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	vpalignr	$8,%ymm1,%ymm2,%ymm11+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	vpaddq	%ymm11,%ymm5,%ymm5+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	vpsrlq	$6,%ymm4,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	vpsllq	$3,%ymm4,%ymm10+	vpaddq	%ymm8,%ymm5,%ymm5+	addq	40+256(%rsp),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	vpsrlq	$19,%ymm4,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	vpaddq	%ymm11,%ymm5,%ymm5+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	vpaddq	32(%rsi),%ymm5,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	vmovdqa	%ymm10,32(%rsp)+	vpalignr	$8,%ymm6,%ymm7,%ymm8+	addq	64+256(%rsp),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	vpalignr	$8,%ymm2,%ymm3,%ymm11+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	vpaddq	%ymm11,%ymm6,%ymm6+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	vpsrlq	$6,%ymm5,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	vpsllq	$3,%ymm5,%ymm10+	vpaddq	%ymm8,%ymm6,%ymm6+	addq	72+256(%rsp),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	vpsrlq	$19,%ymm5,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	vpaddq	%ymm11,%ymm6,%ymm6+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	vpaddq	64(%rsi),%ymm6,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	vmovdqa	%ymm10,64(%rsp)+	vpalignr	$8,%ymm7,%ymm0,%ymm8+	addq	96+256(%rsp),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	vpalignr	$8,%ymm3,%ymm4,%ymm11+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	vpaddq	%ymm11,%ymm7,%ymm7+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	vpsrlq	$6,%ymm6,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	vpsllq	$3,%ymm6,%ymm10+	vpaddq	%ymm8,%ymm7,%ymm7+	addq	104+256(%rsp),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	vpsrlq	$19,%ymm6,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	vpaddq	%ymm11,%ymm7,%ymm7+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	vpaddq	96(%rsi),%ymm7,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	vmovdqa	%ymm10,96(%rsp)+	leaq	256(%rsi),%rsi+	cmpb	$0,-121(%rsi)+	jne	L$avx2_00_47+	addq	0+128(%rsp),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	addq	8+128(%rsp),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	addq	32+128(%rsp),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	addq	40+128(%rsp),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	addq	64+128(%rsp),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	addq	72+128(%rsp),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	addq	96+128(%rsp),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	addq	104+128(%rsp),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	addq	0(%rsp),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	addq	8(%rsp),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	addq	32(%rsp),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	addq	40(%rsp),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	addq	64(%rsp),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	addq	72(%rsp),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	addq	96(%rsp),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	addq	104(%rsp),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	movq	-64(%rbp),%rdi+	addq	%r14,%rax+	movq	-56(%rbp),%r12++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	addq	48(%rdi),%r10+	addq	56(%rdi),%r11++	movq	%rax,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)++	cmpq	-48(%rbp),%r12+	je	L$done_avx2++	leaq	1152(%rsp),%rsi+	xorq	%r14,%r14+	movq	%rbx,%rdi+	xorq	%rcx,%rdi+	movq	%r9,%r12+	jmp	L$ower_avx2+.p2align	4+L$ower_avx2:+	addq	0+16(%rsi),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	addq	8+16(%rsi),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	addq	32+16(%rsi),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	addq	40+16(%rsi),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	addq	64+16(%rsi),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	addq	72+16(%rsi),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	addq	96+16(%rsi),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	addq	104+16(%rsi),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	leaq	-128(%rsi),%rsi+	cmpq	%rsp,%rsi+	jae	L$ower_avx2++	movq	-64(%rbp),%rdi+	addq	%r14,%rax+	movq	-56(%rbp),%rsi+	leaq	1152(%rsp),%rsp++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	leaq	256(%rsi),%rsi+	addq	48(%rdi),%r10+	movq	%rsi,%r12+	addq	56(%rdi),%r11+	cmpq	-48(%rbp),%rsi++	movq	%rax,0(%rdi)+	cmoveq	%rsp,%r12+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)++	jbe	L$oop_avx2++L$done_avx2:+	vzeroupper+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	popq	%rbp+.cfi_adjust_cfa_offset	-8+.cfi_restore	%rbp+.cfi_restore	%r12+.cfi_restore	%r13+.cfi_restore	%r14+.cfi_restore	%r15+.cfi_restore	%rbx+	.byte	0xf3,0xc3+.cfi_endproc	+
+ cbits/asm/sha512-x86_64-mingw64.S view
@@ -0,0 +1,6016 @@+.text	+++.globl	crypton_sha512_asm_block_data_order+.def	crypton_sha512_asm_block_data_order;	.scl 2;	.type 32;	.endef+.p2align	4+crypton_sha512_asm_block_data_order:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_sha512_asm_block_data_order:+++	pushq	%rbp++	movq	%rsp,%rbp++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+	leaq	crypton_ia32cap_P(%rip),%rax+	movl	0(%rax),%r9d+	movl	4(%rax),%r10d+	movl	8(%rax),%eax+	testl	$2048,%r10d+	jnz	.Lxop_shortcut+	andl	$296,%eax+	cmpl	$296,%eax+	je	.Lavx2_shortcut+	andl	$1073741824,%r9d+	andl	$268435968,%r10d+	orl	%r9d,%r10d+	cmpl	$1342177792,%r10d+	je	.Lavx_shortcut+	pushq	%rbx++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	shlq	$4,%rdx+	subq	$128+24,%rsp+++.LSEH_body_crypton_sha512_asm_block_data_order:++	leaq	(%rsi,%rdx,8),%rdx+	movq	%rdi,128+0(%rsp)+	movq	%rsi,128+8(%rsp)+	movq	%rdx,128+16(%rsp)++	movq	0(%rdi),%rax+	movq	8(%rdi),%rbx+	movq	16(%rdi),%rcx+	movq	24(%rdi),%rdx+	movq	32(%rdi),%r8+	movq	40(%rdi),%r9+	movq	48(%rdi),%r10+	movq	56(%rdi),%r11+	jmp	.Lloop++.p2align	4+.Lloop:+	movq	%rbx,%rdi+	leaq	K512(%rip),%rbp+	xorq	%rcx,%rdi+	movq	0(%rsi),%r12+	movq	%r8,%r13+	movq	%rax,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r9,%r15++	xorq	%r8,%r13+	rorq	$5,%r14+	xorq	%r10,%r15++	movq	%r12,0(%rsp)+	xorq	%rax,%r14+	andq	%r8,%r15++	rorq	$4,%r13+	addq	%r11,%r12+	xorq	%r10,%r15++	rorq	$6,%r14+	xorq	%r8,%r13+	addq	%r15,%r12++	movq	%rax,%r15+	addq	(%rbp),%r12+	xorq	%rax,%r14++	xorq	%rbx,%r15+	rorq	$14,%r13+	movq	%rbx,%r11++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r11+	addq	%r12,%rdx+	addq	%r12,%r11++	leaq	8(%rbp),%rbp+	addq	%r14,%r11+	movq	8(%rsi),%r12+	movq	%rdx,%r13+	movq	%r11,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r8,%rdi++	xorq	%rdx,%r13+	rorq	$5,%r14+	xorq	%r9,%rdi++	movq	%r12,8(%rsp)+	xorq	%r11,%r14+	andq	%rdx,%rdi++	rorq	$4,%r13+	addq	%r10,%r12+	xorq	%r9,%rdi++	rorq	$6,%r14+	xorq	%rdx,%r13+	addq	%rdi,%r12++	movq	%r11,%rdi+	addq	(%rbp),%r12+	xorq	%r11,%r14++	xorq	%rax,%rdi+	rorq	$14,%r13+	movq	%rax,%r10++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r10+	addq	%r12,%rcx+	addq	%r12,%r10++	leaq	24(%rbp),%rbp+	addq	%r14,%r10+	movq	16(%rsi),%r12+	movq	%rcx,%r13+	movq	%r10,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rdx,%r15++	xorq	%rcx,%r13+	rorq	$5,%r14+	xorq	%r8,%r15++	movq	%r12,16(%rsp)+	xorq	%r10,%r14+	andq	%rcx,%r15++	rorq	$4,%r13+	addq	%r9,%r12+	xorq	%r8,%r15++	rorq	$6,%r14+	xorq	%rcx,%r13+	addq	%r15,%r12++	movq	%r10,%r15+	addq	(%rbp),%r12+	xorq	%r10,%r14++	xorq	%r11,%r15+	rorq	$14,%r13+	movq	%r11,%r9++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r9+	addq	%r12,%rbx+	addq	%r12,%r9++	leaq	8(%rbp),%rbp+	addq	%r14,%r9+	movq	24(%rsi),%r12+	movq	%rbx,%r13+	movq	%r9,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rcx,%rdi++	xorq	%rbx,%r13+	rorq	$5,%r14+	xorq	%rdx,%rdi++	movq	%r12,24(%rsp)+	xorq	%r9,%r14+	andq	%rbx,%rdi++	rorq	$4,%r13+	addq	%r8,%r12+	xorq	%rdx,%rdi++	rorq	$6,%r14+	xorq	%rbx,%r13+	addq	%rdi,%r12++	movq	%r9,%rdi+	addq	(%rbp),%r12+	xorq	%r9,%r14++	xorq	%r10,%rdi+	rorq	$14,%r13+	movq	%r10,%r8++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r8+	addq	%r12,%rax+	addq	%r12,%r8++	leaq	24(%rbp),%rbp+	addq	%r14,%r8+	movq	32(%rsi),%r12+	movq	%rax,%r13+	movq	%r8,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rbx,%r15++	xorq	%rax,%r13+	rorq	$5,%r14+	xorq	%rcx,%r15++	movq	%r12,32(%rsp)+	xorq	%r8,%r14+	andq	%rax,%r15++	rorq	$4,%r13+	addq	%rdx,%r12+	xorq	%rcx,%r15++	rorq	$6,%r14+	xorq	%rax,%r13+	addq	%r15,%r12++	movq	%r8,%r15+	addq	(%rbp),%r12+	xorq	%r8,%r14++	xorq	%r9,%r15+	rorq	$14,%r13+	movq	%r9,%rdx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rdx+	addq	%r12,%r11+	addq	%r12,%rdx++	leaq	8(%rbp),%rbp+	addq	%r14,%rdx+	movq	40(%rsi),%r12+	movq	%r11,%r13+	movq	%rdx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rax,%rdi++	xorq	%r11,%r13+	rorq	$5,%r14+	xorq	%rbx,%rdi++	movq	%r12,40(%rsp)+	xorq	%rdx,%r14+	andq	%r11,%rdi++	rorq	$4,%r13+	addq	%rcx,%r12+	xorq	%rbx,%rdi++	rorq	$6,%r14+	xorq	%r11,%r13+	addq	%rdi,%r12++	movq	%rdx,%rdi+	addq	(%rbp),%r12+	xorq	%rdx,%r14++	xorq	%r8,%rdi+	rorq	$14,%r13+	movq	%r8,%rcx++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rcx+	addq	%r12,%r10+	addq	%r12,%rcx++	leaq	24(%rbp),%rbp+	addq	%r14,%rcx+	movq	48(%rsi),%r12+	movq	%r10,%r13+	movq	%rcx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r11,%r15++	xorq	%r10,%r13+	rorq	$5,%r14+	xorq	%rax,%r15++	movq	%r12,48(%rsp)+	xorq	%rcx,%r14+	andq	%r10,%r15++	rorq	$4,%r13+	addq	%rbx,%r12+	xorq	%rax,%r15++	rorq	$6,%r14+	xorq	%r10,%r13+	addq	%r15,%r12++	movq	%rcx,%r15+	addq	(%rbp),%r12+	xorq	%rcx,%r14++	xorq	%rdx,%r15+	rorq	$14,%r13+	movq	%rdx,%rbx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rbx+	addq	%r12,%r9+	addq	%r12,%rbx++	leaq	8(%rbp),%rbp+	addq	%r14,%rbx+	movq	56(%rsi),%r12+	movq	%r9,%r13+	movq	%rbx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r10,%rdi++	xorq	%r9,%r13+	rorq	$5,%r14+	xorq	%r11,%rdi++	movq	%r12,56(%rsp)+	xorq	%rbx,%r14+	andq	%r9,%rdi++	rorq	$4,%r13+	addq	%rax,%r12+	xorq	%r11,%rdi++	rorq	$6,%r14+	xorq	%r9,%r13+	addq	%rdi,%r12++	movq	%rbx,%rdi+	addq	(%rbp),%r12+	xorq	%rbx,%r14++	xorq	%rcx,%rdi+	rorq	$14,%r13+	movq	%rcx,%rax++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rax+	addq	%r12,%r8+	addq	%r12,%rax++	leaq	24(%rbp),%rbp+	addq	%r14,%rax+	movq	64(%rsi),%r12+	movq	%r8,%r13+	movq	%rax,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r9,%r15++	xorq	%r8,%r13+	rorq	$5,%r14+	xorq	%r10,%r15++	movq	%r12,64(%rsp)+	xorq	%rax,%r14+	andq	%r8,%r15++	rorq	$4,%r13+	addq	%r11,%r12+	xorq	%r10,%r15++	rorq	$6,%r14+	xorq	%r8,%r13+	addq	%r15,%r12++	movq	%rax,%r15+	addq	(%rbp),%r12+	xorq	%rax,%r14++	xorq	%rbx,%r15+	rorq	$14,%r13+	movq	%rbx,%r11++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r11+	addq	%r12,%rdx+	addq	%r12,%r11++	leaq	8(%rbp),%rbp+	addq	%r14,%r11+	movq	72(%rsi),%r12+	movq	%rdx,%r13+	movq	%r11,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r8,%rdi++	xorq	%rdx,%r13+	rorq	$5,%r14+	xorq	%r9,%rdi++	movq	%r12,72(%rsp)+	xorq	%r11,%r14+	andq	%rdx,%rdi++	rorq	$4,%r13+	addq	%r10,%r12+	xorq	%r9,%rdi++	rorq	$6,%r14+	xorq	%rdx,%r13+	addq	%rdi,%r12++	movq	%r11,%rdi+	addq	(%rbp),%r12+	xorq	%r11,%r14++	xorq	%rax,%rdi+	rorq	$14,%r13+	movq	%rax,%r10++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r10+	addq	%r12,%rcx+	addq	%r12,%r10++	leaq	24(%rbp),%rbp+	addq	%r14,%r10+	movq	80(%rsi),%r12+	movq	%rcx,%r13+	movq	%r10,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rdx,%r15++	xorq	%rcx,%r13+	rorq	$5,%r14+	xorq	%r8,%r15++	movq	%r12,80(%rsp)+	xorq	%r10,%r14+	andq	%rcx,%r15++	rorq	$4,%r13+	addq	%r9,%r12+	xorq	%r8,%r15++	rorq	$6,%r14+	xorq	%rcx,%r13+	addq	%r15,%r12++	movq	%r10,%r15+	addq	(%rbp),%r12+	xorq	%r10,%r14++	xorq	%r11,%r15+	rorq	$14,%r13+	movq	%r11,%r9++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r9+	addq	%r12,%rbx+	addq	%r12,%r9++	leaq	8(%rbp),%rbp+	addq	%r14,%r9+	movq	88(%rsi),%r12+	movq	%rbx,%r13+	movq	%r9,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rcx,%rdi++	xorq	%rbx,%r13+	rorq	$5,%r14+	xorq	%rdx,%rdi++	movq	%r12,88(%rsp)+	xorq	%r9,%r14+	andq	%rbx,%rdi++	rorq	$4,%r13+	addq	%r8,%r12+	xorq	%rdx,%rdi++	rorq	$6,%r14+	xorq	%rbx,%r13+	addq	%rdi,%r12++	movq	%r9,%rdi+	addq	(%rbp),%r12+	xorq	%r9,%r14++	xorq	%r10,%rdi+	rorq	$14,%r13+	movq	%r10,%r8++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r8+	addq	%r12,%rax+	addq	%r12,%r8++	leaq	24(%rbp),%rbp+	addq	%r14,%r8+	movq	96(%rsi),%r12+	movq	%rax,%r13+	movq	%r8,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rbx,%r15++	xorq	%rax,%r13+	rorq	$5,%r14+	xorq	%rcx,%r15++	movq	%r12,96(%rsp)+	xorq	%r8,%r14+	andq	%rax,%r15++	rorq	$4,%r13+	addq	%rdx,%r12+	xorq	%rcx,%r15++	rorq	$6,%r14+	xorq	%rax,%r13+	addq	%r15,%r12++	movq	%r8,%r15+	addq	(%rbp),%r12+	xorq	%r8,%r14++	xorq	%r9,%r15+	rorq	$14,%r13+	movq	%r9,%rdx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rdx+	addq	%r12,%r11+	addq	%r12,%rdx++	leaq	8(%rbp),%rbp+	addq	%r14,%rdx+	movq	104(%rsi),%r12+	movq	%r11,%r13+	movq	%rdx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%rax,%rdi++	xorq	%r11,%r13+	rorq	$5,%r14+	xorq	%rbx,%rdi++	movq	%r12,104(%rsp)+	xorq	%rdx,%r14+	andq	%r11,%rdi++	rorq	$4,%r13+	addq	%rcx,%r12+	xorq	%rbx,%rdi++	rorq	$6,%r14+	xorq	%r11,%r13+	addq	%rdi,%r12++	movq	%rdx,%rdi+	addq	(%rbp),%r12+	xorq	%rdx,%r14++	xorq	%r8,%rdi+	rorq	$14,%r13+	movq	%r8,%rcx++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rcx+	addq	%r12,%r10+	addq	%r12,%rcx++	leaq	24(%rbp),%rbp+	addq	%r14,%rcx+	movq	112(%rsi),%r12+	movq	%r10,%r13+	movq	%rcx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r11,%r15++	xorq	%r10,%r13+	rorq	$5,%r14+	xorq	%rax,%r15++	movq	%r12,112(%rsp)+	xorq	%rcx,%r14+	andq	%r10,%r15++	rorq	$4,%r13+	addq	%rbx,%r12+	xorq	%rax,%r15++	rorq	$6,%r14+	xorq	%r10,%r13+	addq	%r15,%r12++	movq	%rcx,%r15+	addq	(%rbp),%r12+	xorq	%rcx,%r14++	xorq	%rdx,%r15+	rorq	$14,%r13+	movq	%rdx,%rbx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rbx+	addq	%r12,%r9+	addq	%r12,%rbx++	leaq	8(%rbp),%rbp+	addq	%r14,%rbx+	movq	120(%rsi),%r12+	movq	%r9,%r13+	movq	%rbx,%r14+	bswapq	%r12+	rorq	$23,%r13+	movq	%r10,%rdi++	xorq	%r9,%r13+	rorq	$5,%r14+	xorq	%r11,%rdi++	movq	%r12,120(%rsp)+	xorq	%rbx,%r14+	andq	%r9,%rdi++	rorq	$4,%r13+	addq	%rax,%r12+	xorq	%r11,%rdi++	rorq	$6,%r14+	xorq	%r9,%r13+	addq	%rdi,%r12++	movq	%rbx,%rdi+	addq	(%rbp),%r12+	xorq	%rbx,%r14++	xorq	%rcx,%rdi+	rorq	$14,%r13+	movq	%rcx,%rax++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rax+	addq	%r12,%r8+	addq	%r12,%rax++	leaq	24(%rbp),%rbp+	jmp	.Lrounds_16_xx+.p2align	4+.Lrounds_16_xx:+	movq	8(%rsp),%r13+	movq	112(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rax+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	72(%rsp),%r12++	addq	0(%rsp),%r12+	movq	%r8,%r13+	addq	%r15,%r12+	movq	%rax,%r14+	rorq	$23,%r13+	movq	%r9,%r15++	xorq	%r8,%r13+	rorq	$5,%r14+	xorq	%r10,%r15++	movq	%r12,0(%rsp)+	xorq	%rax,%r14+	andq	%r8,%r15++	rorq	$4,%r13+	addq	%r11,%r12+	xorq	%r10,%r15++	rorq	$6,%r14+	xorq	%r8,%r13+	addq	%r15,%r12++	movq	%rax,%r15+	addq	(%rbp),%r12+	xorq	%rax,%r14++	xorq	%rbx,%r15+	rorq	$14,%r13+	movq	%rbx,%r11++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r11+	addq	%r12,%rdx+	addq	%r12,%r11++	leaq	8(%rbp),%rbp+	movq	16(%rsp),%r13+	movq	120(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r11+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	80(%rsp),%r12++	addq	8(%rsp),%r12+	movq	%rdx,%r13+	addq	%rdi,%r12+	movq	%r11,%r14+	rorq	$23,%r13+	movq	%r8,%rdi++	xorq	%rdx,%r13+	rorq	$5,%r14+	xorq	%r9,%rdi++	movq	%r12,8(%rsp)+	xorq	%r11,%r14+	andq	%rdx,%rdi++	rorq	$4,%r13+	addq	%r10,%r12+	xorq	%r9,%rdi++	rorq	$6,%r14+	xorq	%rdx,%r13+	addq	%rdi,%r12++	movq	%r11,%rdi+	addq	(%rbp),%r12+	xorq	%r11,%r14++	xorq	%rax,%rdi+	rorq	$14,%r13+	movq	%rax,%r10++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r10+	addq	%r12,%rcx+	addq	%r12,%r10++	leaq	24(%rbp),%rbp+	movq	24(%rsp),%r13+	movq	0(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r10+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	88(%rsp),%r12++	addq	16(%rsp),%r12+	movq	%rcx,%r13+	addq	%r15,%r12+	movq	%r10,%r14+	rorq	$23,%r13+	movq	%rdx,%r15++	xorq	%rcx,%r13+	rorq	$5,%r14+	xorq	%r8,%r15++	movq	%r12,16(%rsp)+	xorq	%r10,%r14+	andq	%rcx,%r15++	rorq	$4,%r13+	addq	%r9,%r12+	xorq	%r8,%r15++	rorq	$6,%r14+	xorq	%rcx,%r13+	addq	%r15,%r12++	movq	%r10,%r15+	addq	(%rbp),%r12+	xorq	%r10,%r14++	xorq	%r11,%r15+	rorq	$14,%r13+	movq	%r11,%r9++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r9+	addq	%r12,%rbx+	addq	%r12,%r9++	leaq	8(%rbp),%rbp+	movq	32(%rsp),%r13+	movq	8(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r9+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	96(%rsp),%r12++	addq	24(%rsp),%r12+	movq	%rbx,%r13+	addq	%rdi,%r12+	movq	%r9,%r14+	rorq	$23,%r13+	movq	%rcx,%rdi++	xorq	%rbx,%r13+	rorq	$5,%r14+	xorq	%rdx,%rdi++	movq	%r12,24(%rsp)+	xorq	%r9,%r14+	andq	%rbx,%rdi++	rorq	$4,%r13+	addq	%r8,%r12+	xorq	%rdx,%rdi++	rorq	$6,%r14+	xorq	%rbx,%r13+	addq	%rdi,%r12++	movq	%r9,%rdi+	addq	(%rbp),%r12+	xorq	%r9,%r14++	xorq	%r10,%rdi+	rorq	$14,%r13+	movq	%r10,%r8++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r8+	addq	%r12,%rax+	addq	%r12,%r8++	leaq	24(%rbp),%rbp+	movq	40(%rsp),%r13+	movq	16(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r8+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	104(%rsp),%r12++	addq	32(%rsp),%r12+	movq	%rax,%r13+	addq	%r15,%r12+	movq	%r8,%r14+	rorq	$23,%r13+	movq	%rbx,%r15++	xorq	%rax,%r13+	rorq	$5,%r14+	xorq	%rcx,%r15++	movq	%r12,32(%rsp)+	xorq	%r8,%r14+	andq	%rax,%r15++	rorq	$4,%r13+	addq	%rdx,%r12+	xorq	%rcx,%r15++	rorq	$6,%r14+	xorq	%rax,%r13+	addq	%r15,%r12++	movq	%r8,%r15+	addq	(%rbp),%r12+	xorq	%r8,%r14++	xorq	%r9,%r15+	rorq	$14,%r13+	movq	%r9,%rdx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rdx+	addq	%r12,%r11+	addq	%r12,%rdx++	leaq	8(%rbp),%rbp+	movq	48(%rsp),%r13+	movq	24(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rdx+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	112(%rsp),%r12++	addq	40(%rsp),%r12+	movq	%r11,%r13+	addq	%rdi,%r12+	movq	%rdx,%r14+	rorq	$23,%r13+	movq	%rax,%rdi++	xorq	%r11,%r13+	rorq	$5,%r14+	xorq	%rbx,%rdi++	movq	%r12,40(%rsp)+	xorq	%rdx,%r14+	andq	%r11,%rdi++	rorq	$4,%r13+	addq	%rcx,%r12+	xorq	%rbx,%rdi++	rorq	$6,%r14+	xorq	%r11,%r13+	addq	%rdi,%r12++	movq	%rdx,%rdi+	addq	(%rbp),%r12+	xorq	%rdx,%r14++	xorq	%r8,%rdi+	rorq	$14,%r13+	movq	%r8,%rcx++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rcx+	addq	%r12,%r10+	addq	%r12,%rcx++	leaq	24(%rbp),%rbp+	movq	56(%rsp),%r13+	movq	32(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rcx+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	120(%rsp),%r12++	addq	48(%rsp),%r12+	movq	%r10,%r13+	addq	%r15,%r12+	movq	%rcx,%r14+	rorq	$23,%r13+	movq	%r11,%r15++	xorq	%r10,%r13+	rorq	$5,%r14+	xorq	%rax,%r15++	movq	%r12,48(%rsp)+	xorq	%rcx,%r14+	andq	%r10,%r15++	rorq	$4,%r13+	addq	%rbx,%r12+	xorq	%rax,%r15++	rorq	$6,%r14+	xorq	%r10,%r13+	addq	%r15,%r12++	movq	%rcx,%r15+	addq	(%rbp),%r12+	xorq	%rcx,%r14++	xorq	%rdx,%r15+	rorq	$14,%r13+	movq	%rdx,%rbx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rbx+	addq	%r12,%r9+	addq	%r12,%rbx++	leaq	8(%rbp),%rbp+	movq	64(%rsp),%r13+	movq	40(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rbx+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	0(%rsp),%r12++	addq	56(%rsp),%r12+	movq	%r9,%r13+	addq	%rdi,%r12+	movq	%rbx,%r14+	rorq	$23,%r13+	movq	%r10,%rdi++	xorq	%r9,%r13+	rorq	$5,%r14+	xorq	%r11,%rdi++	movq	%r12,56(%rsp)+	xorq	%rbx,%r14+	andq	%r9,%rdi++	rorq	$4,%r13+	addq	%rax,%r12+	xorq	%r11,%rdi++	rorq	$6,%r14+	xorq	%r9,%r13+	addq	%rdi,%r12++	movq	%rbx,%rdi+	addq	(%rbp),%r12+	xorq	%rbx,%r14++	xorq	%rcx,%rdi+	rorq	$14,%r13+	movq	%rcx,%rax++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rax+	addq	%r12,%r8+	addq	%r12,%rax++	leaq	24(%rbp),%rbp+	movq	72(%rsp),%r13+	movq	48(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rax+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	8(%rsp),%r12++	addq	64(%rsp),%r12+	movq	%r8,%r13+	addq	%r15,%r12+	movq	%rax,%r14+	rorq	$23,%r13+	movq	%r9,%r15++	xorq	%r8,%r13+	rorq	$5,%r14+	xorq	%r10,%r15++	movq	%r12,64(%rsp)+	xorq	%rax,%r14+	andq	%r8,%r15++	rorq	$4,%r13+	addq	%r11,%r12+	xorq	%r10,%r15++	rorq	$6,%r14+	xorq	%r8,%r13+	addq	%r15,%r12++	movq	%rax,%r15+	addq	(%rbp),%r12+	xorq	%rax,%r14++	xorq	%rbx,%r15+	rorq	$14,%r13+	movq	%rbx,%r11++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r11+	addq	%r12,%rdx+	addq	%r12,%r11++	leaq	8(%rbp),%rbp+	movq	80(%rsp),%r13+	movq	56(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r11+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	16(%rsp),%r12++	addq	72(%rsp),%r12+	movq	%rdx,%r13+	addq	%rdi,%r12+	movq	%r11,%r14+	rorq	$23,%r13+	movq	%r8,%rdi++	xorq	%rdx,%r13+	rorq	$5,%r14+	xorq	%r9,%rdi++	movq	%r12,72(%rsp)+	xorq	%r11,%r14+	andq	%rdx,%rdi++	rorq	$4,%r13+	addq	%r10,%r12+	xorq	%r9,%rdi++	rorq	$6,%r14+	xorq	%rdx,%r13+	addq	%rdi,%r12++	movq	%r11,%rdi+	addq	(%rbp),%r12+	xorq	%r11,%r14++	xorq	%rax,%rdi+	rorq	$14,%r13+	movq	%rax,%r10++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r10+	addq	%r12,%rcx+	addq	%r12,%r10++	leaq	24(%rbp),%rbp+	movq	88(%rsp),%r13+	movq	64(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r10+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	24(%rsp),%r12++	addq	80(%rsp),%r12+	movq	%rcx,%r13+	addq	%r15,%r12+	movq	%r10,%r14+	rorq	$23,%r13+	movq	%rdx,%r15++	xorq	%rcx,%r13+	rorq	$5,%r14+	xorq	%r8,%r15++	movq	%r12,80(%rsp)+	xorq	%r10,%r14+	andq	%rcx,%r15++	rorq	$4,%r13+	addq	%r9,%r12+	xorq	%r8,%r15++	rorq	$6,%r14+	xorq	%rcx,%r13+	addq	%r15,%r12++	movq	%r10,%r15+	addq	(%rbp),%r12+	xorq	%r10,%r14++	xorq	%r11,%r15+	rorq	$14,%r13+	movq	%r11,%r9++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%r9+	addq	%r12,%rbx+	addq	%r12,%r9++	leaq	8(%rbp),%rbp+	movq	96(%rsp),%r13+	movq	72(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r9+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	32(%rsp),%r12++	addq	88(%rsp),%r12+	movq	%rbx,%r13+	addq	%rdi,%r12+	movq	%r9,%r14+	rorq	$23,%r13+	movq	%rcx,%rdi++	xorq	%rbx,%r13+	rorq	$5,%r14+	xorq	%rdx,%rdi++	movq	%r12,88(%rsp)+	xorq	%r9,%r14+	andq	%rbx,%rdi++	rorq	$4,%r13+	addq	%r8,%r12+	xorq	%rdx,%rdi++	rorq	$6,%r14+	xorq	%rbx,%r13+	addq	%rdi,%r12++	movq	%r9,%rdi+	addq	(%rbp),%r12+	xorq	%r9,%r14++	xorq	%r10,%rdi+	rorq	$14,%r13+	movq	%r10,%r8++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%r8+	addq	%r12,%rax+	addq	%r12,%r8++	leaq	24(%rbp),%rbp+	movq	104(%rsp),%r13+	movq	80(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%r8+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	40(%rsp),%r12++	addq	96(%rsp),%r12+	movq	%rax,%r13+	addq	%r15,%r12+	movq	%r8,%r14+	rorq	$23,%r13+	movq	%rbx,%r15++	xorq	%rax,%r13+	rorq	$5,%r14+	xorq	%rcx,%r15++	movq	%r12,96(%rsp)+	xorq	%r8,%r14+	andq	%rax,%r15++	rorq	$4,%r13+	addq	%rdx,%r12+	xorq	%rcx,%r15++	rorq	$6,%r14+	xorq	%rax,%r13+	addq	%r15,%r12++	movq	%r8,%r15+	addq	(%rbp),%r12+	xorq	%r8,%r14++	xorq	%r9,%r15+	rorq	$14,%r13+	movq	%r9,%rdx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rdx+	addq	%r12,%r11+	addq	%r12,%rdx++	leaq	8(%rbp),%rbp+	movq	112(%rsp),%r13+	movq	88(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rdx+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	48(%rsp),%r12++	addq	104(%rsp),%r12+	movq	%r11,%r13+	addq	%rdi,%r12+	movq	%rdx,%r14+	rorq	$23,%r13+	movq	%rax,%rdi++	xorq	%r11,%r13+	rorq	$5,%r14+	xorq	%rbx,%rdi++	movq	%r12,104(%rsp)+	xorq	%rdx,%r14+	andq	%r11,%rdi++	rorq	$4,%r13+	addq	%rcx,%r12+	xorq	%rbx,%rdi++	rorq	$6,%r14+	xorq	%r11,%r13+	addq	%rdi,%r12++	movq	%rdx,%rdi+	addq	(%rbp),%r12+	xorq	%rdx,%r14++	xorq	%r8,%rdi+	rorq	$14,%r13+	movq	%r8,%rcx++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rcx+	addq	%r12,%r10+	addq	%r12,%rcx++	leaq	24(%rbp),%rbp+	movq	120(%rsp),%r13+	movq	96(%rsp),%r15++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rcx+	movq	%r15,%r14+	rorq	$42,%r15++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%r15+	shrq	$6,%r14++	rorq	$19,%r15+	xorq	%r13,%r12+	xorq	%r14,%r15+	addq	56(%rsp),%r12++	addq	112(%rsp),%r12+	movq	%r10,%r13+	addq	%r15,%r12+	movq	%rcx,%r14+	rorq	$23,%r13+	movq	%r11,%r15++	xorq	%r10,%r13+	rorq	$5,%r14+	xorq	%rax,%r15++	movq	%r12,112(%rsp)+	xorq	%rcx,%r14+	andq	%r10,%r15++	rorq	$4,%r13+	addq	%rbx,%r12+	xorq	%rax,%r15++	rorq	$6,%r14+	xorq	%r10,%r13+	addq	%r15,%r12++	movq	%rcx,%r15+	addq	(%rbp),%r12+	xorq	%rcx,%r14++	xorq	%rdx,%r15+	rorq	$14,%r13+	movq	%rdx,%rbx++	andq	%r15,%rdi+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%rdi,%rbx+	addq	%r12,%r9+	addq	%r12,%rbx++	leaq	8(%rbp),%rbp+	movq	0(%rsp),%r13+	movq	104(%rsp),%rdi++	movq	%r13,%r12+	rorq	$7,%r13+	addq	%r14,%rbx+	movq	%rdi,%r14+	rorq	$42,%rdi++	xorq	%r12,%r13+	shrq	$7,%r12+	rorq	$1,%r13+	xorq	%r14,%rdi+	shrq	$6,%r14++	rorq	$19,%rdi+	xorq	%r13,%r12+	xorq	%r14,%rdi+	addq	64(%rsp),%r12++	addq	120(%rsp),%r12+	movq	%r9,%r13+	addq	%rdi,%r12+	movq	%rbx,%r14+	rorq	$23,%r13+	movq	%r10,%rdi++	xorq	%r9,%r13+	rorq	$5,%r14+	xorq	%r11,%rdi++	movq	%r12,120(%rsp)+	xorq	%rbx,%r14+	andq	%r9,%rdi++	rorq	$4,%r13+	addq	%rax,%r12+	xorq	%r11,%rdi++	rorq	$6,%r14+	xorq	%r9,%r13+	addq	%rdi,%r12++	movq	%rbx,%rdi+	addq	(%rbp),%r12+	xorq	%rbx,%r14++	xorq	%rcx,%rdi+	rorq	$14,%r13+	movq	%rcx,%rax++	andq	%rdi,%r15+	rorq	$28,%r14+	addq	%r13,%r12++	xorq	%r15,%rax+	addq	%r12,%r8+	addq	%r12,%rax++	leaq	24(%rbp),%rbp+	cmpb	$0,7(%rbp)+	jnz	.Lrounds_16_xx++	movq	128+0(%rsp),%rdi+	addq	%r14,%rax+	leaq	128(%rsi),%rsi++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	addq	48(%rdi),%r10+	addq	56(%rdi),%r11++	cmpq	128+16(%rsp),%rsi++	movq	%rax,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)+	jb	.Lloop++	leaq	128+24+48(%rsp),%r11++	movq	128+24(%rsp),%r15+	movq	-40(%r11),%r14+	movq	-32(%r11),%r13+	movq	-24(%r11),%r12+	movq	-16(%r11),%rbx+	movq	-8(%r11),%rbp+.LSEH_epilogue_crypton_sha512_asm_block_data_order:+	mov	8(%r11),%rdi+	mov	16(%r11),%rsi++	leaq	(%r11),%rsp+	.byte	0xf3,0xc3++.LSEH_end_crypton_sha512_asm_block_data_order:+.p2align	6++K512:+.quad	0x428a2f98d728ae22,0x7137449123ef65cd+.quad	0x428a2f98d728ae22,0x7137449123ef65cd+.quad	0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc+.quad	0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc+.quad	0x3956c25bf348b538,0x59f111f1b605d019+.quad	0x3956c25bf348b538,0x59f111f1b605d019+.quad	0x923f82a4af194f9b,0xab1c5ed5da6d8118+.quad	0x923f82a4af194f9b,0xab1c5ed5da6d8118+.quad	0xd807aa98a3030242,0x12835b0145706fbe+.quad	0xd807aa98a3030242,0x12835b0145706fbe+.quad	0x243185be4ee4b28c,0x550c7dc3d5ffb4e2+.quad	0x243185be4ee4b28c,0x550c7dc3d5ffb4e2+.quad	0x72be5d74f27b896f,0x80deb1fe3b1696b1+.quad	0x72be5d74f27b896f,0x80deb1fe3b1696b1+.quad	0x9bdc06a725c71235,0xc19bf174cf692694+.quad	0x9bdc06a725c71235,0xc19bf174cf692694+.quad	0xe49b69c19ef14ad2,0xefbe4786384f25e3+.quad	0xe49b69c19ef14ad2,0xefbe4786384f25e3+.quad	0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65+.quad	0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65+.quad	0x2de92c6f592b0275,0x4a7484aa6ea6e483+.quad	0x2de92c6f592b0275,0x4a7484aa6ea6e483+.quad	0x5cb0a9dcbd41fbd4,0x76f988da831153b5+.quad	0x5cb0a9dcbd41fbd4,0x76f988da831153b5+.quad	0x983e5152ee66dfab,0xa831c66d2db43210+.quad	0x983e5152ee66dfab,0xa831c66d2db43210+.quad	0xb00327c898fb213f,0xbf597fc7beef0ee4+.quad	0xb00327c898fb213f,0xbf597fc7beef0ee4+.quad	0xc6e00bf33da88fc2,0xd5a79147930aa725+.quad	0xc6e00bf33da88fc2,0xd5a79147930aa725+.quad	0x06ca6351e003826f,0x142929670a0e6e70+.quad	0x06ca6351e003826f,0x142929670a0e6e70+.quad	0x27b70a8546d22ffc,0x2e1b21385c26c926+.quad	0x27b70a8546d22ffc,0x2e1b21385c26c926+.quad	0x4d2c6dfc5ac42aed,0x53380d139d95b3df+.quad	0x4d2c6dfc5ac42aed,0x53380d139d95b3df+.quad	0x650a73548baf63de,0x766a0abb3c77b2a8+.quad	0x650a73548baf63de,0x766a0abb3c77b2a8+.quad	0x81c2c92e47edaee6,0x92722c851482353b+.quad	0x81c2c92e47edaee6,0x92722c851482353b+.quad	0xa2bfe8a14cf10364,0xa81a664bbc423001+.quad	0xa2bfe8a14cf10364,0xa81a664bbc423001+.quad	0xc24b8b70d0f89791,0xc76c51a30654be30+.quad	0xc24b8b70d0f89791,0xc76c51a30654be30+.quad	0xd192e819d6ef5218,0xd69906245565a910+.quad	0xd192e819d6ef5218,0xd69906245565a910+.quad	0xf40e35855771202a,0x106aa07032bbd1b8+.quad	0xf40e35855771202a,0x106aa07032bbd1b8+.quad	0x19a4c116b8d2d0c8,0x1e376c085141ab53+.quad	0x19a4c116b8d2d0c8,0x1e376c085141ab53+.quad	0x2748774cdf8eeb99,0x34b0bcb5e19b48a8+.quad	0x2748774cdf8eeb99,0x34b0bcb5e19b48a8+.quad	0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb+.quad	0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb+.quad	0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3+.quad	0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3+.quad	0x748f82ee5defb2fc,0x78a5636f43172f60+.quad	0x748f82ee5defb2fc,0x78a5636f43172f60+.quad	0x84c87814a1f0ab72,0x8cc702081a6439ec+.quad	0x84c87814a1f0ab72,0x8cc702081a6439ec+.quad	0x90befffa23631e28,0xa4506cebde82bde9+.quad	0x90befffa23631e28,0xa4506cebde82bde9+.quad	0xbef9a3f7b2c67915,0xc67178f2e372532b+.quad	0xbef9a3f7b2c67915,0xc67178f2e372532b+.quad	0xca273eceea26619c,0xd186b8c721c0c207+.quad	0xca273eceea26619c,0xd186b8c721c0c207+.quad	0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178+.quad	0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178+.quad	0x06f067aa72176fba,0x0a637dc5a2c898a6+.quad	0x06f067aa72176fba,0x0a637dc5a2c898a6+.quad	0x113f9804bef90dae,0x1b710b35131c471b+.quad	0x113f9804bef90dae,0x1b710b35131c471b+.quad	0x28db77f523047d84,0x32caab7b40c72493+.quad	0x28db77f523047d84,0x32caab7b40c72493+.quad	0x3c9ebe0a15c9bebc,0x431d67c49c100d4c+.quad	0x3c9ebe0a15c9bebc,0x431d67c49c100d4c+.quad	0x4cc5d4becb3e42b6,0x597f299cfc657e2a+.quad	0x4cc5d4becb3e42b6,0x597f299cfc657e2a+.quad	0x5fcb6fab3ad6faec,0x6c44198c4a475817+.quad	0x5fcb6fab3ad6faec,0x6c44198c4a475817++.quad	0x0001020304050607,0x08090a0b0c0d0e0f+.quad	0x0001020304050607,0x08090a0b0c0d0e0f++K512_nodup:+.quad	0x428a2f98d728ae22,0x7137449123ef65cd+.quad	0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc+.quad	0x3956c25bf348b538,0x59f111f1b605d019+.quad	0x923f82a4af194f9b,0xab1c5ed5da6d8118+.quad	0xd807aa98a3030242,0x12835b0145706fbe+.quad	0x243185be4ee4b28c,0x550c7dc3d5ffb4e2+.quad	0x72be5d74f27b896f,0x80deb1fe3b1696b1+.quad	0x9bdc06a725c71235,0xc19bf174cf692694+.quad	0xe49b69c19ef14ad2,0xefbe4786384f25e3+.quad	0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65+.quad	0x2de92c6f592b0275,0x4a7484aa6ea6e483+.quad	0x5cb0a9dcbd41fbd4,0x76f988da831153b5+.quad	0x983e5152ee66dfab,0xa831c66d2db43210+.quad	0xb00327c898fb213f,0xbf597fc7beef0ee4+.quad	0xc6e00bf33da88fc2,0xd5a79147930aa725+.quad	0x06ca6351e003826f,0x142929670a0e6e70+.quad	0x27b70a8546d22ffc,0x2e1b21385c26c926+.quad	0x4d2c6dfc5ac42aed,0x53380d139d95b3df+.quad	0x650a73548baf63de,0x766a0abb3c77b2a8+.quad	0x81c2c92e47edaee6,0x92722c851482353b+.quad	0xa2bfe8a14cf10364,0xa81a664bbc423001+.quad	0xc24b8b70d0f89791,0xc76c51a30654be30+.quad	0xd192e819d6ef5218,0xd69906245565a910+.quad	0xf40e35855771202a,0x106aa07032bbd1b8+.quad	0x19a4c116b8d2d0c8,0x1e376c085141ab53+.quad	0x2748774cdf8eeb99,0x34b0bcb5e19b48a8+.quad	0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb+.quad	0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3+.quad	0x748f82ee5defb2fc,0x78a5636f43172f60+.quad	0x84c87814a1f0ab72,0x8cc702081a6439ec+.quad	0x90befffa23631e28,0xa4506cebde82bde9+.quad	0xbef9a3f7b2c67915,0xc67178f2e372532b+.quad	0xca273eceea26619c,0xd186b8c721c0c207+.quad	0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178+.quad	0x06f067aa72176fba,0x0a637dc5a2c898a6+.quad	0x113f9804bef90dae,0x1b710b35131c471b+.quad	0x28db77f523047d84,0x32caab7b40c72493+.quad	0x3c9ebe0a15c9bebc,0x431d67c49c100d4c+.quad	0x4cc5d4becb3e42b6,0x597f299cfc657e2a+.quad	0x5fcb6fab3ad6faec,0x6c44198c4a475817++.byte	83,72,65,53,49,50,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0+.globl	crypton_sha512_asm_block_data_order_shaext+.def	crypton_sha512_asm_block_data_order_shaext;	.scl 2;	.type 32;	.endef+.p2align	6+crypton_sha512_asm_block_data_order_shaext:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_sha512_asm_block_data_order_shaext:+++	pushq	%rbp++	movq	%rsp,%rbp++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+.Lshaext_shortcut:+	subq	$0x50,%rsp++	movaps	%xmm6,-80(%rbp)+	movaps	%xmm7,-64(%rbp)+	movaps	%xmm8,-48(%rbp)+	movaps	%xmm9,-32(%rbp)+	movaps	%xmm10,-16(%rbp)++.LSEH_body_crypton_sha512_asm_block_data_order_shaext:++	leaq	K512_nodup+128(%rip),%rcx+	vmovdqu	(%rdi),%ymm0+	vmovdqu	32(%rdi),%ymm1+	vmovdqa	-160(%rcx),%ymm8++	vpermq	$27,%ymm0,%ymm0+	vpblendd	$15,%ymm1,%ymm0,%ymm5+	vpblendd	$15,%ymm0,%ymm1,%ymm6+	vpermq	$225,%ymm5,%ymm5+	vpermq	$75,%ymm6,%ymm6+	jmp	.Loop_shaext++.p2align	4+.Loop_shaext:+	vmovdqu	(%rsi),%ymm0+	vmovdqu	32(%rsi),%ymm1+	vmovdqu	64(%rsi),%ymm2+	vpshufb	%ymm8,%ymm0,%ymm0+	vmovdqu	96(%rsi),%ymm3++	vpaddq	0-128(%rcx),%ymm0,%ymm4+	vpshufb	%ymm8,%ymm1,%ymm1+	vmovdqa	%ymm6,%ymm10+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vmovdqa	%ymm5,%ymm9+.byte	196,226,79,203,236++	vpaddq	32-128(%rcx),%ymm1,%ymm4+	vpshufb	%ymm8,%ymm2,%ymm2+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	leaq	128(%rsi),%rsi+.byte	196,226,127,204,193+.byte	196,226,79,203,236++	vpaddq	64-128(%rcx),%ymm2,%ymm4+	vpshufb	%ymm8,%ymm3,%ymm3+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm3,%ymm2,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm0,%ymm0+.byte	196,226,127,204,202+.byte	196,226,79,203,236++	vpaddq	96-128(%rcx),%ymm3,%ymm4+.byte	196,226,127,205,195+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm0,%ymm3,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm1,%ymm1+.byte	196,226,127,204,211+.byte	196,226,79,203,236+	vpaddq	128-128(%rcx),%ymm0,%ymm4+.byte	196,226,127,205,200+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm1,%ymm0,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm2,%ymm2+.byte	196,226,127,204,216+.byte	196,226,79,203,236+	vpaddq	160-128(%rcx),%ymm1,%ymm4+.byte	196,226,127,205,209+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm2,%ymm1,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm3,%ymm3+.byte	196,226,127,204,193+.byte	196,226,79,203,236+	vpaddq	192-128(%rcx),%ymm2,%ymm4+.byte	196,226,127,205,218+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm3,%ymm2,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm0,%ymm0+.byte	196,226,127,204,202+.byte	196,226,79,203,236+	vpaddq	224-128(%rcx),%ymm3,%ymm4+.byte	196,226,127,205,195+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm0,%ymm3,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm1,%ymm1+.byte	196,226,127,204,211+.byte	196,226,79,203,236+	vpaddq	256-128(%rcx),%ymm0,%ymm4+.byte	196,226,127,205,200+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm1,%ymm0,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm2,%ymm2+.byte	196,226,127,204,216+.byte	196,226,79,203,236+	vpaddq	288-128(%rcx),%ymm1,%ymm4+.byte	196,226,127,205,209+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm2,%ymm1,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm3,%ymm3+.byte	196,226,127,204,193+.byte	196,226,79,203,236+	vpaddq	320-128(%rcx),%ymm2,%ymm4+.byte	196,226,127,205,218+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm3,%ymm2,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm0,%ymm0+.byte	196,226,127,204,202+.byte	196,226,79,203,236+	vpaddq	352-128(%rcx),%ymm3,%ymm4+.byte	196,226,127,205,195+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm0,%ymm3,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm1,%ymm1+.byte	196,226,127,204,211+.byte	196,226,79,203,236+	vpaddq	384-128(%rcx),%ymm0,%ymm4+.byte	196,226,127,205,200+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm1,%ymm0,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm2,%ymm2+.byte	196,226,127,204,216+.byte	196,226,79,203,236+	vpaddq	416-128(%rcx),%ymm1,%ymm4+.byte	196,226,127,205,209+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm2,%ymm1,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm3,%ymm3+.byte	196,226,127,204,193+.byte	196,226,79,203,236+	vpaddq	448-128(%rcx),%ymm2,%ymm4+.byte	196,226,127,205,218+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm3,%ymm2,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm0,%ymm0+.byte	196,226,127,204,202+.byte	196,226,79,203,236+	vpaddq	480-128(%rcx),%ymm3,%ymm4+.byte	196,226,127,205,195+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm0,%ymm3,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm1,%ymm1+.byte	196,226,127,204,211+.byte	196,226,79,203,236+	vpaddq	512-128(%rcx),%ymm0,%ymm4+.byte	196,226,127,205,200+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm1,%ymm0,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+	vpaddq	%ymm7,%ymm2,%ymm2+.byte	196,226,127,204,216+.byte	196,226,79,203,236+	vpaddq	544-128(%rcx),%ymm1,%ymm4+.byte	196,226,127,205,209+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	vpblendd	$0x03,%ymm2,%ymm1,%ymm7+	vpermq	$0x39,%ymm7,%ymm7+.byte	196,226,79,203,236+	vpaddq	%ymm7,%ymm3,%ymm3++	vpaddq	576-128(%rcx),%ymm2,%ymm4+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+.byte	196,226,127,205,218+.byte	196,226,79,203,236++	vpaddq	608-128(%rcx),%ymm3,%ymm4+.byte	196,226,87,203,244+	vextracti128	$1,%ymm4,%xmm4+	decq	%rdx+.byte	196,226,79,203,236++	vpaddq	%ymm10,%ymm6,%ymm6+	vpaddq	%ymm9,%ymm5,%ymm5+	jnz	.Loop_shaext++	vpermq	$75,%ymm5,%ymm5+	vpblendd	$240,%ymm6,%ymm5,%ymm1+	vpblendd	$240,%ymm5,%ymm6,%ymm2+	vpermq	$180,%ymm1,%ymm1+	vpermq	$27,%ymm2,%ymm2++	vmovdqu	%ymm1,(%rdi)+	vmovdqu	%ymm2,32(%rdi)++	vzeroupper+	movaps	-80(%rbp),%xmm6+	movaps	-64(%rbp),%xmm7+	movaps	-48(%rbp),%xmm8+	movaps	-32(%rbp),%xmm9+	movaps	-16(%rbp),%xmm10+	movq	%rbp,%rsp++	popq	%rbp++.LSEH_epilogue_crypton_sha512_asm_block_data_order_shaext:+	mov	8(%rsp),%rdi+	mov	16(%rsp),%rsi++	.byte	0xf3,0xc3++.LSEH_end_crypton_sha512_asm_block_data_order_shaext:+.def	crypton_sha512_asm_block_data_order_xop;	.scl 3;	.type 32;	.endef+.p2align	6+crypton_sha512_asm_block_data_order_xop:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_sha512_asm_block_data_order_xop:+++	pushq	%rbp++	movq	%rsp,%rbp++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+.Lxop_shortcut:+	pushq	%rbx++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	shlq	$4,%rdx+	subq	$120,%rsp++	leaq	(%rsi,%rdx,8),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+	movaps	%xmm6,-160(%rbp)+	movaps	%xmm7,-144(%rbp)+	movaps	%xmm8,-128(%rbp)+	movaps	%xmm9,-112(%rbp)++	movaps	%xmm10,-96(%rbp)+	movaps	%xmm11,-80(%rbp)++.LSEH_body_crypton_sha512_asm_block_data_order_xop:+++	leaq	-128(%rsp),%rsp+	vzeroupper+	andq	$-64,%rsp+	movq	0(%rdi),%rax+	movq	8(%rdi),%rbx+	movq	16(%rdi),%rcx+	movq	24(%rdi),%rdx+	movq	32(%rdi),%r8+	movq	40(%rdi),%r9+	movq	48(%rdi),%r10+	movq	56(%rdi),%r11+	jmp	.Lloop_xop+.p2align	4+.Lloop_xop:+	vmovdqa	K512+1280(%rip),%xmm11+	movq	%rsi,-56(%rbp)+	vmovdqu	0(%rsi),%xmm0+	vmovdqu	16(%rsi),%xmm1+	vmovdqu	32(%rsi),%xmm2+	vpshufb	%xmm11,%xmm0,%xmm0+	vmovdqu	48(%rsi),%xmm3+	vpshufb	%xmm11,%xmm1,%xmm1+	vmovdqu	64(%rsi),%xmm4+	vpshufb	%xmm11,%xmm2,%xmm2+	vmovdqu	80(%rsi),%xmm5+	vpshufb	%xmm11,%xmm3,%xmm3+	vmovdqu	96(%rsi),%xmm6+	vpshufb	%xmm11,%xmm4,%xmm4+	vmovdqu	112(%rsi),%xmm7+	leaq	K512+128(%rip),%rsi+	vpshufb	%xmm11,%xmm5,%xmm5+	vpaddq	-128(%rsi),%xmm0,%xmm8+	vpshufb	%xmm11,%xmm6,%xmm6+	vpaddq	-96(%rsi),%xmm1,%xmm9+	vpshufb	%xmm11,%xmm7,%xmm7+	vpaddq	-64(%rsi),%xmm2,%xmm10+	vpaddq	-32(%rsi),%xmm3,%xmm11+	vmovdqa	%xmm8,0(%rsp)+	vpaddq	0(%rsi),%xmm4,%xmm8+	vmovdqa	%xmm9,16(%rsp)+	vpaddq	32(%rsi),%xmm5,%xmm9+	vmovdqa	%xmm10,32(%rsp)+	vpaddq	64(%rsi),%xmm6,%xmm10+	vmovdqa	%xmm11,48(%rsp)+	vpaddq	96(%rsi),%xmm7,%xmm11+	vmovdqa	%xmm8,64(%rsp)+	movq	%rax,%r14+	vmovdqa	%xmm9,80(%rsp)+	movq	%rbx,%rdi+	vmovdqa	%xmm10,96(%rsp)+	xorq	%rcx,%rdi+	vmovdqa	%xmm11,112(%rsp)+	movq	%r8,%r13+	jmp	.Lxop_00_47++.p2align	4+.Lxop_00_47:+	addq	$256,%rsi+	vpalignr	$8,%xmm0,%xmm1,%xmm8+	rorq	$23,%r13+	movq	%r14,%rax+	vpalignr	$8,%xmm4,%xmm5,%xmm11+	movq	%r9,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%r8,%r13+	xorq	%r10,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%rax,%r14+	vpaddq	%xmm11,%xmm0,%xmm0+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	0(%rsp),%r11+	movq	%rax,%r15+.byte	143,72,120,195,209,7+	xorq	%r10,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%rbx,%r15+	addq	%r12,%r11+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,223,3+	xorq	%rax,%r14+	addq	%r13,%r11+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rbx,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm7,%xmm10+	addq	%r11,%rdx+	addq	%rdi,%r11+	vpaddq	%xmm8,%xmm0,%xmm0+	movq	%rdx,%r13+	addq	%r11,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%r11+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%r8,%r12+	rorq	$5,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	vpaddq	%xmm11,%xmm0,%xmm0+	addq	8(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	rorq	$6,%r14+	vpaddq	-128(%rsi),%xmm0,%xmm10+	xorq	%rax,%rdi+	addq	%r12,%r10+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	rorq	$28,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	vmovdqa	%xmm10,0(%rsp)+	vpalignr	$8,%xmm1,%xmm2,%xmm8+	rorq	$23,%r13+	movq	%r14,%r10+	vpalignr	$8,%xmm5,%xmm6,%xmm11+	movq	%rdx,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%rcx,%r13+	xorq	%r8,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%r10,%r14+	vpaddq	%xmm11,%xmm1,%xmm1+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	16(%rsp),%r9+	movq	%r10,%r15+.byte	143,72,120,195,209,7+	xorq	%r8,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%r11,%r15+	addq	%r12,%r9+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,216,3+	xorq	%r10,%r14+	addq	%r13,%r9+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r11,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm0,%xmm10+	addq	%r9,%rbx+	addq	%rdi,%r9+	vpaddq	%xmm8,%xmm1,%xmm1+	movq	%rbx,%r13+	addq	%r9,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%r9+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%rcx,%r12+	rorq	$5,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	vpaddq	%xmm11,%xmm1,%xmm1+	addq	24(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	rorq	$6,%r14+	vpaddq	-96(%rsi),%xmm1,%xmm10+	xorq	%r10,%rdi+	addq	%r12,%r8+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	rorq	$28,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	vmovdqa	%xmm10,16(%rsp)+	vpalignr	$8,%xmm2,%xmm3,%xmm8+	rorq	$23,%r13+	movq	%r14,%r8+	vpalignr	$8,%xmm6,%xmm7,%xmm11+	movq	%rbx,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%rax,%r13+	xorq	%rcx,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%r8,%r14+	vpaddq	%xmm11,%xmm2,%xmm2+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	32(%rsp),%rdx+	movq	%r8,%r15+.byte	143,72,120,195,209,7+	xorq	%rcx,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%r9,%r15+	addq	%r12,%rdx+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,217,3+	xorq	%r8,%r14+	addq	%r13,%rdx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r9,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm1,%xmm10+	addq	%rdx,%r11+	addq	%rdi,%rdx+	vpaddq	%xmm8,%xmm2,%xmm2+	movq	%r11,%r13+	addq	%rdx,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%rdx+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%rax,%r12+	rorq	$5,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	vpaddq	%xmm11,%xmm2,%xmm2+	addq	40(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	rorq	$6,%r14+	vpaddq	-64(%rsi),%xmm2,%xmm10+	xorq	%r8,%rdi+	addq	%r12,%rcx+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	rorq	$28,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	vmovdqa	%xmm10,32(%rsp)+	vpalignr	$8,%xmm3,%xmm4,%xmm8+	rorq	$23,%r13+	movq	%r14,%rcx+	vpalignr	$8,%xmm7,%xmm0,%xmm11+	movq	%r11,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%r10,%r13+	xorq	%rax,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%rcx,%r14+	vpaddq	%xmm11,%xmm3,%xmm3+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	48(%rsp),%rbx+	movq	%rcx,%r15+.byte	143,72,120,195,209,7+	xorq	%rax,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%rdx,%r15+	addq	%r12,%rbx+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,218,3+	xorq	%rcx,%r14+	addq	%r13,%rbx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rdx,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm2,%xmm10+	addq	%rbx,%r9+	addq	%rdi,%rbx+	vpaddq	%xmm8,%xmm3,%xmm3+	movq	%r9,%r13+	addq	%rbx,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%rbx+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%r10,%r12+	rorq	$5,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	vpaddq	%xmm11,%xmm3,%xmm3+	addq	56(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	rorq	$6,%r14+	vpaddq	-32(%rsi),%xmm3,%xmm10+	xorq	%rcx,%rdi+	addq	%r12,%rax+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	rorq	$28,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	vmovdqa	%xmm10,48(%rsp)+	vpalignr	$8,%xmm4,%xmm5,%xmm8+	rorq	$23,%r13+	movq	%r14,%rax+	vpalignr	$8,%xmm0,%xmm1,%xmm11+	movq	%r9,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%r8,%r13+	xorq	%r10,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%rax,%r14+	vpaddq	%xmm11,%xmm4,%xmm4+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	64(%rsp),%r11+	movq	%rax,%r15+.byte	143,72,120,195,209,7+	xorq	%r10,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%rbx,%r15+	addq	%r12,%r11+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,219,3+	xorq	%rax,%r14+	addq	%r13,%r11+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rbx,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm3,%xmm10+	addq	%r11,%rdx+	addq	%rdi,%r11+	vpaddq	%xmm8,%xmm4,%xmm4+	movq	%rdx,%r13+	addq	%r11,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%r11+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%r8,%r12+	rorq	$5,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	vpaddq	%xmm11,%xmm4,%xmm4+	addq	72(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	rorq	$6,%r14+	vpaddq	0(%rsi),%xmm4,%xmm10+	xorq	%rax,%rdi+	addq	%r12,%r10+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	rorq	$28,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	vmovdqa	%xmm10,64(%rsp)+	vpalignr	$8,%xmm5,%xmm6,%xmm8+	rorq	$23,%r13+	movq	%r14,%r10+	vpalignr	$8,%xmm1,%xmm2,%xmm11+	movq	%rdx,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%rcx,%r13+	xorq	%r8,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%r10,%r14+	vpaddq	%xmm11,%xmm5,%xmm5+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	80(%rsp),%r9+	movq	%r10,%r15+.byte	143,72,120,195,209,7+	xorq	%r8,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%r11,%r15+	addq	%r12,%r9+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,220,3+	xorq	%r10,%r14+	addq	%r13,%r9+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r11,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm4,%xmm10+	addq	%r9,%rbx+	addq	%rdi,%r9+	vpaddq	%xmm8,%xmm5,%xmm5+	movq	%rbx,%r13+	addq	%r9,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%r9+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%rcx,%r12+	rorq	$5,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	vpaddq	%xmm11,%xmm5,%xmm5+	addq	88(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	rorq	$6,%r14+	vpaddq	32(%rsi),%xmm5,%xmm10+	xorq	%r10,%rdi+	addq	%r12,%r8+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	rorq	$28,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	vmovdqa	%xmm10,80(%rsp)+	vpalignr	$8,%xmm6,%xmm7,%xmm8+	rorq	$23,%r13+	movq	%r14,%r8+	vpalignr	$8,%xmm2,%xmm3,%xmm11+	movq	%rbx,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%rax,%r13+	xorq	%rcx,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%r8,%r14+	vpaddq	%xmm11,%xmm6,%xmm6+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	96(%rsp),%rdx+	movq	%r8,%r15+.byte	143,72,120,195,209,7+	xorq	%rcx,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%r9,%r15+	addq	%r12,%rdx+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,221,3+	xorq	%r8,%r14+	addq	%r13,%rdx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r9,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm5,%xmm10+	addq	%rdx,%r11+	addq	%rdi,%rdx+	vpaddq	%xmm8,%xmm6,%xmm6+	movq	%r11,%r13+	addq	%rdx,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%rdx+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%rax,%r12+	rorq	$5,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	vpaddq	%xmm11,%xmm6,%xmm6+	addq	104(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	rorq	$6,%r14+	vpaddq	64(%rsi),%xmm6,%xmm10+	xorq	%r8,%rdi+	addq	%r12,%rcx+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	rorq	$28,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	vmovdqa	%xmm10,96(%rsp)+	vpalignr	$8,%xmm7,%xmm0,%xmm8+	rorq	$23,%r13+	movq	%r14,%rcx+	vpalignr	$8,%xmm3,%xmm4,%xmm11+	movq	%r11,%r12+	rorq	$5,%r14+.byte	143,72,120,195,200,56+	xorq	%r10,%r13+	xorq	%rax,%r12+	vpsrlq	$7,%xmm8,%xmm8+	rorq	$4,%r13+	xorq	%rcx,%r14+	vpaddq	%xmm11,%xmm7,%xmm7+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	112(%rsp),%rbx+	movq	%rcx,%r15+.byte	143,72,120,195,209,7+	xorq	%rax,%r12+	rorq	$6,%r14+	vpxor	%xmm9,%xmm8,%xmm8+	xorq	%rdx,%r15+	addq	%r12,%rbx+	rorq	$14,%r13+	andq	%r15,%rdi+.byte	143,104,120,195,222,3+	xorq	%rcx,%r14+	addq	%r13,%rbx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rdx,%rdi+	rorq	$28,%r14+	vpsrlq	$6,%xmm6,%xmm10+	addq	%rbx,%r9+	addq	%rdi,%rbx+	vpaddq	%xmm8,%xmm7,%xmm7+	movq	%r9,%r13+	addq	%rbx,%r14+.byte	143,72,120,195,203,42+	rorq	$23,%r13+	movq	%r14,%rbx+	vpxor	%xmm10,%xmm11,%xmm11+	movq	%r10,%r12+	rorq	$5,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	vpxor	%xmm9,%xmm11,%xmm11+	rorq	$4,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	vpaddq	%xmm11,%xmm7,%xmm7+	addq	120(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	rorq	$6,%r14+	vpaddq	96(%rsi),%xmm7,%xmm10+	xorq	%rcx,%rdi+	addq	%r12,%rax+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	rorq	$28,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	vmovdqa	%xmm10,112(%rsp)+	cmpb	$0,135(%rsi)+	jne	.Lxop_00_47+	rorq	$23,%r13+	movq	%r14,%rax+	movq	%r9,%r12+	rorq	$5,%r14+	xorq	%r8,%r13+	xorq	%r10,%r12+	rorq	$4,%r13+	xorq	%rax,%r14+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	0(%rsp),%r11+	movq	%rax,%r15+	xorq	%r10,%r12+	rorq	$6,%r14+	xorq	%rbx,%r15+	addq	%r12,%r11+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%rax,%r14+	addq	%r13,%r11+	xorq	%rbx,%rdi+	rorq	$28,%r14+	addq	%r11,%rdx+	addq	%rdi,%r11+	movq	%rdx,%r13+	addq	%r11,%r14+	rorq	$23,%r13+	movq	%r14,%r11+	movq	%r8,%r12+	rorq	$5,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	rorq	$4,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	addq	8(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	rorq	$6,%r14+	xorq	%rax,%rdi+	addq	%r12,%r10+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	rorq	$28,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	rorq	$23,%r13+	movq	%r14,%r10+	movq	%rdx,%r12+	rorq	$5,%r14+	xorq	%rcx,%r13+	xorq	%r8,%r12+	rorq	$4,%r13+	xorq	%r10,%r14+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	16(%rsp),%r9+	movq	%r10,%r15+	xorq	%r8,%r12+	rorq	$6,%r14+	xorq	%r11,%r15+	addq	%r12,%r9+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%r10,%r14+	addq	%r13,%r9+	xorq	%r11,%rdi+	rorq	$28,%r14+	addq	%r9,%rbx+	addq	%rdi,%r9+	movq	%rbx,%r13+	addq	%r9,%r14+	rorq	$23,%r13+	movq	%r14,%r9+	movq	%rcx,%r12+	rorq	$5,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	rorq	$4,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	addq	24(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	rorq	$6,%r14+	xorq	%r10,%rdi+	addq	%r12,%r8+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	rorq	$28,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	rorq	$23,%r13+	movq	%r14,%r8+	movq	%rbx,%r12+	rorq	$5,%r14+	xorq	%rax,%r13+	xorq	%rcx,%r12+	rorq	$4,%r13+	xorq	%r8,%r14+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	32(%rsp),%rdx+	movq	%r8,%r15+	xorq	%rcx,%r12+	rorq	$6,%r14+	xorq	%r9,%r15+	addq	%r12,%rdx+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%r8,%r14+	addq	%r13,%rdx+	xorq	%r9,%rdi+	rorq	$28,%r14+	addq	%rdx,%r11+	addq	%rdi,%rdx+	movq	%r11,%r13+	addq	%rdx,%r14+	rorq	$23,%r13+	movq	%r14,%rdx+	movq	%rax,%r12+	rorq	$5,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	rorq	$4,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	addq	40(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	rorq	$6,%r14+	xorq	%r8,%rdi+	addq	%r12,%rcx+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	rorq	$28,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	rorq	$23,%r13+	movq	%r14,%rcx+	movq	%r11,%r12+	rorq	$5,%r14+	xorq	%r10,%r13+	xorq	%rax,%r12+	rorq	$4,%r13+	xorq	%rcx,%r14+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	48(%rsp),%rbx+	movq	%rcx,%r15+	xorq	%rax,%r12+	rorq	$6,%r14+	xorq	%rdx,%r15+	addq	%r12,%rbx+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%rcx,%r14+	addq	%r13,%rbx+	xorq	%rdx,%rdi+	rorq	$28,%r14+	addq	%rbx,%r9+	addq	%rdi,%rbx+	movq	%r9,%r13+	addq	%rbx,%r14+	rorq	$23,%r13+	movq	%r14,%rbx+	movq	%r10,%r12+	rorq	$5,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	rorq	$4,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	addq	56(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	rorq	$6,%r14+	xorq	%rcx,%rdi+	addq	%r12,%rax+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	rorq	$28,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	rorq	$23,%r13+	movq	%r14,%rax+	movq	%r9,%r12+	rorq	$5,%r14+	xorq	%r8,%r13+	xorq	%r10,%r12+	rorq	$4,%r13+	xorq	%rax,%r14+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	64(%rsp),%r11+	movq	%rax,%r15+	xorq	%r10,%r12+	rorq	$6,%r14+	xorq	%rbx,%r15+	addq	%r12,%r11+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%rax,%r14+	addq	%r13,%r11+	xorq	%rbx,%rdi+	rorq	$28,%r14+	addq	%r11,%rdx+	addq	%rdi,%r11+	movq	%rdx,%r13+	addq	%r11,%r14+	rorq	$23,%r13+	movq	%r14,%r11+	movq	%r8,%r12+	rorq	$5,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	rorq	$4,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	addq	72(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	rorq	$6,%r14+	xorq	%rax,%rdi+	addq	%r12,%r10+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	rorq	$28,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	rorq	$23,%r13+	movq	%r14,%r10+	movq	%rdx,%r12+	rorq	$5,%r14+	xorq	%rcx,%r13+	xorq	%r8,%r12+	rorq	$4,%r13+	xorq	%r10,%r14+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	80(%rsp),%r9+	movq	%r10,%r15+	xorq	%r8,%r12+	rorq	$6,%r14+	xorq	%r11,%r15+	addq	%r12,%r9+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%r10,%r14+	addq	%r13,%r9+	xorq	%r11,%rdi+	rorq	$28,%r14+	addq	%r9,%rbx+	addq	%rdi,%r9+	movq	%rbx,%r13+	addq	%r9,%r14+	rorq	$23,%r13+	movq	%r14,%r9+	movq	%rcx,%r12+	rorq	$5,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	rorq	$4,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	addq	88(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	rorq	$6,%r14+	xorq	%r10,%rdi+	addq	%r12,%r8+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	rorq	$28,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	rorq	$23,%r13+	movq	%r14,%r8+	movq	%rbx,%r12+	rorq	$5,%r14+	xorq	%rax,%r13+	xorq	%rcx,%r12+	rorq	$4,%r13+	xorq	%r8,%r14+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	96(%rsp),%rdx+	movq	%r8,%r15+	xorq	%rcx,%r12+	rorq	$6,%r14+	xorq	%r9,%r15+	addq	%r12,%rdx+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%r8,%r14+	addq	%r13,%rdx+	xorq	%r9,%rdi+	rorq	$28,%r14+	addq	%rdx,%r11+	addq	%rdi,%rdx+	movq	%r11,%r13+	addq	%rdx,%r14+	rorq	$23,%r13+	movq	%r14,%rdx+	movq	%rax,%r12+	rorq	$5,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	rorq	$4,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	addq	104(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	rorq	$6,%r14+	xorq	%r8,%rdi+	addq	%r12,%rcx+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	rorq	$28,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	rorq	$23,%r13+	movq	%r14,%rcx+	movq	%r11,%r12+	rorq	$5,%r14+	xorq	%r10,%r13+	xorq	%rax,%r12+	rorq	$4,%r13+	xorq	%rcx,%r14+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	112(%rsp),%rbx+	movq	%rcx,%r15+	xorq	%rax,%r12+	rorq	$6,%r14+	xorq	%rdx,%r15+	addq	%r12,%rbx+	rorq	$14,%r13+	andq	%r15,%rdi+	xorq	%rcx,%r14+	addq	%r13,%rbx+	xorq	%rdx,%rdi+	rorq	$28,%r14+	addq	%rbx,%r9+	addq	%rdi,%rbx+	movq	%r9,%r13+	addq	%rbx,%r14+	rorq	$23,%r13+	movq	%r14,%rbx+	movq	%r10,%r12+	rorq	$5,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	rorq	$4,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	addq	120(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	rorq	$6,%r14+	xorq	%rcx,%rdi+	addq	%r12,%rax+	rorq	$14,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	rorq	$28,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	movq	-64(%rbp),%rdi+	movq	%r14,%rax+	movq	-56(%rbp),%rsi++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	addq	48(%rdi),%r10+	addq	56(%rdi),%r11++	leaq	128(%rsi),%rsi+	cmpq	-48(%rbp),%rsi++	movq	%rax,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)+	jb	.Lloop_xop++	vzeroupper+	movaps	-160(%rbp),%xmm6+	movaps	-144(%rbp),%xmm7+	movaps	-128(%rbp),%xmm8+	movaps	-112(%rbp),%xmm9+	movaps	-96(%rbp),%xmm10+	movaps	-80(%rbp),%xmm11+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp++	popq	%rbp++.LSEH_epilogue_crypton_sha512_asm_block_data_order_xop:+	mov	8(%rsp),%rdi+	mov	16(%rsp),%rsi++	.byte	0xf3,0xc3++.LSEH_end_crypton_sha512_asm_block_data_order_xop:+.def	crypton_sha512_asm_block_data_order_avx;	.scl 3;	.type 32;	.endef+.p2align	6+crypton_sha512_asm_block_data_order_avx:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_sha512_asm_block_data_order_avx:+++	pushq	%rbp++	movq	%rsp,%rbp++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+.Lavx_shortcut:+	pushq	%rbx++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	shlq	$4,%rdx+	subq	$120,%rsp++	leaq	(%rsi,%rdx,8),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+	movaps	%xmm6,-160(%rbp)+	movaps	%xmm7,-144(%rbp)+	movaps	%xmm8,-128(%rbp)+	movaps	%xmm9,-112(%rbp)++	movaps	%xmm10,-96(%rbp)+	movaps	%xmm11,-80(%rbp)++.LSEH_body_crypton_sha512_asm_block_data_order_avx:+++	leaq	-128(%rsp),%rsp+	vzeroupper+	andq	$-64,%rsp+	movq	0(%rdi),%rax+	movq	8(%rdi),%rbx+	movq	16(%rdi),%rcx+	movq	24(%rdi),%rdx+	movq	32(%rdi),%r8+	movq	40(%rdi),%r9+	movq	48(%rdi),%r10+	movq	56(%rdi),%r11+	jmp	.Lloop_avx+.p2align	4+.Lloop_avx:+	vmovdqa	K512+1280(%rip),%xmm11+	movq	%rsi,-56(%rbp)+	vmovdqu	0(%rsi),%xmm0+	vmovdqu	16(%rsi),%xmm1+	vmovdqu	32(%rsi),%xmm2+	vpshufb	%xmm11,%xmm0,%xmm0+	vmovdqu	48(%rsi),%xmm3+	vpshufb	%xmm11,%xmm1,%xmm1+	vmovdqu	64(%rsi),%xmm4+	vpshufb	%xmm11,%xmm2,%xmm2+	vmovdqu	80(%rsi),%xmm5+	vpshufb	%xmm11,%xmm3,%xmm3+	vmovdqu	96(%rsi),%xmm6+	vpshufb	%xmm11,%xmm4,%xmm4+	vmovdqu	112(%rsi),%xmm7+	leaq	K512+128(%rip),%rsi+	vpshufb	%xmm11,%xmm5,%xmm5+	vpaddq	-128(%rsi),%xmm0,%xmm8+	vpshufb	%xmm11,%xmm6,%xmm6+	vpaddq	-96(%rsi),%xmm1,%xmm9+	vpshufb	%xmm11,%xmm7,%xmm7+	vpaddq	-64(%rsi),%xmm2,%xmm10+	vpaddq	-32(%rsi),%xmm3,%xmm11+	vmovdqa	%xmm8,0(%rsp)+	vpaddq	0(%rsi),%xmm4,%xmm8+	vmovdqa	%xmm9,16(%rsp)+	vpaddq	32(%rsi),%xmm5,%xmm9+	vmovdqa	%xmm10,32(%rsp)+	vpaddq	64(%rsi),%xmm6,%xmm10+	vmovdqa	%xmm11,48(%rsp)+	vpaddq	96(%rsi),%xmm7,%xmm11+	vmovdqa	%xmm8,64(%rsp)+	movq	%rax,%r14+	vmovdqa	%xmm9,80(%rsp)+	movq	%rbx,%rdi+	vmovdqa	%xmm10,96(%rsp)+	xorq	%rcx,%rdi+	vmovdqa	%xmm11,112(%rsp)+	movq	%r8,%r13+	jmp	.Lavx_00_47++.p2align	4+.Lavx_00_47:+	addq	$256,%rsi+	vpalignr	$8,%xmm0,%xmm1,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%rax+	vpalignr	$8,%xmm4,%xmm5,%xmm11+	movq	%r9,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%r8,%r13+	xorq	%r10,%r12+	vpaddq	%xmm11,%xmm0,%xmm0+	shrdq	$4,%r13,%r13+	xorq	%rax,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%r8,%r12+	xorq	%r8,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	0(%rsp),%r11+	movq	%rax,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%r10,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%rbx,%r15+	addq	%r12,%r11+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%rax,%r14+	addq	%r13,%r11+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rbx,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm7,%xmm11+	addq	%r11,%rdx+	addq	%rdi,%r11+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%rdx,%r13+	addq	%r11,%r14+	vpsllq	$3,%xmm7,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%r11+	vpaddq	%xmm8,%xmm0,%xmm0+	movq	%r8,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm7,%xmm9+	xorq	%rdx,%r13+	xorq	%r9,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%r11,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%rdx,%r12+	xorq	%rdx,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	8(%rsp),%r10+	movq	%r11,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%r9,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%rax,%rdi+	addq	%r12,%r10+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm0,%xmm0+	xorq	%r11,%r14+	addq	%r13,%r10+	vpaddq	-128(%rsi),%xmm0,%xmm10+	xorq	%rax,%r15+	shrdq	$28,%r14,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	vmovdqa	%xmm10,0(%rsp)+	vpalignr	$8,%xmm1,%xmm2,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%r10+	vpalignr	$8,%xmm5,%xmm6,%xmm11+	movq	%rdx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%rcx,%r13+	xorq	%r8,%r12+	vpaddq	%xmm11,%xmm1,%xmm1+	shrdq	$4,%r13,%r13+	xorq	%r10,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%rcx,%r12+	xorq	%rcx,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	16(%rsp),%r9+	movq	%r10,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%r8,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%r11,%r15+	addq	%r12,%r9+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%r10,%r14+	addq	%r13,%r9+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r11,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm0,%xmm11+	addq	%r9,%rbx+	addq	%rdi,%r9+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%rbx,%r13+	addq	%r9,%r14+	vpsllq	$3,%xmm0,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%r9+	vpaddq	%xmm8,%xmm1,%xmm1+	movq	%rcx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm0,%xmm9+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%r9,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%rbx,%r12+	xorq	%rbx,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	24(%rsp),%r8+	movq	%r9,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%rdx,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%r10,%rdi+	addq	%r12,%r8+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm1,%xmm1+	xorq	%r9,%r14+	addq	%r13,%r8+	vpaddq	-96(%rsi),%xmm1,%xmm10+	xorq	%r10,%r15+	shrdq	$28,%r14,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	vmovdqa	%xmm10,16(%rsp)+	vpalignr	$8,%xmm2,%xmm3,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%r8+	vpalignr	$8,%xmm6,%xmm7,%xmm11+	movq	%rbx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%rax,%r13+	xorq	%rcx,%r12+	vpaddq	%xmm11,%xmm2,%xmm2+	shrdq	$4,%r13,%r13+	xorq	%r8,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%rax,%r12+	xorq	%rax,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	32(%rsp),%rdx+	movq	%r8,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%rcx,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%r9,%r15+	addq	%r12,%rdx+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%r8,%r14+	addq	%r13,%rdx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r9,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm1,%xmm11+	addq	%rdx,%r11+	addq	%rdi,%rdx+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%r11,%r13+	addq	%rdx,%r14+	vpsllq	$3,%xmm1,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%rdx+	vpaddq	%xmm8,%xmm2,%xmm2+	movq	%rax,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm1,%xmm9+	xorq	%r11,%r13+	xorq	%rbx,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%rdx,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%r11,%r12+	xorq	%r11,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	40(%rsp),%rcx+	movq	%rdx,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%rbx,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%r8,%rdi+	addq	%r12,%rcx+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm2,%xmm2+	xorq	%rdx,%r14+	addq	%r13,%rcx+	vpaddq	-64(%rsi),%xmm2,%xmm10+	xorq	%r8,%r15+	shrdq	$28,%r14,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	vmovdqa	%xmm10,32(%rsp)+	vpalignr	$8,%xmm3,%xmm4,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%rcx+	vpalignr	$8,%xmm7,%xmm0,%xmm11+	movq	%r11,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%r10,%r13+	xorq	%rax,%r12+	vpaddq	%xmm11,%xmm3,%xmm3+	shrdq	$4,%r13,%r13+	xorq	%rcx,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%r10,%r12+	xorq	%r10,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	48(%rsp),%rbx+	movq	%rcx,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%rax,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%rdx,%r15+	addq	%r12,%rbx+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%rcx,%r14+	addq	%r13,%rbx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rdx,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm2,%xmm11+	addq	%rbx,%r9+	addq	%rdi,%rbx+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%r9,%r13+	addq	%rbx,%r14+	vpsllq	$3,%xmm2,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%rbx+	vpaddq	%xmm8,%xmm3,%xmm3+	movq	%r10,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm2,%xmm9+	xorq	%r9,%r13+	xorq	%r11,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%rbx,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%r9,%r12+	xorq	%r9,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	56(%rsp),%rax+	movq	%rbx,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%r11,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%rcx,%rdi+	addq	%r12,%rax+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm3,%xmm3+	xorq	%rbx,%r14+	addq	%r13,%rax+	vpaddq	-32(%rsi),%xmm3,%xmm10+	xorq	%rcx,%r15+	shrdq	$28,%r14,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	vmovdqa	%xmm10,48(%rsp)+	vpalignr	$8,%xmm4,%xmm5,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%rax+	vpalignr	$8,%xmm0,%xmm1,%xmm11+	movq	%r9,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%r8,%r13+	xorq	%r10,%r12+	vpaddq	%xmm11,%xmm4,%xmm4+	shrdq	$4,%r13,%r13+	xorq	%rax,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%r8,%r12+	xorq	%r8,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	64(%rsp),%r11+	movq	%rax,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%r10,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%rbx,%r15+	addq	%r12,%r11+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%rax,%r14+	addq	%r13,%r11+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rbx,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm3,%xmm11+	addq	%r11,%rdx+	addq	%rdi,%r11+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%rdx,%r13+	addq	%r11,%r14+	vpsllq	$3,%xmm3,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%r11+	vpaddq	%xmm8,%xmm4,%xmm4+	movq	%r8,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm3,%xmm9+	xorq	%rdx,%r13+	xorq	%r9,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%r11,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%rdx,%r12+	xorq	%rdx,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	72(%rsp),%r10+	movq	%r11,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%r9,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%rax,%rdi+	addq	%r12,%r10+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm4,%xmm4+	xorq	%r11,%r14+	addq	%r13,%r10+	vpaddq	0(%rsi),%xmm4,%xmm10+	xorq	%rax,%r15+	shrdq	$28,%r14,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	vmovdqa	%xmm10,64(%rsp)+	vpalignr	$8,%xmm5,%xmm6,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%r10+	vpalignr	$8,%xmm1,%xmm2,%xmm11+	movq	%rdx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%rcx,%r13+	xorq	%r8,%r12+	vpaddq	%xmm11,%xmm5,%xmm5+	shrdq	$4,%r13,%r13+	xorq	%r10,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%rcx,%r12+	xorq	%rcx,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	80(%rsp),%r9+	movq	%r10,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%r8,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%r11,%r15+	addq	%r12,%r9+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%r10,%r14+	addq	%r13,%r9+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r11,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm4,%xmm11+	addq	%r9,%rbx+	addq	%rdi,%r9+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%rbx,%r13+	addq	%r9,%r14+	vpsllq	$3,%xmm4,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%r9+	vpaddq	%xmm8,%xmm5,%xmm5+	movq	%rcx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm4,%xmm9+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%r9,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%rbx,%r12+	xorq	%rbx,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	88(%rsp),%r8+	movq	%r9,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%rdx,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%r10,%rdi+	addq	%r12,%r8+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm5,%xmm5+	xorq	%r9,%r14+	addq	%r13,%r8+	vpaddq	32(%rsi),%xmm5,%xmm10+	xorq	%r10,%r15+	shrdq	$28,%r14,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	vmovdqa	%xmm10,80(%rsp)+	vpalignr	$8,%xmm6,%xmm7,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%r8+	vpalignr	$8,%xmm2,%xmm3,%xmm11+	movq	%rbx,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%rax,%r13+	xorq	%rcx,%r12+	vpaddq	%xmm11,%xmm6,%xmm6+	shrdq	$4,%r13,%r13+	xorq	%r8,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%rax,%r12+	xorq	%rax,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	96(%rsp),%rdx+	movq	%r8,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%rcx,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%r9,%r15+	addq	%r12,%rdx+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%r8,%r14+	addq	%r13,%rdx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%r9,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm5,%xmm11+	addq	%rdx,%r11+	addq	%rdi,%rdx+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%r11,%r13+	addq	%rdx,%r14+	vpsllq	$3,%xmm5,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%rdx+	vpaddq	%xmm8,%xmm6,%xmm6+	movq	%rax,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm5,%xmm9+	xorq	%r11,%r13+	xorq	%rbx,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%rdx,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%r11,%r12+	xorq	%r11,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	104(%rsp),%rcx+	movq	%rdx,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%rbx,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%r8,%rdi+	addq	%r12,%rcx+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm6,%xmm6+	xorq	%rdx,%r14+	addq	%r13,%rcx+	vpaddq	64(%rsi),%xmm6,%xmm10+	xorq	%r8,%r15+	shrdq	$28,%r14,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	vmovdqa	%xmm10,96(%rsp)+	vpalignr	$8,%xmm7,%xmm0,%xmm8+	shrdq	$23,%r13,%r13+	movq	%r14,%rcx+	vpalignr	$8,%xmm3,%xmm4,%xmm11+	movq	%r11,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$1,%xmm8,%xmm10+	xorq	%r10,%r13+	xorq	%rax,%r12+	vpaddq	%xmm11,%xmm7,%xmm7+	shrdq	$4,%r13,%r13+	xorq	%rcx,%r14+	vpsrlq	$7,%xmm8,%xmm11+	andq	%r10,%r12+	xorq	%r10,%r13+	vpsllq	$56,%xmm8,%xmm9+	addq	112(%rsp),%rbx+	movq	%rcx,%r15+	vpxor	%xmm10,%xmm11,%xmm8+	xorq	%rax,%r12+	shrdq	$6,%r14,%r14+	vpsrlq	$7,%xmm10,%xmm10+	xorq	%rdx,%r15+	addq	%r12,%rbx+	vpxor	%xmm9,%xmm8,%xmm8+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	vpsllq	$7,%xmm9,%xmm9+	xorq	%rcx,%r14+	addq	%r13,%rbx+	vpxor	%xmm10,%xmm8,%xmm8+	xorq	%rdx,%rdi+	shrdq	$28,%r14,%r14+	vpsrlq	$6,%xmm6,%xmm11+	addq	%rbx,%r9+	addq	%rdi,%rbx+	vpxor	%xmm9,%xmm8,%xmm8+	movq	%r9,%r13+	addq	%rbx,%r14+	vpsllq	$3,%xmm6,%xmm10+	shrdq	$23,%r13,%r13+	movq	%r14,%rbx+	vpaddq	%xmm8,%xmm7,%xmm7+	movq	%r10,%r12+	shrdq	$5,%r14,%r14+	vpsrlq	$19,%xmm6,%xmm9+	xorq	%r9,%r13+	xorq	%r11,%r12+	vpxor	%xmm10,%xmm11,%xmm11+	shrdq	$4,%r13,%r13+	xorq	%rbx,%r14+	vpsllq	$42,%xmm10,%xmm10+	andq	%r9,%r12+	xorq	%r9,%r13+	vpxor	%xmm9,%xmm11,%xmm11+	addq	120(%rsp),%rax+	movq	%rbx,%rdi+	vpsrlq	$42,%xmm9,%xmm9+	xorq	%r11,%r12+	shrdq	$6,%r14,%r14+	vpxor	%xmm10,%xmm11,%xmm11+	xorq	%rcx,%rdi+	addq	%r12,%rax+	vpxor	%xmm9,%xmm11,%xmm11+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	vpaddq	%xmm11,%xmm7,%xmm7+	xorq	%rbx,%r14+	addq	%r13,%rax+	vpaddq	96(%rsi),%xmm7,%xmm10+	xorq	%rcx,%r15+	shrdq	$28,%r14,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	vmovdqa	%xmm10,112(%rsp)+	cmpb	$0,135(%rsi)+	jne	.Lavx_00_47+	shrdq	$23,%r13,%r13+	movq	%r14,%rax+	movq	%r9,%r12+	shrdq	$5,%r14,%r14+	xorq	%r8,%r13+	xorq	%r10,%r12+	shrdq	$4,%r13,%r13+	xorq	%rax,%r14+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	0(%rsp),%r11+	movq	%rax,%r15+	xorq	%r10,%r12+	shrdq	$6,%r14,%r14+	xorq	%rbx,%r15+	addq	%r12,%r11+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%rax,%r14+	addq	%r13,%r11+	xorq	%rbx,%rdi+	shrdq	$28,%r14,%r14+	addq	%r11,%rdx+	addq	%rdi,%r11+	movq	%rdx,%r13+	addq	%r11,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r11+	movq	%r8,%r12+	shrdq	$5,%r14,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	shrdq	$4,%r13,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	addq	8(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	shrdq	$6,%r14,%r14+	xorq	%rax,%rdi+	addq	%r12,%r10+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	shrdq	$28,%r14,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r10+	movq	%rdx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rcx,%r13+	xorq	%r8,%r12+	shrdq	$4,%r13,%r13+	xorq	%r10,%r14+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	16(%rsp),%r9+	movq	%r10,%r15+	xorq	%r8,%r12+	shrdq	$6,%r14,%r14+	xorq	%r11,%r15+	addq	%r12,%r9+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%r10,%r14+	addq	%r13,%r9+	xorq	%r11,%rdi+	shrdq	$28,%r14,%r14+	addq	%r9,%rbx+	addq	%rdi,%r9+	movq	%rbx,%r13+	addq	%r9,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r9+	movq	%rcx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	shrdq	$4,%r13,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	addq	24(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r10,%rdi+	addq	%r12,%r8+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	shrdq	$28,%r14,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r8+	movq	%rbx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rax,%r13+	xorq	%rcx,%r12+	shrdq	$4,%r13,%r13+	xorq	%r8,%r14+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	32(%rsp),%rdx+	movq	%r8,%r15+	xorq	%rcx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r9,%r15+	addq	%r12,%rdx+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%r8,%r14+	addq	%r13,%rdx+	xorq	%r9,%rdi+	shrdq	$28,%r14,%r14+	addq	%rdx,%r11+	addq	%rdi,%rdx+	movq	%r11,%r13+	addq	%rdx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rdx+	movq	%rax,%r12+	shrdq	$5,%r14,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	shrdq	$4,%r13,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	addq	40(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r8,%rdi+	addq	%r12,%rcx+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	shrdq	$28,%r14,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rcx+	movq	%r11,%r12+	shrdq	$5,%r14,%r14+	xorq	%r10,%r13+	xorq	%rax,%r12+	shrdq	$4,%r13,%r13+	xorq	%rcx,%r14+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	48(%rsp),%rbx+	movq	%rcx,%r15+	xorq	%rax,%r12+	shrdq	$6,%r14,%r14+	xorq	%rdx,%r15+	addq	%r12,%rbx+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%rcx,%r14+	addq	%r13,%rbx+	xorq	%rdx,%rdi+	shrdq	$28,%r14,%r14+	addq	%rbx,%r9+	addq	%rdi,%rbx+	movq	%r9,%r13+	addq	%rbx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rbx+	movq	%r10,%r12+	shrdq	$5,%r14,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	shrdq	$4,%r13,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	addq	56(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	shrdq	$6,%r14,%r14+	xorq	%rcx,%rdi+	addq	%r12,%rax+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	shrdq	$28,%r14,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rax+	movq	%r9,%r12+	shrdq	$5,%r14,%r14+	xorq	%r8,%r13+	xorq	%r10,%r12+	shrdq	$4,%r13,%r13+	xorq	%rax,%r14+	andq	%r8,%r12+	xorq	%r8,%r13+	addq	64(%rsp),%r11+	movq	%rax,%r15+	xorq	%r10,%r12+	shrdq	$6,%r14,%r14+	xorq	%rbx,%r15+	addq	%r12,%r11+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%rax,%r14+	addq	%r13,%r11+	xorq	%rbx,%rdi+	shrdq	$28,%r14,%r14+	addq	%r11,%rdx+	addq	%rdi,%r11+	movq	%rdx,%r13+	addq	%r11,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r11+	movq	%r8,%r12+	shrdq	$5,%r14,%r14+	xorq	%rdx,%r13+	xorq	%r9,%r12+	shrdq	$4,%r13,%r13+	xorq	%r11,%r14+	andq	%rdx,%r12+	xorq	%rdx,%r13+	addq	72(%rsp),%r10+	movq	%r11,%rdi+	xorq	%r9,%r12+	shrdq	$6,%r14,%r14+	xorq	%rax,%rdi+	addq	%r12,%r10+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%r11,%r14+	addq	%r13,%r10+	xorq	%rax,%r15+	shrdq	$28,%r14,%r14+	addq	%r10,%rcx+	addq	%r15,%r10+	movq	%rcx,%r13+	addq	%r10,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r10+	movq	%rdx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rcx,%r13+	xorq	%r8,%r12+	shrdq	$4,%r13,%r13+	xorq	%r10,%r14+	andq	%rcx,%r12+	xorq	%rcx,%r13+	addq	80(%rsp),%r9+	movq	%r10,%r15+	xorq	%r8,%r12+	shrdq	$6,%r14,%r14+	xorq	%r11,%r15+	addq	%r12,%r9+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%r10,%r14+	addq	%r13,%r9+	xorq	%r11,%rdi+	shrdq	$28,%r14,%r14+	addq	%r9,%rbx+	addq	%rdi,%r9+	movq	%rbx,%r13+	addq	%r9,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r9+	movq	%rcx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rbx,%r13+	xorq	%rdx,%r12+	shrdq	$4,%r13,%r13+	xorq	%r9,%r14+	andq	%rbx,%r12+	xorq	%rbx,%r13+	addq	88(%rsp),%r8+	movq	%r9,%rdi+	xorq	%rdx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r10,%rdi+	addq	%r12,%r8+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%r9,%r14+	addq	%r13,%r8+	xorq	%r10,%r15+	shrdq	$28,%r14,%r14+	addq	%r8,%rax+	addq	%r15,%r8+	movq	%rax,%r13+	addq	%r8,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%r8+	movq	%rbx,%r12+	shrdq	$5,%r14,%r14+	xorq	%rax,%r13+	xorq	%rcx,%r12+	shrdq	$4,%r13,%r13+	xorq	%r8,%r14+	andq	%rax,%r12+	xorq	%rax,%r13+	addq	96(%rsp),%rdx+	movq	%r8,%r15+	xorq	%rcx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r9,%r15+	addq	%r12,%rdx+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%r8,%r14+	addq	%r13,%rdx+	xorq	%r9,%rdi+	shrdq	$28,%r14,%r14+	addq	%rdx,%r11+	addq	%rdi,%rdx+	movq	%r11,%r13+	addq	%rdx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rdx+	movq	%rax,%r12+	shrdq	$5,%r14,%r14+	xorq	%r11,%r13+	xorq	%rbx,%r12+	shrdq	$4,%r13,%r13+	xorq	%rdx,%r14+	andq	%r11,%r12+	xorq	%r11,%r13+	addq	104(%rsp),%rcx+	movq	%rdx,%rdi+	xorq	%rbx,%r12+	shrdq	$6,%r14,%r14+	xorq	%r8,%rdi+	addq	%r12,%rcx+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%rdx,%r14+	addq	%r13,%rcx+	xorq	%r8,%r15+	shrdq	$28,%r14,%r14+	addq	%rcx,%r10+	addq	%r15,%rcx+	movq	%r10,%r13+	addq	%rcx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rcx+	movq	%r11,%r12+	shrdq	$5,%r14,%r14+	xorq	%r10,%r13+	xorq	%rax,%r12+	shrdq	$4,%r13,%r13+	xorq	%rcx,%r14+	andq	%r10,%r12+	xorq	%r10,%r13+	addq	112(%rsp),%rbx+	movq	%rcx,%r15+	xorq	%rax,%r12+	shrdq	$6,%r14,%r14+	xorq	%rdx,%r15+	addq	%r12,%rbx+	shrdq	$14,%r13,%r13+	andq	%r15,%rdi+	xorq	%rcx,%r14+	addq	%r13,%rbx+	xorq	%rdx,%rdi+	shrdq	$28,%r14,%r14+	addq	%rbx,%r9+	addq	%rdi,%rbx+	movq	%r9,%r13+	addq	%rbx,%r14+	shrdq	$23,%r13,%r13+	movq	%r14,%rbx+	movq	%r10,%r12+	shrdq	$5,%r14,%r14+	xorq	%r9,%r13+	xorq	%r11,%r12+	shrdq	$4,%r13,%r13+	xorq	%rbx,%r14+	andq	%r9,%r12+	xorq	%r9,%r13+	addq	120(%rsp),%rax+	movq	%rbx,%rdi+	xorq	%r11,%r12+	shrdq	$6,%r14,%r14+	xorq	%rcx,%rdi+	addq	%r12,%rax+	shrdq	$14,%r13,%r13+	andq	%rdi,%r15+	xorq	%rbx,%r14+	addq	%r13,%rax+	xorq	%rcx,%r15+	shrdq	$28,%r14,%r14+	addq	%rax,%r8+	addq	%r15,%rax+	movq	%r8,%r13+	addq	%rax,%r14+	movq	-64(%rbp),%rdi+	movq	%r14,%rax+	movq	-56(%rbp),%rsi++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	addq	48(%rdi),%r10+	addq	56(%rdi),%r11++	leaq	128(%rsi),%rsi+	cmpq	-48(%rbp),%rsi++	movq	%rax,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)+	jb	.Lloop_avx++	vzeroupper+	movaps	-160(%rbp),%xmm6+	movaps	-144(%rbp),%xmm7+	movaps	-128(%rbp),%xmm8+	movaps	-112(%rbp),%xmm9+	movaps	-96(%rbp),%xmm10+	movaps	-80(%rbp),%xmm11+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp++	popq	%rbp++.LSEH_epilogue_crypton_sha512_asm_block_data_order_avx:+	mov	8(%rsp),%rdi+	mov	16(%rsp),%rsi++	.byte	0xf3,0xc3++.LSEH_end_crypton_sha512_asm_block_data_order_avx:+.def	crypton_sha512_asm_block_data_order_avx2;	.scl 3;	.type 32;	.endef+.p2align	6+crypton_sha512_asm_block_data_order_avx2:+	.byte	0xf3,0x0f,0x1e,0xfa+	movq	%rdi,8(%rsp)+	movq	%rsi,16(%rsp)+	movq	%rsp,%r11+.LSEH_begin_crypton_sha512_asm_block_data_order_avx2:+++	pushq	%rbp++	movq	%rsp,%rbp++	movq	%rcx,%rdi+	movq	%rdx,%rsi+	movq	%r8,%rdx+.Lavx2_shortcut:+	pushq	%rbx++	pushq	%r12++	pushq	%r13++	pushq	%r14++	pushq	%r15++	shlq	$4,%rdx+	subq	$120,%rsp++	leaq	(%rsi,%rdx,8),%rdx+	movq	%rdi,-64(%rbp)++	movq	%rdx,-48(%rbp)+	movaps	%xmm6,-160(%rbp)+	movaps	%xmm7,-144(%rbp)+	movaps	%xmm8,-128(%rbp)+	movaps	%xmm9,-112(%rbp)++	movaps	%xmm10,-96(%rbp)+	movaps	%xmm11,-80(%rbp)++.LSEH_body_crypton_sha512_asm_block_data_order_avx2:+++	leaq	-128(%rsp),%rsp+	vzeroupper+	andq	$-128,%rsp+	subq	$-128,%rsi+	movq	0(%rdi),%rax+	movq	%rsi,%r12+	movq	8(%rdi),%rbx+	cmpq	%rdx,%rsi+	movq	16(%rdi),%rcx+	cmoveq	%rsp,%r12+	movq	24(%rdi),%rdx+	movq	32(%rdi),%r8+	movq	40(%rdi),%r9+	movq	48(%rdi),%r10+	movq	56(%rdi),%r11+	jmp	.Loop_avx2+.p2align	4+.Loop_avx2:+	vmovdqa	K512+1280(%rip),%ymm10+	movq	%rsi,-56(%rbp)+	vmovdqu	-128(%rsi),%xmm0+	vmovdqu	-128+16(%rsi),%xmm1+	vmovdqu	-128+32(%rsi),%xmm2+	vmovdqu	-128+48(%rsi),%xmm3+	vmovdqu	-128+64(%rsi),%xmm4+	vmovdqu	-128+80(%rsi),%xmm5+	vmovdqu	-128+96(%rsi),%xmm6+	vmovdqu	-128+112(%rsi),%xmm7+	leaq	K512+128(%rip),%rsi+	vinserti128	$1,(%r12),%ymm0,%ymm0+	vinserti128	$1,16(%r12),%ymm1,%ymm1+	vpshufb	%ymm10,%ymm0,%ymm0+	vinserti128	$1,32(%r12),%ymm2,%ymm2+	vpshufb	%ymm10,%ymm1,%ymm1+	vinserti128	$1,48(%r12),%ymm3,%ymm3+	vpshufb	%ymm10,%ymm2,%ymm2+	vinserti128	$1,64(%r12),%ymm4,%ymm4+	vpshufb	%ymm10,%ymm3,%ymm3+	vinserti128	$1,80(%r12),%ymm5,%ymm5+	vpshufb	%ymm10,%ymm4,%ymm4+	vinserti128	$1,96(%r12),%ymm6,%ymm6+	vpshufb	%ymm10,%ymm5,%ymm5+	vinserti128	$1,112(%r12),%ymm7,%ymm7++	vpaddq	-128(%rsi),%ymm0,%ymm8+	vpshufb	%ymm10,%ymm6,%ymm6+	vpaddq	-96(%rsi),%ymm1,%ymm9+	vpshufb	%ymm10,%ymm7,%ymm7+	vpaddq	-64(%rsi),%ymm2,%ymm10+	vpaddq	-32(%rsi),%ymm3,%ymm11+	vmovdqa	%ymm8,0(%rsp)+	vpaddq	0(%rsi),%ymm4,%ymm8+	vmovdqa	%ymm9,32(%rsp)+	vpaddq	32(%rsi),%ymm5,%ymm9+	vmovdqa	%ymm10,64(%rsp)+	vpaddq	64(%rsi),%ymm6,%ymm10+	vmovdqa	%ymm11,96(%rsp)+	leaq	-128(%rsp),%rsp+	vpaddq	96(%rsi),%ymm7,%ymm11+	vmovdqa	%ymm8,0(%rsp)+	xorq	%r14,%r14+	vmovdqa	%ymm9,32(%rsp)+	movq	%rbx,%rdi+	vmovdqa	%ymm10,64(%rsp)+	xorq	%rcx,%rdi+	vmovdqa	%ymm11,96(%rsp)+	movq	%r9,%r12+	addq	$32*8,%rsi+	jmp	.Lavx2_00_47++.p2align	4+.Lavx2_00_47:+	leaq	-128(%rsp),%rsp+	vpalignr	$8,%ymm0,%ymm1,%ymm8+	addq	0+256(%rsp),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	vpalignr	$8,%ymm4,%ymm5,%ymm11+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	vpaddq	%ymm11,%ymm0,%ymm0+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	vpsrlq	$6,%ymm7,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	vpsllq	$3,%ymm7,%ymm10+	vpaddq	%ymm8,%ymm0,%ymm0+	addq	8+256(%rsp),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	vpsrlq	$19,%ymm7,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	vpaddq	%ymm11,%ymm0,%ymm0+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	vpaddq	-128(%rsi),%ymm0,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	vmovdqa	%ymm10,0(%rsp)+	vpalignr	$8,%ymm1,%ymm2,%ymm8+	addq	32+256(%rsp),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	vpalignr	$8,%ymm5,%ymm6,%ymm11+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	vpaddq	%ymm11,%ymm1,%ymm1+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	vpsrlq	$6,%ymm0,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	vpsllq	$3,%ymm0,%ymm10+	vpaddq	%ymm8,%ymm1,%ymm1+	addq	40+256(%rsp),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	vpsrlq	$19,%ymm0,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	vpaddq	%ymm11,%ymm1,%ymm1+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	vpaddq	-96(%rsi),%ymm1,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	vmovdqa	%ymm10,32(%rsp)+	vpalignr	$8,%ymm2,%ymm3,%ymm8+	addq	64+256(%rsp),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	vpalignr	$8,%ymm6,%ymm7,%ymm11+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	vpaddq	%ymm11,%ymm2,%ymm2+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	vpsrlq	$6,%ymm1,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	vpsllq	$3,%ymm1,%ymm10+	vpaddq	%ymm8,%ymm2,%ymm2+	addq	72+256(%rsp),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	vpsrlq	$19,%ymm1,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	vpaddq	%ymm11,%ymm2,%ymm2+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	vpaddq	-64(%rsi),%ymm2,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	vmovdqa	%ymm10,64(%rsp)+	vpalignr	$8,%ymm3,%ymm4,%ymm8+	addq	96+256(%rsp),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	vpalignr	$8,%ymm7,%ymm0,%ymm11+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	vpaddq	%ymm11,%ymm3,%ymm3+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	vpsrlq	$6,%ymm2,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	vpsllq	$3,%ymm2,%ymm10+	vpaddq	%ymm8,%ymm3,%ymm3+	addq	104+256(%rsp),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	vpsrlq	$19,%ymm2,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	vpaddq	%ymm11,%ymm3,%ymm3+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	vpaddq	-32(%rsi),%ymm3,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	vmovdqa	%ymm10,96(%rsp)+	leaq	-128(%rsp),%rsp+	vpalignr	$8,%ymm4,%ymm5,%ymm8+	addq	0+256(%rsp),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	vpalignr	$8,%ymm0,%ymm1,%ymm11+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	vpaddq	%ymm11,%ymm4,%ymm4+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	vpsrlq	$6,%ymm3,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	vpsllq	$3,%ymm3,%ymm10+	vpaddq	%ymm8,%ymm4,%ymm4+	addq	8+256(%rsp),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	vpsrlq	$19,%ymm3,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	vpaddq	%ymm11,%ymm4,%ymm4+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	vpaddq	0(%rsi),%ymm4,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	vmovdqa	%ymm10,0(%rsp)+	vpalignr	$8,%ymm5,%ymm6,%ymm8+	addq	32+256(%rsp),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	vpalignr	$8,%ymm1,%ymm2,%ymm11+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	vpaddq	%ymm11,%ymm5,%ymm5+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	vpsrlq	$6,%ymm4,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	vpsllq	$3,%ymm4,%ymm10+	vpaddq	%ymm8,%ymm5,%ymm5+	addq	40+256(%rsp),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	vpsrlq	$19,%ymm4,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	vpaddq	%ymm11,%ymm5,%ymm5+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	vpaddq	32(%rsi),%ymm5,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	vmovdqa	%ymm10,32(%rsp)+	vpalignr	$8,%ymm6,%ymm7,%ymm8+	addq	64+256(%rsp),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	vpalignr	$8,%ymm2,%ymm3,%ymm11+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	vpaddq	%ymm11,%ymm6,%ymm6+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	vpsrlq	$6,%ymm5,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	vpsllq	$3,%ymm5,%ymm10+	vpaddq	%ymm8,%ymm6,%ymm6+	addq	72+256(%rsp),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	vpsrlq	$19,%ymm5,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	vpaddq	%ymm11,%ymm6,%ymm6+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	vpaddq	64(%rsi),%ymm6,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	vmovdqa	%ymm10,64(%rsp)+	vpalignr	$8,%ymm7,%ymm0,%ymm8+	addq	96+256(%rsp),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	vpalignr	$8,%ymm3,%ymm4,%ymm11+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	vpsrlq	$1,%ymm8,%ymm10+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	vpaddq	%ymm11,%ymm7,%ymm7+	vpsrlq	$7,%ymm8,%ymm11+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	vpsllq	$56,%ymm8,%ymm9+	vpxor	%ymm10,%ymm11,%ymm8+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	vpsrlq	$7,%ymm10,%ymm10+	vpxor	%ymm9,%ymm8,%ymm8+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	vpsllq	$7,%ymm9,%ymm9+	vpxor	%ymm10,%ymm8,%ymm8+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	vpsrlq	$6,%ymm6,%ymm11+	vpxor	%ymm9,%ymm8,%ymm8+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	vpsllq	$3,%ymm6,%ymm10+	vpaddq	%ymm8,%ymm7,%ymm7+	addq	104+256(%rsp),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	vpsrlq	$19,%ymm6,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	vpsllq	$42,%ymm10,%ymm10+	vpxor	%ymm9,%ymm11,%ymm11+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	vpsrlq	$42,%ymm9,%ymm9+	vpxor	%ymm10,%ymm11,%ymm11+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	vpxor	%ymm9,%ymm11,%ymm11+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	vpaddq	%ymm11,%ymm7,%ymm7+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	vpaddq	96(%rsi),%ymm7,%ymm10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	vmovdqa	%ymm10,96(%rsp)+	leaq	256(%rsi),%rsi+	cmpb	$0,-121(%rsi)+	jne	.Lavx2_00_47+	addq	0+128(%rsp),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	addq	8+128(%rsp),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	addq	32+128(%rsp),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	addq	40+128(%rsp),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	addq	64+128(%rsp),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	addq	72+128(%rsp),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	addq	96+128(%rsp),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	addq	104+128(%rsp),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	addq	0(%rsp),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	addq	8(%rsp),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	addq	32(%rsp),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	addq	40(%rsp),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	addq	64(%rsp),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	addq	72(%rsp),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	addq	96(%rsp),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	addq	104(%rsp),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	movq	-64(%rbp),%rdi+	addq	%r14,%rax+	movq	-56(%rbp),%r12++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	addq	48(%rdi),%r10+	addq	56(%rdi),%r11++	movq	%rax,0(%rdi)+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)++	cmpq	-48(%rbp),%r12+	je	.Ldone_avx2++	leaq	1152(%rsp),%rsi+	xorq	%r14,%r14+	movq	%rbx,%rdi+	xorq	%rcx,%rdi+	movq	%r9,%r12+	jmp	.Lower_avx2+.p2align	4+.Lower_avx2:+	addq	0+16(%rsi),%r11+	andq	%r8,%r12+	rorxq	$41,%r8,%r13+	rorxq	$18,%r8,%r15+	leaq	(%rax,%r14,1),%rax+	leaq	(%r11,%r12,1),%r11+	andnq	%r10,%r8,%r12+	xorq	%r15,%r13+	rorxq	$14,%r8,%r14+	leaq	(%r11,%r12,1),%r11+	xorq	%r14,%r13+	movq	%rax,%r15+	rorxq	$39,%rax,%r12+	leaq	(%r11,%r13,1),%r11+	xorq	%rbx,%r15+	rorxq	$34,%rax,%r14+	rorxq	$28,%rax,%r13+	leaq	(%rdx,%r11,1),%rdx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rbx,%rdi+	xorq	%r13,%r14+	leaq	(%r11,%rdi,1),%r11+	movq	%r8,%r12+	addq	8+16(%rsi),%r10+	andq	%rdx,%r12+	rorxq	$41,%rdx,%r13+	rorxq	$18,%rdx,%rdi+	leaq	(%r11,%r14,1),%r11+	leaq	(%r10,%r12,1),%r10+	andnq	%r9,%rdx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rdx,%r14+	leaq	(%r10,%r12,1),%r10+	xorq	%r14,%r13+	movq	%r11,%rdi+	rorxq	$39,%r11,%r12+	leaq	(%r10,%r13,1),%r10+	xorq	%rax,%rdi+	rorxq	$34,%r11,%r14+	rorxq	$28,%r11,%r13+	leaq	(%rcx,%r10,1),%rcx+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rax,%r15+	xorq	%r13,%r14+	leaq	(%r10,%r15,1),%r10+	movq	%rdx,%r12+	addq	32+16(%rsi),%r9+	andq	%rcx,%r12+	rorxq	$41,%rcx,%r13+	rorxq	$18,%rcx,%r15+	leaq	(%r10,%r14,1),%r10+	leaq	(%r9,%r12,1),%r9+	andnq	%r8,%rcx,%r12+	xorq	%r15,%r13+	rorxq	$14,%rcx,%r14+	leaq	(%r9,%r12,1),%r9+	xorq	%r14,%r13+	movq	%r10,%r15+	rorxq	$39,%r10,%r12+	leaq	(%r9,%r13,1),%r9+	xorq	%r11,%r15+	rorxq	$34,%r10,%r14+	rorxq	$28,%r10,%r13+	leaq	(%rbx,%r9,1),%rbx+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r11,%rdi+	xorq	%r13,%r14+	leaq	(%r9,%rdi,1),%r9+	movq	%rcx,%r12+	addq	40+16(%rsi),%r8+	andq	%rbx,%r12+	rorxq	$41,%rbx,%r13+	rorxq	$18,%rbx,%rdi+	leaq	(%r9,%r14,1),%r9+	leaq	(%r8,%r12,1),%r8+	andnq	%rdx,%rbx,%r12+	xorq	%rdi,%r13+	rorxq	$14,%rbx,%r14+	leaq	(%r8,%r12,1),%r8+	xorq	%r14,%r13+	movq	%r9,%rdi+	rorxq	$39,%r9,%r12+	leaq	(%r8,%r13,1),%r8+	xorq	%r10,%rdi+	rorxq	$34,%r9,%r14+	rorxq	$28,%r9,%r13+	leaq	(%rax,%r8,1),%rax+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r10,%r15+	xorq	%r13,%r14+	leaq	(%r8,%r15,1),%r8+	movq	%rbx,%r12+	addq	64+16(%rsi),%rdx+	andq	%rax,%r12+	rorxq	$41,%rax,%r13+	rorxq	$18,%rax,%r15+	leaq	(%r8,%r14,1),%r8+	leaq	(%rdx,%r12,1),%rdx+	andnq	%rcx,%rax,%r12+	xorq	%r15,%r13+	rorxq	$14,%rax,%r14+	leaq	(%rdx,%r12,1),%rdx+	xorq	%r14,%r13+	movq	%r8,%r15+	rorxq	$39,%r8,%r12+	leaq	(%rdx,%r13,1),%rdx+	xorq	%r9,%r15+	rorxq	$34,%r8,%r14+	rorxq	$28,%r8,%r13+	leaq	(%r11,%rdx,1),%r11+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%r9,%rdi+	xorq	%r13,%r14+	leaq	(%rdx,%rdi,1),%rdx+	movq	%rax,%r12+	addq	72+16(%rsi),%rcx+	andq	%r11,%r12+	rorxq	$41,%r11,%r13+	rorxq	$18,%r11,%rdi+	leaq	(%rdx,%r14,1),%rdx+	leaq	(%rcx,%r12,1),%rcx+	andnq	%rbx,%r11,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r11,%r14+	leaq	(%rcx,%r12,1),%rcx+	xorq	%r14,%r13+	movq	%rdx,%rdi+	rorxq	$39,%rdx,%r12+	leaq	(%rcx,%r13,1),%rcx+	xorq	%r8,%rdi+	rorxq	$34,%rdx,%r14+	rorxq	$28,%rdx,%r13+	leaq	(%r10,%rcx,1),%r10+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%r8,%r15+	xorq	%r13,%r14+	leaq	(%rcx,%r15,1),%rcx+	movq	%r11,%r12+	addq	96+16(%rsi),%rbx+	andq	%r10,%r12+	rorxq	$41,%r10,%r13+	rorxq	$18,%r10,%r15+	leaq	(%rcx,%r14,1),%rcx+	leaq	(%rbx,%r12,1),%rbx+	andnq	%rax,%r10,%r12+	xorq	%r15,%r13+	rorxq	$14,%r10,%r14+	leaq	(%rbx,%r12,1),%rbx+	xorq	%r14,%r13+	movq	%rcx,%r15+	rorxq	$39,%rcx,%r12+	leaq	(%rbx,%r13,1),%rbx+	xorq	%rdx,%r15+	rorxq	$34,%rcx,%r14+	rorxq	$28,%rcx,%r13+	leaq	(%r9,%rbx,1),%r9+	andq	%r15,%rdi+	xorq	%r12,%r14+	xorq	%rdx,%rdi+	xorq	%r13,%r14+	leaq	(%rbx,%rdi,1),%rbx+	movq	%r10,%r12+	addq	104+16(%rsi),%rax+	andq	%r9,%r12+	rorxq	$41,%r9,%r13+	rorxq	$18,%r9,%rdi+	leaq	(%rbx,%r14,1),%rbx+	leaq	(%rax,%r12,1),%rax+	andnq	%r11,%r9,%r12+	xorq	%rdi,%r13+	rorxq	$14,%r9,%r14+	leaq	(%rax,%r12,1),%rax+	xorq	%r14,%r13+	movq	%rbx,%rdi+	rorxq	$39,%rbx,%r12+	leaq	(%rax,%r13,1),%rax+	xorq	%rcx,%rdi+	rorxq	$34,%rbx,%r14+	rorxq	$28,%rbx,%r13+	leaq	(%r8,%rax,1),%r8+	andq	%rdi,%r15+	xorq	%r12,%r14+	xorq	%rcx,%r15+	xorq	%r13,%r14+	leaq	(%rax,%r15,1),%rax+	movq	%r9,%r12+	leaq	-128(%rsi),%rsi+	cmpq	%rsp,%rsi+	jae	.Lower_avx2++	movq	-64(%rbp),%rdi+	addq	%r14,%rax+	movq	-56(%rbp),%rsi+	leaq	1152(%rsp),%rsp++	addq	0(%rdi),%rax+	addq	8(%rdi),%rbx+	addq	16(%rdi),%rcx+	addq	24(%rdi),%rdx+	addq	32(%rdi),%r8+	addq	40(%rdi),%r9+	leaq	256(%rsi),%rsi+	addq	48(%rdi),%r10+	movq	%rsi,%r12+	addq	56(%rdi),%r11+	cmpq	-48(%rbp),%rsi++	movq	%rax,0(%rdi)+	cmoveq	%rsp,%r12+	movq	%rbx,8(%rdi)+	movq	%rcx,16(%rdi)+	movq	%rdx,24(%rdi)+	movq	%r8,32(%rdi)+	movq	%r9,40(%rdi)+	movq	%r10,48(%rdi)+	movq	%r11,56(%rdi)++	jbe	.Loop_avx2++.Ldone_avx2:+	vzeroupper+	movaps	-160(%rbp),%xmm6+	movaps	-144(%rbp),%xmm7+	movaps	-128(%rbp),%xmm8+	movaps	-112(%rbp),%xmm9+	movaps	-96(%rbp),%xmm10+	movaps	-80(%rbp),%xmm11+	movq	-40(%rbp),%r15+	movq	-32(%rbp),%r14+	movq	-24(%rbp),%r13+	movq	-16(%rbp),%r12+	movq	-8(%rbp),%rbx+	movq	%rbp,%rsp++	popq	%rbp++.LSEH_epilogue_crypton_sha512_asm_block_data_order_avx2:+	mov	8(%rsp),%rdi+	mov	16(%rsp),%rsi++	.byte	0xf3,0xc3++.LSEH_end_crypton_sha512_asm_block_data_order_avx2:+.section	.pdata+.p2align	2+.rva	.LSEH_begin_crypton_sha512_asm_block_data_order+.rva	.LSEH_body_crypton_sha512_asm_block_data_order+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_prologue++.rva	.LSEH_body_crypton_sha512_asm_block_data_order+.rva	.LSEH_epilogue_crypton_sha512_asm_block_data_order+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_body++.rva	.LSEH_epilogue_crypton_sha512_asm_block_data_order+.rva	.LSEH_end_crypton_sha512_asm_block_data_order+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_epilogue++.rva	.LSEH_begin_crypton_sha512_asm_block_data_order_shaext+.rva	.LSEH_body_crypton_sha512_asm_block_data_order_shaext+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_shaext_prologue++.rva	.LSEH_body_crypton_sha512_asm_block_data_order_shaext+.rva	.LSEH_epilogue_crypton_sha512_asm_block_data_order_shaext+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_shaext_body++.rva	.LSEH_epilogue_crypton_sha512_asm_block_data_order_shaext+.rva	.LSEH_end_crypton_sha512_asm_block_data_order_shaext+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_shaext_epilogue++.rva	.LSEH_begin_crypton_sha512_asm_block_data_order_xop+.rva	.LSEH_body_crypton_sha512_asm_block_data_order_xop+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_xop_prologue++.rva	.LSEH_body_crypton_sha512_asm_block_data_order_xop+.rva	.LSEH_epilogue_crypton_sha512_asm_block_data_order_xop+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_xop_body++.rva	.LSEH_epilogue_crypton_sha512_asm_block_data_order_xop+.rva	.LSEH_end_crypton_sha512_asm_block_data_order_xop+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_xop_epilogue++.rva	.LSEH_begin_crypton_sha512_asm_block_data_order_avx+.rva	.LSEH_body_crypton_sha512_asm_block_data_order_avx+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_avx_prologue++.rva	.LSEH_body_crypton_sha512_asm_block_data_order_avx+.rva	.LSEH_epilogue_crypton_sha512_asm_block_data_order_avx+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_avx_body++.rva	.LSEH_epilogue_crypton_sha512_asm_block_data_order_avx+.rva	.LSEH_end_crypton_sha512_asm_block_data_order_avx+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_avx_epilogue++.rva	.LSEH_begin_crypton_sha512_asm_block_data_order_avx2+.rva	.LSEH_body_crypton_sha512_asm_block_data_order_avx2+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_avx2_prologue++.rva	.LSEH_body_crypton_sha512_asm_block_data_order_avx2+.rva	.LSEH_epilogue_crypton_sha512_asm_block_data_order_avx2+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_avx2_body++.rva	.LSEH_epilogue_crypton_sha512_asm_block_data_order_avx2+.rva	.LSEH_end_crypton_sha512_asm_block_data_order_avx2+.rva	.LSEH_info_crypton_sha512_asm_block_data_order_avx2_epilogue++.section	.xdata+.p2align	3+.LSEH_info_crypton_sha512_asm_block_data_order_prologue:+.byte	1,4,6,0x05+.byte	4,0x74,2,0+.byte	4,0x64,3,0+.byte	4,0x53+.byte	1,0x50+.long	0,0+.LSEH_info_crypton_sha512_asm_block_data_order_body:+.byte	1,0,18,0+.byte	0x00,0xf4,0x13,0x00+.byte	0x00,0xe4,0x14,0x00+.byte	0x00,0xd4,0x15,0x00+.byte	0x00,0xc4,0x16,0x00+.byte	0x00,0x34,0x17,0x00+.byte	0x00,0x54,0x18,0x00+.byte	0x00,0x74,0x1a,0x00+.byte	0x00,0x64,0x1b,0x00+.byte	0x00,0x01,0x19,0x00+.byte	0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00+.LSEH_info_crypton_sha512_asm_block_data_order_epilogue:+.byte	1,0,5,11+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0xb3+.byte	0x00,0x00,0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00++.LSEH_info_crypton_sha512_asm_block_data_order_shaext_prologue:+.byte	1,4,6,0x05+.byte	4,0x74,2,0+.byte	4,0x64,3,0+.byte	4,0x53+.byte	1,0x50+.long	0,0+.LSEH_info_crypton_sha512_asm_block_data_order_shaext_body:+.byte	1,0,17,85+.byte	0x00,0x68,0x00,0x00+.byte	0x00,0x78,0x01,0x00+.byte	0x00,0x88,0x02,0x00+.byte	0x00,0x98,0x03,0x00+.byte	0x00,0xa8,0x04,0x00+.byte	0x00,0x74,0x0c,0x00+.byte	0x00,0x64,0x0d,0x00+.byte	0x00,0x53+.byte	0x00,0x92+.byte	0x00,0x50+.byte	0x00,0x00,0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00+.LSEH_info_crypton_sha512_asm_block_data_order_shaext_epilogue:+.byte	1,0,4,0+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0x00,0x00,0x00++.LSEH_info_crypton_sha512_asm_block_data_order_xop_prologue:+.byte	1,4,6,0x05+.byte	4,0x74,2,0+.byte	4,0x64,3,0+.byte	4,0x53+.byte	1,0x50+.long	0,0+.LSEH_info_crypton_sha512_asm_block_data_order_xop_body:+.byte	1,0,30,165+.byte	0x00,0x68,0x00,0x00+.byte	0x00,0x78,0x01,0x00+.byte	0x00,0x88,0x02,0x00+.byte	0x00,0x98,0x03,0x00+.byte	0x00,0xa8,0x04,0x00+.byte	0x00,0xb8,0x05,0x00+.byte	0x00,0xf4,0x0f,0x00+.byte	0x00,0xe4,0x10,0x00+.byte	0x00,0xd4,0x11,0x00+.byte	0x00,0xc4,0x12,0x00+.byte	0x00,0x34,0x13,0x00+.byte	0x00,0x74,0x16,0x00+.byte	0x00,0x64,0x17,0x00+.byte	0x00,0x53+.byte	0x00,0x01,0x14,0x00+.byte	0x00,0x50+.byte	0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00+.LSEH_info_crypton_sha512_asm_block_data_order_xop_epilogue:+.byte	1,0,4,0+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0x00,0x00,0x00++.LSEH_info_crypton_sha512_asm_block_data_order_avx_prologue:+.byte	1,4,6,0x05+.byte	4,0x74,2,0+.byte	4,0x64,3,0+.byte	4,0x53+.byte	1,0x50+.long	0,0+.LSEH_info_crypton_sha512_asm_block_data_order_avx_body:+.byte	1,0,30,165+.byte	0x00,0x68,0x00,0x00+.byte	0x00,0x78,0x01,0x00+.byte	0x00,0x88,0x02,0x00+.byte	0x00,0x98,0x03,0x00+.byte	0x00,0xa8,0x04,0x00+.byte	0x00,0xb8,0x05,0x00+.byte	0x00,0xf4,0x0f,0x00+.byte	0x00,0xe4,0x10,0x00+.byte	0x00,0xd4,0x11,0x00+.byte	0x00,0xc4,0x12,0x00+.byte	0x00,0x34,0x13,0x00+.byte	0x00,0x74,0x16,0x00+.byte	0x00,0x64,0x17,0x00+.byte	0x00,0x53+.byte	0x00,0x01,0x14,0x00+.byte	0x00,0x50+.byte	0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00+.LSEH_info_crypton_sha512_asm_block_data_order_avx_epilogue:+.byte	1,0,4,0+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0x00,0x00,0x00++.LSEH_info_crypton_sha512_asm_block_data_order_avx2_prologue:+.byte	1,4,6,0x05+.byte	4,0x74,2,0+.byte	4,0x64,3,0+.byte	4,0x53+.byte	1,0x50+.long	0,0+.LSEH_info_crypton_sha512_asm_block_data_order_avx2_body:+.byte	1,0,30,165+.byte	0x00,0x68,0x00,0x00+.byte	0x00,0x78,0x01,0x00+.byte	0x00,0x88,0x02,0x00+.byte	0x00,0x98,0x03,0x00+.byte	0x00,0xa8,0x04,0x00+.byte	0x00,0xb8,0x05,0x00+.byte	0x00,0xf4,0x0f,0x00+.byte	0x00,0xe4,0x10,0x00+.byte	0x00,0xd4,0x11,0x00+.byte	0x00,0xc4,0x12,0x00+.byte	0x00,0x34,0x13,0x00+.byte	0x00,0x74,0x16,0x00+.byte	0x00,0x64,0x17,0x00+.byte	0x00,0x53+.byte	0x00,0x01,0x14,0x00+.byte	0x00,0x50+.byte	0x00,0x00,0x00,0x00+.byte	0x00,0x00,0x00,0x00+.LSEH_info_crypton_sha512_asm_block_data_order_avx2_epilogue:+.byte	1,0,4,0+.byte	0x00,0x74,0x01,0x00+.byte	0x00,0x64,0x02,0x00+.byte	0x00,0x00,0x00,0x00+
+ cbits/asm/sha512-x86_64.pl view
@@ -0,0 +1,2519 @@+#!/usr/bin/env perl+#+# ====================================================================+# Written by Andy Polyakov, @dot-asm, initially for the OpenSSL+# project.+# ====================================================================+#+# sha256/512_block procedure for x86_64.+#+# 40% improvement over compiler-generated code on Opteron. On EM64T+# sha256 was observed to run >80% faster and sha512 - >40%. No magical+# tricks, just straight implementation... I really wonder why gcc+# [being armed with inline assembler] fails to generate as fast code.+# The only thing which is cool about this module is that it's very+# same instruction sequence used for both SHA-256 and SHA-512. In+# former case the instructions operate on 32-bit operands, while in+# latter - on 64-bit ones. All I had to do is to get one flavor right,+# the other one passed the test right away:-)+#+# sha256_block runs in ~1005 cycles on Opteron, which gives you+# asymptotic performance of 64*1000/1005=63.7MBps times CPU clock+# frequency in GHz. sha512_block runs in ~1275 cycles, which results+# in 128*1000/1275=100MBps per GHz. Is there room for improvement?+# Well, if you compare it to IA-64 implementation, which maintains+# X[16] in register bank[!], tends to 4 instructions per CPU clock+# cycle and runs in 1003 cycles, 1275 is very good result for 3-way+# issue Opteron pipeline and X[16] maintained in memory. So that *if*+# there is a way to improve it, *then* the only way would be to try to+# offload X[16] updates to SSE unit, but that would require "deeper"+# loop unroll, which in turn would naturally cause size blow-up, not+# to mention increased complexity! And once again, only *if* it's+# actually possible to noticeably improve overall ILP, instruction+# level parallelism, on a given CPU implementation in this case.+#+# Special note on Intel EM64T. While Opteron CPU exhibits perfect+# performance ratio of 1.5 between 64- and 32-bit flavors [see above],+# [currently available] EM64T CPUs apparently are far from it. On the+# contrary, 64-bit version, sha512_block, is ~30% *slower* than 32-bit+# sha256_block:-( This is presumably because 64-bit shifts/rotates+# apparently are not atomic instructions, but implemented in microcode.+#+# May 2012.+#+# Optimization including one of Pavel Semjanov's ideas, alternative+# Maj, resulted in >=5% improvement on most CPUs, +20% SHA256 and+# unfortunately -2% SHA512 on P4 [which nobody should care about+# that much].+#+# June 2012.+#+# Add SIMD code paths, see below for improvement coefficients. SSSE3+# code path was not attempted for SHA512, because improvement is not+# estimated to be high enough, noticeably less than 9%, to justify+# the effort, not on pre-AVX processors. [Obviously with exclusion+# for VIA Nano, but it has SHA512 instruction that is faster and+# should be used instead.] For reference, corresponding estimated+# upper limit for improvement for SSSE3 SHA256 is 28%. The fact that+# higher coefficients are observed on VIA Nano and Bulldozer has more+# to do with specifics of their architecture [which is topic for+# separate discussion].+#+# November 2012.+#+# Add AVX2 code path. Two consecutive input blocks are loaded to+# 256-bit %ymm registers, with data from first block to least+# significant 128-bit halves and data from second to most significant.+# The data is then processed with same SIMD instruction sequence as+# for AVX, but with %ymm as operands. Side effect is increased stack+# frame, 448 additional bytes in SHA256 and 1152 in SHA512, and 1.2KB+# code size increase.+#+# March 2014.+#+# Add support for Intel SHA Extensions.+#+# October 2023.+#+# Add support for Intel SHA512 Extension.++######################################################################+# Current performance in cycles per processed byte (less is better):+#+#		SHA256	SSSE3       AVX/XOP(*)	    SHA512  AVX/XOP(*)+#+# AMD K8	14.9	-	    -		    9.57    -+# P4		17.3	-	    -		    30.8    -+# Core 2	15.6	13.8(+13%)  -		    9.97    -+# Westmere	14.8	12.3(+19%)  -		    9.58    -+# Sandy Bridge	17.4	14.2(+23%)  11.6(+50%(**))  11.2    8.10(+38%(**))+# Ivy Bridge	12.6	10.5(+20%)  10.3(+22%)	    8.17    7.22(+13%)+# Haswell	12.2	9.28(+31%)  7.80(+56%)	    7.66    5.40(+42%)+# Skylake	11.4	9.03(+26%)  7.70(+48%)      7.25    5.20(+40%)+# Cannon Lake	11.4	9.00(+27%)  3.55(+220%)     7.20    5.12(+41%)+# Rocket Lake	10.4	9.13(+14%)  2.43(+330%)     6.66    5.34(+25%)+# Bulldozer	21.1	13.6(+54%)  13.6(+54%(***)) 13.5    8.58(+57%)+# Ryzen		11.0	9.02(+22%)  2.05(+440%)     7.05    5.67(+20%)+# VIA Nano	23.0	16.5(+39%)  -		    14.7    -+# Atom		23.0	18.9(+22%)  -		    14.7    -+# Silvermont	27.4	20.6(+33%)  -               17.5    -+# Knights L	27.4	21.0(+30%)  19.6(+40%)	    17.5    12.8(+37%)+# Goldmont	18.9	14.3(+32%)  4.16(+350%)     12.0    -+#+# (*)	whichever best applicable, including SHAEXT;+# (**)	switch from ror to shrd stands for fair share of improvement;+# (***)	execution time is fully determined by remaining integer-only+#	part, body_00_15; reducing the amount of SIMD instructions+#	below certain limit makes no difference/sense; to conserve+#	space SHA256 XOP code path is therefore omitted;++$flavour = shift;+$output  = pop;+if ($flavour =~ /\./) { $output = $flavour; undef $flavour; }++$win64=0; $win64=1 if ($flavour =~ /[nm]asm|mingw64/ || $output =~ /\.asm$/);++$0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1;+( $xlate="${dir}x86_64-xlate.pl" and -f $xlate ) or+( $xlate="${dir}../../perlasm/x86_64-xlate.pl" and -f $xlate) or+die "can't locate x86_64-xlate.pl";++$avx=undef;+$shaext=1;	### set to zero if compiling for 1.0.1++if (!defined($avx) && $win64 && ($flavour =~ /nasm/ || $ENV{ASM} =~ /nasm/) &&+	   ($ENV{ASM} //= "nasm") &&+	   `"$ENV{ASM}" -v 2>&1` =~ /NASM version ([0-9]+\.[0-9]+)(?:\.([0-9]+))?/) {+	$avx = ($1>=2.09) + ($1>=2.10) + 2 * ($1>=2.12);+	$avx += 2 if ($1==2.11 && $2>=8);+}++if (!defined($avx) && $win64 && ($flavour =~ /masm/ || $ENV{ASM} =~ /ml64/) &&+	   ($ENV{ASM} //= "ml64") &&+	   `"$ENV{ASM}" 2>&1` =~ /Version ([0-9]+)\./) {+	$avx = ($1>=10) + ($1>=12) + 2 * ($1>=14);+}++$ENV{CC} //= "cc";+if (!defined($avx) && `$ENV{CC} -Wa,-v -c -o /dev/zero -x assembler /dev/null 2>&1`+		=~ /GNU assembler version ([0-9]+)\.([0-9]+)/) {+	my $ver = $1 + $2/100.0;	# 3.1->3.01, 3.10->3.10+	$avx = ($ver>=2.19) + ($ver>=2.22) + ($ver>=2.25) + ($ver>=2.26);+}++if (!defined($avx) && `$ENV{CC} -v 2>&1`+		=~ /((?:^clang|LLVM) version|.*based on LLVM) ([0-9]+)\.([0-9]+)/) {+	my $ver = $2 + $3/100.0;	# 3.1->3.01, 3.10->3.10+	$avx = ($ver>=3.0) + ($ver>3.0);+	$avx += 2*($ver>=7.0) if ($1 =~ /^clang/);+}++open STDOUT,"| \"$^X\" \"$xlate\" $flavour \"$output\"";++if ($output =~ /512/) {+	$func="sha512_block_data_order";+	$TABLE="K512";+	$SZ=8;+	@ROT=($A,$B,$C,$D,$E,$F,$G,$H)=("%rax","%rbx","%rcx","%rdx",+					"%r8", "%r9", "%r10","%r11");+	($T1,$a0,$a1,$a2,$a3)=("%r12","%r13","%r14","%r15","%rdi");+	@Sigma0=(28,34,39);+	@Sigma1=(14,18,41);+	@sigma0=(1,  8, 7);+	@sigma1=(19,61, 6);+	$rounds=80;+} else {+	$func="sha256_block_data_order";+	$TABLE="K256";+	$SZ=4;+	@ROT=($A,$B,$C,$D,$E,$F,$G,$H)=("%eax","%ebx","%ecx","%edx",+					"%r8d","%r9d","%r10d","%r11d");+	($T1,$a0,$a1,$a2,$a3)=("%r12d","%r13d","%r14d","%r15d","%edi");+	@Sigma0=( 2,13,22);+	@Sigma1=( 6,11,25);+	@sigma0=( 7,18, 3);+	@sigma1=(17,19,10);+	$rounds=64;+}++$ctx="%rdi";	# 1st arg, zapped by $a3+$inp="%rsi";	# 2nd arg+$Tbl="%rbp";++$_ctx="16*$SZ+0*8(%rsp)";+$_inp="16*$SZ+1*8(%rsp)";+$_end="16*$SZ+2*8(%rsp)";+$framesz="16*$SZ+3*8";+++sub ROUND_00_15()+{ my ($i,$a,$b,$c,$d,$e,$f,$g,$h) = @_;+  my $STRIDE=$SZ;+     $STRIDE += 16 if ($i%(16/$SZ)==(16/$SZ-1));++$code.=<<___;+	ror	\$`$Sigma1[2]-$Sigma1[1]`,$a0+	mov	$f,$a2++	xor	$e,$a0+	ror	\$`$Sigma0[2]-$Sigma0[1]`,$a1+	xor	$g,$a2			# f^g++	mov	$T1,`$SZ*($i&0xf)`(%rsp)+	xor	$a,$a1+	and	$e,$a2			# (f^g)&e++	ror	\$`$Sigma1[1]-$Sigma1[0]`,$a0+	add	$h,$T1			# T1+=h+	xor	$g,$a2			# Ch(e,f,g)=((f^g)&e)^g++	ror	\$`$Sigma0[1]-$Sigma0[0]`,$a1+	xor	$e,$a0+	add	$a2,$T1			# T1+=Ch(e,f,g)++	mov	$a,$a2+	add	($Tbl),$T1		# T1+=K[round]+	xor	$a,$a1++	xor	$b,$a2			# a^b, b^c in next round+	ror	\$$Sigma1[0],$a0	# Sigma1(e)+	mov	$b,$h++	and	$a2,$a3+	ror	\$$Sigma0[0],$a1	# Sigma0(a)+	add	$a0,$T1			# T1+=Sigma1(e)++	xor	$a3,$h			# h=Maj(a,b,c)=Ch(a^b,c,b)+	add	$T1,$d			# d+=T1+	add	$T1,$h			# h+=T1++	lea	$STRIDE($Tbl),$Tbl	# round+++___+$code.=<<___ if ($i<15);+	add	$a1,$h			# h+=Sigma0(a)+___+	($a2,$a3) = ($a3,$a2);+}++sub ROUND_16_XX()+{ my ($i,$a,$b,$c,$d,$e,$f,$g,$h) = @_;++$code.=<<___;+	mov	`$SZ*(($i+1)&0xf)`(%rsp),$a0+	mov	`$SZ*(($i+14)&0xf)`(%rsp),$a2++	mov	$a0,$T1+	ror	\$`$sigma0[1]-$sigma0[0]`,$a0+	add	$a1,$a			# modulo-scheduled h+=Sigma0(a)+	mov	$a2,$a1+	ror	\$`$sigma1[1]-$sigma1[0]`,$a2++	xor	$T1,$a0+	shr	\$$sigma0[2],$T1+	ror	\$$sigma0[0],$a0+	xor	$a1,$a2+	shr	\$$sigma1[2],$a1++	ror	\$$sigma1[0],$a2+	xor	$a0,$T1			# sigma0(X[(i+1)&0xf])+	xor	$a1,$a2			# sigma1(X[(i+14)&0xf])+	add	`$SZ*(($i+9)&0xf)`(%rsp),$T1++	add	`$SZ*($i&0xf)`(%rsp),$T1+	mov	$e,$a0+	add	$a2,$T1+	mov	$a,$a1+___+	&ROUND_00_15(@_);+}++$code=<<___;+.text++.extern	OPENSSL_ia32cap_P+.globl	$func+.type	$func,\@function,3,"unwind"+.align	16+$func:+.cfi_startproc+	push	%rbp+.cfi_push	%rbp+	mov	%rsp,%rbp+.cfi_def_cfa_register	%rbp+___+$code.=<<___ if ($SZ==4 || $avx);+	lea	OPENSSL_ia32cap_P(%rip),%rax+	mov	0(%rax),%r9d+	mov	4(%rax),%r10d+	mov	8(%rax),%eax+___+$code.=<<___ if ($SZ==4 && $shaext);+	test	\$`1<<29`,%eax		# check for SHA+	jnz	.Lshaext_shortcut+___+$code.=<<___ if ($avx && $SZ==8);+	test	\$`1<<11`,%r10d		# check for XOP+	jnz	.Lxop_shortcut+___+$code.=<<___ if ($avx>1);+	and	\$`1<<8|1<<5|1<<3`,%eax	# check for BMI2+AVX2+BMI1+	cmp	\$`1<<8|1<<5|1<<3`,%eax+	je	.Lavx2_shortcut+___+$code.=<<___ if ($avx);+	and	\$`1<<30`,%r9d		# mask "Intel CPU" bit+	and	\$`1<<28|1<<9`,%r10d	# mask AVX and SSSE3 bits+	or	%r9d,%r10d+	cmp	\$`1<<28|1<<9|1<<30`,%r10d+	je	.Lavx_shortcut+___+$code.=<<___ if ($SZ==4);+	test	\$`1<<9`,%r10d+	jnz	.Lssse3_shortcut+___+$code.=<<___;+	push	%rbx+.cfi_push	%rbx+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15+	shl	\$4,%rdx		# num*16+	sub	\$$framesz,%rsp+.cfi_alloca	$framesz+.cfi_def_cfa	%rsp+.cfi_end_prologue+	lea	($inp,%rdx,$SZ),%rdx	# inp+num*16*$SZ+	mov	$ctx,$_ctx		# save ctx, 1st arg+	mov	$inp,$_inp		# save inp, 2nd arh+	mov	%rdx,$_end		# save end pointer, "3rd" arg++	mov	$SZ*0($ctx),$A+	mov	$SZ*1($ctx),$B+	mov	$SZ*2($ctx),$C+	mov	$SZ*3($ctx),$D+	mov	$SZ*4($ctx),$E+	mov	$SZ*5($ctx),$F+	mov	$SZ*6($ctx),$G+	mov	$SZ*7($ctx),$H+	jmp	.Lloop++.align	16+.Lloop:+	mov	$B,$a3+	lea	$TABLE(%rip),$Tbl+	xor	$C,$a3			# magic+___+	for($i=0;$i<16;$i++) {+		$code.="	mov	$SZ*$i($inp),$T1\n";+		$code.="	mov	@ROT[4],$a0\n";+		$code.="	mov	@ROT[0],$a1\n";+		$code.="	bswap	$T1\n";+		&ROUND_00_15($i,@ROT);+		unshift(@ROT,pop(@ROT));+	}+$code.=<<___;+	jmp	.Lrounds_16_xx+.align	16+.Lrounds_16_xx:+___+	for(;$i<32;$i++) {+		&ROUND_16_XX($i,@ROT);+		unshift(@ROT,pop(@ROT));+	}++$code.=<<___;+	cmpb	\$0,`$SZ-1`($Tbl)+	jnz	.Lrounds_16_xx++	mov	$_ctx,$ctx+	add	$a1,$A			# modulo-scheduled h+=Sigma0(a)+	lea	16*$SZ($inp),$inp++	add	$SZ*0($ctx),$A+	add	$SZ*1($ctx),$B+	add	$SZ*2($ctx),$C+	add	$SZ*3($ctx),$D+	add	$SZ*4($ctx),$E+	add	$SZ*5($ctx),$F+	add	$SZ*6($ctx),$G+	add	$SZ*7($ctx),$H++	cmp	$_end,$inp++	mov	$A,$SZ*0($ctx)+	mov	$B,$SZ*1($ctx)+	mov	$C,$SZ*2($ctx)+	mov	$D,$SZ*3($ctx)+	mov	$E,$SZ*4($ctx)+	mov	$F,$SZ*5($ctx)+	mov	$G,$SZ*6($ctx)+	mov	$H,$SZ*7($ctx)+	jb	.Lloop++	lea	$framesz+6*8(%rsp),%r11+.cfi_def_cfa	%r11,8+	mov	$framesz(%rsp),%r15+	mov	-40(%r11),%r14+	mov	-32(%r11),%r13+	mov	-24(%r11),%r12+	mov	-16(%r11),%rbx+	mov	-8(%r11),%rbp+.cfi_epilogue+	lea	(%r11),%rsp+	ret+.cfi_endproc+.size	$func,.-$func+___++if ($SZ==4) {+$code.=<<___;+.align	64+.type	$TABLE,\@object+$TABLE:+	.long	0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5+	.long	0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5+	.long	0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5+	.long	0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5+	.long	0xd807aa98,0x12835b01,0x243185be,0x550c7dc3+	.long	0xd807aa98,0x12835b01,0x243185be,0x550c7dc3+	.long	0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174+	.long	0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174+	.long	0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc+	.long	0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc+	.long	0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da+	.long	0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da+	.long	0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7+	.long	0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7+	.long	0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967+	.long	0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967+	.long	0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13+	.long	0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13+	.long	0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85+	.long	0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85+	.long	0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3+	.long	0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3+	.long	0xd192e819,0xd6990624,0xf40e3585,0x106aa070+	.long	0xd192e819,0xd6990624,0xf40e3585,0x106aa070+	.long	0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5+	.long	0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5+	.long	0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3+	.long	0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3+	.long	0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208+	.long	0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208+	.long	0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2+	.long	0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2++	.long	0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f+	.long	0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f+	.long	0x03020100,0x0b0a0908,0xffffffff,0xffffffff+	.long	0x03020100,0x0b0a0908,0xffffffff,0xffffffff+	.long	0xffffffff,0xffffffff,0x03020100,0x0b0a0908+	.long	0xffffffff,0xffffffff,0x03020100,0x0b0a0908+	.asciz	"SHA256 block transform for x86_64, CRYPTOGAMS by \@dot-asm"+___+} else {+$code.=<<___;+.align	64+.type	$TABLE,\@object+$TABLE:+	.quad	0x428a2f98d728ae22,0x7137449123ef65cd+	.quad	0x428a2f98d728ae22,0x7137449123ef65cd+	.quad	0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc+	.quad	0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc+	.quad	0x3956c25bf348b538,0x59f111f1b605d019+	.quad	0x3956c25bf348b538,0x59f111f1b605d019+	.quad	0x923f82a4af194f9b,0xab1c5ed5da6d8118+	.quad	0x923f82a4af194f9b,0xab1c5ed5da6d8118+	.quad	0xd807aa98a3030242,0x12835b0145706fbe+	.quad	0xd807aa98a3030242,0x12835b0145706fbe+	.quad	0x243185be4ee4b28c,0x550c7dc3d5ffb4e2+	.quad	0x243185be4ee4b28c,0x550c7dc3d5ffb4e2+	.quad	0x72be5d74f27b896f,0x80deb1fe3b1696b1+	.quad	0x72be5d74f27b896f,0x80deb1fe3b1696b1+	.quad	0x9bdc06a725c71235,0xc19bf174cf692694+	.quad	0x9bdc06a725c71235,0xc19bf174cf692694+	.quad	0xe49b69c19ef14ad2,0xefbe4786384f25e3+	.quad	0xe49b69c19ef14ad2,0xefbe4786384f25e3+	.quad	0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65+	.quad	0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65+	.quad	0x2de92c6f592b0275,0x4a7484aa6ea6e483+	.quad	0x2de92c6f592b0275,0x4a7484aa6ea6e483+	.quad	0x5cb0a9dcbd41fbd4,0x76f988da831153b5+	.quad	0x5cb0a9dcbd41fbd4,0x76f988da831153b5+	.quad	0x983e5152ee66dfab,0xa831c66d2db43210+	.quad	0x983e5152ee66dfab,0xa831c66d2db43210+	.quad	0xb00327c898fb213f,0xbf597fc7beef0ee4+	.quad	0xb00327c898fb213f,0xbf597fc7beef0ee4+	.quad	0xc6e00bf33da88fc2,0xd5a79147930aa725+	.quad	0xc6e00bf33da88fc2,0xd5a79147930aa725+	.quad	0x06ca6351e003826f,0x142929670a0e6e70+	.quad	0x06ca6351e003826f,0x142929670a0e6e70+	.quad	0x27b70a8546d22ffc,0x2e1b21385c26c926+	.quad	0x27b70a8546d22ffc,0x2e1b21385c26c926+	.quad	0x4d2c6dfc5ac42aed,0x53380d139d95b3df+	.quad	0x4d2c6dfc5ac42aed,0x53380d139d95b3df+	.quad	0x650a73548baf63de,0x766a0abb3c77b2a8+	.quad	0x650a73548baf63de,0x766a0abb3c77b2a8+	.quad	0x81c2c92e47edaee6,0x92722c851482353b+	.quad	0x81c2c92e47edaee6,0x92722c851482353b+	.quad	0xa2bfe8a14cf10364,0xa81a664bbc423001+	.quad	0xa2bfe8a14cf10364,0xa81a664bbc423001+	.quad	0xc24b8b70d0f89791,0xc76c51a30654be30+	.quad	0xc24b8b70d0f89791,0xc76c51a30654be30+	.quad	0xd192e819d6ef5218,0xd69906245565a910+	.quad	0xd192e819d6ef5218,0xd69906245565a910+	.quad	0xf40e35855771202a,0x106aa07032bbd1b8+	.quad	0xf40e35855771202a,0x106aa07032bbd1b8+	.quad	0x19a4c116b8d2d0c8,0x1e376c085141ab53+	.quad	0x19a4c116b8d2d0c8,0x1e376c085141ab53+	.quad	0x2748774cdf8eeb99,0x34b0bcb5e19b48a8+	.quad	0x2748774cdf8eeb99,0x34b0bcb5e19b48a8+	.quad	0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb+	.quad	0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb+	.quad	0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3+	.quad	0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3+	.quad	0x748f82ee5defb2fc,0x78a5636f43172f60+	.quad	0x748f82ee5defb2fc,0x78a5636f43172f60+	.quad	0x84c87814a1f0ab72,0x8cc702081a6439ec+	.quad	0x84c87814a1f0ab72,0x8cc702081a6439ec+	.quad	0x90befffa23631e28,0xa4506cebde82bde9+	.quad	0x90befffa23631e28,0xa4506cebde82bde9+	.quad	0xbef9a3f7b2c67915,0xc67178f2e372532b+	.quad	0xbef9a3f7b2c67915,0xc67178f2e372532b+	.quad	0xca273eceea26619c,0xd186b8c721c0c207+	.quad	0xca273eceea26619c,0xd186b8c721c0c207+	.quad	0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178+	.quad	0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178+	.quad	0x06f067aa72176fba,0x0a637dc5a2c898a6+	.quad	0x06f067aa72176fba,0x0a637dc5a2c898a6+	.quad	0x113f9804bef90dae,0x1b710b35131c471b+	.quad	0x113f9804bef90dae,0x1b710b35131c471b+	.quad	0x28db77f523047d84,0x32caab7b40c72493+	.quad	0x28db77f523047d84,0x32caab7b40c72493+	.quad	0x3c9ebe0a15c9bebc,0x431d67c49c100d4c+	.quad	0x3c9ebe0a15c9bebc,0x431d67c49c100d4c+	.quad	0x4cc5d4becb3e42b6,0x597f299cfc657e2a+	.quad	0x4cc5d4becb3e42b6,0x597f299cfc657e2a+	.quad	0x5fcb6fab3ad6faec,0x6c44198c4a475817+	.quad	0x5fcb6fab3ad6faec,0x6c44198c4a475817++	.quad	0x0001020304050607,0x08090a0b0c0d0e0f+	.quad	0x0001020304050607,0x08090a0b0c0d0e0f++${TABLE}_nodup:+	.quad	0x428a2f98d728ae22,0x7137449123ef65cd+	.quad	0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc+	.quad	0x3956c25bf348b538,0x59f111f1b605d019+	.quad	0x923f82a4af194f9b,0xab1c5ed5da6d8118+	.quad	0xd807aa98a3030242,0x12835b0145706fbe+	.quad	0x243185be4ee4b28c,0x550c7dc3d5ffb4e2+	.quad	0x72be5d74f27b896f,0x80deb1fe3b1696b1+	.quad	0x9bdc06a725c71235,0xc19bf174cf692694+	.quad	0xe49b69c19ef14ad2,0xefbe4786384f25e3+	.quad	0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65+	.quad	0x2de92c6f592b0275,0x4a7484aa6ea6e483+	.quad	0x5cb0a9dcbd41fbd4,0x76f988da831153b5+	.quad	0x983e5152ee66dfab,0xa831c66d2db43210+	.quad	0xb00327c898fb213f,0xbf597fc7beef0ee4+	.quad	0xc6e00bf33da88fc2,0xd5a79147930aa725+	.quad	0x06ca6351e003826f,0x142929670a0e6e70+	.quad	0x27b70a8546d22ffc,0x2e1b21385c26c926+	.quad	0x4d2c6dfc5ac42aed,0x53380d139d95b3df+	.quad	0x650a73548baf63de,0x766a0abb3c77b2a8+	.quad	0x81c2c92e47edaee6,0x92722c851482353b+	.quad	0xa2bfe8a14cf10364,0xa81a664bbc423001+	.quad	0xc24b8b70d0f89791,0xc76c51a30654be30+	.quad	0xd192e819d6ef5218,0xd69906245565a910+	.quad	0xf40e35855771202a,0x106aa07032bbd1b8+	.quad	0x19a4c116b8d2d0c8,0x1e376c085141ab53+	.quad	0x2748774cdf8eeb99,0x34b0bcb5e19b48a8+	.quad	0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb+	.quad	0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3+	.quad	0x748f82ee5defb2fc,0x78a5636f43172f60+	.quad	0x84c87814a1f0ab72,0x8cc702081a6439ec+	.quad	0x90befffa23631e28,0xa4506cebde82bde9+	.quad	0xbef9a3f7b2c67915,0xc67178f2e372532b+	.quad	0xca273eceea26619c,0xd186b8c721c0c207+	.quad	0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178+	.quad	0x06f067aa72176fba,0x0a637dc5a2c898a6+	.quad	0x113f9804bef90dae,0x1b710b35131c471b+	.quad	0x28db77f523047d84,0x32caab7b40c72493+	.quad	0x3c9ebe0a15c9bebc,0x431d67c49c100d4c+	.quad	0x4cc5d4becb3e42b6,0x597f299cfc657e2a+	.quad	0x5fcb6fab3ad6faec,0x6c44198c4a475817++	.asciz	"SHA512 block transform for x86_64, CRYPTOGAMS by \@dot-asm"+___+}++######################################################################+# SIMD code paths+#+if ($SZ==4 && $shaext) {{{+######################################################################+# Intel SHA Extensions implementation of SHA256 update function.+#+my ($ctx,$inp,$num,$Tbl)=("%rdi","%rsi","%rdx","%rcx");++my ($Wi,$ABEF,$CDGH,$TMP,$BSWAP,$ABEF_SAVE,$CDGH_SAVE)=map("%xmm$_",(0..2,7..10));+my @MSG=map("%xmm$_",(3..6));++$code.=<<___;+.type	sha256_block_data_order_shaext,\@function,3,"unwind"+.align	64+sha256_block_data_order_shaext:+.cfi_startproc+	push	%rbp+.cfi_push	%rbp+	mov	%rsp,%rbp+.cfi_def_cfa_register	%rbp+.Lshaext_shortcut:+___+$code.=<<___ if ($win64);+	sub	\$0x50,%rsp+.cfi_alloca	0x50+	movaps	%xmm6,-0x50(%rbp)+	movaps	%xmm7,-0x40(%rbp)+	movaps	%xmm8,-0x30(%rbp)+	movaps	%xmm9,-0x20(%rbp)+	movaps	%xmm10,-0x10(%rbp)+.cfi_offset	%xmm6-%xmm10,-0x60+___+$code.=<<___;+.cfi_end_prologue+	lea		K256+0x80(%rip),$Tbl+	movdqu		($ctx),$ABEF		# DCBA+	movdqu		16($ctx),$CDGH		# HGFE+	movdqa		0x200-0x80($Tbl),$TMP	# byte swap mask++	pshufd		\$0x1b,$ABEF,$Wi	# ABCD+	pshufd		\$0xb1,$ABEF,$ABEF	# CDAB+	pshufd		\$0x1b,$CDGH,$CDGH	# EFGH+	movdqa		$TMP,$BSWAP		# offload+	palignr		\$8,$CDGH,$ABEF		# ABEF+	punpcklqdq	$Wi,$CDGH		# CDGH+	jmp		.Loop_shaext++.align	16+.Loop_shaext:+	movdqu		($inp),@MSG[0]+	movdqu		0x10($inp),@MSG[1]+	movdqu		0x20($inp),@MSG[2]+	pshufb		$TMP,@MSG[0]+	movdqu		0x30($inp),@MSG[3]++	movdqa		0*32-0x80($Tbl),$Wi+	paddd		@MSG[0],$Wi+	pshufb		$TMP,@MSG[1]+	movdqa		$CDGH,$CDGH_SAVE	# offload+	sha256rnds2	$ABEF,$CDGH		# 0-3+	pshufd		\$0x0e,$Wi,$Wi+	nop+	movdqa		$ABEF,$ABEF_SAVE	# offload+	sha256rnds2	$CDGH,$ABEF++	movdqa		1*32-0x80($Tbl),$Wi+	paddd		@MSG[1],$Wi+	pshufb		$TMP,@MSG[2]+	sha256rnds2	$ABEF,$CDGH		# 4-7+	pshufd		\$0x0e,$Wi,$Wi+	lea		0x40($inp),$inp+	sha256msg1	@MSG[1],@MSG[0]+	sha256rnds2	$CDGH,$ABEF++	movdqa		2*32-0x80($Tbl),$Wi+	paddd		@MSG[2],$Wi+	pshufb		$TMP,@MSG[3]+	sha256rnds2	$ABEF,$CDGH		# 8-11+	pshufd		\$0x0e,$Wi,$Wi+	movdqa		@MSG[3],$TMP+	palignr		\$4,@MSG[2],$TMP+	nop+	paddd		$TMP,@MSG[0]+	sha256msg1	@MSG[2],@MSG[1]+	sha256rnds2	$CDGH,$ABEF++	movdqa		3*32-0x80($Tbl),$Wi+	paddd		@MSG[3],$Wi+	sha256msg2	@MSG[3],@MSG[0]+	sha256rnds2	$ABEF,$CDGH		# 12-15+	pshufd		\$0x0e,$Wi,$Wi+	movdqa		@MSG[0],$TMP+	palignr		\$4,@MSG[3],$TMP+	nop+	paddd		$TMP,@MSG[1]+	sha256msg1	@MSG[3],@MSG[2]+	sha256rnds2	$CDGH,$ABEF+___+for($i=4;$i<16-3;$i++) {+$code.=<<___;+	movdqa		$i*32-0x80($Tbl),$Wi+	paddd		@MSG[0],$Wi+	sha256msg2	@MSG[0],@MSG[1]+	sha256rnds2	$ABEF,$CDGH		# 16-19...+	pshufd		\$0x0e,$Wi,$Wi+	movdqa		@MSG[1],$TMP+	palignr		\$4,@MSG[0],$TMP+	nop+	paddd		$TMP,@MSG[2]+	sha256msg1	@MSG[0],@MSG[3]+	sha256rnds2	$CDGH,$ABEF+___+	push(@MSG,shift(@MSG));+}+$code.=<<___;+	movdqa		13*32-0x80($Tbl),$Wi+	paddd		@MSG[0],$Wi+	sha256msg2	@MSG[0],@MSG[1]+	sha256rnds2	$ABEF,$CDGH		# 52-55+	pshufd		\$0x0e,$Wi,$Wi+	movdqa		@MSG[1],$TMP+	palignr		\$4,@MSG[0],$TMP+	sha256rnds2	$CDGH,$ABEF+	paddd		$TMP,@MSG[2]++	movdqa		14*32-0x80($Tbl),$Wi+	paddd		@MSG[1],$Wi+	sha256rnds2	$ABEF,$CDGH		# 56-59+	pshufd		\$0x0e,$Wi,$Wi+	sha256msg2	@MSG[1],@MSG[2]+	movdqa		$BSWAP,$TMP+	sha256rnds2	$CDGH,$ABEF++	movdqa		15*32-0x80($Tbl),$Wi+	paddd		@MSG[2],$Wi+	nop+	sha256rnds2	$ABEF,$CDGH		# 60-63+	pshufd		\$0x0e,$Wi,$Wi+	dec		$num+	nop+	sha256rnds2	$CDGH,$ABEF++	paddd		$CDGH_SAVE,$CDGH+	paddd		$ABEF_SAVE,$ABEF+	jnz		.Loop_shaext++	pshufd		\$0xb1,$CDGH,$CDGH	# DCHG+	pshufd		\$0x1b,$ABEF,$TMP	# FEBA+	pshufd		\$0xb1,$ABEF,$ABEF	# BAFE+	punpckhqdq	$CDGH,$ABEF		# DCBA+	palignr		\$8,$TMP,$CDGH		# HGFE++	movdqu	$ABEF,($ctx)+	movdqu	$CDGH,16($ctx)+___+$code.=<<___ if ($win64);+	movaps	-0x50(%rbp),%xmm6+	movaps	-0x40(%rbp),%xmm7+	movaps	-0x30(%rbp),%xmm8+	movaps	-0x20(%rbp),%xmm9+	movaps	-0x10(%rbp),%xmm10+	mov	%rbp,%rsp+___+$code.=<<___;+.cfi_def_cfa_register	%rsp+	pop	%rbp+.cfi_pop	%rbp+.cfi_epilogue+	ret+.cfi_endproc+.size	sha256_block_data_order_shaext,.-sha256_block_data_order_shaext+___+}}}+if ($SZ==8 && $shaext && $avx>1) {{{+######################################################################+# Intel SHA Extensions implementation of SHA512 update function.+#+my ($ctx,$inp,$num,$Tbl)=("%rdi","%rsi","%rdx","%rcx");++my ($Wi,$ABEF,$CDGH,$TMP,$BSWAP,$ABEF_SAVE,$CDGH_SAVE)=map("%ymm$_",(4..10));+my @MSG=map("%ymm$_",(0..3));++$code.=<<___;+.globl	sha512_block_data_order_shaext+.type	sha512_block_data_order_shaext,\@function,3,"unwind"+.align	64+sha512_block_data_order_shaext:+.cfi_startproc+	push	%rbp+.cfi_push	%rbp+	mov	%rsp,%rbp+.cfi_def_cfa_register	%rbp+.Lshaext_shortcut:+___+$code.=<<___ if ($win64);+	sub	\$0x50,%rsp+.cfi_alloca	0x50+	movaps	%xmm6,-0x50(%rbp)+	movaps	%xmm7,-0x40(%rbp)+	movaps	%xmm8,-0x30(%rbp)+	movaps	%xmm9,-0x20(%rbp)+	movaps	%xmm10,-0x10(%rbp)+.cfi_offset	%xmm6-%xmm10,-0x60+___+$code.=<<___;+.cfi_end_prologue+	lea		K512_nodup+0x80(%rip),$Tbl+	vmovdqu		($ctx),@MSG[0]				# DCBA+	vmovdqu		32($ctx),@MSG[1]			# HGFE+	vmovdqa		-0xa0($Tbl),$BSWAP++	vpermq		\$0b00011011,@MSG[0],@MSG[0]		# ABCD+	vpblendd	\$0b00001111,@MSG[1],@MSG[0],$ABEF	# ABFE+	vpblendd	\$0b00001111,@MSG[0],@MSG[1],$CDGH	# HGCD+	vpermq		\$0b11100001,$ABEF,$ABEF		# ABEF+	vpermq		\$0b01001011,$CDGH,$CDGH		# CDGH+	jmp		.Loop_shaext++.align	16+.Loop_shaext:+	vmovdqu		($inp),@MSG[0]+	vmovdqu		0x20($inp),@MSG[1]+	vmovdqu		0x40($inp),@MSG[2]+	vpshufb		$BSWAP,@MSG[0],@MSG[0]+	vmovdqu		0x60($inp),@MSG[3]++	vpaddq		0*32-0x80($Tbl),@MSG[0],$Wi+	vpshufb		$BSWAP,@MSG[1],@MSG[1]+	vmovdqa		$CDGH,$CDGH_SAVE			# offload+	vsha512rnds2	%x#$Wi,$ABEF,$CDGH			# 0-3+	vextracti128	\$1,$Wi,%x#$Wi+	vmovdqa		$ABEF,$ABEF_SAVE			# offload+	vsha512rnds2	%x#$Wi,$CDGH,$ABEF++	vpaddq		1*32-0x80($Tbl),@MSG[1],$Wi+	vpshufb		$BSWAP,@MSG[2],@MSG[2]+	vsha512rnds2	%x#$Wi,$ABEF,$CDGH			# 4-7+	vextracti128	\$1,$Wi,%x#$Wi+	lea		0x80($inp),$inp+	vsha512msg1	@MSG[1],@MSG[0]+	vsha512rnds2	%x#$Wi,$CDGH,$ABEF++	vpaddq		2*32-0x80($Tbl),@MSG[2],$Wi+	vpshufb		$BSWAP,@MSG[3],@MSG[3]+	vsha512rnds2	%x#$Wi,$ABEF,$CDGH			# 8-11+	vextracti128	\$1,$Wi,%x#$Wi+	vpblendd	\$0x03,@MSG[3],@MSG[2],$TMP+	vpermq		\$0x39,$TMP,$TMP+	vpaddq		$TMP,@MSG[0],@MSG[0]+	vsha512msg1	@MSG[2],@MSG[1]+	vsha512rnds2	%x#$Wi,$CDGH,$ABEF++	vpaddq		3*32-0x80($Tbl),@MSG[3],$Wi+	vsha512msg2	@MSG[3],@MSG[0]+	vsha512rnds2	%x#$Wi,$ABEF,$CDGH			# 12-15+	vextracti128	\$1,$Wi,%x#$Wi+	vpblendd	\$0x03,@MSG[0],@MSG[3],$TMP+	vpermq		\$0x39,$TMP,$TMP+	vpaddq		$TMP,@MSG[1],@MSG[1]+	vsha512msg1	@MSG[3],@MSG[2]+	vsha512rnds2	%x#$Wi,$CDGH,$ABEF+___+for($i=4;$i<20-3;$i++) {+$code.=<<___;+	vpaddq		$i*32-0x80($Tbl),@MSG[0],$Wi+	vsha512msg2	@MSG[0],@MSG[1]+	vsha512rnds2	%x#$Wi,$ABEF,$CDGH			# 16-19...+	vextracti128	\$1,$Wi,%x#$Wi+	vpblendd	\$0x03,@MSG[1],@MSG[0],$TMP+	vpermq		\$0x39,$TMP,$TMP+	vpaddq		$TMP,@MSG[2],@MSG[2]+	vsha512msg1	@MSG[0],@MSG[3]+	vsha512rnds2	%x#$Wi,$CDGH,$ABEF+___+	push(@MSG,shift(@MSG));+}+$code.=<<___;+	vpaddq		17*32-0x80($Tbl),@MSG[0],$Wi+	vsha512msg2	@MSG[0],@MSG[1]+	vsha512rnds2	%x#$Wi,$ABEF,$CDGH			# 68-71+	vextracti128	\$1,$Wi,%x#$Wi+	vpblendd	\$0x03,@MSG[1],@MSG[0],$TMP+	vpermq		\$0x39,$TMP,$TMP+	vsha512rnds2	%x#$Wi,$CDGH,$ABEF+	vpaddq		$TMP,@MSG[2],@MSG[2]++	vpaddq		18*32-0x80($Tbl),@MSG[1],$Wi+	vsha512rnds2	%x#$Wi,$ABEF,$CDGH			# 72-75+	vextracti128	\$1,$Wi,%x#$Wi+	vsha512msg2	@MSG[1],@MSG[2]+	vsha512rnds2	%x#$Wi,$CDGH,$ABEF++	vpaddq		19*32-0x80($Tbl),@MSG[2],$Wi+	vsha512rnds2	%x#$Wi,$ABEF,$CDGH			# 76-79+	vextracti128	\$1,$Wi,%x#$Wi+	dec		$num+	vsha512rnds2	%x#$Wi,$CDGH,$ABEF++	vpaddq		$CDGH_SAVE,$CDGH,$CDGH+	vpaddq		$ABEF_SAVE,$ABEF,$ABEF+	jnz		.Loop_shaext++	vpermq		\$0b01001011,$ABEF,$ABEF		# EFBA+	vpblendd	\$0b11110000,$CDGH,$ABEF,@MSG[0]	# CDBA+	vpblendd	\$0b11110000,$ABEF,$CDGH,@MSG[1]	# EFGH+	vpermq		\$0b10110100,@MSG[0],@MSG[0]		# DCBA+	vpermq		\$0b00011011,@MSG[1],@MSG[1]		# HGFE++	vmovdqu		@MSG[0],($ctx)+	vmovdqu		@MSG[1],32($ctx)++	vzeroupper+___+$code.=<<___ if ($win64);+	movaps	-0x50(%rbp),%xmm6+	movaps	-0x40(%rbp),%xmm7+	movaps	-0x30(%rbp),%xmm8+	movaps	-0x20(%rbp),%xmm9+	movaps	-0x10(%rbp),%xmm10+	mov	%rbp,%rsp+___+$code.=<<___;+.cfi_def_cfa_register	%rsp+	pop	%rbp+.cfi_pop	%rbp+.cfi_epilogue+	ret+.cfi_endproc+.size	sha512_block_data_order_shaext,.-sha512_block_data_order_shaext+___+}}}+{{{++my $a4=$T1;+my ($a,$b,$c,$d,$e,$f,$g,$h);++sub AUTOLOAD()		# thunk [simplified] 32-bit style perlasm+{ my $opcode = $AUTOLOAD; $opcode =~ s/.*:://;+  my $arg = pop;+    $arg = "\$$arg" if ($arg*1 eq $arg);+    $code .= "\t$opcode\t".join(',',$arg,reverse @_)."\n";+}++sub body_00_15 () {+	(+	'($a,$b,$c,$d,$e,$f,$g,$h)=@ROT;'.++	'&ror	($a0,$Sigma1[2]-$Sigma1[1])',+	'&mov	($a,$a1)',+	'&mov	($a4,$f)',++	'&ror	($a1,$Sigma0[2]-$Sigma0[1])',+	'&xor	($a0,$e)',+	'&xor	($a4,$g)',			# f^g++	'&ror	($a0,$Sigma1[1]-$Sigma1[0])',+	'&xor	($a1,$a)',+	'&and	($a4,$e)',			# (f^g)&e++	'&xor	($a0,$e)',+	'&add	($h,$SZ*($i&15)."(%rsp)")',	# h+=X[i]+K[i]+	'&mov	($a2,$a)',++	'&xor	($a4,$g)',			# Ch(e,f,g)=((f^g)&e)^g+	'&ror	($a1,$Sigma0[1]-$Sigma0[0])',+	'&xor	($a2,$b)',			# a^b, b^c in next round++	'&add	($h,$a4)',			# h+=Ch(e,f,g)+	'&ror	($a0,$Sigma1[0])',		# Sigma1(e)+	'&and	($a3,$a2)',			# (b^c)&(a^b)++	'&xor	($a1,$a)',+	'&add	($h,$a0)',			# h+=Sigma1(e)+	'&xor	($a3,$b)',			# Maj(a,b,c)=Ch(a^b,c,b)++	'&ror	($a1,$Sigma0[0])',		# Sigma0(a)+	'&add	($d,$h)',			# d+=h+	'&add	($h,$a3)',			# h+=Maj(a,b,c)++	'&mov	($a0,$d)',+	'&add	($a1,$h);'.			# h+=Sigma0(a)+	'($a2,$a3) = ($a3,$a2); unshift(@ROT,pop(@ROT)); $i++;'+	);+}++######################################################################+# SSSE3 code path+#+if ($SZ==4) {	# SHA256 only+my $Tbl = $inp;+my $_ctx="-64(%rbp)";+my $_inp="-56(%rbp)";+my $_end="-48(%rbp)";+my $framesz=3*8+$win64*16*4;++my @X = map("%xmm$_",(0..3));+my ($t0,$t1,$t2,$t3, $t4,$t5) = map("%xmm$_",(4..9));++$code.=<<___;+.type	${func}_ssse3,\@function,3,"unwind"+.align	64+${func}_ssse3:+.cfi_startproc+	push	%rbp+.cfi_push	%rbp+	mov	%rsp,%rbp+.cfi_def_cfa_register	%rbp+.Lssse3_shortcut:+	push	%rbx+.cfi_push	%rbx+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15+	shl	\$4,%rdx		# num*16+	sub	\$$framesz,%rsp+.cfi_alloca	$framesz+	lea	($inp,%rdx,$SZ),%rdx	# inp+num*16*$SZ+	mov	$ctx,$_ctx		# save ctx, 1st arg+	#mov	$inp,$_inp		# save inp, 2nd arg+	mov	%rdx,$_end		# save end pointer, "3rd" arg+___+$code.=<<___ if ($win64);+	movaps	%xmm6,-0x80(%rbp)+	movaps	%xmm7,-0x70(%rbp)+	movaps	%xmm8,-0x60(%rbp)+	movaps	%xmm9,-0x50(%rbp)+.cfi_offset	%xmm6-%xmm9,-0x90+___+$code.=<<___;+.cfi_end_prologue++	lea	-16*$SZ(%rsp),%rsp+	mov	$SZ*0($ctx),$A+	and	\$-64,%rsp		# align stack+	mov	$SZ*1($ctx),$B+	mov	$SZ*2($ctx),$C+	mov	$SZ*3($ctx),$D+	mov	$SZ*4($ctx),$E+	mov	$SZ*5($ctx),$F+	mov	$SZ*6($ctx),$G+	mov	$SZ*7($ctx),$H+___++$code.=<<___;+	#movdqa	$TABLE+`$SZ*2*$rounds`+32(%rip),$t4+	#movdqa	$TABLE+`$SZ*2*$rounds`+64(%rip),$t5+	jmp	.Lloop_ssse3+.align	16+.Lloop_ssse3:+	movdqa	$TABLE+`$SZ*2*$rounds`(%rip),$t3+	mov	$inp,$_inp		# offload $inp+	movdqu	0x00($inp),@X[0]+	movdqu	0x10($inp),@X[1]+	movdqu	0x20($inp),@X[2]+	pshufb	$t3,@X[0]+	movdqu	0x30($inp),@X[3]+	lea	$TABLE(%rip),$Tbl+	pshufb	$t3,@X[1]+	movdqa	0x00($Tbl),$t0+	movdqa	0x20($Tbl),$t1+	pshufb	$t3,@X[2]+	paddd	@X[0],$t0+	movdqa	0x40($Tbl),$t2+	pshufb	$t3,@X[3]+	movdqa	0x60($Tbl),$t3+	paddd	@X[1],$t1+	paddd	@X[2],$t2+	paddd	@X[3],$t3+	movdqa	$t0,0x00(%rsp)+	mov	$A,$a1+	movdqa	$t1,0x10(%rsp)+	mov	$B,$a3+	movdqa	$t2,0x20(%rsp)+	xor	$C,$a3			# magic+	movdqa	$t3,0x30(%rsp)+	mov	$E,$a0+	jmp	.Lssse3_00_47++.align	16+.Lssse3_00_47:+	sub	\$`-16*2*$SZ`,$Tbl	# size optimization+___+sub Xupdate_256_SSSE3 () {+	(+	'&movdqa	($t0,@X[1]);',+	'&movdqa	($t3,@X[3])',+	'&palignr	($t0,@X[0],$SZ)',	# X[1..4]+	 '&palignr	($t3,@X[2],$SZ);',	# X[9..12]+	'&movdqa	($t1,$t0)',+	'&movdqa	($t2,$t0);',+	'&psrld		($t0,$sigma0[2])',+	 '&paddd	(@X[0],$t3);',		# X[0..3] += X[9..12]+	'&psrld		($t2,$sigma0[0])',+	 '&pshufd	($t3,@X[3],0b11111010)',# X[14..15]+	'&pslld		($t1,8*$SZ-$sigma0[1]);'.+	'&pxor		($t0,$t2)',+	'&psrld		($t2,$sigma0[1]-$sigma0[0]);'.+	'&pxor		($t0,$t1)',+	'&pslld		($t1,$sigma0[1]-$sigma0[0]);'.+	'&pxor		($t0,$t2);',+	 '&movdqa	($t2,$t3)',+	'&pxor		($t0,$t1);',		# sigma0(X[1..4])+	 '&psrld	($t3,$sigma1[2])',+	'&paddd		(@X[0],$t0);',		# X[0..3] += sigma0(X[1..4])+	 '&psrlq	($t2,$sigma1[0])',+	 '&pxor		($t3,$t2);',+	 '&psrlq	($t2,$sigma1[1]-$sigma1[0])',+	 '&pxor		($t3,$t2)',+	 '&pshufb	($t3,$t4)',		# sigma1(X[14..15])+	'&paddd		(@X[0],$t3)',		# X[0..1] += sigma1(X[14..15])+	 '&pshufd	($t3,@X[0],0b01010000)',# X[16..17]+	 '&movdqa	($t2,$t3);',+	 '&psrld	($t3,$sigma1[2])',+	 '&psrlq	($t2,$sigma1[0])',+	 '&pxor		($t3,$t2);',+	 '&psrlq	($t2,$sigma1[1]-$sigma1[0])',+	 '&pxor		($t3,$t2);',+	'&movdqa	($t2,16*2*$j."($Tbl)")',+	 '&pshufb	($t3,$t5)',+	'&paddd		(@X[0],$t3)'		# X[2..3] += sigma1(X[16..17])+	);+}++sub SSSE3_256_00_47 () {+my $j = shift;+my $body = shift;+my @X = @_;+my @insns = (&$body,&$body,&$body,&$body);	# 104 instructions++    if (0) {+	foreach (Xupdate_256_SSSE3()) {		# 36 instructions+	    eval;+	    eval(shift(@insns));+	    eval(shift(@insns));+	    eval(shift(@insns));+	}+    } else {			# squeeze extra 4% on Westmere and 19% on Atom+	  eval(shift(@insns));	#@+	&movdqa		($t0,@X[1]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&movdqa		($t3,@X[3]);+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	&palignr	($t0,@X[0],$SZ);	# X[1..4]+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &palignr	($t3,@X[2],$SZ);	# X[9..12]+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));	#@+	&movdqa		($t1,$t0);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&movdqa		($t2,$t0);+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	&psrld		($t0,$sigma0[2]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &paddd		(@X[0],$t3);		# X[0..3] += X[9..12]+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	&psrld		($t2,$sigma0[0]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &pshufd	($t3,@X[3],0b11111010);	# X[4..15]+	  eval(shift(@insns));+	  eval(shift(@insns));	#@+	&pslld		($t1,8*$SZ-$sigma0[1]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&pxor		($t0,$t2);+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));	#@+	&psrld		($t2,$sigma0[1]-$sigma0[0]);+	  eval(shift(@insns));+	&pxor		($t0,$t1);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&pslld		($t1,$sigma0[1]-$sigma0[0]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&pxor		($t0,$t2);+	  eval(shift(@insns));+	  eval(shift(@insns));	#@+	 &movdqa	($t2,$t3);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&pxor		($t0,$t1);		# sigma0(X[1..4])+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &psrld		($t3,$sigma1[2]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&paddd		(@X[0],$t0);		# X[0..3] += sigma0(X[1..4])+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	 &psrlq		($t2,$sigma1[0]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &pxor		($t3,$t2);+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));	#@+	 &psrlq		($t2,$sigma1[1]-$sigma1[0]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &pxor		($t3,$t2);+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	  eval(shift(@insns));+	 #&pshufb	($t3,$t4);		# sigma1(X[14..15])+	 &pshufd	($t3,$t3,0b10000000);+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &psrldq	($t3,8);+	  eval(shift(@insns));+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));	#@+	&paddd		(@X[0],$t3);		# X[0..1] += sigma1(X[14..15])+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &pshufd	($t3,@X[0],0b01010000);	# X[16..17]+	  eval(shift(@insns));+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	 &movdqa	($t2,$t3);+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &psrld		($t3,$sigma1[2]);+	  eval(shift(@insns));+	  eval(shift(@insns));	#@+	 &psrlq		($t2,$sigma1[0]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &pxor		($t3,$t2);+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	 &psrlq		($t2,$sigma1[1]-$sigma1[0]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &pxor		($t3,$t2);+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));	#@+	 #&pshufb	($t3,$t5);+	 &pshufd	($t3,$t3,0b00001000);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&movdqa		($t2,16*2*$j."($Tbl)");+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	 &pslldq	($t3,8);+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	&paddd		(@X[0],$t3);		# X[2..3] += sigma1(X[16..17])+	  eval(shift(@insns));	#@+	  eval(shift(@insns));+	  eval(shift(@insns));+    }+	&paddd		($t2,@X[0]);+	  foreach (@insns) { eval; }		# remaining instructions+	&movdqa		(16*$j."(%rsp)",$t2);+}++    for ($i=0,$j=0; $j<4; $j++) {+	&SSSE3_256_00_47($j,\&body_00_15,@X);+	push(@X,shift(@X));			# rotate(@X)+    }+	&cmpb	($SZ-1+16*2*$SZ."($Tbl)",0);+	&jne	(".Lssse3_00_47");++    for ($i=0; $i<16; ) {+	foreach(body_00_15()) { eval; }+    }+$code.=<<___;+	mov	$_ctx,$ctx+	mov	$a1,$A+	mov	$_inp,$inp++	add	$SZ*0($ctx),$A+	add	$SZ*1($ctx),$B+	add	$SZ*2($ctx),$C+	add	$SZ*3($ctx),$D+	add	$SZ*4($ctx),$E+	add	$SZ*5($ctx),$F+	add	$SZ*6($ctx),$G+	add	$SZ*7($ctx),$H++	lea	16*$SZ($inp),$inp+	cmp	$_end,$inp++	mov	$A,$SZ*0($ctx)+	mov	$B,$SZ*1($ctx)+	mov	$C,$SZ*2($ctx)+	mov	$D,$SZ*3($ctx)+	mov	$E,$SZ*4($ctx)+	mov	$F,$SZ*5($ctx)+	mov	$G,$SZ*6($ctx)+	mov	$H,$SZ*7($ctx)+	jb	.Lloop_ssse3++___+$code.=<<___ if ($win64);+	movaps	-0x80(%rbp),%xmm6+	movaps	-0x70(%rbp),%xmm7+	movaps	-0x60(%rbp),%xmm8+	movaps	-0x50(%rbp),%xmm9+___+$code.=<<___;+	mov	-40(%rbp),%r15+	mov	-32(%rbp),%r14+	mov	-24(%rbp),%r13+	mov	-16(%rbp),%r12+	mov	-8(%rbp),%rbx+	mov	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	pop	%rbp+.cfi_pop	%rbp+.cfi_epilogue+	ret+.cfi_endproc+.size	${func}_ssse3,.-${func}_ssse3+___+}++if ($avx) {{+######################################################################+# XOP code path+#+if ($SZ==8) {	# SHA512 only+my $Tbl=$inp;+my $_ctx="-64(%rbp)";+my $_inp="-56(%rbp)";+my $_end="-48(%rbp)";+my $framesz=3*8+$win64*16*6;++$code.=<<___;+.type	${func}_xop,\@function,3,"unwind"+.align	64+${func}_xop:+.cfi_startproc+	push	%rbp+.cfi_push	%rbp+	mov	%rsp,%rbp+.cfi_def_cfa_register	%rbp+.Lxop_shortcut:+	push	%rbx+.cfi_push	%rbx+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15+	shl	\$4,%rdx		# num*16+	sub	\$$framesz,%rsp+.cfi_alloca	$framesz+	lea	($inp,%rdx,$SZ),%rdx	# inp+num*16*$SZ+	mov	$ctx,$_ctx		# save ctx, 1st arg+	#mov	$inp,$_inp		# save inp, 2nd arg+	mov	%rdx,$_end		# save end pointer, "3rd" arg+___+$code.=<<___ if ($win64);+	movaps	%xmm6,-0xa0(%rbp)+	movaps	%xmm7,-0x90(%rbp)+	movaps	%xmm8,-0x80(%rbp)+	movaps	%xmm9,-0x70(%rbp)+.cfi_offset	%xmm6-%xmm9,-0xb0+___+$code.=<<___ if ($win64 && $SZ>4);+	movaps	%xmm10,-0x60(%rbp)+	movaps	%xmm11,-0x50(%rbp)+.cfi_offset	%xmm10-%xmm11,-0x70+___+$code.=<<___;+.cfi_end_prologue++	lea	-16*$SZ(%rsp),%rsp+	vzeroupper+	and	\$-64,%rsp		# align stack+	mov	$SZ*0($ctx),$A+	mov	$SZ*1($ctx),$B+	mov	$SZ*2($ctx),$C+	mov	$SZ*3($ctx),$D+	mov	$SZ*4($ctx),$E+	mov	$SZ*5($ctx),$F+	mov	$SZ*6($ctx),$G+	mov	$SZ*7($ctx),$H+	jmp	.Lloop_xop+___+					if ($SZ==4) {	# SHA256+    my @X = map("%xmm$_",(0..3));+    my ($t0,$t1,$t2,$t3) = map("%xmm$_",(4..7));++$code.=<<___;+.align	16+.Lloop_xop:+	vmovdqa	$TABLE+`$SZ*2*$rounds`(%rip),$t3+	mov	$inp,$_inp		# offload $inp+	vmovdqu	0x00($inp),@X[0]+	vmovdqu	0x10($inp),@X[1]+	vmovdqu	0x20($inp),@X[2]+	vmovdqu	0x30($inp),@X[3]+	vpshufb	$t3,@X[0],@X[0]+	lea	$TABLE(%rip),$Tbl+	vpshufb	$t3,@X[1],@X[1]+	vpshufb	$t3,@X[2],@X[2]+	vpaddd	0x00($Tbl),@X[0],$t0+	vpshufb	$t3,@X[3],@X[3]+	vpaddd	0x20($Tbl),@X[1],$t1+	vpaddd	0x40($Tbl),@X[2],$t2+	vpaddd	0x60($Tbl),@X[3],$t3+	vmovdqa	$t0,0x00(%rsp)+	mov	$A,$a1+	vmovdqa	$t1,0x10(%rsp)+	mov	$B,$a3+	vmovdqa	$t2,0x20(%rsp)+	xor	$C,$a3			# magic+	vmovdqa	$t3,0x30(%rsp)+	mov	$E,$a0+	jmp	.Lxop_00_47++.align	16+.Lxop_00_47:+	sub	\$`-16*2*$SZ`,$Tbl	# size optimization+___+sub XOP_256_00_47 () {+my $j = shift;+my $body = shift;+my @X = @_;+my @insns = (&$body,&$body,&$body,&$body);	# 104 instructions++	&vpalignr	($t0,@X[1],@X[0],$SZ);	# X[1..4]+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vpalignr	($t3,@X[3],@X[2],$SZ);	# X[9..12]+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vprotd		($t1,$t0,8*$SZ-$sigma0[1]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpsrld		($t0,$t0,$sigma0[2]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vpaddd	(@X[0],@X[0],$t3);	# X[0..3] += X[9..12]+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vprotd		($t2,$t1,$sigma0[1]-$sigma0[0]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpxor		($t0,$t0,$t1);+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vprotd	($t3,@X[3],8*$SZ-$sigma1[1]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpxor		($t0,$t0,$t2);		# sigma0(X[1..4])+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vpsrld	($t2,@X[3],$sigma1[2]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpaddd		(@X[0],@X[0],$t0);	# X[0..3] += sigma0(X[1..4])+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vprotd	($t1,$t3,$sigma1[1]-$sigma1[0]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vpxor		($t3,$t3,$t2);+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vpxor		($t3,$t3,$t1);		# sigma1(X[14..15])+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpsrldq	($t3,$t3,8);+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpaddd		(@X[0],@X[0],$t3);	# X[0..1] += sigma1(X[14..15])+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vprotd	($t3,@X[0],8*$SZ-$sigma1[1]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vpsrld	($t2,@X[0],$sigma1[2]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vprotd	($t1,$t3,$sigma1[1]-$sigma1[0]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vpxor		($t3,$t3,$t2);+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vpxor		($t3,$t3,$t1);		# sigma1(X[16..17])+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpslldq	($t3,$t3,8);		# 22 instructions+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpaddd		(@X[0],@X[0],$t3);	# X[2..3] += sigma1(X[16..17])+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpaddd		($t2,@X[0],16*2*$j."($Tbl)");+	  foreach (@insns) { eval; }		# remaining instructions+	&vmovdqa	(16*$j."(%rsp)",$t2);+}++    for ($i=0,$j=0; $j<4; $j++) {+	&XOP_256_00_47($j,\&body_00_15,@X);+	push(@X,shift(@X));			# rotate(@X)+    }+	&cmpb	($SZ-1+16*2*$SZ."($Tbl)",0);+	&jne	(".Lxop_00_47");++    for ($i=0; $i<16; ) {+	foreach(body_00_15()) { eval; }+    }++					} else {	# SHA512+    my @X = map("%xmm$_",(0..7));+    my ($t0,$t1,$t2,$t3) = map("%xmm$_",(8..11));++$code.=<<___;+.align	16+.Lloop_xop:+	vmovdqa	$TABLE+`$SZ*2*$rounds`(%rip),$t3+	mov	$inp,$_inp		# offload $inp+	vmovdqu	0x00($inp),@X[0]+	vmovdqu	0x10($inp),@X[1]+	vmovdqu	0x20($inp),@X[2]+	vpshufb	$t3,@X[0],@X[0]+	vmovdqu	0x30($inp),@X[3]+	vpshufb	$t3,@X[1],@X[1]+	vmovdqu	0x40($inp),@X[4]+	vpshufb	$t3,@X[2],@X[2]+	vmovdqu	0x50($inp),@X[5]+	vpshufb	$t3,@X[3],@X[3]+	vmovdqu	0x60($inp),@X[6]+	vpshufb	$t3,@X[4],@X[4]+	vmovdqu	0x70($inp),@X[7]+	lea	$TABLE+0x80(%rip),$Tbl	# size optimization+	vpshufb	$t3,@X[5],@X[5]+	vpaddq	-0x80($Tbl),@X[0],$t0+	vpshufb	$t3,@X[6],@X[6]+	vpaddq	-0x60($Tbl),@X[1],$t1+	vpshufb	$t3,@X[7],@X[7]+	vpaddq	-0x40($Tbl),@X[2],$t2+	vpaddq	-0x20($Tbl),@X[3],$t3+	vmovdqa	$t0,0x00(%rsp)+	vpaddq	0x00($Tbl),@X[4],$t0+	vmovdqa	$t1,0x10(%rsp)+	vpaddq	0x20($Tbl),@X[5],$t1+	vmovdqa	$t2,0x20(%rsp)+	vpaddq	0x40($Tbl),@X[6],$t2+	vmovdqa	$t3,0x30(%rsp)+	vpaddq	0x60($Tbl),@X[7],$t3+	vmovdqa	$t0,0x40(%rsp)+	mov	$A,$a1+	vmovdqa	$t1,0x50(%rsp)+	mov	$B,$a3+	vmovdqa	$t2,0x60(%rsp)+	xor	$C,$a3			# magic+	vmovdqa	$t3,0x70(%rsp)+	mov	$E,$a0+	jmp	.Lxop_00_47++.align	16+.Lxop_00_47:+	add	\$`16*2*$SZ`,$Tbl+___+sub XOP_512_00_47 () {+my $j = shift;+my $body = shift;+my @X = @_;+my @insns = (&$body,&$body);			# 52 instructions++	&vpalignr	($t0,@X[1],@X[0],$SZ);	# X[1..2]+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vpalignr	($t3,@X[5],@X[4],$SZ);	# X[9..10]+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vprotq		($t1,$t0,8*$SZ-$sigma0[1]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpsrlq		($t0,$t0,$sigma0[2]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vpaddq	(@X[0],@X[0],$t3);	# X[0..1] += X[9..10]+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vprotq		($t2,$t1,$sigma0[1]-$sigma0[0]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpxor		($t0,$t0,$t1);+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vprotq	($t3,@X[7],8*$SZ-$sigma1[1]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpxor		($t0,$t0,$t2);		# sigma0(X[1..2])+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vpsrlq	($t2,@X[7],$sigma1[2]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpaddq		(@X[0],@X[0],$t0);	# X[0..1] += sigma0(X[1..2])+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vprotq	($t1,$t3,$sigma1[1]-$sigma1[0]);+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vpxor		($t3,$t3,$t2);+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	 &vpxor		($t3,$t3,$t1);		# sigma1(X[14..15])+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpaddq		(@X[0],@X[0],$t3);	# X[0..1] += sigma1(X[14..15])+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	  eval(shift(@insns));+	&vpaddq		($t2,@X[0],16*2*$j-0x80."($Tbl)");+	  foreach (@insns) { eval; }		# remaining instructions+	&vmovdqa	(16*$j."(%rsp)",$t2);+}++    for ($i=0,$j=0; $j<8; $j++) {+	&XOP_512_00_47($j,\&body_00_15,@X);+	push(@X,shift(@X));			# rotate(@X)+    }+	&cmpb	($SZ-1+16*2*$SZ-0x80."($Tbl)",0);+	&jne	(".Lxop_00_47");++    for ($i=0; $i<16; ) {+	foreach(body_00_15()) { eval; }+    }+}+$code.=<<___;+	mov	$_ctx,$ctx+	mov	$a1,$A+	mov	$_inp,$inp++	add	$SZ*0($ctx),$A+	add	$SZ*1($ctx),$B+	add	$SZ*2($ctx),$C+	add	$SZ*3($ctx),$D+	add	$SZ*4($ctx),$E+	add	$SZ*5($ctx),$F+	add	$SZ*6($ctx),$G+	add	$SZ*7($ctx),$H++	lea	16*$SZ($inp),$inp+	cmp	$_end,$inp++	mov	$A,$SZ*0($ctx)+	mov	$B,$SZ*1($ctx)+	mov	$C,$SZ*2($ctx)+	mov	$D,$SZ*3($ctx)+	mov	$E,$SZ*4($ctx)+	mov	$F,$SZ*5($ctx)+	mov	$G,$SZ*6($ctx)+	mov	$H,$SZ*7($ctx)+	jb	.Lloop_xop++	vzeroupper+___+$code.=<<___ if ($win64);+	movaps	-0xa0(%rbp),%xmm6+	movaps	-0x90(%rbp),%xmm7+	movaps	-0x80(%rbp),%xmm8+	movaps	-0x70(%rbp),%xmm9+___+$code.=<<___ if ($win64 && $SZ>4);+	movaps	-0x60(%rbp),%xmm10+	movaps	-0x50(%rbp),%xmm11+___+$code.=<<___;+	mov	-40(%rbp),%r15+	mov	-32(%rbp),%r14+	mov	-24(%rbp),%r13+	mov	-16(%rbp),%r12+	mov	-8(%rbp),%rbx+	mov	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	pop	%rbp+.cfi_pop	%rbp+.cfi_epilogue+	ret+.cfi_endproc+.size	${func}_xop,.-${func}_xop+___+}+######################################################################+# AVX+shrd code path+#+my $Tbl=$inp;+my $_ctx="-64(%rbp)";+my $_inp="-56(%rbp)";+my $_end="-48(%rbp)";+my $framesz=3*8+$win64*16*6;++local *ror = sub { &shrd(@_[0],@_) };++$code.=<<___;+.type	${func}_avx,\@function,3,"unwind"+.align	64+${func}_avx:+.cfi_startproc+	push	%rbp+.cfi_push	%rbp+	mov	%rsp,%rbp+.cfi_def_cfa_register	%rbp+.Lavx_shortcut:+	push	%rbx+.cfi_push	%rbx+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15+	shl	\$4,%rdx		# num*16+	sub	\$$framesz,%rsp+.cfi_alloca	$framesz+	lea	($inp,%rdx,$SZ),%rdx	# inp+num*16*$SZ+	mov	$ctx,$_ctx		# save ctx, 1st arg+	#mov	$inp,$_inp		# save inp, 2nd arg+	mov	%rdx,$_end		# save end pointer, "3rd" arg+___+$code.=<<___ if ($win64);+	movaps	%xmm6,-0xa0(%rbp)+	movaps	%xmm7,-0x90(%rbp)+	movaps	%xmm8,-0x80(%rbp)+	movaps	%xmm9,-0x70(%rbp)+.cfi_offset	%xmm6-%xmm9,-0xb0+___+$code.=<<___ if ($win64 && $SZ>4);+	movaps	%xmm10,-0x60(%rbp)+	movaps	%xmm11,-0x50(%rbp)+.cfi_offset	%xmm10-%xmm11,-0x70+___+$code.=<<___;+.cfi_end_prologue++	lea	-16*$SZ(%rsp),%rsp+	vzeroupper+	and	\$-64,%rsp		# align stack+	mov	$SZ*0($ctx),$A+	mov	$SZ*1($ctx),$B+	mov	$SZ*2($ctx),$C+	mov	$SZ*3($ctx),$D+	mov	$SZ*4($ctx),$E+	mov	$SZ*5($ctx),$F+	mov	$SZ*6($ctx),$G+	mov	$SZ*7($ctx),$H+___+					if ($SZ==4) {	# SHA256+    my @X = map("%xmm$_",(0..3));+    my ($t0,$t1,$t2,$t3, $t4,$t5) = map("%xmm$_",(4..9));++$code.=<<___;+	vmovdqa	$TABLE+`$SZ*2*$rounds`+32(%rip),$t4+	vmovdqa	$TABLE+`$SZ*2*$rounds`+64(%rip),$t5+	jmp	.Lloop_avx+.align	16+.Lloop_avx:+	vmovdqa	$TABLE+`$SZ*2*$rounds`(%rip),$t3+	mov	$inp,$_inp		# offload $inp+	vmovdqu	0x00($inp),@X[0]+	vmovdqu	0x10($inp),@X[1]+	vmovdqu	0x20($inp),@X[2]+	vmovdqu	0x30($inp),@X[3]+	vpshufb	$t3,@X[0],@X[0]+	lea	$TABLE(%rip),$Tbl+	vpshufb	$t3,@X[1],@X[1]+	vpshufb	$t3,@X[2],@X[2]+	vpaddd	0x00($Tbl),@X[0],$t0+	vpshufb	$t3,@X[3],@X[3]+	vpaddd	0x20($Tbl),@X[1],$t1+	vpaddd	0x40($Tbl),@X[2],$t2+	vpaddd	0x60($Tbl),@X[3],$t3+	vmovdqa	$t0,0x00(%rsp)+	mov	$A,$a1+	vmovdqa	$t1,0x10(%rsp)+	mov	$B,$a3+	vmovdqa	$t2,0x20(%rsp)+	xor	$C,$a3			# magic+	vmovdqa	$t3,0x30(%rsp)+	mov	$E,$a0+	jmp	.Lavx_00_47++.align	16+.Lavx_00_47:+	sub	\$`-16*2*$SZ`,$Tbl	# size optimization+___+sub Xupdate_256_AVX () {+	(+	'&vpalignr	($t0,@X[1],@X[0],$SZ)',	# X[1..4]+	 '&vpalignr	($t3,@X[3],@X[2],$SZ)',	# X[9..12]+	'&vpsrld	($t2,$t0,$sigma0[0]);',+	 '&vpaddd	(@X[0],@X[0],$t3)',	# X[0..3] += X[9..12]+	'&vpsrld	($t3,$t0,$sigma0[2])',+	'&vpslld	($t1,$t0,8*$SZ-$sigma0[1]);',+	'&vpxor		($t0,$t3,$t2)',+	 '&vpshufd	($t3,@X[3],0b11111010)',# X[14..15]+	'&vpsrld	($t2,$t2,$sigma0[1]-$sigma0[0]);',+	'&vpxor		($t0,$t0,$t1)',+	'&vpslld	($t1,$t1,$sigma0[1]-$sigma0[0]);',+	'&vpxor		($t0,$t0,$t2)',+	 '&vpsrld	($t2,$t3,$sigma1[2]);',+	'&vpxor		($t0,$t0,$t1)',		# sigma0(X[1..4])+	 '&vpsrlq	($t3,$t3,$sigma1[0]);',+	'&vpaddd	(@X[0],@X[0],$t0)',	# X[0..3] += sigma0(X[1..4])+	 '&vpxor	($t2,$t2,$t3);',+	 '&vpsrlq	($t3,$t3,$sigma1[1]-$sigma1[0])',+	 '&vpxor	($t2,$t2,$t3)',+	 '&vpshufb	($t2,$t2,$t4)',		# sigma1(X[14..15])+	'&vpaddd	(@X[0],@X[0],$t2)',	# X[0..1] += sigma1(X[14..15])+	 '&vpshufd	($t3,@X[0],0b01010000)',# X[16..17]+	 '&vpsrld	($t2,$t3,$sigma1[2])',+	 '&vpsrlq	($t3,$t3,$sigma1[0])',+	 '&vpxor	($t2,$t2,$t3);',+	 '&vpsrlq	($t3,$t3,$sigma1[1]-$sigma1[0])',+	 '&vpxor	($t2,$t2,$t3)',+	 '&vpshufb	($t2,$t2,$t5)',+	'&vpaddd	(@X[0],@X[0],$t2)'	# X[2..3] += sigma1(X[16..17])+	);+}++sub AVX_256_00_47 () {+my $j = shift;+my $body = shift;+my @X = @_;+my @insns = (&$body,&$body,&$body,&$body);	# 104 instructions++	foreach (Xupdate_256_AVX()) {		# 29 instructions+	    eval;+	    eval(shift(@insns));+	    eval(shift(@insns));+	    eval(shift(@insns));+	}+	&vpaddd		($t2,@X[0],16*2*$j."($Tbl)");+	  foreach (@insns) { eval; }		# remaining instructions+	&vmovdqa	(16*$j."(%rsp)",$t2);+}++    for ($i=0,$j=0; $j<4; $j++) {+	&AVX_256_00_47($j,\&body_00_15,@X);+	push(@X,shift(@X));			# rotate(@X)+    }+	&cmpb	($SZ-1+16*2*$SZ."($Tbl)",0);+	&jne	(".Lavx_00_47");++    for ($i=0; $i<16; ) {+	foreach(body_00_15()) { eval; }+    }++					} else {	# SHA512+    my @X = map("%xmm$_",(0..7));+    my ($t0,$t1,$t2,$t3) = map("%xmm$_",(8..11));++$code.=<<___;+	jmp	.Lloop_avx+.align	16+.Lloop_avx:+	vmovdqa	$TABLE+`$SZ*2*$rounds`(%rip),$t3+	mov	$inp,$_inp		# offload $inp+	vmovdqu	0x00($inp),@X[0]+	vmovdqu	0x10($inp),@X[1]+	vmovdqu	0x20($inp),@X[2]+	vpshufb	$t3,@X[0],@X[0]+	vmovdqu	0x30($inp),@X[3]+	vpshufb	$t3,@X[1],@X[1]+	vmovdqu	0x40($inp),@X[4]+	vpshufb	$t3,@X[2],@X[2]+	vmovdqu	0x50($inp),@X[5]+	vpshufb	$t3,@X[3],@X[3]+	vmovdqu	0x60($inp),@X[6]+	vpshufb	$t3,@X[4],@X[4]+	vmovdqu	0x70($inp),@X[7]+	lea	$TABLE+0x80(%rip),$Tbl	# size optimization+	vpshufb	$t3,@X[5],@X[5]+	vpaddq	-0x80($Tbl),@X[0],$t0+	vpshufb	$t3,@X[6],@X[6]+	vpaddq	-0x60($Tbl),@X[1],$t1+	vpshufb	$t3,@X[7],@X[7]+	vpaddq	-0x40($Tbl),@X[2],$t2+	vpaddq	-0x20($Tbl),@X[3],$t3+	vmovdqa	$t0,0x00(%rsp)+	vpaddq	0x00($Tbl),@X[4],$t0+	vmovdqa	$t1,0x10(%rsp)+	vpaddq	0x20($Tbl),@X[5],$t1+	vmovdqa	$t2,0x20(%rsp)+	vpaddq	0x40($Tbl),@X[6],$t2+	vmovdqa	$t3,0x30(%rsp)+	vpaddq	0x60($Tbl),@X[7],$t3+	vmovdqa	$t0,0x40(%rsp)+	mov	$A,$a1+	vmovdqa	$t1,0x50(%rsp)+	mov	$B,$a3+	vmovdqa	$t2,0x60(%rsp)+	xor	$C,$a3			# magic+	vmovdqa	$t3,0x70(%rsp)+	mov	$E,$a0+	jmp	.Lavx_00_47++.align	16+.Lavx_00_47:+	add	\$`16*2*$SZ`,$Tbl+___+sub Xupdate_512_AVX () {+	(+	'&vpalignr	($t0,@X[1],@X[0],$SZ)',	# X[1..2]+	 '&vpalignr	($t3,@X[5],@X[4],$SZ)',	# X[9..10]+	'&vpsrlq	($t2,$t0,$sigma0[0])',+	 '&vpaddq	(@X[0],@X[0],$t3);',	# X[0..1] += X[9..10]+	'&vpsrlq	($t3,$t0,$sigma0[2])',+	'&vpsllq	($t1,$t0,8*$SZ-$sigma0[1]);',+	 '&vpxor	($t0,$t3,$t2)',+	'&vpsrlq	($t2,$t2,$sigma0[1]-$sigma0[0]);',+	 '&vpxor	($t0,$t0,$t1)',+	'&vpsllq	($t1,$t1,$sigma0[1]-$sigma0[0]);',+	 '&vpxor	($t0,$t0,$t2)',+	 '&vpsrlq	($t3,@X[7],$sigma1[2]);',+	'&vpxor		($t0,$t0,$t1)',		# sigma0(X[1..2])+	 '&vpsllq	($t2,@X[7],8*$SZ-$sigma1[1]);',+	'&vpaddq	(@X[0],@X[0],$t0)',	# X[0..1] += sigma0(X[1..2])+	 '&vpsrlq	($t1,@X[7],$sigma1[0]);',+	 '&vpxor	($t3,$t3,$t2)',+	 '&vpsllq	($t2,$t2,$sigma1[1]-$sigma1[0]);',+	 '&vpxor	($t3,$t3,$t1)',+	 '&vpsrlq	($t1,$t1,$sigma1[1]-$sigma1[0]);',+	 '&vpxor	($t3,$t3,$t2)',+	 '&vpxor	($t3,$t3,$t1)',		# sigma1(X[14..15])+	'&vpaddq	(@X[0],@X[0],$t3)',	# X[0..1] += sigma1(X[14..15])+	);+}++sub AVX_512_00_47 () {+my $j = shift;+my $body = shift;+my @X = @_;+my @insns = (&$body,&$body);			# 52 instructions++	foreach (Xupdate_512_AVX()) {		# 23 instructions+	    eval;+	    eval(shift(@insns));+	    eval(shift(@insns));+	}+	&vpaddq		($t2,@X[0],16*2*$j-0x80."($Tbl)");+	  foreach (@insns) { eval; }		# remaining instructions+	&vmovdqa	(16*$j."(%rsp)",$t2);+}++    for ($i=0,$j=0; $j<8; $j++) {+	&AVX_512_00_47($j,\&body_00_15,@X);+	push(@X,shift(@X));			# rotate(@X)+    }+	&cmpb	($SZ-1+16*2*$SZ-0x80."($Tbl)",0);+	&jne	(".Lavx_00_47");++    for ($i=0; $i<16; ) {+	foreach(body_00_15()) { eval; }+    }+}+$code.=<<___;+	mov	$_ctx,$ctx+	mov	$a1,$A+	mov	$_inp,$inp++	add	$SZ*0($ctx),$A+	add	$SZ*1($ctx),$B+	add	$SZ*2($ctx),$C+	add	$SZ*3($ctx),$D+	add	$SZ*4($ctx),$E+	add	$SZ*5($ctx),$F+	add	$SZ*6($ctx),$G+	add	$SZ*7($ctx),$H++	lea	16*$SZ($inp),$inp+	cmp	$_end,$inp++	mov	$A,$SZ*0($ctx)+	mov	$B,$SZ*1($ctx)+	mov	$C,$SZ*2($ctx)+	mov	$D,$SZ*3($ctx)+	mov	$E,$SZ*4($ctx)+	mov	$F,$SZ*5($ctx)+	mov	$G,$SZ*6($ctx)+	mov	$H,$SZ*7($ctx)+	jb	.Lloop_avx++	vzeroupper+___+$code.=<<___ if ($win64);+	movaps	-0xa0(%rbp),%xmm6+	movaps	-0x90(%rbp),%xmm7+	movaps	-0x80(%rbp),%xmm8+	movaps	-0x70(%rbp),%xmm9+___+$code.=<<___ if ($win64 && $SZ>4);+	movaps	-0x60(%rbp),%xmm10+	movaps	-0x50(%rbp),%xmm11+___+$code.=<<___;+	mov	-40(%rbp),%r15+	mov	-32(%rbp),%r14+	mov	-24(%rbp),%r13+	mov	-16(%rbp),%r12+	mov	-8(%rbp),%rbx+	mov	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	pop	%rbp+.cfi_pop	%rbp+.cfi_epilogue+	ret+.cfi_endproc+.size	${func}_avx,.-${func}_avx+___++if ($avx>1) {{+######################################################################+# AVX2+BMI code path+#+my $Tbl=$inp;+my $_ctx="-64(%rbp)";+my $_inp="-56(%rbp)";+my $_end="-48(%rbp)";+my $framesz=3*8+$win64*16*6;+my $PUSH8=8*2*$SZ;+use integer;++sub bodyx_00_15 () {+	# at start $a1 should be zero, $a3 - $b^$c and $a4 copy of $f+	(+	'($a,$b,$c,$d,$e,$f,$g,$h)=@ROT;'.++	'&add	($h,(32*($i/(16/$SZ))+$SZ*($i%(16/$SZ)))%$PUSH8.$base)',    # h+=X[i]+K[i]+	'&and	($a4,$e)',		# f&e+	'&rorx	($a0,$e,$Sigma1[2])',+	'&rorx	($a2,$e,$Sigma1[1])',++	'&lea	($a,"($a,$a1)")',	# h+=Sigma0(a) from the past+	'&lea	($h,"($h,$a4)")',+	'&andn	($a4,$e,$g)',		# ~e&g+	'&xor	($a0,$a2)',++	'&rorx	($a1,$e,$Sigma1[0])',+	'&lea	($h,"($h,$a4)")',	# h+=Ch(e,f,g)=(e&f)+(~e&g)+	'&xor	($a0,$a1)',		# Sigma1(e)+	'&mov	($a2,$a)',++	'&rorx	($a4,$a,$Sigma0[2])',+	'&lea	($h,"($h,$a0)")',	# h+=Sigma1(e)+	'&xor	($a2,$b)',		# a^b, b^c in next round+	'&rorx	($a1,$a,$Sigma0[1])',++	'&rorx	($a0,$a,$Sigma0[0])',+	'&lea	($d,"($d,$h)")',	# d+=h+	'&and	($a3,$a2)',		# (b^c)&(a^b)+	'&xor	($a1,$a4)',++	'&xor	($a3,$b)',		# Maj(a,b,c)=Ch(a^b,c,b)+	'&xor	($a1,$a0)',		# Sigma0(a)+	'&lea	($h,"($h,$a3)");'.	# h+=Maj(a,b,c)+	'&mov	($a4,$e)',		# copy of f in future++	'($a2,$a3) = ($a3,$a2); unshift(@ROT,pop(@ROT)); $i++;'+	);+	# and at the finish one has to $a+=$a1+}++$code.=<<___;+.type	${func}_avx2,\@function,3,"unwind"+.align	64+${func}_avx2:+.cfi_startproc+	push	%rbp+.cfi_push	%rbp+	mov	%rsp,%rbp+.cfi_def_cfa_register	%rbp+.Lavx2_shortcut:+	push	%rbx+.cfi_push	%rbx+	push	%r12+.cfi_push	%r12+	push	%r13+.cfi_push	%r13+	push	%r14+.cfi_push	%r14+	push	%r15+.cfi_push	%r15+	shl	\$4,%rdx		# num*16+	sub	\$$framesz,%rsp+.cfi_alloca	$framesz+	lea	($inp,%rdx,$SZ),%rdx	# inp+num*16*$SZ+	mov	$ctx,$_ctx		# save ctx, 1st arg+	#mov	$inp,$_inp		# save inp, 2nd arg+	mov	%rdx,$_end		# save end pointer, "3rd" arg+___+$code.=<<___ if ($win64);+	movaps	%xmm6,-0xa0(%rbp)+	movaps	%xmm7,-0x90(%rbp)+	movaps	%xmm8,-0x80(%rbp)+	movaps	%xmm9,-0x70(%rbp)+.cfi_offset	%xmm6-%xmm9,-0xb0+___+$code.=<<___ if ($win64 && $SZ>4);+	movaps	%xmm10,-0x60(%rbp)+	movaps	%xmm11,-0x50(%rbp)+.cfi_offset	%xmm10-%xmm11,-0x70+___+$code.=<<___;+.cfi_end_prologue++	lea	-$PUSH8(%rsp),%rsp+	vzeroupper+	and	\$-$PUSH8,%rsp		# align stack+	sub	\$-16*$SZ,$inp		# inp++, size optimization+	mov	$SZ*0($ctx),$A+	mov	$inp,%r12		# borrow $T1+	mov	$SZ*1($ctx),$B+	cmp	%rdx,$inp		# $_end+	mov	$SZ*2($ctx),$C+	cmove	%rsp,%r12		# next block or random data+	mov	$SZ*3($ctx),$D+	mov	$SZ*4($ctx),$E+	mov	$SZ*5($ctx),$F+	mov	$SZ*6($ctx),$G+	mov	$SZ*7($ctx),$H+___+					if ($SZ==4) {	# SHA256+    my @X = map("%ymm$_",(0..3));+    my ($t0,$t1,$t2,$t3, $t4,$t5) = map("%ymm$_",(4..9));++$code.=<<___;+	vmovdqa	$TABLE+`$SZ*2*$rounds`+32(%rip),$t4+	vmovdqa	$TABLE+`$SZ*2*$rounds`+64(%rip),$t5+	jmp	.Loop_avx2+.align	16+.Loop_avx2:+	vmovdqa	$TABLE+`$SZ*2*$rounds`(%rip),$t3+	mov	$inp,$_inp		# offload $inp+	vmovdqu	-16*$SZ+0($inp),%xmm0+	vmovdqu	-16*$SZ+16($inp),%xmm1+	vmovdqu	-16*$SZ+32($inp),%xmm2+	vmovdqu	-16*$SZ+48($inp),%xmm3+	lea	$TABLE(%rip),$Tbl+	vinserti128	\$1,(%r12),@X[0],@X[0]+	vinserti128	\$1,16(%r12),@X[1],@X[1]+	vpshufb		$t3,@X[0],@X[0]+	vinserti128	\$1,32(%r12),@X[2],@X[2]+	vpshufb		$t3,@X[1],@X[1]+	vinserti128	\$1,48(%r12),@X[3],@X[3]++	vpshufb	$t3,@X[2],@X[2]+	vpaddd	0x00($Tbl),@X[0],$t0+	vpshufb	$t3,@X[3],@X[3]+	vpaddd	0x20($Tbl),@X[1],$t1+	vpaddd	0x40($Tbl),@X[2],$t2+	vpaddd	0x60($Tbl),@X[3],$t3+	vmovdqa	$t0,0x00(%rsp)+	xor	$a1,$a1+	vmovdqa	$t1,0x20(%rsp)+	lea	-$PUSH8(%rsp),%rsp+	mov	$B,$a3+	vmovdqa	$t2,0x00(%rsp)+	xor	$C,$a3			# magic+	vmovdqa	$t3,0x20(%rsp)+	mov	$F,$a4+	sub	\$-16*2*$SZ,$Tbl	# size optimization+	jmp	.Lavx2_00_47++.align	16+.Lavx2_00_47:+___++sub AVX2_256_00_47 () {+my $j = shift;+my $body = shift;+my @X = @_;+my @insns = (&$body,&$body,&$body,&$body);	# 96 instructions+my $base = "+2*$PUSH8(%rsp)";++	&lea	("%rsp","-$PUSH8(%rsp)")	if (($j%2)==0);+	foreach (Xupdate_256_AVX()) {		# 29 instructions+	    eval;+	    eval(shift(@insns));+	    eval(shift(@insns));+	    eval(shift(@insns));+	}+	&vpaddd		($t2,@X[0],16*2*$j."($Tbl)");+	  foreach (@insns) { eval; }		# remaining instructions+	&vmovdqa	((32*$j)%$PUSH8."(%rsp)",$t2);+}++    for ($i=0,$j=0; $j<4; $j++) {+	&AVX2_256_00_47($j,\&bodyx_00_15,@X);+	push(@X,shift(@X));			# rotate(@X)+    }+	&lea	($Tbl,16*2*$SZ."($Tbl)");+	&cmpb	(($SZ-1)."($Tbl)",0);+	&jne	(".Lavx2_00_47");++    for ($i=0; $i<16; ) {+	my $base=$i<8?"+$PUSH8(%rsp)":"(%rsp)";+	foreach(bodyx_00_15()) { eval; }+    }+					} else {	# SHA512+    my @X = map("%ymm$_",(0..7));+    my ($t0,$t1,$t2,$t3) = map("%ymm$_",(8..11));++$code.=<<___;+	jmp	.Loop_avx2+.align	16+.Loop_avx2:+	vmovdqa	$TABLE+`$SZ*2*$rounds`(%rip),$t2+	mov	$inp,$_inp		# offload $inp+	vmovdqu	-16*$SZ($inp),%xmm0+	vmovdqu	-16*$SZ+16($inp),%xmm1+	vmovdqu	-16*$SZ+32($inp),%xmm2+	vmovdqu	-16*$SZ+48($inp),%xmm3+	vmovdqu	-16*$SZ+64($inp),%xmm4+	vmovdqu	-16*$SZ+80($inp),%xmm5+	vmovdqu	-16*$SZ+96($inp),%xmm6+	vmovdqu	-16*$SZ+112($inp),%xmm7+	lea	$TABLE+0x80(%rip),$Tbl	# size optimization+	vinserti128	\$1,(%r12),@X[0],@X[0]+	vinserti128	\$1,16(%r12),@X[1],@X[1]+	 vpshufb	$t2,@X[0],@X[0]+	vinserti128	\$1,32(%r12),@X[2],@X[2]+	 vpshufb	$t2,@X[1],@X[1]+	vinserti128	\$1,48(%r12),@X[3],@X[3]+	 vpshufb	$t2,@X[2],@X[2]+	vinserti128	\$1,64(%r12),@X[4],@X[4]+	 vpshufb	$t2,@X[3],@X[3]+	vinserti128	\$1,80(%r12),@X[5],@X[5]+	 vpshufb	$t2,@X[4],@X[4]+	vinserti128	\$1,96(%r12),@X[6],@X[6]+	 vpshufb	$t2,@X[5],@X[5]+	vinserti128	\$1,112(%r12),@X[7],@X[7]++	vpaddq	-0x80($Tbl),@X[0],$t0+	vpshufb	$t2,@X[6],@X[6]+	vpaddq	-0x60($Tbl),@X[1],$t1+	vpshufb	$t2,@X[7],@X[7]+	vpaddq	-0x40($Tbl),@X[2],$t2+	vpaddq	-0x20($Tbl),@X[3],$t3+	vmovdqa	$t0,0x00(%rsp)+	vpaddq	0x00($Tbl),@X[4],$t0+	vmovdqa	$t1,0x20(%rsp)+	vpaddq	0x20($Tbl),@X[5],$t1+	vmovdqa	$t2,0x40(%rsp)+	vpaddq	0x40($Tbl),@X[6],$t2+	vmovdqa	$t3,0x60(%rsp)+	lea	-$PUSH8(%rsp),%rsp+	vpaddq	0x60($Tbl),@X[7],$t3+	vmovdqa	$t0,0x00(%rsp)+	xor	$a1,$a1+	vmovdqa	$t1,0x20(%rsp)+	mov	$B,$a3+	vmovdqa	$t2,0x40(%rsp)+	xor	$C,$a3			# magic+	vmovdqa	$t3,0x60(%rsp)+	mov	$F,$a4+	add	\$16*2*$SZ,$Tbl+	jmp	.Lavx2_00_47++.align	16+.Lavx2_00_47:+___++sub AVX2_512_00_47 () {+my $j = shift;+my $body = shift;+my @X = @_;+my @insns = (&$body,&$body);			# 48 instructions+my $base = "+2*$PUSH8(%rsp)";++	&lea	("%rsp","-$PUSH8(%rsp)")	if (($j%4)==0);+	foreach (Xupdate_512_AVX()) {		# 23 instructions+	    eval;+	    if ($_ !~ /\;$/) {+		eval(shift(@insns));+		eval(shift(@insns));+		eval(shift(@insns));+	    }+	}+	&vpaddq		($t2,@X[0],16*2*$j-0x80."($Tbl)");+	  foreach (@insns) { eval; }		# remaining instructions+	&vmovdqa	((32*$j)%$PUSH8."(%rsp)",$t2);+}++    for ($i=0,$j=0; $j<8; $j++) {+	&AVX2_512_00_47($j,\&bodyx_00_15,@X);+	push(@X,shift(@X));			# rotate(@X)+    }+	&lea	($Tbl,16*2*$SZ."($Tbl)");+	&cmpb	(($SZ-1-0x80)."($Tbl)",0);+	&jne	(".Lavx2_00_47");++    for ($i=0; $i<16; ) {+	my $base=$i<8?"+$PUSH8(%rsp)":"(%rsp)";+	foreach(bodyx_00_15()) { eval; }+    }+}+$code.=<<___;+	mov	$_ctx,$ctx+	add	$a1,$A+	mov	$_inp,$a4++	add	$SZ*0($ctx),$A+	add	$SZ*1($ctx),$B+	add	$SZ*2($ctx),$C+	add	$SZ*3($ctx),$D+	add	$SZ*4($ctx),$E+	add	$SZ*5($ctx),$F+	add	$SZ*6($ctx),$G+	add	$SZ*7($ctx),$H++	mov	$A,$SZ*0($ctx)+	mov	$B,$SZ*1($ctx)+	mov	$C,$SZ*2($ctx)+	mov	$D,$SZ*3($ctx)+	mov	$E,$SZ*4($ctx)+	mov	$F,$SZ*5($ctx)+	mov	$G,$SZ*6($ctx)+	mov	$H,$SZ*7($ctx)++	cmp	$_end,$a4+	je	.Ldone_avx2++	lea	`2*$SZ*($rounds-8)`(%rsp),$Tbl+	xor	$a1,$a1+	mov	$B,$a3+	xor	$C,$a3			# magic+	mov	$F,$a4+	jmp	.Lower_avx2+.align	16+.Lower_avx2:+___+    for ($i=0; $i<8; ) {+	my $base="+16($Tbl)";+	foreach(bodyx_00_15()) { eval; }+    }+$code.=<<___;+	lea	-$PUSH8($Tbl),$Tbl+	cmp	%rsp,$Tbl+	jae	.Lower_avx2++	mov	$_ctx,$ctx+	add	$a1,$A+	mov	$_inp,$inp+	lea	`2*$SZ*($rounds-8)`(%rsp),%rsp++	add	$SZ*0($ctx),$A+	add	$SZ*1($ctx),$B+	add	$SZ*2($ctx),$C+	add	$SZ*3($ctx),$D+	add	$SZ*4($ctx),$E+	add	$SZ*5($ctx),$F+	lea	`2*16*$SZ`($inp),$inp	# inp+=2+	add	$SZ*6($ctx),$G+	mov	$inp,%r12+	add	$SZ*7($ctx),$H+	cmp	$_end,$inp++	mov	$A,$SZ*0($ctx)+	cmove	%rsp,%r12		# next block or stale data+	mov	$B,$SZ*1($ctx)+	mov	$C,$SZ*2($ctx)+	mov	$D,$SZ*3($ctx)+	mov	$E,$SZ*4($ctx)+	mov	$F,$SZ*5($ctx)+	mov	$G,$SZ*6($ctx)+	mov	$H,$SZ*7($ctx)++	jbe	.Loop_avx2++.Ldone_avx2:+	vzeroupper+___+$code.=<<___ if ($win64);+	movaps	-0xa0(%rbp),%xmm6+	movaps	-0x90(%rbp),%xmm7+	movaps	-0x80(%rbp),%xmm8+	movaps	-0x70(%rbp),%xmm9+___+$code.=<<___ if ($win64 && $SZ>4);+	movaps	-0x60(%rbp),%xmm10+	movaps	-0x50(%rbp),%xmm11+___+$code.=<<___;+	mov	-40(%rbp),%r15+	mov	-32(%rbp),%r14+	mov	-24(%rbp),%r13+	mov	-16(%rbp),%r12+	mov	-8(%rbp),%rbx+	mov	%rbp,%rsp+.cfi_def_cfa_register	%rsp+	pop	%rbp+.cfi_pop	%rbp+.cfi_epilogue+	ret+.cfi_endproc+.size	${func}_avx2,.-${func}_avx2+___+}}+}}}}}++sub sha256op38 {+    my $instr = shift;+    my %opcodelet = (+		"sha256rnds2" => 0xcb,+  		"sha256msg1"  => 0xcc,+		"sha256msg2"  => 0xcd	);++    if (defined($opcodelet{$instr}) && @_[0] =~ /%xmm([0-7]),\s*%xmm([0-7])/) {+      my @opcode=(0x0f,0x38);+	push @opcode,$opcodelet{$instr};+	push @opcode,0xc0|($1&7)|(($2&7)<<3);		# ModR/M+	return ".byte\t".join(',',@opcode);+    } else {+	return $instr."\t".@_[0];+    }+}++sub vsha512rnds2 {+    my $instr = shift;++    if (@_[0] =~ /%xmm([0-9]+),\s*%ymm([0-9]+),\s*%ymm([0-9]+)/) {+      my @opcode=(0xc4,0xe2,0x7f,0xcb);+	@opcode[1] ^= (($1>>3)<<5)|(($3>>3)<<7);+	@opcode[2] ^= $2<<3;+	push @opcode,0xc0|($1&7)|(($3&7)<<3);		# ModR/M+	return ".byte\t".join(',',@opcode);+    } else {+	return $instr."\t".@_[0];+    }+}++sub vsha512msg {+    my $instr = shift;+    my $op = shift;++    if (@_[0] =~ /%[xy]mm([0-9]+),\s*%ymm([0-9]+)/) {+      my @opcode=(0xc4,0xe2,0x7f,0xcb+$op);+	@opcode[1] ^= (($1>>3)<<5)|(($2>>3)<<7);+	push @opcode,0xc0|($1&7)|(($2&7)<<3);		# ModR/M+	return ".byte\t".join(',',@opcode);+    } else {+	return $instr.$op."\t".@_[0];+    }+}++foreach (split("\n",$code)) {+	s/\`([^\`]*)\`/eval $1/geo;+	s/%x#%[yz]/%x/go;++	s/\b(sha256[^\s]*)\s+(.*)/sha256op38($1,$2)/eo or+	s/\b(vsha512msg)([12])\s+(.*)/vsha512msg($1,$2,$3)/eo or+	s/\b(vsha512rnds2)\s+(.*)/vsha512rnds2($1,$2)/eo;++	print $_,"\n";+}+close STDOUT;
+ cbits/asm/x86_64-xlate.pl view
@@ -0,0 +1,1943 @@+#!/usr/bin/env perl++# Ascetic x86_64 AT&T to MASM/NASM assembler translator by @dot-asm.+#+# Why AT&T to MASM and not vice versa? Several reasons. Because AT&T+# format is way easier to parse. Because it's simpler to "gear" from+# Unix ABI to Windows one [see cross-reference "card" at the end of+# file]. Because Linux targets were available first...+#+# In addition the script also "distills" code suitable for GNU+# assembler, so that it can be compiled with more rigid assemblers,+# such as Solaris /usr/ccs/bin/as.+#+# This translator is not designed to convert *arbitrary* assembler+# code from AT&T format to MASM one. It's designed to convert just+# enough to provide for dual-ABI OpenSSL modules development...+# There *are* limitations and you might have to modify your assembler+# code or this script to achieve the desired result...+#+# Currently recognized limitations:+#+# - can't use multiple ops per line;+#+# Dual-ABI styling rules.+#+# 1. Adhere to Unix register and stack layout [see cross-reference+#    ABI "card" at the end for explanation].+# 2. Forget about "red zone," stick to more traditional blended+#    stack frame allocation. If volatile storage is actually required+#    that is. If not, just leave the stack as is.+# 3. Functions tagged with ".type name,@function" get crafted with+#    unified Win64 prologue and epilogue automatically. If you want+#    to take care of ABI differences yourself, tag functions as+#    ".type name,@abi-omnipotent" instead.+# 4. To optimize the Win64 prologue you can specify number of input+#    arguments as ".type name,@function,N." Keep in mind that if N is+#    larger than 6, then you *have to* write "abi-omnipotent" code,+#    because >6 cases can't be addressed with unified prologue.+# 5. Name local labels as .L*, do *not* use dynamic labels such as 1:+#    (sorry about latter).+# 6. Don't use [or hand-code with .byte] "rep ret." "ret" mnemonic is+#    required to identify the spots, where to inject Win64 epilogue!+#    But on the pros, it's then prefixed with rep automatically:-)+# 7. Stick to explicit ip-relative addressing. If you have to use+#    GOTPCREL addressing, stick to mov symbol@GOTPCREL(%rip),%r??.+#    Both are recognized and translated to proper Win64 addressing+#    modes.+#+# 8. In order to provide for structured exception handling unified+#    Win64 prologue copies %rsp value to %rax. [Unless function is+#    tagged with additional .type tag.] For further details see SEH+#    paragraph at the end.+# 9. .init segment is allowed to contain calls to functions only.+# a. If function accepts more than 4 arguments *and* >4th argument+#    is declared as non 64-bit value, do clear its upper part.+++use strict;++my $flavour = shift;+my $output  = shift;+if ($flavour =~ /\./) { $output = $flavour; undef $flavour; }++open STDOUT,">$output" || die "can't open $output: $!"+	if (defined($output));++my $gas=1;	$gas=0 if ($output =~ /\.asm$/);+my $elf=1;	$elf=0 if (!$gas);+my $dwarf=$elf;+my $win64=0;+my $prefix="";+my $decor=".L";++my $masmref=8 + 50727*2**-32;	# 8.00.50727 shipped with VS2005+my $masm=0;+my $PTR=" PTR";++my $nasmref=2.03;+my $nasm=0;++if    ($flavour eq "mingw64")	{ $gas=1; $elf=0; $win64=1;+				  $prefix=`echo __USER_LABEL_PREFIX__ | \${CC:-false} -E -P -`;+				  $prefix =~ s|\R$||; # Better chomp+				}+elsif ($flavour eq "macosx")	{ $gas=1; $elf=0; $prefix="_"; $decor="L\$"; }+elsif ($flavour eq "masm")	{ $gas=0; $elf=0; $masm=$masmref; $win64=1; $decor="\$L\$"; }+elsif ($flavour eq "nasm")	{ $gas=0; $elf=0; $nasm=$nasmref; $win64=1; $decor="\$L\$"; $PTR=""; }+elsif (!$gas)+{   if ($ENV{ASM} =~ m/nasm/ && `nasm -v` =~ m/version ([0-9]+)\.([0-9]+)/i)+    {	$nasm = $1 + $2*0.01; $PTR="";  }+    elsif (`ml64 2>&1` =~ m/Version ([0-9]+)\.([0-9]+)(\.([0-9]+))?/)+    {	$masm = $1 + $2*2**-16 + $4*2**-32;   }+    die "no assembler found on %PATH%" if (!($nasm || $masm));+    $win64=1;+    $elf=0;+    $decor="\$L\$";+}+my $colon= $masm ? "::" : ":";++$dwarf=0 if($win64);++my $current_segment;+my $current_function;+my %globals;++{ package opcode;	# pick up opcodes+    sub re {+	my	($class, $line) = @_;+	my	$self = {};+	my	$ret;++	if ($$line =~ /^([a-z][a-z0-9]*)/i) {+	    bless $self,$class;+	    $self->{op} = $1;+	    $ret = $self;+	    $$line = substr($$line,@+[0]); $$line =~ s/^\s+//;++	    undef $self->{sz};+	    if ($self->{op} =~ /^(movz)x?([bw]).*/) {	# movz is pain...+		$self->{op} = $1;+		$self->{sz} = $2;+	    } elsif ($self->{op} =~ /cmov[n]?[lb]$/) {+		# pass through+	    } elsif ($self->{op} =~ /call|jmp/) {+		$self->{sz} = "";+	    } elsif ($self->{op} =~ /^p/ && $' !~ /^(ush|op|insrw)/) { # SSEn+		$self->{sz} = "";+	    } elsif ($self->{op} =~ /^[vk]/) { # VEX or k* such as kmov+		$self->{sz} = "";+	    } elsif ($self->{op} =~ /mov[dq]/ && $$line =~ /%xmm/) {+		$self->{sz} = "";+	    } elsif ($self->{op} =~ /([a-z]{3,})([qlwb])$/) {+		$self->{op} = $1;+		$self->{sz} = $2;+	    }+	}+	$ret;+    }+    sub size {+	my ($self, $sz) = @_;+	$self->{sz} = $sz if (defined($sz) && !defined($self->{sz}));+	$self->{sz};+    }+    sub out {+	my $self = shift;+	if ($gas) {+	    if ($self->{op} eq "movz") {	# movz is pain...+		sprintf "%s%s%s",$self->{op},$self->{sz},shift;+	    } elsif ($self->{op} =~ /^set/) {+		"$self->{op}";+	    } elsif ($self->{op} eq "ret") {+		my $epilogue = "";+		if ($win64 && $current_function->{abi} eq "svr4"+			   && !$current_function->{unwind}) {+		    $epilogue = "movq	8(%rsp),%rdi\n\t" .+				"movq	16(%rsp),%rsi\n\t";+		}+		$epilogue . ".byte	0xf3,0xc3";+	    } elsif ($self->{op} eq "call" && !$elf && $current_segment eq ".init") {+		".p2align\t3\n\t.quad";+	    } else {+		"$self->{op}$self->{sz}";+	    }+	} else {+	    $self->{op} =~ s/^movz/movzx/;+	    if ($self->{op} eq "ret") {+		$self->{op} = "";+		if ($win64 && $current_function->{abi} eq "svr4"+			   && !$current_function->{unwind}) {+		    $self->{op} = "mov	rdi,QWORD$PTR\[8+rsp\]\t;WIN64 epilogue\n\t".+				  "mov	rsi,QWORD$PTR\[16+rsp\]\n\t";+		}+		$self->{op} .= "DB\t0F3h,0C3h\t\t;repret";+	    } elsif ($self->{op} =~ /^(pop|push)f/) {+		$self->{op} .= $self->{sz};+	    } elsif ($self->{op} eq "call" && $current_segment eq ".CRT\$XCU") {+		$self->{op} = "\tDQ";+	    }+	    $self->{op};+	}+    }+    sub mnemonic {+	my ($self, $op) = @_;+	$self->{op}=$op if (defined($op));+	$self->{op};+    }+}+{ package const;	# pick up constants, which start with $+    sub re {+	my	($class, $line) = @_;+	my	$self = {};+	my	$ret;++	if ($$line =~ /^\$([^,]+)/) {+	    bless $self, $class;+	    $self->{value} = $1;+	    $ret = $self;+	    $$line = substr($$line,@+[0]); $$line =~ s/^\s+//;+	}+	$ret;+    }+    sub out {+	my $self = shift;++	$self->{value} =~ s/\b(0b[0-1]+)/oct($1)/eig;+	if ($gas) {+	    # Solaris /usr/ccs/bin/as can't handle multiplications+	    # in $self->{value}+	    my $value = $self->{value};+	    no warnings;    # oct might complain about overflow, ignore here...+	    $value =~ s/(?<![\w\$\.])(0x?[0-9a-f]+)/oct($1)/egi;+	    if ($value =~ s/([0-9]+\s*[\*\/\%]\s*[0-9]+)/eval($1)/eg) {+		$self->{value} = $value;+	    }+	    sprintf "\$%s",$self->{value};+	} else {+	    my $value = $self->{value};+	    $value =~ s/0x([0-9a-f]+)/0$1h/ig if ($masm);+	    sprintf "%s",$value;+	}+    }+}+{ package ea;		# pick up effective addresses: expr(%reg,%reg,scale)++    my %szmap = (	b=>"BYTE$PTR",    w=>"WORD$PTR",+			l=>"DWORD$PTR",   d=>"DWORD$PTR",+			q=>"QWORD$PTR",   o=>"OWORD$PTR",+			x=>"XMMWORD$PTR", y=>"YMMWORD$PTR",+			z=>"ZMMWORD$PTR" ) if (!$gas);++    my %sifmap = (	ss=>"d",	sd=>"q",	# broadcast only+			i32x2=>"q",	f32x2=>"q",+			i32x4=>"x",	i64x2=>"x",	i128=>"x",+			f32x4=>"x",	f64x2=>"x",	f128=>"x",+			i32x8=>"y",	i64x4=>"y",+			f32x8=>"y",	f64x4=>"y" ) if (!$gas);++    sub re {+	my	($class, $line, $opcode) = @_;+	my	$self = {};+	my	$ret;++	# optional * ----vvv--- appears in indirect jmp/call+	if ($$line =~ /^(\*?)([^\(,]*)\(([%\w,\s]+)\)((?:{[^}]+})*)/) {+	    bless $self, $class;+	    $self->{asterisk} = $1;+	    $self->{label} = $2;+	    ($self->{base},$self->{index},$self->{scale})=split(/(?:,\s*)/,$3);+	    $self->{scale} = 1 if (!defined($self->{scale}));+	    $self->{opmask} = $4;+	    $ret = $self;+	    $$line = substr($$line,@+[0]); $$line =~ s/^\s+//;++	    if ($win64 && $self->{label} =~ s/\@GOTPCREL//) {+		die if ($opcode->mnemonic() ne "mov");+		$opcode->mnemonic("lea");+	    }+	    $self->{base}  =~ s/^%//;+	    $self->{index} =~ s/^%// if (defined($self->{index}));+	    $self->{opcode} = $opcode;+	}+	$ret;+    }+    sub size {}+    sub out {+	my ($self, $sz) = @_;++	$self->{label} =~ s/([_a-z][_a-z0-9\$]*)/$globals{$1} or $1/gei;+	$self->{label} =~ s/\.L/$decor/g;++	# Silently convert all EAs to 64-bit. This is required for+	# elder GNU assembler and results in more compact code,+	# *but* most importantly AES module depends on this feature!+	$self->{index} =~ s/^[er](.?[0-9xpi])[d]?$/r\1/;+	$self->{base}  =~ s/^[er](.?[0-9xpi])[d]?$/r\1/;++	# Solaris /usr/ccs/bin/as can't handle multiplications+	# in $self->{label}...+	use integer;+	$self->{label} =~ s/(?<![\w\$\.])(0x?[0-9a-f]+)/oct($1)/egi;+	$self->{label} =~ s/\b([0-9]+\s*[\*\/\%]\s*[0-9]+)\b/eval($1)/eg;++	# Some assemblers insist on signed presentation of 32-bit+	# offsets, but sign extension is a tricky business in perl...+	$self->{label} =~ s/\b([0-9]+)\b/unpack("l",pack("L",$1))/eg;++	# if base register is %rbp or %r13, see if it's possible to+	# flip base and index registers [for better performance]+	if (!$self->{label} && $self->{index} && $self->{scale}==1 &&+	    $self->{base} =~ /(rbp|r13)/) {+		$self->{base} = $self->{index}; $self->{index} = $1;+	}++	if ($gas) {+	    $self->{label} =~ s/^___imp_/__imp__/   if ($flavour eq "mingw64");++	    if (defined($self->{index})) {+		sprintf "%s%s(%s,%%%s,%d)%s",+					$self->{asterisk},$self->{label},+					$self->{base}?"%$self->{base}":"",+					$self->{index},$self->{scale},+					$self->{opmask};+	    } else {+		sprintf "%s%s(%%%s)%s",	$self->{asterisk},$self->{label},+					$self->{base},$self->{opmask};+	    }+	} else {+	    $self->{label} =~ s/\./\$/g;+	    $self->{label} =~ s/(?<![\w\$\.])0x([0-9a-f]+)/0$1h/ig;+	    $self->{label} = "($self->{label})" if ($self->{label} =~ /[\*\+\-\/]/);++	    my $mnemonic = $self->{opcode}->mnemonic();+	    ($self->{asterisk})				&& ($sz="q") ||+	    ($mnemonic =~ /^v?mov([qd])$/)		&& ($sz=$1)  ||+	    ($mnemonic =~ /^v?pinsr([qdwb])$/)		&& ($sz=$1)  ||+	    ($mnemonic =~ /^vpbroadcast([qdwb])$/)	&& ($sz=$1)  ||+	    ($mnemonic =~ /^v(?:broadcast|extract|insert)([sif]\w+)$/)+							&& ($sz=$sifmap{$1});++	    $self->{opmask}  =~ s/%(k[0-7])/$1/;++	    if (defined($self->{index})) {+		sprintf "%s[%s%s*%d%s]%s",$szmap{$sz},+					$self->{label}?"$self->{label}+":"",+					$self->{index},$self->{scale},+					$self->{base}?"+$self->{base}":"",+					$self->{opmask};+	    } elsif ($self->{base} eq "rip") {+		sprintf "%s[%s]",$szmap{$sz},$self->{label};+	    } else {+		sprintf "%s[%s%s]%s",	$szmap{$sz},+					$self->{label}?"$self->{label}+":"",+					$self->{base},$self->{opmask};+	    }+	}+    }+}+{ package register;	# pick up registers, which start with %.+    sub re {+	my	($class, $line, $opcode) = @_;+	my	$self = {};+	my	$ret;++	# optional * ----vvv--- appears in indirect jmp/call+	if ($$line =~ /^(\*?)%(\w+)((?:{[^}]+})*)/) {+	    bless $self,$class;+	    $self->{asterisk} = $1;+	    $self->{value} = $2;+	    $self->{opmask} = $3;+	    $opcode->size($self->size());+	    $ret = $self;+	    $$line = substr($$line,@+[0]); $$line =~ s/^\s+//;+	}+	$ret;+    }+    sub size {+	my	$self = shift;+	my	$ret;++	if    ($self->{value} =~ /^r[\d]+b$/i)	{ $ret="b"; }+	elsif ($self->{value} =~ /^r[\d]+w$/i)	{ $ret="w"; }+	elsif ($self->{value} =~ /^r[\d]+d$/i)	{ $ret="l"; }+	elsif ($self->{value} =~ /^r[\w]+$/i)	{ $ret="q"; }+	elsif ($self->{value} =~ /^[a-d][hl]$/i){ $ret="b"; }+	elsif ($self->{value} =~ /^[\w]{2}l$/i)	{ $ret="b"; }+	elsif ($self->{value} =~ /^[\w]{2}$/i)	{ $ret="w"; }+	elsif ($self->{value} =~ /^e[a-z]{2}$/i){ $ret="l"; }++	$ret;+    }+    sub out {+	my $self = shift;+	if ($gas)	{ sprintf "%s%%%s%s",	$self->{asterisk},+						$self->{value},+						$self->{opmask}; }+	else		{ $self->{opmask} =~ s/%(k[0-7])/$1/;+			  $self->{value}.$self->{opmask}; }+    }+}+{ package label;	# pick up labels, which end with :+    sub re {+	my	($class, $line) = @_;+	my	$self = {};+	my	$ret;++	if ($$line =~ /(^[\.\w\$]+)\:/) {+	    bless $self,$class;+	    $self->{value} = $1;+	    $ret = $self;+	    $$line = substr($$line,@+[0]); $$line =~ s/^\s+//;++	    $self->{value} =~ s/^\.L/$decor/;+	}+	$ret;+    }+    sub win64_args {+	my $narg = $current_function->{narg} // 6;+	return undef if ($narg < 0);+	my $arg5 = 4*8 - cfi_directive::cfa_rsp();+	my $arg6 = $arg5 + 8;+	my $args;+	if ($gas) {+	    $args .= "	movq	%rcx,%rdi\n" if ($narg>0);+	    $args .= "	movq	%rdx,%rsi\n" if ($narg>1);+	    $args .= "	movq	%r8,%rdx\n"  if ($narg>2);+	    $args .= "	movq	%r9,%rcx\n"  if ($narg>3);+	    $args .= "	movq	$arg5(%rsp),%r8\n" if ($narg>4);+	    $args .= "	movq	$arg6(%rsp),%r9\n" if ($narg>5);+	} else {+	    $args .= "	mov	rdi,rcx\n" if ($narg>0);+	    $args .= "	mov	rsi,rdx\n" if ($narg>1);+	    $args .= "	mov	rdx,r8\n"  if ($narg>2);+	    $args .= "	mov	rcx,r9\n"  if ($narg>3);+	    $args .= "	mov	r8,QWORD$PTR\[$arg5+rsp\]\n" if ($narg>4);+	    $args .= "	mov	r9,QWORD$PTR\[$arg6+rsp\]\n" if ($narg>5);+	}+	$current_function->{narg} = -1;+	$args;+    }+    sub out {+	my $self = shift;++	if ($gas) {+	    my $func = ($globals{$self->{value}} or $self->{value}) . ":";+	    if ($current_function->{name} eq $self->{value}) {+		$current_function->{pc} = 0;+		$func .= "\n.cfi_".cfi_directive::startproc()   if ($dwarf);+		$func .= "\n	.byte	0xf3,0x0f,0x1e,0xfa\n";	# endbranch+		if ($win64) {+		    if ($current_function->{abi} eq "svr4") {+			my $fp = $current_function->{unwind} ? "%r11" : "%rax";+			$func .= "	movq	%rdi,8(%rsp)\n";+			$func .= "	movq	%rsi,16(%rsp)\n";+			$func .= "	movq	%rsp,$fp\n";+			$func .= "${decor}SEH_begin_$current_function->{name}:\n";+		    } elsif ($current_function->{unwind}) {+			$func .= "	movq	%rsp,%r11\n";+			$func .= "${decor}SEH_begin_$current_function->{name}:\n";+		    }+		}+	    } elsif ($win64 && $current_function->{abi} eq "svr4"+			    && $current_function->{pc} >= 0) {+		$func = win64_args().$func;+	    }+	    $func;+	} elsif ($self->{value} ne "$current_function->{name}") {+	    my $func;+	    if ($win64 && $current_function->{abi} eq "svr4"+		       && $current_function->{pc} >= 0) {+		$func = win64_args();+	    }+	    $func .= $self->{value} . $colon;+	    $func;+	} else {+	    $current_function->{pc} = 0;+	    my $func =	"$current_function->{name}" .+			($nasm ? ":" : "\tPROC $current_function->{scope}") .+			"\n";+	    $func .= "	DB	243,15,30,250\n";	# endbranch+	    if ($current_function->{abi} eq "svr4") {+		my $fp = $current_function->{unwind} ? "r11" : "rax";+		$func .= "	mov	QWORD$PTR\[8+rsp\],rdi\t;WIN64 prologue\n";+		$func .= "	mov	QWORD$PTR\[16+rsp\],rsi\n";+		$func .= "	mov	$fp,rsp\n";+		$func .= "${decor}SEH_begin_$current_function->{name}${colon}\n";+	    } elsif ($current_function->{unwind}) {+		$func .= "	mov	r11,rsp\n";+		$func .= "${decor}SEH_begin_$current_function->{name}${colon}\n";+	    }+	    $func;+	}+    }+}+{ package expr;		# pick up expressions+    sub re {+	my	($class, $line, $opcode) = @_;+	my	$self = {};+	my	$ret;++	if ($$line =~ /(^[^,]+)/) {+	    bless $self,$class;+	    $self->{value} = $1;+	    $ret = $self;+	    $$line = substr($$line,@+[0]); $$line =~ s/^\s+//;++	    $self->{value} =~ s/\@PLT// if (!$elf);+	    $self->{value} =~ s/([_a-z][_a-z0-9\$]*)/$globals{$1} or $1/gei;+	    $self->{value} =~ s/\.L/$decor/g;+	    $self->{opcode} = $opcode;+	}+	$ret;+    }+    sub out {+	my $self = shift;+	$self->{value};+    }+}++my @xdata_seg = (".section	.xdata", ".align	8");+my @pdata_seg = (".section	.pdata", ".align	4");++{ package cfi_directive;+    # CFI directives annotate instructions that are significant for+    # stack unwinding procedure compliant with DWARF specification,+    # see http://dwarfstd.org/. Besides naturally expected for this+    # script platform-specific filtering function, this module adds+    # four auxiliary synthetic directives not recognized by [GNU]+    # assembler:+    #+    # - .cfi_push to annotate push instructions in prologue, which+    #   translates to .cfi_adjust_cfa_offset (if needed) and+    #   .cfi_offset;+    # - .cfi_pop to annotate pop instructions in epilogue, which+    #   translates to .cfi_adjust_cfa_offset (if needed) and+    #   .cfi_restore;+    # - .cfi_alloca to annotate stack pointer adjustments, which+    #   translates to .cfi_adjust_cfa_offset as needed;+    # - [and most notably] .cfi_cfa_expression which encodes+    #   DW_CFA_def_cfa_expression and passes it to .cfi_escape as+    #   byte vector;+    #+    # CFA expressions were introduced in DWARF specification version+    # 3 and describe how to deduce CFA, Canonical Frame Address. This+    # becomes handy if your stack frame is variable and you can't+    # spare register for [previous] frame pointer. Suggested directive+    # syntax is made-up mix of DWARF operator suffixes [subset of]+    # and references to registers with optional bias. Following example+    # describes offloaded *original* stack pointer at specific offset+    # from *current* stack pointer:+    #+    #   .cfi_cfa_expression     %rsp+40,deref,+8+    #+    # Final +8 has everything to do with the fact that CFA is defined+    # as reference to top of caller's stack, and on x86_64 call to+    # subroutine pushes 8-byte return address. In other words original+    # stack pointer upon entry to a subroutine is 8 bytes off from CFA.+    #+    # In addition the .cfi directives are re-purposed even for Win64+    # stack unwinding. Two more synthetic directives were added:+    #+    # - .cfi_end_prologue to denote point when all non-volatile+    #   registers are saved and stack or [chosen] frame pointer is+    #   stable;+    # - .cfi_epilogue to denote point when all non-volatile registers+    #   are restored [and it even adds missing .cfi_restore-s];+    #+    # Though it's not universal "miracle cure," it has its limitations.+    # Most notably .cfi_cfa_expression won't start working... For more+    # information see the end of this file.++    # Below constants are taken from "DWARF Expressions" section of the+    # DWARF specification, section is numbered 7.7 in versions 3 and 4.+    my %DW_OP_simple = (	# no-arg operators, mapped directly+	deref	=> 0x06,	dup	=> 0x12,+	drop	=> 0x13,	over	=> 0x14,+	pick	=> 0x15,	swap	=> 0x16,+	rot	=> 0x17,	xderef	=> 0x18,++	abs	=> 0x19,	and	=> 0x1a,+	div	=> 0x1b,	minus	=> 0x1c,+	mod	=> 0x1d,	mul	=> 0x1e,+	neg	=> 0x1f,	not	=> 0x20,+	or	=> 0x21,	plus	=> 0x22,+	shl	=> 0x24,	shr	=> 0x25,+	shra	=> 0x26,	xor	=> 0x27,+	);++    my %DW_OP_complex = (	# used in specific subroutines+	constu		=> 0x10,	# uleb128+	consts		=> 0x11,	# sleb128+	plus_uconst	=> 0x23,	# uleb128+	lit0 		=> 0x30,	# add 0-31 to opcode+	reg0		=> 0x50,	# add 0-31 to opcode+	breg0		=> 0x70,	# add 0-31 to opcole, sleb128+	regx		=> 0x90,	# uleb28+	fbreg		=> 0x91,	# sleb128+	bregx		=> 0x92,	# uleb128, sleb128+	piece		=> 0x93,	# uleb128+	);++    # Following constants are defined in x86_64 ABI supplement, for+    # example available at https://www.uclibc.org/docs/psABI-x86_64.pdf,+    # see section 3.7 "Stack Unwind Algorithm".+    my %DW_reg_idx = (+	"%rax"=>0,  "%rdx"=>1,  "%rcx"=>2,  "%rbx"=>3,+	"%rsi"=>4,  "%rdi"=>5,  "%rbp"=>6,  "%rsp"=>7,+	"%r8" =>8,  "%r9" =>9,  "%r10"=>10, "%r11"=>11,+	"%r12"=>12, "%r13"=>13, "%r14"=>14, "%r15"=>15+	);++    my ($cfa_reg, $cfa_off, $cfa_rsp, %saved_regs);+    my @cfa_stack;++    sub cfa_rsp { return $cfa_rsp // -8;  }++    # [us]leb128 format is variable-length integer representation base+    # 2^128, with most significant bit of each byte being 0 denoting+    # *last* most significant digit. See "Variable Length Data" in the+    # DWARF specification, numbered 7.6 at least in versions 3 and 4.+    sub sleb128 {+	use integer;	# get right shift extend sign++	my $val = shift;+	my $sign = ($val < 0) ? -1 : 0;+	my @ret = ();++	while(1) {+	    push @ret, $val&0x7f;++	    # see if remaining bits are same and equal to most+	    # significant bit of the current digit, if so, it's+	    # last digit...+	    last if (($val>>6) == $sign);++	    @ret[-1] |= 0x80;+	    $val >>= 7;+	}++	return @ret;+    }+    sub uleb128 {+	my $val = shift;+	my @ret = ();++	while(1) {+	    push @ret, $val&0x7f;++	    # see if it's last significant digit...+	    last if (($val >>= 7) == 0);++	    @ret[-1] |= 0x80;+	}++	return @ret;+    }+    sub const {+	my $val = shift;++	if ($val >= 0 && $val < 32) {+	    return ($DW_OP_complex{lit0}+$val);+	}+	return ($DW_OP_complex{consts}, sleb128($val));+    }+    sub reg {+	my $val = shift;++	return if ($val !~ m/^(%r\w+)(?:([\+\-])((?:0x)?[0-9a-f]+))?/);++	my $reg = $DW_reg_idx{$1};+	my $off = eval ("0 $2 $3");++	return (($DW_OP_complex{breg0} + $reg), sleb128($off));+	# Yes, we use DW_OP_bregX+0 to push register value and not+	# DW_OP_regX, because latter would require even DW_OP_piece,+	# which would be a waste under the circumstances. If you have+	# to use DWP_OP_reg, use "regx:N"...+    }+    sub cfa_expression {+	my $line = shift;+	my @ret;++	foreach my $token (split(/,\s*/,$line)) {+	    if ($token =~ /^%r/) {+		push @ret,reg($token);+	    } elsif ($token =~ /((?:0x)?[0-9a-f]+)\((%r\w+)\)/) {+		push @ret,reg("$2+$1");+	    } elsif ($token =~ /(\w+):(\-?(?:0x)?[0-9a-f]+)(U?)/i) {+		my $i = 1*eval($2);+		push @ret,$DW_OP_complex{$1}, ($3 ? uleb128($i) : sleb128($i));+	    } elsif (my $i = 1*eval($token) or $token eq "0") {+		if ($token =~ /^\+/) {+		    push @ret,$DW_OP_complex{plus_uconst},uleb128($i);+		} else {+		    push @ret,const($i);+		}+	    } else {+		push @ret,$DW_OP_simple{$token};+	    }+	}++	# Finally we return DW_CFA_def_cfa_expression, 15, followed by+	# length of the expression and of course the expression itself.+	return (15,scalar(@ret),@ret);+    }++    # Following constants are defined in "x64 exception handling" at+    # https://docs.microsoft.com/ and match the register sequence in+    # CONTEXT structure defined in winnt.h.+    my %WIN64_reg_idx = (+	"%rax"=>0,  "%rcx"=>1,  "%rdx"=>2,  "%rbx"=>3,+	"%rsp"=>4,  "%rbp"=>5,  "%rsi"=>6,  "%rdi"=>7,+	"%r8" =>8,  "%r9" =>9,  "%r10"=>10, "%r11"=>11,+	"%r12"=>12, "%r13"=>13, "%r14"=>14, "%r15"=>15+	);+    sub xdata {+	our @dat = ();+	our $len = 0;++	sub savereg {+	    my ($key, $offset) = @_;++	    if ($key =~ /%xmm([0-9]+)/) {+		if ($offset < 0x100000) {+		    push @dat, [0,($1<<4)|8,unpack("C2",pack("v",$offset>>4))];+		} else {+		    push @dat, [0,($1<<4)|9,unpack("C4",pack("V",$offset))];+		}+	    } else {+		if ($offset < 0x80000) {+		    push @dat, [0,(($WIN64_reg_idx{$key})<<4)|4,+				unpack("C2",pack("v",$offset>>3))];+		} else {+		    push @dat, [0,(($WIN64_reg_idx{$key})<<4)|5,+				unpack("C4",pack("V",$offset))];+		}+	    }+	    $len += $#{@dat[-1]}+1;+	}++	my $fp_info = 0;++	# allocate stack frame+	if ($cfa_rsp < -8) {+	    my $offset = -8 - $cfa_rsp;+	    if ($cfa_reg ne "%rsp" && $saved_regs{$cfa_reg} == -16) {+		$fp_info = $WIN64_reg_idx{$cfa_reg};+		push @dat, [0,$fp_info<<4];		# UWOP_PUSH_NONVOL+		$len += $#{@dat[-1]}+1;+		$offset -= 8;+	    }+	    if ($offset <= 128) {+		my $alloc = ($offset - 8) >> 3;+		push @dat, [0,$alloc<<4|2];		# UWOP_ALLOC_SMALL+	    } elsif ($offset < 0x80000) {+		push @dat, [0,0x01,unpack("C2",pack("v",$offset>>3))];+	    } else {+		push @dat, [0,0x11,unpack("C4",pack("V",$offset))];+	    }+	    $len += $#{@dat[-1]}+1;+	}++	# save frame pointer [if not pushed already]+	if ($cfa_reg ne "%rsp" && $fp_info == 0) {+	    $fp_info = $WIN64_reg_idx{$cfa_reg};+	    if (defined(my $offset = $saved_regs{$cfa_reg})) {+		$offset -= $cfa_rsp;+		savereg($cfa_reg, $offset);+	    }+	}++	# set up frame pointer+	if ($fp_info) {+	    push @dat, [0,($fp_info<<4)|3];		# UWOP_SET_FPREG+	    $len += $#{@dat[-1]}+1;+	    my $fp_off = $cfa_off - $cfa_rsp;+	    ($fp_off > 240 or $fp_off&0xf) and die "invalid FP offset $fp_off";+	    $fp_info |= $fp_off&-16;+	}++	# save registers+	foreach my $key (sort { $saved_regs{$b} <=> $saved_regs{$a} }+			      keys(%saved_regs)) {+	    next if ($cfa_reg ne "%rsp" && $cfa_reg eq $key);+	    my $offset = $saved_regs{$key} - $cfa_rsp;+	    savereg($key, $offset);+	}++	my @ret;+	# generate 4-byte descriptor+	push @ret, ".byte	1,0,".($len/2).",$fp_info";+	$len += 4;+	# keep objdump happy, pad to 4*n and add a 32-bit zero+	unshift @dat, [(0)x(((-$len)&3)+4)];+	$len += $#{@dat[0]}+1;+	# pad to 8*n+	unshift @dat, [(0)x((-$len)&7)] if ($len&7);+	# emit data+	while(defined(my $row = pop @dat)) {+	    push @ret, ".byte	". join(",",+					map { sprintf "0x%02x",$_ } @{$row});+	}++	return @ret;+    }+    sub startproc {+	return if ($cfa_rsp == -8);+	($cfa_reg, $cfa_off, $cfa_rsp) = ("%rsp", -8, -8);+	%saved_regs = ();+	return "startproc";+    }+    sub endproc {+	return if ($cfa_rsp == 0);+	($cfa_reg, $cfa_off, $cfa_rsp) = ("%rsp", 0, 0);+	%saved_regs = ();+	return "endproc";+    }+    sub re {+	my	($class, $line) = @_;+	my	$self = {};+	my	$ret;++	if ($$line =~ s/^\s*\.cfi_(\w+)\s*//) {+	    bless $self,$class;+	    $ret = $self;+	    undef $self->{value};+	    my $dir = $1;++	    SWITCH: for ($dir) {+	    # What is $cfa_rsp? Effectively it's difference between %rsp+	    # value and current CFA, Canonical Frame Address, which is+	    # why it starts with -8. Recall that CFA is top of caller's+	    # stack...+	    /startproc/	&& do {	$dir = startproc(); last; };+	    /endproc/	&& do {	$dir = endproc();+				# .cfi_remember_state directives that are not+				# matched with .cfi_restore_state are+				# unnecessary.+				die "unpaired .cfi_remember_state" if (@cfa_stack);+				last;+			      };+	    /def_cfa_register/+			&& do {	$cfa_off = $cfa_rsp if ($cfa_reg eq "%rsp");+				$cfa_reg = $$line;+				$cfa_rsp = $cfa_off if ($cfa_reg eq "%rsp");+				last;+			      };+	    /def_cfa_offset/+			&& do {	$cfa_off = -1*eval($$line);+				$cfa_rsp = $cfa_off if ($cfa_reg eq "%rsp");+				last;+			      };+	    /adjust_cfa_offset/+			&& do { my $val = 1*eval($$line);+				$cfa_off -= $val;+				if ($cfa_reg eq "%rsp") {+				    $cfa_rsp -= $val;+				}+				$$line = "$val";+				last;+			      };+	    /alloca/	&& do { $dir = undef;+				my $val = 1*eval($$line);+				$cfa_rsp -= $val;+				if ($cfa_reg eq "%rsp") {+				    $cfa_off -= $val;+				    $dir = "adjust_cfa_offset";+				}+				$$line = "$val";+				last;+			      };+	    /def_cfa/	&& do {	if ($$line =~ /(%r\w+)\s*(?:,\s*(.+))?/) {+				    $cfa_reg = $1;+				    if ($cfa_reg eq "%rsp" && !defined($2)) {+					$cfa_off = $cfa_rsp;+					$$line .= ",".(-$cfa_rsp);+				    } else {+					$cfa_off = -1*eval($2);+					$cfa_rsp = $cfa_off if ($cfa_reg eq "%rsp");+				    }+				}+				last;+			      };+	    /push/	&& do {	$dir = undef;+				$cfa_rsp -= 8;+				if ($cfa_reg eq "%rsp") {+				    $cfa_off = $cfa_rsp;+				    $self->{value} = ".cfi_adjust_cfa_offset\t8\n";+				}+				$saved_regs{$$line} = $cfa_rsp;+				$self->{value} .= ".cfi_offset\t$$line,$cfa_rsp";+				last;+			      };+	    /pop/	&& do {	$dir = undef;+				$cfa_rsp += 8;+				if ($cfa_reg eq "%rsp") {+				    $cfa_off = $cfa_rsp;+				    $self->{value} = ".cfi_adjust_cfa_offset\t-8\n";+				}+				$self->{value} .= ".cfi_restore\t$$line";+				delete $saved_regs{$$line};+				last;+			      };+	    /cfa_expression/+			&& do {	$dir = undef;+				$self->{value} = ".cfi_escape\t" .+					join(",", map(sprintf("0x%02x", $_),+						      cfa_expression($$line)));+				last;+			      };+	    /remember_state/+			&& do {	push @cfa_stack,+				     [$cfa_reg,$cfa_off,$cfa_rsp,%saved_regs];+				last;+			      };+	    /restore_state/+			&& do {	     ($cfa_reg,$cfa_off,$cfa_rsp,%saved_regs)+				= @{pop @cfa_stack};+				last;+			      };+	    /offset/	&& do { if ($$line =~ /(%\w+)(?:-%xmm(\d+))?\s*,\s*(.+)/) {+				    my ($reg, $off, $xmmlast) = ($1, 1*eval($3), $2);+				    if ($reg !~ /%xmm(\d+)/) {+					$saved_regs{$reg} = $off;+				    } else {+					$dir = undef;+					$xmmlast //= $1;+					for (my $i=$1; $i<=$xmmlast; $i++) {+					    $saved_regs{"%xmm$i"} = $off;+					    $off += 16;+					}+				    }+				}+				last;+			      };+	    /restore/	&& do {	delete $saved_regs{$$line}; last; };+	    /end_prologue/+			&& do {	$dir = undef;+				$self->{win64} = ".endprolog";+				last;+			      };+	    /epilogue/	&& do {	$dir = undef;+				$self->{win64} = ".epilogue";+				$self->{value} = join("\n",+						      map { ".cfi_restore\t$_" }+						      sort keys(%saved_regs));+				%saved_regs = ();+				last;+			      };+	    }++	    $self->{value} = ".cfi_$dir\t$$line" if ($dir);++	    $$line = "";+	}++	return $ret;+    }+    sub out {+	my $self = shift;+	return $self->{value} if ($dwarf);++	if ($win64 and $current_function->{unwind}+		   and my $ret = $self->{win64}) {+	    my ($reg, $off) = ($cfa_reg =~ /%(?!rsp)/)  ? ($',    $cfa_off)+							: ("rsp", $cfa_rsp);+	    my $fname = $current_function->{name};++	    if ($ret eq ".endprolog") {+		$ret = "";+		if ($current_function->{abi} eq "svr4") {+		    $ret .= label::win64_args();+		    $saved_regs{"%rdi"} = 0;	# relative to CFA, remember?+		    $saved_regs{"%rsi"} = 8;+		}++		push @pdata_seg,+		    ".rva	.LSEH_begin_${fname}",+		    ".rva	.LSEH_body_${fname}",+		    ".rva	.LSEH_info_${fname}_prologue","";+		push @xdata_seg,+		    ".LSEH_info_${fname}_prologue:";+		if ($current_function->{unwind} eq "%rbp") {+		    if ($current_function->{abi} eq "svr4") {+			push @xdata_seg,+			".byte	1,4,6,0x05",	# 6 unwind codes, %rbp is FP+			".byte	4,0x74,2,0",	# %rdi at 16(%rsp)+			".byte	4,0x64,3,0",	# %rsi at 24(%rsp)+			".byte	4,0x53",	# mov	%rsp, %rbp+			".byte	1,0x50",	# push	%rbp+			".long	0,0"		# pad to keep objdump happy+			;+		    } else {+			push @xdata_seg,+			".byte	1,4,2,0x05",	# 2 unwind codes, %rbp is FP+			".byte	4,0x53",	# mov	%rsp, %rbp+			".byte	1,0x50",	# push	%rbp+			".long	0,0"		# pad to keep objdump happy+			;+		    }+		} else {+		    if ($current_function->{abi} eq "svr4") {+			push @xdata_seg,+			".byte	1,0,5,0x0b",	# 5 unwind codes, %r11 is FP+			".byte	0,0x74,1,0",	# %rdi at 8(%rsp)+			".byte	0,0x64,2,0",	# %rsi at 16(%rsp)+			".byte	0,0xb3",	# set frame pointer+			".byte	0,0",		# padding+			".long	0,0"		# pad to keep objdump happy+			;+		    } else {+			push @xdata_seg,+			".byte	1,0,1,0x0b",	# 1 unwind code, %r11 is FP+			".byte	0,0xb3",	# set frame pointer+			".byte	0,0",		# padding+			".long	0,0"		# pad to keep objdump happy+			;+		    }+		}+		push @pdata_seg,+		    ".rva	.LSEH_body_${fname}",+		    ".rva	.LSEH_epilogue_${fname}",+		    ".rva	.LSEH_info_${fname}_body","";+		push @xdata_seg,".LSEH_info_${fname}_body:", xdata();+		$ret .= "${decor}SEH_body_${fname}${colon}\n";+	    } elsif ($ret eq ".epilogue") {+		%saved_regs = ();+		$cfa_rsp = $cfa_off;+		$ret = "${decor}SEH_epilogue_${fname}${colon}\n";+		if ($current_function->{abi} eq "svr4") {+		    $saved_regs{"%rdi"} = 0;	# relative to CFA, remember?+		    $saved_regs{"%rsi"} = 8;++		    push @pdata_seg,+			".rva	.LSEH_epilogue_${fname}",+			".rva	.LSEH_end_${fname}",+			".rva	.LSEH_info_${fname}_epilogue","";+		    push @xdata_seg,".LSEH_info_${fname}_epilogue:", xdata(), "";+		    if ($gas) {+			$ret .= "	mov	".(0-$off)."(%$reg),%rdi\n";+			$ret .= "	mov	".(8-$off)."(%$reg),%rsi\n";+		    } else {+			$ret .= "	mov	rdi,QWORD$PTR\[".(0-$off)."+$reg\]";+			$ret .= "	;WIN64 epilogue\n";+			$ret .= "	mov	rsi,QWORD$PTR\[".(8-$off)."+$reg\]\n";+		    }+		}+	    }+	    return $ret;+	}+	return;+    }+}+{ package directive;	# pick up directives, which start with .+    sub re {+	my	($class, $line) = @_;+	my	$self = {};+	my	$ret;+	my	$dir;++	# chain-call to cfi_directive+	$ret = cfi_directive->re($line) and return $ret;++	if ($$line =~ /^\s*(\.\w+)/) {+	    bless $self,$class;+	    $dir = $1;+	    $ret = $self;+	    undef $self->{value};+	    $$line = substr($$line,@+[0]); $$line =~ s/^\s+//;++	    SWITCH: for ($dir) {+		/\.global|\.globl|\.extern|\.comm/+			    && do { $$line =~ s/([_a-z][_a-z0-9\$]*)/$prefix\1/gi;+				    $globals{$1} = $prefix.$1 if ($1);+				    last;+				  };+		/\.type/    && do { my ($sym,$type,$narg,$unwind) = split(',',$$line);+				    if ($type eq "\@function") {+					undef $current_function;+					$current_function->{name} = $sym;+					$current_function->{abi}  = "svr4";+					$current_function->{narg} = $narg;+					$current_function->{scope} = defined($globals{$sym})?"PUBLIC":"PRIVATE";+					$current_function->{unwind} = $unwind;+					$current_function->{pc} = -1;+				    } elsif ($type eq "\@abi-omnipotent") {+					undef $current_function;+					$current_function->{name} = $sym;+					$current_function->{scope} = defined($globals{$sym})?"PUBLIC":"PRIVATE";+					$current_function->{unwind} = $unwind;+					$current_function->{pc} = -1;+				    }+				    $$line =~ s/\@abi\-omnipotent/\@function/;+				    $$line =~ s/\@function.*/\@function/;+				    last;+				  };+		/\.asciz/   && do { if ($$line =~ /^"(.*)"$/) {+					$dir  = ".byte";+					$$line = join(",",unpack("C*",$1),0);+				    }+				    last;+				  };+		/\.rva|\.long|\.quad/+			    && do { $$line =~ s/([_a-z][_a-z0-9\$]*)/$globals{$1} or $1/gei;+				    $$line =~ s/\.L/$decor/g;+				    last;+				  };+	    }++	    if ($gas) {+		$self->{value} = $dir . "\t" . $$line;++		if ($dir =~ /\.extern/) {+		    $self->{value} = ""; # swallow extern+		} elsif (!$elf && $dir =~ /\.type/) {+		    $self->{value} = "";+		    $self->{value} = ".def\t" . ($globals{$1} or $1) . ";\t" .+				(defined($globals{$1})?".scl 2;":".scl 3;") .+				"\t.type 32;\t.endef"+				if ($win64 && $$line =~ /([^,]+),\@function/);+		} elsif ($dir =~ /\.size/) {+		    $self->{value} = "" if (!$elf);+		    if ($dwarf and my $endproc = cfi_directive::endproc()) {+			$self->{value} = ".cfi_$endproc\n$self->{value}";+		    } elsif (!$elf && defined($current_function)) {+			$self->{value} .= "${decor}SEH_end_$current_function->{name}:"+				if ($win64 && $current_function->{abi} eq "svr4");+			undef $current_function;+		    }+		} elsif (!$elf && $dir =~ /\.align/) {+		    $self->{value} = ".p2align\t" . (log($$line)/log(2));+		} elsif ($dir eq ".section") {+		    $current_segment=$$line;+		    if (!$elf && $current_segment eq ".init") {+			if	($flavour eq "macosx")	{ $self->{value} = ".mod_init_func"; }+			elsif	($flavour eq "mingw64")	{ $self->{value} = ".section\t.ctors"; }+		    }+		    if (!$elf && $current_segment eq ".rodata") {+			if	($flavour eq "macosx")	{ $self->{value} = ".section\t__TEXT,__const"; }+			elsif	($flavour eq "mingw64")	{ $self->{value} = ".section\t.rdata"; }+		    }+		} elsif ($dir =~ /\.(text|data)/) {+		    $current_segment=".$1";+		} elsif ($dir =~ /\.hidden/) {+		    if    ($flavour eq "macosx")  { $self->{value} = ".private_extern\t$prefix$$line"; }+		    elsif ($flavour eq "mingw64") { $self->{value} = ""; }+		} elsif ($dir =~ /\.comm/) {+		    $self->{value} = "$dir\t$$line";+		    $self->{value} =~ s|,([0-9]+),([0-9]+)$|",$1,".log($2)/log(2)|e if ($flavour eq "macosx");+		}+		$$line = "";+		return $self;+	    }++	    # non-gas case or nasm/masm+	    SWITCH: for ($dir) {+		/\.text/    && do { my $v=undef;+				    if ($nasm) {+					$v="section	.text code align=64\n";+				    } else {+					$v="$current_segment\tENDS\n" if ($current_segment);+					$current_segment = ".text\$";+					$v.="$current_segment\tSEGMENT ";+					$v.=$masm>=$masmref ? "ALIGN(256)" : "PAGE";+					$v.=" 'CODE'";+				    }+				    $self->{value} = $v;+				    last;+				  };+		/\.data/    && do { my $v=undef;+				    if ($nasm) {+					$v="section	.data data align=8\n";+				    } else {+					$v="$current_segment\tENDS\n" if ($current_segment);+					$current_segment = "_DATA";+					$v.="$current_segment\tSEGMENT";+				    }+				    $self->{value} = $v;+				    last;+				  };+		/\.section/ && do { my $v=undef;+				    $$line =~ s/([^,]*).*/$1/;+				    $$line = ".CRT\$XCU" if ($$line eq ".init");+				    $$line = ".rdata" if ($$line eq ".rodata");+				    my %align = ( p=>4, x=>8, r=>256);+				    if ($nasm) {+					$v="section	$$line";+					if ($$line=~/\.([pxr])data/) {+					    $v.=" rdata align=$align{$1}";+					} elsif ($$line=~/\.CRT\$/i) {+					    $v.=" rdata align=8";+					}+				    } else {+					$v="$current_segment\tENDS\n" if ($current_segment);+					$v.="$$line\tSEGMENT";+					if ($$line=~/\.([pxr])data/) {+					    $v.=" READONLY";+					    $v.=" ALIGN($align{$1})" if ($masm>=$masmref);+					} elsif ($$line=~/\.CRT\$/i) {+					    $v.=" READONLY ";+					    $v.=$masm>=$masmref ? "ALIGN(8)" : "DWORD";+					}+				    }+				    $current_segment = $$line;+				    $self->{value} = $v;+				    last;+				  };+		/\.extern/  && do { $self->{value}  = "EXTERN\t".$$line;+				    $self->{value} .= ":NEAR" if ($masm);+				    last;+				  };+		/\.globl|.global/+			    && do { $self->{value}  = $masm?"PUBLIC":"global";+				    $self->{value} .= "\t".$$line;+				    last;+				  };+		/\.size/    && do { if (defined($current_function)) {+					undef $self->{value};+					if ($current_function->{abi} eq "svr4") {+					    $self->{value}="${decor}SEH_end_$current_function->{name}${colon}\n";+					}+					$self->{value}.="$current_function->{name}\tENDP" if($masm && $current_function->{name});+					undef $current_function;+				    }+				    last;+				  };+		/\.align/   && do { my $max = ($masm && $masm>=$masmref) ? 256 : 4096;+				    $self->{value} = "ALIGN\t".($$line>$max?$max:$$line);+				    last;+				  };+		/\.(value|long|rva|quad)/+			    && do { my $sz  = substr($1,0,1);+				    my @arr = split(/,\s*/,$$line);+				    my $last = pop(@arr);+				    my $conv = sub  {	my $var=shift;+							$var=~s/^(0b[0-1]+)/oct($1)/eig;+							$var=~s/^0x([0-9a-f]+)/0$1h/ig if ($masm);+							if ($sz eq "D" && ($current_segment=~/.[px]data/ || $dir eq ".rva"))+							{ $var=~s/^([_a-z\$\@][_a-z0-9\$\@]*)/$nasm?"$1 wrt ..imagebase":"imagerel $1"/egi; }+							$var;+						    };++				    $sz =~ tr/bvlrq/BWDDQ/;+				    $self->{value} = "\tD$sz\t";+				    for (@arr) { $self->{value} .= &$conv($_).","; }+				    $self->{value} .= &$conv($last);+				    last;+				  };+		/\.byte/    && do { my @str=split(/,\s*/,$$line);+				    map(s/(0b[0-1]+)/oct($1)/eig,@str);+				    map(s/0x([0-9a-f]+)/0$1h/ig,@str) if ($masm);+				    while ($#str>15) {+					$self->{value}.="DB\t"+						.join(",",@str[0..15])."\n";+					foreach (0..15) { shift @str; }+				    }+				    $self->{value}.="DB\t"+						.join(",",@str) if (@str);+				    last;+				  };+		/\.comm/    && do { my @str=split(/,\s*/,$$line);+				    my $v=undef;+				    if ($nasm) {+					$v.="common	$prefix@str[0] @str[1]";+				    } else {+					$v="$current_segment\tENDS\n" if ($current_segment);+					$current_segment = "_DATA";+					$v.="$current_segment\tSEGMENT\n";+					$v.="COMM	@str[0]:DWORD:".@str[1]/4;+				    }+				    $self->{value} = $v;+				    last;+				  };+	    }+	    $$line = "";+	}++	$ret;+    }+    sub out {+	my $self = shift;+	$self->{value};+    }+}++# Upon initial x86_64 introduction SSE>2 extensions were not introduced+# yet. In order not to be bothered by tracing exact assembler versions,+# but at the same time to provide a bare security minimum of AES-NI, we+# hard-code some instructions. Extensions past AES-NI on the other hand+# are traced by examining assembler version in individual perlasm+# modules...++my %regrm = (	"%eax"=>0, "%ecx"=>1, "%edx"=>2, "%ebx"=>3,+		"%esp"=>4, "%ebp"=>5, "%esi"=>6, "%edi"=>7	);++sub rex {+ my $opcode=shift;+ my ($dst,$src,$rex)=@_;++   $rex|=0x04 if($dst>=8);+   $rex|=0x01 if($src>=8);+   push @$opcode,($rex|0x40) if ($rex);+}++my $movq = sub {	# elderly gas can't handle inter-register movq+  my $arg = shift;+  my @opcode=(0x66);+    if ($arg =~ /%xmm([0-9]+),\s*%r(\w+)/) {+	my ($src,$dst)=($1,$2);+	if ($dst !~ /[0-9]+/)	{ $dst = $regrm{"%e$dst"}; }+	rex(\@opcode,$src,$dst,0x8);+	push @opcode,0x0f,0x7e;+	push @opcode,0xc0|(($src&7)<<3)|($dst&7);	# ModR/M+	@opcode;+    } elsif ($arg =~ /%r(\w+),\s*%xmm([0-9]+)/) {+	my ($src,$dst)=($2,$1);+	if ($dst !~ /[0-9]+/)	{ $dst = $regrm{"%e$dst"}; }+	rex(\@opcode,$src,$dst,0x8);+	push @opcode,0x0f,0x6e;+	push @opcode,0xc0|(($src&7)<<3)|($dst&7);	# ModR/M+	@opcode;+    } else {+	();+    }+};++my $pextrd = sub {+    if (shift =~ /\$([0-9]+),\s*%xmm([0-9]+),\s*(%\w+)/) {+      my @opcode=(0x66);+	my $imm=$1;+	my $src=$2;+	my $dst=$3;+	if ($dst =~ /%r([0-9]+)d/)	{ $dst = $1; }+	elsif ($dst =~ /%e/)		{ $dst = $regrm{$dst}; }+	rex(\@opcode,$src,$dst);+	push @opcode,0x0f,0x3a,0x16;+	push @opcode,0xc0|(($src&7)<<3)|($dst&7);	# ModR/M+	push @opcode,$imm;+	@opcode;+    } else {+	();+    }+};++my $pinsrd = sub {+    if (shift =~ /\$([0-9]+),\s*(%\w+),\s*%xmm([0-9]+)/) {+      my @opcode=(0x66);+	my $imm=$1;+	my $src=$2;+	my $dst=$3;+	if ($src =~ /%r([0-9]+)/)	{ $src = $1; }+	elsif ($src =~ /%e/)		{ $src = $regrm{$src}; }+	rex(\@opcode,$dst,$src);+	push @opcode,0x0f,0x3a,0x22;+	push @opcode,0xc0|(($dst&7)<<3)|($src&7);	# ModR/M+	push @opcode,$imm;+	@opcode;+    } else {+	();+    }+};++my $pshufb = sub {+    if (shift =~ /%xmm([0-9]+),\s*%xmm([0-9]+)/) {+      my @opcode=(0x66);+	rex(\@opcode,$2,$1);+	push @opcode,0x0f,0x38,0x00;+	push @opcode,0xc0|($1&7)|(($2&7)<<3);		# ModR/M+	@opcode;+    } else {+	();+    }+};++my $palignr = sub {+    if (shift =~ /\$([0-9]+),\s*%xmm([0-9]+),\s*%xmm([0-9]+)/) {+      my @opcode=(0x66);+	rex(\@opcode,$3,$2);+	push @opcode,0x0f,0x3a,0x0f;+	push @opcode,0xc0|($2&7)|(($3&7)<<3);		# ModR/M+	push @opcode,$1;+	@opcode;+    } else {+	();+    }+};++my $pclmulqdq = sub {+    if (shift =~ /\$([x0-9a-f]+),\s*%xmm([0-9]+),\s*%xmm([0-9]+)/) {+      my @opcode=(0x66);+	rex(\@opcode,$3,$2);+	push @opcode,0x0f,0x3a,0x44;+	push @opcode,0xc0|($2&7)|(($3&7)<<3);		# ModR/M+	my $c=$1;+	push @opcode,$c=~/^0/?oct($c):$c;+	@opcode;+    } else {+	();+    }+};++my $rdrand = sub {+    if (shift =~ /%[er](\w+)/) {+      my @opcode=();+      my $dst=$1;+	if ($dst !~ /[0-9]+/) { $dst = $regrm{"%e$dst"}; }+	rex(\@opcode,0,$dst,8);+	push @opcode,0x0f,0xc7,0xf0|($dst&7);+	@opcode;+    } else {+	();+    }+};++my $rdseed = sub {+    if (shift =~ /%[er](\w+)/) {+      my @opcode=();+      my $dst=$1;+	if ($dst !~ /[0-9]+/) { $dst = $regrm{"%e$dst"}; }+	rex(\@opcode,0,$dst,8);+	push @opcode,0x0f,0xc7,0xf8|($dst&7);+	@opcode;+    } else {+	();+    }+};++# Not all AVX-capable assemblers recognize AMD XOP extension. Since we+# are using only two instructions hand-code them in order to be excused+# from chasing assembler versions...++sub rxb {+ my $opcode=shift;+ my ($dst,$src1,$src2,$rxb)=@_;++   $rxb|=0x7<<5;+   $rxb&=~(0x04<<5) if($dst>=8);+   $rxb&=~(0x01<<5) if($src1>=8);+   $rxb&=~(0x02<<5) if($src2>=8);+   push @$opcode,$rxb;+}++my $vprotd = sub {+    if (shift =~ /\$([x0-9a-f]+),\s*%xmm([0-9]+),\s*%xmm([0-9]+)/) {+      my @opcode=(0x8f);+	rxb(\@opcode,$3,$2,-1,0x08);+	push @opcode,0x78,0xc2;+	push @opcode,0xc0|($2&7)|(($3&7)<<3);		# ModR/M+	my $c=$1;+	push @opcode,$c=~/^0/?oct($c):$c;+	@opcode;+    } else {+	();+    }+};++my $vprotq = sub {+    if (shift =~ /\$([x0-9a-f]+),\s*%xmm([0-9]+),\s*%xmm([0-9]+)/) {+      my @opcode=(0x8f);+	rxb(\@opcode,$3,$2,-1,0x08);+	push @opcode,0x78,0xc3;+	push @opcode,0xc0|($2&7)|(($3&7)<<3);		# ModR/M+	my $c=$1;+	push @opcode,$c=~/^0/?oct($c):$c;+	@opcode;+    } else {+	();+    }+};++# Intel Control-flow Enforcement Technology extension. All functions and+# indirect branch targets will have to start with this instruction...+# However, it should not be used in functions' prologues explicitly, as+# it's added automatically [and in the right spot]. Which leaves only+# non-function indirect branch targets, such as in a case-like dispatch+# table, as application area.++my $endbr64 = sub {+    (0xf3,0x0f,0x1e,0xfa);+};++########################################################################++my $preproc_prefix = "#";++if ($nasm) {+    $preproc_prefix = "%";+    print <<___;+default	rel+%define XMMWORD+%define YMMWORD+%define ZMMWORD+___+} elsif ($masm) {+    $preproc_prefix = "";+    print <<___;+OPTION	DOTNAME+___+}++sub process {+    my $line = shift;++    $line =~ s|\R$||;		# Better chomp++    if ($line =~ m/^#\s*(if|elif|else|endif)(.*)/) {	# pass through preproc+	if ($win64 && $current_function->{abi} eq "svr4"+		   && $current_function->{narg} >= 0) {+	    print label::win64_args();+	}+	print $preproc_prefix,$1,$2,"\n";+	next;+    }++    print $1 if ($line =~ s|(\{\w+\})||);++    $line =~ s|[#!].*$||;	# get rid of asm-style comments...+    $line =~ s|/\*.*\*/||;	# ... and C-style comments...+    $line =~ s|^\s+||;		# ... and skip white spaces in beginning+    $line =~ s|\s+$||;		# ... and at the end++    if (my $label=label->re(\$line))	{ print $label->out(); }++    if (my $directive=directive->re(\$line)) {+	printf "%s",$directive->out();+    } elsif (my $opcode=opcode->re(\$line)) {+	my $asm = eval("\$".$opcode->mnemonic());++	if ((ref($asm) eq 'CODE') && scalar(my @bytes=&$asm($line))) {+	    print $gas?".byte\t":"DB\t",join(',',@bytes),"\n";+	    next;+	}++	my @args;+	ARGUMENT: while (1) {+	    my $arg;++	    ($arg=register->re(\$line, $opcode))||+	    ($arg=const->re(\$line))		||+	    ($arg=ea->re(\$line, $opcode))	||+	    ($arg=expr->re(\$line, $opcode))	||+	    last ARGUMENT;++	    push @args,$arg;++	    last ARGUMENT if ($line !~ /^,/);++	    $line =~ s/^,\s*//;+	} # ARGUMENT:++	if ($win64 && $current_function->{abi} eq "svr4"+		   && $current_function->{narg} >= 0) {+	    my $pc = $current_function->{pc};+	    my $op = $opcode->{op};+	    my $a0 = @args[0]->{value} if ($#args>=0);+	    if (!$current_function->{unwind}+		|| $pc == 0 && !($op eq "push" && $a0 eq "rbp")+		|| $pc == 1 && !($op eq "mov" && $a0 eq "rsp"+					      && @args[1]->{value} eq "rbp"+					      && ($current_function->{unwind} = "%rbp"))+		|| $pc > 1) {+		print label::win64_args();+	    }+	}++	if ($#args>=0) {+	    my $insn;+	    my $sz=$opcode->size();++	    if ($gas) {+		$insn = $opcode->out($#args>=1?$args[$#args]->size():$sz);+		@args = map($_->out($sz),@args);+		printf "\t%s\t%s",$insn,join(",",@args);+	    } else {+		$insn = $opcode->out();+		foreach (@args) {+		    my $arg = $_->out();+		    # $insn.=$sz compensates for movq, pinsrw, ...+		    if ($arg =~ /^xmm[0-9]+$/) { $insn.=$sz; $sz="x" if(!$sz); last; }+		    if ($arg =~ /^ymm[0-9]+$/) { $insn.=$sz; $sz="y" if(!$sz); last; }+		    if ($arg =~ /^zmm[0-9]+$/) { $insn.=$sz; $sz="z" if(!$sz); last; }+		    if ($arg =~ /^mm[0-9]+$/)  { $insn.=$sz; $sz="q" if(!$sz); last; }+		}+		@args = reverse(@args);+		undef $sz if ($nasm && $opcode->mnemonic() eq "lea");+		printf "\t%s\t%s",$insn,join(",",map($_->out($sz),@args));+	    }+	} else {+	    printf "\t%s",$opcode->out();+	}++	++$current_function->{pc} if (defined($current_function));+    }++    print $line,"\n";+}++while(<>) { process($_); }++map { process($_) } @pdata_seg if ($win64 && $#pdata_seg>1);+map { process($_) } @xdata_seg if ($win64 && $#xdata_seg>1);++# platform-specific epilogue+if ($masm) {+    print "\n$current_segment\tENDS\n"	if ($current_segment);+    print "END\n";+} elsif ($elf) {+    # -fcf-protection segment, snatched from compiler -S output+    my $align = ($flavour =~ /elf32/) ? 4 : 8;+    print <<___;++.section	.note.gnu.property,"a",\@note+	.long	4,2f-1f,5+	.byte	0x47,0x4E,0x55,0+1:	.long	0xc0000002,4,3+.align	$align+2:+___+}++close STDOUT;++#################################################+# Cross-reference x86_64 ABI "card"+#+# 		Unix		Win64+# %rax		*		*+# %rbx		-		-+# %rcx		#4		#1+# %rdx		#3		#2+# %rsi		#2		-+# %rdi		#1		-+# %rbp		-		-+# %rsp		-		-+# %r8		#5		#3+# %r9		#6		#4+# %r10		*		*+# %r11		*		*+# %r12		-		-+# %r13		-		-+# %r14		-		-+# %r15		-		-+#+# (*)	volatile register+# (-)	preserved by callee+# (#)	Nth argument, volatile+#+# In Unix terms top of stack is argument transfer area for arguments+# which could not be accommodated in registers. Or in other words 7th+# [integer] argument resides at 8(%rsp) upon function entry point.+# 128 bytes above %rsp constitute a "red zone" which is not touched+# by signal handlers and can be used as temporal storage without+# allocating a frame.+#+# In Win64 terms N*8 bytes on top of stack is argument transfer area,+# which belongs to/can be overwritten by callee. N is the number of+# arguments passed to callee, *but* not less than 4! This means that+# upon function entry point 5th argument resides at 40(%rsp), as well+# as that 32 bytes from 8(%rsp) can always be used as temporal+# storage [without allocating a frame]. One can actually argue that+# one can assume a "red zone" above stack pointer under Win64 as well.+# Point is that at apparently no occasion Windows kernel would alter+# the area above user stack pointer in true asynchronous manner...+#+# All the above means that if assembler programmer adheres to Unix+# register and stack layout, but disregards the "red zone" existence,+# it's possible to use following prologue and epilogue to "gear" from+# Unix to Win64 ABI in leaf functions with not more than 6 arguments.+#+# omnipotent_function:+# ifdef WIN64+#	movq	%rdi,8(%rsp)+#	movq	%rsi,16(%rsp)+#	movq	%rcx,%rdi	; if 1st argument is actually present+#	movq	%rdx,%rsi	; if 2nd argument is actually ...+#	movq	%r8,%rdx	; if 3rd argument is ...+#	movq	%r9,%rcx	; if 4th argument ...+#	movq	40(%rsp),%r8	; if 5th ...+#	movq	48(%rsp),%r9	; if 6th ...+# endif+#	...+# ifdef WIN64+#	movq	8(%rsp),%rdi+#	movq	16(%rsp),%rsi+# endif+#	ret+#+#################################################+# Win64 SEH, Structured Exception Handling.+#+# Unlike on Unix systems(*) lack of Win64 stack unwinding information+# has undesired side-effect at run-time: if an exception is raised in+# assembler subroutine such as those in question (basically we're+# referring to segmentation violations caused by malformed input+# parameters), the application is briskly terminated without invoking+# any exception handlers, most notably without generating memory dump+# or any user notification whatsoever. This poses a problem. It's+# possible to address it by registering custom language-specific+# handler that would restore processor context to the state at+# subroutine entry point and return "exception is not handled, keep+# unwinding" code. Writing such handler can be a challenge... But it's+# doable, though requires certain coding convention. Consider following+# snippet:+#+# .type	function,@function+# function:+#	movq	%rsp,%rax	# copy rsp to volatile register+#	pushq	%r15		# save non-volatile registers+#	pushq	%rbx+#	pushq	%rbp+#	movq	%rsp,%r11+#	subq	%rdi,%r11	# prepare [variable] stack frame+#	andq	$-64,%r11+#	movq	%rax,0(%r11)	# check for exceptions+#	movq	%r11,%rsp	# allocate [variable] stack frame+#	movq	%rax,0(%rsp)	# save original rsp value+# magic_point:+#	...+#	movq	0(%rsp),%rcx	# pull original rsp value+#	movq	-24(%rcx),%rbp	# restore non-volatile registers+#	movq	-16(%rcx),%rbx+#	movq	-8(%rcx),%r15+#	movq	%rcx,%rsp	# restore original rsp+# magic_epilogue:+#	ret+# .size function,.-function+#+# The key is that up to magic_point copy of original rsp value remains+# in chosen volatile register and no non-volatile register, except for+# rsp, is modified. While past magic_point rsp remains constant till+# the very end of the function. In this case custom language-specific+# exception handler would look like this:+#+# EXCEPTION_DISPOSITION handler (EXCEPTION_RECORD *rec,ULONG64 frame,+#		CONTEXT *context,DISPATCHER_CONTEXT *disp)+# {	ULONG64 *rsp = (ULONG64 *)context->Rax;+#	ULONG64  rip = context->Rip;+#+#	if (rip >= magic_point)+#	{   rsp = (ULONG64 *)context->Rsp;+#	    if (rip < magic_epilogue)+#	    {	rsp = (ULONG64 *)rsp[0];+#		context->Rbp = rsp[-3];+#		context->Rbx = rsp[-2];+#		context->R15 = rsp[-1];+#	    }+#	}+#	context->Rsp = (ULONG64)rsp;+#	context->Rdi = rsp[1];+#	context->Rsi = rsp[2];+#+#	memcpy (disp->ContextRecord,context,sizeof(CONTEXT));+#	RtlVirtualUnwind(UNW_FLAG_NHANDLER,disp->ImageBase,+#		dips->ControlPc,disp->FunctionEntry,disp->ContextRecord,+#		&disp->HandlerData,&disp->EstablisherFrame,NULL);+#	return ExceptionContinueSearch;+# }+#+# It's appropriate to implement this handler in assembler, directly in+# function's module. In order to do that one has to know members'+# offsets in CONTEXT and DISPATCHER_CONTEXT structures and some constant+# values. Here they are:+#+#	CONTEXT.Rax				120+#	CONTEXT.Rcx				128+#	CONTEXT.Rdx				136+#	CONTEXT.Rbx				144+#	CONTEXT.Rsp				152+#	CONTEXT.Rbp				160+#	CONTEXT.Rsi				168+#	CONTEXT.Rdi				176+#	CONTEXT.R8				184+#	CONTEXT.R9				192+#	CONTEXT.R10				200+#	CONTEXT.R11				208+#	CONTEXT.R12				216+#	CONTEXT.R13				224+#	CONTEXT.R14				232+#	CONTEXT.R15				240+#	CONTEXT.Rip				248+#	CONTEXT.Xmm6				512+#	sizeof(CONTEXT)				1232+#	DISPATCHER_CONTEXT.ControlPc		0+#	DISPATCHER_CONTEXT.ImageBase		8+#	DISPATCHER_CONTEXT.FunctionEntry	16+#	DISPATCHER_CONTEXT.EstablisherFrame	24+#	DISPATCHER_CONTEXT.TargetIp		32+#	DISPATCHER_CONTEXT.ContextRecord	40+#	DISPATCHER_CONTEXT.LanguageHandler	48+#	DISPATCHER_CONTEXT.HandlerData		56+#	UNW_FLAG_NHANDLER			0+#	ExceptionContinueSearch			1+#+# In order to tie the handler to the function one has to compose+# couple of structures: one for .xdata segment and one for .pdata.+#+# UNWIND_INFO structure for .xdata segment would be+#+# function_unwind_info:+#	.byte	9,0,0,0+#	.rva	handler+#+# This structure designates exception handler for a function with+# zero-length prologue, no stack frame or frame register.+#+# To facilitate composing of .pdata structures, auto-generated "gear"+# prologue copies rsp value to rax and denotes next instruction with+# .LSEH_begin_{function_name} label. This essentially defines the SEH+# styling rule mentioned in the beginning. Position of this label is+# chosen in such manner that possible exceptions raised in the "gear"+# prologue would be accounted to caller and unwound from latter's frame.+# End of function is marked with respective .LSEH_end_{function_name}+# label. To summarize, .pdata segment would contain+#+#	.rva	.LSEH_begin_function+#	.rva	.LSEH_end_function+#	.rva	function_unwind_info+#+# Reference to function_unwind_info from .xdata segment is the anchor.+# In case you wonder why references are 32-bit .rvas and not 64-bit+# .quads. References put into these two segments are required to be+# *relative* to the base address of the current binary module, a.k.a.+# image base. No Win64 module, be it .exe or .dll, can be larger than+# 2GB and thus such relative references can be and are accommodated in+# 32 bits.+#+# Having reviewed the example function code, one can argue that "movq+# %rsp,%rax" above is redundant. It is not! Keep in mind that on Unix+# rax would contain an undefined value. If this "offends" you, use+# another register and refrain from modifying rax till magic_point is+# reached, i.e. as if it was a non-volatile register. If more registers+# are required prior [variable] frame setup is completed, note that+# nobody says that you can have only one "magic point." You can+# "liberate" non-volatile registers by denoting last stack off-load+# instruction and reflecting it in finer grade unwind logic in handler.+# After all, isn't it why it's called *language-specific* handler...+#+# SE handlers are also involved in unwinding stack when executable is+# profiled or debugged. Profiling implies additional limitations that+# are too subtle to discuss here. For now it's sufficient to say that+# in order to simplify handlers one should either a) offload original+# %rsp to stack (like discussed above); or b) if you have a register to+# spare for frame pointer, choose volatile one.+#+# (*)	Note that we're talking about run-time, not debug-time. Lack of+#	unwind information makes debugging hard on both Windows and+#	Unix. "Unlike" refers to the fact that on Unix signal handler+#	will always be invoked, core dumped and appropriate exit code+#	returned to parent (for user notification).+#+########################################################################+# As of May 2020 an alternative approach that works with both exceptions+# and debugging/profiling was implemented by re-purposing DWARF .cfi+# annotations even for Win64 unwind tables' generation. Unfortunately,+# but not really unexpectedly, it imposes additional limitations on+# coding style. Probably the most significant limitation is that the+# frame pointer has to be at 16*n distance from the stack pointer at the+# exit from prologue. But first things first. There are two additional+# synthetic .cfi directives, .cfi_end_prologue and .cfi_epilogue,+# that need to be added to all functions marked with additional .type+# tag (see example below). There are "do's and don'ts" for prologue+# and epilogue. It shouldn't come as a surprise that in prologue one may+# not modify non-volatile registers, but one may not modify %r11 either.+# This is because it's used as a temporary frame pointer(*). There are+# two exceptions to this rule. 1) One can set up a non-volatile register+# or %r11 as a frame pointer, but it must be last instruction in the+# prologue. 2) One can use 'push %rbp' as first instruction immediately+# followed by 'mov %rsp,%rbp' to use %rbp as "legacy" frame pointer.+# Constraints for epilogue, or rather on its boundary, depend on whether+# the frame is fixed- or variable-length. In fixed-frame subroutine+# stack pointer has to be restored in the last instruction prior to the+# .cfi_epilogue directive. If it's a variable-frame subroutine, and a+# non-volatile register was used as a frame pointer, then the last+# instruction prior to the directive has to restore its original value.+# This means that final stack pointer adjustment would have to be+# pushed past the directive. Normally this would render the epilogue+# non-unwindable, so special care has to be taken. To resolve the+# dilemma, copy the frame pointer to a volatile register in advance.+# To give an example:+#+# .type	rbp_as_frame_pointer,\@function,3,"unwind"  # mind extra tag!+# rbp_as_frame_pointer:+# .cfi_startproc+#	push	%rbp+# .cfi_push	%rbp+#	push	%rbx+# .cfi_push	%rbx+# 	mov	%rsp,%rbp	# last instruction in prologue+# .cfi_def_cfa_register	%rbp	# %rsp-%rbp has to be 16*n, e.g. 16*0+# .cfi_end_prologue+#	sub	\$40,%rsp+#	and	\$-64,%rsp+#	...+#	mov	%rbp,%r11+# .cfi_def_cfa_register	%r11	# copy frame pointer to volatile %r11+#	mov	0(%rbp),%rbx+#	mov	8(%rbp),%rbp	# last instruction prior epilogue+# .cfi_epilogue			# may not change %r11 in epilogue+#	lea	16(%r11),%rsp+#	ret+# .cfi_endproc+# .size	rbp_as_frame_pointer,.-rbp_as_frame_pointer+#+# An example of "legacy" frame pointer:+#+# .type	legacy_frame_pointer,\@function,3,"unwind"  # mind extra tag!+# legacy_frame_pointer:+# .cfi_startproc+#	push	%rbp+# .cfi_push	%rbp+# 	mov	%rsp,%rbp+# .cfi_def_cfa_register	%rbp+#	push	%rbx+# .cfi_push	%rbx+#	sub	\$40,%rsp+# .cfi_alloca	40+# .cfi_end_prologue		# %rsp-%rbp has to be 16*n+#	and	\$-64,%rsp+#	...+#	mov	-8(%rbp),%rbx+#	mov	%rbp,%rsp+# .cfi_def_cfa_register	%rsp+#	pop	%rbp		# recognized by Windows+# .cfi_pop	%rbp+# .cfi_epilogue+#	ret+# .cfi_endproc+# .size	legacy_frame_pointer,.-legacy_frame_pointer+#+# To give an example of fixed-frame subroutine for reference:+#+# .type	fixed_frame,\@function,3,"unwind"           # mind extra tag!+# fixed_frame:+# .cfi_startproc+#	push	%rbp+# .cfi_push	%rbp+#	push	%rbx+# .cfi_push	%rbx+#	sub	\$40,%rsp+# .cfi_adjust_cfa_offset 40+# .cfi_end_prologue+#	...+#	mov	40(%rsp),%rbx+#	mov	48(%rsp),%rbp+#	lea	56(%rsp),%rsp+# .cfi_adjust_cfa_offset -56+# .cfi_epilogue+#	ret+# .cfi_endproc+# .size	fixed_frame,.-fixed_frame+#+# As for epilogue itself, one can only work on non-volatile registers.+# "Non-volatile" in "Windows" sense, i.e. minus %rdi and %rsi.+#+# On a final note, mixing old-style and modernized subroutines in the+# same file takes some trickery. Ones of the new kind have to appear+# after old-style ones. This has everything to do with the fact that+# entries in the .pdata segment have to appear in strictly same order+# as corresponding subroutines, and auto-generated RUNTIME_FUNCTION+# structures get mechanically appended to whatever existing .pdata.+#+# (*)	Just in case, why %r11 and not %rax. This has everything to do+#	with the way UNWIND_INFO is, one just can't designate %rax as+#	frame pointer.
+ cbits/chacha_avx2.c view
@@ -0,0 +1,146 @@+/*+ * ChaCha with AVX2, eight blocks at a time.+ *+ * The same arrangement as the SSE and NEON versions, twice as wide: word i+ * of eight blocks goes in lane i of one 256-bit register.  Eight blocks is+ * where the register file stops being the constraint -- sixteen registers+ * hold the working state either way, so the wider ones are free.+ *+ * AVX2 is not part of any baseline, so this is reached only after+ * crypton_x86_simd_features() has said the CPU has it and the OS saves the+ * wider registers.  It is compiled into a translation unit that is+ * otherwise baseline, through a function attribute, so nothing here can be+ * emitted anywhere else.+ */++#include <stdint.h>+#include <immintrin.h>+#include "crypton_chacha.h"++#ifdef WITH_TARGET_ATTRIBUTES++#define TARGET __attribute__((target("avx2")))++/* rotating a 32-bit lane by sixteen or eight is a byte shuffle, which AVX2+ * does within each 128-bit half -- which is all this needs */+static const int8_t rot16_tbl[32] = {+	2,3,0,1, 6,7,4,5, 10,11,8,9, 14,15,12,13,+	2,3,0,1, 6,7,4,5, 10,11,8,9, 14,15,12,13,+};+static const int8_t rot8_tbl[32] = {+	3,0,1,2, 7,4,5,6, 11,8,9,10, 15,12,13,14,+	3,0,1,2, 7,4,5,6, 11,8,9,10, 15,12,13,14,+};++#define ROL(x, n)                                                            \+	((n) == 16 ? _mm256_shuffle_epi8((x), _mm256_loadu_si256((const __m256i *) rot16_tbl)) \+	 : (n) == 8 ? _mm256_shuffle_epi8((x), _mm256_loadu_si256((const __m256i *) rot8_tbl)) \+	 : _mm256_or_si256(_mm256_slli_epi32((x), (n)), _mm256_srli_epi32((x), 32 - (n))))++TARGET+static inline void core8(int rounds, const crypton_chacha_state *in,+                         const uint8_t *src, uint8_t *dst, int combine)+{+	__m256i v0, v1, v2, v3, v4, v5, v6, v7;+	__m256i v8, v9, v10, v11, v12, v13, v14, v15;+	const uint32_t c = in->d[12];+	const __m256i ctr = _mm256_setr_epi32((int) c, (int) (c + 1), (int) (c + 2),+	                                      (int) (c + 3), (int) (c + 4), (int) (c + 5),+	                                      (int) (c + 6), (int) (c + 7));+	int i;++#define SET(n) v##n = _mm256_set1_epi32((int) in->d[n])+	SET(0);  SET(1);  SET(2);  SET(3);+	SET(4);  SET(5);  SET(6);  SET(7);+	SET(8);  SET(9);  SET(10); SET(11);+	         SET(13); SET(14); SET(15);+#undef SET+	v12 = ctr;++#define QR(a, b, cc, d)                                                  \+	a = _mm256_add_epi32(a, b); d = ROL(_mm256_xor_si256(d, a), 16);  \+	cc = _mm256_add_epi32(cc, d); b = ROL(_mm256_xor_si256(b, cc), 12); \+	a = _mm256_add_epi32(a, b); d = ROL(_mm256_xor_si256(d, a),  8);  \+	cc = _mm256_add_epi32(cc, d); b = ROL(_mm256_xor_si256(b, cc),  7)++	for (i = rounds; i > 0; i -= 2) {+		QR(v0, v4, v8,  v12);+		QR(v1, v5, v9,  v13);+		QR(v2, v6, v10, v14);+		QR(v3, v7, v11, v15);++		QR(v0, v5, v10, v15);+		QR(v1, v6, v11, v12);+		QR(v2, v7, v8,  v13);+		QR(v3, v4, v9,  v14);+	}+#undef QR++#define ADD(n) v##n = _mm256_add_epi32(v##n, _mm256_set1_epi32((int) in->d[n]))+	ADD(0);  ADD(1);  ADD(2);  ADD(3);+	ADD(4);  ADD(5);  ADD(6);  ADD(7);+	ADD(8);  ADD(9);  ADD(10); ADD(11);+	         ADD(13); ADD(14); ADD(15);+#undef ADD+	v12 = _mm256_add_epi32(v12, ctr);++	/*+	 * The interleave works within each 128-bit half, so four registers+	 * holding word w of blocks 0..7 come apart into words w..w+3 of+	 * blocks 0..3 in the low halves and of blocks 4..7 in the high ones.+	 *+	 * Each piece is exclusive-ored with the input and stored where it+	 * belongs as it comes out.  Writing the keystream to a buffer and+	 * reading it back to combine it cost a pass over every byte, which is+	 * a tenth of what this loop does.+	 */+#define OUT(j, g, v)                                                         \+	do {                                                                 \+		__m128i o_ = (v);                                            \+		if (combine)                                                 \+			o_ = _mm_xor_si128(o_, _mm_loadu_si128(              \+			    (const __m128i *) (src + 64 * (j) + 4 * (g))));  \+		_mm_storeu_si128((__m128i *) (dst + 64 * (j) + 4 * (g)), o_);\+	} while (0)++#define GROUP(g, qa, qb, qc, qd)                                             \+	do {                                                                 \+		__m256i t0_ = _mm256_unpacklo_epi32(qa, qb);                 \+		__m256i t1_ = _mm256_unpackhi_epi32(qa, qb);                 \+		__m256i t2_ = _mm256_unpacklo_epi32(qc, qd);                 \+		__m256i t3_ = _mm256_unpackhi_epi32(qc, qd);                 \+		__m256i u0_ = _mm256_unpacklo_epi64(t0_, t2_);               \+		__m256i u1_ = _mm256_unpackhi_epi64(t0_, t2_);               \+		__m256i u2_ = _mm256_unpacklo_epi64(t1_, t3_);               \+		__m256i u3_ = _mm256_unpackhi_epi64(t1_, t3_);               \+		OUT(0, (g), _mm256_castsi256_si128(u0_));                    \+		OUT(1, (g), _mm256_castsi256_si128(u1_));                    \+		OUT(2, (g), _mm256_castsi256_si128(u2_));                    \+		OUT(3, (g), _mm256_castsi256_si128(u3_));                    \+		OUT(4, (g), _mm256_extracti128_si256(u0_, 1));               \+		OUT(5, (g), _mm256_extracti128_si256(u1_, 1));               \+		OUT(6, (g), _mm256_extracti128_si256(u2_, 1));               \+		OUT(7, (g), _mm256_extracti128_si256(u3_, 1));               \+	} while (0)+	GROUP(0,  v0,  v1,  v2,  v3);+	GROUP(4,  v4,  v5,  v6,  v7);+	GROUP(8,  v8,  v9,  v10, v11);+	GROUP(12, v12, v13, v14, v15);+#undef GROUP+#undef OUT+}++TARGET+void crypton_chacha_avx2_combine(int rounds, uint8_t *dst, const uint8_t *src,+                                 const crypton_chacha_state *in)+{+	core8(rounds, in, src, dst, 1);+}++TARGET+void crypton_chacha_avx2_generate(int rounds, uint8_t *dst, const crypton_chacha_state *in)+{+	core8(rounds, in, NULL, dst, 0);+}++#endif /* WITH_TARGET_ATTRIBUTES */
+ cbits/chacha_neon.c view
@@ -0,0 +1,145 @@+/*+ * ChaCha with NEON, four blocks at a time.+ *+ * The state is sixteen 32-bit words and the quarter rounds touch four of+ * them at once, so a single block vectorises only by shuffling lanes+ * between the column and diagonal rounds.  Four blocks vectorise without+ * any shuffling at all: word i of the four blocks goes in lane i of one+ * register, every quarter round is then the same operation on whole+ * registers, and the blocks are independent because only the counter+ * differs between them.+ *+ * NEON is part of the AArch64 baseline, so unlike the AES, PMULL and SHA+ * work there is nothing to ask about at runtime and no target attribute+ * to attach.+ */++#include <stddef.h>+#include <stdint.h>+#include <arm_neon.h>+#include "crypton_chacha.h"++/* rotate each 32-bit lane left by n */+#define ROL(x, n) vsriq_n_u32(vshlq_n_u32((x), (n)), (x), 32 - (n))+/* by 16 it is a halfword swap, and by 8 a byte shuffle; both beat the pair+ * of shifts */+#define ROL16(x) vreinterpretq_u32_u16(vrev32q_u16(vreinterpretq_u16_u32(x)))+#define ROL8(x)  vreinterpretq_u32_u8(vqtbl1q_u8(vreinterpretq_u8_u32(x), rot8))++#define QR(a, b, c, d)                          \+	a = vaddq_u32(a, b); d = ROL16(veorq_u32(d, a)); \+	c = vaddq_u32(c, d); b = ROL(veorq_u32(b, c), 12); \+	a = vaddq_u32(a, b); d = ROL8(veorq_u32(d, a));  \+	c = vaddq_u32(c, d); b = ROL(veorq_u32(b, c), 7)++/*+ * Turn four registers holding word w of blocks 0..3 into four holding+ * words w..w+3 of one block each, which is the order they are written in.+ */+#define TRANSPOSE(a, b, c, d)                                          \+	do {                                                           \+		uint32x4x2_t t0_ = vtrnq_u32((a), (b));                \+		uint32x4x2_t t1_ = vtrnq_u32((c), (d));                \+		(a) = vcombine_u32(vget_low_u32(t0_.val[0]),           \+		                   vget_low_u32(t1_.val[0]));          \+		(b) = vcombine_u32(vget_low_u32(t0_.val[1]),           \+		                   vget_low_u32(t1_.val[1]));          \+		(c) = vcombine_u32(vget_high_u32(t0_.val[0]),          \+		                   vget_high_u32(t1_.val[0]));         \+		(d) = vcombine_u32(vget_high_u32(t0_.val[1]),          \+		                   vget_high_u32(t1_.val[1]));         \+	} while (0)++/*+ * Four blocks with counters d[12], d[12]+1, d[12]+2 and d[12]+3.  The+ * caller keeps the state's counter, and only calls this when those four+ * do not carry into d[13].+ */+static inline void core4(int rounds, const crypton_chacha_state *in,+                         const uint8_t *src, uint8_t *dst, int combine)+{+	static const uint8_t rot8_tbl[16] =+		{ 3,0,1,2, 7,4,5,6, 11,8,9,10, 15,12,13,14 };+	const uint8x16_t rot8 = vld1q_u8(rot8_tbl);+	uint32x4_t v0, v1, v2, v3, v4, v5, v6, v7;+	uint32x4_t v8, v9, v10, v11, v12, v13, v14, v15;+	const uint32_t c = in->d[12];+	const uint32_t ctr4[4] = { c, c + 1, c + 2, c + 3 };+	int i;++	/*+	 * Only the working state is kept in registers.  The initial state has+	 * to be added back at the end, but holding a second copy of it would+	 * want thirty-two registers for that alone, and the machine has+	 * thirty-two in total; read it again instead, from memory that is+	 * certainly warm.+	 */+#define SET(n) v##n = vdupq_n_u32(in->d[n])+	SET(0);  SET(1);  SET(2);  SET(3);+	SET(4);  SET(5);  SET(6);  SET(7);+	SET(8);  SET(9);  SET(10); SET(11);+	         SET(13); SET(14); SET(15);+#undef SET+	v12 = vld1q_u32(ctr4);++	for (i = rounds; i > 0; i -= 2) {+		QR(v0, v4, v8,  v12);+		QR(v1, v5, v9,  v13);+		QR(v2, v6, v10, v14);+		QR(v3, v7, v11, v15);++		QR(v0, v5, v10, v15);+		QR(v1, v6, v11, v12);+		QR(v2, v7, v8,  v13);+		QR(v3, v4, v9,  v14);+	}++#define ADD(n) v##n = vaddq_u32(v##n, vdupq_n_u32(in->d[n]))+	ADD(0);  ADD(1);  ADD(2);  ADD(3);+	ADD(4);  ADD(5);  ADD(6);  ADD(7);+	ADD(8);  ADD(9);  ADD(10); ADD(11);+	         ADD(13); ADD(14); ADD(15);+#undef ADD+	v12 = vaddq_u32(v12, vld1q_u32(ctr4));++	TRANSPOSE(v0,  v1,  v2,  v3);+	TRANSPOSE(v4,  v5,  v6,  v7);+	TRANSPOSE(v8,  v9,  v10, v11);+	TRANSPOSE(v12, v13, v14, v15);++	/*+	 * Each piece is exclusive-ored with the input and stored where it+	 * belongs as it comes out.  Writing the keystream to a buffer and+	 * reading it back to combine it cost a pass over every byte.+	 */+#define ST(j, g, v)                                                    \+	do {                                                           \+		uint8x16_t o_ = vreinterpretq_u8_u32(v);               \+		if (combine)                                           \+			o_ = veorq_u8(o_, vld1q_u8(src + 64 * (j)      \+			                           + 4 * (g)));        \+		vst1q_u8(dst + 64 * (j) + 4 * (g), o_);                \+	} while (0)+	ST(0, 0, v0);   ST(1, 0, v1);   ST(2, 0, v2);   ST(3, 0, v3);+	ST(0, 4, v4);   ST(1, 4, v5);   ST(2, 4, v6);   ST(3, 4, v7);+	ST(0, 8, v8);   ST(1, 8, v9);   ST(2, 8, v10);  ST(3, 8, v11);+	ST(0, 12, v12); ST(1, 12, v13); ST(2, 12, v14); ST(3, 12, v15);+#undef ST+}++void crypton_chacha_simd_combine(int rounds, uint8_t *dst, const uint8_t *src,+                                  const crypton_chacha_state *in)+{+	core4(rounds, in, src, dst, 1);+}++void crypton_chacha_simd_generate(int rounds, uint8_t *dst, const crypton_chacha_state *in)+{+	core4(rounds, in, NULL, dst, 0);+}++/* NEON has no wider sibling to choose between, so the answer is fixed. */+int crypton_chacha_simd_width(void)+{+	return 4;+}
+ cbits/chacha_sse2.c view
@@ -0,0 +1,105 @@+/*+ * ChaCha with SSE, four blocks at a time, and the choice of which x86+ * version to run.+ *+ * Word i of four blocks goes in lane i of one register, so every quarter+ * round is one operation on whole registers and no lane moves between the+ * column and the diagonal rounds.  Only the counter differs between the+ * four blocks.+ *+ * SSE2 is part of the x86-64 baseline and needs no check.  SSSE3 takes the+ * rotates by sixteen and eight in one instruction each, and AVX2 -- in+ * chacha_avx2.c -- carries eight blocks instead of four; both are reached+ * only after crypton_x86_simd_features() says so.  Both also need function+ * attributes to sit in a translation unit that is otherwise baseline, so+ * with use_target_attributes turned off only the SSE2 version is built.+ */++#include <stdint.h>+#include <emmintrin.h>+#ifdef WITH_TARGET_ATTRIBUTES+#include <tmmintrin.h>+#endif+#include "crypton_chacha.h"+#include "crypton_cpu.h"++#define SIZED(n) n##_sse2+#define TARGET+#define ROL(x, n) _mm_or_si128(_mm_slli_epi32((x), (n)), _mm_srli_epi32((x), 32 - (n)))+#include <chacha_sse_impl.c>+#undef SIZED+#undef TARGET+#undef ROL++#ifdef WITH_TARGET_ATTRIBUTES++static const int8_t rot16_tbl[16] = { 2,3,0,1, 6,7,4,5, 10,11,8,9, 14,15,12,13 };+static const int8_t rot8_tbl[16]  = { 3,0,1,2, 7,4,5,6, 11,8,9,10, 15,12,13,14 };++#define SIZED(n) n##_ssse3+#define TARGET __attribute__((target("ssse3")))+#define ROL(x, n)                                                              \+	((n) == 16 ? _mm_shuffle_epi8((x), _mm_loadu_si128((const __m128i *) rot16_tbl)) \+	 : (n) == 8 ? _mm_shuffle_epi8((x), _mm_loadu_si128((const __m128i *) rot8_tbl)) \+	 : _mm_or_si128(_mm_slli_epi32((x), (n)), _mm_srli_epi32((x), 32 - (n))))+#include <chacha_sse_impl.c>+#undef SIZED+#undef TARGET+#undef ROL++void crypton_chacha_avx2_combine(int rounds, uint8_t *dst, const uint8_t *src,+                                 const crypton_chacha_state *in);+void crypton_chacha_avx2_generate(int rounds, uint8_t *dst, const crypton_chacha_state *in);++#endif++/* how many blocks a call covers, and which version does it */+enum { IMPL_UNRESOLVED = 0, IMPL_SSE2, IMPL_SSSE3, IMPL_AVX2 };++static int impl = IMPL_UNRESOLVED;++/* Two threads racing to answer this both write the same value. */+static int resolve(void)+{+#ifdef WITH_TARGET_ATTRIBUTES+	uint32_t f = crypton_x86_simd_features();++	if (f & CRYPTON_X86_AVX2)+		impl = IMPL_AVX2;+	else if (f & CRYPTON_X86_SSSE3)+		impl = IMPL_SSSE3;+	else+#endif+		impl = IMPL_SSE2;+	return impl;+}++int crypton_chacha_simd_width(void)+{+	int i = impl ? impl : resolve();++	return i == IMPL_AVX2 ? 8 : 4;+}++void crypton_chacha_simd_combine(int rounds, uint8_t *dst, const uint8_t *src,+                                 const crypton_chacha_state *in)+{+	switch (impl ? impl : resolve()) {+#ifdef WITH_TARGET_ATTRIBUTES+	case IMPL_AVX2:  crypton_chacha_avx2_combine(rounds, dst, src, in); return;+	case IMPL_SSSE3: combine_ssse3(rounds, dst, src, in); return;+#endif+	default:         combine_sse2(rounds, dst, src, in); return;+	}+}++void crypton_chacha_simd_generate(int rounds, uint8_t *dst, const crypton_chacha_state *in)+{+	switch (impl ? impl : resolve()) {+#ifdef WITH_TARGET_ATTRIBUTES+	case IMPL_AVX2:  crypton_chacha_avx2_generate(rounds, dst, in); return;+	case IMPL_SSSE3: generate_ssse3(rounds, dst, in); return;+#endif+	default:         generate_sse2(rounds, dst, in); return;+	}+}
+ cbits/chacha_sse_impl.c view
@@ -0,0 +1,114 @@+/*+ * Included from chacha_sse2.c once per instruction set, with SIZED()+ * naming the functions, ROL() rotating a lane and TARGET saying what the+ * functions may use.  The body is identical; only the two rotates by+ * sixteen and eight differ, and only because SSSE3 can do each in one+ * PSHUFB where SSE2 needs a shift, a shift and an or.+ */++/*+ * Four blocks with counters d[12] .. d[12]+3.  The caller keeps the+ * state's counter and only calls this when those four do not carry into+ * d[13].+ */+TARGET+static inline void SIZED(core4)(int rounds, const crypton_chacha_state *in,+                                const uint8_t *src, uint8_t *dst, int combine)+{+	__m128i v0, v1, v2, v3, v4, v5, v6, v7;+	__m128i v8, v9, v10, v11, v12, v13, v14, v15;+	const uint32_t c = in->d[12];+	int i;++	/*+	 * Sixteen registers hold the working state and the machine has+	 * sixteen, so the initial state is read again at the end rather than+	 * kept in a second set.+	 */+#define SET(n) v##n = _mm_set1_epi32((int) in->d[n])+	SET(0);  SET(1);  SET(2);  SET(3);+	SET(4);  SET(5);  SET(6);  SET(7);+	SET(8);  SET(9);  SET(10); SET(11);+	         SET(13); SET(14); SET(15);+#undef SET+	v12 = _mm_setr_epi32((int) c, (int) (c + 1), (int) (c + 2), (int) (c + 3));++#define QR(a, b, cc, d)                                            \+	a = _mm_add_epi32(a, b); d = ROL(_mm_xor_si128(d, a), 16);  \+	cc = _mm_add_epi32(cc, d); b = ROL(_mm_xor_si128(b, cc), 12); \+	a = _mm_add_epi32(a, b); d = ROL(_mm_xor_si128(d, a),  8);  \+	cc = _mm_add_epi32(cc, d); b = ROL(_mm_xor_si128(b, cc),  7)++	for (i = rounds; i > 0; i -= 2) {+		QR(v0, v4, v8,  v12);+		QR(v1, v5, v9,  v13);+		QR(v2, v6, v10, v14);+		QR(v3, v7, v11, v15);++		QR(v0, v5, v10, v15);+		QR(v1, v6, v11, v12);+		QR(v2, v7, v8,  v13);+		QR(v3, v4, v9,  v14);+	}+#undef QR++#define ADD(n) v##n = _mm_add_epi32(v##n, _mm_set1_epi32((int) in->d[n]))+	ADD(0);  ADD(1);  ADD(2);  ADD(3);+	ADD(4);  ADD(5);  ADD(6);  ADD(7);+	ADD(8);  ADD(9);  ADD(10); ADD(11);+	         ADD(13); ADD(14); ADD(15);+#undef ADD+	v12 = _mm_add_epi32(v12, _mm_setr_epi32((int) c, (int) (c + 1),+	                                        (int) (c + 2), (int) (c + 3)));++	/* four registers holding word w of blocks 0..3 become four holding+	 * words w..w+3 of one block each, the order they are written in */+#define TRANSPOSE(qa, qb, qc, qd)                              \+	do {                                                   \+		__m128i t0_ = _mm_unpacklo_epi32(qa, qb);      \+		__m128i t1_ = _mm_unpackhi_epi32(qa, qb);      \+		__m128i t2_ = _mm_unpacklo_epi32(qc, qd);      \+		__m128i t3_ = _mm_unpackhi_epi32(qc, qd);      \+		qa = _mm_unpacklo_epi64(t0_, t2_);             \+		qb = _mm_unpackhi_epi64(t0_, t2_);             \+		qc = _mm_unpacklo_epi64(t1_, t3_);             \+		qd = _mm_unpackhi_epi64(t1_, t3_);             \+	} while (0)+	TRANSPOSE(v0,  v1,  v2,  v3);+	TRANSPOSE(v4,  v5,  v6,  v7);+	TRANSPOSE(v8,  v9,  v10, v11);+	TRANSPOSE(v12, v13, v14, v15);+#undef TRANSPOSE++	/*+	 * Each piece is exclusive-ored with the input and stored where it+	 * belongs as it comes out.  Writing the keystream to a buffer and+	 * reading it back to combine it cost a pass over every byte.+	 */+#define ST(j, g, v)                                                    \+	do {                                                           \+		__m128i o_ = (v);                                      \+		if (combine)                                           \+			o_ = _mm_xor_si128(o_, _mm_loadu_si128(        \+			    (const __m128i *) (src + 64 * (j) + 4 * (g)))); \+		_mm_storeu_si128((__m128i *) (dst + 64 * (j) + 4 * (g)), o_); \+	} while (0)+	ST(0, 0, v0);   ST(1, 0, v1);   ST(2, 0, v2);   ST(3, 0, v3);+	ST(0, 4, v4);   ST(1, 4, v5);   ST(2, 4, v6);   ST(3, 4, v7);+	ST(0, 8, v8);   ST(1, 8, v9);   ST(2, 8, v10);  ST(3, 8, v11);+	ST(0, 12, v12); ST(1, 12, v13); ST(2, 12, v14); ST(3, 12, v15);+#undef ST+}++TARGET+static void SIZED(combine)(int rounds, uint8_t *dst, const uint8_t *src,+                           const crypton_chacha_state *in)+{+	SIZED(core4)(rounds, in, src, dst, 1);+}++TARGET+static void SIZED(generate)(int rounds, uint8_t *dst, const crypton_chacha_state *in)+{+	SIZED(core4)(rounds, in, NULL, dst, 0);+}
cbits/crypton_aes.c view
@@ -38,6 +38,9 @@ #include <aes/generic.h> #include <aes/gf.h> #include <aes/x86ni.h>+#ifdef WITH_GCM_FUSED+#include <aes/gcm_fused_x86.h>+#endif  void crypton_aes_generic_encrypt_ecb(aes_block *output, aes_key *key, aes_block *input, uint32_t nb_blocks); void crypton_aes_generic_decrypt_ecb(aes_block *output, aes_key *key, aes_block *input, uint32_t nb_blocks);@@ -56,6 +59,42 @@ void crypton_aes_generic_ccm_encrypt(uint8_t *output, aes_ccm *ccm, aes_key *key, uint8_t *input, uint32_t length); void crypton_aes_generic_ccm_decrypt(uint8_t *output, aes_ccm *ccm, aes_key *key, uint8_t *input, uint32_t length); +#ifdef WITH_ARMV8_CRYPTO+void crypton_aes_armv8_init(aes_key *key, uint8_t *origkey, uint8_t size);+int crypton_aes_armv8_gcm_fused_dec(uint8_t *out, const block128 *ht,+                                    aes_key *key, const uint8_t *nonce,+                                    const uint8_t *aad, uint32_t aadlen,+                                    const uint8_t *in, uint32_t inlen,+                                    const uint8_t *tag, uint32_t taglen,+                                    uint8_t *outtag);+void crypton_aes_armv8_gcm_fused(uint8_t *out, const block128 *ht,+                                 aes_key *key, const uint8_t *nonce,+                                 const uint8_t *aad, uint32_t aadlen,+                                 const uint8_t *in, uint32_t inlen,+                                 uint32_t taglen, aes_key *hpkey,+                                 uint32_t sampleoff, uint8_t *mask);+#define ARMV8_DECLS(sz) \+	void crypton_aes_armv8_encrypt_block##sz(aes_block *output, aes_key *key, aes_block *input); \+	void crypton_aes_armv8_decrypt_block##sz(aes_block *output, aes_key *key, aes_block *input); \+	void crypton_aes_armv8_encrypt_ecb##sz(aes_block *output, aes_key *key, aes_block *input, uint32_t nb_blocks); \+	void crypton_aes_armv8_decrypt_ecb##sz(aes_block *output, aes_key *key, aes_block *input, uint32_t nb_blocks); \+	void crypton_aes_armv8_encrypt_cbc##sz(aes_block *output, aes_key *key, aes_block *iv, aes_block *input, uint32_t nb_blocks); \+	void crypton_aes_armv8_decrypt_cbc##sz(aes_block *output, aes_key *key, aes_block *iv, aes_block *input, uint32_t nb_blocks); \+	void crypton_aes_armv8_encrypt_ctr##sz(uint8_t *output, aes_key *key, aes_block *iv, uint8_t *input, uint32_t len); \+	void crypton_aes_armv8_gcm_encrypt##sz(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length); \+	void crypton_aes_armv8_gcm_decrypt##sz(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length); \+	void crypton_aes_armv8_encrypt_xts##sz(aes_block *output, aes_key *k1, aes_key *k2, aes_block *dataunit, uint32_t spoint, aes_block *input, uint32_t nb_blocks); \+	void crypton_aes_armv8_decrypt_xts##sz(aes_block *output, aes_key *k1, aes_key *k2, aes_block *dataunit, uint32_t spoint, aes_block *input, uint32_t nb_blocks);+ARMV8_DECLS(128)+ARMV8_DECLS(192)+ARMV8_DECLS(256)+int crypton_aes_armv8_available(void);+int crypton_aes_armv8_pmull_available(void);+void crypton_aes_armv8_hinit_pmull(block128 *htable, const block128 *h);+void crypton_aes_armv8_gf_mul_pmull(block128 *a, const block128 *htable);+void crypton_aes_armv8_gf_mul4_pmull(block128 *a, const block128 *blocks, const block128 *htable);+#endif+ enum { 	/* init */ 	INIT_128, INIT_192, INIT_256,@@ -85,7 +124,7 @@ 	ENCRYPT_CCM_128, ENCRYPT_CCM_192, ENCRYPT_CCM_256, 	DECRYPT_CCM_128, DECRYPT_CCM_192, DECRYPT_CCM_256, 	/* ghash */-	GHASH_HINIT, GHASH_GF_MUL,+	GHASH_HINIT, GHASH_GF_MUL, GHASH_GF_MUL4, };  void *crypton_aes_branch_table[] = {@@ -153,6 +192,7 @@ 	/* GHASH */ 	[GHASH_HINIT]       = crypton_aes_generic_hinit, 	[GHASH_GF_MUL]      = crypton_aes_generic_gf_mul,+	[GHASH_GF_MUL4]     = crypton_aes_generic_gf_mul4, };  typedef void (*init_f)(aes_key *, uint8_t *, uint8_t);@@ -166,8 +206,9 @@ typedef void (*block_f)(aes_block *output, aes_key *key, aes_block *input); typedef void (*hinit_f)(table_4bit htable, const block128 *h); typedef void (*gf_mul_f)(block128 *a, const table_4bit htable);+typedef void (*gf_mul4_f)(block128 *a, const block128 *blocks, const table_4bit htable); -#ifdef WITH_AESNI+#if defined(WITH_AESNI) || defined(WITH_ARMV8_CRYPTO) #define GET_INIT(strength) \ 	((init_f) (crypton_aes_branch_table[INIT_128 + strength])) #define GET_ECB_ENCRYPT(strength) \@@ -206,6 +247,8 @@ 	(((hinit_f) (crypton_aes_branch_table[GHASH_HINIT]))(t,h)) #define crypton_gf_mul(a,t) \ 	(((gf_mul_f) (crypton_aes_branch_table[GHASH_GF_MUL]))(a,t))+#define crypton_gf_mul4(a,b,t) \+	(((gf_mul4_f) (crypton_aes_branch_table[GHASH_GF_MUL4]))(a,b,t)) #else #define GET_INIT(strenght) crypton_aes_generic_init #define GET_ECB_ENCRYPT(strength) crypton_aes_generic_encrypt_ecb@@ -226,6 +269,7 @@ #define crypton_aes_decrypt_block(o,k,i) crypton_aes_generic_decrypt_block(o,k,i) #define crypton_hinit(t,h) crypton_aes_generic_hinit(t,h) #define crypton_gf_mul(a,t) crypton_aes_generic_gf_mul(a,t)+#define crypton_gf_mul4(a,b,t) crypton_aes_generic_gf_mul4(a,b,t) #endif  #define CPU_AESNI        0@@ -242,39 +286,61 @@ 	crypton_aes_cpu_options[CPU_AESNI] = 1;  	crypton_aes_branch_table[INIT_128] = crypton_aesni_init;+	crypton_aes_branch_table[INIT_192] = crypton_aesni_init; 	crypton_aes_branch_table[INIT_256] = crypton_aesni_init;  	crypton_aes_branch_table[ENCRYPT_BLOCK_128] = crypton_aesni_encrypt_block128; 	crypton_aes_branch_table[DECRYPT_BLOCK_128] = crypton_aesni_decrypt_block128;+	crypton_aes_branch_table[ENCRYPT_BLOCK_192] = crypton_aesni_encrypt_block192; 	crypton_aes_branch_table[ENCRYPT_BLOCK_256] = crypton_aesni_encrypt_block256;+	crypton_aes_branch_table[DECRYPT_BLOCK_192] = crypton_aesni_decrypt_block192; 	crypton_aes_branch_table[DECRYPT_BLOCK_256] = crypton_aesni_decrypt_block256; 	/* ECB */ 	crypton_aes_branch_table[ENCRYPT_ECB_128] = crypton_aesni_encrypt_ecb128; 	crypton_aes_branch_table[DECRYPT_ECB_128] = crypton_aesni_decrypt_ecb128;+	crypton_aes_branch_table[ENCRYPT_ECB_192] = crypton_aesni_encrypt_ecb192; 	crypton_aes_branch_table[ENCRYPT_ECB_256] = crypton_aesni_encrypt_ecb256;+	crypton_aes_branch_table[DECRYPT_ECB_192] = crypton_aesni_decrypt_ecb192; 	crypton_aes_branch_table[DECRYPT_ECB_256] = crypton_aesni_decrypt_ecb256; 	/* CBC */ 	crypton_aes_branch_table[ENCRYPT_CBC_128] = crypton_aesni_encrypt_cbc128; 	crypton_aes_branch_table[DECRYPT_CBC_128] = crypton_aesni_decrypt_cbc128;+	crypton_aes_branch_table[ENCRYPT_CBC_192] = crypton_aesni_encrypt_cbc192; 	crypton_aes_branch_table[ENCRYPT_CBC_256] = crypton_aesni_encrypt_cbc256;+	crypton_aes_branch_table[DECRYPT_CBC_192] = crypton_aesni_decrypt_cbc192; 	crypton_aes_branch_table[DECRYPT_CBC_256] = crypton_aesni_decrypt_cbc256; 	/* CTR */ 	crypton_aes_branch_table[ENCRYPT_CTR_128] = crypton_aesni_encrypt_ctr128;+	crypton_aes_branch_table[ENCRYPT_CTR_192] = crypton_aesni_encrypt_ctr192; 	crypton_aes_branch_table[ENCRYPT_CTR_256] = crypton_aesni_encrypt_ctr256; 	/* CTR with 32-bit wrapping */ 	crypton_aes_branch_table[ENCRYPT_C32_128] = crypton_aesni_encrypt_c32_128;+	crypton_aes_branch_table[ENCRYPT_C32_192] = crypton_aesni_encrypt_c32_192; 	crypton_aes_branch_table[ENCRYPT_C32_256] = crypton_aesni_encrypt_c32_256; 	/* XTS */ 	crypton_aes_branch_table[ENCRYPT_XTS_128] = crypton_aesni_encrypt_xts128;+	crypton_aes_branch_table[ENCRYPT_XTS_192] = crypton_aesni_encrypt_xts192; 	crypton_aes_branch_table[ENCRYPT_XTS_256] = crypton_aesni_encrypt_xts256;+	crypton_aes_branch_table[DECRYPT_XTS_128] = crypton_aesni_decrypt_xts128;+	crypton_aes_branch_table[DECRYPT_XTS_192] = crypton_aesni_decrypt_xts192;+	crypton_aes_branch_table[DECRYPT_XTS_256] = crypton_aesni_decrypt_xts256; 	/* GCM */+	/* GCM, where the build has the carry-less multiply, waits below until+	 * the processor is known to have it too: the loop calls the multiply+	 * rather than reaching it through the branch pointer, so that it can+	 * be scheduled against the rounds, and is compiled with the+	 * instruction.  The AArch64 table waits for PMULL for the same reason.+	 */+#ifndef WITH_PCLMUL 	crypton_aes_branch_table[ENCRYPT_GCM_128] = crypton_aesni_gcm_encrypt128;+	crypton_aes_branch_table[ENCRYPT_GCM_192] = crypton_aesni_gcm_encrypt192; 	crypton_aes_branch_table[ENCRYPT_GCM_256] = crypton_aesni_gcm_encrypt256;-	/* OCB */-	/*-	crypton_aes_branch_table[ENCRYPT_OCB_128] = crypton_aesni_ocb_encrypt128;-	crypton_aes_branch_table[ENCRYPT_OCB_256] = crypton_aesni_ocb_encrypt256;-	*/+	crypton_aes_branch_table[DECRYPT_GCM_128] = crypton_aesni_gcm_decrypt128;+	crypton_aes_branch_table[DECRYPT_GCM_192] = crypton_aesni_gcm_decrypt192;+	crypton_aes_branch_table[DECRYPT_GCM_256] = crypton_aesni_gcm_decrypt256;+#endif+	/* OCB drives the ECB paths above a group at a time, so it has no+	 * entries of its own */ #ifdef WITH_PCLMUL 	if (!pclmul) 		return;@@ -283,16 +349,115 @@ 	/* GHASH */ 	crypton_aes_branch_table[GHASH_HINIT]     = crypton_aesni_hinit_pclmul, 	crypton_aes_branch_table[GHASH_GF_MUL]    = crypton_aesni_gf_mul_pclmul,+	crypton_aes_branch_table[GHASH_GF_MUL4]   = crypton_aesni_gf_mul4_pclmul, 	crypton_aesni_init_pclmul();++	/* and GCM, which needs both halves */+	crypton_aes_branch_table[ENCRYPT_GCM_128] = crypton_aesni_gcm_encrypt128;+	crypton_aes_branch_table[ENCRYPT_GCM_192] = crypton_aesni_gcm_encrypt192;+	crypton_aes_branch_table[ENCRYPT_GCM_256] = crypton_aesni_gcm_encrypt256;+	crypton_aes_branch_table[DECRYPT_GCM_128] = crypton_aesni_gcm_decrypt128;+	crypton_aes_branch_table[DECRYPT_GCM_192] = crypton_aesni_gcm_decrypt192;+	crypton_aes_branch_table[DECRYPT_GCM_256] = crypton_aesni_gcm_decrypt256; #endif } #endif -uint8_t *crypton_aes_cpu_init(void)+#ifdef WITH_ARMV8_CRYPTO+static void initialize_table_armv8(void) {+	if (!crypton_aes_armv8_available())+		return;+	crypton_aes_cpu_options[CPU_AESNI] = 1;++	crypton_aes_branch_table[INIT_128] = crypton_aes_armv8_init;+	crypton_aes_branch_table[INIT_192] = crypton_aes_armv8_init;+	crypton_aes_branch_table[INIT_256] = crypton_aes_armv8_init;++	crypton_aes_branch_table[ENCRYPT_BLOCK_128] = crypton_aes_armv8_encrypt_block128;+	crypton_aes_branch_table[DECRYPT_BLOCK_128] = crypton_aes_armv8_decrypt_block128;+	crypton_aes_branch_table[ENCRYPT_BLOCK_192] = crypton_aes_armv8_encrypt_block192;+	crypton_aes_branch_table[ENCRYPT_BLOCK_256] = crypton_aes_armv8_encrypt_block256;+	crypton_aes_branch_table[DECRYPT_BLOCK_192] = crypton_aes_armv8_decrypt_block192;+	crypton_aes_branch_table[DECRYPT_BLOCK_256] = crypton_aes_armv8_decrypt_block256;+	/* ECB */+	crypton_aes_branch_table[ENCRYPT_ECB_128] = crypton_aes_armv8_encrypt_ecb128;+	crypton_aes_branch_table[DECRYPT_ECB_128] = crypton_aes_armv8_decrypt_ecb128;+	crypton_aes_branch_table[ENCRYPT_ECB_192] = crypton_aes_armv8_encrypt_ecb192;+	crypton_aes_branch_table[ENCRYPT_ECB_256] = crypton_aes_armv8_encrypt_ecb256;+	crypton_aes_branch_table[DECRYPT_ECB_192] = crypton_aes_armv8_decrypt_ecb192;+	crypton_aes_branch_table[DECRYPT_ECB_256] = crypton_aes_armv8_decrypt_ecb256;+	/* CBC */+	crypton_aes_branch_table[ENCRYPT_CBC_128] = crypton_aes_armv8_encrypt_cbc128;+	crypton_aes_branch_table[DECRYPT_CBC_128] = crypton_aes_armv8_decrypt_cbc128;+	crypton_aes_branch_table[ENCRYPT_CBC_192] = crypton_aes_armv8_encrypt_cbc192;+	crypton_aes_branch_table[ENCRYPT_CBC_256] = crypton_aes_armv8_encrypt_cbc256;+	crypton_aes_branch_table[DECRYPT_CBC_192] = crypton_aes_armv8_decrypt_cbc192;+	crypton_aes_branch_table[DECRYPT_CBC_256] = crypton_aes_armv8_decrypt_cbc256;+	/* CTR, which the generic loop would otherwise drive one block at a time */+	crypton_aes_branch_table[ENCRYPT_CTR_128] = crypton_aes_armv8_encrypt_ctr128;+	crypton_aes_branch_table[ENCRYPT_CTR_192] = crypton_aes_armv8_encrypt_ctr192;+	crypton_aes_branch_table[ENCRYPT_CTR_256] = crypton_aes_armv8_encrypt_ctr256;+	/* XTS, likewise, in both directions */+	crypton_aes_branch_table[ENCRYPT_XTS_128] = crypton_aes_armv8_encrypt_xts128;+	crypton_aes_branch_table[DECRYPT_XTS_128] = crypton_aes_armv8_decrypt_xts128;+	crypton_aes_branch_table[ENCRYPT_XTS_192] = crypton_aes_armv8_encrypt_xts192;+	crypton_aes_branch_table[ENCRYPT_XTS_256] = crypton_aes_armv8_encrypt_xts256;+	crypton_aes_branch_table[DECRYPT_XTS_192] = crypton_aes_armv8_decrypt_xts192;+	crypton_aes_branch_table[DECRYPT_XTS_256] = crypton_aes_armv8_decrypt_xts256;++	/* GHASH, which GCM spends its time in once AES itself is fast */+	if (!crypton_aes_armv8_pmull_available())+		return;+	crypton_aes_cpu_options[CPU_PCLMUL] = 1;+	crypton_aes_branch_table[GHASH_HINIT]  = crypton_aes_armv8_hinit_pmull;+	crypton_aes_branch_table[GHASH_GF_MUL]  = crypton_aes_armv8_gf_mul_pmull;+	crypton_aes_branch_table[GHASH_GF_MUL4] = crypton_aes_armv8_gf_mul4_pmull;++	/* GCM, which needs both halves and so waits until PMULL is known to+	 * be there; the generic loop stands in otherwise */+	crypton_aes_branch_table[ENCRYPT_GCM_128] = crypton_aes_armv8_gcm_encrypt128;+	crypton_aes_branch_table[DECRYPT_GCM_128] = crypton_aes_armv8_gcm_decrypt128;+	crypton_aes_branch_table[ENCRYPT_GCM_192] = crypton_aes_armv8_gcm_encrypt192;+	crypton_aes_branch_table[ENCRYPT_GCM_256] = crypton_aes_armv8_gcm_encrypt256;+	crypton_aes_branch_table[DECRYPT_GCM_192] = crypton_aes_armv8_gcm_decrypt192;+	crypton_aes_branch_table[DECRYPT_GCM_256] = crypton_aes_armv8_gcm_decrypt256;+}+#endif++/* Which implementation each entry of the branch table names is decided once,+ * before anything else runs.+ *+ * It used to be decided again on every crypton_aes_initkey, which meant two+ * threads taking a key at the same time were writing the whole table at the+ * same time -- a data race for as long as the program used AES, not just at+ * the start.  ThreadSanitizer reports forty-two of them for eight threads+ * doing nothing but taking keys.  The values written are the same ones every+ * time and the table starts out holding valid generic implementations, so+ * nothing has ever come of it; it is a race the standard gives no meaning to+ * all the same.+ *+ * A constructor runs while there is one thread, which is the cheapest way to+ * have no race at all: no flag to test, no lock to take, and one less thing+ * for crypton_aes_initkey to do per key. */+static void crypton_aes_cpu_setup(void)+{ #if defined(ARCH_X86) && defined(WITH_AESNI) 	crypton_aesni_initialize_hw(initialize_table_ni); #endif+#ifdef WITH_ARMV8_CRYPTO+	initialize_table_armv8();+#endif+}++__attribute__((constructor))+static void crypton_aes_cpu_ctor(void)+{+	crypton_aes_cpu_setup();+}++uint8_t *crypton_aes_cpu_init(void)+{ 	return crypton_aes_cpu_options; } @@ -303,7 +468,6 @@ 	case 24: key->nbr = 12; key->strength = 1; break; 	case 32: key->nbr = 14; key->strength = 2; break; 	}-	crypton_aes_cpu_init(); 	init_f _init = GET_INIT(key->strength); 	_init(key, origkey, size); }@@ -332,33 +496,6 @@ 	d(output, key, iv, input, nb_blocks); } -void crypton_aes_gen_ctr(aes_block *output, aes_key *key, const aes_block *iv, uint32_t nb_blocks)-{-	aes_block block;--	/* preload IV in block */-	block128_copy(&block, iv);--	for ( ; nb_blocks-- > 0; output++, block128_inc_be(&block)) {-		crypton_aes_encrypt_block(output, key, &block);-	}-}--void crypton_aes_gen_ctr_cont(aes_block *output, aes_key *key, aes_block *iv, uint32_t nb_blocks)-{-	aes_block block;--	/* preload IV in block */-	block128_copy(&block, iv);--	for ( ; nb_blocks-- > 0; output++, block128_inc_be(&block)) {-		crypton_aes_encrypt_block(output, key, &block);-	}--	/* copy back the IV */-	block128_copy(iv, &block);-}- void crypton_aes_encrypt_ctr(uint8_t *output, aes_key *key, aes_block *iv, uint8_t *input, uint32_t len) { 	ctr_f e = GET_CTR_ENCRYPT(key->strength);@@ -381,7 +518,8 @@ void crypton_aes_decrypt_xts(aes_block *output, aes_key *k1, aes_key *k2, aes_block *dataunit,                      uint32_t spoint, aes_block *input, uint32_t nb_blocks) {-	crypton_aes_generic_decrypt_xts(output, k1, k2, dataunit, spoint, input, nb_blocks);+	xts_f d = GET_XTS_DECRYPT(k1->strength);+	d(output, k1, k2, dataunit, spoint, input, nb_blocks); }  void crypton_aes_gcm_encrypt(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length)@@ -426,20 +564,40 @@ 	crypton_gf_mul(&gcm->tag, gcm->htable); } -void crypton_aes_gcm_init(aes_gcm *gcm, aes_key *key, uint8_t *iv, uint32_t len)+/* Same, for four consecutive blocks.  Where the multiply is a carry-less+ * instruction this costs one reduction instead of four. */+static void gcm_ghash_add4(aes_gcm *gcm, const block128 *b) {+	crypton_gf_mul4(&gcm->tag, b, gcm->htable);+}++/* The part of the state that depends on the key alone: H = encrypt_K(0^128)+ * and the table of its multiples.  It is 256 of the 320 bytes, and a caller+ * that keeps a key can compute it once instead of once per message. */+void crypton_aes_gcm_key_init(aes_gcm_key *gk, aes_key *key)+{ 	block128 h;++	block128_zero(&h);+	crypton_aes_encrypt_block(&h, key, &h);+	crypton_hinit(gk->gcm.htable, &h);+#ifdef WITH_GCM_FUSED+	if (crypton_aes_cpu_options[CPU_AESNI]+	    && crypton_aes_cpu_options[CPU_PCLMUL])+		crypton_gcm_fused_key_init(&gk->fused, key);+#endif+}++/* Everything else: what the nonce and the message determine.  Leaves htable+ * alone, so it runs on a state whose key part is already there. */+static void gcm_message_init(aes_gcm *gcm, uint8_t *iv, uint32_t len)+{ 	gcm->length_aad = 0; 	gcm->length_input = 0; -	block128_zero(&h); 	block128_zero(&gcm->tag); 	block128_zero(&gcm->iv); -	/* prepare H : encrypt_K(0^128) */-	crypton_aes_encrypt_block(&h, key, &h);-	crypton_hinit(gcm->htable, &h);- 	if (len == 12) { 		block128_copy_bytes(&gcm->iv, iv, 12); 		gcm->iv.b[15] = 0x01;@@ -462,9 +620,22 @@ 	block128_copy_aligned(&gcm->civ, &gcm->iv); } +void crypton_aes_gcm_init(aes_gcm *gcm, aes_key *key, uint8_t *iv, uint32_t len)+{+	block128 h;++	block128_zero(&h);+	crypton_aes_encrypt_block(&h, key, &h);+	crypton_hinit(gcm->htable, &h);+	gcm_message_init(gcm, iv, len);+}+ void crypton_aes_gcm_aad(aes_gcm *gcm, uint8_t *input, uint32_t length) { 	gcm->length_aad += length;+	for (; length >= 64; input += 64, length -= 64) {+		gcm_ghash_add4(gcm, (const block128 *) input);+	} 	for (; length >= 16; input += 16, length -= 16) { 		gcm_ghash_add(gcm, (block128 *) input); 	}@@ -495,6 +666,195 @@ 	} } +/* One message, one call.  The key part of the state comes in already built,+ * the rest is set up on the stack, and the additional data, the encryption+ * and the tag all happen before returning, so nothing crosses a language+ * boundary between them and no intermediate state is copied out.  The output+ * buffer takes the ciphertext and then the tag, so it wants length + taglen+ * bytes. */+void crypton_aes_gcm_full_encrypt(uint8_t *output, const aes_gcm_key *gcmkey, aes_key *key,+                                  uint8_t *iv, uint32_t ivlen,+                                  uint8_t *aad, uint32_t aadlen,+                                  uint8_t *input, uint32_t length, uint32_t taglen)+{+	aes_gcm gcm;+	uint8_t tag[16];++#ifdef WITH_GCM_FUSED+	/* Short messages go the other way: the assembly below will not start+	 * on anything under 288 bytes, and under about 1.5 KB the fused path+	 * is ahead of it even where it does. */+	if (ivlen == 12 && length <= CRYPTON_GCM_FUSED_MAX_MESSAGE+	    && crypton_aes_cpu_options[CPU_AESNI]+	    && crypton_aes_cpu_options[CPU_PCLMUL]) {+		crypton_gcm_fused_encrypt(output, &gcmkey->fused, key, iv,+		                          aad, aadlen, input, length, taglen,+		                          NULL, 0, NULL);+		return;+	}+#endif+#ifdef WITH_ARMV8_CRYPTO+	/* The same on AArch64, where the framing is what costs: composing the+	 * additional data, the encryption and the tag reaches each through the+	 * branch table, so the running state goes back to memory between them+	 * and a one-block header pays a reduction of its own.  Measured on an+	 * Apple M4, a 100-byte packet is 3.0x faster taken in one call.+	 *+	 * No length limit, unlike x86: there is no vendored assembly on this+	 * side for a long message to be handed to instead, and measured+	 * against the path this replaces it is never slower -- 1.25x at 1440+	 * bytes, level from about 6 KB up. */+	if (ivlen == 12+	    && crypton_aes_cpu_options[CPU_AESNI]+	    && crypton_aes_cpu_options[CPU_PCLMUL]) {+		crypton_aes_armv8_gcm_fused(output, gcmkey->gcm.htable, key, iv,+		                            aad, aadlen, input, length, taglen,+		                            NULL, 0, NULL);+		return;+	}+#endif+	memcpy(gcm.htable, gcmkey->gcm.htable, sizeof(gcm.htable));+	gcm_message_init(&gcm, iv, ivlen);+	if (aadlen)+		crypton_aes_gcm_aad(&gcm, aad, aadlen);+	if (length)+		crypton_aes_gcm_encrypt(output, &gcm, key, input, length);+	crypton_aes_gcm_finish(tag, &gcm, key);+	memcpy(output + length, tag, taglen);+}++/* The same, and then the header protection mask.  QUIC takes its sample from+ * the ciphertext, so the mask cannot be had before the encryption -- but it+ * can be had before returning, which saves a second crossing for one AES+ * block.  The block itself is about a nanosecond; what it saves is the call.+ * sampleoff is where the sixteen bytes of sample start in the output. */+void crypton_aes_gcm_full_encrypt_mask(uint8_t *output, const aes_gcm_key *gcmkey, aes_key *key,+                                       uint8_t *iv, uint32_t ivlen,+                                       uint8_t *aad, uint32_t aadlen,+                                       uint8_t *input, uint32_t length, uint32_t taglen,+                                       aes_key *hpkey, uint32_t sampleoff, uint8_t *mask)+{+	block128 sample, m;++#ifdef WITH_GCM_FUSED+	/* Here the mask rides in a lane of the AES pipeline that the message+	 * length leaves idle, so it costs very nearly nothing on top of the+	 * encryption rather than a block of its own. */+	if (ivlen == 12 && length <= CRYPTON_GCM_FUSED_MAX_MESSAGE+	    && crypton_aes_cpu_options[CPU_AESNI]+	    && crypton_aes_cpu_options[CPU_PCLMUL]) {+		crypton_gcm_fused_encrypt(output, &gcmkey->fused, key, iv,+		                          aad, aadlen, input, length, taglen,+		                          hpkey, sampleoff, mask);+		return;+	}+#endif+#ifdef WITH_ARMV8_CRYPTO+	/* The same on AArch64, where the framing is what costs: composing the+	 * additional data, the encryption and the tag reaches each through the+	 * branch table, so the running state goes back to memory between them+	 * and a one-block header pays a reduction of its own.  Measured on an+	 * Apple M4, a 100-byte packet is 3.3x faster taken in one call. */+	if (ivlen == 12+	    && crypton_aes_cpu_options[CPU_AESNI]+	    && crypton_aes_cpu_options[CPU_PCLMUL]) {+		crypton_aes_armv8_gcm_fused(output, gcmkey->gcm.htable, key, iv,+		                            aad, aadlen, input, length, taglen,+		                            hpkey, sampleoff, mask);+		return;+	}+#endif+	crypton_aes_gcm_full_encrypt(output, gcmkey, key, iv, ivlen, aad, aadlen,+	                             input, length, taglen);+	/* copied rather than cast: the sample lands wherever the header put it+	 * and a block128 is read as 64-bit words */+	memcpy(&sample, output + sampleoff, 16);+	crypton_aes_encrypt_block(&m, hpkey, &sample);+	memcpy(mask, &m, 16);+}++/* The same the other way, with the tag checked here rather than by the+ * caller: returns 1 when it matches and 0 when it does not, comparing every+ * byte either way.  The plaintext is written whatever the answer, so a caller+ * that gets 0 must not use it. */+/* Decrypt, and either compare the tag or hand it back.+ *+ * With outtag NULL this is the verifying form: the tag is compared here, a+ * byte at a time over its whole length whichever way the answer goes, and the+ * answer is the return value.  With outtag not NULL the computed tag is+ * written there instead and the return value is 1 -- for a caller that holds+ * the expected tag in a form of its own and will compare it itself. */+static int gcm_full_decrypt(uint8_t *output, const aes_gcm_key *gcmkey, aes_key *key,+                            uint8_t *iv, uint32_t ivlen,+                            uint8_t *aad, uint32_t aadlen,+                            uint8_t *input, uint32_t length,+                            const uint8_t *tag, uint32_t taglen,+                            uint8_t *outtag)+{+	aes_gcm gcm;+	uint8_t expected[16];+	uint32_t i;+	uint8_t diff = 0;++#ifdef WITH_GCM_FUSED+	/* The same as the encryption side, and simpler: what GHASH absorbs+	 * here is the ciphertext, which is the input, so the multiplies need+	 * not wait for anything.  Measured on an Intel Haswell, a 100-byte+	 * packet was three times the cost of encrypting one before this. */+	if (ivlen == 12 && length <= CRYPTON_GCM_FUSED_MAX_MESSAGE+	    && crypton_aes_cpu_options[CPU_AESNI]+	    && crypton_aes_cpu_options[CPU_PCLMUL])+		return crypton_gcm_fused_decrypt(output, &gcmkey->fused, key,+		                                 iv, aad, aadlen, input,+		                                 length, tag, taglen, outtag);+#endif+#ifdef WITH_ARMV8_CRYPTO+	if (ivlen == 12+	    && crypton_aes_cpu_options[CPU_AESNI]+	    && crypton_aes_cpu_options[CPU_PCLMUL])+		return crypton_aes_armv8_gcm_fused_dec(output, gcmkey->gcm.htable,+		                                       key, iv, aad, aadlen,+		                                       input, length, tag, taglen,+		                                       outtag);+#endif+	memcpy(gcm.htable, gcmkey->gcm.htable, sizeof(gcm.htable));+	gcm_message_init(&gcm, iv, ivlen);+	if (aadlen)+		crypton_aes_gcm_aad(&gcm, aad, aadlen);+	if (length)+		crypton_aes_gcm_decrypt(output, &gcm, key, input, length);+	crypton_aes_gcm_finish(expected, &gcm, key);++	if (outtag) {+		memcpy(outtag, expected, taglen);+		return 1;+	}+	for (i = 0; i < taglen; i++)+		diff |= (uint8_t) (expected[i] ^ tag[i]);+	return diff == 0;+}++int crypton_aes_gcm_full_decrypt(uint8_t *output, const aes_gcm_key *gcmkey, aes_key *key,+                                 uint8_t *iv, uint32_t ivlen,+                                 uint8_t *aad, uint32_t aadlen,+                                 uint8_t *input, uint32_t length,+                                 const uint8_t *tag, uint32_t taglen)+{+	return gcm_full_decrypt(output, gcmkey, key, iv, ivlen, aad, aadlen,+	                        input, length, tag, taglen, NULL);+}++void crypton_aes_gcm_full_decrypt_tag(uint8_t *output, uint8_t *outtag,+                                      const aes_gcm_key *gcmkey, aes_key *key,+                                      uint8_t *iv, uint32_t ivlen,+                                      uint8_t *aad, uint32_t aadlen,+                                      uint8_t *input, uint32_t length,+                                      uint32_t taglen)+{+	(void) gcm_full_decrypt(output, gcmkey, key, iv, ivlen, aad, aadlen,+	                        input, length, NULL, taglen, outtag);+}+ static inline uint8_t ccm_b0_flags(uint32_t has_adata, uint32_t m, uint32_t l) { 	return 8*m + l + (has_adata? 64: 0);@@ -655,7 +1015,22 @@ #undef L_CACHED } -void crypton_aes_ocb_init(aes_ocb *ocb, aes_key *key, uint8_t *iv, uint32_t len)+/*+ * OCB's offsets are a running exclusive-or, so they have to be worked out in+ * order, but the block cipher calls under them do not depend on each other:+ * every block is offset, encrypted, and offset again.  So the offsets are+ * computed a group at a time and the group goes through ECB together, which+ * is where the code written for the AES instructions interleaves eight blocks+ * and covers the latency of AESENC.  One block at a time left that idle and+ * cost four times what GCM costs on the same machine, for a mode that does+ * less work than GCM.+ *+ * Eight is what the ECB paths interleave; a group beyond that gains nothing+ * and only makes the buffers larger.+ */+#define OCB_WAY 8++void crypton_aes_ocb_init(aes_ocb *ocb, aes_key *key, uint8_t *iv, uint32_t len, uint32_t taglen) { 	block128 tmp, nonce, ktop; 	unsigned char stretch[24];@@ -665,6 +1040,9 @@ 	if (len > 15) { 		len = 15; 	}+	if (taglen > 16) {+		taglen = 16;+	}  	/* create L*, and L$,L0,L1,L2,L3 */ 	block128_zero(&tmp);@@ -678,9 +1056,11 @@  	/* create strech from the nonce */ 	block128_zero(&nonce);-	memcpy(nonce.b + 4, iv, 12);-	nonce.b[0] = (unsigned char)(((16 * 8) % 128) << 1);-	nonce.b[16-12-1] |= 0x01;+	if (len > 0) {+		memcpy(nonce.b + (16 - len), iv, len);+		nonce.b[16 - len - 1] |= 0x01;+	}+	nonce.b[0] |= (unsigned char)(((taglen * 8) % 128) << 1); 	bottom = nonce.b[15] & 0x3F; 	nonce.b[15] &= 0xC0; 	crypton_aes_encrypt_block(&ktop, key, &nonce);@@ -709,9 +1089,23 @@ void crypton_aes_ocb_aad(aes_ocb *ocb, aes_key *key, uint8_t *input, uint32_t length) { 	block128 tmp;-	unsigned int i;+	block128 buf[OCB_WAY];+	uint32_t blocks = length / 16;+	unsigned int i = 1, j; -	for (i=1; i<= length/16; i++, input=input+16) {+	for (; blocks >= OCB_WAY; blocks -= OCB_WAY, input += 16 * OCB_WAY) {+		for (j = 0; j < OCB_WAY; j++, i++) {+			ocb_get_L_i(&tmp, ocb->li, i);+			block128_xor_aligned(&ocb->offset_aad, &tmp);+			block128_vxor(&buf[j], &ocb->offset_aad,+			              (block128 *) (input + 16 * j));+		}+		crypton_aes_encrypt_ecb(buf, key, buf, OCB_WAY);+		for (j = 0; j < OCB_WAY; j++)+			block128_xor_aligned(&ocb->sum_aad, &buf[j]);+	}++	for (; blocks > 0; blocks--, i++, input += 16) { 		ocb_get_L_i(&tmp, ocb->li, i); 		block128_xor_aligned(&ocb->offset_aad, &tmp); @@ -877,6 +1271,20 @@ 	aes_block out;  	gcm->length_input += length;+	/* four blocks at a time, so GHASH can fold them into one reduction */+	for (; length >= 64; input += 64, output += 64, length -= 64) {+		aes_block buf[4];+		int i;++		for (i = 0; i < 4; i++) {+			block128_inc32_be(&gcm->civ);+			crypton_aes_encrypt_block(&buf[i], key, &gcm->civ);+			block128_xor(&buf[i], (block128 *) (input + 16 * i));+		}+		gcm_ghash_add4(gcm, buf);+		for (i = 0; i < 4; i++)+			block128_copy((block128 *) (output + 16 * i), &buf[i]);+	} 	for (; length >= 16; input += 16, output += 16, length -= 16) { 		block128_inc32_be(&gcm->civ); @@ -910,6 +1318,19 @@ 	aes_block out;  	gcm->length_input += length;+	/* GHASH all four ciphertext blocks before writing any plaintext, since+	 * output may be input */+	for (; length >= 64; input += 64, output += 64, length -= 64) {+		int i;++		gcm_ghash_add4(gcm, (const block128 *) input);+		for (i = 0; i < 4; i++) {+			block128_inc32_be(&gcm->civ);+			crypton_aes_encrypt_block(&out, key, &gcm->civ);+			block128_xor(&out, (block128 *) (input + 16 * i));+			block128_copy((block128 *) (output + 16 * i), &out);+		}+	} 	for (; length >= 16; input += 16, output += 16, length -= 16) { 		block128_inc32_be(&gcm->civ); @@ -941,10 +1362,37 @@                               uint8_t *input, uint32_t length, int encrypt) { 	block128 tmp, pad;-	unsigned int i;+	block128 offsets[OCB_WAY], buf[OCB_WAY];+	uint32_t blocks = length / 16;+	unsigned int i = 1, j; -	for (i = 1; i <= length/16; i++, input += 16, output += 16) {-		/* Offset_i = Offset_{i-1} xor L_{ntz(i)} */+	for (; blocks >= OCB_WAY;+	     blocks -= OCB_WAY, input += 16 * OCB_WAY, output += 16 * OCB_WAY) {+		for (j = 0; j < OCB_WAY; j++, i++) {+			/* Offset_i = Offset_{i-1} xor L_{ntz(i)} */+			ocb_get_L_i(&tmp, ocb->li, i);+			block128_xor_aligned(&ocb->offset_enc, &tmp);+			block128_copy_aligned(&offsets[j], &ocb->offset_enc);+			block128_vxor(&buf[j], &ocb->offset_enc,+			              (block128 *) (input + 16 * j));+		}++		if (encrypt)+			crypton_aes_encrypt_ecb(buf, key, buf, OCB_WAY);+		else+			crypton_aes_decrypt_ecb(buf, key, buf, OCB_WAY);++		for (j = 0; j < OCB_WAY; j++) {+			block128_vxor((block128 *) (output + 16 * j),+			              &offsets[j], &buf[j]);+			block128_xor(&ocb->sum_enc,+			             (block128 *) ((encrypt ? input : output)+			                           + 16 * j));+		}+	}++	/* and what is left of the message, a block at a time */+	for (; blocks > 0; blocks--, i++, input += 16, output += 16) { 		ocb_get_L_i(&tmp, ocb->li, i); 		block128_xor_aligned(&ocb->offset_enc, &tmp);
cbits/crypton_aes.h view
@@ -55,6 +55,55 @@ 	uint64_t length_input; } aes_gcm; +/*+ * How many powers of H a key keeps for the fused path in+ * cbits/aes/gcm_fused_x86.c.  A power for every block of the message would+ * fold its whole GHASH into one reduction, which is what picotls does, but+ * then the state grows with the longest message a caller might send and a+ * server holding many keys pays it for each.  A fixed count costs one+ * reduction per this many blocks and keeps the state one size.  Sixteen was+ * measured against 6, 8, 32, 64, 96 and 256: above eight the choice is worth+ * about two per cent, since only messages short enough to take this path at+ * all reach a second batch.  Six is worth avoiding -- at 1440 bytes it is+ * slower than not taking the path.+ */+#define CRYPTON_GCM_FUSED_POWERS 16++/*+ * Beyond this many bytes the stitched assembly in cbits/asm is faster than+ * the fused path, so longer messages go there instead.  Measured on an Intel+ * Haswell: even at 1440 bytes, the assembly ahead by 12 per cent at 3 KB and+ * 20 per cent at 16 KB, and the fused path ahead by 1.9x at 100 bytes and+ * 1.16x at 1200.  QUIC packets fall below this; TLS records do not.+ */+#define CRYPTON_GCM_FUSED_MAX_MESSAGE 1536++/*+ * The powers themselves, each shifted up by one bit, and beside each the+ * halves of it added together for the Karatsuba term.  The two are kept+ * adjacent rather than in two arrays: a multiply wants both, and two arrays+ * put them 256 bytes apart, which is two cache lines where this is one.+ *+ * Defined on every platform so that the key state below is one size+ * everywhere; filled only where the fused path is compiled in.+ */+typedef struct {+	struct {+		aes_block h;+		aes_block r;+	} p[CRYPTON_GCM_FUSED_POWERS];+} aes_gcm_fused;++/*+ * Everything a key determines, built once by crypton_aes_gcm_key_init and+ * read by every message sent under that key: the key half of a GCM state,+ * and the powers of H the fused path reads.  832 bytes.+ */+typedef struct {+	aes_gcm gcm;+	aes_gcm_fused fused;+} aes_gcm_key;+ /* size = 4*16+4*4= 80 */ typedef struct { 	aes_block xi;@@ -95,8 +144,8 @@ void crypton_aes_encrypt_cbc(aes_block *output, aes_key *key, aes_block *iv, aes_block *input, uint32_t nb_blocks); void crypton_aes_decrypt_cbc(aes_block *output, aes_key *key, aes_block *iv, aes_block *input, uint32_t nb_blocks); -void crypton_aes_gen_ctr(aes_block *output, aes_key *key, const aes_block *iv, uint32_t nb_blocks);-void crypton_aes_gen_ctr_cont(aes_block *output, aes_key *key, aes_block *iv, uint32_t nb_blocks);+void crypton_aes_encrypt_ctr(uint8_t *output, aes_key *key, aes_block *iv, uint8_t *input, uint32_t len);+void crypton_aes_encrypt_c32(uint8_t *output, aes_key *key, aes_block *iv, uint8_t *input, uint32_t len);  void crypton_aes_encrypt_xts(aes_block *output, aes_key *key, aes_key *key2, aes_block *sector,                      uint32_t spoint, aes_block *input, uint32_t nb_blocks);@@ -104,12 +153,33 @@                      uint32_t spoint, aes_block *input, uint32_t nb_blocks);  void crypton_aes_gcm_init(aes_gcm *gcm, aes_key *key, uint8_t *iv, uint32_t len);+void crypton_aes_gcm_key_init(aes_gcm_key *gk, aes_key *key);+void crypton_aes_gcm_full_encrypt(uint8_t *output, const aes_gcm_key *gcmkey, aes_key *key,+                                  uint8_t *iv, uint32_t ivlen,+                                  uint8_t *aad, uint32_t aadlen,+                                  uint8_t *input, uint32_t length, uint32_t taglen);+void crypton_aes_gcm_full_encrypt_mask(uint8_t *output, const aes_gcm_key *gcmkey, aes_key *key,+                                       uint8_t *iv, uint32_t ivlen,+                                       uint8_t *aad, uint32_t aadlen,+                                       uint8_t *input, uint32_t length, uint32_t taglen,+                                       aes_key *hpkey, uint32_t sampleoff, uint8_t *mask);+int crypton_aes_gcm_full_decrypt(uint8_t *output, const aes_gcm_key *gcmkey, aes_key *key,+                                 uint8_t *iv, uint32_t ivlen,+                                 uint8_t *aad, uint32_t aadlen,+                                 uint8_t *input, uint32_t length,+                                 const uint8_t *tag, uint32_t taglen);+void crypton_aes_gcm_full_decrypt_tag(uint8_t *output, uint8_t *outtag,+                                      const aes_gcm_key *gcmkey, aes_key *key,+                                      uint8_t *iv, uint32_t ivlen,+                                      uint8_t *aad, uint32_t aadlen,+                                      uint8_t *input, uint32_t length,+                                      uint32_t taglen); void crypton_aes_gcm_aad(aes_gcm *gcm, uint8_t *input, uint32_t length); void crypton_aes_gcm_encrypt(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length); void crypton_aes_gcm_decrypt(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length); void crypton_aes_gcm_finish(uint8_t *tag, aes_gcm *gcm, aes_key *key); -void crypton_aes_ocb_init(aes_ocb *ocb, aes_key *key, uint8_t *iv, uint32_t len);+void crypton_aes_ocb_init(aes_ocb *ocb, aes_key *key, uint8_t *iv, uint32_t len, uint32_t taglen); void crypton_aes_ocb_aad(aes_ocb *ocb, aes_key *key, uint8_t *input, uint32_t length); void crypton_aes_ocb_encrypt(uint8_t *output, aes_ocb *ocb, aes_key *key, uint8_t *input, uint32_t length); void crypton_aes_ocb_decrypt(uint8_t *output, aes_ocb *ocb, aes_key *key, uint8_t *input, uint32_t length);
cbits/crypton_align.h view
@@ -3,6 +3,8 @@  #include "crypton_bitfn.h" +#include <string.h>+ #if (defined(__i386__)) # define UNALIGNED_ACCESS_OK #elif defined(__x86_64__)@@ -34,107 +36,119 @@ #define need_alignment(p,n) IS_ALIGNED(p,n) #endif -static inline uint32_t load_le32_aligned(const uint8_t *p)-{-	return le32_to_cpu(*((uint32_t *) p));		-}+/*+ * Reading and writing a 32- or 64-bit word at a byte pointer.+ *+ * Through memcpy, not a cast to uint32_t * or uint64_t *.  A cast is two+ * things the standard does not allow -- a read of the value through the+ * wrong type, and a read at an address that type is not aligned for -- and+ * this file used to do both wherever UNALIGNED_ACCESS_OK is defined, which+ * is i386 and x86-64.  UndefinedBehaviorSanitizer reported seventy-eight+ * lines of it.+ *+ * Every compiler crypton is built with turns a memcpy of four or eight bytes+ * into the one load or store the cast used to be, so this is the same code+ * with none of the licence.  Where the target cannot do an unaligned load,+ * the compiler is the one that knows, and it emits what the target needs --+ * which is what the byte-at-a-time versions this replaces were for.+ *+ * The _aligned names stay because nineteen files use them.  They no longer+ * ask anything of the pointer.+ */ -static inline void store_le32_aligned(uint8_t *dst, const uint32_t v)+static inline uint32_t load_le32(const uint8_t *p) {-	*((uint32_t *) dst) = cpu_to_le32(v);-}+	uint32_t v; -static inline void xor_le32_aligned(uint8_t *dst, const uint32_t v)-{-	*((uint32_t *) dst) ^= cpu_to_le32(v);+	memcpy(&v, p, sizeof(v));+	return le32_to_cpu(v); } -static inline void store_be32_aligned(uint8_t *dst, const uint32_t v)+static inline uint64_t load_le64(const uint8_t *p) {-	*((uint32_t *) dst) = cpu_to_be32(v);-}+	uint64_t v; -static inline void xor_be32_aligned(uint8_t *dst, const uint32_t v)-{-	*((uint32_t *) dst) ^= cpu_to_be32(v);+	memcpy(&v, p, sizeof(v));+	return le64_to_cpu(v); } -static inline void store_le64_aligned(uint8_t *dst, const uint64_t v)+static inline uint32_t load_be32(const uint8_t *p) {-	*((uint64_t *) dst) = cpu_to_le64(v);-}+	uint32_t v; -static inline void store_be64_aligned(uint8_t *dst, const uint64_t v)-{-	*((uint64_t *) dst) = cpu_to_be64(v);+	memcpy(&v, p, sizeof(v));+	return be32_to_cpu(v); } -static inline void xor_be64_aligned(uint8_t *dst, const uint64_t v)+static inline uint64_t load_be64(const uint8_t *p) {-	*((uint64_t *) dst) ^= cpu_to_be64(v);-}+	uint64_t v; -#ifdef UNALIGNED_ACCESS_OK-#define load_le32(a) load_le32_aligned(a)-#else-static inline uint32_t load_le32(const uint8_t *p)-{-	return ((uint32_t)p[0]) | ((uint32_t)p[1] <<  8) | ((uint32_t)p[2] << 16) | ((uint32_t)p[3] << 24);+	memcpy(&v, p, sizeof(v));+	return be64_to_cpu(v); }-#endif -#ifdef UNALIGNED_ACCESS_OK-#define store_le32(a, b) store_le32_aligned(a, b)-#define xor_le32(a, b) xor_le32_aligned(a, b)-#else static inline void store_le32(uint8_t *dst, const uint32_t v) {-	dst[0] = v; dst[1] = v >> 8; dst[2] = v >> 16; dst[3] = v >> 24;+	uint32_t w = cpu_to_le32(v);++	memcpy(dst, &w, sizeof(w)); }+ static inline void xor_le32(uint8_t *dst, const uint32_t v) {-	dst[0] ^= v; dst[1] ^= v >> 8; dst[2] ^= v >> 16; dst[3] ^= v >> 24;+	store_le32(dst, le32_to_cpu(load_le32(dst)) ^ v); }-#endif -#ifdef UNALIGNED_ACCESS_OK-#define store_be32(a, b) store_be32_aligned(a, b)-#define xor_be32(a, b) xor_be32_aligned(a, b)-#else static inline void store_be32(uint8_t *dst, const uint32_t v) {-	dst[3] = v; dst[2] = v >> 8; dst[1] = v >> 16; dst[0] = v >> 24;+	uint32_t w = cpu_to_be32(v);++	memcpy(dst, &w, sizeof(w)); }+ static inline void xor_be32(uint8_t *dst, const uint32_t v) {-	dst[3] ^= v; dst[2] ^= v >> 8; dst[1] ^= v >> 16; dst[0] ^= v >> 24;+	uint32_t w;++	memcpy(&w, dst, sizeof(w));+	w ^= cpu_to_be32(v);+	memcpy(dst, &w, sizeof(w)); }-#endif -#ifdef UNALIGNED_ACCESS_OK-#define store_le64(a, b) store_le64_aligned(a, b)-#else static inline void store_le64(uint8_t *dst, const uint64_t v) {-	dst[0] = v      ; dst[1] = v >> 8 ; dst[2] = v >> 16; dst[3] = v >> 24;-	dst[4] = v >> 32; dst[5] = v >> 40; dst[6] = v >> 48; dst[7] = v >> 56;+	uint64_t w = cpu_to_le64(v);++	memcpy(dst, &w, sizeof(w)); }-#endif -#ifdef UNALIGNED_ACCESS_OK-#define store_be64(a, b) store_be64_aligned(a, b)-#define xor_be64(a, b) xor_be64_aligned(a, b)-#else static inline void store_be64(uint8_t *dst, const uint64_t v) {-	dst[7] = v      ; dst[6] = v >> 8 ; dst[5] = v >> 16; dst[4] = v >> 24;-	dst[3] = v >> 32; dst[2] = v >> 40; dst[1] = v >> 48; dst[0] = v >> 56;+	uint64_t w = cpu_to_be64(v);++	memcpy(dst, &w, sizeof(w)); }+ static inline void xor_be64(uint8_t *dst, const uint64_t v) {-	dst[7] ^= v      ; dst[6] ^= v >> 8 ; dst[5] ^= v >> 16; dst[4] ^= v >> 24;-	dst[3] ^= v >> 32; dst[2] ^= v >> 40; dst[1] ^= v >> 48; dst[0] ^= v >> 56;+	uint64_t w;++	memcpy(&w, dst, sizeof(w));+	w ^= cpu_to_be64(v);+	memcpy(dst, &w, sizeof(w)); }-#endif++#define load_le32_aligned(p)     load_le32(p)+#define load_le64_aligned(p)     load_le64(p)+#define load_be32_aligned(p)     load_be32(p)+#define load_be64_aligned(p)     load_be64(p)+#define store_le32_aligned(d, v) store_le32(d, v)+#define xor_le32_aligned(d, v)   xor_le32(d, v)+#define store_be32_aligned(d, v) store_be32(d, v)+#define xor_be32_aligned(d, v)   xor_be32(d, v)+#define store_le64_aligned(d, v) store_le64(d, v)+#define store_be64_aligned(d, v) store_be64(d, v)+#define xor_be64_aligned(d, v)   xor_be64(d, v)  #endif
+ cbits/crypton_armv8_target.h view
@@ -0,0 +1,40 @@+/*+ * Asking for an AArch64 extension on one function.+ *+ * The instructions these files use are extensions, so a translation unit+ * compiled for the baseline may not emit them, and the function that does has+ * to say which extension it needs.  The two compilers spell that differently+ * and did not always:+ *+ *   clang          target("+crypto")            for as long as it matters+ *   GCC 13 and up  target("+crypto")            as well+ *   GCC before 13  target("arch=armv8-a+crypto")  -- the bare "+feature" form+ *                  is not understood, and the extension never reaches the+ *                  function, so an always_inline intrinsic that needs it+ *                  fails to inline and the build stops+ *+ * That last case is #273: gcc 12.2 on an aarch64 Linux could not build+ * cbits/sha3_armv8.c at all.  Naming the architecture as well as the+ * extension is understood by every version of both compilers, so GCC is given+ * that spelling and clang keeps the shorter one, which leaves whatever+ * baseline the caller chose alone.+ */+#ifndef CRYPTON_ARMV8_TARGET_H+#define CRYPTON_ARMV8_TARGET_H++#ifdef WITH_TARGET_ATTRIBUTES+#if defined(__clang__)+#define CRYPTON_TARGET_ARMV8_CRYPTO __attribute__((target("+crypto")))+#define CRYPTON_TARGET_ARMV8_SHA3 __attribute__((target("+sha3")))+#else+#define CRYPTON_TARGET_ARMV8_CRYPTO \+	__attribute__((target("arch=armv8-a+crypto")))+#define CRYPTON_TARGET_ARMV8_SHA3 \+	__attribute__((target("arch=armv8.2-a+sha3")))+#endif+#else+#define CRYPTON_TARGET_ARMV8_CRYPTO+#define CRYPTON_TARGET_ARMV8_SHA3+#endif++#endif
+ cbits/crypton_bignum.h view
@@ -0,0 +1,512 @@+/*+ * Arithmetic on numbers held as arrays of limbs, least significant first.+ *+ * The modular multiplication is Montgomery's, and every choice it makes --+ * which of two numbers to keep after the final subtraction, which entry of a+ * table to take -- is made with a mask rather than a branch, so that the+ * values being worked on do not steer the work.  The exponentiation in+ * crypton_powm.c and the curve arithmetic in crypton_ecc.c are both built on+ * this.+ *+ * Everything here is static inline: each file that includes it gets its own+ * copy, which the compiler can specialise to the sizes it uses.+ */+#ifndef CRYPTON_BIGNUM_H+#define CRYPTON_BIGNUM_H++#include <stdint.h>+#include <string.h>++#if defined(__SIZEOF_INT128__)+typedef uint64_t limb_t;+typedef unsigned __int128 dlimb_t;+#define LIMB_BITS 64+#else+typedef uint32_t limb_t;+typedef uint64_t dlimb_t;+#define LIMB_BITS 32+#endif++#define LIMB_BYTES (LIMB_BITS / 8)++/* four bits of exponent per window, so a table of sixteen and no leftover+ * bits: a byte holds exactly two windows */+#define WINDOW_BITS 4+#define TABLE_SIZE (1 << WINDOW_BITS)++/* r = a - b, returning the borrow out of the top */+static inline limb_t sub_n(limb_t *r, const limb_t *a, const limb_t *b, uint32_t n)+{+	limb_t borrow = 0;+	uint32_t i;++	for (i = 0; i < n; i++) {+		limb_t ai = a[i], bi = b[i];+		limb_t d = ai - bi - borrow;+		/* borrow out, without branching */+		borrow = ((~ai & bi) | (~(ai ^ bi) & d)) >> (LIMB_BITS - 1);+		r[i] = d;+	}+	return borrow;+}++/* r = a + b, returning the carry out of the top */+static inline limb_t add_n(limb_t *r, const limb_t *a, const limb_t *b,+                           uint32_t n)+{+	limb_t carry = 0;+	uint32_t i;++	for (i = 0; i < n; i++) {+		dlimb_t s = (dlimb_t) a[i] + b[i] + carry;++		r[i] = (limb_t) s;+		carry = (limb_t) (s >> LIMB_BITS);+	}+	return carry;+}++/* a = 2a, returning the bit shifted out of the top */+static inline limb_t shl1(limb_t *a, uint32_t n)+{+	limb_t carry = 0;+	uint32_t i;++	for (i = 0; i < n; i++) {+		limb_t next = a[i] >> (LIMB_BITS - 1);+		a[i] = (a[i] << 1) | carry;+		carry = next;+	}+	return carry;+}++/* r = take ? a : b */+static inline void select_n(limb_t *r, const limb_t *a, const limb_t *b, limb_t take,+                     uint32_t n)+{+	limb_t mask = (limb_t) 0 - take;+	uint32_t i;++	for (i = 0; i < n; i++)+		r[i] = (a[i] & mask) | (b[i] & ~mask);+}++/* all ones when a and b are equal, zero otherwise */+static inline limb_t eq_mask(limb_t a, limb_t b)+{+	limb_t d = a ^ b;+	limb_t nz = d | ((limb_t) 0 - d); /* top bit set unless d is zero */++	return (limb_t) 0 - ((nz >> (LIMB_BITS - 1)) ^ 1);+}++/* -m^-1 mod 2^LIMB_BITS, for odd m */+static inline limb_t mont_n0(limb_t m0)+{+	limb_t inv = 1;+	int i;++	/* Newton's iteration doubles the number of correct bits each time */+	for (i = 0; i < 6; i++)+		inv *= (limb_t) 2 - m0 * inv;+	return (limb_t) 0 - inv;+}++/*+ * t += a * b over n limbs, returning the carry.  This is where nearly all of+ * the time goes.+ *+ * The C below takes the limbs eight at a time; what is left over at the end+ * is taken one at a time.  On AArch64 the compiler writes each limb as+ * `mul`, `umulh`, `adds`, `cset`, `adds`, `adc`: the carry out of one+ * 128-bit addition leaves the flags for a general register and is added back+ * in the next, because in C each addition is a statement of its own.  Two of+ * those instructions are that round trip.+ *+ * Three attempts to take them back measured worse than the C, and are+ * written down here so that they are not tried again -- one RSA-2048 CRT+ * private operation on an Apple M4, best of many:+ *+ *     this loop in inline assembly, a carry chain per limb       654.7 us+ *     the same with the loads hoisted out of the chain           644.5+ *     the multiply interleaved with its reduction (CIOS)         604.1+ *     the C below                                                590.1+ *     what the AArch64 block does, four limbs and two chains     514.9+ *     the same, with the ragged end of the row written out too   503.4+ *+ * The first two lose because a chain per limb serialises what the spare+ * `cset` lets overlap: `adds`, `adc`, `adds`, `adc` is four dependent steps+ * per limb, and a wide out-of-order core would rather have the extra+ * instruction than the dependency.  The third loses because shifting the+ * accumulator down a limb each round costs more than the round trip through+ * 2n limbs that it saves.  What works is neither: four limbs to an+ * iteration with the flags carrying through two long chains, one for the low+ * halves of the products and one for the high halves a place up.+ *+ * There is more still there.  OpenSSL's armv8-mont.pl runs a 16-limb+ * Montgomery multiplication at about 0.98 multiply-accumulates per cycle;+ * this file was at 0.55 and the block below brings it to 0.64, where 1.0 is+ * the ceiling -- a multiply-accumulate is two instructions and the machine+ * issues two multiplies a cycle.  That code cannot be borrowed: it is in+ * OpenSSL's tree only, under Apache-2.0, and CRYPTOGAMS, which this library+ * does vendor from, publishes no Montgomery generator at all.  BearSSL's+ * only ARM assembly is 32-bit Thumb for Cortex-M0 to M3, with fifteen-bit+ * limbs for cores that have no fast multiplier, and Botan's AArch64 inline+ * assembly is the `mul`/`umulh`/`adds`/`adc` primitive the compiler already+ * emits.+ */+#if defined(__aarch64__) && LIMB_BITS == 64 \+    && (defined(__GNUC__) || defined(__clang__))+/*+ * Four limbs to an iteration, accumulated in two chains rather than one per+ * limb: the low halves of the four products, with the carry coming in, are+ * one run of `adds` and `adcs`, and the high halves shifted up a place are+ * another.  The flags carry the whole way through each, which is what the C+ * above cannot say and what it pays for in `cset` and an extra add.+ *+ * The arrangement is the one in Go's crypto/internal/fips140/bigmod+ * (nat_arm64.s, addMulVVWx), which is BSD-3-Clause like this library --+ * cbits/LICENSE.go carries its notice.  Written out here in the assembler+ * this file's compiler speaks.+ */+static inline limb_t addmul_1(limb_t *t, const limb_t *a, uint32_t n, limb_t b)+{+	uint64_t carry = 0;+	uint64_t x0, x1, x2, x3, z0, z1, z2, z3;+	uint64_t l0, l1, l2, l3, h0, h1, h2, h3;+	uint64_t blocks = n / 4, left = n % 4;++	if (blocks) {+		__asm__ volatile(+		"1:\n\t"+		"ldp	%[x0], %[x1], [%[a]], #16\n\t"+		"ldp	%[x2], %[x3], [%[a]], #16\n\t"+		"ldp	%[z0], %[z1], [%[t]]\n\t"+		/* the low halves, one place up from the second chain, with+		 * the carry that came in */+		"adds	%[z0], %[z0], %[c]\n\t"+		"mul	%[l1], %[x1], %[b]\n\t"+		"adcs	%[z1], %[z1], %[l1]\n\t"+		"mul	%[l2], %[x2], %[b]\n\t"+		"ldp	%[z2], %[z3], [%[t], #16]\n\t"+		"adcs	%[z2], %[z2], %[l2]\n\t"+		"mul	%[l3], %[x3], %[b]\n\t"+		"adcs	%[z3], %[z3], %[l3]\n\t"+		"umulh	%[h3], %[x3], %[b]\n\t"+		"adc	%[h3], %[h3], xzr\n\t"+		/* and the high halves, which is where this block's own carry+		 * ends up */+		"mul	%[l0], %[x0], %[b]\n\t"+		"adds	%[z0], %[z0], %[l0]\n\t"+		"umulh	%[h0], %[x0], %[b]\n\t"+		"adcs	%[z1], %[z1], %[h0]\n\t"+		"umulh	%[h1], %[x1], %[b]\n\t"+		"stp	%[z0], %[z1], [%[t]], #16\n\t"+		"adcs	%[z2], %[z2], %[h1]\n\t"+		"umulh	%[h2], %[x2], %[b]\n\t"+		"adcs	%[z3], %[z3], %[h2]\n\t"+		"stp	%[z2], %[z3], [%[t]], #16\n\t"+		"adc	%[c], %[h3], xzr\n\t"+		"subs	%[k], %[k], #1\n\t"+		"b.ne	1b\n\t"+		: [a] "+r"(a), [t] "+r"(t), [c] "+r"(carry), [k] "+r"(blocks),+		  [x0] "=&r"(x0), [x1] "=&r"(x1), [x2] "=&r"(x2),+		  [x3] "=&r"(x3), [z0] "=&r"(z0), [z1] "=&r"(z1),+		  [z2] "=&r"(z2), [z3] "=&r"(z3), [l0] "=&r"(l0),+		  [l1] "=&r"(l1), [l2] "=&r"(l2), [l3] "=&r"(l3),+		  [h0] "=&r"(h0), [h1] "=&r"(h1), [h2] "=&r"(h2),+		  [h3] "=&r"(h3)+		: [b] "r"(b)+		: "cc", "memory");+	}+	/* What is left of the row: three limbs, two, or one, each the same two+	 * chains cut short.  This is not a rare case to be handed back to C --+	 * mont_sqr asks for every length from n-1 down to 1, so three rows in+	 * four end ragged.  Only 4.7% of the limbs in a 1024-bit exponentiation+	 * arrive here, but they were the dearer ones, and writing them out is+	 * worth the last two per cent in the table above.  The three lengths+	 * are spelled out rather than run as 2+1, because chaining two short+	 * blocks makes the second wait on the first: that costs two thirds of+	 * the gain.+	 */+	if (left == 3) {+		__asm__ volatile(+		"ldp	%[x0], %[x1], [%[a]]\n\t"+		"ldr	%[x2], [%[a], #16]\n\t"+		"add	%[a], %[a], #24\n\t"+		"ldp	%[z0], %[z1], [%[t]]\n\t"+		"ldr	%[z2], [%[t], #16]\n\t"+		"adds	%[z0], %[z0], %[c]\n\t"+		"mul	%[l1], %[x1], %[b]\n\t"+		"adcs	%[z1], %[z1], %[l1]\n\t"+		"mul	%[l2], %[x2], %[b]\n\t"+		"adcs	%[z2], %[z2], %[l2]\n\t"+		"umulh	%[h2], %[x2], %[b]\n\t"+		"adc	%[h2], %[h2], xzr\n\t"+		"mul	%[l0], %[x0], %[b]\n\t"+		"adds	%[z0], %[z0], %[l0]\n\t"+		"umulh	%[h0], %[x0], %[b]\n\t"+		"adcs	%[z1], %[z1], %[h0]\n\t"+		"umulh	%[h1], %[x1], %[b]\n\t"+		"stp	%[z0], %[z1], [%[t]], #16\n\t"+		"adcs	%[z2], %[z2], %[h1]\n\t"+		"str	%[z2], [%[t]], #8\n\t"+		"adc	%[c], %[h2], xzr\n\t"+		: [a] "+r"(a), [t] "+r"(t), [c] "+r"(carry),+		  [x0] "=&r"(x0), [x1] "=&r"(x1), [x2] "=&r"(x2),+		  [z0] "=&r"(z0), [z1] "=&r"(z1), [z2] "=&r"(z2),+		  [l0] "=&r"(l0), [l1] "=&r"(l1), [l2] "=&r"(l2),+		  [h0] "=&r"(h0), [h1] "=&r"(h1), [h2] "=&r"(h2)+		: [b] "r"(b)+		: "cc", "memory");+	} else if (left == 2) {+		__asm__ volatile(+		"ldp	%[x0], %[x1], [%[a]], #16\n\t"+		"ldp	%[z0], %[z1], [%[t]]\n\t"+		"adds	%[z0], %[z0], %[c]\n\t"+		"mul	%[l1], %[x1], %[b]\n\t"+		"adcs	%[z1], %[z1], %[l1]\n\t"+		"umulh	%[h1], %[x1], %[b]\n\t"+		"adc	%[h1], %[h1], xzr\n\t"+		"mul	%[l0], %[x0], %[b]\n\t"+		"adds	%[z0], %[z0], %[l0]\n\t"+		"umulh	%[h0], %[x0], %[b]\n\t"+		"adcs	%[z1], %[z1], %[h0]\n\t"+		"stp	%[z0], %[z1], [%[t]], #16\n\t"+		"adc	%[c], %[h1], xzr\n\t"+		: [a] "+r"(a), [t] "+r"(t), [c] "+r"(carry),+		  [x0] "=&r"(x0), [x1] "=&r"(x1), [z0] "=&r"(z0),+		  [z1] "=&r"(z1), [l0] "=&r"(l0), [l1] "=&r"(l1),+		  [h0] "=&r"(h0), [h1] "=&r"(h1)+		: [b] "r"(b)+		: "cc", "memory");+	} else if (left == 1) {+		__asm__ volatile(+		"ldr	%[x0], [%[a]], #8\n\t"+		"ldr	%[z0], [%[t]]\n\t"+		"mul	%[l0], %[x0], %[b]\n\t"+		"adds	%[z0], %[z0], %[c]\n\t"+		"umulh	%[h0], %[x0], %[b]\n\t"+		"adc	%[h0], %[h0], xzr\n\t"+		"adds	%[z0], %[z0], %[l0]\n\t"+		"str	%[z0], [%[t]], #8\n\t"+		"adc	%[c], %[h0], xzr\n\t"+		: [a] "+r"(a), [t] "+r"(t), [c] "+r"(carry),+		  [x0] "=&r"(x0), [z0] "=&r"(z0), [l0] "=&r"(l0),+		  [h0] "=&r"(h0)+		: [b] "r"(b)+		: "cc", "memory");+	}+	return carry;+}+#else+/* one limb of it, so that the loops below can say how many they take at a+ * time without saying the rest of it four times over */+#define ADDMUL_STEP(k)                                                  \+	p = (dlimb_t) a[i + (k)] * b + t[i + (k)] + carry;                  \+	t[i + (k)] = (limb_t) p;                                            \+	carry = (limb_t) (p >> LIMB_BITS);++static inline limb_t addmul_1(limb_t *t, const limb_t *a, uint32_t n, limb_t b)+{+	limb_t carry = 0;+	uint32_t i = 0;+	dlimb_t p;++	for (; i + 8 <= n; i += 8) {+		ADDMUL_STEP(0) ADDMUL_STEP(1) ADDMUL_STEP(2) ADDMUL_STEP(3)+		ADDMUL_STEP(4) ADDMUL_STEP(5) ADDMUL_STEP(6) ADDMUL_STEP(7)+	}+	for (; i + 4 <= n; i += 4) {+		ADDMUL_STEP(0) ADDMUL_STEP(1) ADDMUL_STEP(2) ADDMUL_STEP(3)+	}+	for (; i + 2 <= n; i += 2) {+		ADDMUL_STEP(0) ADDMUL_STEP(1)+	}+	for (; i < n; i++) {+		ADDMUL_STEP(0)+	}+	return carry;+}+#endif++/* r = t * R^-1 mod m, with t of 2n limbs and destroyed on the way */+static inline void mont_reduce(limb_t *r, limb_t *t, const limb_t *m, limb_t n0,+                        uint32_t n)+{+	limb_t borrow, take, carry = 0;+	uint32_t i;++	for (i = 0; i < n; i++) {+		limb_t u = t[i] * n0;+		limb_t c = addmul_1(t + i, m, n, u);+		dlimb_t s = (dlimb_t) t[n + i] + c + carry;++		t[n + i] = (limb_t) s;+		carry = (limb_t) (s >> LIMB_BITS);+	}++	/* what is left is under 2m, so at most one subtraction; which of the two+	 * to keep is a mask */+	borrow = sub_n(r, t + n, m, n);+	take = carry | (borrow ^ 1);+	select_n(r, r, t + n, take & 1, n);+}++/* t = a * b, the low n limbs, returning the limb above them: addmul_1 with+ * nothing to add to, for the row of a product that lands on empty space. */+static inline limb_t mul_1(limb_t *t, const limb_t *a, uint32_t n, limb_t b)+{+	limb_t carry = 0;+	uint32_t i;++	for (i = 0; i < n; i++) {+		dlimb_t p = (dlimb_t) a[i] * b + carry;++		t[i] = (limb_t) p;+		carry = (limb_t) (p >> LIMB_BITS);+	}+	return carry;+}++/* r = a * b * R^-1 mod m, with t of 2n limbs */+static inline void mont_mul(limb_t *r, const limb_t *a, const limb_t *b,+                     const limb_t *m, limb_t n0, uint32_t n, limb_t *t)+{+	uint32_t i;++	/* Nothing has to be cleared first.  The first row lands on empty space+	 * and writes t[0 .. n], and every row after it reads t[i .. i+n-1],+	 * whose top limb is the one the row before it wrote. */+	t[n] = mul_1(t, a, n, b[0]);+	for (i = 1; i < n; i++)+		t[n + i] = addmul_1(t + i, a, n, b[i]);+	mont_reduce(r, t, m, n0, n);+}++/* r = a * a * R^-1 mod m, with t of 2n limbs.  A square is its own mirror+ * image, so each product off the diagonal is worth two and only half of them+ * are worked out: their sum is doubled, and then the diagonal is added in. */+static inline void mont_sqr(limb_t *r, const limb_t *a, const limb_t *m, limb_t n0,+                     uint32_t n, limb_t *t)+{+	limb_t carry = 0;+	uint32_t i;++	/* The first row lands on empty space here too, on t[1 .. n], and the+	 * rows between them write t[1 .. 2n-2] before any of it is read.  The+	 * diagonal below is the only reader of the two ends. */+	t[0] = 0;+	t[2 * n - 1] = 0;+	if (n > 1)+		t[n] = mul_1(t + 1, a + 1, n - 1, a[0]);+	for (i = 1; i + 1 < n; i++)+		t[n + i] = addmul_1(t + i + i + 1, a + i + 1, n - 1 - i, a[i]);+	shl1(t, 2 * n); /* their sum is under half of what 2n limbs hold */+	for (i = 0; i < n; i++) {+		dlimb_t p = (dlimb_t) a[i] * a[i] + t[i + i] + carry;++		t[i + i] = (limb_t) p;+		p = (dlimb_t) t[i + i + 1] + (limb_t) (p >> LIMB_BITS);+		t[i + i + 1] = (limb_t) p;+		carry = (limb_t) (p >> LIMB_BITS);+	}+	mont_reduce(r, t, m, n0, n);+}++/* a = 2a mod m, for an a already under m */+static inline void dbl_mod(limb_t *a, const limb_t *m, uint32_t n, limb_t *tmp)+{+	limb_t carry = shl1(a, n);+	limb_t borrow = sub_n(tmp, a, m, n);++	select_n(a, tmp, a, (carry | (borrow ^ 1)) & 1, n);+}++/* r2 = R^2 mod m, where R is 2^(n * LIMB_BITS)+ *+ * Doubling the whole way there is 2n * LIMB_BITS steps, and at RSA sizes+ * that is a tenth of the exponentiation it is setting up for.  Only the+ * first half of it has to be done a bit at a time.+ *+ * Write a value as 2^(lgR + d) mod m.  A Montgomery squaring divides by R,+ * so it takes that to 2(lgR + d) - lgR = lgR + 2d: it doubles d.  So double+ * up to R mod m, where d is zero, take one more step to make d one, and then+ * climb to d = lgR by the binary expansion of lgR -- a squaring for each bit+ * and one more doubling where the bit is set.  For a 1024-bit modulus that+ * is ten squarings in place of a thousand and twenty-five doublings, and on+ * an Apple M4 that is 2.1 microseconds against 24.7.+ *+ * Doubling starts at the highest power of two under the modulus rather than+ * at one, since everything below that power is where doubling would go+ * anyway: for a modulus that fills its limbs, that first half is one step.+ *+ * What the trip counts depend on is the modulus' length, which is what the+ * doubling loop showed as well, and nothing else about it.+ */+static inline void mont_r2(limb_t *r2, const limb_t *m, limb_t n0, uint32_t n,+                    limb_t *t)+{+	uint32_t lgr = n * LIMB_BITS;+	uint32_t i, k = 0, msb = 0;++	for (i = n; i > 0 && k == 0; i--)+		if (m[i - 1] != 0) {+			limb_t top = m[i - 1];++			k = (i - 1) * LIMB_BITS;+			while (top != 0) {+				k++;+				top >>= 1;+			}+		}+	memset(r2, 0, n * sizeof(limb_t));+	if (k == 0)+		return; /* a modulus of nothing, which the caller rules out */+	r2[(k - 1) / LIMB_BITS] = (limb_t) 1 << ((k - 1) % LIMB_BITS);+	for (i = k - 1; i < lgr; i++)+		dbl_mod(r2, m, n, t);+	/* r2 is R mod m, so d is zero and squaring would leave it there */+	dbl_mod(r2, m, n, t);+	while ((lgr >> (msb + 1)) != 0)+		msb++;+	for (i = msb; i > 0; i--) {+		mont_sqr(r2, r2, m, n0, n, t);+		if ((lgr >> (i - 1)) & 1)+			dbl_mod(r2, m, n, t);+	}+}++/* big-endian bytes into limbs, least significant limb first; anything above+ * n limbs has to be zero, which is what the contract on the base asks for */+static inline int from_be(limb_t *r, uint32_t n, const uint8_t *src, uint32_t len)+{+	uint32_t i;++	memset(r, 0, n * sizeof(limb_t));+	for (i = 0; i < len; i++) {+		uint8_t byte = src[len - 1 - i];++		if (i / LIMB_BYTES >= n) {+			if (byte != 0)+				return 1;+			continue;+		}+		r[i / LIMB_BYTES] |= (limb_t) byte << (8 * (i % LIMB_BYTES));+	}+	return 0;+}++static inline void to_be(uint8_t *dst, uint32_t len, const limb_t *a, uint32_t n)+{+	uint32_t i;++	for (i = 0; i < len; i++) {+		uint32_t pos = len - 1 - i;+		uint32_t li = i / LIMB_BYTES;++		dst[pos] = li < n ? (uint8_t) (a[li] >> (8 * (i % LIMB_BYTES))) : 0;+	}+}++#endif
+ cbits/crypton_blowfish.c view
@@ -0,0 +1,456 @@+/*+ * Blowfish, and the key setup bcrypt wraps around it.+ *+ * The cipher is the plain one: sixteen Feistel rounds over a schedule of+ * eighteen P words and four S boxes of 256, all of which the key is stirred+ * into.  What makes bcrypt out of it is doing that stirring twice for every+ * count the cost asks for, with the salt in the mix, so that the work is+ * whatever the cost says and cannot be skipped.+ *+ * None of this is constant time, and it is not meant to be: what it is given+ * is a password, and what it leaks by timing is how long the password is,+ * which the format says out loud anyway.  What matters here is that a round+ * costs what it costs, since that is the whole point of the cost parameter.+ */+#include <stdint.h>+#include <string.h>+#include <crypton_blowfish.h>++/* The P array and the four S boxes, which are the digits of pi. */+static const uint32_t initial_p[18] = {+	0x243f6a88U, 0x85a308d3U, 0x13198a2eU, 0x03707344U, 0xa4093822U, 0x299f31d0U,+	0x082efa98U, 0xec4e6c89U, 0x452821e6U, 0x38d01377U, 0xbe5466cfU, 0x34e90c6cU,+	0xc0ac29b7U, 0xc97c50ddU, 0x3f84d5b5U, 0xb5470917U, 0x9216d5d9U, 0x8979fb1bU+};++static const uint32_t initial_s[4][256] = {+	{+		0xd1310ba6U, 0x98dfb5acU, 0x2ffd72dbU, 0xd01adfb7U, 0xb8e1afedU, 0x6a267e96U,+		0xba7c9045U, 0xf12c7f99U, 0x24a19947U, 0xb3916cf7U, 0x0801f2e2U, 0x858efc16U,+		0x636920d8U, 0x71574e69U, 0xa458fea3U, 0xf4933d7eU, 0x0d95748fU, 0x728eb658U,+		0x718bcd58U, 0x82154aeeU, 0x7b54a41dU, 0xc25a59b5U, 0x9c30d539U, 0x2af26013U,+		0xc5d1b023U, 0x286085f0U, 0xca417918U, 0xb8db38efU, 0x8e79dcb0U, 0x603a180eU,+		0x6c9e0e8bU, 0xb01e8a3eU, 0xd71577c1U, 0xbd314b27U, 0x78af2fdaU, 0x55605c60U,+		0xe65525f3U, 0xaa55ab94U, 0x57489862U, 0x63e81440U, 0x55ca396aU, 0x2aab10b6U,+		0xb4cc5c34U, 0x1141e8ceU, 0xa15486afU, 0x7c72e993U, 0xb3ee1411U, 0x636fbc2aU,+		0x2ba9c55dU, 0x741831f6U, 0xce5c3e16U, 0x9b87931eU, 0xafd6ba33U, 0x6c24cf5cU,+		0x7a325381U, 0x28958677U, 0x3b8f4898U, 0x6b4bb9afU, 0xc4bfe81bU, 0x66282193U,+		0x61d809ccU, 0xfb21a991U, 0x487cac60U, 0x5dec8032U, 0xef845d5dU, 0xe98575b1U,+		0xdc262302U, 0xeb651b88U, 0x23893e81U, 0xd396acc5U, 0x0f6d6ff3U, 0x83f44239U,+		0x2e0b4482U, 0xa4842004U, 0x69c8f04aU, 0x9e1f9b5eU, 0x21c66842U, 0xf6e96c9aU,+		0x670c9c61U, 0xabd388f0U, 0x6a51a0d2U, 0xd8542f68U, 0x960fa728U, 0xab5133a3U,+		0x6eef0b6cU, 0x137a3be4U, 0xba3bf050U, 0x7efb2a98U, 0xa1f1651dU, 0x39af0176U,+		0x66ca593eU, 0x82430e88U, 0x8cee8619U, 0x456f9fb4U, 0x7d84a5c3U, 0x3b8b5ebeU,+		0xe06f75d8U, 0x85c12073U, 0x401a449fU, 0x56c16aa6U, 0x4ed3aa62U, 0x363f7706U,+		0x1bfedf72U, 0x429b023dU, 0x37d0d724U, 0xd00a1248U, 0xdb0fead3U, 0x49f1c09bU,+		0x075372c9U, 0x80991b7bU, 0x25d479d8U, 0xf6e8def7U, 0xe3fe501aU, 0xb6794c3bU,+		0x976ce0bdU, 0x04c006baU, 0xc1a94fb6U, 0x409f60c4U, 0x5e5c9ec2U, 0x196a2463U,+		0x68fb6fafU, 0x3e6c53b5U, 0x1339b2ebU, 0x3b52ec6fU, 0x6dfc511fU, 0x9b30952cU,+		0xcc814544U, 0xaf5ebd09U, 0xbee3d004U, 0xde334afdU, 0x660f2807U, 0x192e4bb3U,+		0xc0cba857U, 0x45c8740fU, 0xd20b5f39U, 0xb9d3fbdbU, 0x5579c0bdU, 0x1a60320aU,+		0xd6a100c6U, 0x402c7279U, 0x679f25feU, 0xfb1fa3ccU, 0x8ea5e9f8U, 0xdb3222f8U,+		0x3c7516dfU, 0xfd616b15U, 0x2f501ec8U, 0xad0552abU, 0x323db5faU, 0xfd238760U,+		0x53317b48U, 0x3e00df82U, 0x9e5c57bbU, 0xca6f8ca0U, 0x1a87562eU, 0xdf1769dbU,+		0xd542a8f6U, 0x287effc3U, 0xac6732c6U, 0x8c4f5573U, 0x695b27b0U, 0xbbca58c8U,+		0xe1ffa35dU, 0xb8f011a0U, 0x10fa3d98U, 0xfd2183b8U, 0x4afcb56cU, 0x2dd1d35bU,+		0x9a53e479U, 0xb6f84565U, 0xd28e49bcU, 0x4bfb9790U, 0xe1ddf2daU, 0xa4cb7e33U,+		0x62fb1341U, 0xcee4c6e8U, 0xef20cadaU, 0x36774c01U, 0xd07e9efeU, 0x2bf11fb4U,+		0x95dbda4dU, 0xae909198U, 0xeaad8e71U, 0x6b93d5a0U, 0xd08ed1d0U, 0xafc725e0U,+		0x8e3c5b2fU, 0x8e7594b7U, 0x8ff6e2fbU, 0xf2122b64U, 0x8888b812U, 0x900df01cU,+		0x4fad5ea0U, 0x688fc31cU, 0xd1cff191U, 0xb3a8c1adU, 0x2f2f2218U, 0xbe0e1777U,+		0xea752dfeU, 0x8b021fa1U, 0xe5a0cc0fU, 0xb56f74e8U, 0x18acf3d6U, 0xce89e299U,+		0xb4a84fe0U, 0xfd13e0b7U, 0x7cc43b81U, 0xd2ada8d9U, 0x165fa266U, 0x80957705U,+		0x93cc7314U, 0x211a1477U, 0xe6ad2065U, 0x77b5fa86U, 0xc75442f5U, 0xfb9d35cfU,+		0xebcdaf0cU, 0x7b3e89a0U, 0xd6411bd3U, 0xae1e7e49U, 0x00250e2dU, 0x2071b35eU,+		0x226800bbU, 0x57b8e0afU, 0x2464369bU, 0xf009b91eU, 0x5563911dU, 0x59dfa6aaU,+		0x78c14389U, 0xd95a537fU, 0x207d5ba2U, 0x02e5b9c5U, 0x83260376U, 0x6295cfa9U,+		0x11c81968U, 0x4e734a41U, 0xb3472dcaU, 0x7b14a94aU, 0x1b510052U, 0x9a532915U,+		0xd60f573fU, 0xbc9bc6e4U, 0x2b60a476U, 0x81e67400U, 0x08ba6fb5U, 0x571be91fU,+		0xf296ec6bU, 0x2a0dd915U, 0xb6636521U, 0xe7b9f9b6U, 0xff34052eU, 0xc5855664U,+		0x53b02d5dU, 0xa99f8fa1U, 0x08ba4799U, 0x6e85076aU+	},+	{+		0x4b7a70e9U, 0xb5b32944U, 0xdb75092eU, 0xc4192623U, 0xad6ea6b0U, 0x49a7df7dU,+		0x9cee60b8U, 0x8fedb266U, 0xecaa8c71U, 0x699a17ffU, 0x5664526cU, 0xc2b19ee1U,+		0x193602a5U, 0x75094c29U, 0xa0591340U, 0xe4183a3eU, 0x3f54989aU, 0x5b429d65U,+		0x6b8fe4d6U, 0x99f73fd6U, 0xa1d29c07U, 0xefe830f5U, 0x4d2d38e6U, 0xf0255dc1U,+		0x4cdd2086U, 0x8470eb26U, 0x6382e9c6U, 0x021ecc5eU, 0x09686b3fU, 0x3ebaefc9U,+		0x3c971814U, 0x6b6a70a1U, 0x687f3584U, 0x52a0e286U, 0xb79c5305U, 0xaa500737U,+		0x3e07841cU, 0x7fdeae5cU, 0x8e7d44ecU, 0x5716f2b8U, 0xb03ada37U, 0xf0500c0dU,+		0xf01c1f04U, 0x0200b3ffU, 0xae0cf51aU, 0x3cb574b2U, 0x25837a58U, 0xdc0921bdU,+		0xd19113f9U, 0x7ca92ff6U, 0x94324773U, 0x22f54701U, 0x3ae5e581U, 0x37c2dadcU,+		0xc8b57634U, 0x9af3dda7U, 0xa9446146U, 0x0fd0030eU, 0xecc8c73eU, 0xa4751e41U,+		0xe238cd99U, 0x3bea0e2fU, 0x3280bba1U, 0x183eb331U, 0x4e548b38U, 0x4f6db908U,+		0x6f420d03U, 0xf60a04bfU, 0x2cb81290U, 0x24977c79U, 0x5679b072U, 0xbcaf89afU,+		0xde9a771fU, 0xd9930810U, 0xb38bae12U, 0xdccf3f2eU, 0x5512721fU, 0x2e6b7124U,+		0x501adde6U, 0x9f84cd87U, 0x7a584718U, 0x7408da17U, 0xbc9f9abcU, 0xe94b7d8cU,+		0xec7aec3aU, 0xdb851dfaU, 0x63094366U, 0xc464c3d2U, 0xef1c1847U, 0x3215d908U,+		0xdd433b37U, 0x24c2ba16U, 0x12a14d43U, 0x2a65c451U, 0x50940002U, 0x133ae4ddU,+		0x71dff89eU, 0x10314e55U, 0x81ac77d6U, 0x5f11199bU, 0x043556f1U, 0xd7a3c76bU,+		0x3c11183bU, 0x5924a509U, 0xf28fe6edU, 0x97f1fbfaU, 0x9ebabf2cU, 0x1e153c6eU,+		0x86e34570U, 0xeae96fb1U, 0x860e5e0aU, 0x5a3e2ab3U, 0x771fe71cU, 0x4e3d06faU,+		0x2965dcb9U, 0x99e71d0fU, 0x803e89d6U, 0x5266c825U, 0x2e4cc978U, 0x9c10b36aU,+		0xc6150ebaU, 0x94e2ea78U, 0xa5fc3c53U, 0x1e0a2df4U, 0xf2f74ea7U, 0x361d2b3dU,+		0x1939260fU, 0x19c27960U, 0x5223a708U, 0xf71312b6U, 0xebadfe6eU, 0xeac31f66U,+		0xe3bc4595U, 0xa67bc883U, 0xb17f37d1U, 0x018cff28U, 0xc332ddefU, 0xbe6c5aa5U,+		0x65582185U, 0x68ab9802U, 0xeecea50fU, 0xdb2f953bU, 0x2aef7dadU, 0x5b6e2f84U,+		0x1521b628U, 0x29076170U, 0xecdd4775U, 0x619f1510U, 0x13cca830U, 0xeb61bd96U,+		0x0334fe1eU, 0xaa0363cfU, 0xb5735c90U, 0x4c70a239U, 0xd59e9e0bU, 0xcbaade14U,+		0xeecc86bcU, 0x60622ca7U, 0x9cab5cabU, 0xb2f3846eU, 0x648b1eafU, 0x19bdf0caU,+		0xa02369b9U, 0x655abb50U, 0x40685a32U, 0x3c2ab4b3U, 0x319ee9d5U, 0xc021b8f7U,+		0x9b540b19U, 0x875fa099U, 0x95f7997eU, 0x623d7da8U, 0xf837889aU, 0x97e32d77U,+		0x11ed935fU, 0x16681281U, 0x0e358829U, 0xc7e61fd6U, 0x96dedfa1U, 0x7858ba99U,+		0x57f584a5U, 0x1b227263U, 0x9b83c3ffU, 0x1ac24696U, 0xcdb30aebU, 0x532e3054U,+		0x8fd948e4U, 0x6dbc3128U, 0x58ebf2efU, 0x34c6ffeaU, 0xfe28ed61U, 0xee7c3c73U,+		0x5d4a14d9U, 0xe864b7e3U, 0x42105d14U, 0x203e13e0U, 0x45eee2b6U, 0xa3aaabeaU,+		0xdb6c4f15U, 0xfacb4fd0U, 0xc742f442U, 0xef6abbb5U, 0x654f3b1dU, 0x41cd2105U,+		0xd81e799eU, 0x86854dc7U, 0xe44b476aU, 0x3d816250U, 0xcf62a1f2U, 0x5b8d2646U,+		0xfc8883a0U, 0xc1c7b6a3U, 0x7f1524c3U, 0x69cb7492U, 0x47848a0bU, 0x5692b285U,+		0x095bbf00U, 0xad19489dU, 0x1462b174U, 0x23820e00U, 0x58428d2aU, 0x0c55f5eaU,+		0x1dadf43eU, 0x233f7061U, 0x3372f092U, 0x8d937e41U, 0xd65fecf1U, 0x6c223bdbU,+		0x7cde3759U, 0xcbee7460U, 0x4085f2a7U, 0xce77326eU, 0xa6078084U, 0x19f8509eU,+		0xe8efd855U, 0x61d99735U, 0xa969a7aaU, 0xc50c06c2U, 0x5a04abfcU, 0x800bcadcU,+		0x9e447a2eU, 0xc3453484U, 0xfdd56705U, 0x0e1e9ec9U, 0xdb73dbd3U, 0x105588cdU,+		0x675fda79U, 0xe3674340U, 0xc5c43465U, 0x713e38d8U, 0x3d28f89eU, 0xf16dff20U,+		0x153e21e7U, 0x8fb03d4aU, 0xe6e39f2bU, 0xdb83adf7U+	},+	{+		0xe93d5a68U, 0x948140f7U, 0xf64c261cU, 0x94692934U, 0x411520f7U, 0x7602d4f7U,+		0xbcf46b2eU, 0xd4a20068U, 0xd4082471U, 0x3320f46aU, 0x43b7d4b7U, 0x500061afU,+		0x1e39f62eU, 0x97244546U, 0x14214f74U, 0xbf8b8840U, 0x4d95fc1dU, 0x96b591afU,+		0x70f4ddd3U, 0x66a02f45U, 0xbfbc09ecU, 0x03bd9785U, 0x7fac6dd0U, 0x31cb8504U,+		0x96eb27b3U, 0x55fd3941U, 0xda2547e6U, 0xabca0a9aU, 0x28507825U, 0x530429f4U,+		0x0a2c86daU, 0xe9b66dfbU, 0x68dc1462U, 0xd7486900U, 0x680ec0a4U, 0x27a18deeU,+		0x4f3ffea2U, 0xe887ad8cU, 0xb58ce006U, 0x7af4d6b6U, 0xaace1e7cU, 0xd3375fecU,+		0xce78a399U, 0x406b2a42U, 0x20fe9e35U, 0xd9f385b9U, 0xee39d7abU, 0x3b124e8bU,+		0x1dc9faf7U, 0x4b6d1856U, 0x26a36631U, 0xeae397b2U, 0x3a6efa74U, 0xdd5b4332U,+		0x6841e7f7U, 0xca7820fbU, 0xfb0af54eU, 0xd8feb397U, 0x454056acU, 0xba489527U,+		0x55533a3aU, 0x20838d87U, 0xfe6ba9b7U, 0xd096954bU, 0x55a867bcU, 0xa1159a58U,+		0xcca92963U, 0x99e1db33U, 0xa62a4a56U, 0x3f3125f9U, 0x5ef47e1cU, 0x9029317cU,+		0xfdf8e802U, 0x04272f70U, 0x80bb155cU, 0x05282ce3U, 0x95c11548U, 0xe4c66d22U,+		0x48c1133fU, 0xc70f86dcU, 0x07f9c9eeU, 0x41041f0fU, 0x404779a4U, 0x5d886e17U,+		0x325f51ebU, 0xd59bc0d1U, 0xf2bcc18fU, 0x41113564U, 0x257b7834U, 0x602a9c60U,+		0xdff8e8a3U, 0x1f636c1bU, 0x0e12b4c2U, 0x02e1329eU, 0xaf664fd1U, 0xcad18115U,+		0x6b2395e0U, 0x333e92e1U, 0x3b240b62U, 0xeebeb922U, 0x85b2a20eU, 0xe6ba0d99U,+		0xde720c8cU, 0x2da2f728U, 0xd0127845U, 0x95b794fdU, 0x647d0862U, 0xe7ccf5f0U,+		0x5449a36fU, 0x877d48faU, 0xc39dfd27U, 0xf33e8d1eU, 0x0a476341U, 0x992eff74U,+		0x3a6f6eabU, 0xf4f8fd37U, 0xa812dc60U, 0xa1ebddf8U, 0x991be14cU, 0xdb6e6b0dU,+		0xc67b5510U, 0x6d672c37U, 0x2765d43bU, 0xdcd0e804U, 0xf1290dc7U, 0xcc00ffa3U,+		0xb5390f92U, 0x690fed0bU, 0x667b9ffbU, 0xcedb7d9cU, 0xa091cf0bU, 0xd9155ea3U,+		0xbb132f88U, 0x515bad24U, 0x7b9479bfU, 0x763bd6ebU, 0x37392eb3U, 0xcc115979U,+		0x8026e297U, 0xf42e312dU, 0x6842ada7U, 0xc66a2b3bU, 0x12754cccU, 0x782ef11cU,+		0x6a124237U, 0xb79251e7U, 0x06a1bbe6U, 0x4bfb6350U, 0x1a6b1018U, 0x11caedfaU,+		0x3d25bdd8U, 0xe2e1c3c9U, 0x44421659U, 0x0a121386U, 0xd90cec6eU, 0xd5abea2aU,+		0x64af674eU, 0xda86a85fU, 0xbebfe988U, 0x64e4c3feU, 0x9dbc8057U, 0xf0f7c086U,+		0x60787bf8U, 0x6003604dU, 0xd1fd8346U, 0xf6381fb0U, 0x7745ae04U, 0xd736fcccU,+		0x83426b33U, 0xf01eab71U, 0xb0804187U, 0x3c005e5fU, 0x77a057beU, 0xbde8ae24U,+		0x55464299U, 0xbf582e61U, 0x4e58f48fU, 0xf2ddfda2U, 0xf474ef38U, 0x8789bdc2U,+		0x5366f9c3U, 0xc8b38e74U, 0xb475f255U, 0x46fcd9b9U, 0x7aeb2661U, 0x8b1ddf84U,+		0x846a0e79U, 0x915f95e2U, 0x466e598eU, 0x20b45770U, 0x8cd55591U, 0xc902de4cU,+		0xb90bace1U, 0xbb8205d0U, 0x11a86248U, 0x7574a99eU, 0xb77f19b6U, 0xe0a9dc09U,+		0x662d09a1U, 0xc4324633U, 0xe85a1f02U, 0x09f0be8cU, 0x4a99a025U, 0x1d6efe10U,+		0x1ab93d1dU, 0x0ba5a4dfU, 0xa186f20fU, 0x2868f169U, 0xdcb7da83U, 0x573906feU,+		0xa1e2ce9bU, 0x4fcd7f52U, 0x50115e01U, 0xa70683faU, 0xa002b5c4U, 0x0de6d027U,+		0x9af88c27U, 0x773f8641U, 0xc3604c06U, 0x61a806b5U, 0xf0177a28U, 0xc0f586e0U,+		0x006058aaU, 0x30dc7d62U, 0x11e69ed7U, 0x2338ea63U, 0x53c2dd94U, 0xc2c21634U,+		0xbbcbee56U, 0x90bcb6deU, 0xebfc7da1U, 0xce591d76U, 0x6f05e409U, 0x4b7c0188U,+		0x39720a3dU, 0x7c927c24U, 0x86e3725fU, 0x724d9db9U, 0x1ac15bb4U, 0xd39eb8fcU,+		0xed545578U, 0x08fca5b5U, 0xd83d7cd3U, 0x4dad0fc4U, 0x1e50ef5eU, 0xb161e6f8U,+		0xa28514d9U, 0x6c51133cU, 0x6fd5c7e7U, 0x56e14ec4U, 0x362abfceU, 0xddc6c837U,+		0xd79a3234U, 0x92638212U, 0x670efa8eU, 0x406000e0U+	},+	{+		0x3a39ce37U, 0xd3faf5cfU, 0xabc27737U, 0x5ac52d1bU, 0x5cb0679eU, 0x4fa33742U,+		0xd3822740U, 0x99bc9bbeU, 0xd5118e9dU, 0xbf0f7315U, 0xd62d1c7eU, 0xc700c47bU,+		0xb78c1b6bU, 0x21a19045U, 0xb26eb1beU, 0x6a366eb4U, 0x5748ab2fU, 0xbc946e79U,+		0xc6a376d2U, 0x6549c2c8U, 0x530ff8eeU, 0x468dde7dU, 0xd5730a1dU, 0x4cd04dc6U,+		0x2939bbdbU, 0xa9ba4650U, 0xac9526e8U, 0xbe5ee304U, 0xa1fad5f0U, 0x6a2d519aU,+		0x63ef8ce2U, 0x9a86ee22U, 0xc089c2b8U, 0x43242ef6U, 0xa51e03aaU, 0x9cf2d0a4U,+		0x83c061baU, 0x9be96a4dU, 0x8fe51550U, 0xba645bd6U, 0x2826a2f9U, 0xa73a3ae1U,+		0x4ba99586U, 0xef5562e9U, 0xc72fefd3U, 0xf752f7daU, 0x3f046f69U, 0x77fa0a59U,+		0x80e4a915U, 0x87b08601U, 0x9b09e6adU, 0x3b3ee593U, 0xe990fd5aU, 0x9e34d797U,+		0x2cf0b7d9U, 0x022b8b51U, 0x96d5ac3aU, 0x017da67dU, 0xd1cf3ed6U, 0x7c7d2d28U,+		0x1f9f25cfU, 0xadf2b89bU, 0x5ad6b472U, 0x5a88f54cU, 0xe029ac71U, 0xe019a5e6U,+		0x47b0acfdU, 0xed93fa9bU, 0xe8d3c48dU, 0x283b57ccU, 0xf8d56629U, 0x79132e28U,+		0x785f0191U, 0xed756055U, 0xf7960e44U, 0xe3d35e8cU, 0x15056dd4U, 0x88f46dbaU,+		0x03a16125U, 0x0564f0bdU, 0xc3eb9e15U, 0x3c9057a2U, 0x97271aecU, 0xa93a072aU,+		0x1b3f6d9bU, 0x1e6321f5U, 0xf59c66fbU, 0x26dcf319U, 0x7533d928U, 0xb155fdf5U,+		0x03563482U, 0x8aba3cbbU, 0x28517711U, 0xc20ad9f8U, 0xabcc5167U, 0xccad925fU,+		0x4de81751U, 0x3830dc8eU, 0x379d5862U, 0x9320f991U, 0xea7a90c2U, 0xfb3e7bceU,+		0x5121ce64U, 0x774fbe32U, 0xa8b6e37eU, 0xc3293d46U, 0x48de5369U, 0x6413e680U,+		0xa2ae0810U, 0xdd6db224U, 0x69852dfdU, 0x09072166U, 0xb39a460aU, 0x6445c0ddU,+		0x586cdecfU, 0x1c20c8aeU, 0x5bbef7ddU, 0x1b588d40U, 0xccd2017fU, 0x6bb4e3bbU,+		0xdda26a7eU, 0x3a59ff45U, 0x3e350a44U, 0xbcb4cdd5U, 0x72eacea8U, 0xfa6484bbU,+		0x8d6612aeU, 0xbf3c6f47U, 0xd29be463U, 0x542f5d9eU, 0xaec2771bU, 0xf64e6370U,+		0x740e0d8dU, 0xe75b1357U, 0xf8721671U, 0xaf537d5dU, 0x4040cb08U, 0x4eb4e2ccU,+		0x34d2466aU, 0x0115af84U, 0xe1b00428U, 0x95983a1dU, 0x06b89fb4U, 0xce6ea048U,+		0x6f3f3b82U, 0x3520ab82U, 0x011a1d4bU, 0x277227f8U, 0x611560b1U, 0xe7933fdcU,+		0xbb3a792bU, 0x344525bdU, 0xa08839e1U, 0x51ce794bU, 0x2f32c9b7U, 0xa01fbac9U,+		0xe01cc87eU, 0xbcc7d1f6U, 0xcf0111c3U, 0xa1e8aac7U, 0x1a908749U, 0xd44fbd9aU,+		0xd0dadecbU, 0xd50ada38U, 0x0339c32aU, 0xc6913667U, 0x8df9317cU, 0xe0b12b4fU,+		0xf79e59b7U, 0x43f5bb3aU, 0xf2d519ffU, 0x27d9459cU, 0xbf97222cU, 0x15e6fc2aU,+		0x0f91fc71U, 0x9b941525U, 0xfae59361U, 0xceb69cebU, 0xc2a86459U, 0x12baa8d1U,+		0xb6c1075eU, 0xe3056a0cU, 0x10d25065U, 0xcb03a442U, 0xe0ec6e0eU, 0x1698db3bU,+		0x4c98a0beU, 0x3278e964U, 0x9f1f9532U, 0xe0d392dfU, 0xd3a0342bU, 0x8971f21eU,+		0x1b0a7441U, 0x4ba3348cU, 0xc5be7120U, 0xc37632d8U, 0xdf359f8dU, 0x9b992f2eU,+		0xe60b6f47U, 0x0fe3f11dU, 0xe54cda54U, 0x1edad891U, 0xce6279cfU, 0xcd3e7e6fU,+		0x1618b166U, 0xfd2c1d05U, 0x848fd2c5U, 0xf6fb2299U, 0xf523f357U, 0xa6327623U,+		0x93a83531U, 0x56cccd02U, 0xacf08162U, 0x5a75ebb5U, 0x6e163697U, 0x88d273ccU,+		0xde966292U, 0x81b949d0U, 0x4c50901bU, 0x71c65614U, 0xe6c6c7bdU, 0x327a140aU,+		0x45e1d006U, 0xc3f27b9aU, 0xc9aa53fdU, 0x62a80f00U, 0xbb25bfe2U, 0x35bdd2f6U,+		0x71126905U, 0xb2040222U, 0xb6cbcf7cU, 0xcd769c2bU, 0x53113ec0U, 0x1640e3d3U,+		0x38abbd60U, 0x2547adf0U, 0xba38209cU, 0xf746ce76U, 0x77afa1c5U, 0x20756060U,+		0x85cbfe4eU, 0x8ae88dd8U, 0x7aaaf9b0U, 0x4cf9aa7eU, 0x1948c25cU, 0x02fb8a8cU,+		0x01c36ae4U, 0xd6ebe1f9U, 0x90d4f869U, 0xa65cdea0U, 0x3f09252dU, 0xc208e69fU,+		0xb74e6132U, 0xce77e25bU, 0x578fdfe3U, 0x3ac372e6U+	}+};++#define F(ctx, x)                                                             \+	((((ctx)->s[0][(x) >> 24] + (ctx)->s[1][((x) >> 16) & 0xff])              \+	  ^ (ctx)->s[2][((x) >> 8) & 0xff])                                       \+	 + (ctx)->s[3][(x) & 0xff])++static void block_encrypt(const crypton_blowfish_ctx *ctx, uint32_t *xl,+                          uint32_t *xr)+{+	uint32_t l = *xl, r = *xr;+	int i;++	for (i = 0; i < 16; i += 2) {+		l ^= ctx->p[i];+		r ^= F(ctx, l);+		r ^= ctx->p[i + 1];+		l ^= F(ctx, r);+	}+	l ^= ctx->p[16];+	r ^= ctx->p[17];+	*xl = r;+	*xr = l;+}++static void block_decrypt(const crypton_blowfish_ctx *ctx, uint32_t *xl,+                          uint32_t *xr)+{+	uint32_t l = *xl, r = *xr;+	int i;++	for (i = 16; i > 0; i -= 2) {+		l ^= ctx->p[i + 1];+		r ^= F(ctx, l);+		r ^= ctx->p[i];+		l ^= F(ctx, r);+	}+	l ^= ctx->p[1];+	r ^= ctx->p[0];+	*xl = r;+	*xr = l;+}++/* the next four bytes of the key, taken round and round */+static uint32_t key_word(const uint8_t *key, uint32_t keylen, uint32_t *pos)+{+	uint32_t w = 0, i;++	for (i = 0; i < 4; i++) {+		w = (w << 8) | key[*pos];+		*pos = (*pos + 1) % keylen;+	}+	return w;+}++/* the key into the P array, and then the whole schedule rewritten by+ * encrypting its way through itself */+static void expand_key(crypton_blowfish_ctx *ctx, const uint8_t *key,+                       uint32_t keylen)+{+	uint32_t pos = 0, l = 0, r = 0;+	int i, j;++	if (keylen > 0)+		for (i = 0; i < 18; i++)+			ctx->p[i] ^= key_word(key, keylen, &pos);+	for (i = 0; i < 18; i += 2) {+		block_encrypt(ctx, &l, &r);+		ctx->p[i] = l;+		ctx->p[i + 1] = r;+	}+	for (i = 0; i < 4; i++)+		for (j = 0; j < 256; j += 2) {+			block_encrypt(ctx, &l, &r);+			ctx->s[i][j] = l;+			ctx->s[i][j + 1] = r;+		}+}++/* the same, with the salt exclusive-ored into what is encrypted at every+ * step, which is what makes the schedule depend on it.  The salt is taken+ * round and round as the key is, so a salt of any length will do: bcrypt+ * hands it sixteen bytes and bcrypt_pbkdf hands it sixty-four. */+static void expand_key_with_salt(crypton_blowfish_ctx *ctx, const uint8_t *key,+                                 uint32_t keylen, const uint8_t *salt,+                                 uint32_t saltlen)+{+	uint32_t kpos = 0, spos = 0, l = 0, r = 0;+	int i, j;++	if (keylen > 0)+		for (i = 0; i < 18; i++)+			ctx->p[i] ^= key_word(key, keylen, &kpos);+	for (i = 0; i < 18; i += 2) {+		l ^= key_word(salt, saltlen, &spos);+		r ^= key_word(salt, saltlen, &spos);+		block_encrypt(ctx, &l, &r);+		ctx->p[i] = l;+		ctx->p[i + 1] = r;+	}+	for (i = 0; i < 4; i++)+		for (j = 0; j < 256; j += 2) {+			l ^= key_word(salt, saltlen, &spos);+			r ^= key_word(salt, saltlen, &spos);+			block_encrypt(ctx, &l, &r);+			ctx->s[i][j] = l;+			ctx->s[i][j + 1] = r;+		}+}++void crypton_blowfish_init(crypton_blowfish_ctx *ctx, const uint8_t *key,+                           uint32_t keylen)+{+	memcpy(ctx->p, initial_p, sizeof(ctx->p));+	memcpy(ctx->s, initial_s, sizeof(ctx->s));+	expand_key(ctx, key, keylen);+}++void crypton_blowfish_encrypt(const crypton_blowfish_ctx *ctx, uint8_t *out,+                              const uint8_t *in, uint32_t len)+{+	uint32_t i;++	for (i = 0; i + 8 <= len; i += 8) {+		uint32_t l = ((uint32_t) in[i] << 24) | ((uint32_t) in[i + 1] << 16)+		             | ((uint32_t) in[i + 2] << 8) | (uint32_t) in[i + 3];+		uint32_t r = ((uint32_t) in[i + 4] << 24)+		             | ((uint32_t) in[i + 5] << 16)+		             | ((uint32_t) in[i + 6] << 8) | (uint32_t) in[i + 7];++		block_encrypt(ctx, &l, &r);+		out[i] = (uint8_t) (l >> 24);+		out[i + 1] = (uint8_t) (l >> 16);+		out[i + 2] = (uint8_t) (l >> 8);+		out[i + 3] = (uint8_t) l;+		out[i + 4] = (uint8_t) (r >> 24);+		out[i + 5] = (uint8_t) (r >> 16);+		out[i + 6] = (uint8_t) (r >> 8);+		out[i + 7] = (uint8_t) r;+	}+}++void crypton_blowfish_decrypt(const crypton_blowfish_ctx *ctx, uint8_t *out,+                              const uint8_t *in, uint32_t len)+{+	uint32_t i;++	for (i = 0; i + 8 <= len; i += 8) {+		uint32_t l = ((uint32_t) in[i] << 24) | ((uint32_t) in[i + 1] << 16)+		             | ((uint32_t) in[i + 2] << 8) | (uint32_t) in[i + 3];+		uint32_t r = ((uint32_t) in[i + 4] << 24)+		             | ((uint32_t) in[i + 5] << 16)+		             | ((uint32_t) in[i + 6] << 8) | (uint32_t) in[i + 7];++		block_decrypt(ctx, &l, &r);+		out[i] = (uint8_t) (l >> 24);+		out[i + 1] = (uint8_t) (l >> 16);+		out[i + 2] = (uint8_t) (l >> 8);+		out[i + 3] = (uint8_t) l;+		out[i + 4] = (uint8_t) (r >> 24);+		out[i + 5] = (uint8_t) (r >> 16);+		out[i + 6] = (uint8_t) (r >> 8);+		out[i + 7] = (uint8_t) r;+	}+}++int crypton_bcrypt(uint8_t out[24], uint32_t cost, const uint8_t salt[16],+                   const uint8_t *key, uint32_t keylen)+{+	/* "OrpheanBeholderScryDoubt", which is what bcrypt encrypts */+	static const uint8_t magic[24] = {+		0x4f, 0x72, 0x70, 0x68, 0x65, 0x61, 0x6e, 0x42,+		0x65, 0x68, 0x6f, 0x6c, 0x64, 0x65, 0x72, 0x53,+		0x63, 0x72, 0x79, 0x44, 0x6f, 0x75, 0x62, 0x74+	};+	crypton_blowfish_ctx ctx;+	uint32_t rounds, i;++	if (cost < 4 || cost > 31 || keylen == 0 || keylen > 73)+		return -1;++	memcpy(ctx.p, initial_p, sizeof(ctx.p));+	memcpy(ctx.s, initial_s, sizeof(ctx.s));+	expand_key_with_salt(&ctx, key, keylen, salt, 16);+	rounds = (uint32_t) 1 << cost;+	for (i = 0; i < rounds; i++) {+		expand_key(&ctx, key, keylen);+		expand_key(&ctx, salt, 16);+	}++	memcpy(out, magic, sizeof(magic));+	for (i = 0; i < 64; i++)+		crypton_blowfish_encrypt(&ctx, out, out, 24);++	memset(&ctx, 0, sizeof(ctx));+	return 0;+}++int crypton_bcrypt_pbkdf_hash(uint8_t out[32], const uint8_t *pass,+                              uint32_t passlen, const uint8_t *salt,+                              uint32_t saltlen)+{+	/* "OxychromaticBlowfishSwatDynamite", which is what this one encrypts */+	static const uint8_t magic[32] = {+		0x4f, 0x78, 0x79, 0x63, 0x68, 0x72, 0x6f, 0x6d,+		0x61, 0x74, 0x69, 0x63, 0x42, 0x6c, 0x6f, 0x77,+		0x66, 0x69, 0x73, 0x68, 0x53, 0x77, 0x61, 0x74,+		0x44, 0x79, 0x6e, 0x61, 0x6d, 0x69, 0x74, 0x65+	};+	crypton_blowfish_ctx ctx;+	uint32_t i, j;++	if (passlen == 0 || saltlen == 0)+		return -1;++	memcpy(ctx.p, initial_p, sizeof(ctx.p));+	memcpy(ctx.s, initial_s, sizeof(ctx.s));+	expand_key_with_salt(&ctx, pass, passlen, salt, saltlen);+	for (i = 0; i < 64; i++) {+		expand_key(&ctx, salt, saltlen);+		expand_key(&ctx, pass, passlen);+	}++	/* each block encrypted sixty-four times, and stored with each half the+	 * way round that the original implementation stores it */+	for (i = 0; i < 4; i++) {+		uint32_t l = ((uint32_t) magic[8 * i] << 24)+		             | ((uint32_t) magic[8 * i + 1] << 16)+		             | ((uint32_t) magic[8 * i + 2] << 8)+		             | (uint32_t) magic[8 * i + 3];+		uint32_t r = ((uint32_t) magic[8 * i + 4] << 24)+		             | ((uint32_t) magic[8 * i + 5] << 16)+		             | ((uint32_t) magic[8 * i + 6] << 8)+		             | (uint32_t) magic[8 * i + 7];++		for (j = 0; j < 64; j++)+			block_encrypt(&ctx, &l, &r);+		out[8 * i] = (uint8_t) l;+		out[8 * i + 1] = (uint8_t) (l >> 8);+		out[8 * i + 2] = (uint8_t) (l >> 16);+		out[8 * i + 3] = (uint8_t) (l >> 24);+		out[8 * i + 4] = (uint8_t) r;+		out[8 * i + 5] = (uint8_t) (r >> 8);+		out[8 * i + 6] = (uint8_t) (r >> 16);+		out[8 * i + 7] = (uint8_t) (r >> 24);+	}++	memset(&ctx, 0, sizeof(ctx));+	return 0;+}
+ cbits/crypton_blowfish.h view
@@ -0,0 +1,44 @@+#ifndef CRYPTON_BLOWFISH_H+#define CRYPTON_BLOWFISH_H++#include <stdint.h>++/* The key schedule: the P array and the four S boxes, which is all the state+ * Blowfish has.  The caller keeps it; nothing here allocates. */+typedef struct {+	uint32_t p[18];+	uint32_t s[4][256];+} crypton_blowfish_ctx;++/* Set a schedule up from a key of keylen bytes, which has to be 1 to 56. */+void crypton_blowfish_init(crypton_blowfish_ctx *ctx, const uint8_t *key,+                           uint32_t keylen);++/* Encrypt or decrypt whole blocks: len has to be a multiple of eight, and out+ * may be in. */+void crypton_blowfish_encrypt(const crypton_blowfish_ctx *ctx, uint8_t *out,+                              const uint8_t *in, uint32_t len);+void crypton_blowfish_decrypt(const crypton_blowfish_ctx *ctx, uint8_t *out,+                              const uint8_t *in, uint32_t len);++/* The whole of what bcrypt does with Blowfish: the key setup that costs what+ * the cost says, and then the sixty-four encryptions.  Writes 24 bytes, of+ * which bcrypt keeps 23.  The salt is 16 bytes and the key is the password+ * with its terminating zero, 1 to 72 bytes of it.+ *+ * Returns 0, or -1 for a cost or a length it will not take.+ */+int crypton_bcrypt(uint8_t out[24], uint32_t cost, const uint8_t salt[16],+                   const uint8_t *key, uint32_t keylen);++/* What bcrypt_pbkdf does with Blowfish: the same key setup, sixty-four times+ * over, and then the four blocks of its own magic.  Writes 32 bytes.  The two+ * hashes it is given are 64 bytes each in the only caller there is.+ *+ * Returns 0, or -1 for a length it will not take.+ */+int crypton_bcrypt_pbkdf_hash(uint8_t out[32], const uint8_t *pass,+                              uint32_t passlen, const uint8_t *salt,+                              uint32_t saltlen);++#endif
+ cbits/crypton_bzero.h view
@@ -0,0 +1,27 @@+/*+ * Erasing memory that the compiler is entitled to decide nobody reads.+ *+ * memset on an object that is about to die -- freed, or a local going out of+ * scope -- is a store to memory nothing can observe, and an optimizer may+ * drop it.  That is the whole reason explicit_bzero and memset_s exist.+ * Neither is everywhere, so this writes through a volatile pointer, which the+ * standard says cannot be elided.+ *+ * Use it wherever key material stops being needed.  Plain memset is still+ * right for memory that is about to be read again.+ */+#ifndef CRYPTON_BZERO_H+#define CRYPTON_BZERO_H++#include <stddef.h>+#include <stdint.h>++static inline void crypton_bzero(void *p, size_t n)+{+	volatile uint8_t *q = (volatile uint8_t *)p;++	while (n--)+		*q++ = 0;+}++#endif
+ cbits/crypton_camellia.c view
@@ -0,0 +1,697 @@+/*+ * Camellia with a 128-bit key, as RFC 3713 defines it.+ *+ * SP[i] is generated from that standard: the S-box byte i goes through, spread+ * into the positions the P layer sends it to.  P is an exclusive-or of bytes,+ * so the round function is the exclusive-or of eight lookups.+ */+#include <stdint.h>+#include <crypton_camellia.h>++static const uint64_t SP[8][256] = {+{+	0x7070700070000070ULL, 0x8282820082000082ULL, 0x2c2c2c002c00002cULL, 0xececec00ec0000ecULL,+	0xb3b3b300b30000b3ULL, 0x2727270027000027ULL, 0xc0c0c000c00000c0ULL, 0xe5e5e500e50000e5ULL,+	0xe4e4e400e40000e4ULL, 0x8585850085000085ULL, 0x5757570057000057ULL, 0x3535350035000035ULL,+	0xeaeaea00ea0000eaULL, 0x0c0c0c000c00000cULL, 0xaeaeae00ae0000aeULL, 0x4141410041000041ULL,+	0x2323230023000023ULL, 0xefefef00ef0000efULL, 0x6b6b6b006b00006bULL, 0x9393930093000093ULL,+	0x4545450045000045ULL, 0x1919190019000019ULL, 0xa5a5a500a50000a5ULL, 0x2121210021000021ULL,+	0xededed00ed0000edULL, 0x0e0e0e000e00000eULL, 0x4f4f4f004f00004fULL, 0x4e4e4e004e00004eULL,+	0x1d1d1d001d00001dULL, 0x6565650065000065ULL, 0x9292920092000092ULL, 0xbdbdbd00bd0000bdULL,+	0x8686860086000086ULL, 0xb8b8b800b80000b8ULL, 0xafafaf00af0000afULL, 0x8f8f8f008f00008fULL,+	0x7c7c7c007c00007cULL, 0xebebeb00eb0000ebULL, 0x1f1f1f001f00001fULL, 0xcecece00ce0000ceULL,+	0x3e3e3e003e00003eULL, 0x3030300030000030ULL, 0xdcdcdc00dc0000dcULL, 0x5f5f5f005f00005fULL,+	0x5e5e5e005e00005eULL, 0xc5c5c500c50000c5ULL, 0x0b0b0b000b00000bULL, 0x1a1a1a001a00001aULL,+	0xa6a6a600a60000a6ULL, 0xe1e1e100e10000e1ULL, 0x3939390039000039ULL, 0xcacaca00ca0000caULL,+	0xd5d5d500d50000d5ULL, 0x4747470047000047ULL, 0x5d5d5d005d00005dULL, 0x3d3d3d003d00003dULL,+	0xd9d9d900d90000d9ULL, 0x0101010001000001ULL, 0x5a5a5a005a00005aULL, 0xd6d6d600d60000d6ULL,+	0x5151510051000051ULL, 0x5656560056000056ULL, 0x6c6c6c006c00006cULL, 0x4d4d4d004d00004dULL,+	0x8b8b8b008b00008bULL, 0x0d0d0d000d00000dULL, 0x9a9a9a009a00009aULL, 0x6666660066000066ULL,+	0xfbfbfb00fb0000fbULL, 0xcccccc00cc0000ccULL, 0xb0b0b000b00000b0ULL, 0x2d2d2d002d00002dULL,+	0x7474740074000074ULL, 0x1212120012000012ULL, 0x2b2b2b002b00002bULL, 0x2020200020000020ULL,+	0xf0f0f000f00000f0ULL, 0xb1b1b100b10000b1ULL, 0x8484840084000084ULL, 0x9999990099000099ULL,+	0xdfdfdf00df0000dfULL, 0x4c4c4c004c00004cULL, 0xcbcbcb00cb0000cbULL, 0xc2c2c200c20000c2ULL,+	0x3434340034000034ULL, 0x7e7e7e007e00007eULL, 0x7676760076000076ULL, 0x0505050005000005ULL,+	0x6d6d6d006d00006dULL, 0xb7b7b700b70000b7ULL, 0xa9a9a900a90000a9ULL, 0x3131310031000031ULL,+	0xd1d1d100d10000d1ULL, 0x1717170017000017ULL, 0x0404040004000004ULL, 0xd7d7d700d70000d7ULL,+	0x1414140014000014ULL, 0x5858580058000058ULL, 0x3a3a3a003a00003aULL, 0x6161610061000061ULL,+	0xdedede00de0000deULL, 0x1b1b1b001b00001bULL, 0x1111110011000011ULL, 0x1c1c1c001c00001cULL,+	0x3232320032000032ULL, 0x0f0f0f000f00000fULL, 0x9c9c9c009c00009cULL, 0x1616160016000016ULL,+	0x5353530053000053ULL, 0x1818180018000018ULL, 0xf2f2f200f20000f2ULL, 0x2222220022000022ULL,+	0xfefefe00fe0000feULL, 0x4444440044000044ULL, 0xcfcfcf00cf0000cfULL, 0xb2b2b200b20000b2ULL,+	0xc3c3c300c30000c3ULL, 0xb5b5b500b50000b5ULL, 0x7a7a7a007a00007aULL, 0x9191910091000091ULL,+	0x2424240024000024ULL, 0x0808080008000008ULL, 0xe8e8e800e80000e8ULL, 0xa8a8a800a80000a8ULL,+	0x6060600060000060ULL, 0xfcfcfc00fc0000fcULL, 0x6969690069000069ULL, 0x5050500050000050ULL,+	0xaaaaaa00aa0000aaULL, 0xd0d0d000d00000d0ULL, 0xa0a0a000a00000a0ULL, 0x7d7d7d007d00007dULL,+	0xa1a1a100a10000a1ULL, 0x8989890089000089ULL, 0x6262620062000062ULL, 0x9797970097000097ULL,+	0x5454540054000054ULL, 0x5b5b5b005b00005bULL, 0x1e1e1e001e00001eULL, 0x9595950095000095ULL,+	0xe0e0e000e00000e0ULL, 0xffffff00ff0000ffULL, 0x6464640064000064ULL, 0xd2d2d200d20000d2ULL,+	0x1010100010000010ULL, 0xc4c4c400c40000c4ULL, 0x0000000000000000ULL, 0x4848480048000048ULL,+	0xa3a3a300a30000a3ULL, 0xf7f7f700f70000f7ULL, 0x7575750075000075ULL, 0xdbdbdb00db0000dbULL,+	0x8a8a8a008a00008aULL, 0x0303030003000003ULL, 0xe6e6e600e60000e6ULL, 0xdadada00da0000daULL,+	0x0909090009000009ULL, 0x3f3f3f003f00003fULL, 0xdddddd00dd0000ddULL, 0x9494940094000094ULL,+	0x8787870087000087ULL, 0x5c5c5c005c00005cULL, 0x8383830083000083ULL, 0x0202020002000002ULL,+	0xcdcdcd00cd0000cdULL, 0x4a4a4a004a00004aULL, 0x9090900090000090ULL, 0x3333330033000033ULL,+	0x7373730073000073ULL, 0x6767670067000067ULL, 0xf6f6f600f60000f6ULL, 0xf3f3f300f30000f3ULL,+	0x9d9d9d009d00009dULL, 0x7f7f7f007f00007fULL, 0xbfbfbf00bf0000bfULL, 0xe2e2e200e20000e2ULL,+	0x5252520052000052ULL, 0x9b9b9b009b00009bULL, 0xd8d8d800d80000d8ULL, 0x2626260026000026ULL,+	0xc8c8c800c80000c8ULL, 0x3737370037000037ULL, 0xc6c6c600c60000c6ULL, 0x3b3b3b003b00003bULL,+	0x8181810081000081ULL, 0x9696960096000096ULL, 0x6f6f6f006f00006fULL, 0x4b4b4b004b00004bULL,+	0x1313130013000013ULL, 0xbebebe00be0000beULL, 0x6363630063000063ULL, 0x2e2e2e002e00002eULL,+	0xe9e9e900e90000e9ULL, 0x7979790079000079ULL, 0xa7a7a700a70000a7ULL, 0x8c8c8c008c00008cULL,+	0x9f9f9f009f00009fULL, 0x6e6e6e006e00006eULL, 0xbcbcbc00bc0000bcULL, 0x8e8e8e008e00008eULL,+	0x2929290029000029ULL, 0xf5f5f500f50000f5ULL, 0xf9f9f900f90000f9ULL, 0xb6b6b600b60000b6ULL,+	0x2f2f2f002f00002fULL, 0xfdfdfd00fd0000fdULL, 0xb4b4b400b40000b4ULL, 0x5959590059000059ULL,+	0x7878780078000078ULL, 0x9898980098000098ULL, 0x0606060006000006ULL, 0x6a6a6a006a00006aULL,+	0xe7e7e700e70000e7ULL, 0x4646460046000046ULL, 0x7171710071000071ULL, 0xbababa00ba0000baULL,+	0xd4d4d400d40000d4ULL, 0x2525250025000025ULL, 0xababab00ab0000abULL, 0x4242420042000042ULL,+	0x8888880088000088ULL, 0xa2a2a200a20000a2ULL, 0x8d8d8d008d00008dULL, 0xfafafa00fa0000faULL,+	0x7272720072000072ULL, 0x0707070007000007ULL, 0xb9b9b900b90000b9ULL, 0x5555550055000055ULL,+	0xf8f8f800f80000f8ULL, 0xeeeeee00ee0000eeULL, 0xacacac00ac0000acULL, 0x0a0a0a000a00000aULL,+	0x3636360036000036ULL, 0x4949490049000049ULL, 0x2a2a2a002a00002aULL, 0x6868680068000068ULL,+	0x3c3c3c003c00003cULL, 0x3838380038000038ULL, 0xf1f1f100f10000f1ULL, 0xa4a4a400a40000a4ULL,+	0x4040400040000040ULL, 0x2828280028000028ULL, 0xd3d3d300d30000d3ULL, 0x7b7b7b007b00007bULL,+	0xbbbbbb00bb0000bbULL, 0xc9c9c900c90000c9ULL, 0x4343430043000043ULL, 0xc1c1c100c10000c1ULL,+	0x1515150015000015ULL, 0xe3e3e300e30000e3ULL, 0xadadad00ad0000adULL, 0xf4f4f400f40000f4ULL,+	0x7777770077000077ULL, 0xc7c7c700c70000c7ULL, 0x8080800080000080ULL, 0x9e9e9e009e00009eULL,+},+{+	0x00e0e0e0e0e00000ULL, 0x0005050505050000ULL, 0x0058585858580000ULL, 0x00d9d9d9d9d90000ULL,+	0x0067676767670000ULL, 0x004e4e4e4e4e0000ULL, 0x0081818181810000ULL, 0x00cbcbcbcbcb0000ULL,+	0x00c9c9c9c9c90000ULL, 0x000b0b0b0b0b0000ULL, 0x00aeaeaeaeae0000ULL, 0x006a6a6a6a6a0000ULL,+	0x00d5d5d5d5d50000ULL, 0x0018181818180000ULL, 0x005d5d5d5d5d0000ULL, 0x0082828282820000ULL,+	0x0046464646460000ULL, 0x00dfdfdfdfdf0000ULL, 0x00d6d6d6d6d60000ULL, 0x0027272727270000ULL,+	0x008a8a8a8a8a0000ULL, 0x0032323232320000ULL, 0x004b4b4b4b4b0000ULL, 0x0042424242420000ULL,+	0x00dbdbdbdbdb0000ULL, 0x001c1c1c1c1c0000ULL, 0x009e9e9e9e9e0000ULL, 0x009c9c9c9c9c0000ULL,+	0x003a3a3a3a3a0000ULL, 0x00cacacacaca0000ULL, 0x0025252525250000ULL, 0x007b7b7b7b7b0000ULL,+	0x000d0d0d0d0d0000ULL, 0x0071717171710000ULL, 0x005f5f5f5f5f0000ULL, 0x001f1f1f1f1f0000ULL,+	0x00f8f8f8f8f80000ULL, 0x00d7d7d7d7d70000ULL, 0x003e3e3e3e3e0000ULL, 0x009d9d9d9d9d0000ULL,+	0x007c7c7c7c7c0000ULL, 0x0060606060600000ULL, 0x00b9b9b9b9b90000ULL, 0x00bebebebebe0000ULL,+	0x00bcbcbcbcbc0000ULL, 0x008b8b8b8b8b0000ULL, 0x0016161616160000ULL, 0x0034343434340000ULL,+	0x004d4d4d4d4d0000ULL, 0x00c3c3c3c3c30000ULL, 0x0072727272720000ULL, 0x0095959595950000ULL,+	0x00ababababab0000ULL, 0x008e8e8e8e8e0000ULL, 0x00bababababa0000ULL, 0x007a7a7a7a7a0000ULL,+	0x00b3b3b3b3b30000ULL, 0x0002020202020000ULL, 0x00b4b4b4b4b40000ULL, 0x00adadadadad0000ULL,+	0x00a2a2a2a2a20000ULL, 0x00acacacacac0000ULL, 0x00d8d8d8d8d80000ULL, 0x009a9a9a9a9a0000ULL,+	0x0017171717170000ULL, 0x001a1a1a1a1a0000ULL, 0x0035353535350000ULL, 0x00cccccccccc0000ULL,+	0x00f7f7f7f7f70000ULL, 0x0099999999990000ULL, 0x0061616161610000ULL, 0x005a5a5a5a5a0000ULL,+	0x00e8e8e8e8e80000ULL, 0x0024242424240000ULL, 0x0056565656560000ULL, 0x0040404040400000ULL,+	0x00e1e1e1e1e10000ULL, 0x0063636363630000ULL, 0x0009090909090000ULL, 0x0033333333330000ULL,+	0x00bfbfbfbfbf0000ULL, 0x0098989898980000ULL, 0x0097979797970000ULL, 0x0085858585850000ULL,+	0x0068686868680000ULL, 0x00fcfcfcfcfc0000ULL, 0x00ececececec0000ULL, 0x000a0a0a0a0a0000ULL,+	0x00dadadadada0000ULL, 0x006f6f6f6f6f0000ULL, 0x0053535353530000ULL, 0x0062626262620000ULL,+	0x00a3a3a3a3a30000ULL, 0x002e2e2e2e2e0000ULL, 0x0008080808080000ULL, 0x00afafafafaf0000ULL,+	0x0028282828280000ULL, 0x00b0b0b0b0b00000ULL, 0x0074747474740000ULL, 0x00c2c2c2c2c20000ULL,+	0x00bdbdbdbdbd0000ULL, 0x0036363636360000ULL, 0x0022222222220000ULL, 0x0038383838380000ULL,+	0x0064646464640000ULL, 0x001e1e1e1e1e0000ULL, 0x0039393939390000ULL, 0x002c2c2c2c2c0000ULL,+	0x00a6a6a6a6a60000ULL, 0x0030303030300000ULL, 0x00e5e5e5e5e50000ULL, 0x0044444444440000ULL,+	0x00fdfdfdfdfd0000ULL, 0x0088888888880000ULL, 0x009f9f9f9f9f0000ULL, 0x0065656565650000ULL,+	0x0087878787870000ULL, 0x006b6b6b6b6b0000ULL, 0x00f4f4f4f4f40000ULL, 0x0023232323230000ULL,+	0x0048484848480000ULL, 0x0010101010100000ULL, 0x00d1d1d1d1d10000ULL, 0x0051515151510000ULL,+	0x00c0c0c0c0c00000ULL, 0x00f9f9f9f9f90000ULL, 0x00d2d2d2d2d20000ULL, 0x00a0a0a0a0a00000ULL,+	0x0055555555550000ULL, 0x00a1a1a1a1a10000ULL, 0x0041414141410000ULL, 0x00fafafafafa0000ULL,+	0x0043434343430000ULL, 0x0013131313130000ULL, 0x00c4c4c4c4c40000ULL, 0x002f2f2f2f2f0000ULL,+	0x00a8a8a8a8a80000ULL, 0x00b6b6b6b6b60000ULL, 0x003c3c3c3c3c0000ULL, 0x002b2b2b2b2b0000ULL,+	0x00c1c1c1c1c10000ULL, 0x00ffffffffff0000ULL, 0x00c8c8c8c8c80000ULL, 0x00a5a5a5a5a50000ULL,+	0x0020202020200000ULL, 0x0089898989890000ULL, 0x0000000000000000ULL, 0x0090909090900000ULL,+	0x0047474747470000ULL, 0x00efefefefef0000ULL, 0x00eaeaeaeaea0000ULL, 0x00b7b7b7b7b70000ULL,+	0x0015151515150000ULL, 0x0006060606060000ULL, 0x00cdcdcdcdcd0000ULL, 0x00b5b5b5b5b50000ULL,+	0x0012121212120000ULL, 0x007e7e7e7e7e0000ULL, 0x00bbbbbbbbbb0000ULL, 0x0029292929290000ULL,+	0x000f0f0f0f0f0000ULL, 0x00b8b8b8b8b80000ULL, 0x0007070707070000ULL, 0x0004040404040000ULL,+	0x009b9b9b9b9b0000ULL, 0x0094949494940000ULL, 0x0021212121210000ULL, 0x0066666666660000ULL,+	0x00e6e6e6e6e60000ULL, 0x00cecececece0000ULL, 0x00ededededed0000ULL, 0x00e7e7e7e7e70000ULL,+	0x003b3b3b3b3b0000ULL, 0x00fefefefefe0000ULL, 0x007f7f7f7f7f0000ULL, 0x00c5c5c5c5c50000ULL,+	0x00a4a4a4a4a40000ULL, 0x0037373737370000ULL, 0x00b1b1b1b1b10000ULL, 0x004c4c4c4c4c0000ULL,+	0x0091919191910000ULL, 0x006e6e6e6e6e0000ULL, 0x008d8d8d8d8d0000ULL, 0x0076767676760000ULL,+	0x0003030303030000ULL, 0x002d2d2d2d2d0000ULL, 0x00dedededede0000ULL, 0x0096969696960000ULL,+	0x0026262626260000ULL, 0x007d7d7d7d7d0000ULL, 0x00c6c6c6c6c60000ULL, 0x005c5c5c5c5c0000ULL,+	0x00d3d3d3d3d30000ULL, 0x00f2f2f2f2f20000ULL, 0x004f4f4f4f4f0000ULL, 0x0019191919190000ULL,+	0x003f3f3f3f3f0000ULL, 0x00dcdcdcdcdc0000ULL, 0x0079797979790000ULL, 0x001d1d1d1d1d0000ULL,+	0x0052525252520000ULL, 0x00ebebebebeb0000ULL, 0x00f3f3f3f3f30000ULL, 0x006d6d6d6d6d0000ULL,+	0x005e5e5e5e5e0000ULL, 0x00fbfbfbfbfb0000ULL, 0x0069696969690000ULL, 0x00b2b2b2b2b20000ULL,+	0x00f0f0f0f0f00000ULL, 0x0031313131310000ULL, 0x000c0c0c0c0c0000ULL, 0x00d4d4d4d4d40000ULL,+	0x00cfcfcfcfcf0000ULL, 0x008c8c8c8c8c0000ULL, 0x00e2e2e2e2e20000ULL, 0x0075757575750000ULL,+	0x00a9a9a9a9a90000ULL, 0x004a4a4a4a4a0000ULL, 0x0057575757570000ULL, 0x0084848484840000ULL,+	0x0011111111110000ULL, 0x0045454545450000ULL, 0x001b1b1b1b1b0000ULL, 0x00f5f5f5f5f50000ULL,+	0x00e4e4e4e4e40000ULL, 0x000e0e0e0e0e0000ULL, 0x0073737373730000ULL, 0x00aaaaaaaaaa0000ULL,+	0x00f1f1f1f1f10000ULL, 0x00dddddddddd0000ULL, 0x0059595959590000ULL, 0x0014141414140000ULL,+	0x006c6c6c6c6c0000ULL, 0x0092929292920000ULL, 0x0054545454540000ULL, 0x00d0d0d0d0d00000ULL,+	0x0078787878780000ULL, 0x0070707070700000ULL, 0x00e3e3e3e3e30000ULL, 0x0049494949490000ULL,+	0x0080808080800000ULL, 0x0050505050500000ULL, 0x00a7a7a7a7a70000ULL, 0x00f6f6f6f6f60000ULL,+	0x0077777777770000ULL, 0x0093939393930000ULL, 0x0086868686860000ULL, 0x0083838383830000ULL,+	0x002a2a2a2a2a0000ULL, 0x00c7c7c7c7c70000ULL, 0x005b5b5b5b5b0000ULL, 0x00e9e9e9e9e90000ULL,+	0x00eeeeeeeeee0000ULL, 0x008f8f8f8f8f0000ULL, 0x0001010101010000ULL, 0x003d3d3d3d3d0000ULL,+},+{+	0x3800383800383800ULL, 0x4100414100414100ULL, 0x1600161600161600ULL, 0x7600767600767600ULL,+	0xd900d9d900d9d900ULL, 0x9300939300939300ULL, 0x6000606000606000ULL, 0xf200f2f200f2f200ULL,+	0x7200727200727200ULL, 0xc200c2c200c2c200ULL, 0xab00abab00abab00ULL, 0x9a009a9a009a9a00ULL,+	0x7500757500757500ULL, 0x0600060600060600ULL, 0x5700575700575700ULL, 0xa000a0a000a0a000ULL,+	0x9100919100919100ULL, 0xf700f7f700f7f700ULL, 0xb500b5b500b5b500ULL, 0xc900c9c900c9c900ULL,+	0xa200a2a200a2a200ULL, 0x8c008c8c008c8c00ULL, 0xd200d2d200d2d200ULL, 0x9000909000909000ULL,+	0xf600f6f600f6f600ULL, 0x0700070700070700ULL, 0xa700a7a700a7a700ULL, 0x2700272700272700ULL,+	0x8e008e8e008e8e00ULL, 0xb200b2b200b2b200ULL, 0x4900494900494900ULL, 0xde00dede00dede00ULL,+	0x4300434300434300ULL, 0x5c005c5c005c5c00ULL, 0xd700d7d700d7d700ULL, 0xc700c7c700c7c700ULL,+	0x3e003e3e003e3e00ULL, 0xf500f5f500f5f500ULL, 0x8f008f8f008f8f00ULL, 0x6700676700676700ULL,+	0x1f001f1f001f1f00ULL, 0x1800181800181800ULL, 0x6e006e6e006e6e00ULL, 0xaf00afaf00afaf00ULL,+	0x2f002f2f002f2f00ULL, 0xe200e2e200e2e200ULL, 0x8500858500858500ULL, 0x0d000d0d000d0d00ULL,+	0x5300535300535300ULL, 0xf000f0f000f0f000ULL, 0x9c009c9c009c9c00ULL, 0x6500656500656500ULL,+	0xea00eaea00eaea00ULL, 0xa300a3a300a3a300ULL, 0xae00aeae00aeae00ULL, 0x9e009e9e009e9e00ULL,+	0xec00ecec00ecec00ULL, 0x8000808000808000ULL, 0x2d002d2d002d2d00ULL, 0x6b006b6b006b6b00ULL,+	0xa800a8a800a8a800ULL, 0x2b002b2b002b2b00ULL, 0x3600363600363600ULL, 0xa600a6a600a6a600ULL,+	0xc500c5c500c5c500ULL, 0x8600868600868600ULL, 0x4d004d4d004d4d00ULL, 0x3300333300333300ULL,+	0xfd00fdfd00fdfd00ULL, 0x6600666600666600ULL, 0x5800585800585800ULL, 0x9600969600969600ULL,+	0x3a003a3a003a3a00ULL, 0x0900090900090900ULL, 0x9500959500959500ULL, 0x1000101000101000ULL,+	0x7800787800787800ULL, 0xd800d8d800d8d800ULL, 0x4200424200424200ULL, 0xcc00cccc00cccc00ULL,+	0xef00efef00efef00ULL, 0x2600262600262600ULL, 0xe500e5e500e5e500ULL, 0x6100616100616100ULL,+	0x1a001a1a001a1a00ULL, 0x3f003f3f003f3f00ULL, 0x3b003b3b003b3b00ULL, 0x8200828200828200ULL,+	0xb600b6b600b6b600ULL, 0xdb00dbdb00dbdb00ULL, 0xd400d4d400d4d400ULL, 0x9800989800989800ULL,+	0xe800e8e800e8e800ULL, 0x8b008b8b008b8b00ULL, 0x0200020200020200ULL, 0xeb00ebeb00ebeb00ULL,+	0x0a000a0a000a0a00ULL, 0x2c002c2c002c2c00ULL, 0x1d001d1d001d1d00ULL, 0xb000b0b000b0b000ULL,+	0x6f006f6f006f6f00ULL, 0x8d008d8d008d8d00ULL, 0x8800888800888800ULL, 0x0e000e0e000e0e00ULL,+	0x1900191900191900ULL, 0x8700878700878700ULL, 0x4e004e4e004e4e00ULL, 0x0b000b0b000b0b00ULL,+	0xa900a9a900a9a900ULL, 0x0c000c0c000c0c00ULL, 0x7900797900797900ULL, 0x1100111100111100ULL,+	0x7f007f7f007f7f00ULL, 0x2200222200222200ULL, 0xe700e7e700e7e700ULL, 0x5900595900595900ULL,+	0xe100e1e100e1e100ULL, 0xda00dada00dada00ULL, 0x3d003d3d003d3d00ULL, 0xc800c8c800c8c800ULL,+	0x1200121200121200ULL, 0x0400040400040400ULL, 0x7400747400747400ULL, 0x5400545400545400ULL,+	0x3000303000303000ULL, 0x7e007e7e007e7e00ULL, 0xb400b4b400b4b400ULL, 0x2800282800282800ULL,+	0x5500555500555500ULL, 0x6800686800686800ULL, 0x5000505000505000ULL, 0xbe00bebe00bebe00ULL,+	0xd000d0d000d0d000ULL, 0xc400c4c400c4c400ULL, 0x3100313100313100ULL, 0xcb00cbcb00cbcb00ULL,+	0x2a002a2a002a2a00ULL, 0xad00adad00adad00ULL, 0x0f000f0f000f0f00ULL, 0xca00caca00caca00ULL,+	0x7000707000707000ULL, 0xff00ffff00ffff00ULL, 0x3200323200323200ULL, 0x6900696900696900ULL,+	0x0800080800080800ULL, 0x6200626200626200ULL, 0x0000000000000000ULL, 0x2400242400242400ULL,+	0xd100d1d100d1d100ULL, 0xfb00fbfb00fbfb00ULL, 0xba00baba00baba00ULL, 0xed00eded00eded00ULL,+	0x4500454500454500ULL, 0x8100818100818100ULL, 0x7300737300737300ULL, 0x6d006d6d006d6d00ULL,+	0x8400848400848400ULL, 0x9f009f9f009f9f00ULL, 0xee00eeee00eeee00ULL, 0x4a004a4a004a4a00ULL,+	0xc300c3c300c3c300ULL, 0x2e002e2e002e2e00ULL, 0xc100c1c100c1c100ULL, 0x0100010100010100ULL,+	0xe600e6e600e6e600ULL, 0x2500252500252500ULL, 0x4800484800484800ULL, 0x9900999900999900ULL,+	0xb900b9b900b9b900ULL, 0xb300b3b300b3b300ULL, 0x7b007b7b007b7b00ULL, 0xf900f9f900f9f900ULL,+	0xce00cece00cece00ULL, 0xbf00bfbf00bfbf00ULL, 0xdf00dfdf00dfdf00ULL, 0x7100717100717100ULL,+	0x2900292900292900ULL, 0xcd00cdcd00cdcd00ULL, 0x6c006c6c006c6c00ULL, 0x1300131300131300ULL,+	0x6400646400646400ULL, 0x9b009b9b009b9b00ULL, 0x6300636300636300ULL, 0x9d009d9d009d9d00ULL,+	0xc000c0c000c0c000ULL, 0x4b004b4b004b4b00ULL, 0xb700b7b700b7b700ULL, 0xa500a5a500a5a500ULL,+	0x8900898900898900ULL, 0x5f005f5f005f5f00ULL, 0xb100b1b100b1b100ULL, 0x1700171700171700ULL,+	0xf400f4f400f4f400ULL, 0xbc00bcbc00bcbc00ULL, 0xd300d3d300d3d300ULL, 0x4600464600464600ULL,+	0xcf00cfcf00cfcf00ULL, 0x3700373700373700ULL, 0x5e005e5e005e5e00ULL, 0x4700474700474700ULL,+	0x9400949400949400ULL, 0xfa00fafa00fafa00ULL, 0xfc00fcfc00fcfc00ULL, 0x5b005b5b005b5b00ULL,+	0x9700979700979700ULL, 0xfe00fefe00fefe00ULL, 0x5a005a5a005a5a00ULL, 0xac00acac00acac00ULL,+	0x3c003c3c003c3c00ULL, 0x4c004c4c004c4c00ULL, 0x0300030300030300ULL, 0x3500353500353500ULL,+	0xf300f3f300f3f300ULL, 0x2300232300232300ULL, 0xb800b8b800b8b800ULL, 0x5d005d5d005d5d00ULL,+	0x6a006a6a006a6a00ULL, 0x9200929200929200ULL, 0xd500d5d500d5d500ULL, 0x2100212100212100ULL,+	0x4400444400444400ULL, 0x5100515100515100ULL, 0xc600c6c600c6c600ULL, 0x7d007d7d007d7d00ULL,+	0x3900393900393900ULL, 0x8300838300838300ULL, 0xdc00dcdc00dcdc00ULL, 0xaa00aaaa00aaaa00ULL,+	0x7c007c7c007c7c00ULL, 0x7700777700777700ULL, 0x5600565600565600ULL, 0x0500050500050500ULL,+	0x1b001b1b001b1b00ULL, 0xa400a4a400a4a400ULL, 0x1500151500151500ULL, 0x3400343400343400ULL,+	0x1e001e1e001e1e00ULL, 0x1c001c1c001c1c00ULL, 0xf800f8f800f8f800ULL, 0x5200525200525200ULL,+	0x2000202000202000ULL, 0x1400141400141400ULL, 0xe900e9e900e9e900ULL, 0xbd00bdbd00bdbd00ULL,+	0xdd00dddd00dddd00ULL, 0xe400e4e400e4e400ULL, 0xa100a1a100a1a100ULL, 0xe000e0e000e0e000ULL,+	0x8a008a8a008a8a00ULL, 0xf100f1f100f1f100ULL, 0xd600d6d600d6d600ULL, 0x7a007a7a007a7a00ULL,+	0xbb00bbbb00bbbb00ULL, 0xe300e3e300e3e300ULL, 0x4000404000404000ULL, 0x4f004f4f004f4f00ULL,+},+{+	0x7070007000007070ULL, 0x2c2c002c00002c2cULL, 0xb3b300b30000b3b3ULL, 0xc0c000c00000c0c0ULL,+	0xe4e400e40000e4e4ULL, 0x5757005700005757ULL, 0xeaea00ea0000eaeaULL, 0xaeae00ae0000aeaeULL,+	0x2323002300002323ULL, 0x6b6b006b00006b6bULL, 0x4545004500004545ULL, 0xa5a500a50000a5a5ULL,+	0xeded00ed0000ededULL, 0x4f4f004f00004f4fULL, 0x1d1d001d00001d1dULL, 0x9292009200009292ULL,+	0x8686008600008686ULL, 0xafaf00af0000afafULL, 0x7c7c007c00007c7cULL, 0x1f1f001f00001f1fULL,+	0x3e3e003e00003e3eULL, 0xdcdc00dc0000dcdcULL, 0x5e5e005e00005e5eULL, 0x0b0b000b00000b0bULL,+	0xa6a600a60000a6a6ULL, 0x3939003900003939ULL, 0xd5d500d50000d5d5ULL, 0x5d5d005d00005d5dULL,+	0xd9d900d90000d9d9ULL, 0x5a5a005a00005a5aULL, 0x5151005100005151ULL, 0x6c6c006c00006c6cULL,+	0x8b8b008b00008b8bULL, 0x9a9a009a00009a9aULL, 0xfbfb00fb0000fbfbULL, 0xb0b000b00000b0b0ULL,+	0x7474007400007474ULL, 0x2b2b002b00002b2bULL, 0xf0f000f00000f0f0ULL, 0x8484008400008484ULL,+	0xdfdf00df0000dfdfULL, 0xcbcb00cb0000cbcbULL, 0x3434003400003434ULL, 0x7676007600007676ULL,+	0x6d6d006d00006d6dULL, 0xa9a900a90000a9a9ULL, 0xd1d100d10000d1d1ULL, 0x0404000400000404ULL,+	0x1414001400001414ULL, 0x3a3a003a00003a3aULL, 0xdede00de0000dedeULL, 0x1111001100001111ULL,+	0x3232003200003232ULL, 0x9c9c009c00009c9cULL, 0x5353005300005353ULL, 0xf2f200f20000f2f2ULL,+	0xfefe00fe0000fefeULL, 0xcfcf00cf0000cfcfULL, 0xc3c300c30000c3c3ULL, 0x7a7a007a00007a7aULL,+	0x2424002400002424ULL, 0xe8e800e80000e8e8ULL, 0x6060006000006060ULL, 0x6969006900006969ULL,+	0xaaaa00aa0000aaaaULL, 0xa0a000a00000a0a0ULL, 0xa1a100a10000a1a1ULL, 0x6262006200006262ULL,+	0x5454005400005454ULL, 0x1e1e001e00001e1eULL, 0xe0e000e00000e0e0ULL, 0x6464006400006464ULL,+	0x1010001000001010ULL, 0x0000000000000000ULL, 0xa3a300a30000a3a3ULL, 0x7575007500007575ULL,+	0x8a8a008a00008a8aULL, 0xe6e600e60000e6e6ULL, 0x0909000900000909ULL, 0xdddd00dd0000ddddULL,+	0x8787008700008787ULL, 0x8383008300008383ULL, 0xcdcd00cd0000cdcdULL, 0x9090009000009090ULL,+	0x7373007300007373ULL, 0xf6f600f60000f6f6ULL, 0x9d9d009d00009d9dULL, 0xbfbf00bf0000bfbfULL,+	0x5252005200005252ULL, 0xd8d800d80000d8d8ULL, 0xc8c800c80000c8c8ULL, 0xc6c600c60000c6c6ULL,+	0x8181008100008181ULL, 0x6f6f006f00006f6fULL, 0x1313001300001313ULL, 0x6363006300006363ULL,+	0xe9e900e90000e9e9ULL, 0xa7a700a70000a7a7ULL, 0x9f9f009f00009f9fULL, 0xbcbc00bc0000bcbcULL,+	0x2929002900002929ULL, 0xf9f900f90000f9f9ULL, 0x2f2f002f00002f2fULL, 0xb4b400b40000b4b4ULL,+	0x7878007800007878ULL, 0x0606000600000606ULL, 0xe7e700e70000e7e7ULL, 0x7171007100007171ULL,+	0xd4d400d40000d4d4ULL, 0xabab00ab0000ababULL, 0x8888008800008888ULL, 0x8d8d008d00008d8dULL,+	0x7272007200007272ULL, 0xb9b900b90000b9b9ULL, 0xf8f800f80000f8f8ULL, 0xacac00ac0000acacULL,+	0x3636003600003636ULL, 0x2a2a002a00002a2aULL, 0x3c3c003c00003c3cULL, 0xf1f100f10000f1f1ULL,+	0x4040004000004040ULL, 0xd3d300d30000d3d3ULL, 0xbbbb00bb0000bbbbULL, 0x4343004300004343ULL,+	0x1515001500001515ULL, 0xadad00ad0000adadULL, 0x7777007700007777ULL, 0x8080008000008080ULL,+	0x8282008200008282ULL, 0xecec00ec0000ececULL, 0x2727002700002727ULL, 0xe5e500e50000e5e5ULL,+	0x8585008500008585ULL, 0x3535003500003535ULL, 0x0c0c000c00000c0cULL, 0x4141004100004141ULL,+	0xefef00ef0000efefULL, 0x9393009300009393ULL, 0x1919001900001919ULL, 0x2121002100002121ULL,+	0x0e0e000e00000e0eULL, 0x4e4e004e00004e4eULL, 0x6565006500006565ULL, 0xbdbd00bd0000bdbdULL,+	0xb8b800b80000b8b8ULL, 0x8f8f008f00008f8fULL, 0xebeb00eb0000ebebULL, 0xcece00ce0000ceceULL,+	0x3030003000003030ULL, 0x5f5f005f00005f5fULL, 0xc5c500c50000c5c5ULL, 0x1a1a001a00001a1aULL,+	0xe1e100e10000e1e1ULL, 0xcaca00ca0000cacaULL, 0x4747004700004747ULL, 0x3d3d003d00003d3dULL,+	0x0101000100000101ULL, 0xd6d600d60000d6d6ULL, 0x5656005600005656ULL, 0x4d4d004d00004d4dULL,+	0x0d0d000d00000d0dULL, 0x6666006600006666ULL, 0xcccc00cc0000ccccULL, 0x2d2d002d00002d2dULL,+	0x1212001200001212ULL, 0x2020002000002020ULL, 0xb1b100b10000b1b1ULL, 0x9999009900009999ULL,+	0x4c4c004c00004c4cULL, 0xc2c200c20000c2c2ULL, 0x7e7e007e00007e7eULL, 0x0505000500000505ULL,+	0xb7b700b70000b7b7ULL, 0x3131003100003131ULL, 0x1717001700001717ULL, 0xd7d700d70000d7d7ULL,+	0x5858005800005858ULL, 0x6161006100006161ULL, 0x1b1b001b00001b1bULL, 0x1c1c001c00001c1cULL,+	0x0f0f000f00000f0fULL, 0x1616001600001616ULL, 0x1818001800001818ULL, 0x2222002200002222ULL,+	0x4444004400004444ULL, 0xb2b200b20000b2b2ULL, 0xb5b500b50000b5b5ULL, 0x9191009100009191ULL,+	0x0808000800000808ULL, 0xa8a800a80000a8a8ULL, 0xfcfc00fc0000fcfcULL, 0x5050005000005050ULL,+	0xd0d000d00000d0d0ULL, 0x7d7d007d00007d7dULL, 0x8989008900008989ULL, 0x9797009700009797ULL,+	0x5b5b005b00005b5bULL, 0x9595009500009595ULL, 0xffff00ff0000ffffULL, 0xd2d200d20000d2d2ULL,+	0xc4c400c40000c4c4ULL, 0x4848004800004848ULL, 0xf7f700f70000f7f7ULL, 0xdbdb00db0000dbdbULL,+	0x0303000300000303ULL, 0xdada00da0000dadaULL, 0x3f3f003f00003f3fULL, 0x9494009400009494ULL,+	0x5c5c005c00005c5cULL, 0x0202000200000202ULL, 0x4a4a004a00004a4aULL, 0x3333003300003333ULL,+	0x6767006700006767ULL, 0xf3f300f30000f3f3ULL, 0x7f7f007f00007f7fULL, 0xe2e200e20000e2e2ULL,+	0x9b9b009b00009b9bULL, 0x2626002600002626ULL, 0x3737003700003737ULL, 0x3b3b003b00003b3bULL,+	0x9696009600009696ULL, 0x4b4b004b00004b4bULL, 0xbebe00be0000bebeULL, 0x2e2e002e00002e2eULL,+	0x7979007900007979ULL, 0x8c8c008c00008c8cULL, 0x6e6e006e00006e6eULL, 0x8e8e008e00008e8eULL,+	0xf5f500f50000f5f5ULL, 0xb6b600b60000b6b6ULL, 0xfdfd00fd0000fdfdULL, 0x5959005900005959ULL,+	0x9898009800009898ULL, 0x6a6a006a00006a6aULL, 0x4646004600004646ULL, 0xbaba00ba0000babaULL,+	0x2525002500002525ULL, 0x4242004200004242ULL, 0xa2a200a20000a2a2ULL, 0xfafa00fa0000fafaULL,+	0x0707000700000707ULL, 0x5555005500005555ULL, 0xeeee00ee0000eeeeULL, 0x0a0a000a00000a0aULL,+	0x4949004900004949ULL, 0x6868006800006868ULL, 0x3838003800003838ULL, 0xa4a400a40000a4a4ULL,+	0x2828002800002828ULL, 0x7b7b007b00007b7bULL, 0xc9c900c90000c9c9ULL, 0xc1c100c10000c1c1ULL,+	0xe3e300e30000e3e3ULL, 0xf4f400f40000f4f4ULL, 0xc7c700c70000c7c7ULL, 0x9e9e009e00009e9eULL,+},+{+	0x00e0e0e000e0e0e0ULL, 0x0005050500050505ULL, 0x0058585800585858ULL, 0x00d9d9d900d9d9d9ULL,+	0x0067676700676767ULL, 0x004e4e4e004e4e4eULL, 0x0081818100818181ULL, 0x00cbcbcb00cbcbcbULL,+	0x00c9c9c900c9c9c9ULL, 0x000b0b0b000b0b0bULL, 0x00aeaeae00aeaeaeULL, 0x006a6a6a006a6a6aULL,+	0x00d5d5d500d5d5d5ULL, 0x0018181800181818ULL, 0x005d5d5d005d5d5dULL, 0x0082828200828282ULL,+	0x0046464600464646ULL, 0x00dfdfdf00dfdfdfULL, 0x00d6d6d600d6d6d6ULL, 0x0027272700272727ULL,+	0x008a8a8a008a8a8aULL, 0x0032323200323232ULL, 0x004b4b4b004b4b4bULL, 0x0042424200424242ULL,+	0x00dbdbdb00dbdbdbULL, 0x001c1c1c001c1c1cULL, 0x009e9e9e009e9e9eULL, 0x009c9c9c009c9c9cULL,+	0x003a3a3a003a3a3aULL, 0x00cacaca00cacacaULL, 0x0025252500252525ULL, 0x007b7b7b007b7b7bULL,+	0x000d0d0d000d0d0dULL, 0x0071717100717171ULL, 0x005f5f5f005f5f5fULL, 0x001f1f1f001f1f1fULL,+	0x00f8f8f800f8f8f8ULL, 0x00d7d7d700d7d7d7ULL, 0x003e3e3e003e3e3eULL, 0x009d9d9d009d9d9dULL,+	0x007c7c7c007c7c7cULL, 0x0060606000606060ULL, 0x00b9b9b900b9b9b9ULL, 0x00bebebe00bebebeULL,+	0x00bcbcbc00bcbcbcULL, 0x008b8b8b008b8b8bULL, 0x0016161600161616ULL, 0x0034343400343434ULL,+	0x004d4d4d004d4d4dULL, 0x00c3c3c300c3c3c3ULL, 0x0072727200727272ULL, 0x0095959500959595ULL,+	0x00ababab00abababULL, 0x008e8e8e008e8e8eULL, 0x00bababa00bababaULL, 0x007a7a7a007a7a7aULL,+	0x00b3b3b300b3b3b3ULL, 0x0002020200020202ULL, 0x00b4b4b400b4b4b4ULL, 0x00adadad00adadadULL,+	0x00a2a2a200a2a2a2ULL, 0x00acacac00acacacULL, 0x00d8d8d800d8d8d8ULL, 0x009a9a9a009a9a9aULL,+	0x0017171700171717ULL, 0x001a1a1a001a1a1aULL, 0x0035353500353535ULL, 0x00cccccc00ccccccULL,+	0x00f7f7f700f7f7f7ULL, 0x0099999900999999ULL, 0x0061616100616161ULL, 0x005a5a5a005a5a5aULL,+	0x00e8e8e800e8e8e8ULL, 0x0024242400242424ULL, 0x0056565600565656ULL, 0x0040404000404040ULL,+	0x00e1e1e100e1e1e1ULL, 0x0063636300636363ULL, 0x0009090900090909ULL, 0x0033333300333333ULL,+	0x00bfbfbf00bfbfbfULL, 0x0098989800989898ULL, 0x0097979700979797ULL, 0x0085858500858585ULL,+	0x0068686800686868ULL, 0x00fcfcfc00fcfcfcULL, 0x00ececec00ecececULL, 0x000a0a0a000a0a0aULL,+	0x00dadada00dadadaULL, 0x006f6f6f006f6f6fULL, 0x0053535300535353ULL, 0x0062626200626262ULL,+	0x00a3a3a300a3a3a3ULL, 0x002e2e2e002e2e2eULL, 0x0008080800080808ULL, 0x00afafaf00afafafULL,+	0x0028282800282828ULL, 0x00b0b0b000b0b0b0ULL, 0x0074747400747474ULL, 0x00c2c2c200c2c2c2ULL,+	0x00bdbdbd00bdbdbdULL, 0x0036363600363636ULL, 0x0022222200222222ULL, 0x0038383800383838ULL,+	0x0064646400646464ULL, 0x001e1e1e001e1e1eULL, 0x0039393900393939ULL, 0x002c2c2c002c2c2cULL,+	0x00a6a6a600a6a6a6ULL, 0x0030303000303030ULL, 0x00e5e5e500e5e5e5ULL, 0x0044444400444444ULL,+	0x00fdfdfd00fdfdfdULL, 0x0088888800888888ULL, 0x009f9f9f009f9f9fULL, 0x0065656500656565ULL,+	0x0087878700878787ULL, 0x006b6b6b006b6b6bULL, 0x00f4f4f400f4f4f4ULL, 0x0023232300232323ULL,+	0x0048484800484848ULL, 0x0010101000101010ULL, 0x00d1d1d100d1d1d1ULL, 0x0051515100515151ULL,+	0x00c0c0c000c0c0c0ULL, 0x00f9f9f900f9f9f9ULL, 0x00d2d2d200d2d2d2ULL, 0x00a0a0a000a0a0a0ULL,+	0x0055555500555555ULL, 0x00a1a1a100a1a1a1ULL, 0x0041414100414141ULL, 0x00fafafa00fafafaULL,+	0x0043434300434343ULL, 0x0013131300131313ULL, 0x00c4c4c400c4c4c4ULL, 0x002f2f2f002f2f2fULL,+	0x00a8a8a800a8a8a8ULL, 0x00b6b6b600b6b6b6ULL, 0x003c3c3c003c3c3cULL, 0x002b2b2b002b2b2bULL,+	0x00c1c1c100c1c1c1ULL, 0x00ffffff00ffffffULL, 0x00c8c8c800c8c8c8ULL, 0x00a5a5a500a5a5a5ULL,+	0x0020202000202020ULL, 0x0089898900898989ULL, 0x0000000000000000ULL, 0x0090909000909090ULL,+	0x0047474700474747ULL, 0x00efefef00efefefULL, 0x00eaeaea00eaeaeaULL, 0x00b7b7b700b7b7b7ULL,+	0x0015151500151515ULL, 0x0006060600060606ULL, 0x00cdcdcd00cdcdcdULL, 0x00b5b5b500b5b5b5ULL,+	0x0012121200121212ULL, 0x007e7e7e007e7e7eULL, 0x00bbbbbb00bbbbbbULL, 0x0029292900292929ULL,+	0x000f0f0f000f0f0fULL, 0x00b8b8b800b8b8b8ULL, 0x0007070700070707ULL, 0x0004040400040404ULL,+	0x009b9b9b009b9b9bULL, 0x0094949400949494ULL, 0x0021212100212121ULL, 0x0066666600666666ULL,+	0x00e6e6e600e6e6e6ULL, 0x00cecece00cececeULL, 0x00ededed00edededULL, 0x00e7e7e700e7e7e7ULL,+	0x003b3b3b003b3b3bULL, 0x00fefefe00fefefeULL, 0x007f7f7f007f7f7fULL, 0x00c5c5c500c5c5c5ULL,+	0x00a4a4a400a4a4a4ULL, 0x0037373700373737ULL, 0x00b1b1b100b1b1b1ULL, 0x004c4c4c004c4c4cULL,+	0x0091919100919191ULL, 0x006e6e6e006e6e6eULL, 0x008d8d8d008d8d8dULL, 0x0076767600767676ULL,+	0x0003030300030303ULL, 0x002d2d2d002d2d2dULL, 0x00dedede00dededeULL, 0x0096969600969696ULL,+	0x0026262600262626ULL, 0x007d7d7d007d7d7dULL, 0x00c6c6c600c6c6c6ULL, 0x005c5c5c005c5c5cULL,+	0x00d3d3d300d3d3d3ULL, 0x00f2f2f200f2f2f2ULL, 0x004f4f4f004f4f4fULL, 0x0019191900191919ULL,+	0x003f3f3f003f3f3fULL, 0x00dcdcdc00dcdcdcULL, 0x0079797900797979ULL, 0x001d1d1d001d1d1dULL,+	0x0052525200525252ULL, 0x00ebebeb00ebebebULL, 0x00f3f3f300f3f3f3ULL, 0x006d6d6d006d6d6dULL,+	0x005e5e5e005e5e5eULL, 0x00fbfbfb00fbfbfbULL, 0x0069696900696969ULL, 0x00b2b2b200b2b2b2ULL,+	0x00f0f0f000f0f0f0ULL, 0x0031313100313131ULL, 0x000c0c0c000c0c0cULL, 0x00d4d4d400d4d4d4ULL,+	0x00cfcfcf00cfcfcfULL, 0x008c8c8c008c8c8cULL, 0x00e2e2e200e2e2e2ULL, 0x0075757500757575ULL,+	0x00a9a9a900a9a9a9ULL, 0x004a4a4a004a4a4aULL, 0x0057575700575757ULL, 0x0084848400848484ULL,+	0x0011111100111111ULL, 0x0045454500454545ULL, 0x001b1b1b001b1b1bULL, 0x00f5f5f500f5f5f5ULL,+	0x00e4e4e400e4e4e4ULL, 0x000e0e0e000e0e0eULL, 0x0073737300737373ULL, 0x00aaaaaa00aaaaaaULL,+	0x00f1f1f100f1f1f1ULL, 0x00dddddd00ddddddULL, 0x0059595900595959ULL, 0x0014141400141414ULL,+	0x006c6c6c006c6c6cULL, 0x0092929200929292ULL, 0x0054545400545454ULL, 0x00d0d0d000d0d0d0ULL,+	0x0078787800787878ULL, 0x0070707000707070ULL, 0x00e3e3e300e3e3e3ULL, 0x0049494900494949ULL,+	0x0080808000808080ULL, 0x0050505000505050ULL, 0x00a7a7a700a7a7a7ULL, 0x00f6f6f600f6f6f6ULL,+	0x0077777700777777ULL, 0x0093939300939393ULL, 0x0086868600868686ULL, 0x0083838300838383ULL,+	0x002a2a2a002a2a2aULL, 0x00c7c7c700c7c7c7ULL, 0x005b5b5b005b5b5bULL, 0x00e9e9e900e9e9e9ULL,+	0x00eeeeee00eeeeeeULL, 0x008f8f8f008f8f8fULL, 0x0001010100010101ULL, 0x003d3d3d003d3d3dULL,+},+{+	0x3800383838003838ULL, 0x4100414141004141ULL, 0x1600161616001616ULL, 0x7600767676007676ULL,+	0xd900d9d9d900d9d9ULL, 0x9300939393009393ULL, 0x6000606060006060ULL, 0xf200f2f2f200f2f2ULL,+	0x7200727272007272ULL, 0xc200c2c2c200c2c2ULL, 0xab00ababab00ababULL, 0x9a009a9a9a009a9aULL,+	0x7500757575007575ULL, 0x0600060606000606ULL, 0x5700575757005757ULL, 0xa000a0a0a000a0a0ULL,+	0x9100919191009191ULL, 0xf700f7f7f700f7f7ULL, 0xb500b5b5b500b5b5ULL, 0xc900c9c9c900c9c9ULL,+	0xa200a2a2a200a2a2ULL, 0x8c008c8c8c008c8cULL, 0xd200d2d2d200d2d2ULL, 0x9000909090009090ULL,+	0xf600f6f6f600f6f6ULL, 0x0700070707000707ULL, 0xa700a7a7a700a7a7ULL, 0x2700272727002727ULL,+	0x8e008e8e8e008e8eULL, 0xb200b2b2b200b2b2ULL, 0x4900494949004949ULL, 0xde00dedede00dedeULL,+	0x4300434343004343ULL, 0x5c005c5c5c005c5cULL, 0xd700d7d7d700d7d7ULL, 0xc700c7c7c700c7c7ULL,+	0x3e003e3e3e003e3eULL, 0xf500f5f5f500f5f5ULL, 0x8f008f8f8f008f8fULL, 0x6700676767006767ULL,+	0x1f001f1f1f001f1fULL, 0x1800181818001818ULL, 0x6e006e6e6e006e6eULL, 0xaf00afafaf00afafULL,+	0x2f002f2f2f002f2fULL, 0xe200e2e2e200e2e2ULL, 0x8500858585008585ULL, 0x0d000d0d0d000d0dULL,+	0x5300535353005353ULL, 0xf000f0f0f000f0f0ULL, 0x9c009c9c9c009c9cULL, 0x6500656565006565ULL,+	0xea00eaeaea00eaeaULL, 0xa300a3a3a300a3a3ULL, 0xae00aeaeae00aeaeULL, 0x9e009e9e9e009e9eULL,+	0xec00ececec00ececULL, 0x8000808080008080ULL, 0x2d002d2d2d002d2dULL, 0x6b006b6b6b006b6bULL,+	0xa800a8a8a800a8a8ULL, 0x2b002b2b2b002b2bULL, 0x3600363636003636ULL, 0xa600a6a6a600a6a6ULL,+	0xc500c5c5c500c5c5ULL, 0x8600868686008686ULL, 0x4d004d4d4d004d4dULL, 0x3300333333003333ULL,+	0xfd00fdfdfd00fdfdULL, 0x6600666666006666ULL, 0x5800585858005858ULL, 0x9600969696009696ULL,+	0x3a003a3a3a003a3aULL, 0x0900090909000909ULL, 0x9500959595009595ULL, 0x1000101010001010ULL,+	0x7800787878007878ULL, 0xd800d8d8d800d8d8ULL, 0x4200424242004242ULL, 0xcc00cccccc00ccccULL,+	0xef00efefef00efefULL, 0x2600262626002626ULL, 0xe500e5e5e500e5e5ULL, 0x6100616161006161ULL,+	0x1a001a1a1a001a1aULL, 0x3f003f3f3f003f3fULL, 0x3b003b3b3b003b3bULL, 0x8200828282008282ULL,+	0xb600b6b6b600b6b6ULL, 0xdb00dbdbdb00dbdbULL, 0xd400d4d4d400d4d4ULL, 0x9800989898009898ULL,+	0xe800e8e8e800e8e8ULL, 0x8b008b8b8b008b8bULL, 0x0200020202000202ULL, 0xeb00ebebeb00ebebULL,+	0x0a000a0a0a000a0aULL, 0x2c002c2c2c002c2cULL, 0x1d001d1d1d001d1dULL, 0xb000b0b0b000b0b0ULL,+	0x6f006f6f6f006f6fULL, 0x8d008d8d8d008d8dULL, 0x8800888888008888ULL, 0x0e000e0e0e000e0eULL,+	0x1900191919001919ULL, 0x8700878787008787ULL, 0x4e004e4e4e004e4eULL, 0x0b000b0b0b000b0bULL,+	0xa900a9a9a900a9a9ULL, 0x0c000c0c0c000c0cULL, 0x7900797979007979ULL, 0x1100111111001111ULL,+	0x7f007f7f7f007f7fULL, 0x2200222222002222ULL, 0xe700e7e7e700e7e7ULL, 0x5900595959005959ULL,+	0xe100e1e1e100e1e1ULL, 0xda00dadada00dadaULL, 0x3d003d3d3d003d3dULL, 0xc800c8c8c800c8c8ULL,+	0x1200121212001212ULL, 0x0400040404000404ULL, 0x7400747474007474ULL, 0x5400545454005454ULL,+	0x3000303030003030ULL, 0x7e007e7e7e007e7eULL, 0xb400b4b4b400b4b4ULL, 0x2800282828002828ULL,+	0x5500555555005555ULL, 0x6800686868006868ULL, 0x5000505050005050ULL, 0xbe00bebebe00bebeULL,+	0xd000d0d0d000d0d0ULL, 0xc400c4c4c400c4c4ULL, 0x3100313131003131ULL, 0xcb00cbcbcb00cbcbULL,+	0x2a002a2a2a002a2aULL, 0xad00adadad00adadULL, 0x0f000f0f0f000f0fULL, 0xca00cacaca00cacaULL,+	0x7000707070007070ULL, 0xff00ffffff00ffffULL, 0x3200323232003232ULL, 0x6900696969006969ULL,+	0x0800080808000808ULL, 0x6200626262006262ULL, 0x0000000000000000ULL, 0x2400242424002424ULL,+	0xd100d1d1d100d1d1ULL, 0xfb00fbfbfb00fbfbULL, 0xba00bababa00babaULL, 0xed00ededed00ededULL,+	0x4500454545004545ULL, 0x8100818181008181ULL, 0x7300737373007373ULL, 0x6d006d6d6d006d6dULL,+	0x8400848484008484ULL, 0x9f009f9f9f009f9fULL, 0xee00eeeeee00eeeeULL, 0x4a004a4a4a004a4aULL,+	0xc300c3c3c300c3c3ULL, 0x2e002e2e2e002e2eULL, 0xc100c1c1c100c1c1ULL, 0x0100010101000101ULL,+	0xe600e6e6e600e6e6ULL, 0x2500252525002525ULL, 0x4800484848004848ULL, 0x9900999999009999ULL,+	0xb900b9b9b900b9b9ULL, 0xb300b3b3b300b3b3ULL, 0x7b007b7b7b007b7bULL, 0xf900f9f9f900f9f9ULL,+	0xce00cecece00ceceULL, 0xbf00bfbfbf00bfbfULL, 0xdf00dfdfdf00dfdfULL, 0x7100717171007171ULL,+	0x2900292929002929ULL, 0xcd00cdcdcd00cdcdULL, 0x6c006c6c6c006c6cULL, 0x1300131313001313ULL,+	0x6400646464006464ULL, 0x9b009b9b9b009b9bULL, 0x6300636363006363ULL, 0x9d009d9d9d009d9dULL,+	0xc000c0c0c000c0c0ULL, 0x4b004b4b4b004b4bULL, 0xb700b7b7b700b7b7ULL, 0xa500a5a5a500a5a5ULL,+	0x8900898989008989ULL, 0x5f005f5f5f005f5fULL, 0xb100b1b1b100b1b1ULL, 0x1700171717001717ULL,+	0xf400f4f4f400f4f4ULL, 0xbc00bcbcbc00bcbcULL, 0xd300d3d3d300d3d3ULL, 0x4600464646004646ULL,+	0xcf00cfcfcf00cfcfULL, 0x3700373737003737ULL, 0x5e005e5e5e005e5eULL, 0x4700474747004747ULL,+	0x9400949494009494ULL, 0xfa00fafafa00fafaULL, 0xfc00fcfcfc00fcfcULL, 0x5b005b5b5b005b5bULL,+	0x9700979797009797ULL, 0xfe00fefefe00fefeULL, 0x5a005a5a5a005a5aULL, 0xac00acacac00acacULL,+	0x3c003c3c3c003c3cULL, 0x4c004c4c4c004c4cULL, 0x0300030303000303ULL, 0x3500353535003535ULL,+	0xf300f3f3f300f3f3ULL, 0x2300232323002323ULL, 0xb800b8b8b800b8b8ULL, 0x5d005d5d5d005d5dULL,+	0x6a006a6a6a006a6aULL, 0x9200929292009292ULL, 0xd500d5d5d500d5d5ULL, 0x2100212121002121ULL,+	0x4400444444004444ULL, 0x5100515151005151ULL, 0xc600c6c6c600c6c6ULL, 0x7d007d7d7d007d7dULL,+	0x3900393939003939ULL, 0x8300838383008383ULL, 0xdc00dcdcdc00dcdcULL, 0xaa00aaaaaa00aaaaULL,+	0x7c007c7c7c007c7cULL, 0x7700777777007777ULL, 0x5600565656005656ULL, 0x0500050505000505ULL,+	0x1b001b1b1b001b1bULL, 0xa400a4a4a400a4a4ULL, 0x1500151515001515ULL, 0x3400343434003434ULL,+	0x1e001e1e1e001e1eULL, 0x1c001c1c1c001c1cULL, 0xf800f8f8f800f8f8ULL, 0x5200525252005252ULL,+	0x2000202020002020ULL, 0x1400141414001414ULL, 0xe900e9e9e900e9e9ULL, 0xbd00bdbdbd00bdbdULL,+	0xdd00dddddd00ddddULL, 0xe400e4e4e400e4e4ULL, 0xa100a1a1a100a1a1ULL, 0xe000e0e0e000e0e0ULL,+	0x8a008a8a8a008a8aULL, 0xf100f1f1f100f1f1ULL, 0xd600d6d6d600d6d6ULL, 0x7a007a7a7a007a7aULL,+	0xbb00bbbbbb00bbbbULL, 0xe300e3e3e300e3e3ULL, 0x4000404040004040ULL, 0x4f004f4f4f004f4fULL,+},+{+	0x7070007070700070ULL, 0x2c2c002c2c2c002cULL, 0xb3b300b3b3b300b3ULL, 0xc0c000c0c0c000c0ULL,+	0xe4e400e4e4e400e4ULL, 0x5757005757570057ULL, 0xeaea00eaeaea00eaULL, 0xaeae00aeaeae00aeULL,+	0x2323002323230023ULL, 0x6b6b006b6b6b006bULL, 0x4545004545450045ULL, 0xa5a500a5a5a500a5ULL,+	0xeded00ededed00edULL, 0x4f4f004f4f4f004fULL, 0x1d1d001d1d1d001dULL, 0x9292009292920092ULL,+	0x8686008686860086ULL, 0xafaf00afafaf00afULL, 0x7c7c007c7c7c007cULL, 0x1f1f001f1f1f001fULL,+	0x3e3e003e3e3e003eULL, 0xdcdc00dcdcdc00dcULL, 0x5e5e005e5e5e005eULL, 0x0b0b000b0b0b000bULL,+	0xa6a600a6a6a600a6ULL, 0x3939003939390039ULL, 0xd5d500d5d5d500d5ULL, 0x5d5d005d5d5d005dULL,+	0xd9d900d9d9d900d9ULL, 0x5a5a005a5a5a005aULL, 0x5151005151510051ULL, 0x6c6c006c6c6c006cULL,+	0x8b8b008b8b8b008bULL, 0x9a9a009a9a9a009aULL, 0xfbfb00fbfbfb00fbULL, 0xb0b000b0b0b000b0ULL,+	0x7474007474740074ULL, 0x2b2b002b2b2b002bULL, 0xf0f000f0f0f000f0ULL, 0x8484008484840084ULL,+	0xdfdf00dfdfdf00dfULL, 0xcbcb00cbcbcb00cbULL, 0x3434003434340034ULL, 0x7676007676760076ULL,+	0x6d6d006d6d6d006dULL, 0xa9a900a9a9a900a9ULL, 0xd1d100d1d1d100d1ULL, 0x0404000404040004ULL,+	0x1414001414140014ULL, 0x3a3a003a3a3a003aULL, 0xdede00dedede00deULL, 0x1111001111110011ULL,+	0x3232003232320032ULL, 0x9c9c009c9c9c009cULL, 0x5353005353530053ULL, 0xf2f200f2f2f200f2ULL,+	0xfefe00fefefe00feULL, 0xcfcf00cfcfcf00cfULL, 0xc3c300c3c3c300c3ULL, 0x7a7a007a7a7a007aULL,+	0x2424002424240024ULL, 0xe8e800e8e8e800e8ULL, 0x6060006060600060ULL, 0x6969006969690069ULL,+	0xaaaa00aaaaaa00aaULL, 0xa0a000a0a0a000a0ULL, 0xa1a100a1a1a100a1ULL, 0x6262006262620062ULL,+	0x5454005454540054ULL, 0x1e1e001e1e1e001eULL, 0xe0e000e0e0e000e0ULL, 0x6464006464640064ULL,+	0x1010001010100010ULL, 0x0000000000000000ULL, 0xa3a300a3a3a300a3ULL, 0x7575007575750075ULL,+	0x8a8a008a8a8a008aULL, 0xe6e600e6e6e600e6ULL, 0x0909000909090009ULL, 0xdddd00dddddd00ddULL,+	0x8787008787870087ULL, 0x8383008383830083ULL, 0xcdcd00cdcdcd00cdULL, 0x9090009090900090ULL,+	0x7373007373730073ULL, 0xf6f600f6f6f600f6ULL, 0x9d9d009d9d9d009dULL, 0xbfbf00bfbfbf00bfULL,+	0x5252005252520052ULL, 0xd8d800d8d8d800d8ULL, 0xc8c800c8c8c800c8ULL, 0xc6c600c6c6c600c6ULL,+	0x8181008181810081ULL, 0x6f6f006f6f6f006fULL, 0x1313001313130013ULL, 0x6363006363630063ULL,+	0xe9e900e9e9e900e9ULL, 0xa7a700a7a7a700a7ULL, 0x9f9f009f9f9f009fULL, 0xbcbc00bcbcbc00bcULL,+	0x2929002929290029ULL, 0xf9f900f9f9f900f9ULL, 0x2f2f002f2f2f002fULL, 0xb4b400b4b4b400b4ULL,+	0x7878007878780078ULL, 0x0606000606060006ULL, 0xe7e700e7e7e700e7ULL, 0x7171007171710071ULL,+	0xd4d400d4d4d400d4ULL, 0xabab00ababab00abULL, 0x8888008888880088ULL, 0x8d8d008d8d8d008dULL,+	0x7272007272720072ULL, 0xb9b900b9b9b900b9ULL, 0xf8f800f8f8f800f8ULL, 0xacac00acacac00acULL,+	0x3636003636360036ULL, 0x2a2a002a2a2a002aULL, 0x3c3c003c3c3c003cULL, 0xf1f100f1f1f100f1ULL,+	0x4040004040400040ULL, 0xd3d300d3d3d300d3ULL, 0xbbbb00bbbbbb00bbULL, 0x4343004343430043ULL,+	0x1515001515150015ULL, 0xadad00adadad00adULL, 0x7777007777770077ULL, 0x8080008080800080ULL,+	0x8282008282820082ULL, 0xecec00ececec00ecULL, 0x2727002727270027ULL, 0xe5e500e5e5e500e5ULL,+	0x8585008585850085ULL, 0x3535003535350035ULL, 0x0c0c000c0c0c000cULL, 0x4141004141410041ULL,+	0xefef00efefef00efULL, 0x9393009393930093ULL, 0x1919001919190019ULL, 0x2121002121210021ULL,+	0x0e0e000e0e0e000eULL, 0x4e4e004e4e4e004eULL, 0x6565006565650065ULL, 0xbdbd00bdbdbd00bdULL,+	0xb8b800b8b8b800b8ULL, 0x8f8f008f8f8f008fULL, 0xebeb00ebebeb00ebULL, 0xcece00cecece00ceULL,+	0x3030003030300030ULL, 0x5f5f005f5f5f005fULL, 0xc5c500c5c5c500c5ULL, 0x1a1a001a1a1a001aULL,+	0xe1e100e1e1e100e1ULL, 0xcaca00cacaca00caULL, 0x4747004747470047ULL, 0x3d3d003d3d3d003dULL,+	0x0101000101010001ULL, 0xd6d600d6d6d600d6ULL, 0x5656005656560056ULL, 0x4d4d004d4d4d004dULL,+	0x0d0d000d0d0d000dULL, 0x6666006666660066ULL, 0xcccc00cccccc00ccULL, 0x2d2d002d2d2d002dULL,+	0x1212001212120012ULL, 0x2020002020200020ULL, 0xb1b100b1b1b100b1ULL, 0x9999009999990099ULL,+	0x4c4c004c4c4c004cULL, 0xc2c200c2c2c200c2ULL, 0x7e7e007e7e7e007eULL, 0x0505000505050005ULL,+	0xb7b700b7b7b700b7ULL, 0x3131003131310031ULL, 0x1717001717170017ULL, 0xd7d700d7d7d700d7ULL,+	0x5858005858580058ULL, 0x6161006161610061ULL, 0x1b1b001b1b1b001bULL, 0x1c1c001c1c1c001cULL,+	0x0f0f000f0f0f000fULL, 0x1616001616160016ULL, 0x1818001818180018ULL, 0x2222002222220022ULL,+	0x4444004444440044ULL, 0xb2b200b2b2b200b2ULL, 0xb5b500b5b5b500b5ULL, 0x9191009191910091ULL,+	0x0808000808080008ULL, 0xa8a800a8a8a800a8ULL, 0xfcfc00fcfcfc00fcULL, 0x5050005050500050ULL,+	0xd0d000d0d0d000d0ULL, 0x7d7d007d7d7d007dULL, 0x8989008989890089ULL, 0x9797009797970097ULL,+	0x5b5b005b5b5b005bULL, 0x9595009595950095ULL, 0xffff00ffffff00ffULL, 0xd2d200d2d2d200d2ULL,+	0xc4c400c4c4c400c4ULL, 0x4848004848480048ULL, 0xf7f700f7f7f700f7ULL, 0xdbdb00dbdbdb00dbULL,+	0x0303000303030003ULL, 0xdada00dadada00daULL, 0x3f3f003f3f3f003fULL, 0x9494009494940094ULL,+	0x5c5c005c5c5c005cULL, 0x0202000202020002ULL, 0x4a4a004a4a4a004aULL, 0x3333003333330033ULL,+	0x6767006767670067ULL, 0xf3f300f3f3f300f3ULL, 0x7f7f007f7f7f007fULL, 0xe2e200e2e2e200e2ULL,+	0x9b9b009b9b9b009bULL, 0x2626002626260026ULL, 0x3737003737370037ULL, 0x3b3b003b3b3b003bULL,+	0x9696009696960096ULL, 0x4b4b004b4b4b004bULL, 0xbebe00bebebe00beULL, 0x2e2e002e2e2e002eULL,+	0x7979007979790079ULL, 0x8c8c008c8c8c008cULL, 0x6e6e006e6e6e006eULL, 0x8e8e008e8e8e008eULL,+	0xf5f500f5f5f500f5ULL, 0xb6b600b6b6b600b6ULL, 0xfdfd00fdfdfd00fdULL, 0x5959005959590059ULL,+	0x9898009898980098ULL, 0x6a6a006a6a6a006aULL, 0x4646004646460046ULL, 0xbaba00bababa00baULL,+	0x2525002525250025ULL, 0x4242004242420042ULL, 0xa2a200a2a2a200a2ULL, 0xfafa00fafafa00faULL,+	0x0707000707070007ULL, 0x5555005555550055ULL, 0xeeee00eeeeee00eeULL, 0x0a0a000a0a0a000aULL,+	0x4949004949490049ULL, 0x6868006868680068ULL, 0x3838003838380038ULL, 0xa4a400a4a4a400a4ULL,+	0x2828002828280028ULL, 0x7b7b007b7b7b007bULL, 0xc9c900c9c9c900c9ULL, 0xc1c100c1c1c100c1ULL,+	0xe3e300e3e3e300e3ULL, 0xf4f400f4f4f400f4ULL, 0xc7c700c7c7c700c7ULL, 0x9e9e009e9e9e009eULL,+},+{+	0x7070700070707000ULL, 0x8282820082828200ULL, 0x2c2c2c002c2c2c00ULL, 0xececec00ececec00ULL,+	0xb3b3b300b3b3b300ULL, 0x2727270027272700ULL, 0xc0c0c000c0c0c000ULL, 0xe5e5e500e5e5e500ULL,+	0xe4e4e400e4e4e400ULL, 0x8585850085858500ULL, 0x5757570057575700ULL, 0x3535350035353500ULL,+	0xeaeaea00eaeaea00ULL, 0x0c0c0c000c0c0c00ULL, 0xaeaeae00aeaeae00ULL, 0x4141410041414100ULL,+	0x2323230023232300ULL, 0xefefef00efefef00ULL, 0x6b6b6b006b6b6b00ULL, 0x9393930093939300ULL,+	0x4545450045454500ULL, 0x1919190019191900ULL, 0xa5a5a500a5a5a500ULL, 0x2121210021212100ULL,+	0xededed00ededed00ULL, 0x0e0e0e000e0e0e00ULL, 0x4f4f4f004f4f4f00ULL, 0x4e4e4e004e4e4e00ULL,+	0x1d1d1d001d1d1d00ULL, 0x6565650065656500ULL, 0x9292920092929200ULL, 0xbdbdbd00bdbdbd00ULL,+	0x8686860086868600ULL, 0xb8b8b800b8b8b800ULL, 0xafafaf00afafaf00ULL, 0x8f8f8f008f8f8f00ULL,+	0x7c7c7c007c7c7c00ULL, 0xebebeb00ebebeb00ULL, 0x1f1f1f001f1f1f00ULL, 0xcecece00cecece00ULL,+	0x3e3e3e003e3e3e00ULL, 0x3030300030303000ULL, 0xdcdcdc00dcdcdc00ULL, 0x5f5f5f005f5f5f00ULL,+	0x5e5e5e005e5e5e00ULL, 0xc5c5c500c5c5c500ULL, 0x0b0b0b000b0b0b00ULL, 0x1a1a1a001a1a1a00ULL,+	0xa6a6a600a6a6a600ULL, 0xe1e1e100e1e1e100ULL, 0x3939390039393900ULL, 0xcacaca00cacaca00ULL,+	0xd5d5d500d5d5d500ULL, 0x4747470047474700ULL, 0x5d5d5d005d5d5d00ULL, 0x3d3d3d003d3d3d00ULL,+	0xd9d9d900d9d9d900ULL, 0x0101010001010100ULL, 0x5a5a5a005a5a5a00ULL, 0xd6d6d600d6d6d600ULL,+	0x5151510051515100ULL, 0x5656560056565600ULL, 0x6c6c6c006c6c6c00ULL, 0x4d4d4d004d4d4d00ULL,+	0x8b8b8b008b8b8b00ULL, 0x0d0d0d000d0d0d00ULL, 0x9a9a9a009a9a9a00ULL, 0x6666660066666600ULL,+	0xfbfbfb00fbfbfb00ULL, 0xcccccc00cccccc00ULL, 0xb0b0b000b0b0b000ULL, 0x2d2d2d002d2d2d00ULL,+	0x7474740074747400ULL, 0x1212120012121200ULL, 0x2b2b2b002b2b2b00ULL, 0x2020200020202000ULL,+	0xf0f0f000f0f0f000ULL, 0xb1b1b100b1b1b100ULL, 0x8484840084848400ULL, 0x9999990099999900ULL,+	0xdfdfdf00dfdfdf00ULL, 0x4c4c4c004c4c4c00ULL, 0xcbcbcb00cbcbcb00ULL, 0xc2c2c200c2c2c200ULL,+	0x3434340034343400ULL, 0x7e7e7e007e7e7e00ULL, 0x7676760076767600ULL, 0x0505050005050500ULL,+	0x6d6d6d006d6d6d00ULL, 0xb7b7b700b7b7b700ULL, 0xa9a9a900a9a9a900ULL, 0x3131310031313100ULL,+	0xd1d1d100d1d1d100ULL, 0x1717170017171700ULL, 0x0404040004040400ULL, 0xd7d7d700d7d7d700ULL,+	0x1414140014141400ULL, 0x5858580058585800ULL, 0x3a3a3a003a3a3a00ULL, 0x6161610061616100ULL,+	0xdedede00dedede00ULL, 0x1b1b1b001b1b1b00ULL, 0x1111110011111100ULL, 0x1c1c1c001c1c1c00ULL,+	0x3232320032323200ULL, 0x0f0f0f000f0f0f00ULL, 0x9c9c9c009c9c9c00ULL, 0x1616160016161600ULL,+	0x5353530053535300ULL, 0x1818180018181800ULL, 0xf2f2f200f2f2f200ULL, 0x2222220022222200ULL,+	0xfefefe00fefefe00ULL, 0x4444440044444400ULL, 0xcfcfcf00cfcfcf00ULL, 0xb2b2b200b2b2b200ULL,+	0xc3c3c300c3c3c300ULL, 0xb5b5b500b5b5b500ULL, 0x7a7a7a007a7a7a00ULL, 0x9191910091919100ULL,+	0x2424240024242400ULL, 0x0808080008080800ULL, 0xe8e8e800e8e8e800ULL, 0xa8a8a800a8a8a800ULL,+	0x6060600060606000ULL, 0xfcfcfc00fcfcfc00ULL, 0x6969690069696900ULL, 0x5050500050505000ULL,+	0xaaaaaa00aaaaaa00ULL, 0xd0d0d000d0d0d000ULL, 0xa0a0a000a0a0a000ULL, 0x7d7d7d007d7d7d00ULL,+	0xa1a1a100a1a1a100ULL, 0x8989890089898900ULL, 0x6262620062626200ULL, 0x9797970097979700ULL,+	0x5454540054545400ULL, 0x5b5b5b005b5b5b00ULL, 0x1e1e1e001e1e1e00ULL, 0x9595950095959500ULL,+	0xe0e0e000e0e0e000ULL, 0xffffff00ffffff00ULL, 0x6464640064646400ULL, 0xd2d2d200d2d2d200ULL,+	0x1010100010101000ULL, 0xc4c4c400c4c4c400ULL, 0x0000000000000000ULL, 0x4848480048484800ULL,+	0xa3a3a300a3a3a300ULL, 0xf7f7f700f7f7f700ULL, 0x7575750075757500ULL, 0xdbdbdb00dbdbdb00ULL,+	0x8a8a8a008a8a8a00ULL, 0x0303030003030300ULL, 0xe6e6e600e6e6e600ULL, 0xdadada00dadada00ULL,+	0x0909090009090900ULL, 0x3f3f3f003f3f3f00ULL, 0xdddddd00dddddd00ULL, 0x9494940094949400ULL,+	0x8787870087878700ULL, 0x5c5c5c005c5c5c00ULL, 0x8383830083838300ULL, 0x0202020002020200ULL,+	0xcdcdcd00cdcdcd00ULL, 0x4a4a4a004a4a4a00ULL, 0x9090900090909000ULL, 0x3333330033333300ULL,+	0x7373730073737300ULL, 0x6767670067676700ULL, 0xf6f6f600f6f6f600ULL, 0xf3f3f300f3f3f300ULL,+	0x9d9d9d009d9d9d00ULL, 0x7f7f7f007f7f7f00ULL, 0xbfbfbf00bfbfbf00ULL, 0xe2e2e200e2e2e200ULL,+	0x5252520052525200ULL, 0x9b9b9b009b9b9b00ULL, 0xd8d8d800d8d8d800ULL, 0x2626260026262600ULL,+	0xc8c8c800c8c8c800ULL, 0x3737370037373700ULL, 0xc6c6c600c6c6c600ULL, 0x3b3b3b003b3b3b00ULL,+	0x8181810081818100ULL, 0x9696960096969600ULL, 0x6f6f6f006f6f6f00ULL, 0x4b4b4b004b4b4b00ULL,+	0x1313130013131300ULL, 0xbebebe00bebebe00ULL, 0x6363630063636300ULL, 0x2e2e2e002e2e2e00ULL,+	0xe9e9e900e9e9e900ULL, 0x7979790079797900ULL, 0xa7a7a700a7a7a700ULL, 0x8c8c8c008c8c8c00ULL,+	0x9f9f9f009f9f9f00ULL, 0x6e6e6e006e6e6e00ULL, 0xbcbcbc00bcbcbc00ULL, 0x8e8e8e008e8e8e00ULL,+	0x2929290029292900ULL, 0xf5f5f500f5f5f500ULL, 0xf9f9f900f9f9f900ULL, 0xb6b6b600b6b6b600ULL,+	0x2f2f2f002f2f2f00ULL, 0xfdfdfd00fdfdfd00ULL, 0xb4b4b400b4b4b400ULL, 0x5959590059595900ULL,+	0x7878780078787800ULL, 0x9898980098989800ULL, 0x0606060006060600ULL, 0x6a6a6a006a6a6a00ULL,+	0xe7e7e700e7e7e700ULL, 0x4646460046464600ULL, 0x7171710071717100ULL, 0xbababa00bababa00ULL,+	0xd4d4d400d4d4d400ULL, 0x2525250025252500ULL, 0xababab00ababab00ULL, 0x4242420042424200ULL,+	0x8888880088888800ULL, 0xa2a2a200a2a2a200ULL, 0x8d8d8d008d8d8d00ULL, 0xfafafa00fafafa00ULL,+	0x7272720072727200ULL, 0x0707070007070700ULL, 0xb9b9b900b9b9b900ULL, 0x5555550055555500ULL,+	0xf8f8f800f8f8f800ULL, 0xeeeeee00eeeeee00ULL, 0xacacac00acacac00ULL, 0x0a0a0a000a0a0a00ULL,+	0x3636360036363600ULL, 0x4949490049494900ULL, 0x2a2a2a002a2a2a00ULL, 0x6868680068686800ULL,+	0x3c3c3c003c3c3c00ULL, 0x3838380038383800ULL, 0xf1f1f100f1f1f100ULL, 0xa4a4a400a4a4a400ULL,+	0x4040400040404000ULL, 0x2828280028282800ULL, 0xd3d3d300d3d3d300ULL, 0x7b7b7b007b7b7b00ULL,+	0xbbbbbb00bbbbbb00ULL, 0xc9c9c900c9c9c900ULL, 0x4343430043434300ULL, 0xc1c1c100c1c1c100ULL,+	0x1515150015151500ULL, 0xe3e3e300e3e3e300ULL, 0xadadad00adadad00ULL, 0xf4f4f400f4f4f400ULL,+	0x7777770077777700ULL, 0xc7c7c700c7c7c700ULL, 0x8080800080808000ULL, 0x9e9e9e009e9e9e00ULL,+},+};++static const uint64_t SIGMA[6] = {+	0xA09E667F3BCC908BULL, 0xB67AE8584CAA73B2ULL, 0xC6EF372FE94F82BEULL,+	0x54FF53A5F1D36F1CULL, 0x10E527FADE682D1DULL, 0xB05688C2B3E6C1FDULL+};++static inline uint64_t load_be64(const uint8_t *p)+{+	return ((uint64_t) p[0] << 56) | ((uint64_t) p[1] << 48)+	     | ((uint64_t) p[2] << 40) | ((uint64_t) p[3] << 32)+	     | ((uint64_t) p[4] << 24) | ((uint64_t) p[5] << 16)+	     | ((uint64_t) p[6] << 8)  | ((uint64_t) p[7]);+}++static inline void store_be64(uint8_t *p, uint64_t v)+{+	p[0] = (uint8_t) (v >> 56); p[1] = (uint8_t) (v >> 48);+	p[2] = (uint8_t) (v >> 40); p[3] = (uint8_t) (v >> 32);+	p[4] = (uint8_t) (v >> 24); p[5] = (uint8_t) (v >> 16);+	p[6] = (uint8_t) (v >> 8);  p[7] = (uint8_t) v;+}++static inline uint64_t camellia_f(uint64_t fin, uint64_t ke)+{+	uint64_t x = fin ^ ke;+	return SP[0][(x >> 56) & 0xff] ^ SP[1][(x >> 48) & 0xff]+	     ^ SP[2][(x >> 40) & 0xff] ^ SP[3][(x >> 32) & 0xff]+	     ^ SP[4][(x >> 24) & 0xff] ^ SP[5][(x >> 16) & 0xff]+	     ^ SP[6][(x >>  8) & 0xff] ^ SP[7][ x        & 0xff];+}++static inline uint32_t rotl32(uint32_t v, int n)+{+	return (v << n) | (v >> (32 - n));+}++static inline uint64_t camellia_fl(uint64_t fin, uint64_t ke)+{+	uint32_t x1 = (uint32_t) (fin >> 32), x2 = (uint32_t) fin;+	uint32_t k1 = (uint32_t) (ke >> 32), k2 = (uint32_t) ke;++	x2 ^= rotl32(x1 & k1, 1);+	x1 ^= (x2 | k2);+	return ((uint64_t) x1 << 32) | x2;+}++static inline uint64_t camellia_flinv(uint64_t fin, uint64_t ke)+{+	uint32_t y1 = (uint32_t) (fin >> 32), y2 = (uint32_t) fin;+	uint32_t k1 = (uint32_t) (ke >> 32), k2 = (uint32_t) ke;++	y1 ^= (y2 | k2);+	y2 ^= rotl32(y1 & k1, 1);+	return ((uint64_t) y1 << 32) | y2;+}++/* the halves of a 128-bit value rotated left by n, 0 < n < 128 */+static void rotl128(uint64_t hi, uint64_t lo, int n, uint64_t *rhi, uint64_t *rlo)+{+	if (n >= 64) {+		uint64_t t = hi;+		hi = lo;+		lo = t;+		n -= 64;+	}+	if (n == 0) {+		*rhi = hi;+		*rlo = lo;+	} else {+		*rhi = (hi << n) | (lo >> (64 - n));+		*rlo = (lo << n) | (hi >> (64 - n));+	}+}++void crypton_camellia_init(crypton_camellia_key *ks, const uint8_t *key)+{+	uint64_t klhi = load_be64(key), kllo = load_be64(key + 8);+	uint64_t d1 = klhi, d2 = kllo, kahi, kalo, hi, lo;++	d2 ^= camellia_f(d1, SIGMA[0]);+	d1 ^= camellia_f(d2, SIGMA[1]);+	d1 ^= klhi;+	d2 ^= kllo;+	d2 ^= camellia_f(d1, SIGMA[2]);+	d1 ^= camellia_f(d2, SIGMA[3]);+	kahi = d1;+	kalo = d2;++	ks->kw[0] = klhi;+	ks->kw[1] = kllo;+	ks->k[0] = kahi;+	ks->k[1] = kalo;+	rotl128(klhi, kllo, 15, &hi, &lo);  ks->k[2] = hi;  ks->k[3] = lo;+	rotl128(kahi, kalo, 15, &hi, &lo);  ks->k[4] = hi;  ks->k[5] = lo;+	rotl128(kahi, kalo, 30, &hi, &lo);  ks->ke[0] = hi; ks->ke[1] = lo;+	rotl128(klhi, kllo, 45, &hi, &lo);  ks->k[6] = hi;  ks->k[7] = lo;+	rotl128(kahi, kalo, 45, &hi, &lo);  ks->k[8] = hi;+	rotl128(klhi, kllo, 60, &hi, &lo);  ks->k[9] = lo;+	rotl128(kahi, kalo, 60, &hi, &lo);  ks->k[10] = hi; ks->k[11] = lo;+	rotl128(klhi, kllo, 77, &hi, &lo);  ks->ke[2] = hi; ks->ke[3] = lo;+	rotl128(klhi, kllo, 94, &hi, &lo);  ks->k[12] = hi; ks->k[13] = lo;+	rotl128(kahi, kalo, 94, &hi, &lo);  ks->k[14] = hi; ks->k[15] = lo;+	rotl128(klhi, kllo, 111, &hi, &lo); ks->k[16] = hi; ks->k[17] = lo;+	rotl128(kahi, kalo, 111, &hi, &lo); ks->kw[2] = hi; ks->kw[3] = lo;+}++static void camellia_crypt(uint8_t *out, const uint64_t kw[4], const uint64_t k[18],+                           const uint64_t ke[4], const uint8_t *in, uint32_t nblocks)+{+	uint32_t i;++	for (i = 0; i < nblocks; i++) {+		uint64_t d1 = load_be64(in + 16 * i) ^ kw[0];+		uint64_t d2 = load_be64(in + 16 * i + 8) ^ kw[1];+		int base;++		for (base = 0; base <= 12; base += 6) {+			d2 ^= camellia_f(d1, k[base + 0]);+			d1 ^= camellia_f(d2, k[base + 1]);+			d2 ^= camellia_f(d1, k[base + 2]);+			d1 ^= camellia_f(d2, k[base + 3]);+			d2 ^= camellia_f(d1, k[base + 4]);+			d1 ^= camellia_f(d2, k[base + 5]);+			if (base == 0) {+				d1 = camellia_fl(d1, ke[0]);+				d2 = camellia_flinv(d2, ke[1]);+			} else if (base == 6) {+				d1 = camellia_fl(d1, ke[2]);+				d2 = camellia_flinv(d2, ke[3]);+			}+		}++		store_be64(out + 16 * i, d2 ^ kw[2]);+		store_be64(out + 16 * i + 8, d1 ^ kw[3]);+	}+}++void crypton_camellia_encrypt(uint8_t *out, const crypton_camellia_key *ks,+                              const uint8_t *in, uint32_t nblocks)+{+	camellia_crypt(out, ks->kw, ks->k, ks->ke, in, nblocks);+}++/* Decryption is the same rounds with the subkeys the other way round. */+void crypton_camellia_decrypt(uint8_t *out, const crypton_camellia_key *ks,+                              const uint8_t *in, uint32_t nblocks)+{+	uint64_t kw[4], k[18], ke[4];+	int i;++	kw[0] = ks->kw[2]; kw[1] = ks->kw[3]; kw[2] = ks->kw[0]; kw[3] = ks->kw[1];+	for (i = 0; i < 18; i++)+		k[i] = ks->k[17 - i];+	for (i = 0; i < 4; i++)+		ke[i] = ks->ke[3 - i];+	camellia_crypt(out, kw, k, ke, in, nblocks);+}
+ cbits/crypton_camellia.h view
@@ -0,0 +1,21 @@+#ifndef CRYPTON_CAMELLIA_H+#define CRYPTON_CAMELLIA_H++#include <stdint.h>++/* the subkeys of RFC 3713 section 2.2, for a 128-bit key */+typedef struct {+	uint64_t kw[4];+	uint64_t k[18];+	uint64_t ke[4];+} crypton_camellia_key;++void crypton_camellia_init(crypton_camellia_key *ks, const uint8_t *key);++void crypton_camellia_encrypt(uint8_t *out, const crypton_camellia_key *ks,+                              const uint8_t *in, uint32_t nblocks);++void crypton_camellia_decrypt(uint8_t *out, const crypton_camellia_key *ks,+                              const uint8_t *in, uint32_t nblocks);++#endif
cbits/crypton_chacha.c view
@@ -35,6 +35,71 @@ #include "crypton_align.h" #include <stdio.h> +/*+ * Four blocks at a time with whichever vector unit the target has: NEON in+ * chacha_neon.c, SSE2 in chacha_sse2.c.  Both present the same two entry+ * points, so there is one path here.+ *+ * The state words are held little-endian -- the core below reads them+ * without converting -- so the vector versions, which also do not convert,+ * are left out on a big-endian machine.+ */+#if (defined(WITH_ARMV8_NEON) && !defined(__AARCH64EB__)) || defined(WITH_X86_SSE2)+#define CHACHA_SIMD 1+int crypton_chacha_simd_width(void);+void crypton_chacha_simd_combine(int rounds, uint8_t *dst, const uint8_t *src,+                                 const crypton_chacha_state *in);+void crypton_chacha_simd_generate(int rounds, uint8_t *dst,+                                  const crypton_chacha_state *in);+/* The counters in a group must not carry into d[13], which the crypton_chacha_block loop+ * below handles and the vector one does not; that is one run in 2^29. */+#define CHACHA_SIMD_OK(st, n) ((st)->d[12] <= 0xffffffffU - (uint32_t) (n))+#endif++/*+ * ChaCha20 from CRYPTOGAMS, in cbits/asm/chacha-armv8-*.S.  It keeps four+ * vector blocks and a fifth in the general registers in flight at once, or+ * six and two above 512 bytes, which is more than the intrinsics above can+ * be made to do: the vector registers hold four states and there is no room+ * for another, so the extra parallelism has to come from the integer side,+ * and that means saying which register holds what.+ *+ * Twenty rounds and the 256-bit constants are built into it, and it takes+ * the counter as 32 bits wide, so it is given only the states it fits.+ */+#if (defined(WITH_ARMV8_CHACHA_ASM) && !defined(__AARCH64EB__)) \+    || defined(WITH_X86_CHACHA_ASM)+#define CHACHA_ASM 1+#include "crypton_cpu.h"+void crypton_chacha20_asm_ctr32(uint8_t *out, const uint8_t *in, size_t len,+                                const uint32_t key[8], const uint32_t counter[4]);++/* crypton_cpu.c defines the crypton_armcap_P that the assembly reads to+ * find out whether the processor has NEON. */++/* The four words at the head of the state are the constants that go with a+ * 256-bit key, and the assembly has only those. */+static int chacha_asm_state(const crypton_chacha_state *st)+{+	return st->d[0] == 0x61707865 && st->d[1] == 0x3320646e+	    && st->d[2] == 0x79622d32 && st->d[3] == 0x6b206574;+}++/*+ * How much is worth handing over.  On AArch64 the module's vector path+ * starts at three blocks and below that its scalar path measures level with+ * the C here, so there is nothing to gain; on x86-64 it is ahead from one+ * crypton_chacha_block, the C there having no vector path until eight.+ */+#ifndef CHACHA_ASM_MIN_BLOCKS+#ifdef WITH_X86_CHACHA_ASM+#define CHACHA_ASM_MIN_BLOCKS 1+#else+#define CHACHA_ASM_MIN_BLOCKS 3+#endif+#endif+#endif+ #define QR(a,b,c,d) \ 	a += b; d = rol32(d ^ a,16); \ 	c += d; b = rol32(b ^ c,12); \@@ -47,7 +112,7 @@ static const uint8_t sigma[16] = "expand 32-byte k"; static const uint8_t tau[16] = "expand 16-byte k"; -static void chacha_core(int rounds, block *out, const crypton_chacha_state *in)+static void chacha_core(int rounds, crypton_chacha_block *out, const crypton_chacha_state *in) { 	uint32_t x0, x1, x2, x3, x4, x5, x6, x7, x8, x9, x10, x11, x12, x13, x14, x15; 	int i;@@ -231,7 +296,7 @@  void crypton_chacha_combine(uint8_t *dst, crypton_chacha_context *ctx, const uint8_t *src, uint32_t bytes) {-	block out;+	crypton_chacha_block out; 	crypton_chacha_state *st; 	int i; @@ -256,13 +321,53 @@  	st = &ctx->st; +#ifdef CHACHA_ASM+	if (ctx->nb_rounds == 20 && chacha_asm_state(st)) {+		uint32_t blocks = bytes / 64;++		/* the counter is the caller's to advance, and the assembly+		 * carries it no further than its own 32 bits */+		if (blocks > 0xffffffffU - st->d[12])+			blocks = 0xffffffffU - st->d[12];+		if (blocks >= CHACHA_ASM_MIN_BLOCKS) {+			const uint32_t done = blocks * 64;++#ifdef CRYPTON_X86_ASM+			/* what the module dispatches on, which it reads+			 * directly; resolved once */+			crypton_x86_ia32cap_resolve();+#endif+			crypton_chacha20_asm_ctr32(dst, src, done, &st->d[4],+			                           &st->d[12]);+			st->d[12] += blocks;+			bytes -= done; src += done; dst += done;+		}+	}+#endif++#ifdef CHACHA_SIMD+	{+		const uint32_t nb = (uint32_t) crypton_chacha_simd_width();+		const uint32_t step = 64 * nb;++		while (bytes >= step && CHACHA_SIMD_OK(st, nb)) {+			crypton_chacha_simd_combine(ctx->nb_rounds, dst, src, st);+			st->d[12] += nb;+			bytes -= step; src += step; dst += step;+		}+	}+#endif+ 	/* xor new 64-bytes chunks and store the left over if any */ 	for (; bytes >= 64; bytes -= 64, src += 64, dst += 64) { 		/* generate new chunk and update state */ 		chacha_core(ctx->nb_rounds, &out, st);-		st->d[12] += 1;-		if (st->d[12] == 0)-			st->d[13] += 1;+		uint32_t t0 = le32_to_cpu(st->d[12]);+		st->d[12] = cpu_to_le32(t0 + 1);+		if (st->d[12] == 0) {+			uint32_t t1 = le32_to_cpu(st->d[13]);+			st->d[13] = cpu_to_le32(t1 + 1);+		}  		for (i = 0; i < 64; ++i) 			dst[i] = src[i] ^ out.b[i];@@ -271,14 +376,17 @@ 	if (bytes > 0) { 		/* generate new chunk and update state */ 		chacha_core(ctx->nb_rounds, &out, st);-		st->d[12] += 1;-		if (st->d[12] == 0)-			st->d[13] += 1;+		uint32_t t0 = le32_to_cpu(st->d[12]);+		st->d[12] = cpu_to_le32(t0 + 1);+		if (st->d[12] == 0) {+			uint32_t t1 = le32_to_cpu(st->d[13]);+			st->d[13] = cpu_to_le32(t1 + 1);+		}  		/* xor as much as needed */ 		for (i = 0; i < bytes; i++) 			dst[i] = src[i] ^ out.b[i];-		+ 		/* copy the left over in the buffer */ 		ctx->prev_len = 64 - bytes; 		ctx->prev_ofs = i;@@ -288,10 +396,45 @@ 	} } +uint64_t crypton_chacha_counter64(crypton_chacha_state *st)+{+	uint64_t result = ((uint64_t) le32_to_cpu(st->d[12]))+		| (((uint64_t) le32_to_cpu(st->d[13])) << 32);+	return result;+}++uint32_t crypton_chacha_counter32(crypton_chacha_state *st)+{+	return le32_to_cpu(st->d[12]);+}++void crypton_chacha_set_counter64(crypton_chacha_state *st, uint64_t block_counter)+{+	uint64_t current_counter;+	current_counter = ((uint64_t) le32_to_cpu(st->d[12]))+		| (((uint64_t) le32_to_cpu(st->d[13])) << 32);++	if (current_counter == block_counter)+		return;++	st->d[12] = cpu_to_le32((uint32_t) block_counter);+	st->d[13] = cpu_to_le32((uint32_t) (block_counter >> 32));+}++void crypton_chacha_set_counter32(crypton_chacha_state *st, uint32_t block_counter)+{+	uint32_t current_counter = le32_to_cpu(st->d[12]);++	if (current_counter == block_counter)+		return;++	st->d[12] = cpu_to_le32(block_counter);+}+ void crypton_chacha_generate(uint8_t *dst, crypton_chacha_context *ctx, uint32_t bytes) { 	crypton_chacha_state *st;-	block out;+	crypton_chacha_block out; 	int i;  	if (!bytes)@@ -314,23 +457,42 @@  	st = &ctx->st; +#ifdef CHACHA_SIMD+	{+		const uint32_t nb = (uint32_t) crypton_chacha_simd_width();+		const uint32_t step = 64 * nb;++		while (bytes >= step && CHACHA_SIMD_OK(st, nb)) {+			crypton_chacha_simd_generate(ctx->nb_rounds, dst, st);+			st->d[12] += nb;+			bytes -= step; dst += step;+		}+	}+#endif+ 	if (ALIGNED64(dst)) { 		/* xor new 64-bytes chunks and store the left over if any */ 		for (; bytes >= 64; bytes -= 64, dst += 64) { 			/* generate new chunk and update state */-			chacha_core(ctx->nb_rounds, (block *) dst, st);-			st->d[12] += 1;-			if (st->d[12] == 0)-				st->d[13] += 1;+			chacha_core(ctx->nb_rounds, (crypton_chacha_block *) dst, st);+			uint32_t t0 = le32_to_cpu(st->d[12]);+			st->d[12] = cpu_to_le32(t0 + 1);+			if (st->d[12] == 0) {+				uint32_t t1 = le32_to_cpu(st->d[13]);+				st->d[13] = cpu_to_le32(t1 + 1);+			} 		} 	} else { 		/* xor new 64-bytes chunks and store the left over if any */ 		for (; bytes >= 64; bytes -= 64, dst += 64) { 			/* generate new chunk and update state */ 			chacha_core(ctx->nb_rounds, &out, st);-			st->d[12] += 1;-			if (st->d[12] == 0)-				st->d[13] += 1;+			uint32_t t0 = le32_to_cpu(st->d[12]);+			st->d[12] = cpu_to_le32(t0 + 1);+			if (st->d[12] == 0) {+				uint32_t t1 = le32_to_cpu(st->d[13]);+				st->d[13] = cpu_to_le32(t1 + 1);+			}  			for (i = 0; i < 64; ++i) 				dst[i] = out.b[i];@@ -340,14 +502,17 @@ 	if (bytes > 0) { 		/* generate new chunk and update state */ 		chacha_core(ctx->nb_rounds, &out, st);-		st->d[12] += 1;-		if (st->d[12] == 0)-			st->d[13] += 1;+		uint32_t t0 = le32_to_cpu(st->d[12]);+		st->d[12] = cpu_to_le32(t0 + 1);+		if (st->d[12] == 0) {+			uint32_t t1 = le32_to_cpu(st->d[13]);+			st->d[13] = cpu_to_le32(t1 + 1);+		}  		/* xor as much as needed */ 		for (i = 0; i < bytes; i++) 			dst[i] = out.b[i];-		+ 		/* copy the left over in the buffer */ 		ctx->prev_len = 64 - bytes; 		ctx->prev_ofs = i;@@ -356,9 +521,30 @@ 	} } +void crypton_chacha_generate_simple_block(uint8_t *dst, crypton_chacha_state *st, uint8_t rounds)+{+	if (ALIGNED64(dst)) {+		chacha_core(rounds, (crypton_chacha_block *) dst, st);+	} else {+		crypton_chacha_block out;+		int i;+		chacha_core(rounds, &out, st);+		for (i = 0; i < 64; ++i) {+			dst[i] = out.b[i];+		}+	}++	uint32_t t0 = le32_to_cpu(st->d[12]);+	st->d[12] = cpu_to_le32(t0 + 1);+	if (st->d[12] == 0) {+		uint32_t t1 = le32_to_cpu(st->d[13]);+		st->d[13] = cpu_to_le32(t1 + 1);+	}+}+ void crypton_chacha_random(uint32_t rounds, uint8_t *dst, crypton_chacha_state *st, uint32_t bytes) {-	block out;+	crypton_chacha_block out;  	if (!bytes) 		return;
cbits/crypton_chacha.h view
@@ -34,9 +34,9 @@ 	uint64_t q[8]; 	uint32_t d[16]; 	uint8_t  b[64];-} block;+} crypton_chacha_block; -typedef block crypton_chacha_state;+typedef crypton_chacha_block crypton_chacha_state;  typedef struct { 	crypton_chacha_state st;@@ -51,5 +51,10 @@ void crypton_xchacha_init(crypton_chacha_context *ctx, uint8_t nb_rounds, const uint8_t *key, const uint8_t *iv); void crypton_chacha_combine(uint8_t *dst, crypton_chacha_context *st, const uint8_t *src, uint32_t bytes); void crypton_chacha_generate(uint8_t *dst, crypton_chacha_context *st, uint32_t bytes);-+uint64_t crypton_chacha_counter64(crypton_chacha_state *st);+uint32_t crypton_chacha_counter32(crypton_chacha_state *st);+void crypton_chacha_set_counter64(crypton_chacha_state *st, uint64_t block_counter);+void crypton_chacha_set_counter32(crypton_chacha_state *st, uint32_t block_counter);+void crypton_chacha_generate_simple_block(uint8_t *dst, crypton_chacha_state *st, uint8_t rounds);+#define crypton_chacha_get_state(context) (&((crypton_chacha_context *) context)->st) #endif
+ cbits/crypton_chachapoly.c view
@@ -0,0 +1,156 @@+/*+ * Copyright (c) 2026 Kazu Yamamoto+ *+ * Redistribution and use in source and binary forms, with or without+ * modification, are permitted provided that the following conditions+ * are met:+ * 1. Redistributions of source code must retain the above copyright+ *    notice, this list of conditions and the following disclaimer.+ * 2. Redistributions in binary form must reproduce the above copyright+ *    notice, this list of conditions and the following disclaimer in the+ *    documentation and/or other materials provided with the distribution.+ *+ * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE+ * POSSIBILITY OF SUCH DAMAGE.+ */++#include <stdint.h>+#include <string.h>++#include "crypton_chacha.h"+#include "crypton_chachapoly.h"+#include "crypton_poly1305.h"++/* RFC 8439.  The one-time Poly1305 key is the first 32 bytes of the ChaCha20+ * keystream at counter 0; a whole 64-byte block is generated so the counter+ * lands on 1, which is where the message starts. */+static void chachapoly_start(crypton_chacha_context *cctx, poly1305_ctx *pctx,+                             const uint8_t *key,+                             const uint8_t *nonce, uint32_t noncelen)+{+	uint8_t block[64];++	crypton_chacha_init(cctx, 20, 32, key, noncelen, nonce);+	crypton_chacha_generate(block, cctx, sizeof(block));+	crypton_poly1305_init(pctx, (poly1305_key *) block);+	memset(block, 0, sizeof(block));+}++/* Poly1305 over an associated or encrypted part, then zeros up to the next+ * multiple of sixteen. */+static void absorb_padded(poly1305_ctx *pctx, const uint8_t *p, uint32_t len)+{+	static const uint8_t zeros[16] = {0};+	uint32_t rem;++	if (len)+		crypton_poly1305_update(pctx, (uint8_t *) p, len);+	rem = len % 16;+	if (rem)+		crypton_poly1305_update(pctx, (uint8_t *) zeros, 16 - rem);+}++/* The two lengths, little endian, eight bytes each, which is what the tag+ * ends on. */+static void absorb_lengths(poly1305_ctx *pctx, uint32_t aadlen, uint32_t inlen)+{+	uint8_t lens[16];+	int i;++	for (i = 0; i < 8; i++)+		lens[i] = (uint8_t) (((uint64_t) aadlen) >> (8 * i));+	for (i = 0; i < 8; i++)+		lens[8 + i] = (uint8_t) (((uint64_t) inlen) >> (8 * i));+	crypton_poly1305_update(pctx, lens, sizeof(lens));+}++void crypton_chachapoly_encrypt(uint8_t *out, uint8_t *tag, uint32_t taglen,+                                const uint8_t *key,+                                const uint8_t *nonce, uint32_t noncelen,+                                const uint8_t *aad, uint32_t aadlen,+                                const uint8_t *input, uint32_t inlen)+{+	crypton_chacha_context cctx;+	poly1305_ctx pctx;+	poly1305_mac mac;++	chachapoly_start(&cctx, &pctx, key, nonce, noncelen);+	absorb_padded(&pctx, aad, aadlen);+	if (inlen)+		crypton_chacha_combine(out, &cctx, input, inlen);+	/* what the tag covers is the ciphertext, which is now in out */+	absorb_padded(&pctx, out, inlen);+	absorb_lengths(&pctx, aadlen, inlen);+	crypton_poly1305_finalize(mac, &pctx);+	memcpy(tag, mac, taglen);++	memset(&cctx, 0, sizeof(cctx));+	memset(&pctx, 0, sizeof(pctx));+}++/* Shared by the two decrypting entry points: with outtag NULL the tag is+ * compared here and the answer returned, otherwise it is written there. */+static int chachapoly_decrypt(uint8_t *out, const uint8_t *tag, uint32_t taglen,+                              uint8_t *outtag, const uint8_t *key,+                              const uint8_t *nonce, uint32_t noncelen,+                              const uint8_t *aad, uint32_t aadlen,+                              const uint8_t *input, uint32_t inlen)+{+	crypton_chacha_context cctx;+	poly1305_ctx pctx;+	poly1305_mac mac;+	uint8_t diff = 0;+	uint32_t i;++	chachapoly_start(&cctx, &pctx, key, nonce, noncelen);+	absorb_padded(&pctx, aad, aadlen);+	/* here the ciphertext is the input, so the tag can be taken before the+	 * plaintext is written and out may alias input */+	absorb_padded(&pctx, input, inlen);+	absorb_lengths(&pctx, aadlen, inlen);+	crypton_poly1305_finalize(mac, &pctx);++	if (inlen)+		crypton_chacha_combine(out, &cctx, input, inlen);++	memset(&cctx, 0, sizeof(cctx));+	memset(&pctx, 0, sizeof(pctx));++	if (outtag) {+		memcpy(outtag, mac, taglen);+		return 1;+	}+	for (i = 0; i < taglen; i++)+		diff |= (uint8_t) (mac[i] ^ tag[i]);+	return diff == 0;+}++int crypton_chachapoly_decrypt(uint8_t *out,+                               const uint8_t *tag, uint32_t taglen,+                               const uint8_t *key,+                               const uint8_t *nonce, uint32_t noncelen,+                               const uint8_t *aad, uint32_t aadlen,+                               const uint8_t *input, uint32_t inlen)+{+	return chachapoly_decrypt(out, tag, taglen, NULL, key, nonce, noncelen,+	                          aad, aadlen, input, inlen);+}++void crypton_chachapoly_decrypt_tag(uint8_t *out, uint8_t *outtag,+                                    uint32_t taglen, const uint8_t *key,+                                    const uint8_t *nonce, uint32_t noncelen,+                                    const uint8_t *aad, uint32_t aadlen,+                                    const uint8_t *input, uint32_t inlen)+{+	(void) chachapoly_decrypt(out, NULL, taglen, outtag, key, nonce,+	                          noncelen, aad, aadlen, input, inlen);+}
+ cbits/crypton_chachapoly.h view
@@ -0,0 +1,62 @@+/*+ * Copyright (c) 2026 Kazu Yamamoto+ *+ * Redistribution and use in source and binary forms, with or without+ * modification, are permitted provided that the following conditions+ * are met:+ * 1. Redistributions of source code must retain the above copyright+ *    notice, this list of conditions and the following disclaimer.+ * 2. Redistributions in binary form must reproduce the above copyright+ *    notice, this list of conditions and the following disclaimer in the+ *    documentation and/or other materials provided with the distribution.+ *+ * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE+ * POSSIBILITY OF SUCH DAMAGE.+ */++#ifndef CRYPTON_CHACHAPOLY_H+#define CRYPTON_CHACHAPOLY_H++#include <stdint.h>++/* ChaCha20-Poly1305 (RFC 8439) as one call.+ *+ * The pieces are the ChaCha20 and Poly1305 already here; what these do is+ * hold them together, which the Haskell above used to do at the cost of eight+ * foreign calls and the allocations between them.+ *+ * The nonce is the twelve bytes RFC 8439 defines.  taglen is at most 16.+ */++void crypton_chachapoly_encrypt(uint8_t *out, uint8_t *tag, uint32_t taglen,+                                const uint8_t *key,+                                const uint8_t *nonce, uint32_t noncelen,+                                const uint8_t *aad, uint32_t aadlen,+                                const uint8_t *input, uint32_t inlen);++/* Decrypt and compare, a byte at a time over the whole tag whichever way the+ * answer goes.  Returns non-zero when the tag matched. */+int crypton_chachapoly_decrypt(uint8_t *out,+                               const uint8_t *tag, uint32_t taglen,+                               const uint8_t *key,+                               const uint8_t *nonce, uint32_t noncelen,+                               const uint8_t *aad, uint32_t aadlen,+                               const uint8_t *input, uint32_t inlen);++/* Decrypt and hand the computed tag back rather than comparing it. */+void crypton_chachapoly_decrypt_tag(uint8_t *out, uint8_t *outtag,+                                    uint32_t taglen, const uint8_t *key,+                                    const uint8_t *nonce, uint32_t noncelen,+                                    const uint8_t *aad, uint32_t aadlen,+                                    const uint8_t *input, uint32_t inlen);++#endif
cbits/crypton_cpu.c view
@@ -31,6 +31,18 @@ #include "crypton_cpu.h" #include <stdint.h> +/*+ * The word the assembly reads; crypton_cpu.h says what is in it.  Hidden,+ * so that the reference to it from the assembly resolves at link time in a+ * shared object as well as a static one.  The SHA-256 bit is set by+ * cbits/crypton_sha256.c once it has asked whether the processor has those+ * instructions.+ */+#ifdef CRYPTON_ARM_ASM+__attribute__((visibility("hidden"))) unsigned int crypton_armcap_P =+    CRYPTON_ARMCAP_NEON;+#endif+ #ifdef ARCH_X86 static void cpuid(uint32_t info, uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx) {@@ -51,6 +63,217 @@ #endif 		 :"+a" (*eax), "=S" (*ebx), "=c" (*ecx), "=d" (*edx) 		 : :"edi");+}++/*+ * What the machine will let us use beyond the x86-64 baseline.  XGETBV is+ * spelled out in bytes because it predates some assemblers that are still+ * in use.+ */+static void cpuid_count(uint32_t info, uint32_t sub, uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx)+{+	*eax = info;+	*ecx = sub;+	__asm__ volatile+		(+#ifdef __x86_64__+		 "mov %%rbx, %%rdi;"+#else+		 "mov %%ebx, %%edi;"+#endif+		 "cpuid;"+		 "mov %%ebx, %%esi;"+#ifdef __x86_64__+		 "mov %%rdi, %%rbx;"+#else+		 "mov %%edi, %%ebx;"+#endif+		 :"+a" (*eax), "=S" (*ebx), "+c" (*ecx), "=d" (*edx)+		 : :"edi");+}++static uint64_t xcr0(void)+{+	uint32_t lo, hi;++	__asm__ volatile(".byte 0x0f, 0x01, 0xd0" : "=a" (lo), "=d" (hi) : "c" (0));+	return ((uint64_t) hi << 32) | lo;+}++#ifdef CRYPTON_X86_ASM+__attribute__((visibility("hidden"))) unsigned int crypton_ia32cap_P[4];++/*+ * The AVX-512 bits of leaf 7 EBX -- F, DQ, IFMA, PF, ER, CD, BW and VL,+ * which is every bit from 16 up except 21's neighbours and 29, the SHA+ * extensions, which are not AVX-512 and are wanted.  They are cleared+ * whatever the processor says: the code they would select in the vendored+ * assembly cannot be run, let alone measured, on any machine here, and+ * shipping a path nothing has executed is not worth the few per cent it+ * might be worth.  Turning them on is a one-line change for whoever has+ * the hardware.+ */+#define IA32CAP_AVX512 \+	((1u << 16) | (1u << 17) | (1u << 21) | (1u << 26) | (1u << 27) \+	 | (1u << 28) | (1u << 30) | (1u << 31))++/*+ * cpuid as the assembly reads it, with the two bits it dispatches on -- AVX+ * in leaf 1 and AVX2 in leaf 7 -- left set only where the answer already+ * agreed that the operating system saves the registers.  Two threads racing+ * here write the same values.+ */+void crypton_x86_ia32cap_resolve(void)+{+	static int resolved = 0;++	if (!resolved) {+		uint32_t eax, ebx, ecx, edx, maxleaf;+		uint32_t f = crypton_x86_simd_features();+		uint32_t leaf1_ecx, leaf7_ebx = 0;+		int intel;++		cpuid(0, &eax, &ebx, &ecx, &edx);+		maxleaf = eax;+		/* "GenuineIntel", which OpenSSL records in a bit of leaf 1+		 * EDX that cpuid leaves reserved: some of the assembly asks,+		 * having found a path worth taking on one make and not the+		 * other */+		intel = (ebx == 0x756e6547 && edx == 0x49656e69+		         && ecx == 0x6c65746e);++		cpuid(1, &eax, &ebx, &ecx, &edx);+		crypton_ia32cap_P[0] = intel ? (edx | (1u << 30)) : edx;+		leaf1_ecx = ecx;+		if (!(f & CRYPTON_X86_AVX))+			leaf1_ecx &= ~(1u << 28);+		/*+		 * Bit 11 is not leaf 1's to give.  The assembly reads it as+		 * AMD's XOP, which lives in leaf 0x80000001, and OpenSSL+		 * clears whatever leaf 1 put there before merging the real+		 * flag into the place -- on Intel that is SDBG, the silicon+		 * debug interface, reported since Broadwell, and reading it+		 * as XOP sends SHA-512 and ChaCha20 into a vprotq and a+		 * SIGILL.  It is cleared and left clear: nothing here can run+		 * XOP to test it, and no processor still in service has it,+		 * AMD having carried it from Bulldozer to Excavator and Zen+		 * having dropped it.  That is the reason the AVX-512 bits+		 * above are cleared too.+		 */+		leaf1_ecx &= ~(1u << 11);+		crypton_ia32cap_P[1] = leaf1_ecx;++		if (maxleaf >= 7) {+			cpuid_count(7, 0, &eax, &ebx, &ecx, &edx);+			leaf7_ebx = ebx;+		}+		if (!(f & CRYPTON_X86_AVX2))+			leaf7_ebx &= ~(1u << 5);+		crypton_ia32cap_P[2] = leaf7_ebx & ~IA32CAP_AVX512;++		resolved = 1;+	}+}+#endif++uint32_t crypton_x86_simd_features(void)+{+	static int resolved = 0;+	static uint32_t features = 0;++	if (!resolved) {+		uint32_t eax, ebx, ecx, edx, leaf1, maxleaf, family, f = 0;+		int amd;++		cpuid(0, &eax, &ebx, &ecx, &edx);+		maxleaf = eax;+		/* "AuthenticAMD" arrives as EBX, EDX, ECX in that order */+		amd = (ebx == 0x68747541 && edx == 0x69746e65+		       && ecx == 0x444d4163);++		cpuid(1, &eax, &ebx, &ecx, &edx);+		leaf1 = ecx;+		/* the family is the base one, and the extended field is+		 * added to it only when the base reads 0xf, which is how+		 * every AMD Zen part reports */+		family = (eax >> 8) & 0xf;+		if (family == 0xf)+			family += (eax >> 20) & 0xff;+		if (leaf1 & (1 << 9))+			f |= CRYPTON_X86_SSSE3;+		if (leaf1 & (1 << 1))+			f |= CRYPTON_X86_PCLMUL;+		if (leaf1 & (1 << 22))+			f |= CRYPTON_X86_MOVBE;+		/* AVX asks the same three things as AVX2 below: the+		 * processor has it, OSXSAVE is on, and the operating system+		 * says it saves the registers */+		if ((leaf1 & (1 << 28)) && (leaf1 & (1 << 27))+		    && ((xcr0() & 6) == 6))+			f |= CRYPTON_X86_AVX;++		/* leaf 7 answers for both of the rest, and a processor that+		 * does not have it answers for the highest leaf it does have+		 * instead, so ask what that is first */+		if (maxleaf >= 7) {+			cpuid_count(7, 0, &eax, &ebx, &ecx, &edx);+			/* the SHA extensions work in registers the SSE state+			 * already covers, so they need nothing of the+			 * operating system.  The code that uses them also+			 * wants SSSE3 and SSE4.1, which every processor that+			 * has them has, but ask rather than assume */+			if ((ebx & (1 << 29)) && (leaf1 & (1 << 9))+			    && (leaf1 & (1 << 19)))+				f |= CRYPTON_X86_SHA_NI;+			/* AVX2 has the wider registers, which takes three+			 * things agreeing: the CPU has it, OSXSAVE is on, and+			 * XCR0 says the operating system saves them --+			 * without that last one the upper halves are lost+			 * across a context switch */+			if ((ebx & (1 << 5)) && (leaf1 & (1 << 27))+			    && (leaf1 & (1 << 28)) && ((xcr0() & 6) == 6))+				f |= CRYPTON_X86_AVX2;+			/* BMI2 for MULX and ADX for ADCX/ADOX.  Both are+			 * wanted together and neither touches vector state,+			 * so there is nothing to ask the operating system */+			if ((ebx & (1 << 8)) && (ebx & (1 << 19)))+				f |= CRYPTON_X86_ADX;+			/* VAES and VPCLMULQDQ, leaf 7 ECX bits 9 and 10.+			 * They are wanted together -- one without the other+			 * leaves half of AES-GCM narrow -- and they need the+			 * wide registers, so AVX2 has to have answered first,+			 * which settles the operating system's part. */+			if ((ecx & (1 << 9)) && (ecx & (1 << 10))+			    && (f & CRYPTON_X86_AVX2))+				f |= CRYPTON_X86_VAES;+			/* The same two instructions in their 512-bit form,+			 * which wants AVX-512 F, BW and VL as well -- and+			 * three more bits of XCR0, for the mask registers and+			 * the two upper halves of the vector state.  A+			 * machine can report the instructions and still fault+			 * on them when the operating system has not said it+			 * saves that state, which is what those bits are. */+			if ((f & CRYPTON_X86_VAES)+			    && (ebx & (1 << 16)) && (ebx & (1u << 30))+			    && (ebx & (1u << 31))+			    && ((xcr0() & 0xe6) == 0xe6)+			    /* Not on Zen 4, which is AMD family 19h with+			     * AVX-512: there the 512-bit instructions are two+			     * 256-bit passes through a 256-bit datapath, so+			     * they carry the wider encoding for none of the+			     * throughput, and AES-GCM measures 0.6 to 3+			     * per cent slower than the 256-bit path.  Zen 5+			     * is family 1Ah and does have the wide datapath,+			     * where the same code is half as fast again;+			     * Zen 3, the other family 19h part, has no+			     * AVX-512 at all and never reaches here. */+			    && !(amd && family == 0x19))+				f |= CRYPTON_X86_VAES512;+		}+		features = f;+		resolved = 1;+	}+	return features; }  #ifdef USE_AESNI
cbits/crypton_cpu.h view
@@ -31,9 +31,71 @@ #ifndef CPU_H #define CPU_H +#include <stdint.h>+ #if defined(__i386__) || defined(__x86_64__) #define ARCH_X86 #define USE_AESNI+#endif++/* vector extensions beyond the x86-64 baseline, as cpuid reports them and+ * the OS allows them */+#define CRYPTON_X86_SSSE3  1+#define CRYPTON_X86_AVX2   2+#define CRYPTON_X86_PCLMUL 4+/* the SHA extensions, and the SSSE3 and SSE4.1 the code around them uses */+#define CRYPTON_X86_SHA_NI 8+/* the 128-bit half of AVX, which is what the vendored assembly is written+ * in, and the byte-swapping load it reads the message with */+#define CRYPTON_X86_AVX    16+#define CRYPTON_X86_MOVBE  32+/* MULX, ADCX and ADOX together: the two independent carry chains the+ * vendored s2n-bignum assembly wants.  They are general-purpose register+ * instructions, so unlike the vector ones above they ask nothing of the+ * operating system. */+#define CRYPTON_X86_ADX    64+/* The AES and carry-less multiply instructions in their 256-bit form, which+ * do two blocks where the 128-bit ones do one.  They are VEX-encoded and use+ * the vector registers AVX2 already needs the operating system to save, so+ * they ask nothing further of it -- but AVX2 itself is asked about, since+ * without it there is nowhere to put them. */+#define CRYPTON_X86_VAES   128+/* The same pair in their 512-bit form, four blocks to an instruction.  These+ * are EVEX-encoded and need the AVX-512 state as well, which is three more+ * bits of XCR0 than AVX2 wants: the mask registers and the two upper halves+ * of the vector registers. */+#define CRYPTON_X86_VAES512 256+#ifdef ARCH_X86+uint32_t crypton_x86_simd_features(void);+#endif++/*+ * What the vendored AArch64 assembly asks about the processor, in the way+ * OpenSSL asks it and with OpenSSL's bit numbering.  NEON is not optional+ * on AArch64 and is set from the start; the SHA-256 instructions are, so+ * the bit for them is set once the runtime check has answered.  See+ * cbits/crypton_cpu.c and cbits/asm/README.md.+ */+/*+ * And what the vendored x86-64 assembly asks, which is cpuid's own words in+ * the order OpenSSL keeps them: [0] is leaf 1 EDX, [1] leaf 1 ECX and [2]+ * leaf 7 EBX, with the bits for what the operating system will not preserve+ * cleared.  Filled on first use; see cbits/crypton_cpu.c.+ */+#if defined(WITH_X86_POLY1305_ASM) || defined(WITH_X86_CHACHA_ASM) \+    || defined(WITH_X86_SHA256_ASM) || defined(WITH_X86_SHA512_ASM)+#define CRYPTON_X86_ASM 1+extern unsigned int crypton_ia32cap_P[4];+void crypton_x86_ia32cap_resolve(void);+#endif++#if defined(WITH_ARMV8_CHACHA_ASM) || defined(WITH_ARMV8_POLY1305_ASM) \+    || defined(WITH_ARMV8_SHA1_ASM) || defined(WITH_ARMV8_SHA256_ASM)+#define CRYPTON_ARM_ASM 1+#define CRYPTON_ARMCAP_NEON   1+#define CRYPTON_ARMCAP_SHA1   (1 << 3)+#define CRYPTON_ARMCAP_SHA256 (1 << 4)+extern unsigned int crypton_armcap_P; #endif  #ifdef USE_AESNI
+ cbits/crypton_des.c view
@@ -0,0 +1,1325 @@+/*+ * DES, as FIPS 46-3 defines it.+ *+ * The tables below are generated from the permutations and S-boxes of that+ * standard: SP[i] combines S-box i with the P permutation, IPL/IPR and FPH/FPL+ * apply the initial and final permutations one input byte at a time, and the+ * 48-bit round key is kept as eight six-bit values so that the E expansion is+ * a rotate and a shift rather than a table.+ *+ * DES is here because callers still meet it, not because it should be chosen:+ * its 56-bit key is exhaustible, and this implementation indexes tables with+ * key-dependent values, so it is not constant time.+ */+#include <stdint.h>+#include <string.h>+#include <crypton_des.h>++static const uint32_t SP[8][64] = {+{+	0x00808200U, 0x00000000U, 0x00008000U, 0x00808202U, 0x00808002U, 0x00008202U, 0x00000002U, 0x00008000U,+	0x00000200U, 0x00808200U, 0x00808202U, 0x00000200U, 0x00800202U, 0x00808002U, 0x00800000U, 0x00000002U,+	0x00000202U, 0x00800200U, 0x00800200U, 0x00008200U, 0x00008200U, 0x00808000U, 0x00808000U, 0x00800202U,+	0x00008002U, 0x00800002U, 0x00800002U, 0x00008002U, 0x00000000U, 0x00000202U, 0x00008202U, 0x00800000U,+	0x00008000U, 0x00808202U, 0x00000002U, 0x00808000U, 0x00808200U, 0x00800000U, 0x00800000U, 0x00000200U,+	0x00808002U, 0x00008000U, 0x00008200U, 0x00800002U, 0x00000200U, 0x00000002U, 0x00800202U, 0x00008202U,+	0x00808202U, 0x00008002U, 0x00808000U, 0x00800202U, 0x00800002U, 0x00000202U, 0x00008202U, 0x00808200U,+	0x00000202U, 0x00800200U, 0x00800200U, 0x00000000U, 0x00008002U, 0x00008200U, 0x00000000U, 0x00808002U,+},+{+	0x40084010U, 0x40004000U, 0x00004000U, 0x00084010U, 0x00080000U, 0x00000010U, 0x40080010U, 0x40004010U,+	0x40000010U, 0x40084010U, 0x40084000U, 0x40000000U, 0x40004000U, 0x00080000U, 0x00000010U, 0x40080010U,+	0x00084000U, 0x00080010U, 0x40004010U, 0x00000000U, 0x40000000U, 0x00004000U, 0x00084010U, 0x40080000U,+	0x00080010U, 0x40000010U, 0x00000000U, 0x00084000U, 0x00004010U, 0x40084000U, 0x40080000U, 0x00004010U,+	0x00000000U, 0x00084010U, 0x40080010U, 0x00080000U, 0x40004010U, 0x40080000U, 0x40084000U, 0x00004000U,+	0x40080000U, 0x40004000U, 0x00000010U, 0x40084010U, 0x00084010U, 0x00000010U, 0x00004000U, 0x40000000U,+	0x00004010U, 0x40084000U, 0x00080000U, 0x40000010U, 0x00080010U, 0x40004010U, 0x40000010U, 0x00080010U,+	0x00084000U, 0x00000000U, 0x40004000U, 0x00004010U, 0x40000000U, 0x40080010U, 0x40084010U, 0x00084000U,+},+{+	0x00000104U, 0x04010100U, 0x00000000U, 0x04010004U, 0x04000100U, 0x00000000U, 0x00010104U, 0x04000100U,+	0x00010004U, 0x04000004U, 0x04000004U, 0x00010000U, 0x04010104U, 0x00010004U, 0x04010000U, 0x00000104U,+	0x04000000U, 0x00000004U, 0x04010100U, 0x00000100U, 0x00010100U, 0x04010000U, 0x04010004U, 0x00010104U,+	0x04000104U, 0x00010100U, 0x00010000U, 0x04000104U, 0x00000004U, 0x04010104U, 0x00000100U, 0x04000000U,+	0x04010100U, 0x04000000U, 0x00010004U, 0x00000104U, 0x00010000U, 0x04010100U, 0x04000100U, 0x00000000U,+	0x00000100U, 0x00010004U, 0x04010104U, 0x04000100U, 0x04000004U, 0x00000100U, 0x00000000U, 0x04010004U,+	0x04000104U, 0x00010000U, 0x04000000U, 0x04010104U, 0x00000004U, 0x00010104U, 0x00010100U, 0x04000004U,+	0x04010000U, 0x04000104U, 0x00000104U, 0x04010000U, 0x00010104U, 0x00000004U, 0x04010004U, 0x00010100U,+},+{+	0x80401000U, 0x80001040U, 0x80001040U, 0x00000040U, 0x00401040U, 0x80400040U, 0x80400000U, 0x80001000U,+	0x00000000U, 0x00401000U, 0x00401000U, 0x80401040U, 0x80000040U, 0x00000000U, 0x00400040U, 0x80400000U,+	0x80000000U, 0x00001000U, 0x00400000U, 0x80401000U, 0x00000040U, 0x00400000U, 0x80001000U, 0x00001040U,+	0x80400040U, 0x80000000U, 0x00001040U, 0x00400040U, 0x00001000U, 0x00401040U, 0x80401040U, 0x80000040U,+	0x00400040U, 0x80400000U, 0x00401000U, 0x80401040U, 0x80000040U, 0x00000000U, 0x00000000U, 0x00401000U,+	0x00001040U, 0x00400040U, 0x80400040U, 0x80000000U, 0x80401000U, 0x80001040U, 0x80001040U, 0x00000040U,+	0x80401040U, 0x80000040U, 0x80000000U, 0x00001000U, 0x80400000U, 0x80001000U, 0x00401040U, 0x80400040U,+	0x80001000U, 0x00001040U, 0x00400000U, 0x80401000U, 0x00000040U, 0x00400000U, 0x00001000U, 0x00401040U,+},+{+	0x00000080U, 0x01040080U, 0x01040000U, 0x21000080U, 0x00040000U, 0x00000080U, 0x20000000U, 0x01040000U,+	0x20040080U, 0x00040000U, 0x01000080U, 0x20040080U, 0x21000080U, 0x21040000U, 0x00040080U, 0x20000000U,+	0x01000000U, 0x20040000U, 0x20040000U, 0x00000000U, 0x20000080U, 0x21040080U, 0x21040080U, 0x01000080U,+	0x21040000U, 0x20000080U, 0x00000000U, 0x21000000U, 0x01040080U, 0x01000000U, 0x21000000U, 0x00040080U,+	0x00040000U, 0x21000080U, 0x00000080U, 0x01000000U, 0x20000000U, 0x01040000U, 0x21000080U, 0x20040080U,+	0x01000080U, 0x20000000U, 0x21040000U, 0x01040080U, 0x20040080U, 0x00000080U, 0x01000000U, 0x21040000U,+	0x21040080U, 0x00040080U, 0x21000000U, 0x21040080U, 0x01040000U, 0x00000000U, 0x20040000U, 0x21000000U,+	0x00040080U, 0x01000080U, 0x20000080U, 0x00040000U, 0x00000000U, 0x20040000U, 0x01040080U, 0x20000080U,+},+{+	0x10000008U, 0x10200000U, 0x00002000U, 0x10202008U, 0x10200000U, 0x00000008U, 0x10202008U, 0x00200000U,+	0x10002000U, 0x00202008U, 0x00200000U, 0x10000008U, 0x00200008U, 0x10002000U, 0x10000000U, 0x00002008U,+	0x00000000U, 0x00200008U, 0x10002008U, 0x00002000U, 0x00202000U, 0x10002008U, 0x00000008U, 0x10200008U,+	0x10200008U, 0x00000000U, 0x00202008U, 0x10202000U, 0x00002008U, 0x00202000U, 0x10202000U, 0x10000000U,+	0x10002000U, 0x00000008U, 0x10200008U, 0x00202000U, 0x10202008U, 0x00200000U, 0x00002008U, 0x10000008U,+	0x00200000U, 0x10002000U, 0x10000000U, 0x00002008U, 0x10000008U, 0x10202008U, 0x00202000U, 0x10200000U,+	0x00202008U, 0x10202000U, 0x00000000U, 0x10200008U, 0x00000008U, 0x00002000U, 0x10200000U, 0x00202008U,+	0x00002000U, 0x00200008U, 0x10002008U, 0x00000000U, 0x10202000U, 0x10000000U, 0x00200008U, 0x10002008U,+},+{+	0x00100000U, 0x02100001U, 0x02000401U, 0x00000000U, 0x00000400U, 0x02000401U, 0x00100401U, 0x02100400U,+	0x02100401U, 0x00100000U, 0x00000000U, 0x02000001U, 0x00000001U, 0x02000000U, 0x02100001U, 0x00000401U,+	0x02000400U, 0x00100401U, 0x00100001U, 0x02000400U, 0x02000001U, 0x02100000U, 0x02100400U, 0x00100001U,+	0x02100000U, 0x00000400U, 0x00000401U, 0x02100401U, 0x00100400U, 0x00000001U, 0x02000000U, 0x00100400U,+	0x02000000U, 0x00100400U, 0x00100000U, 0x02000401U, 0x02000401U, 0x02100001U, 0x02100001U, 0x00000001U,+	0x00100001U, 0x02000000U, 0x02000400U, 0x00100000U, 0x02100400U, 0x00000401U, 0x00100401U, 0x02100400U,+	0x00000401U, 0x02000001U, 0x02100401U, 0x02100000U, 0x00100400U, 0x00000000U, 0x00000001U, 0x02100401U,+	0x00000000U, 0x00100401U, 0x02100000U, 0x00000400U, 0x02000001U, 0x02000400U, 0x00000400U, 0x00100001U,+},+{+	0x08000820U, 0x00000800U, 0x00020000U, 0x08020820U, 0x08000000U, 0x08000820U, 0x00000020U, 0x08000000U,+	0x00020020U, 0x08020000U, 0x08020820U, 0x00020800U, 0x08020800U, 0x00020820U, 0x00000800U, 0x00000020U,+	0x08020000U, 0x08000020U, 0x08000800U, 0x00000820U, 0x00020800U, 0x00020020U, 0x08020020U, 0x08020800U,+	0x00000820U, 0x00000000U, 0x00000000U, 0x08020020U, 0x08000020U, 0x08000800U, 0x00020820U, 0x00020000U,+	0x00020820U, 0x00020000U, 0x08020800U, 0x00000800U, 0x00000020U, 0x08020020U, 0x00000800U, 0x00020820U,+	0x08000800U, 0x00000020U, 0x08000020U, 0x08020000U, 0x08020020U, 0x08000000U, 0x00020000U, 0x08000820U,+	0x00000000U, 0x08020820U, 0x00020020U, 0x08000020U, 0x08020000U, 0x08000800U, 0x08000820U, 0x00000000U,+	0x08020820U, 0x00020800U, 0x00020800U, 0x00000820U, 0x00000820U, 0x00020020U, 0x08000000U, 0x08020800U,+},+};++static const uint32_t IPL[8][256] = {+{+	0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U,+	0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U,+	0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U,+	0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U,+	0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U,+	0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U,+	0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U,+	0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U,+	0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U,+	0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U,+	0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U,+	0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U,+	0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U,+	0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U,+	0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U,+	0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U,+	0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U,+	0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U,+	0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U,+	0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U,+	0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U,+	0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U,+	0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U,+	0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U,+	0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U,+	0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U,+	0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U,+	0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U,+	0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U,+	0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U,+	0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U,+	0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U,+},+{+	0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U,+	0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U,+	0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U,+	0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U,+	0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U,+	0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U,+	0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U,+	0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U,+	0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U,+	0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U,+	0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U,+	0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U,+	0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U,+	0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U,+	0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U,+	0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U,+	0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U,+	0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U,+	0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U,+	0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U,+	0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U,+	0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U,+	0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U,+	0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U,+	0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U,+	0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U,+	0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U,+	0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U,+	0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U,+	0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U,+	0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U,+	0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U,+},+{+	0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U,+	0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U,+	0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U,+	0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U,+	0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U,+	0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U,+	0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U,+	0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U,+	0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U,+	0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U,+	0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U,+	0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U,+	0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U,+	0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U,+	0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U,+	0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U,+	0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U,+	0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U,+	0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U,+	0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U,+	0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U,+	0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U,+	0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U,+	0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U,+	0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U,+	0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U,+	0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U,+	0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U,+	0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U,+	0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U,+	0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U,+	0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U,+},+{+	0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U,+	0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U,+	0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U,+	0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U,+	0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U,+	0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U,+	0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U,+	0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U,+	0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U,+	0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U,+	0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U,+	0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U,+	0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U,+	0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U,+	0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U,+	0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U,+	0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U,+	0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U,+	0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U,+	0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U,+	0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U,+	0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U,+	0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U,+	0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U,+	0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U,+	0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U,+	0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U,+	0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U,+	0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U,+	0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U,+	0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U,+	0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U,+},+{+	0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U,+	0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U,+	0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U,+	0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U,+	0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U,+	0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U,+	0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U,+	0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U,+	0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U,+	0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U,+	0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U,+	0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U,+	0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U,+	0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U,+	0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U,+	0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U,+	0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U,+	0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U,+	0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U,+	0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U,+	0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U,+	0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U,+	0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U,+	0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U,+	0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U,+	0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U,+	0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U,+	0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U,+	0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U,+	0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U,+	0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U,+	0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U,+},+{+	0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U,+	0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U,+	0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U,+	0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U,+	0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U,+	0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U,+	0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U,+	0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U,+	0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U,+	0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U,+	0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U,+	0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U,+	0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U,+	0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U,+	0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U,+	0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U,+	0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U,+	0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U,+	0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U,+	0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U,+	0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U,+	0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U,+	0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U,+	0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U,+	0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U,+	0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U,+	0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U,+	0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U,+	0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U,+	0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U,+	0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U,+	0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U,+},+{+	0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U,+	0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U,+	0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U,+	0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U,+	0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U,+	0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U,+	0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U,+	0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U,+	0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U,+	0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U,+	0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U,+	0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U,+	0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U,+	0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U,+	0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U,+	0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U,+	0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U,+	0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U,+	0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U,+	0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U,+	0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U,+	0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U,+	0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U,+	0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U,+	0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U,+	0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U,+	0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U,+	0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U,+	0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U,+	0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U,+	0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U,+	0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U,+},+{+	0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U,+	0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U,+	0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U,+	0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U,+	0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U,+	0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U,+	0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U,+	0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U,+	0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U,+	0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U,+	0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U,+	0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U,+	0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U,+	0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U,+	0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U,+	0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U,+	0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U,+	0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U,+	0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U,+	0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U,+	0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U,+	0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U,+	0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U,+	0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U,+	0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U,+	0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U,+	0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U,+	0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U,+	0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U,+	0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U,+	0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U,+	0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U,+},+};++static const uint32_t IPR[8][256] = {+{+	0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U, 0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U,+	0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U, 0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U,+	0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U, 0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U,+	0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U, 0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U,+	0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U, 0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U,+	0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U, 0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U,+	0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U, 0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U,+	0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U, 0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U,+	0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U, 0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U,+	0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U, 0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U,+	0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U, 0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U,+	0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U, 0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U,+	0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U, 0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U,+	0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U, 0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U,+	0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U, 0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U,+	0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U, 0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U,+	0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U, 0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U,+	0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U, 0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U,+	0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U, 0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U,+	0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U, 0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U,+	0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U, 0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U,+	0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U, 0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U,+	0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U, 0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U,+	0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U, 0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U,+	0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U, 0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U,+	0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U, 0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U,+	0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U, 0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U,+	0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U, 0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U,+	0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U, 0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U,+	0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U, 0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U,+	0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U, 0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U,+	0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U, 0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U,+},+{+	0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U, 0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U,+	0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U, 0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U,+	0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U, 0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U,+	0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U, 0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U,+	0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U, 0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U,+	0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U, 0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U,+	0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U, 0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U,+	0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U, 0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U,+	0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U, 0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U,+	0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U, 0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U,+	0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U, 0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U,+	0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U, 0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U,+	0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U, 0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U,+	0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U, 0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U,+	0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U, 0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U,+	0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U, 0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U,+	0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U, 0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U,+	0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U, 0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U,+	0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U, 0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U,+	0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U, 0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U,+	0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U, 0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U,+	0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U, 0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U,+	0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U, 0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U,+	0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U, 0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U,+	0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U, 0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U,+	0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U, 0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U,+	0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U, 0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U,+	0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U, 0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U,+	0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U, 0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U,+	0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U, 0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U,+	0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U, 0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U,+	0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U, 0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U,+},+{+	0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U, 0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U,+	0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U, 0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U,+	0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U, 0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U,+	0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U, 0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U,+	0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U, 0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U,+	0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U, 0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U,+	0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U, 0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U,+	0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U, 0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U,+	0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U, 0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U,+	0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U, 0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U,+	0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U, 0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U,+	0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U, 0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U,+	0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U, 0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U,+	0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U, 0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U,+	0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U, 0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U,+	0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U, 0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U,+	0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U, 0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U,+	0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U, 0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U,+	0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U, 0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U,+	0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U, 0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U,+	0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U, 0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U,+	0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U, 0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U,+	0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U, 0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U,+	0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U, 0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U,+	0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U, 0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U,+	0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U, 0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U,+	0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U, 0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U,+	0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U, 0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U,+	0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U, 0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U,+	0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U, 0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U,+	0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U, 0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U,+	0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U, 0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U,+},+{+	0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U, 0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U,+	0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U, 0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U,+	0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U, 0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U,+	0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U, 0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U,+	0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U, 0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U,+	0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U, 0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U,+	0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U, 0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U,+	0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U, 0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U,+	0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U, 0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U,+	0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U, 0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U,+	0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U, 0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U,+	0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U, 0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U,+	0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U, 0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U,+	0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U, 0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U,+	0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U, 0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U,+	0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U, 0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U,+	0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U, 0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U,+	0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U, 0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U,+	0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U, 0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U,+	0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U, 0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U,+	0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U, 0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U,+	0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U, 0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U,+	0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U, 0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U,+	0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U, 0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U,+	0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U, 0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U,+	0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U, 0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U,+	0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U, 0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U,+	0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U, 0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U,+	0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U, 0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U,+	0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U, 0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U,+	0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U, 0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U,+	0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U, 0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U,+},+{+	0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U, 0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U,+	0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U, 0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U,+	0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U, 0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U,+	0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U, 0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U,+	0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U, 0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U,+	0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U, 0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U,+	0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U, 0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U,+	0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U, 0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U,+	0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U, 0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U,+	0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U, 0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U,+	0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U, 0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U,+	0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U, 0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U,+	0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U, 0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U,+	0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U, 0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U,+	0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U, 0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U,+	0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U, 0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U,+	0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U, 0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U,+	0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U, 0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U,+	0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U, 0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U,+	0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U, 0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U,+	0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U, 0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U,+	0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U, 0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U,+	0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U, 0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U,+	0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U, 0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U,+	0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U, 0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U,+	0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U, 0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U,+	0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U, 0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U,+	0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U, 0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U,+	0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U, 0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U,+	0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U, 0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U,+	0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U, 0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U,+	0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U, 0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U,+},+{+	0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U, 0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U,+	0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U, 0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U,+	0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U, 0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U,+	0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U, 0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U,+	0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U, 0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U,+	0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U, 0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U,+	0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U, 0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U,+	0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U, 0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U,+	0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U, 0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U,+	0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U, 0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U,+	0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U, 0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U,+	0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U, 0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U,+	0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U, 0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U,+	0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U, 0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U,+	0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U, 0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U,+	0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U, 0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U,+	0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U, 0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U,+	0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U, 0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U,+	0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U, 0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U,+	0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U, 0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U,+	0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U, 0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U,+	0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U, 0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U,+	0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U, 0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U,+	0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U, 0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U,+	0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U, 0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U,+	0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U, 0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U,+	0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U, 0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U,+	0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U, 0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U,+	0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U, 0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U,+	0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U, 0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U,+	0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U, 0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U,+	0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U, 0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U,+},+{+	0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U, 0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U,+	0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U, 0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U,+	0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U, 0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U,+	0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U, 0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U,+	0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U, 0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U,+	0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U, 0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U,+	0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U, 0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U,+	0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U, 0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U,+	0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U, 0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U,+	0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U, 0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U,+	0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U, 0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U,+	0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U, 0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U,+	0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U, 0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U,+	0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U, 0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U,+	0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U, 0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U,+	0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U, 0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U,+	0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U, 0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U,+	0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U, 0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U,+	0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U, 0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U,+	0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U, 0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U,+	0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U, 0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U,+	0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U, 0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U,+	0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U, 0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U,+	0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U, 0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U,+	0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U, 0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U,+	0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U, 0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U,+	0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U, 0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U,+	0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U, 0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U,+	0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U, 0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U,+	0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U, 0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U,+	0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U, 0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U,+	0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U, 0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U,+},+{+	0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U, 0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U,+	0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U, 0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U,+	0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U, 0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U,+	0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U, 0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U,+	0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U, 0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U,+	0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U, 0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U,+	0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U, 0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U,+	0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U, 0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U,+	0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U, 0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U,+	0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U, 0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U,+	0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U, 0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U,+	0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U, 0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U,+	0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U, 0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U,+	0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U, 0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U,+	0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U, 0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U,+	0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U, 0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U,+	0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U, 0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U,+	0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U, 0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U,+	0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U, 0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U,+	0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U, 0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U,+	0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U, 0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U,+	0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U, 0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U,+	0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U, 0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U,+	0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U, 0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U,+	0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U, 0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U,+	0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U, 0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U,+	0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U, 0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U,+	0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U, 0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U,+	0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U, 0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U,+	0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U, 0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U,+	0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U, 0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U,+	0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U, 0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U,+},+};++static const uint32_t FPH[8][256] = {+{+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+	0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U,+	0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U,+},+{+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+	0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U,+	0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U,+},+{+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+	0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U,+	0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U,+},+{+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+	0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U,+	0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U,+},+{+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+	0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U,+	0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U,+},+{+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+	0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U,+	0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U,+},+{+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+	0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U,+	0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U,+},+{+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+	0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U,+	0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U,+},+};++static const uint32_t FPL[8][256] = {+{+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U,+	0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U,+	0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U,+	0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U,+	0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U,+	0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U,+	0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U,+	0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U,+	0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U,+	0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U,+	0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U,+	0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U,+	0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U,+	0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U,+	0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U,+	0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U,+	0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U,+	0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U,+	0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U,+	0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U,+	0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U,+	0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U,+	0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U,+	0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U,+	0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U,+	0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U,+	0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U,+	0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U,+	0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U,+	0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U,+},+{+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U,+	0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U,+	0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U,+	0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U,+	0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U,+	0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U,+	0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U,+	0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U,+	0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U,+	0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U,+	0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U,+	0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U,+	0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U,+	0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U,+	0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U,+	0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U,+	0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U,+	0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U,+	0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U,+	0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U,+	0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U,+	0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U,+	0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U,+	0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U,+	0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U,+	0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U,+	0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U,+	0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U,+	0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U,+	0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U,+},+{+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U,+	0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U,+	0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U,+	0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U,+	0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U,+	0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U,+	0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U,+	0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U,+	0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U,+	0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U,+	0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U,+	0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U,+	0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U,+	0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U,+	0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U,+	0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U,+	0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U,+	0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U,+	0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U,+	0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U,+	0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U,+	0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U,+	0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U,+	0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U,+	0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U,+	0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U,+	0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U,+	0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U,+	0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U,+	0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U,+},+{+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U,+	0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U,+	0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U,+	0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U,+	0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U,+	0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U,+	0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U,+	0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U,+	0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U,+	0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U,+	0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U,+	0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U,+	0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U,+	0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U,+	0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U,+	0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U,+	0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U,+	0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U,+	0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U,+	0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U,+	0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U,+	0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U,+	0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U,+	0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U,+	0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U,+	0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U,+	0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U,+	0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U,+	0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U,+	0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U,+},+{+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U,+	0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U,+	0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U,+	0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U,+	0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U,+	0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U,+	0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U,+	0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U,+	0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U,+	0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U,+	0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U,+	0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U,+	0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U,+	0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U,+	0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U,+	0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U,+	0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U,+	0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U,+	0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U,+	0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U,+	0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U,+	0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U,+	0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U,+	0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U,+	0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U,+	0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U,+	0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U,+	0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U,+	0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U,+	0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U,+},+{+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U,+	0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U,+	0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U,+	0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U,+	0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U,+	0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U,+	0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U,+	0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U,+	0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U,+	0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U,+	0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U,+	0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U,+	0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U,+	0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U,+	0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U,+	0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U,+	0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U,+	0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U,+	0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U,+	0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U,+	0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U,+	0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U,+	0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U,+	0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U,+	0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U,+	0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U,+	0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U,+	0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U,+	0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U,+	0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U,+},+{+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U,+	0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U,+	0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U,+	0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U,+	0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U,+	0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U,+	0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U,+	0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U,+	0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U,+	0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U,+	0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U,+	0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U,+	0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U,+	0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U,+	0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U,+	0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U,+	0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U,+	0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U,+	0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U,+	0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U,+	0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U,+	0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U,+	0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U,+	0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U,+	0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U,+	0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U,+	0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U,+	0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U,+	0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U,+	0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U,+},+{+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U,+	0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U,+	0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U,+	0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U,+	0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U,+	0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U,+	0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U,+	0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U,+	0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U,+	0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U,+	0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U,+	0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U,+	0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U,+	0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U,+	0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U,+	0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U,+	0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U,+	0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U,+	0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U,+	0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U,+	0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U,+	0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U,+	0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U,+	0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U,+	0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U,+	0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U,+	0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U,+	0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U,+	0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U,+	0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U,+	0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U,+},+};++#define ROTL32(v, k) (((v) << (k)) | ((v) >> (32 - (k))))++static inline uint64_t load_be64(const uint8_t *p)+{+	return ((uint64_t) p[0] << 56) | ((uint64_t) p[1] << 48)+	     | ((uint64_t) p[2] << 40) | ((uint64_t) p[3] << 32)+	     | ((uint64_t) p[4] << 24) | ((uint64_t) p[5] << 16)+	     | ((uint64_t) p[6] << 8)  | ((uint64_t) p[7]);+}++static inline void store_be64(uint8_t *p, uint64_t v)+{+	p[0] = (uint8_t) (v >> 56); p[1] = (uint8_t) (v >> 48);+	p[2] = (uint8_t) (v >> 40); p[3] = (uint8_t) (v >> 32);+	p[4] = (uint8_t) (v >> 24); p[5] = (uint8_t) (v >> 16);+	p[6] = (uint8_t) (v >> 8);  p[7] = (uint8_t) v;+}++/* bit i of v, numbered from 1 at the most significant end, as FIPS 46-3 does */+static inline uint32_t bit_of(uint64_t v, int i, int width)+{+	return (uint32_t) ((v >> (width - i)) & 1);+}++static const uint8_t PC1[56] = {+	57,49,41,33,25,17,9,1,58,50,42,34,26,18,10,2,59,51,43,35,27,+	19,11,3,60,52,44,36,63,55,47,39,31,23,15,7,62,54,46,38,30,22,+	14,6,61,53,45,37,29,21,13,5,28,20,12,4+};++static const uint8_t PC2[48] = {+	14,17,11,24,1,5,3,28,15,6,21,10,23,19,12,4,26,8,16,7,27,20,13,2,+	41,52,31,37,47,55,30,40,51,45,33,48,44,49,39,56,34,53,46,42,50,36,29,32+};++static const uint8_t SHIFTS[16] = { 1,1,2,2,2,2,2,2,1,2,2,2,2,2,2,1 };++void crypton_des_init(crypton_des_key *ks, const uint8_t *key, int reverse)+{+	uint64_t k = load_be64(key);+	uint64_t cd = 0;+	uint32_t c, d;+	int round, i;++	for (i = 0; i < 56; i++)+		cd = (cd << 1) | bit_of(k, PC1[i], 64);+	c = (uint32_t) (cd >> 28);+	d = (uint32_t) (cd & 0x0fffffffU);++	for (round = 0; round < 16; round++) {+		uint64_t merged;+		uint8_t *sk;+		int s = SHIFTS[round];++		c = ((c << s) | (c >> (28 - s))) & 0x0fffffffU;+		d = ((d << s) | (d >> (28 - s))) & 0x0fffffffU;+		merged = ((uint64_t) c << 28) | d;++		sk = ks->sk + (reverse ? (15 - round) : round) * 8;+		memset(sk, 0, 8);+		for (i = 0; i < 48; i++)+			sk[i / 6] = (uint8_t) ((sk[i / 6] << 1) | bit_of(merged, PC2[i], 56));+	}+}++static inline uint32_t des_f(uint32_t r, const uint8_t *sk)+{+	return SP[0][((ROTL32(r, 31) >> 26) & 0x3f) ^ sk[0]]+	     | SP[1][((ROTL32(r,  3) >> 26) & 0x3f) ^ sk[1]]+	     | SP[2][((ROTL32(r,  7) >> 26) & 0x3f) ^ sk[2]]+	     | SP[3][((ROTL32(r, 11) >> 26) & 0x3f) ^ sk[3]]+	     | SP[4][((ROTL32(r, 15) >> 26) & 0x3f) ^ sk[4]]+	     | SP[5][((ROTL32(r, 19) >> 26) & 0x3f) ^ sk[5]]+	     | SP[6][((ROTL32(r, 23) >> 26) & 0x3f) ^ sk[6]]+	     | SP[7][((ROTL32(r, 27) >> 26) & 0x3f) ^ sk[7]];+}++static inline uint64_t des_block(uint64_t b, const uint8_t *sk)+{+	uint32_t l, r;+	int round;++	l = IPL[0][(b >> 56) & 0xff] | IPL[1][(b >> 48) & 0xff]+	  | IPL[2][(b >> 40) & 0xff] | IPL[3][(b >> 32) & 0xff]+	  | IPL[4][(b >> 24) & 0xff] | IPL[5][(b >> 16) & 0xff]+	  | IPL[6][(b >>  8) & 0xff] | IPL[7][ b        & 0xff];+	r = IPR[0][(b >> 56) & 0xff] | IPR[1][(b >> 48) & 0xff]+	  | IPR[2][(b >> 40) & 0xff] | IPR[3][(b >> 32) & 0xff]+	  | IPR[4][(b >> 24) & 0xff] | IPR[5][(b >> 16) & 0xff]+	  | IPR[6][(b >>  8) & 0xff] | IPR[7][ b        & 0xff];++	for (round = 0; round < 16; round++) {+		uint32_t t = r;+		r = l ^ des_f(r, sk + round * 8);+		l = t;+	}++	{+		/* the preoutput is the halves the other way round */+		uint64_t p = ((uint64_t) r << 32) | l;+		return ((uint64_t) (FPH[0][(p >> 56) & 0xff] | FPH[1][(p >> 48) & 0xff]+		                  | FPH[2][(p >> 40) & 0xff] | FPH[3][(p >> 32) & 0xff]+		                  | FPH[4][(p >> 24) & 0xff] | FPH[5][(p >> 16) & 0xff]+		                  | FPH[6][(p >>  8) & 0xff] | FPH[7][ p        & 0xff]) << 32)+		     | (FPL[0][(p >> 56) & 0xff] | FPL[1][(p >> 48) & 0xff]+		      | FPL[2][(p >> 40) & 0xff] | FPL[3][(p >> 32) & 0xff]+		      | FPL[4][(p >> 24) & 0xff] | FPL[5][(p >> 16) & 0xff]+		      | FPL[6][(p >>  8) & 0xff] | FPL[7][ p        & 0xff]);+	}+}++/* Run every stage of the schedule over each block: one stage is DES, three are+ * EDE or EEE depending on the directions the schedules were built for. */+void crypton_des_ecb(uint8_t *out, const crypton_des_key *ks, uint32_t nkeys,+                     const uint8_t *in, uint32_t nblocks)+{+	uint32_t i, s;++	for (i = 0; i < nblocks; i++) {+		uint64_t b = load_be64(in + 8 * i);+		for (s = 0; s < nkeys; s++)+			b = des_block(b, ks[s].sk);+		store_be64(out + 8 * i, b);+	}+}
+ cbits/crypton_des.h view
@@ -0,0 +1,20 @@+#ifndef CRYPTON_DES_H+#define CRYPTON_DES_H++#include <stdint.h>++/* the sixteen round keys, as eight six-bit values each */+typedef struct {+	uint8_t sk[16 * 8];+} crypton_des_key;++/* Build a schedule from an eight byte key.  The parity bits are ignored, as+ * FIPS 46-3 says.  With reverse set, the rounds come out in the order that+ * decrypts. */+void crypton_des_init(crypton_des_key *ks, const uint8_t *key, int reverse);++/* Apply nkeys schedules in order to each of nblocks eight byte blocks. */+void crypton_des_ecb(uint8_t *out, const crypton_des_key *ks, uint32_t nkeys,+                     const uint8_t *in, uint32_t nblocks);++#endif
+ cbits/crypton_ecc.c view
@@ -0,0 +1,542 @@+/*+ * Scalar multiplication on a curve over a prime field, doing the same work+ * whatever the scalar is.+ *+ * The scalar is walked four bits at a time: four doublings and one addition+ * of a small multiple of the point, taken from a table of sixteen that is+ * read by touching every entry and keeping one of them with a mask.  So a+ * window costs the same five operations and the same sixteen reads whatever+ * its bits are, and nothing branches on, or indexes memory with, the scalar.+ *+ * The addition and the doubling are the complete formulas of Renes, Costello+ * and Batina (eprint 2015/1060, algorithms 1 and 3), which answer for every+ * pair of points there is -- the same point twice, a point and its negation,+ * the point at infinity -- without a case to choose between.  A formula with+ * cases would need the choice to be made with a mask like everything else+ * here, and would still have to be right about which cases there are; these+ * have none.  They cost about half again what the usual Jacobian formulas do,+ * which is the price of that.+ *+ * Points are kept in homogeneous projective coordinates, where the point at+ * infinity is (0 : 1 : 0), and in Montgomery form, so that the only reduction+ * is the one the multiplication does anyway.+ */+#include <stdlib.h>+#include <crypton_bignum.h>+#include <crypton_bzero.h>+#include <crypton_ecc.h>+#include <crypton_powm.h>+#ifdef CRYPTON_S2N_BIGNUM+#include <crypton_ecc_s2n.h>+#endif++/* four bits of scalar per window, so a table of sixteen and no leftover+ * bits: a byte holds exactly two windows */+#define WINDOW_BITS 4+#define TABLE_SIZE (1 << WINDOW_BITS)++/* the field the curve is over, and what it takes to work in it */+typedef struct {+	uint32_t n; /* limbs in a field element */+	limb_t n0;  /* -p^-1 mod 2^LIMB_BITS */+	const limb_t *p;+	const limb_t *a;  /* the curve's a, in Montgomery form */+	const limb_t *b3; /* three times the curve's b, in Montgomery form */+	const limb_t *zero; /* n limbs of nothing, to subtract from */+	int a_is_zero;      /* a is 0 or p-3 for every curve in use, and then */+	int a_is_minus3;    /* multiplying by it is additions instead */+	limb_t *t;        /* 2n of scratch, for the multiplication */+	limb_t *s;        /* n of scratch, for the addition and subtraction */+	limb_t *s2;       /* n more, for multiplying by a, which may write over+	                   * what it is reading */+} field;++static void fe_mul(const field *f, limb_t *r, const limb_t *x, const limb_t *y)+{+	mont_mul(r, x, y, f->p, f->n0, f->n, f->t);+}++static void fe_sqr(const field *f, limb_t *r, const limb_t *x)+{+	mont_sqr(r, x, f->p, f->n0, f->n, f->t);+}++static void fe_add(const field *f, limb_t *r, const limb_t *x, const limb_t *y)+{+	limb_t carry = add_n(r, x, y, f->n);+	limb_t borrow = sub_n(f->s, r, f->p, f->n);++	select_n(r, f->s, r, (carry | (borrow ^ 1)) & 1, f->n);+}++static void fe_sub(const field *f, limb_t *r, const limb_t *x, const limb_t *y)+{+	limb_t borrow = sub_n(r, x, y, f->n);++	add_n(f->s, r, f->p, f->n);+	select_n(r, f->s, r, borrow, f->n);+}++/* r = -x */+static void fe_neg(const field *f, limb_t *r, const limb_t *x)+{+	fe_sub(f, r, f->zero, x);+}++/* r = a * x, where a is the curve's.  It is zero or minus three on every+ * curve in use, and then this is additions rather than a multiplication.+ * Which of the three it is comes from the curve, which is public. */+static void fe_mul_a(const field *f, limb_t *r, const limb_t *x)+{+	if (f->a_is_zero) {+		memset(r, 0, f->n * sizeof(limb_t));+	} else if (f->a_is_minus3) {+		/* r and x are the same buffer in places, so this goes through one+		 * of its own */+		fe_add(f, f->s2, x, x);+		fe_add(f, f->s2, f->s2, x);+		fe_neg(f, r, f->s2);+	} else {+		fe_mul(f, r, f->a, x);+	}+}++/* Renes-Costello-Batina algorithm 1: r = x + y, for any two points */+static void point_add(const field *f, limb_t *r, const limb_t *x,+                      const limb_t *y, limb_t *w)+{+	uint32_t n = f->n;+	const limb_t *x1 = x, *y1 = x + n, *z1 = x + 2 * n;+	const limb_t *x2 = y, *y2 = y + n, *z2 = y + 2 * n;+	limb_t *t0 = w, *t1 = w + n, *t2 = w + 2 * n, *t3 = w + 3 * n;+	limb_t *t4 = w + 4 * n, *t5 = w + 5 * n;+	limb_t *x3 = w + 6 * n, *y3 = w + 7 * n, *z3 = w + 8 * n;++	fe_mul(f, t0, x1, x2);+	fe_mul(f, t1, y1, y2);+	fe_mul(f, t2, z1, z2);+	fe_add(f, t3, x1, y1);+	fe_add(f, t4, x2, y2);+	fe_mul(f, t3, t3, t4);+	fe_add(f, t4, t0, t1);+	fe_sub(f, t3, t3, t4);+	fe_add(f, t4, x1, z1);+	fe_add(f, t5, x2, z2);+	fe_mul(f, t4, t4, t5);+	fe_add(f, t5, t0, t2);+	fe_sub(f, t4, t4, t5);+	fe_add(f, t5, y1, z1);+	fe_add(f, x3, y2, z2);+	fe_mul(f, t5, t5, x3);+	fe_add(f, x3, t1, t2);+	fe_sub(f, t5, t5, x3);+	fe_mul_a(f, z3, t4);+	fe_mul(f, x3, f->b3, t2);+	fe_add(f, z3, x3, z3);+	fe_sub(f, x3, t1, z3);+	fe_add(f, z3, t1, z3);+	fe_mul(f, y3, x3, z3);+	fe_add(f, t1, t0, t0);+	fe_add(f, t1, t1, t0);+	fe_mul_a(f, t2, t2);+	fe_mul(f, t4, f->b3, t4);+	fe_add(f, t1, t1, t2);+	fe_sub(f, t2, t0, t2);+	fe_mul_a(f, t2, t2);+	fe_add(f, t4, t4, t2);+	fe_mul(f, t0, t1, t4);+	fe_add(f, y3, y3, t0);+	fe_mul(f, t0, t5, t4);+	fe_mul(f, x3, t3, x3);+	fe_sub(f, x3, x3, t0);+	fe_mul(f, t0, t3, t1);+	fe_mul(f, t1, t5, z3);+	fe_add(f, z3, t1, t0);++	memcpy(r, x3, n * sizeof(limb_t));+	memcpy(r + n, y3, n * sizeof(limb_t));+	memcpy(r + 2 * n, z3, n * sizeof(limb_t));+}++/* Renes-Costello-Batina algorithm 3: r = x + x, for any point */+static void point_double(const field *f, limb_t *r, const limb_t *x, limb_t *w)+{+	uint32_t n = f->n;+	const limb_t *px = x, *py = x + n, *pz = x + 2 * n;+	limb_t *t0 = w, *t1 = w + n, *t2 = w + 2 * n, *t3 = w + 3 * n;+	limb_t *x3 = w + 6 * n, *y3 = w + 7 * n, *z3 = w + 8 * n;++	fe_sqr(f, t0, px);+	fe_sqr(f, t1, py);+	fe_sqr(f, t2, pz);+	fe_mul(f, t3, px, py);+	fe_add(f, t3, t3, t3);+	fe_mul(f, z3, px, pz);+	fe_add(f, z3, z3, z3);+	fe_mul_a(f, x3, z3);+	fe_mul(f, y3, f->b3, t2);+	fe_add(f, y3, x3, y3);+	fe_sub(f, x3, t1, y3);+	fe_add(f, y3, t1, y3);+	fe_mul(f, y3, x3, y3);+	fe_mul(f, x3, t3, x3);+	fe_mul(f, z3, f->b3, z3);+	fe_mul_a(f, t2, t2);+	fe_sub(f, t3, t0, t2);+	fe_mul_a(f, t3, t3);+	fe_add(f, t3, t3, z3);+	fe_add(f, z3, t0, t0);+	fe_add(f, t0, z3, t0);+	fe_add(f, t0, t0, t2);+	fe_mul(f, t0, t0, t3);+	fe_add(f, y3, y3, t0);+	fe_mul(f, t2, py, pz);+	fe_add(f, t2, t2, t2);+	fe_mul(f, t0, t2, t3);+	fe_sub(f, x3, x3, t0);+	fe_mul(f, z3, t2, t1);+	fe_add(f, z3, z3, z3);+	fe_add(f, z3, z3, z3);++	memcpy(r, x3, n * sizeof(limb_t));+	memcpy(r + n, y3, n * sizeof(limb_t));+	memcpy(r + 2 * n, z3, n * sizeof(limb_t));+}++/* Everything a curve needs, in one allocation: the field, the buffers the+ * formulas work in, and a table of sixteen points.  The caller frees it with+ * ctx_free. */+typedef struct {+	field f;+	limb_t *space;+	uint32_t words;+	uint32_t n;+	limb_t *r2;   /* R^2 mod p, which is what takes a number to Montgomery form */+	limb_t *one;  /* 1, in Montgomery form */+	limb_t *acc;  /* a point */+	limb_t *sel;  /* a point */+	limb_t *tmp;  /* a point */+	limb_t *work; /* 9n, for the formulas */+	limb_t *table; /* sixteen points */+	uint8_t *bytes; /* 2 * plen, for the inversion */+	uint32_t plen;+} curve_ctx;++static void ctx_free(curve_ctx *c)+{+	/* crypton_bzero rather than memset: this memory is freed on the next+	 * line, and a store to memory about to die is one an optimizer may+	 * drop.  Both buffers have held scalars. */+	if (c->space != NULL) {+		crypton_bzero(c->space, c->words * sizeof(limb_t));+		free(c->space);+	}+	if (c->bytes != NULL) {+		crypton_bzero(c->bytes, 2 * c->plen);+		free(c->bytes);+	}+	c->space = NULL;+	c->bytes = NULL;+}++/* r = x, taken into Montgomery form */+static void to_mont(const curve_ctx *c, limb_t *r, const limb_t *x)+{+	mont_mul(r, x, c->r2, c->f.p, c->f.n0, c->n, c->f.t);+}++/* r = x, taken back out of it */+static void from_mont(const curve_ctx *c, limb_t *r, const limb_t *x)+{+	mont_mul(r, x, c->one, c->f.p, c->f.n0, c->n, c->f.t);+}++static int ctx_init(curve_ctx *c, const uint8_t *a, const uint8_t *b,+                    const uint8_t *p, uint32_t plen)+{+	uint32_t n = (plen + LIMB_BYTES - 1) / LIMB_BYTES;+	limb_t *mp, *ma, *mb3, *zero, *scratch, *mont_t;+	uint32_t i;++	memset(c, 0, sizeof(*c));+	if (plen == 0 || n == 0 || (p[plen - 1] & 1) == 0)+		return -1;++	/* six single numbers, three points, four of scratch, nine for the+	 * formulas, and a table of sixteen points */+	c->n = n;+	c->plen = plen;+	c->words = (6 + 9 + 4 + 9 + 3 * TABLE_SIZE) * n;+	c->space = calloc(c->words, sizeof(limb_t));+	c->bytes = calloc(2, plen);+	if (c->space == NULL || c->bytes == NULL) {+		ctx_free(c);+		return -1;+	}+	mp = c->space;+	ma = mp + n;+	mb3 = ma + n;+	c->r2 = mb3 + n;+	c->one = c->r2 + n;+	zero = c->one + n;+	c->acc = zero + n;+	c->sel = c->acc + 3 * n;+	c->tmp = c->sel + 3 * n;+	scratch = c->tmp + 3 * n;+	mont_t = scratch + 2 * n;+	c->work = mont_t + 2 * n;+	c->table = c->work + 9 * n;++	if (from_be(mp, n, p, plen) != 0) {+		ctx_free(c);+		return -1;+	}+	c->f.n0 = mont_n0(mp[0]);+	mont_r2(c->r2, mp, c->f.n0, n, mont_t);++	c->f.n = n;+	c->f.p = mp;+	c->f.a = ma;+	c->f.b3 = mb3;+	c->f.zero = zero;+	c->f.t = mont_t;+	c->f.s = scratch;+	c->f.s2 = scratch + n;+	c->f.a_is_zero = 0;+	c->f.a_is_minus3 = 0;++	memset(c->one, 0, n * sizeof(limb_t));+	c->one[0] = 1;+	to_mont(c, c->tmp, c->one);+	memcpy(c->one, c->tmp, n * sizeof(limb_t));++	/* the curve's a, and which of the three shapes it has */+	if (from_be(c->tmp, n, a, plen) != 0) {+		ctx_free(c);+		return -1;+	}+	{+		limb_t nonzero = 0, differs = 0;++		for (i = 0; i < n; i++)+			nonzero |= c->tmp[i];+		memset(c->sel, 0, n * sizeof(limb_t));+		c->sel[0] = 3;+		sub_n(c->sel, mp, c->sel, n); /* p - 3 */+		for (i = 0; i < n; i++)+			differs |= c->tmp[i] ^ c->sel[i];+		c->f.a_is_zero = nonzero == 0;+		c->f.a_is_minus3 = differs == 0;+	}+	to_mont(c, ma, c->tmp);++	/* three times the curve's b, which is what the formulas want */+	if (from_be(c->tmp, n, b, plen) != 0) {+		ctx_free(c);+		return -1;+	}+	to_mont(c, mb3, c->tmp);+	fe_add(&c->f, c->tmp, mb3, mb3);+	fe_add(&c->f, mb3, c->tmp, mb3);+	return 0;+}++/* a point, in Montgomery form, from its coordinates */+static int point_from_be(const curve_ctx *c, limb_t *r, const uint8_t *px,+                         const uint8_t *py)+{+	uint32_t n = c->n;++	if (from_be(c->tmp, n, px, c->plen) != 0)+		return -1;+	to_mont(c, r, c->tmp);+	if (from_be(c->tmp, n, py, c->plen) != 0)+		return -1;+	to_mont(c, r + n, c->tmp);+	memcpy(r + 2 * n, c->one, n * sizeof(limb_t));+	return 0;+}++/* x = X/Z and y = Y/Z, with the inverse from Fermat, which is the+ * exponentiation that hides its exponent.  Returns 1 for the point at+ * infinity, which has no coordinates. */+static int point_to_be(curve_ctx *c, uint8_t *outx, uint8_t *outy,+                       const limb_t *pt, const uint8_t *p)+{+	uint32_t n = c->n, plen = c->plen, i;+	limb_t empty = 0;+	uint8_t *zbytes = c->bytes, *pm2 = c->bytes + plen;++	for (i = 0; i < n; i++)+		empty |= pt[2 * n + i];+	if (empty == 0)+		return 1;++	from_mont(c, c->tmp, pt + 2 * n);+	to_be(zbytes, plen, c->tmp, n);+	memset(c->sel, 0, n * sizeof(limb_t));+	c->sel[0] = 2;+	sub_n(c->sel, c->f.p, c->sel, n); /* p - 2 */+	to_be(pm2, plen, c->sel, n);+	if (crypton_powm_sec(zbytes, zbytes, plen, pm2, plen, p, plen) != 0)+		return -1;+	if (from_be(c->tmp, n, zbytes, plen) != 0)+		return -1;+	to_mont(c, c->sel, c->tmp); /* 1/Z, in Montgomery form */++	fe_mul(&c->f, c->tmp, pt, c->sel);+	from_mont(c, c->tmp + n, c->tmp);+	to_be(outx, plen, c->tmp + n, n);++	fe_mul(&c->f, c->tmp, pt + n, c->sel);+	from_mont(c, c->tmp + n, c->tmp);+	to_be(outy, plen, c->tmp + n, n);+	return 0;+}++/* every one of the sixteen entries is read, and a mask keeps the one wanted */+static void table_select(const curve_ctx *c, limb_t *r, const limb_t *table,+                         limb_t w)+{+	uint32_t n = c->n, j, l;++	memset(r, 0, 3 * n * sizeof(limb_t));+	for (j = 0; j < TABLE_SIZE; j++) {+		limb_t mask = eq_mask(j, w);++		for (l = 0; l < 3 * n; l++)+			r[l] |= table[3 * j * n + l] & mask;+	}+}++int crypton_ecc_mul(uint8_t *outx, uint8_t *outy,+                    const uint8_t *px, const uint8_t *py,+                    const uint8_t *k, uint32_t klen,+                    const uint8_t *a, const uint8_t *b,+                    const uint8_t *p, uint32_t plen)+{+	curve_ctx c;+	uint32_t n, i, j;+	int ret = -1;++#ifdef CRYPTON_S2N_BIGNUM+	/* Two of the curves that reach here have hand-written assembly, six+	 * to ten times faster than what follows; see cbits/s2n/README.md.+	 * Anything else, including those two named with a different a or b,+	 * goes on down. */+	{+		int s2n_ret;++		if (crypton_s2n_ecc_mul(&s2n_ret, outx, outy, px, py, k, klen,+		                        a, b, p, plen))+			return s2n_ret;+	}+#endif++	if (klen == 0 || ctx_init(&c, a, b, p, plen) != 0)+		return -1;+	n = c.n;++	/* the table: nothing, the point, and its multiples up to fifteen */+	memset(c.table, 0, 3 * n * sizeof(limb_t));+	memcpy(c.table + n, c.one, n * sizeof(limb_t)); /* (0 : 1 : 0) */+	if (point_from_be(&c, c.table + 3 * n, px, py) != 0)+		goto done;+	for (i = 2; i < TABLE_SIZE; i++)+		point_add(&c.f, c.table + 3 * i * n, c.table + 3 * (i - 1) * n,+		          c.table + 3 * n, c.work);++	/* four bits at a time, from the top */+	memcpy(c.acc, c.table, 3 * n * sizeof(limb_t));+	for (i = klen * 2; i > 0; i--) {+		uint32_t nib = i - 1;+		limb_t w = (k[klen - 1 - nib / 2] >> (4 * (nib % 2))) & 0xf;++		for (j = 0; j < WINDOW_BITS; j++)+			point_double(&c.f, c.acc, c.acc, c.work);+		table_select(&c, c.sel, c.table, w);+		point_add(&c.f, c.acc, c.acc, c.sel, c.work);+	}+	ret = point_to_be(&c, outx, outy, c.acc, p);++done:+	ctx_free(&c);+	return ret;+}++uint32_t crypton_ecc_table_size(uint32_t plen, uint32_t klen)+{+	uint32_t n = (plen + LIMB_BYTES - 1) / LIMB_BYTES;++	if (plen == 0 || klen == 0 || n == 0)+		return 0;+	return klen * 2 * TABLE_SIZE * 3 * n * (uint32_t) sizeof(limb_t);+}++int crypton_ecc_table_build(uint8_t *tab,+                            const uint8_t *gx, const uint8_t *gy,+                            uint32_t klen,+                            const uint8_t *a, const uint8_t *b,+                            const uint8_t *p, uint32_t plen)+{+	curve_ctx c;+	limb_t *t = (limb_t *) (void *) tab;+	uint32_t n, i, j, windows;+	int ret = -1;++	if (klen == 0 || ctx_init(&c, a, b, p, plen) != 0)+		return -1;+	n = c.n;+	windows = klen * 2;++	/* acc walks the powers: at window i it holds 16^i times the point */+	if (point_from_be(&c, c.acc, gx, gy) != 0)+		goto done;+	for (i = 0; i < windows; i++) {+		limb_t *slot = t + (size_t) i * TABLE_SIZE * 3 * n;++		memset(slot, 0, 3 * n * sizeof(limb_t));+		memcpy(slot + n, c.one, n * sizeof(limb_t)); /* (0 : 1 : 0) */+		memcpy(slot + 3 * n, c.acc, 3 * n * sizeof(limb_t));+		for (j = 2; j < TABLE_SIZE; j++)+			point_add(&c.f, slot + 3 * j * n, slot + 3 * (j - 1) * n,+			          c.acc, c.work);+		for (j = 0; j < WINDOW_BITS; j++)+			point_double(&c.f, c.acc, c.acc, c.work);+	}+	ret = 0;++done:+	ctx_free(&c);+	return ret;+}++int crypton_ecc_table_mul(uint8_t *outx, uint8_t *outy, const uint8_t *tab,+                          const uint8_t *k, uint32_t klen,+                          const uint8_t *a, const uint8_t *b,+                          const uint8_t *p, uint32_t plen)+{+	curve_ctx c;+	const limb_t *t = (const limb_t *) (const void *) tab;+	uint32_t n, i;+	int ret;++	if (klen == 0 || ctx_init(&c, a, b, p, plen) != 0)+		return -1;+	n = c.n;++	/* nothing to start with, and one addition for every four bits: the+	 * multiples the doubling would work out are all in the table */+	memset(c.acc, 0, 3 * n * sizeof(limb_t));+	memcpy(c.acc + n, c.one, n * sizeof(limb_t));+	for (i = 0; i < klen * 2; i++) {+		limb_t w = (k[klen - 1 - i / 2] >> (4 * (i % 2))) & 0xf;++		table_select(&c, c.sel, t + (size_t) i * TABLE_SIZE * 3 * n, w);+		point_add(&c.f, c.acc, c.acc, c.sel, c.work);+	}+	ret = point_to_be(&c, outx, outy, c.acc, p);++	ctx_free(&c);+	return ret;+}
+ cbits/crypton_ecc.h view
@@ -0,0 +1,57 @@+#ifndef CRYPTON_ECC_H+#define CRYPTON_ECC_H++#include <stdint.h>++/* Multiply a point of a curve over a prime field by a scalar, doing the same+ * work whatever the scalar is.+ *+ * The curve is y^2 = x^3 + a*x + b over the field of p, which has to be an+ * odd prime; the point has to be on it and not the point at infinity, and its+ * coordinates, a and b have to be below p.  Every number is a big-endian byte+ * string, and the coordinates, a, b and p are all plen bytes.+ *+ * The scalar is walked four bits at a time over every one of the klen bytes+ * it is given, so its value is hidden but its length is not.+ *+ * Returns 0 with the answer in outx and outy, 1 if the answer is the point at+ * infinity, which has no coordinates, and -1 if the arguments are not ones it+ * can work with or memory ran out.+ */+int crypton_ecc_mul(uint8_t *outx, uint8_t *outy,+                    const uint8_t *px, const uint8_t *py,+                    const uint8_t *k, uint32_t klen,+                    const uint8_t *a, const uint8_t *b,+                    const uint8_t *p, uint32_t plen);++/* How many bytes a table for a base point takes, for a prime of plen bytes+ * and scalars of klen.  Zero if those sizes are not ones it can work with. */+uint32_t crypton_ecc_table_size(uint32_t plen, uint32_t klen);++/* Fill that many bytes with the multiples of a point that+ * crypton_ecc_table_mul wants: for every four bits of a scalar, the sixteen+ * points that those bits can call for.  The buffer has to be aligned as a+ * pointer is, which is what an allocator gives.+ *+ * The point, a, b and p are as for crypton_ecc_mul.  Returns 0, or -1 for+ * arguments it cannot work with or memory it could not have.+ */+int crypton_ecc_table_build(uint8_t *table,+                            const uint8_t *gx, const uint8_t *gy,+                            uint32_t klen,+                            const uint8_t *a, const uint8_t *b,+                            const uint8_t *p, uint32_t plen);++/* Multiply the point that table was built for by a scalar of klen bytes,+ * which has to be the klen the table was built for.  One addition for every+ * four bits and no doublings, since the table holds what the doublings would+ * work out.+ *+ * Returns what crypton_ecc_mul returns.+ */+int crypton_ecc_table_mul(uint8_t *outx, uint8_t *outy, const uint8_t *table,+                          const uint8_t *k, uint32_t klen,+                          const uint8_t *a, const uint8_t *b,+                          const uint8_t *p, uint32_t plen);++#endif
+ cbits/crypton_ecc_s2n.c view
@@ -0,0 +1,204 @@+#include <string.h>++#include "crypton_ecc_s2n.h"+#include "crypton_ecc_s2n_curves.h"+#include "crypton_cpu.h"++/* P-384, Montgomery domain, six words a coordinate */+extern void p384_montjscalarmul(uint64_t *res, const uint64_t *s, const uint64_t *p);+extern void p384_montjscalarmul_alt(uint64_t *res, const uint64_t *s, const uint64_t *p);+extern void bignum_tomont_p384(uint64_t *z, const uint64_t *x);+extern void bignum_tomont_p384_alt(uint64_t *z, const uint64_t *x);+extern void bignum_deamont_p384(uint64_t *z, const uint64_t *x);+extern void bignum_deamont_p384_alt(uint64_t *z, const uint64_t *x);+extern void bignum_montmul_p384(uint64_t *z, const uint64_t *x, const uint64_t *y);+extern void bignum_montmul_p384_alt(uint64_t *z, const uint64_t *x, const uint64_t *y);+extern void bignum_montsqr_p384(uint64_t *z, const uint64_t *x);+extern void bignum_montsqr_p384_alt(uint64_t *z, const uint64_t *x);+extern void bignum_montinv_p384(uint64_t *z, const uint64_t *x);++/* P-521, ordinary values, nine words a coordinate */+extern void p521_jscalarmul(uint64_t *res, const uint64_t *s, const uint64_t *p);+extern void p521_jscalarmul_alt(uint64_t *res, const uint64_t *s, const uint64_t *p);+extern void bignum_mul_p521(uint64_t *z, const uint64_t *x, const uint64_t *y);+extern void bignum_mul_p521_alt(uint64_t *z, const uint64_t *x, const uint64_t *y);+extern void bignum_sqr_p521(uint64_t *z, const uint64_t *x);+extern void bignum_sqr_p521_alt(uint64_t *z, const uint64_t *x);+extern void bignum_inv_p521(uint64_t *z, const uint64_t *x);++/* One flavour or the other, all the way through: on x86-64 the plain form+ * wants MULX, ADCX and ADOX and _alt is the fallback, so mixing them would+ * fault on a machine without those. */+struct p384_asm {+	void (*tomont)(uint64_t *, const uint64_t *);+	void (*deamont)(uint64_t *, const uint64_t *);+	void (*montmul)(uint64_t *, const uint64_t *, const uint64_t *);+	void (*montsqr)(uint64_t *, const uint64_t *);+	void (*jscalarmul)(uint64_t *, const uint64_t *, const uint64_t *);+};+struct p521_asm {+	void (*mul)(uint64_t *, const uint64_t *, const uint64_t *);+	void (*sqr)(uint64_t *, const uint64_t *);+	void (*jscalarmul)(uint64_t *, const uint64_t *, const uint64_t *);+};++static const struct p384_asm p384_std = {+	bignum_tomont_p384, bignum_deamont_p384, bignum_montmul_p384,+	bignum_montsqr_p384, p384_montjscalarmul+};+static const struct p384_asm p384_alt = {+	bignum_tomont_p384_alt, bignum_deamont_p384_alt,+	bignum_montmul_p384_alt, bignum_montsqr_p384_alt,+	p384_montjscalarmul_alt+};+static const struct p521_asm p521_std = {+	bignum_mul_p521, bignum_sqr_p521, p521_jscalarmul+};+static const struct p521_asm p521_alt = {+	bignum_mul_p521_alt, bignum_sqr_p521_alt, p521_jscalarmul_alt+};++/* The same question as for P-256, answered the same way; see+ * cbits/p256/p256_s2n.c. */+static int use_alt(void)+{+#if defined(__aarch64__) || defined(__arm64__)+#ifdef __APPLE__+	return 1;+#else+	return 0;+#endif+#else+	return (crypton_x86_simd_features() & CRYPTON_X86_ADX) == 0;+#endif+}++#define MAXWORDS 9++static void be_to_le64(uint64_t *w, const uint8_t *b, uint32_t len)+{+	uint32_t i;++	for (i = 0; i < MAXWORDS; i++)+		w[i] = 0;+	for (i = 0; i < len; i++) {+		uint32_t pos = len - 1 - i;+		w[pos / 8] |= (uint64_t)b[i] << (8 * (pos % 8));+	}+}++static void le64_to_be(uint8_t *b, uint32_t len, const uint64_t *w)+{+	uint32_t i;++	for (i = 0; i < len; i++)+		b[len - 1 - i] = (uint8_t)(w[i / 8] >> (8 * (i % 8)));+}++static int is_zero(const uint64_t *w, int words)+{+	uint64_t acc = 0;+	int i;++	for (i = 0; i < words; i++)+		acc |= w[i];+	return acc == 0;+}++static int mul_p384(uint8_t *outx, uint8_t *outy, const uint8_t *px,+                    const uint8_t *py, const uint8_t *k, uint32_t klen)+{+	const struct p384_asm *f = use_alt() ? &p384_alt : &p384_std;+	uint64_t pt[18], res[18], sc[MAXWORDS], t[MAXWORDS];+	uint64_t zi[6], zi2[6], zi3[6], num[6];+	static const uint64_t one[6] = {1, 0, 0, 0, 0, 0};++	be_to_le64(t, px, P384_PLEN);+	f->tomont(pt, t);+	be_to_le64(t, py, P384_PLEN);+	f->tomont(pt + 6, t);+	f->tomont(pt + 12, one);+	be_to_le64(sc, k, klen);++	f->jscalarmul(res, sc, pt);+	if (is_zero(res + 12, 6))+		return 1;++	/* affine again: x = X/Z^2, y = Y/Z^3, with the inverse taken in the+	 * Montgomery domain so that it lands where the rest of these are */+	bignum_montinv_p384(zi, res + 12);+	f->montsqr(zi2, zi);+	f->montmul(zi3, zi2, zi);+	f->montmul(num, res, zi2);+	f->deamont(t, num);+	le64_to_be(outx, P384_PLEN, t);+	f->montmul(num, res + 6, zi3);+	f->deamont(t, num);+	le64_to_be(outy, P384_PLEN, t);+	return 0;+}++static int mul_p521(uint8_t *outx, uint8_t *outy, const uint8_t *px,+                    const uint8_t *py, const uint8_t *k, uint32_t klen)+{+	const struct p521_asm *f = use_alt() ? &p521_alt : &p521_std;+	uint64_t pt[27], res[27], sc[MAXWORDS], t[MAXWORDS];+	uint64_t zi[9], zi2[9], zi3[9];+	static const uint64_t one[9] = {1, 0, 0, 0, 0, 0, 0, 0, 0};++	be_to_le64(pt, px, P521_PLEN);+	be_to_le64(pt + 9, py, P521_PLEN);+	memcpy(pt + 18, one, sizeof(one));+	be_to_le64(sc, k, klen);++	f->jscalarmul(res, sc, pt);+	if (is_zero(res + 18, 9))+		return 1;++	bignum_inv_p521(zi, res + 18);+	f->sqr(zi2, zi);+	f->mul(zi3, zi2, zi);+	f->mul(t, res, zi2);+	le64_to_be(outx, P521_PLEN, t);+	f->mul(t, res + 9, zi3);+	le64_to_be(outy, P521_PLEN, t);+	return 0;+}++int crypton_s2n_ecc_mul(int *ret, uint8_t *outx, uint8_t *outy,+                        const uint8_t *px, const uint8_t *py,+                        const uint8_t *k, uint32_t klen,+                        const uint8_t *a, const uint8_t *b,+                        const uint8_t *p, uint32_t plen)+{+	int is384;++	if (plen == P384_PLEN && memcmp(p, P384_P, plen) == 0+	    && memcmp(a, P384_A, plen) == 0 && memcmp(b, P384_B, plen) == 0)+		is384 = 1;+	else if (plen == P521_PLEN && memcmp(p, P521_P, plen) == 0+	         && memcmp(a, P521_A, plen) == 0+	         && memcmp(b, P521_B, plen) == 0)+		is384 = 0;+	else+		return 0; /* some other curve; the C answers it */++	/* A scalar longer than the prime is not something the word arrays+	 * here hold, and it is not what any caller of these two curves+	 * sends, so leave it to the C rather than grow a second path. */+	if (klen == 0 || klen > plen)+		return 0;++	/* The C does not require the coordinates to be reduced -- it takes+	 * whatever fits in its limbs and lets the conversion to Montgomery+	 * form reduce it.  Rather than carry a reduction here to match, hand+	 * that case back: nothing sends one, and this way the two cannot+	 * disagree about it.  (A differential test against the C found this;+	 * the first version of this check returned -1 and was wrong.) */+	if (memcmp(px, p, plen) >= 0 || memcmp(py, p, plen) >= 0)+		return 0;++	*ret = is384 ? mul_p384(outx, outy, px, py, k, klen)+	             : mul_p521(outx, outy, px, py, k, klen);+	return 1;+}
+ cbits/crypton_ecc_s2n.h view
@@ -0,0 +1,27 @@+/*+ * P-384 and P-521 through the vendored s2n-bignum assembly, for the two+ * curves it knows among the ones crypton_ecc_mul is asked about.+ *+ * s2n-bignum has no affine wrapper for these two -- only a scalar+ * multiplication on Jacobian points, Montgomery-domain for P-384 and plain+ * for P-521 -- so the conversions in and out are built here out of its own+ * field operations.  See cbits/s2n/README.md.+ */+#ifndef CRYPTON_ECC_S2N_H+#define CRYPTON_ECC_S2N_H++#include <stdint.h>++/*+ * Returns 1 if this was a curve it knows and it answered, with *ret set to+ * what crypton_ecc_mul should return -- 0 and the point in outx and outy, 1+ * for the point at infinity, or -1 for arguments it will not take.  Returns+ * 0 if the curve is not one of its two and nothing was written.+ */+int crypton_s2n_ecc_mul(int *ret, uint8_t *outx, uint8_t *outy,+                        const uint8_t *px, const uint8_t *py,+                        const uint8_t *k, uint32_t klen,+                        const uint8_t *a, const uint8_t *b,+                        const uint8_t *p, uint32_t plen);++#endif
+ cbits/crypton_ecc_s2n_curves.h view
@@ -0,0 +1,76 @@+/*+ * p, a and b of the two curves the vendored s2n-bignum assembly knows, as+ * big-endian bytes, which is how crypton_ecc_mul is given a curve.  They are+ * here to be compared against, not computed with: a caller naming some other+ * curve with the same sizes has to go to the C.+ *+ * Generated from `openssl ecparam -name secp384r1 -param_enc explicit -text`+ * and the same for secp521r1, rather than transcribed.+ */+#ifndef CRYPTON_ECC_S2N_CURVES_H+#define CRYPTON_ECC_S2N_CURVES_H++#include <stdint.h>++#define P384_PLEN 48+static const uint8_t P384_P[48] = {+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xfe,+	0xff,0xff,0xff,0xff,0x00,0x00,0x00,0x00,+	0x00,0x00,0x00,0x00,0xff,0xff,0xff,0xff+};+static const uint8_t P384_A[48] = {+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xfe,+	0xff,0xff,0xff,0xff,0x00,0x00,0x00,0x00,+	0x00,0x00,0x00,0x00,0xff,0xff,0xff,0xfc+};+static const uint8_t P384_B[48] = {+	0xb3,0x31,0x2f,0xa7,0xe2,0x3e,0xe7,0xe4,+	0x98,0x8e,0x05,0x6b,0xe3,0xf8,0x2d,0x19,+	0x18,0x1d,0x9c,0x6e,0xfe,0x81,0x41,0x12,+	0x03,0x14,0x08,0x8f,0x50,0x13,0x87,0x5a,+	0xc6,0x56,0x39,0x8d,0x8a,0x2e,0xd1,0x9d,+	0x2a,0x85,0xc8,0xed,0xd3,0xec,0x2a,0xef+};++#define P521_PLEN 66+static const uint8_t P521_P[66] = {+	0x01,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff+};+static const uint8_t P521_A[66] = {+	0x01,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,+	0xff,0xfc+};+static const uint8_t P521_B[66] = {+	0x00,0x51,0x95,0x3e,0xb9,0x61,0x8e,0x1c,+	0x9a,0x1f,0x92,0x9a,0x21,0xa0,0xb6,0x85,+	0x40,0xee,0xa2,0xda,0x72,0x5b,0x99,0xb3,+	0x15,0xf3,0xb8,0xb4,0x89,0x91,0x8e,0xf1,+	0x09,0xe1,0x56,0x19,0x39,0x51,0xec,0x7e,+	0x93,0x7b,0x16,0x52,0xc0,0xbd,0x3b,0xb1,+	0xbf,0x07,0x35,0x73,0xdf,0x88,0x3d,0x2c,+	0x34,0xf1,0xef,0x45,0x1f,0xd4,0x6b,0x50,+	0x3f,0x00+};++#endif
+ cbits/crypton_f2m.c view
@@ -0,0 +1,555 @@+/*+ * Arithmetic in a binary field, and the scalar multiplication a curve over+ * one needs, doing the same work whatever the scalar is.+ *+ * A carry-less multiplication is the one thing a binary field needs and+ * ordinary arithmetic does not give.  Where the processor has the instruction+ * for it this uses it -- PMULL on aarch64, PCLMULQDQ on x86-64 -- asking the+ * machine at run time where the compiler has not already been told.  Where it+ * does not, each operand is split into four groups of every fourth bit, so+ * that the carries of an ordinary multiplication cannot reach the bits that+ * matter, and masked away afterwards.  None of the three has a table or a+ * branch that depends on what it is multiplying.+ *+ * Reduction folds what is above the degree back in, which the polynomial+ * being a trinomial or a pentanomial with exponents that are public makes+ * cheap.  Inversion is the exponentiation Fermat gives, whose exponent is+ * likewise public.+ *+ * The multiplication itself is Montgomery's ladder: it carries the x+ * coordinates of the multiples of two consecutive numbers, whose difference+ * is therefore the point, and spends one addition and one doubling on every+ * bit of the scalar whichever way the bit goes.+ */+#include <stdint.h>+#include <stdlib.h>+#include <string.h>+#include <crypton_cpu.h>+#include "crypton_armv8_target.h"+#include <crypton_bzero.h>+#include <crypton_f2m.h>++typedef uint64_t limb_t;+#define LIMB_BITS 64+#define LIMB_BYTES 8++/* the four groups, so that no carry of an ordinary multiplication reaches a+ * bit another partial product needs */+static void clmul32(uint32_t x, uint32_t y, limb_t *out)+{+	limb_t x0 = x & 0x11111111u, x1 = x & 0x22222222u;+	limb_t x2 = x & 0x44444444u, x3 = x & 0x88888888u;+	limb_t y0 = y & 0x11111111u, y1 = y & 0x22222222u;+	limb_t y2 = y & 0x44444444u, y3 = y & 0x88888888u;+	limb_t z0 = (x0 * y0) ^ (x1 * y3) ^ (x2 * y2) ^ (x3 * y1);+	limb_t z1 = (x0 * y1) ^ (x1 * y0) ^ (x2 * y3) ^ (x3 * y2);+	limb_t z2 = (x0 * y2) ^ (x1 * y1) ^ (x2 * y0) ^ (x3 * y3);+	limb_t z3 = (x0 * y3) ^ (x1 * y2) ^ (x2 * y1) ^ (x3 * y0);++	*out = (z0 & 0x1111111111111111ULL) | (z1 & 0x2222222222222222ULL)+	       | (z2 & 0x4444444444444444ULL) | (z3 & 0x8888888888888888ULL);+}++static inline void clmul(limb_t a, limb_t b, limb_t *lo, limb_t *hi)+{+	limb_t ah = a >> 32, bh = b >> 32, t0, t1, t2;++	clmul32((uint32_t) a, (uint32_t) b, &t0);+	clmul32((uint32_t) ah, (uint32_t) bh, &t1);+	clmul32((uint32_t) (a ^ ah), (uint32_t) (b ^ bh), &t2);+	t2 ^= t0 ^ t1;+	*lo = t0 ^ (t2 << 32);+	*hi = t1 ^ (t2 >> 32);+}++/* t = a * b, over 2n limbs */+static void poly_mul_generic(limb_t *t, const limb_t *a, const limb_t *b,+                             uint32_t n)+{+	uint32_t i, j;++	memset(t, 0, 2 * n * sizeof(limb_t));+	for (i = 0; i < n; i++)+		for (j = 0; j < n; j++) {+			limb_t lo, hi;++			clmul(a[i], b[j], &lo, &hi);+			t[i + j] ^= lo;+			t[i + j + 1] ^= hi;+		}+}++#if defined(__aarch64__) && (defined(__GNUC__) || defined(__clang__))+#define HAVE_PMULL 1+#include <arm_neon.h>++/* Where the compiler has been told the machine has the crypto extensions --+ * which it is on every Apple processor -- this needs no attribute and no+ * question.  Where it has not, the attribute lets the instruction be emitted+ * in this one function, and the machine is asked before it is called. */+#if defined(__ARM_FEATURE_CRYPTO) || defined(__ARM_FEATURE_AES)+#define PMULL_ATTR+#define PMULL_ALWAYS 1+#else+#define PMULL_ATTR CRYPTON_TARGET_ARMV8_CRYPTO+#define PMULL_ALWAYS 0+#endif++#if !PMULL_ALWAYS+#if defined(__linux__) || defined(__ANDROID__)+#include <asm/hwcap.h>+#include <sys/auxv.h>+#elif defined(__FreeBSD__)+#include <machine/elf.h>+#include <sys/auxv.h>+#elif defined(__APPLE__)+#include <sys/sysctl.h>+#endif+#endif++static int have_pmull(void)+{+#if PMULL_ALWAYS+	return 1;+#elif (defined(__linux__) || defined(__ANDROID__)) && defined(HWCAP_PMULL)+	static int answer = -1;++	if (answer < 0)+		answer = (getauxval(AT_HWCAP) & HWCAP_PMULL) != 0;+	return answer;+#elif defined(__FreeBSD__) && defined(HWCAP_PMULL)+	static int answer = -1;++	if (answer < 0) {+		unsigned long hwcap = 0;++		elf_aux_info(AT_HWCAP, &hwcap, sizeof(hwcap));+		answer = (hwcap & HWCAP_PMULL) != 0;+	}+	return answer;+#elif defined(__APPLE__)+	static int answer = -1;++	if (answer < 0) {+		int has = 0;+		size_t len = sizeof(has);++		answer = sysctlbyname("hw.optional.arm.FEAT_PMULL", &has, &len,+		                      NULL, 0) == 0+		         && has != 0;+	}+	return answer;+#else+	return 0; /* no way to ask, so the four groups it is */+#endif+}++PMULL_ATTR+static void poly_mul_pmull(limb_t *t, const limb_t *a, const limb_t *b,+                           uint32_t n)+{+	uint32_t i, j;++	memset(t, 0, 2 * n * sizeof(limb_t));+	for (i = 0; i < n; i++)+		for (j = 0; j < n; j++) {+			uint64x2_t v = vreinterpretq_u64_p128(+			    vmull_p64((poly64_t) a[i], (poly64_t) b[j]));++			t[i + j] ^= vgetq_lane_u64(v, 0);+			t[i + j + 1] ^= vgetq_lane_u64(v, 1);+		}+}+#else+#define HAVE_PMULL 0+#endif++#if defined(__x86_64__) && (defined(__GNUC__) || defined(__clang__))+#define HAVE_PCLMUL 1+#include <immintrin.h>++/* The same, with the instruction x86 has for it.  The attribute is what lets+ * one file hold both this and the code for a processor without it: the+ * compiler may emit the instruction here and nowhere else, and the caller+ * asks the processor before it comes this way.+ */+__attribute__((target("pclmul,sse2")))+static void poly_mul_pclmul(limb_t *t, const limb_t *a, const limb_t *b,+                            uint32_t n)+{+	uint32_t i, j;++	memset(t, 0, 2 * n * sizeof(limb_t));+	for (i = 0; i < n; i++)+		for (j = 0; j < n; j++) {+			__m128i p = _mm_clmulepi64_si128(+			    _mm_cvtsi64_si128((long long) a[i]),+			    _mm_cvtsi64_si128((long long) b[j]), 0x00);++			t[i + j] ^= (limb_t) _mm_cvtsi128_si64(p);+			t[i + j + 1] ^=+			    (limb_t) _mm_cvtsi128_si64(_mm_srli_si128(p, 8));+		}+}+#else+#define HAVE_PCLMUL 0+#endif++static void poly_mul(limb_t *t, const limb_t *a, const limb_t *b, uint32_t n)+{+#if HAVE_PMULL+	/* what the processor has is not what is being multiplied, so asking is+	 * not a side channel, and the answer is worked out once */+	if (have_pmull()) {+		poly_mul_pmull(t, a, b, n);+		return;+	}+#endif+#if HAVE_PCLMUL+	/* what the processor has is not what is being multiplied, so asking is+	 * not a side channel, and the answer is worked out once */+	if (crypton_x86_simd_features() & CRYPTON_X86_PCLMUL) {+		poly_mul_pclmul(t, a, b, n);+		return;+	}+#endif+	poly_mul_generic(t, a, b, n);+}++/* the bits of a 32-bit half, spread out with a zero between each pair */+static limb_t spread(limb_t x)+{+	x = (x | (x << 16)) & 0x0000ffff0000ffffULL;+	x = (x | (x << 8)) & 0x00ff00ff00ff00ffULL;+	x = (x | (x << 4)) & 0x0f0f0f0f0f0f0f0fULL;+	x = (x | (x << 2)) & 0x3333333333333333ULL;+	x = (x | (x << 1)) & 0x5555555555555555ULL;+	return x;+}++/* t = a * a, which in a binary field is the bits of a spread out */+static void poly_sqr(limb_t *t, const limb_t *a, uint32_t n)+{+	uint32_t i;++	for (i = 0; i < n; i++) {+		t[2 * i] = spread(a[i] & 0xffffffffULL);+		t[2 * i + 1] = spread(a[i] >> 32);+	}+}++/* r = t mod fx, where fx is x^m plus the terms given, which are public+ *+ * Everything above bit m comes back in as those terms, a word at a time, and+ * then what is left above bit m within its own word is folded the same way.+ */+static void poly_reduce(limb_t *r, limb_t *t, uint32_t n, uint32_t m,+                        const uint32_t *terms, uint32_t nterms)+{+	uint32_t mw = m / LIMB_BITS, mb = m % LIMB_BITS, i, j, pass;++	for (i = 2 * n; i > mw + 1; i--) {+		limb_t w = t[i - 1];++		t[i - 1] = 0;+		for (j = 0; j < nterms; j++) {+			uint32_t pos = (i - 1) * LIMB_BITS - m + terms[j];+			uint32_t pw = pos / LIMB_BITS, pb = pos % LIMB_BITS;++			t[pw] ^= w << pb;+			if (pb != 0)+				t[pw + 1] ^= w >> (LIMB_BITS - pb);+		}+	}++	/* what is left above bit m sits in the word that holds it; folding it+	 * can put a little back, so it is done twice */+	for (pass = 0; pass < 2; pass++) {+		limb_t w;++		if (mb == 0)+			break;+		w = t[mw] >> mb;+		t[mw] &= ((limb_t) 1 << mb) - 1;+		for (j = 0; j < nterms; j++) {+			uint32_t pw = terms[j] / LIMB_BITS, pb = terms[j] % LIMB_BITS;++			t[pw] ^= w << pb;+			if (pb != 0 && pw + 1 <= mw)+				t[pw + 1] ^= w >> (LIMB_BITS - pb);+		}+	}+	memcpy(r, t, n * sizeof(limb_t));+}++/* the field: its polynomial, and scratch for a product */+typedef struct {+	uint32_t n;+	uint32_t m;+	uint32_t terms[8]; /* the polynomial without its leading term */+	uint32_t nterms;+	limb_t *t; /* 2n */+} bfield;++static void fe_mul(const bfield *f, limb_t *r, const limb_t *a, const limb_t *b)+{+	poly_mul(f->t, a, b, f->n);+	poly_reduce(r, f->t, f->n, f->m, f->terms, f->nterms);+}++static void fe_sqr(const bfield *f, limb_t *r, const limb_t *a)+{+	poly_sqr(f->t, a, f->n);+	poly_reduce(r, f->t, f->n, f->m, f->terms, f->nterms);+}++static void fe_add(const bfield *f, limb_t *r, const limb_t *a, const limb_t *b)+{+	uint32_t i;++	for (i = 0; i < f->n; i++)+		r[i] = a[i] ^ b[i];+}++static int fe_is_zero(const bfield *f, const limb_t *a)+{+	limb_t acc = 0;+	uint32_t i;++	for (i = 0; i < f->n; i++)+		acc |= a[i];+	return acc == 0;+}++/* r = 1/a, by Fermat: a to the power 2^m - 2, whose exponent is public */+static void fe_inv(const bfield *f, limb_t *r, const limb_t *a, limb_t *tmp)+{+	uint32_t i;++	memcpy(tmp, a, f->n * sizeof(limb_t));+	for (i = 1; i + 1 < f->m; i++) { /* a to the power 2^(m-1) - 1 */+		fe_sqr(f, tmp, tmp);+		fe_mul(f, tmp, tmp, a);+	}+	fe_sqr(f, r, tmp);+}++/* big-endian bytes into limbs, least significant limb first */+static int from_be(limb_t *r, uint32_t n, const uint8_t *src, uint32_t len)+{+	uint32_t i;++	memset(r, 0, n * sizeof(limb_t));+	for (i = 0; i < len; i++) {+		uint8_t byte = src[len - 1 - i];++		if (i / LIMB_BYTES >= n) {+			if (byte != 0)+				return 1;+			continue;+		}+		r[i / LIMB_BYTES] |= (limb_t) byte << (8 * (i % LIMB_BYTES));+	}+	return 0;+}++static void to_be(uint8_t *dst, uint32_t len, const limb_t *a, uint32_t n)+{+	uint32_t i;++	for (i = 0; i < len; i++) {+		uint32_t pos = len - 1 - i, li = i / LIMB_BYTES;++		dst[pos] = li < n ? (uint8_t) (a[li] >> (8 * (i % LIMB_BYTES))) : 0;+	}+}++/* exchange a and b when swap is one */+static void cswap(limb_t *a, limb_t *b, limb_t swap, uint32_t n)+{+	limb_t mask = (limb_t) 0 - swap;+	uint32_t i;++	for (i = 0; i < n; i++) {+		limb_t t = (a[i] ^ b[i]) & mask;++		a[i] ^= t;+		b[i] ^= t;+	}+}++int crypton_f2m_mul(uint8_t *outx, uint8_t *outy,+                    const uint8_t *px, const uint8_t *py,+                    const uint8_t *k, uint32_t klen,+                    const uint8_t *b, uint32_t flen,+                    const uint8_t *fx, uint32_t fxlen)+{+	uint32_t fn = (fxlen + LIMB_BYTES - 1) / LIMB_BYTES;+	uint32_t n, words, i;+	limb_t *space = NULL, *poly, *x, *y, *bb, *x1, *z1, *x2, *z2;+	limb_t *t1, *t2, *t3, *prod;+	bfield f;+	int ret = -1;++	if (flen == 0 || fxlen == 0 || klen == 0 || fn == 0)+		return -1;++	/* the polynomial, and the terms below its leading one */+	{+		limb_t *tmp = calloc(fn, sizeof(limb_t));+		uint32_t m = 0;++		if (tmp == NULL)+			return -1;+		if (from_be(tmp, fn, fx, fxlen) != 0) {+			free(tmp);+			return -1;+		}+		for (i = fn; i > 0 && m == 0; i--)+			if (tmp[i - 1] != 0) {+				limb_t top = tmp[i - 1];++				m = (i - 1) * LIMB_BITS;+				while (top != 0) {+					m++;+					top >>= 1;+				}+				m--; /* the degree is one under the bit count */+			}+		f.m = m;+		f.nterms = 0;+		for (i = 0; i < m; i++)+			if ((tmp[i / LIMB_BITS] >> (i % LIMB_BITS)) & 1) {+				if (f.nterms >= 8) {+					free(tmp);+					return -1; /* more terms than anything in use has */+				}+				f.terms[f.nterms++] = i;+			}+		free(tmp);+		if (m == 0 || f.nterms == 0)+			return -1;+	}++	n = (f.m + LIMB_BITS) / LIMB_BITS; /* room for the degree itself */+	f.n = n;+	words = 12 * n + 2 * n;+	space = calloc(words, sizeof(limb_t));+	if (space == NULL)+		return -1;+	poly = space;      /* unused beyond keeping the layout plain */+	x = poly + n;+	y = x + n;+	bb = y + n;+	x1 = bb + n;+	z1 = x1 + n;+	x2 = z1 + n;+	z2 = x2 + n;+	t1 = z2 + n;+	t2 = t1 + n;+	t3 = t2 + n;+	prod = t3 + n; /* 2n, and one n before it is spare */+	f.t = prod;++	if (from_be(x, n, px, flen) != 0 || from_be(y, n, py, flen) != 0+	    || from_be(bb, n, b, flen) != 0)+		goto done;+	if (fe_is_zero(&f, x))+		goto done; /* the point with no x is the caller's business */++	/* nothing, and the point next to it */+	memset(x1, 0, n * sizeof(limb_t));+	x1[0] = 1;+	memset(z1, 0, n * sizeof(limb_t));+	memcpy(x2, x, n * sizeof(limb_t));+	memset(z2, 0, n * sizeof(limb_t));+	z2[0] = 1;++	for (i = klen * 8; i > 0; i--) {+		uint32_t bit = i - 1;+		limb_t sel = (k[klen - 1 - bit / 8] >> (bit % 8)) & 1;++		/* whichever way the bit goes, one addition and one doubling: the+		 * exchange before and after is what puts them where the bit asks */+		cswap(x1, x2, sel, n);+		cswap(z1, z2, sel, n);++		/* the two added, which their difference being the point allows */+		fe_mul(&f, t1, x1, z2);+		fe_mul(&f, t2, x2, z1);+		fe_add(&f, t3, t1, t2);+		fe_sqr(&f, t3, t3); /* the new z */+		fe_mul(&f, t1, t1, t2);+		fe_mul(&f, t2, x, t3);+		fe_add(&f, t2, t2, t1); /* the new x */++		/* and one of them doubled */+		fe_sqr(&f, x1, x1);+		fe_sqr(&f, z1, z1);+		fe_mul(&f, t1, x1, z1); /* z of the double */+		fe_sqr(&f, x1, x1);+		fe_sqr(&f, z1, z1);+		fe_mul(&f, z1, z1, bb);+		fe_add(&f, x1, x1, z1); /* x of the double */+		memcpy(z1, t1, n * sizeof(limb_t));++		memcpy(x2, t2, n * sizeof(limb_t));+		memcpy(z2, t3, n * sizeof(limb_t));++		cswap(x1, x2, sel, n);+		cswap(z1, z2, sel, n);+	}++	if (fe_is_zero(&f, z1)) {+		ret = 1; /* the multiple is at infinity */+		goto done;+	}+	if (fe_is_zero(&f, z2)) {+		/* the one after it is, so this one is the negation of the point */+		to_be(outx, flen, x, n);+		fe_add(&f, t1, x, y);+		to_be(outy, flen, t1, n);+		ret = 0;+		goto done;+	}++	/* x1/z1 and x2/z2, and the y the ladder does not carry, out of one+	 * inversion: 1/(z1 z2 x) gives each of the three */+	fe_mul(&f, t1, z1, z2);+	fe_mul(&f, t1, t1, x);+	fe_inv(&f, t2, t1, t3);+	{+		limb_t *xa = x1, *xb = x2, *u = t1, *v = t3;++		fe_mul(&f, u, z2, x);+		fe_mul(&f, u, u, t2); /* 1/z1 */+		fe_mul(&f, xa, x1, u);+		fe_mul(&f, v, z1, x);+		fe_mul(&f, v, v, t2); /* 1/z2 */+		fe_mul(&f, xb, x2, v);+		fe_mul(&f, u, z1, z2);+		fe_mul(&f, u, u, t2); /* 1/x */++		fe_add(&f, v, xa, x);          /* x1 + x */+		fe_add(&f, xb, xb, x);         /* x2 + x */+		fe_mul(&f, xb, v, xb);         /* (x1 + x)(x2 + x) */+		fe_sqr(&f, t2, x);+		fe_add(&f, xb, xb, t2);+		fe_add(&f, xb, xb, y);         /* + x^2 + y */+		fe_mul(&f, xb, v, xb);+		fe_mul(&f, xb, xb, u);         /* over x */+		fe_add(&f, xb, xb, y);+		to_be(outx, flen, xa, n);+		to_be(outy, flen, xb, n);+	}+	ret = 0;++done:+	/* freed on the next line, so a plain memset here is a store the+	 * optimizer may drop */+	if (space != NULL) {+		crypton_bzero(space, words * sizeof(limb_t));+		free(space);+	}+	return ret;+}
+ cbits/crypton_f2m.h view
@@ -0,0 +1,31 @@+#ifndef CRYPTON_F2M_H+#define CRYPTON_F2M_H++#include <stdint.h>++/* Multiply a point of a curve over a binary field by a scalar, doing the same+ * work whatever the scalar is.+ *+ * The curve is y^2 + x*y = x^3 + a*x^2 + b over the field of the polynomial+ * fx, and a does not come into it: the ladder carries the x coordinates of+ * two consecutive multiples, and what it takes to add them is b alone.  The+ * point has to be on the curve and to have an x that is not zero -- the one+ * point with none is its own negation, and the caller sees to it.+ *+ * Every number is a big-endian byte string.  The coordinates and b are flen+ * bytes, and fx is the whole polynomial, x^m included, in fxlen.+ *+ * The scalar is walked over every bit of the klen bytes it is given, so its+ * value is hidden but its length is not.+ *+ * Returns 0 with the answer in outx and outy, 1 if the answer is the point at+ * infinity, and -1 for arguments it cannot work with or memory it could not+ * have.+ */+int crypton_f2m_mul(uint8_t *outx, uint8_t *outy,+                    const uint8_t *px, const uint8_t *py,+                    const uint8_t *k, uint32_t klen,+                    const uint8_t *b, uint32_t flen,+                    const uint8_t *fx, uint32_t fxlen);++#endif
cbits/crypton_md4.c view
@@ -48,16 +48,17 @@ #define K3 	0x6ED9EBA1 #define R(a,b,c,d,f,k,s,i) (a = rol32(a + f(b,c,d) + w[i] + k, s)) -static void md4_do_chunk(struct md4_ctx *ctx, uint32_t *buf)+/* The words are read out of the block rather than the block being pointed at+ * as though it were an array of them; see crypton_md5.c. */+static void md4_do_chunk(struct md4_ctx *ctx, const uint8_t *buf) { 	uint32_t a, b, c, d;-#ifdef ARCH_IS_BIG_ENDIAN 	uint32_t w[16];-	cpu_to_le32_array(w, (uint32_t *) buf, 16);-#else-	uint32_t *w = buf;-#endif+	int wi; +	for (wi = 0; wi < 16; wi++)+		w[wi] = load_le32(buf + 4 * wi);+ 	a = ctx->h[0]; b = ctx->h[1]; c = ctx->h[2]; d = ctx->h[3];  	R(a, b, c, d, f1, K1, 3, 0);@@ -125,24 +126,15 @@  	if (index && len >= to_fill) { 		memcpy(ctx->buf + index, data, to_fill);-		md4_do_chunk(ctx, (uint32_t *) ctx->buf);+		md4_do_chunk(ctx, ctx->buf); 		len -= to_fill; 		data += to_fill; 		index = 0; 	} -	if (need_alignment(data, 4)) {-		uint32_t tramp[16];-		ASSERT_ALIGNMENT(tramp, 4);-		for (; len >= 64; len -= 64, data += 64) {-			memcpy(tramp, data, 64);-			md4_do_chunk(ctx, tramp);-		}-	} else {-		/* process as much 64-block as possible */-		for (; len >= 64; len -= 64, data += 64)-			md4_do_chunk(ctx, (uint32_t *) data);-	}+	/* No trampoline: load_le32 does not ask for a boundary. */+	for (; len >= 64; len -= 64, data += 64)+		md4_do_chunk(ctx, data);  	/* append data into buf */ 	if (len)
cbits/crypton_md5.c view
@@ -45,15 +45,22 @@ #define f4(x, y, z)	(y ^ (x | ~z)) #define R(f, a, b, c, d, i, k, s) a += f(b, c, d) + w[i] + k; a = rol32(a, s); a += b -static void md5_do_chunk(struct md5_ctx *ctx, uint32_t *buf)+/* The sixteen words are read out of the block rather than the block being+ * pointed at as though it were an array of them.  A caller's pointer cast to+ * uint32_t * is a pointer the standard says may not exist unless the address+ * is aligned for it, and reading through it is undefined whether or not the+ * machine minds; UndefinedBehaviorSanitizer counted sixty-four of these.+ * load_le32 is a memcpy, which every compiler here turns into the one load+ * the cast used to be, and it takes the endianness with it -- so the two+ * arms this replaces are one. */+static void md5_do_chunk(struct md5_ctx *ctx, const uint8_t *buf) { 	uint32_t a, b, c, d;-#ifdef ARCH_IS_BIG_ENDIAN 	uint32_t w[16];-	cpu_to_le32_array(w, buf, 16);-#else-	uint32_t *w = buf;-#endif+	int wi;++	for (wi = 0; wi < 16; wi++)+		w[wi] = load_le32(buf + 4 * wi); 	a = ctx->h[0]; b = ctx->h[1]; c = ctx->h[2]; d = ctx->h[3];  	R(f1, a, b, c, d, 0, 0xd76aa478, 7);@@ -138,24 +145,16 @@  	if (index && len >= to_fill) { 		memcpy(ctx->buf + index, data, to_fill);-		md5_do_chunk(ctx, (uint32_t *) ctx->buf);+		md5_do_chunk(ctx, ctx->buf); 		len -= to_fill; 		data += to_fill; 		index = 0; 	} -	if (need_alignment(data, 4)) {-		uint32_t tramp[16];-		ASSERT_ALIGNMENT(tramp, 4);-		for (; len >= 64; len -= 64, data += 64) {-			memcpy(tramp, data, 64);-			md5_do_chunk(ctx, tramp);-		}-	} else {-		/* process as much 64-block as possible */-		for (; len >= 64; len -= 64, data += 64)-			md5_do_chunk(ctx, (uint32_t *) data);-	}+	/* No trampoline for a block that is not on a four-byte boundary: the+	 * words are read with load_le32 now, which does not ask. */+	for (; len >= 64; len -= 64, data += 64)+		md5_do_chunk(ctx, data);  	/* append data into buf */ 	if (len)
+ cbits/crypton_memxor.c view
@@ -0,0 +1,29 @@+/*+ * dst = a xor b.+ *+ * Data.ByteArray's xor walks a byte at a time through an IO applicative, and+ * that allocates: fifty bytes of heap for every byte exclusive-ored, which in+ * counter mode cost more than the cipher did.  This is the same operation in+ * one pass of words.+ */++#include <stdint.h>+#include <string.h>++#include "crypton_memxor.h"++void crypton_memxor(uint8_t *dst, const uint8_t *a, const uint8_t *b, uint32_t len)+{+	uint32_t i = 0;++	for (; i + 8 <= len; i += 8) {+		uint64_t x, y;++		memcpy(&x, a + i, 8);+		memcpy(&y, b + i, 8);+		x ^= y;+		memcpy(dst + i, &x, 8);+	}+	for (; i < len; i++)+		dst[i] = a[i] ^ b[i];+}
+ cbits/crypton_memxor.h view
@@ -0,0 +1,8 @@+#ifndef CRYPTON_MEMXOR_H+#define CRYPTON_MEMXOR_H++#include <stdint.h>++void crypton_memxor(uint8_t *dst, const uint8_t *a, const uint8_t *b, uint32_t len);++#endif
+ cbits/crypton_modinv.c view
@@ -0,0 +1,74 @@+/*+ * Inversion modulo an odd number through s2n-bignum, whose routine takes a+ * fixed number of division steps rather than an exponentiation: twenty to+ * thirty times less work than Fermat's little theorem at the sizes here.+ * Measured on an Apple M4, inverting modulo a curve order:+ *+ *              Fermat     this+ *     P-256    6.02 us    0.80+ *     P-384    31.3       1.20+ *     P-521    63.2       2.05+ *+ * It uses no instruction beyond the base architecture on either x86-64 or+ * AArch64, so unlike the rest of the vendored assembly there is nothing to+ * ask the processor first.+ */+#include <string.h>++#include "crypton_modinv.h"++#ifdef CRYPTON_S2N_BIGNUM++extern void bignum_modinv(uint64_t k, uint64_t *z, const uint64_t *a,+                          const uint64_t *b, uint64_t *t);++/* 4096 bits and no more, which covers every modulus that reaches here -- the+ * order of a curve, or a prime factor of an RSA modulus -- and keeps the+ * working space on the stack.  Anything larger is handed back. */+#define MODINV_MAXWORDS 64++int crypton_modinv_sec(uint8_t *z, const uint8_t *a, const uint8_t *m,+                       uint32_t len)+{+	uint64_t aw[MODINV_MAXWORDS], mw[MODINV_MAXWORDS];+	uint64_t zw[MODINV_MAXWORDS], t[3 * MODINV_MAXWORDS];+	uint32_t k = (len + 7) / 8;+	uint32_t i;++	/* An even modulus is the one case it answers without saying it+	 * cannot: it returns a number that is not an inverse rather than+	 * failing, so keep it away from here.  Every caller's modulus is odd. */+	if (len == 0 || k > MODINV_MAXWORDS || (m[len - 1] & 1) == 0)+		return 1;++	for (i = 0; i < k; i++) {+		aw[i] = 0;+		mw[i] = 0;+	}+	for (i = 0; i < len; i++) {+		uint32_t pos = len - 1 - i;++		aw[pos / 8] |= (uint64_t)a[i] << (8 * (pos % 8));+		mw[pos / 8] |= (uint64_t)m[i] << (8 * (pos % 8));+	}++	bignum_modinv(k, zw, aw, mw, t);++	for (i = 0; i < len; i++)+		z[len - 1 - i] = (uint8_t)(zw[i / 8] >> (8 * (i % 8)));+	return 0;+}++#else++int crypton_modinv_sec(uint8_t *z, const uint8_t *a, const uint8_t *m,+                       uint32_t len)+{+	(void)z;+	(void)a;+	(void)m;+	(void)len;+	return 1;+}++#endif
+ cbits/crypton_modinv.h view
@@ -0,0 +1,22 @@+#ifndef CRYPTON_MODINV_H+#define CRYPTON_MODINV_H++#include <stdint.h>++/* z = a^-1 mod m, all three big-endian byte strings of len bytes.+ *+ * Returns 0 with the answer in z, and 1 without touching z when it will not+ * do this one: the assembly it needs is not built, the modulus is even, or+ * the numbers are larger than it keeps room for.  A 1 is not an error, it is+ * "ask something else".+ *+ * The answer is not checked here.  When a has no inverse the routine+ * underneath returns something that is not one rather than saying so, so the+ * caller has to multiply out and look -- which is what Crypto.Number.+ * ModArithmetic.inverseSafe already did for the exponentiation this+ * replaces.+ */+int crypton_modinv_sec(uint8_t *z, const uint8_t *a, const uint8_t *m,+                       uint32_t len);++#endif
cbits/crypton_pbkdf2.c view
@@ -46,6 +46,7 @@  /* Internal function/type names for hash-specific things. */ #define HMAC_CTX(_name) HMAC_ ## _name ## _ctx+#define DIGEST_FITS(_name) pbkdf2_ ## _name ## _digest_fits_in_a_block #define HMAC_INIT(_name) HMAC_ ## _name ## _init #define HMAC_UPDATE(_name) HMAC_ ## _name ## _update #define HMAC_FINAL(_name) HMAC_ ## _name ## _final@@ -79,6 +80,16 @@  */ #define DECL_PBKDF2(_name, _blocksz, _hashsz, _ctx,                           \                     _init, _update, _xform, _final, _xcpy, _xtract, _xxor)    \+  /* HMAC_INIT below shortens a key longer than the block by hashing it,     \+   * which writes _hashsz bytes into a buffer of _blocksz.  An instantiation \+   * whose digest is larger than its block would overflow that buffer, and   \+   * would do it before any check inside the function could say so -- which  \+   * is where the check used to be.  Refuse such an instantiation here       \+   * instead, in front of the person writing it.  The three below are        \+   * SHA-1, SHA-256 and SHA-512, whose digests are 20, 32 and 64 bytes       \+   * against blocks of 64, 64 and 128. */                                    \+  typedef char DIGEST_FITS(_name)[(_hashsz) <= (_blocksz) ? 1 : -1];          \+                                                                              \   typedef struct {                                                            \     _ctx inner;                                                               \     _ctx outer;                                                               \@@ -101,9 +112,6 @@       nkey = _hashsz;                                                         \     }                                                                         \                                                                               \-    /* Standard doesn't cover case where blocksz < hashsz. */                 \-    assert(nkey <= _blocksz);                                                 \-                                                                              \     /* Right zero-pad short keys. */                                          \     if (k != key)                                                             \       memcpy(k, key, nkey);                                                   \@@ -192,7 +200,16 @@                      uint8_t *out, size_t nout)                               \   {                                                                           \     assert(iterations);                                                       \-    assert(out && nout);                                                      \+    assert(out);                                                              \+                                                                              \+    /* Zero bytes of derived key is zero bytes of work.  RFC 8018 asks for a  \+     * positive dkLen and the loop below would write a block regardless, so   \+     * this used to be `assert(out && nout)` -- which aborts the process, in  \+     * a library built without NDEBUG, on a length the caller chose.          \+     * Crypto.KDF.PBKDF2's own tryGenerate returns an empty result for this,  \+     * so return and let the two agree. */                                    \+    if (nout == 0)                                                            \+      return;                                                                 \                                                                               \     /* Starting point for inner loop. */                                      \     HMAC_CTX(_name) ctx;                                                      \
cbits/crypton_poly1305.c view
@@ -39,8 +39,52 @@ #include "crypton_bitfn.h" #include "crypton_align.h" ++/*+ * Poly1305 from CRYPTOGAMS, in cbits/asm/poly1305-armv8-*.S and+ * cbits/asm/poly1305-x86_64-*.S, which is the whole of the arithmetic+ * rather than a bulk loop bolted to the side: it keeps its own accumulator+ * -- in base 2^64 while the message is short and base 2^26 once the vector+ * loop has started, switching between the two itself -- and its own powers+ * of r, so what is left here is the buffering of partial blocks.+ *+ * 'padbit' is the high bit above each block, which is set for every block+ * of the message and clear for the padded last one.+ */+#if (defined(WITH_ARMV8_POLY1305_ASM) && !defined(__AARCH64EB__)) \+    || defined(WITH_X86_POLY1305_ASM)+#define POLY1305_ASM 1+#include "crypton_cpu.h"++typedef void (*poly1305_blocks_f)(void *ctx, const uint8_t *inp, size_t len,+                                  uint32_t padbit);+typedef void (*poly1305_emit_f)(void *ctx, uint8_t mac[16],+                                const uint32_t nonce[4]);++int crypton_poly1305_asm_init(void *ctx, const uint8_t key[16], void *func[2]);++/*+ * Initialisation hands back the pair of functions its own dispatch would+ * use, the vector entry points themselves being local to the module.  They+ * are the same for every context, so they are kept here rather than in each+ * one; two threads racing to fill them write the same values.+ */+static poly1305_blocks_f asm_blocks;+static poly1305_emit_f asm_emit;+#endif+++#ifdef POLY1305_ASM+ static void poly1305_do_chunk(poly1305_ctx *ctx, uint8_t *data, int blocks, int final) {+	asm_blocks(ctx->st.opaque, data, (size_t) blocks * 16, final ? 0 : 1);+}++#else++static void poly1305_do_chunk(poly1305_ctx *ctx, uint8_t *data, int blocks, int final)+{ 	/* following is a cleanup copy of code available poly1305-donna */ 	const uint32_t hibit = (final) ? 0 : (1 << 24); /* 1 << 128 */ 	uint32_t r0,r1,r2,r3,r4;@@ -49,9 +93,10 @@ 	uint64_t d0,d1,d2,d3,d4; 	uint32_t c; + 	/* load r[i], h[i] */-	h0 = ctx->h[0]; h1 = ctx->h[1]; h2 = ctx->h[2]; h3 = ctx->h[3]; h4 = ctx->h[4];-	r0 = ctx->r[0]; r1 = ctx->r[1]; r2 = ctx->r[2]; r3 = ctx->r[3]; r4 = ctx->r[4];+	h0 = ctx->st.limb.h[0]; h1 = ctx->st.limb.h[1]; h2 = ctx->st.limb.h[2]; h3 = ctx->st.limb.h[3]; h4 = ctx->st.limb.h[4];+	r0 = ctx->st.limb.r[0]; r1 = ctx->st.limb.r[1]; r2 = ctx->st.limb.r[2]; r3 = ctx->st.limb.r[3]; r4 = ctx->st.limb.r[4];  	/* s[i] = r[i] * 5 */ 	s1 = r1 * 5; s2 = r2 * 5; s3 = r3 * 5; s4 = r4 * 5;@@ -81,21 +126,37 @@ 	}  	/* store h[i] */-	ctx->h[0] = h0; ctx->h[1] = h1; ctx->h[2] = h2; ctx->h[3] = h3; ctx->h[4] = h4;+	ctx->st.limb.h[0] = h0; ctx->st.limb.h[1] = h1; ctx->st.limb.h[2] = h2; ctx->st.limb.h[3] = h3; ctx->st.limb.h[4] = h4; } +#endif+ void crypton_poly1305_init(poly1305_ctx *ctx, poly1305_key *key) { 	uint8_t *k = (uint8_t *) key;  	memset(ctx, 0, sizeof(poly1305_ctx)); -	ctx->r[0] = (load_le32(&k[ 0])     ) & 0x3ffffff;-	ctx->r[1] = (load_le32(&k[ 3]) >> 2) & 0x3ffff03;-	ctx->r[2] = (load_le32(&k[ 6]) >> 4) & 0x3ffc0ff;-	ctx->r[3] = (load_le32(&k[ 9]) >> 6) & 0x3f03fff;-	ctx->r[4] = (load_le32(&k[12]) >> 8) & 0x00fffff;+#ifdef POLY1305_ASM+	{+		void *func[2]; +#ifdef CRYPTON_X86_ASM+		/* what the module dispatches on, which it reads directly */+		crypton_x86_ia32cap_resolve();+#endif+		crypton_poly1305_asm_init(ctx->st.opaque, k, func);+		asm_blocks = (poly1305_blocks_f) func[0];+		asm_emit = (poly1305_emit_f) func[1];+	}+#else+	ctx->st.limb.r[0] = (load_le32(&k[ 0])     ) & 0x3ffffff;+	ctx->st.limb.r[1] = (load_le32(&k[ 3]) >> 2) & 0x3ffff03;+	ctx->st.limb.r[2] = (load_le32(&k[ 6]) >> 4) & 0x3ffc0ff;+	ctx->st.limb.r[3] = (load_le32(&k[ 9]) >> 6) & 0x3f03fff;+	ctx->st.limb.r[4] = (load_le32(&k[12]) >> 8) & 0x00fffff;+#endif+ 	ctx->pad[0] = load_le32(&k[16]); 	ctx->pad[1] = load_le32(&k[20]); 	ctx->pad[2] = load_le32(&k[24]);@@ -134,11 +195,6 @@  void crypton_poly1305_finalize(poly1305_mac mac8, poly1305_ctx *ctx) {-	uint32_t h0,h1,h2,h3,h4,c;-	uint32_t g0,g1,g2,g3,g4;-	uint64_t f;-	uint32_t mask;-	uint32_t *mac = (uint32_t *) mac8; 	int i;  	if (ctx->index) {@@ -149,10 +205,22 @@ 		poly1305_do_chunk(ctx, ctx->buf, 1, 1); 	} +#ifdef POLY1305_ASM+	/* the carry, the reduction and the addition of the second half of+	 * the key are the assembly's, since the accumulator is its own */+	asm_emit(ctx->st.opaque, mac8, ctx->pad);+#else+	{+	uint32_t h0,h1,h2,h3,h4,c;+	uint32_t g0,g1,g2,g3,g4;+	uint64_t f;+	uint32_t mask;+	uint32_t *mac = (uint32_t *) mac8;+ 	/* following is a cleanup copy of code available poly1305-donna */  	/* fully carry h */-	h0 = ctx->h[0]; h1 = ctx->h[1]; h2 = ctx->h[2]; h3 = ctx->h[3]; h4 = ctx->h[4];+	h0 = ctx->st.limb.h[0]; h1 = ctx->st.limb.h[1]; h2 = ctx->st.limb.h[2]; h3 = ctx->st.limb.h[3]; h4 = ctx->st.limb.h[4];  	             c = h1 >> 26; h1 = h1 & 0x3ffffff; 	h2 +=     c; c = h2 >> 26; h2 = h2 & 0x3ffffff;@@ -200,4 +268,6 @@  	f = (uint64_t)h3 + ctx->pad[3] + (f >> 32); 	mac[3] = cpu_to_le32((uint32_t) f);+	}+#endif }
cbits/crypton_poly1305.h view
@@ -30,11 +30,24 @@ #ifndef CRYPTON_POLY1305_H # define CRYPTON_POLY1305_H -/* 8*8+1*16+1*4 = 84 */+/*+ * Either the 26-bit limbs the C implementation works in, or the state the+ * assembly keeps: its accumulator, in whichever base it is using at the+ * time, the clamped key, and the powers of that laid out for the four-way+ * vector loop, which together come to exactly 192 bytes -- OpenSSL allots+ * the same for the same thing.+ *+ * size = 192+16+4+16 = 228, 232 with the alignment the union asks for+ */ typedef struct {-	uint32_t r[5];-	uint32_t h[5];+	union {+		struct {+			uint32_t r[5];+			uint32_t h[5];+		} limb;+		uint64_t opaque[24];+	} st; 	uint32_t pad[4]; 	uint32_t index; 	uint8_t buf[16]; /* previous partial block */
+ cbits/crypton_powm.c view
@@ -0,0 +1,311 @@+/*+ * Modular exponentiation that does the same work whatever the exponent is.+ *+ * The exponent is walked four bits at a time: four squarings and one+ * multiplication by a small power of the base, taken from a table of sixteen.+ * The table is read by touching all sixteen entries and keeping one of them+ * with a mask, so the address stream does not follow the exponent, and the+ * multiplication itself is Montgomery's, whose only conditional step -- the+ * subtraction at the end -- is also done with a mask.+ *+ * So every window costs the same four squarings, the same multiplication and+ * the same sixteen reads, and nothing here branches on, or indexes memory+ * with, anything derived from the exponent.+ *+ * What is still visible is how many bytes the caller passed: the loop runs+ * over every bit of them, so the exponent's value is hidden but its length is+ * not.  GMP's mpz_powm_sec, which this replaces on the GHCs that no longer+ * offer it, hides the same amount.+ */+#include <stdlib.h>+#include <crypton_bignum.h>+#include <crypton_powm.h>+#include <crypton_bzero.h>++/*+ * At RSA sizes on x86-64, the Montgomery multiplication below is the whole+ * cost, and s2n-bignum's is twice as fast because the C cannot form the two+ * carry chains ADCX and ADOX give.  The window, the table and its masked+ * scan are unchanged: only the multiply and the square are swapped, and+ * only for the sizes s2n-bignum has a Karatsuba multiplication for.+ *+ * Not on AArch64, where the C measures 5% faster than the assembly.+ * See cbits/s2n/README.md.+ */+#if defined(CRYPTON_S2N_BIGNUM) && defined(__x86_64__)+#define CRYPTON_POWM_S2N 1+#include <crypton_cpu.h>++extern void bignum_kmul_16_32(uint64_t *z, const uint64_t *x,+                              const uint64_t *y, uint64_t *t);+extern void bignum_ksqr_16_32(uint64_t *z, const uint64_t *x, uint64_t *t);+extern void bignum_kmul_32_64(uint64_t *z, const uint64_t *x,+                              const uint64_t *y, uint64_t *t);+extern void bignum_ksqr_32_64(uint64_t *z, const uint64_t *x, uint64_t *t);+extern uint64_t bignum_emontredc_8n(uint64_t k, uint64_t *z,+                                    const uint64_t *m, uint64_t w);++/* 16 limbs is 1024 bits and 32 is 2048: the halves a CRT exponentiation+ * works in for RSA-2048 and RSA-4096, and the whole thing without CRT.  The+ * reduction wants ADX, so the answer is a run-time one. */+static int powm_s2n_usable(uint32_t n)+{+	return (n == 16 || n == 32)+	    && (crypton_x86_simd_features() & CRYPTON_X86_ADX) != 0;+}++/* The scratch the widest of them asks for, in multiples of n: kmul_32_64+ * wants 96 limbs for n = 32. */+#define POWM_S2N_SCRATCH 3++/* bignum_emontredc_8n leaves the result in the top half of z with one more+ * bit as its return value, and what is there is under twice the modulus --+ * the same place mont_reduce ends up, and finished the same way. */+static void powm_s2n_finish(limb_t *r, limb_t *z, const limb_t *m,+                            limb_t carry, uint32_t n)+{+	limb_t borrow = sub_n(r, z + n, m, n);+	limb_t take = carry | (borrow ^ 1);++	select_n(r, r, z + n, take & 1, n);+}++static void powm_s2n_mul(limb_t *r, const limb_t *a, const limb_t *b,+                         const limb_t *m, limb_t n0, uint32_t n, limb_t *z,+                         limb_t *scratch)+{+	if (n == 16)+		bignum_kmul_16_32(z, a, b, scratch);+	else+		bignum_kmul_32_64(z, a, b, scratch);+	powm_s2n_finish(r, z, m, bignum_emontredc_8n(n, z, m, n0), n);+}++static void powm_s2n_sqr(limb_t *r, const limb_t *a, const limb_t *m,+                         limb_t n0, uint32_t n, limb_t *z, limb_t *scratch)+{+	if (n == 16)+		bignum_ksqr_16_32(z, a, scratch);+	else+		bignum_ksqr_32_64(z, a, scratch);+	powm_s2n_finish(r, z, m, bignum_emontredc_8n(n, z, m, n0), n);+}+#else+#define POWM_S2N_SCRATCH 0+#endif++/* One or the other, decided once per call */+static void powm_mul(limb_t *r, const limb_t *a, const limb_t *b,+                     const limb_t *m, limb_t n0, uint32_t n, limb_t *t,+                     limb_t *scratch, int s2n)+{+#ifdef CRYPTON_POWM_S2N+	if (s2n) {+		powm_s2n_mul(r, a, b, m, n0, n, t, scratch);+		return;+	}+#else+	(void)scratch;+	(void)s2n;+#endif+	mont_mul(r, a, b, m, n0, n, t);+}++static void powm_sqr(limb_t *r, const limb_t *a, const limb_t *m, limb_t n0,+                     uint32_t n, limb_t *t, limb_t *scratch, int s2n)+{+#ifdef CRYPTON_POWM_S2N+	if (s2n) {+		powm_s2n_sqr(r, a, m, n0, n, t, scratch);+		return;+	}+#else+	(void)scratch;+	(void)s2n;+#endif+	mont_sqr(r, a, m, n0, n, t);+}++/* Four bits of exponent per window, so a table of sixteen and no leftover+ * bits: a byte holds exactly two windows.+ *+ * Five was written and measured, and is not here.  A wider window saves+ * multiplications -- 205 of them against 256 at 1024 bits, with the same+ * 1024 squarings -- and pays for it in the masked scan of a table twice as+ * long, and which way that comes out depends on the machine and on which+ * multiplication is running: 3.5% better on an Apple M4, about 1% worse on+ * an older x86-64, and 7% worse anywhere s2n-bignum's multiplication is+ * used, since that makes the scan the expensive half.  Six measured level+ * with five on the M4 and seven worse.  What would make a wider window pay+ * everywhere is a cheaper scan, not a wider window. */+#define WINDOW_BITS 4+#define TABLE_SIZE (1 << WINDOW_BITS)++/* The masked scan of the table: every entry is read and a mask keeps the one+ * wanted, so that the address stream does not follow the exponent.  At+ * RSA-2048's CRT size that is two kilobytes read per window, and the window+ * loop runs 256 times per exponentiation, which is why it is worth a vector+ * register: removing the scan altogether measures 11% of an exponentiation+ * where s2n-bignum's multiplication runs, and the AVX2 form below gets+ * essentially all of it. */+static void scan_table(limb_t *sel, const limb_t *table, uint32_t n, limb_t w)+{+	uint32_t k, l;++	memset(sel, 0, n * sizeof(limb_t));+	for (k = 0; k < TABLE_SIZE; k++) {+		limb_t mask = eq_mask(k, w);++		for (l = 0; l < n; l++)+			sel[l] |= table[k * n + l] & mask;+	}+}++#if defined(__x86_64__) && defined(WITH_TARGET_ATTRIBUTES) && LIMB_BITS == 64+#define CRYPTON_POWM_SCAN_AVX2 1+#include <crypton_cpu.h>+#include <immintrin.h>++/* The same scan four limbs at a time.  The sixteen masks are worked out+ * once; after that each register of the answer is one pass over the table's+ * column, reading every entry exactly as the scalar form does. */+__attribute__((target("avx2")))+static void scan_table_avx2(limb_t *sel, const limb_t *table, uint32_t n,+                            limb_t w)+{+	__m256i masks[TABLE_SIZE];+	uint32_t k, l;++	for (k = 0; k < TABLE_SIZE; k++)+		masks[k] = _mm256_cmpeq_epi64(+			_mm256_set1_epi64x((long long) k),+			_mm256_set1_epi64x((long long) w));++	for (l = 0; l + 4 <= n; l += 4) {+		__m256i acc = _mm256_setzero_si256();++		for (k = 0; k < TABLE_SIZE; k++) {+			__m256i v = _mm256_loadu_si256(+				(const __m256i *) (table + k * n + l));++			acc = _mm256_or_si256(acc,+			                      _mm256_and_si256(v, masks[k]));+		}+		_mm256_storeu_si256((__m256i *) (sel + l), acc);+	}++	/* a modulus whose limbs do not come in fours ends here */+	for (; l < n; l++) {+		limb_t v = 0;++		for (k = 0; k < TABLE_SIZE; k++)+			v |= table[k * n + l] & eq_mask(k, w);+		sel[l] = v;+	}+}+#endif++int crypton_powm_sec(uint8_t *out,+                     const uint8_t *base, uint32_t baselen,+                     const uint8_t *exp, uint32_t explen,+                     const uint8_t *mod, uint32_t modlen)+{+	uint32_t n = (modlen + LIMB_BYTES - 1) / LIMB_BYTES;+	uint32_t words = (TABLE_SIZE + 7 + POWM_S2N_SCRATCH) * n;+	limb_t *space, *m, *r2, *acc, *sel, *prod, *table, *t, *scratch, n0;+	uint32_t i, j, k;+	int s2n = 0;+#ifdef CRYPTON_POWM_SCAN_AVX2+	int avx2 = (crypton_x86_simd_features() & CRYPTON_X86_AVX2) != 0;+#endif++	if (modlen == 0 || n == 0 || (mod[modlen - 1] & 1) == 0)+		return 1;++	/* the table, five more n-limb numbers and one of 2n */+	space = calloc(words, sizeof(limb_t));+	if (space == NULL)+		return 1;+	m = space;+	r2 = m + n;+	acc = r2 + n;+	sel = acc + n;+	prod = sel + n;+	t = prod + n;+	table = t + 2 * n;+	scratch = table + TABLE_SIZE * n;++#ifdef CRYPTON_POWM_S2N+	s2n = powm_s2n_usable(n);+#endif++	if (from_be(m, n, mod, modlen) != 0)+		goto fail;+	n0 = mont_n0(m[0]);+	mont_r2(r2, m, n0, n, t);++	/* table[k] = base^k in Montgomery form, and table[0] = 1 there */+	memset(table, 0, n * sizeof(limb_t));+	table[0] = 1;+	powm_mul(acc, table, r2, m, n0, n, t, scratch, s2n);+	memcpy(table, acc, n * sizeof(limb_t));++	if (from_be(sel, n, base, baselen) != 0)+		goto fail;+	powm_mul(table + n, sel, r2, m, n0, n, t, scratch, s2n);+	for (k = 2; k < TABLE_SIZE; k++)+		powm_mul(table + k * n, table + (k - 1) * n, table + n, m, n0, n,+			         t, scratch, s2n);++	memcpy(acc, table, n * sizeof(limb_t));++	for (i = explen * 2; i > 0; i--) {+		uint32_t nib = i - 1;+		limb_t w = (exp[explen - 1 - nib / 2] >> (4 * (nib % 2))) & 0xf;++		/* squaring into the other buffer and swapping the two saves a+		 * copy of the modulus' width every time; which of the three+		 * buffers a pointer names is nobody's secret */+		for (j = 0; j < WINDOW_BITS; j++) {+			limb_t *swap;++			powm_sqr(sel, acc, m, n0, n, t, scratch, s2n);+			swap = acc;+			acc = sel;+			sel = swap;+		}++#ifdef CRYPTON_POWM_SCAN_AVX2+		if (avx2)+			scan_table_avx2(sel, table, n, w);+		else+#endif+			scan_table(sel, table, n, w);+		powm_mul(prod, acc, sel, m, n0, n, t, scratch, s2n);+		{+			limb_t *swap = acc;++			acc = prod;+			prod = swap;+		}+	}++	/* out of Montgomery form */+	memset(sel, 0, n * sizeof(limb_t));+	sel[0] = 1;+	powm_mul(prod, acc, sel, m, n0, n, t, scratch, s2n);+	to_be(out, modlen, prod, n);++	/* nothing here is the caller's secret, but the exponent's bits passed+	 * through the accumulators.  crypton_bzero rather than memset, since+	 * this memory is freed on the next line and a store to memory about to+	 * die is one an optimizer may drop. */+	crypton_bzero(space, words * sizeof(limb_t));+	free(space);+	return 0;++fail:+	crypton_bzero(space, words * sizeof(limb_t));+	free(space);+	return 1;+}
+ cbits/crypton_powm.h view
@@ -0,0 +1,23 @@+#ifndef CRYPTON_POWM_H+#define CRYPTON_POWM_H++#include <stdint.h>++/* Modular exponentiation whose work does not depend on the exponent's bits.+ *+ * All three numbers are big-endian byte strings.  The modulus has to be odd+ * and at least one byte, and the base has to be smaller than it: the caller+ * reduces, which it can do in whatever way it likes, because in this library+ * the base is always a public value.+ *+ * The result is written to out, which holds modlen bytes.+ *+ * Returns 0 on success, and nonzero if the modulus is even or memory ran out,+ * in which case out is untouched.+ */+int crypton_powm_sec(uint8_t *out,+                     const uint8_t *base, uint32_t baselen,+                     const uint8_t *exp, uint32_t explen,+                     const uint8_t *mod, uint32_t modlen);++#endif
cbits/crypton_ripemd.c view
@@ -57,16 +57,17 @@ #define R(a, b, c, d, e, f, k, i, s)	\ 	a += f(b, c, d) + w[i] + k; a = rol32(a, s) + e; c = rol32(c, 10) -static void ripemd160_do_chunk(struct ripemd160_ctx *ctx, uint32_t *buf)+/* The words are read out of the block rather than the block being pointed at+ * as though it were an array of them; see crypton_md5.c. */+static void ripemd160_do_chunk(struct ripemd160_ctx *ctx, const uint8_t *buf) { 	uint32_t a1, b1, c1, d1, e1, a2, b2, c2, d2, e2;-#ifdef ARCH_IS_BIG_ENDIAN 	uint32_t w[16];-	cpu_to_le32_array(w, buf, 16);-#else-	uint32_t *w = buf;-#endif+	int wi; +	for (wi = 0; wi < 16; wi++)+		w[wi] = load_le32(buf + 4 * wi);+ 	a1 = ctx->h[0]; b1 = ctx->h[1]; c1 = ctx->h[2]; d1 = ctx->h[3]; e1 = ctx->h[4]; 	a2 = ctx->h[0]; b2 = ctx->h[1]; c2 = ctx->h[2]; d2 = ctx->h[3]; e2 = ctx->h[4]; @@ -260,24 +261,15 @@ 	ctx->sz += len; 	if (index && len >= to_fill) { 		memcpy(ctx->buf + index, data, to_fill);-		ripemd160_do_chunk(ctx, (uint32_t *) ctx->buf);+		ripemd160_do_chunk(ctx, ctx->buf); 		len -= to_fill; 		data += to_fill; 		index = 0; 	} -	if (need_alignment(data, 4)) {-		uint32_t tramp[16];-		ASSERT_ALIGNMENT(tramp, 4);-		for (; len >= 64; len -= 64, data += 64) {-			memcpy(tramp, data, 64);-			ripemd160_do_chunk(ctx, tramp);-		}-	} else {-		/* process as much 64-block as possible */-		for (; len >= 64; len -= 64, data += 64)-			ripemd160_do_chunk(ctx, (uint32_t *) data);-	}+	/* No trampoline: load_le32 does not ask for a boundary. */+	for (; len >= 64; len -= 64, data += 64)+		ripemd160_do_chunk(ctx, data);  	/* append data into buf */ 	if (len)
cbits/crypton_salsa.c view
@@ -59,7 +59,7 @@ 		QR (x15,x12,x13,x14); \ 	} -static void salsa_core(int rounds, block *out, const crypton_salsa_state *in)+static void salsa_core(int rounds, crypton_salsa_block *out, const crypton_salsa_state *in) { 	uint32_t x0, x1, x2, x3, x4, x5, x6, x7, x8, x9, x10, x11, x12, x13, x14, x15; 	int i;@@ -94,7 +94,7 @@ 	out->d[15] = cpu_to_le32(x15); } -void crypton_salsa_core_xor(int rounds, block *out, block *in)+void crypton_salsa_core_xor(int rounds, crypton_salsa_block *out, crypton_salsa_block *in) { 	uint32_t x0, x1, x2, x3, x4, x5, x6, x7, x8, x9, x10, x11, x12, x13, x14, x15; 	int i;@@ -165,7 +165,7 @@  void crypton_salsa_combine(uint8_t *dst, crypton_salsa_context *ctx, const uint8_t *src, uint32_t bytes) {-	block out;+	crypton_salsa_block out; 	crypton_salsa_state *st; 	int i; @@ -225,7 +225,7 @@ void crypton_salsa_generate(uint8_t *dst, crypton_salsa_context *ctx, uint32_t bytes) { 	crypton_salsa_state *st;-	block out;+	crypton_salsa_block out; 	int i;  	if (!bytes)@@ -252,7 +252,7 @@ 		/* xor new 64-bytes chunks and store the left over if any */ 		for (; bytes >= 64; bytes -= 64, dst += 64) { 			/* generate new chunk and update state */-			salsa_core(ctx->nb_rounds, (block *) dst, st);+			salsa_core(ctx->nb_rounds, (crypton_salsa_block *) dst, st); 			st->d[8] += 1; 			if (st->d[8] == 0) 				st->d[9] += 1;
cbits/crypton_salsa.h view
@@ -34,9 +34,9 @@ 	uint64_t q[8]; 	uint32_t d[16]; 	uint8_t  b[64];-} block;+} crypton_salsa_block; -typedef block crypton_salsa_state;+typedef crypton_salsa_block crypton_salsa_state;  typedef struct { 	crypton_salsa_state st;@@ -47,7 +47,7 @@ } crypton_salsa_context;  /* for scrypt */-void crypton_salsa_core_xor(int rounds, block *out, block *in);+void crypton_salsa_core_xor(int rounds, crypton_salsa_block *out, crypton_salsa_block *in);  void crypton_salsa_init_core(crypton_salsa_state *st, uint32_t keylen, const uint8_t *key, uint32_t ivlen, const uint8_t *iv); void crypton_salsa_init(crypton_salsa_context *ctx, uint8_t nb_rounds, uint32_t keylen, const uint8_t *key, uint32_t ivlen, const uint8_t *iv);
cbits/crypton_scrypt.c view
@@ -37,10 +37,10 @@ 	array_copy32(X, &in[(2 * r - 1) * 16], 16);  	for (i = 0; i < 2 * r; i += 2) {-		crypton_salsa_core_xor(8, (block *) X, (block *) &in[i*16]);+		crypton_salsa_core_xor(8, (crypton_salsa_block *) X, (crypton_salsa_block *) &in[i*16]); 		array_copy32(&out[i * 8], X, 16); -		crypton_salsa_core_xor(8, (block *) X, (block *) &in[i*16+16]);+		crypton_salsa_core_xor(8, (crypton_salsa_block *) X, (crypton_salsa_block *) &in[i*16+16]); 		array_copy32(&out[i * 8 + r * 16], X, 16); 	} }
cbits/crypton_sha1.c view
@@ -26,7 +26,54 @@ #include "crypton_sha1.h" #include "crypton_bitfn.h" #include "crypton_align.h"+/*+ * AArch64 can do four rounds at a time with the SHA-1 instructions; see+ * sha1_armv8.c.  They are optional in ARMv8.0, so ask before using them.+ * Two threads racing to answer here both write the same value.+ */+#ifdef WITH_ARMV8_SHA1+extern void crypton_sha1_armv8_do_chunk(uint32_t state[5], const uint8_t buf[64]);+extern void crypton_sha1_armv8_do_chunks(uint32_t state[5], const uint8_t *data,+                                         uint32_t blocks);+extern int crypton_sha1_armv8_available(void); +#ifdef WITH_ARMV8_SHA1_ASM+/*+ * SHA-1 from CRYPTOGAMS, in cbits/asm/sha1-armv8-*.S.  The instructions are+ * the ones the intrinsics beside it use; what the module does with them is+ * schedule the message schedule of the next four rounds against the rounds+ * of this one, which a C function cannot be made to do.+ *+ * The entry point for processors that have the instructions is not+ * exported, so the module's own dispatch is what picks it, and the answer+ * to the question this file already asks goes into the word that dispatch+ * reads.+ */+#define SHA1_ASM 1+#include "crypton_cpu.h"+extern void crypton_sha1_asm_block_data_order(uint32_t state[5],+                                              const void *data, size_t blocks);+#endif++/* Resolved before there is a second thread; see the constructor in+ * cbits/crypton_aes.c for why.  The test below then only ever reads. */+static int sha1_use_armv8 = -1;++__attribute__((constructor))+static void sha1_armv8_ctor(void)+{+	sha1_use_armv8 = crypton_sha1_armv8_available();+#ifdef SHA1_ASM+	if (sha1_use_armv8)+		crypton_armcap_P |= CRYPTON_ARMCAP_SHA1;+#endif+}+#endif++#ifdef WITH_X86_SHA_NI+#include "crypton_cpu.h"+#endif+ void crypton_sha1_init(struct sha1_ctx *ctx) { 	memset(ctx, 0, sizeof(*ctx));@@ -54,11 +101,13 @@ #define M(i)  (w[i & 0x0f] = rol32(w[i & 0x0f] ^ w[(i - 14) & 0x0f] \               ^ w[(i - 8) & 0x0f] ^ w[(i - 3) & 0x0f], 1)) -static inline void sha1_do_chunk(struct sha1_ctx *ctx, uint32_t *buf)+/* The words are read out of the block rather than the block being pointed at+ * as though it were an array of them; see crypton_md5.c. */+static void sha1_do_chunk_generic(struct sha1_ctx *ctx, const uint8_t *buf) { 	uint32_t a, b, c, d, e; 	uint32_t w[16];-#define CPY(i)	w[i] = be32_to_cpu(buf[i])+#define CPY(i)	w[i] = load_be32(buf + 4 * (i)) 	CPY(0); CPY(1); CPY(2); CPY(3); CPY(4); CPY(5); CPY(6); CPY(7); 	CPY(8); CPY(9); CPY(10); CPY(11); CPY(12); CPY(13); CPY(14); CPY(15); #undef CPY@@ -156,6 +205,50 @@ 	ctx->h[4] += e; } +#ifdef WITH_X86_SHA_NI+/*+ * x86 can do four rounds at a time with the SHA extensions; see sha1_x86.c.+ * They arrived long after the x86-64 baseline, so ask before using them.+ * Two threads racing to answer here both write the same value.+ */+extern void crypton_sha1_x86_do_chunk(uint32_t state[5], const uint8_t buf[64]);+extern void crypton_sha1_x86_do_chunks(uint32_t state[5], const uint8_t *data,+                                       uint32_t blocks);++static int sha1_use_x86 = -1;+#endif++static inline void sha1_do_chunk(struct sha1_ctx *ctx, const uint8_t *buf)+{+#ifdef WITH_ARMV8_SHA1+	if (sha1_use_armv8 < 0) {+		sha1_use_armv8 = crypton_sha1_armv8_available();+#ifdef SHA1_ASM+		if (sha1_use_armv8)+			crypton_armcap_P |= CRYPTON_ARMCAP_SHA1;+#endif+	}+	if (sha1_use_armv8) {+#ifdef SHA1_ASM+		crypton_sha1_asm_block_data_order(ctx->h, buf, 1);+#else+		crypton_sha1_armv8_do_chunk(ctx->h, buf);+#endif+		return;+	}+#endif+#ifdef WITH_X86_SHA_NI+	if (sha1_use_x86 < 0)+		sha1_use_x86 =+		    (crypton_x86_simd_features() & CRYPTON_X86_SHA_NI) != 0;+	if (sha1_use_x86) {+		crypton_sha1_x86_do_chunk(ctx->h, buf);+		return;+	}+#endif+	sha1_do_chunk_generic(ctx, buf);+}+ void crypton_sha1_update(struct sha1_ctx *ctx, const uint8_t *data, uint32_t len) { 	uint32_t index, to_fill;@@ -168,24 +261,54 @@ 	/* process partial buffer if there's enough data to make a block */ 	if (index && len >= to_fill) { 		memcpy(ctx->buf + index, data, to_fill);-		sha1_do_chunk(ctx, (uint32_t *) ctx->buf);+		sha1_do_chunk(ctx, ctx->buf); 		len -= to_fill; 		data += to_fill; 		index = 0; 	} -	if (need_alignment(data, 4)) {-		uint32_t tramp[16];-		ASSERT_ALIGNMENT(tramp, 4);-		for (; len >= 64; len -= 64, data += 64) {-			memcpy(tramp, data, 64);-			sha1_do_chunk(ctx, tramp);-		}-	} else {-		/* process as much 64-block as possible */-		for (; len >= 64; len -= 64, data += 64)-			sha1_do_chunk(ctx, (uint32_t *) data);+	/*+	 * Where there are instructions for this, the whole run of blocks+	 * goes over at once: the state then stays in registers from one+	 * block to the next, and the message is read as bytes, so neither+	 * the alignment nor the copy below is wanted.+	 */+#ifdef WITH_ARMV8_SHA1+	if (sha1_use_armv8 < 0) {+		sha1_use_armv8 = crypton_sha1_armv8_available();+#ifdef SHA1_ASM+		if (sha1_use_armv8)+			crypton_armcap_P |= CRYPTON_ARMCAP_SHA1;+#endif 	}+	if (sha1_use_armv8 && len >= 64) {+		uint32_t blocks = len / 64;++#ifdef SHA1_ASM+		crypton_sha1_asm_block_data_order(ctx->h, data, blocks);+#else+		crypton_sha1_armv8_do_chunks(ctx->h, data, blocks);+#endif+		data += blocks * 64;+		len -= blocks * 64;+	}+#endif+#ifdef WITH_X86_SHA_NI+	if (sha1_use_x86 < 0)+		sha1_use_x86 =+		    (crypton_x86_simd_features() & CRYPTON_X86_SHA_NI) != 0;+	if (sha1_use_x86 && len >= 64) {+		uint32_t blocks = len / 64;++		crypton_sha1_x86_do_chunks(ctx->h, data, blocks);+		data += blocks * 64;+		len -= blocks * 64;+	}+#endif++	/* No trampoline: load_be32 does not ask for a boundary. */+	for (; len >= 64; len -= 64, data += 64)+		sha1_do_chunk(ctx, data);  	/* append data into buf */ 	if (len)
cbits/crypton_sha256.c view
@@ -26,6 +26,9 @@ #include "crypton_sha256.h" #include "crypton_bitfn.h" #include "crypton_align.h"+#ifdef WITH_X86_SHA_NI+#include "crypton_cpu.h"+#endif  void crypton_sha224_init(struct sha224_ctx *ctx) {@@ -75,13 +78,16 @@ #define s0(x)       (ror32(x, 7) ^ ror32(x,18) ^ (x >> 3)) #define s1(x)       (ror32(x,17) ^ ror32(x,19) ^ (x >> 10)) -static void sha256_do_chunk(struct sha256_ctx *ctx, uint32_t buf[])+/* The sixteen words are read out of the block rather than the block being+ * pointed at as though it were an array of them; see crypton_md5.c. */+static void sha256_do_chunk_generic(struct sha256_ctx *ctx, const uint8_t *buf) { 	uint32_t a, b, c, d, e, f, g, h, t1, t2; 	int i; 	uint32_t w[64]; -	cpu_to_be32_array(w, buf, 16);+	for (i = 0; i < 16; i++)+		w[i] = load_be32(buf + 4 * i); 	for (i = 16; i < 64; i++) 		w[i] = s1(w[i - 2]) + w[i - 7] + s0(w[i - 15]) + w[i - 16]; @@ -111,6 +117,86 @@ 	ctx->h[4] += e; ctx->h[5] += f; ctx->h[6] += g; ctx->h[7] += h; } +#ifdef WITH_ARMV8_SHA2+/*+ * AArch64 can do four rounds at a time with the SHA-2 instructions; see+ * sha256_armv8.c.  They are optional in ARMv8.0, so ask before using them.+ * Two threads racing to answer here both write the same value.+ */+extern void crypton_sha256_armv8_do_chunk(uint32_t state[8], const uint8_t buf[64]);+extern int crypton_sha256_armv8_available(void);++/* Resolved before there is a second thread; see the constructor in+ * cbits/crypton_aes.c for why.  The test below then only ever reads. */+static int sha256_use_armv8 = -1;++__attribute__((constructor))+static void sha256_armv8_ctor(void)+{+	sha256_use_armv8 = crypton_sha256_armv8_available();+}+#endif++#if (defined(WITH_ARMV8_SHA256_ASM) && defined(WITH_ARMV8_SHA2)) \+    || defined(WITH_X86_SHA256_ASM)+/*+ * SHA-256 from CRYPTOGAMS, in cbits/asm/sha256-armv8-*.S and+ * cbits/asm/sha256-x86_64-*.S, which take any number of blocks at once and+ * schedule the instructions across them -- which is where they are ahead+ * of the intrinsics above, the instructions being the same ones.  Each+ * picks its own path from the word the processor was asked about, so the+ * answer to the runtime check goes there rather than into a branch here.+ */+#define SHA256_ASM 1+#include "crypton_cpu.h"+extern void crypton_sha256_asm_block_data_order(uint32_t state[8],+                                                const void *data, size_t blocks);++#ifdef WITH_ARMV8_SHA256_ASM+/* The assembly picks its path from crypton_armcap_P, so the answer to the+ * runtime check has to reach that word rather than the flag above.  It is+ * set here, before there is a second thread, for the reason the constructor+ * in cbits/crypton_aes.c gives.+ *+ * This ran from sha256_asm_ready below until 2.1.3, guarded by the flag+ * still being unresolved -- which stopped happening when the constructor+ * above was added, so the bit was never set and the assembly took its+ * generic path.  SHA-256 was 5.7 times slower on an Apple M4 for it. */+__attribute__((constructor))+static void sha256_armcap_ctor(void)+{+	if (crypton_sha256_armv8_available())+		crypton_armcap_P |= CRYPTON_ARMCAP_SHA256;+}+#endif++static void sha256_asm_ready(void)+{+#ifndef WITH_ARMV8_SHA256_ASM+	crypton_x86_ia32cap_resolve();+#endif+}+#endif+++static void sha256_do_chunk(struct sha256_ctx *ctx, const uint8_t *buf)+{+#ifdef SHA256_ASM+	sha256_asm_ready();+	crypton_sha256_asm_block_data_order(ctx->h, buf, 1);+	return;+#endif+#if defined(WITH_ARMV8_SHA2) && !defined(SHA256_ASM)+	if (sha256_use_armv8 < 0)+		sha256_use_armv8 = crypton_sha256_armv8_available();+	if (sha256_use_armv8) {+		crypton_sha256_armv8_do_chunk(ctx->h, buf);+		return;+	}+#endif+	sha256_do_chunk_generic(ctx, buf);+}+ void crypton_sha224_update(struct sha224_ctx *ctx, const uint8_t *data, uint32_t len) { 	return crypton_sha256_update(ctx, data, len);@@ -129,24 +215,29 @@ 	/* process partial buffer if there's enough data to make a block */ 	if (index && len >= to_fill) { 		memcpy(ctx->buf + index, data, to_fill);-		sha256_do_chunk(ctx, (uint32_t *) ctx->buf);+		sha256_do_chunk(ctx, ctx->buf); 		len -= to_fill; 		data += to_fill; 		index = 0; 	} -	if (need_alignment(data, 4)) {-		uint32_t tramp[16];-		ASSERT_ALIGNMENT(tramp, 4);-		for (; len >= 64; len -= 64, data += 64) {-			memcpy(tramp, data, 64);-			sha256_do_chunk(ctx, tramp);-		}-	} else {-		/* process as much 64-block as possible */-		for (; len >= 64; len -= 64, data += 64)-			sha256_do_chunk(ctx, (uint32_t *) data);+#ifdef SHA256_ASM+	/* the assembly reads the message a byte at a time as far as the+	 * machine is concerned, so it wants no alignment and no copy, and+	 * it takes the whole run of blocks in one call */+	if (len >= 64) {+		size_t blocks = len / 64;++		sha256_asm_ready();+		crypton_sha256_asm_block_data_order(ctx->h, data, blocks);+		data += blocks * 64;+		len -= (uint32_t) blocks * 64; 	}+#else+	/* No trampoline: load_be32 does not ask for a boundary. */+	for (; len >= 64; len -= 64, data += 64)+		sha256_do_chunk(ctx, data);+#endif  	/* append data into buf */ 	if (len)
cbits/crypton_sha256.h view
@@ -51,6 +51,18 @@  void crypton_sha256_init(struct sha256_ctx *ctx); void crypton_sha256_update(struct sha256_ctx *ctx, const uint8_t *data, uint32_t len);+/* The pointers are all required to be non-null, which is said here so that+ * the compiler knows it too.  Both of these write their digest through a+ * loop -- store_be32(out + 4 * i, ...) -- where sha1 and md5 write theirs at+ * constant offsets, and that is the difference that makes gcc's+ * -Wstringop-overflow reason about out being null: with+ * -fsanitize=undefined, UndefinedBehaviorSanitizer inserts a null check+ * before memcpy, because glibc declares memcpy nonnull, and the check puts a+ * null path in front of the warning pass, which then reports writing into+ * "a region of size 0" at "address zero".  Saying the pointer is never null+ * removes the path rather than the warning.  It costs nothing: compiled as+ * the package compiles it, the assembly is identical with and without. */+__attribute__((nonnull)) void crypton_sha256_finalize(struct sha256_ctx *ctx, uint8_t *out); void crypton_sha256_finalize_prefix(struct sha256_ctx *ctx, const uint8_t *data, uint32_t len, uint32_t n, uint8_t *out);
cbits/crypton_sha3.c view
@@ -50,15 +50,76 @@ static const int keccak_piln[24] = 	{ 10,7,11,17,18,3,5,16,8,21,24,4,15,23,19,13,12,2,20,14,22,9,6,1 }; -static inline void sha3_do_chunk(uint64_t state[25], uint64_t buf[], int bufsz)+/*+ * AArch64 has instructions for this permutation; see sha3_armv8.c.  They are+ * an ARMv8.2 extension, so ask before using them.  Two threads racing to+ * answer here both write the same value.+ */+#ifdef WITH_ARMV8_SHA3+extern void crypton_sha3_armv8_permute(uint64_t state[25]);+extern int crypton_sha3_armv8_available(void);++static int sha3_use_armv8 = -1;++/* Two threads racing to answer this both write the same value. */+static int sha3_armv8_ok(void) {+	if (sha3_use_armv8 < 0)+		sha3_use_armv8 = crypton_sha3_armv8_available();+	return sha3_use_armv8;+}+#endif++#if defined(WITH_ARMV8_SHA3_ASM) && !defined(__AARCH64EB__)+/*+ * Keccak from CRYPTOGAMS, in cbits/asm/keccak1600-armv8-*.S, which takes a+ * run of blocks rather than one at a time and schedules the instructions+ * across the round it is in and the next.  The instructions are the same+ * ones the intrinsics beside it use; the arrangement is what is worth+ * about a tenth here.  It reads the message as bytes, so the run wants+ * neither alignment nor a copy.+ */+#define SHA3_ASM 1+/* the runtime question this file already asks decides whether it is used */+#define SHA3_ASM_OPTIONAL 1+extern size_t crypton_keccak_asm_absorb_cext(uint64_t state[25], const void *inp,+                                             size_t len, size_t bsz);+#define sha3_asm_absorb crypton_keccak_asm_absorb_cext+#endif++#ifdef WITH_X86_SHA3_ASM+/*+ * And the same module for x86-64, where there are no instructions for this+ * and what the assembly has over the C is the arrangement: the twenty-five+ * lanes live in registers across a round, where a compiler given the C+ * below spills them, and the rotations are folded into the operations that+ * consume them.  It needs nothing of the processor beyond the baseline, so+ * unlike the AArch64 one it is used wherever it is compiled in.+ */+#define SHA3_ASM 1+extern size_t crypton_keccak_asm_absorb(uint64_t state[25], const void *inp,+                                        size_t len, size_t bsz);+#define sha3_asm_absorb crypton_keccak_asm_absorb+#endif++/* The words are read out of the block rather than the block being pointed at+ * as though it were an array of them; see crypton_md5.c. */+static inline void sha3_do_chunk(uint64_t state[25], const uint8_t *buf, int bufsz)+{ 	int i, j, r; 	uint64_t tmp, bc[5];  	/* merge buf with state */ 	for (i = 0; i < bufsz; i++)-		state[i] ^= le64_to_cpu(buf[i]);+		state[i] ^= load_le64(buf + 8 * i); +#ifdef WITH_ARMV8_SHA3+	if (sha3_armv8_ok()) {+		crypton_sha3_armv8_permute(state);+		return;+	}+#endif+ 	/* run keccak rounds */ 	for (r = 0; r < KECCAK_NB_ROUNDS; r++) { 		/* compute the parity of each columns */@@ -121,33 +182,38 @@ 	to_fill = ctx->bufsz - ctx->bufindex;  	if (ctx->bufindex == ctx->bufsz) {-		sha3_do_chunk(ctx->state, (uint64_t *) ctx->buf, ctx->bufsz / 8);+		sha3_do_chunk(ctx->state, ctx->buf, ctx->bufsz / 8); 		ctx->bufindex = 0; 	}  	/* process partial buffer if there's enough data to make a block */ 	if (ctx->bufindex && len >= to_fill) { 		memcpy(ctx->buf + ctx->bufindex, data, to_fill);-		sha3_do_chunk(ctx->state, (uint64_t *) ctx->buf, ctx->bufsz / 8);+		sha3_do_chunk(ctx->state, ctx->buf, ctx->bufsz / 8); 		len -= to_fill; 		data += to_fill; 		ctx->bufindex = 0; 	} -	if (need_alignment(data, 8)) {-		uint64_t tramp[SHA3_BUF_SIZE_MAX/8];-		ASSERT_ALIGNMENT(tramp, 8);-		for (; len >= ctx->bufsz; len -= ctx->bufsz, data += ctx->bufsz) {-			memcpy(tramp, data, ctx->bufsz);-			sha3_do_chunk(ctx->state, tramp, ctx->bufsz / 8);-		}-	} else {-		/* process as much ctx->bufsz-block */-		for (; len >= ctx->bufsz; len -= ctx->bufsz, data += ctx->bufsz)-			sha3_do_chunk(ctx->state, (uint64_t *) data, ctx->bufsz / 8);+#ifdef SHA3_ASM+	if (len >= ctx->bufsz+#ifdef SHA3_ASM_OPTIONAL+	    && sha3_armv8_ok()+#endif+	   ) {+		const size_t left = sha3_asm_absorb(ctx->state, data, len,+		                                    ctx->bufsz);++		data += len - left;+		len = (uint32_t) left; 	}+#endif +	/* No trampoline: load_le64 does not ask for a boundary. */+	for (; len >= ctx->bufsz; len -= ctx->bufsz, data += ctx->bufsz)+		sha3_do_chunk(ctx->state, data, ctx->bufsz / 8); + 	/* append data into buf */ 	if (len) { 		memcpy(ctx->buf + ctx->bufindex, data, len);@@ -159,7 +225,7 @@ { 	/* process full buffer if needed */ 	if (ctx->bufindex == ctx->bufsz) {-		sha3_do_chunk(ctx->state, (uint64_t *) ctx->buf, ctx->bufsz / 8);+		sha3_do_chunk(ctx->state, ctx->buf, ctx->bufsz / 8); 		ctx->bufindex = 0; 	} @@ -169,7 +235,7 @@ 	ctx->buf[ctx->bufsz - 1] |= 0x80;  	/* process */-	sha3_do_chunk(ctx->state, (uint64_t *) ctx->buf, ctx->bufsz / 8);+	sha3_do_chunk(ctx->state, ctx->buf, ctx->bufsz / 8); 	ctx->bufindex = 0; }
cbits/crypton_sha512.c view
@@ -91,13 +91,16 @@ #define s0(x)       (ror64(x, 1) ^ ror64(x, 8) ^ (x >> 7)) #define s1(x)       (ror64(x, 19) ^ ror64(x, 61) ^ (x >> 6)) -static void sha512_do_chunk(struct sha512_ctx *ctx, uint64_t *buf)+/* The words are read out of the block rather than the block being pointed at+ * as though it were an array of them; see crypton_md5.c. */+static void sha512_do_chunk_generic(struct sha512_ctx *ctx, const uint8_t *buf) { 	uint64_t a, b, c, d, e, f, g, h, t1, t2; 	int i; 	uint64_t w[80]; -	cpu_to_be64_array(w, buf, 16);+	for (i = 0; i < 16; i++)+		w[i] = load_be64(buf + 8 * i);  	for (i = 16; i < 80; i++) 		w[i] = s1(w[i - 2]) + w[i - 7] + s0(w[i - 15]) + w[i - 16];@@ -128,6 +131,58 @@ 	ctx->h[4] += e; ctx->h[5] += f; ctx->h[6] += g; ctx->h[7] += h; } +#ifdef WITH_ARMV8_SHA512+/*+ * AArch64 can do two rounds at a time with the SHA-512 instructions; see+ * sha512_armv8.c.  They are an optional ARMv8.2 extension and much less+ * widespread than the SHA-256 ones, so ask before using them.  Two threads+ * racing to answer here both write the same value.+ */+extern void crypton_sha512_armv8_do_chunk(uint64_t state[8], const uint8_t buf[128]);+extern int crypton_sha512_armv8_available(void);++/* Resolved before there is a second thread; see the constructor in+ * cbits/crypton_aes.c for why.  The test below then only ever reads. */+static int sha512_use_armv8 = -1;++__attribute__((constructor))+static void sha512_armv8_ctor(void)+{+	sha512_use_armv8 = crypton_sha512_armv8_available();+}+#endif+++#ifdef WITH_X86_SHA512_ASM+/*+ * SHA-512 from CRYPTOGAMS, in cbits/asm/sha512-x86_64-*.S, which takes any+ * number of blocks at once and schedules across them, and picks between+ * AVX2, AVX, SSSE3 and plain integer code from crypton_ia32cap_P.+ */+#define SHA512_ASM 1+#include "crypton_cpu.h"+extern void crypton_sha512_asm_block_data_order(uint64_t state[8],+                                                const void *data, size_t blocks);+#endif++static void sha512_do_chunk(struct sha512_ctx *ctx, const uint8_t *buf)+{+#ifdef SHA512_ASM+	crypton_x86_ia32cap_resolve();+	crypton_sha512_asm_block_data_order(ctx->h, buf, 1);+	return;+#endif+#ifdef WITH_ARMV8_SHA512+	if (sha512_use_armv8 < 0)+		sha512_use_armv8 = crypton_sha512_armv8_available();+	if (sha512_use_armv8) {+		crypton_sha512_armv8_do_chunk(ctx->h, buf);+		return;+	}+#endif+	sha512_do_chunk_generic(ctx, buf);+}+ void crypton_sha384_update(struct sha384_ctx *ctx, const uint8_t *data, uint32_t len) { 	return crypton_sha512_update(ctx, data, len);@@ -148,24 +203,28 @@ 	/* process partial buffer if there's enough data to make a block */ 	if (index && len >= to_fill) { 		memcpy(ctx->buf + index, data, to_fill);-		sha512_do_chunk(ctx, (uint64_t *) ctx->buf);+		sha512_do_chunk(ctx, ctx->buf); 		len -= to_fill; 		data += to_fill; 		index = 0; 	} -	if (need_alignment(data, 8)) {-		uint64_t tramp[16];-		ASSERT_ALIGNMENT(tramp, 8);-		for (; len >= 128; len -= 128, data += 128) {-			memcpy(tramp, data, 128);-			sha512_do_chunk(ctx, tramp);-		}-	} else {-		/* process as much 128-block as possible */-		for (; len >= 128; len -= 128, data += 128)-			sha512_do_chunk(ctx, (uint64_t *) data);+#ifdef SHA512_ASM+	/* the assembly reads the message as bytes, so it wants neither the+	 * alignment nor the copy, and takes the whole run in one call */+	if (len >= 128) {+		size_t blocks = len / 128;++		crypton_x86_ia32cap_resolve();+		crypton_sha512_asm_block_data_order(ctx->h, data, blocks);+		data += blocks * 128;+		len -= (uint32_t) blocks * 128; 	}+#else+	/* No trampoline: load_be64 does not ask for a boundary. */+	for (; len >= 128; len -= 128, data += 128)+		sha512_do_chunk(ctx, data);+#endif  	/* append data into buf */ 	if (len)@@ -287,7 +346,7 @@ 		/* re-init the context, otherwise len is changed */ 		memset(ctx, 0, sizeof(*ctx)); 		for (i = 0; i < 8; i++)-			ctx->h[i] = cpu_to_be64(((uint64_t *) out)[i]);+			ctx->h[i] = load_be64(out + 8 * i); 		} 	} }
cbits/crypton_sha512.h view
@@ -50,6 +50,18 @@  void crypton_sha512_init(struct sha512_ctx *ctx); void crypton_sha512_update(struct sha512_ctx *ctx, const uint8_t *data, uint32_t len);+/* The pointers are all required to be non-null, which is said here so that+ * the compiler knows it too.  Both of these write their digest through a+ * loop -- store_be32(out + 4 * i, ...) -- where sha1 and md5 write theirs at+ * constant offsets, and that is the difference that makes gcc's+ * -Wstringop-overflow reason about out being null: with+ * -fsanitize=undefined, UndefinedBehaviorSanitizer inserts a null check+ * before memcpy, because glibc declares memcpy nonnull, and the check puts a+ * null path in front of the warning pass, which then reports writing into+ * "a region of size 0" at "address zero".  Saying the pointer is never null+ * removes the path rather than the warning.  It costs nothing: compiled as+ * the package compiles it, the assembly is identical with and without. */+__attribute__((nonnull)) void crypton_sha512_finalize(struct sha512_ctx *ctx, uint8_t *out); void crypton_sha512_finalize_prefix(struct sha512_ctx *ctx, const uint8_t *data, uint32_t len, uint32_t n, uint8_t *out);
cbits/crypton_skein256.c view
@@ -37,7 +37,9 @@ static const uint8_t K256_6[2] = { 58, 22, }; static const uint8_t K256_7[2] = { 32, 32, }; -static inline void skein256_do_chunk(struct skein256_ctx *ctx, uint64_t *buf, uint32_t len)+/* The four words are read out of the block rather than the block being+ * pointed at as though it were an array of them; see crypton_md5.c. */+static inline void skein256_do_chunk(struct skein256_ctx *ctx, const uint8_t *bufp, uint32_t len) { 	uint64_t x[4]; 	uint64_t ts[3];@@ -78,11 +80,18 @@ 	ROUND(0,3,2,1,K256_7); \ 	INJECTKEY((i*2) + 2) -	x[0] = le64_to_cpu(buf[0]) + ks[0];-	x[1] = le64_to_cpu(buf[1]) + ks[1] + ts[0];-	x[2] = le64_to_cpu(buf[2]) + ks[2] + ts[1];-	x[3] = le64_to_cpu(buf[3]) + ks[3];+	uint64_t buf[4]; +	buf[0] = load_le64(bufp);+	buf[1] = load_le64(bufp + 8);+	buf[2] = load_le64(bufp + 16);+	buf[3] = load_le64(bufp + 24);++	x[0] = buf[0] + ks[0];+	x[1] = buf[1] + ks[1] + ts[0];+	x[2] = buf[2] + ks[2] + ts[1];+	x[3] = buf[3] + ks[3];+ 	/* 9 pass of 8 rounds = 72 rounds */ 	PASS(0); 	PASS(1);@@ -98,10 +107,10 @@ 	ctx->t0 = ts[0]; 	ctx->t1 = ts[1]; -	ctx->h[0] = x[0] ^ cpu_to_le64(buf[0]);-        ctx->h[1] = x[1] ^ cpu_to_le64(buf[1]);-        ctx->h[2] = x[2] ^ cpu_to_le64(buf[2]);-        ctx->h[3] = x[3] ^ cpu_to_le64(buf[3]);+	ctx->h[0] = x[0] ^ buf[0];+	ctx->h[1] = x[1] ^ buf[1];+	ctx->h[2] = x[2] ^ buf[2];+	ctx->h[3] = x[3] ^ buf[3]; }  void crypton_skein256_init(struct skein256_ctx *ctx, uint32_t hashlen)@@ -115,7 +124,7 @@ 	buf[0] = cpu_to_le64((SKEIN_VERSION << 32) | SKEIN_IDSTRING); 	buf[1] = cpu_to_le64(hashlen); 	buf[2] = 0; /* tree info, not implemented */-	skein256_do_chunk(ctx, buf, 4*8);+	skein256_do_chunk(ctx, (const uint8_t *) buf, 4*8);  	SET_TYPE(ctx, FLAG_FIRST | FLAG_TYPE(TYPE_MSG)); }@@ -130,7 +139,7 @@ 	to_fill = 32 - ctx->bufindex;  	if (ctx->bufindex == 32) {-		skein256_do_chunk(ctx, (uint64_t *) ctx->buf, 32);+		skein256_do_chunk(ctx, ctx->buf, 32); 		ctx->bufindex = 0; 	} @@ -138,24 +147,17 @@ 	 * and there's without doubt further blocks */ 	if (ctx->bufindex && len > to_fill) { 		memcpy(ctx->buf + ctx->bufindex, data, to_fill);-		skein256_do_chunk(ctx, (uint64_t *) ctx->buf, 32);+		skein256_do_chunk(ctx, ctx->buf, 32); 		len -= to_fill; 		data += to_fill; 		ctx->bufindex = 0; 	} -	if (need_alignment(data, 8)) {-		uint64_t tramp[4];-		ASSERT_ALIGNMENT(tramp, 8);-		for (; len > 32; len -= 32, data += 32) {-			memcpy(tramp, data, 32);-			skein256_do_chunk(ctx, tramp, 32);-		}-	} else {-		/* process as much 32-block as possible except the last one in case we finalize */-		for (; len > 32; len -= 32, data += 32)-			skein256_do_chunk(ctx, (uint64_t *) data, 32);-	}+	/* No trampoline for a block that is not on an eight-byte boundary: the+	 * words are read with load_le64 now, which does not ask.  The last+	 * block is left for the finalisation. */+	for (; len > 32; len -= 32, data += 32)+		skein256_do_chunk(ctx, data, 32);  	/* append data into buf */ 	if (len) {@@ -174,7 +176,7 @@ 	/* if buf is not complete pad with 0 bytes */ 	if (ctx->bufindex < 32) 		memset(ctx->buf + ctx->bufindex, '\0', 32 - ctx->bufindex);-	skein256_do_chunk(ctx, (uint64_t *) ctx->buf, ctx->bufindex);+	skein256_do_chunk(ctx, ctx->buf, ctx->bufindex);  	memset(ctx->buf, '\0', 32); @@ -187,9 +189,9 @@ 	/* threefish in counter mode, 0 for 1st 64 bytes, 1 for 2nd 64 bytes, .. */ 	for (i = 0; i*32 < outsize; i++) { 		uint64_t w[4];-		*((uint64_t *) ctx->buf) = cpu_to_le64(i);+		store_le64(ctx->buf, i); 		SET_TYPE(ctx, FLAG_FIRST | FLAG_FINAL | FLAG_TYPE(TYPE_OUT));-		skein256_do_chunk(ctx, (uint64_t *) ctx->buf, sizeof(uint64_t));+		skein256_do_chunk(ctx, ctx->buf, sizeof(uint64_t));  		n = outsize - i * 32; 		if (n >= 32) n = 32;
cbits/crypton_skein256.h view
@@ -37,8 +37,8 @@  #define SKEIN256_CTX_SIZE		sizeof(struct skein256_ctx) -void cryponite_skein256_init(struct skein256_ctx *ctx, uint32_t hashlen);-void cryponite_skein256_update(struct skein256_ctx *ctx, const uint8_t *data, uint32_t len);-void cryponite_skein256_finalize(struct skein256_ctx *ctx, uint32_t hashlen, uint8_t *out);+void crypton_skein256_init(struct skein256_ctx *ctx, uint32_t hashlen);+void crypton_skein256_update(struct skein256_ctx *ctx, const uint8_t *data, uint32_t len);+void crypton_skein256_finalize(struct skein256_ctx *ctx, uint32_t hashlen, uint8_t *out);  #endif
cbits/crypton_skein512.c view
@@ -37,7 +37,9 @@ static const uint8_t K512_6[4] = { 25, 29, 39, 43, }; static const uint8_t K512_7[4] = {  8, 35, 56, 22, }; -static inline void skein512_do_chunk(struct skein512_ctx *ctx, uint64_t *buf, uint32_t len)+/* The words are read out of the block rather than the block being pointed at+ * as though it were an array of them; see crypton_md5.c. */+static inline void skein512_do_chunk(struct skein512_ctx *ctx, const uint8_t *bufp, uint32_t len) { 	uint64_t x[8]; 	uint64_t ts[3];@@ -88,15 +90,21 @@ 	ROUND(6,1,0,7,2,5,4,3,K512_7); \ 	INJECTKEY((i*2) + 2) -	x[0] = le64_to_cpu(buf[0]) + ks[0];-	x[1] = le64_to_cpu(buf[1]) + ks[1];-	x[2] = le64_to_cpu(buf[2]) + ks[2];-	x[3] = le64_to_cpu(buf[3]) + ks[3];-	x[4] = le64_to_cpu(buf[4]) + ks[4];-	x[5] = le64_to_cpu(buf[5]) + ks[5] + ts[0];-	x[6] = le64_to_cpu(buf[6]) + ks[6] + ts[1];-	x[7] = le64_to_cpu(buf[7]) + ks[7];+	uint64_t buf[8];+	int bi; +	for (bi = 0; bi < 8; bi++)+		buf[bi] = load_le64(bufp + 8 * bi);++	x[0] = buf[0] + ks[0];+	x[1] = buf[1] + ks[1];+	x[2] = buf[2] + ks[2];+	x[3] = buf[3] + ks[3];+	x[4] = buf[4] + ks[4];+	x[5] = buf[5] + ks[5] + ts[0];+	x[6] = buf[6] + ks[6] + ts[1];+	x[7] = buf[7] + ks[7];+ 	/* 9 pass of 8 rounds = 72 rounds */ 	PASS(0); 	PASS(1);@@ -112,14 +120,14 @@ 	ctx->t0 = ts[0]; 	ctx->t1 = ts[1]; -	ctx->h[0] = x[0] ^ cpu_to_le64(buf[0]);-        ctx->h[1] = x[1] ^ cpu_to_le64(buf[1]);-        ctx->h[2] = x[2] ^ cpu_to_le64(buf[2]);-        ctx->h[3] = x[3] ^ cpu_to_le64(buf[3]);-        ctx->h[4] = x[4] ^ cpu_to_le64(buf[4]);-        ctx->h[5] = x[5] ^ cpu_to_le64(buf[5]);-        ctx->h[6] = x[6] ^ cpu_to_le64(buf[6]);-        ctx->h[7] = x[7] ^ cpu_to_le64(buf[7]);+	ctx->h[0] = x[0] ^ buf[0];+        ctx->h[1] = x[1] ^ buf[1];+        ctx->h[2] = x[2] ^ buf[2];+        ctx->h[3] = x[3] ^ buf[3];+        ctx->h[4] = x[4] ^ buf[4];+        ctx->h[5] = x[5] ^ buf[5];+        ctx->h[6] = x[6] ^ buf[6];+        ctx->h[7] = x[7] ^ buf[7]; }  void crypton_skein512_init(struct skein512_ctx *ctx, uint32_t hashlen)@@ -133,7 +141,7 @@ 	buf[0] = cpu_to_le64((SKEIN_VERSION << 32) | SKEIN_IDSTRING); 	buf[1] = cpu_to_le64(hashlen); 	buf[2] = 0; /* tree info, not implemented */-	skein512_do_chunk(ctx, buf, 4*8);+	skein512_do_chunk(ctx, (const uint8_t *) buf, 4*8);  	SET_TYPE(ctx, FLAG_FIRST | FLAG_TYPE(TYPE_MSG)); }@@ -148,7 +156,7 @@ 	to_fill = 64 - ctx->bufindex;  	if (ctx->bufindex == 64) {-		skein512_do_chunk(ctx, (uint64_t *) ctx->buf, 64);+		skein512_do_chunk(ctx, ctx->buf, 64); 		ctx->bufindex = 0; 	} @@ -156,24 +164,15 @@ 	 * and there's without doubt further blocks */ 	if (ctx->bufindex && len > to_fill) { 		memcpy(ctx->buf + ctx->bufindex, data, to_fill);-		skein512_do_chunk(ctx, (uint64_t *) ctx->buf, 64);+		skein512_do_chunk(ctx, ctx->buf, 64); 		len -= to_fill; 		data += to_fill; 		ctx->bufindex = 0; 	} -	if (need_alignment(data, 8)) {-		uint64_t tramp[8];-		ASSERT_ALIGNMENT(tramp, 8);-		for (; len > 64; len -= 64, data += 64) {-			memcpy(tramp, data, 64);-			skein512_do_chunk(ctx, tramp, 64);-		}-	} else {-		/* process as much 64-block as possible except the last one in case we finalize */-		for (; len > 64; len -= 64, data += 64)-			skein512_do_chunk(ctx, (uint64_t *) data, 64);-	}+	/* No trampoline: load_le64 does not ask for a boundary. */+	for (; len > 64; len -= 64, data += 64)+		skein512_do_chunk(ctx, data, 64);  	/* append data into buf */ 	if (len) {@@ -192,7 +191,7 @@ 	/* if buf is not complete pad with 0 bytes */ 	if (ctx->bufindex < 64) 		memset(ctx->buf + ctx->bufindex, '\0', 64 - ctx->bufindex);-	skein512_do_chunk(ctx, (uint64_t *) ctx->buf, ctx->bufindex);+	skein512_do_chunk(ctx, ctx->buf, ctx->bufindex);  	memset(ctx->buf, '\0', 64); @@ -205,9 +204,9 @@ 	/* threefish in counter mode, 0 for 1st 64 bytes, 1 for 2nd 64 bytes, .. */ 	for (i = 0; i*64 < outsize; i++) { 		uint64_t w[8];-		*((uint64_t *) ctx->buf) = cpu_to_le64(i);+		store_le64(ctx->buf, i); 		SET_TYPE(ctx, FLAG_FIRST | FLAG_FINAL | FLAG_TYPE(TYPE_OUT));-		skein512_do_chunk(ctx, (uint64_t *) ctx->buf, sizeof(uint64_t));+		skein512_do_chunk(ctx, ctx->buf, sizeof(uint64_t));  		n = outsize - i * 64; 		if (n >= 64) n = 64;
cbits/crypton_skein512.h view
@@ -37,8 +37,8 @@  #define SKEIN512_CTX_SIZE		sizeof(struct skein512_ctx) -void cryponite_skein512_init(struct skein512_ctx *ctx, uint32_t hashlen);-void cryponite_skein512_update(struct skein512_ctx *ctx, const uint8_t *data, uint32_t len);-void cryponite_skein512_finalize(struct skein512_ctx *ctx, uint32_t hashlen, uint8_t *out);+void crypton_skein512_init(struct skein512_ctx *ctx, uint32_t hashlen);+void crypton_skein512_update(struct skein512_ctx *ctx, const uint8_t *data, uint32_t len);+void crypton_skein512_finalize(struct skein512_ctx *ctx, uint32_t hashlen, uint8_t *out);  #endif
cbits/crypton_tiger.c view
@@ -306,7 +306,9 @@ 	ctx->h[2] = 0xf096a5b4c3b2e187ULL; } -static inline void tiger_do_chunk(struct tiger_ctx *ctx, uint64_t *buf)+/* The words are read out of the block rather than the block being pointed at+ * as though it were an array of them; see crypton_md5.c. */+static inline void tiger_do_chunk(struct tiger_ctx *ctx, const uint8_t *buf) { 	uint64_t x0, x1, x2, x3, x4, x5, x6, x7; 	uint64_t a,b,c;@@ -314,8 +316,8 @@ 	b = ctx->h[1]; 	c = ctx->h[2]; -	x0 = cpu_to_le64(buf[0]); x1 = cpu_to_le64(buf[1]); x2 = cpu_to_le64(buf[2]); x3 = cpu_to_le64(buf[3]);-	x4 = cpu_to_le64(buf[4]); x5 = cpu_to_le64(buf[5]); x6 = cpu_to_le64(buf[6]); x7 = cpu_to_le64(buf[7]);+	x0 = load_le64(buf     ); x1 = load_le64(buf +  8); x2 = load_le64(buf + 16); x3 = load_le64(buf + 24);+	x4 = load_le64(buf + 32); x5 = load_le64(buf + 40); x6 = load_le64(buf + 48); x7 = load_le64(buf + 56);  #define BYTEOF(c, n) ((uint8_t) (c >> ((n * 8)))) @@ -376,24 +378,15 @@ 	/* process partial buffer if there's enough data to make a block */ 	if (index && len >= to_fill) { 		memcpy(ctx->buf + index, data, to_fill);-		tiger_do_chunk(ctx, (uint64_t *) ctx->buf);+		tiger_do_chunk(ctx, ctx->buf); 		len -= to_fill; 		data += to_fill; 		index = 0; 	} -	if (need_alignment(data, 8)) {-		uint64_t tramp[8];-		ASSERT_ALIGNMENT(tramp, 8);-		for (; len >= 64; len -= 64, data += 64) {-			memcpy(tramp, data, 64);-			tiger_do_chunk(ctx, tramp);-		}-	} else {-		/* process as much 64-block as possible */-		for (; len >= 64; len -= 64, data += 64)-			tiger_do_chunk(ctx, (uint64_t *) data);-	}+	/* No trampoline: load_le64 does not ask for a boundary. */+	for (; len >= 64; len -= 64, data += 64)+		tiger_do_chunk(ctx, data);  	/* append data into buf */ 	if (len)
cbits/crypton_xsalsa.c view
@@ -41,7 +41,7 @@   memset(ctx, 0, sizeof(*ctx));   ctx->nb_rounds = nb_rounds; -  /* Create initial 512-bit input block:+  /* Create initial 512-bit input crypton_salsa_block:        (x0, x5, x10, x15) is the Salsa20 constant        (x1, x2, x3, x4, x11, x12, x13, x14) is a 256-bit key        (x6, x7, x8, x9) is the first 128 bits of a 192-bit nonce@@ -56,7 +56,7 @@ void crypton_xsalsa_derive(crypton_salsa_context *ctx,                               uint32_t ivlen, const uint8_t *iv) {-  /* Finish creating initial 512-bit input block:+  /* Finish creating initial 512-bit input crypton_salsa_block:        (x6, x7, x8, x9) is the first 128 bits of a 192-bit nonce       Except iv has been shifted by 64 bits so there are now only 128 bits ahead.@@ -65,15 +65,15 @@   ctx->st.d[ 9] += load_le32(iv + 4);    /* Compute (z0, z1, . . . , z15) = doubleround ^(r/2) (x0, x1, . . . , x15) */-  block hSalsa;-  memset(&hSalsa, 0, sizeof(block));+  crypton_salsa_block hSalsa;+  memset(&hSalsa, 0, sizeof(crypton_salsa_block));   crypton_salsa_core_xor(ctx->nb_rounds, &hSalsa, &ctx->st);  -  /* Build a new 512-bit input block (x′0, x′1, . . . , x′15):+  /* Build a new 512-bit input crypton_salsa_block (x′0, x′1, . . . , x′15):        (x′0, x′5, x′10, x′15) is the Salsa20 constant        (x′1,x′2,x′3,x′4,x′11,x′12,x′13,x′14) = (z0,z5,z10,z15,z6,z7,z8,z9)        (x′6,x′7) is the last 64 bits of the 192-bit nonce-       (x′8, x′9) is a 64-bit block counter.+       (x′8, x′9) is a 64-bit crypton_salsa_block counter.   */   ctx->st.d[ 1] = hSalsa.d[ 0] - ctx->st.d[ 0];   ctx->st.d[ 2] = hSalsa.d[ 5] - ctx->st.d[ 5];
+ cbits/curve25519/x25519.c view
@@ -0,0 +1,88 @@+/*+ * X25519 through the vendored s2n-bignum where it is built, and through+ * curve25519-donna where it is not.+ *+ * The fixed-base routine is the interesting half: crypton had none, and asked+ * for the public key by multiplying the base point 9 the general way.  With a+ * table it is four to five times less work, and a TLS handshake generates a+ * key every time.  Measured on an Apple M4:+ *+ *                     donna     s2n+ *     shared secret   18.15 us  12.35+ *     key generation  18.15     3.65+ *+ * and on an x86-64, 41.19 to 26.96 and 41.18 to 8.54.+ */+#include <string.h>++#include "curve25519/x25519.h"++void crypton_curve25519_donna(uint8_t *mypublic, const uint8_t *secret,+                              const uint8_t *basepoint);++/* The assembly takes four little-endian 64-bit words, which is the same bits+ * as the 32 little-endian bytes RFC 7748 sends, so the two cross by copying+ * -- on a little-endian machine, which is the only kind s2n-bignum is for. */+#if defined(CRYPTON_S2N_BIGNUM) && defined(__BYTE_ORDER__) \+    && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__+#define CRYPTON_X25519_S2N 1+#include "crypton_cpu.h"++extern void curve25519_x25519(uint64_t res[4], const uint64_t scalar[4],+                              const uint64_t point[4]);+extern void curve25519_x25519_alt(uint64_t res[4], const uint64_t scalar[4],+                                  const uint64_t point[4]);+extern void curve25519_x25519base(uint64_t res[4], const uint64_t scalar[4]);+extern void curve25519_x25519base_alt(uint64_t res[4], const uint64_t scalar[4]);++/* The same question as everywhere else in cbits/s2n: a microarchitecture one+ * on ARM that no feature bit answers, and exactly a feature bit on x86-64. */+static int use_alt(void)+{+#if defined(__aarch64__) || defined(__arm64__)+#ifdef __APPLE__+	return 1;+#else+	return 0;+#endif+#else+	return (crypton_x86_simd_features() & CRYPTON_X86_ADX) == 0;+#endif+}+#endif++void crypton_x25519(uint8_t out[32], const uint8_t secret[32],+                    const uint8_t point[32])+{+#ifdef CRYPTON_X25519_S2N+	uint64_t r[4], s[4], p[4];++	memcpy(s, secret, 32);+	memcpy(p, point, 32);+	if (use_alt())+		curve25519_x25519_alt(r, s, p);+	else+		curve25519_x25519(r, s, p);+	memcpy(out, r, 32);+#else+	crypton_curve25519_donna(out, secret, point);+#endif+}++void crypton_x25519_base(uint8_t out[32], const uint8_t secret[32])+{+#ifdef CRYPTON_X25519_S2N+	uint64_t r[4], s[4];++	memcpy(s, secret, 32);+	if (use_alt())+		curve25519_x25519base_alt(r, s);+	else+		curve25519_x25519base(r, s);+	memcpy(out, r, 32);+#else+	static const uint8_t nine[32] = {9};++	crypton_curve25519_donna(out, secret, nine);+#endif+}
+ cbits/curve25519/x25519.h view
@@ -0,0 +1,16 @@+#ifndef CRYPTON_X25519_H+#define CRYPTON_X25519_H++#include <stdint.h>++/* out = secret * point, the X25519 of RFC 7748: three 32-byte little-endian+ * strings as they go over the wire. */+void crypton_x25519(uint8_t out[32], const uint8_t secret[32],+                    const uint8_t point[32]);++/* out = secret * G, which is the same thing with the base point 9 -- but+ * where the assembly is built this reads a table instead and is four to five+ * times faster, which is what a key generation costs. */+void crypton_x25519_base(uint8_t out[32], const uint8_t secret[32]);++#endif
cbits/decaf/ed448goldilocks/decaf.c view
@@ -18,6 +18,23 @@ #include <decaf.h> #include <decaf/ed448.h> +/* MSVC has no builtint ctz, this is a fix as in+https://stackoverflow.com/questions/355967/how-to-use-msvc-intrinsics-to-get-the-equivalent-of-this-gcc-code/5468852#5468852+*/+#ifdef _MSC_VER+#include <intrin.h>++uint32_t __inline ctz(uint32_t value)+{+    DWORD trailing_zero = 0;+    if ( _BitScanForward( &trailing_zero, value ) )+        return trailing_zero;+    else+        return 32;  // This is undefined, I better choose 32 than 0+}+#define __builtin_ctz(x) ctz(x)+#endif+ /* Template stuff */ #define API_NS(_id) crypton_decaf_448_##_id #define SCALAR_BITS CRYPTON_DECAF_448_SCALAR_BITS@@ -48,12 +65,25 @@  const uint8_t crypton_decaf_x448_base_point[CRYPTON_DECAF_X448_PUBLIC_BYTES] = { 0x05 }; -#if COFACTOR==8 || EDDSA_USE_SIGMA_ISOGENY-    static const gf SQRT_ONE_MINUS_D = {FIELD_LITERAL(-        /* NONE */-    )};+#define RISTRETTO_FACTOR CRYPTON_DECAF_448_RISTRETTO_FACTOR+const gf RISTRETTO_FACTOR = {FIELD_LITERAL(+    0x42ef0f45572736, 0x7bf6aa20ce5296, 0xf4fd6eded26033, 0x968c14ba839a66, 0xb8d54b64a2d780, 0x6aa0a1f1a7b8a5, 0x683bf68d722fa2, 0x22d962fbeb24f7+)};++#if IMAGINE_TWIST+#define TWISTED_D (-(EDWARDS_D))+#else+#define TWISTED_D ((EDWARDS_D)-1) #endif +#if TWISTED_D < 0+#define EFF_D (-(TWISTED_D))+#define NEG_D 1+#else+#define EFF_D TWISTED_D+#define NEG_D 0+#endif+ /* End of template stuff */  /* Sanity */@@ -109,128 +139,112 @@     crypton_gf_copy(y, t2); } -/** Return high bit of x = low bit of 2x mod p */-static mask_t crypton_gf_lobit(const gf x) {-    gf y;-    crypton_gf_copy(y,x);-    crypton_gf_strong_reduce(y);-    return -(y->limb[0]&1);-}- /** identity = (0,1) */ const point_t API_NS(point_identity) = {{{{{0}}},{{{1}}},{{{1}}},{{{0}}}}}; +/* Predeclare because not static: called by elligator */ void API_NS(deisogenize) (     crypton_gf_s *__restrict__ s,-    crypton_gf_s *__restrict__ minus_t_over_s,+    crypton_gf_s *__restrict__ inv_el_sum,+    crypton_gf_s *__restrict__ inv_el_m1,     const point_t p,-    mask_t toggle_hibit_s,-    mask_t toggle_hibit_t_over_s,+    mask_t toggle_s,+    mask_t toggle_altx,     mask_t toggle_rotation );  void API_NS(deisogenize) (     crypton_gf_s *__restrict__ s,-    crypton_gf_s *__restrict__ minus_t_over_s,+    crypton_gf_s *__restrict__ inv_el_sum,+    crypton_gf_s *__restrict__ inv_el_m1,     const point_t p,-    mask_t toggle_hibit_s,-    mask_t toggle_hibit_t_over_s,+    mask_t toggle_s,+    mask_t toggle_altx,     mask_t toggle_rotation ) { #if COFACTOR == 4 && !IMAGINE_TWIST-    (void) toggle_rotation;-    -    gf b, d;-    crypton_gf_s *c = s, *a = minus_t_over_s;-    crypton_gf_mulw(a, p->y, 1-EDWARDS_D);-    crypton_gf_mul(c, a, p->t);     /* -dYT, with EDWARDS_D = d-1 */-    crypton_gf_mul(a, p->x, p->z); -    crypton_gf_sub(d, c, a);  /* aXZ-dYT with a=-1 */-    crypton_gf_add(a, p->z, p->y); -    crypton_gf_sub(b, p->z, p->y); -    crypton_gf_mul(c, b, a);-    crypton_gf_mulw(b, c, -EDWARDS_D); /* (a-d)(Z+Y)(Z-Y) */-    mask_t ok = crypton_gf_isr (a,b); /* r in the paper */-    (void)ok; assert(ok | crypton_gf_eq(b,ZERO));-    crypton_gf_mulw (b, a, -EDWARDS_D); /* u in the paper */--    crypton_gf_mul(c,a,d); /* r(aZX-dYT) */-    crypton_gf_mul(a,b,p->z); /* uZ */-    crypton_gf_add(a,a,a); /* 2uZ */+    (void)toggle_rotation; /* Only applies to cofactor 8 */+    gf t1;+    crypton_gf_s *t2 = s, *t3=inv_el_sum, *t4=inv_el_m1;     -    mask_t tg = toggle_hibit_t_over_s ^ ~crypton_gf_hibit(minus_t_over_s);-    crypton_gf_cond_neg(minus_t_over_s, tg); /* t/s <-? -t/s */-    crypton_gf_cond_neg(c, tg); /* u <- -u if negative. */+    crypton_gf_add(t1,p->x,p->t);+    crypton_gf_sub(t2,p->x,p->t);+    crypton_gf_mul(t3,t1,t2); /* t3 = num */+    crypton_gf_sqr(t2,p->x);+    crypton_gf_mul(t1,t2,t3);+    crypton_gf_mulw(t2,t1,-1-TWISTED_D); /* -x^2 * (a-d) * num */+    crypton_gf_isr(t1,t2);    /* t1 = isr */+    crypton_gf_mul(t2,t1,t3); /* t2 = ratio */+    crypton_gf_mul(t4,t2,RISTRETTO_FACTOR);+    mask_t negx = crypton_gf_lobit(t4) ^ toggle_altx;+    crypton_gf_cond_neg(t2, negx);+    crypton_gf_mul(t3,t2,p->z);+    crypton_gf_sub(t3,t3,p->t);+    crypton_gf_mul(t2,t3,p->x);+    crypton_gf_mulw(t4,t2,-1-TWISTED_D);+    crypton_gf_mul(s,t4,t1);+    mask_t lobs = crypton_gf_lobit(s);+    crypton_gf_cond_neg(s,lobs);+    crypton_gf_copy(inv_el_m1,p->x);+    crypton_gf_cond_neg(inv_el_m1,~lobs^negx^toggle_s);+    crypton_gf_add(inv_el_m1,inv_el_m1,p->t);     -    crypton_gf_add(d,c,p->y);-    crypton_gf_mul(s,b,d);-    crypton_gf_cond_neg(s, toggle_hibit_s ^ crypton_gf_hibit(s));-#else+#elif COFACTOR == 8 && IMAGINE_TWIST     /* More complicated because of rotation */-    /* MAGIC This code is wrong for certain non-Curve25519 curves;-     * check if it's because of Cofactor==8 or IMAGINE_TWIST */-    -    gf c, d;-    crypton_gf_s *b = s, *a = minus_t_over_s;+    gf t1,t2,t3,t4,t5;+    crypton_gf_add(t1,p->z,p->y);+    crypton_gf_sub(t2,p->z,p->y);+    crypton_gf_mul(t3,t1,t2);      /* t3 = num */+    crypton_gf_mul(t2,p->x,p->y);  /* t2 = den */+    crypton_gf_sqr(t1,t2);+    crypton_gf_mul(t4,t1,t3);+    crypton_gf_mulw(t1,t4,-1-TWISTED_D);+    crypton_gf_isr(t4,t1);         /* isqrt(num*(a-d)*den^2) */+    crypton_gf_mul(t1,t2,t4);+    crypton_gf_mul(t2,t1,RISTRETTO_FACTOR); /* t2 = "iden" in ristretto.sage */+    crypton_gf_mul(t1,t3,t4);                 /* t1 = "inum" in ristretto.sage */ -    #if IMAGINE_TWIST-        gf x, t;-        crypton_gf_div_qnr(x,p->x);-        crypton_gf_div_qnr(t,p->t);-        crypton_gf_add ( a, p->z, x );-        crypton_gf_sub ( b, p->z, x );-        crypton_gf_mul ( c, a, b ); /* "zx" = Z^2 - aX^2 = Z^2 - X^2 */-    #else-        const crypton_gf_s *x = p->x, *t = p->t;-        crypton_gf_sqr ( a, p->z );-        crypton_gf_sqr ( b, p->x );-        crypton_gf_add ( c, a, b ); /* "zx" = Z^2 - aX^2 = Z^2 + X^2 */-    #endif-    /* Here: c = "zx" in the SAGE code = Z^2 - aX^2 */+    /* Calculate altxy = iden*inum*i*t^2*(d-a) */+    crypton_gf_mul(t3,t1,t2);+    crypton_gf_mul_i(t4,t3);+    crypton_gf_mul(t3,t4,p->t);+    crypton_gf_mul(t4,t3,p->t);+    crypton_gf_mulw(t3,t4,TWISTED_D+1);      /* iden*inum*i*t^2*(d-1) */+    mask_t rotate = toggle_rotation ^ crypton_gf_lobit(t3);     -    crypton_gf_mul ( a, p->z, t ); /* "tz" = T*Z */-    crypton_gf_sqr ( b, a );-    crypton_gf_mul ( d, b, c ); /* (TZ)^2 * (Z^2-aX^2) */-    mask_t ok = crypton_gf_isr(b, d);-    (void)ok; assert(ok | crypton_gf_eq(d,ZERO));-    crypton_gf_mul ( d, b, a ); /* "osx" = 1 / sqrt(z^2-ax^2) */-    crypton_gf_mul ( a, b, c ); -    crypton_gf_mul ( b, a, d ); /* 1/tz */--    mask_t rotate;-    #if (COFACTOR == 8)-        gf e;-        crypton_gf_sqr(e, p->z);-        crypton_gf_mul(a, e, b); /* z^2 / tz = z/t = 1/xy */-        rotate = crypton_gf_hibit(a) ^ toggle_rotation;-        /* Curve25519: cond select between zx * 1/tz or sqrt(1-d); y=-x */-        crypton_gf_mul ( a, b, c ); -        crypton_gf_cond_sel ( a, a, SQRT_ONE_MINUS_D, rotate );-        crypton_gf_cond_sel ( e, p->y, x, rotate );-    #else-        const crypton_gf_s *e = x;-        (void)toggle_rotation;-        rotate = 0;-    #endif+    /* Rotate if altxy is negative */+    crypton_gf_cond_swap(t1,t2,rotate);+    crypton_gf_mul_i(t4,p->x);+    crypton_gf_cond_sel(t4,p->y,t4,rotate);  /* t4 = "fac" = ix if rotate, else y */     -    crypton_gf_mul ( c, a, d ); // new "osx"-    crypton_gf_mul ( a, c, p->z );-    crypton_gf_add ( minus_t_over_s, a, a ); // 2 * "osx" * Z-    crypton_gf_mul ( d, b, p->z );+    crypton_gf_mul_i(t5,RISTRETTO_FACTOR); /* t5 = imi */+    crypton_gf_mul(t3,t5,t2);                /* iden * imi */+    crypton_gf_mul(t2,t5,t1);+    crypton_gf_mul(t5,t2,p->t);              /* "altx" = iden*imi*t */+    mask_t negx = crypton_gf_lobit(t5) ^ toggle_altx;     -    mask_t tg = toggle_hibit_t_over_s ^~ crypton_gf_hibit(minus_t_over_s);-    crypton_gf_cond_neg ( minus_t_over_s, tg );-    crypton_gf_cond_neg ( c, rotate ^ tg );-    crypton_gf_add ( d, d, c );-    crypton_gf_mul ( s, d, e ); /* here "x" = y unless rotate */-    crypton_gf_cond_neg ( s, toggle_hibit_s ^ crypton_gf_hibit(s) );+    crypton_gf_cond_neg(t1,negx^rotate);+    crypton_gf_mul(t2,t1,p->z);+    crypton_gf_add(t2,t2,ONE);+    crypton_gf_mul(inv_el_sum,t2,t4);+    crypton_gf_mul(s,inv_el_sum,t3);+    +    mask_t negs = crypton_gf_lobit(s);+    crypton_gf_cond_neg(s,negs);+    +    mask_t negz = ~negs ^ toggle_s ^ negx;+    crypton_gf_copy(inv_el_m1,p->z);+    crypton_gf_cond_neg(inv_el_m1,negz);+    crypton_gf_sub(inv_el_m1,inv_el_m1,t4);+#else+#error "Cofactor must be 4 (with no IMAGINE_TWIST) or 8 (with IMAGINE_TWIST)" #endif }  void API_NS(point_encode)( unsigned char ser[SER_BYTES], const point_t p ) {-    gf s, mtos;-    API_NS(deisogenize)(s,mtos,p,0,0,0);-    crypton_gf_serialize(ser,s,0);+    gf s,ie1,ie2;+    API_NS(deisogenize)(s,ie1,ie2,p,0,0,0);+    crypton_gf_serialize(ser,s); }  crypton_decaf_error_t API_NS(point_decode) (@@ -238,89 +252,54 @@     const unsigned char ser[SER_BYTES],     crypton_decaf_bool_t allow_identity ) {-    gf s, a, b, c, d, e, f;+    gf s, s2, num, tmp;+    crypton_gf_s *tmp2=s2, *ynum=p->z, *isr=p->x, *den=p->t;+         mask_t succ = crypton_gf_deserialize(s, ser, 0);-    mask_t zero = crypton_gf_eq(s, ZERO);-    succ &= bool_to_mask(allow_identity) | ~zero;-    crypton_gf_sqr ( a, s ); /* s^2 */+    succ &= bool_to_mask(allow_identity) | ~crypton_gf_eq(s, ZERO);+    succ &= ~crypton_gf_lobit(s);+    +    crypton_gf_sqr(s2,s);                  /* s^2 = -as^2 */ #if IMAGINE_TWIST-    crypton_gf_sub ( f, ONE, a ); /* f = 1-as^2 = 1-s^2*/-#else-    crypton_gf_add ( f, ONE, a ); /* f = 1-as^2 = 1+s^2 */+    crypton_gf_sub(s2,ZERO,s2);            /* -as^2 */ #endif-    succ &= ~ crypton_gf_eq( f, ZERO );-    crypton_gf_sqr ( b, f );  /* (1-as^2)^2 = 1 - 2as^2 + a^2 s^4 */-    crypton_gf_mulw ( c, a, 4*IMAGINE_TWIST-4*EDWARDS_D ); -    crypton_gf_add ( c, c, b ); /* t^2 = 1 + (2a-4d) s^2 + s^4 */-    crypton_gf_mul ( d, f, s ); /* s * (1-as^2) for denoms */-    crypton_gf_sqr ( e, d );    /* s^2 * (1-as^2)^2 */-    crypton_gf_mul ( b, c, e ); /* t^2 * s^2 * (1-as^2)^2 */+    crypton_gf_sub(den,ONE,s2);            /* 1+as^2 */+    crypton_gf_add(ynum,ONE,s2);           /* 1-as^2 */+    crypton_gf_mulw(num,s2,-4*TWISTED_D);+    crypton_gf_sqr(tmp,den);               /* tmp = den^2 */+    crypton_gf_add(num,tmp,num);           /* num = den^2 - 4*d*s^2 */+    crypton_gf_mul(tmp2,num,tmp);          /* tmp2 = num*den^2 */+    succ &= crypton_gf_isr(isr,tmp2);      /* isr = 1/sqrt(num*den^2) */+    crypton_gf_mul(tmp,isr,den);           /* isr*den */+    crypton_gf_mul(p->y,tmp,ynum);         /* isr*den*(1-as^2) */+    crypton_gf_mul(tmp2,tmp,s);            /* s*isr*den */+    crypton_gf_add(tmp2,tmp2,tmp2);        /* 2*s*isr*den */+    crypton_gf_mul(tmp,tmp2,isr);          /* 2*s*isr^2*den */+    crypton_gf_mul(p->x,tmp,num);          /* 2*s*isr^2*den*num */+    crypton_gf_mul(tmp,tmp2,RISTRETTO_FACTOR); /* 2*s*isr*den*magic */+    crypton_gf_cond_neg(p->x,crypton_gf_lobit(tmp)); /* flip x */     -    succ &= crypton_gf_isr(e,b) | crypton_gf_eq(b,ZERO); /* e = 1/(t s (1-as^2)) */-    crypton_gf_mul ( b, e, d ); /* 1 / t */-    crypton_gf_mul ( d, e, c ); /* t / (s(1-as^2)) */-    crypton_gf_mul ( e, d, f ); /* t / s */-    mask_t negtos = crypton_gf_hibit(e);-    crypton_gf_cond_neg(b, negtos);-    crypton_gf_cond_neg(d, negtos);--#if IMAGINE_TWIST-    crypton_gf_add ( p->z, ONE, a); /* Z = 1+as^2 = 1-s^2 */-#else-    crypton_gf_sub ( p->z, ONE, a); /* Z = 1+as^2 = 1-s^2 */+#if COFACTOR==8+    /* Additionally check y != 0 and x*y*isomagic nonegative */+    succ &= ~crypton_gf_eq(p->y,ZERO);+    crypton_gf_mul(tmp,p->x,p->y);+    crypton_gf_mul(tmp2,tmp,RISTRETTO_FACTOR);+    succ &= ~crypton_gf_lobit(tmp2); #endif -#if COFACTOR == 8-    crypton_gf_mul ( a, p->z, d); /* t(1+s^2) / s(1-s^2) = 2/xy */-    succ &= ~crypton_gf_lobit(a); /* = ~crypton_gf_hibit(a/2), since crypton_gf_hibit(x) = crypton_gf_lobit(2x) */-#endif-    -    crypton_gf_mul ( a, f, b ); /* y = (1-s^2) / t */-    crypton_gf_mul ( p->y, p->z, a ); /* Y = yZ */ #if IMAGINE_TWIST-    crypton_gf_add ( b, s, s );-    crypton_gf_mul(p->x, b, SQRT_MINUS_ONE); /* Curve25519 */-#else-    crypton_gf_add ( p->x, s, s );-#endif-    crypton_gf_mul ( p->t, p->x, a ); /* T = 2s (1-as^2)/t */-    -#if UNSAFE_CURVE_HAS_POINTS_AT_INFINITY-    /* This can't happen for any of the supported configurations.-     *-     * If it can happen (because s=1), it's because the curve has points-     * at infinity, which means that there may be critical security bugs-     * elsewhere in the library.  In that case, it's better that you hit-     * the assertion in point_valid, which will happen in the test suite-     * since it tests s=1.-     *-     * This debugging option is to allow testing of IMAGINE_TWIST = 0 on-     * Ed25519, without hitting that assertion.  Don't use it in-     * production.-     */-    succ &= ~crypton_gf_eq(p->z,ZERO);+    crypton_gf_copy(tmp,p->x);+    crypton_gf_mul_i(p->x,tmp); #endif++    /* Fill in z and t */+    crypton_gf_copy(p->z,ONE);+    crypton_gf_mul(p->t,p->x,p->y);     -    p->y->limb[0] -= zero;     assert(API_NS(point_valid)(p) | ~succ);-         return crypton_decaf_succeed_if(mask_to_bool(succ)); } -#if IMAGINE_TWIST-#define TWISTED_D (-(EDWARDS_D))-#else-#define TWISTED_D ((EDWARDS_D)-1)-#endif--#if TWISTED_D < 0-#define EFF_D (-(TWISTED_D))-#define NEG_D 1-#else-#define EFF_D TWISTED_D-#define NEG_D 0-#endif- void API_NS(point_sub) (     point_t p,     const point_t q,@@ -563,6 +542,7 @@     const point_t b,     const scalar_t scalar ) {+     const int WINDOW = CRYPTON_DECAF_WINDOW_BITS,         WINDOW_MASK = (1<<WINDOW)-1,         WINDOW_T_MASK = WINDOW_MASK >> 1,@@ -573,7 +553,7 @@     API_NS(scalar_halve)(scalar1x,scalar1x);          /* Set up a precomputed table with odd multiples of b. */-    pniels_t pn, multiples[NTABLE];+    pniels_t pn, multiples[1<<((int)(CRYPTON_DECAF_WINDOW_BITS)-1)];  // == NTABLE (MSVC compatibility issue)     point_t tmp;     prepare_fixed_window(multiples, b, NTABLE); @@ -624,12 +604,13 @@     const scalar_t scalarb,     const point_t c,     const scalar_t scalarc-) {+) {    +         const int WINDOW = CRYPTON_DECAF_WINDOW_BITS,         WINDOW_MASK = (1<<WINDOW)-1,         WINDOW_T_MASK = WINDOW_MASK >> 1,         NTABLE = 1<<(WINDOW-1);-        +     scalar_t scalar1x, scalar2x;     API_NS(scalar_add)(scalar1x, scalarb, point_scalarmul_adjustment);     API_NS(scalar_halve)(scalar1x,scalar1x);@@ -637,9 +618,10 @@     API_NS(scalar_halve)(scalar2x,scalar2x);          /* Set up a precomputed table with odd multiples of b. */-    pniels_t pn, multiples1[NTABLE], multiples2[NTABLE];+    pniels_t pn, multiples1[1<<((int)(CRYPTON_DECAF_WINDOW_BITS)-1)], multiples2[1<<((int)(CRYPTON_DECAF_WINDOW_BITS)-1)];+    // Array size above equal NTABLE (MSVC compatibility issue)     point_t tmp;-    prepare_fixed_window(multiples1, b, NTABLE);+    prepare_fixed_window(multiples1, b, NTABLE);       prepare_fixed_window(multiples2, c, NTABLE);      /* Initialize. */@@ -701,11 +683,13 @@     const scalar_t scalar1,     const scalar_t scalar2 ) {+         const int WINDOW = CRYPTON_DECAF_WINDOW_BITS,         WINDOW_MASK = (1<<WINDOW)-1,         WINDOW_T_MASK = WINDOW_MASK >> 1,         NTABLE = 1<<(WINDOW-1);-        ++     scalar_t scalar1x, scalar2x;     API_NS(scalar_add)(scalar1x, scalar1, point_scalarmul_adjustment);     API_NS(scalar_halve)(scalar1x,scalar1x);@@ -713,7 +697,9 @@     API_NS(scalar_halve)(scalar2x,scalar2x);          /* Set up a precomputed table with odd multiples of b. */-    point_t multiples1[NTABLE], multiples2[NTABLE], working, tmp;+    point_t multiples1[1<<((int)(CRYPTON_DECAF_WINDOW_BITS)-1)], multiples2[1<<((int)(CRYPTON_DECAF_WINDOW_BITS)-1)], working, tmp;+    // Array sizes above equal NTABLE (MSVC compatibility issue)+     pniels_t pn;          API_NS(point_copy)(working, b);@@ -801,7 +787,7 @@     crypton_gf_mul ( b, q->y, p->x );     mask_t succ = crypton_gf_eq(a,b);     -    #if (COFACTOR == 8) && IMAGINE_TWIST+    #if (COFACTOR == 8)         crypton_gf_mul ( a, p->y, q->y );         crypton_gf_mul ( b, q->x, p->x );         #if !(IMAGINE_TWIST)@@ -936,11 +922,11 @@     const unsigned int n = COMBS_N, t = COMBS_T, s = COMBS_S;     assert(n*t*s >= SCALAR_BITS);   -    point_t working, start, doubles[t-1];+    point_t working, start, doubles[COMBS_T-1];     API_NS(point_copy)(working, base);     pniels_t pn_tmp;   -    gf zs[n<<(t-1)], zis[n<<(t-1)];+    gf zs[(unsigned int)(COMBS_N)<<(unsigned int)(COMBS_T-1)], zis[(unsigned int)(COMBS_N)<<(unsigned int)(COMBS_T-1)];        unsigned int i,j,k;     @@ -1078,7 +1064,7 @@     return succ; } -void API_NS(point_mul_by_cofactor_and_encode_like_eddsa) (+void API_NS(point_mul_by_ratio_and_encode_like_eddsa) (     uint8_t enc[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES],     const point_t p ) {@@ -1116,15 +1102,20 @@         crypton_gf_mul ( y, u, t ); // (x^2+y^2)(2z^2-y^2+x^2)         crypton_gf_mul ( u, z, t );         crypton_gf_copy( z, u );-        crypton_gf_mul ( u, x, SQRT_ONE_MINUS_D );+        crypton_gf_mul ( u, x, RISTRETTO_FACTOR );+#if IMAGINE_TWIST+        crypton_gf_mul_i( x, u );+#else+#error "... probably wrong"         crypton_gf_copy( x, u );+#endif         crypton_decaf_bzero(u,sizeof(u));     } #elif IMAGINE_TWIST     {         API_NS(point_double)(q,q);         API_NS(point_double)(q,q);-        crypton_gf_mul_qnr(x, q->x);+        crypton_gf_mul_i(x, q->x);         crypton_gf_copy(y, q->y);         crypton_gf_copy(z, q->z);     }@@ -1137,7 +1128,7 @@         crypton_gf_add( u, x, t );         crypton_gf_add( z, q->y, q->x );         crypton_gf_sqr ( y, z);-        crypton_gf_sub ( y, u, y );+        crypton_gf_sub ( y, y, u );         crypton_gf_sub ( z, t, x );         crypton_gf_sqr ( x, q->z );         crypton_gf_add ( t, x, x); @@ -1155,7 +1146,7 @@          /* Encode */     enc[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES-1] = 0;-    crypton_gf_serialize(enc, x, 1);+    crypton_gf_serialize(enc, x);     enc[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES-1] |= 0x80 & crypton_gf_lobit(t);      crypton_decaf_bzero(x,sizeof(x));@@ -1166,7 +1157,7 @@ }  -crypton_decaf_error_t API_NS(point_decode_like_eddsa_and_ignore_cofactor) (+crypton_decaf_error_t API_NS(point_decode_like_eddsa_and_mul_by_ratio) (     point_t p,     const uint8_t enc[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES] ) {@@ -1176,7 +1167,7 @@     mask_t low = ~word_is_zero(enc2[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES-1] & 0x80);     enc2[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES-1] &= ~0x80;     -    mask_t succ = crypton_gf_deserialize(p->y, enc2, 1);+    mask_t succ = crypton_gf_deserialize(p->y, enc2, 0); #if 0 == 0     succ &= word_is_zero(enc2[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES-1]); #endif@@ -1196,7 +1187,7 @@     succ &= crypton_gf_isr(p->t,p->x); /* 1/sqrt(num * denom) */          crypton_gf_mul(p->x,p->t,p->z); /* sqrt(num / denom) */-    crypton_gf_cond_neg(p->x,~crypton_gf_lobit(p->x)^low);+    crypton_gf_cond_neg(p->x,crypton_gf_lobit(p->x)^low);     crypton_gf_copy(p->z,ONE);        #if EDDSA_USE_SIGMA_ISOGENY@@ -1221,8 +1212,9 @@         crypton_gf_sub ( p->t, a, c ); // y^2 - x^2         crypton_gf_sqr ( p->x, p->z );         crypton_gf_add ( p->z, p->x, p->x );-        crypton_gf_sub ( a, p->z, p->t ); // 2z^2 - y^2 + x^2-        crypton_gf_mul ( c, a, SQRT_ONE_MINUS_D );+        crypton_gf_sub ( c, p->z, p->t ); // 2z^2 - y^2 + x^2+        crypton_gf_div_i ( a, c );+        crypton_gf_mul ( c, a, RISTRETTO_FACTOR );         crypton_gf_mul ( p->x, b, p->t); // (2xy)(y^2-x^2)         crypton_gf_mul ( p->z, p->t, c ); // (y^2-x^2)sd(2z^2 - y^2 + x^2)         crypton_gf_mul ( p->y, d, c ); // (y^2+x^2)sd(2z^2 - y^2 + x^2)@@ -1265,6 +1257,7 @@          crypton_decaf_bzero(enc2,sizeof(enc2));     assert(API_NS(point_valid)(p) || ~succ);+         return crypton_decaf_succeed_if(mask_to_bool(succ)); } @@ -1274,7 +1267,7 @@     const uint8_t scalar[X_PRIVATE_BYTES] ) {     gf x1, x2, z2, x3, z3, t1, t2;-    ignore_result(crypton_gf_deserialize(x1,base,1));+    ignore_result(crypton_gf_deserialize(x1,base,0));     crypton_gf_copy(x2,ONE);     crypton_gf_copy(z2,ZERO);     crypton_gf_copy(x3,x1);@@ -1290,8 +1283,7 @@         if (t/8==0) sb &= -(uint8_t)COFACTOR;         else if (t == X_PRIVATE_BITS-1) sb = -1;         -        mask_t k_t = (sb>>(t%8)) & 1;-        k_t = -k_t; /* set to all 0s or all 1s */+        mask_t k_t = bit_to_mask((sb>>(t%8)) & 1);                  swap ^= k_t;         crypton_gf_cond_swap(x2,x3,swap);@@ -1325,7 +1317,7 @@     crypton_gf_cond_swap(z2,z3,swap);     crypton_gf_invert(z2,z2,0);     crypton_gf_mul(x1,x2,z2);-    crypton_gf_serialize(out,x1,1);+    crypton_gf_serialize(out,x1);     mask_t nz = ~crypton_gf_eq(x1,ZERO);          crypton_decaf_bzero(x1,sizeof(x1));@@ -1345,15 +1337,8 @@     const uint8_t ed[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES] ) {     gf y;-    {-        uint8_t enc2[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES];-        memcpy(enc2,ed,sizeof(enc2));--        /* retrieve y from the ed compressed point */-        enc2[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES-1] &= ~0x80;-        ignore_result(crypton_gf_deserialize(y, enc2, 0));-        crypton_decaf_bzero(enc2,sizeof(enc2));-    }+    const uint8_t mask = (uint8_t)(0xFE<<(7));+    ignore_result(crypton_gf_deserialize(y, ed, mask));          {         gf n,d;@@ -1364,7 +1349,7 @@         crypton_gf_sub(d, ONE, y); /* d = 1-y */         crypton_gf_invert(d, d, 0); /* d = 1/(1-y) */         crypton_gf_mul(y, n, d); /* u = (y+1)/(1-y) */-        crypton_gf_serialize(x,y,1);+        crypton_gf_serialize(x,y); #else /* EDDSA_USE_SIGMA_ISOGENY */         /* u = y^2 * (1-dy^2) / (1-y^2) */         crypton_gf_sqr(n,y); /* y^2*/@@ -1374,7 +1359,7 @@         crypton_gf_mulw(d,n,EDWARDS_D); /* dy^2*/         crypton_gf_sub(d, ONE, d); /* 1-dy^2*/         crypton_gf_mul(n, y, d); /* y^2 * (1-dy^2) / (1-y^2) */-        crypton_gf_serialize(x,n,1);+        crypton_gf_serialize(x,n); #endif /* EDDSA_USE_SIGMA_ISOGENY */                  crypton_decaf_bzero(y,sizeof(y));@@ -1390,6 +1375,26 @@     crypton_decaf_x448_derive_public_key(out,scalar); } +void API_NS(point_mul_by_ratio_and_encode_like_x448) (+    uint8_t out[X_PUBLIC_BYTES],+    const point_t p+) {+    point_t q;+#if COFACTOR == 8+    point_double_internal(q,p,1);+#else+    API_NS(point_copy)(q,p);+#endif+    crypton_gf_invert(q->t,q->x,0); /* 1/x */+    crypton_gf_mul(q->z,q->t,q->y); /* y/x */+    crypton_gf_sqr(q->y,q->z); /* (y/x)^2 */+#if IMAGINE_TWIST+    crypton_gf_sub(q->y,ZERO,q->y);+#endif+    crypton_gf_serialize(out,q->y);+    API_NS(point_destroy(q));+}+ void crypton_decaf_x448_derive_public_key (     uint8_t out[X_PUBLIC_BYTES],     const uint8_t scalar[X_PRIVATE_BYTES]@@ -1399,45 +1404,19 @@     memcpy(scalar2,scalar,sizeof(scalar2));     scalar2[0] &= -(uint8_t)COFACTOR;     -    scalar2[X_PRIVATE_BYTES-1] &= ~(-1u<<((X_PRIVATE_BITS+7)%8));+    scalar2[X_PRIVATE_BYTES-1] &= ~(0xFF<<((X_PRIVATE_BITS+7)%8));     scalar2[X_PRIVATE_BYTES-1] |= 1<<((X_PRIVATE_BITS+7)%8);          scalar_t the_scalar;     API_NS(scalar_decode_long)(the_scalar,scalar2,sizeof(scalar2));     -    /* We're gonna isogenize by 2, so divide by 2.-     *-     * Why by 2, even though it's a 4-isogeny?-     *-     * The isogeny map looks like-     * Montgomery <-2-> Jacobi <-2-> Edwards-     *-     * Since the Jacobi base point is the PREimage of the iso to-     * the Montgomery curve, and we're going-     * Jacobi -> Edwards -> Jacobi -> Montgomery,-     * we pick up only a factor of 2 over Jacobi -> Montgomery. -     */-    API_NS(scalar_halve)(the_scalar,the_scalar);+    /* Compensate for the encoding ratio */+    for (unsigned i=1; i<CRYPTON_DECAF_X448_ENCODE_RATIO; i<<=1) {+        API_NS(scalar_halve)(the_scalar,the_scalar);+    }     point_t p;     API_NS(precomputed_scalarmul)(p,API_NS(precomputed_base),the_scalar);-    -    /* Isogenize to Montgomery curve.-     *-     * Why isn't this just a separate function, eg crypton_decaf_encode_like_x448?-     * Basically because in general it does the wrong thing if there is a cofactor-     * component in the input.  In this function though, there isn't a cofactor-     * component in the input.-     */-    crypton_gf_invert(p->t,p->x,0); /* 1/x */-    crypton_gf_mul(p->z,p->t,p->y); /* y/x */-    crypton_gf_sqr(p->y,p->z); /* (y/x)^2 */-#if IMAGINE_TWIST-    crypton_gf_sub(p->y,ZERO,p->y);-#endif-    crypton_gf_serialize(out,p->y,1);-        -    crypton_decaf_bzero(scalar2,sizeof(scalar2));-    API_NS(scalar_destroy)(the_scalar);+    API_NS(point_mul_by_ratio_and_encode_like_x448)(out,p);     API_NS(point_destroy)(p); } @@ -1483,7 +1462,10 @@             uint32_t pos = __builtin_ctz((uint32_t)current), odd = (uint32_t)current >> pos;             int32_t delta = odd & mask;             if (odd & 1<<(table_bits+1)) delta -= (1<<(table_bits+1));-            current -= delta << pos;+            /* delta is negative half the time and shifting a negative+             * value left is undefined; current is unsigned, so the shift is+             * done there and means the same thing. */+            current -= (uint64_t)delta << pos;             control[position].power = pos + 16*(w-1);             control[position].addend = delta;             position--;@@ -1566,13 +1548,13 @@ ) {     const int table_bits_var = CRYPTON_DECAF_WNAF_VAR_TABLE_BITS,         table_bits_pre = CRYPTON_DECAF_WNAF_FIXED_TABLE_BITS;-    struct smvt_control control_var[SCALAR_BITS/(table_bits_var+1)+3];-    struct smvt_control control_pre[SCALAR_BITS/(table_bits_pre+1)+3];+    struct smvt_control control_var[SCALAR_BITS/((int)(CRYPTON_DECAF_WNAF_VAR_TABLE_BITS)+1)+3];+    struct smvt_control control_pre[SCALAR_BITS/((int)(CRYPTON_DECAF_WNAF_FIXED_TABLE_BITS)+1)+3];          int ncb_pre = recode_wnaf(control_pre, scalar1, table_bits_pre);     int ncb_var = recode_wnaf(control_var, scalar2, table_bits_var);   -    pniels_t precmp_var[1<<table_bits_var];+    pniels_t precmp_var[1<<(int)(CRYPTON_DECAF_WNAF_VAR_TABLE_BITS)];     prepare_wnaf_table(precmp_var, base2, table_bits_var);        int contp=0, contv=0, i = control_var[0].power;
cbits/decaf/ed448goldilocks/decaf_tables.c view
@@ -5,350 +5,350 @@  #define API_NS(_id) crypton_decaf_448_##_id const API_NS(point_t) API_NS(point_base) = {{-{FIELD_LITERAL(0x00fffffffffffffe,0x00ffffffffffffff,0x00ffffffffffffff,0x00ffffffffffffff,0x0000000000000003,0x0000000000000000,0x0000000000000000,0x0000000000000000)},-  {FIELD_LITERAL(0x0081e6d37f752992,0x003078ead1c28721,0x00135cfd2394666c,0x0041149c50506061,0x0031d30e4f5490b3,0x00902014990dc141,0x0052341b04c1e328,0x0014237853c10a1b)},-  {FIELD_LITERAL(0x00fffffffffffffb,0x00ffffffffffffff,0x00ffffffffffffff,0x00ffffffffffffff,0x00fffffffffffffe,0x00ffffffffffffff,0x00ffffffffffffff,0x00ffffffffffffff)},-  {FIELD_LITERAL(0x008f205b70660415,0x00881c60cfd3824f,0x00377a638d08500d,0x008c66d5d4672615,0x00e52fa558e08e13,0x0087770ae1b6983d,0x004388f55a0aa7ff,0x00b4d9a785cf1a91)}+{FIELD_LITERAL(0x0000000000000000,0x0000000000000000,0x0000000000000000,0x0080000000000000,0x00fffffffffffffe,0x00ffffffffffffff,0x00ffffffffffffff,0x007fffffffffffff)},+  {FIELD_LITERAL(0x006079b4dfdd4a64,0x000c1e3ab470a1c8,0x0044d73f48e5199b,0x0050452714141818,0x004c74c393d5242c,0x0024080526437050,0x00d48d06c13078ca,0x008508de14f04286)},+  {FIELD_LITERAL(0x0000000000000001,0x0000000000000000,0x0000000000000000,0x0000000000000000,0x0000000000000000,0x0000000000000000,0x0000000000000000,0x0000000000000000)},+  {FIELD_LITERAL(0x00e3c816dc198105,0x0062071833f4e093,0x004dde98e3421403,0x00a319b57519c985,0x00794be956382384,0x00e1ddc2b86da60f,0x0050e23d5682a9ff,0x006d3669e173c6a4)} }}; const gf API_NS(precomputed_base_as_fe)[240] VECTOR_ALIGNED = {-  {FIELD_LITERAL(0x00e614a9f7278dc5,0x002e454ad04c5124,0x00d8f58cee1436f3,0x00c83ed46e4180ec,0x00a41e93274a38fa,0x00c1e7e53257771e,0x0043e0ff03c0392f,0x002c7c6405ce61df)},-  {FIELD_LITERAL(0x0033c4f9dc990b33,0x00c291cb1ceb55c3,0x002ae3f58ade88b2,0x006b1f9f11395474,0x002ded6e4b27ff7c,0x0041012ed4aa10e1,0x003c22d20a36bae7,0x001f584eed472b19)},-  {FIELD_LITERAL(0x00c3514779ee6f60,0x001574c873b20c2b,0x004cd6a46a5a5e65,0x0059a068aeb4204a,0x004c610458bc354d,0x00e94567479d02d2,0x00feaf77ed118e28,0x00f58a8bf115eeb5)},-  {FIELD_LITERAL(0x0046110878fcb20f,0x00df43db21cc6f32,0x00ffdde9f4516644,0x00519917791686b9,0x00b72b441fd34473,0x008d45684cb1c72b,0x0015181370fc17a5,0x00a456d1307f74d3)},-  {FIELD_LITERAL(0x001430f149b607dc,0x00e992ccd16715fc,0x00a62209b0a32a09,0x00b889cedc26b8e4,0x0059bf9a3ac109cf,0x006871bb3b7feac2,0x00f4a4d5fd9a0e6b,0x00b95db460cd69a5)},-  {FIELD_LITERAL(0x0036304418bda702,0x007bc56861561558,0x00f344bc8e30416f,0x00a64537080f59d7,0x00b4c20077d00ace,0x00ee79620b26f8cc,0x00a6a558e0b5403d,0x008f1d2c766f3d19)},-  {FIELD_LITERAL(0x00ef21c0297d3112,0x0073f89bd27c35b1,0x00ec44f9b1ff5e33,0x006bee51d878f1ee,0x001571a4b2aceddb,0x00cd0182d55131d1,0x0026761dbc1844be,0x00f01865af716474)},-  {FIELD_LITERAL(0x0021dfef3f5fe8cc,0x0038c659ed1dbd68,0x0058ded9bcebe283,0x00077bbb094983ee,0x00b7b484e913d70c,0x0063e477a9506397,0x0000b996a6e01629,0x00ab68b41f75cd37)},-  {FIELD_LITERAL(0x00a1fbd946403a4e,0x00be5a4e2d611b05,0x00ea4f210888bc6e,0x0043e9b0e0ae50fe,0x002abc4f6bd86845,0x00c3ed649c67f663,0x00d4eeb391a520e7,0x004b19cf1bfe7584)},-  {FIELD_LITERAL(0x0099a75e6f22999e,0x001f16454c79f659,0x00d776a37fddc812,0x0095fdd63b6b0a78,0x00d232169366e947,0x002ea77dd21e9de7,0x00e8c46e85f97a90,0x00358758651f8cd9)},-  {FIELD_LITERAL(0x002b6f5036a07bdf,0x004f6940af3e2646,0x00866028f8986799,0x00838b26ccb50415,0x0010557417f00b11,0x008a3b6bc447e96b,0x003de3d035e9e0c9,0x00188fca2b6d4011)},-  {FIELD_LITERAL(0x001ca4038635312b,0x0078dc75c1e01c44,0x004340f00b3100a4,0x005e63e36bf6646e,0x008e1efd4b624688,0x00a61c2ffb1525e1,0x0072587505a75b81,0x00a8637140d96e78)},-  {FIELD_LITERAL(0x004a7c41ffac8a41,0x005bf37075b1c20b,0x00c053b570a42408,0x002bb7e278d328e7,0x00b2378b63245100,0x003318bf2a1a368a,0x00f4e3e0bdbe02de,0x0058921e4b1e32f8)},-  {FIELD_LITERAL(0x005e93d6fa1118a0,0x0062b43515d381e2,0x002c42864052e620,0x00af258bae6ccbd3,0x00954247094d654d,0x005db01f5b010810,0x009c8cf25efa8204,0x005f73ced3714ef7)},-  {FIELD_LITERAL(0x0085f89aff2cf49d,0x00f591ee8480f6f0,0x00378ed518114265,0x00f04293e2a09008,0x00c58688db9140ed,0x00e9912696399ff1,0x0055bd1b96367413,0x0023a70cf830f999)},-  {FIELD_LITERAL(0x001c83772944584e,0x00c1ba881e472bcc,0x00af2715a0aef13f,0x00bd0360d25610a6,0x00c42f8b3eebebde,0x00a9e474849788b1,0x00dcd1a1a2efec5c,0x009480d34c2818c0)},-  {FIELD_LITERAL(0x00b4b6e09a565d74,0x0095efcf6175aa48,0x00498defe7ae7810,0x00309b684ed26470,0x007a8873a91d4e44,0x00ea4b3f857eb27a,0x00979b8619d25a9e,0x00721a2770eeb6e9)},-  {FIELD_LITERAL(0x00b422f0f4be195f,0x00e88cfa83bfa2db,0x009fd60666ea4268,0x0095a458f5e801d0,0x00b9eee6882081f6,0x00b27edb37604948,0x00a7f67c4d44d8db,0x00df840ccf290c01)},-  {FIELD_LITERAL(0x00c9fed0d47c9103,0x00ba73ed9294a043,0x005cbbc928e652e1,0x0068419e98ee8215,0x00f63de63786300b,0x009aa9bb6c19f8aa,0x0066c536b573213f,0x00d2b77a5b2f2450)},-  {FIELD_LITERAL(0x00810236c68d5b74,0x00d0a1af1872a011,0x007f23ee29e3801a,0x009a55a678f8dba4,0x0065445dcff9be40,0x00f3978789a9abc5,0x00001f010d23f5e8,0x00ff80042934b0c5)},-  {FIELD_LITERAL(0x00a6749f4b3f9745,0x003ab85f4180e502,0x006a7de9b530ed50,0x0050b5353b0441bf,0x00a093583ac6ede4,0x00c4918ad1406299,0x000f75cf2a353a2b,0x001c6644a0683a56)},-  {FIELD_LITERAL(0x00e8694156c09bfe,0x00f6f3a5bd17ad96,0x0098dbed45edad12,0x00edfe2b84921821,0x0097884330199b67,0x004aab02685b3e9e,0x0068ac0bd2453c30,0x00167c1c1c87d8f5)},-  {FIELD_LITERAL(0x008bba5fbf63f599,0x0059a3c960c7d63f,0x00ce2db75b08b7d9,0x0097e80cb2104171,0x009b68be26a140d0,0x002b9b9954e94c68,0x00023ca8fc411beb,0x00cbc4bcccbada07)},-  {FIELD_LITERAL(0x0053c100e77b678d,0x000f115c400fa96f,0x005928d3de22afa2,0x00e47cd9bdbdbe96,0x00597ecfe84abf19,0x0058bb428e4c7a32,0x00dd582f76ecf584,0x00b1211365eccb79)},-  {FIELD_LITERAL(0x00dbfb9a00a58e68,0x004468189350d82f,0x00b4b12407ee92c6,0x00e27a7908f73455,0x00f071170071b5ae,0x00221a5e6ba229dd,0x001903e3f6a81f83,0x00be36325402775f)},-  {FIELD_LITERAL(0x004d298d6e691756,0x00775644dfce310b,0x00a861887823ea98,0x00cf0b6014fa6e6f,0x005f4e296380826f,0x00bf423392627f90,0x002893bfc8122f6a,0x00440dbc89bea228)},-  {FIELD_LITERAL(0x00acbb4f40a4ab73,0x00d6a82f48fa3366,0x000a7958fc6faac2,0x008a4cdd60a7c33c,0x005e5587dd8b6f1a,0x00e40f63086a88e8,0x0030940cbbcda0ad,0x009a42e3dc35c130)},-  {FIELD_LITERAL(0x00d37716cad825f1,0x00883870cba9552a,0x008ef785f5c762e3,0x006cb253e0469242,0x007b8f17fee9d967,0x00a43de6932b52b6,0x001aca9fe2af783c,0x008967778ff0b680)},-  {FIELD_LITERAL(0x006400c4cdc6c9c3,0x001e8c978691083f,0x00ad74f01f68e0c5,0x00f7feb0372b5f6a,0x002f60d175ade13a,0x0098ec54a221a678,0x00fcfea8a71f244e,0x00dea6660e45ded2)},-  {FIELD_LITERAL(0x002585b4aa8d6752,0x00e62da7615a2089,0x0010c1c741f39b68,0x00569bb1eced9f65,0x00ba6d09e4daa724,0x007d3e20aef281b9,0x00bd7f65aca3ffdc,0x00dea434a50288a8)},-  {FIELD_LITERAL(0x007ba92a2489170f,0x00cd356354d31e9c,0x00a60d47406e5430,0x009c3d5fde8ed877,0x00079eaa50dd08d1,0x0024674d593ffa5f,0x005391be9596c53b,0x00856ca8d50acdd9)},-  {FIELD_LITERAL(0x00d4620aa5e5bdec,0x002303c4b9b5d941,0x003b061f857ebb2a,0x00371f9e856d49fd,0x0071c36c5335051e,0x0040e4346a4d359f,0x00b31dbd959ec40c,0x00d99353a71bf6de)},-  {FIELD_LITERAL(0x0078898adf0f21dd,0x006e09bfedd8604a,0x00efaf0e0f9bb666,0x00b0f685db8852c3,0x0094c86ec566b841,0x00e5c2879ba50dbe,0x00a87cd444cff758,0x00d3e26fd47f23df)},-  {FIELD_LITERAL(0x00b82c07fb1854f8,0x0057f654a06fad9f,0x004c00383250cf92,0x008b91713d291af6,0x002f2521777859b9,0x00533111421f22c8,0x00643da86fab9794,0x00dc7fb0680e3d40)},-  {FIELD_LITERAL(0x00e59ffd40e87788,0x006431e9755a50af,0x00a03ce700fb580a,0x00ad7e70aa3c9b9e,0x0078970a2b4db503,0x00c800451849637a,0x00e7e6a5b49e123f,0x00e1ed15f77bcb4d)},-  {FIELD_LITERAL(0x00bc1d1d1af47f28,0x00ebc5501bbd81f0,0x00aa6b5513547aa4,0x0074ed33551343fe,0x00d2114f6ef7d43b,0x006335b41d518aeb,0x00ebd46919692fb8,0x0052d5d4e3fada95)},-  {FIELD_LITERAL(0x00ebfc9f489799a4,0x00497535b6980688,0x00fef76499e6a51b,0x00018eedde7a18da,0x00f435d9e72b69c7,0x005ab0faa8281675,0x003232d06e290be8,0x005473ec8be0286c)},-  {FIELD_LITERAL(0x00c6eb0d0ebb4874,0x00856a2274119097,0x00380bc7b29e3719,0x00b1ae149f0e424d,0x0009b41855b9de26,0x0098684013d0f53f,0x0082e8554c38a6ff,0x00e76c18c353743a)},-  {FIELD_LITERAL(0x008da1194e1ab61f,0x008edb5f89688805,0x00f4970252f851bd,0x007a46f632b6ad20,0x006d2d1c37e9f90a,0x0060dd09353f665f,0x000a625a80d86657,0x000f93f6fedd0888)},-  {FIELD_LITERAL(0x003b019b31992fb4,0x004f6a2ad1f64c28,0x008a744134e5c571,0x000ca33172f9af3f,0x00d478755a67bb8b,0x009d1f5c48abb223,0x004da4d6f12ee901,0x0084f09541f4140d)},-  {FIELD_LITERAL(0x0031f412f5cacd43,0x00e5afb75dd20e94,0x001ce24b3452740e,0x00176d6dedf30ff1,0x0082e22e564fffca,0x001d56fbe007097f,0x0095b37c851a6918,0x008ec50ef97f8f4c)},-  {FIELD_LITERAL(0x007e2b1c52251f57,0x00cbef37c9380033,0x0037ed652761bceb,0x00f1c2a5dc6dd232,0x0026e1b90d63ce0b,0x00938d732173a6b8,0x00d439aa45da993f,0x00d356b8deaccef7)},-  {FIELD_LITERAL(0x00ed32377f56c67d,0x00c3b6a4de32e4a7,0x00481a36c0dd5d91,0x00bb557d20466ba7,0x00645f6d3200163e,0x005eb4c54df7c48c,0x00fd8e3d08f1e3b4,0x001156353f099147)},-  {FIELD_LITERAL(0x00ae1b4c089b2756,0x00e686d2b916fb5f,0x007ac43ec2437dd8,0x00f7bfdf7e860ed2,0x0097dbcb8b786dc9,0x00ec7a90401c8b2f,0x00425ed017989bdb,0x00444bc9ca6d914d)},-  {FIELD_LITERAL(0x00e5e7b83b53ab7f,0x004e4bed6ca44fc5,0x0008bd7a67c40d4d,0x009dbec74a4a2f0e,0x0077df3f4fc2c73f,0x0046b1af5e73ea8d,0x009f096cb7be8670,0x003ad0a29929141d)},-  {FIELD_LITERAL(0x00991a1222e9b2e1,0x00be7583901d7dc7,0x00fd1d0c8169d3da,0x000fe0a94a68acf9,0x00b77bd05afc78a2,0x00a84f1697f87ebc,0x000097cfdb0c2ecb,0x007d51d70352ed1b)},-  {FIELD_LITERAL(0x0025dc2a60643159,0x001f0d8ff85f95b4,0x00ed74a4bc598a73,0x00f30afe6f0574a9,0x0003788545d4d28c,0x009dc410ad120ac0,0x001950947e69961d,0x001ceb23cb0355b0)},-  {FIELD_LITERAL(0x00ee2202ded9f1bd,0x002fa4fce658976d,0x00e7c15bc9716470,0x004f7ea99d500369,0x004b995a18318376,0x00246c4f8af91911,0x00cc77a07d09dbfe,0x007906f6f1364be6)},-  {FIELD_LITERAL(0x003c97e6384da36e,0x00423d53eac81a09,0x00b70d68f3cdce35,0x00ee7959b354b92c,0x00f4e9718819c8ca,0x009349f12acbffe9,0x005aee7b62cb7da6,0x00d97764154ffc86)},+  {FIELD_LITERAL(0x00cc3b062366f4cc,0x003d6e34e314aa3c,0x00d51c0a7521774d,0x0094e060eec6ab8b,0x00d21291b4d80082,0x00befed12b55ef1e,0x00c3dd2df5c94518,0x00e0a7b112b8d4e6)},+  {FIELD_LITERAL(0x0019eb5608d8723a,0x00d1bab52fb3aedb,0x00270a7311ebc90c,0x0037c12b91be7f13,0x005be16cd8b5c704,0x003e181acda888e1,0x00bc1f00fc3fc6d0,0x00d3839bfa319e20)},+  {FIELD_LITERAL(0x003caeb88611909f,0x00ea8b378c4df3d4,0x00b3295b95a5a19a,0x00a65f97514bdfb5,0x00b39efba743cab1,0x0016ba98b862fd2d,0x0001508812ee71d7,0x000a75740eea114a)},+  {FIELD_LITERAL(0x00ebcf0eb649f823,0x00166d332e98ea03,0x0059ddf64f5cd5f6,0x0047763123d9471b,0x00a64065c53ef62f,0x00978e44c480153d,0x000b5b2a0265f194,0x0046a24b9f32965a)},+  {FIELD_LITERAL(0x00b9eef787034df0,0x0020bc24de3390cd,0x000022160bae99bb,0x00ae66e886e97946,0x0048d4bbe02cbb8b,0x0072ba97b34e38d4,0x00eae7ec8f03e85a,0x005ba92ecf808b2c)},+  {FIELD_LITERAL(0x00c9cfbbe74258fd,0x00843a979ea9eaa7,0x000cbb4371cfbe90,0x0059bac8f7f0a628,0x004b3dff882ff530,0x0011869df4d90733,0x00595aa71f4abfc2,0x0070e2d38990c2e6)},+  {FIELD_LITERAL(0x00de2010c0a01733,0x00c739a612e24297,0x00a7212643141d7c,0x00f88444f6b67c11,0x00484b7b16ec28f2,0x009c1b8856af9c68,0x00ff4669591fe9d6,0x0054974be08a32c8)},+  {FIELD_LITERAL(0x0010de3fd682ceed,0x008c07642d83ca4e,0x0013bb064e00a1cc,0x009411ae27870e11,0x00ea8e5b4d531223,0x0032fe7d2aaece2e,0x00d989e243e7bb41,0x000fe79a508e9b8b)},+  {FIELD_LITERAL(0x005e0426b9bfc5b1,0x0041a5b1d29ee4fa,0x0015b0def7774391,0x00bc164f1f51af01,0x00d543b0942797b9,0x003c129b6398099c,0x002b114c6e5adf18,0x00b4e630e4018a7b)},+  {FIELD_LITERAL(0x00d490afc95f8420,0x00b096bf50c1d9b9,0x00799fd707679866,0x007c74d9334afbea,0x00efaa8be80ff4ed,0x0075c4943bb81694,0x00c21c2fca161f36,0x00e77035d492bfee)},+  {FIELD_LITERAL(0x006658a190dd6661,0x00e0e9bab38609a6,0x0028895c802237ed,0x006a0229c494f587,0x002dcde96c9916b7,0x00d158822de16218,0x00173b917a06856f,0x00ca78a79ae07326)},+  {FIELD_LITERAL(0x00e35bfc79caced4,0x0087238a3e1fe3bb,0x00bcbf0ff4ceff5b,0x00a19c1c94099b91,0x0071e102b49db976,0x0059e3d004eada1e,0x008da78afa58a47e,0x00579c8ebf269187)},+  {FIELD_LITERAL(0x00a16c2905eee75f,0x009d4bcaea2c7e1d,0x00d3bd79bfad19df,0x0050da745193342c,0x006abdb8f6b29ab1,0x00a24fe0a4fef7ef,0x0063730da1057dfb,0x00a08c312c8eb108)},+  {FIELD_LITERAL(0x00b583be005375be,0x00a40c8f8a4e3df4,0x003fac4a8f5bdbf7,0x00d4481d872cd718,0x004dc8749cdbaefe,0x00cce740d5e5c975,0x000b1c1f4241fd21,0x00a76de1b4e1cd07)},+  {FIELD_LITERAL(0x007a076500d30b62,0x000a6e117b7f090f,0x00c8712ae7eebd9a,0x000fbd6c1d5f6ff7,0x003a7977246ebf11,0x00166ed969c6600e,0x00aa42e469c98bec,0x00dc58f307cf0666)},+  {FIELD_LITERAL(0x004b491f65a9a28b,0x006a10309e8a55b7,0x00b67210185187ef,0x00cf6497b12d9b8f,0x0085778c56e2b1ba,0x0015b4c07a814d85,0x00686479e62da561,0x008de5d88f114916)},+  {FIELD_LITERAL(0x00e37c88d6bba7b1,0x003e4577e1b8d433,0x0050d8ea5f510ec0,0x0042fc9f2da9ef59,0x003bd074c1141420,0x00561b8b7b68774e,0x00232e5e5d1013a3,0x006b7f2cb3d7e73f)},+  {FIELD_LITERAL(0x004bdd0f0b41e6a0,0x001773057c405d24,0x006029f99915bd97,0x006a5ba70a17fe2f,0x0046111977df7e08,0x004d8124c89fb6b7,0x00580983b2bb2724,0x00207bf330d6f3fe)},+  {FIELD_LITERAL(0x007efdc93972a48b,0x002f5e50e78d5fee,0x0080dc11d61c7fe5,0x0065aa598707245b,0x009abba2300641be,0x000c68787656543a,0x00ffe0fef2dc0a17,0x00007ffbd6cb4f3a)},+  {FIELD_LITERAL(0x0036012f2b836efc,0x00458c126d6b5fbc,0x00a34436d719ad1e,0x0097be6167117dea,0x0009c219c879cff3,0x0065564493e60755,0x00993ac94a8cdec0,0x002d4885a4d0dbaf)},+  {FIELD_LITERAL(0x00598b60b4c068ba,0x00c547a0be7f1afd,0x009582164acf12af,0x00af4acac4fbbe40,0x005f6ca7c539121a,0x003b6e752ebf9d66,0x00f08a30d5cac5d4,0x00e399bb5f97c5a9)},+  {FIELD_LITERAL(0x007445a0409c0a66,0x00a65c369f3829c0,0x0031d248a4f74826,0x006817f34defbe8e,0x00649741d95ebf2e,0x00d46466ab16b397,0x00fdc35703bee414,0x00343b43334525f8)},+  {FIELD_LITERAL(0x001796bea93f6401,0x00090c5a42e85269,0x00672412ba1252ed,0x001201d47b6de7de,0x006877bccfe66497,0x00b554fd97a4c161,0x009753f42dbac3cf,0x00e983e3e378270a)},+  {FIELD_LITERAL(0x00ac3eff18849872,0x00f0eea3bff05690,0x00a6d72c21dd505d,0x001b832642424169,0x00a6813017b540e5,0x00a744bd71b385cd,0x0022a7d089130a7b,0x004edeec9a133486)},+  {FIELD_LITERAL(0x00b2d6729196e8a9,0x0088a9bb2031cef4,0x00579e7787dc1567,0x0030f49feb059190,0x00a0b1d69c7f7d8f,0x0040bdcc6d9d806f,0x00d76c4037edd095,0x00bbf24376415dd7)},+  {FIELD_LITERAL(0x00240465ff5a7197,0x00bb97e76caf27d0,0x004b4edbf8116d39,0x001d8586f708cbaa,0x000f8ee8ff8e4a50,0x00dde5a1945dd622,0x00e6fc1c0957e07c,0x0041c9cdabfd88a0)},+  {FIELD_LITERAL(0x005344b0bf5b548c,0x002957d0b705cc99,0x00f586a70390553d,0x0075b3229f583cc3,0x00a1aa78227490e4,0x001bf09cf7957717,0x00cf6bf344325f52,0x0065bd1c23ca3ecf)},+  {FIELD_LITERAL(0x009bff3b3239363c,0x00e17368796ef7c0,0x00528b0fe0971f3a,0x0008014fc8d4a095,0x00d09f2e8a521ec4,0x006713ab5dde5987,0x0003015758e0dbb1,0x00215999f1ba212d)},+  {FIELD_LITERAL(0x002c88e93527da0e,0x0077c78f3456aad5,0x0071087a0a389d1c,0x00934dac1fb96dbd,0x008470e801162697,0x005bc2196cd4ad49,0x00e535601d5087c3,0x00769888700f497f)},+  {FIELD_LITERAL(0x00da7a4b557298ad,0x0019d2589ea5df76,0x00ef3e38be0c6497,0x00a9644e1312609a,0x004592f61b2558da,0x0082c1df510d7e46,0x0042809a535c0023,0x00215bcb5afd7757)},+  {FIELD_LITERAL(0x002b9df55a1a4213,0x00dcfc3b464a26be,0x00c4f9e07a8144d5,0x00c8e0617a92b602,0x008e3c93accafae0,0x00bf1bcb95b2ca60,0x004ce2426a613bf3,0x00266cac58e40921)},+  {FIELD_LITERAL(0x008456d5db76e8f0,0x0032ca9cab2ce163,0x0059f2b8bf91abcf,0x0063c2a021712788,0x00f86155af22f72d,0x00db98b2a6c005a0,0x00ac6e416a693ac4,0x007a93572af53226)},+  {FIELD_LITERAL(0x0087767520f0de22,0x0091f64012279fb5,0x001050f1f0644999,0x004f097a2477ad3c,0x006b37913a9947bd,0x001a3d78645af241,0x0057832bbb3008a7,0x002c1d902b80dc20)},+  {FIELD_LITERAL(0x001a6002bf178877,0x009bce168aa5af50,0x005fc318ff04a7f5,0x0052818f55c36461,0x008768f5d4b24afb,0x0037ffbae7b69c85,0x0018195a4b61edc0,0x001e12ea088434b2)},+  {FIELD_LITERAL(0x0047d3f804e7ab07,0x00a809ab5f905260,0x00b3ffc7cdaf306d,0x00746e8ec2d6e509,0x00d0dade8887a645,0x00acceeebde0dd37,0x009bc2579054686b,0x0023804f97f1c2bf)},+  {FIELD_LITERAL(0x0043e2e2e50b80d7,0x00143aafe4427e0f,0x005594aaecab855b,0x008b12ccaaecbc01,0x002deeb091082bc3,0x009cca4be2ae7514,0x00142b96e696d047,0x00ad2a2b1c05256a)},+  {FIELD_LITERAL(0x003914f2f144b78b,0x007a95dd8bee6f68,0x00c7f4384d61c8e6,0x004e51eb60f1bdb2,0x00f64be7aa4621d8,0x006797bfec2f0ac0,0x007d17aab3c75900,0x001893e73cac8bc5)},+  {FIELD_LITERAL(0x00140360b768665b,0x00b68aca4967f977,0x0001089b66195ae4,0x00fe71122185e725,0x000bca2618d49637,0x00a54f0557d7e98a,0x00cdcd2f91d6f417,0x00ab8c13741fd793)},+  {FIELD_LITERAL(0x00725ee6b1e549e0,0x007124a0769777fa,0x000b68fdad07ae42,0x0085b909cd4952df,0x0092d2e3c81606f4,0x009f22f6cac099a0,0x00f59da57f2799a8,0x00f06c090122f777)},+  {FIELD_LITERAL(0x00ce0bed0a3532bc,0x001a5048a22df16b,0x00e31db4cbad8bf1,0x00e89292120cf00e,0x007d1dd1a9b00034,0x00e2a9041ff8f680,0x006a4c837ae596e7,0x00713af1068070b3)},+  {FIELD_LITERAL(0x00c4fe64ce66d04b,0x00b095d52e09b3d7,0x00758bbecb1a3a8e,0x00f35cce8d0650c0,0x002b878aa5984473,0x0062e0a3b7544ddc,0x00b25b290ed116fe,0x007b0f6abe0bebf2)},+  {FIELD_LITERAL(0x0081d4e3addae0a8,0x003410c836c7ffcc,0x00c8129ad89e4314,0x000e3d5a23922dcd,0x00d91e46f29c31f3,0x006c728cde8c5947,0x002bc655ba2566c0,0x002ca94721533108)},+  {FIELD_LITERAL(0x0051e4b3f764d8a9,0x0019792d46e904a0,0x00853bc13dbc8227,0x000840208179f12d,0x0068243474879235,0x0013856fbfe374d0,0x00bda12fe8676424,0x00bbb43635926eb2)},+  {FIELD_LITERAL(0x0012cdc880a93982,0x003c495b21cd1b58,0x00b7e5c93f22a26e,0x0044aa82dfb99458,0x009ba092cdffe9c0,0x00a14b3ab2083b73,0x000271c2f70e1c4b,0x00eea9cac0f66eb8)},+  {FIELD_LITERAL(0x001a1847c4ac5480,0x00b1b412935bb03a,0x00f74285983bf2b2,0x00624138b5b5d0f1,0x008820c0b03d38bf,0x00b94e50a18c1572,0x0060f6934841798f,0x00c52f5d66d6ebe2)},+  {FIELD_LITERAL(0x00da23d59f9bcea6,0x00e0f27007a06a4b,0x00128b5b43a6758c,0x000cf50190fa8b56,0x00fc877aba2b2d72,0x00623bef52edf53f,0x00e6af6b819669e2,0x00e314dc34fcaa4f)},+  {FIELD_LITERAL(0x0066e5eddd164d1e,0x00418a7c6fe28238,0x0002e2f37e962c25,0x00f01f56b5975306,0x0048842fa503875c,0x0057b0e968078143,0x00ff683024f3d134,0x0082ae28fcad12e4)},+  {FIELD_LITERAL(0x0011ddfd21260e42,0x00d05b0319a76892,0x00183ea4368e9b8f,0x00b0815662affc96,0x00b466a5e7ce7c88,0x00db93b07506e6ee,0x0033885f82f62401,0x0086f9090ec9b419)},   {FIELD_LITERAL(0x00d95d1c5fcb435a,0x0016d1ed6b5086f9,0x00792aa0b7e54d71,0x0067b65715f1925d,0x00a219755ec6176b,0x00bc3f026b12c28f,0x00700c897ffeb93e,0x0089b83f6ec50b46)},-  {FIELD_LITERAL(0x00ad9cdb4544b923,0x00d11664c7284061,0x00815ae86b8f910b,0x005414fb2591c3c6,0x0094ba83e2d7ef9e,0x0001dbc16599386c,0x00c8721f0493911b,0x00c1be6b463c346c)},-  {FIELD_LITERAL(0x0079680ce111ed3b,0x001a1ed82806122c,0x000c2e7466d15df3,0x002c407f6f7150fd,0x00c5e7c96b1b0ce3,0x009aa44626863ff9,0x00887b8b5b80be42,0x00b6023cec964825)},+  {FIELD_LITERAL(0x003c97e6384da36e,0x00423d53eac81a09,0x00b70d68f3cdce35,0x00ee7959b354b92c,0x00f4e9718819c8ca,0x009349f12acbffe9,0x005aee7b62cb7da6,0x00d97764154ffc86)},+  {FIELD_LITERAL(0x00526324babb46dc,0x002ee99b38d7bf9e,0x007ea51794706ef4,0x00abeb04da6e3c39,0x006b457c1d281060,0x00fe243e9a66c793,0x00378de0fb6c6ee4,0x003e4194b9c3cb93)},   {FIELD_LITERAL(0x00fed3cd80ca2292,0x0015b043a73ca613,0x000a9fd7bf9be227,0x003b5e03de2db983,0x005af72d46904ef7,0x00c0f1b5c49faa99,0x00dc86fc3bd305e1,0x00c92f08c1cb1797)},-  {FIELD_LITERAL(0x001b571efb768f37,0x009d778487cf5cfd,0x00430e37327ebfd4,0x00a92447e5970a41,0x00eb13127c0edbac,0x00ec61e5aefeaf20,0x00447eebf57d2e5c,0x00f01433e550e558)},-  {FIELD_LITERAL(0x0039dd7ce7fc6860,0x00d64f6425653da1,0x003e037c7f57d0af,0x0063477a06e2bcf2,0x001727dbb7ac67e6,0x0049589f5efafe2e,0x00fc0fef2e813d54,0x008baa5d087fb50d)},+  {FIELD_LITERAL(0x0079680ce111ed3b,0x001a1ed82806122c,0x000c2e7466d15df3,0x002c407f6f7150fd,0x00c5e7c96b1b0ce3,0x009aa44626863ff9,0x00887b8b5b80be42,0x00b6023cec964825)},+  {FIELD_LITERAL(0x00e4a8e1048970c8,0x0062887b7830a302,0x00bcf1c8cd81402b,0x0056dbb81a68f5be,0x0014eced83f12452,0x00139e1a510150df,0x00bb81140a82d1a3,0x000febcc1aaf1aa7)},   {FIELD_LITERAL(0x00a7527958238159,0x0013ec9537a84cd6,0x001d7fee7d562525,0x00b9eefa6191d5e5,0x00dbc97db70bcb8a,0x00481affc7a4d395,0x006f73d3e70c31bb,0x00183f324ed96a61)},-  {FIELD_LITERAL(0x00db04a6264ba838,0x00582b1f9fddc1b3,0x003ee72e4aaa027f,0x007d1de938cd0dd5,0x0032d5d66cf76afa,0x00c9c717c95c1ec2,0x00f27aa11764b8d6,0x00713a482b7ef36e)},-  {FIELD_LITERAL(0x00ece96f95f2b66f,0x00ece7952813a27b,0x0026fc36592e489e,0x007157d1a2de0f66,0x00759dc111d86ddf,0x0012881e5780bb0f,0x00c8ccc83ad29496,0x0012b9bd1929eb71)},+  {FIELD_LITERAL(0x0039dd7ce7fc6860,0x00d64f6425653da1,0x003e037c7f57d0af,0x0063477a06e2bcf2,0x001727dbb7ac67e6,0x0049589f5efafe2e,0x00fc0fef2e813d54,0x008baa5d087fb50d)},+  {FIELD_LITERAL(0x0024fb59d9b457c7,0x00a7d4e060223e4c,0x00c118d1b555fd80,0x0082e216c732f22a,0x00cd2a2993089504,0x003638e836a3e13d,0x000d855ee89b4729,0x008ec5b7d4810c91)},   {FIELD_LITERAL(0x001bf51f7d65cdfd,0x00d14cdafa16a97d,0x002c38e60fcd10e7,0x00a27446e393efbd,0x000b5d8946a71fdd,0x0063df2cde128f2f,0x006c8679569b1888,0x0059ffc4925d732d)},-  {FIELD_LITERAL(0x00f05ea5df25a20f,0x00cb6224e5b932ce,0x00d3aed52e2718d9,0x00fb89ee0996ce72,0x006197045a6e1e80,0x00bcdf20057fc6f9,0x0059bf78b6ae5c2c,0x0049cacb87455db0)},-  {FIELD_LITERAL(0x006a15bb20f75c0c,0x0079a144027a5d0c,0x00d19116ce0b4d70,0x0059b83bcb0b268e,0x005f58f63f16c127,0x0079958318ee2c37,0x00defbb063d07f82,0x00f1f0b931d2d446)},+  {FIELD_LITERAL(0x00ece96f95f2b66f,0x00ece7952813a27b,0x0026fc36592e489e,0x007157d1a2de0f66,0x00759dc111d86ddf,0x0012881e5780bb0f,0x00c8ccc83ad29496,0x0012b9bd1929eb71)},+  {FIELD_LITERAL(0x000fa15a20da5df0,0x00349ddb1a46cd31,0x002c512ad1d8e726,0x00047611f669318d,0x009e68fba591e17e,0x004320dffa803906,0x00a640874951a3d3,0x00b6353478baa24f)},   {FIELD_LITERAL(0x009696510000d333,0x00ec2f788bc04826,0x000e4d02b1f67ba5,0x00659aa8dace08b6,0x00d7a38a3a3ae533,0x008856defa8c746b,0x004d7a4402d3da1a,0x00ea82e06229260f)},-  {FIELD_LITERAL(0x0034a1b3c3ca2bdd,0x0072077a35bca880,0x0005af4e935c1b8e,0x00a5f1a71e8b7737,0x004d3133292cb2e5,0x000fe2a2dca1c916,0x0024d181b41935bb,0x00d9f54880ca0332)},-  {FIELD_LITERAL(0x009ffd90abfeae96,0x00cba3c2b624a516,0x005ef08bcee46c91,0x00e6fde30afb6185,0x00f0b4db4f818ce4,0x006c54f45d2127f5,0x00040125035854c7,0x00372658a3287e13)},+  {FIELD_LITERAL(0x006a15bb20f75c0c,0x0079a144027a5d0c,0x00d19116ce0b4d70,0x0059b83bcb0b268e,0x005f58f63f16c127,0x0079958318ee2c37,0x00defbb063d07f82,0x00f1f0b931d2d446)},+  {FIELD_LITERAL(0x00cb5e4c3c35d422,0x008df885ca43577f,0x00fa50b16ca3e471,0x005a0e58e17488c8,0x00b2ceccd6d34d19,0x00f01d5d235e36e9,0x00db2e7e4be6ca44,0x00260ab77f35fccd)},   {FIELD_LITERAL(0x006f6fd9baac61d5,0x002a7710a020a895,0x009de0db7fc03d4d,0x00cdedcb1875f40b,0x00050caf9b6b1e22,0x005e3a6654456ab0,0x00775fdf8c4423d4,0x0028701ea5738b5d)},-  {FIELD_LITERAL(0x0028f8f04e414d54,0x0087037ba56c7694,0x00976b5b4d0ddb59,0x00a4227e6d462421,0x004c77c678b4c560,0x0006c9e74fb485a8,0x00c1c138a02d3981,0x0040a19403d6b6b5)},-  {FIELD_LITERAL(0x0045e8dda9400888,0x002ff12e5fc05db7,0x00a7098d54afe69c,0x00cdbe846a500585,0x00879c1593ca1882,0x003f7a7fea76c8b0,0x002cd73dd0c8e0a1,0x00645d6ce96f51fe)},+  {FIELD_LITERAL(0x009ffd90abfeae96,0x00cba3c2b624a516,0x005ef08bcee46c91,0x00e6fde30afb6185,0x00f0b4db4f818ce4,0x006c54f45d2127f5,0x00040125035854c7,0x00372658a3287e13)},+  {FIELD_LITERAL(0x00d7070fb1beb2ab,0x0078fc845a93896b,0x006894a4b2f224a6,0x005bdd8192b9dbde,0x00b38839874b3a9e,0x00f93618b04b7a57,0x003e3ec75fd2c67e,0x00bf5e6bfc29494a)},   {FIELD_LITERAL(0x00f19224ebba2aa5,0x0074f89d358e694d,0x00eea486597135ad,0x0081579a4555c7e1,0x0010b9b872930a9d,0x00f002e87a30ecc0,0x009b9d66b6de56e2,0x00a3c4f45e8004eb)},-  {FIELD_LITERAL(0x00d4817c1edc2929,0x00c67cb908be637f,0x00bd6dd1aa6bfe9c,0x00a1803a9fe7795c,0x001770d311e2cefb,0x0018054eca0d1c88,0x004fa667b240f212,0x00f631f7f055a447)},-  {FIELD_LITERAL(0x00f89335c2a59286,0x00a0f5c905d55141,0x00b41fb836ee9382,0x00e235d51730ca43,0x00a5cb37b5c0a69a,0x009b966ffe136c45,0x00cb2ea10bf80ed1,0x00fb2b370b40dc35)},+  {FIELD_LITERAL(0x0045e8dda9400888,0x002ff12e5fc05db7,0x00a7098d54afe69c,0x00cdbe846a500585,0x00879c1593ca1882,0x003f7a7fea76c8b0,0x002cd73dd0c8e0a1,0x00645d6ce96f51fe)},+  {FIELD_LITERAL(0x002b7e83e123d6d6,0x00398346f7419c80,0x0042922e55940163,0x005e7fc5601886a3,0x00e88f2cee1d3103,0x00e7fab135f2e377,0x00b059984dbf0ded,0x0009ce080faa5bb8)},   {FIELD_LITERAL(0x0085e78af7758979,0x00275a4ee1631a3a,0x00d26bc0ed78b683,0x004f8355ea21064f,0x00d618e1a32696e5,0x008d8d7b150e5680,0x00a74cd854b278d2,0x001dd62702203ea0)},-  {FIELD_LITERAL(0x0029782e92b11745,0x008eadf422f96200,0x00217a39f2cdcaa2,0x00782d1ca9aefd0b,0x00321c6e47203654,0x001e72961020101a,0x00b562fa6e6ab16e,0x0005c92274af111a)},-  {FIELD_LITERAL(0x006bc3d53011f470,0x00032d6e692b83e8,0x00059722f497cd0b,0x0009b4e6f0c497cc,0x0058a804b7cce6c0,0x002b71d3302bbd5d,0x00e2f82a36765fce,0x008dded99524c703)},+  {FIELD_LITERAL(0x00f89335c2a59286,0x00a0f5c905d55141,0x00b41fb836ee9382,0x00e235d51730ca43,0x00a5cb37b5c0a69a,0x009b966ffe136c45,0x00cb2ea10bf80ed1,0x00fb2b370b40dc35)},+  {FIELD_LITERAL(0x00d687d16d4ee8ba,0x0071520bdd069dff,0x00de85c60d32355d,0x0087d2e3565102f4,0x00cde391b8dfc9aa,0x00e18d69efdfefe5,0x004a9d0591954e91,0x00fa36dd8b50eee5)},   {FIELD_LITERAL(0x002e788749a865f7,0x006e4dc3116861ea,0x009f1428c37276e6,0x00e7d2e0fc1e1226,0x003aeebc6b6c45f6,0x0071a8073bf500c9,0x004b22ad986b530c,0x00f439e63c0d79d4)},-  {FIELD_LITERAL(0x00b2fa76ac8b829b,0x008fe6bf01865590,0x0059df538e389f40,0x006acd49eeea748a,0x00ab81280b990cfe,0x00c34a54ac57bfe5,0x003889ce9731cedf,0x0081b71cc1b4654d)},-  {FIELD_LITERAL(0x002f194eaafa46dc,0x008e38f57fe87613,0x00dc8e5ae25f4ab2,0x000a17809575e6bd,0x00d3ec7923ba366a,0x003a7e72e0ad75e3,0x0010024b88436e0a,0x00ed3c5444b64051)},+  {FIELD_LITERAL(0x006bc3d53011f470,0x00032d6e692b83e8,0x00059722f497cd0b,0x0009b4e6f0c497cc,0x0058a804b7cce6c0,0x002b71d3302bbd5d,0x00e2f82a36765fce,0x008dded99524c703)},+  {FIELD_LITERAL(0x004d058953747d64,0x00701940fe79aa6f,0x00a620ac71c760bf,0x009532b611158b75,0x00547ed7f466f300,0x003cb5ab53a8401a,0x00c7763168ce3120,0x007e48e33e4b9ab2)},   {FIELD_LITERAL(0x001b2fc57bf3c738,0x006a3f918993fb80,0x0026f7a14fdec288,0x0075a2cdccef08db,0x00d3ecbc9eecdbf1,0x0048c40f06e5bf7f,0x00d63e423009896b,0x000598bc99c056a8)},-  {FIELD_LITERAL(0x007ce03ecbf50cbd,0x00369ba996b992ca,0x00896d4b33a5f7f0,0x00602b5b8536da60,0x00e1122082ba6d73,0x00c3fbb903ba0d74,0x00d3f8ec55c1daf8,0x006a8f96ca0f0be1)},-  {FIELD_LITERAL(0x001fb73475c45509,0x00d2b2e5ea43345a,0x00cb3c3842077bd1,0x0029f90ad820946e,0x007c11b2380778aa,0x009e54ece62c1704,0x004bc60c41ca01c3,0x004525679a5a0b03)},+  {FIELD_LITERAL(0x002f194eaafa46dc,0x008e38f57fe87613,0x00dc8e5ae25f4ab2,0x000a17809575e6bd,0x00d3ec7923ba366a,0x003a7e72e0ad75e3,0x0010024b88436e0a,0x00ed3c5444b64051)},+  {FIELD_LITERAL(0x00831fc1340af342,0x00c9645669466d35,0x007692b4cc5a080f,0x009fd4a47ac9259f,0x001eeddf7d45928b,0x003c0446fc45f28b,0x002c0713aa3e2507,0x0095706935f0f41e)},   {FIELD_LITERAL(0x00766ae4190ec6d8,0x0065768cabc71380,0x00b902598416cdc2,0x00380021ad38df52,0x008f0b89d6551134,0x004254d4cc62c5a5,0x000d79f4484b9b94,0x00b516732ae3c50e)},-  {FIELD_LITERAL(0x0039b0422412784c,0x00bf9fe2ee8ce055,0x0063ddb8a4906298,0x00db48625178a0ea,0x009e9012c0fd3c4e,0x00ff30c60950d2c4,0x003b9453f5565977,0x0054dc1d7ff25dfb)},-  {FIELD_LITERAL(0x0017085f4a346148,0x00c7cf7a37f62272,0x001776e129bc5c30,0x009955134c9eef2a,0x001ba5bdf1df07be,0x00ec39497103a55c,0x006578354fda6cfb,0x005f02719d4f15ee)},+  {FIELD_LITERAL(0x001fb73475c45509,0x00d2b2e5ea43345a,0x00cb3c3842077bd1,0x0029f90ad820946e,0x007c11b2380778aa,0x009e54ece62c1704,0x004bc60c41ca01c3,0x004525679a5a0b03)},+  {FIELD_LITERAL(0x00c64fbddbed87b3,0x0040601d11731faa,0x009c22475b6f9d67,0x0024b79dae875f15,0x00616fed3f02c3b0,0x0000cf39f6af2d3b,0x00c46bac0aa9a688,0x00ab23e2800da204)},   {FIELD_LITERAL(0x000b3a37617632b0,0x00597199fe1cfb6c,0x0042a7ccdfeafdd6,0x004cc9f15ebcea17,0x00f436e596a6b4a4,0x00168861142df0d8,0x000753edfec26af5,0x000c495d7e388116)},-  {FIELD_LITERAL(0x00ad46264a269aa2,0x002b13845e4b9e3c,0x0006a20b68b0d7f4,0x00c271a35ee514ae,0x002b67e14a58f4d8,0x00f5065b099a60d6,0x00ba6737b90514bc,0x00b6265e7c5b898f)},-  {FIELD_LITERAL(0x00b60167d9e7d065,0x00e60ba0d07381e8,0x003a4f17b725c2d4,0x006c19fe176b64fa,0x003b57b31af86ccb,0x0021047c286180fd,0x00bdc8fb00c6dbb6,0x00fe4a9f4bab4f3f)},+  {FIELD_LITERAL(0x0017085f4a346148,0x00c7cf7a37f62272,0x001776e129bc5c30,0x009955134c9eef2a,0x001ba5bdf1df07be,0x00ec39497103a55c,0x006578354fda6cfb,0x005f02719d4f15ee)},+  {FIELD_LITERAL(0x0052b9d9b5d9655d,0x00d4ec7ba1b461c3,0x00f95df4974f280b,0x003d8e5ca11aeb51,0x00d4981eb5a70b26,0x000af9a4f6659f29,0x004598c846faeb43,0x0049d9a183a47670)},   {FIELD_LITERAL(0x000a72d23dcb3f1f,0x00a3737f84011727,0x00f870c0fbbf4a47,0x00a7aadd04b5c9ca,0x000c7715c67bd072,0x00015a136afcd74e,0x0080d5caea499634,0x0026b448ec7514b7)},-  {FIELD_LITERAL(0x0077003c5e9eee08,0x006eaa1bdba2f437,0x007ae297ddfa8d2a,0x00aa8531e1aeb2d6,0x00ce283cc626efdc,0x00efe2f51d153115,0x00db954c07c84995,0x002ade92c7e00acf)},-  {FIELD_LITERAL(0x00a6295218dc136a,0x00563b3af0e9c012,0x00d3753b0145db1b,0x004550389c043dc1,0x00ea94ae27401bdf,0x002b0b949f2b7956,0x00c63f780ad8e23c,0x00e591c47d6bab15)},+  {FIELD_LITERAL(0x00b60167d9e7d065,0x00e60ba0d07381e8,0x003a4f17b725c2d4,0x006c19fe176b64fa,0x003b57b31af86ccb,0x0021047c286180fd,0x00bdc8fb00c6dbb6,0x00fe4a9f4bab4f3f)},+  {FIELD_LITERAL(0x0088ffc3a16111f7,0x009155e4245d0bc8,0x00851d68220572d5,0x00557ace1e514d29,0x0031d7c339d91022,0x00101d0ae2eaceea,0x00246ab3f837b66a,0x00d5216d381ff530)},   {FIELD_LITERAL(0x0057e7ea35f36dae,0x00f47d7ad15de22e,0x00d757ea4b105115,0x008311457d579d7e,0x00b49b75b1edd4eb,0x0081c7ff742fd63a,0x00ddda3187433df6,0x00475727d55f9c66)},-  {FIELD_LITERAL(0x00be93a7d4fa7149,0x00bef825a4d3396a,0x004c32daa951139b,0x003f4be7d981a85e,0x00e866d6ca8642d0,0x00b912bba6f1b2f8,0x00e28ba64c9cf5e1,0x0039504574996955)},-  {FIELD_LITERAL(0x002419222c607674,0x00a7f23af89188b3,0x00ad127284e73d1c,0x008bba582fae1c51,0x00fc6aa7ca9ecab1,0x003df5319eb6c2ba,0x002a05af8a8b199a,0x004bf8354558407c)},+  {FIELD_LITERAL(0x00a6295218dc136a,0x00563b3af0e9c012,0x00d3753b0145db1b,0x004550389c043dc1,0x00ea94ae27401bdf,0x002b0b949f2b7956,0x00c63f780ad8e23c,0x00e591c47d6bab15)},+  {FIELD_LITERAL(0x00416c582b058eb6,0x004107da5b2cc695,0x00b3cd2556aeec64,0x00c0b418267e57a1,0x001799293579bd2e,0x0046ed44590e4d07,0x001d7459b3630a1e,0x00c6afba8b6696aa)},   {FIELD_LITERAL(0x008d6009b26da3f8,0x00898e88ca06b1ca,0x00edb22b2ed7fe62,0x00fbc93516aabe80,0x008b4b470c42ce0d,0x00e0032ba7d0dcbb,0x00d76da3a956ecc8,0x007f20fe74e3852a)},-  {FIELD_LITERAL(0x003182b5cf0f0340,0x002fd3d8d9d60fc2,0x00b73ffe08bff43d,0x00d3dec97fee6a72,0x00675aafc6e16949,0x00d27f499c6f0c86,0x00e0578789f3387a,0x00e52031ab49ec2a)},-  {FIELD_LITERAL(0x006b7a0674f9f8de,0x00a742414e5c7cff,0x0041cbf3c6e13221,0x00e3a64fd207af24,0x0087c05f15fbe8d1,0x004c50936d9e8a33,0x001306ec21042b6d,0x00a4f4137d1141c2)},+  {FIELD_LITERAL(0x002419222c607674,0x00a7f23af89188b3,0x00ad127284e73d1c,0x008bba582fae1c51,0x00fc6aa7ca9ecab1,0x003df5319eb6c2ba,0x002a05af8a8b199a,0x004bf8354558407c)},+  {FIELD_LITERAL(0x00ce7d4a30f0fcbf,0x00d02c272629f03d,0x0048c001f7400bc2,0x002c21368011958d,0x0098a550391e96b5,0x002d80b66390f379,0x001fa878760cc785,0x001adfce54b613d5)},   {FIELD_LITERAL(0x001ed4dc71fa2523,0x005d0bff19bf9b5c,0x00c3801cee065a64,0x001ed0b504323fbf,0x0003ab9fdcbbc593,0x00df82070178b8d2,0x00a2bcaa9c251f85,0x00c628a3674bd02e)},-  {FIELD_LITERAL(0x00f619046dea974f,0x004c39fedfde6ee7,0x00d593cb9f22afc5,0x00624e10ee9ab4ab,0x009c1b40f41869fd,0x0098f2cb44da6d46,0x002311d093becf31,0x004d97d1771880ab)},-  {FIELD_LITERAL(0x00ddbe0750dd1add,0x004b3c7b885844b8,0x00363e7ecf12f1ae,0x0062e953e6438f9d,0x0023cc73b076afe9,0x00b09fa083b4da32,0x00c7c3d2456c541d,0x005b591ec6b694d4)},+  {FIELD_LITERAL(0x006b7a0674f9f8de,0x00a742414e5c7cff,0x0041cbf3c6e13221,0x00e3a64fd207af24,0x0087c05f15fbe8d1,0x004c50936d9e8a33,0x001306ec21042b6d,0x00a4f4137d1141c2)},+  {FIELD_LITERAL(0x0009e6fb921568b0,0x00b3c60120219118,0x002a6c3460dd503a,0x009db1ef11654b54,0x0063e4bf0be79601,0x00670d34bb2592b9,0x00dcee2f6c4130ce,0x00b2682e88e77f54)},   {FIELD_LITERAL(0x000d5b4b3da135ab,0x00838f3e5064d81d,0x00d44eb50f6d94ed,0x0008931ab502ac6d,0x00debe01ca3d3586,0x0025c206775f0641,0x005ad4b6ae912763,0x007e2c318ad8f247)},-  {FIELD_LITERAL(0x00d79a91e629d030,0x00ad5b50fc20eb72,0x00edd89a222eb1bd,0x000ddad6fb098ea8,0x00b8be69a49c90c4,0x009bbe2d69ecd346,0x00a1def906a95a48,0x00db8fd6a6d2cca3)},-  {FIELD_LITERAL(0x00c41d1f9c1f1ac1,0x007b2df4e9f19146,0x00b469355fd5ba7a,0x00b5e1965afc852a,0x00388d5f1e2d8217,0x0022079e4c09ae93,0x0014268acd4ef518,0x00c1dd8d9640464c)},+  {FIELD_LITERAL(0x00ddbe0750dd1add,0x004b3c7b885844b8,0x00363e7ecf12f1ae,0x0062e953e6438f9d,0x0023cc73b076afe9,0x00b09fa083b4da32,0x00c7c3d2456c541d,0x005b591ec6b694d4)},+  {FIELD_LITERAL(0x0028656e19d62fcf,0x0052a4af03df148d,0x00122765ddd14e42,0x00f2252904f67157,0x004741965b636f3a,0x006441d296132cb9,0x005e2106f956a5b7,0x00247029592d335c)},   {FIELD_LITERAL(0x003fe038eb92f894,0x000e6da1b72e8e32,0x003a1411bfcbe0fa,0x00b55d473164a9e4,0x00b9a775ac2df48d,0x0002ddf350659e21,0x00a279a69eb19cb3,0x00f844eab25cba44)},-  {FIELD_LITERAL(0x00c7ad952112f3aa,0x00229739f81c017a,0x0008b9222b75a2a8,0x00bd0d6ad469c483,0x00e344297892a13c,0x00a1cbeb8f435a3d,0x0078e2be1f7a0bec,0x001ac54f670ba8cd)},-  {FIELD_LITERAL(0x00adb2c1566e8b8f,0x0096c68a35771a9a,0x00869933356f334a,0x00ba9c93459f5962,0x009ec73fb6e8ca4b,0x003c3802c27202e1,0x0031f5b733e0c008,0x00f9058c19611fa9)},+  {FIELD_LITERAL(0x00c41d1f9c1f1ac1,0x007b2df4e9f19146,0x00b469355fd5ba7a,0x00b5e1965afc852a,0x00388d5f1e2d8217,0x0022079e4c09ae93,0x0014268acd4ef518,0x00c1dd8d9640464c)},+  {FIELD_LITERAL(0x0038526adeed0c55,0x00dd68c607e3fe85,0x00f746ddd48a5d57,0x0042f2952b963b7c,0x001cbbd6876d5ec2,0x005e341470bca5c2,0x00871d41e085f413,0x00e53ab098f45732)},   {FIELD_LITERAL(0x004d51124797c831,0x008f5ae3750347ad,0x0070ced94c1a0c8e,0x00f6db2043898e64,0x000d00c9a5750cd0,0x000741ec59bad712,0x003c9d11aab37b7f,0x00a67ba169807714)},-  {FIELD_LITERAL(0x00dc70fe7eb5cbde,0x003cda5bb49331d7,0x00dec9068514f18c,0x00f3537d975b501d,0x00dd02de725b8e4b,0x0062327200072106,0x0034607e7e266644,0x00ebc51a91215cb6)},-  {FIELD_LITERAL(0x00a5187e6ee7341b,0x00e6d52e82d83b6e,0x00df3c41323094a7,0x00b3324f444e9de9,0x00689eb21a35bfe5,0x00f16363becd548d,0x00e187cc98e7f60f,0x00127d9062f0ccab)},+  {FIELD_LITERAL(0x00adb2c1566e8b8f,0x0096c68a35771a9a,0x00869933356f334a,0x00ba9c93459f5962,0x009ec73fb6e8ca4b,0x003c3802c27202e1,0x0031f5b733e0c008,0x00f9058c19611fa9)},+  {FIELD_LITERAL(0x00238f01814a3421,0x00c325a44b6cce28,0x002136f97aeb0e73,0x000cac8268a4afe2,0x0022fd218da471b3,0x009dcd8dfff8def9,0x00cb9f8181d999bb,0x00143ae56edea349)},   {FIELD_LITERAL(0x0000623bf87622c5,0x00a1966fdd069496,0x00c315b7b812f9fc,0x00bdf5efcd128b97,0x001d464f532e3e16,0x003cd94f081bfd7e,0x00ed9dae12ce4009,0x002756f5736eee70)},-  {FIELD_LITERAL(0x00b528e4ce3d61bf,0x005a03531ed051d6,0x00bbda4aa68d7f12,0x001810a28e93ccb9,0x00ef4ac525bef536,0x006dcefdd9f9f364,0x006e3d9ed78d6381,0x00774bd6ff0713c4)},-  {FIELD_LITERAL(0x00c13c5aae3ae341,0x009c6c9ed98373e7,0x00098f26864577a8,0x0015b886e9488b45,0x0037692c42aadba5,0x00b83170b8e7791c,0x001670952ece1b44,0x00fd932a39276da2)},+  {FIELD_LITERAL(0x00a5187e6ee7341b,0x00e6d52e82d83b6e,0x00df3c41323094a7,0x00b3324f444e9de9,0x00689eb21a35bfe5,0x00f16363becd548d,0x00e187cc98e7f60f,0x00127d9062f0ccab)},+  {FIELD_LITERAL(0x004ad71b31c29e40,0x00a5fcace12fae29,0x004425b5597280ed,0x00e7ef5d716c3346,0x0010b53ada410ac8,0x0092310226060c9b,0x0091c26128729c7e,0x0088b42900f8ec3b)},   {FIELD_LITERAL(0x00f1e26e9762d4a8,0x00d9d74082183414,0x00ffec9bd57a0282,0x000919e128fd497a,0x00ab7ae7d00fe5f8,0x0054dc442851ff68,0x00c9ebeb3b861687,0x00507f7cab8b698f)},-  {FIELD_LITERAL(0x007e5cda6410cc67,0x00ab7f000be9ef84,0x0031b09f82de4167,0x00c003f7b4be2064,0x00bc2f44effafd2d,0x0013ca0a8a45cd9e,0x0035e70988cff10c,0x001744f57d827ab7)},-  {FIELD_LITERAL(0x009ae3b93a56c404,0x004a410b7a456699,0x00023a619355e6b2,0x009cdc7297387257,0x0055b94d4ae70d04,0x002cbd607f65b005,0x003208b489697166,0x00ea2aa058867370)},+  {FIELD_LITERAL(0x00c13c5aae3ae341,0x009c6c9ed98373e7,0x00098f26864577a8,0x0015b886e9488b45,0x0037692c42aadba5,0x00b83170b8e7791c,0x001670952ece1b44,0x00fd932a39276da2)},+  {FIELD_LITERAL(0x0081a3259bef3398,0x005480fff416107b,0x00ce4f607d21be98,0x003ffc084b41df9b,0x0043d0bb100502d1,0x00ec35f575ba3261,0x00ca18f677300ef3,0x00e8bb0a827d8548)},   {FIELD_LITERAL(0x00df76b3328ada72,0x002e20621604a7c2,0x00f910638a105b09,0x00ef4724d96ef2cd,0x00377d83d6b8a2f7,0x00b4f48805ade324,0x001cd5da8b152018,0x0045af671a20ca7f)},-  {FIELD_LITERAL(0x000d62da6711c0cd,0x004b53ac7a27d523,0x0089cc150fb20e64,0x0055d2c2883154fe,0x00b5dcfd03448874,0x006d80dda2a505cb,0x00b57162afb80dc8,0x007ddb5162431acf)},-  {FIELD_LITERAL(0x00c845923c084294,0x00072419a201bc25,0x0045f408b5f8e669,0x00e9d6a186b74dfe,0x00e19108c68fa075,0x0017b91d874177b7,0x002f0ca2c7912c5a,0x009400aa385a90a2)},+  {FIELD_LITERAL(0x009ae3b93a56c404,0x004a410b7a456699,0x00023a619355e6b2,0x009cdc7297387257,0x0055b94d4ae70d04,0x002cbd607f65b005,0x003208b489697166,0x00ea2aa058867370)},+  {FIELD_LITERAL(0x00f29d2598ee3f32,0x00b4ac5385d82adc,0x007633eaf04df19b,0x00aa2d3d77ceab01,0x004a2302fcbb778a,0x00927f225d5afa34,0x004a8e9d5047f237,0x008224ae9dbce530)},   {FIELD_LITERAL(0x001cf640859b02f8,0x00758d1d5d5ce427,0x00763c784ef4604c,0x005fa81aee205270,0x00ac537bfdfc44cb,0x004b919bd342d670,0x00238508d9bf4b7a,0x00154888795644f3)},-  {FIELD_LITERAL(0x008eeef4feb7de7b,0x003012ffbb0d4107,0x00cb0d6fe30b99d1,0x00c4b51d598067cb,0x003356469016b7ee,0x00addaf85188542f,0x004538bdd8de18c1,0x00999dd4f0c59d4f)},-  {FIELD_LITERAL(0x0026ef1614e160af,0x00c023f9edfc9c76,0x00cff090da5f57ba,0x0076db7a66643ae9,0x0019462f8c646999,0x008fec00b3854b22,0x00d55041692a0a1c,0x0065db894215ca00)},+  {FIELD_LITERAL(0x00c845923c084294,0x00072419a201bc25,0x0045f408b5f8e669,0x00e9d6a186b74dfe,0x00e19108c68fa075,0x0017b91d874177b7,0x002f0ca2c7912c5a,0x009400aa385a90a2)},+  {FIELD_LITERAL(0x0071110b01482184,0x00cfed0044f2bef8,0x0034f2901cf4662e,0x003b4ae2a67f9834,0x00cca9b96fe94810,0x00522507ae77abd0,0x00bac7422721e73e,0x0066622b0f3a62b0)},   {FIELD_LITERAL(0x00f8ac5cf4705b6a,0x00867d82dcb457e3,0x007e13ab2ccc2ce9,0x009ee9a018d3930e,0x008370f8ecb42df8,0x002d9f019add263e,0x003302385b92d196,0x00a15654536e2c0c)},-  {FIELD_LITERAL(0x0056dafc91f5bae3,0x00d5fc6f3c94933e,0x000d8fdf26f76b0b,0x00726f2ad342c280,0x001e2fec8c6d0c46,0x000fe83ea74ae570,0x00353cec2c128243,0x0046657e1c14bd2c)},-  {FIELD_LITERAL(0x008cc9cd236315c0,0x0031d9c5b39fda54,0x00a5713ef37e1171,0x00293d5ae2886325,0x00c4aba3e05015e1,0x0003f35ef78e4fc6,0x0039d6bd3ac1527b,0x0019d7c3afb77106)},+  {FIELD_LITERAL(0x0026ef1614e160af,0x00c023f9edfc9c76,0x00cff090da5f57ba,0x0076db7a66643ae9,0x0019462f8c646999,0x008fec00b3854b22,0x00d55041692a0a1c,0x0065db894215ca00)},+  {FIELD_LITERAL(0x00a925036e0a451c,0x002a0390c36b6cc1,0x00f27020d90894f4,0x008d90d52cbd3d7f,0x00e1d0137392f3b8,0x00f017c158b51a8f,0x00cac313d3ed7dbc,0x00b99a81e3eb42d3)},   {FIELD_LITERAL(0x00b54850275fe626,0x0053a3fd1ec71140,0x00e3d2d7dbe096fa,0x00e4ac7b595cce4c,0x0077bad449c0a494,0x00b7c98814afd5b3,0x0057226f58486cf9,0x00b1557154f0cc57)},-  {FIELD_LITERAL(0x0084e9d6ce567a50,0x0052bf5d1f2558ec,0x00920d83bff60ee7,0x00afc160b1d17413,0x008ae58837d3e7d1,0x00fd676c8896dba4,0x00004e170540611a,0x00f7ccb8f91f6541)},-  {FIELD_LITERAL(0x004246bfcecc627a,0x004ba431246c03a4,0x00bd1d101872d497,0x003b73d3f185ee16,0x001feb2e2678c0e3,0x00ff13c5a89dec76,0x00ed06042e771d8f,0x00a4fd2a897a83dd)},+  {FIELD_LITERAL(0x008cc9cd236315c0,0x0031d9c5b39fda54,0x00a5713ef37e1171,0x00293d5ae2886325,0x00c4aba3e05015e1,0x0003f35ef78e4fc6,0x0039d6bd3ac1527b,0x0019d7c3afb77106)},+  {FIELD_LITERAL(0x007b162931a985af,0x00ad40a2e0daa713,0x006df27c4009f118,0x00503e9f4e2e8bec,0x00751a77c82c182d,0x000298937769245b,0x00ffb1e8fabf9ee5,0x0008334706e09abe)},   {FIELD_LITERAL(0x00dbca4e98a7dcd9,0x00ee29cfc78bde99,0x00e4a3b6995f52e9,0x0045d70189ae8096,0x00fd2a8a3b9b0d1b,0x00af1793b107d8e1,0x00dbf92cbe4afa20,0x00da60f798e3681d)},-  {FIELD_LITERAL(0x0065b5c41af29a68,0x0021ce9a03a5ef69,0x00b0c0a91cba4f38,0x0008408de2a54743,0x00bcec1b84f673ae,0x001b382a3f1e5244,0x00d1c1c24c9afae1,0x005b7f3d32956904)},-  {FIELD_LITERAL(0x004ede34af2813f3,0x00d4a8e11c9e8216,0x004796d5041de8a5,0x00c4c6b4d21cc987,0x00e8a433ee07fa1e,0x0055720b5abcc5a1,0x008873ea9c74b080,0x005b3fec1ab65d48)},+  {FIELD_LITERAL(0x004246bfcecc627a,0x004ba431246c03a4,0x00bd1d101872d497,0x003b73d3f185ee16,0x001feb2e2678c0e3,0x00ff13c5a89dec76,0x00ed06042e771d8f,0x00a4fd2a897a83dd)},+  {FIELD_LITERAL(0x009a4a3be50d6597,0x00de3165fc5a1096,0x004f3f56e345b0c7,0x00f7bf721d5ab8bc,0x004313e47b098c50,0x00e4c7d5c0e1adbb,0x002e3e3db365051e,0x00a480c2cd6a96fb)},   {FIELD_LITERAL(0x00417fa30a7119ed,0x00af257758419751,0x00d358a487b463d4,0x0089703cc720b00d,0x00ce56314ff7f271,0x0064db171ade62c1,0x00640b36d4a22fed,0x00424eb88696d23f)},-  {FIELD_LITERAL(0x00b81ad88248f13a,0x00f5f69399248294,0x004be9b33e8cfea6,0x00b56087c018df01,0x0057e8846bbb6242,0x006a5db00b65a660,0x00963e3a87daf343,0x00badfe6dec2140b)},-  {FIELD_LITERAL(0x001bd59c09e982ea,0x00f72daeb937b289,0x0018b76dca908e0e,0x00edb498512384ad,0x00ce0243b6cc9538,0x00f96ff690cb4e70,0x007c77bf9f673c8d,0x005bf704c088a528)},+  {FIELD_LITERAL(0x004ede34af2813f3,0x00d4a8e11c9e8216,0x004796d5041de8a5,0x00c4c6b4d21cc987,0x00e8a433ee07fa1e,0x0055720b5abcc5a1,0x008873ea9c74b080,0x005b3fec1ab65d48)},+  {FIELD_LITERAL(0x0047e5277db70ec5,0x000a096c66db7d6b,0x00b4164cc1730159,0x004a9f783fe720fe,0x00a8177b94449dbc,0x0095a24ff49a599f,0x0069c1c578250cbc,0x00452019213debf4)},   {FIELD_LITERAL(0x0021ce99e09ebda3,0x00fcbd9f91875ad0,0x009bbf6b7b7a0b5f,0x00388886a69b1940,0x00926a56d0f81f12,0x00e12903c3358d46,0x005dfce4e8e1ce9d,0x0044cfa94e2f7e23)},-  {FIELD_LITERAL(0x006c2b9d7234cc41,0x006ad9c2ae2bda7d,0x00b64cdddba701f9,0x00180318c49ac580,0x00c35d14319f4c95,0x003a21dc65cd415b,0x009c474c28e04940,0x00c65114875e57c6)},-  {FIELD_LITERAL(0x00fb22bb5fd3ce50,0x0017b48aada7ae54,0x00fd5c44ad19a536,0x000ccc4e4e55e45c,0x00fd637d45b4c3f5,0x0038914e023c37cf,0x00ac1881d6a8d898,0x00611ed8d3d943a8)},+  {FIELD_LITERAL(0x001bd59c09e982ea,0x00f72daeb937b289,0x0018b76dca908e0e,0x00edb498512384ad,0x00ce0243b6cc9538,0x00f96ff690cb4e70,0x007c77bf9f673c8d,0x005bf704c088a528)},+  {FIELD_LITERAL(0x0093d4628dcb33be,0x0095263d51d42582,0x0049b3222458fe06,0x00e7fce73b653a7f,0x003ca2ebce60b369,0x00c5de239a32bea4,0x0063b8b3d71fb6bf,0x0039aeeb78a1a839)},   {FIELD_LITERAL(0x007dc52da400336c,0x001fded1e15b9457,0x00902e00f5568e3a,0x00219bef40456d2d,0x005684161fb3dbc9,0x004a4e9be49a76ea,0x006e685ae88b78ff,0x0021c42f13042d3c)},-  {FIELD_LITERAL(0x00a91dda62eec2d4,0x00a6b7e64d7b13e9,0x00384086b44c9969,0x008de118af683239,0x0008e416fb85d76c,0x0020945ebda9b120,0x0096a7f485e7b172,0x000fa91c7035f011)},-  {FIELD_LITERAL(0x005e8694077a1535,0x008bef75f71c8f1d,0x000a7c1316423511,0x00906e1d70604320,0x003fc46c1a2ffbd6,0x00d1d5022e68f360,0x002515fba37bbf46,0x00ca16234e023b44)},+  {FIELD_LITERAL(0x00fb22bb5fd3ce50,0x0017b48aada7ae54,0x00fd5c44ad19a536,0x000ccc4e4e55e45c,0x00fd637d45b4c3f5,0x0038914e023c37cf,0x00ac1881d6a8d898,0x00611ed8d3d943a8)},+  {FIELD_LITERAL(0x0056e2259d113d2b,0x00594819b284ec16,0x00c7bf794bb36696,0x00721ee75097cdc6,0x00f71be9047a2892,0x00df6ba142564edf,0x0069580b7a184e8d,0x00f056e38fca0fee)},   {FIELD_LITERAL(0x009df98566a18c6d,0x00cf3a200968f219,0x0044ba60da6d9086,0x00dbc9c0e344da03,0x000f9401c4466855,0x00d46a57c5b0a8d1,0x00875a635d7ac7c6,0x00ef4a933b7e0ae6)},-  {FIELD_LITERAL(0x00878366a9e0b96f,0x0057a8573ea9e0d8,0x005ef206ddc3f601,0x0046756a9d1c4eab,0x00bccf478bb3c12c,0x001f97ed7f813a3b,0x001b309582460e1c,0x0026a4f760ecd5cb)},-  {FIELD_LITERAL(0x00139078397030bd,0x000e3c447e859a00,0x0064a5b334c82393,0x00b8aabeb7358093,0x00020778bb9ae73b,0x0032ee94c7892a18,0x008215253cb41bda,0x005e2797593517ae)},+  {FIELD_LITERAL(0x005e8694077a1535,0x008bef75f71c8f1d,0x000a7c1316423511,0x00906e1d70604320,0x003fc46c1a2ffbd6,0x00d1d5022e68f360,0x002515fba37bbf46,0x00ca16234e023b44)},+  {FIELD_LITERAL(0x00787c99561f4690,0x00a857a8c1561f27,0x00a10df9223c09fe,0x00b98a9562e3b154,0x004330b8744c3ed2,0x00e06812807ec5c4,0x00e4cf6a7db9f1e3,0x00d95b089f132a34)},   {FIELD_LITERAL(0x002922b39ca33eec,0x0090d12a5f3ab194,0x00ab60c02fb5f8ed,0x00188d292abba1cf,0x00e10edec9698f6e,0x0069a4d9934133c8,0x0024aac40e6d3d06,0x001702c2177661b0)},-  {FIELD_LITERAL(0x007c89a5a07aa2b5,0x00ae492ecae4711d,0x00ee921ab74f0844,0x007842778fc5005f,0x006a4d33cb28022c,0x007b327e4ac0f437,0x007a9d0366acaf12,0x005c6544e6c9ae1c)},-  {FIELD_LITERAL(0x0091868594265aa2,0x00797accae98ca6d,0x0008d8c5f0f8a184,0x00d1f4f1c2b2fe6e,0x0036783dfb48a006,0x008c165120503527,0x0025fd780058ce9b,0x0068beb007be7d27)},+  {FIELD_LITERAL(0x00139078397030bd,0x000e3c447e859a00,0x0064a5b334c82393,0x00b8aabeb7358093,0x00020778bb9ae73b,0x0032ee94c7892a18,0x008215253cb41bda,0x005e2797593517ae)},+  {FIELD_LITERAL(0x0083765a5f855d4a,0x0051b6d1351b8ee2,0x00116de548b0f7bb,0x0087bd88703affa0,0x0095b2cc34d7fdd2,0x0084cd81b53f0bc8,0x008562fc995350ed,0x00a39abb193651e3)},   {FIELD_LITERAL(0x0019e23f0474b114,0x00eb94c2ad3b437e,0x006ddb34683b75ac,0x00391f9209b564c6,0x00083b3bb3bff7aa,0x00eedcd0f6dceefc,0x00b50817f794fe01,0x0036474deaaa75c9)},-  {FIELD_LITERAL(0x002f007755836f3d,0x004d39f2530acc6b,0x006b58d7b2699929,0x004126fdd3185e62,0x003aeaac0f32897c,0x003c0478f4edb66d,0x0072f43ac66a9364,0x0003730da744777a)},-  {FIELD_LITERAL(0x0045fdc16487cda3,0x00b2d8e844cf2ed7,0x00612c50e88c1607,0x00a08aabc66c1672,0x006031fdcbb24d97,0x001b639525744b93,0x004409d62639ab17,0x00a1853d0347ab1d)},+  {FIELD_LITERAL(0x0091868594265aa2,0x00797accae98ca6d,0x0008d8c5f0f8a184,0x00d1f4f1c2b2fe6e,0x0036783dfb48a006,0x008c165120503527,0x0025fd780058ce9b,0x0068beb007be7d27)},+  {FIELD_LITERAL(0x00d0ff88aa7c90c2,0x00b2c60dacf53394,0x0094a7284d9666d6,0x00bed9022ce7a19d,0x00c51553f0cd7682,0x00c3fb870b124992,0x008d0bc539956c9b,0x00fc8cf258bb8885)},   {FIELD_LITERAL(0x003667bf998406f8,0x0000115c43a12975,0x001e662f3b20e8fd,0x0019ffa534cb24eb,0x00016be0dc8efb45,0x00ff76a8b26243f5,0x00ae20d241a541e3,0x0069bd6af13cd430)},-  {FIELD_LITERAL(0x008a5e5a9140a3de,0x005c18d41653ac12,0x0010321e9d6e8f3d,0x00fbdda016e10aca,0x0077fb6038c20257,0x00b5438b7a81ed77,0x00db1dbcb9a8ce83,0x0026734c2c1aabc3)},-  {FIELD_LITERAL(0x007e32c049b5c477,0x009d2bfdbd9bcfd8,0x00636e93045938c6,0x007fde4af7687298,0x0046a5184fafa5d3,0x0079b1e7f13a359b,0x00875adf1fb927d6,0x00333e21c61bcad2)},+  {FIELD_LITERAL(0x0045fdc16487cda3,0x00b2d8e844cf2ed7,0x00612c50e88c1607,0x00a08aabc66c1672,0x006031fdcbb24d97,0x001b639525744b93,0x004409d62639ab17,0x00a1853d0347ab1d)},+  {FIELD_LITERAL(0x0075a1a56ebf5c21,0x00a3e72be9ac53ed,0x00efcde1629170c2,0x0004225fe91ef535,0x0088049fc73dfda7,0x004abc74857e1288,0x0024e2434657317c,0x00d98cb3d3e5543c)},   {FIELD_LITERAL(0x00b4b53eab6bdb19,0x009b22d8b43711d0,0x00d948b9d961785d,0x00cb167b6f279ead,0x00191de3a678e1c9,0x00d9dd9511095c2e,0x00f284324cd43067,0x00ed74fa535151dd)},-  {FIELD_LITERAL(0x00fb7feb08c27472,0x008a97b55f699c77,0x006d41820f923b83,0x006831432f0aa975,0x00a58ffb263b3955,0x004f13449a66db38,0x0026fccd22b6d583,0x00a803eb20eeb6c2)},-  {FIELD_LITERAL(0x007df6cbb926830b,0x00d336058ae37865,0x007af47dac696423,0x0048d3011ec64ac8,0x006b87666e40049f,0x0036a2e0e51303d7,0x00ba319bd79dbc55,0x003e2737ecc94f53)},+  {FIELD_LITERAL(0x007e32c049b5c477,0x009d2bfdbd9bcfd8,0x00636e93045938c6,0x007fde4af7687298,0x0046a5184fafa5d3,0x0079b1e7f13a359b,0x00875adf1fb927d6,0x00333e21c61bcad2)},+  {FIELD_LITERAL(0x00048014f73d8b8d,0x0075684aa0966388,0x0092be7df06dc47c,0x0097cebcd0f5568a,0x005a7004d9c4c6a9,0x00b0ecbb659924c7,0x00d90332dd492a7c,0x0057fc14df11493d)},   {FIELD_LITERAL(0x0008ed8ea0ad95be,0x0041d324b9709645,0x00e25412257a19b4,0x0058df9f3423d8d2,0x00a9ab20def71304,0x009ae0dbf8ac4a81,0x00c9565977e4392a,0x003c9269444baf55)},-  {FIELD_LITERAL(0x002d69008d9d8d26,0x00092f686d7030a8,0x001f19e95aa28fec,0x002150bab1261538,0x008c5a941210b26c,0x009330209036d1e6,0x0062e11ec8e58de7,0x0011c3d11bb9d27f)},-  {FIELD_LITERAL(0x008132ae5c5d8cd1,0x00121d68324a1d9f,0x00d6be9dafcb8c76,0x00684d9070edf745,0x00519fbc96d7448e,0x00388182fdc1f27e,0x000235baed41f158,0x00bf6cf6f1a1796a)},+  {FIELD_LITERAL(0x007df6cbb926830b,0x00d336058ae37865,0x007af47dac696423,0x0048d3011ec64ac8,0x006b87666e40049f,0x0036a2e0e51303d7,0x00ba319bd79dbc55,0x003e2737ecc94f53)},+  {FIELD_LITERAL(0x00d296ff726272d9,0x00f6d097928fcf57,0x00e0e616a55d7013,0x00deaf454ed9eac7,0x0073a56bedef4d92,0x006ccfdf6fc92e19,0x009d1ee1371a7218,0x00ee3c2ee4462d80)},   {FIELD_LITERAL(0x00437bce9bccdf9d,0x00e0c8e2f85dc0a3,0x00c91a7073995a19,0x00856ec9fe294559,0x009e4b33394b156e,0x00e245b0dc497e5c,0x006a54e687eeaeff,0x00f1cd1cd00fdb7c)},-  {FIELD_LITERAL(0x00d523b4b2eb7de6,0x00cf7b525f2c56f5,0x00b9217554f0d1b1,0x00bad2cbd5984a02,0x002b4af0fe2b21dd,0x002492603f310486,0x0073e7b3795b9d32,0x001e837c89b2bd25)},-  {FIELD_LITERAL(0x00ce382dc7993d92,0x00021153e938b4c8,0x00096f7567f48f51,0x0058f81ddfe4b0d5,0x00cc379a56b355c7,0x002c760770d3e819,0x00ee22d1d26e5a40,0x00de6d93d5b082d7)},+  {FIELD_LITERAL(0x008132ae5c5d8cd1,0x00121d68324a1d9f,0x00d6be9dafcb8c76,0x00684d9070edf745,0x00519fbc96d7448e,0x00388182fdc1f27e,0x000235baed41f158,0x00bf6cf6f1a1796a)},+  {FIELD_LITERAL(0x002adc4b4d148219,0x003084ada0d3a90a,0x0046de8aab0f2e4e,0x00452d342a67b5fd,0x00d4b50f01d4de21,0x00db6d9fc0cefb79,0x008c184c86a462cd,0x00e17c83764d42da)},   {FIELD_LITERAL(0x007b2743b9a1e01a,0x007847ffd42688c4,0x006c7844d610a316,0x00f0cb8b250aa4b0,0x00a19060143b3ae6,0x0014eb10b77cfd80,0x000170905729dd06,0x00063b5b9cd72477)},-  {FIELD_LITERAL(0x00f56e5bd3ad1fa9,0x00e7a09488031815,0x00f7fc3ae69d094a,0x00ddad7a7d45a9c2,0x00bc07fbf167a928,0x007a5d6137e0479f,0x00a0659eeab60a00,0x003e068b1342b4f9)},-  {FIELD_LITERAL(0x00ffc5c89d2b0cba,0x00d363d42e3e6fc3,0x0019a1a0118e2e8a,0x00f7baeff48882e1,0x001bd5af28c6b514,0x0055476ca2253cb2,0x00d8eb1977e2ddf3,0x00b173b1adb228a1)},+  {FIELD_LITERAL(0x00ce382dc7993d92,0x00021153e938b4c8,0x00096f7567f48f51,0x0058f81ddfe4b0d5,0x00cc379a56b355c7,0x002c760770d3e819,0x00ee22d1d26e5a40,0x00de6d93d5b082d7)},+  {FIELD_LITERAL(0x000a91a42c52e056,0x00185f6b77fce7ea,0x000803c51962f6b5,0x0022528582ba563d,0x0043f8040e9856d6,0x0085a29ec81fb860,0x005f9a611549f5ff,0x00c1f974ecbd4b06)},   {FIELD_LITERAL(0x005b64c6fd65ec97,0x00c1fdd7f877bc7f,0x000d9cc6c89f841c,0x005c97b7f1aff9ad,0x0075e3c61475d47e,0x001ecb1ba8153011,0x00fe7f1c8d71d40d,0x003fa9757a229832)},-  {FIELD_LITERAL(0x000d346622f528f8,0x001e1f7497a62227,0x00fff70d2f9af433,0x002812c6d079ea3c,0x006898af56b25d7f,0x00c17c44f1349645,0x00207172ea3eb539,0x000608e8bd6a263d)},-  {FIELD_LITERAL(0x002389319450f9ba,0x003677f31aa1250a,0x0092c3db642f38cb,0x00f8b64c0dfc9773,0x00cd49fe3505b795,0x0068105a4090a510,0x00df0ba2072a8bb6,0x00eb396143afd8be)},+  {FIELD_LITERAL(0x00ffc5c89d2b0cba,0x00d363d42e3e6fc3,0x0019a1a0118e2e8a,0x00f7baeff48882e1,0x001bd5af28c6b514,0x0055476ca2253cb2,0x00d8eb1977e2ddf3,0x00b173b1adb228a1)},+  {FIELD_LITERAL(0x00f2cb99dd0ad707,0x00e1e08b6859ddd8,0x000008f2d0650bcc,0x00d7ed392f8615c3,0x00976750a94da27f,0x003e83bb0ecb69ba,0x00df8e8d15c14ac6,0x00f9f7174295d9c2)},   {FIELD_LITERAL(0x00f11cc8e0e70bcb,0x00e5dc689974e7dd,0x0014e409f9ee5870,0x00826e6689acbd63,0x008a6f4e3d895d88,0x00b26a8da41fd4ad,0x000fb7723f83efd7,0x009c749db0a5f6c3)},-  {FIELD_LITERAL(0x005f2b1304db3200,0x0022507ff7459b86,0x000f4c1c92b4f0bb,0x00c8cb42c50e0eb9,0x004781d1038aad80,0x002dcf20aa2254af,0x00d9ecda851a93e2,0x0043f6b92eca6cb2)},-  {FIELD_LITERAL(0x0067f8f0c4fe26c9,0x0079c4a3cc8f67b9,0x0082b1e62f23550d,0x00f2d409caefd7f5,0x0080e67dcdb26e81,0x0087ae993ea1f98a,0x00aa108becf61d03,0x001acf11efb608a3)},+  {FIELD_LITERAL(0x002389319450f9ba,0x003677f31aa1250a,0x0092c3db642f38cb,0x00f8b64c0dfc9773,0x00cd49fe3505b795,0x0068105a4090a510,0x00df0ba2072a8bb6,0x00eb396143afd8be)},+  {FIELD_LITERAL(0x00a0d4ecfb24cdff,0x00ddaf8008ba6479,0x00f0b3e36d4b0f44,0x003734bd3af1f146,0x00b87e2efc75527e,0x00d230df55ddab50,0x002613257ae56c1d,0x00bc0946d135934d)},   {FIELD_LITERAL(0x00468711bd994651,0x0033108fa67561bf,0x0089d760192a54b4,0x00adc433de9f1871,0x000467d05f36e050,0x007847e0f0579f7f,0x00a2314ad320052d,0x00b3a93649f0b243)},-  {FIELD_LITERAL(0x007dda014454af26,0x000c49fa1b22df7c,0x005cd4d7e761dc2d,0x002af81a1a14b368,0x00a5e57b1cfd7ddf,0x00f90ab3e3a0f738,0x005cb83734d7bc0f,0x00f608c16abb405a)},-  {FIELD_LITERAL(0x00e828333c297f8b,0x009ef3cf8c3f7e1f,0x00ab45f8fff31cb9,0x00c8b4178cb0b013,0x00d0c50dd3260a3f,0x0097126ac257f5bc,0x0042376cc90c705a,0x001d96fdb4a1071e)},+  {FIELD_LITERAL(0x0067f8f0c4fe26c9,0x0079c4a3cc8f67b9,0x0082b1e62f23550d,0x00f2d409caefd7f5,0x0080e67dcdb26e81,0x0087ae993ea1f98a,0x00aa108becf61d03,0x001acf11efb608a3)},+  {FIELD_LITERAL(0x008225febbab50d9,0x00f3b605e4dd2083,0x00a32b28189e23d2,0x00d507e5e5eb4c97,0x005a1a84e302821f,0x0006f54c1c5f08c7,0x00a347c8cb2843f0,0x0009f73e9544bfa5)},   {FIELD_LITERAL(0x006c59c9ae744185,0x009fc32f1b4282cd,0x004d6348ca59b1ac,0x00105376881be067,0x00af4096013147dc,0x004abfb5a5cb3124,0x000d2a7f8626c354,0x009c6ed568e07431)},-  {FIELD_LITERAL(0x00abd2bb27611e57,0x00cf99bd1fbbd267,0x006f7ac78d478cc7,0x00dc9d340dd23fbb,0x00d3ddd520099c46,0x009836dbb6a03486,0x00f19de267c36883,0x0020885613349904)},-  {FIELD_LITERAL(0x00832d02369b482c,0x00cba52ff0d93450,0x003fa9c908d554db,0x008d1e357b54122f,0x00abd91c2dc950c6,0x007eff1df4c0ec69,0x003f6aeb13fb2d31,0x00002d6179fc5b2c)},+  {FIELD_LITERAL(0x00e828333c297f8b,0x009ef3cf8c3f7e1f,0x00ab45f8fff31cb9,0x00c8b4178cb0b013,0x00d0c50dd3260a3f,0x0097126ac257f5bc,0x0042376cc90c705a,0x001d96fdb4a1071e)},+  {FIELD_LITERAL(0x00542d44d89ee1a8,0x00306642e0442d98,0x0090853872b87338,0x002362cbf22dc044,0x002c222adff663b8,0x0067c924495fcb79,0x000e621d983c977c,0x00df77a9eccb66fb)},   {FIELD_LITERAL(0x002809e4bbf1814a,0x00b9e854f9fafb32,0x00d35e67c10f7a67,0x008f1bcb76e748cf,0x004224d9515687d2,0x005ba0b774e620c4,0x00b5e57db5d54119,0x00e15babe5683282)},-  {FIELD_LITERAL(0x00b9361257e36376,0x0049f348e3709d03,0x00dd0a597c455aa7,0x00078ce603320668,0x00635f64ae3195dc,0x00a4ed450b508288,0x0075b9adb5e1cc1d,0x00fca588167741f2)},-  {FIELD_LITERAL(0x00a9e7730a819691,0x00d9cc73c4992b70,0x00e299bde067de5a,0x008c314eb705192a,0x00e7226f17e8a3cc,0x0029dfd956e65a47,0x0053a8e839073b12,0x006f942b2ab1597e)},+  {FIELD_LITERAL(0x00832d02369b482c,0x00cba52ff0d93450,0x003fa9c908d554db,0x008d1e357b54122f,0x00abd91c2dc950c6,0x007eff1df4c0ec69,0x003f6aeb13fb2d31,0x00002d6179fc5b2c)},+  {FIELD_LITERAL(0x0046c9eda81c9c89,0x00b60cb71c8f62fc,0x0022f5a683baa558,0x00f87319fccdf997,0x009ca09b51ce6a22,0x005b12baf4af7d77,0x008a46524a1e33e2,0x00035a77e988be0d)},   {FIELD_LITERAL(0x00a7efe46a7dbe2f,0x002f66fd55014fe7,0x006a428afa1ff026,0x0056caaa9604ab72,0x0033f3bcd7fac8ae,0x00ccb1aa01c86764,0x00158d1edf13bf40,0x009848ee76fcf3b4)},-  {FIELD_LITERAL(0x00e3c287f132a1c6,0x006b0db804233a01,0x002a387902ad889b,0x00490b258b0f24d5,0x007f0e0745232a02,0x000c95c8c52d1dc4,0x0007fb060bcbc40d,0x002e50bf139dc67d)},-  {FIELD_LITERAL(0x0039343746531ebe,0x00c8509d835d429d,0x00e79eceff6b0018,0x004abfd31e8efce5,0x007bbfaaa1e20210,0x00e3be89c193e179,0x001c420f4c31d585,0x00f414a315bef5ae)},+  {FIELD_LITERAL(0x00a9e7730a819691,0x00d9cc73c4992b70,0x00e299bde067de5a,0x008c314eb705192a,0x00e7226f17e8a3cc,0x0029dfd956e65a47,0x0053a8e839073b12,0x006f942b2ab1597e)},+  {FIELD_LITERAL(0x001c3d780ecd5e39,0x0094f247fbdcc5fe,0x00d5c786fd527764,0x00b6f4da74f0db2a,0x0080f1f8badcd5fc,0x00f36a373ad2e23b,0x00f804f9f4343bf2,0x00d1af40ec623982)},   {FIELD_LITERAL(0x0082aeace5f1b144,0x00f68b3108cf4dd3,0x00634af01dde3020,0x000beab5df5c2355,0x00e8b790d1b49b0b,0x00e48d15854e36f4,0x0040ab2d95f3db9f,0x002711c4ed9e899a)},-  {FIELD_LITERAL(0x0083d695db66f207,0x002a2f8ada58aa77,0x002271eec16b4818,0x008443a70141f337,0x00d60ae50640352b,0x00816cee1385490c,0x006577b21e989cbc,0x00af2a0d2317b416)},-  {FIELD_LITERAL(0x0098cddc8b39549a,0x006da37e3b05d22c,0x00ce633cfd4eb3cb,0x00fda288ef526acd,0x0025338878c5d30a,0x00f34438c4e5a1b4,0x00584efea7c310f1,0x0041a551f1b660ad)},+  {FIELD_LITERAL(0x0039343746531ebe,0x00c8509d835d429d,0x00e79eceff6b0018,0x004abfd31e8efce5,0x007bbfaaa1e20210,0x00e3be89c193e179,0x001c420f4c31d585,0x00f414a315bef5ae)},+  {FIELD_LITERAL(0x007c296a24990df8,0x00d5d07525a75588,0x00dd8e113e94b7e7,0x007bbc58febe0cc8,0x0029f51af9bfcad3,0x007e9311ec7ab6f3,0x009a884de1676343,0x0050d5f2dce84be9)},   {FIELD_LITERAL(0x005fa020cca2450a,0x00491c29db6416d8,0x0037cefe3f9f9a85,0x003d405230647066,0x0049e835f0fdbe89,0x00feb78ac1a0815c,0x00828e4b32dc9724,0x00db84f2dc8d6fd4)},-  {FIELD_LITERAL(0x002808570429bc85,0x009d78dbec40c8ac,0x0052b4434bc3a7b4,0x00801b6419fe281c,0x008839a68764540a,0x0014ba034f958be4,0x00a31dbb6ec068f7,0x0077bd9bfe8c9cd9)},-  {FIELD_LITERAL(0x00a0b68ec1eb72d2,0x002c03235c0d45a0,0x00553627323fe8c5,0x006186e94b17af94,0x00a9906196e29f14,0x0025b3aee6567733,0x007e0dd840080517,0x0018eb5801a4ba93)},+  {FIELD_LITERAL(0x0098cddc8b39549a,0x006da37e3b05d22c,0x00ce633cfd4eb3cb,0x00fda288ef526acd,0x0025338878c5d30a,0x00f34438c4e5a1b4,0x00584efea7c310f1,0x0041a551f1b660ad)},+  {FIELD_LITERAL(0x00d7f7a8fbd6437a,0x0062872413bf3753,0x00ad4bbcb43c584b,0x007fe49be601d7e3,0x0077c659789babf4,0x00eb45fcb06a741b,0x005ce244913f9708,0x0088426401736326)},   {FIELD_LITERAL(0x007bf562ca768d7c,0x006c1f3a174e387c,0x00f024b447fee939,0x007e7af75f01143f,0x003adb70b4eed89d,0x00e43544021ad79a,0x0091f7f7042011f6,0x0093c1a1ee3a0ddc)},-  {FIELD_LITERAL(0x0028018fe84095bf,0x0091c0f9db41f3bd,0x0000445dfaca7dba,0x000603d307e6bdc6,0x00726c4c840ea4b0,0x009220d1c741716a,0x00d4918640a03006,0x0054caa25bda1d21)},-  {FIELD_LITERAL(0x003973d8938971d6,0x002aca26fa80c1f5,0x00108af1faa6b513,0x00daae275d7924e6,0x0053634ced721308,0x00d2355fe0bbd443,0x00357612b2d22095,0x00f9bb9dd4136cf3)},+  {FIELD_LITERAL(0x00a0b68ec1eb72d2,0x002c03235c0d45a0,0x00553627323fe8c5,0x006186e94b17af94,0x00a9906196e29f14,0x0025b3aee6567733,0x007e0dd840080517,0x0018eb5801a4ba93)},+  {FIELD_LITERAL(0x00d7fe7017bf6a40,0x006e3f0624be0c42,0x00ffbba205358245,0x00f9fc2cf8194239,0x008d93b37bf15b4e,0x006ddf2e38be8e95,0x002b6e79bf5fcff9,0x00ab355da425e2de)},   {FIELD_LITERAL(0x00938f97e20be973,0x0099141a36aaf306,0x0057b0ca29e545a1,0x0085db571f9fbc13,0x008b333c554b4693,0x0043ab6ef3e241cb,0x0054fb20aa1e5c70,0x00be0ff852760adf)},-  {FIELD_LITERAL(0x00d400ed30a1fc5a,0x00e424e0575e6307,0x0036e3986c07b2c6,0x0007960e4d145650,0x00a643ab823cdc93,0x0026e9ee292c7976,0x001f9d2555d3fdeb,0x0012c3fb833d437d)},-  {FIELD_LITERAL(0x0062dd0fb31be374,0x00fcc96b84c8e727,0x003f64f1375e6ae3,0x0057d9b6dd1af004,0x00d6a167b1103c7b,0x00dd28f3180fb537,0x004ff27ad7167128,0x008934c33461f2ac)},+  {FIELD_LITERAL(0x003973d8938971d6,0x002aca26fa80c1f5,0x00108af1faa6b513,0x00daae275d7924e6,0x0053634ced721308,0x00d2355fe0bbd443,0x00357612b2d22095,0x00f9bb9dd4136cf3)},+  {FIELD_LITERAL(0x002bff12cf5e03a5,0x001bdb1fa8a19cf8,0x00c91c6793f84d39,0x00f869f1b2eba9af,0x0059bc547dc3236b,0x00d91611d6d38689,0x00e062daaa2c0214,0x00ed3c047cc2bc82)},   {FIELD_LITERAL(0x000050d70c32b31a,0x001939d576d437b3,0x00d709e598bf9fe6,0x00a885b34bd2ee9e,0x00dd4b5c08ab1a50,0x0091bebd50b55639,0x00cf79ff64acdbc6,0x006067a39d826336)},-  {FIELD_LITERAL(0x009a4b8d486fffbc,0x00458102d00ef9b4,0x00f498293b3cfdf0,0x00ed2d7b960b1b92,0x00ce3cd6c68fc137,0x004b60f431eccf99,0x00081efbe9e7e2b8,0x00a36f0ae7981133)},-  {FIELD_LITERAL(0x0006918f5dfce6dc,0x00d4bf1c793c57fb,0x0069a3f649435364,0x00e89a50e5b0cd6e,0x00b9f6a237e973af,0x006d4ed8b104e41d,0x00498946a3924cd2,0x00c136ec5ac9d4f7)},+  {FIELD_LITERAL(0x0062dd0fb31be374,0x00fcc96b84c8e727,0x003f64f1375e6ae3,0x0057d9b6dd1af004,0x00d6a167b1103c7b,0x00dd28f3180fb537,0x004ff27ad7167128,0x008934c33461f2ac)},+  {FIELD_LITERAL(0x0065b472b7900043,0x00ba7efd2ff1064b,0x000b67d6c4c3020f,0x0012d28469f4e46d,0x0031c32939703ec7,0x00b49f0bce133066,0x00f7e10416181d47,0x005c90f51867eecc)},   {FIELD_LITERAL(0x0051207abd179101,0x00fc2a5c20d9c5da,0x00fb9d5f2701b6df,0x002dd040fdea82b8,0x00f163b0738442ff,0x00d9736bd68855b8,0x00e0d8e93005e61c,0x00df5a40b3988570)},-  {FIELD_LITERAL(0x00ee563d6f53acc9,0x00d465d2b5959acc,0x006575973bba26c8,0x00c9e4d84f81a1a3,0x00c3fbc4e8aa468a,0x0048149930eeaa11,0x008850a6f611000d,0x006709f6788337f9)},-  {FIELD_LITERAL(0x00b373076597455f,0x00e83f1af53ac0f5,0x0041f63c01dc6840,0x0097dea19b0c6f4b,0x007f9d63b4c1572c,0x00e692d492d0f5f0,0x00cbcb392e83b4ad,0x0069c0f39ed9b1a8)},+  {FIELD_LITERAL(0x0006918f5dfce6dc,0x00d4bf1c793c57fb,0x0069a3f649435364,0x00e89a50e5b0cd6e,0x00b9f6a237e973af,0x006d4ed8b104e41d,0x00498946a3924cd2,0x00c136ec5ac9d4f7)},+  {FIELD_LITERAL(0x0011a9c290ac5336,0x002b9a2d4a6a6533,0x009a8a68c445d937,0x00361b27b07e5e5c,0x003c043b1755b974,0x00b7eb66cf1155ee,0x0077af5909eefff2,0x0098f609877cc806)},   {FIELD_LITERAL(0x00ab13af436bf8f4,0x000bcf0a0dac8574,0x00d50c864f705045,0x00c40e611debc842,0x0085010489bd5caa,0x007c5050acec026f,0x00f67d943c8da6d1,0x00de1da0278074c6)},-  {FIELD_LITERAL(0x0079efcffed8f836,0x00604423802b5504,0x0070a6e294aab7dd,0x0020f75be15e7521,0x0062827c19bd5414,0x006738e425c48700,0x00dd37618fde0ffa,0x00bb2d65c01e1c3b)},-  {FIELD_LITERAL(0x00c903ee6d825540,0x00add6c4cf98473e,0x007636efed4227f1,0x00905124ae55e772,0x00e6b38fab12ed53,0x0045e132b863fe55,0x003974662edb366a,0x00b1787052be8208)},+  {FIELD_LITERAL(0x00b373076597455f,0x00e83f1af53ac0f5,0x0041f63c01dc6840,0x0097dea19b0c6f4b,0x007f9d63b4c1572c,0x00e692d492d0f5f0,0x00cbcb392e83b4ad,0x0069c0f39ed9b1a8)},+  {FIELD_LITERAL(0x00861030012707c9,0x009fbbdc7fd4aafb,0x008f591d6b554822,0x00df08a41ea18ade,0x009d7d83e642abea,0x0098c71bda3b78ff,0x0022c89e7021f005,0x0044d29a3fe1e3c4)},   {FIELD_LITERAL(0x00e748cd7b5c52f2,0x00ea9df883f89cc3,0x0018970df156b6c7,0x00c5a46c2a33a847,0x00cbde395e32aa09,0x0072474ebb423140,0x00fb00053086a23d,0x001dafcfe22d4e1f)},-  {FIELD_LITERAL(0x0059eb4ff288a383,0x00283876be3388ab,0x00bdd22974a2543b,0x0059eef0fe982d74,0x0097a5cf63dad778,0x004bc6002aebc99f,0x00c9a91d6118c690,0x0038364612a527ab)},-  {FIELD_LITERAL(0x00006e34a35d9fbc,0x00eee4e48b2f019a,0x006b344743003a5f,0x00541d514f04a7e3,0x00e81f9ee7647455,0x005e2b916c438f81,0x00116f8137b7eff0,0x009bd3decc7039d1)},+  {FIELD_LITERAL(0x00c903ee6d825540,0x00add6c4cf98473e,0x007636efed4227f1,0x00905124ae55e772,0x00e6b38fab12ed53,0x0045e132b863fe55,0x003974662edb366a,0x00b1787052be8208)},+  {FIELD_LITERAL(0x00a614b00d775c7c,0x00d7c78941cc7754,0x00422dd68b5dabc4,0x00a6110f0167d28b,0x00685a309c252886,0x00b439ffd5143660,0x003656e29ee7396f,0x00c7c9b9ed5ad854)},   {FIELD_LITERAL(0x0040f7e7c5b37bf2,0x0064e4dc81181bba,0x00a8767ae2a366b6,0x001496b4f90546f2,0x002a28493f860441,0x0021f59513049a3a,0x00852d369a8b7ee3,0x00dd2e7d8b7d30a9)},-  {FIELD_LITERAL(0x00fa2dd90bcbeef2,0x00507d774710de2a,0x00b585ad10e7e373,0x0041f487e4b4f921,0x00191c9d8212f81d,0x001bc55cbdd8d474,0x0017954bdba8827b,0x0004d6d3a991ca44)},-  {FIELD_LITERAL(0x00e38abece3c82ab,0x005a51f18a2c7a86,0x009dafa2e86d592e,0x00495a62eb688678,0x00b79df74c0eb212,0x0023e8cc78b75982,0x005998cb91075e13,0x00735aa9ba61bc76)},+  {FIELD_LITERAL(0x00006e34a35d9fbc,0x00eee4e48b2f019a,0x006b344743003a5f,0x00541d514f04a7e3,0x00e81f9ee7647455,0x005e2b916c438f81,0x00116f8137b7eff0,0x009bd3decc7039d1)},+  {FIELD_LITERAL(0x0005d226f434110d,0x00af8288b8ef21d5,0x004a7a52ef181c8c,0x00be0b781b4b06de,0x00e6e3627ded07e1,0x00e43aa342272b8b,0x00e86ab424577d84,0x00fb292c566e35bb)},   {FIELD_LITERAL(0x00334f5303ea1222,0x00dfb3dbeb0a5d3e,0x002940d9592335c1,0x00706a7a63e8938a,0x005a533558bc4caf,0x00558e33192022a9,0x00970d9faf74c133,0x002979fcb63493ca)},-  {FIELD_LITERAL(0x00260857d22419d7,0x005e0387d77651f0,0x008e0025ed2eb499,0x00c830b135804c2a,0x0037f43dbd3a77f6,0x008a4073d2f7379c,0x0072be0ce503ad58,0x00e6869d130c78be)},-  {FIELD_LITERAL(0x00bfc5fa1e4ea21f,0x00c21d7b6bb892e6,0x00cf043f3acf0291,0x00c13f2f849b3c90,0x00d1a97ebef10891,0x0061e130a445e7fe,0x0019513fdedbf22b,0x001d60c813bff841)},+  {FIELD_LITERAL(0x00e38abece3c82ab,0x005a51f18a2c7a86,0x009dafa2e86d592e,0x00495a62eb688678,0x00b79df74c0eb212,0x0023e8cc78b75982,0x005998cb91075e13,0x00735aa9ba61bc76)},+  {FIELD_LITERAL(0x00d9f7a82ddbe628,0x00a1fc782889ae0f,0x0071ffda12d14b66,0x0037cf4eca7fb3d5,0x00c80bc242c58808,0x0075bf8c2d08c863,0x008d41f31afc52a7,0x00197962ecf38741)},   {FIELD_LITERAL(0x006e9f475cccf2ee,0x00454b9cd506430c,0x00224a4fb79ee479,0x0062e3347ef0b5e2,0x0034fd2a3512232a,0x00b8b3cb0f457046,0x00eb20165daa38ec,0x00128eebc2d9c0f7)},-  {FIELD_LITERAL(0x00e6a9e38030fdec,0x001c23597bc14288,0x0097156a46356df1,0x00642048f0daca6a,0x003970a6e7955fd4,0x00a511e335e3cfc6,0x0054865756c85e31,0x00465f1ab66a6190)},-  {FIELD_LITERAL(0x003e4964fa8a8fc8,0x00f6a1cdbcf41689,0x00943cb18fe7fda7,0x00606dafbf34440a,0x005d37a86399c789,0x00e79a2a69417403,0x00fe34f7e68b8866,0x0011f448ed2df10e)},+  {FIELD_LITERAL(0x00bfc5fa1e4ea21f,0x00c21d7b6bb892e6,0x00cf043f3acf0291,0x00c13f2f849b3c90,0x00d1a97ebef10891,0x0061e130a445e7fe,0x0019513fdedbf22b,0x001d60c813bff841)},+  {FIELD_LITERAL(0x0019561c7fcf0213,0x00e3dca6843ebd77,0x0068ea95b9ca920e,0x009bdfb70f253595,0x00c68f59186aa02a,0x005aee1cca1c3039,0x00ab79a8a937a1ce,0x00b9a0e549959e6f)},   {FIELD_LITERAL(0x00c79e0b6d97dfbd,0x00917c71fd2bc6e8,0x00db7529ccfb63d8,0x00be5be957f17866,0x00a9e11fdc2cdac1,0x007b91a8e1f44443,0x00a3065e4057d80f,0x004825f5b8d5f6d4)},-  {FIELD_LITERAL(0x000e0a81033e033b,0x00aec986ee821eab,0x00d1a4a48379273c,0x00609b79a9e06304,0x00e9618b4fe8f307,0x006ffdfa50b50969,0x009530224887ac0c,0x0020e7b36f0cef97)},-  {FIELD_LITERAL(0x00fd579ffb691713,0x00b76af4f81c412d,0x00f239de96110f82,0x00e965fb437f0306,0x00ca7e9436900921,0x00e487f1325fa24a,0x00633907de476380,0x00721c62ac5b8ea0)},+  {FIELD_LITERAL(0x003e4964fa8a8fc8,0x00f6a1cdbcf41689,0x00943cb18fe7fda7,0x00606dafbf34440a,0x005d37a86399c789,0x00e79a2a69417403,0x00fe34f7e68b8866,0x0011f448ed2df10e)},+  {FIELD_LITERAL(0x00f1f57efcc1fcc4,0x00513679117de154,0x002e5b5b7c86d8c3,0x009f6486561f9cfb,0x00169e74b0170cf7,0x00900205af4af696,0x006acfddb77853f3,0x00df184c90f31068)},   {FIELD_LITERAL(0x00b37396c3320791,0x00fc7b67175c5783,0x00c36d2cd73ecc38,0x0080ebcc0b328fc5,0x0043a5b22b35d35d,0x00466c9f1713c9da,0x0026ad346dcaa8da,0x007c684e701183a6)},-  {FIELD_LITERAL(0x003f2ab1abd14b06,0x00b129a8e8e37230,0x0048bc5b083d5c64,0x0002606c12933a98,0x00cf8051ceec1a73,0x00a755a8836c3ce6,0x002dabaa90ca4cb9,0x00b6e5525ddfc0f2)},-  {FIELD_LITERAL(0x00c4a1fb48635413,0x00b5dd54423ad59f,0x009ff5d53fd24a88,0x003c98d267fc06a7,0x002db7cb20013641,0x00bd1d6716e191f2,0x006dbc8b29094241,0x0044bbf233dafa2c)},+  {FIELD_LITERAL(0x00fd579ffb691713,0x00b76af4f81c412d,0x00f239de96110f82,0x00e965fb437f0306,0x00ca7e9436900921,0x00e487f1325fa24a,0x00633907de476380,0x00721c62ac5b8ea0)},+  {FIELD_LITERAL(0x00c0d54e542eb4f9,0x004ed657171c8dcf,0x00b743a4f7c2a39b,0x00fd9f93ed6cc567,0x00307fae3113e58b,0x0058aa577c93c319,0x00d254556f35b346,0x00491aada2203f0d)},   {FIELD_LITERAL(0x00dff3103786ff34,0x000144553b1f20c3,0x0095613baeb930e4,0x00098058275ea5d4,0x007cd1402b046756,0x0074d74e4d58aee3,0x005f93fc343ff69b,0x00873df17296b3b0)},-  {FIELD_LITERAL(0x00aa7c72be0ace19,0x004095d22fc37e4d,0x00a7d85f9e3b7c61,0x00ff21d344c9553c,0x00d105d6268e8b86,0x000616d733758845,0x003ecb4ba7210610,0x006a75e7dddc03b7)},-  {FIELD_LITERAL(0x007860d99db787cf,0x00fda8983018f4a8,0x008c8866bac4743c,0x00ef471f84c82a3f,0x00abea5976d3b8e7,0x00714882896cd015,0x00b49fae584ddac5,0x008e33a1a0b69c81)},+  {FIELD_LITERAL(0x00c4a1fb48635413,0x00b5dd54423ad59f,0x009ff5d53fd24a88,0x003c98d267fc06a7,0x002db7cb20013641,0x00bd1d6716e191f2,0x006dbc8b29094241,0x0044bbf233dafa2c)},+  {FIELD_LITERAL(0x0055838d41f531e6,0x00bf6a2dd03c81b2,0x005827a061c4839e,0x0000de2cbb36aac3,0x002efa29d9717478,0x00f9e928cc8a77ba,0x00c134b458def9ef,0x00958a182223fc48)},   {FIELD_LITERAL(0x000a9ee23c06881f,0x002c727d3d871945,0x00f47d971512d24a,0x00671e816f9ef31a,0x00883af2cfaad673,0x00601f98583d6c9a,0x00b435f5adc79655,0x00ad87b71c04bff2)},-  {FIELD_LITERAL(0x0084911d36175613,0x00dbaa24427629dd,0x009b6f30b1554fc7,0x0026da093cf7ea9e,0x00eac4cfb8218c7c,0x00c4bde074231490,0x0089e5b5afb62587,0x0067fcb73adfdbcc)},-  {FIELD_LITERAL(0x00eebfd4e2312cc3,0x00474b2564e4fc8c,0x003303ef14b1da9b,0x003c93e0e66beb1d,0x0013619b0566925a,0x008817c24d901bf3,0x00b62bd8898d218b,0x0075a7716f1e88a2)},+  {FIELD_LITERAL(0x007860d99db787cf,0x00fda8983018f4a8,0x008c8866bac4743c,0x00ef471f84c82a3f,0x00abea5976d3b8e7,0x00714882896cd015,0x00b49fae584ddac5,0x008e33a1a0b69c81)},+  {FIELD_LITERAL(0x007b6ee2c9e8a9ec,0x002455dbbd89d622,0x006490cf4eaab038,0x00d925f6c3081561,0x00153b3047de7382,0x003b421f8bdceb6f,0x00761a4a5049da78,0x00980348c5202433)},   {FIELD_LITERAL(0x007f8a43da97dd5c,0x00058539c800fc7b,0x0040f3cf5a28414a,0x00d68dd0d95283d6,0x004adce9da90146e,0x00befa41c7d4f908,0x007603bc2e3c3060,0x00bdf360ab3545db)},-  {FIELD_LITERAL(0x00f6de725e1976f0,0x00d96f80a02fda8a,0x00b25412a0e629fa,0x00c540e7e78fdb62,0x004ad02fb7336d3a,0x004922ae1bea5a3a,0x0026147d42d4bfeb,0x00d379a5bc4b94bc)},-  {FIELD_LITERAL(0x00c338b915d8fef0,0x00a893292045c39a,0x0028ab4f2eba6887,0x0060743cb519fd61,0x0006213964093ac0,0x007c0b7a43f6266d,0x008e3557c4fa5bda,0x002da976de7b8d9d)},+  {FIELD_LITERAL(0x00eebfd4e2312cc3,0x00474b2564e4fc8c,0x003303ef14b1da9b,0x003c93e0e66beb1d,0x0013619b0566925a,0x008817c24d901bf3,0x00b62bd8898d218b,0x0075a7716f1e88a2)},+  {FIELD_LITERAL(0x0009218da1e6890f,0x0026907f5fd02575,0x004dabed5f19d605,0x003abf181870249d,0x00b52fd048cc92c4,0x00b6dd51e415a5c5,0x00d9eb82bd2b4014,0x002c865a43b46b43)},   {FIELD_LITERAL(0x0070047189452f4c,0x00f7ad12e1ce78d5,0x00af1ba51ec44a8b,0x005f39f63e667cd6,0x00058eac4648425e,0x00d7fdab42bea03b,0x0028576a5688de15,0x00af973209e77c10)},-  {FIELD_LITERAL(0x00b78d6075749232,0x0001dc47a33b2cdc,0x0018c7b2e91b24f1,0x00b5bdc68f9876bd,0x0013f489ccba2b44,0x003b8846066128de,0x003d6252c8884dcf,0x00e3ae84b9908209)},-  {FIELD_LITERAL(0x00aa2261022d883f,0x00ebcca4548010ac,0x002528512e28a437,0x0070ca7676b66082,0x0084bda170f7c6d3,0x00581b4747c9b8bb,0x005c96a01061c7e2,0x00fb7c4a362b5273)},+  {FIELD_LITERAL(0x00c338b915d8fef0,0x00a893292045c39a,0x0028ab4f2eba6887,0x0060743cb519fd61,0x0006213964093ac0,0x007c0b7a43f6266d,0x008e3557c4fa5bda,0x002da976de7b8d9d)},+  {FIELD_LITERAL(0x0048729f8a8b6dcd,0x00fe23b85cc4d323,0x00e7384d16e4db0e,0x004a423970678942,0x00ec0b763345d4ba,0x00c477b9f99ed721,0x00c29dad3777b230,0x001c517b466f7df6)},   {FIELD_LITERAL(0x006366c380f7b574,0x001c7d1f09ff0438,0x003e20a7301f5b22,0x00d3efb1916d28f6,0x0049f4f81060ce83,0x00c69d91ea43ced1,0x002b6f3e5cd269ed,0x005b0fb22ce9ec65)},-  {FIELD_LITERAL(0x003cffdf14aed2fd,0x009f0d77d7c5b2d9,0x004812ec41321d9f,0x008a1448bddf0916,0x008fef86030175df,0x00e3d703200a76c7,0x00d1babb470b2094,0x009f3a43b0e5828c)},-  {FIELD_LITERAL(0x00a94700032a093f,0x0076e96c225216e7,0x00a63a4316e45f91,0x007d8bbb4645d3b2,0x00340a6ff22793eb,0x006f935d4572aeb7,0x00b1fb69f00afa28,0x009e8f3423161ed3)},+  {FIELD_LITERAL(0x00aa2261022d883f,0x00ebcca4548010ac,0x002528512e28a437,0x0070ca7676b66082,0x0084bda170f7c6d3,0x00581b4747c9b8bb,0x005c96a01061c7e2,0x00fb7c4a362b5273)},+  {FIELD_LITERAL(0x00c30020eb512d02,0x0060f288283a4d26,0x00b7ed13becde260,0x0075ebb74220f6e9,0x00701079fcfe8a1f,0x001c28fcdff58938,0x002e4544b8f4df6b,0x0060c5bc4f1a7d73)},   {FIELD_LITERAL(0x00ae307cf069f701,0x005859f222dd618b,0x00212d6c46ec0b0d,0x00a0fe4642afb62d,0x00420d8e4a0a8903,0x00a80ff639bdf7b0,0x0019bee1490b5d8e,0x007439e4b9c27a86)},-  {FIELD_LITERAL(0x00610b6394a312e8,0x005aaa19d96160f5,0x008190e286138c4a,0x006538796a5cd53b,0x00fe28804432a97c,0x007315e011f55112,0x000bd4157d5acb9d,0x00d1b95469350336)},-  {FIELD_LITERAL(0x0060db815bc4786c,0x006fab25beedc434,0x00c610d06084797c,0x000c48f08537bec0,0x0031aba51c5b93da,0x007968fa6e01f347,0x0030070da52840c6,0x00c043c225a4837f)},+  {FIELD_LITERAL(0x00a94700032a093f,0x0076e96c225216e7,0x00a63a4316e45f91,0x007d8bbb4645d3b2,0x00340a6ff22793eb,0x006f935d4572aeb7,0x00b1fb69f00afa28,0x009e8f3423161ed3)},+  {FIELD_LITERAL(0x009ef49c6b5ced17,0x00a555e6269e9f0a,0x007e6f1d79ec73b5,0x009ac78695a32ac4,0x0001d77fbbcd5682,0x008cea1fee0aaeed,0x00f42bea82a53462,0x002e46ab96cafcc9)},   {FIELD_LITERAL(0x0051cfcc5885377a,0x00dce566cb1803ca,0x00430c7643f2c7d4,0x00dce1a1337bdcc0,0x0010d5bd7283c128,0x003b1b547f9b46fe,0x000f245e37e770ab,0x007b72511f022b37)},-  {FIELD_LITERAL(0x00e4302ff9b6116c,0x0092314b81d5f02a,0x000d31425f30702f,0x004946262e04213c,0x007ead9d19b6f9ed,0x001080a31ce8989f,0x001b632f36672a74,0x00a03933d9645a83)},-  {FIELD_LITERAL(0x004a2902926f8d3f,0x00ad79b42637ab75,0x0088f60b90f2d4e8,0x0030f54ef0e398c4,0x00021dc9bf99681e,0x007ebf66fde74ee3,0x004ade654386e9a4,0x00e7485066be4c27)},+  {FIELD_LITERAL(0x0060db815bc4786c,0x006fab25beedc434,0x00c610d06084797c,0x000c48f08537bec0,0x0031aba51c5b93da,0x007968fa6e01f347,0x0030070da52840c6,0x00c043c225a4837f)},+  {FIELD_LITERAL(0x001bcfd00649ee93,0x006dceb47e2a0fd5,0x00f2cebda0cf8fd0,0x00b6b9d9d1fbdec3,0x00815262e6490611,0x00ef7f5ce3176760,0x00e49cd0c998d58b,0x005fc6cc269ba57c)},   {FIELD_LITERAL(0x008940211aa0d633,0x00addae28136571d,0x00d68fdbba20d673,0x003bc6129bc9e21a,0x000346cf184ebe9a,0x0068774d741ebc7f,0x0019d5e9e6966557,0x0003cbd7f981b651)},-  {FIELD_LITERAL(0x00bba0ed9c67c41f,0x00b30c8e225ba195,0x008bb5762a5cef18,0x00e0df31b06fb7cc,0x0018b912141991d5,0x00f6ed54e093eac2,0x0009e288264dbbb3,0x00feb663299b89ef)}+  {FIELD_LITERAL(0x004a2902926f8d3f,0x00ad79b42637ab75,0x0088f60b90f2d4e8,0x0030f54ef0e398c4,0x00021dc9bf99681e,0x007ebf66fde74ee3,0x004ade654386e9a4,0x00e7485066be4c27)},+  {FIELD_LITERAL(0x00445f1263983be0,0x004cf371dda45e6a,0x00744a89d5a310e7,0x001f20ce4f904833,0x00e746edebe66e29,0x000912ab1f6c153d,0x00f61d77d9b2444c,0x0001499cd6647610)} }; const gf API_NS(precomputed_wnaf_as_fe)[96] VECTOR_ALIGNED = {-  {FIELD_LITERAL(0x00cfc32590115acd,0x0079f0e2a5c7af1b,0x00dd94605b8d7332,0x0097dd6c75f5f3f3,0x00d9c59e36156de9,0x00edfbfd6cde47d7,0x0095b97c9f67c39a,0x007d7b90f587debc)},-  {FIELD_LITERAL(0x00cfc32590115acd,0x0079f0e2a5c7af1b,0x00dd94605b8d7332,0x0017dd6c75f5f3f3,0x00d9c59e36156de8,0x00edfbfd6cde47d7,0x0095b97c9f67c39a,0x00fd7b90f587debc)},-  {FIELD_LITERAL(0x001071dd4d8ae672,0x004f14ebe5f4f174,0x00e0987625c34c73,0x0092d00712c6f8c1,0x009ef424965e980b,0x00a8e0cf9369764b,0x000aa81907b4d207,0x00d5002c74d37924)},-  {FIELD_LITERAL(0x00f3c4efe62b8b17,0x001e6acc1b6add7b,0x003367ef45836df5,0x000efc2d87a6ba53,0x00405a96933964ca,0x00572c2ae16357c6,0x00a9dc34ba6a7946,0x00151831e32ad161)},-  {FIELD_LITERAL(0x00315f0372d1774a,0x007de9ed2960e79d,0x008b3d7c4c198add,0x00a5e6a45fa57892,0x00f32201aa80115a,0x007fb9386a433a1a,0x00abf6960b291ee6,0x002d8069294ebc2a)},-  {FIELD_LITERAL(0x00fa5e878ae22827,0x00d33c7bb3963bd0,0x0053401a101efac6,0x0063df0bcbce59a5,0x007bca269c8b584b,0x00611a8a9978842c,0x00bb96e8da12b8a8,0x00e17844d01d394d)},-  {FIELD_LITERAL(0x00c107c50e9b4d0d,0x00f6b65a5fada2f2,0x000bb67e79353fae,0x0018853f610ed92d,0x008c51f4d36d6915,0x00e3e9c096dd1c12,0x009d6b9ea6cde415,0x00304864dd66f4c6)},-  {FIELD_LITERAL(0x00f3123b214085fb,0x00d005bafffb8f53,0x00d1606987dfe6ea,0x00e825edf73b018d,0x0082aa733829a933,0x00c857d8d7830d76,0x00ebdb8d2cbbe7e6,0x0063de0e9930722e)},-  {FIELD_LITERAL(0x004ffebce35619ab,0x00d281a1543365c5,0x00ad17eeb3d098b8,0x008653b06bb7806d,0x0040026e64a28b62,0x00d9e06d52ea19df,0x008e7c684856876a,0x003ebbc191443f3b)},-  {FIELD_LITERAL(0x00c0a062813b8884,0x0054d18cc36e636b,0x00e4493fcadba51a,0x005cda5b6577c9cf,0x00cc165615c315cf,0x001bbd5e155f17bb,0x004dee92a4f18e47,0x003e95412929bfb8)},-  {FIELD_LITERAL(0x0015326f3e1f5fb6,0x0076886ca4eb6041,0x00fb34645ee36c23,0x006042a4cb8f7bb2,0x00b43e736403dd2f,0x00a8986566e7c60c,0x0010ea48904bf6d1,0x008b5ae8c5ddafbe)},-  {FIELD_LITERAL(0x003a9f4a12faee9a,0x00e6ba523a29af6b,0x001dde79a8ef06ef,0x0033ed4361647314,0x00b0556ae76eb1c9,0x00e8b892762bd092,0x004709c83705e374,0x0077382d86f79b47)},-  {FIELD_LITERAL(0x006638c5cee4113d,0x005c100c7276ed52,0x00d10562e281768d,0x0008e851e1eb2ed9,0x00d7cc086a7af373,0x00993ed528eb7942,0x0051677625b7df14,0x0029fbbcf6aaa3f7)},-  {FIELD_LITERAL(0x001081503e396419,0x007a2c7aa8870415,0x00d372a4baf3490a,0x00b18821a1e18013,0x00b83fa876c54211,0x00e4bcf47a2ae1e9,0x0069a384ba9bf3c3,0x00b784d44ee9d468)},-  {FIELD_LITERAL(0x00b4e3ad7c2ea1be,0x009962715cf7008a,0x00fbc6fdcc089d5e,0x001e29847c349313,0x00c1145569b3874d,0x0094f50069a1499b,0x004cec2bb8f423c8,0x0077eb0034c34627)},-  {FIELD_LITERAL(0x008f00d279b21a44,0x00a5c81149c8116a,0x00cc8be3da721e9f,0x001935a34e6770b9,0x00e315426d5db99d,0x00cf6a842aff01bf,0x00e3cc9d5016ed3a,0x00ae78776098742d)},-  {FIELD_LITERAL(0x0068db473197248f,0x0089874a12ff90c2,0x00420b4763f5428c,0x00d668b71fb38392,0x0022279b6d3c3687,0x003a5801405cf566,0x00127b8ea4b4fd44,0x00ce6a975208fb79)},-  {FIELD_LITERAL(0x00797ca039d44238,0x0063cae935b6ef5e,0x006a938e072ff87c,0x006a3870309cdca0,0x0003800945fa3ddc,0x0032274c0728b5ad,0x0053a51e9217da91,0x00162b41712b79db)},-  {FIELD_LITERAL(0x000911f06768bdc6,0x00bd27650f82c5b0,0x007b948017bcb94a,0x0095de039572c65e,0x0053743dabe00d25,0x0092b1d5888cd8cd,0x0065c6496b33c0d0,0x007a3f55d5bfb370)},-  {FIELD_LITERAL(0x003f31eebfa20d27,0x00b1c0c84d6c2849,0x00dbefe8d1e53924,0x00472400b407ebc2,0x00c584bf62a91498,0x00c1f095f2010650,0x007e3b1b2c9ba41e,0x003189f894ed89dc)},-  {FIELD_LITERAL(0x004d9eefe5de7ab7,0x003e35169bdbd884,0x0079625f58822d97,0x0043f4f607137c15,0x0029efd80717d455,0x0055b37a66623198,0x00153cecd460c01e,0x000464f30e396a2d)},-  {FIELD_LITERAL(0x0057b28375dc4b6e,0x00771e6557974d80,0x00fa6792bc187316,0x000d7fed0f9f92d7,0x00e821281efdb64b,0x00a12bf7b4dc5064,0x00464f56bfa9bb8d,0x00526fa933114e0b)},-  {FIELD_LITERAL(0x00bcf86d6aaed0f2,0x00b95ff679e8a71f,0x00c11d7bd57f8c87,0x00cb3362ed671b05,0x0068bb14b2ce4c10,0x00505313699af32f,0x005376e4cec89e51,0x00179b292d918f75)},-  {FIELD_LITERAL(0x00246e4ca8018aa1,0x005e55abb4eaca63,0x0050b6ce5fe6aa8b,0x008979edb01ee510,0x002e152c38461080,0x00550a03a7f073ea,0x0018d841eb811e13,0x00c39e3e1ea88479)},-  {FIELD_LITERAL(0x007f1264364f8cc7,0x000315388ba2d9ad,0x007562aa0a0d3396,0x0069318d20cfe53a,0x000acdcd1868b277,0x008e8d738518c6b8,0x006faf89fda8f887,0x00347e30277c4e4d)},-  {FIELD_LITERAL(0x0062c03567cddf30,0x0032ee53437ac23b,0x00e8a6fbf62d80e2,0x002de89967f7d7fd,0x0005fedae4d7c736,0x0022d685f264ae39,0x0028936d3fba7df5,0x00acb4383b936fcc)},-  {FIELD_LITERAL(0x00afee55215c8c25,0x00c57a8713769fcb,0x000df59aca05928e,0x00aead2ce1a57830,0x00d453e3719735cd,0x004f1cdc24b3ec7e,0x000e2a69482a51da,0x00151ba7f6834b1f)},-  {FIELD_LITERAL(0x003eaec329954173,0x00fec61feee76bb2,0x009b544347f7f444,0x004c4f7dfdb8cebd,0x0039d610da25dbfb,0x000f513ccef26480,0x00af4ddd8b8d2732,0x00093756dd2be04b)},-  {FIELD_LITERAL(0x006df537f064f2de,0x0007f0808cbfedb9,0x00792c87b64aa829,0x00fd42b4ce848ad1,0x004d9b9c66c5bd43,0x00df8fbdd58c4ed6,0x00cbe5355fc7f34c,0x00abe6eb22995e4d)},-  {FIELD_LITERAL(0x00ef8a330d9484e0,0x0044944dece8fbcc,0x0016b6e52d9d2586,0x00610b0b72d2c7b3,0x00766d88f8990f61,0x00ea7bc69494eefe,0x0050c07989360110,0x00db9fc3bfd96ee7)},-  {FIELD_LITERAL(0x0069991db096c6b8,0x0008ebceed962ba0,0x00ef0053e2f37ae3,0x009917f3c8c9cb68,0x000e0b52fef39f4e,0x00ea378bf7b8f008,0x009ae2a16388995b,0x007ec77e628ee921)},-  {FIELD_LITERAL(0x0062284cece6ad83,0x00e18536b7278c56,0x0005ab4b910698c5,0x009910472a4fd019,0x008ab4e2c6d75150,0x00fbd9d538d59094,0x0086482b65914fd9,0x00ced958acabfefd)},-  {FIELD_LITERAL(0x00c6cb4ee3a8dac4,0x0010cf7120de0b91,0x001ab166385e9e67,0x007f2a8eca89b19c,0x008ae3d846b943da,0x0022c7631b161ed6,0x005e5d402e327b23,0x00d0518c1aeb64cd)},-  {FIELD_LITERAL(0x000d45c95be55ebb,0x005f3dd26b911e70,0x00755171065eb066,0x00110b2864e644c9,0x00718a31c2d84e02,0x0059a255fc4d65d8,0x0026337c97b14eba,0x0061e127f33d128b)},-  {FIELD_LITERAL(0x006ee9a82004b322,0x003eff4833aac2f9,0x00bb62f8a13b9833,0x008f9deff439b18f,0x00bc30790842de17,0x000bfe23b4868215,0x00addb504d09d19a,0x002e121c04a5bd41)},-  {FIELD_LITERAL(0x004126ac2e668677,0x0046c12e8a5dbed7,0x0078e3a69c049c9a,0x0035d20dffeb5878,0x000a263e2f4cbcdc,0x00090a6bd7e724f5,0x00b33f6e0b6366f9,0x00175e7759f40060)},-  {FIELD_LITERAL(0x0083b4b835838c18,0x00ac69ddefc68cb4,0x00749b220f1ba281,0x004052a50d7a193d,0x007138ee3a4e5e56,0x003099ccfedc8067,0x006e811c0e9aaed9,0x00bead0cc8101227)},-  {FIELD_LITERAL(0x00cd3889dfcd0517,0x001bf78dcd1f43de,0x000898cbb491727a,0x00440c964893d55d,0x0075e0b9391ea8f2,0x00ec9732687fc960,0x008ca65c62f86bcf,0x00fc9b9aed6debcb)},-  {FIELD_LITERAL(0x00f8381236cfa255,0x00f5999b0d8c8fe3,0x000918786a1dff4e,0x00a2fa46132db8c1,0x00eb0a0e8379a878,0x003802d2e990566a,0x00b6c65d27147f1f,0x00ddbb45f6bd3e66)},-  {FIELD_LITERAL(0x000f68a71ee1c67a,0x00e96102429b052c,0x0017776482925329,0x00ca322a71577df6,0x004325b8a79280b5,0x00c322234d786f77,0x00e9258fe7816ab4,0x006aa915d16d5532)},-  {FIELD_LITERAL(0x00cde18980fd9d30,0x00d1a82889350971,0x0040d36b7eb0fbc8,0x003cc6e695329dd0,0x00e24b3318e1d88e,0x00e212a22459111d,0x00879f754eaab372,0x00f9801f5489c9a4)},-  {FIELD_LITERAL(0x007354e942e00768,0x004c7668d3208ac0,0x0015712e1b92023f,0x00b018106b3a760b,0x00d4751647fa130b,0x00da3f7276d78b5a,0x00dc6c71672bb3b3,0x0008a6ecb3540963)},-  {FIELD_LITERAL(0x00e13a624c26a6f1,0x00e161c0e3c0e7d2,0x00ba563c13d354eb,0x00f7e67a8d51498c,0x0088c48bf9742e97,0x00edaca155c6abcb,0x00bb24561c4448b5,0x00d045b2c38b42f1)},-  {FIELD_LITERAL(0x0093d57b9871b4c4,0x0085e6b5532e7970,0x0012fdda50bdb89e,0x0025f590d6c39b47,0x00ef9d53a39585e6,0x00cf0a88a575110b,0x00fd53552894850f,0x00bef47029c5a860)},-  {FIELD_LITERAL(0x00bd40f701996dd3,0x00cce747044b6173,0x0028a6b9ffb55eb3,0x0009fea794bd40e3,0x0038b30e26ed0198,0x005434c968b4cf52,0x00814878df362d47,0x0060ab54842b207a)},-  {FIELD_LITERAL(0x00bd19d97479e8ae,0x00f722fb96aff3e9,0x004ae4a83cc75c02,0x0033bb6827a30094,0x00d0ec294a83cb5a,0x007c9ad150cfeefa,0x0033cbbd6b336c57,0x009f0b2fd7ef1d8f)},-  {FIELD_LITERAL(0x00246036b708c7d9,0x000574c8b9127116,0x00ecd349a550414d,0x003c900c0186da47,0x007c82512cac2d00,0x001399e41f99830b,0x00a414712d16fdfb,0x0028822961a9b698)},-  {FIELD_LITERAL(0x00576abc9c32ae74,0x0052e8eedb433484,0x009a0b95b52551ff,0x00e4e5a4d5691aff,0x00bc01db07dccd79,0x00996692751e0d3c,0x003acf0cd9be9606,0x003f06d2f83095a8)},-  {FIELD_LITERAL(0x0028c4051a1ff7bb,0x0040ba689904a0ad,0x009e4b0a5acec321,0x00bc6d2b3c46aaeb,0x00f2caae4ef88adb,0x00ff6677bf11a28e,0x0092191cbfbb7484,0x00dae55afb78a291)},-  {FIELD_LITERAL(0x00c95aa397ea26bc,0x007372e21066c24c,0x00d1f1e17008ce70,0x00277c5b46d24ff5,0x00d0a187e51cc6f8,0x00e58d524dca3f92,0x000d1a618c916355,0x00e5b4a71cfce6eb)},-  {FIELD_LITERAL(0x00c40cbcbd853cbd,0x00523f5879bd473a,0x00fc476ce8a57ceb,0x009e5cb521a8fc43,0x0015c157448e29cc,0x0041f2065e0e673d,0x00b9227183e9ca04,0x000eadc022da2a1a)},-  {FIELD_LITERAL(0x00d6313aad8c08f2,0x008fbb11d8a39cbf,0x00bf09c856cfea1d,0x00cc7448724a5516,0x00eb6e4d59ecdeb7,0x005eda293019421c,0x00a0853a9e457996,0x00e2a1515c045530)},-  {FIELD_LITERAL(0x009cc09c03622bf9,0x0018ec007f1fb5bc,0x009f39168f0d29de,0x005a83280f20e76e,0x000dbf95aaf9af43,0x004f9bd6f102397b,0x00e154febb2e86e9,0x0032ea079c3d6c54)},-  {FIELD_LITERAL(0x00fab169ca1c41ce,0x00f1bc0ce1d78d41,0x002fa4e361cc67be,0x009053af427e0267,0x0032387ad15144f5,0x00b00ae64f9e66e4,0x006f6617ef82b37a,0x00d8c1db3c95b59e)},-  {FIELD_LITERAL(0x0035175500c7799c,0x00a167c5ca225e38,0x00854efcf271c80b,0x001b76bf0a2fcd01,0x0095c90610cf4ccd,0x0064190fc6a738a8,0x0079dce31456ebff,0x00742f0847dc1855)},-  {FIELD_LITERAL(0x00f8f4bbbe10d3b9,0x00105a4fd7fe5ef6,0x0040f473c119b520,0x0075981f4cbad167,0x00e6e94e0d05858a,0x00287e587009323c,0x00797d31a81a36e6,0x0033eef622def25c)},-  {FIELD_LITERAL(0x003077e1410a5ba5,0x00b14158718390d3,0x006f256df630d95f,0x0021d4d1b388a47b,0x008e29fce3c3ea50,0x002616d810e8828f,0x0076b1173dc76902,0x001c4c4bfe1be552)},-  {FIELD_LITERAL(0x00a2657cac024d24,0x00aa33dfb739670f,0x00093b53769a8de7,0x00adafcb28c0514d,0x00bca8890425c381,0x008f15acedcdc343,0x0085efa2bb2f9604,0x0092437292387955)},-  {FIELD_LITERAL(0x00dfb010d979be8f,0x007e6d963a211f07,0x00404b8ec1368699,0x00d9cc6590cb2087,0x00e0d919b389e23c,0x001001c50cec349f,0x001e848fec709fe4,0x000e91e3326121a1)},-  {FIELD_LITERAL(0x00e8300e632c6b13,0x00010847ef6dda78,0x0019b7c68f200ab7,0x00220c952978bd9b,0x0019e887adc0331c,0x006c5993f36c4db5,0x0002c98eeb248079,0x0089ad282231d922)},-  {FIELD_LITERAL(0x0059811830606614,0x00a8ec4d8a0d0097,0x000e2ac957beaec2,0x007dc4a64fdb8ed1,0x0063b9462f2c7312,0x00324ea6a55d282b,0x007c8a4cbdc26507,0x00f54f4ae9268708)},-  {FIELD_LITERAL(0x0026d312845ed7bc,0x0051563888e17918,0x00b99c696ccab084,0x0059d7244957f3b8,0x00c5f4faf8c8d6ab,0x00bdeeec54ba3f26,0x001aba0f7c9d5485,0x00d731f784b29269)},-  {FIELD_LITERAL(0x00bd7234c3aef4f0,0x00a7a9f815db44b1,0x00c8c940e9fc9785,0x003b81a973b01c38,0x00c32ffd7d7b79f9,0x00bc5b783c46e6c6,0x00b003fb1ef6a5f9,0x005b36765c2b46e7)},-  {FIELD_LITERAL(0x0030b09f9659a719,0x00ac35ad7a6bc959,0x009b466b281c1ee8,0x0034b96465f80acb,0x00304970c66162b7,0x000f2347253e3918,0x000d54980ac74c5a,0x00aaabb0e875468a)},-  {FIELD_LITERAL(0x00578872f1bd6085,0x00b3fd4fa6efa597,0x00e99ac49f625c00,0x002aef842e5ed2d8,0x004b8f706588e353,0x00449c499dfcc096,0x008d0cdddbf18dea,0x00e6bba4a6396ddd)},-  {FIELD_LITERAL(0x0066485d97a2ac73,0x001d0e768483ffe7,0x00c5253731b7251c,0x00f76d892a3af3f3,0x00e8d035f85298e7,0x0034e58d0abf961a,0x00b11bd0eccaba4c,0x0087a079aec9d0e9)},-  {FIELD_LITERAL(0x00d38488bd2e2026,0x00d35414e79dc3fe,0x00faa0a1c1fbbbb9,0x0093df0c4b10ab45,0x0039ffebe1394c9f,0x00cab0bc80e5cd5c,0x00453b9db5cadf06,0x003b7c08cb56f96e)},-  {FIELD_LITERAL(0x00b63453c7af61ee,0x00eadcbafa2bd320,0x0086b04f4a7bf0e3,0x00b69bc8cbbfba5a,0x00ce4926bb1b064e,0x004df8ce753e0a27,0x00ff37bf2580a3a2,0x00ad90c8c5a377eb)},-  {FIELD_LITERAL(0x00ac58c82bdd6e72,0x0008035e278a79da,0x003c9fcc92524fb3,0x000c71c26ea75e47,0x009631c4be717b38,0x00a2e968135e9152,0x00074295ca131ec2,0x00877a203d4a5015)},-  {FIELD_LITERAL(0x00a49896f002be26,0x00ad6b0d720ae906,0x005786d8dbed0346,0x00f6749d6592e372,0x000542c37faf79a4,0x003281a4f5c7863a,0x00eacdc7def0cbdc,0x00ca8353efe160bd)},-  {FIELD_LITERAL(0x003c9e851d9f8893,0x004df23c1696dd28,0x005e587fddb98f95,0x00359afa5adbfdbb,0x00ddb949d26e687c,0x00ebc6efd285564c,0x001750eec619bdd3,0x0037772e4ad0d4fa)},-  {FIELD_LITERAL(0x0076e84babbbb048,0x000a6db83681bbe4,0x0059dff597eaead2,0x00f65bdd79fe2dab,0x00e3fc9faa642c8a,0x008a9cc9dfc634c9,0x00428a4b728b1cd4,0x00e80aea53cb6617)},-  {FIELD_LITERAL(0x002ab17fdf7d2bd3,0x005aa55f23183393,0x009b88469f8c0eb9,0x007d101b314bca6b,0x0056dd4345fd97b9,0x00880e62e548ae7d,0x003d44d8c87b91a6,0x00fb2811386e22cc)},-  {FIELD_LITERAL(0x00eacd58001be3a5,0x0014e1231ca72940,0x0022453384987584,0x0075848f0c37be5c,0x000e6dc40d82c0b2,0x00f4d8ec1270878c,0x00550981d6fb86fd,0x00bb66b58f4c6892)},-  {FIELD_LITERAL(0x00bba772e57e297f,0x004f56f68df71b07,0x00ded9facaf23a81,0x00d78e832d78eedc,0x0004f7c3eff02685,0x00ba5fa931f9c020,0x005a29fb4b2295be,0x00e2543f745b1dc9)},-  {FIELD_LITERAL(0x00712177652580f9,0x00e9ee16e21d1eca,0x0002465ba75b8e46,0x00a9cb7b1fc8ef2e,0x00ce337e6da1cf8e,0x009d3684c507fffa,0x00058cc115d71214,0x0017dba81e144377)},-  {FIELD_LITERAL(0x003b778e67285805,0x00dbb06704ba87b5,0x00ba6ee1ea5ea2fe,0x00e2cdc2c8b3f699,0x006983c6eae69a9c,0x00c6c8c542d0c398,0x00f2d3a9ebcedbdc,0x00be30ddeabbd31c)},-  {FIELD_LITERAL(0x0095f20a016490a6,0x005f2b00b9fbf26d,0x00b583124906cdaf,0x002e2077aa473ca8,0x0018c5b9f7902fa6,0x00b704f5229201a6,0x00e1fc5d70e4b1c2,0x00578e366ccf7289)},-  {FIELD_LITERAL(0x00932127be1d579d,0x00e6729f50f54904,0x00e70f6247f618af,0x00b1953989fe9d9c,0x0015032e9df69633,0x00d3687b35cb6e82,0x00ab0fff86869218,0x0026054a3a68ddfb)},-  {FIELD_LITERAL(0x00cf244d2e899137,0x00a793f52ec7aaa1,0x002e5cb0616e3883,0x009cbf752f176feb,0x0029edce4fa090a3,0x00f6540a960a0275,0x00513985eef0e3bc,0x00ce2e586f6c7228)},-  {FIELD_LITERAL(0x00b42f011dbc757c,0x004a8e19d4f07c42,0x00a6d7828318b7ff,0x0004c9ce49ba3c0f,0x005fe71688087b6a,0x006e1d8f9a3d84ed,0x0089693e7e8e9a1f,0x0073bf4183ba45c5)},-  {FIELD_LITERAL(0x0029e8ce35530d30,0x00d20f389f61fe3a,0x00cf9e8ddf74e1d4,0x004bec01b04d4979,0x007d92c9f6fd5ddd,0x00c072fa91981808,0x009afda4fe8a1676,0x00c96522ee879a14)},-  {FIELD_LITERAL(0x005f0cd9cd83497b,0x00e382f098d97f00,0x0073e37e004eed2e,0x000707fe98b12237,0x0016d92a2b73d561,0x00a42926ab390165,0x00b394db4b1cc8fc,0x002fa14a3f6efa33)},-  {FIELD_LITERAL(0x0055076a513d05ee,0x00f076d43cec14ad,0x00a4e386b252faf4,0x00c0713b79b313eb,0x00507efa72f46f19,0x00141bc1e7c66844,0x005629ef060c19ea,0x0085327113d1772c)},-  {FIELD_LITERAL(0x00ed490108514e35,0x006bed897e6b4958,0x0000f2cae0dc546c,0x008175eb3e5008e4,0x0093e3fe8f3aed42,0x00e9dbc15fd54d1a,0x00844979a4cfc0c1,0x00ea3194d64ea60b)},-  {FIELD_LITERAL(0x00b64d054ec7ed5c,0x007b924cd329fbce,0x00fe8805a8737293,0x00fb82f1d52b43ae,0x004ea745c72e1a76,0x0095ba2552861c0c,0x00f66846c3547784,0x003b815bd05dc23c)},-  {FIELD_LITERAL(0x00669e32fd197ef7,0x001dfca2c5e2f7c9,0x00a2ae0964a1e5e2,0x00b4334b15c91232,0x0096419585110d96,0x009c0b2262172a58,0x009d7c87cf6d35ca,0x008a5ce50d3cabf6)},-  {FIELD_LITERAL(0x00888b9c1cf73530,0x00375346c6afecd2,0x00142240b35b74d3,0x00d952835f86a5f5,0x000665c2658eaf9a,0x00f29f43062b2033,0x00a19a58c5bc85f9,0x00e62ac95724a937)},-  {FIELD_LITERAL(0x003bedc9ae9d1730,0x00fedd7c04cbc775,0x00c19abc4540c61d,0x00115294c57fb687,0x00663fceb174cd8f,0x001671f572b885b0,0x002d14694ed85978,0x00127282078a8e44)},-  {FIELD_LITERAL(0x00e6d2822aa72eca,0x00d832957cdc0058,0x00dc60e5bed23e18,0x00b94b4c418b03a3,0x00df3b85d410a430,0x0055e81b70bc79d4,0x00081d9369cbd1a0,0x00f7fee3acf0c656)},-  {FIELD_LITERAL(0x003baba41b5abffb,0x00661ee09fca8193,0x00e0c6c92e6aea59,0x00886c207bcbe591,0x00aef9e7798e8004,0x00164f599f4d707a,0x00bb1597a76d21f2,0x00fda82d5e025626)},-  {FIELD_LITERAL(0x00552b53a9640f0e,0x005985236f4d88bf,0x00b7aaec965a8ae5,0x00cedada7b5ccf95,0x007b1ea2088f1902,0x0028445e38b4a7fa,0x0057f10ddc50efed,0x007637a3147bc5cb)},-  {FIELD_LITERAL(0x008174fe4db53757,0x00930c4f4a35ecc8,0x000e9f82c1c95a8f,0x00c6480547d66e5e,0x00dce888f9a7bf39,0x006671a5022cb906,0x004823c19b5337a0,0x00455338b7fec529)},-  {FIELD_LITERAL(0x005ac123fdc45964,0x00395057c2221d17,0x003c09c74cf84eb1,0x00b5ca859bbebf9d,0x001b26b274a7d235,0x00e8c63508e96a48,0x00edbce4d51d721e,0x00c49436797d6f83)},-  {FIELD_LITERAL(0x0071595be88a7f40,0x00a05e6ac1c0fc87,0x00a01bf6538b29eb,0x00badcd80b881fb8,0x005bfe7af8049f8b,0x0084918e6ae35537,0x00ed4bd54759316e,0x007f135988d6b548)},-  {FIELD_LITERAL(0x0075656c41e06629,0x0086059d83396637,0x004f304ecb457b37,0x00e3b4887db6be65,0x0020b54c263bb0be,0x0060a69193e561c3,0x00e6863f20dc8ce9,0x00afe16ac56e6478)}+  {FIELD_LITERAL(0x00303cda6feea532,0x00860f1d5a3850e4,0x00226b9fa4728ccd,0x00e822938a0a0c0c,0x00263a61c9ea9216,0x001204029321b828,0x006a468360983c65,0x0002846f0a782143)},+  {FIELD_LITERAL(0x00303cda6feea532,0x00860f1d5a3850e4,0x00226b9fa4728ccd,0x006822938a0a0c0c,0x00263a61c9ea9215,0x001204029321b828,0x006a468360983c65,0x0082846f0a782143)},+  {FIELD_LITERAL(0x00ef8e22b275198d,0x00b0eb141a0b0e8b,0x001f6789da3cb38c,0x006d2ff8ed39073e,0x00610bdb69a167f3,0x00571f306c9689b4,0x00f557e6f84b2df8,0x002affd38b2c86db)},+  {FIELD_LITERAL(0x00cea0fc8d2e88b5,0x00821612d69f1862,0x0074c283b3e67522,0x005a195ba05a876d,0x000cddfe557feea4,0x008046c795bcc5e5,0x00540969f4d6e119,0x00d27f96d6b143d5)},+  {FIELD_LITERAL(0x000c3b1019d474e8,0x00e19533e4952284,0x00cc9810ba7c920a,0x00f103d2785945ac,0x00bfa5696cc69b34,0x00a8d3d51e9ca839,0x005623cb459586b9,0x00eae7ce1cd52e9e)},+  {FIELD_LITERAL(0x0005a178751dd7d8,0x002cc3844c69c42f,0x00acbfe5efe10539,0x009c20f43431a65a,0x008435d96374a7b3,0x009ee57566877bd3,0x0044691725ed4757,0x001e87bb2fe2c6b2)},+  {FIELD_LITERAL(0x000cedc4debf7a04,0x002ffa45000470ac,0x002e9f9678201915,0x0017da1208c4fe72,0x007d558cc7d656cb,0x0037a827287cf289,0x00142472d3441819,0x009c21f166cf8dd1)},+  {FIELD_LITERAL(0x003ef83af164b2f2,0x000949a5a0525d0d,0x00f4498186cac051,0x00e77ac09ef126d2,0x0073ae0b2c9296e9,0x001c163f6922e3ed,0x0062946159321bea,0x00cfb79b22990b39)},+  {FIELD_LITERAL(0x00b001431ca9e654,0x002d7e5eabcc9a3a,0x0052e8114c2f6747,0x0079ac4f94487f92,0x00bffd919b5d749c,0x00261f92ad15e620,0x00718397b7a97895,0x00c1443e6ebbc0c4)},+  {FIELD_LITERAL(0x00eacd90c1e0a049,0x008977935b149fbe,0x0004cb9ba11c93dc,0x009fbd5b3470844d,0x004bc18c9bfc22cf,0x0057679a991839f3,0x00ef15b76fb4092e,0x0074a5173a225041)},+  {FIELD_LITERAL(0x003f5f9d7ec4777b,0x00ab2e733c919c94,0x001bb6c035245ae5,0x00a325a49a883630,0x0033e9a9ea3cea2f,0x00e442a1eaa0e844,0x00b2116d5b0e71b8,0x00c16abed6d64047)},+  {FIELD_LITERAL(0x00c560b5ed051165,0x001945adc5d65094,0x00e221865710f910,0x00cc12bc9e9b8ceb,0x004faa9518914e35,0x0017476d89d42f6d,0x00b8f637c8fa1c8b,0x0088c7d2790864b8)},+  {FIELD_LITERAL(0x00ef7eafc1c69be6,0x0085d3855778fbea,0x002c8d5b450cb6f5,0x004e77de5e1e7fec,0x0047c057893abded,0x001b430b85d51e16,0x00965c7b45640c3c,0x00487b2bb1162b97)},+  {FIELD_LITERAL(0x0099c73a311beec2,0x00a3eff38d8912ad,0x002efa9d1d7e8972,0x00f717ae1e14d126,0x002833f795850c8b,0x0066c12ad71486bd,0x00ae9889da4820eb,0x00d6044309555c08)},+  {FIELD_LITERAL(0x004b1c5283d15e41,0x00669d8ea308ff75,0x0004390233f762a1,0x00e1d67b83cb6cec,0x003eebaa964c78b1,0x006b0aff965eb664,0x00b313d4470bdc37,0x008814ffcb3cb9d8)},+  {FIELD_LITERAL(0x009724b8ce68db70,0x007678b5ed006f3d,0x00bdf4b89c0abd73,0x00299748e04c7c6d,0x00ddd86492c3c977,0x00c5a7febfa30a99,0x00ed84715b4b02bb,0x00319568adf70486)},+  {FIELD_LITERAL(0x0070ff2d864de5bb,0x005a37eeb637ee95,0x0033741c258de160,0x00e6ca5cb1988f46,0x001ceabd92a24661,0x0030957bd500fe40,0x001c3362afe912c5,0x005187889f678bd2)},+  {FIELD_LITERAL(0x0086835fc62bbdc7,0x009c3516ca4910a1,0x00956c71f8d00783,0x0095c78fcf63235f,0x00fc7ff6ba05c222,0x00cdd8b3f8d74a52,0x00ac5ae16de8256e,0x00e9d4be8ed48624)},+  {FIELD_LITERAL(0x00c0ce11405df2d8,0x004e3f37b293d7b6,0x002410172e1ac6db,0x00b8dbff4bf8143d,0x003a7b409d56eb66,0x003e0f6a0dfef9af,0x0081c4e4d3645be1,0x00ce76076b127623)},+  {FIELD_LITERAL(0x00f6ee0f98974239,0x0042d89af07d3a4f,0x00846b7fe84346b5,0x006a21fc6a8d39a1,0x00ac8bc2541ff2d9,0x006d4e2a77732732,0x009a39b694cc3f2f,0x0085c0aa2a404c8f)},+  {FIELD_LITERAL(0x00b261101a218548,0x00c1cae96424277b,0x00869da0a77dd268,0x00bc0b09f8ec83ea,0x00d61027f8e82ba9,0x00aa4c85999dce67,0x00eac3132b9f3fe1,0x00fb9b0cf1c695d2)},+  {FIELD_LITERAL(0x0043079295512f0d,0x0046a009861758e0,0x003ee2842a807378,0x0034cc9d1298e4fa,0x009744eb4d31b3ee,0x00afacec96650cd0,0x00ac891b313761ae,0x00e864d6d26e708a)},+  {FIELD_LITERAL(0x00a84d7c8a23b491,0x0088e19aa868b27f,0x0005986d43e78ce9,0x00f28012f0606d28,0x0017ded7e10249b3,0x005ed4084b23af9b,0x00b9b0a940564472,0x00ad9056cceeb1f4)},+  {FIELD_LITERAL(0x00db91b357fe755e,0x00a1aa544b15359c,0x00af4931a0195574,0x007686124fe11aef,0x00d1ead3c7b9ef7e,0x00aaf5fc580f8c15,0x00e727be147ee1ec,0x003c61c1e1577b86)},+  {FIELD_LITERAL(0x009d3fca983220cf,0x00cd11acbc853dc4,0x0017590409d27f1d,0x00d2176698082802,0x00fa01251b2838c8,0x00dd297a0d9b51c6,0x00d76c92c045820a,0x00534bc7c46c9033)},+  {FIELD_LITERAL(0x0080ed9bc9b07338,0x00fceac7745d2652,0x008a9d55f5f2cc69,0x0096ce72df301ac5,0x00f53232e7974d87,0x0071728c7ae73947,0x0090507602570778,0x00cb81cfd883b1b2)},+  {FIELD_LITERAL(0x005011aadea373da,0x003a8578ec896034,0x00f20a6535fa6d71,0x005152d31e5a87cf,0x002bac1c8e68ca31,0x00b0e323db4c1381,0x00f1d596b7d5ae25,0x00eae458097cb4e0)},+  {FIELD_LITERAL(0x00920ac80f9b0d21,0x00f80f7f73401246,0x0086d37849b557d6,0x0002bd4b317b752e,0x00b26463993a42bb,0x002070422a73b129,0x00341acaa0380cb3,0x00541914dd66a1b2)},+  {FIELD_LITERAL(0x00c1513cd66abe8c,0x000139e01118944d,0x0064abbcb8080bbb,0x00b3b08202473142,0x00c629ef25da2403,0x00f0aec3310d9b7f,0x0050b2227472d8cd,0x00f6c8a922d41fb4)},+  {FIELD_LITERAL(0x001075ccf26b7b1f,0x00bb6bb213170433,0x00e9491ad262da79,0x009ef4f48d2d384c,0x008992770766f09d,0x001584396b6b1101,0x00af3f8676c9feef,0x0024603c40269118)},+  {FIELD_LITERAL(0x009dd7b31319527c,0x001e7ac948d873a9,0x00fa54b46ef9673a,0x0066efb8d5b02fe6,0x00754b1d3928aeae,0x0004262ac72a6f6b,0x0079b7d49a6eb026,0x003126a753540102)},+  {FIELD_LITERAL(0x009666e24f693947,0x00f714311269d45f,0x0010ffac1d0c851c,0x0066e80c37363497,0x00f1f4ad010c60b0,0x0015c87408470ff7,0x00651d5e9c7766a4,0x008138819d7116de)},+  {FIELD_LITERAL(0x003934b11c57253b,0x00ef308edf21f46e,0x00e54e99c7a16198,0x0080d57135764e63,0x00751c27b946bc24,0x00dd389ce4e9e129,0x00a1a2bfd1cd84dc,0x002fae73e5149b32)},+  {FIELD_LITERAL(0x00911657dffb4cdd,0x00c100b7cc553d06,0x00449d075ec467cc,0x007062100bc64e70,0x0043cf86f7bd21e7,0x00f401dc4b797dea,0x005224afb2f62e65,0x00d1ede3fb5a42be)},+  {FIELD_LITERAL(0x00f2ba36a41aa144,0x00a0c22d946ee18f,0x008aae8ef9a14f99,0x00eef4d79b19bb36,0x008e75ce3d27b1fc,0x00a65daa03b29a27,0x00d9cc83684eb145,0x009e1ed80cc2ed74)},+  {FIELD_LITERAL(0x00bed953d1997988,0x00b93ed175a24128,0x00871c5963fb6365,0x00ca2df20014a787,0x00f5d9c1d0b34322,0x00f6f5942818db0a,0x004cc091f49c9906,0x00e8a188a60bff9f)},+  {FIELD_LITERAL(0x0032c7762032fae8,0x00e4087232e0bc21,0x00f767344b6e8d85,0x00bbf369b76c2aa2,0x008a1f46c6e1570c,0x001368cd9780369f,0x007359a39d079430,0x0003646512921434)},+  {FIELD_LITERAL(0x007c4b47ca7c73e7,0x005396221039734b,0x008b64ddf0e45d7e,0x00bfad5af285e6c2,0x008ec711c5b1a1a8,0x00cf663301237f98,0x00917ee3f1655126,0x004152f337efedd8)},+  {FIELD_LITERAL(0x0007c7edc9305daa,0x000a6664f273701c,0x00f6e78795e200b1,0x005d05b9ecd2473e,0x0014f5f17c865786,0x00c7fd2d166fa995,0x004939a2d8eb80e0,0x002244ba0942c199)},+  {FIELD_LITERAL(0x00321e767f0262cf,0x002e57d776caf68e,0x00bf2c94814f0437,0x00c339196acd622f,0x001db4cce71e2770,0x001ded5ddba6eee2,0x0078608ab1554c8d,0x00067fe0ab76365b)},+  {FIELD_LITERAL(0x00f09758e11e3985,0x00169efdbd64fad3,0x00e8889b7d6dacd6,0x0035cdd58ea88209,0x00bcda47586d7f49,0x003cdddcb2879088,0x0016da70187e954b,0x009556ea2e92aacd)},+  {FIELD_LITERAL(0x008cab16bd1ff897,0x00b389972cdf753f,0x00ea8ed1e46dfdc0,0x004fe7ef94c589f4,0x002b8ae9b805ecf3,0x0025c08d892874a5,0x0023938e98d44c4c,0x00f759134cabf69c)},+  {FIELD_LITERAL(0x006c2a84678e4b3b,0x007a194aacd1868f,0x00ed0225af424761,0x00da0a6f293c64b8,0x001062ac5c6a7a18,0x0030f5775a8aeef4,0x0002acaad76b7af0,0x00410b8fd63a579f)},+  {FIELD_LITERAL(0x001ec59db3d9590e,0x001e9e3f1c3f182d,0x0045a9c3ec2cab14,0x0008198572aeb673,0x00773b74068bd167,0x0012535eaa395434,0x0044dba9e3bbb74a,0x002fba4d3c74bd0e)},+  {FIELD_LITERAL(0x0042bf08fe66922c,0x003318b8fbb49e8c,0x00d75946004aa14c,0x00f601586b42bf1c,0x00c74cf1d912fe66,0x00abcb36974b30ad,0x007eb78720c9d2b8,0x009f54ab7bd4df85)},+  {FIELD_LITERAL(0x00db9fc948f73826,0x00fa8b3746ed8ee9,0x00132cb65aafbeb2,0x00c36ff3fe7925b8,0x00837daed353d2fe,0x00ec661be0667cf4,0x005beb8ed2e90204,0x00d77dd69e564967)},+  {FIELD_LITERAL(0x0042e6268b861751,0x0008dd0469500c16,0x00b51b57c338a3fd,0x00cc4497d85cff6b,0x002f13d6b57c34a4,0x0083652eaf301105,0x00cc344294cc93a8,0x0060f4d02810e270)},+  {FIELD_LITERAL(0x00a8954363cd518b,0x00ad171124bccb7b,0x0065f46a4adaae00,0x001b1a5b2a96e500,0x0043fe24f8233285,0x0066996d8ae1f2c3,0x00c530f3264169f9,0x00c0f92d07cf6a57)},+  {FIELD_LITERAL(0x0036a55c6815d943,0x008c8d1def993db3,0x002e0e1e8ff7318f,0x00d883a4b92db00a,0x002f5e781ae33906,0x001a72adb235c06d,0x00f2e59e736e9caa,0x001a4b58e3031914)},+  {FIELD_LITERAL(0x00d73bfae5e00844,0x00bf459766fb5f52,0x0061b4f5a5313cde,0x004392d4c3b95514,0x000d3551b1077523,0x0000998840ee5d71,0x006de6e340448b7b,0x00251aa504875d6e)},+  {FIELD_LITERAL(0x003bf343427ac342,0x00adc0a78642b8c5,0x0003b893175a8314,0x0061a34ade5703bc,0x00ea3ea8bb71d632,0x00be0df9a1f198c2,0x0046dd8e7c1635fb,0x00f1523fdd25d5e5)},+  {FIELD_LITERAL(0x00633f63fc9dd406,0x00e713ff80e04a43,0x0060c6e970f2d621,0x00a57cd7f0df1891,0x00f2406a550650bb,0x00b064290efdc684,0x001eab0144d17916,0x00cd15f863c293ab)},+  {FIELD_LITERAL(0x0029cec55273f70d,0x007044ee275c6340,0x0040f637a93015e2,0x00338bb78db5aae9,0x001491b2a6132147,0x00a125d6cfe6bde3,0x005f7ac561ba8669,0x001d5eaea3fbaacf)},+  {FIELD_LITERAL(0x00054e9635e3be31,0x000e43f31e2872be,0x00d05b1c9e339841,0x006fac50bd81fd98,0x00cdc7852eaebb09,0x004ff519b061991b,0x009099e8107d4c85,0x00273e24c36a4a61)},+  {FIELD_LITERAL(0x00070b4441ef2c46,0x00efa5b02801a109,0x00bf0b8c3ee64adf,0x008a67e0b3452e98,0x001916b1f2fa7a74,0x00d781a78ff6cdc3,0x008682ce57e5c919,0x00cc1109dd210da3)},+  {FIELD_LITERAL(0x00cae8aaff388663,0x005e983a35dda1c7,0x007ab1030d8e37f4,0x00e48940f5d032fe,0x006a36f9ef30b331,0x009be6f03958c757,0x0086231ceba91400,0x008bd0f7b823e7aa)},+  {FIELD_LITERAL(0x00cf881ebef5a45a,0x004ebea78e7c6f2c,0x0090da9209cf26a0,0x00de2b2e4c775b84,0x0071d6031c3c15ae,0x00d9e927ef177d70,0x00894ee8c23896fd,0x00e3b3b401e41aad)},+  {FIELD_LITERAL(0x00204fef26864170,0x00819269c5dee0f8,0x00bfb4713ec97966,0x0026339a6f34df78,0x001f26e64c761dc2,0x00effe3af313cb60,0x00e17b70138f601b,0x00f16e1ccd9ede5e)},+  {FIELD_LITERAL(0x005d9a8353fdb2db,0x0055cc2048c698f0,0x00f6c4ac89657218,0x00525034d73faeb2,0x00435776fbda3c7d,0x0070ea5312323cbc,0x007a105d44d069fb,0x006dbc8d6dc786aa)},+  {FIELD_LITERAL(0x0017cff19cd394ec,0x00fef7b810922587,0x00e6483970dff548,0x00ddf36ad6874264,0x00e61778523fcce2,0x0093a66c0c93b24a,0x00fd367114db7f86,0x007652d7ddce26dd)},+  {FIELD_LITERAL(0x00d92ced7ba12843,0x00aea9c7771e86e7,0x0046639693354f7b,0x00a628dbb6a80c47,0x003a0b0507372953,0x00421113ab45c0d9,0x00e545f08362ab7a,0x0028ce087b4d6d96)},+  {FIELD_LITERAL(0x00a67ee7cf9f99eb,0x005713b275f2ff68,0x00f1d536a841513d,0x00823b59b024712e,0x009c46b9d0d38cec,0x00cdb1595aa2d7d4,0x008375b3423d9af8,0x000ab0b516d978f7)},+  {FIELD_LITERAL(0x00428dcb3c510b0f,0x00585607ea24bb4e,0x003736bf1603687a,0x00c47e568c4fe3c7,0x003cd00282848605,0x0043a487c3b91939,0x004ffc04e1095a06,0x00a4c989a3d4b918)},+  {FIELD_LITERAL(0x00a8778d0e429f7a,0x004c02b059105a68,0x0016653b609da3ff,0x00d5107bd1a12d27,0x00b4708f9a771cab,0x00bb63b662033f69,0x0072f322240e7215,0x0019445b59c69222)},+  {FIELD_LITERAL(0x00cf4f6069a658e6,0x0053ca52859436a6,0x0064b994d7e3e117,0x00cb469b9a07f534,0x00cfb68f399e9d47,0x00f0dcb8dac1c6e7,0x00f2ab67f538b3a5,0x0055544f178ab975)},+  {FIELD_LITERAL(0x0099b7a2685d538c,0x00e2f1897b7c0018,0x003adac8ce48dae3,0x00089276d5c50c0c,0x00172fca07ad6717,0x00cb1a72f54069e5,0x004ee42f133545b3,0x00785f8651362f16)},+  {FIELD_LITERAL(0x0049cbac38509e11,0x0015234505d42cdf,0x00794fb0b5840f1c,0x00496437344045a5,0x0031b6d944e4f9b0,0x00b207318ac1f5d8,0x0000c840da7f5c5d,0x00526f373a5c8814)},+  {FIELD_LITERAL(0x002c7b7742d1dfd9,0x002cabeb18623c01,0x00055f5e3e044446,0x006c20f3b4ef54ba,0x00c600141ec6b35f,0x00354f437f1a32a3,0x00bac4624a3520f9,0x00c483f734a90691)},+  {FIELD_LITERAL(0x0053a737d422918d,0x00f7fca1d8758625,0x00c360336dadb04c,0x00f38e3d9158a1b8,0x0069ce3b418e84c6,0x005d1697eca16ead,0x00f8bd6a35ece13d,0x007885dfc2b5afea)},+  {FIELD_LITERAL(0x00c3617ae260776c,0x00b20dc3e96922d7,0x00a1a7802246706a,0x00ca6505a5240244,0x002246b62d919782,0x001439102d7aa9b3,0x00e8af1139e6422c,0x00c888d1b52f2b05)},+  {FIELD_LITERAL(0x005b67690ffd41d9,0x005294f28df516f9,0x00a879272412fcb9,0x00098b629a6d1c8d,0x00fabd3c8050865a,0x00cd7e5b0a3879c5,0x00153238210f3423,0x00357cac101e9f42)},+  {FIELD_LITERAL(0x008917b454444fb7,0x00f59247c97e441b,0x00a6200a6815152d,0x0009a4228601d254,0x001c0360559bd374,0x007563362039cb36,0x00bd75b48d74e32b,0x0017f515ac3499e8)},+  {FIELD_LITERAL(0x001532a7ffe41c5a,0x00eb1edce358d6bf,0x00ddbacc7b678a7b,0x008a7b70f3c841a3,0x00f1923bf27d3f4c,0x000b2713ed8f7873,0x00aaf67e29047902,0x0044994a70b3976d)},+  {FIELD_LITERAL(0x00d54e802082d42c,0x00a55aa0dce7cc6c,0x006477b96073f146,0x0082efe4ceb43594,0x00a922bcba026845,0x0077f19d1ab75182,0x00c2bb2737846e59,0x0004d7eec791dd33)},+  {FIELD_LITERAL(0x0044588d1a81d680,0x00b0a9097208e4f8,0x00212605350dc57e,0x0028717cd2871123,0x00fb083c100fd979,0x0045a056ce063fdf,0x00a5d604b4dd6a41,0x001dabc08ba4e236)},+  {FIELD_LITERAL(0x00c4887198d7a7fa,0x00244f98fb45784a,0x0045911e15a15d01,0x001d323d374c0966,0x00967c3915196562,0x0039373abd2f3c67,0x000d2c5614312423,0x0041cf2215442ce3)},+  {FIELD_LITERAL(0x008ede889ada7f06,0x001611e91de2e135,0x00fdb9a458a471b9,0x00563484e03710d1,0x0031cc81925e3070,0x0062c97b3af80005,0x00fa733eea28edeb,0x00e82457e1ebbc88)},+  {FIELD_LITERAL(0x006a0df5fe9b6f59,0x00a0d4ff46040d92,0x004a7cedb6f93250,0x00d1df8855b8c357,0x00e73a46086fd058,0x0048fb0add6dfe59,0x001e03a28f1b4e3d,0x00a871c993308d76)},+  {FIELD_LITERAL(0x0030dbb2d1766ec8,0x00586c0ad138555e,0x00d1a34f9e91c77c,0x0063408ad0e89014,0x00d61231b05f6f5b,0x0009abf569f5fd8a,0x00aec67a110f1c43,0x0031d1a790938dd7)},+  {FIELD_LITERAL(0x006cded841e2a862,0x00198d60af0ab6fb,0x0018f09db809e750,0x004e6ac676016263,0x00eafcd1620969cb,0x002c9784ca34917d,0x0054f00079796de7,0x00d9fab5c5972204)},+  {FIELD_LITERAL(0x004bd0fee2438a83,0x00b571e62b0f83bd,0x0059287d7ce74800,0x00fb3631b645c3f0,0x00a018e977f78494,0x0091e27065c27b12,0x007696c1817165e0,0x008c40be7c45ba3a)},+  {FIELD_LITERAL(0x00a0f326327cb684,0x001c7d0f672680ff,0x008c1c81ffb112d1,0x00f8f801674eddc8,0x00e926d5d48c2a9d,0x005bd6d954c6fe9a,0x004c6b24b4e33703,0x00d05eb5c09105cc)},+  {FIELD_LITERAL(0x00d61731caacf2cf,0x002df0c7609e01c5,0x00306172208b1e2b,0x00b413fe4fb2b686,0x00826d360902a221,0x003f8d056e67e7f7,0x0065025b0175e989,0x00369add117865eb)},+  {FIELD_LITERAL(0x00aaf895aec2fa11,0x000f892bc313eb52,0x005b1c794dad050b,0x003f8ec4864cec14,0x00af81058d0b90e5,0x00ebe43e183997bb,0x00a9d610f9f3e615,0x007acd8eec2e88d3)},+  {FIELD_LITERAL(0x0049b2fab13812a3,0x00846db32cd60431,0x000177fa578c8d6c,0x00047d0e2ad4bc51,0x00b158ba38d1e588,0x006a45daad79e3f3,0x000997b93cab887b,0x00c47ea42fa23dc3)},+  {FIELD_LITERAL(0x0012b6fef7aeb1ca,0x009412768194b6a7,0x00ff0d351f23ab93,0x007e8a14c1aff71b,0x006c1c0170c512bc,0x0016243ea02ab2e5,0x007bb6865b303f3e,0x0015ce6b29b159f4)},+  {FIELD_LITERAL(0x009961cd02e68108,0x00e2035d3a1d0836,0x005d51f69b5e1a1d,0x004bccb4ea36edcd,0x0069be6a7aeef268,0x0063f4dd9de8d5a7,0x006283783092ca35,0x0075a31af2c35409)},+  {FIELD_LITERAL(0x00c412365162e8cf,0x00012283fb34388a,0x003e6543babf39e2,0x00eead6b3a804978,0x0099c0314e8b326f,0x00e98e0a8d477a4f,0x00d2eb96b127a687,0x00ed8d7df87571bb)},+  {FIELD_LITERAL(0x00777463e308cacf,0x00c8acb93950132d,0x00ebddbf4ca48b2c,0x0026ad7ca0795a0a,0x00f99a3d9a715064,0x000d60bcf9d4dfcc,0x005e65a73a437a06,0x0019d536a8db56c8)},+  {FIELD_LITERAL(0x00192d7dd558d135,0x0027cd6a8323ffa7,0x00239f1a412dc1e7,0x0046b4b3be74fc5c,0x0020c47a2bef5bce,0x00aa17e48f43862b,0x00f7e26c96342e5f,0x0008011c530f39a9)},+  {FIELD_LITERAL(0x00aad4ac569bf0f1,0x00a67adc90b27740,0x0048551369a5751a,0x0031252584a3306a,0x0084e15df770e6fc,0x00d7bba1c74b5805,0x00a80ef223af1012,0x0089c85ceb843a34)},+  {FIELD_LITERAL(0x00c4545be4a54004,0x0099e11f60357e6c,0x001f3936d19515a6,0x007793df84341a6e,0x0051061886717ffa,0x00e9b0a660b28f85,0x0044ea685892de0d,0x000257d2a1fda9d9)},+  {FIELD_LITERAL(0x007e8b01b24ac8a8,0x006cf3b0b5ca1337,0x00f1607d3e36a570,0x0039b7fab82991a1,0x00231777065840c5,0x00998e5afdd346f9,0x00b7dc3e64acc85f,0x00baacc748013ad6)},+  {FIELD_LITERAL(0x008ea6a4177580bf,0x005fa1953e3f0378,0x005fe409ac74d614,0x00452327f477e047,0x00a4018507fb6073,0x007b6e71951caac8,0x0012b42ab8a6ce91,0x0080eca677294ab7)},+  {FIELD_LITERAL(0x00a53edc023ba69b,0x00c6afa83ddde2e8,0x00c3f638b307b14e,0x004a357a64414062,0x00e4d94d8b582dc9,0x001739caf71695b7,0x0012431b2ae28de1,0x003b6bc98682907c)},+  {FIELD_LITERAL(0x008a9a93be1f99d6,0x0079fa627cc699c8,0x00b0cfb134ba84c8,0x001c4b778249419a,0x00df4ab3d9c44f40,0x009f596e6c1a9e3c,0x001979c0df237316,0x00501e953a919b87)} };
cbits/decaf/ed448goldilocks/eddsa.c view
@@ -31,18 +31,13 @@ #define NO_CONTEXT CRYPTON_DECAF_EDDSA_448_SUPPORTS_CONTEXTLESS_SIGS #define EDDSA_USE_SIGMA_ISOGENY 0 #define COFACTOR 4+#define EDDSA_PREHASH_BYTES 64  #if NO_CONTEXT const uint8_t CRYPTON_NO_CONTEXT_POINTS_HERE = 0; const uint8_t * const CRYPTON_DECAF_ED448_NO_CONTEXT = &CRYPTON_NO_CONTEXT_POINTS_HERE; #endif -/* EDDSA_BASE_POINT_RATIO = 1 or 2- * Because EdDSA25519 is not on E_d but on the isogenous E_sigma_d,- * its base point is twice ours.- */-#define EDDSA_BASE_POINT_RATIO (1+EDDSA_USE_SIGMA_ISOGENY)- static void clamp (     uint8_t secret_scalar_ser[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES] ) {@@ -128,22 +123,22 @@      * the decaf base point is on Etwist_d, and when converted it effectively      * picks up a factor of 2 from the isogenies.  So we might start at 2 instead of 1.       */-    for (unsigned int c = EDDSA_BASE_POINT_RATIO; c < COFACTOR; c <<= 1) {+    for (unsigned int c=1; c<CRYPTON_DECAF_448_EDDSA_ENCODE_RATIO; c <<= 1) {         API_NS(scalar_halve)(secret_scalar,secret_scalar);     }          API_NS(point_t) p;     API_NS(precomputed_scalarmul)(p,API_NS(precomputed_base),secret_scalar);     -    API_NS(point_mul_by_cofactor_and_encode_like_eddsa)(pubkey, p);+    API_NS(point_mul_by_ratio_and_encode_like_eddsa)(pubkey, p);              /* Cleanup */     API_NS(scalar_destroy)(secret_scalar);     API_NS(point_destroy)(p);     crypton_decaf_bzero(secret_scalar_ser, sizeof(secret_scalar_ser)); }--void crypton_decaf_ed448_sign (+        +static void crypton_decaf_ed448_sign_internal (     uint8_t signature[CRYPTON_DECAF_EDDSA_448_SIGNATURE_BYTES],     const uint8_t privkey[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES],     const uint8_t pubkey[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES],@@ -191,13 +186,13 @@         /* Scalarmul to create the nonce-point */         API_NS(scalar_t) nonce_scalar_2;         API_NS(scalar_halve)(nonce_scalar_2,nonce_scalar);-        for (unsigned int c = 2*EDDSA_BASE_POINT_RATIO; c < COFACTOR; c <<= 1) {+        for (unsigned int c = 2; c < CRYPTON_DECAF_448_EDDSA_ENCODE_RATIO; c <<= 1) {             API_NS(scalar_halve)(nonce_scalar_2,nonce_scalar_2);         }                  API_NS(point_t) p;         API_NS(precomputed_scalarmul)(p,API_NS(precomputed_base),nonce_scalar_2);-        API_NS(point_mul_by_cofactor_and_encode_like_eddsa)(nonce_point, p);+        API_NS(point_mul_by_ratio_and_encode_like_eddsa)(nonce_point, p);         API_NS(point_destroy)(p);         API_NS(scalar_destroy)(nonce_scalar_2);     }@@ -228,6 +223,26 @@     API_NS(scalar_destroy)(challenge_scalar); } +void crypton_decaf_ed448_sign (+    uint8_t signature[CRYPTON_DECAF_EDDSA_448_SIGNATURE_BYTES],+    const uint8_t privkey[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES],+    const uint8_t pubkey[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES],+    const uint8_t *message,+    size_t message_len,+    uint8_t prehashed,+    const uint8_t *context,+    uint8_t context_len+) {+    /* rederivation already performed in Crypto.PubKey.Ed448.sign+    uint8_t rederived_pubkey[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES];+    crypton_decaf_ed448_derive_public_key(rederived_pubkey, privkey);+    if (CRYPTON_DECAF_TRUE != crypton_decaf_memeq(rederived_pubkey, pubkey, sizeof(rederived_pubkey))) {+        abort();+    }+    */+    crypton_decaf_ed448_sign_internal(signature,privkey,/*rederived_*/pubkey,message,+        message_len,prehashed,context,context_len);+}  void crypton_decaf_ed448_sign_prehash (     uint8_t signature[CRYPTON_DECAF_EDDSA_448_SIGNATURE_BYTES],@@ -237,7 +252,7 @@     const uint8_t *context,     uint8_t context_len ) {-    uint8_t hash_output[64]; /* MAGIC but true for all existing schemes */+    uint8_t hash_output[EDDSA_PREHASH_BYTES];     {         crypton_decaf_ed448_prehash_ctx_t hash_too;         memcpy(hash_too,hash,sizeof(hash_too));@@ -245,10 +260,78 @@         hash_destroy(hash_too);     } -    crypton_decaf_ed448_sign(signature,privkey,pubkey,hash_output,sizeof(hash_output),1,context,context_len);+    uint8_t rederived_pubkey[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES];+    crypton_decaf_ed448_derive_public_key(rederived_pubkey, privkey);+    if (CRYPTON_DECAF_TRUE != crypton_decaf_memeq(rederived_pubkey, pubkey, sizeof(rederived_pubkey))) {+        abort();+    }++    crypton_decaf_ed448_sign_internal(signature,privkey,rederived_pubkey,hash_output,+        sizeof(hash_output),1,context,context_len);     crypton_decaf_bzero(hash_output,sizeof(hash_output)); } +void crypton_decaf_ed448_derive_keypair (+    crypton_decaf_eddsa_448_keypair_t keypair,+    const uint8_t privkey[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES]+) {+    memcpy(keypair->privkey, privkey, CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES);+    crypton_decaf_ed448_derive_public_key(keypair->pubkey, keypair->privkey);+}++void crypton_decaf_ed448_keypair_extract_public_key (+    uint8_t pubkey[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES],+    const crypton_decaf_eddsa_448_keypair_t keypair+) {+    memcpy(pubkey,keypair->pubkey,CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES);+}++void crypton_decaf_ed448_keypair_extract_private_key (+    uint8_t privkey[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES],+    const crypton_decaf_eddsa_448_keypair_t keypair+) {+    memcpy(privkey,keypair->privkey,CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES);+}++void crypton_decaf_ed448_keypair_destroy (+    crypton_decaf_eddsa_448_keypair_t keypair+) {+    crypton_decaf_bzero(keypair, sizeof(crypton_decaf_eddsa_448_keypair_t));+}++void crypton_decaf_ed448_keypair_sign (+    uint8_t signature[CRYPTON_DECAF_EDDSA_448_SIGNATURE_BYTES],+    const crypton_decaf_eddsa_448_keypair_t keypair,+    const uint8_t *message,+    size_t message_len,+    uint8_t prehashed,+    const uint8_t *context,+    uint8_t context_len+) {+    crypton_decaf_ed448_sign_internal(signature,keypair->privkey,keypair->pubkey,message,+        message_len,prehashed,context,context_len);+}++void crypton_decaf_ed448_keypair_sign_prehash (+    uint8_t signature[CRYPTON_DECAF_EDDSA_448_SIGNATURE_BYTES],+    const crypton_decaf_eddsa_448_keypair_t keypair,+    const crypton_decaf_ed448_prehash_ctx_t hash,+    const uint8_t *context,+    uint8_t context_len+) {+    uint8_t hash_output[EDDSA_PREHASH_BYTES];+    {+        crypton_decaf_ed448_prehash_ctx_t hash_too;+        memcpy(hash_too,hash,sizeof(hash_too));+        hash_final(hash_too,hash_output,sizeof(hash_output));+        hash_destroy(hash_too);+    }++    crypton_decaf_ed448_sign_internal(signature,keypair->privkey,keypair->pubkey,hash_output,+        sizeof(hash_output),1,context,context_len);+    crypton_decaf_bzero(hash_output,sizeof(hash_output));+}+ crypton_decaf_error_t crypton_decaf_ed448_verify (     const uint8_t signature[CRYPTON_DECAF_EDDSA_448_SIGNATURE_BYTES],     const uint8_t pubkey[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES],@@ -259,10 +342,10 @@     uint8_t context_len ) {      API_NS(point_t) pk_point, r_point;-    crypton_decaf_error_t error = API_NS(point_decode_like_eddsa_and_ignore_cofactor)(pk_point,pubkey);+    crypton_decaf_error_t error = API_NS(point_decode_like_eddsa_and_mul_by_ratio)(pk_point,pubkey);     if (CRYPTON_DECAF_SUCCESS != error) { return error; }     -    error = API_NS(point_decode_like_eddsa_and_ignore_cofactor)(r_point,signature);+    error = API_NS(point_decode_like_eddsa_and_mul_by_ratio)(r_point,signature);     if (CRYPTON_DECAF_SUCCESS != error) { return error; }          API_NS(scalar_t) challenge_scalar;@@ -282,16 +365,27 @@     API_NS(scalar_sub)(challenge_scalar, API_NS(scalar_zero), challenge_scalar);          API_NS(scalar_t) response_scalar;-    API_NS(scalar_decode_long)(+    error = API_NS(scalar_decode)(         response_scalar,-        &signature[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES],-        CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES+        &signature[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES]     );-#if EDDSA_BASE_POINT_RATIO == 2-    API_NS(scalar_add)(response_scalar,response_scalar,response_scalar);+    if (CRYPTON_DECAF_SUCCESS != error) { return error; }++#if CRYPTON_DECAF_448_SCALAR_BYTES < CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES+    for (unsigned i = CRYPTON_DECAF_448_SCALAR_BYTES;+         i < CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES;+         i++) {+        if (signature[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES+i] != 0x00) {+            return CRYPTON_DECAF_FAILURE;+        }+    } #endif     +    for (unsigned c=1; c<CRYPTON_DECAF_448_EDDSA_DECODE_RATIO; c<<=1) {+        API_NS(scalar_add)(response_scalar,response_scalar,response_scalar);+    }     +         /* pk_point = -c(x(P)) + (cx + k)G = kG */     API_NS(base_double_scalarmul_non_secret)(         pk_point,@@ -312,7 +406,7 @@ ) {     crypton_decaf_error_t ret;     -    uint8_t hash_output[64]; /* MAGIC but true for all existing schemes */+    uint8_t hash_output[EDDSA_PREHASH_BYTES];     {         crypton_decaf_ed448_prehash_ctx_t hash_too;         memcpy(hash_too,hash,sizeof(hash_too));
cbits/decaf/ed448goldilocks/scalar.c view
@@ -48,15 +48,15 @@     unsigned int i;     for (i=0; i<SCALAR_LIMBS; i++) {         chain = (chain + accum[i]) - sub->limb[i];-        out->limb[i] = chain;+        out->limb[i] = (crypton_decaf_word_t)chain;         chain >>= WBITS;     }-    crypton_decaf_word_t borrow = chain+extra; /* = 0 or -1 */+    crypton_decaf_word_t borrow = (crypton_decaf_word_t)chain+extra; /* = 0 or -1 */          chain = 0;     for (i=0; i<SCALAR_LIMBS; i++) {         chain = (chain + out->limb[i]) + (p->limb[i] & borrow);-        out->limb[i] = chain;+        out->limb[i] = (crypton_decaf_word_t)chain;         chain >>= WBITS;     } }@@ -77,22 +77,22 @@         crypton_decaf_dword_t chain = 0;         for (j=0; j<SCALAR_LIMBS; j++) {             chain += ((crypton_decaf_dword_t)mand)*mier[j] + accum[j];-            accum[j] = chain;+            accum[j] = (crypton_decaf_word_t)chain;             chain >>= WBITS;         }-        accum[j] = chain;+        accum[j] = (crypton_decaf_word_t)chain;                  mand = accum[0] * MONTGOMERY_FACTOR;         chain = 0;         mier = sc_p->limb;         for (j=0; j<SCALAR_LIMBS; j++) {             chain += (crypton_decaf_dword_t)mand*mier[j] + accum[j];-            if (j) accum[j-1] = chain;+            if (j) accum[j-1] = (crypton_decaf_word_t)chain;             chain >>= WBITS;         }         chain += accum[j];         chain += hi_carry;-        accum[j-1] = chain;+        accum[j-1] = (crypton_decaf_word_t)chain;         hi_carry = chain >> WBITS;     }     @@ -121,7 +121,7 @@      * Sliding window is fine here because the modulus isn't secret.      */     const int SCALAR_WINDOW_BITS = 3;-    scalar_t precmp[1<<SCALAR_WINDOW_BITS];+    scalar_t precmp[1<<3];  // Rewritten from SCALAR_WINDOW_BITS for windows compatibility     const int LAST = (1<<SCALAR_WINDOW_BITS)-1;      /* Precompute precmp = [a^1,a^3,...] */@@ -190,10 +190,10 @@     unsigned int i;     for (i=0; i<SCALAR_LIMBS; i++) {         chain = (chain + a->limb[i]) + b->limb[i];-        out->limb[i] = chain;+        out->limb[i] = (crypton_decaf_word_t)chain;         chain >>= WBITS;     }-    sc_subx(out, out->limb, sc_p, sc_p, chain);+    sc_subx(out, out->limb, sc_p, sc_p, (crypton_decaf_word_t)chain); }  void@@ -204,7 +204,7 @@     memset(out,0,sizeof(scalar_t));     unsigned int i = 0;     for (; i<sizeof(uint64_t)/sizeof(crypton_decaf_word_t); i++) {-        out->limb[i] = w;+        out->limb[i] = (crypton_decaf_word_t)w; #if CRYPTON_DECAF_WORD_BITS < 64         w >>= 8*sizeof(crypton_decaf_word_t); #endif@@ -227,7 +227,7 @@ static CRYPTON_DECAF_INLINE void scalar_decode_short (     scalar_t s,     const unsigned char *ser,-    unsigned int nbytes+    size_t nbytes ) {     unsigned int i,j,k=0;     for (i=0; i<SCALAR_LIMBS; i++) {@@ -253,7 +253,7 @@          API_NS(scalar_mul)(s,s,API_NS(scalar_one)); /* ham-handed reduce */     -    return crypton_decaf_succeed_if(~word_is_zero(accum));+    return crypton_decaf_succeed_if(~word_is_zero((crypton_decaf_word_t)accum)); }  void API_NS(scalar_destroy) (@@ -325,17 +325,17 @@     scalar_t out,     const scalar_t a ) {-    crypton_decaf_word_t mask = -(a->limb[0] & 1);+    crypton_decaf_word_t mask = bit_to_mask((a->limb[0]) & 1);     crypton_decaf_dword_t chain = 0;     unsigned int i;     for (i=0; i<SCALAR_LIMBS; i++) {         chain = (chain + a->limb[i]) + (sc_p->limb[i] & mask);-        out->limb[i] = chain;+        out->limb[i] = (crypton_decaf_word_t)chain;         chain >>= CRYPTON_DECAF_WORD_BITS;     }     for (i=0; i<SCALAR_LIMBS-1; i++) {         out->limb[i] = out->limb[i]>>1 | out->limb[i+1]<<(WBITS-1);     }-    out->limb[i] = out->limb[i]>>1 | chain<<(WBITS-1);+    out->limb[i] = out->limb[i]>>1 | (crypton_decaf_word_t)(chain<<(WBITS-1)); }
cbits/decaf/include/arch_32/arch_intrinsics.h view
@@ -7,6 +7,11 @@  #define ARCH_WORD_BITS 32 +#if defined _MSC_VER+#define __attribute(x)+#define __inline__ __inline+#endif // MSVC+ static __inline__ __attribute((always_inline,unused)) uint32_t word_is_zero(uint32_t a) {     /* let's hope the compiler isn't clever enough to optimize this. */
cbits/decaf/include/decaf/common.h view
@@ -13,7 +13,9 @@ #define __CRYPTON_DECAF_COMMON_H__ 1  #include <stdint.h>+#if defined (__GNUC__)  // File only exists for GNU compilers #include <sys/types.h>+#endif  #ifdef __cplusplus extern "C" {@@ -21,14 +23,35 @@  /* Goldilocks' build flags default to hidden and stripping executables. */ /** @cond internal */-#if defined(DOXYGEN) && !defined(__attribute__)-#define __attribute__((x))+#if DOXYGEN || defined(__attribute__)+#define __attribute__(x)+#define NOINLINE #endif++/* Aliasing MSVC preprocessing to GNU preprocessing */+#if defined _MSC_VER+#   define __attribute__(x)        // Turn off attribute code+#   define __attribute(x)+#   define __restrict__ __restrict  // Use MSVC restrict code+#   if defined _DLL+#       define CRYPTON_DECAF_API_VIS __declspec(dllexport)  // MSVC for visibility+#   else+#       define CRYPTON_DECAF_API_VIS __declspec(dllimport)+#   endif++//#   define CRYPTON_DECAF_NOINLINE __declspec(noinline) // MSVC for noinline+//#   define CRYPTON_DECAF_INLINE __forceinline // MSVC for always inline+//#   define CRYPTON_DECAF_WARN_UNUSED _Check_return_    +#else // MSVC #define CRYPTON_DECAF_API_VIS __attribute__((visibility("default")))+#define CRYPTON_DECAF_API_IMPORT+#endif++// The following are disabled for MSVC #define CRYPTON_DECAF_NOINLINE  __attribute__((noinline))-#define CRYPTON_DECAF_WARN_UNUSED __attribute__((warn_unused_result))-#define CRYPTON_DECAF_NONNULL __attribute__((nonnull)) #define CRYPTON_DECAF_INLINE inline __attribute__((always_inline,unused))+#define CRYPTON_DECAF_WARN_UNUSED __attribute__((warn_unused_result))+#define CRYPTON_DECAF_NONNULL __attribute__((nonnull))   // Cribbed from libnotmuch #if defined (__clang_major__) && __clang_major__ >= 3 \     || defined (__GNUC__) && __GNUC__ >= 5 \@@ -70,8 +93,25 @@ #error "Only supporting CRYPTON_DECAF_WORD_BITS = 32 or 64 for now" #endif     -/** CRYPTON_DECAF_TRUE = -1 so that CRYPTON_DECAF_TRUE & x = x */-static const crypton_decaf_bool_t CRYPTON_DECAF_TRUE = -(crypton_decaf_bool_t)1;+/* MSCV compiler doesn't like the trick to have -1 assigned to an unsigned int to+ * set it to all ones, so do it openly */+#if CRYPTON_DECAF_WORD_BITS == 64+/** CRYPTON_DECAF_TRUE = all ones so that CRYPTON_DECAF_TRUE & x = x */+static const crypton_decaf_bool_t CRYPTON_DECAF_TRUE = (crypton_decaf_bool_t)0xFFFFFFFFFFFFFFFF;+/** CRYPTON_DECAF_WORD_ALL_SET : all ones */+static const crypton_decaf_word_t CRYPTON_DECAF_WORD_ALL_SET = (crypton_decaf_word_t)0xFFFFFFFFFFFFFFFF;+/** CRYPTON_DECAF_WORD_ALL_UNSET : all zeros */+static const crypton_decaf_word_t CRYPTON_DECAF_WORD_ALL_UNSET = (crypton_decaf_word_t)0x0;+#elif CRYPTON_DECAF_WORD_BITS == 32         /**< The number of bits in a word */+/** CRYPTON_DECAF_TRUE = all ones so that CRYPTON_DECAF_TRUE & x = x */+static const crypton_decaf_bool_t CRYPTON_DECAF_TRUE = (crypton_decaf_bool_t)0xFFFFFFFF;+/** CRYPTON_DECAF_WORD_ALL_SET : all ones */+static const crypton_decaf_word_t CRYPTON_DECAF_WORD_ALL_SET = (crypton_decaf_word_t)0xFFFFFFFF;+/** CRYPTON_DECAF_WORD_ALL_UNSET : all zeros */+static const crypton_decaf_word_t CRYPTON_DECAF_WORD_ALL_UNSET = (crypton_decaf_word_t)0x0;+#else+#error "Only supporting CRYPTON_DECAF_WORD_BITS = 32 or 64 for now"+#endif  /** CRYPTON_DECAF_FALSE = 0 so that CRYPTON_DECAF_FALSE & x = 0 */ static const crypton_decaf_bool_t CRYPTON_DECAF_FALSE = 0;@@ -92,22 +132,23 @@ /** Return CRYPTON_DECAF_TRUE iff x == CRYPTON_DECAF_SUCCESS */ static CRYPTON_DECAF_INLINE crypton_decaf_bool_t crypton_decaf_successful(crypton_decaf_error_t e) {-    crypton_decaf_dword_t w = ((crypton_decaf_word_t)e) ^  ((crypton_decaf_word_t)CRYPTON_DECAF_SUCCESS);+    crypton_decaf_word_t succ = CRYPTON_DECAF_SUCCESS;+    crypton_decaf_dword_t w = ((crypton_decaf_word_t)e) ^  succ;     return (w-1)>>CRYPTON_DECAF_WORD_BITS; }      /** Overwrite data with zeros.  Uses memset_s if available. */-void crypton_decaf_bzero (+void CRYPTON_DECAF_API_VIS crypton_decaf_bzero (     void *data,     size_t size-) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_API_VIS;+) CRYPTON_DECAF_NONNULL;  /** Compare two buffers, returning CRYPTON_DECAF_TRUE if they are equal. */-crypton_decaf_bool_t crypton_decaf_memeq (+crypton_decaf_bool_t CRYPTON_DECAF_API_VIS crypton_decaf_memeq (     const void *data1,     const void *data2,     size_t size-) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_API_VIS;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_WARN_UNUSED;      #ifdef __cplusplus } /* extern "C" */
cbits/decaf/include/decaf/ed448.h view
@@ -33,14 +33,45 @@ #define CRYPTON_DECAF_EDDSA_448_SIGNATURE_BYTES (CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES + CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES)  /** Does EdDSA support non-contextual signatures? */+#if defined _MSC_VER  /* Different syntax for exposing API */ #define CRYPTON_DECAF_EDDSA_448_SUPPORTS_CONTEXTLESS_SIGS 0 -/** Prehash context renaming macros. */+#else+#define CRYPTON_DECAF_EDDSA_448_SUPPORTS_CONTEXTLESS_SIGS 0++#endif++/** Prehash context (raw), because each EdDSA instance has a different prehash. */ #define crypton_decaf_ed448_prehash_ctx_s   crypton_decaf_shake256_ctx_s++/** Prehash context, array[1] form. */ #define crypton_decaf_ed448_prehash_ctx_t   crypton_decaf_shake256_ctx_t+    +/** Prehash update. */ #define crypton_decaf_ed448_prehash_update  crypton_decaf_shake256_update+    +/** Prehash destroy. */ #define crypton_decaf_ed448_prehash_destroy crypton_decaf_shake256_destroy +/** EdDSA encoding ratio. */+#define CRYPTON_DECAF_448_EDDSA_ENCODE_RATIO 4++/** EdDSA decoding ratio. */+#define CRYPTON_DECAF_448_EDDSA_DECODE_RATIO (4 / 4)+    +#ifndef CRYPTON_DECAF_EDDSA_NON_KEYPAIR_API_IS_DEPRECATED+/** If 1, add deprecation attribute to non-keypair API functions. Now deprecated. */+#define CRYPTON_DECAF_EDDSA_NON_KEYPAIR_API_IS_DEPRECATED 1+#endif++/** @cond internal */+/** @brief Scheduled EdDSA keypair */+typedef struct crypton_decaf_eddsa_448_keypair_s {+    uint8_t privkey[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES];+    uint8_t pubkey[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES];+}  crypton_decaf_eddsa_448_keypair_s, crypton_decaf_eddsa_448_keypair_t[1];+/** @endcond */+ /**  * @brief EdDSA key generation.  This function uses a different (non-Decaf)  * encoding.@@ -48,14 +79,60 @@  * @param [out] pubkey The public key.  * @param [in] privkey The private key.  */    -void crypton_decaf_ed448_derive_public_key (+void CRYPTON_DECAF_API_VIS crypton_decaf_ed448_derive_public_key (     uint8_t pubkey[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES],     const uint8_t privkey[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES]-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**- * @brief EdDSA signing.+ * @brief EdDSA keypair scheduling.  This is to add a safer version of the signing algorithm,+ * where it is harder to use the wrong pubkey for your private key..  *+ * @param [out] keypair The scheduled keypair.+ * @param [in] privkey The private key.+ */    +void CRYPTON_DECAF_API_VIS crypton_decaf_ed448_derive_keypair (+    crypton_decaf_eddsa_448_keypair_t keypair,+    const uint8_t privkey[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES]+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;++/**+ * @brief Extract the public key from an EdDSA keypair.+ *+ * @param [out] pubkey The public key.+ * @param [in] keypair The keypair.+ */    +void CRYPTON_DECAF_API_VIS crypton_decaf_ed448_keypair_extract_public_key (+    uint8_t pubkey[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES],+    const crypton_decaf_eddsa_448_keypair_t keypair+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;++/**+ * @brief Extract the private key from an EdDSA keypair.+ *+ * @param [out] privkey The private key.+ * @param [in] keypair The keypair.+ */    +void CRYPTON_DECAF_API_VIS crypton_decaf_ed448_keypair_extract_private_key (+    uint8_t privkey[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES],+    const crypton_decaf_eddsa_448_keypair_t keypair+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;++/**+ * @brief EdDSA keypair destructor.+ * @param [in] pubkey The keypair.+ */    +void CRYPTON_DECAF_API_VIS crypton_decaf_ed448_keypair_destroy (+    crypton_decaf_eddsa_448_keypair_t keypair+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;++/**+ * @brief EdDSA signing.  However, this API is deprecated because it isn't safe: if the wrong+ * public key is passed, it would reveal the private key.  Instead, this function checks that+ * the public key is correct, and otherwise aborts.+ *+ * @deprecated Use CRYPTON_DECAF_API_VIS crypton_decaf_ed448_keypair_sign instead.+ *  * @param [out] signature The signature.  * @param [in] privkey The private key.  * @param [in] pubkey The public key.@@ -70,7 +147,7 @@  * safe.  The C++ wrapper is designed to make it harder to screw this up, but this C code gives  * you no seat belt.  */  -void crypton_decaf_ed448_sign (+void CRYPTON_DECAF_API_VIS crypton_decaf_ed448_sign (     uint8_t signature[CRYPTON_DECAF_EDDSA_448_SIGNATURE_BYTES],     const uint8_t privkey[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES],     const uint8_t pubkey[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES],@@ -79,40 +156,85 @@     uint8_t prehashed,     const uint8_t *context,     uint8_t context_len-) CRYPTON_DECAF_API_VIS __attribute__((nonnull(1,2,3))) CRYPTON_DECAF_NOINLINE;+) __attribute__((nonnull(1,2,3))) CRYPTON_DECAF_NOINLINE+#if CRYPTON_DECAF_EDDSA_NON_KEYPAIR_API_IS_DEPRECATED+  CRYPTON_DECAF_DEPRECATED("Passing the pubkey and privkey separately is unsafe, use crypton_decaf_ed448_keypair_sign")+#endif+;  /**- * @brief EdDSA signing with prehash.+ * @brief EdDSA signing with prehash.  However, this API is deprecated because it isn't safe: if the wrong+ * public key is passed, it would reveal the private key.  Instead, this function checks that+ * the public key is correct, and otherwise aborts.  *+ * @deprecated Use CRYPTON_DECAF_API_VIS crypton_decaf_ed448_keypair_sign_prehash instead.+ *  * @param [out] signature The signature.  * @param [in] privkey The private key.  * @param [in] pubkey The public key.  * @param [in] hash The hash of the message.  This object will not be modified by the call.  * @param [in] context A "context" for this signature of up to 255 bytes.  Must be the same as what was used for the prehash.  * @param [in] context_len Length of the context.- *- * @warning For Ed25519, it is unsafe to use the same key for both prehashed and non-prehashed- * messages, at least without some very careful protocol-level disambiguation.  For Ed448 it is- * safe.  The C++ wrapper is designed to make it harder to screw this up, but this C code gives- * you no seat belt.  */  -void crypton_decaf_ed448_sign_prehash (+void CRYPTON_DECAF_API_VIS crypton_decaf_ed448_sign_prehash (     uint8_t signature[CRYPTON_DECAF_EDDSA_448_SIGNATURE_BYTES],     const uint8_t privkey[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES],     const uint8_t pubkey[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES],     const crypton_decaf_ed448_prehash_ctx_t hash,     const uint8_t *context,     uint8_t context_len-) CRYPTON_DECAF_API_VIS __attribute__((nonnull(1,2,3,4))) CRYPTON_DECAF_NOINLINE;+) __attribute__((nonnull(1,2,3,4))) CRYPTON_DECAF_NOINLINE+#if CRYPTON_DECAF_EDDSA_NON_KEYPAIR_API_IS_DEPRECATED+  CRYPTON_DECAF_DEPRECATED("Passing the pubkey and privkey separately is unsafe, use crypton_decaf_ed448_keypair_sign_prehash")+#endif+;++/**+ * @brief EdDSA signing.+ *+ * @param [out] signature The signature.+ * @param [in] keypair The private and public key.+ * @param [in] message The message to sign.+ * @param [in] message_len The length of the message.+ * @param [in] prehashed Nonzero if the message is actually the hash of something you want to sign.+ * @param [in] context A "context" for this signature of up to 255 bytes.+ * @param [in] context_len Length of the context.+ */  +void CRYPTON_DECAF_API_VIS crypton_decaf_ed448_keypair_sign (+    uint8_t signature[CRYPTON_DECAF_EDDSA_448_SIGNATURE_BYTES],+    const crypton_decaf_eddsa_448_keypair_t keypair,+    const uint8_t *message,+    size_t message_len,+    uint8_t prehashed,+    const uint8_t *context,+    uint8_t context_len+) __attribute__((nonnull(1,2,3))) CRYPTON_DECAF_NOINLINE;++/**+ * @brief EdDSA signing with prehash.+ *+ * @param [out] signature The signature.+ * @param [in] keypair The private and public key.+ * @param [in] hash The hash of the message.  This object will not be modified by the call.+ * @param [in] context A "context" for this signature of up to 255 bytes.  Must be the same as what was used for the prehash.+ * @param [in] context_len Length of the context.+ */  +void CRYPTON_DECAF_API_VIS crypton_decaf_ed448_keypair_sign_prehash (+    uint8_t signature[CRYPTON_DECAF_EDDSA_448_SIGNATURE_BYTES],+    const crypton_decaf_eddsa_448_keypair_t keypair,+    const crypton_decaf_ed448_prehash_ctx_t hash,+    const uint8_t *context,+    uint8_t context_len+) __attribute__((nonnull(1,2,3,4))) CRYPTON_DECAF_NOINLINE;      /**  * @brief Prehash initialization, with contexts if supported.  *  * @param [out] hash The hash object to be initialized.  */-void crypton_decaf_ed448_prehash_init (+void CRYPTON_DECAF_API_VIS crypton_decaf_ed448_prehash_init (     crypton_decaf_ed448_prehash_ctx_t hash-) CRYPTON_DECAF_API_VIS __attribute__((nonnull(1))) CRYPTON_DECAF_NOINLINE;+) __attribute__((nonnull(1))) CRYPTON_DECAF_NOINLINE;  /**  * @brief EdDSA signature verification.@@ -132,7 +254,7 @@  * safe.  The C++ wrapper is designed to make it harder to screw this up, but this C code gives  * you no seat belt.  */-crypton_decaf_error_t crypton_decaf_ed448_verify (+crypton_decaf_error_t CRYPTON_DECAF_API_VIS crypton_decaf_ed448_verify (     const uint8_t signature[CRYPTON_DECAF_EDDSA_448_SIGNATURE_BYTES],     const uint8_t pubkey[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES],     const uint8_t *message,@@ -140,7 +262,7 @@     uint8_t prehashed,     const uint8_t *context,     uint8_t context_len-) CRYPTON_DECAF_API_VIS __attribute__((nonnull(1,2))) CRYPTON_DECAF_NOINLINE;+) __attribute__((nonnull(1,2))) CRYPTON_DECAF_NOINLINE;  /**  * @brief EdDSA signature verification.@@ -158,38 +280,56 @@  * safe.  The C++ wrapper is designed to make it harder to screw this up, but this C code gives  * you no seat belt.  */-crypton_decaf_error_t crypton_decaf_ed448_verify_prehash (+crypton_decaf_error_t CRYPTON_DECAF_API_VIS crypton_decaf_ed448_verify_prehash (     const uint8_t signature[CRYPTON_DECAF_EDDSA_448_SIGNATURE_BYTES],     const uint8_t pubkey[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES],     const crypton_decaf_ed448_prehash_ctx_t hash,     const uint8_t *context,     uint8_t context_len-) CRYPTON_DECAF_API_VIS __attribute__((nonnull(1,2))) CRYPTON_DECAF_NOINLINE;+) __attribute__((nonnull(1,2))) CRYPTON_DECAF_NOINLINE;  /**  * @brief EdDSA point encoding.  Used internally, exposed externally.- * Multiplies the point by the current cofactor first.+ * Multiplies by CRYPTON_DECAF_448_EDDSA_ENCODE_RATIO first.  *+ * The multiplication is required because the EdDSA encoding represents+ * the cofactor information, but the Decaf encoding ignores it (which+ * is the whole point).  So if you decode from EdDSA and re-encode to+ * EdDSA, the cofactor info must get cleared, because the intermediate+ * representation doesn't track it.+ *+ * The way libdecaf handles this is to multiply by+ * CRYPTON_DECAF_448_EDDSA_DECODE_RATIO when decoding, and by+ * CRYPTON_DECAF_448_EDDSA_ENCODE_RATIO when encoding.  The product of these+ * ratios is always exactly the cofactor 4, so the cofactor+ * ends up cleared one way or another.  But exactly how that shakes+ * out depends on the base points specified in RFC 8032.+ *+ * The upshot is that if you pass the Decaf/Ristretto base point to+ * this function, you will get CRYPTON_DECAF_448_EDDSA_ENCODE_RATIO times the+ * EdDSA base point.+ *  * @param [out] enc The encoded point.  * @param [in] p The point.  */       -void crypton_decaf_448_point_mul_by_cofactor_and_encode_like_eddsa (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_mul_by_ratio_and_encode_like_eddsa (     uint8_t enc[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES],     const crypton_decaf_448_point_t p-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**- * @brief EdDSA point decoding.  Remember that while points on the- * EdDSA curves have cofactor information, Decaf ignores (quotients- * out) all cofactor information.+ * @brief EdDSA point decoding.  Multiplies by CRYPTON_DECAF_448_EDDSA_DECODE_RATIO,+ * and ignores cofactor information.  *+ * See notes on crypton_decaf_448_point_mul_by_ratio_and_encode_like_eddsa+ *  * @param [out] enc The encoded point.  * @param [in] p The point.  */       -crypton_decaf_error_t crypton_decaf_448_point_decode_like_eddsa_and_ignore_cofactor (+crypton_decaf_error_t CRYPTON_DECAF_API_VIS crypton_decaf_448_point_decode_like_eddsa_and_mul_by_ratio (     crypton_decaf_448_point_t p,     const uint8_t enc[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES]-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief EdDSA to ECDH public key conversion@@ -202,10 +342,10 @@  * @param[out] x The ECDH public key as in RFC7748(point on Montgomery curve)  * @param[in] ed The EdDSA public key(point on Edwards curve)  */-void crypton_decaf_ed448_convert_public_key_to_x448 (+void CRYPTON_DECAF_API_VIS crypton_decaf_ed448_convert_public_key_to_x448 (     uint8_t x[CRYPTON_DECAF_X448_PUBLIC_BYTES],     const uint8_t ed[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES]-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief EdDSA to ECDH private key conversion@@ -215,10 +355,10 @@  * @param[out] x The ECDH private key as in RFC7748  * @param[in] ed The EdDSA private key  */-void crypton_decaf_ed448_convert_private_key_to_x448 (+void CRYPTON_DECAF_API_VIS crypton_decaf_ed448_convert_private_key_to_x448 (     uint8_t x[CRYPTON_DECAF_X448_PRIVATE_BYTES],     const uint8_t ed[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES]-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  #ifdef __cplusplus } /* extern "C" */
cbits/decaf/include/decaf/point_448.h view
@@ -34,7 +34,7 @@ /** @brief Galois field element internal structure */ typedef struct crypton_gf_448_s {     crypton_decaf_word_t limb[512/CRYPTON_DECAF_WORD_BITS];-} __attribute__((aligned(16))) crypton_gf_448_s, crypton_gf_448_t[1];+} __attribute__((aligned(32))) crypton_gf_448_s, crypton_gf_448_t[1]; #endif /* __CRYPTON_DECAF_448_GF_DEFINED__ */ /** @endcond */ @@ -52,16 +52,22 @@ /** Number of bits in the "which" field of an elligator inverse */ #define CRYPTON_DECAF_448_INVERT_ELLIGATOR_WHICH_BITS 3 +/** The cofactor the curve would have, if we hadn't removed it */+#define CRYPTON_DECAF_448_REMOVED_COFACTOR 4++/** X448 encoding ratio. */+#define CRYPTON_DECAF_X448_ENCODE_RATIO 2+ /** Number of bytes in an x448 public key */ #define CRYPTON_DECAF_X448_PUBLIC_BYTES 56  /** Number of bytes in an x448 private key */ #define CRYPTON_DECAF_X448_PRIVATE_BYTES 56 -/** Twisted Edwards extended homogeneous coordinates */+/** Representation of a point on the elliptic curve. */ typedef struct crypton_decaf_448_point_s {     /** @cond internal */-    crypton_gf_448_t x,y,z,t;+    crypton_gf_448_t x,y,z,t; /* Twisted extended homogeneous coordinates */     /** @endcond */ } crypton_decaf_448_point_t[1]; @@ -72,30 +78,51 @@ typedef struct crypton_decaf_448_precomputed_s crypton_decaf_448_precomputed_s;   /** Size and alignment of precomputed point tables. */-extern const size_t crypton_decaf_448_sizeof_precomputed_s CRYPTON_DECAF_API_VIS, crypton_decaf_448_alignof_precomputed_s CRYPTON_DECAF_API_VIS;+CRYPTON_DECAF_API_VIS extern const size_t crypton_decaf_448_sizeof_precomputed_s, crypton_decaf_448_alignof_precomputed_s; -/** Scalar is stored packed, because we don't need the speed. */+/** Representation of an element of the scalar field. */ typedef struct crypton_decaf_448_scalar_s {     /** @cond internal */     crypton_decaf_word_t limb[CRYPTON_DECAF_448_SCALAR_LIMBS];     /** @endcond */ } crypton_decaf_448_scalar_t[1]; -/** A scalar equal to 1. */-extern const crypton_decaf_448_scalar_t crypton_decaf_448_scalar_one CRYPTON_DECAF_API_VIS;+#if defined _MSC_VER -/** A scalar equal to 0. */-extern const crypton_decaf_448_scalar_t crypton_decaf_448_scalar_zero CRYPTON_DECAF_API_VIS;+/** The scalar 1. */+extern const crypton_decaf_448_scalar_t CRYPTON_DECAF_API_VIS crypton_decaf_448_scalar_one; -/** The identity point on the curve. */-extern const crypton_decaf_448_point_t crypton_decaf_448_point_identity CRYPTON_DECAF_API_VIS;+/** The scalar 0. */+extern const crypton_decaf_448_scalar_t CRYPTON_DECAF_API_VIS crypton_decaf_448_scalar_zero; -/** An arbitrarily chosen base point on the curve. */-extern const crypton_decaf_448_point_t crypton_decaf_448_point_base CRYPTON_DECAF_API_VIS;+/** The identity (zero) point on the curve. */+extern const crypton_decaf_448_point_t CRYPTON_DECAF_API_VIS crypton_decaf_448_point_identity; -/** Precomputed table for the base point on the curve. */-extern const struct crypton_decaf_448_precomputed_s *crypton_decaf_448_precomputed_base CRYPTON_DECAF_API_VIS;+/** An arbitrarily-chosen base point on the curve. */+extern const crypton_decaf_448_point_t CRYPTON_DECAF_API_VIS crypton_decaf_448_point_base; +/** Precomputed table of multiples of the base point on the curve. */+extern const struct CRYPTON_DECAF_API_VIS crypton_decaf_448_precomputed_s *crypton_decaf_448_precomputed_base;+++#else // _MSC_VER++/** The scalar 1. */+CRYPTON_DECAF_API_VIS extern const crypton_decaf_448_scalar_t crypton_decaf_448_scalar_one;++/** The scalar 0. */+CRYPTON_DECAF_API_VIS extern const crypton_decaf_448_scalar_t crypton_decaf_448_scalar_zero;++/** The identity (zero) point on the curve. */+CRYPTON_DECAF_API_VIS extern const crypton_decaf_448_point_t crypton_decaf_448_point_identity;++/** An arbitrarily-chosen base point on the curve. */+CRYPTON_DECAF_API_VIS extern const crypton_decaf_448_point_t crypton_decaf_448_point_base;++/** Precomputed table of multiples of the base point on the curve. */+CRYPTON_DECAF_API_VIS extern const struct crypton_decaf_448_precomputed_s *crypton_decaf_448_precomputed_base;++#endif // _MSC_VER /**  * @brief Read a scalar from wire format or from bytes.  *@@ -106,10 +133,10 @@  * @retval CRYPTON_DECAF_FAILURE The scalar was greater than the modulus,  * and has been reduced modulo that modulus.  */-crypton_decaf_error_t crypton_decaf_448_scalar_decode (+crypton_decaf_error_t CRYPTON_DECAF_API_VIS crypton_decaf_448_scalar_decode (     crypton_decaf_448_scalar_t out,     const unsigned char ser[CRYPTON_DECAF_448_SCALAR_BYTES]-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Read a scalar from wire format or from bytes.  Reduces mod@@ -119,11 +146,11 @@  * @param [in] ser_len Length of serialized form.  * @param [out] out Deserialized form.  */-void crypton_decaf_448_scalar_decode_long (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_scalar_decode_long (     crypton_decaf_448_scalar_t out,     const unsigned char *ser,     size_t ser_len-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;      /**  * @brief Serialize a scalar to wire format.@@ -131,10 +158,10 @@  * @param [out] ser Serialized form of a scalar.  * @param [in] s Deserialized scalar.  */-void crypton_decaf_448_scalar_encode (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_scalar_encode (     unsigned char ser[CRYPTON_DECAF_448_SCALAR_BYTES],     const crypton_decaf_448_scalar_t s-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE CRYPTON_DECAF_NOINLINE;          /**  * @brief Add two scalars.  The scalars may use the same memory.@@ -142,11 +169,11 @@  * @param [in] b Another scalar.  * @param [out] out a+b.  */-void crypton_decaf_448_scalar_add (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_scalar_add (     crypton_decaf_448_scalar_t out,     const crypton_decaf_448_scalar_t a,     const crypton_decaf_448_scalar_t b-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Compare two scalars.@@ -155,10 +182,10 @@  * @retval CRYPTON_DECAF_TRUE The scalars are equal.  * @retval CRYPTON_DECAF_FALSE The scalars are not equal.  */    -crypton_decaf_bool_t crypton_decaf_448_scalar_eq (+crypton_decaf_bool_t CRYPTON_DECAF_API_VIS crypton_decaf_448_scalar_eq (     const crypton_decaf_448_scalar_t a,     const crypton_decaf_448_scalar_t b-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Subtract two scalars.  The scalars may use the same memory.@@ -166,11 +193,11 @@  * @param [in] b Another scalar.  * @param [out] out a-b.  */  -void crypton_decaf_448_scalar_sub (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_scalar_sub (     crypton_decaf_448_scalar_t out,     const crypton_decaf_448_scalar_t a,     const crypton_decaf_448_scalar_t b-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Multiply two scalars.  The scalars may use the same memory.@@ -178,21 +205,21 @@  * @param [in] b Another scalar.  * @param [out] out a*b.  */  -void crypton_decaf_448_scalar_mul (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_scalar_mul (     crypton_decaf_448_scalar_t out,     const crypton_decaf_448_scalar_t a,     const crypton_decaf_448_scalar_t b-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;          /** * @brief Halve a scalar.  The scalars may use the same memory. * @param [in] a A scalar. * @param [out] out a/2. */-void crypton_decaf_448_scalar_halve (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_scalar_halve (    crypton_decaf_448_scalar_t out,    const crypton_decaf_448_scalar_t a-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Invert a scalar.  When passed zero, return 0.  The input and output may alias.@@ -200,10 +227,10 @@  * @param [out] out 1/a.  * @return CRYPTON_DECAF_SUCCESS The input is nonzero.  */  -crypton_decaf_error_t crypton_decaf_448_scalar_invert (+crypton_decaf_error_t CRYPTON_DECAF_API_VIS crypton_decaf_448_scalar_invert (     crypton_decaf_448_scalar_t out,     const crypton_decaf_448_scalar_t a-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Copy a scalar.  The scalars may use the same memory, in which@@ -223,10 +250,10 @@  * @param [in] a An integer.  * @param [out] out Will become equal to a.  */  -void crypton_decaf_448_scalar_set_unsigned (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_scalar_set_unsigned (     crypton_decaf_448_scalar_t out,     uint64_t a-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL;+) CRYPTON_DECAF_NONNULL;  /**  * @brief Encode a point as a sequence of bytes.@@ -234,10 +261,10 @@  * @param [out] ser The byte representation of the point.  * @param [in] pt The point to encode.  */-void crypton_decaf_448_point_encode (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_encode (     uint8_t ser[CRYPTON_DECAF_448_SER_BYTES],     const crypton_decaf_448_point_t pt-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Decode a point from a sequence of bytes.@@ -253,11 +280,11 @@  * @retval CRYPTON_DECAF_FAILURE The decoding didn't succeed, because  * ser does not represent a point.  */-crypton_decaf_error_t crypton_decaf_448_point_decode (+crypton_decaf_error_t CRYPTON_DECAF_API_VIS crypton_decaf_448_point_decode (     crypton_decaf_448_point_t pt,     const uint8_t ser[CRYPTON_DECAF_448_SER_BYTES],     crypton_decaf_bool_t allow_identity-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Copy a point.  The input and output may alias,@@ -282,10 +309,10 @@  * @retval CRYPTON_DECAF_TRUE The points are equal.  * @retval CRYPTON_DECAF_FALSE The points are not equal.  */-crypton_decaf_bool_t crypton_decaf_448_point_eq (+crypton_decaf_bool_t CRYPTON_DECAF_API_VIS crypton_decaf_448_point_eq (     const crypton_decaf_448_point_t a,     const crypton_decaf_448_point_t b-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Add two points to produce a third point.  The@@ -296,11 +323,11 @@  * @param [in] a An addend.  * @param [in] b An addend.  */-void crypton_decaf_448_point_add (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_add (     crypton_decaf_448_point_t sum,     const crypton_decaf_448_point_t a,     const crypton_decaf_448_point_t b-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL;+) CRYPTON_DECAF_NONNULL;  /**  * @brief Double a point.  Equivalent to@@ -309,10 +336,10 @@  * @param [out] two_a The sum a+a.  * @param [in] a A point.  */-void crypton_decaf_448_point_double (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_double (     crypton_decaf_448_point_t two_a,     const crypton_decaf_448_point_t a-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL;+) CRYPTON_DECAF_NONNULL;  /**  * @brief Subtract two points to produce a third point.  The@@ -323,11 +350,11 @@  * @param [in] a The minuend.  * @param [in] b The subtrahend.  */-void crypton_decaf_448_point_sub (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_sub (     crypton_decaf_448_point_t diff,     const crypton_decaf_448_point_t a,     const crypton_decaf_448_point_t b-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL;+) CRYPTON_DECAF_NONNULL;      /**  * @brief Negate a point to produce another point.  The input@@ -336,10 +363,10 @@  * @param [out] nega The negated input point  * @param [in] a The input point.  */-void crypton_decaf_448_point_negate (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_negate (    crypton_decaf_448_point_t nega,    const crypton_decaf_448_point_t a-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL;+) CRYPTON_DECAF_NONNULL;  /**  * @brief Multiply a base point by a scalar: scaled = scalar*base.@@ -348,11 +375,11 @@  * @param [in] base The point to be scaled.  * @param [in] scalar The scalar to multiply by.  */-void crypton_decaf_448_point_scalarmul (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_scalarmul (     crypton_decaf_448_point_t scaled,     const crypton_decaf_448_point_t base,     const crypton_decaf_448_scalar_t scalar-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Multiply a base point by a scalar: scaled = scalar*base.@@ -371,34 +398,64 @@  * @retval CRYPTON_DECAF_FAILURE The scalarmul didn't succeed, because  * base does not represent a point.  */-crypton_decaf_error_t crypton_decaf_448_direct_scalarmul (+crypton_decaf_error_t CRYPTON_DECAF_API_VIS crypton_decaf_448_direct_scalarmul (     uint8_t scaled[CRYPTON_DECAF_448_SER_BYTES],     const uint8_t base[CRYPTON_DECAF_448_SER_BYTES],     const crypton_decaf_448_scalar_t scalar,     crypton_decaf_bool_t allow_identity,     crypton_decaf_bool_t short_circuit-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NOINLINE;  /**- * @brief RFC 7748 Diffie-Hellman scalarmul.  This function uses a different- * (non-Decaf) encoding.+ * @brief RFC 7748 Diffie-Hellman scalarmul, used to compute shared secrets.+ * This function uses a different (non-Decaf) encoding.  *- * @param [out] scaled The scaled point base*scalar- * @param [in] base The point to be scaled.- * @param [in] scalar The scalar to multiply by.+ * @param [out] shared The shared secret base*scalar+ * @param [in] base The other party's public key, used as the base of the scalarmul.+ * @param [in] scalar The private scalar to multiply by.  *  * @retval CRYPTON_DECAF_SUCCESS The scalarmul succeeded.  * @retval CRYPTON_DECAF_FAILURE The scalarmul didn't succeed, because the base  * point is in a small subgroup.  */-crypton_decaf_error_t crypton_decaf_x448 (-    uint8_t out[CRYPTON_DECAF_X448_PUBLIC_BYTES],+crypton_decaf_error_t CRYPTON_DECAF_API_VIS crypton_decaf_x448 (+    uint8_t shared[CRYPTON_DECAF_X448_PUBLIC_BYTES],     const uint8_t base[CRYPTON_DECAF_X448_PUBLIC_BYTES],     const uint8_t scalar[CRYPTON_DECAF_X448_PRIVATE_BYTES]-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NOINLINE; +/**+ * @brief Multiply a point by CRYPTON_DECAF_X448_ENCODE_RATIO,+ * then encode it like RFC 7748.+ *+ * This function is mainly used internally, but is exported in case+ * it will be useful.+ *+ * The ratio is necessary because the internal representation doesn't+ * track the cofactor information, so on output we must clear the cofactor.+ * This would multiply by the cofactor, but in fact internally libdecaf's+ * points are always even, so it multiplies by half the cofactor instead.+ *+ * As it happens, this aligns with the base point definitions; that is,+ * if you pass the Decaf/Ristretto base point to this function, the result+ * will be CRYPTON_DECAF_X448_ENCODE_RATIO times the X448+ * base point.+ *+ * @param [out] out The scaled and encoded point.+ * @param [in] p The point to be scaled and encoded.+ */+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_mul_by_ratio_and_encode_like_x448 (+    uint8_t out[CRYPTON_DECAF_X448_PUBLIC_BYTES],+    const crypton_decaf_448_point_t p+) CRYPTON_DECAF_NONNULL;+ /** The base point for X448 Diffie-Hellman */-extern const uint8_t crypton_decaf_x448_base_point[CRYPTON_DECAF_X448_PUBLIC_BYTES] CRYPTON_DECAF_API_VIS;+extern const uint8_t+#ifndef DOXYGEN+    /* For some reason Doxygen chokes on this despite the defense in common.h... */+    CRYPTON_DECAF_API_VIS+#endif+    crypton_decaf_x448_base_point[CRYPTON_DECAF_X448_PUBLIC_BYTES];  /**  * @brief RFC 7748 Diffie-Hellman base point scalarmul.  This function uses@@ -407,13 +464,13 @@  * @deprecated Renamed to crypton_decaf_x448_derive_public_key.  * I have no particular timeline for removing this name.  *- * @param [out] scaled The scaled point base*scalar- * @param [in] scalar The scalar to multiply by.+ * @param [out] out The public key base*scalar.+ * @param [in] scalar The private scalar.  */-void crypton_decaf_x448_generate_key (+void CRYPTON_DECAF_API_VIS crypton_decaf_x448_generate_key (     uint8_t out[CRYPTON_DECAF_X448_PUBLIC_BYTES],     const uint8_t scalar[CRYPTON_DECAF_X448_PRIVATE_BYTES]-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE CRYPTON_DECAF_DEPRECATED("Renamed to crypton_decaf_x448_derive_public_key");+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE CRYPTON_DECAF_DEPRECATED("Renamed to crypton_decaf_x448_derive_public_key");      /**  * @brief RFC 7748 Diffie-Hellman base point scalarmul.  This function uses@@ -422,13 +479,13 @@  * Does exactly the same thing as crypton_decaf_x448_generate_key,  * but has a better name.  *- * @param [out] scaled The scaled point base*scalar- * @param [in] scalar The scalar to multiply by.+ * @param [out] out The public key base*scalar+ * @param [in] scalar The private scalar.  */-void crypton_decaf_x448_derive_public_key (+void CRYPTON_DECAF_API_VIS crypton_decaf_x448_derive_public_key (     uint8_t out[CRYPTON_DECAF_X448_PUBLIC_BYTES],     const uint8_t scalar[CRYPTON_DECAF_X448_PRIVATE_BYTES]-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /* FUTURE: uint8_t crypton_decaf_448_encode_like_curve448) */ @@ -441,10 +498,10 @@  * @param [out] a A precomputed table of multiples of the point.  * @param [in] b Any point.  */-void crypton_decaf_448_precompute (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_precompute (     crypton_decaf_448_precomputed_s *a,     const crypton_decaf_448_point_t b-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Multiply a precomputed base point by a scalar:@@ -457,11 +514,11 @@  * @param [in] base The point to be scaled.  * @param [in] scalar The scalar to multiply by.  */-void crypton_decaf_448_precomputed_scalarmul (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_precomputed_scalarmul (     crypton_decaf_448_point_t scaled,     const crypton_decaf_448_precomputed_s *base,     const crypton_decaf_448_scalar_t scalar-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Multiply two base points by two scalars:@@ -476,13 +533,13 @@  * @param [in] base2 A second point to be scaled.  * @param [in] scalar2 A second scalar to multiply by.  */-void crypton_decaf_448_point_double_scalarmul (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_double_scalarmul (     crypton_decaf_448_point_t combo,     const crypton_decaf_448_point_t base1,     const crypton_decaf_448_scalar_t scalar1,     const crypton_decaf_448_point_t base2,     const crypton_decaf_448_scalar_t scalar2-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;      /**  * Multiply one base point by two scalars:@@ -499,13 +556,13 @@  * @param [in] scalar1 A first scalar to multiply by.  * @param [in] scalar2 A second scalar to multiply by.  */-void crypton_decaf_448_point_dual_scalarmul (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_dual_scalarmul (     crypton_decaf_448_point_t a1,     crypton_decaf_448_point_t a2,     const crypton_decaf_448_point_t base1,     const crypton_decaf_448_scalar_t scalar1,     const crypton_decaf_448_scalar_t scalar2-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Multiply two base points by two scalars:@@ -522,12 +579,12 @@  * @warning: This function takes variable time, and may leak the scalars  * used.  It is designed for signature verification.  */-void crypton_decaf_448_base_double_scalarmul_non_secret (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_base_double_scalarmul_non_secret (     crypton_decaf_448_point_t combo,     const crypton_decaf_448_scalar_t scalar1,     const crypton_decaf_448_point_t base2,     const crypton_decaf_448_scalar_t scalar2-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Constant-time decision between two points.  If pick_b@@ -538,12 +595,12 @@  * @param [in] b Any point.  * @param [in] pick_b If nonzero, choose point b.  */-void crypton_decaf_448_point_cond_sel (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_cond_sel (     crypton_decaf_448_point_t out,     const crypton_decaf_448_point_t a,     const crypton_decaf_448_point_t b,     crypton_decaf_word_t pick_b-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Constant-time decision between two scalars.  If pick_b@@ -554,12 +611,12 @@  * @param [in] b Any scalar.  * @param [in] pick_b If nonzero, choose scalar b.  */-void crypton_decaf_448_scalar_cond_sel (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_scalar_cond_sel (     crypton_decaf_448_scalar_t out,     const crypton_decaf_448_scalar_t a,     const crypton_decaf_448_scalar_t b,     crypton_decaf_word_t pick_b-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Test that a point is valid, for debugging purposes.@@ -568,9 +625,9 @@  * @retval CRYPTON_DECAF_TRUE The point is valid.  * @retval CRYPTON_DECAF_FALSE The point is invalid.  */-crypton_decaf_bool_t crypton_decaf_448_point_valid (+crypton_decaf_bool_t CRYPTON_DECAF_API_VIS crypton_decaf_448_point_valid (     const crypton_decaf_448_point_t to_test-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_WARN_UNUSED CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Torque a point, for debugging purposes.  The output@@ -579,10 +636,10 @@  * @param [out] q The point to torque.  * @param [in] p The point to torque.  */-void crypton_decaf_448_point_debugging_torque (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_debugging_torque (     crypton_decaf_448_point_t q,     const crypton_decaf_448_point_t p-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Projectively scale a point, for debugging purposes.@@ -593,11 +650,11 @@  * @param [in] p The point to scale.  * @param [in] factor Serialized GF factor to scale.  */-void crypton_decaf_448_point_debugging_pscale (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_debugging_pscale (     crypton_decaf_448_point_t q,     const crypton_decaf_448_point_t p,     const unsigned char factor[CRYPTON_DECAF_448_SER_BYTES]-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Almost-Elligator-like hash to curve.@@ -627,11 +684,11 @@  * @param [in] hashed_data Output of some hash function.  * @param [out] pt The data hashed to the curve.  */-void+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_from_hash_nonuniform (     crypton_decaf_448_point_t pt,     const unsigned char hashed_data[CRYPTON_DECAF_448_HASH_BYTES]-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Indifferentiable hash function encoding to curve.@@ -641,10 +698,10 @@  * @param [in] hashed_data Output of some hash function.  * @param [out] pt The data hashed to the curve.  */ -void crypton_decaf_448_point_from_hash_uniform (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_from_hash_uniform (     crypton_decaf_448_point_t pt,     const unsigned char hashed_data[2*CRYPTON_DECAF_448_HASH_BYTES]-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE;  /**  * @brief Inverse of elligator-like hash to curve.@@ -656,6 +713,16 @@  * inverse sampling, this function succeeds or fails  * independently for different "which" values.  *+ * This function isn't guaranteed to find every possible+ * preimage, but it finds all except a small finite number.+ * In particular, when the number of bits in the modulus isn't+ * a multiple of 8 (i.e. for curve25519), it sets the high bits+ * independently, which enables the generated data to be uniform.+ * But it doesn't add p, so you'll never get exactly p from this+ * function.  This might change in the future, especially if+ * we ever support eg Brainpool curves, where this could cause+ * real nonuniformity.+ *  * @param [out] recovered_hash Encoded data.  * @param [in] pt The point to encode.  * @param [in] which A value determining which inverse point@@ -664,12 +731,12 @@  * @retval CRYPTON_DECAF_SUCCESS The inverse succeeded.  * @retval CRYPTON_DECAF_FAILURE The inverse failed.  */-crypton_decaf_error_t+crypton_decaf_error_t CRYPTON_DECAF_API_VIS crypton_decaf_448_invert_elligator_nonuniform (     unsigned char recovered_hash[CRYPTON_DECAF_448_HASH_BYTES],     const crypton_decaf_448_point_t pt,     uint32_t which-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE CRYPTON_DECAF_WARN_UNUSED;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE CRYPTON_DECAF_WARN_UNUSED;  /**  * @brief Inverse of elligator-like hash to curve.@@ -689,33 +756,31 @@  * @retval CRYPTON_DECAF_SUCCESS The inverse succeeded.  * @retval CRYPTON_DECAF_FAILURE The inverse failed.  */-crypton_decaf_error_t+crypton_decaf_error_t CRYPTON_DECAF_API_VIS crypton_decaf_448_invert_elligator_uniform (     unsigned char recovered_hash[2*CRYPTON_DECAF_448_HASH_BYTES],     const crypton_decaf_448_point_t pt,     uint32_t which-) CRYPTON_DECAF_API_VIS CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE CRYPTON_DECAF_WARN_UNUSED;+) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_NOINLINE CRYPTON_DECAF_WARN_UNUSED; -/**- * @brief Overwrite scalar with zeros.- */-void crypton_decaf_448_scalar_destroy (+/** Securely erase a scalar. */+void CRYPTON_DECAF_API_VIS crypton_decaf_448_scalar_destroy (     crypton_decaf_448_scalar_t scalar-) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_API_VIS;+) CRYPTON_DECAF_NONNULL; -/**- * @brief Overwrite point with zeros.+/** Securely erase a point by overwriting it with zeros.+ * @warning This causes the point object to become invalid.  */-void crypton_decaf_448_point_destroy (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_point_destroy (     crypton_decaf_448_point_t point-) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_API_VIS;+) CRYPTON_DECAF_NONNULL; -/**- * @brief Overwrite precomputed table with zeros.+/** Securely erase a precomputed table by overwriting it with zeros.+ * @warning This causes the table object to become invalid.  */-void crypton_decaf_448_precomputed_destroy (+void CRYPTON_DECAF_API_VIS crypton_decaf_448_precomputed_destroy (     crypton_decaf_448_precomputed_s *pre-) CRYPTON_DECAF_NONNULL CRYPTON_DECAF_API_VIS;+) CRYPTON_DECAF_NONNULL;  #ifdef __cplusplus } /* extern "C" */
cbits/decaf/include/field.h view
@@ -103,5 +103,10 @@ #endif } +#if P_MOD_8 == 5+#define crypton_gf_mul_i crypton_gf_mul_qnr+#define crypton_gf_div_i crypton_gf_div_qnr+#endif+  #endif // __GF_H__
cbits/decaf/include/word.h view
@@ -13,6 +13,11 @@ extern int posix_memalign(void **, size_t, size_t); #endif +// MSVC has no posix_memalign+#if defined(_MSC_VER)+#define posix_memalign(p, a, s) (((*(p)) = _aligned_malloc((s), (a))), *(p) ?0 :errno)+#endif+ #include <assert.h> #include <stdint.h> #include "arch_intrinsics.h"@@ -58,6 +63,22 @@ #else     #error "For now, libdecaf only supports 32- and 64-bit architectures." #endif++/**+ * Expand bit 0 of the given uint8_t to a mask_t all 1 or all 0+ * The input must be either 0 or 1+ */+static CRYPTON_DECAF_INLINE mask_t bit_to_mask(uint8_t bit) {+#ifdef _MSC_VER+#pragma warning ( push)+#pragma warning ( disable : 4146)+#endif+	return -(mask_t)bit;+#ifdef _MSC_VER+#pragma warning ( pop)+#endif++}      /* Scalar limbs are keyed off of the API word size instead of the arch word size. */ #if CRYPTON_DECAF_WORD_BITS == 64@@ -130,7 +151,7 @@     br_set_to_mask(mask_t x) {         return vdupq_n_u32(x);     }-#elif _WIN64 || __amd64__ || __X86_64__ || __aarch64__+#elif __amd64__ || __X86_64__ || __aarch64__ || __loongarch_lp64 || __PPC64__ || __riscv ||  __s390x__ || __alpha__ || __powerpc64__ || (__sparc__ && __arch64__) /* || _WIN64 -> WIN64 does not support int128 so force the build on arch32 default so do not use this define for _WIN64*/     #define VECTOR_ALIGNED __attribute__((aligned(8)))     typedef uint64_t big_register_t, uint64xn_t;
cbits/decaf/p448/arch_32/f_impl.c view
@@ -88,11 +88,11 @@      accum0 += accum8 + c[8];     c[8] = accum0 & mask;-    c[9] += accum0 >> 28;+    c[9] += (uint32_t)(accum0 >> 28);      accum8 += c[0];     c[0] = accum8 & mask;-    c[1] += accum8 >> 28;+    c[1] += (uint32_t)(accum8 >> 28); }  void crypton_gf_sqr (crypton_gf_s *__restrict__ cs, const gf as) {
cbits/decaf/p448/f_field.h view
@@ -23,11 +23,10 @@  #define __CRYPTON_DECAF_448_GF_DEFINED__ 1 #define NLIMBS (64/sizeof(word_t))-#define X_SER_BYTES 56 #define SER_BYTES 56 typedef struct crypton_gf_448_s {     word_t limb[NLIMBS];-} __attribute__((aligned(16))) crypton_gf_448_s, crypton_gf_448_t[1];+} __attribute__((aligned(32))) crypton_gf_448_s, crypton_gf_448_t[1];  #define GF_LIT_LIMB_BITS  56 #define GF_BITS           448@@ -37,7 +36,7 @@ #define gf                crypton_gf_448_t #define crypton_gf_s              crypton_gf_448_s #define crypton_gf_eq             crypton_gf_448_eq-#define crypton_gf_hibit          crypton_gf_448_hibit+#define crypton_gf_lobit          crypton_gf_448_lobit #define crypton_gf_copy           crypton_gf_448_copy #define crypton_gf_add            crypton_gf_448_add #define crypton_gf_sub            crypton_gf_448_sub@@ -54,7 +53,7 @@ #define crypton_gf_deserialize    crypton_gf_448_deserialize  /* RFC 7748 support */-#define X_PUBLIC_BYTES  X_SER_BYTES+#define X_PUBLIC_BYTES  SER_BYTES #define X_PRIVATE_BYTES X_PUBLIC_BYTES #define X_PRIVATE_BITS  448 @@ -81,10 +80,10 @@ void crypton_gf_sqr (crypton_gf_s *__restrict__ out, const gf a); mask_t crypton_gf_isr(gf a, const gf x); /** a^2 x = 1, QNR, or 0 if x=0.  Return true if successful */ mask_t crypton_gf_eq (const gf x, const gf y);-mask_t crypton_gf_hibit (const gf x);+mask_t crypton_gf_lobit (const gf x); -void crypton_gf_serialize (uint8_t *serial, const gf x,int with_highbit);-mask_t crypton_gf_deserialize (gf x, const uint8_t serial[SER_BYTES],int with_highbit);+void crypton_gf_serialize (uint8_t serial[SER_BYTES], const gf x);+mask_t crypton_gf_deserialize (gf x, const uint8_t serial[SER_BYTES],uint8_t hi_nmask);   #ifdef __cplusplus
cbits/decaf/p448/f_generic.c view
@@ -24,52 +24,52 @@ #endif  /** Serialize to wire format. */-void crypton_gf_serialize (uint8_t serial[SER_BYTES], const gf x, int with_hibit) {+void crypton_gf_serialize (uint8_t serial[SER_BYTES], const gf x) {     gf red;     crypton_gf_copy(red, x);     crypton_gf_strong_reduce(red);-    if (!with_hibit) { assert(crypton_gf_hibit(red) == 0); }          unsigned int j=0, fill=0;     dword_t buffer = 0;-    UNROLL for (unsigned int i=0; i<(with_hibit ? X_SER_BYTES : SER_BYTES); i++) {+    UNROLL for (unsigned int i=0; i<SER_BYTES; i++) {         if (fill < 8 && j < NLIMBS) {             buffer |= ((dword_t)red->limb[LIMBPERM(j)]) << fill;             fill += LIMB_PLACE_VALUE(LIMBPERM(j));             j++;         }-        serial[i] = buffer;+        serial[i] = (uint8_t)buffer;         fill -= 8;         buffer >>= 8;     } }  /** Return high bit of x = low bit of 2x mod p */-mask_t crypton_gf_hibit(const gf x) {+mask_t crypton_gf_lobit(const gf x) {     gf y;-    crypton_gf_add(y,x,x);+    crypton_gf_copy(y,x);     crypton_gf_strong_reduce(y);-    return -(y->limb[0]&1);+    return bit_to_mask((y->limb[0]) & 1); }  /** Deserialize from wire format; return -1 on success and 0 on failure. */-mask_t crypton_gf_deserialize (gf x, const uint8_t serial[SER_BYTES], int with_hibit) {+mask_t crypton_gf_deserialize (gf x, const uint8_t serial[SER_BYTES], uint8_t hi_nmask) {     unsigned int j=0, fill=0;     dword_t buffer = 0;     dsword_t scarry = 0;     UNROLL for (unsigned int i=0; i<NLIMBS; i++) {-        UNROLL while (fill < LIMB_PLACE_VALUE(LIMBPERM(i)) && j < (with_hibit ? X_SER_BYTES : SER_BYTES)) {-            buffer |= ((dword_t)serial[j]) << fill;+        UNROLL while (fill < (unsigned int)(LIMB_PLACE_VALUE(LIMBPERM(i))) && j < SER_BYTES) {+            uint8_t sj = serial[j];+            if (j==SER_BYTES-1) sj &= ~hi_nmask;+            buffer |= ((dword_t)sj) << fill;             fill += 8;             j++;         }-        x->limb[LIMBPERM(i)] = (i<NLIMBS-1) ? buffer & LIMB_MASK(LIMBPERM(i)) : buffer;+        x->limb[LIMBPERM(i)] = (word_t)((i<NLIMBS-1) ? buffer & LIMB_MASK(LIMBPERM(i)) : buffer);         fill -= LIMB_PLACE_VALUE(LIMBPERM(i));         buffer >>= LIMB_PLACE_VALUE(LIMBPERM(i));         scarry = (scarry + x->limb[LIMBPERM(i)] - MODULUS->limb[LIMBPERM(i)]) >> (8*sizeof(word_t));     }-    mask_t succ = with_hibit ? -(mask_t)1 : ~crypton_gf_hibit(x);-    return succ & word_is_zero(buffer) & ~word_is_zero(scarry);+    return word_is_zero((word_t)buffer) & ~word_is_zero((word_t)scarry); }  /** Reduce to canonical form. */@@ -91,9 +91,9 @@      * common case: it was < p, so now scarry = -1 and this = x - p + 2^255      * so let's add back in p.  will carry back off the top for 2^255.      */-    assert(word_is_zero(scarry) | word_is_zero(scarry+1));+    assert(word_is_zero((word_t)scarry) | word_is_zero((word_t)scarry+1)); -    word_t scarry_0 = scarry;+    word_t scarry_0 = (word_t)scarry;     dword_t carry = 0;      /* add it back */@@ -103,7 +103,7 @@         carry >>= LIMB_PLACE_VALUE(LIMBPERM(i));     } -    assert(word_is_zero(carry + scarry_0));+    assert(word_is_zero((word_t)(carry) + scarry_0)); }  /** Subtract two gf elements d=a-b */
cbits/ed25519/ed25519.c view
@@ -12,8 +12,33 @@ #include "ed25519-randombytes.h" #include "ed25519-hash.h" #include "ed25519-crypton-exts.h"+#include "ed25519/ed25519_s2n.h"  /*+	The base point multiplied by a scalar, packed.  s2n-bignum's assembly+	where it is built -- twice the speed of the table below, and signing+	does this twice -- and ed25519-donna's own table where it is not.+*/+static void+ed25519_base_pack(ed25519_public_key out, const bignum256modm s) {+	ge25519 ALIGN(16) p;++#if defined(CRYPTON_S2N_BIGNUM) && defined(__BYTE_ORDER__) \+    && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__+	unsigned char e[32];++	contract256_modm(e, s);+	if (crypton_ed25519_base_mult(out, e)) {+		memset(e, 0, sizeof e);+		return;+	}+	memset(e, 0, sizeof e);+#endif+	ge25519_scalarmult_base_niels(&p, ge25519_niels_base_multiples, s);+	ge25519_pack(out, &p);+}++/* 	Generates a (extsk[0..31]) and aExt (extsk[32..63]) */ @@ -38,14 +63,12 @@ void ED25519_FN(ed25519_publickey) (const ed25519_secret_key sk, ed25519_public_key pk) { 	bignum256modm a;-	ge25519 ALIGN(16) A; 	hash_512bits extsk;  	/* A = aB */ 	ed25519_extsk(extsk, sk); 	expand256_modm(a, extsk, 32);-	ge25519_scalarmult_base_niels(&A, ge25519_niels_base_multiples, a);-	ge25519_pack(pk, &A);+	ed25519_base_pack(pk, a); }  @@ -53,7 +76,6 @@ ED25519_FN(ed25519_sign) (const unsigned char *m, size_t mlen, const ed25519_secret_key sk, const ed25519_public_key pk, ed25519_signature RS) { 	ed25519_hash_context ctx; 	bignum256modm r, S, a;-	ge25519 ALIGN(16) R; 	hash_512bits extsk, hashr, hram;  	ed25519_extsk(extsk, sk);@@ -66,8 +88,7 @@ 	expand256_modm(r, hashr, 64);  	/* R = rB */-	ge25519_scalarmult_base_niels(&R, ge25519_niels_base_multiples, r);-	ge25519_pack(RS, &R);+	ed25519_base_pack(RS, r);  	/* S = H(R,A,m).. */ 	ed25519_hram(hram, RS, pk, m, mlen);@@ -89,7 +110,7 @@ 	ge25519 ALIGN(16) R, A; 	hash_512bits hash; 	bignum256modm hram, S;-	unsigned char checkR[32];+	unsigned char checkR[32], checkS[32];  	if ((RS[63] & 224) || !ge25519_unpack_negative_vartime(&A, pk)) 		return -1;@@ -100,6 +121,11 @@  	/* S */ 	expand256_modm(S, RS + 32, 32);++	/* check that S is canonical */+	contract256_modm(checkS, S);+	if (!ed25519_verify(RS + 32, checkS, 32))+		return -1;  	/* SB - H(R,A,m)A */ 	ge25519_double_scalarmult_vartime(&R, &A, hram, S);
+ cbits/ed25519/ed25519_s2n.c view
@@ -0,0 +1,65 @@+/*+ * Ed25519's base point multiplication through the vendored s2n-bignum.+ *+ * Signing does this twice: once for the nonce's point R, and once for the+ * public key, which crypton derives from the secret key at every signature+ * rather than trusting the one it is handed.  Both go through here, so both+ * halves of a signature move at once.+ *+ * Measured on an Apple M4, one multiplication with the encoding:+ * ed25519-donna 7.5 us against s2n-bignum 3.5.+ */+#include <string.h>++#include "ed25519/ed25519_s2n.h"++#if defined(CRYPTON_S2N_BIGNUM) && defined(__BYTE_ORDER__) \+    && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__+#define CRYPTON_ED25519_S2N 1+#include "crypton_cpu.h"++extern void edwards25519_scalarmulbase(uint64_t res[8],+                                       const uint64_t scalar[4]);+extern void edwards25519_scalarmulbase_alt(uint64_t res[8],+                                           const uint64_t scalar[4]);+extern void edwards25519_encode(uint8_t z[32], const uint64_t p[8]);++/* The same question as everywhere else in cbits/s2n: a microarchitecture one+ * on ARM that no feature bit answers, and exactly a feature bit on x86-64. */+static int use_alt(void)+{+#if defined(__aarch64__) || defined(__arm64__)+#ifdef __APPLE__+	return 1;+#else+	return 0;+#endif+#else+	return (crypton_x86_simd_features() & CRYPTON_X86_ADX) == 0;+#endif+}+#endif++int crypton_ed25519_base_mult(uint8_t out[32], const uint8_t scalar[32])+{+#ifdef CRYPTON_ED25519_S2N+	/* the assembly takes four little-endian 64-bit words, which is the+	 * same bits as the 32 little-endian bytes the scalar is kept in */+	uint64_t s[4], p[8];++	memcpy(s, scalar, 32);+	if (use_alt())+		edwards25519_scalarmulbase_alt(p, s);+	else+		edwards25519_scalarmulbase(p, s);+	edwards25519_encode(out, p);++	memset(s, 0, sizeof s);+	memset(p, 0, sizeof p);+	return 1;+#else+	(void) out;+	(void) scalar;+	return 0;+#endif+}
+ cbits/ed25519/ed25519_s2n.h view
@@ -0,0 +1,14 @@+#ifndef CRYPTON_ED25519_S2N_H+#define CRYPTON_ED25519_S2N_H++#include <stdint.h>++/* The base point multiplied by a scalar, packed into Ed25519's 32-byte+ * encoding.  The scalar is 32 little-endian bytes, already reduced.+ *+ * Returns 1 when the vendored s2n-bignum did the work and 0 when it is not+ * built, in which case the caller multiplies the base point itself.+ */+int crypton_ed25519_base_mult(uint8_t out[32], const uint8_t scalar[32]);++#endif
cbits/include32/p256/p256.h view
@@ -83,16 +83,9 @@     const crypton_p256_int* b,     crypton_p256_int* c); -// b := 1 / a % MOD-// MOD best be SECP256r1_n-void crypton_p256_modinv(-    const crypton_p256_int* MOD,-    const crypton_p256_int* a,-    crypton_p256_int* b);--// b := 1 / a % MOD+// b := 1 / a % MOD, in time that depends on a // MOD best be SECP256r1_n-// Faster than crypton_p256_modinv()+// Answers zero for an a that has no inverse, which is zero and MOD void crypton_p256_modinv_vartime(     const crypton_p256_int* MOD,     const crypton_p256_int* a,@@ -130,13 +123,6 @@ void crypton_p256_base_point_mul(const crypton_p256_int *n,                          crypton_p256_int *out_x,                          crypton_p256_int *out_y);--// {out_x,out_y} := n{in_x,in_y}-void crypton_p256_point_mul(const crypton_p256_int *n,-                    const crypton_p256_int *in_x,-                    const crypton_p256_int *in_y,-                    crypton_p256_int *out_x,-                    crypton_p256_int *out_y);  // {out_x,out_y} := n1G + n2{in_x,in_y} void crypton_p256_points_mul_vartime(
cbits/include32/p256/p256_gf.h view
@@ -76,6 +76,13 @@     0 }; static const felem kZero = {0};++/* the curve's b, in Montgomery form, for the complete addition formula */+static const felem kB = {+    0x13897bbf, 0x9cdf622, 0x43090d8, 0x2e67c4,+    0x176b5678, 0x2afdc84, 0xd196888, 0xb090e90,+    0xb8600c3+}; static const felem kP = {     0x1fffffff, 0xfffffff, 0x1fffffff, 0x3ff,     0, 0, 0x200000, 0xf000000,@@ -86,96 +93,99 @@     0, 0, 0x400000, 0xe000000,     0x1fffffff };-/* kPrecomputed contains precomputed values to aid the calculation of scalar- * multiples of the base point, G. It's actually two, equal length, tables- * concatenated.+/* kPrecomputed holds the multiples of the base point G that the comb in+ * scalar_base_mult reads.  Two tables of sixteen affine points, one after the+ * other.  *- * The first table contains (x,y) felem pairs for 16 multiples of the base- * point, G.+ * The comb takes five bits of the signed all-bits-set representation at a+ * time, from positions 52 apart, and the two tables are offset from each+ * other by 26:  *- *   Index  |  Index (binary) | Value- *       0  |           0000  | 0G (all zeros, omitted)- *       1  |           0001  | G- *       2  |           0010  | 2**64G- *       3  |           0011  | 2**64G + G- *       4  |           0100  | 2**128G- *       5  |           0101  | 2**128G + G- *       6  |           0110  | 2**128G + 2**64G- *       7  |           0111  | 2**128G + 2**64G + G- *       8  |           1000  | 2**192G- *       9  |           1001  | 2**192G + G- *      10  |           1010  | 2**192G + 2**64G- *      11  |           1011  | 2**192G + 2**64G + G- *      12  |           1100  | 2**192G + 2**128G- *      13  |           1101  | 2**192G + 2**128G + G- *      14  |           1110  | 2**192G + 2**128G + 2**64G- *      15  |           1111  | 2**192G + 2**128G + 2**64G + G+ *   first table    i, 52+i, 104+i, 156+i, 208+i+ *   second table   26+i, 78+i, 130+i, 182+i, 234+i  *- * The second table follows the same style, but the terms are 2**32G,- * 2**96G, 2**160G, 2**224G.+ * for i from 25 down to 0, which covers all 260 bits between them.  *+ * Every digit of that representation is +-1, so a block of five teeth takes+ * one of thirty-two values -- and they come in pairs that differ only by+ * sign.  So sixteen entries are enough: the top tooth is taken positive, bit+ * j of the index says that tooth j agrees with it, and where the top tooth is+ * negative the caller negates y, which costs a subtraction.  Entry zero is a+ * point like any other here, unlike the unsigned table this replaces, where+ * it stood for the infinity.+ *+ *   Index  |  Index (binary) | Value+ *       0  |           0000  | 2**208G - 2**156G - 2**104G - 2**52G - G+ *       1  |           0001  | 2**208G - 2**156G - 2**104G - 2**52G + G+ *     ...  |            ...  | ...+ *      15  |           1111  | 2**208G + 2**156G + 2**104G + 2**52G + G+ *  * This is ~2KB of data. */-static const limb kPrecomputed[NLIMBS * 2 * 15 * 2] = {-    0x11522878, 0xe730d41, 0xdb60179, 0x4afe2ff, 0x12883add, 0xcaddd88, 0x119e7edc, 0xd4a6eab, 0x3120bee,-    0x1d2aac15, 0xf25357c, 0x19e45cdd, 0x5c721d0, 0x1992c5a5, 0xa237487, 0x154ba21, 0x14b10bb, 0xae3fe3,-    0xd41a576, 0x922fc51, 0x234994f, 0x60b60d3, 0x164586ae, 0xce95f18, 0x1fe49073, 0x3fa36cc, 0x5ebcd2c,-    0xb402f2f, 0x15c70bf, 0x1561925c, 0x5a26704, 0xda91e90, 0xcdc1c7f, 0x1ea12446, 0xe1ade1e, 0xec91f22,-    0x26f7778, 0x566847e, 0xa0bec9e, 0x234f453, 0x1a31f21a, 0xd85e75c, 0x56c7109, 0xa267a00, 0xb57c050,-    0x98fb57, 0xaa837cc, 0x60c0792, 0xcfa5e19, 0x61bab9e, 0x589e39b, 0xa324c5, 0x7d6dee7, 0x2976e4b,-    0x1fc4124a, 0xa8c244b, 0x1ce86762, 0xcd61c7e, 0x1831c8e0, 0x75774e1, 0x1d96a5a9, 0x843a649, 0xc3ab0fa,-    0x6e2e7d5, 0x7673a2a, 0x178b65e8, 0x4003e9b, 0x1a1f11c2, 0x7816ea, 0xf643e11, 0x58c43df, 0xf423fc2,-    0x19633ffa, 0x891f2b2, 0x123c231c, 0x46add8c, 0x54700dd, 0x59e2b17, 0x172db40f, 0x83e277d, 0xb0dd609,-    0xfd1da12, 0x35c6e52, 0x19ede20c, 0xd19e0c0, 0x97d0f40, 0xb015b19, 0x449e3f5, 0xe10c9e, 0x33ab581,-    0x56a67ab, 0x577734d, 0x1dddc062, 0xc57b10d, 0x149b39d, 0x26a9e7b, 0xc35df9f, 0x48764cd, 0x76dbcca,-    0xca4b366, 0xe9303ab, 0x1a7480e7, 0x57e9e81, 0x1e13eb50, 0xf466cf3, 0x6f16b20, 0x4ba3173, 0xc168c33,-    0x15cb5439, 0x6a38e11, 0x73658bd, 0xb29564f, 0x3f6dc5b, 0x53b97e, 0x1322c4c0, 0x65dd7ff, 0x3a1e4f6,-    0x14e614aa, 0x9246317, 0x1bc83aca, 0xad97eed, 0xd38ce4a, 0xf82b006, 0x341f077, 0xa6add89, 0x4894acd,-    0x9f162d5, 0xf8410ef, 0x1b266a56, 0xd7f223, 0x3e0cb92, 0xe39b672, 0x6a2901a, 0x69a8556, 0x7e7c0,-    0x9b7d8d3, 0x309a80, 0x1ad05f7f, 0xc2fb5dd, 0xcbfd41d, 0x9ceb638, 0x1051825c, 0xda0cf5b, 0x812e881,-    0x6f35669, 0x6a56f2c, 0x1df8d184, 0x345820, 0x1477d477, 0x1645db1, 0xbe80c51, 0xc22be3e, 0xe35e65a,-    0x1aeb7aa0, 0xc375315, 0xf67bc99, 0x7fdd7b9, 0x191fc1be, 0x61235d, 0x2c184e9, 0x1c5a839, 0x47a1e26,-    0xb7cb456, 0x93e225d, 0x14f3c6ed, 0xccc1ac9, 0x17fe37f3, 0x4988989, 0x1a90c502, 0x2f32042, 0xa17769b,-    0xafd8c7c, 0x8191c6e, 0x1dcdb237, 0x16200c0, 0x107b32a1, 0x66c08db, 0x10d06a02, 0x3fc93, 0x5620023,-    0x16722b27, 0x68b5c59, 0x270fcfc, 0xfad0ecc, 0xe5de1c2, 0xeab466b, 0x2fc513c, 0x407f75c, 0xbaab133,-    0x9705fe9, 0xb88b8e7, 0x734c993, 0x1e1ff8f, 0x19156970, 0xabd0f00, 0x10469ea7, 0x3293ac0, 0xcdc98aa,-    0x1d843fd, 0xe14bfe8, 0x15be825f, 0x8b5212, 0xeb3fb67, 0x81cbd29, 0xbc62f16, 0x2b6fcc7, 0xf5a4e29,-    0x13560b66, 0xc0b6ac2, 0x51ae690, 0xd41e271, 0xf3e9bd4, 0x1d70aab, 0x1029f72, 0x73e1c35, 0xee70fbc,-    0xad81baf, 0x9ecc49a, 0x86c741e, 0xfe6be30, 0x176752e7, 0x23d416, 0x1f83de85, 0x27de188, 0x66f70b8,-    0x181cd51f, 0x96b6e4c, 0x188f2335, 0xa5df759, 0x17a77eb6, 0xfeb0e73, 0x154ae914, 0x2f3ec51, 0x3826b59,-    0xb91f17d, 0x1c72949, 0x1362bf0a, 0xe23fddf, 0xa5614b0, 0xf7d8f, 0x79061, 0x823d9d2, 0x8213f39,-    0x1128ae0b, 0xd095d05, 0xb85c0c2, 0x1ecb2ef, 0x24ddc84, 0xe35e901, 0x18411a4a, 0xf5ddc3d, 0x3786689,-    0x52260e8, 0x5ae3564, 0x542b10d, 0x8d93a45, 0x19952aa4, 0x996cc41, 0x1051a729, 0x4be3499, 0x52b23aa,-    0x109f307e, 0x6f5b6bb, 0x1f84e1e7, 0x77a0cfa, 0x10c4df3f, 0x25a02ea, 0xb048035, 0xe31de66, 0xc6ecaa3,-    0x28ea335, 0x2886024, 0x1372f020, 0xf55d35, 0x15e4684c, 0xf2a9e17, 0x1a4a7529, 0xcb7beb1, 0xb2a78a1,-    0x1ab21f1f, 0x6361ccf, 0x6c9179d, 0xb135627, 0x1267b974, 0x4408bad, 0x1cbff658, 0xe3d6511, 0xc7d76f,-    0x1cc7a69, 0xe7ee31b, 0x54fab4f, 0x2b914f, 0x1ad27a30, 0xcd3579e, 0xc50124c, 0x50daa90, 0xb13f72,-    0xb06aa75, 0x70f5cc6, 0x1649e5aa, 0x84a5312, 0x329043c, 0x41c4011, 0x13d32411, 0xb04a838, 0xd760d2d,-    0x1713b532, 0xbaa0c03, 0x84022ab, 0x6bcf5c1, 0x2f45379, 0x18ae070, 0x18c9e11e, 0x20bca9a, 0x66f496b,-    0x3eef294, 0x67500d2, 0xd7f613c, 0x2dbbeb, 0xb741038, 0xe04133f, 0x1582968d, 0xbe985f7, 0x1acbc1a,-    0x1a6a939f, 0x33e50f6, 0xd665ed4, 0xb4b7bd6, 0x1e5a3799, 0x6b33847, 0x17fa56ff, 0x65ef930, 0x21dc4a,-    0x2b37659, 0x450fe17, 0xb357b65, 0xdf5efac, 0x15397bef, 0x9d35a7f, 0x112ac15f, 0x624e62e, 0xa90ae2f,-    0x107eecd2, 0x1f69bbe, 0x77d6bce, 0x5741394, 0x13c684fc, 0x950c910, 0x725522b, 0xdc78583, 0x40eeabb,-    0x1fde328a, 0xbd61d96, 0xd28c387, 0x9e77d89, 0x12550c40, 0x759cb7d, 0x367ef34, 0xae2a960, 0x91b8bdc,-    0x93462a9, 0xf469ef, 0xb2e9aef, 0xd2ca771, 0x54e1f42, 0x7aaa49, 0x6316abb, 0x2413c8e, 0x5425bf9,-    0x1bed3e3a, 0xf272274, 0x1f5e7326, 0x6416517, 0xea27072, 0x9cedea7, 0x6e7633, 0x7c91952, 0xd806dce,-    0x8e2a7e1, 0xe421e1a, 0x418c9e1, 0x1dbc890, 0x1b395c36, 0xa1dc175, 0x1dc4ef73, 0x8956f34, 0xe4b5cf2,-    0x1b0d3a18, 0x3194a36, 0x6c2641f, 0xe44124c, 0xa2f4eaa, 0xa8c25ba, 0xf927ed7, 0x627b614, 0x7371cca,-    0xba16694, 0x417bc03, 0x7c0a7e3, 0x9c35c19, 0x1168a205, 0x8b6b00d, 0x10e3edc9, 0x9c19bf2, 0x5882229,-    0x1b2b4162, 0xa5cef1a, 0x1543622b, 0x9bd433e, 0x364e04d, 0x7480792, 0x5c9b5b3, 0xe85ff25, 0x408ef57,-    0x1814cfa4, 0x121b41b, 0xd248a0f, 0x3b05222, 0x39bb16a, 0xc75966d, 0xa038113, 0xa4a1769, 0x11fbc6c,-    0x917e50e, 0xeec3da8, 0x169d6eac, 0x10c1699, 0xa416153, 0xf724912, 0x15cd60b7, 0x4acbad9, 0x5efc5fa,-    0xf150ed7, 0x122b51, 0x1104b40a, 0xcb7f442, 0xfbb28ff, 0x6ac53ca, 0x196142cc, 0x7bf0fa9, 0x957651,-    0x4e0f215, 0xed439f8, 0x3f46bd5, 0x5ace82f, 0x110916b6, 0x6db078, 0xffd7d57, 0xf2ecaac, 0xca86dec,-    0x15d6b2da, 0x965ecc9, 0x1c92b4c2, 0x1f3811, 0x1cb080f5, 0x2d8b804, 0x19d1c12d, 0xf20bd46, 0x1951fa7,-    0xa3656c3, 0x523a425, 0xfcd0692, 0xd44ddc8, 0x131f0f5b, 0xaf80e4a, 0xcd9fc74, 0x99bb618, 0x2db944c,-    0xa673090, 0x1c210e1, 0x178c8d23, 0x1474383, 0x10b8743d, 0x985a55b, 0x2e74779, 0x576138, 0x9587927,-    0x133130fa, 0xbe05516, 0x9f4d619, 0xbb62570, 0x99ec591, 0xd9468fe, 0x1d07782d, 0xfc72e0b, 0x701b298,-    0x1863863b, 0x85954b8, 0x121a0c36, 0x9e7fedf, 0xf64b429, 0x9b9d71e, 0x14e2f5d8, 0xf858d3a, 0x942eea8,-    0xda5b765, 0x6edafff, 0xa9d18cc, 0xc65e4ba, 0x1c747e86, 0xe4ea915, 0x1981d7a1, 0x8395659, 0x52ed4e2,-    0x87d43b7, 0x37ab11b, 0x19d292ce, 0xf8d4692, 0x18c3053f, 0x8863e13, 0x4c146c0, 0x6bdf55a, 0x4e4457d,-    0x16152289, 0xac78ec2, 0x1a59c5a2, 0x2028b97, 0x71c2d01, 0x295851f, 0x404747b, 0x878558d, 0x7d29aa4,-    0x13d8341f, 0x8daefd7, 0x139c972d, 0x6b7ea75, 0xd4a9dde, 0xff163d8, 0x81d55d7, 0xa5bef68, 0xb7b30d8,-    0xbe73d6f, 0xaa88141, 0xd976c81, 0x7e7a9cc, 0x18beb771, 0xd773cbd, 0x13f51951, 0x9d0c177, 0x1c49a78,+static const limb kPrecomputed[NLIMBS * 2 * 16 * 2] = {+    0xe01bd76, 0xa0be8b3, 0x8494c1d, 0x609ab3d, 0x1188042f, 0x499c03d, 0x1df7cd26, 0x51b33c5, 0x1fb3bce,+    0x39cdd45, 0xdc0dd9b, 0xe3053d7, 0x1ffaf46, 0x9ac284a, 0xac051d4, 0x1c09fe1b, 0x8227cbf, 0x5bf049b,+    0xb9487d, 0x2ecb75f, 0x194825bf, 0xd70cf28, 0x14e528f3, 0x4d8670c, 0x35bbabb, 0x6b692ca, 0xd96d08,+    0x1db081dc, 0xa87ea7b, 0x190e5549, 0xa4cf420, 0x1e151385, 0xaf3d4bd, 0x4057e9f, 0x5078feb, 0x154519a,+    0xbf15dea, 0x453fa25, 0x1171c85a, 0x576c824, 0x154e7060, 0x71ede6e, 0x160467a2, 0xdea8a44, 0x81ffcb1,+    0x61a56fa, 0x76119b9, 0x110bfb9b, 0x3d527ea, 0x1997bdb4, 0xe1d1253, 0x180ce91c, 0x11950ee, 0x53d5938,+    0x694e7c9, 0xe0cf337, 0x16d8ae50, 0x202517f, 0x4d02e16, 0xd13b5fd, 0xfae97eb, 0xa1c7f60, 0x1206fe8,+    0x11b1c908, 0xf8a82f, 0x6ab17a0, 0x48058e8, 0x2d0feb, 0xfada550, 0x658edb9, 0xa17567a, 0x8daa44d,+    0x6361dc9, 0xec00c0e, 0x151a7b1c, 0xb35a683, 0x1643fe02, 0x70155c0, 0x1f131d45, 0x3998068, 0x25beef8,+    0x88138de, 0x8995ce4, 0x18565c50, 0x60f12b6, 0xc47a656, 0x4a82bd9, 0xb547a17, 0xb333474, 0xe86513a,+    0x7f8d2bc, 0x7f0f16c, 0x8cde475, 0x1ac3d5c, 0x1a832c9a, 0x6a93e7a, 0x19833281, 0xcec82db, 0x4f08cc,+    0x72c4394, 0x4686520, 0x1e845ce, 0xfb181a1, 0xf5135a5, 0xa1265d6, 0x6c63ce8, 0xe81797e, 0x5dbcd5a,+    0x2a5d603, 0x1ad4e91, 0xc86e1b3, 0x793abea, 0x1b8610a4, 0x8d5b975, 0x74dd850, 0xbbca81, 0xd7c35d8,+    0x1bab7afc, 0x4df749, 0x4acb4ea, 0xfae8c89, 0x14552ae5, 0x6dc1c20, 0x14f629f, 0x2368fe5, 0xe5a9cba,+    0x67576f7, 0x9c77c50, 0x1d63c92a, 0xbbb9ef8, 0xa7530d4, 0x963335, 0xfa09c54, 0xb6d03e3, 0xed1a022,+    0x19c59f49, 0x45c823d, 0x17a28df1, 0x80ae516, 0xe2ada82, 0x19b97fb, 0x13a9ebf, 0xf1e7606, 0xde03632,+    0x14e318b9, 0x7b57b83, 0x51a9a92, 0x3378a17, 0x1cde9289, 0x45956c2, 0x2bbab5e, 0x780b1f5, 0x8356034,+    0x75eca28, 0x6f39648, 0xf2fdbda, 0x4c65cd9, 0xb3759e7, 0x710c0b1, 0xda24432, 0x8d236aa, 0xf4c449f,+    0xaf9ba24, 0xb83ea7f, 0x1e46ecc3, 0x3f277b0, 0x6acdecd, 0x1f597d1, 0x8483d72, 0x57d29dd, 0x66d4060,+    0x167c14af, 0xc142ded, 0xc1689ca, 0x651aa52, 0x8d05768, 0x9709cfa, 0x165fd283, 0x9f6f583, 0x9833b54,+    0xd17e2d6, 0x2915c32, 0x1970ef24, 0xe8ca45b, 0x11c29e09, 0x8121f26, 0xb5afbe1, 0x10c80ec, 0x834a25c,+    0xa37f0b0, 0xd6bac16, 0xed484fc, 0x8799206, 0x13db8bb1, 0xdce615c, 0x13320329, 0x79dc25, 0x914e7af,+    0x860a414, 0xb8a9434, 0x50396ce, 0x902d3e6, 0x15c152f3, 0x3753c64, 0x970c055, 0xba296fb, 0x64bd63b,+    0x118cbb94, 0x9d4274f, 0x121cdbfe, 0xb9137cf, 0xc8bddf1, 0xa1598d0, 0x446ab41, 0x11f1df2, 0x68115f1,+    0x2f1f708, 0xee35192, 0x1dfeb3fc, 0x4e1e1a6, 0x1d9adcbb, 0x6688662, 0x63ad21b, 0x9d9a9e1, 0xb0f8b4b,+    0xcd8bc3a, 0x3577d8, 0x1ffcb97d, 0xd31e8d6, 0x1c776310, 0x95b4ef7, 0x185a82ed, 0xe40bbb0, 0xbca1ea,+    0x19462e0b, 0x2252179, 0x14f14f09, 0x565e68d, 0x7ba5f37, 0x4cd1858, 0x167941b3, 0x4d1c7a7, 0x7aef1ab,+    0x1599efe9, 0x658e78d, 0x1ad33917, 0x7e74797, 0x19152edc, 0xdf7dc18, 0xdf677c, 0x9315d96, 0x4ba8eec,+    0xc45cd82, 0x1acfa03, 0xe265b49, 0xcfa6eb, 0x89d7619, 0x7b05, 0x1ba11068, 0xe1672d3, 0x655622a,+    0x1607ca6, 0x339049, 0x5454f70, 0x10edd75, 0x1133ceb7, 0xe3eec39, 0xc263156, 0xeb6ddbe, 0x10360a7,+    0xfd5f981, 0x47dd502, 0x1e66dbbe, 0x8820b63, 0xeee91ef, 0xffde293, 0xb66325d, 0x3a5a2a, 0x6a2acbb,+    0x11e949bc, 0xf6a6d57, 0x6b2ca24, 0xad903ec, 0x1e55de0, 0xe7074ed, 0xe681934, 0xea44010, 0xaa490cc,+    0x512324a, 0x6a5eb00, 0xee5e100, 0xd60a02b, 0x1b89c993, 0x5cffb70, 0xa49030c, 0x405aee, 0xe1b27cc,+    0x2e73a15, 0x9ca8dc0, 0x781dbff, 0x9fd85e0, 0x1884e4c8, 0x40873f6, 0x32d4b69, 0xf42f753, 0xeaf4c38,+    0x2802a4c, 0x1283dac, 0x759100a, 0xcb3ba75, 0xf3203d3, 0xf89b8aa, 0x7caa59e, 0x384a60a, 0x37f69e7,+    0x1e56d569, 0x120c552, 0x181734aa, 0x4fcd9b4, 0x7918e4e, 0xcd7938a, 0x2bbd8b2, 0x19f4f97, 0xa7af533,+    0xbb69948, 0x3eff33e, 0x1f3ac118, 0x3739770, 0x58898fd, 0x623fafb, 0xa7e6d93, 0xd31a676, 0x615192d,+    0xf543d28, 0xe61ce5a, 0x10ae4b39, 0xcd5a8d7, 0x1b34c6de, 0x81997ad, 0x198e2093, 0xd9b6ff7, 0x9a5954f,+    0x16782589, 0xed3c1ab, 0x62dc4a5, 0xac12d0c, 0x8bc8b7c, 0x168ec4e, 0x177e11dc, 0x407df09, 0x4056e85,+    0x9858305, 0xfe445e9, 0x18b1230a, 0x815ee9f, 0xa852eb4, 0x89444e0, 0x1a83481, 0x479359b, 0x2192576,+    0x16d5e61b, 0xfe480c4, 0x1d60b8b7, 0x1c6e798, 0x1a01310, 0x9998572, 0x7c59f75, 0x49dda87, 0xe75e0b6,+    0x1b2b7536, 0xb267d8, 0x15443085, 0x45e5924, 0x7fb947f, 0x296915d, 0x38fc56b, 0x4bae39f, 0xb218e7c,+    0x115c3b16, 0x9d95f0c, 0xa50ac4c, 0xcce8037, 0xc3c7ba3, 0xf02773a, 0xbc2ad54, 0x26914c1, 0x19a4b8d,+    0x3f8a1a6, 0xa0b8459, 0x1f77a521, 0x7d93297, 0x1dddb4b2, 0x9e4cd1c, 0x6e28403, 0xd7ce413, 0x5575b62,+    0x12bb7dc1, 0xbdfb15e, 0xa542867, 0xe943d3e, 0x1367fbdd, 0x37b387, 0x14e4d75f, 0xb90b09d, 0xbf6ec28,+    0xa5182c8, 0x1e5b34e, 0xabe4602, 0x1c13efa, 0x8d1182, 0x1c2947a, 0x1e04e0e3, 0x6caecdb, 0x40f14e8,+    0x1b845e4a, 0xa9fb149, 0xb34f513, 0x3a0fdbb, 0xfad2335, 0x5bb9342, 0x18c5ad62, 0xc97fdc3, 0x31225c7,+    0xb28a9ee, 0x585915, 0x1e355da1, 0x26ed08e, 0xc7d06a, 0xa65f219, 0x1fdf45cd, 0xc8323ea, 0x297b9ee,+    0x1c031098, 0x9c39cf0, 0x1287d79f, 0x69a9e32, 0x10015650, 0x1c3dfc3, 0x12dcd848, 0x3155a59, 0xeff3212,+    0x1a6ecdd0, 0xd26bd07, 0x18e077ab, 0x442b477, 0x46b735f, 0x495d60c, 0x1b57a6e5, 0x76a368a, 0xf53bd50,+    0xf12c5e0, 0x80a9b4, 0x15562060, 0x4102113, 0xa144ab5, 0x5fa4e9, 0x1009f5e9, 0xe34343a, 0x26fda5a,+    0x159e06a4, 0x5fa3aad, 0x10259b5f, 0xa69947d, 0x1190417e, 0x987da3f, 0x14e1e868, 0xdcb7e1e, 0x8890f9f,+    0x14dece80, 0x94bbf5c, 0x18513a17, 0x4ca31ca, 0xc0a2713, 0xbc46074, 0x1536f6a5, 0x43991aa, 0xb9f8f1c,+    0x987ba48, 0xb0829ae, 0xd29d324, 0x6339c35, 0x18ace0a4, 0x5d53b55, 0xff829f4, 0xe882ecf, 0xc05164c,+    0x6bd6bba, 0x9dfc14f, 0x1981cab3, 0xcfebf18, 0x1ddac868, 0x94ec6d4, 0x5abd4b, 0x737fdb3, 0x18f531f,+    0xd3c2a71, 0x337178f, 0x9f7c32e, 0xd9d7fda, 0x137d191f, 0xdd0757b, 0x14b6d65, 0x179f37a, 0x67b10e1,+    0x1bfb2cfd, 0xfe4ca43, 0x1fee2930, 0x98c2aa0, 0x9826788, 0xeaf4ceb, 0x17a6be82, 0xc899ed1, 0x500fb01,+    0x10918e6f, 0x36179ed, 0xbca6643, 0x2d80942, 0xf1ef61, 0xadca21c, 0xbe5b3a7, 0xadae157, 0x12daac,+    0x1337dda8, 0x6326e5d, 0x2738e1b, 0x5cb5c54, 0x98ec8a0, 0x252647d, 0x1d5c173c, 0xbdf848d, 0x9e5217b,+    0x1d64f447, 0xb71d1a, 0xb2c2360, 0xccb6bee, 0x1245995e, 0x94a9130, 0x5b93d91, 0x76c57ff, 0xeaa91d1,+    0x3941881, 0xc2aafbd, 0x1c0540d0, 0x1a938f0, 0x1304b724, 0x8524e10, 0x1bef780f, 0xbc0ea48, 0xbe90dae,+    0x1d10d5d8, 0xca979e2, 0x10db5cc4, 0x54e2493, 0x44d38f3, 0xbcb73b, 0x12dcff4, 0xd0ab219, 0xde69db2,+    0x13594366, 0xc30e05f, 0xfc245d4, 0x8c5b52f, 0x81901c7, 0xa9d1e03, 0x11ead62e, 0xb7be89b, 0xc9c8486,+    0x132a6fa0, 0x56af9b8, 0x41cb561, 0xf74418c, 0x141c461a, 0xbc18514, 0x1d6bbb68, 0x96d43c2, 0x7108696 };
cbits/include64/p256/p256.h view
@@ -83,16 +83,9 @@     const crypton_p256_int* b,     crypton_p256_int* c); -// b := 1 / a % MOD-// MOD best be SECP256r1_n-void crypton_p256_modinv(-    const crypton_p256_int* MOD,-    const crypton_p256_int* a,-    crypton_p256_int* b);--// b := 1 / a % MOD+// b := 1 / a % MOD, in time that depends on a // MOD best be SECP256r1_n-// Faster than crypton_p256_modinv()+// Answers zero for an a that has no inverse, which is zero and MOD void crypton_p256_modinv_vartime(     const crypton_p256_int* MOD,     const crypton_p256_int* a,@@ -130,13 +123,6 @@ void crypton_p256_base_point_mul(const crypton_p256_int *n,                          crypton_p256_int *out_x,                          crypton_p256_int *out_y);--// {out_x,out_y} := n{in_x,in_y}-void crypton_p256_point_mul(const crypton_p256_int *n,-                    const crypton_p256_int *in_x,-                    const crypton_p256_int *in_y,-                    crypton_p256_int *out_x,-                    crypton_p256_int *out_y);  // {out_x,out_y} := n1G + n2{in_x,in_y} void crypton_p256_points_mul_vartime(
cbits/include64/p256/p256_gf.h view
@@ -63,6 +63,38 @@ #define NLIMBS 5 typedef limb felem[NLIMBS]; +/* On AArch64, the three functions that do the field arithmetic are asked to+ * be inlined rather than left for the compiler to decide.+ *+ * felem_mul and felem_square end in felem_reduce_degree, a carry chain the+ * whole width of the number, and that chain is what their latency is: one+ * product feeding the next costs 18.1 ns on an Apple M4, while four+ * independent ones cost 11.4 ns each.  The curve arithmetic has independent+ * products to offer -- the two squarings that open a point doubling, the+ * multiplication and the squaring that close it -- but only if the compiler+ * can see one reduction while the other is still going.  Left alone it emits+ * felem_reduce_degree once and calls it, and a call is a fence: the two+ * chains cannot overlap.  Plain `inline` does not change its mind.+ *+ * Asking costs code: this file's object goes from 30 to 116 kilobytes.  That+ * is worth it where there are registers to hold two chains at once and not+ * where there are not, which is the architecture talking rather than the+ * compiler.  Measured on a variable-point scalar multiplication:+ *+ *   Apple M4, Apple clang 21      1.23x+ *   Neoverse, clang 18            1.12x+ *   Neoverse, gcc 13              1.05x+ *   EPYC 7763, clang 18           0.95x+ *   Xeon 8370C, gcc 13            0.82x+ *+ * so x86-64 keeps the compiler's own judgement.+ */+#if defined(__aarch64__) && (defined(__GNUC__) || defined(__clang__))+#define FELEM_INLINE static inline __attribute__((always_inline))+#else+#define FELEM_INLINE static+#endif+ static const limb kBottom51Bits = 0x7ffffffffffff; static const limb kBottom52Bits = 0xfffffffffffff; @@ -72,102 +104,111 @@     2, 0xfc00000000000, 0x7ffffffffffff, 0xfff7fffffffff, 0x7ffff }; static const felem kZero = {0};++/* the curve's b, in Montgomery form, for the complete addition formula */+static const felem kB = {+    0x1bec453897bbf, 0x33e210c243627, 0x484bb5ab3c017, 0x41a32d11055fb,+    0x2e18030ec243a+}; static const felem kP = {     0x7ffffffffffff, 0x1fffffffffff, 0, 0x4000000000, 0x3fffffffc0000 }; static const felem k2P = {     0x7fffffffffffe, 0x3fffffffffff, 0, 0x8000000000, 0x7fffffff80000 };-/* kPrecomputed contains precomputed values to aid the calculation of scalar- * multiples of the base point, G. It's actually two, equal length, tables- * concatenated.+/* kPrecomputed holds the multiples of the base point G that the comb in+ * scalar_base_mult reads.  Two tables of sixteen affine points, one after the+ * other.  *- * The first table contains (x,y) felem pairs for 16 multiples of the base- * point, G.+ * The comb takes five bits of the signed all-bits-set representation at a+ * time, from positions 52 apart, and the two tables are offset from each+ * other by 26:  *- *   Index  |  Index (binary) | Value- *       0  |           0000  | 0G (all zeros, omitted)- *       1  |           0001  | G- *       2  |           0010  | 2**64G- *       3  |           0011  | 2**64G + G- *       4  |           0100  | 2**128G- *       5  |           0101  | 2**128G + G- *       6  |           0110  | 2**128G + 2**64G- *       7  |           0111  | 2**128G + 2**64G + G- *       8  |           1000  | 2**192G- *       9  |           1001  | 2**192G + G- *      10  |           1010  | 2**192G + 2**64G- *      11  |           1011  | 2**192G + 2**64G + G- *      12  |           1100  | 2**192G + 2**128G- *      13  |           1101  | 2**192G + 2**128G + G- *      14  |           1110  | 2**192G + 2**128G + 2**64G- *      15  |           1111  | 2**192G + 2**128G + 2**64G + G+ *   first table    i, 52+i, 104+i, 156+i, 208+i+ *   second table   26+i, 78+i, 130+i, 182+i, 234+i  *- * The second table follows the same style, but the terms are 2**32G,- * 2**96G, 2**160G, 2**224G.+ * for i from 25 down to 0, which covers all 260 bits between them.  *+ * Every digit of that representation is +-1, so a block of five teeth takes+ * one of thirty-two values -- and they come in pairs that differ only by+ * sign.  So sixteen entries are enough: the top tooth is taken positive, bit+ * j of the index says that tooth j agrees with it, and where the top tooth is+ * negative the caller negates y, which costs a subtraction.  Entry zero is a+ * point like any other here, unlike the unsigned table this replaces, where+ * it stood for the infinity.+ *+ *   Index  |  Index (binary) | Value+ *       0  |           0000  | 2**208G - 2**156G - 2**104G - 2**52G - G+ *       1  |           0001  | 2**208G - 2**156G - 2**104G - 2**52G + G+ *     ...  |            ...  | ...+ *      15  |           1111  | 2**208G + 2**156G + 2**104G + 2**52G + G+ *  * This is ~2KB of data. */-static const limb kPrecomputed[NLIMBS * 2 * 15 * 2] = {-    0x661a831522878, 0xf17fb6d805e79, 0x5889441d6ea57, 0xae33cfdb995bb, 0xc482fbb529ba,-    0x4a6af9d2aac15, 0x90e867917377c, 0x487cc962d2ae3, 0xec2a97443446e, 0x2b8ff8c52c42,-    0x45f8a2d41a576, 0xb06988d2653e4, 0x718b22c357305, 0x33fc920e79d2b, 0x17af34b0fe8db,-    0x38e17eb402f2f, 0x3382558649705, 0x47f6d48f482d1, 0x7bd42488d9b83, 0x3b247c8b86b78,-    0x4d08fc26f7778, 0x7a29a82fb2795, 0x75cd18f90d11a, 0xad8e213b0bc, 0x2d5f0142899e8,-    0x506f98098fb57, 0x2f0c98301e4aa, 0x39b30dd5cf67d, 0x9c146498ab13c, 0xa5db92df5b7b,-    0x184897fc4124a, 0xe3f73a19d8aa, 0x4e1c18e47066b, 0x27b2d4b52eaee, 0x30eac3ea10e99,-    0x4e74546e2e7d5, 0x1f4dde2d97a1d, 0x6ead0f88e1200, 0x7dec87c220f02, 0x3d08ff096310f,-    0x23e5659633ffa, 0x6ec648f08c722, 0x3172a3806ea35, 0xf6e5b681eb3c5, 0x2c3758260f89d,-    0x38dca4fd1da12, 0xf06067b78830d, 0x3194be87a068c, 0x78893c7eb602b, 0xcead60438432,-    0x6ee69a56a67ab, 0xd886f77701895, 0x67b0a4d9cee2b, 0x3586bbf3e4d53, 0x1db6f32921d93,-    0x260756ca4b366, 0x4f40e9d2039fa, 0x4f3f09f5a82bf, 0xccde2d641e8cd, 0x305a30cd2e8c5,-    0x471c235cb5439, 0xab279cd962f5a, 0x17e1fb6e2dd94, 0xfe64589800a77, 0xe8793d99775f,-    0x48c62f4e614aa, 0xbf76ef20eb2a4, 0x669c672556c, 0x24683e0eff056, 0x12252b369ab76,-    0x821de9f162d5, 0xf911ec99a95be, 0x6721f065c906b, 0x58d452035c736, 0x1f9f01a6a15,-    0x6135009b7d8d3, 0xdaeeeb417dfc0, 0x63865fea0ee17, 0x6e0a304b939d6, 0x204ba2076833d,-    0x4ade586f35669, 0x2c1077e34611a, 0x5b1a3bea3b81a, 0xf97d018a22c8b, 0x38d7996b08af8,-    0x6ea62baeb7aa0, 0xebdcbd9ef2670, 0x35dc8fe0df3fe, 0xe458309d20c24, 0x11e87898716a0,-    0x7c44bab7cb456, 0xd64d3cf1bb64, 0x189bff1bf9e66, 0xb5218a049311, 0x285dda6cbcc81,-    0x3238dcafd8c7c, 0x607736c8de0, 0xdb83d99508b1, 0x4e1a0d404cd81, 0x1588008c00ff2,-    0x16b8b36722b27, 0x876609c3f3f1a, 0x66b72ef0e17d6, 0x705f8a279d568, 0x2eaac4cd01fdd,-    0x1171ce9705fe9, 0xffc79cd3264ee, 0x700c8ab4b80f0, 0x208d3d4f57a1, 0x337262a8ca4eb,-    0x297fd01d843fd, 0xa90956fa097f8, 0x529759fdb3845, 0x1d78c5e2d0397, 0x3d6938a4adbf3,-    0x16d5853560b66, 0xf138946b9a430, 0x2ab79f4dea6a0, 0xd42053ee43ae1, 0x3b9c3ef1cf870,-    0x598934ad81baf, 0x5f1821b1d07a7, 0x416bb3a973ff3, 0x23f07bd0a047a, 0x19bdc2e09f786,-    0x56dc9981cd51f, 0xfbace23c8cd65, 0x673bd3bf5b52e, 0x46a95d229fd61, 0xe09ad64bcfb1,-    0xe5292b91f17d, 0xfeefcd8afc287, 0x58f52b0a58711, 0x4800f20c201ef, 0x2084fce608f67,-    0x12ba0b128ae0b, 0x5977ae17030b4, 0x101126ee420f6, 0xf70823495c6bd, 0xde19a27d7770,-    0x5c6ac852260e8, 0x9d22950ac4356, 0x441cca955246c, 0x660a34e5332d9, 0x14ac8ea92f8d2,-    0x6b6d7709f307e, 0x67d7e13879db, 0x2ea8626f9fbbd, 0x99609006a4b40, 0x31bb2a8f8c779,-    0x10c04828ea335, 0xae9acdcbc080a, 0x617af2342607a, 0xc7494ea53e553, 0x2ca9e2872defa,-    0x6c399fab21f1f, 0xab139b245e758, 0x3ad933dcba589, 0x4797fecb08811, 0x31f5dbf8f594,-    0x7dc6361cc7a69, 0xc8a7953ead3f9, 0x79ed693d18015, 0x418a024999a6a, 0x2c4fdc9436aa,-    0x1eb98cb06aa75, 0x2989592796a9c, 0x11194821e425, 0xe27a648228388, 0x35d834b6c12a0,-    0x541807713b532, 0x7ae0a1008aaee, 0x7017a29bcb5e, 0x6b193c23c315c, 0x19bd25ac82f2a,-    0x6a01a43eef294, 0xddf5b5fd84f19, 0x33f5ba081c016, 0xdeb052d1bc082, 0x6b2f06afa617,-    0x7ca1eda6a939f, 0xbdeb35997b50c, 0x47f2d1bccda5, 0xc2ff4adfed667, 0x87712997be4,-    0x21fc2e2b37659, 0xf7d62cd5ed951, 0x27fa9cbdf7efa, 0xba25582bf3a6b, 0x2a42b8bd89398,-    0x6d377d07eecd2, 0x9ca1df5af387, 0x1109e3427e2ba, 0xce4aa4572a19, 0x103baaef71e16,-    0x2c3b2dfde328a, 0xbec4b4a30e1ef, 0x37d92a86204f3, 0x806cfde68eb39, 0x246e2f72b8aa5,-    0x68d3de93462a9, 0x53b8acba6bbc3, 0x2492a70fa1696, 0x38c62d5760f55, 0x15096fe4904f2,-    0x4e44e9bed3e3a, 0xb28bfd79cc9bc, 0x6a77513839320, 0x480dcec6739db, 0x3601b739f2465,-    0x43c348e2a7e1, 0xe448106327879, 0x175d9cae1b0ed, 0xd3b89dee743b8, 0x392d73ca255bc,-    0x32946db0d3a18, 0x9261b09907cc, 0x5ba517a755722, 0x51f24fdaf5184, 0x1cdc732989ed8,-    0x2f7806ba16694, 0xae0c9f029f8d0, 0xd8b45102ce1, 0xca1c7db9316d6, 0x162088a67066f,-    0x39de35b2b4162, 0xa19f550d88ae9, 0x7921b27026cde, 0x94b936b66e900, 0x1023bd5fa17fc,-    0x436837814cfa4, 0x29113492283c4, 0x66d1cdd8b51d8, 0xa540702278eb2, 0x47ef1b29285d,-    0x587b50917e50e, 0xb4cda75bab3b, 0x112520b0a9886, 0x66b9ac16fee49, 0x17bf17e92b2eb,-    0x2456a2f150ed7, 0xfa214412d0280, 0x3ca7dd947fe5b, 0xa72c28598d58a, 0x255d945efc3e,-    0x2873f04e0f215, 0x74178fd1af57b, 0x788848b5b2d6, 0xb1ffafaae0db6, 0x32a1b7b3cbb2a,-    0x4bd9935d6b2da, 0x9c08f24ad30a5, 0x4e58407a80f, 0x1b3a3825a5b17, 0x6547e9fc82f5,-    0x47484aa3656c3, 0x6ee43f341a494, 0x64a98f87adea2, 0x619b3f8e95f01, 0xb6e513266ed8,-    0x421c2a673090, 0xa1c1de32348c7, 0x55b85c3a1e8a3, 0xe05ce8ef330b4, 0x2561e49c15d84,-    0x40aa2d33130fa, 0x12b827d35866f, 0xfe4cf62c8ddb, 0x2fa0ef05bb28d, 0x1c06ca63f1cb8,-    0x32a971863863b, 0xff6fc86830da1, 0x71e7b25a14cf3, 0xea9c5ebb1373a, 0x250bbaa3e1634,-    0x5b5ffeda5b765, 0xf25d2a746331b, 0x115e3a3f43632, 0x67303af43c9d5, 0x14bb538a0e559,-    0x75623687d43b7, 0xa349674a4b38d, 0x613c61829ffc6, 0x689828d8110c7, 0x139115f5af7d5,-    0xf1d856152289, 0x45cbe967168ab, 0x51f38e1680901, 0x34808e8f652b0, 0x1f4a6a921e156,-    0x35dfaf3d8341f, 0xf53ace725cb63, 0x3d86a54eef35b, 0xa103aabaffe2c, 0x2decc36296fbd,-    0x510282be73d6f, 0xd4e6365db206a, 0x4bdc5f5bb8bf3, 0xde7ea32a3aee7, 0x71269e274305,+static const limb kPrecomputed[NLIMBS * 2 * 16 * 2] = {+    0x17d166e01bd76, 0xd59ea12530768, 0x3d8c40217b04, 0x17bef9a4c9338, 0x7ecef3946ccf,+    0x1bb3639cdd45, 0xd7a338c14f5f7, 0x1d44d614250ff, 0xff813fc37580a, 0x16fc126e089f2,+    0x596ebe0b9487d, 0x6794652096fcb, 0x70ca729479eb8, 0x286b775769b0c, 0x365b421ada4b,+    0xfd4f7db081dc, 0x7a1064395526a, 0x4bdf0a89c2d26, 0xac80afd3f5e7a, 0x551466941e3f,+    0x27f44abf15dea, 0x641245c721691, 0x66eaa738302bb, 0x12c08cf44e3db, 0x207ff2c77aa29,+    0x42337261a56fa, 0x93f5442fee6dd, 0x253ccbdeda1ea, 0xbb019d239c3a2, 0x14f564e046543,+    0x19e66e694e7c9, 0x28bfdb62b9438, 0x5fd268170b101, 0x81f5d2fd7a276, 0x481bfa2871fd,+    0x71505f1b1c908, 0x2c741aac5e803, 0x55001687f5a40, 0xe8cb1db73f5b4, 0x236a913685d59,+    0x181c6361dc9, 0xd341d469ec73b, 0x5c0b21ff0159a, 0xa3e263a8ae02a, 0x96fbbe0e6601,+    0x32b9c888138de, 0x895b615971422, 0x3d9623d32b307, 0xd16a8f42e9505, 0x3a1944eacccd1,+    0x61e2d87f8d2bc, 0x1eae233791d5f, 0x67ad41964d0d6, 0x6f3066502d527, 0x13c23333b20b,+    0x50ca4072c4394, 0xc0d087a117391, 0x5d67a89ad2fd8, 0xf8d8c79d1424c, 0x176f356ba05e5,+    0x5a9d222a5d603, 0xd5f5321b86cc6, 0x175dc308523c9, 0x4e9bb0a11ab7, 0x35f0d7602ef2a,+    0x1bee93bab7afc, 0x464492b2d3a81, 0x420a2a9572fd7, 0x9429ec53edb83, 0x396a72e88da3f,+    0xef8a067576f7, 0xcf7c758f24aa7, 0x33553a986a5dd, 0x8df4138a812c6, 0x3b46808adb40f,+    0x39047b9c59f49, 0x728b5e8a37c51, 0x7fb7156d41405, 0x182753d7e3372, 0x3780d8cbc79d8,+    0x6af7074e318b9, 0xc50b946a6a49e, 0x6c2e6f494499b, 0xd457756bc8b2a, 0x20d580d1e02c7,+    0x672c9075eca28, 0x2e6cbcbf6f69b, 0xb159bacf3a63, 0xa9b448864e218, 0x3d31127e348da,+    0x7d4feaf9ba24, 0x3bd8791bb30ee, 0x7d13566f669f9, 0x750907ae43eb2, 0x19b501815f4a7,+    0x285bdb67c14af, 0xd529305a272b0, 0x4fa4682bb4328, 0xecbfa5072e13, 0x260ced527dbd6,+    0x22b864d17e2d6, 0x522de5c3bc90a, 0x7268e14f04f46, 0xb16b5f7c30243, 0x20d2897043203,+    0x57582ca37f0b0, 0xc9033b5213f35, 0x15c9edc5d8c3c, 0x966640653b9cc, 0x24539ebc1e770,+    0x152868860a414, 0x69f3140e5b3ae, 0x464ae0a979c81, 0xed2e180aa6ea7, 0x192f58eee8a5b,+    0x284e9f18cbb94, 0x9be7c8736ffa7, 0xd0645eef8dc8, 0xc888d568342b3, 0x1a0457c447c77,+    0x46a3242f1f708, 0xf0d377facff3b, 0x662ecd6e5da70, 0x84c75a436cd10, 0x2c3e2d2e766a7,+    0x6aefb0cd8bc3a, 0xf46b7ff2e5f40, 0x6f7e3bb188698, 0xc30b505db2b69, 0x2f287ab902ee,+    0x4a42f39462e0b, 0xf346d3c53c248, 0x583dd2f9bab2, 0x9ecf2836699a3, 0x1ebbc6ad3471e,+    0x31cf1b599efe9, 0xa3cbeb4ce45d9, 0x418c8a976e3f3, 0x581becef9befb, 0x12ea3bb24c576,+    0x59f406c45cd82, 0xd375b8996d246, 0x30544ebb0c867, 0x4f74220d0000f, 0x195588ab859cb,+    0x6720921607ca6, 0x6eba95153dc00, 0x439899e75b887, 0xf984c62adc7dd, 0x40d829fadb76,+    0x7baa04fd5f981, 0x5b1f99b6ef91, 0x29377748f7c41, 0xa96cc64bbffbc, 0x1a8ab2ec0e968,+    0x54daaf1e949bc, 0x81f61acb2893d, 0x4ed0f2aef056c, 0x41cd03269ce0e, 0x2a924333a9100,+    0x4bd600512324a, 0x5015bb978401a, 0x370dc4e4c9eb0, 0xb94920618b9ff, 0x386c9f301016b,+    0x151b802e73a15, 0xc2f01e076ffe7, 0x3f6c4272644fe, 0x4c65a96d2810e, 0x3abd30e3d0bdd,+    0x507b582802a4c, 0xdd3a9d6440284, 0xaa79901e9e59, 0x28f954b3df137, 0xdfda79ce1298,+    0x418aa5e56d569, 0x6cda605cd2a84, 0x38a3c8c72727e, 0x5c577b1659af2, 0x29ebd4cc67d3e,+    0x5fe67cbb69948, 0xcbb87ceb0460f, 0x2fb2c44c7e9b9, 0xd94fcdb26c47f, 0x185464b74c699,+    0x439cb4f543d28, 0xd46bc2b92ce79, 0x7add9a636f66a, 0xdf31c41270332, 0x2696553f66dbf,+    0x2783576782589, 0x968618b71297b, 0x44e45e45be560, 0x26efc23b82d1d, 0x1015ba1501f7c,+    0x488bd29858305, 0xf74fe2c48c2bf, 0x4e0542975a40a, 0x6c35069031288, 0x86495d91e4d6,+    0x4901896d5e61b, 0x73cc7582e2dff, 0x5720d009880e3, 0x1cf8b3eeb3330, 0x39d782d92776a,+    0x64cfb1b2b7536, 0x2c925510c2142, 0x15d3fdca3fa2f, 0x7c71f8ad652d2, 0x2c8639f12eb8e,+    0x32be1915c3b16, 0x401ba942b1327, 0x73a61e3dd1e67, 0x57855aa9e04e, 0x6692e349a453,+    0x1708b23f8a1a6, 0x994bfdde94868, 0x51ceeeda593ec, 0x4cdc508073c99, 0x155d6d8b5f390,+    0x3f62bd2bb7dc1, 0x1e9f2950a19ef, 0x3879b3fdeef4a, 0x769c9aebe06f6, 0x2fdbb0a2e42c2,+    0x4b669ca5182c8, 0x9f7d2af918087, 0x47a04688c10e0, 0x6fc09c1c63852, 0x103c53a1b2bb3,+    0x3f6293b845e4a, 0x7eddacd3d44ea, 0x3427d6919a9d0, 0xf18b5ac4b772, 0xc48971f25ff7,+    0x30b22ab28a9ee, 0x684778d576841, 0x219063e835137, 0xabfbe8b9b4cbe, 0xa5ee7bb20c8f,+    0x739e1c031098, 0x4f194a1f5e7e7, 0x7c3800ab2834d, 0x665b9b090387b, 0x3bfcc848c5569,+    0x4d7a0fa6ecdd0, 0x5a3be381deaf4, 0x60c235b9afa21, 0x2b6af4dca92ba, 0x3d4ef541da8da,+    0x15368f12c5e0, 0x1089d55881802, 0x4e950a255aa08, 0xea013ebd20bf4, 0x9bf696b8d0d0,+    0x74755b59e06a4, 0xca3ec0966d7d7, 0x23f8c820bf534, 0x7a9c3d0d130fb, 0x22243e7f72df8,+    0x177eb94dece80, 0x18e56144e85e5, 0x746051389a65, 0xaaa6ded4b788c, 0x2e7e3c710e646,+    0x10535c987ba48, 0xce1ab4a74c92c, 0x355c567052319, 0x3dff053e8baa7, 0x30145933a20bb,+    0x3f829e6bd6bba, 0x5f8c66072ace7, 0x6d4eed643467f, 0xcc0b57a9729d8, 0x63d4c7dcdff6,+    0x6e2f1ed3c2a71, 0xbfed27df0cb8c, 0x57b9be8c8fece, 0xe8296dacbba0e, 0x19ec43845e7cd,+    0x499487bfb2cfd, 0x15507fb8a4c3f, 0x4eb4c133c44c6, 0x46f4d7d05d5e9, 0x1403ec072267b,+    0x42f3db0918e6f, 0x4a12f29990cd, 0x21c078f7b096c, 0x5d7cb674f5b94, 0x4b6ab2b6b85,+    0x64dcbb337dda8, 0xae2a09ce386d8, 0x47d4c764502e5, 0x37ab82e784a4c, 0x279485eef7e12,+    0x6e3a35d64f447, 0xb5f72cb08d802, 0x130922ccaf665, 0xfcb727b232952, 0x3aaa4745db15f,+    0x555f7a3941881, 0x9c78701503430, 0x6109825b920d4, 0x237def01f0a49, 0x2fa436baf03a9,+    0x52f3c5d10d5d8, 0x1249c36d73132, 0x73b2269c79aa7, 0x6425b9fe81796, 0x379a76cb42ac8,+    0x61c0bf3594366, 0xda97bf0917530, 0x60340c80e3c62, 0x6e3d5ac5d53a3, 0x3272121adefa2,+    0x55f37132a6fa0, 0x20c61072d5855, 0x514a0e230d7ba, 0xbad776d17830, 0x1c421a5a5b50f };  @@ -278,7 +319,7 @@  *  * On entry: tmp[i] < 2**128  * On exit: out[0,2,...] < 2**52, out[1,3,...] < 2**53 */-static void felem_reduce_degree(felem out, u128 tmp[9]) {+FELEM_INLINE void felem_reduce_degree(felem out, u128 tmp[9]) {    /* The following table may be helpful when reading this code:     *     * Limb number:   0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10@@ -468,7 +509,7 @@  *  * On entry: in[0,2,...] < 2**52, in[1,3,...] < 2**53.  * On exit: out[0,2,...] < 2**52, out[1,3,...] < 2**53. */-static void felem_square(felem out, const felem in) {+FELEM_INLINE void felem_square(felem out, const felem in) {   u128 tmp[9], x1x1, x3x3;    x1x1 = ((u128) in[1]) * in[1];@@ -496,7 +537,7 @@  * On entry: in[0,2,...] < 2**52, in[1,3,...] < 2**53 and  *           in2[0,2,...] < 2**52, in2[1,3,...] < 2**53.  * On exit: out[0,2,...] < 2**52, out[1,3,...] < 2**53. */-static void felem_mul(felem out, const felem in, const felem in2) {+FELEM_INLINE void felem_mul(felem out, const felem in, const felem in2) {   u128 tmp[9], x1y1, x1y3, x3y1, x3y3;    x1y1 = ((u128) in[1]) * in2[1];
+ cbits/include64/p256/p256_s2n.h view
@@ -0,0 +1,25 @@+/*+ * The vendored s2n-bignum assembly, behind one call that picks the variant+ * the machine wants.  See cbits/s2n/README.md for which and why.+ *+ * Only declared in the 64-bit field build: s2n-bignum is x86-64 and AArch64+ * only, and this interface is the four little-endian 64-bit words those+ * architectures give crypton_p256_int anyway.+ */+#ifndef CRYPTON_P256_S2N_H+#define CRYPTON_P256_S2N_H++#include <stdint.h>++/* res = scalar * point, all of them affine and not in Montgomery form:+ * point is x then y, four words each, and res the same.  The point at+ * infinity goes in and comes out as (0, 0). */+void crypton_s2n_p256_scalarmul(uint64_t res[8], const uint64_t scalar[4],+                                const uint64_t point[8]);++/* res = scalar * G, the same shape out.  The table it reads and the window+ * width it was built for are in cbits/p256/p256_base_table.c, which+ * cbits/p256/gen_base_table.py writes. */+void crypton_s2n_p256_scalarmulbase(uint64_t res[8], const uint64_t scalar[4]);++#endif
+ cbits/p256/gen_base_table.py view
@@ -0,0 +1,126 @@+#!/usr/bin/env python3+"""Build the two tables of multiples of the base point that crypton reads.++    ./gen_base_table.py 5 p256_base_table.c        # for p256_scalarmulbase+    ./gen_base_table.py odd 7 p256_wnaf_table.c    # for p256_verify.c++Its own words for the layout: for each i, j with blocksize*i <= 256 and+1 <= j <= B, where B = 2^(blocksize-1), the multiple 2^(blocksize*i) * j * P+goes at tab + 64*(B*i + (j-1)), as a Montgomery-affine pair, four+little-endian 64-bit words each.++Five is the blocksize crypton ships: on an Apple M4 it is 6.40 microseconds+against 7.15 for four and 6.10 for six, and the table is 52 KiB against 33+and 88.  Nothing outside this file knows the number -- it is written into+the generated file and read from there.++The curve constants come from `openssl ecparam`, not from memory, and the+generated table is checked against crypton's own base-point multiplication+by the test suite, so a mistake here does not pass quietly.+"""+import subprocess, re, sys++def curve():+    out = subprocess.run(["openssl","ecparam","-name","prime256v1",+                          "-param_enc","explicit","-text","-noout"],+                         capture_output=True, text=True).stdout+    f, cur = {}, None+    for line in out.splitlines():+        m = re.match(r'^(Prime|A|B|Generator \(uncompressed\)|Order):?\s*$', line.strip())+        if m:+            cur = m.group(1); f[cur] = ""; continue+        if cur and re.match(r'^\s+[0-9a-f]{2}[:0-9a-f]*:?\s*$', line):+            f[cur] += line.strip()+        elif cur and line and not line.startswith(' '):+            cur = None+    def num(s):+        return int.from_bytes(bytes(int(v,16) for v in s.strip(':').split(':') if v), 'big')+    g = bytes(int(v,16) for v in f['Generator (uncompressed)'].strip(':').split(':') if v)+    assert g[0] == 4 and len(g) == 65+    return (num(f['Prime']), num(f['A']), num(f['B']),+            int.from_bytes(g[1:33],'big'), int.from_bytes(g[33:],'big'), num(f['Order']))++P, A, B, GX, GY, N = curve()+assert (GY*GY - GX**3 - A*GX - B) % P == 0, "the generator is not on the curve"++def add(p, q):+    if p is None: return q+    if q is None: return p+    (x1,y1),(x2,y2) = p,q+    if x1 == x2:+        if (y1 + y2) % P == 0: return None+        l = (3*x1*x1 + A) * pow(2*y1, -1, P) % P+    else:+        l = (y2-y1) * pow(x2-x1, -1, P) % P+    x3 = (l*l - x1 - x2) % P+    return (x3, (l*(x1-x3) - y1) % P)++R = 1 << 256+def mont(v):  return v * R % P+def words(v): return [(v >> (64*k)) & 0xFFFFFFFFFFFFFFFF for k in range(4)]++def table(blocksize):+    Bn = 1 << (blocksize - 1)+    blocks = 256 // blocksize + 1+    out = []+    base = (GX, GY)                      # 2^(blocksize*i) * P+    for i in range(blocks):+        acc = None+        for j in range(1, Bn + 1):+            acc = add(acc, base)         # j * base+            out.append(acc)+        for _ in range(blocksize):+            base = add(base, base)+    return blocks, Bn, out++def odd_table(width):+    """The odd multiples 1P, 3P, ..., (2^(width-1)-1)P, which is what the+    windowed form in cbits/p256/p256_verify.c reads for the base point."""+    n = 1 << (width - 2)+    twice = add((GX, GY), (GX, GY))+    out, acc = [], (GX, GY)+    for _ in range(n):+        out.append(acc)+        acc = add(acc, twice)+    return out++def emit_odd(width, path):+    pts = odd_table(width)+    with open(path, 'w') as fh:+        fh.write("/* Generated by cbits/p256/gen_base_table.py; do not hand-edit.\n"+                 " * The odd multiples of the base point, in Montgomery-affine\n"+                 " * form, which cbits/p256/p256_verify.c reads.  A verification\n"+                 " * is public, so this table is walked in variable time. */\n")+        fh.write("#include <stdint.h>\n\n")+        fh.write(f"const uint64_t crypton_p256_wnaf_width = {width};\n\n")+        fh.write(f"const uint64_t crypton_p256_wnaf_table[{len(pts)*8}] = {{\n")+        for (x, y) in pts:+            ws = words(mont(x)) + words(mont(y))+            fh.write("\t" + ",".join(f"0x{w:016x}ULL" for w in ws) + ",\n")+        fh.write("};\n")+    return len(pts), len(pts) * 64++def emit(blocksize, path):+    blocks, Bn, pts = table(blocksize)+    with open(path, 'w') as fh:+        fh.write("/* Generated by cbits/p256/gen_base_table.py; do not hand-edit.\n"+                 " * The multiples of the base point that s2n-bignum's\n"+                 " * p256_scalarmulbase reads, in Montgomery-affine form. */\n")+        fh.write("#include <stdint.h>\n\n")+        fh.write(f"const uint64_t crypton_p256_s2n_base_blocksize = {blocksize};\n\n")+        fh.write(f"const uint64_t crypton_p256_s2n_base_table[{len(pts)*8}] = {{\n")+        for (x, y) in pts:+            ws = words(mont(x)) + words(mont(y))+            fh.write("\t" + ",".join(f"0x{w:016x}ULL" for w in ws) + ",\n")+        fh.write("};\n")+    return blocks, Bn, len(pts)*64++if __name__ == "__main__":+    if sys.argv[1] == "odd":+        w = int(sys.argv[2]); path = sys.argv[3]+        n, size = emit_odd(w, path)+        print(f"width {w}: {n} odd multiples = {size} bytes")+    else:+        b = int(sys.argv[1]); path = sys.argv[2]+        blocks, Bn, size = emit(b, path)+        print(f"blocksize {b}: {blocks} blocks x {Bn} = {size} bytes")
cbits/p256/p256.c view
@@ -39,6 +39,15 @@  #include "p256/p256.h" +#ifdef CRYPTON_S2N_BIGNUM+extern void bignum_modinv(uint64_t k, uint64_t *z, const uint64_t *a,+                          const uint64_t *b, uint64_t *t);+/* the digits are handed over as they lie */+#if P256_BITSPERDIGIT != 64 || P256_NDIGITS != 4+#error "CRYPTON_S2N_BIGNUM wants the 64-bit crypton_p256_int"+#endif+#endif+ const crypton_p256_int crypton_SECP256r1_n =  // curve order   {{P256_LITERAL(0xfc632551, 0xf3b9cac2), P256_LITERAL(0xa7179e84, 0xbce6faad),     P256_LITERAL(-1, -1), P256_LITERAL(0, -1)}};@@ -104,7 +113,9 @@   borrow += top_c;   borrow -= top_a;   top_c = (crypton_p256_digit)borrow;-  assert((borrow >> P256_BITSPERDIGIT) == 0);+  /* A borrow out is a legitimate outcome: the quotient estimate in+     crypton_p256_modmul can exceed the true quotient by one.  Report it in+     the returned top digit (all ones) and let the caller correct. */   return top_c; } @@ -178,6 +189,13 @@     // Subtract reducer from top | tmp.     top = subTop(top_reducer, reducer, top, tmp + i); +    // The quotient estimate above can exceed the true quotient by one --+    // with 64-bit digits, whenever the low half of top is zero and the+    // digits below it are small -- and the subtraction then borrows.  The+    // deficit is always less than MOD, so adding MOD back once restores+    // the invariant.+    top = addM(MOD, top, tmp + i, 0 - (top >> (P256_BITSPERDIGIT - 1)));+     // top is now either 0 or 1. Make it 0, fixed-timing.     assert(top <= 1); @@ -205,7 +223,11 @@   n %= P256_BITSPERDIGIT;   for (i = P256_NDIGITS - 1; i > 0; --i) {     crypton_p256_digit accu = (P256_DIGIT(a, i) << n);-    accu |= (P256_DIGIT(a, i - 1) >> (P256_BITSPERDIGIT - n));+    /* n is zero as often as it is anything else, and a digit shifted by its+     * own width is undefined: x86 takes the count modulo the width and hands+     * back the whole digit, ARM hands back nothing.  Two shifts that are each+     * inside the width say the intended nothing on both. */+    accu |= (P256_DIGIT(a, i - 1) >> (P256_BITSPERDIGIT - 1 - n) >> 1);     P256_DIGIT(b, i) = accu;   }   P256_DIGIT(b, i) = (P256_DIGIT(a, i) << n);@@ -221,7 +243,8 @@   n %= P256_BITSPERDIGIT;   for (i = 0; i < P256_NDIGITS - 1; ++i) {     crypton_p256_digit accu = (P256_DIGIT(a, i) >> n);-    accu |= (P256_DIGIT(a, i + 1) << (P256_BITSPERDIGIT - n));+    /* the same full-width shift as in crypton_p256_shl above */+    accu |= (P256_DIGIT(a, i + 1) << (P256_BITSPERDIGIT - 1 - n) << 1);     P256_DIGIT(b, i) = accu;   }   P256_DIGIT(b, i) = (P256_DIGIT(a, i) >> n);@@ -235,8 +258,11 @@     accu |= (P256_DIGIT(a, i + 1) << (P256_BITSPERDIGIT - 1));     P256_DIGIT(b, i) = accu;   }+  /* The shift is unsigned: highbit is a carry, zero or one, and one shifted+   * into the sign bit of the signed digit is an overflow the standard leaves+   * undefined.  The value put into b is the same either way. */   P256_DIGIT(b, i) = (P256_DIGIT(a, i) >> 1) |-      (((crypton_p256_sdigit) highbit) << (P256_BITSPERDIGIT - 1));+      (((crypton_p256_digit) highbit) << (P256_BITSPERDIGIT - 1)); }  // Return -1, 0, 1 for a < b, a == b or a > b respectively.@@ -303,6 +329,22 @@   crypton_p256_int U = *MOD;   crypton_p256_int V = *a; +  /* Zero has no inverse, and the loop below never finds that out: V stays+     even forever, so it is halved forever, and the only break is in the+     branch both U and V have to be odd to reach.  The other input without an+     inverse is MOD itself -- 2*MOD does not fit in 256 bits, so there is no+     third -- and that one already leaves here as zero, which is also what+     Crypto.PubKey.ECC.P256's scalarInvSafe answers for both.  Answer the+     same for zero rather than not answering.++     Reachable: Crypto.PubKey.ECC.P256 exports scalarInv, and scalarFromBinary+     accepts any 256 bits.  A hang inside a foreign call cannot be interrupted+     by System.Timeout either. */+  if (crypton_p256_is_zero(a)) {+    crypton_p256_clear(b);+    return;+  }+   for (;;) {     if (crypton_p256_is_even(&U)) {       crypton_p256_shr1(&U, 0, &U);@@ -343,13 +385,12 @@ }  // Verify y^2 == x^3 - 3x + b mod p-// and 0 < x < p and 0 < y < p+// and 0 <= x < p and 0 < y < p int crypton_p256_is_valid_point(const crypton_p256_int* x, const crypton_p256_int* y) {   crypton_p256_int y2, x3;    if (crypton_p256_cmp(&crypton_SECP256r1_p, x) <= 0 ||       crypton_p256_cmp(&crypton_SECP256r1_p, y) <= 0 ||-      crypton_p256_is_zero(x) ||       crypton_p256_is_zero(y)) return 0;    crypton_p256_modmul(&crypton_SECP256r1_p, y, 0, y, &y2);  // y^2@@ -361,6 +402,7 @@   if (crypton_p256_sub(&x3, x, &x3)) crypton_p256_add(&x3, &crypton_SECP256r1_p, &x3);  // x^3 - 3x   if (crypton_p256_add(&x3, &crypton_SECP256r1_b, &x3))  // x^3 - 3x + b     crypton_p256_sub(&x3, &crypton_SECP256r1_p, &x3);+  crypton_p256_mod(&crypton_SECP256r1_p, &x3, &x3);    return crypton_p256_cmp(&y2, &x3) == 0; }@@ -471,6 +513,16 @@  // b = 1/a mod n, using Fermat's little theorem. void crypton_p256e_scalar_invert(const crypton_p256_int* a, crypton_p256_int* b) {+#ifdef CRYPTON_S2N_BIGNUM+  /* The assembly, which takes a fixed number of division steps instead: 0.80+   * microseconds against 6.02 on an Apple M4.  It answers zero where a has no+   * inverse, which is what the chain below does as well, and the caller reads+   * a zero as "no inverse". */+  uint64_t t[12];++  bignum_modinv(4, P256_DIGITS(b), P256_DIGITS(a),+                P256_DIGITS(&crypton_SECP256r1_n), t);+#else   crypton_p256_int _1, _10, _11, _101, _111, _1010, _1111;   crypton_p256_int _10101, _101010, _101111, x6, x8, x16, x32;   int i;@@ -525,4 +577,5 @@    // Demontgomerize   crypton_p256e_montmul(b, &crypton_SECP256r1_one, b);+#endif }
+ cbits/p256/p256_base_table.c view
@@ -0,0 +1,841 @@+/* Generated by cbits/p256/gen_base_table.py; do not hand-edit.+ * The multiples of the base point that s2n-bignum's+ * p256_scalarmulbase reads, in Montgomery-affine form. */+#include <stdint.h>++const uint64_t crypton_p256_s2n_base_blocksize = 5;++const uint64_t crypton_p256_s2n_base_table[6656] = {+	0x79e730d418a9143cULL,0x75ba95fc5fedb601ULL,0x79fb732b77622510ULL,0x18905f76a53755c6ULL,0xddf25357ce95560aULL,0x8b4ab8e4ba19e45cULL,0xd2e88688dd21f325ULL,0x8571ff1825885d85ULL,+	0x850046d410ddd64dULL,0xaa6ae3c1a433827dULL,0x732205038d1490d9ULL,0xf6bb32e43dcf3a3bULL,0x2f3648d361bee1a5ULL,0x152cd7cbeb236ff8ULL,0x19a8fb0e92042dbeULL,0x78c577510a5b8a3bULL,+	0xffac3f904eebc127ULL,0xb027f84a087d81fbULL,0x66ad77dd87cbbc98ULL,0x26936a3fb6ff747eULL,0xb04c5c1fc983a7ebULL,0x583e47ad0861fe1aULL,0x788208311a2ee98eULL,0xd5f06a29e587cc07ULL,+	0x74b0b50d46918dccULL,0x4650a6edc623c173ULL,0x0cdaacace8100af2ULL,0x577362f541b0176bULL,0x2d96f24ce4cbaba6ULL,0x17628471fad6f447ULL,0x6b6c36dee5ddd22eULL,0x84b14c394c5ab863ULL,+	0xbe1b8aaec45c61f5ULL,0x90ec649a94b9537dULL,0x941cb5aad076c20cULL,0xc9079605890523c8ULL,0xeb309b4ae7ba4f10ULL,0x73c568efe5eb882bULL,0x3540a9877e7a1f68ULL,0x73a076bb2dd1e916ULL,+	0x403947373e77664aULL,0x55ae744f346cee3eULL,0xd50a961a5b17a3adULL,0x13074b5954213673ULL,0x93d36220d377e44bULL,0x299c2b53adff14b5ULL,0xf424d44cef639f11ULL,0xa4c9916d4a07f75fULL,+	0x0746354ea0173b4fULL,0x2bd20213d23c00f7ULL,0xf43eaab50c23bb08ULL,0x13ba5119c3123e03ULL,0x2847d0303f5b9d4dULL,0x6742f2f25da67bddULL,0xef933bdc77c94195ULL,0xeaedd9156e240867ULL,+	0x27f14cd19499a78fULL,0x462ab5c56f9b3455ULL,0x8f90f02af02cfc6bULL,0xb763891eb265230dULL,0xf59da3a9532d4977ULL,0x21e3327dcf9eba15ULL,0x123c7b84be60bbf0ULL,0x56ec12f27706df76ULL,+	0x75c96e8f264e20e8ULL,0xabe6bfed59a7a841ULL,0x2cc09c0444c8eb00ULL,0xe05b3080f0c4e16bULL,0x1eb7777aa45f3314ULL,0x56af7bedce5d45e3ULL,0x2b6e019a88b12f1aULL,0x086659cdfd835f9bULL,+	0x2c18dbd19dc21ec8ULL,0x98f9868a0fcf8139ULL,0x737d2cd648250b49ULL,0xcc61c94724b3428fULL,0x0c2b407880dd9e76ULL,0xc43a8991383fbe08ULL,0x5f7d2d65779be5d2ULL,0x78719a54eb3b4ab5ULL,+	0xea7d260a6245e404ULL,0x9de407956e7fdfe0ULL,0x1ff3a4158dac1ab5ULL,0x3e7090f1649c9073ULL,0x1a7685612b944e88ULL,0x250f939ee57f61c8ULL,0x0c0daa891ead643dULL,0x68930023e125b88eULL,+	0x04b71aa7d2697768ULL,0xabdedef5ca345a33ULL,0x2409d29dee37385eULL,0x4ee1df77cb83e156ULL,0x0cac12d91cbb5b43ULL,0x170ed2f6ca895637ULL,0x28228cfa8ade6d66ULL,0x7ff57c9553238acaULL,+	0xccc425634b2ed709ULL,0x0e356769856fd30dULL,0xbcbcd43f559e9811ULL,0x738477ac5395b759ULL,0x35752b90c00ee17fULL,0x68748390742ed2e3ULL,0x7cd06422bd1f5bc1ULL,0xfbc08769c9e7b797ULL,+	0xa242a35bb0cf664aULL,0x126e48f77f9707e3ULL,0x1717bf54c6832660ULL,0xfaae7332fd12c72eULL,0x27b52db7995d586bULL,0xbe29569e832237c2ULL,0xe8e4193e2a65e7dbULL,0x152706dc2eaa1bbbULL,+	0x72bcd8b7bc60055bULL,0x03cc23ee56e27e4bULL,0xee337424e4819370ULL,0xe2aa0e430ad3da09ULL,0x40b8524f6383c45dULL,0xd766355442a41b25ULL,0x64efa6de778a4797ULL,0x2042170a7079adf4ULL,+	0x808b0b650bc6fb80ULL,0x5882e0753ffe2e6bULL,0xd5ef2f7c2c83f549ULL,0x54d63c809103b723ULL,0xf2f11bd652a23f9bULL,0x3670c3194b0b6587ULL,0x55c4623bb1580e9eULL,0x64edf7b201efe220ULL,+	0xd8c5fccfc5e3a3d8ULL,0xbefd904c4079dfbfULL,0xbc6d6a58fead0197ULL,0x39227077695532a4ULL,0x09e23e6ddbef42f5ULL,0x7e449b64480a9908ULL,0x7b969c1aad9a2e40ULL,0x6231d7929591c2a4ULL,+	0xdb6d96f305968b80ULL,0x380a0913089f73b9ULL,0x7da70b83c2c61e01ULL,0x95fb8394569b38c7ULL,0x9a3c651280edfe2fULL,0x8f726bb98faeaf82ULL,0x8010a4a078424bf8ULL,0x296720440e844970ULL,+	0x802b8d2333e12b70ULL,0x6d490a4b19dd329bULL,0x14f356cc6abc354dULL,0x11eddf7fd0a0da0dULL,0x1e208328d87fd1d8ULL,0xfd2f4f8cfd025813ULL,0x03b48cc47c29bca2ULL,0x3f2a78b3241a2b71ULL,+	0x63c5cb817a2ad62aULL,0x7ef2b6b9ac62ff54ULL,0x3749bba4b3ad9db5ULL,0xad311f2c46d5a617ULL,0xb77a8087c2ff3b6dULL,0xb46feaf3367834ffULL,0xf8aa266d75d6b138ULL,0xfa38d320ec008188ULL,+	0xc04afa1ae3451a09ULL,0x7cc69103bc117423ULL,0x876be3aa51cf56eeULL,0xe7577d57ad844a25ULL,0x266fed8cdb77f341ULL,0xcfa258dc23ae4a2aULL,0x53a7a98cda782760ULL,0x04b48868ceaf7d4aULL,+	0xc0f2affc4e6916c6ULL,0x6fb94957811842daULL,0x6034bcb624b4d157ULL,0xde2efdc7992efb90ULL,0xd66f7eceac793c87ULL,0x02f026267dc6fdcdULL,0x90d3235c9aa1c501ULL,0xf6e494962b4666f0ULL,+	0x082736d19c0859c3ULL,0x89ea5516b269386aULL,0xf25071871aa87b33ULL,0xe9d82f5f704e8236ULL,0x7834612442e855f5ULL,0x209f50fe395e00d8ULL,0xcd9e03aae6e7e62bULL,0xb4b4959e5e5be37bULL,+	0x486d8ffa696946fcULL,0x50fbc6d8b9cba56dULL,0x7e3d423e90f35a15ULL,0x7c3da195c0dd962cULL,0xe673fdb03cfd5d8bULL,0x0704b7c2889dfca5ULL,0xf6ce581ff52305aaULL,0x399d49eb914d5e53ULL,+	0x7966afbb10e6d950ULL,0x37e4a4c4e2bf970aULL,0x23d0c8559d54ca2aULL,0x13d62865fee39a10ULL,0x15f53c38d3bd15e9ULL,0x014b8bed84a80bccULL,0x10674c77bfd8f608ULL,0x4dfab986c93fbfefULL,+	0x24c97c367b92d453ULL,0xd2c271a2249a26c0ULL,0x60eb4b2b89d14a39ULL,0x1198de20432e8005ULL,0x9eabea75799b80d9ULL,0xab6e0c2b8f826ae5ULL,0xca004eedd10061ecULL,0x31a9f439e99c4fd8ULL,+	0x734c8b75b5498a7cULL,0xaeccad8a29f64c2dULL,0x95dd54fa295b1677ULL,0x383902a0b4c54968ULL,0x78cb276feb9d33a9ULL,0x00aedca1af552869ULL,0xa01d14594c5c1630ULL,0xfeba17067cf7d50dULL,+	0x380a496d6ec293cdULL,0x733dbda78e7051f5ULL,0x037e388db849140aULL,0xee4b32b05946dbf6ULL,0xb1c4fda9cae368d1ULL,0x5001a7b0fdb0b2f3ULL,0x6df593742e3ac46eULL,0x4af675f239b3e656ULL,+	0x1a1fffdcc01b0a46ULL,0x07ad675f83f843c2ULL,0xbcec2d076738e81aULL,0x910aec75aa8b7da8ULL,0x13b4d740a4509ba7ULL,0x057010734c7b8216ULL,0x3d75c8f71591f1e5ULL,0x134c1b6f0dfe1d90ULL,+	0xb20e7c44d67826ebULL,0x1212d3cfac379637ULL,0x614f67877de03a5dULL,0x7538a2fc802baa26ULL,0x133c37a19d252415ULL,0x7db390506eb4b587ULL,0x5d40d7574c49d1d9ULL,0xf1126b99a801c4baULL,+	0x39a7aeb9c56729fcULL,0x21a59448e7a8bd85ULL,0xd7c6da5d049d10c6ULL,0x93a4c4a8c5197afbULL,0xd185539c25933861ULL,0x4994bcba27494d9dULL,0xef033de14ce7bfa9ULL,0x22e9b000321f9236ULL,+	0x44e3811039949296ULL,0x5b63827b361db1b5ULL,0x3e5323ed206eaff5ULL,0x942370d2c21f4290ULL,0xf2caaf2ee0d985a1ULL,0x192cc64b7239846dULL,0x7c0b8f47ae6312f8ULL,0x7dc61f9196620108ULL,+	0x4f7081e144cc3addULL,0xd5ffa1d687be82cfULL,0x89890b6c0edd6472ULL,0xada26e1a3ed17863ULL,0x276f271563483caaULL,0xe6924cd92f6077fdULL,0x05a7fe980a466e3cULL,0xf1c794b0b1902d1fULL,+	0x3d2b24b9eb7926b8ULL,0xbff88cb3cdbe5509ULL,0xd0f399afe4dd640bULL,0x3c5fe1302f76ed45ULL,0x6f3562f43764fb3dULL,0x7b5af3183151b62dULL,0xd5bd0bc7d79ce5f3ULL,0xfdaf6b20ec66890fULL,+	0x32027fe891e5d7d3ULL,0xf14b7d1773a07678ULL,0xf88497b3c0dfdd61ULL,0xf7c2eec02a8c4f48ULL,0xaa5573f43756e621ULL,0xc013a2401825b948ULL,0x1c03b34563878572ULL,0xa0472bea653a4184ULL,+	0x6772b0e5ab4b35a2ULL,0x1d8b6001f5eeaacfULL,0x728f7ce4795b9580ULL,0x4a20ed2a41fb81daULL,0x9f685cd44fec01e6ULL,0x3ed7ddcca7ff50adULL,0x460fd2640c2d97fdULL,0x3a241426eb82f4f9ULL,+	0x29ae2cf983dfedc9ULL,0xf84371348d87631aULL,0xaf5717117429c8d2ULL,0x18d15867146d9272ULL,0x83053ecf69769bb7ULL,0xc55eb856c479ab82ULL,0x5ef7791c21b0f4b2ULL,0xaa5956ba3d491525ULL,+	0x84cfbfa1c5c5ea50ULL,0xd3baf14c67960681ULL,0x263984030dd50942ULL,0xe4b7839c4716a663ULL,0xd5f1f794e7de6dc0ULL,0x5cd0f4d4622aa7ceULL,0x5295f3f159acfeecULL,0x8d933552953e0607ULL,+	0xe652533b3cef0d7dULL,0xd94f7b182bbb4381ULL,0x838752be0e80f500ULL,0x8e6e24889e9c9bfbULL,0xc975169716caca6aULL,0x866c49d838531ad9ULL,0xc917e2397151ade1ULL,0x2d016ec16037c407ULL,+	0x80009862d5d721d5ULL,0x0c3357a35bd3a182ULL,0x27f3a83b7aa2cda4ULL,0xb58ae74ef6f83085ULL,0x2a911a812e6dad6bULL,0xde286051f43d6c5bULL,0x4bdccc41f996c4d8ULL,0xe7312ec00ae1e24eULL,+	0x903f6e3960e913afULL,0xb2b58bee98bf140dULL,0x9deff025354890b8ULL,0x155810068d2e924eULL,0xb5755db493c95e5bULL,0x3fac42f0dae20eb8ULL,0x9377c8c109b6d8e0ULL,0xa43e2b46ab47ceffULL,+	0x5f57b2fbfacfa459ULL,0x874b1498c1b5aa6bULL,0xb9e89acac4db2092ULL,0x1362bf8ddf4381daULL,0x25d76830b76328a0ULL,0x38188b7098572ae4ULL,0xb43e941429132f7dULL,0x7895a29f22dd42c9ULL,+	0xcbde78dd5e22cbb2ULL,0xf449c85b76bb4391ULL,0x4289f357b6a4273bULL,0x9fce23fd48e84a19ULL,0xcfc32730939eb3b4ULL,0x8b3d982c16c32280ULL,0x5ac234bad5f1346cULL,0x781954b470769fc9ULL,+	0x6faf68feaae6ee70ULL,0x78f4cc155602b0c9ULL,0x7e3321a86e94052aULL,0x2fb3a0d6734d5d80ULL,0xf3b98f3bb25a43baULL,0x30bf803119ee2951ULL,0x7ffee43321b0612aULL,0x12f775e42eb821d0ULL,+	0x4fdff805f57209b5ULL,0x9bd65ac3f952ac8dULL,0x02a3abd3c7969a6fULL,0x1359927ef523775fULL,0xe09b463f88d2e861ULL,0x661d2199623287c3ULL,0x821e64495a70eb7aULL,0x0afbbb1dd67dc684ULL,+	0x7418e3d3acff89f9ULL,0x227f16aed852251fULL,0xdd5bc6e4eb84658bULL,0xf066b9c8f90a9f7eULL,0xc2369071800a7f87ULL,0x383ddc0d5a72862aULL,0x5b48465d8a776da5ULL,0x3d82f64f5e2d8318ULL,+	0x5852104b87453b28ULL,0x073e8128b387344dULL,0x300e78e4817cfc08ULL,0x3a82ed4799362088ULL,0xe222304c88de46a4ULL,0x666c94fd57fadf4aULL,0x40b2d08ea0c8e108ULL,0x4b2955b909e050faULL,+	0xf8d112e76e6485b3ULL,0x4d3e24db771c52f8ULL,0x48e3ee41684a2f6dULL,0x7161957d21d95551ULL,0x19631283cdb12a6cULL,0xbf3fa8822e50e164ULL,0xf6254b633166cc73ULL,0x3aefa7aeaee8cc38ULL,+	0x79b0fe623b36f9fdULL,0x26543b23fde19fc0ULL,0x136e64a0958482efULL,0x23f637719b095825ULL,0x14cfd596b6a1142eULL,0x5ea6aac6335aac0bULL,0x86a0e8bdf3081dd5ULL,0x5fb89d79003dc12aULL,+	0x0f0165fce3779ee3ULL,0xe00e7f9dbd495d9eULL,0x1fa4efa220284e7aULL,0x4564bade47ac6219ULL,0x90e6312ac4708e8eULL,0x4f5725fba71e9adfULL,0xe95f55ae3d684b9fULL,0x47f7ccb11e94b415ULL,+	0xda3a77e5522e6b69ULL,0x69c908c3bbcd6c18ULL,0x1f1b9e48d924fd56ULL,0x37c64e36aa4bb3f7ULL,0x5a4fdbdfee478d7dULL,0xba75c8bc0193f7a0ULL,0x84bc1e8456cd16dfULL,0x1fb08f0846fad151ULL,+	0x3617890361a341c1ULL,0x3604dc600cfd6142ULL,0x022295eb8533316cULL,0x3dbde4ac44af2922ULL,0x898afc5d1c7eef69ULL,0x58896805d14f4fa1ULL,0x05002160203c21caULL,0x6f0d1f3040ef730bULL,+	0x48201b4b12cfe297ULL,0x3eee129c292f74e5ULL,0xe1fe114ec9e874e8ULL,0x899b055c92c5fc41ULL,0x4e477a643a39c8cfULL,0x82f09efe78963cc9ULL,0x6fd3fd8fd333f863ULL,0x85132b2adc949c63ULL,+	0xbd9b8b1dbe7a2af3ULL,0xec51caa94fb74a72ULL,0xb9937a4b63879697ULL,0x7c9a9d20ec2687d5ULL,0x1773e44f6ef5f014ULL,0x8abcf412e90c6900ULL,0x387bd0228142161eULL,0x50393755fcb6ff2aULL,+	0x766e072232398baaULL,0x205fee425cfca031ULL,0xa49f53417a029cf2ULL,0xa88c68b84023890dULL,0xbc2750417337aaa8ULL,0x9ed364ad0eb384f4ULL,0xe0816f8529aba92fULL,0x2e9e194104e38a88ULL,+	0xfabf770977f7195aULL,0x8ec86167adeb838fULL,0xea1285a8bb4f012dULL,0xd68835039a3eab3fULL,0xee5d24f8309004c2ULL,0xa96e4b7613ffe95eULL,0x0cdffe12bd223ea4ULL,0x8f5c2ee5b6739a53ULL,+	0xecace1dda1887395ULL,0x40960f36932a65deULL,0x9611ff5c3aa95529ULL,0xc58215b07c1e5a36ULL,0xd48c9b58f0e1a524ULL,0xb406856bf590dfb8ULL,0xc7605e049cd95662ULL,0x0dd036eea33ecf82ULL,+	0x3d61333959145a65ULL,0xcd9bc368fa406337ULL,0x82d11be32d8a52a0ULL,0xf6877b2797a1c590ULL,0x837a819bf5cbdb25ULL,0x2a4fd1d8de090249ULL,0x622a7de774990e5fULL,0x840fa5a07945511bULL,+	0xfe2893277946d3f9ULL,0xe132bd2407472273ULL,0xeeeb510c1eb6ae86ULL,0x777708c5f0595067ULL,0x18e2c8cd1297029eULL,0x2c61095cbbf9305eULL,0xe466c2586b85d6d9ULL,0x8ac06c36da1ea530ULL,+	0xe58e90b36b0cf82eULL,0x6438d2462615b5e7ULL,0x07b1f8fc669c145aULL,0xb0d8b2da36f1e1cbULL,0x54d5dadbd9184c4dULL,0x3dbb18d5f93d9976ULL,0x0a3e0f56d1147d47ULL,0x2afa8c8da0a48609ULL,+	0x871239ad653ae326ULL,0x14bcf72aa74cbb43ULL,0x8737650e20d4c083ULL,0x3df86536110ed4afULL,0xd2d86fe7b53ca555ULL,0x688cb00dabd5d538ULL,0xcf81bda31ad38468ULL,0x7ccfe3ccf01167b6ULL,+	0x26e08c07e3533d77ULL,0xd7222e6a2e341c99ULL,0x9d60ec3d8d2dc4edULL,0xbdfe0d8f7c476cf8ULL,0x1fe59ab61d056605ULL,0xa9ea9df686a8551fULL,0x8489941e47fb8d8cULL,0xfeb874eb4a7f1b10ULL,+	0x61fc181060a71676ULL,0xe852d1a8f66a8ad1ULL,0x172bbd656417231eULL,0x0d6de7bd3babb11fULL,0x6fde6f88c8e347f8ULL,0x1c5875479bd99cc3ULL,0x78e54ed034076950ULL,0x97f0f334796e83baULL,+	0xed406aa9bd763802ULL,0xc21486a065303da1ULL,0x61ae291ec7e62ec4ULL,0x622a0492df99333eULL,0x7fd80c9dbb7a8ee0ULL,0xdc2ed3bc6c01aedbULL,0x35c35a1208be74ecULL,0xd540cb1a469f671fULL,+	0xa7a8746a584c5e20ULL,0x267e4ea1b9dc7035ULL,0x593a15cfb9548c9bULL,0x5e6e21354bd012f3ULL,0xdf31cc6a8c8f936eULL,0x8af84d04b5c241dcULL,0x63990a6f345efb86ULL,0x6fef4e61b9b962cbULL,+	0xf6368f0925722608ULL,0x131260db131cf5c6ULL,0x40eb353bfab4f7acULL,0x85c7888037eee829ULL,0x4c1581ffc3bdf24eULL,0x5bff75cbf5c3c5a8ULL,0x35e8c83fa14e6f40ULL,0xb81d1c0f0295e0caULL,+	0xf2efe23d442a8ad1ULL,0xc3816a7d06b9c164ULL,0xa9df2d8bdc0aa5e5ULL,0x191ae46f120a8e65ULL,0x83667f8700611c5bULL,0x83171ed7ff109948ULL,0x33a2ecf8ca695952ULL,0xfa4a73eef48d1a13ULL,+	0xfcde7cc8f43a730fULL,0xe89b6f3c33ab590eULL,0xc823f529ad03240bULL,0x82b79afe98bea5dbULL,0x568f2856962fe5deULL,0x0c590adb60c591f3ULL,0x1fc74a144a28a858ULL,0x3b662498b3203f4cULL,+	0x48fc4ed082dd1b6aULL,0x5783a13867b703afULL,0x2463cb9a005d6aaaULL,0xd31ec55c706ecd43ULL,0x9f8ed33f8e9a7641ULL,0x625453ed098d9e7aULL,0xa3beade4ec887493ULL,0x442b80505a795566ULL,+	0x91e3cf0d6c39765aULL,0xa2db3acdac3cca0bULL,0x288f2f08cb953b50ULL,0x2414582ccf43cf1aULL,0x8dec8bbc60eee9a8ULL,0x54c79f02729aa042ULL,0xd81cd5ec6532f5d5ULL,0xa672303acf82e15fULL,+	0x46df582d3bfab839ULL,0x92474e042f8adadeULL,0x36a7766a147a1bc3ULL,0xb6940f540dc0f979ULL,0x44738ef2f2759f25ULL,0x9dd95789a719f4c6ULL,0x2859b7f40750c345ULL,0x5e788bf2b22180d5ULL,+	0x376aafa8719c0563ULL,0xcd8ad2dcbc5fc79fULL,0x303fdb9fcb750cd3ULL,0x14ff052f4418b08eULL,0xf75084cf3e2d6520ULL,0x7ebdf0f8144ed509ULL,0xf43bf0f2d3f25b98ULL,0x86ad71cfa354d837ULL,+	0xa839c9fdfd67ca25ULL,0x023e626860f2015cULL,0x2414a7930e7b2a65ULL,0x92dbe372b13edcbbULL,0xf64981ee64c2200fULL,0x94fb9cdf8446f2f3ULL,0x01411a6a3f1367bbULL,0x7985c1915a1e8331ULL,+	0xb827fe9226f43572ULL,0xdfd3ab5b5d824758ULL,0x315dd23a539094c1ULL,0x85c0e37a66623d68ULL,0x575c79727be19ae0ULL,0x616a3396df0d36b5ULL,0xa1ebb3c826b1ff7eULL,0x635b9485140ad453ULL,+	0xc8123c6037e2efeaULL,0x8d49b502034a96f6ULL,0x466a346b973e4a95ULL,0xf176b5bab7de00ffULL,0x1c58fa3b82dfa945ULL,0x2eb27a9609e429aeULL,0x57c67a67a12b187cULL,0xb155ba82e2298bbaULL,+	0x92bf3cdada430c0bULL,0x4702850e3a96dac6ULL,0xc91cf0a515ac326aULL,0x95de4f49ab8c25e4ULL,0xb01bad09e265c17cULL,0x24e45464087b3881ULL,0xd43e583ce1fac5caULL,0xe17cb3186ead97a6ULL,+	0xf1a542073d99bcfaULL,0x59db703ce8becf6dULL,0x2e455142d2459569ULL,0xb0ee5143a901b910ULL,0xfc05d451e26d994fULL,0x7a6062b41360caafULL,0xdf1ded5f4fa639b1ULL,0xaf930348d335b8b0ULL,+	0x6cc3924374dcec46ULL,0x33cfc02d54c2b73fULL,0x82917844f26cd99cULL,0x8819dd95d1773f89ULL,0x09572aa60871f427ULL,0x8e0cf365f6f01c34ULL,0x7fa52988bff1f5afULL,0x4eb357eae75e8e50ULL,+	0x3d8f248a21fd0861ULL,0xade3bd649bd5a4b6ULL,0xcb56c953c2e2a6bfULL,0x699cd2b5287d6c5fULL,0xdebce1be47d05e8fULL,0x1a4fbb13a8f53732ULL,0x97163beaa5852b08ULL,0x92c49e6ceec6987aULL,+	0xd9d0c8c4868af75dULL,0xd7325cff45c8c7eaULL,0xab471996cc81ecb0ULL,0xff5d55f3611824edULL,0xbe3145411977a0eeULL,0x5085c4c5722038c6ULL,0x2d5335bff94bb495ULL,0x894ad8a6c8e2a082ULL,+	0x540234b22c11bb37ULL,0x2d0366dded4c74a3ULL,0xf9a968daeec5f25dULL,0x3660106867b63142ULL,0x07cd6d2c68d7b6d4ULL,0xa8f74f090c842942ULL,0xe27514047768b1eeULL,0x4b5f7e89fe62aee4ULL,+	0xf2369f0b879fbbedULL,0x0ff0ae86da9d1869ULL,0x5251d75956766f45ULL,0x4984d8c02be8d0fcULL,0x7ecc95a6d21008f0ULL,0x29bd54a03a1a1c49ULL,0xab9828c5d26c50f3ULL,0x32c0087c51d0d251ULL,+	0x47feeb6662b5f3afULL,0xcefab5610abb3734ULL,0x449de60e19f35cb1ULL,0x39f8db14157f0eb9ULL,0xffaecc5b3c61bfd6ULL,0xa5a4d41d41216703ULL,0x7f8fabed224e1cc2ULL,0x0d5a8186871ad953ULL,+	0x190d8ea601799a52ULL,0xa20cec41b86d2952ULL,0x3062ffb27fff2a7cULL,0x741b32e579f19d37ULL,0xf80d81814eb57d47ULL,0x7a2d0ed416aef06bULL,0x09735fb01cecb588ULL,0x1641caaac6061f5bULL,+	0xae2ad171656e8c3aULL,0xc0e2a4631acd0705ULL,0x006f6a8aa0b6055cULL,0xaf4513d72b65a26eULL,0x3f549e14d616d5bcULL,0x64ee395571253b1fULL,0xe8b10bc1b8ce243aULL,0xbcbeace5913a4e77ULL,+	0x68d32256f779d6a7ULL,0x20423b4d19e7284eULL,0xde19aa1b38f3b153ULL,0x73d0b6c28444f703ULL,0x666161489c64e6a3ULL,0x99587c3737256224ULL,0x6f5277fe61331563ULL,0x7174ad4aa656502eULL,+	0xec1ffc920133918dULL,0x15d9bf5eb2c9ef97ULL,0x7885b542fc6cbb9aULL,0x8abe64535720cf5dULL,0x4e715dced4ec68abULL,0x57a67614279c24a3ULL,0x788ed52a31bb61cfULL,0xaba82444f437a003ULL,+	0x7f99824f20151427ULL,0x206828b692430206ULL,0xaa9097d7e1112357ULL,0xacf9a2f209e414ecULL,0xdbdac9da27915356ULL,0x7e0734b7001efee3ULL,0x54fab5bbd2b288e2ULL,0x4c630fc4f62dd09cULL,+	0x6d883e66bd70a5e1ULL,0x9bd884c1d05de713ULL,0x48d9d445d4d8487dULL,0x8440cd8b0eeae405ULL,0xa3cd0f293d26f83eULL,0x3fd9453022a3c5e4ULL,0x63a078663ffa2ca7ULL,0xbfeadd2900d4a097ULL,+	0x13aea559d68432baULL,0x940043fbd33915dbULL,0x476c7c94e2e8da08ULL,0x6b1630c4443065bbULL,0xd6d8546d19f06c75ULL,0x7e1e98c22eb35abdULL,0x751c9310f0157d8eULL,0xdef84327f6fa6075ULL,+	0x765b993c3a7a4e0eULL,0xfb8f4aef9d79d314ULL,0x5ea2c50e2a19eb24ULL,0x925016e5b891bc8cULL,0x8f70afdc3ad1ece7ULL,0x0abba84cf08fabe2ULL,0x0a9922c815f6132bULL,0x6c0213735d076d77ULL,+	0x00af2615a4453961ULL,0x1705494b993d112cULL,0x0032e12aa1cdd652ULL,0x0ebfa612046d9cb3ULL,0x9f03a9f31b63728cULL,0x8f3618a2de022b05ULL,0xd5b24d903e77c5c9ULL,0x837838a372acb77cULL,+	0xfe65cbd98fceb047ULL,0xde872ef53f8b11a1ULL,0x9aba0d3d8fec802fULL,0x139f1d329a9f381dULL,0x0721aed9c587958dULL,0x066a015cb9f6a7daULL,0x059ec4e3ed5d9d06ULL,0x144285716cbaca1fULL,+	0x66fe0fffe298cdf5ULL,0x3f61bea47b2e51b6ULL,0x7d372117bad3afa4ULL,0x6521a09cef656e2fULL,0xb3b8c966e8a58fe7ULL,0x25203a115a47ebc7ULL,0xfe81588d5c4be573ULL,0x6132e2f31f49a03cULL,+	0xec1ab0130f8c2c99ULL,0x60e8968ff17725c2ULL,0xe1a4a593940a980bULL,0x15ed15b020e9ccb1ULL,0x77d754cc64a00becULL,0x90d09c3341687382ULL,0x294fe02dc31fb651ULL,0x8372cd1a1c94ae53ULL,+	0x8537107a1ac2703bULL,0xb49258d86bc857b5ULL,0x57df14debcdaccd1ULL,0x24ab68d7c4ae8529ULL,0x7ed8b5d4734e59d0ULL,0x5f8740c8c495cc80ULL,0x84aedd5a291db9b3ULL,0x80b360f84fb995beULL,+	0x55d5c68da61a76faULL,0x598b441dca1554dcULL,0xd39923b9773b279cULL,0x33331d3c36bf9efcULL,0x2d4c848e298de399ULL,0xcfdb8e77a1a27f56ULL,0x94c855ea57b8ab70ULL,0xdcdb9dae6f7879baULL,+	0x4c4f07367f636a38ULL,0x9f943fb70e76d5cbULL,0xb03510baa8b68b8bULL,0xc246780a9ed07a1fULL,0x3c0514156d549fc2ULL,0xc2953f31607781caULL,0x955e2c69d8d95413ULL,0xb300fadc7bd282e3ULL,+	0xa14b1163c27901b4ULL,0xfd9236e0899b8bf3ULL,0x42b091eccbc6da0aULL,0xbb1dac6f5ad1d297ULL,0x80e61d53a91cf76eULL,0x4110a412d31f1ee7ULL,0x2d87c3ba13efcf77ULL,0x1f374bb4df450d76ULL,+	0x202886024147519aULL,0xd0981eac26b372f0ULL,0xa9d4a7caa785ebc8ULL,0xd953c50ddbdf58e9ULL,0x9d6361ccfd590f8fULL,0x72e9626b44e6c917ULL,0x7fd9611022eb64cfULL,0x863ebb7e9eb288f3ULL,+	0x15e10a0a097e4403ULL,0xcb3d0a8619854665ULL,0x88d8e211d67d4826ULL,0xb39af66e0b9d2839ULL,0xa5f94588bd475ca8ULL,0xe06b7966c077b80bULL,0xfedb1485da27c26cULL,0xd290d33afe0fd5e0ULL,+	0x686202f31306583dULL,0x05b10da0437c622eULL,0xbf9aaa0f076a7bc8ULL,0x25e94efb8f8f4e43ULL,0x8a35c9b7fa3dc26dULL,0xe0e5fb9396ff03c5ULL,0xa77e3843ebc394ceULL,0xcede65958361de60ULL,+	0xbf9075090c22b540ULL,0x2cde42aab7c267d4ULL,0xba18f9ed5ab0d693ULL,0x3ba62aa66e4660d9ULL,0xb24bf97bab9ea96aULL,0x5d039642e3b60e32ULL,0x4e6a45067c4d9bd5ULL,0x666c5b9e7ed4a6a4ULL,+	0x877b7cf5678a31b0ULL,0xd50301ae3998b620ULL,0x734257c5c00fb396ULL,0xf9fb18a004e672a6ULL,0xff8bd8ebe8758851ULL,0x1e64e4c65d99ba44ULL,0x4b8eaedf7dfd93b7ULL,0xba2f2a9804e76b8cULL,+	0xae02a2f09b56ddbdULL,0x1339b5ac8a2f1cf3ULL,0xf2b569c7839dff0dULL,0xb0b9e864fee9a43dULL,0x4ff8ca4177bb064eULL,0x145a2812fd249f63ULL,0x3ab7beacf86f689aULL,0x9bafec2701d35f5eULL,+	0xbba23fddae57c7b7ULL,0x345342f21b932522ULL,0xfd9c80fe556d4aa3ULL,0xa03907ba6525bb61ULL,0x38b010e1ff218933ULL,0xc066b654aa52117bULL,0x8e14192094f2e6eaULL,0x66a27dca0d32f2b2ULL,+	0xb2e6b121fbabbe92ULL,0x281850fbe1330076ULL,0x093581ec97890015ULL,0x69b1dded75ff77f5ULL,0x7cf0b18fab105105ULL,0x953ced31a89ccfefULL,0x3151f85feb914009ULL,0x3c9f1b8788ed48adULL,+	0xa18f07e0e90fb21eULL,0x00fd2b80bba7fca1ULL,0x20387f2795cd67b5ULL,0x5b89a4e7d39707f7ULL,0x8f83ad3f894407ceULL,0xa0025b946c226132ULL,0xc79563c7f906c13bULL,0x5f548f314e7bb025ULL,+	0xff7589494fde0c1fULL,0xbf8a1abee5b6ec20ULL,0x702278fb87e1db6cULL,0xc447ad7a35ed658fULL,0x48d4aa3803d0ccf2ULL,0x80acb338819a7c03ULL,0x9bc7c89e6e17ceccULL,0x46736b8b03be1d82ULL,+	0x2126f742d43b5eaaULL,0x054a0766dfa59b85ULL,0x9d0d5e36126bfd45ULL,0xa1f8fbd7384f8a8fULL,0x317680f5d563fcccULL,0x48ca5055f280a928ULL,0xe00b81b227b578cfULL,0x10aad9182994a514ULL,+	0xe63f301f358bcdc0ULL,0x07689e990a9d47f8ULL,0x1f689e2f4f43d43aULL,0x4d542a1690920904ULL,0xaea293d59ca0a707ULL,0xd061fe458ac68065ULL,0x1033bf1b0090008cULL,0x29749558c08a6db6ULL,+	0x0ee6d3a7c35d8794ULL,0x042e65580356bae5ULL,0x9f59698d643322fdULL,0x9379ae1550a61967ULL,0x64b9ae62fcc9981eULL,0xaed3d6316d2934c6ULL,0x2454b3025e4e65ebULL,0xab09f647f9950428ULL,+	0xb2083a1222248accULL,0x1f6ec0ef3264e366ULL,0x5659b7045afdee28ULL,0x7a823a40e6430bb5ULL,0x24592a04e1900a79ULL,0xcde09d4ac9ee6576ULL,0x52b6463f4b5ea54aULL,0x1efe9ed3d3ca65a7ULL,+	0xe27a6dbe305406ddULL,0x8eb7dc7fdd5d1957ULL,0xf54a6876387d4d8fULL,0x9c479409c7762de4ULL,0xbe4d5b5d99b30778ULL,0x25380c566e793682ULL,0x602d37f3dac740e3ULL,0x140deabe1566e4aeULL,+	0x4481d067afd32acfULL,0xd8f0fccae1f71ccfULL,0xd208dd0cb596f2daULL,0xd049d7309aad93f9ULL,0xc79f263d42ab580eULL,0x09411bb123f707b4ULL,0x8cfde1ff835e0edaULL,0x7270749090f03402ULL,+	0xeaee6126c49a861eULL,0x024f3b65e14f0d06ULL,0x51a3f1e8c69bfc17ULL,0xc3c3a8e9a7686381ULL,0x3400752cb103d4c8ULL,0x02bc46139218b36bULL,0xc67f75eb7651504aULL,0xd6848b56d02aebfaULL,+	0xbd9802e6c30fa92bULL,0x5a70d96d9a552784ULL,0x9085c4ea3f83169bULL,0xfa9423bb06908228ULL,0x2ffebe12fe97a5b9ULL,0x85da604971b99118ULL,0x9cbc2f7f63178846ULL,0xfd96bc709153218eULL,+	0x958381db1782269bULL,0xae34bf792597e550ULL,0xbb5c60645f385153ULL,0x6f0e96afe3088048ULL,0xbf6a021577884456ULL,0xb3b5688c69310ea7ULL,0x17c9429504fad2deULL,0xe020f0e517896d4dULL,+	0x730ba0ab0976505fULL,0x567f6813095e2ec5ULL,0x470620106331ab71ULL,0x72cfa97741d22b9fULL,0x33e55ead8a2373daULL,0xa8d0d5f47ba45a68ULL,0xba1d8f9c03029d15ULL,0x8f34f1ccfc55b9f3ULL,+	0xcca4428dbbe5a1a9ULL,0x8187fd5f3126bd67ULL,0x0036973a48105826ULL,0xa39b6663b8bd61a0ULL,0x6d42deef2d65a808ULL,0x4969044f94636b19ULL,0xf611ee47dd5d564cULL,0x7b2f3a49d2873077ULL,+	0x94157d45300eb294ULL,0x2b2a656e169c1494ULL,0xc000dd76d3a47aa9ULL,0xa2864e4fa6243ea4ULL,0x82716c47db89842eULL,0x12dfd7d761479fb7ULL,0x3b9a2c56e0b2f6dcULL,0x46be862ad7f85d67ULL,+	0x03b0d8dd0f82b214ULL,0x460c34f9f103cbc6ULL,0xf32e5c0318d79e19ULL,0x8b8888baa84117f8ULL,0x8f3c37dcc0722677ULL,0x10d21be91c1c0f27ULL,0xd47c8468e0f7a0c6ULL,0x9bf02213adecc0e0ULL,+	0x0baa7d1242b48b99ULL,0x1bcb665d48424096ULL,0x8b847cd6ebfb5cfbULL,0x87c2ae569ad4d10dULL,0xf1cbb1220de36726ULL,0xe7043c683fdfbd21ULL,0x4bd0826a4e79d460ULL,0x11f5e5984bd1a2cbULL,+	0x97554160b7fe7b6eULL,0x7d16189a400a3fb2ULL,0xd73e9beae328ca1eULL,0x0dd04b97e793d8ccULL,0xa9c83c9b506db8ccULL,0x5cd47aaecf38814cULL,0x26fc430db64b45e6ULL,0x079b5499d818ea84ULL,+	0xebb01102c1c24a3bULL,0xca24e5681c161c1aULL,0x103eea6936f00a4aULL,0x9ad76ee876176c7bULL,0x97451fc2538e0ff7ULL,0x94f898096604b3b0ULL,0x6311436e3249cfd7ULL,0x27b4a7bd41224f69ULL,+	0x03b5d21ae0ac2941ULL,0x279b0254c2d31937ULL,0x3307c052cac992d0ULL,0x6aa7cb92efa8b1f3ULL,0x5a1825800d37c7a5ULL,0x13380c37342d5422ULL,0x92ac2d66d5d2ef92ULL,0x035a70c9030c63c6ULL,+	0xc16025dd4ce4f152ULL,0x1f419a71f9df7c06ULL,0x6d5b221491e4bb14ULL,0xfc43c6cc839fb4ceULL,0x49f06591925d6b2dULL,0x4b37d9d362186598ULL,0x8c54a971d01b1629ULL,0xe1a9c29f51d50e05ULL,+	0x5109b78571ba1861ULL,0x48b22d5cd0c8f93dULL,0xe8fa84a78633bb93ULL,0x53fba6ba5aebbd08ULL,0x7ff27df3e5eea7d8ULL,0x521c879668ca7158ULL,0xb9d5133bce6f1a05ULL,0x2d50cd53fd0ebee4ULL,+	0x889f6d65533ef217ULL,0x7158c7e4c3ca2e87ULL,0xfb670dfbdc2b4167ULL,0x75910a01844c257fULL,0xf336bf07cf88577dULL,0x22245250e45e2aceULL,0x2ed92e8d7ca23d85ULL,0x29f8be4c2b812f58ULL,+	0xfbb9b2452133ffd9ULL,0x39a8b2f1830f1a20ULL,0x484bc97dd5a1f52aULL,0xd6aebf56a40eddf8ULL,0x32257acb76ccdac6ULL,0xaf4d36ec1586ff27ULL,0x8eaa8863f8de7dd1ULL,0x0045d5cf88647c16ULL,+	0xc51e414351facc61ULL,0xbaf2647de68a25bcULL,0x8f5271a00ff872edULL,0x8f32ef993d2d9659ULL,0xca12488c7593cbd4ULL,0xed266c5d02b82fabULL,0x0a2f78ad14eb3f16ULL,0xc34049484d47afe3ULL,+	0xa6f3d574c005979dULL,0xc2072b426a40e350ULL,0xfca5c1568de2ecf9ULL,0xa8c8bf5ba515344eULL,0x97aee555114df14aULL,0xd4374a4dfdc5ec6bULL,0x754cc28f2ca85418ULL,0x71cb9e27d3c41f78ULL,+	0x09c1670209470496ULL,0xa489a5edebd23815ULL,0xc4dde4648edd4398ULL,0x3ca7b94a80111696ULL,0x3c385d682ad636a4ULL,0x6702702508dc5f1eULL,0x0c1965deafa21943ULL,0x18666e16610be69eULL,+	0x6792fd350369c8e1ULL,0x9271aa62b9dc843bULL,0x8711a4b14d02e2abULL,0x02b2a3e27ee1a383ULL,0xb226e35f0e2b379bULL,0x3d3de39cd652ab25ULL,0xaca6d4c93b560106ULL,0xeced0cf4c95bd877ULL,+	0x45beb4ca2a604b3bULL,0x56f651843a616762ULL,0xf52f5a70978b806eULL,0x7aa3978711dc4480ULL,0xe13fac2a0e01fabcULL,0x7c6ee8a5237d99f9ULL,0x251384ee05211ffeULL,0x4ff6976d1bc9d3ebULL,+	0x8910507903605c39ULL,0xf0843d9ea142c96cULL,0xf374493416923684ULL,0x732caa2ffa0a2893ULL,0xb2e8c27061160170ULL,0xc32788cc437fbaa3ULL,0x39cd818ea6eda3acULL,0xe2e942399e2b2e07ULL,+	0xdde0492316e043a2ULL,0x98a452611dd3d209ULL,0xeaf9f61bd431ebe8ULL,0x00919f4dbaf56abdULL,0xe42417db6d8774b1ULL,0x5fc5279c58e0e309ULL,0x64aa40613adf81eaULL,0xef419edabc627c7fULL,+	0x3919759239ef620fULL,0x9d47284074fa29c4ULL,0x4e428fa39d416d83ULL,0xd1a7c25129f30269ULL,0x46076e1cd746218fULL,0xf3ad6ee8110d967eULL,0xfbb5f434a00ae61fULL,0x3cd2c01980d4c929ULL,+	0xfa24d0537a4af00fULL,0x3f938926ca294614ULL,0x0d700c183982182eULL,0x801334434cc59947ULL,0xf0397106ec87c925ULL,0x62bd59fc0ed6665cULL,0xe8414348c7cca8b5ULL,0x574c76209f9f0a30ULL,+	0x6967d39b0260e52aULL,0xd42585cc90653325ULL,0x0d9bd60521ca7954ULL,0x4fa2087781ed57b3ULL,0x60c1eff8e34a0bbeULL,0x56b0040c84f6ef64ULL,0x28be2b24b1af8483ULL,0xb2278163f5531614ULL,+	0x95be42e2bb8b6a07ULL,0x64be74eeca23f86aULL,0xa73d74fd154ce470ULL,0x1c2d2857d8dc076aULL,0xb1fa1c575a887868ULL,0x38df8e0b3de64818ULL,0xd88e52f9c34e8967ULL,0x274b4f018b4cc76cULL,+	0x59c81ec2c11c208bULL,0x240207da4e3a7234ULL,0x4957eb80f4a089d8ULL,0xc86960bc2137a072ULL,0xde1154fd7f1a932cULL,0xb517fe74d3f2fa17ULL,0x2d5d94557111c0ebULL,0x2aa3378980929d36ULL,+	0x3f5c05b4f8b7559dULL,0x0be4c7acfae29200ULL,0xdd6d3ef756532accULL,0xf6c3ed87eea7a285ULL,0xe463b0a8f46ec59bULL,0x531d9b14ecea6c83ULL,0x3d6bdbafc2dc836bULL,0x3ee501e92ab27f0bULL,+	0x8df275455922ac1cULL,0xa7b3ef5ca52b3f63ULL,0x8e77b21471de57c4ULL,0x31682c10834c008bULL,0xc76824f04bd55d31ULL,0xb6d1c08617b61c71ULL,0x31db0903c2a5089dULL,0x9c092172184e5d3fULL,+	0x5ace5035ea6c3997ULL,0x54259aaac2610befULL,0xef18bb3f3c80dd39ULL,0x6910b95b5fc3fa39ULL,0xfce2f51043e09aeeULL,0xced56c9fa7675665ULL,0x10e265acd872db61ULL,0x6982812eae9fce69ULL,+	0xb8fa3d931341ed7aULL,0x4223272ca7b59d49ULL,0x3dcb194783b8c4a4ULL,0x4e413c01ed1302e4ULL,0x6d999127e17e44ceULL,0xee86bf7533b3adfbULL,0xf6902fe625aa96caULL,0xb73540e4e5aae47dULL,+	0xcc50ef6c872b4a60ULL,0xab2a34a44613521bULL,0x39c5c190983e15d1ULL,0x61dde5df59905512ULL,0xe417f6219f2275f3ULL,0x0750c8b6451d894bULL,0x75b04ab978b0bdaaULL,0x3bfd9fd4458589bdULL,+	0xaafcbfabaf95894cULL,0x7b9bdc07276b2241ULL,0xeaf983625bdda48bULL,0x5977faf2a3fcb4dfULL,0xbed042ef052c4b5bULL,0x9fe87f71067591f0ULL,0xc89c73ca22f24ec7ULL,0x7d37fa9ee64a9f1bULL,+	0xbd78148045ee2f40ULL,0x75e354af416b60cfULL,0xde0b58a18d49a8c4ULL,0xe40e94e2fa359536ULL,0xbd4fa59f62accd76ULL,0x05cf466a8c762837ULL,0xb5abda99448c277bULL,0x5a9e01bf48b13740ULL,+	0xb51e55e6f2606a82ULL,0xe25f706190f2fb57ULL,0xacef6c2ab1a4e37cULL,0x864e359d5dcf2706ULL,0x479e6b187ce57316ULL,0x2cab25003a96b23dULL,0xed4898628ef16df7ULL,0x2056538cef3758b5ULL,+	0xef69a0c3d41d3bd3ULL,0xb533b8c907a26bdeULL,0xe2801d97db2edf9fULL,0xdc4a8269e1877af0ULL,0x6c1c58513d590dbeULL,0x84632f6bee4e9357ULL,0xd36d36b779b33374ULL,0xb46833e39bbca2e6ULL,+	0xcc7a64880a750c0fULL,0x39bacfe34e548e83ULL,0x3d418c760c110f05ULL,0x3e4daa4cb1f11588ULL,0x2733e7b55ffc69ffULL,0x46f147bc92053127ULL,0x885b2434d722df94ULL,0x6a444f65e6fc6b7cULL,+	0x008be4bb346ac9cbULL,0x2e6cb02429811bb2ULL,0xa6ccc747f41540c1ULL,0xf119334efbf6de47ULL,0x4e6bffe9cc97fe6eULL,0x7f4b578bf3d82883ULL,0x459db722753dd1c7ULL,0x05843cd82066b495ULL,+	0x7cb88cd5dbf2af33ULL,0xded9566eaf6b0eabULL,0xd1caf5488e40d844ULL,0x3ae59bd67ce1f67fULL,0xacf4ad33dd82429eULL,0xcc7ebd2bbc4467c7ULL,0xf4f6e95de783d011ULL,0x5197e39116e92db7ULL,+	0xd4f2031885e7a4afULL,0x291bf9a0f7a6878dULL,0xadd8b6eebd051913ULL,0x174f5124f36be034ULL,0x2ac3364527f60189ULL,0xd8902eed8b37ec73ULL,0x546166a11cfdb42eULL,0x31c4e3b807076421ULL,+	0x7de29dfdbc6ebdfbULL,0xa1ee450589de549bULL,0xfd7181aa3ef26a64ULL,0x003179eedf2f3980ULL,0x1bce4d30fd71bc78ULL,0xeb842b14be86a583ULL,0xe00125dc5eb85711ULL,0x9f586983ff11f405ULL,+	0x5b5a089eb833eccbULL,0xc1b3077a7bc51c79ULL,0xe581157f9fbe0e93ULL,0xab487d695b29b172ULL,0xc72551082222f24bULL,0x338cd22cf6dd35dfULL,0x92010e6f5fc24afeULL,0xf8298f15681d46edULL,+	0x2ce7f723042389edULL,0xdf7de0d5a54b1972ULL,0xb8ea2e142c251d75ULL,0xd22a4d37f4e8e8c8ULL,0xe99958566c29c8bcULL,0x9fefdbfe1d1e201eULL,0xb97d946edac885c9ULL,0xf6ea9767d38eac70ULL,+	0x9aaca8e974e75a2eULL,0xcda6fc1eadac968fULL,0x3f6651bf46306befULL,0xc7445e4ecb2ed7f9ULL,0x584a12d8d1571ac1ULL,0x684846c4daf3a679ULL,0xcfc622a9863fbc43ULL,0x2f9e101dea6814f2ULL,+	0x7a1a465ac3f16ea8ULL,0x115a461db2f1d11cULL,0x4767dd956c68a172ULL,0x3392f2ebd13a4698ULL,0xc7a99ccde526cdc7ULL,0x8e537fdc22292b81ULL,0x76d8cf69a6d39198ULL,0xffc5ff432446852dULL,+	0x97b14f7ea90567e6ULL,0x513257b7b6ae5cb7ULL,0x85454a3c9f10903dULL,0xd8d2c9ad69bc3724ULL,0x38da93246b29cb44ULL,0xb540a21d77c8cbacULL,0x9bbfe43501918e42ULL,0xfffa707a56c3614eULL,+	0x56c2e05b1cb76219ULL,0x0ec0bf9171567e7eULL,0xe7076f8661c4c910ULL,0xd67b085bbabc04d9ULL,0x9fb904595e93a96aULL,0x7526c1eafbdc249aULL,0x0d44d367ecdd0bb7ULL,0x953999179dc0d695ULL,+	0x2ccbc583a4c506ecULL,0x957ed188d1acfe97ULL,0x8baed83312f1aea2ULL,0xef2a6cb48325362dULL,0x130dde428e195c43ULL,0xc842025a0e6050c6ULL,0x2da972a708686a5dULL,0xb52999a1e508b4a8ULL,+	0x83f49167ceca9754ULL,0x426d2cf64b7939a0ULL,0x2555e355723fd0bfULL,0xa96e6d06c4f144e2ULL,0x4768a8dd87880e61ULL,0x15543815e508e4d5ULL,0x09d7e772b1b65e15ULL,0x63439dd6ac302fa0ULL,+	0xb11df8e1698e04ccULL,0x877be203169005c8ULL,0x32749e8c4f3c6179ULL,0x2dbc9d0a7853fc05ULL,0x187d4f939454d937ULL,0xe682ce9db4800e1bULL,0xa9129ad8165e68e8ULL,0x0fe29735be7f785bULL,+	0x31019ccf3a4434b4ULL,0xa34581111a7954dcULL,0xa9dac80de34972a7ULL,0xb043d05474f6b8ddULL,0x021c319e11137b1aULL,0x00a754ceed5cc03fULL,0x0aa2c794cbea5ad4ULL,0x093e67f470c015b6ULL,+	0x20ac0351d598d710ULL,0x272c4166cb3a4da4ULL,0xdb82fe1aca71de1fULL,0x746e79f2d8f54b0fULL,0x6e7fc7364b573e9bULL,0x75d03f46fd4b5040ULL,0x5c1cc36d0b98d87bULL,0x513ba3f11f472da1ULL,+	0x859d3145983c38b5ULL,0xb14f176c637abc8bULL,0x2793fb9dcaff7be6ULL,0xebe5a55f35a66a5aULL,0x7cec1dcd9f87dc59ULL,0x7c595cd3fbdbf560ULL,0x5b543b2226eb3257ULL,0x69080646c4c935fdULL,+	0xdca3b70678a6513bULL,0x92ea4a2a9edb1943ULL,0x02642216db6e2dd8ULL,0x9b45d0b49fd57894ULL,0x114e70dbc69d11aeULL,0x1477dd194c57595fULL,0xbc2208b4ec77c272ULL,0x95c5b4d7db68f59cULL,+	0xfe541fa47ea67c77ULL,0x952bd2afe3ea810cULL,0x791fef568d01d374ULL,0xa3a1c6210f11336eULL,0x5ad0d5a9c7ec6d79ULL,0xff7038af3225c342ULL,0x003c6689bc69601bULL,0x25059bc745e8747dULL,+	0x9a75c80676cb2566ULL,0x8f76acb1b24892d9ULL,0x7ae7b9cc1f08fe45ULL,0x19ef73296a4907d8ULL,0x2db4ab715f228bf0ULL,0xf3cdea39817032d7ULL,0x0b1f482edcabe3c0ULL,0x3baf76b4bb86325cULL,+	0x6aac688eadd70482ULL,0x708de92a7b4a4e8aULL,0x75b6dd73758a6eefULL,0xea4bf352725b3c43ULL,0x10041f2c87912868ULL,0xb1b1be95ef09297aULL,0x19ae23c5a9f3860aULL,0xc4f0f839515dcf4bULL,+	0xc71e27bf8538a5c6ULL,0x195c63dd89abff17ULL,0xfd3152851b71e3daULL,0x9cbdfda7fa680fa0ULL,0x9db876ca849d7eabULL,0xebe2764b3c273271ULL,0x663357e3f208dceaULL,0x8c5bd833565b1b70ULL,+	0x75900d7c2fa4f126ULL,0x08a3b8655c99a232ULL,0x2478b6bfdb25e0c3ULL,0x482cc2c271db2edfULL,0x37df7e645f321bb8ULL,0x8a93821b9a8005b4ULL,0x3fa2f10ccc8c1958ULL,0x0d3322182c269d0aULL,+	0xba5514df3fd165e8ULL,0x499fd6a9061f8811ULL,0x72cd1fe0bfef9f00ULL,0x120a4bb979ad7e8aULL,0xf2ffd0955f4a5ac5ULL,0xcfd174f195a7a2f0ULL,0xd42301ba9d17baf1ULL,0xd2fa487a77f22089ULL,+	0xb93452381d531696ULL,0x57201c0088cdde69ULL,0xdde922519a86afc7ULL,0xe3043895bd35cea8ULL,0x7608c1e18555970dULL,0x8267dfa92535935eULL,0xd4c60a57322ea38bULL,0xe0bf7977804ef8b5ULL,+	0x6233ea68c094dbb5ULL,0xb77d062ed968d410ULL,0x3e719bbc58b3002dULL,0x68e7dd3d3dc49d58ULL,0x8d825740013a5e58ULL,0x213117473c9e3c1bULL,0x0cb0a2a77c99b6abULL,0x5c48a3b3c2f888f2ULL,+	0xc7913e91991724f3ULL,0x5eda799c39cbd686ULL,0xddb595c763d4fc1eULL,0x6b63b80bac4fed54ULL,0x6ea0fc697e5fb516ULL,0x737708bad0f1c964ULL,0x9628745f11a92ca5ULL,0x61f379589a86967aULL,+	0x5320fd610979e6b3ULL,0x1d0bd3e593d40723ULL,0x0ac006fcaa80baccULL,0xb1d9c60ed2002515ULL,0x610e7ed0af780b92ULL,0xf5bf446e80a9ce31ULL,0x441c011d3c20b54bULL,0x77f76da1191596c3ULL,+	0x9af39b2caa665072ULL,0x78322fa4efd324efULL,0x3d153394c327bd31ULL,0x81d5f2713129dab0ULL,0xc72e0c42f48027f5ULL,0xaa40cdbc8536e717ULL,0xf45a657a2d369d0fULL,0xb03bbfc4ea7f74e6ULL,+	0x7a04d5429b6a42eaULL,0xfa597e853daf41b9ULL,0x4c58ec27726b0b89ULL,0xed8eb16a030d43deULL,0x65e1e5e1ec9dcf57ULL,0xb7a770c1697cff81ULL,0x1e6d918f9f6e2b22ULL,0x7c277a9ac64e82b7ULL,+	0x46a8c4180d738dedULL,0x6f1a5bb0e0de5729ULL,0xf10230b98ba81675ULL,0x32c6f30c112b33d4ULL,0x7559129dd8fffb62ULL,0x6a281b47b459bf05ULL,0x77c1bd3afa3b6776ULL,0x0709b3807829973aULL,+	0x0875e0c1da36cb47ULL,0xfdf5b7cb1e0210f0ULL,0x7e0c7e4d3a3787c8ULL,0xf043f5262b1c741fULL,0x76df1b006d74d72dULL,0x0514df7338b45ba9ULL,0xaecf7c3e0a6b797aULL,0x5e30b285ddeaac39ULL,+	0x8c26b232a3326505ULL,0x38d69272ee1d41bfULL,0x0459453effe32afaULL,0xce8143ad7cb3ea87ULL,0x932ec1fa7e6ab666ULL,0x6cd2d23022286264ULL,0x459a46fe6736f8edULL,0x50bf0d009eca85bbULL,+	0x15c78f7d605238a5ULL,0xe9d87842448496abULL,0xcfcf75d0d210acc1ULL,0x2948f2295c8c14e2ULL,0xe81fd76de8daf0cbULL,0xd02d11e4a03be800ULL,0x0c4df3518778d30cULL,0x3482bc96965139cbULL,+	0x0b825852877a21ecULL,0x300414a70f537a94ULL,0x3f1cba4021a9a6a2ULL,0x50824eee76943c00ULL,0xa0dbfcecf83cba5dULL,0xf953814893b4f3c0ULL,0x6174416248f24dd7ULL,0x5322d64de4fb09ddULL,+	0x56b6cf278a448bbcULL,0x0ca898dfc85251daULL,0x9082cad836e79b24ULL,0x2e7b9ed31a8e51a7ULL,0xdc7d318a43e1c802ULL,0x4750e523cbf8689dULL,0x9887a072f0071b1aULL,0x52090f87814bfdc1ULL,+	0x574473843d9325f3ULL,0xa9bef2d0f371cb84ULL,0x77d2188ba61e36c5ULL,0xbbd6a7d7c602df72ULL,0xba3aa9028f61bc0bULL,0xf49085ed6ed0b6a1ULL,0x8bc625d6ae6e8298ULL,0x832b0b1da2e9c01dULL,+	0x5bebf2196196bc6eULL,0x0e66736bef097efdULL,0x1128f3b8ea87293aULL,0x2998e4056addfcdeULL,0x55cc31b85d16c961ULL,0x87a434e12418f056ULL,0x2e94aaccdc9fe819ULL,0x94a486c1a56490c6ULL,+	0xa337c447f1f0ced1ULL,0x800cc7939492dd2bULL,0x4b93151dbea08efaULL,0x820cf3f8de0a741eULL,0xff1982dc1c0f7d13ULL,0xef92196084dde6caULL,0x1ad7d97245f96ee3ULL,0x319c8dbe29dea0c7ULL,+	0x1ff2385bde259ec8ULL,0xf6b0836a30f67b0dULL,0x04cc65b006661cffULL,0x467e6358d4230f5cULL,0xce468c802dbed3d3ULL,0x7b984262f1920da6ULL,0x34d257421537479fULL,0x5c8aa88c87bb8de1ULL,+	0xd3ea38717b82b99bULL,0x75922d4d470eb624ULL,0x8f66ec543b95d466ULL,0x66e673ccbee1e346ULL,0x6afe67c4b5f2b89aULL,0x3de9c1e6290e5cd3ULL,0x8c278bb6310a2adaULL,0x420fa3840bdb323bULL,+	0x646f96796424c49bULL,0xf888dfe867c241c9ULL,0xe12d4b9324f68b49ULL,0x9a6b62d8a571df20ULL,0x81b4b26d179483cbULL,0x666f96329511fae2ULL,0xd281b3e4d53aa51fULL,0x7f96a7657f3dbd16ULL,+	0x8553d37c051af62bULL,0xe9a998eb0bf94496ULL,0xe0844f9fb0d59aa1ULL,0x983fd558e6afb813ULL,0x9670c0ca65d69804ULL,0x732b22de6ea5ff2dULL,0xd7640ba95fd8623bULL,0x9f619163a6351782ULL,+	0xf167b4e0bdefdd4fULL,0x69958465f366e401ULL,0x5aa368aba73bbec0ULL,0x121487097b240c21ULL,0x378c323318969006ULL,0xcb4d73cee1fe53d1ULL,0x5f50a80e130c4361ULL,0xd67f59517ef5212bULL,+	0x332f81088cad38c0ULL,0x471b7e906bd68ae2ULL,0x56ac3fb20d8e27a3ULL,0xb54660db136b4b0dULL,0x123a1e11a6fd8de4ULL,0x44dbffeaa37799efULL,0x4540b977ce6ac17cULL,0x495173a8af60acefULL,+	0xeb4437434573eab0ULL,0x11570dfbd1ac6031ULL,0xf7d9b45b44dd9afdULL,0xb8066add22067231ULL,0x15f92ad8f8a3f0b4ULL,0x9e0e4899e0ace2a2ULL,0xbdcd0aadfab38b80ULL,0x46506ae917020052ULL,+	0x429a69f78fca399dULL,0xfe9e27d20207bb63ULL,0xec655ed68788f582ULL,0xa426d748adb75f6eULL,0x18695c02ca81c66dULL,0x84fb8d27a531d425ULL,0x3a3a8956deff48baULL,0xaf1d0d56766d2247ULL,+	0x5a059565352c4b5cULL,0x49261531590bc3e2ULL,0x809f7521f66f9f5fULL,0x2baef6bfc70a4a9bULL,0xe7e6fa6509ed3561ULL,0x11370233984b230cULL,0x2151659bd04cdc69ULL,0xbdb83c63f007d416ULL,+	0x9ebb284d391c2a82ULL,0xbcdd4863158308e8ULL,0x006f16ec83f1edcaULL,0xa13e2c37695dc6c8ULL,0x2ab756f04a057a87ULL,0xa8765500a6b48f98ULL,0x4252face68651c44ULL,0xa52b540be1765e02ULL,+	0xcb35a1a85ca37ff0ULL,0xe1a04f1ccd2f1c8fULL,0x238816ce15a26112ULL,0xe206a111095b177eULL,0x3c10b6048a424149ULL,0xc6a3f56774752cfbULL,0xbf16a37a47f1dbb8ULL,0x7c372f9ad31a3dfbULL,+	0x122d05b5c20b4d2aULL,0xff659cf50c662a67ULL,0xed57c128e8ffc9e9ULL,0x0fbb15859e987683ULL,0xadd70df247319a2bULL,0x4b98baba374be470ULL,0xf03d747356a7b307ULL,0x342e696e3efebf30ULL,+	0xf84b48f7864ac537ULL,0x04713409a6940d3dULL,0x014db22d6174c7aeULL,0xc73a1c438c213034ULL,0x18ac4ea5ffdd93ecULL,0x724fc7576102783eULL,0x9fe13fcc91c3e83fULL,0x92a8c2c8f08f0bf5ULL,+	0x7f2e22fd8f67f6deULL,0xab018833d8693177ULL,0x266d1db6863eca95ULL,0x6bb7732b31b5ef2bULL,0x4fa927c6915f80ceULL,0x6fa1d6d25f90efd8ULL,0x7bd75de8456b48adULL,0xd2cb507a845b6429ULL,+	0xa72cf82ae255d7ecULL,0x52025c23a460e204ULL,0x10ae542d7d5b0a44ULL,0xa85143109305aedaULL,0x958315f5a14bbfe8ULL,0x3f361826385365feULL,0xc2b3a36b66d95040ULL,0x12c7b3347cf4eda2ULL,+	0xa545b4d1e744119dULL,0x4c93b169829a71e7ULL,0x2dbb908c6117fcbfULL,0x4dc97320b35a3d85ULL,0x94c04f856bf88105ULL,0x452e1bce1b51bc1fULL,0x0c41ff50b3013af3ULL,0xf07af445224d7e24ULL,+	0xbdb9e57ca3d24f6aULL,0x8a8246d7f345a763ULL,0x73bd2a6d98cfbb5fULL,0x1dd8e85e86ed04dbULL,0x76f2da42c01f420bULL,0x7ef0547364407bc7ULL,0x7e98ba7faff548f5ULL,0x6b7afbeefd30b64aULL,+	0x4f922fc516a0d2bbULL,0x0d5cc16c1a623499ULL,0x9241cf3a57c62c8bULL,0x2f5e6961fd1b667fULL,0x5c15c70bf5a01797ULL,0x3d20b44d60956192ULL,0x04911b37071fdb52ULL,0xf648f9168d6f0f7bULL,+	0x027cc8b8fac61d9aULL,0x7d25e062e3c6fe8aULL,0xe08805bfe5bff503ULL,0x13271e6c6ff632f7ULL,0x55dca6c0232f76a5ULL,0x8957c32d701ef426ULL,0xee728bcba10a5178ULL,0x5ea60411b62c5173ULL,+	0x9ad5462bb4d8bc50ULL,0x181c0b16a9195770ULL,0xebd4fe1c78412a68ULL,0xae0341bcc0dff48cULL,0xb6bc45cf7003e866ULL,0xf11a6dea8a24a41bULL,0x5407151ad04c24c2ULL,0x62c9d27dda5b7b68ULL,+	0x32865719a8afd30bULL,0x867983288a826dceULL,0xdf04e891c4a8fbe0ULL,0xbb6b6e1bebf56ad3ULL,0x0a695b11471f1ff0ULL,0xd76c3389be15baf0ULL,0x018edb95be96c43eULL,0xf2beaaf490794158ULL,+	0x0a50b12e523b8bf6ULL,0x8009eb5b8f910c1bULL,0xf535af824a167588ULL,0x0f835f9cfb2a2abdULL,0xf59b29312afceb62ULL,0xc797df2a169d383fULL,0xeb3f5fb066ac02b0ULL,0x029d4c6fdaa2d0caULL,+	0x87a7ebd1e0a1b12aULL,0x1e4ef88d770ba95fULL,0x8c33345cdc2ae9cbULL,0xcecf127601cc8403ULL,0x687c012e1b39b80fULL,0xfd90d0ad35c33ba4ULL,0xa3ef5a675c9661c2ULL,0x368fc88ee017429eULL,+	0xb82226052b7ce542ULL,0xe6d4ce997472bde1ULL,0x53e16ebe09d2f4daULL,0x180ff42e53b92b2eULL,0xc59bcc022c34a1c6ULL,0x3803d6f9422c46c2ULL,0x18aff74f5c14a8a2ULL,0x55aebf8010a08b28ULL,+	0xb956970e2fdd23ccULL,0xb80288bc5682e971ULL,0xe6e6d91e9ae86ebcULL,0x0564c83f8c9f1939ULL,0x551932a239560368ULL,0xe893752b049c28e2ULL,0x0b03cee5a6a158c3ULL,0xe12d656b04964263ULL,+	0x58af2010f5b343bcULL,0x0f2e400af2f142feULL,0x3483bfdea85f4bdfULL,0xf0b1d09303bfeaa9ULL,0x2ea01b95c7081603ULL,0xe943e4c93dba1097ULL,0x47be92adb438f3a6ULL,0x00bb7742e5bf6636ULL,+	0x4ed714576be5f7deULL,0xd93006f8c2263c9eULL,0xe073694ccacacb36ULL,0x2ff7a5b43ae118abULL,0x3cce53f1cd871236ULL,0xf156a39dc2aa6d52ULL,0x9cc5f271b198d76dULL,0xbc615b6f81383d39ULL,+	0x137a4fb486df2a61ULL,0xa1ed9c07ecf7b4a2ULL,0xb2e460e27bd042ffULL,0xb7f5e2fa5f62f5ecULL,0x7aa6ec6bcc2423b7ULL,0x75ce0a7fba63eea7ULL,0x67a45fb1f250a6e1ULL,0x93bc919ce53cdc9fULL,+	0x67930af231f63950ULL,0xa77797c114caa2c9ULL,0x526e80ee27ac7e62ULL,0xe1e6e62658b28aecULL,0x636178b0b3c9fef0ULL,0xaf7752e06d5f90beULL,0x94ecaf18eece51cfULL,0x2864d0edca806e1fULL,+	0xec2fccaaddce3345ULL,0x2a6811b7012a4350ULL,0x96760ff1ac598bdcULL,0x054d652ad1bf4128ULL,0x0a1151d492a21005ULL,0xad7f397133110fdfULL,0x8c95928c1960100fULL,0x6c91c8257bf03362ULL,+	0x17785b7799eb6df0ULL,0x26c3cc517386b779ULL,0x345ed9886417a48eULL,0xe990b4e407d6ef31ULL,0x0f456b7e2586abbaULL,0x239ca6a559c96e9aULL,0xe327459ce2eb4206ULL,0x3a4c3313a002b90aULL,+	0x19e6125dec3f1decULL,0x07b1f040911178daULL,0xd93ededa904a6738ULL,0x55187a5a0bebedcdULL,0xf7d04722eb329d41ULL,0xf449099ef170b391ULL,0xfd317a69ca99f828ULL,0x50c3db2b34a4976dULL,+	0x3806b69b92222f1fULL,0x5a2459ca6cf7ae70ULL,0x6789f69ca85217eeULL,0x5f232b5ee3dc85acULL,0x660e3ec548e9e516ULL,0x124b4e473197eb31ULL,0x10a0cb13aafcca23ULL,0x7bd63ba48213224fULL,+	0xb674481b7bfe7178ULL,0x4e1debae65405868ULL,0x061b2821c48c867dULL,0x69c15b35513b30eaULL,0x3b4a166636871088ULL,0xe5e29f5d1220b1ffULL,0x4b82bb35233d9f4dULL,0x4e07633318cdc675ULL,+	0x0d53f5c7a3e6fcedULL,0xe8cbbdd5f45fbdebULL,0xf85c01df13339a70ULL,0x0ff71880142ceb81ULL,0x4c4e8774bd70437aULL,0x5fb32891ba0bda6aULL,0x1cdbebd2f18bd26eULL,0x2f9526f103a9d522ULL,+	0x40ce305192c4d684ULL,0x8b04d7257612efcdULL,0xb9dcda366f9cae20ULL,0x0edc4d24f058856cULL,0x64f2e6bf85427900ULL,0x3de81295dc09dfeaULL,0xd41b4487379bf26cULL,0x50b62c6d6df135a9ULL,+	0xd4f8e3b4c72dfe67ULL,0xc416b0f690e19fdfULL,0x18b9098d4c13bd35ULL,0xac11118a15b8cb9eULL,0xf598a318f0062841ULL,0xbfe0602f89f356f4ULL,0x7ae3637e30177a0cULL,0x3409774761136537ULL,+	0x0db2fb5ed005832aULL,0x5f5efd3b91042e4fULL,0x8c4ffdc6ed70f8caULL,0xe4645d0bb52da9ccULL,0x9596f58bc9001d1fULL,0x52c8f0bc4e117205ULL,0xfd4aa0d2e398a084ULL,0x815bfe3a104f49deULL,+	0x97e5443f23885e5fULL,0xf72f8f99e8433aabULL,0xbd00b154e4d4e604ULL,0xd0b35e6ae5e173ffULL,0x57b2a0489164722dULL,0x3e3c665b88761ec8ULL,0x6bdd13973da83832ULL,0x3c8b1a1e73dafe3bULL,+	0x4497ace654317cacULL,0xbe600ab9521771b3ULL,0xb42e409eb0dfe8b8ULL,0x386a67d73942310fULL,0x25548d8d4431cc28ULL,0xa7cff142985dc524ULL,0x4d60f5a193c4be32ULL,0x83ebd5c8d071c6e1ULL,+	0xba3a80a7b1fd2b0bULL,0x9b3ad3965bec33e8ULL,0xb3868d6179743fb3ULL,0xcfd169fcfdb462faULL,0xd3b499d79ce0a6afULL,0x55dc1cf1e42d3ff8ULL,0x04fb9e6cc6c3e1b2ULL,0x47e6961d6f69a474ULL,+	0x54eb3acce548b37bULL,0xb38e754284d40549ULL,0x8c3daa517b341b4fULL,0x2f6928ec690bf7faULL,0x0496b32386ce6c41ULL,0x01be1c5510adadcdULL,0xc04e67e74bb5faf9ULL,0x3cbaf678e15c9985ULL,+	0x8cd1214550ca4247ULL,0xba1aa47ae7dd30aaULL,0x2f81ddf1e58fee24ULL,0x03452936eec9b0e8ULL,0x8bdc3b81243aea96ULL,0x9a2919af15c3d0e5ULL,0x9ea640ec10948361ULL,0x5ac86d5b6e0bcccfULL,+	0xf892d918c36cf440ULL,0xaed3e837c939719cULL,0xb07b08d2c0218b64ULL,0x6f1bcbbace9790ddULL,0x4a84d6ed60919b8eULL,0xd89007918ac1f9ebULL,0xf84941aa0dd5daefULL,0xb22fe40a67fd62c5ULL,+	0x97e15ba2157f2db3ULL,0xbda2fc8f8e28ca9cULL,0x5d050da437b9f454ULL,0x3d57eb572379d72eULL,0xe9b5eba2fb5ee997ULL,0x01648ca2e11538caULL,0x32bb76f6f6327974ULL,0x338f14b8ff3f4bb7ULL,+	0x524d226ad7ab9a2dULL,0x9c00090d7dfae958ULL,0x0ba5f5398751d8c2ULL,0x8afcbcdd3ab8262dULL,0x57392729e99d043bULL,0xef51263baebc943aULL,0x9feace9320862935ULL,0x639efc03b06c817bULL,+	0x1fe054b366b4be7aULL,0x3f25a9de84a37a1eULL,0xf39ef1ad78d75cd9ULL,0xd7b58f495062c1b5ULL,0x6f74f9a9ff563436ULL,0xf718ff29e8af51e7ULL,0x5234d31315e97fecULL,0xb6a8e2b1292f1c0aULL,+	0xa7f53aa8327720c1ULL,0x956ca322ba092cc8ULL,0x8f03d64a28746c4dULL,0x51fe178266d0d392ULL,0xd19b34db3c832c80ULL,0x60dccc5c6da2e3b4ULL,0x245dd62e0a104cccULL,0xa7ab1de1620b21fdULL,+	0xb293ae0b3893d123ULL,0xf7b75783b15ee71cULL,0x5aa3c61442a9468bULL,0xd686123cdb15d744ULL,0x8c616891a7ab4116ULL,0x6fcd72c8a4e6a459ULL,0xac21911077e5fad7ULL,0xfb6a20e7704fa46bULL,+	0xe839be7d341d81dcULL,0xcddb688932148379ULL,0xda6211a1f7026eadULL,0xf3b2575ff4d1cc5eULL,0x40cfc8f6a7a73ae6ULL,0x83879a5e61d5b483ULL,0xc5acb1ed41a50ebcULL,0x59a60cc83c07d8faULL,+	0x439530b665c7322dULL,0xcf12cc01b3c1b3fbULL,0xc70b01860172f685ULL,0xb915ee221b58391dULL,0x9afdf03ba317db24ULL,0x87dec65917b8ffc4ULL,0x7f46597be4d3d050ULL,0x80a1c1ed006500e7ULL,+	0x3e22a7b397acf4ecULL,0x0426c4005ea8b640ULL,0x5e3295a64e969285ULL,0x22aabc59a6a45670ULL,0xb929714c5f5942bcULL,0x9a6168bdfa3182edULL,0x2216a665104152baULL,0x46908d03b6926368ULL,+	0x52cb8ac6c2babcc1ULL,0x4748d448fe81ae8dULL,0x5844f03f80f1a711ULL,0x3db784b2f8df4ac4ULL,0xad918f122df1fe36ULL,0xe40f25b9f33cc7c0ULL,0x4700d0e73b5e5555ULL,0x5c28fa08b03326f9ULL,+	0xa9f5d8745a1251fbULL,0x967747a8c72725c7ULL,0x195c33e531ffe89eULL,0x609d210fe964935eULL,0xcafd6ca82fe12227ULL,0xaf9b5b960426469dULL,0x2e9ee04c5693183cULL,0x1084a333c8146fefULL,+	0x56dab1c8321a518cULL,0xfd4439a68bce226fULL,0xe0b30d194facb9faULL,0xb5052f307583571bULL,0x1442641012afd476ULL,0xd02e417203fe624aULL,0xfc394f65531c92e6ULL,0x16d4bf5ad4bc0b52ULL,+	0xce06b88210395755ULL,0x117ce6345ec1df80ULL,0xfefae513eff55e96ULL,0xcf36cba6fd7fed1eULL,0x7340eca9a40ebf88ULL,0xe6ec1bcfb3d37e12ULL,0xca51b64e86bbf9ffULL,0x4e0dbb588b40e05eULL,+	0x120ad0bf5f8b8a84ULL,0xbfa00f36866f3edeULL,0xa8c6064ec30ebc2cULL,0xc39e40b823001e3bULL,0x9614f7cde7b1cbabULL,0xcd4420c704a56284ULL,0x8446a8f316857a19ULL,0x519b93b1e6f706cbULL,+	0x96649933aed1d1f7ULL,0x566eaff350563090ULL,0x345057f0ad2e39cfULL,0x148ff65b1f832124ULL,0x042e89d4cf94cf0dULL,0x319bec84520c58b3ULL,0x2a2676265361aa0dULL,0xc86fa3028fbc87adULL,+	0x3805e5b80863b664ULL,0x8be7ac6b3cabdb53ULL,0x9f7d70505dfeff91ULL,0x7dea8bd095896206ULL,0x28005a3b410e3c4eULL,0x24a4f0e9bc603ebfULL,0xcc4fd5ae4aec15d2ULL,0x4dc253f80f96641dULL,+	0x359d7b9c7ea2ee34ULL,0x3fd0d94c09cc3a71ULL,0xbb53c31c3a1ea37aULL,0x533425facf818c87ULL,0x7cd199c3810156e0ULL,0x0ea020e430c16448ULL,0xe557ba094a642542ULL,0xe657e7e79465f5eaULL,+	0xea723ad1fbc82439ULL,0xc726868bf896e9fdULL,0xd97c913c511c33ffULL,0x8a3fa8e2e2114231ULL,0x2a6c0e5608445b3eULL,0x2c4cd884f8d098d3ULL,0x7bf51faf2fea77d4ULL,0x709f208a7b1c4f71ULL,+	0xfc83d2ab5c8b06d5ULL,0xb1a785a2fe4eac46ULL,0xb99315bc846f7779ULL,0xcf31d816ef9ea505ULL,0x2391fe6a15d7dc85ULL,0x2f132b04b4016b33ULL,0x29547fe3181cb4c7ULL,0xdb66d8a6650155a1ULL,+	0x5b9e4843f45aac50ULL,0xc31e042e91eaaad0ULL,0x6e8c0b345a54eea0ULL,0xf0437b94962c7a57ULL,0xe4531ce8fe1d348bULL,0xe1489378e3786432ULL,0xd5e19c4a3f510d38ULL,0x4df3f016ce348b00ULL,+	0x59cd0e8b593d070fULL,0x437575165255625dULL,0x551fdda75b7a0399ULL,0x7bb6e6b02dec1eebULL,0x729bb662334c0922ULL,0x3df631df0cf41b79ULL,0x01abf3c578f32402ULL,0xfcb4666c9cd33c88ULL,+	0xb805b445735e843cULL,0x2a8e890d97379134ULL,0xebc7c10c52ab9f87ULL,0xcbb5e1ecb80a92b6ULL,0xd6ada2d9dc2c4efeULL,0xfccf504eae8cc7bdULL,0x650115acb2418a74ULL,0x8dd90e06c52bd80cULL,+	0x6b66d7e1adc1696fULL,0x98ebe5930acd72d0ULL,0x65f24550cc1b7435ULL,0xce231393b4b9a5ecULL,0x234a22d4db067df9ULL,0x98dda095caff9b00ULL,0x1bbc75a06100c9c1ULL,0x1560a9c8939cf695ULL,+	0xe4050f1cf1c367caULL,0x9bc85a9bc90fbc7dULL,0xa373c4a2e1a11032ULL,0xb64232b7ad0393a9ULL,0xf5577eb0167dad29ULL,0x1604f30194b78ab2ULL,0x0baa94afe829348bULL,0x77fbd8dd41654342ULL,+	0x31f14802fcf0a7fdULL,0x42fd07895488b01eULL,0x71d78d6d9952b498ULL,0x8eb572d907ac5201ULL,0xe0a2a44c4d194a88ULL,0xd2b63fd9ba017e66ULL,0x78efc6c8f888aefcULL,0xb76f6bda4a881a11ULL,+	0xa2f7932c68af43eeULL,0x5502468e703d00bdULL,0xe5dc978f2fb061f5ULL,0xc9a1904a28c815adULL,0xd3af538d470c56a4ULL,0x159abc5f193d8cedULL,0x2a37245f20108ef3ULL,0xfa17081e223f7178ULL,+	0x1fe2a9b2b4b4b67cULL,0xc1d10df0e8020604ULL,0x9d64abfcbc8058d8ULL,0x8943b9b2712a0fbbULL,0x90eed9143b3def04ULL,0x85ab3aa24ce775ffULL,0x605fd4ca7bbc9040ULL,0x8b34a564e2c75dfbULL,+	0x5c18acf88e2f7d90ULL,0xfdbf33d777be32cdULL,0x0a085cd7d2eb5ee9ULL,0x2d702cfbb3201115ULL,0xb6e0ebdb85c88ce8ULL,0x23a3ce3c1e01d617ULL,0x3041618e567333acULL,0x9dd0fd8f157edb6bULL,+	0xb2b2610798fa7aaaULL,0x41209ee4f073aa4eULL,0xf1570359f2d6b19bULL,0xcbe6868cfc577cafULL,0x186c4bdc32c04dd3ULL,0xa6c35faecfeee397ULL,0xb4a1b312f086c0cfULL,0xe0a5ccc6d9461fe2ULL,+	0x516ff3a36fa6110cULL,0x74fb1eb1fb93561fULL,0x6c0c90478457522bULL,0xcfd321046bb8bdc6ULL,0x2d6884a2cc80ad57ULL,0x7c27fc3586a9b637ULL,0x3461baedadf4e8cdULL,0x1d56251a617242f0ULL,+	0xb84011a9431dd80eULL,0xeb7c7cca73306cd9ULL,0x20fadd29d1b3b730ULL,0x83858b5bfe37b3d3ULL,0xbf4cd193b6251d5cULL,0x1cca1fd31352d952ULL,0xc66157a490fbc051ULL,0x7990a63889b98636ULL,+	0x892c81a321175ec1ULL,0x9159a505ee018109ULL,0xc70130532d8be316ULL,0x76060c21426fa2e5ULL,0x074d2dfc6b6f0f22ULL,0x9725fc64ca01a671ULL,0x3f6679b92770bd8eULL,0x8fe6604fd7c9b3feULL,+	0xce711154b6e00a84ULL,0xd9fe7e4224890e60ULL,0xd10bc6c34560988fULL,0xbdc2ef526859b004ULL,0xdcf0d868d5c890eeULL,0x893115e6119c47dcULL,0xe97966fbee714567ULL,0x117813355c85aa53ULL,+	0x71d530cc73204349ULL,0xc9df473d94a0679cULL,0xc572f0014261e031ULL,0x9786b71f22f135feULL,0xed6505fa6b64e56fULL,0xe2fb48e905219c46ULL,0x0dbec45bedf53d71ULL,0xd7d782f2c589f406ULL,+	0x350fb66b73ed0966ULL,0x968e4b082886032bULL,0x5a16ed6e88390493ULL,0x0a83ce84a121edbeULL,0x7c86fc10a3d9cda0ULL,0x5a40a6d595ce67fbULL,0x6cb8cda7f937dbf6ULL,0x95b44768651f6283ULL,+	0x06513c8a446cd7f4ULL,0x158c423b906d52a6ULL,0x71503261c423866cULL,0x4b96f57093c148eeULL,0x5daf9cc7239a8523ULL,0x611b597695ac4b8bULL,0xde3981db724bf7f6ULL,0x7e7d0f7867afc443ULL,+	0x984f101ed6fc3837ULL,0x340bf99f5e1b3a09ULL,0xbb96036f06942626ULL,0x7bc878ab0c7da618ULL,0xb37416441c6fb035ULL,0xc65bd5aea182fe9fULL,0x1b9c2fb86cc7a67aULL,0x8d1b19af5ce68d7dULL,+	0x3d1ab80c8ce59954ULL,0x742c5a9478222ac0ULL,0x3ddacbf894f878ddULL,0xfc085117e7d54a99ULL,0xfb0f1dfa21e38ec2ULL,0x1c7b59cb16f4ff7fULL,0x988752397ea888feULL,0x705d270cb10dc889ULL,+	0xe5aa692a87dec0e1ULL,0x010ded8df7b39d00ULL,0x7b1b80c854cfa0b5ULL,0x66beb876a0f8ea28ULL,0x50d7f5313476cd0eULL,0xa63d0e65b08d3949ULL,0x1a09eea953479fc6ULL,0x82ae9891f499e742ULL,+	0xab58b9105ca7d866ULL,0x582967e23adb3b34ULL,0x89ae4447cceac0bcULL,0x919c667c7bf56af5ULL,0x9aec17b160f5dcd7ULL,0xec697b9fddcaadbcULL,0x0b98f341463467f5ULL,0xb187f1f7a967132fULL,+	0xeb5ddcb6ec7fae9fULL,0x995f2714efb66e5aULL,0xdee95d8e69445d52ULL,0x1b6c2d4609e27620ULL,0x32621c318129d716ULL,0xb03909f10958c1aaULL,0x8c468ef91af4af63ULL,0x162c429ffba5cdf6ULL,+	0xe8cb5eef9c053df7ULL,0x8de25b37b300ea6fULL,0xdb03fa92c849cffbULL,0x242e43a7e84169bbULL,0xe4fa51f4dd6f958eULL,0x6925a77ff4445a8dULL,0xe6e72a50e90d8949ULL,0xc66648e32b1f6390ULL,+	0x6c3b96f31711ebecULL,0x2da40f1fce98fdc4ULL,0xb99774d357b4411fULL,0x87c8bdf415b65bb6ULL,0xda3a89e3c2eef12dULL,0xde95bb9b3c7471f3ULL,0x600f225bd812c594ULL,0x54907c5d2b75a56bULL,+	0x699e4d2945c1dd53ULL,0xcadc5898231debb5ULL,0xdf49fcc7a77f00e0ULL,0x93057bbfa73e5a0eULL,0x2f8b7ecd027a4cd1ULL,0x114734b3c614011aULL,0xe7a01db767677c68ULL,0x89d9be5e7e273f4fULL,+	0xe0deee5931fba239ULL,0xf47424d398bd91d1ULL,0x0f8886f4071a3c1dULL,0x3f7d41e8a819233bULL,0x708623c2cf6eb998ULL,0x86bb49af609a287fULL,0x942bb24963c90762ULL,0x0ef6eea555a9654bULL,+	0x4add4a2e649d4e57ULL,0xcd53a2b01917526eULL,0xc526233020b44ac4ULL,0x4028746abaa2c31dULL,0x5131839064291d4cULL,0xbf48f151ee5ad909ULL,0xcce57f597b185681ULL,0x7c3ac1b04854d442ULL,+	0xa80d1db6f79588c0ULL,0xfa52fc69b55768ccULL,0x0b4df1ae7f54438aULL,0x0cadd1a7f9b46a4fULL,0xb40ea6b31803dd6fULL,0x488e4fa555eaae35ULL,0x9f047d55382e4e16ULL,0xc9b5b7e02f6e0c98ULL,+	0xc19972d0b611c24bULL,0x1d468e6560a8f351ULL,0xeb7580697bcf6421ULL,0xec9dd0ee88fbc491ULL,0x5b59d2bf956c2e32ULL,0x73dc6864dcddf94eULL,0xfd5e2321bcee7665ULL,0xa7b4f8ef5e9a06c4ULL,+	0x03cc8f17cf41c6e8ULL,0xf1f03c2a037b925cULL,0xc39c19cc66d2427cULL,0x823d24ba7b6c18e4ULL,0x32ef9013901f0b4fULL,0x684360f1f8941c2eULL,0x0ebaff522c28092eULL,0x7891e4e3256c932fULL,+	0x174e8f82d8d38a9bULL,0x2e97c600e7de1391ULL,0xc5709850a1c175ddULL,0x969041a032ae5035ULL,0xcbfd533b76a2086bULL,0xd6bba71bd7c2e8feULL,0xb2d58ee6099dfb67ULL,0x3a8b342d064a85d9ULL,+	0xf83cbf0502f40d9aULL,0x4681c4682c318a4dULL,0x985756180e9c2674ULL,0xbe79d0461847092eULL,0xaf1e480a78bd01e0ULL,0x6dd359e472a51db9ULL,0x62ce3821e3afbab6ULL,0xc5cee5b617733199ULL,+	0x671ed8fc3b922bf8ULL,0xe4d8c0a04c29b133ULL,0x87eb12393b6e99c4ULL,0xaff3974c8793bebaULL,0x037494052c18df9bULL,0xc5c3a29391007139ULL,0x6a77234fe37a0b95ULL,0x02c29a21b661c96bULL,+	0x5a52fe2e34d74e31ULL,0xa352c3103bf79ab6ULL,0x97ff6c5aabfeeb8fULL,0xbfbe8feff5c97305ULL,0xd6081ce6a7904608ULL,0x1f812f3ac4fca249ULL,0x9b24bc9ab9e5e200ULL,0x91022c6738012ee8ULL,+	0x184de7d7cc5f4394ULL,0xb5551b5c4536e142ULL,0x2e89b212d34aa60aULL,0x14a96feaf50051d5ULL,0x4e21ef740d12bb0bULL,0xc522f02060b9677eULL,0x8b12e4672df7731dULL,0x39f803827b326d31ULL,+	0xc12738b67c4a658aULL,0xb3c4763940e72182ULL,0x3b77be468798e44fULL,0xdc047df217a7f85fULL,0x2439d4c55e59d92dULL,0xcedca475e8e64d8dULL,0xa724cd0d87ca9b16ULL,0x35e4fd59a5540dfeULL,+	0x9894344f3a29467aULL,0xde81e949c51eba6dULL,0xdaea066ba5e5c2f2ULL,0x3fc8a61408c8c7b3ULL,0x7adff88f06d0de9fULL,0xbbc11cf53b75ce0aULL,0x9fbb7accfbbc87d5ULL,0xa1458e267badfde2ULL,+	0x1cb43668e039c256ULL,0x5f26fb8b7c17fd5dULL,0xeee426af79aa062bULL,0x072002d0d78fbf04ULL,0x4c9ca237e84fb7e3ULL,0xb401d8a10c82133dULL,0xaaa525926d7e4181ULL,0xe943083373dbb152ULL,+	0x2f5fad1e6ee7c983ULL,0xeb0f9d7cb41328f5ULL,0x9ba68b441d78d5f7ULL,0xa06b3b9e35bc726fULL,0xa2550255593e1ff1ULL,0x552dd43ddfbec115ULL,0x2c48a7abbba8f046ULL,0xd4fc56a3ad0bf133ULL,+	0xf92dda31be24319aULL,0x03f7d28be095a8e7ULL,0xa52fe84098782185ULL,0x276ddafe29c24dbcULL,0x80cd54961d7a64ebULL,0xe43608897f1dbe42ULL,0x2f81a8778438d2d5ULL,0x7e4d52a885169036ULL,+	0x62f21cefba04b5d1ULL,0x9a442707224c7352ULL,0xbf07966c33c6171fULL,0xdb7ba8911e0816b0ULL,0x306fea59033745a9ULL,0x2aacf7e0c0a78f67ULL,0xb5aa3883ad251bf9ULL,0x345aede926bd7086ULL,+	0x19e3d5b11d59715dULL,0xc7eaa762d788983eULL,0xe5a730b0abf1f248ULL,0xfbab8084fae3fd83ULL,0x65e50d2153765b2fULL,0xbdd4e083fa127f3dULL,0x9cf3c074397b1b10ULL,0x59f8090cb1b59fd3ULL,+	0xdc1de17d98119f10ULL,0x74353c5d488c36a6ULL,0x14aaf33a3d8e23dfULL,0x31e075c078baf593ULL,0x0f7ca03a46d1ca3cULL,0x99c5e3ac47b660c7ULL,0x70d0241388fe2e59ULL,0x2e9a6be12a7ec005ULL,+	0x7b15fd9d615faa8fULL,0x8fa1eb40968554edULL,0x7bb4447e7aa44882ULL,0x2bb2d0d1029fff32ULL,0x075e2a646caa6d2fULL,0x8eb879de22e7351bULL,0xbcd5624e9a506c62ULL,0x218eaef0a87e24dcULL,+	0xac449695241fbd6fULL,0x67c9b170081c1223ULL,0x16868f21b56aac6fULL,0x34bd8fa3f8bcb721ULL,0x06b6bd33b6691c76ULL,0x6c924766381a7973ULL,0x6a12444ca54078dbULL,0xd02e91a96d1051ccULL,+	0x37e5684744ddfa35ULL,0x9ccfc5c5dab3f747ULL,0x9ac1df3f1ee96cf4ULL,0x0c0571a13b480b8fULL,0x2fbeb3d54b3a7b3cULL,0x35c036695dcdbb99ULL,0x52a0f5dcb2415b3aULL,0xd57759b44413ed9aULL,+	0x077379c00b33d3f8ULL,0x421883c67064e409ULL,0x2d0873d76c29c8f6ULL,0xbfa433a3d274c0c8ULL,0x56dc778f23a5891eULL,0xd663bf6535e2de04ULL,0x488fdb485db517ceULL,0x00bba55e19b226c2ULL,+	0x1fe647d83d30a2c5ULL,0x0857f77ef78a81dcULL,0x11d5a334131a4a9bULL,0xc0a94af929d393f5ULL,0xbc3a5c0bdaa6ec1aULL,0xba9fe49388d2d7edULL,0xbb4335b4bb614797ULL,0x991c4d6872f83533ULL,+	0xedbbeee78a058fb6ULL,0xb9d19ddcfb09121aULL,0xa41bb45bd34dddceULL,0x2dbc80b900964bc4ULL,0x4ed9137d1d6cb654ULL,0x1b9016db483d01c5ULL,0x5fc501bc6528e22eULL,0xb2d2f8816cad646bULL,+	0x53258c28d2f01cb3ULL,0x93d6eaa3d75db0b1ULL,0x419a2b0de87d0db4ULL,0xa1e48f03d8fe8493ULL,0xf747faf6c508b23aULL,0xf137571a35d53549ULL,0x9f5e58e2fcf9b838ULL,0xc7186ceea7fd3cf5ULL,+	0x5e76fb2f286bad39ULL,0xbad9efe39dcad1e2ULL,0x60e75190edc7e904ULL,0x6a6f063e0fecb5a5ULL,0x5150ed85aed8acc3ULL,0xb56ccfbc6d20af6cULL,0x7e0d1e982c69dbfaULL,0xabf5628a7c7e10a9ULL,+	0x77b868cee978a1d3ULL,0xe3a68b337ab92d04ULL,0x5102979487a5b862ULL,0x5f0606c33a61d41dULL,0x2814be276f9326f1ULL,0x2f521c14c6fe3c2eULL,0x17464d7dacdf7351ULL,0x10f5f9d3777f7e44ULL,+	0xb06b1244c5f95cd8ULL,0xda8c8af0f4ab95f4ULL,0x1bae59c2b9e5836dULL,0x07d51e7e3acffffcULL,0x01e15e6ac2ccbcdaULL,0x3bc1923f8528c3e0ULL,0x43324577a49fead4ULL,0x61a1b8842aa7a711ULL,+	0x4fe7ee31b0e63d34ULL,0xf4600572a9e54fabULL,0xc0493334d5e7b5a4ULL,0x8589fb9206d54831ULL,0xaa70f5cc6583553aULL,0x0879094ae25649e5ULL,0xcc90450710044652ULL,0xebb0696d02541c4fULL,+	0x172a5247d95ea168ULL,0x1758fada2970764aULL,0xac803a511d978169ULL,0x299cfe2ede77e01bULL,0x652a1e17b0a98927ULL,0x2e26e1d120014495ULL,0x7ae0af9f7175b56aULL,0xc2e22a80d64b9f95ULL,+	0x758c1a3ea2dee7a6ULL,0xdcde2f3c734b2284ULL,0xaba445d24eaba6adULL,0x35aaf66876cee0a7ULL,0x7e0b04a9e5aa049aULL,0xe74083ad91103e84ULL,0xbeb183ce40afecc3ULL,0x6b89de9fea043f7aULL,+	0xafbfb1eda4f7e665ULL,0x403cce6aa23df8e7ULL,0xb49cc83f1312c2f4ULL,0xe1cc2366771a9c34ULL,0x7ab6a6c0db92faacULL,0xacd15e0dec3befe1ULL,0x7f8ed988583a0f36ULL,0x1821de7705f9be09ULL,+	0xb99f0e0399375235ULL,0x7614c847b9917970ULL,0xfec93ce9524ec067ULL,0xe40e7bf89b122520ULL,0xb5670631ee4c4774ULL,0x6f03847a3b04914cULL,0xc96e9429dc9dd226ULL,0x43489b6c8c57c1f8ULL,+	0x7b9722a5c5f26464ULL,0xca4c3dfba442809aULL,0x7d10986723644810ULL,0x9e4951723c924f82ULL,0xef4a6968a2c5bc14ULL,0x750eac4f68de6b7aULL,0x4e01884d52a2cbb5ULL,0xac40830af4a5f446ULL,+	0x0e299d23fe67ba66ULL,0x9145076093cf2f34ULL,0xf45b5ea997fcf913ULL,0x5be008438bd7dddaULL,0x358c3e05d53ff04dULL,0xbf7ccdc35de91ef7ULL,0xad684dbfb69ec1a0ULL,0x367e7cf2801fd997ULL,+	0x8c54f1076103adf8ULL,0x2d813d6cbe8e9810ULL,0xb1466fa85fbd3c9bULL,0x68c65d2240e1ca76ULL,0xb81baa40255f9164ULL,0x5b34c3eed1a864b2ULL,0x3602209b122ca141ULL,0xe7d7248e885badebULL,+	0x46ffd227cc2338fbULL,0x89ff6fa990e26153ULL,0xbe570779331a0076ULL,0x43d241c506e1f3afULL,0xfdcdb97dde9b62a3ULL,0x6a06e984a0ae30eaULL,0xc9bf16804fbddf7dULL,0x170471a2d36163c4ULL,+	0xfd23e207a6469d43ULL,0xcb9f5f112f753a85ULL,0xde2625d4fbb5ca72ULL,0x82e4e54ab7b1c78bULL,0x2cd0ca5378b9e814ULL,0xfcd44051125b817aULL,0xb68f719f30cfd965ULL,0x31644719d848a974ULL,+	0xff5ba8ae3113655eULL,0xfa2c6e2b57b83180ULL,0x1c48271977e0eabeULL,0xf9f3c555337fea97ULL,0x340f7022a42581cbULL,0xe1de0bc218f710e3ULL,0xee640adef62e5aa8ULL,0x16b2389149428940ULL,+	0x5045738f25f653f5ULL,0xe42b8cb83764f635ULL,0xb4f89406dc11ffc3ULL,0x99593144b6b3e4aaULL,0x81c849f3c9740052ULL,0x2c9cf4c155ffc48bULL,0x6299e52177f67a49ULL,0x5869f6e3c00c6c62ULL,+	0x361619e455950cc3ULL,0xc71d665c56b66bb8ULL,0xea034b34afac6d84ULL,0xa987f832e5e4c7e3ULL,0xa07427727a79a6a7ULL,0x56e5d017e26d6c23ULL,0x7e50b97638167e10ULL,0xaa6c81efe88aa84eULL,+	0xb84186f75fe01576ULL,0x446e276cdea51395ULL,0xf3c5ef8105f3f8d4ULL,0x3d7f7df674f7f142ULL,0x7c69b565f9ef1656ULL,0x87efa4247c414ef6ULL,0xeb7e620d3d292060ULL,0x50b1de346eec21aeULL,+	0x0ca1f3b7b0dc8595ULL,0x27de46089f1d9f2eULL,0x1af3bf39badd82a7ULL,0x79356a7965862448ULL,0xc0602345f5f9a052ULL,0x1a8b0f89139a42f9ULL,0xb53eee42844d40fcULL,0x93b0bfe54e5b6368ULL,+	0x0f893a5dc8de610bULL,0xe8c515fb67e223ceULL,0x7774bfa64ead6dc5ULL,0x89d20f95925c728fULL,0x7a1e0966098583ceULL,0xa2eedb9493f2a7d7ULL,0x1b2820974c304d4aULL,0x0842e3dac077282dULL,+	0xa1010e9d74cd06ffULL,0x9c17c7dfaca3eeacULL,0x74c86cd38063aa2bULL,0x8595c4b3734614ffULL,0xa3de00ca990f62ccULL,0xd9bed213ca0c3be5ULL,0x7886078adf8ce9f5ULL,0xddb27ce35cd44444ULL,+	0x5a3097befc15aa1eULL,0x40d12548b54b0745ULL,0x5bad4706519a5f12ULL,0xed03f717a439dee6ULL,0x0794bb6c4a02c499ULL,0xf725083dcffe71d2ULL,0x2cad75190f3adcafULL,0x7f68ea1c43729310ULL,+	0x9c7c581d26ee8382ULL,0xcf17dcc5359d638eULL,0xee8273abb728ae3dULL,0x1d112926f821f047ULL,0x1149847750491a74ULL,0x687fa761fde0dfb9ULL,0x2c2580227ea435abULL,0x6b8bdb9491ce7e3fULL,+	0x9c806d8af7f91d0fULL,0x3b61b0f1a82a5728ULL,0x4640032d94d76754ULL,0x273eb5de47d834c6ULL,0x2988abf77b4e4d53ULL,0xb7ce66bfde401777ULL,0x9fba6b32715071b3ULL,0x82413c24ad3a1a98ULL,+	0x75537b7e3cc8ac85ULL,0x8d725f57dd02753bULL,0xfd05ff64b737df2fULL,0x55fe8712f6d2531dULL,0x57ce04a96ab6b01cULL,0x69a02a897cd93724ULL,0x4f82ac35cf86699bULL,0x8242d3ad9cb4b232ULL,+	0x69c435269be47be0ULL,0x323b7dd8cb28fea1ULL,0xfa5538ba3a6c67e5ULL,0xef921d701d378e46ULL,0xf92961fc3c4b880eULL,0x3f6f914e98940a67ULL,0xa990eb0afef0ff39ULL,0xa6c2920ff0eeff9cULL,+	0xb23a03a553fb2b56ULL,0x6ce141e74e057f78ULL,0x796525c389e490d9ULL,0x0bc95725a31a7e75ULL,0x1ec567911220fd06ULL,0x716e3a3c408b0bd6ULL,0x31cd6bf7e8ebeba9ULL,0xa7326ca6bee6b670ULL,+	0x70b63d32343bf1a9ULL,0x8fd3bd2837d1a6b1ULL,0x0454879c316865b4ULL,0xee959ff6c458efa2ULL,0x0461dcf89706dc3fULL,0x737db0e2164e4b2eULL,0x092626802f8843c8ULL,0x54498bbc7745e6f6ULL,+	0x5341352b5acf6e10ULL,0xc50343fdafe652c3ULL,0x4af3792d18577a7fULL,0xe1a4c617af16823dULL,0x9b26d0cd33425d0aULL,0x306399ed9b7bc47fULL,0x2a792f33706bb20bULL,0x3121961498111055ULL,+	0x4c1f428cd5f30851ULL,0x94dfed272a4f6630ULL,0x4df53772fc5d48a4ULL,0xdd2d5a2f933260ceULL,0x574115bdd44cc7a5ULL,0x4ba6b20dbd12533aULL,0x30e93cb8243057c9ULL,0x794c486a14de320eULL,+	0x6095355699f241d7ULL,0xee4adbd7001a349dULL,0x0b35bf6aaa89e491ULL,0x7f0076f4136f7546ULL,0xd19a18ba9264da3dULL,0x6eb2d2cd62a7a28bULL,0xcdba941f8761c971ULL,0x1550518ba3be4a5dULL,+	0xc232d97302f1cd1eULL,0xce87eacb1dd212a4ULL,0x6e4c8c73e69802f7ULL,0x12ef02901fffddbdULL,0x941ec74e1bcea6e2ULL,0xd0b540243cb92cbbULL,0x809fb9d47e8f9d05ULL,0x3bf16159f2992aaeULL,+	0xb2497007eafbb1e1ULL,0xd75c9ce6e75b7a93ULL,0x3558352defb68d78ULL,0xa2f26699223f6396ULL,0xeb911ecfe469b17aULL,0x62545779e72d3ec2ULL,0x8ea47de782cb113fULL,0xebe4b0864e1fa98dULL,+	0xbdb8e675b055cb40ULL,0x898f8e7b977b5167ULL,0xecc65651b82fb863ULL,0x565448146d88f01fULL,0xb0928e95263a75a9ULL,0xcfb6836f1a22fcdaULL,0x651d14db3f3bd37cULL,0x1d3837fbb6ad4664ULL,+	0x20d3c982cf7d62d2ULL,0x1f36e29d23ba8150ULL,0x48ae0bf092763f9eULL,0x7a527e6b1d3a7007ULL,0xb4a89097581a85e3ULL,0x1f1a520fdc158be5ULL,0xf98db37d167d726eULL,0x8802786e1113e862ULL,+	0xefb2149e36f09ab0ULL,0x03f163ca4a10bb5bULL,0xd029704506e20998ULL,0x56f0af001b5a3babULL,0x7af4cfec70880e0dULL,0x7332a66fbe3d913fULL,0x32e6c84a7eceb4bdULL,0xedc4a79a9c228f55ULL,+	0xc37c7dd0c55c4496ULL,0xa6a9635725bbabd2ULL,0x5b7e63f2add7f363ULL,0x9dce37822e73f1dfULL,0xe1e5a16ab2b91f71ULL,0xe44898235ba0163cULL,0xf2759c32f6e515adULL,0xa5e2f1f88615eecfULL,+	0x74519be7abded551ULL,0x03d358b8c8b74410ULL,0x4d00b10b0e10d9a9ULL,0x6392b0b128da52b7ULL,0x6744a2980b75c904ULL,0xc305b0aea8f7f96cULL,0x042e421d182cf932ULL,0xf6fc5d509e4636caULL,+	0x795847c9d64cc78cULL,0x6c50621b9b6cb27bULL,0x07099bf8df8022abULL,0x48f862ebc04eda1dULL,0xd12732ede1603c16ULL,0x19a80e0f5c9a9450ULL,0xe2257f54b429b4fcULL,0x66d3b2c645460515ULL,+	0x6ca4f87e822e37beULL,0x73f237b4253bda4eULL,0xf747f3a241190aebULL,0xf06fa36f804cf284ULL,0x0a6bbb6efc621c12ULL,0x5d624b6440b80ec6ULL,0x4b0724257ba556f3ULL,0x7fa0c3543e2d20a8ULL,+	0xe921fa31e3229d41ULL,0xa929c65294531bd4ULL,0x84156027a6d38209ULL,0xf3d69f736bdb97bdULL,0x8906d19a16833631ULL,0x68a34c2e03d51be3ULL,0xcb59583b0e511cd8ULL,0x99ce6bfdfdc132a8ULL,+	0x3facdaaaffcdb463ULL,0x658bbc1a34a38b08ULL,0x12a801f8f1a9078dULL,0x1567bcf96ab855deULL,0xe08498e03572359bULL,0xcf0353e58659e68bULL,0xbb86e9c87d23807cULL,0xbc08728d2198e8a2ULL,+	0x8de2b7bc453cadd6ULL,0x203900a7bc0bc1f8ULL,0xbcd86e47a6abd3afULL,0x911cac128502effbULL,0x2d550242ec965469ULL,0x0e9f769229e0017eULL,0x633f078f65979885ULL,0xfb87d4494cf751efULL,+	0xe1790e4bfc25419aULL,0x364672034bff3cfdULL,0xc8db638625b6e83fULL,0x6cc69f236cad6fd2ULL,0x0219e45a6bc68bb9ULL,0xe43d79b6297f7334ULL,0x7d445368465dc97cULL,0x4b9eea322a0b949aULL,+	0x1b96c6ba6102d021ULL,0xeaafac782f4461eaULL,0xd4b85c41c49f19a8ULL,0x275c28e4cf538875ULL,0x35451a9ddd2e54e0ULL,0x6991adb50605618bULL,0x5b8b4bcd7b36cd24ULL,0x372a4f8c56f37216ULL,+	0xc890bd73a6a5da60ULL,0x6f083da0dc4c9ff0ULL,0xf4e14d94f0536e57ULL,0xf9ee1edaaaec8243ULL,0x571241ec8bdcf8e7ULL,0xa5db82710b041e26ULL,0x9a0b9a99e3fff040ULL,0xcaaf21dd7c271202ULL,+	0xb4e2b2e14f0dd2e8ULL,0xe77e7c4f0a377ac7ULL,0x69202c3f0d7a2198ULL,0xf759b7ff28200eb8ULL,0xc87526eddcfe314eULL,0xeb84c52453d5cf99ULL,0xb1b52ace515138b6ULL,0x5aa7ff8c23fca3f4ULL,+	0xff0b13c3b9791a26ULL,0x960022dacdd58b16ULL,0xdbd55c9257aad2deULL,0x3baaaaa3f30fe619ULL,0x9a4b23460d881efdULL,0x506416c046325e2aULL,0x91381e76035c18d4ULL,0xb3bb68bef27817b0ULL,+	0x15bfb8bf5116f937ULL,0x7c64a586c1268943ULL,0x71e25cc38419a2c8ULL,0x9fd6b0c48335f463ULL,0x4bf0ba3ce8ee0e0eULL,0x6f6fba60298c21faULL,0x57d57b39ae66bee0ULL,0x292d513022672544ULL,+	0xf451105dbab093b3ULL,0x012f59b902839986ULL,0x8a9158023474a89cULL,0x048c919c2de03e97ULL,0xc476a2b591071cd5ULL,0x791ed89a034970a5ULL,0x89bd9042e1b7994bULL,0x8eaf5179a1057ffdULL,+	0x6066e2a2d551ee10ULL,0x87a8f1d8727e09a6ULL,0x00d08bab2c01148dULL,0x6da8e4f1424f33feULL,0x466d17f0cf9a4e71ULL,0xff5020103bf5cb19ULL,0xdccf97d8d062ecc0ULL,0x80c0d9af81d80ac4ULL,+	0x98857ceb1bf4581cULL,0xe635e186aca7b166ULL,0x278ddd22659722acULL,0xa0903c4c1db68007ULL,0x366e458948f21402ULL,0x31b49c14b96abda2ULL,0x329c4b09e0403190ULL,0x97197ca3d29f43feULL,+	0x03e2de1cf3480d4aULL,0xf0d8edc7bc8acf1aULL,0xf23e330368295a9cULL,0xfadd5f68c546a97dULL,0x895597ad96f8acb1ULL,0xbddd49d5671bdae2ULL,0x16fcd52821dd43f4ULL,0xa5a454126619141aULL,+	0xfa32c79f439f29cfULL,0x7bf321c04dd82a3bULL,0xff127f54eaa2c1b1ULL,0xa8365f2df35e9618ULL,0x852d29024d0ef8ddULL,0x395ce2c159228c4aULL,0xb69f44e8215afed3ULL,0x16c1f898b27458e3ULL,+	0x8ce9b6bfc360e25aULL,0xe6425195075a1a78ULL,0x9dc756a8481732f4ULL,0x83c0440f5432b57aULL,0xc670b3f1d720281fULL,0x2205910ed135e051ULL,0xded14b0edb052be7ULL,0x697b3d27c568ea39ULL,+	0xdef29005dc453233ULL,0xc208c47a2fac4bcfULL,0x6057a3feac3d55acULL,0x1723725d902c1207ULL,0x9c31c62733eb0fecULL,0x4913ccdbbb3c63bbULL,0x113e542b07305838ULL,0x9d48e72a310c2d97ULL,+	0x54424ec4d9bbeb3dULL,0x34ceafe3d7b2921cULL,0x5e68022e296d37c5ULL,0xa28e0a2b359f16b0ULL,0xfdd82dd9bc3f9d73ULL,0x6939a8f9baf3e1ffULL,0x55cff45c31736dc4ULL,0x910f56427892e8e7ULL,+	0xde09b349dfe39260ULL,0x984612f773549093ULL,0x7cede28167853b02ULL,0x7d809bb429d17703ULL,0x450b6bfb2756c4f0ULL,0xc59ff9ba93f02b80ULL,0x3e69545720ad9561ULL,0x0c7cf0be3331e2c7ULL,+	0x2e599b9afb3ff9edULL,0x28c2e0ab17f6515cULL,0x1cbee4fd474da449ULL,0x071279a44f364452ULL,0x97abff6601fbe855ULL,0x3ee394e85fda51c4ULL,0x190385f667597c0bULL,0x6e9fccc6a27ee34bULL,+	0xced2d419362fb228ULL,0x894637c206aa0be4ULL,0x7a4fc55eb294b197ULL,0xd9cbac1cfd4ca1e8ULL,0x068b74800ccdb6ceULL,0xca4c556580dfaa49ULL,0xe835382176033b78ULL,0x35db7525c5ce98a7ULL,+	0x8cf5927274fd1997ULL,0x987d2031ffadeb58ULL,0x5647c6c3d4db260cULL,0xf08bb13b985543beULL,0x566eeb1056fad695ULL,0x39e17dde68334cc7ULL,0x8a97b3bb7ddd1db2ULL,0xf91199d8c1c5a300ULL,+	0x402fa437bdc47fe4ULL,0x35bd25234f3751f0ULL,0x8c2281b7cf57d485ULL,0x50083ea58f607cb6ULL,0x6f41e480df6d730dULL,0x1164e47e91bb06a2ULL,0xd9040c22391fb48aULL,0xe12df251da23dda7ULL,+	0x0b89de9314092ebbULL,0xf17256bd428e240cULL,0xcf89a7f393d2f064ULL,0x4f57841ee1ed3b14ULL,0x4ee14405e708d855ULL,0x856aae7203f1c3d0ULL,0xc8e5424fbdd7eed5ULL,0x3333e4ef73ab4270ULL,+	0xf9cf2b5148f835b1ULL,0x4a70faf32adaaba8ULL,0xa0d2e24fed7beadeULL,0x2b1e4e6715f04032ULL,0xeadbc0b91fb3ceeeULL,0xb817b0863a54b0b4ULL,0x14a787257fd4a188ULL,0xff13304623482e60ULL,+	0x312012ec4efe42dcULL,0x7251b2c2c664b2bcULL,0x996c2e6b798f9cb9ULL,0x3543376a3b8f3465ULL,0x337ae8416fbfb6a9ULL,0xe0893a840cb91a03ULL,0x53744e9d7b02d855ULL,0x7e673186206d473fULL,+	0x9509996f3ef143dbULL,0x9cf1e82e399378bcULL,0xb59cd09b0b8ccc89ULL,0x64b4a2fc52cda6beULL,0xd3bda8b31bcd55dfULL,0xfec4e87d2507cb37ULL,0x3d48a85eb610e49bULL,0x02fbe1bd1f9cc445ULL,+	0x3bc77adedda492f8ULL,0xc11a3aea78297205ULL,0x5e89a3e734931b4cULL,0x17512e2e9f5694bbULL,0x5dc349f3177bf8b6ULL,0x232ea4ba08c7ff3eULL,0x9c4f9d16f511145dULL,0xccf109a333b379c3ULL,+	0x60e2857080eb24a9ULL,0x7bedfb4d488e0cfdULL,0x721ebbd7c259cdb8ULL,0x0b0da855bc6390a9ULL,0x2b4d04dbde314c70ULL,0xcdbf1fbc6c32e846ULL,0x33833eabb162fc9eULL,0x9939b48bb0dd3ab7ULL,+	0x96892c1f711b0eb9ULL,0xb905f2c8780ab954ULL,0xace26309a20792dbULL,0xec8ac9b30684e126ULL,0x486ad8b6b40a2447ULL,0x60121fc19fe3fb24ULL,0x5626fccf1a8e3b3fULL,0x4e5686226ad1f394ULL,+	0x5cfbbb22236f4a98ULL,0x0b0c59e9066800bbULL,0x4ac69a8f5a9a7774ULL,0x2b33f804d6bec948ULL,0xb372929532e6c466ULL,0x68956d0f4e599c73ULL,0xa47a249f155c31ccULL,0x24d80f0de1ce284eULL,+	0x5a4b46c64a8a3d62ULL,0x8469c4d0247743d2ULL,0x2bb3a13d88f7e433ULL,0x62b23a1001be5849ULL,0xe83596b4a63d1a4cULL,0x454e7fea7d183f3eULL,0x643fce6117afb01cULL,0x4e65e5e61c4c3638ULL,+	0xb7e830e3dc09508bULL,0xfaf6d2cf74317655ULL,0x72606cebdf690355ULL,0x48bb92b3d0c3ded6ULL,0x65b754845c7cf892ULL,0xf6cd7ac9d5d5f01fULL,0xc2c30a5996401d69ULL,0x91268650ed921878ULL,+	0xe5db77176add8545ULL,0x1b71cb6672c49b66ULL,0xd856073968421d77ULL,0x03840fe883e3afeaULL,0xb391dad51ec69977ULL,0xae243fb9307f6726ULL,0xc88ac87be8ca160cULL,0x5174cced4ce355f4ULL,+	0x752067f8ff81578eULL,0x786221509045447dULL,0xc0c22fcf0505aa6fULL,0x1030f0a66bed1c77ULL,0x31f29f151f0bd739ULL,0x2d7989c7e6debe85ULL,0x5c070e728e677e98ULL,0x0a817bd306e81fd5ULL,+	0xc1e17eb6cbc613e5ULL,0x33131d55497ea61cULL,0x2f69d39eaf7eded5ULL,0x73c2f434de6af11bULL,0x4ca52493a4a375faULL,0x5f06787cb833c5c2ULL,0x814e091f3e6e71cfULL,0x76451f578b746666ULL,+	0x01c7e082e1539388ULL,0xfd286f30759a9c6bULL,0x94581041176bacfbULL,0xe580f07c3bc3de53ULL,0xdcdb6f75884d772fULL,0xd75c3bb840bb9d4fULL,0xdc6c2e4edb083011ULL,0xe18789a2cd9c298dULL,+	0x5ee6ab8495fe1347ULL,0xab0f6c396f24503cULL,0x807e3ffb4486dd6bULL,0xf00b6c748002fef5ULL,0x48bff9a6a7862999ULL,0x85e5a06cbed89e26ULL,0x86d311af3d8419ebULL,0x24f3ad7834733f16ULL,+	0x5e20551311820d44ULL,0x0c651bcc86c4473aULL,0x1d230c2b9bd80eefULL,0x042c4a1207515be5ULL,0x42517ca0bfe9e284ULL,0xe8f605782369827aULL,0x184f04f01638699dULL,0x174618edf2bc6d05ULL,+	0x5e3e03fc6c68d687ULL,0x3e732c3d1ff052c7ULL,0xf2d0efa66ed16e7aULL,0x63d92b26b65bb746ULL,0xffcd82badd44867cULL,0xa71b4a9ef8c081b8ULL,0x6c1676a7736c8785ULL,0xbe2c06169d8932d0ULL,+	0x7bfa1a4b9eca1c9fULL,0x960bc1dc8ce5e535ULL,0xe267d9f317eec30aULL,0x06fb89ef6c257d38ULL,0x2328999ad364a26dULL,0x69b794cb26eaab58ULL,0xad28ab1fb85ba596ULL,0x05dcbff356d0aa94ULL,+	0x53376d282bcffbc4ULL,0x708817a706eadb7aULL,0x6ff50e05cd35ae69ULL,0x63b5fb7574bc7fdeULL,0x71c9e953e7fe08c4ULL,0xb4d8bfd4f583ca18ULL,0xde8d788245e81c5cULL,0xa5f5e93ce0474138ULL,+	0x426c160f293c9f31ULL,0x8eb56333e864d7a7ULL,0xbe1164023ebbba30ULL,0x64c2bae32fd5a302ULL,0x800601e1265aff7bULL,0x52d8a88066fd4b14ULL,0x5aba20e746075a9cULL,0xdaa32bf87e1234c6ULL,+	0x80f9bdef694db7e0ULL,0xedca8787b9fcddc6ULL,0x51981c3403b8dce1ULL,0x4274dcf170e10ba1ULL,0xf72743b86def6d1aULL,0xd25b1670ebdb1866ULL,0xc4491e8c050c6f58ULL,0x2be2b2ab87fbd7f5ULL,+	0x3e0e5c9dd111f8ecULL,0xbcc33f8db7c4e760ULL,0x702f9a91bd392a51ULL,0x7da4a795c132e92dULL,0x1a0b0ae30bb1151bULL,0x54febac802e32251ULL,0xea3a5082694e9e78ULL,0xe58ffec1e4fe40b8ULL,+	0xfbb8349d29c4120bULL,0x9f94391fc0d0d915ULL,0xc4074fa75410ba51ULL,0xa66adbf6150a5911ULL,0xc164543c34bfca38ULL,0xe0f27560b9e1ccfcULL,0x99da0f53e820219cULL,0xe8234498c6b4997aULL,+	0x7b23c513516e19e4ULL,0x56e2e847c5c4d593ULL,0x9f727d735ce71ef6ULL,0x5b6304a6f79a44c5ULL,0x6638a7363ab7e433ULL,0x1adea470fe742f83ULL,0xe054b8545b7fc19fULL,0xf935381aba1d0698ULL,+	0xb5504f9d918e4936ULL,0x65035ef6b2513982ULL,0x0553a0c26f4d9cb9ULL,0x6cb10d56bea85509ULL,0x48d957b7a242da11ULL,0x16a4d3dd672b7268ULL,0x3d7e637c8502a96bULL,0x27c7032b730d463bULL,+	0x55366b7d5846426fULL,0xe7d09e89247d441dULL,0x510b404d736fbf48ULL,0x7fa003d0e784bd7dULL,0x25f7614f17fd9596ULL,0x49e0e0a135cb98dbULL,0x2c65957b2e83a76aULL,0x5d40da8dcddbe0f8ULL,+	0x37f68bb4a595939dULL,0x0355647928740217ULL,0x8e740e7c84ad7612ULL,0xd89bc8439044695fULL,0xf7f3da5d85a9184dULL,0x562563bb9fc0b074ULL,0x06d2e6aaf88a888eULL,0x612d8643161fbe7cULL,+	0x9fb3bba354530bb2ULL,0xbde3ef77cb0869eaULL,0x89bc90460b431163ULL,0x4d03d7d2e4819a35ULL,0x33ae4f9e43b6a782ULL,0x216db3079c88a686ULL,0x91dd88e000ffedd9ULL,0xb280da9f12bd4840ULL,+	0x458f86913e538cd7ULL,0xa7001f6c8e08ad53ULL,0x52b8c6e6bf5d15ffULL,0x548234a4011215ddULL,0xff5a9d2d3d5b4045ULL,0xb0ffeeb64a904190ULL,0x55a3aca448607f8bULL,0x8cbd665c30a0672aULL,+	0xa37f3573f37f5937ULL,0xeb0f6c7dd1e4fca5ULL,0x2965a554ac8ab0fcULL,0x17fbf56c274676acULL,0x2e2f6bd9acf7d720ULL,0x41fc8f8810224766ULL,0x517a14b385d53befULL,0xdae327a57d76a7d1ULL,+	0x515d5c891f5f82dcULL,0x9a7f67d76361079eULL,0xa8da81e311a35330ULL,0xe44990c44b18be1bULL,0xc7d5ed95af103e59ULL,0xece8aba78dac9261ULL,0xbe82b0999394b8d3ULL,0x6830f09a16adfe83ULL,+	0x43c41ac194d7d9b1ULL,0x5bafdd82c82e7f17ULL,0xdf0614c15fda0fcaULL,0x74b043a7a8ae37adULL,0x3ba6afa19e71734cULL,0x15d5437e9c450f2eULL,0x4a5883fe67e242b1ULL,0x5143bdc22c1953c2ULL,+	0xa2a9ce7c6b53f5f9ULL,0x642465951b176d99ULL,0xb1298d36b95c081bULL,0x53505bb81d9a9ee6ULL,0x3f6f9e61f2ba70b0ULL,0xd07e16c98afad453ULL,0x9f1694bbe7eb4a6aULL,0xdfebced93cb0bc8eULL,+	0xc676d7f2b1f3390bULL,0x9f7a1b8ca5b61272ULL,0x4ebebfc9c2e127a9ULL,0x4602500c5dd997bfULL,0x7f09771c4711230fULL,0x058eb37c020f09c1ULL,0xab693d4bfee5e38bULL,0x9289eb1f4653cbc0ULL,+	0xa44d2b391770f5a7ULL,0xe4d4d7910e44eb82ULL,0x42e69d1e3f69712aULL,0xbf11c4d6ac6a820eULL,0xb5e7f3e542c4224cULL,0xd6b4e81c449d941cULL,0x5d72bd165450e878ULL,0x6a61e28aee25ac54ULL,+	0x54da9dc7ab952578ULL,0xb5423df226e84d0bULL,0xa8b64eeb9b872042ULL,0xac2057825990f6dfULL,0x4ff696eb21f4c77aULL,0x1a79c3e4aab273afULL,0x29bc922e9436b3f1ULL,0xff807ef8d6d9a27aULL,+	0xc7f3a8f833f6746cULL,0x21e46f65fea990caULL,0x915fd5c5caddb0a9ULL,0xbd41f01678614555ULL,0x346f4434426ffb58ULL,0x8055943614dbc204ULL,0xf3dd20fe5a969b7fULL,0x9d59e956e899a39aULL,+	0x3c2f0ba9b733aa5fULL,0xdece47cbf05af235ULL,0xf8e3f715a2ac82a5ULL,0xc97ba6412203f18aULL,0xc3af550409c11060ULL,0x56ea2c0546af512dULL,0xfac28daff3f28146ULL,0x87fab43a959ef494ULL,+	0x09891641d4c5105fULL,0x1ae80f8e6d7fbd65ULL,0x9d67225fbee6bdb0ULL,0x3b433b597fc4d860ULL,0x44e66db693e85638ULL,0xf7b59252e3e9862fULL,0xdb785157665c32ecULL,0x702fefd7ae362f50ULL,+	0xfe756a5c97290293ULL,0xbf04a19cd388acbfULL,0xfbbbb9cf5e916bdaULL,0xf489527391f93becULL,0xdee07ec32a5923d7ULL,0xc7bc949bfde0c370ULL,0xbd5121750419d8fcULL,0x54f5d4763fdcc93fULL,+	0x3754475d0fefb0c3ULL,0xd48fb56b46d7c35dULL,0xa070b633363798a4ULL,0xae89f3d28fdb98e6ULL,0x970b89c86363d14cULL,0x8981752167abd27dULL,0x9bf7d47444d5a021ULL,0xb3083bafcac72aeeULL,+	0x0acda2ffcc5e62e9ULL,0x3b8b7d755edb02a4ULL,0xa700741c66120c76ULL,0xf77e847f7d974064ULL,0x0d310678a5e3d464ULL,0xde68b1f346bf35a3ULL,0xcae83028d32f9043ULL,0x724e4717517cb0cbULL,+	0x389741debe949a44ULL,0x638e9388546a4fa5ULL,0x3fe6419ca0047bdcULL,0x7047f648aaea57caULL,0x54e48a9041fbab17ULL,0xda8e0b28576bdba2ULL,0xe807eebcc72afddcULL,0x07d3336df42577bfULL,+	0xfae8563b4f3011dbULL,0xda33776536610c03ULL,0xad4f6f2a6381af0bULL,0xc277984cd95378ecULL,0x5ab0a10c5751d2b2ULL,0x70a18bb97a4bf3f4ULL,0x38d07ad4eae3caf3ULL,0xe8ef552fec430361ULL,+	0x62a8c244bfe20925ULL,0x91c19ac38fdce867ULL,0x5a96a5d5dd387063ULL,0x61d587d421d324f6ULL,0xe87673a2a37173eaULL,0x2384800853778b65ULL,0x10f8441e05bab43eULL,0xfa11fe124621efbeULL,+	0xedd0389a8391d54bULL,0xdac74c08c8afe546ULL,0x8525a4ad5be60bbdULL,0x2419ac9690ad7b87ULL,0x078a3a0277eee51bULL,0xe86ecf367239768dULL,0xcb0a259d8fd48035ULL,0x94db43cb9d29ca5bULL,+	0x047b772e81685d7bULL,0x23f27d81bf34a976ULL,0xc27608e2915f48efULL,0x3b0b43faa521d5c3ULL,0x7613fb2663ca7284ULL,0x7f5729b41d4db837ULL,0x87b14898583b526bULL,0x00b732a6bbadd3d1ULL,+	0x4fffa25b90f8550aULL,0x254db3d31c8dae9dULL,0x58cef963c1fbb232ULL,0xad1cb481a4bfdf0fULL,0x84d26ea1958773c2ULL,0x58622664010114f1ULL,0xeadb3a87f051e67eULL,0xf3185722b69e45c6ULL,+	0x8e02f4262048e396ULL,0x436b50b6383d9de4ULL,0xf78d3481471e85adULL,0x8b01ea6ad005c8d6ULL,0xd3c7afee97015c07ULL,0x46cdf1a94e3ba2aeULL,0x7a42e50183d3a1d2ULL,0xd54b5268b541dff4ULL,+	0xe120c4f948a48e22ULL,0xf24977eed0335a96ULL,0x9af3442336151c7bULL,0xe2815a7fd36648d8ULL,0xb4d2deba66e0a6b2ULL,0x783a84cf817515e8ULL,0x78424c0bff3ff24aULL,0x12dd1bb638e1c5d9ULL,+	0x3f24cf304e23e9bcULL,0x4387f816126e3624ULL,0x26a46a033b0b6d61ULL,0xaf1bc8458b2d777cULL,0x25c401ba527de79cULL,0x0e1346d44261bbb6ULL,0x4b96c44b287b4bc7ULL,0x658493c75254562fULL,+	0xfcb0e9d8fdd41d98ULL,0x8be980c1bcc7c7fbULL,0xa72e86506f8fa4d9ULL,0x356b14ad748cb88dULL,0xfd9fccefea6178f2ULL,0xc84a620d78bb0e35ULL,0xbc75367c62f391a9ULL,0x6a83a5c623100c05ULL,+	0x23f949feb8a24a20ULL,0x17ebfed1f52ca53fULL,0x9b691bbebcfb4853ULL,0x5617ff6b6278a05dULL,0x241b34c5e3c99ebdULL,0xfc64242e1784156aULL,0x4206482f695d67dfULL,0xb967ce0eee27c011ULL,+	0xb4480f0441c23fa3ULL,0xb4712eb0c1989a2eULL,0x3ccbba0f93a29ca7ULL,0x6e205c14d619428cULL,0x90db7957b3641686ULL,0x0432691d45ac8b4eULL,0x07a759acf64e0350ULL,0x0514d89c9c972517ULL,+	0xe3b22c6bc4fe3c39ULL,0xba4a81536c7bebdfULL,0xf23ab6b725693459ULL,0x53bc377014922b11ULL,0x4645c8ab5afc60dbULL,0xaa02235520b9f2a3ULL,0x52a2954cce0fc507ULL,0x8c2731bb7ce1c2e7ULL,+	0x5066efb6d9790ed6ULL,0xa77a0cbca6aa793bULL,0x1a915f3c223e042eULL,0x1c5def0469c5874bULL,0x0e83007873b6c1daULL,0x55cf85d2fcd8557aULL,0x0f7c7c760460f3b1ULL,0x87052acb46e58063ULL,+	0x6a7091c2e48fb889ULL,0x26882c137b8a9d06ULL,0xa24986631b82a0e2ULL,0x844ed7363518152dULL,0x282f476fd86e27c7ULL,0xa04edaca04afefdcULL,0x8b256ebc6119e34dULL,0x56a413e90787d78bULL,+	0x16eab6a20d645fd6ULL,0x632cbd8df61d3148ULL,0xcc1bf7cf62079ae9ULL,0x257ee5c7f33eccbbULL,0xbf6b34a81680ac73ULL,0xaa084e8872c77aa0ULL,0x7b5a864e05a0a1d1ULL,0x0641f6db359a1b16ULL,+	0xf01d095dc8385050ULL,0x0d54a5d5df4b441cULL,0x2a37ccb40927706aULL,0xdf008f5445d7eb7eULL,0x74eb34f35bf716c7ULL,0x57a65b58641bd6caULL,0xef345e4835e6fa02ULL,0x191f913b88342a09ULL,+	0x1554d46da670ff1dULL,0x24833d88cb97a1ccULL,0x8fa6ab3cded97493ULL,0x215e037189926498ULL,0x549bd592e56d74ffULL,0x58a8caf543b5e1ecULL,0x3c6087a323e93cb9ULL,0x8b0549875648b83cULL,+	0x82ee061d5a74be50ULL,0xe41781c4dea16ff5ULL,0xe0b0c81e99bfc8a2ULL,0x624f4d690b547e2dULL,0x3a83545dbdcc9ae4ULL,0x2573dbb6409b1e8eULL,0x482960c4a6c93539ULL,0xf01059ad5ae18798ULL,+	0xc431a238013ff83bULL,0x7c0018b2fad69d08ULL,0x99aeb52a4c9589eaULL,0x121f41ab9b1cf19fULL,0x0cfbbcbaef0f5958ULL,0x8deb3aeb7be8fbdcULL,0x12b954081f15aa31ULL,0x5acc09b34c0c06fdULL,+	0x775cbfa86d518ffbULL,0xdecee1f6930f124bULL,0x9a402804f5e81d0fULL,0x0e8225c52a0eeb2fULL,0x884a5d39fee9e867ULL,0x9540428ffb505454ULL,0xb2bf2e20107a70d1ULL,0xd9917c3ba010b2aaULL,+	0xa98f42fa3d843d53ULL,0x33777cc613ef927aULL,0xc440cdbecb84ca74ULL,0x8c22f9631dc7c5ddULL,0x4bc82b70c8d94708ULL,0x7e0b43fcc814364fULL,0x286d4e2486f59b7eULL,0x1abc895e4d6bf4c4ULL,+	0x38151e274d559d96ULL,0x4f18c0d3b8db6c01ULL,0x49a3aa836f9921afULL,0xdbeab27b8c046029ULL,0x242b9eaa7040bf3bULL,0x39c479e51614b091ULL,0x338ede2b0e4baf5dULL,0x5bb192b7f0a53945ULL,+	0x896d572337e440d7ULL,0x685c5fd9ade23f68ULL,0xb5b1a26dc2c64918ULL,0xb9390e30dad6580cULL,0x87911c4e7dee5b9bULL,0xb90c5053deb04f6eULL,0x37b942a18f065aa6ULL,0x34acdf2a1ca0928dULL,+	0x733b64d39de40ca3ULL,0x1d4b6d6fd2f3857eULL,0xbe2be8e9b2ed92f7ULL,0x64ca7047b77da248ULL,0xc65dae9b8da99315ULL,0x9c1451750fc698a4ULL,0x8a296b94ff958c27ULL,0x38684e0843950097ULL,+	0x7872e34b3390ff23ULL,0x968ce4abde7d18efULL,0x9b4a745e627fe7b1ULL,0x9607b0a0caff3e2aULL,0x1b05818eeb40e3a5ULL,0x6ac62204c0fa8d7aULL,0xb5b9058571ed4809ULL,0xb2432ef0f7cb65f2ULL,+	0x715c9f973112795fULL,0xe8244437984e6ee1ULL,0x55cb4858ecb66bcdULL,0x7c136735abaffbeeULL,0x546615955dbec38eULL,0x51c0782c388ad153ULL,0x9ba4c53ac6e0952fULL,0x27e6782a1b21dfa8ULL,+	0xfeb09740e2c2bf15ULL,0x627a2205a9e99704ULL,0xec8d73d0c2fbc565ULL,0x223eed8fc20c8de8ULL,0x1ee32583a8363b49ULL,0x1a0b6cb9c9c2b0a6ULL,0x49f7c3d290dbc85cULL,0xa8dfbb971ef4c1acULL,+	0xc16c236e846e364fULL,0x7f33527cdea50ca0ULL,0xc48107750926b86dULL,0x6c2a36090598e70cULL,0xa6755e52f024e924ULL,0xe0fa07a49db4afcaULL,0x15c3ce7d66831790ULL,0x5b4ef350a6cbb0d6ULL,+	0x42806b2da6dc1d29ULL,0xd3030009f871e144ULL,0xa1feb333aaf49276ULL,0xb5583b9ec70bc04bULL,0x1db0be7895695f20ULL,0xfc84181189d012b5ULL,0x6409f27205f61643ULL,0x40d34174d5883128ULL,+	0x05214c050f15dde9ULL,0xa47a76a80d5f2b82ULL,0xbb254d3062e82b62ULL,0x11a05fe03ec955eeULL,0x7eaff46e9d529b36ULL,0x55ab13018f9e3df6ULL,0xc463e37199317698ULL,0xfd251438ccda47adULL,+	0x8c3c669c72ba075bULL,0x89f78b55ba469015ULL,0x5706aade3e9f8ba8ULL,0x6d8bd565b32d7ed7ULL,0x25f4e63b805f08d6ULL,0x7f48200dc3bcc1b5ULL,0x4e801968b025d847ULL,0x74afac0487cbe0a8ULL,+	0xe2a37598a9d82abfULL,0x5f188ccbe6c170f5ULL,0x816822005066b087ULL,0xda22c212c7155adaULL,0x151e5d3afbddb479ULL,0x4b606b846d715b99ULL,0x4a73b54bf997cb2eULL,0x9a1bfe433ecd8b66ULL,+	0x79732522cccc18adULL,0xaadf3f8df1a6e027ULL,0xf7382c9317c2354dULL,0x5ce1680cd818b689ULL,0x359ebbfcd9ecbee9ULL,0x4330689c1cae62acULL,0xb55ce5b4c51ac38aULL,0x7921dfeafe238ee8ULL,+	0xe13122f3dbfb894eULL,0xbe9b79f6ce274b18ULL,0x85a49de5ca58aadfULL,0x2495775811487351ULL,0x111def61bb939099ULL,0x1d6a974a26d13694ULL,0x4474b4ced3fc253bULL,0x3a1485e64c5db15eULL,+	0x65994ddb0f5f27caULL,0xe85461fba80d59ffULL,0xff05481a66601023ULL,0xc665427afc9ebbfbULL,0xb0571a697587fd52ULL,0x935289f88d49efceULL,0x61becc60ea420688ULL,0xb22639d913a786afULL,+	0x5afddab61430c9abULL,0x0bdd41d32238e997ULL,0xf0947430418042aeULL,0x71f9addacdddc4cbULL,0x7090c016c52dd907ULL,0xd9bdf44d29e2047fULL,0xe6f1fe801b1011a6ULL,0xb63accbcd9acdc78ULL,+	0x264c76680448087cULL,0xac30903f71432daeULL,0x3851b26600f9bf47ULL,0x400ed3116cdd6d03ULL,0x045e79fef8fd2424ULL,0xfdfd974afa6da98bULL,0x45c9f6410c1e673aULL,0x76f2e7335b2c5168ULL,+	0x7817acab4baef62eULL,0x9f5a2202a85b91e8ULL,0x9666ebe66ce57610ULL,0x32ad31f3f73bfe03ULL,0x628330a425bcf4d6ULL,0xea950593515056e6ULL,0x59811c89e1332156ULL,0xc89cf1fe8c11b2d7ULL,+	0x889e5acbc46d7ce1ULL,0x9a515bb78b085877ULL,0xfac1a03d0b7a5050ULL,0x7d3e738af2926035ULL,0x861cc2ce2a6cb0ebULL,0x6f2e29558f7adc79ULL,0x61c4d45133016376ULL,0xd9fd2c805ad59090ULL,+	0x0ad7337ac0b7eff3ULL,0x8552225ec5e48b3cULL,0xe6f78b0c73f13a5fULL,0x5e70062e82349cbeULL,0x6b8d5048e7073969ULL,0x392d2a29c33cb3d2ULL,0xee4f727c4ecaa20fULL,0xa068c99e2ccde707ULL,+	0x1888d65861a023efULL,0x1d72aab4b9e5246eULL,0xa9a26348e5563ec0ULL,0xa0971963c3439a43ULL,0x567dd54badb9b5b7ULL,0x73fac1a1c45a524bULL,0x8fe97ef7fe38e608ULL,0x608748d23f384f48ULL,+	0xebde86ec1ed66f18ULL,0x225d906bd61fce43ULL,0x5cab07d6e8bed74dULL,0x16e4617f27855ab7ULL,0x6568aaddb2fbc3ddULL,0xedb5484f8aeddf5bULL,0x878f20e86dcf2fadULL,0x3516497c615f5699ULL,+	0xef0a3fecfa181e69ULL,0x9ea02f8130d69a98ULL,0xb2e9cf8e66eab95dULL,0x520f2beb24720021ULL,0x621c540a1df84361ULL,0x1203772171fa6d5dULL,0x6e3c7b510ff5f6ffULL,0x817a069babb2bef3ULL,+	0x83572fb6b294cda6ULL,0x6ce9bf75b9039f34ULL,0x20e012f0095cbb21ULL,0xa0aecc1bd063f0daULL,0x57c21c3af02909e5ULL,0xc7d59ecf48ce9cdcULL,0x2732b8448ae336f8ULL,0x056e37233f4f85f4ULL,+	0x8a10b53189e800caULL,0x50fe0c17145208fdULL,0x9e43c0d3b714ba37ULL,0x427d200e34189accULL,0x05dee24fe616e2c0ULL,0x9c25f4c8ee1854c1ULL,0x4d3222a58f342a73ULL,0x0807804fa027c952ULL,+	0xc222653a4f0d56f3ULL,0x961e4047ca28b805ULL,0x2c03f8b04a73434bULL,0x4c966787ab712a19ULL,0xcc196c42864fee42ULL,0xc1be93da5b0ece5cULL,0xa87d9f22c131c159ULL,0x2bb6d593dce45655ULL,+	0x22c49ec9b809b7ceULL,0x8a41486be2c72c2cULL,0x813b9420fea0bf36ULL,0xb3d36ee9a66dac69ULL,0x6fddc08a328cc987ULL,0x0a3bcd2c3a326461ULL,0x7103c49dd810dbbaULL,0xf9d81a284b78a4c4ULL,+	0x3de865ade4d55941ULL,0xdedafa5e30384087ULL,0x6f414abb4ef18b9bULL,0x9ee9ea42faee5268ULL,0x260faa1637a55a4aULL,0xeb19a514015f93b9ULL,0x51d7ebd29e9c3598ULL,0x523fc56d1932178eULL,+	0x501d070cb98fe684ULL,0xd60fbe9a124a1458ULL,0xa45761c892bc6b3fULL,0xf5384858fe6f27cbULL,0x4b0271f7b59e763bULL,0x3d4606a95b5a8e5eULL,0x1eda5d9b05a48292ULL,0xda7731d0e6fec446ULL,+	0xa3e3369390d45871ULL,0xe976404006166d8dULL,0xb5c3368289a90403ULL,0x4bd1798372f1d637ULL,0xa616679ed5d2c53aULL,0x5ec4bcd8fdcf3b87ULL,0xae6d7613b66a694eULL,0x7460fc76e3fc27e5ULL,+	0x70469b8295caabeeULL,0xde024ca5889501e3ULL,0x6bdadc06076ed265ULL,0x0cb1236b5a0ef8b2ULL,0x4065ddbf0972ebf9ULL,0xf1dd387522aca432ULL,0xa88b97cf744aff76ULL,0xd1359afdfe8e3d24ULL,+	0x52a3ba2b91502cf3ULL,0x2c3832a8084db75dULL,0x04a12dddde30b1c9ULL,0x7802eabce31fd60cULL,0x33707327a37fddabULL,0x65d6f2abfaafa973ULL,0x3525c5b811e6f91aULL,0x76aeb0c95f46530bULL,+	0xe8815ff62f93a675ULL,0xa6ec968405f48679ULL,0x6dcbb556358ae884ULL,0x0af61472e19e3873ULL,0x72334372a5f696beULL,0xc65e57ea6f22fb70ULL,0x268da30c946cea90ULL,0x136a8a8765681b2aULL,+	0xad5e81dc0f9f44d4ULL,0xf09a69602c46585aULL,0xd1649164c447d1b1ULL,0x3b4b36c8879dc8b1ULL,0x20d4177b3b6b234cULL,0x096a25051730d9d0ULL,0x0611b9b8ef80531dULL,0xba904b3b64bb495dULL,+	0x1192d9d493a3147aULL,0x9f30a5dc9a565545ULL,0x90b1f9cb6ef07212ULL,0x299585460d87fc13ULL,0xd3323effc17db9baULL,0xcb18548ccb1644a8ULL,0x18a306d44f49ffbcULL,0x28d658f14c2e8684ULL,+	0x44ba60cda99f8c71ULL,0x67b7abdb4bf742ffULL,0x66310f9c914b3f99ULL,0xae430a32f412c161ULL,0x1e6776d388ace52fULL,0x4bc0fa2452d7067dULL,0x03c286aa8f07cd1bULL,0x4cb8f38ca985b2c1ULL,+	0x83ccbe808c3bff36ULL,0x005a0bd25263e575ULL,0x460d7dda259bdcd1ULL,0x4a1c5642fa5cab6bULL,0x2b7bdbb99fe4fc88ULL,0x09418e28cc97bbb5ULL,0xd8274fb4a12321aeULL,0xb137007d5c87b64eULL,+	0x80531fe1c63c4962ULL,0x50541e89981fdb25ULL,0xdc1291a1fd4c2b6bULL,0xc0693a17a6df4fcaULL,0xb2c4604e0117f203ULL,0x245f19630a99b8d0ULL,0xaedc20aac6212c44ULL,0xb1ed4e56520f52a8ULL,+	0xb5560fb6700a1acdULL,0xe823fd73fd999681ULL,0xda915d1f6cb4e1baULL,0x0d0301186ebe00a3ULL,0x744fb0c989fca8cdULL,0x970d01dbf9da0e0bULL,0x0ad8c5647931d76fULL,0xb15737bff659b96aULL,+	0xa12b384ece53c2d0ULL,0x779d897d5e4606daULL,0xa53e47b073ec12b0ULL,0x462dbbba5756f1adULL,0x69fe09f2cafe37b6ULL,0x273d1ebfecce2e17ULL,0x8ac1d5383cf607fdULL,0x8035f7ff12e10c25ULL,+	0xca442d5a2093c22aULL,0xebd0bd31d5703aedULL,0x308f2afd653287b6ULL,0x9bb88bac0d1bc8baULL,0xfbaf853875c1e3b2ULL,0xbd2ac950ca11447cULL,0x286d816cea5c4c8dULL,0xdc3aa80028dc3208ULL,+	0x854d34c77e6c5520ULL,0xc27df9efdcb9ea58ULL,0x405f2369d686666dULL,0x29d1febf0417aa85ULL,0x9846819e93470afeULL,0x3e6a9669e2a27f9eULL,0x24d008a2e31e6504ULL,0xdba7cecf9cb7680aULL,+	0x26a43e41d07fa53dULL,0x3154a78a74e35bc5ULL,0x7b768924e0da2f8cULL,0xba964a2b23613f9aULL,0x5a548d35ba1d16c4ULL,0x2e1bfed1fb54d057ULL,0xff992136bc640205ULL,0xf39cb9148156df29ULL,+	0xc913e64699b444adULL,0xddfce99dc40504c5ULL,0x58482a99d42e53dbULL,0x9aaf2c25d1aff537ULL,0xee90f7962664cf67ULL,0x74ab5c99f1393e2bULL,0xfad0faeae6225bb0ULL,0xc355648c2d63dd6cULL,+	0xe4e31d271d91cf9dULL,0xcb35d4fdb377b20aULL,0x74de1e45055e1327ULL,0x3298e31b28703e75ULL,0x55087237de013339ULL,0x32cbf30123d101c6ULL,0xc70dba22e8aab0dcULL,0x4a52623d3d155bb9ULL,+	0xecaff541338d6e43ULL,0x56f7dd734541d5ccULL,0xb5d426de96bc88caULL,0x48d94f6b9ed3a2c3ULL,0x6354a3bb2ef8279cULL,0xd575465b0b1867f2ULL,0xef99b0ff95225151ULL,0xf3e19d88f94500d8ULL,+	0xdbf435acc85dcf57ULL,0x61745658c88f5415ULL,0x26367e9a17c55807ULL,0x22d077a5ca90c56fULL,0xfbf72258a2e04e76ULL,0xba965d3e6e06e405ULL,0x5724d06fe3e6f954ULL,0x3e47d47581251a74ULL,+	0xb8ba0151e0fb82f7ULL,0x0d160726d5668ac2ULL,0x622ba25814d711b0ULL,0x6addf5577f3fe2f2ULL,0x2b831e1c6b9c9435ULL,0xce3a060ab73826bdULL,0x93fa11c11c240f89ULL,0x4f9cc8d8956e303aULL,+	0x4b9331f6641f82c9ULL,0xd97c7c54dffec756ULL,0xf5ee6d1f1a9158abULL,0x054493a385c3da7fULL,0xa57a05f5eb7d96dfULL,0xa3afd447e4473a39ULL,0x42a4d9c488e16d55ULL,0x83e144f5f5f876aeULL,+	0x92a83268e32dd620ULL,0x913ec99f627849a2ULL,0xedd8fdfa2c378882ULL,0xaf96f33eee6f8cfeULL,0xc06737e5dc3fa8a5ULL,0x236bb531b0b03a1dULL,0x33e59f2989f037b0ULL,0x13f9b5a7d9a12a53ULL,+	0x50d8ae9559029aa6ULL,0xd74e292c5a4db2edULL,0x0b9c3355848f373dULL,0xec018db6ac45ab38ULL,0x1f44690269cc53a8ULL,0x8c4b628d1a879864ULL,0x1c743d284b13475eULL,0xbf4a933873de19f6ULL,+	0x4a8a4f47f0cefa69ULL,0xdc8e4cbaa4546866ULL,0x359ba69b23f603c1ULL,0xdab4d601187b7ac5ULL,0xa6ca4337c1ebc8d9ULL,0x9fa6585452b4074bULL,0x1a4b4f81902fb733ULL,0xd2bb5d7aa525deaaULL,+	0xeb2e92d5f81f9567ULL,0x54cb95ea4d698470ULL,0x5f2acb28e04c81ebULL,0x1c1ebfc4f8ceec64ULL,0x8f799fac06e07423ULL,0x72225f9937fa0c85ULL,0xb0cd861634f4db44ULL,0x5ec36159752c9091ULL,+	0x0d0df6ce51efb310ULL,0xcb5b2eb4958df5beULL,0xd6459e2936158e59ULL,0x82aae2b91466e336ULL,0xfb658a39411aa636ULL,0x7152ecc5d4c0a933ULL,0xf10c758a49f026b7ULL,0xf4837f97cb09311fULL,+	0x994f523a626332d5ULL,0x7bc388335561bb44ULL,0x005ed4b03d845ea2ULL,0xd39d3ee1c2a1f08aULL,0x6561fdd3e7676b0dULL,0x620e35fffb706017ULL,0x36ce424ff264f9a8ULL,0xc4c3419fda2681f7ULL,+	0x00f831769bb81648ULL,0xd69eb485653120d0ULL,0xd17d75f44ccabc62ULL,0x34a07f82b749fcb1ULL,0x2c3af787bbfb5554ULL,0xb06ed4d062e283f8ULL,0x5722889fa19213a0ULL,0x162b085edcf3c7b4ULL,+	0x36d90ddaeb300f7aULL,0x9dcf7dfcedb5e801ULL,0x645cb26874d5244cULL,0xa127ee79348e3aa2ULL,0x488acc53575f1dbbULL,0x95037e8580e6161eULL,0x57e59283292650d0ULL,0xabe67d9914938216ULL,+	0x32670d2f7189e71fULL,0xc64387485ecf91e7ULL,0x15758e57db757a21ULL,0x427d09f8290a9ce5ULL,0x846a308f38384a7aULL,0xaac3acb4b0732b99ULL,0x9e94100917845819ULL,0x95cba111a7ce5e03ULL,+	0xeb81aa377378058eULL,0x41c746a104411154ULL,0xa10c73bcfb828ac7ULL,0x6439be919d972b29ULL,0x4bf3b4b043a2fbadULL,0x39e6dadf82b5e840ULL,0x4f7164086397bd4cULL,0x0f7de5687f1eeccbULL,+	0xdb332a73f37ec3c3ULL,0xc65259bddd59eba0ULL,0x2291709cdb4d3257ULL,0x9a793b25bd389390ULL,0xf39fe34be43756f0ULL,0x2f76bdce9afb56c9ULL,0x9f37867a61208b27ULL,0xea1d4307089972c3ULL,+	0xd0a744878a429f4fULL,0x0649712bdb516609ULL,0xb826ba57e769b5dfULL,0x82335df21fc7aaf2ULL,0x2389f0675c93d995ULL,0x59ac367a68677be6ULL,0xa77985ff21d9951bULL,0x038956fb85011cceULL,+	0x97b7851aaaca5e9bULL,0x518aa52156713b97ULL,0x3357e8c7150a61f6ULL,0x7842e7e2ec2c2b69ULL,0x8dffaf656868a548ULL,0xd963bd82e068fc81ULL,0x64da5c8b65917733ULL,0x927090ff7b247328ULL,+	0xd6ffdb942f6d0d97ULL,0x05c3ee41443b9373ULL,0xb2e541ebffd36db6ULL,0xb7415a96ce1dcc3eULL,0xe383682e163aa2f6ULL,0x46febdd42f3af218ULL,0x90a0507ebafdbadbULL,0x4ca8ab4dce52e21aULL,+	0xa3f0832e1b7cfb73ULL,0x7a8afe523ec354c9ULL,0xae91c97e378eadcaULL,0x7449c599ac3b32bbULL,0xa619c3710b1c4655ULL,0x692e4c6af79da87eULL,0xff3f5d86de38d96aULL,0xc5320f421c08c0ecULL,+	0x92a6f2bc8fec5decULL,0xa71383ff84d6786cULL,0x87588c06dbffa084ULL,0x0d85f5ca6857e715ULL,0xe87311b3b6c774d4ULL,0x672357c84c3521a8ULL,0xe5fe74615b29fe0fULL,0x02bc51105b7158cdULL,+	0x37a01e48a105fc8eULL,0x769d754a289ba48cULL,0xc08c6fe1d51c2180ULL,0xb032dd33b7bd1387ULL,0x953826db020b0aa6ULL,0x05137e800664c73cULL,0xc66302c4660cf95dULL,0x99004e11b2cef28aULL,+	0x824f5b284f973536ULL,0xb43e299ed35b04eaULL,0xc72c88f74da03089ULL,0x8269d57a45a2e42cULL,0x7c1e63fc6607b38eULL,0xe89e2aaf29390b0cULL,0xc7c740da1bee2869ULL,0x8556f6fcaf3fb974ULL,+	0x1a0a3995a4b6bed1ULL,0x2dab579597095c54ULL,0x06c6a1ff2aa73ce9ULL,0xadd0a54b4de438a7ULL,0x160b6b1afca906cdULL,0x25fc601629de10ddULL,0x348e9c99d3633da3ULL,0x1fe3f746158a4d5aULL,+	0xc253edc88be85c1dULL,0xdd3d0e483ca09cb6ULL,0xb997f6879ae3055aULL,0x0c929ad007431dbfULL,0xcef1621584d2db42ULL,0xb50df3ef078828cdULL,0x4589da9d6dbd4b66ULL,0xbc4fd2e3d99c2b04ULL,+	0x214bc9a7d298c241ULL,0xe3b697ba56807cfdULL,0xef1c78024564eadbULL,0xdde8cdcfb48149c5ULL,0x946bf0a75a4d2604ULL,0x27154d7f6c1538afULL,0x95cc9230de5b1fccULL,0xd88519e966864f82ULL,+	0xb828dd1a7cb1282cULL,0xa08d7626be46973aULL,0x6baf8d40e708d6b2ULL,0x72571fa14daeb3f3ULL,0x85b1732ff22dfd98ULL,0x87ab01a70087108dULL,0xaaaafea85988207aULL,0xccc832f869f00755ULL,+	0x488f1185ca8d9d1aULL,0xadf2c77dd987ded2ULL,0x5f3039f060c46124ULL,0xe5d70b7571e095f4ULL,0x82d586506260e70fULL,0x39d75ea7f750d105ULL,0x8cf3d0b175bac364ULL,0xf3a7564d21d01329ULL,+	0x94ab4700ec3128c2ULL,0x6c76d8628e383f49ULL,0xdc36b150c03024ebULL,0xfb43947753daac69ULL,0xfc68764a8dc79623ULL,0x5b86995db440fbb2ULL,0xd66879bfccc5ee0dULL,0x0522894295aa8bd3ULL,+	0xb24aa43e3fcd3efcULL,0xdd26c034b8088e9aULL,0xa5ef4dc9bd3d46eaULL,0xa2f99d588a4c6a6fULL,0xddabd3552f1da46cULL,0x72c3f8ce1afacdd1ULL,0xd90c4eee92d40578ULL,0xd28bb41fca623b94ULL,+	0x5e7c3becee8314f3ULL,0x1c068aeddbea298fULL,0x08d381f17c80acecULL,0x03b56be8e330495bULL,0xaeffb8f29222882dULL,0x95ff38f6c4af8bf7ULL,0x50e32d351fc57d8cULL,0x6635be5217b444f0ULL,+	0x242792d2e7417ce1ULL,0xff42bc71970ee7f5ULL,0x1ff4dc6d5c67a41eULL,0x77709b7b20882a58ULL,0x3554731dbe217f2cULL,0x2af2a8cd5bb72177ULL,0x58eee769591dd059ULL,0xbb2930c94bba6477ULL,+	0x174a9126cecdaa7aULL,0xfc8c7e0e0b13247bULL,0x29c110d23484c1c4ULL,0xf8eb8757831dfc3bULL,0x022f0212c0067452ULL,0x3f6f69ee7b9b926cULL,0x09032da0ef42daf4ULL,0x79f00ade83f80de4ULL,+	0x1e6adddaf176f2c0ULL,0x01ca4604e2572658ULL,0x0a404ded85342ffbULL,0x8cf60f96441838d6ULL,0x9bbc691cc9071c4aULL,0xfd58874434442803ULL,0x97101c85809c0d81ULL,0xa7fb754c8c456f7fULL,+	0xf8559ff4c1e99d81ULL,0x08e1a7d6a3c617c0ULL,0xb398fd43248c6ba7ULL,0x6ffedd91d1283794ULL,0x8a6a59d2d629d208ULL,0xa9d141d53490530eULL,0x42f6fc1838505989ULL,0x09bf250d479d94eeULL,+	0x6af7a1d5af71013fULL,0xe68216e50bedc946ULL,0xf4cba30bd27370a0ULL,0x7981afbf870421ccULL,0x02496a679449f0e1ULL,0x86cfc4be0a47edaeULL,0x3073c936b1feca22ULL,0xf569461203f8f8fbULL,+	0xec14f9e12cb7191eULL,0x78ea1bd8e5b08ea6ULL,0x3c65aa9b46332bb9ULL,0x84cc22b3bf80ce25ULL,0x0098e9e9d49d5bf1ULL,0xcd4ec1c619087da4ULL,0x3c9d07c5aef6e357ULL,0x839a02689f8f64b8ULL,+	0xbcadd6715bde48f8ULL,0xc97038732189bc7dULL,0x5d45299ec709ee8aULL,0xd1287ee2845aaff8ULL,0x7d1f8874db1dbf1fULL,0xea46588b990c88d6ULL,0x60ba649a84368313ULL,0xd5fdcbce60d543aeULL,+	0xf795643037577dd8ULL,0x83b82af429c5fe88ULL,0x9c1bea26cdbdc132ULL,0x589fa0869c04339eULL,0x033e9538b13799dfULL,0x85fa8b21d295d034ULL,0xdf17f73fbd9ddccaULL,0xf32bd122ddb66334ULL,+	0xcf3de9959890272dULL,0x75f3432a3e713a10ULL,0x5e13479fe28227b8ULL,0xb8561ea9fefacdc8ULL,0xa6a297a08332aafdULL,0x9b0d8bb573809b62ULL,0xd2fa1cfd0c63036fULL,0x7a16eb55bd64bda8ULL,+	0x4cc34ec13cf48283ULL,0xb09daa259c8a705eULL,0xd1e9d0d05b7d4f84ULL,0x4df6ef64db38929dULL,0xe16b0763aa21ba46ULL,0xc6b1d178a293f8fbULL,0x0ff5b602d520aabfULL,0x94d671bdc339397aULL,+	0xf7e48e8a2ac13e27ULL,0x4494f6df4eb1a9f5ULL,0xedbf84eb981f0a62ULL,0x49badc32536438f0ULL,0x50bea541004f7571ULL,0xbac67d10df1c94eeULL,0x253d73a1b727bc31ULL,0xb3d01cf230686e28ULL,+	0xd433e50f6d3549cfULL,0x6f33696ffacd665eULL,0x695bfdacce11fcb4ULL,0x810ee252af7c9860ULL,0x65450fe17159bb2cULL,0xf7dfbebe758b357bULL,0x2b057e74d69fea72ULL,0xd485717a92731745ULL,+	0x896c42e8ee36860cULL,0xdaf04dfd4113c22dULL,0x1adbb7b744104213ULL,0xe5fd5fa11fd394eaULL,0x68235d941a4e0551ULL,0x6772cfbe18d10151ULL,0x276071e309984523ULL,0xe4e879de5a56ba98ULL,+	0x6c8d0aa9b898fd52ULL,0x2fb38a57be9af1a7ULL,0xe1f2b9a93b4f03f8ULL,0x2b1aad44c3f0cc6fULL,0x58b5332e7cf2c084ULL,0x1c57d96f0367d26dULL,0x2297eabdfa6e4a8dULL,0x65a947ee4a0e2b6aULL,+	0xaaafafb0285b9491ULL,0x01a0be881e4c705eULL,0xff1d4f5d2ad9caabULL,0x6e349a4ac37a233fULL,0xcf1c12464a1c6a16ULL,0xd99e6b6629383260ULL,0xea3d43665f6d5471ULL,0x36974d04ff8cc89bULL,+	0xf535b616fdd5b854ULL,0x592549c85728719fULL,0xe231468606921cadULL,0x98c8ce34311b1ef8ULL,0x28b937e7e9090b36ULL,0x67fc3ab90bf7bbb7ULL,0x12337097a9d87974ULL,0x3e5adca1f970e3feULL,+	0xc26c49a1cfe89d80ULL,0xb42c026dda9c8371ULL,0xca6c013adad066d2ULL,0xfb8f722856a4f3eeULL,0x08b579ecd850935bULL,0x34c1a74cd631e1b3ULL,0xcb5fe596ac198534ULL,0x39ff21f6e1f24f25ULL,+	0xcdcc68a7b3f85ff0ULL,0xacd21cdd1a888044ULL,0xb6719b2e05dbe894ULL,0xfae1d3d88b8260d4ULL,0xedfedece8a1c5d92ULL,0xbca01a94dc52077eULL,0xc085549c16dd13edULL,0xdc5c3bae495ebaadULL,+	0x27f29e148f929057ULL,0x7a64ae06c0c853dfULL,0x256cd18358e9c5ceULL,0x9d9cce82ded092a5ULL,0xcc6e59796e93b7c7ULL,0xe1e4709231bb9e27ULL,0xb70b3083aa9e29a0ULL,0xbf181a753785e644ULL,+	0xcc17063fbe7b643aULL,0x7872e1c846085760ULL,0x86b0fffbb4214c9eULL,0xb18bbc0e72bf3638ULL,0x8b17de0c722591c9ULL,0x1edeab1948c29e0cULL,0x9fbfd98ef4304f20ULL,0x2d1dbb6b9c77ffb6ULL,+	0xf53f2c658ead09f7ULL,0x1335e1d59780d14dULL,0x69cc20e0cd1b66bcULL,0x9b670a37bbe0bfc8ULL,0xce53dc8128efbeedULL,0x0c74e77c8326a6e5ULL,0x3604e0d2b88e9a63ULL,0xbab38fca13dc2248ULL,+	0x255616d3c7141771ULL,0xa86691ab2f226b66ULL,0xda19fea4b3ca63a9ULL,0xfc05dc42ae672f2bULL,0xa9c6e786718ba28fULL,0x07b7995b9c66b984ULL,0x0f434f551b3702f2ULL,0xd6f6212fda84eeffULL,+	0x8ed6e8c85c0a3f1eULL,0xbcad24927c87c37fULL,0xfdfb62bb9ee3b78dULL,0xeba8e477cbceba46ULL,0x37d38cb0eeaede4bULL,0x0bc498e87976deb6ULL,0xb2944c046b6147fbULL,0x8b123f35f71f9609ULL,+	0x4b0e7987b5b41d78ULL,0xea7df9074bf0c4f8ULL,0xb4d03560fab80ecdULL,0x6cf306f6fb1db7e5ULL,0x0d59fb5689fd4773ULL,0xab254f4000f9be33ULL,0x18a09a9277352da4ULL,0xf81862f5641ea3efULL,+	0xa155dcc7de79dc24ULL,0xf1168a32558f69cdULL,0xbac215950d1850dfULL,0x15c8295bb204c848ULL,0xf661aa367d8184ffULL,0xc396228e30447bdbULL,0x11cd5143bde4a59eULL,0xe3a26e3b6beab5e6ULL,+	0xb59b01579f759d01ULL,0xa2923d2f7eae4fdeULL,0x18327757690ba8c0ULL,0x4bf7e38b44f51443ULL,0xb6812563b413fc26ULL,0xedb7d36379e53b36ULL,0x4fa585c4c389f66dULL,0x8e1adc3154bd3416ULL,+	0xd3b3a13f1402b9d0ULL,0x573441c32c7bc863ULL,0x4b301ec4578c3e6eULL,0xc26fc9c40adaf57eULL,0x96e71bfd7493cea3ULL,0xd05d4b3f1af81456ULL,0xdaca2a8a6a8c608fULL,0x53ef07f60725b276ULL,+	0xa6b5c9d646ac49d2ULL,0x42c77c0b83137aa9ULL,0x24d000fc68225a38ULL,0x0f63cfc82fe1e907ULL,0x22d1b01bc6441f95ULL,0x7d38f719ec8e448fULL,0x9b33fa5f787fb1baULL,0x94dcfda1190158dfULL,+	0x211cde10296c36efULL,0x7ee8967282c4da77ULL,0xb617d270a57836daULL,0xf0cd9c319cb7560bULL,0x01fdcbf7e455fe90ULL,0x3fb53cbb7e7334f3ULL,0x781e2ea44e7de4ecULL,0x8adab3ad0b384fd0ULL,+	0x01778a2b599ff0f9ULL,0x68a923d78104fc6bULL,0x5bfa44dfda694ff3ULL,0x4f7199dbf7667f12ULL,0xc06d8ff6e46f2a79ULL,0x08b5deade9f8131dULL,0x02519a59abb4ce7cULL,0xc4f710bcb42aec3eULL,+	0x3014368b4ed80940ULL,0x67e6d0567a6fceddULL,0x7c208c49ca97579fULL,0xfe3d7a81a23597f6ULL,0x5e2032027e096ae2ULL,0xb1f3e1e724b39366ULL,0x26da26f32fdcdffcULL,0x79422f1d6097be83ULL,+	0x50549c748c878145ULL,0x67f14edf39c63565ULL,0x22ddf78c9bcf2d5eULL,0xffaa842f68201d10ULL,0x47d94a9dd1b2de28ULL,0xc09c4be8054be414ULL,0xac80e178ca82755bULL,0xe3251d105697c3bdULL,+	0x2bbe09d35001417bULL,0x795e84ee5962ed5eULL,0x5b79d1ca279f46c3ULL,0x1f7f8a3b83836a2eULL,0x692200b14a64dc32ULL,0xc84243350f84f739ULL,0xf110da07cc9155c0ULL,0xee8fbe61594b0507ULL,+	0x2f6a5391035703ccULL,0x9899bf6a40c7e24dULL,0xc3f7f248bbfcbb9aULL,0xf65027ded555875bULL,0xa7a16b69ffff3b37ULL,0x67b6eb54145b4431ULL,0x19d7e1d249afd679ULL,0xbd819bab110fccdfULL,+	0x263a2cfb9db3b381ULL,0x9c3a2deed4df0a4bULL,0x728d06e97d04e61fULL,0x8b1adfbc42449325ULL,0x6ec1d9397e053a1bULL,0xee2be5c766daf707ULL,0x80ba1e14810ac7abULL,0xdd2ae778f530f174ULL,+	0x3e708e703b9f0426ULL,0xe5b02fb60c84f17cULL,0x2f4ff35be3b70a0bULL,0x781b3c5f9b15565dULL,0xe76c636a6e124c3aULL,0xbde81eba8b496784ULL,0xa412f8e2443f0370ULL,0x15d42362999be45dULL,+	0x0f503ae2a4af76c1ULL,0x550e66dc08276fe7ULL,0x11e0c1fcbf3a33c6ULL,0x42be231006629f85ULL,0xedf7743e516ace49ULL,0xce436668436a2262ULL,0xe1ac7036446ca192ULL,0x73631cb5476fe13eULL,+	0xd47f82d4a160bcfaULL,0x258fb075b8ceb1f2ULL,0x4f818e8fdf5a8d25ULL,0x6685475e6fd31c9dULL,0xcef6385ae1e9b13fULL,0xe0a42594f0508bdbULL,0x5ad7ae16aef1f90dULL,0x45a155ef63f8a81eULL,+	0x8ca407c28034b95eULL,0xc93eb97617bdc560ULL,0x4ec24e8d339807e8ULL,0x91b734d6dd64a4ebULL,0xd0fece398f668b26ULL,0x4822cc4b141823d5ULL,0xb953bc32f09e4e00ULL,0xca0a7c6006d861aeULL,+	0x4c74d4470f33e712ULL,0x3cec1e0625a87cb0ULL,0x5cec0610e5962db4ULL,0xd971af1571a256aaULL,0xa044c983a2ef4ac9ULL,0xcaa1da63d74e9d00ULL,0xfb972d834673d881ULL,0x50747c5a03a26c8bULL,+	0x04349982a3e25566ULL,0xeef9075e18e1b896ULL,0x4c7bead092b2d24bULL,0xd99f72fb0a21ba55ULL,0xb93e09315005e541ULL,0x2a7a98389ece3205ULL,0xeb388ed11462f2f6ULL,0xb488b15a2e3460a6ULL,+	0x43f6cd67969d56afULL,0x9e0d872cdfc58a8bULL,0x401c1509a4e70377ULL,0x103d1a1308ad646cULL,0x078ee37e9d062427ULL,0x4e69c5acb9bef78cULL,0x521ec00136e66142ULL,0x8de1ecb2a634cd82ULL,+	0x0435d97a205b9d8bULL,0x6eb8f064056756d4ULL,0xd5e88a8bb6f8210eULL,0x070ef12dec9fd9eaULL,0x4d8495053bcc876aULL,0x12a75338a7404ce3ULL,0xd22b49e1b8a1db5eULL,0xec1f205114bfa5adULL,+	0x43ed81b5c4e83d33ULL,0xd9f358795efd488bULL,0x164a620f9deb4d0fULL,0xc6927bdbac6a7394ULL,0x45c28df79f9e0f03ULL,0x2868661efcd7e1a9ULL,0x7cf4e8d0ffa348f1ULL,0x6bd4c284398538e0ULL,+	0x56036e8c06d75fc1ULL,0x2dcf7bb73249a89fULL,0x81dd1d3de245e7ddULL,0xf578dc4bebd6e2a7ULL,0x4c028903df2ce7a0ULL,0xaee362889c39afacULL,0xdc847c31146404abULL,0x6304c0d8a4e97818ULL,+	0xfb6836c327d02dccULL,0x5ad009827a68bcc2ULL,0x1b24b44c005e912dULL,0xcc83d20f811fdcfeULL,0x36527ec1666fba0cULL,0x6994819714754635ULL,0xfcdcb1a8556da9c2ULL,0xa593426781a732b2ULL,+	0xe4ac8b33070d3aabULL,0x2643672b9a2cd5e5ULL,0x52eff79b1cfc9173ULL,0x665ca49b90a7c13fULL,0x5a8dda59b3efb998ULL,0x8a5b922d052f1341ULL,0xae9ebbab3cf9a530ULL,0x35986e7bf56da4d7ULL,+	0x63ee4cbd8088b454ULL,0xdb7f32f79a9e0c8aULL,0xb377d4186b2447cbULL,0xe3e982aad370219bULL,0x06ccc1e4c2a2a593ULL,0x72c368650773f24fULL,0xa13b4da795859423ULL,0x8bbf1d3375040c8fULL,+	0x03c187d0ad886aacULL,0x5c16878ab771b645ULL,0xb07dfc6fc74045abULL,0x2c6360bf7800caedULL,0x24295bb5b9c972a3ULL,0xc9e6f88e7c9a6dbaULL,0x90ffbf2492a79aa6ULL,0xde29d50a41c26ac2ULL,+	0xb1f0fb68d84d835dULL,0xc90caf39861dc1e6ULL,0x12e5b0467594f8d7ULL,0x26897ae265012b92ULL,0xbcf68a08a4d6755dULL,0x403ee41c0991fbdaULL,0x733e343e3bbf17e8ULL,0xd2c7980d679b3d65ULL,+	0x534acf4fda79e5acULL,0x68b83b3a8630215fULL,0x5c748b2ed085756eULL,0xb0317258e5d37cb2ULL,0x6735841ac5ccc2c4ULL,0x7d7dc96b3d9d5069ULL,0xa147e410fd1754bdULL,0x65296e94d399ddd5ULL,+	0x1e71c9a1deb8568bULL,0xa35daea080fb3d32ULL,0xe8b6f2662cf8fb81ULL,0x6d51afe89490696aULL,0x81beac6e51803a19ULL,0xe3d24b7f86219080ULL,0x727cfd9ddf6f463cULL,0x8c6865ca72284ee8ULL,+	0xe00df169d23233f3ULL,0x3e32279677cb637fULL,0x1f897c0e1da0cf6cULL,0xa651f5d831d6bbddULL,0xdd61af191a230c76ULL,0xbd527272cdaa5e4aULL,0xca753636d0abcd7eULL,0x78bdd37c370bd8dcULL,+	0xcddb27c17078c432ULL,0xe1961b9cb77fedb7ULL,0x1edc2f5cc2290570ULL,0x2c3fefca19cbd886ULL,0xcf880a36c2af389aULL,0x96c610fdbda71ceaULL,0xf03977a932aa8463ULL,0x8eb7763f8586d90aULL,+	0x831ab3edf0290a8fULL,0xcae81966cb47c387ULL,0xaad7dece184efb4fULL,0xdcfc53b34749110eULL,0x6698f23c4cb632f9ULL,0xc42a1ad6b91f8067ULL,0xb116a81d6284180aULL,0xebedf5f8e901326fULL,+	0x91633f0ab2cf8940ULL,0x72b0b1786f948f51ULL,0x2d28dc30782653c8ULL,0x88829849db903a05ULL,0xb8095d0c6a19d2bbULL,0x4b9e7f0c86f782cbULL,0x7af739882d907064ULL,0xd12be0fe8b32643cULL,+	0x9561f28b638a7e81ULL,0x54155cdf5980ddc3ULL,0xb2db4a96d26f247aULL,0x9d774e4e4787d100ULL,0x1a9e6e2e078637d2ULL,0x1c363e2d5e0ae06aULL,0x7493483ee9cfa354ULL,0x76843cb37f74b98dULL,+	0x0491f1bc789a283bULL,0x72d3ac3d880836f4ULL,0xaa1c5ea388e5402dULL,0x1b192421d5cc473dULL,0x5c0b99989dc84cacULL,0xb0a8482d9c6e75b8ULL,0x639961d03a191ce2ULL,0xda3bc8656d837930ULL,+	0xd7e0c4cdb30cfb3aULL,0x6d09b8c16c9db4c8ULL,0x40ba1a4207c8d9dfULL,0x6fd495f71c52c66dULL,0xfb0e169f275264daULL,0x80c2b746e57d8362ULL,0xedd987f749ad7222ULL,0xfdc229af4398ec7bULL,+	0xb0d1ed8452666a58ULL,0x4bcb6e00e6a9c3c2ULL,0x3c57411c26906408ULL,0xcfc2075513556400ULL,0xa08b1c505294dba3ULL,0xa30ba2868b7dd31eULL,0xd70ba90e991eca74ULL,0x094e142ce762c2b9ULL,+	0xb81d783e979f3925ULL,0x1efd130aaf4c89a7ULL,0x525c2144fd1bf7faULL,0x4b2969041b265a9eULL,0xed8e9634b9db65b6ULL,0x35c82e3203599d8aULL,0xdaa7a54f403563f3ULL,0x9df088ad022c38abULL,+	0xe5cfb066bb3fd30aULL,0x429169daeff0354eULL,0x809cf8523524e36cULL,0x136f4fb30155be1dULL,0x4826af011fbba712ULL,0x6ef0f0b4506ba1a1ULL,0xd9928b3177aea73eULL,0xe2bf6af25eaa244eULL,+	0x8d084f124237b64bULL,0x688ebe99e3ecfd07ULL,0x57b8a70cf6845dd8ULL,0x808fc59c5da4a325ULL,0xa9032b2ba3585862ULL,0xb66825d5edf29386ULL,0xb5a5a8db431ec29bULL,0xbb143a983a1e8dc8ULL,+	0x35ee94ce12ae381bULL,0x3a7f176c86ccda90ULL,0xc63a657e4606eacaULL,0x9ae5a38043cd04dfULL,0x9bec8d15ed251b46ULL,0x1f5d6d30caca5e64ULL,0x347b3b359ff20f07ULL,0x4d65f034f7e4b286ULL,+	0x9e93ba24f111661eULL,0xedced484b105eb04ULL,0x96dc9ba1f424b578ULL,0xbf8f66b7e83e9069ULL,0x872d4df4d7ed8216ULL,0xbf07f3778e2cbecfULL,0x4281d89998e73754ULL,0xfec85fbb8aab8708ULL,+	0x9a3c0deea5ba5b0bULL,0xe6a116ce42d05299ULL,0xae9775fee9b02d42ULL,0x72b05200a1545cb6ULL,0xbc506f7d31a3b4eaULL,0xe58930788bbd9b32ULL,0xc8bc5f37e4b12a97ULL,0x6b000c064a73b671ULL,+	0x13b5bf22765fa7d0ULL,0x59805bf01d6a5370ULL,0x67a5e29d4280db98ULL,0x4f53916f776b1ce3ULL,0x714ff61f33ddf626ULL,0x4206238ea085d103ULL,0x1c50d4b7e5809ee3ULL,0x999f450d85f8eb1dULL,+	0x658a6051e4c79e9bULL,0x1394cb73c66a9feaULL,0x27f31ed5c6be7b23ULL,0xf4c88f365aa6f8feULL,0x0fb0721f4aaa499eULL,0x68b3a7d5e3fb2a6bULL,0xa788097d3a92851dULL,0x060e7f8ae96f4913ULL,+	0x82eebe731a3a93bcULL,0x42bbf465a21adc1aULL,0xc10b6fa4ef030efdULL,0x247aa4c787b097bbULL,0x8b8dc632f60c77daULL,0x6ffbc26ac223523eULL,0xa4f6ff11344579cfULL,0x5825653c980250f6ULL,+	0xb2dd097ebc1aa2b9ULL,0x0788939337a0333aULL,0x1cf55e7137a0db38ULL,0x2648487f792c1613ULL,0xdad013363fcef261ULL,0x6239c81d0eabf129ULL,0x8ee761de9d276be2ULL,0x406a7a341eda6ad3ULL,+	0x4bf367ba4a493b31ULL,0x54f20a529bf7f026ULL,0xb696e0629795914bULL,0xcddab96d8bf236acULL,0x4ff2c70aed25ea13ULL,0xfa1d09eb81cbbbe7ULL,0x88fc8c87468544c5ULL,0x847a670d696b3317ULL,+	0xf133421e64bcb626ULL,0xaea638c826dee0b5ULL,0xd6e7680bb310346cULL,0xe06f4097d5d4ced3ULL,0x099614527512a30bULL,0xf3d867fde589a59aULL,0x2e73254f52d0c180ULL,0x9063d8a3333c74acULL,+	0xeda6c595d314e7bcULL,0x2ee7464b467899edULL,0x1cef423c0a1ed5d3ULL,0x217e76ea69cc7613ULL,0x27ccce1fe7cda917ULL,0x12d8016b8a893f16ULL,0xbcd6de849fc74f6bULL,0xfa5817e2f3144e61ULL,+	0x1f3541640821ee4cULL,0x1583eab40bc61992ULL,0x7490caf61d72879fULL,0x998ad9f3f76ae7b2ULL,0x1e181950a41157f7ULL,0xa9d7e1e6e8da3a7eULL,0x963784eb8426b95fULL,0x0ee4ed6e542e2a10ULL,+	0xb79d4cc5ac751e7bULL,0x93f96472fd4211bdULL,0x8c72d3d2c8de4fc6ULL,0x7b69cbf5df44f064ULL,0x3da90ca2f4bf94e1ULL,0x1a5325f8f12894e2ULL,0x0a437f6c7917d60bULL,0x9be7048696c9cb5dULL,+	0x949c9976e1337c26ULL,0x6faadebdd73d68e5ULL,0x9e158614f1b768d9ULL,0x22dfa5579cc4f069ULL,0xccd6da17be93c6d6ULL,0x24866c61a504f5b9ULL,0x2121353c8d694da1ULL,0x1c6ca5800140b8c6ULL,+	0xf1604a7dd4b79bb8ULL,0xaee806fb52c878c8ULL,0x34144f118d47b8e8ULL,0x72edf52b949f9054ULL,0xebfca84e2127015aULL,0x9051d0c09cb7cef3ULL,0x86e8fe58296deec8ULL,0x33b2818841010d74ULL,+	0xbd5660ed9aed9f40ULL,0x70ca6ad1532a8c99ULL,0xc4978bfb95c371eaULL,0xe5464d0d7003109dULL,0x1af32fdfd9e535efULL,0xabf57ea798c9185bULL,0xed7a741712b42488ULL,0x8e0296a7e97286faULL,+	0x01079383171b445fULL,0x9bcf21e38131ad4cULL,0x8cdfe205c93987e8ULL,0xe63f4152c92e8c8fULL,0x729462a930add43dULL,0x62ebb143c980f05aULL,0x4f3954e53b06e968ULL,0xfe1d75ad242cf6b1ULL,+	0x8b57416e1f017d5eULL,0x375333967674e99bULL,0x6e6d94c0e8f488a0ULL,0xb93a787adc16f95eULL,0xc3ac51a2dcc99cccULL,0xc134b4139aa47c1dULL,0xf28fcdafafdfd8d5ULL,0x0d57bd8e10b831edULL,+	0x9276fbccf0bcfc46ULL,0x3a822aceb5cffee6ULL,0x328ed2fec75d915bULL,0xa145c113c359476cULL,0xf61a81538be17bcdULL,0x01e867c3aa6c3d8fULL,0x5634e15d6516c82fULL,0xc1437bd26948b9b0ULL,+	0xd2fcd2006c19d4c7ULL,0xa0f3c437e1b1e976ULL,0xf0545ff694f237e8ULL,0xdd10ec3fc0bf8bb1ULL,0x4f89696cac7cd3e1ULL,0xed3714ec5f24bfe6ULL,0x363eb1d85faf7706ULL,0xfcbd604dc027cc32ULL,+	0x5f95c6c7af8685c8ULL,0xd4c1c8ce2f8f01aaULL,0xc44bbe322574692aULL,0xb8003478d4a4a068ULL,0x7c8fc6e52eca3cdbULL,0xea1db16bec04d399ULL,0xb05bc82e8f2bc5cfULL,0x763d517ff44793d2ULL,+	0x16ce8eddc355363bULL,0x4af2f70ff8820d6eULL,0xcb7ed4d27661a508ULL,0x41d3444edd195472ULL,0x17fea2b438da9649ULL,0x9bf69356aeb4a200ULL,0xa13b5f916ab19c3dULL,0xc0519c14dc9360a6ULL,+	0xc2571ae92e42e171ULL,0xcb31ab63ed41ccf9ULL,0x37f3c576b5c8854fULL,0x66e5191bc62392a1ULL,0x71565a1c6cd5683bULL,0x484b0283606fe689ULL,0xf3a25d6767e2fda6ULL,0x87ba21de8a65c0a4ULL,+	0xde74e49ca70684d1ULL,0x3ae8766133e80c3dULL,0x5984a2a916a5c34dULL,0x09a83eccb8298c35ULL,0x9a19867caa4ca4c0ULL,0x02085610b375b8ffULL,0xf296328bf70396dcULL,0x9c9ddc4cde6fae63ULL,+	0x4451c1b808bd98d0ULL,0x644b1cd46575f240ULL,0x6907eb337375d270ULL,0x56c8bebdfa2286bdULL,0xc713d2acc4632b46ULL,0x17da427aafd60242ULL,0x313065b7c95c7546ULL,0xf8239898bf17a3deULL,+	0x94683d260b083b6eULL,0x0a3752eb06f6a54dULL,0x48bedc23752074ddULL,0x637622fc3e822593ULL,0xea0005136be55d3bULL,0x9f5e12f4324d006dULL,0x529486a964fc0270ULL,0x09ba0d0c923399e6ULL,+	0x363858473a977080ULL,0x4cf8e1b80c6a6ab6ULL,0x919a5c6c0482261eULL,0x517a9ad0e5ce4806ULL,0x2792d40c056aa7aaULL,0x4c7c6adae56c61b0ULL,0xf19cb178a4b19e0cULL,0x046d5c4fe4ba267fULL,+	0xd3e926ab121550b3ULL,0xe4975e4ac147ce84ULL,0x7a8be0f95eff722aULL,0x71e4702c6fd4f2a0ULL,0x13b92acf3cb7b280ULL,0xc588716d28272d73ULL,0x862c7bf3daa9fe5cULL,0x78c008f2e2a79e42ULL,+	0xf3b7963f4c830320ULL,0x842c7aa0903203e3ULL,0xaf22ca0ae7327afbULL,0x38e13092967609b6ULL,0x73b8fb62757558f1ULL,0x3cc3e831f7eca8c1ULL,0xe4174474f6331627ULL,0xa77989cac3c40234ULL,+	0xae8317f4b0166f7aULL,0xfbd3e3f7ceec74e6ULL,0xfdb516ace0874bfdULL,0x3d846019c681f3a3ULL,0x0b12ee5c7c1620b0ULL,0xba68b4dd2b63c501ULL,0xac03cd326668c51eULL,0x2a6279f74e0bcb5bULL,+	0xfd8e139f8f5fcda8ULL,0xf3e558c4bdee5bfdULL,0xd76cbaf4e33f9f77ULL,0x3a4c97a471771969ULL,0xda27e84bf6dce6a7ULL,0xff373d9613e6c2d1ULL,0xf115193cd759a6e9ULL,0x3f9b702563d2262cULL,+	0x12536fea87baa627ULL,0x58c1fec1f72aa680ULL,0x6c29b637601e5dc9ULL,0x9e3c3c1cde9e01b9ULL,0xefc8127b2bcfe0b0ULL,0x351071022a12f50dULL,0x6ccd6cb14879b397ULL,0xf792f804f8a82f21ULL,+	0x8c3184911a335cc8ULL,0x563459ba6a5913e4ULL,0x1b920d61c7b32919ULL,0x805ab8b6a02425adULL,0x2ac512da8d006086ULL,0x6ca4846abcf5c0fdULL,0xafea51d8ac2138d7ULL,0xcb647545344cd443ULL,+	0xa3f4f521e447f2c4ULL,0x81b8da7a604291f0ULL,0xd680bc467d5926deULL,0x84f21fd534a1202fULL,0x1d1e31814e9df3d8ULL,0x1ca4861a39ab8d34ULL,0x809ddeec5b19aa4aULL,0x59f72f7e4d329366ULL,+	0x9f1b2466cdedca85ULL,0x140bb7101a09538cULL,0xac8ae8515e11115dULL,0x0d63ff676f03f59eULL,0x755e55517d234afbULL,0x61c2db4e7e208fc1ULL,0xaa9859cef28a4b5dULL,0xbdd6d4fc34af030fULL,+	0x3f39e67f906151e5ULL,0xcea27f5f55e10649ULL,0xdca1d4e1c17cf7b7ULL,0x0c326d122fe2362dULL,0x05f7ac337dd35df3ULL,0x0c3b7639c396dbdfULL,0x0912f5ac03b7db1cULL,0x9dea4b705c9ed4a9ULL,+	0x511053e453544774ULL,0x834d0ecc3adba2bcULL,0x4215d7f7bae371f5ULL,0xfcfd57bf6c8663bcULL,0xded2383dd6901b1dULL,0x3b49fbb4b5587dc3ULL,0xfd44a08d07625f62ULL,0x3ee4d65b9de9b762ULL,+	0x3e56fe5bdf53aad0ULL,0x51314de5e4604a67ULL,0x386ad98607a261a0ULL,0x8b7e021217afcc91ULL,0xcbf411273b72aec5ULL,0x13c85d05c4f9f509ULL,0xeda56845b6484b57ULL,0x13cb1642b3d0995bULL,+	0x5a994b6e717815deULL,0xd995c7a0a7e131d1ULL,0xc8b46df226c023aaULL,0x8cfd094d702afcedULL,0x7bc743cdced6a886ULL,0xb7d70ec41fcabe75ULL,0x2a6c9e47ddac9390ULL,0x720694259310aa90ULL,+	0xa607b3263e8f793cULL,0xa541166ecde3b289ULL,0xf44ff2924a915b21ULL,0x68bea906b58ecda6ULL,0xd85b37b440292897ULL,0xd2a508ae4b768423ULL,0xd10bb79da413bbbeULL,0x0262f481061491f4ULL,+	0xd17e80f55464d0ebULL,0x89d3e1a767a613a7ULL,0x77791260c8c97dadULL,0xec2ff21fe4f0cbe7ULL,0xed984ac2e9e6bc10ULL,0xe3c53de877cba305ULL,0x9bbaf9b283624fdcULL,0x5e9451cd1485c0ecULL,+	0xfdfc9f63b7abad11ULL,0xc7ec3a9263a46189ULL,0x49ebee42f67037b7ULL,0x8247f504cdac1710ULL,0xfc518f8d397583e5ULL,0xe7d24de70c3f8c2eULL,0x354832669c5edcb1ULL,0x94bba483f8c2cefcULL,+	0x1f13756db1761ec8ULL,0xe53c8b98a4b97e55ULL,0xb2aee3f84096cc28ULL,0x48c361a0920f1a8dULL,0xa98b672d8c31190aULL,0x7bc1e7d1001855d4ULL,0x242cfb07bf3f4b2aULL,0x9bf44a3f32a28bc4ULL,+	0xeba8976299da550cULL,0xb2c1781a16baa042ULL,0x3068b082788c2f9dULL,0xc0fa414594869a9eULL,0x73bd9e39d50b693fULL,0xb79e2a9c988e2c5eULL,0xf1cb8de40f8f9f62ULL,0x415b04ee1ea50c7bULL,+	0x64e5137d0d63d1faULL,0x658fc05202a9d89fULL,0x4889487450436309ULL,0xe9ae30f8d598da61ULL,0x2ed710d1818baf91ULL,0xe27e9e068b6a0c20ULL,0x1e28dcfb1c1a6b44ULL,0x883acb64d6ac57dcULL,+	0x8735728dc2c6ff70ULL,0x79d6122fc5dc2235ULL,0x23f5d00319e277f9ULL,0x7ee84e25dded8cc7ULL,0x91a8afb063cd880aULL,0x3f3ea7c63574af60ULL,0x0cfcdc8402de7f42ULL,0x62d0792fb31aa152ULL,+	0xb02c83f9dec31a21ULL,0x988c8b236ad9d573ULL,0x53e983aea57be365ULL,0xe968734d646f834eULL,0x9137ea8f5da6309bULL,0x10f3a624c1f1ce16ULL,0x782a9ea2ca440921ULL,0xdf94739e5b46f1b5ULL,+	0x1df165a434a35ea8ULL,0x3418e0f74d4412f6ULL,0x5af1f8af518836c3ULL,0x42ceef4d130e1965ULL,0x5560ca0b543a1957ULL,0xc33761e5886cb123ULL,0x66624b1ffe98ed30ULL,0xf772f4bf1090997dULL,+	0x56f8410ef4f8b16aULL,0x97241afec47b266aULL,0x0a406b8e6d9c87c1ULL,0x803f3e02cd42ab1bULL,0x7f0309a804dbec69ULL,0xa83b85f73bbad05fULL,0xc6097273ad8e197fULL,0xc097440e5067adc1ULL,+	0xc507b6dd418e7dddULL,0x39888d93472f19d6ULL,0x7eae26be0c27eb4dULL,0x17b53ed3fbabb884ULL,0xfc27021b2b01ae4fULL,0x88462e87cf488682ULL,0xbee096ec215e2d87ULL,0xeb2fea9ad242e29bULL,+	0xbbcc00c756b95bceULL,0x5ec03906616da680ULL,0x79162ee672214252ULL,0x43132b6386a892d2ULL,0x4bdd3ff22f3263bfULL,0xd5b3733c9cd0a142ULL,0x592eaa8244415ccbULL,0x663e89248d5474eaULL,+	0x0d38ab35ce7c42d4ULL,0x9fd493ef82feab10ULL,0x46056b6d82111b45ULL,0xda11dae173efc5c3ULL,0xdc7402785545a7fbULL,0xbdb2601c40d507e6ULL,0x121dfeeb7066fa58ULL,0x214369a839ae8c2aULL,+	0x3f747fa0b311898cULL,0xe2a272e4cd0eac65ULL,0x4bba5851f914d0bcULL,0x7a1a9660c4a43ee3ULL,0xe5a367cea1c8cde9ULL,0x9d958ba97271abe3ULL,0xf3ff7eb63d1615cdULL,0xa2280dcef5ae20b0ULL,+	0x8c0ed566d4312483ULL,0x5179a95d643e216fULL,0xcc185fec17044493ULL,0xb306333954991a21ULL,0xd801ecdb0081a726ULL,0x0149b0c64fa89bbbULL,0xafe9065a4391b6b9ULL,0xedc92786d633f3a3ULL,+	0xd6d9d9e37a6a308bULL,0x623758304c2767d3ULL,0x874a8bc6f38cbeb6ULL,0xd94d3f1accb6fd9eULL,0x92a9735bba21f248ULL,0x272ad0e56cd1efb0ULL,0x7437b69c05b03284ULL,0xe7f047026948c225ULL,+	0x9ae868a9e9adfe1cULL,0x3984403d314e39bbULL,0xb5875720f2fe378fULL,0x33f901e0ba44a628ULL,0xea1125fe3652438cULL,0xae9ec4e69dd1f20bULL,0x1e740d9ebebf7fbdULL,0x6dbd3ddc42dbe79cULL,+	0x266344a43794f8dcULL,0xdcca923a483c5c36ULL,0x2d6b6bbf3f9d10a0ULL,0xb320c5ca81d9bdf3ULL,0x620e28ff47b50a95ULL,0x933e3b01cef03371ULL,0xf081bf8599100153ULL,0x183be9a0c3a8c8d6ULL,+	0x8a9443d77613aa81ULL,0x8010080085fe6584ULL,0x70fc4dbc7fb10288ULL,0xf58280d3e86beee8ULL,0x14fdd82f7c978c38ULL,0xdf1204c10de44d7bULL,0xa08a1c844160252fULL,0x591554cac17646a5ULL,+	0x7ddc81ea77b46a08ULL,0xcf5a6477c7480699ULL,0x43a8cb346633f683ULL,0x1b867e6b92363c60ULL,0x439211141f60558eULL,0xcdbcdd632f41450eULL,0x7fc04601cc630e8bULL,0xea7c66d597038b43ULL,+	0x34fc8820fbeee3f9ULL,0x93e5349049091afdULL,0x764b9be59a31f35cULL,0x71f3786457e3d924ULL,0x02fb34e0943aa75eULL,0xa18c9c58ab8ff6e4ULL,0x080f31b133cf0d19ULL,0x5c9682db083518a7ULL,+	0xb6c185c341dca566ULL,0x7de7fedad8622aa3ULL,0x99e84d92901b6dfbULL,0x30a02b0e7c4ad288ULL,0xc7c81daa2fd3cf36ULL,0xd1319547df89e59fULL,0xb2be8184cd496733ULL,0xd5f449eb93d3412bULL,+	0xc492ec644cd8f64cULL,0x58a2d790279d7b51ULL,0x0ced1fc51fc75256ULL,0x3e658aed8f433017ULL,0x0b61942e05da59ebULL,0xba3d60a30ddc3722ULL,0x7c311cd1742e7f87ULL,0x6473ffeef6b01b6eULL,+	0x8303604f692ac542ULL,0xf079ffe1227b91d3ULL,0x19f63e6315aaf9bdULL,0xf99ee565f1f344fbULL,0x8a1d661fd6219199ULL,0x8c883bc6d48ce41cULL,0x1065118f3c74d904ULL,0x713889ee0faf8b1bULL,+	0xc035f697960eb8c7ULL,0xf1599f2ce2de04d3ULL,0x892450f8d2ad9228ULL,0x7d48129bb829c1abULL,0x24d785e13a50afc9ULL,0x2745ba2763a96ee0ULL,0x956534013bfb6d7bULL,0x536202671bad2a42ULL,+	0x972b3f8f81a1b3beULL,0x4f3ce145ce2764a0ULL,0xe2d0f1cc28c4f5f7ULL,0xdeee0c0dc7f3985bULL,0x7df4adc0d39e25c3ULL,0x40619820c467a080ULL,0x440ebc9361cf5a58ULL,0x527729a6422ad600ULL,+	0xa691398a4a9eb3f0ULL,0x56c1dbff3b99a48fULL,0x9a87e1b91b4b5b32ULL,0xad6396145378b5feULL,0x437a243ec26b5302ULL,0x0275878c3ccb4c10ULL,0x0e81e4a21de07015ULL,0x0c6265c9850df3c0ULL,+	0xca6c0937b1b76ba6ULL,0x1a2eab854d2026dcULL,0xb1715e1519d9ae0aULL,0xf1ad9199bac4a026ULL,0x35b3dfb807ea7b0eULL,0xedf5496f3ed9eb89ULL,0x8932e5ff2d6d08abULL,0xf314874e25bd2731ULL,+	0xc8327149a8c25ff6ULL,0x29bf2556782e6569ULL,0x9012f5c6cd68fc38ULL,0x3e67e8bd3b982ad5ULL,0x5e3a75386ecdca88ULL,0xf297eaa6c1753a04ULL,0x10121e5405db3256ULL,0xab9697d4f0851055ULL,+	0xefb26a753f73f449ULL,0x1d1c94f88d44fc79ULL,0x49f0fbc53bc0dc4dULL,0xb747ea0b3698a0d0ULL,0x5218c3fe228d291eULL,0x35b804b543c129d6ULL,0xfac859b8d1acc516ULL,0x6c10697d95d6e668ULL,+	0xe5d27171f6bdf1bfULL,0x0b77b876facb0d8fULL,0xda95471d8496a31bULL,0x46a50dbb3f16b103ULL,0x2a4f3f977b865bffULL,0x848195e66b1c198cULL,0x491ad08821702ea6ULL,0x3f20b43749035228ULL,+	0xc38e438f0876fd4eULL,0x45f0c30783d2f383ULL,0x203cc2ecb10934cbULL,0x6a8f24392c9d46eeULL,0xf16b431b65ccde7bULL,0x41e2cd1827e76a6fULL,0xb9c8cf8f4e3484d7ULL,0x64426efd8315244aULL,+	0xe6ec98093a69fc01ULL,0x7e20fecbfaa9dfc2ULL,0x5cfdbb07f56f2a55ULL,0xb1cd68680bbdbfdfULL,0x247b4995986eb9edULL,0x74785bf53dd0955eULL,0x88f74f61c0c7a201ULL,0x8861a15b5d01a80dULL,+	0x1c0a8e44fc94dea3ULL,0x34c8cdbfdad6a0b0ULL,0x919c384004113cefULL,0xfd32fba415490ffaULL,0x58d190f6795dcfb7ULL,0xfef01b0383588bafULL,0x9e6d1d63ca1fc1c0ULL,0x53173f96f0a41ac9ULL,+	0x54637e4182997cc1ULL,0x08c5a96ce3720c9cULL,0x78bce01c11de5d45ULL,0x49d623e50dfdd75aULL,0x8c72a4680fb2a3acULL,0xcc53bbff319c25afULL,0x198eba7978a92421ULL,0xcd61f28ba3bdecf3ULL,+	0x2b1d402aba16f73bULL,0x2fb310148cf9b9fcULL,0x2d51e60e446ef7bfULL,0xc731021bb91e1745ULL,0x9d3b47244fee99d4ULL,0x4bca48b6fac5c1eaULL,0x70f5f514bbea9af7ULL,0x751f55a5974c283aULL,+	0x23899fe8662595c2ULL,0x495d672711a80773ULL,0x86c971d2b0d1d43bULL,0xb518637c93b7a65fULL,0x30e453bad98c99ceULL,0xba6e0d4a14d39f5bULL,0xf7db02a6431ce415ULL,0xcd909c7cf6e1d823ULL,+	0x6e30251acb452fdbULL,0x31ee696550f30650ULL,0xb0b3e508933548d9ULL,0xb8949a4ff4b0ef5bULL,0x208b83263c88f3bdULL,0xab147c30db1d9989ULL,0xed6515fd44d4df03ULL,0x17a12f75e72eb0c5ULL,+	0x25914f7881fdad90ULL,0xcf638f560d2cf6abULL,0xb90bc03fcc054de5ULL,0x932811a718b06350ULL,0x2f00b3309bbd11ffULL,0x76108a6fb4044974ULL,0x801bb9e0a851d266ULL,0x0dd099bebf8990c1ULL,+	0x14c6dd8a58d6cd46ULL,0x9cb633b58e6634d2ULL,0xc1305047f81bc328ULL,0x12ede0e226a177e5ULL,0x332cca62065a6f4fULL,0xc3a47ecd67be487bULL,0x741eb1870f47ed1cULL,0x99e66e58e7598b14ULL,+	0xebd6a6777b0ac93dULL,0xa6e37b0d78f5e0d7ULL,0x2516c09676f5492bULL,0x1e4bf8889ac05f3aULL,0xcdb42ce04df0ba2bULL,0x935d5cfd5062341bULL,0x8a30333382acac20ULL,0x429438c45198b00eULL,+	0xfb2838be67e573e0ULL,0x05891db94084c44bULL,0x9131137396c1c2c5ULL,0x6aebfa3fd958444bULL,0xac9cdce9e56e55c1ULL,0x7148ced32caa46d0ULL,0x2e10c7efb61fe8ebULL,0x9fd835daff97cf4dULL,+	0x6c626f56c1770616ULL,0x5351909e09da9a2dULL,0xe58e6825a3730e45ULL,0x9d8c8bc003ef0a79ULL,0x543f78b6056becfdULL,0x33f13253a090b36dULL,0x82ad4997794432f9ULL,0x1386493c4721f502ULL,+	0x3794eefa5abea82aULL,0x8dc611b993fe62d4ULL,0x69f1af37281ef606ULL,0x6af546c839839e69ULL,0x625578c7c977ec23ULL,0xa8de294cbd5c0576ULL,0xe2ddaf0f7cd1a4c0ULL,0x8243fc704f95f4d4ULL,+	0xe566f400b008733aULL,0xcba0697d512e1f57ULL,0x9537c2b240509cd0ULL,0x5f989c6957353d8cULL,0x7dbec9724c3c2b2fULL,0x90e02fa8ff031fa8ULL,0xf4d15c53cfd5d11fULL,0xb3404fae48314dfcULL,+	0xa36da109081e9387ULL,0xfb9780d78c935828ULL,0xd5940332e540b015ULL,0xc9d7b51be0f466faULL,0xfaadcd41d6d9f671ULL,0xba6c1e28b1a2ac17ULL,0x066a7833ed201e5fULL,0x19d99719f90f462bULL,+	0xf02cc3a9f327a07fULL,0xefb27a9b4490937dULL,0x81451e96b1b3afa5ULL,0x67e24de891883be4ULL,0x1ad65d4770869e54ULL,0xd36291a464a3856aULL,0x070a1abf7132e880ULL,0x9511d0a30e28dfdfULL,+	0xfdeed650f8d1cac4ULL,0xeb99194b6d16bda5ULL,0xb53b19f71cabbe46ULL,0x5f45af5039b9276cULL,0xd0784c6126ee9d77ULL,0xf7a1558b0c02ca5dULL,0xb61d6c59f032e720ULL,0xae3ffb95470cf3f7ULL,+	0x9b185facc72a4be5ULL,0xf66de2364d848089ULL,0xba14d07c717afea9ULL,0x25bfbfc02d551c1cULL,0x2cef0ecd4cdf3d88ULL,0x8cee2aa3647f73c4ULL,0xc10a7d3d722d67f7ULL,0x090037a294564a21ULL,+	0xe6567987fa9ced27ULL,0x36b2d8842a9e5dbcULL,0xf4bdeec6380d8e8cULL,0xb5e6a6b0dbc300d0ULL,0x7ba7e9b9592bef36ULL,0x2b373c4fee81b749ULL,0x484b5e01f0ce596bULL,0x7cc51c62c0bf54ccULL,+	0x6ac07bb84f3815c4ULL,0xddb9f6241aa9017eULL,0x31e30228ca85720aULL,0xe59d63f57cb75838ULL,0x69e18e777baad2d0ULL,0x2cfdb784d42f5d73ULL,0x025dd53df5774983ULL,0x2f80e7cee042cd52ULL,+	0x4bf56bafec695bb0ULL,0x22da1ca8f13c78adULL,0x0f9c4b131182abb0ULL,0x02ea555aae7d249eULL,0x868583f25e05d9e3ULL,0xe09cdcdf70382afaULL,0xdf072ec787080408ULL,0x0a317847cbf75658ULL,+	0x43f18d7f4d6ee4abULL,0xd3ac8cde9570c3dcULL,0x527e49070b8c9b2aULL,0x716709a7c5a4c0f1ULL,0x930852b0916a26b1ULL,0x3cc17fcf4e071177ULL,0x34f5e3d459694868ULL,0xee0341aba28f655dULL,+	0xf431f462060b5f61ULL,0xa56f46b47bd057c2ULL,0x348dca6c47e1bf65ULL,0x9a38783e41bcf1ffULL,0x7a5d33a9da710718ULL,0x5a7799872e0aeaf6ULL,0xca87314d2d29d187ULL,0xfa0edc3ec687d733ULL,+	0x1c894849cb198ac7ULL,0xa884a93d0f264665ULL,0x2da964ef9b200678ULL,0x3c351b87009834e6ULL,0xafb2ef9fe2c4b44bULL,0x580f6c473326790cULL,0xb84805210b02264aULL,0x8ba6f9e242a194e2ULL,+	0x39d934abd3c095f1ULL,0x04b261bee4b76d71ULL,0x1d2e6970e73e6984ULL,0x879fb23b5e5fcb11ULL,0x11506c72dfd75490ULL,0x3a97d08561bcf1c1ULL,0x43201d82bf5e7007ULL,0x7f0ac52f798232a7ULL,+	0xb25101fb319d7682ULL,0xb02931290a982feeULL,0x51c1c9b90261b344ULL,0x0e008c5bbfd371faULL,0xd866dd1c0278ca33ULL,0x666f76a6e5aa53b1ULL,0xe5cfb7796013a2cfULL,0x1d3a1aada3521836ULL,+	0x76b4131a567193ecULL,0xaf3c305ae5f6e70bULL,0x9587bd39031eebddULL,0x5709def871bbe831ULL,0x570599830eb2b669ULL,0x4d80ce1b875b7029ULL,0x838a7da80364ac16ULL,0x2f431d23be1c83abULL,+	0xe781276638235d4eULL,0x1c62bd67496e3298ULL,0x8378660c3f175bc8ULL,0x4d04e18917afdd4dULL,0x32a8160185a8068cULL,0xdb58e4e192b29a85ULL,0xe8a65b86c70d8a3bULL,0x5f0e6f4e98a0403bULL,+	0x2c2492b73f894ae0ULL,0xf59df3e5b75f18ceULL,0x7cb740d28f53cad0ULL,0x3eb585fbc4f01294ULL,0x17da0c8632c7f717ULL,0xeb8c795baf943f4cULL,0x4ee23fb5f67c51d2ULL,0xef18757568889949ULL,+	0x3ea011a4e822f0d0ULL,0xbc647ad15a8704f8ULL,0xbb315b3550c6820fULL,0x863dec3db7e76becULL,0x01ff5d3af017bfc7ULL,0x20054439976b8229ULL,0x067fca370bbd0d3bULL,0xf63dde647f5e3d0fULL,+	0x75d9bc15adf7cccfULL,0x81a3e5d6dfa1e1b0ULL,0x8c39e444249bc17eULL,0xf37dccb28ea7fd43ULL,0xda654873907fba12ULL,0x35daa6da4a372904ULL,0x0564cfc66283a6c5ULL,0xd09fa4f64a9395bfULL,+	0xb510b3b56aa39dffULL,0x59b43da29f8e4d8cULL,0xa8ce31fd9e4c4b9fULL,0x0e20be26c1303c01ULL,0x18187182e8ee47c9ULL,0xd9687cdb7db98101ULL,0x7a520e4da1e14ff6ULL,0x429808ba8836d572ULL,+	0x174d46996d16768eULL,0x9fc4ff6a628bf217ULL,0x77705a94154e490dULL,0x9d96dd288d2d997aULL,0x77e2d9d8ce5d72c4ULL,0x9d06c5a4c11c714fULL,0x02aa513679e4a03eULL,0x1386b3c2030ff28bULL,+	0x26a42d69ea40dc3aULL,0xdc84ad22aecc018fULL,0x25c36c7b3270f04aULL,0x46ba6d4750fa72edULL,0x6c37d1c593e58a8eULL,0xa2394731120c088cULL,0xc3be4263cb6e86daULL,0x2c417d367126d038ULL,+	0xcca523bb440e2229ULL,0x324673a273ef4d04ULL,0xaf3adf343e11ec39ULL,0x6136d7f1dc5968d3ULL,0x7a7b2899b053a927ULL,0x3eaa2661ae067ecdULL,0x8549b9c802779cd9ULL,0x061d7940c53385eaULL,+	0xe07141fcaaa2902bULL,0x539ad799e4f69ad3ULL,0xa6453f94813f9ffdULL,0xc58d3c48375bc2f7ULL,0xb3326fad5dc64e96ULL,0x3aafcaa9b240e354ULL,0x1d1b0903aca1e7a9ULL,0x4ceb97671211b8a0ULL,+	0x1eb4de4687032d58ULL,0xc54f3d835e2c79e0ULL,0x07818df45d04ef23ULL,0x55faa9c8673d41b4ULL,0xced64f6f89b95355ULL,0x4860d2eab7415c84ULL,0x5fdb9bd2050ebad3ULL,0xdb53e0cc6685a5bfULL,+	0x919fca5fabfae1caULL,0x937afaac1a21459bULL,0x9e0ca91c1f66a4d2ULL,0x194cc7f323ec1331ULL,0xad25143a8aa11690ULL,0xbe40ad8d09b59e08ULL,0x37d60d9be750860aULL,0x6c53b008c6bf434cULL,+	0x832d7080eb6b242dULL,0xd30bd0233b71e246ULL,0x7027991bbe31139dULL,0x68797e91462e4e53ULL,0x423fe20a6b4e185aULL,0x82f2c67e42d9b707ULL,0x25c817684cf7811bULL,0xbd53005e045bb95dULL,+	0xe5f649be9d8e68fdULL,0xdb0f05331b044320ULL,0xf6fde9b3e0c33398ULL,0x92f4209b66c8cfaeULL,0xe9d1afcc1a739d4bULL,0x09aea75fa28ab8deULL,0x14375fb5eac6f1d0ULL,0x6420b560708f7aa5ULL,+	0x9eae499c6254dc41ULL,0x7e2939247a837e7eULL,0x74aec08c090524a7ULL,0xf82b92198d6f55f2ULL,0x493c962e1402cec5ULL,0x9f17ca17fa2f30e7ULL,0xbcd783e8e9b879cbULL,0xea3d8c145a6f145fULL,+	0xdede15e75e0dee6eULL,0x74f24872dc628aa2ULL,0xd3e9c4fe7861bb93ULL,0x56d4822a6187b2e0ULL,0xb66417cfc59826f9ULL,0xca2609692408169eULL,0xedf69d06c79ef885ULL,0x00031f8adc7d138fULL,+	0x103c46e60ebcf726ULL,0x4482b8316231470eULL,0x6f6dfaca487c2109ULL,0x2e0ace9762e666efULL,0x3246a9d31f8d1f42ULL,0x1b1e83f1574944d2ULL,0x13dfa63aa57f334bULL,0x0cf8daed9f025d81ULL,+	0x30d78ea800ee11c1ULL,0xeb053cd4b5e3dd75ULL,0x9b65b13ed58c43c5ULL,0xc3ad49bdbd151663ULL,0x99fd8e41b6427990ULL,0x12cf15bd707eae1eULL,0x29ad4f1b1aabb71eULL,0x5143e74d07545d0eULL,+	0x30266336c88bdee1ULL,0x25f293065876767cULL,0x9c078571c6731996ULL,0xc88690b2ed552951ULL,0x274f2c2d852705b4ULL,0xb0bf8d444e09552dULL,0x7628beeb986575d1ULL,0x407be2387f864651ULL,+	0x0e5e3049a639fc6bULL,0xe75c35d986003625ULL,0x0cf35bd85dcc1646ULL,0x8bcaced26c26273aULL,0xe22ecf1db5536742ULL,0x013dd8971a9e068bULL,0x17f411cb8a7909c5ULL,0x5757ac98861dd506ULL,+	0x85de1f0d1e935abbULL,0xdefd10b4154de37aULL,0xb8d9e392369cebb5ULL,0x54d5ef9b761324beULL,0x4d6341ba74f17e26ULL,0xc0a0e3c878c1dde4ULL,0xa6d7758187d918fdULL,0x6687601502ca3a13ULL,+	0xc7313e9cf36658f0ULL,0xc433ef1c71f8057eULL,0x853262461b6a835aULL,0xc8f053987c86394cULL,0xff398cdfe983c4a1ULL,0xbf5e816203b7b931ULL,0x93193c46b7b9045bULL,0x1e4ebf5da4a6e46bULL,+	0xf9942a6043a24fe7ULL,0x29c1191effb3492bULL,0x9f662449902fde05ULL,0xc792a7ac6713c32dULL,0x2fd88ad8b737982cULL,0x7e3a0319a21e60e3ULL,0x09b0de447383591aULL,0x6df141ee8310a456ULL,+	0xaec1a039e6d6f471ULL,0x14b2ba0f1198d12eULL,0xebc1a1603aeee5acULL,0x401f4836e0b964ceULL,0x2ee437964fd03f66ULL,0x3fdb4e49dd8f3f12ULL,0x6ef267f629380f18ULL,0x3e8e96708da64d16ULL,+	0xbc19180c207674f1ULL,0x112e09a733ae8fdbULL,0x996675546aaeb71eULL,0x79432af1e101b1c7ULL,0xd5eb558fde2ddec6ULL,0x81392d1f5357753fULL,0xa7a76b973ae1158aULL,0x416fbbff4a899991ULL,+	0x9e65fdfd0d4a9dcfULL,0x7bc29e48944ddf12ULL,0xbc1a92d93c856866ULL,0x273c69056e98dfe2ULL,0x69fce418cdfaa6b8ULL,0x606bd8235061c69fULL,0x42d495a06af75e27ULL,0x8ed3d5056d873a1fULL,+	0xaf5528416ab25b6aULL,0xc6c0ffc72b1a4523ULL,0xab18827b21c99e03ULL,0x060e86489034691bULL,0x5207f90f93c7f398ULL,0x9f4a96cb82f8d10bULL,0xdd71cd793ad0f9e3ULL,0x84f435d2fc3a54f5ULL,+	0x4b03c55b8e33787fULL,0xef42f975a6384673ULL,0xff7304f75051b9f0ULL,0x18aca1dc741c87c2ULL,0x56f120a72d4bfe80ULL,0xfd823b3d053e732cULL,0x11bccfe47537ca16ULL,0xdf6c9c741b5a996bULL,+	0xee7332c7904fc3faULL,0x14a23f45c7e3636aULL,0xc38659c3f091d9aaULL,0x4a995e5db12d8540ULL,0x20a53becf3a5598aULL,0x56534b17b1eaa995ULL,0x9ed3dca4bf04e03cULL,0x716c563ad8d56268ULL,+	0x5043dea7e0f222c2ULL,0x309d42ac72e65142ULL,0x94fe9ddd9216cd30ULL,0xd6539c7d0f87feecULL,0x03c5a57c432ac7d7ULL,0x72692cf0327fda10ULL,0xec28c85f280698deULL,0x2331fb467ec283b1ULL,+	0xa0158eeae457a477ULL,0xd19857dbee6ddc05ULL,0xb326522418c41671ULL,0x3ffdfc7e3c2c0d58ULL,0x3a3a525426ee7cdaULL,0x341b0869df02c3a8ULL,0xa023bf42723bbfc8ULL,0x3d15002a14452691ULL,+	0xc961b2f687500b96ULL,0x795510e72dcd9425ULL,0x0308172978615433ULL,0xe5d0145465445029ULL,0x5bd13302bf690cbeULL,0x44e48831731eca67ULL,0x73306bc72b8038a4ULL,0x351d151ebf57bf02ULL,+	0x5ef7324c85edfa30ULL,0x2597655487d4f3daULL,0x352f5bc0dcb50c86ULL,0x8f6927b04832a96cULL,0xd08ee1ba55f2f94cULL,0x6a996f99344b45faULL,0xe133cb8da8aa455dULL,0x5d0721ec758dc1f7ULL,+	0xf3d44e1f9a876441ULL,0x82bc0c14147a818dULL,0x33ddc1170603eddeULL,0x77163f2e0a25f260ULL,0xb54c02caa3bfaa53ULL,0xbd1b2502e2256982ULL,0xc4e1728c1a6f37dcULL,0xfe36c213814c94a5ULL,+	0xf3cae7e9262a3539ULL,0x78a49d1d6670d59eULL,0x37de0f63c1c5e1b9ULL,0x3072c30c69cb7c1cULL,0x1d278a5277c850e6ULL,0x84f15f8f1f6a3de6ULL,0x46a8bb45592ca7adULL,0x1912e3eee4d424b8ULL,+	0xc1ffe2d490e31734ULL,0x91b1f1267fca007cULL,0x5459b1d0ae3f77e8ULL,0x262b051d46425c88ULL,0xcf5c8f765c51e274ULL,0x997481e2304e6146ULL,0x6fc1198bd84046b5ULL,0x1cb0a6bbe7f7a6bdULL,+	0x6ba7a92079e5fb67ULL,0xe1331feb70aa725eULL,0x5080ccf57df5d837ULL,0xe4cae01d7ff72e21ULL,0xd9243ee60412a77dULL,0x06ff7cacdf449025ULL,0xbe75f7cd23ef5a31ULL,0xbc9578220ddef7a8ULL,+	0xc4737ad11b7f30b0ULL,0x525ab2c63629dcf9ULL,0x963f4cc1186ae160ULL,0x8507671373e6b6e0ULL,0xd9be3180f6998bcfULL,0x93d91da3b1c8d8d8ULL,0xf902ce661b8c0054ULL,0x47e7924d74a8a768ULL,+	0xdc988086365e668bULL,0xada8dcdaaabda5fbULL,0xbc146b4c255f1fbeULL,0x9cfcde29cf34cfc3ULL,0xacbb453e7e85d1e4ULL,0x9ca09679f92358b5ULL,0x15fc2d96240823ffULL,0x8d65adf70c11d11eULL,+	0x323cfd177e19a46fULL,0x0948a7a786161156ULL,0x50d06b977e7d3363ULL,0x41c47ec1a702579dULL,0x9455998e59e9260bULL,0xc865e44446c24260ULL,0x393021ec13bc3744ULL,0x4981994ecd92a14aULL,+	0x8cf7230cb0ce1c55ULL,0x5b534d050bbfb607ULL,0xee1ef1130e16363bULL,0x27e0aa7ab4999e82ULL,0xce1dac2d79362c41ULL,0x67920c9091bb6cb0ULL,0x1e648d632223df24ULL,0x0f7d9eefe32e8f28ULL,+	0x9766f264d66f51c0ULL,0x644317ab5d9ccea5ULL,0xa39b37bcd721e232ULL,0x98bdde0bb9daf737ULL,0xc2ecc758165166bdULL,0x0951a285a2802108ULL,0xe39fbf24997aa66fULL,0x1a2f6862db62da27ULL,+	0x775557f10296f4fdULL,0x1dca76a3ea51b436ULL,0xf3e98f60fb950805ULL,0x31ff32ea831cf7f1ULL,0x643e7bf18d2c714bULL,0x64b5c3392e9d2acaULL,0xa9fd9ccc6adc2d23ULL,0xfc2397eccc721b9bULL,+	0xab651fc764465367ULL,0xe43870152b098f57ULL,0x0a91d519d382376fULL,0xb5afdb0a53cad929ULL,0x457e1875138d5523ULL,0xa92becae1aecacfbULL,0x0762f6e811484f49ULL,0x114b5c86dda16c2bULL,+	0x6943f39afa833834ULL,0x22951722a6328562ULL,0x81d63dd54170fc10ULL,0x9f5fa58faecc2e6dULL,0xb66c8725e77d9a3bULL,0x11235cea6384ebe0ULL,0x06a8c1185845e24aULL,0x0137b286ebd093b1ULL,+	0xdb567d6ac42bd6d2ULL,0x6df86468bb1f96aeULL,0x0efe5b1a4843b28eULL,0x961bbb056379b240ULL,0xb6caf5f070a6a26bULL,0x70686c0d328e6e39ULL,0x80da06cf895fc8d3ULL,0x804d8810b363fdc9ULL,+	0x63b99ce74462007dULL,0xb8ab48a54cb5f5b7ULL,0x9ec673d2f55edde7ULL,0xd1567f748cfaefdaULL,0x46381b6b0887bcecULL,0x694497cee178f3c2ULL,0x5e6525e31e6266cbULL,0x5931de26697d6413ULL,+	0x14e49da11f17a34cULL,0x5420ab39235a1456ULL,0xb76372412f50363bULL,0x7b15d623c3fabb6eULL,0xa0ef40b1e274e49cULL,0x5cf5074496b1860aULL,0xd6583fbf66afe5a4ULL,0x44240510f47e3e9aULL,+	0x142b55021a93507aULL,0xb4cd11878d3c06cfULL,0xdf70e76a91ec3f40ULL,0x484e81ad4e7553c2ULL,0x830f87b5272e9d6eULL,0xea1c93e5c6ff514aULL,0x67cc2adcc4192a8eULL,0xc77e27e242f4535aULL,+	0xb5358b1e48ac2840ULL,0x18311294ecba9477ULL,0xda58f990a6946b43ULL,0x3098baf99ab41819ULL,0x66c4c1584198da52ULL,0xab4fc17c146bfd1bULL,0x2f0a4c3cbf36a908ULL,0x2ae9e34b58cf7838ULL,+	0x45eb40ec0ccced58ULL,0x25cd4b9c0da44f98ULL,0x43e06458871812c6ULL,0x99f80d5516cef651ULL,0x571340c9ce6dc153ULL,0x138d5117d8665521ULL,0xacdb45bc4e07014dULL,0x2f34bb3884b60b91ULL,+	0x417499e84a34f239ULL,0x15fdb83cb90402d5ULL,0xb75f46bf433aa832ULL,0xb61e15af63215db1ULL,0xaabe59d4a127f89aULL,0x5d541e0c07e816daULL,0xaaba0659a618b692ULL,0x5532773317266026ULL,+	0x8cda9cf2d0c05199ULL,0x502fbc22fae78454ULL,0xc0bda9dff572a182ULL,0x5f9b71b86158b372ULL,0xe0f33a592b82dd07ULL,0x763027359523032eULL,0x7fe1a721c4505a32ULL,0x7b6e3e82f796409fULL,+	0x023c155d3f6effc7ULL,0x1fbd69ff9c90f0c7ULL,0xe5d7da8abeec2c5dULL,0x8813872bd7e86273ULL,0x9f3bc2c655f5e228ULL,0x11482869b0923b41ULL,0x65d75c741aa307caULL,0xda92c2577f24eee5ULL,+	0x26357732edc665d1ULL,0x9fb5b731a939ef1bULL,0x7db720fb94968089ULL,0x36f75f2c33138c52ULL,0xf8b793ec48d3cb97ULL,0x8dff1d456d261726ULL,0xfb65791b885c4ffbULL,0xf7c79e2df1a3a870ULL,+	0x08dd1028754c92e1ULL,0xca90b57acf0fef34ULL,0x1a9b84ac8af55919ULL,0xaa95e0e1ed93686bULL,0x46737315167021a4ULL,0x6cb6a0da20d5ff98ULL,0xecc4801a1092e706ULL,0xedcab23a3c5e61a6ULL,+	0xb4c66356ea37ba9eULL,0x1adc84150afff55dULL,0xf0080ef9596cc862ULL,0x756c85b86d647ab6ULL,0xc9db94aa1db9c215ULL,0x2dd36db12013b1a5ULL,0xde6ac61c4286c903ULL,0x3fd32f88c76cf884ULL,+	0x7f1290fca06d107eULL,0x697261fdb7661137ULL,0x1bb5be4e947b4b38ULL,0xb49826b63bb79130ULL,0x019ddfe85ba8bffbULL,0xb1af79007e3fa8e4ULL,0x72e1bdf201bcfe7fULL,0x2ed3ca8fd1169aeaULL,+	0x3befda8cd745fff1ULL,0x70b9e9b669b9924eULL,0xa5df48cfd1511381ULL,0x84f93fe2d06bc535ULL,0xaa42c5a9b279a6c3ULL,0x651da6c4d8f96132ULL,0xb0368c8b01b6aea5ULL,0x64e44c47ac7862a2ULL,+	0xe17a9947d9de99a8ULL,0xc2e61b2dc93477bdULL,0x57f684d41d19e287ULL,0x843c2122fe358135ULL,0xe2d3e2e904f7e8abULL,0xbf93ffe9b5f27aeeULL,0x29830d1d7b1858c4ULL,0xa8f449648106adbfULL,+	0xe3417bc035d0b34aULL,0x440b386b8327c0a7ULL,0x8fb7262dac0362d1ULL,0x2c41114ce0cdf943ULL,0x2ba5cef1ad95a0b1ULL,0xc09b37a867d54362ULL,0x26d6cdd201e486c9ULL,0x20477abf42ff9297ULL,+	0xa004dcb3292a9287ULL,0xddc15cf677b092c7ULL,0x083a8464806c0605ULL,0x4a68df703db997b0ULL,0x9c134e4505bf7dd0ULL,0xa4e63d398ccf7f8cULL,0xa6e6517f41b5f8afULL,0xaa8b9342ad7bc1ccULL,+	0xc41764717af715d2ULL,0xe2f7f594d0134a96ULL,0x2c1873efa41ec956ULL,0xe4e7b4f677821304ULL,0xe5c8ff9788d5374aULL,0x2b915e6380823d5bULL,0xea6bc755b2ee8fe2ULL,0x6657624ce7112651ULL,+	0xd6f800e07442f1d5ULL,0x475607d166e0e3abULL,0x82807f16b7c64047ULL,0x8858e1e3a749883dULL,0x5859120b8231ee10ULL,0x1b80e7eb638a1eceULL,0xcb72525ac6aa73a4ULL,0xa7cdea3d844423acULL,+	0x57477b11e51732d2ULL,0xdfd6eb282538fc0eULL,0x5c43b0cc3b39eec5ULL,0x6af12778cb36cc57ULL,0x70b0852d06c425aeULL,0x6df92f8c5c221b9bULL,0x6c8d4f9ece826d9cULL,0xf59aba7bb49359c3ULL,+	0xd2eb2cf152bfda05ULL,0xe0e4c4e96197b98cULL,0x1d35076cf8a1726fULL,0x6c06085b2db11e3dULL,0x15c0c4d74463ba14ULL,0x9d292f830030238cULL,0x1311ee8b3727536dULL,0xfeea86efbeaedc1eULL,+	0xa7f96054afa05dd8ULL,0x26dfcf21fcaf119eULL,0xe20ef2e30564bb59ULL,0xef4dca5061cb02b8ULL,0xcda7838a65d30672ULL,0x8b08d534fd657e86ULL,0x4c5b439546d595c8ULL,0x39b58725425cb836ULL,+	0xfda853931a62cc26ULL,0x23c69b9650c0e052ULL,0xa227df15bfc633f3ULL,0x2ac788481bae7d48ULL,0x487878f9187d073dULL,0x6c2be919967f807dULL,0x765861d8336e6d8fULL,0x88b8974cce528a43ULL,+	0xc37e2c2e421d3aa4ULL,0xf926407ce84fa840ULL,0x18abc03d1454e41cULL,0x26605ecd3f7af644ULL,0x242341a6d6a5eabfULL,0x1edb84f4216b668eULL,0xd836edb804010102ULL,0x5b337ce7945e1d8cULL,+	0x666ba2dccc78cf66ULL,0xb30181746fdbff77ULL,0x8d4dd0db168d4668ULL,0x259455d01dab3a2aULL,0xf58564c5cde3acecULL,0x7714192513adb276ULL,0x527d725d8a303f65ULL,0x55deb6c9e6f38f7bULL,+	0x864d05d73272d838ULL,0xe22924f9fa6295c5ULL,0x8189593f6c2fda32ULL,0x330d7189b184b544ULL,0x79efa62cbde1f714ULL,0x35771c94e5cb1a63ULL,0x2f4826b8641c8332ULL,0x00a894fbc8cee854ULL,+	0xdcdacd0a1058a318ULL,0x369cf3f578053a9aULL,0xc6c3de5031c68de2ULL,0x4653a5763c4b6d9fULL,0x1688dd5aaa4e5c97ULL,0x5be80aa1b7ab3c74ULL,0x70cefe7cbc65c283ULL,0x57f95f1306867091ULL,+	0xc240b6de34eaacdaULL,0xd9e116e82ba0f1deULL,0xcbe45ec779438e55ULL,0x91787c9d96f752d7ULL,0x897f532bf129ac2fULL,0xd307b7c85a36e22cULL,0x91940675749fb8f3ULL,0xd14f95d0157fdb28ULL,+	0x1625360416df4285ULL,0xb0c9babbd0c56ae2ULL,0x73032b19cfc5cfc3ULL,0xe497e5c309752056ULL,0x12096bb4164bda96ULL,0x1ee42419a0b74da1ULL,0x8fc36243403826baULL,0x0c8f0069dc09e660ULL,+	0xc44b74a15b0ec6f5ULL,0x47989fe45289b2b8ULL,0x745f848458d6fc73ULL,0xec362a6ff61c70abULL,0x070c98a7b3a8ad41ULL,0x73a20fc07b63db51ULL,0xed2c2173f44c35f4ULL,0x8a56149d9acc9dcaULL,+	0xa395c36f35d33ae7ULL,0x200ea12350bb5a94ULL,0x20c789bd0bafe84bULL,0x243ef52d0919276aULL,0x3934c577e23ae233ULL,0xb93807afa460d1ecULL,0xb72a53b1f8fa76a4ULL,0xd8914cb0c3ca4491ULL,+	0x4c076b86d23ddc82ULL,0x03fd344c7e0143f0ULL,0xa95362ff317af2c5ULL,0x0add3db7e18b7a4fULL,0x9c673e3f8260e01bULL,0xfbeb49e554a1cc91ULL,0x91351bf292f2e433ULL,0xc755e7ec851141ebULL,+	0x2bf5db47f23206d5ULL,0x2f6d34201d260152ULL,0x17b876533f8ff89aULL,0x5157c30c378fa458ULL,0x7517c5c52d4fb936ULL,0xef22f7ace6518cdcULL,0xdeb483e6bf847a64ULL,0xf508455892e0fa89ULL,+	0xab9659d8df7304d4ULL,0xb71bcf1bff210e8eULL,0xa9a2438bd73fbd60ULL,0x4595cd1f5d11b4deULL,0x9c0d329a4835859dULL,0x4a0f0d2d7dbb6e56ULL,0xc6038e5edf928a4eULL,0xc94296218f5ad154ULL,+	0x08a33840a70c6ec4ULL,0x9e8819f7e0311195ULL,0xed209d96708ab202ULL,0x10d7c4e7ce943a27ULL,0x372fb317a29b49a1ULL,0x57a67fb346627d1fULL,0xf912561e7cdf39ecULL,0xfa3ce6f26f7c8f17ULL,+	0x91213462f23f2d92ULL,0x6cab71bd60b94078ULL,0x6bdd0a63176cde20ULL,0x54c9b20cee4d54bcULL,0x3cd2d8aa9f2ac02fULL,0x03f8e617206eedb0ULL,0xc7f68e1693086434ULL,0x831469c592dd3db9ULL,+	0xfe7d7465653f3c5fULL,0x283dd45ef040feb1ULL,0x91fe599bf3b7edfeULL,0x5ff039ad80379311ULL,0xbf76995b4e96fa49ULL,0x2640b6b2a3e25094ULL,0x8b096341c1c83f74ULL,0xd2bec884fa560ac3ULL,+	0x8521df248f981354ULL,0x587e23ec3588a259ULL,0xcbedf281d7a0992cULL,0x06930a5538961407ULL,0x09320debbe5bbe21ULL,0xa7ffa5b52491817fULL,0xe6c8b4d909065160ULL,0xac4f3992fff6d2a9ULL,+	0x2e4eb2a3cc53da66ULL,0x04708a71f17a9b4eULL,0xdbfdc7b20de05b2cULL,0x4cdc9aee4907a201ULL,0xe5cc8dfc6c475566ULL,0x2b83cbfb47be1691ULL,0x695833a74c05c3fdULL,0xee938a7243b0deb0ULL,+	0x7aa7a1583ae9c1bdULL,0xe0af6d98e37ce240ULL,0xe54342d928ab38b4ULL,0xe8b750070a1c98caULL,0xefce86afe02358f2ULL,0x31b8b856ea921228ULL,0x052a19120a1c67fcULL,0xb4069ea4e3aead59ULL,+	0x3b826205c6ffcfe6ULL,0x2eb31256aa39418aULL,0xc4b4a9e1c18521fbULL,0x48614dd8db610615ULL,0x04c362bbeea7475fULL,0x916ab969a8ead162ULL,0x6a7975bc0310a876ULL,0xea4e7d11ecc77ec4ULL,+	0x3232d6e27fa03cb3ULL,0xdb938e5b0fdd7d88ULL,0x04c1d2cd2ccbfc5dULL,0xd2f45c12af3a580fULL,0x592620b57883e614ULL,0x5fd27e68be7c5f26ULL,0x139e45a91567e1e3ULL,0x2cc71d2d44d8aaafULL,+	0x6ef493c706ba8e6fULL,0x05c07bf8123deffdULL,0xcebf9b4d9c2b5a4fULL,0xf958147e2d038e9cULL,0x5af5cee03e5561c4ULL,0x8d0b5a7a794a6afcULL,0x9de22df13772168bULL,0xe5d249c5d84097fdULL,+	0x4a9090cde36d0757ULL,0xf722d7b1d9a29382ULL,0xfb7fb04c04b48ddfULL,0x628ad2a7ebe16f43ULL,0xcd3fbfb520226040ULL,0x6c34ecb15104b6c4ULL,0x30c0754ec903c188ULL,0xec336b082d23cab0ULL,+	0x74c70b4295c69248ULL,0x8813259dfed90303ULL,0xa3e330684b8cc87aULL,0xe689371c111a7a95ULL,0x52bfbbf7fbbbc20bULL,0x73a8543d65f9a6e2ULL,0x67bb2fdd7e413e6dULL,0xa066b3efc5cf2032ULL,+	0x473d62a21e206ee5ULL,0xf1e274808c49a633ULL,0x87ab956ce9f6b2c3ULL,0x61830b4862b606eaULL,0x67cd6846e78e815fULL,0xfe40139f4c02082aULL,0x52bbbfcb952ec365ULL,0x74c116426b9836abULL,+	0xcf61b89c44f48971ULL,0xe2d700f76d660683ULL,0x72ef285cd1d431bdULL,0x0593b24e9bdebf4aULL,0x4084bc5b0561f8a1ULL,0x84ce74d0aa16f256ULL,0x9cbc79d309f6d277ULL,0xa94fe2fe4139bdfeULL,+	0x9f51439e558df019ULL,0x230da4baac712b27ULL,0x518919e355185a24ULL,0x4dcefcdd84b78f50ULL,0xa7d90fb2a47d4c5aULL,0x55ac9abfb30e009eULL,0xfd2fc35974eed273ULL,0xb72d824cdbea8fafULL,+	0x549db2b5ef7d9289ULL,0x2480d4a8197f015aULL,0x61d5590bc40493b6ULL,0x3a55b52e6f780331ULL,0x40eb8115309eadb0ULL,0xdea7de5a92e5c625ULL,0x64d631f0cc6a3d5aULL,0x9d5e9d7c93e8dd61ULL,+	0x196860411e84e0e5ULL,0xa5db84d3aea34c93ULL,0xf9d5bb197073a732ULL,0xb8d2fe566bcfd7c0ULL,0x45775f36f3eb82faULL,0x8cb20cccfdff8b58ULL,0x1659b65f8374c110ULL,0xb8b4a422330c789aULL,+	0xc925ff87aedbae9fULL,0x7daf0eb936880a54ULL,0x9284ddf59c4d0e71ULL,0x1581cf93316f8cf5ULL,0x3eeca8873ac1f452ULL,0xb417fce9fb6aeffeULL,0xa5918046eefb8dc3ULL,0x73d318ac02209400ULL,+	0xc4f4cda3af2ebc2fULL,0xa0af843dcb4efe24ULL,0x53b857c19ccd10b1ULL,0xddc9d1eb914d3e04ULL,0x7bdec8bb62771debULL,0x829277aa91c5aa81ULL,0x7af18dd6832391aeULL,0x1740f316c71a84caULL,+	0xdd4a12d8e12b31f8ULL,0x577e29bc177736e6ULL,0x2353722ba88935e8ULL,0xca1d3729015f286dULL,0x86c7b6a239a3e035ULL,0x6e5250bfd3b03a9fULL,0x79d98930fd0d536eULL,0x8c4cbbabfa0c3832ULL,+	0x2d500910cab91f1eULL,0xbedd9e444d1cd216ULL,0xd634b74fedd02252ULL,0xbd60f8e11258617aULL,0xd8c7537b9e05614aULL,0xfd26c766e7af5fc5ULL,0x0660b581582bd926ULL,0x87019244acf07fc8ULL,+	0xafa26ccfaf64ecb6ULL,0xe6054f974bd72775ULL,0xbbcbab5b140f695aULL,0xbc71b4a4e348efdeULL,0xfc2e52becc96d963ULL,0x150abf5f5e5d9018ULL,0xbd182fa604568771ULL,0x35b4c06170339f83ULL,+	0x8928e99aeeaf8c49ULL,0xee7aa73d6e24d728ULL,0x4c5007c2e72b156cULL,0x5fcf57c5ed408a1dULL,0x9f719e39b6057604ULL,0x7d343c01c2868bbfULL,0x2cca254b7e103e2dULL,0xe6eb38a9f131bea2ULL,+	0xc624a04e5f40ff52ULL,0x3611d7cffcd2914aULL,0xb7e8b42b1fd3bfd6ULL,0x6cde40fdfa85063aULL,0x7811c449178b7e5bULL,0x4cb609312972cc13ULL,0xf579125e33b46135ULL,0xca102ef788a4e56eULL,+	0x0ba4e3520a981b0dULL,0x1c354cb3bd1a41a4ULL,0x1aabaa3adf9fab9cULL,0x0701a7d153c418d5ULL,0xdd1a7cefdcf2b921ULL,0x6ceef0b3bcf48061ULL,0x1083b598de25cce6ULL,0x890a54c7e90a5e34ULL,+	0xc535956640ac1807ULL,0xd4c1e56684da0b1eULL,0xc4b33f97e0b82421ULL,0xd0bd23177b41be00ULL,0x53a4b42e147b72e1ULL,0xf8d39f5ff777104cULL,0x36e64e64d3fb530dULL,0xa7a6ba6756074fddULL,+	0x405718db4f6d01b1ULL,0xe73c6bc28f11e8a0ULL,0xac11bb8ca0591a3bULL,0x12d09a5a0acc4531ULL,0xcbf174eee7de13f4ULL,0x177e2be6044fd682ULL,0x65f574cb1c48af70ULL,0xce5966929961cb7cULL,+	0x989fe84ebc6fab9fULL,0xe70ce6b1c80f6474ULL,0xbe9ff3053b02a1bcULL,0x12ef486699c0afd3ULL,0x22d957f9e3411a26ULL,0x0b41d8817b485b98ULL,0x820e56d04ae20119ULL,0x81e3d01f328528e0ULL,+	0xc59eed6c048752a1ULL,0x41f2702ea01341b4ULL,0x6e35903b9dc6b092ULL,0x4291aba81f5b5b23ULL,0x8173aa70a653d61dULL,0xd1b648d44f2eb51eULL,0x31b7ce065ab93f8fULL,0xa55408ee99e2f4feULL,+	0x1fa4ed0985e34160ULL,0xa26d7dc37a03cde2ULL,0x1dac0848fa84df2dULL,0x5e9a28d61b697108ULL,0xa88004d914ea0ea1ULL,0xa8d154283ffe5520ULL,0x4f422dae639b139cULL,0xeedccc0dd4b5a861ULL,+	0xb33e624f8be762b4ULL,0x2a9ee4d1058e3413ULL,0x968e636967d805faULL,0x9848949b7db8bfd7ULL,0x5308d7e5d23a8417ULL,0x892f3b1df3e29da5ULL,0xc95c139e3dee471fULL,0x8631594dd757e089ULL,+	0x1083e2ea1f095615ULL,0x0a28ad7714e68c33ULL,0x6bfc02523d8818beULL,0xb585113af35850cdULL,0x7d935f0b30df8aa1ULL,0xaddda07c4ab7e3acULL,0x92c34299552f00cbULL,0xc33ed1de2909df6cULL,+	0x2dc40d483e07113cULL,0x6e4a5d397d8b63aeULL,0x5582a94b79684c2bULL,0x932b33d4622da26cULL,0xf534f6510dbbf08dULL,0x211d07c964c23a52ULL,0x0eeece0fee5bdc9bULL,0xdf178168f7015558ULL,+	0xabe7905a83cdd60eULL,0x50602fb5a1170184ULL,0x689886cdb023642aULL,0xd568d090a6e1fb00ULL,0x5b1922c70259217fULL,0x93831cd9c43141e4ULL,0xdfca35870c95f86eULL,0xdec2057a568ae828ULL,+	0x568f8925913cc16dULL,0x18bc5b6de1a26f5aULL,0xdfa413bef5f499aeULL,0xf8835decc3f0ae84ULL,0xb6e60bd865a40ab0ULL,0x65596439194b377eULL,0xbcd8562592084a69ULL,0x5ce433b94f23ede0ULL,+	0x860d523d42e06189ULL,0xbf0779414e3aff13ULL,0x0b616dcac1b20650ULL,0xe66dd6d12131300dULL,0xd4a0fd67ff99abdeULL,0xc9903550c7aac50dULL,0x022ecf8b7c46b2d7ULL,0x3333b1e83abf92afULL,+	0xc0da65e784d6365dULL,0xbcb7443f8f759fb8ULL,0x35c712b17ae81930ULL,0x80428dff4c6e08abULL,0xf19dafefa4faf843ULL,0xced8538dffa9855fULL,0x20ac409cbe3ac7ceULL,0x358c1fb6882da71eULL,+	0xefecdef7be42a582ULL,0xd3fc608065046be6ULL,0xc9af13c809e8dba9ULL,0x1e6c9847641491ffULL,0x3b574925d30c31f7ULL,0xb7eb72baac2a2122ULL,0x776a0dacef0859e7ULL,0x06fec31421900942ULL,+	0x324794b07e50122bULL,0xdd744f8b4af07ca5ULL,0x30a12f08d63fc97bULL,0x39650f1a76626d9dULL,0x101b47f71fa38477ULL,0x3d815f19d4dc124fULL,0x1569ae95b26eb58aULL,0xc3cde18895fb1887ULL,+	0x7ec62fbbf4737f21ULL,0xd8dba5ab6209f5acULL,0x24b5d7a9a5f9adbeULL,0x707d28f7a61dc768ULL,0x7711460bcaa999eaULL,0xba7b174d1c92e4ccULL,0x3c4bab6618d4bf2dULL,0xb8f0c980eb8bd279ULL,+	0x9d658932790691bfULL,0xed61058906b736aeULL,0x712c2f04c0d63b6eULL,0x5cf06fd5c63d488fULL,0x97363facd9588e41ULL,0x1f9bf7622b93257eULL,0xa9d1ffc4667acaceULL,0x1cf4a1aa0a061ecfULL,+	0x28d675b2c0519a23ULL,0x9ebf94fe4f6952e3ULL,0xf28bb767a2294a8aULL,0x85512b4dfe0af3f5ULL,0x18958ba899b16a0dULL,0x95c2430cba7548a7ULL,0xb30d1b10a16be615ULL,0xe3ebbb9785bfb74cULL,+	0x07b53f5eb2e63645ULL,0xbe57e54784c84232ULL,0xd779c2167214d5cfULL,0x617969cd029a3acaULL,0xd17668cd8a7017a0ULL,0x77b4d19abe9b7ee8ULL,0x58fd0e939c161776ULL,0xa8c4f4efd5968a72ULL,+	0x81eeb865d2fdca23ULL,0x5a15ee08cc8ef895ULL,0x768fa10a01905614ULL,0xeff5b8ef880ee19bULL,0xf0c0cabbcb1c8a0eULL,0x2e1ee9cdb8c838f9ULL,0x0587d8b88a4a14c0ULL,0xf6f278962ff698e5ULL,+	0x519d34b3bf44da80ULL,0x283834f95ab32e66ULL,0x6e6087976278a000ULL,0x1e62960e627312f6ULL,0x9b87b27be6901c55ULL,0x80e7853824fdbc1fULL,0xbbbc09512facc27dULL,0x06394239ac143b5aULL,+	0x9c4b646e9e2fce99ULL,0x68a210811e80857fULL,0x06d54e443643b52aULL,0xde8d6d630d8eb843ULL,0x7032156342146a0aULL,0x8ba826f25eaa3622ULL,0x227a58bd86138787ULL,0x43b6c03c10281d37ULL,+	0x02b37a952f41deffULL,0x0e44a59ae63b89b7ULL,0x673257dc143ff951ULL,0x19c02205d752baf4ULL,0x46c23069c4b7d692ULL,0x2e6392c3fd1502acULL,0x6057b1a21b220846ULL,0xe51ff9460c1b5b63ULL,+	0x6e85cb51566c5c43ULL,0xcff9c9193597f046ULL,0x9354e90c4994d94aULL,0xe0a393322147927dULL,0x8427fac10dc1eb2bULL,0x88cfd8c22ff319faULL,0xe2d4e68401965274ULL,0xfa2e067d67aaa746ULL,+	0xb6d92a7f3e5f9f11ULL,0x9afe153ad6cb3b8eULL,0x4d1a6dd7ddf800bdULL,0xf6c13cc0caf17e19ULL,0x15f6c58e325fc3eeULL,0x71095400a31dc3b2ULL,0x168e7c07afa3d3e7ULL,0x3f8417a194c7ae2dULL,+	0xec234772813b230dULL,0x634d0f5f17344427ULL,0x11548ab1d77fc56aULL,0x7fab1750ce06af77ULL,0xb62c10a74f7c4f83ULL,0xa7d2edc4220a67d9ULL,0x1c404170921209a0ULL,0x0b9815a0face59f0ULL,+	0x2842589b319540c3ULL,0x18490f59a283d6f8ULL,0xa2731f84daae9fcbULL,0x3db6d960c3683ba0ULL,0xc85c63bb14611069ULL,0xb19436af0788bf05ULL,0x905459df347460d2ULL,0x73f6e094e11a7db1ULL,+	0xdc7f938eb6357f37ULL,0xc5d00f792bd8aa62ULL,0xc878dcb92ca979fcULL,0x37e83ed9eb023a99ULL,0x6b23e2731560bf3dULL,0x1086e4591d0fae61ULL,0x782483169a9414bdULL,0x1b956bc0f0ea9ea1ULL,+	0x7b85bb91c31b9c38ULL,0x0c5aa90b48ef57b5ULL,0xdedeb169af3bab6fULL,0xe610ad732d373685ULL,0xf13870df02ba8e15ULL,0x0337edb68ca7f771ULL,0xe4acf747b62c036cULL,0xd921d576b6b94e81ULL,+	0xdbc864392c422f7aULL,0xfb635362ed348898ULL,0x83084668c45bfcd1ULL,0xc357c9e32b315e11ULL,0xb173b5405b2e5b8cULL,0x7e946931e102b9a4ULL,0x17c890eb7b0fb199ULL,0xec225a83d61b662bULL,+	0xf306a3c8ee3c76cbULL,0x3cf11623d32a1f6eULL,0xe6d5ab646863e956ULL,0x3b8a4cbe5c005c26ULL,0xdcd529a59ce6bb27ULL,0xc4afaa5204d4b16fULL,0xb0624a267923798dULL,0x85e56df66b307fabULL,+	0x0281893c2bf29698ULL,0x91fc19a4d7ce7603ULL,0x75a5dca3ad9a558fULL,0x40ceb3fa4d50bf77ULL,0x1baf6060bc9ba369ULL,0x927e1037597888c2ULL,0xd936bf1986a34c07ULL,0xd4cf10c1c34ae980ULL,+	0x3a3e5334859dd614ULL,0x9c475b5b18d0c8eeULL,0x63080d1f07cd51d5ULL,0xc9c0d0a6b88b4326ULL,0x1ac98691c234296fULL,0x2a0a83a494887fb6ULL,0x565114270cea9cf2ULL,0x5230a6e8a24802f5ULL,+	0xf7a2bf0f72e3d5c1ULL,0x377174464f21439eULL,0xfedcbf259ce30334ULL,0xe0030a787ce202f9ULL,0x6f2d9ebf1202e9caULL,0xe79dde6c75e6e591ULL,0xf52072aff1dac4f8ULL,0x6c8d087ebb9b404dULL,+	0xad0fc73dbce913afULL,0x909e587b458a07cbULL,0x1300da84d4f00c8aULL,0x425cd048b54466acULL,0xb59cb9be90e9d8bfULL,0x991616db3e431b0eULL,0xd3aa117a531aecffULL,0x91af92d359f4dc3bULL,+	0x9b1ec292e93fda29ULL,0x76bb6c17e97d91bcULL,0x7509d95faface1e6ULL,0x3653fe47be855ae3ULL,0x73180b280f680e75ULL,0x75eefd1beeb6c26cULL,0xa4cdf29fb66d4236ULL,0x2d70a9976b5821d8ULL,+	0x7a3ee20720445c36ULL,0x71d1ac8259877174ULL,0x0fc539f7949f73e9ULL,0xd05cf3d7982e3081ULL,0x8758e20b7b1c7129ULL,0xffadcc20569e61f2ULL,0xb05d3a2f59544c2dULL,0xbe16f5c19fff5e53ULL,+	0x73cf65b8aad58135ULL,0x622c2119037aa5beULL,0x79373b3f646fd6a0ULL,0x0e029db50d3978cfULL,0x8bdfc43794fba037ULL,0xaefbd687620797a6ULL,0x3fa5382bbd30d38eULL,0x7627cfbf585d7464ULL,+	0xb2330fef4e4ca463ULL,0xbcef72873566cc63ULL,0xd161d2cacf780900ULL,0x135dc5395b54827dULL,0x638f052e27bf1bc6ULL,0x10a224f007dfa06cULL,0xe973586d6d3321daULL,0x8b0c573826152c8fULL,+	0x9910ba6b23a5d896ULL,0x1fe19e357fe4364eULL,0x6e1da8c39a33c677ULL,0x15b4488b29fd9fd0ULL,0x1f4392541a1f22bfULL,0x920a8a70ab8163e8ULL,0x3fd1b24907e5658eULL,0xf2c4f79cb6ec839bULL,+	0x8b5c619c76497ee8ULL,0x5d2b0ac6c717370eULL,0x98204cb64fcf68e1ULL,0x0bdec21162bc6792ULL,0x6973ccefa63b1011ULL,0xf9e3fa97e0de1ac5ULL,0x5efb693e3d0e0c8bULL,0x037248e9d2d4fcb4ULL,+	0xd3694e2ab20364e4ULL,0x62699718e770b20dULL,0x6183291b6ed77d1cULL,0x69aada7f6d6180a5ULL,0x51f9054bf185509bULL,0xfd7e845678701077ULL,0xd6a2308a7fb96d8dULL,0xd53e48d228dc87eeULL,+	0x80802dc91ec34f9eULL,0xd8772d3533810603ULL,0x3f06d66c530cb4f3ULL,0x7be5ed0dc475c129ULL,0xcb9e3c1931e82b10ULL,0xc63d2857c9ff6b4cULL,0xb92118c692a1b45eULL,0x0aec44147285bbcaULL,+	0x37071afbe8316c45ULL,0x982be4fd46700b8bULL,0x8d5d177c64ff8578ULL,0x5ec40582c9a82fa7ULL,0x5518e37bcfa86678ULL,0x24e809f49f031284ULL,0x312f39604bbbb74cULL,0xad4b4f6fc0c14de6ULL,+	0xa3d6f4868eb6e843ULL,0x6415834bac4ab3abULL,0x028a81514f3cf2dcULL,0xf3b4962f5f3e6c3eULL,0x9119ae90987dd2f2ULL,0x437a6d8ae10bce55ULL,0xc31cdd6b9b149ed6ULL,0x1b77791d06871332ULL,+	0x9c34b650e16e05e9ULL,0x965a774094e74640ULL,0xa3fd22fbcea3f029ULL,0x1eb6a9688f95277cULL,0x2520a63d7bad84f6ULL,0xad917201f58f2feeULL,0xea92c1669b840d48ULL,0x12109c4aacef5cbdULL,+	0xfc189ae71e29a3efULL,0xcbe906f04c93302eULL,0xd0107914ceaae10eULL,0xb7a23f34b68e19f8ULL,0xe9d875c2efd2119dULL,0x03198c6efcadc9c8ULL,0x65591bf64da17113ULL,0x3cf0bbf83d443038ULL,+	0xab293027aad991c1ULL,0x598d0bf8849be4b7ULL,0x8c94a21ab972da90ULL,0xada4cfdd7ecfa840ULL,0x93d4b9c0fbcec63aULL,0x7ca617a203219a34ULL,0x900424eb6a652a55ULL,0xaf9346e9eb8562e0ULL,+	0xc3e04d7902381461ULL,0xb1643ab5911bc478ULL,0xc92becfa390b3ef2ULL,0x54476778acd2f1b6ULL,0x8daa0c4d66bf3aafULL,0x2bc1287b2c21c65aULL,0xee182910b5a13ac3ULL,0xbb04730090b0790aULL,+	0x83766947d17d4e0bULL,0xc5772beefdc3a47bULL,0x765a50db1a6fd0ffULL,0x17f904ba45b0995eULL,0xcee643832883487eULL,0xf56db7f3c270aaedULL,0x6738d94f46cb1fd9ULL,0xc8fa426a142fd4d5ULL,+	0xae485bb72b724759ULL,0x945353e1b2d4c63aULL,0x82159d07de7d6f2cULL,0x389caef34ec5b109ULL,0x4a8ebb53db65ef14ULL,0x2dc2cb7edd99de43ULL,0x816fa3ed83f2405fULL,0x73429bb9c14208a3ULL,+	0x08ed8febd56daf06ULL,0x8d98277b4a837f69ULL,0x9947c636a9b6e05aULL,0x58c8a77ac0d58abdULL,0xf45496a45f121e4fULL,0x16cd67c71076d3d3ULL,0xecbd1958e3fb0c5dULL,0xfbe185ec38e1eb47ULL,+	0xb7ef9760ff79d2eeULL,0xdd4d06aff39e7832ULL,0xfd025001b905b499ULL,0x98fe1c61f5b61d31ULL,0xa9f83980c5f12805ULL,0x376e3b783009cd9aULL,0xa322b09f514eb16dULL,0x08e213122c3832dfULL,+	0xaea68626dc4ad4f4ULL,0x5dc516824ddbc0b6ULL,0xa76697bd602e9065ULL,0xbeeb3ea58c37888eULL,0x1ec4a2f214569113ULL,0xe48b820ca35f4484ULL,0x9fb560949ae44df2ULL,0x6ca1346292cc09fdULL,+	0xb618d590b01e6e27ULL,0x047e2ccde180b2dcULL,0xd1b299b504aea4a9ULL,0x412c9e1e9fa403a4ULL,0x88d28a3679407552ULL,0x49c50136f332b8e3ULL,0x3a1b6fcce668de19ULL,0x178851bc75122b97ULL,+	0x197dd46d95a7b1a2ULL,0x9c4e7ad63c6341fbULL,0x426eca29484c2eceULL,0x9211e489de7f4f8aULL,0x14997f6ec78ef1f4ULL,0x2b2c091006574586ULL,0x17286a6e1c3eede8ULL,0x25f92e470f60e018ULL,+	0xb4e370af3aeac968ULL,0xe4f7fee9c4b63266ULL,0xb4acd4c2e3ac5664ULL,0xf8910bd2ceb38cbfULL,0x1c3ae50cc9c0726eULL,0x15309569d97b40bfULL,0x70884b7ffd5a5a1bULL,0x3890896aef8314cdULL,+	0x090d7d205ffe7b37ULL,0x3b7f3efb1747d2daULL,0xa2cb525fb54fc519ULL,0x6e220932f66a971eULL,0xddc160dfb486d440ULL,0x7fcfec463fe13465ULL,0x83da7e4e76e4c151ULL,0xd6fa48a1d8d302b5ULL,+	0x5ced3c9f82e4c634ULL,0x8efb83143a4464f8ULL,0xe706381b7a1dca25ULL,0x6cd15a3c5a2a412bULL,0x9347a8fdbfcd8fb5ULL,0x31db2eef6e54cd22ULL,0xc4aeb11ef8d8932fULL,0x11e7c1ed344411afULL,+	0xae4065ef12045cf9ULL,0x6fcb2caf9ccce8bdULL,0x1fa0ba4ef2cf6525ULL,0xf683125dcb72c312ULL,0xa01da4eae312410eULL,0x67e286776cd8e830ULL,0xabd9575298fb3f07ULL,0x05f11e11eef649a5ULL,+	0x996884f5903fa271ULL,0xe6da0fd2b9da921eULL,0xa6f2f2695db01e54ULL,0x1ee3e9bd6876214eULL,0xa26e181ce27a9497ULL,0x36d254e48e215e04ULL,0x42f32a6c252cabcaULL,0x9948148780b57614ULL,+	0xea961058fa28a7e0ULL,0xc726cf250bf5ec74ULL,0xe74d55c8db229666ULL,0x0bd9abbfa57f5799ULL,0x7479ef074dfc47b3ULL,0xd9c65fc30c52f91dULL,0x8e0283fe36a8bde2ULL,0xa32a8b5e7d4b7280ULL,+	0xab41b43a43228d83ULL,0x24ae1c304ad63f99ULL,0x8e525f1a46a51229ULL,0x14af860fcd26d2b4ULL,0xd6baef613f714aa1ULL,0xf51865adeb78795eULL,0xd3e21fcee6a9d694ULL,0x82ceb1dd8a37b527ULL,+	0x8605d27d48307682ULL,0x745aaba3e10566daULL,0xe57cae36bff2d7abULL,0x91332ba14b127823ULL,0xcb5c3638f3429f43ULL,0x43a21c4fec462929ULL,0xe9bc95352b18b7bdULL,0x8c2addf3e78cb0c6ULL,+	0x4a665bfd2f9fd51aULL,0x7f2f1fe2481b97f7ULL,0xcad05d69ad36ce50ULL,0x314fc2a4844f4dedULL,0xd5593d8cb55fc5c6ULL,0xe3510ce8bfb1e23dULL,0xf9b7be6937453cceULL,0xd3541b7969fae631ULL,+	0xb013cbcad7154cedULL,0x949b28573f52651aULL,0x03b41f6e9f5e642eULL,0xc3a3462986ed94a4ULL,0xf3c86cd6222b24dcULL,0x7578fe8a028c9c26ULL,0xaac7bfa12edad3b6ULL,0x4112c5d78847940aULL,+	0x99296525eca445dfULL,0xf1af24f22cdfa4c6ULL,0xf5b4eb61eba6d3bcULL,0x4560910c98972cc7ULL,0x54751c32093eaa32ULL,0x018313497d3c67bbULL,0x3bd90ce62d871110ULL,0x75fc863a538baa7eULL,+	0x8fe1f8b64ae3a278ULL,0x160c5306137cdf65ULL,0x22f029e733be0492ULL,0x79a680427a75cd82ULL,0x1d8c094a5b3f3adaULL,0x5d723bbe165c3250ULL,0x08b958ffa5792e22ULL,0x829fa986b11c1eaaULL,+	0x711b8a4176a9f05dULL,0x06ca4e4b9011d488ULL,0x543bc62ba248a65eULL,0x017535ffc9290894ULL,0x840b84ce406851d7ULL,0xafa3acdf90e960b4ULL,0xac3394af7128fd34ULL,0x54eb4d5b2ac0f92cULL,+	0x923ea73e4a14f836ULL,0xc8cc8c57f0946328ULL,0xce3f117fe917a4dfULL,0x6372f933f72ce929ULL,0x75000249c29ba567ULL,0xcbd437e68c829ccaULL,0x6c63aaabdd02ec7aULL,0xe9b2f90c7b42bd17ULL,+	0xdb09e87355dbd4b3ULL,0x1f8799286639bbb1ULL,0xb83e47e51c651962ULL,0xd4ef0fb6c43fb574ULL,0x27d3b9d8f1bfb12aULL,0x6ab877e86e5e8b72ULL,0x8eebdc9d157b9014ULL,0x4c2110053aa5cb64ULL,+};
cbits/p256/p256_ec.c view
@@ -33,8 +33,18 @@ // See http://www.imperialviolet.org/2010/12/04/ecc.html ([1]) for background.  #include "p256/p256_gf.h"+#include "crypton_bzero.h" +#ifdef CRYPTON_S2N_BIGNUM+#include "p256/p256_s2n.h"+#include "p256/p256_verify.h"+/* the memcpy below is the two representations being the same thing */+#if P256_BITSPERDIGIT != 64 || P256_NDIGITS != 4+#error "CRYPTON_S2N_BIGNUM wants the 64-bit crypton_p256_int"+#endif+#endif + /* Field element operations: */  /* felem_inv calculates |out| = |in|^{-1}@@ -43,69 +53,95 @@  *   a^p = a (mod p)  *   a^{p-1} = 1 (mod p)  *   a^{p-2} = a^{-1} (mod p)- */+ *+ * The exponent is built left to right from the shape of p - 2, which for+ * this prime is+ *+ *   ffffffff 00000001 00000000 00000000 00000000 ffffffff ffffffff fffffffd+ *   \__32 ones__/ \_31 zeros, one 1_/ \______ 96 zeros ______/ \_94 ones, 0, 1_/+ *+ * A run of k zeros is k squarings; a run of k ones is k squarings and one+ * multiplication by a^(2^k - 1), which is why the powers below are kept.  The+ * whole chain is 255 squarings, which is the least an exponent of 256 bits+ * can be done in, and 13 multiplications.+ *+ * The chain this replaces built the low 94 ones in a second accumulator and+ * multiplied the two at the end, which cost 32 squarings more than the 255. */ static void felem_inv(felem out, const felem in) {-  felem ftmp, ftmp2;-  /* each e_I will hold |in|^{2^I - 1} */-  felem e2, e4, e8, e16, e32, e64;+  felem ftmp, x2, x4, x8, x16, x32;   unsigned i; -  felem_square(ftmp, in); /* 2^1 */-  felem_mul(ftmp, in, ftmp); /* 2^2 - 2^0 */-  felem_assign(e2, ftmp);-  felem_square(ftmp, ftmp); /* 2^3 - 2^1 */-  felem_square(ftmp, ftmp); /* 2^4 - 2^2 */-  felem_mul(ftmp, ftmp, e2); /* 2^4 - 2^0 */-  felem_assign(e4, ftmp);-  felem_square(ftmp, ftmp); /* 2^5 - 2^1 */-  felem_square(ftmp, ftmp); /* 2^6 - 2^2 */-  felem_square(ftmp, ftmp); /* 2^7 - 2^3 */-  felem_square(ftmp, ftmp); /* 2^8 - 2^4 */-  felem_mul(ftmp, ftmp, e4); /* 2^8 - 2^0 */-  felem_assign(e8, ftmp);+  /* x{k} holds in^(2^k - 1), a run of k ones. */+  felem_square(ftmp, in);+  felem_mul(x2, ftmp, in); /* 2^2 - 1 */++  felem_square(ftmp, x2);+  felem_square(ftmp, ftmp);+  felem_mul(x4, ftmp, x2); /* 2^4 - 1 */++  felem_assign(ftmp, x4);+  for (i = 0; i < 4; i++) {+    felem_square(ftmp, ftmp);+  }+  felem_mul(x8, ftmp, x4); /* 2^8 - 1 */++  felem_assign(ftmp, x8);   for (i = 0; i < 8; i++) {     felem_square(ftmp, ftmp);-  } /* 2^16 - 2^8 */-  felem_mul(ftmp, ftmp, e8); /* 2^16 - 2^0 */-  felem_assign(e16, ftmp);+  }+  felem_mul(x16, ftmp, x8); /* 2^16 - 1 */++  felem_assign(ftmp, x16);   for (i = 0; i < 16; i++) {     felem_square(ftmp, ftmp);-  } /* 2^32 - 2^16 */-  felem_mul(ftmp, ftmp, e16); /* 2^32 - 2^0 */-  felem_assign(e32, ftmp);+  }+  felem_mul(x32, ftmp, x16); /* 2^32 - 1 */++  /* The top 32 ones. */+  felem_assign(ftmp, x32);++  /* 31 zeros and a one: the 00000001 word. */   for (i = 0; i < 32; i++) {     felem_square(ftmp, ftmp);-  } /* 2^64 - 2^32 */-  felem_assign(e64, ftmp);-  felem_mul(ftmp, ftmp, in); /* 2^64 - 2^32 + 2^0 */-  for (i = 0; i < 192; i++) {+  }+  felem_mul(ftmp, ftmp, in);++  /* 96 zeros. */+  for (i = 0; i < 96; i++) {     felem_square(ftmp, ftmp);-  } /* 2^256 - 2^224 + 2^192 */+  } -  felem_mul(ftmp2, e64, e32); /* 2^64 - 2^0 */+  /* 94 ones, as 32 + 32 + 16 + 8 + 4 + 2. */+  for (i = 0; i < 32; i++) {+    felem_square(ftmp, ftmp);+  }+  felem_mul(ftmp, ftmp, x32);+  for (i = 0; i < 32; i++) {+    felem_square(ftmp, ftmp);+  }+  felem_mul(ftmp, ftmp, x32);   for (i = 0; i < 16; i++) {-    felem_square(ftmp2, ftmp2);-  } /* 2^80 - 2^16 */-  felem_mul(ftmp2, ftmp2, e16); /* 2^80 - 2^0 */+    felem_square(ftmp, ftmp);+  }+  felem_mul(ftmp, ftmp, x16);   for (i = 0; i < 8; i++) {-    felem_square(ftmp2, ftmp2);-  } /* 2^88 - 2^8 */-  felem_mul(ftmp2, ftmp2, e8); /* 2^88 - 2^0 */+    felem_square(ftmp, ftmp);+  }+  felem_mul(ftmp, ftmp, x8);   for (i = 0; i < 4; i++) {-    felem_square(ftmp2, ftmp2);-  } /* 2^92 - 2^4 */-  felem_mul(ftmp2, ftmp2, e4); /* 2^92 - 2^0 */-  felem_square(ftmp2, ftmp2); /* 2^93 - 2^1 */-  felem_square(ftmp2, ftmp2); /* 2^94 - 2^2 */-  felem_mul(ftmp2, ftmp2, e2); /* 2^94 - 2^0 */-  felem_square(ftmp2, ftmp2); /* 2^95 - 2^1 */-  felem_square(ftmp2, ftmp2); /* 2^96 - 2^2 */-  felem_mul(ftmp2, ftmp2, in); /* 2^96 - 3 */+    felem_square(ftmp, ftmp);+  }+  felem_mul(ftmp, ftmp, x4);+  felem_square(ftmp, ftmp);+  felem_square(ftmp, ftmp);+  felem_mul(ftmp, ftmp, x2); -  felem_mul(out, ftmp2, ftmp); /* 2^256 - 2^224 + 2^192 + 2^96 - 3 */+  /* A zero and a one: the d of fffffffd. */+  felem_square(ftmp, ftmp);+  felem_square(ftmp, ftmp);+  felem_mul(out, ftmp, in); } - /* Group operations:  *  * Elements of the elliptic curve group are represented in Jacobian@@ -298,7 +334,8 @@ }  /* select_affine_point sets {out_x,out_y} to the index'th entry of table.- * On entry: index < 16, table[0] must be zero. */+ * On entry: index < 16.  Every entry is a point of its own -- the signed+ * representation has no zero digit -- so the scan starts at zero. */ static void select_affine_point(felem out_x, felem out_y, const limb* table,                                 limb index) {   limb i, j;@@ -306,7 +343,7 @@   memset(out_x, 0, sizeof(felem));   memset(out_y, 0, sizeof(felem)); -  for (i = 1; i < 16; i++) {+  for (i = 0; i < 16; i++) {     limb mask = i ^ index;     mask |= mask >> 2;     mask |= mask >> 1;@@ -321,95 +358,306 @@   } } -/* select_jacobian_point sets {out_x,out_y,out_z} to the index'th entry of- * table. On entry: index < 16, table[0] must be zero. */-static void select_jacobian_point(felem out_x, felem out_y, felem out_z,-                                  const limb* table, limb index) {-  limb i, j;+/* words_are_zero returns 1 when |v| is zero and 0 otherwise, without a+ * branch. */+static u32 words_are_zero(u32 v) {+  v |= v >> 16;+  v |= v >> 8;+  v |= v >> 4;+  v |= v >> 2;+  v |= v >> 1;+  return (v & 1) ^ 1;+} -  memset(out_x, 0, sizeof(felem));-  memset(out_y, 0, sizeof(felem));-  memset(out_z, 0, sizeof(felem));+/* The comb: five teeth to a block, the teeth of a block 52 apart and the two+ * blocks 26 from each other, so 26 steps cover all 260 bits between them.+ *+ *   first block    i, 52+i, 104+i, 156+i, 208+i+ *   second block   26+i, 78+i, 130+i, 182+i, 234+i+ *+ * Five teeth would want a table of 32, but the signed representation makes+ * every digit +-1, so the thirty-two values come in pairs that differ by sign+ * and sixteen entries serve: kPrecomputed holds the ones whose top tooth is+ * positive and the other sign is a negated y.  That is the whole of the gain+ * over the four-tooth unsigned comb this replaces, which took 32 steps for+ * the same 256 bits: 25 doublings and 52 mixed additions against 31 and 64.+ */+#define COMB_TEETH 5+#define COMB_STEPS 26+#define COMB_SPAN (2 * COMB_STEPS) /* 52, the distance between a block's teeth */+#define COMB_WORDS 9               /* 260 bits of signs, with room to add into */+#define COMB_BIT(t, q) (((t)[(q) >> 5] >> ((q) & 31)) & 1) -  /* The implicit value at index 0 is all zero. We don't need to perform that-   * iteration of the loop because we already set out_* to zero. */-  table += 3 * NLIMBS;+/* comb_recode writes into |out| the value whose bits are the signs of the+ * scalar's all-bits-set representation: digit j is +1 where bit j of |out| is+ * set and -1 where it is not.+ *+ * For an odd k below 2^260 there is exactly one such representation, and it+ * is (k + 2^260 - 1) / 2 read as bits -- an addition and a shift, and that is+ * all the recoding is.  An even scalar has the order added to make it odd,+ * which changes the scalar and not the point it selects.+ *+ * Constant time in the scalar: every branch below is on a loop counter. */+static void comb_recode(u32 out[COMB_WORDS],+                        const crypton_p256_int* scalar) {+  u32 k[COMB_WORDS], ord[COMB_WORDS];+  u64 carry;+  int i; -  // Hit all entries to obscure cache profiling.-  for (i = 1; i < 16; i++) {-    limb mask = i ^ index;-    mask |= mask >> 2;-    mask |= mask >> 1;-    mask &= 1;-    mask--;-    for (j = 0; j < NLIMBS; j++, table++) {-      out_x[j] |= *table & mask;-    }-    for (j = 0; j < NLIMBS; j++, table++) {-      out_y[j] |= *table & mask;-    }-    for (j = 0; j < NLIMBS; j++, table++) {-      out_z[j] |= *table & mask;+  for (i = 0; i < COMB_WORDS; i++) {+    k[i] = 0;+    ord[i] = 0;+  }+  for (i = 0; i < 256; i += 32) {+    k[i >> 5] = (u32)(P256_DIGIT(scalar, i / P256_BITSPERDIGIT)+                      >> (i % P256_BITSPERDIGIT));+    ord[i >> 5] = (u32)(P256_DIGIT(&crypton_SECP256r1_n, i / P256_BITSPERDIGIT)+                        >> (i % P256_BITSPERDIGIT));+  }++  /* an even scalar becomes odd by taking on the order */+  {+    u32 addmask = (u32)0 - (u32)((k[0] & 1) ^ 1);++    carry = 0;+    for (i = 0; i < COMB_WORDS; i++) {+      u64 v = (u64)k[i] + (u64)(ord[i] & addmask) + carry;+      k[i] = (u32)v;+      carry = v >> 32;     }   }++  /* out = (k + 2^260 - 1) >> 1 */+  carry = 0;+  for (i = 0; i < COMB_WORDS; i++) {+    u64 v = (u64)k[i] + (u64)(i < 8 ? 0xffffffffu : 0xfu) + carry;+    out[i] = (u32)v;+    carry = v >> 32;+  }+  for (i = 0; i < COMB_WORDS - 1; i++) {+    out[i] = (out[i] >> 1) | (out[i + 1] << 31);+  }+  out[COMB_WORDS - 1] >>= 1;+ } +/* point_add_complete_mixed sets {x3,y3,z3} = {x1,y1,z1} + {x2,y2}, where the+ * accumulator is in projective coordinates and the added point is affine.+ *+ * This is Renes-Costello-Batina algorithm 5, for a curve with a = -3.  It is+ * complete: it is right when the two points are the same, when either is the+ * point at infinity, and when they are each other's negation, which is what+ * point_add_mixed cannot say.  It costs 11 multiplications and two by b,+ * against point_add_mixed's 8 multiplications and 3 squarings.+ *+ * The table this comb reads is affine, and an affine point has Z = 1, so+ * Z = Z^2 = Z^3 and the same three numbers are the point in projective+ * coordinates and in Jacobian ones.  That is what lets a comb built on+ * Jacobian arithmetic step into this formula for one addition and back out.+ */+static void point_add_complete_mixed(felem x3, felem y3, felem z3,+                                     const felem x1, const felem y1,+                                     const felem z1, const felem x2,+                                     const felem y2) {+  felem t0, t1, t2, t3, t4, xx, yy, zz;++  felem_mul(t0, x1, x2);+  felem_mul(t1, y1, y2);+  felem_sum(t3, x2, y2);+  felem_sum(t4, x1, y1);+  felem_mul(t3, t3, t4);+  felem_sum(t4, t0, t1);+  felem_diff(t3, t3, t4);+  felem_mul(t4, y2, z1);+  felem_sum(t4, t4, y1);+  felem_mul(yy, x2, z1);+  felem_sum(yy, yy, x1);+  felem_mul(zz, kB, z1);+  felem_diff(xx, yy, zz);+  felem_sum(zz, xx, xx);+  felem_sum(xx, xx, zz);+  felem_diff(zz, t1, xx);+  felem_sum(xx, t1, xx);+  felem_mul(yy, kB, yy);+  felem_sum(t1, z1, z1);+  felem_sum(t2, t1, z1);+  felem_diff(yy, yy, t2);+  felem_diff(yy, yy, t0);+  felem_sum(t1, yy, yy);+  felem_sum(yy, t1, yy);+  felem_sum(t1, t0, t0);+  felem_sum(t0, t1, t0);+  felem_diff(t0, t0, t2);+  felem_mul(t1, t4, yy);+  felem_mul(t2, t0, yy);+  felem_mul(yy, xx, zz);+  felem_sum(y3, yy, t2);+  felem_mul(xx, t3, xx);+  felem_diff(x3, xx, t1);+  felem_mul(zz, t4, zz);+  felem_mul(t1, t3, t0);+  felem_sum(z3, zz, t1);+}++/* point_add_complete sets {x3,y3,z3} = {x1,y1,z1} + {x2,y2,z2}, both in+ * projective coordinates.+ *+ * Renes-Costello-Batina algorithm 4, for a = -3, and complete for the same+ * reasons as the mixed one above.  It costs 12 multiplications and two by b,+ * against point_add's 11 multiplications and 5 squarings.+ */+static void point_add_complete(felem x3, felem y3, felem z3, const felem x1,+                               const felem y1, const felem z1, const felem x2,+                               const felem y2, const felem z2) {+  felem t0, t1, t2, t3, t4, xx, yy, zz;++  felem_mul(t0, x1, x2);+  felem_mul(t1, y1, y2);+  felem_mul(t2, z1, z2);+  felem_sum(t3, x1, y1);+  felem_sum(t4, x2, y2);+  felem_mul(t3, t3, t4);+  felem_sum(t4, t0, t1);+  felem_diff(t3, t3, t4);+  felem_sum(t4, y1, z1);+  felem_sum(xx, y2, z2);+  felem_mul(t4, t4, xx);+  felem_sum(xx, t1, t2);+  felem_diff(t4, t4, xx);+  felem_sum(xx, x1, z1);+  felem_sum(yy, x2, z2);+  felem_mul(xx, xx, yy);+  felem_sum(yy, t0, t2);+  felem_diff(yy, xx, yy);+  felem_mul(zz, kB, t2);+  felem_diff(xx, yy, zz);+  felem_sum(zz, xx, xx);+  felem_sum(xx, xx, zz);+  felem_diff(zz, t1, xx);+  felem_sum(xx, t1, xx);+  felem_mul(yy, kB, yy);+  felem_sum(t1, t2, t2);+  felem_sum(t2, t1, t2);+  felem_diff(yy, yy, t2);+  felem_diff(yy, yy, t0);+  felem_sum(t1, yy, yy);+  felem_sum(yy, t1, yy);+  felem_sum(t1, t0, t0);+  felem_sum(t0, t1, t0);+  felem_diff(t0, t0, t2);+  felem_mul(t1, t4, yy);+  felem_mul(t2, t0, yy);+  felem_mul(yy, xx, zz);+  felem_sum(y3, yy, t2);+  felem_mul(xx, t3, xx);+  felem_diff(x3, xx, t1);+  felem_mul(zz, t4, zz);+  felem_mul(t1, t3, t0);+  felem_sum(z3, zz, t1);+}++/* jacobian_to_projective sets {x2,y2,z2} to the projective form of the+ * Jacobian point {x1,y1,z1}: (X/Z^2, Y/Z^3) is (XZ : Y : Z^3). */+static void jacobian_to_projective(felem x2, felem y2, felem z2,+                                   const felem x1, const felem y1,+                                   const felem z1) {+  felem zz, zzz;++  felem_square(zz, z1);+  felem_mul(zzz, zz, z1);+  felem_mul(x2, x1, z1);+  memcpy(y2, y1, sizeof(felem));+  memcpy(z2, zzz, sizeof(felem));+}++/* projective_to_jacobian is the other way: (X/Z) is (XZ : YZ^2 : Z). */+static void projective_to_jacobian(felem x2, felem y2, felem z2,+                                   const felem x1, const felem y1,+                                   const felem z1) {+  felem zz;++  felem_square(zz, z1);+  felem_mul(x2, x1, z1);+  felem_mul(y2, y1, zz);+  memcpy(z2, z1, sizeof(felem));+}+ /* scalar_base_mult sets {nx,ny,nz} = scalar*G where scalar is a little-endian  * number. Note that the value of scalar must be less than the order of the  * group. */ static void scalar_base_mult(felem nx, felem ny, felem nz,                              const crypton_p256_int* scalar) {-  int i, j;-  limb n_is_infinity_mask = -1, p_is_noninfinite_mask, mask;-  u32 table_offset;+  u32 rec[COMB_WORDS];+  limb n_is_infinity_mask = -1, mask;+  felem px, py, negy, tx, ty, tz, jx, jy, jz, cx, cy, cz;+  int i, blk, j; -  felem px, py;-  felem tx, ty, tz;+  comb_recode(rec, scalar);    memset(nx, 0, sizeof(felem));   memset(ny, 0, sizeof(felem));   memset(nz, 0, sizeof(felem)); -  /* The loop adds bits at positions 0, 64, 128 and 192, followed by-   * positions 32,96,160 and 224 and does this 32 times. */-  for (i = 0; i < 32; i++) {-    if (i) {+  for (i = COMB_STEPS - 1; i >= 0; i--) {+    if (i != COMB_STEPS - 1) {       point_double(nx, ny, nz, nx, ny, nz);     }-    table_offset = 0;-    for (j = 0; j <= 32; j += 32) {-      char bit0 = crypton_p256_get_bit(scalar, 31 - i + j);-      char bit1 = crypton_p256_get_bit(scalar, 95 - i + j);-      char bit2 = crypton_p256_get_bit(scalar, 159 - i + j);-      char bit3 = crypton_p256_get_bit(scalar, 223 - i + j);-      limb index = bit0 | (bit1 << 1) | (bit2 << 2) | (bit3 << 3); -      select_affine_point(px, py, kPrecomputed + table_offset, index);-      table_offset += 30 * NLIMBS;+    for (blk = 0; blk < 2; blk++) {+      u32 base = (u32)(blk * COMB_STEPS + i);+      u32 top = COMB_BIT(rec, base + (COMB_TEETH - 1) * COMB_SPAN);+      limb index = 0; -      /* Since scalar is less than the order of the group, we know that-       * {nx,ny,nz} != {px,py,1}, unless both are zero, which we handle-       * below. */-      point_add_mixed(tx, ty, tz, nx, ny, nz, px, py);-      /* The result of point_add_mixed is incorrect if {nx,ny,nz} is zero-       * (a.k.a.  the point at infinity). We handle that situation by-       * copying the point from the table. */+      for (j = 0; j < COMB_TEETH - 1; j++) {+        /* a tooth agreeing with the top one is a set bit of the index */+        index |= (limb)(COMB_BIT(rec, base + (u32)j * COMB_SPAN) ^ top ^ 1)+                 << j;+      }++      select_affine_point(px, py, kPrecomputed + blk * 16 * 2 * NLIMBS, index);+      felem_diff(negy, kZero, py);+      copy_conditional(py, negy, (limb)0 - (limb)(top ^ 1));++      /* The last addition is the one that can be a point added to itself:+       * entering it the accumulator is (k' - B)*G and what it adds is B*G,+       * where B is the second block's digit at step zero, so the two meet+       * when k' = 2B modulo the order.  One scalar below the order does+       * that, and point_add_mixed cannot answer it.+       *+       * So that one addition goes through the complete formula instead.+       * The table is affine, so the point just selected is the same three+       * numbers read as projective coordinates as read as Jacobian ones --+       * which is what lets a comb built on Jacobian arithmetic step into+       * the formula for an addition and back out of it.  The accumulator+       * is not affine and is converted.+       *+       * Measured on an M4, thread CPU time, the whole base point+       * multiplication is 17.34 us this way against 17.30 us with an extra+       * doubling and a mask, which is inside the spread of either. */+      if (i == 0 && blk == 1) {+        jacobian_to_projective(jx, jy, jz, nx, ny, nz);+        point_add_complete_mixed(cx, cy, cz, jx, jy, jz, px, py);+        projective_to_jacobian(tx, ty, tz, cx, cy, cz);+      } else {+        point_add_mixed(tx, ty, tz, nx, ny, nz, px, py);+      }++      /* The accumulator is the infinity until the first of these, and+       * point_add_mixed cannot start from it; the point itself is the sum. */       copy_conditional(nx, px, n_is_infinity_mask);       copy_conditional(ny, py, n_is_infinity_mask);       copy_conditional(nz, kOne, n_is_infinity_mask);--      /* Equally, the result is also wrong if the point from the table is-       * zero, which happens when the index is zero. We handle that by-       * only copying from {tx,ty,tz} to {nx,ny,nz} if index != 0. */-      p_is_noninfinite_mask = NON_ZERO_TO_ALL_ONES(index);-      mask = p_is_noninfinite_mask & ~n_is_infinity_mask;+      mask = ~n_is_infinity_mask;       copy_conditional(nx, tx, mask);       copy_conditional(ny, ty, mask);       copy_conditional(nz, tz, mask);-      /* If p was not zero, then n is now non-zero. */-      n_is_infinity_mask &= ~p_is_noninfinite_mask;+      n_is_infinity_mask = 0;     }   }++  /* The recoded scalar is the private key in another representation, so it+   * does not stay on the stack. */+  crypton_bzero(rec, sizeof(rec)); }  /* point_to_affine converts a Jacobian point to an affine point. If the input@@ -424,67 +672,309 @@   felem_mul(y_out, ny, z_inv); } -/* scalar_base_mult sets {nx,ny,nz} = scalar*{x,y}. */-static void scalar_mult(felem nx, felem ny, felem nz, const felem x,-                        const felem y, const crypton_p256_int* scalar) {-  int i;-  felem px, py, pz, tx, ty, tz;-  felem precomp[16][3];-  limb n_is_infinity_mask, index, p_is_noninfinite_mask, mask;+/* point_add_mixed_pm sets {xp,yp,zp} = {x1,y1,z1} + {x2,y2} and+ * {xm,ym,zm} = {x1,y1,z1} - {x2,y2}, where {x2,y2} is affine.+ *+ * Negating the second point changes the sign of s2 and so of r, and nothing+ * else: z1z1, tmp, u2, z1z1z1, h, i, j, v, the output z and the product y1*j+ * are common to the two.  What the second point costs over the first is one+ * squaring (r*r) and one multiplication (by r), rather than another eleven.+ *+ * The same restrictions as point_add_mixed: this does not handle P+P,+ * infinity+P nor P+infinity. */+static void point_add_mixed_pm(felem xp, felem yp, felem zp,+                               felem xm, felem ym, felem zm,+                               const felem x1, const felem y1, const felem z1,+                               const felem x2, const felem y2) {+  felem z1z1, z1z1z1, s2, u2, h, i, j, r, rr, v, y1j, tmp; -  /* We precompute 0,1,2,... times {x,y}. */-  memset(precomp, 0, sizeof(felem) * 3);-  memcpy(&precomp[1][0], x, sizeof(felem));-  memcpy(&precomp[1][1], y, sizeof(felem));-  memcpy(&precomp[1][2], kOne, sizeof(felem));+  felem_square(z1z1, z1);+  felem_sum(tmp, z1, z1); -  for (i = 2; i < 16; i += 2) {-    point_double(precomp[i][0], precomp[i][1], precomp[i][2],-                 precomp[i / 2][0], precomp[i / 2][1], precomp[i / 2][2]);+  felem_mul(u2, x2, z1z1);+  felem_mul(z1z1z1, z1, z1z1);+  felem_mul(s2, y2, z1z1z1);+  felem_diff(h, u2, x1);+  felem_sum(i, h, h);+  felem_square(i, i);+  felem_mul(j, h, i);+  felem_mul(v, x1, i);+  felem_mul(y1j, y1, j); -    point_add_mixed(precomp[i + 1][0], precomp[i + 1][1], precomp[i + 1][2],-                    precomp[i][0], precomp[i][1], precomp[i][2], x, y);+  /* The two points share their z. */+  felem_mul(zp, tmp, h);+  felem_assign(zm, zp);++  /* X + P */+  felem_diff(r, s2, y1);+  felem_sum(r, r, r);+  felem_square(rr, r);+  felem_diff(xp, rr, j);+  felem_diff(xp, xp, v);+  felem_diff(xp, xp, v);+  felem_diff(tmp, v, xp);+  felem_mul(yp, tmp, r);+  felem_diff(yp, yp, y1j);+  felem_diff(yp, yp, y1j);++  /* X - P.  Negating the point negates s2, so r becomes -q where+   * q = 2*(s2 + y1).  The square is the same either way, and the sign is+   * carried into y by taking (xm - v) where the other took (v - xp):+   *   xm = q^2 - j - 2v+   *   ym = (v - xm)*(-q) - 2*y1*j = (xm - v)*q - 2*y1*j+   * so no field negation is needed. */+  felem_sum(r, s2, y1);+  felem_sum(r, r, r);+  felem_square(rr, r);+  felem_diff(xm, rr, j);+  felem_diff(xm, xm, v);+  felem_diff(xm, xm, v);+  felem_diff(tmp, xm, v);+  felem_mul(ym, tmp, r);+  felem_diff(ym, ym, y1j);+  felem_diff(ym, ym, y1j);+}++/* select_jacobian_odd sets {out_x,out_y,out_z} to the index'th of the 16+ * entries of table, for index < 16.  There is no implicit infinity at index+ * zero, as the unsigned window this replaces had: every entry is a real+ * point, which is what lets the signed representation below do without the+ * infinity masks. */+static void select_jacobian_odd(felem out_x, felem out_y, felem out_z,+                                const limb* table, limb index) {+  limb i, j;++  memset(out_x, 0, sizeof(felem));+  memset(out_y, 0, sizeof(felem));+  memset(out_z, 0, sizeof(felem));++  for (i = 0; i < 16; i++) {+    limb mask = i ^ index;+    mask |= mask >> 2;+    mask |= mask >> 1;+    mask &= 1;+    mask--;+    for (j = 0; j < NLIMBS; j++, table++) {+      out_x[j] |= *table & mask;+    }+    for (j = 0; j < NLIMBS; j++, table++) {+      out_y[j] |= *table & mask;+    }+    for (j = 0; j < NLIMBS; j++, table++) {+      out_z[j] |= *table & mask;+    }   }+} -  memset(nx, 0, sizeof(felem));-  memset(ny, 0, sizeof(felem));-  memset(nz, 0, sizeof(felem));-  n_is_infinity_mask = -1;+/* The scalar, recoded: 52 signed odd digits, each in {+-1,+-3,...,+-31}, so+ * that scalar = sum d_i * 32^i.  A digit is one byte: the low four bits are+ * the table index (|d|-1)/2, and bit four is set when d is negative.  One+ * byte rather than a byte and a word because this is the private key in+ * another form and has to be wiped afterwards. */+#define SABS_DIGITS 52+#define SABS_INDEX(b) ((limb)((b) & 15))+#define SABS_NEGMASK(b) ((limb)0 - (limb)((b) >> 4))+typedef struct {+  u8 digit[SABS_DIGITS];+} sabs_scalar; -  /* We add in a window of four bits each iteration and do this 64 times. */-  for (i = 0; i < 256; i += 4) {-    if (i) {-      point_double(nx, ny, nz, nx, ny, nz);-      point_double(nx, ny, nz, nx, ny, nz);-      point_double(nx, ny, nz, nx, ny, nz);-      point_double(nx, ny, nz, nx, ny, nz);++/* sabs_recode writes the signed representation of |scalar| into |out|.+ *+ * The recoding is the regular one of Joye and Tunstall: take the low six bits,+ * subtract 32, and carry the difference upwards.  It needs an odd input, which+ * is arranged by adding the group order to an even scalar -- that changes the+ * scalar but not the point it selects, the order being the order.  A zero+ * scalar is replaced by one and the caller is told, since zero times a point+ * is the infinity this code deliberately cannot represent.+ *+ * Constant time in the scalar: every branch below is on a loop counter. */+static limb sabs_recode(sabs_scalar* out,+                        const crypton_p256_int* scalar) {+  u32 k[9], n[9];+  u32 nonzero;+  limb is_zero_mask;+  int i, b;++  for (i = 0; i < 9; i++) {+    k[i] = 0;+    n[i] = 0;+  }+  /* A word at a time.  Bit at a time would be 512 calls into another+   * translation unit, which the compiler cannot inline away. */+  for (b = 0; b < 256; b += 32) {+    k[b >> 5] = (u32)(P256_DIGIT(scalar, b / P256_BITSPERDIGIT)+                      >> (b % P256_BITSPERDIGIT));+    n[b >> 5] = (u32)(P256_DIGIT(&crypton_SECP256r1_n, b / P256_BITSPERDIGIT)+                      >> (b % P256_BITSPERDIGIT));+  }++  /* Replace a zero scalar by one, and report it. */+  nonzero = 0;+  for (i = 0; i < 9; i++) {+    nonzero |= k[i];+  }+  {+    u32 z = words_are_zero(nonzero);+    k[0] |= z;+    is_zero_mask = (limb)0 - (limb)z;+  }++  /* An even scalar becomes odd by adding the order.  The sum is below 2^257,+   * which is why nine words and fifty-two digits are enough. */+  {+    u32 addmask = (u32)0 - (u32)((k[0] & 1) ^ 1);+    u64 carry = 0;+    for (i = 0; i < 9; i++) {+      u64 t = (u64)k[i] + (u64)(n[i] & addmask) + carry;+      k[i] = (u32)t;+      carry = t >> 32;     }+  } -    index = (crypton_p256_get_bit(scalar, 255 - i - 0) << 3) |-            (crypton_p256_get_bit(scalar, 255 - i - 1) << 2) |-            (crypton_p256_get_bit(scalar, 255 - i - 2) << 1) |-            crypton_p256_get_bit(scalar, 255 - i - 3);+  for (i = 0; i < SABS_DIGITS - 1; i++) {+    u32 r6 = k[0] & 63;            /* odd, so never 32 */+    u32 hi = (r6 >> 5) & 1;        /* 1 when the digit is positive */+    u32 wabs = ((r6 - 32) & (0u - hi)) | ((32 - r6) & (hi - 1));+    u32 mlo = 32u - r6;            /* two's complement of the digit's negation */+    u32 ext = 0u - hi;             /* its sign extension */+    u64 carry = 0;+    int w; -    /* See the comments in scalar_base_mult about handling infinities. */-    select_jacobian_point(px, py, pz, precomp[0][0], index);-    point_add(tx, ty, tz, nx, ny, nz, px, py, pz);-    copy_conditional(nx, px, n_is_infinity_mask);-    copy_conditional(ny, py, n_is_infinity_mask);-    copy_conditional(nz, pz, n_is_infinity_mask);+    out->digit[i] = (u8)(((wabs - 1) >> 1) | ((hi ^ 1) << 4)); -    p_is_noninfinite_mask = NON_ZERO_TO_ALL_ONES(index);-    mask = p_is_noninfinite_mask & ~n_is_infinity_mask;+    /* k -= digit, i.e. k += -digit, sign extended over the nine words. */+    for (w = 0; w < 9; w++) {+      u64 t = (u64)k[w] + (u64)(w == 0 ? mlo : ext) + carry;+      k[w] = (u32)t;+      carry = t >> 32;+    }+    /* k >>= 5 */+    for (w = 0; w < 8; w++) {+      k[w] = (k[w] >> 5) | (k[w + 1] << 27);+    }+    k[8] >>= 5;+  } -    copy_conditional(nx, tx, mask);-    copy_conditional(ny, ty, mask);-    copy_conditional(nz, tz, mask);-    n_is_infinity_mask &= ~p_is_noninfinite_mask;+  /* What is left is odd, positive and at most five: the scalar is below+   * 2^257 and fifty-one digits have taken 255 bits off it, each leaving a+   * remainder below one. */+  out->digit[SABS_DIGITS - 1] = (u8)((k[0] - 1) >> 1);++  return is_zero_mask;+}++/* scalar_mult sets {nx,ny,nz} = scalar*{x,y}.+ *+ * A five-bit signed window.  The scalar is recoded into 52 digits, every one+ * of them odd and none of them zero, so the table holds only the odd+ * multiples P, 3P, ..., 31P and a negative digit is served by negating y,+ * which is free.  Against the four-bit unsigned window this replaces, the+ * main loop trades 252 doublings and 64 additions for 255 and 51, and --+ * because no digit is zero and no partial sum is the infinity -- it drops the+ * masks that stood in for infinity on every iteration.+ *+ * The table is built so that each pair of neighbouring odd multiples comes+ * out of one doubling and one shared addition:+ *+ *   2P = 2*P                3P  = 2P + P+ *   6P = 2*(3P)             5P  = 6P - P,  7P  = 6P + P+ *   10P = 2*(5P)            9P  = 10P - P, 11P = 10P + P+ *   ...+ *   30P = 2*(15P)           29P = 30P - P, 31P = 30P + P+ *+ * which is eight doublings, one mixed addition and seven shared pairs. */+static void scalar_mult(felem nx, felem ny, felem nz, const felem x,+                        const felem y, const crypton_p256_int* scalar) {+  /* odd[k] is (2k+1)*P, for k in 0..15. */+  felem odd[16][3];+  felem dx, dy, dz, px, py, pz, negy, ddx, ddy, ddz, qx, qy, qz, cx, cy, cz;+  sabs_scalar rec;+  limb is_zero_mask;+  int i, k;++  is_zero_mask = sabs_recode(&rec, scalar);++  felem_assign(odd[0][0], x);+  felem_assign(odd[0][1], y);+  memcpy(odd[0][2], kOne, sizeof(felem));++  /* 3P = 2P + P */+  point_double(dx, dy, dz, x, y, kOne);+  point_add_mixed(odd[1][0], odd[1][1], odd[1][2], dx, dy, dz, x, y);++  /* (4k+2)P from (2k+1)P, then (4k+1)P and (4k+3)P from it. */+  for (k = 1; k < 8; k++) {+    point_double(dx, dy, dz, odd[k][0], odd[k][1], odd[k][2]);+    point_add_mixed_pm(odd[2 * k + 1][0], odd[2 * k + 1][1], odd[2 * k + 1][2],+                       odd[2 * k][0], odd[2 * k][1], odd[2 * k][2],+                       dx, dy, dz, x, y);   }++  /* The top digit initialises the accumulator; it is always positive. */+  select_jacobian_odd(nx, ny, nz, odd[0][0],+                      SABS_INDEX(rec.digit[SABS_DIGITS - 1]));++  for (i = SABS_DIGITS - 2; i >= 0; i--) {+    point_double(nx, ny, nz, nx, ny, nz);+    point_double(nx, ny, nz, nx, ny, nz);+    point_double(nx, ny, nz, nx, ny, nz);+    point_double(nx, ny, nz, nx, ny, nz);+    point_double(nx, ny, nz, nx, ny, nz);++    select_jacobian_odd(px, py, pz, odd[0][0], SABS_INDEX(rec.digit[i]));+    felem_diff(negy, kZero, py);+    copy_conditional(py, negy, SABS_NEGMASK(rec.digit[i]));++    /* On the last step alone the accumulator can be the very point being+     * added -- the accumulator is (k' - d0)*P and it adds d0*P, so the two+     * meet when k' = 2*d0 modulo the order, which the scalar 30 does with a+     * digit of 15 -- and point_add answers the infinity where the truth is+     * twice that point.  That one addition goes through the complete formula+     * instead, with both points converted to projective coordinates and the+     * answer converted back.  One of fifty-one iterations pays for it, and+     * it comes to a field multiplication less than the doubling and the mask+     * it replaces.+     *+     * point_add finishes with z before it touches x, and with each of x and+     * y before the next, so on every other step the accumulator can be its+     * own output. */+    if (i == 0) {+      jacobian_to_projective(ddx, ddy, ddz, nx, ny, nz);+      jacobian_to_projective(qx, qy, qz, px, py, pz);+      point_add_complete(cx, cy, cz, ddx, ddy, ddz, qx, qy, qz);+      projective_to_jacobian(nx, ny, nz, cx, cy, cz);+    } else {+      point_add(nx, ny, nz, nx, ny, nz, px, py, pz);+    }+  }++  /* Zero was replaced by one on the way in; put the infinity back.  All+   * three coordinates, not just z: crypton_p256_points_mul_vartime reads the+   * comment above it as saying the whole point is zero. */+  for (i = 0; i < NLIMBS; i++) {+    nx[i] &= ~is_zero_mask;+    ny[i] &= ~is_zero_mask;+    nz[i] &= ~is_zero_mask;+  }++  /* The recoded scalar is the private key in another representation, so it+   * does not stay on the stack. */+  crypton_bzero(&rec, sizeof(rec)); }  /* crypton_p256_base_point_mul sets {out_x,out_y} = nG, where n is < the  * order of the group. */ void crypton_p256_base_point_mul(const crypton_p256_int* n, crypton_p256_int* out_x, crypton_p256_int* out_y) {+#ifdef CRYPTON_S2N_BIGNUM+  /* The assembly, four times faster, and constant time as this has to be:+   * it is how a public key is derived from a private one. */+  uint64_t res[8];++  crypton_s2n_p256_scalarmulbase(res, P256_DIGITS(n));+  memcpy(P256_DIGITS(out_x), res, P256_NBYTES);+  memcpy(P256_DIGITS(out_y), res + P256_NDIGITS, P256_NBYTES);+#else   felem x, y, z;    scalar_base_mult(x, y, z, n);@@ -496,8 +986,29 @@     from_montgomery(out_x, x_affine);     from_montgomery(out_y, y_affine);   }+#endif } +#ifdef CRYPTON_S2N_BIGNUM+/* An affine point from the assembly as the Jacobian triple the addition+ * below wants, keeping this file's convention that the point at infinity is+ * all three coordinates zero -- the assembly says it with (0, 0). */+static void s2n_lift(felem x, felem y, felem z, const uint64_t res[8]) {+  crypton_p256_int t;+  uint64_t any = 0;+  int i;++  for (i = 0; i < 2 * P256_NDIGITS; i++)+    any |= res[i];++  memcpy(P256_DIGITS(&t), res, P256_NBYTES);+  to_montgomery(x, &t);+  memcpy(P256_DIGITS(&t), res + P256_NDIGITS, P256_NBYTES);+  to_montgomery(y, &t);+  memcpy(z, any != 0 ? kOne : kZero, sizeof(felem));+}+#endif+ /* crypton_p256_points_mul_vartime sets {out_x,out_y} = n1*G + n2*{in_x,in_y}, where  * n1 and n2 are < the order of the group.  *@@ -516,10 +1027,48 @@     return;   } +#ifdef CRYPTON_S2N_BIGNUM+  {+    /* Both scalars at once, in variable time, which is what the two+     * multiplications below are not: they are constant time, and pay for it,+     * over values that are all public here.  It gives up on the one case the+     * assembly's addition does not cover -- adding a point to itself -- and+     * then the constant-time pair answers instead. */+    uint64_t jr[3 * P256_NDIGITS];++    if (crypton_p256_verify_mul(jr, P256_DIGITS(n1), P256_DIGITS(n2),+                                P256_DIGITS(in_x), P256_DIGITS(in_y))) {+      crypton_p256_int t;++      memcpy(P256_DIGITS(&t), jr, P256_NBYTES);+      to_montgomery(x1, &t);+      memcpy(P256_DIGITS(&t), jr + P256_NDIGITS, P256_NBYTES);+      to_montgomery(y1, &t);+      memcpy(P256_DIGITS(&t), jr + 2 * P256_NDIGITS, P256_NBYTES);+      to_montgomery(z1, &t);++      point_to_affine(px, py, x1, y1, z1);+      from_montgomery(out_x, px);+      from_montgomery(out_y, py);+      return;+    }+  }+  {+    uint64_t r1[8], r2[8], pt[2 * P256_NDIGITS];++    memcpy(pt, P256_DIGITS(in_x), P256_NBYTES);+    memcpy(pt + P256_NDIGITS, P256_DIGITS(in_y), P256_NBYTES);+    crypton_s2n_p256_scalarmulbase(r1, P256_DIGITS(n1));+    crypton_s2n_p256_scalarmul(r2, P256_DIGITS(n2), pt);+    s2n_lift(x1, y1, z1, r1);+    s2n_lift(x2, y2, z2, r2);+  }+#else   to_montgomery(px, in_x);   to_montgomery(py, in_y);   scalar_base_mult(x1, y1, z1, n1);   scalar_mult(x2, y2, z2, px, py, n2);+#endif    if (crypton_p256_is_zero(n2) != 0) {     /* If n2 == 0, then {x2,y2,z2} is zero and the result is just@@ -581,6 +1130,19 @@ void crypton_p256e_point_mul(const crypton_p256_int* n,     const crypton_p256_int* in_x, const crypton_p256_int* in_y,     crypton_p256_int* out_x, crypton_p256_int* out_y) {+#ifdef CRYPTON_S2N_BIGNUM+  /* The vendored assembly, which answers the same thing two and a half to+   * three times faster.  Both sides are four little-endian 64-bit words of+   * an ordinary affine coordinate, so the points cross as they are, and+   * both give (0, 0) for the point at infinity.  See cbits/s2n/README.md. */+  uint64_t point[8], res[8];++  memcpy(point, P256_DIGITS(in_x), P256_NBYTES);+  memcpy(point + P256_NDIGITS, P256_DIGITS(in_y), P256_NBYTES);+  crypton_s2n_p256_scalarmul(res, P256_DIGITS(n), point);+  memcpy(P256_DIGITS(out_x), res, P256_NBYTES);+  memcpy(P256_DIGITS(out_y), res + P256_NDIGITS, P256_NBYTES);+#else   felem x, y, z, px, py;    to_montgomery(px, in_x);@@ -589,4 +1151,5 @@   point_to_affine(px, py, x, y, z);   from_montgomery(out_x, px);   from_montgomery(out_y, py);+#endif }
+ cbits/p256/p256_s2n.c view
@@ -0,0 +1,61 @@+/*+ * Choosing between s2n-bignum's two forms of each routine.  The reasoning+ * is in cbits/s2n/README.md; in short, on ARM the choice is a+ * microarchitecture one that no feature bit answers, and on x86-64 it is+ * exactly a feature bit.+ */+#include "p256/p256_s2n.h"+#include "crypton_cpu.h"++extern void p256_scalarmul(uint64_t res[8], const uint64_t scalar[4],+                           const uint64_t point[8]);+extern void p256_scalarmul_alt(uint64_t res[8], const uint64_t scalar[4],+                               const uint64_t point[8]);+extern void p256_scalarmulbase(uint64_t res[8], const uint64_t scalar[4],+                               uint64_t blocksize, const uint64_t *table);+extern void p256_scalarmulbase_alt(uint64_t res[8], const uint64_t scalar[4],+                                   uint64_t blocksize, const uint64_t *table);++extern const uint64_t crypton_p256_s2n_base_blocksize;+extern const uint64_t crypton_p256_s2n_base_table[];++void crypton_s2n_p256_scalarmul(uint64_t res[8], const uint64_t scalar[4],+                                const uint64_t point[8])+{+#if defined(__aarch64__) || defined(__arm64__)+#ifdef __APPLE__+	/* the _alt form is the one written for high multiplier throughput,+	 * and it is 30-40% faster on Apple silicon */+	p256_scalarmul_alt(res, scalar, point);+#else+	p256_scalarmul(res, scalar, point);+#endif+#else+	if (crypton_x86_simd_features() & CRYPTON_X86_ADX)+		p256_scalarmul(res, scalar, point);+	else+		p256_scalarmul_alt(res, scalar, point);+#endif+}++void crypton_s2n_p256_scalarmulbase(uint64_t res[8], const uint64_t scalar[4])+{+#if defined(__aarch64__) || defined(__arm64__)+#ifdef __APPLE__+	p256_scalarmulbase_alt(res, scalar, crypton_p256_s2n_base_blocksize,+	                       crypton_p256_s2n_base_table);+#else+	p256_scalarmulbase(res, scalar, crypton_p256_s2n_base_blocksize,+	                   crypton_p256_s2n_base_table);+#endif+#else+	if (crypton_x86_simd_features() & CRYPTON_X86_ADX)+		p256_scalarmulbase(res, scalar,+		                   crypton_p256_s2n_base_blocksize,+		                   crypton_p256_s2n_base_table);+	else+		p256_scalarmulbase_alt(res, scalar,+		                       crypton_p256_s2n_base_blocksize,+		                       crypton_p256_s2n_base_table);+#endif+}
+ cbits/p256/p256_verify.c view
@@ -0,0 +1,300 @@+/*+ * The double scalar multiplication ECDSA verification does, in variable+ * time.+ *+ * Verification asks for u1*G + u2*Q, and crypton used to work that out as+ * two separate constant-time multiplications and an addition.  Nothing here+ * is secret -- the message, the signature and the public key are all sent in+ * the clear -- so the constant-time work is paid for nothing, and the two+ * multiplications can share their doublings besides.  Measured, those two+ * were 84% of a verification.+ *+ * So this is the usual interleaved windowed form: both scalars in+ * width-5 non-adjacent form, a table of odd multiples of each point, and one+ * pass down the digits with a doubling at every step and an addition where a+ * digit is not zero.  It is s2n-bignum's Jacobian point arithmetic+ * underneath, the same assembly the constant-time paths use.+ *+ * s2n-bignum's p256_montjadd is correct except when its two arguments are+ * the same point -- that is the side condition its proof carries -- so every+ * sum is checked for the sign of that, which is a zero z where neither+ * argument had one.  There the answer is given up on and the caller falls+ * back to the constant-time pair, which has no such condition.  With random+ * inputs it does not happen; it is here because an attacker chooses the+ * public key and the signature.+ */+#include <string.h>++#include "p256/p256.h"++#ifdef CRYPTON_S2N_BIGNUM++#include "crypton_cpu.h"+#include "p256/p256_verify.h"++/* a Jacobian triple in the Montgomery domain, as the assembly keeps them */+#define JAC 12+#define AFF 8++extern void p256_montjadd(uint64_t p3[JAC], const uint64_t p1[JAC],+                          const uint64_t p2[JAC]);+extern void p256_montjadd_alt(uint64_t p3[JAC], const uint64_t p1[JAC],+                              const uint64_t p2[JAC]);+extern void p256_montjdouble(uint64_t p3[JAC], const uint64_t p1[JAC]);+extern void p256_montjdouble_alt(uint64_t p3[JAC], const uint64_t p1[JAC]);+extern void p256_montjmixadd(uint64_t p3[JAC], const uint64_t p1[JAC],+                             const uint64_t p2[AFF]);+extern void p256_montjmixadd_alt(uint64_t p3[JAC], const uint64_t p1[JAC],+                                 const uint64_t p2[AFF]);++/* the odd multiples of the base point, from cbits/p256/gen_base_table.py */+extern const uint64_t crypton_p256_wnaf_width;+extern const uint64_t crypton_p256_wnaf_table[];+extern void bignum_tomont_p256(uint64_t z[4], const uint64_t x[4]);+extern void bignum_demont_p256(uint64_t z[4], const uint64_t x[4]);+extern void bignum_neg_p256(uint64_t z[4], const uint64_t x[4]);+#if !defined(__aarch64__) && !defined(__arm64__)+extern void bignum_tomont_p256_alt(uint64_t z[4], const uint64_t x[4]);+extern void bignum_demont_p256_alt(uint64_t z[4], const uint64_t x[4]);+#endif++/* The same question as everywhere else in cbits/s2n: a microarchitecture one+ * on ARM that no feature bit answers, and exactly a feature bit on x86-64. */+static int use_alt(void)+{+#if defined(__aarch64__) || defined(__arm64__)+#ifdef __APPLE__+	return 1;+#else+	return 0;+#endif+#else+	return (crypton_x86_simd_features() & CRYPTON_X86_ADX) == 0;+#endif+}++static void padd(uint64_t r[JAC], const uint64_t a[JAC], const uint64_t b[JAC],+                 int alt)+{+	if (alt)+		p256_montjadd_alt(r, a, b);+	else+		p256_montjadd(r, a, b);+}++static void pmixadd(uint64_t r[JAC], const uint64_t a[JAC],+                    const uint64_t b[AFF], int alt)+{+	if (alt)+		p256_montjmixadd_alt(r, a, b);+	else+		p256_montjmixadd(r, a, b);+}++static void pdouble(uint64_t r[JAC], const uint64_t a[JAC], int alt)+{+	if (alt)+		p256_montjdouble_alt(r, a);+	else+		p256_montjdouble(r, a);+}++static void tomont(uint64_t z[4], const uint64_t x[4])+{+#if defined(__aarch64__) || defined(__arm64__)+	bignum_tomont_p256(z, x);+#else+	if (use_alt())+		bignum_tomont_p256_alt(z, x);+	else+		bignum_tomont_p256(z, x);+#endif+}++static void demont(uint64_t z[4], const uint64_t x[4])+{+#if defined(__aarch64__) || defined(__arm64__)+	bignum_demont_p256(z, x);+#else+	if (use_alt())+		bignum_demont_p256_alt(z, x);+	else+		bignum_demont_p256(z, x);+#endif+}++static int is_infinity(const uint64_t p[JAC])+{+	return (p[8] | p[9] | p[10] | p[11]) == 0;+}++/*+ * The base point's table is a constant, so its window is as wide as the+ * table is worth carrying: seven bits, thirty-two odd multiples, two+ * kilobytes, and one addition every eight digits.  The public key's table+ * has to be built for each verification, so five bits is the trade there --+ * eight entries, seven point operations to build, and one addition every+ * six digits.+ */+#define WG 7+#define TG (1 << (WG - 2))+#define WQ 5+#define TQ (1 << (WQ - 2))+#define NAF_MAX 258++/*+ * The width-W non-adjacent form of a scalar, one signed digit per bit+ * position: odd or zero, and never two non-zero digits within W of each+ * other.  Returns how many digits were written.+ *+ * The scalar is public, so the loop may look at it.+ */+static int wnaf(int8_t out[NAF_MAX], const uint64_t in[4], int w)+{+	uint64_t k[5];+	int len = 0;++	memcpy(k, in, 32);+	k[4] = 0;++	while (k[0] | k[1] | k[2] | k[3] | k[4]) {+		int d = 0;++		if (k[0] & 1) {+			d = (int) (k[0] & ((1u << w) - 1));+			if (d >= (1 << (w - 1)))+				d -= 1 << w;+			if (d > 0) {+				uint64_t borrow = (uint64_t) d;+				int i;++				for (i = 0; i < 5 && borrow; i++) {+					uint64_t t = k[i];++					k[i] = t - borrow;+					borrow = (k[i] > t);+				}+			} else {+				uint64_t carry = (uint64_t) (-d);+				int i;++				for (i = 0; i < 5 && carry; i++) {+					k[i] += carry;+					carry = (k[i] < carry);+				}+			}+		}+		out[len++] = (int8_t) d;++		{ /* k >>= 1 */+			int i;++			for (i = 0; i < 4; i++)+				k[i] = (k[i] >> 1) | (k[i + 1] << 63);+			k[4] >>= 1;+		}+	}+	return len;+}++/* P, 3P, 5P, ..., (2*TQ-1)P from a Jacobian P */+static int build_table(uint64_t t[TQ][JAC], const uint64_t p[JAC], int alt)+{+	uint64_t twice[JAC];+	int i;++	memcpy(t[0], p, sizeof(uint64_t) * JAC);+	pdouble(twice, p, alt);+	for (i = 1; i < TQ; i++) {+		padd(t[i], t[i - 1], twice, alt);+		/* the table is built from a point and its double, which are+		 * never the same point unless the point has order two, and+		 * P-256 has none */+		if (is_infinity(t[i]) && !is_infinity(t[i - 1])+		    && !is_infinity(twice))+			return 0;+	}+	return 1;+}++/* the table entry for a digit, negated when the digit is */+static void pick(uint64_t out[JAC], const uint64_t t[TQ][JAC], int digit)+{+	int idx = (digit > 0 ? digit : -digit) / 2;++	memcpy(out, t[idx], sizeof(uint64_t) * JAC);+	if (digit < 0)+		bignum_neg_p256(out + 4, out + 4);+}++/* the same from the base point's affine table */+static void pick_affine(uint64_t out[AFF], int digit)+{+	int idx = (digit > 0 ? digit : -digit) / 2;++	memcpy(out, crypton_p256_wnaf_table + (size_t) idx * AFF,+	       sizeof(uint64_t) * AFF);+	if (digit < 0)+		bignum_neg_p256(out + 4, out + 4);+}++int crypton_p256_verify_mul(uint64_t out[JAC], const uint64_t n1[4],+                            const uint64_t n2[4], const uint64_t qx[4],+                            const uint64_t qy[4])+{+	/* the Montgomery form of one, which is the z of an affine point */+	static const uint64_t mont_one[4] = {+		0x0000000000000001ULL, 0xffffffff00000000ULL,+		0xffffffffffffffffULL, 0x00000000fffffffeULL+	};+	uint64_t tq[TQ][JAC];+	uint64_t q[JAC], acc[JAC], addend[JAC], aff[AFF], sum[JAC];+	int8_t naf1[NAF_MAX], naf2[NAF_MAX];+	int len1, len2, len, i;+	/* asked once rather than at every point operation */+	const int alt = use_alt();++	/* the generated table has to be the width this file walks it at */+	if (crypton_p256_wnaf_width != WG)+		return 0;++	tomont(q, qx);+	tomont(q + 4, qy);+	memcpy(q + 8, mont_one, sizeof mont_one);++	if (!build_table(tq, q, alt))+		return 0;++	len1 = wnaf(naf1, n1, WG);+	len2 = wnaf(naf2, n2, WQ);+	len = len1 > len2 ? len1 : len2;++	memset(acc, 0, sizeof acc);+	for (i = len - 1; i >= 0; i--) {+		pdouble(acc, acc, alt);++		if (i < len1 && naf1[i] != 0) {+			/* the base point's entries are affine, which is a+			 * cheaper addition and no table to build */+			pick_affine(aff, naf1[i]);+			pmixadd(sum, acc, aff, alt);+			if (is_infinity(sum) && !is_infinity(acc))+				return 0;+			memcpy(acc, sum, sizeof acc);+		}+		if (i < len2 && naf2[i] != 0) {+			pick(addend, tq, naf2[i]);+			padd(sum, acc, addend, alt);+			if (is_infinity(sum) && !is_infinity(acc))+				return 0;+			memcpy(acc, sum, sizeof acc);+		}+	}++	demont(out, acc);+	demont(out + 4, acc + 4);+	demont(out + 8, acc + 8);+	return 1;+}++#endif
+ cbits/p256/p256_verify.h view
@@ -0,0 +1,19 @@+#ifndef CRYPTON_P256_VERIFY_H+#define CRYPTON_P256_VERIFY_H++#include <stdint.h>++/*+ * n1*G + n2*Q, in variable time, as a Jacobian triple in the plain domain.+ *+ * Returns 1 when the answer is in `out`, and 0 when the walk met the one+ * case s2n-bignum's point addition does not cover -- adding a point to+ * itself -- in which case the caller works the answer out the constant-time+ * way instead.  Nothing here is secret: it is all in the signature and the+ * public key.+ */+int crypton_p256_verify_mul(uint64_t out[12], const uint64_t n1[4],+                            const uint64_t n2[4], const uint64_t qx[4],+                            const uint64_t qy[4]);++#endif
+ cbits/p256/p256_wnaf_table.c view
@@ -0,0 +1,42 @@+/* Generated by cbits/p256/gen_base_table.py; do not hand-edit.+ * The odd multiples of the base point, in Montgomery-affine+ * form, which cbits/p256/p256_verify.c reads.  A verification+ * is public, so this table is walked in variable time. */+#include <stdint.h>++const uint64_t crypton_p256_wnaf_width = 7;++const uint64_t crypton_p256_wnaf_table[256] = {+	0x79e730d418a9143cULL,0x75ba95fc5fedb601ULL,0x79fb732b77622510ULL,0x18905f76a53755c6ULL,0xddf25357ce95560aULL,0x8b4ab8e4ba19e45cULL,0xd2e88688dd21f325ULL,0x8571ff1825885d85ULL,+	0xffac3f904eebc127ULL,0xb027f84a087d81fbULL,0x66ad77dd87cbbc98ULL,0x26936a3fb6ff747eULL,0xb04c5c1fc983a7ebULL,0x583e47ad0861fe1aULL,0x788208311a2ee98eULL,0xd5f06a29e587cc07ULL,+	0xbe1b8aaec45c61f5ULL,0x90ec649a94b9537dULL,0x941cb5aad076c20cULL,0xc9079605890523c8ULL,0xeb309b4ae7ba4f10ULL,0x73c568efe5eb882bULL,0x3540a9877e7a1f68ULL,0x73a076bb2dd1e916ULL,+	0x0746354ea0173b4fULL,0x2bd20213d23c00f7ULL,0xf43eaab50c23bb08ULL,0x13ba5119c3123e03ULL,0x2847d0303f5b9d4dULL,0x6742f2f25da67bddULL,0xef933bdc77c94195ULL,0xeaedd9156e240867ULL,+	0x75c96e8f264e20e8ULL,0xabe6bfed59a7a841ULL,0x2cc09c0444c8eb00ULL,0xe05b3080f0c4e16bULL,0x1eb7777aa45f3314ULL,0x56af7bedce5d45e3ULL,0x2b6e019a88b12f1aULL,0x086659cdfd835f9bULL,+	0xea7d260a6245e404ULL,0x9de407956e7fdfe0ULL,0x1ff3a4158dac1ab5ULL,0x3e7090f1649c9073ULL,0x1a7685612b944e88ULL,0x250f939ee57f61c8ULL,0x0c0daa891ead643dULL,0x68930023e125b88eULL,+	0xccc425634b2ed709ULL,0x0e356769856fd30dULL,0xbcbcd43f559e9811ULL,0x738477ac5395b759ULL,0x35752b90c00ee17fULL,0x68748390742ed2e3ULL,0x7cd06422bd1f5bc1ULL,0xfbc08769c9e7b797ULL,+	0x72bcd8b7bc60055bULL,0x03cc23ee56e27e4bULL,0xee337424e4819370ULL,0xe2aa0e430ad3da09ULL,0x40b8524f6383c45dULL,0xd766355442a41b25ULL,0x64efa6de778a4797ULL,0x2042170a7079adf4ULL,+	0x97091dcbd53c5c9dULL,0xf17624b6ac0a177bULL,0xb0f139752cfe2dffULL,0xc1a35c0a6c7a574eULL,0x227d314693e79987ULL,0x0575bf30e89cb80eULL,0x2f4e247f0d1883bbULL,0xebd512263274c3d0ULL,+	0xfea912baa5659ae8ULL,0x68363aba25e1a16eULL,0xb8842277752c41acULL,0xfe545c282897c3fcULL,0x2d36e9e7dc4c696bULL,0x5806244afba977c5ULL,0x85665e9be39508c1ULL,0xf720ee256d12597bULL,+	0x562e4cecc135b208ULL,0x74e1b2654783f47dULL,0x6d2a506c5a3f3b30ULL,0xecead9f4c16762fcULL,0xf29dd4b2e286e5b9ULL,0x1b0fadc083bb3c61ULL,0x7a75023e7fac29a4ULL,0xc086d5f1c9477fa3ULL,+	0xf4f876532de45068ULL,0x37c7a7e89e2e1f6eULL,0xd0825fa2a3584069ULL,0xaf2cea7c1727bf42ULL,0x0360a4fb9e4785a9ULL,0xe5fda49c27299f4aULL,0x48068e1371ac2f71ULL,0x83d0687b9077666fULL,+	0xa4a319acd837879fULL,0x6fc1b49eed6b67b0ULL,0xe395993332f1f3afULL,0x966742eb65432a2eULL,0x4b8dc9feb4966228ULL,0x96cc631243f43950ULL,0x12068859c9b731eeULL,0x7b948dc356f79968ULL,+	0x042c2af497e2feb4ULL,0xd36a42d7aebf7313ULL,0x49d2c9eb084ffdd7ULL,0x9f8aa54b2ef7c76aULL,0x9200b7ba09895e70ULL,0x3bd0c66fddb7fb58ULL,0x2d97d10878eb4cbbULL,0x2d431068d84bde31ULL,+	0x5e5db46acb66e132ULL,0xf1be963a0d925880ULL,0x944a70270317b9e2ULL,0xe266f95948603d48ULL,0x98db66735c208899ULL,0x90472447a2fb18a3ULL,0x8a966939777c619fULL,0x3798142a2a3be21bULL,+	0xe2f73c696755ff89ULL,0xdd3cf7e7473017e6ULL,0x8ef5689d3cf7600dULL,0x948dc4f8b1fc87b4ULL,0xd9e9fe814ea53299ULL,0x2d921ca298eb6028ULL,0xfaecedfd0c9803fcULL,0xf38ae8914d7b4745ULL,+	0x871514560f664534ULL,0x85ceae7c4b68f103ULL,0xac09c4ae65578ab9ULL,0x33ec6868f044b10cULL,0x6ac4832b3a8ec1f1ULL,0x5509d1285847d5efULL,0xf909604f763f1574ULL,0xb16c4303c32f63c4ULL,+	0xfd16847fdec67ef5ULL,0x742ee464233e76b7ULL,0x0b8e4134efc2b4c8ULL,0xca640b8642a3e521ULL,0x653a01908ceb6aa9ULL,0x313c300c547852d5ULL,0x24e4ab126b237af7ULL,0x2ba901628bb47af8ULL,+	0x00467bc58cce08b5ULL,0xb636458c7f178d55ULL,0xc5748baea677d806ULL,0x2763a387dfa394ebULL,0xa12b448a7d3cebb6ULL,0xe7adda3e6f20d850ULL,0xf63ebce51558462cULL,0x58b36143620088a8ULL,+	0xa9d89488a059c142ULL,0x6f5ae714ff0b9346ULL,0x068f237d16fb3664ULL,0x5853e4c4363186acULL,0xe2d87d2363c52f98ULL,0x2ec4a76681828876ULL,0x47b864fae14e7b1cULL,0x0c0bc0e569192408ULL,+	0x624d60492ed22e91ULL,0x6fdfe0b56f072822ULL,0xeeca111539ce2271ULL,0x98100a4fdb01614fULL,0xb6b0daa2a35c628fULL,0xb6f94d2ec87e9a47ULL,0xc67732591d57d9ceULL,0xf70bfeec03884a7bULL,+	0x4ff23ffd248a7d06ULL,0x80c5bfb4878873faULL,0xb7d9ad9005745981ULL,0x179c85db3db01994ULL,0xba41b06261a6966cULL,0x4d82d052eadce5a8ULL,0x9e91cd3ba5e6a318ULL,0x47795f4f95b2dda0ULL,+	0x1ee426ccd5cd79bfULL,0x0032940b946c6e18ULL,0x1b1e8ae057477f58ULL,0xe94f7d346d823278ULL,0xc747cb96782ba21aULL,0xc5254469f72b33a5ULL,0x772ef6dec7f80c81ULL,0xd73acbfe2cd9e6b5ULL,+	0x283c7513caa76097ULL,0x0a624fa936c83906ULL,0x6b20afec715af2c7ULL,0x4b969974eba78bfdULL,0x220755ccd921d60eULL,0x9b944e107baeca13ULL,0x04819d515ded93d4ULL,0x9bbff86e6dddfd27ULL,+	0x21950b421ff6acd3ULL,0xffe7048453dc6909ULL,0xff4cd0b228766127ULL,0xabdbe6084fb7db2bULL,0x837c92285e1109e8ULL,0x26147d27f4645b5aULL,0x4d78f592f7818ed8ULL,0xd394077ef247fa36ULL,+	0x508cec1c3b3f64c9ULL,0xe20bc0ba1e5edf3fULL,0xda1deb852f4318d4ULL,0xd20ebe0d5c3fa443ULL,0x370b4ea773241ea3ULL,0x61f1511c5e1a5f65ULL,0x99a5e23d82681c62ULL,0xd731e383a2f54c2dULL,+	0x97359638546c4d8dULL,0x5f9c3fc492f24679ULL,0x912e8beda8c8acd9ULL,0xec3a318d306634b0ULL,0x80167f41c31cb264ULL,0x3db82f6f522113f2ULL,0xb155bcd2dcafe197ULL,0xfba1da5943465283ULL,+	0x258bbbf9e7305683ULL,0x31eea5bf07ef5be6ULL,0x0deb0e4a46c814c1ULL,0x5cee8449a7b730ddULL,0xeab495c5a0182bdeULL,0xee759f879e27a6b4ULL,0xc2cf6a6880e518caULL,0x25e8013ff14cf3f4ULL,+	0x3ec832e77acaca28ULL,0x1bfeea57c7385b29ULL,0x068212e3fd1eaf38ULL,0xc13298306acf8cccULL,0xb909f2db2aac9e59ULL,0x5748060db661782aULL,0xc5ab2632c79b7a01ULL,0xda44c6c600017626ULL,+	0x69d44ed65c46aa8eULL,0x2100d5d3a8d063d1ULL,0xcb9727eaa2d17c36ULL,0x4c2bab1b8add53b7ULL,0xa084e90c15426704ULL,0x778afcd3a837ebeaULL,0x6651f7017ce477f8ULL,0xa062499846fb7a8bULL,+	0x3667eb1a7f4c04ccULL,0x59556621a9404f84ULL,0x71cdf6537eceb50aULL,0x994a44a69b8335faULL,0xd7faf819dbeb9b69ULL,0x473c5680eed4350dULL,0xb6658466da44bba2ULL,0x0d1bc780872bdbf3ULL,+	0xb8d3d9319ff91fe5ULL,0x039c4800f0518eedULL,0x95c376329182cb26ULL,0x0763a43482fc568dULL,0x707c04d5383e76baULL,0xac98b930824e8197ULL,0x92bf7c8f91230de0ULL,0x90876a0140959b70ULL,+};
+ cbits/s2n/COMMIT view
@@ -0,0 +1,1 @@+62ec77dc6c2c8cc4c48c768f5f785240b55a47bf
+ cbits/s2n/LICENSE view
@@ -0,0 +1,222 @@+SPDX-License-Identifier: Apache-2.0 OR ISC or MIT-0+++Apache 2.0 license+-------------------------------------+++                                 Apache License+                           Version 2.0, January 2004+                        http://www.apache.org/licenses/++   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION++   1. Definitions.++      "License" shall mean the terms and conditions for use, reproduction,+      and distribution as defined by Sections 1 through 9 of this document.++      "Licensor" shall mean the copyright owner or entity authorized by+      the copyright owner that is granting the License.++      "Legal Entity" shall mean the union of the acting entity and all+      other entities that control, are controlled by, or are under common+      control with that entity. For the purposes of this definition,+      "control" means (i) the power, direct or indirect, to cause the+      direction or management of such entity, whether by contract or+      otherwise, or (ii) ownership of fifty percent (50%) or more of the+      outstanding shares, or (iii) beneficial ownership of such entity.++      "You" (or "Your") shall mean an individual or Legal Entity+      exercising permissions granted by this License.++      "Source" form shall mean the preferred form for making modifications,+      including but not limited to software source code, documentation+      source, and configuration files.++      "Object" form shall mean any form resulting from mechanical+      transformation or translation of a Source form, including but+      not limited to compiled object code, generated documentation,+      and conversions to other media types.++      "Work" shall mean the work of authorship, whether in Source or+      Object form, made available under the License, as indicated by a+      copyright notice that is included in or attached to the work+      (an example is provided in the Appendix below).++      "Derivative Works" shall mean any work, whether in Source or Object+      form, that is based on (or derived from) the Work and for which the+      editorial revisions, annotations, elaborations, or other modifications+      represent, as a whole, an original work of authorship. For the purposes+      of this License, Derivative Works shall not include works that remain+      separable from, or merely link (or bind by name) to the interfaces of,+      the Work and Derivative Works thereof.++      "Contribution" shall mean any work of authorship, including+      the original version of the Work and any modifications or additions+      to that Work or Derivative Works thereof, that is intentionally+      submitted to Licensor for inclusion in the Work by the copyright owner+      or by an individual or Legal Entity authorized to submit on behalf of+      the copyright owner. For the purposes of this definition, "submitted"+      means any form of electronic, verbal, or written communication sent+      to the Licensor or its representatives, including but not limited to+      communication on electronic mailing lists, source code control systems,+      and issue tracking systems that are managed by, or on behalf of, the+      Licensor for the purpose of discussing and improving the Work, but+      excluding communication that is conspicuously marked or otherwise+      designated in writing by the copyright owner as "Not a Contribution."++      "Contributor" shall mean Licensor and any individual or Legal Entity+      on behalf of whom a Contribution has been received by Licensor and+      subsequently incorporated within the Work.++   2. Grant of Copyright License. Subject to the terms and conditions of+      this License, each Contributor hereby grants to You a perpetual,+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable+      copyright license to reproduce, prepare Derivative Works of,+      publicly display, publicly perform, sublicense, and distribute the+      Work and such Derivative Works in Source or Object form.++   3. Grant of Patent License. Subject to the terms and conditions of+      this License, each Contributor hereby grants to You a perpetual,+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable+      (except as stated in this section) patent license to make, have made,+      use, offer to sell, sell, import, and otherwise transfer the Work,+      where such license applies only to those patent claims licensable+      by such Contributor that are necessarily infringed by their+      Contribution(s) alone or by combination of their Contribution(s)+      with the Work to which such Contribution(s) was submitted. If You+      institute patent litigation against any entity (including a+      cross-claim or counterclaim in a lawsuit) alleging that the Work+      or a Contribution incorporated within the Work constitutes direct+      or contributory patent infringement, then any patent licenses+      granted to You under this License for that Work shall terminate+      as of the date such litigation is filed.++   4. Redistribution. You may reproduce and distribute copies of the+      Work or Derivative Works thereof in any medium, with or without+      modifications, and in Source or Object form, provided that You+      meet the following conditions:++      (a) You must give any other recipients of the Work or+          Derivative Works a copy of this License; and++      (b) You must cause any modified files to carry prominent notices+          stating that You changed the files; and++      (c) You must retain, in the Source form of any Derivative Works+          that You distribute, all copyright, patent, trademark, and+          attribution notices from the Source form of the Work,+          excluding those notices that do not pertain to any part of+          the Derivative Works; and++      (d) If the Work includes a "NOTICE" text file as part of its+          distribution, then any Derivative Works that You distribute must+          include a readable copy of the attribution notices contained+          within such NOTICE file, excluding those notices that do not+          pertain to any part of the Derivative Works, in at least one+          of the following places: within a NOTICE text file distributed+          as part of the Derivative Works; within the Source form or+          documentation, if provided along with the Derivative Works; or,+          within a display generated by the Derivative Works, if and+          wherever such third-party notices normally appear. The contents+          of the NOTICE file are for informational purposes only and+          do not modify the License. You may add Your own attribution+          notices within Derivative Works that You distribute, alongside+          or as an addendum to the NOTICE text from the Work, provided+          that such additional attribution notices cannot be construed+          as modifying the License.++      You may add Your own copyright statement to Your modifications and+      may provide additional or different license terms and conditions+      for use, reproduction, or distribution of Your modifications, or+      for any such Derivative Works as a whole, provided Your use,+      reproduction, and distribution of the Work otherwise complies with+      the conditions stated in this License.++   5. Submission of Contributions. Unless You explicitly state otherwise,+      any Contribution intentionally submitted for inclusion in the Work+      by You to the Licensor shall be under the terms and conditions of+      this License, without any additional terms or conditions.+      Notwithstanding the above, nothing herein shall supersede or modify+      the terms of any separate license agreement you may have executed+      with Licensor regarding such Contributions.++   6. Trademarks. This License does not grant permission to use the trade+      names, trademarks, service marks, or product names of the Licensor,+      except as required for reasonable and customary use in describing the+      origin of the Work and reproducing the content of the NOTICE file.++   7. Disclaimer of Warranty. Unless required by applicable law or+      agreed to in writing, Licensor provides the Work (and each+      Contributor provides its Contributions) on an "AS IS" BASIS,+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or+      implied, including, without limitation, any warranties or conditions+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A+      PARTICULAR PURPOSE. You are solely responsible for determining the+      appropriateness of using or redistributing the Work and assume any+      risks associated with Your exercise of permissions under this License.++   8. Limitation of Liability. In no event and under no legal theory,+      whether in tort (including negligence), contract, or otherwise,+      unless required by applicable law (such as deliberate and grossly+      negligent acts) or agreed to in writing, shall any Contributor be+      liable to You for damages, including any direct, indirect, special,+      incidental, or consequential damages of any character arising as a+      result of this License or out of the use or inability to use the+      Work (including but not limited to damages for loss of goodwill,+      work stoppage, computer failure or malfunction, or any and all+      other commercial damages or losses), even if such Contributor+      has been advised of the possibility of such damages.++   9. Accepting Warranty or Additional Liability. While redistributing+      the Work or Derivative Works thereof, You may choose to offer,+      and charge a fee for, acceptance of support, warranty, indemnity,+      or other liability obligations and/or rights consistent with this+      License. However, in accepting such obligations, You may act only+      on Your own behalf and on Your sole responsibility, not on behalf+      of any other Contributor, and only if You agree to indemnify,+      defend, and hold each Contributor harmless for any liability+      incurred by, or claims asserted against, such Contributor by reason+      of your accepting any such warranty or additional liability.++   END OF TERMS AND CONDITIONS+++ISC license+-------------------------------------++Copyright Amazon.com, Inc. or its affiliates.++Permission to use, copy, modify, and/or distribute this software for any+purpose with or without fee is hereby granted, provided that the above+copyright notice and this permission notice appear in all copies.++THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES+WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF+MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR+ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES+WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN+ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF+OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.+++MIT-0 license+-------------------------------------++Copyright 2021-2024 Amazon.com, Inc. or its affiliates.++Permission is hereby granted, free of charge, to any person obtaining a+copy of this software and associated documentation files (the "Software"),+to deal in the Software without restriction, including without limitation+the rights to use, copy, modify, merge, publish, distribute, sublicense,+and/or sell copies of the Software, and to permit persons to whom the+Software is furnished to do so.++THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER+DEALINGS IN THE SOFTWARE.
+ cbits/s2n/README.md view
@@ -0,0 +1,100 @@+# s2n-bignum++Assembly for the NIST prime curves from AWS's+[s2n-bignum](https://github.com/awslabs/s2n-bignum), vendored here.++`COMMIT` holds the upstream revision these files came from, and `import.sh`+fetches them again.  The files are unmodified: the choice between the two+variants of each routine is made in crypton's own C, not by editing theirs.++## Why++crypton's P-256 is portable C, and on the two architectures s2n-bignum+covers, hand-written assembly beats it by a lot.  Measured against crypton's+own code in the same process, agreeing with it on every scalar tried:++| | crypton | s2n-bignum |+| --- | ---: | ---: |+| Apple M4 | 52.9 us | **20.6** |+| x86-64 with ADX (EPYC 9V74) | 175.9 | **54.5** |+| x86-64, ADX not advertised | 156.1 | **59.4** |++The third row is the `_alt` path, which is what crypton picks where `CPUID`+does not report ADX.  It was measured on a KVM guest whose `CPUID` says so+while the host underneath runs ADX instructions anyway, so it says what the+two implementations cost relative to each other on that path, not what an+actual pre-Broadwell part would do.++Each routine also carries a machine-checked proof in HOL-Light that it+computes what it says, and is written in a constant-time style.++## Licence++`Apache-2.0 OR ISC OR MIT-0`, one of which is on every file and all three in+`LICENSE`.  crypton is BSD-3, so it takes these under **ISC** -- the MIT-0+option would do as well, and the Apache one is the reason OpenSSL's and+BoringSSL's `ecp_nistz256`, which is Apache-2.0 only, cannot be used here.++## Which variant++Both forms of each routine are vendored, because which one is faster is not+the same question on the two architectures:++* **On ARM**, `_alt` is written for parts with high multiplier throughput,+  which is what Apple silicon is, and it wins there by 30-40%.  The plain+  form is for parts that pipeline `UMULH` less well.  There is no feature bit+  for this, so the choice is made at compile time on `__APPLE__`.+* **On x86-64**, the plain form uses `MULX`, `ADCX` and `ADOX`, and `_alt` is+  the fallback for processors without them.  That *is* a feature bit, so the+  choice is made at run time, from `crypton_x86_simd_features()`.++## RSA++`crypton_powm.c`'s modular exponentiation also borrows from here, but only its+innermost step and only on x86-64: `bignum_kmul_16_32`, `bignum_ksqr_16_32`,+`bignum_kmul_32_64`, `bignum_ksqr_32_64` and `bignum_emontredc_8n` replace the+C's Montgomery multiplication and square.  The window, the table and its masked+scan are crypton's throughout.  Sixteen limbs is 1024 bits and thirty-two is+2048: the halves a CRT exponentiation works in for RSA-2048 and RSA-4096, and+the whole thing without CRT.  The reduction wants ADX, so the choice is made at+run time like the other x86-64 ones.++It is twice the C, because the C cannot form the two carry chains `ADCX` and+`ADOX` give.  Measured on an EPYC 7763 at 1024 bits:++| | C | s2n-bignum |+| --- | ---: | ---: |+| multiplication | 0.4832 us | **0.2479** |+| square | 0.3984 | **0.1994** |++**Nothing is vendored for AArch64**, where the same five routines measure level+with the C.  That took three attempts to establish, and the first two were+wrong in opposite directions: a reading when the x86-64 routines went in put+the C 5% ahead, and one on 2026-09-26 put the assembly 4% ahead.  Both were+wall-clock times on a machine with other work on it, where a swing of that size+says nothing.  Measured by thread CPU time, taking the best of twenty-five+batches and alternating the two binaries with the order swapped, one RSA-2048+CRT private operation is 598.9 us through the C and 597.7 through the assembly:+two tenths of a per cent, which is nothing.  The two agree on 200 random cases+at 1024 and 2048 bits, so both were computing the same thing.  Five files for+nothing is not a trade, so the C stays.++The gap to OpenSSL on Apple silicon -- about half its speed -- is assembly and+not the C, and no portable C closes it.  BearSSL's `i62`, which is as far as+portable C is known to go -- 62-bit limbs in 64-bit words, so that a+multiply-accumulate fits an `__int128` with no carry chain at all, and a+five-bit window against crypton's four -- was built and measured the same way+on the same machine: 626.4 us against crypton's C at 668.0, a tie.++`bignum_emontredc_8n_cdiff` was tried too and is not the answer either: it+wants a precomputation this test did not give it, and was slower besides.  What+the window is worth, and why it is four and not five, is in+`cbits/crypton_powm.c` beside the constant.++## What is not here++s2n-bignum has only x86-64 and AArch64, so crypton's C stays and is what+every other architecture uses.  `p384_montjscalarmul` and `p521_jscalarmul`+have no affine wrapper upstream; crypton's is in `cbits/p256/p256_s2n.c`'s+sibling for those curves.  There is no fixed-base routine for P-384 or+P-521 at all, so signing on those curves keeps crypton's comb.
+ cbits/s2n/arm/bignum_deamont_p384.S view
@@ -0,0 +1,151 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Convert from almost-Montgomery form, z := (x / 2^384) mod p_384+// Input x[6]; output z[6]+//+//    extern void bignum_deamont_p384(uint64_t z[static 6],+//                                    const uint64_t x[static 6]);+//+// Convert a 6-digit bignum x out of its (optionally almost) Montgomery form,+// "almost" meaning any 6-digit input will work, with no range restriction.+//+// Standard ARM ABI: X0 = z, X1 = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_deamont_p384)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_deamont_p384)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_deamont_p384)+        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_deamont_p384_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_deamont_p384_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_deamont_p384_alt)+        .text+        .balign 4++// ---------------------------------------------------------------------------+// Core one-step "short" Montgomery reduction macro. Takes input in+// [d5;d4;d3;d2;d1;d0] and returns result in [d6;d5;d4;d3;d2;d1],+// adding to the existing contents of [d5;d4;d3;d2;d1]. It is fine+// for d6 to be the same register as d0.+//+// We want to add (2^384 - 2^128 - 2^96 + 2^32 - 1) * w+// where w = [d0 + (d0<<32)] mod 2^64+// ---------------------------------------------------------------------------++#define montreds(d6,d5,d4,d3,d2,d1,d0, t3,t2,t1)                            \+/* Our correction multiplier is w = [d0 + (d0<<32)] mod 2^64            */  \+/* Recycle d0 (which we know gets implicitly cancelled) to store it     */  \+        lsl     t1, d0, #32 __LF                                       \+        add     d0, t1, d0 __LF                                        \+/* Now let [t2;t1] = 2^64 * w - w + w_hi where w_hi = floor(w/2^32)     */  \+/* We need to subtract 2^32 * this, and we can ignore its lower 32      */  \+/* bits since by design it will cancel anyway; we only need the w_hi    */  \+/* part to get the carry propagation going.                             */  \+        lsr     t1, d0, #32 __LF                                       \+        subs    t1, t1, d0 __LF                                        \+        sbc     t2, d0, xzr __LF                                       \+/* Now select in t1 the field to subtract from d1                       */  \+        extr    t1, t2, t1, #32 __LF                                   \+/* And now get the terms to subtract from d2 and d3                     */  \+        lsr     t2, t2, #32 __LF                                       \+        adds    t2, t2, d0 __LF                                        \+        adc     t3, xzr, xzr __LF                                      \+/* Do the subtraction of that portion                                   */  \+        subs    d1, d1, t1 __LF                                        \+        sbcs    d2, d2, t2 __LF                                        \+        sbcs    d3, d3, t3 __LF                                        \+        sbcs    d4, d4, xzr __LF                                       \+        sbcs    d5, d5, xzr __LF                                       \+/* Now effectively add 2^384 * w by taking d0 as the input for last sbc */  \+        sbc     d6, d0, xzr++// Input parameters++#define z x0+#define x x1++// Rotating registers for the intermediate windows++#define d0 x2+#define d1 x3+#define d2 x4+#define d3 x5+#define d4 x6+#define d5 x7++// Other temporaries++#define u x8+#define v x9+#define w x10++S2N_BN_SYMBOL(bignum_deamont_p384):++S2N_BN_SYMBOL(bignum_deamont_p384_alt):+        CFI_START++// Set up an initial window with the input x and an extra leading zero++        ldp     d0, d1, [x]+        ldp     d2, d3, [x, #16]+        ldp     d4, d5, [x, #32]++// Systematically scroll left doing 1-step reductions++        montreds(d0,d5,d4,d3,d2,d1,d0, u,v,w)++        montreds(d1,d0,d5,d4,d3,d2,d1, u,v,w)++        montreds(d2,d1,d0,d5,d4,d3,d2, u,v,w)++        montreds(d3,d2,d1,d0,d5,d4,d3, u,v,w)++        montreds(d4,d3,d2,d1,d0,d5,d4, u,v,w)++        montreds(d5,d4,d3,d2,d1,d0,d5, u,v,w)++// Now compare end result in [d5;d4;d3;d2;d1;d0] = dd with p_384 by *adding*+// 2^384 - p_384 = [0;0;0;w;v;u]. This will set CF if+// dd + (2^384 - p_384) >= 2^384, hence iff dd >= p_384++        mov     u, #0xffffffff00000001+        mov     v, #0x00000000ffffffff+        mov     w, #0x0000000000000001++        adds    xzr, d0, u+        adcs    xzr, d1, v+        adcs    xzr, d2, w+        adcs    xzr, d3, xzr+        adcs    xzr, d4, xzr+        adcs    xzr, d5, xzr++// Convert the condition dd >= p_384 into a bitmask in w and do a masked+// subtraction of p_384, via a masked addition of 2^384 - p_384:++        csetm   w, cs+        and     u, u, w+        adds    d0, d0, u+        and     v, v, w+        adcs    d1, d1, v+        and     w, w, #1+        adcs    d2, d2, w+        adcs    d3, d3, xzr+        adcs    d4, d4, xzr+        adc     d5, d5, xzr++// Store it back++        stp     d0, d1, [z]+        stp     d2, d3, [z, #16]+        stp     d4, d5, [z, #32]++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_deamont_p384)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/bignum_demont_p256.S view
@@ -0,0 +1,99 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Convert from Montgomery form z := (x / 2^256) mod p_256, assuming x reduced+// Input x[4]; output z[4]+//+//    extern void bignum_demont_p256(uint64_t z[static 4],+//                                   const uint64_t x[static 4]);+//+// This assumes the input is < p_256 for correctness. If this is not the case,+// use the variant "bignum_deamont_p256" instead.+//+// Standard ARM ABI: X0 = z, X1 = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_demont_p256)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_demont_p256)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_demont_p256)+        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_demont_p256_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_demont_p256_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_demont_p256_alt)+        .text+        .balign 4++// ---------------------------------------------------------------------------+// Core one-step "short" Montgomery reduction macro. Takes input in+// [d3;d2;d1;d0] and returns result in [d4;d3;d2;d1], adding to the+// existing contents of [d3;d2;d1] and generating d4 from zero, re-using+// d0 as a temporary internally together with t0, t1 and t2.+// It is fine for d4 to be the same register as d0, and it often is.+// ---------------------------------------------------------------------------++#define montreds(d4,d3,d2,d1,d0, t2,t1,t0)                                  \+/* Let w = d0, the original word we use as offset; d0 gets recycled      */ \+/* First let [t2;t1] = 2^32 * w                                          */ \+/* then let [d0;t0] = (2^64 - 2^32 + 1) * w (overwrite old d0)           */ \+        lsl     t1, d0, #32 __LF                                       \+        subs    t0, d0, t1 __LF                                        \+        lsr     t2, d0, #32 __LF                                       \+        sbc     d0, d0, t2 __LF                                        \+/* Hence [d4;..;d1] := [d3;d2;d1;0] + (2^256 - 2^224 + 2^192 + 2^96) * w */ \+        adds    d1, d1, t1 __LF                                        \+        adcs    d2, d2, t2 __LF                                        \+        adcs    d3, d3, t0 __LF                                        \+        adc     d4, d0, xzr++// Input parameters++#define z x0+#define x x1++// Rotating registers for the intermediate windows (with repetitions)++#define d0 x2+#define d1 x3+#define d2 x4+#define d3 x5++// Other temporaries++#define u x6+#define v x7+#define w x8++S2N_BN_SYMBOL(bignum_demont_p256):++S2N_BN_SYMBOL(bignum_demont_p256_alt):+        CFI_START++// Set up an initial window with the input x and an extra leading zero++        ldp     d0, d1, [x]+        ldp     d2, d3, [x, #16]++// Systematically scroll left doing 1-step reductions++        montreds(d0,d3,d2,d1,d0, u,v,w)++        montreds(d1,d0,d3,d2,d1, u,v,w)++        montreds(d2,d1,d0,d3,d2, u,v,w)++        montreds(d3,d2,d1,d0,d3, u,v,w)++// Write back result++        stp     d0, d1, [z]+        stp     d2, d3, [z, #16]++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_demont_p256)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/bignum_inv_p521.S view
@@ -0,0 +1,1701 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Modular inverse modulo p_521 =  2^521 - 1+// Input x[9]; output z[9]+//+// extern void bignum_inv_p521(uint64_t z[static 9],const uint64_t x[static 9]);+//+// Assuming the 9-digit input x is coprime to p_521, i.e. is not divisible+// by it, returns z < p_521 such that x * z == 1 (mod p_521). Note that+// x does not need to be reduced modulo p_521, but the output always is.+//+// Standard ARM ABI: X0 = z, X1 = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_inv_p521)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_inv_p521)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_inv_p521)++        .text+        .balign 4++// Size in bytes of a 64-bit word++#define N 8++// Used for the return pointer++#define res x20++// Loop counter and d = 2 * delta value for divstep++#define i x21+#define d x22++// Registers used for matrix element magnitudes and signs++#define m00 x10+#define m01 x11+#define m10 x12+#define m11 x13+#define s00 x14+#define s01 x15+#define s10 x16+#define s11 x17++// Initial carries for combinations++#define car0 x9+#define car1 x19++// Input and output, plain registers treated according to pattern++#define reg0 x0, #0+#define reg1 x1, #0+#define reg2 x2, #0+#define reg3 x3, #0+#define reg4 x4, #0++#define x x1, #0+#define z x0, #0++// Pointer-offset pairs for temporaries on stack++#define f sp, #0+#define g sp, #(9*N)+#define u sp, #(18*N)+#define v sp, #(27*N)++// Total size to reserve on the stack++#define NSPACE 36*N++// Very similar to a subroutine call to the s2n-bignum word_divstep59.+// But different in register usage and returning the final matrix in+// registers as follows+//+// [ m00  m01]+// [ m10  m11]++#define divstep59()                                                     \+        and     x4, x2, #0xfffff __LF                                      \+        orr     x4, x4, #0xfffffe0000000000 __LF                           \+        and     x5, x3, #0xfffff __LF                                      \+        orr     x5, x5, #0xc000000000000000 __LF                           \+        tst     x5, #0x1 __LF                                              \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        asr     x5, x5, #1 __LF                                            \+        add     x8, x4, #0x100, lsl #12 __LF                               \+        sbfx    x8, x8, #21, #21 __LF                                      \+        mov     x11, #0x100000 __LF                                        \+        add     x11, x11, x11, lsl #21 __LF                                \+        add     x9, x4, x11 __LF                                           \+        asr     x9, x9, #42 __LF                                           \+        add     x10, x5, #0x100, lsl #12 __LF                              \+        sbfx    x10, x10, #21, #21 __LF                                    \+        add     x11, x5, x11 __LF                                          \+        asr     x11, x11, #42 __LF                                         \+        mul     x6, x8, x2 __LF                                            \+        mul     x7, x9, x3 __LF                                            \+        mul     x2, x10, x2 __LF                                           \+        mul     x3, x11, x3 __LF                                           \+        add     x4, x6, x7 __LF                                            \+        add     x5, x2, x3 __LF                                            \+        asr     x2, x4, #20 __LF                                           \+        asr     x3, x5, #20 __LF                                           \+        and     x4, x2, #0xfffff __LF                                      \+        orr     x4, x4, #0xfffffe0000000000 __LF                           \+        and     x5, x3, #0xfffff __LF                                      \+        orr     x5, x5, #0xc000000000000000 __LF                           \+        tst     x5, #0x1 __LF                                              \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        asr     x5, x5, #1 __LF                                            \+        add     x12, x4, #0x100, lsl #12 __LF                              \+        sbfx    x12, x12, #21, #21 __LF                                    \+        mov     x15, #0x100000 __LF                                        \+        add     x15, x15, x15, lsl #21 __LF                                \+        add     x13, x4, x15 __LF                                          \+        asr     x13, x13, #42 __LF                                         \+        add     x14, x5, #0x100, lsl #12 __LF                              \+        sbfx    x14, x14, #21, #21 __LF                                    \+        add     x15, x5, x15 __LF                                          \+        asr     x15, x15, #42 __LF                                         \+        mul     x6, x12, x2 __LF                                           \+        mul     x7, x13, x3 __LF                                           \+        mul     x2, x14, x2 __LF                                           \+        mul     x3, x15, x3 __LF                                           \+        add     x4, x6, x7 __LF                                            \+        add     x5, x2, x3 __LF                                            \+        asr     x2, x4, #20 __LF                                           \+        asr     x3, x5, #20 __LF                                           \+        and     x4, x2, #0xfffff __LF                                      \+        orr     x4, x4, #0xfffffe0000000000 __LF                           \+        and     x5, x3, #0xfffff __LF                                      \+        orr     x5, x5, #0xc000000000000000 __LF                           \+        tst     x5, #0x1 __LF                                              \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        mul     x2, x12, x8 __LF                                           \+        mul     x3, x12, x9 __LF                                           \+        mul     x6, x14, x8 __LF                                           \+        mul     x7, x14, x9 __LF                                           \+        madd    x8, x13, x10, x2 __LF                                      \+        madd    x9, x13, x11, x3 __LF                                      \+        madd    x16, x15, x10, x6 __LF                                     \+        madd    x17, x15, x11, x7 __LF                                     \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        asr     x5, x5, #1 __LF                                            \+        add     x12, x4, #0x100, lsl #12 __LF                              \+        sbfx    x12, x12, #22, #21 __LF                                    \+        mov     x15, #0x100000 __LF                                        \+        add     x15, x15, x15, lsl #21 __LF                                \+        add     x13, x4, x15 __LF                                          \+        asr     x13, x13, #43 __LF                                         \+        add     x14, x5, #0x100, lsl #12 __LF                              \+        sbfx    x14, x14, #22, #21 __LF                                    \+        add     x15, x5, x15 __LF                                          \+        asr     x15, x15, #43 __LF                                         \+        mneg    x2, x12, x8 __LF                                           \+        mneg    x3, x12, x9 __LF                                           \+        mneg    x4, x14, x8 __LF                                           \+        mneg    x5, x14, x9 __LF                                           \+        msub    m00, x13, x16, x2 __LF                                     \+        msub    m01, x13, x17, x3 __LF                                     \+        msub    m10, x15, x16, x4 __LF                                     \+        msub    m11, x15, x17, x5++// Loading large constants++#define movbig(nn,n3,n2,n1,n0)                                      \+        movz    nn, n0 __LF                                            \+        movk    nn, n1, lsl #16 __LF                                   \+        movk    nn, n2, lsl #32 __LF                                   \+        movk    nn, n3, lsl #48++S2N_BN_SYMBOL(bignum_inv_p521):+        CFI_START++// Save registers and make room for temporaries++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_DEC_SP(NSPACE)++// Save the return pointer for the end so we can overwrite x0 later++        mov     res, x0++// Copy the prime p_521 = 2^521 - 1 into the f variable++        mov     x10, #0xFFFFFFFFFFFFFFFF+        stp     x10, x10, [f]+        stp     x10, x10, [f+16]+        stp     x10, x10, [f+32]+        stp     x10, x10, [f+48]+        mov     x11, #0x1FF+        str     x11, [f+64]++// Copy the input into the g variable, but reduce it strictly mod p_521+// so that g <= f as assumed in the bound proof. This code fragment is+// very similar to bignum_mod_p521_9 complete with carry condensation.++        ldr     x8, [x1, #64]+        lsr     x9, x8, #9++        subs    xzr, xzr, xzr+        ldp     x10, x11, [x1]+        adcs    xzr, x10, x9+        adcs    xzr, x11, xzr+        ldp     x12, x13, [x1, #16]+        and     x7, x12, x13+        adcs    xzr, x7, xzr+        ldp     x14, x15, [x1, #32]+        and     x7, x14, x15+        adcs    xzr, x7, xzr+        ldp     x16, x17, [x1, #48]+        and     x7, x16, x17+        adcs    xzr, x7, xzr+        orr     x7, x8, #~0x1FF+        adcs    x7, x7, xzr++        adcs    x10, x10, x9+        adcs    x11, x11, xzr+        adcs    x12, x12, xzr+        adcs    x13, x13, xzr+        adcs    x14, x14, xzr+        adcs    x15, x15, xzr+        adcs    x16, x16, xzr+        adcs    x17, x17, xzr+        adc     x8, x8, xzr+        and     x8, x8, #0x1FF++        stp     x10, x11, [g]+        stp     x12, x13, [g+16]+        stp     x14, x15, [g+32]+        stp     x16, x17, [g+48]+        str     x8, [g+64]++// Also maintain weakly reduced < 2*p_521 vector [u,v] such that+// [f,g] == x * 2^{1239-59*i} * [u,v] (mod p_521)+// starting with [p_521,x] == x * 2^{1239-59*0} * [0,2^-1239] (mod p_521)+// Note that because (2^{a+521} == 2^a) (mod p_521) we simply have+// (2^-1239 == 2^324) (mod p_521) so the constant initializer is simple.+//+// Based on the standard divstep bound, for inputs <= 2^b we need at least+// n >= (9437 * b + 1) / 4096. Since b is 521, that means 1201 iterations.+// Since we package divstep in multiples of 59 bits, we do 21 blocks of 59+// making *1239* total. (With a bit more effort we could avoid the full 59+// divsteps and use a shorter tail computation, but we keep it simple.)+// Hence, after the 21st iteration we have [f,g] == x * [u,v] and since+// |f| = 1 we get the modular inverse from u by flipping its sign with f.++        stp     xzr, xzr, [u]+        stp     xzr, xzr, [u+16]+        stp     xzr, xzr, [u+32]+        stp     xzr, xzr, [u+48]+        str     xzr, [u+64]++        mov     x10, #16+        stp     xzr, xzr, [v]+        stp     xzr, xzr, [v+16]+        stp     xzr, x10, [v+32]+        stp     xzr, xzr, [v+48]+        str     xzr, [v+64]++// Start of main loop. We jump into the middle so that the divstep+// portion is common to the special 21st iteration after a uniform+// first 20.++        mov     i, #21+        mov     d, #1+        b       Lbignum_inv_p521_midloop++Lbignum_inv_p521_loop:++// Separate the matrix elements into sign-magnitude pairs++        cmp     m00, xzr+        csetm   s00, mi+        cneg    m00, m00, mi++        cmp     m01, xzr+        csetm   s01, mi+        cneg    m01, m01, mi++        cmp     m10, xzr+        csetm   s10, mi+        cneg    m10, m10, mi++        cmp     m11, xzr+        csetm   s11, mi+        cneg    m11, m11, mi++// Adjust the initial values to allow for complement instead of negation+// This initial offset is the same for [f,g] and [u,v] compositions.+// Save it in stable registers for the [u,v] part and do [f,g] first.++        and     x0, m00, s00+        and     x1, m01, s01+        add     car0, x0, x1++        and     x0, m10, s10+        and     x1, m11, s11+        add     car1, x0, x1++// Now the computation of the updated f and g values. This maintains a+// 2-word carry between stages so we can conveniently insert the shift+// right by 59 before storing back, and not overwrite digits we need+// again of the old f and g values.+//+// Digit 0 of [f,g]++        ldr     x7, [f]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x4, car0, x0+        adc     x2, xzr, x1+        ldr     x8, [g]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x4, x4, x0+        adc     x2, x2, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x5, car1, x0+        adc     x3, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x5, x5, x0+        adc     x3, x3, x1++// Digit 1 of [f,g]++        ldr     x7, [f+N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [g+N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [f]++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [g]++// Digit 2 of [f,g]++        ldr     x7, [f+2*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [g+2*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [f+N]++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [g+N]++// Digit 3 of [f,g]++        ldr     x7, [f+3*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x5, x5, x0+        adc     x3, xzr, x1+        ldr     x8, [g+3*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x5, x6, #59+        str     x6, [f+2*N]++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [g+2*N]++// Digit 4 of [f,g]++        ldr     x7, [f+4*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x3, x3, x0+        adc     x4, xzr, x1+        ldr     x8, [g+4*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [f+3*N]++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [g+3*N]++// Digit 5 of [f,g]++        ldr     x7, [f+5*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x4, x4, x0+        adc     x2, xzr, x1+        ldr     x8, [g+5*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [f+4*N]++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x5, x5, x0+        adc     x3, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x5, x6, #59+        str     x6, [g+4*N]++// Digit 6 of [f,g]++        ldr     x7, [f+6*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [g+6*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [f+5*N]++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [g+5*N]++// Digit 7 of [f,g]++        ldr     x7, [f+7*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [g+7*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [f+6*N]++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [g+6*N]++// Digits 8 and 9 of [f,g]++        ldr     x7, [f+8*N]+        eor     x1, x7, s00+        asr     x3, x1, #63+        and     x3, x3, m00+        neg     x3, x3+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [g+8*N]+        eor     x1, x8, s01+        asr     x0, x1, #63+        and     x0, x0, m01+        sub     x3, x3, x0+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x5, x6, #59+        str     x6, [f+7*N]+        extr    x5, x3, x5, #59+        str     x5, [f+8*N]++        eor     x1, x7, s10+        asr     x5, x1, #63+        and     x5, x5, m10+        neg     x5, x5+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, s11+        asr     x0, x1, #63+        and     x0, x0, m11+        sub     x5, x5, x0+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x4, x2, x4, #59+        str     x4, [g+7*N]+        extr    x2, x5, x2, #59+        str     x2, [g+8*N]++// Now the computation of the updated u and v values and their+// modular reductions. A very similar accumulation except that+// the top words of u and v are unsigned and we don't shift.+//+// Digit 0 of [u,v]++        ldr     x7, [u]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x4, car0, x0+        adc     x2, xzr, x1+        ldr     x8, [v]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x4, x4, x0+        str     x4, [u]+        adc     x2, x2, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x5, car1, x0+        adc     x3, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x5, x5, x0+        str     x5, [v]+        adc     x3, x3, x1++// Digit 1 of [u,v]++        ldr     x7, [u+N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [v+N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x2, x2, x0+        str     x2, [u+N]+        adc     x6, x6, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x3, x3, x0+        str     x3, [v+N]+        adc     x4, x4, x1++// Digit 2 of [u,v]++        ldr     x7, [u+2*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [v+2*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x6, x6, x0+        str     x6, [u+2*N]+        adc     x5, x5, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x4, x4, x0+        str     x4, [v+2*N]+        adc     x2, x2, x1++// Digit 3 of [u,v]++        ldr     x7, [u+3*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x5, x5, x0+        adc     x3, xzr, x1+        ldr     x8, [v+3*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x5, x5, x0+        str     x5, [u+3*N]+        adc     x3, x3, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x2, x2, x0+        str     x2, [v+3*N]+        adc     x6, x6, x1++// Digit 4 of [u,v]++        ldr     x7, [u+4*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x3, x3, x0+        adc     x4, xzr, x1+        ldr     x8, [v+4*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x3, x3, x0+        str     x3, [u+4*N]+        adc     x4, x4, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x6, x6, x0+        str     x6, [v+4*N]+        adc     x5, x5, x1++// Digit 5 of [u,v]++        ldr     x7, [u+5*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x4, x4, x0+        adc     x2, xzr, x1+        ldr     x8, [v+5*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x4, x4, x0+        str     x4, [u+5*N]+        adc     x2, x2, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x5, x5, x0+        adc     x3, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x5, x5, x0+        str     x5, [v+5*N]+        adc     x3, x3, x1++// Digit 6 of [u,v]++        ldr     x7, [u+6*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [v+6*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x2, x2, x0+        str     x2, [u+6*N]+        adc     x6, x6, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x3, x3, x0+        str     x3, [v+6*N]+        adc     x4, x4, x1++// Digit 7 of [u,v]++        ldr     x7, [u+7*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [v+7*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x6, x6, x0+        str     x6, [u+7*N]+        adc     x5, x5, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x4, x4, x0+        str     x4, [v+7*N]+        adc     x2, x2, x1++// Digits 8 and 9 of u (top is unsigned)++        ldr     x7, [u+8*N]+        eor     x1, x7, s00+        and     x3, s00, m00+        neg     x3, x3+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [v+8*N]+        eor     x1, x8, s01+        and     x0, s01, m01+        sub     x3, x3, x0+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x5, x5, x0+        adc     x3, x3, x1++// Modular reduction of u, reloading as needed from u[0],...,u[7],x5,x3++        extr    x6, x3, x5, #9+        ldp     x0, x1, [u]+        add     x6, x6, x3, asr #63+        sub     x5, x5, x6, lsl #9+        adds    x0, x0, x6+        asr     x6, x6, #63+        adcs    x1, x1, x6+        stp     x0, x1, [u]+        ldp     x0, x1, [u+16]+        adcs    x0, x0, x6+        adcs    x1, x1, x6+        stp     x0, x1, [u+16]+        ldp     x0, x1, [u+32]+        adcs    x0, x0, x6+        adcs    x1, x1, x6+        stp     x0, x1, [u+32]+        ldp     x0, x1, [u+48]+        adcs    x0, x0, x6+        adcs    x1, x1, x6+        stp     x0, x1, [u+48]+        adc     x5, x5, x6+        str     x5, [u+64]++// Digits 8 and 9 of v (top is unsigned)++        eor     x1, x7, s10+        and     x5, s10, m10+        neg     x5, x5+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, s11+        and     x0, s11, m11+        sub     x5, x5, x0+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x2, x2, x0+        adc     x5, x5, x1++// Modular reduction of v, reloading as needed from v[0],...,v[7],x2,x5++        extr    x6, x5, x2, #9+        ldp     x0, x1, [v]+        add     x6, x6, x5, asr #63+        sub     x2, x2, x6, lsl #9+        adds    x0, x0, x6+        asr     x6, x6, #63+        adcs    x1, x1, x6+        stp     x0, x1, [v]+        ldp     x0, x1, [v+16]+        adcs    x0, x0, x6+        adcs    x1, x1, x6+        stp     x0, x1, [v+16]+        ldp     x0, x1, [v+32]+        adcs    x0, x0, x6+        adcs    x1, x1, x6+        stp     x0, x1, [v+32]+        ldp     x0, x1, [v+48]+        adcs    x0, x0, x6+        adcs    x1, x1, x6+        stp     x0, x1, [v+48]+        adc     x2, x2, x6+        str     x2, [v+64]++Lbignum_inv_p521_midloop:++        mov     x1, d+        ldr     x2, [f]+        ldr     x3, [g]+        divstep59()+        mov     d, x1++// Next iteration++        subs    i, i, #1+        bne     Lbignum_inv_p521_loop++// The 21st and last iteration does not need anything except the+// u value and the sign of f; the latter can be obtained from the+// lowest word of f. So it's done differently from the main loop.+// Find the sign of the new f. For this we just need one digit+// since we know (for in-scope cases) that f is either +1 or -1.+// We don't explicitly shift right by 59 either, but looking at+// bit 63 (or any bit >= 60) of the unshifted result is enough+// to distinguish -1 from +1; this is then made into a mask.++        ldr     x0, [f]+        ldr     x1, [g]+        mul     x0, x0, m00+        madd    x1, x1, m01, x0+        asr     x0, x1, #63++// Now separate out the matrix into sign-magnitude pairs+// and adjust each one based on the sign of f.+//+// Note that at this point we expect |f|=1 and we got its+// sign above, so then since [f,0] == x * [u,v] (mod p_521)+// we want to flip the sign of u according to that of f.++        cmp     m00, xzr+        csetm   s00, mi+        cneg    m00, m00, mi+        eor     s00, s00, x0++        cmp     m01, xzr+        csetm   s01, mi+        cneg    m01, m01, mi+        eor     s01, s01, x0++        cmp     m10, xzr+        csetm   s10, mi+        cneg    m10, m10, mi+        eor     s10, s10, x0++        cmp     m11, xzr+        csetm   s11, mi+        cneg    m11, m11, mi+        eor     s11, s11, x0++// Adjust the initial value to allow for complement instead of negation++        and     x0, m00, s00+        and     x1, m01, s01+        add     car0, x0, x1++// Digit 0 of [u]++        ldr     x7, [u]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x4, car0, x0+        adc     x2, xzr, x1+        ldr     x8, [v]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x4, x4, x0+        str     x4, [u]+        adc     x2, x2, x1++// Digit 1 of [u]++        ldr     x7, [u+N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [v+N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x2, x2, x0+        str     x2, [u+N]+        adc     x6, x6, x1++// Digit 2 of [u]++        ldr     x7, [u+2*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [v+2*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x6, x6, x0+        str     x6, [u+2*N]+        adc     x5, x5, x1++// Digit 3 of [u]++        ldr     x7, [u+3*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x5, x5, x0+        adc     x3, xzr, x1+        ldr     x8, [v+3*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x5, x5, x0+        str     x5, [u+3*N]+        adc     x3, x3, x1++// Digit 4 of [u]++        ldr     x7, [u+4*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x3, x3, x0+        adc     x4, xzr, x1+        ldr     x8, [v+4*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x3, x3, x0+        str     x3, [u+4*N]+        adc     x4, x4, x1++// Digit 5 of [u]++        ldr     x7, [u+5*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x4, x4, x0+        adc     x2, xzr, x1+        ldr     x8, [v+5*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x4, x4, x0+        str     x4, [u+5*N]+        adc     x2, x2, x1++// Digit 6 of [u]++        ldr     x7, [u+6*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [v+6*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x2, x2, x0+        str     x2, [u+6*N]+        adc     x6, x6, x1++// Digit 7 of [u]++        ldr     x7, [u+7*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [v+7*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x6, x6, x0+        str     x6, [u+7*N]+        adc     x5, x5, x1++// Digits 8 and 9 of u (top is unsigned)++        ldr     x7, [u+8*N]+        eor     x1, x7, s00+        and     x3, s00, m00+        neg     x3, x3+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [v+8*N]+        eor     x1, x8, s01+        and     x0, s01, m01+        sub     x3, x3, x0+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x5, x5, x0+        adc     x3, x3, x1++// Modular reduction of u, reloading as needed from u[0],...,u[7],x5,x3++        extr    x6, x3, x5, #9+        ldp     x10, x11, [u]+        add     x6, x6, x3, asr #63+        sub     x5, x5, x6, lsl #9+        adds    x10, x10, x6+        asr     x6, x6, #63+        adcs    x11, x11, x6+        ldp     x12, x13, [u+16]+        adcs    x12, x12, x6+        adcs    x13, x13, x6+        ldp     x14, x15, [u+32]+        adcs    x14, x14, x6+        adcs    x15, x15, x6+        ldp     x16, x17, [u+48]+        adcs    x16, x16, x6+        adcs    x17, x17, x6+        adc     x19, x5, x6++// Further strict reduction ready for the output, which just means+// a conditional subtraction of p_521++        subs    x0, x10, #-1+        adcs    x1, x11, xzr+        adcs    x2, x12, xzr+        adcs    x3, x13, xzr+        adcs    x4, x14, xzr+        adcs    x5, x15, xzr+        adcs    x6, x16, xzr+        adcs    x7, x17, xzr+        mov     x8, #0x1FF+        sbcs    x8, x19, x8++        csel    x0, x0, x10, cs+        csel    x1, x1, x11, cs+        csel    x2, x2, x12, cs+        csel    x3, x3, x13, cs+        csel    x4, x4, x14, cs+        csel    x5, x5, x15, cs+        csel    x6, x6, x16, cs+        csel    x7, x7, x17, cs+        csel    x8, x8, x19, cs++// Store it back to the final output++        stp     x0, x1, [res]+        stp     x2, x3, [res, #16]+        stp     x4, x5, [res, #32]+        stp     x6, x7, [res, #48]+        str     x8, [res, #64]++// Restore stack and registers++        CFI_INC_SP(NSPACE)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_inv_p521)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/bignum_modinv.S view
@@ -0,0 +1,613 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Invert modulo m, z = (1/a) mod b, assuming b is an odd number > 1, coprime a+// Inputs a[k], b[k]; output z[k]; temporary buffer t[>=3*k]+//+//    extern void bignum_modinv(uint64_t k, uint64_t *z, const uint64_t *a,+//                              const uint64_t *b, uint64_t *t);+//+// k-digit (digit=64 bits) "z := a^-1 mod b" (modular inverse of a modulo b)+// using t as a temporary buffer (t at least 3*k words = 24*k bytes), and+// assuming that a and b are coprime *and* that b is an odd number > 1.+//+// Standard ARM ABI: X0 = k, X1 = z, X2 = a, X3 = b, X4 = t+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_modinv)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_modinv)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_modinv)+        .text+        .balign 4++// We get CHUNKSIZE bits per outer iteration, 64 minus a few for proxy errors++#define CHUNKSIZE 58++// Pervasive variables++#define k x0+#define z x1+#define b x3+#define w x4++// This one is recycled after initial copying in of a as outer loop counter++#define a x2+#define t x2++// Additional variables; later ones are currently rather high regs++#define l x5++#define m x21+#define n x22++// The matrix of update factors to apply to m and n+// Also used a couple of additional temporary variables for the swapping loop+// Also used as an extra down-counter in corrective negation loops++#define m_m x6+#define m_n x7+#define n_m x8+#define n_n x9++#define j x6++// General temporary variables and loop counters++#define i x10+#define t1 x11+#define t2 x12++// High and low proxies for the inner loop+// Then re-used for high and carry words during actual cross-multiplications++#define m_hi x13+#define n_hi x14+#define m_lo x15+#define n_lo x16++#define h1 x13+#define h2 x14+#define l1 x15+#define l2 x16++#define c1 x17+#define c2 x19++// Negated modular inverse for Montgomery++#define v x20++// Some more intuitive names for temp regs in initial word-level negmodinv.+// These just use t1 and t2 again, though carefully since t1 = initial b[0]++#define one t2+#define e1 t2+#define e2 t1+#define e4 t2+#define e8 t1++S2N_BN_SYMBOL(bignum_modinv):+        CFI_START++// We make use of registers beyond the modifiable++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)++// If k = 0 then do nothing (this is out of scope anyway)++        cbz     k, Lbignum_modinv_end++// Set up the additional two buffers m and n beyond w in temp space++        lsl     i, k, #3+        add     m, w, i+        add     n, m, i++// Initialize the main buffers with their starting values:+// m = a, n = b, w = b (to be tweaked to b - 1) and z = 0++        mov     i, xzr+Lbignum_modinv_copyloop:+        ldr     t1, [a, i, lsl #3]+        ldr     t2, [b, i, lsl #3]+        str     t1, [m, i, lsl #3]+        str     t2, [n, i, lsl #3]+        str     t2, [w, i, lsl #3]+        str     xzr, [z, i, lsl #3]+        add     i, i, #1+        cmp     i, k+        bcc     Lbignum_modinv_copyloop++// Tweak down w to b - 1 (this crude approach is safe as b needs to be odd+// for it to be in scope). We have then established the congruence invariant:+//+//   a * w == -m (mod b)+//   a * z == n (mod b)+//+// This, with the bound w <= b and z <= b, is maintained round the outer loop++        ldr     t1, [w]+        sub     t2, t1, #1+        str     t2, [w]++// Compute v = negated modular inverse of b mod 2^64, reusing t1 from above+// This is used for Montgomery reduction operations each time round the loop++        lsl     v, t1, #2+        sub     v, t1, v+        eor     v, v, #2+        mov     one, #1+        madd    e1, t1, v, one+        mul     e2, e1, e1+        madd    v, e1, v, v+        mul     e4, e2, e2+        madd    v, e2, v, v+        mul     e8, e4, e4+        madd    v, e4, v, v+        madd    v, e8, v, v++// Set up the outer loop count of 128 * k+// The invariant is that m * n < 2^t at all times.++        lsl     t, k, #7++// Start of the main outer loop iterated t / CHUNKSIZE times++Lbignum_modinv_outerloop:++// We need only bother with sharper l = min k (ceil(t/64)) digits+// for the computations on m and n (but we still need k for w and z).+// Either both m and n fit in l digits, or m has become zero and so+// nothing happens in the loop anyway and this makes no difference.++        add     i, t, #63+        lsr     l, i, #6+        cmp     l, k+        csel    l, k, l, cs++// Select upper and lower proxies for both m and n to drive the inner+// loop. The lower proxies are simply the lowest digits themselves,+// m_lo = m[0] and n_lo = n[0], while the upper proxies are bitfields+// of the two inputs selected so their top bit (63) aligns with the+// most significant bit of *either* of the two inputs.++        mov     h1, xzr // Previous high and low for m+        mov     l1, xzr+        mov     h2, xzr // Previous high and low for n+        mov     l2, xzr+        mov     c2, xzr // Mask flag: previous word of one was nonzero+        // and in this case h1 and h2 are those words+        mov     i, xzr+Lbignum_modinv_toploop:+        ldr     t1, [m, i, lsl #3]+        ldr     t2, [n, i, lsl #3]+        orr     c1, t1, t2+        cmp     c1, xzr+        and     c1, c2, h1+        csel    l1, c1, l1, ne+        and     c1, c2, h2+        csel    l2, c1, l2, ne+        csel    h1, t1, h1, ne+        csel    h2, t2, h2, ne+        csetm   c2, ne+        add     i, i, #1+        cmp     i, l+        bcc     Lbignum_modinv_toploop++        orr     t1, h1, h2+        clz     t2, t1+        negs    c1, t2+        lsl     h1, h1, t2+        csel    l1, l1, xzr, ne+        lsl     h2, h2, t2+        csel    l2, l2, xzr, ne+        lsr     l1, l1, c1+        lsr     l2, l2, c1+        orr     m_hi, h1, l1+        orr     n_hi, h2, l2++        ldr     m_lo, [m]+        ldr     n_lo, [n]++// Now the inner loop, with i as loop counter from CHUNKSIZE down.+// This records a matrix of updates to apply to the initial+// values of m and n with, at stage j:+//+//     sgn * m' = (m_m * m - m_n * n) / 2^j+//    -sgn * n' = (n_m * m - n_n * n) / 2^j+//+// where "sgn" is either +1 or -1, and we lose track of which except+// that both instance above are the same. This throwing away the sign+// costs nothing (since we have to correct in general anyway because+// of the proxied comparison) and makes things a bit simpler. But it+// is simply the parity of the number of times the first condition,+// used as the swapping criterion, fires in this loop.++        mov     m_m, #1+        mov     m_n, xzr+        mov     n_m, xzr+        mov     n_n, #1++        mov     i, #CHUNKSIZE++// Conceptually in the inner loop we follow these steps:+//+// * If m_lo is odd and m_hi < n_hi, then swap the four pairs+//    (m_hi,n_hi); (m_lo,n_lo); (m_m,n_m); (m_n,n_n)+//+// * Now, if m_lo is odd (old or new, doesn't matter as initial n_lo is odd)+//    m_hi := m_hi - n_hi, m_lo := m_lo - n_lo+//    m_m  := m_m + n_m, m_n := m_n + n_n+//+// * Halve and double them+//     m_hi := m_hi / 2, m_lo := m_lo / 2+//     n_m := n_m * 2, n_n := n_n * 2+//+// The actual computation computes updates before actually swapping and+// then corrects as needed. It also maintains the invariant ~ZF <=> odd(m_lo),+// since it seems to reduce the dependent latency. Set that up first.++        ands    xzr, m_lo, #1++Lbignum_modinv_innerloop:++// At the start of the loop ~ZF <=> m_lo is odd; mask values accordingly+// Set the flags for m_hi - [~ZF] * n_hi so we know to flip things.++        csel    t1, n_hi, xzr, ne+        csel    t2, n_lo, xzr, ne+        csel    c1, n_m, xzr, ne+        csel    c2, n_n, xzr, ne+        ccmp    m_hi, n_hi, #0x2, ne++// Compute subtractive updates, trivial in the case ZF <=> even(m_lo).++        sub     t1, m_hi, t1+        sub     t2, m_lo, t2++// If the subtraction borrows, swap things appropriately, negating where+// we've already subtracted so things are as if we actually swapped first.++        csel    n_hi, n_hi, m_hi, cs+        cneg    t1, t1, cc+        csel    n_lo, n_lo, m_lo, cs+        cneg    m_lo, t2, cc+        csel    n_m, n_m, m_m, cs+        csel    n_n, n_n, m_n, cs++// Update and shift while setting oddness flag for next iteration+// We look at bit 1 of t2 (m_lo before possible negation), which is+// safe because it is even.++        ands    xzr, t2, #2+        add     m_m, m_m, c1+        add     m_n, m_n, c2+        lsr     m_hi, t1, #1+        lsr     m_lo, m_lo, #1+        add     n_m, n_m, n_m+        add     n_n, n_n, n_n++// Next iteration; don't disturb the flags since they are used at entry++        sub     i, i, #1+        cbnz    i, Lbignum_modinv_innerloop++// Apply the update to w and z, using addition in this case, and also take+// the chance to shift an additional 6 = 64-CHUNKSIZE bits to be ready for a+// Montgomery multiplication. Because we know that m_m + m_n <= 2^58 and+// w, z <= b < 2^{64k}, we know that both of these fit in k+1 words.+// We do this before the m-n update to allow us to play with c1 and c2 here.+//+//    h1::w = 2^6 * (m_m * w + m_n * z)+//    h2::z = 2^6 * (n_m * w + n_n * z)+//+// with c1 and c2 recording previous words for the shifting part++        mov     h1, xzr+        mov     h2, xzr+        mov     c1, xzr+        mov     c2, xzr++        mov     i, xzr+Lbignum_modinv_congloop:+        ldr     t1, [w, i, lsl #3]+        ldr     t2, [z, i, lsl #3]++        mul     l1, m_m, t1+        mul     l2, m_n, t2+        adds    l1, l1, h1+        umulh   h1, m_m, t1+        adc     h1, h1, xzr+        adds    l1, l1, l2+        extr    c1, l1, c1, #CHUNKSIZE+        str     c1, [w, i, lsl #3]+        mov     c1, l1+        umulh   l1, m_n, t2+        adc     h1, h1, l1++        mul     l1, n_m, t1+        mul     l2, n_n, t2+        adds    l1, l1, h2+        umulh   h2, n_m, t1+        adc     h2, h2, xzr+        adds    l1, l1, l2+        extr    c2, l1, c2, #CHUNKSIZE+        str     c2, [z, i, lsl #3]+        mov     c2, l1+        umulh   l1, n_n, t2+        adc     h2, h2, l1++        add     i, i, #1+        cmp     i, k+        bcc     Lbignum_modinv_congloop++        extr    h1, h1, c1, #CHUNKSIZE+        extr    h2, h2, c2, #CHUNKSIZE++// Do a Montgomery reduction of h1::w++        ldr     t1, [w]+        mul     c1, t1, v+        ldr     t2, [b]+        mul     l1, c1, t2+        umulh   l2, c1, t2+        adds    t1, t1, l1      // Will be zero but want the carry++        mov     i, #1+        sub     t1, k, #1+        cbz     t1, Lbignum_modinv_wmontend+Lbignum_modinv_wmontloop:+        ldr     t1, [b, i, lsl #3]+        ldr     t2, [w, i, lsl #3]+        mul     l1, c1, t1+        adcs    t2, t2, l2+        umulh   l2, c1, t1+        adc     l2, l2, xzr+        adds    t2, t2, l1+        sub     l1, i, #1+        str     t2, [w, l1, lsl #3]+        add     i, i, #1+        sub     t1, i, k+        cbnz    t1, Lbignum_modinv_wmontloop+Lbignum_modinv_wmontend:+        adcs    l2, l2, h1+        adc     h1, xzr, xzr+        sub     l1, i, #1+        str     l2, [w, l1, lsl #3]++        subs    i, xzr, xzr+Lbignum_modinv_wcmploop:+        ldr     t1, [w, i, lsl #3]+        ldr     t2, [b, i, lsl #3]+        sbcs    xzr, t1, t2+        add     i, i, #1+        sub     t1, i, k+        cbnz    t1, Lbignum_modinv_wcmploop++        sbcs    xzr, h1, xzr+        csetm   h1, cs++        subs    i, xzr, xzr+Lbignum_modinv_wcorrloop:+        ldr     t1, [w, i, lsl #3]+        ldr     t2, [b, i, lsl #3]+        and     t2, t2, h1+        sbcs    t1, t1, t2+        str     t1, [w, i, lsl #3]+        add     i, i, #1+        sub     t1, i, k+        cbnz    t1, Lbignum_modinv_wcorrloop++// Do a Montgomery reduction of h2::z++        ldr     t1, [z]+        mul     c1, t1, v+        ldr     t2, [b]+        mul     l1, c1, t2+        umulh   l2, c1, t2+        adds    t1, t1, l1      // Will be zero but want the carry++        mov     i, #1+        sub     t1, k, #1+        cbz     t1, Lbignum_modinv_zmontend+Lbignum_modinv_zmontloop:+        ldr     t1, [b, i, lsl #3]+        ldr     t2, [z, i, lsl #3]+        mul     l1, c1, t1+        adcs    t2, t2, l2+        umulh   l2, c1, t1+        adc     l2, l2, xzr+        adds    t2, t2, l1+        sub     l1, i, #1+        str     t2, [z, l1, lsl #3]+        add     i, i, #1+        sub     t1, i, k+        cbnz    t1, Lbignum_modinv_zmontloop+Lbignum_modinv_zmontend:+        adcs    l2, l2, h2+        adc     h2, xzr, xzr+        sub     l1, i, #1+        str     l2, [z, l1, lsl #3]++        subs    i, xzr, xzr+Lbignum_modinv_zcmploop:+        ldr     t1, [z, i, lsl #3]+        ldr     t2, [b, i, lsl #3]+        sbcs    xzr, t1, t2+        add     i, i, #1+        sub     t1, i, k+        cbnz    t1, Lbignum_modinv_zcmploop++        sbcs    xzr, h2, xzr+        csetm   h2, cs++        subs    i, xzr, xzr+Lbignum_modinv_zcorrloop:+        ldr     t1, [z, i, lsl #3]+        ldr     t2, [b, i, lsl #3]+        and     t2, t2, h2+        sbcs    t1, t1, t2+        str     t1, [z, i, lsl #3]+        add     i, i, #1+        sub     t1, i, k+        cbnz    t1, Lbignum_modinv_zcorrloop++// Now actually compute the updates to m and n corresponding to the matrix,+// and correct the signs if they have gone negative. First we compute the+// (k+1)-sized updates with the following invariant (here c1 and c2 are in+// fact carry bitmasks, either 0 or -1):+//+//    c1::h1::m = m_m * m - m_n * n+//    c2::h2::n = n_m * m - n_n * n++        mov     h1, xzr+        mov     h2, xzr+        mov     c1, xzr+        mov     c2, xzr+        mov     i, xzr+Lbignum_modinv_crossloop:+        ldr     t1, [m, i, lsl #3]+        ldr     t2, [n, i, lsl #3]++        mul     l1, m_m, t1+        mul     l2, m_n, t2+        adds    l1, l1, h1+        umulh   h1, m_m, t1+        adc     h1, h1, xzr+        subs    l1, l1, l2+        str     l1, [m, i, lsl #3]+        umulh   l1, m_n, t2+        sub     c1, l1, c1+        sbcs    h1, h1, c1+        csetm   c1, cc++        mul     l1, n_m, t1+        mul     l2, n_n, t2+        adds    l1, l1, h2+        umulh   h2, n_m, t1+        adc     h2, h2, xzr+        subs    l1, l1, l2+        str     l1, [n, i, lsl #3]+        umulh   l1, n_n, t2+        sub     c2, l1, c2+        sbcs    h2, h2, c2+        csetm   c2, cc++        add     i, i, #1+        cmp     i, l+        bcc     Lbignum_modinv_crossloop++// Write back m optionally negated and shifted right CHUNKSIZE bits++        adds    xzr, c1, c1++        ldr     l1, [m]+        mov     i, xzr+        sub     j, l, #1+        cbz     j, Lbignum_modinv_negskip1++Lbignum_modinv_negloop1:+        add     t1, i, #8+        ldr     t2, [m, t1]+        extr    l1, t2, l1, #CHUNKSIZE+        eor     l1, l1, c1+        adcs    l1, l1, xzr+        str     l1, [m, i]+        mov     l1, t2+        add     i, i, #8+        sub     j, j, #1+        cbnz    j, Lbignum_modinv_negloop1+Lbignum_modinv_negskip1:+        extr    l1, h1, l1, #CHUNKSIZE+        eor     l1, l1, c1+        adcs    l1, l1, xzr+        str     l1, [m, i]++// Write back n optionally negated and shifted right CHUNKSIZE bits++        adds    xzr, c2, c2++        ldr     l1, [n]+        mov     i, xzr+        sub     j, l, #1+        cbz     j, Lbignum_modinv_negskip2+Lbignum_modinv_negloop2:+        add     t1, i, #8+        ldr     t2, [n, t1]+        extr    l1, t2, l1, #CHUNKSIZE+        eor     l1, l1, c2+        adcs    l1, l1, xzr+        str     l1, [n, i]+        mov     l1, t2+        add     i, i, #8+        sub     j, j, #1+        cbnz    j, Lbignum_modinv_negloop2+Lbignum_modinv_negskip2:+        extr    l1, h2, l1, #CHUNKSIZE+        eor     l1, l1, c2+        adcs    l1, l1, xzr+        str     l1, [n, i]++// Finally, use the signs c1 and c2 to do optional modular negations of+// w and z respectively, flipping c2 to make signs work. We don't make+// any checks for zero values, but we certainly retain w <= b and z <= b.+// This is enough for the Montgomery step in the next iteration to give+// strict reduction w < b amd z < b, and anyway when we terminate we+// could not have z = b since it violates the coprimality assumption for+// in-scope cases.++        mov     i, xzr+        adds    xzr, c1, c1+Lbignum_modinv_wfliploop:+        ldr     t1, [b, i, lsl #3]+        ldr     t2, [w, i, lsl #3]+        and     t1, t1, c1+        eor     t2, t2, c1+        adcs    t1, t1, t2+        str     t1, [w, i, lsl #3]+        add     i, i, #1+        sub     t1, i, k+        cbnz    t1, Lbignum_modinv_wfliploop++        mvn     c2, c2++        mov     i, xzr+        adds    xzr, c2, c2+Lbignum_modinv_zfliploop:+        ldr     t1, [b, i, lsl #3]+        ldr     t2, [z, i, lsl #3]+        and     t1, t1, c2+        eor     t2, t2, c2+        adcs    t1, t1, t2+        str     t1, [z, i, lsl #3]+        add     i, i, #1+        sub     t1, i, k+        cbnz    t1, Lbignum_modinv_zfliploop++// End of main loop. We can stop if t' <= 0 since then m * n < 2^0, which+// since n is odd and m and n are coprime (in the in-scope cases) means+// m = 0, n = 1 and hence from the congruence invariant a * z == 1 (mod b).+// Moreover we do in fact need to maintain strictly t > 0 in the main loop,+// or the computation of the optimized digit bound l could collapse to 0.++        subs    t, t, #CHUNKSIZE+        bhi     Lbignum_modinv_outerloop++Lbignum_modinv_end:+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_modinv)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/bignum_montinv_p384.S view
@@ -0,0 +1,1493 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Montgomery inverse modulo p_384 = 2^384 - 2^128 - 2^96 + 2^32 - 1+// Input x[6]; output z[6]+//+// extern void bignum_montinv_p384(uint64_t z[static 6],+//                                 const uint64_t x[static 6]);+//+// If the 6-digit input x is coprime to p_384, i.e. is not divisible+// by it, returns z < p_384 such that x * z == 2^768 (mod p_384). This+// is effectively "Montgomery inverse" because if we consider x and z as+// Montgomery forms of X and Z, i.e. x == 2^384 * X and z == 2^384 * Z+// (both mod p_384) then X * Z == 1 (mod p_384). That is, this function+// gives the analog of the modular inverse bignum_inv_p384 but with both+// input and output in the Montgomery domain. Note that x does not need+// to be reduced modulo p_384, but the output always is. If the input+// is divisible (i.e. is 0 or p_384), then there can be no solution to+// the congruence x * z == 2^768 (mod p_384), and z = 0 is returned.+//+// Standard ARM ABI: X0 = z, X1 = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_montinv_p384)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_montinv_p384)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_montinv_p384)++        .text+        .balign 4++// Size in bytes of a 64-bit word++#define N 8++// Used for the return pointer++#define res x20++// Loop counter and d = 2 * delta value for divstep++#define i x21+#define d x22++// Registers used for matrix element magnitudes and signs++#define m00 x10+#define m01 x11+#define m10 x12+#define m11 x13+#define s00 x14+#define s01 x15+#define s10 x16+#define s11 x17++// Initial carries for combinations++#define car0 x9+#define car1 x19++// Input and output, plain registers treated according to pattern++#define reg0 x0, #0+#define reg1 x1, #0+#define reg2 x2, #0+#define reg3 x3, #0+#define reg4 x4, #0++#define x x1, #0+#define z x0, #0++// Pointer-offset pairs for temporaries on stack+// The u and v variables are 6 words each as expected, but the f and g+// variables are 8 words each -- they need to have at least one extra+// word for a sign word, and to preserve alignment we "round up" to 8.+// In fact, we currently keep an extra word in u and v as well.++#define f sp, #0+#define g sp, #(8*N)+#define u sp, #(16*N)+#define v sp, #(24*N)++// Total size to reserve on the stack++#define NSPACE 32*N++// ---------------------------------------------------------------------------+// Core signed almost-Montgomery reduction macro. Takes input in+// [d6;d5;d4;d3;d2;d1;d0] and returns result in [d6;d5d4;d3;d2;d1], adding+// to the existing [d6;d5;d4;d3;d2;d1], and re-using d0 as a temporary+// internally as well as t0, t1, t2. This is almost-Montgomery, i.e. the+// result fits in 6 digits but is not necessarily strictly reduced mod p_384.+// ---------------------------------------------------------------------------++#define amontred(d6,d5,d4,d3,d2,d1,d0, t3,t2,t1)                            \+/* We only know the input is -2^444 < x < 2^444. To do traditional  */      \+/* unsigned Montgomery reduction, start by adding 2^61 * p_384.     */      \+        mov     t1, #0xe000000000000000 __LF                           \+        adds    d0, d0, t1 __LF                                        \+        mov     t2, #0x000000001fffffff __LF                           \+        adcs    d1, d1, t2 __LF                                        \+        mov     t3, #0xffffffffe0000000 __LF                           \+        bic     t3, t3, #0x2000000000000000 __LF                       \+        adcs    d2, d2, t3 __LF                                        \+        sbcs    d3, d3, xzr __LF                                       \+        sbcs    d4, d4, xzr __LF                                       \+        sbcs    d5, d5, xzr __LF                                       \+        mov     t1, #0x1fffffffffffffff __LF                           \+        adc     d6, d6, t1 __LF                                        \+/* Our correction multiplier is w = [d0 + (d0<<32)] mod 2^64  */    \+/* Store it back into d0 since we no longer need that digit.  */    \+        add     d0, d0, d0, lsl #32 __LF                               \+/* Now let [t3;t2;t1;-] = (2^384 - p_384) * w                 */    \+/* We know the lowest word will cancel d0 so we don't need it */    \+        mov     t1, #0xffffffff00000001 __LF                           \+        umulh   t1, t1, d0 __LF                                        \+        mov     t2, #0x00000000ffffffff __LF                           \+        mul     t3, t2, d0 __LF                                        \+        umulh   t2, t2, d0 __LF                                        \+        adds    t1, t1, t3 __LF                                        \+        adcs    t2, t2, d0 __LF                                        \+        cset    t3, cs __LF                                            \+/* Now x + p_384 * w = (x + 2^384 * w) - (2^384 - p_384) * w */     \+/* We catch the net top carry from add-subtract in the digit d0 */  \+        adds    d6, d6, d0 __LF                                        \+        cset    d0, cs __LF                                            \+        subs    d1, d1, t1 __LF                                        \+        sbcs    d2, d2, t2 __LF                                        \+        sbcs    d3, d3, t3 __LF                                        \+        sbcs    d4, d4, xzr __LF                                       \+        sbcs    d5, d5, xzr __LF                                       \+        sbcs    d6, d6, xzr __LF                                       \+        sbcs    d0, d0, xzr __LF                                       \+/* Now if d0 is nonzero we subtract p_384 (almost-Montgomery) */    \+        neg     d0, d0 __LF                                            \+        and     t1, d0, #0x00000000ffffffff __LF                       \+        and     t2, d0, #0xffffffff00000000 __LF                       \+        and     t3, d0, #0xfffffffffffffffe __LF                       \+        subs    d1, d1, t1 __LF                                        \+        sbcs    d2, d2, t2 __LF                                        \+        sbcs    d3, d3, t3 __LF                                        \+        sbcs    d4, d4, d0 __LF                                        \+        sbcs    d5, d5, d0 __LF                                        \+        sbc     d6, d6, d0++// Very similar to a subroutine call to the s2n-bignum word_divstep59.+// But different in register usage and returning the final matrix in+// registers as follows+//+// [ m00  m01]+// [ m10  m11]++#define divstep59()                                                     \+        and     x4, x2, #0xfffff __LF                                      \+        orr     x4, x4, #0xfffffe0000000000 __LF                           \+        and     x5, x3, #0xfffff __LF                                      \+        orr     x5, x5, #0xc000000000000000 __LF                           \+        tst     x5, #0x1 __LF                                              \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        asr     x5, x5, #1 __LF                                            \+        add     x8, x4, #0x100, lsl #12 __LF                               \+        sbfx    x8, x8, #21, #21 __LF                                      \+        mov     x11, #0x100000 __LF                                        \+        add     x11, x11, x11, lsl #21 __LF                                \+        add     x9, x4, x11 __LF                                           \+        asr     x9, x9, #42 __LF                                           \+        add     x10, x5, #0x100, lsl #12 __LF                              \+        sbfx    x10, x10, #21, #21 __LF                                    \+        add     x11, x5, x11 __LF                                          \+        asr     x11, x11, #42 __LF                                         \+        mul     x6, x8, x2 __LF                                            \+        mul     x7, x9, x3 __LF                                            \+        mul     x2, x10, x2 __LF                                           \+        mul     x3, x11, x3 __LF                                           \+        add     x4, x6, x7 __LF                                            \+        add     x5, x2, x3 __LF                                            \+        asr     x2, x4, #20 __LF                                           \+        asr     x3, x5, #20 __LF                                           \+        and     x4, x2, #0xfffff __LF                                      \+        orr     x4, x4, #0xfffffe0000000000 __LF                           \+        and     x5, x3, #0xfffff __LF                                      \+        orr     x5, x5, #0xc000000000000000 __LF                           \+        tst     x5, #0x1 __LF                                              \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        asr     x5, x5, #1 __LF                                            \+        add     x12, x4, #0x100, lsl #12 __LF                              \+        sbfx    x12, x12, #21, #21 __LF                                    \+        mov     x15, #0x100000 __LF                                        \+        add     x15, x15, x15, lsl #21 __LF                                \+        add     x13, x4, x15 __LF                                          \+        asr     x13, x13, #42 __LF                                         \+        add     x14, x5, #0x100, lsl #12 __LF                              \+        sbfx    x14, x14, #21, #21 __LF                                    \+        add     x15, x5, x15 __LF                                          \+        asr     x15, x15, #42 __LF                                         \+        mul     x6, x12, x2 __LF                                           \+        mul     x7, x13, x3 __LF                                           \+        mul     x2, x14, x2 __LF                                           \+        mul     x3, x15, x3 __LF                                           \+        add     x4, x6, x7 __LF                                            \+        add     x5, x2, x3 __LF                                            \+        asr     x2, x4, #20 __LF                                           \+        asr     x3, x5, #20 __LF                                           \+        and     x4, x2, #0xfffff __LF                                      \+        orr     x4, x4, #0xfffffe0000000000 __LF                           \+        and     x5, x3, #0xfffff __LF                                      \+        orr     x5, x5, #0xc000000000000000 __LF                           \+        tst     x5, #0x1 __LF                                              \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        mul     x2, x12, x8 __LF                                           \+        mul     x3, x12, x9 __LF                                           \+        mul     x6, x14, x8 __LF                                           \+        mul     x7, x14, x9 __LF                                           \+        madd    x8, x13, x10, x2 __LF                                      \+        madd    x9, x13, x11, x3 __LF                                      \+        madd    x16, x15, x10, x6 __LF                                     \+        madd    x17, x15, x11, x7 __LF                                     \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        tst     x5, #0x2 __LF                                              \+        asr     x5, x5, #1 __LF                                            \+        csel    x6, x4, xzr, ne __LF                                       \+        ccmp    x1, xzr, #0x8, ne __LF                                     \+        cneg    x1, x1, ge __LF                                            \+        cneg    x6, x6, ge __LF                                            \+        csel    x4, x5, x4, ge __LF                                        \+        add     x5, x5, x6 __LF                                            \+        add     x1, x1, #0x2 __LF                                          \+        asr     x5, x5, #1 __LF                                            \+        add     x12, x4, #0x100, lsl #12 __LF                              \+        sbfx    x12, x12, #22, #21 __LF                                    \+        mov     x15, #0x100000 __LF                                        \+        add     x15, x15, x15, lsl #21 __LF                                \+        add     x13, x4, x15 __LF                                          \+        asr     x13, x13, #43 __LF                                         \+        add     x14, x5, #0x100, lsl #12 __LF                              \+        sbfx    x14, x14, #22, #21 __LF                                    \+        add     x15, x5, x15 __LF                                          \+        asr     x15, x15, #43 __LF                                         \+        mneg    x2, x12, x8 __LF                                           \+        mneg    x3, x12, x9 __LF                                           \+        mneg    x4, x14, x8 __LF                                           \+        mneg    x5, x14, x9 __LF                                           \+        msub    m00, x13, x16, x2 __LF                                     \+        msub    m01, x13, x17, x3 __LF                                     \+        msub    m10, x15, x16, x4 __LF                                     \+        msub    m11, x15, x17, x5++S2N_BN_SYMBOL(bignum_montinv_p384):+        CFI_START++// Save registers and make room for temporaries++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_DEC_SP(NSPACE)++// Save the return pointer for the end so we can overwrite x0 later++        mov     res, x0++// Copy the prime and input into the main f and g variables respectively.+// Make sure x is reduced so that g <= f as assumed in the bound proof.++        mov     x10, #0x00000000ffffffff+        mov     x11, #0xffffffff00000000+        mov     x12, #0xfffffffffffffffe+        mov     x15, #0xffffffffffffffff+        stp     x10, x11, [f]+        stp     x12, x15, [f+2*N]+        stp     x15, x15, [f+4*N]+        str     xzr, [f+6*N]++        ldp     x2, x3, [x1]+        subs    x10, x2, x10+        sbcs    x11, x3, x11+        ldp     x4, x5, [x1, #(2*N)]+        sbcs    x12, x4, x12+        sbcs    x13, x5, x15+        ldp     x6, x7, [x1, #(4*N)]+        sbcs    x14, x6, x15+        sbcs    x15, x7, x15++        csel    x2, x2, x10, cc+        csel    x3, x3, x11, cc+        csel    x4, x4, x12, cc+        csel    x5, x5, x13, cc+        csel    x6, x6, x14, cc+        csel    x7, x7, x15, cc++        stp     x2, x3, [g]+        stp     x4, x5, [g+2*N]+        stp     x6, x7, [g+4*N]+        str     xzr, [g+6*N]++// Also maintain reduced < 2^384 vector [u,v] such that+// [f,g] == x * 2^{5*i-843} * [u,v] (mod p_384)+// starting with [p_384,x] == x * 2^{5*0-843} * [0,2^843] (mod p_384)+// The weird-looking 5*i modifications come in because we are doing+// 64-bit word-sized Montgomery reductions at each stage, which is+// 5 bits more than the 59-bit requirement to keep things stable.+// After the 15th and last iteration and sign adjustment, when+// f == 1 for in-scope cases, we have x * 2^{75-843} * u == 1, i.e.+// x * u == 2^768 as required.++        stp     xzr, xzr, [u]+        stp     xzr, xzr, [u+2*N]+        stp     xzr, xzr, [u+4*N]++// The starting constant 2^843 mod p_384 is+// 0x0000000000000800:00001000000007ff:fffff00000000000+//  :00001000000007ff:fffff00000000800:0000000000000000+// where colons separate 64-bit subwords, least significant at the right.+// Not all of these are single loads on ARM so this is a bit dynamic++        mov     x12, #0xfffff00000000000+        orr     x10, x12, #0x0000000000000800+        stp     xzr, x10, [v]+        mov     x11, #0x00000000000007ff+        orr     x11, x11, #0x0000100000000000+        stp     x11, x12, [v+2*N]+        mov     x12, #0x0000000000000800+        stp     x11, x12, [v+4*N]++// Start of main loop. We jump into the middle so that the divstep+// portion is common to the special fifteenth iteration after a uniform+// first 14.++        mov     i, #15+        mov     d, #1+        b       Lbignum_montinv_p384_midloop++Lbignum_montinv_p384_loop:++// Separate the matrix elements into sign-magnitude pairs++        cmp     m00, xzr+        csetm   s00, mi+        cneg    m00, m00, mi++        cmp     m01, xzr+        csetm   s01, mi+        cneg    m01, m01, mi++        cmp     m10, xzr+        csetm   s10, mi+        cneg    m10, m10, mi++        cmp     m11, xzr+        csetm   s11, mi+        cneg    m11, m11, mi++// Adjust the initial values to allow for complement instead of negation+// This initial offset is the same for [f,g] and [u,v] compositions.+// Save it in stable registers for the [u,v] part and do [f,g] first.++        and     x0, m00, s00+        and     x1, m01, s01+        add     car0, x0, x1++        and     x0, m10, s10+        and     x1, m11, s11+        add     car1, x0, x1++// Now the computation of the updated f and g values. This maintains a+// 2-word carry between stages so we can conveniently insert the shift+// right by 59 before storing back, and not overwrite digits we need+// again of the old f and g values.+//+// Digit 0 of [f,g]++        ldr     x7, [f]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x4, car0, x0+        adc     x2, xzr, x1+        ldr     x8, [g]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x4, x4, x0+        adc     x2, x2, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x5, car1, x0+        adc     x3, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x5, x5, x0+        adc     x3, x3, x1++// Digit 1 of [f,g]++        ldr     x7, [f+N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [g+N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [f]++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [g]++// Digit 2 of [f,g]++        ldr     x7, [f+2*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [g+2*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [f+N]++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [g+N]++// Digit 3 of [f,g]++        ldr     x7, [f+3*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x5, x5, x0+        adc     x3, xzr, x1+        ldr     x8, [g+3*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x5, x6, #59+        str     x6, [f+2*N]++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [g+2*N]++// Digit 4 of [f,g]++        ldr     x7, [f+4*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x3, x3, x0+        adc     x4, xzr, x1+        ldr     x8, [g+4*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [f+3*N]++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [g+3*N]++// Digits 5 and 6 of [f,g]++        ldr     x7, [f+5*N]+        eor     x1, x7, s00+        ldr     x23, [f+6*N]+        eor     x2, x23, s00+        and     x2, x2, m00+        neg     x2, x2+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x4, x4, x0+        adc     x2, x2, x1+        ldr     x8, [g+5*N]+        eor     x1, x8, s01+        ldr     x24, [g+6*N]+        eor     x0, x24, s01+        and     x0, x0, m01+        sub     x2, x2, x0+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [f+4*N]+        extr    x4, x2, x4, #59+        str     x4, [f+5*N]+        asr     x2, x2, #59+        str     x2, [f+6*N]++        eor     x1, x7, s10+        eor     x4, x23, s10+        and     x4, x4, m10+        neg     x4, x4+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x5, x5, x0+        adc     x4, x4, x1+        eor     x1, x8, s11+        eor     x0, x24, s11+        and     x0, x0, m11+        sub     x4, x4, x0+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x5, x5, x0+        adc     x4, x4, x1+        extr    x6, x5, x6, #59+        str     x6, [g+4*N]+        extr    x5, x4, x5, #59+        str     x5, [g+5*N]+        asr     x4, x4, #59+        str     x4, [g+6*N]++// Now the computation of the updated u and v values and their+// Montgomery reductions. A very similar accumulation except that+// the top words of u and v are unsigned and we don't shift.+//+// Digit 0 of [u,v]++        ldr     x7, [u]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x4, car0, x0+        adc     x2, xzr, x1+        ldr     x8, [v]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x4, x4, x0+        str     x4, [u]+        adc     x2, x2, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x5, car1, x0+        adc     x3, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x5, x5, x0+        str     x5, [v]+        adc     x3, x3, x1++// Digit 1 of [u,v]++        ldr     x7, [u+N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [v+N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x2, x2, x0+        str     x2, [u+N]+        adc     x6, x6, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x3, x3, x0+        str     x3, [v+N]+        adc     x4, x4, x1++// Digit 2 of [u,v]++        ldr     x7, [u+2*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [v+2*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x6, x6, x0+        str     x6, [u+2*N]+        adc     x5, x5, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x4, x4, x0+        str     x4, [v+2*N]+        adc     x2, x2, x1++// Digit 3 of [u,v]++        ldr     x7, [u+3*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x5, x5, x0+        adc     x3, xzr, x1+        ldr     x8, [v+3*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x5, x5, x0+        str     x5, [u+3*N]+        adc     x3, x3, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x2, x2, x0+        str     x2, [v+3*N]+        adc     x6, x6, x1++// Digit 4 of [u,v]++        ldr     x7, [u+4*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x3, x3, x0+        adc     x4, xzr, x1+        ldr     x8, [v+4*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x3, x3, x0+        str     x3, [u+4*N]+        adc     x4, x4, x1++        eor     x1, x7, s10+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        eor     x1, x8, s11+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x6, x6, x0+        str     x6, [v+4*N]+        adc     x5, x5, x1++// Digits 5 and 6 of [u,v] (top is unsigned)++        ldr     x7, [u+5*N]+        eor     x1, x7, s00+        and     x2, s00, m00+        neg     x2, x2+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x4, x4, x0+        adc     x2, x2, x1+        ldr     x8, [v+5*N]+        eor     x1, x8, s01+        and     x0, s01, m01+        sub     x2, x2, x0+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x4, x4, x0+        str     x4, [u+5*N]+        adc     x2, x2, x1+        str     x2, [u+6*N]++        eor     x1, x7, s10+        and     x4, s10, m10+        neg     x4, x4+        mul     x0, x1, m10+        umulh   x1, x1, m10+        adds    x5, x5, x0+        adc     x4, x4, x1+        eor     x1, x8, s11+        and     x0, s11, m11+        sub     x4, x4, x0+        mul     x0, x1, m11+        umulh   x1, x1, m11+        adds    x5, x5, x0+        str     x5, [v+5*N]+        adc     x4, x4, x1+        str     x4, [v+6*N]++// Montgomery reduction of u++        ldp     x0, x1, [u]+        ldp     x2, x3, [u+16]+        ldp     x4, x5, [u+32]+        ldr     x6, [u+48]+        amontred(x6,x5,x4,x3,x2,x1,x0, x9,x8,x7)+        stp     x1, x2, [u]+        stp     x3, x4, [u+16]+        stp     x5, x6, [u+32]++// Montgomery reduction of v++        ldp     x0, x1, [v]+        ldp     x2, x3, [v+16]+        ldp     x4, x5, [v+32]+        ldr     x6, [v+48]+        amontred(x6,x5,x4,x3,x2,x1,x0, x9,x8,x7)+        stp     x1, x2, [v]+        stp     x3, x4, [v+16]+        stp     x5, x6, [v+32]++Lbignum_montinv_p384_midloop:++        mov     x1, d+        ldr     x2, [f]+        ldr     x3, [g]+        divstep59()+        mov     d, x1++// Next iteration++        subs    i, i, #1+        bne     Lbignum_montinv_p384_loop++// The 15th and last iteration does not need anything except the+// u value and the sign of f; the latter can be obtained from the+// lowest word of f. So it's done differently from the main loop.+// Find the sign of the new f. For this we just need one digit+// since we know (for in-scope cases) that f is either +1 or -1.+// We don't explicitly shift right by 59 either, but looking at+// bit 63 (or any bit >= 60) of the unshifted result is enough+// to distinguish -1 from +1; this is then made into a mask.++        ldr     x0, [f]+        ldr     x1, [g]+        mul     x0, x0, m00+        madd    x1, x1, m01, x0+        asr     x0, x1, #63++// Now separate out the matrix into sign-magnitude pairs+// and adjust each one based on the sign of f.+//+// Note that at this point we expect |f|=1 and we got its+// sign above, so then since [f,0] == x * 2^{-768} [u,v] (mod p_384)+// we want to flip the sign of u according to that of f.++        cmp     m00, xzr+        csetm   s00, mi+        cneg    m00, m00, mi+        eor     s00, s00, x0++        cmp     m01, xzr+        csetm   s01, mi+        cneg    m01, m01, mi+        eor     s01, s01, x0++        cmp     m10, xzr+        csetm   s10, mi+        cneg    m10, m10, mi+        eor     s10, s10, x0++        cmp     m11, xzr+        csetm   s11, mi+        cneg    m11, m11, mi+        eor     s11, s11, x0++// Adjust the initial value to allow for complement instead of negation++        and     x0, m00, s00+        and     x1, m01, s01+        add     car0, x0, x1++// Digit 0 of [u]++        ldr     x7, [u]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x4, car0, x0+        adc     x2, xzr, x1+        ldr     x8, [v]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x4, x4, x0+        str     x4, [u]+        adc     x2, x2, x1++// Digit 1 of [u]++        ldr     x7, [u+N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [v+N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x2, x2, x0+        str     x2, [u+N]+        adc     x6, x6, x1++// Digit 2 of [u]++        ldr     x7, [u+2*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [v+2*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x6, x6, x0+        str     x6, [u+2*N]+        adc     x5, x5, x1++// Digit 3 of [u]++        ldr     x7, [u+3*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x5, x5, x0+        adc     x3, xzr, x1+        ldr     x8, [v+3*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x5, x5, x0+        str     x5, [u+3*N]+        adc     x3, x3, x1++// Digit 4 of [u]++        ldr     x7, [u+4*N]+        eor     x1, x7, s00+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x3, x3, x0+        adc     x4, xzr, x1+        ldr     x8, [v+4*N]+        eor     x1, x8, s01+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x3, x3, x0+        str     x3, [u+4*N]+        adc     x4, x4, x1++// Digits 5 and 6 of [u] (top is unsigned)++        ldr     x7, [u+5*N]+        eor     x1, x7, s00+        and     x2, s00, m00+        neg     x2, x2+        mul     x0, x1, m00+        umulh   x1, x1, m00+        adds    x4, x4, x0+        adc     x2, x2, x1+        ldr     x8, [v+5*N]+        eor     x1, x8, s01+        and     x0, s01, m01+        sub     x2, x2, x0+        mul     x0, x1, m01+        umulh   x1, x1, m01+        adds    x4, x4, x0+        str     x4, [u+5*N]+        adc     x2, x2, x1+        str     x2, [u+6*N]++// Montgomery reduction of u. This needs to be strict not "almost"+// so it is followed by an optional subtraction of p_384++        ldp     x10, x0, [u]+        ldp     x1, x2, [u+16]+        ldp     x3, x4, [u+32]+        ldr     x5, [u+48]+        amontred(x5,x4,x3,x2,x1,x0,x10, x9,x8,x7)++        mov     x10, #0x00000000ffffffff+        subs    x10, x0, x10+        mov     x11, #0xffffffff00000000+        sbcs    x11, x1, x11+        mov     x12, #0xfffffffffffffffe+        sbcs    x12, x2, x12+        mov     x15, #0xffffffffffffffff+        sbcs    x13, x3, x15+        sbcs    x14, x4, x15+        sbcs    x15, x5, x15++        csel    x0, x0, x10, cc+        csel    x1, x1, x11, cc+        csel    x2, x2, x12, cc+        csel    x3, x3, x13, cc+        csel    x4, x4, x14, cc+        csel    x5, x5, x15, cc++// Store it back to the final output++        stp     x0, x1, [res]+        stp     x2, x3, [res, #16]+        stp     x4, x5, [res, #32]++// Restore stack and registers++        CFI_INC_SP(NSPACE)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_montinv_p384)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/bignum_montmul_p384.S view
@@ -0,0 +1,891 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Montgomery multiply, z := (x * y / 2^384) mod p_384+// Inputs x[6], y[6]; output z[6]+//+//    extern void bignum_montmul_p384(uint64_t z[static 6],+//                                    const uint64_t x[static 6],+//                                    const uint64_t y[static 6]);+//+// Does z := (2^{-384} * x * y) mod p_384, assuming that the inputs x and y+// satisfy x * y <= 2^384 * p_384 (in particular this is true if we are in+// the "usual" case x < p_384 and y < p_384).+//+// Standard ARM ABI: X0 = z, X1 = x, X2 = y+// ----------------------------------------------------------------------------++// bignum_montmul_p384 is functionally equivalent to+// unopt/bignum_montmul_p384_base.+// It is written in a way that+// 1. A subset of scalar multiplications in bignum_montmul_p384 are carefully+//    chosen and vectorized+// 2. The vectorized assembly is rescheduled using the SLOTHY superoptimizer.+//    https://github.com/slothy-optimizer/slothy+//+// The output program of step 1. is as follows:+//+//        stp	x19, x20, [sp, #-16]!+//        stp	x21, x22, [sp, #-16]!+//        stp	x23, x24, [sp, #-16]!+//        ldp x3, x21, [x1]+//        ldr q30, [x1]+//        ldp x8, x24, [x1, #16]+//        ldp x5, x10, [x1, #32]+//        ldp x13, x23, [x2]+//        ldr q19, [x2]+//        ldp x6, x14, [x2, #16]+//        ldp x15, x17, [x2, #32]+//        ldr q1, [x1, #32]+//        ldr q28, [x2, #32]+//        uzp1 v5.4S, v19.4S, v30.4S+//        rev64 v19.4S, v19.4S+//        uzp1 v0.4S, v30.4S, v30.4S+//        mul v21.4S, v19.4S, v30.4S+//        uaddlp v19.2D, v21.4S+//        shl v19.2D, v19.2D, #32+//        umlal v19.2D, v0.2S, v5.2S+//        mov x12, v19.d[0]+//        mov x16, v19.d[1]+//        mul x20, x8, x6+//        umulh x4, x3, x13+//        umulh x1, x21, x23+//        umulh x2, x8, x6+//        adds x4, x4, x16+//        adcs x19, x1, x20+//        adc x20, x2, xzr+//        adds x11, x4, x12+//        adcs x16, x19, x4+//        adcs x1, x20, x19+//        adc x2, x20, xzr+//        adds x7, x16, x12+//        adcs x4, x1, x4+//        adcs x9, x2, x19+//        adc x19, x20, xzr+//        subs x2, x3, x21+//        cneg x20, x2, cc+//        csetm x16, cc+//        subs x2, x23, x13+//        cneg x2, x2, cc+//        mul x1, x20, x2+//        umulh x2, x20, x2+//        cinv x16, x16, cc+//        eor x1, x1, x16+//        eor x2, x2, x16+//        cmn x16, #0x1+//        adcs x11, x11, x1+//        adcs x7, x7, x2+//        adcs x4, x4, x16+//        adcs x9, x9, x16+//        adc x19, x19, x16+//        subs x2, x3, x8+//        cneg x20, x2, cc+//        csetm x16, cc+//        subs x2, x6, x13+//        cneg x2, x2, cc+//        mul x1, x20, x2+//        umulh x2, x20, x2+//        cinv x16, x16, cc+//        eor x1, x1, x16+//        eor x2, x2, x16+//        cmn x16, #0x1+//        adcs x7, x7, x1+//        adcs x4, x4, x2+//        adcs x9, x9, x16+//        adc x19, x19, x16+//        subs x2, x21, x8+//        cneg x20, x2, cc+//        csetm x16, cc+//        subs x2, x6, x23+//        cneg x2, x2, cc+//        mul x1, x20, x2+//        umulh x2, x20, x2+//        cinv x16, x16, cc+//        eor x1, x1, x16+//        eor x2, x2, x16+//        cmn x16, #0x1+//        adcs x4, x4, x1+//        adcs x20, x9, x2+//        adc x16, x19, x16+//        lsl x2, x12, #32+//        add x19, x2, x12+//        lsr x2, x19, #32+//        subs x1, x2, x19+//        sbc x2, x19, xzr+//        extr x1, x2, x1, #32+//        lsr x2, x2, #32+//        adds x12, x2, x19+//        adc x2, xzr, xzr+//        subs x1, x11, x1+//        sbcs x7, x7, x12+//        sbcs x4, x4, x2+//        sbcs x20, x20, xzr+//        sbcs x16, x16, xzr+//        sbc x9, x19, xzr+//        lsl x2, x1, #32+//        add x19, x2, x1+//        lsr x2, x19, #32+//        subs x1, x2, x19+//        sbc x2, x19, xzr+//        extr x1, x2, x1, #32+//        lsr x2, x2, #32+//        adds x12, x2, x19+//        adc x2, xzr, xzr+//        subs x1, x7, x1+//        sbcs x4, x4, x12+//        sbcs x20, x20, x2+//        sbcs x16, x16, xzr+//        sbcs x7, x9, xzr+//        sbc x9, x19, xzr+//        lsl x2, x1, #32+//        add x19, x2, x1+//        lsr x2, x19, #32+//        subs x1, x2, x19+//        sbc x2, x19, xzr+//        extr x12, x2, x1, #32+//        lsr x2, x2, #32+//        adds x1, x2, x19+//        adc x2, xzr, xzr+//        subs x4, x4, x12+//        sbcs x20, x20, x1+//        sbcs x16, x16, x2+//        sbcs x12, x7, xzr+//        sbcs x1, x9, xzr+//        sbc x2, x19, xzr+//        stp x4, x20, [x0]                       // @slothy:writes=buffer0+//        stp x16, x12, [x0, #16]                 // @slothy:writes=buffer16+//        stp x1, x2, [x0, #32]                   // @slothy:writes=buffer32+//        mul x22, x24, x14+//        movi v31.2D, #0x00000000ffffffff+//        uzp2 v16.4S, v28.4S, v28.4S+//        xtn v6.2S, v1.2D+//        xtn v30.2S, v28.2D+//        rev64 v28.4S, v28.4S+//        umull v5.2D, v6.2S, v30.2S+//        umull v0.2D, v6.2S, v16.2S+//        uzp2 v19.4S, v1.4S, v1.4S+//        mul v20.4S, v28.4S, v1.4S+//        usra v0.2D, v5.2D, #32+//        umull v1.2D, v19.2S, v16.2S+//        uaddlp v24.2D, v20.4S+//        and v5.16B, v0.16B, v31.16B+//        umlal v5.2D, v19.2S, v30.2S+//        shl v19.2D, v24.2D, #32+//        usra v1.2D, v0.2D, #32+//        umlal v19.2D, v6.2S, v30.2S+//        usra v1.2D, v5.2D, #32+//        mov x20, v19.d[0]+//        mov x16, v19.d[1]+//        umulh x12, x24, x14+//        mov x1, v1.d[0]+//        mov x2, v1.d[1]+//        adds x4, x12, x20+//        adcs x20, x1, x16+//        adc x16, x2, xzr+//        adds x7, x4, x22+//        adcs x12, x20, x4+//        adcs x1, x16, x20+//        adc x2, x16, xzr+//        adds x9, x12, x22+//        adcs x19, x1, x4+//        adcs x4, x2, x20+//        adc x20, x16, xzr+//        subs x2, x24, x5+//        cneg x16, x2, cc+//        csetm x12, cc+//        subs x2, x15, x14+//        cneg x2, x2, cc+//        mul x1, x16, x2+//        umulh x2, x16, x2+//        cinv x12, x12, cc+//        eor x1, x1, x12+//        eor x2, x2, x12+//        cmn x12, #0x1+//        adcs x11, x7, x1+//        adcs x9, x9, x2+//        adcs x19, x19, x12+//        adcs x4, x4, x12+//        adc x20, x20, x12+//        subs x2, x24, x10+//        cneg x16, x2, cc+//        csetm x12, cc+//        subs x2, x17, x14+//        cneg x2, x2, cc+//        mul x1, x16, x2+//        umulh x2, x16, x2+//        cinv x12, x12, cc+//        eor x1, x1, x12+//        eor x2, x2, x12+//        cmn x12, #0x1+//        adcs x7, x9, x1+//        adcs x19, x19, x2+//        adcs x4, x4, x12+//        adc x20, x20, x12+//        subs x2, x5, x10+//        cneg x16, x2, cc+//        csetm x12, cc+//        subs x2, x17, x15+//        cneg x2, x2, cc+//        mul x1, x16, x2+//        umulh x2, x16, x2+//        cinv x16, x12, cc+//        eor x1, x1, x16+//        eor x2, x2, x16+//        cmn x16, #0x1+//        adcs x19, x19, x1+//        adcs x12, x4, x2+//        adc x1, x20, x16+//        subs x2, x24, x3+//        sbcs x24, x5, x21+//        sbcs x21, x10, x8+//        ngc x5, xzr+//        cmn x5, #0x1+//        eor x2, x2, x5+//        adcs x4, x2, xzr+//        eor x2, x24, x5+//        adcs x20, x2, xzr+//        eor x2, x21, x5+//        adc x16, x2, xzr+//        subs x2, x13, x14+//        sbcs x24, x23, x15+//        sbcs x8, x6, x17+//        ngc x21, xzr+//        cmn x21, #0x1+//        eor x2, x2, x21+//        adcs x15, x2, xzr+//        eor x2, x24, x21+//        adcs x14, x2, xzr+//        eor x2, x8, x21+//        adc x6, x2, xzr+//        eor x9, x5, x21+//        ldp x21, x2, [x0]                       // @slothy:reads=buffer0+//        adds x10, x22, x21+//        adcs x5, x11, x2+//        ldp x21, x2, [x0, #16]                  // @slothy:reads=buffer16+//        adcs x24, x7, x21+//        adcs x8, x19, x2+//        ldp x21, x2, [x0, #32]                  // @slothy:reads=buffer32+//        adcs x21, x12, x21+//        adcs x2, x1, x2+//        adc x19, xzr, xzr+//        stp x10, x5, [x0]                       // @slothy:writes=buffer0+//        stp x24, x8, [x0, #16]                  // @slothy:writes=buffer16+//        stp x21, x2, [x0, #32]                  // @slothy:writes=buffer32+//        mul x12, x4, x15+//        mul x5, x20, x14+//        mul x24, x16, x6+//        umulh x8, x4, x15+//        umulh x21, x20, x14+//        umulh x2, x16, x6+//        adds x10, x8, x5+//        adcs x5, x21, x24+//        adc x24, x2, xzr+//        adds x23, x10, x12+//        adcs x8, x5, x10+//        adcs x21, x24, x5+//        adc x2, x24, xzr+//        adds x13, x8, x12+//        adcs x1, x21, x10+//        adcs x10, x2, x5+//        adc x5, x24, xzr+//        subs x2, x4, x20+//        cneg x24, x2, cc+//        csetm x8, cc+//        subs x2, x14, x15+//        cneg x2, x2, cc+//        mul x21, x24, x2+//        umulh x2, x24, x2+//        cinv x8, x8, cc+//        eor x21, x21, x8+//        eor x2, x2, x8+//        cmn x8, #0x1+//        adcs x23, x23, x21+//        adcs x13, x13, x2+//        adcs x1, x1, x8+//        adcs x10, x10, x8+//        adc x5, x5, x8+//        subs x2, x4, x16+//        cneg x24, x2, cc+//        csetm x8, cc+//        subs x2, x6, x15+//        cneg x2, x2, cc+//        mul x21, x24, x2+//        umulh x2, x24, x2+//        cinv x8, x8, cc+//        eor x21, x21, x8+//        eor x2, x2, x8+//        cmn x8, #0x1+//        adcs x4, x13, x21+//        adcs x13, x1, x2+//        adcs x1, x10, x8+//        adc x10, x5, x8+//        subs x2, x20, x16+//        cneg x24, x2, cc+//        csetm x8, cc+//        subs x2, x6, x14+//        cneg x2, x2, cc+//        mul x21, x24, x2+//        umulh x2, x24, x2+//        cinv x5, x8, cc+//        eor x21, x21, x5+//        eor x2, x2, x5+//        cmn x5, #0x1+//        adcs x24, x13, x21+//        adcs x8, x1, x2+//        adc x21, x10, x5+//        ldp x20, x16, [x0]                      // @slothy:reads=buffer0+//        ldp x17, x15, [x0, #16]                 // @slothy:reads=buffer16+//        ldp x14, x6, [x0, #32]                  // @slothy:reads=buffer32+//        cmn x9, #0x1+//        eor x2, x12, x9+//        adcs x12, x2, x20+//        eor x2, x23, x9+//        adcs x23, x2, x16+//        eor x2, x4, x9+//        adcs x13, x2, x17+//        eor x2, x24, x9+//        adcs x10, x2, x15+//        eor x2, x8, x9+//        adcs x5, x2, x14+//        eor x2, x21, x9+//        adcs x24, x2, x6+//        adcs x1, x9, x19+//        adcs x8, x9, xzr+//        adcs x21, x9, xzr+//        adc x2, x9, xzr+//        adds x10, x10, x20+//        adcs x5, x5, x16+//        adcs x24, x24, x17+//        adcs x17, x1, x15+//        adcs x15, x8, x14+//        adcs x14, x21, x6+//        adc x6, x2, x19+//        lsl x2, x12, #32+//        add x1, x2, x12+//        lsr x2, x1, #32+//        subs x21, x2, x1+//        sbc x2, x1, xzr+//        extr x21, x2, x21, #32+//        lsr x2, x2, #32+//        adds x8, x2, x1+//        adc x2, xzr, xzr+//        subs x21, x23, x21+//        sbcs x23, x13, x8+//        sbcs x10, x10, x2+//        sbcs x5, x5, xzr+//        sbcs x24, x24, xzr+//        sbc x13, x1, xzr+//        lsl x2, x21, #32+//        add x1, x2, x21+//        lsr x2, x1, #32+//        subs x21, x2, x1+//        sbc x2, x1, xzr+//        extr x21, x2, x21, #32+//        lsr x2, x2, #32+//        adds x8, x2, x1+//        adc x2, xzr, xzr+//        subs x21, x23, x21+//        sbcs x10, x10, x8+//        sbcs x5, x5, x2+//        sbcs x24, x24, xzr+//        sbcs x23, x13, xzr+//        sbc x13, x1, xzr+//        lsl x2, x21, #32+//        add x1, x2, x21+//        lsr x2, x1, #32+//        subs x21, x2, x1+//        sbc x2, x1, xzr+//        extr x8, x2, x21, #32+//        lsr x2, x2, #32+//        adds x21, x2, x1+//        adc x2, xzr, xzr+//        subs x10, x10, x8+//        sbcs x5, x5, x21+//        sbcs x24, x24, x2+//        sbcs x8, x23, xzr+//        sbcs x21, x13, xzr+//        sbc x2, x1, xzr+//        adds x23, x17, x8+//        adcs x13, x15, x21+//        adcs x1, x14, x2+//        adc x2, x6, xzr+//        add x8, x2, #0x1+//        lsl x2, x8, #32+//        subs x21, x8, x2+//        sbc x2, x2, xzr+//        adds x10, x10, x21+//        adcs x5, x5, x2+//        adcs x24, x24, x8+//        adcs x8, x23, xzr+//        adcs x21, x13, xzr+//        adcs x13, x1, xzr+//        csetm x1, cc+//        mov x2, #0xffffffff+//        and x2, x2, x1+//        adds x10, x10, x2+//        eor x2, x2, x1+//        adcs x5, x5, x2+//        mov x2, #0xfffffffffffffffe+//        and x2, x2, x1+//        adcs x24, x24, x2+//        adcs x8, x8, x1+//        adcs x21, x21, x1+//        adc x2, x13, x1+//        stp x10, x5, [x0]                       // @slothy:writes=buffer0+//        stp x24, x8, [x0, #16]                  // @slothy:writes=buffer16+//        stp x21, x2, [x0, #32]                  // @slothy:writes=buffer32+//        ldp	x23, x24, [sp], #16+//        ldp	x21, x22, [sp], #16+//        ldp	x19, x20, [sp], #16+//        ret+//+// The bash script used for step 2 is as follows:+//+//        # Store the assembly instructions except the last 'ret' and+//        # callee-register store/loads as, say, 'input.S'.+//        export OUTPUTS="[hint_buffer0,hint_buffer16,hint_buffer32]"+//        export RESERVED_REGS="[x18,x25,x26,x27,x28,x29,x30,sp,q8,q9,q10,q11,q12,q13,q14,q15,v8,v9,v10,v11,v12,v13,v14,v15]"+//        <s2n-bignum>/tools/external/slothy.sh input.S my_out_dir+//        # my_out_dir/3.opt.s is the optimized assembly. Its output may differ+//        # from this file since the sequence is non-deterministically chosen.+//        # Please add 'ret' at the end of the output assembly.+++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_montmul_p384)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_montmul_p384)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_montmul_p384)+        .text+        .balign 4++S2N_BN_SYMBOL(bignum_montmul_p384):+        CFI_START++// Save some registers++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)++        ldr q3, [x1]+        ldr q25, [x2]+        ldp x13, x23, [x2]+        ldp x3, x21, [x1]+        rev64 v23.4S, v25.4S+        uzp1 v17.4S, v25.4S, v3.4S+        umulh x15, x3, x13+        mul v6.4S, v23.4S, v3.4S+        uzp1 v3.4S, v3.4S, v3.4S+        ldr q27, [x2, #32]+        ldp x8, x24, [x1, #16]+        subs x6, x3, x21+        ldr q0, [x1, #32]+        movi v23.2D, #0x00000000ffffffff+        csetm x10, cc+        umulh x19, x21, x23+        rev64 v4.4S, v27.4S+        uzp2 v25.4S, v27.4S, v27.4S+        cneg x4, x6, cc+        subs x7, x23, x13+        xtn v22.2S, v0.2D+        xtn v24.2S, v27.2D+        cneg x20, x7, cc+        ldp x6, x14, [x2, #16]+        mul v27.4S, v4.4S, v0.4S+        uaddlp v20.2D, v6.4S+        cinv x5, x10, cc+        mul x16, x4, x20+        uzp2 v6.4S, v0.4S, v0.4S+        umull v21.2D, v22.2S, v25.2S+        shl v0.2D, v20.2D, #32+        umlal v0.2D, v3.2S, v17.2S+        mul x22, x8, x6+        umull v1.2D, v6.2S, v25.2S+        subs x12, x3, x8+        umull v20.2D, v22.2S, v24.2S+        cneg x17, x12, cc+        umulh x9, x8, x6+        mov x12, v0.d[1]+        eor x11, x16, x5+        mov x7, v0.d[0]+        csetm x10, cc+        usra v21.2D, v20.2D, #32+        adds x15, x15, x12+        adcs x12, x19, x22+        umulh x20, x4, x20+        adc x19, x9, xzr+        usra v1.2D, v21.2D, #32+        adds x22, x15, x7+        and v26.16B, v21.16B, v23.16B+        adcs x16, x12, x15+        uaddlp v25.2D, v27.4S+        adcs x9, x19, x12+        umlal v26.2D, v6.2S, v24.2S+        adc x4, x19, xzr+        adds x16, x16, x7+        shl v27.2D, v25.2D, #32+        adcs x9, x9, x15+        adcs x4, x4, x12+        eor x12, x20, x5+        adc x15, x19, xzr+        subs x20, x6, x13+        cneg x20, x20, cc+        cinv x10, x10, cc+        cmn x5, #0x1+        mul x19, x17, x20+        adcs x11, x22, x11+        adcs x12, x16, x12+        adcs x9, x9, x5+        umulh x17, x17, x20+        adcs x22, x4, x5+        adc x5, x15, x5+        subs x16, x21, x8+        cneg x20, x16, cc+        eor x19, x19, x10+        csetm x4, cc+        subs x16, x6, x23+        cneg x16, x16, cc+        umlal v27.2D, v22.2S, v24.2S+        mul x15, x20, x16+        cinv x4, x4, cc+        cmn x10, #0x1+        usra v1.2D, v26.2D, #32+        adcs x19, x12, x19+        eor x17, x17, x10+        adcs x9, x9, x17+        adcs x22, x22, x10+        lsl x12, x7, #32+        umulh x20, x20, x16+        eor x16, x15, x4+        ldp x15, x17, [x2, #32]+        add x2, x12, x7+        adc x7, x5, x10+        ldp x5, x10, [x1, #32]+        lsr x1, x2, #32+        eor x12, x20, x4+        subs x1, x1, x2+        sbc x20, x2, xzr+        cmn x4, #0x1+        adcs x9, x9, x16+        extr x1, x20, x1, #32+        lsr x20, x20, #32+        adcs x22, x22, x12+        adc x16, x7, x4+        adds x12, x20, x2+        umulh x7, x24, x14+        adc x4, xzr, xzr+        subs x1, x11, x1+        sbcs x20, x19, x12+        sbcs x12, x9, x4+        lsl x9, x1, #32+        add x1, x9, x1+        sbcs x9, x22, xzr+        mul x22, x24, x14+        sbcs x16, x16, xzr+        lsr x4, x1, #32+        sbc x19, x2, xzr+        subs x4, x4, x1+        sbc x11, x1, xzr+        extr x2, x11, x4, #32+        lsr x4, x11, #32+        adds x4, x4, x1+        adc x11, xzr, xzr+        subs x2, x20, x2+        sbcs x4, x12, x4+        sbcs x20, x9, x11+        lsl x12, x2, #32+        add x2, x12, x2+        sbcs x9, x16, xzr+        lsr x11, x2, #32+        sbcs x19, x19, xzr+        sbc x1, x1, xzr+        subs x16, x11, x2+        sbc x12, x2, xzr+        extr x16, x12, x16, #32+        lsr x12, x12, #32+        adds x11, x12, x2+        adc x12, xzr, xzr+        subs x16, x4, x16+        mov x4, v27.d[0]+        sbcs x11, x20, x11+        sbcs x20, x9, x12+        stp x16, x11, [x0]+        sbcs x11, x19, xzr+        sbcs x9, x1, xzr+        stp x20, x11, [x0, #16]+        mov x1, v1.d[0]+        sbc x20, x2, xzr+        subs x12, x24, x5+        mov x11, v27.d[1]+        cneg x16, x12, cc+        csetm x2, cc+        subs x19, x15, x14+        mov x12, v1.d[1]+        cinv x2, x2, cc+        cneg x19, x19, cc+        stp x9, x20, [x0, #32]+        mul x9, x16, x19+        adds x4, x7, x4+        adcs x11, x1, x11+        adc x1, x12, xzr+        adds x20, x4, x22+        umulh x19, x16, x19+        adcs x7, x11, x4+        eor x16, x9, x2+        adcs x9, x1, x11+        adc x12, x1, xzr+        adds x7, x7, x22+        adcs x4, x9, x4+        adcs x9, x12, x11+        adc x12, x1, xzr+        cmn x2, #0x1+        eor x1, x19, x2+        adcs x11, x20, x16+        adcs x19, x7, x1+        adcs x1, x4, x2+        adcs x20, x9, x2+        adc x2, x12, x2+        subs x12, x24, x10+        cneg x16, x12, cc+        csetm x12, cc+        subs x9, x17, x14+        cinv x12, x12, cc+        cneg x9, x9, cc+        subs x3, x24, x3+        sbcs x21, x5, x21+        mul x24, x16, x9+        sbcs x4, x10, x8+        ngc x8, xzr+        subs x10, x5, x10+        eor x5, x24, x12+        csetm x7, cc+        cneg x24, x10, cc+        subs x10, x17, x15+        cinv x7, x7, cc+        cneg x10, x10, cc+        subs x14, x13, x14+        sbcs x15, x23, x15+        eor x13, x21, x8+        mul x23, x24, x10+        sbcs x17, x6, x17+        eor x6, x3, x8+        ngc x21, xzr+        umulh x9, x16, x9+        cmn x8, #0x1+        eor x3, x23, x7+        adcs x23, x6, xzr+        adcs x13, x13, xzr+        eor x16, x4, x8+        adc x16, x16, xzr+        eor x4, x17, x21+        umulh x17, x24, x10+        cmn x21, #0x1+        eor x24, x14, x21+        eor x6, x15, x21+        adcs x15, x24, xzr+        adcs x14, x6, xzr+        adc x6, x4, xzr+        cmn x12, #0x1+        eor x4, x9, x12+        adcs x19, x19, x5+        umulh x5, x23, x15+        adcs x1, x1, x4+        adcs x10, x20, x12+        eor x4, x17, x7+        ldp x20, x9, [x0]+        adc x2, x2, x12+        cmn x7, #0x1+        adcs x12, x1, x3+        ldp x17, x24, [x0, #16]+        mul x1, x16, x6+        adcs x3, x10, x4+        adc x2, x2, x7+        ldp x7, x4, [x0, #32]+        adds x20, x22, x20+        mul x10, x13, x14+        adcs x11, x11, x9+        eor x9, x8, x21+        adcs x21, x19, x17+        stp x20, x11, [x0]+        adcs x12, x12, x24+        mul x8, x23, x15+        adcs x3, x3, x7+        stp x21, x12, [x0, #16]+        adcs x12, x2, x4+        adc x19, xzr, xzr+        subs x21, x23, x16+        umulh x2, x16, x6+        stp x3, x12, [x0, #32]+        cneg x3, x21, cc+        csetm x24, cc+        umulh x11, x13, x14+        subs x21, x13, x16+        eor x7, x8, x9+        cneg x17, x21, cc+        csetm x16, cc+        subs x21, x6, x15+        cneg x22, x21, cc+        cinv x21, x24, cc+        subs x20, x23, x13+        umulh x12, x3, x22+        cneg x23, x20, cc+        csetm x24, cc+        subs x20, x14, x15+        cinv x24, x24, cc+        mul x22, x3, x22+        cneg x3, x20, cc+        subs x13, x6, x14+        cneg x20, x13, cc+        cinv x15, x16, cc+        adds x13, x5, x10+        mul x4, x23, x3+        adcs x11, x11, x1+        adc x14, x2, xzr+        adds x5, x13, x8+        adcs x16, x11, x13+        umulh x23, x23, x3+        adcs x3, x14, x11+        adc x1, x14, xzr+        adds x10, x16, x8+        adcs x6, x3, x13+        adcs x8, x1, x11+        umulh x13, x17, x20+        eor x1, x4, x24+        adc x4, x14, xzr+        cmn x24, #0x1+        adcs x1, x5, x1+        eor x16, x23, x24+        eor x11, x1, x9+        adcs x23, x10, x16+        eor x2, x22, x21+        adcs x3, x6, x24+        mul x14, x17, x20+        eor x17, x13, x15+        adcs x13, x8, x24+        adc x8, x4, x24+        cmn x21, #0x1+        adcs x6, x23, x2+        mov x16, #0xfffffffffffffffe+        eor x20, x12, x21+        adcs x20, x3, x20+        eor x23, x14, x15+        adcs x2, x13, x21+        adc x8, x8, x21+        cmn x15, #0x1+        ldp x5, x4, [x0]+        ldp x21, x12, [x0, #16]+        adcs x22, x20, x23+        eor x23, x22, x9+        adcs x17, x2, x17+        adc x22, x8, x15+        cmn x9, #0x1+        adcs x15, x7, x5+        ldp x10, x14, [x0, #32]+        eor x1, x6, x9+        lsl x2, x15, #32+        adcs x8, x11, x4+        adcs x13, x1, x21+        eor x1, x22, x9+        adcs x24, x23, x12+        eor x11, x17, x9+        adcs x23, x11, x10+        adcs x7, x1, x14+        adcs x17, x9, x19+        adcs x20, x9, xzr+        add x1, x2, x15+        lsr x3, x1, #32+        adcs x11, x9, xzr+        adc x9, x9, xzr+        subs x3, x3, x1+        sbc x6, x1, xzr+        adds x24, x24, x5+        adcs x4, x23, x4+        extr x3, x6, x3, #32+        lsr x6, x6, #32+        adcs x21, x7, x21+        adcs x15, x17, x12+        adcs x7, x20, x10+        adcs x20, x11, x14+        mov x14, #0xffffffff+        adc x22, x9, x19+        adds x12, x6, x1+        adc x10, xzr, xzr+        subs x3, x8, x3+        sbcs x12, x13, x12+        lsl x9, x3, #32+        add x3, x9, x3+        sbcs x10, x24, x10+        sbcs x24, x4, xzr+        lsr x9, x3, #32+        sbcs x21, x21, xzr+        sbc x1, x1, xzr+        subs x9, x9, x3+        sbc x13, x3, xzr+        extr x9, x13, x9, #32+        lsr x13, x13, #32+        adds x13, x13, x3+        adc x6, xzr, xzr+        subs x12, x12, x9+        sbcs x17, x10, x13+        lsl x2, x12, #32+        sbcs x10, x24, x6+        add x9, x2, x12+        sbcs x6, x21, xzr+        lsr x5, x9, #32+        sbcs x21, x1, xzr+        sbc x13, x3, xzr+        subs x8, x5, x9+        sbc x19, x9, xzr+        lsr x12, x19, #32+        extr x3, x19, x8, #32+        adds x8, x12, x9+        adc x1, xzr, xzr+        subs x2, x17, x3+        sbcs x12, x10, x8+        sbcs x5, x6, x1+        sbcs x3, x21, xzr+        sbcs x19, x13, xzr+        sbc x24, x9, xzr+        adds x23, x15, x3+        adcs x8, x7, x19+        adcs x11, x20, x24+        adc x9, x22, xzr+        add x24, x9, #0x1+        lsl x7, x24, #32+        subs x21, x24, x7+        sbc x10, x7, xzr+        adds x6, x2, x21+        adcs x7, x12, x10+        adcs x24, x5, x24+        adcs x13, x23, xzr+        adcs x8, x8, xzr+        adcs x15, x11, xzr+        csetm x23, cc+        and x11, x16, x23+        and x20, x14, x23+        adds x22, x6, x20+        eor x3, x20, x23+        adcs x5, x7, x3+        adcs x14, x24, x11+        stp x22, x5, [x0]+        adcs x5, x13, x23+        adcs x21, x8, x23+        stp x14, x5, [x0, #16]+        adc x12, x15, x23+        stp x21, x12, [x0, #32]++// Restore registers and return++        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_montmul_p384)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/bignum_montmul_p384_alt.S view
@@ -0,0 +1,345 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Montgomery multiply, z := (x * y / 2^384) mod p_384+// Inputs x[6], y[6]; output z[6]+//+//    extern void bignum_montmul_p384_alt(uint64_t z[static 6],+//                                        const uint64_t x[static 6],+//                                        const uint64_t y[static 6]);+//+// Does z := (2^{-384} * x * y) mod p_384, assuming that the inputs x and y+// satisfy x * y <= 2^384 * p_384 (in particular this is true if we are in+// the "usual" case x < p_384 and y < p_384).+//+// Standard ARM ABI: X0 = z, X1 = x, X2 = y+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_montmul_p384_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_montmul_p384_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_montmul_p384_alt)+        .text+        .balign 4++// ---------------------------------------------------------------------------+// Core one-step "short" Montgomery reduction macro. Takes input in+// [d5;d4;d3;d2;d1;d0] and returns result in [d6;d5;d4;d3;d2;d1],+// adding to the existing contents of [d5;d4;d3;d2;d1]. It is fine+// for d6 to be the same register as d0.+//+// We want to add (2^384 - 2^128 - 2^96 + 2^32 - 1) * w+// where w = [d0 + (d0<<32)] mod 2^64+// ---------------------------------------------------------------------------++#define montreds(d6,d5,d4,d3,d2,d1,d0, t3,t2,t1)                            \+/* Our correction multiplier is w = [d0 + (d0<<32)] mod 2^64            */  \+/* Store it in d6 to make the 2^384 * w contribution already            */  \+        lsl     t1, d0, #32 __LF                                       \+        add     d6, t1, d0 __LF                                        \+/* Now let [t3;t2;t1;-] = (2^384 - p_384) * w                    */         \+/* We know the lowest word will cancel d0 so we don't need it    */         \+        mov     t1, #0xffffffff00000001 __LF                           \+        umulh   t1, t1, d6 __LF                                        \+        mov     t2, #0x00000000ffffffff __LF                           \+        mul     t3, t2, d6 __LF                                        \+        umulh   t2, t2, d6 __LF                                        \+        adds    t1, t1, t3 __LF                                        \+        adcs    t2, t2, d6 __LF                                        \+        adc     t3, xzr, xzr __LF                                      \+/* Now add it, by subtracting from 2^384 * w + x */                         \+        subs    d1, d1, t1 __LF                                        \+        sbcs    d2, d2, t2 __LF                                        \+        sbcs    d3, d3, t3 __LF                                        \+        sbcs    d4, d4, xzr __LF                                       \+        sbcs    d5, d5, xzr __LF                                       \+        sbc     d6, d6, xzr+++#define z x0+#define x x1+#define y x2++// These are repeated mod 2 as we load pairs of inputs++#define a0 x3+#define a1 x4+#define a2 x3+#define a3 x4+#define a4 x3+#define a5 x4++#define b0 x5+#define b1 x6+#define b2 x7+#define b3 x8+#define b4 x9+#define b5 x10++#define l x11++#define u0 x12+#define u1 x13+#define u2 x14+#define u3 x15+#define u4 x16+#define u5 x17+#define u6 x19+#define u7 x20+#define u8 x21+#define u9 x22+#define u10 x2 // same as y+#define u11 x1 // same as x+#define h b5 // same as b5++S2N_BN_SYMBOL(bignum_montmul_p384_alt):+        CFI_START++// Save more registers++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)++// Load operands and set up row 0 = [u6;...;u0] = a0 * [b5;...;b0]++        ldp     a0, a1, [x]+        ldp     b0, b1, [y]++        mul     u0, a0, b0+        umulh   u1, a0, b0+        mul     l, a0, b1+        umulh   u2, a0, b1+        adds    u1, u1, l++        ldp     b2, b3, [y, #16]++        mul     l, a0, b2+        umulh   u3, a0, b2+        adcs    u2, u2, l++        mul     l, a0, b3+        umulh   u4, a0, b3+        adcs    u3, u3, l++        ldp     b4, b5, [y, #32]++        mul     l, a0, b4+        umulh   u5, a0, b4+        adcs    u4, u4, l++        mul     l, a0, b5+        umulh   u6, a0, b5+        adcs    u5, u5, l++        adc     u6, u6, xzr++// Row 1 = [u7;...;u0] = [a1;a0] * [b5;...;b0]++        mul     l, a1, b0+        adds    u1, u1, l+        mul     l, a1, b1+        adcs    u2, u2, l+        mul     l, a1, b2+        adcs    u3, u3, l+        mul     l, a1, b3+        adcs    u4, u4, l+        mul     l, a1, b4+        adcs    u5, u5, l+        mul     l, a1, b5+        adcs    u6, u6, l+        cset    u7, cs++        umulh   l, a1, b0+        adds    u2, u2, l+        umulh   l, a1, b1+        adcs    u3, u3, l+        umulh   l, a1, b2+        adcs    u4, u4, l+        umulh   l, a1, b3+        adcs    u5, u5, l+        umulh   l, a1, b4+        adcs    u6, u6, l+        umulh   l, a1, b5+        adc     u7, u7, l++// Row 2 = [u8;...;u0] = [a2;a1;a0] * [b5;...;b0]++        ldp     a2, a3, [x, #16]++        mul     l, a2, b0+        adds    u2, u2, l+        mul     l, a2, b1+        adcs    u3, u3, l+        mul     l, a2, b2+        adcs    u4, u4, l+        mul     l, a2, b3+        adcs    u5, u5, l+        mul     l, a2, b4+        adcs    u6, u6, l+        mul     l, a2, b5+        adcs    u7, u7, l+        cset    u8, cs++        umulh   l, a2, b0+        adds    u3, u3, l+        umulh   l, a2, b1+        adcs    u4, u4, l+        umulh   l, a2, b2+        adcs    u5, u5, l+        umulh   l, a2, b3+        adcs    u6, u6, l+        umulh   l, a2, b4+        adcs    u7, u7, l+        umulh   l, a2, b5+        adc     u8, u8, l++// Row 3 = [u9;...;u0] = [a3;a2;a1;a0] * [b5;...;b0]++        mul     l, a3, b0+        adds    u3, u3, l+        mul     l, a3, b1+        adcs    u4, u4, l+        mul     l, a3, b2+        adcs    u5, u5, l+        mul     l, a3, b3+        adcs    u6, u6, l+        mul     l, a3, b4+        adcs    u7, u7, l+        mul     l, a3, b5+        adcs    u8, u8, l+        cset    u9, cs++        umulh   l, a3, b0+        adds    u4, u4, l+        umulh   l, a3, b1+        adcs    u5, u5, l+        umulh   l, a3, b2+        adcs    u6, u6, l+        umulh   l, a3, b3+        adcs    u7, u7, l+        umulh   l, a3, b4+        adcs    u8, u8, l+        umulh   l, a3, b5+        adc     u9, u9, l++// Row 4 = [u10;...;u0] = [a4;a3;a2;a1;a0] * [b5;...;b0]++        ldp     a4, a5, [x, #32]++        mul     l, a4, b0+        adds    u4, u4, l+        mul     l, a4, b1+        adcs    u5, u5, l+        mul     l, a4, b2+        adcs    u6, u6, l+        mul     l, a4, b3+        adcs    u7, u7, l+        mul     l, a4, b4+        adcs    u8, u8, l+        mul     l, a4, b5+        adcs    u9, u9, l+        cset    u10, cs++        umulh   l, a4, b0+        adds    u5, u5, l+        umulh   l, a4, b1+        adcs    u6, u6, l+        umulh   l, a4, b2+        adcs    u7, u7, l+        umulh   l, a4, b3+        adcs    u8, u8, l+        umulh   l, a4, b4+        adcs    u9, u9, l+        umulh   l, a4, b5+        adc     u10, u10, l++// Row 5 = [u11;...;u0] = [a5;a4;a3;a2;a1;a0] * [b5;...;b0]++        mul     l, a5, b0+        adds    u5, u5, l+        mul     l, a5, b1+        adcs    u6, u6, l+        mul     l, a5, b2+        adcs    u7, u7, l+        mul     l, a5, b3+        adcs    u8, u8, l+        mul     l, a5, b4+        adcs    u9, u9, l+        mul     l, a5, b5+        adcs    u10, u10, l+        cset    u11, cs++        umulh   l, a5, b0+        adds    u6, u6, l+        umulh   l, a5, b1+        adcs    u7, u7, l+        umulh   l, a5, b2+        adcs    u8, u8, l+        umulh   l, a5, b3+        adcs    u9, u9, l+        umulh   l, a5, b4+        adcs    u10, u10, l+        umulh   l, a5, b5+        adc     u11, u11, l++// Montgomery rotate the low half++        montreds(u0,u5,u4,u3,u2,u1,u0, b0,b1,b2)+        montreds(u1,u0,u5,u4,u3,u2,u1, b0,b1,b2)+        montreds(u2,u1,u0,u5,u4,u3,u2, b0,b1,b2)+        montreds(u3,u2,u1,u0,u5,u4,u3, b0,b1,b2)+        montreds(u4,u3,u2,u1,u0,u5,u4, b0,b1,b2)+        montreds(u5,u4,u3,u2,u1,u0,u5, b0,b1,b2)++// Add up the high and low parts as [h; u5;u4;u3;u2;u1;u0] = z++        adds    u0, u0, u6+        adcs    u1, u1, u7+        adcs    u2, u2, u8+        adcs    u3, u3, u9+        adcs    u4, u4, u10+        adcs    u5, u5, u11+        adc     h, xzr, xzr++// Now add [h; u11;u10;u9;u8;u7;u6] = z + (2^384 - p_384)++        mov     l, #0xffffffff00000001+        adds    u6, u0, l+        mov     l, #0x00000000ffffffff+        adcs    u7, u1, l+        mov     l, #0x0000000000000001+        adcs    u8, u2, l+        adcs    u9, u3, xzr+        adcs    u10, u4, xzr+        adcs    u11, u5, xzr+        adcs    h, h, xzr++// Now z >= p_384 iff h is nonzero, so select accordingly++        csel    u0, u0, u6, eq+        csel    u1, u1, u7, eq+        csel    u2, u2, u8, eq+        csel    u3, u3, u9, eq+        csel    u4, u4, u10, eq+        csel    u5, u5, u11, eq++// Store back final result++        stp     u0, u1, [z]+        stp     u2, u3, [z, #16]+        stp     u4, u5, [z, #32]++// Restore registers++        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_montmul_p384_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/bignum_montsqr_p384.S view
@@ -0,0 +1,671 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Montgomery square, z := (x^2 / 2^384) mod p_384+// Input x[6]; output z[6]+//+//    extern void bignum_montsqr_p384(uint64_t z[static 6],+//                                    const uint64_t x[static 6]);+//+// Does z := (x^2 / 2^384) mod p_384, assuming x^2 <= 2^384 * p_384, which is+// guaranteed in particular if x < p_384 initially (the "intended" case).+//+// Standard ARM ABI: X0 = z, X1 = x+// ----------------------------------------------------------------------------++// bignum_montsqr_p384 is functionally equivalent to+// unopt/bignum_montsqr_p384_base.+// It is written in a way that+// 1. A subset of scalar multiplications in bignum_montsqr_p384 are carefully+//    chosen and vectorized+// 2. The vectorized assembly is rescheduled using the SLOTHY superoptimizer.+//    https://github.com/slothy-optimizer/slothy+//+// The output program of step 1. is as follows:+//+//        ldp x9, x2, [x1]+//        ldr q18, [x1]+//        ldr q19, [x1]+//        ldp x4, x6, [x1, #16]+//        ldp x5, x10, [x1, #32]+//        ldr q21, [x1, #32]+//        ldr q28, [x1, #32]+//        mul x12, x9, x2+//        mul x1, x9, x4+//        mul x13, x2, x4+//        movi v0.2D, #0x00000000ffffffff+//        uzp2 v5.4S, v19.4S, v19.4S+//        xtn v25.2S, v18.2D+//        xtn v4.2S, v19.2D+//        rev64 v23.4S, v19.4S+//        umull v20.2D, v25.2S, v4.2S+//        umull v30.2D, v25.2S, v5.2S+//        uzp2 v19.4S, v18.4S, v18.4S+//        mul v22.4S, v23.4S, v18.4S+//        usra v30.2D, v20.2D, #32+//        umull v18.2D, v19.2S, v5.2S+//        uaddlp v22.2D, v22.4S+//        and v20.16B, v30.16B, v0.16B+//        umlal v20.2D, v19.2S, v4.2S+//        shl v19.2D, v22.2D, #32+//        usra v18.2D, v30.2D, #32+//        umlal v19.2D, v25.2S, v4.2S+//        usra v18.2D, v20.2D, #32+//        mov x7, v19.d[0]+//        mov x17, v19.d[1]+//        mul x16, x4, x4+//        umulh x3, x9, x2+//        adds x15, x1, x3+//        umulh x1, x9, x4+//        adcs x13, x13, x1+//        umulh x1, x2, x4+//        adcs x8, x1, xzr+//        mov x11, v18.d[0]+//        mov x14, v18.d[1]+//        umulh x1, x4, x4+//        adds x3, x12, x12+//        adcs x15, x15, x15+//        adcs x13, x13, x13+//        adcs x12, x8, x8+//        adc x1, x1, xzr+//        adds x11, x11, x3+//        adcs x3, x17, x15+//        adcs x17, x14, x13+//        adcs x15, x16, x12+//        adc x13, x1, xzr+//        lsl x1, x7, #32+//        add x16, x1, x7+//        lsr x1, x16, #32+//        subs x12, x1, x16+//        sbc x1, x16, xzr+//        extr x12, x1, x12, #32+//        lsr x1, x1, #32+//        adds x7, x1, x16+//        adc x1, xzr, xzr+//        subs x12, x11, x12+//        sbcs x11, x3, x7+//        sbcs x17, x17, x1+//        sbcs x15, x15, xzr+//        sbcs x13, x13, xzr+//        sbc x3, x16, xzr+//        lsl x1, x12, #32+//        add x16, x1, x12+//        lsr x1, x16, #32+//        subs x12, x1, x16+//        sbc x1, x16, xzr+//        extr x12, x1, x12, #32+//        lsr x1, x1, #32+//        adds x7, x1, x16+//        adc x1, xzr, xzr+//        subs x12, x11, x12+//        sbcs x17, x17, x7+//        sbcs x15, x15, x1+//        sbcs x13, x13, xzr+//        sbcs x11, x3, xzr+//        sbc x3, x16, xzr+//        lsl x1, x12, #32+//        add x16, x1, x12+//        lsr x1, x16, #32+//        subs x12, x1, x16+//        sbc x1, x16, xzr+//        extr x7, x1, x12, #32+//        lsr x1, x1, #32+//        adds x12, x1, x16+//        adc x1, xzr, xzr+//        subs x17, x17, x7+//        sbcs x15, x15, x12+//        sbcs x13, x13, x1+//        sbcs x7, x11, xzr+//        sbcs x12, x3, xzr+//        sbc x1, x16, xzr+//        stp x17, x15, [x0]                     // @slothy:writes=buffer0+//        stp x13, x7, [x0, #16]                 // @slothy:writes=buffer16+//        stp x12, x1, [x0, #32]                 // @slothy:writes=buffer32+//        mul x14, x9, x6+//        mul x15, x2, x5+//        mul x13, x4, x10+//        umulh x7, x9, x6+//        umulh x12, x2, x5+//        umulh x1, x4, x10+//        adds x15, x7, x15+//        adcs x16, x12, x13+//        adc x13, x1, xzr+//        adds x11, x15, x14+//        adcs x7, x16, x15+//        adcs x12, x13, x16+//        adc x1, x13, xzr+//        adds x17, x7, x14+//        adcs x15, x12, x15+//        adcs x3, x1, x16+//        adc x16, x13, xzr+//        subs x1, x9, x2+//        cneg x13, x1, cc+//        csetm x7, cc+//        subs x1, x5, x6+//        cneg x1, x1, cc+//        mul x12, x13, x1+//        umulh x1, x13, x1+//        cinv x7, x7, cc+//        eor x12, x12, x7+//        eor x1, x1, x7+//        cmn x7, #0x1+//        adcs x11, x11, x12+//        adcs x17, x17, x1+//        adcs x15, x15, x7+//        adcs x3, x3, x7+//        adc x16, x16, x7+//        subs x9, x9, x4+//        cneg x13, x9, cc+//        csetm x7, cc+//        subs x1, x10, x6+//        cneg x1, x1, cc+//        mul x12, x13, x1+//        umulh x1, x13, x1+//        cinv x7, x7, cc+//        eor x12, x12, x7+//        eor x1, x1, x7+//        cmn x7, #0x1+//        adcs x17, x17, x12+//        adcs x15, x15, x1+//        adcs x13, x3, x7+//        adc x7, x16, x7+//        subs x2, x2, x4+//        cneg x12, x2, cc+//        csetm x1, cc+//        subs x2, x10, x5+//        cneg x2, x2, cc+//        mul x4, x12, x2+//        umulh x2, x12, x2+//        cinv x1, x1, cc+//        eor x4, x4, x1+//        eor x2, x2, x1+//        cmn x1, #0x1+//        adcs x12, x15, x4+//        adcs x4, x13, x2+//        adc x2, x7, x1+//        adds x1, x14, x14+//        adcs x16, x11, x11+//        adcs x17, x17, x17+//        adcs x15, x12, x12+//        adcs x13, x4, x4+//        adcs x7, x2, x2+//        adc x12, xzr, xzr+//        ldp x4, x2, [x0]                       // @slothy:reads=buffer0+//        adds x1, x1, x4+//        adcs x16, x16, x2+//        ldp x4, x2, [x0, #16]                  // @slothy:reads=buffer16+//        adcs x17, x17, x4+//        adcs x15, x15, x2+//        ldp x4, x2, [x0, #32]                  // @slothy:reads=buffer32+//        adcs x13, x13, x4+//        adcs x7, x7, x2+//        adc x11, x12, xzr+//        lsl x2, x1, #32+//        add x12, x2, x1+//        lsr x2, x12, #32+//        subs x4, x2, x12+//        sbc x2, x12, xzr+//        extr x4, x2, x4, #32+//        lsr x2, x2, #32+//        adds x1, x2, x12+//        adc x2, xzr, xzr+//        subs x4, x16, x4+//        sbcs x16, x17, x1+//        sbcs x17, x15, x2+//        sbcs x15, x13, xzr+//        sbcs x13, x7, xzr+//        sbc x7, x12, xzr+//        lsl x2, x4, #32+//        add x12, x2, x4+//        lsr x2, x12, #32+//        subs x4, x2, x12+//        sbc x2, x12, xzr+//        extr x4, x2, x4, #32+//        lsr x2, x2, #32+//        adds x1, x2, x12+//        adc x2, xzr, xzr+//        subs x4, x16, x4+//        sbcs x16, x17, x1+//        sbcs x17, x15, x2+//        sbcs x15, x13, xzr+//        sbcs x13, x7, xzr+//        sbc x7, x12, xzr+//        lsl x2, x4, #32+//        add x12, x2, x4+//        lsr x2, x12, #32+//        subs x4, x2, x12+//        sbc x2, x12, xzr+//        extr x1, x2, x4, #32+//        lsr x2, x2, #32+//        adds x4, x2, x12+//        adc x2, xzr, xzr+//        subs x3, x16, x1+//        sbcs x17, x17, x4+//        sbcs x15, x15, x2+//        sbcs x1, x13, xzr+//        sbcs x4, x7, xzr+//        sbc x2, x12, xzr+//        adds x13, x11, x1+//        adcs x7, x4, xzr+//        adcs x12, x2, xzr+//        adcs x16, xzr, xzr+//        mul x2, x6, x6+//        adds x3, x3, x2+//        xtn v30.2S, v28.2D+//        shrn v26.2S, v28.2D, #32+//        umull v26.2D, v30.2S, v26.2S+//        shl v19.2D, v26.2D, #33+//        umlal v19.2D, v30.2S, v30.2S+//        mov x1, v19.d[0]+//        mov x4, v19.d[1]+//        umulh x2, x6, x6+//        adcs x17, x17, x2+//        umulh x2, x5, x5+//        adcs x15, x15, x1+//        adcs x13, x13, x2+//        umulh x2, x10, x10+//        adcs x7, x7, x4+//        adcs x12, x12, x2+//        adc x16, x16, xzr+//        dup v28.2D, x6+//        movi v0.2D, #0x00000000ffffffff+//        uzp2 v5.4S, v21.4S, v21.4S+//        xtn v25.2S, v28.2D+//        xtn v4.2S, v21.2D+//        rev64 v19.4S, v21.4S+//        umull v30.2D, v25.2S, v4.2S+//        umull v23.2D, v25.2S, v5.2S+//        uzp2 v20.4S, v28.4S, v28.4S+//        mul v19.4S, v19.4S, v28.4S+//        usra v23.2D, v30.2D, #32+//        umull v18.2D, v20.2S, v5.2S+//        uaddlp v19.2D, v19.4S+//        and v30.16B, v23.16B, v0.16B+//        umlal v30.2D, v20.2S, v4.2S+//        shl v19.2D, v19.2D, #32+//        usra v18.2D, v23.2D, #32+//        umlal v19.2D, v25.2S, v4.2S+//        usra v18.2D, v30.2D, #32+//        mov x6, v19.d[0]+//        mov x1, v19.d[1]+//        mul x4, x5, x10+//        mov x2, v18.d[0]+//        adds x1, x1, x2+//        mov x2, v18.d[1]+//        adcs x4, x4, x2+//        umulh x5, x5, x10+//        adc x2, x5, xzr+//        adds x5, x6, x6+//        adcs x6, x1, x1+//        adcs x1, x4, x4+//        adcs x4, x2, x2+//        adc x2, xzr, xzr+//        adds x17, x17, x5+//        adcs x15, x15, x6+//        adcs x13, x13, x1+//        adcs x7, x7, x4+//        adcs x12, x12, x2+//        adc x2, x16, xzr+//        mov x5, #0xffffffff00000001+//        mov x6, #0xffffffff+//        mov x1, #0x1+//        cmn x3, x5+//        adcs xzr, x17, x6+//        adcs xzr, x15, x1+//        adcs xzr, x13, xzr+//        adcs xzr, x7, xzr+//        adcs xzr, x12, xzr+//        adc x2, x2, xzr+//        neg x4, x2+//        and x2, x5, x4+//        adds x10, x3, x2+//        and x2, x6, x4+//        adcs x5, x17, x2+//        and x2, x1, x4+//        adcs x6, x15, x2+//        adcs x1, x13, xzr+//        adcs x4, x7, xzr+//        adc x2, x12, xzr+//        stp x10, x5, [x0]                      // @slothy:writes=buffer0+//        stp x6, x1, [x0, #16]                  // @slothy:writes=buffer16+//        stp x4, x2, [x0, #32]                  // @slothy:writes=buffer32+//        ret+//+// The bash script used for step 2 is as follows:+//+//        # Store the assembly instructions except the last 'ret' as, say, 'input.S'.+//        export OUTPUTS="[hint_buffer0,hint_buffer16,hint_buffer32]"+//        export RESERVED_REGS="[x18,x19,x20,x21,x22,x23,x24,x25,x26,x27,x28,x29,x30,sp,q8,q9,q10,q11,q12,q13,q14,q15,v8,v9,v10,v11,v12,v13,v14,v15]"+//        <s2n-bignum>/tools/external/slothy.sh input.S my_out_dir+//        # my_out_dir/3.opt.s is the optimized assembly. Its output may differ+//        # from this file since the sequence is non-deterministically chosen.+//        # Please add 'ret' at the end of the output assembly.++++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_montsqr_p384)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_montsqr_p384)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_montsqr_p384)+        .text+        .balign 4++S2N_BN_SYMBOL(bignum_montsqr_p384):+        CFI_START++        ldr q1, [x1]+        ldp x9, x2, [x1]+        ldr q0, [x1]+        ldp x4, x6, [x1, #16]+        rev64 v21.4S, v1.4S+        uzp2 v28.4S, v1.4S, v1.4S+        umulh x7, x9, x2+        xtn v17.2S, v1.2D+        mul v27.4S, v21.4S, v0.4S+        ldr q20, [x1, #32]+        xtn v30.2S, v0.2D+        ldr q1, [x1, #32]+        uzp2 v31.4S, v0.4S, v0.4S+        ldp x5, x10, [x1, #32]+        umulh x8, x9, x4+        uaddlp v3.2D, v27.4S+        umull v16.2D, v30.2S, v17.2S+        mul x16, x9, x4+        umull v27.2D, v30.2S, v28.2S+        shrn v0.2S, v20.2D, #32+        xtn v7.2S, v20.2D+        shl v20.2D, v3.2D, #32+        umull v3.2D, v31.2S, v28.2S+        mul x3, x2, x4+        umlal v20.2D, v30.2S, v17.2S+        umull v22.2D, v7.2S, v0.2S+        usra v27.2D, v16.2D, #32+        umulh x11, x2, x4+        movi v21.2D, #0x00000000ffffffff+        uzp2 v28.4S, v1.4S, v1.4S+        adds x15, x16, x7+        and v5.16B, v27.16B, v21.16B+        adcs x3, x3, x8+        usra v3.2D, v27.2D, #32+        dup v29.2D, x6+        adcs x16, x11, xzr+        mov x14, v20.d[0]+        umlal v5.2D, v31.2S, v17.2S+        mul x8, x9, x2+        mov x7, v20.d[1]+        shl v19.2D, v22.2D, #33+        xtn v25.2S, v29.2D+        rev64 v31.4S, v1.4S+        lsl x13, x14, #32+        uzp2 v6.4S, v29.4S, v29.4S+        umlal v19.2D, v7.2S, v7.2S+        usra v3.2D, v5.2D, #32+        adds x1, x8, x8+        umulh x8, x4, x4+        add x12, x13, x14+        mul v17.4S, v31.4S, v29.4S+        xtn v4.2S, v1.2D+        adcs x14, x15, x15+        lsr x13, x12, #32+        adcs x15, x3, x3+        umull v31.2D, v25.2S, v28.2S+        adcs x11, x16, x16+        umull v21.2D, v25.2S, v4.2S+        mov x17, v3.d[0]+        umull v18.2D, v6.2S, v28.2S+        adc x16, x8, xzr+        uaddlp v16.2D, v17.4S+        movi v1.2D, #0x00000000ffffffff+        subs x13, x13, x12+        usra v31.2D, v21.2D, #32+        sbc x8, x12, xzr+        adds x17, x17, x1+        mul x1, x4, x4+        shl v28.2D, v16.2D, #32+        mov x3, v3.d[1]+        adcs x14, x7, x14+        extr x7, x8, x13, #32+        adcs x13, x3, x15+        and v3.16B, v31.16B, v1.16B+        adcs x11, x1, x11+        lsr x1, x8, #32+        umlal v3.2D, v6.2S, v4.2S+        usra v18.2D, v31.2D, #32+        adc x3, x16, xzr+        adds x1, x1, x12+        umlal v28.2D, v25.2S, v4.2S+        adc x16, xzr, xzr+        subs x15, x17, x7+        sbcs x7, x14, x1+        lsl x1, x15, #32+        sbcs x16, x13, x16+        add x8, x1, x15+        usra v18.2D, v3.2D, #32+        sbcs x14, x11, xzr+        lsr x1, x8, #32+        sbcs x17, x3, xzr+        sbc x11, x12, xzr+        subs x13, x1, x8+        umulh x12, x4, x10+        sbc x1, x8, xzr+        extr x13, x1, x13, #32+        lsr x1, x1, #32+        adds x15, x1, x8+        adc x1, xzr, xzr+        subs x7, x7, x13+        sbcs x13, x16, x15+        lsl x3, x7, #32+        umulh x16, x2, x5+        sbcs x15, x14, x1+        add x7, x3, x7+        sbcs x3, x17, xzr+        lsr x1, x7, #32+        sbcs x14, x11, xzr+        sbc x11, x8, xzr+        subs x8, x1, x7+        sbc x1, x7, xzr+        extr x8, x1, x8, #32+        lsr x1, x1, #32+        adds x1, x1, x7+        adc x17, xzr, xzr+        subs x13, x13, x8+        umulh x8, x9, x6+        sbcs x1, x15, x1+        sbcs x15, x3, x17+        sbcs x3, x14, xzr+        mul x17, x2, x5+        sbcs x11, x11, xzr+        stp x13, x1, [x0]                       // @slothy:writes=buffer0+        sbc x14, x7, xzr+        mul x7, x4, x10+        subs x1, x9, x2+        stp x15, x3, [x0, #16]                  // @slothy:writes=buffer16+        csetm x15, cc+        cneg x1, x1, cc+        stp x11, x14, [x0, #32]                 // @slothy:writes=buffer32+        mul x14, x9, x6+        adds x17, x8, x17+        adcs x7, x16, x7+        adc x13, x12, xzr+        subs x12, x5, x6+        cneg x3, x12, cc+        cinv x16, x15, cc+        mul x8, x1, x3+        umulh x1, x1, x3+        eor x12, x8, x16+        adds x11, x17, x14+        adcs x3, x7, x17+        adcs x15, x13, x7+        adc x8, x13, xzr+        adds x3, x3, x14+        adcs x15, x15, x17+        adcs x17, x8, x7+        eor x1, x1, x16+        adc x13, x13, xzr+        subs x9, x9, x4+        csetm x8, cc+        cneg x9, x9, cc+        subs x4, x2, x4+        cneg x4, x4, cc+        csetm x7, cc+        subs x2, x10, x6+        cinv x8, x8, cc+        cneg x2, x2, cc+        cmn x16, #0x1+        adcs x11, x11, x12+        mul x12, x9, x2+        adcs x3, x3, x1+        adcs x15, x15, x16+        umulh x9, x9, x2+        adcs x17, x17, x16+        adc x13, x13, x16+        subs x1, x10, x5+        cinv x2, x7, cc+        cneg x1, x1, cc+        eor x9, x9, x8+        cmn x8, #0x1+        eor x7, x12, x8+        mul x12, x4, x1+        adcs x3, x3, x7+        adcs x7, x15, x9+        adcs x15, x17, x8+        ldp x9, x17, [x0, #16]                  // @slothy:reads=buffer16+        umulh x4, x4, x1+        adc x8, x13, x8+        cmn x2, #0x1+        eor x1, x12, x2+        adcs x1, x7, x1+        ldp x7, x16, [x0]                       // @slothy:reads=buffer0+        eor x12, x4, x2+        adcs x4, x15, x12+        ldp x15, x12, [x0, #32]                 // @slothy:reads=buffer32+        adc x8, x8, x2+        adds x13, x14, x14+        umulh x14, x5, x10+        adcs x2, x11, x11+        adcs x3, x3, x3+        adcs x1, x1, x1+        adcs x4, x4, x4+        adcs x11, x8, x8+        adc x8, xzr, xzr+        adds x13, x13, x7+        adcs x2, x2, x16+        mul x16, x5, x10+        adcs x3, x3, x9+        adcs x1, x1, x17+        umulh x5, x5, x5+        lsl x9, x13, #32+        add x9, x9, x13+        adcs x4, x4, x15+        mov x13, v28.d[1]+        adcs x15, x11, x12+        lsr x7, x9, #32+        adc x11, x8, xzr+        subs x7, x7, x9+        umulh x10, x10, x10+        sbc x17, x9, xzr+        extr x7, x17, x7, #32+        lsr x17, x17, #32+        adds x17, x17, x9+        adc x12, xzr, xzr+        subs x8, x2, x7+        sbcs x17, x3, x17+        lsl x7, x8, #32+        sbcs x2, x1, x12+        add x3, x7, x8+        sbcs x12, x4, xzr+        lsr x1, x3, #32+        sbcs x7, x15, xzr+        sbc x15, x9, xzr+        subs x1, x1, x3+        sbc x4, x3, xzr+        lsr x9, x4, #32+        extr x8, x4, x1, #32+        adds x9, x9, x3+        adc x4, xzr, xzr+        subs x1, x17, x8+        lsl x17, x1, #32+        sbcs x8, x2, x9+        sbcs x9, x12, x4+        add x17, x17, x1+        mov x1, v18.d[1]+        lsr x2, x17, #32+        sbcs x7, x7, xzr+        mov x12, v18.d[0]+        sbcs x15, x15, xzr+        sbc x3, x3, xzr+        subs x4, x2, x17+        sbc x2, x17, xzr+        adds x12, x13, x12+        adcs x16, x16, x1+        lsr x13, x2, #32+        extr x1, x2, x4, #32+        adc x2, x14, xzr+        adds x4, x13, x17+        mul x13, x6, x6+        adc x14, xzr, xzr+        subs x1, x8, x1+        sbcs x4, x9, x4+        mov x9, v28.d[0]+        sbcs x7, x7, x14+        sbcs x8, x15, xzr+        sbcs x3, x3, xzr+        sbc x14, x17, xzr+        adds x17, x9, x9+        adcs x12, x12, x12+        mov x15, v19.d[0]+        adcs x9, x16, x16+        umulh x6, x6, x6+        adcs x16, x2, x2+        adc x2, xzr, xzr+        adds x11, x11, x8+        adcs x3, x3, xzr+        adcs x14, x14, xzr+        adcs x8, xzr, xzr+        adds x13, x1, x13+        mov x1, v19.d[1]+        adcs x6, x4, x6+        mov x4, #0xffffffff+        adcs x15, x7, x15+        adcs x7, x11, x5+        adcs x1, x3, x1+        adcs x14, x14, x10+        adc x11, x8, xzr+        adds x6, x6, x17+        adcs x8, x15, x12+        adcs x3, x7, x9+        adcs x15, x1, x16+        mov x16, #0xffffffff00000001+        adcs x14, x14, x2+        mov x2, #0x1+        adc x17, x11, xzr+        cmn x13, x16+        adcs xzr, x6, x4+        adcs xzr, x8, x2+        adcs xzr, x3, xzr+        adcs xzr, x15, xzr+        adcs xzr, x14, xzr+        adc x1, x17, xzr+        neg x9, x1+        and x1, x16, x9+        adds x11, x13, x1+        and x13, x4, x9+        adcs x5, x6, x13+        and x1, x2, x9+        adcs x7, x8, x1+        stp x11, x5, [x0]                       // @slothy:writes=buffer0+        adcs x11, x3, xzr+        adcs x2, x15, xzr+        stp x7, x11, [x0, #16]                  // @slothy:writes=buffer16+        adc x17, x14, xzr+        stp x2, x17, [x0, #32]                  // depth 72 // @slothy:writes=buffer32++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_montsqr_p384)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/bignum_montsqr_p384_alt.S view
@@ -0,0 +1,273 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Montgomery square, z := (x^2 / 2^384) mod p_384+// Input x[6]; output z[6]+//+//    extern void bignum_montsqr_p384_alt(uint64_t z[static 6],+//                                        const uint64_t x[static 6]);+//+// Does z := (x^2 / 2^384) mod p_384, assuming x^2 <= 2^384 * p_384, which is+// guaranteed in particular if x < p_384 initially (the "intended" case).+//+// Standard ARM ABI: X0 = z, X1 = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_montsqr_p384_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_montsqr_p384_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_montsqr_p384_alt)+        .text+        .balign 4++// ---------------------------------------------------------------------------+// Core one-step "short" Montgomery reduction macro. Takes input in+// [d5;d4;d3;d2;d1;d0] and returns result in [d6;d5;d4;d3;d2;d1],+// adding to the existing contents of [d5;d4;d3;d2;d1]. It is fine+// for d6 to be the same register as d0.+//+// We want to add (2^384 - 2^128 - 2^96 + 2^32 - 1) * w+// where w = [d0 + (d0<<32)] mod 2^64+// ---------------------------------------------------------------------------++#define montreds(d6,d5,d4,d3,d2,d1,d0, t3,t2,t1)                            \+/* Our correction multiplier is w = [d0 + (d0<<32)] mod 2^64            */  \+/* Store it in d6 to make the 2^384 * w contribution already            */  \+        lsl     t1, d0, #32 __LF                                       \+        add     d6, t1, d0 __LF                                        \+/* Now let [t3;t2;t1;-] = (2^384 - p_384) * w                    */         \+/* We know the lowest word will cancel d0 so we don't need it    */         \+        mov     t1, #0xffffffff00000001 __LF                           \+        umulh   t1, t1, d6 __LF                                        \+        mov     t2, #0x00000000ffffffff __LF                           \+        mul     t3, t2, d6 __LF                                        \+        umulh   t2, t2, d6 __LF                                        \+        adds    t1, t1, t3 __LF                                        \+        adcs    t2, t2, d6 __LF                                        \+        adc     t3, xzr, xzr __LF                                      \+/* Now add it, by subtracting from 2^384 * w + x */                         \+        subs    d1, d1, t1 __LF                                        \+        sbcs    d2, d2, t2 __LF                                        \+        sbcs    d3, d3, t3 __LF                                        \+        sbcs    d4, d4, xzr __LF                                       \+        sbcs    d5, d5, xzr __LF                                       \+        sbc     d6, d6, xzr++#define z x0+#define x x1++#define a0 x2+#define a1 x3+#define a2 x4+#define a3 x5+#define a4 x6+#define a5 x7++#define l x8++#define u0 x2 // The same as a0, which is safe+#define u1 x9+#define u2 x10+#define u3 x11+#define u4 x12+#define u5 x13+#define u6 x14+#define u7 x15+#define u8 x16+#define u9 x17+#define u10 x19+#define u11 x20+#define h x6 // same as a4++S2N_BN_SYMBOL(bignum_montsqr_p384_alt):+        CFI_START++// It's convenient to have two more registers to play with++        CFI_PUSH2(x19,x20)++// Load all the elements as [a5;a4;a3;a2;a1;a0], set up an initial+// window [u8;u7; u6;u5; u4;u3; u2;u1] = [34;05;03;01], and then+// chain in the addition of 02 + 12 + 13 + 14 + 15 to that window+// (no carry-out possible since we add it to the top of a product).++        ldp     a0, a1, [x]++        mul     u1, a0, a1+        umulh   u2, a0, a1++        ldp     a2, a3, [x, #16]++        mul     l, a0, a2+        adds    u2, u2, l++        mul     u3, a0, a3+        mul     l, a1, a2+        adcs    u3, u3, l++        umulh   u4, a0, a3+        mul     l, a1, a3+        adcs    u4, u4, l++        ldp     a4, a5, [x, #32]++        mul     u5, a0, a5+        mul     l, a1, a4+        adcs    u5, u5, l++        umulh   u6, a0, a5+        mul     l, a1, a5+        adcs    u6, u6, l++        mul     u7, a3, a4+        adcs    u7, u7, xzr++        umulh   u8, a3, a4+        adc     u8, u8, xzr++        umulh   l, a0, a2+        adds    u3, u3, l+        umulh   l, a1, a2+        adcs    u4, u4, l+        umulh   l, a1, a3+        adcs    u5, u5, l+        umulh   l, a1, a4+        adcs    u6, u6, l+        umulh   l, a1, a5+        adcs    u7, u7, l+        adc     u8, u8, xzr++// Now chain in the 04 + 23 + 24 + 25 + 35 + 45 terms++        mul     l, a0, a4+        adds    u4, u4, l+        mul     l, a2, a3+        adcs    u5, u5, l+        mul     l, a2, a4+        adcs    u6, u6, l+        mul     l, a2, a5+        adcs    u7, u7, l+        mul     l, a3, a5+        adcs    u8, u8, l+        mul     u9, a4, a5+        adcs    u9, u9, xzr+        umulh   u10, a4, a5+        adc     u10, u10, xzr++        umulh   l, a0, a4+        adds    u5, u5, l+        umulh   l, a2, a3+        adcs    u6, u6, l+        umulh   l, a2, a4+        adcs    u7, u7, l+        umulh   l, a2, a5+        adcs    u8, u8, l+        umulh   l, a3, a5+        adcs    u9, u9, l+        adc     u10, u10, xzr++// Double that, with u11 holding the top carry++        adds    u1, u1, u1+        adcs    u2, u2, u2+        adcs    u3, u3, u3+        adcs    u4, u4, u4+        adcs    u5, u5, u5+        adcs    u6, u6, u6+        adcs    u7, u7, u7+        adcs    u8, u8, u8+        adcs    u9, u9, u9+        adcs    u10, u10, u10+        cset    u11, cs++// Add the homogeneous terms 00 + 11 + 22 + 33 + 44 + 55++        umulh   l, a0, a0+        mul     u0, a0, a0+        adds    u1, u1, l++        mul     l, a1, a1+        adcs    u2, u2, l+        umulh   l, a1, a1+        adcs    u3, u3, l++        mul     l, a2, a2+        adcs    u4, u4, l+        umulh   l, a2, a2+        adcs    u5, u5, l++        mul     l, a3, a3+        adcs    u6, u6, l+        umulh   l, a3, a3+        adcs    u7, u7, l++        mul     l, a4, a4+        adcs    u8, u8, l+        umulh   l, a4, a4+        adcs    u9, u9, l++        mul     l, a5, a5+        adcs    u10, u10, l+        umulh   l, a5, a5+        adc     u11, u11, l++// Montgomery rotate the low half++        montreds(u0,u5,u4,u3,u2,u1,u0, a1,a2,a3)+        montreds(u1,u0,u5,u4,u3,u2,u1, a1,a2,a3)+        montreds(u2,u1,u0,u5,u4,u3,u2, a1,a2,a3)+        montreds(u3,u2,u1,u0,u5,u4,u3, a1,a2,a3)+        montreds(u4,u3,u2,u1,u0,u5,u4, a1,a2,a3)+        montreds(u5,u4,u3,u2,u1,u0,u5, a1,a2,a3)++// Add up the high and low parts as [h; u5;u4;u3;u2;u1;u0] = z++        adds    u0, u0, u6+        adcs    u1, u1, u7+        adcs    u2, u2, u8+        adcs    u3, u3, u9+        adcs    u4, u4, u10+        adcs    u5, u5, u11+        adc     h, xzr, xzr++// Now add [h; u11;u10;u9;u8;u7;u6] = z + (2^384 - p_384)++        mov     l, #0xffffffff00000001+        adds    u6, u0, l+        mov     l, #0x00000000ffffffff+        adcs    u7, u1, l+        mov     l, #0x0000000000000001+        adcs    u8, u2, l+        adcs    u9, u3, xzr+        adcs    u10, u4, xzr+        adcs    u11, u5, xzr+        adcs    h, h, xzr++// Now z >= p_384 iff h is nonzero, so select accordingly++        csel    u0, u0, u6, eq+        csel    u1, u1, u7, eq+        csel    u2, u2, u8, eq+        csel    u3, u3, u9, eq+        csel    u4, u4, u10, eq+        csel    u5, u5, u11, eq++// Store back final result++        stp     u0, u1, [z]+        stp     u2, u3, [z, #16]+        stp     u4, u5, [z, #32]++// Restore registers++        CFI_POP2(x19,x20)++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_montsqr_p384_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/bignum_mul_p521.S view
@@ -0,0 +1,1407 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Multiply modulo p_521, z := (x * y) mod p_521, assuming x and y reduced+// Inputs x[9], y[9]; output z[9]+//+//    extern void bignum_mul_p521(uint64_t z[static 9], const uint64_t x[static 9],+//                                const uint64_t y[static 9]);+//+// Standard ARM ABI: X0 = z, X1 = x, X2 = y+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++// bignum_mul_p521 is functionally equivalent to unopt/bignum_mul_p521_base.+// It is written in a way that+// 1. A subset of scalar multiplications in bignum_montmul_p384 are carefully+//    chosen and vectorized+// 2. The vectorized assembly is rescheduled using the SLOTHY superoptimizer.+//    https://github.com/slothy-optimizer/slothy+//+// The output program of step 1. is as follows:+//+//        stp     x19, x20, [sp, #-16]!+//        stp     x21, x22, [sp, #-16]!+//        stp     x23, x24, [sp, #-16]!+//        stp     x25, x26, [sp, #-16]!+//        sub     sp, sp, #80+//        ldp x15, x21, [x1]+//        ldp x10, x17, [x1, #16]+//        ldp x13, x16, [x2]+//        ldr q18, [x1]+//        ldr q28, [x2]+//        ldp x5, x20, [x2, #16]+//        movi v16.2D, #0x00000000ffffffff+//        uzp2 v7.4S, v28.4S, v28.4S+//        xtn v4.2S, v18.2D+//        xtn v1.2S, v28.2D+//        rev64 v27.4S, v28.4S+//        umull v21.2D, v4.2S, v1.2S+//        umull v28.2D, v4.2S, v7.2S+//        uzp2 v5.4S, v18.4S, v18.4S+//        mul v18.4S, v27.4S, v18.4S+//        usra v28.2D, v21.2D, #32+//        umull v29.2D, v5.2S, v7.2S+//        uaddlp v18.2D, v18.4S+//        and v16.16B, v28.16B, v16.16B+//        umlal v16.2D, v5.2S, v1.2S+//        shl v18.2D, v18.2D, #32+//        usra v29.2D, v28.2D, #32+//        umlal v18.2D, v4.2S, v1.2S+//        usra v29.2D, v16.2D, #32+//        mov x8, v18.d[0]+//        mov x9, v18.d[1]+//        mul x6, x10, x5+//        mul x19, x17, x20+//        mov x14, v29.d[0]+//        adds x9, x9, x14+//        mov x14, v29.d[1]+//        adcs x6, x6, x14+//        umulh x14, x10, x5+//        adcs x19, x19, x14+//        umulh x14, x17, x20+//        adc x14, x14, xzr+//        adds x11, x9, x8+//        adcs x9, x6, x9+//        adcs x6, x19, x6+//        adcs x19, x14, x19+//        adc x14, xzr, x14+//        adds x3, x9, x8+//        adcs x24, x6, x11+//        adcs x9, x19, x9+//        adcs x6, x14, x6+//        adcs x19, xzr, x19+//        adc x14, xzr, x14+//        subs x4, x10, x17+//        cneg x4, x4, cc+//        csetm x7, cc+//        subs x23, x20, x5+//        cneg x23, x23, cc+//        mul x22, x4, x23+//        umulh x4, x4, x23+//        cinv x7, x7, cc+//        cmn x7, #0x1+//        eor x23, x22, x7+//        adcs x6, x6, x23+//        eor x4, x4, x7+//        adcs x19, x19, x4+//        adc x14, x14, x7+//        subs x4, x15, x21+//        cneg x4, x4, cc+//        csetm x7, cc+//        subs x23, x16, x13+//        cneg x23, x23, cc+//        mul x22, x4, x23+//        umulh x4, x4, x23+//        cinv x7, x7, cc+//        cmn x7, #0x1+//        eor x23, x22, x7+//        adcs x11, x11, x23+//        eor x4, x4, x7+//        adcs x3, x3, x4+//        adcs x24, x24, x7+//        adcs x9, x9, x7+//        adcs x6, x6, x7+//        adcs x19, x19, x7+//        adc x14, x14, x7+//        subs x4, x21, x17+//        cneg x4, x4, cc+//        csetm x7, cc+//        subs x23, x20, x16+//        cneg x23, x23, cc+//        mul x22, x4, x23+//        umulh x4, x4, x23+//        cinv x7, x7, cc+//        cmn x7, #0x1+//        eor x23, x22, x7+//        adcs x9, x9, x23+//        eor x4, x4, x7+//        adcs x6, x6, x4+//        adcs x19, x19, x7+//        adc x14, x14, x7+//        subs x4, x15, x10+//        cneg x4, x4, cc+//        csetm x7, cc+//        subs x23, x5, x13+//        cneg x23, x23, cc+//        mul x22, x4, x23+//        umulh x4, x4, x23+//        cinv x7, x7, cc+//        cmn x7, #0x1+//        eor x23, x22, x7+//        adcs x3, x3, x23+//        eor x4, x4, x7+//        adcs x24, x24, x4+//        adcs x9, x9, x7+//        adcs x6, x6, x7+//        adcs x19, x19, x7+//        adc x14, x14, x7+//        subs x17, x15, x17+//        cneg x17, x17, cc+//        csetm x4, cc+//        subs x13, x20, x13+//        cneg x13, x13, cc+//        mul x20, x17, x13+//        umulh x17, x17, x13+//        cinv x13, x4, cc+//        cmn x13, #0x1+//        eor x20, x20, x13+//        adcs x20, x24, x20+//        eor x17, x17, x13+//        adcs x17, x9, x17+//        adcs x9, x6, x13+//        adcs x6, x19, x13+//        adc x13, x14, x13+//        subs x21, x21, x10+//        cneg x21, x21, cc+//        csetm x10, cc+//        subs x16, x5, x16+//        cneg x16, x16, cc+//        mul x5, x21, x16+//        umulh x21, x21, x16+//        cinv x10, x10, cc+//        cmn x10, #0x1+//        eor x16, x5, x10+//        adcs x16, x20, x16+//        eor x21, x21, x10+//        adcs x21, x17, x21+//        adcs x17, x9, x10+//        adcs x5, x6, x10+//        adc x10, x13, x10+//        lsl x13, x8, #9+//        extr x20, x11, x8, #55+//        extr x8, x3, x11, #55+//        extr x9, x16, x3, #55+//        lsr x16, x16, #55+//        stp x21, x17, [sp]                       // @slothy:writes=stack0+//        stp x5, x10, [sp, #16]                   // @slothy:writes=stack16+//        stp x13, x20, [sp, #32]                  // @slothy:writes=stack32+//        stp x8, x9, [sp, #48]                    // @slothy:writes=stack48+//        str x16, [sp, #64]                       // @slothy:writes=stack64+//        ldp x21, x10, [x1, #32]+//        ldp x17, x13, [x1, #48]+//        ldp x16, x5, [x2, #32]+//        ldr q18, [x1, #32]+//        ldr q28, [x2, #32]+//        ldp x20, x8, [x2, #48]+//        movi v16.2D, #0x00000000ffffffff+//        uzp2 v7.4S, v28.4S, v28.4S+//        xtn v4.2S, v18.2D+//        xtn v1.2S, v28.2D+//        rev64 v28.4S, v28.4S+//        umull v27.2D, v4.2S, v1.2S+//        umull v29.2D, v4.2S, v7.2S+//        uzp2 v21.4S, v18.4S, v18.4S+//        mul v28.4S, v28.4S, v18.4S+//        usra v29.2D, v27.2D, #32+//        umull v18.2D, v21.2S, v7.2S+//        uaddlp v28.2D, v28.4S+//        and v16.16B, v29.16B, v16.16B+//        umlal v16.2D, v21.2S, v1.2S+//        shl v28.2D, v28.2D, #32+//        usra v18.2D, v29.2D, #32+//        umlal v28.2D, v4.2S, v1.2S+//        usra v18.2D, v16.2D, #32+//        mov x9, v28.d[0]+//        mov x6, v28.d[1]+//        mul x19, x17, x20+//        mul x14, x13, x8+//        mov x11, v18.d[0]+//        adds x6, x6, x11+//        mov x11, v18.d[1]+//        adcs x19, x19, x11+//        umulh x11, x17, x20+//        adcs x14, x14, x11+//        umulh x11, x13, x8+//        adc x11, x11, xzr+//        adds x3, x6, x9+//        adcs x6, x19, x6+//        adcs x19, x14, x19+//        adcs x14, x11, x14+//        adc x11, xzr, x11+//        adds x24, x6, x9+//        adcs x4, x19, x3+//        adcs x6, x14, x6+//        adcs x19, x11, x19+//        adcs x14, xzr, x14+//        adc x11, xzr, x11+//        subs x7, x17, x13+//        cneg x7, x7, cc+//        csetm x23, cc+//        subs x22, x8, x20+//        cneg x22, x22, cc+//        mul x12, x7, x22+//        umulh x7, x7, x22+//        cinv x23, x23, cc+//        cmn x23, #0x1+//        eor x22, x12, x23+//        adcs x19, x19, x22+//        eor x7, x7, x23+//        adcs x14, x14, x7+//        adc x11, x11, x23+//        subs x7, x21, x10+//        cneg x7, x7, cc+//        csetm x23, cc+//        subs x22, x5, x16+//        cneg x22, x22, cc+//        mul x12, x7, x22+//        umulh x7, x7, x22+//        cinv x23, x23, cc+//        cmn x23, #0x1+//        eor x22, x12, x23+//        adcs x3, x3, x22+//        eor x7, x7, x23+//        adcs x24, x24, x7+//        adcs x4, x4, x23+//        adcs x6, x6, x23+//        adcs x19, x19, x23+//        adcs x14, x14, x23+//        adc x11, x11, x23+//        subs x7, x10, x13+//        cneg x7, x7, cc+//        csetm x23, cc+//        subs x22, x8, x5+//        cneg x22, x22, cc+//        mul x12, x7, x22+//        umulh x7, x7, x22+//        cinv x23, x23, cc+//        cmn x23, #0x1+//        eor x22, x12, x23+//        adcs x6, x6, x22+//        eor x7, x7, x23+//        adcs x19, x19, x7+//        adcs x14, x14, x23+//        adc x11, x11, x23+//        subs x7, x21, x17+//        cneg x7, x7, cc+//        csetm x23, cc+//        subs x22, x20, x16+//        cneg x22, x22, cc+//        mul x12, x7, x22+//        umulh x7, x7, x22+//        cinv x23, x23, cc+//        cmn x23, #0x1+//        eor x22, x12, x23+//        adcs x24, x24, x22+//        eor x7, x7, x23+//        adcs x4, x4, x7+//        adcs x6, x6, x23+//        adcs x19, x19, x23+//        adcs x14, x14, x23+//        adc x11, x11, x23+//        subs x7, x21, x13+//        cneg x7, x7, cc+//        csetm x23, cc+//        subs x22, x8, x16+//        cneg x22, x22, cc+//        mul x12, x7, x22+//        umulh x7, x7, x22+//        cinv x23, x23, cc+//        cmn x23, #0x1+//        eor x22, x12, x23+//        adcs x4, x4, x22+//        eor x7, x7, x23+//        adcs x6, x6, x7+//        adcs x19, x19, x23+//        adcs x14, x14, x23+//        adc x11, x11, x23+//        subs x7, x10, x17+//        cneg x7, x7, cc+//        csetm x23, cc+//        subs x22, x20, x5+//        cneg x22, x22, cc+//        mul x12, x7, x22+//        umulh x7, x7, x22+//        cinv x23, x23, cc+//        cmn x23, #0x1+//        eor x22, x12, x23+//        adcs x4, x4, x22+//        eor x7, x7, x23+//        adcs x6, x6, x7+//        adcs x19, x19, x23+//        adcs x14, x14, x23+//        adc x11, x11, x23+//        ldp x7, x23, [sp]                        // @slothy:reads=stack0+//        adds x9, x9, x7+//        adcs x3, x3, x23+//        stp x9, x3, [sp]                         // @slothy:writes=stack0+//        ldp x9, x3, [sp, #16]                    // @slothy:reads=stack16+//        adcs x9, x24, x9+//        adcs x3, x4, x3+//        stp x9, x3, [sp, #16]                    // @slothy:writes=stack16+//        ldp x9, x3, [sp, #32]                    // @slothy:reads=stack32+//        adcs x9, x6, x9+//        adcs x6, x19, x3+//        stp x9, x6, [sp, #32]                    // @slothy:writes=stack32+//        ldp x9, x6, [sp, #48]                    // @slothy:reads=stack48+//        adcs x9, x14, x9+//        adcs x6, x11, x6+//        stp x9, x6, [sp, #48]                    // @slothy:writes=stack48+//        ldr x9, [sp, #64]                        // @slothy:reads=stack64+//        adc x9, x9, xzr+//        str x9, [sp, #64]                        // @slothy:writes=stack64+//        ldp x9, x6, [x1]+//        subs x21, x21, x9+//        sbcs x10, x10, x6+//        ldp x9, x6, [x1, #16]+//        sbcs x17, x17, x9+//        sbcs x13, x13, x6+//        csetm x9, cc+//        ldp x6, x19, [x2]+//        subs x16, x6, x16+//        sbcs x5, x19, x5+//        ldp x6, x19, [x2, #16]+//        sbcs x20, x6, x20+//        sbcs x8, x19, x8+//        csetm x6, cc+//        eor x21, x21, x9+//        subs x21, x21, x9+//        eor x10, x10, x9+//        sbcs x10, x10, x9+//        eor x17, x17, x9+//        sbcs x17, x17, x9+//        eor x13, x13, x9+//        sbc x13, x13, x9+//        eor x16, x16, x6+//        subs x16, x16, x6+//        eor x5, x5, x6+//        sbcs x5, x5, x6+//        eor x20, x20, x6+//        sbcs x20, x20, x6+//        eor x8, x8, x6+//        sbc x8, x8, x6+//        eor x9, x6, x9+//        mul x6, x21, x16+//        mul x19, x10, x5+//        mul x14, x17, x20+//        mul x11, x13, x8+//        umulh x3, x21, x16+//        adds x19, x19, x3+//        umulh x3, x10, x5+//        adcs x14, x14, x3+//        umulh x3, x17, x20+//        adcs x11, x11, x3+//        umulh x3, x13, x8+//        adc x3, x3, xzr+//        adds x24, x19, x6+//        adcs x19, x14, x19+//        adcs x14, x11, x14+//        adcs x11, x3, x11+//        adc x3, xzr, x3+//        adds x4, x19, x6+//        adcs x7, x14, x24+//        adcs x19, x11, x19+//        adcs x14, x3, x14+//        adcs x11, xzr, x11+//        adc x3, xzr, x3+//        subs x23, x17, x13+//        cneg x23, x23, cc+//        csetm x22, cc+//        subs x12, x8, x20+//        cneg x12, x12, cc+//        mul x15, x23, x12+//        umulh x23, x23, x12+//        cinv x22, x22, cc+//        cmn x22, #0x1+//        eor x12, x15, x22+//        adcs x14, x14, x12+//        eor x23, x23, x22+//        adcs x11, x11, x23+//        adc x3, x3, x22+//        subs x23, x21, x10+//        cneg x23, x23, cc+//        csetm x22, cc+//        subs x12, x5, x16+//        cneg x12, x12, cc+//        mul x15, x23, x12+//        umulh x23, x23, x12+//        cinv x22, x22, cc+//        cmn x22, #0x1+//        eor x12, x15, x22+//        adcs x24, x24, x12+//        eor x23, x23, x22+//        adcs x4, x4, x23+//        adcs x7, x7, x22+//        adcs x19, x19, x22+//        adcs x14, x14, x22+//        adcs x11, x11, x22+//        adc x3, x3, x22+//        subs x23, x10, x13+//        cneg x23, x23, cc+//        csetm x22, cc+//        subs x12, x8, x5+//        cneg x12, x12, cc+//        mul x15, x23, x12+//        umulh x23, x23, x12+//        cinv x22, x22, cc+//        cmn x22, #0x1+//        eor x12, x15, x22+//        adcs x19, x19, x12+//        eor x23, x23, x22+//        adcs x14, x14, x23+//        adcs x11, x11, x22+//        adc x3, x3, x22+//        subs x23, x21, x17+//        cneg x23, x23, cc+//        csetm x22, cc+//        subs x12, x20, x16+//        cneg x12, x12, cc+//        mul x15, x23, x12+//        umulh x23, x23, x12+//        cinv x22, x22, cc+//        cmn x22, #0x1+//        eor x12, x15, x22+//        adcs x4, x4, x12+//        eor x23, x23, x22+//        adcs x7, x7, x23+//        adcs x19, x19, x22+//        adcs x14, x14, x22+//        adcs x11, x11, x22+//        adc x3, x3, x22+//        subs x21, x21, x13+//        cneg x21, x21, cc+//        csetm x13, cc+//        subs x16, x8, x16+//        cneg x16, x16, cc+//        mul x8, x21, x16+//        umulh x21, x21, x16+//        cinv x13, x13, cc+//        cmn x13, #0x1+//        eor x16, x8, x13+//        adcs x16, x7, x16+//        eor x21, x21, x13+//        adcs x21, x19, x21+//        adcs x8, x14, x13+//        adcs x19, x11, x13+//        adc x13, x3, x13+//        subs x10, x10, x17+//        cneg x10, x10, cc+//        csetm x17, cc+//        subs x5, x20, x5+//        cneg x5, x5, cc+//        mul x20, x10, x5+//        umulh x10, x10, x5+//        cinv x17, x17, cc+//        cmn x17, #0x1+//        eor x5, x20, x17+//        adcs x16, x16, x5+//        eor x10, x10, x17+//        adcs x21, x21, x10+//        adcs x10, x8, x17+//        adcs x5, x19, x17+//        adc x17, x13, x17+//        ldp x13, x20, [sp]                       // @slothy:reads=stack0+//        ldp x8, x19, [sp, #16]                   // @slothy:reads=stack16+//        eor x6, x6, x9+//        adds x6, x6, x13+//        eor x14, x24, x9+//        adcs x14, x14, x20+//        eor x11, x4, x9+//        adcs x11, x11, x8+//        eor x16, x16, x9+//        adcs x16, x16, x19+//        eor x21, x21, x9+//        ldp x3, x24, [sp, #32]                   // @slothy:reads=stack32+//        ldp x4, x7, [sp, #48]                    // @slothy:reads=stack48+//        ldr x23, [sp, #64]                       // @slothy:reads=stack64+//        adcs x21, x21, x3+//        eor x10, x10, x9+//        adcs x10, x10, x24+//        eor x5, x5, x9+//        adcs x5, x5, x4+//        eor x17, x17, x9+//        adcs x17, x17, x7+//        adc x22, x23, xzr+//        adds x21, x21, x13+//        adcs x10, x10, x20+//        adcs x13, x5, x8+//        adcs x17, x17, x19+//        and x5, x9, #0x1ff+//        lsl x20, x6, #9+//        orr x5, x20, x5+//        adcs x5, x3, x5+//        extr x20, x14, x6, #55+//        adcs x20, x24, x20+//        extr x8, x11, x14, #55+//        adcs x8, x4, x8+//        extr x9, x16, x11, #55+//        adcs x9, x7, x9+//        lsr x16, x16, #55+//        adc x16, x16, x23+//        ldr x6, [x2, #64]+//        ldp x19, x14, [x1]+//        and x11, x19, #0xfffffffffffff+//        mul x11, x6, x11+//        ldr x3, [x1, #64]+//        ldp x24, x4, [x2]+//        and x7, x24, #0xfffffffffffff+//        mul x7, x3, x7+//        add x11, x11, x7+//        extr x19, x14, x19, #52+//        and x19, x19, #0xfffffffffffff+//        mul x19, x6, x19+//        extr x24, x4, x24, #52+//        and x24, x24, #0xfffffffffffff+//        mul x24, x3, x24+//        add x19, x19, x24+//        lsr x24, x11, #52+//        add x19, x19, x24+//        lsl x11, x11, #12+//        extr x11, x19, x11, #12+//        adds x21, x21, x11+//        ldp x11, x24, [x1, #16]+//        ldp x7, x23, [x2, #16]+//        extr x14, x11, x14, #40+//        and x14, x14, #0xfffffffffffff+//        mul x14, x6, x14+//        extr x4, x7, x4, #40+//        and x4, x4, #0xfffffffffffff+//        mul x4, x3, x4+//        add x14, x14, x4+//        lsr x4, x19, #52+//        add x14, x14, x4+//        lsl x19, x19, #12+//        extr x19, x14, x19, #24+//        adcs x10, x10, x19+//        extr x19, x24, x11, #28+//        and x19, x19, #0xfffffffffffff+//        mul x19, x6, x19+//        extr x11, x23, x7, #28+//        and x11, x11, #0xfffffffffffff+//        mul x11, x3, x11+//        add x19, x19, x11+//        lsr x11, x14, #52+//        add x19, x19, x11+//        lsl x14, x14, #12+//        extr x14, x19, x14, #36+//        adcs x13, x13, x14+//        and x14, x10, x13+//        ldp x11, x4, [x1, #32]+//        ldp x7, x12, [x2, #32]+//        extr x24, x11, x24, #16+//        and x24, x24, #0xfffffffffffff+//        mul x24, x6, x24+//        extr x23, x7, x23, #16+//        and x23, x23, #0xfffffffffffff+//        mul x23, x3, x23+//        add x24, x24, x23+//        lsl x23, x22, #48+//        add x24, x24, x23+//        lsr x23, x19, #52+//        add x24, x24, x23+//        lsl x19, x19, #12+//        extr x19, x24, x19, #48+//        adcs x17, x17, x19+//        and x19, x14, x17+//        lsr x14, x11, #4+//        and x14, x14, #0xfffffffffffff+//        mul x14, x6, x14+//        lsr x23, x7, #4+//        and x23, x23, #0xfffffffffffff+//        mul x23, x3, x23+//        add x14, x14, x23+//        lsr x23, x24, #52+//        add x14, x14, x23+//        lsl x24, x24, #12+//        extr x24, x14, x24, #60+//        extr x11, x4, x11, #56+//        and x11, x11, #0xfffffffffffff+//        mul x11, x6, x11+//        extr x7, x12, x7, #56+//        and x7, x7, #0xfffffffffffff+//        mul x7, x3, x7+//        add x11, x11, x7+//        lsr x14, x14, #52+//        add x14, x11, x14+//        lsl x11, x24, #8+//        extr x11, x14, x11, #8+//        adcs x5, x5, x11+//        and x19, x19, x5+//        ldp x11, x24, [x1, #48]+//        ldp x2, x7, [x2, #48]+//        extr x4, x11, x4, #44+//        and x4, x4, #0xfffffffffffff+//        mul x4, x6, x4+//        extr x23, x2, x12, #44+//        and x23, x23, #0xfffffffffffff+//        mul x23, x3, x23+//        add x4, x4, x23+//        lsr x23, x14, #52+//        add x4, x4, x23+//        lsl x14, x14, #12+//        extr x14, x4, x14, #20+//        adcs x20, x20, x14+//        and x19, x19, x20+//        extr x14, x24, x11, #32+//        and x14, x14, #0xfffffffffffff+//        mul x14, x6, x14+//        extr x2, x7, x2, #32+//        and x2, x2, #0xfffffffffffff+//        mul x2, x3, x2+//        add x2, x14, x2+//        lsr x14, x4, #52+//        add x2, x2, x14+//        lsl x14, x4, #12+//        extr x14, x2, x14, #32+//        adcs x8, x8, x14+//        and x19, x19, x8+//        lsr x14, x24, #20+//        mul x14, x6, x14+//        lsr x11, x7, #20+//        mul x11, x3, x11+//        add x14, x14, x11+//        lsr x11, x2, #52+//        add x14, x14, x11+//        lsl x2, x2, #12+//        extr x2, x14, x2, #44+//        adcs x9, x9, x2+//        and x2, x19, x9+//        mul x6, x6, x3+//        lsr x19, x14, #44+//        add x6, x6, x19+//        adc x16, x16, x6+//        lsr x6, x16, #9+//        orr x16, x16, #0xfffffffffffffe00+//        cmp xzr, xzr+//        adcs xzr, x21, x6+//        adcs xzr, x2, xzr+//        adcs xzr, x16, xzr+//        adcs x21, x21, x6+//        adcs x10, x10, xzr+//        adcs x13, x13, xzr+//        adcs x17, x17, xzr+//        adcs x5, x5, xzr+//        adcs x20, x20, xzr+//        adcs x8, x8, xzr+//        adcs x9, x9, xzr+//        adc x16, x16, xzr+//        and x2, x21, #0x1ff+//        extr x21, x10, x21, #9+//        extr x10, x13, x10, #9+//        stp x21, x10, [x0]                       // @slothy:writes=buffer0+//        extr x21, x17, x13, #9+//        extr x10, x5, x17, #9+//        stp x21, x10, [x0, #16]                  // @slothy:writes=buffer16+//        extr x21, x20, x5, #9+//        extr x10, x8, x20, #9+//        stp x21, x10, [x0, #32]                  // @slothy:writes=buffer32+//        extr x21, x9, x8, #9+//        extr x10, x16, x9, #9+//        stp x21, x10, [x0, #48]                  // @slothy:writes=buffer48+//        str x2, [x0, #64]                        // @slothy:writes=buffer64+//        add     sp, sp, #80+//        ldp     x25, x26, [sp], #16+//        ldp     x23, x24, [sp], #16+//        ldp     x21, x22, [sp], #16+//        ldp     x19, x20, [sp], #16+//        ret+//+// The bash script used for step 2 is as follows:+//+//        # Store the assembly instructions except the last 'ret',+//        # callee-register store/loads and add/sub sp #80 as, say, 'input.S'.+//        export OUTPUTS="[hint_buffer0,hint_buffer16,hint_buffer32,hint_buffer48,hint_buffer64]"+//        export RESERVED_REGS="[x18,x27,x28,x29,x30,sp,q8,q9,q10,q11,q12,q13,q14,q15,v8,v9,v10,v11,v12,v13,v14,v15]"+//        <s2n-bignum>/tools/external/slothy.sh input.S my_out_dir+//        # my_out_dir/3.opt.s is the optimized assembly. Its output may differ+//        # from this file since the sequence is non-deterministically chosen.+//        # Please add 'ret' at the end of the output assembly.++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_mul_p521)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_mul_p521)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_mul_p521)+        .text+        .balign 4++S2N_BN_SYMBOL(bignum_mul_p521):+        CFI_START++// Save registers and make space for the temporary buffer++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x26)+        CFI_DEC_SP(80)++        ldr q6, [x2]+        ldp x10, x17, [x1, #16]+        ldr q4, [x1]+        ldr q16, [x2, #32]+        ldp x5, x20, [x2, #16]+        ldr q2, [x1, #32]+        movi v31.2D, #0x00000000ffffffff+        uzp2 v17.4S, v6.4S, v6.4S+        rev64 v7.4S, v6.4S+        ldp x15, x21, [x1]+        xtn v25.2S, v6.2D+        xtn v22.2S, v4.2D+        subs x14, x10, x17+        mul v7.4S, v7.4S, v4.4S+        csetm x8, cc+        rev64 v3.4S, v16.4S+        xtn v1.2S, v16.2D+        ldp x13, x16, [x2]+        mul x26, x10, x5+        uzp2 v16.4S, v16.4S, v16.4S+        uaddlp v26.2D, v7.4S+        cneg x4, x14, cc+        subs x24, x15, x21+        xtn v5.2S, v2.2D+        mul v28.4S, v3.4S, v2.4S+        shl v26.2D, v26.2D, #32+        mul x22, x17, x20+        umull v20.2D, v22.2S, v25.2S+        uzp2 v6.4S, v4.4S, v4.4S+        umull v18.2D, v22.2S, v17.2S+        uzp2 v4.4S, v2.4S, v2.4S+        cneg x14, x24, cc+        csetm x7, cc+        umulh x11, x17, x20+        usra v18.2D, v20.2D, #32+        uaddlp v7.2D, v28.4S+        subs x19, x16, x13+        umlal v26.2D, v22.2S, v25.2S+        cneg x19, x19, cc+        shl v28.2D, v7.2D, #32+        umull v7.2D, v5.2S, v1.2S+        umull v30.2D, v5.2S, v16.2S+        cinv x6, x7, cc+        mul x25, x14, x19+        umlal v28.2D, v5.2S, v1.2S+        umull v21.2D, v6.2S, v17.2S+        umulh x14, x14, x19+        usra v30.2D, v7.2D, #32+        subs x9, x20, x5+        and v29.16B, v18.16B, v31.16B+        cinv x23, x8, cc+        mov x8, v26.d[1]+        cneg x12, x9, cc+        usra v21.2D, v18.2D, #32+        umlal v29.2D, v6.2S, v25.2S+        mul x24, x4, x12+        umull v18.2D, v4.2S, v16.2S+        movi v25.2D, #0x00000000ffffffff+        eor x9, x14, x6+        and v7.16B, v30.16B, v25.16B+        usra v21.2D, v29.2D, #32+        umulh x7, x10, x5+        usra v18.2D, v30.2D, #32+        umlal v7.2D, v4.2S, v1.2S+        mov x19, v21.d[0]+        umulh x3, x4, x12+        mov x14, v21.d[1]+        usra v18.2D, v7.2D, #32+        adds x4, x8, x19+        mov x8, v26.d[0]+        adcs x19, x26, x14+        adcs x14, x22, x7+        adc x12, x11, xzr+        adds x11, x4, x8+        adcs x26, x19, x4+        adcs x22, x14, x19+        eor x4, x24, x23+        adcs x14, x12, x14+        eor x7, x25, x6+        adc x25, xzr, x12+        eor x19, x3, x23+        adds x3, x26, x8+        adcs x24, x22, x11+        adcs x12, x14, x26+        adcs x22, x25, x22+        adcs x26, xzr, x14+        adc x14, xzr, x25+        cmn x23, #0x1+        adcs x22, x22, x4+        adcs x19, x26, x19+        adc x25, x14, x23+        subs x14, x21, x17+        cneg x23, x14, cc+        csetm x26, cc+        subs x4, x20, x16+        cneg x14, x4, cc+        cinv x4, x26, cc+        cmn x6, #0x1+        adcs x11, x11, x7+        mul x7, x23, x14+        adcs x9, x3, x9+        adcs x26, x24, x6+        umulh x3, x23, x14+        adcs x14, x12, x6+        adcs x22, x22, x6+        adcs x12, x19, x6+        extr x24, x11, x8, #55+        adc x6, x25, x6+        subs x19, x15, x17+        csetm x17, cc+        cneg x23, x19, cc+        subs x19, x20, x13+        lsl x25, x8, #9+        eor x8, x7, x4+        cneg x20, x19, cc+        umulh x7, x23, x20+        cinv x19, x17, cc+        subs x17, x15, x10+        csetm x15, cc+        stp x25, x24, [sp, #32]+        cneg x24, x17, cc+        mul x20, x23, x20+        subs x25, x5, x13+        cneg x13, x25, cc+        cinv x15, x15, cc+        mul x25, x24, x13+        subs x21, x21, x10+        csetm x23, cc+        cneg x17, x21, cc+        subs x21, x5, x16+        umulh x13, x24, x13+        cinv x10, x23, cc+        cneg x23, x21, cc+        cmn x4, #0x1+        adcs x14, x14, x8+        eor x21, x3, x4+        adcs x21, x22, x21+        eor x5, x20, x19+        adcs x24, x12, x4+        mul x12, x17, x23+        eor x8, x25, x15+        adc x25, x6, x4+        cmn x15, #0x1+        adcs x6, x9, x8+        ldp x20, x8, [x2, #48]+        eor x9, x13, x15+        adcs x4, x26, x9+        umulh x26, x17, x23+        ldp x17, x13, [x1, #48]+        adcs x9, x14, x15+        adcs x16, x21, x15+        adcs x14, x24, x15+        eor x21, x7, x19+        mul x23, x17, x20+        adc x24, x25, x15+        cmn x19, #0x1+        adcs x7, x4, x5+        adcs x9, x9, x21+        umulh x3, x13, x8+        adcs x16, x16, x19+        adcs x22, x14, x19+        eor x5, x12, x10+        adc x12, x24, x19+        cmn x10, #0x1+        adcs x19, x7, x5+        eor x14, x26, x10+        mov x7, v28.d[1]+        adcs x24, x9, x14+        extr x4, x19, x6, #55+        umulh x15, x17, x20+        mov x14, v18.d[1]+        lsr x9, x19, #55+        adcs x5, x16, x10+        mov x16, v18.d[0]+        adcs x19, x22, x10+        str x9, [sp, #64]+        extr x25, x6, x11, #55+        adc x21, x12, x10+        subs x26, x17, x13+        stp x25, x4, [sp, #48]+        stp x19, x21, [sp, #16]+        csetm x6, cc+        cneg x4, x26, cc+        mul x19, x13, x8+        subs x11, x8, x20+        stp x24, x5, [sp]+        ldp x21, x10, [x1, #32]+        cinv x12, x6, cc+        cneg x6, x11, cc+        mov x9, v28.d[0]+        umulh x25, x4, x6+        adds x22, x7, x16+        ldp x16, x5, [x2, #32]+        adcs x14, x23, x14+        adcs x11, x19, x15+        adc x24, x3, xzr+        adds x3, x22, x9+        adcs x15, x14, x22+        mul x22, x4, x6+        adcs x6, x11, x14+        adcs x4, x24, x11+        eor x14, x25, x12+        adc x26, xzr, x24+        subs x7, x21, x10+        csetm x23, cc+        cneg x19, x7, cc+        subs x24, x5, x16+        cneg x11, x24, cc+        cinv x7, x23, cc+        adds x25, x15, x9+        eor x23, x22, x12+        adcs x22, x6, x3+        mul x24, x19, x11+        adcs x15, x4, x15+        adcs x6, x26, x6+        umulh x19, x19, x11+        adcs x11, xzr, x4+        adc x26, xzr, x26+        cmn x12, #0x1+        adcs x4, x6, x23+        eor x6, x24, x7+        adcs x14, x11, x14+        adc x26, x26, x12+        subs x11, x10, x13+        cneg x12, x11, cc+        csetm x11, cc+        eor x19, x19, x7+        subs x24, x8, x5+        cinv x11, x11, cc+        cneg x24, x24, cc+        cmn x7, #0x1+        adcs x3, x3, x6+        mul x23, x12, x24+        adcs x25, x25, x19+        adcs x6, x22, x7+        umulh x19, x12, x24+        adcs x22, x15, x7+        adcs x12, x4, x7+        eor x24, x23, x11+        adcs x4, x14, x7+        adc x26, x26, x7+        eor x19, x19, x11+        subs x14, x21, x17+        cneg x7, x14, cc+        csetm x14, cc+        subs x23, x20, x16+        cinv x14, x14, cc+        cneg x23, x23, cc+        cmn x11, #0x1+        adcs x22, x22, x24+        mul x24, x7, x23+        adcs x15, x12, x19+        adcs x4, x4, x11+        adc x19, x26, x11+        umulh x26, x7, x23+        subs x7, x21, x13+        eor x11, x24, x14+        cneg x23, x7, cc+        csetm x12, cc+        subs x7, x8, x16+        cneg x7, x7, cc+        cinv x12, x12, cc+        cmn x14, #0x1+        eor x26, x26, x14+        adcs x11, x25, x11+        mul x25, x23, x7+        adcs x26, x6, x26+        adcs x6, x22, x14+        adcs x24, x15, x14+        umulh x23, x23, x7+        adcs x4, x4, x14+        adc x22, x19, x14+        eor x14, x25, x12+        eor x7, x23, x12+        cmn x12, #0x1+        adcs x14, x26, x14+        ldp x19, x25, [x2]+        ldp x15, x23, [x2, #16]+        adcs x26, x6, x7+        adcs x24, x24, x12+        adcs x7, x4, x12+        adc x4, x22, x12+        subs x19, x19, x16+        ldp x16, x22, [x1]+        sbcs x6, x25, x5+        ldp x12, x25, [x1, #16]+        sbcs x15, x15, x20+        sbcs x8, x23, x8+        csetm x23, cc+        subs x21, x21, x16+        eor x16, x19, x23+        sbcs x19, x10, x22+        eor x22, x6, x23+        eor x8, x8, x23+        sbcs x6, x17, x12+        sbcs x13, x13, x25+        csetm x12, cc+        subs x10, x10, x17+        cneg x17, x10, cc+        csetm x25, cc+        subs x5, x20, x5+        eor x10, x19, x12+        cneg x19, x5, cc+        eor x20, x15, x23+        eor x21, x21, x12+        cinv x15, x25, cc+        mul x25, x17, x19+        subs x16, x16, x23+        sbcs x5, x22, x23+        eor x6, x6, x12+        sbcs x20, x20, x23+        eor x22, x13, x12+        sbc x8, x8, x23+        subs x21, x21, x12+        umulh x19, x17, x19+        sbcs x10, x10, x12+        sbcs x17, x6, x12+        eor x6, x19, x15+        eor x19, x25, x15+        umulh x25, x17, x20+        sbc x13, x22, x12+        cmn x15, #0x1+        adcs x22, x14, x19+        adcs x19, x26, x6+        ldp x6, x26, [sp]+        adcs x14, x24, x15+        umulh x24, x21, x16+        adcs x7, x7, x15+        adc x15, x4, x15+        adds x4, x9, x6+        eor x9, x23, x12+        adcs x12, x3, x26+        stp x4, x12, [sp]+        ldp x4, x26, [sp, #16]+        umulh x12, x10, x5+        ldp x6, x23, [sp, #32]+        adcs x3, x11, x4+        mul x4, x13, x8+        adcs x26, x22, x26+        ldp x22, x11, [sp, #48]+        adcs x6, x19, x6+        stp x3, x26, [sp, #16]+        mul x26, x10, x5+        adcs x14, x14, x23+        stp x6, x14, [sp, #32]+        ldr x6, [sp, #64]+        adcs x22, x7, x22+        adcs x14, x15, x11+        mul x11, x17, x20+        adc x19, x6, xzr+        stp x22, x14, [sp, #48]+        adds x14, x26, x24+        str x19, [sp, #64]+        umulh x19, x13, x8+        adcs x7, x11, x12+        adcs x22, x4, x25+        mul x6, x21, x16+        adc x19, x19, xzr+        subs x11, x17, x13+        cneg x12, x11, cc+        csetm x11, cc+        subs x24, x8, x20+        cinv x11, x11, cc+        cneg x24, x24, cc+        adds x4, x14, x6+        adcs x14, x7, x14+        mul x3, x12, x24+        adcs x7, x22, x7+        adcs x22, x19, x22+        umulh x12, x12, x24+        adc x24, xzr, x19+        adds x19, x14, x6+        eor x3, x3, x11+        adcs x26, x7, x4+        adcs x14, x22, x14+        adcs x25, x24, x7+        adcs x23, xzr, x22+        eor x7, x12, x11+        adc x12, xzr, x24+        subs x22, x21, x10+        cneg x24, x22, cc+        csetm x22, cc+        subs x15, x5, x16+        cinv x22, x22, cc+        cneg x15, x15, cc+        cmn x11, #0x1+        adcs x3, x25, x3+        mul x25, x24, x15+        adcs x23, x23, x7+        adc x11, x12, x11+        subs x7, x10, x13+        umulh x15, x24, x15+        cneg x12, x7, cc+        csetm x7, cc+        eor x24, x25, x22+        eor x25, x15, x22+        cmn x22, #0x1+        adcs x24, x4, x24+        adcs x19, x19, x25+        adcs x15, x26, x22+        adcs x4, x14, x22+        adcs x26, x3, x22+        adcs x25, x23, x22+        adc x23, x11, x22+        subs x14, x21, x17+        cneg x3, x14, cc+        csetm x11, cc+        subs x14, x8, x5+        cneg x14, x14, cc+        cinv x7, x7, cc+        subs x13, x21, x13+        cneg x21, x13, cc+        csetm x13, cc+        mul x22, x12, x14+        subs x8, x8, x16+        cinv x13, x13, cc+        umulh x14, x12, x14+        cneg x12, x8, cc+        subs x8, x20, x16+        cneg x8, x8, cc+        cinv x16, x11, cc+        eor x22, x22, x7+        cmn x7, #0x1+        eor x14, x14, x7+        adcs x4, x4, x22+        mul x11, x3, x8+        adcs x22, x26, x14+        adcs x14, x25, x7+        eor x25, x24, x9+        adc x26, x23, x7+        umulh x7, x3, x8+        subs x17, x10, x17+        cneg x24, x17, cc+        eor x3, x11, x16+        csetm x11, cc+        subs x20, x20, x5+        cneg x5, x20, cc+        cinv x11, x11, cc+        cmn x16, #0x1+        mul x17, x21, x12+        eor x8, x7, x16+        adcs x10, x19, x3+        and x19, x9, #0x1ff+        adcs x20, x15, x8+        umulh x15, x21, x12+        eor x12, x10, x9+        eor x8, x6, x9+        adcs x6, x4, x16+        adcs x4, x22, x16+        adcs x21, x14, x16+        adc x7, x26, x16+        mul x10, x24, x5+        cmn x13, #0x1+        ldp x3, x14, [x1]+        eor x17, x17, x13+        umulh x5, x24, x5+        adcs x20, x20, x17+        eor x17, x15, x13+        adcs x16, x6, x17+        eor x22, x10, x11+        adcs x23, x4, x13+        extr x10, x14, x3, #52+        and x26, x3, #0xfffffffffffff+        adcs x24, x21, x13+        and x15, x10, #0xfffffffffffff+        adc x6, x7, x13+        cmn x11, #0x1+        adcs x17, x20, x22+        eor x4, x5, x11+        ldp x21, x10, [sp]+        adcs x7, x16, x4+        eor x16, x17, x9+        eor x13, x7, x9+        ldp x3, x17, [sp, #16]+        adcs x7, x23, x11+        eor x23, x7, x9+        ldp x5, x22, [sp, #32]+        adcs x7, x24, x11+        adc x24, x6, x11+        ldr x6, [x2, #64]+        adds x20, x8, x21+        lsl x11, x20, #9+        eor x4, x7, x9+        orr x7, x11, x19+        eor x8, x24, x9+        adcs x11, x25, x10+        mul x26, x6, x26+        ldp x19, x24, [sp, #48]+        adcs x12, x12, x3+        adcs x16, x16, x17+        adcs x9, x13, x5+        ldr x25, [sp, #64]+        extr x20, x11, x20, #55+        adcs x13, x23, x22+        adcs x4, x4, x19+        extr x23, x12, x11, #55+        adcs x8, x8, x24+        adc x11, x25, xzr+        adds x21, x9, x21+        extr x9, x16, x12, #55+        lsr x12, x16, #55+        adcs x10, x13, x10+        mul x15, x6, x15+        adcs x13, x4, x3+        ldp x16, x4, [x2]+        ldr x3, [x1, #64]+        adcs x17, x8, x17+        adcs x5, x5, x7+        adcs x20, x22, x20+        adcs x8, x19, x23+        and x22, x16, #0xfffffffffffff+        ldp x19, x7, [x1, #16]+        adcs x9, x24, x9+        extr x24, x4, x16, #52+        adc x16, x12, x25+        mul x22, x3, x22+        and x25, x24, #0xfffffffffffff+        extr x14, x19, x14, #40+        and x12, x14, #0xfffffffffffff+        extr x23, x7, x19, #28+        ldp x19, x24, [x2, #16]+        mul x14, x3, x25+        and x23, x23, #0xfffffffffffff+        add x22, x26, x22+        lsl x11, x11, #48+        lsr x26, x22, #52+        lsl x25, x22, #12+        mul x22, x6, x12+        extr x12, x19, x4, #40+        add x4, x15, x14+        mul x15, x6, x23+        add x4, x4, x26+        extr x23, x24, x19, #28+        ldp x14, x19, [x1, #32]+        and x26, x12, #0xfffffffffffff+        extr x12, x4, x25, #12+        and x25, x23, #0xfffffffffffff+        adds x21, x21, x12+        mul x12, x3, x26+        extr x23, x14, x7, #16+        and x23, x23, #0xfffffffffffff+        mul x7, x3, x25+        ldp x25, x26, [x2, #32]+        add x12, x22, x12+        extr x22, x19, x14, #56+        mul x23, x6, x23+        lsr x14, x14, #4+        extr x24, x25, x24, #16+        add x7, x15, x7+        and x15, x24, #0xfffffffffffff+        and x22, x22, #0xfffffffffffff+        lsr x24, x4, #52+        mul x15, x3, x15+        and x14, x14, #0xfffffffffffff+        add x12, x12, x24+        lsl x24, x4, #12+        lsr x4, x12, #52+        extr x24, x12, x24, #24+        adcs x10, x10, x24+        lsl x24, x12, #12+        add x12, x7, x4+        mul x22, x6, x22+        add x4, x23, x15+        extr x7, x12, x24, #36+        adcs x13, x13, x7+        lsl x15, x12, #12+        add x7, x4, x11+        lsr x24, x12, #52+        ldp x23, x11, [x2, #48]+        add x4, x7, x24+        mul x12, x6, x14+        extr x7, x26, x25, #56+        extr x14, x4, x15, #48+        and x2, x7, #0xfffffffffffff+        extr x24, x11, x23, #32+        ldp x15, x7, [x1, #48]+        and x1, x24, #0xfffffffffffff+        lsr x24, x4, #52+        mul x2, x3, x2+        extr x26, x23, x26, #44+        lsr x23, x25, #4+        and x23, x23, #0xfffffffffffff+        and x25, x26, #0xfffffffffffff+        extr x26, x7, x15, #32+        extr x19, x15, x19, #44+        mul x23, x3, x23+        and x15, x26, #0xfffffffffffff+        lsl x26, x4, #12+        and x4, x19, #0xfffffffffffff+        lsr x11, x11, #20+        mul x19, x6, x4+        adcs x17, x17, x14+        add x14, x22, x2+        add x22, x12, x23+        lsr x7, x7, #20+        add x22, x22, x24+        extr x2, x22, x26, #60+        mul x24, x3, x25+        lsr x22, x22, #52+        add x14, x14, x22+        lsl x22, x2, #8+        extr x22, x14, x22, #8+        lsl x2, x14, #12+        mul x1, x3, x1+        adcs x12, x5, x22+        mul x5, x6, x15+        and x26, x10, x13+        and x4, x26, x17+        add x23, x19, x24+        lsr x14, x14, #52+        mul x22, x3, x11+        add x11, x23, x14+        extr x25, x11, x2, #20+        lsl x19, x11, #12+        adcs x25, x20, x25+        and x14, x4, x12+        add x1, x5, x1+        and x14, x14, x25+        mul x15, x6, x7+        add x26, x15, x22+        mul x6, x6, x3+        lsr x22, x11, #52+        add x4, x1, x22+        lsr x1, x4, #52+        extr x3, x4, x19, #32+        lsl x15, x4, #12+        add x7, x26, x1+        adcs x23, x8, x3+        extr x20, x7, x15, #44+        and x3, x14, x23+        lsr x19, x7, #44+        adcs x7, x9, x20+        add x11, x6, x19+        adc x4, x16, x11+        lsr x14, x4, #9+        cmp xzr, xzr+        and x15, x3, x7+        orr x3, x4, #0xfffffffffffffe00+        adcs xzr, x21, x14+        adcs xzr, x15, xzr+        adcs xzr, x3, xzr+        adcs x11, x21, x14+        and x14, x11, #0x1ff+        adcs x1, x10, xzr+        extr x10, x1, x11, #9+        str x14, [x0, #64]+        adcs x14, x13, xzr+        extr x11, x14, x1, #9+        adcs x1, x17, xzr+        extr x4, x1, x14, #9+        stp x10, x11, [x0]+        adcs x11, x12, xzr+        extr x14, x11, x1, #9+        adcs x10, x25, xzr+        extr x11, x10, x11, #9+        stp x4, x14, [x0, #16]+        adcs x14, x23, xzr+        extr x10, x14, x10, #9+        adcs x1, x7, xzr+        stp x11, x10, [x0, #32]+        extr x14, x1, x14, #9+        adc x10, x3, xzr+        extr x26, x10, x1, #9+        stp x14, x26, [x0, #48]++// Restore regs and return++        CFI_INC_SP(80)+        CFI_POP2(x25,x26)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_mul_p521)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/bignum_mul_p521_alt.S view
@@ -0,0 +1,538 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Multiply modulo p_521, z := (x * y) mod p_521, assuming x and y reduced+// Inputs x[9], y[9]; output z[9]+//+//    extern void bignum_mul_p521_alt(uint64_t z[static 9],+//                                    const uint64_t x[static 9],+//                                    const uint64_t y[static 9]);+//+// Standard ARM ABI: X0 = z, X1 = x, X2 = y+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_mul_p521_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_mul_p521_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_mul_p521_alt)+        .text+        .balign 4++#define z x0+#define x x1+#define y x2++// These are repeated mod 2 as we load paris of inputs++#define a0 x3+#define a1 x4+#define a2 x3+#define a3 x4+#define a4 x3+#define a5 x4+#define a6 x3+#define a7 x4+#define a8 x3++#define b0 x5+#define b1 x6+#define b2 x7+#define b3 x8+#define b4 x9+#define b5 x10+#define b6 x11+#define b7 x12+#define b8 x13++#define t x14++// These repeat mod 11 as we stash some intermediate results in the+// output buffer.++#define u0 x15+#define u1 x16+#define u2 x17+#define u3 x19+#define u4 x20+#define u5 x21+#define u6 x22+#define u7 x23+#define u8 x24+#define u9 x25+#define u10 x26+#define u11 x15+#define u12 x16+#define u13 x17+#define u14 x19+#define u15 x20+#define u16 x21++S2N_BN_SYMBOL(bignum_mul_p521_alt):+        CFI_START++// Save more registers and make temporary space on stack++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x26)+        CFI_DEC_SP(64)++// Load operands and set up row 0 = [u9;...;u0] = a0 * [b8;...;b0]++        ldp     a0, a1, [x]+        ldp     b0, b1, [y]++        mul     u0, a0, b0+        umulh   u1, a0, b0+        mul     t, a0, b1+        umulh   u2, a0, b1+        adds    u1, u1, t++        ldp     b2, b3, [y, #16]++        mul     t, a0, b2+        umulh   u3, a0, b2+        adcs    u2, u2, t++        mul     t, a0, b3+        umulh   u4, a0, b3+        adcs    u3, u3, t++        ldp     b4, b5, [y, #32]++        mul     t, a0, b4+        umulh   u5, a0, b4+        adcs    u4, u4, t++        mul     t, a0, b5+        umulh   u6, a0, b5+        adcs    u5, u5, t++        ldp     b6, b7, [y, #48]++        mul     t, a0, b6+        umulh   u7, a0, b6+        adcs    u6, u6, t++        ldr     b8, [y, #64]++        mul     t, a0, b7+        umulh   u8, a0, b7+        adcs    u7, u7, t++        mul     t, a0, b8+        umulh   u9, a0, b8+        adcs    u8, u8, t++        adc     u9, u9, xzr++// Row 1 = [u10;...;u0] = [a1;a0] * [b8;...;b0]++        mul     t, a1, b0+        adds    u1, u1, t+        mul     t, a1, b1+        adcs    u2, u2, t+        mul     t, a1, b2+        adcs    u3, u3, t+        mul     t, a1, b3+        adcs    u4, u4, t+        mul     t, a1, b4+        adcs    u5, u5, t+        mul     t, a1, b5+        adcs    u6, u6, t+        mul     t, a1, b6+        adcs    u7, u7, t+        mul     t, a1, b7+        adcs    u8, u8, t+        mul     t, a1, b8+        adcs    u9, u9, t+        cset    u10, cs++        umulh   t, a1, b0+        adds    u2, u2, t+        umulh   t, a1, b1+        adcs    u3, u3, t+        umulh   t, a1, b2+        adcs    u4, u4, t+        umulh   t, a1, b3+        adcs    u5, u5, t+        umulh   t, a1, b4+        adcs    u6, u6, t+        umulh   t, a1, b5+        adcs    u7, u7, t+        umulh   t, a1, b6+        adcs    u8, u8, t+        umulh   t, a1, b7+        adcs    u9, u9, t+        umulh   t, a1, b8+        adc     u10, u10, t++        stp     u0, u1, [sp]++// Row 2 = [u11;...;u0] = [a2;a1;a0] * [b8;...;b0]++        ldp     a2, a3, [x, #16]++        mul     t, a2, b0+        adds    u2, u2, t+        mul     t, a2, b1+        adcs    u3, u3, t+        mul     t, a2, b2+        adcs    u4, u4, t+        mul     t, a2, b3+        adcs    u5, u5, t+        mul     t, a2, b4+        adcs    u6, u6, t+        mul     t, a2, b5+        adcs    u7, u7, t+        mul     t, a2, b6+        adcs    u8, u8, t+        mul     t, a2, b7+        adcs    u9, u9, t+        mul     t, a2, b8+        adcs    u10, u10, t+        cset    u11, cs++        umulh   t, a2, b0+        adds    u3, u3, t+        umulh   t, a2, b1+        adcs    u4, u4, t+        umulh   t, a2, b2+        adcs    u5, u5, t+        umulh   t, a2, b3+        adcs    u6, u6, t+        umulh   t, a2, b4+        adcs    u7, u7, t+        umulh   t, a2, b5+        adcs    u8, u8, t+        umulh   t, a2, b6+        adcs    u9, u9, t+        umulh   t, a2, b7+        adcs    u10, u10, t+        umulh   t, a2, b8+        adc     u11, u11, t++// Row 3 = [u12;...;u0] = [a3;a2;a1;a0] * [b8;...;b0]++        mul     t, a3, b0+        adds    u3, u3, t+        mul     t, a3, b1+        adcs    u4, u4, t+        mul     t, a3, b2+        adcs    u5, u5, t+        mul     t, a3, b3+        adcs    u6, u6, t+        mul     t, a3, b4+        adcs    u7, u7, t+        mul     t, a3, b5+        adcs    u8, u8, t+        mul     t, a3, b6+        adcs    u9, u9, t+        mul     t, a3, b7+        adcs    u10, u10, t+        mul     t, a3, b8+        adcs    u11, u11, t+        cset    u12, cs++        umulh   t, a3, b0+        adds    u4, u4, t+        umulh   t, a3, b1+        adcs    u5, u5, t+        umulh   t, a3, b2+        adcs    u6, u6, t+        umulh   t, a3, b3+        adcs    u7, u7, t+        umulh   t, a3, b4+        adcs    u8, u8, t+        umulh   t, a3, b5+        adcs    u9, u9, t+        umulh   t, a3, b6+        adcs    u10, u10, t+        umulh   t, a3, b7+        adcs    u11, u11, t+        umulh   t, a3, b8+        adc     u12, u12, t++        stp     u2, u3, [sp, #16]++// Row 4 = [u13;...;u0] = [a4;a3;a2;a1;a0] * [b8;...;b0]++        ldp     a4, a5, [x, #32]++        mul     t, a4, b0+        adds    u4, u4, t+        mul     t, a4, b1+        adcs    u5, u5, t+        mul     t, a4, b2+        adcs    u6, u6, t+        mul     t, a4, b3+        adcs    u7, u7, t+        mul     t, a4, b4+        adcs    u8, u8, t+        mul     t, a4, b5+        adcs    u9, u9, t+        mul     t, a4, b6+        adcs    u10, u10, t+        mul     t, a4, b7+        adcs    u11, u11, t+        mul     t, a4, b8+        adcs    u12, u12, t+        cset    u13, cs++        umulh   t, a4, b0+        adds    u5, u5, t+        umulh   t, a4, b1+        adcs    u6, u6, t+        umulh   t, a4, b2+        adcs    u7, u7, t+        umulh   t, a4, b3+        adcs    u8, u8, t+        umulh   t, a4, b4+        adcs    u9, u9, t+        umulh   t, a4, b5+        adcs    u10, u10, t+        umulh   t, a4, b6+        adcs    u11, u11, t+        umulh   t, a4, b7+        adcs    u12, u12, t+        umulh   t, a4, b8+        adc     u13, u13, t++// Row 5 = [u14;...;u0] = [a5;a4;a3;a2;a1;a0] * [b8;...;b0]++        mul     t, a5, b0+        adds    u5, u5, t+        mul     t, a5, b1+        adcs    u6, u6, t+        mul     t, a5, b2+        adcs    u7, u7, t+        mul     t, a5, b3+        adcs    u8, u8, t+        mul     t, a5, b4+        adcs    u9, u9, t+        mul     t, a5, b5+        adcs    u10, u10, t+        mul     t, a5, b6+        adcs    u11, u11, t+        mul     t, a5, b7+        adcs    u12, u12, t+        mul     t, a5, b8+        adcs    u13, u13, t+        cset    u14, cs++        umulh   t, a5, b0+        adds    u6, u6, t+        umulh   t, a5, b1+        adcs    u7, u7, t+        umulh   t, a5, b2+        adcs    u8, u8, t+        umulh   t, a5, b3+        adcs    u9, u9, t+        umulh   t, a5, b4+        adcs    u10, u10, t+        umulh   t, a5, b5+        adcs    u11, u11, t+        umulh   t, a5, b6+        adcs    u12, u12, t+        umulh   t, a5, b7+        adcs    u13, u13, t+        umulh   t, a5, b8+        adc     u14, u14, t++        stp     u4, u5, [sp, #32]++// Row 6 = [u15;...;u0] = [a6;a5;a4;a3;a2;a1;a0] * [b8;...;b0]++        ldp     a6, a7, [x, #48]++        mul     t, a6, b0+        adds    u6, u6, t+        mul     t, a6, b1+        adcs    u7, u7, t+        mul     t, a6, b2+        adcs    u8, u8, t+        mul     t, a6, b3+        adcs    u9, u9, t+        mul     t, a6, b4+        adcs    u10, u10, t+        mul     t, a6, b5+        adcs    u11, u11, t+        mul     t, a6, b6+        adcs    u12, u12, t+        mul     t, a6, b7+        adcs    u13, u13, t+        mul     t, a6, b8+        adcs    u14, u14, t+        cset    u15, cs++        umulh   t, a6, b0+        adds    u7, u7, t+        umulh   t, a6, b1+        adcs    u8, u8, t+        umulh   t, a6, b2+        adcs    u9, u9, t+        umulh   t, a6, b3+        adcs    u10, u10, t+        umulh   t, a6, b4+        adcs    u11, u11, t+        umulh   t, a6, b5+        adcs    u12, u12, t+        umulh   t, a6, b6+        adcs    u13, u13, t+        umulh   t, a6, b7+        adcs    u14, u14, t+        umulh   t, a6, b8+        adc     u15, u15, t++// Row 7 = [u16;...;u0] = [a7;a6;a5;a4;a3;a2;a1;a0] * [b8;...;b0]++        mul     t, a7, b0+        adds    u7, u7, t+        mul     t, a7, b1+        adcs    u8, u8, t+        mul     t, a7, b2+        adcs    u9, u9, t+        mul     t, a7, b3+        adcs    u10, u10, t+        mul     t, a7, b4+        adcs    u11, u11, t+        mul     t, a7, b5+        adcs    u12, u12, t+        mul     t, a7, b6+        adcs    u13, u13, t+        mul     t, a7, b7+        adcs    u14, u14, t+        mul     t, a7, b8+        adcs    u15, u15, t+        cset    u16, cs++        umulh   t, a7, b0+        adds    u8, u8, t+        umulh   t, a7, b1+        adcs    u9, u9, t+        umulh   t, a7, b2+        adcs    u10, u10, t+        umulh   t, a7, b3+        adcs    u11, u11, t+        umulh   t, a7, b4+        adcs    u12, u12, t+        umulh   t, a7, b5+        adcs    u13, u13, t+        umulh   t, a7, b6+        adcs    u14, u14, t+        umulh   t, a7, b7+        adcs    u15, u15, t+        umulh   t, a7, b8+        adc     u16, u16, t++        stp     u6, u7, [sp, #48]++// Row 8 = [u16;...;u0] = [a8;a7;a6;a5;a4;a3;a2;a1;a0] * [b8;...;b0]++        ldr     a8, [x, #64]++        mul     t, a8, b0+        adds    u8, u8, t+        mul     t, a8, b1+        adcs    u9, u9, t+        mul     t, a8, b2+        adcs    u10, u10, t+        mul     t, a8, b3+        adcs    u11, u11, t+        mul     t, a8, b4+        adcs    u12, u12, t+        mul     t, a8, b5+        adcs    u13, u13, t+        mul     t, a8, b6+        adcs    u14, u14, t+        mul     t, a8, b7+        adcs    u15, u15, t+        mul     t, a8, b8+        adc     u16, u16, t++        umulh   t, a8, b0+        adds    u9, u9, t+        umulh   t, a8, b1+        adcs    u10, u10, t+        umulh   t, a8, b2+        adcs    u11, u11, t+        umulh   t, a8, b3+        adcs    u12, u12, t+        umulh   t, a8, b4+        adcs    u13, u13, t+        umulh   t, a8, b5+        adcs    u14, u14, t+        umulh   t, a8, b6+        adcs    u15, u15, t+        umulh   t, a8, b7+        adc     u16, u16, t++// Now we have the full product, which we consider as+// 2^521 * h + l. Form h + l + 1++        subs    xzr, xzr, xzr+        ldp     b0, b1, [sp]+        extr    t, u9, u8, #9+        adcs    b0, b0, t+        extr    t, u10, u9, #9+        adcs    b1, b1, t+        ldp     b2, b3, [sp, #16]+        extr    t, u11, u10, #9+        adcs    b2, b2, t+        extr    t, u12, u11, #9+        adcs    b3, b3, t+        ldp     b4, b5, [sp, #32]+        extr    t, u13, u12, #9+        adcs    b4, b4, t+        extr    t, u14, u13, #9+        adcs    b5, b5, t+        ldp     b6, b7, [sp, #48]+        extr    t, u15, u14, #9+        adcs    b6, b6, t+        extr    t, u16, u15, #9+        adcs    b7, b7, t+        orr     b8, u8, #~0x1FF+        lsr     t, u16, #9+        adcs    b8, b8, t++// Now CF is set if h + l + 1 >= 2^521, which means it's already+// the answer, while if ~CF the answer is h + l so we should subtract+// 1 (all considered in 521 bits). Hence subtract ~CF and mask.++        sbcs    b0, b0, xzr+        sbcs    b1, b1, xzr+        sbcs    b2, b2, xzr+        sbcs    b3, b3, xzr+        sbcs    b4, b4, xzr+        sbcs    b5, b5, xzr+        sbcs    b6, b6, xzr+        sbcs    b7, b7, xzr+        sbc     b8, b8, xzr+        and     b8, b8, #0x1FF++// Store back digits of final result++        stp     b0, b1, [z]+        stp     b2, b3, [z, #16]+        stp     b4, b5, [z, #32]+        stp     b6, b7, [z, #48]+        str     b8, [z, #64]++// Restore registers++        CFI_INC_SP(64)+        CFI_POP2(x25,x26)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_mul_p521_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/bignum_neg_p256.S view
@@ -0,0 +1,72 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Negate modulo p_256, z := (-x) mod p_256, assuming x reduced+// Input x[4]; output z[4]+//+//    extern void bignum_neg_p256(uint64_t z[static 4], const uint64_t x[static 4]);+//+// Standard ARM ABI: X0 = z, X1 = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_neg_p256)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_neg_p256)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_neg_p256)+        .text+        .balign 4++#define z x0+#define x x1++#define p x2+#define t x3++#define d0 x4+#define d1 x5+#define d2 x6+#define d3 x7+++S2N_BN_SYMBOL(bignum_neg_p256):+        CFI_START++// Load the 4 digits of x++        ldp     d0, d1, [x]+        ldp     d2, d3, [x, #16]++// Set a bitmask p for the input being nonzero, so that we avoid doing+// -0 = p_256 and hence maintain strict modular reduction++        orr     t, d0, d1+        orr     p, d2, d3+        orr     p, p, t+        cmp     p, #0+        csetm   p, ne++// Mask the nontrivial words of p_256 = [n3;0;n1;-1] and subtract++        subs    d0, p, d0+        and     t, p, #0x00000000ffffffff+        sbcs    d1, t, d1+        sbcs    d2, xzr, d2+        and     t, p, #0xffffffff00000001+        sbc     d3, t, d3++// Write back the result++        stp     d0, d1, [z]+        stp     d2, d3, [z, #16]++// Return++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_neg_p256)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/bignum_sqr_p521.S view
@@ -0,0 +1,1125 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Square modulo p_521, z := (x^2) mod p_521, assuming x reduced+// Input x[9]; output z[9]+//+//    extern void bignum_sqr_p521(uint64_t z[static 9], const uint64_t x[static 9]);+//+// Standard ARM ABI: X0 = z, X1 = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++// bignum_sqr_p521 is functionally equivalent to unopt/bignum_sqr_p521_base.+// It is written in a way that+// 1. A subset of scalar multiplications in bignum_montmul_p384 are carefully+//    chosen and vectorized+// 2. The vectorized assembly is rescheduled using the SLOTHY superoptimizer.+//    https://github.com/slothy-optimizer/slothy+//+// The output program of step 1. is as follows:+//+//        stp     x19, x20, [sp, #-16]!+//        stp     x21, x22, [sp, #-16]!+//        stp     x23, x24, [sp, #-16]!+//        ldp x20, x19, [x1]+//        ldr q23, [x1]+//        ldr q1, [x1]+//        ldr q16, [x1]+//        ldp x14, x12, [x1, #16]+//        ldr q28, [x1, #16]+//        ldr q31, [x1, #16]+//        ldp x9, x2, [x1, #32]+//        ldr q29, [x1, #32]+//        ldr q4, [x1, #32]+//        ldr q5, [x1]+//        ldr q2, [x1, #32]+//        ldp x6, x13, [x1, #48]+//        ldr q24, [x1, #48]+//        ldr q27, [x1, #48]+//        ldr q0, [x1, #16]+//        ldr q30, [x1, #48]+//        mul x17, x9, x6+//        mul x10, x2, x13+//        umulh x24, x9, x6+//        subs x4, x9, x2+//        cneg x4, x4, cc+//        csetm x16, cc+//        subs x3, x13, x6+//        cneg x23, x3, cc+//        mul x3, x4, x23+//        umulh x4, x4, x23+//        cinv x22, x16, cc+//        eor x23, x3, x22+//        eor x16, x4, x22+//        adds x3, x17, x24+//        adc x24, x24, xzr+//        umulh x4, x2, x13+//        adds x3, x3, x10+//        adcs x24, x24, x4+//        adc x4, x4, xzr+//        adds x24, x24, x10+//        adc x10, x4, xzr+//        cmn x22, #0x1+//        adcs x4, x3, x23+//        adcs x24, x24, x16+//        adc x10, x10, x22+//        adds x8, x17, x17+//        adcs x22, x4, x4+//        adcs x5, x24, x24+//        adcs x11, x10, x10+//        adc x23, xzr, xzr+//        movi v25.2D, #0xffffffff+//        uzp2 v19.4S, v4.4S, v4.4S+//        xtn v26.2S, v29.2D+//        xtn v22.2S, v4.2D+//        rev64 v4.4S, v4.4S+//        umull v7.2D, v26.2S, v22.2S+//        umull v21.2D, v26.2S, v19.2S+//        uzp2 v17.4S, v29.4S, v29.4S+//        mul v4.4S, v4.4S, v29.4S+//        usra v21.2D, v7.2D, #32+//        umull v18.2D, v17.2S, v19.2S+//        uaddlp v4.2D, v4.4S+//        and v7.16B, v21.16B, v25.16B+//        umlal v7.2D, v17.2S, v22.2S+//        shl v4.2D, v4.2D, #32+//        usra v18.2D, v21.2D, #32+//        umlal v4.2D, v26.2S, v22.2S+//        usra v18.2D, v7.2D, #32+//        mov x15, v4.d[0]+//        mov x16, v4.d[1]+//        mul x3, x9, x2+//        mov x10, v18.d[0]+//        mov x17, v18.d[1]+//        umulh x4, x9, x2+//        adds x24, x10, x3+//        adcs x10, x16, x4+//        adc x17, x17, xzr+//        adds x7, x24, x3+//        adcs x10, x10, x4+//        adc x17, x17, xzr+//        adds x8, x8, x10+//        adcs x22, x22, x17+//        adcs x21, x5, xzr+//        adcs x5, x11, xzr+//        adc x11, x23, xzr+//        movi v25.2D, #0xffffffff+//        uzp2 v19.4S, v27.4S, v27.4S+//        xtn v26.2S, v24.2D+//        xtn v22.2S, v27.2D+//        rev64 v4.4S, v27.4S+//        umull v7.2D, v26.2S, v22.2S+//        umull v21.2D, v26.2S, v19.2S+//        uzp2 v17.4S, v24.4S, v24.4S+//        mul v4.4S, v4.4S, v24.4S+//        usra v21.2D, v7.2D, #32+//        umull v18.2D, v17.2S, v19.2S+//        uaddlp v4.2D, v4.4S+//        and v7.16B, v21.16B, v25.16B+//        umlal v7.2D, v17.2S, v22.2S+//        shl v4.2D, v4.2D, #32+//        usra v18.2D, v21.2D, #32+//        umlal v4.2D, v26.2S, v22.2S+//        usra v18.2D, v7.2D, #32+//        mov x23, v4.d[0]+//        mov x16, v4.d[1]+//        mul x3, x6, x13+//        mov x10, v18.d[0]+//        mov x17, v18.d[1]+//        umulh x4, x6, x13+//        adds x24, x10, x3+//        adcs x10, x16, x4+//        adc x17, x17, xzr+//        adds x24, x24, x3+//        adcs x10, x10, x4+//        adc x17, x17, xzr+//        adds x23, x23, x21+//        adcs x16, x24, x5+//        adcs x3, x10, x11+//        adc x21, x17, xzr+//        ldr x17, [x1, #64]+//        add x5, x17, x17+//        mul x11, x17, x17+//        and x17, x20, #0xfffffffffffff+//        mul x4, x5, x17+//        extr x17, x19, x20, #52+//        and x17, x17, #0xfffffffffffff+//        mul x10, x5, x17+//        lsr x17, x4, #52+//        add x24, x10, x17+//        lsl x17, x4, #12+//        extr x17, x24, x17, #12+//        adds x15, x15, x17+//        extr x17, x14, x19, #40+//        and x17, x17, #0xfffffffffffff+//        mul x10, x5, x17+//        lsr x17, x24, #52+//        add x4, x10, x17+//        lsl x17, x24, #12+//        extr x17, x4, x17, #24+//        adcs x7, x7, x17+//        extr x17, x12, x14, #28+//        and x17, x17, #0xfffffffffffff+//        mul x10, x5, x17+//        lsr x17, x4, #52+//        add x24, x10, x17+//        lsl x17, x4, #12+//        extr x17, x24, x17, #36+//        adcs x8, x8, x17+//        extr x17, x9, x12, #16+//        and x17, x17, #0xfffffffffffff+//        mul x10, x5, x17+//        lsr x17, x24, #52+//        add x4, x10, x17+//        lsl x17, x24, #12+//        extr x17, x4, x17, #48+//        adcs x22, x22, x17+//        lsr x17, x9, #4+//        and x17, x17, #0xfffffffffffff+//        mul x10, x5, x17+//        lsr x17, x4, #52+//        add x24, x10, x17+//        lsl x17, x4, #12+//        extr x4, x24, x17, #60+//        extr x17, x2, x9, #56+//        and x17, x17, #0xfffffffffffff+//        mul x10, x5, x17+//        lsr x17, x24, #52+//        add x24, x10, x17+//        lsl x17, x4, #8+//        extr x17, x24, x17, #8+//        adcs x23, x23, x17+//        extr x17, x6, x2, #44+//        and x17, x17, #0xfffffffffffff+//        mul x10, x5, x17+//        lsr x17, x24, #52+//        add x4, x10, x17+//        lsl x17, x24, #12+//        extr x17, x4, x17, #20+//        adcs x16, x16, x17+//        extr x17, x13, x6, #32+//        and x17, x17, #0xfffffffffffff+//        mul x10, x5, x17+//        lsr x17, x4, #52+//        add x24, x10, x17+//        lsl x17, x4, #12+//        extr x17, x24, x17, #32+//        adcs x3, x3, x17+//        lsr x17, x13, #20+//        mul x10, x5, x17+//        lsr x17, x24, #52+//        add x10, x10, x17+//        lsl x17, x24, #12+//        extr x17, x10, x17, #44+//        adcs x4, x21, x17+//        lsr x17, x10, #44+//        adc x24, x11, x17+//        extr x10, x7, x15, #9+//        extr x17, x8, x7, #9+//        stp x10, x17, [x0]                       // @slothy:writes=buffer0+//        extr x10, x22, x8, #9+//        extr x17, x23, x22, #9+//        stp x10, x17, [x0, #16]                  // @slothy:writes=buffer16+//        extr x10, x16, x23, #9+//        extr x17, x3, x16, #9+//        stp x10, x17, [x0, #32]                  // @slothy:writes=buffer32+//        extr x10, x4, x3, #9+//        extr x17, x24, x4, #9+//        stp x10, x17, [x0, #48]                  // @slothy:writes=buffer48+//        and x10, x15, #0x1ff+//        lsr x17, x24, #9+//        add x17, x10, x17+//        str x17, [x0, #64]                       // @slothy:writes=buffer64+//        uzp1 v17.4S, v28.4S, v23.4S+//        rev64 v4.4S, v28.4S+//        uzp1 v7.4S, v23.4S, v23.4S+//        mul v4.4S, v4.4S, v23.4S+//        uaddlp v4.2D, v4.4S+//        shl v4.2D, v4.2D, #32+//        umlal v4.2D, v7.2S, v17.2S+//        mov x8, v4.d[0]+//        mov x22, v4.d[1]+//        umulh x23, x20, x14+//        subs x17, x20, x19+//        cneg x4, x17, cc+//        csetm x24, cc+//        subs x17, x12, x14+//        cneg x17, x17, cc+//        mul x10, x4, x17+//        umulh x17, x4, x17+//        cinv x16, x24, cc+//        eor x3, x10, x16+//        eor x4, x17, x16+//        adds x24, x8, x23+//        adc x10, x23, xzr+//        umulh x17, x19, x12+//        adds x24, x24, x22+//        adcs x10, x10, x17+//        adc x17, x17, xzr+//        adds x10, x10, x22+//        adc x17, x17, xzr+//        cmn x16, #0x1+//        adcs x24, x24, x3+//        adcs x10, x10, x4+//        adc x17, x17, x16+//        adds x15, x8, x8+//        adcs x7, x24, x24+//        adcs x8, x10, x10+//        adcs x22, x17, x17+//        adc x23, xzr, xzr+//        movi v25.2D, #0xffffffff+//        uzp2 v19.4S, v16.4S, v16.4S+//        xtn v26.2S, v1.2D+//        xtn v22.2S, v16.2D+//        rev64 v4.4S, v16.4S+//        umull v7.2D, v26.2S, v22.2S+//        umull v21.2D, v26.2S, v19.2S+//        uzp2 v17.4S, v1.4S, v1.4S+//        mul v4.4S, v4.4S, v1.4S+//        usra v21.2D, v7.2D, #32+//        umull v18.2D, v17.2S, v19.2S+//        uaddlp v4.2D, v4.4S+//        and v7.16B, v21.16B, v25.16B+//        umlal v7.2D, v17.2S, v22.2S+//        shl v4.2D, v4.2D, #32+//        usra v18.2D, v21.2D, #32+//        umlal v4.2D, v26.2S, v22.2S+//        usra v18.2D, v7.2D, #32+//        mov x21, v4.d[0]+//        mov x16, v4.d[1]+//        mul x3, x20, x19+//        mov x10, v18.d[0]+//        mov x17, v18.d[1]+//        umulh x4, x20, x19+//        adds x24, x10, x3+//        adcs x10, x16, x4+//        adc x17, x17, xzr+//        adds x5, x24, x3+//        adcs x10, x10, x4+//        adc x17, x17, xzr+//        adds x11, x15, x10+//        adcs x15, x7, x17+//        adcs x7, x8, xzr+//        adcs x8, x22, xzr+//        adc x22, x23, xzr+//        xtn v7.2S, v31.2D+//        shrn v4.2S, v31.2D, #32+//        umull v4.2D, v7.2S, v4.2S+//        shl v4.2D, v4.2D, #33+//        umlal v4.2D, v7.2S, v7.2S+//        mov x23, v4.d[0]+//        mov x16, v4.d[1]+//        mul x3, x14, x12+//        umulh x10, x14, x14+//        umulh x17, x12, x12+//        umulh x4, x14, x12+//        adds x24, x10, x3+//        adcs x10, x16, x4+//        adc x17, x17, xzr+//        adds x24, x24, x3+//        adcs x10, x10, x4+//        adc x17, x17, xzr+//        adds x16, x23, x7+//        adcs x3, x24, x8+//        adcs x4, x10, x22+//        adc x24, x17, xzr+//        ldp x10, x17, [x0]                       // @slothy:reads=buffer0+//        adds x10, x10, x21+//        adcs x17, x17, x5+//        stp x10, x17, [x0]                       // @slothy:writes=buffer0+//        ldp x10, x17, [x0, #16]                  // @slothy:reads=buffer16+//        adcs x10, x10, x11+//        adcs x17, x17, x15+//        stp x10, x17, [x0, #16]                  // @slothy:writes=buffer16+//        ldp x10, x17, [x0, #32]                  // @slothy:reads=buffer32+//        adcs x10, x10, x16+//        adcs x17, x17, x3+//        stp x10, x17, [x0, #32]                  // @slothy:writes=buffer32+//        ldp x10, x17, [x0, #48]                  // @slothy:reads=buffer48+//        adcs x10, x10, x4+//        adcs x17, x17, x24+//        stp x10, x17, [x0, #48]                  // @slothy:writes=buffer48+//        ldr x17, [x0, #64]                       // @slothy:reads=buffer64+//        adc x17, x17, xzr+//        str x17, [x0, #64]                       // @slothy:writes=buffer64+//        movi v25.2D, #0xffffffff+//        uzp2 v19.4S, v2.4S, v2.4S+//        xtn v26.2S, v5.2D+//        xtn v22.2S, v2.2D+//        rev64 v4.4S, v2.4S+//        umull v7.2D, v26.2S, v22.2S+//        umull v21.2D, v26.2S, v19.2S+//        uzp2 v17.4S, v5.4S, v5.4S+//        mul v4.4S, v4.4S, v5.4S+//        usra v21.2D, v7.2D, #32+//        umull v18.2D, v17.2S, v19.2S+//        uaddlp v4.2D, v4.4S+//        and v7.16B, v21.16B, v25.16B+//        umlal v7.2D, v17.2S, v22.2S+//        shl v4.2D, v4.2D, #32+//        usra v18.2D, v21.2D, #32+//        umlal v4.2D, v26.2S, v22.2S+//        usra v18.2D, v7.2D, #32+//        mov x5, v4.d[0]+//        mov x4, v4.d[1]+//        movi v25.2D, #0xffffffff+//        uzp2 v17.4S, v30.4S, v30.4S+//        xtn v19.2S, v0.2D+//        xtn v26.2S, v30.2D+//        rev64 v4.4S, v30.4S+//        umull v7.2D, v19.2S, v26.2S+//        umull v22.2D, v19.2S, v17.2S+//        uzp2 v21.4S, v0.4S, v0.4S+//        mul v4.4S, v4.4S, v0.4S+//        usra v22.2D, v7.2D, #32+//        umull v17.2D, v21.2S, v17.2S+//        uaddlp v4.2D, v4.4S+//        and v7.16B, v22.16B, v25.16B+//        umlal v7.2D, v21.2S, v26.2S+//        shl v4.2D, v4.2D, #32+//        usra v17.2D, v22.2D, #32+//        umlal v4.2D, v19.2S, v26.2S+//        usra v17.2D, v7.2D, #32+//        mov x24, v4.d[0]+//        mov x10, v4.d[1]+//        mov x17, v18.d[0]+//        adds x4, x4, x17+//        mov x17, v18.d[1]+//        adcs x24, x24, x17+//        mov x17, v17.d[0]+//        adcs x10, x10, x17+//        mov x17, v17.d[1]+//        adc x17, x17, xzr+//        adds x15, x4, x5+//        adcs x4, x24, x4+//        adcs x24, x10, x24+//        adcs x10, x17, x10+//        adc x17, xzr, x17+//        adds x7, x4, x5+//        adcs x8, x24, x15+//        adcs x22, x10, x4+//        adcs x23, x17, x24+//        adcs x16, xzr, x10+//        adc x3, xzr, x17+//        subs x17, x14, x12+//        cneg x24, x17, cc+//        csetm x4, cc+//        subs x17, x13, x6+//        cneg x10, x17, cc+//        mul x17, x24, x10+//        umulh x24, x24, x10+//        cinv x10, x4, cc+//        cmn x10, #0x1+//        eor x17, x17, x10+//        adcs x23, x23, x17+//        eor x17, x24, x10+//        adcs x16, x16, x17+//        adc x3, x3, x10+//        subs x17, x20, x19+//        cneg x24, x17, cc+//        csetm x4, cc+//        subs x17, x2, x9+//        cneg x10, x17, cc+//        mul x17, x24, x10+//        umulh x24, x24, x10+//        cinv x10, x4, cc+//        cmn x10, #0x1+//        eor x17, x17, x10+//        adcs x11, x15, x17+//        eor x17, x24, x10+//        adcs x15, x7, x17+//        adcs x7, x8, x10+//        adcs x22, x22, x10+//        adcs x23, x23, x10+//        adcs x16, x16, x10+//        adc x3, x3, x10+//        subs x17, x19, x12+//        cneg x24, x17, cc+//        csetm x4, cc+//        subs x17, x13, x2+//        cneg x10, x17, cc+//        mul x17, x24, x10+//        umulh x24, x24, x10+//        cinv x10, x4, cc+//        cmn x10, #0x1+//        eor x17, x17, x10+//        adcs x8, x22, x17+//        eor x17, x24, x10+//        adcs x23, x23, x17+//        adcs x16, x16, x10+//        adc x3, x3, x10+//        subs x17, x20, x14+//        cneg x24, x17, cc+//        csetm x4, cc+//        subs x17, x6, x9+//        cneg x10, x17, cc+//        mul x17, x24, x10+//        umulh x24, x24, x10+//        cinv x10, x4, cc+//        cmn x10, #0x1+//        eor x17, x17, x10+//        adcs x22, x15, x17+//        eor x17, x24, x10+//        adcs x4, x7, x17+//        adcs x24, x8, x10+//        adcs x23, x23, x10+//        adcs x16, x16, x10+//        adc x3, x3, x10+//        subs x12, x20, x12+//        cneg x10, x12, cc+//        csetm x17, cc+//        subs x12, x13, x9+//        cneg x9, x12, cc+//        mul x12, x10, x9+//        umulh x13, x10, x9+//        cinv x9, x17, cc+//        cmn x9, #0x1+//        eor x12, x12, x9+//        adcs x4, x4, x12+//        eor x12, x13, x9+//        adcs x24, x24, x12+//        adcs x10, x23, x9+//        adcs x17, x16, x9+//        adc x13, x3, x9+//        subs x19, x19, x14+//        cneg x12, x19, cc+//        csetm x9, cc+//        subs x6, x6, x2+//        cneg x14, x6, cc+//        mul x19, x12, x14+//        umulh x12, x12, x14+//        cinv x14, x9, cc+//        cmn x14, #0x1+//        eor x19, x19, x14+//        adcs x23, x4, x19+//        eor x19, x12, x14+//        adcs x16, x24, x19+//        adcs x6, x10, x14+//        adcs x2, x17, x14+//        adc x9, x13, x14+//        ldp x12, x14, [x0]                       // @slothy:reads=buffer0+//        extr x19, x6, x16, #8+//        adds x10, x19, x12+//        extr x19, x2, x6, #8+//        adcs x17, x19, x14+//        ldp x14, x12, [x0, #16]                  // @slothy:reads=buffer16+//        extr x19, x9, x2, #8+//        adcs x13, x19, x14+//        and x14, x17, x13+//        lsr x19, x9, #8+//        adcs x6, x19, x12+//        and x9, x14, x6+//        ldp x14, x12, [x0, #32]                  // @slothy:reads=buffer32+//        lsl x19, x5, #1+//        adcs x2, x19, x14+//        and x14, x9, x2+//        extr x19, x11, x5, #63+//        adcs x3, x19, x12+//        and x9, x14, x3+//        ldp x14, x12, [x0, #48]                  // @slothy:reads=buffer48+//        extr x19, x22, x11, #63+//        adcs x4, x19, x14+//        and x14, x9, x4+//        extr x19, x23, x22, #63+//        adcs x24, x19, x12+//        and x12, x14, x24+//        ldr x14, [x0, #64]                       // @slothy:reads=buffer64+//        extr x19, x16, x23, #63+//        and x19, x19, #0x1ff+//        adc x19, x14, x19+//        lsr x14, x19, #9+//        orr x19, x19, #0xfffffffffffffe00+//        cmp xzr, xzr+//        adcs xzr, x10, x14+//        adcs xzr, x12, xzr+//        adcs xzr, x19, xzr+//        adcs x10, x10, x14+//        adcs x17, x17, xzr+//        adcs x13, x13, xzr+//        adcs x6, x6, xzr+//        adcs x2, x2, xzr+//        adcs x9, x3, xzr+//        adcs x12, x4, xzr+//        adcs x14, x24, xzr+//        adc x19, x19, xzr+//        and x19, x19, #0x1ff+//        stp x10, x17, [x0]                       // @slothy:writes=buffer0+//        stp x13, x6, [x0, #16]                   // @slothy:writes=buffer16+//        stp x2, x9, [x0, #32]                    // @slothy:writes=buffer32+//        stp x12, x14, [x0, #48]                  // @slothy:writes=buffer48+//        str x19, [x0, #64]                       // @slothy:writes=buffer64+//        ldp     x23, x24, [sp], #16+//        ldp     x21, x22, [sp], #16+//        ldp     x19, x20, [sp], #16+//        ret+//+// The bash script used for step 2 is as follows:+//+//        # Store the assembly instructions except the last 'ret',+//        # callee-register store/loads as, say, 'input.S'.+//        export OUTPUTS="[hint_buffer0,hint_buffer16,hint_buffer32,hint_buffer48,hint_buffer64]"+//        export RESERVED_REGS="[x18,x25,x26,x27,x28,x29,x30,sp,q8,q9,q10,q11,q12,q13,q14,q15,v8,v9,v10,v11,v12,v13,v14,v15]"+//        <s2n-bignum>/tools/external/slothy.sh input.S my_out_dir+//        # my_out_dir/3.opt.s is the optimized assembly. Its output may differ+//        # from this file since the sequence is non-deterministically chosen.+//        # Please add 'ret' at the end of the output assembly.++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_sqr_p521)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_sqr_p521)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_sqr_p521)+        .text+        .balign 4++S2N_BN_SYMBOL(bignum_sqr_p521):+        CFI_START++// Save registers++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)++        ldr q23, [x1, #32]+        ldp x9, x2, [x1, #32]+        ldr q16, [x1, #32]+        ldr q20, [x1, #48]+        ldp x6, x13, [x1, #48]+        rev64 v2.4S, v23.4S+        mul x14, x9, x2+        ldr q31, [x1, #48]+        subs x22, x9, x2+        uzp2 v26.4S, v23.4S, v23.4S+        mul v30.4S, v2.4S, v16.4S+        xtn v0.2S, v20.2D+        csetm x12, cc+        xtn v21.2S, v16.2D+        xtn v23.2S, v23.2D+        umulh x10, x9, x6+        rev64 v27.4S, v31.4S+        umull v2.2D, v21.2S, v26.2S+        cneg x23, x22, cc+        uaddlp v25.2D, v30.4S+        umull v18.2D, v21.2S, v23.2S+        mul x22, x9, x6+        mul v6.4S, v27.4S, v20.4S+        uzp2 v17.4S, v20.4S, v20.4S+        shl v20.2D, v25.2D, #32+        uzp2 v27.4S, v31.4S, v31.4S+        mul x16, x2, x13+        umlal v20.2D, v21.2S, v23.2S+        usra v2.2D, v18.2D, #32+        adds x8, x22, x10+        umull v25.2D, v17.2S, v27.2S+        xtn v31.2S, v31.2D+        movi v1.2D, #0xffffffff+        adc x3, x10, xzr+        umulh x21, x2, x13+        uzp2 v21.4S, v16.4S, v16.4S+        umull v18.2D, v0.2S, v27.2S+        subs x19, x13, x6+        and v7.16B, v2.16B, v1.16B+        umull v27.2D, v0.2S, v31.2S+        cneg x20, x19, cc+        movi v30.2D, #0xffffffff+        umull v16.2D, v21.2S, v26.2S+        umlal v7.2D, v21.2S, v23.2S+        mul x19, x23, x20+        cinv x7, x12, cc+        uaddlp v6.2D, v6.4S+        eor x12, x19, x7+        adds x11, x8, x16+        umulh x10, x23, x20+        ldr q1, [x1]+        usra v16.2D, v2.2D, #32+        adcs x19, x3, x21+        shl v2.2D, v6.2D, #32+        adc x20, x21, xzr+        adds x17, x19, x16+        usra v18.2D, v27.2D, #32+        adc x19, x20, xzr+        cmn x7, #0x1+        umlal v2.2D, v0.2S, v31.2S+        umulh x16, x9, x2+        adcs x8, x11, x12+        usra v16.2D, v7.2D, #32+        ldr x12, [x1, #64]+        eor x20, x10, x7+        umulh x10, x6, x13+        mov x23, v2.d[0]+        mov x3, v2.d[1]+        adcs x21, x17, x20+        usra v25.2D, v18.2D, #32+        and v23.16B, v18.16B, v30.16B+        adc x7, x19, x7+        adds x22, x22, x22+        ldr q7, [x1, #16]+        adcs x17, x8, x8+        umlal v23.2D, v17.2S, v31.2S+        mov x19, v16.d[0]+        mul x11, x12, x12+        ldr q4, [x1]+        usra v25.2D, v23.2D, #32+        add x5, x12, x12+        adcs x15, x21, x21+        ldr q28, [x1]+        mov x12, v20.d[1]+        adcs x24, x7, x7+        mov x21, v16.d[1]+        adc x4, xzr, xzr+        adds x19, x19, x14+        ldr q18, [x1, #16]+        xtn v26.2S, v1.2D+        adcs x8, x12, x16+        adc x21, x21, xzr+        adds x7, x19, x14+        xtn v23.2S, v7.2D+        rev64 v21.4S, v28.4S+        adcs x12, x8, x16+        ldp x20, x19, [x1]+        mov x16, v25.d[1]+        xtn v22.2S, v28.2D+        adc x14, x21, xzr+        adds x8, x22, x12+        uzp2 v24.4S, v28.4S, v28.4S+        rev64 v28.4S, v18.4S+        mul x12, x6, x13+        mul v16.4S, v21.4S, v1.4S+        shrn v31.2S, v7.2D, #32+        adcs x22, x17, x14+        mov x14, v25.d[0]+        and x21, x20, #0xfffffffffffff+        umull v17.2D, v26.2S, v24.2S+        ldr q2, [x1, #32]+        adcs x17, x15, xzr+        ldr q30, [x1, #48]+        umull v7.2D, v26.2S, v22.2S+        adcs x15, x24, xzr+        ldr q0, [x1, #16]+        movi v6.2D, #0xffffffff+        adc x4, x4, xzr+        adds x14, x14, x12+        uzp1 v27.4S, v18.4S, v4.4S+        uzp2 v19.4S, v1.4S, v1.4S+        adcs x24, x3, x10+        mul x3, x5, x21+        umull v29.2D, v23.2S, v31.2S+        ldr q5, [x1]+        adc x21, x16, xzr+        adds x16, x14, x12+        extr x12, x19, x20, #52+        umull v18.2D, v19.2S, v24.2S+        adcs x24, x24, x10+        and x10, x12, #0xfffffffffffff+        ldp x14, x12, [x1, #16]+        usra v17.2D, v7.2D, #32+        adc x21, x21, xzr+        adds x23, x23, x17+        mul x17, x5, x10+        shl v21.2D, v29.2D, #33+        lsl x10, x3, #12+        lsr x1, x3, #52+        rev64 v29.4S, v2.4S+        uaddlp v25.2D, v16.4S+        add x17, x17, x1+        adcs x16, x16, x15+        extr x3, x14, x19, #40+        mov x15, v20.d[0]+        extr x10, x17, x10, #12+        and x3, x3, #0xfffffffffffff+        shl v3.2D, v25.2D, #32+        and v6.16B, v17.16B, v6.16B+        mul x1, x5, x3+        usra v18.2D, v17.2D, #32+        adcs x3, x24, x4+        extr x4, x12, x14, #28+        umlal v6.2D, v19.2S, v22.2S+        xtn v20.2S, v2.2D+        umlal v3.2D, v26.2S, v22.2S+        movi v26.2D, #0xffffffff+        lsr x24, x17, #52+        and x4, x4, #0xfffffffffffff+        uzp2 v19.4S, v2.4S, v2.4S+        add x1, x1, x24+        mul x24, x5, x4+        lsl x4, x17, #12+        xtn v24.2S, v5.2D+        extr x17, x1, x4, #24+        adc x21, x21, xzr+        umlal v21.2D, v23.2S, v23.2S+        adds x4, x15, x10+        lsl x10, x1, #12+        adcs x15, x7, x17+        mul v23.4S, v28.4S, v4.4S+        and x7, x4, #0x1ff+        lsr x17, x1, #52+        umulh x1, x19, x12+        uzp2 v17.4S, v5.4S, v5.4S+        extr x4, x15, x4, #9+        add x24, x24, x17+        mul v29.4S, v29.4S, v5.4S+        extr x17, x24, x10, #36+        extr x10, x9, x12, #16+        uzp1 v28.4S, v4.4S, v4.4S+        adcs x17, x8, x17+        and x8, x10, #0xfffffffffffff+        umull v16.2D, v24.2S, v20.2S+        extr x10, x17, x15, #9+        mul x15, x5, x8+        stp x4, x10, [x0]+        lsl x4, x24, #12+        lsr x8, x9, #4+        uaddlp v4.2D, v23.4S+        and x8, x8, #0xfffffffffffff+        umull v23.2D, v24.2S, v19.2S+        mul x8, x5, x8+        extr x10, x2, x9, #56+        lsr x24, x24, #52+        and x10, x10, #0xfffffffffffff+        add x15, x15, x24+        extr x4, x15, x4, #48+        mul x24, x5, x10+        lsr x10, x15, #52+        usra v23.2D, v16.2D, #32+        add x10, x8, x10+        shl v4.2D, v4.2D, #32+        adcs x22, x22, x4+        extr x4, x6, x2, #44+        lsl x15, x15, #12+        lsr x8, x10, #52+        extr x15, x10, x15, #60+        and x10, x4, #0xfffffffffffff+        umlal v4.2D, v28.2S, v27.2S+        add x8, x24, x8+        extr x4, x13, x6, #32+        mul x24, x5, x10+        uzp2 v16.4S, v30.4S, v30.4S+        lsl x10, x15, #8+        rev64 v28.4S, v30.4S+        and x15, x4, #0xfffffffffffff+        extr x4, x8, x10, #8+        mul x10, x5, x15+        lsl x15, x8, #12+        adcs x23, x23, x4+        lsr x4, x8, #52+        lsr x8, x13, #20+        add x4, x24, x4+        mul x8, x5, x8+        lsr x24, x4, #52+        extr x15, x4, x15, #20+        lsl x4, x4, #12+        add x10, x10, x24+        adcs x15, x16, x15+        extr x4, x10, x4, #32+        umulh x5, x20, x14+        adcs x3, x3, x4+        usra v18.2D, v6.2D, #32+        lsl x16, x10, #12+        extr x24, x15, x23, #9+        lsr x10, x10, #52+        uzp2 v27.4S, v0.4S, v0.4S+        add x8, x8, x10+        extr x10, x3, x15, #9+        extr x4, x22, x17, #9+        and v25.16B, v23.16B, v26.16B+        lsr x17, x8, #44+        extr x15, x8, x16, #44+        extr x16, x23, x22, #9+        xtn v7.2S, v30.2D+        mov x8, v4.d[0]+        stp x24, x10, [x0, #32]+        uaddlp v30.2D, v29.4S+        stp x4, x16, [x0, #16]+        umulh x24, x20, x19+        adcs x15, x21, x15+        adc x16, x11, x17+        subs x11, x20, x19+        xtn v5.2S, v0.2D+        csetm x17, cc+        extr x3, x15, x3, #9+        mov x22, v4.d[1]+        cneg x21, x11, cc+        subs x10, x12, x14+        mul v31.4S, v28.4S, v0.4S+        cneg x10, x10, cc+        cinv x11, x17, cc+        shl v4.2D, v30.2D, #32+        umull v28.2D, v5.2S, v16.2S+        extr x23, x16, x15, #9+        adds x4, x8, x5+        mul x17, x21, x10+        umull v22.2D, v5.2S, v7.2S+        adc x15, x5, xzr+        adds x4, x4, x22+        uaddlp v2.2D, v31.4S+        lsr x5, x16, #9+        adcs x16, x15, x1+        mov x15, v18.d[0]+        adc x1, x1, xzr+        umulh x10, x21, x10+        adds x22, x16, x22+        umlal v4.2D, v24.2S, v20.2S+        umull v30.2D, v27.2S, v16.2S+        stp x3, x23, [x0, #48]+        add x3, x7, x5+        adc x16, x1, xzr+        usra v28.2D, v22.2D, #32+        mul x23, x20, x19+        eor x1, x17, x11+        cmn x11, #0x1+        mov x17, v18.d[1]+        umull v18.2D, v17.2S, v19.2S+        adcs x7, x4, x1+        eor x1, x10, x11+        umlal v25.2D, v17.2S, v20.2S+        movi v16.2D, #0xffffffff+        adcs x22, x22, x1+        usra v18.2D, v23.2D, #32+        umulh x4, x14, x14+        adc x1, x16, x11+        adds x10, x8, x8+        shl v23.2D, v2.2D, #32+        str x3, [x0, #64]+        adcs x5, x7, x7+        and v16.16B, v28.16B, v16.16B+        usra v30.2D, v28.2D, #32+        adcs x7, x22, x22+        mov x21, v3.d[1]+        adcs x11, x1, x1+        umlal v16.2D, v27.2S, v7.2S+        adc x22, xzr, xzr+        adds x16, x15, x23+        mul x8, x14, x12+        umlal v23.2D, v5.2S, v7.2S+        usra v18.2D, v25.2D, #32+        umulh x15, x14, x12+        adcs x21, x21, x24+        usra v30.2D, v16.2D, #32+        adc x1, x17, xzr+        adds x3, x16, x23+        adcs x21, x21, x24+        adc x1, x1, xzr+        adds x24, x10, x21+        umulh x21, x12, x12+        adcs x16, x5, x1+        adcs x10, x7, xzr+        mov x17, v21.d[1]+        adcs x23, x11, xzr+        adc x5, x22, xzr+        adds x1, x4, x8+        adcs x22, x17, x15+        ldp x17, x4, [x0]+        mov x11, v21.d[0]+        adc x21, x21, xzr+        adds x1, x1, x8+        adcs x15, x22, x15+        adc x8, x21, xzr+        adds x22, x11, x10+        mov x21, v3.d[0]+        adcs x11, x1, x23+        ldp x1, x10, [x0, #16]+        adcs x15, x15, x5+        adc x7, x8, xzr+        adds x8, x17, x21+        mov x23, v4.d[1]+        ldp x5, x21, [x0, #32]+        adcs x17, x4, x3+        ldr x4, [x0, #64]+        mov x3, v18.d[0]+        adcs x24, x1, x24+        stp x8, x17, [x0]+        adcs x17, x10, x16+        ldp x1, x16, [x0, #48]+        adcs x5, x5, x22+        adcs x8, x21, x11+        stp x5, x8, [x0, #32]+        adcs x1, x1, x15+        mov x15, v23.d[1]+        adcs x21, x16, x7+        stp x1, x21, [x0, #48]+        adc x10, x4, xzr+        subs x7, x14, x12+        mov x16, v18.d[1]+        cneg x5, x7, cc+        csetm x4, cc+        subs x11, x13, x6+        mov x8, v23.d[0]+        cneg x7, x11, cc+        cinv x21, x4, cc+        mov x11, v30.d[0]+        adds x4, x23, x3+        mul x22, x5, x7+        mov x23, v30.d[1]+        adcs x8, x8, x16+        adcs x16, x15, x11+        adc x11, x23, xzr+        umulh x3, x5, x7+        stp x24, x17, [x0, #16]+        mov x5, v4.d[0]+        subs x15, x20, x19+        cneg x7, x15, cc+        str x10, [x0, #64]+        csetm x1, cc+        subs x24, x2, x9+        cneg x17, x24, cc+        cinv x15, x1, cc+        adds x23, x4, x5+        umulh x1, x7, x17+        adcs x24, x8, x4+        adcs x10, x16, x8+        eor x8, x22, x21+        adcs x16, x11, x16+        mul x22, x7, x17+        eor x17, x1, x15+        adc x1, xzr, x11+        adds x11, x24, x5+        eor x7, x3, x21+        adcs x3, x10, x23+        adcs x24, x16, x24+        adcs x4, x1, x10+        eor x10, x22, x15+        adcs x16, xzr, x16+        adc x1, xzr, x1+        cmn x21, #0x1+        adcs x8, x4, x8+        adcs x22, x16, x7+        adc x7, x1, x21+        subs x21, x19, x12+        csetm x4, cc+        cneg x1, x21, cc+        subs x21, x13, x2+        cinv x16, x4, cc+        cneg x4, x21, cc+        cmn x15, #0x1+        adcs x21, x23, x10+        mul x23, x1, x4+        adcs x11, x11, x17+        adcs x3, x3, x15+        umulh x1, x1, x4+        adcs x24, x24, x15+        adcs x8, x8, x15+        adcs x22, x22, x15+        eor x17, x23, x16+        adc x15, x7, x15+        subs x7, x20, x14+        cneg x7, x7, cc+        csetm x4, cc+        subs x10, x20, x12+        cneg x23, x10, cc+        csetm x10, cc+        subs x12, x6, x9+        cinv x20, x4, cc+        cneg x12, x12, cc+        cmn x16, #0x1+        eor x1, x1, x16+        adcs x17, x24, x17+        mul x4, x7, x12+        adcs x8, x8, x1+        umulh x1, x7, x12+        adcs x24, x22, x16+        adc x7, x15, x16+        subs x12, x13, x9+        cneg x12, x12, cc+        cinv x13, x10, cc+        subs x19, x19, x14+        mul x9, x23, x12+        cneg x19, x19, cc+        csetm x10, cc+        eor x16, x1, x20+        subs x22, x6, x2+        umulh x12, x23, x12+        eor x1, x4, x20+        cinv x4, x10, cc+        cneg x22, x22, cc+        cmn x20, #0x1+        adcs x15, x11, x1+        eor x6, x12, x13+        adcs x10, x3, x16+        adcs x17, x17, x20+        eor x23, x9, x13+        adcs x2, x8, x20+        mul x11, x19, x22+        adcs x24, x24, x20+        adc x7, x7, x20+        cmn x13, #0x1+        adcs x3, x10, x23+        umulh x22, x19, x22+        adcs x17, x17, x6+        eor x12, x22, x4+        extr x22, x15, x21, #63+        adcs x8, x2, x13+        extr x21, x21, x5, #63+        ldp x16, x23, [x0]+        adcs x20, x24, x13+        eor x1, x11, x4+        adc x6, x7, x13+        cmn x4, #0x1+        ldp x2, x7, [x0, #16]+        adcs x1, x3, x1+        extr x19, x1, x15, #63+        adcs x14, x17, x12+        extr x1, x14, x1, #63+        lsl x17, x5, #1+        adcs x8, x8, x4+        extr x12, x8, x14, #8+        ldp x15, x11, [x0, #32]+        adcs x9, x20, x4+        adc x3, x6, x4+        adds x16, x12, x16+        extr x6, x9, x8, #8+        ldp x14, x12, [x0, #48]+        extr x8, x3, x9, #8+        adcs x20, x6, x23+        ldr x24, [x0, #64]+        lsr x6, x3, #8+        adcs x8, x8, x2+        and x2, x1, #0x1ff+        and x1, x20, x8+        adcs x4, x6, x7+        adcs x3, x17, x15+        and x1, x1, x4+        adcs x9, x21, x11+        and x1, x1, x3+        adcs x6, x22, x14+        and x1, x1, x9+        and x21, x1, x6+        adcs x14, x19, x12+        adc x1, x24, x2+        cmp xzr, xzr+        orr x12, x1, #0xfffffffffffffe00+        lsr x1, x1, #9+        adcs xzr, x16, x1+        and x21, x21, x14+        adcs xzr, x21, xzr+        adcs xzr, x12, xzr+        adcs x21, x16, x1+        adcs x1, x20, xzr+        adcs x19, x8, xzr+        stp x21, x1, [x0]+        adcs x1, x4, xzr+        adcs x21, x3, xzr+        stp x19, x1, [x0, #16]+        adcs x1, x9, xzr+        stp x21, x1, [x0, #32]+        adcs x21, x6, xzr+        adcs x1, x14, xzr+        stp x21, x1, [x0, #48]+        adc x1, x12, xzr+        and x1, x1, #0x1ff+        str x1, [x0, #64]++// Restore regs and return++        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_sqr_p521)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/bignum_sqr_p521_alt.S view
@@ -0,0 +1,374 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Square modulo p_521, z := (x^2) mod p_521, assuming x reduced+// Input x[9]; output z[9]+//+//    extern void bignum_sqr_p521_alt(uint64_t z[static 9],+//                                    const uint64_t x[static 9]);+//+// Standard ARM ABI: X0 = z, X1 = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_sqr_p521_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_sqr_p521_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_sqr_p521_alt)+        .text+        .balign 4++#define z x0+#define x x1++#define a0 x2+#define a1 x3+#define a2 x4+#define a3 x5+#define a4 x6+#define a5 x7+#define a6 x8+#define a7 x9+#define a8 x1 // Overwrites input argument at last load++#define l x10++#define u0 x2 // The same as a0+#define u1 x11+#define u2 x12+#define u3 x13+#define u4 x14+#define u5 x15+#define u6 x16+#define u7 x17+#define u8 x19+#define u9 x20+#define u10 x21+#define u11 x22+#define u12 x23+#define u13 x24+#define u14 x25+#define u15 x26+#define u16 x4 // The same as a2++S2N_BN_SYMBOL(bignum_sqr_p521_alt):+        CFI_START++// It's convenient to have more registers to play with++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x26)++// Load low 8 elements as [a7;a6;a5;a4;a3;a2;a1;a0], set up an initial+// window [u8;u7;u6;u5;u4;u3;u2;u1] =  10 + 20 + 30 + 40 + 50 + 60 + 70++        ldp     a0, a1, [x]++        mul     u1, a0, a1+        umulh   u2, a0, a1++        ldp     a2, a3, [x, #16]++        mul     l, a0, a2+        umulh   u3, a0, a2+        adds    u2, u2, l++        ldp     a4, a5, [x, #32]++        mul     l, a0, a3+        umulh   u4, a0, a3+        adcs    u3, u3, l++        ldp     a6, a7, [x, #48]++        mul     l, a0, a4+        umulh   u5, a0, a4+        adcs    u4, u4, l++        mul     l, a0, a5+        umulh   u6, a0, a5+        adcs    u5, u5, l++        mul     l, a0, a6+        umulh   u7, a0, a6+        adcs    u6, u6, l++        mul     l, a0, a7+        umulh   u8, a0, a7+        adcs    u7, u7, l++        adc     u8, u8, xzr++// Add in the next diagonal = 21 + 31 + 41 + 51 + 61 + 71 + 54++        mul     l, a1, a2+        adds    u3, u3, l+        mul     l, a1, a3+        adcs    u4, u4, l+        mul     l, a1, a4+        adcs    u5, u5, l+        mul     l, a1, a5+        adcs    u6, u6, l+        mul     l, a1, a6+        adcs    u7, u7, l+        mul     l, a1, a7+        adcs    u8, u8, l+        cset    u9, cs++        umulh   l, a1, a2+        adds    u4, u4, l+        umulh   l, a1, a3+        adcs    u5, u5, l+        umulh   l, a1, a4+        adcs    u6, u6, l+        umulh   l, a1, a5+        adcs    u7, u7, l+        umulh   l, a1, a6+        adcs    u8, u8, l+        umulh   l, a1, a7+        adc     u9, u9, l+        mul     l, a4, a5+        umulh   u10, a4, a5+        adds    u9, u9, l+        adc     u10, u10, xzr++// And the next one = 32 + 42 + 52 + 62 + 72 + 64 + 65++        mul     l, a2, a3+        adds    u5, u5, l+        mul     l, a2, a4+        adcs    u6, u6, l+        mul     l, a2, a5+        adcs    u7, u7, l+        mul     l, a2, a6+        adcs    u8, u8, l+        mul     l, a2, a7+        adcs    u9, u9, l+        mul     l, a4, a6+        adcs    u10, u10, l+        cset    u11, cs++        umulh   l, a2, a3+        adds    u6, u6, l+        umulh   l, a2, a4+        adcs    u7, u7, l+        umulh   l, a2, a5+        adcs    u8, u8, l+        umulh   l, a2, a6+        adcs    u9, u9, l+        umulh   l, a2, a7+        adcs    u10, u10, l+        umulh   l, a4, a6+        adc     u11, u11, l+        mul     l, a5, a6+        umulh   u12, a5, a6+        adds    u11, u11, l+        adc     u12, u12, xzr++// And the final one = 43 + 53 + 63 + 73 + 74 + 75 + 76++        mul     l, a3, a4+        adds    u7, u7, l+        mul     l, a3, a5+        adcs    u8, u8, l+        mul     l, a3, a6+        adcs    u9, u9, l+        mul     l, a3, a7+        adcs    u10, u10, l+        mul     l, a4, a7+        adcs    u11, u11, l+        mul     l, a5, a7+        adcs    u12, u12, l+        cset    u13, cs++        umulh   l, a3, a4+        adds    u8, u8, l+        umulh   l, a3, a5+        adcs    u9, u9, l+        umulh   l, a3, a6+        adcs    u10, u10, l+        umulh   l, a3, a7+        adcs    u11, u11, l+        umulh   l, a4, a7+        adcs    u12, u12, l+        umulh   l, a5, a7+        adc     u13, u13, l+        mul     l, a6, a7+        umulh   u14, a6, a7+        adds    u13, u13, l+        adc     u14, u14, xzr++// Double that, with u15 holding the top carry++        adds    u1, u1, u1+        adcs    u2, u2, u2+        adcs    u3, u3, u3+        adcs    u4, u4, u4+        adcs    u5, u5, u5+        adcs    u6, u6, u6+        adcs    u7, u7, u7+        adcs    u8, u8, u8+        adcs    u9, u9, u9+        adcs    u10, u10, u10+        adcs    u11, u11, u11+        adcs    u12, u12, u12+        adcs    u13, u13, u13+        adcs    u14, u14, u14+        cset    u15, cs++// Add the homogeneous terms 00 + 11 + 22 + 33 + 44 + 55 + 66 + 77++        umulh   l, a0, a0+        adds    u1, u1, l++        mul     l, a1, a1+        adcs    u2, u2, l+        umulh   l, a1, a1+        adcs    u3, u3, l++        mul     l, a2, a2+        adcs    u4, u4, l+        umulh   l, a2, a2+        adcs    u5, u5, l++        mul     l, a3, a3+        adcs    u6, u6, l+        umulh   l, a3, a3+        adcs    u7, u7, l++        mul     l, a4, a4+        adcs    u8, u8, l+        umulh   l, a4, a4+        adcs    u9, u9, l++        mul     l, a5, a5+        adcs    u10, u10, l+        umulh   l, a5, a5+        adcs    u11, u11, l++        mul     l, a6, a6+        adcs    u12, u12, l+        umulh   l, a6, a6+        adcs    u13, u13, l++        mul     l, a7, a7+        adcs    u14, u14, l+        umulh   l, a7, a7+        adc     u15, u15, l++// Now load in the top digit a8, and immediately double the register++        ldr     a8, [x, #64]+        add     a8, a8, a8++// Add (2 * a8) * [a7;...;a0] into the top of the buffer+// At the end of the first chain we form u16 = a8 ^ 2.+// This needs us to shift right the modified a8 again but it saves a+// register, and the overall performance impact seems slightly positive.++        mul     l, a8, a0+        adds    u8, u8, l+        umulh   l, a8, a0+        adcs    u9, u9, l+        mul     l, a8, a2+        adcs    u10, u10, l+        umulh   l, a8, a2+        adcs    u11, u11, l+        mul     l, a8, a4+        adcs    u12, u12, l+        umulh   l, a8, a4+        adcs    u13, u13, l+        mul     l, a8, a6+        adcs    u14, u14, l+        umulh   l, a8, a6+        adcs    u15, u15, l+        lsr     u16, a8, #1+        mul     u16, u16, u16+        adc     u16, u16, xzr++        mul     l, a8, a1+        adds    u9, u9, l+        umulh   l, a8, a1+        adcs    u10, u10, l+        mul     l, a8, a3+        adcs    u11, u11, l+        umulh   l, a8, a3+        adcs    u12, u12, l+        mul     l, a8, a5+        adcs    u13, u13, l+        umulh   l, a8, a5+        adcs    u14, u14, l+        mul     l, a8, a7+        adcs    u15, u15, l+        umulh   l, a8, a7+        adc     u16, u16, l++// Finally squeeze in the lowest mul. This didn't need to be involved+// in the addition chains and moreover lets us re-use u0 == a0++        mul     u0, a0, a0++// Now we have the full product, which we consider as+// 2^521 * h + l. Form h + l + 1++        subs    xzr, xzr, xzr+        extr    l, u9, u8, #9+        adcs    u0, u0, l+        extr    l, u10, u9, #9+        adcs    u1, u1, l+        extr    l, u11, u10, #9+        adcs    u2, u2, l+        extr    l, u12, u11, #9+        adcs    u3, u3, l+        extr    l, u13, u12, #9+        adcs    u4, u4, l+        extr    l, u14, u13, #9+        adcs    u5, u5, l+        extr    l, u15, u14, #9+        adcs    u6, u6, l+        extr    l, u16, u15, #9+        adcs    u7, u7, l+        orr     u8, u8, #~0x1FF+        lsr     l, u16, #9+        adcs    u8, u8, l++// Now CF is set if h + l + 1 >= 2^521, which means it's already+// the answer, while if ~CF the answer is h + l so we should subtract+// 1 (all considered in 521 bits). Hence subtract ~CF and mask.++        sbcs    u0, u0, xzr+        sbcs    u1, u1, xzr+        sbcs    u2, u2, xzr+        sbcs    u3, u3, xzr+        sbcs    u4, u4, xzr+        sbcs    u5, u5, xzr+        sbcs    u6, u6, xzr+        sbcs    u7, u7, xzr+        sbc     u8, u8, xzr+        and     u8, u8, #0x1FF++// Store back digits of final result++        stp     u0, u1, [z]+        stp     u2, u3, [z, #16]+        stp     u4, u5, [z, #32]+        stp     u6, u7, [z, #48]+        str     u8, [z, #64]++// Restore registers and return++        CFI_POP2(x25,x26)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_sqr_p521_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/bignum_tomont_p256.S view
@@ -0,0 +1,122 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Convert to Montgomery form z := (2^256 * x) mod p_256+// Input x[4]; output z[4]+//+//    extern void bignum_tomont_p256(uint64_t z[static 4],+//                                   const uint64_t x[static 4]);+//+// Standard ARM ABI: X0 = z, X1 = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_tomont_p256)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_tomont_p256)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_tomont_p256)+        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_tomont_p256_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_tomont_p256_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_tomont_p256_alt)+        .text+        .balign 4++// ----------------------------------------------------------------------------+// Core "x |-> (2^64 * x) mod p_256" macro, with x assumed to be < p_256.+// Input is in [d4;d3;d2;d1] and output in [d3;d2;d1;d0]+// using d4 as well as t1, t2, t3 as temporaries.+// ----------------------------------------------------------------------------++#define modstep_p256(d4, d3,d2,d1,d0, t1,t2,t3)                             \+/* Writing the input as z = 2^256 * h + 2^192 * l + t = 2^192 * hl + t,  */ \+/* our quotient approximation is MIN ((hl + hl>>32 + 1)>>64) (2^64 - 1). */ \+        subs    xzr, xzr, xzr __LF/* Set carry flag for +1 */          \+        extr    t3, d4, d3, #32 __LF                                   \+        adcs    xzr, d3, t3 __LF                                       \+        lsr     t3, d4, #32 __LF                                       \+        adcs    t3, d4, t3 __LF                                        \+        csetm   d0, cs __LF                                            \+        orr     t3, t3, d0 __LF                                        \+/* First do [t2;t1] = 2^32 * q, which we use twice                       */ \+        lsl     t1, t3, #32 __LF                                       \+        lsr     t2, t3, #32 __LF                                       \+/* Add 2^224 * q to sum                                                  */ \+        adds    d3, d3, t1 __LF                                        \+        adc     d4, d4, t2 __LF                                        \+/* Accumulate [t2;t1;d0] = (2^96 - 1) * q                                */ \+        subs    d0, xzr, t3 __LF                                       \+        sbcs    t1, t1, xzr __LF                                       \+        sbc     t2, t2, xzr __LF                                       \+/* Subtract (2^256 + 2^192 + 2^96 - 1) * q                               */ \+        subs    d0, xzr, d0 __LF                                       \+        sbcs    d1, d1, t1 __LF                                        \+        sbcs    d2, d2, t2 __LF                                        \+        sbcs    d3, d3, t3 __LF                                        \+        sbcs    d4, d4, t3 __LF                                        \+/* Use top word as mask to correct                                       */ \+        adds    d0, d0, d4 __LF                                        \+        mov     t1, #0x00000000ffffffff __LF                           \+        and     t1, t1, d4 __LF                                        \+        adcs    d1, d1, t1 __LF                                        \+        adcs    d2, d2, xzr __LF                                       \+        mov     t1, #0xffffffff00000001 __LF                           \+        and     t1, t1, d4 __LF                                        \+        adc     d3, d3, t1++#define d0 x2+#define d1 x3+#define d2 x4+#define d3 x5+#define d4 x6++#define t0 x1+#define t1 x7+#define t2 x8+#define t3 x9++S2N_BN_SYMBOL(bignum_tomont_p256):++S2N_BN_SYMBOL(bignum_tomont_p256_alt):+        CFI_START++// Load the input++        ldp     d0, d1, [x1]+        ldp     d2, d3, [x1, #16]++// Do an initial reduction to make sure this is < p_256, using just+// a copy of the bignum_mod_p256_4 code. This is needed to set up the+// invariant "input < p_256" for the main modular reduction steps.++        mov     t0, #0xffffffffffffffff+        mov     t1, #0x00000000ffffffff+        mov     t3, #0xffffffff00000001+        subs    t0, d0, t0+        sbcs    t1, d1, t1+        sbcs    t2, d2, xzr+        sbcs    t3, d3, t3+        csel    d0, d0, t0, cc+        csel    d1, d1, t1, cc+        csel    d2, d2, t2, cc+        csel    d3, d3, t3, cc++// Successively multiply by 2^64 and reduce++        modstep_p256(d3,d2,d1,d0,d4, t1,t2,t3)+        modstep_p256(d2,d1,d0,d4,d3, t1,t2,t3)+        modstep_p256(d1,d0,d4,d3,d2, t1,t2,t3)+        modstep_p256(d0,d4,d3,d2,d1, t1,t2,t3)++// Store the result and return++        stp     d1, d2, [x0]+        stp     d3, d4, [x0, #16]++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_tomont_p256)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/bignum_tomont_p384.S view
@@ -0,0 +1,138 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Convert to Montgomery form z := (2^384 * x) mod p_384+// Input x[6]; output z[6]+//+//    extern void bignum_tomont_p384(uint64_t z[static 6],+//                                   const uint64_t x[static 6]);+//+// Standard ARM ABI: X0 = z, X1 = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_tomont_p384)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_tomont_p384)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_tomont_p384)+        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_tomont_p384_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_tomont_p384_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_tomont_p384_alt)+        .text+        .balign 4++// ----------------------------------------------------------------------------+// Core "x |-> (2^64 * x) mod p_384" macro, with x assumed to be < p_384.+// Input is in [d6;d5;d4;d3;d2;d1] and output in [d5;d4;d3;d2;d1;d0]+// using d6 as well as t1, t2, t3 as temporaries.+// ----------------------------------------------------------------------------++#define modstep_p384(d6,d5,d4,d3,d2,d1,d0, t1,t2,t3)                        \+/* Initial quotient approximation q = min (h + 1) (2^64 - 1) */             \+        adds    d6, d6, #1 __LF                                        \+        csetm   t3, cs __LF                                            \+        add     d6, d6, t3 __LF                                        \+        orn     t3, xzr, t3 __LF                                       \+        sub     t2, d6, #1 __LF                                        \+        sub     t1, xzr, d6 __LF                                       \+/* Correction term [d6;t2;t1;d0] = q * (2^384 - p_384) */                   \+        lsl     d0, t1, #32 __LF                                       \+        extr    t1, t2, t1, #32 __LF                                   \+        lsr     t2, t2, #32 __LF                                       \+        adds    d0, d0, d6 __LF                                        \+        adcs    t1, t1, xzr __LF                                       \+        adcs    t2, t2, d6 __LF                                        \+        adc     d6, xzr, xzr __LF                                      \+/* Addition to the initial value */                                         \+        adds    d1, d1, t1 __LF                                        \+        adcs    d2, d2, t2 __LF                                        \+        adcs    d3, d3, d6 __LF                                        \+        adcs    d4, d4, xzr __LF                                       \+        adcs    d5, d5, xzr __LF                                       \+        adc     t3, t3, xzr __LF                                       \+/* Use net top of the 7-word answer in t3 for masked correction */          \+        mov     t1, #0x00000000ffffffff __LF                           \+        and     t1, t1, t3 __LF                                        \+        adds    d0, d0, t1 __LF                                        \+        eor     t1, t1, t3 __LF                                        \+        adcs    d1, d1, t1 __LF                                        \+        mov     t1, #0xfffffffffffffffe __LF                           \+        and     t1, t1, t3 __LF                                        \+        adcs    d2, d2, t1 __LF                                        \+        adcs    d3, d3, t3 __LF                                        \+        adcs    d4, d4, t3 __LF                                        \+        adc     d5, d5, t3++S2N_BN_SYMBOL(bignum_tomont_p384):++S2N_BN_SYMBOL(bignum_tomont_p384_alt):+        CFI_START++#define d0 x2+#define d1 x3+#define d2 x4+#define d3 x5+#define d4 x6+#define d5 x7+#define d6 x8++#define t1 x9+#define t2 x10+#define t3 x11++#define n0 x8+#define n1 x9+#define n2 x10+#define n3 x11+#define n4 x12+#define n5 x1++// Load the inputs++        ldp     d0, d1, [x1]+        ldp     d2, d3, [x1, #16]+        ldp     d4, d5, [x1, #32]++// Do an initial reduction to make sure this is < p_384, using just+// a copy of the bignum_mod_p384_6 code. This is needed to set up the+// invariant "input < p_384" for the main modular reduction steps.++        mov     n0, #0x00000000ffffffff+        mov     n1, #0xffffffff00000000+        mov     n2, #0xfffffffffffffffe+        subs    n0, d0, n0+        sbcs    n1, d1, n1+        sbcs    n2, d2, n2+        adcs    n3, d3, xzr+        adcs    n4, d4, xzr+        adcs    n5, d5, xzr+        csel    d0, d0, n0, cc+        csel    d1, d1, n1, cc+        csel    d2, d2, n2, cc+        csel    d3, d3, n3, cc+        csel    d4, d4, n4, cc+        csel    d5, d5, n5, cc++// Successively multiply by 2^64 and reduce++        modstep_p384(d5,d4,d3,d2,d1,d0,d6, t1,t2,t3)+        modstep_p384(d4,d3,d2,d1,d0,d6,d5, t1,t2,t3)+        modstep_p384(d3,d2,d1,d0,d6,d5,d4, t1,t2,t3)+        modstep_p384(d2,d1,d0,d6,d5,d4,d3, t1,t2,t3)+        modstep_p384(d1,d0,d6,d5,d4,d3,d2, t1,t2,t3)+        modstep_p384(d0,d6,d5,d4,d3,d2,d1, t1,t2,t3)++// Store the result and return++        stp     d1, d2, [x0]+        stp     d3, d4, [x0, #16]+        stp     d5, d6, [x0, #32]++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_tomont_p384)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/curve25519_x25519.S view
@@ -0,0 +1,2596 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// **********************************************************************+// This code is substantially derived from Emil Lenngren's implementation+//+//      https://github.com/Emill/X25519-AArch64/blob/master/X25519_AArch64.pdf+//      https://github.com/Emill/X25519-AArch64+//+// and the SLOTHY-based re-engineering of that code by Abdulrahman, Becker,+// Kannwischer and Klein:+//+//      https://eprint.iacr.org/2022/1303.pdf+//      https://github.com/slothy-optimizer/slothy/tree/main/paper+// **********************************************************************++// ----------------------------------------------------------------------------+// The x25519 function for curve25519+// Inputs scalar[4], point[4]; output res[4]+//+// extern void curve25519_x25519+//   (uint64_t res[static 4],const uint64_t scalar[static 4],+//    const uint64_t point[static 4]);+//+// Given a scalar n and the X coordinate of an input point P = (X,Y) on+// curve25519 (Y can live in any extension field of characteristic 2^255-19),+// this returns the X coordinate of n * P = (X, Y), or 0 when n * P is the+// point at infinity. Both n and X inputs are first slightly modified/mangled+// as specified in the relevant RFC (https://www.rfc-editor.org/rfc/rfc7748);+// in particular the lower three bits of n are set to zero. Does not implement+// the zero-check specified in Section 6.1.+//+// Standard ARM ABI: X0 = res, X1 = scalar, X2 = point+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(curve25519_x25519)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(curve25519_x25519)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(curve25519_x25519)++        .text+        .balign 4++// Pointer-offset pairs for temporaries on stack++#define scalar sp, #0+#define pointx sp, #32+#define mask1 sp, #72+#define mask2 sp, #80+#define tmpa sp, #88+#define tmpb sp, #128+#define xn sp, #128+#define zn sp, #160++#define res sp, #192+#define i sp, #200+#define swap sp, #208++// Total size to reserve on the stack++#define NSPACE 224+#define regsave sp, #NSPACE++S2N_BN_SYMBOL(curve25519_x25519):+        CFI_START++// Save registers and make additional room #NSPACE for temporaries.+// We only need to save the low 64-bits of the Q8...Q15 registers+// according to the ABI, so we use a save of the D8...D15 forms.++        CFI_DEC_SP(NSPACE+160)+        CFI_STACKSAVE2(d8,d9,NSPACE+0)+        CFI_STACKSAVE2(d10,d11,NSPACE+16)+        CFI_STACKSAVE2(d12,d13,NSPACE+32)+        CFI_STACKSAVE2(d14,d15,NSPACE+48)+        CFI_STACKSAVE2(x19,x20,NSPACE+64)+        CFI_STACKSAVE2(x21,x22,NSPACE+80)+        CFI_STACKSAVE2(x23,x24,NSPACE+96)+        CFI_STACKSAVE2(x25,x26,NSPACE+112)+        CFI_STACKSAVE2(x27,x28,NSPACE+128)+        CFI_STACKSAVE2(x29,x30,NSPACE+144)++// Move the output pointer to a stable place++        str     x0, [res]++// Copy the scalar to the corresponding local variable while+// mangling it. In principle it becomes 01xxx...xxx000 where+// the xxx are the corresponding bits of the original input+// scalar. We actually don't bother forcing the MSB to zero,+// but rather start the main loop below at 254 instead of 255.++        ldp     x10, x11, [x1]+        bic     x10, x10, #7+        stp     x10, x11, [scalar]+        ldp     x12, x13, [x1, #16]+        orr     x13, x13, #0x4000000000000000+        stp     x12, x13, [scalar+16]++// Discard the MSB of the point X coordinate (this is in+// accordance with the RFC, mod 2^255, *not* 2^255-19).+// Then recode it into the unsaturated base 25.5 form.++        ldp     x0, x1, [x2]+        ldp     x2, x3, [x2, #16]++        lsr     x12, x0, #51+        lsr     x17, x2, #51+        orr     x12, x12, x1, lsl #13+        orr     x17, x17, x3, lsl #13+        ubfx    x8, x3, #12, #26+        ubfx    x9, x3, #38, #25+        ubfx    x11, x0, #26, #25+        ubfx    x13, x1, #13, #25+        lsr     x14, x1, #38+        ubfx    x16, x2, #25, #26+        and     x10, x0, #0x3ffffff+        and     x12, x12, #0x3ffffff+        and     x15, x2, #0x1ffffff+        and     x17, x17, #0x1ffffff+        orr     x10, x10, x11, lsl #32+        orr     x11, x12, x13, lsl #32+        orr     x12, x14, x15, lsl #32+        orr     x13, x16, x17, lsl #32+        orr     x14, x8, x9, lsl #32++        stp     x10, x11, [pointx+0]+        stp     x12, x13, [pointx+16]+        str     x14, [pointx+32]++// Initialize (X2,Z2) = (1,0), the identity (projective point at infinity)++        mov     x1, #1+        mov     v0.d[0], x1+        mov     v2.d[0], xzr+        mov     v4.d[0], xzr+        mov     v6.d[0], xzr+        mov     v8.d[0], xzr++        mov     v1.d[0], xzr+        mov     v3.d[0], xzr+        mov     v5.d[0], xzr+        mov     v7.d[0], xzr+        mov     v9.d[0], xzr++// Initialize (X3,Z3) = (X,1), projective representation of X++        mov     v10.d[0], x10+        mov     v12.d[0], x11+        mov     v14.d[0], x12+        mov     v16.d[0], x13+        mov     v18.d[0], x14++        mov     v11.d[0], x1+        mov     v13.d[0], xzr+        mov     v15.d[0], xzr+        mov     v17.d[0], xzr+        mov     v19.d[0], xzr++// Set up some constants used repeatedly in the main loop:+//+// Q31 = 0x1300000013 (two 32-bit copies of 19)+// Q30 = 0x3ffffff0000000003ffffff (two 64-bit copies of 2^26-1)+// Q29 = mask1 = (0x07ffffc,0x07fffffe)+// Q28 = mask2 = (0x07ffffb4,0x07fffffe)++        mov     w0, #19+        add     x0, x0, x0, lsl #32+        mov     v31.d[0], x0+        mov     v31.d[1], xzr++        mov     x0, #67108863 // #(1<<26)-1+        mov     v30.d[0], x0+        mov     v30.d[1], x0++        mov     x0, #0x07fffffe07fffffe+        sub     x1, x0, #74 // #0xfe-0xb4+        sub     x0, x0, #2++        stp     x0, x1, [mask1]+        ldp     d29, d28, [mask1]++// The main loop over (modified) bits from i = 254, ..., i = 0 (inclusive);+// we explicitly skip bit 255 because it should be forced to zero initially.+// This is a classic Montgomery ladder using a "swap" variable.+// It's assumed x0 = i at the start of the loop, but that is volatile and+// needs to be reloaded from memory at the end of the loop.++        str     xzr, [swap]+        mov     x0, #254+        str     x0, [i]++Lcurve25519_x25519_scalarloop:++        lsr     x1, x0, #6+        ldr     x2, [sp, x1, lsl #3]    // Exploiting scalar = sp exactly+        lsr     x2, x2, x0+        and     x2, x2, #1++        ldr     x0, [swap]+        cmp     x0, x2+        str     x2, [swap]++// The following inner loop code is derived closely following Lenngren's+// implementation available at "https://github.com/Emill/X25519-AArch64".+// In particular, the basic dataflow and the organization between integer+// and SIMD units is identical, with only a few minor changes to some+// individual instructions (for miscellaneous reasons). The scheduling+// was redone from scratch by SLOTHY starting from the un-interleaved+// form in the SLOTHY work cited above, and using the same scripts.+//+// The intermediate value annotations were added to provide data that+// is used in the formal proof, indicating which lines assign specific+// digits of the various intermediate results (mainly of field+// operations, sometimes other transformations). The names used for+// the intermediate results are similar but not identical to those in+// the abstract Algorithm 1 description in Lenngren's paper. Almost+// all equations are to be interpreted as field operations, i.e. as+// arithmetic modulo 2^255-19, not simple numeric equalities.+//+//      b = x2 - z2+//      d = x3 - z3+//      a = x2 + z2+//      c = x3 + z3+//      f = if flip then c else a+//      g = if flip then d else b+//      aa = f^2+//      bb = g^2+//      bbalt = bb (change of representation)+//      e = aa - bb+//      bce = bbalt + 121666 * e+//      z4 = bce * e+//      bc = b * c+//      ad = a * d+//      t1 = ad + bc+//      t2 = ad - bc+//      x5 = t1^2+//      t3 = t2^2+//      x4 = aa * bb+//      z5 = x * t3+//+// Then the main variables are updated for the next iteration as+//+//      (x2',z2') = (x4,z4)+//      (x3',z3') = (x5,z5)++        add     v22.2s, v2.2s, v3.2s            // ubignum_of_qreglist 1 // INTERMEDIATE a+        sub     v21.2s, v28.2s, v1.2s+        add     v25.2s, v0.2s, v1.2s            // ubignum_of_qreglist 0 // INTERMEDIATE a+        sub     v24.2s, v29.2s, v3.2s+        add     v3.2s, v18.2s, v19.2s           // ubignum_of_qreglist 4 // INTERMEDIATE c+        add     v0.2s, v0.2s, v21.2s            // ubignum_of_qreglist 0 // INTERMEDIATE b+        sub     v20.2s, v29.2s, v15.2s+        sub     v1.2s, v29.2s, v5.2s+        sub     v26.2s, v28.2s, v11.2s+        sub     v21.2s, v29.2s, v19.2s+        add     v19.2s, v10.2s, v11.2s          // ubignum_of_qreglist 0 // INTERMEDIATE c+        add     v11.2s, v14.2s, v20.2s          // ubignum_of_qreglist 2 // INTERMEDIATE d+        add     v21.2s, v18.2s, v21.2s          // ubignum_of_qreglist 4 // INTERMEDIATE d+        sub     v20.2s, v29.2s, v17.2s+        add     v18.2s, v2.2s, v24.2s           // ubignum_of_qreglist 1 // INTERMEDIATE b+        add     v14.2s, v14.2s, v15.2s          // ubignum_of_qreglist 2 // INTERMEDIATE c+        add     v15.2s, v16.2s, v17.2s          // ubignum_of_qreglist 3 // INTERMEDIATE c+        add     v2.2s, v16.2s, v20.2s           // ubignum_of_qreglist 3 // INTERMEDIATE d+        add     v24.2s, v12.2s, v13.2s          // ubignum_of_qreglist 1 // INTERMEDIATE c+        add     v26.2s, v10.2s, v26.2s          // ubignum_of_qreglist 0 // INTERMEDIATE d+        sub     v10.2s, v29.2s, v13.2s+        sub     v13.2s, v29.2s, v7.2s+        add     v23.2s, v6.2s, v7.2s            // ubignum_of_qreglist 3 // INTERMEDIATE a+        sub     v7.2s, v29.2s, v9.2s+        add     v27.2s, v12.2s, v10.2s          // ubignum_of_qreglist 1 // INTERMEDIATE d+        fcsel   d20, d22, d24, eq               // ubignum_of_qreglist 1 // INTERMEDIATE f+        add     v28.2s, v4.2s, v5.2s            // ubignum_of_qreglist 2 // INTERMEDIATE a+        fcsel   d12, d23, d15, eq               // ubignum_of_qreglist 3 // INTERMEDIATE f+        add     v7.2s, v8.2s, v7.2s             // ubignum_of_qreglist 4 // INTERMEDIATE b+        fcsel   d16, d25, d19, eq               // ubignum_of_qreglist 0 // INTERMEDIATE f+        mov     x0, v20.d[0]+        fcsel   d5, d28, d14, eq                // ubignum_of_qreglist 2 // INTERMEDIATE f+        mov     x21, v12.d[0]+        fcsel   d29, d7, d21, eq                // ubignum_of_qreglist 4 // INTERMEDIATE g+        mov     x5, v16.d[0]+        lsr     x26, x0, #32+        add     x29, x21, x21+        umull   x15, w5, w29+        add     v13.2s, v6.2s, v13.2s           // ubignum_of_qreglist 3 // INTERMEDIATE b+        add     x12, x26, x26+        mov     x30, v5.d[0]+        fcsel   d10, d18, d27, eq               // ubignum_of_qreglist 1 // INTERMEDIATE g+        lsr     x11, x5, #32+        lsr     x10, x30, #32+        trn2    v20.2s, v21.2s, v3.2s+        add     v9.2s, v8.2s, v9.2s             // ubignum_of_qreglist 4 // INTERMEDIATE a+        add     x14, x11, x11+        trn2    v6.2s, v2.2s, v15.2s+        trn1    v12.2s, v25.2s, v0.2s+        add     v1.2s, v4.2s, v1.2s             // ubignum_of_qreglist 2 // INTERMEDIATE b+        trn1    v16.2s, v23.2s, v13.2s+        fcsel   d8, d13, d2, eq                 // ubignum_of_qreglist 3 // INTERMEDIATE g+        trn2    v17.2s, v27.2s, v24.2s+        str     d29, [tmpb+32]+        add     x17, x10, x10+        trn2    v4.2s, v28.2s, v1.2s+        trn1    v5.2s, v28.2s, v1.2s+        trn1    v28.2s, v2.2s, v15.2s+        trn1    v2.2s, v22.2s, v18.2s+        fcsel   d29, d0, d26, eq                // ubignum_of_qreglist 0 // INTERMEDIATE g+        trn2    v15.2s, v22.2s, v18.2s+        umull   v22.2d, v12.2s, v20.2s+        umull   x22, w30, w17+        stp     d29, d10, [tmpb+0]+        trn2    v10.2s, v23.2s, v13.2s+        trn2    v23.2s, v11.2s, v14.2s+        trn1    v13.2s, v27.2s, v24.2s+        fcsel   d27, d1, d11, eq                // ubignum_of_qreglist 2 // INTERMEDIATE g+        trn1    v14.2s, v11.2s, v14.2s+        umlal   v22.2d, v2.2s, v6.2s+        umull   x25, w30, w30+        umlal   v22.2d, v5.2s, v23.2s+        add     x3, x30, x30+        umlal   v22.2d, v16.2s, v17.2s+        add     w30, w21, w21, lsl #1;+        stp     d27, d8, [tmpb+16]+        add     w30, w30, w21, lsl #4+        trn1    v11.2s, v26.2s, v19.2s+        trn2    v8.2s, v26.2s, v19.2s+        trn2    v19.2s, v25.2s, v0.2s+        mul     v29.2s, v20.2s, v31.2s+        ldr     x20, [tmpb+24]+        umull   v25.2d, v19.2s, v6.2s+        add     x1, x0, x0+        umull   v27.2d, v19.2s, v23.2s+        umull   x9, w5, w1+        umull   v0.2d, v12.2s, v23.2s+        lsr     x24, x20, #32+        mul     v20.2s, v23.2s, v31.2s+        lsr     x16, x21, #32+        umlal   v25.2d, v15.2s, v23.2s+        umaddl  x13, w11, w14, x9+        umlal   v25.2d, v4.2s, v17.2s+        umaddl  x9, w14, w17, x15+        umull   v24.2d, v12.2s, v6.2s+        add     w2, w16, w16, lsl #1;+        fcsel   d26, d9, d3, eq                 // ubignum_of_qreglist 4 // INTERMEDIATE f+        add     w2, w2, w16, lsl #4+        trn1    v18.2s, v21.2s, v3.2s+        umull   v3.2d, v19.2s, v29.2s+        umull   x28, w5, w3+        mul     v1.2s, v6.2s, v31.2s+        umull   x8, w5, w5+        umlal   v24.2d, v2.2s, v23.2s+        umaddl  x13, w21, w30, x13+        mul     v23.2s, v17.2s, v31.2s+        umaddl  x27, w14, w12, x28+        trn2    v6.2s, v9.2s, v7.2s+        mov     x6, v26.d[0]+        umlal   v3.2d, v15.2s, v1.2s+        add     x16, x16, x16+        umlal   v3.2d, v4.2s, v20.2s+        lsr     x4, x6, #32+        umlal   v3.2d, v10.2s, v23.2s+        add     x7, x6, x6+        umull   v26.2d, v19.2s, v8.2s+        add     x23, x4, x4+        umaddl  x28, w5, w23, x22+        trn1    v7.2s, v9.2s, v7.2s+        umlal   v27.2d, v15.2s, v17.2s+        add     w15, w4, w4, lsl #1;+        umlal   v27.2d, v4.2s, v8.2s+        add     w15, w15, w4, lsl #4+        add     w22, w10, w10, lsl #1;+        umlal   v24.2d, v5.2s, v17.2s+        add     w22, w22, w10, lsl #4+        umaddl  x10, w11, w7, x28+        umlal   v25.2d, v10.2s, v8.2s+        umull   x21, w5, w16+        umlal   v25.2d, v6.2s, v29.2s+        umaddl  x23, w15, w23, x25+        umlal   v27.2d, v10.2s, v29.2s+        umull   x19, w5, w12+        umlal   v27.2d, v6.2s, v1.2s+        umaddl  x25, w11, w29, x21+        umlal   v0.2d, v2.2s, v17.2s+        umaddl  x28, w0, w3, x9+        shl     v21.2d, v25.2d, #1+        umaddl  x4, w11, w1, x19+        umaddl  x21, w2, w29, x4+        mul     v25.2s, v8.2s, v31.2s+        umlal   v24.2d, v16.2s, v8.2s+        umaddl  x19, w0, w17, x25+        umlal   v24.2d, v7.2s, v29.2s+        umull   x25, w5, w17+        umlal   v24.2d, v19.2s, v28.2s+        umaddl  x4, w0, w16, x10+        umull   v9.2d, v12.2s, v8.2s+        umaddl  x23, w5, w7, x23+        umlal   v21.2d, v12.2s, v18.2s+        add     w10, w6, w6, lsl #1;+        shl     v27.2d, v27.2d, #1+        add     w10, w10, w6, lsl #4+        umaddl  x28, w26, w12, x28+        umlal   v26.2d, v15.2s, v29.2s+        umaddl  x9, w14, w16, x23+        umlal   v9.2d, v2.2s, v29.2s+        umaddl  x22, w22, w17, x8+        umlal   v21.2d, v2.2s, v28.2s+        umaddl  x28, w6, w10, x28+        umaddl  x27, w0, w0, x27+        add     x8, x14, x14+        umlal   v0.2d, v5.2s, v8.2s+        umull   x5, w5, w14+        umlal   v9.2d, v5.2s, v1.2s+        umaddl  x14, w0, w29, x9+        umlal   v26.2d, v4.2s, v1.2s+        umaddl  x6, w2, w16, x27+        umlal   v22.2d, v7.2s, v8.2s+        umaddl  x5, w30, w17, x5+        umaddl  x5, w2, w3, x5+        add     x23, x17, x17+        umlal   v27.2d, v12.2s, v28.2s+        umaddl  x13, w2, w23, x13+        umlal   v26.2d, v10.2s, v20.2s+        add     x9, x12, x12+        umlal   v9.2d, v16.2s, v20.2s+        umaddl  x27, w10, w29, x6+        umlal   v0.2d, v16.2s, v29.2s+        umaddl  x6, w11, w3, x25+        umlal   v22.2d, v19.2s, v18.2s+        umaddl  x19, w26, w3, x19+        mul     v18.2s, v18.2s, v31.2s+        umaddl  x23, w15, w23, x27+        umlal   v3.2d, v6.2s, v25.2s+        umaddl  x0, w0, w12, x6+        umlal   v0.2d, v7.2s, v1.2s+        add     x11, x16, x16+        umlal   v9.2d, v7.2s, v23.2s+        umaddl  x6, w12, w17, x14+        umlal   v9.2d, v19.2s, v11.2s+        umaddl  x25, w26, w29, x4+        umlal   v9.2d, v15.2s, v18.2s+        umaddl  x14, w10, w3, x13+        umull   v25.2d, v12.2s, v17.2s+        umaddl  x27, w10, w16, x0+        umlal   v26.2d, v6.2s, v23.2s+        add     x0, x25, x6, lsr #26+        mul     v23.2s, v28.2s, v31.2s+        umaddl  x12, w10, w12, x5+        shl     v3.2d, v3.2d, #1+        add     x16, x22, x0, lsr #25+        umlal   v21.2d, v5.2s, v14.2s+        bic     x22, x0, #0x1ffffff+        umlal   v3.2d, v12.2s, v11.2s+        add     x26, x16, x22, lsr #24+        umlal   v3.2d, v2.2s, v18.2s+        umaddl  x16, w10, w17, x21+        umlal   v3.2d, v5.2s, v23.2s+        add     x22, x26, x22, lsr #21+        umlal   v9.2d, v4.2s, v23.2s+        umaddl  x5, w15, w29, x27+        umull   v17.2d, v19.2s, v17.2s+        umaddl  x17, w30, w3, x22+        umlal   v25.2d, v2.2s, v8.2s+        umaddl  x25, w15, w3, x16+        umlal   v25.2d, v5.2s, v29.2s+        umaddl  x26, w15, w7, x19+        umlal   v0.2d, v19.2s, v14.2s+        umaddl  x17, w2, w9, x17+        umlal   v17.2d, v15.2s, v8.2s+        ldr     x19, [tmpb+0]+        umlal   v17.2d, v4.2s, v29.2s+        ldr     x7, [tmpb+8]+        shl     v29.2d, v26.2d, #1+        umaddl  x13, w10, w1, x17+        umlal   v0.2d, v15.2s, v13.2s+        lsr     x2, x19, #32+        umlal   v29.2d, v12.2s, v13.2s+        umaddl  x27, w15, w1, x12+        umlal   v29.2d, v2.2s, v11.2s+        umaddl  x30, w15, w8, x13+        umlal   v29.2d, v5.2s, v18.2s+        add     x4, x7, x7+        umlal   v29.2d, v16.2s, v23.2s+        umaddl  x29, w15, w9, x14+        umlal   v0.2d, v4.2s, v11.2s+        add     x17, x27, x30, lsr #26+        umlal   v0.2d, v10.2s, v18.2s+        umaddl  x16, w15, w11, x28+        umlal   v0.2d, v6.2s, v23.2s+        add     x1, x29, x17, lsr #25+        umlal   v25.2d, v16.2s, v1.2s+        umull   x11, w19, w4+        ldr     x8, [tmpb+32]+        mul     v26.2s, v14.2s, v31.2s+        umlal   v17.2d, v10.2s, v1.2s+        ldr     x15, [tmpb+16]+        umlal   v17.2d, v6.2s, v20.2s+        and     x9, x30, #0x3ffffff+        bfi     x9, x17, #32, #25               // ubignum_of_preglist 0 // INTERMEDIATE aa+        add     x17, x2, x2+        lsr     x10, x15, #32+        add     x27, x25, x1, lsr #26+        umlal   v25.2d, v7.2s, v20.2s+        add     x13, x10, x10+        umlal   v25.2d, v19.2s, v13.2s+        add     x29, x23, x27, lsr #25+        umlal   v25.2d, v15.2s, v11.2s+        lsr     x30, x8, #32+        umlal   v25.2d, v4.2s, v18.2s+        add     x23, x5, x29, lsr #26+        umlal   v25.2d, v10.2s, v23.2s+        and     x14, x29, #0x3ffffff+        umlal   v25.2d, v6.2s, v26.2s+        add     x5, x16, x23, lsr #25+        shl     v8.2d, v17.2d, #1+        umaddl  x12, w2, w17, x11+        and     x29, x5, #0x3ffffff+        umull   x21, w19, w19+        umlal   v29.2d, v7.2s, v26.2s+        add     w16, w10, w10, lsl #1;+        umlal   v3.2d, v16.2s, v26.2s+        add     w16, w16, w10, lsl #4+        bfi     x14, x23, #32, #25              // ubignum_of_preglist 2 // INTERMEDIATE aa+        add     w10, w24, w24, lsl #1;+        add     x22, x26, x5, lsr #26+        add     w10, w10, w24, lsl #4+        umlal   v8.2d, v12.2s, v14.2s+        umaddl  x25, w16, w13, x21+        umlal   v8.2d, v2.2s, v13.2s+        bfi     x29, x22, #32, #25              // ubignum_of_preglist 3 // INTERMEDIATE aa+        umlal   v8.2d, v5.2s, v11.2s+        add     x26, x24, x24+        umlal   v8.2d, v16.2s, v18.2s+        stp     x14, x29, [tmpa+16]+        umlal   v8.2d, v7.2s, v23.2s+        add     w24, w30, w30, lsl #1;+        usra    v25.2d, v29.2d, #26+        add     w24, w24, w30, lsl #4+        umull   x29, w15, w15+        umlal   v27.2d, v2.2s, v14.2s+        umull   x3, w15, w13+        umlal   v27.2d, v5.2s, v13.2s+        add     x21, x20, x20+        umlal   v24.2d, v15.2s, v14.2s+        umull   x5, w19, w21+        umlal   v24.2d, v4.2s, v13.2s+        and     x11, x1, #0x3ffffff+        usra    v8.2d, v25.2d, #25+        and     x1, x0, #0x1ffffff+        umlal   v27.2d, v16.2s, v11.2s+        umaddl  x23, w17, w13, x5+        umlal   v27.2d, v7.2s, v18.2s+        add     x5, x30, x30+        usra    v0.2d, v8.2d, #26+        add     x0, x15, x15+        umlal   v24.2d, v10.2s, v11.2s+        umaddl  x23, w7, w0, x23+        umlal   v24.2d, v6.2s, v18.2s+        lsr     x30, x7, #32+        usra    v27.2d, v0.2d, #25+        add     x16, x30, x30+        and     v20.16b, v8.16b, v30.16b        // ubignum_of_hreglist 4 + ubignum_of_lreglist 4 // INTERMEDIATE H|L = bc|ad+        umaddl  x15, w30, w16, x23+        ushr    v23.2d, v30.2d, #1+        add     w23, w8, w8, lsl #1;+        usra    v24.2d, v27.2d, #26+        add     w23, w23, w8, lsl #4+        umaddl  x14, w19, w5, x3+        and     v8.16b, v27.16b, v30.16b        // ubignum_of_hreglist 6 + ubignum_of_lreglist 6 // INTERMEDIATE H|L = bc|ad+        add     x28, x8, x8+        and     v27.16b, v0.16b, v23.16b        // ubignum_of_hreglist 5 + ubignum_of_lreglist 5 // INTERMEDIATE H|L = bc|ad+        umaddl  x8, w8, w23, x15+        and     v5.16b, v24.16b, v23.16b        // ubignum_of_hreglist 7 + ubignum_of_lreglist 7 // INTERMEDIATE H|L = bc|ad+        umaddl  x3, w2, w28, x14+        umlal   v22.2d, v15.2s, v28.2s+        bfi     x11, x27, #32, #25              // ubignum_of_preglist 1 // INTERMEDIATE aa+        uzp1    v5.4s, v8.4s, v5.4s+        umaddl  x14, w24, w5, x29+        umaddl  x5, w19, w28, x14+        ldr     d18, [mask1]+        mov     v18.d[1], v18.d[0]+        umaddl  x15, w7, w26, x3+        mul     v12.2s, v13.2s, v31.2s+        umlal   v21.2d, v16.2s, v13.2s+        stp     x9, x11, [tmpa+0]+        umlal   v21.2d, v7.2s, v11.2s+        umaddl  x29, w17, w26, x5+        umlal   v22.2d, v4.2s, v14.2s+        add     w14, w20, w20, lsl #1;+        umlal   v22.2d, v10.2s, v13.2s+        add     w14, w14, w20, lsl #4+        umull   x3, w19, w0+        umlal   v22.2d, v6.2s, v11.2s+        umaddl  x29, w7, w21, x29+        usra    v21.2d, v24.2d, #25+        umaddl  x11, w20, w14, x12+        and     v0.16b, v25.16b, v23.16b+        umaddl  x5, w30, w21, x15+        and     v14.16b, v29.16b, v30.16b+        umaddl  x12, w16, w13, x29+        usra    v22.2d, v21.2d, #26+        umaddl  x29, w17, w16, x3+        umlal   v3.2d, v7.2s, v12.2s+        add     x9, x26, x26+        and     v1.16b, v21.16b, v30.16b        // ubignum_of_hreglist 8 + ubignum_of_lreglist 8 // INTERMEDIATE H|L = bc|ad+        add     x27, x5, x12, lsr #26+        bic     v8.16b, v22.16b, v23.16b+        umaddl  x29, w7, w7, x29+        and     v17.16b, v22.16b, v23.16b       // ubignum_of_hreglist 9 + ubignum_of_lreglist 9 // INTERMEDIATE H|L = bc|ad+        add     x5, x25, x27, lsr #25+        usra    v3.2d, v8.2d, #25+        umaddl  x25, w24, w9, x8+        umlal   v9.2d, v10.2s, v26.2s+        add     x8, x13, x13+        trn1    v22.4s, v1.4s, v17.4s+        umaddl  x11, w10, w8, x11+        usra    v3.2d, v8.2d, #24+        umull   x20, w19, w16+        add     v26.2s, v22.2s, v18.2s+        ldr     d28, [mask2]+        umlal   v9.2d, v6.2s, v12.2s+        umaddl  x3, w23, w0, x11+        usra    v3.2d, v8.2d, #21+        umaddl  x29, w10, w26, x29+        uzp1    v11.4s, v20.4s, v27.4s+        umaddl  x20, w2, w4, x20+        umaddl  x9, w10, w21, x20+        mov     v17.d[0], v22.d[1]+        usra    v9.2d, v3.2d, #26+        umull   x15, w19, w13+        and     v7.16b, v3.16b, v30.16b         // ubignum_of_hreglist 0 + ubignum_of_lreglist 0 // INTERMEDIATE H|L = bc|ad+        add     x11, x16, x16+        uzp2    v1.4s, v11.4s, v5.4s+        umaddl  x20, w23, w13, x9+        and     v8.16b, v9.16b, v23.16b         // ubignum_of_hreglist 1 + ubignum_of_lreglist 1 // INTERMEDIATE H|L = bc|ad+        umaddl  x9, w2, w0, x15+        usra    v14.2d, v9.2d, #25+        and     x6, x6, #0x3ffffff+        uzp1    v7.4s, v7.4s, v8.4s+        umaddl  x29, w23, w21, x29+        uzp1    v27.4s, v11.4s, v5.4s+        umull   x15, w19, w26+        usra    v0.2d, v14.2d, #26              // ubignum_of_hreglist 3 + ubignum_of_lreglist 3 // INTERMEDIATE H|L = bc|ad+        add     x6, x6, x22, lsr #25+        and     v3.16b, v14.16b, v30.16b        // ubignum_of_hreglist 2 + ubignum_of_lreglist 2 // INTERMEDIATE H|L = bc|ad+        bic     x22, x27, #0x1ffffff+        sub     v2.2s, v26.2s, v17.2s+        add     v9.2s, v22.2s, v17.2s+        uzp1    v14.4s, v3.4s, v0.4s+        umaddl  x2, w2, w21, x15+        add     v5.4s, v27.4s, v18.4s+        add     x5, x5, x22, lsr #24+        zip1    v22.2s, v2.2s, v9.2s            // ubignum_of_h32reglist 8 + ubignum_of_l32reglist 8 // INTERMEDIATE H|L = t1|t2+        mov     v18.b[0], v28.b[0]+        uzp1    v8.4s, v7.4s, v14.4s+        add     x22, x5, x22, lsr #21+        uzp2    v3.4s, v7.4s, v14.4s+        umaddl  x5, w7, w16, x9+        add     v25.4s, v8.4s, v18.4s+        umaddl  x15, w14, w0, x22+        add     v12.4s, v27.4s, v1.4s+        add     x9, x17, x17+        sub     v14.4s, v5.4s, v1.4s+        umull   x19, w19, w17+        sub     v18.4s, v25.4s, v3.4s+        ldr     x22, [tmpa+8]+        add     v20.4s, v8.4s, v3.4s+        umaddl  x15, w10, w11, x15+        zip1    v16.4s, v14.4s, v12.4s          // ubignum_of_h32reglist 4 + ubignum_of_l32reglist 4 // INTERMEDIATE H|L = t1|t2+        umaddl  x14, w14, w13, x19+        zip2    v14.4s, v14.4s, v12.4s          // ubignum_of_h32reglist 6 + ubignum_of_l32reglist 6 // INTERMEDIATE H|L = t1|t2+        and     x17, x27, #0x1ffffff+        zip2    v0.4s, v18.4s, v20.4s           // ubignum_of_h32reglist 2 + ubignum_of_l32reglist 2 // INTERMEDIATE H|L = t1|t2+        umaddl  x15, w23, w4, x15+        zip1    v1.4s, v18.4s, v20.4s           // ubignum_of_h32reglist 0 + ubignum_of_l32reglist 0 // INTERMEDIATE H|L = t1|t2+        umaddl  x10, w10, w0, x14+        zip2    v5.2s, v2.2s, v9.2s             // ubignum_of_h32reglist 9 + ubignum_of_l32reglist 9 // INTERMEDIATE H|L = t1|t2+        shl     v24.2s, v0.2s, #1+        mov     v19.d[0], v1.d[1]               // ubignum_of_h32reglist 1 + ubignum_of_l32reglist 1 // INTERMEDIATE H|L = t1|t2+        shl     v26.2s, v22.2s, #1+        shl     v17.2s, v16.2s, #1+        mov     v15.d[0], v0.d[1]               // ubignum_of_h32reglist 3 + ubignum_of_l32reglist 3 // INTERMEDIATE H|L = t1|t2+        shl     v7.2s, v5.2s, #1+        shl     v18.2s, v19.2s, #1+        umull   v11.2d, v1.2s, v24.2s+        umaddl  x19, w23, w16, x10+        umull   v6.2d, v1.2s, v17.2s+        umaddl  x10, w7, w13, x2+        mov     v4.d[0], v16.d[1]               // ubignum_of_h32reglist 5 + ubignum_of_l32reglist 5 // INTERMEDIATE H|L = t1|t2+        mov     v10.d[0], v14.d[1]              // ubignum_of_h32reglist 7 + ubignum_of_l32reglist 7 // INTERMEDIATE H|L = t1|t2+        umull   v9.2d, v1.2s, v26.2s+        ldr     x13, [tmpa+0]+        shl     v28.2s, v15.2s, #1+        shl     v3.2s, v10.2s, #1+        ldr     x14, [tmpa+16]+        mul     v12.2s, v10.2s, v31.2s+        umull   v25.2d, v1.2s, v7.2s+        ldr     x2, [tmpa+24]+        umlal   v6.2d, v18.2s, v28.2s+        umaddl  x27, w30, w0, x10+        umaddl  x16, w24, w0, x20+        shl     v13.2s, v14.2s, #1+        umaddl  x5, w23, w26, x5+        mul     v2.2s, v22.2s, v31.2s+        umull   v21.2d, v1.2s, v13.2s+        umaddl  x23, w24, w8, x29+        umlal   v11.2d, v18.2s, v19.2s+        mov     x10, #0x07fffffe07fffffe+        sub     x10, x10, #2+        umaddl  x26, w24, w21, x5+        mul     v29.2s, v14.2s, v31.2s+        umlal   v25.2d, v19.2s, v26.2s+        add     x7, x1, x6, lsr #26+        mul     v20.2s, v4.2s, v31.2s+        and     x6, x6, #0x3ffffff+        shl     v8.2s, v18.2s, #1+        shl     v4.2s, v4.2s, #1+        umlal   v11.2d, v29.2s, v14.2s+        bfi     x6, x7, #32, #26                // ubignum_of_preglist 4 // INTERMEDIATE aa+        umlal   v25.2d, v0.2s, v3.2s+        umaddl  x0, w24, w4, x19+        umlal   v25.2d, v15.2s, v13.2s+        str     x6, [tmpa+32]+        umlal   v21.2d, v18.2s, v4.2s+        umaddl  x8, w24, w11, x3+        umlal   v21.2d, v0.2s, v17.2s+        ldr     x30, [tmpa+32]+        mul     v14.2s, v5.2s, v31.2s+        add     x2, x2, x10+        shl     v5.2s, v28.2s, #1+        shl     v27.2s, v4.2s, #1+        umlal   v6.2d, v0.2s, v0.2s+        umaddl  x11, w24, w9, x15+        umlal   v6.2d, v12.2s, v3.2s+        add     x4, x30, x10+        umlal   v11.2d, v14.2s, v5.2s+        add     x3, x22, x10+        umlal   v11.2d, v2.2s, v17.2s+        add     x6, x0, x11, lsr #26+        umlal   v11.2d, v12.2s, v27.2s+        add     x14, x14, x10+        umlal   v6.2d, v14.2s, v27.2s+        add     x8, x8, x6, lsr #25+        umlal   v6.2d, v2.2s, v13.2s+        movk    x10, #0xffb4+        umlal   v25.2d, v16.2s, v4.2s+        add     x29, x16, x8, lsr #26+        umull   v27.2d, v1.2s, v3.2s+        and     x11, x11, #0x3ffffff+        umlal   v9.2d, v18.2s, v3.2s+        add     x19, x13, x10+        umlal   v9.2d, v0.2s, v13.2s+        and     x5, x8, #0x3ffffff+        umlal   v9.2d, v28.2s, v4.2s+        bfi     x11, x6, #32, #25               // ubignum_of_preglist 0 // INTERMEDIATE bb+        umlal   v9.2d, v16.2s, v16.2s+        umaddl  x30, w24, w28, x27+        umlal   v9.2d, v14.2s, v7.2s+        sub     x13, x19, x11+        umull   v10.2d, v1.2s, v18.2s+        add     x7, x23, x29, lsr #25+        umlal   v21.2d, v28.2s, v15.2s+        lsr     x16, x13, #32                   // ubignum_of_wreglist 1 + ubignum_of_wreglist 0 // INTERMEDIATE e+        umlal   v21.2d, v2.2s, v22.2s+        add     x0, x26, x7, lsr #26+        usra    v25.2d, v9.2d, #26+        and     x20, x7, #0x3ffffff+        umull   v22.2d, v1.2s, v1.2s+        add     x8, x25, x0, lsr #25+        umull   v7.2d, v1.2s, v28.2s+        and     x1, x29, #0x1ffffff             // ubignum_of_xreglist 3 // INTERMEDIATE bbalt+        bic     v18.16b, v25.16b, v23.16b+        and     x19, x8, #0x3ffffff+        and     v16.16b, v9.16b, v30.16b+        and     x7, x12, #0x3ffffff+        usra    v22.2d, v18.2d, #25+        add     x10, x30, x8, lsr #26+        umlal   v7.2d, v19.2s, v24.2s+        bfi     x5, x29, #32, #25               // ubignum_of_preglist 1 // INTERMEDIATE bb+        and     v9.16b, v25.16b, v23.16b+        add     x27, x7, x10, lsr #25+        usra    v22.2d, v18.2d, #24+        mov     x21, #60833+        lsl     x21, x21, #1+        add     x15, x17, x27, lsr #26+        shl     v25.2s, v3.2s, #1+        umlal   v7.2d, v14.2s, v17.2s+        and     x29, x27, #0x3ffffff+        usra    v22.2d, v18.2d, #21+        bfi     x29, x15, #32, #26              // ubignum_of_preglist 4 // INTERMEDIATE bb // ***SOURCE*** ubignum_of_xreglist 9 // INTERMEDIATE bbalt+        umlal   v10.2d, v14.2s, v24.2s+        and     x17, x6, #0x1ffffff             // ubignum_of_xreglist 1 // INTERMEDIATE bbalt+        umlal   v10.2d, v2.2s, v28.2s+        sub     x6, x3, x5+        umlal   v10.2d, v12.2s, v17.2s+        umaddl  x25, w16, w21, x17+        umlal   v10.2d, v29.2s, v4.2s+        mov     w12, w5                         // ubignum_of_xreglist 2 // INTERMEDIATE bbalt+        umlal   v22.2d, v20.2s, v4.2s+        lsr     x26, x6, #32                    // ubignum_of_wreglist 3 + ubignum_of_wreglist 2 // INTERMEDIATE e+        umlal   v22.2d, v14.2s, v8.2s+        and     x24, x0, #0x1ffffff             // ubignum_of_xreglist 5 // INTERMEDIATE bbalt+        umlal   v22.2d, v2.2s, v24.2s+        stp     x11, x5, [tmpb+0]+        umlal   v22.2d, v12.2s, v5.2s+        bfi     x20, x0, #32, #25               // ubignum_of_preglist 2 // INTERMEDIATE bb+        umlal   v22.2d, v29.2s, v17.2s+        umaddl  x12, w6, w21, x12+        umull   v18.2d, v1.2s, v4.2s+        bfi     x19, x10, #32, #25              // ubignum_of_preglist 3 // INTERMEDIATE bb+        umlal   v7.2d, v2.2s, v4.2s+        sub     x7, x14, x20+        umlal   v27.2d, v19.2s, v13.2s+        mov     w8, w20                         // ubignum_of_xreglist 4 // INTERMEDIATE bbalt+        usra    v10.2d, v22.2d, #26+        lsr     x14, x7, #32                    // ubignum_of_wreglist 5 + ubignum_of_wreglist 4 // INTERMEDIATE e+        umlal   v18.2d, v19.2s, v17.2s+        and     x28, x10, #0x1ffffff            // ubignum_of_xreglist 7 // INTERMEDIATE bbalt+        umlal   v7.2d, v12.2s, v13.2s+        sub     x5, x2, x19+        usra    v11.2d, v10.2d, #25+        mov     w2, w19                         // ubignum_of_xreglist 6 // INTERMEDIATE bbalt+        umlal   v27.2d, v0.2s, v4.2s+        umlal   v21.2d, v14.2s, v25.2s+        sub     x23, x4, x29+        usra    v7.2d, v11.2d, #26+        mov     w0, w29                         // ubignum_of_xreglist 8 // INTERMEDIATE bbalt+        umlal   v18.2d, v0.2s, v28.2s+        lsr     x22, x23, #32                   // ubignum_of_wreglist 9 + ubignum_of_wreglist 8 // INTERMEDIATE e+        umlal   v27.2d, v15.2s, v17.2s+        str     x29, [tmpb+32]+        usra    v6.2d, v7.2d, #25+        mov     w17, w11                        // ubignum_of_xreglist 0 // INTERMEDIATE bbalt+        and     v0.16b, v22.16b, v30.16b        // ubignum_of_hreglist 0 + ubignum_of_lreglist 0 // INTERMEDIATE H|L = x5|t3+        umaddl  x27, w26, w21, x1+        umlal   v18.2d, v14.2s, v13.2s+        umaddl  x30, w23, w21, x0+        umlal   v18.2d, v2.2s, v3.2s+        lsr     x10, x5, #32                    // ubignum_of_wreglist 7 + ubignum_of_wreglist 6 // INTERMEDIATE e+        and     v4.16b, v6.16b, v30.16b         // ubignum_of_hreglist 4 + ubignum_of_lreglist 4 // INTERMEDIATE H|L = x5|t3+        and     v1.16b, v10.16b, v23.16b        // ubignum_of_hreglist 1 + ubignum_of_lreglist 1 // INTERMEDIATE H|L = x5|t3+        umaddl  x4, w14, w21, x24+        ldr     x0, [tmpa+0]+        mov     v0.s[1], w0+        lsr     x0, x0, #32+        mov     v1.s[1], w0+        umaddl  x9, w7, w21, x8+        usra    v18.2d, v6.2d, #26+        umaddl  x24, w10, w21, x28+        and     v3.16b, v7.16b, v23.16b         // ubignum_of_hreglist 3 + ubignum_of_lreglist 3 // INTERMEDIATE H|L = x5|t3+        umaddl  x8, w22, w21, x15+        umlal   v27.2d, v14.2s, v26.2s+        umaddl  x15, w13, w21, x17+        usra    v21.2d, v18.2d, #25+        stp     x20, x19, [tmpb+16]+        and     v2.16b, v11.16b, v30.16b        // ubignum_of_hreglist 2 + ubignum_of_lreglist 2 // INTERMEDIATE H|L = x5|t3+        lsr     x29, x8, #25+        ldr     x3, [tmpb+0]+        mov     v10.s[1], w3+        lsr     x3, x3, #32+        mov     v11.s[1], w3+        add     x17, x15, x29+        usra    v27.2d, v21.2d, #26+        add     x28, x17, x29, lsl #1+        and     v6.16b, v21.16b, v30.16b        // ubignum_of_hreglist 6 + ubignum_of_lreglist 6 // INTERMEDIATE H|L = x5|t3+        and     x20, x8, #0x1ffffff+        and     v5.16b, v18.16b, v23.16b        // ubignum_of_hreglist 5 + ubignum_of_lreglist 5 // INTERMEDIATE H|L = x5|t3+        add     x17, x28, x29, lsl #4+        and     v7.16b, v27.16b, v23.16b        // ubignum_of_hreglist 7 + ubignum_of_lreglist 7 // INTERMEDIATE H|L = x5|t3+        ldr     x3, [tmpb+8]+        mov     v22.s[1], w3+        lsr     x3, x3, #32+        mov     v23.s[1], w3+        add     x29, x25, x17, lsr #26+        ldr     x15, [pointx+0]+        mov     v10.s[0], w15+        lsr     x15, x15, #32+        mov     v11.s[0], w15+        and     x11, x17, #0x3ffffff            // ubignum_of_xreglist 0 // INTERMEDIATE bce+        usra    v16.2d, v27.2d, #25+        add     x8, x12, x29, lsr #25+        ldr     x3, [tmpb+16]+        mov     v14.s[1], w3+        lsr     x3, x3, #32+        mov     v15.s[1], w3+        and     x12, x29, #0x1ffffff            // ubignum_of_xreglist 1 // INTERMEDIATE bce+        ldr     x15, [pointx+8]+        mov     v22.s[0], w15+        lsr     x15, x15, #32+        mov     v23.s[0], w15+        add     x28, x27, x8, lsr #26+        and     v8.16b, v16.16b, v30.16b        // ubignum_of_hreglist 8 + ubignum_of_lreglist 8 // INTERMEDIATE H|L = x5|t3+        umull   x1, w12, w10+        ldr     x3, [tmpb+24]+        mov     v17.s[1], w3+        lsr     x3, x3, #32+        mov     v18.s[1], w3+        add     x25, x9, x28, lsr #25+        ldr     x15, [pointx+16]+        mov     v14.s[0], w15+        lsr     x15, x15, #32+        mov     v15.s[0], w15+        umaddl  x19, w5, w21, x2+        usra    v9.2d, v16.2d, #26              // ubignum_of_hreglist 8 + ubignum_of_lreglist 8 // INTERMEDIATE H|L = x5|t3+        add     x2, x4, x25, lsr #26+        ldr     x3, [tmpb+32]+        mov     v24.s[1], w3+        lsr     x3, x3, #32+        mov     v25.s[1], w3+        umull   x3, w12, w23+        ldr     x15, [pointx+24]+        mov     v17.s[0], w15+        lsr     x15, x15, #32+        mov     v18.s[0], w15+        add     x29, x19, x2, lsr #25+        umull   v26.2d, v0.2s, v23.2s+        and     x21, x28, #0x1ffffff            // ubignum_of_xreglist 3 // INTERMEDIATE bce+        ldr     x0, [tmpa+8]+        mov     v2.s[1], w0+        lsr     x0, x0, #32+        mov     v3.s[1], w0+        umaddl  x27, w21, w5, x3+        ldr     x15, [pointx+32]+        mov     v24.s[0], w15+        lsr     x15, x15, #32+        mov     v25.s[0], w15+        add     x17, x24, x29, lsr #26+        umull   v29.2d, v1.2s, v18.2s+        and     x15, x8, #0x3ffffff             // ubignum_of_xreglist 2 // INTERMEDIATE bce+        umull   v20.2d, v0.2s, v15.2s+        add     x19, x30, x17, lsr #25+        and     x3, x17, #0x1ffffff             // ubignum_of_xreglist 7 // INTERMEDIATE bce+        mul     v12.2s, v25.2s, v31.2s+        ldr     x0, [tmpa+16]+        mov     v4.s[1], w0+        lsr     x0, x0, #32+        mov     v5.s[1], w0+        add     x4, x20, x19, lsr #26           // ubignum_of_xreglist 9 // INTERMEDIATE bce+        umlal   v26.2d, v2.2s, v11.2s+        add     w28, w3, w3, lsl #1;+        umlal   v20.2d, v2.2s, v23.2s+        add     w28, w28, w3, lsl #4+        umull   x8, w12, w5+        ldr     x0, [tmpa+24]+        mov     v6.s[1], w0+        lsr     x0, x0, #32+        mov     v7.s[1], w0+        and     x30, x25, #0x3ffffff            // ubignum_of_xreglist 4 // INTERMEDIATE bce+        mul     v16.2s, v18.2s, v31.2s+        add     w17, w4, w4, lsl #1;+        umull   v21.2d, v1.2s, v15.2s+        add     w17, w17, w4, lsl #4+        umaddl  x25, w21, w7, x8+        umlal   v20.2d, v4.2s, v11.2s+        add     w8, w21, w21, lsl #1;+        ldr     x0, [tmpa+32]+        add     w8, w8, w21, lsl #4+        mov     v8.s[1], w0+        lsr     x0, x0, #32+        mov     v9.s[1], w0+        and     x2, x2, #0x1ffffff              // ubignum_of_xreglist 5 // INTERMEDIATE bce+        umlal   v29.2d, v3.2s, v15.2s+        umaddl  x24, w2, w6, x25+        umull   v13.2d, v0.2s, v25.2s+        umaddl  x25, w2, w7, x27+        umaddl  x0, w3, w6, x25+        mul     v19.2s, v15.2s, v31.2s+        umull   v27.2d, v0.2s, v18.2s+        umaddl  x20, w3, w13, x24+        umlal   v20.2d, v6.2s, v12.2s+        umaddl  x24, w21, w14, x1+        umlal   v13.2d, v2.2s, v18.2s+        umaddl  x9, w4, w13, x0+        umull   v25.2d, v0.2s, v11.2s+        umaddl  x20, w17, w23, x20+        umlal   v27.2d, v2.2s, v15.2s+        umaddl  x0, w2, w26, x24+        umull   v28.2d, v1.2s, v11.2s+        umull   x24, w17, w5+        umlal   v29.2d, v5.2s, v23.2s+        umaddl  x9, w11, w22, x9+        umlal   v13.2d, v4.2s, v15.2s+        umaddl  x27, w3, w16, x0+        umlal   v27.2d, v4.2s, v23.2s+        umull   x0, w17, w14+        umlal   v27.2d, v6.2s, v11.2s+        umull   x4, w12, w14+        umlal   v27.2d, v8.2s, v12.2s+        umaddl  x25, w11, w10, x20+        umlal   v27.2d, v1.2s, v17.2s+        umaddl  x0, w28, w10, x0+        umlal   v13.2d, v6.2s, v23.2s+        umull   x3, w17, w6+        umlal   v13.2d, v8.2s, v11.2s+        umaddl  x1, w21, w26, x4+        umlal   v20.2d, v8.2s, v16.2s+        umaddl  x4, w2, w13, x24+        umlal   v28.2d, v3.2s, v12.2s+        umaddl  x20, w28, w7, x3+        umlal   v29.2d, v7.2s, v11.2s+        and     x3, x19, #0x3ffffff             // ubignum_of_xreglist 9 // INTERMEDIATE bce+        umlal   v29.2d, v9.2s, v12.2s+        umaddl  x19, w17, w22, x27+        add     w27, w2, w2, lsl #1;+        mul     v18.2s, v24.2s, v31.2s+        add     w27, w27, w2, lsl #4+        umlal   v21.2d, v3.2s, v23.2s+        umull   x24, w17, w7+        umlal   v13.2d, v1.2s, v24.2s+        add     x19, x19, x19+        shl     v29.2d, v29.2d, #1+        umaddl  x1, w2, w16, x1+        umull   v15.2d, v1.2s, v23.2s+        umaddl  x0, w27, w22, x0+        umlal   v29.2d, v0.2s, v24.2s+        umaddl  x2, w28, w5, x24+        mul     v24.2s, v23.2s, v31.2s+        umaddl  x4, w28, w23, x4+        umlal   v21.2d, v5.2s, v11.2s+        umaddl  x24, w27, w5, x20+        umlal   v20.2d, v1.2s, v14.2s+        umaddl  x20, w11, w23, x19+        umlal   v26.2d, v4.2s, v12.2s+        umaddl  x19, w27, w23, x2+        umlal   v26.2d, v6.2s, v16.2s+        umaddl  x2, w21, w6, x4+        umlal   v29.2d, v2.2s, v17.2s+        umaddl  x24, w8, w23, x24+        umlal   v15.2d, v3.2s, v11.2s+        umaddl  x0, w21, w16, x0+        umaddl  x4, w21, w13, x19+        mul     v23.2s, v11.2s, v31.2s+        umlal   v20.2d, v3.2s, v22.2s+        umaddl  x2, w12, w7, x2+        umlal   v20.2d, v5.2s, v10.2s+        umaddl  x19, w12, w26, x0+        umlal   v29.2d, v4.2s, v14.2s+        umaddl  x0, w12, w13, x24+        umlal   v26.2d, v8.2s, v19.2s+        umaddl  x20, w15, w5, x20+        umlal   v26.2d, v1.2s, v22.2s+        umaddl  x21, w15, w10, x9+        umlal   v26.2d, v3.2s, v10.2s+        and     x9, x29, #0x3ffffff             // ubignum_of_xreglist 6 // INTERMEDIATE bce+        umlal   v29.2d, v6.2s, v22.2s+        umaddl  x20, w30, w7, x20+        umaddl  x1, w28, w22, x1+        add     x24, x19, x19+        umull   v11.2d, v1.2s, v12.2s+        add     w19, w3, w3, lsl #1;+        umlal   v26.2d, v5.2s, v18.2s+        add     w19, w19, w3, lsl #4+        umaddl  x20, w9, w6, x20+        umlal   v29.2d, v8.2s, v10.2s+        add     w29, w9, w9, lsl #1;+        umlal   v13.2d, v3.2s, v17.2s+        add     w29, w29, w9, lsl #4+        umaddl  x2, w19, w10, x2+        umlal   v11.2d, v3.2s, v16.2s+        umaddl  x21, w30, w14, x21+        umlal   v11.2d, v5.2s, v19.2s+        umaddl  x20, w3, w13, x20+        umlal   v11.2d, v7.2s, v24.2s+        umaddl  x2, w29, w22, x2+        umlal   v11.2d, v9.2s, v23.2s+        umaddl  x21, w9, w26, x21+        ushr    v23.2d, v30.2d, #1+        umaddl  x1, w17, w10, x1+        umlal   v13.2d, v5.2s, v14.2s+        umaddl  x24, w19, w5, x24+        umlal   v27.2d, v3.2s, v14.2s+        umaddl  x21, w3, w16, x21+        shl     v11.2d, v11.2d, #1+        add     w3, w30, w30, lsl #1;+        umlal   v28.2d, v5.2s, v16.2s+        add     w3, w3, w30, lsl #4+        umaddl  x24, w29, w23, x24+        umlal   v28.2d, v7.2s, v19.2s+        add     x1, x1, x1+        umlal   v28.2d, v9.2s, v24.2s+        umaddl  x1, w11, w5, x1+        umlal   v15.2d, v5.2s, v12.2s+        umaddl  x24, w30, w13, x24+        umlal   v15.2d, v7.2s, v16.2s+        umaddl  x25, w15, w14, x25+        umlal   v15.2d, v9.2s, v19.2s+        umaddl  x1, w15, w7, x1+        shl     v28.2d, v28.2d, #1+        umaddl  x24, w15, w6, x24+        umlal   v21.2d, v7.2s, v12.2s+        umaddl  x2, w30, w16, x2+        umlal   v21.2d, v9.2s, v16.2s+        umaddl  x25, w30, w26, x25+        shl     v15.2d, v15.2d, #1+        umaddl  x30, w30, w6, x1+        umlal   v28.2d, v0.2s, v22.2s+        umaddl  x1, w15, w26, x2+        umlal   v28.2d, v2.2s, v10.2s+        umaddl  x2, w9, w16, x25+        shl     v21.2d, v21.2d, #1+        umaddl  x24, w11, w7, x24+        umlal   v15.2d, v0.2s, v14.2s+        umaddl  x1, w11, w14, x1+        umlal   v21.2d, v0.2s, v17.2s+        umaddl  x25, w9, w13, x30+        umlal   v28.2d, v4.2s, v18.2s+        umaddl  x0, w19, w26, x0+        umlal   v25.2d, v2.2s, v12.2s+        add     x1, x1, x24, lsr #26+        umlal   v25.2d, v4.2s, v16.2s+        umaddl  x30, w19, w22, x2+        umlal   v21.2d, v2.2s, v14.2s+        umaddl  x4, w12, w6, x4+        mul     v14.2s, v14.2s, v31.2s+        umaddl  x25, w19, w23, x25+        and     x2, x1, #0x1ffffff+        mul     v16.2s, v17.2s, v31.2s+        umlal   v25.2d, v6.2s, v19.2s+        umaddl  x9, w19, w14, x4+        umlal   v13.2d, v7.2s, v22.2s+        add     x25, x25, x1, lsr #25+        umlal   v21.2d, v4.2s, v22.2s+        umaddl  x0, w29, w14, x0+        umlal   v26.2d, v7.2s, v16.2s+        add     x30, x30, x25, lsr #26+        umlal   v26.2d, v9.2s, v14.2s+        add     w1, w15, w15, lsl #1;+        umlal   v28.2d, v6.2s, v16.2s+        add     w1, w1, w15, lsl #4+        add     x4, x20, x30, lsr #25+        umlal   v28.2d, v8.2s, v14.2s+        and     x25, x25, #0x3ffffff+        umlal   v15.2d, v2.2s, v22.2s+        add     x21, x21, x4, lsr #26+        umlal   v11.2d, v0.2s, v10.2s+        bfi     x25, x30, #32, #25              // ubignum_of_preglist 3 // INTERMEDIATE z4+        umlal   v11.2d, v2.2s, v18.2s+        bic     x30, x21, #0x3ffffff+        usra    v26.2d, v28.2d, #26+        lsr     x20, x30, #26+        umlal   v15.2d, v4.2s, v10.2s+        add     x20, x20, x30, lsr #25+        umlal   v15.2d, v6.2s, v18.2s+        umaddl  x9, w29, w10, x9+        umlal   v15.2d, v8.2s, v16.2s+        add     x30, x20, x30, lsr #22+        umlal   v27.2d, v5.2s, v22.2s+        umull   x20, w17, w26+        umlal   v20.2d, v7.2s, v18.2s+        umaddl  x30, w17, w16, x30+        umlal   v20.2d, v9.2s, v16.2s+        umaddl  x17, w3, w10, x0+        usra    v15.2d, v26.2d, #25+        umaddl  x0, w28, w14, x20+        umlal   v27.2d, v7.2s, v10.2s+        umaddl  x20, w28, w26, x30+        umlal   v27.2d, v9.2s, v18.2s+        add     w28, w12, w12, lsl #1;+        usra    v20.2d, v15.2d, #26+        add     w28, w28, w12, lsl #4+        umaddl  x30, w27, w10, x0+        and     v17.16b, v15.16b, v30.16b       // ubignum_of_hreglist 4 + ubignum_of_lreglist 4 // INTERMEDIATE H|L = x4|z5+        umaddl  x27, w27, w14, x20+        umaddl  x0, w8, w10, x27+        mul     v12.2s, v22.2s, v31.2s+        and     v15.16b, v20.16b, v23.16b       // ubignum_of_hreglist 5 + ubignum_of_lreglist 5 // INTERMEDIATE H|L = x4|z5+        umaddl  x14, w3, w22, x9+        umlal   v21.2d, v6.2s, v10.2s+        umaddl  x27, w8, w22, x30+        trn1    v15.4s, v17.4s, v15.4s          // FINAL z3+        umaddl  x10, w28, w22, x0+        umlal   v11.2d, v4.2s, v16.2s+        umaddl  x30, w15, w16, x14+        and     v26.16b, v26.16b, v23.16b+        umaddl  x28, w12, w16, x27+        umlal   v21.2d, v8.2s, v18.2s+        add     x10, x10, x10+        umlal   v25.2d, v8.2s, v24.2s+        umaddl  x20, w19, w6, x10+        umlal   v25.2d, v1.2s, v10.2s+        add     x28, x28, x28+        umlal   v25.2d, v3.2s, v18.2s+        umaddl  x28, w19, w7, x28+        usra    v21.2d, v20.2d, #25+        umaddl  x0, w29, w7, x20+        umlal   v11.2d, v6.2s, v14.2s+        umaddl  x10, w11, w26, x30+        umlal   v13.2d, v9.2s, v10.2s+        umaddl  x19, w29, w5, x28+        usra    v27.2d, v21.2d, #26+        umaddl  x0, w3, w5, x0+        umlal   v25.2d, v5.2s, v16.2s+        umaddl  x20, w1, w22, x17+        and     v20.16b, v28.16b, v30.16b+        umaddl  x29, w3, w23, x19+        usra    v29.2d, v27.2d, #25+        umaddl  x3, w1, w23, x0+        and     v27.16b, v27.16b, v23.16b       // ubignum_of_hreglist 7 + ubignum_of_lreglist 7 // INTERMEDIATE H|L = x4|z5+        umlal   v11.2d, v8.2s, v12.2s+        umaddl  x12, w15, w13, x29+        usra    v13.2d, v29.2d, #26+        umaddl  x7, w11, w13, x3+        trn1    v6.4s, v6.4s, v7.4s+        umaddl  x17, w11, w16, x20+        umlal   v25.2d, v7.2s, v14.2s+        and     x23, x4, #0x3ffffff+        bic     v19.16b, v13.16b, v23.16b+        umaddl  x19, w11, w6, x12+        and     v28.16b, v13.16b, v23.16b       // ubignum_of_hreglist 9 + ubignum_of_lreglist 9 // INTERMEDIATE H|L = x4|z5+        add     x3, x17, x7, lsr #26+        usra    v11.2d, v19.2d, #25+        trn1    v2.4s, v2.4s, v3.4s+        add     x17, x19, x3, lsr #25+        and     v13.16b, v21.16b, v30.16b       // ubignum_of_hreglist 6 + ubignum_of_lreglist 6 // INTERMEDIATE H|L = x4|z5+        and     x5, x7, #0x3ffffff+        usra    v11.2d, v19.2d, #24+        add     x7, x10, x17, lsr #26+        trn1    v0.4s, v0.4s, v1.4s+        and     x19, x24, #0x3ffffff+        and     v21.16b, v29.16b, v30.16b       // ubignum_of_hreglist 8 + ubignum_of_lreglist 8 // INTERMEDIATE H|L = x4|z5+        add     x29, x19, x7, lsr #25+        usra    v11.2d, v19.2d, #21+        bfi     x5, x3, #32, #25                // ubignum_of_preglist 0 // INTERMEDIATE z4+        trn1    v17.4s, v13.4s, v27.4s          // FINAL z3+        add     x19, x2, x29, lsr #26+        trn1    v19.4s, v21.4s, v28.4s          // FINAL z3+        and     x3, x29, #0x3ffffff+        mov     v16.d[0], v6.d[1]               // FINAL x3+        mov     v6.d[0], v17.d[1]               // FINAL x2+        trn1    v8.4s, v8.4s, v9.4s+        bfi     x3, x19, #32, #26               // ubignum_of_preglist 2 // INTERMEDIATE z4+        and     v21.16b, v11.16b, v30.16b       // ubignum_of_hreglist 0 + ubignum_of_lreglist 0 // INTERMEDIATE H|L = x4|z5+        bfi     x23, x21, #32, #26              // ubignum_of_preglist 4 // INTERMEDIATE z4+        mov     v18.d[0], v8.d[1]               // FINAL x3+        mov     v8.d[0], v19.d[1]               // FINAL x2+        umlal   v25.2d, v9.2s, v12.2s+        mov     v9.d[0], x23                    // FINAL z2+        mov     v7.d[0], x25                    // FINAL z2+        ldr     d29, [mask1]+        mov     v12.d[0], v2.d[1]               // FINAL x3+        trn1    v4.4s, v4.4s, v5.4s+        and     x17, x17, #0x3ffffff+        usra    v25.2d, v11.2d, #26+        mov     v10.d[0], v0.d[1]               // FINAL x3+        mov     v14.d[0], v4.d[1]               // FINAL x3+        mov     v4.d[0], v15.d[1]               // FINAL x2+        usra    v20.2d, v25.2d, #25+        and     v27.16b, v25.16b, v23.16b       // ubignum_of_hreglist 1 + ubignum_of_lreglist 1 // INTERMEDIATE H|L = x4|z5+        bfi     x17, x7, #32, #25               // ubignum_of_preglist 1 // INTERMEDIATE z4+        mov     v5.d[0], x3+        mov     v1.d[0], x5                     // FINAL z2+        usra    v26.2d, v20.2d, #26             // ubignum_of_hreglist 3 + ubignum_of_lreglist 3 // INTERMEDIATE H|L = x4|z5+        and     v28.16b, v20.16b, v30.16b       // ubignum_of_hreglist 2 + ubignum_of_lreglist 2 // INTERMEDIATE H|L = x4|z5+        trn1    v11.4s, v21.4s, v27.4s          // FINAL z3+        trn1    v13.4s, v28.4s, v26.4s          // FINAL z3+        mov     v0.d[0], v11.d[1]               // FINAL x2+        mov     v3.d[0], x17                    // FINAL z2+        mov     v2.d[0], v13.d[1]               // FINAL x2+        ldr     d28, [mask2]++        ldr     x0, [i]+        subs    x0, x0, #1+        str     x0, [i]+        bcs     Lcurve25519_x25519_scalarloop++// Repack X2 into the saturated representation as 256-bit value xn.+// This does not fully normalize mod 2^255-19 but stays within 256 bits.++        mov     w0, v0.s[0]+        mov     w1, v0.s[1]+        mov     w2, v2.s[0]+        mov     w3, v2.s[1]+        mov     w4, v4.s[0]+        mov     w5, v4.s[1]+        mov     w6, v6.s[0]+        mov     w7, v6.s[1]+        mov     w8, v8.s[0]+        mov     w9, v8.s[1]++        add     x0, x0, x1, lsl #26+        add     x1, x2, x3, lsl #26+        add     x2, x4, x5, lsl #26+        add     x3, x6, x7, lsl #26+        add     x4, x8, x9, lsl #26++        adds    x0, x0, x1, lsl #51+        lsr     x6, x1, #13+        lsl     x7, x2, #38+        adcs    x1, x6, x7+        lsr     x8, x2, #26+        lsl     x9, x3, #25+        adcs    x2, x8, x9+        lsr     x10, x3, #39+        lsl     x11, x4, #12+        adc     x3, x10, x11+        stp     x0, x1, [xn]+        stp     x2, x3, [xn+16]++// Repack Z2 into the saturated representation as 256-bit value zn.+// This does not fully normalize mod 2^255-19. However since Z2,+// unlike X2, was not repacked (within the last multiplication) in+// right-to-left order, its top digit can be any 26-bit value, on+// the face of it. To make sure we don't overflow 256 bits here+// we remove b = 25th bit of the 9th digit (now scaled by 2^230+// giving bit 25 a final weighting of 2^255) and add 19 * b to+// to the bottom of the sum here to compensate mod 2^255-19.++        mov     w0, v1.s[0]+        mov     w1, v1.s[1]+        mov     w2, v3.s[0]+        mov     w3, v3.s[1]+        mov     w4, v5.s[0]+        mov     w5, v5.s[1]+        mov     w6, v7.s[0]+        mov     w7, v7.s[1]+        mov     w8, v9.s[0]+        mov     w9, v9.s[1]++        mov     w10, #19+        add     x0, x0, x1, lsl #26+        tst     x9, #0x2000000+        add     x1, x2, x3, lsl #26+        csel    x10, x10, xzr, ne+        add     x2, x4, x5, lsl #26+        and     x9, x9, #0x1FFFFFF+        add     x3, x6, x7, lsl #26+        add     x0, x0, x10+        add     x4, x8, x9, lsl #26++        adds    x0, x0, x1, lsl #51+        lsr     x6, x1, #13+        lsl     x7, x2, #38+        adcs    x1, x6, x7+        lsr     x8, x2, #26+        lsl     x9, x3, #25+        adcs    x2, x8, x9+        lsr     x10, x3, #39+        lsl     x11, x4, #12+        adc     x3, x10, x11+        stp     x0, x1, [zn]+        stp     x2, x3, [zn+16]++// Because the lowest bit (indeed, the three lowest bits) of the scalar+// were forced to zero, we know that the projective result of the scalar+// multiplication was in (X2,Z2) and is now (xn,zn) in saturated form.+// Prepare to call the modular inverse function to get zn' = 1/zn.++        add     x0, zn+        add     x1, zn++// Inline copy of bignum_inv_p25519, identical except for stripping out+// the prologue and epilogue saving and restoring registers and making+// and reclaiming room on the stack. For more details and explanations see+// "arm/curve25519/bignum_inv_p25519.S". Note that the stack it uses for+// its own temporaries is 128 bytes, so it has no effect on variables+// that are needed in the rest of our computation here: res, xn, and zn.++        mov     x20, x0+        mov     x10, #0xffffffffffffffed+        mov     x11, #0xffffffffffffffff+        stp     x10, x11, [sp]+        mov     x12, #0x7fffffffffffffff+        stp     x11, x12, [sp, #16]+        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]+        mov     x7, #0x13+        lsr     x6, x5, #63+        madd    x6, x7, x6, x7+        adds    x2, x2, x6+        adcs    x3, x3, xzr+        adcs    x4, x4, xzr+        orr     x5, x5, #0x8000000000000000+        adcs    x5, x5, xzr+        csel    x6, x7, xzr, cc+        subs    x2, x2, x6+        sbcs    x3, x3, xzr+        sbcs    x4, x4, xzr+        sbc     x5, x5, xzr+        and     x5, x5, #0x7fffffffffffffff+        stp     x2, x3, [sp, #32]+        stp     x4, x5, [sp, #48]+        stp     xzr, xzr, [sp, #64]+        stp     xzr, xzr, [sp, #80]+        mov     x10, #0x2099+        movk    x10, #0x7502, lsl #16+        movk    x10, #0x9e23, lsl #32+        movk    x10, #0xa0f9, lsl #48+        mov     x11, #0x2595+        movk    x11, #0x1d13, lsl #16+        movk    x11, #0x8f3f, lsl #32+        movk    x11, #0xa8c6, lsl #48+        mov     x12, #0x5242+        movk    x12, #0x5ac, lsl #16+        movk    x12, #0x8938, lsl #32+        movk    x12, #0x6c6c, lsl #48+        mov     x13, #0x615+        movk    x13, #0x4177, lsl #16+        movk    x13, #0x8b2, lsl #32+        movk    x13, #0x2765, lsl #48+        stp     x10, x11, [sp, #96]+        stp     x12, x13, [sp, #112]+        mov     x21, #0xa+        mov     x22, #0x1+        b       Lcurve25519_x25519_invmidloop+Lcurve25519_x25519_invloop:+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        and     x0, x12, x16+        and     x1, x13, x17+        add     x19, x0, x1+        ldr     x7, [sp]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #32]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x7, [sp, #8]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #40]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [sp]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [sp, #32]+        ldr     x7, [sp, #16]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #48]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [sp, #8]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [sp, #40]+        ldr     x7, [sp, #24]+        eor     x1, x7, x14+        asr     x3, x1, #63+        and     x3, x3, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #56]+        eor     x1, x8, x15+        asr     x0, x1, #63+        and     x0, x0, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x5, x6, #59+        str     x6, [sp, #16]+        extr    x5, x3, x5, #59+        str     x5, [sp, #24]+        eor     x1, x7, x16+        asr     x5, x1, #63+        and     x5, x5, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        asr     x0, x1, #63+        and     x0, x0, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x4, x2, x4, #59+        str     x4, [sp, #48]+        extr    x2, x5, x2, #59+        str     x2, [sp, #56]+        ldr     x7, [sp, #64]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #96]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #64]+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        str     x5, [sp, #96]+        adc     x3, x3, x1+        ldr     x7, [sp, #72]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #104]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #72]+        adc     x6, x6, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        str     x3, [sp, #104]+        adc     x4, x4, x1+        ldr     x7, [sp, #80]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #112]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #80]+        adc     x5, x5, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        str     x4, [sp, #112]+        adc     x2, x2, x1+        ldr     x7, [sp, #88]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #120]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x3, x5, #63+        ldp     x0, x1, [sp, #64]+        add     x6, x6, x3, asr #63+        mov     x3, #0x13+        mul     x4, x6, x3+        add     x5, x5, x6, lsl #63+        smulh   x3, x6, x3+        ldr     x6, [sp, #80]+        adds    x0, x0, x4+        adcs    x1, x1, x3+        asr     x3, x3, #63+        adcs    x6, x6, x3+        adc     x5, x5, x3+        stp     x0, x1, [sp, #64]+        stp     x6, x5, [sp, #80]+        eor     x1, x7, x16+        and     x5, x16, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        and     x0, x17, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x6, x5, x2, #63+        ldp     x0, x1, [sp, #96]+        add     x6, x6, x5, asr #63+        mov     x5, #0x13+        mul     x4, x6, x5+        add     x2, x2, x6, lsl #63+        smulh   x5, x6, x5+        ldr     x3, [sp, #112]+        adds    x0, x0, x4+        adcs    x1, x1, x5+        asr     x5, x5, #63+        adcs    x3, x3, x5+        adc     x2, x2, x5+        stp     x0, x1, [sp, #96]+        stp     x3, x2, [sp, #112]+Lcurve25519_x25519_invmidloop:+        mov     x1, x22+        ldr     x2, [sp]+        ldr     x3, [sp, #32]+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x8, x4, #0x100, lsl #12+        sbfx    x8, x8, #21, #21+        mov     x11, #0x100000+        add     x11, x11, x11, lsl #21+        add     x9, x4, x11+        asr     x9, x9, #42+        add     x10, x5, #0x100, lsl #12+        sbfx    x10, x10, #21, #21+        add     x11, x5, x11+        asr     x11, x11, #42+        mul     x6, x8, x2+        mul     x7, x9, x3+        mul     x2, x10, x2+        mul     x3, x11, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #21, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #42+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #21, #21+        add     x15, x5, x15+        asr     x15, x15, #42+        mul     x6, x12, x2+        mul     x7, x13, x3+        mul     x2, x14, x2+        mul     x3, x15, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        mul     x2, x12, x8+        mul     x3, x12, x9+        mul     x6, x14, x8+        mul     x7, x14, x9+        madd    x8, x13, x10, x2+        madd    x9, x13, x11, x3+        madd    x16, x15, x10, x6+        madd    x17, x15, x11, x7+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #22, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #43+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #22, #21+        add     x15, x5, x15+        asr     x15, x15, #43+        mneg    x2, x12, x8+        mneg    x3, x12, x9+        mneg    x4, x14, x8+        mneg    x5, x14, x9+        msub    x10, x13, x16, x2+        msub    x11, x13, x17, x3+        msub    x12, x15, x16, x4+        msub    x13, x15, x17, x5+        mov     x22, x1+        subs    x21, x21, #0x1+        b.ne    Lcurve25519_x25519_invloop+        ldr     x0, [sp]+        ldr     x1, [sp, #32]+        mul     x0, x0, x10+        madd    x1, x1, x11, x0+        asr     x0, x1, #63+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        eor     x14, x14, x0+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        eor     x15, x15, x0+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        eor     x16, x16, x0+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        eor     x17, x17, x0+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        ldr     x7, [sp, #64]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #96]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #64]+        adc     x2, x2, x1+        ldr     x7, [sp, #72]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #104]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #72]+        adc     x6, x6, x1+        ldr     x7, [sp, #80]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #112]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #80]+        adc     x5, x5, x1+        ldr     x7, [sp, #88]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #120]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x3, x5, #63+        ldp     x0, x1, [sp, #64]+        tst     x3, x3+        cinc    x6, x6, pl+        mov     x3, #0x13+        mul     x4, x6, x3+        add     x5, x5, x6, lsl #63+        smulh   x6, x6, x3+        ldr     x2, [sp, #80]+        adds    x0, x0, x4+        adcs    x1, x1, x6+        asr     x6, x6, #63+        adcs    x2, x2, x6+        adcs    x5, x5, x6+        csel    x3, x3, xzr, mi+        subs    x0, x0, x3+        sbcs    x1, x1, xzr+        sbcs    x2, x2, xzr+        sbc     x5, x5, xzr+        and     x5, x5, #0x7fffffffffffffff+        mov     x4, x20+        stp     x0, x1, [x4]+        stp     x2, x5, [x4, #16]++// Now the result is xn * (1/zn), fully reduced modulo p.+// Note that in the degenerate case zn = 0 (mod p_25519), the+// modular inverse code above will produce 1/zn = 0, giving+// the correct overall X25519 result of zero for the point at+// infinity. The multiplication below is just an inlined+// version of bignum_mul_p25519 except for the detailed+// addressing of inputs and outputs++        ldr     x17, [res]++        ldp     x3, x4, [xn]+        ldp     x5, x6, [zn]+        umull   x7, w3, w5+        lsr     x0, x3, #32+        umull   x15, w0, w5+        lsr     x16, x5, #32+        umull   x8, w16, w0+        umull   x16, w3, w16+        adds    x7, x7, x15, lsl #32+        lsr     x15, x15, #32+        adc     x8, x8, x15+        adds    x7, x7, x16, lsl #32+        lsr     x16, x16, #32+        adc     x8, x8, x16+        mul     x9, x4, x6+        umulh   x10, x4, x6+        subs    x4, x4, x3+        cneg    x4, x4, cc+        csetm   x16, cc+        adds    x9, x9, x8+        adc     x10, x10, xzr+        subs    x3, x5, x6+        cneg    x3, x3, cc+        cinv    x16, x16, cc+        mul     x15, x4, x3+        umulh   x3, x4, x3+        adds    x8, x7, x9+        adcs    x9, x9, x10+        adc     x10, x10, xzr+        cmn     x16, #0x1+        eor     x15, x15, x16+        adcs    x8, x15, x8+        eor     x3, x3, x16+        adcs    x9, x3, x9+        adc     x10, x10, x16+        ldp     x3, x4, [xn+16]+        ldp     x5, x6, [zn+16]+        umull   x11, w3, w5+        lsr     x0, x3, #32+        umull   x15, w0, w5+        lsr     x16, x5, #32+        umull   x12, w16, w0+        umull   x16, w3, w16+        adds    x11, x11, x15, lsl #32+        lsr     x15, x15, #32+        adc     x12, x12, x15+        adds    x11, x11, x16, lsl #32+        lsr     x16, x16, #32+        adc     x12, x12, x16+        mul     x13, x4, x6+        umulh   x14, x4, x6+        subs    x4, x4, x3+        cneg    x4, x4, cc+        csetm   x16, cc+        adds    x13, x13, x12+        adc     x14, x14, xzr+        subs    x3, x5, x6+        cneg    x3, x3, cc+        cinv    x16, x16, cc+        mul     x15, x4, x3+        umulh   x3, x4, x3+        adds    x12, x11, x13+        adcs    x13, x13, x14+        adc     x14, x14, xzr+        cmn     x16, #0x1+        eor     x15, x15, x16+        adcs    x12, x15, x12+        eor     x3, x3, x16+        adcs    x13, x3, x13+        adc     x14, x14, x16+        ldp     x3, x4, [xn+16]+        ldp     x15, x16, [xn]+        subs    x3, x3, x15+        sbcs    x4, x4, x16+        csetm   x16, cc+        ldp     x15, x0, [zn]+        subs    x5, x15, x5+        sbcs    x6, x0, x6+        csetm   x0, cc+        eor     x3, x3, x16+        subs    x3, x3, x16+        eor     x4, x4, x16+        sbc     x4, x4, x16+        eor     x5, x5, x0+        subs    x5, x5, x0+        eor     x6, x6, x0+        sbc     x6, x6, x0+        eor     x16, x0, x16+        adds    x11, x11, x9+        adcs    x12, x12, x10+        adcs    x13, x13, xzr+        adc     x14, x14, xzr+        mul     x2, x3, x5+        umulh   x0, x3, x5+        mul     x15, x4, x6+        umulh   x1, x4, x6+        subs    x4, x4, x3+        cneg    x4, x4, cc+        csetm   x9, cc+        adds    x15, x15, x0+        adc     x1, x1, xzr+        subs    x6, x5, x6+        cneg    x6, x6, cc+        cinv    x9, x9, cc+        mul     x5, x4, x6+        umulh   x6, x4, x6+        adds    x0, x2, x15+        adcs    x15, x15, x1+        adc     x1, x1, xzr+        cmn     x9, #0x1+        eor     x5, x5, x9+        adcs    x0, x5, x0+        eor     x6, x6, x9+        adcs    x15, x6, x15+        adc     x1, x1, x9+        adds    x9, x11, x7+        adcs    x10, x12, x8+        adcs    x11, x13, x11+        adcs    x12, x14, x12+        adcs    x13, x13, xzr+        adc     x14, x14, xzr+        cmn     x16, #0x1+        eor     x2, x2, x16+        adcs    x9, x2, x9+        eor     x0, x0, x16+        adcs    x10, x0, x10+        eor     x15, x15, x16+        adcs    x11, x15, x11+        eor     x1, x1, x16+        adcs    x12, x1, x12+        adcs    x13, x13, x16+        adc     x14, x14, x16+        mov     x3, #0x26+        umull   x4, w11, w3+        add     x4, x4, w7, uxtw+        lsr     x7, x7, #32+        lsr     x11, x11, #32+        umaddl  x11, w11, w3, x7+        mov     x7, x4+        umull   x4, w12, w3+        add     x4, x4, w8, uxtw+        lsr     x8, x8, #32+        lsr     x12, x12, #32+        umaddl  x12, w12, w3, x8+        mov     x8, x4+        umull   x4, w13, w3+        add     x4, x4, w9, uxtw+        lsr     x9, x9, #32+        lsr     x13, x13, #32+        umaddl  x13, w13, w3, x9+        mov     x9, x4+        umull   x4, w14, w3+        add     x4, x4, w10, uxtw+        lsr     x10, x10, #32+        lsr     x14, x14, #32+        umaddl  x14, w14, w3, x10+        mov     x10, x4+        lsr     x0, x14, #31+        mov     x5, #0x13+        umaddl  x5, w5, w0, x5+        add     x7, x7, x5+        adds    x7, x7, x11, lsl #32+        extr    x3, x12, x11, #32+        adcs    x8, x8, x3+        extr    x3, x13, x12, #32+        adcs    x9, x9, x3+        extr    x3, x14, x13, #32+        lsl     x5, x0, #63+        eor     x10, x10, x5+        adc     x10, x10, x3+        mov     x3, #0x13+        tst     x10, #0x8000000000000000+        csel    x3, x3, xzr, pl+        subs    x7, x7, x3+        sbcs    x8, x8, xzr+        sbcs    x9, x9, xzr+        sbc     x10, x10, xzr+        and     x10, x10, #0x7fffffffffffffff+        stp     x7, x8, [x17]+        stp     x9, x10, [x17, #16]++// Restore stack and registers (this will zero the tops of Q8...Q15).++        CFI_STACKLOAD2(d8,d9,NSPACE+0)+        CFI_STACKLOAD2(d10,d11,NSPACE+16)+        CFI_STACKLOAD2(d12,d13,NSPACE+32)+        CFI_STACKLOAD2(d14,d15,NSPACE+48)+        CFI_STACKLOAD2(x19,x20,NSPACE+64)+        CFI_STACKLOAD2(x21,x22,NSPACE+80)+        CFI_STACKLOAD2(x23,x24,NSPACE+96)+        CFI_STACKLOAD2(x25,x26,NSPACE+112)+        CFI_STACKLOAD2(x27,x28,NSPACE+128)+        CFI_STACKLOAD2(x29,x30,NSPACE+144)+        CFI_INC_SP((NSPACE+160))+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(curve25519_x25519)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/curve25519_x25519_alt.S view
@@ -0,0 +1,1702 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// The x25519 function for curve25519+// Inputs scalar[4], point[4]; output res[4]+//+// extern void curve25519_x25519_alt+//   (uint64_t res[static 4],const uint64_t scalar[static 4],+//    const uint64_t point[static 4]);+//+// Given a scalar n and the X coordinate of an input point P = (X,Y) on+// curve25519 (Y can live in any extension field of characteristic 2^255-19),+// this returns the X coordinate of n * P = (X, Y), or 0 when n * P is the+// point at infinity. Both n and X inputs are first slightly modified/mangled+// as specified in the relevant RFC (https://www.rfc-editor.org/rfc/rfc7748);+// in particular the lower three bits of n are set to zero. Does not implement+// the zero-check specified in Section 6.1.+//+// Standard ARM ABI: X0 = res, X1 = scalar, X2 = point+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(curve25519_x25519_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(curve25519_x25519_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(curve25519_x25519_alt)++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Stable homes for the input result argument during the whole body+// and other variables that are only needed prior to the modular inverse.++#define res x23+#define i x20+#define swap x21++// Pointers to result x coord to be written++#define resx res, #0++// Pointer-offset pairs for temporaries on stack with some aliasing.++#define scalar sp, #(0*NUMSIZE)++#define pointx sp, #(1*NUMSIZE)++#define zm sp, #(2*NUMSIZE)+#define sm sp, #(2*NUMSIZE)+#define dpro sp, #(2*NUMSIZE)++#define sn sp, #(3*NUMSIZE)++#define dm sp, #(4*NUMSIZE)++#define zn sp, #(5*NUMSIZE)+#define dn sp, #(5*NUMSIZE)+#define e sp, #(5*NUMSIZE)++#define dmsn sp, #(6*NUMSIZE)+#define p sp, #(6*NUMSIZE)++#define xm sp, #(7*NUMSIZE)+#define dnsm sp, #(7*NUMSIZE)+#define spro sp, #(7*NUMSIZE)++#define d sp, #(8*NUMSIZE)++#define xn sp, #(9*NUMSIZE)+#define s sp, #(9*NUMSIZE)++// Total size to reserve on the stack++#define NSPACE 10*NUMSIZE++// Macro wrapping up the basic field operation bignum_mul_p25519_alt, only+// trivially different from a pure function call to that subroutine.++#define mul_p25519(P0,P1,P2)                    \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x7, x8, [P2] __LF                  \+        mul     x12, x3, x7 __LF                   \+        umulh   x13, x3, x7 __LF                   \+        mul     x11, x3, x8 __LF                   \+        umulh   x14, x3, x8 __LF                   \+        adds    x13, x13, x11 __LF                 \+        ldp     x9, x10, [P2+16] __LF              \+        mul     x11, x3, x9 __LF                   \+        umulh   x15, x3, x9 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x3, x10 __LF                  \+        umulh   x16, x3, x10 __LF                  \+        adcs    x15, x15, x11 __LF                 \+        adc     x16, x16, xzr __LF                 \+        ldp     x5, x6, [P1+16] __LF               \+        mul     x11, x4, x7 __LF                   \+        adds    x13, x13, x11 __LF                 \+        mul     x11, x4, x8 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x4, x9 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        mul     x11, x4, x10 __LF                  \+        adcs    x16, x16, x11 __LF                 \+        umulh   x3, x4, x10 __LF                   \+        adc     x3, x3, xzr __LF                   \+        umulh   x11, x4, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        umulh   x11, x4, x8 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        umulh   x11, x4, x9 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        adc     x3, x3, xzr __LF                   \+        mul     x11, x5, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        mul     x11, x5, x8 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        mul     x11, x5, x9 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        mul     x11, x5, x10 __LF                  \+        adcs    x3, x3, x11 __LF                   \+        umulh   x4, x5, x10 __LF                   \+        adc     x4, x4, xzr __LF                   \+        umulh   x11, x5, x7 __LF                   \+        adds    x15, x15, x11 __LF                 \+        umulh   x11, x5, x8 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        umulh   x11, x5, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        adc     x4, x4, xzr __LF                   \+        mul     x11, x6, x7 __LF                   \+        adds    x15, x15, x11 __LF                 \+        mul     x11, x6, x8 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        mul     x11, x6, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        mul     x11, x6, x10 __LF                  \+        adcs    x4, x4, x11 __LF                   \+        umulh   x5, x6, x10 __LF                   \+        adc     x5, x5, xzr __LF                   \+        umulh   x11, x6, x7 __LF                   \+        adds    x16, x16, x11 __LF                 \+        umulh   x11, x6, x8 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        umulh   x11, x6, x9 __LF                   \+        adcs    x4, x4, x11 __LF                   \+        adc     x5, x5, xzr __LF                   \+        mov     x7, #0x26 __LF                     \+        mul     x11, x7, x16 __LF                  \+        umulh   x9, x7, x16 __LF                   \+        adds    x12, x12, x11 __LF                 \+        mul     x11, x7, x3 __LF                   \+        umulh   x3, x7, x3 __LF                    \+        adcs    x13, x13, x11 __LF                 \+        mul     x11, x7, x4 __LF                   \+        umulh   x4, x7, x4 __LF                    \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x7, x5 __LF                   \+        umulh   x5, x7, x5 __LF                    \+        adcs    x15, x15, x11 __LF                 \+        cset    x16, cs __LF                       \+        adds    x15, x15, x4 __LF                  \+        adc     x16, x16, x5 __LF                  \+        cmn     x15, x15 __LF                      \+        orr     x15, x15, #0x8000000000000000 __LF \+        adc     x8, x16, x16 __LF                  \+        mov     x7, #0x13 __LF                     \+        madd    x11, x7, x8, x7 __LF               \+        adds    x12, x12, x11 __LF                 \+        adcs    x13, x13, x9 __LF                  \+        adcs    x14, x14, x3 __LF                  \+        adcs    x15, x15, xzr __LF                 \+        csel    x7, x7, xzr, cc __LF               \+        subs    x12, x12, x7 __LF                  \+        sbcs    x13, x13, xzr __LF                 \+        sbcs    x14, x14, xzr __LF                 \+        sbc     x15, x15, xzr __LF                 \+        and     x15, x15, #0x7fffffffffffffff __LF \+        stp     x12, x13, [P0] __LF                \+        stp     x14, x15, [P0+16]++// A version of multiplication that only guarantees output < 2 * p_25519.+// This basically skips the +1 and final correction in quotient estimation.++#define mul_4(P0,P1,P2)                         \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x7, x8, [P2] __LF                  \+        mul     x12, x3, x7 __LF                   \+        umulh   x13, x3, x7 __LF                   \+        mul     x11, x3, x8 __LF                   \+        umulh   x14, x3, x8 __LF                   \+        adds    x13, x13, x11 __LF                 \+        ldp     x9, x10, [P2+16] __LF              \+        mul     x11, x3, x9 __LF                   \+        umulh   x15, x3, x9 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x3, x10 __LF                  \+        umulh   x16, x3, x10 __LF                  \+        adcs    x15, x15, x11 __LF                 \+        adc     x16, x16, xzr __LF                 \+        ldp     x5, x6, [P1+16] __LF               \+        mul     x11, x4, x7 __LF                   \+        adds    x13, x13, x11 __LF                 \+        mul     x11, x4, x8 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x4, x9 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        mul     x11, x4, x10 __LF                  \+        adcs    x16, x16, x11 __LF                 \+        umulh   x3, x4, x10 __LF                   \+        adc     x3, x3, xzr __LF                   \+        umulh   x11, x4, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        umulh   x11, x4, x8 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        umulh   x11, x4, x9 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        adc     x3, x3, xzr __LF                   \+        mul     x11, x5, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        mul     x11, x5, x8 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        mul     x11, x5, x9 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        mul     x11, x5, x10 __LF                  \+        adcs    x3, x3, x11 __LF                   \+        umulh   x4, x5, x10 __LF                   \+        adc     x4, x4, xzr __LF                   \+        umulh   x11, x5, x7 __LF                   \+        adds    x15, x15, x11 __LF                 \+        umulh   x11, x5, x8 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        umulh   x11, x5, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        adc     x4, x4, xzr __LF                   \+        mul     x11, x6, x7 __LF                   \+        adds    x15, x15, x11 __LF                 \+        mul     x11, x6, x8 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        mul     x11, x6, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        mul     x11, x6, x10 __LF                  \+        adcs    x4, x4, x11 __LF                   \+        umulh   x5, x6, x10 __LF                   \+        adc     x5, x5, xzr __LF                   \+        umulh   x11, x6, x7 __LF                   \+        adds    x16, x16, x11 __LF                 \+        umulh   x11, x6, x8 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        umulh   x11, x6, x9 __LF                   \+        adcs    x4, x4, x11 __LF                   \+        adc     x5, x5, xzr __LF                   \+        mov     x7, #0x26 __LF                     \+        mul     x11, x7, x16 __LF                  \+        umulh   x9, x7, x16 __LF                   \+        adds    x12, x12, x11 __LF                 \+        mul     x11, x7, x3 __LF                   \+        umulh   x3, x7, x3 __LF                    \+        adcs    x13, x13, x11 __LF                 \+        mul     x11, x7, x4 __LF                   \+        umulh   x4, x7, x4 __LF                    \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x7, x5 __LF                   \+        umulh   x5, x7, x5 __LF                    \+        adcs    x15, x15, x11 __LF                 \+        cset    x16, cs __LF                       \+        adds    x15, x15, x4 __LF                  \+        adc     x16, x16, x5 __LF                  \+        cmn     x15, x15 __LF                      \+        bic     x15, x15, #0x8000000000000000 __LF \+        adc     x8, x16, x16 __LF                  \+        mov     x7, #0x13 __LF                     \+        mul     x11, x7, x8 __LF                   \+        adds    x12, x12, x11 __LF                 \+        adcs    x13, x13, x9 __LF                  \+        adcs    x14, x14, x3 __LF                  \+        adc     x15, x15, xzr __LF                 \+        stp     x12, x13, [P0] __LF                \+        stp     x14, x15, [P0+16]++// Squaring just giving a result < 2 * p_25519, which is done by+// basically skipping the +1 in the quotient estimate and the final+// optional correction.++#define sqr_4(P0,P1)                            \+        ldp     x2, x3, [P1] __LF                  \+        mul     x9, x2, x3 __LF                    \+        umulh   x10, x2, x3 __LF                   \+        ldp     x4, x5, [P1+16] __LF               \+        mul     x11, x2, x5 __LF                   \+        umulh   x12, x2, x5 __LF                   \+        mul     x7, x2, x4 __LF                    \+        umulh   x6, x2, x4 __LF                    \+        adds    x10, x10, x7 __LF                  \+        adcs    x11, x11, x6 __LF                  \+        mul     x7, x3, x4 __LF                    \+        umulh   x6, x3, x4 __LF                    \+        adc     x6, x6, xzr __LF                   \+        adds    x11, x11, x7 __LF                  \+        mul     x13, x4, x5 __LF                   \+        umulh   x14, x4, x5 __LF                   \+        adcs    x12, x12, x6 __LF                  \+        mul     x7, x3, x5 __LF                    \+        umulh   x6, x3, x5 __LF                    \+        adc     x6, x6, xzr __LF                   \+        adds    x12, x12, x7 __LF                  \+        adcs    x13, x13, x6 __LF                  \+        adc     x14, x14, xzr __LF                 \+        adds    x9, x9, x9 __LF                    \+        adcs    x10, x10, x10 __LF                 \+        adcs    x11, x11, x11 __LF                 \+        adcs    x12, x12, x12 __LF                 \+        adcs    x13, x13, x13 __LF                 \+        adcs    x14, x14, x14 __LF                 \+        cset    x6, cs __LF                        \+        umulh   x7, x2, x2 __LF                    \+        mul     x8, x2, x2 __LF                    \+        adds    x9, x9, x7 __LF                    \+        mul     x7, x3, x3 __LF                    \+        adcs    x10, x10, x7 __LF                  \+        umulh   x7, x3, x3 __LF                    \+        adcs    x11, x11, x7 __LF                  \+        mul     x7, x4, x4 __LF                    \+        adcs    x12, x12, x7 __LF                  \+        umulh   x7, x4, x4 __LF                    \+        adcs    x13, x13, x7 __LF                  \+        mul     x7, x5, x5 __LF                    \+        adcs    x14, x14, x7 __LF                  \+        umulh   x7, x5, x5 __LF                    \+        adc     x6, x6, x7 __LF                    \+        mov     x3, #0x26 __LF                     \+        mul     x7, x3, x12 __LF                   \+        umulh   x4, x3, x12 __LF                   \+        adds    x8, x8, x7 __LF                    \+        mul     x7, x3, x13 __LF                   \+        umulh   x13, x3, x13 __LF                  \+        adcs    x9, x9, x7 __LF                    \+        mul     x7, x3, x14 __LF                   \+        umulh   x14, x3, x14 __LF                  \+        adcs    x10, x10, x7 __LF                  \+        mul     x7, x3, x6 __LF                    \+        umulh   x6, x3, x6 __LF                    \+        adcs    x11, x11, x7 __LF                  \+        cset    x12, cs __LF                       \+        adds    x11, x11, x14 __LF                 \+        adc     x12, x12, x6 __LF                  \+        cmn     x11, x11 __LF                      \+        bic     x11, x11, #0x8000000000000000 __LF \+        adc     x2, x12, x12 __LF                  \+        mov     x3, #0x13 __LF                     \+        mul     x7, x3, x2 __LF                    \+        adds    x8, x8, x7 __LF                    \+        adcs    x9, x9, x4 __LF                    \+        adcs    x10, x10, x13 __LF                 \+        adc     x11, x11, xzr __LF                 \+        stp     x8, x9, [P0] __LF                  \+        stp     x10, x11, [P0+16]++// Modular addition with double modulus 2 * p_25519 = 2^256 - 38.+// This only ensures that the result fits in 4 digits, not that it is reduced+// even w.r.t. double modulus. The result is always correct modulo provided+// the sum of the inputs is < 2^256 + 2^256 - 38, so in particular provided+// at least one of them is reduced double modulo.++#define add_twice4(P0,P1,P2)                    \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x7, x8, [P2] __LF                  \+        adds    x3, x3, x7 __LF                    \+        adcs    x4, x4, x8 __LF                    \+        ldp     x5, x6, [P1+16] __LF               \+        ldp     x7, x8, [P2+16] __LF               \+        adcs    x5, x5, x7 __LF                    \+        adcs    x6, x6, x8 __LF                    \+        mov     x9, #38 __LF                       \+        csel    x9, x9, xzr, cs __LF               \+        adds    x3, x3, x9 __LF                    \+        adcs    x4, x4, xzr __LF                   \+        adcs    x5, x5, xzr __LF                   \+        adc     x6, x6, xzr __LF                   \+        stp     x3, x4, [P0] __LF                  \+        stp     x5, x6, [P0+16]++// Modular subtraction with double modulus 2 * p_25519 = 2^256 - 38++#define sub_twice4(p0,p1,p2)                    \+        ldp     x5, x6, [p1] __LF                  \+        ldp     x4, x3, [p2] __LF                  \+        subs    x5, x5, x4 __LF                    \+        sbcs    x6, x6, x3 __LF                    \+        ldp     x7, x8, [p1+16] __LF               \+        ldp     x4, x3, [p2+16] __LF               \+        sbcs    x7, x7, x4 __LF                    \+        sbcs    x8, x8, x3 __LF                    \+        mov     x4, #38 __LF                       \+        csel    x3, x4, xzr, lo __LF               \+        subs    x5, x5, x3 __LF                    \+        sbcs    x6, x6, xzr __LF                   \+        sbcs    x7, x7, xzr __LF                   \+        sbc     x8, x8, xzr __LF                   \+        stp     x5, x6, [p0] __LF                  \+        stp     x7, x8, [p0+16]++// Combined z = c * x + y with reduction only < 2 * p_25519+// where c is initially in the X1 register. It is assumed+// that 19 * (c * x + y) < 2^60 * 2^256 so we don't need a+// high mul in the final part.++#define cmadd_4(p0,p2,p3)                       \+        ldp     x7, x8, [p2] __LF                  \+        ldp     x9, x10, [p2+16] __LF              \+        mul     x3, x1, x7 __LF                    \+        mul     x4, x1, x8 __LF                    \+        mul     x5, x1, x9 __LF                    \+        mul     x6, x1, x10 __LF                   \+        umulh   x7, x1, x7 __LF                    \+        umulh   x8, x1, x8 __LF                    \+        umulh   x9, x1, x9 __LF                    \+        umulh   x10, x1, x10 __LF                  \+        adds    x4, x4, x7 __LF                    \+        adcs    x5, x5, x8 __LF                    \+        adcs    x6, x6, x9 __LF                    \+        adc     x10, x10, xzr __LF                 \+        ldp     x7, x8, [p3] __LF                  \+        adds    x3, x3, x7 __LF                    \+        adcs    x4, x4, x8 __LF                    \+        ldp     x7, x8, [p3+16] __LF               \+        adcs    x5, x5, x7 __LF                    \+        adcs    x6, x6, x8 __LF                    \+        adc     x10, x10, xzr __LF                 \+        cmn     x6, x6 __LF                        \+        bic     x6, x6, #0x8000000000000000 __LF   \+        adc     x8, x10, x10 __LF                  \+        mov     x9, #19 __LF                       \+        mul     x7, x8, x9 __LF                    \+        adds    x3, x3, x7 __LF                    \+        adcs    x4, x4, xzr __LF                   \+        adcs    x5, x5, xzr __LF                   \+        adc     x6, x6, xzr __LF                   \+        stp     x3, x4, [p0] __LF                  \+        stp     x5, x6, [p0+16]++// Multiplex: z := if NZ then x else y++#define mux_4(p0,p1,p2)                         \+        ldp     x0, x1, [p1] __LF                  \+        ldp     x2, x3, [p2] __LF                  \+        csel    x0, x0, x2, ne __LF                \+        csel    x1, x1, x3, ne __LF                \+        stp     x0, x1, [p0] __LF                  \+        ldp     x0, x1, [p1+16] __LF               \+        ldp     x2, x3, [p2+16] __LF               \+        csel    x0, x0, x2, ne __LF                \+        csel    x1, x1, x3, ne __LF                \+        stp     x0, x1, [p0+16]++S2N_BN_SYMBOL(curve25519_x25519_alt):+        CFI_START++// Save regs and make room for temporaries++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_DEC_SP(NSPACE)++// Move the output pointer to a stable place++        mov     res, x0++// Copy the inputs to the local variables with minimal mangling:+//+//  - The scalar is in principle turned into 01xxx...xxx000 but+//    in the structure below the special handling of these bits is+//    explicit in the main computation; the scalar is just copied.+//+//  - The point x coord is reduced mod 2^255 by masking off the+//    top bit. In the main loop we only need reduction < 2 * p_25519.++        ldp     x10, x11, [x1]+        stp     x10, x11, [scalar]+        ldp     x12, x13, [x1, #16]+        stp     x12, x13, [scalar+16]++        ldp     x10, x11, [x2]+        stp     x10, x11, [pointx]+        ldp     x12, x13, [x2, #16]+        and     x13, x13, #0x7fffffffffffffff+        stp     x12, x13, [pointx+16]++// Initialize with explicit doubling in order to handle set bit 254.+// Set swap = 1 and (xm,zm) = (x,1) then double as (xn,zn) = 2 * (x,1).+// We use the fact that the point x coordinate is still in registers.+// Since zm = 1 we could do the doubling with an operation count of+// 2 * S + M instead of 2 * S + 2 * M, but it doesn't seem worth+// the slight complication arising from a different linear combination.++        mov     swap, #1+        stp     x10, x11, [xm]+        stp     x12, x13, [xm+16]+        stp     swap, xzr, [zm]+        stp     xzr, xzr, [zm+16]++        sub_twice4(d,xm,zm)+        add_twice4(s,xm,zm)+        sqr_4(d,d)+        sqr_4(s,s)+        sub_twice4(p,s,d)+        mov     x1, 0xdb42+        orr     x1, x1, 0x10000+        cmadd_4(e,p,d)+        mul_4(xn,s,d)+        mul_4(zn,p,e)++// The main loop over unmodified bits from i = 253, ..., i = 3 (inclusive).+// This is a classic Montgomery ladder, with the main coordinates only+// reduced mod 2 * p_25519, some intermediate results even more loosely.++        mov     i, #253++Lcurve25519_x25519_alt_scalarloop:++// sm = xm + zm; sn = xn + zn; dm = xm - zm; dn = xn - zn++        sub_twice4(dm,xm,zm)+        add_twice4(sn,xn,zn)+        sub_twice4(dn,xn,zn)+        add_twice4(sm,xm,zm)++// ADDING: dmsn = dm * sn+// DOUBLING: mux d = xt - zt and s = xt + zt for appropriate choice of (xt,zt)++        mul_4(dmsn,sn,dm)++        lsr     x0, i, #6+        ldr     x2, [sp, x0, lsl #3]    // Exploiting scalar = sp exactly+        lsr     x2, x2, i+        and     x2, x2, #1++        cmp     swap, x2+        mov     swap, x2++        mux_4(d,dm,dn)+        mux_4(s,sm,sn)++// ADDING: dnsm = sm * dn++        mul_4(dnsm,sm,dn)++// DOUBLING: d = (xt - zt)^2++        sqr_4(d,d)++// ADDING: dpro = (dmsn - dnsm)^2, spro = (dmsn + dnsm)^2+// DOUBLING: s = (xt + zt)^2++        sub_twice4(dpro,dmsn,dnsm)+        sqr_4(s,s)+        add_twice4(spro,dmsn,dnsm)+        sqr_4(dpro,dpro)++// DOUBLING: p = 4 * xt * zt = s - d++        sub_twice4(p,s,d)++// ADDING: xm' = (dmsn + dnsm)^2++        sqr_4(xm,spro)++// DOUBLING: e = 121666 * p + d++        mov     x1, 0xdb42+        orr     x1, x1, 0x10000+        cmadd_4(e,p,d)++// DOUBLING: xn' = (xt + zt)^2 * (xt - zt)^2 = s * d++        mul_4(xn,s,d)++// ADDING: zm' = x * (dmsn - dnsm)^2++        mul_4(zm,dpro,pointx)++// DOUBLING: zn' = (4 * xt * zt) * ((xt - zt)^2 + 121666 * (4 * xt * zt))+//               = p * (d + 121666 * p)++        mul_4(zn,p,e)++// Loop down as far as 3 (inclusive)++        sub     i, i, #1+        cmp     i, #3+        bcs     Lcurve25519_x25519_alt_scalarloop++// Multiplex directly into (xn,zn) then do three pure doubling steps;+// this accounts for the implicit zeroing of the three lowest bits+// of the scalar.++        cmp     swap, xzr+        mux_4(xn,xm,xn)+        mux_4(zn,zm,zn)++        sub_twice4(d,xn,zn)+        add_twice4(s,xn,zn)+        sqr_4(d,d)+        sqr_4(s,s)+        sub_twice4(p,s,d)+        mov     x1, 0xdb42+        orr     x1, x1, 0x10000+        cmadd_4(e,p,d)+        mul_4(xn,s,d)+        mul_4(zn,p,e)++        sub_twice4(d,xn,zn)+        add_twice4(s,xn,zn)+        sqr_4(d,d)+        sqr_4(s,s)+        sub_twice4(p,s,d)+        mov     x1, 0xdb42+        orr     x1, x1, 0x10000+        cmadd_4(e,p,d)+        mul_4(xn,s,d)+        mul_4(zn,p,e)++        sub_twice4(d,xn,zn)+        add_twice4(s,xn,zn)+        sqr_4(d,d)+        sqr_4(s,s)+        sub_twice4(p,s,d)+        mov     x1, 0xdb42+        orr     x1, x1, 0x10000+        cmadd_4(e,p,d)+        mul_4(xn,s,d)+        mul_4(zn,p,e)++// The projective result of the scalar multiplication is now (xn,zn).+// Prepare to call the modular inverse function to get zn' = 1/zn++        add     x0, zn+        add     x1, zn++// Inline copy of bignum_inv_p25519, identical except for stripping out+// the prologue and epilogue saving and restoring registers and making+// and reclaiming room on the stack. For more details and explanations see+// "arm/curve25519/bignum_inv_p25519.S". Note that the stack it uses for+// its own temporaries is 128 bytes, so it has no effect on variables+// that are needed in the rest of our computation here: res, xn and zn.++        mov     x20, x0+        mov     x10, #0xffffffffffffffed+        mov     x11, #0xffffffffffffffff+        stp     x10, x11, [sp]+        mov     x12, #0x7fffffffffffffff+        stp     x11, x12, [sp, #16]+        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]+        mov     x7, #0x13+        lsr     x6, x5, #63+        madd    x6, x7, x6, x7+        adds    x2, x2, x6+        adcs    x3, x3, xzr+        adcs    x4, x4, xzr+        orr     x5, x5, #0x8000000000000000+        adcs    x5, x5, xzr+        csel    x6, x7, xzr, cc+        subs    x2, x2, x6+        sbcs    x3, x3, xzr+        sbcs    x4, x4, xzr+        sbc     x5, x5, xzr+        and     x5, x5, #0x7fffffffffffffff+        stp     x2, x3, [sp, #32]+        stp     x4, x5, [sp, #48]+        stp     xzr, xzr, [sp, #64]+        stp     xzr, xzr, [sp, #80]+        mov     x10, #0x2099+        movk    x10, #0x7502, lsl #16+        movk    x10, #0x9e23, lsl #32+        movk    x10, #0xa0f9, lsl #48+        mov     x11, #0x2595+        movk    x11, #0x1d13, lsl #16+        movk    x11, #0x8f3f, lsl #32+        movk    x11, #0xa8c6, lsl #48+        mov     x12, #0x5242+        movk    x12, #0x5ac, lsl #16+        movk    x12, #0x8938, lsl #32+        movk    x12, #0x6c6c, lsl #48+        mov     x13, #0x615+        movk    x13, #0x4177, lsl #16+        movk    x13, #0x8b2, lsl #32+        movk    x13, #0x2765, lsl #48+        stp     x10, x11, [sp, #96]+        stp     x12, x13, [sp, #112]+        mov     x21, #0xa+        mov     x22, #0x1+        b       Lcurve25519_x25519_alt_invmidloop+Lcurve25519_x25519_alt_invloop:+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        and     x0, x12, x16+        and     x1, x13, x17+        add     x19, x0, x1+        ldr     x7, [sp]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #32]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x7, [sp, #8]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #40]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [sp]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [sp, #32]+        ldr     x7, [sp, #16]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #48]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [sp, #8]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [sp, #40]+        ldr     x7, [sp, #24]+        eor     x1, x7, x14+        asr     x3, x1, #63+        and     x3, x3, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #56]+        eor     x1, x8, x15+        asr     x0, x1, #63+        and     x0, x0, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x5, x6, #59+        str     x6, [sp, #16]+        extr    x5, x3, x5, #59+        str     x5, [sp, #24]+        eor     x1, x7, x16+        asr     x5, x1, #63+        and     x5, x5, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        asr     x0, x1, #63+        and     x0, x0, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x4, x2, x4, #59+        str     x4, [sp, #48]+        extr    x2, x5, x2, #59+        str     x2, [sp, #56]+        ldr     x7, [sp, #64]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #96]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #64]+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        str     x5, [sp, #96]+        adc     x3, x3, x1+        ldr     x7, [sp, #72]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #104]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #72]+        adc     x6, x6, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        str     x3, [sp, #104]+        adc     x4, x4, x1+        ldr     x7, [sp, #80]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #112]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #80]+        adc     x5, x5, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        str     x4, [sp, #112]+        adc     x2, x2, x1+        ldr     x7, [sp, #88]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #120]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x3, x5, #63+        ldp     x0, x1, [sp, #64]+        add     x6, x6, x3, asr #63+        mov     x3, #0x13+        mul     x4, x6, x3+        add     x5, x5, x6, lsl #63+        smulh   x3, x6, x3+        ldr     x6, [sp, #80]+        adds    x0, x0, x4+        adcs    x1, x1, x3+        asr     x3, x3, #63+        adcs    x6, x6, x3+        adc     x5, x5, x3+        stp     x0, x1, [sp, #64]+        stp     x6, x5, [sp, #80]+        eor     x1, x7, x16+        and     x5, x16, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        and     x0, x17, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x6, x5, x2, #63+        ldp     x0, x1, [sp, #96]+        add     x6, x6, x5, asr #63+        mov     x5, #0x13+        mul     x4, x6, x5+        add     x2, x2, x6, lsl #63+        smulh   x5, x6, x5+        ldr     x3, [sp, #112]+        adds    x0, x0, x4+        adcs    x1, x1, x5+        asr     x5, x5, #63+        adcs    x3, x3, x5+        adc     x2, x2, x5+        stp     x0, x1, [sp, #96]+        stp     x3, x2, [sp, #112]+Lcurve25519_x25519_alt_invmidloop:+        mov     x1, x22+        ldr     x2, [sp]+        ldr     x3, [sp, #32]+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x8, x4, #0x100, lsl #12+        sbfx    x8, x8, #21, #21+        mov     x11, #0x100000+        add     x11, x11, x11, lsl #21+        add     x9, x4, x11+        asr     x9, x9, #42+        add     x10, x5, #0x100, lsl #12+        sbfx    x10, x10, #21, #21+        add     x11, x5, x11+        asr     x11, x11, #42+        mul     x6, x8, x2+        mul     x7, x9, x3+        mul     x2, x10, x2+        mul     x3, x11, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #21, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #42+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #21, #21+        add     x15, x5, x15+        asr     x15, x15, #42+        mul     x6, x12, x2+        mul     x7, x13, x3+        mul     x2, x14, x2+        mul     x3, x15, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        mul     x2, x12, x8+        mul     x3, x12, x9+        mul     x6, x14, x8+        mul     x7, x14, x9+        madd    x8, x13, x10, x2+        madd    x9, x13, x11, x3+        madd    x16, x15, x10, x6+        madd    x17, x15, x11, x7+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #22, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #43+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #22, #21+        add     x15, x5, x15+        asr     x15, x15, #43+        mneg    x2, x12, x8+        mneg    x3, x12, x9+        mneg    x4, x14, x8+        mneg    x5, x14, x9+        msub    x10, x13, x16, x2+        msub    x11, x13, x17, x3+        msub    x12, x15, x16, x4+        msub    x13, x15, x17, x5+        mov     x22, x1+        subs    x21, x21, #0x1+        b.ne    Lcurve25519_x25519_alt_invloop+        ldr     x0, [sp]+        ldr     x1, [sp, #32]+        mul     x0, x0, x10+        madd    x1, x1, x11, x0+        asr     x0, x1, #63+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        eor     x14, x14, x0+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        eor     x15, x15, x0+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        eor     x16, x16, x0+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        eor     x17, x17, x0+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        ldr     x7, [sp, #64]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #96]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #64]+        adc     x2, x2, x1+        ldr     x7, [sp, #72]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #104]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #72]+        adc     x6, x6, x1+        ldr     x7, [sp, #80]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #112]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #80]+        adc     x5, x5, x1+        ldr     x7, [sp, #88]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #120]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x3, x5, #63+        ldp     x0, x1, [sp, #64]+        tst     x3, x3+        cinc    x6, x6, pl+        mov     x3, #0x13+        mul     x4, x6, x3+        add     x5, x5, x6, lsl #63+        smulh   x6, x6, x3+        ldr     x2, [sp, #80]+        adds    x0, x0, x4+        adcs    x1, x1, x6+        asr     x6, x6, #63+        adcs    x2, x2, x6+        adcs    x5, x5, x6+        csel    x3, x3, xzr, mi+        subs    x0, x0, x3+        sbcs    x1, x1, xzr+        sbcs    x2, x2, xzr+        sbc     x5, x5, xzr+        and     x5, x5, #0x7fffffffffffffff+        mov     x4, x20+        stp     x0, x1, [x4]+        stp     x2, x5, [x4, #16]++// Now the result is xn * (1/zn), fully reduced modulo p.+// Note that in the degenerate case zn = 0 (mod p_25519), the+// modular inverse code above will produce 1/zn = 0, giving+// the correct overall X25519 result of zero for the point at+// infinity.++        mul_p25519(resx,xn,zn)++// Restore stack and registers++        CFI_INC_SP(NSPACE)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(curve25519_x25519_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/curve25519_x25519base.S view
@@ -0,0 +1,9591 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// The x25519 function for curve25519 on base element 9+// Input scalar[4]; output res[4]+//+// extern void curve25519_x25519base+//   (uint64_t res[static 4],const uint64_t scalar[static 4]);+//+// Given a scalar n, returns the X coordinate of n * G where G = (9,...) is+// the standard generator. The scalar is first slightly modified/mangled+// as specified in the relevant RFC (https://www.rfc-editor.org/rfc/rfc7748).+//+// Standard ARM ABI: X0 = res, X1 = scalar+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(curve25519_x25519base)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(curve25519_x25519base)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(curve25519_x25519base)++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Stable home for the input result argument during the whole body++#define res x23++// Other variables that are only needed prior to the modular inverse.++#define tab x19++#define i x20++#define bias x21++#define bf x22+#define ix x22++// Pointer-offset pairs for result and temporaries on stack with some aliasing.++#define resx res, #(0*NUMSIZE)++#define scalar sp, #(0*NUMSIZE)++#define tabent sp, #(1*NUMSIZE)+#define ymx_2 sp, #(1*NUMSIZE)+#define xpy_2 sp, #(2*NUMSIZE)+#define kxy_2 sp, #(3*NUMSIZE)++#define acc sp, #(4*NUMSIZE)+#define x_1 sp, #(4*NUMSIZE)+#define y_1 sp, #(5*NUMSIZE)+#define z_1 sp, #(6*NUMSIZE)+#define w_1 sp, #(7*NUMSIZE)+#define x_3 sp, #(4*NUMSIZE)+#define y_3 sp, #(5*NUMSIZE)+#define z_3 sp, #(6*NUMSIZE)+#define w_3 sp, #(7*NUMSIZE)++#define tmpspace sp, #(8*NUMSIZE)+#define t0 sp, #(8*NUMSIZE)+#define t1 sp, #(9*NUMSIZE)+#define t2 sp, #(10*NUMSIZE)+#define t3 sp, #(11*NUMSIZE)+#define t4 sp, #(12*NUMSIZE)+#define t5 sp, #(13*NUMSIZE)++// Total size to reserve on the stack++#define NSPACE 14*NUMSIZE++// Macro wrapping up the basic field operation bignum_mul_p25519, only+// trivially different from a pure function call to that subroutine.++#define mul_p25519(P0,P1,P2)                    \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x5, x6, [P2] __LF                  \+        umull   x7, w3, w5 __LF                    \+        lsr     x0, x3, #32 __LF                   \+        umull   x15, w0, w5 __LF                   \+        lsr     x16, x5, #32 __LF                  \+        umull   x8, w16, w0 __LF                   \+        umull   x16, w3, w16 __LF                  \+        adds    x7, x7, x15, lsl #32 __LF          \+        lsr     x15, x15, #32 __LF                 \+        adc     x8, x8, x15 __LF                   \+        adds    x7, x7, x16, lsl #32 __LF          \+        lsr     x16, x16, #32 __LF                 \+        adc     x8, x8, x16 __LF                   \+        mul     x9, x4, x6 __LF                    \+        umulh   x10, x4, x6 __LF                   \+        subs    x4, x4, x3 __LF                    \+        cneg    x4, x4, cc __LF                    \+        csetm   x16, cc __LF                       \+        adds    x9, x9, x8 __LF                    \+        adc     x10, x10, xzr __LF                 \+        subs    x3, x5, x6 __LF                    \+        cneg    x3, x3, cc __LF                    \+        cinv    x16, x16, cc __LF                  \+        mul     x15, x4, x3 __LF                   \+        umulh   x3, x4, x3 __LF                    \+        adds    x8, x7, x9 __LF                    \+        adcs    x9, x9, x10 __LF                   \+        adc     x10, x10, xzr __LF                 \+        cmn     x16, #0x1 __LF                     \+        eor     x15, x15, x16 __LF                 \+        adcs    x8, x15, x8 __LF                   \+        eor     x3, x3, x16 __LF                   \+        adcs    x9, x3, x9 __LF                    \+        adc     x10, x10, x16 __LF                 \+        ldp     x3, x4, [P1+16] __LF               \+        ldp     x5, x6, [P2+16] __LF               \+        umull   x11, w3, w5 __LF                   \+        lsr     x0, x3, #32 __LF                   \+        umull   x15, w0, w5 __LF                   \+        lsr     x16, x5, #32 __LF                  \+        umull   x12, w16, w0 __LF                  \+        umull   x16, w3, w16 __LF                  \+        adds    x11, x11, x15, lsl #32 __LF        \+        lsr     x15, x15, #32 __LF                 \+        adc     x12, x12, x15 __LF                 \+        adds    x11, x11, x16, lsl #32 __LF        \+        lsr     x16, x16, #32 __LF                 \+        adc     x12, x12, x16 __LF                 \+        mul     x13, x4, x6 __LF                   \+        umulh   x14, x4, x6 __LF                   \+        subs    x4, x4, x3 __LF                    \+        cneg    x4, x4, cc __LF                    \+        csetm   x16, cc __LF                       \+        adds    x13, x13, x12 __LF                 \+        adc     x14, x14, xzr __LF                 \+        subs    x3, x5, x6 __LF                    \+        cneg    x3, x3, cc __LF                    \+        cinv    x16, x16, cc __LF                  \+        mul     x15, x4, x3 __LF                   \+        umulh   x3, x4, x3 __LF                    \+        adds    x12, x11, x13 __LF                 \+        adcs    x13, x13, x14 __LF                 \+        adc     x14, x14, xzr __LF                 \+        cmn     x16, #0x1 __LF                     \+        eor     x15, x15, x16 __LF                 \+        adcs    x12, x15, x12 __LF                 \+        eor     x3, x3, x16 __LF                   \+        adcs    x13, x3, x13 __LF                  \+        adc     x14, x14, x16 __LF                 \+        ldp     x3, x4, [P1+16] __LF               \+        ldp     x15, x16, [P1] __LF                \+        subs    x3, x3, x15 __LF                   \+        sbcs    x4, x4, x16 __LF                   \+        csetm   x16, cc __LF                       \+        ldp     x15, x0, [P2] __LF                 \+        subs    x5, x15, x5 __LF                   \+        sbcs    x6, x0, x6 __LF                    \+        csetm   x0, cc __LF                        \+        eor     x3, x3, x16 __LF                   \+        subs    x3, x3, x16 __LF                   \+        eor     x4, x4, x16 __LF                   \+        sbc     x4, x4, x16 __LF                   \+        eor     x5, x5, x0 __LF                    \+        subs    x5, x5, x0 __LF                    \+        eor     x6, x6, x0 __LF                    \+        sbc     x6, x6, x0 __LF                    \+        eor     x16, x0, x16 __LF                  \+        adds    x11, x11, x9 __LF                  \+        adcs    x12, x12, x10 __LF                 \+        adcs    x13, x13, xzr __LF                 \+        adc     x14, x14, xzr __LF                 \+        mul     x2, x3, x5 __LF                    \+        umulh   x0, x3, x5 __LF                    \+        mul     x15, x4, x6 __LF                   \+        umulh   x1, x4, x6 __LF                    \+        subs    x4, x4, x3 __LF                    \+        cneg    x4, x4, cc __LF                    \+        csetm   x9, cc __LF                        \+        adds    x15, x15, x0 __LF                  \+        adc     x1, x1, xzr __LF                   \+        subs    x6, x5, x6 __LF                    \+        cneg    x6, x6, cc __LF                    \+        cinv    x9, x9, cc __LF                    \+        mul     x5, x4, x6 __LF                    \+        umulh   x6, x4, x6 __LF                    \+        adds    x0, x2, x15 __LF                   \+        adcs    x15, x15, x1 __LF                  \+        adc     x1, x1, xzr __LF                   \+        cmn     x9, #0x1 __LF                      \+        eor     x5, x5, x9 __LF                    \+        adcs    x0, x5, x0 __LF                    \+        eor     x6, x6, x9 __LF                    \+        adcs    x15, x6, x15 __LF                  \+        adc     x1, x1, x9 __LF                    \+        adds    x9, x11, x7 __LF                   \+        adcs    x10, x12, x8 __LF                  \+        adcs    x11, x13, x11 __LF                 \+        adcs    x12, x14, x12 __LF                 \+        adcs    x13, x13, xzr __LF                 \+        adc     x14, x14, xzr __LF                 \+        cmn     x16, #0x1 __LF                     \+        eor     x2, x2, x16 __LF                   \+        adcs    x9, x2, x9 __LF                    \+        eor     x0, x0, x16 __LF                   \+        adcs    x10, x0, x10 __LF                  \+        eor     x15, x15, x16 __LF                 \+        adcs    x11, x15, x11 __LF                 \+        eor     x1, x1, x16 __LF                   \+        adcs    x12, x1, x12 __LF                  \+        adcs    x13, x13, x16 __LF                 \+        adc     x14, x14, x16 __LF                 \+        mov     x3, #0x26 __LF                     \+        umull   x4, w11, w3 __LF                   \+        add     x4, x4, w7, uxtw __LF              \+        lsr     x7, x7, #32 __LF                   \+        lsr     x11, x11, #32 __LF                 \+        umaddl  x11, w11, w3, x7 __LF              \+        mov     x7, x4 __LF                        \+        umull   x4, w12, w3 __LF                   \+        add     x4, x4, w8, uxtw __LF              \+        lsr     x8, x8, #32 __LF                   \+        lsr     x12, x12, #32 __LF                 \+        umaddl  x12, w12, w3, x8 __LF              \+        mov     x8, x4 __LF                        \+        umull   x4, w13, w3 __LF                   \+        add     x4, x4, w9, uxtw __LF              \+        lsr     x9, x9, #32 __LF                   \+        lsr     x13, x13, #32 __LF                 \+        umaddl  x13, w13, w3, x9 __LF              \+        mov     x9, x4 __LF                        \+        umull   x4, w14, w3 __LF                   \+        add     x4, x4, w10, uxtw __LF             \+        lsr     x10, x10, #32 __LF                 \+        lsr     x14, x14, #32 __LF                 \+        umaddl  x14, w14, w3, x10 __LF             \+        mov     x10, x4 __LF                       \+        lsr     x0, x14, #31 __LF                  \+        mov     x5, #0x13 __LF                     \+        umaddl  x5, w5, w0, x5 __LF                \+        add     x7, x7, x5 __LF                    \+        adds    x7, x7, x11, lsl #32 __LF          \+        extr    x3, x12, x11, #32 __LF             \+        adcs    x8, x8, x3 __LF                    \+        extr    x3, x13, x12, #32 __LF             \+        adcs    x9, x9, x3 __LF                    \+        extr    x3, x14, x13, #32 __LF             \+        lsl     x5, x0, #63 __LF                   \+        eor     x10, x10, x5 __LF                  \+        adc     x10, x10, x3 __LF                  \+        mov     x3, #0x13 __LF                     \+        tst     x10, #0x8000000000000000 __LF      \+        csel    x3, x3, xzr, pl __LF               \+        subs    x7, x7, x3 __LF                    \+        sbcs    x8, x8, xzr __LF                   \+        sbcs    x9, x9, xzr __LF                   \+        sbc     x10, x10, xzr __LF                 \+        and     x10, x10, #0x7fffffffffffffff __LF \+        stp     x7, x8, [P0] __LF                  \+        stp     x9, x10, [P0+16]++// A version of multiplication that only guarantees output < 2 * p_25519.+// This basically skips the +1 and final correction in quotient estimation.++#define mul_4(P0,P1,P2)                         \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x5, x6, [P2] __LF                  \+        umull   x7, w3, w5 __LF                    \+        lsr     x0, x3, #32 __LF                   \+        umull   x15, w0, w5 __LF                   \+        lsr     x16, x5, #32 __LF                  \+        umull   x8, w16, w0 __LF                   \+        umull   x16, w3, w16 __LF                  \+        adds    x7, x7, x15, lsl #32 __LF          \+        lsr     x15, x15, #32 __LF                 \+        adc     x8, x8, x15 __LF                   \+        adds    x7, x7, x16, lsl #32 __LF          \+        lsr     x16, x16, #32 __LF                 \+        adc     x8, x8, x16 __LF                   \+        mul     x9, x4, x6 __LF                    \+        umulh   x10, x4, x6 __LF                   \+        subs    x4, x4, x3 __LF                    \+        cneg    x4, x4, cc __LF                    \+        csetm   x16, cc __LF                       \+        adds    x9, x9, x8 __LF                    \+        adc     x10, x10, xzr __LF                 \+        subs    x3, x5, x6 __LF                    \+        cneg    x3, x3, cc __LF                    \+        cinv    x16, x16, cc __LF                  \+        mul     x15, x4, x3 __LF                   \+        umulh   x3, x4, x3 __LF                    \+        adds    x8, x7, x9 __LF                    \+        adcs    x9, x9, x10 __LF                   \+        adc     x10, x10, xzr __LF                 \+        cmn     x16, #0x1 __LF                     \+        eor     x15, x15, x16 __LF                 \+        adcs    x8, x15, x8 __LF                   \+        eor     x3, x3, x16 __LF                   \+        adcs    x9, x3, x9 __LF                    \+        adc     x10, x10, x16 __LF                 \+        ldp     x3, x4, [P1+16] __LF               \+        ldp     x5, x6, [P2+16] __LF               \+        umull   x11, w3, w5 __LF                   \+        lsr     x0, x3, #32 __LF                   \+        umull   x15, w0, w5 __LF                   \+        lsr     x16, x5, #32 __LF                  \+        umull   x12, w16, w0 __LF                  \+        umull   x16, w3, w16 __LF                  \+        adds    x11, x11, x15, lsl #32 __LF        \+        lsr     x15, x15, #32 __LF                 \+        adc     x12, x12, x15 __LF                 \+        adds    x11, x11, x16, lsl #32 __LF        \+        lsr     x16, x16, #32 __LF                 \+        adc     x12, x12, x16 __LF                 \+        mul     x13, x4, x6 __LF                   \+        umulh   x14, x4, x6 __LF                   \+        subs    x4, x4, x3 __LF                    \+        cneg    x4, x4, cc __LF                    \+        csetm   x16, cc __LF                       \+        adds    x13, x13, x12 __LF                 \+        adc     x14, x14, xzr __LF                 \+        subs    x3, x5, x6 __LF                    \+        cneg    x3, x3, cc __LF                    \+        cinv    x16, x16, cc __LF                  \+        mul     x15, x4, x3 __LF                   \+        umulh   x3, x4, x3 __LF                    \+        adds    x12, x11, x13 __LF                 \+        adcs    x13, x13, x14 __LF                 \+        adc     x14, x14, xzr __LF                 \+        cmn     x16, #0x1 __LF                     \+        eor     x15, x15, x16 __LF                 \+        adcs    x12, x15, x12 __LF                 \+        eor     x3, x3, x16 __LF                   \+        adcs    x13, x3, x13 __LF                  \+        adc     x14, x14, x16 __LF                 \+        ldp     x3, x4, [P1+16] __LF               \+        ldp     x15, x16, [P1] __LF                \+        subs    x3, x3, x15 __LF                   \+        sbcs    x4, x4, x16 __LF                   \+        csetm   x16, cc __LF                       \+        ldp     x15, x0, [P2] __LF                 \+        subs    x5, x15, x5 __LF                   \+        sbcs    x6, x0, x6 __LF                    \+        csetm   x0, cc __LF                        \+        eor     x3, x3, x16 __LF                   \+        subs    x3, x3, x16 __LF                   \+        eor     x4, x4, x16 __LF                   \+        sbc     x4, x4, x16 __LF                   \+        eor     x5, x5, x0 __LF                    \+        subs    x5, x5, x0 __LF                    \+        eor     x6, x6, x0 __LF                    \+        sbc     x6, x6, x0 __LF                    \+        eor     x16, x0, x16 __LF                  \+        adds    x11, x11, x9 __LF                  \+        adcs    x12, x12, x10 __LF                 \+        adcs    x13, x13, xzr __LF                 \+        adc     x14, x14, xzr __LF                 \+        mul     x2, x3, x5 __LF                    \+        umulh   x0, x3, x5 __LF                    \+        mul     x15, x4, x6 __LF                   \+        umulh   x1, x4, x6 __LF                    \+        subs    x4, x4, x3 __LF                    \+        cneg    x4, x4, cc __LF                    \+        csetm   x9, cc __LF                        \+        adds    x15, x15, x0 __LF                  \+        adc     x1, x1, xzr __LF                   \+        subs    x6, x5, x6 __LF                    \+        cneg    x6, x6, cc __LF                    \+        cinv    x9, x9, cc __LF                    \+        mul     x5, x4, x6 __LF                    \+        umulh   x6, x4, x6 __LF                    \+        adds    x0, x2, x15 __LF                   \+        adcs    x15, x15, x1 __LF                  \+        adc     x1, x1, xzr __LF                   \+        cmn     x9, #0x1 __LF                      \+        eor     x5, x5, x9 __LF                    \+        adcs    x0, x5, x0 __LF                    \+        eor     x6, x6, x9 __LF                    \+        adcs    x15, x6, x15 __LF                  \+        adc     x1, x1, x9 __LF                    \+        adds    x9, x11, x7 __LF                   \+        adcs    x10, x12, x8 __LF                  \+        adcs    x11, x13, x11 __LF                 \+        adcs    x12, x14, x12 __LF                 \+        adcs    x13, x13, xzr __LF                 \+        adc     x14, x14, xzr __LF                 \+        cmn     x16, #0x1 __LF                     \+        eor     x2, x2, x16 __LF                   \+        adcs    x9, x2, x9 __LF                    \+        eor     x0, x0, x16 __LF                   \+        adcs    x10, x0, x10 __LF                  \+        eor     x15, x15, x16 __LF                 \+        adcs    x11, x15, x11 __LF                 \+        eor     x1, x1, x16 __LF                   \+        adcs    x12, x1, x12 __LF                  \+        adcs    x13, x13, x16 __LF                 \+        adc     x14, x14, x16 __LF                 \+        mov     x3, #0x26 __LF                     \+        umull   x4, w11, w3 __LF                   \+        add     x4, x4, w7, uxtw __LF              \+        lsr     x7, x7, #32 __LF                   \+        lsr     x11, x11, #32 __LF                 \+        umaddl  x11, w11, w3, x7 __LF              \+        mov     x7, x4 __LF                        \+        umull   x4, w12, w3 __LF                   \+        add     x4, x4, w8, uxtw __LF              \+        lsr     x8, x8, #32 __LF                   \+        lsr     x12, x12, #32 __LF                 \+        umaddl  x12, w12, w3, x8 __LF              \+        mov     x8, x4 __LF                        \+        umull   x4, w13, w3 __LF                   \+        add     x4, x4, w9, uxtw __LF              \+        lsr     x9, x9, #32 __LF                   \+        lsr     x13, x13, #32 __LF                 \+        umaddl  x13, w13, w3, x9 __LF              \+        mov     x9, x4 __LF                        \+        umull   x4, w14, w3 __LF                   \+        add     x4, x4, w10, uxtw __LF             \+        lsr     x10, x10, #32 __LF                 \+        lsr     x14, x14, #32 __LF                 \+        umaddl  x14, w14, w3, x10 __LF             \+        mov     x10, x4 __LF                       \+        lsr     x0, x14, #31 __LF                  \+        mov     x5, #0x13 __LF                     \+        umull   x5, w5, w0 __LF                    \+        add     x7, x7, x5 __LF                    \+        adds    x7, x7, x11, lsl #32 __LF          \+        extr    x3, x12, x11, #32 __LF             \+        adcs    x8, x8, x3 __LF                    \+        extr    x3, x13, x12, #32 __LF             \+        adcs    x9, x9, x3 __LF                    \+        extr    x3, x14, x13, #32 __LF             \+        lsl     x5, x0, #63 __LF                   \+        eor     x10, x10, x5 __LF                  \+        adc     x10, x10, x3 __LF                  \+        stp     x7, x8, [P0] __LF                  \+        stp     x9, x10, [P0+16]++// Modular subtraction with double modulus 2 * p_25519 = 2^256 - 38++#define sub_twice4(P0,P1,P2)                    \+        ldp     x5, x6, [P1] __LF                  \+        ldp     x4, x3, [P2] __LF                  \+        subs    x5, x5, x4 __LF                    \+        sbcs    x6, x6, x3 __LF                    \+        ldp     x7, x8, [P1+16] __LF               \+        ldp     x4, x3, [P2+16] __LF               \+        sbcs    x7, x7, x4 __LF                    \+        sbcs    x8, x8, x3 __LF                    \+        mov     x4, #38 __LF                       \+        csel    x3, x4, xzr, lo __LF               \+        subs    x5, x5, x3 __LF                    \+        sbcs    x6, x6, xzr __LF                   \+        sbcs    x7, x7, xzr __LF                   \+        sbc     x8, x8, xzr __LF                   \+        stp     x5, x6, [P0] __LF                  \+        stp     x7, x8, [P0+16]++// Modular addition and doubling with double modulus 2 * p_25519 = 2^256 - 38.+// This only ensures that the result fits in 4 digits, not that it is reduced+// even w.r.t. double modulus. The result is always correct modulo provided+// the sum of the inputs is < 2^256 + 2^256 - 38, so in particular provided+// at least one of them is reduced double modulo.++#define add_twice4(P0,P1,P2)                    \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x7, x8, [P2] __LF                  \+        adds    x3, x3, x7 __LF                    \+        adcs    x4, x4, x8 __LF                    \+        ldp     x5, x6, [P1+16] __LF               \+        ldp     x7, x8, [P2+16] __LF               \+        adcs    x5, x5, x7 __LF                    \+        adcs    x6, x6, x8 __LF                    \+        mov     x9, #38 __LF                       \+        csel    x9, x9, xzr, cs __LF               \+        adds    x3, x3, x9 __LF                    \+        adcs    x4, x4, xzr __LF                   \+        adcs    x5, x5, xzr __LF                   \+        adc     x6, x6, xzr __LF                   \+        stp     x3, x4, [P0] __LF                  \+        stp     x5, x6, [P0+16]++#define double_twice4(P0,P1)                    \+        ldp     x3, x4, [P1] __LF                  \+        adds    x3, x3, x3 __LF                    \+        adcs    x4, x4, x4 __LF                    \+        ldp     x5, x6, [P1+16] __LF               \+        adcs    x5, x5, x5 __LF                    \+        adcs    x6, x6, x6 __LF                    \+        mov     x9, #38 __LF                       \+        csel    x9, x9, xzr, cs __LF               \+        adds    x3, x3, x9 __LF                    \+        adcs    x4, x4, xzr __LF                   \+        adcs    x5, x5, xzr __LF                   \+        adc     x6, x6, xzr __LF                   \+        stp     x3, x4, [P0] __LF                  \+        stp     x5, x6, [P0+16]++S2N_BN_SYMBOL(curve25519_x25519base):+        CFI_START++// Save regs and make room for temporaries++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_DEC_SP(NSPACE)++// Move the output pointer to a stable place++        mov     res, x0++// Copy the input scalar to its local variable while mangling it.+// In principle the mangling is into 01xxx...xxx000, but actually+// we only clear the top two bits so 00xxx...xxxxxx. The additional+// 2^254 * G is taken care of by the starting value for the addition+// chain below, while we never look at the three low bits at all.++        ldp     x10, x11, [x1]+        stp     x10, x11, [scalar]+        ldp     x12, x13, [x1, #16]+        bic     x13, x13, #0xc000000000000000+        stp     x12, x13, [scalar+16]++// The main part of the computation is on the edwards25519 curve in+// extended-projective coordinates (X,Y,Z,T), representing a point+// (x,y) via x = X/Z, y = Y/Z and x * y = T/Z (so X * Y = T * Z).+// Only at the very end do we translate back to curve25519. So G+// below means the generator within edwards25519 corresponding to+// (9,...) for curve25519, via the standard isomorphism.+//+// Initialize accumulator "acc" to either (2^254 + 8) * G or just 2^254 * G+// depending on bit 3 of the scalar, the only nonzero bit of the bottom 4.+// Thus, we have effectively dealt with bits 0, 1, 2, 3, 254 and 255.++        ldr     x0, [scalar]+        ands    xzr, x0, #8++#if defined(__ELF__)+        adrp    tab, S2N_BN_SYMBOL(curve25519_x25519base_constant)+        add     tab, tab, :lo12:S2N_BN_SYMBOL(curve25519_x25519base_constant)+#else+        adrp    tab, S2N_BN_SYMBOL(curve25519_x25519base_constant)@PAGE+        add     tab, tab, S2N_BN_SYMBOL(curve25519_x25519base_constant)@PAGEOFF+#endif++        ldp     x0, x1, [tab]+        ldp     x2, x3, [tab, #96]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc]++        ldp     x0, x1, [tab, #1*16]+        ldp     x2, x3, [tab, #96+1*16]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+1*16]++        ldp     x0, x1, [tab, #2*16]+        ldp     x2, x3, [tab, #96+2*16]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+2*16]++        ldp     x0, x1, [tab, #3*16]+        ldp     x2, x3, [tab, #96+3*16]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+3*16]++        mov     x0, #1+        stp     x0, xzr, [acc+4*16]+        stp     xzr, xzr, [acc+5*16]++        ldp     x0, x1, [tab, #4*16]+        ldp     x2, x3, [tab, #96+4*16]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+6*16]++        ldp     x0, x1, [tab, #5*16]+        ldp     x2, x3, [tab, #96+5*16]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+7*16]++// The counter "i" tracks the bit position for which the scalar has+// already been absorbed, starting at 4 and going up in chunks of 4.+//+// The pointer "tab" points at the current block of the table for+// multiples (2^i * j) * G at the current bit position i; 1 <= j <= 8.+//+// The bias is always either 0 and 1 and needs to be added to the+// partially processed scalar implicitly. This is used to absorb 4 bits+// of scalar per iteration from 3-bit table indexing by exploiting+// negation: (16 * h + l) * G = (16 * (h + 1) - (16 - l)) * G is used+// when l >= 9. Note that we can't have any bias left over at the+// end because of the clearing of bit 255 of the scalar, meaning the+// l >= 9 case cannot arise on the last iteration.++        mov     i, 4+        add     tab, tab, #192+        mov     bias, xzr++// Start of the main loop, repeated 63 times for i = 4, 8, ..., 252++Lcurve25519_x25519base_scalarloop:++// Look at the next 4-bit field "bf", adding the previous bias as well.+// Choose the table index "ix" as bf when bf <= 8 and 16 - bf for bf >= 9,+// setting the bias to 1 for the next iteration in the latter case.++        lsr     x0, i, #6+        ldr     x2, [sp, x0, lsl #3]    // Exploiting scalar = sp exactly+        lsr     x2, x2, i+        and     x2, x2, #15+        add     bf, x2, bias++        cmp     bf, 9+        cset    bias, cs++        mov     x0, 16+        sub     x0, x0, bf+        cmp     bias, xzr+        csel    ix, x0, bf, ne++// Perform constant-time lookup in the table to get element number "ix".+// The table entry for the affine point (x,y) is actually a triple+// (y - x,x + y,2 * d * x * y) to precompute parts of the addition.+// Note that "ix" can be 0, so we set up the appropriate identity first.++        mov     x0, #1+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, #1+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        mov     x9, xzr+        mov     x10, xzr+        mov     x11, xzr++        cmp     ix, #1+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #2+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #3+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #4+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #5+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #6+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #7+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #8+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++// We now have the triple from the table in registers as follows+//+//      [x3;x2;x1;x0] = y - x+//      [x7;x6;x5;x4] = x + y+//      [x11;x10;x9;x8] = 2 * d * x * y+//+// In case bias = 1 we need to negate this. For Edwards curves+// -(x,y) = (-x,y), i.e. we need to negate the x coordinate.+// In this processed encoding, that amounts to swapping the+// first two fields and negating the third.+//+// The optional negation here also pretends bias = 0 whenever+// ix = 0 so that it doesn't need to handle the case of zero+// inputs, since no non-trivial table entries are zero. Note+// that in the zero case the whole negation is trivial, and+// so indeed is the swapping.++        cmp     bias, #0++        csel    x12, x0, x4, eq+        csel    x13, x1, x5, eq+        csel    x14, x2, x6, eq+        csel    x15, x3, x7, eq+        stp     x12, x13, [tabent]+        stp     x14, x15, [tabent+16]++        csel    x12, x0, x4, ne+        csel    x13, x1, x5, ne+        csel    x14, x2, x6, ne+        csel    x15, x3, x7, ne+        stp     x12, x13, [tabent+32]+        stp     x14, x15, [tabent+48]++        mov     x0, #-19+        subs    x0, x0, x8+        mov     x2, #-1+        sbcs    x1, x2, x9+        sbcs    x2, x2, x10+        mov     x3, #0x7FFFFFFFFFFFFFFF+        sbc     x3, x3, x11++        cmp     ix, xzr+        ccmp    bias, xzr, #4, ne++        csel    x0, x0, x8, ne+        csel    x1, x1, x9, ne+        stp     x0, x1, [tabent+64]+        csel    x2, x2, x10, ne+        csel    x3, x3, x11, ne+        stp     x2, x3, [tabent+80]++// Extended-projective and precomputed mixed addition.+// This is effectively the same as calling the standalone+// function edwards25519_pepadd(acc,acc,tabent), but we+// only retain slightly weaker normalization < 2 * p_25519+// throughout the inner loop, so the computation is+// slightly different, and faster overall.++        double_twice4(t0,z_1)+        sub_twice4(t1,y_1,x_1)+        add_twice4(t2,y_1,x_1)+        mul_4(t3,w_1,kxy_2)+        mul_4(t1,t1,ymx_2)+        mul_4(t2,t2,xpy_2)+        sub_twice4(t4,t0,t3)+        add_twice4(t0,t0,t3)+        sub_twice4(t5,t2,t1)+        add_twice4(t1,t2,t1)+        mul_4(z_3,t4,t0)+        mul_4(x_3,t5,t4)+        mul_4(y_3,t0,t1)+        mul_4(w_3,t5,t1)++// End of the main loop; move on by 4 bits.++        add     i, i, 4+        cmp     i, 256+        bcc     Lcurve25519_x25519base_scalarloop++// Now we need to translate from Edwards curve edwards25519 back+// to the Montgomery form curve25519. The mapping in the affine+// representations is+//+// (x,y) |-> ((1 + y) / (1 - y), c * (1 + y) / ((1 - y) * x))+//+// For x25519, we only need the x coordinate, and we compute this as+//+// (1 + y) / (1 - y) = (x + x * y) / (x - x * y)+//                   = (X/Z + T/Z) / (X/Z - T/Z)+//                   = (X + T) / (X - T)+//                   = (X + T) * inverse(X - T)+//+// We could equally well use (Z + Y) / (Z - Y), but the above has the+// same cost, and it more explicitly forces zero output whenever X = 0,+// regardless of how the modular inverse behaves on zero inputs. In+// the present setting (base point 9, mangled scalar) that doesn't+// really matter anyway since X = 0 never arises, but it seems a+// little bit tidier. Note that both Edwards point (0,1) which maps to+// the Montgomery point at infinity, and Edwards (0,-1) which maps to+// Montgomery (0,0) [this is the 2-torsion point] are both by definition+// mapped to 0 by the X coordinate mapping used to define curve25519.+//+// First the addition and subtraction:++        add_twice4(t1,x_3,w_3)+        sub_twice4(t2,x_3,w_3)++// Prepare to call the modular inverse function to get t0 = 1/t2+// Note that this works for the weakly normalized z_3 equally well.+// The non-coprime case z_3 == 0 (mod p_25519) cannot arise anyway.++        add     x0, t0+        add     x1, t2++// Inline copy of bignum_inv_p25519, identical except for stripping out+// the prologue and epilogue saving and restoring registers and making+// and reclaiming room on the stack. For more details and explanations see+// "arm/curve25519/bignum_inv_p25519.S". Note that the stack it uses for+// its own temporaries is 128 bytes, so it has no effect on variables+// that are needed in the rest of our computation here: res, t0, t1, t2.++        mov     x20, x0+        mov     x10, #0xffffffffffffffed+        mov     x11, #0xffffffffffffffff+        stp     x10, x11, [sp]+        mov     x12, #0x7fffffffffffffff+        stp     x11, x12, [sp, #16]+        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]+        mov     x7, #0x13+        lsr     x6, x5, #63+        madd    x6, x7, x6, x7+        adds    x2, x2, x6+        adcs    x3, x3, xzr+        adcs    x4, x4, xzr+        orr     x5, x5, #0x8000000000000000+        adcs    x5, x5, xzr+        csel    x6, x7, xzr, cc+        subs    x2, x2, x6+        sbcs    x3, x3, xzr+        sbcs    x4, x4, xzr+        sbc     x5, x5, xzr+        and     x5, x5, #0x7fffffffffffffff+        stp     x2, x3, [sp, #32]+        stp     x4, x5, [sp, #48]+        stp     xzr, xzr, [sp, #64]+        stp     xzr, xzr, [sp, #80]+        mov     x10, #0x2099+        movk    x10, #0x7502, lsl #16+        movk    x10, #0x9e23, lsl #32+        movk    x10, #0xa0f9, lsl #48+        mov     x11, #0x2595+        movk    x11, #0x1d13, lsl #16+        movk    x11, #0x8f3f, lsl #32+        movk    x11, #0xa8c6, lsl #48+        mov     x12, #0x5242+        movk    x12, #0x5ac, lsl #16+        movk    x12, #0x8938, lsl #32+        movk    x12, #0x6c6c, lsl #48+        mov     x13, #0x615+        movk    x13, #0x4177, lsl #16+        movk    x13, #0x8b2, lsl #32+        movk    x13, #0x2765, lsl #48+        stp     x10, x11, [sp, #96]+        stp     x12, x13, [sp, #112]+        mov     x21, #0xa+        mov     x22, #0x1+        b       Lcurve25519_x25519base_invmidloop+Lcurve25519_x25519base_invloop:+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        and     x0, x12, x16+        and     x1, x13, x17+        add     x19, x0, x1+        ldr     x7, [sp]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #32]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x7, [sp, #8]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #40]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [sp]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [sp, #32]+        ldr     x7, [sp, #16]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #48]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [sp, #8]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [sp, #40]+        ldr     x7, [sp, #24]+        eor     x1, x7, x14+        asr     x3, x1, #63+        and     x3, x3, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #56]+        eor     x1, x8, x15+        asr     x0, x1, #63+        and     x0, x0, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x5, x6, #59+        str     x6, [sp, #16]+        extr    x5, x3, x5, #59+        str     x5, [sp, #24]+        eor     x1, x7, x16+        asr     x5, x1, #63+        and     x5, x5, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        asr     x0, x1, #63+        and     x0, x0, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x4, x2, x4, #59+        str     x4, [sp, #48]+        extr    x2, x5, x2, #59+        str     x2, [sp, #56]+        ldr     x7, [sp, #64]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #96]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #64]+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        str     x5, [sp, #96]+        adc     x3, x3, x1+        ldr     x7, [sp, #72]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #104]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #72]+        adc     x6, x6, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        str     x3, [sp, #104]+        adc     x4, x4, x1+        ldr     x7, [sp, #80]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #112]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #80]+        adc     x5, x5, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        str     x4, [sp, #112]+        adc     x2, x2, x1+        ldr     x7, [sp, #88]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #120]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x3, x5, #63+        ldp     x0, x1, [sp, #64]+        add     x6, x6, x3, asr #63+        mov     x3, #0x13+        mul     x4, x6, x3+        add     x5, x5, x6, lsl #63+        smulh   x3, x6, x3+        ldr     x6, [sp, #80]+        adds    x0, x0, x4+        adcs    x1, x1, x3+        asr     x3, x3, #63+        adcs    x6, x6, x3+        adc     x5, x5, x3+        stp     x0, x1, [sp, #64]+        stp     x6, x5, [sp, #80]+        eor     x1, x7, x16+        and     x5, x16, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        and     x0, x17, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x6, x5, x2, #63+        ldp     x0, x1, [sp, #96]+        add     x6, x6, x5, asr #63+        mov     x5, #0x13+        mul     x4, x6, x5+        add     x2, x2, x6, lsl #63+        smulh   x5, x6, x5+        ldr     x3, [sp, #112]+        adds    x0, x0, x4+        adcs    x1, x1, x5+        asr     x5, x5, #63+        adcs    x3, x3, x5+        adc     x2, x2, x5+        stp     x0, x1, [sp, #96]+        stp     x3, x2, [sp, #112]+Lcurve25519_x25519base_invmidloop:+        mov     x1, x22+        ldr     x2, [sp]+        ldr     x3, [sp, #32]+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x8, x4, #0x100, lsl #12+        sbfx    x8, x8, #21, #21+        mov     x11, #0x100000+        add     x11, x11, x11, lsl #21+        add     x9, x4, x11+        asr     x9, x9, #42+        add     x10, x5, #0x100, lsl #12+        sbfx    x10, x10, #21, #21+        add     x11, x5, x11+        asr     x11, x11, #42+        mul     x6, x8, x2+        mul     x7, x9, x3+        mul     x2, x10, x2+        mul     x3, x11, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #21, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #42+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #21, #21+        add     x15, x5, x15+        asr     x15, x15, #42+        mul     x6, x12, x2+        mul     x7, x13, x3+        mul     x2, x14, x2+        mul     x3, x15, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        mul     x2, x12, x8+        mul     x3, x12, x9+        mul     x6, x14, x8+        mul     x7, x14, x9+        madd    x8, x13, x10, x2+        madd    x9, x13, x11, x3+        madd    x16, x15, x10, x6+        madd    x17, x15, x11, x7+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #22, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #43+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #22, #21+        add     x15, x5, x15+        asr     x15, x15, #43+        mneg    x2, x12, x8+        mneg    x3, x12, x9+        mneg    x4, x14, x8+        mneg    x5, x14, x9+        msub    x10, x13, x16, x2+        msub    x11, x13, x17, x3+        msub    x12, x15, x16, x4+        msub    x13, x15, x17, x5+        mov     x22, x1+        subs    x21, x21, #0x1+        b.ne    Lcurve25519_x25519base_invloop+        ldr     x0, [sp]+        ldr     x1, [sp, #32]+        mul     x0, x0, x10+        madd    x1, x1, x11, x0+        asr     x0, x1, #63+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        eor     x14, x14, x0+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        eor     x15, x15, x0+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        eor     x16, x16, x0+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        eor     x17, x17, x0+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        ldr     x7, [sp, #64]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #96]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #64]+        adc     x2, x2, x1+        ldr     x7, [sp, #72]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #104]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #72]+        adc     x6, x6, x1+        ldr     x7, [sp, #80]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #112]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #80]+        adc     x5, x5, x1+        ldr     x7, [sp, #88]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #120]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x3, x5, #63+        ldp     x0, x1, [sp, #64]+        tst     x3, x3+        cinc    x6, x6, pl+        mov     x3, #0x13+        mul     x4, x6, x3+        add     x5, x5, x6, lsl #63+        smulh   x6, x6, x3+        ldr     x2, [sp, #80]+        adds    x0, x0, x4+        adcs    x1, x1, x6+        asr     x6, x6, #63+        adcs    x2, x2, x6+        adcs    x5, x5, x6+        csel    x3, x3, xzr, mi+        subs    x0, x0, x3+        sbcs    x1, x1, xzr+        sbcs    x2, x2, xzr+        sbc     x5, x5, xzr+        and     x5, x5, #0x7fffffffffffffff+        mov     x4, x20+        stp     x0, x1, [x4]+        stp     x2, x5, [x4, #16]++// The final result is (X + T) / (X - T)+// This is the only operation in the whole computation that+// fully reduces modulo p_25519 since now we want the canonical+// answer as output.++        mul_p25519(resx,t1,t0)++// Restore stack and registers++        CFI_INC_SP(NSPACE)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(curve25519_x25519base)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif++// ****************************************************************************+// The precomputed data (all read-only).+// ****************************************************************************++#if defined(__ELF__)+.section .rodata+.type S2N_BN_SYMBOL(curve25519_x25519base_constant), %object+.size S2N_BN_SYMBOL(curve25519_x25519base_constant), 48576+#elif defined(__APPLE__)+.const_data+#endif++S2N_BN_SYMBOL(curve25519_x25519base_constant):++// 2^254 * G and (2^254 + 8) * G in extended-projective coordinates+// but with Z = 1 assumed and hence left out, so they are (X,Y,T) only.++        .quad   0x251037f7cf4e861d+        .quad   0x10ede0fb19fb128f+        .quad   0x96c033b175f5e2c8+        .quad   0x055f070d6c15fb0d+        .quad   0x7c52af2c97473e69+        .quad   0x022f82391bad8378+        .quad   0x9991e1b02adb476f+        .quad   0x511144a03a99b855+        .quad   0x5fafc3b88ff2e4ae+        .quad   0x855e4ff0de1230ff+        .quad   0x72e302a348492870+        .quad   0x1253c19e53dbe1bc++        .quad   0x331d086e0d9abcaa+        .quad   0x1e23c96d311a10c9+        .quad   0x96d0f95e58c13478+        .quad   0x2f72f7384fcfcc59+        .quad   0x39a6cd1cfd7d87c9+        .quad   0x9867a0abd8ae153a+        .quad   0xa49d2a5f35986745+        .quad   0x57012940cdfe82e1+        .quad   0x5046a6532ec5544a+        .quad   0x6d674004739ff6c9+        .quad   0x9bbaa44b234a70e3+        .quad   0x5e6d8901138cf386++// Precomputed table of multiples of generator for edwards25519+// all in precomputed extended-projective (y-x,x+y,2*d*x*y) triples.++        // 2^4 * 1 * G++        .quad   0x7ec851ca553e2df3+        .quad   0xa71284cba64878b3+        .quad   0xe6b5e4193288d1e7+        .quad   0x4cf210ec5a9a8883+        .quad   0x322d04a52d9021f6+        .quad   0xb9c19f3375c6bf9c+        .quad   0x587a3a4342d20b09+        .quad   0x143b1cf8aa64fe61+        .quad   0x9f867c7d968acaab+        .quad   0x5f54258e27092729+        .quad   0xd0a7d34bea180975+        .quad   0x21b546a3374126e1++        // 2^4 * 2 * G++        .quad   0xa94ff858a2888343+        .quad   0xce0ed4565313ed3c+        .quad   0xf55c3dcfb5bf34fa+        .quad   0x0a653ca5c9eab371+        .quad   0x490a7a45d185218f+        .quad   0x9a15377846049335+        .quad   0x0060ea09cc31e1f6+        .quad   0x7e041577f86ee965+        .quad   0x66b2a496ce5b67f3+        .quad   0xff5492d8bd569796+        .quad   0x503cec294a592cd0+        .quad   0x566943650813acb2++        // 2^4 * 3 * G++        .quad   0xb818db0c26620798+        .quad   0x5d5c31d9606e354a+        .quad   0x0982fa4f00a8cdc7+        .quad   0x17e12bcd4653e2d4+        .quad   0x5672f9eb1dabb69d+        .quad   0xba70b535afe853fc+        .quad   0x47ac0f752796d66d+        .quad   0x32a5351794117275+        .quad   0xd3a644a6df648437+        .quad   0x703b6559880fbfdd+        .quad   0xcb852540ad3a1aa5+        .quad   0x0900b3f78e4c6468++        // 2^4 * 4 * G++        .quad   0x0a851b9f679d651b+        .quad   0xe108cb61033342f2+        .quad   0xd601f57fe88b30a3+        .quad   0x371f3acaed2dd714+        .quad   0xed280fbec816ad31+        .quad   0x52d9595bd8e6efe3+        .quad   0x0fe71772f6c623f5+        .quad   0x4314030b051e293c+        .quad   0xd560005efbf0bcad+        .quad   0x8eb70f2ed1870c5e+        .quad   0x201f9033d084e6a0+        .quad   0x4c3a5ae1ce7b6670++        // 2^4 * 5 * G++        .quad   0x4138a434dcb8fa95+        .quad   0x870cf67d6c96840b+        .quad   0xde388574297be82c+        .quad   0x7c814db27262a55a+        .quad   0xbaf875e4c93da0dd+        .quad   0xb93282a771b9294d+        .quad   0x80d63fb7f4c6c460+        .quad   0x6de9c73dea66c181+        .quad   0x478904d5a04df8f2+        .quad   0xfafbae4ab10142d3+        .quad   0xf6c8ac63555d0998+        .quad   0x5aac4a412f90b104++        // 2^4 * 6 * G++        .quad   0xc64f326b3ac92908+        .quad   0x5551b282e663e1e0+        .quad   0x476b35f54a1a4b83+        .quad   0x1b9da3fe189f68c2+        .quad   0x603a0d0abd7f5134+        .quad   0x8089c932e1d3ae46+        .quad   0xdf2591398798bd63+        .quad   0x1c145cd274ba0235+        .quad   0x32e8386475f3d743+        .quad   0x365b8baf6ae5d9ef+        .quad   0x825238b6385b681e+        .quad   0x234929c1167d65e1++        // 2^4 * 7 * G++        .quad   0x984decaba077ade8+        .quad   0x383f77ad19eb389d+        .quad   0xc7ec6b7e2954d794+        .quad   0x59c77b3aeb7c3a7a+        .quad   0x48145cc21d099fcf+        .quad   0x4535c192cc28d7e5+        .quad   0x80e7c1e548247e01+        .quad   0x4a5f28743b2973ee+        .quad   0xd3add725225ccf62+        .quad   0x911a3381b2152c5d+        .quad   0xd8b39fad5b08f87d+        .quad   0x6f05606b4799fe3b++        // 2^4 * 8 * G++        .quad   0x9ffe9e92177ba962+        .quad   0x98aee71d0de5cae1+        .quad   0x3ff4ae942d831044+        .quad   0x714de12e58533ac8+        .quad   0x5b433149f91b6483+        .quad   0xadb5dc655a2cbf62+        .quad   0x87fa8412632827b3+        .quad   0x60895e91ab49f8d8+        .quad   0xe9ecf2ed0cf86c18+        .quad   0xb46d06120735dfd4+        .quad   0xbc9da09804b96be7+        .quad   0x73e2e62fd96dc26b++        // 2^8 * 1 * G++        .quad   0xed5b635449aa515e+        .quad   0xa865c49f0bc6823a+        .quad   0x850c1fe95b42d1c4+        .quad   0x30d76d6f03d315b9+        .quad   0x2eccdd0e632f9c1d+        .quad   0x51d0b69676893115+        .quad   0x52dfb76ba8637a58+        .quad   0x6dd37d49a00eef39+        .quad   0x6c4444172106e4c7+        .quad   0xfb53d680928d7f69+        .quad   0xb4739ea4694d3f26+        .quad   0x10c697112e864bb0++        // 2^8 * 2 * G++        .quad   0x6493c4277dbe5fde+        .quad   0x265d4fad19ad7ea2+        .quad   0x0e00dfc846304590+        .quad   0x25e61cabed66fe09+        .quad   0x0ca62aa08358c805+        .quad   0x6a3d4ae37a204247+        .quad   0x7464d3a63b11eddc+        .quad   0x03bf9baf550806ef+        .quad   0x3f13e128cc586604+        .quad   0x6f5873ecb459747e+        .quad   0xa0b63dedcc1268f5+        .quad   0x566d78634586e22c++        // 2^8 * 3 * G++        .quad   0x1637a49f9cc10834+        .quad   0xbc8e56d5a89bc451+        .quad   0x1cb5ec0f7f7fd2db+        .quad   0x33975bca5ecc35d9+        .quad   0xa1054285c65a2fd0+        .quad   0x6c64112af31667c3+        .quad   0x680ae240731aee58+        .quad   0x14fba5f34793b22a+        .quad   0x3cd746166985f7d4+        .quad   0x593e5e84c9c80057+        .quad   0x2fc3f2b67b61131e+        .quad   0x14829cea83fc526c++        // 2^8 * 4 * G++        .quad   0xff437b8497dd95c2+        .quad   0x6c744e30aa4eb5a7+        .quad   0x9e0c5d613c85e88b+        .quad   0x2fd9c71e5f758173+        .quad   0x21e70b2f4e71ecb8+        .quad   0xe656ddb940a477e3+        .quad   0xbf6556cece1d4f80+        .quad   0x05fc3bc4535d7b7e+        .quad   0x24b8b3ae52afdedd+        .quad   0x3495638ced3b30cf+        .quad   0x33a4bc83a9be8195+        .quad   0x373767475c651f04++        // 2^8 * 5 * G++        .quad   0x2fba99fd40d1add9+        .quad   0xb307166f96f4d027+        .quad   0x4363f05215f03bae+        .quad   0x1fbea56c3b18f999+        .quad   0x634095cb14246590+        .quad   0xef12144016c15535+        .quad   0x9e38140c8910bc60+        .quad   0x6bf5905730907c8c+        .quad   0x0fa778f1e1415b8a+        .quad   0x06409ff7bac3a77e+        .quad   0x6f52d7b89aa29a50+        .quad   0x02521cf67a635a56++        // 2^8 * 6 * G++        .quad   0x513fee0b0a9d5294+        .quad   0x8f98e75c0fdf5a66+        .quad   0xd4618688bfe107ce+        .quad   0x3fa00a7e71382ced+        .quad   0xb1146720772f5ee4+        .quad   0xe8f894b196079ace+        .quad   0x4af8224d00ac824a+        .quad   0x001753d9f7cd6cc4+        .quad   0x3c69232d963ddb34+        .quad   0x1dde87dab4973858+        .quad   0xaad7d1f9a091f285+        .quad   0x12b5fe2fa048edb6++        // 2^8 * 7 * G++        .quad   0x71f0fbc496fce34d+        .quad   0x73b9826badf35bed+        .quad   0xd2047261ff28c561+        .quad   0x749b76f96fb1206f+        .quad   0xdf2b7c26ad6f1e92+        .quad   0x4b66d323504b8913+        .quad   0x8c409dc0751c8bc3+        .quad   0x6f7e93c20796c7b8+        .quad   0x1f5af604aea6ae05+        .quad   0xc12351f1bee49c99+        .quad   0x61a808b5eeff6b66+        .quad   0x0fcec10f01e02151++        // 2^8 * 8 * G++        .quad   0x644d58a649fe1e44+        .quad   0x21fcaea231ad777e+        .quad   0x02441c5a887fd0d2+        .quad   0x4901aa7183c511f3+        .quad   0x3df2d29dc4244e45+        .quad   0x2b020e7493d8de0a+        .quad   0x6cc8067e820c214d+        .quad   0x413779166feab90a+        .quad   0x08b1b7548c1af8f0+        .quad   0xce0f7a7c246299b4+        .quad   0xf760b0f91e06d939+        .quad   0x41bb887b726d1213++        // 2^12 * 1 * G++        .quad   0x9267806c567c49d8+        .quad   0x066d04ccca791e6a+        .quad   0xa69f5645e3cc394b+        .quad   0x5c95b686a0788cd2+        .quad   0x97d980e0aa39f7d2+        .quad   0x35d0384252c6b51c+        .quad   0x7d43f49307cd55aa+        .quad   0x56bd36cfb78ac362+        .quad   0x2ac519c10d14a954+        .quad   0xeaf474b494b5fa90+        .quad   0xe6af8382a9f87a5a+        .quad   0x0dea6db1879be094++        // 2^12 * 2 * G++        .quad   0xaa66bf547344e5ab+        .quad   0xda1258888f1b4309+        .quad   0x5e87d2b3fd564b2f+        .quad   0x5b2c78885483b1dd+        .quad   0x15baeb74d6a8797a+        .quad   0x7ef55cf1fac41732+        .quad   0x29001f5a3c8b05c5+        .quad   0x0ad7cc8752eaccfb+        .quad   0x52151362793408cf+        .quad   0xeb0f170319963d94+        .quad   0xa833b2fa883d9466+        .quad   0x093a7fa775003c78++        // 2^12 * 3 * G++        .quad   0xe5107de63a16d7be+        .quad   0xa377ffdc9af332cf+        .quad   0x70d5bf18440b677f+        .quad   0x6a252b19a4a31403+        .quad   0xb8e9604460a91286+        .quad   0x7f3fd8047778d3de+        .quad   0x67d01e31bf8a5e2d+        .quad   0x7b038a06c27b653e+        .quad   0x9ed919d5d36990f3+        .quad   0x5213aebbdb4eb9f2+        .quad   0xc708ea054cb99135+        .quad   0x58ded57f72260e56++        // 2^12 * 4 * G++        .quad   0x78e79dade9413d77+        .quad   0xf257f9d59729e67d+        .quad   0x59db910ee37aa7e6+        .quad   0x6aa11b5bbb9e039c+        .quad   0xda6d53265b0fd48b+        .quad   0x8960823193bfa988+        .quad   0xd78ac93261d57e28+        .quad   0x79f2942d3a5c8143+        .quad   0x97da2f25b6c88de9+        .quad   0x251ba7eaacf20169+        .quad   0x09b44f87ef4eb4e4+        .quad   0x7d90ab1bbc6a7da5++        // 2^12 * 5 * G++        .quad   0x9acca683a7016bfe+        .quad   0x90505f4df2c50b6d+        .quad   0x6b610d5fcce435aa+        .quad   0x19a10d446198ff96+        .quad   0x1a07a3f496b3c397+        .quad   0x11ceaa188f4e2532+        .quad   0x7d9498d5a7751bf0+        .quad   0x19ed161f508dd8a0+        .quad   0x560a2cd687dce6ca+        .quad   0x7f3568c48664cf4d+        .quad   0x8741e95222803a38+        .quad   0x483bdab1595653fc++        // 2^12 * 6 * G++        .quad   0xfa780f148734fa49+        .quad   0x106f0b70360534e0+        .quad   0x2210776fe3e307bd+        .quad   0x3286c109dde6a0fe+        .quad   0xd6cf4d0ab4da80f6+        .quad   0x82483e45f8307fe0+        .quad   0x05005269ae6f9da4+        .quad   0x1c7052909cf7877a+        .quad   0x32ee7de2874e98d4+        .quad   0x14c362e9b97e0c60+        .quad   0x5781dcde6a60a38a+        .quad   0x217dd5eaaa7aa840++        // 2^12 * 7 * G++        .quad   0x9db7c4d0248e1eb0+        .quad   0xe07697e14d74bf52+        .quad   0x1e6a9b173c562354+        .quad   0x7fa7c21f795a4965+        .quad   0x8bdf1fb9be8c0ec8+        .quad   0x00bae7f8e30a0282+        .quad   0x4963991dad6c4f6c+        .quad   0x07058a6e5df6f60a+        .quad   0xe9eb02c4db31f67f+        .quad   0xed25fd8910bcfb2b+        .quad   0x46c8131f5c5cddb4+        .quad   0x33b21c13a0cb9bce++        // 2^12 * 8 * G++        .quad   0x360692f8087d8e31+        .quad   0xf4dcc637d27163f7+        .quad   0x25a4e62065ea5963+        .quad   0x659bf72e5ac160d9+        .quad   0x9aafb9b05ee38c5b+        .quad   0xbf9d2d4e071a13c7+        .quad   0x8eee6e6de933290a+        .quad   0x1c3bab17ae109717+        .quad   0x1c9ab216c7cab7b0+        .quad   0x7d65d37407bbc3cc+        .quad   0x52744750504a58d5+        .quad   0x09f2606b131a2990++        // 2^16 * 1 * G++        .quad   0x40e87d44744346be+        .quad   0x1d48dad415b52b25+        .quad   0x7c3a8a18a13b603e+        .quad   0x4eb728c12fcdbdf7+        .quad   0x7e234c597c6691ae+        .quad   0x64889d3d0a85b4c8+        .quad   0xdae2c90c354afae7+        .quad   0x0a871e070c6a9e1d+        .quad   0x3301b5994bbc8989+        .quad   0x736bae3a5bdd4260+        .quad   0x0d61ade219d59e3c+        .quad   0x3ee7300f2685d464++        // 2^16 * 2 * G++        .quad   0xf5d255e49e7dd6b7+        .quad   0x8016115c610b1eac+        .quad   0x3c99975d92e187ca+        .quad   0x13815762979125c2+        .quad   0x43fa7947841e7518+        .quad   0xe5c6fa59639c46d7+        .quad   0xa1065e1de3052b74+        .quad   0x7d47c6a2cfb89030+        .quad   0x3fdad0148ef0d6e0+        .quad   0x9d3e749a91546f3c+        .quad   0x71ec621026bb8157+        .quad   0x148cf58d34c9ec80++        // 2^16 * 3 * G++        .quad   0x46a492f67934f027+        .quad   0x469984bef6840aa9+        .quad   0x5ca1bc2a89611854+        .quad   0x3ff2fa1ebd5dbbd4+        .quad   0xe2572f7d9ae4756d+        .quad   0x56c345bb88f3487f+        .quad   0x9fd10b6d6960a88d+        .quad   0x278febad4eaea1b9+        .quad   0xb1aa681f8c933966+        .quad   0x8c21949c20290c98+        .quad   0x39115291219d3c52+        .quad   0x4104dd02fe9c677b++        // 2^16 * 4 * G++        .quad   0x72b2bf5e1124422a+        .quad   0xa1fa0c3398a33ab5+        .quad   0x94cb6101fa52b666+        .quad   0x2c863b00afaf53d5+        .quad   0x81214e06db096ab8+        .quad   0x21a8b6c90ce44f35+        .quad   0x6524c12a409e2af5+        .quad   0x0165b5a48efca481+        .quad   0xf190a474a0846a76+        .quad   0x12eff984cd2f7cc0+        .quad   0x695e290658aa2b8f+        .quad   0x591b67d9bffec8b8++        // 2^16 * 5 * G++        .quad   0x312f0d1c80b49bfa+        .quad   0x5979515eabf3ec8a+        .quad   0x727033c09ef01c88+        .quad   0x3de02ec7ca8f7bcb+        .quad   0x99b9b3719f18b55d+        .quad   0xe465e5faa18c641e+        .quad   0x61081136c29f05ed+        .quad   0x489b4f867030128b+        .quad   0xd232102d3aeb92ef+        .quad   0xe16253b46116a861+        .quad   0x3d7eabe7190baa24+        .quad   0x49f5fbba496cbebf++        // 2^16 * 6 * G++        .quad   0x30949a108a5bcfd4+        .quad   0xdc40dd70bc6473eb+        .quad   0x92c294c1307c0d1c+        .quad   0x5604a86dcbfa6e74+        .quad   0x155d628c1e9c572e+        .quad   0x8a4d86acc5884741+        .quad   0x91a352f6515763eb+        .quad   0x06a1a6c28867515b+        .quad   0x7288d1d47c1764b6+        .quad   0x72541140e0418b51+        .quad   0x9f031a6018acf6d1+        .quad   0x20989e89fe2742c6++        // 2^16 * 7 * G++        .quad   0x499777fd3a2dcc7f+        .quad   0x32857c2ca54fd892+        .quad   0xa279d864d207e3a0+        .quad   0x0403ed1d0ca67e29+        .quad   0x1674278b85eaec2e+        .quad   0x5621dc077acb2bdf+        .quad   0x640a4c1661cbf45a+        .quad   0x730b9950f70595d3+        .quad   0xc94b2d35874ec552+        .quad   0xc5e6c8cf98246f8d+        .quad   0xf7cb46fa16c035ce+        .quad   0x5bd7454308303dcc++        // 2^16 * 8 * G++        .quad   0x7f9ad19528b24cc2+        .quad   0x7f6b54656335c181+        .quad   0x66b8b66e4fc07236+        .quad   0x133a78007380ad83+        .quad   0x85c4932115e7792a+        .quad   0xc64c89a2bdcdddc9+        .quad   0x9d1e3da8ada3d762+        .quad   0x5bb7db123067f82c+        .quad   0x0961f467c6ca62be+        .quad   0x04ec21d6211952ee+        .quad   0x182360779bd54770+        .quad   0x740dca6d58f0e0d2++        // 2^20 * 1 * G++        .quad   0x50b70bf5d3f0af0b+        .quad   0x4feaf48ae32e71f7+        .quad   0x60e84ed3a55bbd34+        .quad   0x00ed489b3f50d1ed+        .quad   0x3906c72aed261ae5+        .quad   0x9ab68fd988e100f7+        .quad   0xf5e9059af3360197+        .quad   0x0e53dc78bf2b6d47+        .quad   0xb90829bf7971877a+        .quad   0x5e4444636d17e631+        .quad   0x4d05c52e18276893+        .quad   0x27632d9a5a4a4af5++        // 2^20 * 2 * G++        .quad   0xd11ff05154b260ce+        .quad   0xd86dc38e72f95270+        .quad   0x601fcd0d267cc138+        .quad   0x2b67916429e90ccd+        .quad   0xa98285d187eaffdb+        .quad   0xa5b4fbbbd8d0a864+        .quad   0xb658f27f022663f7+        .quad   0x3bbc2b22d99ce282+        .quad   0xb917c952583c0a58+        .quad   0x653ff9b80fe4c6f3+        .quad   0x9b0da7d7bcdf3c0c+        .quad   0x43a0eeb6ab54d60e++        // 2^20 * 3 * G++        .quad   0x396966a46d4a5487+        .quad   0xf811a18aac2bb3ba+        .quad   0x66e4685b5628b26b+        .quad   0x70a477029d929b92+        .quad   0x3ac6322357875fe8+        .quad   0xd9d4f4ecf5fbcb8f+        .quad   0x8dee8493382bb620+        .quad   0x50c5eaa14c799fdc+        .quad   0xdd0edc8bd6f2fb3c+        .quad   0x54c63aa79cc7b7a0+        .quad   0xae0b032b2c8d9f1a+        .quad   0x6f9ce107602967fb++        // 2^20 * 4 * G++        .quad   0xad1054b1cde1c22a+        .quad   0xc4a8e90248eb32df+        .quad   0x5f3e7b33accdc0ea+        .quad   0x72364713fc79963e+        .quad   0x139693063520e0b5+        .quad   0x437fcf7c88ea03fe+        .quad   0xf7d4c40bd3c959bc+        .quad   0x699154d1f893ded9+        .quad   0x315d5c75b4b27526+        .quad   0xcccb842d0236daa5+        .quad   0x22f0c8a3345fee8e+        .quad   0x73975a617d39dbed++        // 2^20 * 5 * G++        .quad   0xe4024df96375da10+        .quad   0x78d3251a1830c870+        .quad   0x902b1948658cd91c+        .quad   0x7e18b10b29b7438a+        .quad   0x6f37f392f4433e46+        .quad   0x0e19b9a11f566b18+        .quad   0x220fb78a1fd1d662+        .quad   0x362a4258a381c94d+        .quad   0x9071d9132b6beb2f+        .quad   0x0f26e9ad28418247+        .quad   0xeab91ec9bdec925d+        .quad   0x4be65bc8f48af2de++        // 2^20 * 6 * G++        .quad   0x78487feba36e7028+        .quad   0x5f3f13001dd8ce34+        .quad   0x934fb12d4b30c489+        .quad   0x056c244d397f0a2b+        .quad   0x1d50fba257c26234+        .quad   0x7bd4823adeb0678b+        .quad   0xc2b0dc6ea6538af5+        .quad   0x5665eec6351da73e+        .quad   0xdb3ee00943bfb210+        .quad   0x4972018720800ac2+        .quad   0x26ab5d6173bd8667+        .quad   0x20b209c2ab204938++        // 2^20 * 7 * G++        .quad   0x549e342ac07fb34b+        .quad   0x02d8220821373d93+        .quad   0xbc262d70acd1f567+        .quad   0x7a92c9fdfbcac784+        .quad   0x1fcca94516bd3289+        .quad   0x448d65aa41420428+        .quad   0x59c3b7b216a55d62+        .quad   0x49992cc64e612cd8+        .quad   0x65bd1bea70f801de+        .quad   0x1befb7c0fe49e28a+        .quad   0xa86306cdb1b2ae4a+        .quad   0x3b7ac0cd265c2a09++        // 2^20 * 8 * G++        .quad   0x822bee438c01bcec+        .quad   0x530cb525c0fbc73b+        .quad   0x48519034c1953fe9+        .quad   0x265cc261e09a0f5b+        .quad   0xf0d54e4f22ed39a7+        .quad   0xa2aae91e5608150a+        .quad   0xf421b2e9eddae875+        .quad   0x31bc531d6b7de992+        .quad   0xdf3d134da980f971+        .quad   0x7a4fb8d1221a22a7+        .quad   0x3df7d42035aad6d8+        .quad   0x2a14edcc6a1a125e++        // 2^24 * 1 * G++        .quad   0xdf48ee0752cfce4e+        .quad   0xc3fffaf306ec08b7+        .quad   0x05710b2ab95459c4+        .quad   0x161d25fa963ea38d+        .quad   0x231a8c570478433c+        .quad   0xb7b5270ec281439d+        .quad   0xdbaa99eae3d9079f+        .quad   0x2c03f5256c2b03d9+        .quad   0x790f18757b53a47d+        .quad   0x307b0130cf0c5879+        .quad   0x31903d77257ef7f9+        .quad   0x699468bdbd96bbaf++        // 2^24 * 2 * G++        .quad   0xbd1f2f46f4dafecf+        .quad   0x7cef0114a47fd6f7+        .quad   0xd31ffdda4a47b37f+        .quad   0x525219a473905785+        .quad   0xd8dd3de66aa91948+        .quad   0x485064c22fc0d2cc+        .quad   0x9b48246634fdea2f+        .quad   0x293e1c4e6c4a2e3a+        .quad   0x376e134b925112e1+        .quad   0x703778b5dca15da0+        .quad   0xb04589af461c3111+        .quad   0x5b605c447f032823++        // 2^24 * 3 * G++        .quad   0xb965805920c47c89+        .quad   0xe7f0100c923b8fcc+        .quad   0x0001256502e2ef77+        .quad   0x24a76dcea8aeb3ee+        .quad   0x3be9fec6f0e7f04c+        .quad   0x866a579e75e34962+        .quad   0x5542ef161e1de61a+        .quad   0x2f12fef4cc5abdd5+        .quad   0x0a4522b2dfc0c740+        .quad   0x10d06e7f40c9a407+        .quad   0xc6cf144178cff668+        .quad   0x5e607b2518a43790++        // 2^24 * 4 * G++        .quad   0x58b31d8f6cdf1818+        .quad   0x35cfa74fc36258a2+        .quad   0xe1b3ff4f66e61d6e+        .quad   0x5067acab6ccdd5f7+        .quad   0xa02c431ca596cf14+        .quad   0xe3c42d40aed3e400+        .quad   0xd24526802e0f26db+        .quad   0x201f33139e457068+        .quad   0xfd527f6b08039d51+        .quad   0x18b14964017c0006+        .quad   0xd5220eb02e25a4a8+        .quad   0x397cba8862460375++        // 2^24 * 5 * G++        .quad   0x30c13093f05959b2+        .quad   0xe23aa18de9a97976+        .quad   0x222fd491721d5e26+        .quad   0x2339d320766e6c3a+        .quad   0x7815c3fbc81379e7+        .quad   0xa6619420dde12af1+        .quad   0xffa9c0f885a8fdd5+        .quad   0x771b4022c1e1c252+        .quad   0xd87dd986513a2fa7+        .quad   0xf5ac9b71f9d4cf08+        .quad   0xd06bc31b1ea283b3+        .quad   0x331a189219971a76++        // 2^24 * 6 * G++        .quad   0xf5166f45fb4f80c6+        .quad   0x9c36c7de61c775cf+        .quad   0xe3d4e81b9041d91c+        .quad   0x31167c6b83bdfe21+        .quad   0x26512f3a9d7572af+        .quad   0x5bcbe28868074a9e+        .quad   0x84edc1c11180f7c4+        .quad   0x1ac9619ff649a67b+        .quad   0xf22b3842524b1068+        .quad   0x5068343bee9ce987+        .quad   0xfc9d71844a6250c8+        .quad   0x612436341f08b111++        // 2^24 * 7 * G++        .quad   0xd99d41db874e898d+        .quad   0x09fea5f16c07dc20+        .quad   0x793d2c67d00f9bbc+        .quad   0x46ebe2309e5eff40+        .quad   0x8b6349e31a2d2638+        .quad   0x9ddfb7009bd3fd35+        .quad   0x7f8bf1b8a3a06ba4+        .quad   0x1522aa3178d90445+        .quad   0x2c382f5369614938+        .quad   0xdafe409ab72d6d10+        .quad   0xe8c83391b646f227+        .quad   0x45fe70f50524306c++        // 2^24 * 8 * G++        .quad   0xda4875a6960c0b8c+        .quad   0x5b68d076ef0e2f20+        .quad   0x07fb51cf3d0b8fd4+        .quad   0x428d1623a0e392d4+        .quad   0x62f24920c8951491+        .quad   0x05f007c83f630ca2+        .quad   0x6fbb45d2f5c9d4b8+        .quad   0x16619f6db57a2245+        .quad   0x084f4a4401a308fd+        .quad   0xa82219c376a5caac+        .quad   0xdeb8de4643d1bc7d+        .quad   0x1d81592d60bd38c6++        // 2^28 * 1 * G++        .quad   0xd833d7beec2a4c38+        .quad   0x2c9162830acc20ed+        .quad   0xe93a47aa92df7581+        .quad   0x702d67a3333c4a81+        .quad   0x3a4a369a2f89c8a1+        .quad   0x63137a1d7c8de80d+        .quad   0xbcac008a78eda015+        .quad   0x2cb8b3a5b483b03f+        .quad   0x36e417cbcb1b90a1+        .quad   0x33b3ddaa7f11794e+        .quad   0x3f510808885bc607+        .quad   0x24141dc0e6a8020d++        // 2^28 * 2 * G++        .quad   0x59f73c773fefee9d+        .quad   0xb3f1ef89c1cf989d+        .quad   0xe35dfb42e02e545f+        .quad   0x5766120b47a1b47c+        .quad   0x91925dccbd83157d+        .quad   0x3ca1205322cc8094+        .quad   0x28e57f183f90d6e4+        .quad   0x1a4714cede2e767b+        .quad   0xdb20ba0fb8b6b7ff+        .quad   0xb732c3b677511fa1+        .quad   0xa92b51c099f02d89+        .quad   0x4f3875ad489ca5f1++        // 2^28 * 3 * G++        .quad   0xc7fc762f4932ab22+        .quad   0x7ac0edf72f4c3c1b+        .quad   0x5f6b55aa9aa895e8+        .quad   0x3680274dad0a0081+        .quad   0x79ed13f6ee73eec0+        .quad   0xa5c6526d69110bb1+        .quad   0xe48928c38603860c+        .quad   0x722a1446fd7059f5+        .quad   0xd0959fe9a8cf8819+        .quad   0xd0a995508475a99c+        .quad   0x6eac173320b09cc5+        .quad   0x628ecf04331b1095++        // 2^28 * 4 * G++        .quad   0x98bcb118a9d0ddbc+        .quad   0xee449e3408b4802b+        .quad   0x87089226b8a6b104+        .quad   0x685f349a45c7915d+        .quad   0x9b41acf85c74ccf1+        .quad   0xb673318108265251+        .quad   0x99c92aed11adb147+        .quad   0x7a47d70d34ecb40f+        .quad   0x60a0c4cbcc43a4f5+        .quad   0x775c66ca3677bea9+        .quad   0xa17aa1752ff8f5ed+        .quad   0x11ded9020e01fdc0++        // 2^28 * 5 * G++        .quad   0x890e7809caefe704+        .quad   0x8728296de30e8c6c+        .quad   0x4c5cd2a392aeb1c9+        .quad   0x194263d15771531f+        .quad   0x471f95b03bea93b7+        .quad   0x0552d7d43313abd3+        .quad   0xbd9370e2e17e3f7b+        .quad   0x7b120f1db20e5bec+        .quad   0x17d2fb3d86502d7a+        .quad   0xb564d84450a69352+        .quad   0x7da962c8a60ed75d+        .quad   0x00d0f85b318736aa++        // 2^28 * 6 * G++        .quad   0x978b142e777c84fd+        .quad   0xf402644705a8c062+        .quad   0xa67ad51be7e612c7+        .quad   0x2f7b459698dd6a33+        .quad   0xa6753c1efd7621c1+        .quad   0x69c0b4a7445671f5+        .quad   0x971f527405b23c11+        .quad   0x387bc74851a8c7cd+        .quad   0x81894b4d4a52a9a8+        .quad   0xadd93e12f6b8832f+        .quad   0x184d8548b61bd638+        .quad   0x3f1c62dbd6c9f6cd++        // 2^28 * 7 * G++        .quad   0x2e8f1f0091910c1f+        .quad   0xa4df4fe0bff2e12c+        .quad   0x60c6560aee927438+        .quad   0x6338283facefc8fa+        .quad   0x3fad3e40148f693d+        .quad   0x052656e194eb9a72+        .quad   0x2f4dcbfd184f4e2f+        .quad   0x406f8db1c482e18b+        .quad   0x9e630d2c7f191ee4+        .quad   0x4fbf8301bc3ff670+        .quad   0x787d8e4e7afb73c4+        .quad   0x50d83d5be8f58fa5++        // 2^28 * 8 * G++        .quad   0x85683916c11a1897+        .quad   0x2d69a4efe506d008+        .quad   0x39af1378f664bd01+        .quad   0x65942131361517c6+        .quad   0xc0accf90b4d3b66d+        .quad   0xa7059de561732e60+        .quad   0x033d1f7870c6b0ba+        .quad   0x584161cd26d946e4+        .quad   0xbbf2b1a072d27ca2+        .quad   0xbf393c59fbdec704+        .quad   0xe98dbbcee262b81e+        .quad   0x02eebd0b3029b589++        // 2^32 * 1 * G++        .quad   0x61368756a60dac5f+        .quad   0x17e02f6aebabdc57+        .quad   0x7f193f2d4cce0f7d+        .quad   0x20234a7789ecdcf0+        .quad   0x8765b69f7b85c5e8+        .quad   0x6ff0678bd168bab2+        .quad   0x3a70e77c1d330f9b+        .quad   0x3a5f6d51b0af8e7c+        .quad   0x76d20db67178b252+        .quad   0x071c34f9d51ed160+        .quad   0xf62a4a20b3e41170+        .quad   0x7cd682353cffe366++        // 2^32 * 2 * G++        .quad   0x0be1a45bd887fab6+        .quad   0x2a846a32ba403b6e+        .quad   0xd9921012e96e6000+        .quad   0x2838c8863bdc0943+        .quad   0xa665cd6068acf4f3+        .quad   0x42d92d183cd7e3d3+        .quad   0x5759389d336025d9+        .quad   0x3ef0253b2b2cd8ff+        .quad   0xd16bb0cf4a465030+        .quad   0xfa496b4115c577ab+        .quad   0x82cfae8af4ab419d+        .quad   0x21dcb8a606a82812++        // 2^32 * 3 * G++        .quad   0x5c6004468c9d9fc8+        .quad   0x2540096ed42aa3cb+        .quad   0x125b4d4c12ee2f9c+        .quad   0x0bc3d08194a31dab+        .quad   0x9a8d00fabe7731ba+        .quad   0x8203607e629e1889+        .quad   0xb2cc023743f3d97f+        .quad   0x5d840dbf6c6f678b+        .quad   0x706e380d309fe18b+        .quad   0x6eb02da6b9e165c7+        .quad   0x57bbba997dae20ab+        .quad   0x3a4276232ac196dd++        // 2^32 * 4 * G++        .quad   0x4b42432c8a7084fa+        .quad   0x898a19e3dfb9e545+        .quad   0xbe9f00219c58e45d+        .quad   0x1ff177cea16debd1+        .quad   0x3bf8c172db447ecb+        .quad   0x5fcfc41fc6282dbd+        .quad   0x80acffc075aa15fe+        .quad   0x0770c9e824e1a9f9+        .quad   0xcf61d99a45b5b5fd+        .quad   0x860984e91b3a7924+        .quad   0xe7300919303e3e89+        .quad   0x39f264fd41500b1e++        // 2^32 * 5 * G++        .quad   0xa7ad3417dbe7e29c+        .quad   0xbd94376a2b9c139c+        .quad   0xa0e91b8e93597ba9+        .quad   0x1712d73468889840+        .quad   0xd19b4aabfe097be1+        .quad   0xa46dfce1dfe01929+        .quad   0xc3c908942ca6f1ff+        .quad   0x65c621272c35f14e+        .quad   0xe72b89f8ce3193dd+        .quad   0x4d103356a125c0bb+        .quad   0x0419a93d2e1cfe83+        .quad   0x22f9800ab19ce272++        // 2^32 * 6 * G++        .quad   0x605a368a3e9ef8cb+        .quad   0xe3e9c022a5504715+        .quad   0x553d48b05f24248f+        .quad   0x13f416cd647626e5+        .quad   0x42029fdd9a6efdac+        .quad   0xb912cebe34a54941+        .quad   0x640f64b987bdf37b+        .quad   0x4171a4d38598cab4+        .quad   0xfa2758aa99c94c8c+        .quad   0x23006f6fb000b807+        .quad   0xfbd291ddadda5392+        .quad   0x508214fa574bd1ab++        // 2^32 * 7 * G++        .quad   0xc20269153ed6fe4b+        .quad   0xa65a6739511d77c4+        .quad   0xcbde26462c14af94+        .quad   0x22f960ec6faba74b+        .quad   0x461a15bb53d003d6+        .quad   0xb2102888bcf3c965+        .quad   0x27c576756c683a5a+        .quad   0x3a7758a4c86cb447+        .quad   0x548111f693ae5076+        .quad   0x1dae21df1dfd54a6+        .quad   0x12248c90f3115e65+        .quad   0x5d9fd15f8de7f494++        // 2^32 * 8 * G++        .quad   0x031408d36d63727f+        .quad   0x6a379aefd7c7b533+        .quad   0xa9e18fc5ccaee24b+        .quad   0x332f35914f8fbed3+        .quad   0x3f244d2aeed7521e+        .quad   0x8e3a9028432e9615+        .quad   0xe164ba772e9c16d4+        .quad   0x3bc187fa47eb98d8+        .quad   0x6d470115ea86c20c+        .quad   0x998ab7cb6c46d125+        .quad   0xd77832b53a660188+        .quad   0x450d81ce906fba03++        // 2^36 * 1 * G++        .quad   0xf8ae4d2ad8453902+        .quad   0x7018058ee8db2d1d+        .quad   0xaab3995fc7d2c11e+        .quad   0x53b16d2324ccca79+        .quad   0x23264d66b2cae0b5+        .quad   0x7dbaed33ebca6576+        .quad   0x030ebed6f0d24ac8+        .quad   0x2a887f78f7635510+        .quad   0x2a23b9e75c012d4f+        .quad   0x0c974651cae1f2ea+        .quad   0x2fb63273675d70ca+        .quad   0x0ba7250b864403f5++        // 2^36 * 2 * G++        .quad   0xbb0d18fd029c6421+        .quad   0xbc2d142189298f02+        .quad   0x8347f8e68b250e96+        .quad   0x7b9f2fe8032d71c9+        .quad   0xdd63589386f86d9c+        .quad   0x61699176e13a85a4+        .quad   0x2e5111954eaa7d57+        .quad   0x32c21b57fb60bdfb+        .quad   0xd87823cd319e0780+        .quad   0xefc4cfc1897775c5+        .quad   0x4854fb129a0ab3f7+        .quad   0x12c49d417238c371++        // 2^36 * 3 * G++        .quad   0x0950b533ffe83769+        .quad   0x21861c1d8e1d6bd1+        .quad   0xf022d8381302e510+        .quad   0x2509200c6391cab4+        .quad   0x09b3a01783799542+        .quad   0x626dd08faad5ee3f+        .quad   0xba00bceeeb70149f+        .quad   0x1421b246a0a444c9+        .quad   0x4aa43a8e8c24a7c7+        .quad   0x04c1f540d8f05ef5+        .quad   0xadba5e0c0b3eb9dc+        .quad   0x2ab5504448a49ce3++        // 2^36 * 4 * G++        .quad   0x2ed227266f0f5dec+        .quad   0x9824ee415ed50824+        .quad   0x807bec7c9468d415+        .quad   0x7093bae1b521e23f+        .quad   0xdc07ac631c5d3afa+        .quad   0x58615171f9df8c6c+        .quad   0x72a079d89d73e2b0+        .quad   0x7301f4ceb4eae15d+        .quad   0x6409e759d6722c41+        .quad   0xa674e1cf72bf729b+        .quad   0xbc0a24eb3c21e569+        .quad   0x390167d24ebacb23++        // 2^36 * 5 * G++        .quad   0x27f58e3bba353f1c+        .quad   0x4c47764dbf6a4361+        .quad   0xafbbc4e56e562650+        .quad   0x07db2ee6aae1a45d+        .quad   0xd7bb054ba2f2120b+        .quad   0xe2b9ceaeb10589b7+        .quad   0x3fe8bac8f3c0edbe+        .quad   0x4cbd40767112cb69+        .quad   0x0b603cc029c58176+        .quad   0x5988e3825cb15d61+        .quad   0x2bb61413dcf0ad8d+        .quad   0x7b8eec6c74183287++        // 2^36 * 6 * G++        .quad   0xe4ca40782cd27cb0+        .quad   0xdaf9c323fbe967bd+        .quad   0xb29bd34a8ad41e9e+        .quad   0x72810497626ede4d+        .quad   0x32fee570fc386b73+        .quad   0xda8b0141da3a8cc7+        .quad   0x975ffd0ac8968359+        .quad   0x6ee809a1b132a855+        .quad   0x9444bb31fcfd863a+        .quad   0x2fe3690a3e4e48c5+        .quad   0xdc29c867d088fa25+        .quad   0x13bd1e38d173292e++        // 2^36 * 7 * G++        .quad   0xd32b4cd8696149b5+        .quad   0xe55937d781d8aab7+        .quad   0x0bcb2127ae122b94+        .quad   0x41e86fcfb14099b0+        .quad   0x223fb5cf1dfac521+        .quad   0x325c25316f554450+        .quad   0x030b98d7659177ac+        .quad   0x1ed018b64f88a4bd+        .quad   0x3630dfa1b802a6b0+        .quad   0x880f874742ad3bd5+        .quad   0x0af90d6ceec5a4d4+        .quad   0x746a247a37cdc5d9++        // 2^36 * 8 * G++        .quad   0xd531b8bd2b7b9af6+        .quad   0x5005093537fc5b51+        .quad   0x232fcf25c593546d+        .quad   0x20a365142bb40f49+        .quad   0x6eccd85278d941ed+        .quad   0x2254ae83d22f7843+        .quad   0xc522d02e7bbfcdb7+        .quad   0x681e3351bff0e4e2+        .quad   0x8b64b59d83034f45+        .quad   0x2f8b71f21fa20efb+        .quad   0x69249495ba6550e4+        .quad   0x539ef98e45d5472b++        // 2^40 * 1 * G++        .quad   0x6e7bb6a1a6205275+        .quad   0xaa4f21d7413c8e83+        .quad   0x6f56d155e88f5cb2+        .quad   0x2de25d4ba6345be1+        .quad   0xd074d8961cae743f+        .quad   0xf86d18f5ee1c63ed+        .quad   0x97bdc55be7f4ed29+        .quad   0x4cbad279663ab108+        .quad   0x80d19024a0d71fcd+        .quad   0xc525c20afb288af8+        .quad   0xb1a3974b5f3a6419+        .quad   0x7d7fbcefe2007233++        // 2^40 * 2 * G++        .quad   0xfaef1e6a266b2801+        .quad   0x866c68c4d5739f16+        .quad   0xf68a2fbc1b03762c+        .quad   0x5975435e87b75a8d+        .quad   0xcd7c5dc5f3c29094+        .quad   0xc781a29a2a9105ab+        .quad   0x80c61d36421c3058+        .quad   0x4f9cd196dcd8d4d7+        .quad   0x199297d86a7b3768+        .quad   0xd0d058241ad17a63+        .quad   0xba029cad5c1c0c17+        .quad   0x7ccdd084387a0307++        // 2^40 * 3 * G++        .quad   0xdca6422c6d260417+        .quad   0xae153d50948240bd+        .quad   0xa9c0c1b4fb68c677+        .quad   0x428bd0ed61d0cf53+        .quad   0x9b0c84186760cc93+        .quad   0xcdae007a1ab32a99+        .quad   0xa88dec86620bda18+        .quad   0x3593ca848190ca44+        .quad   0x9213189a5e849aa7+        .quad   0xd4d8c33565d8facd+        .quad   0x8c52545b53fdbbd1+        .quad   0x27398308da2d63e6++        // 2^40 * 4 * G++        .quad   0x42c38d28435ed413+        .quad   0xbd50f3603278ccc9+        .quad   0xbb07ab1a79da03ef+        .quad   0x269597aebe8c3355+        .quad   0xb9a10e4c0a702453+        .quad   0x0fa25866d57d1bde+        .quad   0xffb9d9b5cd27daf7+        .quad   0x572c2945492c33fd+        .quad   0xc77fc745d6cd30be+        .quad   0xe4dfe8d3e3baaefb+        .quad   0xa22c8830aa5dda0c+        .quad   0x7f985498c05bca80++        // 2^40 * 5 * G++        .quad   0x3849ce889f0be117+        .quad   0x8005ad1b7b54a288+        .quad   0x3da3c39f23fc921c+        .quad   0x76c2ec470a31f304+        .quad   0xd35615520fbf6363+        .quad   0x08045a45cf4dfba6+        .quad   0xeec24fbc873fa0c2+        .quad   0x30f2653cd69b12e7+        .quad   0x8a08c938aac10c85+        .quad   0x46179b60db276bcb+        .quad   0xa920c01e0e6fac70+        .quad   0x2f1273f1596473da++        // 2^40 * 6 * G++        .quad   0x4739fc7c8ae01e11+        .quad   0xfd5274904a6aab9f+        .quad   0x41d98a8287728f2e+        .quad   0x5d9e572ad85b69f2+        .quad   0x30488bd755a70bc0+        .quad   0x06d6b5a4f1d442e7+        .quad   0xead1a69ebc596162+        .quad   0x38ac1997edc5f784+        .quad   0x0666b517a751b13b+        .quad   0x747d06867e9b858c+        .quad   0xacacc011454dde49+        .quad   0x22dfcd9cbfe9e69c++        // 2^40 * 7 * G++        .quad   0x8ddbd2e0c30d0cd9+        .quad   0xad8e665facbb4333+        .quad   0x8f6b258c322a961f+        .quad   0x6b2916c05448c1c7+        .quad   0x56ec59b4103be0a1+        .quad   0x2ee3baecd259f969+        .quad   0x797cb29413f5cd32+        .quad   0x0fe9877824cde472+        .quad   0x7edb34d10aba913b+        .quad   0x4ea3cd822e6dac0e+        .quad   0x66083dff6578f815+        .quad   0x4c303f307ff00a17++        // 2^40 * 8 * G++        .quad   0xd30a3bd617b28c85+        .quad   0xc5d377b739773bea+        .quad   0xc6c6e78c1e6a5cbf+        .quad   0x0d61b8f78b2ab7c4+        .quad   0x29fc03580dd94500+        .quad   0xecd27aa46fbbec93+        .quad   0x130a155fc2e2a7f8+        .quad   0x416b151ab706a1d5+        .quad   0x56a8d7efe9c136b0+        .quad   0xbd07e5cd58e44b20+        .quad   0xafe62fda1b57e0ab+        .quad   0x191a2af74277e8d2++        // 2^44 * 1 * G++        .quad   0xd550095bab6f4985+        .quad   0x04f4cd5b4fbfaf1a+        .quad   0x9d8e2ed12a0c7540+        .quad   0x2bc24e04b2212286+        .quad   0x09d4b60b2fe09a14+        .quad   0xc384f0afdbb1747e+        .quad   0x58e2ea8978b5fd6e+        .quad   0x519ef577b5e09b0a+        .quad   0x1863d7d91124cca9+        .quad   0x7ac08145b88a708e+        .quad   0x2bcd7309857031f5+        .quad   0x62337a6e8ab8fae5++        // 2^44 * 2 * G++        .quad   0x4bcef17f06ffca16+        .quad   0xde06e1db692ae16a+        .quad   0x0753702d614f42b0+        .quad   0x5f6041b45b9212d0+        .quad   0xd1ab324e1b3a1273+        .quad   0x18947cf181055340+        .quad   0x3b5d9567a98c196e+        .quad   0x7fa00425802e1e68+        .quad   0x7d531574028c2705+        .quad   0x80317d69db0d75fe+        .quad   0x30fface8ef8c8ddd+        .quad   0x7e9de97bb6c3e998++        // 2^44 * 3 * G++        .quad   0x1558967b9e6585a3+        .quad   0x97c99ce098e98b92+        .quad   0x10af149b6eb3adad+        .quad   0x42181fe8f4d38cfa+        .quad   0xf004be62a24d40dd+        .quad   0xba0659910452d41f+        .quad   0x81c45ee162a44234+        .quad   0x4cb829d8a22266ef+        .quad   0x1dbcaa8407b86681+        .quad   0x081f001e8b26753b+        .quad   0x3cd7ce6a84048e81+        .quad   0x78af11633f25f22c++        // 2^44 * 4 * G++        .quad   0x8416ebd40b50babc+        .quad   0x1508722628208bee+        .quad   0xa3148fafb9c1c36d+        .quad   0x0d07daacd32d7d5d+        .quad   0x3241c00e7d65318c+        .quad   0xe6bee5dcd0e86de7+        .quad   0x118b2dc2fbc08c26+        .quad   0x680d04a7fc603dc3+        .quad   0xf9c2414a695aa3eb+        .quad   0xdaa42c4c05a68f21+        .quad   0x7c6c23987f93963e+        .quad   0x210e8cd30c3954e3++        // 2^44 * 5 * G++        .quad   0xac4201f210a71c06+        .quad   0x6a65e0aef3bfb021+        .quad   0xbc42c35c393632f7+        .quad   0x56ea8db1865f0742+        .quad   0x2b50f16137fe6c26+        .quad   0xe102bcd856e404d8+        .quad   0x12b0f1414c561f6b+        .quad   0x51b17bc8d028ec91+        .quad   0xfff5fb4bcf535119+        .quad   0xf4989d79df1108a0+        .quad   0xbdfcea659a3ba325+        .quad   0x18a11f1174d1a6f2++        // 2^44 * 6 * G++        .quad   0x407375ab3f6bba29+        .quad   0x9ec3b6d8991e482e+        .quad   0x99c80e82e55f92e9+        .quad   0x307c13b6fb0c0ae1+        .quad   0xfbd63cdad27a5f2c+        .quad   0xf00fc4bc8aa106d7+        .quad   0x53fb5c1a8e64a430+        .quad   0x04eaabe50c1a2e85+        .quad   0x24751021cb8ab5e7+        .quad   0xfc2344495c5010eb+        .quad   0x5f1e717b4e5610a1+        .quad   0x44da5f18c2710cd5++        // 2^44 * 7 * G++        .quad   0x033cc55ff1b82eb5+        .quad   0xb15ae36d411cae52+        .quad   0xba40b6198ffbacd3+        .quad   0x768edce1532e861f+        .quad   0x9156fe6b89d8eacc+        .quad   0xe6b79451e23126a1+        .quad   0xbd7463d93944eb4e+        .quad   0x726373f6767203ae+        .quad   0xe305ca72eb7ef68a+        .quad   0x662cf31f70eadb23+        .quad   0x18f026fdb4c45b68+        .quad   0x513b5384b5d2ecbd++        // 2^44 * 8 * G++        .quad   0x46d46280c729989e+        .quad   0x4b93fbd05368a5dd+        .quad   0x63df3f81d1765a89+        .quad   0x34cebd64b9a0a223+        .quad   0x5e2702878af34ceb+        .quad   0x900b0409b946d6ae+        .quad   0x6512ebf7dabd8512+        .quad   0x61d9b76988258f81+        .quad   0xa6c5a71349b7d94b+        .quad   0xa3f3d15823eb9446+        .quad   0x0416fbd277484834+        .quad   0x69d45e6f2c70812f++        // 2^48 * 1 * G++        .quad   0xce16f74bc53c1431+        .quad   0x2b9725ce2072edde+        .quad   0xb8b9c36fb5b23ee7+        .quad   0x7e2e0e450b5cc908+        .quad   0x9fe62b434f460efb+        .quad   0xded303d4a63607d6+        .quad   0xf052210eb7a0da24+        .quad   0x237e7dbe00545b93+        .quad   0x013575ed6701b430+        .quad   0x231094e69f0bfd10+        .quad   0x75320f1583e47f22+        .quad   0x71afa699b11155e3++        // 2^48 * 2 * G++        .quad   0x65ce6f9b3953b61d+        .quad   0xc65839eaafa141e6+        .quad   0x0f435ffda9f759fe+        .quad   0x021142e9c2b1c28e+        .quad   0xea423c1c473b50d6+        .quad   0x51e87a1f3b38ef10+        .quad   0x9b84bf5fb2c9be95+        .quad   0x00731fbc78f89a1c+        .quad   0xe430c71848f81880+        .quad   0xbf960c225ecec119+        .quad   0xb6dae0836bba15e3+        .quad   0x4c4d6f3347e15808++        // 2^48 * 3 * G++        .quad   0x18f7eccfc17d1fc9+        .quad   0x6c75f5a651403c14+        .quad   0xdbde712bf7ee0cdf+        .quad   0x193fddaaa7e47a22+        .quad   0x2f0cddfc988f1970+        .quad   0x6b916227b0b9f51b+        .quad   0x6ec7b6c4779176be+        .quad   0x38bf9500a88f9fa8+        .quad   0x1fd2c93c37e8876f+        .quad   0xa2f61e5a18d1462c+        .quad   0x5080f58239241276+        .quad   0x6a6fb99ebf0d4969++        // 2^48 * 4 * G++        .quad   0x6a46c1bb560855eb+        .quad   0x2416bb38f893f09d+        .quad   0xd71d11378f71acc1+        .quad   0x75f76914a31896ea+        .quad   0xeeb122b5b6e423c6+        .quad   0x939d7010f286ff8e+        .quad   0x90a92a831dcf5d8c+        .quad   0x136fda9f42c5eb10+        .quad   0xf94cdfb1a305bdd1+        .quad   0x0f364b9d9ff82c08+        .quad   0x2a87d8a5c3bb588a+        .quad   0x022183510be8dcba++        // 2^48 * 5 * G++        .quad   0x4af766385ead2d14+        .quad   0xa08ed880ca7c5830+        .quad   0x0d13a6e610211e3d+        .quad   0x6a071ce17b806c03+        .quad   0x9d5a710143307a7f+        .quad   0xb063de9ec47da45f+        .quad   0x22bbfe52be927ad3+        .quad   0x1387c441fd40426c+        .quad   0xb5d3c3d187978af8+        .quad   0x722b5a3d7f0e4413+        .quad   0x0d7b4848bb477ca0+        .quad   0x3171b26aaf1edc92++        // 2^48 * 6 * G++        .quad   0xa92f319097564ca8+        .quad   0xff7bb84c2275e119+        .quad   0x4f55fe37a4875150+        .quad   0x221fd4873cf0835a+        .quad   0xa60db7d8b28a47d1+        .quad   0xa6bf14d61770a4f1+        .quad   0xd4a1f89353ddbd58+        .quad   0x6c514a63344243e9+        .quad   0x2322204f3a156341+        .quad   0xfb73e0e9ba0a032d+        .quad   0xfce0dd4c410f030e+        .quad   0x48daa596fb924aaa++        // 2^48 * 7 * G++        .quad   0x6eca8e665ca59cc7+        .quad   0xa847254b2e38aca0+        .quad   0x31afc708d21e17ce+        .quad   0x676dd6fccad84af7+        .quad   0x14f61d5dc84c9793+        .quad   0x9941f9e3ef418206+        .quad   0xcdf5b88f346277ac+        .quad   0x58c837fa0e8a79a9+        .quad   0x0cf9688596fc9058+        .quad   0x1ddcbbf37b56a01b+        .quad   0xdcc2e77d4935d66a+        .quad   0x1c4f73f2c6a57f0a++        // 2^48 * 8 * G++        .quad   0x0e7a4fbd305fa0bb+        .quad   0x829d4ce054c663ad+        .quad   0xf421c3832fe33848+        .quad   0x795ac80d1bf64c42+        .quad   0xb36e706efc7c3484+        .quad   0x73dfc9b4c3c1cf61+        .quad   0xeb1d79c9781cc7e5+        .quad   0x70459adb7daf675c+        .quad   0x1b91db4991b42bb3+        .quad   0x572696234b02dcca+        .quad   0x9fdf9ee51f8c78dc+        .quad   0x5fe162848ce21fd3++        // 2^52 * 1 * G++        .quad   0xe2790aae4d077c41+        .quad   0x8b938270db7469a3+        .quad   0x6eb632dc8abd16a2+        .quad   0x720814ecaa064b72+        .quad   0x315c29c795115389+        .quad   0xd7e0e507862f74ce+        .quad   0x0c4a762185927432+        .quad   0x72de6c984a25a1e4+        .quad   0xae9ab553bf6aa310+        .quad   0x050a50a9806d6e1b+        .quad   0x92bb7403adff5139+        .quad   0x0394d27645be618b++        // 2^52 * 2 * G++        .quad   0x4d572251857eedf4+        .quad   0xe3724edde19e93c5+        .quad   0x8a71420e0b797035+        .quad   0x3b3c833687abe743+        .quad   0xf5396425b23545a4+        .quad   0x15a7a27e98fbb296+        .quad   0xab6c52bc636fdd86+        .quad   0x79d995a8419334ee+        .quad   0xcd8a8ea61195dd75+        .quad   0xa504d8a81dd9a82f+        .quad   0x540dca81a35879b6+        .quad   0x60dd16a379c86a8a++        // 2^52 * 3 * G++        .quad   0x35a2c8487381e559+        .quad   0x596ffea6d78082cb+        .quad   0xcb9771ebdba7b653+        .quad   0x5a08b5019b4da685+        .quad   0x3501d6f8153e47b8+        .quad   0xb7a9675414a2f60c+        .quad   0x112ee8b6455d9523+        .quad   0x4e62a3c18112ea8a+        .quad   0xc8d4ac04516ab786+        .quad   0x595af3215295b23d+        .quad   0xd6edd234db0230c1+        .quad   0x0929efe8825b41cc++        // 2^52 * 4 * G++        .quad   0x5f0601d1cbd0f2d3+        .quad   0x736e412f6132bb7f+        .quad   0x83604432238dde87+        .quad   0x1e3a5272f5c0753c+        .quad   0x8b3172b7ad56651d+        .quad   0x01581b7a3fabd717+        .quad   0x2dc94df6424df6e4+        .quad   0x30376e5d2c29284f+        .quad   0xd2918da78159a59c+        .quad   0x6bdc1cd93f0713f3+        .quad   0x565f7a934acd6590+        .quad   0x53daacec4cb4c128++        // 2^52 * 5 * G++        .quad   0x4ca73bd79cc8a7d6+        .quad   0x4d4a738f47e9a9b2+        .quad   0xf4cbf12942f5fe00+        .quad   0x01a13ff9bdbf0752+        .quad   0x99852bc3852cfdb0+        .quad   0x2cc12e9559d6ed0b+        .quad   0x70f9e2bf9b5ac27b+        .quad   0x4f3b8c117959ae99+        .quad   0x55b6c9c82ff26412+        .quad   0x1ac4a8c91fb667a8+        .quad   0xd527bfcfeb778bf2+        .quad   0x303337da7012a3be++        // 2^52 * 6 * G++        .quad   0x955422228c1c9d7c+        .quad   0x01fac1371a9b340f+        .quad   0x7e8d9177925b48d7+        .quad   0x53f8ad5661b3e31b+        .quad   0x976d3ccbfad2fdd1+        .quad   0xcb88839737a640a8+        .quad   0x2ff00c1d6734cb25+        .quad   0x269ff4dc789c2d2b+        .quad   0x0c003fbdc08d678d+        .quad   0x4d982fa37ead2b17+        .quad   0xc07e6bcdb2e582f1+        .quad   0x296c7291df412a44++        // 2^52 * 7 * G++        .quad   0x7903de2b33daf397+        .quad   0xd0ff0619c9a624b3+        .quad   0x8a1d252b555b3e18+        .quad   0x2b6d581c52e0b7c0+        .quad   0xdfb23205dab8b59e+        .quad   0x465aeaa0c8092250+        .quad   0xd133c1189a725d18+        .quad   0x2327370261f117d1+        .quad   0x3d0543d3623e7986+        .quad   0x679414c2c278a354+        .quad   0xae43f0cc726196f6+        .quad   0x7836c41f8245eaba++        // 2^52 * 8 * G++        .quad   0xe7a254db49e95a81+        .quad   0x5192d5d008b0ad73+        .quad   0x4d20e5b1d00afc07+        .quad   0x5d55f8012cf25f38+        .quad   0xca651e848011937c+        .quad   0xc6b0c46e6ef41a28+        .quad   0xb7021ba75f3f8d52+        .quad   0x119dff99ead7b9fd+        .quad   0x43eadfcbf4b31d4d+        .quad   0xc6503f7411148892+        .quad   0xfeee68c5060d3b17+        .quad   0x329293b3dd4a0ac8++        // 2^56 * 1 * G++        .quad   0x4e59214fe194961a+        .quad   0x49be7dc70d71cd4f+        .quad   0x9300cfd23b50f22d+        .quad   0x4789d446fc917232+        .quad   0x2879852d5d7cb208+        .quad   0xb8dedd70687df2e7+        .quad   0xdc0bffab21687891+        .quad   0x2b44c043677daa35+        .quad   0x1a1c87ab074eb78e+        .quad   0xfac6d18e99daf467+        .quad   0x3eacbbcd484f9067+        .quad   0x60c52eef2bb9a4e4++        // 2^56 * 2 * G++        .quad   0x0b5d89bc3bfd8bf1+        .quad   0xb06b9237c9f3551a+        .quad   0x0e4c16b0d53028f5+        .quad   0x10bc9c312ccfcaab+        .quad   0x702bc5c27cae6d11+        .quad   0x44c7699b54a48cab+        .quad   0xefbc4056ba492eb2+        .quad   0x70d77248d9b6676d+        .quad   0xaa8ae84b3ec2a05b+        .quad   0x98699ef4ed1781e0+        .quad   0x794513e4708e85d1+        .quad   0x63755bd3a976f413++        // 2^56 * 3 * G++        .quad   0xb55fa03e2ad10853+        .quad   0x356f75909ee63569+        .quad   0x9ff9f1fdbe69b890+        .quad   0x0d8cc1c48bc16f84+        .quad   0x3dc7101897f1acb7+        .quad   0x5dda7d5ec165bbd8+        .quad   0x508e5b9c0fa1020f+        .quad   0x2763751737c52a56+        .quad   0x029402d36eb419a9+        .quad   0xf0b44e7e77b460a5+        .quad   0xcfa86230d43c4956+        .quad   0x70c2dd8a7ad166e7++        // 2^56 * 4 * G++        .quad   0x656194509f6fec0e+        .quad   0xee2e7ea946c6518d+        .quad   0x9733c1f367e09b5c+        .quad   0x2e0fac6363948495+        .quad   0x91d4967db8ed7e13+        .quad   0x74252f0ad776817a+        .quad   0xe40982e00d852564+        .quad   0x32b8613816a53ce5+        .quad   0x79e7f7bee448cd64+        .quad   0x6ac83a67087886d0+        .quad   0xf89fd4d9a0e4db2e+        .quad   0x4179215c735a4f41++        // 2^56 * 5 * G++        .quad   0x8c7094e7d7dced2a+        .quad   0x97fb8ac347d39c70+        .quad   0xe13be033a906d902+        .quad   0x700344a30cd99d76+        .quad   0xe4ae33b9286bcd34+        .quad   0xb7ef7eb6559dd6dc+        .quad   0x278b141fb3d38e1f+        .quad   0x31fa85662241c286+        .quad   0xaf826c422e3622f4+        .quad   0xc12029879833502d+        .quad   0x9bc1b7e12b389123+        .quad   0x24bb2312a9952489++        // 2^56 * 6 * G++        .quad   0xb1a8ed1732de67c3+        .quad   0x3cb49418461b4948+        .quad   0x8ebd434376cfbcd2+        .quad   0x0fee3e871e188008+        .quad   0x41f80c2af5f85c6b+        .quad   0x687284c304fa6794+        .quad   0x8945df99a3ba1bad+        .quad   0x0d1d2af9ffeb5d16+        .quad   0xa9da8aa132621edf+        .quad   0x30b822a159226579+        .quad   0x4004197ba79ac193+        .quad   0x16acd79718531d76++        // 2^56 * 7 * G++        .quad   0x72df72af2d9b1d3d+        .quad   0x63462a36a432245a+        .quad   0x3ecea07916b39637+        .quad   0x123e0ef6b9302309+        .quad   0xc959c6c57887b6ad+        .quad   0x94e19ead5f90feba+        .quad   0x16e24e62a342f504+        .quad   0x164ed34b18161700+        .quad   0x487ed94c192fe69a+        .quad   0x61ae2cea3a911513+        .quad   0x877bf6d3b9a4de27+        .quad   0x78da0fc61073f3eb++        // 2^56 * 8 * G++        .quad   0x5bf15d28e52bc66a+        .quad   0x2c47e31870f01a8e+        .quad   0x2419afbc06c28bdd+        .quad   0x2d25deeb256b173a+        .quad   0xa29f80f1680c3a94+        .quad   0x71f77e151ae9e7e6+        .quad   0x1100f15848017973+        .quad   0x054aa4b316b38ddd+        .quad   0xdfc8468d19267cb8+        .quad   0x0b28789c66e54daf+        .quad   0x2aeb1d2a666eec17+        .quad   0x134610a6ab7da760++        // 2^60 * 1 * G++        .quad   0xcaf55ec27c59b23f+        .quad   0x99aeed3e154d04f2+        .quad   0x68441d72e14141f4+        .quad   0x140345133932a0a2+        .quad   0xd91430e0dc028c3c+        .quad   0x0eb955a85217c771+        .quad   0x4b09e1ed2c99a1fa+        .quad   0x42881af2bd6a743c+        .quad   0x7bfec69aab5cad3d+        .quad   0xc23e8cd34cb2cfad+        .quad   0x685dd14bfb37d6a2+        .quad   0x0ad6d64415677a18++        // 2^60 * 2 * G++        .quad   0x781a439e417becb5+        .quad   0x4ac5938cd10e0266+        .quad   0x5da385110692ac24+        .quad   0x11b065a2ade31233+        .quad   0x7914892847927e9f+        .quad   0x33dad6ef370aa877+        .quad   0x1f8f24fa11122703+        .quad   0x5265ac2f2adf9592+        .quad   0x405fdd309afcb346+        .quad   0xd9723d4428e63f54+        .quad   0x94c01df05f65aaae+        .quad   0x43e4dc3ae14c0809++        // 2^60 * 3 * G++        .quad   0xbc12c7f1a938a517+        .quad   0x473028ab3180b2e1+        .quad   0x3f78571efbcd254a+        .quad   0x74e534426ff6f90f+        .quad   0xea6f7ac3adc2c6a3+        .quad   0xd0e928f6e9717c94+        .quad   0xe2d379ead645eaf5+        .quad   0x46dd8785c51ffbbe+        .quad   0x709801be375c8898+        .quad   0x4b06dab5e3fd8348+        .quad   0x75880ced27230714+        .quad   0x2b09468fdd2f4c42++        // 2^60 * 4 * G++        .quad   0x97c749eeb701cb96+        .quad   0x83f438d4b6a369c3+        .quad   0x62962b8b9a402cd9+        .quad   0x6976c7509888df7b+        .quad   0x5b97946582ffa02a+        .quad   0xda096a51fea8f549+        .quad   0xa06351375f77af9b+        .quad   0x1bcfde61201d1e76+        .quad   0x4a4a5490246a59a2+        .quad   0xd63ebddee87fdd90+        .quad   0xd9437c670d2371fa+        .quad   0x69e87308d30f8ed6++        // 2^60 * 5 * G++        .quad   0x435a8bb15656beb0+        .quad   0xf8fac9ba4f4d5bca+        .quad   0xb9b278c41548c075+        .quad   0x3eb0ef76e892b622+        .quad   0x0f80bf028bc80303+        .quad   0x6aae16b37a18cefb+        .quad   0xdd47ea47d72cd6a3+        .quad   0x61943588f4ed39aa+        .quad   0xd26e5c3e91039f85+        .quad   0xc0e9e77df6f33aa9+        .quad   0xe8968c5570066a93+        .quad   0x3c34d1881faaaddd++        // 2^60 * 6 * G++        .quad   0x3f9d2b5ea09f9ec0+        .quad   0x1dab3b6fb623a890+        .quad   0xa09ba3ea72d926c4+        .quad   0x374193513fd8b36d+        .quad   0xbd5b0b8f2fffe0d9+        .quad   0x6aa254103ed24fb9+        .quad   0x2ac7d7bcb26821c4+        .quad   0x605b394b60dca36a+        .quad   0xb4e856e45a9d1ed2+        .quad   0xefe848766c97a9a2+        .quad   0xb104cf641e5eee7d+        .quad   0x2f50b81c88a71c8f++        // 2^60 * 7 * G++        .quad   0x31723c61fc6811bb+        .quad   0x9cb450486211800f+        .quad   0x768933d347995753+        .quad   0x3491a53502752fcd+        .quad   0x2b552ca0a7da522a+        .quad   0x3230b336449b0250+        .quad   0xf2c4c5bca4b99fb9+        .quad   0x7b2c674958074a22+        .quad   0xd55165883ed28cdf+        .quad   0x12d84fd2d362de39+        .quad   0x0a874ad3e3378e4f+        .quad   0x000d2b1f7c763e74++        // 2^60 * 8 * G++        .quad   0x3d420811d06d4a67+        .quad   0xbefc048590e0ffe3+        .quad   0xf870c6b7bd487bde+        .quad   0x6e2a7316319afa28+        .quad   0x9624778c3e94a8ab+        .quad   0x0ad6f3cee9a78bec+        .quad   0x948ac7810d743c4f+        .quad   0x76627935aaecfccc+        .quad   0x56a8ac24d6d59a9f+        .quad   0xc8db753e3096f006+        .quad   0x477f41e68f4c5299+        .quad   0x588d851cf6c86114++        // 2^64 * 1 * G++        .quad   0x51138ec78df6b0fe+        .quad   0x5397da89e575f51b+        .quad   0x09207a1d717af1b9+        .quad   0x2102fdba2b20d650+        .quad   0xcd2a65e777d1f515+        .quad   0x548991878faa60f1+        .quad   0xb1b73bbcdabc06e5+        .quad   0x654878cba97cc9fb+        .quad   0x969ee405055ce6a1+        .quad   0x36bca7681251ad29+        .quad   0x3a1af517aa7da415+        .quad   0x0ad725db29ecb2ba++        // 2^64 * 2 * G++        .quad   0xdc4267b1834e2457+        .quad   0xb67544b570ce1bc5+        .quad   0x1af07a0bf7d15ed7+        .quad   0x4aefcffb71a03650+        .quad   0xfec7bc0c9b056f85+        .quad   0x537d5268e7f5ffd7+        .quad   0x77afc6624312aefa+        .quad   0x4f675f5302399fd9+        .quad   0xc32d36360415171e+        .quad   0xcd2bef118998483b+        .quad   0x870a6eadd0945110+        .quad   0x0bccbb72a2a86561++        // 2^64 * 3 * G++        .quad   0x185e962feab1a9c8+        .quad   0x86e7e63565147dcd+        .quad   0xb092e031bb5b6df2+        .quad   0x4024f0ab59d6b73e+        .quad   0x186d5e4c50fe1296+        .quad   0xe0397b82fee89f7e+        .quad   0x3bc7f6c5507031b0+        .quad   0x6678fd69108f37c2+        .quad   0x1586fa31636863c2+        .quad   0x07f68c48572d33f2+        .quad   0x4f73cc9f789eaefc+        .quad   0x2d42e2108ead4701++        // 2^64 * 4 * G++        .quad   0x97f5131594dfd29b+        .quad   0x6155985d313f4c6a+        .quad   0xeba13f0708455010+        .quad   0x676b2608b8d2d322+        .quad   0x21717b0d0f537593+        .quad   0x914e690b131e064c+        .quad   0x1bb687ae752ae09f+        .quad   0x420bf3a79b423c6e+        .quad   0x8138ba651c5b2b47+        .quad   0x8671b6ec311b1b80+        .quad   0x7bff0cb1bc3135b0+        .quad   0x745d2ffa9c0cf1e0++        // 2^64 * 5 * G++        .quad   0xbf525a1e2bc9c8bd+        .quad   0xea5b260826479d81+        .quad   0xd511c70edf0155db+        .quad   0x1ae23ceb960cf5d0+        .quad   0x6036df5721d34e6a+        .quad   0xb1db8827997bb3d0+        .quad   0xd3c209c3c8756afa+        .quad   0x06e15be54c1dc839+        .quad   0x5b725d871932994a+        .quad   0x32351cb5ceb1dab0+        .quad   0x7dc41549dab7ca05+        .quad   0x58ded861278ec1f7++        // 2^64 * 6 * G++        .quad   0xd8173793f266c55c+        .quad   0xc8c976c5cc454e49+        .quad   0x5ce382f8bc26c3a8+        .quad   0x2ff39de85485f6f9+        .quad   0x2dfb5ba8b6c2c9a8+        .quad   0x48eeef8ef52c598c+        .quad   0x33809107f12d1573+        .quad   0x08ba696b531d5bd8+        .quad   0x77ed3eeec3efc57a+        .quad   0x04e05517d4ff4811+        .quad   0xea3d7a3ff1a671cb+        .quad   0x120633b4947cfe54++        // 2^64 * 7 * G++        .quad   0x0b94987891610042+        .quad   0x4ee7b13cecebfae8+        .quad   0x70be739594f0a4c0+        .quad   0x35d30a99b4d59185+        .quad   0x82bd31474912100a+        .quad   0xde237b6d7e6fbe06+        .quad   0xe11e761911ea79c6+        .quad   0x07433be3cb393bde+        .quad   0xff7944c05ce997f4+        .quad   0x575d3de4b05c51a3+        .quad   0x583381fd5a76847c+        .quad   0x2d873ede7af6da9f++        // 2^64 * 8 * G++        .quad   0x157a316443373409+        .quad   0xfab8b7eef4aa81d9+        .quad   0xb093fee6f5a64806+        .quad   0x2e773654707fa7b6+        .quad   0xaa6202e14e5df981+        .quad   0xa20d59175015e1f5+        .quad   0x18a275d3bae21d6c+        .quad   0x0543618a01600253+        .quad   0x0deabdf4974c23c1+        .quad   0xaa6f0a259dce4693+        .quad   0x04202cb8a29aba2c+        .quad   0x4b1443362d07960d++        // 2^68 * 1 * G++        .quad   0x47b837f753242cec+        .quad   0x256dc48cc04212f2+        .quad   0xe222fbfbe1d928c5+        .quad   0x48ea295bad8a2c07+        .quad   0x299b1c3f57c5715e+        .quad   0x96cb929e6b686d90+        .quad   0x3004806447235ab3+        .quad   0x2c435c24a44d9fe1+        .quad   0x0607c97c80f8833f+        .quad   0x0e851578ca25ec5b+        .quad   0x54f7450b161ebb6f+        .quad   0x7bcb4792a0def80e++        // 2^68 * 2 * G++        .quad   0x8487e3d02bc73659+        .quad   0x4baf8445059979df+        .quad   0xd17c975adcad6fbf+        .quad   0x57369f0bdefc96b6+        .quad   0x1cecd0a0045224c2+        .quad   0x757f1b1b69e53952+        .quad   0x775b7a925289f681+        .quad   0x1b6cc62016736148+        .quad   0xf1a9990175638698+        .quad   0x353dd1beeeaa60d3+        .quad   0x849471334c9ba488+        .quad   0x63fa6e6843ade311++        // 2^68 * 3 * G++        .quad   0xd15c20536597c168+        .quad   0x9f73740098d28789+        .quad   0x18aee7f13257ba1f+        .quad   0x3418bfda07346f14+        .quad   0x2195becdd24b5eb7+        .quad   0x5e41f18cc0cd44f9+        .quad   0xdf28074441ca9ede+        .quad   0x07073b98f35b7d67+        .quad   0xd03c676c4ce530d4+        .quad   0x0b64c0473b5df9f4+        .quad   0x065cef8b19b3a31e+        .quad   0x3084d661533102c9++        // 2^68 * 4 * G++        .quad   0xe1f6b79ebf8469ad+        .quad   0x15801004e2663135+        .quad   0x9a498330af74181b+        .quad   0x3ba2504f049b673c+        .quad   0x9a6ce876760321fd+        .quad   0x7fe2b5109eb63ad8+        .quad   0x00e7d4ae8ac80592+        .quad   0x73d86b7abb6f723a+        .quad   0x0b52b5606dba5ab6+        .quad   0xa9134f0fbbb1edab+        .quad   0x30a9520d9b04a635+        .quad   0x6813b8f37973e5db++        // 2^68 * 5 * G++        .quad   0x9854b054334127c1+        .quad   0x105d047882fbff25+        .quad   0xdb49f7f944186f4f+        .quad   0x1768e838bed0b900+        .quad   0xf194ca56f3157e29+        .quad   0x136d35705ef528a5+        .quad   0xdd4cef778b0599bc+        .quad   0x7d5472af24f833ed+        .quad   0xd0ef874daf33da47+        .quad   0x00d3be5db6e339f9+        .quad   0x3f2a8a2f9c9ceece+        .quad   0x5d1aeb792352435a++        // 2^68 * 6 * G++        .quad   0xf59e6bb319cd63ca+        .quad   0x670c159221d06839+        .quad   0xb06d565b2150cab6+        .quad   0x20fb199d104f12a3+        .quad   0x12c7bfaeb61ba775+        .quad   0xb84e621fe263bffd+        .quad   0x0b47a5c35c840dcf+        .quad   0x7e83be0bccaf8634+        .quad   0x61943dee6d99c120+        .quad   0x86101f2e460b9fe0+        .quad   0x6bb2f1518ee8598d+        .quad   0x76b76289fcc475cc++        // 2^68 * 7 * G++        .quad   0x791b4cc1756286fa+        .quad   0xdbced317d74a157c+        .quad   0x7e732421ea72bde6+        .quad   0x01fe18491131c8e9+        .quad   0x4245f1a1522ec0b3+        .quad   0x558785b22a75656d+        .quad   0x1d485a2548a1b3c0+        .quad   0x60959eccd58fe09f+        .quad   0x3ebfeb7ba8ed7a09+        .quad   0x49fdc2bbe502789c+        .quad   0x44ebce5d3c119428+        .quad   0x35e1eb55be947f4a++        // 2^68 * 8 * G++        .quad   0xdbdae701c5738dd3+        .quad   0xf9c6f635b26f1bee+        .quad   0x61e96a8042f15ef4+        .quad   0x3aa1d11faf60a4d8+        .quad   0x14fd6dfa726ccc74+        .quad   0x3b084cfe2f53b965+        .quad   0xf33ae4f552a2c8b4+        .quad   0x59aab07a0d40166a+        .quad   0x77bcec4c925eac25+        .quad   0x1848718460137738+        .quad   0x5b374337fea9f451+        .quad   0x1865e78ec8e6aa46++        // 2^72 * 1 * G++        .quad   0xccc4b7c7b66e1f7a+        .quad   0x44157e25f50c2f7e+        .quad   0x3ef06dfc713eaf1c+        .quad   0x582f446752da63f7+        .quad   0x967c54e91c529ccb+        .quad   0x30f6269264c635fb+        .quad   0x2747aff478121965+        .quad   0x17038418eaf66f5c+        .quad   0xc6317bd320324ce4+        .quad   0xa81042e8a4488bc4+        .quad   0xb21ef18b4e5a1364+        .quad   0x0c2a1c4bcda28dc9++        // 2^72 * 2 * G++        .quad   0xd24dc7d06f1f0447+        .quad   0xb2269e3edb87c059+        .quad   0xd15b0272fbb2d28f+        .quad   0x7c558bd1c6f64877+        .quad   0xedc4814869bd6945+        .quad   0x0d6d907dbe1c8d22+        .quad   0xc63bd212d55cc5ab+        .quad   0x5a6a9b30a314dc83+        .quad   0xd0ec1524d396463d+        .quad   0x12bb628ac35a24f0+        .quad   0xa50c3a791cbc5fa4+        .quad   0x0404a5ca0afbafc3++        // 2^72 * 3 * G++        .quad   0x8c1f40070aa743d6+        .quad   0xccbad0cb5b265ee8+        .quad   0x574b046b668fd2de+        .quad   0x46395bfdcadd9633+        .quad   0x62bc9e1b2a416fd1+        .quad   0xb5c6f728e350598b+        .quad   0x04343fd83d5d6967+        .quad   0x39527516e7f8ee98+        .quad   0x117fdb2d1a5d9a9c+        .quad   0x9c7745bcd1005c2a+        .quad   0xefd4bef154d56fea+        .quad   0x76579a29e822d016++        // 2^72 * 4 * G++        .quad   0x45b68e7e49c02a17+        .quad   0x23cd51a2bca9a37f+        .quad   0x3ed65f11ec224c1b+        .quad   0x43a384dc9e05bdb1+        .quad   0x333cb51352b434f2+        .quad   0xd832284993de80e1+        .quad   0xb5512887750d35ce+        .quad   0x02c514bb2a2777c1+        .quad   0x684bd5da8bf1b645+        .quad   0xfb8bd37ef6b54b53+        .quad   0x313916d7a9b0d253+        .quad   0x1160920961548059++        // 2^72 * 5 * G++        .quad   0xb44d166929dacfaa+        .quad   0xda529f4c8413598f+        .quad   0xe9ef63ca453d5559+        .quad   0x351e125bc5698e0b+        .quad   0x7a385616369b4dcd+        .quad   0x75c02ca7655c3563+        .quad   0x7dc21bf9d4f18021+        .quad   0x2f637d7491e6e042+        .quad   0xd4b49b461af67bbe+        .quad   0xd603037ac8ab8961+        .quad   0x71dee19ff9a699fb+        .quad   0x7f182d06e7ce2a9a++        // 2^72 * 6 * G++        .quad   0x7a7c8e64ab0168ec+        .quad   0xcb5a4a5515edc543+        .quad   0x095519d347cd0eda+        .quad   0x67d4ac8c343e93b0+        .quad   0x09454b728e217522+        .quad   0xaa58e8f4d484b8d8+        .quad   0xd358254d7f46903c+        .quad   0x44acc043241c5217+        .quad   0x1c7d6bbb4f7a5777+        .quad   0x8b35fed4918313e1+        .quad   0x4adca1c6c96b4684+        .quad   0x556d1c8312ad71bd++        // 2^72 * 7 * G++        .quad   0x17ef40e30c8d3982+        .quad   0x31f7073e15a3fa34+        .quad   0x4f21f3cb0773646e+        .quad   0x746c6c6d1d824eff+        .quad   0x81f06756b11be821+        .quad   0x0faff82310a3f3dd+        .quad   0xf8b2d0556a99465d+        .quad   0x097abe38cc8c7f05+        .quad   0x0c49c9877ea52da4+        .quad   0x4c4369559bdc1d43+        .quad   0x022c3809f7ccebd2+        .quad   0x577e14a34bee84bd++        // 2^72 * 8 * G++        .quad   0xf0e268ac61a73b0a+        .quad   0xf2fafa103791a5f5+        .quad   0xc1e13e826b6d00e9+        .quad   0x60fa7ee96fd78f42+        .quad   0x94fecebebd4dd72b+        .quad   0xf46a4fda060f2211+        .quad   0x124a5977c0c8d1ff+        .quad   0x705304b8fb009295+        .quad   0xb63d1d354d296ec6+        .quad   0xf3c3053e5fad31d8+        .quad   0x670b958cb4bd42ec+        .quad   0x21398e0ca16353fd++        // 2^76 * 1 * G++        .quad   0x216ab2ca8da7d2ef+        .quad   0x366ad9dd99f42827+        .quad   0xae64b9004fdd3c75+        .quad   0x403a395b53909e62+        .quad   0x86c5fc16861b7e9a+        .quad   0xf6a330476a27c451+        .quad   0x01667267a1e93597+        .quad   0x05ffb9cd6082dfeb+        .quad   0xa617fa9ff53f6139+        .quad   0x60f2b5e513e66cb6+        .quad   0xd7a8beefb3448aa4+        .quad   0x7a2932856f5ea192++        // 2^76 * 2 * G++        .quad   0x0b39d761b02de888+        .quad   0x5f550e7ed2414e1f+        .quad   0xa6bfa45822e1a940+        .quad   0x050a2f7dfd447b99+        .quad   0xb89c444879639302+        .quad   0x4ae4f19350c67f2c+        .quad   0xf0b35da8c81af9c6+        .quad   0x39d0003546871017+        .quad   0x437c3b33a650db77+        .quad   0x6bafe81dbac52bb2+        .quad   0xfe99402d2db7d318+        .quad   0x2b5b7eec372ba6ce++        // 2^76 * 3 * G++        .quad   0xb3bc4bbd83f50eef+        .quad   0x508f0c998c927866+        .quad   0x43e76587c8b7e66e+        .quad   0x0f7655a3a47f98d9+        .quad   0xa694404d613ac8f4+        .quad   0x500c3c2bfa97e72c+        .quad   0x874104d21fcec210+        .quad   0x1b205fb38604a8ee+        .quad   0x55ecad37d24b133c+        .quad   0x441e147d6038c90b+        .quad   0x656683a1d62c6fee+        .quad   0x0157d5dc87e0ecae++        // 2^76 * 4 * G++        .quad   0xf2a7af510354c13d+        .quad   0xd7a0b145aa372b60+        .quad   0x2869b96a05a3d470+        .quad   0x6528e42d82460173+        .quad   0x95265514d71eb524+        .quad   0xe603d8815df14593+        .quad   0x147cdf410d4de6b7+        .quad   0x5293b1730437c850+        .quad   0x23d0e0814bccf226+        .quad   0x92c745cd8196fb93+        .quad   0x8b61796c59541e5b+        .quad   0x40a44df0c021f978++        // 2^76 * 5 * G++        .quad   0xdaa869894f20ea6a+        .quad   0xea14a3d14c620618+        .quad   0x6001fccb090bf8be+        .quad   0x35f4e822947e9cf0+        .quad   0x86c96e514bc5d095+        .quad   0xf20d4098fca6804a+        .quad   0x27363d89c826ea5d+        .quad   0x39ca36565719cacf+        .quad   0x97506f2f6f87b75c+        .quad   0xc624aea0034ae070+        .quad   0x1ec856e3aad34dd6+        .quad   0x055b0be0e440e58f++        // 2^76 * 6 * G++        .quad   0x6469a17d89735d12+        .quad   0xdb6f27d5e662b9f1+        .quad   0x9fcba3286a395681+        .quad   0x363b8004d269af25+        .quad   0x4d12a04b6ea33da2+        .quad   0x57cf4c15e36126dd+        .quad   0x90ec9675ee44d967+        .quad   0x64ca348d2a985aac+        .quad   0x99588e19e4c4912d+        .quad   0xefcc3b4e1ca5ce6b+        .quad   0x4522ea60fa5b98d5+        .quad   0x7064bbab1de4a819++        // 2^76 * 7 * G++        .quad   0xb919e1515a770641+        .quad   0xa9a2e2c74e7f8039+        .quad   0x7527250b3df23109+        .quad   0x756a7330ac27b78b+        .quad   0xa290c06142542129+        .quad   0xf2e2c2aebe8d5b90+        .quad   0xcf2458db76abfe1b+        .quad   0x02157ade83d626bf+        .quad   0x3e46972a1b9a038b+        .quad   0x2e4ee66a7ee03fb4+        .quad   0x81a248776edbb4ca+        .quad   0x1a944ee88ecd0563++        // 2^76 * 8 * G++        .quad   0xd5a91d1151039372+        .quad   0x2ed377b799ca26de+        .quad   0xa17202acfd366b6b+        .quad   0x0730291bd6901995+        .quad   0xbb40a859182362d6+        .quad   0xb99f55778a4d1abb+        .quad   0x8d18b427758559f6+        .quad   0x26c20fe74d26235a+        .quad   0x648d1d9fe9cc22f5+        .quad   0x66bc561928dd577c+        .quad   0x47d3ed21652439d1+        .quad   0x49d271acedaf8b49++        // 2^80 * 1 * G++        .quad   0x89f5058a382b33f3+        .quad   0x5ae2ba0bad48c0b4+        .quad   0x8f93b503a53db36e+        .quad   0x5aa3ed9d95a232e6+        .quad   0x2798aaf9b4b75601+        .quad   0x5eac72135c8dad72+        .quad   0xd2ceaa6161b7a023+        .quad   0x1bbfb284e98f7d4e+        .quad   0x656777e9c7d96561+        .quad   0xcb2b125472c78036+        .quad   0x65053299d9506eee+        .quad   0x4a07e14e5e8957cc++        // 2^80 * 2 * G++        .quad   0x4ee412cb980df999+        .quad   0xa315d76f3c6ec771+        .quad   0xbba5edde925c77fd+        .quad   0x3f0bac391d313402+        .quad   0x240b58cdc477a49b+        .quad   0xfd38dade6447f017+        .quad   0x19928d32a7c86aad+        .quad   0x50af7aed84afa081+        .quad   0x6e4fde0115f65be5+        .quad   0x29982621216109b2+        .quad   0x780205810badd6d9+        .quad   0x1921a316baebd006++        // 2^80 * 3 * G++        .quad   0x89422f7edfb870fc+        .quad   0x2c296beb4f76b3bd+        .quad   0x0738f1d436c24df7+        .quad   0x6458df41e273aeb0+        .quad   0xd75aad9ad9f3c18b+        .quad   0x566a0eef60b1c19c+        .quad   0x3e9a0bac255c0ed9+        .quad   0x7b049deca062c7f5+        .quad   0xdccbe37a35444483+        .quad   0x758879330fedbe93+        .quad   0x786004c312c5dd87+        .quad   0x6093dccbc2950e64++        // 2^80 * 4 * G++        .quad   0x1ff39a8585e0706d+        .quad   0x36d0a5d8b3e73933+        .quad   0x43b9f2e1718f453b+        .quad   0x57d1ea084827a97c+        .quad   0x6bdeeebe6084034b+        .quad   0x3199c2b6780fb854+        .quad   0x973376abb62d0695+        .quad   0x6e3180c98b647d90+        .quad   0xee7ab6e7a128b071+        .quad   0xa4c1596d93a88baa+        .quad   0xf7b4de82b2216130+        .quad   0x363e999ddd97bd18++        // 2^80 * 5 * G++        .quad   0x96a843c135ee1fc4+        .quad   0x976eb35508e4c8cf+        .quad   0xb42f6801b58cd330+        .quad   0x48ee9b78693a052b+        .quad   0x2f1848dce24baec6+        .quad   0x769b7255babcaf60+        .quad   0x90cb3c6e3cefe931+        .quad   0x231f979bc6f9b355+        .quad   0x5c31de4bcc2af3c6+        .quad   0xb04bb030fe208d1f+        .quad   0xb78d7009c14fb466+        .quad   0x079bfa9b08792413++        // 2^80 * 6 * G++        .quad   0xe3903a51da300df4+        .quad   0x843964233da95ab0+        .quad   0xed3cf12d0b356480+        .quad   0x038c77f684817194+        .quad   0xf3c9ed80a2d54245+        .quad   0x0aa08b7877f63952+        .quad   0xd76dac63d1085475+        .quad   0x1ef4fb159470636b+        .quad   0x854e5ee65b167bec+        .quad   0x59590a4296d0cdc2+        .quad   0x72b2df3498102199+        .quad   0x575ee92a4a0bff56++        // 2^80 * 7 * G++        .quad   0xd4c080908a182fcf+        .quad   0x30e170c299489dbd+        .quad   0x05babd5752f733de+        .quad   0x43d4e7112cd3fd00+        .quad   0x5d46bc450aa4d801+        .quad   0xc3af1227a533b9d8+        .quad   0x389e3b262b8906c2+        .quad   0x200a1e7e382f581b+        .quad   0x518db967eaf93ac5+        .quad   0x71bc989b056652c0+        .quad   0xfe2b85d9567197f5+        .quad   0x050eca52651e4e38++        // 2^80 * 8 * G++        .quad   0xc3431ade453f0c9c+        .quad   0xe9f5045eff703b9b+        .quad   0xfcd97ac9ed847b3d+        .quad   0x4b0ee6c21c58f4c6+        .quad   0x97ac397660e668ea+        .quad   0x9b19bbfe153ab497+        .quad   0x4cb179b534eca79f+        .quad   0x6151c09fa131ae57+        .quad   0x3af55c0dfdf05d96+        .quad   0xdd262ee02ab4ee7a+        .quad   0x11b2bb8712171709+        .quad   0x1fef24fa800f030b++        // 2^84 * 1 * G++        .quad   0xb496123a6b6c6609+        .quad   0xa750fe8580ab5938+        .quad   0xf471bf39b7c27a5f+        .quad   0x507903ce77ac193c+        .quad   0xff91a66a90166220+        .quad   0xf22552ae5bf1e009+        .quad   0x7dff85d87f90df7c+        .quad   0x4f620ffe0c736fb9+        .quad   0x62f90d65dfde3e34+        .quad   0xcf28c592b9fa5fad+        .quad   0x99c86ef9c6164510+        .quad   0x25d448044a256c84++        // 2^84 * 2 * G++        .quad   0xbd68230ec7e9b16f+        .quad   0x0eb1b9c1c1c5795d+        .quad   0x7943c8c495b6b1ff+        .quad   0x2f9faf620bbacf5e+        .quad   0x2c7c4415c9022b55+        .quad   0x56a0d241812eb1fe+        .quad   0xf02ea1c9d7b65e0d+        .quad   0x4180512fd5323b26+        .quad   0xa4ff3e698a48a5db+        .quad   0xba6a3806bd95403b+        .quad   0x9f7ce1af47d5b65d+        .quad   0x15e087e55939d2fb++        // 2^84 * 3 * G++        .quad   0x12207543745c1496+        .quad   0xdaff3cfdda38610c+        .quad   0xe4e797272c71c34f+        .quad   0x39c07b1934bdede9+        .quad   0x8894186efb963f38+        .quad   0x48a00e80dc639bd5+        .quad   0xa4e8092be96c1c99+        .quad   0x5a097d54ca573661+        .quad   0x2d45892b17c9e755+        .quad   0xd033fd7289308df8+        .quad   0x6c2fe9d9525b8bd9+        .quad   0x2edbecf1c11cc079++        // 2^84 * 4 * G++        .quad   0x1616a4e3c715a0d2+        .quad   0x53623cb0f8341d4d+        .quad   0x96ef5329c7e899cb+        .quad   0x3d4e8dbba668baa6+        .quad   0xee0f0fddd087a25f+        .quad   0x9c7531555c3e34ee+        .quad   0x660c572e8fab3ab5+        .quad   0x0854fc44544cd3b2+        .quad   0x61eba0c555edad19+        .quad   0x24b533fef0a83de6+        .quad   0x3b77042883baa5f8+        .quad   0x678f82b898a47e8d++        // 2^84 * 5 * G++        .quad   0xb1491d0bd6900c54+        .quad   0x3539722c9d132636+        .quad   0x4db928920b362bc9+        .quad   0x4d7cd1fea68b69df+        .quad   0x1e09d94057775696+        .quad   0xeed1265c3cd951db+        .quad   0xfa9dac2b20bce16f+        .quad   0x0f7f76e0e8d089f4+        .quad   0x36d9ebc5d485b00c+        .quad   0xa2596492e4adb365+        .quad   0xc1659480c2119ccd+        .quad   0x45306349186e0d5f++        // 2^84 * 6 * G++        .quad   0x94ddd0c1a6cdff1d+        .quad   0x55f6f115e84213ae+        .quad   0x6c935f85992fcf6a+        .quad   0x067ee0f54a37f16f+        .quad   0x96a414ec2b072491+        .quad   0x1bb2218127a7b65b+        .quad   0x6d2849596e8a4af0+        .quad   0x65f3b08ccd27765f+        .quad   0xecb29fff199801f7+        .quad   0x9d361d1fa2a0f72f+        .quad   0x25f11d2375fd2f49+        .quad   0x124cefe80fe10fe2++        // 2^84 * 7 * G++        .quad   0x4c126cf9d18df255+        .quad   0xc1d471e9147a63b6+        .quad   0x2c6d3c73f3c93b5f+        .quad   0x6be3a6a2e3ff86a2+        .quad   0x1518e85b31b16489+        .quad   0x8faadcb7db710bfb+        .quad   0x39b0bdf4a14ae239+        .quad   0x05f4cbea503d20c1+        .quad   0xce040e9ec04145bc+        .quad   0xc71ff4e208f6834c+        .quad   0xbd546e8dab8847a3+        .quad   0x64666aa0a4d2aba5++        // 2^84 * 8 * G++        .quad   0x6841435a7c06d912+        .quad   0xca123c21bb3f830b+        .quad   0xd4b37b27b1cbe278+        .quad   0x1d753b84c76f5046+        .quad   0xb0c53bf73337e94c+        .quad   0x7cb5697e11e14f15+        .quad   0x4b84abac1930c750+        .quad   0x28dd4abfe0640468+        .quad   0x7dc0b64c44cb9f44+        .quad   0x18a3e1ace3925dbf+        .quad   0x7a3034862d0457c4+        .quad   0x4c498bf78a0c892e++        // 2^88 * 1 * G++        .quad   0x37d653fb1aa73196+        .quad   0x0f9495303fd76418+        .quad   0xad200b09fb3a17b2+        .quad   0x544d49292fc8613e+        .quad   0x22d2aff530976b86+        .quad   0x8d90b806c2d24604+        .quad   0xdca1896c4de5bae5+        .quad   0x28005fe6c8340c17+        .quad   0x6aefba9f34528688+        .quad   0x5c1bff9425107da1+        .quad   0xf75bbbcd66d94b36+        .quad   0x72e472930f316dfa++        // 2^88 * 2 * G++        .quad   0x2695208c9781084f+        .quad   0xb1502a0b23450ee1+        .quad   0xfd9daea603efde02+        .quad   0x5a9d2e8c2733a34c+        .quad   0x07f3f635d32a7627+        .quad   0x7aaa4d865f6566f0+        .quad   0x3c85e79728d04450+        .quad   0x1fee7f000fe06438+        .quad   0x765305da03dbf7e5+        .quad   0xa4daf2491434cdbd+        .quad   0x7b4ad5cdd24a88ec+        .quad   0x00f94051ee040543++        // 2^88 * 3 * G++        .quad   0x8d356b23c3d330b2+        .quad   0xf21c8b9bb0471b06+        .quad   0xb36c316c6e42b83c+        .quad   0x07d79c7e8beab10d+        .quad   0xd7ef93bb07af9753+        .quad   0x583ed0cf3db766a7+        .quad   0xce6998bf6e0b1ec5+        .quad   0x47b7ffd25dd40452+        .quad   0x87fbfb9cbc08dd12+        .quad   0x8a066b3ae1eec29b+        .quad   0x0d57242bdb1fc1bf+        .quad   0x1c3520a35ea64bb6++        // 2^88 * 4 * G++        .quad   0x80d253a6bccba34a+        .quad   0x3e61c3a13838219b+        .quad   0x90c3b6019882e396+        .quad   0x1c3d05775d0ee66f+        .quad   0xcda86f40216bc059+        .quad   0x1fbb231d12bcd87e+        .quad   0xb4956a9e17c70990+        .quad   0x38750c3b66d12e55+        .quad   0x692ef1409422e51a+        .quad   0xcbc0c73c2b5df671+        .quad   0x21014fe7744ce029+        .quad   0x0621e2c7d330487c++        // 2^88 * 5 * G++        .quad   0xaf9860cc8259838d+        .quad   0x90ea48c1c69f9adc+        .quad   0x6526483765581e30+        .quad   0x0007d6097bd3a5bc+        .quad   0xb7ae1796b0dbf0f3+        .quad   0x54dfafb9e17ce196+        .quad   0x25923071e9aaa3b4+        .quad   0x5d8e589ca1002e9d+        .quad   0xc0bf1d950842a94b+        .quad   0xb2d3c363588f2e3e+        .quad   0x0a961438bb51e2ef+        .quad   0x1583d7783c1cbf86++        // 2^88 * 6 * G++        .quad   0xeceea2ef5da27ae1+        .quad   0x597c3a1455670174+        .quad   0xc9a62a126609167a+        .quad   0x252a5f2e81ed8f70+        .quad   0x90034704cc9d28c7+        .quad   0x1d1b679ef72cc58f+        .quad   0x16e12b5fbe5b8726+        .quad   0x4958064e83c5580a+        .quad   0x0d2894265066e80d+        .quad   0xfcc3f785307c8c6b+        .quad   0x1b53da780c1112fd+        .quad   0x079c170bd843b388++        // 2^88 * 7 * G++        .quad   0x0506ece464fa6fff+        .quad   0xbee3431e6205e523+        .quad   0x3579422451b8ea42+        .quad   0x6dec05e34ac9fb00+        .quad   0xcdd6cd50c0d5d056+        .quad   0x9af7686dbb03573b+        .quad   0x3ca6723ff3c3ef48+        .quad   0x6768c0d7317b8acc+        .quad   0x94b625e5f155c1b3+        .quad   0x417bf3a7997b7b91+        .quad   0xc22cbddc6d6b2600+        .quad   0x51445e14ddcd52f4++        // 2^88 * 8 * G++        .quad   0x57502b4b3b144951+        .quad   0x8e67ff6b444bbcb3+        .quad   0xb8bd6927166385db+        .quad   0x13186f31e39295c8+        .quad   0x893147ab2bbea455+        .quad   0x8c53a24f92079129+        .quad   0x4b49f948be30f7a7+        .quad   0x12e990086e4fd43d+        .quad   0xf10c96b37fdfbb2e+        .quad   0x9f9a935e121ceaf9+        .quad   0xdf1136c43a5b983f+        .quad   0x77b2e3f05d3e99af++        // 2^92 * 1 * G++        .quad   0xfd0d75879cf12657+        .quad   0xe82fef94e53a0e29+        .quad   0xcc34a7f05bbb4be7+        .quad   0x0b251172a50c38a2+        .quad   0x9532f48fcc5cd29b+        .quad   0x2ba851bea3ce3671+        .quad   0x32dacaa051122941+        .quad   0x478d99d9350004f2+        .quad   0x1d5ad94890bb02c0+        .quad   0x50e208b10ec25115+        .quad   0xa26a22894ef21702+        .quad   0x4dc923343b524805++        // 2^92 * 2 * G++        .quad   0xe3828c400f8086b6+        .quad   0x3f77e6f7979f0dc8+        .quad   0x7ef6de304df42cb4+        .quad   0x5265797cb6abd784+        .quad   0x3ad3e3ebf36c4975+        .quad   0xd75d25a537862125+        .quad   0xe873943da025a516+        .quad   0x6bbc7cb4c411c847+        .quad   0x3c6f9cd1d4a50d56+        .quad   0xb6244077c6feab7e+        .quad   0x6ff9bf483580972e+        .quad   0x00375883b332acfb++        // 2^92 * 3 * G++        .quad   0x0001b2cd28cb0940+        .quad   0x63fb51a06f1c24c9+        .quad   0xb5ad8691dcd5ca31+        .quad   0x67238dbd8c450660+        .quad   0xc98bec856c75c99c+        .quad   0xe44184c000e33cf4+        .quad   0x0a676b9bba907634+        .quad   0x669e2cb571f379d7+        .quad   0xcb116b73a49bd308+        .quad   0x025aad6b2392729e+        .quad   0xb4793efa3f55d9b1+        .quad   0x72a1056140678bb9++        // 2^92 * 4 * G++        .quad   0xa2b6812b1cc9249d+        .quad   0x62866eee21211f58+        .quad   0x2cb5c5b85df10ece+        .quad   0x03a6b259e263ae00+        .quad   0x0d8d2909e2e505b6+        .quad   0x98ca78abc0291230+        .quad   0x77ef5569a9b12327+        .quad   0x7c77897b81439b47+        .quad   0xf1c1b5e2de331cb5+        .quad   0x5a9f5d8e15fca420+        .quad   0x9fa438f17bd932b1+        .quad   0x2a381bf01c6146e7++        // 2^92 * 5 * G++        .quad   0xac9b9879cfc811c1+        .quad   0x8b7d29813756e567+        .quad   0x50da4e607c70edfc+        .quad   0x5dbca62f884400b6+        .quad   0xf7c0be32b534166f+        .quad   0x27e6ca6419cf70d4+        .quad   0x934df7d7a957a759+        .quad   0x5701461dabdec2aa+        .quad   0x2c6747402c915c25+        .quad   0x1bdcd1a80b0d340a+        .quad   0x5e5601bd07b43f5f+        .quad   0x2555b4e05539a242++        // 2^92 * 6 * G++        .quad   0x6fc09f5266ddd216+        .quad   0xdce560a7c8e37048+        .quad   0xec65939da2df62fd+        .quad   0x7a869ae7e52ed192+        .quad   0x78409b1d87e463d4+        .quad   0xad4da95acdfb639d+        .quad   0xec28773755259b9c+        .quad   0x69c806e9c31230ab+        .quad   0x7b48f57414bb3f22+        .quad   0x68c7cee4aedccc88+        .quad   0xed2f936179ed80be+        .quad   0x25d70b885f77bc4b++        // 2^92 * 7 * G++        .quad   0x4151c3d9762bf4de+        .quad   0x083f435f2745d82b+        .quad   0x29775a2e0d23ddd5+        .quad   0x138e3a6269a5db24+        .quad   0x98459d29bb1ae4d4+        .quad   0x56b9c4c739f954ec+        .quad   0x832743f6c29b4b3e+        .quad   0x21ea8e2798b6878a+        .quad   0x87bef4b46a5a7b9c+        .quad   0xd2299d1b5fc1d062+        .quad   0x82409818dd321648+        .quad   0x5c5abeb1e5a2e03d++        // 2^92 * 8 * G++        .quad   0x14722af4b73c2ddb+        .quad   0xbc470c5f5a05060d+        .quad   0x00943eac2581b02e+        .quad   0x0e434b3b1f499c8f+        .quad   0x02cde6de1306a233+        .quad   0x7b5a52a2116f8ec7+        .quad   0xe1c681f4c1163b5b+        .quad   0x241d350660d32643+        .quad   0x6be4404d0ebc52c7+        .quad   0xae46233bb1a791f5+        .quad   0x2aec170ed25db42b+        .quad   0x1d8dfd966645d694++        // 2^96 * 1 * G++        .quad   0x296fa9c59c2ec4de+        .quad   0xbc8b61bf4f84f3cb+        .quad   0x1c7706d917a8f908+        .quad   0x63b795fc7ad3255d+        .quad   0xd598639c12ddb0a4+        .quad   0xa5d19f30c024866b+        .quad   0xd17c2f0358fce460+        .quad   0x07a195152e095e8a+        .quad   0xa8368f02389e5fc8+        .quad   0x90433b02cf8de43b+        .quad   0xafa1fd5dc5412643+        .quad   0x3e8fe83d032f0137++        // 2^96 * 2 * G++        .quad   0x2f8b15b90570a294+        .quad   0x94f2427067084549+        .quad   0xde1c5ae161bbfd84+        .quad   0x75ba3b797fac4007+        .quad   0x08704c8de8efd13c+        .quad   0xdfc51a8e33e03731+        .quad   0xa59d5da51260cde3+        .quad   0x22d60899a6258c86+        .quad   0x6239dbc070cdd196+        .quad   0x60fe8a8b6c7d8a9a+        .quad   0xb38847bceb401260+        .quad   0x0904d07b87779e5e++        // 2^96 * 3 * G++        .quad   0xb4ce1fd4ddba919c+        .quad   0xcf31db3ec74c8daa+        .quad   0x2c63cc63ad86cc51+        .quad   0x43e2143fbc1dde07+        .quad   0xf4322d6648f940b9+        .quad   0x06952f0cbd2d0c39+        .quad   0x167697ada081f931+        .quad   0x6240aacebaf72a6c+        .quad   0xf834749c5ba295a0+        .quad   0xd6947c5bca37d25a+        .quad   0x66f13ba7e7c9316a+        .quad   0x56bdaf238db40cac++        // 2^96 * 4 * G++        .quad   0x362ab9e3f53533eb+        .quad   0x338568d56eb93d40+        .quad   0x9e0e14521d5a5572+        .quad   0x1d24a86d83741318+        .quad   0x1310d36cc19d3bb2+        .quad   0x062a6bb7622386b9+        .quad   0x7c9b8591d7a14f5c+        .quad   0x03aa31507e1e5754+        .quad   0xf4ec7648ffd4ce1f+        .quad   0xe045eaf054ac8c1c+        .quad   0x88d225821d09357c+        .quad   0x43b261dc9aeb4859++        // 2^96 * 5 * G++        .quad   0xe55b1e1988bb79bb+        .quad   0xa09ed07dc17a359d+        .quad   0xb02c2ee2603dea33+        .quad   0x326055cf5b276bc2+        .quad   0x19513d8b6c951364+        .quad   0x94fe7126000bf47b+        .quad   0x028d10ddd54f9567+        .quad   0x02b4d5e242940964+        .quad   0xb4a155cb28d18df2+        .quad   0xeacc4646186ce508+        .quad   0xc49cf4936c824389+        .quad   0x27a6c809ae5d3410++        // 2^96 * 6 * G++        .quad   0x8ba6ebcd1f0db188+        .quad   0x37d3d73a675a5be8+        .quad   0xf22edfa315f5585a+        .quad   0x2cb67174ff60a17e+        .quad   0xcd2c270ac43d6954+        .quad   0xdd4a3e576a66cab2+        .quad   0x79fa592469d7036c+        .quad   0x221503603d8c2599+        .quad   0x59eecdf9390be1d0+        .quad   0xa9422044728ce3f1+        .quad   0x82891c667a94f0f4+        .quad   0x7b1df4b73890f436++        // 2^96 * 7 * G++        .quad   0xe492f2e0b3b2a224+        .quad   0x7c6c9e062b551160+        .quad   0x15eb8fe20d7f7b0e+        .quad   0x61fcef2658fc5992+        .quad   0x5f2e221807f8f58c+        .quad   0xe3555c9fd49409d4+        .quad   0xb2aaa88d1fb6a630+        .quad   0x68698245d352e03d+        .quad   0xdbb15d852a18187a+        .quad   0xf3e4aad386ddacd7+        .quad   0x44bae2810ff6c482+        .quad   0x46cf4c473daf01cf++        // 2^96 * 8 * G++        .quad   0x426525ed9ec4e5f9+        .quad   0x0e5eda0116903303+        .quad   0x72b1a7f2cbe5cadc+        .quad   0x29387bcd14eb5f40+        .quad   0x213c6ea7f1498140+        .quad   0x7c1e7ef8392b4854+        .quad   0x2488c38c5629ceba+        .quad   0x1065aae50d8cc5bb+        .quad   0x1c2c4525df200d57+        .quad   0x5c3b2dd6bfca674a+        .quad   0x0a07e7b1e1834030+        .quad   0x69a198e64f1ce716++        // 2^100 * 1 * G++        .quad   0x7afcd613efa9d697+        .quad   0x0cc45aa41c067959+        .quad   0xa56fe104c1fada96+        .quad   0x3a73b70472e40365+        .quad   0x7b26e56b9e2d4734+        .quad   0xc4c7132b81c61675+        .quad   0xef5c9525ec9cde7f+        .quad   0x39c80b16e71743ad+        .quad   0x0f196e0d1b826c68+        .quad   0xf71ff0e24960e3db+        .quad   0x6113167023b7436c+        .quad   0x0cf0ea5877da7282++        // 2^100 * 2 * G++        .quad   0x196c80a4ddd4ccbd+        .quad   0x22e6f55d95f2dd9d+        .quad   0xc75e33c740d6c71b+        .quad   0x7bb51279cb3c042f+        .quad   0xe332ced43ba6945a+        .quad   0xde0b1361e881c05d+        .quad   0x1ad40f095e67ed3b+        .quad   0x5da8acdab8c63d5d+        .quad   0xc4b6664a3a70159f+        .quad   0x76194f0f0a904e14+        .quad   0xa5614c39a4096c13+        .quad   0x6cd0ff50979feced++        // 2^100 * 3 * G++        .quad   0xc0e067e78f4428ac+        .quad   0x14835ab0a61135e3+        .quad   0xf21d14f338062935+        .quad   0x6390a4c8df04849c+        .quad   0x7fecfabdb04ba18e+        .quad   0xd0fc7bfc3bddbcf7+        .quad   0xa41d486e057a131c+        .quad   0x641a4391f2223a61+        .quad   0xc5c6b95aa606a8db+        .quad   0x914b7f9eb06825f1+        .quad   0x2a731f6b44fc9eff+        .quad   0x30ddf38562705cfc++        // 2^100 * 4 * G++        .quad   0x4e3dcbdad1bff7f9+        .quad   0xc9118e8220645717+        .quad   0xbacccebc0f189d56+        .quad   0x1b4822e9d4467668+        .quad   0x33bef2bd68bcd52c+        .quad   0xc649dbb069482ef2+        .quad   0xb5b6ee0c41cb1aee+        .quad   0x5c294d270212a7e5+        .quad   0xab360a7f25563781+        .quad   0x2512228a480f7958+        .quad   0xc75d05276114b4e3+        .quad   0x222d9625d976fe2a++        // 2^100 * 5 * G++        .quad   0x1c717f85b372ace1+        .quad   0x81930e694638bf18+        .quad   0x239cad056bc08b58+        .quad   0x0b34271c87f8fff4+        .quad   0x0f94be7e0a344f85+        .quad   0xeb2faa8c87f22c38+        .quad   0x9ce1e75e4ee16f0f+        .quad   0x43e64e5418a08dea+        .quad   0x8155e2521a35ce63+        .quad   0xbe100d4df912028e+        .quad   0xbff80bf8a57ddcec+        .quad   0x57342dc96d6bc6e4++        // 2^100 * 6 * G++        .quad   0xefeef065c8ce5998+        .quad   0xbf029510b5cbeaa2+        .quad   0x8c64a10620b7c458+        .quad   0x35134fb231c24855+        .quad   0xf3c3bcb71e707bf6+        .quad   0x351d9b8c7291a762+        .quad   0x00502e6edad69a33+        .quad   0x522f521f1ec8807f+        .quad   0x272c1f46f9a3902b+        .quad   0xc91ba3b799657bcc+        .quad   0xae614b304f8a1c0e+        .quad   0x7afcaad70b99017b++        // 2^100 * 7 * G++        .quad   0xc25ded54a4b8be41+        .quad   0x902d13e11bb0e2dd+        .quad   0x41f43233cde82ab2+        .quad   0x1085faa5c3aae7cb+        .quad   0xa88141ecef842b6b+        .quad   0x55e7b14797abe6c5+        .quad   0x8c748f9703784ffe+        .quad   0x5b50a1f7afcd00b7+        .quad   0x9b840f66f1361315+        .quad   0x18462242701003e9+        .quad   0x65ed45fae4a25080+        .quad   0x0a2862393fda7320++        // 2^100 * 8 * G++        .quad   0x46ab13c8347cbc9d+        .quad   0x3849e8d499c12383+        .quad   0x4cea314087d64ac9+        .quad   0x1f354134b1a29ee7+        .quad   0x960e737b6ecb9d17+        .quad   0xfaf24948d67ceae1+        .quad   0x37e7a9b4d55e1b89+        .quad   0x5cb7173cb46c59eb+        .quad   0x4a89e68b82b7abf0+        .quad   0xf41cd9279ba6b7b9+        .quad   0x16e6c210e18d876f+        .quad   0x7cacdb0f7f1b09c6++        // 2^104 * 1 * G++        .quad   0x9062b2e0d91a78bc+        .quad   0x47c9889cc8509667+        .quad   0x9df54a66405070b8+        .quad   0x7369e6a92493a1bf+        .quad   0xe1014434dcc5caed+        .quad   0x47ed5d963c84fb33+        .quad   0x70019576ed86a0e7+        .quad   0x25b2697bd267f9e4+        .quad   0x9d673ffb13986864+        .quad   0x3ca5fbd9415dc7b8+        .quad   0xe04ecc3bdf273b5e+        .quad   0x1420683db54e4cd2++        // 2^104 * 2 * G++        .quad   0xb478bd1e249dd197+        .quad   0x620c35005e58c102+        .quad   0xfb02d32fccbaac5c+        .quad   0x60b63bebf508a72d+        .quad   0x34eebb6fc1cc5ad0+        .quad   0x6a1b0ce99646ac8b+        .quad   0xd3b0da49a66bde53+        .quad   0x31e83b4161d081c1+        .quad   0x97e8c7129e062b4f+        .quad   0x49e48f4f29320ad8+        .quad   0x5bece14b6f18683f+        .quad   0x55cf1eb62d550317++        // 2^104 * 3 * G++        .quad   0x5879101065c23d58+        .quad   0x8b9d086d5094819c+        .quad   0xe2402fa912c55fa7+        .quad   0x669a6564570891d4+        .quad   0x3076b5e37df58c52+        .quad   0xd73ab9dde799cc36+        .quad   0xbd831ce34913ee20+        .quad   0x1a56fbaa62ba0133+        .quad   0x943e6b505c9dc9ec+        .quad   0x302557bba77c371a+        .quad   0x9873ae5641347651+        .quad   0x13c4836799c58a5c++        // 2^104 * 4 * G++        .quad   0x423a5d465ab3e1b9+        .quad   0xfc13c187c7f13f61+        .quad   0x19f83664ecb5b9b6+        .quad   0x66f80c93a637b607+        .quad   0xc4dcfb6a5d8bd080+        .quad   0xdeebc4ec571a4842+        .quad   0xd4b2e883b8e55365+        .quad   0x50bdc87dc8e5b827+        .quad   0x606d37836edfe111+        .quad   0x32353e15f011abd9+        .quad   0x64b03ac325b73b96+        .quad   0x1dd56444725fd5ae++        // 2^104 * 5 * G++        .quad   0x8fa47ff83362127d+        .quad   0xbc9f6ac471cd7c15+        .quad   0x6e71454349220c8b+        .quad   0x0e645912219f732e+        .quad   0xc297e60008bac89a+        .quad   0x7d4cea11eae1c3e0+        .quad   0xf3e38be19fe7977c+        .quad   0x3a3a450f63a305cd+        .quad   0x078f2f31d8394627+        .quad   0x389d3183de94a510+        .quad   0xd1e36c6d17996f80+        .quad   0x318c8d9393a9a87b++        // 2^104 * 6 * G++        .quad   0xf2745d032afffe19+        .quad   0x0c9f3c497f24db66+        .quad   0xbc98d3e3ba8598ef+        .quad   0x224c7c679a1d5314+        .quad   0x5d669e29ab1dd398+        .quad   0xfc921658342d9e3b+        .quad   0x55851dfdf35973cd+        .quad   0x509a41c325950af6+        .quad   0xbdc06edca6f925e9+        .quad   0x793ef3f4641b1f33+        .quad   0x82ec12809d833e89+        .quad   0x05bff02328a11389++        // 2^104 * 7 * G++        .quad   0x3632137023cae00b+        .quad   0x544acf0ad1accf59+        .quad   0x96741049d21a1c88+        .quad   0x780b8cc3fa2a44a7+        .quad   0x6881a0dd0dc512e4+        .quad   0x4fe70dc844a5fafe+        .quad   0x1f748e6b8f4a5240+        .quad   0x576277cdee01a3ea+        .quad   0x1ef38abc234f305f+        .quad   0x9a577fbd1405de08+        .quad   0x5e82a51434e62a0d+        .quad   0x5ff418726271b7a1++        // 2^104 * 8 * G++        .quad   0x398e080c1789db9d+        .quad   0xa7602025f3e778f5+        .quad   0xfa98894c06bd035d+        .quad   0x106a03dc25a966be+        .quad   0xe5db47e813b69540+        .quad   0xf35d2a3b432610e1+        .quad   0xac1f26e938781276+        .quad   0x29d4db8ca0a0cb69+        .quad   0xd9ad0aaf333353d0+        .quad   0x38669da5acd309e5+        .quad   0x3c57658ac888f7f0+        .quad   0x4ab38a51052cbefa++        // 2^108 * 1 * G++        .quad   0xdfdacbee4324c0e9+        .quad   0x054442883f955bb7+        .quad   0xdef7aaa8ea31609f+        .quad   0x68aee70642287cff+        .quad   0xf68fe2e8809de054+        .quad   0xe3bc096a9c82bad1+        .quad   0x076353d40aadbf45+        .quad   0x7b9b1fb5dea1959e+        .quad   0xf01cc8f17471cc0c+        .quad   0x95242e37579082bb+        .quad   0x27776093d3e46b5f+        .quad   0x2d13d55a28bd85fb++        // 2^108 * 2 * G++        .quad   0xfac5d2065b35b8da+        .quad   0xa8da8a9a85624bb7+        .quad   0xccd2ca913d21cd0f+        .quad   0x6b8341ee8bf90d58+        .quad   0xbf019cce7aee7a52+        .quad   0xa8ded2b6e454ead3+        .quad   0x3c619f0b87a8bb19+        .quad   0x3619b5d7560916d8+        .quad   0x3579f26b0282c4b2+        .quad   0x64d592f24fafefae+        .quad   0xb7cded7b28c8c7c0+        .quad   0x6a927b6b7173a8d7++        // 2^108 * 3 * G++        .quad   0x1f6db24f986e4656+        .quad   0x1021c02ed1e9105b+        .quad   0xf8ff3fff2cc0a375+        .quad   0x1d2a6bf8c6c82592+        .quad   0x8d7040863ece88eb+        .quad   0xf0e307a980eec08c+        .quad   0xac2250610d788fda+        .quad   0x056d92a43a0d478d+        .quad   0x1b05a196fc3da5a1+        .quad   0x77d7a8c243b59ed0+        .quad   0x06da3d6297d17918+        .quad   0x66fbb494f12353f7++        // 2^108 * 4 * G++        .quad   0x751a50b9d85c0fb8+        .quad   0xd1afdc258bcf097b+        .quad   0x2f16a6a38309a969+        .quad   0x14ddff9ee5b00659+        .quad   0xd6d70996f12309d6+        .quad   0xdbfb2385e9c3d539+        .quad   0x46d602b0f7552411+        .quad   0x270a0b0557843e0c+        .quad   0x61ff0640a7862bcc+        .quad   0x81cac09a5f11abfe+        .quad   0x9047830455d12abb+        .quad   0x19a4bde1945ae873++        // 2^108 * 5 * G++        .quad   0x9b9f26f520a6200a+        .quad   0x64804443cf13eaf8+        .quad   0x8a63673f8631edd3+        .quad   0x72bbbce11ed39dc1+        .quad   0x40c709dec076c49f+        .quad   0x657bfaf27f3e53f6+        .quad   0x40662331eca042c4+        .quad   0x14b375487eb4df04+        .quad   0xae853c94ab66dc47+        .quad   0xeb62343edf762d6e+        .quad   0xf08e0e186fb2f7d1+        .quad   0x4f0b1c02700ab37a++        // 2^108 * 6 * G++        .quad   0xe1706787d81951fa+        .quad   0xa10a2c8eb290c77b+        .quad   0xe7382fa03ed66773+        .quad   0x0a4d84710bcc4b54+        .quad   0x79fd21ccc1b2e23f+        .quad   0x4ae7c281453df52a+        .quad   0xc8172ec9d151486b+        .quad   0x68abe9443e0a7534+        .quad   0xda12c6c407831dcb+        .quad   0x0da230d74d5c510d+        .quad   0x4ab1531e6bd404e1+        .quad   0x4106b166bcf440ef++        // 2^108 * 7 * G++        .quad   0x02e57a421cd23668+        .quad   0x4ad9fb5d0eaef6fd+        .quad   0x954e6727b1244480+        .quad   0x7f792f9d2699f331+        .quad   0xa485ccd539e4ecf2+        .quad   0x5aa3f3ad0555bab5+        .quad   0x145e3439937df82d+        .quad   0x1238b51e1214283f+        .quad   0x0b886b925fd4d924+        .quad   0x60906f7a3626a80d+        .quad   0xecd367b4b98abd12+        .quad   0x2876beb1def344cf++        // 2^108 * 8 * G++        .quad   0xdc84e93563144691+        .quad   0x632fe8a0d61f23f4+        .quad   0x4caa800612a9a8d5+        .quad   0x48f9dbfa0e9918d3+        .quad   0xd594b3333a8a85f8+        .quad   0x4ea37689e78d7d58+        .quad   0x73bf9f455e8e351f+        .quad   0x5507d7d2bc41ebb4+        .quad   0x1ceb2903299572fc+        .quad   0x7c8ccaa29502d0ee+        .quad   0x91bfa43411cce67b+        .quad   0x5784481964a831e7++        // 2^112 * 1 * G++        .quad   0xda7c2b256768d593+        .quad   0x98c1c0574422ca13+        .quad   0xf1a80bd5ca0ace1d+        .quad   0x29cdd1adc088a690+        .quad   0xd6cfd1ef5fddc09c+        .quad   0xe82b3efdf7575dce+        .quad   0x25d56b5d201634c2+        .quad   0x3041c6bb04ed2b9b+        .quad   0x0ff2f2f9d956e148+        .quad   0xade797759f356b2e+        .quad   0x1a4698bb5f6c025c+        .quad   0x104bbd6814049a7b++        // 2^112 * 2 * G++        .quad   0x51f0fd3168f1ed67+        .quad   0x2c811dcdd86f3bc2+        .quad   0x44dc5c4304d2f2de+        .quad   0x5be8cc57092a7149+        .quad   0xa95d9a5fd67ff163+        .quad   0xe92be69d4cc75681+        .quad   0xb7f8024cde20f257+        .quad   0x204f2a20fb072df5+        .quad   0xc8143b3d30ebb079+        .quad   0x7589155abd652e30+        .quad   0x653c3c318f6d5c31+        .quad   0x2570fb17c279161f++        // 2^112 * 3 * G++        .quad   0x3efa367f2cb61575+        .quad   0xf5f96f761cd6026c+        .quad   0xe8c7142a65b52562+        .quad   0x3dcb65ea53030acd+        .quad   0x192ea9550bb8245a+        .quad   0xc8e6fba88f9050d1+        .quad   0x7986ea2d88a4c935+        .quad   0x241c5f91de018668+        .quad   0x28d8172940de6caa+        .quad   0x8fbf2cf022d9733a+        .quad   0x16d7fcdd235b01d1+        .quad   0x08420edd5fcdf0e5++        // 2^112 * 4 * G++        .quad   0xcdff20ab8362fa4a+        .quad   0x57e118d4e21a3e6e+        .quad   0xe3179617fc39e62b+        .quad   0x0d9a53efbc1769fd+        .quad   0x0358c34e04f410ce+        .quad   0xb6135b5a276e0685+        .quad   0x5d9670c7ebb91521+        .quad   0x04d654f321db889c+        .quad   0x5e7dc116ddbdb5d5+        .quad   0x2954deb68da5dd2d+        .quad   0x1cb608173334a292+        .quad   0x4a7a4f2618991ad7++        // 2^112 * 5 * G++        .quad   0xf4a718025fb15f95+        .quad   0x3df65f346b5c1b8f+        .quad   0xcdfcf08500e01112+        .quad   0x11b50c4cddd31848+        .quad   0x24c3b291af372a4b+        .quad   0x93da8270718147f2+        .quad   0xdd84856486899ef2+        .quad   0x4a96314223e0ee33+        .quad   0xa6e8274408a4ffd6+        .quad   0x738e177e9c1576d9+        .quad   0x773348b63d02b3f2+        .quad   0x4f4bce4dce6bcc51++        // 2^112 * 6 * G++        .quad   0xa71fce5ae2242584+        .quad   0x26ea725692f58a9e+        .quad   0xd21a09d71cea3cf4+        .quad   0x73fcdd14b71c01e6+        .quad   0x30e2616ec49d0b6f+        .quad   0xe456718fcaec2317+        .quad   0x48eb409bf26b4fa6+        .quad   0x3042cee561595f37+        .quad   0x427e7079449bac41+        .quad   0x855ae36dbce2310a+        .quad   0x4cae76215f841a7c+        .quad   0x389e740c9a9ce1d6++        // 2^112 * 7 * G++        .quad   0x64fcb3ae34dcb9ce+        .quad   0x97500323e348d0ad+        .quad   0x45b3f07d62c6381b+        .quad   0x61545379465a6788+        .quad   0xc9bd78f6570eac28+        .quad   0xe55b0b3227919ce1+        .quad   0x65fc3eaba19b91ed+        .quad   0x25c425e5d6263690+        .quad   0x3f3e06a6f1d7de6e+        .quad   0x3ef976278e062308+        .quad   0x8c14f6264e8a6c77+        .quad   0x6539a08915484759++        // 2^112 * 8 * G++        .quad   0xe9d21f74c3d2f773+        .quad   0xc150544125c46845+        .quad   0x624e5ce8f9b99e33+        .quad   0x11c5e4aac5cd186c+        .quad   0xddc4dbd414bb4a19+        .quad   0x19b2bc3c98424f8e+        .quad   0x48a89fd736ca7169+        .quad   0x0f65320ef019bd90+        .quad   0xd486d1b1cafde0c6+        .quad   0x4f3fe6e3163b5181+        .quad   0x59a8af0dfaf2939a+        .quad   0x4cabc7bdec33072a++        // 2^116 * 1 * G++        .quad   0x16faa8fb532f7428+        .quad   0xdbd42ea046a4e272+        .quad   0x5337653b8b9ea480+        .quad   0x4065947223973f03+        .quad   0xf7c0a19c1a54a044+        .quad   0x4a1c5e2477bd9fbb+        .quad   0xa6e3ca115af22972+        .quad   0x1819bb953f2e9e0d+        .quad   0x498fbb795e042e84+        .quad   0x7d0dd89a7698b714+        .quad   0x8bfb0ba427fe6295+        .quad   0x36ba82e721200524++        // 2^116 * 2 * G++        .quad   0xd60ecbb74245ec41+        .quad   0xfd9be89e34348716+        .quad   0xc9240afee42284de+        .quad   0x4472f648d0531db4+        .quad   0xc8d69d0a57274ed5+        .quad   0x45ba803260804b17+        .quad   0xdf3cda102255dfac+        .quad   0x77d221232709b339+        .quad   0x498a6d7064ad94d8+        .quad   0xa5b5c8fd9af62263+        .quad   0x8ca8ed0545c141f4+        .quad   0x2c63bec3662d358c++        // 2^116 * 3 * G++        .quad   0x7fe60d8bea787955+        .quad   0xb9dc117eb5f401b7+        .quad   0x91c7c09a19355cce+        .quad   0x22692ef59442bedf+        .quad   0x9a518b3a8586f8bf+        .quad   0x9ee71af6cbb196f0+        .quad   0xaa0625e6a2385cf2+        .quad   0x1deb2176ddd7c8d1+        .quad   0x8563d19a2066cf6c+        .quad   0x401bfd8c4dcc7cd7+        .quad   0xd976a6becd0d8f62+        .quad   0x67cfd773a278b05e++        // 2^116 * 4 * G++        .quad   0x8dec31faef3ee475+        .quad   0x99dbff8a9e22fd92+        .quad   0x512d11594e26cab1+        .quad   0x0cde561eec4310b9+        .quad   0x2d5fa9855a4e586a+        .quad   0x65f8f7a449beab7e+        .quad   0xaa074dddf21d33d3+        .quad   0x185cba721bcb9dee+        .quad   0x93869da3f4e3cb41+        .quad   0xbf0392f540f7977e+        .quad   0x026204fcd0463b83+        .quad   0x3ec91a769eec6eed++        // 2^116 * 5 * G++        .quad   0x1e9df75bf78166ad+        .quad   0x4dfda838eb0cd7af+        .quad   0xba002ed8c1eaf988+        .quad   0x13fedb3e11f33cfc+        .quad   0x0fad2fb7b0a3402f+        .quad   0x46615ecbfb69f4a8+        .quad   0xf745bcc8c5f8eaa6+        .quad   0x7a5fa8794a94e896+        .quad   0x52958faa13cd67a1+        .quad   0x965ee0818bdbb517+        .quad   0x16e58daa2e8845b3+        .quad   0x357d397d5499da8f++        // 2^116 * 6 * G++        .quad   0x1ebfa05fb0bace6c+        .quad   0xc934620c1caf9a1e+        .quad   0xcc771cc41d82b61a+        .quad   0x2d94a16aa5f74fec+        .quad   0x481dacb4194bfbf8+        .quad   0x4d77e3f1bae58299+        .quad   0x1ef4612e7d1372a0+        .quad   0x3a8d867e70ff69e1+        .quad   0x6f58cd5d55aff958+        .quad   0xba3eaa5c75567721+        .quad   0x75c123999165227d+        .quad   0x69be1343c2f2b35e++        // 2^116 * 7 * G++        .quad   0x0e091d5ee197c92a+        .quad   0x4f51019f2945119f+        .quad   0x143679b9f034e99c+        .quad   0x7d88112e4d24c696+        .quad   0x82bbbdac684b8de3+        .quad   0xa2f4c7d03fca0718+        .quad   0x337f92fbe096aaa8+        .quad   0x200d4d8c63587376+        .quad   0x208aed4b4893b32b+        .quad   0x3efbf23ebe59b964+        .quad   0xd762deb0dba5e507+        .quad   0x69607bd681bd9d94++        // 2^116 * 8 * G++        .quad   0xf6be021068de1ce1+        .quad   0xe8d518e70edcbc1f+        .quad   0xe3effdd01b5505a5+        .quad   0x35f63353d3ec3fd0+        .quad   0x3b7f3bd49323a902+        .quad   0x7c21b5566b2c6e53+        .quad   0xe5ba8ff53a7852a7+        .quad   0x28bc77a5838ece00+        .quad   0x63ba78a8e25d8036+        .quad   0x63651e0094333490+        .quad   0x48d82f20288ce532+        .quad   0x3a31abfa36b57524++        // 2^120 * 1 * G++        .quad   0x239e9624089c0a2e+        .quad   0xc748c4c03afe4738+        .quad   0x17dbed2a764fa12a+        .quad   0x639b93f0321c8582+        .quad   0xc08f788f3f78d289+        .quad   0xfe30a72ca1404d9f+        .quad   0xf2778bfccf65cc9d+        .quad   0x7ee498165acb2021+        .quad   0x7bd508e39111a1c3+        .quad   0x2b2b90d480907489+        .quad   0xe7d2aec2ae72fd19+        .quad   0x0edf493c85b602a6++        // 2^120 * 2 * G++        .quad   0xaecc8158599b5a68+        .quad   0xea574f0febade20e+        .quad   0x4fe41d7422b67f07+        .quad   0x403b92e3019d4fb4+        .quad   0x6767c4d284764113+        .quad   0xa090403ff7f5f835+        .quad   0x1c8fcffacae6bede+        .quad   0x04c00c54d1dfa369+        .quad   0x4dc22f818b465cf8+        .quad   0x71a0f35a1480eff8+        .quad   0xaee8bfad04c7d657+        .quad   0x355bb12ab26176f4++        // 2^120 * 3 * G++        .quad   0xa71e64cc7493bbf4+        .quad   0xe5bd84d9eca3b0c3+        .quad   0x0a6bc50cfa05e785+        .quad   0x0f9b8132182ec312+        .quad   0xa301dac75a8c7318+        .quad   0xed90039db3ceaa11+        .quad   0x6f077cbf3bae3f2d+        .quad   0x7518eaf8e052ad8e+        .quad   0xa48859c41b7f6c32+        .quad   0x0f2d60bcf4383298+        .quad   0x1815a929c9b1d1d9+        .quad   0x47c3871bbb1755c4++        // 2^120 * 4 * G++        .quad   0x5144539771ec4f48+        .quad   0xf805b17dc98c5d6e+        .quad   0xf762c11a47c3c66b+        .quad   0x00b89b85764699dc+        .quad   0xfbe65d50c85066b0+        .quad   0x62ecc4b0b3a299b0+        .quad   0xe53754ea441ae8e0+        .quad   0x08fea02ce8d48d5f+        .quad   0x824ddd7668deead0+        .quad   0xc86445204b685d23+        .quad   0xb514cfcd5d89d665+        .quad   0x473829a74f75d537++        // 2^120 * 5 * G++        .quad   0x82d2da754679c418+        .quad   0xe63bd7d8b2618df0+        .quad   0x355eef24ac47eb0a+        .quad   0x2078684c4833c6b4+        .quad   0x23d9533aad3902c9+        .quad   0x64c2ddceef03588f+        .quad   0x15257390cfe12fb4+        .quad   0x6c668b4d44e4d390+        .quad   0x3b48cf217a78820c+        .quad   0xf76a0ab281273e97+        .quad   0xa96c65a78c8eed7b+        .quad   0x7411a6054f8a433f++        // 2^120 * 6 * G++        .quad   0x4d659d32b99dc86d+        .quad   0x044cdc75603af115+        .quad   0xb34c712cdcc2e488+        .quad   0x7c136574fb8134ff+        .quad   0x579ae53d18b175b4+        .quad   0x68713159f392a102+        .quad   0x8455ecba1eef35f5+        .quad   0x1ec9a872458c398f+        .quad   0xb8e6a4d400a2509b+        .quad   0x9b81d7020bc882b4+        .quad   0x57e7cc9bf1957561+        .quad   0x3add88a5c7cd6460++        // 2^120 * 7 * G++        .quad   0xab895770b635dcf2+        .quad   0x02dfef6cf66c1fbc+        .quad   0x85530268beb6d187+        .quad   0x249929fccc879e74+        .quad   0x85c298d459393046+        .quad   0x8f7e35985ff659ec+        .quad   0x1d2ca22af2f66e3a+        .quad   0x61ba1131a406a720+        .quad   0xa3d0a0f116959029+        .quad   0x023b6b6cba7ebd89+        .quad   0x7bf15a3e26783307+        .quad   0x5620310cbbd8ece7++        // 2^120 * 8 * G++        .quad   0x528993434934d643+        .quad   0xb9dbf806a51222f5+        .quad   0x8f6d878fc3f41c22+        .quad   0x37676a2a4d9d9730+        .quad   0x6646b5f477e285d6+        .quad   0x40e8ff676c8f6193+        .quad   0xa6ec7311abb594dd+        .quad   0x7ec846f3658cec4d+        .quad   0x9b5e8f3f1da22ec7+        .quad   0x130f1d776c01cd13+        .quad   0x214c8fcfa2989fb8+        .quad   0x6daaf723399b9dd5++        // 2^124 * 1 * G++        .quad   0x591e4a5610628564+        .quad   0x2a4bb87ca8b4df34+        .quad   0xde2a2572e7a38e43+        .quad   0x3cbdabd9fee5046e+        .quad   0x81aebbdd2cd13070+        .quad   0x962e4325f85a0e9e+        .quad   0xde9391aacadffecb+        .quad   0x53177fda52c230e6+        .quad   0xa7bc970650b9de79+        .quad   0x3d12a7fbc301b59b+        .quad   0x02652e68d36ae38c+        .quad   0x79d739835a6199dc++        // 2^124 * 2 * G++        .quad   0xd9354df64131c1bd+        .quad   0x758094a186ec5822+        .quad   0x4464ee12e459f3c2+        .quad   0x6c11fce4cb133282+        .quad   0x21c9d9920d591737+        .quad   0x9bea41d2e9b46cd6+        .quad   0xe20e84200d89bfca+        .quad   0x79d99f946eae5ff8+        .quad   0xf17b483568673205+        .quad   0x387deae83caad96c+        .quad   0x61b471fd56ffe386+        .quad   0x31741195b745a599++        // 2^124 * 3 * G++        .quad   0xe8d10190b77a360b+        .quad   0x99b983209995e702+        .quad   0xbd4fdff8fa0247aa+        .quad   0x2772e344e0d36a87+        .quad   0x17f8ba683b02a047+        .quad   0x50212096feefb6c8+        .quad   0x70139be21556cbe2+        .quad   0x203e44a11d98915b+        .quad   0xd6863eba37b9e39f+        .quad   0x105bc169723b5a23+        .quad   0x104f6459a65c0762+        .quad   0x567951295b4d38d4++        // 2^124 * 4 * G++        .quad   0x535fd60613037524+        .quad   0xe210adf6b0fbc26a+        .quad   0xac8d0a9b23e990ae+        .quad   0x47204d08d72fdbf9+        .quad   0x07242eb30d4b497f+        .quad   0x1ef96306b9bccc87+        .quad   0x37950934d8116f45+        .quad   0x05468d6201405b04+        .quad   0x00f565a9f93267de+        .quad   0xcecfd78dc0d58e8a+        .quad   0xa215e2dcf318e28e+        .quad   0x4599ee919b633352++        // 2^124 * 5 * G++        .quad   0xd3c220ca70e0e76b+        .quad   0xb12bea58ea9f3094+        .quad   0x294ddec8c3271282+        .quad   0x0c3539e1a1d1d028+        .quad   0xac746d6b861ae579+        .quad   0x31ab0650f6aea9dc+        .quad   0x241d661140256d4c+        .quad   0x2f485e853d21a5de+        .quad   0x329744839c0833f3+        .quad   0x6fe6257fd2abc484+        .quad   0x5327d1814b358817+        .quad   0x65712585893fe9bc++        // 2^124 * 6 * G++        .quad   0x9c102fb732a61161+        .quad   0xe48e10dd34d520a8+        .quad   0x365c63546f9a9176+        .quad   0x32f6fe4c046f6006+        .quad   0x81c29f1bd708ee3f+        .quad   0xddcb5a05ae6407d0+        .quad   0x97aec1d7d2a3eba7+        .quad   0x1590521a91d50831+        .quad   0x40a3a11ec7910acc+        .quad   0x9013dff8f16d27ae+        .quad   0x1a9720d8abb195d4+        .quad   0x1bb9fe452ea98463++        // 2^124 * 7 * G++        .quad   0xe9d1d950b3d54f9e+        .quad   0x2d5f9cbee00d33c1+        .quad   0x51c2c656a04fc6ac+        .quad   0x65c091ee3c1cbcc9+        .quad   0xcf5e6c95cc36747c+        .quad   0x294201536b0bc30d+        .quad   0x453ac67cee797af0+        .quad   0x5eae6ab32a8bb3c9+        .quad   0x7083661114f118ea+        .quad   0x2b37b87b94349cad+        .quad   0x7273f51cb4e99f40+        .quad   0x78a2a95823d75698++        // 2^124 * 8 * G++        .quad   0xa2b072e95c8c2ace+        .quad   0x69cffc96651e9c4b+        .quad   0x44328ef842e7b42b+        .quad   0x5dd996c122aadeb3+        .quad   0xb4f23c425ef83207+        .quad   0xabf894d3c9a934b5+        .quad   0xd0708c1339fd87f7+        .quad   0x1876789117166130+        .quad   0x925b5ef0670c507c+        .quad   0x819bc842b93c33bf+        .quad   0x10792e9a70dd003f+        .quad   0x59ad4b7a6e28dc74++        // 2^128 * 1 * G++        .quad   0x5f3a7562eb3dbe47+        .quad   0xf7ea38548ebda0b8+        .quad   0x00c3e53145747299+        .quad   0x1304e9e71627d551+        .quad   0x583b04bfacad8ea2+        .quad   0x29b743e8148be884+        .quad   0x2b1e583b0810c5db+        .quad   0x2b5449e58eb3bbaa+        .quad   0x789814d26adc9cfe+        .quad   0x3c1bab3f8b48dd0b+        .quad   0xda0fe1fff979c60a+        .quad   0x4468de2d7c2dd693++        // 2^128 * 2 * G++        .quad   0x51bb355e9419469e+        .quad   0x33e6dc4c23ddc754+        .quad   0x93a5b6d6447f9962+        .quad   0x6cce7c6ffb44bd63+        .quad   0x4b9ad8c6f86307ce+        .quad   0x21113531435d0c28+        .quad   0xd4a866c5657a772c+        .quad   0x5da6427e63247352+        .quad   0x1a94c688deac22ca+        .quad   0xb9066ef7bbae1ff8+        .quad   0x88ad8c388d59580f+        .quad   0x58f29abfe79f2ca8++        // 2^128 * 3 * G++        .quad   0xe90ecfab8de73e68+        .quad   0x54036f9f377e76a5+        .quad   0xf0495b0bbe015982+        .quad   0x577629c4a7f41e36+        .quad   0x4b5a64bf710ecdf6+        .quad   0xb14ce538462c293c+        .quad   0x3643d056d50b3ab9+        .quad   0x6af93724185b4870+        .quad   0x3220024509c6a888+        .quad   0xd2e036134b558973+        .quad   0x83e236233c33289f+        .quad   0x701f25bb0caec18f++        // 2^128 * 4 * G++        .quad   0xc3a8b0f8e4616ced+        .quad   0xf700660e9e25a87d+        .quad   0x61e3061ff4bca59c+        .quad   0x2e0c92bfbdc40be9+        .quad   0x9d18f6d97cbec113+        .quad   0x844a06e674bfdbe4+        .quad   0x20f5b522ac4e60d6+        .quad   0x720a5bc050955e51+        .quad   0x0c3f09439b805a35+        .quad   0xe84e8b376242abfc+        .quad   0x691417f35c229346+        .quad   0x0e9b9cbb144ef0ec++        // 2^128 * 5 * G++        .quad   0xfbbad48ffb5720ad+        .quad   0xee81916bdbf90d0e+        .quad   0xd4813152635543bf+        .quad   0x221104eb3f337bd8+        .quad   0x8dee9bd55db1beee+        .quad   0xc9c3ab370a723fb9+        .quad   0x44a8f1bf1c68d791+        .quad   0x366d44191cfd3cde+        .quad   0x9e3c1743f2bc8c14+        .quad   0x2eda26fcb5856c3b+        .quad   0xccb82f0e68a7fb97+        .quad   0x4167a4e6bc593244++        // 2^128 * 6 * G++        .quad   0x643b9d2876f62700+        .quad   0x5d1d9d400e7668eb+        .quad   0x1b4b430321fc0684+        .quad   0x7938bb7e2255246a+        .quad   0xc2be2665f8ce8fee+        .quad   0xe967ff14e880d62c+        .quad   0xf12e6e7e2f364eee+        .quad   0x34b33370cb7ed2f6+        .quad   0xcdc591ee8681d6cc+        .quad   0xce02109ced85a753+        .quad   0xed7485c158808883+        .quad   0x1176fc6e2dfe65e4++        // 2^128 * 7 * G++        .quad   0xb4af6cd05b9c619b+        .quad   0x2ddfc9f4b2a58480+        .quad   0x3d4fa502ebe94dc4+        .quad   0x08fc3a4c677d5f34+        .quad   0xdb90e28949770eb8+        .quad   0x98fbcc2aacf440a3+        .quad   0x21354ffeded7879b+        .quad   0x1f6a3e54f26906b6+        .quad   0x60a4c199d30734ea+        .quad   0x40c085b631165cd6+        .quad   0xe2333e23f7598295+        .quad   0x4f2fad0116b900d1++        // 2^128 * 8 * G++        .quad   0x44beb24194ae4e54+        .quad   0x5f541c511857ef6c+        .quad   0xa61e6b2d368d0498+        .quad   0x445484a4972ef7ab+        .quad   0x962cd91db73bb638+        .quad   0xe60577aafc129c08+        .quad   0x6f619b39f3b61689+        .quad   0x3451995f2944ee81+        .quad   0x9152fcd09fea7d7c+        .quad   0x4a816c94b0935cf6+        .quad   0x258e9aaa47285c40+        .quad   0x10b89ca6042893b7++        // 2^132 * 1 * G++        .quad   0x9b2a426e3b646025+        .quad   0x32127190385ce4cf+        .quad   0xa25cffc2dd6dea45+        .quad   0x06409010bea8de75+        .quad   0xd67cded679d34aa0+        .quad   0xcc0b9ec0cc4db39f+        .quad   0xa535a456e35d190f+        .quad   0x2e05d9eaf61f6fef+        .quad   0xc447901ad61beb59+        .quad   0x661f19bce5dc880a+        .quad   0x24685482b7ca6827+        .quad   0x293c778cefe07f26++        // 2^132 * 2 * G++        .quad   0x86809e7007069096+        .quad   0xaad75b15e4e50189+        .quad   0x07f35715a21a0147+        .quad   0x0487f3f112815d5e+        .quad   0x16c795d6a11ff200+        .quad   0xcb70d0e2b15815c9+        .quad   0x89f293209b5395b5+        .quad   0x50b8c2d031e47b4f+        .quad   0x48350c08068a4962+        .quad   0x6ffdd05351092c9a+        .quad   0x17af4f4aaf6fc8dd+        .quad   0x4b0553b53cdba58b++        // 2^132 * 3 * G++        .quad   0x9c65fcbe1b32ff79+        .quad   0xeb75ea9f03b50f9b+        .quad   0xfced2a6c6c07e606+        .quad   0x35106cd551717908+        .quad   0xbf05211b27c152d4+        .quad   0x5ec26849bd1af639+        .quad   0x5e0b2caa8e6fab98+        .quad   0x054c8bdd50bd0840+        .quad   0x38a0b12f1dcf073d+        .quad   0x4b60a8a3b7f6a276+        .quad   0xfed5ac25d3404f9a+        .quad   0x72e82d5e5505c229++        // 2^132 * 4 * G++        .quad   0x6b0b697ff0d844c8+        .quad   0xbb12f85cd979cb49+        .quad   0xd2a541c6c1da0f1f+        .quad   0x7b7c242958ce7211+        .quad   0x00d9cdfd69771d02+        .quad   0x410276cd6cfbf17e+        .quad   0x4c45306c1cb12ec7+        .quad   0x2857bf1627500861+        .quad   0x9f21903f0101689e+        .quad   0xd779dfd3bf861005+        .quad   0xa122ee5f3deb0f1b+        .quad   0x510df84b485a00d4++        // 2^132 * 5 * G++        .quad   0xa54133bb9277a1fa+        .quad   0x74ec3b6263991237+        .quad   0x1a3c54dc35d2f15a+        .quad   0x2d347144e482ba3a+        .quad   0x24b3c887c70ac15e+        .quad   0xb0f3a557fb81b732+        .quad   0x9b2cde2fe578cc1b+        .quad   0x4cf7ed0703b54f8e+        .quad   0x6bd47c6598fbee0f+        .quad   0x9e4733e2ab55be2d+        .quad   0x1093f624127610c5+        .quad   0x4e05e26ad0a1eaa4++        // 2^132 * 6 * G++        .quad   0xda9b6b624b531f20+        .quad   0x429a760e77509abb+        .quad   0xdbe9f522e823cb80+        .quad   0x618f1856880c8f82+        .quad   0x1833c773e18fe6c0+        .quad   0xe3c4711ad3c87265+        .quad   0x3bfd3c4f0116b283+        .quad   0x1955875eb4cd4db8+        .quad   0x6da6de8f0e399799+        .quad   0x7ad61aa440fda178+        .quad   0xb32cd8105e3563dd+        .quad   0x15f6beae2ae340ae++        // 2^132 * 7 * G++        .quad   0x862bcb0c31ec3a62+        .quad   0x810e2b451138f3c2+        .quad   0x788ec4b839dac2a4+        .quad   0x28f76867ae2a9281+        .quad   0xba9a0f7b9245e215+        .quad   0xf368612dd98c0dbb+        .quad   0x2e84e4cbf220b020+        .quad   0x6ba92fe962d90eda+        .quad   0x3e4df9655884e2aa+        .quad   0xbd62fbdbdbd465a5+        .quad   0xd7596caa0de9e524+        .quad   0x6e8042ccb2b1b3d7++        // 2^132 * 8 * G++        .quad   0xf10d3c29ce28ca6e+        .quad   0xbad34540fcb6093d+        .quad   0xe7426ed7a2ea2d3f+        .quad   0x08af9d4e4ff298b9+        .quad   0x1530653616521f7e+        .quad   0x660d06b896203dba+        .quad   0x2d3989bc545f0879+        .quad   0x4b5303af78ebd7b0+        .quad   0x72f8a6c3bebcbde8+        .quad   0x4f0fca4adc3a8e89+        .quad   0x6fa9d4e8c7bfdf7a+        .quad   0x0dcf2d679b624eb7++        // 2^136 * 1 * G++        .quad   0x3d5947499718289c+        .quad   0x12ebf8c524533f26+        .quad   0x0262bfcb14c3ef15+        .quad   0x20b878d577b7518e+        .quad   0x753941be5a45f06e+        .quad   0xd07caeed6d9c5f65+        .quad   0x11776b9c72ff51b6+        .quad   0x17d2d1d9ef0d4da9+        .quad   0x27f2af18073f3e6a+        .quad   0xfd3fe519d7521069+        .quad   0x22e3b72c3ca60022+        .quad   0x72214f63cc65c6a7++        // 2^136 * 2 * G++        .quad   0xb4e37f405307a693+        .quad   0xaba714d72f336795+        .quad   0xd6fbd0a773761099+        .quad   0x5fdf48c58171cbc9+        .quad   0x1d9db7b9f43b29c9+        .quad   0xd605824a4f518f75+        .quad   0xf2c072bd312f9dc4+        .quad   0x1f24ac855a1545b0+        .quad   0x24d608328e9505aa+        .quad   0x4748c1d10c1420ee+        .quad   0xc7ffe45c06fb25a2+        .quad   0x00ba739e2ae395e6++        // 2^136 * 3 * G++        .quad   0x592e98de5c8790d6+        .quad   0xe5bfb7d345c2a2df+        .quad   0x115a3b60f9b49922+        .quad   0x03283a3e67ad78f3+        .quad   0xae4426f5ea88bb26+        .quad   0x360679d984973bfb+        .quad   0x5c9f030c26694e50+        .quad   0x72297de7d518d226+        .quad   0x48241dc7be0cb939+        .quad   0x32f19b4d8b633080+        .quad   0xd3dfc90d02289308+        .quad   0x05e1296846271945++        // 2^136 * 4 * G++        .quad   0xba82eeb32d9c495a+        .quad   0xceefc8fcf12bb97c+        .quad   0xb02dabae93b5d1e0+        .quad   0x39c00c9c13698d9b+        .quad   0xadbfbbc8242c4550+        .quad   0xbcc80cecd03081d9+        .quad   0x843566a6f5c8df92+        .quad   0x78cf25d38258ce4c+        .quad   0x15ae6b8e31489d68+        .quad   0xaa851cab9c2bf087+        .quad   0xc9a75a97f04efa05+        .quad   0x006b52076b3ff832++        // 2^136 * 5 * G++        .quad   0x29e0cfe19d95781c+        .quad   0xb681df18966310e2+        .quad   0x57df39d370516b39+        .quad   0x4d57e3443bc76122+        .quad   0xf5cb7e16b9ce082d+        .quad   0x3407f14c417abc29+        .quad   0xd4b36bce2bf4a7ab+        .quad   0x7de2e9561a9f75ce+        .quad   0xde70d4f4b6a55ecb+        .quad   0x4801527f5d85db99+        .quad   0xdbc9c440d3ee9a81+        .quad   0x6b2a90af1a6029ed++        // 2^136 * 6 * G++        .quad   0x6923f4fc9ae61e97+        .quad   0x5735281de03f5fd1+        .quad   0xa764ae43e6edd12d+        .quad   0x5fd8f4e9d12d3e4a+        .quad   0x77ebf3245bb2d80a+        .quad   0xd8301b472fb9079b+        .quad   0xc647e6f24cee7333+        .quad   0x465812c8276c2109+        .quad   0x4d43beb22a1062d9+        .quad   0x7065fb753831dc16+        .quad   0x180d4a7bde2968d7+        .quad   0x05b32c2b1cb16790++        // 2^136 * 7 * G++        .quad   0xc8c05eccd24da8fd+        .quad   0xa1cf1aac05dfef83+        .quad   0xdbbeeff27df9cd61+        .quad   0x3b5556a37b471e99+        .quad   0xf7fca42c7ad58195+        .quad   0x3214286e4333f3cc+        .quad   0xb6c29d0d340b979d+        .quad   0x31771a48567307e1+        .quad   0x32b0c524e14dd482+        .quad   0xedb351541a2ba4b6+        .quad   0xa3d16048282b5af3+        .quad   0x4fc079d27a7336eb++        // 2^136 * 8 * G++        .quad   0x51c938b089bf2f7f+        .quad   0x2497bd6502dfe9a7+        .quad   0xffffc09c7880e453+        .quad   0x124567cecaf98e92+        .quad   0xdc348b440c86c50d+        .quad   0x1337cbc9cc94e651+        .quad   0x6422f74d643e3cb9+        .quad   0x241170c2bae3cd08+        .quad   0x3ff9ab860ac473b4+        .quad   0xf0911dee0113e435+        .quad   0x4ae75060ebc6c4af+        .quad   0x3f8612966c87000d++        // 2^140 * 1 * G++        .quad   0x0c9c5303f7957be4+        .quad   0xa3c31a20e085c145+        .quad   0xb0721d71d0850050+        .quad   0x0aba390eab0bf2da+        .quad   0x529fdffe638c7bf3+        .quad   0xdf2b9e60388b4995+        .quad   0xe027b34f1bad0249+        .quad   0x7bc92fc9b9fa74ed+        .quad   0x9f97ef2e801ad9f9+        .quad   0x83697d5479afda3a+        .quad   0xe906b3ffbd596b50+        .quad   0x02672b37dd3fb8e0++        // 2^140 * 2 * G++        .quad   0x48b2ca8b260885e4+        .quad   0xa4286bec82b34c1c+        .quad   0x937e1a2617f58f74+        .quad   0x741d1fcbab2ca2a5+        .quad   0xee9ba729398ca7f5+        .quad   0xeb9ca6257a4849db+        .quad   0x29eb29ce7ec544e1+        .quad   0x232ca21ef736e2c8+        .quad   0xbf61423d253fcb17+        .quad   0x08803ceafa39eb14+        .quad   0xf18602df9851c7af+        .quad   0x0400f3a049e3414b++        // 2^140 * 3 * G++        .quad   0xabce0476ba61c55b+        .quad   0x36a3d6d7c4d39716+        .quad   0x6eb259d5e8d82d09+        .quad   0x0c9176e984d756fb+        .quad   0x2efba412a06e7b06+        .quad   0x146785452c8d2560+        .quad   0xdf9713ebd67a91c7+        .quad   0x32830ac7157eadf3+        .quad   0x0e782a7ab73769e8+        .quad   0x04a05d7875b18e2c+        .quad   0x29525226ebcceae1+        .quad   0x0d794f8383eba820++        // 2^140 * 4 * G++        .quad   0xff35f5cb9e1516f4+        .quad   0xee805bcf648aae45+        .quad   0xf0d73c2bb93a9ef3+        .quad   0x097b0bf22092a6c2+        .quad   0x7be44ce7a7a2e1ac+        .quad   0x411fd93efad1b8b7+        .quad   0x1734a1d70d5f7c9b+        .quad   0x0d6592233127db16+        .quad   0xc48bab1521a9d733+        .quad   0xa6c2eaead61abb25+        .quad   0x625c6c1cc6cb4305+        .quad   0x7fc90fea93eb3a67++        // 2^140 * 5 * G++        .quad   0x0408f1fe1f5c5926+        .quad   0x1a8f2f5e3b258bf4+        .quad   0x40a951a2fdc71669+        .quad   0x6598ee93c98b577e+        .quad   0xc527deb59c7cb23d+        .quad   0x955391695328404e+        .quad   0xd64392817ccf2c7a+        .quad   0x6ce97dabf7d8fa11+        .quad   0x25b5a8e50ef7c48f+        .quad   0xeb6034116f2ce532+        .quad   0xc5e75173e53de537+        .quad   0x73119fa08c12bb03++        // 2^140 * 6 * G++        .quad   0xed30129453f1a4cb+        .quad   0xbce621c9c8f53787+        .quad   0xfacb2b1338bee7b9+        .quad   0x3025798a9ea8428c+        .quad   0x7845b94d21f4774d+        .quad   0xbf62f16c7897b727+        .quad   0x671857c03c56522b+        .quad   0x3cd6a85295621212+        .quad   0x3fecde923aeca999+        .quad   0xbdaa5b0062e8c12f+        .quad   0x67b99dfc96988ade+        .quad   0x3f52c02852661036++        // 2^140 * 7 * G++        .quad   0xffeaa48e2a1351c6+        .quad   0x28624754fa7f53d7+        .quad   0x0b5ba9e57582ddf1+        .quad   0x60c0104ba696ac59+        .quad   0x9258bf99eec416c6+        .quad   0xac8a5017a9d2f671+        .quad   0x629549ab16dea4ab+        .quad   0x05d0e85c99091569+        .quad   0x051de020de9cbe97+        .quad   0xfa07fc56b50bcf74+        .quad   0x378cec9f0f11df65+        .quad   0x36853c69ab96de4d++        // 2^140 * 8 * G++        .quad   0x36d9b8de78f39b2d+        .quad   0x7f42ed71a847b9ec+        .quad   0x241cd1d679bd3fde+        .quad   0x6a704fec92fbce6b+        .quad   0x4433c0b0fac5e7be+        .quad   0x724bae854c08dcbe+        .quad   0xf1f24cc446978f9b+        .quad   0x4a0aff6d62825fc8+        .quad   0xe917fb9e61095301+        .quad   0xc102df9402a092f8+        .quad   0xbf09e2f5fa66190b+        .quad   0x681109bee0dcfe37++        // 2^144 * 1 * G++        .quad   0x559a0cc9782a0dde+        .quad   0x551dcdb2ea718385+        .quad   0x7f62865b31ef238c+        .quad   0x504aa7767973613d+        .quad   0x9c18fcfa36048d13+        .quad   0x29159db373899ddd+        .quad   0xdc9f350b9f92d0aa+        .quad   0x26f57eee878a19d4+        .quad   0x0cab2cd55687efb1+        .quad   0x5180d162247af17b+        .quad   0x85c15a344f5a2467+        .quad   0x4041943d9dba3069++        // 2^144 * 2 * G++        .quad   0xc3c0eeba43ebcc96+        .quad   0x8d749c9c26ea9caf+        .quad   0xd9fa95ee1c77ccc6+        .quad   0x1420a1d97684340f+        .quad   0x4b217743a26caadd+        .quad   0x47a6b424648ab7ce+        .quad   0xcb1d4f7a03fbc9e3+        .quad   0x12d931429800d019+        .quad   0x00c67799d337594f+        .quad   0x5e3c5140b23aa47b+        .quad   0x44182854e35ff395+        .quad   0x1b4f92314359a012++        // 2^144 * 3 * G++        .quad   0x3e5c109d89150951+        .quad   0x39cefa912de9696a+        .quad   0x20eae43f975f3020+        .quad   0x239b572a7f132dae+        .quad   0x33cf3030a49866b1+        .quad   0x251f73d2215f4859+        .quad   0xab82aa4051def4f6+        .quad   0x5ff191d56f9a23f6+        .quad   0x819ed433ac2d9068+        .quad   0x2883ab795fc98523+        .quad   0xef4572805593eb3d+        .quad   0x020c526a758f36cb++        // 2^144 * 4 * G++        .quad   0x779834f89ed8dbbc+        .quad   0xc8f2aaf9dc7ca46c+        .quad   0xa9524cdca3e1b074+        .quad   0x02aacc4615313877+        .quad   0xe931ef59f042cc89+        .quad   0x2c589c9d8e124bb6+        .quad   0xadc8e18aaec75997+        .quad   0x452cfe0a5602c50c+        .quad   0x86a0f7a0647877df+        .quad   0xbbc464270e607c9f+        .quad   0xab17ea25f1fb11c9+        .quad   0x4cfb7d7b304b877b++        // 2^144 * 5 * G++        .quad   0x72b43d6cb89b75fe+        .quad   0x54c694d99c6adc80+        .quad   0xb8c3aa373ee34c9f+        .quad   0x14b4622b39075364+        .quad   0xe28699c29789ef12+        .quad   0x2b6ecd71df57190d+        .quad   0xc343c857ecc970d0+        .quad   0x5b1d4cbc434d3ac5+        .quad   0xb6fb2615cc0a9f26+        .quad   0x3a4f0e2bb88dcce5+        .quad   0x1301498b3369a705+        .quad   0x2f98f71258592dd1++        // 2^144 * 6 * G++        .quad   0x0c94a74cb50f9e56+        .quad   0x5b1ff4a98e8e1320+        .quad   0x9a2acc2182300f67+        .quad   0x3a6ae249d806aaf9+        .quad   0x2e12ae444f54a701+        .quad   0xfcfe3ef0a9cbd7de+        .quad   0xcebf890d75835de0+        .quad   0x1d8062e9e7614554+        .quad   0x657ada85a9907c5a+        .quad   0x1a0ea8b591b90f62+        .quad   0x8d0e1dfbdf34b4e9+        .quad   0x298b8ce8aef25ff3++        // 2^144 * 7 * G++        .quad   0x2a927953eff70cb2+        .quad   0x4b89c92a79157076+        .quad   0x9418457a30a7cf6a+        .quad   0x34b8a8404d5ce485+        .quad   0x837a72ea0a2165de+        .quad   0x3fab07b40bcf79f6+        .quad   0x521636c77738ae70+        .quad   0x6ba6271803a7d7dc+        .quad   0xc26eecb583693335+        .quad   0xd5a813df63b5fefd+        .quad   0xa293aa9aa4b22573+        .quad   0x71d62bdd465e1c6a++        // 2^144 * 8 * G++        .quad   0x6533cc28d378df80+        .quad   0xf6db43790a0fa4b4+        .quad   0xe3645ff9f701da5a+        .quad   0x74d5f317f3172ba4+        .quad   0xcd2db5dab1f75ef5+        .quad   0xd77f95cf16b065f5+        .quad   0x14571fea3f49f085+        .quad   0x1c333621262b2b3d+        .quad   0xa86fe55467d9ca81+        .quad   0x398b7c752b298c37+        .quad   0xda6d0892e3ac623b+        .quad   0x4aebcc4547e9d98c++        // 2^148 * 1 * G++        .quad   0x53175a7205d21a77+        .quad   0xb0c04422d3b934d4+        .quad   0xadd9f24bdd5deadc+        .quad   0x074f46e69f10ff8c+        .quad   0x0de9b204a059a445+        .quad   0xe15cb4aa4b17ad0f+        .quad   0xe1bbec521f79c557+        .quad   0x2633f1b9d071081b+        .quad   0xc1fb4177018b9910+        .quad   0xa6ea20dc6c0fe140+        .quad   0xd661f3e74354c6ff+        .quad   0x5ecb72e6f1a3407a++        // 2^148 * 2 * G++        .quad   0xa515a31b2259fb4e+        .quad   0x0960f3972bcac52f+        .quad   0xedb52fec8d3454cb+        .quad   0x382e2720c476c019+        .quad   0xfeeae106e8e86997+        .quad   0x9863337f98d09383+        .quad   0x9470480eaa06ebef+        .quad   0x038b6898d4c5c2d0+        .quad   0xf391c51d8ace50a6+        .quad   0x3142d0b9ae2d2948+        .quad   0xdb4d5a1a7f24ca80+        .quad   0x21aeba8b59250ea8++        // 2^148 * 3 * G++        .quad   0x24f13b34cf405530+        .quad   0x3c44ea4a43088af7+        .quad   0x5dd5c5170006a482+        .quad   0x118eb8f8890b086d+        .quad   0x53853600f0087f23+        .quad   0x4c461879da7d5784+        .quad   0x6af303deb41f6860+        .quad   0x0a3c16c5c27c18ed+        .quad   0x17e49c17cc947f3d+        .quad   0xccc6eda6aac1d27b+        .quad   0xdf6092ceb0f08e56+        .quad   0x4909b3e22c67c36b++        // 2^148 * 4 * G++        .quad   0x9c9c85ea63fe2e89+        .quad   0xbe1baf910e9412ec+        .quad   0x8f7baa8a86fbfe7b+        .quad   0x0fb17f9fef968b6c+        .quad   0x59a16676706ff64e+        .quad   0x10b953dd0d86a53d+        .quad   0x5848e1e6ce5c0b96+        .quad   0x2d8b78e712780c68+        .quad   0x79d5c62eafc3902b+        .quad   0x773a215289e80728+        .quad   0xc38ae640e10120b9+        .quad   0x09ae23717b2b1a6d++        // 2^148 * 5 * G++        .quad   0xbb6a192a4e4d083c+        .quad   0x34ace0630029e192+        .quad   0x98245a59aafabaeb+        .quad   0x6d9c8a9ada97faac+        .quad   0x10ab8fa1ad32b1d0+        .quad   0xe9aced1be2778b24+        .quad   0xa8856bc0373de90f+        .quad   0x66f35ddddda53996+        .quad   0xd27d9afb24997323+        .quad   0x1bb7e07ef6f01d2e+        .quad   0x2ba7472df52ecc7f+        .quad   0x03019b4f646f9dc8++        // 2^148 * 6 * G++        .quad   0x04a186b5565345cd+        .quad   0xeee76610bcc4116a+        .quad   0x689c73b478fb2a45+        .quad   0x387dcbff65697512+        .quad   0xaf09b214e6b3dc6b+        .quad   0x3f7573b5ad7d2f65+        .quad   0xd019d988100a23b0+        .quad   0x392b63a58b5c35f7+        .quad   0x4093addc9c07c205+        .quad   0xc565be15f532c37e+        .quad   0x63dbecfd1583402a+        .quad   0x61722b4aef2e032e++        // 2^148 * 7 * G++        .quad   0x0012aafeecbd47af+        .quad   0x55a266fb1cd46309+        .quad   0xf203eb680967c72c+        .quad   0x39633944ca3c1429+        .quad   0xd6b07a5581cb0e3c+        .quad   0x290ff006d9444969+        .quad   0x08680b6a16dcda1f+        .quad   0x5568d2b75a06de59+        .quad   0x8d0cb88c1b37cfe1+        .quad   0x05b6a5a3053818f3+        .quad   0xf2e9bc04b787d959+        .quad   0x6beba1249add7f64++        // 2^148 * 8 * G++        .quad   0x1d06005ca5b1b143+        .quad   0x6d4c6bb87fd1cda2+        .quad   0x6ef5967653fcffe7+        .quad   0x097c29e8c1ce1ea5+        .quad   0x5c3cecb943f5a53b+        .quad   0x9cc9a61d06c08df2+        .quad   0xcfba639a85895447+        .quad   0x5a845ae80df09fd5+        .quad   0x4ce97dbe5deb94ca+        .quad   0x38d0a4388c709c48+        .quad   0xc43eced4a169d097+        .quad   0x0a1249fff7e587c3++        // 2^152 * 1 * G++        .quad   0x12f0071b276d01c9+        .quad   0xe7b8bac586c48c70+        .quad   0x5308129b71d6fba9+        .quad   0x5d88fbf95a3db792+        .quad   0x0b408d9e7354b610+        .quad   0x806b32535ba85b6e+        .quad   0xdbe63a034a58a207+        .quad   0x173bd9ddc9a1df2c+        .quad   0x2b500f1efe5872df+        .quad   0x58d6582ed43918c1+        .quad   0xe6ed278ec9673ae0+        .quad   0x06e1cd13b19ea319++        // 2^152 * 2 * G++        .quad   0x40d0ad516f166f23+        .quad   0x118e32931fab6abe+        .quad   0x3fe35e14a04d088e+        .quad   0x3080603526e16266+        .quad   0x472baf629e5b0353+        .quad   0x3baa0b90278d0447+        .quad   0x0c785f469643bf27+        .quad   0x7f3a6a1a8d837b13+        .quad   0xf7e644395d3d800b+        .quad   0x95a8d555c901edf6+        .quad   0x68cd7830592c6339+        .quad   0x30d0fded2e51307e++        // 2^152 * 3 * G++        .quad   0xe0594d1af21233b3+        .quad   0x1bdbe78ef0cc4d9c+        .quad   0x6965187f8f499a77+        .quad   0x0a9214202c099868+        .quad   0x9cb4971e68b84750+        .quad   0xa09572296664bbcf+        .quad   0x5c8de72672fa412b+        .quad   0x4615084351c589d9+        .quad   0xbc9019c0aeb9a02e+        .quad   0x55c7110d16034cae+        .quad   0x0e6df501659932ec+        .quad   0x3bca0d2895ca5dfe++        // 2^152 * 4 * G++        .quad   0x40f031bc3c5d62a4+        .quad   0x19fc8b3ecff07a60+        .quad   0x98183da2130fb545+        .quad   0x5631deddae8f13cd+        .quad   0x9c688eb69ecc01bf+        .quad   0xf0bc83ada644896f+        .quad   0xca2d955f5f7a9fe2+        .quad   0x4ea8b4038df28241+        .quad   0x2aed460af1cad202+        .quad   0x46305305a48cee83+        .quad   0x9121774549f11a5f+        .quad   0x24ce0930542ca463++        // 2^152 * 5 * G++        .quad   0x1fe890f5fd06c106+        .quad   0xb5c468355d8810f2+        .quad   0x827808fe6e8caf3e+        .quad   0x41d4e3c28a06d74b+        .quad   0x3fcfa155fdf30b85+        .quad   0xd2f7168e36372ea4+        .quad   0xb2e064de6492f844+        .quad   0x549928a7324f4280+        .quad   0xf26e32a763ee1a2e+        .quad   0xae91e4b7d25ffdea+        .quad   0xbc3bd33bd17f4d69+        .quad   0x491b66dec0dcff6a++        // 2^152 * 6 * G++        .quad   0x98f5b13dc7ea32a7+        .quad   0xe3d5f8cc7e16db98+        .quad   0xac0abf52cbf8d947+        .quad   0x08f338d0c85ee4ac+        .quad   0x75f04a8ed0da64a1+        .quad   0xed222caf67e2284b+        .quad   0x8234a3791f7b7ba4+        .quad   0x4cf6b8b0b7018b67+        .quad   0xc383a821991a73bd+        .quad   0xab27bc01df320c7a+        .quad   0xc13d331b84777063+        .quad   0x530d4a82eb078a99++        // 2^152 * 7 * G++        .quad   0x004c3630e1f94825+        .quad   0x7e2d78268cab535a+        .quad   0xc7482323cc84ff8b+        .quad   0x65ea753f101770b9+        .quad   0x6d6973456c9abf9e+        .quad   0x257fb2fc4900a880+        .quad   0x2bacf412c8cfb850+        .quad   0x0db3e7e00cbfbd5b+        .quad   0x3d66fc3ee2096363+        .quad   0x81d62c7f61b5cb6b+        .quad   0x0fbe044213443b1a+        .quad   0x02a4ec1921e1a1db++        // 2^152 * 8 * G++        .quad   0x5ce6259a3b24b8a2+        .quad   0xb8577acc45afa0b8+        .quad   0xcccbe6e88ba07037+        .quad   0x3d143c51127809bf+        .quad   0xf5c86162f1cf795f+        .quad   0x118c861926ee57f2+        .quad   0x172124851c063578+        .quad   0x36d12b5dec067fcf+        .quad   0x126d279179154557+        .quad   0xd5e48f5cfc783a0a+        .quad   0x36bdb6e8df179bac+        .quad   0x2ef517885ba82859++        // 2^156 * 1 * G++        .quad   0x88bd438cd11e0d4a+        .quad   0x30cb610d43ccf308+        .quad   0xe09a0e3791937bcc+        .quad   0x4559135b25b1720c+        .quad   0x1ea436837c6da1e9+        .quad   0xf9c189af1fb9bdbe+        .quad   0x303001fcce5dd155+        .quad   0x28a7c99ebc57be52+        .quad   0xb8fd9399e8d19e9d+        .quad   0x908191cb962423ff+        .quad   0xb2b948d747c742a3+        .quad   0x37f33226d7fb44c4++        // 2^156 * 2 * G++        .quad   0x0dae8767b55f6e08+        .quad   0x4a43b3b35b203a02+        .quad   0xe3725a6e80af8c79+        .quad   0x0f7a7fd1705fa7a3+        .quad   0x33912553c821b11d+        .quad   0x66ed42c241e301df+        .quad   0x066fcc11104222fd+        .quad   0x307a3b41c192168f+        .quad   0x8eeb5d076eb55ce0+        .quad   0x2fc536bfaa0d925a+        .quad   0xbe81830fdcb6c6e8+        .quad   0x556c7045827baf52++        // 2^156 * 3 * G++        .quad   0x8e2b517302e9d8b7+        .quad   0xe3e52269248714e8+        .quad   0xbd4fbd774ca960b5+        .quad   0x6f4b4199c5ecada9+        .quad   0xb94b90022bf44406+        .quad   0xabd4237eff90b534+        .quad   0x7600a960faf86d3a+        .quad   0x2f45abdac2322ee3+        .quad   0x61af4912c8ef8a6a+        .quad   0xe58fa4fe43fb6e5e+        .quad   0xb5afcc5d6fd427cf+        .quad   0x6a5393281e1e11eb++        // 2^156 * 4 * G++        .quad   0xf3da5139a5d1ee89+        .quad   0x8145457cff936988+        .quad   0x3f622fed00e188c4+        .quad   0x0f513815db8b5a3d+        .quad   0x0fff04fe149443cf+        .quad   0x53cac6d9865cddd7+        .quad   0x31385b03531ed1b7+        .quad   0x5846a27cacd1039d+        .quad   0x4ff5cdac1eb08717+        .quad   0x67e8b29590f2e9bc+        .quad   0x44093b5e237afa99+        .quad   0x0d414bed8708b8b2++        // 2^156 * 5 * G++        .quad   0xcfb68265fd0e75f6+        .quad   0xe45b3e28bb90e707+        .quad   0x7242a8de9ff92c7a+        .quad   0x685b3201933202dd+        .quad   0x81886a92294ac9e8+        .quad   0x23162b45d55547be+        .quad   0x94cfbc4403715983+        .quad   0x50eb8fdb134bc401+        .quad   0xc0b73ec6d6b330cd+        .quad   0x84e44807132faff1+        .quad   0x732b7352c4a5dee1+        .quad   0x5d7c7cf1aa7cd2d2++        // 2^156 * 6 * G++        .quad   0xaf3b46bf7a4aafa2+        .quad   0xb78705ec4d40d411+        .quad   0x114f0c6aca7c15e3+        .quad   0x3f364faaa9489d4d+        .quad   0x33d1013e9b73a562+        .quad   0x925cef5748ec26e1+        .quad   0xa7fce614dd468058+        .quad   0x78b0fad41e9aa438+        .quad   0xbf56a431ed05b488+        .quad   0xa533e66c9c495c7e+        .quad   0xe8652baf87f3651a+        .quad   0x0241800059d66c33++        // 2^156 * 7 * G++        .quad   0xceb077fea37a5be4+        .quad   0xdb642f02e5a5eeb7+        .quad   0xc2e6d0c5471270b8+        .quad   0x4771b65538e4529c+        .quad   0x28350c7dcf38ea01+        .quad   0x7c6cdbc0b2917ab6+        .quad   0xace7cfbe857082f7+        .quad   0x4d2845aba2d9a1e0+        .quad   0xbb537fe0447070de+        .quad   0xcba744436dd557df+        .quad   0xd3b5a3473600dbcb+        .quad   0x4aeabbe6f9ffd7f8++        // 2^156 * 8 * G++        .quad   0x4630119e40d8f78c+        .quad   0xa01a9bc53c710e11+        .quad   0x486d2b258910dd79+        .quad   0x1e6c47b3db0324e5+        .quad   0x6a2134bcc4a9c8f2+        .quad   0xfbf8fd1c8ace2e37+        .quad   0x000ae3049911a0ba+        .quad   0x046e3a616bc89b9e+        .quad   0x14e65442f03906be+        .quad   0x4a019d54e362be2a+        .quad   0x68ccdfec8dc230c7+        .quad   0x7cfb7e3faf6b861c++        // 2^160 * 1 * G++        .quad   0x4637974e8c58aedc+        .quad   0xb9ef22fbabf041a4+        .quad   0xe185d956e980718a+        .quad   0x2f1b78fab143a8a6+        .quad   0x96eebffb305b2f51+        .quad   0xd3f938ad889596b8+        .quad   0xf0f52dc746d5dd25+        .quad   0x57968290bb3a0095+        .quad   0xf71ab8430a20e101+        .quad   0xf393658d24f0ec47+        .quad   0xcf7509a86ee2eed1+        .quad   0x7dc43e35dc2aa3e1++        // 2^160 * 2 * G++        .quad   0x85966665887dd9c3+        .quad   0xc90f9b314bb05355+        .quad   0xc6e08df8ef2079b1+        .quad   0x7ef72016758cc12f+        .quad   0x5a782a5c273e9718+        .quad   0x3576c6995e4efd94+        .quad   0x0f2ed8051f237d3e+        .quad   0x044fb81d82d50a99+        .quad   0xc1df18c5a907e3d9+        .quad   0x57b3371dce4c6359+        .quad   0xca704534b201bb49+        .quad   0x7f79823f9c30dd2e++        // 2^160 * 3 * G++        .quad   0x8334d239a3b513e8+        .quad   0xc13670d4b91fa8d8+        .quad   0x12b54136f590bd33+        .quad   0x0a4e0373d784d9b4+        .quad   0x6a9c1ff068f587ba+        .quad   0x0827894e0050c8de+        .quad   0x3cbf99557ded5be7+        .quad   0x64a9b0431c06d6f0+        .quad   0x2eb3d6a15b7d2919+        .quad   0xb0b4f6a0d53a8235+        .quad   0x7156ce4389a45d47+        .quad   0x071a7d0ace18346c++        // 2^160 * 4 * G++        .quad   0xd3072daac887ba0b+        .quad   0x01262905bfa562ee+        .quad   0xcf543002c0ef768b+        .quad   0x2c3bcc7146ea7e9c+        .quad   0xcc0c355220e14431+        .quad   0x0d65950709b15141+        .quad   0x9af5621b209d5f36+        .quad   0x7c69bcf7617755d3+        .quad   0x07f0d7eb04e8295f+        .quad   0x10db18252f50f37d+        .quad   0xe951a9a3171798d7+        .quad   0x6f5a9a7322aca51d++        // 2^160 * 5 * G++        .quad   0x8ba1000c2f41c6c5+        .quad   0xc49f79c10cfefb9b+        .quad   0x4efa47703cc51c9f+        .quad   0x494e21a2e147afca+        .quad   0xe729d4eba3d944be+        .quad   0x8d9e09408078af9e+        .quad   0x4525567a47869c03+        .quad   0x02ab9680ee8d3b24+        .quad   0xefa48a85dde50d9a+        .quad   0x219a224e0fb9a249+        .quad   0xfa091f1dd91ef6d9+        .quad   0x6b5d76cbea46bb34++        // 2^160 * 6 * G++        .quad   0x8857556cec0cd994+        .quad   0x6472dc6f5cd01dba+        .quad   0xaf0169148f42b477+        .quad   0x0ae333f685277354+        .quad   0xe0f941171e782522+        .quad   0xf1e6ae74036936d3+        .quad   0x408b3ea2d0fcc746+        .quad   0x16fb869c03dd313e+        .quad   0x288e199733b60962+        .quad   0x24fc72b4d8abe133+        .quad   0x4811f7ed0991d03e+        .quad   0x3f81e38b8f70d075++        // 2^160 * 7 * G++        .quad   0x7f910fcc7ed9affe+        .quad   0x545cb8a12465874b+        .quad   0xa8397ed24b0c4704+        .quad   0x50510fc104f50993+        .quad   0x0adb7f355f17c824+        .quad   0x74b923c3d74299a4+        .quad   0xd57c3e8bcbf8eaf7+        .quad   0x0ad3e2d34cdedc3d+        .quad   0x6f0c0fc5336e249d+        .quad   0x745ede19c331cfd9+        .quad   0xf2d6fd0009eefe1c+        .quad   0x127c158bf0fa1ebe++        // 2^160 * 8 * G++        .quad   0xf6197c422e9879a2+        .quad   0xa44addd452ca3647+        .quad   0x9b413fc14b4eaccb+        .quad   0x354ef87d07ef4f68+        .quad   0xdea28fc4ae51b974+        .quad   0x1d9973d3744dfe96+        .quad   0x6240680b873848a8+        .quad   0x4ed82479d167df95+        .quad   0xfee3b52260c5d975+        .quad   0x50352efceb41b0b8+        .quad   0x8808ac30a9f6653c+        .quad   0x302d92d20539236d++        // 2^164 * 1 * G++        .quad   0x4c59023fcb3efb7c+        .quad   0x6c2fcb99c63c2a94+        .quad   0xba4190e2c3c7e084+        .quad   0x0e545daea51874d9+        .quad   0x957b8b8b0df53c30+        .quad   0x2a1c770a8e60f098+        .quad   0xbbc7a670345796de+        .quad   0x22a48f9a90c99bc9+        .quad   0x6b7dc0dc8d3fac58+        .quad   0x5497cd6ce6e42bfd+        .quad   0x542f7d1bf400d305+        .quad   0x4159f47f048d9136++        // 2^164 * 2 * G++        .quad   0x20ad660839e31e32+        .quad   0xf81e1bd58405be50+        .quad   0xf8064056f4dabc69+        .quad   0x14d23dd4ce71b975+        .quad   0x748515a8bbd24839+        .quad   0x77128347afb02b55+        .quad   0x50ba2ac649a2a17f+        .quad   0x060525513ad730f1+        .quad   0xf2398e098aa27f82+        .quad   0x6d7982bb89a1b024+        .quad   0xfa694084214dd24c+        .quad   0x71ab966fa32301c3++        // 2^164 * 3 * G++        .quad   0x2dcbd8e34ded02fc+        .quad   0x1151f3ec596f22aa+        .quad   0xbca255434e0328da+        .quad   0x35768fbe92411b22+        .quad   0xb1088a0702809955+        .quad   0x43b273ea0b43c391+        .quad   0xca9b67aefe0686ed+        .quad   0x605eecbf8335f4ed+        .quad   0x83200a656c340431+        .quad   0x9fcd71678ee59c2f+        .quad   0x75d4613f71300f8a+        .quad   0x7a912faf60f542f9++        // 2^164 * 4 * G++        .quad   0xb204585e5edc1a43+        .quad   0x9f0e16ee5897c73c+        .quad   0x5b82c0ae4e70483c+        .quad   0x624a170e2bddf9be+        .quad   0x253f4f8dfa2d5597+        .quad   0x25e49c405477130c+        .quad   0x00c052e5996b1102+        .quad   0x33cb966e33bb6c4a+        .quad   0x597028047f116909+        .quad   0x828ac41c1e564467+        .quad   0x70417dbde6217387+        .quad   0x721627aefbac4384++        // 2^164 * 5 * G++        .quad   0x97d03bc38736add5+        .quad   0x2f1422afc532b130+        .quad   0x3aa68a057101bbc4+        .quad   0x4c946cf7e74f9fa7+        .quad   0xfd3097bc410b2f22+        .quad   0xf1a05da7b5cfa844+        .quad   0x61289a1def57ca74+        .quad   0x245ea199bb821902+        .quad   0xaedca66978d477f8+        .quad   0x1898ba3c29117fe1+        .quad   0xcf73f983720cbd58+        .quad   0x67da12e6b8b56351++        // 2^164 * 6 * G++        .quad   0x7067e187b4bd6e07+        .quad   0x6e8f0203c7d1fe74+        .quad   0x93c6aa2f38c85a30+        .quad   0x76297d1f3d75a78a+        .quad   0x2b7ef3d38ec8308c+        .quad   0x828fd7ec71eb94ab+        .quad   0x807c3b36c5062abd+        .quad   0x0cb64cb831a94141+        .quad   0x3030fc33534c6378+        .quad   0xb9635c5ce541e861+        .quad   0x15d9a9bed9b2c728+        .quad   0x49233ea3f3775dcb++        // 2^164 * 7 * G++        .quad   0x629398fa8dbffc3a+        .quad   0xe12fe52dd54db455+        .quad   0xf3be11dfdaf25295+        .quad   0x628b140dce5e7b51+        .quad   0x7b3985fe1c9f249b+        .quad   0x4fd6b2d5a1233293+        .quad   0xceb345941adf4d62+        .quad   0x6987ff6f542de50c+        .quad   0x47e241428f83753c+        .quad   0x6317bebc866af997+        .quad   0xdabb5b433d1a9829+        .quad   0x074d8d245287fb2d++        // 2^164 * 8 * G++        .quad   0x8337d9cd440bfc31+        .quad   0x729d2ca1af318fd7+        .quad   0xa040a4a4772c2070+        .quad   0x46002ef03a7349be+        .quad   0x481875c6c0e31488+        .quad   0x219429b2e22034b4+        .quad   0x7223c98a31283b65+        .quad   0x3420d60b342277f9+        .quad   0xfaa23adeaffe65f7+        .quad   0x78261ed45be0764c+        .quad   0x441c0a1e2f164403+        .quad   0x5aea8e567a87d395++        // 2^168 * 1 * G++        .quad   0x7813c1a2bca4283d+        .quad   0xed62f091a1863dd9+        .quad   0xaec7bcb8c268fa86+        .quad   0x10e5d3b76f1cae4c+        .quad   0x2dbc6fb6e4e0f177+        .quad   0x04e1bf29a4bd6a93+        .quad   0x5e1966d4787af6e8+        .quad   0x0edc5f5eb426d060+        .quad   0x5453bfd653da8e67+        .quad   0xe9dc1eec24a9f641+        .quad   0xbf87263b03578a23+        .quad   0x45b46c51361cba72++        // 2^168 * 2 * G++        .quad   0xa9402abf314f7fa1+        .quad   0xe257f1dc8e8cf450+        .quad   0x1dbbd54b23a8be84+        .quad   0x2177bfa36dcb713b+        .quad   0xce9d4ddd8a7fe3e4+        .quad   0xab13645676620e30+        .quad   0x4b594f7bb30e9958+        .quad   0x5c1c0aef321229df+        .quad   0x37081bbcfa79db8f+        .quad   0x6048811ec25f59b3+        .quad   0x087a76659c832487+        .quad   0x4ae619387d8ab5bb++        // 2^168 * 3 * G++        .quad   0x8ddbf6aa5344a32e+        .quad   0x7d88eab4b41b4078+        .quad   0x5eb0eb974a130d60+        .quad   0x1a00d91b17bf3e03+        .quad   0x61117e44985bfb83+        .quad   0xfce0462a71963136+        .quad   0x83ac3448d425904b+        .quad   0x75685abe5ba43d64+        .quad   0x6e960933eb61f2b2+        .quad   0x543d0fa8c9ff4952+        .quad   0xdf7275107af66569+        .quad   0x135529b623b0e6aa++        // 2^168 * 4 * G++        .quad   0x18f0dbd7add1d518+        .quad   0x979f7888cfc11f11+        .quad   0x8732e1f07114759b+        .quad   0x79b5b81a65ca3a01+        .quad   0xf5c716bce22e83fe+        .quad   0xb42beb19e80985c1+        .quad   0xec9da63714254aae+        .quad   0x5972ea051590a613+        .quad   0x0fd4ac20dc8f7811+        .quad   0x9a9ad294ac4d4fa8+        .quad   0xc01b2d64b3360434+        .quad   0x4f7e9c95905f3bdb++        // 2^168 * 5 * G++        .quad   0x62674bbc5781302e+        .quad   0xd8520f3989addc0f+        .quad   0x8c2999ae53fbd9c6+        .quad   0x31993ad92e638e4c+        .quad   0x71c8443d355299fe+        .quad   0x8bcd3b1cdbebead7+        .quad   0x8092499ef1a49466+        .quad   0x1942eec4a144adc8+        .quad   0x7dac5319ae234992+        .quad   0x2c1b3d910cea3e92+        .quad   0x553ce494253c1122+        .quad   0x2a0a65314ef9ca75++        // 2^168 * 6 * G++        .quad   0x2db7937ff7f927c2+        .quad   0xdb741f0617d0a635+        .quad   0x5982f3a21155af76+        .quad   0x4cf6e218647c2ded+        .quad   0xcf361acd3c1c793a+        .quad   0x2f9ebcac5a35bc3b+        .quad   0x60e860e9a8cda6ab+        .quad   0x055dc39b6dea1a13+        .quad   0xb119227cc28d5bb6+        .quad   0x07e24ebc774dffab+        .quad   0xa83c78cee4a32c89+        .quad   0x121a307710aa24b6++        // 2^168 * 7 * G++        .quad   0xe4db5d5e9f034a97+        .quad   0xe153fc093034bc2d+        .quad   0x460546919551d3b1+        .quad   0x333fc76c7a40e52d+        .quad   0xd659713ec77483c9+        .quad   0x88bfe077b82b96af+        .quad   0x289e28231097bcd3+        .quad   0x527bb94a6ced3a9b+        .quad   0x563d992a995b482e+        .quad   0x3405d07c6e383801+        .quad   0x485035de2f64d8e5+        .quad   0x6b89069b20a7a9f7++        // 2^168 * 8 * G++        .quad   0x812aa0416270220d+        .quad   0x995a89faf9245b4e+        .quad   0xffadc4ce5072ef05+        .quad   0x23bc2103aa73eb73+        .quad   0x4082fa8cb5c7db77+        .quad   0x068686f8c734c155+        .quad   0x29e6c8d9f6e7a57e+        .quad   0x0473d308a7639bcf+        .quad   0xcaee792603589e05+        .quad   0x2b4b421246dcc492+        .quad   0x02a1ef74e601a94f+        .quad   0x102f73bfde04341a++        // 2^172 * 1 * G++        .quad   0xb5a2d50c7ec20d3e+        .quad   0xc64bdd6ea0c97263+        .quad   0x56e89052c1ff734d+        .quad   0x4929c6f72b2ffaba+        .quad   0x358ecba293a36247+        .quad   0xaf8f9862b268fd65+        .quad   0x412f7e9968a01c89+        .quad   0x5786f312cd754524+        .quad   0x337788ffca14032c+        .quad   0xf3921028447f1ee3+        .quad   0x8b14071f231bccad+        .quad   0x4c817b4bf2344783++        // 2^172 * 2 * G++        .quad   0x0ff853852871b96e+        .quad   0xe13e9fab60c3f1bb+        .quad   0xeefd595325344402+        .quad   0x0a37c37075b7744b+        .quad   0x413ba057a40b4484+        .quad   0xba4c2e1a4f5f6a43+        .quad   0x614ba0a5aee1d61c+        .quad   0x78a1531a8b05dc53+        .quad   0x6cbdf1703ad0562b+        .quad   0x8ecf4830c92521a3+        .quad   0xdaebd303fd8424e7+        .quad   0x72ad82a42e5ec56f++        // 2^172 * 3 * G++        .quad   0x3f9e8e35bafb65f6+        .quad   0x39d69ec8f27293a1+        .quad   0x6cb8cd958cf6a3d0+        .quad   0x1734778173adae6d+        .quad   0xc368939167024bc3+        .quad   0x8e69d16d49502fda+        .quad   0xfcf2ec3ce45f4b29+        .quad   0x065f669ea3b4cbc4+        .quad   0x8a00aec75532db4d+        .quad   0xb869a4e443e31bb1+        .quad   0x4a0f8552d3a7f515+        .quad   0x19adeb7c303d7c08++        // 2^172 * 4 * G++        .quad   0xc720cb6153ead9a3+        .quad   0x55b2c97f512b636e+        .quad   0xb1e35b5fd40290b1+        .quad   0x2fd9ccf13b530ee2+        .quad   0x9d05ba7d43c31794+        .quad   0x2470c8ff93322526+        .quad   0x8323dec816197438+        .quad   0x2852709881569b53+        .quad   0x07bd475b47f796b8+        .quad   0xd2c7b013542c8f54+        .quad   0x2dbd23f43b24f87e+        .quad   0x6551afd77b0901d6++        // 2^172 * 5 * G++        .quad   0x4546baaf54aac27f+        .quad   0xf6f66fecb2a45a28+        .quad   0x582d1b5b562bcfe8+        .quad   0x44b123f3920f785f+        .quad   0x68a24ce3a1d5c9ac+        .quad   0xbb77a33d10ff6461+        .quad   0x0f86ce4425d3166e+        .quad   0x56507c0950b9623b+        .quad   0x1206f0b7d1713e63+        .quad   0x353fe3d915bafc74+        .quad   0x194ceb970ad9d94d+        .quad   0x62fadd7cf9d03ad3++        // 2^172 * 6 * G++        .quad   0xc6b5967b5598a074+        .quad   0x5efe91ce8e493e25+        .quad   0xd4b72c4549280888+        .quad   0x20ef1149a26740c2+        .quad   0x3cd7bc61e7ce4594+        .quad   0xcd6b35a9b7dd267e+        .quad   0xa080abc84366ef27+        .quad   0x6ec7c46f59c79711+        .quad   0x2f07ad636f09a8a2+        .quad   0x8697e6ce24205e7d+        .quad   0xc0aefc05ee35a139+        .quad   0x15e80958b5f9d897++        // 2^172 * 7 * G++        .quad   0x25a5ef7d0c3e235b+        .quad   0x6c39c17fbe134ee7+        .quad   0xc774e1342dc5c327+        .quad   0x021354b892021f39+        .quad   0x4dd1ed355bb061c4+        .quad   0x42dc0cef941c0700+        .quad   0x61305dc1fd86340e+        .quad   0x56b2cc930e55a443+        .quad   0x1df79da6a6bfc5a2+        .quad   0x02f3a2749fde4369+        .quad   0xb323d9f2cda390a7+        .quad   0x7be0847b8774d363++        // 2^172 * 8 * G++        .quad   0x8c99cc5a8b3f55c3+        .quad   0x0611d7253fded2a0+        .quad   0xed2995ff36b70a36+        .quad   0x1f699a54d78a2619+        .quad   0x1466f5af5307fa11+        .quad   0x817fcc7ded6c0af2+        .quad   0x0a6de44ec3a4a3fb+        .quad   0x74071475bc927d0b+        .quad   0xe77292f373e7ea8a+        .quad   0x296537d2cb045a31+        .quad   0x1bd0653ed3274fde+        .quad   0x2f9a2c4476bd2966++        // 2^176 * 1 * G++        .quad   0xeb18b9ab7f5745c6+        .quad   0x023a8aee5787c690+        .quad   0xb72712da2df7afa9+        .quad   0x36597d25ea5c013d+        .quad   0xa2b4dae0b5511c9a+        .quad   0x7ac860292bffff06+        .quad   0x981f375df5504234+        .quad   0x3f6bd725da4ea12d+        .quad   0x734d8d7b106058ac+        .quad   0xd940579e6fc6905f+        .quad   0x6466f8f99202932d+        .quad   0x7b7ecc19da60d6d0++        // 2^176 * 2 * G++        .quad   0x78c2373c695c690d+        .quad   0xdd252e660642906e+        .quad   0x951d44444ae12bd2+        .quad   0x4235ad7601743956+        .quad   0x6dae4a51a77cfa9b+        .quad   0x82263654e7a38650+        .quad   0x09bbffcd8f2d82db+        .quad   0x03bedc661bf5caba+        .quad   0x6258cb0d078975f5+        .quad   0x492942549189f298+        .quad   0xa0cab423e2e36ee4+        .quad   0x0e7ce2b0cdf066a1++        // 2^176 * 3 * G++        .quad   0xc494643ac48c85a3+        .quad   0xfd361df43c6139ad+        .quad   0x09db17dd3ae94d48+        .quad   0x666e0a5d8fb4674a+        .quad   0xfea6fedfd94b70f9+        .quad   0xf130c051c1fcba2d+        .quad   0x4882d47e7f2fab89+        .quad   0x615256138aeceeb5+        .quad   0x2abbf64e4870cb0d+        .quad   0xcd65bcf0aa458b6b+        .quad   0x9abe4eba75e8985d+        .quad   0x7f0bc810d514dee4++        // 2^176 * 4 * G++        .quad   0xb9006ba426f4136f+        .quad   0x8d67369e57e03035+        .quad   0xcbc8dfd94f463c28+        .quad   0x0d1f8dbcf8eedbf5+        .quad   0x83ac9dad737213a0+        .quad   0x9ff6f8ba2ef72e98+        .quad   0x311e2edd43ec6957+        .quad   0x1d3a907ddec5ab75+        .quad   0xba1693313ed081dc+        .quad   0x29329fad851b3480+        .quad   0x0128013c030321cb+        .quad   0x00011b44a31bfde3++        // 2^176 * 5 * G++        .quad   0x3fdfa06c3fc66c0c+        .quad   0x5d40e38e4dd60dd2+        .quad   0x7ae38b38268e4d71+        .quad   0x3ac48d916e8357e1+        .quad   0x16561f696a0aa75c+        .quad   0xc1bf725c5852bd6a+        .quad   0x11a8dd7f9a7966ad+        .quad   0x63d988a2d2851026+        .quad   0x00120753afbd232e+        .quad   0xe92bceb8fdd8f683+        .quad   0xf81669b384e72b91+        .quad   0x33fad52b2368a066++        // 2^176 * 6 * G++        .quad   0x540649c6c5e41e16+        .quad   0x0af86430333f7735+        .quad   0xb2acfcd2f305e746+        .quad   0x16c0f429a256dca7+        .quad   0x8d2cc8d0c422cfe8+        .quad   0x072b4f7b05a13acb+        .quad   0xa3feb6e6ecf6a56f+        .quad   0x3cc355ccb90a71e2+        .quad   0xe9b69443903e9131+        .quad   0xb8a494cb7a5637ce+        .quad   0xc87cd1a4baba9244+        .quad   0x631eaf426bae7568++        // 2^176 * 7 * G++        .quad   0xb3e90410da66fe9f+        .quad   0x85dd4b526c16e5a6+        .quad   0xbc3d97611ef9bf83+        .quad   0x5599648b1ea919b5+        .quad   0x47d975b9a3700de8+        .quad   0x7280c5fbe2f80552+        .quad   0x53658f2732e45de1+        .quad   0x431f2c7f665f80b5+        .quad   0xd6026344858f7b19+        .quad   0x14ab352fa1ea514a+        .quad   0x8900441a2090a9d7+        .quad   0x7b04715f91253b26++        // 2^176 * 8 * G++        .quad   0x83edbd28acf6ae43+        .quad   0x86357c8b7d5c7ab4+        .quad   0xc0404769b7eb2c44+        .quad   0x59b37bf5c2f6583f+        .quad   0xb376c280c4e6bac6+        .quad   0x970ed3dd6d1d9b0b+        .quad   0xb09a9558450bf944+        .quad   0x48d0acfa57cde223+        .quad   0xb60f26e47dabe671+        .quad   0xf1d1a197622f3a37+        .quad   0x4208ce7ee9960394+        .quad   0x16234191336d3bdb++        // 2^180 * 1 * G++        .quad   0xf19aeac733a63aef+        .quad   0x2c7fba5d4442454e+        .quad   0x5da87aa04795e441+        .quad   0x413051e1a4e0b0f5+        .quad   0x852dd1fd3d578bbe+        .quad   0x2b65ce72c3286108+        .quad   0x658c07f4eace2273+        .quad   0x0933f804ec38ab40+        .quad   0xa7ab69798d496476+        .quad   0x8121aadefcb5abc8+        .quad   0xa5dc12ef7b539472+        .quad   0x07fd47065e45351a++        // 2^180 * 2 * G++        .quad   0xc8583c3d258d2bcd+        .quad   0x17029a4daf60b73f+        .quad   0xfa0fc9d6416a3781+        .quad   0x1c1e5fba38b3fb23+        .quad   0x304211559ae8e7c3+        .quad   0xf281b229944882a5+        .quad   0x8a13ac2e378250e4+        .quad   0x014afa0954ba48f4+        .quad   0xcb3197001bb3666c+        .quad   0x330060524bffecb9+        .quad   0x293711991a88233c+        .quad   0x291884363d4ed364++        // 2^180 * 3 * G++        .quad   0x033c6805dc4babfa+        .quad   0x2c15bf5e5596ecc1+        .quad   0x1bc70624b59b1d3b+        .quad   0x3ede9850a19f0ec5+        .quad   0xfb9d37c3bc1ab6eb+        .quad   0x02be14534d57a240+        .quad   0xf4d73415f8a5e1f6+        .quad   0x5964f4300ccc8188+        .quad   0xe44a23152d096800+        .quad   0x5c08c55970866996+        .quad   0xdf2db60a46affb6e+        .quad   0x579155c1f856fd89++        // 2^180 * 4 * G++        .quad   0x96324edd12e0c9ef+        .quad   0x468b878df2420297+        .quad   0x199a3776a4f573be+        .quad   0x1e7fbcf18e91e92a+        .quad   0xb5f16b630817e7a6+        .quad   0x808c69233c351026+        .quad   0x324a983b54cef201+        .quad   0x53c092084a485345+        .quad   0xd2d41481f1cbafbf+        .quad   0x231d2db6716174e5+        .quad   0x0b7d7656e2a55c98+        .quad   0x3e955cd82aa495f6++        // 2^180 * 5 * G++        .quad   0xe48f535e3ed15433+        .quad   0xd075692a0d7270a3+        .quad   0x40fbd21daade6387+        .quad   0x14264887cf4495f5+        .quad   0xab39f3ef61bb3a3f+        .quad   0x8eb400652eb9193e+        .quad   0xb5de6ecc38c11f74+        .quad   0x654d7e9626f3c49f+        .quad   0xe564cfdd5c7d2ceb+        .quad   0x82eeafded737ccb9+        .quad   0x6107db62d1f9b0ab+        .quad   0x0b6baac3b4358dbb++        // 2^180 * 6 * G++        .quad   0x7ae62bcb8622fe98+        .quad   0x47762256ceb891af+        .quad   0x1a5a92bcf2e406b4+        .quad   0x7d29401784e41501+        .quad   0x204abad63700a93b+        .quad   0xbe0023d3da779373+        .quad   0xd85f0346633ab709+        .quad   0x00496dc490820412+        .quad   0x1c74b88dc27e6360+        .quad   0x074854268d14850c+        .quad   0xa145fb7b3e0dcb30+        .quad   0x10843f1b43803b23++        // 2^180 * 7 * G++        .quad   0xc5f90455376276dd+        .quad   0xce59158dd7645cd9+        .quad   0x92f65d511d366b39+        .quad   0x11574b6e526996c4+        .quad   0xd56f672de324689b+        .quad   0xd1da8aedb394a981+        .quad   0xdd7b58fe9168cfed+        .quad   0x7ce246cd4d56c1e8+        .quad   0xb8f4308e7f80be53+        .quad   0x5f3cb8cb34a9d397+        .quad   0x18a961bd33cc2b2c+        .quad   0x710045fb3a9af671++        // 2^180 * 8 * G++        .quad   0x73f93d36101b95eb+        .quad   0xfaef33794f6f4486+        .quad   0x5651735f8f15e562+        .quad   0x7fa3f19058b40da1+        .quad   0xa03fc862059d699e+        .quad   0x2370cfa19a619e69+        .quad   0xc4fe3b122f823deb+        .quad   0x1d1b056fa7f0844e+        .quad   0x1bc64631e56bf61f+        .quad   0xd379ab106e5382a3+        .quad   0x4d58c57e0540168d+        .quad   0x566256628442d8e4++        // 2^184 * 1 * G++        .quad   0xb9e499def6267ff6+        .quad   0x7772ca7b742c0843+        .quad   0x23a0153fe9a4f2b1+        .quad   0x2cdfdfecd5d05006+        .quad   0xdd499cd61ff38640+        .quad   0x29cd9bc3063625a0+        .quad   0x51e2d8023dd73dc3+        .quad   0x4a25707a203b9231+        .quad   0x2ab7668a53f6ed6a+        .quad   0x304242581dd170a1+        .quad   0x4000144c3ae20161+        .quad   0x5721896d248e49fc++        // 2^184 * 2 * G++        .quad   0x0b6e5517fd181bae+        .quad   0x9022629f2bb963b4+        .quad   0x5509bce932064625+        .quad   0x578edd74f63c13da+        .quad   0x285d5091a1d0da4e+        .quad   0x4baa6fa7b5fe3e08+        .quad   0x63e5177ce19393b3+        .quad   0x03c935afc4b030fd+        .quad   0x997276c6492b0c3d+        .quad   0x47ccc2c4dfe205fc+        .quad   0xdcd29b84dd623a3c+        .quad   0x3ec2ab590288c7a2++        // 2^184 * 3 * G++        .quad   0xa1a0d27be4d87bb9+        .quad   0xa98b4deb61391aed+        .quad   0x99a0ddd073cb9b83+        .quad   0x2dd5c25a200fcace+        .quad   0xa7213a09ae32d1cb+        .quad   0x0f2b87df40f5c2d5+        .quad   0x0baea4c6e81eab29+        .quad   0x0e1bf66c6adbac5e+        .quad   0xe2abd5e9792c887e+        .quad   0x1a020018cb926d5d+        .quad   0xbfba69cdbaae5f1e+        .quad   0x730548b35ae88f5f++        // 2^184 * 4 * G++        .quad   0xc43551a3cba8b8ee+        .quad   0x65a26f1db2115f16+        .quad   0x760f4f52ab8c3850+        .quad   0x3043443b411db8ca+        .quad   0x805b094ba1d6e334+        .quad   0xbf3ef17709353f19+        .quad   0x423f06cb0622702b+        .quad   0x585a2277d87845dd+        .quad   0xa18a5f8233d48962+        .quad   0x6698c4b5ec78257f+        .quad   0xa78e6fa5373e41ff+        .quad   0x7656278950ef981f++        // 2^184 * 5 * G++        .quad   0x38c3cf59d51fc8c0+        .quad   0x9bedd2fd0506b6f2+        .quad   0x26bf109fab570e8f+        .quad   0x3f4160a8c1b846a6+        .quad   0xe17073a3ea86cf9d+        .quad   0x3a8cfbb707155fdc+        .quad   0x4853e7fc31838a8e+        .quad   0x28bbf484b613f616+        .quad   0xf2612f5c6f136c7c+        .quad   0xafead107f6dd11be+        .quad   0x527e9ad213de6f33+        .quad   0x1e79cb358188f75d++        // 2^184 * 6 * G++        .quad   0x013436c3eef7e3f1+        .quad   0x828b6a7ffe9e10f8+        .quad   0x7ff908e5bcf9defc+        .quad   0x65d7951b3a3b3831+        .quad   0x77e953d8f5e08181+        .quad   0x84a50c44299dded9+        .quad   0xdc6c2d0c864525e5+        .quad   0x478ab52d39d1f2f4+        .quad   0x66a6a4d39252d159+        .quad   0xe5dde1bc871ac807+        .quad   0xb82c6b40a6c1c96f+        .quad   0x16d87a411a212214++        // 2^184 * 7 * G++        .quad   0xb3bd7e5a42066215+        .quad   0x879be3cd0c5a24c1+        .quad   0x57c05db1d6f994b7+        .quad   0x28f87c8165f38ca6+        .quad   0xfba4d5e2d54e0583+        .quad   0xe21fafd72ebd99fa+        .quad   0x497ac2736ee9778f+        .quad   0x1f990b577a5a6dde+        .quad   0xa3344ead1be8f7d6+        .quad   0x7d1e50ebacea798f+        .quad   0x77c6569e520de052+        .quad   0x45882fe1534d6d3e++        // 2^184 * 8 * G++        .quad   0x6669345d757983d6+        .quad   0x62b6ed1117aa11a6+        .quad   0x7ddd1857985e128f+        .quad   0x688fe5b8f626f6dd+        .quad   0xd8ac9929943c6fe4+        .quad   0xb5f9f161a38392a2+        .quad   0x2699db13bec89af3+        .quad   0x7dcf843ce405f074+        .quad   0x6c90d6484a4732c0+        .quad   0xd52143fdca563299+        .quad   0xb3be28c3915dc6e1+        .quad   0x6739687e7327191b++        // 2^188 * 1 * G++        .quad   0x9f65c5ea200814cf+        .quad   0x840536e169a31740+        .quad   0x8b0ed13925c8b4ad+        .quad   0x0080dbafe936361d+        .quad   0x8ce5aad0c9cb971f+        .quad   0x1156aaa99fd54a29+        .quad   0x41f7247015af9b78+        .quad   0x1fe8cca8420f49aa+        .quad   0x72a1848f3c0cc82a+        .quad   0x38c560c2877c9e54+        .quad   0x5004e228ce554140+        .quad   0x042418a103429d71++        // 2^188 * 2 * G++        .quad   0x899dea51abf3ff5f+        .quad   0x9b93a8672fc2d8ba+        .quad   0x2c38cb97be6ebd5c+        .quad   0x114d578497263b5d+        .quad   0x58e84c6f20816247+        .quad   0x8db2b2b6e36fd793+        .quad   0x977182561d484d85+        .quad   0x0822024f8632abd7+        .quad   0xb301bb7c6b1beca3+        .quad   0x55393f6dc6eb1375+        .quad   0x910d281097b6e4eb+        .quad   0x1ad4548d9d479ea3++        // 2^188 * 3 * G++        .quad   0xcd5a7da0389a48fd+        .quad   0xb38fa4aa9a78371e+        .quad   0xc6d9761b2cdb8e6c+        .quad   0x35cf51dbc97e1443+        .quad   0xa06fe66d0fe9fed3+        .quad   0xa8733a401c587909+        .quad   0x30d14d800df98953+        .quad   0x41ce5876c7b30258+        .quad   0x59ac3bc5d670c022+        .quad   0xeae67c109b119406+        .quad   0x9798bdf0b3782fda+        .quad   0x651e3201fd074092++        // 2^188 * 4 * G++        .quad   0xd63d8483ef30c5cf+        .quad   0x4cd4b4962361cc0c+        .quad   0xee90e500a48426ac+        .quad   0x0af51d7d18c14eeb+        .quad   0xa57ba4a01efcae9e+        .quad   0x769f4beedc308a94+        .quad   0xd1f10eeb3603cb2e+        .quad   0x4099ce5e7e441278+        .quad   0x1ac98e4f8a5121e9+        .quad   0x7dae9544dbfa2fe0+        .quad   0x8320aa0dd6430df9+        .quad   0x667282652c4a2fb5++        // 2^188 * 5 * G++        .quad   0x874621f4d86bc9ab+        .quad   0xb54c7bbe56fe6fea+        .quad   0x077a24257fadc22c+        .quad   0x1ab53be419b90d39+        .quad   0xada8b6e02946db23+        .quad   0x1c0ce51a7b253ab7+        .quad   0x8448c85a66dd485b+        .quad   0x7f1fc025d0675adf+        .quad   0xd8ee1b18319ea6aa+        .quad   0x004d88083a21f0da+        .quad   0x3bd6aa1d883a4f4b+        .quad   0x4db9a3a6dfd9fd14++        // 2^188 * 6 * G++        .quad   0x8ce7b23bb99c0755+        .quad   0x35c5d6edc4f50f7a+        .quad   0x7e1e2ed2ed9b50c3+        .quad   0x36305f16e8934da1+        .quad   0xd95b00bbcbb77c68+        .quad   0xddbc846a91f17849+        .quad   0x7cf700aebe28d9b3+        .quad   0x5ce1285c85d31f3e+        .quad   0x31b6972d98b0bde8+        .quad   0x7d920706aca6de5b+        .quad   0xe67310f8908a659f+        .quad   0x50fac2a6efdf0235++        // 2^188 * 7 * G++        .quad   0xf3d3a9f35b880f5a+        .quad   0xedec050cdb03e7c2+        .quad   0xa896981ff9f0b1a2+        .quad   0x49a4ae2bac5e34a4+        .quad   0x295b1c86f6f449bc+        .quad   0x51b2e84a1f0ab4dd+        .quad   0xc001cb30aa8e551d+        .quad   0x6a28d35944f43662+        .quad   0x28bb12ee04a740e0+        .quad   0x14313bbd9bce8174+        .quad   0x72f5b5e4e8c10c40+        .quad   0x7cbfb19936adcd5b++        // 2^188 * 8 * G++        .quad   0xa311ddc26b89792d+        .quad   0x1b30b4c6da512664+        .quad   0x0ca77b4ccf150859+        .quad   0x1de443df1b009408+        .quad   0x8e793a7acc36e6e0+        .quad   0xf9fab7a37d586eed+        .quad   0x3a4f9692bae1f4e4+        .quad   0x1c14b03eff5f447e+        .quad   0x19647bd114a85291+        .quad   0x57b76cb21034d3af+        .quad   0x6329db440f9d6dfa+        .quad   0x5ef43e586a571493++        // 2^192 * 1 * G++        .quad   0xef782014385675a6+        .quad   0xa2649f30aafda9e8+        .quad   0x4cd1eb505cdfa8cb+        .quad   0x46115aba1d4dc0b3+        .quad   0xa66dcc9dc80c1ac0+        .quad   0x97a05cf41b38a436+        .quad   0xa7ebf3be95dbd7c6+        .quad   0x7da0b8f68d7e7dab+        .quad   0xd40f1953c3b5da76+        .quad   0x1dac6f7321119e9b+        .quad   0x03cc6021feb25960+        .quad   0x5a5f887e83674b4b++        // 2^192 * 2 * G++        .quad   0x8f6301cf70a13d11+        .quad   0xcfceb815350dd0c4+        .quad   0xf70297d4a4bca47e+        .quad   0x3669b656e44d1434+        .quad   0x9e9628d3a0a643b9+        .quad   0xb5c3cb00e6c32064+        .quad   0x9b5302897c2dec32+        .quad   0x43e37ae2d5d1c70c+        .quad   0x387e3f06eda6e133+        .quad   0x67301d5199a13ac0+        .quad   0xbd5ad8f836263811+        .quad   0x6a21e6cd4fd5e9be++        // 2^192 * 3 * G++        .quad   0xf1c6170a3046e65f+        .quad   0x58712a2a00d23524+        .quad   0x69dbbd3c8c82b755+        .quad   0x586bf9f1a195ff57+        .quad   0xef4129126699b2e3+        .quad   0x71d30847708d1301+        .quad   0x325432d01182b0bd+        .quad   0x45371b07001e8b36+        .quad   0xa6db088d5ef8790b+        .quad   0x5278f0dc610937e5+        .quad   0xac0349d261a16eb8+        .quad   0x0eafb03790e52179++        // 2^192 * 4 * G++        .quad   0x960555c13748042f+        .quad   0x219a41e6820baa11+        .quad   0x1c81f73873486d0c+        .quad   0x309acc675a02c661+        .quad   0x5140805e0f75ae1d+        .quad   0xec02fbe32662cc30+        .quad   0x2cebdf1eea92396d+        .quad   0x44ae3344c5435bb3+        .quad   0x9cf289b9bba543ee+        .quad   0xf3760e9d5ac97142+        .quad   0x1d82e5c64f9360aa+        .quad   0x62d5221b7f94678f++        // 2^192 * 5 * G++        .quad   0x524c299c18d0936d+        .quad   0xc86bb56c8a0c1a0c+        .quad   0xa375052edb4a8631+        .quad   0x5c0efde4bc754562+        .quad   0x7585d4263af77a3c+        .quad   0xdfae7b11fee9144d+        .quad   0xa506708059f7193d+        .quad   0x14f29a5383922037+        .quad   0xdf717edc25b2d7f5+        .quad   0x21f970db99b53040+        .quad   0xda9234b7c3ed4c62+        .quad   0x5e72365c7bee093e++        // 2^192 * 6 * G++        .quad   0x575bfc074571217f+        .quad   0x3779675d0694d95b+        .quad   0x9a0a37bbf4191e33+        .quad   0x77f1104c47b4eabc+        .quad   0x7d9339062f08b33e+        .quad   0x5b9659e5df9f32be+        .quad   0xacff3dad1f9ebdfd+        .quad   0x70b20555cb7349b7+        .quad   0xbe5113c555112c4c+        .quad   0x6688423a9a881fcd+        .quad   0x446677855e503b47+        .quad   0x0e34398f4a06404a++        // 2^192 * 7 * G++        .quad   0xb67d22d93ecebde8+        .quad   0x09b3e84127822f07+        .quad   0x743fa61fb05b6d8d+        .quad   0x5e5405368a362372+        .quad   0x18930b093e4b1928+        .quad   0x7de3e10e73f3f640+        .quad   0xf43217da73395d6f+        .quad   0x6f8aded6ca379c3e+        .quad   0xe340123dfdb7b29a+        .quad   0x487b97e1a21ab291+        .quad   0xf9967d02fde6949e+        .quad   0x780de72ec8d3de97++        // 2^192 * 8 * G++        .quad   0x0ae28545089ae7bc+        .quad   0x388ddecf1c7f4d06+        .quad   0x38ac15510a4811b8+        .quad   0x0eb28bf671928ce4+        .quad   0x671feaf300f42772+        .quad   0x8f72eb2a2a8c41aa+        .quad   0x29a17fd797373292+        .quad   0x1defc6ad32b587a6+        .quad   0xaf5bbe1aef5195a7+        .quad   0x148c1277917b15ed+        .quad   0x2991f7fb7ae5da2e+        .quad   0x467d201bf8dd2867++        // 2^196 * 1 * G++        .quad   0x7906ee72f7bd2e6b+        .quad   0x05d270d6109abf4e+        .quad   0x8d5cfe45b941a8a4+        .quad   0x44c218671c974287+        .quad   0x745f9d56296bc318+        .quad   0x993580d4d8152e65+        .quad   0xb0e5b13f5839e9ce+        .quad   0x51fc2b28d43921c0+        .quad   0x1b8fd11795e2a98c+        .quad   0x1c4e5ee12b6b6291+        .quad   0x5b30e7107424b572+        .quad   0x6e6b9de84c4f4ac6++        // 2^196 * 2 * G++        .quad   0xdff25fce4b1de151+        .quad   0xd841c0c7e11c4025+        .quad   0x2554b3c854749c87+        .quad   0x2d292459908e0df9+        .quad   0x6b7c5f10f80cb088+        .quad   0x736b54dc56e42151+        .quad   0xc2b620a5c6ef99c4+        .quad   0x5f4c802cc3a06f42+        .quad   0x9b65c8f17d0752da+        .quad   0x881ce338c77ee800+        .quad   0xc3b514f05b62f9e3+        .quad   0x66ed5dd5bec10d48++        // 2^196 * 3 * G++        .quad   0x7d38a1c20bb2089d+        .quad   0x808334e196ccd412+        .quad   0xc4a70b8c6c97d313+        .quad   0x2eacf8bc03007f20+        .quad   0xf0adf3c9cbca047d+        .quad   0x81c3b2cbf4552f6b+        .quad   0xcfda112d44735f93+        .quad   0x1f23a0c77e20048c+        .quad   0xf235467be5bc1570+        .quad   0x03d2d9020dbab38c+        .quad   0x27529aa2fcf9e09e+        .quad   0x0840bef29d34bc50++        // 2^196 * 4 * G++        .quad   0x796dfb35dc10b287+        .quad   0x27176bcd5c7ff29d+        .quad   0x7f3d43e8c7b24905+        .quad   0x0304f5a191c54276+        .quad   0xcd54e06b7f37e4eb+        .quad   0x8cc15f87f5e96cca+        .quad   0xb8248bb0d3597dce+        .quad   0x246affa06074400c+        .quad   0x37d88e68fbe45321+        .quad   0x86097548c0d75032+        .quad   0x4e9b13ef894a0d35+        .quad   0x25a83cac5753d325++        // 2^196 * 5 * G++        .quad   0x10222f48eed8165e+        .quad   0x623fc1234b8bcf3a+        .quad   0x1e145c09c221e8f0+        .quad   0x7ccfa59fca782630+        .quad   0x9f0f66293952b6e2+        .quad   0x33db5e0e0934267b+        .quad   0xff45252bd609fedc+        .quad   0x06be10f5c506e0c9+        .quad   0x1a9615a9b62a345f+        .quad   0x22050c564a52fecc+        .quad   0xa7a2788528bc0dfe+        .quad   0x5e82770a1a1ee71d++        // 2^196 * 6 * G++        .quad   0x35425183ad896a5c+        .quad   0xe8673afbe78d52f6+        .quad   0x2c66f25f92a35f64+        .quad   0x09d04f3b3b86b102+        .quad   0xe802e80a42339c74+        .quad   0x34175166a7fffae5+        .quad   0x34865d1f1c408cae+        .quad   0x2cca982c605bc5ee+        .quad   0xfd2d5d35197dbe6e+        .quad   0x207c2eea8be4ffa3+        .quad   0x2613d8db325ae918+        .quad   0x7a325d1727741d3e++        // 2^196 * 7 * G++        .quad   0xd036b9bbd16dfde2+        .quad   0xa2055757c497a829+        .quad   0x8e6cc966a7f12667+        .quad   0x4d3b1a791239c180+        .quad   0xecd27d017e2a076a+        .quad   0xd788689f1636495e+        .quad   0x52a61af0919233e5+        .quad   0x2a479df17bb1ae64+        .quad   0x9e5eee8e33db2710+        .quad   0x189854ded6c43ca5+        .quad   0xa41c22c592718138+        .quad   0x27ad5538a43a5e9b++        // 2^196 * 8 * G++        .quad   0x2746dd4b15350d61+        .quad   0xd03fcbc8ee9521b7+        .quad   0xe86e365a138672ca+        .quad   0x510e987f7e7d89e2+        .quad   0xcb5a7d638e47077c+        .quad   0x8db7536120a1c059+        .quad   0x549e1e4d8bedfdcc+        .quad   0x080153b7503b179d+        .quad   0xdda69d930a3ed3e3+        .quad   0x3d386ef1cd60a722+        .quad   0xc817ad58bdaa4ee6+        .quad   0x23be8d554fe7372a++        // 2^200 * 1 * G++        .quad   0x95fe919a74ef4fad+        .quad   0x3a827becf6a308a2+        .quad   0x964e01d309a47b01+        .quad   0x71c43c4f5ba3c797+        .quad   0xbc1ef4bd567ae7a9+        .quad   0x3f624cb2d64498bd+        .quad   0xe41064d22c1f4ec8+        .quad   0x2ef9c5a5ba384001+        .quad   0xb6fd6df6fa9e74cd+        .quad   0xf18278bce4af267a+        .quad   0x8255b3d0f1ef990e+        .quad   0x5a758ca390c5f293++        // 2^200 * 2 * G++        .quad   0xa2b72710d9462495+        .quad   0x3aa8c6d2d57d5003+        .quad   0xe3d400bfa0b487ca+        .quad   0x2dbae244b3eb72ec+        .quad   0x8ce0918b1d61dc94+        .quad   0x8ded36469a813066+        .quad   0xd4e6a829afe8aad3+        .quad   0x0a738027f639d43f+        .quad   0x980f4a2f57ffe1cc+        .quad   0x00670d0de1839843+        .quad   0x105c3f4a49fb15fd+        .quad   0x2698ca635126a69c++        // 2^200 * 3 * G++        .quad   0xe765318832b0ba78+        .quad   0x381831f7925cff8b+        .quad   0x08a81b91a0291fcc+        .quad   0x1fb43dcc49caeb07+        .quad   0x2e3d702f5e3dd90e+        .quad   0x9e3f0918e4d25386+        .quad   0x5e773ef6024da96a+        .quad   0x3c004b0c4afa3332+        .quad   0x9aa946ac06f4b82b+        .quad   0x1ca284a5a806c4f3+        .quad   0x3ed3265fc6cd4787+        .quad   0x6b43fd01cd1fd217++        // 2^200 * 4 * G++        .quad   0xc7a75d4b4697c544+        .quad   0x15fdf848df0fffbf+        .quad   0x2868b9ebaa46785a+        .quad   0x5a68d7105b52f714+        .quad   0xb5c742583e760ef3+        .quad   0x75dc52b9ee0ab990+        .quad   0xbf1427c2072b923f+        .quad   0x73420b2d6ff0d9f0+        .quad   0xaf2cf6cb9e851e06+        .quad   0x8f593913c62238c4+        .quad   0xda8ab89699fbf373+        .quad   0x3db5632fea34bc9e++        // 2^200 * 5 * G++        .quad   0xf46eee2bf75dd9d8+        .quad   0x0d17b1f6396759a5+        .quad   0x1bf2d131499e7273+        .quad   0x04321adf49d75f13+        .quad   0x2e4990b1829825d5+        .quad   0xedeaeb873e9a8991+        .quad   0xeef03d394c704af8+        .quad   0x59197ea495df2b0e+        .quad   0x04e16019e4e55aae+        .quad   0xe77b437a7e2f92e9+        .quad   0xc7ce2dc16f159aa4+        .quad   0x45eafdc1f4d70cc0++        // 2^200 * 6 * G++        .quad   0x698401858045d72b+        .quad   0x4c22faa2cf2f0651+        .quad   0x941a36656b222dc6+        .quad   0x5a5eebc80362dade+        .quad   0xb60e4624cfccb1ed+        .quad   0x59dbc292bd5c0395+        .quad   0x31a09d1ddc0481c9+        .quad   0x3f73ceea5d56d940+        .quad   0xb7a7bfd10a4e8dc6+        .quad   0xbe57007e44c9b339+        .quad   0x60c1207f1557aefa+        .quad   0x26058891266218db++        // 2^200 * 7 * G++        .quad   0x59f704a68360ff04+        .quad   0xc3d93fde7661e6f4+        .quad   0x831b2a7312873551+        .quad   0x54ad0c2e4e615d57+        .quad   0x4c818e3cc676e542+        .quad   0x5e422c9303ceccad+        .quad   0xec07cccab4129f08+        .quad   0x0dedfa10b24443b8+        .quad   0xee3b67d5b82b522a+        .quad   0x36f163469fa5c1eb+        .quad   0xa5b4d2f26ec19fd3+        .quad   0x62ecb2baa77a9408++        // 2^200 * 8 * G++        .quad   0xe5ed795261152b3d+        .quad   0x4962357d0eddd7d1+        .quad   0x7482c8d0b96b4c71+        .quad   0x2e59f919a966d8be+        .quad   0x92072836afb62874+        .quad   0x5fcd5e8579e104a5+        .quad   0x5aad01adc630a14a+        .quad   0x61913d5075663f98+        .quad   0x0dc62d361a3231da+        .quad   0xfa47583294200270+        .quad   0x02d801513f9594ce+        .quad   0x3ddbc2a131c05d5c++        // 2^204 * 1 * G++        .quad   0x3f50a50a4ffb81ef+        .quad   0xb1e035093bf420bf+        .quad   0x9baa8e1cc6aa2cd0+        .quad   0x32239861fa237a40+        .quad   0xfb735ac2004a35d1+        .quad   0x31de0f433a6607c3+        .quad   0x7b8591bfc528d599+        .quad   0x55be9a25f5bb050c+        .quad   0x0d005acd33db3dbf+        .quad   0x0111b37c80ac35e2+        .quad   0x4892d66c6f88ebeb+        .quad   0x770eadb16508fbcd++        // 2^204 * 2 * G++        .quad   0x8451f9e05e4e89dd+        .quad   0xc06302ffbc793937+        .quad   0x5d22749556a6495c+        .quad   0x09a6755ca05603fb+        .quad   0xf1d3b681a05071b9+        .quad   0x2207659a3592ff3a+        .quad   0x5f0169297881e40e+        .quad   0x16bedd0e86ba374e+        .quad   0x5ecccc4f2c2737b5+        .quad   0x43b79e0c2dccb703+        .quad   0x33e008bc4ec43df3+        .quad   0x06c1b840f07566c0++        // 2^204 * 3 * G++        .quad   0x7688a5c6a388f877+        .quad   0x02a96c14deb2b6ac+        .quad   0x64c9f3431b8c2af8+        .quad   0x3628435554a1eed6+        .quad   0x69ee9e7f9b02805c+        .quad   0xcbff828a547d1640+        .quad   0x3d93a869b2430968+        .quad   0x46b7b8cd3fe26972+        .quad   0xe9812086fe7eebe0+        .quad   0x4cba6be72f515437+        .quad   0x1d04168b516efae9+        .quad   0x5ea1391043982cb9++        // 2^204 * 4 * G++        .quad   0x49125c9cf4702ee1+        .quad   0x4520b71f8b25b32d+        .quad   0x33193026501fef7e+        .quad   0x656d8997c8d2eb2b+        .quad   0x6f2b3be4d5d3b002+        .quad   0xafec33d96a09c880+        .quad   0x035f73a4a8bcc4cc+        .quad   0x22c5b9284662198b+        .quad   0xcb58c8fe433d8939+        .quad   0x89a0cb2e6a8d7e50+        .quad   0x79ca955309fbbe5a+        .quad   0x0c626616cd7fc106++        // 2^204 * 5 * G++        .quad   0x1ffeb80a4879b61f+        .quad   0x6396726e4ada21ed+        .quad   0x33c7b093368025ba+        .quad   0x471aa0c6f3c31788+        .quad   0x8fdfc379fbf454b1+        .quad   0x45a5a970f1a4b771+        .quad   0xac921ef7bad35915+        .quad   0x42d088dca81c2192+        .quad   0x8fda0f37a0165199+        .quad   0x0adadb77c8a0e343+        .quad   0x20fbfdfcc875e820+        .quad   0x1cf2bea80c2206e7++        // 2^204 * 6 * G++        .quad   0xc2ddf1deb36202ac+        .quad   0x92a5fe09d2e27aa5+        .quad   0x7d1648f6fc09f1d3+        .quad   0x74c2cc0513bc4959+        .quad   0x982d6e1a02c0412f+        .quad   0x90fa4c83db58e8fe+        .quad   0x01c2f5bcdcb18bc0+        .quad   0x686e0c90216abc66+        .quad   0x1fadbadba54395a7+        .quad   0xb41a02a0ae0da66a+        .quad   0xbf19f598bba37c07+        .quad   0x6a12b8acde48430d++        // 2^204 * 7 * G++        .quad   0xf8daea1f39d495d9+        .quad   0x592c190e525f1dfc+        .quad   0xdb8cbd04c9991d1b+        .quad   0x11f7fda3d88f0cb7+        .quad   0x793bdd801aaeeb5f+        .quad   0x00a2a0aac1518871+        .quad   0xe8a373a31f2136b4+        .quad   0x48aab888fc91ef19+        .quad   0x041f7e925830f40e+        .quad   0x002d6ca979661c06+        .quad   0x86dc9ff92b046a2e+        .quad   0x760360928b0493d1++        // 2^204 * 8 * G++        .quad   0x21bb41c6120cf9c6+        .quad   0xeab2aa12decda59b+        .quad   0xc1a72d020aa48b34+        .quad   0x215d4d27e87d3b68+        .quad   0xb43108e5695a0b05+        .quad   0x6cb00ee8ad37a38b+        .quad   0x5edad6eea3537381+        .quad   0x3f2602d4b6dc3224+        .quad   0xc8b247b65bcaf19c+        .quad   0x49779dc3b1b2c652+        .quad   0x89a180bbd5ece2e2+        .quad   0x13f098a3cec8e039++        // 2^208 * 1 * G++        .quad   0x9adc0ff9ce5ec54b+        .quad   0x039c2a6b8c2f130d+        .quad   0x028007c7f0f89515+        .quad   0x78968314ac04b36b+        .quad   0xf3aa57a22796bb14+        .quad   0x883abab79b07da21+        .quad   0xe54be21831a0391c+        .quad   0x5ee7fb38d83205f9+        .quad   0x538dfdcb41446a8e+        .quad   0xa5acfda9434937f9+        .quad   0x46af908d263c8c78+        .quad   0x61d0633c9bca0d09++        // 2^208 * 2 * G++        .quad   0x63744935ffdb2566+        .quad   0xc5bd6b89780b68bb+        .quad   0x6f1b3280553eec03+        .quad   0x6e965fd847aed7f5+        .quad   0xada328bcf8fc73df+        .quad   0xee84695da6f037fc+        .quad   0x637fb4db38c2a909+        .quad   0x5b23ac2df8067bdc+        .quad   0x9ad2b953ee80527b+        .quad   0xe88f19aafade6d8d+        .quad   0x0e711704150e82cf+        .quad   0x79b9bbb9dd95dedc++        // 2^208 * 3 * G++        .quad   0xebb355406a3126c2+        .quad   0xd26383a868c8c393+        .quad   0x6c0c6429e5b97a82+        .quad   0x5065f158c9fd2147+        .quad   0xd1997dae8e9f7374+        .quad   0xa032a2f8cfbb0816+        .quad   0xcd6cba126d445f0a+        .quad   0x1ba811460accb834+        .quad   0x708169fb0c429954+        .quad   0xe14600acd76ecf67+        .quad   0x2eaab98a70e645ba+        .quad   0x3981f39e58a4faf2++        // 2^208 * 4 * G++        .quad   0x18fb8a7559230a93+        .quad   0x1d168f6960e6f45d+        .quad   0x3a85a94514a93cb5+        .quad   0x38dc083705acd0fd+        .quad   0xc845dfa56de66fde+        .quad   0xe152a5002c40483a+        .quad   0xe9d2e163c7b4f632+        .quad   0x30f4452edcbc1b65+        .quad   0x856d2782c5759740+        .quad   0xfa134569f99cbecc+        .quad   0x8844fc73c0ea4e71+        .quad   0x632d9a1a593f2469++        // 2^208 * 5 * G++        .quad   0xf6bb6b15b807cba6+        .quad   0x1823c7dfbc54f0d7+        .quad   0xbb1d97036e29670b+        .quad   0x0b24f48847ed4a57+        .quad   0xbf09fd11ed0c84a7+        .quad   0x63f071810d9f693a+        .quad   0x21908c2d57cf8779+        .quad   0x3a5a7df28af64ba2+        .quad   0xdcdad4be511beac7+        .quad   0xa4538075ed26ccf2+        .quad   0xe19cff9f005f9a65+        .quad   0x34fcf74475481f63++        // 2^208 * 6 * G++        .quad   0xc197e04c789767ca+        .quad   0xb8714dcb38d9467d+        .quad   0x55de888283f95fa8+        .quad   0x3d3bdc164dfa63f7+        .quad   0xa5bb1dab78cfaa98+        .quad   0x5ceda267190b72f2+        .quad   0x9309c9110a92608e+        .quad   0x0119a3042fb374b0+        .quad   0x67a2d89ce8c2177d+        .quad   0x669da5f66895d0c1+        .quad   0xf56598e5b282a2b0+        .quad   0x56c088f1ede20a73++        // 2^208 * 7 * G++        .quad   0x336d3d1110a86e17+        .quad   0xd7f388320b75b2fa+        .quad   0xf915337625072988+        .quad   0x09674c6b99108b87+        .quad   0x581b5fac24f38f02+        .quad   0xa90be9febae30cbd+        .quad   0x9a2169028acf92f0+        .quad   0x038b7ea48359038f+        .quad   0x9f4ef82199316ff8+        .quad   0x2f49d282eaa78d4f+        .quad   0x0971a5ab5aef3174+        .quad   0x6e5e31025969eb65++        // 2^208 * 8 * G++        .quad   0xb16c62f587e593fb+        .quad   0x4999eddeca5d3e71+        .quad   0xb491c1e014cc3e6d+        .quad   0x08f5114789a8dba8+        .quad   0x3304fb0e63066222+        .quad   0xfb35068987acba3f+        .quad   0xbd1924778c1061a3+        .quad   0x3058ad43d1838620+        .quad   0x323c0ffde57663d0+        .quad   0x05c3df38a22ea610+        .quad   0xbdc78abdac994f9a+        .quad   0x26549fa4efe3dc99++        // 2^212 * 1 * G++        .quad   0x738b38d787ce8f89+        .quad   0xb62658e24179a88d+        .quad   0x30738c9cf151316d+        .quad   0x49128c7f727275c9+        .quad   0x04dbbc17f75396b9+        .quad   0x69e6a2d7d2f86746+        .quad   0xc6409d99f53eabc6+        .quad   0x606175f6332e25d2+        .quad   0x4021370ef540e7dd+        .quad   0x0910d6f5a1f1d0a5+        .quad   0x4634aacd5b06b807+        .quad   0x6a39e6356944f235++        // 2^212 * 2 * G++        .quad   0x96cd5640df90f3e7+        .quad   0x6c3a760edbfa25ea+        .quad   0x24f3ef0959e33cc4+        .quad   0x42889e7e530d2e58+        .quad   0x1da1965774049e9d+        .quad   0xfbcd6ea198fe352b+        .quad   0xb1cbcd50cc5236a6+        .quad   0x1f5ec83d3f9846e2+        .quad   0x8efb23c3328ccb75+        .quad   0xaf42a207dd876ee9+        .quad   0x20fbdadc5dfae796+        .quad   0x241e246b06bf9f51++        // 2^212 * 3 * G++        .quad   0x29e68e57ad6e98f6+        .quad   0x4c9260c80b462065+        .quad   0x3f00862ea51ebb4b+        .quad   0x5bc2c77fb38d9097+        .quad   0x7eaafc9a6280bbb8+        .quad   0x22a70f12f403d809+        .quad   0x31ce40bb1bfc8d20+        .quad   0x2bc65635e8bd53ee+        .quad   0xe8d5dc9fa96bad93+        .quad   0xe58fb17dde1947dc+        .quad   0x681532ea65185fa3+        .quad   0x1fdd6c3b034a7830++        // 2^212 * 4 * G++        .quad   0x0a64e28c55dc18fe+        .quad   0xe3df9e993399ebdd+        .quad   0x79ac432370e2e652+        .quad   0x35ff7fc33ae4cc0e+        .quad   0x9c13a6a52dd8f7a9+        .quad   0x2dbb1f8c3efdcabf+        .quad   0x961e32405e08f7b5+        .quad   0x48c8a121bbe6c9e5+        .quad   0xfc415a7c59646445+        .quad   0xd224b2d7c128b615+        .quad   0x6035c9c905fbb912+        .quad   0x42d7a91274429fab++        // 2^212 * 5 * G++        .quad   0x4e6213e3eaf72ed3+        .quad   0x6794981a43acd4e7+        .quad   0xff547cde6eb508cb+        .quad   0x6fed19dd10fcb532+        .quad   0xa9a48947933da5bc+        .quad   0x4a58920ec2e979ec+        .quad   0x96d8800013e5ac4c+        .quad   0x453692d74b48b147+        .quad   0xdd775d99a8559c6f+        .quad   0xf42a2140df003e24+        .quad   0x5223e229da928a66+        .quad   0x063f46ba6d38f22c++        // 2^212 * 6 * G++        .quad   0xd2d242895f536694+        .quad   0xca33a2c542939b2c+        .quad   0x986fada6c7ddb95c+        .quad   0x5a152c042f712d5d+        .quad   0x39843cb737346921+        .quad   0xa747fb0738c89447+        .quad   0xcb8d8031a245307e+        .quad   0x67810f8e6d82f068+        .quad   0x3eeb8fbcd2287db4+        .quad   0x72c7d3a301a03e93+        .quad   0x5473e88cbd98265a+        .quad   0x7324aa515921b403++        // 2^212 * 7 * G++        .quad   0x857942f46c3cbe8e+        .quad   0xa1d364b14730c046+        .quad   0x1c8ed914d23c41bf+        .quad   0x0838e161eef6d5d2+        .quad   0xad23f6dae82354cb+        .quad   0x6962502ab6571a6d+        .quad   0x9b651636e38e37d1+        .quad   0x5cac5005d1a3312f+        .quad   0x8cc154cce9e39904+        .quad   0x5b3a040b84de6846+        .quad   0xc4d8a61cb1be5d6e+        .quad   0x40fb897bd8861f02++        // 2^212 * 8 * G++        .quad   0x84c5aa9062de37a1+        .quad   0x421da5000d1d96e1+        .quad   0x788286306a9242d9+        .quad   0x3c5e464a690d10da+        .quad   0xe57ed8475ab10761+        .quad   0x71435e206fd13746+        .quad   0x342f824ecd025632+        .quad   0x4b16281ea8791e7b+        .quad   0xd1c101d50b813381+        .quad   0xdee60f1176ee6828+        .quad   0x0cb68893383f6409+        .quad   0x6183c565f6ff484a++        // 2^216 * 1 * G++        .quad   0x741d5a461e6bf9d6+        .quad   0x2305b3fc7777a581+        .quad   0xd45574a26474d3d9+        .quad   0x1926e1dc6401e0ff+        .quad   0xdb468549af3f666e+        .quad   0xd77fcf04f14a0ea5+        .quad   0x3df23ff7a4ba0c47+        .quad   0x3a10dfe132ce3c85+        .quad   0xe07f4e8aea17cea0+        .quad   0x2fd515463a1fc1fd+        .quad   0x175322fd31f2c0f1+        .quad   0x1fa1d01d861e5d15++        // 2^216 * 2 * G++        .quad   0xcc8055947d599832+        .quad   0x1e4656da37f15520+        .quad   0x99f6f7744e059320+        .quad   0x773563bc6a75cf33+        .quad   0x38dcac00d1df94ab+        .quad   0x2e712bddd1080de9+        .quad   0x7f13e93efdd5e262+        .quad   0x73fced18ee9a01e5+        .quad   0x06b1e90863139cb3+        .quad   0xa493da67c5a03ecd+        .quad   0x8d77cec8ad638932+        .quad   0x1f426b701b864f44++        // 2^216 * 3 * G++        .quad   0xefc9264c41911c01+        .quad   0xf1a3b7b817a22c25+        .quad   0x5875da6bf30f1447+        .quad   0x4e1af5271d31b090+        .quad   0xf17e35c891a12552+        .quad   0xb76b8153575e9c76+        .quad   0xfa83406f0d9b723e+        .quad   0x0b76bb1b3fa7e438+        .quad   0x08b8c1f97f92939b+        .quad   0xbe6771cbd444ab6e+        .quad   0x22e5646399bb8017+        .quad   0x7b6dd61eb772a955++        // 2^216 * 4 * G++        .quad   0xb7adc1e850f33d92+        .quad   0x7998fa4f608cd5cf+        .quad   0xad962dbd8dfc5bdb+        .quad   0x703e9bceaf1d2f4f+        .quad   0x5730abf9ab01d2c7+        .quad   0x16fb76dc40143b18+        .quad   0x866cbe65a0cbb281+        .quad   0x53fa9b659bff6afe+        .quad   0x6c14c8e994885455+        .quad   0x843a5d6665aed4e5+        .quad   0x181bb73ebcd65af1+        .quad   0x398d93e5c4c61f50++        // 2^216 * 5 * G++        .quad   0x1c4bd16733e248f3+        .quad   0xbd9e128715bf0a5f+        .quad   0xd43f8cf0a10b0376+        .quad   0x53b09b5ddf191b13+        .quad   0xc3877c60d2e7e3f2+        .quad   0x3b34aaa030828bb1+        .quad   0x283e26e7739ef138+        .quad   0x699c9c9002c30577+        .quad   0xf306a7235946f1cc+        .quad   0x921718b5cce5d97d+        .quad   0x28cdd24781b4e975+        .quad   0x51caf30c6fcdd907++        // 2^216 * 6 * G++        .quad   0xa60ba7427674e00a+        .quad   0x630e8570a17a7bf3+        .quad   0x3758563dcf3324cc+        .quad   0x5504aa292383fdaa+        .quad   0x737af99a18ac54c7+        .quad   0x903378dcc51cb30f+        .quad   0x2b89bc334ce10cc7+        .quad   0x12ae29c189f8e99a+        .quad   0xa99ec0cb1f0d01cf+        .quad   0x0dd1efcc3a34f7ae+        .quad   0x55ca7521d09c4e22+        .quad   0x5fd14fe958eba5ea++        // 2^216 * 7 * G++        .quad   0xb5dc2ddf2845ab2c+        .quad   0x069491b10a7fe993+        .quad   0x4daaf3d64002e346+        .quad   0x093ff26e586474d1+        .quad   0x3c42fe5ebf93cb8e+        .quad   0xbedfa85136d4565f+        .quad   0xe0f0859e884220e8+        .quad   0x7dd73f960725d128+        .quad   0xb10d24fe68059829+        .quad   0x75730672dbaf23e5+        .quad   0x1367253ab457ac29+        .quad   0x2f59bcbc86b470a4++        // 2^216 * 8 * G++        .quad   0x83847d429917135f+        .quad   0xad1b911f567d03d7+        .quad   0x7e7748d9be77aad1+        .quad   0x5458b42e2e51af4a+        .quad   0x7041d560b691c301+        .quad   0x85201b3fadd7e71e+        .quad   0x16c2e16311335585+        .quad   0x2aa55e3d010828b1+        .quad   0xed5192e60c07444f+        .quad   0x42c54e2d74421d10+        .quad   0x352b4c82fdb5c864+        .quad   0x13e9004a8a768664++        // 2^220 * 1 * G++        .quad   0xcbb5b5556c032bff+        .quad   0xdf7191b729297a3a+        .quad   0xc1ff7326aded81bb+        .quad   0x71ade8bb68be03f5+        .quad   0x1e6284c5806b467c+        .quad   0xc5f6997be75d607b+        .quad   0x8b67d958b378d262+        .quad   0x3d88d66a81cd8b70+        .quad   0x8b767a93204ed789+        .quad   0x762fcacb9fa0ae2a+        .quad   0x771febcc6dce4887+        .quad   0x343062158ff05fb3++        // 2^220 * 2 * G++        .quad   0xe05da1a7e1f5bf49+        .quad   0x26457d6dd4736092+        .quad   0x77dcb07773cc32f6+        .quad   0x0a5d94969cdd5fcd+        .quad   0xfce219072a7b31b4+        .quad   0x4d7adc75aa578016+        .quad   0x0ec276a687479324+        .quad   0x6d6d9d5d1fda4beb+        .quad   0x22b1a58ae9b08183+        .quad   0xfd95d071c15c388b+        .quad   0xa9812376850a0517+        .quad   0x33384cbabb7f335e++        // 2^220 * 3 * G++        .quad   0x3c6fa2680ca2c7b5+        .quad   0x1b5082046fb64fda+        .quad   0xeb53349c5431d6de+        .quad   0x5278b38f6b879c89+        .quad   0x33bc627a26218b8d+        .quad   0xea80b21fc7a80c61+        .quad   0x9458b12b173e9ee6+        .quad   0x076247be0e2f3059+        .quad   0x52e105f61416375a+        .quad   0xec97af3685abeba4+        .quad   0x26e6b50623a67c36+        .quad   0x5cf0e856f3d4fb01++        // 2^220 * 4 * G++        .quad   0xf6c968731ae8cab4+        .quad   0x5e20741ecb4f92c5+        .quad   0x2da53be58ccdbc3e+        .quad   0x2dddfea269970df7+        .quad   0xbeaece313db342a8+        .quad   0xcba3635b842db7ee+        .quad   0xe88c6620817f13ef+        .quad   0x1b9438aa4e76d5c6+        .quad   0x8a50777e166f031a+        .quad   0x067b39f10fb7a328+        .quad   0x1925c9a6010fbd76+        .quad   0x6df9b575cc740905++        // 2^220 * 5 * G++        .quad   0x42c1192927f6bdcf+        .quad   0x8f91917a403d61ca+        .quad   0xdc1c5a668b9e1f61+        .quad   0x1596047804ec0f8d+        .quad   0xecdfc35b48cade41+        .quad   0x6a88471fb2328270+        .quad   0x740a4a2440a01b6a+        .quad   0x471e5796003b5f29+        .quad   0xda96bbb3aced37ac+        .quad   0x7a2423b5e9208cea+        .quad   0x24cc5c3038aebae2+        .quad   0x50c356afdc5dae2f++        // 2^220 * 6 * G++        .quad   0x09dcbf4341c30318+        .quad   0xeeba061183181dce+        .quad   0xc179c0cedc1e29a1+        .quad   0x1dbf7b89073f35b0+        .quad   0xcfed9cdf1b31b964+        .quad   0xf486a9858ca51af3+        .quad   0x14897265ea8c1f84+        .quad   0x784a53dd932acc00+        .quad   0x2d99f9df14fc4920+        .quad   0x76ccb60cc4499fe5+        .quad   0xa4132cbbe5cf0003+        .quad   0x3f93d82354f000ea++        // 2^220 * 7 * G++        .quad   0x8183e7689e04ce85+        .quad   0x678fb71e04465341+        .quad   0xad92058f6688edac+        .quad   0x5da350d3532b099a+        .quad   0xeaac12d179e14978+        .quad   0xff923ff3bbebff5e+        .quad   0x4af663e40663ce27+        .quad   0x0fd381a811a5f5ff+        .quad   0xf256aceca436df54+        .quad   0x108b6168ae69d6e8+        .quad   0x20d986cb6b5d036c+        .quad   0x655957b9fee2af50++        // 2^220 * 8 * G++        .quad   0xaea8b07fa902030f+        .quad   0xf88c766af463d143+        .quad   0x15b083663c787a60+        .quad   0x08eab1148267a4a8+        .quad   0xbdc1409bd002d0ac+        .quad   0x66660245b5ccd9a6+        .quad   0x82317dc4fade85ec+        .quad   0x02fe934b6ad7df0d+        .quad   0xef5cf100cfb7ea74+        .quad   0x22897633a1cb42ac+        .quad   0xd4ce0c54cef285e2+        .quad   0x30408c048a146a55++        // 2^224 * 1 * G++        .quad   0x739d8845832fcedb+        .quad   0xfa38d6c9ae6bf863+        .quad   0x32bc0dcab74ffef7+        .quad   0x73937e8814bce45e+        .quad   0xbb2e00c9193b877f+        .quad   0xece3a890e0dc506b+        .quad   0xecf3b7c036de649f+        .quad   0x5f46040898de9e1a+        .quad   0xb9037116297bf48d+        .quad   0xa9d13b22d4f06834+        .quad   0xe19715574696bdc6+        .quad   0x2cf8a4e891d5e835++        // 2^224 * 2 * G++        .quad   0x6d93fd8707110f67+        .quad   0xdd4c09d37c38b549+        .quad   0x7cb16a4cc2736a86+        .quad   0x2049bd6e58252a09+        .quad   0x2cb5487e17d06ba2+        .quad   0x24d2381c3950196b+        .quad   0xd7659c8185978a30+        .quad   0x7a6f7f2891d6a4f6+        .quad   0x7d09fd8d6a9aef49+        .quad   0xf0ee60be5b3db90b+        .quad   0x4c21b52c519ebfd4+        .quad   0x6011aadfc545941d++        // 2^224 * 3 * G++        .quad   0x5f67926dcf95f83c+        .quad   0x7c7e856171289071+        .quad   0xd6a1e7f3998f7a5b+        .quad   0x6fc5cc1b0b62f9e0+        .quad   0x63ded0c802cbf890+        .quad   0xfbd098ca0dff6aaa+        .quad   0x624d0afdb9b6ed99+        .quad   0x69ce18b779340b1e+        .quad   0xd1ef5528b29879cb+        .quad   0xdd1aae3cd47e9092+        .quad   0x127e0442189f2352+        .quad   0x15596b3ae57101f1++        // 2^224 * 4 * G++        .quad   0x462739d23f9179a2+        .quad   0xff83123197d6ddcf+        .quad   0x1307deb553f2148a+        .quad   0x0d2237687b5f4dda+        .quad   0x09ff31167e5124ca+        .quad   0x0be4158bd9c745df+        .quad   0x292b7d227ef556e5+        .quad   0x3aa4e241afb6d138+        .quad   0x2cc138bf2a3305f5+        .quad   0x48583f8fa2e926c3+        .quad   0x083ab1a25549d2eb+        .quad   0x32fcaa6e4687a36c++        // 2^224 * 5 * G++        .quad   0x7bc56e8dc57d9af5+        .quad   0x3e0bd2ed9df0bdf2+        .quad   0xaac014de22efe4a3+        .quad   0x4627e9cefebd6a5c+        .quad   0x3207a4732787ccdf+        .quad   0x17e31908f213e3f8+        .quad   0xd5b2ecd7f60d964e+        .quad   0x746f6336c2600be9+        .quad   0x3f4af345ab6c971c+        .quad   0xe288eb729943731f+        .quad   0x33596a8a0344186d+        .quad   0x7b4917007ed66293++        // 2^224 * 6 * G++        .quad   0x2d85fb5cab84b064+        .quad   0x497810d289f3bc14+        .quad   0x476adc447b15ce0c+        .quad   0x122ba376f844fd7b+        .quad   0x54341b28dd53a2dd+        .quad   0xaa17905bdf42fc3f+        .quad   0x0ff592d94dd2f8f4+        .quad   0x1d03620fe08cd37d+        .quad   0xc20232cda2b4e554+        .quad   0x9ed0fd42115d187f+        .quad   0x2eabb4be7dd479d9+        .quad   0x02c70bf52b68ec4c++        // 2^224 * 7 * G++        .quad   0xa287ec4b5d0b2fbb+        .quad   0x415c5790074882ca+        .quad   0xe044a61ec1d0815c+        .quad   0x26334f0a409ef5e0+        .quad   0xace532bf458d72e1+        .quad   0x5be768e07cb73cb5+        .quad   0x56cf7d94ee8bbde7+        .quad   0x6b0697e3feb43a03+        .quad   0xb6c8f04adf62a3c0+        .quad   0x3ef000ef076da45d+        .quad   0x9c9cb95849f0d2a9+        .quad   0x1cc37f43441b2fae++        // 2^224 * 8 * G++        .quad   0x508f565a5cc7324f+        .quad   0xd061c4c0e506a922+        .quad   0xfb18abdb5c45ac19+        .quad   0x6c6809c10380314a+        .quad   0xd76656f1c9ceaeb9+        .quad   0x1c5b15f818e5656a+        .quad   0x26e72832844c2334+        .quad   0x3a346f772f196838+        .quad   0xd2d55112e2da6ac8+        .quad   0xe9bd0331b1e851ed+        .quad   0x960746dd8ec67262+        .quad   0x05911b9f6ef7c5d0++        // 2^228 * 1 * G++        .quad   0xe9dcd756b637ff2d+        .quad   0xec4c348fc987f0c4+        .quad   0xced59285f3fbc7b7+        .quad   0x3305354793e1ea87+        .quad   0x01c18980c5fe9f94+        .quad   0xcd656769716fd5c8+        .quad   0x816045c3d195a086+        .quad   0x6e2b7f3266cc7982+        .quad   0xcc802468f7c3568f+        .quad   0x9de9ba8219974cb3+        .quad   0xabb7229cb5b81360+        .quad   0x44e2017a6fbeba62++        // 2^228 * 2 * G++        .quad   0xc4c2a74354dab774+        .quad   0x8e5d4c3c4eaf031a+        .quad   0xb76c23d242838f17+        .quad   0x749a098f68dce4ea+        .quad   0x87f82cf3b6ca6ecd+        .quad   0x580f893e18f4a0c2+        .quad   0x058930072604e557+        .quad   0x6cab6ac256d19c1d+        .quad   0xdcdfe0a02cc1de60+        .quad   0x032665ff51c5575b+        .quad   0x2c0c32f1073abeeb+        .quad   0x6a882014cd7b8606++        // 2^228 * 3 * G++        .quad   0xa52a92fea4747fb5+        .quad   0xdc12a4491fa5ab89+        .quad   0xd82da94bb847a4ce+        .quad   0x4d77edce9512cc4e+        .quad   0xd111d17caf4feb6e+        .quad   0x050bba42b33aa4a3+        .quad   0x17514c3ceeb46c30+        .quad   0x54bedb8b1bc27d75+        .quad   0x77c8e14577e2189c+        .quad   0xa3e46f6aff99c445+        .quad   0x3144dfc86d335343+        .quad   0x3a96559e7c4216a9++        // 2^228 * 4 * G++        .quad   0x12550d37f42ad2ee+        .quad   0x8b78e00498a1fbf5+        .quad   0x5d53078233894cb2+        .quad   0x02c84e4e3e498d0c+        .quad   0x4493896880baaa52+        .quad   0x4c98afc4f285940e+        .quad   0xef4aa79ba45448b6+        .quad   0x5278c510a57aae7f+        .quad   0xa54dd074294c0b94+        .quad   0xf55d46b8df18ffb6+        .quad   0xf06fecc58dae8366+        .quad   0x588657668190d165++        // 2^228 * 5 * G++        .quad   0xd47712311aef7117+        .quad   0x50343101229e92c7+        .quad   0x7a95e1849d159b97+        .quad   0x2449959b8b5d29c9+        .quad   0xbf5834f03de25cc3+        .quad   0xb887c8aed6815496+        .quad   0x5105221a9481e892+        .quad   0x6760ed19f7723f93+        .quad   0x669ba3b7ac35e160+        .quad   0x2eccf73fba842056+        .quad   0x1aec1f17c0804f07+        .quad   0x0d96bc031856f4e7++        // 2^228 * 6 * G++        .quad   0x3318be7775c52d82+        .quad   0x4cb764b554d0aab9+        .quad   0xabcf3d27cc773d91+        .quad   0x3bf4d1848123288a+        .quad   0xb1d534b0cc7505e1+        .quad   0x32cd003416c35288+        .quad   0xcb36a5800762c29d+        .quad   0x5bfe69b9237a0bf8+        .quad   0x183eab7e78a151ab+        .quad   0xbbe990c999093763+        .quad   0xff717d6e4ac7e335+        .quad   0x4c5cddb325f39f88++        // 2^228 * 7 * G++        .quad   0xc0f6b74d6190a6eb+        .quad   0x20ea81a42db8f4e4+        .quad   0xa8bd6f7d97315760+        .quad   0x33b1d60262ac7c21+        .quad   0x57750967e7a9f902+        .quad   0x2c37fdfc4f5b467e+        .quad   0xb261663a3177ba46+        .quad   0x3a375e78dc2d532b+        .quad   0x8141e72f2d4dddea+        .quad   0xe6eafe9862c607c8+        .quad   0x23c28458573cafd0+        .quad   0x46b9476f4ff97346++        // 2^228 * 8 * G++        .quad   0x0c1ffea44f901e5c+        .quad   0x2b0b6fb72184b782+        .quad   0xe587ff910114db88+        .quad   0x37130f364785a142+        .quad   0x1215505c0d58359f+        .quad   0x2a2013c7fc28c46b+        .quad   0x24a0a1af89ea664e+        .quad   0x4400b638a1130e1f+        .quad   0x3a01b76496ed19c3+        .quad   0x31e00ab0ed327230+        .quad   0x520a885783ca15b1+        .quad   0x06aab9875accbec7++        // 2^232 * 1 * G++        .quad   0xc1339983f5df0ebb+        .quad   0xc0f3758f512c4cac+        .quad   0x2cf1130a0bb398e1+        .quad   0x6b3cecf9aa270c62+        .quad   0x5349acf3512eeaef+        .quad   0x20c141d31cc1cb49+        .quad   0x24180c07a99a688d+        .quad   0x555ef9d1c64b2d17+        .quad   0x36a770ba3b73bd08+        .quad   0x624aef08a3afbf0c+        .quad   0x5737ff98b40946f2+        .quad   0x675f4de13381749d++        // 2^232 * 2 * G++        .quad   0x0e2c52036b1782fc+        .quad   0x64816c816cad83b4+        .quad   0xd0dcbdd96964073e+        .quad   0x13d99df70164c520+        .quad   0xa12ff6d93bdab31d+        .quad   0x0725d80f9d652dfe+        .quad   0x019c4ff39abe9487+        .quad   0x60f450b882cd3c43+        .quad   0x014b5ec321e5c0ca+        .quad   0x4fcb69c9d719bfa2+        .quad   0x4e5f1c18750023a0+        .quad   0x1c06de9e55edac80++        // 2^232 * 3 * G++        .quad   0x990f7ad6a33ec4e2+        .quad   0x6608f938be2ee08e+        .quad   0x9ca143c563284515+        .quad   0x4cf38a1fec2db60d+        .quad   0xffd52b40ff6d69aa+        .quad   0x34530b18dc4049bb+        .quad   0x5e4a5c2fa34d9897+        .quad   0x78096f8e7d32ba2d+        .quad   0xa0aaaa650dfa5ce7+        .quad   0xf9c49e2a48b5478c+        .quad   0x4f09cc7d7003725b+        .quad   0x373cad3a26091abe++        // 2^232 * 4 * G++        .quad   0xb294634d82c9f57c+        .quad   0x1fcbfde124934536+        .quad   0x9e9c4db3418cdb5a+        .quad   0x0040f3d9454419fc+        .quad   0xf1bea8fb89ddbbad+        .quad   0x3bcb2cbc61aeaecb+        .quad   0x8f58a7bb1f9b8d9d+        .quad   0x21547eda5112a686+        .quad   0xdefde939fd5986d3+        .quad   0xf4272c89510a380c+        .quad   0xb72ba407bb3119b9+        .quad   0x63550a334a254df4++        // 2^232 * 5 * G++        .quad   0x6507d6edb569cf37+        .quad   0x178429b00ca52ee1+        .quad   0xea7c0090eb6bd65d+        .quad   0x3eea62c7daf78f51+        .quad   0x9bba584572547b49+        .quad   0xf305c6fae2c408e0+        .quad   0x60e8fa69c734f18d+        .quad   0x39a92bafaa7d767a+        .quad   0x9d24c713e693274e+        .quad   0x5f63857768dbd375+        .quad   0x70525560eb8ab39a+        .quad   0x68436a0665c9c4cd++        // 2^232 * 6 * G++        .quad   0xbc0235e8202f3f27+        .quad   0xc75c00e264f975b0+        .quad   0x91a4e9d5a38c2416+        .quad   0x17b6e7f68ab789f9+        .quad   0x1e56d317e820107c+        .quad   0xc5266844840ae965+        .quad   0xc1e0a1c6320ffc7a+        .quad   0x5373669c91611472+        .quad   0x5d2814ab9a0e5257+        .quad   0x908f2084c9cab3fc+        .quad   0xafcaf5885b2d1eca+        .quad   0x1cb4b5a678f87d11++        // 2^232 * 7 * G++        .quad   0xb664c06b394afc6c+        .quad   0x0c88de2498da5fb1+        .quad   0x4f8d03164bcad834+        .quad   0x330bca78de7434a2+        .quad   0x6b74aa62a2a007e7+        .quad   0xf311e0b0f071c7b1+        .quad   0x5707e438000be223+        .quad   0x2dc0fd2d82ef6eac+        .quad   0x982eff841119744e+        .quad   0xf9695e962b074724+        .quad   0xc58ac14fbfc953fb+        .quad   0x3c31be1b369f1cf5++        // 2^232 * 8 * G++        .quad   0xb0f4864d08948aee+        .quad   0x07dc19ee91ba1c6f+        .quad   0x7975cdaea6aca158+        .quad   0x330b61134262d4bb+        .quad   0xc168bc93f9cb4272+        .quad   0xaeb8711fc7cedb98+        .quad   0x7f0e52aa34ac8d7a+        .quad   0x41cec1097e7d55bb+        .quad   0xf79619d7a26d808a+        .quad   0xbb1fd49e1d9e156d+        .quad   0x73d7c36cdba1df27+        .quad   0x26b44cd91f28777d++        // 2^236 * 1 * G++        .quad   0x300a9035393aa6d8+        .quad   0x2b501131a12bb1cd+        .quad   0x7b1ff677f093c222+        .quad   0x4309c1f8cab82bad+        .quad   0xaf44842db0285f37+        .quad   0x8753189047efc8df+        .quad   0x9574e091f820979a+        .quad   0x0e378d6069615579+        .quad   0xd9fa917183075a55+        .quad   0x4bdb5ad26b009fdc+        .quad   0x7829ad2cd63def0e+        .quad   0x078fc54975fd3877++        // 2^236 * 2 * G++        .quad   0x87dfbd1428878f2d+        .quad   0x134636dd1e9421a1+        .quad   0x4f17c951257341a3+        .quad   0x5df98d4bad296cb8+        .quad   0xe2004b5bb833a98a+        .quad   0x44775dec2d4c3330+        .quad   0x3aa244067eace913+        .quad   0x272630e3d58e00a9+        .quad   0xf3678fd0ecc90b54+        .quad   0xf001459b12043599+        .quad   0x26725fbc3758b89b+        .quad   0x4325e4aa73a719ae++        // 2^236 * 3 * G++        .quad   0x657dc6ef433c3493+        .quad   0x65375e9f80dbf8c3+        .quad   0x47fd2d465b372dae+        .quad   0x4966ab79796e7947+        .quad   0xed24629acf69f59d+        .quad   0x2a4a1ccedd5abbf4+        .quad   0x3535ca1f56b2d67b+        .quad   0x5d8c68d043b1b42d+        .quad   0xee332d4de3b42b0a+        .quad   0xd84e5a2b16a4601c+        .quad   0x78243877078ba3e4+        .quad   0x77ed1eb4184ee437++        // 2^236 * 4 * G++        .quad   0xbfd4e13f201839a0+        .quad   0xaeefffe23e3df161+        .quad   0xb65b04f06b5d1fe3+        .quad   0x52e085fb2b62fbc0+        .quad   0x185d43f89e92ed1a+        .quad   0xb04a1eeafe4719c6+        .quad   0x499fbe88a6f03f4f+        .quad   0x5d8b0d2f3c859bdd+        .quad   0x124079eaa54cf2ba+        .quad   0xd72465eb001b26e7+        .quad   0x6843bcfdc97af7fd+        .quad   0x0524b42b55eacd02++        // 2^236 * 5 * G++        .quad   0xfd0d5dbee45447b0+        .quad   0x6cec351a092005ee+        .quad   0x99a47844567579cb+        .quad   0x59d242a216e7fa45+        .quad   0xbc18dcad9b829eac+        .quad   0x23ae7d28b5f579d0+        .quad   0xc346122a69384233+        .quad   0x1a6110b2e7d4ac89+        .quad   0x4f833f6ae66997ac+        .quad   0x6849762a361839a4+        .quad   0x6985dec1970ab525+        .quad   0x53045e89dcb1f546++        // 2^236 * 6 * G++        .quad   0xcb8bb346d75353db+        .quad   0xfcfcb24bae511e22+        .quad   0xcba48d40d50ae6ef+        .quad   0x26e3bae5f4f7cb5d+        .quad   0x84da3cde8d45fe12+        .quad   0xbd42c218e444e2d2+        .quad   0xa85196781f7e3598+        .quad   0x7642c93f5616e2b2+        .quad   0x2323daa74595f8e4+        .quad   0xde688c8b857abeb4+        .quad   0x3fc48e961c59326e+        .quad   0x0b2e73ca15c9b8ba++        // 2^236 * 7 * G++        .quad   0xd6bb4428c17f5026+        .quad   0x9eb27223fb5a9ca7+        .quad   0xe37ba5031919c644+        .quad   0x21ce380db59a6602+        .quad   0x0e3fbfaf79c03a55+        .quad   0x3077af054cbb5acf+        .quad   0xd5c55245db3de39f+        .quad   0x015e68c1476a4af7+        .quad   0xc1d5285220066a38+        .quad   0x95603e523570aef3+        .quad   0x832659a7226b8a4d+        .quad   0x5dd689091f8eedc9++        // 2^236 * 8 * G++        .quad   0xcbac84debfd3c856+        .quad   0x1624c348b35ff244+        .quad   0xb7f88dca5d9cad07+        .quad   0x3b0e574da2c2ebe8+        .quad   0x1d022591a5313084+        .quad   0xca2d4aaed6270872+        .quad   0x86a12b852f0bfd20+        .quad   0x56e6c439ad7da748+        .quad   0xc704ff4942bdbae6+        .quad   0x5e21ade2b2de1f79+        .quad   0xe95db3f35652fad8+        .quad   0x0822b5378f08ebc1++        // 2^240 * 1 * G++        .quad   0x51f048478f387475+        .quad   0xb25dbcf49cbecb3c+        .quad   0x9aab1244d99f2055+        .quad   0x2c709e6c1c10a5d6+        .quad   0xe1b7f29362730383+        .quad   0x4b5279ffebca8a2c+        .quad   0xdafc778abfd41314+        .quad   0x7deb10149c72610f+        .quad   0xcb62af6a8766ee7a+        .quad   0x66cbec045553cd0e+        .quad   0x588001380f0be4b5+        .quad   0x08e68e9ff62ce2ea++        // 2^240 * 2 * G++        .quad   0x34ad500a4bc130ad+        .quad   0x8d38db493d0bd49c+        .quad   0xa25c3d98500a89be+        .quad   0x2f1f3f87eeba3b09+        .quad   0x2f2d09d50ab8f2f9+        .quad   0xacb9218dc55923df+        .quad   0x4a8f342673766cb9+        .quad   0x4cb13bd738f719f5+        .quad   0xf7848c75e515b64a+        .quad   0xa59501badb4a9038+        .quad   0xc20d313f3f751b50+        .quad   0x19a1e353c0ae2ee8++        // 2^240 * 3 * G++        .quad   0x7d1c7560bafa05c3+        .quad   0xb3e1a0a0c6e55e61+        .quad   0xe3529718c0d66473+        .quad   0x41546b11c20c3486+        .quad   0xb42172cdd596bdbd+        .quad   0x93e0454398eefc40+        .quad   0x9fb15347b44109b5+        .quad   0x736bd3990266ae34+        .quad   0x85532d509334b3b4+        .quad   0x46fd114b60816573+        .quad   0xcc5f5f30425c8375+        .quad   0x412295a2b87fab5c++        // 2^240 * 4 * G++        .quad   0x19c99b88f57ed6e9+        .quad   0x5393cb266df8c825+        .quad   0x5cee3213b30ad273+        .quad   0x14e153ebb52d2e34+        .quad   0x2e655261e293eac6+        .quad   0x845a92032133acdb+        .quad   0x460975cb7900996b+        .quad   0x0760bb8d195add80+        .quad   0x413e1a17cde6818a+        .quad   0x57156da9ed69a084+        .quad   0x2cbf268f46caccb1+        .quad   0x6b34be9bc33ac5f2++        // 2^240 * 5 * G++        .quad   0xf3df2f643a78c0b2+        .quad   0x4c3e971ef22e027c+        .quad   0xec7d1c5e49c1b5a3+        .quad   0x2012c18f0922dd2d+        .quad   0x11fc69656571f2d3+        .quad   0xc6c9e845530e737a+        .quad   0xe33ae7a2d4fe5035+        .quad   0x01b9c7b62e6dd30b+        .quad   0x880b55e55ac89d29+        .quad   0x1483241f45a0a763+        .quad   0x3d36efdfc2e76c1f+        .quad   0x08af5b784e4bade8++        // 2^240 * 6 * G++        .quad   0x283499dc881f2533+        .quad   0x9d0525da779323b6+        .quad   0x897addfb673441f4+        .quad   0x32b79d71163a168d+        .quad   0xe27314d289cc2c4b+        .quad   0x4be4bd11a287178d+        .quad   0x18d528d6fa3364ce+        .quad   0x6423c1d5afd9826e+        .quad   0xcc85f8d9edfcb36a+        .quad   0x22bcc28f3746e5f9+        .quad   0xe49de338f9e5d3cd+        .quad   0x480a5efbc13e2dcc++        // 2^240 * 7 * G++        .quad   0x0b51e70b01622071+        .quad   0x06b505cf8b1dafc5+        .quad   0x2c6bb061ef5aabcd+        .quad   0x47aa27600cb7bf31+        .quad   0xb6614ce442ce221f+        .quad   0x6e199dcc4c053928+        .quad   0x663fb4a4dc1cbe03+        .quad   0x24b31d47691c8e06+        .quad   0x2a541eedc015f8c3+        .quad   0x11a4fe7e7c693f7c+        .quad   0xf0af66134ea278d6+        .quad   0x545b585d14dda094++        // 2^240 * 8 * G++        .quad   0x67bf275ea0d43a0f+        .quad   0xade68e34089beebe+        .quad   0x4289134cd479e72e+        .quad   0x0f62f9c332ba5454+        .quad   0x6204e4d0e3b321e1+        .quad   0x3baa637a28ff1e95+        .quad   0x0b0ccffd5b99bd9e+        .quad   0x4d22dc3e64c8d071+        .quad   0xfcb46589d63b5f39+        .quad   0x5cae6a3f57cbcf61+        .quad   0xfebac2d2953afa05+        .quad   0x1c0fa01a36371436++        // 2^244 * 1 * G++        .quad   0xe7547449bc7cd692+        .quad   0x0f9abeaae6f73ddf+        .quad   0x4af01ca700837e29+        .quad   0x63ab1b5d3f1bc183+        .quad   0xc11ee5e854c53fae+        .quad   0x6a0b06c12b4f3ff4+        .quad   0x33540f80e0b67a72+        .quad   0x15f18fc3cd07e3ef+        .quad   0x32750763b028f48c+        .quad   0x06020740556a065f+        .quad   0xd53bd812c3495b58+        .quad   0x08706c9b865f508d++        // 2^244 * 2 * G++        .quad   0xf37ca2ab3d343dff+        .quad   0x1a8c6a2d80abc617+        .quad   0x8e49e035d4ccffca+        .quad   0x48b46beebaa1d1b9+        .quad   0xcc991b4138b41246+        .quad   0x243b9c526f9ac26b+        .quad   0xb9ef494db7cbabbd+        .quad   0x5fba433dd082ed00+        .quad   0x9c49e355c9941ad0+        .quad   0xb9734ade74498f84+        .quad   0x41c3fed066663e5c+        .quad   0x0ecfedf8e8e710b3++        // 2^244 * 3 * G++        .quad   0x76430f9f9cd470d9+        .quad   0xb62acc9ba42f6008+        .quad   0x1898297c59adad5e+        .quad   0x7789dd2db78c5080+        .quad   0x744f7463e9403762+        .quad   0xf79a8dee8dfcc9c9+        .quad   0x163a649655e4cde3+        .quad   0x3b61788db284f435+        .quad   0xb22228190d6ef6b2+        .quad   0xa94a66b246ce4bfa+        .quad   0x46c1a77a4f0b6cc7+        .quad   0x4236ccffeb7338cf++        // 2^244 * 4 * G++        .quad   0x8497404d0d55e274+        .quad   0x6c6663d9c4ad2b53+        .quad   0xec2fb0d9ada95734+        .quad   0x2617e120cdb8f73c+        .quad   0x3bd82dbfda777df6+        .quad   0x71b177cc0b98369e+        .quad   0x1d0e8463850c3699+        .quad   0x5a71945b48e2d1f1+        .quad   0x6f203dd5405b4b42+        .quad   0x327ec60410b24509+        .quad   0x9c347230ac2a8846+        .quad   0x77de29fc11ffeb6a++        // 2^244 * 5 * G++        .quad   0xb0ac57c983b778a8+        .quad   0x53cdcca9d7fe912c+        .quad   0x61c2b854ff1f59dc+        .quad   0x3a1a2cf0f0de7dac+        .quad   0x835e138fecced2ca+        .quad   0x8c9eaf13ea963b9a+        .quad   0xc95fbfc0b2160ea6+        .quad   0x575e66f3ad877892+        .quad   0x99803a27c88fcb3a+        .quad   0x345a6789275ec0b0+        .quad   0x459789d0ff6c2be5+        .quad   0x62f882651e70a8b2++        // 2^244 * 6 * G++        .quad   0x085ae2c759ff1be4+        .quad   0x149145c93b0e40b7+        .quad   0xc467e7fa7ff27379+        .quad   0x4eeecf0ad5c73a95+        .quad   0x6d822986698a19e0+        .quad   0xdc9821e174d78a71+        .quad   0x41a85f31f6cb1f47+        .quad   0x352721c2bcda9c51+        .quad   0x48329952213fc985+        .quad   0x1087cf0d368a1746+        .quad   0x8e5261b166c15aa5+        .quad   0x2d5b2d842ed24c21++        // 2^244 * 7 * G++        .quad   0x02cfebd9ebd3ded1+        .quad   0xd45b217739021974+        .quad   0x7576f813fe30a1b7+        .quad   0x5691b6f9a34ef6c2+        .quad   0x5eb7d13d196ac533+        .quad   0x377234ecdb80be2b+        .quad   0xe144cffc7cf5ae24+        .quad   0x5226bcf9c441acec+        .quad   0x79ee6c7223e5b547+        .quad   0x6f5f50768330d679+        .quad   0xed73e1e96d8adce9+        .quad   0x27c3da1e1d8ccc03++        // 2^244 * 8 * G++        .quad   0x7eb9efb23fe24c74+        .quad   0x3e50f49f1651be01+        .quad   0x3ea732dc21858dea+        .quad   0x17377bd75bb810f9+        .quad   0x28302e71630ef9f6+        .quad   0xc2d4a2032b64cee0+        .quad   0x090820304b6292be+        .quad   0x5fca747aa82adf18+        .quad   0x232a03c35c258ea5+        .quad   0x86f23a2c6bcb0cf1+        .quad   0x3dad8d0d2e442166+        .quad   0x04a8933cab76862b++        // 2^248 * 1 * G++        .quad   0xd2c604b622943dff+        .quad   0xbc8cbece44cfb3a0+        .quad   0x5d254ff397808678+        .quad   0x0fa3614f3b1ca6bf+        .quad   0x69082b0e8c936a50+        .quad   0xf9c9a035c1dac5b6+        .quad   0x6fb73e54c4dfb634+        .quad   0x4005419b1d2bc140+        .quad   0xa003febdb9be82f0+        .quad   0x2089c1af3a44ac90+        .quad   0xf8499f911954fa8e+        .quad   0x1fba218aef40ab42++        // 2^248 * 2 * G++        .quad   0xab549448fac8f53e+        .quad   0x81f6e89a7ba63741+        .quad   0x74fd6c7d6c2b5e01+        .quad   0x392e3acaa8c86e42+        .quad   0x4f3e57043e7b0194+        .quad   0xa81d3eee08daaf7f+        .quad   0xc839c6ab99dcdef1+        .quad   0x6c535d13ff7761d5+        .quad   0x4cbd34e93e8a35af+        .quad   0x2e0781445887e816+        .quad   0x19319c76f29ab0ab+        .quad   0x25e17fe4d50ac13b++        // 2^248 * 3 * G++        .quad   0x0a289bd71e04f676+        .quad   0x208e1c52d6420f95+        .quad   0x5186d8b034691fab+        .quad   0x255751442a9fb351+        .quad   0x915f7ff576f121a7+        .quad   0xc34a32272fcd87e3+        .quad   0xccba2fde4d1be526+        .quad   0x6bba828f8969899b+        .quad   0xe2d1bc6690fe3901+        .quad   0x4cb54a18a0997ad5+        .quad   0x971d6914af8460d4+        .quad   0x559d504f7f6b7be4++        // 2^248 * 4 * G++        .quad   0xa7738378b3eb54d5+        .quad   0x1d69d366a5553c7c+        .quad   0x0a26cf62f92800ba+        .quad   0x01ab12d5807e3217+        .quad   0x9c4891e7f6d266fd+        .quad   0x0744a19b0307781b+        .quad   0x88388f1d6061e23b+        .quad   0x123ea6a3354bd50e+        .quad   0x118d189041e32d96+        .quad   0xb9ede3c2d8315848+        .quad   0x1eab4271d83245d9+        .quad   0x4a3961e2c918a154++        // 2^248 * 5 * G++        .quad   0x71dc3be0f8e6bba0+        .quad   0xd6cef8347effe30a+        .quad   0xa992425fe13a476a+        .quad   0x2cd6bce3fb1db763+        .quad   0x0327d644f3233f1e+        .quad   0x499a260e34fcf016+        .quad   0x83b5a716f2dab979+        .quad   0x68aceead9bd4111f+        .quad   0x38b4c90ef3d7c210+        .quad   0x308e6e24b7ad040c+        .quad   0x3860d9f1b7e73e23+        .quad   0x595760d5b508f597++        // 2^248 * 6 * G++        .quad   0x6129bfe104aa6397+        .quad   0x8f960008a4a7fccb+        .quad   0x3f8bc0897d909458+        .quad   0x709fa43edcb291a9+        .quad   0x882acbebfd022790+        .quad   0x89af3305c4115760+        .quad   0x65f492e37d3473f4+        .quad   0x2cb2c5df54515a2b+        .quad   0xeb0a5d8c63fd2aca+        .quad   0xd22bc1662e694eff+        .quad   0x2723f36ef8cbb03a+        .quad   0x70f029ecf0c8131f++        // 2^248 * 7 * G++        .quad   0x461307b32eed3e33+        .quad   0xae042f33a45581e7+        .quad   0xc94449d3195f0366+        .quad   0x0b7d5d8a6c314858+        .quad   0x2a6aafaa5e10b0b9+        .quad   0x78f0a370ef041aa9+        .quad   0x773efb77aa3ad61f+        .quad   0x44eca5a2a74bd9e1+        .quad   0x25d448327b95d543+        .quad   0x70d38300a3340f1d+        .quad   0xde1c531c60e1c52b+        .quad   0x272224512c7de9e4++        // 2^248 * 8 * G++        .quad   0x1abc92af49c5342e+        .quad   0xffeed811b2e6fad0+        .quad   0xefa28c8dfcc84e29+        .quad   0x11b5df18a44cc543+        .quad   0xbf7bbb8a42a975fc+        .quad   0x8c5c397796ada358+        .quad   0xe27fc76fcdedaa48+        .quad   0x19735fd7f6bc20a6+        .quad   0xe3ab90d042c84266+        .quad   0xeb848e0f7f19547e+        .quad   0x2503a1d065a497b9+        .quad   0x0fef911191df895f++        // 2^252 * 1 * G++        .quad   0xb1507ca1ab1c6eb9+        .quad   0xbd448f3e16b687b3+        .quad   0x3455fb7f2c7a91ab+        .quad   0x7579229e2f2adec1+        .quad   0x6ab5dcb85b1c16b7+        .quad   0x94c0fce83c7b27a5+        .quad   0xa4b11c1a735517be+        .quad   0x499238d0ba0eafaa+        .quad   0xecf46e527aba8b57+        .quad   0x15a08c478bd1647b+        .quad   0x7af1c6a65f706fef+        .quad   0x6345fa78f03a30d5++        // 2^252 * 2 * G++        .quad   0xdf02f95f1015e7a1+        .quad   0x790ec41da9b40263+        .quad   0x4d3a0ea133ea1107+        .quad   0x54f70be7e33af8c9+        .quad   0x93d3cbe9bdd8f0a4+        .quad   0xdb152c1bfd177302+        .quad   0x7dbddc6d7f17a875+        .quad   0x3e1a71cc8f426efe+        .quad   0xc83ca3e390babd62+        .quad   0x80ede3670291c833+        .quad   0xc88038ccd37900c4+        .quad   0x2c5fc0231ec31fa1++        // 2^252 * 3 * G++        .quad   0xfeba911717038b4f+        .quad   0xe5123721c9deef81+        .quad   0x1c97e4e75d0d8834+        .quad   0x68afae7a23dc3bc6+        .quad   0xc422e4d102456e65+        .quad   0x87414ac1cad47b91+        .quad   0x1592e2bba2b6ffdd+        .quad   0x75d9d2bff5c2100f+        .quad   0x5bd9b4763626e81c+        .quad   0x89966936bca02edd+        .quad   0x0a41193d61f077b3+        .quad   0x3097a24200ce5471++        // 2^252 * 4 * G++        .quad   0x57427734c7f8b84c+        .quad   0xf141a13e01b270e9+        .quad   0x02d1adfeb4e564a6+        .quad   0x4bb23d92ce83bd48+        .quad   0xa162e7246695c486+        .quad   0x131d633435a89607+        .quad   0x30521561a0d12a37+        .quad   0x56704bada6afb363+        .quad   0xaf6c4aa752f912b9+        .quad   0x5e665f6cd86770c8+        .quad   0x4c35ac83a3c8cd58+        .quad   0x2b7a29c010a58a7e++        // 2^252 * 5 * G++        .quad   0xc4007f77d0c1cec3+        .quad   0x8d1020b6bac492f8+        .quad   0x32ec29d57e69daaf+        .quad   0x599408759d95fce0+        .quad   0x33810a23bf00086e+        .quad   0xafce925ee736ff7c+        .quad   0x3d60e670e24922d4+        .quad   0x11ce9e714f96061b+        .quad   0x219ef713d815bac1+        .quad   0xf141465d485be25c+        .quad   0x6d5447cc4e513c51+        .quad   0x174926be5ef44393++        // 2^252 * 6 * G++        .quad   0xb5deb2f9fc5bd5bb+        .quad   0x92daa72ae1d810e1+        .quad   0xafc4cfdcb72a1c59+        .quad   0x497d78813fc22a24+        .quad   0x3ef5d41593ea022e+        .quad   0x5cbcc1a20ed0eed6+        .quad   0x8fd24ecf07382c8c+        .quad   0x6fa42ead06d8e1ad+        .quad   0xe276824a1f73371f+        .quad   0x7f7cf01c4f5b6736+        .quad   0x7e201fe304fa46e7+        .quad   0x785a36a357808c96++        // 2^252 * 7 * G++        .quad   0x825fbdfd63014d2b+        .quad   0xc852369c6ca7578b+        .quad   0x5b2fcd285c0b5df0+        .quad   0x12ab214c58048c8f+        .quad   0x070442985d517bc3+        .quad   0x6acd56c7ae653678+        .quad   0x00a27983985a7763+        .quad   0x5167effae512662b+        .quad   0xbd4ea9e10f53c4b6+        .quad   0x1673dc5f8ac91a14+        .quad   0xa8f81a4e2acc1aba+        .quad   0x33a92a7924332a25++        // 2^252 * 8 * G++        .quad   0x9dd1f49927996c02+        .quad   0x0cb3b058e04d1752+        .quad   0x1f7e88967fd02c3e+        .quad   0x2f964268cb8b3eb1+        .quad   0x7ba95ba0218f2ada+        .quad   0xcff42287330fb9ca+        .quad   0xdada496d56c6d907+        .quad   0x5380c296f4beee54+        .quad   0x9d4f270466898d0a+        .quad   0x3d0987990aff3f7a+        .quad   0xd09ef36267daba45+        .quad   0x7761455e7b1c669c
+ cbits/s2n/arm/curve25519_x25519base_alt.S view
@@ -0,0 +1,9434 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// The x25519 function for curve25519 on base element 9+// Input scalar[4]; output res[4]+//+// extern void curve25519_x25519base_alt+//   (uint64_t res[static 4],const uint64_t scalar[static 4]);+//+// Given a scalar n, returns the X coordinate of n * G where G = (9,...) is+// the standard generator. The scalar is first slightly modified/mangled+// as specified in the relevant RFC (https://www.rfc-editor.org/rfc/rfc7748).+//+// Standard ARM ABI: X0 = res, X1 = scalar+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(curve25519_x25519base_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(curve25519_x25519base_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(curve25519_x25519base_alt)++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Stable home for the input result argument during the whole body++#define res x23++// Other variables that are only needed prior to the modular inverse.++#define tab x19++#define i x20++#define bias x21++#define bf x22+#define ix x22++// Pointer-offset pairs for result and temporaries on stack with some aliasing.++#define resx res, #(0*NUMSIZE)++#define scalar sp, #(0*NUMSIZE)++#define tabent sp, #(1*NUMSIZE)+#define ymx_2 sp, #(1*NUMSIZE)+#define xpy_2 sp, #(2*NUMSIZE)+#define kxy_2 sp, #(3*NUMSIZE)++#define acc sp, #(4*NUMSIZE)+#define x_1 sp, #(4*NUMSIZE)+#define y_1 sp, #(5*NUMSIZE)+#define z_1 sp, #(6*NUMSIZE)+#define w_1 sp, #(7*NUMSIZE)+#define x_3 sp, #(4*NUMSIZE)+#define y_3 sp, #(5*NUMSIZE)+#define z_3 sp, #(6*NUMSIZE)+#define w_3 sp, #(7*NUMSIZE)++#define tmpspace sp, #(8*NUMSIZE)+#define t0 sp, #(8*NUMSIZE)+#define t1 sp, #(9*NUMSIZE)+#define t2 sp, #(10*NUMSIZE)+#define t3 sp, #(11*NUMSIZE)+#define t4 sp, #(12*NUMSIZE)+#define t5 sp, #(13*NUMSIZE)++// Total size to reserve on the stack++#define NSPACE 14*NUMSIZE++// Macro wrapping up the basic field operation bignum_mul_p25519_alt, only+// trivially different from a pure function call to that subroutine.++#define mul_p25519(P0,P1,P2)                    \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x7, x8, [P2] __LF                  \+        mul     x12, x3, x7 __LF                   \+        umulh   x13, x3, x7 __LF                   \+        mul     x11, x3, x8 __LF                   \+        umulh   x14, x3, x8 __LF                   \+        adds    x13, x13, x11 __LF                 \+        ldp     x9, x10, [P2+16] __LF              \+        mul     x11, x3, x9 __LF                   \+        umulh   x15, x3, x9 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x3, x10 __LF                  \+        umulh   x16, x3, x10 __LF                  \+        adcs    x15, x15, x11 __LF                 \+        adc     x16, x16, xzr __LF                 \+        ldp     x5, x6, [P1+16] __LF               \+        mul     x11, x4, x7 __LF                   \+        adds    x13, x13, x11 __LF                 \+        mul     x11, x4, x8 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x4, x9 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        mul     x11, x4, x10 __LF                  \+        adcs    x16, x16, x11 __LF                 \+        umulh   x3, x4, x10 __LF                   \+        adc     x3, x3, xzr __LF                   \+        umulh   x11, x4, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        umulh   x11, x4, x8 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        umulh   x11, x4, x9 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        adc     x3, x3, xzr __LF                   \+        mul     x11, x5, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        mul     x11, x5, x8 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        mul     x11, x5, x9 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        mul     x11, x5, x10 __LF                  \+        adcs    x3, x3, x11 __LF                   \+        umulh   x4, x5, x10 __LF                   \+        adc     x4, x4, xzr __LF                   \+        umulh   x11, x5, x7 __LF                   \+        adds    x15, x15, x11 __LF                 \+        umulh   x11, x5, x8 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        umulh   x11, x5, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        adc     x4, x4, xzr __LF                   \+        mul     x11, x6, x7 __LF                   \+        adds    x15, x15, x11 __LF                 \+        mul     x11, x6, x8 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        mul     x11, x6, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        mul     x11, x6, x10 __LF                  \+        adcs    x4, x4, x11 __LF                   \+        umulh   x5, x6, x10 __LF                   \+        adc     x5, x5, xzr __LF                   \+        umulh   x11, x6, x7 __LF                   \+        adds    x16, x16, x11 __LF                 \+        umulh   x11, x6, x8 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        umulh   x11, x6, x9 __LF                   \+        adcs    x4, x4, x11 __LF                   \+        adc     x5, x5, xzr __LF                   \+        mov     x7, #0x26 __LF                     \+        mul     x11, x7, x16 __LF                  \+        umulh   x9, x7, x16 __LF                   \+        adds    x12, x12, x11 __LF                 \+        mul     x11, x7, x3 __LF                   \+        umulh   x3, x7, x3 __LF                    \+        adcs    x13, x13, x11 __LF                 \+        mul     x11, x7, x4 __LF                   \+        umulh   x4, x7, x4 __LF                    \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x7, x5 __LF                   \+        umulh   x5, x7, x5 __LF                    \+        adcs    x15, x15, x11 __LF                 \+        cset    x16, cs __LF                       \+        adds    x15, x15, x4 __LF                  \+        adc     x16, x16, x5 __LF                  \+        cmn     x15, x15 __LF                      \+        orr     x15, x15, #0x8000000000000000 __LF \+        adc     x8, x16, x16 __LF                  \+        mov     x7, #0x13 __LF                     \+        madd    x11, x7, x8, x7 __LF               \+        adds    x12, x12, x11 __LF                 \+        adcs    x13, x13, x9 __LF                  \+        adcs    x14, x14, x3 __LF                  \+        adcs    x15, x15, xzr __LF                 \+        csel    x7, x7, xzr, cc __LF               \+        subs    x12, x12, x7 __LF                  \+        sbcs    x13, x13, xzr __LF                 \+        sbcs    x14, x14, xzr __LF                 \+        sbc     x15, x15, xzr __LF                 \+        and     x15, x15, #0x7fffffffffffffff __LF \+        stp     x12, x13, [P0] __LF                \+        stp     x14, x15, [P0+16]++// A version of multiplication that only guarantees output < 2 * p_25519.+// This basically skips the +1 and final correction in quotient estimation.++#define mul_4(P0,P1,P2)                         \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x7, x8, [P2] __LF                  \+        mul     x12, x3, x7 __LF                   \+        umulh   x13, x3, x7 __LF                   \+        mul     x11, x3, x8 __LF                   \+        umulh   x14, x3, x8 __LF                   \+        adds    x13, x13, x11 __LF                 \+        ldp     x9, x10, [P2+16] __LF              \+        mul     x11, x3, x9 __LF                   \+        umulh   x15, x3, x9 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x3, x10 __LF                  \+        umulh   x16, x3, x10 __LF                  \+        adcs    x15, x15, x11 __LF                 \+        adc     x16, x16, xzr __LF                 \+        ldp     x5, x6, [P1+16] __LF               \+        mul     x11, x4, x7 __LF                   \+        adds    x13, x13, x11 __LF                 \+        mul     x11, x4, x8 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x4, x9 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        mul     x11, x4, x10 __LF                  \+        adcs    x16, x16, x11 __LF                 \+        umulh   x3, x4, x10 __LF                   \+        adc     x3, x3, xzr __LF                   \+        umulh   x11, x4, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        umulh   x11, x4, x8 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        umulh   x11, x4, x9 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        adc     x3, x3, xzr __LF                   \+        mul     x11, x5, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        mul     x11, x5, x8 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        mul     x11, x5, x9 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        mul     x11, x5, x10 __LF                  \+        adcs    x3, x3, x11 __LF                   \+        umulh   x4, x5, x10 __LF                   \+        adc     x4, x4, xzr __LF                   \+        umulh   x11, x5, x7 __LF                   \+        adds    x15, x15, x11 __LF                 \+        umulh   x11, x5, x8 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        umulh   x11, x5, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        adc     x4, x4, xzr __LF                   \+        mul     x11, x6, x7 __LF                   \+        adds    x15, x15, x11 __LF                 \+        mul     x11, x6, x8 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        mul     x11, x6, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        mul     x11, x6, x10 __LF                  \+        adcs    x4, x4, x11 __LF                   \+        umulh   x5, x6, x10 __LF                   \+        adc     x5, x5, xzr __LF                   \+        umulh   x11, x6, x7 __LF                   \+        adds    x16, x16, x11 __LF                 \+        umulh   x11, x6, x8 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        umulh   x11, x6, x9 __LF                   \+        adcs    x4, x4, x11 __LF                   \+        adc     x5, x5, xzr __LF                   \+        mov     x7, #0x26 __LF                     \+        mul     x11, x7, x16 __LF                  \+        umulh   x9, x7, x16 __LF                   \+        adds    x12, x12, x11 __LF                 \+        mul     x11, x7, x3 __LF                   \+        umulh   x3, x7, x3 __LF                    \+        adcs    x13, x13, x11 __LF                 \+        mul     x11, x7, x4 __LF                   \+        umulh   x4, x7, x4 __LF                    \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x7, x5 __LF                   \+        umulh   x5, x7, x5 __LF                    \+        adcs    x15, x15, x11 __LF                 \+        cset    x16, cs __LF                       \+        adds    x15, x15, x4 __LF                  \+        adc     x16, x16, x5 __LF                  \+        cmn     x15, x15 __LF                      \+        bic     x15, x15, #0x8000000000000000 __LF \+        adc     x8, x16, x16 __LF                  \+        mov     x7, #0x13 __LF                     \+        mul     x11, x7, x8 __LF                   \+        adds    x12, x12, x11 __LF                 \+        adcs    x13, x13, x9 __LF                  \+        adcs    x14, x14, x3 __LF                  \+        adc     x15, x15, xzr __LF                 \+        stp     x12, x13, [P0] __LF                \+        stp     x14, x15, [P0+16]++// Modular subtraction with double modulus 2 * p_25519 = 2^256 - 38++#define sub_twice4(P0,P1,P2)                    \+        ldp     x5, x6, [P1] __LF                  \+        ldp     x4, x3, [P2] __LF                  \+        subs    x5, x5, x4 __LF                    \+        sbcs    x6, x6, x3 __LF                    \+        ldp     x7, x8, [P1+16] __LF               \+        ldp     x4, x3, [P2+16] __LF               \+        sbcs    x7, x7, x4 __LF                    \+        sbcs    x8, x8, x3 __LF                    \+        mov     x4, #38 __LF                       \+        csel    x3, x4, xzr, lo __LF               \+        subs    x5, x5, x3 __LF                    \+        sbcs    x6, x6, xzr __LF                   \+        sbcs    x7, x7, xzr __LF                   \+        sbc     x8, x8, xzr __LF                   \+        stp     x5, x6, [P0] __LF                  \+        stp     x7, x8, [P0+16]++// Modular addition and doubling with double modulus 2 * p_25519 = 2^256 - 38.+// This only ensures that the result fits in 4 digits, not that it is reduced+// even w.r.t. double modulus. The result is always correct modulo provided+// the sum of the inputs is < 2^256 + 2^256 - 38, so in particular provided+// at least one of them is reduced double modulo.++#define add_twice4(P0,P1,P2)                    \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x7, x8, [P2] __LF                  \+        adds    x3, x3, x7 __LF                    \+        adcs    x4, x4, x8 __LF                    \+        ldp     x5, x6, [P1+16] __LF               \+        ldp     x7, x8, [P2+16] __LF               \+        adcs    x5, x5, x7 __LF                    \+        adcs    x6, x6, x8 __LF                    \+        mov     x9, #38 __LF                       \+        csel    x9, x9, xzr, cs __LF               \+        adds    x3, x3, x9 __LF                    \+        adcs    x4, x4, xzr __LF                   \+        adcs    x5, x5, xzr __LF                   \+        adc     x6, x6, xzr __LF                   \+        stp     x3, x4, [P0] __LF                  \+        stp     x5, x6, [P0+16]++#define double_twice4(P0,P1)                    \+        ldp     x3, x4, [P1] __LF                  \+        adds    x3, x3, x3 __LF                    \+        adcs    x4, x4, x4 __LF                    \+        ldp     x5, x6, [P1+16] __LF               \+        adcs    x5, x5, x5 __LF                    \+        adcs    x6, x6, x6 __LF                    \+        mov     x9, #38 __LF                       \+        csel    x9, x9, xzr, cs __LF               \+        adds    x3, x3, x9 __LF                    \+        adcs    x4, x4, xzr __LF                   \+        adcs    x5, x5, xzr __LF                   \+        adc     x6, x6, xzr __LF                   \+        stp     x3, x4, [P0] __LF                  \+        stp     x5, x6, [P0+16]++S2N_BN_SYMBOL(curve25519_x25519base_alt):+        CFI_START++// Save regs and make room for temporaries++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_DEC_SP(NSPACE)++// Move the output pointer to a stable place++        mov     res, x0++// Copy the input scalar to its local variable while mangling it.+// In principle the mangling is into 01xxx...xxx000, but actually+// we only clear the top two bits so 00xxx...xxxxxx. The additional+// 2^254 * G is taken care of by the starting value for the addition+// chain below, while we never look at the three low bits at all.++        ldp     x10, x11, [x1]+        stp     x10, x11, [scalar]+        ldp     x12, x13, [x1, #16]+        bic     x13, x13, #0xc000000000000000+        stp     x12, x13, [scalar+16]++// The main part of the computation is on the edwards25519 curve in+// extended-projective coordinates (X,Y,Z,T), representing a point+// (x,y) via x = X/Z, y = Y/Z and x * y = T/Z (so X * Y = T * Z).+// Only at the very end do we translate back to curve25519. So G+// below means the generator within edwards25519 corresponding to+// (9,...) for curve25519, via the standard isomorphism.+//+// Initialize accumulator "acc" to either (2^254 + 8) * G or just 2^254 * G+// depending on bit 3 of the scalar, the only nonzero bit of the bottom 4.+// Thus, we have effectively dealt with bits 0, 1, 2, 3, 254 and 255.++        ldr     x0, [scalar]+        ands    xzr, x0, #8++#if defined(__ELF__)+        adrp    tab, S2N_BN_SYMBOL(curve25519_x25519base_alt_constant)+        add     tab, tab, :lo12:S2N_BN_SYMBOL(curve25519_x25519base_alt_constant)+#else+        adrp    tab, S2N_BN_SYMBOL(curve25519_x25519base_alt_constant)@PAGE+        add     tab, tab, S2N_BN_SYMBOL(curve25519_x25519base_alt_constant)@PAGEOFF+#endif++        ldp     x0, x1, [tab]+        ldp     x2, x3, [tab, #96]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc]++        ldp     x0, x1, [tab, #1*16]+        ldp     x2, x3, [tab, #96+1*16]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+1*16]++        ldp     x0, x1, [tab, #2*16]+        ldp     x2, x3, [tab, #96+2*16]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+2*16]++        ldp     x0, x1, [tab, #3*16]+        ldp     x2, x3, [tab, #96+3*16]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+3*16]++        mov     x0, #1+        stp     x0, xzr, [acc+4*16]+        stp     xzr, xzr, [acc+5*16]++        ldp     x0, x1, [tab, #4*16]+        ldp     x2, x3, [tab, #96+4*16]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+6*16]++        ldp     x0, x1, [tab, #5*16]+        ldp     x2, x3, [tab, #96+5*16]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+7*16]++// The counter "i" tracks the bit position for which the scalar has+// already been absorbed, starting at 4 and going up in chunks of 4.+//+// The pointer "tab" points at the current block of the table for+// multiples (2^i * j) * G at the current bit position i; 1 <= j <= 8.+//+// The bias is always either 0 and 1 and needs to be added to the+// partially processed scalar implicitly. This is used to absorb 4 bits+// of scalar per iteration from 3-bit table indexing by exploiting+// negation: (16 * h + l) * G = (16 * (h + 1) - (16 - l)) * G is used+// when l >= 9. Note that we can't have any bias left over at the+// end because of the clearing of bit 255 of the scalar, meaning the+// l >= 9 case cannot arise on the last iteration.++        mov     i, 4+        add     tab, tab, #192+        mov     bias, xzr++// Start of the main loop, repeated 63 times for i = 4, 8, ..., 252++Lcurve25519_x25519base_alt_scalarloop:++// Look at the next 4-bit field "bf", adding the previous bias as well.+// Choose the table index "ix" as bf when bf <= 8 and 16 - bf for bf >= 9,+// setting the bias to 1 for the next iteration in the latter case.++        lsr     x0, i, #6+        ldr     x2, [sp, x0, lsl #3]    // Exploiting scalar = sp exactly+        lsr     x2, x2, i+        and     x2, x2, #15+        add     bf, x2, bias++        cmp     bf, 9+        cset    bias, cs++        mov     x0, 16+        sub     x0, x0, bf+        cmp     bias, xzr+        csel    ix, x0, bf, ne++// Perform constant-time lookup in the table to get element number "ix".+// The table entry for the affine point (x,y) is actually a triple+// (y - x,x + y,2 * d * x * y) to precompute parts of the addition.+// Note that "ix" can be 0, so we set up the appropriate identity first.++        mov     x0, #1+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, #1+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        mov     x9, xzr+        mov     x10, xzr+        mov     x11, xzr++        cmp     ix, #1+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #2+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #3+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #4+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #5+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #6+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #7+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #8+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++// We now have the triple from the table in registers as follows+//+//      [x3;x2;x1;x0] = y - x+//      [x7;x6;x5;x4] = x + y+//      [x11;x10;x9;x8] = 2 * d * x * y+//+// In case bias = 1 we need to negate this. For Edwards curves+// -(x,y) = (-x,y), i.e. we need to negate the x coordinate.+// In this processed encoding, that amounts to swapping the+// first two fields and negating the third.+//+// The optional negation here also pretends bias = 0 whenever+// ix = 0 so that it doesn't need to handle the case of zero+// inputs, since no non-trivial table entries are zero. Note+// that in the zero case the whole negation is trivial, and+// so indeed is the swapping.++        cmp     bias, #0++        csel    x12, x0, x4, eq+        csel    x13, x1, x5, eq+        csel    x14, x2, x6, eq+        csel    x15, x3, x7, eq+        stp     x12, x13, [tabent]+        stp     x14, x15, [tabent+16]++        csel    x12, x0, x4, ne+        csel    x13, x1, x5, ne+        csel    x14, x2, x6, ne+        csel    x15, x3, x7, ne+        stp     x12, x13, [tabent+32]+        stp     x14, x15, [tabent+48]++        mov     x0, #-19+        subs    x0, x0, x8+        mov     x2, #-1+        sbcs    x1, x2, x9+        sbcs    x2, x2, x10+        mov     x3, #0x7FFFFFFFFFFFFFFF+        sbc     x3, x3, x11++        cmp     ix, xzr+        ccmp    bias, xzr, #4, ne++        csel    x0, x0, x8, ne+        csel    x1, x1, x9, ne+        stp     x0, x1, [tabent+64]+        csel    x2, x2, x10, ne+        csel    x3, x3, x11, ne+        stp     x2, x3, [tabent+80]++// Extended-projective and precomputed mixed addition.+// This is effectively the same as calling the standalone+// function edwards25519_pepadd(acc,acc,tabent), but we+// only retain slightly weaker normalization < 2 * p_25519+// throughout the inner loop, so the computation is+// slightly different, and faster overall.++        double_twice4(t0,z_1)+        sub_twice4(t1,y_1,x_1)+        add_twice4(t2,y_1,x_1)+        mul_4(t3,w_1,kxy_2)+        mul_4(t1,t1,ymx_2)+        mul_4(t2,t2,xpy_2)+        sub_twice4(t4,t0,t3)+        add_twice4(t0,t0,t3)+        sub_twice4(t5,t2,t1)+        add_twice4(t1,t2,t1)+        mul_4(z_3,t4,t0)+        mul_4(x_3,t5,t4)+        mul_4(y_3,t0,t1)+        mul_4(w_3,t5,t1)++// End of the main loop; move on by 4 bits.++        add     i, i, 4+        cmp     i, 256+        bcc     Lcurve25519_x25519base_alt_scalarloop++// Now we need to translate from Edwards curve edwards25519 back+// to the Montgomery form curve25519. The mapping in the affine+// representations is+//+// (x,y) |-> ((1 + y) / (1 - y), c * (1 + y) / ((1 - y) * x))+//+// For x25519, we only need the x coordinate, and we compute this as+//+// (1 + y) / (1 - y) = (x + x * y) / (x - x * y)+//                   = (X/Z + T/Z) / (X/Z - T/Z)+//                   = (X + T) / (X - T)+//                   = (X + T) * inverse(X - T)+//+// We could equally well use (Z + Y) / (Z - Y), but the above has the+// same cost, and it more explicitly forces zero output whenever X = 0,+// regardless of how the modular inverse behaves on zero inputs. In+// the present setting (base point 9, mangled scalar) that doesn't+// really matter anyway since X = 0 never arises, but it seems a+// little bit tidier. Note that both Edwards point (0,1) which maps to+// the Montgomery point at infinity, and Edwards (0,-1) which maps to+// Montgomery (0,0) [this is the 2-torsion point] are both by definition+// mapped to 0 by the X coordinate mapping used to define curve25519.+//+// First the addition and subtraction:++        add_twice4(t1,x_3,w_3)+        sub_twice4(t2,x_3,w_3)++// Prepare to call the modular inverse function to get t0 = 1/t2+// Note that this works for the weakly normalized z_3 equally well.+// The non-coprime case z_3 == 0 (mod p_25519) cannot arise anyway.++        add     x0, t0+        add     x1, t2++// Inline copy of bignum_inv_p25519, identical except for stripping out+// the prologue and epilogue saving and restoring registers and making+// and reclaiming room on the stack. For more details and explanations see+// "arm/curve25519/bignum_inv_p25519.S". Note that the stack it uses for+// its own temporaries is 128 bytes, so it has no effect on variables+// that are needed in the rest of our computation here: res, t0, t1, t2.++        mov     x20, x0+        mov     x10, #0xffffffffffffffed+        mov     x11, #0xffffffffffffffff+        stp     x10, x11, [sp]+        mov     x12, #0x7fffffffffffffff+        stp     x11, x12, [sp, #16]+        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]+        mov     x7, #0x13+        lsr     x6, x5, #63+        madd    x6, x7, x6, x7+        adds    x2, x2, x6+        adcs    x3, x3, xzr+        adcs    x4, x4, xzr+        orr     x5, x5, #0x8000000000000000+        adcs    x5, x5, xzr+        csel    x6, x7, xzr, cc+        subs    x2, x2, x6+        sbcs    x3, x3, xzr+        sbcs    x4, x4, xzr+        sbc     x5, x5, xzr+        and     x5, x5, #0x7fffffffffffffff+        stp     x2, x3, [sp, #32]+        stp     x4, x5, [sp, #48]+        stp     xzr, xzr, [sp, #64]+        stp     xzr, xzr, [sp, #80]+        mov     x10, #0x2099+        movk    x10, #0x7502, lsl #16+        movk    x10, #0x9e23, lsl #32+        movk    x10, #0xa0f9, lsl #48+        mov     x11, #0x2595+        movk    x11, #0x1d13, lsl #16+        movk    x11, #0x8f3f, lsl #32+        movk    x11, #0xa8c6, lsl #48+        mov     x12, #0x5242+        movk    x12, #0x5ac, lsl #16+        movk    x12, #0x8938, lsl #32+        movk    x12, #0x6c6c, lsl #48+        mov     x13, #0x615+        movk    x13, #0x4177, lsl #16+        movk    x13, #0x8b2, lsl #32+        movk    x13, #0x2765, lsl #48+        stp     x10, x11, [sp, #96]+        stp     x12, x13, [sp, #112]+        mov     x21, #0xa+        mov     x22, #0x1+        b       Lcurve25519_x25519base_alt_invmidloop+Lcurve25519_x25519base_alt_invloop:+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        and     x0, x12, x16+        and     x1, x13, x17+        add     x19, x0, x1+        ldr     x7, [sp]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #32]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x7, [sp, #8]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #40]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [sp]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [sp, #32]+        ldr     x7, [sp, #16]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #48]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [sp, #8]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [sp, #40]+        ldr     x7, [sp, #24]+        eor     x1, x7, x14+        asr     x3, x1, #63+        and     x3, x3, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #56]+        eor     x1, x8, x15+        asr     x0, x1, #63+        and     x0, x0, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x5, x6, #59+        str     x6, [sp, #16]+        extr    x5, x3, x5, #59+        str     x5, [sp, #24]+        eor     x1, x7, x16+        asr     x5, x1, #63+        and     x5, x5, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        asr     x0, x1, #63+        and     x0, x0, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x4, x2, x4, #59+        str     x4, [sp, #48]+        extr    x2, x5, x2, #59+        str     x2, [sp, #56]+        ldr     x7, [sp, #64]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #96]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #64]+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        str     x5, [sp, #96]+        adc     x3, x3, x1+        ldr     x7, [sp, #72]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #104]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #72]+        adc     x6, x6, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        str     x3, [sp, #104]+        adc     x4, x4, x1+        ldr     x7, [sp, #80]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #112]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #80]+        adc     x5, x5, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        str     x4, [sp, #112]+        adc     x2, x2, x1+        ldr     x7, [sp, #88]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #120]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x3, x5, #63+        ldp     x0, x1, [sp, #64]+        add     x6, x6, x3, asr #63+        mov     x3, #0x13+        mul     x4, x6, x3+        add     x5, x5, x6, lsl #63+        smulh   x3, x6, x3+        ldr     x6, [sp, #80]+        adds    x0, x0, x4+        adcs    x1, x1, x3+        asr     x3, x3, #63+        adcs    x6, x6, x3+        adc     x5, x5, x3+        stp     x0, x1, [sp, #64]+        stp     x6, x5, [sp, #80]+        eor     x1, x7, x16+        and     x5, x16, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        and     x0, x17, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x6, x5, x2, #63+        ldp     x0, x1, [sp, #96]+        add     x6, x6, x5, asr #63+        mov     x5, #0x13+        mul     x4, x6, x5+        add     x2, x2, x6, lsl #63+        smulh   x5, x6, x5+        ldr     x3, [sp, #112]+        adds    x0, x0, x4+        adcs    x1, x1, x5+        asr     x5, x5, #63+        adcs    x3, x3, x5+        adc     x2, x2, x5+        stp     x0, x1, [sp, #96]+        stp     x3, x2, [sp, #112]+Lcurve25519_x25519base_alt_invmidloop:+        mov     x1, x22+        ldr     x2, [sp]+        ldr     x3, [sp, #32]+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x8, x4, #0x100, lsl #12+        sbfx    x8, x8, #21, #21+        mov     x11, #0x100000+        add     x11, x11, x11, lsl #21+        add     x9, x4, x11+        asr     x9, x9, #42+        add     x10, x5, #0x100, lsl #12+        sbfx    x10, x10, #21, #21+        add     x11, x5, x11+        asr     x11, x11, #42+        mul     x6, x8, x2+        mul     x7, x9, x3+        mul     x2, x10, x2+        mul     x3, x11, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #21, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #42+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #21, #21+        add     x15, x5, x15+        asr     x15, x15, #42+        mul     x6, x12, x2+        mul     x7, x13, x3+        mul     x2, x14, x2+        mul     x3, x15, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        mul     x2, x12, x8+        mul     x3, x12, x9+        mul     x6, x14, x8+        mul     x7, x14, x9+        madd    x8, x13, x10, x2+        madd    x9, x13, x11, x3+        madd    x16, x15, x10, x6+        madd    x17, x15, x11, x7+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #22, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #43+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #22, #21+        add     x15, x5, x15+        asr     x15, x15, #43+        mneg    x2, x12, x8+        mneg    x3, x12, x9+        mneg    x4, x14, x8+        mneg    x5, x14, x9+        msub    x10, x13, x16, x2+        msub    x11, x13, x17, x3+        msub    x12, x15, x16, x4+        msub    x13, x15, x17, x5+        mov     x22, x1+        subs    x21, x21, #0x1+        b.ne    Lcurve25519_x25519base_alt_invloop+        ldr     x0, [sp]+        ldr     x1, [sp, #32]+        mul     x0, x0, x10+        madd    x1, x1, x11, x0+        asr     x0, x1, #63+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        eor     x14, x14, x0+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        eor     x15, x15, x0+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        eor     x16, x16, x0+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        eor     x17, x17, x0+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        ldr     x7, [sp, #64]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #96]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #64]+        adc     x2, x2, x1+        ldr     x7, [sp, #72]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #104]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #72]+        adc     x6, x6, x1+        ldr     x7, [sp, #80]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #112]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #80]+        adc     x5, x5, x1+        ldr     x7, [sp, #88]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #120]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x3, x5, #63+        ldp     x0, x1, [sp, #64]+        tst     x3, x3+        cinc    x6, x6, pl+        mov     x3, #0x13+        mul     x4, x6, x3+        add     x5, x5, x6, lsl #63+        smulh   x6, x6, x3+        ldr     x2, [sp, #80]+        adds    x0, x0, x4+        adcs    x1, x1, x6+        asr     x6, x6, #63+        adcs    x2, x2, x6+        adcs    x5, x5, x6+        csel    x3, x3, xzr, mi+        subs    x0, x0, x3+        sbcs    x1, x1, xzr+        sbcs    x2, x2, xzr+        sbc     x5, x5, xzr+        and     x5, x5, #0x7fffffffffffffff+        mov     x4, x20+        stp     x0, x1, [x4]+        stp     x2, x5, [x4, #16]++// The final result is (X + T) / (X - T)+// This is the only operation in the whole computation that+// fully reduces modulo p_25519 since now we want the canonical+// answer as output.++        mul_p25519(resx,t1,t0)++// Restore stack and registers++        CFI_INC_SP(NSPACE)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(curve25519_x25519base_alt)+++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif++// ****************************************************************************+// The precomputed data (all read-only).+// ****************************************************************************++#if defined(__ELF__)+.section .rodata+.type S2N_BN_SYMBOL(curve25519_x25519base_alt_constant), %object+.size S2N_BN_SYMBOL(curve25519_x25519base_alt_constant), 48576+#elif defined(__APPLE__)+.const_data+#endif++S2N_BN_SYMBOL(curve25519_x25519base_alt_constant):++// 2^254 * G and (2^254 + 8) * G in extended-projective coordinates+// but with Z = 1 assumed and hence left out, so they are (X,Y,T) only.++        .quad   0x251037f7cf4e861d+        .quad   0x10ede0fb19fb128f+        .quad   0x96c033b175f5e2c8+        .quad   0x055f070d6c15fb0d+        .quad   0x7c52af2c97473e69+        .quad   0x022f82391bad8378+        .quad   0x9991e1b02adb476f+        .quad   0x511144a03a99b855+        .quad   0x5fafc3b88ff2e4ae+        .quad   0x855e4ff0de1230ff+        .quad   0x72e302a348492870+        .quad   0x1253c19e53dbe1bc++        .quad   0x331d086e0d9abcaa+        .quad   0x1e23c96d311a10c9+        .quad   0x96d0f95e58c13478+        .quad   0x2f72f7384fcfcc59+        .quad   0x39a6cd1cfd7d87c9+        .quad   0x9867a0abd8ae153a+        .quad   0xa49d2a5f35986745+        .quad   0x57012940cdfe82e1+        .quad   0x5046a6532ec5544a+        .quad   0x6d674004739ff6c9+        .quad   0x9bbaa44b234a70e3+        .quad   0x5e6d8901138cf386++// Precomputed table of multiples of generator for edwards25519+// all in precomputed extended-projective (y-x,x+y,2*d*x*y) triples.++        // 2^4 * 1 * G++        .quad   0x7ec851ca553e2df3+        .quad   0xa71284cba64878b3+        .quad   0xe6b5e4193288d1e7+        .quad   0x4cf210ec5a9a8883+        .quad   0x322d04a52d9021f6+        .quad   0xb9c19f3375c6bf9c+        .quad   0x587a3a4342d20b09+        .quad   0x143b1cf8aa64fe61+        .quad   0x9f867c7d968acaab+        .quad   0x5f54258e27092729+        .quad   0xd0a7d34bea180975+        .quad   0x21b546a3374126e1++        // 2^4 * 2 * G++        .quad   0xa94ff858a2888343+        .quad   0xce0ed4565313ed3c+        .quad   0xf55c3dcfb5bf34fa+        .quad   0x0a653ca5c9eab371+        .quad   0x490a7a45d185218f+        .quad   0x9a15377846049335+        .quad   0x0060ea09cc31e1f6+        .quad   0x7e041577f86ee965+        .quad   0x66b2a496ce5b67f3+        .quad   0xff5492d8bd569796+        .quad   0x503cec294a592cd0+        .quad   0x566943650813acb2++        // 2^4 * 3 * G++        .quad   0xb818db0c26620798+        .quad   0x5d5c31d9606e354a+        .quad   0x0982fa4f00a8cdc7+        .quad   0x17e12bcd4653e2d4+        .quad   0x5672f9eb1dabb69d+        .quad   0xba70b535afe853fc+        .quad   0x47ac0f752796d66d+        .quad   0x32a5351794117275+        .quad   0xd3a644a6df648437+        .quad   0x703b6559880fbfdd+        .quad   0xcb852540ad3a1aa5+        .quad   0x0900b3f78e4c6468++        // 2^4 * 4 * G++        .quad   0x0a851b9f679d651b+        .quad   0xe108cb61033342f2+        .quad   0xd601f57fe88b30a3+        .quad   0x371f3acaed2dd714+        .quad   0xed280fbec816ad31+        .quad   0x52d9595bd8e6efe3+        .quad   0x0fe71772f6c623f5+        .quad   0x4314030b051e293c+        .quad   0xd560005efbf0bcad+        .quad   0x8eb70f2ed1870c5e+        .quad   0x201f9033d084e6a0+        .quad   0x4c3a5ae1ce7b6670++        // 2^4 * 5 * G++        .quad   0x4138a434dcb8fa95+        .quad   0x870cf67d6c96840b+        .quad   0xde388574297be82c+        .quad   0x7c814db27262a55a+        .quad   0xbaf875e4c93da0dd+        .quad   0xb93282a771b9294d+        .quad   0x80d63fb7f4c6c460+        .quad   0x6de9c73dea66c181+        .quad   0x478904d5a04df8f2+        .quad   0xfafbae4ab10142d3+        .quad   0xf6c8ac63555d0998+        .quad   0x5aac4a412f90b104++        // 2^4 * 6 * G++        .quad   0xc64f326b3ac92908+        .quad   0x5551b282e663e1e0+        .quad   0x476b35f54a1a4b83+        .quad   0x1b9da3fe189f68c2+        .quad   0x603a0d0abd7f5134+        .quad   0x8089c932e1d3ae46+        .quad   0xdf2591398798bd63+        .quad   0x1c145cd274ba0235+        .quad   0x32e8386475f3d743+        .quad   0x365b8baf6ae5d9ef+        .quad   0x825238b6385b681e+        .quad   0x234929c1167d65e1++        // 2^4 * 7 * G++        .quad   0x984decaba077ade8+        .quad   0x383f77ad19eb389d+        .quad   0xc7ec6b7e2954d794+        .quad   0x59c77b3aeb7c3a7a+        .quad   0x48145cc21d099fcf+        .quad   0x4535c192cc28d7e5+        .quad   0x80e7c1e548247e01+        .quad   0x4a5f28743b2973ee+        .quad   0xd3add725225ccf62+        .quad   0x911a3381b2152c5d+        .quad   0xd8b39fad5b08f87d+        .quad   0x6f05606b4799fe3b++        // 2^4 * 8 * G++        .quad   0x9ffe9e92177ba962+        .quad   0x98aee71d0de5cae1+        .quad   0x3ff4ae942d831044+        .quad   0x714de12e58533ac8+        .quad   0x5b433149f91b6483+        .quad   0xadb5dc655a2cbf62+        .quad   0x87fa8412632827b3+        .quad   0x60895e91ab49f8d8+        .quad   0xe9ecf2ed0cf86c18+        .quad   0xb46d06120735dfd4+        .quad   0xbc9da09804b96be7+        .quad   0x73e2e62fd96dc26b++        // 2^8 * 1 * G++        .quad   0xed5b635449aa515e+        .quad   0xa865c49f0bc6823a+        .quad   0x850c1fe95b42d1c4+        .quad   0x30d76d6f03d315b9+        .quad   0x2eccdd0e632f9c1d+        .quad   0x51d0b69676893115+        .quad   0x52dfb76ba8637a58+        .quad   0x6dd37d49a00eef39+        .quad   0x6c4444172106e4c7+        .quad   0xfb53d680928d7f69+        .quad   0xb4739ea4694d3f26+        .quad   0x10c697112e864bb0++        // 2^8 * 2 * G++        .quad   0x6493c4277dbe5fde+        .quad   0x265d4fad19ad7ea2+        .quad   0x0e00dfc846304590+        .quad   0x25e61cabed66fe09+        .quad   0x0ca62aa08358c805+        .quad   0x6a3d4ae37a204247+        .quad   0x7464d3a63b11eddc+        .quad   0x03bf9baf550806ef+        .quad   0x3f13e128cc586604+        .quad   0x6f5873ecb459747e+        .quad   0xa0b63dedcc1268f5+        .quad   0x566d78634586e22c++        // 2^8 * 3 * G++        .quad   0x1637a49f9cc10834+        .quad   0xbc8e56d5a89bc451+        .quad   0x1cb5ec0f7f7fd2db+        .quad   0x33975bca5ecc35d9+        .quad   0xa1054285c65a2fd0+        .quad   0x6c64112af31667c3+        .quad   0x680ae240731aee58+        .quad   0x14fba5f34793b22a+        .quad   0x3cd746166985f7d4+        .quad   0x593e5e84c9c80057+        .quad   0x2fc3f2b67b61131e+        .quad   0x14829cea83fc526c++        // 2^8 * 4 * G++        .quad   0xff437b8497dd95c2+        .quad   0x6c744e30aa4eb5a7+        .quad   0x9e0c5d613c85e88b+        .quad   0x2fd9c71e5f758173+        .quad   0x21e70b2f4e71ecb8+        .quad   0xe656ddb940a477e3+        .quad   0xbf6556cece1d4f80+        .quad   0x05fc3bc4535d7b7e+        .quad   0x24b8b3ae52afdedd+        .quad   0x3495638ced3b30cf+        .quad   0x33a4bc83a9be8195+        .quad   0x373767475c651f04++        // 2^8 * 5 * G++        .quad   0x2fba99fd40d1add9+        .quad   0xb307166f96f4d027+        .quad   0x4363f05215f03bae+        .quad   0x1fbea56c3b18f999+        .quad   0x634095cb14246590+        .quad   0xef12144016c15535+        .quad   0x9e38140c8910bc60+        .quad   0x6bf5905730907c8c+        .quad   0x0fa778f1e1415b8a+        .quad   0x06409ff7bac3a77e+        .quad   0x6f52d7b89aa29a50+        .quad   0x02521cf67a635a56++        // 2^8 * 6 * G++        .quad   0x513fee0b0a9d5294+        .quad   0x8f98e75c0fdf5a66+        .quad   0xd4618688bfe107ce+        .quad   0x3fa00a7e71382ced+        .quad   0xb1146720772f5ee4+        .quad   0xe8f894b196079ace+        .quad   0x4af8224d00ac824a+        .quad   0x001753d9f7cd6cc4+        .quad   0x3c69232d963ddb34+        .quad   0x1dde87dab4973858+        .quad   0xaad7d1f9a091f285+        .quad   0x12b5fe2fa048edb6++        // 2^8 * 7 * G++        .quad   0x71f0fbc496fce34d+        .quad   0x73b9826badf35bed+        .quad   0xd2047261ff28c561+        .quad   0x749b76f96fb1206f+        .quad   0xdf2b7c26ad6f1e92+        .quad   0x4b66d323504b8913+        .quad   0x8c409dc0751c8bc3+        .quad   0x6f7e93c20796c7b8+        .quad   0x1f5af604aea6ae05+        .quad   0xc12351f1bee49c99+        .quad   0x61a808b5eeff6b66+        .quad   0x0fcec10f01e02151++        // 2^8 * 8 * G++        .quad   0x644d58a649fe1e44+        .quad   0x21fcaea231ad777e+        .quad   0x02441c5a887fd0d2+        .quad   0x4901aa7183c511f3+        .quad   0x3df2d29dc4244e45+        .quad   0x2b020e7493d8de0a+        .quad   0x6cc8067e820c214d+        .quad   0x413779166feab90a+        .quad   0x08b1b7548c1af8f0+        .quad   0xce0f7a7c246299b4+        .quad   0xf760b0f91e06d939+        .quad   0x41bb887b726d1213++        // 2^12 * 1 * G++        .quad   0x9267806c567c49d8+        .quad   0x066d04ccca791e6a+        .quad   0xa69f5645e3cc394b+        .quad   0x5c95b686a0788cd2+        .quad   0x97d980e0aa39f7d2+        .quad   0x35d0384252c6b51c+        .quad   0x7d43f49307cd55aa+        .quad   0x56bd36cfb78ac362+        .quad   0x2ac519c10d14a954+        .quad   0xeaf474b494b5fa90+        .quad   0xe6af8382a9f87a5a+        .quad   0x0dea6db1879be094++        // 2^12 * 2 * G++        .quad   0xaa66bf547344e5ab+        .quad   0xda1258888f1b4309+        .quad   0x5e87d2b3fd564b2f+        .quad   0x5b2c78885483b1dd+        .quad   0x15baeb74d6a8797a+        .quad   0x7ef55cf1fac41732+        .quad   0x29001f5a3c8b05c5+        .quad   0x0ad7cc8752eaccfb+        .quad   0x52151362793408cf+        .quad   0xeb0f170319963d94+        .quad   0xa833b2fa883d9466+        .quad   0x093a7fa775003c78++        // 2^12 * 3 * G++        .quad   0xe5107de63a16d7be+        .quad   0xa377ffdc9af332cf+        .quad   0x70d5bf18440b677f+        .quad   0x6a252b19a4a31403+        .quad   0xb8e9604460a91286+        .quad   0x7f3fd8047778d3de+        .quad   0x67d01e31bf8a5e2d+        .quad   0x7b038a06c27b653e+        .quad   0x9ed919d5d36990f3+        .quad   0x5213aebbdb4eb9f2+        .quad   0xc708ea054cb99135+        .quad   0x58ded57f72260e56++        // 2^12 * 4 * G++        .quad   0x78e79dade9413d77+        .quad   0xf257f9d59729e67d+        .quad   0x59db910ee37aa7e6+        .quad   0x6aa11b5bbb9e039c+        .quad   0xda6d53265b0fd48b+        .quad   0x8960823193bfa988+        .quad   0xd78ac93261d57e28+        .quad   0x79f2942d3a5c8143+        .quad   0x97da2f25b6c88de9+        .quad   0x251ba7eaacf20169+        .quad   0x09b44f87ef4eb4e4+        .quad   0x7d90ab1bbc6a7da5++        // 2^12 * 5 * G++        .quad   0x9acca683a7016bfe+        .quad   0x90505f4df2c50b6d+        .quad   0x6b610d5fcce435aa+        .quad   0x19a10d446198ff96+        .quad   0x1a07a3f496b3c397+        .quad   0x11ceaa188f4e2532+        .quad   0x7d9498d5a7751bf0+        .quad   0x19ed161f508dd8a0+        .quad   0x560a2cd687dce6ca+        .quad   0x7f3568c48664cf4d+        .quad   0x8741e95222803a38+        .quad   0x483bdab1595653fc++        // 2^12 * 6 * G++        .quad   0xfa780f148734fa49+        .quad   0x106f0b70360534e0+        .quad   0x2210776fe3e307bd+        .quad   0x3286c109dde6a0fe+        .quad   0xd6cf4d0ab4da80f6+        .quad   0x82483e45f8307fe0+        .quad   0x05005269ae6f9da4+        .quad   0x1c7052909cf7877a+        .quad   0x32ee7de2874e98d4+        .quad   0x14c362e9b97e0c60+        .quad   0x5781dcde6a60a38a+        .quad   0x217dd5eaaa7aa840++        // 2^12 * 7 * G++        .quad   0x9db7c4d0248e1eb0+        .quad   0xe07697e14d74bf52+        .quad   0x1e6a9b173c562354+        .quad   0x7fa7c21f795a4965+        .quad   0x8bdf1fb9be8c0ec8+        .quad   0x00bae7f8e30a0282+        .quad   0x4963991dad6c4f6c+        .quad   0x07058a6e5df6f60a+        .quad   0xe9eb02c4db31f67f+        .quad   0xed25fd8910bcfb2b+        .quad   0x46c8131f5c5cddb4+        .quad   0x33b21c13a0cb9bce++        // 2^12 * 8 * G++        .quad   0x360692f8087d8e31+        .quad   0xf4dcc637d27163f7+        .quad   0x25a4e62065ea5963+        .quad   0x659bf72e5ac160d9+        .quad   0x9aafb9b05ee38c5b+        .quad   0xbf9d2d4e071a13c7+        .quad   0x8eee6e6de933290a+        .quad   0x1c3bab17ae109717+        .quad   0x1c9ab216c7cab7b0+        .quad   0x7d65d37407bbc3cc+        .quad   0x52744750504a58d5+        .quad   0x09f2606b131a2990++        // 2^16 * 1 * G++        .quad   0x40e87d44744346be+        .quad   0x1d48dad415b52b25+        .quad   0x7c3a8a18a13b603e+        .quad   0x4eb728c12fcdbdf7+        .quad   0x7e234c597c6691ae+        .quad   0x64889d3d0a85b4c8+        .quad   0xdae2c90c354afae7+        .quad   0x0a871e070c6a9e1d+        .quad   0x3301b5994bbc8989+        .quad   0x736bae3a5bdd4260+        .quad   0x0d61ade219d59e3c+        .quad   0x3ee7300f2685d464++        // 2^16 * 2 * G++        .quad   0xf5d255e49e7dd6b7+        .quad   0x8016115c610b1eac+        .quad   0x3c99975d92e187ca+        .quad   0x13815762979125c2+        .quad   0x43fa7947841e7518+        .quad   0xe5c6fa59639c46d7+        .quad   0xa1065e1de3052b74+        .quad   0x7d47c6a2cfb89030+        .quad   0x3fdad0148ef0d6e0+        .quad   0x9d3e749a91546f3c+        .quad   0x71ec621026bb8157+        .quad   0x148cf58d34c9ec80++        // 2^16 * 3 * G++        .quad   0x46a492f67934f027+        .quad   0x469984bef6840aa9+        .quad   0x5ca1bc2a89611854+        .quad   0x3ff2fa1ebd5dbbd4+        .quad   0xe2572f7d9ae4756d+        .quad   0x56c345bb88f3487f+        .quad   0x9fd10b6d6960a88d+        .quad   0x278febad4eaea1b9+        .quad   0xb1aa681f8c933966+        .quad   0x8c21949c20290c98+        .quad   0x39115291219d3c52+        .quad   0x4104dd02fe9c677b++        // 2^16 * 4 * G++        .quad   0x72b2bf5e1124422a+        .quad   0xa1fa0c3398a33ab5+        .quad   0x94cb6101fa52b666+        .quad   0x2c863b00afaf53d5+        .quad   0x81214e06db096ab8+        .quad   0x21a8b6c90ce44f35+        .quad   0x6524c12a409e2af5+        .quad   0x0165b5a48efca481+        .quad   0xf190a474a0846a76+        .quad   0x12eff984cd2f7cc0+        .quad   0x695e290658aa2b8f+        .quad   0x591b67d9bffec8b8++        // 2^16 * 5 * G++        .quad   0x312f0d1c80b49bfa+        .quad   0x5979515eabf3ec8a+        .quad   0x727033c09ef01c88+        .quad   0x3de02ec7ca8f7bcb+        .quad   0x99b9b3719f18b55d+        .quad   0xe465e5faa18c641e+        .quad   0x61081136c29f05ed+        .quad   0x489b4f867030128b+        .quad   0xd232102d3aeb92ef+        .quad   0xe16253b46116a861+        .quad   0x3d7eabe7190baa24+        .quad   0x49f5fbba496cbebf++        // 2^16 * 6 * G++        .quad   0x30949a108a5bcfd4+        .quad   0xdc40dd70bc6473eb+        .quad   0x92c294c1307c0d1c+        .quad   0x5604a86dcbfa6e74+        .quad   0x155d628c1e9c572e+        .quad   0x8a4d86acc5884741+        .quad   0x91a352f6515763eb+        .quad   0x06a1a6c28867515b+        .quad   0x7288d1d47c1764b6+        .quad   0x72541140e0418b51+        .quad   0x9f031a6018acf6d1+        .quad   0x20989e89fe2742c6++        // 2^16 * 7 * G++        .quad   0x499777fd3a2dcc7f+        .quad   0x32857c2ca54fd892+        .quad   0xa279d864d207e3a0+        .quad   0x0403ed1d0ca67e29+        .quad   0x1674278b85eaec2e+        .quad   0x5621dc077acb2bdf+        .quad   0x640a4c1661cbf45a+        .quad   0x730b9950f70595d3+        .quad   0xc94b2d35874ec552+        .quad   0xc5e6c8cf98246f8d+        .quad   0xf7cb46fa16c035ce+        .quad   0x5bd7454308303dcc++        // 2^16 * 8 * G++        .quad   0x7f9ad19528b24cc2+        .quad   0x7f6b54656335c181+        .quad   0x66b8b66e4fc07236+        .quad   0x133a78007380ad83+        .quad   0x85c4932115e7792a+        .quad   0xc64c89a2bdcdddc9+        .quad   0x9d1e3da8ada3d762+        .quad   0x5bb7db123067f82c+        .quad   0x0961f467c6ca62be+        .quad   0x04ec21d6211952ee+        .quad   0x182360779bd54770+        .quad   0x740dca6d58f0e0d2++        // 2^20 * 1 * G++        .quad   0x50b70bf5d3f0af0b+        .quad   0x4feaf48ae32e71f7+        .quad   0x60e84ed3a55bbd34+        .quad   0x00ed489b3f50d1ed+        .quad   0x3906c72aed261ae5+        .quad   0x9ab68fd988e100f7+        .quad   0xf5e9059af3360197+        .quad   0x0e53dc78bf2b6d47+        .quad   0xb90829bf7971877a+        .quad   0x5e4444636d17e631+        .quad   0x4d05c52e18276893+        .quad   0x27632d9a5a4a4af5++        // 2^20 * 2 * G++        .quad   0xd11ff05154b260ce+        .quad   0xd86dc38e72f95270+        .quad   0x601fcd0d267cc138+        .quad   0x2b67916429e90ccd+        .quad   0xa98285d187eaffdb+        .quad   0xa5b4fbbbd8d0a864+        .quad   0xb658f27f022663f7+        .quad   0x3bbc2b22d99ce282+        .quad   0xb917c952583c0a58+        .quad   0x653ff9b80fe4c6f3+        .quad   0x9b0da7d7bcdf3c0c+        .quad   0x43a0eeb6ab54d60e++        // 2^20 * 3 * G++        .quad   0x396966a46d4a5487+        .quad   0xf811a18aac2bb3ba+        .quad   0x66e4685b5628b26b+        .quad   0x70a477029d929b92+        .quad   0x3ac6322357875fe8+        .quad   0xd9d4f4ecf5fbcb8f+        .quad   0x8dee8493382bb620+        .quad   0x50c5eaa14c799fdc+        .quad   0xdd0edc8bd6f2fb3c+        .quad   0x54c63aa79cc7b7a0+        .quad   0xae0b032b2c8d9f1a+        .quad   0x6f9ce107602967fb++        // 2^20 * 4 * G++        .quad   0xad1054b1cde1c22a+        .quad   0xc4a8e90248eb32df+        .quad   0x5f3e7b33accdc0ea+        .quad   0x72364713fc79963e+        .quad   0x139693063520e0b5+        .quad   0x437fcf7c88ea03fe+        .quad   0xf7d4c40bd3c959bc+        .quad   0x699154d1f893ded9+        .quad   0x315d5c75b4b27526+        .quad   0xcccb842d0236daa5+        .quad   0x22f0c8a3345fee8e+        .quad   0x73975a617d39dbed++        // 2^20 * 5 * G++        .quad   0xe4024df96375da10+        .quad   0x78d3251a1830c870+        .quad   0x902b1948658cd91c+        .quad   0x7e18b10b29b7438a+        .quad   0x6f37f392f4433e46+        .quad   0x0e19b9a11f566b18+        .quad   0x220fb78a1fd1d662+        .quad   0x362a4258a381c94d+        .quad   0x9071d9132b6beb2f+        .quad   0x0f26e9ad28418247+        .quad   0xeab91ec9bdec925d+        .quad   0x4be65bc8f48af2de++        // 2^20 * 6 * G++        .quad   0x78487feba36e7028+        .quad   0x5f3f13001dd8ce34+        .quad   0x934fb12d4b30c489+        .quad   0x056c244d397f0a2b+        .quad   0x1d50fba257c26234+        .quad   0x7bd4823adeb0678b+        .quad   0xc2b0dc6ea6538af5+        .quad   0x5665eec6351da73e+        .quad   0xdb3ee00943bfb210+        .quad   0x4972018720800ac2+        .quad   0x26ab5d6173bd8667+        .quad   0x20b209c2ab204938++        // 2^20 * 7 * G++        .quad   0x549e342ac07fb34b+        .quad   0x02d8220821373d93+        .quad   0xbc262d70acd1f567+        .quad   0x7a92c9fdfbcac784+        .quad   0x1fcca94516bd3289+        .quad   0x448d65aa41420428+        .quad   0x59c3b7b216a55d62+        .quad   0x49992cc64e612cd8+        .quad   0x65bd1bea70f801de+        .quad   0x1befb7c0fe49e28a+        .quad   0xa86306cdb1b2ae4a+        .quad   0x3b7ac0cd265c2a09++        // 2^20 * 8 * G++        .quad   0x822bee438c01bcec+        .quad   0x530cb525c0fbc73b+        .quad   0x48519034c1953fe9+        .quad   0x265cc261e09a0f5b+        .quad   0xf0d54e4f22ed39a7+        .quad   0xa2aae91e5608150a+        .quad   0xf421b2e9eddae875+        .quad   0x31bc531d6b7de992+        .quad   0xdf3d134da980f971+        .quad   0x7a4fb8d1221a22a7+        .quad   0x3df7d42035aad6d8+        .quad   0x2a14edcc6a1a125e++        // 2^24 * 1 * G++        .quad   0xdf48ee0752cfce4e+        .quad   0xc3fffaf306ec08b7+        .quad   0x05710b2ab95459c4+        .quad   0x161d25fa963ea38d+        .quad   0x231a8c570478433c+        .quad   0xb7b5270ec281439d+        .quad   0xdbaa99eae3d9079f+        .quad   0x2c03f5256c2b03d9+        .quad   0x790f18757b53a47d+        .quad   0x307b0130cf0c5879+        .quad   0x31903d77257ef7f9+        .quad   0x699468bdbd96bbaf++        // 2^24 * 2 * G++        .quad   0xbd1f2f46f4dafecf+        .quad   0x7cef0114a47fd6f7+        .quad   0xd31ffdda4a47b37f+        .quad   0x525219a473905785+        .quad   0xd8dd3de66aa91948+        .quad   0x485064c22fc0d2cc+        .quad   0x9b48246634fdea2f+        .quad   0x293e1c4e6c4a2e3a+        .quad   0x376e134b925112e1+        .quad   0x703778b5dca15da0+        .quad   0xb04589af461c3111+        .quad   0x5b605c447f032823++        // 2^24 * 3 * G++        .quad   0xb965805920c47c89+        .quad   0xe7f0100c923b8fcc+        .quad   0x0001256502e2ef77+        .quad   0x24a76dcea8aeb3ee+        .quad   0x3be9fec6f0e7f04c+        .quad   0x866a579e75e34962+        .quad   0x5542ef161e1de61a+        .quad   0x2f12fef4cc5abdd5+        .quad   0x0a4522b2dfc0c740+        .quad   0x10d06e7f40c9a407+        .quad   0xc6cf144178cff668+        .quad   0x5e607b2518a43790++        // 2^24 * 4 * G++        .quad   0x58b31d8f6cdf1818+        .quad   0x35cfa74fc36258a2+        .quad   0xe1b3ff4f66e61d6e+        .quad   0x5067acab6ccdd5f7+        .quad   0xa02c431ca596cf14+        .quad   0xe3c42d40aed3e400+        .quad   0xd24526802e0f26db+        .quad   0x201f33139e457068+        .quad   0xfd527f6b08039d51+        .quad   0x18b14964017c0006+        .quad   0xd5220eb02e25a4a8+        .quad   0x397cba8862460375++        // 2^24 * 5 * G++        .quad   0x30c13093f05959b2+        .quad   0xe23aa18de9a97976+        .quad   0x222fd491721d5e26+        .quad   0x2339d320766e6c3a+        .quad   0x7815c3fbc81379e7+        .quad   0xa6619420dde12af1+        .quad   0xffa9c0f885a8fdd5+        .quad   0x771b4022c1e1c252+        .quad   0xd87dd986513a2fa7+        .quad   0xf5ac9b71f9d4cf08+        .quad   0xd06bc31b1ea283b3+        .quad   0x331a189219971a76++        // 2^24 * 6 * G++        .quad   0xf5166f45fb4f80c6+        .quad   0x9c36c7de61c775cf+        .quad   0xe3d4e81b9041d91c+        .quad   0x31167c6b83bdfe21+        .quad   0x26512f3a9d7572af+        .quad   0x5bcbe28868074a9e+        .quad   0x84edc1c11180f7c4+        .quad   0x1ac9619ff649a67b+        .quad   0xf22b3842524b1068+        .quad   0x5068343bee9ce987+        .quad   0xfc9d71844a6250c8+        .quad   0x612436341f08b111++        // 2^24 * 7 * G++        .quad   0xd99d41db874e898d+        .quad   0x09fea5f16c07dc20+        .quad   0x793d2c67d00f9bbc+        .quad   0x46ebe2309e5eff40+        .quad   0x8b6349e31a2d2638+        .quad   0x9ddfb7009bd3fd35+        .quad   0x7f8bf1b8a3a06ba4+        .quad   0x1522aa3178d90445+        .quad   0x2c382f5369614938+        .quad   0xdafe409ab72d6d10+        .quad   0xe8c83391b646f227+        .quad   0x45fe70f50524306c++        // 2^24 * 8 * G++        .quad   0xda4875a6960c0b8c+        .quad   0x5b68d076ef0e2f20+        .quad   0x07fb51cf3d0b8fd4+        .quad   0x428d1623a0e392d4+        .quad   0x62f24920c8951491+        .quad   0x05f007c83f630ca2+        .quad   0x6fbb45d2f5c9d4b8+        .quad   0x16619f6db57a2245+        .quad   0x084f4a4401a308fd+        .quad   0xa82219c376a5caac+        .quad   0xdeb8de4643d1bc7d+        .quad   0x1d81592d60bd38c6++        // 2^28 * 1 * G++        .quad   0xd833d7beec2a4c38+        .quad   0x2c9162830acc20ed+        .quad   0xe93a47aa92df7581+        .quad   0x702d67a3333c4a81+        .quad   0x3a4a369a2f89c8a1+        .quad   0x63137a1d7c8de80d+        .quad   0xbcac008a78eda015+        .quad   0x2cb8b3a5b483b03f+        .quad   0x36e417cbcb1b90a1+        .quad   0x33b3ddaa7f11794e+        .quad   0x3f510808885bc607+        .quad   0x24141dc0e6a8020d++        // 2^28 * 2 * G++        .quad   0x59f73c773fefee9d+        .quad   0xb3f1ef89c1cf989d+        .quad   0xe35dfb42e02e545f+        .quad   0x5766120b47a1b47c+        .quad   0x91925dccbd83157d+        .quad   0x3ca1205322cc8094+        .quad   0x28e57f183f90d6e4+        .quad   0x1a4714cede2e767b+        .quad   0xdb20ba0fb8b6b7ff+        .quad   0xb732c3b677511fa1+        .quad   0xa92b51c099f02d89+        .quad   0x4f3875ad489ca5f1++        // 2^28 * 3 * G++        .quad   0xc7fc762f4932ab22+        .quad   0x7ac0edf72f4c3c1b+        .quad   0x5f6b55aa9aa895e8+        .quad   0x3680274dad0a0081+        .quad   0x79ed13f6ee73eec0+        .quad   0xa5c6526d69110bb1+        .quad   0xe48928c38603860c+        .quad   0x722a1446fd7059f5+        .quad   0xd0959fe9a8cf8819+        .quad   0xd0a995508475a99c+        .quad   0x6eac173320b09cc5+        .quad   0x628ecf04331b1095++        // 2^28 * 4 * G++        .quad   0x98bcb118a9d0ddbc+        .quad   0xee449e3408b4802b+        .quad   0x87089226b8a6b104+        .quad   0x685f349a45c7915d+        .quad   0x9b41acf85c74ccf1+        .quad   0xb673318108265251+        .quad   0x99c92aed11adb147+        .quad   0x7a47d70d34ecb40f+        .quad   0x60a0c4cbcc43a4f5+        .quad   0x775c66ca3677bea9+        .quad   0xa17aa1752ff8f5ed+        .quad   0x11ded9020e01fdc0++        // 2^28 * 5 * G++        .quad   0x890e7809caefe704+        .quad   0x8728296de30e8c6c+        .quad   0x4c5cd2a392aeb1c9+        .quad   0x194263d15771531f+        .quad   0x471f95b03bea93b7+        .quad   0x0552d7d43313abd3+        .quad   0xbd9370e2e17e3f7b+        .quad   0x7b120f1db20e5bec+        .quad   0x17d2fb3d86502d7a+        .quad   0xb564d84450a69352+        .quad   0x7da962c8a60ed75d+        .quad   0x00d0f85b318736aa++        // 2^28 * 6 * G++        .quad   0x978b142e777c84fd+        .quad   0xf402644705a8c062+        .quad   0xa67ad51be7e612c7+        .quad   0x2f7b459698dd6a33+        .quad   0xa6753c1efd7621c1+        .quad   0x69c0b4a7445671f5+        .quad   0x971f527405b23c11+        .quad   0x387bc74851a8c7cd+        .quad   0x81894b4d4a52a9a8+        .quad   0xadd93e12f6b8832f+        .quad   0x184d8548b61bd638+        .quad   0x3f1c62dbd6c9f6cd++        // 2^28 * 7 * G++        .quad   0x2e8f1f0091910c1f+        .quad   0xa4df4fe0bff2e12c+        .quad   0x60c6560aee927438+        .quad   0x6338283facefc8fa+        .quad   0x3fad3e40148f693d+        .quad   0x052656e194eb9a72+        .quad   0x2f4dcbfd184f4e2f+        .quad   0x406f8db1c482e18b+        .quad   0x9e630d2c7f191ee4+        .quad   0x4fbf8301bc3ff670+        .quad   0x787d8e4e7afb73c4+        .quad   0x50d83d5be8f58fa5++        // 2^28 * 8 * G++        .quad   0x85683916c11a1897+        .quad   0x2d69a4efe506d008+        .quad   0x39af1378f664bd01+        .quad   0x65942131361517c6+        .quad   0xc0accf90b4d3b66d+        .quad   0xa7059de561732e60+        .quad   0x033d1f7870c6b0ba+        .quad   0x584161cd26d946e4+        .quad   0xbbf2b1a072d27ca2+        .quad   0xbf393c59fbdec704+        .quad   0xe98dbbcee262b81e+        .quad   0x02eebd0b3029b589++        // 2^32 * 1 * G++        .quad   0x61368756a60dac5f+        .quad   0x17e02f6aebabdc57+        .quad   0x7f193f2d4cce0f7d+        .quad   0x20234a7789ecdcf0+        .quad   0x8765b69f7b85c5e8+        .quad   0x6ff0678bd168bab2+        .quad   0x3a70e77c1d330f9b+        .quad   0x3a5f6d51b0af8e7c+        .quad   0x76d20db67178b252+        .quad   0x071c34f9d51ed160+        .quad   0xf62a4a20b3e41170+        .quad   0x7cd682353cffe366++        // 2^32 * 2 * G++        .quad   0x0be1a45bd887fab6+        .quad   0x2a846a32ba403b6e+        .quad   0xd9921012e96e6000+        .quad   0x2838c8863bdc0943+        .quad   0xa665cd6068acf4f3+        .quad   0x42d92d183cd7e3d3+        .quad   0x5759389d336025d9+        .quad   0x3ef0253b2b2cd8ff+        .quad   0xd16bb0cf4a465030+        .quad   0xfa496b4115c577ab+        .quad   0x82cfae8af4ab419d+        .quad   0x21dcb8a606a82812++        // 2^32 * 3 * G++        .quad   0x5c6004468c9d9fc8+        .quad   0x2540096ed42aa3cb+        .quad   0x125b4d4c12ee2f9c+        .quad   0x0bc3d08194a31dab+        .quad   0x9a8d00fabe7731ba+        .quad   0x8203607e629e1889+        .quad   0xb2cc023743f3d97f+        .quad   0x5d840dbf6c6f678b+        .quad   0x706e380d309fe18b+        .quad   0x6eb02da6b9e165c7+        .quad   0x57bbba997dae20ab+        .quad   0x3a4276232ac196dd++        // 2^32 * 4 * G++        .quad   0x4b42432c8a7084fa+        .quad   0x898a19e3dfb9e545+        .quad   0xbe9f00219c58e45d+        .quad   0x1ff177cea16debd1+        .quad   0x3bf8c172db447ecb+        .quad   0x5fcfc41fc6282dbd+        .quad   0x80acffc075aa15fe+        .quad   0x0770c9e824e1a9f9+        .quad   0xcf61d99a45b5b5fd+        .quad   0x860984e91b3a7924+        .quad   0xe7300919303e3e89+        .quad   0x39f264fd41500b1e++        // 2^32 * 5 * G++        .quad   0xa7ad3417dbe7e29c+        .quad   0xbd94376a2b9c139c+        .quad   0xa0e91b8e93597ba9+        .quad   0x1712d73468889840+        .quad   0xd19b4aabfe097be1+        .quad   0xa46dfce1dfe01929+        .quad   0xc3c908942ca6f1ff+        .quad   0x65c621272c35f14e+        .quad   0xe72b89f8ce3193dd+        .quad   0x4d103356a125c0bb+        .quad   0x0419a93d2e1cfe83+        .quad   0x22f9800ab19ce272++        // 2^32 * 6 * G++        .quad   0x605a368a3e9ef8cb+        .quad   0xe3e9c022a5504715+        .quad   0x553d48b05f24248f+        .quad   0x13f416cd647626e5+        .quad   0x42029fdd9a6efdac+        .quad   0xb912cebe34a54941+        .quad   0x640f64b987bdf37b+        .quad   0x4171a4d38598cab4+        .quad   0xfa2758aa99c94c8c+        .quad   0x23006f6fb000b807+        .quad   0xfbd291ddadda5392+        .quad   0x508214fa574bd1ab++        // 2^32 * 7 * G++        .quad   0xc20269153ed6fe4b+        .quad   0xa65a6739511d77c4+        .quad   0xcbde26462c14af94+        .quad   0x22f960ec6faba74b+        .quad   0x461a15bb53d003d6+        .quad   0xb2102888bcf3c965+        .quad   0x27c576756c683a5a+        .quad   0x3a7758a4c86cb447+        .quad   0x548111f693ae5076+        .quad   0x1dae21df1dfd54a6+        .quad   0x12248c90f3115e65+        .quad   0x5d9fd15f8de7f494++        // 2^32 * 8 * G++        .quad   0x031408d36d63727f+        .quad   0x6a379aefd7c7b533+        .quad   0xa9e18fc5ccaee24b+        .quad   0x332f35914f8fbed3+        .quad   0x3f244d2aeed7521e+        .quad   0x8e3a9028432e9615+        .quad   0xe164ba772e9c16d4+        .quad   0x3bc187fa47eb98d8+        .quad   0x6d470115ea86c20c+        .quad   0x998ab7cb6c46d125+        .quad   0xd77832b53a660188+        .quad   0x450d81ce906fba03++        // 2^36 * 1 * G++        .quad   0xf8ae4d2ad8453902+        .quad   0x7018058ee8db2d1d+        .quad   0xaab3995fc7d2c11e+        .quad   0x53b16d2324ccca79+        .quad   0x23264d66b2cae0b5+        .quad   0x7dbaed33ebca6576+        .quad   0x030ebed6f0d24ac8+        .quad   0x2a887f78f7635510+        .quad   0x2a23b9e75c012d4f+        .quad   0x0c974651cae1f2ea+        .quad   0x2fb63273675d70ca+        .quad   0x0ba7250b864403f5++        // 2^36 * 2 * G++        .quad   0xbb0d18fd029c6421+        .quad   0xbc2d142189298f02+        .quad   0x8347f8e68b250e96+        .quad   0x7b9f2fe8032d71c9+        .quad   0xdd63589386f86d9c+        .quad   0x61699176e13a85a4+        .quad   0x2e5111954eaa7d57+        .quad   0x32c21b57fb60bdfb+        .quad   0xd87823cd319e0780+        .quad   0xefc4cfc1897775c5+        .quad   0x4854fb129a0ab3f7+        .quad   0x12c49d417238c371++        // 2^36 * 3 * G++        .quad   0x0950b533ffe83769+        .quad   0x21861c1d8e1d6bd1+        .quad   0xf022d8381302e510+        .quad   0x2509200c6391cab4+        .quad   0x09b3a01783799542+        .quad   0x626dd08faad5ee3f+        .quad   0xba00bceeeb70149f+        .quad   0x1421b246a0a444c9+        .quad   0x4aa43a8e8c24a7c7+        .quad   0x04c1f540d8f05ef5+        .quad   0xadba5e0c0b3eb9dc+        .quad   0x2ab5504448a49ce3++        // 2^36 * 4 * G++        .quad   0x2ed227266f0f5dec+        .quad   0x9824ee415ed50824+        .quad   0x807bec7c9468d415+        .quad   0x7093bae1b521e23f+        .quad   0xdc07ac631c5d3afa+        .quad   0x58615171f9df8c6c+        .quad   0x72a079d89d73e2b0+        .quad   0x7301f4ceb4eae15d+        .quad   0x6409e759d6722c41+        .quad   0xa674e1cf72bf729b+        .quad   0xbc0a24eb3c21e569+        .quad   0x390167d24ebacb23++        // 2^36 * 5 * G++        .quad   0x27f58e3bba353f1c+        .quad   0x4c47764dbf6a4361+        .quad   0xafbbc4e56e562650+        .quad   0x07db2ee6aae1a45d+        .quad   0xd7bb054ba2f2120b+        .quad   0xe2b9ceaeb10589b7+        .quad   0x3fe8bac8f3c0edbe+        .quad   0x4cbd40767112cb69+        .quad   0x0b603cc029c58176+        .quad   0x5988e3825cb15d61+        .quad   0x2bb61413dcf0ad8d+        .quad   0x7b8eec6c74183287++        // 2^36 * 6 * G++        .quad   0xe4ca40782cd27cb0+        .quad   0xdaf9c323fbe967bd+        .quad   0xb29bd34a8ad41e9e+        .quad   0x72810497626ede4d+        .quad   0x32fee570fc386b73+        .quad   0xda8b0141da3a8cc7+        .quad   0x975ffd0ac8968359+        .quad   0x6ee809a1b132a855+        .quad   0x9444bb31fcfd863a+        .quad   0x2fe3690a3e4e48c5+        .quad   0xdc29c867d088fa25+        .quad   0x13bd1e38d173292e++        // 2^36 * 7 * G++        .quad   0xd32b4cd8696149b5+        .quad   0xe55937d781d8aab7+        .quad   0x0bcb2127ae122b94+        .quad   0x41e86fcfb14099b0+        .quad   0x223fb5cf1dfac521+        .quad   0x325c25316f554450+        .quad   0x030b98d7659177ac+        .quad   0x1ed018b64f88a4bd+        .quad   0x3630dfa1b802a6b0+        .quad   0x880f874742ad3bd5+        .quad   0x0af90d6ceec5a4d4+        .quad   0x746a247a37cdc5d9++        // 2^36 * 8 * G++        .quad   0xd531b8bd2b7b9af6+        .quad   0x5005093537fc5b51+        .quad   0x232fcf25c593546d+        .quad   0x20a365142bb40f49+        .quad   0x6eccd85278d941ed+        .quad   0x2254ae83d22f7843+        .quad   0xc522d02e7bbfcdb7+        .quad   0x681e3351bff0e4e2+        .quad   0x8b64b59d83034f45+        .quad   0x2f8b71f21fa20efb+        .quad   0x69249495ba6550e4+        .quad   0x539ef98e45d5472b++        // 2^40 * 1 * G++        .quad   0x6e7bb6a1a6205275+        .quad   0xaa4f21d7413c8e83+        .quad   0x6f56d155e88f5cb2+        .quad   0x2de25d4ba6345be1+        .quad   0xd074d8961cae743f+        .quad   0xf86d18f5ee1c63ed+        .quad   0x97bdc55be7f4ed29+        .quad   0x4cbad279663ab108+        .quad   0x80d19024a0d71fcd+        .quad   0xc525c20afb288af8+        .quad   0xb1a3974b5f3a6419+        .quad   0x7d7fbcefe2007233++        // 2^40 * 2 * G++        .quad   0xfaef1e6a266b2801+        .quad   0x866c68c4d5739f16+        .quad   0xf68a2fbc1b03762c+        .quad   0x5975435e87b75a8d+        .quad   0xcd7c5dc5f3c29094+        .quad   0xc781a29a2a9105ab+        .quad   0x80c61d36421c3058+        .quad   0x4f9cd196dcd8d4d7+        .quad   0x199297d86a7b3768+        .quad   0xd0d058241ad17a63+        .quad   0xba029cad5c1c0c17+        .quad   0x7ccdd084387a0307++        // 2^40 * 3 * G++        .quad   0xdca6422c6d260417+        .quad   0xae153d50948240bd+        .quad   0xa9c0c1b4fb68c677+        .quad   0x428bd0ed61d0cf53+        .quad   0x9b0c84186760cc93+        .quad   0xcdae007a1ab32a99+        .quad   0xa88dec86620bda18+        .quad   0x3593ca848190ca44+        .quad   0x9213189a5e849aa7+        .quad   0xd4d8c33565d8facd+        .quad   0x8c52545b53fdbbd1+        .quad   0x27398308da2d63e6++        // 2^40 * 4 * G++        .quad   0x42c38d28435ed413+        .quad   0xbd50f3603278ccc9+        .quad   0xbb07ab1a79da03ef+        .quad   0x269597aebe8c3355+        .quad   0xb9a10e4c0a702453+        .quad   0x0fa25866d57d1bde+        .quad   0xffb9d9b5cd27daf7+        .quad   0x572c2945492c33fd+        .quad   0xc77fc745d6cd30be+        .quad   0xe4dfe8d3e3baaefb+        .quad   0xa22c8830aa5dda0c+        .quad   0x7f985498c05bca80++        // 2^40 * 5 * G++        .quad   0x3849ce889f0be117+        .quad   0x8005ad1b7b54a288+        .quad   0x3da3c39f23fc921c+        .quad   0x76c2ec470a31f304+        .quad   0xd35615520fbf6363+        .quad   0x08045a45cf4dfba6+        .quad   0xeec24fbc873fa0c2+        .quad   0x30f2653cd69b12e7+        .quad   0x8a08c938aac10c85+        .quad   0x46179b60db276bcb+        .quad   0xa920c01e0e6fac70+        .quad   0x2f1273f1596473da++        // 2^40 * 6 * G++        .quad   0x4739fc7c8ae01e11+        .quad   0xfd5274904a6aab9f+        .quad   0x41d98a8287728f2e+        .quad   0x5d9e572ad85b69f2+        .quad   0x30488bd755a70bc0+        .quad   0x06d6b5a4f1d442e7+        .quad   0xead1a69ebc596162+        .quad   0x38ac1997edc5f784+        .quad   0x0666b517a751b13b+        .quad   0x747d06867e9b858c+        .quad   0xacacc011454dde49+        .quad   0x22dfcd9cbfe9e69c++        // 2^40 * 7 * G++        .quad   0x8ddbd2e0c30d0cd9+        .quad   0xad8e665facbb4333+        .quad   0x8f6b258c322a961f+        .quad   0x6b2916c05448c1c7+        .quad   0x56ec59b4103be0a1+        .quad   0x2ee3baecd259f969+        .quad   0x797cb29413f5cd32+        .quad   0x0fe9877824cde472+        .quad   0x7edb34d10aba913b+        .quad   0x4ea3cd822e6dac0e+        .quad   0x66083dff6578f815+        .quad   0x4c303f307ff00a17++        // 2^40 * 8 * G++        .quad   0xd30a3bd617b28c85+        .quad   0xc5d377b739773bea+        .quad   0xc6c6e78c1e6a5cbf+        .quad   0x0d61b8f78b2ab7c4+        .quad   0x29fc03580dd94500+        .quad   0xecd27aa46fbbec93+        .quad   0x130a155fc2e2a7f8+        .quad   0x416b151ab706a1d5+        .quad   0x56a8d7efe9c136b0+        .quad   0xbd07e5cd58e44b20+        .quad   0xafe62fda1b57e0ab+        .quad   0x191a2af74277e8d2++        // 2^44 * 1 * G++        .quad   0xd550095bab6f4985+        .quad   0x04f4cd5b4fbfaf1a+        .quad   0x9d8e2ed12a0c7540+        .quad   0x2bc24e04b2212286+        .quad   0x09d4b60b2fe09a14+        .quad   0xc384f0afdbb1747e+        .quad   0x58e2ea8978b5fd6e+        .quad   0x519ef577b5e09b0a+        .quad   0x1863d7d91124cca9+        .quad   0x7ac08145b88a708e+        .quad   0x2bcd7309857031f5+        .quad   0x62337a6e8ab8fae5++        // 2^44 * 2 * G++        .quad   0x4bcef17f06ffca16+        .quad   0xde06e1db692ae16a+        .quad   0x0753702d614f42b0+        .quad   0x5f6041b45b9212d0+        .quad   0xd1ab324e1b3a1273+        .quad   0x18947cf181055340+        .quad   0x3b5d9567a98c196e+        .quad   0x7fa00425802e1e68+        .quad   0x7d531574028c2705+        .quad   0x80317d69db0d75fe+        .quad   0x30fface8ef8c8ddd+        .quad   0x7e9de97bb6c3e998++        // 2^44 * 3 * G++        .quad   0x1558967b9e6585a3+        .quad   0x97c99ce098e98b92+        .quad   0x10af149b6eb3adad+        .quad   0x42181fe8f4d38cfa+        .quad   0xf004be62a24d40dd+        .quad   0xba0659910452d41f+        .quad   0x81c45ee162a44234+        .quad   0x4cb829d8a22266ef+        .quad   0x1dbcaa8407b86681+        .quad   0x081f001e8b26753b+        .quad   0x3cd7ce6a84048e81+        .quad   0x78af11633f25f22c++        // 2^44 * 4 * G++        .quad   0x8416ebd40b50babc+        .quad   0x1508722628208bee+        .quad   0xa3148fafb9c1c36d+        .quad   0x0d07daacd32d7d5d+        .quad   0x3241c00e7d65318c+        .quad   0xe6bee5dcd0e86de7+        .quad   0x118b2dc2fbc08c26+        .quad   0x680d04a7fc603dc3+        .quad   0xf9c2414a695aa3eb+        .quad   0xdaa42c4c05a68f21+        .quad   0x7c6c23987f93963e+        .quad   0x210e8cd30c3954e3++        // 2^44 * 5 * G++        .quad   0xac4201f210a71c06+        .quad   0x6a65e0aef3bfb021+        .quad   0xbc42c35c393632f7+        .quad   0x56ea8db1865f0742+        .quad   0x2b50f16137fe6c26+        .quad   0xe102bcd856e404d8+        .quad   0x12b0f1414c561f6b+        .quad   0x51b17bc8d028ec91+        .quad   0xfff5fb4bcf535119+        .quad   0xf4989d79df1108a0+        .quad   0xbdfcea659a3ba325+        .quad   0x18a11f1174d1a6f2++        // 2^44 * 6 * G++        .quad   0x407375ab3f6bba29+        .quad   0x9ec3b6d8991e482e+        .quad   0x99c80e82e55f92e9+        .quad   0x307c13b6fb0c0ae1+        .quad   0xfbd63cdad27a5f2c+        .quad   0xf00fc4bc8aa106d7+        .quad   0x53fb5c1a8e64a430+        .quad   0x04eaabe50c1a2e85+        .quad   0x24751021cb8ab5e7+        .quad   0xfc2344495c5010eb+        .quad   0x5f1e717b4e5610a1+        .quad   0x44da5f18c2710cd5++        // 2^44 * 7 * G++        .quad   0x033cc55ff1b82eb5+        .quad   0xb15ae36d411cae52+        .quad   0xba40b6198ffbacd3+        .quad   0x768edce1532e861f+        .quad   0x9156fe6b89d8eacc+        .quad   0xe6b79451e23126a1+        .quad   0xbd7463d93944eb4e+        .quad   0x726373f6767203ae+        .quad   0xe305ca72eb7ef68a+        .quad   0x662cf31f70eadb23+        .quad   0x18f026fdb4c45b68+        .quad   0x513b5384b5d2ecbd++        // 2^44 * 8 * G++        .quad   0x46d46280c729989e+        .quad   0x4b93fbd05368a5dd+        .quad   0x63df3f81d1765a89+        .quad   0x34cebd64b9a0a223+        .quad   0x5e2702878af34ceb+        .quad   0x900b0409b946d6ae+        .quad   0x6512ebf7dabd8512+        .quad   0x61d9b76988258f81+        .quad   0xa6c5a71349b7d94b+        .quad   0xa3f3d15823eb9446+        .quad   0x0416fbd277484834+        .quad   0x69d45e6f2c70812f++        // 2^48 * 1 * G++        .quad   0xce16f74bc53c1431+        .quad   0x2b9725ce2072edde+        .quad   0xb8b9c36fb5b23ee7+        .quad   0x7e2e0e450b5cc908+        .quad   0x9fe62b434f460efb+        .quad   0xded303d4a63607d6+        .quad   0xf052210eb7a0da24+        .quad   0x237e7dbe00545b93+        .quad   0x013575ed6701b430+        .quad   0x231094e69f0bfd10+        .quad   0x75320f1583e47f22+        .quad   0x71afa699b11155e3++        // 2^48 * 2 * G++        .quad   0x65ce6f9b3953b61d+        .quad   0xc65839eaafa141e6+        .quad   0x0f435ffda9f759fe+        .quad   0x021142e9c2b1c28e+        .quad   0xea423c1c473b50d6+        .quad   0x51e87a1f3b38ef10+        .quad   0x9b84bf5fb2c9be95+        .quad   0x00731fbc78f89a1c+        .quad   0xe430c71848f81880+        .quad   0xbf960c225ecec119+        .quad   0xb6dae0836bba15e3+        .quad   0x4c4d6f3347e15808++        // 2^48 * 3 * G++        .quad   0x18f7eccfc17d1fc9+        .quad   0x6c75f5a651403c14+        .quad   0xdbde712bf7ee0cdf+        .quad   0x193fddaaa7e47a22+        .quad   0x2f0cddfc988f1970+        .quad   0x6b916227b0b9f51b+        .quad   0x6ec7b6c4779176be+        .quad   0x38bf9500a88f9fa8+        .quad   0x1fd2c93c37e8876f+        .quad   0xa2f61e5a18d1462c+        .quad   0x5080f58239241276+        .quad   0x6a6fb99ebf0d4969++        // 2^48 * 4 * G++        .quad   0x6a46c1bb560855eb+        .quad   0x2416bb38f893f09d+        .quad   0xd71d11378f71acc1+        .quad   0x75f76914a31896ea+        .quad   0xeeb122b5b6e423c6+        .quad   0x939d7010f286ff8e+        .quad   0x90a92a831dcf5d8c+        .quad   0x136fda9f42c5eb10+        .quad   0xf94cdfb1a305bdd1+        .quad   0x0f364b9d9ff82c08+        .quad   0x2a87d8a5c3bb588a+        .quad   0x022183510be8dcba++        // 2^48 * 5 * G++        .quad   0x4af766385ead2d14+        .quad   0xa08ed880ca7c5830+        .quad   0x0d13a6e610211e3d+        .quad   0x6a071ce17b806c03+        .quad   0x9d5a710143307a7f+        .quad   0xb063de9ec47da45f+        .quad   0x22bbfe52be927ad3+        .quad   0x1387c441fd40426c+        .quad   0xb5d3c3d187978af8+        .quad   0x722b5a3d7f0e4413+        .quad   0x0d7b4848bb477ca0+        .quad   0x3171b26aaf1edc92++        // 2^48 * 6 * G++        .quad   0xa92f319097564ca8+        .quad   0xff7bb84c2275e119+        .quad   0x4f55fe37a4875150+        .quad   0x221fd4873cf0835a+        .quad   0xa60db7d8b28a47d1+        .quad   0xa6bf14d61770a4f1+        .quad   0xd4a1f89353ddbd58+        .quad   0x6c514a63344243e9+        .quad   0x2322204f3a156341+        .quad   0xfb73e0e9ba0a032d+        .quad   0xfce0dd4c410f030e+        .quad   0x48daa596fb924aaa++        // 2^48 * 7 * G++        .quad   0x6eca8e665ca59cc7+        .quad   0xa847254b2e38aca0+        .quad   0x31afc708d21e17ce+        .quad   0x676dd6fccad84af7+        .quad   0x14f61d5dc84c9793+        .quad   0x9941f9e3ef418206+        .quad   0xcdf5b88f346277ac+        .quad   0x58c837fa0e8a79a9+        .quad   0x0cf9688596fc9058+        .quad   0x1ddcbbf37b56a01b+        .quad   0xdcc2e77d4935d66a+        .quad   0x1c4f73f2c6a57f0a++        // 2^48 * 8 * G++        .quad   0x0e7a4fbd305fa0bb+        .quad   0x829d4ce054c663ad+        .quad   0xf421c3832fe33848+        .quad   0x795ac80d1bf64c42+        .quad   0xb36e706efc7c3484+        .quad   0x73dfc9b4c3c1cf61+        .quad   0xeb1d79c9781cc7e5+        .quad   0x70459adb7daf675c+        .quad   0x1b91db4991b42bb3+        .quad   0x572696234b02dcca+        .quad   0x9fdf9ee51f8c78dc+        .quad   0x5fe162848ce21fd3++        // 2^52 * 1 * G++        .quad   0xe2790aae4d077c41+        .quad   0x8b938270db7469a3+        .quad   0x6eb632dc8abd16a2+        .quad   0x720814ecaa064b72+        .quad   0x315c29c795115389+        .quad   0xd7e0e507862f74ce+        .quad   0x0c4a762185927432+        .quad   0x72de6c984a25a1e4+        .quad   0xae9ab553bf6aa310+        .quad   0x050a50a9806d6e1b+        .quad   0x92bb7403adff5139+        .quad   0x0394d27645be618b++        // 2^52 * 2 * G++        .quad   0x4d572251857eedf4+        .quad   0xe3724edde19e93c5+        .quad   0x8a71420e0b797035+        .quad   0x3b3c833687abe743+        .quad   0xf5396425b23545a4+        .quad   0x15a7a27e98fbb296+        .quad   0xab6c52bc636fdd86+        .quad   0x79d995a8419334ee+        .quad   0xcd8a8ea61195dd75+        .quad   0xa504d8a81dd9a82f+        .quad   0x540dca81a35879b6+        .quad   0x60dd16a379c86a8a++        // 2^52 * 3 * G++        .quad   0x35a2c8487381e559+        .quad   0x596ffea6d78082cb+        .quad   0xcb9771ebdba7b653+        .quad   0x5a08b5019b4da685+        .quad   0x3501d6f8153e47b8+        .quad   0xb7a9675414a2f60c+        .quad   0x112ee8b6455d9523+        .quad   0x4e62a3c18112ea8a+        .quad   0xc8d4ac04516ab786+        .quad   0x595af3215295b23d+        .quad   0xd6edd234db0230c1+        .quad   0x0929efe8825b41cc++        // 2^52 * 4 * G++        .quad   0x5f0601d1cbd0f2d3+        .quad   0x736e412f6132bb7f+        .quad   0x83604432238dde87+        .quad   0x1e3a5272f5c0753c+        .quad   0x8b3172b7ad56651d+        .quad   0x01581b7a3fabd717+        .quad   0x2dc94df6424df6e4+        .quad   0x30376e5d2c29284f+        .quad   0xd2918da78159a59c+        .quad   0x6bdc1cd93f0713f3+        .quad   0x565f7a934acd6590+        .quad   0x53daacec4cb4c128++        // 2^52 * 5 * G++        .quad   0x4ca73bd79cc8a7d6+        .quad   0x4d4a738f47e9a9b2+        .quad   0xf4cbf12942f5fe00+        .quad   0x01a13ff9bdbf0752+        .quad   0x99852bc3852cfdb0+        .quad   0x2cc12e9559d6ed0b+        .quad   0x70f9e2bf9b5ac27b+        .quad   0x4f3b8c117959ae99+        .quad   0x55b6c9c82ff26412+        .quad   0x1ac4a8c91fb667a8+        .quad   0xd527bfcfeb778bf2+        .quad   0x303337da7012a3be++        // 2^52 * 6 * G++        .quad   0x955422228c1c9d7c+        .quad   0x01fac1371a9b340f+        .quad   0x7e8d9177925b48d7+        .quad   0x53f8ad5661b3e31b+        .quad   0x976d3ccbfad2fdd1+        .quad   0xcb88839737a640a8+        .quad   0x2ff00c1d6734cb25+        .quad   0x269ff4dc789c2d2b+        .quad   0x0c003fbdc08d678d+        .quad   0x4d982fa37ead2b17+        .quad   0xc07e6bcdb2e582f1+        .quad   0x296c7291df412a44++        // 2^52 * 7 * G++        .quad   0x7903de2b33daf397+        .quad   0xd0ff0619c9a624b3+        .quad   0x8a1d252b555b3e18+        .quad   0x2b6d581c52e0b7c0+        .quad   0xdfb23205dab8b59e+        .quad   0x465aeaa0c8092250+        .quad   0xd133c1189a725d18+        .quad   0x2327370261f117d1+        .quad   0x3d0543d3623e7986+        .quad   0x679414c2c278a354+        .quad   0xae43f0cc726196f6+        .quad   0x7836c41f8245eaba++        // 2^52 * 8 * G++        .quad   0xe7a254db49e95a81+        .quad   0x5192d5d008b0ad73+        .quad   0x4d20e5b1d00afc07+        .quad   0x5d55f8012cf25f38+        .quad   0xca651e848011937c+        .quad   0xc6b0c46e6ef41a28+        .quad   0xb7021ba75f3f8d52+        .quad   0x119dff99ead7b9fd+        .quad   0x43eadfcbf4b31d4d+        .quad   0xc6503f7411148892+        .quad   0xfeee68c5060d3b17+        .quad   0x329293b3dd4a0ac8++        // 2^56 * 1 * G++        .quad   0x4e59214fe194961a+        .quad   0x49be7dc70d71cd4f+        .quad   0x9300cfd23b50f22d+        .quad   0x4789d446fc917232+        .quad   0x2879852d5d7cb208+        .quad   0xb8dedd70687df2e7+        .quad   0xdc0bffab21687891+        .quad   0x2b44c043677daa35+        .quad   0x1a1c87ab074eb78e+        .quad   0xfac6d18e99daf467+        .quad   0x3eacbbcd484f9067+        .quad   0x60c52eef2bb9a4e4++        // 2^56 * 2 * G++        .quad   0x0b5d89bc3bfd8bf1+        .quad   0xb06b9237c9f3551a+        .quad   0x0e4c16b0d53028f5+        .quad   0x10bc9c312ccfcaab+        .quad   0x702bc5c27cae6d11+        .quad   0x44c7699b54a48cab+        .quad   0xefbc4056ba492eb2+        .quad   0x70d77248d9b6676d+        .quad   0xaa8ae84b3ec2a05b+        .quad   0x98699ef4ed1781e0+        .quad   0x794513e4708e85d1+        .quad   0x63755bd3a976f413++        // 2^56 * 3 * G++        .quad   0xb55fa03e2ad10853+        .quad   0x356f75909ee63569+        .quad   0x9ff9f1fdbe69b890+        .quad   0x0d8cc1c48bc16f84+        .quad   0x3dc7101897f1acb7+        .quad   0x5dda7d5ec165bbd8+        .quad   0x508e5b9c0fa1020f+        .quad   0x2763751737c52a56+        .quad   0x029402d36eb419a9+        .quad   0xf0b44e7e77b460a5+        .quad   0xcfa86230d43c4956+        .quad   0x70c2dd8a7ad166e7++        // 2^56 * 4 * G++        .quad   0x656194509f6fec0e+        .quad   0xee2e7ea946c6518d+        .quad   0x9733c1f367e09b5c+        .quad   0x2e0fac6363948495+        .quad   0x91d4967db8ed7e13+        .quad   0x74252f0ad776817a+        .quad   0xe40982e00d852564+        .quad   0x32b8613816a53ce5+        .quad   0x79e7f7bee448cd64+        .quad   0x6ac83a67087886d0+        .quad   0xf89fd4d9a0e4db2e+        .quad   0x4179215c735a4f41++        // 2^56 * 5 * G++        .quad   0x8c7094e7d7dced2a+        .quad   0x97fb8ac347d39c70+        .quad   0xe13be033a906d902+        .quad   0x700344a30cd99d76+        .quad   0xe4ae33b9286bcd34+        .quad   0xb7ef7eb6559dd6dc+        .quad   0x278b141fb3d38e1f+        .quad   0x31fa85662241c286+        .quad   0xaf826c422e3622f4+        .quad   0xc12029879833502d+        .quad   0x9bc1b7e12b389123+        .quad   0x24bb2312a9952489++        // 2^56 * 6 * G++        .quad   0xb1a8ed1732de67c3+        .quad   0x3cb49418461b4948+        .quad   0x8ebd434376cfbcd2+        .quad   0x0fee3e871e188008+        .quad   0x41f80c2af5f85c6b+        .quad   0x687284c304fa6794+        .quad   0x8945df99a3ba1bad+        .quad   0x0d1d2af9ffeb5d16+        .quad   0xa9da8aa132621edf+        .quad   0x30b822a159226579+        .quad   0x4004197ba79ac193+        .quad   0x16acd79718531d76++        // 2^56 * 7 * G++        .quad   0x72df72af2d9b1d3d+        .quad   0x63462a36a432245a+        .quad   0x3ecea07916b39637+        .quad   0x123e0ef6b9302309+        .quad   0xc959c6c57887b6ad+        .quad   0x94e19ead5f90feba+        .quad   0x16e24e62a342f504+        .quad   0x164ed34b18161700+        .quad   0x487ed94c192fe69a+        .quad   0x61ae2cea3a911513+        .quad   0x877bf6d3b9a4de27+        .quad   0x78da0fc61073f3eb++        // 2^56 * 8 * G++        .quad   0x5bf15d28e52bc66a+        .quad   0x2c47e31870f01a8e+        .quad   0x2419afbc06c28bdd+        .quad   0x2d25deeb256b173a+        .quad   0xa29f80f1680c3a94+        .quad   0x71f77e151ae9e7e6+        .quad   0x1100f15848017973+        .quad   0x054aa4b316b38ddd+        .quad   0xdfc8468d19267cb8+        .quad   0x0b28789c66e54daf+        .quad   0x2aeb1d2a666eec17+        .quad   0x134610a6ab7da760++        // 2^60 * 1 * G++        .quad   0xcaf55ec27c59b23f+        .quad   0x99aeed3e154d04f2+        .quad   0x68441d72e14141f4+        .quad   0x140345133932a0a2+        .quad   0xd91430e0dc028c3c+        .quad   0x0eb955a85217c771+        .quad   0x4b09e1ed2c99a1fa+        .quad   0x42881af2bd6a743c+        .quad   0x7bfec69aab5cad3d+        .quad   0xc23e8cd34cb2cfad+        .quad   0x685dd14bfb37d6a2+        .quad   0x0ad6d64415677a18++        // 2^60 * 2 * G++        .quad   0x781a439e417becb5+        .quad   0x4ac5938cd10e0266+        .quad   0x5da385110692ac24+        .quad   0x11b065a2ade31233+        .quad   0x7914892847927e9f+        .quad   0x33dad6ef370aa877+        .quad   0x1f8f24fa11122703+        .quad   0x5265ac2f2adf9592+        .quad   0x405fdd309afcb346+        .quad   0xd9723d4428e63f54+        .quad   0x94c01df05f65aaae+        .quad   0x43e4dc3ae14c0809++        // 2^60 * 3 * G++        .quad   0xbc12c7f1a938a517+        .quad   0x473028ab3180b2e1+        .quad   0x3f78571efbcd254a+        .quad   0x74e534426ff6f90f+        .quad   0xea6f7ac3adc2c6a3+        .quad   0xd0e928f6e9717c94+        .quad   0xe2d379ead645eaf5+        .quad   0x46dd8785c51ffbbe+        .quad   0x709801be375c8898+        .quad   0x4b06dab5e3fd8348+        .quad   0x75880ced27230714+        .quad   0x2b09468fdd2f4c42++        // 2^60 * 4 * G++        .quad   0x97c749eeb701cb96+        .quad   0x83f438d4b6a369c3+        .quad   0x62962b8b9a402cd9+        .quad   0x6976c7509888df7b+        .quad   0x5b97946582ffa02a+        .quad   0xda096a51fea8f549+        .quad   0xa06351375f77af9b+        .quad   0x1bcfde61201d1e76+        .quad   0x4a4a5490246a59a2+        .quad   0xd63ebddee87fdd90+        .quad   0xd9437c670d2371fa+        .quad   0x69e87308d30f8ed6++        // 2^60 * 5 * G++        .quad   0x435a8bb15656beb0+        .quad   0xf8fac9ba4f4d5bca+        .quad   0xb9b278c41548c075+        .quad   0x3eb0ef76e892b622+        .quad   0x0f80bf028bc80303+        .quad   0x6aae16b37a18cefb+        .quad   0xdd47ea47d72cd6a3+        .quad   0x61943588f4ed39aa+        .quad   0xd26e5c3e91039f85+        .quad   0xc0e9e77df6f33aa9+        .quad   0xe8968c5570066a93+        .quad   0x3c34d1881faaaddd++        // 2^60 * 6 * G++        .quad   0x3f9d2b5ea09f9ec0+        .quad   0x1dab3b6fb623a890+        .quad   0xa09ba3ea72d926c4+        .quad   0x374193513fd8b36d+        .quad   0xbd5b0b8f2fffe0d9+        .quad   0x6aa254103ed24fb9+        .quad   0x2ac7d7bcb26821c4+        .quad   0x605b394b60dca36a+        .quad   0xb4e856e45a9d1ed2+        .quad   0xefe848766c97a9a2+        .quad   0xb104cf641e5eee7d+        .quad   0x2f50b81c88a71c8f++        // 2^60 * 7 * G++        .quad   0x31723c61fc6811bb+        .quad   0x9cb450486211800f+        .quad   0x768933d347995753+        .quad   0x3491a53502752fcd+        .quad   0x2b552ca0a7da522a+        .quad   0x3230b336449b0250+        .quad   0xf2c4c5bca4b99fb9+        .quad   0x7b2c674958074a22+        .quad   0xd55165883ed28cdf+        .quad   0x12d84fd2d362de39+        .quad   0x0a874ad3e3378e4f+        .quad   0x000d2b1f7c763e74++        // 2^60 * 8 * G++        .quad   0x3d420811d06d4a67+        .quad   0xbefc048590e0ffe3+        .quad   0xf870c6b7bd487bde+        .quad   0x6e2a7316319afa28+        .quad   0x9624778c3e94a8ab+        .quad   0x0ad6f3cee9a78bec+        .quad   0x948ac7810d743c4f+        .quad   0x76627935aaecfccc+        .quad   0x56a8ac24d6d59a9f+        .quad   0xc8db753e3096f006+        .quad   0x477f41e68f4c5299+        .quad   0x588d851cf6c86114++        // 2^64 * 1 * G++        .quad   0x51138ec78df6b0fe+        .quad   0x5397da89e575f51b+        .quad   0x09207a1d717af1b9+        .quad   0x2102fdba2b20d650+        .quad   0xcd2a65e777d1f515+        .quad   0x548991878faa60f1+        .quad   0xb1b73bbcdabc06e5+        .quad   0x654878cba97cc9fb+        .quad   0x969ee405055ce6a1+        .quad   0x36bca7681251ad29+        .quad   0x3a1af517aa7da415+        .quad   0x0ad725db29ecb2ba++        // 2^64 * 2 * G++        .quad   0xdc4267b1834e2457+        .quad   0xb67544b570ce1bc5+        .quad   0x1af07a0bf7d15ed7+        .quad   0x4aefcffb71a03650+        .quad   0xfec7bc0c9b056f85+        .quad   0x537d5268e7f5ffd7+        .quad   0x77afc6624312aefa+        .quad   0x4f675f5302399fd9+        .quad   0xc32d36360415171e+        .quad   0xcd2bef118998483b+        .quad   0x870a6eadd0945110+        .quad   0x0bccbb72a2a86561++        // 2^64 * 3 * G++        .quad   0x185e962feab1a9c8+        .quad   0x86e7e63565147dcd+        .quad   0xb092e031bb5b6df2+        .quad   0x4024f0ab59d6b73e+        .quad   0x186d5e4c50fe1296+        .quad   0xe0397b82fee89f7e+        .quad   0x3bc7f6c5507031b0+        .quad   0x6678fd69108f37c2+        .quad   0x1586fa31636863c2+        .quad   0x07f68c48572d33f2+        .quad   0x4f73cc9f789eaefc+        .quad   0x2d42e2108ead4701++        // 2^64 * 4 * G++        .quad   0x97f5131594dfd29b+        .quad   0x6155985d313f4c6a+        .quad   0xeba13f0708455010+        .quad   0x676b2608b8d2d322+        .quad   0x21717b0d0f537593+        .quad   0x914e690b131e064c+        .quad   0x1bb687ae752ae09f+        .quad   0x420bf3a79b423c6e+        .quad   0x8138ba651c5b2b47+        .quad   0x8671b6ec311b1b80+        .quad   0x7bff0cb1bc3135b0+        .quad   0x745d2ffa9c0cf1e0++        // 2^64 * 5 * G++        .quad   0xbf525a1e2bc9c8bd+        .quad   0xea5b260826479d81+        .quad   0xd511c70edf0155db+        .quad   0x1ae23ceb960cf5d0+        .quad   0x6036df5721d34e6a+        .quad   0xb1db8827997bb3d0+        .quad   0xd3c209c3c8756afa+        .quad   0x06e15be54c1dc839+        .quad   0x5b725d871932994a+        .quad   0x32351cb5ceb1dab0+        .quad   0x7dc41549dab7ca05+        .quad   0x58ded861278ec1f7++        // 2^64 * 6 * G++        .quad   0xd8173793f266c55c+        .quad   0xc8c976c5cc454e49+        .quad   0x5ce382f8bc26c3a8+        .quad   0x2ff39de85485f6f9+        .quad   0x2dfb5ba8b6c2c9a8+        .quad   0x48eeef8ef52c598c+        .quad   0x33809107f12d1573+        .quad   0x08ba696b531d5bd8+        .quad   0x77ed3eeec3efc57a+        .quad   0x04e05517d4ff4811+        .quad   0xea3d7a3ff1a671cb+        .quad   0x120633b4947cfe54++        // 2^64 * 7 * G++        .quad   0x0b94987891610042+        .quad   0x4ee7b13cecebfae8+        .quad   0x70be739594f0a4c0+        .quad   0x35d30a99b4d59185+        .quad   0x82bd31474912100a+        .quad   0xde237b6d7e6fbe06+        .quad   0xe11e761911ea79c6+        .quad   0x07433be3cb393bde+        .quad   0xff7944c05ce997f4+        .quad   0x575d3de4b05c51a3+        .quad   0x583381fd5a76847c+        .quad   0x2d873ede7af6da9f++        // 2^64 * 8 * G++        .quad   0x157a316443373409+        .quad   0xfab8b7eef4aa81d9+        .quad   0xb093fee6f5a64806+        .quad   0x2e773654707fa7b6+        .quad   0xaa6202e14e5df981+        .quad   0xa20d59175015e1f5+        .quad   0x18a275d3bae21d6c+        .quad   0x0543618a01600253+        .quad   0x0deabdf4974c23c1+        .quad   0xaa6f0a259dce4693+        .quad   0x04202cb8a29aba2c+        .quad   0x4b1443362d07960d++        // 2^68 * 1 * G++        .quad   0x47b837f753242cec+        .quad   0x256dc48cc04212f2+        .quad   0xe222fbfbe1d928c5+        .quad   0x48ea295bad8a2c07+        .quad   0x299b1c3f57c5715e+        .quad   0x96cb929e6b686d90+        .quad   0x3004806447235ab3+        .quad   0x2c435c24a44d9fe1+        .quad   0x0607c97c80f8833f+        .quad   0x0e851578ca25ec5b+        .quad   0x54f7450b161ebb6f+        .quad   0x7bcb4792a0def80e++        // 2^68 * 2 * G++        .quad   0x8487e3d02bc73659+        .quad   0x4baf8445059979df+        .quad   0xd17c975adcad6fbf+        .quad   0x57369f0bdefc96b6+        .quad   0x1cecd0a0045224c2+        .quad   0x757f1b1b69e53952+        .quad   0x775b7a925289f681+        .quad   0x1b6cc62016736148+        .quad   0xf1a9990175638698+        .quad   0x353dd1beeeaa60d3+        .quad   0x849471334c9ba488+        .quad   0x63fa6e6843ade311++        // 2^68 * 3 * G++        .quad   0xd15c20536597c168+        .quad   0x9f73740098d28789+        .quad   0x18aee7f13257ba1f+        .quad   0x3418bfda07346f14+        .quad   0x2195becdd24b5eb7+        .quad   0x5e41f18cc0cd44f9+        .quad   0xdf28074441ca9ede+        .quad   0x07073b98f35b7d67+        .quad   0xd03c676c4ce530d4+        .quad   0x0b64c0473b5df9f4+        .quad   0x065cef8b19b3a31e+        .quad   0x3084d661533102c9++        // 2^68 * 4 * G++        .quad   0xe1f6b79ebf8469ad+        .quad   0x15801004e2663135+        .quad   0x9a498330af74181b+        .quad   0x3ba2504f049b673c+        .quad   0x9a6ce876760321fd+        .quad   0x7fe2b5109eb63ad8+        .quad   0x00e7d4ae8ac80592+        .quad   0x73d86b7abb6f723a+        .quad   0x0b52b5606dba5ab6+        .quad   0xa9134f0fbbb1edab+        .quad   0x30a9520d9b04a635+        .quad   0x6813b8f37973e5db++        // 2^68 * 5 * G++        .quad   0x9854b054334127c1+        .quad   0x105d047882fbff25+        .quad   0xdb49f7f944186f4f+        .quad   0x1768e838bed0b900+        .quad   0xf194ca56f3157e29+        .quad   0x136d35705ef528a5+        .quad   0xdd4cef778b0599bc+        .quad   0x7d5472af24f833ed+        .quad   0xd0ef874daf33da47+        .quad   0x00d3be5db6e339f9+        .quad   0x3f2a8a2f9c9ceece+        .quad   0x5d1aeb792352435a++        // 2^68 * 6 * G++        .quad   0xf59e6bb319cd63ca+        .quad   0x670c159221d06839+        .quad   0xb06d565b2150cab6+        .quad   0x20fb199d104f12a3+        .quad   0x12c7bfaeb61ba775+        .quad   0xb84e621fe263bffd+        .quad   0x0b47a5c35c840dcf+        .quad   0x7e83be0bccaf8634+        .quad   0x61943dee6d99c120+        .quad   0x86101f2e460b9fe0+        .quad   0x6bb2f1518ee8598d+        .quad   0x76b76289fcc475cc++        // 2^68 * 7 * G++        .quad   0x791b4cc1756286fa+        .quad   0xdbced317d74a157c+        .quad   0x7e732421ea72bde6+        .quad   0x01fe18491131c8e9+        .quad   0x4245f1a1522ec0b3+        .quad   0x558785b22a75656d+        .quad   0x1d485a2548a1b3c0+        .quad   0x60959eccd58fe09f+        .quad   0x3ebfeb7ba8ed7a09+        .quad   0x49fdc2bbe502789c+        .quad   0x44ebce5d3c119428+        .quad   0x35e1eb55be947f4a++        // 2^68 * 8 * G++        .quad   0xdbdae701c5738dd3+        .quad   0xf9c6f635b26f1bee+        .quad   0x61e96a8042f15ef4+        .quad   0x3aa1d11faf60a4d8+        .quad   0x14fd6dfa726ccc74+        .quad   0x3b084cfe2f53b965+        .quad   0xf33ae4f552a2c8b4+        .quad   0x59aab07a0d40166a+        .quad   0x77bcec4c925eac25+        .quad   0x1848718460137738+        .quad   0x5b374337fea9f451+        .quad   0x1865e78ec8e6aa46++        // 2^72 * 1 * G++        .quad   0xccc4b7c7b66e1f7a+        .quad   0x44157e25f50c2f7e+        .quad   0x3ef06dfc713eaf1c+        .quad   0x582f446752da63f7+        .quad   0x967c54e91c529ccb+        .quad   0x30f6269264c635fb+        .quad   0x2747aff478121965+        .quad   0x17038418eaf66f5c+        .quad   0xc6317bd320324ce4+        .quad   0xa81042e8a4488bc4+        .quad   0xb21ef18b4e5a1364+        .quad   0x0c2a1c4bcda28dc9++        // 2^72 * 2 * G++        .quad   0xd24dc7d06f1f0447+        .quad   0xb2269e3edb87c059+        .quad   0xd15b0272fbb2d28f+        .quad   0x7c558bd1c6f64877+        .quad   0xedc4814869bd6945+        .quad   0x0d6d907dbe1c8d22+        .quad   0xc63bd212d55cc5ab+        .quad   0x5a6a9b30a314dc83+        .quad   0xd0ec1524d396463d+        .quad   0x12bb628ac35a24f0+        .quad   0xa50c3a791cbc5fa4+        .quad   0x0404a5ca0afbafc3++        // 2^72 * 3 * G++        .quad   0x8c1f40070aa743d6+        .quad   0xccbad0cb5b265ee8+        .quad   0x574b046b668fd2de+        .quad   0x46395bfdcadd9633+        .quad   0x62bc9e1b2a416fd1+        .quad   0xb5c6f728e350598b+        .quad   0x04343fd83d5d6967+        .quad   0x39527516e7f8ee98+        .quad   0x117fdb2d1a5d9a9c+        .quad   0x9c7745bcd1005c2a+        .quad   0xefd4bef154d56fea+        .quad   0x76579a29e822d016++        // 2^72 * 4 * G++        .quad   0x45b68e7e49c02a17+        .quad   0x23cd51a2bca9a37f+        .quad   0x3ed65f11ec224c1b+        .quad   0x43a384dc9e05bdb1+        .quad   0x333cb51352b434f2+        .quad   0xd832284993de80e1+        .quad   0xb5512887750d35ce+        .quad   0x02c514bb2a2777c1+        .quad   0x684bd5da8bf1b645+        .quad   0xfb8bd37ef6b54b53+        .quad   0x313916d7a9b0d253+        .quad   0x1160920961548059++        // 2^72 * 5 * G++        .quad   0xb44d166929dacfaa+        .quad   0xda529f4c8413598f+        .quad   0xe9ef63ca453d5559+        .quad   0x351e125bc5698e0b+        .quad   0x7a385616369b4dcd+        .quad   0x75c02ca7655c3563+        .quad   0x7dc21bf9d4f18021+        .quad   0x2f637d7491e6e042+        .quad   0xd4b49b461af67bbe+        .quad   0xd603037ac8ab8961+        .quad   0x71dee19ff9a699fb+        .quad   0x7f182d06e7ce2a9a++        // 2^72 * 6 * G++        .quad   0x7a7c8e64ab0168ec+        .quad   0xcb5a4a5515edc543+        .quad   0x095519d347cd0eda+        .quad   0x67d4ac8c343e93b0+        .quad   0x09454b728e217522+        .quad   0xaa58e8f4d484b8d8+        .quad   0xd358254d7f46903c+        .quad   0x44acc043241c5217+        .quad   0x1c7d6bbb4f7a5777+        .quad   0x8b35fed4918313e1+        .quad   0x4adca1c6c96b4684+        .quad   0x556d1c8312ad71bd++        // 2^72 * 7 * G++        .quad   0x17ef40e30c8d3982+        .quad   0x31f7073e15a3fa34+        .quad   0x4f21f3cb0773646e+        .quad   0x746c6c6d1d824eff+        .quad   0x81f06756b11be821+        .quad   0x0faff82310a3f3dd+        .quad   0xf8b2d0556a99465d+        .quad   0x097abe38cc8c7f05+        .quad   0x0c49c9877ea52da4+        .quad   0x4c4369559bdc1d43+        .quad   0x022c3809f7ccebd2+        .quad   0x577e14a34bee84bd++        // 2^72 * 8 * G++        .quad   0xf0e268ac61a73b0a+        .quad   0xf2fafa103791a5f5+        .quad   0xc1e13e826b6d00e9+        .quad   0x60fa7ee96fd78f42+        .quad   0x94fecebebd4dd72b+        .quad   0xf46a4fda060f2211+        .quad   0x124a5977c0c8d1ff+        .quad   0x705304b8fb009295+        .quad   0xb63d1d354d296ec6+        .quad   0xf3c3053e5fad31d8+        .quad   0x670b958cb4bd42ec+        .quad   0x21398e0ca16353fd++        // 2^76 * 1 * G++        .quad   0x216ab2ca8da7d2ef+        .quad   0x366ad9dd99f42827+        .quad   0xae64b9004fdd3c75+        .quad   0x403a395b53909e62+        .quad   0x86c5fc16861b7e9a+        .quad   0xf6a330476a27c451+        .quad   0x01667267a1e93597+        .quad   0x05ffb9cd6082dfeb+        .quad   0xa617fa9ff53f6139+        .quad   0x60f2b5e513e66cb6+        .quad   0xd7a8beefb3448aa4+        .quad   0x7a2932856f5ea192++        // 2^76 * 2 * G++        .quad   0x0b39d761b02de888+        .quad   0x5f550e7ed2414e1f+        .quad   0xa6bfa45822e1a940+        .quad   0x050a2f7dfd447b99+        .quad   0xb89c444879639302+        .quad   0x4ae4f19350c67f2c+        .quad   0xf0b35da8c81af9c6+        .quad   0x39d0003546871017+        .quad   0x437c3b33a650db77+        .quad   0x6bafe81dbac52bb2+        .quad   0xfe99402d2db7d318+        .quad   0x2b5b7eec372ba6ce++        // 2^76 * 3 * G++        .quad   0xb3bc4bbd83f50eef+        .quad   0x508f0c998c927866+        .quad   0x43e76587c8b7e66e+        .quad   0x0f7655a3a47f98d9+        .quad   0xa694404d613ac8f4+        .quad   0x500c3c2bfa97e72c+        .quad   0x874104d21fcec210+        .quad   0x1b205fb38604a8ee+        .quad   0x55ecad37d24b133c+        .quad   0x441e147d6038c90b+        .quad   0x656683a1d62c6fee+        .quad   0x0157d5dc87e0ecae++        // 2^76 * 4 * G++        .quad   0xf2a7af510354c13d+        .quad   0xd7a0b145aa372b60+        .quad   0x2869b96a05a3d470+        .quad   0x6528e42d82460173+        .quad   0x95265514d71eb524+        .quad   0xe603d8815df14593+        .quad   0x147cdf410d4de6b7+        .quad   0x5293b1730437c850+        .quad   0x23d0e0814bccf226+        .quad   0x92c745cd8196fb93+        .quad   0x8b61796c59541e5b+        .quad   0x40a44df0c021f978++        // 2^76 * 5 * G++        .quad   0xdaa869894f20ea6a+        .quad   0xea14a3d14c620618+        .quad   0x6001fccb090bf8be+        .quad   0x35f4e822947e9cf0+        .quad   0x86c96e514bc5d095+        .quad   0xf20d4098fca6804a+        .quad   0x27363d89c826ea5d+        .quad   0x39ca36565719cacf+        .quad   0x97506f2f6f87b75c+        .quad   0xc624aea0034ae070+        .quad   0x1ec856e3aad34dd6+        .quad   0x055b0be0e440e58f++        // 2^76 * 6 * G++        .quad   0x6469a17d89735d12+        .quad   0xdb6f27d5e662b9f1+        .quad   0x9fcba3286a395681+        .quad   0x363b8004d269af25+        .quad   0x4d12a04b6ea33da2+        .quad   0x57cf4c15e36126dd+        .quad   0x90ec9675ee44d967+        .quad   0x64ca348d2a985aac+        .quad   0x99588e19e4c4912d+        .quad   0xefcc3b4e1ca5ce6b+        .quad   0x4522ea60fa5b98d5+        .quad   0x7064bbab1de4a819++        // 2^76 * 7 * G++        .quad   0xb919e1515a770641+        .quad   0xa9a2e2c74e7f8039+        .quad   0x7527250b3df23109+        .quad   0x756a7330ac27b78b+        .quad   0xa290c06142542129+        .quad   0xf2e2c2aebe8d5b90+        .quad   0xcf2458db76abfe1b+        .quad   0x02157ade83d626bf+        .quad   0x3e46972a1b9a038b+        .quad   0x2e4ee66a7ee03fb4+        .quad   0x81a248776edbb4ca+        .quad   0x1a944ee88ecd0563++        // 2^76 * 8 * G++        .quad   0xd5a91d1151039372+        .quad   0x2ed377b799ca26de+        .quad   0xa17202acfd366b6b+        .quad   0x0730291bd6901995+        .quad   0xbb40a859182362d6+        .quad   0xb99f55778a4d1abb+        .quad   0x8d18b427758559f6+        .quad   0x26c20fe74d26235a+        .quad   0x648d1d9fe9cc22f5+        .quad   0x66bc561928dd577c+        .quad   0x47d3ed21652439d1+        .quad   0x49d271acedaf8b49++        // 2^80 * 1 * G++        .quad   0x89f5058a382b33f3+        .quad   0x5ae2ba0bad48c0b4+        .quad   0x8f93b503a53db36e+        .quad   0x5aa3ed9d95a232e6+        .quad   0x2798aaf9b4b75601+        .quad   0x5eac72135c8dad72+        .quad   0xd2ceaa6161b7a023+        .quad   0x1bbfb284e98f7d4e+        .quad   0x656777e9c7d96561+        .quad   0xcb2b125472c78036+        .quad   0x65053299d9506eee+        .quad   0x4a07e14e5e8957cc++        // 2^80 * 2 * G++        .quad   0x4ee412cb980df999+        .quad   0xa315d76f3c6ec771+        .quad   0xbba5edde925c77fd+        .quad   0x3f0bac391d313402+        .quad   0x240b58cdc477a49b+        .quad   0xfd38dade6447f017+        .quad   0x19928d32a7c86aad+        .quad   0x50af7aed84afa081+        .quad   0x6e4fde0115f65be5+        .quad   0x29982621216109b2+        .quad   0x780205810badd6d9+        .quad   0x1921a316baebd006++        // 2^80 * 3 * G++        .quad   0x89422f7edfb870fc+        .quad   0x2c296beb4f76b3bd+        .quad   0x0738f1d436c24df7+        .quad   0x6458df41e273aeb0+        .quad   0xd75aad9ad9f3c18b+        .quad   0x566a0eef60b1c19c+        .quad   0x3e9a0bac255c0ed9+        .quad   0x7b049deca062c7f5+        .quad   0xdccbe37a35444483+        .quad   0x758879330fedbe93+        .quad   0x786004c312c5dd87+        .quad   0x6093dccbc2950e64++        // 2^80 * 4 * G++        .quad   0x1ff39a8585e0706d+        .quad   0x36d0a5d8b3e73933+        .quad   0x43b9f2e1718f453b+        .quad   0x57d1ea084827a97c+        .quad   0x6bdeeebe6084034b+        .quad   0x3199c2b6780fb854+        .quad   0x973376abb62d0695+        .quad   0x6e3180c98b647d90+        .quad   0xee7ab6e7a128b071+        .quad   0xa4c1596d93a88baa+        .quad   0xf7b4de82b2216130+        .quad   0x363e999ddd97bd18++        // 2^80 * 5 * G++        .quad   0x96a843c135ee1fc4+        .quad   0x976eb35508e4c8cf+        .quad   0xb42f6801b58cd330+        .quad   0x48ee9b78693a052b+        .quad   0x2f1848dce24baec6+        .quad   0x769b7255babcaf60+        .quad   0x90cb3c6e3cefe931+        .quad   0x231f979bc6f9b355+        .quad   0x5c31de4bcc2af3c6+        .quad   0xb04bb030fe208d1f+        .quad   0xb78d7009c14fb466+        .quad   0x079bfa9b08792413++        // 2^80 * 6 * G++        .quad   0xe3903a51da300df4+        .quad   0x843964233da95ab0+        .quad   0xed3cf12d0b356480+        .quad   0x038c77f684817194+        .quad   0xf3c9ed80a2d54245+        .quad   0x0aa08b7877f63952+        .quad   0xd76dac63d1085475+        .quad   0x1ef4fb159470636b+        .quad   0x854e5ee65b167bec+        .quad   0x59590a4296d0cdc2+        .quad   0x72b2df3498102199+        .quad   0x575ee92a4a0bff56++        // 2^80 * 7 * G++        .quad   0xd4c080908a182fcf+        .quad   0x30e170c299489dbd+        .quad   0x05babd5752f733de+        .quad   0x43d4e7112cd3fd00+        .quad   0x5d46bc450aa4d801+        .quad   0xc3af1227a533b9d8+        .quad   0x389e3b262b8906c2+        .quad   0x200a1e7e382f581b+        .quad   0x518db967eaf93ac5+        .quad   0x71bc989b056652c0+        .quad   0xfe2b85d9567197f5+        .quad   0x050eca52651e4e38++        // 2^80 * 8 * G++        .quad   0xc3431ade453f0c9c+        .quad   0xe9f5045eff703b9b+        .quad   0xfcd97ac9ed847b3d+        .quad   0x4b0ee6c21c58f4c6+        .quad   0x97ac397660e668ea+        .quad   0x9b19bbfe153ab497+        .quad   0x4cb179b534eca79f+        .quad   0x6151c09fa131ae57+        .quad   0x3af55c0dfdf05d96+        .quad   0xdd262ee02ab4ee7a+        .quad   0x11b2bb8712171709+        .quad   0x1fef24fa800f030b++        // 2^84 * 1 * G++        .quad   0xb496123a6b6c6609+        .quad   0xa750fe8580ab5938+        .quad   0xf471bf39b7c27a5f+        .quad   0x507903ce77ac193c+        .quad   0xff91a66a90166220+        .quad   0xf22552ae5bf1e009+        .quad   0x7dff85d87f90df7c+        .quad   0x4f620ffe0c736fb9+        .quad   0x62f90d65dfde3e34+        .quad   0xcf28c592b9fa5fad+        .quad   0x99c86ef9c6164510+        .quad   0x25d448044a256c84++        // 2^84 * 2 * G++        .quad   0xbd68230ec7e9b16f+        .quad   0x0eb1b9c1c1c5795d+        .quad   0x7943c8c495b6b1ff+        .quad   0x2f9faf620bbacf5e+        .quad   0x2c7c4415c9022b55+        .quad   0x56a0d241812eb1fe+        .quad   0xf02ea1c9d7b65e0d+        .quad   0x4180512fd5323b26+        .quad   0xa4ff3e698a48a5db+        .quad   0xba6a3806bd95403b+        .quad   0x9f7ce1af47d5b65d+        .quad   0x15e087e55939d2fb++        // 2^84 * 3 * G++        .quad   0x12207543745c1496+        .quad   0xdaff3cfdda38610c+        .quad   0xe4e797272c71c34f+        .quad   0x39c07b1934bdede9+        .quad   0x8894186efb963f38+        .quad   0x48a00e80dc639bd5+        .quad   0xa4e8092be96c1c99+        .quad   0x5a097d54ca573661+        .quad   0x2d45892b17c9e755+        .quad   0xd033fd7289308df8+        .quad   0x6c2fe9d9525b8bd9+        .quad   0x2edbecf1c11cc079++        // 2^84 * 4 * G++        .quad   0x1616a4e3c715a0d2+        .quad   0x53623cb0f8341d4d+        .quad   0x96ef5329c7e899cb+        .quad   0x3d4e8dbba668baa6+        .quad   0xee0f0fddd087a25f+        .quad   0x9c7531555c3e34ee+        .quad   0x660c572e8fab3ab5+        .quad   0x0854fc44544cd3b2+        .quad   0x61eba0c555edad19+        .quad   0x24b533fef0a83de6+        .quad   0x3b77042883baa5f8+        .quad   0x678f82b898a47e8d++        // 2^84 * 5 * G++        .quad   0xb1491d0bd6900c54+        .quad   0x3539722c9d132636+        .quad   0x4db928920b362bc9+        .quad   0x4d7cd1fea68b69df+        .quad   0x1e09d94057775696+        .quad   0xeed1265c3cd951db+        .quad   0xfa9dac2b20bce16f+        .quad   0x0f7f76e0e8d089f4+        .quad   0x36d9ebc5d485b00c+        .quad   0xa2596492e4adb365+        .quad   0xc1659480c2119ccd+        .quad   0x45306349186e0d5f++        // 2^84 * 6 * G++        .quad   0x94ddd0c1a6cdff1d+        .quad   0x55f6f115e84213ae+        .quad   0x6c935f85992fcf6a+        .quad   0x067ee0f54a37f16f+        .quad   0x96a414ec2b072491+        .quad   0x1bb2218127a7b65b+        .quad   0x6d2849596e8a4af0+        .quad   0x65f3b08ccd27765f+        .quad   0xecb29fff199801f7+        .quad   0x9d361d1fa2a0f72f+        .quad   0x25f11d2375fd2f49+        .quad   0x124cefe80fe10fe2++        // 2^84 * 7 * G++        .quad   0x4c126cf9d18df255+        .quad   0xc1d471e9147a63b6+        .quad   0x2c6d3c73f3c93b5f+        .quad   0x6be3a6a2e3ff86a2+        .quad   0x1518e85b31b16489+        .quad   0x8faadcb7db710bfb+        .quad   0x39b0bdf4a14ae239+        .quad   0x05f4cbea503d20c1+        .quad   0xce040e9ec04145bc+        .quad   0xc71ff4e208f6834c+        .quad   0xbd546e8dab8847a3+        .quad   0x64666aa0a4d2aba5++        // 2^84 * 8 * G++        .quad   0x6841435a7c06d912+        .quad   0xca123c21bb3f830b+        .quad   0xd4b37b27b1cbe278+        .quad   0x1d753b84c76f5046+        .quad   0xb0c53bf73337e94c+        .quad   0x7cb5697e11e14f15+        .quad   0x4b84abac1930c750+        .quad   0x28dd4abfe0640468+        .quad   0x7dc0b64c44cb9f44+        .quad   0x18a3e1ace3925dbf+        .quad   0x7a3034862d0457c4+        .quad   0x4c498bf78a0c892e++        // 2^88 * 1 * G++        .quad   0x37d653fb1aa73196+        .quad   0x0f9495303fd76418+        .quad   0xad200b09fb3a17b2+        .quad   0x544d49292fc8613e+        .quad   0x22d2aff530976b86+        .quad   0x8d90b806c2d24604+        .quad   0xdca1896c4de5bae5+        .quad   0x28005fe6c8340c17+        .quad   0x6aefba9f34528688+        .quad   0x5c1bff9425107da1+        .quad   0xf75bbbcd66d94b36+        .quad   0x72e472930f316dfa++        // 2^88 * 2 * G++        .quad   0x2695208c9781084f+        .quad   0xb1502a0b23450ee1+        .quad   0xfd9daea603efde02+        .quad   0x5a9d2e8c2733a34c+        .quad   0x07f3f635d32a7627+        .quad   0x7aaa4d865f6566f0+        .quad   0x3c85e79728d04450+        .quad   0x1fee7f000fe06438+        .quad   0x765305da03dbf7e5+        .quad   0xa4daf2491434cdbd+        .quad   0x7b4ad5cdd24a88ec+        .quad   0x00f94051ee040543++        // 2^88 * 3 * G++        .quad   0x8d356b23c3d330b2+        .quad   0xf21c8b9bb0471b06+        .quad   0xb36c316c6e42b83c+        .quad   0x07d79c7e8beab10d+        .quad   0xd7ef93bb07af9753+        .quad   0x583ed0cf3db766a7+        .quad   0xce6998bf6e0b1ec5+        .quad   0x47b7ffd25dd40452+        .quad   0x87fbfb9cbc08dd12+        .quad   0x8a066b3ae1eec29b+        .quad   0x0d57242bdb1fc1bf+        .quad   0x1c3520a35ea64bb6++        // 2^88 * 4 * G++        .quad   0x80d253a6bccba34a+        .quad   0x3e61c3a13838219b+        .quad   0x90c3b6019882e396+        .quad   0x1c3d05775d0ee66f+        .quad   0xcda86f40216bc059+        .quad   0x1fbb231d12bcd87e+        .quad   0xb4956a9e17c70990+        .quad   0x38750c3b66d12e55+        .quad   0x692ef1409422e51a+        .quad   0xcbc0c73c2b5df671+        .quad   0x21014fe7744ce029+        .quad   0x0621e2c7d330487c++        // 2^88 * 5 * G++        .quad   0xaf9860cc8259838d+        .quad   0x90ea48c1c69f9adc+        .quad   0x6526483765581e30+        .quad   0x0007d6097bd3a5bc+        .quad   0xb7ae1796b0dbf0f3+        .quad   0x54dfafb9e17ce196+        .quad   0x25923071e9aaa3b4+        .quad   0x5d8e589ca1002e9d+        .quad   0xc0bf1d950842a94b+        .quad   0xb2d3c363588f2e3e+        .quad   0x0a961438bb51e2ef+        .quad   0x1583d7783c1cbf86++        // 2^88 * 6 * G++        .quad   0xeceea2ef5da27ae1+        .quad   0x597c3a1455670174+        .quad   0xc9a62a126609167a+        .quad   0x252a5f2e81ed8f70+        .quad   0x90034704cc9d28c7+        .quad   0x1d1b679ef72cc58f+        .quad   0x16e12b5fbe5b8726+        .quad   0x4958064e83c5580a+        .quad   0x0d2894265066e80d+        .quad   0xfcc3f785307c8c6b+        .quad   0x1b53da780c1112fd+        .quad   0x079c170bd843b388++        // 2^88 * 7 * G++        .quad   0x0506ece464fa6fff+        .quad   0xbee3431e6205e523+        .quad   0x3579422451b8ea42+        .quad   0x6dec05e34ac9fb00+        .quad   0xcdd6cd50c0d5d056+        .quad   0x9af7686dbb03573b+        .quad   0x3ca6723ff3c3ef48+        .quad   0x6768c0d7317b8acc+        .quad   0x94b625e5f155c1b3+        .quad   0x417bf3a7997b7b91+        .quad   0xc22cbddc6d6b2600+        .quad   0x51445e14ddcd52f4++        // 2^88 * 8 * G++        .quad   0x57502b4b3b144951+        .quad   0x8e67ff6b444bbcb3+        .quad   0xb8bd6927166385db+        .quad   0x13186f31e39295c8+        .quad   0x893147ab2bbea455+        .quad   0x8c53a24f92079129+        .quad   0x4b49f948be30f7a7+        .quad   0x12e990086e4fd43d+        .quad   0xf10c96b37fdfbb2e+        .quad   0x9f9a935e121ceaf9+        .quad   0xdf1136c43a5b983f+        .quad   0x77b2e3f05d3e99af++        // 2^92 * 1 * G++        .quad   0xfd0d75879cf12657+        .quad   0xe82fef94e53a0e29+        .quad   0xcc34a7f05bbb4be7+        .quad   0x0b251172a50c38a2+        .quad   0x9532f48fcc5cd29b+        .quad   0x2ba851bea3ce3671+        .quad   0x32dacaa051122941+        .quad   0x478d99d9350004f2+        .quad   0x1d5ad94890bb02c0+        .quad   0x50e208b10ec25115+        .quad   0xa26a22894ef21702+        .quad   0x4dc923343b524805++        // 2^92 * 2 * G++        .quad   0xe3828c400f8086b6+        .quad   0x3f77e6f7979f0dc8+        .quad   0x7ef6de304df42cb4+        .quad   0x5265797cb6abd784+        .quad   0x3ad3e3ebf36c4975+        .quad   0xd75d25a537862125+        .quad   0xe873943da025a516+        .quad   0x6bbc7cb4c411c847+        .quad   0x3c6f9cd1d4a50d56+        .quad   0xb6244077c6feab7e+        .quad   0x6ff9bf483580972e+        .quad   0x00375883b332acfb++        // 2^92 * 3 * G++        .quad   0x0001b2cd28cb0940+        .quad   0x63fb51a06f1c24c9+        .quad   0xb5ad8691dcd5ca31+        .quad   0x67238dbd8c450660+        .quad   0xc98bec856c75c99c+        .quad   0xe44184c000e33cf4+        .quad   0x0a676b9bba907634+        .quad   0x669e2cb571f379d7+        .quad   0xcb116b73a49bd308+        .quad   0x025aad6b2392729e+        .quad   0xb4793efa3f55d9b1+        .quad   0x72a1056140678bb9++        // 2^92 * 4 * G++        .quad   0xa2b6812b1cc9249d+        .quad   0x62866eee21211f58+        .quad   0x2cb5c5b85df10ece+        .quad   0x03a6b259e263ae00+        .quad   0x0d8d2909e2e505b6+        .quad   0x98ca78abc0291230+        .quad   0x77ef5569a9b12327+        .quad   0x7c77897b81439b47+        .quad   0xf1c1b5e2de331cb5+        .quad   0x5a9f5d8e15fca420+        .quad   0x9fa438f17bd932b1+        .quad   0x2a381bf01c6146e7++        // 2^92 * 5 * G++        .quad   0xac9b9879cfc811c1+        .quad   0x8b7d29813756e567+        .quad   0x50da4e607c70edfc+        .quad   0x5dbca62f884400b6+        .quad   0xf7c0be32b534166f+        .quad   0x27e6ca6419cf70d4+        .quad   0x934df7d7a957a759+        .quad   0x5701461dabdec2aa+        .quad   0x2c6747402c915c25+        .quad   0x1bdcd1a80b0d340a+        .quad   0x5e5601bd07b43f5f+        .quad   0x2555b4e05539a242++        // 2^92 * 6 * G++        .quad   0x6fc09f5266ddd216+        .quad   0xdce560a7c8e37048+        .quad   0xec65939da2df62fd+        .quad   0x7a869ae7e52ed192+        .quad   0x78409b1d87e463d4+        .quad   0xad4da95acdfb639d+        .quad   0xec28773755259b9c+        .quad   0x69c806e9c31230ab+        .quad   0x7b48f57414bb3f22+        .quad   0x68c7cee4aedccc88+        .quad   0xed2f936179ed80be+        .quad   0x25d70b885f77bc4b++        // 2^92 * 7 * G++        .quad   0x4151c3d9762bf4de+        .quad   0x083f435f2745d82b+        .quad   0x29775a2e0d23ddd5+        .quad   0x138e3a6269a5db24+        .quad   0x98459d29bb1ae4d4+        .quad   0x56b9c4c739f954ec+        .quad   0x832743f6c29b4b3e+        .quad   0x21ea8e2798b6878a+        .quad   0x87bef4b46a5a7b9c+        .quad   0xd2299d1b5fc1d062+        .quad   0x82409818dd321648+        .quad   0x5c5abeb1e5a2e03d++        // 2^92 * 8 * G++        .quad   0x14722af4b73c2ddb+        .quad   0xbc470c5f5a05060d+        .quad   0x00943eac2581b02e+        .quad   0x0e434b3b1f499c8f+        .quad   0x02cde6de1306a233+        .quad   0x7b5a52a2116f8ec7+        .quad   0xe1c681f4c1163b5b+        .quad   0x241d350660d32643+        .quad   0x6be4404d0ebc52c7+        .quad   0xae46233bb1a791f5+        .quad   0x2aec170ed25db42b+        .quad   0x1d8dfd966645d694++        // 2^96 * 1 * G++        .quad   0x296fa9c59c2ec4de+        .quad   0xbc8b61bf4f84f3cb+        .quad   0x1c7706d917a8f908+        .quad   0x63b795fc7ad3255d+        .quad   0xd598639c12ddb0a4+        .quad   0xa5d19f30c024866b+        .quad   0xd17c2f0358fce460+        .quad   0x07a195152e095e8a+        .quad   0xa8368f02389e5fc8+        .quad   0x90433b02cf8de43b+        .quad   0xafa1fd5dc5412643+        .quad   0x3e8fe83d032f0137++        // 2^96 * 2 * G++        .quad   0x2f8b15b90570a294+        .quad   0x94f2427067084549+        .quad   0xde1c5ae161bbfd84+        .quad   0x75ba3b797fac4007+        .quad   0x08704c8de8efd13c+        .quad   0xdfc51a8e33e03731+        .quad   0xa59d5da51260cde3+        .quad   0x22d60899a6258c86+        .quad   0x6239dbc070cdd196+        .quad   0x60fe8a8b6c7d8a9a+        .quad   0xb38847bceb401260+        .quad   0x0904d07b87779e5e++        // 2^96 * 3 * G++        .quad   0xb4ce1fd4ddba919c+        .quad   0xcf31db3ec74c8daa+        .quad   0x2c63cc63ad86cc51+        .quad   0x43e2143fbc1dde07+        .quad   0xf4322d6648f940b9+        .quad   0x06952f0cbd2d0c39+        .quad   0x167697ada081f931+        .quad   0x6240aacebaf72a6c+        .quad   0xf834749c5ba295a0+        .quad   0xd6947c5bca37d25a+        .quad   0x66f13ba7e7c9316a+        .quad   0x56bdaf238db40cac++        // 2^96 * 4 * G++        .quad   0x362ab9e3f53533eb+        .quad   0x338568d56eb93d40+        .quad   0x9e0e14521d5a5572+        .quad   0x1d24a86d83741318+        .quad   0x1310d36cc19d3bb2+        .quad   0x062a6bb7622386b9+        .quad   0x7c9b8591d7a14f5c+        .quad   0x03aa31507e1e5754+        .quad   0xf4ec7648ffd4ce1f+        .quad   0xe045eaf054ac8c1c+        .quad   0x88d225821d09357c+        .quad   0x43b261dc9aeb4859++        // 2^96 * 5 * G++        .quad   0xe55b1e1988bb79bb+        .quad   0xa09ed07dc17a359d+        .quad   0xb02c2ee2603dea33+        .quad   0x326055cf5b276bc2+        .quad   0x19513d8b6c951364+        .quad   0x94fe7126000bf47b+        .quad   0x028d10ddd54f9567+        .quad   0x02b4d5e242940964+        .quad   0xb4a155cb28d18df2+        .quad   0xeacc4646186ce508+        .quad   0xc49cf4936c824389+        .quad   0x27a6c809ae5d3410++        // 2^96 * 6 * G++        .quad   0x8ba6ebcd1f0db188+        .quad   0x37d3d73a675a5be8+        .quad   0xf22edfa315f5585a+        .quad   0x2cb67174ff60a17e+        .quad   0xcd2c270ac43d6954+        .quad   0xdd4a3e576a66cab2+        .quad   0x79fa592469d7036c+        .quad   0x221503603d8c2599+        .quad   0x59eecdf9390be1d0+        .quad   0xa9422044728ce3f1+        .quad   0x82891c667a94f0f4+        .quad   0x7b1df4b73890f436++        // 2^96 * 7 * G++        .quad   0xe492f2e0b3b2a224+        .quad   0x7c6c9e062b551160+        .quad   0x15eb8fe20d7f7b0e+        .quad   0x61fcef2658fc5992+        .quad   0x5f2e221807f8f58c+        .quad   0xe3555c9fd49409d4+        .quad   0xb2aaa88d1fb6a630+        .quad   0x68698245d352e03d+        .quad   0xdbb15d852a18187a+        .quad   0xf3e4aad386ddacd7+        .quad   0x44bae2810ff6c482+        .quad   0x46cf4c473daf01cf++        // 2^96 * 8 * G++        .quad   0x426525ed9ec4e5f9+        .quad   0x0e5eda0116903303+        .quad   0x72b1a7f2cbe5cadc+        .quad   0x29387bcd14eb5f40+        .quad   0x213c6ea7f1498140+        .quad   0x7c1e7ef8392b4854+        .quad   0x2488c38c5629ceba+        .quad   0x1065aae50d8cc5bb+        .quad   0x1c2c4525df200d57+        .quad   0x5c3b2dd6bfca674a+        .quad   0x0a07e7b1e1834030+        .quad   0x69a198e64f1ce716++        // 2^100 * 1 * G++        .quad   0x7afcd613efa9d697+        .quad   0x0cc45aa41c067959+        .quad   0xa56fe104c1fada96+        .quad   0x3a73b70472e40365+        .quad   0x7b26e56b9e2d4734+        .quad   0xc4c7132b81c61675+        .quad   0xef5c9525ec9cde7f+        .quad   0x39c80b16e71743ad+        .quad   0x0f196e0d1b826c68+        .quad   0xf71ff0e24960e3db+        .quad   0x6113167023b7436c+        .quad   0x0cf0ea5877da7282++        // 2^100 * 2 * G++        .quad   0x196c80a4ddd4ccbd+        .quad   0x22e6f55d95f2dd9d+        .quad   0xc75e33c740d6c71b+        .quad   0x7bb51279cb3c042f+        .quad   0xe332ced43ba6945a+        .quad   0xde0b1361e881c05d+        .quad   0x1ad40f095e67ed3b+        .quad   0x5da8acdab8c63d5d+        .quad   0xc4b6664a3a70159f+        .quad   0x76194f0f0a904e14+        .quad   0xa5614c39a4096c13+        .quad   0x6cd0ff50979feced++        // 2^100 * 3 * G++        .quad   0xc0e067e78f4428ac+        .quad   0x14835ab0a61135e3+        .quad   0xf21d14f338062935+        .quad   0x6390a4c8df04849c+        .quad   0x7fecfabdb04ba18e+        .quad   0xd0fc7bfc3bddbcf7+        .quad   0xa41d486e057a131c+        .quad   0x641a4391f2223a61+        .quad   0xc5c6b95aa606a8db+        .quad   0x914b7f9eb06825f1+        .quad   0x2a731f6b44fc9eff+        .quad   0x30ddf38562705cfc++        // 2^100 * 4 * G++        .quad   0x4e3dcbdad1bff7f9+        .quad   0xc9118e8220645717+        .quad   0xbacccebc0f189d56+        .quad   0x1b4822e9d4467668+        .quad   0x33bef2bd68bcd52c+        .quad   0xc649dbb069482ef2+        .quad   0xb5b6ee0c41cb1aee+        .quad   0x5c294d270212a7e5+        .quad   0xab360a7f25563781+        .quad   0x2512228a480f7958+        .quad   0xc75d05276114b4e3+        .quad   0x222d9625d976fe2a++        // 2^100 * 5 * G++        .quad   0x1c717f85b372ace1+        .quad   0x81930e694638bf18+        .quad   0x239cad056bc08b58+        .quad   0x0b34271c87f8fff4+        .quad   0x0f94be7e0a344f85+        .quad   0xeb2faa8c87f22c38+        .quad   0x9ce1e75e4ee16f0f+        .quad   0x43e64e5418a08dea+        .quad   0x8155e2521a35ce63+        .quad   0xbe100d4df912028e+        .quad   0xbff80bf8a57ddcec+        .quad   0x57342dc96d6bc6e4++        // 2^100 * 6 * G++        .quad   0xefeef065c8ce5998+        .quad   0xbf029510b5cbeaa2+        .quad   0x8c64a10620b7c458+        .quad   0x35134fb231c24855+        .quad   0xf3c3bcb71e707bf6+        .quad   0x351d9b8c7291a762+        .quad   0x00502e6edad69a33+        .quad   0x522f521f1ec8807f+        .quad   0x272c1f46f9a3902b+        .quad   0xc91ba3b799657bcc+        .quad   0xae614b304f8a1c0e+        .quad   0x7afcaad70b99017b++        // 2^100 * 7 * G++        .quad   0xc25ded54a4b8be41+        .quad   0x902d13e11bb0e2dd+        .quad   0x41f43233cde82ab2+        .quad   0x1085faa5c3aae7cb+        .quad   0xa88141ecef842b6b+        .quad   0x55e7b14797abe6c5+        .quad   0x8c748f9703784ffe+        .quad   0x5b50a1f7afcd00b7+        .quad   0x9b840f66f1361315+        .quad   0x18462242701003e9+        .quad   0x65ed45fae4a25080+        .quad   0x0a2862393fda7320++        // 2^100 * 8 * G++        .quad   0x46ab13c8347cbc9d+        .quad   0x3849e8d499c12383+        .quad   0x4cea314087d64ac9+        .quad   0x1f354134b1a29ee7+        .quad   0x960e737b6ecb9d17+        .quad   0xfaf24948d67ceae1+        .quad   0x37e7a9b4d55e1b89+        .quad   0x5cb7173cb46c59eb+        .quad   0x4a89e68b82b7abf0+        .quad   0xf41cd9279ba6b7b9+        .quad   0x16e6c210e18d876f+        .quad   0x7cacdb0f7f1b09c6++        // 2^104 * 1 * G++        .quad   0x9062b2e0d91a78bc+        .quad   0x47c9889cc8509667+        .quad   0x9df54a66405070b8+        .quad   0x7369e6a92493a1bf+        .quad   0xe1014434dcc5caed+        .quad   0x47ed5d963c84fb33+        .quad   0x70019576ed86a0e7+        .quad   0x25b2697bd267f9e4+        .quad   0x9d673ffb13986864+        .quad   0x3ca5fbd9415dc7b8+        .quad   0xe04ecc3bdf273b5e+        .quad   0x1420683db54e4cd2++        // 2^104 * 2 * G++        .quad   0xb478bd1e249dd197+        .quad   0x620c35005e58c102+        .quad   0xfb02d32fccbaac5c+        .quad   0x60b63bebf508a72d+        .quad   0x34eebb6fc1cc5ad0+        .quad   0x6a1b0ce99646ac8b+        .quad   0xd3b0da49a66bde53+        .quad   0x31e83b4161d081c1+        .quad   0x97e8c7129e062b4f+        .quad   0x49e48f4f29320ad8+        .quad   0x5bece14b6f18683f+        .quad   0x55cf1eb62d550317++        // 2^104 * 3 * G++        .quad   0x5879101065c23d58+        .quad   0x8b9d086d5094819c+        .quad   0xe2402fa912c55fa7+        .quad   0x669a6564570891d4+        .quad   0x3076b5e37df58c52+        .quad   0xd73ab9dde799cc36+        .quad   0xbd831ce34913ee20+        .quad   0x1a56fbaa62ba0133+        .quad   0x943e6b505c9dc9ec+        .quad   0x302557bba77c371a+        .quad   0x9873ae5641347651+        .quad   0x13c4836799c58a5c++        // 2^104 * 4 * G++        .quad   0x423a5d465ab3e1b9+        .quad   0xfc13c187c7f13f61+        .quad   0x19f83664ecb5b9b6+        .quad   0x66f80c93a637b607+        .quad   0xc4dcfb6a5d8bd080+        .quad   0xdeebc4ec571a4842+        .quad   0xd4b2e883b8e55365+        .quad   0x50bdc87dc8e5b827+        .quad   0x606d37836edfe111+        .quad   0x32353e15f011abd9+        .quad   0x64b03ac325b73b96+        .quad   0x1dd56444725fd5ae++        // 2^104 * 5 * G++        .quad   0x8fa47ff83362127d+        .quad   0xbc9f6ac471cd7c15+        .quad   0x6e71454349220c8b+        .quad   0x0e645912219f732e+        .quad   0xc297e60008bac89a+        .quad   0x7d4cea11eae1c3e0+        .quad   0xf3e38be19fe7977c+        .quad   0x3a3a450f63a305cd+        .quad   0x078f2f31d8394627+        .quad   0x389d3183de94a510+        .quad   0xd1e36c6d17996f80+        .quad   0x318c8d9393a9a87b++        // 2^104 * 6 * G++        .quad   0xf2745d032afffe19+        .quad   0x0c9f3c497f24db66+        .quad   0xbc98d3e3ba8598ef+        .quad   0x224c7c679a1d5314+        .quad   0x5d669e29ab1dd398+        .quad   0xfc921658342d9e3b+        .quad   0x55851dfdf35973cd+        .quad   0x509a41c325950af6+        .quad   0xbdc06edca6f925e9+        .quad   0x793ef3f4641b1f33+        .quad   0x82ec12809d833e89+        .quad   0x05bff02328a11389++        // 2^104 * 7 * G++        .quad   0x3632137023cae00b+        .quad   0x544acf0ad1accf59+        .quad   0x96741049d21a1c88+        .quad   0x780b8cc3fa2a44a7+        .quad   0x6881a0dd0dc512e4+        .quad   0x4fe70dc844a5fafe+        .quad   0x1f748e6b8f4a5240+        .quad   0x576277cdee01a3ea+        .quad   0x1ef38abc234f305f+        .quad   0x9a577fbd1405de08+        .quad   0x5e82a51434e62a0d+        .quad   0x5ff418726271b7a1++        // 2^104 * 8 * G++        .quad   0x398e080c1789db9d+        .quad   0xa7602025f3e778f5+        .quad   0xfa98894c06bd035d+        .quad   0x106a03dc25a966be+        .quad   0xe5db47e813b69540+        .quad   0xf35d2a3b432610e1+        .quad   0xac1f26e938781276+        .quad   0x29d4db8ca0a0cb69+        .quad   0xd9ad0aaf333353d0+        .quad   0x38669da5acd309e5+        .quad   0x3c57658ac888f7f0+        .quad   0x4ab38a51052cbefa++        // 2^108 * 1 * G++        .quad   0xdfdacbee4324c0e9+        .quad   0x054442883f955bb7+        .quad   0xdef7aaa8ea31609f+        .quad   0x68aee70642287cff+        .quad   0xf68fe2e8809de054+        .quad   0xe3bc096a9c82bad1+        .quad   0x076353d40aadbf45+        .quad   0x7b9b1fb5dea1959e+        .quad   0xf01cc8f17471cc0c+        .quad   0x95242e37579082bb+        .quad   0x27776093d3e46b5f+        .quad   0x2d13d55a28bd85fb++        // 2^108 * 2 * G++        .quad   0xfac5d2065b35b8da+        .quad   0xa8da8a9a85624bb7+        .quad   0xccd2ca913d21cd0f+        .quad   0x6b8341ee8bf90d58+        .quad   0xbf019cce7aee7a52+        .quad   0xa8ded2b6e454ead3+        .quad   0x3c619f0b87a8bb19+        .quad   0x3619b5d7560916d8+        .quad   0x3579f26b0282c4b2+        .quad   0x64d592f24fafefae+        .quad   0xb7cded7b28c8c7c0+        .quad   0x6a927b6b7173a8d7++        // 2^108 * 3 * G++        .quad   0x1f6db24f986e4656+        .quad   0x1021c02ed1e9105b+        .quad   0xf8ff3fff2cc0a375+        .quad   0x1d2a6bf8c6c82592+        .quad   0x8d7040863ece88eb+        .quad   0xf0e307a980eec08c+        .quad   0xac2250610d788fda+        .quad   0x056d92a43a0d478d+        .quad   0x1b05a196fc3da5a1+        .quad   0x77d7a8c243b59ed0+        .quad   0x06da3d6297d17918+        .quad   0x66fbb494f12353f7++        // 2^108 * 4 * G++        .quad   0x751a50b9d85c0fb8+        .quad   0xd1afdc258bcf097b+        .quad   0x2f16a6a38309a969+        .quad   0x14ddff9ee5b00659+        .quad   0xd6d70996f12309d6+        .quad   0xdbfb2385e9c3d539+        .quad   0x46d602b0f7552411+        .quad   0x270a0b0557843e0c+        .quad   0x61ff0640a7862bcc+        .quad   0x81cac09a5f11abfe+        .quad   0x9047830455d12abb+        .quad   0x19a4bde1945ae873++        // 2^108 * 5 * G++        .quad   0x9b9f26f520a6200a+        .quad   0x64804443cf13eaf8+        .quad   0x8a63673f8631edd3+        .quad   0x72bbbce11ed39dc1+        .quad   0x40c709dec076c49f+        .quad   0x657bfaf27f3e53f6+        .quad   0x40662331eca042c4+        .quad   0x14b375487eb4df04+        .quad   0xae853c94ab66dc47+        .quad   0xeb62343edf762d6e+        .quad   0xf08e0e186fb2f7d1+        .quad   0x4f0b1c02700ab37a++        // 2^108 * 6 * G++        .quad   0xe1706787d81951fa+        .quad   0xa10a2c8eb290c77b+        .quad   0xe7382fa03ed66773+        .quad   0x0a4d84710bcc4b54+        .quad   0x79fd21ccc1b2e23f+        .quad   0x4ae7c281453df52a+        .quad   0xc8172ec9d151486b+        .quad   0x68abe9443e0a7534+        .quad   0xda12c6c407831dcb+        .quad   0x0da230d74d5c510d+        .quad   0x4ab1531e6bd404e1+        .quad   0x4106b166bcf440ef++        // 2^108 * 7 * G++        .quad   0x02e57a421cd23668+        .quad   0x4ad9fb5d0eaef6fd+        .quad   0x954e6727b1244480+        .quad   0x7f792f9d2699f331+        .quad   0xa485ccd539e4ecf2+        .quad   0x5aa3f3ad0555bab5+        .quad   0x145e3439937df82d+        .quad   0x1238b51e1214283f+        .quad   0x0b886b925fd4d924+        .quad   0x60906f7a3626a80d+        .quad   0xecd367b4b98abd12+        .quad   0x2876beb1def344cf++        // 2^108 * 8 * G++        .quad   0xdc84e93563144691+        .quad   0x632fe8a0d61f23f4+        .quad   0x4caa800612a9a8d5+        .quad   0x48f9dbfa0e9918d3+        .quad   0xd594b3333a8a85f8+        .quad   0x4ea37689e78d7d58+        .quad   0x73bf9f455e8e351f+        .quad   0x5507d7d2bc41ebb4+        .quad   0x1ceb2903299572fc+        .quad   0x7c8ccaa29502d0ee+        .quad   0x91bfa43411cce67b+        .quad   0x5784481964a831e7++        // 2^112 * 1 * G++        .quad   0xda7c2b256768d593+        .quad   0x98c1c0574422ca13+        .quad   0xf1a80bd5ca0ace1d+        .quad   0x29cdd1adc088a690+        .quad   0xd6cfd1ef5fddc09c+        .quad   0xe82b3efdf7575dce+        .quad   0x25d56b5d201634c2+        .quad   0x3041c6bb04ed2b9b+        .quad   0x0ff2f2f9d956e148+        .quad   0xade797759f356b2e+        .quad   0x1a4698bb5f6c025c+        .quad   0x104bbd6814049a7b++        // 2^112 * 2 * G++        .quad   0x51f0fd3168f1ed67+        .quad   0x2c811dcdd86f3bc2+        .quad   0x44dc5c4304d2f2de+        .quad   0x5be8cc57092a7149+        .quad   0xa95d9a5fd67ff163+        .quad   0xe92be69d4cc75681+        .quad   0xb7f8024cde20f257+        .quad   0x204f2a20fb072df5+        .quad   0xc8143b3d30ebb079+        .quad   0x7589155abd652e30+        .quad   0x653c3c318f6d5c31+        .quad   0x2570fb17c279161f++        // 2^112 * 3 * G++        .quad   0x3efa367f2cb61575+        .quad   0xf5f96f761cd6026c+        .quad   0xe8c7142a65b52562+        .quad   0x3dcb65ea53030acd+        .quad   0x192ea9550bb8245a+        .quad   0xc8e6fba88f9050d1+        .quad   0x7986ea2d88a4c935+        .quad   0x241c5f91de018668+        .quad   0x28d8172940de6caa+        .quad   0x8fbf2cf022d9733a+        .quad   0x16d7fcdd235b01d1+        .quad   0x08420edd5fcdf0e5++        // 2^112 * 4 * G++        .quad   0xcdff20ab8362fa4a+        .quad   0x57e118d4e21a3e6e+        .quad   0xe3179617fc39e62b+        .quad   0x0d9a53efbc1769fd+        .quad   0x0358c34e04f410ce+        .quad   0xb6135b5a276e0685+        .quad   0x5d9670c7ebb91521+        .quad   0x04d654f321db889c+        .quad   0x5e7dc116ddbdb5d5+        .quad   0x2954deb68da5dd2d+        .quad   0x1cb608173334a292+        .quad   0x4a7a4f2618991ad7++        // 2^112 * 5 * G++        .quad   0xf4a718025fb15f95+        .quad   0x3df65f346b5c1b8f+        .quad   0xcdfcf08500e01112+        .quad   0x11b50c4cddd31848+        .quad   0x24c3b291af372a4b+        .quad   0x93da8270718147f2+        .quad   0xdd84856486899ef2+        .quad   0x4a96314223e0ee33+        .quad   0xa6e8274408a4ffd6+        .quad   0x738e177e9c1576d9+        .quad   0x773348b63d02b3f2+        .quad   0x4f4bce4dce6bcc51++        // 2^112 * 6 * G++        .quad   0xa71fce5ae2242584+        .quad   0x26ea725692f58a9e+        .quad   0xd21a09d71cea3cf4+        .quad   0x73fcdd14b71c01e6+        .quad   0x30e2616ec49d0b6f+        .quad   0xe456718fcaec2317+        .quad   0x48eb409bf26b4fa6+        .quad   0x3042cee561595f37+        .quad   0x427e7079449bac41+        .quad   0x855ae36dbce2310a+        .quad   0x4cae76215f841a7c+        .quad   0x389e740c9a9ce1d6++        // 2^112 * 7 * G++        .quad   0x64fcb3ae34dcb9ce+        .quad   0x97500323e348d0ad+        .quad   0x45b3f07d62c6381b+        .quad   0x61545379465a6788+        .quad   0xc9bd78f6570eac28+        .quad   0xe55b0b3227919ce1+        .quad   0x65fc3eaba19b91ed+        .quad   0x25c425e5d6263690+        .quad   0x3f3e06a6f1d7de6e+        .quad   0x3ef976278e062308+        .quad   0x8c14f6264e8a6c77+        .quad   0x6539a08915484759++        // 2^112 * 8 * G++        .quad   0xe9d21f74c3d2f773+        .quad   0xc150544125c46845+        .quad   0x624e5ce8f9b99e33+        .quad   0x11c5e4aac5cd186c+        .quad   0xddc4dbd414bb4a19+        .quad   0x19b2bc3c98424f8e+        .quad   0x48a89fd736ca7169+        .quad   0x0f65320ef019bd90+        .quad   0xd486d1b1cafde0c6+        .quad   0x4f3fe6e3163b5181+        .quad   0x59a8af0dfaf2939a+        .quad   0x4cabc7bdec33072a++        // 2^116 * 1 * G++        .quad   0x16faa8fb532f7428+        .quad   0xdbd42ea046a4e272+        .quad   0x5337653b8b9ea480+        .quad   0x4065947223973f03+        .quad   0xf7c0a19c1a54a044+        .quad   0x4a1c5e2477bd9fbb+        .quad   0xa6e3ca115af22972+        .quad   0x1819bb953f2e9e0d+        .quad   0x498fbb795e042e84+        .quad   0x7d0dd89a7698b714+        .quad   0x8bfb0ba427fe6295+        .quad   0x36ba82e721200524++        // 2^116 * 2 * G++        .quad   0xd60ecbb74245ec41+        .quad   0xfd9be89e34348716+        .quad   0xc9240afee42284de+        .quad   0x4472f648d0531db4+        .quad   0xc8d69d0a57274ed5+        .quad   0x45ba803260804b17+        .quad   0xdf3cda102255dfac+        .quad   0x77d221232709b339+        .quad   0x498a6d7064ad94d8+        .quad   0xa5b5c8fd9af62263+        .quad   0x8ca8ed0545c141f4+        .quad   0x2c63bec3662d358c++        // 2^116 * 3 * G++        .quad   0x7fe60d8bea787955+        .quad   0xb9dc117eb5f401b7+        .quad   0x91c7c09a19355cce+        .quad   0x22692ef59442bedf+        .quad   0x9a518b3a8586f8bf+        .quad   0x9ee71af6cbb196f0+        .quad   0xaa0625e6a2385cf2+        .quad   0x1deb2176ddd7c8d1+        .quad   0x8563d19a2066cf6c+        .quad   0x401bfd8c4dcc7cd7+        .quad   0xd976a6becd0d8f62+        .quad   0x67cfd773a278b05e++        // 2^116 * 4 * G++        .quad   0x8dec31faef3ee475+        .quad   0x99dbff8a9e22fd92+        .quad   0x512d11594e26cab1+        .quad   0x0cde561eec4310b9+        .quad   0x2d5fa9855a4e586a+        .quad   0x65f8f7a449beab7e+        .quad   0xaa074dddf21d33d3+        .quad   0x185cba721bcb9dee+        .quad   0x93869da3f4e3cb41+        .quad   0xbf0392f540f7977e+        .quad   0x026204fcd0463b83+        .quad   0x3ec91a769eec6eed++        // 2^116 * 5 * G++        .quad   0x1e9df75bf78166ad+        .quad   0x4dfda838eb0cd7af+        .quad   0xba002ed8c1eaf988+        .quad   0x13fedb3e11f33cfc+        .quad   0x0fad2fb7b0a3402f+        .quad   0x46615ecbfb69f4a8+        .quad   0xf745bcc8c5f8eaa6+        .quad   0x7a5fa8794a94e896+        .quad   0x52958faa13cd67a1+        .quad   0x965ee0818bdbb517+        .quad   0x16e58daa2e8845b3+        .quad   0x357d397d5499da8f++        // 2^116 * 6 * G++        .quad   0x1ebfa05fb0bace6c+        .quad   0xc934620c1caf9a1e+        .quad   0xcc771cc41d82b61a+        .quad   0x2d94a16aa5f74fec+        .quad   0x481dacb4194bfbf8+        .quad   0x4d77e3f1bae58299+        .quad   0x1ef4612e7d1372a0+        .quad   0x3a8d867e70ff69e1+        .quad   0x6f58cd5d55aff958+        .quad   0xba3eaa5c75567721+        .quad   0x75c123999165227d+        .quad   0x69be1343c2f2b35e++        // 2^116 * 7 * G++        .quad   0x0e091d5ee197c92a+        .quad   0x4f51019f2945119f+        .quad   0x143679b9f034e99c+        .quad   0x7d88112e4d24c696+        .quad   0x82bbbdac684b8de3+        .quad   0xa2f4c7d03fca0718+        .quad   0x337f92fbe096aaa8+        .quad   0x200d4d8c63587376+        .quad   0x208aed4b4893b32b+        .quad   0x3efbf23ebe59b964+        .quad   0xd762deb0dba5e507+        .quad   0x69607bd681bd9d94++        // 2^116 * 8 * G++        .quad   0xf6be021068de1ce1+        .quad   0xe8d518e70edcbc1f+        .quad   0xe3effdd01b5505a5+        .quad   0x35f63353d3ec3fd0+        .quad   0x3b7f3bd49323a902+        .quad   0x7c21b5566b2c6e53+        .quad   0xe5ba8ff53a7852a7+        .quad   0x28bc77a5838ece00+        .quad   0x63ba78a8e25d8036+        .quad   0x63651e0094333490+        .quad   0x48d82f20288ce532+        .quad   0x3a31abfa36b57524++        // 2^120 * 1 * G++        .quad   0x239e9624089c0a2e+        .quad   0xc748c4c03afe4738+        .quad   0x17dbed2a764fa12a+        .quad   0x639b93f0321c8582+        .quad   0xc08f788f3f78d289+        .quad   0xfe30a72ca1404d9f+        .quad   0xf2778bfccf65cc9d+        .quad   0x7ee498165acb2021+        .quad   0x7bd508e39111a1c3+        .quad   0x2b2b90d480907489+        .quad   0xe7d2aec2ae72fd19+        .quad   0x0edf493c85b602a6++        // 2^120 * 2 * G++        .quad   0xaecc8158599b5a68+        .quad   0xea574f0febade20e+        .quad   0x4fe41d7422b67f07+        .quad   0x403b92e3019d4fb4+        .quad   0x6767c4d284764113+        .quad   0xa090403ff7f5f835+        .quad   0x1c8fcffacae6bede+        .quad   0x04c00c54d1dfa369+        .quad   0x4dc22f818b465cf8+        .quad   0x71a0f35a1480eff8+        .quad   0xaee8bfad04c7d657+        .quad   0x355bb12ab26176f4++        // 2^120 * 3 * G++        .quad   0xa71e64cc7493bbf4+        .quad   0xe5bd84d9eca3b0c3+        .quad   0x0a6bc50cfa05e785+        .quad   0x0f9b8132182ec312+        .quad   0xa301dac75a8c7318+        .quad   0xed90039db3ceaa11+        .quad   0x6f077cbf3bae3f2d+        .quad   0x7518eaf8e052ad8e+        .quad   0xa48859c41b7f6c32+        .quad   0x0f2d60bcf4383298+        .quad   0x1815a929c9b1d1d9+        .quad   0x47c3871bbb1755c4++        // 2^120 * 4 * G++        .quad   0x5144539771ec4f48+        .quad   0xf805b17dc98c5d6e+        .quad   0xf762c11a47c3c66b+        .quad   0x00b89b85764699dc+        .quad   0xfbe65d50c85066b0+        .quad   0x62ecc4b0b3a299b0+        .quad   0xe53754ea441ae8e0+        .quad   0x08fea02ce8d48d5f+        .quad   0x824ddd7668deead0+        .quad   0xc86445204b685d23+        .quad   0xb514cfcd5d89d665+        .quad   0x473829a74f75d537++        // 2^120 * 5 * G++        .quad   0x82d2da754679c418+        .quad   0xe63bd7d8b2618df0+        .quad   0x355eef24ac47eb0a+        .quad   0x2078684c4833c6b4+        .quad   0x23d9533aad3902c9+        .quad   0x64c2ddceef03588f+        .quad   0x15257390cfe12fb4+        .quad   0x6c668b4d44e4d390+        .quad   0x3b48cf217a78820c+        .quad   0xf76a0ab281273e97+        .quad   0xa96c65a78c8eed7b+        .quad   0x7411a6054f8a433f++        // 2^120 * 6 * G++        .quad   0x4d659d32b99dc86d+        .quad   0x044cdc75603af115+        .quad   0xb34c712cdcc2e488+        .quad   0x7c136574fb8134ff+        .quad   0x579ae53d18b175b4+        .quad   0x68713159f392a102+        .quad   0x8455ecba1eef35f5+        .quad   0x1ec9a872458c398f+        .quad   0xb8e6a4d400a2509b+        .quad   0x9b81d7020bc882b4+        .quad   0x57e7cc9bf1957561+        .quad   0x3add88a5c7cd6460++        // 2^120 * 7 * G++        .quad   0xab895770b635dcf2+        .quad   0x02dfef6cf66c1fbc+        .quad   0x85530268beb6d187+        .quad   0x249929fccc879e74+        .quad   0x85c298d459393046+        .quad   0x8f7e35985ff659ec+        .quad   0x1d2ca22af2f66e3a+        .quad   0x61ba1131a406a720+        .quad   0xa3d0a0f116959029+        .quad   0x023b6b6cba7ebd89+        .quad   0x7bf15a3e26783307+        .quad   0x5620310cbbd8ece7++        // 2^120 * 8 * G++        .quad   0x528993434934d643+        .quad   0xb9dbf806a51222f5+        .quad   0x8f6d878fc3f41c22+        .quad   0x37676a2a4d9d9730+        .quad   0x6646b5f477e285d6+        .quad   0x40e8ff676c8f6193+        .quad   0xa6ec7311abb594dd+        .quad   0x7ec846f3658cec4d+        .quad   0x9b5e8f3f1da22ec7+        .quad   0x130f1d776c01cd13+        .quad   0x214c8fcfa2989fb8+        .quad   0x6daaf723399b9dd5++        // 2^124 * 1 * G++        .quad   0x591e4a5610628564+        .quad   0x2a4bb87ca8b4df34+        .quad   0xde2a2572e7a38e43+        .quad   0x3cbdabd9fee5046e+        .quad   0x81aebbdd2cd13070+        .quad   0x962e4325f85a0e9e+        .quad   0xde9391aacadffecb+        .quad   0x53177fda52c230e6+        .quad   0xa7bc970650b9de79+        .quad   0x3d12a7fbc301b59b+        .quad   0x02652e68d36ae38c+        .quad   0x79d739835a6199dc++        // 2^124 * 2 * G++        .quad   0xd9354df64131c1bd+        .quad   0x758094a186ec5822+        .quad   0x4464ee12e459f3c2+        .quad   0x6c11fce4cb133282+        .quad   0x21c9d9920d591737+        .quad   0x9bea41d2e9b46cd6+        .quad   0xe20e84200d89bfca+        .quad   0x79d99f946eae5ff8+        .quad   0xf17b483568673205+        .quad   0x387deae83caad96c+        .quad   0x61b471fd56ffe386+        .quad   0x31741195b745a599++        // 2^124 * 3 * G++        .quad   0xe8d10190b77a360b+        .quad   0x99b983209995e702+        .quad   0xbd4fdff8fa0247aa+        .quad   0x2772e344e0d36a87+        .quad   0x17f8ba683b02a047+        .quad   0x50212096feefb6c8+        .quad   0x70139be21556cbe2+        .quad   0x203e44a11d98915b+        .quad   0xd6863eba37b9e39f+        .quad   0x105bc169723b5a23+        .quad   0x104f6459a65c0762+        .quad   0x567951295b4d38d4++        // 2^124 * 4 * G++        .quad   0x535fd60613037524+        .quad   0xe210adf6b0fbc26a+        .quad   0xac8d0a9b23e990ae+        .quad   0x47204d08d72fdbf9+        .quad   0x07242eb30d4b497f+        .quad   0x1ef96306b9bccc87+        .quad   0x37950934d8116f45+        .quad   0x05468d6201405b04+        .quad   0x00f565a9f93267de+        .quad   0xcecfd78dc0d58e8a+        .quad   0xa215e2dcf318e28e+        .quad   0x4599ee919b633352++        // 2^124 * 5 * G++        .quad   0xd3c220ca70e0e76b+        .quad   0xb12bea58ea9f3094+        .quad   0x294ddec8c3271282+        .quad   0x0c3539e1a1d1d028+        .quad   0xac746d6b861ae579+        .quad   0x31ab0650f6aea9dc+        .quad   0x241d661140256d4c+        .quad   0x2f485e853d21a5de+        .quad   0x329744839c0833f3+        .quad   0x6fe6257fd2abc484+        .quad   0x5327d1814b358817+        .quad   0x65712585893fe9bc++        // 2^124 * 6 * G++        .quad   0x9c102fb732a61161+        .quad   0xe48e10dd34d520a8+        .quad   0x365c63546f9a9176+        .quad   0x32f6fe4c046f6006+        .quad   0x81c29f1bd708ee3f+        .quad   0xddcb5a05ae6407d0+        .quad   0x97aec1d7d2a3eba7+        .quad   0x1590521a91d50831+        .quad   0x40a3a11ec7910acc+        .quad   0x9013dff8f16d27ae+        .quad   0x1a9720d8abb195d4+        .quad   0x1bb9fe452ea98463++        // 2^124 * 7 * G++        .quad   0xe9d1d950b3d54f9e+        .quad   0x2d5f9cbee00d33c1+        .quad   0x51c2c656a04fc6ac+        .quad   0x65c091ee3c1cbcc9+        .quad   0xcf5e6c95cc36747c+        .quad   0x294201536b0bc30d+        .quad   0x453ac67cee797af0+        .quad   0x5eae6ab32a8bb3c9+        .quad   0x7083661114f118ea+        .quad   0x2b37b87b94349cad+        .quad   0x7273f51cb4e99f40+        .quad   0x78a2a95823d75698++        // 2^124 * 8 * G++        .quad   0xa2b072e95c8c2ace+        .quad   0x69cffc96651e9c4b+        .quad   0x44328ef842e7b42b+        .quad   0x5dd996c122aadeb3+        .quad   0xb4f23c425ef83207+        .quad   0xabf894d3c9a934b5+        .quad   0xd0708c1339fd87f7+        .quad   0x1876789117166130+        .quad   0x925b5ef0670c507c+        .quad   0x819bc842b93c33bf+        .quad   0x10792e9a70dd003f+        .quad   0x59ad4b7a6e28dc74++        // 2^128 * 1 * G++        .quad   0x5f3a7562eb3dbe47+        .quad   0xf7ea38548ebda0b8+        .quad   0x00c3e53145747299+        .quad   0x1304e9e71627d551+        .quad   0x583b04bfacad8ea2+        .quad   0x29b743e8148be884+        .quad   0x2b1e583b0810c5db+        .quad   0x2b5449e58eb3bbaa+        .quad   0x789814d26adc9cfe+        .quad   0x3c1bab3f8b48dd0b+        .quad   0xda0fe1fff979c60a+        .quad   0x4468de2d7c2dd693++        // 2^128 * 2 * G++        .quad   0x51bb355e9419469e+        .quad   0x33e6dc4c23ddc754+        .quad   0x93a5b6d6447f9962+        .quad   0x6cce7c6ffb44bd63+        .quad   0x4b9ad8c6f86307ce+        .quad   0x21113531435d0c28+        .quad   0xd4a866c5657a772c+        .quad   0x5da6427e63247352+        .quad   0x1a94c688deac22ca+        .quad   0xb9066ef7bbae1ff8+        .quad   0x88ad8c388d59580f+        .quad   0x58f29abfe79f2ca8++        // 2^128 * 3 * G++        .quad   0xe90ecfab8de73e68+        .quad   0x54036f9f377e76a5+        .quad   0xf0495b0bbe015982+        .quad   0x577629c4a7f41e36+        .quad   0x4b5a64bf710ecdf6+        .quad   0xb14ce538462c293c+        .quad   0x3643d056d50b3ab9+        .quad   0x6af93724185b4870+        .quad   0x3220024509c6a888+        .quad   0xd2e036134b558973+        .quad   0x83e236233c33289f+        .quad   0x701f25bb0caec18f++        // 2^128 * 4 * G++        .quad   0xc3a8b0f8e4616ced+        .quad   0xf700660e9e25a87d+        .quad   0x61e3061ff4bca59c+        .quad   0x2e0c92bfbdc40be9+        .quad   0x9d18f6d97cbec113+        .quad   0x844a06e674bfdbe4+        .quad   0x20f5b522ac4e60d6+        .quad   0x720a5bc050955e51+        .quad   0x0c3f09439b805a35+        .quad   0xe84e8b376242abfc+        .quad   0x691417f35c229346+        .quad   0x0e9b9cbb144ef0ec++        // 2^128 * 5 * G++        .quad   0xfbbad48ffb5720ad+        .quad   0xee81916bdbf90d0e+        .quad   0xd4813152635543bf+        .quad   0x221104eb3f337bd8+        .quad   0x8dee9bd55db1beee+        .quad   0xc9c3ab370a723fb9+        .quad   0x44a8f1bf1c68d791+        .quad   0x366d44191cfd3cde+        .quad   0x9e3c1743f2bc8c14+        .quad   0x2eda26fcb5856c3b+        .quad   0xccb82f0e68a7fb97+        .quad   0x4167a4e6bc593244++        // 2^128 * 6 * G++        .quad   0x643b9d2876f62700+        .quad   0x5d1d9d400e7668eb+        .quad   0x1b4b430321fc0684+        .quad   0x7938bb7e2255246a+        .quad   0xc2be2665f8ce8fee+        .quad   0xe967ff14e880d62c+        .quad   0xf12e6e7e2f364eee+        .quad   0x34b33370cb7ed2f6+        .quad   0xcdc591ee8681d6cc+        .quad   0xce02109ced85a753+        .quad   0xed7485c158808883+        .quad   0x1176fc6e2dfe65e4++        // 2^128 * 7 * G++        .quad   0xb4af6cd05b9c619b+        .quad   0x2ddfc9f4b2a58480+        .quad   0x3d4fa502ebe94dc4+        .quad   0x08fc3a4c677d5f34+        .quad   0xdb90e28949770eb8+        .quad   0x98fbcc2aacf440a3+        .quad   0x21354ffeded7879b+        .quad   0x1f6a3e54f26906b6+        .quad   0x60a4c199d30734ea+        .quad   0x40c085b631165cd6+        .quad   0xe2333e23f7598295+        .quad   0x4f2fad0116b900d1++        // 2^128 * 8 * G++        .quad   0x44beb24194ae4e54+        .quad   0x5f541c511857ef6c+        .quad   0xa61e6b2d368d0498+        .quad   0x445484a4972ef7ab+        .quad   0x962cd91db73bb638+        .quad   0xe60577aafc129c08+        .quad   0x6f619b39f3b61689+        .quad   0x3451995f2944ee81+        .quad   0x9152fcd09fea7d7c+        .quad   0x4a816c94b0935cf6+        .quad   0x258e9aaa47285c40+        .quad   0x10b89ca6042893b7++        // 2^132 * 1 * G++        .quad   0x9b2a426e3b646025+        .quad   0x32127190385ce4cf+        .quad   0xa25cffc2dd6dea45+        .quad   0x06409010bea8de75+        .quad   0xd67cded679d34aa0+        .quad   0xcc0b9ec0cc4db39f+        .quad   0xa535a456e35d190f+        .quad   0x2e05d9eaf61f6fef+        .quad   0xc447901ad61beb59+        .quad   0x661f19bce5dc880a+        .quad   0x24685482b7ca6827+        .quad   0x293c778cefe07f26++        // 2^132 * 2 * G++        .quad   0x86809e7007069096+        .quad   0xaad75b15e4e50189+        .quad   0x07f35715a21a0147+        .quad   0x0487f3f112815d5e+        .quad   0x16c795d6a11ff200+        .quad   0xcb70d0e2b15815c9+        .quad   0x89f293209b5395b5+        .quad   0x50b8c2d031e47b4f+        .quad   0x48350c08068a4962+        .quad   0x6ffdd05351092c9a+        .quad   0x17af4f4aaf6fc8dd+        .quad   0x4b0553b53cdba58b++        // 2^132 * 3 * G++        .quad   0x9c65fcbe1b32ff79+        .quad   0xeb75ea9f03b50f9b+        .quad   0xfced2a6c6c07e606+        .quad   0x35106cd551717908+        .quad   0xbf05211b27c152d4+        .quad   0x5ec26849bd1af639+        .quad   0x5e0b2caa8e6fab98+        .quad   0x054c8bdd50bd0840+        .quad   0x38a0b12f1dcf073d+        .quad   0x4b60a8a3b7f6a276+        .quad   0xfed5ac25d3404f9a+        .quad   0x72e82d5e5505c229++        // 2^132 * 4 * G++        .quad   0x6b0b697ff0d844c8+        .quad   0xbb12f85cd979cb49+        .quad   0xd2a541c6c1da0f1f+        .quad   0x7b7c242958ce7211+        .quad   0x00d9cdfd69771d02+        .quad   0x410276cd6cfbf17e+        .quad   0x4c45306c1cb12ec7+        .quad   0x2857bf1627500861+        .quad   0x9f21903f0101689e+        .quad   0xd779dfd3bf861005+        .quad   0xa122ee5f3deb0f1b+        .quad   0x510df84b485a00d4++        // 2^132 * 5 * G++        .quad   0xa54133bb9277a1fa+        .quad   0x74ec3b6263991237+        .quad   0x1a3c54dc35d2f15a+        .quad   0x2d347144e482ba3a+        .quad   0x24b3c887c70ac15e+        .quad   0xb0f3a557fb81b732+        .quad   0x9b2cde2fe578cc1b+        .quad   0x4cf7ed0703b54f8e+        .quad   0x6bd47c6598fbee0f+        .quad   0x9e4733e2ab55be2d+        .quad   0x1093f624127610c5+        .quad   0x4e05e26ad0a1eaa4++        // 2^132 * 6 * G++        .quad   0xda9b6b624b531f20+        .quad   0x429a760e77509abb+        .quad   0xdbe9f522e823cb80+        .quad   0x618f1856880c8f82+        .quad   0x1833c773e18fe6c0+        .quad   0xe3c4711ad3c87265+        .quad   0x3bfd3c4f0116b283+        .quad   0x1955875eb4cd4db8+        .quad   0x6da6de8f0e399799+        .quad   0x7ad61aa440fda178+        .quad   0xb32cd8105e3563dd+        .quad   0x15f6beae2ae340ae++        // 2^132 * 7 * G++        .quad   0x862bcb0c31ec3a62+        .quad   0x810e2b451138f3c2+        .quad   0x788ec4b839dac2a4+        .quad   0x28f76867ae2a9281+        .quad   0xba9a0f7b9245e215+        .quad   0xf368612dd98c0dbb+        .quad   0x2e84e4cbf220b020+        .quad   0x6ba92fe962d90eda+        .quad   0x3e4df9655884e2aa+        .quad   0xbd62fbdbdbd465a5+        .quad   0xd7596caa0de9e524+        .quad   0x6e8042ccb2b1b3d7++        // 2^132 * 8 * G++        .quad   0xf10d3c29ce28ca6e+        .quad   0xbad34540fcb6093d+        .quad   0xe7426ed7a2ea2d3f+        .quad   0x08af9d4e4ff298b9+        .quad   0x1530653616521f7e+        .quad   0x660d06b896203dba+        .quad   0x2d3989bc545f0879+        .quad   0x4b5303af78ebd7b0+        .quad   0x72f8a6c3bebcbde8+        .quad   0x4f0fca4adc3a8e89+        .quad   0x6fa9d4e8c7bfdf7a+        .quad   0x0dcf2d679b624eb7++        // 2^136 * 1 * G++        .quad   0x3d5947499718289c+        .quad   0x12ebf8c524533f26+        .quad   0x0262bfcb14c3ef15+        .quad   0x20b878d577b7518e+        .quad   0x753941be5a45f06e+        .quad   0xd07caeed6d9c5f65+        .quad   0x11776b9c72ff51b6+        .quad   0x17d2d1d9ef0d4da9+        .quad   0x27f2af18073f3e6a+        .quad   0xfd3fe519d7521069+        .quad   0x22e3b72c3ca60022+        .quad   0x72214f63cc65c6a7++        // 2^136 * 2 * G++        .quad   0xb4e37f405307a693+        .quad   0xaba714d72f336795+        .quad   0xd6fbd0a773761099+        .quad   0x5fdf48c58171cbc9+        .quad   0x1d9db7b9f43b29c9+        .quad   0xd605824a4f518f75+        .quad   0xf2c072bd312f9dc4+        .quad   0x1f24ac855a1545b0+        .quad   0x24d608328e9505aa+        .quad   0x4748c1d10c1420ee+        .quad   0xc7ffe45c06fb25a2+        .quad   0x00ba739e2ae395e6++        // 2^136 * 3 * G++        .quad   0x592e98de5c8790d6+        .quad   0xe5bfb7d345c2a2df+        .quad   0x115a3b60f9b49922+        .quad   0x03283a3e67ad78f3+        .quad   0xae4426f5ea88bb26+        .quad   0x360679d984973bfb+        .quad   0x5c9f030c26694e50+        .quad   0x72297de7d518d226+        .quad   0x48241dc7be0cb939+        .quad   0x32f19b4d8b633080+        .quad   0xd3dfc90d02289308+        .quad   0x05e1296846271945++        // 2^136 * 4 * G++        .quad   0xba82eeb32d9c495a+        .quad   0xceefc8fcf12bb97c+        .quad   0xb02dabae93b5d1e0+        .quad   0x39c00c9c13698d9b+        .quad   0xadbfbbc8242c4550+        .quad   0xbcc80cecd03081d9+        .quad   0x843566a6f5c8df92+        .quad   0x78cf25d38258ce4c+        .quad   0x15ae6b8e31489d68+        .quad   0xaa851cab9c2bf087+        .quad   0xc9a75a97f04efa05+        .quad   0x006b52076b3ff832++        // 2^136 * 5 * G++        .quad   0x29e0cfe19d95781c+        .quad   0xb681df18966310e2+        .quad   0x57df39d370516b39+        .quad   0x4d57e3443bc76122+        .quad   0xf5cb7e16b9ce082d+        .quad   0x3407f14c417abc29+        .quad   0xd4b36bce2bf4a7ab+        .quad   0x7de2e9561a9f75ce+        .quad   0xde70d4f4b6a55ecb+        .quad   0x4801527f5d85db99+        .quad   0xdbc9c440d3ee9a81+        .quad   0x6b2a90af1a6029ed++        // 2^136 * 6 * G++        .quad   0x6923f4fc9ae61e97+        .quad   0x5735281de03f5fd1+        .quad   0xa764ae43e6edd12d+        .quad   0x5fd8f4e9d12d3e4a+        .quad   0x77ebf3245bb2d80a+        .quad   0xd8301b472fb9079b+        .quad   0xc647e6f24cee7333+        .quad   0x465812c8276c2109+        .quad   0x4d43beb22a1062d9+        .quad   0x7065fb753831dc16+        .quad   0x180d4a7bde2968d7+        .quad   0x05b32c2b1cb16790++        // 2^136 * 7 * G++        .quad   0xc8c05eccd24da8fd+        .quad   0xa1cf1aac05dfef83+        .quad   0xdbbeeff27df9cd61+        .quad   0x3b5556a37b471e99+        .quad   0xf7fca42c7ad58195+        .quad   0x3214286e4333f3cc+        .quad   0xb6c29d0d340b979d+        .quad   0x31771a48567307e1+        .quad   0x32b0c524e14dd482+        .quad   0xedb351541a2ba4b6+        .quad   0xa3d16048282b5af3+        .quad   0x4fc079d27a7336eb++        // 2^136 * 8 * G++        .quad   0x51c938b089bf2f7f+        .quad   0x2497bd6502dfe9a7+        .quad   0xffffc09c7880e453+        .quad   0x124567cecaf98e92+        .quad   0xdc348b440c86c50d+        .quad   0x1337cbc9cc94e651+        .quad   0x6422f74d643e3cb9+        .quad   0x241170c2bae3cd08+        .quad   0x3ff9ab860ac473b4+        .quad   0xf0911dee0113e435+        .quad   0x4ae75060ebc6c4af+        .quad   0x3f8612966c87000d++        // 2^140 * 1 * G++        .quad   0x0c9c5303f7957be4+        .quad   0xa3c31a20e085c145+        .quad   0xb0721d71d0850050+        .quad   0x0aba390eab0bf2da+        .quad   0x529fdffe638c7bf3+        .quad   0xdf2b9e60388b4995+        .quad   0xe027b34f1bad0249+        .quad   0x7bc92fc9b9fa74ed+        .quad   0x9f97ef2e801ad9f9+        .quad   0x83697d5479afda3a+        .quad   0xe906b3ffbd596b50+        .quad   0x02672b37dd3fb8e0++        // 2^140 * 2 * G++        .quad   0x48b2ca8b260885e4+        .quad   0xa4286bec82b34c1c+        .quad   0x937e1a2617f58f74+        .quad   0x741d1fcbab2ca2a5+        .quad   0xee9ba729398ca7f5+        .quad   0xeb9ca6257a4849db+        .quad   0x29eb29ce7ec544e1+        .quad   0x232ca21ef736e2c8+        .quad   0xbf61423d253fcb17+        .quad   0x08803ceafa39eb14+        .quad   0xf18602df9851c7af+        .quad   0x0400f3a049e3414b++        // 2^140 * 3 * G++        .quad   0xabce0476ba61c55b+        .quad   0x36a3d6d7c4d39716+        .quad   0x6eb259d5e8d82d09+        .quad   0x0c9176e984d756fb+        .quad   0x2efba412a06e7b06+        .quad   0x146785452c8d2560+        .quad   0xdf9713ebd67a91c7+        .quad   0x32830ac7157eadf3+        .quad   0x0e782a7ab73769e8+        .quad   0x04a05d7875b18e2c+        .quad   0x29525226ebcceae1+        .quad   0x0d794f8383eba820++        // 2^140 * 4 * G++        .quad   0xff35f5cb9e1516f4+        .quad   0xee805bcf648aae45+        .quad   0xf0d73c2bb93a9ef3+        .quad   0x097b0bf22092a6c2+        .quad   0x7be44ce7a7a2e1ac+        .quad   0x411fd93efad1b8b7+        .quad   0x1734a1d70d5f7c9b+        .quad   0x0d6592233127db16+        .quad   0xc48bab1521a9d733+        .quad   0xa6c2eaead61abb25+        .quad   0x625c6c1cc6cb4305+        .quad   0x7fc90fea93eb3a67++        // 2^140 * 5 * G++        .quad   0x0408f1fe1f5c5926+        .quad   0x1a8f2f5e3b258bf4+        .quad   0x40a951a2fdc71669+        .quad   0x6598ee93c98b577e+        .quad   0xc527deb59c7cb23d+        .quad   0x955391695328404e+        .quad   0xd64392817ccf2c7a+        .quad   0x6ce97dabf7d8fa11+        .quad   0x25b5a8e50ef7c48f+        .quad   0xeb6034116f2ce532+        .quad   0xc5e75173e53de537+        .quad   0x73119fa08c12bb03++        // 2^140 * 6 * G++        .quad   0xed30129453f1a4cb+        .quad   0xbce621c9c8f53787+        .quad   0xfacb2b1338bee7b9+        .quad   0x3025798a9ea8428c+        .quad   0x7845b94d21f4774d+        .quad   0xbf62f16c7897b727+        .quad   0x671857c03c56522b+        .quad   0x3cd6a85295621212+        .quad   0x3fecde923aeca999+        .quad   0xbdaa5b0062e8c12f+        .quad   0x67b99dfc96988ade+        .quad   0x3f52c02852661036++        // 2^140 * 7 * G++        .quad   0xffeaa48e2a1351c6+        .quad   0x28624754fa7f53d7+        .quad   0x0b5ba9e57582ddf1+        .quad   0x60c0104ba696ac59+        .quad   0x9258bf99eec416c6+        .quad   0xac8a5017a9d2f671+        .quad   0x629549ab16dea4ab+        .quad   0x05d0e85c99091569+        .quad   0x051de020de9cbe97+        .quad   0xfa07fc56b50bcf74+        .quad   0x378cec9f0f11df65+        .quad   0x36853c69ab96de4d++        // 2^140 * 8 * G++        .quad   0x36d9b8de78f39b2d+        .quad   0x7f42ed71a847b9ec+        .quad   0x241cd1d679bd3fde+        .quad   0x6a704fec92fbce6b+        .quad   0x4433c0b0fac5e7be+        .quad   0x724bae854c08dcbe+        .quad   0xf1f24cc446978f9b+        .quad   0x4a0aff6d62825fc8+        .quad   0xe917fb9e61095301+        .quad   0xc102df9402a092f8+        .quad   0xbf09e2f5fa66190b+        .quad   0x681109bee0dcfe37++        // 2^144 * 1 * G++        .quad   0x559a0cc9782a0dde+        .quad   0x551dcdb2ea718385+        .quad   0x7f62865b31ef238c+        .quad   0x504aa7767973613d+        .quad   0x9c18fcfa36048d13+        .quad   0x29159db373899ddd+        .quad   0xdc9f350b9f92d0aa+        .quad   0x26f57eee878a19d4+        .quad   0x0cab2cd55687efb1+        .quad   0x5180d162247af17b+        .quad   0x85c15a344f5a2467+        .quad   0x4041943d9dba3069++        // 2^144 * 2 * G++        .quad   0xc3c0eeba43ebcc96+        .quad   0x8d749c9c26ea9caf+        .quad   0xd9fa95ee1c77ccc6+        .quad   0x1420a1d97684340f+        .quad   0x4b217743a26caadd+        .quad   0x47a6b424648ab7ce+        .quad   0xcb1d4f7a03fbc9e3+        .quad   0x12d931429800d019+        .quad   0x00c67799d337594f+        .quad   0x5e3c5140b23aa47b+        .quad   0x44182854e35ff395+        .quad   0x1b4f92314359a012++        // 2^144 * 3 * G++        .quad   0x3e5c109d89150951+        .quad   0x39cefa912de9696a+        .quad   0x20eae43f975f3020+        .quad   0x239b572a7f132dae+        .quad   0x33cf3030a49866b1+        .quad   0x251f73d2215f4859+        .quad   0xab82aa4051def4f6+        .quad   0x5ff191d56f9a23f6+        .quad   0x819ed433ac2d9068+        .quad   0x2883ab795fc98523+        .quad   0xef4572805593eb3d+        .quad   0x020c526a758f36cb++        // 2^144 * 4 * G++        .quad   0x779834f89ed8dbbc+        .quad   0xc8f2aaf9dc7ca46c+        .quad   0xa9524cdca3e1b074+        .quad   0x02aacc4615313877+        .quad   0xe931ef59f042cc89+        .quad   0x2c589c9d8e124bb6+        .quad   0xadc8e18aaec75997+        .quad   0x452cfe0a5602c50c+        .quad   0x86a0f7a0647877df+        .quad   0xbbc464270e607c9f+        .quad   0xab17ea25f1fb11c9+        .quad   0x4cfb7d7b304b877b++        // 2^144 * 5 * G++        .quad   0x72b43d6cb89b75fe+        .quad   0x54c694d99c6adc80+        .quad   0xb8c3aa373ee34c9f+        .quad   0x14b4622b39075364+        .quad   0xe28699c29789ef12+        .quad   0x2b6ecd71df57190d+        .quad   0xc343c857ecc970d0+        .quad   0x5b1d4cbc434d3ac5+        .quad   0xb6fb2615cc0a9f26+        .quad   0x3a4f0e2bb88dcce5+        .quad   0x1301498b3369a705+        .quad   0x2f98f71258592dd1++        // 2^144 * 6 * G++        .quad   0x0c94a74cb50f9e56+        .quad   0x5b1ff4a98e8e1320+        .quad   0x9a2acc2182300f67+        .quad   0x3a6ae249d806aaf9+        .quad   0x2e12ae444f54a701+        .quad   0xfcfe3ef0a9cbd7de+        .quad   0xcebf890d75835de0+        .quad   0x1d8062e9e7614554+        .quad   0x657ada85a9907c5a+        .quad   0x1a0ea8b591b90f62+        .quad   0x8d0e1dfbdf34b4e9+        .quad   0x298b8ce8aef25ff3++        // 2^144 * 7 * G++        .quad   0x2a927953eff70cb2+        .quad   0x4b89c92a79157076+        .quad   0x9418457a30a7cf6a+        .quad   0x34b8a8404d5ce485+        .quad   0x837a72ea0a2165de+        .quad   0x3fab07b40bcf79f6+        .quad   0x521636c77738ae70+        .quad   0x6ba6271803a7d7dc+        .quad   0xc26eecb583693335+        .quad   0xd5a813df63b5fefd+        .quad   0xa293aa9aa4b22573+        .quad   0x71d62bdd465e1c6a++        // 2^144 * 8 * G++        .quad   0x6533cc28d378df80+        .quad   0xf6db43790a0fa4b4+        .quad   0xe3645ff9f701da5a+        .quad   0x74d5f317f3172ba4+        .quad   0xcd2db5dab1f75ef5+        .quad   0xd77f95cf16b065f5+        .quad   0x14571fea3f49f085+        .quad   0x1c333621262b2b3d+        .quad   0xa86fe55467d9ca81+        .quad   0x398b7c752b298c37+        .quad   0xda6d0892e3ac623b+        .quad   0x4aebcc4547e9d98c++        // 2^148 * 1 * G++        .quad   0x53175a7205d21a77+        .quad   0xb0c04422d3b934d4+        .quad   0xadd9f24bdd5deadc+        .quad   0x074f46e69f10ff8c+        .quad   0x0de9b204a059a445+        .quad   0xe15cb4aa4b17ad0f+        .quad   0xe1bbec521f79c557+        .quad   0x2633f1b9d071081b+        .quad   0xc1fb4177018b9910+        .quad   0xa6ea20dc6c0fe140+        .quad   0xd661f3e74354c6ff+        .quad   0x5ecb72e6f1a3407a++        // 2^148 * 2 * G++        .quad   0xa515a31b2259fb4e+        .quad   0x0960f3972bcac52f+        .quad   0xedb52fec8d3454cb+        .quad   0x382e2720c476c019+        .quad   0xfeeae106e8e86997+        .quad   0x9863337f98d09383+        .quad   0x9470480eaa06ebef+        .quad   0x038b6898d4c5c2d0+        .quad   0xf391c51d8ace50a6+        .quad   0x3142d0b9ae2d2948+        .quad   0xdb4d5a1a7f24ca80+        .quad   0x21aeba8b59250ea8++        // 2^148 * 3 * G++        .quad   0x24f13b34cf405530+        .quad   0x3c44ea4a43088af7+        .quad   0x5dd5c5170006a482+        .quad   0x118eb8f8890b086d+        .quad   0x53853600f0087f23+        .quad   0x4c461879da7d5784+        .quad   0x6af303deb41f6860+        .quad   0x0a3c16c5c27c18ed+        .quad   0x17e49c17cc947f3d+        .quad   0xccc6eda6aac1d27b+        .quad   0xdf6092ceb0f08e56+        .quad   0x4909b3e22c67c36b++        // 2^148 * 4 * G++        .quad   0x9c9c85ea63fe2e89+        .quad   0xbe1baf910e9412ec+        .quad   0x8f7baa8a86fbfe7b+        .quad   0x0fb17f9fef968b6c+        .quad   0x59a16676706ff64e+        .quad   0x10b953dd0d86a53d+        .quad   0x5848e1e6ce5c0b96+        .quad   0x2d8b78e712780c68+        .quad   0x79d5c62eafc3902b+        .quad   0x773a215289e80728+        .quad   0xc38ae640e10120b9+        .quad   0x09ae23717b2b1a6d++        // 2^148 * 5 * G++        .quad   0xbb6a192a4e4d083c+        .quad   0x34ace0630029e192+        .quad   0x98245a59aafabaeb+        .quad   0x6d9c8a9ada97faac+        .quad   0x10ab8fa1ad32b1d0+        .quad   0xe9aced1be2778b24+        .quad   0xa8856bc0373de90f+        .quad   0x66f35ddddda53996+        .quad   0xd27d9afb24997323+        .quad   0x1bb7e07ef6f01d2e+        .quad   0x2ba7472df52ecc7f+        .quad   0x03019b4f646f9dc8++        // 2^148 * 6 * G++        .quad   0x04a186b5565345cd+        .quad   0xeee76610bcc4116a+        .quad   0x689c73b478fb2a45+        .quad   0x387dcbff65697512+        .quad   0xaf09b214e6b3dc6b+        .quad   0x3f7573b5ad7d2f65+        .quad   0xd019d988100a23b0+        .quad   0x392b63a58b5c35f7+        .quad   0x4093addc9c07c205+        .quad   0xc565be15f532c37e+        .quad   0x63dbecfd1583402a+        .quad   0x61722b4aef2e032e++        // 2^148 * 7 * G++        .quad   0x0012aafeecbd47af+        .quad   0x55a266fb1cd46309+        .quad   0xf203eb680967c72c+        .quad   0x39633944ca3c1429+        .quad   0xd6b07a5581cb0e3c+        .quad   0x290ff006d9444969+        .quad   0x08680b6a16dcda1f+        .quad   0x5568d2b75a06de59+        .quad   0x8d0cb88c1b37cfe1+        .quad   0x05b6a5a3053818f3+        .quad   0xf2e9bc04b787d959+        .quad   0x6beba1249add7f64++        // 2^148 * 8 * G++        .quad   0x1d06005ca5b1b143+        .quad   0x6d4c6bb87fd1cda2+        .quad   0x6ef5967653fcffe7+        .quad   0x097c29e8c1ce1ea5+        .quad   0x5c3cecb943f5a53b+        .quad   0x9cc9a61d06c08df2+        .quad   0xcfba639a85895447+        .quad   0x5a845ae80df09fd5+        .quad   0x4ce97dbe5deb94ca+        .quad   0x38d0a4388c709c48+        .quad   0xc43eced4a169d097+        .quad   0x0a1249fff7e587c3++        // 2^152 * 1 * G++        .quad   0x12f0071b276d01c9+        .quad   0xe7b8bac586c48c70+        .quad   0x5308129b71d6fba9+        .quad   0x5d88fbf95a3db792+        .quad   0x0b408d9e7354b610+        .quad   0x806b32535ba85b6e+        .quad   0xdbe63a034a58a207+        .quad   0x173bd9ddc9a1df2c+        .quad   0x2b500f1efe5872df+        .quad   0x58d6582ed43918c1+        .quad   0xe6ed278ec9673ae0+        .quad   0x06e1cd13b19ea319++        // 2^152 * 2 * G++        .quad   0x40d0ad516f166f23+        .quad   0x118e32931fab6abe+        .quad   0x3fe35e14a04d088e+        .quad   0x3080603526e16266+        .quad   0x472baf629e5b0353+        .quad   0x3baa0b90278d0447+        .quad   0x0c785f469643bf27+        .quad   0x7f3a6a1a8d837b13+        .quad   0xf7e644395d3d800b+        .quad   0x95a8d555c901edf6+        .quad   0x68cd7830592c6339+        .quad   0x30d0fded2e51307e++        // 2^152 * 3 * G++        .quad   0xe0594d1af21233b3+        .quad   0x1bdbe78ef0cc4d9c+        .quad   0x6965187f8f499a77+        .quad   0x0a9214202c099868+        .quad   0x9cb4971e68b84750+        .quad   0xa09572296664bbcf+        .quad   0x5c8de72672fa412b+        .quad   0x4615084351c589d9+        .quad   0xbc9019c0aeb9a02e+        .quad   0x55c7110d16034cae+        .quad   0x0e6df501659932ec+        .quad   0x3bca0d2895ca5dfe++        // 2^152 * 4 * G++        .quad   0x40f031bc3c5d62a4+        .quad   0x19fc8b3ecff07a60+        .quad   0x98183da2130fb545+        .quad   0x5631deddae8f13cd+        .quad   0x9c688eb69ecc01bf+        .quad   0xf0bc83ada644896f+        .quad   0xca2d955f5f7a9fe2+        .quad   0x4ea8b4038df28241+        .quad   0x2aed460af1cad202+        .quad   0x46305305a48cee83+        .quad   0x9121774549f11a5f+        .quad   0x24ce0930542ca463++        // 2^152 * 5 * G++        .quad   0x1fe890f5fd06c106+        .quad   0xb5c468355d8810f2+        .quad   0x827808fe6e8caf3e+        .quad   0x41d4e3c28a06d74b+        .quad   0x3fcfa155fdf30b85+        .quad   0xd2f7168e36372ea4+        .quad   0xb2e064de6492f844+        .quad   0x549928a7324f4280+        .quad   0xf26e32a763ee1a2e+        .quad   0xae91e4b7d25ffdea+        .quad   0xbc3bd33bd17f4d69+        .quad   0x491b66dec0dcff6a++        // 2^152 * 6 * G++        .quad   0x98f5b13dc7ea32a7+        .quad   0xe3d5f8cc7e16db98+        .quad   0xac0abf52cbf8d947+        .quad   0x08f338d0c85ee4ac+        .quad   0x75f04a8ed0da64a1+        .quad   0xed222caf67e2284b+        .quad   0x8234a3791f7b7ba4+        .quad   0x4cf6b8b0b7018b67+        .quad   0xc383a821991a73bd+        .quad   0xab27bc01df320c7a+        .quad   0xc13d331b84777063+        .quad   0x530d4a82eb078a99++        // 2^152 * 7 * G++        .quad   0x004c3630e1f94825+        .quad   0x7e2d78268cab535a+        .quad   0xc7482323cc84ff8b+        .quad   0x65ea753f101770b9+        .quad   0x6d6973456c9abf9e+        .quad   0x257fb2fc4900a880+        .quad   0x2bacf412c8cfb850+        .quad   0x0db3e7e00cbfbd5b+        .quad   0x3d66fc3ee2096363+        .quad   0x81d62c7f61b5cb6b+        .quad   0x0fbe044213443b1a+        .quad   0x02a4ec1921e1a1db++        // 2^152 * 8 * G++        .quad   0x5ce6259a3b24b8a2+        .quad   0xb8577acc45afa0b8+        .quad   0xcccbe6e88ba07037+        .quad   0x3d143c51127809bf+        .quad   0xf5c86162f1cf795f+        .quad   0x118c861926ee57f2+        .quad   0x172124851c063578+        .quad   0x36d12b5dec067fcf+        .quad   0x126d279179154557+        .quad   0xd5e48f5cfc783a0a+        .quad   0x36bdb6e8df179bac+        .quad   0x2ef517885ba82859++        // 2^156 * 1 * G++        .quad   0x88bd438cd11e0d4a+        .quad   0x30cb610d43ccf308+        .quad   0xe09a0e3791937bcc+        .quad   0x4559135b25b1720c+        .quad   0x1ea436837c6da1e9+        .quad   0xf9c189af1fb9bdbe+        .quad   0x303001fcce5dd155+        .quad   0x28a7c99ebc57be52+        .quad   0xb8fd9399e8d19e9d+        .quad   0x908191cb962423ff+        .quad   0xb2b948d747c742a3+        .quad   0x37f33226d7fb44c4++        // 2^156 * 2 * G++        .quad   0x0dae8767b55f6e08+        .quad   0x4a43b3b35b203a02+        .quad   0xe3725a6e80af8c79+        .quad   0x0f7a7fd1705fa7a3+        .quad   0x33912553c821b11d+        .quad   0x66ed42c241e301df+        .quad   0x066fcc11104222fd+        .quad   0x307a3b41c192168f+        .quad   0x8eeb5d076eb55ce0+        .quad   0x2fc536bfaa0d925a+        .quad   0xbe81830fdcb6c6e8+        .quad   0x556c7045827baf52++        // 2^156 * 3 * G++        .quad   0x8e2b517302e9d8b7+        .quad   0xe3e52269248714e8+        .quad   0xbd4fbd774ca960b5+        .quad   0x6f4b4199c5ecada9+        .quad   0xb94b90022bf44406+        .quad   0xabd4237eff90b534+        .quad   0x7600a960faf86d3a+        .quad   0x2f45abdac2322ee3+        .quad   0x61af4912c8ef8a6a+        .quad   0xe58fa4fe43fb6e5e+        .quad   0xb5afcc5d6fd427cf+        .quad   0x6a5393281e1e11eb++        // 2^156 * 4 * G++        .quad   0xf3da5139a5d1ee89+        .quad   0x8145457cff936988+        .quad   0x3f622fed00e188c4+        .quad   0x0f513815db8b5a3d+        .quad   0x0fff04fe149443cf+        .quad   0x53cac6d9865cddd7+        .quad   0x31385b03531ed1b7+        .quad   0x5846a27cacd1039d+        .quad   0x4ff5cdac1eb08717+        .quad   0x67e8b29590f2e9bc+        .quad   0x44093b5e237afa99+        .quad   0x0d414bed8708b8b2++        // 2^156 * 5 * G++        .quad   0xcfb68265fd0e75f6+        .quad   0xe45b3e28bb90e707+        .quad   0x7242a8de9ff92c7a+        .quad   0x685b3201933202dd+        .quad   0x81886a92294ac9e8+        .quad   0x23162b45d55547be+        .quad   0x94cfbc4403715983+        .quad   0x50eb8fdb134bc401+        .quad   0xc0b73ec6d6b330cd+        .quad   0x84e44807132faff1+        .quad   0x732b7352c4a5dee1+        .quad   0x5d7c7cf1aa7cd2d2++        // 2^156 * 6 * G++        .quad   0xaf3b46bf7a4aafa2+        .quad   0xb78705ec4d40d411+        .quad   0x114f0c6aca7c15e3+        .quad   0x3f364faaa9489d4d+        .quad   0x33d1013e9b73a562+        .quad   0x925cef5748ec26e1+        .quad   0xa7fce614dd468058+        .quad   0x78b0fad41e9aa438+        .quad   0xbf56a431ed05b488+        .quad   0xa533e66c9c495c7e+        .quad   0xe8652baf87f3651a+        .quad   0x0241800059d66c33++        // 2^156 * 7 * G++        .quad   0xceb077fea37a5be4+        .quad   0xdb642f02e5a5eeb7+        .quad   0xc2e6d0c5471270b8+        .quad   0x4771b65538e4529c+        .quad   0x28350c7dcf38ea01+        .quad   0x7c6cdbc0b2917ab6+        .quad   0xace7cfbe857082f7+        .quad   0x4d2845aba2d9a1e0+        .quad   0xbb537fe0447070de+        .quad   0xcba744436dd557df+        .quad   0xd3b5a3473600dbcb+        .quad   0x4aeabbe6f9ffd7f8++        // 2^156 * 8 * G++        .quad   0x4630119e40d8f78c+        .quad   0xa01a9bc53c710e11+        .quad   0x486d2b258910dd79+        .quad   0x1e6c47b3db0324e5+        .quad   0x6a2134bcc4a9c8f2+        .quad   0xfbf8fd1c8ace2e37+        .quad   0x000ae3049911a0ba+        .quad   0x046e3a616bc89b9e+        .quad   0x14e65442f03906be+        .quad   0x4a019d54e362be2a+        .quad   0x68ccdfec8dc230c7+        .quad   0x7cfb7e3faf6b861c++        // 2^160 * 1 * G++        .quad   0x4637974e8c58aedc+        .quad   0xb9ef22fbabf041a4+        .quad   0xe185d956e980718a+        .quad   0x2f1b78fab143a8a6+        .quad   0x96eebffb305b2f51+        .quad   0xd3f938ad889596b8+        .quad   0xf0f52dc746d5dd25+        .quad   0x57968290bb3a0095+        .quad   0xf71ab8430a20e101+        .quad   0xf393658d24f0ec47+        .quad   0xcf7509a86ee2eed1+        .quad   0x7dc43e35dc2aa3e1++        // 2^160 * 2 * G++        .quad   0x85966665887dd9c3+        .quad   0xc90f9b314bb05355+        .quad   0xc6e08df8ef2079b1+        .quad   0x7ef72016758cc12f+        .quad   0x5a782a5c273e9718+        .quad   0x3576c6995e4efd94+        .quad   0x0f2ed8051f237d3e+        .quad   0x044fb81d82d50a99+        .quad   0xc1df18c5a907e3d9+        .quad   0x57b3371dce4c6359+        .quad   0xca704534b201bb49+        .quad   0x7f79823f9c30dd2e++        // 2^160 * 3 * G++        .quad   0x8334d239a3b513e8+        .quad   0xc13670d4b91fa8d8+        .quad   0x12b54136f590bd33+        .quad   0x0a4e0373d784d9b4+        .quad   0x6a9c1ff068f587ba+        .quad   0x0827894e0050c8de+        .quad   0x3cbf99557ded5be7+        .quad   0x64a9b0431c06d6f0+        .quad   0x2eb3d6a15b7d2919+        .quad   0xb0b4f6a0d53a8235+        .quad   0x7156ce4389a45d47+        .quad   0x071a7d0ace18346c++        // 2^160 * 4 * G++        .quad   0xd3072daac887ba0b+        .quad   0x01262905bfa562ee+        .quad   0xcf543002c0ef768b+        .quad   0x2c3bcc7146ea7e9c+        .quad   0xcc0c355220e14431+        .quad   0x0d65950709b15141+        .quad   0x9af5621b209d5f36+        .quad   0x7c69bcf7617755d3+        .quad   0x07f0d7eb04e8295f+        .quad   0x10db18252f50f37d+        .quad   0xe951a9a3171798d7+        .quad   0x6f5a9a7322aca51d++        // 2^160 * 5 * G++        .quad   0x8ba1000c2f41c6c5+        .quad   0xc49f79c10cfefb9b+        .quad   0x4efa47703cc51c9f+        .quad   0x494e21a2e147afca+        .quad   0xe729d4eba3d944be+        .quad   0x8d9e09408078af9e+        .quad   0x4525567a47869c03+        .quad   0x02ab9680ee8d3b24+        .quad   0xefa48a85dde50d9a+        .quad   0x219a224e0fb9a249+        .quad   0xfa091f1dd91ef6d9+        .quad   0x6b5d76cbea46bb34++        // 2^160 * 6 * G++        .quad   0x8857556cec0cd994+        .quad   0x6472dc6f5cd01dba+        .quad   0xaf0169148f42b477+        .quad   0x0ae333f685277354+        .quad   0xe0f941171e782522+        .quad   0xf1e6ae74036936d3+        .quad   0x408b3ea2d0fcc746+        .quad   0x16fb869c03dd313e+        .quad   0x288e199733b60962+        .quad   0x24fc72b4d8abe133+        .quad   0x4811f7ed0991d03e+        .quad   0x3f81e38b8f70d075++        // 2^160 * 7 * G++        .quad   0x7f910fcc7ed9affe+        .quad   0x545cb8a12465874b+        .quad   0xa8397ed24b0c4704+        .quad   0x50510fc104f50993+        .quad   0x0adb7f355f17c824+        .quad   0x74b923c3d74299a4+        .quad   0xd57c3e8bcbf8eaf7+        .quad   0x0ad3e2d34cdedc3d+        .quad   0x6f0c0fc5336e249d+        .quad   0x745ede19c331cfd9+        .quad   0xf2d6fd0009eefe1c+        .quad   0x127c158bf0fa1ebe++        // 2^160 * 8 * G++        .quad   0xf6197c422e9879a2+        .quad   0xa44addd452ca3647+        .quad   0x9b413fc14b4eaccb+        .quad   0x354ef87d07ef4f68+        .quad   0xdea28fc4ae51b974+        .quad   0x1d9973d3744dfe96+        .quad   0x6240680b873848a8+        .quad   0x4ed82479d167df95+        .quad   0xfee3b52260c5d975+        .quad   0x50352efceb41b0b8+        .quad   0x8808ac30a9f6653c+        .quad   0x302d92d20539236d++        // 2^164 * 1 * G++        .quad   0x4c59023fcb3efb7c+        .quad   0x6c2fcb99c63c2a94+        .quad   0xba4190e2c3c7e084+        .quad   0x0e545daea51874d9+        .quad   0x957b8b8b0df53c30+        .quad   0x2a1c770a8e60f098+        .quad   0xbbc7a670345796de+        .quad   0x22a48f9a90c99bc9+        .quad   0x6b7dc0dc8d3fac58+        .quad   0x5497cd6ce6e42bfd+        .quad   0x542f7d1bf400d305+        .quad   0x4159f47f048d9136++        // 2^164 * 2 * G++        .quad   0x20ad660839e31e32+        .quad   0xf81e1bd58405be50+        .quad   0xf8064056f4dabc69+        .quad   0x14d23dd4ce71b975+        .quad   0x748515a8bbd24839+        .quad   0x77128347afb02b55+        .quad   0x50ba2ac649a2a17f+        .quad   0x060525513ad730f1+        .quad   0xf2398e098aa27f82+        .quad   0x6d7982bb89a1b024+        .quad   0xfa694084214dd24c+        .quad   0x71ab966fa32301c3++        // 2^164 * 3 * G++        .quad   0x2dcbd8e34ded02fc+        .quad   0x1151f3ec596f22aa+        .quad   0xbca255434e0328da+        .quad   0x35768fbe92411b22+        .quad   0xb1088a0702809955+        .quad   0x43b273ea0b43c391+        .quad   0xca9b67aefe0686ed+        .quad   0x605eecbf8335f4ed+        .quad   0x83200a656c340431+        .quad   0x9fcd71678ee59c2f+        .quad   0x75d4613f71300f8a+        .quad   0x7a912faf60f542f9++        // 2^164 * 4 * G++        .quad   0xb204585e5edc1a43+        .quad   0x9f0e16ee5897c73c+        .quad   0x5b82c0ae4e70483c+        .quad   0x624a170e2bddf9be+        .quad   0x253f4f8dfa2d5597+        .quad   0x25e49c405477130c+        .quad   0x00c052e5996b1102+        .quad   0x33cb966e33bb6c4a+        .quad   0x597028047f116909+        .quad   0x828ac41c1e564467+        .quad   0x70417dbde6217387+        .quad   0x721627aefbac4384++        // 2^164 * 5 * G++        .quad   0x97d03bc38736add5+        .quad   0x2f1422afc532b130+        .quad   0x3aa68a057101bbc4+        .quad   0x4c946cf7e74f9fa7+        .quad   0xfd3097bc410b2f22+        .quad   0xf1a05da7b5cfa844+        .quad   0x61289a1def57ca74+        .quad   0x245ea199bb821902+        .quad   0xaedca66978d477f8+        .quad   0x1898ba3c29117fe1+        .quad   0xcf73f983720cbd58+        .quad   0x67da12e6b8b56351++        // 2^164 * 6 * G++        .quad   0x7067e187b4bd6e07+        .quad   0x6e8f0203c7d1fe74+        .quad   0x93c6aa2f38c85a30+        .quad   0x76297d1f3d75a78a+        .quad   0x2b7ef3d38ec8308c+        .quad   0x828fd7ec71eb94ab+        .quad   0x807c3b36c5062abd+        .quad   0x0cb64cb831a94141+        .quad   0x3030fc33534c6378+        .quad   0xb9635c5ce541e861+        .quad   0x15d9a9bed9b2c728+        .quad   0x49233ea3f3775dcb++        // 2^164 * 7 * G++        .quad   0x629398fa8dbffc3a+        .quad   0xe12fe52dd54db455+        .quad   0xf3be11dfdaf25295+        .quad   0x628b140dce5e7b51+        .quad   0x7b3985fe1c9f249b+        .quad   0x4fd6b2d5a1233293+        .quad   0xceb345941adf4d62+        .quad   0x6987ff6f542de50c+        .quad   0x47e241428f83753c+        .quad   0x6317bebc866af997+        .quad   0xdabb5b433d1a9829+        .quad   0x074d8d245287fb2d++        // 2^164 * 8 * G++        .quad   0x8337d9cd440bfc31+        .quad   0x729d2ca1af318fd7+        .quad   0xa040a4a4772c2070+        .quad   0x46002ef03a7349be+        .quad   0x481875c6c0e31488+        .quad   0x219429b2e22034b4+        .quad   0x7223c98a31283b65+        .quad   0x3420d60b342277f9+        .quad   0xfaa23adeaffe65f7+        .quad   0x78261ed45be0764c+        .quad   0x441c0a1e2f164403+        .quad   0x5aea8e567a87d395++        // 2^168 * 1 * G++        .quad   0x7813c1a2bca4283d+        .quad   0xed62f091a1863dd9+        .quad   0xaec7bcb8c268fa86+        .quad   0x10e5d3b76f1cae4c+        .quad   0x2dbc6fb6e4e0f177+        .quad   0x04e1bf29a4bd6a93+        .quad   0x5e1966d4787af6e8+        .quad   0x0edc5f5eb426d060+        .quad   0x5453bfd653da8e67+        .quad   0xe9dc1eec24a9f641+        .quad   0xbf87263b03578a23+        .quad   0x45b46c51361cba72++        // 2^168 * 2 * G++        .quad   0xa9402abf314f7fa1+        .quad   0xe257f1dc8e8cf450+        .quad   0x1dbbd54b23a8be84+        .quad   0x2177bfa36dcb713b+        .quad   0xce9d4ddd8a7fe3e4+        .quad   0xab13645676620e30+        .quad   0x4b594f7bb30e9958+        .quad   0x5c1c0aef321229df+        .quad   0x37081bbcfa79db8f+        .quad   0x6048811ec25f59b3+        .quad   0x087a76659c832487+        .quad   0x4ae619387d8ab5bb++        // 2^168 * 3 * G++        .quad   0x8ddbf6aa5344a32e+        .quad   0x7d88eab4b41b4078+        .quad   0x5eb0eb974a130d60+        .quad   0x1a00d91b17bf3e03+        .quad   0x61117e44985bfb83+        .quad   0xfce0462a71963136+        .quad   0x83ac3448d425904b+        .quad   0x75685abe5ba43d64+        .quad   0x6e960933eb61f2b2+        .quad   0x543d0fa8c9ff4952+        .quad   0xdf7275107af66569+        .quad   0x135529b623b0e6aa++        // 2^168 * 4 * G++        .quad   0x18f0dbd7add1d518+        .quad   0x979f7888cfc11f11+        .quad   0x8732e1f07114759b+        .quad   0x79b5b81a65ca3a01+        .quad   0xf5c716bce22e83fe+        .quad   0xb42beb19e80985c1+        .quad   0xec9da63714254aae+        .quad   0x5972ea051590a613+        .quad   0x0fd4ac20dc8f7811+        .quad   0x9a9ad294ac4d4fa8+        .quad   0xc01b2d64b3360434+        .quad   0x4f7e9c95905f3bdb++        // 2^168 * 5 * G++        .quad   0x62674bbc5781302e+        .quad   0xd8520f3989addc0f+        .quad   0x8c2999ae53fbd9c6+        .quad   0x31993ad92e638e4c+        .quad   0x71c8443d355299fe+        .quad   0x8bcd3b1cdbebead7+        .quad   0x8092499ef1a49466+        .quad   0x1942eec4a144adc8+        .quad   0x7dac5319ae234992+        .quad   0x2c1b3d910cea3e92+        .quad   0x553ce494253c1122+        .quad   0x2a0a65314ef9ca75++        // 2^168 * 6 * G++        .quad   0x2db7937ff7f927c2+        .quad   0xdb741f0617d0a635+        .quad   0x5982f3a21155af76+        .quad   0x4cf6e218647c2ded+        .quad   0xcf361acd3c1c793a+        .quad   0x2f9ebcac5a35bc3b+        .quad   0x60e860e9a8cda6ab+        .quad   0x055dc39b6dea1a13+        .quad   0xb119227cc28d5bb6+        .quad   0x07e24ebc774dffab+        .quad   0xa83c78cee4a32c89+        .quad   0x121a307710aa24b6++        // 2^168 * 7 * G++        .quad   0xe4db5d5e9f034a97+        .quad   0xe153fc093034bc2d+        .quad   0x460546919551d3b1+        .quad   0x333fc76c7a40e52d+        .quad   0xd659713ec77483c9+        .quad   0x88bfe077b82b96af+        .quad   0x289e28231097bcd3+        .quad   0x527bb94a6ced3a9b+        .quad   0x563d992a995b482e+        .quad   0x3405d07c6e383801+        .quad   0x485035de2f64d8e5+        .quad   0x6b89069b20a7a9f7++        // 2^168 * 8 * G++        .quad   0x812aa0416270220d+        .quad   0x995a89faf9245b4e+        .quad   0xffadc4ce5072ef05+        .quad   0x23bc2103aa73eb73+        .quad   0x4082fa8cb5c7db77+        .quad   0x068686f8c734c155+        .quad   0x29e6c8d9f6e7a57e+        .quad   0x0473d308a7639bcf+        .quad   0xcaee792603589e05+        .quad   0x2b4b421246dcc492+        .quad   0x02a1ef74e601a94f+        .quad   0x102f73bfde04341a++        // 2^172 * 1 * G++        .quad   0xb5a2d50c7ec20d3e+        .quad   0xc64bdd6ea0c97263+        .quad   0x56e89052c1ff734d+        .quad   0x4929c6f72b2ffaba+        .quad   0x358ecba293a36247+        .quad   0xaf8f9862b268fd65+        .quad   0x412f7e9968a01c89+        .quad   0x5786f312cd754524+        .quad   0x337788ffca14032c+        .quad   0xf3921028447f1ee3+        .quad   0x8b14071f231bccad+        .quad   0x4c817b4bf2344783++        // 2^172 * 2 * G++        .quad   0x0ff853852871b96e+        .quad   0xe13e9fab60c3f1bb+        .quad   0xeefd595325344402+        .quad   0x0a37c37075b7744b+        .quad   0x413ba057a40b4484+        .quad   0xba4c2e1a4f5f6a43+        .quad   0x614ba0a5aee1d61c+        .quad   0x78a1531a8b05dc53+        .quad   0x6cbdf1703ad0562b+        .quad   0x8ecf4830c92521a3+        .quad   0xdaebd303fd8424e7+        .quad   0x72ad82a42e5ec56f++        // 2^172 * 3 * G++        .quad   0x3f9e8e35bafb65f6+        .quad   0x39d69ec8f27293a1+        .quad   0x6cb8cd958cf6a3d0+        .quad   0x1734778173adae6d+        .quad   0xc368939167024bc3+        .quad   0x8e69d16d49502fda+        .quad   0xfcf2ec3ce45f4b29+        .quad   0x065f669ea3b4cbc4+        .quad   0x8a00aec75532db4d+        .quad   0xb869a4e443e31bb1+        .quad   0x4a0f8552d3a7f515+        .quad   0x19adeb7c303d7c08++        // 2^172 * 4 * G++        .quad   0xc720cb6153ead9a3+        .quad   0x55b2c97f512b636e+        .quad   0xb1e35b5fd40290b1+        .quad   0x2fd9ccf13b530ee2+        .quad   0x9d05ba7d43c31794+        .quad   0x2470c8ff93322526+        .quad   0x8323dec816197438+        .quad   0x2852709881569b53+        .quad   0x07bd475b47f796b8+        .quad   0xd2c7b013542c8f54+        .quad   0x2dbd23f43b24f87e+        .quad   0x6551afd77b0901d6++        // 2^172 * 5 * G++        .quad   0x4546baaf54aac27f+        .quad   0xf6f66fecb2a45a28+        .quad   0x582d1b5b562bcfe8+        .quad   0x44b123f3920f785f+        .quad   0x68a24ce3a1d5c9ac+        .quad   0xbb77a33d10ff6461+        .quad   0x0f86ce4425d3166e+        .quad   0x56507c0950b9623b+        .quad   0x1206f0b7d1713e63+        .quad   0x353fe3d915bafc74+        .quad   0x194ceb970ad9d94d+        .quad   0x62fadd7cf9d03ad3++        // 2^172 * 6 * G++        .quad   0xc6b5967b5598a074+        .quad   0x5efe91ce8e493e25+        .quad   0xd4b72c4549280888+        .quad   0x20ef1149a26740c2+        .quad   0x3cd7bc61e7ce4594+        .quad   0xcd6b35a9b7dd267e+        .quad   0xa080abc84366ef27+        .quad   0x6ec7c46f59c79711+        .quad   0x2f07ad636f09a8a2+        .quad   0x8697e6ce24205e7d+        .quad   0xc0aefc05ee35a139+        .quad   0x15e80958b5f9d897++        // 2^172 * 7 * G++        .quad   0x25a5ef7d0c3e235b+        .quad   0x6c39c17fbe134ee7+        .quad   0xc774e1342dc5c327+        .quad   0x021354b892021f39+        .quad   0x4dd1ed355bb061c4+        .quad   0x42dc0cef941c0700+        .quad   0x61305dc1fd86340e+        .quad   0x56b2cc930e55a443+        .quad   0x1df79da6a6bfc5a2+        .quad   0x02f3a2749fde4369+        .quad   0xb323d9f2cda390a7+        .quad   0x7be0847b8774d363++        // 2^172 * 8 * G++        .quad   0x8c99cc5a8b3f55c3+        .quad   0x0611d7253fded2a0+        .quad   0xed2995ff36b70a36+        .quad   0x1f699a54d78a2619+        .quad   0x1466f5af5307fa11+        .quad   0x817fcc7ded6c0af2+        .quad   0x0a6de44ec3a4a3fb+        .quad   0x74071475bc927d0b+        .quad   0xe77292f373e7ea8a+        .quad   0x296537d2cb045a31+        .quad   0x1bd0653ed3274fde+        .quad   0x2f9a2c4476bd2966++        // 2^176 * 1 * G++        .quad   0xeb18b9ab7f5745c6+        .quad   0x023a8aee5787c690+        .quad   0xb72712da2df7afa9+        .quad   0x36597d25ea5c013d+        .quad   0xa2b4dae0b5511c9a+        .quad   0x7ac860292bffff06+        .quad   0x981f375df5504234+        .quad   0x3f6bd725da4ea12d+        .quad   0x734d8d7b106058ac+        .quad   0xd940579e6fc6905f+        .quad   0x6466f8f99202932d+        .quad   0x7b7ecc19da60d6d0++        // 2^176 * 2 * G++        .quad   0x78c2373c695c690d+        .quad   0xdd252e660642906e+        .quad   0x951d44444ae12bd2+        .quad   0x4235ad7601743956+        .quad   0x6dae4a51a77cfa9b+        .quad   0x82263654e7a38650+        .quad   0x09bbffcd8f2d82db+        .quad   0x03bedc661bf5caba+        .quad   0x6258cb0d078975f5+        .quad   0x492942549189f298+        .quad   0xa0cab423e2e36ee4+        .quad   0x0e7ce2b0cdf066a1++        // 2^176 * 3 * G++        .quad   0xc494643ac48c85a3+        .quad   0xfd361df43c6139ad+        .quad   0x09db17dd3ae94d48+        .quad   0x666e0a5d8fb4674a+        .quad   0xfea6fedfd94b70f9+        .quad   0xf130c051c1fcba2d+        .quad   0x4882d47e7f2fab89+        .quad   0x615256138aeceeb5+        .quad   0x2abbf64e4870cb0d+        .quad   0xcd65bcf0aa458b6b+        .quad   0x9abe4eba75e8985d+        .quad   0x7f0bc810d514dee4++        // 2^176 * 4 * G++        .quad   0xb9006ba426f4136f+        .quad   0x8d67369e57e03035+        .quad   0xcbc8dfd94f463c28+        .quad   0x0d1f8dbcf8eedbf5+        .quad   0x83ac9dad737213a0+        .quad   0x9ff6f8ba2ef72e98+        .quad   0x311e2edd43ec6957+        .quad   0x1d3a907ddec5ab75+        .quad   0xba1693313ed081dc+        .quad   0x29329fad851b3480+        .quad   0x0128013c030321cb+        .quad   0x00011b44a31bfde3++        // 2^176 * 5 * G++        .quad   0x3fdfa06c3fc66c0c+        .quad   0x5d40e38e4dd60dd2+        .quad   0x7ae38b38268e4d71+        .quad   0x3ac48d916e8357e1+        .quad   0x16561f696a0aa75c+        .quad   0xc1bf725c5852bd6a+        .quad   0x11a8dd7f9a7966ad+        .quad   0x63d988a2d2851026+        .quad   0x00120753afbd232e+        .quad   0xe92bceb8fdd8f683+        .quad   0xf81669b384e72b91+        .quad   0x33fad52b2368a066++        // 2^176 * 6 * G++        .quad   0x540649c6c5e41e16+        .quad   0x0af86430333f7735+        .quad   0xb2acfcd2f305e746+        .quad   0x16c0f429a256dca7+        .quad   0x8d2cc8d0c422cfe8+        .quad   0x072b4f7b05a13acb+        .quad   0xa3feb6e6ecf6a56f+        .quad   0x3cc355ccb90a71e2+        .quad   0xe9b69443903e9131+        .quad   0xb8a494cb7a5637ce+        .quad   0xc87cd1a4baba9244+        .quad   0x631eaf426bae7568++        // 2^176 * 7 * G++        .quad   0xb3e90410da66fe9f+        .quad   0x85dd4b526c16e5a6+        .quad   0xbc3d97611ef9bf83+        .quad   0x5599648b1ea919b5+        .quad   0x47d975b9a3700de8+        .quad   0x7280c5fbe2f80552+        .quad   0x53658f2732e45de1+        .quad   0x431f2c7f665f80b5+        .quad   0xd6026344858f7b19+        .quad   0x14ab352fa1ea514a+        .quad   0x8900441a2090a9d7+        .quad   0x7b04715f91253b26++        // 2^176 * 8 * G++        .quad   0x83edbd28acf6ae43+        .quad   0x86357c8b7d5c7ab4+        .quad   0xc0404769b7eb2c44+        .quad   0x59b37bf5c2f6583f+        .quad   0xb376c280c4e6bac6+        .quad   0x970ed3dd6d1d9b0b+        .quad   0xb09a9558450bf944+        .quad   0x48d0acfa57cde223+        .quad   0xb60f26e47dabe671+        .quad   0xf1d1a197622f3a37+        .quad   0x4208ce7ee9960394+        .quad   0x16234191336d3bdb++        // 2^180 * 1 * G++        .quad   0xf19aeac733a63aef+        .quad   0x2c7fba5d4442454e+        .quad   0x5da87aa04795e441+        .quad   0x413051e1a4e0b0f5+        .quad   0x852dd1fd3d578bbe+        .quad   0x2b65ce72c3286108+        .quad   0x658c07f4eace2273+        .quad   0x0933f804ec38ab40+        .quad   0xa7ab69798d496476+        .quad   0x8121aadefcb5abc8+        .quad   0xa5dc12ef7b539472+        .quad   0x07fd47065e45351a++        // 2^180 * 2 * G++        .quad   0xc8583c3d258d2bcd+        .quad   0x17029a4daf60b73f+        .quad   0xfa0fc9d6416a3781+        .quad   0x1c1e5fba38b3fb23+        .quad   0x304211559ae8e7c3+        .quad   0xf281b229944882a5+        .quad   0x8a13ac2e378250e4+        .quad   0x014afa0954ba48f4+        .quad   0xcb3197001bb3666c+        .quad   0x330060524bffecb9+        .quad   0x293711991a88233c+        .quad   0x291884363d4ed364++        // 2^180 * 3 * G++        .quad   0x033c6805dc4babfa+        .quad   0x2c15bf5e5596ecc1+        .quad   0x1bc70624b59b1d3b+        .quad   0x3ede9850a19f0ec5+        .quad   0xfb9d37c3bc1ab6eb+        .quad   0x02be14534d57a240+        .quad   0xf4d73415f8a5e1f6+        .quad   0x5964f4300ccc8188+        .quad   0xe44a23152d096800+        .quad   0x5c08c55970866996+        .quad   0xdf2db60a46affb6e+        .quad   0x579155c1f856fd89++        // 2^180 * 4 * G++        .quad   0x96324edd12e0c9ef+        .quad   0x468b878df2420297+        .quad   0x199a3776a4f573be+        .quad   0x1e7fbcf18e91e92a+        .quad   0xb5f16b630817e7a6+        .quad   0x808c69233c351026+        .quad   0x324a983b54cef201+        .quad   0x53c092084a485345+        .quad   0xd2d41481f1cbafbf+        .quad   0x231d2db6716174e5+        .quad   0x0b7d7656e2a55c98+        .quad   0x3e955cd82aa495f6++        // 2^180 * 5 * G++        .quad   0xe48f535e3ed15433+        .quad   0xd075692a0d7270a3+        .quad   0x40fbd21daade6387+        .quad   0x14264887cf4495f5+        .quad   0xab39f3ef61bb3a3f+        .quad   0x8eb400652eb9193e+        .quad   0xb5de6ecc38c11f74+        .quad   0x654d7e9626f3c49f+        .quad   0xe564cfdd5c7d2ceb+        .quad   0x82eeafded737ccb9+        .quad   0x6107db62d1f9b0ab+        .quad   0x0b6baac3b4358dbb++        // 2^180 * 6 * G++        .quad   0x7ae62bcb8622fe98+        .quad   0x47762256ceb891af+        .quad   0x1a5a92bcf2e406b4+        .quad   0x7d29401784e41501+        .quad   0x204abad63700a93b+        .quad   0xbe0023d3da779373+        .quad   0xd85f0346633ab709+        .quad   0x00496dc490820412+        .quad   0x1c74b88dc27e6360+        .quad   0x074854268d14850c+        .quad   0xa145fb7b3e0dcb30+        .quad   0x10843f1b43803b23++        // 2^180 * 7 * G++        .quad   0xc5f90455376276dd+        .quad   0xce59158dd7645cd9+        .quad   0x92f65d511d366b39+        .quad   0x11574b6e526996c4+        .quad   0xd56f672de324689b+        .quad   0xd1da8aedb394a981+        .quad   0xdd7b58fe9168cfed+        .quad   0x7ce246cd4d56c1e8+        .quad   0xb8f4308e7f80be53+        .quad   0x5f3cb8cb34a9d397+        .quad   0x18a961bd33cc2b2c+        .quad   0x710045fb3a9af671++        // 2^180 * 8 * G++        .quad   0x73f93d36101b95eb+        .quad   0xfaef33794f6f4486+        .quad   0x5651735f8f15e562+        .quad   0x7fa3f19058b40da1+        .quad   0xa03fc862059d699e+        .quad   0x2370cfa19a619e69+        .quad   0xc4fe3b122f823deb+        .quad   0x1d1b056fa7f0844e+        .quad   0x1bc64631e56bf61f+        .quad   0xd379ab106e5382a3+        .quad   0x4d58c57e0540168d+        .quad   0x566256628442d8e4++        // 2^184 * 1 * G++        .quad   0xb9e499def6267ff6+        .quad   0x7772ca7b742c0843+        .quad   0x23a0153fe9a4f2b1+        .quad   0x2cdfdfecd5d05006+        .quad   0xdd499cd61ff38640+        .quad   0x29cd9bc3063625a0+        .quad   0x51e2d8023dd73dc3+        .quad   0x4a25707a203b9231+        .quad   0x2ab7668a53f6ed6a+        .quad   0x304242581dd170a1+        .quad   0x4000144c3ae20161+        .quad   0x5721896d248e49fc++        // 2^184 * 2 * G++        .quad   0x0b6e5517fd181bae+        .quad   0x9022629f2bb963b4+        .quad   0x5509bce932064625+        .quad   0x578edd74f63c13da+        .quad   0x285d5091a1d0da4e+        .quad   0x4baa6fa7b5fe3e08+        .quad   0x63e5177ce19393b3+        .quad   0x03c935afc4b030fd+        .quad   0x997276c6492b0c3d+        .quad   0x47ccc2c4dfe205fc+        .quad   0xdcd29b84dd623a3c+        .quad   0x3ec2ab590288c7a2++        // 2^184 * 3 * G++        .quad   0xa1a0d27be4d87bb9+        .quad   0xa98b4deb61391aed+        .quad   0x99a0ddd073cb9b83+        .quad   0x2dd5c25a200fcace+        .quad   0xa7213a09ae32d1cb+        .quad   0x0f2b87df40f5c2d5+        .quad   0x0baea4c6e81eab29+        .quad   0x0e1bf66c6adbac5e+        .quad   0xe2abd5e9792c887e+        .quad   0x1a020018cb926d5d+        .quad   0xbfba69cdbaae5f1e+        .quad   0x730548b35ae88f5f++        // 2^184 * 4 * G++        .quad   0xc43551a3cba8b8ee+        .quad   0x65a26f1db2115f16+        .quad   0x760f4f52ab8c3850+        .quad   0x3043443b411db8ca+        .quad   0x805b094ba1d6e334+        .quad   0xbf3ef17709353f19+        .quad   0x423f06cb0622702b+        .quad   0x585a2277d87845dd+        .quad   0xa18a5f8233d48962+        .quad   0x6698c4b5ec78257f+        .quad   0xa78e6fa5373e41ff+        .quad   0x7656278950ef981f++        // 2^184 * 5 * G++        .quad   0x38c3cf59d51fc8c0+        .quad   0x9bedd2fd0506b6f2+        .quad   0x26bf109fab570e8f+        .quad   0x3f4160a8c1b846a6+        .quad   0xe17073a3ea86cf9d+        .quad   0x3a8cfbb707155fdc+        .quad   0x4853e7fc31838a8e+        .quad   0x28bbf484b613f616+        .quad   0xf2612f5c6f136c7c+        .quad   0xafead107f6dd11be+        .quad   0x527e9ad213de6f33+        .quad   0x1e79cb358188f75d++        // 2^184 * 6 * G++        .quad   0x013436c3eef7e3f1+        .quad   0x828b6a7ffe9e10f8+        .quad   0x7ff908e5bcf9defc+        .quad   0x65d7951b3a3b3831+        .quad   0x77e953d8f5e08181+        .quad   0x84a50c44299dded9+        .quad   0xdc6c2d0c864525e5+        .quad   0x478ab52d39d1f2f4+        .quad   0x66a6a4d39252d159+        .quad   0xe5dde1bc871ac807+        .quad   0xb82c6b40a6c1c96f+        .quad   0x16d87a411a212214++        // 2^184 * 7 * G++        .quad   0xb3bd7e5a42066215+        .quad   0x879be3cd0c5a24c1+        .quad   0x57c05db1d6f994b7+        .quad   0x28f87c8165f38ca6+        .quad   0xfba4d5e2d54e0583+        .quad   0xe21fafd72ebd99fa+        .quad   0x497ac2736ee9778f+        .quad   0x1f990b577a5a6dde+        .quad   0xa3344ead1be8f7d6+        .quad   0x7d1e50ebacea798f+        .quad   0x77c6569e520de052+        .quad   0x45882fe1534d6d3e++        // 2^184 * 8 * G++        .quad   0x6669345d757983d6+        .quad   0x62b6ed1117aa11a6+        .quad   0x7ddd1857985e128f+        .quad   0x688fe5b8f626f6dd+        .quad   0xd8ac9929943c6fe4+        .quad   0xb5f9f161a38392a2+        .quad   0x2699db13bec89af3+        .quad   0x7dcf843ce405f074+        .quad   0x6c90d6484a4732c0+        .quad   0xd52143fdca563299+        .quad   0xb3be28c3915dc6e1+        .quad   0x6739687e7327191b++        // 2^188 * 1 * G++        .quad   0x9f65c5ea200814cf+        .quad   0x840536e169a31740+        .quad   0x8b0ed13925c8b4ad+        .quad   0x0080dbafe936361d+        .quad   0x8ce5aad0c9cb971f+        .quad   0x1156aaa99fd54a29+        .quad   0x41f7247015af9b78+        .quad   0x1fe8cca8420f49aa+        .quad   0x72a1848f3c0cc82a+        .quad   0x38c560c2877c9e54+        .quad   0x5004e228ce554140+        .quad   0x042418a103429d71++        // 2^188 * 2 * G++        .quad   0x899dea51abf3ff5f+        .quad   0x9b93a8672fc2d8ba+        .quad   0x2c38cb97be6ebd5c+        .quad   0x114d578497263b5d+        .quad   0x58e84c6f20816247+        .quad   0x8db2b2b6e36fd793+        .quad   0x977182561d484d85+        .quad   0x0822024f8632abd7+        .quad   0xb301bb7c6b1beca3+        .quad   0x55393f6dc6eb1375+        .quad   0x910d281097b6e4eb+        .quad   0x1ad4548d9d479ea3++        // 2^188 * 3 * G++        .quad   0xcd5a7da0389a48fd+        .quad   0xb38fa4aa9a78371e+        .quad   0xc6d9761b2cdb8e6c+        .quad   0x35cf51dbc97e1443+        .quad   0xa06fe66d0fe9fed3+        .quad   0xa8733a401c587909+        .quad   0x30d14d800df98953+        .quad   0x41ce5876c7b30258+        .quad   0x59ac3bc5d670c022+        .quad   0xeae67c109b119406+        .quad   0x9798bdf0b3782fda+        .quad   0x651e3201fd074092++        // 2^188 * 4 * G++        .quad   0xd63d8483ef30c5cf+        .quad   0x4cd4b4962361cc0c+        .quad   0xee90e500a48426ac+        .quad   0x0af51d7d18c14eeb+        .quad   0xa57ba4a01efcae9e+        .quad   0x769f4beedc308a94+        .quad   0xd1f10eeb3603cb2e+        .quad   0x4099ce5e7e441278+        .quad   0x1ac98e4f8a5121e9+        .quad   0x7dae9544dbfa2fe0+        .quad   0x8320aa0dd6430df9+        .quad   0x667282652c4a2fb5++        // 2^188 * 5 * G++        .quad   0x874621f4d86bc9ab+        .quad   0xb54c7bbe56fe6fea+        .quad   0x077a24257fadc22c+        .quad   0x1ab53be419b90d39+        .quad   0xada8b6e02946db23+        .quad   0x1c0ce51a7b253ab7+        .quad   0x8448c85a66dd485b+        .quad   0x7f1fc025d0675adf+        .quad   0xd8ee1b18319ea6aa+        .quad   0x004d88083a21f0da+        .quad   0x3bd6aa1d883a4f4b+        .quad   0x4db9a3a6dfd9fd14++        // 2^188 * 6 * G++        .quad   0x8ce7b23bb99c0755+        .quad   0x35c5d6edc4f50f7a+        .quad   0x7e1e2ed2ed9b50c3+        .quad   0x36305f16e8934da1+        .quad   0xd95b00bbcbb77c68+        .quad   0xddbc846a91f17849+        .quad   0x7cf700aebe28d9b3+        .quad   0x5ce1285c85d31f3e+        .quad   0x31b6972d98b0bde8+        .quad   0x7d920706aca6de5b+        .quad   0xe67310f8908a659f+        .quad   0x50fac2a6efdf0235++        // 2^188 * 7 * G++        .quad   0xf3d3a9f35b880f5a+        .quad   0xedec050cdb03e7c2+        .quad   0xa896981ff9f0b1a2+        .quad   0x49a4ae2bac5e34a4+        .quad   0x295b1c86f6f449bc+        .quad   0x51b2e84a1f0ab4dd+        .quad   0xc001cb30aa8e551d+        .quad   0x6a28d35944f43662+        .quad   0x28bb12ee04a740e0+        .quad   0x14313bbd9bce8174+        .quad   0x72f5b5e4e8c10c40+        .quad   0x7cbfb19936adcd5b++        // 2^188 * 8 * G++        .quad   0xa311ddc26b89792d+        .quad   0x1b30b4c6da512664+        .quad   0x0ca77b4ccf150859+        .quad   0x1de443df1b009408+        .quad   0x8e793a7acc36e6e0+        .quad   0xf9fab7a37d586eed+        .quad   0x3a4f9692bae1f4e4+        .quad   0x1c14b03eff5f447e+        .quad   0x19647bd114a85291+        .quad   0x57b76cb21034d3af+        .quad   0x6329db440f9d6dfa+        .quad   0x5ef43e586a571493++        // 2^192 * 1 * G++        .quad   0xef782014385675a6+        .quad   0xa2649f30aafda9e8+        .quad   0x4cd1eb505cdfa8cb+        .quad   0x46115aba1d4dc0b3+        .quad   0xa66dcc9dc80c1ac0+        .quad   0x97a05cf41b38a436+        .quad   0xa7ebf3be95dbd7c6+        .quad   0x7da0b8f68d7e7dab+        .quad   0xd40f1953c3b5da76+        .quad   0x1dac6f7321119e9b+        .quad   0x03cc6021feb25960+        .quad   0x5a5f887e83674b4b++        // 2^192 * 2 * G++        .quad   0x8f6301cf70a13d11+        .quad   0xcfceb815350dd0c4+        .quad   0xf70297d4a4bca47e+        .quad   0x3669b656e44d1434+        .quad   0x9e9628d3a0a643b9+        .quad   0xb5c3cb00e6c32064+        .quad   0x9b5302897c2dec32+        .quad   0x43e37ae2d5d1c70c+        .quad   0x387e3f06eda6e133+        .quad   0x67301d5199a13ac0+        .quad   0xbd5ad8f836263811+        .quad   0x6a21e6cd4fd5e9be++        // 2^192 * 3 * G++        .quad   0xf1c6170a3046e65f+        .quad   0x58712a2a00d23524+        .quad   0x69dbbd3c8c82b755+        .quad   0x586bf9f1a195ff57+        .quad   0xef4129126699b2e3+        .quad   0x71d30847708d1301+        .quad   0x325432d01182b0bd+        .quad   0x45371b07001e8b36+        .quad   0xa6db088d5ef8790b+        .quad   0x5278f0dc610937e5+        .quad   0xac0349d261a16eb8+        .quad   0x0eafb03790e52179++        // 2^192 * 4 * G++        .quad   0x960555c13748042f+        .quad   0x219a41e6820baa11+        .quad   0x1c81f73873486d0c+        .quad   0x309acc675a02c661+        .quad   0x5140805e0f75ae1d+        .quad   0xec02fbe32662cc30+        .quad   0x2cebdf1eea92396d+        .quad   0x44ae3344c5435bb3+        .quad   0x9cf289b9bba543ee+        .quad   0xf3760e9d5ac97142+        .quad   0x1d82e5c64f9360aa+        .quad   0x62d5221b7f94678f++        // 2^192 * 5 * G++        .quad   0x524c299c18d0936d+        .quad   0xc86bb56c8a0c1a0c+        .quad   0xa375052edb4a8631+        .quad   0x5c0efde4bc754562+        .quad   0x7585d4263af77a3c+        .quad   0xdfae7b11fee9144d+        .quad   0xa506708059f7193d+        .quad   0x14f29a5383922037+        .quad   0xdf717edc25b2d7f5+        .quad   0x21f970db99b53040+        .quad   0xda9234b7c3ed4c62+        .quad   0x5e72365c7bee093e++        // 2^192 * 6 * G++        .quad   0x575bfc074571217f+        .quad   0x3779675d0694d95b+        .quad   0x9a0a37bbf4191e33+        .quad   0x77f1104c47b4eabc+        .quad   0x7d9339062f08b33e+        .quad   0x5b9659e5df9f32be+        .quad   0xacff3dad1f9ebdfd+        .quad   0x70b20555cb7349b7+        .quad   0xbe5113c555112c4c+        .quad   0x6688423a9a881fcd+        .quad   0x446677855e503b47+        .quad   0x0e34398f4a06404a++        // 2^192 * 7 * G++        .quad   0xb67d22d93ecebde8+        .quad   0x09b3e84127822f07+        .quad   0x743fa61fb05b6d8d+        .quad   0x5e5405368a362372+        .quad   0x18930b093e4b1928+        .quad   0x7de3e10e73f3f640+        .quad   0xf43217da73395d6f+        .quad   0x6f8aded6ca379c3e+        .quad   0xe340123dfdb7b29a+        .quad   0x487b97e1a21ab291+        .quad   0xf9967d02fde6949e+        .quad   0x780de72ec8d3de97++        // 2^192 * 8 * G++        .quad   0x0ae28545089ae7bc+        .quad   0x388ddecf1c7f4d06+        .quad   0x38ac15510a4811b8+        .quad   0x0eb28bf671928ce4+        .quad   0x671feaf300f42772+        .quad   0x8f72eb2a2a8c41aa+        .quad   0x29a17fd797373292+        .quad   0x1defc6ad32b587a6+        .quad   0xaf5bbe1aef5195a7+        .quad   0x148c1277917b15ed+        .quad   0x2991f7fb7ae5da2e+        .quad   0x467d201bf8dd2867++        // 2^196 * 1 * G++        .quad   0x7906ee72f7bd2e6b+        .quad   0x05d270d6109abf4e+        .quad   0x8d5cfe45b941a8a4+        .quad   0x44c218671c974287+        .quad   0x745f9d56296bc318+        .quad   0x993580d4d8152e65+        .quad   0xb0e5b13f5839e9ce+        .quad   0x51fc2b28d43921c0+        .quad   0x1b8fd11795e2a98c+        .quad   0x1c4e5ee12b6b6291+        .quad   0x5b30e7107424b572+        .quad   0x6e6b9de84c4f4ac6++        // 2^196 * 2 * G++        .quad   0xdff25fce4b1de151+        .quad   0xd841c0c7e11c4025+        .quad   0x2554b3c854749c87+        .quad   0x2d292459908e0df9+        .quad   0x6b7c5f10f80cb088+        .quad   0x736b54dc56e42151+        .quad   0xc2b620a5c6ef99c4+        .quad   0x5f4c802cc3a06f42+        .quad   0x9b65c8f17d0752da+        .quad   0x881ce338c77ee800+        .quad   0xc3b514f05b62f9e3+        .quad   0x66ed5dd5bec10d48++        // 2^196 * 3 * G++        .quad   0x7d38a1c20bb2089d+        .quad   0x808334e196ccd412+        .quad   0xc4a70b8c6c97d313+        .quad   0x2eacf8bc03007f20+        .quad   0xf0adf3c9cbca047d+        .quad   0x81c3b2cbf4552f6b+        .quad   0xcfda112d44735f93+        .quad   0x1f23a0c77e20048c+        .quad   0xf235467be5bc1570+        .quad   0x03d2d9020dbab38c+        .quad   0x27529aa2fcf9e09e+        .quad   0x0840bef29d34bc50++        // 2^196 * 4 * G++        .quad   0x796dfb35dc10b287+        .quad   0x27176bcd5c7ff29d+        .quad   0x7f3d43e8c7b24905+        .quad   0x0304f5a191c54276+        .quad   0xcd54e06b7f37e4eb+        .quad   0x8cc15f87f5e96cca+        .quad   0xb8248bb0d3597dce+        .quad   0x246affa06074400c+        .quad   0x37d88e68fbe45321+        .quad   0x86097548c0d75032+        .quad   0x4e9b13ef894a0d35+        .quad   0x25a83cac5753d325++        // 2^196 * 5 * G++        .quad   0x10222f48eed8165e+        .quad   0x623fc1234b8bcf3a+        .quad   0x1e145c09c221e8f0+        .quad   0x7ccfa59fca782630+        .quad   0x9f0f66293952b6e2+        .quad   0x33db5e0e0934267b+        .quad   0xff45252bd609fedc+        .quad   0x06be10f5c506e0c9+        .quad   0x1a9615a9b62a345f+        .quad   0x22050c564a52fecc+        .quad   0xa7a2788528bc0dfe+        .quad   0x5e82770a1a1ee71d++        // 2^196 * 6 * G++        .quad   0x35425183ad896a5c+        .quad   0xe8673afbe78d52f6+        .quad   0x2c66f25f92a35f64+        .quad   0x09d04f3b3b86b102+        .quad   0xe802e80a42339c74+        .quad   0x34175166a7fffae5+        .quad   0x34865d1f1c408cae+        .quad   0x2cca982c605bc5ee+        .quad   0xfd2d5d35197dbe6e+        .quad   0x207c2eea8be4ffa3+        .quad   0x2613d8db325ae918+        .quad   0x7a325d1727741d3e++        // 2^196 * 7 * G++        .quad   0xd036b9bbd16dfde2+        .quad   0xa2055757c497a829+        .quad   0x8e6cc966a7f12667+        .quad   0x4d3b1a791239c180+        .quad   0xecd27d017e2a076a+        .quad   0xd788689f1636495e+        .quad   0x52a61af0919233e5+        .quad   0x2a479df17bb1ae64+        .quad   0x9e5eee8e33db2710+        .quad   0x189854ded6c43ca5+        .quad   0xa41c22c592718138+        .quad   0x27ad5538a43a5e9b++        // 2^196 * 8 * G++        .quad   0x2746dd4b15350d61+        .quad   0xd03fcbc8ee9521b7+        .quad   0xe86e365a138672ca+        .quad   0x510e987f7e7d89e2+        .quad   0xcb5a7d638e47077c+        .quad   0x8db7536120a1c059+        .quad   0x549e1e4d8bedfdcc+        .quad   0x080153b7503b179d+        .quad   0xdda69d930a3ed3e3+        .quad   0x3d386ef1cd60a722+        .quad   0xc817ad58bdaa4ee6+        .quad   0x23be8d554fe7372a++        // 2^200 * 1 * G++        .quad   0x95fe919a74ef4fad+        .quad   0x3a827becf6a308a2+        .quad   0x964e01d309a47b01+        .quad   0x71c43c4f5ba3c797+        .quad   0xbc1ef4bd567ae7a9+        .quad   0x3f624cb2d64498bd+        .quad   0xe41064d22c1f4ec8+        .quad   0x2ef9c5a5ba384001+        .quad   0xb6fd6df6fa9e74cd+        .quad   0xf18278bce4af267a+        .quad   0x8255b3d0f1ef990e+        .quad   0x5a758ca390c5f293++        // 2^200 * 2 * G++        .quad   0xa2b72710d9462495+        .quad   0x3aa8c6d2d57d5003+        .quad   0xe3d400bfa0b487ca+        .quad   0x2dbae244b3eb72ec+        .quad   0x8ce0918b1d61dc94+        .quad   0x8ded36469a813066+        .quad   0xd4e6a829afe8aad3+        .quad   0x0a738027f639d43f+        .quad   0x980f4a2f57ffe1cc+        .quad   0x00670d0de1839843+        .quad   0x105c3f4a49fb15fd+        .quad   0x2698ca635126a69c++        // 2^200 * 3 * G++        .quad   0xe765318832b0ba78+        .quad   0x381831f7925cff8b+        .quad   0x08a81b91a0291fcc+        .quad   0x1fb43dcc49caeb07+        .quad   0x2e3d702f5e3dd90e+        .quad   0x9e3f0918e4d25386+        .quad   0x5e773ef6024da96a+        .quad   0x3c004b0c4afa3332+        .quad   0x9aa946ac06f4b82b+        .quad   0x1ca284a5a806c4f3+        .quad   0x3ed3265fc6cd4787+        .quad   0x6b43fd01cd1fd217++        // 2^200 * 4 * G++        .quad   0xc7a75d4b4697c544+        .quad   0x15fdf848df0fffbf+        .quad   0x2868b9ebaa46785a+        .quad   0x5a68d7105b52f714+        .quad   0xb5c742583e760ef3+        .quad   0x75dc52b9ee0ab990+        .quad   0xbf1427c2072b923f+        .quad   0x73420b2d6ff0d9f0+        .quad   0xaf2cf6cb9e851e06+        .quad   0x8f593913c62238c4+        .quad   0xda8ab89699fbf373+        .quad   0x3db5632fea34bc9e++        // 2^200 * 5 * G++        .quad   0xf46eee2bf75dd9d8+        .quad   0x0d17b1f6396759a5+        .quad   0x1bf2d131499e7273+        .quad   0x04321adf49d75f13+        .quad   0x2e4990b1829825d5+        .quad   0xedeaeb873e9a8991+        .quad   0xeef03d394c704af8+        .quad   0x59197ea495df2b0e+        .quad   0x04e16019e4e55aae+        .quad   0xe77b437a7e2f92e9+        .quad   0xc7ce2dc16f159aa4+        .quad   0x45eafdc1f4d70cc0++        // 2^200 * 6 * G++        .quad   0x698401858045d72b+        .quad   0x4c22faa2cf2f0651+        .quad   0x941a36656b222dc6+        .quad   0x5a5eebc80362dade+        .quad   0xb60e4624cfccb1ed+        .quad   0x59dbc292bd5c0395+        .quad   0x31a09d1ddc0481c9+        .quad   0x3f73ceea5d56d940+        .quad   0xb7a7bfd10a4e8dc6+        .quad   0xbe57007e44c9b339+        .quad   0x60c1207f1557aefa+        .quad   0x26058891266218db++        // 2^200 * 7 * G++        .quad   0x59f704a68360ff04+        .quad   0xc3d93fde7661e6f4+        .quad   0x831b2a7312873551+        .quad   0x54ad0c2e4e615d57+        .quad   0x4c818e3cc676e542+        .quad   0x5e422c9303ceccad+        .quad   0xec07cccab4129f08+        .quad   0x0dedfa10b24443b8+        .quad   0xee3b67d5b82b522a+        .quad   0x36f163469fa5c1eb+        .quad   0xa5b4d2f26ec19fd3+        .quad   0x62ecb2baa77a9408++        // 2^200 * 8 * G++        .quad   0xe5ed795261152b3d+        .quad   0x4962357d0eddd7d1+        .quad   0x7482c8d0b96b4c71+        .quad   0x2e59f919a966d8be+        .quad   0x92072836afb62874+        .quad   0x5fcd5e8579e104a5+        .quad   0x5aad01adc630a14a+        .quad   0x61913d5075663f98+        .quad   0x0dc62d361a3231da+        .quad   0xfa47583294200270+        .quad   0x02d801513f9594ce+        .quad   0x3ddbc2a131c05d5c++        // 2^204 * 1 * G++        .quad   0x3f50a50a4ffb81ef+        .quad   0xb1e035093bf420bf+        .quad   0x9baa8e1cc6aa2cd0+        .quad   0x32239861fa237a40+        .quad   0xfb735ac2004a35d1+        .quad   0x31de0f433a6607c3+        .quad   0x7b8591bfc528d599+        .quad   0x55be9a25f5bb050c+        .quad   0x0d005acd33db3dbf+        .quad   0x0111b37c80ac35e2+        .quad   0x4892d66c6f88ebeb+        .quad   0x770eadb16508fbcd++        // 2^204 * 2 * G++        .quad   0x8451f9e05e4e89dd+        .quad   0xc06302ffbc793937+        .quad   0x5d22749556a6495c+        .quad   0x09a6755ca05603fb+        .quad   0xf1d3b681a05071b9+        .quad   0x2207659a3592ff3a+        .quad   0x5f0169297881e40e+        .quad   0x16bedd0e86ba374e+        .quad   0x5ecccc4f2c2737b5+        .quad   0x43b79e0c2dccb703+        .quad   0x33e008bc4ec43df3+        .quad   0x06c1b840f07566c0++        // 2^204 * 3 * G++        .quad   0x7688a5c6a388f877+        .quad   0x02a96c14deb2b6ac+        .quad   0x64c9f3431b8c2af8+        .quad   0x3628435554a1eed6+        .quad   0x69ee9e7f9b02805c+        .quad   0xcbff828a547d1640+        .quad   0x3d93a869b2430968+        .quad   0x46b7b8cd3fe26972+        .quad   0xe9812086fe7eebe0+        .quad   0x4cba6be72f515437+        .quad   0x1d04168b516efae9+        .quad   0x5ea1391043982cb9++        // 2^204 * 4 * G++        .quad   0x49125c9cf4702ee1+        .quad   0x4520b71f8b25b32d+        .quad   0x33193026501fef7e+        .quad   0x656d8997c8d2eb2b+        .quad   0x6f2b3be4d5d3b002+        .quad   0xafec33d96a09c880+        .quad   0x035f73a4a8bcc4cc+        .quad   0x22c5b9284662198b+        .quad   0xcb58c8fe433d8939+        .quad   0x89a0cb2e6a8d7e50+        .quad   0x79ca955309fbbe5a+        .quad   0x0c626616cd7fc106++        // 2^204 * 5 * G++        .quad   0x1ffeb80a4879b61f+        .quad   0x6396726e4ada21ed+        .quad   0x33c7b093368025ba+        .quad   0x471aa0c6f3c31788+        .quad   0x8fdfc379fbf454b1+        .quad   0x45a5a970f1a4b771+        .quad   0xac921ef7bad35915+        .quad   0x42d088dca81c2192+        .quad   0x8fda0f37a0165199+        .quad   0x0adadb77c8a0e343+        .quad   0x20fbfdfcc875e820+        .quad   0x1cf2bea80c2206e7++        // 2^204 * 6 * G++        .quad   0xc2ddf1deb36202ac+        .quad   0x92a5fe09d2e27aa5+        .quad   0x7d1648f6fc09f1d3+        .quad   0x74c2cc0513bc4959+        .quad   0x982d6e1a02c0412f+        .quad   0x90fa4c83db58e8fe+        .quad   0x01c2f5bcdcb18bc0+        .quad   0x686e0c90216abc66+        .quad   0x1fadbadba54395a7+        .quad   0xb41a02a0ae0da66a+        .quad   0xbf19f598bba37c07+        .quad   0x6a12b8acde48430d++        // 2^204 * 7 * G++        .quad   0xf8daea1f39d495d9+        .quad   0x592c190e525f1dfc+        .quad   0xdb8cbd04c9991d1b+        .quad   0x11f7fda3d88f0cb7+        .quad   0x793bdd801aaeeb5f+        .quad   0x00a2a0aac1518871+        .quad   0xe8a373a31f2136b4+        .quad   0x48aab888fc91ef19+        .quad   0x041f7e925830f40e+        .quad   0x002d6ca979661c06+        .quad   0x86dc9ff92b046a2e+        .quad   0x760360928b0493d1++        // 2^204 * 8 * G++        .quad   0x21bb41c6120cf9c6+        .quad   0xeab2aa12decda59b+        .quad   0xc1a72d020aa48b34+        .quad   0x215d4d27e87d3b68+        .quad   0xb43108e5695a0b05+        .quad   0x6cb00ee8ad37a38b+        .quad   0x5edad6eea3537381+        .quad   0x3f2602d4b6dc3224+        .quad   0xc8b247b65bcaf19c+        .quad   0x49779dc3b1b2c652+        .quad   0x89a180bbd5ece2e2+        .quad   0x13f098a3cec8e039++        // 2^208 * 1 * G++        .quad   0x9adc0ff9ce5ec54b+        .quad   0x039c2a6b8c2f130d+        .quad   0x028007c7f0f89515+        .quad   0x78968314ac04b36b+        .quad   0xf3aa57a22796bb14+        .quad   0x883abab79b07da21+        .quad   0xe54be21831a0391c+        .quad   0x5ee7fb38d83205f9+        .quad   0x538dfdcb41446a8e+        .quad   0xa5acfda9434937f9+        .quad   0x46af908d263c8c78+        .quad   0x61d0633c9bca0d09++        // 2^208 * 2 * G++        .quad   0x63744935ffdb2566+        .quad   0xc5bd6b89780b68bb+        .quad   0x6f1b3280553eec03+        .quad   0x6e965fd847aed7f5+        .quad   0xada328bcf8fc73df+        .quad   0xee84695da6f037fc+        .quad   0x637fb4db38c2a909+        .quad   0x5b23ac2df8067bdc+        .quad   0x9ad2b953ee80527b+        .quad   0xe88f19aafade6d8d+        .quad   0x0e711704150e82cf+        .quad   0x79b9bbb9dd95dedc++        // 2^208 * 3 * G++        .quad   0xebb355406a3126c2+        .quad   0xd26383a868c8c393+        .quad   0x6c0c6429e5b97a82+        .quad   0x5065f158c9fd2147+        .quad   0xd1997dae8e9f7374+        .quad   0xa032a2f8cfbb0816+        .quad   0xcd6cba126d445f0a+        .quad   0x1ba811460accb834+        .quad   0x708169fb0c429954+        .quad   0xe14600acd76ecf67+        .quad   0x2eaab98a70e645ba+        .quad   0x3981f39e58a4faf2++        // 2^208 * 4 * G++        .quad   0x18fb8a7559230a93+        .quad   0x1d168f6960e6f45d+        .quad   0x3a85a94514a93cb5+        .quad   0x38dc083705acd0fd+        .quad   0xc845dfa56de66fde+        .quad   0xe152a5002c40483a+        .quad   0xe9d2e163c7b4f632+        .quad   0x30f4452edcbc1b65+        .quad   0x856d2782c5759740+        .quad   0xfa134569f99cbecc+        .quad   0x8844fc73c0ea4e71+        .quad   0x632d9a1a593f2469++        // 2^208 * 5 * G++        .quad   0xf6bb6b15b807cba6+        .quad   0x1823c7dfbc54f0d7+        .quad   0xbb1d97036e29670b+        .quad   0x0b24f48847ed4a57+        .quad   0xbf09fd11ed0c84a7+        .quad   0x63f071810d9f693a+        .quad   0x21908c2d57cf8779+        .quad   0x3a5a7df28af64ba2+        .quad   0xdcdad4be511beac7+        .quad   0xa4538075ed26ccf2+        .quad   0xe19cff9f005f9a65+        .quad   0x34fcf74475481f63++        // 2^208 * 6 * G++        .quad   0xc197e04c789767ca+        .quad   0xb8714dcb38d9467d+        .quad   0x55de888283f95fa8+        .quad   0x3d3bdc164dfa63f7+        .quad   0xa5bb1dab78cfaa98+        .quad   0x5ceda267190b72f2+        .quad   0x9309c9110a92608e+        .quad   0x0119a3042fb374b0+        .quad   0x67a2d89ce8c2177d+        .quad   0x669da5f66895d0c1+        .quad   0xf56598e5b282a2b0+        .quad   0x56c088f1ede20a73++        // 2^208 * 7 * G++        .quad   0x336d3d1110a86e17+        .quad   0xd7f388320b75b2fa+        .quad   0xf915337625072988+        .quad   0x09674c6b99108b87+        .quad   0x581b5fac24f38f02+        .quad   0xa90be9febae30cbd+        .quad   0x9a2169028acf92f0+        .quad   0x038b7ea48359038f+        .quad   0x9f4ef82199316ff8+        .quad   0x2f49d282eaa78d4f+        .quad   0x0971a5ab5aef3174+        .quad   0x6e5e31025969eb65++        // 2^208 * 8 * G++        .quad   0xb16c62f587e593fb+        .quad   0x4999eddeca5d3e71+        .quad   0xb491c1e014cc3e6d+        .quad   0x08f5114789a8dba8+        .quad   0x3304fb0e63066222+        .quad   0xfb35068987acba3f+        .quad   0xbd1924778c1061a3+        .quad   0x3058ad43d1838620+        .quad   0x323c0ffde57663d0+        .quad   0x05c3df38a22ea610+        .quad   0xbdc78abdac994f9a+        .quad   0x26549fa4efe3dc99++        // 2^212 * 1 * G++        .quad   0x738b38d787ce8f89+        .quad   0xb62658e24179a88d+        .quad   0x30738c9cf151316d+        .quad   0x49128c7f727275c9+        .quad   0x04dbbc17f75396b9+        .quad   0x69e6a2d7d2f86746+        .quad   0xc6409d99f53eabc6+        .quad   0x606175f6332e25d2+        .quad   0x4021370ef540e7dd+        .quad   0x0910d6f5a1f1d0a5+        .quad   0x4634aacd5b06b807+        .quad   0x6a39e6356944f235++        // 2^212 * 2 * G++        .quad   0x96cd5640df90f3e7+        .quad   0x6c3a760edbfa25ea+        .quad   0x24f3ef0959e33cc4+        .quad   0x42889e7e530d2e58+        .quad   0x1da1965774049e9d+        .quad   0xfbcd6ea198fe352b+        .quad   0xb1cbcd50cc5236a6+        .quad   0x1f5ec83d3f9846e2+        .quad   0x8efb23c3328ccb75+        .quad   0xaf42a207dd876ee9+        .quad   0x20fbdadc5dfae796+        .quad   0x241e246b06bf9f51++        // 2^212 * 3 * G++        .quad   0x29e68e57ad6e98f6+        .quad   0x4c9260c80b462065+        .quad   0x3f00862ea51ebb4b+        .quad   0x5bc2c77fb38d9097+        .quad   0x7eaafc9a6280bbb8+        .quad   0x22a70f12f403d809+        .quad   0x31ce40bb1bfc8d20+        .quad   0x2bc65635e8bd53ee+        .quad   0xe8d5dc9fa96bad93+        .quad   0xe58fb17dde1947dc+        .quad   0x681532ea65185fa3+        .quad   0x1fdd6c3b034a7830++        // 2^212 * 4 * G++        .quad   0x0a64e28c55dc18fe+        .quad   0xe3df9e993399ebdd+        .quad   0x79ac432370e2e652+        .quad   0x35ff7fc33ae4cc0e+        .quad   0x9c13a6a52dd8f7a9+        .quad   0x2dbb1f8c3efdcabf+        .quad   0x961e32405e08f7b5+        .quad   0x48c8a121bbe6c9e5+        .quad   0xfc415a7c59646445+        .quad   0xd224b2d7c128b615+        .quad   0x6035c9c905fbb912+        .quad   0x42d7a91274429fab++        // 2^212 * 5 * G++        .quad   0x4e6213e3eaf72ed3+        .quad   0x6794981a43acd4e7+        .quad   0xff547cde6eb508cb+        .quad   0x6fed19dd10fcb532+        .quad   0xa9a48947933da5bc+        .quad   0x4a58920ec2e979ec+        .quad   0x96d8800013e5ac4c+        .quad   0x453692d74b48b147+        .quad   0xdd775d99a8559c6f+        .quad   0xf42a2140df003e24+        .quad   0x5223e229da928a66+        .quad   0x063f46ba6d38f22c++        // 2^212 * 6 * G++        .quad   0xd2d242895f536694+        .quad   0xca33a2c542939b2c+        .quad   0x986fada6c7ddb95c+        .quad   0x5a152c042f712d5d+        .quad   0x39843cb737346921+        .quad   0xa747fb0738c89447+        .quad   0xcb8d8031a245307e+        .quad   0x67810f8e6d82f068+        .quad   0x3eeb8fbcd2287db4+        .quad   0x72c7d3a301a03e93+        .quad   0x5473e88cbd98265a+        .quad   0x7324aa515921b403++        // 2^212 * 7 * G++        .quad   0x857942f46c3cbe8e+        .quad   0xa1d364b14730c046+        .quad   0x1c8ed914d23c41bf+        .quad   0x0838e161eef6d5d2+        .quad   0xad23f6dae82354cb+        .quad   0x6962502ab6571a6d+        .quad   0x9b651636e38e37d1+        .quad   0x5cac5005d1a3312f+        .quad   0x8cc154cce9e39904+        .quad   0x5b3a040b84de6846+        .quad   0xc4d8a61cb1be5d6e+        .quad   0x40fb897bd8861f02++        // 2^212 * 8 * G++        .quad   0x84c5aa9062de37a1+        .quad   0x421da5000d1d96e1+        .quad   0x788286306a9242d9+        .quad   0x3c5e464a690d10da+        .quad   0xe57ed8475ab10761+        .quad   0x71435e206fd13746+        .quad   0x342f824ecd025632+        .quad   0x4b16281ea8791e7b+        .quad   0xd1c101d50b813381+        .quad   0xdee60f1176ee6828+        .quad   0x0cb68893383f6409+        .quad   0x6183c565f6ff484a++        // 2^216 * 1 * G++        .quad   0x741d5a461e6bf9d6+        .quad   0x2305b3fc7777a581+        .quad   0xd45574a26474d3d9+        .quad   0x1926e1dc6401e0ff+        .quad   0xdb468549af3f666e+        .quad   0xd77fcf04f14a0ea5+        .quad   0x3df23ff7a4ba0c47+        .quad   0x3a10dfe132ce3c85+        .quad   0xe07f4e8aea17cea0+        .quad   0x2fd515463a1fc1fd+        .quad   0x175322fd31f2c0f1+        .quad   0x1fa1d01d861e5d15++        // 2^216 * 2 * G++        .quad   0xcc8055947d599832+        .quad   0x1e4656da37f15520+        .quad   0x99f6f7744e059320+        .quad   0x773563bc6a75cf33+        .quad   0x38dcac00d1df94ab+        .quad   0x2e712bddd1080de9+        .quad   0x7f13e93efdd5e262+        .quad   0x73fced18ee9a01e5+        .quad   0x06b1e90863139cb3+        .quad   0xa493da67c5a03ecd+        .quad   0x8d77cec8ad638932+        .quad   0x1f426b701b864f44++        // 2^216 * 3 * G++        .quad   0xefc9264c41911c01+        .quad   0xf1a3b7b817a22c25+        .quad   0x5875da6bf30f1447+        .quad   0x4e1af5271d31b090+        .quad   0xf17e35c891a12552+        .quad   0xb76b8153575e9c76+        .quad   0xfa83406f0d9b723e+        .quad   0x0b76bb1b3fa7e438+        .quad   0x08b8c1f97f92939b+        .quad   0xbe6771cbd444ab6e+        .quad   0x22e5646399bb8017+        .quad   0x7b6dd61eb772a955++        // 2^216 * 4 * G++        .quad   0xb7adc1e850f33d92+        .quad   0x7998fa4f608cd5cf+        .quad   0xad962dbd8dfc5bdb+        .quad   0x703e9bceaf1d2f4f+        .quad   0x5730abf9ab01d2c7+        .quad   0x16fb76dc40143b18+        .quad   0x866cbe65a0cbb281+        .quad   0x53fa9b659bff6afe+        .quad   0x6c14c8e994885455+        .quad   0x843a5d6665aed4e5+        .quad   0x181bb73ebcd65af1+        .quad   0x398d93e5c4c61f50++        // 2^216 * 5 * G++        .quad   0x1c4bd16733e248f3+        .quad   0xbd9e128715bf0a5f+        .quad   0xd43f8cf0a10b0376+        .quad   0x53b09b5ddf191b13+        .quad   0xc3877c60d2e7e3f2+        .quad   0x3b34aaa030828bb1+        .quad   0x283e26e7739ef138+        .quad   0x699c9c9002c30577+        .quad   0xf306a7235946f1cc+        .quad   0x921718b5cce5d97d+        .quad   0x28cdd24781b4e975+        .quad   0x51caf30c6fcdd907++        // 2^216 * 6 * G++        .quad   0xa60ba7427674e00a+        .quad   0x630e8570a17a7bf3+        .quad   0x3758563dcf3324cc+        .quad   0x5504aa292383fdaa+        .quad   0x737af99a18ac54c7+        .quad   0x903378dcc51cb30f+        .quad   0x2b89bc334ce10cc7+        .quad   0x12ae29c189f8e99a+        .quad   0xa99ec0cb1f0d01cf+        .quad   0x0dd1efcc3a34f7ae+        .quad   0x55ca7521d09c4e22+        .quad   0x5fd14fe958eba5ea++        // 2^216 * 7 * G++        .quad   0xb5dc2ddf2845ab2c+        .quad   0x069491b10a7fe993+        .quad   0x4daaf3d64002e346+        .quad   0x093ff26e586474d1+        .quad   0x3c42fe5ebf93cb8e+        .quad   0xbedfa85136d4565f+        .quad   0xe0f0859e884220e8+        .quad   0x7dd73f960725d128+        .quad   0xb10d24fe68059829+        .quad   0x75730672dbaf23e5+        .quad   0x1367253ab457ac29+        .quad   0x2f59bcbc86b470a4++        // 2^216 * 8 * G++        .quad   0x83847d429917135f+        .quad   0xad1b911f567d03d7+        .quad   0x7e7748d9be77aad1+        .quad   0x5458b42e2e51af4a+        .quad   0x7041d560b691c301+        .quad   0x85201b3fadd7e71e+        .quad   0x16c2e16311335585+        .quad   0x2aa55e3d010828b1+        .quad   0xed5192e60c07444f+        .quad   0x42c54e2d74421d10+        .quad   0x352b4c82fdb5c864+        .quad   0x13e9004a8a768664++        // 2^220 * 1 * G++        .quad   0xcbb5b5556c032bff+        .quad   0xdf7191b729297a3a+        .quad   0xc1ff7326aded81bb+        .quad   0x71ade8bb68be03f5+        .quad   0x1e6284c5806b467c+        .quad   0xc5f6997be75d607b+        .quad   0x8b67d958b378d262+        .quad   0x3d88d66a81cd8b70+        .quad   0x8b767a93204ed789+        .quad   0x762fcacb9fa0ae2a+        .quad   0x771febcc6dce4887+        .quad   0x343062158ff05fb3++        // 2^220 * 2 * G++        .quad   0xe05da1a7e1f5bf49+        .quad   0x26457d6dd4736092+        .quad   0x77dcb07773cc32f6+        .quad   0x0a5d94969cdd5fcd+        .quad   0xfce219072a7b31b4+        .quad   0x4d7adc75aa578016+        .quad   0x0ec276a687479324+        .quad   0x6d6d9d5d1fda4beb+        .quad   0x22b1a58ae9b08183+        .quad   0xfd95d071c15c388b+        .quad   0xa9812376850a0517+        .quad   0x33384cbabb7f335e++        // 2^220 * 3 * G++        .quad   0x3c6fa2680ca2c7b5+        .quad   0x1b5082046fb64fda+        .quad   0xeb53349c5431d6de+        .quad   0x5278b38f6b879c89+        .quad   0x33bc627a26218b8d+        .quad   0xea80b21fc7a80c61+        .quad   0x9458b12b173e9ee6+        .quad   0x076247be0e2f3059+        .quad   0x52e105f61416375a+        .quad   0xec97af3685abeba4+        .quad   0x26e6b50623a67c36+        .quad   0x5cf0e856f3d4fb01++        // 2^220 * 4 * G++        .quad   0xf6c968731ae8cab4+        .quad   0x5e20741ecb4f92c5+        .quad   0x2da53be58ccdbc3e+        .quad   0x2dddfea269970df7+        .quad   0xbeaece313db342a8+        .quad   0xcba3635b842db7ee+        .quad   0xe88c6620817f13ef+        .quad   0x1b9438aa4e76d5c6+        .quad   0x8a50777e166f031a+        .quad   0x067b39f10fb7a328+        .quad   0x1925c9a6010fbd76+        .quad   0x6df9b575cc740905++        // 2^220 * 5 * G++        .quad   0x42c1192927f6bdcf+        .quad   0x8f91917a403d61ca+        .quad   0xdc1c5a668b9e1f61+        .quad   0x1596047804ec0f8d+        .quad   0xecdfc35b48cade41+        .quad   0x6a88471fb2328270+        .quad   0x740a4a2440a01b6a+        .quad   0x471e5796003b5f29+        .quad   0xda96bbb3aced37ac+        .quad   0x7a2423b5e9208cea+        .quad   0x24cc5c3038aebae2+        .quad   0x50c356afdc5dae2f++        // 2^220 * 6 * G++        .quad   0x09dcbf4341c30318+        .quad   0xeeba061183181dce+        .quad   0xc179c0cedc1e29a1+        .quad   0x1dbf7b89073f35b0+        .quad   0xcfed9cdf1b31b964+        .quad   0xf486a9858ca51af3+        .quad   0x14897265ea8c1f84+        .quad   0x784a53dd932acc00+        .quad   0x2d99f9df14fc4920+        .quad   0x76ccb60cc4499fe5+        .quad   0xa4132cbbe5cf0003+        .quad   0x3f93d82354f000ea++        // 2^220 * 7 * G++        .quad   0x8183e7689e04ce85+        .quad   0x678fb71e04465341+        .quad   0xad92058f6688edac+        .quad   0x5da350d3532b099a+        .quad   0xeaac12d179e14978+        .quad   0xff923ff3bbebff5e+        .quad   0x4af663e40663ce27+        .quad   0x0fd381a811a5f5ff+        .quad   0xf256aceca436df54+        .quad   0x108b6168ae69d6e8+        .quad   0x20d986cb6b5d036c+        .quad   0x655957b9fee2af50++        // 2^220 * 8 * G++        .quad   0xaea8b07fa902030f+        .quad   0xf88c766af463d143+        .quad   0x15b083663c787a60+        .quad   0x08eab1148267a4a8+        .quad   0xbdc1409bd002d0ac+        .quad   0x66660245b5ccd9a6+        .quad   0x82317dc4fade85ec+        .quad   0x02fe934b6ad7df0d+        .quad   0xef5cf100cfb7ea74+        .quad   0x22897633a1cb42ac+        .quad   0xd4ce0c54cef285e2+        .quad   0x30408c048a146a55++        // 2^224 * 1 * G++        .quad   0x739d8845832fcedb+        .quad   0xfa38d6c9ae6bf863+        .quad   0x32bc0dcab74ffef7+        .quad   0x73937e8814bce45e+        .quad   0xbb2e00c9193b877f+        .quad   0xece3a890e0dc506b+        .quad   0xecf3b7c036de649f+        .quad   0x5f46040898de9e1a+        .quad   0xb9037116297bf48d+        .quad   0xa9d13b22d4f06834+        .quad   0xe19715574696bdc6+        .quad   0x2cf8a4e891d5e835++        // 2^224 * 2 * G++        .quad   0x6d93fd8707110f67+        .quad   0xdd4c09d37c38b549+        .quad   0x7cb16a4cc2736a86+        .quad   0x2049bd6e58252a09+        .quad   0x2cb5487e17d06ba2+        .quad   0x24d2381c3950196b+        .quad   0xd7659c8185978a30+        .quad   0x7a6f7f2891d6a4f6+        .quad   0x7d09fd8d6a9aef49+        .quad   0xf0ee60be5b3db90b+        .quad   0x4c21b52c519ebfd4+        .quad   0x6011aadfc545941d++        // 2^224 * 3 * G++        .quad   0x5f67926dcf95f83c+        .quad   0x7c7e856171289071+        .quad   0xd6a1e7f3998f7a5b+        .quad   0x6fc5cc1b0b62f9e0+        .quad   0x63ded0c802cbf890+        .quad   0xfbd098ca0dff6aaa+        .quad   0x624d0afdb9b6ed99+        .quad   0x69ce18b779340b1e+        .quad   0xd1ef5528b29879cb+        .quad   0xdd1aae3cd47e9092+        .quad   0x127e0442189f2352+        .quad   0x15596b3ae57101f1++        // 2^224 * 4 * G++        .quad   0x462739d23f9179a2+        .quad   0xff83123197d6ddcf+        .quad   0x1307deb553f2148a+        .quad   0x0d2237687b5f4dda+        .quad   0x09ff31167e5124ca+        .quad   0x0be4158bd9c745df+        .quad   0x292b7d227ef556e5+        .quad   0x3aa4e241afb6d138+        .quad   0x2cc138bf2a3305f5+        .quad   0x48583f8fa2e926c3+        .quad   0x083ab1a25549d2eb+        .quad   0x32fcaa6e4687a36c++        // 2^224 * 5 * G++        .quad   0x7bc56e8dc57d9af5+        .quad   0x3e0bd2ed9df0bdf2+        .quad   0xaac014de22efe4a3+        .quad   0x4627e9cefebd6a5c+        .quad   0x3207a4732787ccdf+        .quad   0x17e31908f213e3f8+        .quad   0xd5b2ecd7f60d964e+        .quad   0x746f6336c2600be9+        .quad   0x3f4af345ab6c971c+        .quad   0xe288eb729943731f+        .quad   0x33596a8a0344186d+        .quad   0x7b4917007ed66293++        // 2^224 * 6 * G++        .quad   0x2d85fb5cab84b064+        .quad   0x497810d289f3bc14+        .quad   0x476adc447b15ce0c+        .quad   0x122ba376f844fd7b+        .quad   0x54341b28dd53a2dd+        .quad   0xaa17905bdf42fc3f+        .quad   0x0ff592d94dd2f8f4+        .quad   0x1d03620fe08cd37d+        .quad   0xc20232cda2b4e554+        .quad   0x9ed0fd42115d187f+        .quad   0x2eabb4be7dd479d9+        .quad   0x02c70bf52b68ec4c++        // 2^224 * 7 * G++        .quad   0xa287ec4b5d0b2fbb+        .quad   0x415c5790074882ca+        .quad   0xe044a61ec1d0815c+        .quad   0x26334f0a409ef5e0+        .quad   0xace532bf458d72e1+        .quad   0x5be768e07cb73cb5+        .quad   0x56cf7d94ee8bbde7+        .quad   0x6b0697e3feb43a03+        .quad   0xb6c8f04adf62a3c0+        .quad   0x3ef000ef076da45d+        .quad   0x9c9cb95849f0d2a9+        .quad   0x1cc37f43441b2fae++        // 2^224 * 8 * G++        .quad   0x508f565a5cc7324f+        .quad   0xd061c4c0e506a922+        .quad   0xfb18abdb5c45ac19+        .quad   0x6c6809c10380314a+        .quad   0xd76656f1c9ceaeb9+        .quad   0x1c5b15f818e5656a+        .quad   0x26e72832844c2334+        .quad   0x3a346f772f196838+        .quad   0xd2d55112e2da6ac8+        .quad   0xe9bd0331b1e851ed+        .quad   0x960746dd8ec67262+        .quad   0x05911b9f6ef7c5d0++        // 2^228 * 1 * G++        .quad   0xe9dcd756b637ff2d+        .quad   0xec4c348fc987f0c4+        .quad   0xced59285f3fbc7b7+        .quad   0x3305354793e1ea87+        .quad   0x01c18980c5fe9f94+        .quad   0xcd656769716fd5c8+        .quad   0x816045c3d195a086+        .quad   0x6e2b7f3266cc7982+        .quad   0xcc802468f7c3568f+        .quad   0x9de9ba8219974cb3+        .quad   0xabb7229cb5b81360+        .quad   0x44e2017a6fbeba62++        // 2^228 * 2 * G++        .quad   0xc4c2a74354dab774+        .quad   0x8e5d4c3c4eaf031a+        .quad   0xb76c23d242838f17+        .quad   0x749a098f68dce4ea+        .quad   0x87f82cf3b6ca6ecd+        .quad   0x580f893e18f4a0c2+        .quad   0x058930072604e557+        .quad   0x6cab6ac256d19c1d+        .quad   0xdcdfe0a02cc1de60+        .quad   0x032665ff51c5575b+        .quad   0x2c0c32f1073abeeb+        .quad   0x6a882014cd7b8606++        // 2^228 * 3 * G++        .quad   0xa52a92fea4747fb5+        .quad   0xdc12a4491fa5ab89+        .quad   0xd82da94bb847a4ce+        .quad   0x4d77edce9512cc4e+        .quad   0xd111d17caf4feb6e+        .quad   0x050bba42b33aa4a3+        .quad   0x17514c3ceeb46c30+        .quad   0x54bedb8b1bc27d75+        .quad   0x77c8e14577e2189c+        .quad   0xa3e46f6aff99c445+        .quad   0x3144dfc86d335343+        .quad   0x3a96559e7c4216a9++        // 2^228 * 4 * G++        .quad   0x12550d37f42ad2ee+        .quad   0x8b78e00498a1fbf5+        .quad   0x5d53078233894cb2+        .quad   0x02c84e4e3e498d0c+        .quad   0x4493896880baaa52+        .quad   0x4c98afc4f285940e+        .quad   0xef4aa79ba45448b6+        .quad   0x5278c510a57aae7f+        .quad   0xa54dd074294c0b94+        .quad   0xf55d46b8df18ffb6+        .quad   0xf06fecc58dae8366+        .quad   0x588657668190d165++        // 2^228 * 5 * G++        .quad   0xd47712311aef7117+        .quad   0x50343101229e92c7+        .quad   0x7a95e1849d159b97+        .quad   0x2449959b8b5d29c9+        .quad   0xbf5834f03de25cc3+        .quad   0xb887c8aed6815496+        .quad   0x5105221a9481e892+        .quad   0x6760ed19f7723f93+        .quad   0x669ba3b7ac35e160+        .quad   0x2eccf73fba842056+        .quad   0x1aec1f17c0804f07+        .quad   0x0d96bc031856f4e7++        // 2^228 * 6 * G++        .quad   0x3318be7775c52d82+        .quad   0x4cb764b554d0aab9+        .quad   0xabcf3d27cc773d91+        .quad   0x3bf4d1848123288a+        .quad   0xb1d534b0cc7505e1+        .quad   0x32cd003416c35288+        .quad   0xcb36a5800762c29d+        .quad   0x5bfe69b9237a0bf8+        .quad   0x183eab7e78a151ab+        .quad   0xbbe990c999093763+        .quad   0xff717d6e4ac7e335+        .quad   0x4c5cddb325f39f88++        // 2^228 * 7 * G++        .quad   0xc0f6b74d6190a6eb+        .quad   0x20ea81a42db8f4e4+        .quad   0xa8bd6f7d97315760+        .quad   0x33b1d60262ac7c21+        .quad   0x57750967e7a9f902+        .quad   0x2c37fdfc4f5b467e+        .quad   0xb261663a3177ba46+        .quad   0x3a375e78dc2d532b+        .quad   0x8141e72f2d4dddea+        .quad   0xe6eafe9862c607c8+        .quad   0x23c28458573cafd0+        .quad   0x46b9476f4ff97346++        // 2^228 * 8 * G++        .quad   0x0c1ffea44f901e5c+        .quad   0x2b0b6fb72184b782+        .quad   0xe587ff910114db88+        .quad   0x37130f364785a142+        .quad   0x1215505c0d58359f+        .quad   0x2a2013c7fc28c46b+        .quad   0x24a0a1af89ea664e+        .quad   0x4400b638a1130e1f+        .quad   0x3a01b76496ed19c3+        .quad   0x31e00ab0ed327230+        .quad   0x520a885783ca15b1+        .quad   0x06aab9875accbec7++        // 2^232 * 1 * G++        .quad   0xc1339983f5df0ebb+        .quad   0xc0f3758f512c4cac+        .quad   0x2cf1130a0bb398e1+        .quad   0x6b3cecf9aa270c62+        .quad   0x5349acf3512eeaef+        .quad   0x20c141d31cc1cb49+        .quad   0x24180c07a99a688d+        .quad   0x555ef9d1c64b2d17+        .quad   0x36a770ba3b73bd08+        .quad   0x624aef08a3afbf0c+        .quad   0x5737ff98b40946f2+        .quad   0x675f4de13381749d++        // 2^232 * 2 * G++        .quad   0x0e2c52036b1782fc+        .quad   0x64816c816cad83b4+        .quad   0xd0dcbdd96964073e+        .quad   0x13d99df70164c520+        .quad   0xa12ff6d93bdab31d+        .quad   0x0725d80f9d652dfe+        .quad   0x019c4ff39abe9487+        .quad   0x60f450b882cd3c43+        .quad   0x014b5ec321e5c0ca+        .quad   0x4fcb69c9d719bfa2+        .quad   0x4e5f1c18750023a0+        .quad   0x1c06de9e55edac80++        // 2^232 * 3 * G++        .quad   0x990f7ad6a33ec4e2+        .quad   0x6608f938be2ee08e+        .quad   0x9ca143c563284515+        .quad   0x4cf38a1fec2db60d+        .quad   0xffd52b40ff6d69aa+        .quad   0x34530b18dc4049bb+        .quad   0x5e4a5c2fa34d9897+        .quad   0x78096f8e7d32ba2d+        .quad   0xa0aaaa650dfa5ce7+        .quad   0xf9c49e2a48b5478c+        .quad   0x4f09cc7d7003725b+        .quad   0x373cad3a26091abe++        // 2^232 * 4 * G++        .quad   0xb294634d82c9f57c+        .quad   0x1fcbfde124934536+        .quad   0x9e9c4db3418cdb5a+        .quad   0x0040f3d9454419fc+        .quad   0xf1bea8fb89ddbbad+        .quad   0x3bcb2cbc61aeaecb+        .quad   0x8f58a7bb1f9b8d9d+        .quad   0x21547eda5112a686+        .quad   0xdefde939fd5986d3+        .quad   0xf4272c89510a380c+        .quad   0xb72ba407bb3119b9+        .quad   0x63550a334a254df4++        // 2^232 * 5 * G++        .quad   0x6507d6edb569cf37+        .quad   0x178429b00ca52ee1+        .quad   0xea7c0090eb6bd65d+        .quad   0x3eea62c7daf78f51+        .quad   0x9bba584572547b49+        .quad   0xf305c6fae2c408e0+        .quad   0x60e8fa69c734f18d+        .quad   0x39a92bafaa7d767a+        .quad   0x9d24c713e693274e+        .quad   0x5f63857768dbd375+        .quad   0x70525560eb8ab39a+        .quad   0x68436a0665c9c4cd++        // 2^232 * 6 * G++        .quad   0xbc0235e8202f3f27+        .quad   0xc75c00e264f975b0+        .quad   0x91a4e9d5a38c2416+        .quad   0x17b6e7f68ab789f9+        .quad   0x1e56d317e820107c+        .quad   0xc5266844840ae965+        .quad   0xc1e0a1c6320ffc7a+        .quad   0x5373669c91611472+        .quad   0x5d2814ab9a0e5257+        .quad   0x908f2084c9cab3fc+        .quad   0xafcaf5885b2d1eca+        .quad   0x1cb4b5a678f87d11++        // 2^232 * 7 * G++        .quad   0xb664c06b394afc6c+        .quad   0x0c88de2498da5fb1+        .quad   0x4f8d03164bcad834+        .quad   0x330bca78de7434a2+        .quad   0x6b74aa62a2a007e7+        .quad   0xf311e0b0f071c7b1+        .quad   0x5707e438000be223+        .quad   0x2dc0fd2d82ef6eac+        .quad   0x982eff841119744e+        .quad   0xf9695e962b074724+        .quad   0xc58ac14fbfc953fb+        .quad   0x3c31be1b369f1cf5++        // 2^232 * 8 * G++        .quad   0xb0f4864d08948aee+        .quad   0x07dc19ee91ba1c6f+        .quad   0x7975cdaea6aca158+        .quad   0x330b61134262d4bb+        .quad   0xc168bc93f9cb4272+        .quad   0xaeb8711fc7cedb98+        .quad   0x7f0e52aa34ac8d7a+        .quad   0x41cec1097e7d55bb+        .quad   0xf79619d7a26d808a+        .quad   0xbb1fd49e1d9e156d+        .quad   0x73d7c36cdba1df27+        .quad   0x26b44cd91f28777d++        // 2^236 * 1 * G++        .quad   0x300a9035393aa6d8+        .quad   0x2b501131a12bb1cd+        .quad   0x7b1ff677f093c222+        .quad   0x4309c1f8cab82bad+        .quad   0xaf44842db0285f37+        .quad   0x8753189047efc8df+        .quad   0x9574e091f820979a+        .quad   0x0e378d6069615579+        .quad   0xd9fa917183075a55+        .quad   0x4bdb5ad26b009fdc+        .quad   0x7829ad2cd63def0e+        .quad   0x078fc54975fd3877++        // 2^236 * 2 * G++        .quad   0x87dfbd1428878f2d+        .quad   0x134636dd1e9421a1+        .quad   0x4f17c951257341a3+        .quad   0x5df98d4bad296cb8+        .quad   0xe2004b5bb833a98a+        .quad   0x44775dec2d4c3330+        .quad   0x3aa244067eace913+        .quad   0x272630e3d58e00a9+        .quad   0xf3678fd0ecc90b54+        .quad   0xf001459b12043599+        .quad   0x26725fbc3758b89b+        .quad   0x4325e4aa73a719ae++        // 2^236 * 3 * G++        .quad   0x657dc6ef433c3493+        .quad   0x65375e9f80dbf8c3+        .quad   0x47fd2d465b372dae+        .quad   0x4966ab79796e7947+        .quad   0xed24629acf69f59d+        .quad   0x2a4a1ccedd5abbf4+        .quad   0x3535ca1f56b2d67b+        .quad   0x5d8c68d043b1b42d+        .quad   0xee332d4de3b42b0a+        .quad   0xd84e5a2b16a4601c+        .quad   0x78243877078ba3e4+        .quad   0x77ed1eb4184ee437++        // 2^236 * 4 * G++        .quad   0xbfd4e13f201839a0+        .quad   0xaeefffe23e3df161+        .quad   0xb65b04f06b5d1fe3+        .quad   0x52e085fb2b62fbc0+        .quad   0x185d43f89e92ed1a+        .quad   0xb04a1eeafe4719c6+        .quad   0x499fbe88a6f03f4f+        .quad   0x5d8b0d2f3c859bdd+        .quad   0x124079eaa54cf2ba+        .quad   0xd72465eb001b26e7+        .quad   0x6843bcfdc97af7fd+        .quad   0x0524b42b55eacd02++        // 2^236 * 5 * G++        .quad   0xfd0d5dbee45447b0+        .quad   0x6cec351a092005ee+        .quad   0x99a47844567579cb+        .quad   0x59d242a216e7fa45+        .quad   0xbc18dcad9b829eac+        .quad   0x23ae7d28b5f579d0+        .quad   0xc346122a69384233+        .quad   0x1a6110b2e7d4ac89+        .quad   0x4f833f6ae66997ac+        .quad   0x6849762a361839a4+        .quad   0x6985dec1970ab525+        .quad   0x53045e89dcb1f546++        // 2^236 * 6 * G++        .quad   0xcb8bb346d75353db+        .quad   0xfcfcb24bae511e22+        .quad   0xcba48d40d50ae6ef+        .quad   0x26e3bae5f4f7cb5d+        .quad   0x84da3cde8d45fe12+        .quad   0xbd42c218e444e2d2+        .quad   0xa85196781f7e3598+        .quad   0x7642c93f5616e2b2+        .quad   0x2323daa74595f8e4+        .quad   0xde688c8b857abeb4+        .quad   0x3fc48e961c59326e+        .quad   0x0b2e73ca15c9b8ba++        // 2^236 * 7 * G++        .quad   0xd6bb4428c17f5026+        .quad   0x9eb27223fb5a9ca7+        .quad   0xe37ba5031919c644+        .quad   0x21ce380db59a6602+        .quad   0x0e3fbfaf79c03a55+        .quad   0x3077af054cbb5acf+        .quad   0xd5c55245db3de39f+        .quad   0x015e68c1476a4af7+        .quad   0xc1d5285220066a38+        .quad   0x95603e523570aef3+        .quad   0x832659a7226b8a4d+        .quad   0x5dd689091f8eedc9++        // 2^236 * 8 * G++        .quad   0xcbac84debfd3c856+        .quad   0x1624c348b35ff244+        .quad   0xb7f88dca5d9cad07+        .quad   0x3b0e574da2c2ebe8+        .quad   0x1d022591a5313084+        .quad   0xca2d4aaed6270872+        .quad   0x86a12b852f0bfd20+        .quad   0x56e6c439ad7da748+        .quad   0xc704ff4942bdbae6+        .quad   0x5e21ade2b2de1f79+        .quad   0xe95db3f35652fad8+        .quad   0x0822b5378f08ebc1++        // 2^240 * 1 * G++        .quad   0x51f048478f387475+        .quad   0xb25dbcf49cbecb3c+        .quad   0x9aab1244d99f2055+        .quad   0x2c709e6c1c10a5d6+        .quad   0xe1b7f29362730383+        .quad   0x4b5279ffebca8a2c+        .quad   0xdafc778abfd41314+        .quad   0x7deb10149c72610f+        .quad   0xcb62af6a8766ee7a+        .quad   0x66cbec045553cd0e+        .quad   0x588001380f0be4b5+        .quad   0x08e68e9ff62ce2ea++        // 2^240 * 2 * G++        .quad   0x34ad500a4bc130ad+        .quad   0x8d38db493d0bd49c+        .quad   0xa25c3d98500a89be+        .quad   0x2f1f3f87eeba3b09+        .quad   0x2f2d09d50ab8f2f9+        .quad   0xacb9218dc55923df+        .quad   0x4a8f342673766cb9+        .quad   0x4cb13bd738f719f5+        .quad   0xf7848c75e515b64a+        .quad   0xa59501badb4a9038+        .quad   0xc20d313f3f751b50+        .quad   0x19a1e353c0ae2ee8++        // 2^240 * 3 * G++        .quad   0x7d1c7560bafa05c3+        .quad   0xb3e1a0a0c6e55e61+        .quad   0xe3529718c0d66473+        .quad   0x41546b11c20c3486+        .quad   0xb42172cdd596bdbd+        .quad   0x93e0454398eefc40+        .quad   0x9fb15347b44109b5+        .quad   0x736bd3990266ae34+        .quad   0x85532d509334b3b4+        .quad   0x46fd114b60816573+        .quad   0xcc5f5f30425c8375+        .quad   0x412295a2b87fab5c++        // 2^240 * 4 * G++        .quad   0x19c99b88f57ed6e9+        .quad   0x5393cb266df8c825+        .quad   0x5cee3213b30ad273+        .quad   0x14e153ebb52d2e34+        .quad   0x2e655261e293eac6+        .quad   0x845a92032133acdb+        .quad   0x460975cb7900996b+        .quad   0x0760bb8d195add80+        .quad   0x413e1a17cde6818a+        .quad   0x57156da9ed69a084+        .quad   0x2cbf268f46caccb1+        .quad   0x6b34be9bc33ac5f2++        // 2^240 * 5 * G++        .quad   0xf3df2f643a78c0b2+        .quad   0x4c3e971ef22e027c+        .quad   0xec7d1c5e49c1b5a3+        .quad   0x2012c18f0922dd2d+        .quad   0x11fc69656571f2d3+        .quad   0xc6c9e845530e737a+        .quad   0xe33ae7a2d4fe5035+        .quad   0x01b9c7b62e6dd30b+        .quad   0x880b55e55ac89d29+        .quad   0x1483241f45a0a763+        .quad   0x3d36efdfc2e76c1f+        .quad   0x08af5b784e4bade8++        // 2^240 * 6 * G++        .quad   0x283499dc881f2533+        .quad   0x9d0525da779323b6+        .quad   0x897addfb673441f4+        .quad   0x32b79d71163a168d+        .quad   0xe27314d289cc2c4b+        .quad   0x4be4bd11a287178d+        .quad   0x18d528d6fa3364ce+        .quad   0x6423c1d5afd9826e+        .quad   0xcc85f8d9edfcb36a+        .quad   0x22bcc28f3746e5f9+        .quad   0xe49de338f9e5d3cd+        .quad   0x480a5efbc13e2dcc++        // 2^240 * 7 * G++        .quad   0x0b51e70b01622071+        .quad   0x06b505cf8b1dafc5+        .quad   0x2c6bb061ef5aabcd+        .quad   0x47aa27600cb7bf31+        .quad   0xb6614ce442ce221f+        .quad   0x6e199dcc4c053928+        .quad   0x663fb4a4dc1cbe03+        .quad   0x24b31d47691c8e06+        .quad   0x2a541eedc015f8c3+        .quad   0x11a4fe7e7c693f7c+        .quad   0xf0af66134ea278d6+        .quad   0x545b585d14dda094++        // 2^240 * 8 * G++        .quad   0x67bf275ea0d43a0f+        .quad   0xade68e34089beebe+        .quad   0x4289134cd479e72e+        .quad   0x0f62f9c332ba5454+        .quad   0x6204e4d0e3b321e1+        .quad   0x3baa637a28ff1e95+        .quad   0x0b0ccffd5b99bd9e+        .quad   0x4d22dc3e64c8d071+        .quad   0xfcb46589d63b5f39+        .quad   0x5cae6a3f57cbcf61+        .quad   0xfebac2d2953afa05+        .quad   0x1c0fa01a36371436++        // 2^244 * 1 * G++        .quad   0xe7547449bc7cd692+        .quad   0x0f9abeaae6f73ddf+        .quad   0x4af01ca700837e29+        .quad   0x63ab1b5d3f1bc183+        .quad   0xc11ee5e854c53fae+        .quad   0x6a0b06c12b4f3ff4+        .quad   0x33540f80e0b67a72+        .quad   0x15f18fc3cd07e3ef+        .quad   0x32750763b028f48c+        .quad   0x06020740556a065f+        .quad   0xd53bd812c3495b58+        .quad   0x08706c9b865f508d++        // 2^244 * 2 * G++        .quad   0xf37ca2ab3d343dff+        .quad   0x1a8c6a2d80abc617+        .quad   0x8e49e035d4ccffca+        .quad   0x48b46beebaa1d1b9+        .quad   0xcc991b4138b41246+        .quad   0x243b9c526f9ac26b+        .quad   0xb9ef494db7cbabbd+        .quad   0x5fba433dd082ed00+        .quad   0x9c49e355c9941ad0+        .quad   0xb9734ade74498f84+        .quad   0x41c3fed066663e5c+        .quad   0x0ecfedf8e8e710b3++        // 2^244 * 3 * G++        .quad   0x76430f9f9cd470d9+        .quad   0xb62acc9ba42f6008+        .quad   0x1898297c59adad5e+        .quad   0x7789dd2db78c5080+        .quad   0x744f7463e9403762+        .quad   0xf79a8dee8dfcc9c9+        .quad   0x163a649655e4cde3+        .quad   0x3b61788db284f435+        .quad   0xb22228190d6ef6b2+        .quad   0xa94a66b246ce4bfa+        .quad   0x46c1a77a4f0b6cc7+        .quad   0x4236ccffeb7338cf++        // 2^244 * 4 * G++        .quad   0x8497404d0d55e274+        .quad   0x6c6663d9c4ad2b53+        .quad   0xec2fb0d9ada95734+        .quad   0x2617e120cdb8f73c+        .quad   0x3bd82dbfda777df6+        .quad   0x71b177cc0b98369e+        .quad   0x1d0e8463850c3699+        .quad   0x5a71945b48e2d1f1+        .quad   0x6f203dd5405b4b42+        .quad   0x327ec60410b24509+        .quad   0x9c347230ac2a8846+        .quad   0x77de29fc11ffeb6a++        // 2^244 * 5 * G++        .quad   0xb0ac57c983b778a8+        .quad   0x53cdcca9d7fe912c+        .quad   0x61c2b854ff1f59dc+        .quad   0x3a1a2cf0f0de7dac+        .quad   0x835e138fecced2ca+        .quad   0x8c9eaf13ea963b9a+        .quad   0xc95fbfc0b2160ea6+        .quad   0x575e66f3ad877892+        .quad   0x99803a27c88fcb3a+        .quad   0x345a6789275ec0b0+        .quad   0x459789d0ff6c2be5+        .quad   0x62f882651e70a8b2++        // 2^244 * 6 * G++        .quad   0x085ae2c759ff1be4+        .quad   0x149145c93b0e40b7+        .quad   0xc467e7fa7ff27379+        .quad   0x4eeecf0ad5c73a95+        .quad   0x6d822986698a19e0+        .quad   0xdc9821e174d78a71+        .quad   0x41a85f31f6cb1f47+        .quad   0x352721c2bcda9c51+        .quad   0x48329952213fc985+        .quad   0x1087cf0d368a1746+        .quad   0x8e5261b166c15aa5+        .quad   0x2d5b2d842ed24c21++        // 2^244 * 7 * G++        .quad   0x02cfebd9ebd3ded1+        .quad   0xd45b217739021974+        .quad   0x7576f813fe30a1b7+        .quad   0x5691b6f9a34ef6c2+        .quad   0x5eb7d13d196ac533+        .quad   0x377234ecdb80be2b+        .quad   0xe144cffc7cf5ae24+        .quad   0x5226bcf9c441acec+        .quad   0x79ee6c7223e5b547+        .quad   0x6f5f50768330d679+        .quad   0xed73e1e96d8adce9+        .quad   0x27c3da1e1d8ccc03++        // 2^244 * 8 * G++        .quad   0x7eb9efb23fe24c74+        .quad   0x3e50f49f1651be01+        .quad   0x3ea732dc21858dea+        .quad   0x17377bd75bb810f9+        .quad   0x28302e71630ef9f6+        .quad   0xc2d4a2032b64cee0+        .quad   0x090820304b6292be+        .quad   0x5fca747aa82adf18+        .quad   0x232a03c35c258ea5+        .quad   0x86f23a2c6bcb0cf1+        .quad   0x3dad8d0d2e442166+        .quad   0x04a8933cab76862b++        // 2^248 * 1 * G++        .quad   0xd2c604b622943dff+        .quad   0xbc8cbece44cfb3a0+        .quad   0x5d254ff397808678+        .quad   0x0fa3614f3b1ca6bf+        .quad   0x69082b0e8c936a50+        .quad   0xf9c9a035c1dac5b6+        .quad   0x6fb73e54c4dfb634+        .quad   0x4005419b1d2bc140+        .quad   0xa003febdb9be82f0+        .quad   0x2089c1af3a44ac90+        .quad   0xf8499f911954fa8e+        .quad   0x1fba218aef40ab42++        // 2^248 * 2 * G++        .quad   0xab549448fac8f53e+        .quad   0x81f6e89a7ba63741+        .quad   0x74fd6c7d6c2b5e01+        .quad   0x392e3acaa8c86e42+        .quad   0x4f3e57043e7b0194+        .quad   0xa81d3eee08daaf7f+        .quad   0xc839c6ab99dcdef1+        .quad   0x6c535d13ff7761d5+        .quad   0x4cbd34e93e8a35af+        .quad   0x2e0781445887e816+        .quad   0x19319c76f29ab0ab+        .quad   0x25e17fe4d50ac13b++        // 2^248 * 3 * G++        .quad   0x0a289bd71e04f676+        .quad   0x208e1c52d6420f95+        .quad   0x5186d8b034691fab+        .quad   0x255751442a9fb351+        .quad   0x915f7ff576f121a7+        .quad   0xc34a32272fcd87e3+        .quad   0xccba2fde4d1be526+        .quad   0x6bba828f8969899b+        .quad   0xe2d1bc6690fe3901+        .quad   0x4cb54a18a0997ad5+        .quad   0x971d6914af8460d4+        .quad   0x559d504f7f6b7be4++        // 2^248 * 4 * G++        .quad   0xa7738378b3eb54d5+        .quad   0x1d69d366a5553c7c+        .quad   0x0a26cf62f92800ba+        .quad   0x01ab12d5807e3217+        .quad   0x9c4891e7f6d266fd+        .quad   0x0744a19b0307781b+        .quad   0x88388f1d6061e23b+        .quad   0x123ea6a3354bd50e+        .quad   0x118d189041e32d96+        .quad   0xb9ede3c2d8315848+        .quad   0x1eab4271d83245d9+        .quad   0x4a3961e2c918a154++        // 2^248 * 5 * G++        .quad   0x71dc3be0f8e6bba0+        .quad   0xd6cef8347effe30a+        .quad   0xa992425fe13a476a+        .quad   0x2cd6bce3fb1db763+        .quad   0x0327d644f3233f1e+        .quad   0x499a260e34fcf016+        .quad   0x83b5a716f2dab979+        .quad   0x68aceead9bd4111f+        .quad   0x38b4c90ef3d7c210+        .quad   0x308e6e24b7ad040c+        .quad   0x3860d9f1b7e73e23+        .quad   0x595760d5b508f597++        // 2^248 * 6 * G++        .quad   0x6129bfe104aa6397+        .quad   0x8f960008a4a7fccb+        .quad   0x3f8bc0897d909458+        .quad   0x709fa43edcb291a9+        .quad   0x882acbebfd022790+        .quad   0x89af3305c4115760+        .quad   0x65f492e37d3473f4+        .quad   0x2cb2c5df54515a2b+        .quad   0xeb0a5d8c63fd2aca+        .quad   0xd22bc1662e694eff+        .quad   0x2723f36ef8cbb03a+        .quad   0x70f029ecf0c8131f++        // 2^248 * 7 * G++        .quad   0x461307b32eed3e33+        .quad   0xae042f33a45581e7+        .quad   0xc94449d3195f0366+        .quad   0x0b7d5d8a6c314858+        .quad   0x2a6aafaa5e10b0b9+        .quad   0x78f0a370ef041aa9+        .quad   0x773efb77aa3ad61f+        .quad   0x44eca5a2a74bd9e1+        .quad   0x25d448327b95d543+        .quad   0x70d38300a3340f1d+        .quad   0xde1c531c60e1c52b+        .quad   0x272224512c7de9e4++        // 2^248 * 8 * G++        .quad   0x1abc92af49c5342e+        .quad   0xffeed811b2e6fad0+        .quad   0xefa28c8dfcc84e29+        .quad   0x11b5df18a44cc543+        .quad   0xbf7bbb8a42a975fc+        .quad   0x8c5c397796ada358+        .quad   0xe27fc76fcdedaa48+        .quad   0x19735fd7f6bc20a6+        .quad   0xe3ab90d042c84266+        .quad   0xeb848e0f7f19547e+        .quad   0x2503a1d065a497b9+        .quad   0x0fef911191df895f++        // 2^252 * 1 * G++        .quad   0xb1507ca1ab1c6eb9+        .quad   0xbd448f3e16b687b3+        .quad   0x3455fb7f2c7a91ab+        .quad   0x7579229e2f2adec1+        .quad   0x6ab5dcb85b1c16b7+        .quad   0x94c0fce83c7b27a5+        .quad   0xa4b11c1a735517be+        .quad   0x499238d0ba0eafaa+        .quad   0xecf46e527aba8b57+        .quad   0x15a08c478bd1647b+        .quad   0x7af1c6a65f706fef+        .quad   0x6345fa78f03a30d5++        // 2^252 * 2 * G++        .quad   0xdf02f95f1015e7a1+        .quad   0x790ec41da9b40263+        .quad   0x4d3a0ea133ea1107+        .quad   0x54f70be7e33af8c9+        .quad   0x93d3cbe9bdd8f0a4+        .quad   0xdb152c1bfd177302+        .quad   0x7dbddc6d7f17a875+        .quad   0x3e1a71cc8f426efe+        .quad   0xc83ca3e390babd62+        .quad   0x80ede3670291c833+        .quad   0xc88038ccd37900c4+        .quad   0x2c5fc0231ec31fa1++        // 2^252 * 3 * G++        .quad   0xfeba911717038b4f+        .quad   0xe5123721c9deef81+        .quad   0x1c97e4e75d0d8834+        .quad   0x68afae7a23dc3bc6+        .quad   0xc422e4d102456e65+        .quad   0x87414ac1cad47b91+        .quad   0x1592e2bba2b6ffdd+        .quad   0x75d9d2bff5c2100f+        .quad   0x5bd9b4763626e81c+        .quad   0x89966936bca02edd+        .quad   0x0a41193d61f077b3+        .quad   0x3097a24200ce5471++        // 2^252 * 4 * G++        .quad   0x57427734c7f8b84c+        .quad   0xf141a13e01b270e9+        .quad   0x02d1adfeb4e564a6+        .quad   0x4bb23d92ce83bd48+        .quad   0xa162e7246695c486+        .quad   0x131d633435a89607+        .quad   0x30521561a0d12a37+        .quad   0x56704bada6afb363+        .quad   0xaf6c4aa752f912b9+        .quad   0x5e665f6cd86770c8+        .quad   0x4c35ac83a3c8cd58+        .quad   0x2b7a29c010a58a7e++        // 2^252 * 5 * G++        .quad   0xc4007f77d0c1cec3+        .quad   0x8d1020b6bac492f8+        .quad   0x32ec29d57e69daaf+        .quad   0x599408759d95fce0+        .quad   0x33810a23bf00086e+        .quad   0xafce925ee736ff7c+        .quad   0x3d60e670e24922d4+        .quad   0x11ce9e714f96061b+        .quad   0x219ef713d815bac1+        .quad   0xf141465d485be25c+        .quad   0x6d5447cc4e513c51+        .quad   0x174926be5ef44393++        // 2^252 * 6 * G++        .quad   0xb5deb2f9fc5bd5bb+        .quad   0x92daa72ae1d810e1+        .quad   0xafc4cfdcb72a1c59+        .quad   0x497d78813fc22a24+        .quad   0x3ef5d41593ea022e+        .quad   0x5cbcc1a20ed0eed6+        .quad   0x8fd24ecf07382c8c+        .quad   0x6fa42ead06d8e1ad+        .quad   0xe276824a1f73371f+        .quad   0x7f7cf01c4f5b6736+        .quad   0x7e201fe304fa46e7+        .quad   0x785a36a357808c96++        // 2^252 * 7 * G++        .quad   0x825fbdfd63014d2b+        .quad   0xc852369c6ca7578b+        .quad   0x5b2fcd285c0b5df0+        .quad   0x12ab214c58048c8f+        .quad   0x070442985d517bc3+        .quad   0x6acd56c7ae653678+        .quad   0x00a27983985a7763+        .quad   0x5167effae512662b+        .quad   0xbd4ea9e10f53c4b6+        .quad   0x1673dc5f8ac91a14+        .quad   0xa8f81a4e2acc1aba+        .quad   0x33a92a7924332a25++        // 2^252 * 8 * G++        .quad   0x9dd1f49927996c02+        .quad   0x0cb3b058e04d1752+        .quad   0x1f7e88967fd02c3e+        .quad   0x2f964268cb8b3eb1+        .quad   0x7ba95ba0218f2ada+        .quad   0xcff42287330fb9ca+        .quad   0xdada496d56c6d907+        .quad   0x5380c296f4beee54+        .quad   0x9d4f270466898d0a+        .quad   0x3d0987990aff3f7a+        .quad   0xd09ef36267daba45+        .quad   0x7761455e7b1c669c
+ cbits/s2n/arm/edwards25519_encode.S view
@@ -0,0 +1,136 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Encode edwards25519 point into compressed form as 256-bit number+// Input p[8]; output z[32] (bytes)+//+//    extern void edwards25519_encode(uint8_t z[static 32],+//                                    const uint64_t p[static 8]);+//+// This assumes that the input buffer p points to a pair of 256-bit+// numbers x (at p) and y (at p+4) representing a point (x,y) on the+// edwards25519 curve. It is assumed that both x and y are < p_25519+// but there is no checking of this, nor of the fact that (x,y) is+// in fact on the curve.+//+// The output in z is a little-endian array of bytes corresponding to+// the standard compressed encoding of a point as 2^255 * x_0 + y+// where x_0 is the least significant bit of x.+// See "https://datatracker.ietf.org/doc/html/rfc8032#section-5.1.2"+// In this implementation, y is simply truncated to 255 bits, but if+// it is reduced mod p_25519 as expected this does not affect values.+//+// Standard ARM ABI: X0 = z, X1 = p+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(edwards25519_encode)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(edwards25519_encode)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(edwards25519_encode)+        .text+        .balign 4++#define z x0+#define p x1++#define y0 x2+#define y1 x3+#define y2 x4+#define y3 x5+#define y0short w2+#define y1short w3+#define y2short w4+#define y3short w5+#define xb x6++S2N_BN_SYMBOL(edwards25519_encode):+        CFI_START++// Load lowest word of x coordinate in xb and full y as [y3;y2;y1;y0].++        ldr     xb, [p]+        ldp     y0, y1, [p, #32]+        ldp     y2, y3, [p, #48]++// Compute the encoded form, making the LSB of x the MSB of the encoding++        and     y3, y3, #0x7FFFFFFFFFFFFFFF+        orr     y3, y3, xb, lsl #63++// Write back in a byte-oriented fashion to be independent of endianness++        strb    y0short, [z]+        lsr     y0, y0, #8+        strb    y0short, [z, #1]+        lsr     y0, y0, #8+        strb    y0short, [z, #2]+        lsr     y0, y0, #8+        strb    y0short, [z, #3]+        lsr     y0, y0, #8+        strb    y0short, [z, #4]+        lsr     y0, y0, #8+        strb    y0short, [z, #5]+        lsr     y0, y0, #8+        strb    y0short, [z, #6]+        lsr     y0, y0, #8+        strb    y0short, [z, #7]++        strb    y1short, [z, #8]+        lsr     y1, y1, #8+        strb    y1short, [z, #9]+        lsr     y1, y1, #8+        strb    y1short, [z, #10]+        lsr     y1, y1, #8+        strb    y1short, [z, #11]+        lsr     y1, y1, #8+        strb    y1short, [z, #12]+        lsr     y1, y1, #8+        strb    y1short, [z, #13]+        lsr     y1, y1, #8+        strb    y1short, [z, #14]+        lsr     y1, y1, #8+        strb    y1short, [z, #15]++        strb    y2short, [z, #16]+        lsr     y2, y2, #8+        strb    y2short, [z, #17]+        lsr     y2, y2, #8+        strb    y2short, [z, #18]+        lsr     y2, y2, #8+        strb    y2short, [z, #19]+        lsr     y2, y2, #8+        strb    y2short, [z, #20]+        lsr     y2, y2, #8+        strb    y2short, [z, #21]+        lsr     y2, y2, #8+        strb    y2short, [z, #22]+        lsr     y2, y2, #8+        strb    y2short, [z, #23]++        strb    y3short, [z, #24]+        lsr     y3, y3, #8+        strb    y3short, [z, #25]+        lsr     y3, y3, #8+        strb    y3short, [z, #26]+        lsr     y3, y3, #8+        strb    y3short, [z, #27]+        lsr     y3, y3, #8+        strb    y3short, [z, #28]+        lsr     y3, y3, #8+        strb    y3short, [z, #29]+        lsr     y3, y3, #8+        strb    y3short, [z, #30]+        lsr     y3, y3, #8+        strb    y3short, [z, #31]++// Return++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(edwards25519_encode)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/arm/edwards25519_scalarmulbase.S view
@@ -0,0 +1,9635 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Scalar multiplication for the edwards25519 standard basepoint+// Input scalar[4]; output res[8]+//+// extern void edwards25519_scalarmulbase+//   (uint64_t res[static 8],const uint64_t scalar[static 4]);+//+// Given a scalar n, returns point (X,Y) = n * B where B = (...,4/5) is+// the standard basepoint for the edwards25519 (Ed25519) curve.+//+// Standard ARM ABI: X0 = res, X1 = scalar+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(edwards25519_scalarmulbase)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(edwards25519_scalarmulbase)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(edwards25519_scalarmulbase)++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Stable home for the input result argument during the whole body++#define res x23++// Other variables that are only needed prior to the modular inverse.++#define tab x19++#define i x20++#define bias x21++#define bf x22+#define ix x22++// Pointer-offset pairs for result and temporaries on stack with some aliasing.++#define resx res, #(0*NUMSIZE)+#define resy res, #(1*NUMSIZE)++#define scalar sp, #(0*NUMSIZE)++#define tabent sp, #(1*NUMSIZE)+#define ymx_2 sp, #(1*NUMSIZE)+#define xpy_2 sp, #(2*NUMSIZE)+#define kxy_2 sp, #(3*NUMSIZE)++#define acc sp, #(4*NUMSIZE)+#define x_1 sp, #(4*NUMSIZE)+#define y_1 sp, #(5*NUMSIZE)+#define z_1 sp, #(6*NUMSIZE)+#define w_1 sp, #(7*NUMSIZE)+#define x_3 sp, #(4*NUMSIZE)+#define y_3 sp, #(5*NUMSIZE)+#define z_3 sp, #(6*NUMSIZE)+#define w_3 sp, #(7*NUMSIZE)++#define tmpspace sp, #(8*NUMSIZE)+#define t0 sp, #(8*NUMSIZE)+#define t1 sp, #(9*NUMSIZE)+#define t2 sp, #(10*NUMSIZE)+#define t3 sp, #(11*NUMSIZE)+#define t4 sp, #(12*NUMSIZE)+#define t5 sp, #(13*NUMSIZE)++// Total size to reserve on the stack++#define NSPACE 14*NUMSIZE++// Load 64-bit immediate into a register++#define movbig(nn,n3,n2,n1,n0)                                      \+        movz    nn, n0 __LF                                            \+        movk    nn, n1, lsl #16 __LF                                   \+        movk    nn, n2, lsl #32 __LF                                   \+        movk    nn, n3, lsl #48++// Macro wrapping up the basic field operation bignum_mul_p25519, only+// trivially different from a pure function call to that subroutine.++#define mul_p25519(P0,P1,P2)                    \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x5, x6, [P2] __LF                  \+        umull   x7, w3, w5 __LF                    \+        lsr     x0, x3, #32 __LF                   \+        umull   x15, w0, w5 __LF                   \+        lsr     x16, x5, #32 __LF                  \+        umull   x8, w16, w0 __LF                   \+        umull   x16, w3, w16 __LF                  \+        adds    x7, x7, x15, lsl #32 __LF          \+        lsr     x15, x15, #32 __LF                 \+        adc     x8, x8, x15 __LF                   \+        adds    x7, x7, x16, lsl #32 __LF          \+        lsr     x16, x16, #32 __LF                 \+        adc     x8, x8, x16 __LF                   \+        mul     x9, x4, x6 __LF                    \+        umulh   x10, x4, x6 __LF                   \+        subs    x4, x4, x3 __LF                    \+        cneg    x4, x4, cc __LF                    \+        csetm   x16, cc __LF                       \+        adds    x9, x9, x8 __LF                    \+        adc     x10, x10, xzr __LF                 \+        subs    x3, x5, x6 __LF                    \+        cneg    x3, x3, cc __LF                    \+        cinv    x16, x16, cc __LF                  \+        mul     x15, x4, x3 __LF                   \+        umulh   x3, x4, x3 __LF                    \+        adds    x8, x7, x9 __LF                    \+        adcs    x9, x9, x10 __LF                   \+        adc     x10, x10, xzr __LF                 \+        cmn     x16, #0x1 __LF                     \+        eor     x15, x15, x16 __LF                 \+        adcs    x8, x15, x8 __LF                   \+        eor     x3, x3, x16 __LF                   \+        adcs    x9, x3, x9 __LF                    \+        adc     x10, x10, x16 __LF                 \+        ldp     x3, x4, [P1+16] __LF               \+        ldp     x5, x6, [P2+16] __LF               \+        umull   x11, w3, w5 __LF                   \+        lsr     x0, x3, #32 __LF                   \+        umull   x15, w0, w5 __LF                   \+        lsr     x16, x5, #32 __LF                  \+        umull   x12, w16, w0 __LF                  \+        umull   x16, w3, w16 __LF                  \+        adds    x11, x11, x15, lsl #32 __LF        \+        lsr     x15, x15, #32 __LF                 \+        adc     x12, x12, x15 __LF                 \+        adds    x11, x11, x16, lsl #32 __LF        \+        lsr     x16, x16, #32 __LF                 \+        adc     x12, x12, x16 __LF                 \+        mul     x13, x4, x6 __LF                   \+        umulh   x14, x4, x6 __LF                   \+        subs    x4, x4, x3 __LF                    \+        cneg    x4, x4, cc __LF                    \+        csetm   x16, cc __LF                       \+        adds    x13, x13, x12 __LF                 \+        adc     x14, x14, xzr __LF                 \+        subs    x3, x5, x6 __LF                    \+        cneg    x3, x3, cc __LF                    \+        cinv    x16, x16, cc __LF                  \+        mul     x15, x4, x3 __LF                   \+        umulh   x3, x4, x3 __LF                    \+        adds    x12, x11, x13 __LF                 \+        adcs    x13, x13, x14 __LF                 \+        adc     x14, x14, xzr __LF                 \+        cmn     x16, #0x1 __LF                     \+        eor     x15, x15, x16 __LF                 \+        adcs    x12, x15, x12 __LF                 \+        eor     x3, x3, x16 __LF                   \+        adcs    x13, x3, x13 __LF                  \+        adc     x14, x14, x16 __LF                 \+        ldp     x3, x4, [P1+16] __LF               \+        ldp     x15, x16, [P1] __LF                \+        subs    x3, x3, x15 __LF                   \+        sbcs    x4, x4, x16 __LF                   \+        csetm   x16, cc __LF                       \+        ldp     x15, x0, [P2] __LF                 \+        subs    x5, x15, x5 __LF                   \+        sbcs    x6, x0, x6 __LF                    \+        csetm   x0, cc __LF                        \+        eor     x3, x3, x16 __LF                   \+        subs    x3, x3, x16 __LF                   \+        eor     x4, x4, x16 __LF                   \+        sbc     x4, x4, x16 __LF                   \+        eor     x5, x5, x0 __LF                    \+        subs    x5, x5, x0 __LF                    \+        eor     x6, x6, x0 __LF                    \+        sbc     x6, x6, x0 __LF                    \+        eor     x16, x0, x16 __LF                  \+        adds    x11, x11, x9 __LF                  \+        adcs    x12, x12, x10 __LF                 \+        adcs    x13, x13, xzr __LF                 \+        adc     x14, x14, xzr __LF                 \+        mul     x2, x3, x5 __LF                    \+        umulh   x0, x3, x5 __LF                    \+        mul     x15, x4, x6 __LF                   \+        umulh   x1, x4, x6 __LF                    \+        subs    x4, x4, x3 __LF                    \+        cneg    x4, x4, cc __LF                    \+        csetm   x9, cc __LF                        \+        adds    x15, x15, x0 __LF                  \+        adc     x1, x1, xzr __LF                   \+        subs    x6, x5, x6 __LF                    \+        cneg    x6, x6, cc __LF                    \+        cinv    x9, x9, cc __LF                    \+        mul     x5, x4, x6 __LF                    \+        umulh   x6, x4, x6 __LF                    \+        adds    x0, x2, x15 __LF                   \+        adcs    x15, x15, x1 __LF                  \+        adc     x1, x1, xzr __LF                   \+        cmn     x9, #0x1 __LF                      \+        eor     x5, x5, x9 __LF                    \+        adcs    x0, x5, x0 __LF                    \+        eor     x6, x6, x9 __LF                    \+        adcs    x15, x6, x15 __LF                  \+        adc     x1, x1, x9 __LF                    \+        adds    x9, x11, x7 __LF                   \+        adcs    x10, x12, x8 __LF                  \+        adcs    x11, x13, x11 __LF                 \+        adcs    x12, x14, x12 __LF                 \+        adcs    x13, x13, xzr __LF                 \+        adc     x14, x14, xzr __LF                 \+        cmn     x16, #0x1 __LF                     \+        eor     x2, x2, x16 __LF                   \+        adcs    x9, x2, x9 __LF                    \+        eor     x0, x0, x16 __LF                   \+        adcs    x10, x0, x10 __LF                  \+        eor     x15, x15, x16 __LF                 \+        adcs    x11, x15, x11 __LF                 \+        eor     x1, x1, x16 __LF                   \+        adcs    x12, x1, x12 __LF                  \+        adcs    x13, x13, x16 __LF                 \+        adc     x14, x14, x16 __LF                 \+        mov     x3, #0x26 __LF                     \+        umull   x4, w11, w3 __LF                   \+        add     x4, x4, w7, uxtw __LF              \+        lsr     x7, x7, #32 __LF                   \+        lsr     x11, x11, #32 __LF                 \+        umaddl  x11, w11, w3, x7 __LF              \+        mov     x7, x4 __LF                        \+        umull   x4, w12, w3 __LF                   \+        add     x4, x4, w8, uxtw __LF              \+        lsr     x8, x8, #32 __LF                   \+        lsr     x12, x12, #32 __LF                 \+        umaddl  x12, w12, w3, x8 __LF              \+        mov     x8, x4 __LF                        \+        umull   x4, w13, w3 __LF                   \+        add     x4, x4, w9, uxtw __LF              \+        lsr     x9, x9, #32 __LF                   \+        lsr     x13, x13, #32 __LF                 \+        umaddl  x13, w13, w3, x9 __LF              \+        mov     x9, x4 __LF                        \+        umull   x4, w14, w3 __LF                   \+        add     x4, x4, w10, uxtw __LF             \+        lsr     x10, x10, #32 __LF                 \+        lsr     x14, x14, #32 __LF                 \+        umaddl  x14, w14, w3, x10 __LF             \+        mov     x10, x4 __LF                       \+        lsr     x0, x14, #31 __LF                  \+        mov     x5, #0x13 __LF                     \+        umaddl  x5, w5, w0, x5 __LF                \+        add     x7, x7, x5 __LF                    \+        adds    x7, x7, x11, lsl #32 __LF          \+        extr    x3, x12, x11, #32 __LF             \+        adcs    x8, x8, x3 __LF                    \+        extr    x3, x13, x12, #32 __LF             \+        adcs    x9, x9, x3 __LF                    \+        extr    x3, x14, x13, #32 __LF             \+        lsl     x5, x0, #63 __LF                   \+        eor     x10, x10, x5 __LF                  \+        adc     x10, x10, x3 __LF                  \+        mov     x3, #0x13 __LF                     \+        tst     x10, #0x8000000000000000 __LF      \+        csel    x3, x3, xzr, pl __LF               \+        subs    x7, x7, x3 __LF                    \+        sbcs    x8, x8, xzr __LF                   \+        sbcs    x9, x9, xzr __LF                   \+        sbc     x10, x10, xzr __LF                 \+        and     x10, x10, #0x7fffffffffffffff __LF \+        stp     x7, x8, [P0] __LF                  \+        stp     x9, x10, [P0+16]++// A version of multiplication that only guarantees output < 2 * p_25519.+// This basically skips the +1 and final correction in quotient estimation.++#define mul_4(P0,P1,P2)                         \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x5, x6, [P2] __LF                  \+        umull   x7, w3, w5 __LF                    \+        lsr     x0, x3, #32 __LF                   \+        umull   x15, w0, w5 __LF                   \+        lsr     x16, x5, #32 __LF                  \+        umull   x8, w16, w0 __LF                   \+        umull   x16, w3, w16 __LF                  \+        adds    x7, x7, x15, lsl #32 __LF          \+        lsr     x15, x15, #32 __LF                 \+        adc     x8, x8, x15 __LF                   \+        adds    x7, x7, x16, lsl #32 __LF          \+        lsr     x16, x16, #32 __LF                 \+        adc     x8, x8, x16 __LF                   \+        mul     x9, x4, x6 __LF                    \+        umulh   x10, x4, x6 __LF                   \+        subs    x4, x4, x3 __LF                    \+        cneg    x4, x4, cc __LF                    \+        csetm   x16, cc __LF                       \+        adds    x9, x9, x8 __LF                    \+        adc     x10, x10, xzr __LF                 \+        subs    x3, x5, x6 __LF                    \+        cneg    x3, x3, cc __LF                    \+        cinv    x16, x16, cc __LF                  \+        mul     x15, x4, x3 __LF                   \+        umulh   x3, x4, x3 __LF                    \+        adds    x8, x7, x9 __LF                    \+        adcs    x9, x9, x10 __LF                   \+        adc     x10, x10, xzr __LF                 \+        cmn     x16, #0x1 __LF                     \+        eor     x15, x15, x16 __LF                 \+        adcs    x8, x15, x8 __LF                   \+        eor     x3, x3, x16 __LF                   \+        adcs    x9, x3, x9 __LF                    \+        adc     x10, x10, x16 __LF                 \+        ldp     x3, x4, [P1+16] __LF               \+        ldp     x5, x6, [P2+16] __LF               \+        umull   x11, w3, w5 __LF                   \+        lsr     x0, x3, #32 __LF                   \+        umull   x15, w0, w5 __LF                   \+        lsr     x16, x5, #32 __LF                  \+        umull   x12, w16, w0 __LF                  \+        umull   x16, w3, w16 __LF                  \+        adds    x11, x11, x15, lsl #32 __LF        \+        lsr     x15, x15, #32 __LF                 \+        adc     x12, x12, x15 __LF                 \+        adds    x11, x11, x16, lsl #32 __LF        \+        lsr     x16, x16, #32 __LF                 \+        adc     x12, x12, x16 __LF                 \+        mul     x13, x4, x6 __LF                   \+        umulh   x14, x4, x6 __LF                   \+        subs    x4, x4, x3 __LF                    \+        cneg    x4, x4, cc __LF                    \+        csetm   x16, cc __LF                       \+        adds    x13, x13, x12 __LF                 \+        adc     x14, x14, xzr __LF                 \+        subs    x3, x5, x6 __LF                    \+        cneg    x3, x3, cc __LF                    \+        cinv    x16, x16, cc __LF                  \+        mul     x15, x4, x3 __LF                   \+        umulh   x3, x4, x3 __LF                    \+        adds    x12, x11, x13 __LF                 \+        adcs    x13, x13, x14 __LF                 \+        adc     x14, x14, xzr __LF                 \+        cmn     x16, #0x1 __LF                     \+        eor     x15, x15, x16 __LF                 \+        adcs    x12, x15, x12 __LF                 \+        eor     x3, x3, x16 __LF                   \+        adcs    x13, x3, x13 __LF                  \+        adc     x14, x14, x16 __LF                 \+        ldp     x3, x4, [P1+16] __LF               \+        ldp     x15, x16, [P1] __LF                \+        subs    x3, x3, x15 __LF                   \+        sbcs    x4, x4, x16 __LF                   \+        csetm   x16, cc __LF                       \+        ldp     x15, x0, [P2] __LF                 \+        subs    x5, x15, x5 __LF                   \+        sbcs    x6, x0, x6 __LF                    \+        csetm   x0, cc __LF                        \+        eor     x3, x3, x16 __LF                   \+        subs    x3, x3, x16 __LF                   \+        eor     x4, x4, x16 __LF                   \+        sbc     x4, x4, x16 __LF                   \+        eor     x5, x5, x0 __LF                    \+        subs    x5, x5, x0 __LF                    \+        eor     x6, x6, x0 __LF                    \+        sbc     x6, x6, x0 __LF                    \+        eor     x16, x0, x16 __LF                  \+        adds    x11, x11, x9 __LF                  \+        adcs    x12, x12, x10 __LF                 \+        adcs    x13, x13, xzr __LF                 \+        adc     x14, x14, xzr __LF                 \+        mul     x2, x3, x5 __LF                    \+        umulh   x0, x3, x5 __LF                    \+        mul     x15, x4, x6 __LF                   \+        umulh   x1, x4, x6 __LF                    \+        subs    x4, x4, x3 __LF                    \+        cneg    x4, x4, cc __LF                    \+        csetm   x9, cc __LF                        \+        adds    x15, x15, x0 __LF                  \+        adc     x1, x1, xzr __LF                   \+        subs    x6, x5, x6 __LF                    \+        cneg    x6, x6, cc __LF                    \+        cinv    x9, x9, cc __LF                    \+        mul     x5, x4, x6 __LF                    \+        umulh   x6, x4, x6 __LF                    \+        adds    x0, x2, x15 __LF                   \+        adcs    x15, x15, x1 __LF                  \+        adc     x1, x1, xzr __LF                   \+        cmn     x9, #0x1 __LF                      \+        eor     x5, x5, x9 __LF                    \+        adcs    x0, x5, x0 __LF                    \+        eor     x6, x6, x9 __LF                    \+        adcs    x15, x6, x15 __LF                  \+        adc     x1, x1, x9 __LF                    \+        adds    x9, x11, x7 __LF                   \+        adcs    x10, x12, x8 __LF                  \+        adcs    x11, x13, x11 __LF                 \+        adcs    x12, x14, x12 __LF                 \+        adcs    x13, x13, xzr __LF                 \+        adc     x14, x14, xzr __LF                 \+        cmn     x16, #0x1 __LF                     \+        eor     x2, x2, x16 __LF                   \+        adcs    x9, x2, x9 __LF                    \+        eor     x0, x0, x16 __LF                   \+        adcs    x10, x0, x10 __LF                  \+        eor     x15, x15, x16 __LF                 \+        adcs    x11, x15, x11 __LF                 \+        eor     x1, x1, x16 __LF                   \+        adcs    x12, x1, x12 __LF                  \+        adcs    x13, x13, x16 __LF                 \+        adc     x14, x14, x16 __LF                 \+        mov     x3, #0x26 __LF                     \+        umull   x4, w11, w3 __LF                   \+        add     x4, x4, w7, uxtw __LF              \+        lsr     x7, x7, #32 __LF                   \+        lsr     x11, x11, #32 __LF                 \+        umaddl  x11, w11, w3, x7 __LF              \+        mov     x7, x4 __LF                        \+        umull   x4, w12, w3 __LF                   \+        add     x4, x4, w8, uxtw __LF              \+        lsr     x8, x8, #32 __LF                   \+        lsr     x12, x12, #32 __LF                 \+        umaddl  x12, w12, w3, x8 __LF              \+        mov     x8, x4 __LF                        \+        umull   x4, w13, w3 __LF                   \+        add     x4, x4, w9, uxtw __LF              \+        lsr     x9, x9, #32 __LF                   \+        lsr     x13, x13, #32 __LF                 \+        umaddl  x13, w13, w3, x9 __LF              \+        mov     x9, x4 __LF                        \+        umull   x4, w14, w3 __LF                   \+        add     x4, x4, w10, uxtw __LF             \+        lsr     x10, x10, #32 __LF                 \+        lsr     x14, x14, #32 __LF                 \+        umaddl  x14, w14, w3, x10 __LF             \+        mov     x10, x4 __LF                       \+        lsr     x0, x14, #31 __LF                  \+        mov     x5, #0x13 __LF                     \+        umull   x5, w5, w0 __LF                    \+        add     x7, x7, x5 __LF                    \+        adds    x7, x7, x11, lsl #32 __LF          \+        extr    x3, x12, x11, #32 __LF             \+        adcs    x8, x8, x3 __LF                    \+        extr    x3, x13, x12, #32 __LF             \+        adcs    x9, x9, x3 __LF                    \+        extr    x3, x14, x13, #32 __LF             \+        lsl     x5, x0, #63 __LF                   \+        eor     x10, x10, x5 __LF                  \+        adc     x10, x10, x3 __LF                  \+        stp     x7, x8, [P0] __LF                  \+        stp     x9, x10, [P0+16]++// Modular subtraction with double modulus 2 * p_25519 = 2^256 - 38++#define sub_twice4(P0,P1,P2)                    \+        ldp     x5, x6, [P1] __LF                  \+        ldp     x4, x3, [P2] __LF                  \+        subs    x5, x5, x4 __LF                    \+        sbcs    x6, x6, x3 __LF                    \+        ldp     x7, x8, [P1+16] __LF               \+        ldp     x4, x3, [P2+16] __LF               \+        sbcs    x7, x7, x4 __LF                    \+        sbcs    x8, x8, x3 __LF                    \+        mov     x4, #38 __LF                       \+        csel    x3, x4, xzr, lo __LF               \+        subs    x5, x5, x3 __LF                    \+        sbcs    x6, x6, xzr __LF                   \+        sbcs    x7, x7, xzr __LF                   \+        sbc     x8, x8, xzr __LF                   \+        stp     x5, x6, [P0] __LF                  \+        stp     x7, x8, [P0+16]++// Modular addition and doubling with double modulus 2 * p_25519 = 2^256 - 38.+// This only ensures that the result fits in 4 digits, not that it is reduced+// even w.r.t. double modulus. The result is always correct modulo provided+// the sum of the inputs is < 2^256 + 2^256 - 38, so in particular provided+// at least one of them is reduced double modulo.++#define add_twice4(P0,P1,P2)                    \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x7, x8, [P2] __LF                  \+        adds    x3, x3, x7 __LF                    \+        adcs    x4, x4, x8 __LF                    \+        ldp     x5, x6, [P1+16] __LF               \+        ldp     x7, x8, [P2+16] __LF               \+        adcs    x5, x5, x7 __LF                    \+        adcs    x6, x6, x8 __LF                    \+        mov     x9, #38 __LF                       \+        csel    x9, x9, xzr, cs __LF               \+        adds    x3, x3, x9 __LF                    \+        adcs    x4, x4, xzr __LF                   \+        adcs    x5, x5, xzr __LF                   \+        adc     x6, x6, xzr __LF                   \+        stp     x3, x4, [P0] __LF                  \+        stp     x5, x6, [P0+16]++#define double_twice4(P0,P1)                    \+        ldp     x3, x4, [P1] __LF                  \+        adds    x3, x3, x3 __LF                    \+        adcs    x4, x4, x4 __LF                    \+        ldp     x5, x6, [P1+16] __LF               \+        adcs    x5, x5, x5 __LF                    \+        adcs    x6, x6, x6 __LF                    \+        mov     x9, #38 __LF                       \+        csel    x9, x9, xzr, cs __LF               \+        adds    x3, x3, x9 __LF                    \+        adcs    x4, x4, xzr __LF                   \+        adcs    x5, x5, xzr __LF                   \+        adc     x6, x6, xzr __LF                   \+        stp     x3, x4, [P0] __LF                  \+        stp     x5, x6, [P0+16]++S2N_BN_SYMBOL(edwards25519_scalarmulbase):+        CFI_START++// Save regs and make room for temporaries++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_DEC_SP(NSPACE)++// Move the output pointer to a stable place++        mov     res, x0++// Copy the input scalar x to its local variable while reducing it+// modulo 2^252 + m where m = 27742317777372353535851937790883648493;+// this is the order of the basepoint so this doesn't change the result.+// First do q = floor(x/2^252) and x' = x - q * (2^252 + m), which gives+// an initial result -15 * m <= x' < 2^252++        ldp     x10, x11, [x1]+        ldp     x12, x13, [x1, #16]++        lsr     x9, x13, #60++        movbig(x0,#0x5812,#0x631a,#0x5cf5,#0xd3ed);+        movbig(x1,#0x14de,#0xf9de,#0xa2f7,#0x9cd6);++        mul     x2, x9, x0+        mul     x3, x9, x1+        umulh   x4, x9, x0+        umulh   x5, x9, x1++        adds    x3, x3, x4+        adc     x4, x5, xzr+        lsl     x5, x9, #60++        subs    x10, x10, x2+        sbcs    x11, x11, x3+        sbcs    x12, x12, x4+        sbcs    x13, x13, x5++// If x' < 0 then just directly negate it; this makes sure the+// reduced argument is strictly 0 <= x' < 2^252, but now we need+// to record (done via bit 255 of the reduced scalar, which is+// ignored in the main loop) when we negated so we can flip+// the sign of the eventual point to compensate.++        csetm   x9, cc+        adds    xzr, x9, x9+        eor     x10, x10, x9+        adcs    x10, x10, xzr+        eor     x11, x11, x9+        adcs    x11, x11, xzr+        eor     x12, x12, x9+        adcs    x12, x12, xzr+        eor     x13, x13, x9+        adc     x13, x13, xzr++        and     x9, x9, #0x8000000000000000+        orr     x13, x13, x9++// And before we store the scalar, test and reset bit 251 to+// initialize the main loop just below.++        stp     x10, x11, [scalar]+        tst     x13, #0x0800000000000000+        bic     x13, x13, #0x0800000000000000+        stp     x12, x13, [scalar+16]++// The main part of the computation is in extended-projective coordinates+// (X,Y,Z,T), representing an affine point on the edwards25519 curve+// (x,y) via x = X/Z, y = Y/Z and x * y = T/Z (so X * Y = T * Z).+// In comments B means the standard basepoint (x,4/5) =+// (0x216....f25d51a,0x6666..666658).+//+// Initialize accumulator "acc" to either 0 or 2^251 * B depending on+// bit 251 of the (reduced) scalar. That leaves bits 0..250 to handle.++#if defined(__ELF__)+        adrp    tab, S2N_BN_SYMBOL(edwards25519_scalarmulbase_constant)+        add     tab, tab, :lo12:S2N_BN_SYMBOL(edwards25519_scalarmulbase_constant)+#else+        adrp    tab, S2N_BN_SYMBOL(edwards25519_scalarmulbase_constant)@PAGE+        add     tab, tab, S2N_BN_SYMBOL(edwards25519_scalarmulbase_constant)@PAGEOFF+#endif++        ldp     x0, x1, [tab]+        ldp     x2, x3, [tab, #96]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc]++        ldp     x0, x1, [tab, #1*16]+        ldp     x2, x3, [tab, #(96+1*16)]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+1*16]++        ldp     x0, x1, [tab, #2*16]+        ldp     x2, x3, [tab, #(96+2*16)]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+2*16]++        ldp     x0, x1, [tab, #3*16]+        ldp     x2, x3, [tab, #(96+3*16)]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+3*16]++        mov     x0, #1+        stp     x0, xzr, [acc+4*16]+        stp     xzr, xzr, [acc+5*16]++        ldp     x0, x1, [tab, #4*16]+        ldp     x2, x3, [tab, #(96+4*16)]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+6*16]++        ldp     x0, x1, [tab, #5*16]+        ldp     x2, x3, [tab, #(96+5*16)]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+7*16]++// The counter "i" tracks the bit position for which the scalar has+// already been absorbed, starting at 0 and going up in chunks of 4.+//+// The pointer "tab" points at the current block of the table for+// multiples (2^i * j) * B at the current bit position i; 1 <= j <= 8.+//+// The bias is always either 0 and 1 and needs to be added to the+// partially processed scalar implicitly. This is used to absorb 4 bits+// of scalar per iteration from 3-bit table indexing by exploiting+// negation: (16 * h + l) * B = (16 * (h + 1) - (16 - l)) * B is used+// when l >= 9. Note that we can't have any bias left over at the+// end because we made sure bit 251 is clear in the reduced scalar.++        mov     i, 0+        add     tab, tab, #192+        mov     bias, xzr++// Start of the main loop, repeated 63 times for i = 0, 4, 8, ..., 248++Ledwards25519_scalarmulbase_scalarloop:++// Look at the next 4-bit field "bf", adding the previous bias as well.+// Choose the table index "ix" as bf when bf <= 8 and 16 - bf for bf >= 9,+// setting the bias to 1 for the next iteration in the latter case.++        lsr     x0, i, #6+        ldr     x2, [sp, x0, lsl #3]    // Exploiting scalar = sp exactly+        lsr     x2, x2, i+        and     x2, x2, #15+        add     bf, x2, bias++        cmp     bf, 9+        cset    bias, cs++        mov     x0, 16+        sub     x0, x0, bf+        cmp     bias, xzr+        csel    ix, x0, bf, ne++// Perform constant-time lookup in the table to get element number "ix".+// The table entry for the affine point (x,y) is actually a triple+// (y - x,x + y,2 * d * x * y) to precompute parts of the addition.+// Note that "ix" can be 0, so we set up the appropriate identity first.++        mov     x0, #1+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, #1+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        mov     x9, xzr+        mov     x10, xzr+        mov     x11, xzr++        cmp     ix, #1+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #2+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #3+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #4+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #5+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #6+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #7+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #8+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++// We now have the triple from the table in registers as follows+//+//      [x3;x2;x1;x0] = y - x+//      [x7;x6;x5;x4] = x + y+//      [x11;x10;x9;x8] = 2 * d * x * y+//+// In case bias = 1 we need to negate this. For Edwards curves+// -(x,y) = (-x,y), i.e. we need to negate the x coordinate.+// In this processed encoding, that amounts to swapping the+// first two fields and negating the third.+//+// The optional negation here also pretends bias = 0 whenever+// ix = 0 so that it doesn't need to handle the case of zero+// inputs, since no non-trivial table entries are zero. Note+// that in the zero case the whole negation is trivial, and+// so indeed is the swapping.++        cmp     bias, #0++        csel    x12, x0, x4, eq+        csel    x13, x1, x5, eq+        csel    x14, x2, x6, eq+        csel    x15, x3, x7, eq+        stp     x12, x13, [tabent]+        stp     x14, x15, [tabent+16]++        csel    x12, x0, x4, ne+        csel    x13, x1, x5, ne+        csel    x14, x2, x6, ne+        csel    x15, x3, x7, ne+        stp     x12, x13, [tabent+32]+        stp     x14, x15, [tabent+48]++        mov     x0, #-19+        subs    x0, x0, x8+        mov     x2, #-1+        sbcs    x1, x2, x9+        sbcs    x2, x2, x10+        mov     x3, #0x7FFFFFFFFFFFFFFF+        sbc     x3, x3, x11++        cmp     ix, xzr+        ccmp    bias, xzr, #4, ne++        csel    x0, x0, x8, ne+        csel    x1, x1, x9, ne+        stp     x0, x1, [tabent+64]+        csel    x2, x2, x10, ne+        csel    x3, x3, x11, ne+        stp     x2, x3, [tabent+80]++// Extended-projective and precomputed mixed addition.+// This is effectively the same as calling the standalone+// function edwards25519_pepadd_alt(acc,acc,tabent), but we+// only retain slightly weaker normalization < 2 * p_25519+// throughout the inner loop, so the computation is+// slightly different, and faster overall.++        double_twice4(t0,z_1)+        sub_twice4(t1,y_1,x_1)+        add_twice4(t2,y_1,x_1)+        mul_4(t3,w_1,kxy_2)+        mul_4(t1,t1,ymx_2)+        mul_4(t2,t2,xpy_2)+        sub_twice4(t4,t0,t3)+        add_twice4(t0,t0,t3)+        sub_twice4(t5,t2,t1)+        add_twice4(t1,t2,t1)+        mul_4(z_3,t4,t0)+        mul_4(x_3,t5,t4)+        mul_4(y_3,t0,t1)+        mul_4(w_3,t5,t1)++// End of the main loop; move on by 4 bits.++        add     i, i, 4+        cmp     i, 252+        bcc     Ledwards25519_scalarmulbase_scalarloop++// Insert the optional negation of the projective X coordinate, and+// so by extension the final affine x coordinate x = X/Z and thus+// the point P = (x,y). We only know X < 2 * p_25519, so we do the+// negation as 2 * p_25519 - X to keep it nonnegative. From this+// point on we don't need any normalization of the coordinates+// except for making sure that they fit in 4 digits.++        ldp     x0, x1, [x_3]+        ldp     x2, x3, [x_3+16]+        mov     x4, #0xffffffffffffffda+        subs    x4, x4, x0+        mov     x7, #0xffffffffffffffff+        sbcs    x5, x7, x1+        sbcs    x6, x7, x2+        sbc     x7, x7, x3+        ldr     x10, [scalar+24]+        tst     x10, #0x8000000000000000+        csel    x0, x4, x0, ne+        csel    x1, x5, x1, ne+        csel    x2, x6, x2, ne+        csel    x3, x7, x3, ne+        stp     x0, x1, [x_3]+        stp     x2, x3, [x_3+16]++// Now we need to map out of the extended-projective representation+// (X,Y,Z,W) back to the affine form (x,y) = (X/Z,Y/Z). This means+// first calling the modular inverse to get w_3 = 1/z_3.++        add     x0, w_3+        add     x1, z_3++// Inline copy of bignum_inv_p25519, identical except for stripping out+// the prologue and epilogue saving and restoring registers and making+// and reclaiming room on the stack. For more details and explanations see+// "arm/curve25519/bignum_inv_p25519.S". Note that the stack it uses for+// its own temporaries is 128 bytes, so it has no effect on variables+// that are needed in the rest of our computation here: res, w_3, x_3+// and y_3.++        mov     x20, x0+        mov     x10, #0xffffffffffffffed+        mov     x11, #0xffffffffffffffff+        stp     x10, x11, [sp]+        mov     x12, #0x7fffffffffffffff+        stp     x11, x12, [sp, #16]+        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]+        mov     x7, #0x13+        lsr     x6, x5, #63+        madd    x6, x7, x6, x7+        adds    x2, x2, x6+        adcs    x3, x3, xzr+        adcs    x4, x4, xzr+        orr     x5, x5, #0x8000000000000000+        adcs    x5, x5, xzr+        csel    x6, x7, xzr, cc+        subs    x2, x2, x6+        sbcs    x3, x3, xzr+        sbcs    x4, x4, xzr+        sbc     x5, x5, xzr+        and     x5, x5, #0x7fffffffffffffff+        stp     x2, x3, [sp, #32]+        stp     x4, x5, [sp, #48]+        stp     xzr, xzr, [sp, #64]+        stp     xzr, xzr, [sp, #80]+        mov     x10, #0x2099+        movk    x10, #0x7502, lsl #16+        movk    x10, #0x9e23, lsl #32+        movk    x10, #0xa0f9, lsl #48+        mov     x11, #0x2595+        movk    x11, #0x1d13, lsl #16+        movk    x11, #0x8f3f, lsl #32+        movk    x11, #0xa8c6, lsl #48+        mov     x12, #0x5242+        movk    x12, #0x5ac, lsl #16+        movk    x12, #0x8938, lsl #32+        movk    x12, #0x6c6c, lsl #48+        mov     x13, #0x615+        movk    x13, #0x4177, lsl #16+        movk    x13, #0x8b2, lsl #32+        movk    x13, #0x2765, lsl #48+        stp     x10, x11, [sp, #96]+        stp     x12, x13, [sp, #112]+        mov     x21, #0xa+        mov     x22, #0x1+        b       Ledwards25519_scalarmulbase_invmidloop+Ledwards25519_scalarmulbase_invloop:+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        and     x0, x12, x16+        and     x1, x13, x17+        add     x19, x0, x1+        ldr     x7, [sp]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #32]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x7, [sp, #8]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #40]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [sp]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [sp, #32]+        ldr     x7, [sp, #16]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #48]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [sp, #8]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [sp, #40]+        ldr     x7, [sp, #24]+        eor     x1, x7, x14+        asr     x3, x1, #63+        and     x3, x3, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #56]+        eor     x1, x8, x15+        asr     x0, x1, #63+        and     x0, x0, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x5, x6, #59+        str     x6, [sp, #16]+        extr    x5, x3, x5, #59+        str     x5, [sp, #24]+        eor     x1, x7, x16+        asr     x5, x1, #63+        and     x5, x5, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        asr     x0, x1, #63+        and     x0, x0, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x4, x2, x4, #59+        str     x4, [sp, #48]+        extr    x2, x5, x2, #59+        str     x2, [sp, #56]+        ldr     x7, [sp, #64]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #96]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #64]+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        str     x5, [sp, #96]+        adc     x3, x3, x1+        ldr     x7, [sp, #72]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #104]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #72]+        adc     x6, x6, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        str     x3, [sp, #104]+        adc     x4, x4, x1+        ldr     x7, [sp, #80]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #112]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #80]+        adc     x5, x5, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        str     x4, [sp, #112]+        adc     x2, x2, x1+        ldr     x7, [sp, #88]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #120]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x3, x5, #63+        ldp     x0, x1, [sp, #64]+        add     x6, x6, x3, asr #63+        mov     x3, #0x13+        mul     x4, x6, x3+        add     x5, x5, x6, lsl #63+        smulh   x3, x6, x3+        ldr     x6, [sp, #80]+        adds    x0, x0, x4+        adcs    x1, x1, x3+        asr     x3, x3, #63+        adcs    x6, x6, x3+        adc     x5, x5, x3+        stp     x0, x1, [sp, #64]+        stp     x6, x5, [sp, #80]+        eor     x1, x7, x16+        and     x5, x16, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        and     x0, x17, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x6, x5, x2, #63+        ldp     x0, x1, [sp, #96]+        add     x6, x6, x5, asr #63+        mov     x5, #0x13+        mul     x4, x6, x5+        add     x2, x2, x6, lsl #63+        smulh   x5, x6, x5+        ldr     x3, [sp, #112]+        adds    x0, x0, x4+        adcs    x1, x1, x5+        asr     x5, x5, #63+        adcs    x3, x3, x5+        adc     x2, x2, x5+        stp     x0, x1, [sp, #96]+        stp     x3, x2, [sp, #112]+Ledwards25519_scalarmulbase_invmidloop:+        mov     x1, x22+        ldr     x2, [sp]+        ldr     x3, [sp, #32]+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x8, x4, #0x100, lsl #12+        sbfx    x8, x8, #21, #21+        mov     x11, #0x100000+        add     x11, x11, x11, lsl #21+        add     x9, x4, x11+        asr     x9, x9, #42+        add     x10, x5, #0x100, lsl #12+        sbfx    x10, x10, #21, #21+        add     x11, x5, x11+        asr     x11, x11, #42+        mul     x6, x8, x2+        mul     x7, x9, x3+        mul     x2, x10, x2+        mul     x3, x11, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #21, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #42+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #21, #21+        add     x15, x5, x15+        asr     x15, x15, #42+        mul     x6, x12, x2+        mul     x7, x13, x3+        mul     x2, x14, x2+        mul     x3, x15, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        mul     x2, x12, x8+        mul     x3, x12, x9+        mul     x6, x14, x8+        mul     x7, x14, x9+        madd    x8, x13, x10, x2+        madd    x9, x13, x11, x3+        madd    x16, x15, x10, x6+        madd    x17, x15, x11, x7+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #22, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #43+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #22, #21+        add     x15, x5, x15+        asr     x15, x15, #43+        mneg    x2, x12, x8+        mneg    x3, x12, x9+        mneg    x4, x14, x8+        mneg    x5, x14, x9+        msub    x10, x13, x16, x2+        msub    x11, x13, x17, x3+        msub    x12, x15, x16, x4+        msub    x13, x15, x17, x5+        mov     x22, x1+        subs    x21, x21, #0x1+        b.ne    Ledwards25519_scalarmulbase_invloop+        ldr     x0, [sp]+        ldr     x1, [sp, #32]+        mul     x0, x0, x10+        madd    x1, x1, x11, x0+        asr     x0, x1, #63+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        eor     x14, x14, x0+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        eor     x15, x15, x0+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        eor     x16, x16, x0+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        eor     x17, x17, x0+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        ldr     x7, [sp, #64]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #96]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #64]+        adc     x2, x2, x1+        ldr     x7, [sp, #72]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #104]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #72]+        adc     x6, x6, x1+        ldr     x7, [sp, #80]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #112]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #80]+        adc     x5, x5, x1+        ldr     x7, [sp, #88]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #120]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x3, x5, #63+        ldp     x0, x1, [sp, #64]+        tst     x3, x3+        cinc    x6, x6, pl+        mov     x3, #0x13+        mul     x4, x6, x3+        add     x5, x5, x6, lsl #63+        smulh   x6, x6, x3+        ldr     x2, [sp, #80]+        adds    x0, x0, x4+        adcs    x1, x1, x6+        asr     x6, x6, #63+        adcs    x2, x2, x6+        adcs    x5, x5, x6+        csel    x3, x3, xzr, mi+        subs    x0, x0, x3+        sbcs    x1, x1, xzr+        sbcs    x2, x2, xzr+        sbc     x5, x5, xzr+        and     x5, x5, #0x7fffffffffffffff+        mov     x4, x20+        stp     x0, x1, [x4]+        stp     x2, x5, [x4, #16]++// The final result is x = X * inv(Z), y = Y * inv(Z).+// These are the only operations in the whole computation that+// fully reduce modulo p_25519 since now we want the canonical+// answer as output.++        mul_p25519(resx,x_3,w_3)+        mul_p25519(resy,y_3,w_3)++// Restore stack and registers++        CFI_INC_SP(NSPACE)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(edwards25519_scalarmulbase)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif++// ****************************************************************************+// The precomputed data (all read-only).+// ****************************************************************************++#if defined(__ELF__)+.section .rodata+.type S2N_BN_SYMBOL(edwards25519_scalarmulbase_constant), %object+.size S2N_BN_SYMBOL(edwards25519_scalarmulbase_constant), 48576+#elif defined(__APPLE__)+.const_data+#endif++S2N_BN_SYMBOL(edwards25519_scalarmulbase_constant):++// 0 * B = 0 and 2^251 * B in extended-projective coordinates+// but with Z = 1 assumed and hence left out, so they are (X,Y,T) only.++        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000001+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000++        .quad   0x525f946d7c7220e7+        .quad   0x4636b0b2f1e35444+        .quad   0x796e9d70e892ae0f+        .quad   0x03dec05fa937adb1+        .quad   0x6d1c271cc6375515+        .quad   0x462588c4a4ca4f14+        .quad   0x691129fee55afc39+        .quad   0x15949f784d8472f5+        .quad   0xbd89e510afad0049+        .quad   0x4d1f08c073b9860e+        .quad   0x07716e8b2d00af9d+        .quad   0x70d685f68f859714++// Precomputed table of multiples of generator for edwards25519+// all in precomputed extended-projective (y-x,x+y,2*d*x*y) triples.++        // 2^0 * 1 * G++        .quad   0x9d103905d740913e+        .quad   0xfd399f05d140beb3+        .quad   0xa5c18434688f8a09+        .quad   0x44fd2f9298f81267+        .quad   0x2fbc93c6f58c3b85+        .quad   0xcf932dc6fb8c0e19+        .quad   0x270b4898643d42c2+        .quad   0x07cf9d3a33d4ba65+        .quad   0xabc91205877aaa68+        .quad   0x26d9e823ccaac49e+        .quad   0x5a1b7dcbdd43598c+        .quad   0x6f117b689f0c65a8++        // 2^0 * 2 * G++        .quad   0x8a99a56042b4d5a8+        .quad   0x8f2b810c4e60acf6+        .quad   0xe09e236bb16e37aa+        .quad   0x6bb595a669c92555+        .quad   0x9224e7fc933c71d7+        .quad   0x9f469d967a0ff5b5+        .quad   0x5aa69a65e1d60702+        .quad   0x590c063fa87d2e2e+        .quad   0x43faa8b3a59b7a5f+        .quad   0x36c16bdd5d9acf78+        .quad   0x500fa0840b3d6a31+        .quad   0x701af5b13ea50b73++        // 2^0 * 3 * G++        .quad   0x56611fe8a4fcd265+        .quad   0x3bd353fde5c1ba7d+        .quad   0x8131f31a214bd6bd+        .quad   0x2ab91587555bda62+        .quad   0xaf25b0a84cee9730+        .quad   0x025a8430e8864b8a+        .quad   0xc11b50029f016732+        .quad   0x7a164e1b9a80f8f4+        .quad   0x14ae933f0dd0d889+        .quad   0x589423221c35da62+        .quad   0xd170e5458cf2db4c+        .quad   0x5a2826af12b9b4c6++        // 2^0 * 4 * G++        .quad   0x95fe050a056818bf+        .quad   0x327e89715660faa9+        .quad   0xc3e8e3cd06a05073+        .quad   0x27933f4c7445a49a+        .quad   0x287351b98efc099f+        .quad   0x6765c6f47dfd2538+        .quad   0xca348d3dfb0a9265+        .quad   0x680e910321e58727+        .quad   0x5a13fbe9c476ff09+        .quad   0x6e9e39457b5cc172+        .quad   0x5ddbdcf9102b4494+        .quad   0x7f9d0cbf63553e2b++        // 2^0 * 5 * G++        .quad   0x7f9182c3a447d6ba+        .quad   0xd50014d14b2729b7+        .quad   0xe33cf11cb864a087+        .quad   0x154a7e73eb1b55f3+        .quad   0xa212bc4408a5bb33+        .quad   0x8d5048c3c75eed02+        .quad   0xdd1beb0c5abfec44+        .quad   0x2945ccf146e206eb+        .quad   0xbcbbdbf1812a8285+        .quad   0x270e0807d0bdd1fc+        .quad   0xb41b670b1bbda72d+        .quad   0x43aabe696b3bb69a++        // 2^0 * 6 * G++        .quad   0x499806b67b7d8ca4+        .quad   0x575be28427d22739+        .quad   0xbb085ce7204553b9+        .quad   0x38b64c41ae417884+        .quad   0x3a0ceeeb77157131+        .quad   0x9b27158900c8af88+        .quad   0x8065b668da59a736+        .quad   0x51e57bb6a2cc38bd+        .quad   0x85ac326702ea4b71+        .quad   0xbe70e00341a1bb01+        .quad   0x53e4a24b083bc144+        .quad   0x10b8e91a9f0d61e3++        // 2^0 * 7 * G++        .quad   0xba6f2c9aaa3221b1+        .quad   0x6ca021533bba23a7+        .quad   0x9dea764f92192c3a+        .quad   0x1d6edd5d2e5317e0+        .quad   0x6b1a5cd0944ea3bf+        .quad   0x7470353ab39dc0d2+        .quad   0x71b2528228542e49+        .quad   0x461bea69283c927e+        .quad   0xf1836dc801b8b3a2+        .quad   0xb3035f47053ea49a+        .quad   0x529c41ba5877adf3+        .quad   0x7a9fbb1c6a0f90a7++        // 2^0 * 8 * G++        .quad   0xe2a75dedf39234d9+        .quad   0x963d7680e1b558f9+        .quad   0x2c2741ac6e3c23fb+        .quad   0x3a9024a1320e01c3+        .quad   0x59b7596604dd3e8f+        .quad   0x6cb30377e288702c+        .quad   0xb1339c665ed9c323+        .quad   0x0915e76061bce52f+        .quad   0xe7c1f5d9c9a2911a+        .quad   0xb8a371788bcca7d7+        .quad   0x636412190eb62a32+        .quad   0x26907c5c2ecc4e95++        // 2^4 * 1 * B++        .quad   0x7ec851ca553e2df3+        .quad   0xa71284cba64878b3+        .quad   0xe6b5e4193288d1e7+        .quad   0x4cf210ec5a9a8883+        .quad   0x322d04a52d9021f6+        .quad   0xb9c19f3375c6bf9c+        .quad   0x587a3a4342d20b09+        .quad   0x143b1cf8aa64fe61+        .quad   0x9f867c7d968acaab+        .quad   0x5f54258e27092729+        .quad   0xd0a7d34bea180975+        .quad   0x21b546a3374126e1++        // 2^4 * 2 * B++        .quad   0xa94ff858a2888343+        .quad   0xce0ed4565313ed3c+        .quad   0xf55c3dcfb5bf34fa+        .quad   0x0a653ca5c9eab371+        .quad   0x490a7a45d185218f+        .quad   0x9a15377846049335+        .quad   0x0060ea09cc31e1f6+        .quad   0x7e041577f86ee965+        .quad   0x66b2a496ce5b67f3+        .quad   0xff5492d8bd569796+        .quad   0x503cec294a592cd0+        .quad   0x566943650813acb2++        // 2^4 * 3 * B++        .quad   0xb818db0c26620798+        .quad   0x5d5c31d9606e354a+        .quad   0x0982fa4f00a8cdc7+        .quad   0x17e12bcd4653e2d4+        .quad   0x5672f9eb1dabb69d+        .quad   0xba70b535afe853fc+        .quad   0x47ac0f752796d66d+        .quad   0x32a5351794117275+        .quad   0xd3a644a6df648437+        .quad   0x703b6559880fbfdd+        .quad   0xcb852540ad3a1aa5+        .quad   0x0900b3f78e4c6468++        // 2^4 * 4 * B++        .quad   0x0a851b9f679d651b+        .quad   0xe108cb61033342f2+        .quad   0xd601f57fe88b30a3+        .quad   0x371f3acaed2dd714+        .quad   0xed280fbec816ad31+        .quad   0x52d9595bd8e6efe3+        .quad   0x0fe71772f6c623f5+        .quad   0x4314030b051e293c+        .quad   0xd560005efbf0bcad+        .quad   0x8eb70f2ed1870c5e+        .quad   0x201f9033d084e6a0+        .quad   0x4c3a5ae1ce7b6670++        // 2^4 * 5 * B++        .quad   0x4138a434dcb8fa95+        .quad   0x870cf67d6c96840b+        .quad   0xde388574297be82c+        .quad   0x7c814db27262a55a+        .quad   0xbaf875e4c93da0dd+        .quad   0xb93282a771b9294d+        .quad   0x80d63fb7f4c6c460+        .quad   0x6de9c73dea66c181+        .quad   0x478904d5a04df8f2+        .quad   0xfafbae4ab10142d3+        .quad   0xf6c8ac63555d0998+        .quad   0x5aac4a412f90b104++        // 2^4 * 6 * B++        .quad   0xc64f326b3ac92908+        .quad   0x5551b282e663e1e0+        .quad   0x476b35f54a1a4b83+        .quad   0x1b9da3fe189f68c2+        .quad   0x603a0d0abd7f5134+        .quad   0x8089c932e1d3ae46+        .quad   0xdf2591398798bd63+        .quad   0x1c145cd274ba0235+        .quad   0x32e8386475f3d743+        .quad   0x365b8baf6ae5d9ef+        .quad   0x825238b6385b681e+        .quad   0x234929c1167d65e1++        // 2^4 * 7 * B++        .quad   0x984decaba077ade8+        .quad   0x383f77ad19eb389d+        .quad   0xc7ec6b7e2954d794+        .quad   0x59c77b3aeb7c3a7a+        .quad   0x48145cc21d099fcf+        .quad   0x4535c192cc28d7e5+        .quad   0x80e7c1e548247e01+        .quad   0x4a5f28743b2973ee+        .quad   0xd3add725225ccf62+        .quad   0x911a3381b2152c5d+        .quad   0xd8b39fad5b08f87d+        .quad   0x6f05606b4799fe3b++        // 2^4 * 8 * B++        .quad   0x9ffe9e92177ba962+        .quad   0x98aee71d0de5cae1+        .quad   0x3ff4ae942d831044+        .quad   0x714de12e58533ac8+        .quad   0x5b433149f91b6483+        .quad   0xadb5dc655a2cbf62+        .quad   0x87fa8412632827b3+        .quad   0x60895e91ab49f8d8+        .quad   0xe9ecf2ed0cf86c18+        .quad   0xb46d06120735dfd4+        .quad   0xbc9da09804b96be7+        .quad   0x73e2e62fd96dc26b++        // 2^8 * 1 * B++        .quad   0xed5b635449aa515e+        .quad   0xa865c49f0bc6823a+        .quad   0x850c1fe95b42d1c4+        .quad   0x30d76d6f03d315b9+        .quad   0x2eccdd0e632f9c1d+        .quad   0x51d0b69676893115+        .quad   0x52dfb76ba8637a58+        .quad   0x6dd37d49a00eef39+        .quad   0x6c4444172106e4c7+        .quad   0xfb53d680928d7f69+        .quad   0xb4739ea4694d3f26+        .quad   0x10c697112e864bb0++        // 2^8 * 2 * B++        .quad   0x6493c4277dbe5fde+        .quad   0x265d4fad19ad7ea2+        .quad   0x0e00dfc846304590+        .quad   0x25e61cabed66fe09+        .quad   0x0ca62aa08358c805+        .quad   0x6a3d4ae37a204247+        .quad   0x7464d3a63b11eddc+        .quad   0x03bf9baf550806ef+        .quad   0x3f13e128cc586604+        .quad   0x6f5873ecb459747e+        .quad   0xa0b63dedcc1268f5+        .quad   0x566d78634586e22c++        // 2^8 * 3 * B++        .quad   0x1637a49f9cc10834+        .quad   0xbc8e56d5a89bc451+        .quad   0x1cb5ec0f7f7fd2db+        .quad   0x33975bca5ecc35d9+        .quad   0xa1054285c65a2fd0+        .quad   0x6c64112af31667c3+        .quad   0x680ae240731aee58+        .quad   0x14fba5f34793b22a+        .quad   0x3cd746166985f7d4+        .quad   0x593e5e84c9c80057+        .quad   0x2fc3f2b67b61131e+        .quad   0x14829cea83fc526c++        // 2^8 * 4 * B++        .quad   0xff437b8497dd95c2+        .quad   0x6c744e30aa4eb5a7+        .quad   0x9e0c5d613c85e88b+        .quad   0x2fd9c71e5f758173+        .quad   0x21e70b2f4e71ecb8+        .quad   0xe656ddb940a477e3+        .quad   0xbf6556cece1d4f80+        .quad   0x05fc3bc4535d7b7e+        .quad   0x24b8b3ae52afdedd+        .quad   0x3495638ced3b30cf+        .quad   0x33a4bc83a9be8195+        .quad   0x373767475c651f04++        // 2^8 * 5 * B++        .quad   0x2fba99fd40d1add9+        .quad   0xb307166f96f4d027+        .quad   0x4363f05215f03bae+        .quad   0x1fbea56c3b18f999+        .quad   0x634095cb14246590+        .quad   0xef12144016c15535+        .quad   0x9e38140c8910bc60+        .quad   0x6bf5905730907c8c+        .quad   0x0fa778f1e1415b8a+        .quad   0x06409ff7bac3a77e+        .quad   0x6f52d7b89aa29a50+        .quad   0x02521cf67a635a56++        // 2^8 * 6 * B++        .quad   0x513fee0b0a9d5294+        .quad   0x8f98e75c0fdf5a66+        .quad   0xd4618688bfe107ce+        .quad   0x3fa00a7e71382ced+        .quad   0xb1146720772f5ee4+        .quad   0xe8f894b196079ace+        .quad   0x4af8224d00ac824a+        .quad   0x001753d9f7cd6cc4+        .quad   0x3c69232d963ddb34+        .quad   0x1dde87dab4973858+        .quad   0xaad7d1f9a091f285+        .quad   0x12b5fe2fa048edb6++        // 2^8 * 7 * B++        .quad   0x71f0fbc496fce34d+        .quad   0x73b9826badf35bed+        .quad   0xd2047261ff28c561+        .quad   0x749b76f96fb1206f+        .quad   0xdf2b7c26ad6f1e92+        .quad   0x4b66d323504b8913+        .quad   0x8c409dc0751c8bc3+        .quad   0x6f7e93c20796c7b8+        .quad   0x1f5af604aea6ae05+        .quad   0xc12351f1bee49c99+        .quad   0x61a808b5eeff6b66+        .quad   0x0fcec10f01e02151++        // 2^8 * 8 * B++        .quad   0x644d58a649fe1e44+        .quad   0x21fcaea231ad777e+        .quad   0x02441c5a887fd0d2+        .quad   0x4901aa7183c511f3+        .quad   0x3df2d29dc4244e45+        .quad   0x2b020e7493d8de0a+        .quad   0x6cc8067e820c214d+        .quad   0x413779166feab90a+        .quad   0x08b1b7548c1af8f0+        .quad   0xce0f7a7c246299b4+        .quad   0xf760b0f91e06d939+        .quad   0x41bb887b726d1213++        // 2^12 * 1 * B++        .quad   0x9267806c567c49d8+        .quad   0x066d04ccca791e6a+        .quad   0xa69f5645e3cc394b+        .quad   0x5c95b686a0788cd2+        .quad   0x97d980e0aa39f7d2+        .quad   0x35d0384252c6b51c+        .quad   0x7d43f49307cd55aa+        .quad   0x56bd36cfb78ac362+        .quad   0x2ac519c10d14a954+        .quad   0xeaf474b494b5fa90+        .quad   0xe6af8382a9f87a5a+        .quad   0x0dea6db1879be094++        // 2^12 * 2 * B++        .quad   0xaa66bf547344e5ab+        .quad   0xda1258888f1b4309+        .quad   0x5e87d2b3fd564b2f+        .quad   0x5b2c78885483b1dd+        .quad   0x15baeb74d6a8797a+        .quad   0x7ef55cf1fac41732+        .quad   0x29001f5a3c8b05c5+        .quad   0x0ad7cc8752eaccfb+        .quad   0x52151362793408cf+        .quad   0xeb0f170319963d94+        .quad   0xa833b2fa883d9466+        .quad   0x093a7fa775003c78++        // 2^12 * 3 * B++        .quad   0xe5107de63a16d7be+        .quad   0xa377ffdc9af332cf+        .quad   0x70d5bf18440b677f+        .quad   0x6a252b19a4a31403+        .quad   0xb8e9604460a91286+        .quad   0x7f3fd8047778d3de+        .quad   0x67d01e31bf8a5e2d+        .quad   0x7b038a06c27b653e+        .quad   0x9ed919d5d36990f3+        .quad   0x5213aebbdb4eb9f2+        .quad   0xc708ea054cb99135+        .quad   0x58ded57f72260e56++        // 2^12 * 4 * B++        .quad   0x78e79dade9413d77+        .quad   0xf257f9d59729e67d+        .quad   0x59db910ee37aa7e6+        .quad   0x6aa11b5bbb9e039c+        .quad   0xda6d53265b0fd48b+        .quad   0x8960823193bfa988+        .quad   0xd78ac93261d57e28+        .quad   0x79f2942d3a5c8143+        .quad   0x97da2f25b6c88de9+        .quad   0x251ba7eaacf20169+        .quad   0x09b44f87ef4eb4e4+        .quad   0x7d90ab1bbc6a7da5++        // 2^12 * 5 * B++        .quad   0x9acca683a7016bfe+        .quad   0x90505f4df2c50b6d+        .quad   0x6b610d5fcce435aa+        .quad   0x19a10d446198ff96+        .quad   0x1a07a3f496b3c397+        .quad   0x11ceaa188f4e2532+        .quad   0x7d9498d5a7751bf0+        .quad   0x19ed161f508dd8a0+        .quad   0x560a2cd687dce6ca+        .quad   0x7f3568c48664cf4d+        .quad   0x8741e95222803a38+        .quad   0x483bdab1595653fc++        // 2^12 * 6 * B++        .quad   0xfa780f148734fa49+        .quad   0x106f0b70360534e0+        .quad   0x2210776fe3e307bd+        .quad   0x3286c109dde6a0fe+        .quad   0xd6cf4d0ab4da80f6+        .quad   0x82483e45f8307fe0+        .quad   0x05005269ae6f9da4+        .quad   0x1c7052909cf7877a+        .quad   0x32ee7de2874e98d4+        .quad   0x14c362e9b97e0c60+        .quad   0x5781dcde6a60a38a+        .quad   0x217dd5eaaa7aa840++        // 2^12 * 7 * B++        .quad   0x9db7c4d0248e1eb0+        .quad   0xe07697e14d74bf52+        .quad   0x1e6a9b173c562354+        .quad   0x7fa7c21f795a4965+        .quad   0x8bdf1fb9be8c0ec8+        .quad   0x00bae7f8e30a0282+        .quad   0x4963991dad6c4f6c+        .quad   0x07058a6e5df6f60a+        .quad   0xe9eb02c4db31f67f+        .quad   0xed25fd8910bcfb2b+        .quad   0x46c8131f5c5cddb4+        .quad   0x33b21c13a0cb9bce++        // 2^12 * 8 * B++        .quad   0x360692f8087d8e31+        .quad   0xf4dcc637d27163f7+        .quad   0x25a4e62065ea5963+        .quad   0x659bf72e5ac160d9+        .quad   0x9aafb9b05ee38c5b+        .quad   0xbf9d2d4e071a13c7+        .quad   0x8eee6e6de933290a+        .quad   0x1c3bab17ae109717+        .quad   0x1c9ab216c7cab7b0+        .quad   0x7d65d37407bbc3cc+        .quad   0x52744750504a58d5+        .quad   0x09f2606b131a2990++        // 2^16 * 1 * B++        .quad   0x40e87d44744346be+        .quad   0x1d48dad415b52b25+        .quad   0x7c3a8a18a13b603e+        .quad   0x4eb728c12fcdbdf7+        .quad   0x7e234c597c6691ae+        .quad   0x64889d3d0a85b4c8+        .quad   0xdae2c90c354afae7+        .quad   0x0a871e070c6a9e1d+        .quad   0x3301b5994bbc8989+        .quad   0x736bae3a5bdd4260+        .quad   0x0d61ade219d59e3c+        .quad   0x3ee7300f2685d464++        // 2^16 * 2 * B++        .quad   0xf5d255e49e7dd6b7+        .quad   0x8016115c610b1eac+        .quad   0x3c99975d92e187ca+        .quad   0x13815762979125c2+        .quad   0x43fa7947841e7518+        .quad   0xe5c6fa59639c46d7+        .quad   0xa1065e1de3052b74+        .quad   0x7d47c6a2cfb89030+        .quad   0x3fdad0148ef0d6e0+        .quad   0x9d3e749a91546f3c+        .quad   0x71ec621026bb8157+        .quad   0x148cf58d34c9ec80++        // 2^16 * 3 * B++        .quad   0x46a492f67934f027+        .quad   0x469984bef6840aa9+        .quad   0x5ca1bc2a89611854+        .quad   0x3ff2fa1ebd5dbbd4+        .quad   0xe2572f7d9ae4756d+        .quad   0x56c345bb88f3487f+        .quad   0x9fd10b6d6960a88d+        .quad   0x278febad4eaea1b9+        .quad   0xb1aa681f8c933966+        .quad   0x8c21949c20290c98+        .quad   0x39115291219d3c52+        .quad   0x4104dd02fe9c677b++        // 2^16 * 4 * B++        .quad   0x72b2bf5e1124422a+        .quad   0xa1fa0c3398a33ab5+        .quad   0x94cb6101fa52b666+        .quad   0x2c863b00afaf53d5+        .quad   0x81214e06db096ab8+        .quad   0x21a8b6c90ce44f35+        .quad   0x6524c12a409e2af5+        .quad   0x0165b5a48efca481+        .quad   0xf190a474a0846a76+        .quad   0x12eff984cd2f7cc0+        .quad   0x695e290658aa2b8f+        .quad   0x591b67d9bffec8b8++        // 2^16 * 5 * B++        .quad   0x312f0d1c80b49bfa+        .quad   0x5979515eabf3ec8a+        .quad   0x727033c09ef01c88+        .quad   0x3de02ec7ca8f7bcb+        .quad   0x99b9b3719f18b55d+        .quad   0xe465e5faa18c641e+        .quad   0x61081136c29f05ed+        .quad   0x489b4f867030128b+        .quad   0xd232102d3aeb92ef+        .quad   0xe16253b46116a861+        .quad   0x3d7eabe7190baa24+        .quad   0x49f5fbba496cbebf++        // 2^16 * 6 * B++        .quad   0x30949a108a5bcfd4+        .quad   0xdc40dd70bc6473eb+        .quad   0x92c294c1307c0d1c+        .quad   0x5604a86dcbfa6e74+        .quad   0x155d628c1e9c572e+        .quad   0x8a4d86acc5884741+        .quad   0x91a352f6515763eb+        .quad   0x06a1a6c28867515b+        .quad   0x7288d1d47c1764b6+        .quad   0x72541140e0418b51+        .quad   0x9f031a6018acf6d1+        .quad   0x20989e89fe2742c6++        // 2^16 * 7 * B++        .quad   0x499777fd3a2dcc7f+        .quad   0x32857c2ca54fd892+        .quad   0xa279d864d207e3a0+        .quad   0x0403ed1d0ca67e29+        .quad   0x1674278b85eaec2e+        .quad   0x5621dc077acb2bdf+        .quad   0x640a4c1661cbf45a+        .quad   0x730b9950f70595d3+        .quad   0xc94b2d35874ec552+        .quad   0xc5e6c8cf98246f8d+        .quad   0xf7cb46fa16c035ce+        .quad   0x5bd7454308303dcc++        // 2^16 * 8 * B++        .quad   0x7f9ad19528b24cc2+        .quad   0x7f6b54656335c181+        .quad   0x66b8b66e4fc07236+        .quad   0x133a78007380ad83+        .quad   0x85c4932115e7792a+        .quad   0xc64c89a2bdcdddc9+        .quad   0x9d1e3da8ada3d762+        .quad   0x5bb7db123067f82c+        .quad   0x0961f467c6ca62be+        .quad   0x04ec21d6211952ee+        .quad   0x182360779bd54770+        .quad   0x740dca6d58f0e0d2++        // 2^20 * 1 * B++        .quad   0x50b70bf5d3f0af0b+        .quad   0x4feaf48ae32e71f7+        .quad   0x60e84ed3a55bbd34+        .quad   0x00ed489b3f50d1ed+        .quad   0x3906c72aed261ae5+        .quad   0x9ab68fd988e100f7+        .quad   0xf5e9059af3360197+        .quad   0x0e53dc78bf2b6d47+        .quad   0xb90829bf7971877a+        .quad   0x5e4444636d17e631+        .quad   0x4d05c52e18276893+        .quad   0x27632d9a5a4a4af5++        // 2^20 * 2 * B++        .quad   0xd11ff05154b260ce+        .quad   0xd86dc38e72f95270+        .quad   0x601fcd0d267cc138+        .quad   0x2b67916429e90ccd+        .quad   0xa98285d187eaffdb+        .quad   0xa5b4fbbbd8d0a864+        .quad   0xb658f27f022663f7+        .quad   0x3bbc2b22d99ce282+        .quad   0xb917c952583c0a58+        .quad   0x653ff9b80fe4c6f3+        .quad   0x9b0da7d7bcdf3c0c+        .quad   0x43a0eeb6ab54d60e++        // 2^20 * 3 * B++        .quad   0x396966a46d4a5487+        .quad   0xf811a18aac2bb3ba+        .quad   0x66e4685b5628b26b+        .quad   0x70a477029d929b92+        .quad   0x3ac6322357875fe8+        .quad   0xd9d4f4ecf5fbcb8f+        .quad   0x8dee8493382bb620+        .quad   0x50c5eaa14c799fdc+        .quad   0xdd0edc8bd6f2fb3c+        .quad   0x54c63aa79cc7b7a0+        .quad   0xae0b032b2c8d9f1a+        .quad   0x6f9ce107602967fb++        // 2^20 * 4 * B++        .quad   0xad1054b1cde1c22a+        .quad   0xc4a8e90248eb32df+        .quad   0x5f3e7b33accdc0ea+        .quad   0x72364713fc79963e+        .quad   0x139693063520e0b5+        .quad   0x437fcf7c88ea03fe+        .quad   0xf7d4c40bd3c959bc+        .quad   0x699154d1f893ded9+        .quad   0x315d5c75b4b27526+        .quad   0xcccb842d0236daa5+        .quad   0x22f0c8a3345fee8e+        .quad   0x73975a617d39dbed++        // 2^20 * 5 * B++        .quad   0xe4024df96375da10+        .quad   0x78d3251a1830c870+        .quad   0x902b1948658cd91c+        .quad   0x7e18b10b29b7438a+        .quad   0x6f37f392f4433e46+        .quad   0x0e19b9a11f566b18+        .quad   0x220fb78a1fd1d662+        .quad   0x362a4258a381c94d+        .quad   0x9071d9132b6beb2f+        .quad   0x0f26e9ad28418247+        .quad   0xeab91ec9bdec925d+        .quad   0x4be65bc8f48af2de++        // 2^20 * 6 * B++        .quad   0x78487feba36e7028+        .quad   0x5f3f13001dd8ce34+        .quad   0x934fb12d4b30c489+        .quad   0x056c244d397f0a2b+        .quad   0x1d50fba257c26234+        .quad   0x7bd4823adeb0678b+        .quad   0xc2b0dc6ea6538af5+        .quad   0x5665eec6351da73e+        .quad   0xdb3ee00943bfb210+        .quad   0x4972018720800ac2+        .quad   0x26ab5d6173bd8667+        .quad   0x20b209c2ab204938++        // 2^20 * 7 * B++        .quad   0x549e342ac07fb34b+        .quad   0x02d8220821373d93+        .quad   0xbc262d70acd1f567+        .quad   0x7a92c9fdfbcac784+        .quad   0x1fcca94516bd3289+        .quad   0x448d65aa41420428+        .quad   0x59c3b7b216a55d62+        .quad   0x49992cc64e612cd8+        .quad   0x65bd1bea70f801de+        .quad   0x1befb7c0fe49e28a+        .quad   0xa86306cdb1b2ae4a+        .quad   0x3b7ac0cd265c2a09++        // 2^20 * 8 * B++        .quad   0x822bee438c01bcec+        .quad   0x530cb525c0fbc73b+        .quad   0x48519034c1953fe9+        .quad   0x265cc261e09a0f5b+        .quad   0xf0d54e4f22ed39a7+        .quad   0xa2aae91e5608150a+        .quad   0xf421b2e9eddae875+        .quad   0x31bc531d6b7de992+        .quad   0xdf3d134da980f971+        .quad   0x7a4fb8d1221a22a7+        .quad   0x3df7d42035aad6d8+        .quad   0x2a14edcc6a1a125e++        // 2^24 * 1 * B++        .quad   0xdf48ee0752cfce4e+        .quad   0xc3fffaf306ec08b7+        .quad   0x05710b2ab95459c4+        .quad   0x161d25fa963ea38d+        .quad   0x231a8c570478433c+        .quad   0xb7b5270ec281439d+        .quad   0xdbaa99eae3d9079f+        .quad   0x2c03f5256c2b03d9+        .quad   0x790f18757b53a47d+        .quad   0x307b0130cf0c5879+        .quad   0x31903d77257ef7f9+        .quad   0x699468bdbd96bbaf++        // 2^24 * 2 * B++        .quad   0xbd1f2f46f4dafecf+        .quad   0x7cef0114a47fd6f7+        .quad   0xd31ffdda4a47b37f+        .quad   0x525219a473905785+        .quad   0xd8dd3de66aa91948+        .quad   0x485064c22fc0d2cc+        .quad   0x9b48246634fdea2f+        .quad   0x293e1c4e6c4a2e3a+        .quad   0x376e134b925112e1+        .quad   0x703778b5dca15da0+        .quad   0xb04589af461c3111+        .quad   0x5b605c447f032823++        // 2^24 * 3 * B++        .quad   0xb965805920c47c89+        .quad   0xe7f0100c923b8fcc+        .quad   0x0001256502e2ef77+        .quad   0x24a76dcea8aeb3ee+        .quad   0x3be9fec6f0e7f04c+        .quad   0x866a579e75e34962+        .quad   0x5542ef161e1de61a+        .quad   0x2f12fef4cc5abdd5+        .quad   0x0a4522b2dfc0c740+        .quad   0x10d06e7f40c9a407+        .quad   0xc6cf144178cff668+        .quad   0x5e607b2518a43790++        // 2^24 * 4 * B++        .quad   0x58b31d8f6cdf1818+        .quad   0x35cfa74fc36258a2+        .quad   0xe1b3ff4f66e61d6e+        .quad   0x5067acab6ccdd5f7+        .quad   0xa02c431ca596cf14+        .quad   0xe3c42d40aed3e400+        .quad   0xd24526802e0f26db+        .quad   0x201f33139e457068+        .quad   0xfd527f6b08039d51+        .quad   0x18b14964017c0006+        .quad   0xd5220eb02e25a4a8+        .quad   0x397cba8862460375++        // 2^24 * 5 * B++        .quad   0x30c13093f05959b2+        .quad   0xe23aa18de9a97976+        .quad   0x222fd491721d5e26+        .quad   0x2339d320766e6c3a+        .quad   0x7815c3fbc81379e7+        .quad   0xa6619420dde12af1+        .quad   0xffa9c0f885a8fdd5+        .quad   0x771b4022c1e1c252+        .quad   0xd87dd986513a2fa7+        .quad   0xf5ac9b71f9d4cf08+        .quad   0xd06bc31b1ea283b3+        .quad   0x331a189219971a76++        // 2^24 * 6 * B++        .quad   0xf5166f45fb4f80c6+        .quad   0x9c36c7de61c775cf+        .quad   0xe3d4e81b9041d91c+        .quad   0x31167c6b83bdfe21+        .quad   0x26512f3a9d7572af+        .quad   0x5bcbe28868074a9e+        .quad   0x84edc1c11180f7c4+        .quad   0x1ac9619ff649a67b+        .quad   0xf22b3842524b1068+        .quad   0x5068343bee9ce987+        .quad   0xfc9d71844a6250c8+        .quad   0x612436341f08b111++        // 2^24 * 7 * B++        .quad   0xd99d41db874e898d+        .quad   0x09fea5f16c07dc20+        .quad   0x793d2c67d00f9bbc+        .quad   0x46ebe2309e5eff40+        .quad   0x8b6349e31a2d2638+        .quad   0x9ddfb7009bd3fd35+        .quad   0x7f8bf1b8a3a06ba4+        .quad   0x1522aa3178d90445+        .quad   0x2c382f5369614938+        .quad   0xdafe409ab72d6d10+        .quad   0xe8c83391b646f227+        .quad   0x45fe70f50524306c++        // 2^24 * 8 * B++        .quad   0xda4875a6960c0b8c+        .quad   0x5b68d076ef0e2f20+        .quad   0x07fb51cf3d0b8fd4+        .quad   0x428d1623a0e392d4+        .quad   0x62f24920c8951491+        .quad   0x05f007c83f630ca2+        .quad   0x6fbb45d2f5c9d4b8+        .quad   0x16619f6db57a2245+        .quad   0x084f4a4401a308fd+        .quad   0xa82219c376a5caac+        .quad   0xdeb8de4643d1bc7d+        .quad   0x1d81592d60bd38c6++        // 2^28 * 1 * B++        .quad   0xd833d7beec2a4c38+        .quad   0x2c9162830acc20ed+        .quad   0xe93a47aa92df7581+        .quad   0x702d67a3333c4a81+        .quad   0x3a4a369a2f89c8a1+        .quad   0x63137a1d7c8de80d+        .quad   0xbcac008a78eda015+        .quad   0x2cb8b3a5b483b03f+        .quad   0x36e417cbcb1b90a1+        .quad   0x33b3ddaa7f11794e+        .quad   0x3f510808885bc607+        .quad   0x24141dc0e6a8020d++        // 2^28 * 2 * B++        .quad   0x59f73c773fefee9d+        .quad   0xb3f1ef89c1cf989d+        .quad   0xe35dfb42e02e545f+        .quad   0x5766120b47a1b47c+        .quad   0x91925dccbd83157d+        .quad   0x3ca1205322cc8094+        .quad   0x28e57f183f90d6e4+        .quad   0x1a4714cede2e767b+        .quad   0xdb20ba0fb8b6b7ff+        .quad   0xb732c3b677511fa1+        .quad   0xa92b51c099f02d89+        .quad   0x4f3875ad489ca5f1++        // 2^28 * 3 * B++        .quad   0xc7fc762f4932ab22+        .quad   0x7ac0edf72f4c3c1b+        .quad   0x5f6b55aa9aa895e8+        .quad   0x3680274dad0a0081+        .quad   0x79ed13f6ee73eec0+        .quad   0xa5c6526d69110bb1+        .quad   0xe48928c38603860c+        .quad   0x722a1446fd7059f5+        .quad   0xd0959fe9a8cf8819+        .quad   0xd0a995508475a99c+        .quad   0x6eac173320b09cc5+        .quad   0x628ecf04331b1095++        // 2^28 * 4 * B++        .quad   0x98bcb118a9d0ddbc+        .quad   0xee449e3408b4802b+        .quad   0x87089226b8a6b104+        .quad   0x685f349a45c7915d+        .quad   0x9b41acf85c74ccf1+        .quad   0xb673318108265251+        .quad   0x99c92aed11adb147+        .quad   0x7a47d70d34ecb40f+        .quad   0x60a0c4cbcc43a4f5+        .quad   0x775c66ca3677bea9+        .quad   0xa17aa1752ff8f5ed+        .quad   0x11ded9020e01fdc0++        // 2^28 * 5 * B++        .quad   0x890e7809caefe704+        .quad   0x8728296de30e8c6c+        .quad   0x4c5cd2a392aeb1c9+        .quad   0x194263d15771531f+        .quad   0x471f95b03bea93b7+        .quad   0x0552d7d43313abd3+        .quad   0xbd9370e2e17e3f7b+        .quad   0x7b120f1db20e5bec+        .quad   0x17d2fb3d86502d7a+        .quad   0xb564d84450a69352+        .quad   0x7da962c8a60ed75d+        .quad   0x00d0f85b318736aa++        // 2^28 * 6 * B++        .quad   0x978b142e777c84fd+        .quad   0xf402644705a8c062+        .quad   0xa67ad51be7e612c7+        .quad   0x2f7b459698dd6a33+        .quad   0xa6753c1efd7621c1+        .quad   0x69c0b4a7445671f5+        .quad   0x971f527405b23c11+        .quad   0x387bc74851a8c7cd+        .quad   0x81894b4d4a52a9a8+        .quad   0xadd93e12f6b8832f+        .quad   0x184d8548b61bd638+        .quad   0x3f1c62dbd6c9f6cd++        // 2^28 * 7 * B++        .quad   0x2e8f1f0091910c1f+        .quad   0xa4df4fe0bff2e12c+        .quad   0x60c6560aee927438+        .quad   0x6338283facefc8fa+        .quad   0x3fad3e40148f693d+        .quad   0x052656e194eb9a72+        .quad   0x2f4dcbfd184f4e2f+        .quad   0x406f8db1c482e18b+        .quad   0x9e630d2c7f191ee4+        .quad   0x4fbf8301bc3ff670+        .quad   0x787d8e4e7afb73c4+        .quad   0x50d83d5be8f58fa5++        // 2^28 * 8 * B++        .quad   0x85683916c11a1897+        .quad   0x2d69a4efe506d008+        .quad   0x39af1378f664bd01+        .quad   0x65942131361517c6+        .quad   0xc0accf90b4d3b66d+        .quad   0xa7059de561732e60+        .quad   0x033d1f7870c6b0ba+        .quad   0x584161cd26d946e4+        .quad   0xbbf2b1a072d27ca2+        .quad   0xbf393c59fbdec704+        .quad   0xe98dbbcee262b81e+        .quad   0x02eebd0b3029b589++        // 2^32 * 1 * B++        .quad   0x61368756a60dac5f+        .quad   0x17e02f6aebabdc57+        .quad   0x7f193f2d4cce0f7d+        .quad   0x20234a7789ecdcf0+        .quad   0x8765b69f7b85c5e8+        .quad   0x6ff0678bd168bab2+        .quad   0x3a70e77c1d330f9b+        .quad   0x3a5f6d51b0af8e7c+        .quad   0x76d20db67178b252+        .quad   0x071c34f9d51ed160+        .quad   0xf62a4a20b3e41170+        .quad   0x7cd682353cffe366++        // 2^32 * 2 * B++        .quad   0x0be1a45bd887fab6+        .quad   0x2a846a32ba403b6e+        .quad   0xd9921012e96e6000+        .quad   0x2838c8863bdc0943+        .quad   0xa665cd6068acf4f3+        .quad   0x42d92d183cd7e3d3+        .quad   0x5759389d336025d9+        .quad   0x3ef0253b2b2cd8ff+        .quad   0xd16bb0cf4a465030+        .quad   0xfa496b4115c577ab+        .quad   0x82cfae8af4ab419d+        .quad   0x21dcb8a606a82812++        // 2^32 * 3 * B++        .quad   0x5c6004468c9d9fc8+        .quad   0x2540096ed42aa3cb+        .quad   0x125b4d4c12ee2f9c+        .quad   0x0bc3d08194a31dab+        .quad   0x9a8d00fabe7731ba+        .quad   0x8203607e629e1889+        .quad   0xb2cc023743f3d97f+        .quad   0x5d840dbf6c6f678b+        .quad   0x706e380d309fe18b+        .quad   0x6eb02da6b9e165c7+        .quad   0x57bbba997dae20ab+        .quad   0x3a4276232ac196dd++        // 2^32 * 4 * B++        .quad   0x4b42432c8a7084fa+        .quad   0x898a19e3dfb9e545+        .quad   0xbe9f00219c58e45d+        .quad   0x1ff177cea16debd1+        .quad   0x3bf8c172db447ecb+        .quad   0x5fcfc41fc6282dbd+        .quad   0x80acffc075aa15fe+        .quad   0x0770c9e824e1a9f9+        .quad   0xcf61d99a45b5b5fd+        .quad   0x860984e91b3a7924+        .quad   0xe7300919303e3e89+        .quad   0x39f264fd41500b1e++        // 2^32 * 5 * B++        .quad   0xa7ad3417dbe7e29c+        .quad   0xbd94376a2b9c139c+        .quad   0xa0e91b8e93597ba9+        .quad   0x1712d73468889840+        .quad   0xd19b4aabfe097be1+        .quad   0xa46dfce1dfe01929+        .quad   0xc3c908942ca6f1ff+        .quad   0x65c621272c35f14e+        .quad   0xe72b89f8ce3193dd+        .quad   0x4d103356a125c0bb+        .quad   0x0419a93d2e1cfe83+        .quad   0x22f9800ab19ce272++        // 2^32 * 6 * B++        .quad   0x605a368a3e9ef8cb+        .quad   0xe3e9c022a5504715+        .quad   0x553d48b05f24248f+        .quad   0x13f416cd647626e5+        .quad   0x42029fdd9a6efdac+        .quad   0xb912cebe34a54941+        .quad   0x640f64b987bdf37b+        .quad   0x4171a4d38598cab4+        .quad   0xfa2758aa99c94c8c+        .quad   0x23006f6fb000b807+        .quad   0xfbd291ddadda5392+        .quad   0x508214fa574bd1ab++        // 2^32 * 7 * B++        .quad   0xc20269153ed6fe4b+        .quad   0xa65a6739511d77c4+        .quad   0xcbde26462c14af94+        .quad   0x22f960ec6faba74b+        .quad   0x461a15bb53d003d6+        .quad   0xb2102888bcf3c965+        .quad   0x27c576756c683a5a+        .quad   0x3a7758a4c86cb447+        .quad   0x548111f693ae5076+        .quad   0x1dae21df1dfd54a6+        .quad   0x12248c90f3115e65+        .quad   0x5d9fd15f8de7f494++        // 2^32 * 8 * B++        .quad   0x031408d36d63727f+        .quad   0x6a379aefd7c7b533+        .quad   0xa9e18fc5ccaee24b+        .quad   0x332f35914f8fbed3+        .quad   0x3f244d2aeed7521e+        .quad   0x8e3a9028432e9615+        .quad   0xe164ba772e9c16d4+        .quad   0x3bc187fa47eb98d8+        .quad   0x6d470115ea86c20c+        .quad   0x998ab7cb6c46d125+        .quad   0xd77832b53a660188+        .quad   0x450d81ce906fba03++        // 2^36 * 1 * B++        .quad   0xf8ae4d2ad8453902+        .quad   0x7018058ee8db2d1d+        .quad   0xaab3995fc7d2c11e+        .quad   0x53b16d2324ccca79+        .quad   0x23264d66b2cae0b5+        .quad   0x7dbaed33ebca6576+        .quad   0x030ebed6f0d24ac8+        .quad   0x2a887f78f7635510+        .quad   0x2a23b9e75c012d4f+        .quad   0x0c974651cae1f2ea+        .quad   0x2fb63273675d70ca+        .quad   0x0ba7250b864403f5++        // 2^36 * 2 * B++        .quad   0xbb0d18fd029c6421+        .quad   0xbc2d142189298f02+        .quad   0x8347f8e68b250e96+        .quad   0x7b9f2fe8032d71c9+        .quad   0xdd63589386f86d9c+        .quad   0x61699176e13a85a4+        .quad   0x2e5111954eaa7d57+        .quad   0x32c21b57fb60bdfb+        .quad   0xd87823cd319e0780+        .quad   0xefc4cfc1897775c5+        .quad   0x4854fb129a0ab3f7+        .quad   0x12c49d417238c371++        // 2^36 * 3 * B++        .quad   0x0950b533ffe83769+        .quad   0x21861c1d8e1d6bd1+        .quad   0xf022d8381302e510+        .quad   0x2509200c6391cab4+        .quad   0x09b3a01783799542+        .quad   0x626dd08faad5ee3f+        .quad   0xba00bceeeb70149f+        .quad   0x1421b246a0a444c9+        .quad   0x4aa43a8e8c24a7c7+        .quad   0x04c1f540d8f05ef5+        .quad   0xadba5e0c0b3eb9dc+        .quad   0x2ab5504448a49ce3++        // 2^36 * 4 * B++        .quad   0x2ed227266f0f5dec+        .quad   0x9824ee415ed50824+        .quad   0x807bec7c9468d415+        .quad   0x7093bae1b521e23f+        .quad   0xdc07ac631c5d3afa+        .quad   0x58615171f9df8c6c+        .quad   0x72a079d89d73e2b0+        .quad   0x7301f4ceb4eae15d+        .quad   0x6409e759d6722c41+        .quad   0xa674e1cf72bf729b+        .quad   0xbc0a24eb3c21e569+        .quad   0x390167d24ebacb23++        // 2^36 * 5 * B++        .quad   0x27f58e3bba353f1c+        .quad   0x4c47764dbf6a4361+        .quad   0xafbbc4e56e562650+        .quad   0x07db2ee6aae1a45d+        .quad   0xd7bb054ba2f2120b+        .quad   0xe2b9ceaeb10589b7+        .quad   0x3fe8bac8f3c0edbe+        .quad   0x4cbd40767112cb69+        .quad   0x0b603cc029c58176+        .quad   0x5988e3825cb15d61+        .quad   0x2bb61413dcf0ad8d+        .quad   0x7b8eec6c74183287++        // 2^36 * 6 * B++        .quad   0xe4ca40782cd27cb0+        .quad   0xdaf9c323fbe967bd+        .quad   0xb29bd34a8ad41e9e+        .quad   0x72810497626ede4d+        .quad   0x32fee570fc386b73+        .quad   0xda8b0141da3a8cc7+        .quad   0x975ffd0ac8968359+        .quad   0x6ee809a1b132a855+        .quad   0x9444bb31fcfd863a+        .quad   0x2fe3690a3e4e48c5+        .quad   0xdc29c867d088fa25+        .quad   0x13bd1e38d173292e++        // 2^36 * 7 * B++        .quad   0xd32b4cd8696149b5+        .quad   0xe55937d781d8aab7+        .quad   0x0bcb2127ae122b94+        .quad   0x41e86fcfb14099b0+        .quad   0x223fb5cf1dfac521+        .quad   0x325c25316f554450+        .quad   0x030b98d7659177ac+        .quad   0x1ed018b64f88a4bd+        .quad   0x3630dfa1b802a6b0+        .quad   0x880f874742ad3bd5+        .quad   0x0af90d6ceec5a4d4+        .quad   0x746a247a37cdc5d9++        // 2^36 * 8 * B++        .quad   0xd531b8bd2b7b9af6+        .quad   0x5005093537fc5b51+        .quad   0x232fcf25c593546d+        .quad   0x20a365142bb40f49+        .quad   0x6eccd85278d941ed+        .quad   0x2254ae83d22f7843+        .quad   0xc522d02e7bbfcdb7+        .quad   0x681e3351bff0e4e2+        .quad   0x8b64b59d83034f45+        .quad   0x2f8b71f21fa20efb+        .quad   0x69249495ba6550e4+        .quad   0x539ef98e45d5472b++        // 2^40 * 1 * B++        .quad   0x6e7bb6a1a6205275+        .quad   0xaa4f21d7413c8e83+        .quad   0x6f56d155e88f5cb2+        .quad   0x2de25d4ba6345be1+        .quad   0xd074d8961cae743f+        .quad   0xf86d18f5ee1c63ed+        .quad   0x97bdc55be7f4ed29+        .quad   0x4cbad279663ab108+        .quad   0x80d19024a0d71fcd+        .quad   0xc525c20afb288af8+        .quad   0xb1a3974b5f3a6419+        .quad   0x7d7fbcefe2007233++        // 2^40 * 2 * B++        .quad   0xfaef1e6a266b2801+        .quad   0x866c68c4d5739f16+        .quad   0xf68a2fbc1b03762c+        .quad   0x5975435e87b75a8d+        .quad   0xcd7c5dc5f3c29094+        .quad   0xc781a29a2a9105ab+        .quad   0x80c61d36421c3058+        .quad   0x4f9cd196dcd8d4d7+        .quad   0x199297d86a7b3768+        .quad   0xd0d058241ad17a63+        .quad   0xba029cad5c1c0c17+        .quad   0x7ccdd084387a0307++        // 2^40 * 3 * B++        .quad   0xdca6422c6d260417+        .quad   0xae153d50948240bd+        .quad   0xa9c0c1b4fb68c677+        .quad   0x428bd0ed61d0cf53+        .quad   0x9b0c84186760cc93+        .quad   0xcdae007a1ab32a99+        .quad   0xa88dec86620bda18+        .quad   0x3593ca848190ca44+        .quad   0x9213189a5e849aa7+        .quad   0xd4d8c33565d8facd+        .quad   0x8c52545b53fdbbd1+        .quad   0x27398308da2d63e6++        // 2^40 * 4 * B++        .quad   0x42c38d28435ed413+        .quad   0xbd50f3603278ccc9+        .quad   0xbb07ab1a79da03ef+        .quad   0x269597aebe8c3355+        .quad   0xb9a10e4c0a702453+        .quad   0x0fa25866d57d1bde+        .quad   0xffb9d9b5cd27daf7+        .quad   0x572c2945492c33fd+        .quad   0xc77fc745d6cd30be+        .quad   0xe4dfe8d3e3baaefb+        .quad   0xa22c8830aa5dda0c+        .quad   0x7f985498c05bca80++        // 2^40 * 5 * B++        .quad   0x3849ce889f0be117+        .quad   0x8005ad1b7b54a288+        .quad   0x3da3c39f23fc921c+        .quad   0x76c2ec470a31f304+        .quad   0xd35615520fbf6363+        .quad   0x08045a45cf4dfba6+        .quad   0xeec24fbc873fa0c2+        .quad   0x30f2653cd69b12e7+        .quad   0x8a08c938aac10c85+        .quad   0x46179b60db276bcb+        .quad   0xa920c01e0e6fac70+        .quad   0x2f1273f1596473da++        // 2^40 * 6 * B++        .quad   0x4739fc7c8ae01e11+        .quad   0xfd5274904a6aab9f+        .quad   0x41d98a8287728f2e+        .quad   0x5d9e572ad85b69f2+        .quad   0x30488bd755a70bc0+        .quad   0x06d6b5a4f1d442e7+        .quad   0xead1a69ebc596162+        .quad   0x38ac1997edc5f784+        .quad   0x0666b517a751b13b+        .quad   0x747d06867e9b858c+        .quad   0xacacc011454dde49+        .quad   0x22dfcd9cbfe9e69c++        // 2^40 * 7 * B++        .quad   0x8ddbd2e0c30d0cd9+        .quad   0xad8e665facbb4333+        .quad   0x8f6b258c322a961f+        .quad   0x6b2916c05448c1c7+        .quad   0x56ec59b4103be0a1+        .quad   0x2ee3baecd259f969+        .quad   0x797cb29413f5cd32+        .quad   0x0fe9877824cde472+        .quad   0x7edb34d10aba913b+        .quad   0x4ea3cd822e6dac0e+        .quad   0x66083dff6578f815+        .quad   0x4c303f307ff00a17++        // 2^40 * 8 * B++        .quad   0xd30a3bd617b28c85+        .quad   0xc5d377b739773bea+        .quad   0xc6c6e78c1e6a5cbf+        .quad   0x0d61b8f78b2ab7c4+        .quad   0x29fc03580dd94500+        .quad   0xecd27aa46fbbec93+        .quad   0x130a155fc2e2a7f8+        .quad   0x416b151ab706a1d5+        .quad   0x56a8d7efe9c136b0+        .quad   0xbd07e5cd58e44b20+        .quad   0xafe62fda1b57e0ab+        .quad   0x191a2af74277e8d2++        // 2^44 * 1 * B++        .quad   0xd550095bab6f4985+        .quad   0x04f4cd5b4fbfaf1a+        .quad   0x9d8e2ed12a0c7540+        .quad   0x2bc24e04b2212286+        .quad   0x09d4b60b2fe09a14+        .quad   0xc384f0afdbb1747e+        .quad   0x58e2ea8978b5fd6e+        .quad   0x519ef577b5e09b0a+        .quad   0x1863d7d91124cca9+        .quad   0x7ac08145b88a708e+        .quad   0x2bcd7309857031f5+        .quad   0x62337a6e8ab8fae5++        // 2^44 * 2 * B++        .quad   0x4bcef17f06ffca16+        .quad   0xde06e1db692ae16a+        .quad   0x0753702d614f42b0+        .quad   0x5f6041b45b9212d0+        .quad   0xd1ab324e1b3a1273+        .quad   0x18947cf181055340+        .quad   0x3b5d9567a98c196e+        .quad   0x7fa00425802e1e68+        .quad   0x7d531574028c2705+        .quad   0x80317d69db0d75fe+        .quad   0x30fface8ef8c8ddd+        .quad   0x7e9de97bb6c3e998++        // 2^44 * 3 * B++        .quad   0x1558967b9e6585a3+        .quad   0x97c99ce098e98b92+        .quad   0x10af149b6eb3adad+        .quad   0x42181fe8f4d38cfa+        .quad   0xf004be62a24d40dd+        .quad   0xba0659910452d41f+        .quad   0x81c45ee162a44234+        .quad   0x4cb829d8a22266ef+        .quad   0x1dbcaa8407b86681+        .quad   0x081f001e8b26753b+        .quad   0x3cd7ce6a84048e81+        .quad   0x78af11633f25f22c++        // 2^44 * 4 * B++        .quad   0x8416ebd40b50babc+        .quad   0x1508722628208bee+        .quad   0xa3148fafb9c1c36d+        .quad   0x0d07daacd32d7d5d+        .quad   0x3241c00e7d65318c+        .quad   0xe6bee5dcd0e86de7+        .quad   0x118b2dc2fbc08c26+        .quad   0x680d04a7fc603dc3+        .quad   0xf9c2414a695aa3eb+        .quad   0xdaa42c4c05a68f21+        .quad   0x7c6c23987f93963e+        .quad   0x210e8cd30c3954e3++        // 2^44 * 5 * B++        .quad   0xac4201f210a71c06+        .quad   0x6a65e0aef3bfb021+        .quad   0xbc42c35c393632f7+        .quad   0x56ea8db1865f0742+        .quad   0x2b50f16137fe6c26+        .quad   0xe102bcd856e404d8+        .quad   0x12b0f1414c561f6b+        .quad   0x51b17bc8d028ec91+        .quad   0xfff5fb4bcf535119+        .quad   0xf4989d79df1108a0+        .quad   0xbdfcea659a3ba325+        .quad   0x18a11f1174d1a6f2++        // 2^44 * 6 * B++        .quad   0x407375ab3f6bba29+        .quad   0x9ec3b6d8991e482e+        .quad   0x99c80e82e55f92e9+        .quad   0x307c13b6fb0c0ae1+        .quad   0xfbd63cdad27a5f2c+        .quad   0xf00fc4bc8aa106d7+        .quad   0x53fb5c1a8e64a430+        .quad   0x04eaabe50c1a2e85+        .quad   0x24751021cb8ab5e7+        .quad   0xfc2344495c5010eb+        .quad   0x5f1e717b4e5610a1+        .quad   0x44da5f18c2710cd5++        // 2^44 * 7 * B++        .quad   0x033cc55ff1b82eb5+        .quad   0xb15ae36d411cae52+        .quad   0xba40b6198ffbacd3+        .quad   0x768edce1532e861f+        .quad   0x9156fe6b89d8eacc+        .quad   0xe6b79451e23126a1+        .quad   0xbd7463d93944eb4e+        .quad   0x726373f6767203ae+        .quad   0xe305ca72eb7ef68a+        .quad   0x662cf31f70eadb23+        .quad   0x18f026fdb4c45b68+        .quad   0x513b5384b5d2ecbd++        // 2^44 * 8 * B++        .quad   0x46d46280c729989e+        .quad   0x4b93fbd05368a5dd+        .quad   0x63df3f81d1765a89+        .quad   0x34cebd64b9a0a223+        .quad   0x5e2702878af34ceb+        .quad   0x900b0409b946d6ae+        .quad   0x6512ebf7dabd8512+        .quad   0x61d9b76988258f81+        .quad   0xa6c5a71349b7d94b+        .quad   0xa3f3d15823eb9446+        .quad   0x0416fbd277484834+        .quad   0x69d45e6f2c70812f++        // 2^48 * 1 * B++        .quad   0xce16f74bc53c1431+        .quad   0x2b9725ce2072edde+        .quad   0xb8b9c36fb5b23ee7+        .quad   0x7e2e0e450b5cc908+        .quad   0x9fe62b434f460efb+        .quad   0xded303d4a63607d6+        .quad   0xf052210eb7a0da24+        .quad   0x237e7dbe00545b93+        .quad   0x013575ed6701b430+        .quad   0x231094e69f0bfd10+        .quad   0x75320f1583e47f22+        .quad   0x71afa699b11155e3++        // 2^48 * 2 * B++        .quad   0x65ce6f9b3953b61d+        .quad   0xc65839eaafa141e6+        .quad   0x0f435ffda9f759fe+        .quad   0x021142e9c2b1c28e+        .quad   0xea423c1c473b50d6+        .quad   0x51e87a1f3b38ef10+        .quad   0x9b84bf5fb2c9be95+        .quad   0x00731fbc78f89a1c+        .quad   0xe430c71848f81880+        .quad   0xbf960c225ecec119+        .quad   0xb6dae0836bba15e3+        .quad   0x4c4d6f3347e15808++        // 2^48 * 3 * B++        .quad   0x18f7eccfc17d1fc9+        .quad   0x6c75f5a651403c14+        .quad   0xdbde712bf7ee0cdf+        .quad   0x193fddaaa7e47a22+        .quad   0x2f0cddfc988f1970+        .quad   0x6b916227b0b9f51b+        .quad   0x6ec7b6c4779176be+        .quad   0x38bf9500a88f9fa8+        .quad   0x1fd2c93c37e8876f+        .quad   0xa2f61e5a18d1462c+        .quad   0x5080f58239241276+        .quad   0x6a6fb99ebf0d4969++        // 2^48 * 4 * B++        .quad   0x6a46c1bb560855eb+        .quad   0x2416bb38f893f09d+        .quad   0xd71d11378f71acc1+        .quad   0x75f76914a31896ea+        .quad   0xeeb122b5b6e423c6+        .quad   0x939d7010f286ff8e+        .quad   0x90a92a831dcf5d8c+        .quad   0x136fda9f42c5eb10+        .quad   0xf94cdfb1a305bdd1+        .quad   0x0f364b9d9ff82c08+        .quad   0x2a87d8a5c3bb588a+        .quad   0x022183510be8dcba++        // 2^48 * 5 * B++        .quad   0x4af766385ead2d14+        .quad   0xa08ed880ca7c5830+        .quad   0x0d13a6e610211e3d+        .quad   0x6a071ce17b806c03+        .quad   0x9d5a710143307a7f+        .quad   0xb063de9ec47da45f+        .quad   0x22bbfe52be927ad3+        .quad   0x1387c441fd40426c+        .quad   0xb5d3c3d187978af8+        .quad   0x722b5a3d7f0e4413+        .quad   0x0d7b4848bb477ca0+        .quad   0x3171b26aaf1edc92++        // 2^48 * 6 * B++        .quad   0xa92f319097564ca8+        .quad   0xff7bb84c2275e119+        .quad   0x4f55fe37a4875150+        .quad   0x221fd4873cf0835a+        .quad   0xa60db7d8b28a47d1+        .quad   0xa6bf14d61770a4f1+        .quad   0xd4a1f89353ddbd58+        .quad   0x6c514a63344243e9+        .quad   0x2322204f3a156341+        .quad   0xfb73e0e9ba0a032d+        .quad   0xfce0dd4c410f030e+        .quad   0x48daa596fb924aaa++        // 2^48 * 7 * B++        .quad   0x6eca8e665ca59cc7+        .quad   0xa847254b2e38aca0+        .quad   0x31afc708d21e17ce+        .quad   0x676dd6fccad84af7+        .quad   0x14f61d5dc84c9793+        .quad   0x9941f9e3ef418206+        .quad   0xcdf5b88f346277ac+        .quad   0x58c837fa0e8a79a9+        .quad   0x0cf9688596fc9058+        .quad   0x1ddcbbf37b56a01b+        .quad   0xdcc2e77d4935d66a+        .quad   0x1c4f73f2c6a57f0a++        // 2^48 * 8 * B++        .quad   0x0e7a4fbd305fa0bb+        .quad   0x829d4ce054c663ad+        .quad   0xf421c3832fe33848+        .quad   0x795ac80d1bf64c42+        .quad   0xb36e706efc7c3484+        .quad   0x73dfc9b4c3c1cf61+        .quad   0xeb1d79c9781cc7e5+        .quad   0x70459adb7daf675c+        .quad   0x1b91db4991b42bb3+        .quad   0x572696234b02dcca+        .quad   0x9fdf9ee51f8c78dc+        .quad   0x5fe162848ce21fd3++        // 2^52 * 1 * B++        .quad   0xe2790aae4d077c41+        .quad   0x8b938270db7469a3+        .quad   0x6eb632dc8abd16a2+        .quad   0x720814ecaa064b72+        .quad   0x315c29c795115389+        .quad   0xd7e0e507862f74ce+        .quad   0x0c4a762185927432+        .quad   0x72de6c984a25a1e4+        .quad   0xae9ab553bf6aa310+        .quad   0x050a50a9806d6e1b+        .quad   0x92bb7403adff5139+        .quad   0x0394d27645be618b++        // 2^52 * 2 * B++        .quad   0x4d572251857eedf4+        .quad   0xe3724edde19e93c5+        .quad   0x8a71420e0b797035+        .quad   0x3b3c833687abe743+        .quad   0xf5396425b23545a4+        .quad   0x15a7a27e98fbb296+        .quad   0xab6c52bc636fdd86+        .quad   0x79d995a8419334ee+        .quad   0xcd8a8ea61195dd75+        .quad   0xa504d8a81dd9a82f+        .quad   0x540dca81a35879b6+        .quad   0x60dd16a379c86a8a++        // 2^52 * 3 * B++        .quad   0x35a2c8487381e559+        .quad   0x596ffea6d78082cb+        .quad   0xcb9771ebdba7b653+        .quad   0x5a08b5019b4da685+        .quad   0x3501d6f8153e47b8+        .quad   0xb7a9675414a2f60c+        .quad   0x112ee8b6455d9523+        .quad   0x4e62a3c18112ea8a+        .quad   0xc8d4ac04516ab786+        .quad   0x595af3215295b23d+        .quad   0xd6edd234db0230c1+        .quad   0x0929efe8825b41cc++        // 2^52 * 4 * B++        .quad   0x5f0601d1cbd0f2d3+        .quad   0x736e412f6132bb7f+        .quad   0x83604432238dde87+        .quad   0x1e3a5272f5c0753c+        .quad   0x8b3172b7ad56651d+        .quad   0x01581b7a3fabd717+        .quad   0x2dc94df6424df6e4+        .quad   0x30376e5d2c29284f+        .quad   0xd2918da78159a59c+        .quad   0x6bdc1cd93f0713f3+        .quad   0x565f7a934acd6590+        .quad   0x53daacec4cb4c128++        // 2^52 * 5 * B++        .quad   0x4ca73bd79cc8a7d6+        .quad   0x4d4a738f47e9a9b2+        .quad   0xf4cbf12942f5fe00+        .quad   0x01a13ff9bdbf0752+        .quad   0x99852bc3852cfdb0+        .quad   0x2cc12e9559d6ed0b+        .quad   0x70f9e2bf9b5ac27b+        .quad   0x4f3b8c117959ae99+        .quad   0x55b6c9c82ff26412+        .quad   0x1ac4a8c91fb667a8+        .quad   0xd527bfcfeb778bf2+        .quad   0x303337da7012a3be++        // 2^52 * 6 * B++        .quad   0x955422228c1c9d7c+        .quad   0x01fac1371a9b340f+        .quad   0x7e8d9177925b48d7+        .quad   0x53f8ad5661b3e31b+        .quad   0x976d3ccbfad2fdd1+        .quad   0xcb88839737a640a8+        .quad   0x2ff00c1d6734cb25+        .quad   0x269ff4dc789c2d2b+        .quad   0x0c003fbdc08d678d+        .quad   0x4d982fa37ead2b17+        .quad   0xc07e6bcdb2e582f1+        .quad   0x296c7291df412a44++        // 2^52 * 7 * B++        .quad   0x7903de2b33daf397+        .quad   0xd0ff0619c9a624b3+        .quad   0x8a1d252b555b3e18+        .quad   0x2b6d581c52e0b7c0+        .quad   0xdfb23205dab8b59e+        .quad   0x465aeaa0c8092250+        .quad   0xd133c1189a725d18+        .quad   0x2327370261f117d1+        .quad   0x3d0543d3623e7986+        .quad   0x679414c2c278a354+        .quad   0xae43f0cc726196f6+        .quad   0x7836c41f8245eaba++        // 2^52 * 8 * B++        .quad   0xe7a254db49e95a81+        .quad   0x5192d5d008b0ad73+        .quad   0x4d20e5b1d00afc07+        .quad   0x5d55f8012cf25f38+        .quad   0xca651e848011937c+        .quad   0xc6b0c46e6ef41a28+        .quad   0xb7021ba75f3f8d52+        .quad   0x119dff99ead7b9fd+        .quad   0x43eadfcbf4b31d4d+        .quad   0xc6503f7411148892+        .quad   0xfeee68c5060d3b17+        .quad   0x329293b3dd4a0ac8++        // 2^56 * 1 * B++        .quad   0x4e59214fe194961a+        .quad   0x49be7dc70d71cd4f+        .quad   0x9300cfd23b50f22d+        .quad   0x4789d446fc917232+        .quad   0x2879852d5d7cb208+        .quad   0xb8dedd70687df2e7+        .quad   0xdc0bffab21687891+        .quad   0x2b44c043677daa35+        .quad   0x1a1c87ab074eb78e+        .quad   0xfac6d18e99daf467+        .quad   0x3eacbbcd484f9067+        .quad   0x60c52eef2bb9a4e4++        // 2^56 * 2 * B++        .quad   0x0b5d89bc3bfd8bf1+        .quad   0xb06b9237c9f3551a+        .quad   0x0e4c16b0d53028f5+        .quad   0x10bc9c312ccfcaab+        .quad   0x702bc5c27cae6d11+        .quad   0x44c7699b54a48cab+        .quad   0xefbc4056ba492eb2+        .quad   0x70d77248d9b6676d+        .quad   0xaa8ae84b3ec2a05b+        .quad   0x98699ef4ed1781e0+        .quad   0x794513e4708e85d1+        .quad   0x63755bd3a976f413++        // 2^56 * 3 * B++        .quad   0xb55fa03e2ad10853+        .quad   0x356f75909ee63569+        .quad   0x9ff9f1fdbe69b890+        .quad   0x0d8cc1c48bc16f84+        .quad   0x3dc7101897f1acb7+        .quad   0x5dda7d5ec165bbd8+        .quad   0x508e5b9c0fa1020f+        .quad   0x2763751737c52a56+        .quad   0x029402d36eb419a9+        .quad   0xf0b44e7e77b460a5+        .quad   0xcfa86230d43c4956+        .quad   0x70c2dd8a7ad166e7++        // 2^56 * 4 * B++        .quad   0x656194509f6fec0e+        .quad   0xee2e7ea946c6518d+        .quad   0x9733c1f367e09b5c+        .quad   0x2e0fac6363948495+        .quad   0x91d4967db8ed7e13+        .quad   0x74252f0ad776817a+        .quad   0xe40982e00d852564+        .quad   0x32b8613816a53ce5+        .quad   0x79e7f7bee448cd64+        .quad   0x6ac83a67087886d0+        .quad   0xf89fd4d9a0e4db2e+        .quad   0x4179215c735a4f41++        // 2^56 * 5 * B++        .quad   0x8c7094e7d7dced2a+        .quad   0x97fb8ac347d39c70+        .quad   0xe13be033a906d902+        .quad   0x700344a30cd99d76+        .quad   0xe4ae33b9286bcd34+        .quad   0xb7ef7eb6559dd6dc+        .quad   0x278b141fb3d38e1f+        .quad   0x31fa85662241c286+        .quad   0xaf826c422e3622f4+        .quad   0xc12029879833502d+        .quad   0x9bc1b7e12b389123+        .quad   0x24bb2312a9952489++        // 2^56 * 6 * B++        .quad   0xb1a8ed1732de67c3+        .quad   0x3cb49418461b4948+        .quad   0x8ebd434376cfbcd2+        .quad   0x0fee3e871e188008+        .quad   0x41f80c2af5f85c6b+        .quad   0x687284c304fa6794+        .quad   0x8945df99a3ba1bad+        .quad   0x0d1d2af9ffeb5d16+        .quad   0xa9da8aa132621edf+        .quad   0x30b822a159226579+        .quad   0x4004197ba79ac193+        .quad   0x16acd79718531d76++        // 2^56 * 7 * B++        .quad   0x72df72af2d9b1d3d+        .quad   0x63462a36a432245a+        .quad   0x3ecea07916b39637+        .quad   0x123e0ef6b9302309+        .quad   0xc959c6c57887b6ad+        .quad   0x94e19ead5f90feba+        .quad   0x16e24e62a342f504+        .quad   0x164ed34b18161700+        .quad   0x487ed94c192fe69a+        .quad   0x61ae2cea3a911513+        .quad   0x877bf6d3b9a4de27+        .quad   0x78da0fc61073f3eb++        // 2^56 * 8 * B++        .quad   0x5bf15d28e52bc66a+        .quad   0x2c47e31870f01a8e+        .quad   0x2419afbc06c28bdd+        .quad   0x2d25deeb256b173a+        .quad   0xa29f80f1680c3a94+        .quad   0x71f77e151ae9e7e6+        .quad   0x1100f15848017973+        .quad   0x054aa4b316b38ddd+        .quad   0xdfc8468d19267cb8+        .quad   0x0b28789c66e54daf+        .quad   0x2aeb1d2a666eec17+        .quad   0x134610a6ab7da760++        // 2^60 * 1 * B++        .quad   0xcaf55ec27c59b23f+        .quad   0x99aeed3e154d04f2+        .quad   0x68441d72e14141f4+        .quad   0x140345133932a0a2+        .quad   0xd91430e0dc028c3c+        .quad   0x0eb955a85217c771+        .quad   0x4b09e1ed2c99a1fa+        .quad   0x42881af2bd6a743c+        .quad   0x7bfec69aab5cad3d+        .quad   0xc23e8cd34cb2cfad+        .quad   0x685dd14bfb37d6a2+        .quad   0x0ad6d64415677a18++        // 2^60 * 2 * B++        .quad   0x781a439e417becb5+        .quad   0x4ac5938cd10e0266+        .quad   0x5da385110692ac24+        .quad   0x11b065a2ade31233+        .quad   0x7914892847927e9f+        .quad   0x33dad6ef370aa877+        .quad   0x1f8f24fa11122703+        .quad   0x5265ac2f2adf9592+        .quad   0x405fdd309afcb346+        .quad   0xd9723d4428e63f54+        .quad   0x94c01df05f65aaae+        .quad   0x43e4dc3ae14c0809++        // 2^60 * 3 * B++        .quad   0xbc12c7f1a938a517+        .quad   0x473028ab3180b2e1+        .quad   0x3f78571efbcd254a+        .quad   0x74e534426ff6f90f+        .quad   0xea6f7ac3adc2c6a3+        .quad   0xd0e928f6e9717c94+        .quad   0xe2d379ead645eaf5+        .quad   0x46dd8785c51ffbbe+        .quad   0x709801be375c8898+        .quad   0x4b06dab5e3fd8348+        .quad   0x75880ced27230714+        .quad   0x2b09468fdd2f4c42++        // 2^60 * 4 * B++        .quad   0x97c749eeb701cb96+        .quad   0x83f438d4b6a369c3+        .quad   0x62962b8b9a402cd9+        .quad   0x6976c7509888df7b+        .quad   0x5b97946582ffa02a+        .quad   0xda096a51fea8f549+        .quad   0xa06351375f77af9b+        .quad   0x1bcfde61201d1e76+        .quad   0x4a4a5490246a59a2+        .quad   0xd63ebddee87fdd90+        .quad   0xd9437c670d2371fa+        .quad   0x69e87308d30f8ed6++        // 2^60 * 5 * B++        .quad   0x435a8bb15656beb0+        .quad   0xf8fac9ba4f4d5bca+        .quad   0xb9b278c41548c075+        .quad   0x3eb0ef76e892b622+        .quad   0x0f80bf028bc80303+        .quad   0x6aae16b37a18cefb+        .quad   0xdd47ea47d72cd6a3+        .quad   0x61943588f4ed39aa+        .quad   0xd26e5c3e91039f85+        .quad   0xc0e9e77df6f33aa9+        .quad   0xe8968c5570066a93+        .quad   0x3c34d1881faaaddd++        // 2^60 * 6 * B++        .quad   0x3f9d2b5ea09f9ec0+        .quad   0x1dab3b6fb623a890+        .quad   0xa09ba3ea72d926c4+        .quad   0x374193513fd8b36d+        .quad   0xbd5b0b8f2fffe0d9+        .quad   0x6aa254103ed24fb9+        .quad   0x2ac7d7bcb26821c4+        .quad   0x605b394b60dca36a+        .quad   0xb4e856e45a9d1ed2+        .quad   0xefe848766c97a9a2+        .quad   0xb104cf641e5eee7d+        .quad   0x2f50b81c88a71c8f++        // 2^60 * 7 * B++        .quad   0x31723c61fc6811bb+        .quad   0x9cb450486211800f+        .quad   0x768933d347995753+        .quad   0x3491a53502752fcd+        .quad   0x2b552ca0a7da522a+        .quad   0x3230b336449b0250+        .quad   0xf2c4c5bca4b99fb9+        .quad   0x7b2c674958074a22+        .quad   0xd55165883ed28cdf+        .quad   0x12d84fd2d362de39+        .quad   0x0a874ad3e3378e4f+        .quad   0x000d2b1f7c763e74++        // 2^60 * 8 * B++        .quad   0x3d420811d06d4a67+        .quad   0xbefc048590e0ffe3+        .quad   0xf870c6b7bd487bde+        .quad   0x6e2a7316319afa28+        .quad   0x9624778c3e94a8ab+        .quad   0x0ad6f3cee9a78bec+        .quad   0x948ac7810d743c4f+        .quad   0x76627935aaecfccc+        .quad   0x56a8ac24d6d59a9f+        .quad   0xc8db753e3096f006+        .quad   0x477f41e68f4c5299+        .quad   0x588d851cf6c86114++        // 2^64 * 1 * B++        .quad   0x51138ec78df6b0fe+        .quad   0x5397da89e575f51b+        .quad   0x09207a1d717af1b9+        .quad   0x2102fdba2b20d650+        .quad   0xcd2a65e777d1f515+        .quad   0x548991878faa60f1+        .quad   0xb1b73bbcdabc06e5+        .quad   0x654878cba97cc9fb+        .quad   0x969ee405055ce6a1+        .quad   0x36bca7681251ad29+        .quad   0x3a1af517aa7da415+        .quad   0x0ad725db29ecb2ba++        // 2^64 * 2 * B++        .quad   0xdc4267b1834e2457+        .quad   0xb67544b570ce1bc5+        .quad   0x1af07a0bf7d15ed7+        .quad   0x4aefcffb71a03650+        .quad   0xfec7bc0c9b056f85+        .quad   0x537d5268e7f5ffd7+        .quad   0x77afc6624312aefa+        .quad   0x4f675f5302399fd9+        .quad   0xc32d36360415171e+        .quad   0xcd2bef118998483b+        .quad   0x870a6eadd0945110+        .quad   0x0bccbb72a2a86561++        // 2^64 * 3 * B++        .quad   0x185e962feab1a9c8+        .quad   0x86e7e63565147dcd+        .quad   0xb092e031bb5b6df2+        .quad   0x4024f0ab59d6b73e+        .quad   0x186d5e4c50fe1296+        .quad   0xe0397b82fee89f7e+        .quad   0x3bc7f6c5507031b0+        .quad   0x6678fd69108f37c2+        .quad   0x1586fa31636863c2+        .quad   0x07f68c48572d33f2+        .quad   0x4f73cc9f789eaefc+        .quad   0x2d42e2108ead4701++        // 2^64 * 4 * B++        .quad   0x97f5131594dfd29b+        .quad   0x6155985d313f4c6a+        .quad   0xeba13f0708455010+        .quad   0x676b2608b8d2d322+        .quad   0x21717b0d0f537593+        .quad   0x914e690b131e064c+        .quad   0x1bb687ae752ae09f+        .quad   0x420bf3a79b423c6e+        .quad   0x8138ba651c5b2b47+        .quad   0x8671b6ec311b1b80+        .quad   0x7bff0cb1bc3135b0+        .quad   0x745d2ffa9c0cf1e0++        // 2^64 * 5 * B++        .quad   0xbf525a1e2bc9c8bd+        .quad   0xea5b260826479d81+        .quad   0xd511c70edf0155db+        .quad   0x1ae23ceb960cf5d0+        .quad   0x6036df5721d34e6a+        .quad   0xb1db8827997bb3d0+        .quad   0xd3c209c3c8756afa+        .quad   0x06e15be54c1dc839+        .quad   0x5b725d871932994a+        .quad   0x32351cb5ceb1dab0+        .quad   0x7dc41549dab7ca05+        .quad   0x58ded861278ec1f7++        // 2^64 * 6 * B++        .quad   0xd8173793f266c55c+        .quad   0xc8c976c5cc454e49+        .quad   0x5ce382f8bc26c3a8+        .quad   0x2ff39de85485f6f9+        .quad   0x2dfb5ba8b6c2c9a8+        .quad   0x48eeef8ef52c598c+        .quad   0x33809107f12d1573+        .quad   0x08ba696b531d5bd8+        .quad   0x77ed3eeec3efc57a+        .quad   0x04e05517d4ff4811+        .quad   0xea3d7a3ff1a671cb+        .quad   0x120633b4947cfe54++        // 2^64 * 7 * B++        .quad   0x0b94987891610042+        .quad   0x4ee7b13cecebfae8+        .quad   0x70be739594f0a4c0+        .quad   0x35d30a99b4d59185+        .quad   0x82bd31474912100a+        .quad   0xde237b6d7e6fbe06+        .quad   0xe11e761911ea79c6+        .quad   0x07433be3cb393bde+        .quad   0xff7944c05ce997f4+        .quad   0x575d3de4b05c51a3+        .quad   0x583381fd5a76847c+        .quad   0x2d873ede7af6da9f++        // 2^64 * 8 * B++        .quad   0x157a316443373409+        .quad   0xfab8b7eef4aa81d9+        .quad   0xb093fee6f5a64806+        .quad   0x2e773654707fa7b6+        .quad   0xaa6202e14e5df981+        .quad   0xa20d59175015e1f5+        .quad   0x18a275d3bae21d6c+        .quad   0x0543618a01600253+        .quad   0x0deabdf4974c23c1+        .quad   0xaa6f0a259dce4693+        .quad   0x04202cb8a29aba2c+        .quad   0x4b1443362d07960d++        // 2^68 * 1 * B++        .quad   0x47b837f753242cec+        .quad   0x256dc48cc04212f2+        .quad   0xe222fbfbe1d928c5+        .quad   0x48ea295bad8a2c07+        .quad   0x299b1c3f57c5715e+        .quad   0x96cb929e6b686d90+        .quad   0x3004806447235ab3+        .quad   0x2c435c24a44d9fe1+        .quad   0x0607c97c80f8833f+        .quad   0x0e851578ca25ec5b+        .quad   0x54f7450b161ebb6f+        .quad   0x7bcb4792a0def80e++        // 2^68 * 2 * B++        .quad   0x8487e3d02bc73659+        .quad   0x4baf8445059979df+        .quad   0xd17c975adcad6fbf+        .quad   0x57369f0bdefc96b6+        .quad   0x1cecd0a0045224c2+        .quad   0x757f1b1b69e53952+        .quad   0x775b7a925289f681+        .quad   0x1b6cc62016736148+        .quad   0xf1a9990175638698+        .quad   0x353dd1beeeaa60d3+        .quad   0x849471334c9ba488+        .quad   0x63fa6e6843ade311++        // 2^68 * 3 * B++        .quad   0xd15c20536597c168+        .quad   0x9f73740098d28789+        .quad   0x18aee7f13257ba1f+        .quad   0x3418bfda07346f14+        .quad   0x2195becdd24b5eb7+        .quad   0x5e41f18cc0cd44f9+        .quad   0xdf28074441ca9ede+        .quad   0x07073b98f35b7d67+        .quad   0xd03c676c4ce530d4+        .quad   0x0b64c0473b5df9f4+        .quad   0x065cef8b19b3a31e+        .quad   0x3084d661533102c9++        // 2^68 * 4 * B++        .quad   0xe1f6b79ebf8469ad+        .quad   0x15801004e2663135+        .quad   0x9a498330af74181b+        .quad   0x3ba2504f049b673c+        .quad   0x9a6ce876760321fd+        .quad   0x7fe2b5109eb63ad8+        .quad   0x00e7d4ae8ac80592+        .quad   0x73d86b7abb6f723a+        .quad   0x0b52b5606dba5ab6+        .quad   0xa9134f0fbbb1edab+        .quad   0x30a9520d9b04a635+        .quad   0x6813b8f37973e5db++        // 2^68 * 5 * B++        .quad   0x9854b054334127c1+        .quad   0x105d047882fbff25+        .quad   0xdb49f7f944186f4f+        .quad   0x1768e838bed0b900+        .quad   0xf194ca56f3157e29+        .quad   0x136d35705ef528a5+        .quad   0xdd4cef778b0599bc+        .quad   0x7d5472af24f833ed+        .quad   0xd0ef874daf33da47+        .quad   0x00d3be5db6e339f9+        .quad   0x3f2a8a2f9c9ceece+        .quad   0x5d1aeb792352435a++        // 2^68 * 6 * B++        .quad   0xf59e6bb319cd63ca+        .quad   0x670c159221d06839+        .quad   0xb06d565b2150cab6+        .quad   0x20fb199d104f12a3+        .quad   0x12c7bfaeb61ba775+        .quad   0xb84e621fe263bffd+        .quad   0x0b47a5c35c840dcf+        .quad   0x7e83be0bccaf8634+        .quad   0x61943dee6d99c120+        .quad   0x86101f2e460b9fe0+        .quad   0x6bb2f1518ee8598d+        .quad   0x76b76289fcc475cc++        // 2^68 * 7 * B++        .quad   0x791b4cc1756286fa+        .quad   0xdbced317d74a157c+        .quad   0x7e732421ea72bde6+        .quad   0x01fe18491131c8e9+        .quad   0x4245f1a1522ec0b3+        .quad   0x558785b22a75656d+        .quad   0x1d485a2548a1b3c0+        .quad   0x60959eccd58fe09f+        .quad   0x3ebfeb7ba8ed7a09+        .quad   0x49fdc2bbe502789c+        .quad   0x44ebce5d3c119428+        .quad   0x35e1eb55be947f4a++        // 2^68 * 8 * B++        .quad   0xdbdae701c5738dd3+        .quad   0xf9c6f635b26f1bee+        .quad   0x61e96a8042f15ef4+        .quad   0x3aa1d11faf60a4d8+        .quad   0x14fd6dfa726ccc74+        .quad   0x3b084cfe2f53b965+        .quad   0xf33ae4f552a2c8b4+        .quad   0x59aab07a0d40166a+        .quad   0x77bcec4c925eac25+        .quad   0x1848718460137738+        .quad   0x5b374337fea9f451+        .quad   0x1865e78ec8e6aa46++        // 2^72 * 1 * B++        .quad   0xccc4b7c7b66e1f7a+        .quad   0x44157e25f50c2f7e+        .quad   0x3ef06dfc713eaf1c+        .quad   0x582f446752da63f7+        .quad   0x967c54e91c529ccb+        .quad   0x30f6269264c635fb+        .quad   0x2747aff478121965+        .quad   0x17038418eaf66f5c+        .quad   0xc6317bd320324ce4+        .quad   0xa81042e8a4488bc4+        .quad   0xb21ef18b4e5a1364+        .quad   0x0c2a1c4bcda28dc9++        // 2^72 * 2 * B++        .quad   0xd24dc7d06f1f0447+        .quad   0xb2269e3edb87c059+        .quad   0xd15b0272fbb2d28f+        .quad   0x7c558bd1c6f64877+        .quad   0xedc4814869bd6945+        .quad   0x0d6d907dbe1c8d22+        .quad   0xc63bd212d55cc5ab+        .quad   0x5a6a9b30a314dc83+        .quad   0xd0ec1524d396463d+        .quad   0x12bb628ac35a24f0+        .quad   0xa50c3a791cbc5fa4+        .quad   0x0404a5ca0afbafc3++        // 2^72 * 3 * B++        .quad   0x8c1f40070aa743d6+        .quad   0xccbad0cb5b265ee8+        .quad   0x574b046b668fd2de+        .quad   0x46395bfdcadd9633+        .quad   0x62bc9e1b2a416fd1+        .quad   0xb5c6f728e350598b+        .quad   0x04343fd83d5d6967+        .quad   0x39527516e7f8ee98+        .quad   0x117fdb2d1a5d9a9c+        .quad   0x9c7745bcd1005c2a+        .quad   0xefd4bef154d56fea+        .quad   0x76579a29e822d016++        // 2^72 * 4 * B++        .quad   0x45b68e7e49c02a17+        .quad   0x23cd51a2bca9a37f+        .quad   0x3ed65f11ec224c1b+        .quad   0x43a384dc9e05bdb1+        .quad   0x333cb51352b434f2+        .quad   0xd832284993de80e1+        .quad   0xb5512887750d35ce+        .quad   0x02c514bb2a2777c1+        .quad   0x684bd5da8bf1b645+        .quad   0xfb8bd37ef6b54b53+        .quad   0x313916d7a9b0d253+        .quad   0x1160920961548059++        // 2^72 * 5 * B++        .quad   0xb44d166929dacfaa+        .quad   0xda529f4c8413598f+        .quad   0xe9ef63ca453d5559+        .quad   0x351e125bc5698e0b+        .quad   0x7a385616369b4dcd+        .quad   0x75c02ca7655c3563+        .quad   0x7dc21bf9d4f18021+        .quad   0x2f637d7491e6e042+        .quad   0xd4b49b461af67bbe+        .quad   0xd603037ac8ab8961+        .quad   0x71dee19ff9a699fb+        .quad   0x7f182d06e7ce2a9a++        // 2^72 * 6 * B++        .quad   0x7a7c8e64ab0168ec+        .quad   0xcb5a4a5515edc543+        .quad   0x095519d347cd0eda+        .quad   0x67d4ac8c343e93b0+        .quad   0x09454b728e217522+        .quad   0xaa58e8f4d484b8d8+        .quad   0xd358254d7f46903c+        .quad   0x44acc043241c5217+        .quad   0x1c7d6bbb4f7a5777+        .quad   0x8b35fed4918313e1+        .quad   0x4adca1c6c96b4684+        .quad   0x556d1c8312ad71bd++        // 2^72 * 7 * B++        .quad   0x17ef40e30c8d3982+        .quad   0x31f7073e15a3fa34+        .quad   0x4f21f3cb0773646e+        .quad   0x746c6c6d1d824eff+        .quad   0x81f06756b11be821+        .quad   0x0faff82310a3f3dd+        .quad   0xf8b2d0556a99465d+        .quad   0x097abe38cc8c7f05+        .quad   0x0c49c9877ea52da4+        .quad   0x4c4369559bdc1d43+        .quad   0x022c3809f7ccebd2+        .quad   0x577e14a34bee84bd++        // 2^72 * 8 * B++        .quad   0xf0e268ac61a73b0a+        .quad   0xf2fafa103791a5f5+        .quad   0xc1e13e826b6d00e9+        .quad   0x60fa7ee96fd78f42+        .quad   0x94fecebebd4dd72b+        .quad   0xf46a4fda060f2211+        .quad   0x124a5977c0c8d1ff+        .quad   0x705304b8fb009295+        .quad   0xb63d1d354d296ec6+        .quad   0xf3c3053e5fad31d8+        .quad   0x670b958cb4bd42ec+        .quad   0x21398e0ca16353fd++        // 2^76 * 1 * B++        .quad   0x216ab2ca8da7d2ef+        .quad   0x366ad9dd99f42827+        .quad   0xae64b9004fdd3c75+        .quad   0x403a395b53909e62+        .quad   0x86c5fc16861b7e9a+        .quad   0xf6a330476a27c451+        .quad   0x01667267a1e93597+        .quad   0x05ffb9cd6082dfeb+        .quad   0xa617fa9ff53f6139+        .quad   0x60f2b5e513e66cb6+        .quad   0xd7a8beefb3448aa4+        .quad   0x7a2932856f5ea192++        // 2^76 * 2 * B++        .quad   0x0b39d761b02de888+        .quad   0x5f550e7ed2414e1f+        .quad   0xa6bfa45822e1a940+        .quad   0x050a2f7dfd447b99+        .quad   0xb89c444879639302+        .quad   0x4ae4f19350c67f2c+        .quad   0xf0b35da8c81af9c6+        .quad   0x39d0003546871017+        .quad   0x437c3b33a650db77+        .quad   0x6bafe81dbac52bb2+        .quad   0xfe99402d2db7d318+        .quad   0x2b5b7eec372ba6ce++        // 2^76 * 3 * B++        .quad   0xb3bc4bbd83f50eef+        .quad   0x508f0c998c927866+        .quad   0x43e76587c8b7e66e+        .quad   0x0f7655a3a47f98d9+        .quad   0xa694404d613ac8f4+        .quad   0x500c3c2bfa97e72c+        .quad   0x874104d21fcec210+        .quad   0x1b205fb38604a8ee+        .quad   0x55ecad37d24b133c+        .quad   0x441e147d6038c90b+        .quad   0x656683a1d62c6fee+        .quad   0x0157d5dc87e0ecae++        // 2^76 * 4 * B++        .quad   0xf2a7af510354c13d+        .quad   0xd7a0b145aa372b60+        .quad   0x2869b96a05a3d470+        .quad   0x6528e42d82460173+        .quad   0x95265514d71eb524+        .quad   0xe603d8815df14593+        .quad   0x147cdf410d4de6b7+        .quad   0x5293b1730437c850+        .quad   0x23d0e0814bccf226+        .quad   0x92c745cd8196fb93+        .quad   0x8b61796c59541e5b+        .quad   0x40a44df0c021f978++        // 2^76 * 5 * B++        .quad   0xdaa869894f20ea6a+        .quad   0xea14a3d14c620618+        .quad   0x6001fccb090bf8be+        .quad   0x35f4e822947e9cf0+        .quad   0x86c96e514bc5d095+        .quad   0xf20d4098fca6804a+        .quad   0x27363d89c826ea5d+        .quad   0x39ca36565719cacf+        .quad   0x97506f2f6f87b75c+        .quad   0xc624aea0034ae070+        .quad   0x1ec856e3aad34dd6+        .quad   0x055b0be0e440e58f++        // 2^76 * 6 * B++        .quad   0x6469a17d89735d12+        .quad   0xdb6f27d5e662b9f1+        .quad   0x9fcba3286a395681+        .quad   0x363b8004d269af25+        .quad   0x4d12a04b6ea33da2+        .quad   0x57cf4c15e36126dd+        .quad   0x90ec9675ee44d967+        .quad   0x64ca348d2a985aac+        .quad   0x99588e19e4c4912d+        .quad   0xefcc3b4e1ca5ce6b+        .quad   0x4522ea60fa5b98d5+        .quad   0x7064bbab1de4a819++        // 2^76 * 7 * B++        .quad   0xb919e1515a770641+        .quad   0xa9a2e2c74e7f8039+        .quad   0x7527250b3df23109+        .quad   0x756a7330ac27b78b+        .quad   0xa290c06142542129+        .quad   0xf2e2c2aebe8d5b90+        .quad   0xcf2458db76abfe1b+        .quad   0x02157ade83d626bf+        .quad   0x3e46972a1b9a038b+        .quad   0x2e4ee66a7ee03fb4+        .quad   0x81a248776edbb4ca+        .quad   0x1a944ee88ecd0563++        // 2^76 * 8 * B++        .quad   0xd5a91d1151039372+        .quad   0x2ed377b799ca26de+        .quad   0xa17202acfd366b6b+        .quad   0x0730291bd6901995+        .quad   0xbb40a859182362d6+        .quad   0xb99f55778a4d1abb+        .quad   0x8d18b427758559f6+        .quad   0x26c20fe74d26235a+        .quad   0x648d1d9fe9cc22f5+        .quad   0x66bc561928dd577c+        .quad   0x47d3ed21652439d1+        .quad   0x49d271acedaf8b49++        // 2^80 * 1 * B++        .quad   0x89f5058a382b33f3+        .quad   0x5ae2ba0bad48c0b4+        .quad   0x8f93b503a53db36e+        .quad   0x5aa3ed9d95a232e6+        .quad   0x2798aaf9b4b75601+        .quad   0x5eac72135c8dad72+        .quad   0xd2ceaa6161b7a023+        .quad   0x1bbfb284e98f7d4e+        .quad   0x656777e9c7d96561+        .quad   0xcb2b125472c78036+        .quad   0x65053299d9506eee+        .quad   0x4a07e14e5e8957cc++        // 2^80 * 2 * B++        .quad   0x4ee412cb980df999+        .quad   0xa315d76f3c6ec771+        .quad   0xbba5edde925c77fd+        .quad   0x3f0bac391d313402+        .quad   0x240b58cdc477a49b+        .quad   0xfd38dade6447f017+        .quad   0x19928d32a7c86aad+        .quad   0x50af7aed84afa081+        .quad   0x6e4fde0115f65be5+        .quad   0x29982621216109b2+        .quad   0x780205810badd6d9+        .quad   0x1921a316baebd006++        // 2^80 * 3 * B++        .quad   0x89422f7edfb870fc+        .quad   0x2c296beb4f76b3bd+        .quad   0x0738f1d436c24df7+        .quad   0x6458df41e273aeb0+        .quad   0xd75aad9ad9f3c18b+        .quad   0x566a0eef60b1c19c+        .quad   0x3e9a0bac255c0ed9+        .quad   0x7b049deca062c7f5+        .quad   0xdccbe37a35444483+        .quad   0x758879330fedbe93+        .quad   0x786004c312c5dd87+        .quad   0x6093dccbc2950e64++        // 2^80 * 4 * B++        .quad   0x1ff39a8585e0706d+        .quad   0x36d0a5d8b3e73933+        .quad   0x43b9f2e1718f453b+        .quad   0x57d1ea084827a97c+        .quad   0x6bdeeebe6084034b+        .quad   0x3199c2b6780fb854+        .quad   0x973376abb62d0695+        .quad   0x6e3180c98b647d90+        .quad   0xee7ab6e7a128b071+        .quad   0xa4c1596d93a88baa+        .quad   0xf7b4de82b2216130+        .quad   0x363e999ddd97bd18++        // 2^80 * 5 * B++        .quad   0x96a843c135ee1fc4+        .quad   0x976eb35508e4c8cf+        .quad   0xb42f6801b58cd330+        .quad   0x48ee9b78693a052b+        .quad   0x2f1848dce24baec6+        .quad   0x769b7255babcaf60+        .quad   0x90cb3c6e3cefe931+        .quad   0x231f979bc6f9b355+        .quad   0x5c31de4bcc2af3c6+        .quad   0xb04bb030fe208d1f+        .quad   0xb78d7009c14fb466+        .quad   0x079bfa9b08792413++        // 2^80 * 6 * B++        .quad   0xe3903a51da300df4+        .quad   0x843964233da95ab0+        .quad   0xed3cf12d0b356480+        .quad   0x038c77f684817194+        .quad   0xf3c9ed80a2d54245+        .quad   0x0aa08b7877f63952+        .quad   0xd76dac63d1085475+        .quad   0x1ef4fb159470636b+        .quad   0x854e5ee65b167bec+        .quad   0x59590a4296d0cdc2+        .quad   0x72b2df3498102199+        .quad   0x575ee92a4a0bff56++        // 2^80 * 7 * B++        .quad   0xd4c080908a182fcf+        .quad   0x30e170c299489dbd+        .quad   0x05babd5752f733de+        .quad   0x43d4e7112cd3fd00+        .quad   0x5d46bc450aa4d801+        .quad   0xc3af1227a533b9d8+        .quad   0x389e3b262b8906c2+        .quad   0x200a1e7e382f581b+        .quad   0x518db967eaf93ac5+        .quad   0x71bc989b056652c0+        .quad   0xfe2b85d9567197f5+        .quad   0x050eca52651e4e38++        // 2^80 * 8 * B++        .quad   0xc3431ade453f0c9c+        .quad   0xe9f5045eff703b9b+        .quad   0xfcd97ac9ed847b3d+        .quad   0x4b0ee6c21c58f4c6+        .quad   0x97ac397660e668ea+        .quad   0x9b19bbfe153ab497+        .quad   0x4cb179b534eca79f+        .quad   0x6151c09fa131ae57+        .quad   0x3af55c0dfdf05d96+        .quad   0xdd262ee02ab4ee7a+        .quad   0x11b2bb8712171709+        .quad   0x1fef24fa800f030b++        // 2^84 * 1 * B++        .quad   0xb496123a6b6c6609+        .quad   0xa750fe8580ab5938+        .quad   0xf471bf39b7c27a5f+        .quad   0x507903ce77ac193c+        .quad   0xff91a66a90166220+        .quad   0xf22552ae5bf1e009+        .quad   0x7dff85d87f90df7c+        .quad   0x4f620ffe0c736fb9+        .quad   0x62f90d65dfde3e34+        .quad   0xcf28c592b9fa5fad+        .quad   0x99c86ef9c6164510+        .quad   0x25d448044a256c84++        // 2^84 * 2 * B++        .quad   0xbd68230ec7e9b16f+        .quad   0x0eb1b9c1c1c5795d+        .quad   0x7943c8c495b6b1ff+        .quad   0x2f9faf620bbacf5e+        .quad   0x2c7c4415c9022b55+        .quad   0x56a0d241812eb1fe+        .quad   0xf02ea1c9d7b65e0d+        .quad   0x4180512fd5323b26+        .quad   0xa4ff3e698a48a5db+        .quad   0xba6a3806bd95403b+        .quad   0x9f7ce1af47d5b65d+        .quad   0x15e087e55939d2fb++        // 2^84 * 3 * B++        .quad   0x12207543745c1496+        .quad   0xdaff3cfdda38610c+        .quad   0xe4e797272c71c34f+        .quad   0x39c07b1934bdede9+        .quad   0x8894186efb963f38+        .quad   0x48a00e80dc639bd5+        .quad   0xa4e8092be96c1c99+        .quad   0x5a097d54ca573661+        .quad   0x2d45892b17c9e755+        .quad   0xd033fd7289308df8+        .quad   0x6c2fe9d9525b8bd9+        .quad   0x2edbecf1c11cc079++        // 2^84 * 4 * B++        .quad   0x1616a4e3c715a0d2+        .quad   0x53623cb0f8341d4d+        .quad   0x96ef5329c7e899cb+        .quad   0x3d4e8dbba668baa6+        .quad   0xee0f0fddd087a25f+        .quad   0x9c7531555c3e34ee+        .quad   0x660c572e8fab3ab5+        .quad   0x0854fc44544cd3b2+        .quad   0x61eba0c555edad19+        .quad   0x24b533fef0a83de6+        .quad   0x3b77042883baa5f8+        .quad   0x678f82b898a47e8d++        // 2^84 * 5 * B++        .quad   0xb1491d0bd6900c54+        .quad   0x3539722c9d132636+        .quad   0x4db928920b362bc9+        .quad   0x4d7cd1fea68b69df+        .quad   0x1e09d94057775696+        .quad   0xeed1265c3cd951db+        .quad   0xfa9dac2b20bce16f+        .quad   0x0f7f76e0e8d089f4+        .quad   0x36d9ebc5d485b00c+        .quad   0xa2596492e4adb365+        .quad   0xc1659480c2119ccd+        .quad   0x45306349186e0d5f++        // 2^84 * 6 * B++        .quad   0x94ddd0c1a6cdff1d+        .quad   0x55f6f115e84213ae+        .quad   0x6c935f85992fcf6a+        .quad   0x067ee0f54a37f16f+        .quad   0x96a414ec2b072491+        .quad   0x1bb2218127a7b65b+        .quad   0x6d2849596e8a4af0+        .quad   0x65f3b08ccd27765f+        .quad   0xecb29fff199801f7+        .quad   0x9d361d1fa2a0f72f+        .quad   0x25f11d2375fd2f49+        .quad   0x124cefe80fe10fe2++        // 2^84 * 7 * B++        .quad   0x4c126cf9d18df255+        .quad   0xc1d471e9147a63b6+        .quad   0x2c6d3c73f3c93b5f+        .quad   0x6be3a6a2e3ff86a2+        .quad   0x1518e85b31b16489+        .quad   0x8faadcb7db710bfb+        .quad   0x39b0bdf4a14ae239+        .quad   0x05f4cbea503d20c1+        .quad   0xce040e9ec04145bc+        .quad   0xc71ff4e208f6834c+        .quad   0xbd546e8dab8847a3+        .quad   0x64666aa0a4d2aba5++        // 2^84 * 8 * B++        .quad   0x6841435a7c06d912+        .quad   0xca123c21bb3f830b+        .quad   0xd4b37b27b1cbe278+        .quad   0x1d753b84c76f5046+        .quad   0xb0c53bf73337e94c+        .quad   0x7cb5697e11e14f15+        .quad   0x4b84abac1930c750+        .quad   0x28dd4abfe0640468+        .quad   0x7dc0b64c44cb9f44+        .quad   0x18a3e1ace3925dbf+        .quad   0x7a3034862d0457c4+        .quad   0x4c498bf78a0c892e++        // 2^88 * 1 * B++        .quad   0x37d653fb1aa73196+        .quad   0x0f9495303fd76418+        .quad   0xad200b09fb3a17b2+        .quad   0x544d49292fc8613e+        .quad   0x22d2aff530976b86+        .quad   0x8d90b806c2d24604+        .quad   0xdca1896c4de5bae5+        .quad   0x28005fe6c8340c17+        .quad   0x6aefba9f34528688+        .quad   0x5c1bff9425107da1+        .quad   0xf75bbbcd66d94b36+        .quad   0x72e472930f316dfa++        // 2^88 * 2 * B++        .quad   0x2695208c9781084f+        .quad   0xb1502a0b23450ee1+        .quad   0xfd9daea603efde02+        .quad   0x5a9d2e8c2733a34c+        .quad   0x07f3f635d32a7627+        .quad   0x7aaa4d865f6566f0+        .quad   0x3c85e79728d04450+        .quad   0x1fee7f000fe06438+        .quad   0x765305da03dbf7e5+        .quad   0xa4daf2491434cdbd+        .quad   0x7b4ad5cdd24a88ec+        .quad   0x00f94051ee040543++        // 2^88 * 3 * B++        .quad   0x8d356b23c3d330b2+        .quad   0xf21c8b9bb0471b06+        .quad   0xb36c316c6e42b83c+        .quad   0x07d79c7e8beab10d+        .quad   0xd7ef93bb07af9753+        .quad   0x583ed0cf3db766a7+        .quad   0xce6998bf6e0b1ec5+        .quad   0x47b7ffd25dd40452+        .quad   0x87fbfb9cbc08dd12+        .quad   0x8a066b3ae1eec29b+        .quad   0x0d57242bdb1fc1bf+        .quad   0x1c3520a35ea64bb6++        // 2^88 * 4 * B++        .quad   0x80d253a6bccba34a+        .quad   0x3e61c3a13838219b+        .quad   0x90c3b6019882e396+        .quad   0x1c3d05775d0ee66f+        .quad   0xcda86f40216bc059+        .quad   0x1fbb231d12bcd87e+        .quad   0xb4956a9e17c70990+        .quad   0x38750c3b66d12e55+        .quad   0x692ef1409422e51a+        .quad   0xcbc0c73c2b5df671+        .quad   0x21014fe7744ce029+        .quad   0x0621e2c7d330487c++        // 2^88 * 5 * B++        .quad   0xaf9860cc8259838d+        .quad   0x90ea48c1c69f9adc+        .quad   0x6526483765581e30+        .quad   0x0007d6097bd3a5bc+        .quad   0xb7ae1796b0dbf0f3+        .quad   0x54dfafb9e17ce196+        .quad   0x25923071e9aaa3b4+        .quad   0x5d8e589ca1002e9d+        .quad   0xc0bf1d950842a94b+        .quad   0xb2d3c363588f2e3e+        .quad   0x0a961438bb51e2ef+        .quad   0x1583d7783c1cbf86++        // 2^88 * 6 * B++        .quad   0xeceea2ef5da27ae1+        .quad   0x597c3a1455670174+        .quad   0xc9a62a126609167a+        .quad   0x252a5f2e81ed8f70+        .quad   0x90034704cc9d28c7+        .quad   0x1d1b679ef72cc58f+        .quad   0x16e12b5fbe5b8726+        .quad   0x4958064e83c5580a+        .quad   0x0d2894265066e80d+        .quad   0xfcc3f785307c8c6b+        .quad   0x1b53da780c1112fd+        .quad   0x079c170bd843b388++        // 2^88 * 7 * B++        .quad   0x0506ece464fa6fff+        .quad   0xbee3431e6205e523+        .quad   0x3579422451b8ea42+        .quad   0x6dec05e34ac9fb00+        .quad   0xcdd6cd50c0d5d056+        .quad   0x9af7686dbb03573b+        .quad   0x3ca6723ff3c3ef48+        .quad   0x6768c0d7317b8acc+        .quad   0x94b625e5f155c1b3+        .quad   0x417bf3a7997b7b91+        .quad   0xc22cbddc6d6b2600+        .quad   0x51445e14ddcd52f4++        // 2^88 * 8 * B++        .quad   0x57502b4b3b144951+        .quad   0x8e67ff6b444bbcb3+        .quad   0xb8bd6927166385db+        .quad   0x13186f31e39295c8+        .quad   0x893147ab2bbea455+        .quad   0x8c53a24f92079129+        .quad   0x4b49f948be30f7a7+        .quad   0x12e990086e4fd43d+        .quad   0xf10c96b37fdfbb2e+        .quad   0x9f9a935e121ceaf9+        .quad   0xdf1136c43a5b983f+        .quad   0x77b2e3f05d3e99af++        // 2^92 * 1 * B++        .quad   0xfd0d75879cf12657+        .quad   0xe82fef94e53a0e29+        .quad   0xcc34a7f05bbb4be7+        .quad   0x0b251172a50c38a2+        .quad   0x9532f48fcc5cd29b+        .quad   0x2ba851bea3ce3671+        .quad   0x32dacaa051122941+        .quad   0x478d99d9350004f2+        .quad   0x1d5ad94890bb02c0+        .quad   0x50e208b10ec25115+        .quad   0xa26a22894ef21702+        .quad   0x4dc923343b524805++        // 2^92 * 2 * B++        .quad   0xe3828c400f8086b6+        .quad   0x3f77e6f7979f0dc8+        .quad   0x7ef6de304df42cb4+        .quad   0x5265797cb6abd784+        .quad   0x3ad3e3ebf36c4975+        .quad   0xd75d25a537862125+        .quad   0xe873943da025a516+        .quad   0x6bbc7cb4c411c847+        .quad   0x3c6f9cd1d4a50d56+        .quad   0xb6244077c6feab7e+        .quad   0x6ff9bf483580972e+        .quad   0x00375883b332acfb++        // 2^92 * 3 * B++        .quad   0x0001b2cd28cb0940+        .quad   0x63fb51a06f1c24c9+        .quad   0xb5ad8691dcd5ca31+        .quad   0x67238dbd8c450660+        .quad   0xc98bec856c75c99c+        .quad   0xe44184c000e33cf4+        .quad   0x0a676b9bba907634+        .quad   0x669e2cb571f379d7+        .quad   0xcb116b73a49bd308+        .quad   0x025aad6b2392729e+        .quad   0xb4793efa3f55d9b1+        .quad   0x72a1056140678bb9++        // 2^92 * 4 * B++        .quad   0xa2b6812b1cc9249d+        .quad   0x62866eee21211f58+        .quad   0x2cb5c5b85df10ece+        .quad   0x03a6b259e263ae00+        .quad   0x0d8d2909e2e505b6+        .quad   0x98ca78abc0291230+        .quad   0x77ef5569a9b12327+        .quad   0x7c77897b81439b47+        .quad   0xf1c1b5e2de331cb5+        .quad   0x5a9f5d8e15fca420+        .quad   0x9fa438f17bd932b1+        .quad   0x2a381bf01c6146e7++        // 2^92 * 5 * B++        .quad   0xac9b9879cfc811c1+        .quad   0x8b7d29813756e567+        .quad   0x50da4e607c70edfc+        .quad   0x5dbca62f884400b6+        .quad   0xf7c0be32b534166f+        .quad   0x27e6ca6419cf70d4+        .quad   0x934df7d7a957a759+        .quad   0x5701461dabdec2aa+        .quad   0x2c6747402c915c25+        .quad   0x1bdcd1a80b0d340a+        .quad   0x5e5601bd07b43f5f+        .quad   0x2555b4e05539a242++        // 2^92 * 6 * B++        .quad   0x6fc09f5266ddd216+        .quad   0xdce560a7c8e37048+        .quad   0xec65939da2df62fd+        .quad   0x7a869ae7e52ed192+        .quad   0x78409b1d87e463d4+        .quad   0xad4da95acdfb639d+        .quad   0xec28773755259b9c+        .quad   0x69c806e9c31230ab+        .quad   0x7b48f57414bb3f22+        .quad   0x68c7cee4aedccc88+        .quad   0xed2f936179ed80be+        .quad   0x25d70b885f77bc4b++        // 2^92 * 7 * B++        .quad   0x4151c3d9762bf4de+        .quad   0x083f435f2745d82b+        .quad   0x29775a2e0d23ddd5+        .quad   0x138e3a6269a5db24+        .quad   0x98459d29bb1ae4d4+        .quad   0x56b9c4c739f954ec+        .quad   0x832743f6c29b4b3e+        .quad   0x21ea8e2798b6878a+        .quad   0x87bef4b46a5a7b9c+        .quad   0xd2299d1b5fc1d062+        .quad   0x82409818dd321648+        .quad   0x5c5abeb1e5a2e03d++        // 2^92 * 8 * B++        .quad   0x14722af4b73c2ddb+        .quad   0xbc470c5f5a05060d+        .quad   0x00943eac2581b02e+        .quad   0x0e434b3b1f499c8f+        .quad   0x02cde6de1306a233+        .quad   0x7b5a52a2116f8ec7+        .quad   0xe1c681f4c1163b5b+        .quad   0x241d350660d32643+        .quad   0x6be4404d0ebc52c7+        .quad   0xae46233bb1a791f5+        .quad   0x2aec170ed25db42b+        .quad   0x1d8dfd966645d694++        // 2^96 * 1 * B++        .quad   0x296fa9c59c2ec4de+        .quad   0xbc8b61bf4f84f3cb+        .quad   0x1c7706d917a8f908+        .quad   0x63b795fc7ad3255d+        .quad   0xd598639c12ddb0a4+        .quad   0xa5d19f30c024866b+        .quad   0xd17c2f0358fce460+        .quad   0x07a195152e095e8a+        .quad   0xa8368f02389e5fc8+        .quad   0x90433b02cf8de43b+        .quad   0xafa1fd5dc5412643+        .quad   0x3e8fe83d032f0137++        // 2^96 * 2 * B++        .quad   0x2f8b15b90570a294+        .quad   0x94f2427067084549+        .quad   0xde1c5ae161bbfd84+        .quad   0x75ba3b797fac4007+        .quad   0x08704c8de8efd13c+        .quad   0xdfc51a8e33e03731+        .quad   0xa59d5da51260cde3+        .quad   0x22d60899a6258c86+        .quad   0x6239dbc070cdd196+        .quad   0x60fe8a8b6c7d8a9a+        .quad   0xb38847bceb401260+        .quad   0x0904d07b87779e5e++        // 2^96 * 3 * B++        .quad   0xb4ce1fd4ddba919c+        .quad   0xcf31db3ec74c8daa+        .quad   0x2c63cc63ad86cc51+        .quad   0x43e2143fbc1dde07+        .quad   0xf4322d6648f940b9+        .quad   0x06952f0cbd2d0c39+        .quad   0x167697ada081f931+        .quad   0x6240aacebaf72a6c+        .quad   0xf834749c5ba295a0+        .quad   0xd6947c5bca37d25a+        .quad   0x66f13ba7e7c9316a+        .quad   0x56bdaf238db40cac++        // 2^96 * 4 * B++        .quad   0x362ab9e3f53533eb+        .quad   0x338568d56eb93d40+        .quad   0x9e0e14521d5a5572+        .quad   0x1d24a86d83741318+        .quad   0x1310d36cc19d3bb2+        .quad   0x062a6bb7622386b9+        .quad   0x7c9b8591d7a14f5c+        .quad   0x03aa31507e1e5754+        .quad   0xf4ec7648ffd4ce1f+        .quad   0xe045eaf054ac8c1c+        .quad   0x88d225821d09357c+        .quad   0x43b261dc9aeb4859++        // 2^96 * 5 * B++        .quad   0xe55b1e1988bb79bb+        .quad   0xa09ed07dc17a359d+        .quad   0xb02c2ee2603dea33+        .quad   0x326055cf5b276bc2+        .quad   0x19513d8b6c951364+        .quad   0x94fe7126000bf47b+        .quad   0x028d10ddd54f9567+        .quad   0x02b4d5e242940964+        .quad   0xb4a155cb28d18df2+        .quad   0xeacc4646186ce508+        .quad   0xc49cf4936c824389+        .quad   0x27a6c809ae5d3410++        // 2^96 * 6 * B++        .quad   0x8ba6ebcd1f0db188+        .quad   0x37d3d73a675a5be8+        .quad   0xf22edfa315f5585a+        .quad   0x2cb67174ff60a17e+        .quad   0xcd2c270ac43d6954+        .quad   0xdd4a3e576a66cab2+        .quad   0x79fa592469d7036c+        .quad   0x221503603d8c2599+        .quad   0x59eecdf9390be1d0+        .quad   0xa9422044728ce3f1+        .quad   0x82891c667a94f0f4+        .quad   0x7b1df4b73890f436++        // 2^96 * 7 * B++        .quad   0xe492f2e0b3b2a224+        .quad   0x7c6c9e062b551160+        .quad   0x15eb8fe20d7f7b0e+        .quad   0x61fcef2658fc5992+        .quad   0x5f2e221807f8f58c+        .quad   0xe3555c9fd49409d4+        .quad   0xb2aaa88d1fb6a630+        .quad   0x68698245d352e03d+        .quad   0xdbb15d852a18187a+        .quad   0xf3e4aad386ddacd7+        .quad   0x44bae2810ff6c482+        .quad   0x46cf4c473daf01cf++        // 2^96 * 8 * B++        .quad   0x426525ed9ec4e5f9+        .quad   0x0e5eda0116903303+        .quad   0x72b1a7f2cbe5cadc+        .quad   0x29387bcd14eb5f40+        .quad   0x213c6ea7f1498140+        .quad   0x7c1e7ef8392b4854+        .quad   0x2488c38c5629ceba+        .quad   0x1065aae50d8cc5bb+        .quad   0x1c2c4525df200d57+        .quad   0x5c3b2dd6bfca674a+        .quad   0x0a07e7b1e1834030+        .quad   0x69a198e64f1ce716++        // 2^100 * 1 * B++        .quad   0x7afcd613efa9d697+        .quad   0x0cc45aa41c067959+        .quad   0xa56fe104c1fada96+        .quad   0x3a73b70472e40365+        .quad   0x7b26e56b9e2d4734+        .quad   0xc4c7132b81c61675+        .quad   0xef5c9525ec9cde7f+        .quad   0x39c80b16e71743ad+        .quad   0x0f196e0d1b826c68+        .quad   0xf71ff0e24960e3db+        .quad   0x6113167023b7436c+        .quad   0x0cf0ea5877da7282++        // 2^100 * 2 * B++        .quad   0x196c80a4ddd4ccbd+        .quad   0x22e6f55d95f2dd9d+        .quad   0xc75e33c740d6c71b+        .quad   0x7bb51279cb3c042f+        .quad   0xe332ced43ba6945a+        .quad   0xde0b1361e881c05d+        .quad   0x1ad40f095e67ed3b+        .quad   0x5da8acdab8c63d5d+        .quad   0xc4b6664a3a70159f+        .quad   0x76194f0f0a904e14+        .quad   0xa5614c39a4096c13+        .quad   0x6cd0ff50979feced++        // 2^100 * 3 * B++        .quad   0xc0e067e78f4428ac+        .quad   0x14835ab0a61135e3+        .quad   0xf21d14f338062935+        .quad   0x6390a4c8df04849c+        .quad   0x7fecfabdb04ba18e+        .quad   0xd0fc7bfc3bddbcf7+        .quad   0xa41d486e057a131c+        .quad   0x641a4391f2223a61+        .quad   0xc5c6b95aa606a8db+        .quad   0x914b7f9eb06825f1+        .quad   0x2a731f6b44fc9eff+        .quad   0x30ddf38562705cfc++        // 2^100 * 4 * B++        .quad   0x4e3dcbdad1bff7f9+        .quad   0xc9118e8220645717+        .quad   0xbacccebc0f189d56+        .quad   0x1b4822e9d4467668+        .quad   0x33bef2bd68bcd52c+        .quad   0xc649dbb069482ef2+        .quad   0xb5b6ee0c41cb1aee+        .quad   0x5c294d270212a7e5+        .quad   0xab360a7f25563781+        .quad   0x2512228a480f7958+        .quad   0xc75d05276114b4e3+        .quad   0x222d9625d976fe2a++        // 2^100 * 5 * B++        .quad   0x1c717f85b372ace1+        .quad   0x81930e694638bf18+        .quad   0x239cad056bc08b58+        .quad   0x0b34271c87f8fff4+        .quad   0x0f94be7e0a344f85+        .quad   0xeb2faa8c87f22c38+        .quad   0x9ce1e75e4ee16f0f+        .quad   0x43e64e5418a08dea+        .quad   0x8155e2521a35ce63+        .quad   0xbe100d4df912028e+        .quad   0xbff80bf8a57ddcec+        .quad   0x57342dc96d6bc6e4++        // 2^100 * 6 * B++        .quad   0xefeef065c8ce5998+        .quad   0xbf029510b5cbeaa2+        .quad   0x8c64a10620b7c458+        .quad   0x35134fb231c24855+        .quad   0xf3c3bcb71e707bf6+        .quad   0x351d9b8c7291a762+        .quad   0x00502e6edad69a33+        .quad   0x522f521f1ec8807f+        .quad   0x272c1f46f9a3902b+        .quad   0xc91ba3b799657bcc+        .quad   0xae614b304f8a1c0e+        .quad   0x7afcaad70b99017b++        // 2^100 * 7 * B++        .quad   0xc25ded54a4b8be41+        .quad   0x902d13e11bb0e2dd+        .quad   0x41f43233cde82ab2+        .quad   0x1085faa5c3aae7cb+        .quad   0xa88141ecef842b6b+        .quad   0x55e7b14797abe6c5+        .quad   0x8c748f9703784ffe+        .quad   0x5b50a1f7afcd00b7+        .quad   0x9b840f66f1361315+        .quad   0x18462242701003e9+        .quad   0x65ed45fae4a25080+        .quad   0x0a2862393fda7320++        // 2^100 * 8 * B++        .quad   0x46ab13c8347cbc9d+        .quad   0x3849e8d499c12383+        .quad   0x4cea314087d64ac9+        .quad   0x1f354134b1a29ee7+        .quad   0x960e737b6ecb9d17+        .quad   0xfaf24948d67ceae1+        .quad   0x37e7a9b4d55e1b89+        .quad   0x5cb7173cb46c59eb+        .quad   0x4a89e68b82b7abf0+        .quad   0xf41cd9279ba6b7b9+        .quad   0x16e6c210e18d876f+        .quad   0x7cacdb0f7f1b09c6++        // 2^104 * 1 * B++        .quad   0x9062b2e0d91a78bc+        .quad   0x47c9889cc8509667+        .quad   0x9df54a66405070b8+        .quad   0x7369e6a92493a1bf+        .quad   0xe1014434dcc5caed+        .quad   0x47ed5d963c84fb33+        .quad   0x70019576ed86a0e7+        .quad   0x25b2697bd267f9e4+        .quad   0x9d673ffb13986864+        .quad   0x3ca5fbd9415dc7b8+        .quad   0xe04ecc3bdf273b5e+        .quad   0x1420683db54e4cd2++        // 2^104 * 2 * B++        .quad   0xb478bd1e249dd197+        .quad   0x620c35005e58c102+        .quad   0xfb02d32fccbaac5c+        .quad   0x60b63bebf508a72d+        .quad   0x34eebb6fc1cc5ad0+        .quad   0x6a1b0ce99646ac8b+        .quad   0xd3b0da49a66bde53+        .quad   0x31e83b4161d081c1+        .quad   0x97e8c7129e062b4f+        .quad   0x49e48f4f29320ad8+        .quad   0x5bece14b6f18683f+        .quad   0x55cf1eb62d550317++        // 2^104 * 3 * B++        .quad   0x5879101065c23d58+        .quad   0x8b9d086d5094819c+        .quad   0xe2402fa912c55fa7+        .quad   0x669a6564570891d4+        .quad   0x3076b5e37df58c52+        .quad   0xd73ab9dde799cc36+        .quad   0xbd831ce34913ee20+        .quad   0x1a56fbaa62ba0133+        .quad   0x943e6b505c9dc9ec+        .quad   0x302557bba77c371a+        .quad   0x9873ae5641347651+        .quad   0x13c4836799c58a5c++        // 2^104 * 4 * B++        .quad   0x423a5d465ab3e1b9+        .quad   0xfc13c187c7f13f61+        .quad   0x19f83664ecb5b9b6+        .quad   0x66f80c93a637b607+        .quad   0xc4dcfb6a5d8bd080+        .quad   0xdeebc4ec571a4842+        .quad   0xd4b2e883b8e55365+        .quad   0x50bdc87dc8e5b827+        .quad   0x606d37836edfe111+        .quad   0x32353e15f011abd9+        .quad   0x64b03ac325b73b96+        .quad   0x1dd56444725fd5ae++        // 2^104 * 5 * B++        .quad   0x8fa47ff83362127d+        .quad   0xbc9f6ac471cd7c15+        .quad   0x6e71454349220c8b+        .quad   0x0e645912219f732e+        .quad   0xc297e60008bac89a+        .quad   0x7d4cea11eae1c3e0+        .quad   0xf3e38be19fe7977c+        .quad   0x3a3a450f63a305cd+        .quad   0x078f2f31d8394627+        .quad   0x389d3183de94a510+        .quad   0xd1e36c6d17996f80+        .quad   0x318c8d9393a9a87b++        // 2^104 * 6 * B++        .quad   0xf2745d032afffe19+        .quad   0x0c9f3c497f24db66+        .quad   0xbc98d3e3ba8598ef+        .quad   0x224c7c679a1d5314+        .quad   0x5d669e29ab1dd398+        .quad   0xfc921658342d9e3b+        .quad   0x55851dfdf35973cd+        .quad   0x509a41c325950af6+        .quad   0xbdc06edca6f925e9+        .quad   0x793ef3f4641b1f33+        .quad   0x82ec12809d833e89+        .quad   0x05bff02328a11389++        // 2^104 * 7 * B++        .quad   0x3632137023cae00b+        .quad   0x544acf0ad1accf59+        .quad   0x96741049d21a1c88+        .quad   0x780b8cc3fa2a44a7+        .quad   0x6881a0dd0dc512e4+        .quad   0x4fe70dc844a5fafe+        .quad   0x1f748e6b8f4a5240+        .quad   0x576277cdee01a3ea+        .quad   0x1ef38abc234f305f+        .quad   0x9a577fbd1405de08+        .quad   0x5e82a51434e62a0d+        .quad   0x5ff418726271b7a1++        // 2^104 * 8 * B++        .quad   0x398e080c1789db9d+        .quad   0xa7602025f3e778f5+        .quad   0xfa98894c06bd035d+        .quad   0x106a03dc25a966be+        .quad   0xe5db47e813b69540+        .quad   0xf35d2a3b432610e1+        .quad   0xac1f26e938781276+        .quad   0x29d4db8ca0a0cb69+        .quad   0xd9ad0aaf333353d0+        .quad   0x38669da5acd309e5+        .quad   0x3c57658ac888f7f0+        .quad   0x4ab38a51052cbefa++        // 2^108 * 1 * B++        .quad   0xdfdacbee4324c0e9+        .quad   0x054442883f955bb7+        .quad   0xdef7aaa8ea31609f+        .quad   0x68aee70642287cff+        .quad   0xf68fe2e8809de054+        .quad   0xe3bc096a9c82bad1+        .quad   0x076353d40aadbf45+        .quad   0x7b9b1fb5dea1959e+        .quad   0xf01cc8f17471cc0c+        .quad   0x95242e37579082bb+        .quad   0x27776093d3e46b5f+        .quad   0x2d13d55a28bd85fb++        // 2^108 * 2 * B++        .quad   0xfac5d2065b35b8da+        .quad   0xa8da8a9a85624bb7+        .quad   0xccd2ca913d21cd0f+        .quad   0x6b8341ee8bf90d58+        .quad   0xbf019cce7aee7a52+        .quad   0xa8ded2b6e454ead3+        .quad   0x3c619f0b87a8bb19+        .quad   0x3619b5d7560916d8+        .quad   0x3579f26b0282c4b2+        .quad   0x64d592f24fafefae+        .quad   0xb7cded7b28c8c7c0+        .quad   0x6a927b6b7173a8d7++        // 2^108 * 3 * B++        .quad   0x1f6db24f986e4656+        .quad   0x1021c02ed1e9105b+        .quad   0xf8ff3fff2cc0a375+        .quad   0x1d2a6bf8c6c82592+        .quad   0x8d7040863ece88eb+        .quad   0xf0e307a980eec08c+        .quad   0xac2250610d788fda+        .quad   0x056d92a43a0d478d+        .quad   0x1b05a196fc3da5a1+        .quad   0x77d7a8c243b59ed0+        .quad   0x06da3d6297d17918+        .quad   0x66fbb494f12353f7++        // 2^108 * 4 * B++        .quad   0x751a50b9d85c0fb8+        .quad   0xd1afdc258bcf097b+        .quad   0x2f16a6a38309a969+        .quad   0x14ddff9ee5b00659+        .quad   0xd6d70996f12309d6+        .quad   0xdbfb2385e9c3d539+        .quad   0x46d602b0f7552411+        .quad   0x270a0b0557843e0c+        .quad   0x61ff0640a7862bcc+        .quad   0x81cac09a5f11abfe+        .quad   0x9047830455d12abb+        .quad   0x19a4bde1945ae873++        // 2^108 * 5 * B++        .quad   0x9b9f26f520a6200a+        .quad   0x64804443cf13eaf8+        .quad   0x8a63673f8631edd3+        .quad   0x72bbbce11ed39dc1+        .quad   0x40c709dec076c49f+        .quad   0x657bfaf27f3e53f6+        .quad   0x40662331eca042c4+        .quad   0x14b375487eb4df04+        .quad   0xae853c94ab66dc47+        .quad   0xeb62343edf762d6e+        .quad   0xf08e0e186fb2f7d1+        .quad   0x4f0b1c02700ab37a++        // 2^108 * 6 * B++        .quad   0xe1706787d81951fa+        .quad   0xa10a2c8eb290c77b+        .quad   0xe7382fa03ed66773+        .quad   0x0a4d84710bcc4b54+        .quad   0x79fd21ccc1b2e23f+        .quad   0x4ae7c281453df52a+        .quad   0xc8172ec9d151486b+        .quad   0x68abe9443e0a7534+        .quad   0xda12c6c407831dcb+        .quad   0x0da230d74d5c510d+        .quad   0x4ab1531e6bd404e1+        .quad   0x4106b166bcf440ef++        // 2^108 * 7 * B++        .quad   0x02e57a421cd23668+        .quad   0x4ad9fb5d0eaef6fd+        .quad   0x954e6727b1244480+        .quad   0x7f792f9d2699f331+        .quad   0xa485ccd539e4ecf2+        .quad   0x5aa3f3ad0555bab5+        .quad   0x145e3439937df82d+        .quad   0x1238b51e1214283f+        .quad   0x0b886b925fd4d924+        .quad   0x60906f7a3626a80d+        .quad   0xecd367b4b98abd12+        .quad   0x2876beb1def344cf++        // 2^108 * 8 * B++        .quad   0xdc84e93563144691+        .quad   0x632fe8a0d61f23f4+        .quad   0x4caa800612a9a8d5+        .quad   0x48f9dbfa0e9918d3+        .quad   0xd594b3333a8a85f8+        .quad   0x4ea37689e78d7d58+        .quad   0x73bf9f455e8e351f+        .quad   0x5507d7d2bc41ebb4+        .quad   0x1ceb2903299572fc+        .quad   0x7c8ccaa29502d0ee+        .quad   0x91bfa43411cce67b+        .quad   0x5784481964a831e7++        // 2^112 * 1 * B++        .quad   0xda7c2b256768d593+        .quad   0x98c1c0574422ca13+        .quad   0xf1a80bd5ca0ace1d+        .quad   0x29cdd1adc088a690+        .quad   0xd6cfd1ef5fddc09c+        .quad   0xe82b3efdf7575dce+        .quad   0x25d56b5d201634c2+        .quad   0x3041c6bb04ed2b9b+        .quad   0x0ff2f2f9d956e148+        .quad   0xade797759f356b2e+        .quad   0x1a4698bb5f6c025c+        .quad   0x104bbd6814049a7b++        // 2^112 * 2 * B++        .quad   0x51f0fd3168f1ed67+        .quad   0x2c811dcdd86f3bc2+        .quad   0x44dc5c4304d2f2de+        .quad   0x5be8cc57092a7149+        .quad   0xa95d9a5fd67ff163+        .quad   0xe92be69d4cc75681+        .quad   0xb7f8024cde20f257+        .quad   0x204f2a20fb072df5+        .quad   0xc8143b3d30ebb079+        .quad   0x7589155abd652e30+        .quad   0x653c3c318f6d5c31+        .quad   0x2570fb17c279161f++        // 2^112 * 3 * B++        .quad   0x3efa367f2cb61575+        .quad   0xf5f96f761cd6026c+        .quad   0xe8c7142a65b52562+        .quad   0x3dcb65ea53030acd+        .quad   0x192ea9550bb8245a+        .quad   0xc8e6fba88f9050d1+        .quad   0x7986ea2d88a4c935+        .quad   0x241c5f91de018668+        .quad   0x28d8172940de6caa+        .quad   0x8fbf2cf022d9733a+        .quad   0x16d7fcdd235b01d1+        .quad   0x08420edd5fcdf0e5++        // 2^112 * 4 * B++        .quad   0xcdff20ab8362fa4a+        .quad   0x57e118d4e21a3e6e+        .quad   0xe3179617fc39e62b+        .quad   0x0d9a53efbc1769fd+        .quad   0x0358c34e04f410ce+        .quad   0xb6135b5a276e0685+        .quad   0x5d9670c7ebb91521+        .quad   0x04d654f321db889c+        .quad   0x5e7dc116ddbdb5d5+        .quad   0x2954deb68da5dd2d+        .quad   0x1cb608173334a292+        .quad   0x4a7a4f2618991ad7++        // 2^112 * 5 * B++        .quad   0xf4a718025fb15f95+        .quad   0x3df65f346b5c1b8f+        .quad   0xcdfcf08500e01112+        .quad   0x11b50c4cddd31848+        .quad   0x24c3b291af372a4b+        .quad   0x93da8270718147f2+        .quad   0xdd84856486899ef2+        .quad   0x4a96314223e0ee33+        .quad   0xa6e8274408a4ffd6+        .quad   0x738e177e9c1576d9+        .quad   0x773348b63d02b3f2+        .quad   0x4f4bce4dce6bcc51++        // 2^112 * 6 * B++        .quad   0xa71fce5ae2242584+        .quad   0x26ea725692f58a9e+        .quad   0xd21a09d71cea3cf4+        .quad   0x73fcdd14b71c01e6+        .quad   0x30e2616ec49d0b6f+        .quad   0xe456718fcaec2317+        .quad   0x48eb409bf26b4fa6+        .quad   0x3042cee561595f37+        .quad   0x427e7079449bac41+        .quad   0x855ae36dbce2310a+        .quad   0x4cae76215f841a7c+        .quad   0x389e740c9a9ce1d6++        // 2^112 * 7 * B++        .quad   0x64fcb3ae34dcb9ce+        .quad   0x97500323e348d0ad+        .quad   0x45b3f07d62c6381b+        .quad   0x61545379465a6788+        .quad   0xc9bd78f6570eac28+        .quad   0xe55b0b3227919ce1+        .quad   0x65fc3eaba19b91ed+        .quad   0x25c425e5d6263690+        .quad   0x3f3e06a6f1d7de6e+        .quad   0x3ef976278e062308+        .quad   0x8c14f6264e8a6c77+        .quad   0x6539a08915484759++        // 2^112 * 8 * B++        .quad   0xe9d21f74c3d2f773+        .quad   0xc150544125c46845+        .quad   0x624e5ce8f9b99e33+        .quad   0x11c5e4aac5cd186c+        .quad   0xddc4dbd414bb4a19+        .quad   0x19b2bc3c98424f8e+        .quad   0x48a89fd736ca7169+        .quad   0x0f65320ef019bd90+        .quad   0xd486d1b1cafde0c6+        .quad   0x4f3fe6e3163b5181+        .quad   0x59a8af0dfaf2939a+        .quad   0x4cabc7bdec33072a++        // 2^116 * 1 * B++        .quad   0x16faa8fb532f7428+        .quad   0xdbd42ea046a4e272+        .quad   0x5337653b8b9ea480+        .quad   0x4065947223973f03+        .quad   0xf7c0a19c1a54a044+        .quad   0x4a1c5e2477bd9fbb+        .quad   0xa6e3ca115af22972+        .quad   0x1819bb953f2e9e0d+        .quad   0x498fbb795e042e84+        .quad   0x7d0dd89a7698b714+        .quad   0x8bfb0ba427fe6295+        .quad   0x36ba82e721200524++        // 2^116 * 2 * B++        .quad   0xd60ecbb74245ec41+        .quad   0xfd9be89e34348716+        .quad   0xc9240afee42284de+        .quad   0x4472f648d0531db4+        .quad   0xc8d69d0a57274ed5+        .quad   0x45ba803260804b17+        .quad   0xdf3cda102255dfac+        .quad   0x77d221232709b339+        .quad   0x498a6d7064ad94d8+        .quad   0xa5b5c8fd9af62263+        .quad   0x8ca8ed0545c141f4+        .quad   0x2c63bec3662d358c++        // 2^116 * 3 * B++        .quad   0x7fe60d8bea787955+        .quad   0xb9dc117eb5f401b7+        .quad   0x91c7c09a19355cce+        .quad   0x22692ef59442bedf+        .quad   0x9a518b3a8586f8bf+        .quad   0x9ee71af6cbb196f0+        .quad   0xaa0625e6a2385cf2+        .quad   0x1deb2176ddd7c8d1+        .quad   0x8563d19a2066cf6c+        .quad   0x401bfd8c4dcc7cd7+        .quad   0xd976a6becd0d8f62+        .quad   0x67cfd773a278b05e++        // 2^116 * 4 * B++        .quad   0x8dec31faef3ee475+        .quad   0x99dbff8a9e22fd92+        .quad   0x512d11594e26cab1+        .quad   0x0cde561eec4310b9+        .quad   0x2d5fa9855a4e586a+        .quad   0x65f8f7a449beab7e+        .quad   0xaa074dddf21d33d3+        .quad   0x185cba721bcb9dee+        .quad   0x93869da3f4e3cb41+        .quad   0xbf0392f540f7977e+        .quad   0x026204fcd0463b83+        .quad   0x3ec91a769eec6eed++        // 2^116 * 5 * B++        .quad   0x1e9df75bf78166ad+        .quad   0x4dfda838eb0cd7af+        .quad   0xba002ed8c1eaf988+        .quad   0x13fedb3e11f33cfc+        .quad   0x0fad2fb7b0a3402f+        .quad   0x46615ecbfb69f4a8+        .quad   0xf745bcc8c5f8eaa6+        .quad   0x7a5fa8794a94e896+        .quad   0x52958faa13cd67a1+        .quad   0x965ee0818bdbb517+        .quad   0x16e58daa2e8845b3+        .quad   0x357d397d5499da8f++        // 2^116 * 6 * B++        .quad   0x1ebfa05fb0bace6c+        .quad   0xc934620c1caf9a1e+        .quad   0xcc771cc41d82b61a+        .quad   0x2d94a16aa5f74fec+        .quad   0x481dacb4194bfbf8+        .quad   0x4d77e3f1bae58299+        .quad   0x1ef4612e7d1372a0+        .quad   0x3a8d867e70ff69e1+        .quad   0x6f58cd5d55aff958+        .quad   0xba3eaa5c75567721+        .quad   0x75c123999165227d+        .quad   0x69be1343c2f2b35e++        // 2^116 * 7 * B++        .quad   0x0e091d5ee197c92a+        .quad   0x4f51019f2945119f+        .quad   0x143679b9f034e99c+        .quad   0x7d88112e4d24c696+        .quad   0x82bbbdac684b8de3+        .quad   0xa2f4c7d03fca0718+        .quad   0x337f92fbe096aaa8+        .quad   0x200d4d8c63587376+        .quad   0x208aed4b4893b32b+        .quad   0x3efbf23ebe59b964+        .quad   0xd762deb0dba5e507+        .quad   0x69607bd681bd9d94++        // 2^116 * 8 * B++        .quad   0xf6be021068de1ce1+        .quad   0xe8d518e70edcbc1f+        .quad   0xe3effdd01b5505a5+        .quad   0x35f63353d3ec3fd0+        .quad   0x3b7f3bd49323a902+        .quad   0x7c21b5566b2c6e53+        .quad   0xe5ba8ff53a7852a7+        .quad   0x28bc77a5838ece00+        .quad   0x63ba78a8e25d8036+        .quad   0x63651e0094333490+        .quad   0x48d82f20288ce532+        .quad   0x3a31abfa36b57524++        // 2^120 * 1 * B++        .quad   0x239e9624089c0a2e+        .quad   0xc748c4c03afe4738+        .quad   0x17dbed2a764fa12a+        .quad   0x639b93f0321c8582+        .quad   0xc08f788f3f78d289+        .quad   0xfe30a72ca1404d9f+        .quad   0xf2778bfccf65cc9d+        .quad   0x7ee498165acb2021+        .quad   0x7bd508e39111a1c3+        .quad   0x2b2b90d480907489+        .quad   0xe7d2aec2ae72fd19+        .quad   0x0edf493c85b602a6++        // 2^120 * 2 * B++        .quad   0xaecc8158599b5a68+        .quad   0xea574f0febade20e+        .quad   0x4fe41d7422b67f07+        .quad   0x403b92e3019d4fb4+        .quad   0x6767c4d284764113+        .quad   0xa090403ff7f5f835+        .quad   0x1c8fcffacae6bede+        .quad   0x04c00c54d1dfa369+        .quad   0x4dc22f818b465cf8+        .quad   0x71a0f35a1480eff8+        .quad   0xaee8bfad04c7d657+        .quad   0x355bb12ab26176f4++        // 2^120 * 3 * B++        .quad   0xa71e64cc7493bbf4+        .quad   0xe5bd84d9eca3b0c3+        .quad   0x0a6bc50cfa05e785+        .quad   0x0f9b8132182ec312+        .quad   0xa301dac75a8c7318+        .quad   0xed90039db3ceaa11+        .quad   0x6f077cbf3bae3f2d+        .quad   0x7518eaf8e052ad8e+        .quad   0xa48859c41b7f6c32+        .quad   0x0f2d60bcf4383298+        .quad   0x1815a929c9b1d1d9+        .quad   0x47c3871bbb1755c4++        // 2^120 * 4 * B++        .quad   0x5144539771ec4f48+        .quad   0xf805b17dc98c5d6e+        .quad   0xf762c11a47c3c66b+        .quad   0x00b89b85764699dc+        .quad   0xfbe65d50c85066b0+        .quad   0x62ecc4b0b3a299b0+        .quad   0xe53754ea441ae8e0+        .quad   0x08fea02ce8d48d5f+        .quad   0x824ddd7668deead0+        .quad   0xc86445204b685d23+        .quad   0xb514cfcd5d89d665+        .quad   0x473829a74f75d537++        // 2^120 * 5 * B++        .quad   0x82d2da754679c418+        .quad   0xe63bd7d8b2618df0+        .quad   0x355eef24ac47eb0a+        .quad   0x2078684c4833c6b4+        .quad   0x23d9533aad3902c9+        .quad   0x64c2ddceef03588f+        .quad   0x15257390cfe12fb4+        .quad   0x6c668b4d44e4d390+        .quad   0x3b48cf217a78820c+        .quad   0xf76a0ab281273e97+        .quad   0xa96c65a78c8eed7b+        .quad   0x7411a6054f8a433f++        // 2^120 * 6 * B++        .quad   0x4d659d32b99dc86d+        .quad   0x044cdc75603af115+        .quad   0xb34c712cdcc2e488+        .quad   0x7c136574fb8134ff+        .quad   0x579ae53d18b175b4+        .quad   0x68713159f392a102+        .quad   0x8455ecba1eef35f5+        .quad   0x1ec9a872458c398f+        .quad   0xb8e6a4d400a2509b+        .quad   0x9b81d7020bc882b4+        .quad   0x57e7cc9bf1957561+        .quad   0x3add88a5c7cd6460++        // 2^120 * 7 * B++        .quad   0xab895770b635dcf2+        .quad   0x02dfef6cf66c1fbc+        .quad   0x85530268beb6d187+        .quad   0x249929fccc879e74+        .quad   0x85c298d459393046+        .quad   0x8f7e35985ff659ec+        .quad   0x1d2ca22af2f66e3a+        .quad   0x61ba1131a406a720+        .quad   0xa3d0a0f116959029+        .quad   0x023b6b6cba7ebd89+        .quad   0x7bf15a3e26783307+        .quad   0x5620310cbbd8ece7++        // 2^120 * 8 * B++        .quad   0x528993434934d643+        .quad   0xb9dbf806a51222f5+        .quad   0x8f6d878fc3f41c22+        .quad   0x37676a2a4d9d9730+        .quad   0x6646b5f477e285d6+        .quad   0x40e8ff676c8f6193+        .quad   0xa6ec7311abb594dd+        .quad   0x7ec846f3658cec4d+        .quad   0x9b5e8f3f1da22ec7+        .quad   0x130f1d776c01cd13+        .quad   0x214c8fcfa2989fb8+        .quad   0x6daaf723399b9dd5++        // 2^124 * 1 * B++        .quad   0x591e4a5610628564+        .quad   0x2a4bb87ca8b4df34+        .quad   0xde2a2572e7a38e43+        .quad   0x3cbdabd9fee5046e+        .quad   0x81aebbdd2cd13070+        .quad   0x962e4325f85a0e9e+        .quad   0xde9391aacadffecb+        .quad   0x53177fda52c230e6+        .quad   0xa7bc970650b9de79+        .quad   0x3d12a7fbc301b59b+        .quad   0x02652e68d36ae38c+        .quad   0x79d739835a6199dc++        // 2^124 * 2 * B++        .quad   0xd9354df64131c1bd+        .quad   0x758094a186ec5822+        .quad   0x4464ee12e459f3c2+        .quad   0x6c11fce4cb133282+        .quad   0x21c9d9920d591737+        .quad   0x9bea41d2e9b46cd6+        .quad   0xe20e84200d89bfca+        .quad   0x79d99f946eae5ff8+        .quad   0xf17b483568673205+        .quad   0x387deae83caad96c+        .quad   0x61b471fd56ffe386+        .quad   0x31741195b745a599++        // 2^124 * 3 * B++        .quad   0xe8d10190b77a360b+        .quad   0x99b983209995e702+        .quad   0xbd4fdff8fa0247aa+        .quad   0x2772e344e0d36a87+        .quad   0x17f8ba683b02a047+        .quad   0x50212096feefb6c8+        .quad   0x70139be21556cbe2+        .quad   0x203e44a11d98915b+        .quad   0xd6863eba37b9e39f+        .quad   0x105bc169723b5a23+        .quad   0x104f6459a65c0762+        .quad   0x567951295b4d38d4++        // 2^124 * 4 * B++        .quad   0x535fd60613037524+        .quad   0xe210adf6b0fbc26a+        .quad   0xac8d0a9b23e990ae+        .quad   0x47204d08d72fdbf9+        .quad   0x07242eb30d4b497f+        .quad   0x1ef96306b9bccc87+        .quad   0x37950934d8116f45+        .quad   0x05468d6201405b04+        .quad   0x00f565a9f93267de+        .quad   0xcecfd78dc0d58e8a+        .quad   0xa215e2dcf318e28e+        .quad   0x4599ee919b633352++        // 2^124 * 5 * B++        .quad   0xd3c220ca70e0e76b+        .quad   0xb12bea58ea9f3094+        .quad   0x294ddec8c3271282+        .quad   0x0c3539e1a1d1d028+        .quad   0xac746d6b861ae579+        .quad   0x31ab0650f6aea9dc+        .quad   0x241d661140256d4c+        .quad   0x2f485e853d21a5de+        .quad   0x329744839c0833f3+        .quad   0x6fe6257fd2abc484+        .quad   0x5327d1814b358817+        .quad   0x65712585893fe9bc++        // 2^124 * 6 * B++        .quad   0x9c102fb732a61161+        .quad   0xe48e10dd34d520a8+        .quad   0x365c63546f9a9176+        .quad   0x32f6fe4c046f6006+        .quad   0x81c29f1bd708ee3f+        .quad   0xddcb5a05ae6407d0+        .quad   0x97aec1d7d2a3eba7+        .quad   0x1590521a91d50831+        .quad   0x40a3a11ec7910acc+        .quad   0x9013dff8f16d27ae+        .quad   0x1a9720d8abb195d4+        .quad   0x1bb9fe452ea98463++        // 2^124 * 7 * B++        .quad   0xe9d1d950b3d54f9e+        .quad   0x2d5f9cbee00d33c1+        .quad   0x51c2c656a04fc6ac+        .quad   0x65c091ee3c1cbcc9+        .quad   0xcf5e6c95cc36747c+        .quad   0x294201536b0bc30d+        .quad   0x453ac67cee797af0+        .quad   0x5eae6ab32a8bb3c9+        .quad   0x7083661114f118ea+        .quad   0x2b37b87b94349cad+        .quad   0x7273f51cb4e99f40+        .quad   0x78a2a95823d75698++        // 2^124 * 8 * B++        .quad   0xa2b072e95c8c2ace+        .quad   0x69cffc96651e9c4b+        .quad   0x44328ef842e7b42b+        .quad   0x5dd996c122aadeb3+        .quad   0xb4f23c425ef83207+        .quad   0xabf894d3c9a934b5+        .quad   0xd0708c1339fd87f7+        .quad   0x1876789117166130+        .quad   0x925b5ef0670c507c+        .quad   0x819bc842b93c33bf+        .quad   0x10792e9a70dd003f+        .quad   0x59ad4b7a6e28dc74++        // 2^128 * 1 * B++        .quad   0x5f3a7562eb3dbe47+        .quad   0xf7ea38548ebda0b8+        .quad   0x00c3e53145747299+        .quad   0x1304e9e71627d551+        .quad   0x583b04bfacad8ea2+        .quad   0x29b743e8148be884+        .quad   0x2b1e583b0810c5db+        .quad   0x2b5449e58eb3bbaa+        .quad   0x789814d26adc9cfe+        .quad   0x3c1bab3f8b48dd0b+        .quad   0xda0fe1fff979c60a+        .quad   0x4468de2d7c2dd693++        // 2^128 * 2 * B++        .quad   0x51bb355e9419469e+        .quad   0x33e6dc4c23ddc754+        .quad   0x93a5b6d6447f9962+        .quad   0x6cce7c6ffb44bd63+        .quad   0x4b9ad8c6f86307ce+        .quad   0x21113531435d0c28+        .quad   0xd4a866c5657a772c+        .quad   0x5da6427e63247352+        .quad   0x1a94c688deac22ca+        .quad   0xb9066ef7bbae1ff8+        .quad   0x88ad8c388d59580f+        .quad   0x58f29abfe79f2ca8++        // 2^128 * 3 * B++        .quad   0xe90ecfab8de73e68+        .quad   0x54036f9f377e76a5+        .quad   0xf0495b0bbe015982+        .quad   0x577629c4a7f41e36+        .quad   0x4b5a64bf710ecdf6+        .quad   0xb14ce538462c293c+        .quad   0x3643d056d50b3ab9+        .quad   0x6af93724185b4870+        .quad   0x3220024509c6a888+        .quad   0xd2e036134b558973+        .quad   0x83e236233c33289f+        .quad   0x701f25bb0caec18f++        // 2^128 * 4 * B++        .quad   0xc3a8b0f8e4616ced+        .quad   0xf700660e9e25a87d+        .quad   0x61e3061ff4bca59c+        .quad   0x2e0c92bfbdc40be9+        .quad   0x9d18f6d97cbec113+        .quad   0x844a06e674bfdbe4+        .quad   0x20f5b522ac4e60d6+        .quad   0x720a5bc050955e51+        .quad   0x0c3f09439b805a35+        .quad   0xe84e8b376242abfc+        .quad   0x691417f35c229346+        .quad   0x0e9b9cbb144ef0ec++        // 2^128 * 5 * B++        .quad   0xfbbad48ffb5720ad+        .quad   0xee81916bdbf90d0e+        .quad   0xd4813152635543bf+        .quad   0x221104eb3f337bd8+        .quad   0x8dee9bd55db1beee+        .quad   0xc9c3ab370a723fb9+        .quad   0x44a8f1bf1c68d791+        .quad   0x366d44191cfd3cde+        .quad   0x9e3c1743f2bc8c14+        .quad   0x2eda26fcb5856c3b+        .quad   0xccb82f0e68a7fb97+        .quad   0x4167a4e6bc593244++        // 2^128 * 6 * B++        .quad   0x643b9d2876f62700+        .quad   0x5d1d9d400e7668eb+        .quad   0x1b4b430321fc0684+        .quad   0x7938bb7e2255246a+        .quad   0xc2be2665f8ce8fee+        .quad   0xe967ff14e880d62c+        .quad   0xf12e6e7e2f364eee+        .quad   0x34b33370cb7ed2f6+        .quad   0xcdc591ee8681d6cc+        .quad   0xce02109ced85a753+        .quad   0xed7485c158808883+        .quad   0x1176fc6e2dfe65e4++        // 2^128 * 7 * B++        .quad   0xb4af6cd05b9c619b+        .quad   0x2ddfc9f4b2a58480+        .quad   0x3d4fa502ebe94dc4+        .quad   0x08fc3a4c677d5f34+        .quad   0xdb90e28949770eb8+        .quad   0x98fbcc2aacf440a3+        .quad   0x21354ffeded7879b+        .quad   0x1f6a3e54f26906b6+        .quad   0x60a4c199d30734ea+        .quad   0x40c085b631165cd6+        .quad   0xe2333e23f7598295+        .quad   0x4f2fad0116b900d1++        // 2^128 * 8 * B++        .quad   0x44beb24194ae4e54+        .quad   0x5f541c511857ef6c+        .quad   0xa61e6b2d368d0498+        .quad   0x445484a4972ef7ab+        .quad   0x962cd91db73bb638+        .quad   0xe60577aafc129c08+        .quad   0x6f619b39f3b61689+        .quad   0x3451995f2944ee81+        .quad   0x9152fcd09fea7d7c+        .quad   0x4a816c94b0935cf6+        .quad   0x258e9aaa47285c40+        .quad   0x10b89ca6042893b7++        // 2^132 * 1 * B++        .quad   0x9b2a426e3b646025+        .quad   0x32127190385ce4cf+        .quad   0xa25cffc2dd6dea45+        .quad   0x06409010bea8de75+        .quad   0xd67cded679d34aa0+        .quad   0xcc0b9ec0cc4db39f+        .quad   0xa535a456e35d190f+        .quad   0x2e05d9eaf61f6fef+        .quad   0xc447901ad61beb59+        .quad   0x661f19bce5dc880a+        .quad   0x24685482b7ca6827+        .quad   0x293c778cefe07f26++        // 2^132 * 2 * B++        .quad   0x86809e7007069096+        .quad   0xaad75b15e4e50189+        .quad   0x07f35715a21a0147+        .quad   0x0487f3f112815d5e+        .quad   0x16c795d6a11ff200+        .quad   0xcb70d0e2b15815c9+        .quad   0x89f293209b5395b5+        .quad   0x50b8c2d031e47b4f+        .quad   0x48350c08068a4962+        .quad   0x6ffdd05351092c9a+        .quad   0x17af4f4aaf6fc8dd+        .quad   0x4b0553b53cdba58b++        // 2^132 * 3 * B++        .quad   0x9c65fcbe1b32ff79+        .quad   0xeb75ea9f03b50f9b+        .quad   0xfced2a6c6c07e606+        .quad   0x35106cd551717908+        .quad   0xbf05211b27c152d4+        .quad   0x5ec26849bd1af639+        .quad   0x5e0b2caa8e6fab98+        .quad   0x054c8bdd50bd0840+        .quad   0x38a0b12f1dcf073d+        .quad   0x4b60a8a3b7f6a276+        .quad   0xfed5ac25d3404f9a+        .quad   0x72e82d5e5505c229++        // 2^132 * 4 * B++        .quad   0x6b0b697ff0d844c8+        .quad   0xbb12f85cd979cb49+        .quad   0xd2a541c6c1da0f1f+        .quad   0x7b7c242958ce7211+        .quad   0x00d9cdfd69771d02+        .quad   0x410276cd6cfbf17e+        .quad   0x4c45306c1cb12ec7+        .quad   0x2857bf1627500861+        .quad   0x9f21903f0101689e+        .quad   0xd779dfd3bf861005+        .quad   0xa122ee5f3deb0f1b+        .quad   0x510df84b485a00d4++        // 2^132 * 5 * B++        .quad   0xa54133bb9277a1fa+        .quad   0x74ec3b6263991237+        .quad   0x1a3c54dc35d2f15a+        .quad   0x2d347144e482ba3a+        .quad   0x24b3c887c70ac15e+        .quad   0xb0f3a557fb81b732+        .quad   0x9b2cde2fe578cc1b+        .quad   0x4cf7ed0703b54f8e+        .quad   0x6bd47c6598fbee0f+        .quad   0x9e4733e2ab55be2d+        .quad   0x1093f624127610c5+        .quad   0x4e05e26ad0a1eaa4++        // 2^132 * 6 * B++        .quad   0xda9b6b624b531f20+        .quad   0x429a760e77509abb+        .quad   0xdbe9f522e823cb80+        .quad   0x618f1856880c8f82+        .quad   0x1833c773e18fe6c0+        .quad   0xe3c4711ad3c87265+        .quad   0x3bfd3c4f0116b283+        .quad   0x1955875eb4cd4db8+        .quad   0x6da6de8f0e399799+        .quad   0x7ad61aa440fda178+        .quad   0xb32cd8105e3563dd+        .quad   0x15f6beae2ae340ae++        // 2^132 * 7 * B++        .quad   0x862bcb0c31ec3a62+        .quad   0x810e2b451138f3c2+        .quad   0x788ec4b839dac2a4+        .quad   0x28f76867ae2a9281+        .quad   0xba9a0f7b9245e215+        .quad   0xf368612dd98c0dbb+        .quad   0x2e84e4cbf220b020+        .quad   0x6ba92fe962d90eda+        .quad   0x3e4df9655884e2aa+        .quad   0xbd62fbdbdbd465a5+        .quad   0xd7596caa0de9e524+        .quad   0x6e8042ccb2b1b3d7++        // 2^132 * 8 * B++        .quad   0xf10d3c29ce28ca6e+        .quad   0xbad34540fcb6093d+        .quad   0xe7426ed7a2ea2d3f+        .quad   0x08af9d4e4ff298b9+        .quad   0x1530653616521f7e+        .quad   0x660d06b896203dba+        .quad   0x2d3989bc545f0879+        .quad   0x4b5303af78ebd7b0+        .quad   0x72f8a6c3bebcbde8+        .quad   0x4f0fca4adc3a8e89+        .quad   0x6fa9d4e8c7bfdf7a+        .quad   0x0dcf2d679b624eb7++        // 2^136 * 1 * B++        .quad   0x3d5947499718289c+        .quad   0x12ebf8c524533f26+        .quad   0x0262bfcb14c3ef15+        .quad   0x20b878d577b7518e+        .quad   0x753941be5a45f06e+        .quad   0xd07caeed6d9c5f65+        .quad   0x11776b9c72ff51b6+        .quad   0x17d2d1d9ef0d4da9+        .quad   0x27f2af18073f3e6a+        .quad   0xfd3fe519d7521069+        .quad   0x22e3b72c3ca60022+        .quad   0x72214f63cc65c6a7++        // 2^136 * 2 * B++        .quad   0xb4e37f405307a693+        .quad   0xaba714d72f336795+        .quad   0xd6fbd0a773761099+        .quad   0x5fdf48c58171cbc9+        .quad   0x1d9db7b9f43b29c9+        .quad   0xd605824a4f518f75+        .quad   0xf2c072bd312f9dc4+        .quad   0x1f24ac855a1545b0+        .quad   0x24d608328e9505aa+        .quad   0x4748c1d10c1420ee+        .quad   0xc7ffe45c06fb25a2+        .quad   0x00ba739e2ae395e6++        // 2^136 * 3 * B++        .quad   0x592e98de5c8790d6+        .quad   0xe5bfb7d345c2a2df+        .quad   0x115a3b60f9b49922+        .quad   0x03283a3e67ad78f3+        .quad   0xae4426f5ea88bb26+        .quad   0x360679d984973bfb+        .quad   0x5c9f030c26694e50+        .quad   0x72297de7d518d226+        .quad   0x48241dc7be0cb939+        .quad   0x32f19b4d8b633080+        .quad   0xd3dfc90d02289308+        .quad   0x05e1296846271945++        // 2^136 * 4 * B++        .quad   0xba82eeb32d9c495a+        .quad   0xceefc8fcf12bb97c+        .quad   0xb02dabae93b5d1e0+        .quad   0x39c00c9c13698d9b+        .quad   0xadbfbbc8242c4550+        .quad   0xbcc80cecd03081d9+        .quad   0x843566a6f5c8df92+        .quad   0x78cf25d38258ce4c+        .quad   0x15ae6b8e31489d68+        .quad   0xaa851cab9c2bf087+        .quad   0xc9a75a97f04efa05+        .quad   0x006b52076b3ff832++        // 2^136 * 5 * B++        .quad   0x29e0cfe19d95781c+        .quad   0xb681df18966310e2+        .quad   0x57df39d370516b39+        .quad   0x4d57e3443bc76122+        .quad   0xf5cb7e16b9ce082d+        .quad   0x3407f14c417abc29+        .quad   0xd4b36bce2bf4a7ab+        .quad   0x7de2e9561a9f75ce+        .quad   0xde70d4f4b6a55ecb+        .quad   0x4801527f5d85db99+        .quad   0xdbc9c440d3ee9a81+        .quad   0x6b2a90af1a6029ed++        // 2^136 * 6 * B++        .quad   0x6923f4fc9ae61e97+        .quad   0x5735281de03f5fd1+        .quad   0xa764ae43e6edd12d+        .quad   0x5fd8f4e9d12d3e4a+        .quad   0x77ebf3245bb2d80a+        .quad   0xd8301b472fb9079b+        .quad   0xc647e6f24cee7333+        .quad   0x465812c8276c2109+        .quad   0x4d43beb22a1062d9+        .quad   0x7065fb753831dc16+        .quad   0x180d4a7bde2968d7+        .quad   0x05b32c2b1cb16790++        // 2^136 * 7 * B++        .quad   0xc8c05eccd24da8fd+        .quad   0xa1cf1aac05dfef83+        .quad   0xdbbeeff27df9cd61+        .quad   0x3b5556a37b471e99+        .quad   0xf7fca42c7ad58195+        .quad   0x3214286e4333f3cc+        .quad   0xb6c29d0d340b979d+        .quad   0x31771a48567307e1+        .quad   0x32b0c524e14dd482+        .quad   0xedb351541a2ba4b6+        .quad   0xa3d16048282b5af3+        .quad   0x4fc079d27a7336eb++        // 2^136 * 8 * B++        .quad   0x51c938b089bf2f7f+        .quad   0x2497bd6502dfe9a7+        .quad   0xffffc09c7880e453+        .quad   0x124567cecaf98e92+        .quad   0xdc348b440c86c50d+        .quad   0x1337cbc9cc94e651+        .quad   0x6422f74d643e3cb9+        .quad   0x241170c2bae3cd08+        .quad   0x3ff9ab860ac473b4+        .quad   0xf0911dee0113e435+        .quad   0x4ae75060ebc6c4af+        .quad   0x3f8612966c87000d++        // 2^140 * 1 * B++        .quad   0x0c9c5303f7957be4+        .quad   0xa3c31a20e085c145+        .quad   0xb0721d71d0850050+        .quad   0x0aba390eab0bf2da+        .quad   0x529fdffe638c7bf3+        .quad   0xdf2b9e60388b4995+        .quad   0xe027b34f1bad0249+        .quad   0x7bc92fc9b9fa74ed+        .quad   0x9f97ef2e801ad9f9+        .quad   0x83697d5479afda3a+        .quad   0xe906b3ffbd596b50+        .quad   0x02672b37dd3fb8e0++        // 2^140 * 2 * B++        .quad   0x48b2ca8b260885e4+        .quad   0xa4286bec82b34c1c+        .quad   0x937e1a2617f58f74+        .quad   0x741d1fcbab2ca2a5+        .quad   0xee9ba729398ca7f5+        .quad   0xeb9ca6257a4849db+        .quad   0x29eb29ce7ec544e1+        .quad   0x232ca21ef736e2c8+        .quad   0xbf61423d253fcb17+        .quad   0x08803ceafa39eb14+        .quad   0xf18602df9851c7af+        .quad   0x0400f3a049e3414b++        // 2^140 * 3 * B++        .quad   0xabce0476ba61c55b+        .quad   0x36a3d6d7c4d39716+        .quad   0x6eb259d5e8d82d09+        .quad   0x0c9176e984d756fb+        .quad   0x2efba412a06e7b06+        .quad   0x146785452c8d2560+        .quad   0xdf9713ebd67a91c7+        .quad   0x32830ac7157eadf3+        .quad   0x0e782a7ab73769e8+        .quad   0x04a05d7875b18e2c+        .quad   0x29525226ebcceae1+        .quad   0x0d794f8383eba820++        // 2^140 * 4 * B++        .quad   0xff35f5cb9e1516f4+        .quad   0xee805bcf648aae45+        .quad   0xf0d73c2bb93a9ef3+        .quad   0x097b0bf22092a6c2+        .quad   0x7be44ce7a7a2e1ac+        .quad   0x411fd93efad1b8b7+        .quad   0x1734a1d70d5f7c9b+        .quad   0x0d6592233127db16+        .quad   0xc48bab1521a9d733+        .quad   0xa6c2eaead61abb25+        .quad   0x625c6c1cc6cb4305+        .quad   0x7fc90fea93eb3a67++        // 2^140 * 5 * B++        .quad   0x0408f1fe1f5c5926+        .quad   0x1a8f2f5e3b258bf4+        .quad   0x40a951a2fdc71669+        .quad   0x6598ee93c98b577e+        .quad   0xc527deb59c7cb23d+        .quad   0x955391695328404e+        .quad   0xd64392817ccf2c7a+        .quad   0x6ce97dabf7d8fa11+        .quad   0x25b5a8e50ef7c48f+        .quad   0xeb6034116f2ce532+        .quad   0xc5e75173e53de537+        .quad   0x73119fa08c12bb03++        // 2^140 * 6 * B++        .quad   0xed30129453f1a4cb+        .quad   0xbce621c9c8f53787+        .quad   0xfacb2b1338bee7b9+        .quad   0x3025798a9ea8428c+        .quad   0x7845b94d21f4774d+        .quad   0xbf62f16c7897b727+        .quad   0x671857c03c56522b+        .quad   0x3cd6a85295621212+        .quad   0x3fecde923aeca999+        .quad   0xbdaa5b0062e8c12f+        .quad   0x67b99dfc96988ade+        .quad   0x3f52c02852661036++        // 2^140 * 7 * B++        .quad   0xffeaa48e2a1351c6+        .quad   0x28624754fa7f53d7+        .quad   0x0b5ba9e57582ddf1+        .quad   0x60c0104ba696ac59+        .quad   0x9258bf99eec416c6+        .quad   0xac8a5017a9d2f671+        .quad   0x629549ab16dea4ab+        .quad   0x05d0e85c99091569+        .quad   0x051de020de9cbe97+        .quad   0xfa07fc56b50bcf74+        .quad   0x378cec9f0f11df65+        .quad   0x36853c69ab96de4d++        // 2^140 * 8 * B++        .quad   0x36d9b8de78f39b2d+        .quad   0x7f42ed71a847b9ec+        .quad   0x241cd1d679bd3fde+        .quad   0x6a704fec92fbce6b+        .quad   0x4433c0b0fac5e7be+        .quad   0x724bae854c08dcbe+        .quad   0xf1f24cc446978f9b+        .quad   0x4a0aff6d62825fc8+        .quad   0xe917fb9e61095301+        .quad   0xc102df9402a092f8+        .quad   0xbf09e2f5fa66190b+        .quad   0x681109bee0dcfe37++        // 2^144 * 1 * B++        .quad   0x559a0cc9782a0dde+        .quad   0x551dcdb2ea718385+        .quad   0x7f62865b31ef238c+        .quad   0x504aa7767973613d+        .quad   0x9c18fcfa36048d13+        .quad   0x29159db373899ddd+        .quad   0xdc9f350b9f92d0aa+        .quad   0x26f57eee878a19d4+        .quad   0x0cab2cd55687efb1+        .quad   0x5180d162247af17b+        .quad   0x85c15a344f5a2467+        .quad   0x4041943d9dba3069++        // 2^144 * 2 * B++        .quad   0xc3c0eeba43ebcc96+        .quad   0x8d749c9c26ea9caf+        .quad   0xd9fa95ee1c77ccc6+        .quad   0x1420a1d97684340f+        .quad   0x4b217743a26caadd+        .quad   0x47a6b424648ab7ce+        .quad   0xcb1d4f7a03fbc9e3+        .quad   0x12d931429800d019+        .quad   0x00c67799d337594f+        .quad   0x5e3c5140b23aa47b+        .quad   0x44182854e35ff395+        .quad   0x1b4f92314359a012++        // 2^144 * 3 * B++        .quad   0x3e5c109d89150951+        .quad   0x39cefa912de9696a+        .quad   0x20eae43f975f3020+        .quad   0x239b572a7f132dae+        .quad   0x33cf3030a49866b1+        .quad   0x251f73d2215f4859+        .quad   0xab82aa4051def4f6+        .quad   0x5ff191d56f9a23f6+        .quad   0x819ed433ac2d9068+        .quad   0x2883ab795fc98523+        .quad   0xef4572805593eb3d+        .quad   0x020c526a758f36cb++        // 2^144 * 4 * B++        .quad   0x779834f89ed8dbbc+        .quad   0xc8f2aaf9dc7ca46c+        .quad   0xa9524cdca3e1b074+        .quad   0x02aacc4615313877+        .quad   0xe931ef59f042cc89+        .quad   0x2c589c9d8e124bb6+        .quad   0xadc8e18aaec75997+        .quad   0x452cfe0a5602c50c+        .quad   0x86a0f7a0647877df+        .quad   0xbbc464270e607c9f+        .quad   0xab17ea25f1fb11c9+        .quad   0x4cfb7d7b304b877b++        // 2^144 * 5 * B++        .quad   0x72b43d6cb89b75fe+        .quad   0x54c694d99c6adc80+        .quad   0xb8c3aa373ee34c9f+        .quad   0x14b4622b39075364+        .quad   0xe28699c29789ef12+        .quad   0x2b6ecd71df57190d+        .quad   0xc343c857ecc970d0+        .quad   0x5b1d4cbc434d3ac5+        .quad   0xb6fb2615cc0a9f26+        .quad   0x3a4f0e2bb88dcce5+        .quad   0x1301498b3369a705+        .quad   0x2f98f71258592dd1++        // 2^144 * 6 * B++        .quad   0x0c94a74cb50f9e56+        .quad   0x5b1ff4a98e8e1320+        .quad   0x9a2acc2182300f67+        .quad   0x3a6ae249d806aaf9+        .quad   0x2e12ae444f54a701+        .quad   0xfcfe3ef0a9cbd7de+        .quad   0xcebf890d75835de0+        .quad   0x1d8062e9e7614554+        .quad   0x657ada85a9907c5a+        .quad   0x1a0ea8b591b90f62+        .quad   0x8d0e1dfbdf34b4e9+        .quad   0x298b8ce8aef25ff3++        // 2^144 * 7 * B++        .quad   0x2a927953eff70cb2+        .quad   0x4b89c92a79157076+        .quad   0x9418457a30a7cf6a+        .quad   0x34b8a8404d5ce485+        .quad   0x837a72ea0a2165de+        .quad   0x3fab07b40bcf79f6+        .quad   0x521636c77738ae70+        .quad   0x6ba6271803a7d7dc+        .quad   0xc26eecb583693335+        .quad   0xd5a813df63b5fefd+        .quad   0xa293aa9aa4b22573+        .quad   0x71d62bdd465e1c6a++        // 2^144 * 8 * B++        .quad   0x6533cc28d378df80+        .quad   0xf6db43790a0fa4b4+        .quad   0xe3645ff9f701da5a+        .quad   0x74d5f317f3172ba4+        .quad   0xcd2db5dab1f75ef5+        .quad   0xd77f95cf16b065f5+        .quad   0x14571fea3f49f085+        .quad   0x1c333621262b2b3d+        .quad   0xa86fe55467d9ca81+        .quad   0x398b7c752b298c37+        .quad   0xda6d0892e3ac623b+        .quad   0x4aebcc4547e9d98c++        // 2^148 * 1 * B++        .quad   0x53175a7205d21a77+        .quad   0xb0c04422d3b934d4+        .quad   0xadd9f24bdd5deadc+        .quad   0x074f46e69f10ff8c+        .quad   0x0de9b204a059a445+        .quad   0xe15cb4aa4b17ad0f+        .quad   0xe1bbec521f79c557+        .quad   0x2633f1b9d071081b+        .quad   0xc1fb4177018b9910+        .quad   0xa6ea20dc6c0fe140+        .quad   0xd661f3e74354c6ff+        .quad   0x5ecb72e6f1a3407a++        // 2^148 * 2 * B++        .quad   0xa515a31b2259fb4e+        .quad   0x0960f3972bcac52f+        .quad   0xedb52fec8d3454cb+        .quad   0x382e2720c476c019+        .quad   0xfeeae106e8e86997+        .quad   0x9863337f98d09383+        .quad   0x9470480eaa06ebef+        .quad   0x038b6898d4c5c2d0+        .quad   0xf391c51d8ace50a6+        .quad   0x3142d0b9ae2d2948+        .quad   0xdb4d5a1a7f24ca80+        .quad   0x21aeba8b59250ea8++        // 2^148 * 3 * B++        .quad   0x24f13b34cf405530+        .quad   0x3c44ea4a43088af7+        .quad   0x5dd5c5170006a482+        .quad   0x118eb8f8890b086d+        .quad   0x53853600f0087f23+        .quad   0x4c461879da7d5784+        .quad   0x6af303deb41f6860+        .quad   0x0a3c16c5c27c18ed+        .quad   0x17e49c17cc947f3d+        .quad   0xccc6eda6aac1d27b+        .quad   0xdf6092ceb0f08e56+        .quad   0x4909b3e22c67c36b++        // 2^148 * 4 * B++        .quad   0x9c9c85ea63fe2e89+        .quad   0xbe1baf910e9412ec+        .quad   0x8f7baa8a86fbfe7b+        .quad   0x0fb17f9fef968b6c+        .quad   0x59a16676706ff64e+        .quad   0x10b953dd0d86a53d+        .quad   0x5848e1e6ce5c0b96+        .quad   0x2d8b78e712780c68+        .quad   0x79d5c62eafc3902b+        .quad   0x773a215289e80728+        .quad   0xc38ae640e10120b9+        .quad   0x09ae23717b2b1a6d++        // 2^148 * 5 * B++        .quad   0xbb6a192a4e4d083c+        .quad   0x34ace0630029e192+        .quad   0x98245a59aafabaeb+        .quad   0x6d9c8a9ada97faac+        .quad   0x10ab8fa1ad32b1d0+        .quad   0xe9aced1be2778b24+        .quad   0xa8856bc0373de90f+        .quad   0x66f35ddddda53996+        .quad   0xd27d9afb24997323+        .quad   0x1bb7e07ef6f01d2e+        .quad   0x2ba7472df52ecc7f+        .quad   0x03019b4f646f9dc8++        // 2^148 * 6 * B++        .quad   0x04a186b5565345cd+        .quad   0xeee76610bcc4116a+        .quad   0x689c73b478fb2a45+        .quad   0x387dcbff65697512+        .quad   0xaf09b214e6b3dc6b+        .quad   0x3f7573b5ad7d2f65+        .quad   0xd019d988100a23b0+        .quad   0x392b63a58b5c35f7+        .quad   0x4093addc9c07c205+        .quad   0xc565be15f532c37e+        .quad   0x63dbecfd1583402a+        .quad   0x61722b4aef2e032e++        // 2^148 * 7 * B++        .quad   0x0012aafeecbd47af+        .quad   0x55a266fb1cd46309+        .quad   0xf203eb680967c72c+        .quad   0x39633944ca3c1429+        .quad   0xd6b07a5581cb0e3c+        .quad   0x290ff006d9444969+        .quad   0x08680b6a16dcda1f+        .quad   0x5568d2b75a06de59+        .quad   0x8d0cb88c1b37cfe1+        .quad   0x05b6a5a3053818f3+        .quad   0xf2e9bc04b787d959+        .quad   0x6beba1249add7f64++        // 2^148 * 8 * B++        .quad   0x1d06005ca5b1b143+        .quad   0x6d4c6bb87fd1cda2+        .quad   0x6ef5967653fcffe7+        .quad   0x097c29e8c1ce1ea5+        .quad   0x5c3cecb943f5a53b+        .quad   0x9cc9a61d06c08df2+        .quad   0xcfba639a85895447+        .quad   0x5a845ae80df09fd5+        .quad   0x4ce97dbe5deb94ca+        .quad   0x38d0a4388c709c48+        .quad   0xc43eced4a169d097+        .quad   0x0a1249fff7e587c3++        // 2^152 * 1 * B++        .quad   0x12f0071b276d01c9+        .quad   0xe7b8bac586c48c70+        .quad   0x5308129b71d6fba9+        .quad   0x5d88fbf95a3db792+        .quad   0x0b408d9e7354b610+        .quad   0x806b32535ba85b6e+        .quad   0xdbe63a034a58a207+        .quad   0x173bd9ddc9a1df2c+        .quad   0x2b500f1efe5872df+        .quad   0x58d6582ed43918c1+        .quad   0xe6ed278ec9673ae0+        .quad   0x06e1cd13b19ea319++        // 2^152 * 2 * B++        .quad   0x40d0ad516f166f23+        .quad   0x118e32931fab6abe+        .quad   0x3fe35e14a04d088e+        .quad   0x3080603526e16266+        .quad   0x472baf629e5b0353+        .quad   0x3baa0b90278d0447+        .quad   0x0c785f469643bf27+        .quad   0x7f3a6a1a8d837b13+        .quad   0xf7e644395d3d800b+        .quad   0x95a8d555c901edf6+        .quad   0x68cd7830592c6339+        .quad   0x30d0fded2e51307e++        // 2^152 * 3 * B++        .quad   0xe0594d1af21233b3+        .quad   0x1bdbe78ef0cc4d9c+        .quad   0x6965187f8f499a77+        .quad   0x0a9214202c099868+        .quad   0x9cb4971e68b84750+        .quad   0xa09572296664bbcf+        .quad   0x5c8de72672fa412b+        .quad   0x4615084351c589d9+        .quad   0xbc9019c0aeb9a02e+        .quad   0x55c7110d16034cae+        .quad   0x0e6df501659932ec+        .quad   0x3bca0d2895ca5dfe++        // 2^152 * 4 * B++        .quad   0x40f031bc3c5d62a4+        .quad   0x19fc8b3ecff07a60+        .quad   0x98183da2130fb545+        .quad   0x5631deddae8f13cd+        .quad   0x9c688eb69ecc01bf+        .quad   0xf0bc83ada644896f+        .quad   0xca2d955f5f7a9fe2+        .quad   0x4ea8b4038df28241+        .quad   0x2aed460af1cad202+        .quad   0x46305305a48cee83+        .quad   0x9121774549f11a5f+        .quad   0x24ce0930542ca463++        // 2^152 * 5 * B++        .quad   0x1fe890f5fd06c106+        .quad   0xb5c468355d8810f2+        .quad   0x827808fe6e8caf3e+        .quad   0x41d4e3c28a06d74b+        .quad   0x3fcfa155fdf30b85+        .quad   0xd2f7168e36372ea4+        .quad   0xb2e064de6492f844+        .quad   0x549928a7324f4280+        .quad   0xf26e32a763ee1a2e+        .quad   0xae91e4b7d25ffdea+        .quad   0xbc3bd33bd17f4d69+        .quad   0x491b66dec0dcff6a++        // 2^152 * 6 * B++        .quad   0x98f5b13dc7ea32a7+        .quad   0xe3d5f8cc7e16db98+        .quad   0xac0abf52cbf8d947+        .quad   0x08f338d0c85ee4ac+        .quad   0x75f04a8ed0da64a1+        .quad   0xed222caf67e2284b+        .quad   0x8234a3791f7b7ba4+        .quad   0x4cf6b8b0b7018b67+        .quad   0xc383a821991a73bd+        .quad   0xab27bc01df320c7a+        .quad   0xc13d331b84777063+        .quad   0x530d4a82eb078a99++        // 2^152 * 7 * B++        .quad   0x004c3630e1f94825+        .quad   0x7e2d78268cab535a+        .quad   0xc7482323cc84ff8b+        .quad   0x65ea753f101770b9+        .quad   0x6d6973456c9abf9e+        .quad   0x257fb2fc4900a880+        .quad   0x2bacf412c8cfb850+        .quad   0x0db3e7e00cbfbd5b+        .quad   0x3d66fc3ee2096363+        .quad   0x81d62c7f61b5cb6b+        .quad   0x0fbe044213443b1a+        .quad   0x02a4ec1921e1a1db++        // 2^152 * 8 * B++        .quad   0x5ce6259a3b24b8a2+        .quad   0xb8577acc45afa0b8+        .quad   0xcccbe6e88ba07037+        .quad   0x3d143c51127809bf+        .quad   0xf5c86162f1cf795f+        .quad   0x118c861926ee57f2+        .quad   0x172124851c063578+        .quad   0x36d12b5dec067fcf+        .quad   0x126d279179154557+        .quad   0xd5e48f5cfc783a0a+        .quad   0x36bdb6e8df179bac+        .quad   0x2ef517885ba82859++        // 2^156 * 1 * B++        .quad   0x88bd438cd11e0d4a+        .quad   0x30cb610d43ccf308+        .quad   0xe09a0e3791937bcc+        .quad   0x4559135b25b1720c+        .quad   0x1ea436837c6da1e9+        .quad   0xf9c189af1fb9bdbe+        .quad   0x303001fcce5dd155+        .quad   0x28a7c99ebc57be52+        .quad   0xb8fd9399e8d19e9d+        .quad   0x908191cb962423ff+        .quad   0xb2b948d747c742a3+        .quad   0x37f33226d7fb44c4++        // 2^156 * 2 * B++        .quad   0x0dae8767b55f6e08+        .quad   0x4a43b3b35b203a02+        .quad   0xe3725a6e80af8c79+        .quad   0x0f7a7fd1705fa7a3+        .quad   0x33912553c821b11d+        .quad   0x66ed42c241e301df+        .quad   0x066fcc11104222fd+        .quad   0x307a3b41c192168f+        .quad   0x8eeb5d076eb55ce0+        .quad   0x2fc536bfaa0d925a+        .quad   0xbe81830fdcb6c6e8+        .quad   0x556c7045827baf52++        // 2^156 * 3 * B++        .quad   0x8e2b517302e9d8b7+        .quad   0xe3e52269248714e8+        .quad   0xbd4fbd774ca960b5+        .quad   0x6f4b4199c5ecada9+        .quad   0xb94b90022bf44406+        .quad   0xabd4237eff90b534+        .quad   0x7600a960faf86d3a+        .quad   0x2f45abdac2322ee3+        .quad   0x61af4912c8ef8a6a+        .quad   0xe58fa4fe43fb6e5e+        .quad   0xb5afcc5d6fd427cf+        .quad   0x6a5393281e1e11eb++        // 2^156 * 4 * B++        .quad   0xf3da5139a5d1ee89+        .quad   0x8145457cff936988+        .quad   0x3f622fed00e188c4+        .quad   0x0f513815db8b5a3d+        .quad   0x0fff04fe149443cf+        .quad   0x53cac6d9865cddd7+        .quad   0x31385b03531ed1b7+        .quad   0x5846a27cacd1039d+        .quad   0x4ff5cdac1eb08717+        .quad   0x67e8b29590f2e9bc+        .quad   0x44093b5e237afa99+        .quad   0x0d414bed8708b8b2++        // 2^156 * 5 * B++        .quad   0xcfb68265fd0e75f6+        .quad   0xe45b3e28bb90e707+        .quad   0x7242a8de9ff92c7a+        .quad   0x685b3201933202dd+        .quad   0x81886a92294ac9e8+        .quad   0x23162b45d55547be+        .quad   0x94cfbc4403715983+        .quad   0x50eb8fdb134bc401+        .quad   0xc0b73ec6d6b330cd+        .quad   0x84e44807132faff1+        .quad   0x732b7352c4a5dee1+        .quad   0x5d7c7cf1aa7cd2d2++        // 2^156 * 6 * B++        .quad   0xaf3b46bf7a4aafa2+        .quad   0xb78705ec4d40d411+        .quad   0x114f0c6aca7c15e3+        .quad   0x3f364faaa9489d4d+        .quad   0x33d1013e9b73a562+        .quad   0x925cef5748ec26e1+        .quad   0xa7fce614dd468058+        .quad   0x78b0fad41e9aa438+        .quad   0xbf56a431ed05b488+        .quad   0xa533e66c9c495c7e+        .quad   0xe8652baf87f3651a+        .quad   0x0241800059d66c33++        // 2^156 * 7 * B++        .quad   0xceb077fea37a5be4+        .quad   0xdb642f02e5a5eeb7+        .quad   0xc2e6d0c5471270b8+        .quad   0x4771b65538e4529c+        .quad   0x28350c7dcf38ea01+        .quad   0x7c6cdbc0b2917ab6+        .quad   0xace7cfbe857082f7+        .quad   0x4d2845aba2d9a1e0+        .quad   0xbb537fe0447070de+        .quad   0xcba744436dd557df+        .quad   0xd3b5a3473600dbcb+        .quad   0x4aeabbe6f9ffd7f8++        // 2^156 * 8 * B++        .quad   0x4630119e40d8f78c+        .quad   0xa01a9bc53c710e11+        .quad   0x486d2b258910dd79+        .quad   0x1e6c47b3db0324e5+        .quad   0x6a2134bcc4a9c8f2+        .quad   0xfbf8fd1c8ace2e37+        .quad   0x000ae3049911a0ba+        .quad   0x046e3a616bc89b9e+        .quad   0x14e65442f03906be+        .quad   0x4a019d54e362be2a+        .quad   0x68ccdfec8dc230c7+        .quad   0x7cfb7e3faf6b861c++        // 2^160 * 1 * B++        .quad   0x4637974e8c58aedc+        .quad   0xb9ef22fbabf041a4+        .quad   0xe185d956e980718a+        .quad   0x2f1b78fab143a8a6+        .quad   0x96eebffb305b2f51+        .quad   0xd3f938ad889596b8+        .quad   0xf0f52dc746d5dd25+        .quad   0x57968290bb3a0095+        .quad   0xf71ab8430a20e101+        .quad   0xf393658d24f0ec47+        .quad   0xcf7509a86ee2eed1+        .quad   0x7dc43e35dc2aa3e1++        // 2^160 * 2 * B++        .quad   0x85966665887dd9c3+        .quad   0xc90f9b314bb05355+        .quad   0xc6e08df8ef2079b1+        .quad   0x7ef72016758cc12f+        .quad   0x5a782a5c273e9718+        .quad   0x3576c6995e4efd94+        .quad   0x0f2ed8051f237d3e+        .quad   0x044fb81d82d50a99+        .quad   0xc1df18c5a907e3d9+        .quad   0x57b3371dce4c6359+        .quad   0xca704534b201bb49+        .quad   0x7f79823f9c30dd2e++        // 2^160 * 3 * B++        .quad   0x8334d239a3b513e8+        .quad   0xc13670d4b91fa8d8+        .quad   0x12b54136f590bd33+        .quad   0x0a4e0373d784d9b4+        .quad   0x6a9c1ff068f587ba+        .quad   0x0827894e0050c8de+        .quad   0x3cbf99557ded5be7+        .quad   0x64a9b0431c06d6f0+        .quad   0x2eb3d6a15b7d2919+        .quad   0xb0b4f6a0d53a8235+        .quad   0x7156ce4389a45d47+        .quad   0x071a7d0ace18346c++        // 2^160 * 4 * B++        .quad   0xd3072daac887ba0b+        .quad   0x01262905bfa562ee+        .quad   0xcf543002c0ef768b+        .quad   0x2c3bcc7146ea7e9c+        .quad   0xcc0c355220e14431+        .quad   0x0d65950709b15141+        .quad   0x9af5621b209d5f36+        .quad   0x7c69bcf7617755d3+        .quad   0x07f0d7eb04e8295f+        .quad   0x10db18252f50f37d+        .quad   0xe951a9a3171798d7+        .quad   0x6f5a9a7322aca51d++        // 2^160 * 5 * B++        .quad   0x8ba1000c2f41c6c5+        .quad   0xc49f79c10cfefb9b+        .quad   0x4efa47703cc51c9f+        .quad   0x494e21a2e147afca+        .quad   0xe729d4eba3d944be+        .quad   0x8d9e09408078af9e+        .quad   0x4525567a47869c03+        .quad   0x02ab9680ee8d3b24+        .quad   0xefa48a85dde50d9a+        .quad   0x219a224e0fb9a249+        .quad   0xfa091f1dd91ef6d9+        .quad   0x6b5d76cbea46bb34++        // 2^160 * 6 * B++        .quad   0x8857556cec0cd994+        .quad   0x6472dc6f5cd01dba+        .quad   0xaf0169148f42b477+        .quad   0x0ae333f685277354+        .quad   0xe0f941171e782522+        .quad   0xf1e6ae74036936d3+        .quad   0x408b3ea2d0fcc746+        .quad   0x16fb869c03dd313e+        .quad   0x288e199733b60962+        .quad   0x24fc72b4d8abe133+        .quad   0x4811f7ed0991d03e+        .quad   0x3f81e38b8f70d075++        // 2^160 * 7 * B++        .quad   0x7f910fcc7ed9affe+        .quad   0x545cb8a12465874b+        .quad   0xa8397ed24b0c4704+        .quad   0x50510fc104f50993+        .quad   0x0adb7f355f17c824+        .quad   0x74b923c3d74299a4+        .quad   0xd57c3e8bcbf8eaf7+        .quad   0x0ad3e2d34cdedc3d+        .quad   0x6f0c0fc5336e249d+        .quad   0x745ede19c331cfd9+        .quad   0xf2d6fd0009eefe1c+        .quad   0x127c158bf0fa1ebe++        // 2^160 * 8 * B++        .quad   0xf6197c422e9879a2+        .quad   0xa44addd452ca3647+        .quad   0x9b413fc14b4eaccb+        .quad   0x354ef87d07ef4f68+        .quad   0xdea28fc4ae51b974+        .quad   0x1d9973d3744dfe96+        .quad   0x6240680b873848a8+        .quad   0x4ed82479d167df95+        .quad   0xfee3b52260c5d975+        .quad   0x50352efceb41b0b8+        .quad   0x8808ac30a9f6653c+        .quad   0x302d92d20539236d++        // 2^164 * 1 * B++        .quad   0x4c59023fcb3efb7c+        .quad   0x6c2fcb99c63c2a94+        .quad   0xba4190e2c3c7e084+        .quad   0x0e545daea51874d9+        .quad   0x957b8b8b0df53c30+        .quad   0x2a1c770a8e60f098+        .quad   0xbbc7a670345796de+        .quad   0x22a48f9a90c99bc9+        .quad   0x6b7dc0dc8d3fac58+        .quad   0x5497cd6ce6e42bfd+        .quad   0x542f7d1bf400d305+        .quad   0x4159f47f048d9136++        // 2^164 * 2 * B++        .quad   0x20ad660839e31e32+        .quad   0xf81e1bd58405be50+        .quad   0xf8064056f4dabc69+        .quad   0x14d23dd4ce71b975+        .quad   0x748515a8bbd24839+        .quad   0x77128347afb02b55+        .quad   0x50ba2ac649a2a17f+        .quad   0x060525513ad730f1+        .quad   0xf2398e098aa27f82+        .quad   0x6d7982bb89a1b024+        .quad   0xfa694084214dd24c+        .quad   0x71ab966fa32301c3++        // 2^164 * 3 * B++        .quad   0x2dcbd8e34ded02fc+        .quad   0x1151f3ec596f22aa+        .quad   0xbca255434e0328da+        .quad   0x35768fbe92411b22+        .quad   0xb1088a0702809955+        .quad   0x43b273ea0b43c391+        .quad   0xca9b67aefe0686ed+        .quad   0x605eecbf8335f4ed+        .quad   0x83200a656c340431+        .quad   0x9fcd71678ee59c2f+        .quad   0x75d4613f71300f8a+        .quad   0x7a912faf60f542f9++        // 2^164 * 4 * B++        .quad   0xb204585e5edc1a43+        .quad   0x9f0e16ee5897c73c+        .quad   0x5b82c0ae4e70483c+        .quad   0x624a170e2bddf9be+        .quad   0x253f4f8dfa2d5597+        .quad   0x25e49c405477130c+        .quad   0x00c052e5996b1102+        .quad   0x33cb966e33bb6c4a+        .quad   0x597028047f116909+        .quad   0x828ac41c1e564467+        .quad   0x70417dbde6217387+        .quad   0x721627aefbac4384++        // 2^164 * 5 * B++        .quad   0x97d03bc38736add5+        .quad   0x2f1422afc532b130+        .quad   0x3aa68a057101bbc4+        .quad   0x4c946cf7e74f9fa7+        .quad   0xfd3097bc410b2f22+        .quad   0xf1a05da7b5cfa844+        .quad   0x61289a1def57ca74+        .quad   0x245ea199bb821902+        .quad   0xaedca66978d477f8+        .quad   0x1898ba3c29117fe1+        .quad   0xcf73f983720cbd58+        .quad   0x67da12e6b8b56351++        // 2^164 * 6 * B++        .quad   0x7067e187b4bd6e07+        .quad   0x6e8f0203c7d1fe74+        .quad   0x93c6aa2f38c85a30+        .quad   0x76297d1f3d75a78a+        .quad   0x2b7ef3d38ec8308c+        .quad   0x828fd7ec71eb94ab+        .quad   0x807c3b36c5062abd+        .quad   0x0cb64cb831a94141+        .quad   0x3030fc33534c6378+        .quad   0xb9635c5ce541e861+        .quad   0x15d9a9bed9b2c728+        .quad   0x49233ea3f3775dcb++        // 2^164 * 7 * B++        .quad   0x629398fa8dbffc3a+        .quad   0xe12fe52dd54db455+        .quad   0xf3be11dfdaf25295+        .quad   0x628b140dce5e7b51+        .quad   0x7b3985fe1c9f249b+        .quad   0x4fd6b2d5a1233293+        .quad   0xceb345941adf4d62+        .quad   0x6987ff6f542de50c+        .quad   0x47e241428f83753c+        .quad   0x6317bebc866af997+        .quad   0xdabb5b433d1a9829+        .quad   0x074d8d245287fb2d++        // 2^164 * 8 * B++        .quad   0x8337d9cd440bfc31+        .quad   0x729d2ca1af318fd7+        .quad   0xa040a4a4772c2070+        .quad   0x46002ef03a7349be+        .quad   0x481875c6c0e31488+        .quad   0x219429b2e22034b4+        .quad   0x7223c98a31283b65+        .quad   0x3420d60b342277f9+        .quad   0xfaa23adeaffe65f7+        .quad   0x78261ed45be0764c+        .quad   0x441c0a1e2f164403+        .quad   0x5aea8e567a87d395++        // 2^168 * 1 * B++        .quad   0x7813c1a2bca4283d+        .quad   0xed62f091a1863dd9+        .quad   0xaec7bcb8c268fa86+        .quad   0x10e5d3b76f1cae4c+        .quad   0x2dbc6fb6e4e0f177+        .quad   0x04e1bf29a4bd6a93+        .quad   0x5e1966d4787af6e8+        .quad   0x0edc5f5eb426d060+        .quad   0x5453bfd653da8e67+        .quad   0xe9dc1eec24a9f641+        .quad   0xbf87263b03578a23+        .quad   0x45b46c51361cba72++        // 2^168 * 2 * B++        .quad   0xa9402abf314f7fa1+        .quad   0xe257f1dc8e8cf450+        .quad   0x1dbbd54b23a8be84+        .quad   0x2177bfa36dcb713b+        .quad   0xce9d4ddd8a7fe3e4+        .quad   0xab13645676620e30+        .quad   0x4b594f7bb30e9958+        .quad   0x5c1c0aef321229df+        .quad   0x37081bbcfa79db8f+        .quad   0x6048811ec25f59b3+        .quad   0x087a76659c832487+        .quad   0x4ae619387d8ab5bb++        // 2^168 * 3 * B++        .quad   0x8ddbf6aa5344a32e+        .quad   0x7d88eab4b41b4078+        .quad   0x5eb0eb974a130d60+        .quad   0x1a00d91b17bf3e03+        .quad   0x61117e44985bfb83+        .quad   0xfce0462a71963136+        .quad   0x83ac3448d425904b+        .quad   0x75685abe5ba43d64+        .quad   0x6e960933eb61f2b2+        .quad   0x543d0fa8c9ff4952+        .quad   0xdf7275107af66569+        .quad   0x135529b623b0e6aa++        // 2^168 * 4 * B++        .quad   0x18f0dbd7add1d518+        .quad   0x979f7888cfc11f11+        .quad   0x8732e1f07114759b+        .quad   0x79b5b81a65ca3a01+        .quad   0xf5c716bce22e83fe+        .quad   0xb42beb19e80985c1+        .quad   0xec9da63714254aae+        .quad   0x5972ea051590a613+        .quad   0x0fd4ac20dc8f7811+        .quad   0x9a9ad294ac4d4fa8+        .quad   0xc01b2d64b3360434+        .quad   0x4f7e9c95905f3bdb++        // 2^168 * 5 * B++        .quad   0x62674bbc5781302e+        .quad   0xd8520f3989addc0f+        .quad   0x8c2999ae53fbd9c6+        .quad   0x31993ad92e638e4c+        .quad   0x71c8443d355299fe+        .quad   0x8bcd3b1cdbebead7+        .quad   0x8092499ef1a49466+        .quad   0x1942eec4a144adc8+        .quad   0x7dac5319ae234992+        .quad   0x2c1b3d910cea3e92+        .quad   0x553ce494253c1122+        .quad   0x2a0a65314ef9ca75++        // 2^168 * 6 * B++        .quad   0x2db7937ff7f927c2+        .quad   0xdb741f0617d0a635+        .quad   0x5982f3a21155af76+        .quad   0x4cf6e218647c2ded+        .quad   0xcf361acd3c1c793a+        .quad   0x2f9ebcac5a35bc3b+        .quad   0x60e860e9a8cda6ab+        .quad   0x055dc39b6dea1a13+        .quad   0xb119227cc28d5bb6+        .quad   0x07e24ebc774dffab+        .quad   0xa83c78cee4a32c89+        .quad   0x121a307710aa24b6++        // 2^168 * 7 * B++        .quad   0xe4db5d5e9f034a97+        .quad   0xe153fc093034bc2d+        .quad   0x460546919551d3b1+        .quad   0x333fc76c7a40e52d+        .quad   0xd659713ec77483c9+        .quad   0x88bfe077b82b96af+        .quad   0x289e28231097bcd3+        .quad   0x527bb94a6ced3a9b+        .quad   0x563d992a995b482e+        .quad   0x3405d07c6e383801+        .quad   0x485035de2f64d8e5+        .quad   0x6b89069b20a7a9f7++        // 2^168 * 8 * B++        .quad   0x812aa0416270220d+        .quad   0x995a89faf9245b4e+        .quad   0xffadc4ce5072ef05+        .quad   0x23bc2103aa73eb73+        .quad   0x4082fa8cb5c7db77+        .quad   0x068686f8c734c155+        .quad   0x29e6c8d9f6e7a57e+        .quad   0x0473d308a7639bcf+        .quad   0xcaee792603589e05+        .quad   0x2b4b421246dcc492+        .quad   0x02a1ef74e601a94f+        .quad   0x102f73bfde04341a++        // 2^172 * 1 * B++        .quad   0xb5a2d50c7ec20d3e+        .quad   0xc64bdd6ea0c97263+        .quad   0x56e89052c1ff734d+        .quad   0x4929c6f72b2ffaba+        .quad   0x358ecba293a36247+        .quad   0xaf8f9862b268fd65+        .quad   0x412f7e9968a01c89+        .quad   0x5786f312cd754524+        .quad   0x337788ffca14032c+        .quad   0xf3921028447f1ee3+        .quad   0x8b14071f231bccad+        .quad   0x4c817b4bf2344783++        // 2^172 * 2 * B++        .quad   0x0ff853852871b96e+        .quad   0xe13e9fab60c3f1bb+        .quad   0xeefd595325344402+        .quad   0x0a37c37075b7744b+        .quad   0x413ba057a40b4484+        .quad   0xba4c2e1a4f5f6a43+        .quad   0x614ba0a5aee1d61c+        .quad   0x78a1531a8b05dc53+        .quad   0x6cbdf1703ad0562b+        .quad   0x8ecf4830c92521a3+        .quad   0xdaebd303fd8424e7+        .quad   0x72ad82a42e5ec56f++        // 2^172 * 3 * B++        .quad   0x3f9e8e35bafb65f6+        .quad   0x39d69ec8f27293a1+        .quad   0x6cb8cd958cf6a3d0+        .quad   0x1734778173adae6d+        .quad   0xc368939167024bc3+        .quad   0x8e69d16d49502fda+        .quad   0xfcf2ec3ce45f4b29+        .quad   0x065f669ea3b4cbc4+        .quad   0x8a00aec75532db4d+        .quad   0xb869a4e443e31bb1+        .quad   0x4a0f8552d3a7f515+        .quad   0x19adeb7c303d7c08++        // 2^172 * 4 * B++        .quad   0xc720cb6153ead9a3+        .quad   0x55b2c97f512b636e+        .quad   0xb1e35b5fd40290b1+        .quad   0x2fd9ccf13b530ee2+        .quad   0x9d05ba7d43c31794+        .quad   0x2470c8ff93322526+        .quad   0x8323dec816197438+        .quad   0x2852709881569b53+        .quad   0x07bd475b47f796b8+        .quad   0xd2c7b013542c8f54+        .quad   0x2dbd23f43b24f87e+        .quad   0x6551afd77b0901d6++        // 2^172 * 5 * B++        .quad   0x4546baaf54aac27f+        .quad   0xf6f66fecb2a45a28+        .quad   0x582d1b5b562bcfe8+        .quad   0x44b123f3920f785f+        .quad   0x68a24ce3a1d5c9ac+        .quad   0xbb77a33d10ff6461+        .quad   0x0f86ce4425d3166e+        .quad   0x56507c0950b9623b+        .quad   0x1206f0b7d1713e63+        .quad   0x353fe3d915bafc74+        .quad   0x194ceb970ad9d94d+        .quad   0x62fadd7cf9d03ad3++        // 2^172 * 6 * B++        .quad   0xc6b5967b5598a074+        .quad   0x5efe91ce8e493e25+        .quad   0xd4b72c4549280888+        .quad   0x20ef1149a26740c2+        .quad   0x3cd7bc61e7ce4594+        .quad   0xcd6b35a9b7dd267e+        .quad   0xa080abc84366ef27+        .quad   0x6ec7c46f59c79711+        .quad   0x2f07ad636f09a8a2+        .quad   0x8697e6ce24205e7d+        .quad   0xc0aefc05ee35a139+        .quad   0x15e80958b5f9d897++        // 2^172 * 7 * B++        .quad   0x25a5ef7d0c3e235b+        .quad   0x6c39c17fbe134ee7+        .quad   0xc774e1342dc5c327+        .quad   0x021354b892021f39+        .quad   0x4dd1ed355bb061c4+        .quad   0x42dc0cef941c0700+        .quad   0x61305dc1fd86340e+        .quad   0x56b2cc930e55a443+        .quad   0x1df79da6a6bfc5a2+        .quad   0x02f3a2749fde4369+        .quad   0xb323d9f2cda390a7+        .quad   0x7be0847b8774d363++        // 2^172 * 8 * B++        .quad   0x8c99cc5a8b3f55c3+        .quad   0x0611d7253fded2a0+        .quad   0xed2995ff36b70a36+        .quad   0x1f699a54d78a2619+        .quad   0x1466f5af5307fa11+        .quad   0x817fcc7ded6c0af2+        .quad   0x0a6de44ec3a4a3fb+        .quad   0x74071475bc927d0b+        .quad   0xe77292f373e7ea8a+        .quad   0x296537d2cb045a31+        .quad   0x1bd0653ed3274fde+        .quad   0x2f9a2c4476bd2966++        // 2^176 * 1 * B++        .quad   0xeb18b9ab7f5745c6+        .quad   0x023a8aee5787c690+        .quad   0xb72712da2df7afa9+        .quad   0x36597d25ea5c013d+        .quad   0xa2b4dae0b5511c9a+        .quad   0x7ac860292bffff06+        .quad   0x981f375df5504234+        .quad   0x3f6bd725da4ea12d+        .quad   0x734d8d7b106058ac+        .quad   0xd940579e6fc6905f+        .quad   0x6466f8f99202932d+        .quad   0x7b7ecc19da60d6d0++        // 2^176 * 2 * B++        .quad   0x78c2373c695c690d+        .quad   0xdd252e660642906e+        .quad   0x951d44444ae12bd2+        .quad   0x4235ad7601743956+        .quad   0x6dae4a51a77cfa9b+        .quad   0x82263654e7a38650+        .quad   0x09bbffcd8f2d82db+        .quad   0x03bedc661bf5caba+        .quad   0x6258cb0d078975f5+        .quad   0x492942549189f298+        .quad   0xa0cab423e2e36ee4+        .quad   0x0e7ce2b0cdf066a1++        // 2^176 * 3 * B++        .quad   0xc494643ac48c85a3+        .quad   0xfd361df43c6139ad+        .quad   0x09db17dd3ae94d48+        .quad   0x666e0a5d8fb4674a+        .quad   0xfea6fedfd94b70f9+        .quad   0xf130c051c1fcba2d+        .quad   0x4882d47e7f2fab89+        .quad   0x615256138aeceeb5+        .quad   0x2abbf64e4870cb0d+        .quad   0xcd65bcf0aa458b6b+        .quad   0x9abe4eba75e8985d+        .quad   0x7f0bc810d514dee4++        // 2^176 * 4 * B++        .quad   0xb9006ba426f4136f+        .quad   0x8d67369e57e03035+        .quad   0xcbc8dfd94f463c28+        .quad   0x0d1f8dbcf8eedbf5+        .quad   0x83ac9dad737213a0+        .quad   0x9ff6f8ba2ef72e98+        .quad   0x311e2edd43ec6957+        .quad   0x1d3a907ddec5ab75+        .quad   0xba1693313ed081dc+        .quad   0x29329fad851b3480+        .quad   0x0128013c030321cb+        .quad   0x00011b44a31bfde3++        // 2^176 * 5 * B++        .quad   0x3fdfa06c3fc66c0c+        .quad   0x5d40e38e4dd60dd2+        .quad   0x7ae38b38268e4d71+        .quad   0x3ac48d916e8357e1+        .quad   0x16561f696a0aa75c+        .quad   0xc1bf725c5852bd6a+        .quad   0x11a8dd7f9a7966ad+        .quad   0x63d988a2d2851026+        .quad   0x00120753afbd232e+        .quad   0xe92bceb8fdd8f683+        .quad   0xf81669b384e72b91+        .quad   0x33fad52b2368a066++        // 2^176 * 6 * B++        .quad   0x540649c6c5e41e16+        .quad   0x0af86430333f7735+        .quad   0xb2acfcd2f305e746+        .quad   0x16c0f429a256dca7+        .quad   0x8d2cc8d0c422cfe8+        .quad   0x072b4f7b05a13acb+        .quad   0xa3feb6e6ecf6a56f+        .quad   0x3cc355ccb90a71e2+        .quad   0xe9b69443903e9131+        .quad   0xb8a494cb7a5637ce+        .quad   0xc87cd1a4baba9244+        .quad   0x631eaf426bae7568++        // 2^176 * 7 * B++        .quad   0xb3e90410da66fe9f+        .quad   0x85dd4b526c16e5a6+        .quad   0xbc3d97611ef9bf83+        .quad   0x5599648b1ea919b5+        .quad   0x47d975b9a3700de8+        .quad   0x7280c5fbe2f80552+        .quad   0x53658f2732e45de1+        .quad   0x431f2c7f665f80b5+        .quad   0xd6026344858f7b19+        .quad   0x14ab352fa1ea514a+        .quad   0x8900441a2090a9d7+        .quad   0x7b04715f91253b26++        // 2^176 * 8 * B++        .quad   0x83edbd28acf6ae43+        .quad   0x86357c8b7d5c7ab4+        .quad   0xc0404769b7eb2c44+        .quad   0x59b37bf5c2f6583f+        .quad   0xb376c280c4e6bac6+        .quad   0x970ed3dd6d1d9b0b+        .quad   0xb09a9558450bf944+        .quad   0x48d0acfa57cde223+        .quad   0xb60f26e47dabe671+        .quad   0xf1d1a197622f3a37+        .quad   0x4208ce7ee9960394+        .quad   0x16234191336d3bdb++        // 2^180 * 1 * B++        .quad   0xf19aeac733a63aef+        .quad   0x2c7fba5d4442454e+        .quad   0x5da87aa04795e441+        .quad   0x413051e1a4e0b0f5+        .quad   0x852dd1fd3d578bbe+        .quad   0x2b65ce72c3286108+        .quad   0x658c07f4eace2273+        .quad   0x0933f804ec38ab40+        .quad   0xa7ab69798d496476+        .quad   0x8121aadefcb5abc8+        .quad   0xa5dc12ef7b539472+        .quad   0x07fd47065e45351a++        // 2^180 * 2 * B++        .quad   0xc8583c3d258d2bcd+        .quad   0x17029a4daf60b73f+        .quad   0xfa0fc9d6416a3781+        .quad   0x1c1e5fba38b3fb23+        .quad   0x304211559ae8e7c3+        .quad   0xf281b229944882a5+        .quad   0x8a13ac2e378250e4+        .quad   0x014afa0954ba48f4+        .quad   0xcb3197001bb3666c+        .quad   0x330060524bffecb9+        .quad   0x293711991a88233c+        .quad   0x291884363d4ed364++        // 2^180 * 3 * B++        .quad   0x033c6805dc4babfa+        .quad   0x2c15bf5e5596ecc1+        .quad   0x1bc70624b59b1d3b+        .quad   0x3ede9850a19f0ec5+        .quad   0xfb9d37c3bc1ab6eb+        .quad   0x02be14534d57a240+        .quad   0xf4d73415f8a5e1f6+        .quad   0x5964f4300ccc8188+        .quad   0xe44a23152d096800+        .quad   0x5c08c55970866996+        .quad   0xdf2db60a46affb6e+        .quad   0x579155c1f856fd89++        // 2^180 * 4 * B++        .quad   0x96324edd12e0c9ef+        .quad   0x468b878df2420297+        .quad   0x199a3776a4f573be+        .quad   0x1e7fbcf18e91e92a+        .quad   0xb5f16b630817e7a6+        .quad   0x808c69233c351026+        .quad   0x324a983b54cef201+        .quad   0x53c092084a485345+        .quad   0xd2d41481f1cbafbf+        .quad   0x231d2db6716174e5+        .quad   0x0b7d7656e2a55c98+        .quad   0x3e955cd82aa495f6++        // 2^180 * 5 * B++        .quad   0xe48f535e3ed15433+        .quad   0xd075692a0d7270a3+        .quad   0x40fbd21daade6387+        .quad   0x14264887cf4495f5+        .quad   0xab39f3ef61bb3a3f+        .quad   0x8eb400652eb9193e+        .quad   0xb5de6ecc38c11f74+        .quad   0x654d7e9626f3c49f+        .quad   0xe564cfdd5c7d2ceb+        .quad   0x82eeafded737ccb9+        .quad   0x6107db62d1f9b0ab+        .quad   0x0b6baac3b4358dbb++        // 2^180 * 6 * B++        .quad   0x7ae62bcb8622fe98+        .quad   0x47762256ceb891af+        .quad   0x1a5a92bcf2e406b4+        .quad   0x7d29401784e41501+        .quad   0x204abad63700a93b+        .quad   0xbe0023d3da779373+        .quad   0xd85f0346633ab709+        .quad   0x00496dc490820412+        .quad   0x1c74b88dc27e6360+        .quad   0x074854268d14850c+        .quad   0xa145fb7b3e0dcb30+        .quad   0x10843f1b43803b23++        // 2^180 * 7 * B++        .quad   0xc5f90455376276dd+        .quad   0xce59158dd7645cd9+        .quad   0x92f65d511d366b39+        .quad   0x11574b6e526996c4+        .quad   0xd56f672de324689b+        .quad   0xd1da8aedb394a981+        .quad   0xdd7b58fe9168cfed+        .quad   0x7ce246cd4d56c1e8+        .quad   0xb8f4308e7f80be53+        .quad   0x5f3cb8cb34a9d397+        .quad   0x18a961bd33cc2b2c+        .quad   0x710045fb3a9af671++        // 2^180 * 8 * B++        .quad   0x73f93d36101b95eb+        .quad   0xfaef33794f6f4486+        .quad   0x5651735f8f15e562+        .quad   0x7fa3f19058b40da1+        .quad   0xa03fc862059d699e+        .quad   0x2370cfa19a619e69+        .quad   0xc4fe3b122f823deb+        .quad   0x1d1b056fa7f0844e+        .quad   0x1bc64631e56bf61f+        .quad   0xd379ab106e5382a3+        .quad   0x4d58c57e0540168d+        .quad   0x566256628442d8e4++        // 2^184 * 1 * B++        .quad   0xb9e499def6267ff6+        .quad   0x7772ca7b742c0843+        .quad   0x23a0153fe9a4f2b1+        .quad   0x2cdfdfecd5d05006+        .quad   0xdd499cd61ff38640+        .quad   0x29cd9bc3063625a0+        .quad   0x51e2d8023dd73dc3+        .quad   0x4a25707a203b9231+        .quad   0x2ab7668a53f6ed6a+        .quad   0x304242581dd170a1+        .quad   0x4000144c3ae20161+        .quad   0x5721896d248e49fc++        // 2^184 * 2 * B++        .quad   0x0b6e5517fd181bae+        .quad   0x9022629f2bb963b4+        .quad   0x5509bce932064625+        .quad   0x578edd74f63c13da+        .quad   0x285d5091a1d0da4e+        .quad   0x4baa6fa7b5fe3e08+        .quad   0x63e5177ce19393b3+        .quad   0x03c935afc4b030fd+        .quad   0x997276c6492b0c3d+        .quad   0x47ccc2c4dfe205fc+        .quad   0xdcd29b84dd623a3c+        .quad   0x3ec2ab590288c7a2++        // 2^184 * 3 * B++        .quad   0xa1a0d27be4d87bb9+        .quad   0xa98b4deb61391aed+        .quad   0x99a0ddd073cb9b83+        .quad   0x2dd5c25a200fcace+        .quad   0xa7213a09ae32d1cb+        .quad   0x0f2b87df40f5c2d5+        .quad   0x0baea4c6e81eab29+        .quad   0x0e1bf66c6adbac5e+        .quad   0xe2abd5e9792c887e+        .quad   0x1a020018cb926d5d+        .quad   0xbfba69cdbaae5f1e+        .quad   0x730548b35ae88f5f++        // 2^184 * 4 * B++        .quad   0xc43551a3cba8b8ee+        .quad   0x65a26f1db2115f16+        .quad   0x760f4f52ab8c3850+        .quad   0x3043443b411db8ca+        .quad   0x805b094ba1d6e334+        .quad   0xbf3ef17709353f19+        .quad   0x423f06cb0622702b+        .quad   0x585a2277d87845dd+        .quad   0xa18a5f8233d48962+        .quad   0x6698c4b5ec78257f+        .quad   0xa78e6fa5373e41ff+        .quad   0x7656278950ef981f++        // 2^184 * 5 * B++        .quad   0x38c3cf59d51fc8c0+        .quad   0x9bedd2fd0506b6f2+        .quad   0x26bf109fab570e8f+        .quad   0x3f4160a8c1b846a6+        .quad   0xe17073a3ea86cf9d+        .quad   0x3a8cfbb707155fdc+        .quad   0x4853e7fc31838a8e+        .quad   0x28bbf484b613f616+        .quad   0xf2612f5c6f136c7c+        .quad   0xafead107f6dd11be+        .quad   0x527e9ad213de6f33+        .quad   0x1e79cb358188f75d++        // 2^184 * 6 * B++        .quad   0x013436c3eef7e3f1+        .quad   0x828b6a7ffe9e10f8+        .quad   0x7ff908e5bcf9defc+        .quad   0x65d7951b3a3b3831+        .quad   0x77e953d8f5e08181+        .quad   0x84a50c44299dded9+        .quad   0xdc6c2d0c864525e5+        .quad   0x478ab52d39d1f2f4+        .quad   0x66a6a4d39252d159+        .quad   0xe5dde1bc871ac807+        .quad   0xb82c6b40a6c1c96f+        .quad   0x16d87a411a212214++        // 2^184 * 7 * B++        .quad   0xb3bd7e5a42066215+        .quad   0x879be3cd0c5a24c1+        .quad   0x57c05db1d6f994b7+        .quad   0x28f87c8165f38ca6+        .quad   0xfba4d5e2d54e0583+        .quad   0xe21fafd72ebd99fa+        .quad   0x497ac2736ee9778f+        .quad   0x1f990b577a5a6dde+        .quad   0xa3344ead1be8f7d6+        .quad   0x7d1e50ebacea798f+        .quad   0x77c6569e520de052+        .quad   0x45882fe1534d6d3e++        // 2^184 * 8 * B++        .quad   0x6669345d757983d6+        .quad   0x62b6ed1117aa11a6+        .quad   0x7ddd1857985e128f+        .quad   0x688fe5b8f626f6dd+        .quad   0xd8ac9929943c6fe4+        .quad   0xb5f9f161a38392a2+        .quad   0x2699db13bec89af3+        .quad   0x7dcf843ce405f074+        .quad   0x6c90d6484a4732c0+        .quad   0xd52143fdca563299+        .quad   0xb3be28c3915dc6e1+        .quad   0x6739687e7327191b++        // 2^188 * 1 * B++        .quad   0x9f65c5ea200814cf+        .quad   0x840536e169a31740+        .quad   0x8b0ed13925c8b4ad+        .quad   0x0080dbafe936361d+        .quad   0x8ce5aad0c9cb971f+        .quad   0x1156aaa99fd54a29+        .quad   0x41f7247015af9b78+        .quad   0x1fe8cca8420f49aa+        .quad   0x72a1848f3c0cc82a+        .quad   0x38c560c2877c9e54+        .quad   0x5004e228ce554140+        .quad   0x042418a103429d71++        // 2^188 * 2 * B++        .quad   0x899dea51abf3ff5f+        .quad   0x9b93a8672fc2d8ba+        .quad   0x2c38cb97be6ebd5c+        .quad   0x114d578497263b5d+        .quad   0x58e84c6f20816247+        .quad   0x8db2b2b6e36fd793+        .quad   0x977182561d484d85+        .quad   0x0822024f8632abd7+        .quad   0xb301bb7c6b1beca3+        .quad   0x55393f6dc6eb1375+        .quad   0x910d281097b6e4eb+        .quad   0x1ad4548d9d479ea3++        // 2^188 * 3 * B++        .quad   0xcd5a7da0389a48fd+        .quad   0xb38fa4aa9a78371e+        .quad   0xc6d9761b2cdb8e6c+        .quad   0x35cf51dbc97e1443+        .quad   0xa06fe66d0fe9fed3+        .quad   0xa8733a401c587909+        .quad   0x30d14d800df98953+        .quad   0x41ce5876c7b30258+        .quad   0x59ac3bc5d670c022+        .quad   0xeae67c109b119406+        .quad   0x9798bdf0b3782fda+        .quad   0x651e3201fd074092++        // 2^188 * 4 * B++        .quad   0xd63d8483ef30c5cf+        .quad   0x4cd4b4962361cc0c+        .quad   0xee90e500a48426ac+        .quad   0x0af51d7d18c14eeb+        .quad   0xa57ba4a01efcae9e+        .quad   0x769f4beedc308a94+        .quad   0xd1f10eeb3603cb2e+        .quad   0x4099ce5e7e441278+        .quad   0x1ac98e4f8a5121e9+        .quad   0x7dae9544dbfa2fe0+        .quad   0x8320aa0dd6430df9+        .quad   0x667282652c4a2fb5++        // 2^188 * 5 * B++        .quad   0x874621f4d86bc9ab+        .quad   0xb54c7bbe56fe6fea+        .quad   0x077a24257fadc22c+        .quad   0x1ab53be419b90d39+        .quad   0xada8b6e02946db23+        .quad   0x1c0ce51a7b253ab7+        .quad   0x8448c85a66dd485b+        .quad   0x7f1fc025d0675adf+        .quad   0xd8ee1b18319ea6aa+        .quad   0x004d88083a21f0da+        .quad   0x3bd6aa1d883a4f4b+        .quad   0x4db9a3a6dfd9fd14++        // 2^188 * 6 * B++        .quad   0x8ce7b23bb99c0755+        .quad   0x35c5d6edc4f50f7a+        .quad   0x7e1e2ed2ed9b50c3+        .quad   0x36305f16e8934da1+        .quad   0xd95b00bbcbb77c68+        .quad   0xddbc846a91f17849+        .quad   0x7cf700aebe28d9b3+        .quad   0x5ce1285c85d31f3e+        .quad   0x31b6972d98b0bde8+        .quad   0x7d920706aca6de5b+        .quad   0xe67310f8908a659f+        .quad   0x50fac2a6efdf0235++        // 2^188 * 7 * B++        .quad   0xf3d3a9f35b880f5a+        .quad   0xedec050cdb03e7c2+        .quad   0xa896981ff9f0b1a2+        .quad   0x49a4ae2bac5e34a4+        .quad   0x295b1c86f6f449bc+        .quad   0x51b2e84a1f0ab4dd+        .quad   0xc001cb30aa8e551d+        .quad   0x6a28d35944f43662+        .quad   0x28bb12ee04a740e0+        .quad   0x14313bbd9bce8174+        .quad   0x72f5b5e4e8c10c40+        .quad   0x7cbfb19936adcd5b++        // 2^188 * 8 * B++        .quad   0xa311ddc26b89792d+        .quad   0x1b30b4c6da512664+        .quad   0x0ca77b4ccf150859+        .quad   0x1de443df1b009408+        .quad   0x8e793a7acc36e6e0+        .quad   0xf9fab7a37d586eed+        .quad   0x3a4f9692bae1f4e4+        .quad   0x1c14b03eff5f447e+        .quad   0x19647bd114a85291+        .quad   0x57b76cb21034d3af+        .quad   0x6329db440f9d6dfa+        .quad   0x5ef43e586a571493++        // 2^192 * 1 * B++        .quad   0xef782014385675a6+        .quad   0xa2649f30aafda9e8+        .quad   0x4cd1eb505cdfa8cb+        .quad   0x46115aba1d4dc0b3+        .quad   0xa66dcc9dc80c1ac0+        .quad   0x97a05cf41b38a436+        .quad   0xa7ebf3be95dbd7c6+        .quad   0x7da0b8f68d7e7dab+        .quad   0xd40f1953c3b5da76+        .quad   0x1dac6f7321119e9b+        .quad   0x03cc6021feb25960+        .quad   0x5a5f887e83674b4b++        // 2^192 * 2 * B++        .quad   0x8f6301cf70a13d11+        .quad   0xcfceb815350dd0c4+        .quad   0xf70297d4a4bca47e+        .quad   0x3669b656e44d1434+        .quad   0x9e9628d3a0a643b9+        .quad   0xb5c3cb00e6c32064+        .quad   0x9b5302897c2dec32+        .quad   0x43e37ae2d5d1c70c+        .quad   0x387e3f06eda6e133+        .quad   0x67301d5199a13ac0+        .quad   0xbd5ad8f836263811+        .quad   0x6a21e6cd4fd5e9be++        // 2^192 * 3 * B++        .quad   0xf1c6170a3046e65f+        .quad   0x58712a2a00d23524+        .quad   0x69dbbd3c8c82b755+        .quad   0x586bf9f1a195ff57+        .quad   0xef4129126699b2e3+        .quad   0x71d30847708d1301+        .quad   0x325432d01182b0bd+        .quad   0x45371b07001e8b36+        .quad   0xa6db088d5ef8790b+        .quad   0x5278f0dc610937e5+        .quad   0xac0349d261a16eb8+        .quad   0x0eafb03790e52179++        // 2^192 * 4 * B++        .quad   0x960555c13748042f+        .quad   0x219a41e6820baa11+        .quad   0x1c81f73873486d0c+        .quad   0x309acc675a02c661+        .quad   0x5140805e0f75ae1d+        .quad   0xec02fbe32662cc30+        .quad   0x2cebdf1eea92396d+        .quad   0x44ae3344c5435bb3+        .quad   0x9cf289b9bba543ee+        .quad   0xf3760e9d5ac97142+        .quad   0x1d82e5c64f9360aa+        .quad   0x62d5221b7f94678f++        // 2^192 * 5 * B++        .quad   0x524c299c18d0936d+        .quad   0xc86bb56c8a0c1a0c+        .quad   0xa375052edb4a8631+        .quad   0x5c0efde4bc754562+        .quad   0x7585d4263af77a3c+        .quad   0xdfae7b11fee9144d+        .quad   0xa506708059f7193d+        .quad   0x14f29a5383922037+        .quad   0xdf717edc25b2d7f5+        .quad   0x21f970db99b53040+        .quad   0xda9234b7c3ed4c62+        .quad   0x5e72365c7bee093e++        // 2^192 * 6 * B++        .quad   0x575bfc074571217f+        .quad   0x3779675d0694d95b+        .quad   0x9a0a37bbf4191e33+        .quad   0x77f1104c47b4eabc+        .quad   0x7d9339062f08b33e+        .quad   0x5b9659e5df9f32be+        .quad   0xacff3dad1f9ebdfd+        .quad   0x70b20555cb7349b7+        .quad   0xbe5113c555112c4c+        .quad   0x6688423a9a881fcd+        .quad   0x446677855e503b47+        .quad   0x0e34398f4a06404a++        // 2^192 * 7 * B++        .quad   0xb67d22d93ecebde8+        .quad   0x09b3e84127822f07+        .quad   0x743fa61fb05b6d8d+        .quad   0x5e5405368a362372+        .quad   0x18930b093e4b1928+        .quad   0x7de3e10e73f3f640+        .quad   0xf43217da73395d6f+        .quad   0x6f8aded6ca379c3e+        .quad   0xe340123dfdb7b29a+        .quad   0x487b97e1a21ab291+        .quad   0xf9967d02fde6949e+        .quad   0x780de72ec8d3de97++        // 2^192 * 8 * B++        .quad   0x0ae28545089ae7bc+        .quad   0x388ddecf1c7f4d06+        .quad   0x38ac15510a4811b8+        .quad   0x0eb28bf671928ce4+        .quad   0x671feaf300f42772+        .quad   0x8f72eb2a2a8c41aa+        .quad   0x29a17fd797373292+        .quad   0x1defc6ad32b587a6+        .quad   0xaf5bbe1aef5195a7+        .quad   0x148c1277917b15ed+        .quad   0x2991f7fb7ae5da2e+        .quad   0x467d201bf8dd2867++        // 2^196 * 1 * B++        .quad   0x7906ee72f7bd2e6b+        .quad   0x05d270d6109abf4e+        .quad   0x8d5cfe45b941a8a4+        .quad   0x44c218671c974287+        .quad   0x745f9d56296bc318+        .quad   0x993580d4d8152e65+        .quad   0xb0e5b13f5839e9ce+        .quad   0x51fc2b28d43921c0+        .quad   0x1b8fd11795e2a98c+        .quad   0x1c4e5ee12b6b6291+        .quad   0x5b30e7107424b572+        .quad   0x6e6b9de84c4f4ac6++        // 2^196 * 2 * B++        .quad   0xdff25fce4b1de151+        .quad   0xd841c0c7e11c4025+        .quad   0x2554b3c854749c87+        .quad   0x2d292459908e0df9+        .quad   0x6b7c5f10f80cb088+        .quad   0x736b54dc56e42151+        .quad   0xc2b620a5c6ef99c4+        .quad   0x5f4c802cc3a06f42+        .quad   0x9b65c8f17d0752da+        .quad   0x881ce338c77ee800+        .quad   0xc3b514f05b62f9e3+        .quad   0x66ed5dd5bec10d48++        // 2^196 * 3 * B++        .quad   0x7d38a1c20bb2089d+        .quad   0x808334e196ccd412+        .quad   0xc4a70b8c6c97d313+        .quad   0x2eacf8bc03007f20+        .quad   0xf0adf3c9cbca047d+        .quad   0x81c3b2cbf4552f6b+        .quad   0xcfda112d44735f93+        .quad   0x1f23a0c77e20048c+        .quad   0xf235467be5bc1570+        .quad   0x03d2d9020dbab38c+        .quad   0x27529aa2fcf9e09e+        .quad   0x0840bef29d34bc50++        // 2^196 * 4 * B++        .quad   0x796dfb35dc10b287+        .quad   0x27176bcd5c7ff29d+        .quad   0x7f3d43e8c7b24905+        .quad   0x0304f5a191c54276+        .quad   0xcd54e06b7f37e4eb+        .quad   0x8cc15f87f5e96cca+        .quad   0xb8248bb0d3597dce+        .quad   0x246affa06074400c+        .quad   0x37d88e68fbe45321+        .quad   0x86097548c0d75032+        .quad   0x4e9b13ef894a0d35+        .quad   0x25a83cac5753d325++        // 2^196 * 5 * B++        .quad   0x10222f48eed8165e+        .quad   0x623fc1234b8bcf3a+        .quad   0x1e145c09c221e8f0+        .quad   0x7ccfa59fca782630+        .quad   0x9f0f66293952b6e2+        .quad   0x33db5e0e0934267b+        .quad   0xff45252bd609fedc+        .quad   0x06be10f5c506e0c9+        .quad   0x1a9615a9b62a345f+        .quad   0x22050c564a52fecc+        .quad   0xa7a2788528bc0dfe+        .quad   0x5e82770a1a1ee71d++        // 2^196 * 6 * B++        .quad   0x35425183ad896a5c+        .quad   0xe8673afbe78d52f6+        .quad   0x2c66f25f92a35f64+        .quad   0x09d04f3b3b86b102+        .quad   0xe802e80a42339c74+        .quad   0x34175166a7fffae5+        .quad   0x34865d1f1c408cae+        .quad   0x2cca982c605bc5ee+        .quad   0xfd2d5d35197dbe6e+        .quad   0x207c2eea8be4ffa3+        .quad   0x2613d8db325ae918+        .quad   0x7a325d1727741d3e++        // 2^196 * 7 * B++        .quad   0xd036b9bbd16dfde2+        .quad   0xa2055757c497a829+        .quad   0x8e6cc966a7f12667+        .quad   0x4d3b1a791239c180+        .quad   0xecd27d017e2a076a+        .quad   0xd788689f1636495e+        .quad   0x52a61af0919233e5+        .quad   0x2a479df17bb1ae64+        .quad   0x9e5eee8e33db2710+        .quad   0x189854ded6c43ca5+        .quad   0xa41c22c592718138+        .quad   0x27ad5538a43a5e9b++        // 2^196 * 8 * B++        .quad   0x2746dd4b15350d61+        .quad   0xd03fcbc8ee9521b7+        .quad   0xe86e365a138672ca+        .quad   0x510e987f7e7d89e2+        .quad   0xcb5a7d638e47077c+        .quad   0x8db7536120a1c059+        .quad   0x549e1e4d8bedfdcc+        .quad   0x080153b7503b179d+        .quad   0xdda69d930a3ed3e3+        .quad   0x3d386ef1cd60a722+        .quad   0xc817ad58bdaa4ee6+        .quad   0x23be8d554fe7372a++        // 2^200 * 1 * B++        .quad   0x95fe919a74ef4fad+        .quad   0x3a827becf6a308a2+        .quad   0x964e01d309a47b01+        .quad   0x71c43c4f5ba3c797+        .quad   0xbc1ef4bd567ae7a9+        .quad   0x3f624cb2d64498bd+        .quad   0xe41064d22c1f4ec8+        .quad   0x2ef9c5a5ba384001+        .quad   0xb6fd6df6fa9e74cd+        .quad   0xf18278bce4af267a+        .quad   0x8255b3d0f1ef990e+        .quad   0x5a758ca390c5f293++        // 2^200 * 2 * B++        .quad   0xa2b72710d9462495+        .quad   0x3aa8c6d2d57d5003+        .quad   0xe3d400bfa0b487ca+        .quad   0x2dbae244b3eb72ec+        .quad   0x8ce0918b1d61dc94+        .quad   0x8ded36469a813066+        .quad   0xd4e6a829afe8aad3+        .quad   0x0a738027f639d43f+        .quad   0x980f4a2f57ffe1cc+        .quad   0x00670d0de1839843+        .quad   0x105c3f4a49fb15fd+        .quad   0x2698ca635126a69c++        // 2^200 * 3 * B++        .quad   0xe765318832b0ba78+        .quad   0x381831f7925cff8b+        .quad   0x08a81b91a0291fcc+        .quad   0x1fb43dcc49caeb07+        .quad   0x2e3d702f5e3dd90e+        .quad   0x9e3f0918e4d25386+        .quad   0x5e773ef6024da96a+        .quad   0x3c004b0c4afa3332+        .quad   0x9aa946ac06f4b82b+        .quad   0x1ca284a5a806c4f3+        .quad   0x3ed3265fc6cd4787+        .quad   0x6b43fd01cd1fd217++        // 2^200 * 4 * B++        .quad   0xc7a75d4b4697c544+        .quad   0x15fdf848df0fffbf+        .quad   0x2868b9ebaa46785a+        .quad   0x5a68d7105b52f714+        .quad   0xb5c742583e760ef3+        .quad   0x75dc52b9ee0ab990+        .quad   0xbf1427c2072b923f+        .quad   0x73420b2d6ff0d9f0+        .quad   0xaf2cf6cb9e851e06+        .quad   0x8f593913c62238c4+        .quad   0xda8ab89699fbf373+        .quad   0x3db5632fea34bc9e++        // 2^200 * 5 * B++        .quad   0xf46eee2bf75dd9d8+        .quad   0x0d17b1f6396759a5+        .quad   0x1bf2d131499e7273+        .quad   0x04321adf49d75f13+        .quad   0x2e4990b1829825d5+        .quad   0xedeaeb873e9a8991+        .quad   0xeef03d394c704af8+        .quad   0x59197ea495df2b0e+        .quad   0x04e16019e4e55aae+        .quad   0xe77b437a7e2f92e9+        .quad   0xc7ce2dc16f159aa4+        .quad   0x45eafdc1f4d70cc0++        // 2^200 * 6 * B++        .quad   0x698401858045d72b+        .quad   0x4c22faa2cf2f0651+        .quad   0x941a36656b222dc6+        .quad   0x5a5eebc80362dade+        .quad   0xb60e4624cfccb1ed+        .quad   0x59dbc292bd5c0395+        .quad   0x31a09d1ddc0481c9+        .quad   0x3f73ceea5d56d940+        .quad   0xb7a7bfd10a4e8dc6+        .quad   0xbe57007e44c9b339+        .quad   0x60c1207f1557aefa+        .quad   0x26058891266218db++        // 2^200 * 7 * B++        .quad   0x59f704a68360ff04+        .quad   0xc3d93fde7661e6f4+        .quad   0x831b2a7312873551+        .quad   0x54ad0c2e4e615d57+        .quad   0x4c818e3cc676e542+        .quad   0x5e422c9303ceccad+        .quad   0xec07cccab4129f08+        .quad   0x0dedfa10b24443b8+        .quad   0xee3b67d5b82b522a+        .quad   0x36f163469fa5c1eb+        .quad   0xa5b4d2f26ec19fd3+        .quad   0x62ecb2baa77a9408++        // 2^200 * 8 * B++        .quad   0xe5ed795261152b3d+        .quad   0x4962357d0eddd7d1+        .quad   0x7482c8d0b96b4c71+        .quad   0x2e59f919a966d8be+        .quad   0x92072836afb62874+        .quad   0x5fcd5e8579e104a5+        .quad   0x5aad01adc630a14a+        .quad   0x61913d5075663f98+        .quad   0x0dc62d361a3231da+        .quad   0xfa47583294200270+        .quad   0x02d801513f9594ce+        .quad   0x3ddbc2a131c05d5c++        // 2^204 * 1 * B++        .quad   0x3f50a50a4ffb81ef+        .quad   0xb1e035093bf420bf+        .quad   0x9baa8e1cc6aa2cd0+        .quad   0x32239861fa237a40+        .quad   0xfb735ac2004a35d1+        .quad   0x31de0f433a6607c3+        .quad   0x7b8591bfc528d599+        .quad   0x55be9a25f5bb050c+        .quad   0x0d005acd33db3dbf+        .quad   0x0111b37c80ac35e2+        .quad   0x4892d66c6f88ebeb+        .quad   0x770eadb16508fbcd++        // 2^204 * 2 * B++        .quad   0x8451f9e05e4e89dd+        .quad   0xc06302ffbc793937+        .quad   0x5d22749556a6495c+        .quad   0x09a6755ca05603fb+        .quad   0xf1d3b681a05071b9+        .quad   0x2207659a3592ff3a+        .quad   0x5f0169297881e40e+        .quad   0x16bedd0e86ba374e+        .quad   0x5ecccc4f2c2737b5+        .quad   0x43b79e0c2dccb703+        .quad   0x33e008bc4ec43df3+        .quad   0x06c1b840f07566c0++        // 2^204 * 3 * B++        .quad   0x7688a5c6a388f877+        .quad   0x02a96c14deb2b6ac+        .quad   0x64c9f3431b8c2af8+        .quad   0x3628435554a1eed6+        .quad   0x69ee9e7f9b02805c+        .quad   0xcbff828a547d1640+        .quad   0x3d93a869b2430968+        .quad   0x46b7b8cd3fe26972+        .quad   0xe9812086fe7eebe0+        .quad   0x4cba6be72f515437+        .quad   0x1d04168b516efae9+        .quad   0x5ea1391043982cb9++        // 2^204 * 4 * B++        .quad   0x49125c9cf4702ee1+        .quad   0x4520b71f8b25b32d+        .quad   0x33193026501fef7e+        .quad   0x656d8997c8d2eb2b+        .quad   0x6f2b3be4d5d3b002+        .quad   0xafec33d96a09c880+        .quad   0x035f73a4a8bcc4cc+        .quad   0x22c5b9284662198b+        .quad   0xcb58c8fe433d8939+        .quad   0x89a0cb2e6a8d7e50+        .quad   0x79ca955309fbbe5a+        .quad   0x0c626616cd7fc106++        // 2^204 * 5 * B++        .quad   0x1ffeb80a4879b61f+        .quad   0x6396726e4ada21ed+        .quad   0x33c7b093368025ba+        .quad   0x471aa0c6f3c31788+        .quad   0x8fdfc379fbf454b1+        .quad   0x45a5a970f1a4b771+        .quad   0xac921ef7bad35915+        .quad   0x42d088dca81c2192+        .quad   0x8fda0f37a0165199+        .quad   0x0adadb77c8a0e343+        .quad   0x20fbfdfcc875e820+        .quad   0x1cf2bea80c2206e7++        // 2^204 * 6 * B++        .quad   0xc2ddf1deb36202ac+        .quad   0x92a5fe09d2e27aa5+        .quad   0x7d1648f6fc09f1d3+        .quad   0x74c2cc0513bc4959+        .quad   0x982d6e1a02c0412f+        .quad   0x90fa4c83db58e8fe+        .quad   0x01c2f5bcdcb18bc0+        .quad   0x686e0c90216abc66+        .quad   0x1fadbadba54395a7+        .quad   0xb41a02a0ae0da66a+        .quad   0xbf19f598bba37c07+        .quad   0x6a12b8acde48430d++        // 2^204 * 7 * B++        .quad   0xf8daea1f39d495d9+        .quad   0x592c190e525f1dfc+        .quad   0xdb8cbd04c9991d1b+        .quad   0x11f7fda3d88f0cb7+        .quad   0x793bdd801aaeeb5f+        .quad   0x00a2a0aac1518871+        .quad   0xe8a373a31f2136b4+        .quad   0x48aab888fc91ef19+        .quad   0x041f7e925830f40e+        .quad   0x002d6ca979661c06+        .quad   0x86dc9ff92b046a2e+        .quad   0x760360928b0493d1++        // 2^204 * 8 * B++        .quad   0x21bb41c6120cf9c6+        .quad   0xeab2aa12decda59b+        .quad   0xc1a72d020aa48b34+        .quad   0x215d4d27e87d3b68+        .quad   0xb43108e5695a0b05+        .quad   0x6cb00ee8ad37a38b+        .quad   0x5edad6eea3537381+        .quad   0x3f2602d4b6dc3224+        .quad   0xc8b247b65bcaf19c+        .quad   0x49779dc3b1b2c652+        .quad   0x89a180bbd5ece2e2+        .quad   0x13f098a3cec8e039++        // 2^208 * 1 * B++        .quad   0x9adc0ff9ce5ec54b+        .quad   0x039c2a6b8c2f130d+        .quad   0x028007c7f0f89515+        .quad   0x78968314ac04b36b+        .quad   0xf3aa57a22796bb14+        .quad   0x883abab79b07da21+        .quad   0xe54be21831a0391c+        .quad   0x5ee7fb38d83205f9+        .quad   0x538dfdcb41446a8e+        .quad   0xa5acfda9434937f9+        .quad   0x46af908d263c8c78+        .quad   0x61d0633c9bca0d09++        // 2^208 * 2 * B++        .quad   0x63744935ffdb2566+        .quad   0xc5bd6b89780b68bb+        .quad   0x6f1b3280553eec03+        .quad   0x6e965fd847aed7f5+        .quad   0xada328bcf8fc73df+        .quad   0xee84695da6f037fc+        .quad   0x637fb4db38c2a909+        .quad   0x5b23ac2df8067bdc+        .quad   0x9ad2b953ee80527b+        .quad   0xe88f19aafade6d8d+        .quad   0x0e711704150e82cf+        .quad   0x79b9bbb9dd95dedc++        // 2^208 * 3 * B++        .quad   0xebb355406a3126c2+        .quad   0xd26383a868c8c393+        .quad   0x6c0c6429e5b97a82+        .quad   0x5065f158c9fd2147+        .quad   0xd1997dae8e9f7374+        .quad   0xa032a2f8cfbb0816+        .quad   0xcd6cba126d445f0a+        .quad   0x1ba811460accb834+        .quad   0x708169fb0c429954+        .quad   0xe14600acd76ecf67+        .quad   0x2eaab98a70e645ba+        .quad   0x3981f39e58a4faf2++        // 2^208 * 4 * B++        .quad   0x18fb8a7559230a93+        .quad   0x1d168f6960e6f45d+        .quad   0x3a85a94514a93cb5+        .quad   0x38dc083705acd0fd+        .quad   0xc845dfa56de66fde+        .quad   0xe152a5002c40483a+        .quad   0xe9d2e163c7b4f632+        .quad   0x30f4452edcbc1b65+        .quad   0x856d2782c5759740+        .quad   0xfa134569f99cbecc+        .quad   0x8844fc73c0ea4e71+        .quad   0x632d9a1a593f2469++        // 2^208 * 5 * B++        .quad   0xf6bb6b15b807cba6+        .quad   0x1823c7dfbc54f0d7+        .quad   0xbb1d97036e29670b+        .quad   0x0b24f48847ed4a57+        .quad   0xbf09fd11ed0c84a7+        .quad   0x63f071810d9f693a+        .quad   0x21908c2d57cf8779+        .quad   0x3a5a7df28af64ba2+        .quad   0xdcdad4be511beac7+        .quad   0xa4538075ed26ccf2+        .quad   0xe19cff9f005f9a65+        .quad   0x34fcf74475481f63++        // 2^208 * 6 * B++        .quad   0xc197e04c789767ca+        .quad   0xb8714dcb38d9467d+        .quad   0x55de888283f95fa8+        .quad   0x3d3bdc164dfa63f7+        .quad   0xa5bb1dab78cfaa98+        .quad   0x5ceda267190b72f2+        .quad   0x9309c9110a92608e+        .quad   0x0119a3042fb374b0+        .quad   0x67a2d89ce8c2177d+        .quad   0x669da5f66895d0c1+        .quad   0xf56598e5b282a2b0+        .quad   0x56c088f1ede20a73++        // 2^208 * 7 * B++        .quad   0x336d3d1110a86e17+        .quad   0xd7f388320b75b2fa+        .quad   0xf915337625072988+        .quad   0x09674c6b99108b87+        .quad   0x581b5fac24f38f02+        .quad   0xa90be9febae30cbd+        .quad   0x9a2169028acf92f0+        .quad   0x038b7ea48359038f+        .quad   0x9f4ef82199316ff8+        .quad   0x2f49d282eaa78d4f+        .quad   0x0971a5ab5aef3174+        .quad   0x6e5e31025969eb65++        // 2^208 * 8 * B++        .quad   0xb16c62f587e593fb+        .quad   0x4999eddeca5d3e71+        .quad   0xb491c1e014cc3e6d+        .quad   0x08f5114789a8dba8+        .quad   0x3304fb0e63066222+        .quad   0xfb35068987acba3f+        .quad   0xbd1924778c1061a3+        .quad   0x3058ad43d1838620+        .quad   0x323c0ffde57663d0+        .quad   0x05c3df38a22ea610+        .quad   0xbdc78abdac994f9a+        .quad   0x26549fa4efe3dc99++        // 2^212 * 1 * B++        .quad   0x738b38d787ce8f89+        .quad   0xb62658e24179a88d+        .quad   0x30738c9cf151316d+        .quad   0x49128c7f727275c9+        .quad   0x04dbbc17f75396b9+        .quad   0x69e6a2d7d2f86746+        .quad   0xc6409d99f53eabc6+        .quad   0x606175f6332e25d2+        .quad   0x4021370ef540e7dd+        .quad   0x0910d6f5a1f1d0a5+        .quad   0x4634aacd5b06b807+        .quad   0x6a39e6356944f235++        // 2^212 * 2 * B++        .quad   0x96cd5640df90f3e7+        .quad   0x6c3a760edbfa25ea+        .quad   0x24f3ef0959e33cc4+        .quad   0x42889e7e530d2e58+        .quad   0x1da1965774049e9d+        .quad   0xfbcd6ea198fe352b+        .quad   0xb1cbcd50cc5236a6+        .quad   0x1f5ec83d3f9846e2+        .quad   0x8efb23c3328ccb75+        .quad   0xaf42a207dd876ee9+        .quad   0x20fbdadc5dfae796+        .quad   0x241e246b06bf9f51++        // 2^212 * 3 * B++        .quad   0x29e68e57ad6e98f6+        .quad   0x4c9260c80b462065+        .quad   0x3f00862ea51ebb4b+        .quad   0x5bc2c77fb38d9097+        .quad   0x7eaafc9a6280bbb8+        .quad   0x22a70f12f403d809+        .quad   0x31ce40bb1bfc8d20+        .quad   0x2bc65635e8bd53ee+        .quad   0xe8d5dc9fa96bad93+        .quad   0xe58fb17dde1947dc+        .quad   0x681532ea65185fa3+        .quad   0x1fdd6c3b034a7830++        // 2^212 * 4 * B++        .quad   0x0a64e28c55dc18fe+        .quad   0xe3df9e993399ebdd+        .quad   0x79ac432370e2e652+        .quad   0x35ff7fc33ae4cc0e+        .quad   0x9c13a6a52dd8f7a9+        .quad   0x2dbb1f8c3efdcabf+        .quad   0x961e32405e08f7b5+        .quad   0x48c8a121bbe6c9e5+        .quad   0xfc415a7c59646445+        .quad   0xd224b2d7c128b615+        .quad   0x6035c9c905fbb912+        .quad   0x42d7a91274429fab++        // 2^212 * 5 * B++        .quad   0x4e6213e3eaf72ed3+        .quad   0x6794981a43acd4e7+        .quad   0xff547cde6eb508cb+        .quad   0x6fed19dd10fcb532+        .quad   0xa9a48947933da5bc+        .quad   0x4a58920ec2e979ec+        .quad   0x96d8800013e5ac4c+        .quad   0x453692d74b48b147+        .quad   0xdd775d99a8559c6f+        .quad   0xf42a2140df003e24+        .quad   0x5223e229da928a66+        .quad   0x063f46ba6d38f22c++        // 2^212 * 6 * B++        .quad   0xd2d242895f536694+        .quad   0xca33a2c542939b2c+        .quad   0x986fada6c7ddb95c+        .quad   0x5a152c042f712d5d+        .quad   0x39843cb737346921+        .quad   0xa747fb0738c89447+        .quad   0xcb8d8031a245307e+        .quad   0x67810f8e6d82f068+        .quad   0x3eeb8fbcd2287db4+        .quad   0x72c7d3a301a03e93+        .quad   0x5473e88cbd98265a+        .quad   0x7324aa515921b403++        // 2^212 * 7 * B++        .quad   0x857942f46c3cbe8e+        .quad   0xa1d364b14730c046+        .quad   0x1c8ed914d23c41bf+        .quad   0x0838e161eef6d5d2+        .quad   0xad23f6dae82354cb+        .quad   0x6962502ab6571a6d+        .quad   0x9b651636e38e37d1+        .quad   0x5cac5005d1a3312f+        .quad   0x8cc154cce9e39904+        .quad   0x5b3a040b84de6846+        .quad   0xc4d8a61cb1be5d6e+        .quad   0x40fb897bd8861f02++        // 2^212 * 8 * B++        .quad   0x84c5aa9062de37a1+        .quad   0x421da5000d1d96e1+        .quad   0x788286306a9242d9+        .quad   0x3c5e464a690d10da+        .quad   0xe57ed8475ab10761+        .quad   0x71435e206fd13746+        .quad   0x342f824ecd025632+        .quad   0x4b16281ea8791e7b+        .quad   0xd1c101d50b813381+        .quad   0xdee60f1176ee6828+        .quad   0x0cb68893383f6409+        .quad   0x6183c565f6ff484a++        // 2^216 * 1 * B++        .quad   0x741d5a461e6bf9d6+        .quad   0x2305b3fc7777a581+        .quad   0xd45574a26474d3d9+        .quad   0x1926e1dc6401e0ff+        .quad   0xdb468549af3f666e+        .quad   0xd77fcf04f14a0ea5+        .quad   0x3df23ff7a4ba0c47+        .quad   0x3a10dfe132ce3c85+        .quad   0xe07f4e8aea17cea0+        .quad   0x2fd515463a1fc1fd+        .quad   0x175322fd31f2c0f1+        .quad   0x1fa1d01d861e5d15++        // 2^216 * 2 * B++        .quad   0xcc8055947d599832+        .quad   0x1e4656da37f15520+        .quad   0x99f6f7744e059320+        .quad   0x773563bc6a75cf33+        .quad   0x38dcac00d1df94ab+        .quad   0x2e712bddd1080de9+        .quad   0x7f13e93efdd5e262+        .quad   0x73fced18ee9a01e5+        .quad   0x06b1e90863139cb3+        .quad   0xa493da67c5a03ecd+        .quad   0x8d77cec8ad638932+        .quad   0x1f426b701b864f44++        // 2^216 * 3 * B++        .quad   0xefc9264c41911c01+        .quad   0xf1a3b7b817a22c25+        .quad   0x5875da6bf30f1447+        .quad   0x4e1af5271d31b090+        .quad   0xf17e35c891a12552+        .quad   0xb76b8153575e9c76+        .quad   0xfa83406f0d9b723e+        .quad   0x0b76bb1b3fa7e438+        .quad   0x08b8c1f97f92939b+        .quad   0xbe6771cbd444ab6e+        .quad   0x22e5646399bb8017+        .quad   0x7b6dd61eb772a955++        // 2^216 * 4 * B++        .quad   0xb7adc1e850f33d92+        .quad   0x7998fa4f608cd5cf+        .quad   0xad962dbd8dfc5bdb+        .quad   0x703e9bceaf1d2f4f+        .quad   0x5730abf9ab01d2c7+        .quad   0x16fb76dc40143b18+        .quad   0x866cbe65a0cbb281+        .quad   0x53fa9b659bff6afe+        .quad   0x6c14c8e994885455+        .quad   0x843a5d6665aed4e5+        .quad   0x181bb73ebcd65af1+        .quad   0x398d93e5c4c61f50++        // 2^216 * 5 * B++        .quad   0x1c4bd16733e248f3+        .quad   0xbd9e128715bf0a5f+        .quad   0xd43f8cf0a10b0376+        .quad   0x53b09b5ddf191b13+        .quad   0xc3877c60d2e7e3f2+        .quad   0x3b34aaa030828bb1+        .quad   0x283e26e7739ef138+        .quad   0x699c9c9002c30577+        .quad   0xf306a7235946f1cc+        .quad   0x921718b5cce5d97d+        .quad   0x28cdd24781b4e975+        .quad   0x51caf30c6fcdd907++        // 2^216 * 6 * B++        .quad   0xa60ba7427674e00a+        .quad   0x630e8570a17a7bf3+        .quad   0x3758563dcf3324cc+        .quad   0x5504aa292383fdaa+        .quad   0x737af99a18ac54c7+        .quad   0x903378dcc51cb30f+        .quad   0x2b89bc334ce10cc7+        .quad   0x12ae29c189f8e99a+        .quad   0xa99ec0cb1f0d01cf+        .quad   0x0dd1efcc3a34f7ae+        .quad   0x55ca7521d09c4e22+        .quad   0x5fd14fe958eba5ea++        // 2^216 * 7 * B++        .quad   0xb5dc2ddf2845ab2c+        .quad   0x069491b10a7fe993+        .quad   0x4daaf3d64002e346+        .quad   0x093ff26e586474d1+        .quad   0x3c42fe5ebf93cb8e+        .quad   0xbedfa85136d4565f+        .quad   0xe0f0859e884220e8+        .quad   0x7dd73f960725d128+        .quad   0xb10d24fe68059829+        .quad   0x75730672dbaf23e5+        .quad   0x1367253ab457ac29+        .quad   0x2f59bcbc86b470a4++        // 2^216 * 8 * B++        .quad   0x83847d429917135f+        .quad   0xad1b911f567d03d7+        .quad   0x7e7748d9be77aad1+        .quad   0x5458b42e2e51af4a+        .quad   0x7041d560b691c301+        .quad   0x85201b3fadd7e71e+        .quad   0x16c2e16311335585+        .quad   0x2aa55e3d010828b1+        .quad   0xed5192e60c07444f+        .quad   0x42c54e2d74421d10+        .quad   0x352b4c82fdb5c864+        .quad   0x13e9004a8a768664++        // 2^220 * 1 * B++        .quad   0xcbb5b5556c032bff+        .quad   0xdf7191b729297a3a+        .quad   0xc1ff7326aded81bb+        .quad   0x71ade8bb68be03f5+        .quad   0x1e6284c5806b467c+        .quad   0xc5f6997be75d607b+        .quad   0x8b67d958b378d262+        .quad   0x3d88d66a81cd8b70+        .quad   0x8b767a93204ed789+        .quad   0x762fcacb9fa0ae2a+        .quad   0x771febcc6dce4887+        .quad   0x343062158ff05fb3++        // 2^220 * 2 * B++        .quad   0xe05da1a7e1f5bf49+        .quad   0x26457d6dd4736092+        .quad   0x77dcb07773cc32f6+        .quad   0x0a5d94969cdd5fcd+        .quad   0xfce219072a7b31b4+        .quad   0x4d7adc75aa578016+        .quad   0x0ec276a687479324+        .quad   0x6d6d9d5d1fda4beb+        .quad   0x22b1a58ae9b08183+        .quad   0xfd95d071c15c388b+        .quad   0xa9812376850a0517+        .quad   0x33384cbabb7f335e++        // 2^220 * 3 * B++        .quad   0x3c6fa2680ca2c7b5+        .quad   0x1b5082046fb64fda+        .quad   0xeb53349c5431d6de+        .quad   0x5278b38f6b879c89+        .quad   0x33bc627a26218b8d+        .quad   0xea80b21fc7a80c61+        .quad   0x9458b12b173e9ee6+        .quad   0x076247be0e2f3059+        .quad   0x52e105f61416375a+        .quad   0xec97af3685abeba4+        .quad   0x26e6b50623a67c36+        .quad   0x5cf0e856f3d4fb01++        // 2^220 * 4 * B++        .quad   0xf6c968731ae8cab4+        .quad   0x5e20741ecb4f92c5+        .quad   0x2da53be58ccdbc3e+        .quad   0x2dddfea269970df7+        .quad   0xbeaece313db342a8+        .quad   0xcba3635b842db7ee+        .quad   0xe88c6620817f13ef+        .quad   0x1b9438aa4e76d5c6+        .quad   0x8a50777e166f031a+        .quad   0x067b39f10fb7a328+        .quad   0x1925c9a6010fbd76+        .quad   0x6df9b575cc740905++        // 2^220 * 5 * B++        .quad   0x42c1192927f6bdcf+        .quad   0x8f91917a403d61ca+        .quad   0xdc1c5a668b9e1f61+        .quad   0x1596047804ec0f8d+        .quad   0xecdfc35b48cade41+        .quad   0x6a88471fb2328270+        .quad   0x740a4a2440a01b6a+        .quad   0x471e5796003b5f29+        .quad   0xda96bbb3aced37ac+        .quad   0x7a2423b5e9208cea+        .quad   0x24cc5c3038aebae2+        .quad   0x50c356afdc5dae2f++        // 2^220 * 6 * B++        .quad   0x09dcbf4341c30318+        .quad   0xeeba061183181dce+        .quad   0xc179c0cedc1e29a1+        .quad   0x1dbf7b89073f35b0+        .quad   0xcfed9cdf1b31b964+        .quad   0xf486a9858ca51af3+        .quad   0x14897265ea8c1f84+        .quad   0x784a53dd932acc00+        .quad   0x2d99f9df14fc4920+        .quad   0x76ccb60cc4499fe5+        .quad   0xa4132cbbe5cf0003+        .quad   0x3f93d82354f000ea++        // 2^220 * 7 * B++        .quad   0x8183e7689e04ce85+        .quad   0x678fb71e04465341+        .quad   0xad92058f6688edac+        .quad   0x5da350d3532b099a+        .quad   0xeaac12d179e14978+        .quad   0xff923ff3bbebff5e+        .quad   0x4af663e40663ce27+        .quad   0x0fd381a811a5f5ff+        .quad   0xf256aceca436df54+        .quad   0x108b6168ae69d6e8+        .quad   0x20d986cb6b5d036c+        .quad   0x655957b9fee2af50++        // 2^220 * 8 * B++        .quad   0xaea8b07fa902030f+        .quad   0xf88c766af463d143+        .quad   0x15b083663c787a60+        .quad   0x08eab1148267a4a8+        .quad   0xbdc1409bd002d0ac+        .quad   0x66660245b5ccd9a6+        .quad   0x82317dc4fade85ec+        .quad   0x02fe934b6ad7df0d+        .quad   0xef5cf100cfb7ea74+        .quad   0x22897633a1cb42ac+        .quad   0xd4ce0c54cef285e2+        .quad   0x30408c048a146a55++        // 2^224 * 1 * B++        .quad   0x739d8845832fcedb+        .quad   0xfa38d6c9ae6bf863+        .quad   0x32bc0dcab74ffef7+        .quad   0x73937e8814bce45e+        .quad   0xbb2e00c9193b877f+        .quad   0xece3a890e0dc506b+        .quad   0xecf3b7c036de649f+        .quad   0x5f46040898de9e1a+        .quad   0xb9037116297bf48d+        .quad   0xa9d13b22d4f06834+        .quad   0xe19715574696bdc6+        .quad   0x2cf8a4e891d5e835++        // 2^224 * 2 * B++        .quad   0x6d93fd8707110f67+        .quad   0xdd4c09d37c38b549+        .quad   0x7cb16a4cc2736a86+        .quad   0x2049bd6e58252a09+        .quad   0x2cb5487e17d06ba2+        .quad   0x24d2381c3950196b+        .quad   0xd7659c8185978a30+        .quad   0x7a6f7f2891d6a4f6+        .quad   0x7d09fd8d6a9aef49+        .quad   0xf0ee60be5b3db90b+        .quad   0x4c21b52c519ebfd4+        .quad   0x6011aadfc545941d++        // 2^224 * 3 * B++        .quad   0x5f67926dcf95f83c+        .quad   0x7c7e856171289071+        .quad   0xd6a1e7f3998f7a5b+        .quad   0x6fc5cc1b0b62f9e0+        .quad   0x63ded0c802cbf890+        .quad   0xfbd098ca0dff6aaa+        .quad   0x624d0afdb9b6ed99+        .quad   0x69ce18b779340b1e+        .quad   0xd1ef5528b29879cb+        .quad   0xdd1aae3cd47e9092+        .quad   0x127e0442189f2352+        .quad   0x15596b3ae57101f1++        // 2^224 * 4 * B++        .quad   0x462739d23f9179a2+        .quad   0xff83123197d6ddcf+        .quad   0x1307deb553f2148a+        .quad   0x0d2237687b5f4dda+        .quad   0x09ff31167e5124ca+        .quad   0x0be4158bd9c745df+        .quad   0x292b7d227ef556e5+        .quad   0x3aa4e241afb6d138+        .quad   0x2cc138bf2a3305f5+        .quad   0x48583f8fa2e926c3+        .quad   0x083ab1a25549d2eb+        .quad   0x32fcaa6e4687a36c++        // 2^224 * 5 * B++        .quad   0x7bc56e8dc57d9af5+        .quad   0x3e0bd2ed9df0bdf2+        .quad   0xaac014de22efe4a3+        .quad   0x4627e9cefebd6a5c+        .quad   0x3207a4732787ccdf+        .quad   0x17e31908f213e3f8+        .quad   0xd5b2ecd7f60d964e+        .quad   0x746f6336c2600be9+        .quad   0x3f4af345ab6c971c+        .quad   0xe288eb729943731f+        .quad   0x33596a8a0344186d+        .quad   0x7b4917007ed66293++        // 2^224 * 6 * B++        .quad   0x2d85fb5cab84b064+        .quad   0x497810d289f3bc14+        .quad   0x476adc447b15ce0c+        .quad   0x122ba376f844fd7b+        .quad   0x54341b28dd53a2dd+        .quad   0xaa17905bdf42fc3f+        .quad   0x0ff592d94dd2f8f4+        .quad   0x1d03620fe08cd37d+        .quad   0xc20232cda2b4e554+        .quad   0x9ed0fd42115d187f+        .quad   0x2eabb4be7dd479d9+        .quad   0x02c70bf52b68ec4c++        // 2^224 * 7 * B++        .quad   0xa287ec4b5d0b2fbb+        .quad   0x415c5790074882ca+        .quad   0xe044a61ec1d0815c+        .quad   0x26334f0a409ef5e0+        .quad   0xace532bf458d72e1+        .quad   0x5be768e07cb73cb5+        .quad   0x56cf7d94ee8bbde7+        .quad   0x6b0697e3feb43a03+        .quad   0xb6c8f04adf62a3c0+        .quad   0x3ef000ef076da45d+        .quad   0x9c9cb95849f0d2a9+        .quad   0x1cc37f43441b2fae++        // 2^224 * 8 * B++        .quad   0x508f565a5cc7324f+        .quad   0xd061c4c0e506a922+        .quad   0xfb18abdb5c45ac19+        .quad   0x6c6809c10380314a+        .quad   0xd76656f1c9ceaeb9+        .quad   0x1c5b15f818e5656a+        .quad   0x26e72832844c2334+        .quad   0x3a346f772f196838+        .quad   0xd2d55112e2da6ac8+        .quad   0xe9bd0331b1e851ed+        .quad   0x960746dd8ec67262+        .quad   0x05911b9f6ef7c5d0++        // 2^228 * 1 * B++        .quad   0xe9dcd756b637ff2d+        .quad   0xec4c348fc987f0c4+        .quad   0xced59285f3fbc7b7+        .quad   0x3305354793e1ea87+        .quad   0x01c18980c5fe9f94+        .quad   0xcd656769716fd5c8+        .quad   0x816045c3d195a086+        .quad   0x6e2b7f3266cc7982+        .quad   0xcc802468f7c3568f+        .quad   0x9de9ba8219974cb3+        .quad   0xabb7229cb5b81360+        .quad   0x44e2017a6fbeba62++        // 2^228 * 2 * B++        .quad   0xc4c2a74354dab774+        .quad   0x8e5d4c3c4eaf031a+        .quad   0xb76c23d242838f17+        .quad   0x749a098f68dce4ea+        .quad   0x87f82cf3b6ca6ecd+        .quad   0x580f893e18f4a0c2+        .quad   0x058930072604e557+        .quad   0x6cab6ac256d19c1d+        .quad   0xdcdfe0a02cc1de60+        .quad   0x032665ff51c5575b+        .quad   0x2c0c32f1073abeeb+        .quad   0x6a882014cd7b8606++        // 2^228 * 3 * B++        .quad   0xa52a92fea4747fb5+        .quad   0xdc12a4491fa5ab89+        .quad   0xd82da94bb847a4ce+        .quad   0x4d77edce9512cc4e+        .quad   0xd111d17caf4feb6e+        .quad   0x050bba42b33aa4a3+        .quad   0x17514c3ceeb46c30+        .quad   0x54bedb8b1bc27d75+        .quad   0x77c8e14577e2189c+        .quad   0xa3e46f6aff99c445+        .quad   0x3144dfc86d335343+        .quad   0x3a96559e7c4216a9++        // 2^228 * 4 * B++        .quad   0x12550d37f42ad2ee+        .quad   0x8b78e00498a1fbf5+        .quad   0x5d53078233894cb2+        .quad   0x02c84e4e3e498d0c+        .quad   0x4493896880baaa52+        .quad   0x4c98afc4f285940e+        .quad   0xef4aa79ba45448b6+        .quad   0x5278c510a57aae7f+        .quad   0xa54dd074294c0b94+        .quad   0xf55d46b8df18ffb6+        .quad   0xf06fecc58dae8366+        .quad   0x588657668190d165++        // 2^228 * 5 * B++        .quad   0xd47712311aef7117+        .quad   0x50343101229e92c7+        .quad   0x7a95e1849d159b97+        .quad   0x2449959b8b5d29c9+        .quad   0xbf5834f03de25cc3+        .quad   0xb887c8aed6815496+        .quad   0x5105221a9481e892+        .quad   0x6760ed19f7723f93+        .quad   0x669ba3b7ac35e160+        .quad   0x2eccf73fba842056+        .quad   0x1aec1f17c0804f07+        .quad   0x0d96bc031856f4e7++        // 2^228 * 6 * B++        .quad   0x3318be7775c52d82+        .quad   0x4cb764b554d0aab9+        .quad   0xabcf3d27cc773d91+        .quad   0x3bf4d1848123288a+        .quad   0xb1d534b0cc7505e1+        .quad   0x32cd003416c35288+        .quad   0xcb36a5800762c29d+        .quad   0x5bfe69b9237a0bf8+        .quad   0x183eab7e78a151ab+        .quad   0xbbe990c999093763+        .quad   0xff717d6e4ac7e335+        .quad   0x4c5cddb325f39f88++        // 2^228 * 7 * B++        .quad   0xc0f6b74d6190a6eb+        .quad   0x20ea81a42db8f4e4+        .quad   0xa8bd6f7d97315760+        .quad   0x33b1d60262ac7c21+        .quad   0x57750967e7a9f902+        .quad   0x2c37fdfc4f5b467e+        .quad   0xb261663a3177ba46+        .quad   0x3a375e78dc2d532b+        .quad   0x8141e72f2d4dddea+        .quad   0xe6eafe9862c607c8+        .quad   0x23c28458573cafd0+        .quad   0x46b9476f4ff97346++        // 2^228 * 8 * B++        .quad   0x0c1ffea44f901e5c+        .quad   0x2b0b6fb72184b782+        .quad   0xe587ff910114db88+        .quad   0x37130f364785a142+        .quad   0x1215505c0d58359f+        .quad   0x2a2013c7fc28c46b+        .quad   0x24a0a1af89ea664e+        .quad   0x4400b638a1130e1f+        .quad   0x3a01b76496ed19c3+        .quad   0x31e00ab0ed327230+        .quad   0x520a885783ca15b1+        .quad   0x06aab9875accbec7++        // 2^232 * 1 * B++        .quad   0xc1339983f5df0ebb+        .quad   0xc0f3758f512c4cac+        .quad   0x2cf1130a0bb398e1+        .quad   0x6b3cecf9aa270c62+        .quad   0x5349acf3512eeaef+        .quad   0x20c141d31cc1cb49+        .quad   0x24180c07a99a688d+        .quad   0x555ef9d1c64b2d17+        .quad   0x36a770ba3b73bd08+        .quad   0x624aef08a3afbf0c+        .quad   0x5737ff98b40946f2+        .quad   0x675f4de13381749d++        // 2^232 * 2 * B++        .quad   0x0e2c52036b1782fc+        .quad   0x64816c816cad83b4+        .quad   0xd0dcbdd96964073e+        .quad   0x13d99df70164c520+        .quad   0xa12ff6d93bdab31d+        .quad   0x0725d80f9d652dfe+        .quad   0x019c4ff39abe9487+        .quad   0x60f450b882cd3c43+        .quad   0x014b5ec321e5c0ca+        .quad   0x4fcb69c9d719bfa2+        .quad   0x4e5f1c18750023a0+        .quad   0x1c06de9e55edac80++        // 2^232 * 3 * B++        .quad   0x990f7ad6a33ec4e2+        .quad   0x6608f938be2ee08e+        .quad   0x9ca143c563284515+        .quad   0x4cf38a1fec2db60d+        .quad   0xffd52b40ff6d69aa+        .quad   0x34530b18dc4049bb+        .quad   0x5e4a5c2fa34d9897+        .quad   0x78096f8e7d32ba2d+        .quad   0xa0aaaa650dfa5ce7+        .quad   0xf9c49e2a48b5478c+        .quad   0x4f09cc7d7003725b+        .quad   0x373cad3a26091abe++        // 2^232 * 4 * B++        .quad   0xb294634d82c9f57c+        .quad   0x1fcbfde124934536+        .quad   0x9e9c4db3418cdb5a+        .quad   0x0040f3d9454419fc+        .quad   0xf1bea8fb89ddbbad+        .quad   0x3bcb2cbc61aeaecb+        .quad   0x8f58a7bb1f9b8d9d+        .quad   0x21547eda5112a686+        .quad   0xdefde939fd5986d3+        .quad   0xf4272c89510a380c+        .quad   0xb72ba407bb3119b9+        .quad   0x63550a334a254df4++        // 2^232 * 5 * B++        .quad   0x6507d6edb569cf37+        .quad   0x178429b00ca52ee1+        .quad   0xea7c0090eb6bd65d+        .quad   0x3eea62c7daf78f51+        .quad   0x9bba584572547b49+        .quad   0xf305c6fae2c408e0+        .quad   0x60e8fa69c734f18d+        .quad   0x39a92bafaa7d767a+        .quad   0x9d24c713e693274e+        .quad   0x5f63857768dbd375+        .quad   0x70525560eb8ab39a+        .quad   0x68436a0665c9c4cd++        // 2^232 * 6 * B++        .quad   0xbc0235e8202f3f27+        .quad   0xc75c00e264f975b0+        .quad   0x91a4e9d5a38c2416+        .quad   0x17b6e7f68ab789f9+        .quad   0x1e56d317e820107c+        .quad   0xc5266844840ae965+        .quad   0xc1e0a1c6320ffc7a+        .quad   0x5373669c91611472+        .quad   0x5d2814ab9a0e5257+        .quad   0x908f2084c9cab3fc+        .quad   0xafcaf5885b2d1eca+        .quad   0x1cb4b5a678f87d11++        // 2^232 * 7 * B++        .quad   0xb664c06b394afc6c+        .quad   0x0c88de2498da5fb1+        .quad   0x4f8d03164bcad834+        .quad   0x330bca78de7434a2+        .quad   0x6b74aa62a2a007e7+        .quad   0xf311e0b0f071c7b1+        .quad   0x5707e438000be223+        .quad   0x2dc0fd2d82ef6eac+        .quad   0x982eff841119744e+        .quad   0xf9695e962b074724+        .quad   0xc58ac14fbfc953fb+        .quad   0x3c31be1b369f1cf5++        // 2^232 * 8 * B++        .quad   0xb0f4864d08948aee+        .quad   0x07dc19ee91ba1c6f+        .quad   0x7975cdaea6aca158+        .quad   0x330b61134262d4bb+        .quad   0xc168bc93f9cb4272+        .quad   0xaeb8711fc7cedb98+        .quad   0x7f0e52aa34ac8d7a+        .quad   0x41cec1097e7d55bb+        .quad   0xf79619d7a26d808a+        .quad   0xbb1fd49e1d9e156d+        .quad   0x73d7c36cdba1df27+        .quad   0x26b44cd91f28777d++        // 2^236 * 1 * B++        .quad   0x300a9035393aa6d8+        .quad   0x2b501131a12bb1cd+        .quad   0x7b1ff677f093c222+        .quad   0x4309c1f8cab82bad+        .quad   0xaf44842db0285f37+        .quad   0x8753189047efc8df+        .quad   0x9574e091f820979a+        .quad   0x0e378d6069615579+        .quad   0xd9fa917183075a55+        .quad   0x4bdb5ad26b009fdc+        .quad   0x7829ad2cd63def0e+        .quad   0x078fc54975fd3877++        // 2^236 * 2 * B++        .quad   0x87dfbd1428878f2d+        .quad   0x134636dd1e9421a1+        .quad   0x4f17c951257341a3+        .quad   0x5df98d4bad296cb8+        .quad   0xe2004b5bb833a98a+        .quad   0x44775dec2d4c3330+        .quad   0x3aa244067eace913+        .quad   0x272630e3d58e00a9+        .quad   0xf3678fd0ecc90b54+        .quad   0xf001459b12043599+        .quad   0x26725fbc3758b89b+        .quad   0x4325e4aa73a719ae++        // 2^236 * 3 * B++        .quad   0x657dc6ef433c3493+        .quad   0x65375e9f80dbf8c3+        .quad   0x47fd2d465b372dae+        .quad   0x4966ab79796e7947+        .quad   0xed24629acf69f59d+        .quad   0x2a4a1ccedd5abbf4+        .quad   0x3535ca1f56b2d67b+        .quad   0x5d8c68d043b1b42d+        .quad   0xee332d4de3b42b0a+        .quad   0xd84e5a2b16a4601c+        .quad   0x78243877078ba3e4+        .quad   0x77ed1eb4184ee437++        // 2^236 * 4 * B++        .quad   0xbfd4e13f201839a0+        .quad   0xaeefffe23e3df161+        .quad   0xb65b04f06b5d1fe3+        .quad   0x52e085fb2b62fbc0+        .quad   0x185d43f89e92ed1a+        .quad   0xb04a1eeafe4719c6+        .quad   0x499fbe88a6f03f4f+        .quad   0x5d8b0d2f3c859bdd+        .quad   0x124079eaa54cf2ba+        .quad   0xd72465eb001b26e7+        .quad   0x6843bcfdc97af7fd+        .quad   0x0524b42b55eacd02++        // 2^236 * 5 * B++        .quad   0xfd0d5dbee45447b0+        .quad   0x6cec351a092005ee+        .quad   0x99a47844567579cb+        .quad   0x59d242a216e7fa45+        .quad   0xbc18dcad9b829eac+        .quad   0x23ae7d28b5f579d0+        .quad   0xc346122a69384233+        .quad   0x1a6110b2e7d4ac89+        .quad   0x4f833f6ae66997ac+        .quad   0x6849762a361839a4+        .quad   0x6985dec1970ab525+        .quad   0x53045e89dcb1f546++        // 2^236 * 6 * B++        .quad   0xcb8bb346d75353db+        .quad   0xfcfcb24bae511e22+        .quad   0xcba48d40d50ae6ef+        .quad   0x26e3bae5f4f7cb5d+        .quad   0x84da3cde8d45fe12+        .quad   0xbd42c218e444e2d2+        .quad   0xa85196781f7e3598+        .quad   0x7642c93f5616e2b2+        .quad   0x2323daa74595f8e4+        .quad   0xde688c8b857abeb4+        .quad   0x3fc48e961c59326e+        .quad   0x0b2e73ca15c9b8ba++        // 2^236 * 7 * B++        .quad   0xd6bb4428c17f5026+        .quad   0x9eb27223fb5a9ca7+        .quad   0xe37ba5031919c644+        .quad   0x21ce380db59a6602+        .quad   0x0e3fbfaf79c03a55+        .quad   0x3077af054cbb5acf+        .quad   0xd5c55245db3de39f+        .quad   0x015e68c1476a4af7+        .quad   0xc1d5285220066a38+        .quad   0x95603e523570aef3+        .quad   0x832659a7226b8a4d+        .quad   0x5dd689091f8eedc9++        // 2^236 * 8 * B++        .quad   0xcbac84debfd3c856+        .quad   0x1624c348b35ff244+        .quad   0xb7f88dca5d9cad07+        .quad   0x3b0e574da2c2ebe8+        .quad   0x1d022591a5313084+        .quad   0xca2d4aaed6270872+        .quad   0x86a12b852f0bfd20+        .quad   0x56e6c439ad7da748+        .quad   0xc704ff4942bdbae6+        .quad   0x5e21ade2b2de1f79+        .quad   0xe95db3f35652fad8+        .quad   0x0822b5378f08ebc1++        // 2^240 * 1 * B++        .quad   0x51f048478f387475+        .quad   0xb25dbcf49cbecb3c+        .quad   0x9aab1244d99f2055+        .quad   0x2c709e6c1c10a5d6+        .quad   0xe1b7f29362730383+        .quad   0x4b5279ffebca8a2c+        .quad   0xdafc778abfd41314+        .quad   0x7deb10149c72610f+        .quad   0xcb62af6a8766ee7a+        .quad   0x66cbec045553cd0e+        .quad   0x588001380f0be4b5+        .quad   0x08e68e9ff62ce2ea++        // 2^240 * 2 * B++        .quad   0x34ad500a4bc130ad+        .quad   0x8d38db493d0bd49c+        .quad   0xa25c3d98500a89be+        .quad   0x2f1f3f87eeba3b09+        .quad   0x2f2d09d50ab8f2f9+        .quad   0xacb9218dc55923df+        .quad   0x4a8f342673766cb9+        .quad   0x4cb13bd738f719f5+        .quad   0xf7848c75e515b64a+        .quad   0xa59501badb4a9038+        .quad   0xc20d313f3f751b50+        .quad   0x19a1e353c0ae2ee8++        // 2^240 * 3 * B++        .quad   0x7d1c7560bafa05c3+        .quad   0xb3e1a0a0c6e55e61+        .quad   0xe3529718c0d66473+        .quad   0x41546b11c20c3486+        .quad   0xb42172cdd596bdbd+        .quad   0x93e0454398eefc40+        .quad   0x9fb15347b44109b5+        .quad   0x736bd3990266ae34+        .quad   0x85532d509334b3b4+        .quad   0x46fd114b60816573+        .quad   0xcc5f5f30425c8375+        .quad   0x412295a2b87fab5c++        // 2^240 * 4 * B++        .quad   0x19c99b88f57ed6e9+        .quad   0x5393cb266df8c825+        .quad   0x5cee3213b30ad273+        .quad   0x14e153ebb52d2e34+        .quad   0x2e655261e293eac6+        .quad   0x845a92032133acdb+        .quad   0x460975cb7900996b+        .quad   0x0760bb8d195add80+        .quad   0x413e1a17cde6818a+        .quad   0x57156da9ed69a084+        .quad   0x2cbf268f46caccb1+        .quad   0x6b34be9bc33ac5f2++        // 2^240 * 5 * B++        .quad   0xf3df2f643a78c0b2+        .quad   0x4c3e971ef22e027c+        .quad   0xec7d1c5e49c1b5a3+        .quad   0x2012c18f0922dd2d+        .quad   0x11fc69656571f2d3+        .quad   0xc6c9e845530e737a+        .quad   0xe33ae7a2d4fe5035+        .quad   0x01b9c7b62e6dd30b+        .quad   0x880b55e55ac89d29+        .quad   0x1483241f45a0a763+        .quad   0x3d36efdfc2e76c1f+        .quad   0x08af5b784e4bade8++        // 2^240 * 6 * B++        .quad   0x283499dc881f2533+        .quad   0x9d0525da779323b6+        .quad   0x897addfb673441f4+        .quad   0x32b79d71163a168d+        .quad   0xe27314d289cc2c4b+        .quad   0x4be4bd11a287178d+        .quad   0x18d528d6fa3364ce+        .quad   0x6423c1d5afd9826e+        .quad   0xcc85f8d9edfcb36a+        .quad   0x22bcc28f3746e5f9+        .quad   0xe49de338f9e5d3cd+        .quad   0x480a5efbc13e2dcc++        // 2^240 * 7 * B++        .quad   0x0b51e70b01622071+        .quad   0x06b505cf8b1dafc5+        .quad   0x2c6bb061ef5aabcd+        .quad   0x47aa27600cb7bf31+        .quad   0xb6614ce442ce221f+        .quad   0x6e199dcc4c053928+        .quad   0x663fb4a4dc1cbe03+        .quad   0x24b31d47691c8e06+        .quad   0x2a541eedc015f8c3+        .quad   0x11a4fe7e7c693f7c+        .quad   0xf0af66134ea278d6+        .quad   0x545b585d14dda094++        // 2^240 * 8 * B++        .quad   0x67bf275ea0d43a0f+        .quad   0xade68e34089beebe+        .quad   0x4289134cd479e72e+        .quad   0x0f62f9c332ba5454+        .quad   0x6204e4d0e3b321e1+        .quad   0x3baa637a28ff1e95+        .quad   0x0b0ccffd5b99bd9e+        .quad   0x4d22dc3e64c8d071+        .quad   0xfcb46589d63b5f39+        .quad   0x5cae6a3f57cbcf61+        .quad   0xfebac2d2953afa05+        .quad   0x1c0fa01a36371436++        // 2^244 * 1 * B++        .quad   0xe7547449bc7cd692+        .quad   0x0f9abeaae6f73ddf+        .quad   0x4af01ca700837e29+        .quad   0x63ab1b5d3f1bc183+        .quad   0xc11ee5e854c53fae+        .quad   0x6a0b06c12b4f3ff4+        .quad   0x33540f80e0b67a72+        .quad   0x15f18fc3cd07e3ef+        .quad   0x32750763b028f48c+        .quad   0x06020740556a065f+        .quad   0xd53bd812c3495b58+        .quad   0x08706c9b865f508d++        // 2^244 * 2 * B++        .quad   0xf37ca2ab3d343dff+        .quad   0x1a8c6a2d80abc617+        .quad   0x8e49e035d4ccffca+        .quad   0x48b46beebaa1d1b9+        .quad   0xcc991b4138b41246+        .quad   0x243b9c526f9ac26b+        .quad   0xb9ef494db7cbabbd+        .quad   0x5fba433dd082ed00+        .quad   0x9c49e355c9941ad0+        .quad   0xb9734ade74498f84+        .quad   0x41c3fed066663e5c+        .quad   0x0ecfedf8e8e710b3++        // 2^244 * 3 * B++        .quad   0x76430f9f9cd470d9+        .quad   0xb62acc9ba42f6008+        .quad   0x1898297c59adad5e+        .quad   0x7789dd2db78c5080+        .quad   0x744f7463e9403762+        .quad   0xf79a8dee8dfcc9c9+        .quad   0x163a649655e4cde3+        .quad   0x3b61788db284f435+        .quad   0xb22228190d6ef6b2+        .quad   0xa94a66b246ce4bfa+        .quad   0x46c1a77a4f0b6cc7+        .quad   0x4236ccffeb7338cf++        // 2^244 * 4 * B++        .quad   0x8497404d0d55e274+        .quad   0x6c6663d9c4ad2b53+        .quad   0xec2fb0d9ada95734+        .quad   0x2617e120cdb8f73c+        .quad   0x3bd82dbfda777df6+        .quad   0x71b177cc0b98369e+        .quad   0x1d0e8463850c3699+        .quad   0x5a71945b48e2d1f1+        .quad   0x6f203dd5405b4b42+        .quad   0x327ec60410b24509+        .quad   0x9c347230ac2a8846+        .quad   0x77de29fc11ffeb6a++        // 2^244 * 5 * B++        .quad   0xb0ac57c983b778a8+        .quad   0x53cdcca9d7fe912c+        .quad   0x61c2b854ff1f59dc+        .quad   0x3a1a2cf0f0de7dac+        .quad   0x835e138fecced2ca+        .quad   0x8c9eaf13ea963b9a+        .quad   0xc95fbfc0b2160ea6+        .quad   0x575e66f3ad877892+        .quad   0x99803a27c88fcb3a+        .quad   0x345a6789275ec0b0+        .quad   0x459789d0ff6c2be5+        .quad   0x62f882651e70a8b2++        // 2^244 * 6 * B++        .quad   0x085ae2c759ff1be4+        .quad   0x149145c93b0e40b7+        .quad   0xc467e7fa7ff27379+        .quad   0x4eeecf0ad5c73a95+        .quad   0x6d822986698a19e0+        .quad   0xdc9821e174d78a71+        .quad   0x41a85f31f6cb1f47+        .quad   0x352721c2bcda9c51+        .quad   0x48329952213fc985+        .quad   0x1087cf0d368a1746+        .quad   0x8e5261b166c15aa5+        .quad   0x2d5b2d842ed24c21++        // 2^244 * 7 * B++        .quad   0x02cfebd9ebd3ded1+        .quad   0xd45b217739021974+        .quad   0x7576f813fe30a1b7+        .quad   0x5691b6f9a34ef6c2+        .quad   0x5eb7d13d196ac533+        .quad   0x377234ecdb80be2b+        .quad   0xe144cffc7cf5ae24+        .quad   0x5226bcf9c441acec+        .quad   0x79ee6c7223e5b547+        .quad   0x6f5f50768330d679+        .quad   0xed73e1e96d8adce9+        .quad   0x27c3da1e1d8ccc03++        // 2^244 * 8 * B++        .quad   0x7eb9efb23fe24c74+        .quad   0x3e50f49f1651be01+        .quad   0x3ea732dc21858dea+        .quad   0x17377bd75bb810f9+        .quad   0x28302e71630ef9f6+        .quad   0xc2d4a2032b64cee0+        .quad   0x090820304b6292be+        .quad   0x5fca747aa82adf18+        .quad   0x232a03c35c258ea5+        .quad   0x86f23a2c6bcb0cf1+        .quad   0x3dad8d0d2e442166+        .quad   0x04a8933cab76862b++        // 2^248 * 1 * B++        .quad   0xd2c604b622943dff+        .quad   0xbc8cbece44cfb3a0+        .quad   0x5d254ff397808678+        .quad   0x0fa3614f3b1ca6bf+        .quad   0x69082b0e8c936a50+        .quad   0xf9c9a035c1dac5b6+        .quad   0x6fb73e54c4dfb634+        .quad   0x4005419b1d2bc140+        .quad   0xa003febdb9be82f0+        .quad   0x2089c1af3a44ac90+        .quad   0xf8499f911954fa8e+        .quad   0x1fba218aef40ab42++        // 2^248 * 2 * B++        .quad   0xab549448fac8f53e+        .quad   0x81f6e89a7ba63741+        .quad   0x74fd6c7d6c2b5e01+        .quad   0x392e3acaa8c86e42+        .quad   0x4f3e57043e7b0194+        .quad   0xa81d3eee08daaf7f+        .quad   0xc839c6ab99dcdef1+        .quad   0x6c535d13ff7761d5+        .quad   0x4cbd34e93e8a35af+        .quad   0x2e0781445887e816+        .quad   0x19319c76f29ab0ab+        .quad   0x25e17fe4d50ac13b++        // 2^248 * 3 * B++        .quad   0x0a289bd71e04f676+        .quad   0x208e1c52d6420f95+        .quad   0x5186d8b034691fab+        .quad   0x255751442a9fb351+        .quad   0x915f7ff576f121a7+        .quad   0xc34a32272fcd87e3+        .quad   0xccba2fde4d1be526+        .quad   0x6bba828f8969899b+        .quad   0xe2d1bc6690fe3901+        .quad   0x4cb54a18a0997ad5+        .quad   0x971d6914af8460d4+        .quad   0x559d504f7f6b7be4++        // 2^248 * 4 * B++        .quad   0xa7738378b3eb54d5+        .quad   0x1d69d366a5553c7c+        .quad   0x0a26cf62f92800ba+        .quad   0x01ab12d5807e3217+        .quad   0x9c4891e7f6d266fd+        .quad   0x0744a19b0307781b+        .quad   0x88388f1d6061e23b+        .quad   0x123ea6a3354bd50e+        .quad   0x118d189041e32d96+        .quad   0xb9ede3c2d8315848+        .quad   0x1eab4271d83245d9+        .quad   0x4a3961e2c918a154++        // 2^248 * 5 * B++        .quad   0x71dc3be0f8e6bba0+        .quad   0xd6cef8347effe30a+        .quad   0xa992425fe13a476a+        .quad   0x2cd6bce3fb1db763+        .quad   0x0327d644f3233f1e+        .quad   0x499a260e34fcf016+        .quad   0x83b5a716f2dab979+        .quad   0x68aceead9bd4111f+        .quad   0x38b4c90ef3d7c210+        .quad   0x308e6e24b7ad040c+        .quad   0x3860d9f1b7e73e23+        .quad   0x595760d5b508f597++        // 2^248 * 6 * B++        .quad   0x6129bfe104aa6397+        .quad   0x8f960008a4a7fccb+        .quad   0x3f8bc0897d909458+        .quad   0x709fa43edcb291a9+        .quad   0x882acbebfd022790+        .quad   0x89af3305c4115760+        .quad   0x65f492e37d3473f4+        .quad   0x2cb2c5df54515a2b+        .quad   0xeb0a5d8c63fd2aca+        .quad   0xd22bc1662e694eff+        .quad   0x2723f36ef8cbb03a+        .quad   0x70f029ecf0c8131f++        // 2^248 * 7 * B++        .quad   0x461307b32eed3e33+        .quad   0xae042f33a45581e7+        .quad   0xc94449d3195f0366+        .quad   0x0b7d5d8a6c314858+        .quad   0x2a6aafaa5e10b0b9+        .quad   0x78f0a370ef041aa9+        .quad   0x773efb77aa3ad61f+        .quad   0x44eca5a2a74bd9e1+        .quad   0x25d448327b95d543+        .quad   0x70d38300a3340f1d+        .quad   0xde1c531c60e1c52b+        .quad   0x272224512c7de9e4++        // 2^248 * 8 * B++        .quad   0x1abc92af49c5342e+        .quad   0xffeed811b2e6fad0+        .quad   0xefa28c8dfcc84e29+        .quad   0x11b5df18a44cc543+        .quad   0xbf7bbb8a42a975fc+        .quad   0x8c5c397796ada358+        .quad   0xe27fc76fcdedaa48+        .quad   0x19735fd7f6bc20a6+        .quad   0xe3ab90d042c84266+        .quad   0xeb848e0f7f19547e+        .quad   0x2503a1d065a497b9+        .quad   0x0fef911191df895f
+ cbits/s2n/arm/edwards25519_scalarmulbase_alt.S view
@@ -0,0 +1,9477 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Scalar multiplication for the edwards25519 standard basepoint+// Input scalar[4]; output res[8]+//+// extern void edwards25519_scalarmulbase_alt+//   (uint64_t res[static 8],const uint64_t scalar[static 4]);+//+// Given a scalar n, returns point (X,Y) = n * B where B = (...,4/5) is+// the standard basepoint for the edwards25519 (Ed25519) curve.+//+// Standard ARM ABI: X0 = res, X1 = scalar+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(edwards25519_scalarmulbase_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(edwards25519_scalarmulbase_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(edwards25519_scalarmulbase_alt)++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Stable home for the input result argument during the whole body++#define res x23++// Other variables that are only needed prior to the modular inverse.++#define tab x19++#define i x20++#define bias x21++#define bf x22+#define ix x22++// Pointer-offset pairs for result and temporaries on stack with some aliasing.++#define resx res, #(0*NUMSIZE)+#define resy res, #(1*NUMSIZE)++#define scalar sp, #(0*NUMSIZE)++#define tabent sp, #(1*NUMSIZE)+#define ymx_2 sp, #(1*NUMSIZE)+#define xpy_2 sp, #(2*NUMSIZE)+#define kxy_2 sp, #(3*NUMSIZE)++#define acc sp, #(4*NUMSIZE)+#define x_1 sp, #(4*NUMSIZE)+#define y_1 sp, #(5*NUMSIZE)+#define z_1 sp, #(6*NUMSIZE)+#define w_1 sp, #(7*NUMSIZE)+#define x_3 sp, #(4*NUMSIZE)+#define y_3 sp, #(5*NUMSIZE)+#define z_3 sp, #(6*NUMSIZE)+#define w_3 sp, #(7*NUMSIZE)++#define tmpspace sp, #(8*NUMSIZE)+#define t0 sp, #(8*NUMSIZE)+#define t1 sp, #(9*NUMSIZE)+#define t2 sp, #(10*NUMSIZE)+#define t3 sp, #(11*NUMSIZE)+#define t4 sp, #(12*NUMSIZE)+#define t5 sp, #(13*NUMSIZE)++// Total size to reserve on the stack++#define NSPACE 14*NUMSIZE++// Load 64-bit immediate into a register++#define movbig(nn,n3,n2,n1,n0)                                      \+        movz    nn, n0 __LF                                            \+        movk    nn, n1, lsl #16 __LF                                   \+        movk    nn, n2, lsl #32 __LF                                   \+        movk    nn, n3, lsl #48++// Macro wrapping up the basic field operation bignum_mul_p25519_alt, only+// trivially different from a pure function call to that subroutine.++#define mul_p25519(P0,P1,P2)                    \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x7, x8, [P2] __LF                  \+        mul     x12, x3, x7 __LF                   \+        umulh   x13, x3, x7 __LF                   \+        mul     x11, x3, x8 __LF                   \+        umulh   x14, x3, x8 __LF                   \+        adds    x13, x13, x11 __LF                 \+        ldp     x9, x10, [P2+16] __LF              \+        mul     x11, x3, x9 __LF                   \+        umulh   x15, x3, x9 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x3, x10 __LF                  \+        umulh   x16, x3, x10 __LF                  \+        adcs    x15, x15, x11 __LF                 \+        adc     x16, x16, xzr __LF                 \+        ldp     x5, x6, [P1+16] __LF               \+        mul     x11, x4, x7 __LF                   \+        adds    x13, x13, x11 __LF                 \+        mul     x11, x4, x8 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x4, x9 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        mul     x11, x4, x10 __LF                  \+        adcs    x16, x16, x11 __LF                 \+        umulh   x3, x4, x10 __LF                   \+        adc     x3, x3, xzr __LF                   \+        umulh   x11, x4, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        umulh   x11, x4, x8 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        umulh   x11, x4, x9 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        adc     x3, x3, xzr __LF                   \+        mul     x11, x5, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        mul     x11, x5, x8 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        mul     x11, x5, x9 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        mul     x11, x5, x10 __LF                  \+        adcs    x3, x3, x11 __LF                   \+        umulh   x4, x5, x10 __LF                   \+        adc     x4, x4, xzr __LF                   \+        umulh   x11, x5, x7 __LF                   \+        adds    x15, x15, x11 __LF                 \+        umulh   x11, x5, x8 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        umulh   x11, x5, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        adc     x4, x4, xzr __LF                   \+        mul     x11, x6, x7 __LF                   \+        adds    x15, x15, x11 __LF                 \+        mul     x11, x6, x8 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        mul     x11, x6, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        mul     x11, x6, x10 __LF                  \+        adcs    x4, x4, x11 __LF                   \+        umulh   x5, x6, x10 __LF                   \+        adc     x5, x5, xzr __LF                   \+        umulh   x11, x6, x7 __LF                   \+        adds    x16, x16, x11 __LF                 \+        umulh   x11, x6, x8 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        umulh   x11, x6, x9 __LF                   \+        adcs    x4, x4, x11 __LF                   \+        adc     x5, x5, xzr __LF                   \+        mov     x7, #0x26 __LF                     \+        mul     x11, x7, x16 __LF                  \+        umulh   x9, x7, x16 __LF                   \+        adds    x12, x12, x11 __LF                 \+        mul     x11, x7, x3 __LF                   \+        umulh   x3, x7, x3 __LF                    \+        adcs    x13, x13, x11 __LF                 \+        mul     x11, x7, x4 __LF                   \+        umulh   x4, x7, x4 __LF                    \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x7, x5 __LF                   \+        umulh   x5, x7, x5 __LF                    \+        adcs    x15, x15, x11 __LF                 \+        cset    x16, cs __LF                       \+        adds    x15, x15, x4 __LF                  \+        adc     x16, x16, x5 __LF                  \+        cmn     x15, x15 __LF                      \+        orr     x15, x15, #0x8000000000000000 __LF \+        adc     x8, x16, x16 __LF                  \+        mov     x7, #0x13 __LF                     \+        madd    x11, x7, x8, x7 __LF               \+        adds    x12, x12, x11 __LF                 \+        adcs    x13, x13, x9 __LF                  \+        adcs    x14, x14, x3 __LF                  \+        adcs    x15, x15, xzr __LF                 \+        csel    x7, x7, xzr, cc __LF               \+        subs    x12, x12, x7 __LF                  \+        sbcs    x13, x13, xzr __LF                 \+        sbcs    x14, x14, xzr __LF                 \+        sbc     x15, x15, xzr __LF                 \+        and     x15, x15, #0x7fffffffffffffff __LF \+        stp     x12, x13, [P0] __LF                \+        stp     x14, x15, [P0+16]++// A version of multiplication that only guarantees output < 2 * p_25519.+// This basically skips the +1 and final correction in quotient estimation.++#define mul_4(P0,P1,P2)                         \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x7, x8, [P2] __LF                  \+        mul     x12, x3, x7 __LF                   \+        umulh   x13, x3, x7 __LF                   \+        mul     x11, x3, x8 __LF                   \+        umulh   x14, x3, x8 __LF                   \+        adds    x13, x13, x11 __LF                 \+        ldp     x9, x10, [P2+16] __LF              \+        mul     x11, x3, x9 __LF                   \+        umulh   x15, x3, x9 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x3, x10 __LF                  \+        umulh   x16, x3, x10 __LF                  \+        adcs    x15, x15, x11 __LF                 \+        adc     x16, x16, xzr __LF                 \+        ldp     x5, x6, [P1+16] __LF               \+        mul     x11, x4, x7 __LF                   \+        adds    x13, x13, x11 __LF                 \+        mul     x11, x4, x8 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x4, x9 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        mul     x11, x4, x10 __LF                  \+        adcs    x16, x16, x11 __LF                 \+        umulh   x3, x4, x10 __LF                   \+        adc     x3, x3, xzr __LF                   \+        umulh   x11, x4, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        umulh   x11, x4, x8 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        umulh   x11, x4, x9 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        adc     x3, x3, xzr __LF                   \+        mul     x11, x5, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        mul     x11, x5, x8 __LF                   \+        adcs    x15, x15, x11 __LF                 \+        mul     x11, x5, x9 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        mul     x11, x5, x10 __LF                  \+        adcs    x3, x3, x11 __LF                   \+        umulh   x4, x5, x10 __LF                   \+        adc     x4, x4, xzr __LF                   \+        umulh   x11, x5, x7 __LF                   \+        adds    x15, x15, x11 __LF                 \+        umulh   x11, x5, x8 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        umulh   x11, x5, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        adc     x4, x4, xzr __LF                   \+        mul     x11, x6, x7 __LF                   \+        adds    x15, x15, x11 __LF                 \+        mul     x11, x6, x8 __LF                   \+        adcs    x16, x16, x11 __LF                 \+        mul     x11, x6, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        mul     x11, x6, x10 __LF                  \+        adcs    x4, x4, x11 __LF                   \+        umulh   x5, x6, x10 __LF                   \+        adc     x5, x5, xzr __LF                   \+        umulh   x11, x6, x7 __LF                   \+        adds    x16, x16, x11 __LF                 \+        umulh   x11, x6, x8 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        umulh   x11, x6, x9 __LF                   \+        adcs    x4, x4, x11 __LF                   \+        adc     x5, x5, xzr __LF                   \+        mov     x7, #0x26 __LF                     \+        mul     x11, x7, x16 __LF                  \+        umulh   x9, x7, x16 __LF                   \+        adds    x12, x12, x11 __LF                 \+        mul     x11, x7, x3 __LF                   \+        umulh   x3, x7, x3 __LF                    \+        adcs    x13, x13, x11 __LF                 \+        mul     x11, x7, x4 __LF                   \+        umulh   x4, x7, x4 __LF                    \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x7, x5 __LF                   \+        umulh   x5, x7, x5 __LF                    \+        adcs    x15, x15, x11 __LF                 \+        cset    x16, cs __LF                       \+        adds    x15, x15, x4 __LF                  \+        adc     x16, x16, x5 __LF                  \+        cmn     x15, x15 __LF                      \+        bic     x15, x15, #0x8000000000000000 __LF \+        adc     x8, x16, x16 __LF                  \+        mov     x7, #0x13 __LF                     \+        mul     x11, x7, x8 __LF                   \+        adds    x12, x12, x11 __LF                 \+        adcs    x13, x13, x9 __LF                  \+        adcs    x14, x14, x3 __LF                  \+        adc     x15, x15, xzr __LF                 \+        stp     x12, x13, [P0] __LF                \+        stp     x14, x15, [P0+16]++// Modular subtraction with double modulus 2 * p_25519 = 2^256 - 38++#define sub_twice4(P0,P1,P2)                    \+        ldp     x5, x6, [P1] __LF                  \+        ldp     x4, x3, [P2] __LF                  \+        subs    x5, x5, x4 __LF                    \+        sbcs    x6, x6, x3 __LF                    \+        ldp     x7, x8, [P1+16] __LF               \+        ldp     x4, x3, [P2+16] __LF               \+        sbcs    x7, x7, x4 __LF                    \+        sbcs    x8, x8, x3 __LF                    \+        mov     x4, #38 __LF                       \+        csel    x3, x4, xzr, lo __LF               \+        subs    x5, x5, x3 __LF                    \+        sbcs    x6, x6, xzr __LF                   \+        sbcs    x7, x7, xzr __LF                   \+        sbc     x8, x8, xzr __LF                   \+        stp     x5, x6, [P0] __LF                  \+        stp     x7, x8, [P0+16]++// Modular addition and doubling with double modulus 2 * p_25519 = 2^256 - 38.+// This only ensures that the result fits in 4 digits, not that it is reduced+// even w.r.t. double modulus. The result is always correct modulo provided+// the sum of the inputs is < 2^256 + 2^256 - 38, so in particular provided+// at least one of them is reduced double modulo.++#define add_twice4(P0,P1,P2)                    \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x7, x8, [P2] __LF                  \+        adds    x3, x3, x7 __LF                    \+        adcs    x4, x4, x8 __LF                    \+        ldp     x5, x6, [P1+16] __LF               \+        ldp     x7, x8, [P2+16] __LF               \+        adcs    x5, x5, x7 __LF                    \+        adcs    x6, x6, x8 __LF                    \+        mov     x9, #38 __LF                       \+        csel    x9, x9, xzr, cs __LF               \+        adds    x3, x3, x9 __LF                    \+        adcs    x4, x4, xzr __LF                   \+        adcs    x5, x5, xzr __LF                   \+        adc     x6, x6, xzr __LF                   \+        stp     x3, x4, [P0] __LF                  \+        stp     x5, x6, [P0+16]++#define double_twice4(P0,P1)                    \+        ldp     x3, x4, [P1] __LF                  \+        adds    x3, x3, x3 __LF                    \+        adcs    x4, x4, x4 __LF                    \+        ldp     x5, x6, [P1+16] __LF               \+        adcs    x5, x5, x5 __LF                    \+        adcs    x6, x6, x6 __LF                    \+        mov     x9, #38 __LF                       \+        csel    x9, x9, xzr, cs __LF               \+        adds    x3, x3, x9 __LF                    \+        adcs    x4, x4, xzr __LF                   \+        adcs    x5, x5, xzr __LF                   \+        adc     x6, x6, xzr __LF                   \+        stp     x3, x4, [P0] __LF                  \+        stp     x5, x6, [P0+16]++S2N_BN_SYMBOL(edwards25519_scalarmulbase_alt):+        CFI_START++// Save regs and make room for temporaries++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_DEC_SP(NSPACE)++// Move the output pointer to a stable place++        mov     res, x0++// Copy the input scalar x to its local variable while reducing it+// modulo 2^252 + m where m = 27742317777372353535851937790883648493;+// this is the order of the basepoint so this doesn't change the result.+// First do q = floor(x/2^252) and x' = x - q * (2^252 + m), which gives+// an initial result -15 * m <= x' < 2^252++        ldp     x10, x11, [x1]+        ldp     x12, x13, [x1, #16]++        lsr     x9, x13, #60++        movbig(x0,#0x5812,#0x631a,#0x5cf5,#0xd3ed);+        movbig(x1,#0x14de,#0xf9de,#0xa2f7,#0x9cd6);++        mul     x2, x9, x0+        mul     x3, x9, x1+        umulh   x4, x9, x0+        umulh   x5, x9, x1++        adds    x3, x3, x4+        adc     x4, x5, xzr+        lsl     x5, x9, #60++        subs    x10, x10, x2+        sbcs    x11, x11, x3+        sbcs    x12, x12, x4+        sbcs    x13, x13, x5++// If x' < 0 then just directly negate it; this makes sure the+// reduced argument is strictly 0 <= x' < 2^252, but now we need+// to record (done via bit 255 of the reduced scalar, which is+// ignored in the main loop) when we negated so we can flip+// the sign of the eventual point to compensate.++        csetm   x9, cc+        adds    xzr, x9, x9+        eor     x10, x10, x9+        adcs    x10, x10, xzr+        eor     x11, x11, x9+        adcs    x11, x11, xzr+        eor     x12, x12, x9+        adcs    x12, x12, xzr+        eor     x13, x13, x9+        adc     x13, x13, xzr++        and     x9, x9, #0x8000000000000000+        orr     x13, x13, x9++// And before we store the scalar, test and reset bit 251 to+// initialize the main loop just below.++        stp     x10, x11, [scalar]+        tst     x13, #0x0800000000000000+        bic     x13, x13, #0x0800000000000000+        stp     x12, x13, [scalar+16]++// The main part of the computation is in extended-projective coordinates+// (X,Y,Z,T), representing an affine point on the edwards25519 curve+// (x,y) via x = X/Z, y = Y/Z and x * y = T/Z (so X * Y = T * Z).+// In comments B means the standard basepoint (x,4/5) =+// (0x216....f25d51a,0x6666..666658).+//+// Initialize accumulator "acc" to either 0 or 2^251 * B depending on+// bit 251 of the (reduced) scalar. That leaves bits 0..250 to handle.++#if defined(__ELF__)+        adrp    tab, S2N_BN_SYMBOL(edwards25519_scalarmulbase_alt_constant)+        add     tab, tab, :lo12:S2N_BN_SYMBOL(edwards25519_scalarmulbase_alt_constant)+#else+        adrp    tab, S2N_BN_SYMBOL(edwards25519_scalarmulbase_alt_constant)@PAGE+        add     tab, tab, S2N_BN_SYMBOL(edwards25519_scalarmulbase_alt_constant)@PAGEOFF+#endif++        ldp     x0, x1, [tab]+        ldp     x2, x3, [tab, #96]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc]++        ldp     x0, x1, [tab, #1*16]+        ldp     x2, x3, [tab, #(96+1*16)]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+1*16]++        ldp     x0, x1, [tab, #2*16]+        ldp     x2, x3, [tab, #(96+2*16)]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+2*16]++        ldp     x0, x1, [tab, #3*16]+        ldp     x2, x3, [tab, #(96+3*16)]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+3*16]++        mov     x0, #1+        stp     x0, xzr, [acc+4*16]+        stp     xzr, xzr, [acc+5*16]++        ldp     x0, x1, [tab, #4*16]+        ldp     x2, x3, [tab, #(96+4*16)]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+6*16]++        ldp     x0, x1, [tab, #5*16]+        ldp     x2, x3, [tab, #(96+5*16)]+        csel    x0, x0, x2, eq+        csel    x1, x1, x3, eq+        stp     x0, x1, [acc+7*16]++// The counter "i" tracks the bit position for which the scalar has+// already been absorbed, starting at 0 and going up in chunks of 4.+//+// The pointer "tab" points at the current block of the table for+// multiples (2^i * j) * B at the current bit position i; 1 <= j <= 8.+//+// The bias is always either 0 and 1 and needs to be added to the+// partially processed scalar implicitly. This is used to absorb 4 bits+// of scalar per iteration from 3-bit table indexing by exploiting+// negation: (16 * h + l) * B = (16 * (h + 1) - (16 - l)) * B is used+// when l >= 9. Note that we can't have any bias left over at the+// end because we made sure bit 251 is clear in the reduced scalar.++        mov     i, 0+        add     tab, tab, #192+        mov     bias, xzr++// Start of the main loop, repeated 63 times for i = 0, 4, 8, ..., 248++Ledwards25519_scalarmulbase_alt_scalarloop:++// Look at the next 4-bit field "bf", adding the previous bias as well.+// Choose the table index "ix" as bf when bf <= 8 and 16 - bf for bf >= 9,+// setting the bias to 1 for the next iteration in the latter case.++        lsr     x0, i, #6+        ldr     x2, [sp, x0, lsl #3]    // Exploiting scalar = sp exactly+        lsr     x2, x2, i+        and     x2, x2, #15+        add     bf, x2, bias++        cmp     bf, 9+        cset    bias, cs++        mov     x0, 16+        sub     x0, x0, bf+        cmp     bias, xzr+        csel    ix, x0, bf, ne++// Perform constant-time lookup in the table to get element number "ix".+// The table entry for the affine point (x,y) is actually a triple+// (y - x,x + y,2 * d * x * y) to precompute parts of the addition.+// Note that "ix" can be 0, so we set up the appropriate identity first.++        mov     x0, #1+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, #1+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        mov     x9, xzr+        mov     x10, xzr+        mov     x11, xzr++        cmp     ix, #1+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #2+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #3+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #4+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #5+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #6+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #7+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++        cmp     ix, #8+        ldp     x12, x13, [tab]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x12, x13, [tab, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x12, x13, [tab, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x12, x13, [tab, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x12, x13, [tab, #64]+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x12, x13, [tab, #80]+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        add     tab, tab, #96++// We now have the triple from the table in registers as follows+//+//      [x3;x2;x1;x0] = y - x+//      [x7;x6;x5;x4] = x + y+//      [x11;x10;x9;x8] = 2 * d * x * y+//+// In case bias = 1 we need to negate this. For Edwards curves+// -(x,y) = (-x,y), i.e. we need to negate the x coordinate.+// In this processed encoding, that amounts to swapping the+// first two fields and negating the third.+//+// The optional negation here also pretends bias = 0 whenever+// ix = 0 so that it doesn't need to handle the case of zero+// inputs, since no non-trivial table entries are zero. Note+// that in the zero case the whole negation is trivial, and+// so indeed is the swapping.++        cmp     bias, #0++        csel    x12, x0, x4, eq+        csel    x13, x1, x5, eq+        csel    x14, x2, x6, eq+        csel    x15, x3, x7, eq+        stp     x12, x13, [tabent]+        stp     x14, x15, [tabent+16]++        csel    x12, x0, x4, ne+        csel    x13, x1, x5, ne+        csel    x14, x2, x6, ne+        csel    x15, x3, x7, ne+        stp     x12, x13, [tabent+32]+        stp     x14, x15, [tabent+48]++        mov     x0, #-19+        subs    x0, x0, x8+        mov     x2, #-1+        sbcs    x1, x2, x9+        sbcs    x2, x2, x10+        mov     x3, #0x7FFFFFFFFFFFFFFF+        sbc     x3, x3, x11++        cmp     ix, xzr+        ccmp    bias, xzr, #4, ne++        csel    x0, x0, x8, ne+        csel    x1, x1, x9, ne+        stp     x0, x1, [tabent+64]+        csel    x2, x2, x10, ne+        csel    x3, x3, x11, ne+        stp     x2, x3, [tabent+80]++// Extended-projective and precomputed mixed addition.+// This is effectively the same as calling the standalone+// function edwards25519_pepadd_alt(acc,acc,tabent), but we+// only retain slightly weaker normalization < 2 * p_25519+// throughout the inner loop, so the computation is+// slightly different, and faster overall.++        double_twice4(t0,z_1)+        sub_twice4(t1,y_1,x_1)+        add_twice4(t2,y_1,x_1)+        mul_4(t3,w_1,kxy_2)+        mul_4(t1,t1,ymx_2)+        mul_4(t2,t2,xpy_2)+        sub_twice4(t4,t0,t3)+        add_twice4(t0,t0,t3)+        sub_twice4(t5,t2,t1)+        add_twice4(t1,t2,t1)+        mul_4(z_3,t4,t0)+        mul_4(x_3,t5,t4)+        mul_4(y_3,t0,t1)+        mul_4(w_3,t5,t1)++// End of the main loop; move on by 4 bits.++        add     i, i, 4+        cmp     i, 252+        bcc     Ledwards25519_scalarmulbase_alt_scalarloop++// Insert the optional negation of the projective X coordinate, and+// so by extension the final affine x coordinate x = X/Z and thus+// the point P = (x,y). We only know X < 2 * p_25519, so we do the+// negation as 2 * p_25519 - X to keep it nonnegative. From this+// point on we don't need any normalization of the coordinates+// except for making sure that they fit in 4 digits.++        ldp     x0, x1, [x_3]+        ldp     x2, x3, [x_3+16]+        mov     x4, #0xffffffffffffffda+        subs    x4, x4, x0+        mov     x7, #0xffffffffffffffff+        sbcs    x5, x7, x1+        sbcs    x6, x7, x2+        sbc     x7, x7, x3+        ldr     x10, [scalar+24]+        tst     x10, #0x8000000000000000+        csel    x0, x4, x0, ne+        csel    x1, x5, x1, ne+        csel    x2, x6, x2, ne+        csel    x3, x7, x3, ne+        stp     x0, x1, [x_3]+        stp     x2, x3, [x_3+16]++// Now we need to map out of the extended-projective representation+// (X,Y,Z,W) back to the affine form (x,y) = (X/Z,Y/Z). This means+// first calling the modular inverse to get w_3 = 1/z_3.++        add     x0, w_3+        add     x1, z_3++// Inline copy of bignum_inv_p25519, identical except for stripping out+// the prologue and epilogue saving and restoring registers and making+// and reclaiming room on the stack. For more details and explanations see+// "arm/curve25519/bignum_inv_p25519.S". Note that the stack it uses for+// its own temporaries is 128 bytes, so it has no effect on variables+// that are needed in the rest of our computation here: res, w_3, x_3+// and y_3.++        mov     x20, x0+        mov     x10, #0xffffffffffffffed+        mov     x11, #0xffffffffffffffff+        stp     x10, x11, [sp]+        mov     x12, #0x7fffffffffffffff+        stp     x11, x12, [sp, #16]+        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]+        mov     x7, #0x13+        lsr     x6, x5, #63+        madd    x6, x7, x6, x7+        adds    x2, x2, x6+        adcs    x3, x3, xzr+        adcs    x4, x4, xzr+        orr     x5, x5, #0x8000000000000000+        adcs    x5, x5, xzr+        csel    x6, x7, xzr, cc+        subs    x2, x2, x6+        sbcs    x3, x3, xzr+        sbcs    x4, x4, xzr+        sbc     x5, x5, xzr+        and     x5, x5, #0x7fffffffffffffff+        stp     x2, x3, [sp, #32]+        stp     x4, x5, [sp, #48]+        stp     xzr, xzr, [sp, #64]+        stp     xzr, xzr, [sp, #80]+        mov     x10, #0x2099+        movk    x10, #0x7502, lsl #16+        movk    x10, #0x9e23, lsl #32+        movk    x10, #0xa0f9, lsl #48+        mov     x11, #0x2595+        movk    x11, #0x1d13, lsl #16+        movk    x11, #0x8f3f, lsl #32+        movk    x11, #0xa8c6, lsl #48+        mov     x12, #0x5242+        movk    x12, #0x5ac, lsl #16+        movk    x12, #0x8938, lsl #32+        movk    x12, #0x6c6c, lsl #48+        mov     x13, #0x615+        movk    x13, #0x4177, lsl #16+        movk    x13, #0x8b2, lsl #32+        movk    x13, #0x2765, lsl #48+        stp     x10, x11, [sp, #96]+        stp     x12, x13, [sp, #112]+        mov     x21, #0xa+        mov     x22, #0x1+        b       Ledwards25519_scalarmulbase_alt_invmidloop+Ledwards25519_scalarmulbase_alt_invloop:+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        and     x0, x12, x16+        and     x1, x13, x17+        add     x19, x0, x1+        ldr     x7, [sp]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #32]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x7, [sp, #8]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #40]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [sp]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [sp, #32]+        ldr     x7, [sp, #16]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #48]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [sp, #8]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [sp, #40]+        ldr     x7, [sp, #24]+        eor     x1, x7, x14+        asr     x3, x1, #63+        and     x3, x3, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #56]+        eor     x1, x8, x15+        asr     x0, x1, #63+        and     x0, x0, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x5, x6, #59+        str     x6, [sp, #16]+        extr    x5, x3, x5, #59+        str     x5, [sp, #24]+        eor     x1, x7, x16+        asr     x5, x1, #63+        and     x5, x5, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        asr     x0, x1, #63+        and     x0, x0, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x4, x2, x4, #59+        str     x4, [sp, #48]+        extr    x2, x5, x2, #59+        str     x2, [sp, #56]+        ldr     x7, [sp, #64]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #96]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #64]+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        str     x5, [sp, #96]+        adc     x3, x3, x1+        ldr     x7, [sp, #72]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #104]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #72]+        adc     x6, x6, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        str     x3, [sp, #104]+        adc     x4, x4, x1+        ldr     x7, [sp, #80]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #112]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #80]+        adc     x5, x5, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        str     x4, [sp, #112]+        adc     x2, x2, x1+        ldr     x7, [sp, #88]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #120]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x3, x5, #63+        ldp     x0, x1, [sp, #64]+        add     x6, x6, x3, asr #63+        mov     x3, #0x13+        mul     x4, x6, x3+        add     x5, x5, x6, lsl #63+        smulh   x3, x6, x3+        ldr     x6, [sp, #80]+        adds    x0, x0, x4+        adcs    x1, x1, x3+        asr     x3, x3, #63+        adcs    x6, x6, x3+        adc     x5, x5, x3+        stp     x0, x1, [sp, #64]+        stp     x6, x5, [sp, #80]+        eor     x1, x7, x16+        and     x5, x16, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        and     x0, x17, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x6, x5, x2, #63+        ldp     x0, x1, [sp, #96]+        add     x6, x6, x5, asr #63+        mov     x5, #0x13+        mul     x4, x6, x5+        add     x2, x2, x6, lsl #63+        smulh   x5, x6, x5+        ldr     x3, [sp, #112]+        adds    x0, x0, x4+        adcs    x1, x1, x5+        asr     x5, x5, #63+        adcs    x3, x3, x5+        adc     x2, x2, x5+        stp     x0, x1, [sp, #96]+        stp     x3, x2, [sp, #112]+Ledwards25519_scalarmulbase_alt_invmidloop:+        mov     x1, x22+        ldr     x2, [sp]+        ldr     x3, [sp, #32]+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x8, x4, #0x100, lsl #12+        sbfx    x8, x8, #21, #21+        mov     x11, #0x100000+        add     x11, x11, x11, lsl #21+        add     x9, x4, x11+        asr     x9, x9, #42+        add     x10, x5, #0x100, lsl #12+        sbfx    x10, x10, #21, #21+        add     x11, x5, x11+        asr     x11, x11, #42+        mul     x6, x8, x2+        mul     x7, x9, x3+        mul     x2, x10, x2+        mul     x3, x11, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #21, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #42+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #21, #21+        add     x15, x5, x15+        asr     x15, x15, #42+        mul     x6, x12, x2+        mul     x7, x13, x3+        mul     x2, x14, x2+        mul     x3, x15, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        mul     x2, x12, x8+        mul     x3, x12, x9+        mul     x6, x14, x8+        mul     x7, x14, x9+        madd    x8, x13, x10, x2+        madd    x9, x13, x11, x3+        madd    x16, x15, x10, x6+        madd    x17, x15, x11, x7+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #22, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #43+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #22, #21+        add     x15, x5, x15+        asr     x15, x15, #43+        mneg    x2, x12, x8+        mneg    x3, x12, x9+        mneg    x4, x14, x8+        mneg    x5, x14, x9+        msub    x10, x13, x16, x2+        msub    x11, x13, x17, x3+        msub    x12, x15, x16, x4+        msub    x13, x15, x17, x5+        mov     x22, x1+        subs    x21, x21, #0x1+        b.ne    Ledwards25519_scalarmulbase_alt_invloop+        ldr     x0, [sp]+        ldr     x1, [sp, #32]+        mul     x0, x0, x10+        madd    x1, x1, x11, x0+        asr     x0, x1, #63+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        eor     x14, x14, x0+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        eor     x15, x15, x0+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        eor     x16, x16, x0+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        eor     x17, x17, x0+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        ldr     x7, [sp, #64]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #96]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #64]+        adc     x2, x2, x1+        ldr     x7, [sp, #72]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #104]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #72]+        adc     x6, x6, x1+        ldr     x7, [sp, #80]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #112]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #80]+        adc     x5, x5, x1+        ldr     x7, [sp, #88]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #120]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x3, x5, #63+        ldp     x0, x1, [sp, #64]+        tst     x3, x3+        cinc    x6, x6, pl+        mov     x3, #0x13+        mul     x4, x6, x3+        add     x5, x5, x6, lsl #63+        smulh   x6, x6, x3+        ldr     x2, [sp, #80]+        adds    x0, x0, x4+        adcs    x1, x1, x6+        asr     x6, x6, #63+        adcs    x2, x2, x6+        adcs    x5, x5, x6+        csel    x3, x3, xzr, mi+        subs    x0, x0, x3+        sbcs    x1, x1, xzr+        sbcs    x2, x2, xzr+        sbc     x5, x5, xzr+        and     x5, x5, #0x7fffffffffffffff+        mov     x4, x20+        stp     x0, x1, [x4]+        stp     x2, x5, [x4, #16]++// The final result is x = X * inv(Z), y = Y * inv(Z).+// These are the only operations in the whole computation that+// fully reduce modulo p_25519 since now we want the canonical+// answer as output.++        mul_p25519(resx,x_3,w_3)+        mul_p25519(resy,y_3,w_3)++// Restore stack and registers++        CFI_INC_SP(NSPACE)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)++        CFI_RET++S2N_BN_SIZE_DIRECTIVE(edwards25519_scalarmulbase_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif++// ****************************************************************************+// The precomputed data (all read-only).+// ****************************************************************************++#if defined(__ELF__)+.section .rodata+.type S2N_BN_SYMBOL(edwards25519_scalarmulbase_alt_constant), %object+.size S2N_BN_SYMBOL(edwards25519_scalarmulbase_alt_constant), 48576+#elif defined(__APPLE__)+.const_data+#endif++S2N_BN_SYMBOL(edwards25519_scalarmulbase_alt_constant):++// 0 * B = 0 and 2^251 * B in extended-projective coordinates+// but with Z = 1 assumed and hence left out, so they are (X,Y,T) only.++        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000001+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000++        .quad   0x525f946d7c7220e7+        .quad   0x4636b0b2f1e35444+        .quad   0x796e9d70e892ae0f+        .quad   0x03dec05fa937adb1+        .quad   0x6d1c271cc6375515+        .quad   0x462588c4a4ca4f14+        .quad   0x691129fee55afc39+        .quad   0x15949f784d8472f5+        .quad   0xbd89e510afad0049+        .quad   0x4d1f08c073b9860e+        .quad   0x07716e8b2d00af9d+        .quad   0x70d685f68f859714++// Precomputed table of multiples of generator for edwards25519+// all in precomputed extended-projective (y-x,x+y,2*d*x*y) triples.++        // 2^0 * 1 * G++        .quad   0x9d103905d740913e+        .quad   0xfd399f05d140beb3+        .quad   0xa5c18434688f8a09+        .quad   0x44fd2f9298f81267+        .quad   0x2fbc93c6f58c3b85+        .quad   0xcf932dc6fb8c0e19+        .quad   0x270b4898643d42c2+        .quad   0x07cf9d3a33d4ba65+        .quad   0xabc91205877aaa68+        .quad   0x26d9e823ccaac49e+        .quad   0x5a1b7dcbdd43598c+        .quad   0x6f117b689f0c65a8++        // 2^0 * 2 * G++        .quad   0x8a99a56042b4d5a8+        .quad   0x8f2b810c4e60acf6+        .quad   0xe09e236bb16e37aa+        .quad   0x6bb595a669c92555+        .quad   0x9224e7fc933c71d7+        .quad   0x9f469d967a0ff5b5+        .quad   0x5aa69a65e1d60702+        .quad   0x590c063fa87d2e2e+        .quad   0x43faa8b3a59b7a5f+        .quad   0x36c16bdd5d9acf78+        .quad   0x500fa0840b3d6a31+        .quad   0x701af5b13ea50b73++        // 2^0 * 3 * G++        .quad   0x56611fe8a4fcd265+        .quad   0x3bd353fde5c1ba7d+        .quad   0x8131f31a214bd6bd+        .quad   0x2ab91587555bda62+        .quad   0xaf25b0a84cee9730+        .quad   0x025a8430e8864b8a+        .quad   0xc11b50029f016732+        .quad   0x7a164e1b9a80f8f4+        .quad   0x14ae933f0dd0d889+        .quad   0x589423221c35da62+        .quad   0xd170e5458cf2db4c+        .quad   0x5a2826af12b9b4c6++        // 2^0 * 4 * G++        .quad   0x95fe050a056818bf+        .quad   0x327e89715660faa9+        .quad   0xc3e8e3cd06a05073+        .quad   0x27933f4c7445a49a+        .quad   0x287351b98efc099f+        .quad   0x6765c6f47dfd2538+        .quad   0xca348d3dfb0a9265+        .quad   0x680e910321e58727+        .quad   0x5a13fbe9c476ff09+        .quad   0x6e9e39457b5cc172+        .quad   0x5ddbdcf9102b4494+        .quad   0x7f9d0cbf63553e2b++        // 2^0 * 5 * G++        .quad   0x7f9182c3a447d6ba+        .quad   0xd50014d14b2729b7+        .quad   0xe33cf11cb864a087+        .quad   0x154a7e73eb1b55f3+        .quad   0xa212bc4408a5bb33+        .quad   0x8d5048c3c75eed02+        .quad   0xdd1beb0c5abfec44+        .quad   0x2945ccf146e206eb+        .quad   0xbcbbdbf1812a8285+        .quad   0x270e0807d0bdd1fc+        .quad   0xb41b670b1bbda72d+        .quad   0x43aabe696b3bb69a++        // 2^0 * 6 * G++        .quad   0x499806b67b7d8ca4+        .quad   0x575be28427d22739+        .quad   0xbb085ce7204553b9+        .quad   0x38b64c41ae417884+        .quad   0x3a0ceeeb77157131+        .quad   0x9b27158900c8af88+        .quad   0x8065b668da59a736+        .quad   0x51e57bb6a2cc38bd+        .quad   0x85ac326702ea4b71+        .quad   0xbe70e00341a1bb01+        .quad   0x53e4a24b083bc144+        .quad   0x10b8e91a9f0d61e3++        // 2^0 * 7 * G++        .quad   0xba6f2c9aaa3221b1+        .quad   0x6ca021533bba23a7+        .quad   0x9dea764f92192c3a+        .quad   0x1d6edd5d2e5317e0+        .quad   0x6b1a5cd0944ea3bf+        .quad   0x7470353ab39dc0d2+        .quad   0x71b2528228542e49+        .quad   0x461bea69283c927e+        .quad   0xf1836dc801b8b3a2+        .quad   0xb3035f47053ea49a+        .quad   0x529c41ba5877adf3+        .quad   0x7a9fbb1c6a0f90a7++        // 2^0 * 8 * G++        .quad   0xe2a75dedf39234d9+        .quad   0x963d7680e1b558f9+        .quad   0x2c2741ac6e3c23fb+        .quad   0x3a9024a1320e01c3+        .quad   0x59b7596604dd3e8f+        .quad   0x6cb30377e288702c+        .quad   0xb1339c665ed9c323+        .quad   0x0915e76061bce52f+        .quad   0xe7c1f5d9c9a2911a+        .quad   0xb8a371788bcca7d7+        .quad   0x636412190eb62a32+        .quad   0x26907c5c2ecc4e95++        // 2^4 * 1 * B++        .quad   0x7ec851ca553e2df3+        .quad   0xa71284cba64878b3+        .quad   0xe6b5e4193288d1e7+        .quad   0x4cf210ec5a9a8883+        .quad   0x322d04a52d9021f6+        .quad   0xb9c19f3375c6bf9c+        .quad   0x587a3a4342d20b09+        .quad   0x143b1cf8aa64fe61+        .quad   0x9f867c7d968acaab+        .quad   0x5f54258e27092729+        .quad   0xd0a7d34bea180975+        .quad   0x21b546a3374126e1++        // 2^4 * 2 * B++        .quad   0xa94ff858a2888343+        .quad   0xce0ed4565313ed3c+        .quad   0xf55c3dcfb5bf34fa+        .quad   0x0a653ca5c9eab371+        .quad   0x490a7a45d185218f+        .quad   0x9a15377846049335+        .quad   0x0060ea09cc31e1f6+        .quad   0x7e041577f86ee965+        .quad   0x66b2a496ce5b67f3+        .quad   0xff5492d8bd569796+        .quad   0x503cec294a592cd0+        .quad   0x566943650813acb2++        // 2^4 * 3 * B++        .quad   0xb818db0c26620798+        .quad   0x5d5c31d9606e354a+        .quad   0x0982fa4f00a8cdc7+        .quad   0x17e12bcd4653e2d4+        .quad   0x5672f9eb1dabb69d+        .quad   0xba70b535afe853fc+        .quad   0x47ac0f752796d66d+        .quad   0x32a5351794117275+        .quad   0xd3a644a6df648437+        .quad   0x703b6559880fbfdd+        .quad   0xcb852540ad3a1aa5+        .quad   0x0900b3f78e4c6468++        // 2^4 * 4 * B++        .quad   0x0a851b9f679d651b+        .quad   0xe108cb61033342f2+        .quad   0xd601f57fe88b30a3+        .quad   0x371f3acaed2dd714+        .quad   0xed280fbec816ad31+        .quad   0x52d9595bd8e6efe3+        .quad   0x0fe71772f6c623f5+        .quad   0x4314030b051e293c+        .quad   0xd560005efbf0bcad+        .quad   0x8eb70f2ed1870c5e+        .quad   0x201f9033d084e6a0+        .quad   0x4c3a5ae1ce7b6670++        // 2^4 * 5 * B++        .quad   0x4138a434dcb8fa95+        .quad   0x870cf67d6c96840b+        .quad   0xde388574297be82c+        .quad   0x7c814db27262a55a+        .quad   0xbaf875e4c93da0dd+        .quad   0xb93282a771b9294d+        .quad   0x80d63fb7f4c6c460+        .quad   0x6de9c73dea66c181+        .quad   0x478904d5a04df8f2+        .quad   0xfafbae4ab10142d3+        .quad   0xf6c8ac63555d0998+        .quad   0x5aac4a412f90b104++        // 2^4 * 6 * B++        .quad   0xc64f326b3ac92908+        .quad   0x5551b282e663e1e0+        .quad   0x476b35f54a1a4b83+        .quad   0x1b9da3fe189f68c2+        .quad   0x603a0d0abd7f5134+        .quad   0x8089c932e1d3ae46+        .quad   0xdf2591398798bd63+        .quad   0x1c145cd274ba0235+        .quad   0x32e8386475f3d743+        .quad   0x365b8baf6ae5d9ef+        .quad   0x825238b6385b681e+        .quad   0x234929c1167d65e1++        // 2^4 * 7 * B++        .quad   0x984decaba077ade8+        .quad   0x383f77ad19eb389d+        .quad   0xc7ec6b7e2954d794+        .quad   0x59c77b3aeb7c3a7a+        .quad   0x48145cc21d099fcf+        .quad   0x4535c192cc28d7e5+        .quad   0x80e7c1e548247e01+        .quad   0x4a5f28743b2973ee+        .quad   0xd3add725225ccf62+        .quad   0x911a3381b2152c5d+        .quad   0xd8b39fad5b08f87d+        .quad   0x6f05606b4799fe3b++        // 2^4 * 8 * B++        .quad   0x9ffe9e92177ba962+        .quad   0x98aee71d0de5cae1+        .quad   0x3ff4ae942d831044+        .quad   0x714de12e58533ac8+        .quad   0x5b433149f91b6483+        .quad   0xadb5dc655a2cbf62+        .quad   0x87fa8412632827b3+        .quad   0x60895e91ab49f8d8+        .quad   0xe9ecf2ed0cf86c18+        .quad   0xb46d06120735dfd4+        .quad   0xbc9da09804b96be7+        .quad   0x73e2e62fd96dc26b++        // 2^8 * 1 * B++        .quad   0xed5b635449aa515e+        .quad   0xa865c49f0bc6823a+        .quad   0x850c1fe95b42d1c4+        .quad   0x30d76d6f03d315b9+        .quad   0x2eccdd0e632f9c1d+        .quad   0x51d0b69676893115+        .quad   0x52dfb76ba8637a58+        .quad   0x6dd37d49a00eef39+        .quad   0x6c4444172106e4c7+        .quad   0xfb53d680928d7f69+        .quad   0xb4739ea4694d3f26+        .quad   0x10c697112e864bb0++        // 2^8 * 2 * B++        .quad   0x6493c4277dbe5fde+        .quad   0x265d4fad19ad7ea2+        .quad   0x0e00dfc846304590+        .quad   0x25e61cabed66fe09+        .quad   0x0ca62aa08358c805+        .quad   0x6a3d4ae37a204247+        .quad   0x7464d3a63b11eddc+        .quad   0x03bf9baf550806ef+        .quad   0x3f13e128cc586604+        .quad   0x6f5873ecb459747e+        .quad   0xa0b63dedcc1268f5+        .quad   0x566d78634586e22c++        // 2^8 * 3 * B++        .quad   0x1637a49f9cc10834+        .quad   0xbc8e56d5a89bc451+        .quad   0x1cb5ec0f7f7fd2db+        .quad   0x33975bca5ecc35d9+        .quad   0xa1054285c65a2fd0+        .quad   0x6c64112af31667c3+        .quad   0x680ae240731aee58+        .quad   0x14fba5f34793b22a+        .quad   0x3cd746166985f7d4+        .quad   0x593e5e84c9c80057+        .quad   0x2fc3f2b67b61131e+        .quad   0x14829cea83fc526c++        // 2^8 * 4 * B++        .quad   0xff437b8497dd95c2+        .quad   0x6c744e30aa4eb5a7+        .quad   0x9e0c5d613c85e88b+        .quad   0x2fd9c71e5f758173+        .quad   0x21e70b2f4e71ecb8+        .quad   0xe656ddb940a477e3+        .quad   0xbf6556cece1d4f80+        .quad   0x05fc3bc4535d7b7e+        .quad   0x24b8b3ae52afdedd+        .quad   0x3495638ced3b30cf+        .quad   0x33a4bc83a9be8195+        .quad   0x373767475c651f04++        // 2^8 * 5 * B++        .quad   0x2fba99fd40d1add9+        .quad   0xb307166f96f4d027+        .quad   0x4363f05215f03bae+        .quad   0x1fbea56c3b18f999+        .quad   0x634095cb14246590+        .quad   0xef12144016c15535+        .quad   0x9e38140c8910bc60+        .quad   0x6bf5905730907c8c+        .quad   0x0fa778f1e1415b8a+        .quad   0x06409ff7bac3a77e+        .quad   0x6f52d7b89aa29a50+        .quad   0x02521cf67a635a56++        // 2^8 * 6 * B++        .quad   0x513fee0b0a9d5294+        .quad   0x8f98e75c0fdf5a66+        .quad   0xd4618688bfe107ce+        .quad   0x3fa00a7e71382ced+        .quad   0xb1146720772f5ee4+        .quad   0xe8f894b196079ace+        .quad   0x4af8224d00ac824a+        .quad   0x001753d9f7cd6cc4+        .quad   0x3c69232d963ddb34+        .quad   0x1dde87dab4973858+        .quad   0xaad7d1f9a091f285+        .quad   0x12b5fe2fa048edb6++        // 2^8 * 7 * B++        .quad   0x71f0fbc496fce34d+        .quad   0x73b9826badf35bed+        .quad   0xd2047261ff28c561+        .quad   0x749b76f96fb1206f+        .quad   0xdf2b7c26ad6f1e92+        .quad   0x4b66d323504b8913+        .quad   0x8c409dc0751c8bc3+        .quad   0x6f7e93c20796c7b8+        .quad   0x1f5af604aea6ae05+        .quad   0xc12351f1bee49c99+        .quad   0x61a808b5eeff6b66+        .quad   0x0fcec10f01e02151++        // 2^8 * 8 * B++        .quad   0x644d58a649fe1e44+        .quad   0x21fcaea231ad777e+        .quad   0x02441c5a887fd0d2+        .quad   0x4901aa7183c511f3+        .quad   0x3df2d29dc4244e45+        .quad   0x2b020e7493d8de0a+        .quad   0x6cc8067e820c214d+        .quad   0x413779166feab90a+        .quad   0x08b1b7548c1af8f0+        .quad   0xce0f7a7c246299b4+        .quad   0xf760b0f91e06d939+        .quad   0x41bb887b726d1213++        // 2^12 * 1 * B++        .quad   0x9267806c567c49d8+        .quad   0x066d04ccca791e6a+        .quad   0xa69f5645e3cc394b+        .quad   0x5c95b686a0788cd2+        .quad   0x97d980e0aa39f7d2+        .quad   0x35d0384252c6b51c+        .quad   0x7d43f49307cd55aa+        .quad   0x56bd36cfb78ac362+        .quad   0x2ac519c10d14a954+        .quad   0xeaf474b494b5fa90+        .quad   0xe6af8382a9f87a5a+        .quad   0x0dea6db1879be094++        // 2^12 * 2 * B++        .quad   0xaa66bf547344e5ab+        .quad   0xda1258888f1b4309+        .quad   0x5e87d2b3fd564b2f+        .quad   0x5b2c78885483b1dd+        .quad   0x15baeb74d6a8797a+        .quad   0x7ef55cf1fac41732+        .quad   0x29001f5a3c8b05c5+        .quad   0x0ad7cc8752eaccfb+        .quad   0x52151362793408cf+        .quad   0xeb0f170319963d94+        .quad   0xa833b2fa883d9466+        .quad   0x093a7fa775003c78++        // 2^12 * 3 * B++        .quad   0xe5107de63a16d7be+        .quad   0xa377ffdc9af332cf+        .quad   0x70d5bf18440b677f+        .quad   0x6a252b19a4a31403+        .quad   0xb8e9604460a91286+        .quad   0x7f3fd8047778d3de+        .quad   0x67d01e31bf8a5e2d+        .quad   0x7b038a06c27b653e+        .quad   0x9ed919d5d36990f3+        .quad   0x5213aebbdb4eb9f2+        .quad   0xc708ea054cb99135+        .quad   0x58ded57f72260e56++        // 2^12 * 4 * B++        .quad   0x78e79dade9413d77+        .quad   0xf257f9d59729e67d+        .quad   0x59db910ee37aa7e6+        .quad   0x6aa11b5bbb9e039c+        .quad   0xda6d53265b0fd48b+        .quad   0x8960823193bfa988+        .quad   0xd78ac93261d57e28+        .quad   0x79f2942d3a5c8143+        .quad   0x97da2f25b6c88de9+        .quad   0x251ba7eaacf20169+        .quad   0x09b44f87ef4eb4e4+        .quad   0x7d90ab1bbc6a7da5++        // 2^12 * 5 * B++        .quad   0x9acca683a7016bfe+        .quad   0x90505f4df2c50b6d+        .quad   0x6b610d5fcce435aa+        .quad   0x19a10d446198ff96+        .quad   0x1a07a3f496b3c397+        .quad   0x11ceaa188f4e2532+        .quad   0x7d9498d5a7751bf0+        .quad   0x19ed161f508dd8a0+        .quad   0x560a2cd687dce6ca+        .quad   0x7f3568c48664cf4d+        .quad   0x8741e95222803a38+        .quad   0x483bdab1595653fc++        // 2^12 * 6 * B++        .quad   0xfa780f148734fa49+        .quad   0x106f0b70360534e0+        .quad   0x2210776fe3e307bd+        .quad   0x3286c109dde6a0fe+        .quad   0xd6cf4d0ab4da80f6+        .quad   0x82483e45f8307fe0+        .quad   0x05005269ae6f9da4+        .quad   0x1c7052909cf7877a+        .quad   0x32ee7de2874e98d4+        .quad   0x14c362e9b97e0c60+        .quad   0x5781dcde6a60a38a+        .quad   0x217dd5eaaa7aa840++        // 2^12 * 7 * B++        .quad   0x9db7c4d0248e1eb0+        .quad   0xe07697e14d74bf52+        .quad   0x1e6a9b173c562354+        .quad   0x7fa7c21f795a4965+        .quad   0x8bdf1fb9be8c0ec8+        .quad   0x00bae7f8e30a0282+        .quad   0x4963991dad6c4f6c+        .quad   0x07058a6e5df6f60a+        .quad   0xe9eb02c4db31f67f+        .quad   0xed25fd8910bcfb2b+        .quad   0x46c8131f5c5cddb4+        .quad   0x33b21c13a0cb9bce++        // 2^12 * 8 * B++        .quad   0x360692f8087d8e31+        .quad   0xf4dcc637d27163f7+        .quad   0x25a4e62065ea5963+        .quad   0x659bf72e5ac160d9+        .quad   0x9aafb9b05ee38c5b+        .quad   0xbf9d2d4e071a13c7+        .quad   0x8eee6e6de933290a+        .quad   0x1c3bab17ae109717+        .quad   0x1c9ab216c7cab7b0+        .quad   0x7d65d37407bbc3cc+        .quad   0x52744750504a58d5+        .quad   0x09f2606b131a2990++        // 2^16 * 1 * B++        .quad   0x40e87d44744346be+        .quad   0x1d48dad415b52b25+        .quad   0x7c3a8a18a13b603e+        .quad   0x4eb728c12fcdbdf7+        .quad   0x7e234c597c6691ae+        .quad   0x64889d3d0a85b4c8+        .quad   0xdae2c90c354afae7+        .quad   0x0a871e070c6a9e1d+        .quad   0x3301b5994bbc8989+        .quad   0x736bae3a5bdd4260+        .quad   0x0d61ade219d59e3c+        .quad   0x3ee7300f2685d464++        // 2^16 * 2 * B++        .quad   0xf5d255e49e7dd6b7+        .quad   0x8016115c610b1eac+        .quad   0x3c99975d92e187ca+        .quad   0x13815762979125c2+        .quad   0x43fa7947841e7518+        .quad   0xe5c6fa59639c46d7+        .quad   0xa1065e1de3052b74+        .quad   0x7d47c6a2cfb89030+        .quad   0x3fdad0148ef0d6e0+        .quad   0x9d3e749a91546f3c+        .quad   0x71ec621026bb8157+        .quad   0x148cf58d34c9ec80++        // 2^16 * 3 * B++        .quad   0x46a492f67934f027+        .quad   0x469984bef6840aa9+        .quad   0x5ca1bc2a89611854+        .quad   0x3ff2fa1ebd5dbbd4+        .quad   0xe2572f7d9ae4756d+        .quad   0x56c345bb88f3487f+        .quad   0x9fd10b6d6960a88d+        .quad   0x278febad4eaea1b9+        .quad   0xb1aa681f8c933966+        .quad   0x8c21949c20290c98+        .quad   0x39115291219d3c52+        .quad   0x4104dd02fe9c677b++        // 2^16 * 4 * B++        .quad   0x72b2bf5e1124422a+        .quad   0xa1fa0c3398a33ab5+        .quad   0x94cb6101fa52b666+        .quad   0x2c863b00afaf53d5+        .quad   0x81214e06db096ab8+        .quad   0x21a8b6c90ce44f35+        .quad   0x6524c12a409e2af5+        .quad   0x0165b5a48efca481+        .quad   0xf190a474a0846a76+        .quad   0x12eff984cd2f7cc0+        .quad   0x695e290658aa2b8f+        .quad   0x591b67d9bffec8b8++        // 2^16 * 5 * B++        .quad   0x312f0d1c80b49bfa+        .quad   0x5979515eabf3ec8a+        .quad   0x727033c09ef01c88+        .quad   0x3de02ec7ca8f7bcb+        .quad   0x99b9b3719f18b55d+        .quad   0xe465e5faa18c641e+        .quad   0x61081136c29f05ed+        .quad   0x489b4f867030128b+        .quad   0xd232102d3aeb92ef+        .quad   0xe16253b46116a861+        .quad   0x3d7eabe7190baa24+        .quad   0x49f5fbba496cbebf++        // 2^16 * 6 * B++        .quad   0x30949a108a5bcfd4+        .quad   0xdc40dd70bc6473eb+        .quad   0x92c294c1307c0d1c+        .quad   0x5604a86dcbfa6e74+        .quad   0x155d628c1e9c572e+        .quad   0x8a4d86acc5884741+        .quad   0x91a352f6515763eb+        .quad   0x06a1a6c28867515b+        .quad   0x7288d1d47c1764b6+        .quad   0x72541140e0418b51+        .quad   0x9f031a6018acf6d1+        .quad   0x20989e89fe2742c6++        // 2^16 * 7 * B++        .quad   0x499777fd3a2dcc7f+        .quad   0x32857c2ca54fd892+        .quad   0xa279d864d207e3a0+        .quad   0x0403ed1d0ca67e29+        .quad   0x1674278b85eaec2e+        .quad   0x5621dc077acb2bdf+        .quad   0x640a4c1661cbf45a+        .quad   0x730b9950f70595d3+        .quad   0xc94b2d35874ec552+        .quad   0xc5e6c8cf98246f8d+        .quad   0xf7cb46fa16c035ce+        .quad   0x5bd7454308303dcc++        // 2^16 * 8 * B++        .quad   0x7f9ad19528b24cc2+        .quad   0x7f6b54656335c181+        .quad   0x66b8b66e4fc07236+        .quad   0x133a78007380ad83+        .quad   0x85c4932115e7792a+        .quad   0xc64c89a2bdcdddc9+        .quad   0x9d1e3da8ada3d762+        .quad   0x5bb7db123067f82c+        .quad   0x0961f467c6ca62be+        .quad   0x04ec21d6211952ee+        .quad   0x182360779bd54770+        .quad   0x740dca6d58f0e0d2++        // 2^20 * 1 * B++        .quad   0x50b70bf5d3f0af0b+        .quad   0x4feaf48ae32e71f7+        .quad   0x60e84ed3a55bbd34+        .quad   0x00ed489b3f50d1ed+        .quad   0x3906c72aed261ae5+        .quad   0x9ab68fd988e100f7+        .quad   0xf5e9059af3360197+        .quad   0x0e53dc78bf2b6d47+        .quad   0xb90829bf7971877a+        .quad   0x5e4444636d17e631+        .quad   0x4d05c52e18276893+        .quad   0x27632d9a5a4a4af5++        // 2^20 * 2 * B++        .quad   0xd11ff05154b260ce+        .quad   0xd86dc38e72f95270+        .quad   0x601fcd0d267cc138+        .quad   0x2b67916429e90ccd+        .quad   0xa98285d187eaffdb+        .quad   0xa5b4fbbbd8d0a864+        .quad   0xb658f27f022663f7+        .quad   0x3bbc2b22d99ce282+        .quad   0xb917c952583c0a58+        .quad   0x653ff9b80fe4c6f3+        .quad   0x9b0da7d7bcdf3c0c+        .quad   0x43a0eeb6ab54d60e++        // 2^20 * 3 * B++        .quad   0x396966a46d4a5487+        .quad   0xf811a18aac2bb3ba+        .quad   0x66e4685b5628b26b+        .quad   0x70a477029d929b92+        .quad   0x3ac6322357875fe8+        .quad   0xd9d4f4ecf5fbcb8f+        .quad   0x8dee8493382bb620+        .quad   0x50c5eaa14c799fdc+        .quad   0xdd0edc8bd6f2fb3c+        .quad   0x54c63aa79cc7b7a0+        .quad   0xae0b032b2c8d9f1a+        .quad   0x6f9ce107602967fb++        // 2^20 * 4 * B++        .quad   0xad1054b1cde1c22a+        .quad   0xc4a8e90248eb32df+        .quad   0x5f3e7b33accdc0ea+        .quad   0x72364713fc79963e+        .quad   0x139693063520e0b5+        .quad   0x437fcf7c88ea03fe+        .quad   0xf7d4c40bd3c959bc+        .quad   0x699154d1f893ded9+        .quad   0x315d5c75b4b27526+        .quad   0xcccb842d0236daa5+        .quad   0x22f0c8a3345fee8e+        .quad   0x73975a617d39dbed++        // 2^20 * 5 * B++        .quad   0xe4024df96375da10+        .quad   0x78d3251a1830c870+        .quad   0x902b1948658cd91c+        .quad   0x7e18b10b29b7438a+        .quad   0x6f37f392f4433e46+        .quad   0x0e19b9a11f566b18+        .quad   0x220fb78a1fd1d662+        .quad   0x362a4258a381c94d+        .quad   0x9071d9132b6beb2f+        .quad   0x0f26e9ad28418247+        .quad   0xeab91ec9bdec925d+        .quad   0x4be65bc8f48af2de++        // 2^20 * 6 * B++        .quad   0x78487feba36e7028+        .quad   0x5f3f13001dd8ce34+        .quad   0x934fb12d4b30c489+        .quad   0x056c244d397f0a2b+        .quad   0x1d50fba257c26234+        .quad   0x7bd4823adeb0678b+        .quad   0xc2b0dc6ea6538af5+        .quad   0x5665eec6351da73e+        .quad   0xdb3ee00943bfb210+        .quad   0x4972018720800ac2+        .quad   0x26ab5d6173bd8667+        .quad   0x20b209c2ab204938++        // 2^20 * 7 * B++        .quad   0x549e342ac07fb34b+        .quad   0x02d8220821373d93+        .quad   0xbc262d70acd1f567+        .quad   0x7a92c9fdfbcac784+        .quad   0x1fcca94516bd3289+        .quad   0x448d65aa41420428+        .quad   0x59c3b7b216a55d62+        .quad   0x49992cc64e612cd8+        .quad   0x65bd1bea70f801de+        .quad   0x1befb7c0fe49e28a+        .quad   0xa86306cdb1b2ae4a+        .quad   0x3b7ac0cd265c2a09++        // 2^20 * 8 * B++        .quad   0x822bee438c01bcec+        .quad   0x530cb525c0fbc73b+        .quad   0x48519034c1953fe9+        .quad   0x265cc261e09a0f5b+        .quad   0xf0d54e4f22ed39a7+        .quad   0xa2aae91e5608150a+        .quad   0xf421b2e9eddae875+        .quad   0x31bc531d6b7de992+        .quad   0xdf3d134da980f971+        .quad   0x7a4fb8d1221a22a7+        .quad   0x3df7d42035aad6d8+        .quad   0x2a14edcc6a1a125e++        // 2^24 * 1 * B++        .quad   0xdf48ee0752cfce4e+        .quad   0xc3fffaf306ec08b7+        .quad   0x05710b2ab95459c4+        .quad   0x161d25fa963ea38d+        .quad   0x231a8c570478433c+        .quad   0xb7b5270ec281439d+        .quad   0xdbaa99eae3d9079f+        .quad   0x2c03f5256c2b03d9+        .quad   0x790f18757b53a47d+        .quad   0x307b0130cf0c5879+        .quad   0x31903d77257ef7f9+        .quad   0x699468bdbd96bbaf++        // 2^24 * 2 * B++        .quad   0xbd1f2f46f4dafecf+        .quad   0x7cef0114a47fd6f7+        .quad   0xd31ffdda4a47b37f+        .quad   0x525219a473905785+        .quad   0xd8dd3de66aa91948+        .quad   0x485064c22fc0d2cc+        .quad   0x9b48246634fdea2f+        .quad   0x293e1c4e6c4a2e3a+        .quad   0x376e134b925112e1+        .quad   0x703778b5dca15da0+        .quad   0xb04589af461c3111+        .quad   0x5b605c447f032823++        // 2^24 * 3 * B++        .quad   0xb965805920c47c89+        .quad   0xe7f0100c923b8fcc+        .quad   0x0001256502e2ef77+        .quad   0x24a76dcea8aeb3ee+        .quad   0x3be9fec6f0e7f04c+        .quad   0x866a579e75e34962+        .quad   0x5542ef161e1de61a+        .quad   0x2f12fef4cc5abdd5+        .quad   0x0a4522b2dfc0c740+        .quad   0x10d06e7f40c9a407+        .quad   0xc6cf144178cff668+        .quad   0x5e607b2518a43790++        // 2^24 * 4 * B++        .quad   0x58b31d8f6cdf1818+        .quad   0x35cfa74fc36258a2+        .quad   0xe1b3ff4f66e61d6e+        .quad   0x5067acab6ccdd5f7+        .quad   0xa02c431ca596cf14+        .quad   0xe3c42d40aed3e400+        .quad   0xd24526802e0f26db+        .quad   0x201f33139e457068+        .quad   0xfd527f6b08039d51+        .quad   0x18b14964017c0006+        .quad   0xd5220eb02e25a4a8+        .quad   0x397cba8862460375++        // 2^24 * 5 * B++        .quad   0x30c13093f05959b2+        .quad   0xe23aa18de9a97976+        .quad   0x222fd491721d5e26+        .quad   0x2339d320766e6c3a+        .quad   0x7815c3fbc81379e7+        .quad   0xa6619420dde12af1+        .quad   0xffa9c0f885a8fdd5+        .quad   0x771b4022c1e1c252+        .quad   0xd87dd986513a2fa7+        .quad   0xf5ac9b71f9d4cf08+        .quad   0xd06bc31b1ea283b3+        .quad   0x331a189219971a76++        // 2^24 * 6 * B++        .quad   0xf5166f45fb4f80c6+        .quad   0x9c36c7de61c775cf+        .quad   0xe3d4e81b9041d91c+        .quad   0x31167c6b83bdfe21+        .quad   0x26512f3a9d7572af+        .quad   0x5bcbe28868074a9e+        .quad   0x84edc1c11180f7c4+        .quad   0x1ac9619ff649a67b+        .quad   0xf22b3842524b1068+        .quad   0x5068343bee9ce987+        .quad   0xfc9d71844a6250c8+        .quad   0x612436341f08b111++        // 2^24 * 7 * B++        .quad   0xd99d41db874e898d+        .quad   0x09fea5f16c07dc20+        .quad   0x793d2c67d00f9bbc+        .quad   0x46ebe2309e5eff40+        .quad   0x8b6349e31a2d2638+        .quad   0x9ddfb7009bd3fd35+        .quad   0x7f8bf1b8a3a06ba4+        .quad   0x1522aa3178d90445+        .quad   0x2c382f5369614938+        .quad   0xdafe409ab72d6d10+        .quad   0xe8c83391b646f227+        .quad   0x45fe70f50524306c++        // 2^24 * 8 * B++        .quad   0xda4875a6960c0b8c+        .quad   0x5b68d076ef0e2f20+        .quad   0x07fb51cf3d0b8fd4+        .quad   0x428d1623a0e392d4+        .quad   0x62f24920c8951491+        .quad   0x05f007c83f630ca2+        .quad   0x6fbb45d2f5c9d4b8+        .quad   0x16619f6db57a2245+        .quad   0x084f4a4401a308fd+        .quad   0xa82219c376a5caac+        .quad   0xdeb8de4643d1bc7d+        .quad   0x1d81592d60bd38c6++        // 2^28 * 1 * B++        .quad   0xd833d7beec2a4c38+        .quad   0x2c9162830acc20ed+        .quad   0xe93a47aa92df7581+        .quad   0x702d67a3333c4a81+        .quad   0x3a4a369a2f89c8a1+        .quad   0x63137a1d7c8de80d+        .quad   0xbcac008a78eda015+        .quad   0x2cb8b3a5b483b03f+        .quad   0x36e417cbcb1b90a1+        .quad   0x33b3ddaa7f11794e+        .quad   0x3f510808885bc607+        .quad   0x24141dc0e6a8020d++        // 2^28 * 2 * B++        .quad   0x59f73c773fefee9d+        .quad   0xb3f1ef89c1cf989d+        .quad   0xe35dfb42e02e545f+        .quad   0x5766120b47a1b47c+        .quad   0x91925dccbd83157d+        .quad   0x3ca1205322cc8094+        .quad   0x28e57f183f90d6e4+        .quad   0x1a4714cede2e767b+        .quad   0xdb20ba0fb8b6b7ff+        .quad   0xb732c3b677511fa1+        .quad   0xa92b51c099f02d89+        .quad   0x4f3875ad489ca5f1++        // 2^28 * 3 * B++        .quad   0xc7fc762f4932ab22+        .quad   0x7ac0edf72f4c3c1b+        .quad   0x5f6b55aa9aa895e8+        .quad   0x3680274dad0a0081+        .quad   0x79ed13f6ee73eec0+        .quad   0xa5c6526d69110bb1+        .quad   0xe48928c38603860c+        .quad   0x722a1446fd7059f5+        .quad   0xd0959fe9a8cf8819+        .quad   0xd0a995508475a99c+        .quad   0x6eac173320b09cc5+        .quad   0x628ecf04331b1095++        // 2^28 * 4 * B++        .quad   0x98bcb118a9d0ddbc+        .quad   0xee449e3408b4802b+        .quad   0x87089226b8a6b104+        .quad   0x685f349a45c7915d+        .quad   0x9b41acf85c74ccf1+        .quad   0xb673318108265251+        .quad   0x99c92aed11adb147+        .quad   0x7a47d70d34ecb40f+        .quad   0x60a0c4cbcc43a4f5+        .quad   0x775c66ca3677bea9+        .quad   0xa17aa1752ff8f5ed+        .quad   0x11ded9020e01fdc0++        // 2^28 * 5 * B++        .quad   0x890e7809caefe704+        .quad   0x8728296de30e8c6c+        .quad   0x4c5cd2a392aeb1c9+        .quad   0x194263d15771531f+        .quad   0x471f95b03bea93b7+        .quad   0x0552d7d43313abd3+        .quad   0xbd9370e2e17e3f7b+        .quad   0x7b120f1db20e5bec+        .quad   0x17d2fb3d86502d7a+        .quad   0xb564d84450a69352+        .quad   0x7da962c8a60ed75d+        .quad   0x00d0f85b318736aa++        // 2^28 * 6 * B++        .quad   0x978b142e777c84fd+        .quad   0xf402644705a8c062+        .quad   0xa67ad51be7e612c7+        .quad   0x2f7b459698dd6a33+        .quad   0xa6753c1efd7621c1+        .quad   0x69c0b4a7445671f5+        .quad   0x971f527405b23c11+        .quad   0x387bc74851a8c7cd+        .quad   0x81894b4d4a52a9a8+        .quad   0xadd93e12f6b8832f+        .quad   0x184d8548b61bd638+        .quad   0x3f1c62dbd6c9f6cd++        // 2^28 * 7 * B++        .quad   0x2e8f1f0091910c1f+        .quad   0xa4df4fe0bff2e12c+        .quad   0x60c6560aee927438+        .quad   0x6338283facefc8fa+        .quad   0x3fad3e40148f693d+        .quad   0x052656e194eb9a72+        .quad   0x2f4dcbfd184f4e2f+        .quad   0x406f8db1c482e18b+        .quad   0x9e630d2c7f191ee4+        .quad   0x4fbf8301bc3ff670+        .quad   0x787d8e4e7afb73c4+        .quad   0x50d83d5be8f58fa5++        // 2^28 * 8 * B++        .quad   0x85683916c11a1897+        .quad   0x2d69a4efe506d008+        .quad   0x39af1378f664bd01+        .quad   0x65942131361517c6+        .quad   0xc0accf90b4d3b66d+        .quad   0xa7059de561732e60+        .quad   0x033d1f7870c6b0ba+        .quad   0x584161cd26d946e4+        .quad   0xbbf2b1a072d27ca2+        .quad   0xbf393c59fbdec704+        .quad   0xe98dbbcee262b81e+        .quad   0x02eebd0b3029b589++        // 2^32 * 1 * B++        .quad   0x61368756a60dac5f+        .quad   0x17e02f6aebabdc57+        .quad   0x7f193f2d4cce0f7d+        .quad   0x20234a7789ecdcf0+        .quad   0x8765b69f7b85c5e8+        .quad   0x6ff0678bd168bab2+        .quad   0x3a70e77c1d330f9b+        .quad   0x3a5f6d51b0af8e7c+        .quad   0x76d20db67178b252+        .quad   0x071c34f9d51ed160+        .quad   0xf62a4a20b3e41170+        .quad   0x7cd682353cffe366++        // 2^32 * 2 * B++        .quad   0x0be1a45bd887fab6+        .quad   0x2a846a32ba403b6e+        .quad   0xd9921012e96e6000+        .quad   0x2838c8863bdc0943+        .quad   0xa665cd6068acf4f3+        .quad   0x42d92d183cd7e3d3+        .quad   0x5759389d336025d9+        .quad   0x3ef0253b2b2cd8ff+        .quad   0xd16bb0cf4a465030+        .quad   0xfa496b4115c577ab+        .quad   0x82cfae8af4ab419d+        .quad   0x21dcb8a606a82812++        // 2^32 * 3 * B++        .quad   0x5c6004468c9d9fc8+        .quad   0x2540096ed42aa3cb+        .quad   0x125b4d4c12ee2f9c+        .quad   0x0bc3d08194a31dab+        .quad   0x9a8d00fabe7731ba+        .quad   0x8203607e629e1889+        .quad   0xb2cc023743f3d97f+        .quad   0x5d840dbf6c6f678b+        .quad   0x706e380d309fe18b+        .quad   0x6eb02da6b9e165c7+        .quad   0x57bbba997dae20ab+        .quad   0x3a4276232ac196dd++        // 2^32 * 4 * B++        .quad   0x4b42432c8a7084fa+        .quad   0x898a19e3dfb9e545+        .quad   0xbe9f00219c58e45d+        .quad   0x1ff177cea16debd1+        .quad   0x3bf8c172db447ecb+        .quad   0x5fcfc41fc6282dbd+        .quad   0x80acffc075aa15fe+        .quad   0x0770c9e824e1a9f9+        .quad   0xcf61d99a45b5b5fd+        .quad   0x860984e91b3a7924+        .quad   0xe7300919303e3e89+        .quad   0x39f264fd41500b1e++        // 2^32 * 5 * B++        .quad   0xa7ad3417dbe7e29c+        .quad   0xbd94376a2b9c139c+        .quad   0xa0e91b8e93597ba9+        .quad   0x1712d73468889840+        .quad   0xd19b4aabfe097be1+        .quad   0xa46dfce1dfe01929+        .quad   0xc3c908942ca6f1ff+        .quad   0x65c621272c35f14e+        .quad   0xe72b89f8ce3193dd+        .quad   0x4d103356a125c0bb+        .quad   0x0419a93d2e1cfe83+        .quad   0x22f9800ab19ce272++        // 2^32 * 6 * B++        .quad   0x605a368a3e9ef8cb+        .quad   0xe3e9c022a5504715+        .quad   0x553d48b05f24248f+        .quad   0x13f416cd647626e5+        .quad   0x42029fdd9a6efdac+        .quad   0xb912cebe34a54941+        .quad   0x640f64b987bdf37b+        .quad   0x4171a4d38598cab4+        .quad   0xfa2758aa99c94c8c+        .quad   0x23006f6fb000b807+        .quad   0xfbd291ddadda5392+        .quad   0x508214fa574bd1ab++        // 2^32 * 7 * B++        .quad   0xc20269153ed6fe4b+        .quad   0xa65a6739511d77c4+        .quad   0xcbde26462c14af94+        .quad   0x22f960ec6faba74b+        .quad   0x461a15bb53d003d6+        .quad   0xb2102888bcf3c965+        .quad   0x27c576756c683a5a+        .quad   0x3a7758a4c86cb447+        .quad   0x548111f693ae5076+        .quad   0x1dae21df1dfd54a6+        .quad   0x12248c90f3115e65+        .quad   0x5d9fd15f8de7f494++        // 2^32 * 8 * B++        .quad   0x031408d36d63727f+        .quad   0x6a379aefd7c7b533+        .quad   0xa9e18fc5ccaee24b+        .quad   0x332f35914f8fbed3+        .quad   0x3f244d2aeed7521e+        .quad   0x8e3a9028432e9615+        .quad   0xe164ba772e9c16d4+        .quad   0x3bc187fa47eb98d8+        .quad   0x6d470115ea86c20c+        .quad   0x998ab7cb6c46d125+        .quad   0xd77832b53a660188+        .quad   0x450d81ce906fba03++        // 2^36 * 1 * B++        .quad   0xf8ae4d2ad8453902+        .quad   0x7018058ee8db2d1d+        .quad   0xaab3995fc7d2c11e+        .quad   0x53b16d2324ccca79+        .quad   0x23264d66b2cae0b5+        .quad   0x7dbaed33ebca6576+        .quad   0x030ebed6f0d24ac8+        .quad   0x2a887f78f7635510+        .quad   0x2a23b9e75c012d4f+        .quad   0x0c974651cae1f2ea+        .quad   0x2fb63273675d70ca+        .quad   0x0ba7250b864403f5++        // 2^36 * 2 * B++        .quad   0xbb0d18fd029c6421+        .quad   0xbc2d142189298f02+        .quad   0x8347f8e68b250e96+        .quad   0x7b9f2fe8032d71c9+        .quad   0xdd63589386f86d9c+        .quad   0x61699176e13a85a4+        .quad   0x2e5111954eaa7d57+        .quad   0x32c21b57fb60bdfb+        .quad   0xd87823cd319e0780+        .quad   0xefc4cfc1897775c5+        .quad   0x4854fb129a0ab3f7+        .quad   0x12c49d417238c371++        // 2^36 * 3 * B++        .quad   0x0950b533ffe83769+        .quad   0x21861c1d8e1d6bd1+        .quad   0xf022d8381302e510+        .quad   0x2509200c6391cab4+        .quad   0x09b3a01783799542+        .quad   0x626dd08faad5ee3f+        .quad   0xba00bceeeb70149f+        .quad   0x1421b246a0a444c9+        .quad   0x4aa43a8e8c24a7c7+        .quad   0x04c1f540d8f05ef5+        .quad   0xadba5e0c0b3eb9dc+        .quad   0x2ab5504448a49ce3++        // 2^36 * 4 * B++        .quad   0x2ed227266f0f5dec+        .quad   0x9824ee415ed50824+        .quad   0x807bec7c9468d415+        .quad   0x7093bae1b521e23f+        .quad   0xdc07ac631c5d3afa+        .quad   0x58615171f9df8c6c+        .quad   0x72a079d89d73e2b0+        .quad   0x7301f4ceb4eae15d+        .quad   0x6409e759d6722c41+        .quad   0xa674e1cf72bf729b+        .quad   0xbc0a24eb3c21e569+        .quad   0x390167d24ebacb23++        // 2^36 * 5 * B++        .quad   0x27f58e3bba353f1c+        .quad   0x4c47764dbf6a4361+        .quad   0xafbbc4e56e562650+        .quad   0x07db2ee6aae1a45d+        .quad   0xd7bb054ba2f2120b+        .quad   0xe2b9ceaeb10589b7+        .quad   0x3fe8bac8f3c0edbe+        .quad   0x4cbd40767112cb69+        .quad   0x0b603cc029c58176+        .quad   0x5988e3825cb15d61+        .quad   0x2bb61413dcf0ad8d+        .quad   0x7b8eec6c74183287++        // 2^36 * 6 * B++        .quad   0xe4ca40782cd27cb0+        .quad   0xdaf9c323fbe967bd+        .quad   0xb29bd34a8ad41e9e+        .quad   0x72810497626ede4d+        .quad   0x32fee570fc386b73+        .quad   0xda8b0141da3a8cc7+        .quad   0x975ffd0ac8968359+        .quad   0x6ee809a1b132a855+        .quad   0x9444bb31fcfd863a+        .quad   0x2fe3690a3e4e48c5+        .quad   0xdc29c867d088fa25+        .quad   0x13bd1e38d173292e++        // 2^36 * 7 * B++        .quad   0xd32b4cd8696149b5+        .quad   0xe55937d781d8aab7+        .quad   0x0bcb2127ae122b94+        .quad   0x41e86fcfb14099b0+        .quad   0x223fb5cf1dfac521+        .quad   0x325c25316f554450+        .quad   0x030b98d7659177ac+        .quad   0x1ed018b64f88a4bd+        .quad   0x3630dfa1b802a6b0+        .quad   0x880f874742ad3bd5+        .quad   0x0af90d6ceec5a4d4+        .quad   0x746a247a37cdc5d9++        // 2^36 * 8 * B++        .quad   0xd531b8bd2b7b9af6+        .quad   0x5005093537fc5b51+        .quad   0x232fcf25c593546d+        .quad   0x20a365142bb40f49+        .quad   0x6eccd85278d941ed+        .quad   0x2254ae83d22f7843+        .quad   0xc522d02e7bbfcdb7+        .quad   0x681e3351bff0e4e2+        .quad   0x8b64b59d83034f45+        .quad   0x2f8b71f21fa20efb+        .quad   0x69249495ba6550e4+        .quad   0x539ef98e45d5472b++        // 2^40 * 1 * B++        .quad   0x6e7bb6a1a6205275+        .quad   0xaa4f21d7413c8e83+        .quad   0x6f56d155e88f5cb2+        .quad   0x2de25d4ba6345be1+        .quad   0xd074d8961cae743f+        .quad   0xf86d18f5ee1c63ed+        .quad   0x97bdc55be7f4ed29+        .quad   0x4cbad279663ab108+        .quad   0x80d19024a0d71fcd+        .quad   0xc525c20afb288af8+        .quad   0xb1a3974b5f3a6419+        .quad   0x7d7fbcefe2007233++        // 2^40 * 2 * B++        .quad   0xfaef1e6a266b2801+        .quad   0x866c68c4d5739f16+        .quad   0xf68a2fbc1b03762c+        .quad   0x5975435e87b75a8d+        .quad   0xcd7c5dc5f3c29094+        .quad   0xc781a29a2a9105ab+        .quad   0x80c61d36421c3058+        .quad   0x4f9cd196dcd8d4d7+        .quad   0x199297d86a7b3768+        .quad   0xd0d058241ad17a63+        .quad   0xba029cad5c1c0c17+        .quad   0x7ccdd084387a0307++        // 2^40 * 3 * B++        .quad   0xdca6422c6d260417+        .quad   0xae153d50948240bd+        .quad   0xa9c0c1b4fb68c677+        .quad   0x428bd0ed61d0cf53+        .quad   0x9b0c84186760cc93+        .quad   0xcdae007a1ab32a99+        .quad   0xa88dec86620bda18+        .quad   0x3593ca848190ca44+        .quad   0x9213189a5e849aa7+        .quad   0xd4d8c33565d8facd+        .quad   0x8c52545b53fdbbd1+        .quad   0x27398308da2d63e6++        // 2^40 * 4 * B++        .quad   0x42c38d28435ed413+        .quad   0xbd50f3603278ccc9+        .quad   0xbb07ab1a79da03ef+        .quad   0x269597aebe8c3355+        .quad   0xb9a10e4c0a702453+        .quad   0x0fa25866d57d1bde+        .quad   0xffb9d9b5cd27daf7+        .quad   0x572c2945492c33fd+        .quad   0xc77fc745d6cd30be+        .quad   0xe4dfe8d3e3baaefb+        .quad   0xa22c8830aa5dda0c+        .quad   0x7f985498c05bca80++        // 2^40 * 5 * B++        .quad   0x3849ce889f0be117+        .quad   0x8005ad1b7b54a288+        .quad   0x3da3c39f23fc921c+        .quad   0x76c2ec470a31f304+        .quad   0xd35615520fbf6363+        .quad   0x08045a45cf4dfba6+        .quad   0xeec24fbc873fa0c2+        .quad   0x30f2653cd69b12e7+        .quad   0x8a08c938aac10c85+        .quad   0x46179b60db276bcb+        .quad   0xa920c01e0e6fac70+        .quad   0x2f1273f1596473da++        // 2^40 * 6 * B++        .quad   0x4739fc7c8ae01e11+        .quad   0xfd5274904a6aab9f+        .quad   0x41d98a8287728f2e+        .quad   0x5d9e572ad85b69f2+        .quad   0x30488bd755a70bc0+        .quad   0x06d6b5a4f1d442e7+        .quad   0xead1a69ebc596162+        .quad   0x38ac1997edc5f784+        .quad   0x0666b517a751b13b+        .quad   0x747d06867e9b858c+        .quad   0xacacc011454dde49+        .quad   0x22dfcd9cbfe9e69c++        // 2^40 * 7 * B++        .quad   0x8ddbd2e0c30d0cd9+        .quad   0xad8e665facbb4333+        .quad   0x8f6b258c322a961f+        .quad   0x6b2916c05448c1c7+        .quad   0x56ec59b4103be0a1+        .quad   0x2ee3baecd259f969+        .quad   0x797cb29413f5cd32+        .quad   0x0fe9877824cde472+        .quad   0x7edb34d10aba913b+        .quad   0x4ea3cd822e6dac0e+        .quad   0x66083dff6578f815+        .quad   0x4c303f307ff00a17++        // 2^40 * 8 * B++        .quad   0xd30a3bd617b28c85+        .quad   0xc5d377b739773bea+        .quad   0xc6c6e78c1e6a5cbf+        .quad   0x0d61b8f78b2ab7c4+        .quad   0x29fc03580dd94500+        .quad   0xecd27aa46fbbec93+        .quad   0x130a155fc2e2a7f8+        .quad   0x416b151ab706a1d5+        .quad   0x56a8d7efe9c136b0+        .quad   0xbd07e5cd58e44b20+        .quad   0xafe62fda1b57e0ab+        .quad   0x191a2af74277e8d2++        // 2^44 * 1 * B++        .quad   0xd550095bab6f4985+        .quad   0x04f4cd5b4fbfaf1a+        .quad   0x9d8e2ed12a0c7540+        .quad   0x2bc24e04b2212286+        .quad   0x09d4b60b2fe09a14+        .quad   0xc384f0afdbb1747e+        .quad   0x58e2ea8978b5fd6e+        .quad   0x519ef577b5e09b0a+        .quad   0x1863d7d91124cca9+        .quad   0x7ac08145b88a708e+        .quad   0x2bcd7309857031f5+        .quad   0x62337a6e8ab8fae5++        // 2^44 * 2 * B++        .quad   0x4bcef17f06ffca16+        .quad   0xde06e1db692ae16a+        .quad   0x0753702d614f42b0+        .quad   0x5f6041b45b9212d0+        .quad   0xd1ab324e1b3a1273+        .quad   0x18947cf181055340+        .quad   0x3b5d9567a98c196e+        .quad   0x7fa00425802e1e68+        .quad   0x7d531574028c2705+        .quad   0x80317d69db0d75fe+        .quad   0x30fface8ef8c8ddd+        .quad   0x7e9de97bb6c3e998++        // 2^44 * 3 * B++        .quad   0x1558967b9e6585a3+        .quad   0x97c99ce098e98b92+        .quad   0x10af149b6eb3adad+        .quad   0x42181fe8f4d38cfa+        .quad   0xf004be62a24d40dd+        .quad   0xba0659910452d41f+        .quad   0x81c45ee162a44234+        .quad   0x4cb829d8a22266ef+        .quad   0x1dbcaa8407b86681+        .quad   0x081f001e8b26753b+        .quad   0x3cd7ce6a84048e81+        .quad   0x78af11633f25f22c++        // 2^44 * 4 * B++        .quad   0x8416ebd40b50babc+        .quad   0x1508722628208bee+        .quad   0xa3148fafb9c1c36d+        .quad   0x0d07daacd32d7d5d+        .quad   0x3241c00e7d65318c+        .quad   0xe6bee5dcd0e86de7+        .quad   0x118b2dc2fbc08c26+        .quad   0x680d04a7fc603dc3+        .quad   0xf9c2414a695aa3eb+        .quad   0xdaa42c4c05a68f21+        .quad   0x7c6c23987f93963e+        .quad   0x210e8cd30c3954e3++        // 2^44 * 5 * B++        .quad   0xac4201f210a71c06+        .quad   0x6a65e0aef3bfb021+        .quad   0xbc42c35c393632f7+        .quad   0x56ea8db1865f0742+        .quad   0x2b50f16137fe6c26+        .quad   0xe102bcd856e404d8+        .quad   0x12b0f1414c561f6b+        .quad   0x51b17bc8d028ec91+        .quad   0xfff5fb4bcf535119+        .quad   0xf4989d79df1108a0+        .quad   0xbdfcea659a3ba325+        .quad   0x18a11f1174d1a6f2++        // 2^44 * 6 * B++        .quad   0x407375ab3f6bba29+        .quad   0x9ec3b6d8991e482e+        .quad   0x99c80e82e55f92e9+        .quad   0x307c13b6fb0c0ae1+        .quad   0xfbd63cdad27a5f2c+        .quad   0xf00fc4bc8aa106d7+        .quad   0x53fb5c1a8e64a430+        .quad   0x04eaabe50c1a2e85+        .quad   0x24751021cb8ab5e7+        .quad   0xfc2344495c5010eb+        .quad   0x5f1e717b4e5610a1+        .quad   0x44da5f18c2710cd5++        // 2^44 * 7 * B++        .quad   0x033cc55ff1b82eb5+        .quad   0xb15ae36d411cae52+        .quad   0xba40b6198ffbacd3+        .quad   0x768edce1532e861f+        .quad   0x9156fe6b89d8eacc+        .quad   0xe6b79451e23126a1+        .quad   0xbd7463d93944eb4e+        .quad   0x726373f6767203ae+        .quad   0xe305ca72eb7ef68a+        .quad   0x662cf31f70eadb23+        .quad   0x18f026fdb4c45b68+        .quad   0x513b5384b5d2ecbd++        // 2^44 * 8 * B++        .quad   0x46d46280c729989e+        .quad   0x4b93fbd05368a5dd+        .quad   0x63df3f81d1765a89+        .quad   0x34cebd64b9a0a223+        .quad   0x5e2702878af34ceb+        .quad   0x900b0409b946d6ae+        .quad   0x6512ebf7dabd8512+        .quad   0x61d9b76988258f81+        .quad   0xa6c5a71349b7d94b+        .quad   0xa3f3d15823eb9446+        .quad   0x0416fbd277484834+        .quad   0x69d45e6f2c70812f++        // 2^48 * 1 * B++        .quad   0xce16f74bc53c1431+        .quad   0x2b9725ce2072edde+        .quad   0xb8b9c36fb5b23ee7+        .quad   0x7e2e0e450b5cc908+        .quad   0x9fe62b434f460efb+        .quad   0xded303d4a63607d6+        .quad   0xf052210eb7a0da24+        .quad   0x237e7dbe00545b93+        .quad   0x013575ed6701b430+        .quad   0x231094e69f0bfd10+        .quad   0x75320f1583e47f22+        .quad   0x71afa699b11155e3++        // 2^48 * 2 * B++        .quad   0x65ce6f9b3953b61d+        .quad   0xc65839eaafa141e6+        .quad   0x0f435ffda9f759fe+        .quad   0x021142e9c2b1c28e+        .quad   0xea423c1c473b50d6+        .quad   0x51e87a1f3b38ef10+        .quad   0x9b84bf5fb2c9be95+        .quad   0x00731fbc78f89a1c+        .quad   0xe430c71848f81880+        .quad   0xbf960c225ecec119+        .quad   0xb6dae0836bba15e3+        .quad   0x4c4d6f3347e15808++        // 2^48 * 3 * B++        .quad   0x18f7eccfc17d1fc9+        .quad   0x6c75f5a651403c14+        .quad   0xdbde712bf7ee0cdf+        .quad   0x193fddaaa7e47a22+        .quad   0x2f0cddfc988f1970+        .quad   0x6b916227b0b9f51b+        .quad   0x6ec7b6c4779176be+        .quad   0x38bf9500a88f9fa8+        .quad   0x1fd2c93c37e8876f+        .quad   0xa2f61e5a18d1462c+        .quad   0x5080f58239241276+        .quad   0x6a6fb99ebf0d4969++        // 2^48 * 4 * B++        .quad   0x6a46c1bb560855eb+        .quad   0x2416bb38f893f09d+        .quad   0xd71d11378f71acc1+        .quad   0x75f76914a31896ea+        .quad   0xeeb122b5b6e423c6+        .quad   0x939d7010f286ff8e+        .quad   0x90a92a831dcf5d8c+        .quad   0x136fda9f42c5eb10+        .quad   0xf94cdfb1a305bdd1+        .quad   0x0f364b9d9ff82c08+        .quad   0x2a87d8a5c3bb588a+        .quad   0x022183510be8dcba++        // 2^48 * 5 * B++        .quad   0x4af766385ead2d14+        .quad   0xa08ed880ca7c5830+        .quad   0x0d13a6e610211e3d+        .quad   0x6a071ce17b806c03+        .quad   0x9d5a710143307a7f+        .quad   0xb063de9ec47da45f+        .quad   0x22bbfe52be927ad3+        .quad   0x1387c441fd40426c+        .quad   0xb5d3c3d187978af8+        .quad   0x722b5a3d7f0e4413+        .quad   0x0d7b4848bb477ca0+        .quad   0x3171b26aaf1edc92++        // 2^48 * 6 * B++        .quad   0xa92f319097564ca8+        .quad   0xff7bb84c2275e119+        .quad   0x4f55fe37a4875150+        .quad   0x221fd4873cf0835a+        .quad   0xa60db7d8b28a47d1+        .quad   0xa6bf14d61770a4f1+        .quad   0xd4a1f89353ddbd58+        .quad   0x6c514a63344243e9+        .quad   0x2322204f3a156341+        .quad   0xfb73e0e9ba0a032d+        .quad   0xfce0dd4c410f030e+        .quad   0x48daa596fb924aaa++        // 2^48 * 7 * B++        .quad   0x6eca8e665ca59cc7+        .quad   0xa847254b2e38aca0+        .quad   0x31afc708d21e17ce+        .quad   0x676dd6fccad84af7+        .quad   0x14f61d5dc84c9793+        .quad   0x9941f9e3ef418206+        .quad   0xcdf5b88f346277ac+        .quad   0x58c837fa0e8a79a9+        .quad   0x0cf9688596fc9058+        .quad   0x1ddcbbf37b56a01b+        .quad   0xdcc2e77d4935d66a+        .quad   0x1c4f73f2c6a57f0a++        // 2^48 * 8 * B++        .quad   0x0e7a4fbd305fa0bb+        .quad   0x829d4ce054c663ad+        .quad   0xf421c3832fe33848+        .quad   0x795ac80d1bf64c42+        .quad   0xb36e706efc7c3484+        .quad   0x73dfc9b4c3c1cf61+        .quad   0xeb1d79c9781cc7e5+        .quad   0x70459adb7daf675c+        .quad   0x1b91db4991b42bb3+        .quad   0x572696234b02dcca+        .quad   0x9fdf9ee51f8c78dc+        .quad   0x5fe162848ce21fd3++        // 2^52 * 1 * B++        .quad   0xe2790aae4d077c41+        .quad   0x8b938270db7469a3+        .quad   0x6eb632dc8abd16a2+        .quad   0x720814ecaa064b72+        .quad   0x315c29c795115389+        .quad   0xd7e0e507862f74ce+        .quad   0x0c4a762185927432+        .quad   0x72de6c984a25a1e4+        .quad   0xae9ab553bf6aa310+        .quad   0x050a50a9806d6e1b+        .quad   0x92bb7403adff5139+        .quad   0x0394d27645be618b++        // 2^52 * 2 * B++        .quad   0x4d572251857eedf4+        .quad   0xe3724edde19e93c5+        .quad   0x8a71420e0b797035+        .quad   0x3b3c833687abe743+        .quad   0xf5396425b23545a4+        .quad   0x15a7a27e98fbb296+        .quad   0xab6c52bc636fdd86+        .quad   0x79d995a8419334ee+        .quad   0xcd8a8ea61195dd75+        .quad   0xa504d8a81dd9a82f+        .quad   0x540dca81a35879b6+        .quad   0x60dd16a379c86a8a++        // 2^52 * 3 * B++        .quad   0x35a2c8487381e559+        .quad   0x596ffea6d78082cb+        .quad   0xcb9771ebdba7b653+        .quad   0x5a08b5019b4da685+        .quad   0x3501d6f8153e47b8+        .quad   0xb7a9675414a2f60c+        .quad   0x112ee8b6455d9523+        .quad   0x4e62a3c18112ea8a+        .quad   0xc8d4ac04516ab786+        .quad   0x595af3215295b23d+        .quad   0xd6edd234db0230c1+        .quad   0x0929efe8825b41cc++        // 2^52 * 4 * B++        .quad   0x5f0601d1cbd0f2d3+        .quad   0x736e412f6132bb7f+        .quad   0x83604432238dde87+        .quad   0x1e3a5272f5c0753c+        .quad   0x8b3172b7ad56651d+        .quad   0x01581b7a3fabd717+        .quad   0x2dc94df6424df6e4+        .quad   0x30376e5d2c29284f+        .quad   0xd2918da78159a59c+        .quad   0x6bdc1cd93f0713f3+        .quad   0x565f7a934acd6590+        .quad   0x53daacec4cb4c128++        // 2^52 * 5 * B++        .quad   0x4ca73bd79cc8a7d6+        .quad   0x4d4a738f47e9a9b2+        .quad   0xf4cbf12942f5fe00+        .quad   0x01a13ff9bdbf0752+        .quad   0x99852bc3852cfdb0+        .quad   0x2cc12e9559d6ed0b+        .quad   0x70f9e2bf9b5ac27b+        .quad   0x4f3b8c117959ae99+        .quad   0x55b6c9c82ff26412+        .quad   0x1ac4a8c91fb667a8+        .quad   0xd527bfcfeb778bf2+        .quad   0x303337da7012a3be++        // 2^52 * 6 * B++        .quad   0x955422228c1c9d7c+        .quad   0x01fac1371a9b340f+        .quad   0x7e8d9177925b48d7+        .quad   0x53f8ad5661b3e31b+        .quad   0x976d3ccbfad2fdd1+        .quad   0xcb88839737a640a8+        .quad   0x2ff00c1d6734cb25+        .quad   0x269ff4dc789c2d2b+        .quad   0x0c003fbdc08d678d+        .quad   0x4d982fa37ead2b17+        .quad   0xc07e6bcdb2e582f1+        .quad   0x296c7291df412a44++        // 2^52 * 7 * B++        .quad   0x7903de2b33daf397+        .quad   0xd0ff0619c9a624b3+        .quad   0x8a1d252b555b3e18+        .quad   0x2b6d581c52e0b7c0+        .quad   0xdfb23205dab8b59e+        .quad   0x465aeaa0c8092250+        .quad   0xd133c1189a725d18+        .quad   0x2327370261f117d1+        .quad   0x3d0543d3623e7986+        .quad   0x679414c2c278a354+        .quad   0xae43f0cc726196f6+        .quad   0x7836c41f8245eaba++        // 2^52 * 8 * B++        .quad   0xe7a254db49e95a81+        .quad   0x5192d5d008b0ad73+        .quad   0x4d20e5b1d00afc07+        .quad   0x5d55f8012cf25f38+        .quad   0xca651e848011937c+        .quad   0xc6b0c46e6ef41a28+        .quad   0xb7021ba75f3f8d52+        .quad   0x119dff99ead7b9fd+        .quad   0x43eadfcbf4b31d4d+        .quad   0xc6503f7411148892+        .quad   0xfeee68c5060d3b17+        .quad   0x329293b3dd4a0ac8++        // 2^56 * 1 * B++        .quad   0x4e59214fe194961a+        .quad   0x49be7dc70d71cd4f+        .quad   0x9300cfd23b50f22d+        .quad   0x4789d446fc917232+        .quad   0x2879852d5d7cb208+        .quad   0xb8dedd70687df2e7+        .quad   0xdc0bffab21687891+        .quad   0x2b44c043677daa35+        .quad   0x1a1c87ab074eb78e+        .quad   0xfac6d18e99daf467+        .quad   0x3eacbbcd484f9067+        .quad   0x60c52eef2bb9a4e4++        // 2^56 * 2 * B++        .quad   0x0b5d89bc3bfd8bf1+        .quad   0xb06b9237c9f3551a+        .quad   0x0e4c16b0d53028f5+        .quad   0x10bc9c312ccfcaab+        .quad   0x702bc5c27cae6d11+        .quad   0x44c7699b54a48cab+        .quad   0xefbc4056ba492eb2+        .quad   0x70d77248d9b6676d+        .quad   0xaa8ae84b3ec2a05b+        .quad   0x98699ef4ed1781e0+        .quad   0x794513e4708e85d1+        .quad   0x63755bd3a976f413++        // 2^56 * 3 * B++        .quad   0xb55fa03e2ad10853+        .quad   0x356f75909ee63569+        .quad   0x9ff9f1fdbe69b890+        .quad   0x0d8cc1c48bc16f84+        .quad   0x3dc7101897f1acb7+        .quad   0x5dda7d5ec165bbd8+        .quad   0x508e5b9c0fa1020f+        .quad   0x2763751737c52a56+        .quad   0x029402d36eb419a9+        .quad   0xf0b44e7e77b460a5+        .quad   0xcfa86230d43c4956+        .quad   0x70c2dd8a7ad166e7++        // 2^56 * 4 * B++        .quad   0x656194509f6fec0e+        .quad   0xee2e7ea946c6518d+        .quad   0x9733c1f367e09b5c+        .quad   0x2e0fac6363948495+        .quad   0x91d4967db8ed7e13+        .quad   0x74252f0ad776817a+        .quad   0xe40982e00d852564+        .quad   0x32b8613816a53ce5+        .quad   0x79e7f7bee448cd64+        .quad   0x6ac83a67087886d0+        .quad   0xf89fd4d9a0e4db2e+        .quad   0x4179215c735a4f41++        // 2^56 * 5 * B++        .quad   0x8c7094e7d7dced2a+        .quad   0x97fb8ac347d39c70+        .quad   0xe13be033a906d902+        .quad   0x700344a30cd99d76+        .quad   0xe4ae33b9286bcd34+        .quad   0xb7ef7eb6559dd6dc+        .quad   0x278b141fb3d38e1f+        .quad   0x31fa85662241c286+        .quad   0xaf826c422e3622f4+        .quad   0xc12029879833502d+        .quad   0x9bc1b7e12b389123+        .quad   0x24bb2312a9952489++        // 2^56 * 6 * B++        .quad   0xb1a8ed1732de67c3+        .quad   0x3cb49418461b4948+        .quad   0x8ebd434376cfbcd2+        .quad   0x0fee3e871e188008+        .quad   0x41f80c2af5f85c6b+        .quad   0x687284c304fa6794+        .quad   0x8945df99a3ba1bad+        .quad   0x0d1d2af9ffeb5d16+        .quad   0xa9da8aa132621edf+        .quad   0x30b822a159226579+        .quad   0x4004197ba79ac193+        .quad   0x16acd79718531d76++        // 2^56 * 7 * B++        .quad   0x72df72af2d9b1d3d+        .quad   0x63462a36a432245a+        .quad   0x3ecea07916b39637+        .quad   0x123e0ef6b9302309+        .quad   0xc959c6c57887b6ad+        .quad   0x94e19ead5f90feba+        .quad   0x16e24e62a342f504+        .quad   0x164ed34b18161700+        .quad   0x487ed94c192fe69a+        .quad   0x61ae2cea3a911513+        .quad   0x877bf6d3b9a4de27+        .quad   0x78da0fc61073f3eb++        // 2^56 * 8 * B++        .quad   0x5bf15d28e52bc66a+        .quad   0x2c47e31870f01a8e+        .quad   0x2419afbc06c28bdd+        .quad   0x2d25deeb256b173a+        .quad   0xa29f80f1680c3a94+        .quad   0x71f77e151ae9e7e6+        .quad   0x1100f15848017973+        .quad   0x054aa4b316b38ddd+        .quad   0xdfc8468d19267cb8+        .quad   0x0b28789c66e54daf+        .quad   0x2aeb1d2a666eec17+        .quad   0x134610a6ab7da760++        // 2^60 * 1 * B++        .quad   0xcaf55ec27c59b23f+        .quad   0x99aeed3e154d04f2+        .quad   0x68441d72e14141f4+        .quad   0x140345133932a0a2+        .quad   0xd91430e0dc028c3c+        .quad   0x0eb955a85217c771+        .quad   0x4b09e1ed2c99a1fa+        .quad   0x42881af2bd6a743c+        .quad   0x7bfec69aab5cad3d+        .quad   0xc23e8cd34cb2cfad+        .quad   0x685dd14bfb37d6a2+        .quad   0x0ad6d64415677a18++        // 2^60 * 2 * B++        .quad   0x781a439e417becb5+        .quad   0x4ac5938cd10e0266+        .quad   0x5da385110692ac24+        .quad   0x11b065a2ade31233+        .quad   0x7914892847927e9f+        .quad   0x33dad6ef370aa877+        .quad   0x1f8f24fa11122703+        .quad   0x5265ac2f2adf9592+        .quad   0x405fdd309afcb346+        .quad   0xd9723d4428e63f54+        .quad   0x94c01df05f65aaae+        .quad   0x43e4dc3ae14c0809++        // 2^60 * 3 * B++        .quad   0xbc12c7f1a938a517+        .quad   0x473028ab3180b2e1+        .quad   0x3f78571efbcd254a+        .quad   0x74e534426ff6f90f+        .quad   0xea6f7ac3adc2c6a3+        .quad   0xd0e928f6e9717c94+        .quad   0xe2d379ead645eaf5+        .quad   0x46dd8785c51ffbbe+        .quad   0x709801be375c8898+        .quad   0x4b06dab5e3fd8348+        .quad   0x75880ced27230714+        .quad   0x2b09468fdd2f4c42++        // 2^60 * 4 * B++        .quad   0x97c749eeb701cb96+        .quad   0x83f438d4b6a369c3+        .quad   0x62962b8b9a402cd9+        .quad   0x6976c7509888df7b+        .quad   0x5b97946582ffa02a+        .quad   0xda096a51fea8f549+        .quad   0xa06351375f77af9b+        .quad   0x1bcfde61201d1e76+        .quad   0x4a4a5490246a59a2+        .quad   0xd63ebddee87fdd90+        .quad   0xd9437c670d2371fa+        .quad   0x69e87308d30f8ed6++        // 2^60 * 5 * B++        .quad   0x435a8bb15656beb0+        .quad   0xf8fac9ba4f4d5bca+        .quad   0xb9b278c41548c075+        .quad   0x3eb0ef76e892b622+        .quad   0x0f80bf028bc80303+        .quad   0x6aae16b37a18cefb+        .quad   0xdd47ea47d72cd6a3+        .quad   0x61943588f4ed39aa+        .quad   0xd26e5c3e91039f85+        .quad   0xc0e9e77df6f33aa9+        .quad   0xe8968c5570066a93+        .quad   0x3c34d1881faaaddd++        // 2^60 * 6 * B++        .quad   0x3f9d2b5ea09f9ec0+        .quad   0x1dab3b6fb623a890+        .quad   0xa09ba3ea72d926c4+        .quad   0x374193513fd8b36d+        .quad   0xbd5b0b8f2fffe0d9+        .quad   0x6aa254103ed24fb9+        .quad   0x2ac7d7bcb26821c4+        .quad   0x605b394b60dca36a+        .quad   0xb4e856e45a9d1ed2+        .quad   0xefe848766c97a9a2+        .quad   0xb104cf641e5eee7d+        .quad   0x2f50b81c88a71c8f++        // 2^60 * 7 * B++        .quad   0x31723c61fc6811bb+        .quad   0x9cb450486211800f+        .quad   0x768933d347995753+        .quad   0x3491a53502752fcd+        .quad   0x2b552ca0a7da522a+        .quad   0x3230b336449b0250+        .quad   0xf2c4c5bca4b99fb9+        .quad   0x7b2c674958074a22+        .quad   0xd55165883ed28cdf+        .quad   0x12d84fd2d362de39+        .quad   0x0a874ad3e3378e4f+        .quad   0x000d2b1f7c763e74++        // 2^60 * 8 * B++        .quad   0x3d420811d06d4a67+        .quad   0xbefc048590e0ffe3+        .quad   0xf870c6b7bd487bde+        .quad   0x6e2a7316319afa28+        .quad   0x9624778c3e94a8ab+        .quad   0x0ad6f3cee9a78bec+        .quad   0x948ac7810d743c4f+        .quad   0x76627935aaecfccc+        .quad   0x56a8ac24d6d59a9f+        .quad   0xc8db753e3096f006+        .quad   0x477f41e68f4c5299+        .quad   0x588d851cf6c86114++        // 2^64 * 1 * B++        .quad   0x51138ec78df6b0fe+        .quad   0x5397da89e575f51b+        .quad   0x09207a1d717af1b9+        .quad   0x2102fdba2b20d650+        .quad   0xcd2a65e777d1f515+        .quad   0x548991878faa60f1+        .quad   0xb1b73bbcdabc06e5+        .quad   0x654878cba97cc9fb+        .quad   0x969ee405055ce6a1+        .quad   0x36bca7681251ad29+        .quad   0x3a1af517aa7da415+        .quad   0x0ad725db29ecb2ba++        // 2^64 * 2 * B++        .quad   0xdc4267b1834e2457+        .quad   0xb67544b570ce1bc5+        .quad   0x1af07a0bf7d15ed7+        .quad   0x4aefcffb71a03650+        .quad   0xfec7bc0c9b056f85+        .quad   0x537d5268e7f5ffd7+        .quad   0x77afc6624312aefa+        .quad   0x4f675f5302399fd9+        .quad   0xc32d36360415171e+        .quad   0xcd2bef118998483b+        .quad   0x870a6eadd0945110+        .quad   0x0bccbb72a2a86561++        // 2^64 * 3 * B++        .quad   0x185e962feab1a9c8+        .quad   0x86e7e63565147dcd+        .quad   0xb092e031bb5b6df2+        .quad   0x4024f0ab59d6b73e+        .quad   0x186d5e4c50fe1296+        .quad   0xe0397b82fee89f7e+        .quad   0x3bc7f6c5507031b0+        .quad   0x6678fd69108f37c2+        .quad   0x1586fa31636863c2+        .quad   0x07f68c48572d33f2+        .quad   0x4f73cc9f789eaefc+        .quad   0x2d42e2108ead4701++        // 2^64 * 4 * B++        .quad   0x97f5131594dfd29b+        .quad   0x6155985d313f4c6a+        .quad   0xeba13f0708455010+        .quad   0x676b2608b8d2d322+        .quad   0x21717b0d0f537593+        .quad   0x914e690b131e064c+        .quad   0x1bb687ae752ae09f+        .quad   0x420bf3a79b423c6e+        .quad   0x8138ba651c5b2b47+        .quad   0x8671b6ec311b1b80+        .quad   0x7bff0cb1bc3135b0+        .quad   0x745d2ffa9c0cf1e0++        // 2^64 * 5 * B++        .quad   0xbf525a1e2bc9c8bd+        .quad   0xea5b260826479d81+        .quad   0xd511c70edf0155db+        .quad   0x1ae23ceb960cf5d0+        .quad   0x6036df5721d34e6a+        .quad   0xb1db8827997bb3d0+        .quad   0xd3c209c3c8756afa+        .quad   0x06e15be54c1dc839+        .quad   0x5b725d871932994a+        .quad   0x32351cb5ceb1dab0+        .quad   0x7dc41549dab7ca05+        .quad   0x58ded861278ec1f7++        // 2^64 * 6 * B++        .quad   0xd8173793f266c55c+        .quad   0xc8c976c5cc454e49+        .quad   0x5ce382f8bc26c3a8+        .quad   0x2ff39de85485f6f9+        .quad   0x2dfb5ba8b6c2c9a8+        .quad   0x48eeef8ef52c598c+        .quad   0x33809107f12d1573+        .quad   0x08ba696b531d5bd8+        .quad   0x77ed3eeec3efc57a+        .quad   0x04e05517d4ff4811+        .quad   0xea3d7a3ff1a671cb+        .quad   0x120633b4947cfe54++        // 2^64 * 7 * B++        .quad   0x0b94987891610042+        .quad   0x4ee7b13cecebfae8+        .quad   0x70be739594f0a4c0+        .quad   0x35d30a99b4d59185+        .quad   0x82bd31474912100a+        .quad   0xde237b6d7e6fbe06+        .quad   0xe11e761911ea79c6+        .quad   0x07433be3cb393bde+        .quad   0xff7944c05ce997f4+        .quad   0x575d3de4b05c51a3+        .quad   0x583381fd5a76847c+        .quad   0x2d873ede7af6da9f++        // 2^64 * 8 * B++        .quad   0x157a316443373409+        .quad   0xfab8b7eef4aa81d9+        .quad   0xb093fee6f5a64806+        .quad   0x2e773654707fa7b6+        .quad   0xaa6202e14e5df981+        .quad   0xa20d59175015e1f5+        .quad   0x18a275d3bae21d6c+        .quad   0x0543618a01600253+        .quad   0x0deabdf4974c23c1+        .quad   0xaa6f0a259dce4693+        .quad   0x04202cb8a29aba2c+        .quad   0x4b1443362d07960d++        // 2^68 * 1 * B++        .quad   0x47b837f753242cec+        .quad   0x256dc48cc04212f2+        .quad   0xe222fbfbe1d928c5+        .quad   0x48ea295bad8a2c07+        .quad   0x299b1c3f57c5715e+        .quad   0x96cb929e6b686d90+        .quad   0x3004806447235ab3+        .quad   0x2c435c24a44d9fe1+        .quad   0x0607c97c80f8833f+        .quad   0x0e851578ca25ec5b+        .quad   0x54f7450b161ebb6f+        .quad   0x7bcb4792a0def80e++        // 2^68 * 2 * B++        .quad   0x8487e3d02bc73659+        .quad   0x4baf8445059979df+        .quad   0xd17c975adcad6fbf+        .quad   0x57369f0bdefc96b6+        .quad   0x1cecd0a0045224c2+        .quad   0x757f1b1b69e53952+        .quad   0x775b7a925289f681+        .quad   0x1b6cc62016736148+        .quad   0xf1a9990175638698+        .quad   0x353dd1beeeaa60d3+        .quad   0x849471334c9ba488+        .quad   0x63fa6e6843ade311++        // 2^68 * 3 * B++        .quad   0xd15c20536597c168+        .quad   0x9f73740098d28789+        .quad   0x18aee7f13257ba1f+        .quad   0x3418bfda07346f14+        .quad   0x2195becdd24b5eb7+        .quad   0x5e41f18cc0cd44f9+        .quad   0xdf28074441ca9ede+        .quad   0x07073b98f35b7d67+        .quad   0xd03c676c4ce530d4+        .quad   0x0b64c0473b5df9f4+        .quad   0x065cef8b19b3a31e+        .quad   0x3084d661533102c9++        // 2^68 * 4 * B++        .quad   0xe1f6b79ebf8469ad+        .quad   0x15801004e2663135+        .quad   0x9a498330af74181b+        .quad   0x3ba2504f049b673c+        .quad   0x9a6ce876760321fd+        .quad   0x7fe2b5109eb63ad8+        .quad   0x00e7d4ae8ac80592+        .quad   0x73d86b7abb6f723a+        .quad   0x0b52b5606dba5ab6+        .quad   0xa9134f0fbbb1edab+        .quad   0x30a9520d9b04a635+        .quad   0x6813b8f37973e5db++        // 2^68 * 5 * B++        .quad   0x9854b054334127c1+        .quad   0x105d047882fbff25+        .quad   0xdb49f7f944186f4f+        .quad   0x1768e838bed0b900+        .quad   0xf194ca56f3157e29+        .quad   0x136d35705ef528a5+        .quad   0xdd4cef778b0599bc+        .quad   0x7d5472af24f833ed+        .quad   0xd0ef874daf33da47+        .quad   0x00d3be5db6e339f9+        .quad   0x3f2a8a2f9c9ceece+        .quad   0x5d1aeb792352435a++        // 2^68 * 6 * B++        .quad   0xf59e6bb319cd63ca+        .quad   0x670c159221d06839+        .quad   0xb06d565b2150cab6+        .quad   0x20fb199d104f12a3+        .quad   0x12c7bfaeb61ba775+        .quad   0xb84e621fe263bffd+        .quad   0x0b47a5c35c840dcf+        .quad   0x7e83be0bccaf8634+        .quad   0x61943dee6d99c120+        .quad   0x86101f2e460b9fe0+        .quad   0x6bb2f1518ee8598d+        .quad   0x76b76289fcc475cc++        // 2^68 * 7 * B++        .quad   0x791b4cc1756286fa+        .quad   0xdbced317d74a157c+        .quad   0x7e732421ea72bde6+        .quad   0x01fe18491131c8e9+        .quad   0x4245f1a1522ec0b3+        .quad   0x558785b22a75656d+        .quad   0x1d485a2548a1b3c0+        .quad   0x60959eccd58fe09f+        .quad   0x3ebfeb7ba8ed7a09+        .quad   0x49fdc2bbe502789c+        .quad   0x44ebce5d3c119428+        .quad   0x35e1eb55be947f4a++        // 2^68 * 8 * B++        .quad   0xdbdae701c5738dd3+        .quad   0xf9c6f635b26f1bee+        .quad   0x61e96a8042f15ef4+        .quad   0x3aa1d11faf60a4d8+        .quad   0x14fd6dfa726ccc74+        .quad   0x3b084cfe2f53b965+        .quad   0xf33ae4f552a2c8b4+        .quad   0x59aab07a0d40166a+        .quad   0x77bcec4c925eac25+        .quad   0x1848718460137738+        .quad   0x5b374337fea9f451+        .quad   0x1865e78ec8e6aa46++        // 2^72 * 1 * B++        .quad   0xccc4b7c7b66e1f7a+        .quad   0x44157e25f50c2f7e+        .quad   0x3ef06dfc713eaf1c+        .quad   0x582f446752da63f7+        .quad   0x967c54e91c529ccb+        .quad   0x30f6269264c635fb+        .quad   0x2747aff478121965+        .quad   0x17038418eaf66f5c+        .quad   0xc6317bd320324ce4+        .quad   0xa81042e8a4488bc4+        .quad   0xb21ef18b4e5a1364+        .quad   0x0c2a1c4bcda28dc9++        // 2^72 * 2 * B++        .quad   0xd24dc7d06f1f0447+        .quad   0xb2269e3edb87c059+        .quad   0xd15b0272fbb2d28f+        .quad   0x7c558bd1c6f64877+        .quad   0xedc4814869bd6945+        .quad   0x0d6d907dbe1c8d22+        .quad   0xc63bd212d55cc5ab+        .quad   0x5a6a9b30a314dc83+        .quad   0xd0ec1524d396463d+        .quad   0x12bb628ac35a24f0+        .quad   0xa50c3a791cbc5fa4+        .quad   0x0404a5ca0afbafc3++        // 2^72 * 3 * B++        .quad   0x8c1f40070aa743d6+        .quad   0xccbad0cb5b265ee8+        .quad   0x574b046b668fd2de+        .quad   0x46395bfdcadd9633+        .quad   0x62bc9e1b2a416fd1+        .quad   0xb5c6f728e350598b+        .quad   0x04343fd83d5d6967+        .quad   0x39527516e7f8ee98+        .quad   0x117fdb2d1a5d9a9c+        .quad   0x9c7745bcd1005c2a+        .quad   0xefd4bef154d56fea+        .quad   0x76579a29e822d016++        // 2^72 * 4 * B++        .quad   0x45b68e7e49c02a17+        .quad   0x23cd51a2bca9a37f+        .quad   0x3ed65f11ec224c1b+        .quad   0x43a384dc9e05bdb1+        .quad   0x333cb51352b434f2+        .quad   0xd832284993de80e1+        .quad   0xb5512887750d35ce+        .quad   0x02c514bb2a2777c1+        .quad   0x684bd5da8bf1b645+        .quad   0xfb8bd37ef6b54b53+        .quad   0x313916d7a9b0d253+        .quad   0x1160920961548059++        // 2^72 * 5 * B++        .quad   0xb44d166929dacfaa+        .quad   0xda529f4c8413598f+        .quad   0xe9ef63ca453d5559+        .quad   0x351e125bc5698e0b+        .quad   0x7a385616369b4dcd+        .quad   0x75c02ca7655c3563+        .quad   0x7dc21bf9d4f18021+        .quad   0x2f637d7491e6e042+        .quad   0xd4b49b461af67bbe+        .quad   0xd603037ac8ab8961+        .quad   0x71dee19ff9a699fb+        .quad   0x7f182d06e7ce2a9a++        // 2^72 * 6 * B++        .quad   0x7a7c8e64ab0168ec+        .quad   0xcb5a4a5515edc543+        .quad   0x095519d347cd0eda+        .quad   0x67d4ac8c343e93b0+        .quad   0x09454b728e217522+        .quad   0xaa58e8f4d484b8d8+        .quad   0xd358254d7f46903c+        .quad   0x44acc043241c5217+        .quad   0x1c7d6bbb4f7a5777+        .quad   0x8b35fed4918313e1+        .quad   0x4adca1c6c96b4684+        .quad   0x556d1c8312ad71bd++        // 2^72 * 7 * B++        .quad   0x17ef40e30c8d3982+        .quad   0x31f7073e15a3fa34+        .quad   0x4f21f3cb0773646e+        .quad   0x746c6c6d1d824eff+        .quad   0x81f06756b11be821+        .quad   0x0faff82310a3f3dd+        .quad   0xf8b2d0556a99465d+        .quad   0x097abe38cc8c7f05+        .quad   0x0c49c9877ea52da4+        .quad   0x4c4369559bdc1d43+        .quad   0x022c3809f7ccebd2+        .quad   0x577e14a34bee84bd++        // 2^72 * 8 * B++        .quad   0xf0e268ac61a73b0a+        .quad   0xf2fafa103791a5f5+        .quad   0xc1e13e826b6d00e9+        .quad   0x60fa7ee96fd78f42+        .quad   0x94fecebebd4dd72b+        .quad   0xf46a4fda060f2211+        .quad   0x124a5977c0c8d1ff+        .quad   0x705304b8fb009295+        .quad   0xb63d1d354d296ec6+        .quad   0xf3c3053e5fad31d8+        .quad   0x670b958cb4bd42ec+        .quad   0x21398e0ca16353fd++        // 2^76 * 1 * B++        .quad   0x216ab2ca8da7d2ef+        .quad   0x366ad9dd99f42827+        .quad   0xae64b9004fdd3c75+        .quad   0x403a395b53909e62+        .quad   0x86c5fc16861b7e9a+        .quad   0xf6a330476a27c451+        .quad   0x01667267a1e93597+        .quad   0x05ffb9cd6082dfeb+        .quad   0xa617fa9ff53f6139+        .quad   0x60f2b5e513e66cb6+        .quad   0xd7a8beefb3448aa4+        .quad   0x7a2932856f5ea192++        // 2^76 * 2 * B++        .quad   0x0b39d761b02de888+        .quad   0x5f550e7ed2414e1f+        .quad   0xa6bfa45822e1a940+        .quad   0x050a2f7dfd447b99+        .quad   0xb89c444879639302+        .quad   0x4ae4f19350c67f2c+        .quad   0xf0b35da8c81af9c6+        .quad   0x39d0003546871017+        .quad   0x437c3b33a650db77+        .quad   0x6bafe81dbac52bb2+        .quad   0xfe99402d2db7d318+        .quad   0x2b5b7eec372ba6ce++        // 2^76 * 3 * B++        .quad   0xb3bc4bbd83f50eef+        .quad   0x508f0c998c927866+        .quad   0x43e76587c8b7e66e+        .quad   0x0f7655a3a47f98d9+        .quad   0xa694404d613ac8f4+        .quad   0x500c3c2bfa97e72c+        .quad   0x874104d21fcec210+        .quad   0x1b205fb38604a8ee+        .quad   0x55ecad37d24b133c+        .quad   0x441e147d6038c90b+        .quad   0x656683a1d62c6fee+        .quad   0x0157d5dc87e0ecae++        // 2^76 * 4 * B++        .quad   0xf2a7af510354c13d+        .quad   0xd7a0b145aa372b60+        .quad   0x2869b96a05a3d470+        .quad   0x6528e42d82460173+        .quad   0x95265514d71eb524+        .quad   0xe603d8815df14593+        .quad   0x147cdf410d4de6b7+        .quad   0x5293b1730437c850+        .quad   0x23d0e0814bccf226+        .quad   0x92c745cd8196fb93+        .quad   0x8b61796c59541e5b+        .quad   0x40a44df0c021f978++        // 2^76 * 5 * B++        .quad   0xdaa869894f20ea6a+        .quad   0xea14a3d14c620618+        .quad   0x6001fccb090bf8be+        .quad   0x35f4e822947e9cf0+        .quad   0x86c96e514bc5d095+        .quad   0xf20d4098fca6804a+        .quad   0x27363d89c826ea5d+        .quad   0x39ca36565719cacf+        .quad   0x97506f2f6f87b75c+        .quad   0xc624aea0034ae070+        .quad   0x1ec856e3aad34dd6+        .quad   0x055b0be0e440e58f++        // 2^76 * 6 * B++        .quad   0x6469a17d89735d12+        .quad   0xdb6f27d5e662b9f1+        .quad   0x9fcba3286a395681+        .quad   0x363b8004d269af25+        .quad   0x4d12a04b6ea33da2+        .quad   0x57cf4c15e36126dd+        .quad   0x90ec9675ee44d967+        .quad   0x64ca348d2a985aac+        .quad   0x99588e19e4c4912d+        .quad   0xefcc3b4e1ca5ce6b+        .quad   0x4522ea60fa5b98d5+        .quad   0x7064bbab1de4a819++        // 2^76 * 7 * B++        .quad   0xb919e1515a770641+        .quad   0xa9a2e2c74e7f8039+        .quad   0x7527250b3df23109+        .quad   0x756a7330ac27b78b+        .quad   0xa290c06142542129+        .quad   0xf2e2c2aebe8d5b90+        .quad   0xcf2458db76abfe1b+        .quad   0x02157ade83d626bf+        .quad   0x3e46972a1b9a038b+        .quad   0x2e4ee66a7ee03fb4+        .quad   0x81a248776edbb4ca+        .quad   0x1a944ee88ecd0563++        // 2^76 * 8 * B++        .quad   0xd5a91d1151039372+        .quad   0x2ed377b799ca26de+        .quad   0xa17202acfd366b6b+        .quad   0x0730291bd6901995+        .quad   0xbb40a859182362d6+        .quad   0xb99f55778a4d1abb+        .quad   0x8d18b427758559f6+        .quad   0x26c20fe74d26235a+        .quad   0x648d1d9fe9cc22f5+        .quad   0x66bc561928dd577c+        .quad   0x47d3ed21652439d1+        .quad   0x49d271acedaf8b49++        // 2^80 * 1 * B++        .quad   0x89f5058a382b33f3+        .quad   0x5ae2ba0bad48c0b4+        .quad   0x8f93b503a53db36e+        .quad   0x5aa3ed9d95a232e6+        .quad   0x2798aaf9b4b75601+        .quad   0x5eac72135c8dad72+        .quad   0xd2ceaa6161b7a023+        .quad   0x1bbfb284e98f7d4e+        .quad   0x656777e9c7d96561+        .quad   0xcb2b125472c78036+        .quad   0x65053299d9506eee+        .quad   0x4a07e14e5e8957cc++        // 2^80 * 2 * B++        .quad   0x4ee412cb980df999+        .quad   0xa315d76f3c6ec771+        .quad   0xbba5edde925c77fd+        .quad   0x3f0bac391d313402+        .quad   0x240b58cdc477a49b+        .quad   0xfd38dade6447f017+        .quad   0x19928d32a7c86aad+        .quad   0x50af7aed84afa081+        .quad   0x6e4fde0115f65be5+        .quad   0x29982621216109b2+        .quad   0x780205810badd6d9+        .quad   0x1921a316baebd006++        // 2^80 * 3 * B++        .quad   0x89422f7edfb870fc+        .quad   0x2c296beb4f76b3bd+        .quad   0x0738f1d436c24df7+        .quad   0x6458df41e273aeb0+        .quad   0xd75aad9ad9f3c18b+        .quad   0x566a0eef60b1c19c+        .quad   0x3e9a0bac255c0ed9+        .quad   0x7b049deca062c7f5+        .quad   0xdccbe37a35444483+        .quad   0x758879330fedbe93+        .quad   0x786004c312c5dd87+        .quad   0x6093dccbc2950e64++        // 2^80 * 4 * B++        .quad   0x1ff39a8585e0706d+        .quad   0x36d0a5d8b3e73933+        .quad   0x43b9f2e1718f453b+        .quad   0x57d1ea084827a97c+        .quad   0x6bdeeebe6084034b+        .quad   0x3199c2b6780fb854+        .quad   0x973376abb62d0695+        .quad   0x6e3180c98b647d90+        .quad   0xee7ab6e7a128b071+        .quad   0xa4c1596d93a88baa+        .quad   0xf7b4de82b2216130+        .quad   0x363e999ddd97bd18++        // 2^80 * 5 * B++        .quad   0x96a843c135ee1fc4+        .quad   0x976eb35508e4c8cf+        .quad   0xb42f6801b58cd330+        .quad   0x48ee9b78693a052b+        .quad   0x2f1848dce24baec6+        .quad   0x769b7255babcaf60+        .quad   0x90cb3c6e3cefe931+        .quad   0x231f979bc6f9b355+        .quad   0x5c31de4bcc2af3c6+        .quad   0xb04bb030fe208d1f+        .quad   0xb78d7009c14fb466+        .quad   0x079bfa9b08792413++        // 2^80 * 6 * B++        .quad   0xe3903a51da300df4+        .quad   0x843964233da95ab0+        .quad   0xed3cf12d0b356480+        .quad   0x038c77f684817194+        .quad   0xf3c9ed80a2d54245+        .quad   0x0aa08b7877f63952+        .quad   0xd76dac63d1085475+        .quad   0x1ef4fb159470636b+        .quad   0x854e5ee65b167bec+        .quad   0x59590a4296d0cdc2+        .quad   0x72b2df3498102199+        .quad   0x575ee92a4a0bff56++        // 2^80 * 7 * B++        .quad   0xd4c080908a182fcf+        .quad   0x30e170c299489dbd+        .quad   0x05babd5752f733de+        .quad   0x43d4e7112cd3fd00+        .quad   0x5d46bc450aa4d801+        .quad   0xc3af1227a533b9d8+        .quad   0x389e3b262b8906c2+        .quad   0x200a1e7e382f581b+        .quad   0x518db967eaf93ac5+        .quad   0x71bc989b056652c0+        .quad   0xfe2b85d9567197f5+        .quad   0x050eca52651e4e38++        // 2^80 * 8 * B++        .quad   0xc3431ade453f0c9c+        .quad   0xe9f5045eff703b9b+        .quad   0xfcd97ac9ed847b3d+        .quad   0x4b0ee6c21c58f4c6+        .quad   0x97ac397660e668ea+        .quad   0x9b19bbfe153ab497+        .quad   0x4cb179b534eca79f+        .quad   0x6151c09fa131ae57+        .quad   0x3af55c0dfdf05d96+        .quad   0xdd262ee02ab4ee7a+        .quad   0x11b2bb8712171709+        .quad   0x1fef24fa800f030b++        // 2^84 * 1 * B++        .quad   0xb496123a6b6c6609+        .quad   0xa750fe8580ab5938+        .quad   0xf471bf39b7c27a5f+        .quad   0x507903ce77ac193c+        .quad   0xff91a66a90166220+        .quad   0xf22552ae5bf1e009+        .quad   0x7dff85d87f90df7c+        .quad   0x4f620ffe0c736fb9+        .quad   0x62f90d65dfde3e34+        .quad   0xcf28c592b9fa5fad+        .quad   0x99c86ef9c6164510+        .quad   0x25d448044a256c84++        // 2^84 * 2 * B++        .quad   0xbd68230ec7e9b16f+        .quad   0x0eb1b9c1c1c5795d+        .quad   0x7943c8c495b6b1ff+        .quad   0x2f9faf620bbacf5e+        .quad   0x2c7c4415c9022b55+        .quad   0x56a0d241812eb1fe+        .quad   0xf02ea1c9d7b65e0d+        .quad   0x4180512fd5323b26+        .quad   0xa4ff3e698a48a5db+        .quad   0xba6a3806bd95403b+        .quad   0x9f7ce1af47d5b65d+        .quad   0x15e087e55939d2fb++        // 2^84 * 3 * B++        .quad   0x12207543745c1496+        .quad   0xdaff3cfdda38610c+        .quad   0xe4e797272c71c34f+        .quad   0x39c07b1934bdede9+        .quad   0x8894186efb963f38+        .quad   0x48a00e80dc639bd5+        .quad   0xa4e8092be96c1c99+        .quad   0x5a097d54ca573661+        .quad   0x2d45892b17c9e755+        .quad   0xd033fd7289308df8+        .quad   0x6c2fe9d9525b8bd9+        .quad   0x2edbecf1c11cc079++        // 2^84 * 4 * B++        .quad   0x1616a4e3c715a0d2+        .quad   0x53623cb0f8341d4d+        .quad   0x96ef5329c7e899cb+        .quad   0x3d4e8dbba668baa6+        .quad   0xee0f0fddd087a25f+        .quad   0x9c7531555c3e34ee+        .quad   0x660c572e8fab3ab5+        .quad   0x0854fc44544cd3b2+        .quad   0x61eba0c555edad19+        .quad   0x24b533fef0a83de6+        .quad   0x3b77042883baa5f8+        .quad   0x678f82b898a47e8d++        // 2^84 * 5 * B++        .quad   0xb1491d0bd6900c54+        .quad   0x3539722c9d132636+        .quad   0x4db928920b362bc9+        .quad   0x4d7cd1fea68b69df+        .quad   0x1e09d94057775696+        .quad   0xeed1265c3cd951db+        .quad   0xfa9dac2b20bce16f+        .quad   0x0f7f76e0e8d089f4+        .quad   0x36d9ebc5d485b00c+        .quad   0xa2596492e4adb365+        .quad   0xc1659480c2119ccd+        .quad   0x45306349186e0d5f++        // 2^84 * 6 * B++        .quad   0x94ddd0c1a6cdff1d+        .quad   0x55f6f115e84213ae+        .quad   0x6c935f85992fcf6a+        .quad   0x067ee0f54a37f16f+        .quad   0x96a414ec2b072491+        .quad   0x1bb2218127a7b65b+        .quad   0x6d2849596e8a4af0+        .quad   0x65f3b08ccd27765f+        .quad   0xecb29fff199801f7+        .quad   0x9d361d1fa2a0f72f+        .quad   0x25f11d2375fd2f49+        .quad   0x124cefe80fe10fe2++        // 2^84 * 7 * B++        .quad   0x4c126cf9d18df255+        .quad   0xc1d471e9147a63b6+        .quad   0x2c6d3c73f3c93b5f+        .quad   0x6be3a6a2e3ff86a2+        .quad   0x1518e85b31b16489+        .quad   0x8faadcb7db710bfb+        .quad   0x39b0bdf4a14ae239+        .quad   0x05f4cbea503d20c1+        .quad   0xce040e9ec04145bc+        .quad   0xc71ff4e208f6834c+        .quad   0xbd546e8dab8847a3+        .quad   0x64666aa0a4d2aba5++        // 2^84 * 8 * B++        .quad   0x6841435a7c06d912+        .quad   0xca123c21bb3f830b+        .quad   0xd4b37b27b1cbe278+        .quad   0x1d753b84c76f5046+        .quad   0xb0c53bf73337e94c+        .quad   0x7cb5697e11e14f15+        .quad   0x4b84abac1930c750+        .quad   0x28dd4abfe0640468+        .quad   0x7dc0b64c44cb9f44+        .quad   0x18a3e1ace3925dbf+        .quad   0x7a3034862d0457c4+        .quad   0x4c498bf78a0c892e++        // 2^88 * 1 * B++        .quad   0x37d653fb1aa73196+        .quad   0x0f9495303fd76418+        .quad   0xad200b09fb3a17b2+        .quad   0x544d49292fc8613e+        .quad   0x22d2aff530976b86+        .quad   0x8d90b806c2d24604+        .quad   0xdca1896c4de5bae5+        .quad   0x28005fe6c8340c17+        .quad   0x6aefba9f34528688+        .quad   0x5c1bff9425107da1+        .quad   0xf75bbbcd66d94b36+        .quad   0x72e472930f316dfa++        // 2^88 * 2 * B++        .quad   0x2695208c9781084f+        .quad   0xb1502a0b23450ee1+        .quad   0xfd9daea603efde02+        .quad   0x5a9d2e8c2733a34c+        .quad   0x07f3f635d32a7627+        .quad   0x7aaa4d865f6566f0+        .quad   0x3c85e79728d04450+        .quad   0x1fee7f000fe06438+        .quad   0x765305da03dbf7e5+        .quad   0xa4daf2491434cdbd+        .quad   0x7b4ad5cdd24a88ec+        .quad   0x00f94051ee040543++        // 2^88 * 3 * B++        .quad   0x8d356b23c3d330b2+        .quad   0xf21c8b9bb0471b06+        .quad   0xb36c316c6e42b83c+        .quad   0x07d79c7e8beab10d+        .quad   0xd7ef93bb07af9753+        .quad   0x583ed0cf3db766a7+        .quad   0xce6998bf6e0b1ec5+        .quad   0x47b7ffd25dd40452+        .quad   0x87fbfb9cbc08dd12+        .quad   0x8a066b3ae1eec29b+        .quad   0x0d57242bdb1fc1bf+        .quad   0x1c3520a35ea64bb6++        // 2^88 * 4 * B++        .quad   0x80d253a6bccba34a+        .quad   0x3e61c3a13838219b+        .quad   0x90c3b6019882e396+        .quad   0x1c3d05775d0ee66f+        .quad   0xcda86f40216bc059+        .quad   0x1fbb231d12bcd87e+        .quad   0xb4956a9e17c70990+        .quad   0x38750c3b66d12e55+        .quad   0x692ef1409422e51a+        .quad   0xcbc0c73c2b5df671+        .quad   0x21014fe7744ce029+        .quad   0x0621e2c7d330487c++        // 2^88 * 5 * B++        .quad   0xaf9860cc8259838d+        .quad   0x90ea48c1c69f9adc+        .quad   0x6526483765581e30+        .quad   0x0007d6097bd3a5bc+        .quad   0xb7ae1796b0dbf0f3+        .quad   0x54dfafb9e17ce196+        .quad   0x25923071e9aaa3b4+        .quad   0x5d8e589ca1002e9d+        .quad   0xc0bf1d950842a94b+        .quad   0xb2d3c363588f2e3e+        .quad   0x0a961438bb51e2ef+        .quad   0x1583d7783c1cbf86++        // 2^88 * 6 * B++        .quad   0xeceea2ef5da27ae1+        .quad   0x597c3a1455670174+        .quad   0xc9a62a126609167a+        .quad   0x252a5f2e81ed8f70+        .quad   0x90034704cc9d28c7+        .quad   0x1d1b679ef72cc58f+        .quad   0x16e12b5fbe5b8726+        .quad   0x4958064e83c5580a+        .quad   0x0d2894265066e80d+        .quad   0xfcc3f785307c8c6b+        .quad   0x1b53da780c1112fd+        .quad   0x079c170bd843b388++        // 2^88 * 7 * B++        .quad   0x0506ece464fa6fff+        .quad   0xbee3431e6205e523+        .quad   0x3579422451b8ea42+        .quad   0x6dec05e34ac9fb00+        .quad   0xcdd6cd50c0d5d056+        .quad   0x9af7686dbb03573b+        .quad   0x3ca6723ff3c3ef48+        .quad   0x6768c0d7317b8acc+        .quad   0x94b625e5f155c1b3+        .quad   0x417bf3a7997b7b91+        .quad   0xc22cbddc6d6b2600+        .quad   0x51445e14ddcd52f4++        // 2^88 * 8 * B++        .quad   0x57502b4b3b144951+        .quad   0x8e67ff6b444bbcb3+        .quad   0xb8bd6927166385db+        .quad   0x13186f31e39295c8+        .quad   0x893147ab2bbea455+        .quad   0x8c53a24f92079129+        .quad   0x4b49f948be30f7a7+        .quad   0x12e990086e4fd43d+        .quad   0xf10c96b37fdfbb2e+        .quad   0x9f9a935e121ceaf9+        .quad   0xdf1136c43a5b983f+        .quad   0x77b2e3f05d3e99af++        // 2^92 * 1 * B++        .quad   0xfd0d75879cf12657+        .quad   0xe82fef94e53a0e29+        .quad   0xcc34a7f05bbb4be7+        .quad   0x0b251172a50c38a2+        .quad   0x9532f48fcc5cd29b+        .quad   0x2ba851bea3ce3671+        .quad   0x32dacaa051122941+        .quad   0x478d99d9350004f2+        .quad   0x1d5ad94890bb02c0+        .quad   0x50e208b10ec25115+        .quad   0xa26a22894ef21702+        .quad   0x4dc923343b524805++        // 2^92 * 2 * B++        .quad   0xe3828c400f8086b6+        .quad   0x3f77e6f7979f0dc8+        .quad   0x7ef6de304df42cb4+        .quad   0x5265797cb6abd784+        .quad   0x3ad3e3ebf36c4975+        .quad   0xd75d25a537862125+        .quad   0xe873943da025a516+        .quad   0x6bbc7cb4c411c847+        .quad   0x3c6f9cd1d4a50d56+        .quad   0xb6244077c6feab7e+        .quad   0x6ff9bf483580972e+        .quad   0x00375883b332acfb++        // 2^92 * 3 * B++        .quad   0x0001b2cd28cb0940+        .quad   0x63fb51a06f1c24c9+        .quad   0xb5ad8691dcd5ca31+        .quad   0x67238dbd8c450660+        .quad   0xc98bec856c75c99c+        .quad   0xe44184c000e33cf4+        .quad   0x0a676b9bba907634+        .quad   0x669e2cb571f379d7+        .quad   0xcb116b73a49bd308+        .quad   0x025aad6b2392729e+        .quad   0xb4793efa3f55d9b1+        .quad   0x72a1056140678bb9++        // 2^92 * 4 * B++        .quad   0xa2b6812b1cc9249d+        .quad   0x62866eee21211f58+        .quad   0x2cb5c5b85df10ece+        .quad   0x03a6b259e263ae00+        .quad   0x0d8d2909e2e505b6+        .quad   0x98ca78abc0291230+        .quad   0x77ef5569a9b12327+        .quad   0x7c77897b81439b47+        .quad   0xf1c1b5e2de331cb5+        .quad   0x5a9f5d8e15fca420+        .quad   0x9fa438f17bd932b1+        .quad   0x2a381bf01c6146e7++        // 2^92 * 5 * B++        .quad   0xac9b9879cfc811c1+        .quad   0x8b7d29813756e567+        .quad   0x50da4e607c70edfc+        .quad   0x5dbca62f884400b6+        .quad   0xf7c0be32b534166f+        .quad   0x27e6ca6419cf70d4+        .quad   0x934df7d7a957a759+        .quad   0x5701461dabdec2aa+        .quad   0x2c6747402c915c25+        .quad   0x1bdcd1a80b0d340a+        .quad   0x5e5601bd07b43f5f+        .quad   0x2555b4e05539a242++        // 2^92 * 6 * B++        .quad   0x6fc09f5266ddd216+        .quad   0xdce560a7c8e37048+        .quad   0xec65939da2df62fd+        .quad   0x7a869ae7e52ed192+        .quad   0x78409b1d87e463d4+        .quad   0xad4da95acdfb639d+        .quad   0xec28773755259b9c+        .quad   0x69c806e9c31230ab+        .quad   0x7b48f57414bb3f22+        .quad   0x68c7cee4aedccc88+        .quad   0xed2f936179ed80be+        .quad   0x25d70b885f77bc4b++        // 2^92 * 7 * B++        .quad   0x4151c3d9762bf4de+        .quad   0x083f435f2745d82b+        .quad   0x29775a2e0d23ddd5+        .quad   0x138e3a6269a5db24+        .quad   0x98459d29bb1ae4d4+        .quad   0x56b9c4c739f954ec+        .quad   0x832743f6c29b4b3e+        .quad   0x21ea8e2798b6878a+        .quad   0x87bef4b46a5a7b9c+        .quad   0xd2299d1b5fc1d062+        .quad   0x82409818dd321648+        .quad   0x5c5abeb1e5a2e03d++        // 2^92 * 8 * B++        .quad   0x14722af4b73c2ddb+        .quad   0xbc470c5f5a05060d+        .quad   0x00943eac2581b02e+        .quad   0x0e434b3b1f499c8f+        .quad   0x02cde6de1306a233+        .quad   0x7b5a52a2116f8ec7+        .quad   0xe1c681f4c1163b5b+        .quad   0x241d350660d32643+        .quad   0x6be4404d0ebc52c7+        .quad   0xae46233bb1a791f5+        .quad   0x2aec170ed25db42b+        .quad   0x1d8dfd966645d694++        // 2^96 * 1 * B++        .quad   0x296fa9c59c2ec4de+        .quad   0xbc8b61bf4f84f3cb+        .quad   0x1c7706d917a8f908+        .quad   0x63b795fc7ad3255d+        .quad   0xd598639c12ddb0a4+        .quad   0xa5d19f30c024866b+        .quad   0xd17c2f0358fce460+        .quad   0x07a195152e095e8a+        .quad   0xa8368f02389e5fc8+        .quad   0x90433b02cf8de43b+        .quad   0xafa1fd5dc5412643+        .quad   0x3e8fe83d032f0137++        // 2^96 * 2 * B++        .quad   0x2f8b15b90570a294+        .quad   0x94f2427067084549+        .quad   0xde1c5ae161bbfd84+        .quad   0x75ba3b797fac4007+        .quad   0x08704c8de8efd13c+        .quad   0xdfc51a8e33e03731+        .quad   0xa59d5da51260cde3+        .quad   0x22d60899a6258c86+        .quad   0x6239dbc070cdd196+        .quad   0x60fe8a8b6c7d8a9a+        .quad   0xb38847bceb401260+        .quad   0x0904d07b87779e5e++        // 2^96 * 3 * B++        .quad   0xb4ce1fd4ddba919c+        .quad   0xcf31db3ec74c8daa+        .quad   0x2c63cc63ad86cc51+        .quad   0x43e2143fbc1dde07+        .quad   0xf4322d6648f940b9+        .quad   0x06952f0cbd2d0c39+        .quad   0x167697ada081f931+        .quad   0x6240aacebaf72a6c+        .quad   0xf834749c5ba295a0+        .quad   0xd6947c5bca37d25a+        .quad   0x66f13ba7e7c9316a+        .quad   0x56bdaf238db40cac++        // 2^96 * 4 * B++        .quad   0x362ab9e3f53533eb+        .quad   0x338568d56eb93d40+        .quad   0x9e0e14521d5a5572+        .quad   0x1d24a86d83741318+        .quad   0x1310d36cc19d3bb2+        .quad   0x062a6bb7622386b9+        .quad   0x7c9b8591d7a14f5c+        .quad   0x03aa31507e1e5754+        .quad   0xf4ec7648ffd4ce1f+        .quad   0xe045eaf054ac8c1c+        .quad   0x88d225821d09357c+        .quad   0x43b261dc9aeb4859++        // 2^96 * 5 * B++        .quad   0xe55b1e1988bb79bb+        .quad   0xa09ed07dc17a359d+        .quad   0xb02c2ee2603dea33+        .quad   0x326055cf5b276bc2+        .quad   0x19513d8b6c951364+        .quad   0x94fe7126000bf47b+        .quad   0x028d10ddd54f9567+        .quad   0x02b4d5e242940964+        .quad   0xb4a155cb28d18df2+        .quad   0xeacc4646186ce508+        .quad   0xc49cf4936c824389+        .quad   0x27a6c809ae5d3410++        // 2^96 * 6 * B++        .quad   0x8ba6ebcd1f0db188+        .quad   0x37d3d73a675a5be8+        .quad   0xf22edfa315f5585a+        .quad   0x2cb67174ff60a17e+        .quad   0xcd2c270ac43d6954+        .quad   0xdd4a3e576a66cab2+        .quad   0x79fa592469d7036c+        .quad   0x221503603d8c2599+        .quad   0x59eecdf9390be1d0+        .quad   0xa9422044728ce3f1+        .quad   0x82891c667a94f0f4+        .quad   0x7b1df4b73890f436++        // 2^96 * 7 * B++        .quad   0xe492f2e0b3b2a224+        .quad   0x7c6c9e062b551160+        .quad   0x15eb8fe20d7f7b0e+        .quad   0x61fcef2658fc5992+        .quad   0x5f2e221807f8f58c+        .quad   0xe3555c9fd49409d4+        .quad   0xb2aaa88d1fb6a630+        .quad   0x68698245d352e03d+        .quad   0xdbb15d852a18187a+        .quad   0xf3e4aad386ddacd7+        .quad   0x44bae2810ff6c482+        .quad   0x46cf4c473daf01cf++        // 2^96 * 8 * B++        .quad   0x426525ed9ec4e5f9+        .quad   0x0e5eda0116903303+        .quad   0x72b1a7f2cbe5cadc+        .quad   0x29387bcd14eb5f40+        .quad   0x213c6ea7f1498140+        .quad   0x7c1e7ef8392b4854+        .quad   0x2488c38c5629ceba+        .quad   0x1065aae50d8cc5bb+        .quad   0x1c2c4525df200d57+        .quad   0x5c3b2dd6bfca674a+        .quad   0x0a07e7b1e1834030+        .quad   0x69a198e64f1ce716++        // 2^100 * 1 * B++        .quad   0x7afcd613efa9d697+        .quad   0x0cc45aa41c067959+        .quad   0xa56fe104c1fada96+        .quad   0x3a73b70472e40365+        .quad   0x7b26e56b9e2d4734+        .quad   0xc4c7132b81c61675+        .quad   0xef5c9525ec9cde7f+        .quad   0x39c80b16e71743ad+        .quad   0x0f196e0d1b826c68+        .quad   0xf71ff0e24960e3db+        .quad   0x6113167023b7436c+        .quad   0x0cf0ea5877da7282++        // 2^100 * 2 * B++        .quad   0x196c80a4ddd4ccbd+        .quad   0x22e6f55d95f2dd9d+        .quad   0xc75e33c740d6c71b+        .quad   0x7bb51279cb3c042f+        .quad   0xe332ced43ba6945a+        .quad   0xde0b1361e881c05d+        .quad   0x1ad40f095e67ed3b+        .quad   0x5da8acdab8c63d5d+        .quad   0xc4b6664a3a70159f+        .quad   0x76194f0f0a904e14+        .quad   0xa5614c39a4096c13+        .quad   0x6cd0ff50979feced++        // 2^100 * 3 * B++        .quad   0xc0e067e78f4428ac+        .quad   0x14835ab0a61135e3+        .quad   0xf21d14f338062935+        .quad   0x6390a4c8df04849c+        .quad   0x7fecfabdb04ba18e+        .quad   0xd0fc7bfc3bddbcf7+        .quad   0xa41d486e057a131c+        .quad   0x641a4391f2223a61+        .quad   0xc5c6b95aa606a8db+        .quad   0x914b7f9eb06825f1+        .quad   0x2a731f6b44fc9eff+        .quad   0x30ddf38562705cfc++        // 2^100 * 4 * B++        .quad   0x4e3dcbdad1bff7f9+        .quad   0xc9118e8220645717+        .quad   0xbacccebc0f189d56+        .quad   0x1b4822e9d4467668+        .quad   0x33bef2bd68bcd52c+        .quad   0xc649dbb069482ef2+        .quad   0xb5b6ee0c41cb1aee+        .quad   0x5c294d270212a7e5+        .quad   0xab360a7f25563781+        .quad   0x2512228a480f7958+        .quad   0xc75d05276114b4e3+        .quad   0x222d9625d976fe2a++        // 2^100 * 5 * B++        .quad   0x1c717f85b372ace1+        .quad   0x81930e694638bf18+        .quad   0x239cad056bc08b58+        .quad   0x0b34271c87f8fff4+        .quad   0x0f94be7e0a344f85+        .quad   0xeb2faa8c87f22c38+        .quad   0x9ce1e75e4ee16f0f+        .quad   0x43e64e5418a08dea+        .quad   0x8155e2521a35ce63+        .quad   0xbe100d4df912028e+        .quad   0xbff80bf8a57ddcec+        .quad   0x57342dc96d6bc6e4++        // 2^100 * 6 * B++        .quad   0xefeef065c8ce5998+        .quad   0xbf029510b5cbeaa2+        .quad   0x8c64a10620b7c458+        .quad   0x35134fb231c24855+        .quad   0xf3c3bcb71e707bf6+        .quad   0x351d9b8c7291a762+        .quad   0x00502e6edad69a33+        .quad   0x522f521f1ec8807f+        .quad   0x272c1f46f9a3902b+        .quad   0xc91ba3b799657bcc+        .quad   0xae614b304f8a1c0e+        .quad   0x7afcaad70b99017b++        // 2^100 * 7 * B++        .quad   0xc25ded54a4b8be41+        .quad   0x902d13e11bb0e2dd+        .quad   0x41f43233cde82ab2+        .quad   0x1085faa5c3aae7cb+        .quad   0xa88141ecef842b6b+        .quad   0x55e7b14797abe6c5+        .quad   0x8c748f9703784ffe+        .quad   0x5b50a1f7afcd00b7+        .quad   0x9b840f66f1361315+        .quad   0x18462242701003e9+        .quad   0x65ed45fae4a25080+        .quad   0x0a2862393fda7320++        // 2^100 * 8 * B++        .quad   0x46ab13c8347cbc9d+        .quad   0x3849e8d499c12383+        .quad   0x4cea314087d64ac9+        .quad   0x1f354134b1a29ee7+        .quad   0x960e737b6ecb9d17+        .quad   0xfaf24948d67ceae1+        .quad   0x37e7a9b4d55e1b89+        .quad   0x5cb7173cb46c59eb+        .quad   0x4a89e68b82b7abf0+        .quad   0xf41cd9279ba6b7b9+        .quad   0x16e6c210e18d876f+        .quad   0x7cacdb0f7f1b09c6++        // 2^104 * 1 * B++        .quad   0x9062b2e0d91a78bc+        .quad   0x47c9889cc8509667+        .quad   0x9df54a66405070b8+        .quad   0x7369e6a92493a1bf+        .quad   0xe1014434dcc5caed+        .quad   0x47ed5d963c84fb33+        .quad   0x70019576ed86a0e7+        .quad   0x25b2697bd267f9e4+        .quad   0x9d673ffb13986864+        .quad   0x3ca5fbd9415dc7b8+        .quad   0xe04ecc3bdf273b5e+        .quad   0x1420683db54e4cd2++        // 2^104 * 2 * B++        .quad   0xb478bd1e249dd197+        .quad   0x620c35005e58c102+        .quad   0xfb02d32fccbaac5c+        .quad   0x60b63bebf508a72d+        .quad   0x34eebb6fc1cc5ad0+        .quad   0x6a1b0ce99646ac8b+        .quad   0xd3b0da49a66bde53+        .quad   0x31e83b4161d081c1+        .quad   0x97e8c7129e062b4f+        .quad   0x49e48f4f29320ad8+        .quad   0x5bece14b6f18683f+        .quad   0x55cf1eb62d550317++        // 2^104 * 3 * B++        .quad   0x5879101065c23d58+        .quad   0x8b9d086d5094819c+        .quad   0xe2402fa912c55fa7+        .quad   0x669a6564570891d4+        .quad   0x3076b5e37df58c52+        .quad   0xd73ab9dde799cc36+        .quad   0xbd831ce34913ee20+        .quad   0x1a56fbaa62ba0133+        .quad   0x943e6b505c9dc9ec+        .quad   0x302557bba77c371a+        .quad   0x9873ae5641347651+        .quad   0x13c4836799c58a5c++        // 2^104 * 4 * B++        .quad   0x423a5d465ab3e1b9+        .quad   0xfc13c187c7f13f61+        .quad   0x19f83664ecb5b9b6+        .quad   0x66f80c93a637b607+        .quad   0xc4dcfb6a5d8bd080+        .quad   0xdeebc4ec571a4842+        .quad   0xd4b2e883b8e55365+        .quad   0x50bdc87dc8e5b827+        .quad   0x606d37836edfe111+        .quad   0x32353e15f011abd9+        .quad   0x64b03ac325b73b96+        .quad   0x1dd56444725fd5ae++        // 2^104 * 5 * B++        .quad   0x8fa47ff83362127d+        .quad   0xbc9f6ac471cd7c15+        .quad   0x6e71454349220c8b+        .quad   0x0e645912219f732e+        .quad   0xc297e60008bac89a+        .quad   0x7d4cea11eae1c3e0+        .quad   0xf3e38be19fe7977c+        .quad   0x3a3a450f63a305cd+        .quad   0x078f2f31d8394627+        .quad   0x389d3183de94a510+        .quad   0xd1e36c6d17996f80+        .quad   0x318c8d9393a9a87b++        // 2^104 * 6 * B++        .quad   0xf2745d032afffe19+        .quad   0x0c9f3c497f24db66+        .quad   0xbc98d3e3ba8598ef+        .quad   0x224c7c679a1d5314+        .quad   0x5d669e29ab1dd398+        .quad   0xfc921658342d9e3b+        .quad   0x55851dfdf35973cd+        .quad   0x509a41c325950af6+        .quad   0xbdc06edca6f925e9+        .quad   0x793ef3f4641b1f33+        .quad   0x82ec12809d833e89+        .quad   0x05bff02328a11389++        // 2^104 * 7 * B++        .quad   0x3632137023cae00b+        .quad   0x544acf0ad1accf59+        .quad   0x96741049d21a1c88+        .quad   0x780b8cc3fa2a44a7+        .quad   0x6881a0dd0dc512e4+        .quad   0x4fe70dc844a5fafe+        .quad   0x1f748e6b8f4a5240+        .quad   0x576277cdee01a3ea+        .quad   0x1ef38abc234f305f+        .quad   0x9a577fbd1405de08+        .quad   0x5e82a51434e62a0d+        .quad   0x5ff418726271b7a1++        // 2^104 * 8 * B++        .quad   0x398e080c1789db9d+        .quad   0xa7602025f3e778f5+        .quad   0xfa98894c06bd035d+        .quad   0x106a03dc25a966be+        .quad   0xe5db47e813b69540+        .quad   0xf35d2a3b432610e1+        .quad   0xac1f26e938781276+        .quad   0x29d4db8ca0a0cb69+        .quad   0xd9ad0aaf333353d0+        .quad   0x38669da5acd309e5+        .quad   0x3c57658ac888f7f0+        .quad   0x4ab38a51052cbefa++        // 2^108 * 1 * B++        .quad   0xdfdacbee4324c0e9+        .quad   0x054442883f955bb7+        .quad   0xdef7aaa8ea31609f+        .quad   0x68aee70642287cff+        .quad   0xf68fe2e8809de054+        .quad   0xe3bc096a9c82bad1+        .quad   0x076353d40aadbf45+        .quad   0x7b9b1fb5dea1959e+        .quad   0xf01cc8f17471cc0c+        .quad   0x95242e37579082bb+        .quad   0x27776093d3e46b5f+        .quad   0x2d13d55a28bd85fb++        // 2^108 * 2 * B++        .quad   0xfac5d2065b35b8da+        .quad   0xa8da8a9a85624bb7+        .quad   0xccd2ca913d21cd0f+        .quad   0x6b8341ee8bf90d58+        .quad   0xbf019cce7aee7a52+        .quad   0xa8ded2b6e454ead3+        .quad   0x3c619f0b87a8bb19+        .quad   0x3619b5d7560916d8+        .quad   0x3579f26b0282c4b2+        .quad   0x64d592f24fafefae+        .quad   0xb7cded7b28c8c7c0+        .quad   0x6a927b6b7173a8d7++        // 2^108 * 3 * B++        .quad   0x1f6db24f986e4656+        .quad   0x1021c02ed1e9105b+        .quad   0xf8ff3fff2cc0a375+        .quad   0x1d2a6bf8c6c82592+        .quad   0x8d7040863ece88eb+        .quad   0xf0e307a980eec08c+        .quad   0xac2250610d788fda+        .quad   0x056d92a43a0d478d+        .quad   0x1b05a196fc3da5a1+        .quad   0x77d7a8c243b59ed0+        .quad   0x06da3d6297d17918+        .quad   0x66fbb494f12353f7++        // 2^108 * 4 * B++        .quad   0x751a50b9d85c0fb8+        .quad   0xd1afdc258bcf097b+        .quad   0x2f16a6a38309a969+        .quad   0x14ddff9ee5b00659+        .quad   0xd6d70996f12309d6+        .quad   0xdbfb2385e9c3d539+        .quad   0x46d602b0f7552411+        .quad   0x270a0b0557843e0c+        .quad   0x61ff0640a7862bcc+        .quad   0x81cac09a5f11abfe+        .quad   0x9047830455d12abb+        .quad   0x19a4bde1945ae873++        // 2^108 * 5 * B++        .quad   0x9b9f26f520a6200a+        .quad   0x64804443cf13eaf8+        .quad   0x8a63673f8631edd3+        .quad   0x72bbbce11ed39dc1+        .quad   0x40c709dec076c49f+        .quad   0x657bfaf27f3e53f6+        .quad   0x40662331eca042c4+        .quad   0x14b375487eb4df04+        .quad   0xae853c94ab66dc47+        .quad   0xeb62343edf762d6e+        .quad   0xf08e0e186fb2f7d1+        .quad   0x4f0b1c02700ab37a++        // 2^108 * 6 * B++        .quad   0xe1706787d81951fa+        .quad   0xa10a2c8eb290c77b+        .quad   0xe7382fa03ed66773+        .quad   0x0a4d84710bcc4b54+        .quad   0x79fd21ccc1b2e23f+        .quad   0x4ae7c281453df52a+        .quad   0xc8172ec9d151486b+        .quad   0x68abe9443e0a7534+        .quad   0xda12c6c407831dcb+        .quad   0x0da230d74d5c510d+        .quad   0x4ab1531e6bd404e1+        .quad   0x4106b166bcf440ef++        // 2^108 * 7 * B++        .quad   0x02e57a421cd23668+        .quad   0x4ad9fb5d0eaef6fd+        .quad   0x954e6727b1244480+        .quad   0x7f792f9d2699f331+        .quad   0xa485ccd539e4ecf2+        .quad   0x5aa3f3ad0555bab5+        .quad   0x145e3439937df82d+        .quad   0x1238b51e1214283f+        .quad   0x0b886b925fd4d924+        .quad   0x60906f7a3626a80d+        .quad   0xecd367b4b98abd12+        .quad   0x2876beb1def344cf++        // 2^108 * 8 * B++        .quad   0xdc84e93563144691+        .quad   0x632fe8a0d61f23f4+        .quad   0x4caa800612a9a8d5+        .quad   0x48f9dbfa0e9918d3+        .quad   0xd594b3333a8a85f8+        .quad   0x4ea37689e78d7d58+        .quad   0x73bf9f455e8e351f+        .quad   0x5507d7d2bc41ebb4+        .quad   0x1ceb2903299572fc+        .quad   0x7c8ccaa29502d0ee+        .quad   0x91bfa43411cce67b+        .quad   0x5784481964a831e7++        // 2^112 * 1 * B++        .quad   0xda7c2b256768d593+        .quad   0x98c1c0574422ca13+        .quad   0xf1a80bd5ca0ace1d+        .quad   0x29cdd1adc088a690+        .quad   0xd6cfd1ef5fddc09c+        .quad   0xe82b3efdf7575dce+        .quad   0x25d56b5d201634c2+        .quad   0x3041c6bb04ed2b9b+        .quad   0x0ff2f2f9d956e148+        .quad   0xade797759f356b2e+        .quad   0x1a4698bb5f6c025c+        .quad   0x104bbd6814049a7b++        // 2^112 * 2 * B++        .quad   0x51f0fd3168f1ed67+        .quad   0x2c811dcdd86f3bc2+        .quad   0x44dc5c4304d2f2de+        .quad   0x5be8cc57092a7149+        .quad   0xa95d9a5fd67ff163+        .quad   0xe92be69d4cc75681+        .quad   0xb7f8024cde20f257+        .quad   0x204f2a20fb072df5+        .quad   0xc8143b3d30ebb079+        .quad   0x7589155abd652e30+        .quad   0x653c3c318f6d5c31+        .quad   0x2570fb17c279161f++        // 2^112 * 3 * B++        .quad   0x3efa367f2cb61575+        .quad   0xf5f96f761cd6026c+        .quad   0xe8c7142a65b52562+        .quad   0x3dcb65ea53030acd+        .quad   0x192ea9550bb8245a+        .quad   0xc8e6fba88f9050d1+        .quad   0x7986ea2d88a4c935+        .quad   0x241c5f91de018668+        .quad   0x28d8172940de6caa+        .quad   0x8fbf2cf022d9733a+        .quad   0x16d7fcdd235b01d1+        .quad   0x08420edd5fcdf0e5++        // 2^112 * 4 * B++        .quad   0xcdff20ab8362fa4a+        .quad   0x57e118d4e21a3e6e+        .quad   0xe3179617fc39e62b+        .quad   0x0d9a53efbc1769fd+        .quad   0x0358c34e04f410ce+        .quad   0xb6135b5a276e0685+        .quad   0x5d9670c7ebb91521+        .quad   0x04d654f321db889c+        .quad   0x5e7dc116ddbdb5d5+        .quad   0x2954deb68da5dd2d+        .quad   0x1cb608173334a292+        .quad   0x4a7a4f2618991ad7++        // 2^112 * 5 * B++        .quad   0xf4a718025fb15f95+        .quad   0x3df65f346b5c1b8f+        .quad   0xcdfcf08500e01112+        .quad   0x11b50c4cddd31848+        .quad   0x24c3b291af372a4b+        .quad   0x93da8270718147f2+        .quad   0xdd84856486899ef2+        .quad   0x4a96314223e0ee33+        .quad   0xa6e8274408a4ffd6+        .quad   0x738e177e9c1576d9+        .quad   0x773348b63d02b3f2+        .quad   0x4f4bce4dce6bcc51++        // 2^112 * 6 * B++        .quad   0xa71fce5ae2242584+        .quad   0x26ea725692f58a9e+        .quad   0xd21a09d71cea3cf4+        .quad   0x73fcdd14b71c01e6+        .quad   0x30e2616ec49d0b6f+        .quad   0xe456718fcaec2317+        .quad   0x48eb409bf26b4fa6+        .quad   0x3042cee561595f37+        .quad   0x427e7079449bac41+        .quad   0x855ae36dbce2310a+        .quad   0x4cae76215f841a7c+        .quad   0x389e740c9a9ce1d6++        // 2^112 * 7 * B++        .quad   0x64fcb3ae34dcb9ce+        .quad   0x97500323e348d0ad+        .quad   0x45b3f07d62c6381b+        .quad   0x61545379465a6788+        .quad   0xc9bd78f6570eac28+        .quad   0xe55b0b3227919ce1+        .quad   0x65fc3eaba19b91ed+        .quad   0x25c425e5d6263690+        .quad   0x3f3e06a6f1d7de6e+        .quad   0x3ef976278e062308+        .quad   0x8c14f6264e8a6c77+        .quad   0x6539a08915484759++        // 2^112 * 8 * B++        .quad   0xe9d21f74c3d2f773+        .quad   0xc150544125c46845+        .quad   0x624e5ce8f9b99e33+        .quad   0x11c5e4aac5cd186c+        .quad   0xddc4dbd414bb4a19+        .quad   0x19b2bc3c98424f8e+        .quad   0x48a89fd736ca7169+        .quad   0x0f65320ef019bd90+        .quad   0xd486d1b1cafde0c6+        .quad   0x4f3fe6e3163b5181+        .quad   0x59a8af0dfaf2939a+        .quad   0x4cabc7bdec33072a++        // 2^116 * 1 * B++        .quad   0x16faa8fb532f7428+        .quad   0xdbd42ea046a4e272+        .quad   0x5337653b8b9ea480+        .quad   0x4065947223973f03+        .quad   0xf7c0a19c1a54a044+        .quad   0x4a1c5e2477bd9fbb+        .quad   0xa6e3ca115af22972+        .quad   0x1819bb953f2e9e0d+        .quad   0x498fbb795e042e84+        .quad   0x7d0dd89a7698b714+        .quad   0x8bfb0ba427fe6295+        .quad   0x36ba82e721200524++        // 2^116 * 2 * B++        .quad   0xd60ecbb74245ec41+        .quad   0xfd9be89e34348716+        .quad   0xc9240afee42284de+        .quad   0x4472f648d0531db4+        .quad   0xc8d69d0a57274ed5+        .quad   0x45ba803260804b17+        .quad   0xdf3cda102255dfac+        .quad   0x77d221232709b339+        .quad   0x498a6d7064ad94d8+        .quad   0xa5b5c8fd9af62263+        .quad   0x8ca8ed0545c141f4+        .quad   0x2c63bec3662d358c++        // 2^116 * 3 * B++        .quad   0x7fe60d8bea787955+        .quad   0xb9dc117eb5f401b7+        .quad   0x91c7c09a19355cce+        .quad   0x22692ef59442bedf+        .quad   0x9a518b3a8586f8bf+        .quad   0x9ee71af6cbb196f0+        .quad   0xaa0625e6a2385cf2+        .quad   0x1deb2176ddd7c8d1+        .quad   0x8563d19a2066cf6c+        .quad   0x401bfd8c4dcc7cd7+        .quad   0xd976a6becd0d8f62+        .quad   0x67cfd773a278b05e++        // 2^116 * 4 * B++        .quad   0x8dec31faef3ee475+        .quad   0x99dbff8a9e22fd92+        .quad   0x512d11594e26cab1+        .quad   0x0cde561eec4310b9+        .quad   0x2d5fa9855a4e586a+        .quad   0x65f8f7a449beab7e+        .quad   0xaa074dddf21d33d3+        .quad   0x185cba721bcb9dee+        .quad   0x93869da3f4e3cb41+        .quad   0xbf0392f540f7977e+        .quad   0x026204fcd0463b83+        .quad   0x3ec91a769eec6eed++        // 2^116 * 5 * B++        .quad   0x1e9df75bf78166ad+        .quad   0x4dfda838eb0cd7af+        .quad   0xba002ed8c1eaf988+        .quad   0x13fedb3e11f33cfc+        .quad   0x0fad2fb7b0a3402f+        .quad   0x46615ecbfb69f4a8+        .quad   0xf745bcc8c5f8eaa6+        .quad   0x7a5fa8794a94e896+        .quad   0x52958faa13cd67a1+        .quad   0x965ee0818bdbb517+        .quad   0x16e58daa2e8845b3+        .quad   0x357d397d5499da8f++        // 2^116 * 6 * B++        .quad   0x1ebfa05fb0bace6c+        .quad   0xc934620c1caf9a1e+        .quad   0xcc771cc41d82b61a+        .quad   0x2d94a16aa5f74fec+        .quad   0x481dacb4194bfbf8+        .quad   0x4d77e3f1bae58299+        .quad   0x1ef4612e7d1372a0+        .quad   0x3a8d867e70ff69e1+        .quad   0x6f58cd5d55aff958+        .quad   0xba3eaa5c75567721+        .quad   0x75c123999165227d+        .quad   0x69be1343c2f2b35e++        // 2^116 * 7 * B++        .quad   0x0e091d5ee197c92a+        .quad   0x4f51019f2945119f+        .quad   0x143679b9f034e99c+        .quad   0x7d88112e4d24c696+        .quad   0x82bbbdac684b8de3+        .quad   0xa2f4c7d03fca0718+        .quad   0x337f92fbe096aaa8+        .quad   0x200d4d8c63587376+        .quad   0x208aed4b4893b32b+        .quad   0x3efbf23ebe59b964+        .quad   0xd762deb0dba5e507+        .quad   0x69607bd681bd9d94++        // 2^116 * 8 * B++        .quad   0xf6be021068de1ce1+        .quad   0xe8d518e70edcbc1f+        .quad   0xe3effdd01b5505a5+        .quad   0x35f63353d3ec3fd0+        .quad   0x3b7f3bd49323a902+        .quad   0x7c21b5566b2c6e53+        .quad   0xe5ba8ff53a7852a7+        .quad   0x28bc77a5838ece00+        .quad   0x63ba78a8e25d8036+        .quad   0x63651e0094333490+        .quad   0x48d82f20288ce532+        .quad   0x3a31abfa36b57524++        // 2^120 * 1 * B++        .quad   0x239e9624089c0a2e+        .quad   0xc748c4c03afe4738+        .quad   0x17dbed2a764fa12a+        .quad   0x639b93f0321c8582+        .quad   0xc08f788f3f78d289+        .quad   0xfe30a72ca1404d9f+        .quad   0xf2778bfccf65cc9d+        .quad   0x7ee498165acb2021+        .quad   0x7bd508e39111a1c3+        .quad   0x2b2b90d480907489+        .quad   0xe7d2aec2ae72fd19+        .quad   0x0edf493c85b602a6++        // 2^120 * 2 * B++        .quad   0xaecc8158599b5a68+        .quad   0xea574f0febade20e+        .quad   0x4fe41d7422b67f07+        .quad   0x403b92e3019d4fb4+        .quad   0x6767c4d284764113+        .quad   0xa090403ff7f5f835+        .quad   0x1c8fcffacae6bede+        .quad   0x04c00c54d1dfa369+        .quad   0x4dc22f818b465cf8+        .quad   0x71a0f35a1480eff8+        .quad   0xaee8bfad04c7d657+        .quad   0x355bb12ab26176f4++        // 2^120 * 3 * B++        .quad   0xa71e64cc7493bbf4+        .quad   0xe5bd84d9eca3b0c3+        .quad   0x0a6bc50cfa05e785+        .quad   0x0f9b8132182ec312+        .quad   0xa301dac75a8c7318+        .quad   0xed90039db3ceaa11+        .quad   0x6f077cbf3bae3f2d+        .quad   0x7518eaf8e052ad8e+        .quad   0xa48859c41b7f6c32+        .quad   0x0f2d60bcf4383298+        .quad   0x1815a929c9b1d1d9+        .quad   0x47c3871bbb1755c4++        // 2^120 * 4 * B++        .quad   0x5144539771ec4f48+        .quad   0xf805b17dc98c5d6e+        .quad   0xf762c11a47c3c66b+        .quad   0x00b89b85764699dc+        .quad   0xfbe65d50c85066b0+        .quad   0x62ecc4b0b3a299b0+        .quad   0xe53754ea441ae8e0+        .quad   0x08fea02ce8d48d5f+        .quad   0x824ddd7668deead0+        .quad   0xc86445204b685d23+        .quad   0xb514cfcd5d89d665+        .quad   0x473829a74f75d537++        // 2^120 * 5 * B++        .quad   0x82d2da754679c418+        .quad   0xe63bd7d8b2618df0+        .quad   0x355eef24ac47eb0a+        .quad   0x2078684c4833c6b4+        .quad   0x23d9533aad3902c9+        .quad   0x64c2ddceef03588f+        .quad   0x15257390cfe12fb4+        .quad   0x6c668b4d44e4d390+        .quad   0x3b48cf217a78820c+        .quad   0xf76a0ab281273e97+        .quad   0xa96c65a78c8eed7b+        .quad   0x7411a6054f8a433f++        // 2^120 * 6 * B++        .quad   0x4d659d32b99dc86d+        .quad   0x044cdc75603af115+        .quad   0xb34c712cdcc2e488+        .quad   0x7c136574fb8134ff+        .quad   0x579ae53d18b175b4+        .quad   0x68713159f392a102+        .quad   0x8455ecba1eef35f5+        .quad   0x1ec9a872458c398f+        .quad   0xb8e6a4d400a2509b+        .quad   0x9b81d7020bc882b4+        .quad   0x57e7cc9bf1957561+        .quad   0x3add88a5c7cd6460++        // 2^120 * 7 * B++        .quad   0xab895770b635dcf2+        .quad   0x02dfef6cf66c1fbc+        .quad   0x85530268beb6d187+        .quad   0x249929fccc879e74+        .quad   0x85c298d459393046+        .quad   0x8f7e35985ff659ec+        .quad   0x1d2ca22af2f66e3a+        .quad   0x61ba1131a406a720+        .quad   0xa3d0a0f116959029+        .quad   0x023b6b6cba7ebd89+        .quad   0x7bf15a3e26783307+        .quad   0x5620310cbbd8ece7++        // 2^120 * 8 * B++        .quad   0x528993434934d643+        .quad   0xb9dbf806a51222f5+        .quad   0x8f6d878fc3f41c22+        .quad   0x37676a2a4d9d9730+        .quad   0x6646b5f477e285d6+        .quad   0x40e8ff676c8f6193+        .quad   0xa6ec7311abb594dd+        .quad   0x7ec846f3658cec4d+        .quad   0x9b5e8f3f1da22ec7+        .quad   0x130f1d776c01cd13+        .quad   0x214c8fcfa2989fb8+        .quad   0x6daaf723399b9dd5++        // 2^124 * 1 * B++        .quad   0x591e4a5610628564+        .quad   0x2a4bb87ca8b4df34+        .quad   0xde2a2572e7a38e43+        .quad   0x3cbdabd9fee5046e+        .quad   0x81aebbdd2cd13070+        .quad   0x962e4325f85a0e9e+        .quad   0xde9391aacadffecb+        .quad   0x53177fda52c230e6+        .quad   0xa7bc970650b9de79+        .quad   0x3d12a7fbc301b59b+        .quad   0x02652e68d36ae38c+        .quad   0x79d739835a6199dc++        // 2^124 * 2 * B++        .quad   0xd9354df64131c1bd+        .quad   0x758094a186ec5822+        .quad   0x4464ee12e459f3c2+        .quad   0x6c11fce4cb133282+        .quad   0x21c9d9920d591737+        .quad   0x9bea41d2e9b46cd6+        .quad   0xe20e84200d89bfca+        .quad   0x79d99f946eae5ff8+        .quad   0xf17b483568673205+        .quad   0x387deae83caad96c+        .quad   0x61b471fd56ffe386+        .quad   0x31741195b745a599++        // 2^124 * 3 * B++        .quad   0xe8d10190b77a360b+        .quad   0x99b983209995e702+        .quad   0xbd4fdff8fa0247aa+        .quad   0x2772e344e0d36a87+        .quad   0x17f8ba683b02a047+        .quad   0x50212096feefb6c8+        .quad   0x70139be21556cbe2+        .quad   0x203e44a11d98915b+        .quad   0xd6863eba37b9e39f+        .quad   0x105bc169723b5a23+        .quad   0x104f6459a65c0762+        .quad   0x567951295b4d38d4++        // 2^124 * 4 * B++        .quad   0x535fd60613037524+        .quad   0xe210adf6b0fbc26a+        .quad   0xac8d0a9b23e990ae+        .quad   0x47204d08d72fdbf9+        .quad   0x07242eb30d4b497f+        .quad   0x1ef96306b9bccc87+        .quad   0x37950934d8116f45+        .quad   0x05468d6201405b04+        .quad   0x00f565a9f93267de+        .quad   0xcecfd78dc0d58e8a+        .quad   0xa215e2dcf318e28e+        .quad   0x4599ee919b633352++        // 2^124 * 5 * B++        .quad   0xd3c220ca70e0e76b+        .quad   0xb12bea58ea9f3094+        .quad   0x294ddec8c3271282+        .quad   0x0c3539e1a1d1d028+        .quad   0xac746d6b861ae579+        .quad   0x31ab0650f6aea9dc+        .quad   0x241d661140256d4c+        .quad   0x2f485e853d21a5de+        .quad   0x329744839c0833f3+        .quad   0x6fe6257fd2abc484+        .quad   0x5327d1814b358817+        .quad   0x65712585893fe9bc++        // 2^124 * 6 * B++        .quad   0x9c102fb732a61161+        .quad   0xe48e10dd34d520a8+        .quad   0x365c63546f9a9176+        .quad   0x32f6fe4c046f6006+        .quad   0x81c29f1bd708ee3f+        .quad   0xddcb5a05ae6407d0+        .quad   0x97aec1d7d2a3eba7+        .quad   0x1590521a91d50831+        .quad   0x40a3a11ec7910acc+        .quad   0x9013dff8f16d27ae+        .quad   0x1a9720d8abb195d4+        .quad   0x1bb9fe452ea98463++        // 2^124 * 7 * B++        .quad   0xe9d1d950b3d54f9e+        .quad   0x2d5f9cbee00d33c1+        .quad   0x51c2c656a04fc6ac+        .quad   0x65c091ee3c1cbcc9+        .quad   0xcf5e6c95cc36747c+        .quad   0x294201536b0bc30d+        .quad   0x453ac67cee797af0+        .quad   0x5eae6ab32a8bb3c9+        .quad   0x7083661114f118ea+        .quad   0x2b37b87b94349cad+        .quad   0x7273f51cb4e99f40+        .quad   0x78a2a95823d75698++        // 2^124 * 8 * B++        .quad   0xa2b072e95c8c2ace+        .quad   0x69cffc96651e9c4b+        .quad   0x44328ef842e7b42b+        .quad   0x5dd996c122aadeb3+        .quad   0xb4f23c425ef83207+        .quad   0xabf894d3c9a934b5+        .quad   0xd0708c1339fd87f7+        .quad   0x1876789117166130+        .quad   0x925b5ef0670c507c+        .quad   0x819bc842b93c33bf+        .quad   0x10792e9a70dd003f+        .quad   0x59ad4b7a6e28dc74++        // 2^128 * 1 * B++        .quad   0x5f3a7562eb3dbe47+        .quad   0xf7ea38548ebda0b8+        .quad   0x00c3e53145747299+        .quad   0x1304e9e71627d551+        .quad   0x583b04bfacad8ea2+        .quad   0x29b743e8148be884+        .quad   0x2b1e583b0810c5db+        .quad   0x2b5449e58eb3bbaa+        .quad   0x789814d26adc9cfe+        .quad   0x3c1bab3f8b48dd0b+        .quad   0xda0fe1fff979c60a+        .quad   0x4468de2d7c2dd693++        // 2^128 * 2 * B++        .quad   0x51bb355e9419469e+        .quad   0x33e6dc4c23ddc754+        .quad   0x93a5b6d6447f9962+        .quad   0x6cce7c6ffb44bd63+        .quad   0x4b9ad8c6f86307ce+        .quad   0x21113531435d0c28+        .quad   0xd4a866c5657a772c+        .quad   0x5da6427e63247352+        .quad   0x1a94c688deac22ca+        .quad   0xb9066ef7bbae1ff8+        .quad   0x88ad8c388d59580f+        .quad   0x58f29abfe79f2ca8++        // 2^128 * 3 * B++        .quad   0xe90ecfab8de73e68+        .quad   0x54036f9f377e76a5+        .quad   0xf0495b0bbe015982+        .quad   0x577629c4a7f41e36+        .quad   0x4b5a64bf710ecdf6+        .quad   0xb14ce538462c293c+        .quad   0x3643d056d50b3ab9+        .quad   0x6af93724185b4870+        .quad   0x3220024509c6a888+        .quad   0xd2e036134b558973+        .quad   0x83e236233c33289f+        .quad   0x701f25bb0caec18f++        // 2^128 * 4 * B++        .quad   0xc3a8b0f8e4616ced+        .quad   0xf700660e9e25a87d+        .quad   0x61e3061ff4bca59c+        .quad   0x2e0c92bfbdc40be9+        .quad   0x9d18f6d97cbec113+        .quad   0x844a06e674bfdbe4+        .quad   0x20f5b522ac4e60d6+        .quad   0x720a5bc050955e51+        .quad   0x0c3f09439b805a35+        .quad   0xe84e8b376242abfc+        .quad   0x691417f35c229346+        .quad   0x0e9b9cbb144ef0ec++        // 2^128 * 5 * B++        .quad   0xfbbad48ffb5720ad+        .quad   0xee81916bdbf90d0e+        .quad   0xd4813152635543bf+        .quad   0x221104eb3f337bd8+        .quad   0x8dee9bd55db1beee+        .quad   0xc9c3ab370a723fb9+        .quad   0x44a8f1bf1c68d791+        .quad   0x366d44191cfd3cde+        .quad   0x9e3c1743f2bc8c14+        .quad   0x2eda26fcb5856c3b+        .quad   0xccb82f0e68a7fb97+        .quad   0x4167a4e6bc593244++        // 2^128 * 6 * B++        .quad   0x643b9d2876f62700+        .quad   0x5d1d9d400e7668eb+        .quad   0x1b4b430321fc0684+        .quad   0x7938bb7e2255246a+        .quad   0xc2be2665f8ce8fee+        .quad   0xe967ff14e880d62c+        .quad   0xf12e6e7e2f364eee+        .quad   0x34b33370cb7ed2f6+        .quad   0xcdc591ee8681d6cc+        .quad   0xce02109ced85a753+        .quad   0xed7485c158808883+        .quad   0x1176fc6e2dfe65e4++        // 2^128 * 7 * B++        .quad   0xb4af6cd05b9c619b+        .quad   0x2ddfc9f4b2a58480+        .quad   0x3d4fa502ebe94dc4+        .quad   0x08fc3a4c677d5f34+        .quad   0xdb90e28949770eb8+        .quad   0x98fbcc2aacf440a3+        .quad   0x21354ffeded7879b+        .quad   0x1f6a3e54f26906b6+        .quad   0x60a4c199d30734ea+        .quad   0x40c085b631165cd6+        .quad   0xe2333e23f7598295+        .quad   0x4f2fad0116b900d1++        // 2^128 * 8 * B++        .quad   0x44beb24194ae4e54+        .quad   0x5f541c511857ef6c+        .quad   0xa61e6b2d368d0498+        .quad   0x445484a4972ef7ab+        .quad   0x962cd91db73bb638+        .quad   0xe60577aafc129c08+        .quad   0x6f619b39f3b61689+        .quad   0x3451995f2944ee81+        .quad   0x9152fcd09fea7d7c+        .quad   0x4a816c94b0935cf6+        .quad   0x258e9aaa47285c40+        .quad   0x10b89ca6042893b7++        // 2^132 * 1 * B++        .quad   0x9b2a426e3b646025+        .quad   0x32127190385ce4cf+        .quad   0xa25cffc2dd6dea45+        .quad   0x06409010bea8de75+        .quad   0xd67cded679d34aa0+        .quad   0xcc0b9ec0cc4db39f+        .quad   0xa535a456e35d190f+        .quad   0x2e05d9eaf61f6fef+        .quad   0xc447901ad61beb59+        .quad   0x661f19bce5dc880a+        .quad   0x24685482b7ca6827+        .quad   0x293c778cefe07f26++        // 2^132 * 2 * B++        .quad   0x86809e7007069096+        .quad   0xaad75b15e4e50189+        .quad   0x07f35715a21a0147+        .quad   0x0487f3f112815d5e+        .quad   0x16c795d6a11ff200+        .quad   0xcb70d0e2b15815c9+        .quad   0x89f293209b5395b5+        .quad   0x50b8c2d031e47b4f+        .quad   0x48350c08068a4962+        .quad   0x6ffdd05351092c9a+        .quad   0x17af4f4aaf6fc8dd+        .quad   0x4b0553b53cdba58b++        // 2^132 * 3 * B++        .quad   0x9c65fcbe1b32ff79+        .quad   0xeb75ea9f03b50f9b+        .quad   0xfced2a6c6c07e606+        .quad   0x35106cd551717908+        .quad   0xbf05211b27c152d4+        .quad   0x5ec26849bd1af639+        .quad   0x5e0b2caa8e6fab98+        .quad   0x054c8bdd50bd0840+        .quad   0x38a0b12f1dcf073d+        .quad   0x4b60a8a3b7f6a276+        .quad   0xfed5ac25d3404f9a+        .quad   0x72e82d5e5505c229++        // 2^132 * 4 * B++        .quad   0x6b0b697ff0d844c8+        .quad   0xbb12f85cd979cb49+        .quad   0xd2a541c6c1da0f1f+        .quad   0x7b7c242958ce7211+        .quad   0x00d9cdfd69771d02+        .quad   0x410276cd6cfbf17e+        .quad   0x4c45306c1cb12ec7+        .quad   0x2857bf1627500861+        .quad   0x9f21903f0101689e+        .quad   0xd779dfd3bf861005+        .quad   0xa122ee5f3deb0f1b+        .quad   0x510df84b485a00d4++        // 2^132 * 5 * B++        .quad   0xa54133bb9277a1fa+        .quad   0x74ec3b6263991237+        .quad   0x1a3c54dc35d2f15a+        .quad   0x2d347144e482ba3a+        .quad   0x24b3c887c70ac15e+        .quad   0xb0f3a557fb81b732+        .quad   0x9b2cde2fe578cc1b+        .quad   0x4cf7ed0703b54f8e+        .quad   0x6bd47c6598fbee0f+        .quad   0x9e4733e2ab55be2d+        .quad   0x1093f624127610c5+        .quad   0x4e05e26ad0a1eaa4++        // 2^132 * 6 * B++        .quad   0xda9b6b624b531f20+        .quad   0x429a760e77509abb+        .quad   0xdbe9f522e823cb80+        .quad   0x618f1856880c8f82+        .quad   0x1833c773e18fe6c0+        .quad   0xe3c4711ad3c87265+        .quad   0x3bfd3c4f0116b283+        .quad   0x1955875eb4cd4db8+        .quad   0x6da6de8f0e399799+        .quad   0x7ad61aa440fda178+        .quad   0xb32cd8105e3563dd+        .quad   0x15f6beae2ae340ae++        // 2^132 * 7 * B++        .quad   0x862bcb0c31ec3a62+        .quad   0x810e2b451138f3c2+        .quad   0x788ec4b839dac2a4+        .quad   0x28f76867ae2a9281+        .quad   0xba9a0f7b9245e215+        .quad   0xf368612dd98c0dbb+        .quad   0x2e84e4cbf220b020+        .quad   0x6ba92fe962d90eda+        .quad   0x3e4df9655884e2aa+        .quad   0xbd62fbdbdbd465a5+        .quad   0xd7596caa0de9e524+        .quad   0x6e8042ccb2b1b3d7++        // 2^132 * 8 * B++        .quad   0xf10d3c29ce28ca6e+        .quad   0xbad34540fcb6093d+        .quad   0xe7426ed7a2ea2d3f+        .quad   0x08af9d4e4ff298b9+        .quad   0x1530653616521f7e+        .quad   0x660d06b896203dba+        .quad   0x2d3989bc545f0879+        .quad   0x4b5303af78ebd7b0+        .quad   0x72f8a6c3bebcbde8+        .quad   0x4f0fca4adc3a8e89+        .quad   0x6fa9d4e8c7bfdf7a+        .quad   0x0dcf2d679b624eb7++        // 2^136 * 1 * B++        .quad   0x3d5947499718289c+        .quad   0x12ebf8c524533f26+        .quad   0x0262bfcb14c3ef15+        .quad   0x20b878d577b7518e+        .quad   0x753941be5a45f06e+        .quad   0xd07caeed6d9c5f65+        .quad   0x11776b9c72ff51b6+        .quad   0x17d2d1d9ef0d4da9+        .quad   0x27f2af18073f3e6a+        .quad   0xfd3fe519d7521069+        .quad   0x22e3b72c3ca60022+        .quad   0x72214f63cc65c6a7++        // 2^136 * 2 * B++        .quad   0xb4e37f405307a693+        .quad   0xaba714d72f336795+        .quad   0xd6fbd0a773761099+        .quad   0x5fdf48c58171cbc9+        .quad   0x1d9db7b9f43b29c9+        .quad   0xd605824a4f518f75+        .quad   0xf2c072bd312f9dc4+        .quad   0x1f24ac855a1545b0+        .quad   0x24d608328e9505aa+        .quad   0x4748c1d10c1420ee+        .quad   0xc7ffe45c06fb25a2+        .quad   0x00ba739e2ae395e6++        // 2^136 * 3 * B++        .quad   0x592e98de5c8790d6+        .quad   0xe5bfb7d345c2a2df+        .quad   0x115a3b60f9b49922+        .quad   0x03283a3e67ad78f3+        .quad   0xae4426f5ea88bb26+        .quad   0x360679d984973bfb+        .quad   0x5c9f030c26694e50+        .quad   0x72297de7d518d226+        .quad   0x48241dc7be0cb939+        .quad   0x32f19b4d8b633080+        .quad   0xd3dfc90d02289308+        .quad   0x05e1296846271945++        // 2^136 * 4 * B++        .quad   0xba82eeb32d9c495a+        .quad   0xceefc8fcf12bb97c+        .quad   0xb02dabae93b5d1e0+        .quad   0x39c00c9c13698d9b+        .quad   0xadbfbbc8242c4550+        .quad   0xbcc80cecd03081d9+        .quad   0x843566a6f5c8df92+        .quad   0x78cf25d38258ce4c+        .quad   0x15ae6b8e31489d68+        .quad   0xaa851cab9c2bf087+        .quad   0xc9a75a97f04efa05+        .quad   0x006b52076b3ff832++        // 2^136 * 5 * B++        .quad   0x29e0cfe19d95781c+        .quad   0xb681df18966310e2+        .quad   0x57df39d370516b39+        .quad   0x4d57e3443bc76122+        .quad   0xf5cb7e16b9ce082d+        .quad   0x3407f14c417abc29+        .quad   0xd4b36bce2bf4a7ab+        .quad   0x7de2e9561a9f75ce+        .quad   0xde70d4f4b6a55ecb+        .quad   0x4801527f5d85db99+        .quad   0xdbc9c440d3ee9a81+        .quad   0x6b2a90af1a6029ed++        // 2^136 * 6 * B++        .quad   0x6923f4fc9ae61e97+        .quad   0x5735281de03f5fd1+        .quad   0xa764ae43e6edd12d+        .quad   0x5fd8f4e9d12d3e4a+        .quad   0x77ebf3245bb2d80a+        .quad   0xd8301b472fb9079b+        .quad   0xc647e6f24cee7333+        .quad   0x465812c8276c2109+        .quad   0x4d43beb22a1062d9+        .quad   0x7065fb753831dc16+        .quad   0x180d4a7bde2968d7+        .quad   0x05b32c2b1cb16790++        // 2^136 * 7 * B++        .quad   0xc8c05eccd24da8fd+        .quad   0xa1cf1aac05dfef83+        .quad   0xdbbeeff27df9cd61+        .quad   0x3b5556a37b471e99+        .quad   0xf7fca42c7ad58195+        .quad   0x3214286e4333f3cc+        .quad   0xb6c29d0d340b979d+        .quad   0x31771a48567307e1+        .quad   0x32b0c524e14dd482+        .quad   0xedb351541a2ba4b6+        .quad   0xa3d16048282b5af3+        .quad   0x4fc079d27a7336eb++        // 2^136 * 8 * B++        .quad   0x51c938b089bf2f7f+        .quad   0x2497bd6502dfe9a7+        .quad   0xffffc09c7880e453+        .quad   0x124567cecaf98e92+        .quad   0xdc348b440c86c50d+        .quad   0x1337cbc9cc94e651+        .quad   0x6422f74d643e3cb9+        .quad   0x241170c2bae3cd08+        .quad   0x3ff9ab860ac473b4+        .quad   0xf0911dee0113e435+        .quad   0x4ae75060ebc6c4af+        .quad   0x3f8612966c87000d++        // 2^140 * 1 * B++        .quad   0x0c9c5303f7957be4+        .quad   0xa3c31a20e085c145+        .quad   0xb0721d71d0850050+        .quad   0x0aba390eab0bf2da+        .quad   0x529fdffe638c7bf3+        .quad   0xdf2b9e60388b4995+        .quad   0xe027b34f1bad0249+        .quad   0x7bc92fc9b9fa74ed+        .quad   0x9f97ef2e801ad9f9+        .quad   0x83697d5479afda3a+        .quad   0xe906b3ffbd596b50+        .quad   0x02672b37dd3fb8e0++        // 2^140 * 2 * B++        .quad   0x48b2ca8b260885e4+        .quad   0xa4286bec82b34c1c+        .quad   0x937e1a2617f58f74+        .quad   0x741d1fcbab2ca2a5+        .quad   0xee9ba729398ca7f5+        .quad   0xeb9ca6257a4849db+        .quad   0x29eb29ce7ec544e1+        .quad   0x232ca21ef736e2c8+        .quad   0xbf61423d253fcb17+        .quad   0x08803ceafa39eb14+        .quad   0xf18602df9851c7af+        .quad   0x0400f3a049e3414b++        // 2^140 * 3 * B++        .quad   0xabce0476ba61c55b+        .quad   0x36a3d6d7c4d39716+        .quad   0x6eb259d5e8d82d09+        .quad   0x0c9176e984d756fb+        .quad   0x2efba412a06e7b06+        .quad   0x146785452c8d2560+        .quad   0xdf9713ebd67a91c7+        .quad   0x32830ac7157eadf3+        .quad   0x0e782a7ab73769e8+        .quad   0x04a05d7875b18e2c+        .quad   0x29525226ebcceae1+        .quad   0x0d794f8383eba820++        // 2^140 * 4 * B++        .quad   0xff35f5cb9e1516f4+        .quad   0xee805bcf648aae45+        .quad   0xf0d73c2bb93a9ef3+        .quad   0x097b0bf22092a6c2+        .quad   0x7be44ce7a7a2e1ac+        .quad   0x411fd93efad1b8b7+        .quad   0x1734a1d70d5f7c9b+        .quad   0x0d6592233127db16+        .quad   0xc48bab1521a9d733+        .quad   0xa6c2eaead61abb25+        .quad   0x625c6c1cc6cb4305+        .quad   0x7fc90fea93eb3a67++        // 2^140 * 5 * B++        .quad   0x0408f1fe1f5c5926+        .quad   0x1a8f2f5e3b258bf4+        .quad   0x40a951a2fdc71669+        .quad   0x6598ee93c98b577e+        .quad   0xc527deb59c7cb23d+        .quad   0x955391695328404e+        .quad   0xd64392817ccf2c7a+        .quad   0x6ce97dabf7d8fa11+        .quad   0x25b5a8e50ef7c48f+        .quad   0xeb6034116f2ce532+        .quad   0xc5e75173e53de537+        .quad   0x73119fa08c12bb03++        // 2^140 * 6 * B++        .quad   0xed30129453f1a4cb+        .quad   0xbce621c9c8f53787+        .quad   0xfacb2b1338bee7b9+        .quad   0x3025798a9ea8428c+        .quad   0x7845b94d21f4774d+        .quad   0xbf62f16c7897b727+        .quad   0x671857c03c56522b+        .quad   0x3cd6a85295621212+        .quad   0x3fecde923aeca999+        .quad   0xbdaa5b0062e8c12f+        .quad   0x67b99dfc96988ade+        .quad   0x3f52c02852661036++        // 2^140 * 7 * B++        .quad   0xffeaa48e2a1351c6+        .quad   0x28624754fa7f53d7+        .quad   0x0b5ba9e57582ddf1+        .quad   0x60c0104ba696ac59+        .quad   0x9258bf99eec416c6+        .quad   0xac8a5017a9d2f671+        .quad   0x629549ab16dea4ab+        .quad   0x05d0e85c99091569+        .quad   0x051de020de9cbe97+        .quad   0xfa07fc56b50bcf74+        .quad   0x378cec9f0f11df65+        .quad   0x36853c69ab96de4d++        // 2^140 * 8 * B++        .quad   0x36d9b8de78f39b2d+        .quad   0x7f42ed71a847b9ec+        .quad   0x241cd1d679bd3fde+        .quad   0x6a704fec92fbce6b+        .quad   0x4433c0b0fac5e7be+        .quad   0x724bae854c08dcbe+        .quad   0xf1f24cc446978f9b+        .quad   0x4a0aff6d62825fc8+        .quad   0xe917fb9e61095301+        .quad   0xc102df9402a092f8+        .quad   0xbf09e2f5fa66190b+        .quad   0x681109bee0dcfe37++        // 2^144 * 1 * B++        .quad   0x559a0cc9782a0dde+        .quad   0x551dcdb2ea718385+        .quad   0x7f62865b31ef238c+        .quad   0x504aa7767973613d+        .quad   0x9c18fcfa36048d13+        .quad   0x29159db373899ddd+        .quad   0xdc9f350b9f92d0aa+        .quad   0x26f57eee878a19d4+        .quad   0x0cab2cd55687efb1+        .quad   0x5180d162247af17b+        .quad   0x85c15a344f5a2467+        .quad   0x4041943d9dba3069++        // 2^144 * 2 * B++        .quad   0xc3c0eeba43ebcc96+        .quad   0x8d749c9c26ea9caf+        .quad   0xd9fa95ee1c77ccc6+        .quad   0x1420a1d97684340f+        .quad   0x4b217743a26caadd+        .quad   0x47a6b424648ab7ce+        .quad   0xcb1d4f7a03fbc9e3+        .quad   0x12d931429800d019+        .quad   0x00c67799d337594f+        .quad   0x5e3c5140b23aa47b+        .quad   0x44182854e35ff395+        .quad   0x1b4f92314359a012++        // 2^144 * 3 * B++        .quad   0x3e5c109d89150951+        .quad   0x39cefa912de9696a+        .quad   0x20eae43f975f3020+        .quad   0x239b572a7f132dae+        .quad   0x33cf3030a49866b1+        .quad   0x251f73d2215f4859+        .quad   0xab82aa4051def4f6+        .quad   0x5ff191d56f9a23f6+        .quad   0x819ed433ac2d9068+        .quad   0x2883ab795fc98523+        .quad   0xef4572805593eb3d+        .quad   0x020c526a758f36cb++        // 2^144 * 4 * B++        .quad   0x779834f89ed8dbbc+        .quad   0xc8f2aaf9dc7ca46c+        .quad   0xa9524cdca3e1b074+        .quad   0x02aacc4615313877+        .quad   0xe931ef59f042cc89+        .quad   0x2c589c9d8e124bb6+        .quad   0xadc8e18aaec75997+        .quad   0x452cfe0a5602c50c+        .quad   0x86a0f7a0647877df+        .quad   0xbbc464270e607c9f+        .quad   0xab17ea25f1fb11c9+        .quad   0x4cfb7d7b304b877b++        // 2^144 * 5 * B++        .quad   0x72b43d6cb89b75fe+        .quad   0x54c694d99c6adc80+        .quad   0xb8c3aa373ee34c9f+        .quad   0x14b4622b39075364+        .quad   0xe28699c29789ef12+        .quad   0x2b6ecd71df57190d+        .quad   0xc343c857ecc970d0+        .quad   0x5b1d4cbc434d3ac5+        .quad   0xb6fb2615cc0a9f26+        .quad   0x3a4f0e2bb88dcce5+        .quad   0x1301498b3369a705+        .quad   0x2f98f71258592dd1++        // 2^144 * 6 * B++        .quad   0x0c94a74cb50f9e56+        .quad   0x5b1ff4a98e8e1320+        .quad   0x9a2acc2182300f67+        .quad   0x3a6ae249d806aaf9+        .quad   0x2e12ae444f54a701+        .quad   0xfcfe3ef0a9cbd7de+        .quad   0xcebf890d75835de0+        .quad   0x1d8062e9e7614554+        .quad   0x657ada85a9907c5a+        .quad   0x1a0ea8b591b90f62+        .quad   0x8d0e1dfbdf34b4e9+        .quad   0x298b8ce8aef25ff3++        // 2^144 * 7 * B++        .quad   0x2a927953eff70cb2+        .quad   0x4b89c92a79157076+        .quad   0x9418457a30a7cf6a+        .quad   0x34b8a8404d5ce485+        .quad   0x837a72ea0a2165de+        .quad   0x3fab07b40bcf79f6+        .quad   0x521636c77738ae70+        .quad   0x6ba6271803a7d7dc+        .quad   0xc26eecb583693335+        .quad   0xd5a813df63b5fefd+        .quad   0xa293aa9aa4b22573+        .quad   0x71d62bdd465e1c6a++        // 2^144 * 8 * B++        .quad   0x6533cc28d378df80+        .quad   0xf6db43790a0fa4b4+        .quad   0xe3645ff9f701da5a+        .quad   0x74d5f317f3172ba4+        .quad   0xcd2db5dab1f75ef5+        .quad   0xd77f95cf16b065f5+        .quad   0x14571fea3f49f085+        .quad   0x1c333621262b2b3d+        .quad   0xa86fe55467d9ca81+        .quad   0x398b7c752b298c37+        .quad   0xda6d0892e3ac623b+        .quad   0x4aebcc4547e9d98c++        // 2^148 * 1 * B++        .quad   0x53175a7205d21a77+        .quad   0xb0c04422d3b934d4+        .quad   0xadd9f24bdd5deadc+        .quad   0x074f46e69f10ff8c+        .quad   0x0de9b204a059a445+        .quad   0xe15cb4aa4b17ad0f+        .quad   0xe1bbec521f79c557+        .quad   0x2633f1b9d071081b+        .quad   0xc1fb4177018b9910+        .quad   0xa6ea20dc6c0fe140+        .quad   0xd661f3e74354c6ff+        .quad   0x5ecb72e6f1a3407a++        // 2^148 * 2 * B++        .quad   0xa515a31b2259fb4e+        .quad   0x0960f3972bcac52f+        .quad   0xedb52fec8d3454cb+        .quad   0x382e2720c476c019+        .quad   0xfeeae106e8e86997+        .quad   0x9863337f98d09383+        .quad   0x9470480eaa06ebef+        .quad   0x038b6898d4c5c2d0+        .quad   0xf391c51d8ace50a6+        .quad   0x3142d0b9ae2d2948+        .quad   0xdb4d5a1a7f24ca80+        .quad   0x21aeba8b59250ea8++        // 2^148 * 3 * B++        .quad   0x24f13b34cf405530+        .quad   0x3c44ea4a43088af7+        .quad   0x5dd5c5170006a482+        .quad   0x118eb8f8890b086d+        .quad   0x53853600f0087f23+        .quad   0x4c461879da7d5784+        .quad   0x6af303deb41f6860+        .quad   0x0a3c16c5c27c18ed+        .quad   0x17e49c17cc947f3d+        .quad   0xccc6eda6aac1d27b+        .quad   0xdf6092ceb0f08e56+        .quad   0x4909b3e22c67c36b++        // 2^148 * 4 * B++        .quad   0x9c9c85ea63fe2e89+        .quad   0xbe1baf910e9412ec+        .quad   0x8f7baa8a86fbfe7b+        .quad   0x0fb17f9fef968b6c+        .quad   0x59a16676706ff64e+        .quad   0x10b953dd0d86a53d+        .quad   0x5848e1e6ce5c0b96+        .quad   0x2d8b78e712780c68+        .quad   0x79d5c62eafc3902b+        .quad   0x773a215289e80728+        .quad   0xc38ae640e10120b9+        .quad   0x09ae23717b2b1a6d++        // 2^148 * 5 * B++        .quad   0xbb6a192a4e4d083c+        .quad   0x34ace0630029e192+        .quad   0x98245a59aafabaeb+        .quad   0x6d9c8a9ada97faac+        .quad   0x10ab8fa1ad32b1d0+        .quad   0xe9aced1be2778b24+        .quad   0xa8856bc0373de90f+        .quad   0x66f35ddddda53996+        .quad   0xd27d9afb24997323+        .quad   0x1bb7e07ef6f01d2e+        .quad   0x2ba7472df52ecc7f+        .quad   0x03019b4f646f9dc8++        // 2^148 * 6 * B++        .quad   0x04a186b5565345cd+        .quad   0xeee76610bcc4116a+        .quad   0x689c73b478fb2a45+        .quad   0x387dcbff65697512+        .quad   0xaf09b214e6b3dc6b+        .quad   0x3f7573b5ad7d2f65+        .quad   0xd019d988100a23b0+        .quad   0x392b63a58b5c35f7+        .quad   0x4093addc9c07c205+        .quad   0xc565be15f532c37e+        .quad   0x63dbecfd1583402a+        .quad   0x61722b4aef2e032e++        // 2^148 * 7 * B++        .quad   0x0012aafeecbd47af+        .quad   0x55a266fb1cd46309+        .quad   0xf203eb680967c72c+        .quad   0x39633944ca3c1429+        .quad   0xd6b07a5581cb0e3c+        .quad   0x290ff006d9444969+        .quad   0x08680b6a16dcda1f+        .quad   0x5568d2b75a06de59+        .quad   0x8d0cb88c1b37cfe1+        .quad   0x05b6a5a3053818f3+        .quad   0xf2e9bc04b787d959+        .quad   0x6beba1249add7f64++        // 2^148 * 8 * B++        .quad   0x1d06005ca5b1b143+        .quad   0x6d4c6bb87fd1cda2+        .quad   0x6ef5967653fcffe7+        .quad   0x097c29e8c1ce1ea5+        .quad   0x5c3cecb943f5a53b+        .quad   0x9cc9a61d06c08df2+        .quad   0xcfba639a85895447+        .quad   0x5a845ae80df09fd5+        .quad   0x4ce97dbe5deb94ca+        .quad   0x38d0a4388c709c48+        .quad   0xc43eced4a169d097+        .quad   0x0a1249fff7e587c3++        // 2^152 * 1 * B++        .quad   0x12f0071b276d01c9+        .quad   0xe7b8bac586c48c70+        .quad   0x5308129b71d6fba9+        .quad   0x5d88fbf95a3db792+        .quad   0x0b408d9e7354b610+        .quad   0x806b32535ba85b6e+        .quad   0xdbe63a034a58a207+        .quad   0x173bd9ddc9a1df2c+        .quad   0x2b500f1efe5872df+        .quad   0x58d6582ed43918c1+        .quad   0xe6ed278ec9673ae0+        .quad   0x06e1cd13b19ea319++        // 2^152 * 2 * B++        .quad   0x40d0ad516f166f23+        .quad   0x118e32931fab6abe+        .quad   0x3fe35e14a04d088e+        .quad   0x3080603526e16266+        .quad   0x472baf629e5b0353+        .quad   0x3baa0b90278d0447+        .quad   0x0c785f469643bf27+        .quad   0x7f3a6a1a8d837b13+        .quad   0xf7e644395d3d800b+        .quad   0x95a8d555c901edf6+        .quad   0x68cd7830592c6339+        .quad   0x30d0fded2e51307e++        // 2^152 * 3 * B++        .quad   0xe0594d1af21233b3+        .quad   0x1bdbe78ef0cc4d9c+        .quad   0x6965187f8f499a77+        .quad   0x0a9214202c099868+        .quad   0x9cb4971e68b84750+        .quad   0xa09572296664bbcf+        .quad   0x5c8de72672fa412b+        .quad   0x4615084351c589d9+        .quad   0xbc9019c0aeb9a02e+        .quad   0x55c7110d16034cae+        .quad   0x0e6df501659932ec+        .quad   0x3bca0d2895ca5dfe++        // 2^152 * 4 * B++        .quad   0x40f031bc3c5d62a4+        .quad   0x19fc8b3ecff07a60+        .quad   0x98183da2130fb545+        .quad   0x5631deddae8f13cd+        .quad   0x9c688eb69ecc01bf+        .quad   0xf0bc83ada644896f+        .quad   0xca2d955f5f7a9fe2+        .quad   0x4ea8b4038df28241+        .quad   0x2aed460af1cad202+        .quad   0x46305305a48cee83+        .quad   0x9121774549f11a5f+        .quad   0x24ce0930542ca463++        // 2^152 * 5 * B++        .quad   0x1fe890f5fd06c106+        .quad   0xb5c468355d8810f2+        .quad   0x827808fe6e8caf3e+        .quad   0x41d4e3c28a06d74b+        .quad   0x3fcfa155fdf30b85+        .quad   0xd2f7168e36372ea4+        .quad   0xb2e064de6492f844+        .quad   0x549928a7324f4280+        .quad   0xf26e32a763ee1a2e+        .quad   0xae91e4b7d25ffdea+        .quad   0xbc3bd33bd17f4d69+        .quad   0x491b66dec0dcff6a++        // 2^152 * 6 * B++        .quad   0x98f5b13dc7ea32a7+        .quad   0xe3d5f8cc7e16db98+        .quad   0xac0abf52cbf8d947+        .quad   0x08f338d0c85ee4ac+        .quad   0x75f04a8ed0da64a1+        .quad   0xed222caf67e2284b+        .quad   0x8234a3791f7b7ba4+        .quad   0x4cf6b8b0b7018b67+        .quad   0xc383a821991a73bd+        .quad   0xab27bc01df320c7a+        .quad   0xc13d331b84777063+        .quad   0x530d4a82eb078a99++        // 2^152 * 7 * B++        .quad   0x004c3630e1f94825+        .quad   0x7e2d78268cab535a+        .quad   0xc7482323cc84ff8b+        .quad   0x65ea753f101770b9+        .quad   0x6d6973456c9abf9e+        .quad   0x257fb2fc4900a880+        .quad   0x2bacf412c8cfb850+        .quad   0x0db3e7e00cbfbd5b+        .quad   0x3d66fc3ee2096363+        .quad   0x81d62c7f61b5cb6b+        .quad   0x0fbe044213443b1a+        .quad   0x02a4ec1921e1a1db++        // 2^152 * 8 * B++        .quad   0x5ce6259a3b24b8a2+        .quad   0xb8577acc45afa0b8+        .quad   0xcccbe6e88ba07037+        .quad   0x3d143c51127809bf+        .quad   0xf5c86162f1cf795f+        .quad   0x118c861926ee57f2+        .quad   0x172124851c063578+        .quad   0x36d12b5dec067fcf+        .quad   0x126d279179154557+        .quad   0xd5e48f5cfc783a0a+        .quad   0x36bdb6e8df179bac+        .quad   0x2ef517885ba82859++        // 2^156 * 1 * B++        .quad   0x88bd438cd11e0d4a+        .quad   0x30cb610d43ccf308+        .quad   0xe09a0e3791937bcc+        .quad   0x4559135b25b1720c+        .quad   0x1ea436837c6da1e9+        .quad   0xf9c189af1fb9bdbe+        .quad   0x303001fcce5dd155+        .quad   0x28a7c99ebc57be52+        .quad   0xb8fd9399e8d19e9d+        .quad   0x908191cb962423ff+        .quad   0xb2b948d747c742a3+        .quad   0x37f33226d7fb44c4++        // 2^156 * 2 * B++        .quad   0x0dae8767b55f6e08+        .quad   0x4a43b3b35b203a02+        .quad   0xe3725a6e80af8c79+        .quad   0x0f7a7fd1705fa7a3+        .quad   0x33912553c821b11d+        .quad   0x66ed42c241e301df+        .quad   0x066fcc11104222fd+        .quad   0x307a3b41c192168f+        .quad   0x8eeb5d076eb55ce0+        .quad   0x2fc536bfaa0d925a+        .quad   0xbe81830fdcb6c6e8+        .quad   0x556c7045827baf52++        // 2^156 * 3 * B++        .quad   0x8e2b517302e9d8b7+        .quad   0xe3e52269248714e8+        .quad   0xbd4fbd774ca960b5+        .quad   0x6f4b4199c5ecada9+        .quad   0xb94b90022bf44406+        .quad   0xabd4237eff90b534+        .quad   0x7600a960faf86d3a+        .quad   0x2f45abdac2322ee3+        .quad   0x61af4912c8ef8a6a+        .quad   0xe58fa4fe43fb6e5e+        .quad   0xb5afcc5d6fd427cf+        .quad   0x6a5393281e1e11eb++        // 2^156 * 4 * B++        .quad   0xf3da5139a5d1ee89+        .quad   0x8145457cff936988+        .quad   0x3f622fed00e188c4+        .quad   0x0f513815db8b5a3d+        .quad   0x0fff04fe149443cf+        .quad   0x53cac6d9865cddd7+        .quad   0x31385b03531ed1b7+        .quad   0x5846a27cacd1039d+        .quad   0x4ff5cdac1eb08717+        .quad   0x67e8b29590f2e9bc+        .quad   0x44093b5e237afa99+        .quad   0x0d414bed8708b8b2++        // 2^156 * 5 * B++        .quad   0xcfb68265fd0e75f6+        .quad   0xe45b3e28bb90e707+        .quad   0x7242a8de9ff92c7a+        .quad   0x685b3201933202dd+        .quad   0x81886a92294ac9e8+        .quad   0x23162b45d55547be+        .quad   0x94cfbc4403715983+        .quad   0x50eb8fdb134bc401+        .quad   0xc0b73ec6d6b330cd+        .quad   0x84e44807132faff1+        .quad   0x732b7352c4a5dee1+        .quad   0x5d7c7cf1aa7cd2d2++        // 2^156 * 6 * B++        .quad   0xaf3b46bf7a4aafa2+        .quad   0xb78705ec4d40d411+        .quad   0x114f0c6aca7c15e3+        .quad   0x3f364faaa9489d4d+        .quad   0x33d1013e9b73a562+        .quad   0x925cef5748ec26e1+        .quad   0xa7fce614dd468058+        .quad   0x78b0fad41e9aa438+        .quad   0xbf56a431ed05b488+        .quad   0xa533e66c9c495c7e+        .quad   0xe8652baf87f3651a+        .quad   0x0241800059d66c33++        // 2^156 * 7 * B++        .quad   0xceb077fea37a5be4+        .quad   0xdb642f02e5a5eeb7+        .quad   0xc2e6d0c5471270b8+        .quad   0x4771b65538e4529c+        .quad   0x28350c7dcf38ea01+        .quad   0x7c6cdbc0b2917ab6+        .quad   0xace7cfbe857082f7+        .quad   0x4d2845aba2d9a1e0+        .quad   0xbb537fe0447070de+        .quad   0xcba744436dd557df+        .quad   0xd3b5a3473600dbcb+        .quad   0x4aeabbe6f9ffd7f8++        // 2^156 * 8 * B++        .quad   0x4630119e40d8f78c+        .quad   0xa01a9bc53c710e11+        .quad   0x486d2b258910dd79+        .quad   0x1e6c47b3db0324e5+        .quad   0x6a2134bcc4a9c8f2+        .quad   0xfbf8fd1c8ace2e37+        .quad   0x000ae3049911a0ba+        .quad   0x046e3a616bc89b9e+        .quad   0x14e65442f03906be+        .quad   0x4a019d54e362be2a+        .quad   0x68ccdfec8dc230c7+        .quad   0x7cfb7e3faf6b861c++        // 2^160 * 1 * B++        .quad   0x4637974e8c58aedc+        .quad   0xb9ef22fbabf041a4+        .quad   0xe185d956e980718a+        .quad   0x2f1b78fab143a8a6+        .quad   0x96eebffb305b2f51+        .quad   0xd3f938ad889596b8+        .quad   0xf0f52dc746d5dd25+        .quad   0x57968290bb3a0095+        .quad   0xf71ab8430a20e101+        .quad   0xf393658d24f0ec47+        .quad   0xcf7509a86ee2eed1+        .quad   0x7dc43e35dc2aa3e1++        // 2^160 * 2 * B++        .quad   0x85966665887dd9c3+        .quad   0xc90f9b314bb05355+        .quad   0xc6e08df8ef2079b1+        .quad   0x7ef72016758cc12f+        .quad   0x5a782a5c273e9718+        .quad   0x3576c6995e4efd94+        .quad   0x0f2ed8051f237d3e+        .quad   0x044fb81d82d50a99+        .quad   0xc1df18c5a907e3d9+        .quad   0x57b3371dce4c6359+        .quad   0xca704534b201bb49+        .quad   0x7f79823f9c30dd2e++        // 2^160 * 3 * B++        .quad   0x8334d239a3b513e8+        .quad   0xc13670d4b91fa8d8+        .quad   0x12b54136f590bd33+        .quad   0x0a4e0373d784d9b4+        .quad   0x6a9c1ff068f587ba+        .quad   0x0827894e0050c8de+        .quad   0x3cbf99557ded5be7+        .quad   0x64a9b0431c06d6f0+        .quad   0x2eb3d6a15b7d2919+        .quad   0xb0b4f6a0d53a8235+        .quad   0x7156ce4389a45d47+        .quad   0x071a7d0ace18346c++        // 2^160 * 4 * B++        .quad   0xd3072daac887ba0b+        .quad   0x01262905bfa562ee+        .quad   0xcf543002c0ef768b+        .quad   0x2c3bcc7146ea7e9c+        .quad   0xcc0c355220e14431+        .quad   0x0d65950709b15141+        .quad   0x9af5621b209d5f36+        .quad   0x7c69bcf7617755d3+        .quad   0x07f0d7eb04e8295f+        .quad   0x10db18252f50f37d+        .quad   0xe951a9a3171798d7+        .quad   0x6f5a9a7322aca51d++        // 2^160 * 5 * B++        .quad   0x8ba1000c2f41c6c5+        .quad   0xc49f79c10cfefb9b+        .quad   0x4efa47703cc51c9f+        .quad   0x494e21a2e147afca+        .quad   0xe729d4eba3d944be+        .quad   0x8d9e09408078af9e+        .quad   0x4525567a47869c03+        .quad   0x02ab9680ee8d3b24+        .quad   0xefa48a85dde50d9a+        .quad   0x219a224e0fb9a249+        .quad   0xfa091f1dd91ef6d9+        .quad   0x6b5d76cbea46bb34++        // 2^160 * 6 * B++        .quad   0x8857556cec0cd994+        .quad   0x6472dc6f5cd01dba+        .quad   0xaf0169148f42b477+        .quad   0x0ae333f685277354+        .quad   0xe0f941171e782522+        .quad   0xf1e6ae74036936d3+        .quad   0x408b3ea2d0fcc746+        .quad   0x16fb869c03dd313e+        .quad   0x288e199733b60962+        .quad   0x24fc72b4d8abe133+        .quad   0x4811f7ed0991d03e+        .quad   0x3f81e38b8f70d075++        // 2^160 * 7 * B++        .quad   0x7f910fcc7ed9affe+        .quad   0x545cb8a12465874b+        .quad   0xa8397ed24b0c4704+        .quad   0x50510fc104f50993+        .quad   0x0adb7f355f17c824+        .quad   0x74b923c3d74299a4+        .quad   0xd57c3e8bcbf8eaf7+        .quad   0x0ad3e2d34cdedc3d+        .quad   0x6f0c0fc5336e249d+        .quad   0x745ede19c331cfd9+        .quad   0xf2d6fd0009eefe1c+        .quad   0x127c158bf0fa1ebe++        // 2^160 * 8 * B++        .quad   0xf6197c422e9879a2+        .quad   0xa44addd452ca3647+        .quad   0x9b413fc14b4eaccb+        .quad   0x354ef87d07ef4f68+        .quad   0xdea28fc4ae51b974+        .quad   0x1d9973d3744dfe96+        .quad   0x6240680b873848a8+        .quad   0x4ed82479d167df95+        .quad   0xfee3b52260c5d975+        .quad   0x50352efceb41b0b8+        .quad   0x8808ac30a9f6653c+        .quad   0x302d92d20539236d++        // 2^164 * 1 * B++        .quad   0x4c59023fcb3efb7c+        .quad   0x6c2fcb99c63c2a94+        .quad   0xba4190e2c3c7e084+        .quad   0x0e545daea51874d9+        .quad   0x957b8b8b0df53c30+        .quad   0x2a1c770a8e60f098+        .quad   0xbbc7a670345796de+        .quad   0x22a48f9a90c99bc9+        .quad   0x6b7dc0dc8d3fac58+        .quad   0x5497cd6ce6e42bfd+        .quad   0x542f7d1bf400d305+        .quad   0x4159f47f048d9136++        // 2^164 * 2 * B++        .quad   0x20ad660839e31e32+        .quad   0xf81e1bd58405be50+        .quad   0xf8064056f4dabc69+        .quad   0x14d23dd4ce71b975+        .quad   0x748515a8bbd24839+        .quad   0x77128347afb02b55+        .quad   0x50ba2ac649a2a17f+        .quad   0x060525513ad730f1+        .quad   0xf2398e098aa27f82+        .quad   0x6d7982bb89a1b024+        .quad   0xfa694084214dd24c+        .quad   0x71ab966fa32301c3++        // 2^164 * 3 * B++        .quad   0x2dcbd8e34ded02fc+        .quad   0x1151f3ec596f22aa+        .quad   0xbca255434e0328da+        .quad   0x35768fbe92411b22+        .quad   0xb1088a0702809955+        .quad   0x43b273ea0b43c391+        .quad   0xca9b67aefe0686ed+        .quad   0x605eecbf8335f4ed+        .quad   0x83200a656c340431+        .quad   0x9fcd71678ee59c2f+        .quad   0x75d4613f71300f8a+        .quad   0x7a912faf60f542f9++        // 2^164 * 4 * B++        .quad   0xb204585e5edc1a43+        .quad   0x9f0e16ee5897c73c+        .quad   0x5b82c0ae4e70483c+        .quad   0x624a170e2bddf9be+        .quad   0x253f4f8dfa2d5597+        .quad   0x25e49c405477130c+        .quad   0x00c052e5996b1102+        .quad   0x33cb966e33bb6c4a+        .quad   0x597028047f116909+        .quad   0x828ac41c1e564467+        .quad   0x70417dbde6217387+        .quad   0x721627aefbac4384++        // 2^164 * 5 * B++        .quad   0x97d03bc38736add5+        .quad   0x2f1422afc532b130+        .quad   0x3aa68a057101bbc4+        .quad   0x4c946cf7e74f9fa7+        .quad   0xfd3097bc410b2f22+        .quad   0xf1a05da7b5cfa844+        .quad   0x61289a1def57ca74+        .quad   0x245ea199bb821902+        .quad   0xaedca66978d477f8+        .quad   0x1898ba3c29117fe1+        .quad   0xcf73f983720cbd58+        .quad   0x67da12e6b8b56351++        // 2^164 * 6 * B++        .quad   0x7067e187b4bd6e07+        .quad   0x6e8f0203c7d1fe74+        .quad   0x93c6aa2f38c85a30+        .quad   0x76297d1f3d75a78a+        .quad   0x2b7ef3d38ec8308c+        .quad   0x828fd7ec71eb94ab+        .quad   0x807c3b36c5062abd+        .quad   0x0cb64cb831a94141+        .quad   0x3030fc33534c6378+        .quad   0xb9635c5ce541e861+        .quad   0x15d9a9bed9b2c728+        .quad   0x49233ea3f3775dcb++        // 2^164 * 7 * B++        .quad   0x629398fa8dbffc3a+        .quad   0xe12fe52dd54db455+        .quad   0xf3be11dfdaf25295+        .quad   0x628b140dce5e7b51+        .quad   0x7b3985fe1c9f249b+        .quad   0x4fd6b2d5a1233293+        .quad   0xceb345941adf4d62+        .quad   0x6987ff6f542de50c+        .quad   0x47e241428f83753c+        .quad   0x6317bebc866af997+        .quad   0xdabb5b433d1a9829+        .quad   0x074d8d245287fb2d++        // 2^164 * 8 * B++        .quad   0x8337d9cd440bfc31+        .quad   0x729d2ca1af318fd7+        .quad   0xa040a4a4772c2070+        .quad   0x46002ef03a7349be+        .quad   0x481875c6c0e31488+        .quad   0x219429b2e22034b4+        .quad   0x7223c98a31283b65+        .quad   0x3420d60b342277f9+        .quad   0xfaa23adeaffe65f7+        .quad   0x78261ed45be0764c+        .quad   0x441c0a1e2f164403+        .quad   0x5aea8e567a87d395++        // 2^168 * 1 * B++        .quad   0x7813c1a2bca4283d+        .quad   0xed62f091a1863dd9+        .quad   0xaec7bcb8c268fa86+        .quad   0x10e5d3b76f1cae4c+        .quad   0x2dbc6fb6e4e0f177+        .quad   0x04e1bf29a4bd6a93+        .quad   0x5e1966d4787af6e8+        .quad   0x0edc5f5eb426d060+        .quad   0x5453bfd653da8e67+        .quad   0xe9dc1eec24a9f641+        .quad   0xbf87263b03578a23+        .quad   0x45b46c51361cba72++        // 2^168 * 2 * B++        .quad   0xa9402abf314f7fa1+        .quad   0xe257f1dc8e8cf450+        .quad   0x1dbbd54b23a8be84+        .quad   0x2177bfa36dcb713b+        .quad   0xce9d4ddd8a7fe3e4+        .quad   0xab13645676620e30+        .quad   0x4b594f7bb30e9958+        .quad   0x5c1c0aef321229df+        .quad   0x37081bbcfa79db8f+        .quad   0x6048811ec25f59b3+        .quad   0x087a76659c832487+        .quad   0x4ae619387d8ab5bb++        // 2^168 * 3 * B++        .quad   0x8ddbf6aa5344a32e+        .quad   0x7d88eab4b41b4078+        .quad   0x5eb0eb974a130d60+        .quad   0x1a00d91b17bf3e03+        .quad   0x61117e44985bfb83+        .quad   0xfce0462a71963136+        .quad   0x83ac3448d425904b+        .quad   0x75685abe5ba43d64+        .quad   0x6e960933eb61f2b2+        .quad   0x543d0fa8c9ff4952+        .quad   0xdf7275107af66569+        .quad   0x135529b623b0e6aa++        // 2^168 * 4 * B++        .quad   0x18f0dbd7add1d518+        .quad   0x979f7888cfc11f11+        .quad   0x8732e1f07114759b+        .quad   0x79b5b81a65ca3a01+        .quad   0xf5c716bce22e83fe+        .quad   0xb42beb19e80985c1+        .quad   0xec9da63714254aae+        .quad   0x5972ea051590a613+        .quad   0x0fd4ac20dc8f7811+        .quad   0x9a9ad294ac4d4fa8+        .quad   0xc01b2d64b3360434+        .quad   0x4f7e9c95905f3bdb++        // 2^168 * 5 * B++        .quad   0x62674bbc5781302e+        .quad   0xd8520f3989addc0f+        .quad   0x8c2999ae53fbd9c6+        .quad   0x31993ad92e638e4c+        .quad   0x71c8443d355299fe+        .quad   0x8bcd3b1cdbebead7+        .quad   0x8092499ef1a49466+        .quad   0x1942eec4a144adc8+        .quad   0x7dac5319ae234992+        .quad   0x2c1b3d910cea3e92+        .quad   0x553ce494253c1122+        .quad   0x2a0a65314ef9ca75++        // 2^168 * 6 * B++        .quad   0x2db7937ff7f927c2+        .quad   0xdb741f0617d0a635+        .quad   0x5982f3a21155af76+        .quad   0x4cf6e218647c2ded+        .quad   0xcf361acd3c1c793a+        .quad   0x2f9ebcac5a35bc3b+        .quad   0x60e860e9a8cda6ab+        .quad   0x055dc39b6dea1a13+        .quad   0xb119227cc28d5bb6+        .quad   0x07e24ebc774dffab+        .quad   0xa83c78cee4a32c89+        .quad   0x121a307710aa24b6++        // 2^168 * 7 * B++        .quad   0xe4db5d5e9f034a97+        .quad   0xe153fc093034bc2d+        .quad   0x460546919551d3b1+        .quad   0x333fc76c7a40e52d+        .quad   0xd659713ec77483c9+        .quad   0x88bfe077b82b96af+        .quad   0x289e28231097bcd3+        .quad   0x527bb94a6ced3a9b+        .quad   0x563d992a995b482e+        .quad   0x3405d07c6e383801+        .quad   0x485035de2f64d8e5+        .quad   0x6b89069b20a7a9f7++        // 2^168 * 8 * B++        .quad   0x812aa0416270220d+        .quad   0x995a89faf9245b4e+        .quad   0xffadc4ce5072ef05+        .quad   0x23bc2103aa73eb73+        .quad   0x4082fa8cb5c7db77+        .quad   0x068686f8c734c155+        .quad   0x29e6c8d9f6e7a57e+        .quad   0x0473d308a7639bcf+        .quad   0xcaee792603589e05+        .quad   0x2b4b421246dcc492+        .quad   0x02a1ef74e601a94f+        .quad   0x102f73bfde04341a++        // 2^172 * 1 * B++        .quad   0xb5a2d50c7ec20d3e+        .quad   0xc64bdd6ea0c97263+        .quad   0x56e89052c1ff734d+        .quad   0x4929c6f72b2ffaba+        .quad   0x358ecba293a36247+        .quad   0xaf8f9862b268fd65+        .quad   0x412f7e9968a01c89+        .quad   0x5786f312cd754524+        .quad   0x337788ffca14032c+        .quad   0xf3921028447f1ee3+        .quad   0x8b14071f231bccad+        .quad   0x4c817b4bf2344783++        // 2^172 * 2 * B++        .quad   0x0ff853852871b96e+        .quad   0xe13e9fab60c3f1bb+        .quad   0xeefd595325344402+        .quad   0x0a37c37075b7744b+        .quad   0x413ba057a40b4484+        .quad   0xba4c2e1a4f5f6a43+        .quad   0x614ba0a5aee1d61c+        .quad   0x78a1531a8b05dc53+        .quad   0x6cbdf1703ad0562b+        .quad   0x8ecf4830c92521a3+        .quad   0xdaebd303fd8424e7+        .quad   0x72ad82a42e5ec56f++        // 2^172 * 3 * B++        .quad   0x3f9e8e35bafb65f6+        .quad   0x39d69ec8f27293a1+        .quad   0x6cb8cd958cf6a3d0+        .quad   0x1734778173adae6d+        .quad   0xc368939167024bc3+        .quad   0x8e69d16d49502fda+        .quad   0xfcf2ec3ce45f4b29+        .quad   0x065f669ea3b4cbc4+        .quad   0x8a00aec75532db4d+        .quad   0xb869a4e443e31bb1+        .quad   0x4a0f8552d3a7f515+        .quad   0x19adeb7c303d7c08++        // 2^172 * 4 * B++        .quad   0xc720cb6153ead9a3+        .quad   0x55b2c97f512b636e+        .quad   0xb1e35b5fd40290b1+        .quad   0x2fd9ccf13b530ee2+        .quad   0x9d05ba7d43c31794+        .quad   0x2470c8ff93322526+        .quad   0x8323dec816197438+        .quad   0x2852709881569b53+        .quad   0x07bd475b47f796b8+        .quad   0xd2c7b013542c8f54+        .quad   0x2dbd23f43b24f87e+        .quad   0x6551afd77b0901d6++        // 2^172 * 5 * B++        .quad   0x4546baaf54aac27f+        .quad   0xf6f66fecb2a45a28+        .quad   0x582d1b5b562bcfe8+        .quad   0x44b123f3920f785f+        .quad   0x68a24ce3a1d5c9ac+        .quad   0xbb77a33d10ff6461+        .quad   0x0f86ce4425d3166e+        .quad   0x56507c0950b9623b+        .quad   0x1206f0b7d1713e63+        .quad   0x353fe3d915bafc74+        .quad   0x194ceb970ad9d94d+        .quad   0x62fadd7cf9d03ad3++        // 2^172 * 6 * B++        .quad   0xc6b5967b5598a074+        .quad   0x5efe91ce8e493e25+        .quad   0xd4b72c4549280888+        .quad   0x20ef1149a26740c2+        .quad   0x3cd7bc61e7ce4594+        .quad   0xcd6b35a9b7dd267e+        .quad   0xa080abc84366ef27+        .quad   0x6ec7c46f59c79711+        .quad   0x2f07ad636f09a8a2+        .quad   0x8697e6ce24205e7d+        .quad   0xc0aefc05ee35a139+        .quad   0x15e80958b5f9d897++        // 2^172 * 7 * B++        .quad   0x25a5ef7d0c3e235b+        .quad   0x6c39c17fbe134ee7+        .quad   0xc774e1342dc5c327+        .quad   0x021354b892021f39+        .quad   0x4dd1ed355bb061c4+        .quad   0x42dc0cef941c0700+        .quad   0x61305dc1fd86340e+        .quad   0x56b2cc930e55a443+        .quad   0x1df79da6a6bfc5a2+        .quad   0x02f3a2749fde4369+        .quad   0xb323d9f2cda390a7+        .quad   0x7be0847b8774d363++        // 2^172 * 8 * B++        .quad   0x8c99cc5a8b3f55c3+        .quad   0x0611d7253fded2a0+        .quad   0xed2995ff36b70a36+        .quad   0x1f699a54d78a2619+        .quad   0x1466f5af5307fa11+        .quad   0x817fcc7ded6c0af2+        .quad   0x0a6de44ec3a4a3fb+        .quad   0x74071475bc927d0b+        .quad   0xe77292f373e7ea8a+        .quad   0x296537d2cb045a31+        .quad   0x1bd0653ed3274fde+        .quad   0x2f9a2c4476bd2966++        // 2^176 * 1 * B++        .quad   0xeb18b9ab7f5745c6+        .quad   0x023a8aee5787c690+        .quad   0xb72712da2df7afa9+        .quad   0x36597d25ea5c013d+        .quad   0xa2b4dae0b5511c9a+        .quad   0x7ac860292bffff06+        .quad   0x981f375df5504234+        .quad   0x3f6bd725da4ea12d+        .quad   0x734d8d7b106058ac+        .quad   0xd940579e6fc6905f+        .quad   0x6466f8f99202932d+        .quad   0x7b7ecc19da60d6d0++        // 2^176 * 2 * B++        .quad   0x78c2373c695c690d+        .quad   0xdd252e660642906e+        .quad   0x951d44444ae12bd2+        .quad   0x4235ad7601743956+        .quad   0x6dae4a51a77cfa9b+        .quad   0x82263654e7a38650+        .quad   0x09bbffcd8f2d82db+        .quad   0x03bedc661bf5caba+        .quad   0x6258cb0d078975f5+        .quad   0x492942549189f298+        .quad   0xa0cab423e2e36ee4+        .quad   0x0e7ce2b0cdf066a1++        // 2^176 * 3 * B++        .quad   0xc494643ac48c85a3+        .quad   0xfd361df43c6139ad+        .quad   0x09db17dd3ae94d48+        .quad   0x666e0a5d8fb4674a+        .quad   0xfea6fedfd94b70f9+        .quad   0xf130c051c1fcba2d+        .quad   0x4882d47e7f2fab89+        .quad   0x615256138aeceeb5+        .quad   0x2abbf64e4870cb0d+        .quad   0xcd65bcf0aa458b6b+        .quad   0x9abe4eba75e8985d+        .quad   0x7f0bc810d514dee4++        // 2^176 * 4 * B++        .quad   0xb9006ba426f4136f+        .quad   0x8d67369e57e03035+        .quad   0xcbc8dfd94f463c28+        .quad   0x0d1f8dbcf8eedbf5+        .quad   0x83ac9dad737213a0+        .quad   0x9ff6f8ba2ef72e98+        .quad   0x311e2edd43ec6957+        .quad   0x1d3a907ddec5ab75+        .quad   0xba1693313ed081dc+        .quad   0x29329fad851b3480+        .quad   0x0128013c030321cb+        .quad   0x00011b44a31bfde3++        // 2^176 * 5 * B++        .quad   0x3fdfa06c3fc66c0c+        .quad   0x5d40e38e4dd60dd2+        .quad   0x7ae38b38268e4d71+        .quad   0x3ac48d916e8357e1+        .quad   0x16561f696a0aa75c+        .quad   0xc1bf725c5852bd6a+        .quad   0x11a8dd7f9a7966ad+        .quad   0x63d988a2d2851026+        .quad   0x00120753afbd232e+        .quad   0xe92bceb8fdd8f683+        .quad   0xf81669b384e72b91+        .quad   0x33fad52b2368a066++        // 2^176 * 6 * B++        .quad   0x540649c6c5e41e16+        .quad   0x0af86430333f7735+        .quad   0xb2acfcd2f305e746+        .quad   0x16c0f429a256dca7+        .quad   0x8d2cc8d0c422cfe8+        .quad   0x072b4f7b05a13acb+        .quad   0xa3feb6e6ecf6a56f+        .quad   0x3cc355ccb90a71e2+        .quad   0xe9b69443903e9131+        .quad   0xb8a494cb7a5637ce+        .quad   0xc87cd1a4baba9244+        .quad   0x631eaf426bae7568++        // 2^176 * 7 * B++        .quad   0xb3e90410da66fe9f+        .quad   0x85dd4b526c16e5a6+        .quad   0xbc3d97611ef9bf83+        .quad   0x5599648b1ea919b5+        .quad   0x47d975b9a3700de8+        .quad   0x7280c5fbe2f80552+        .quad   0x53658f2732e45de1+        .quad   0x431f2c7f665f80b5+        .quad   0xd6026344858f7b19+        .quad   0x14ab352fa1ea514a+        .quad   0x8900441a2090a9d7+        .quad   0x7b04715f91253b26++        // 2^176 * 8 * B++        .quad   0x83edbd28acf6ae43+        .quad   0x86357c8b7d5c7ab4+        .quad   0xc0404769b7eb2c44+        .quad   0x59b37bf5c2f6583f+        .quad   0xb376c280c4e6bac6+        .quad   0x970ed3dd6d1d9b0b+        .quad   0xb09a9558450bf944+        .quad   0x48d0acfa57cde223+        .quad   0xb60f26e47dabe671+        .quad   0xf1d1a197622f3a37+        .quad   0x4208ce7ee9960394+        .quad   0x16234191336d3bdb++        // 2^180 * 1 * B++        .quad   0xf19aeac733a63aef+        .quad   0x2c7fba5d4442454e+        .quad   0x5da87aa04795e441+        .quad   0x413051e1a4e0b0f5+        .quad   0x852dd1fd3d578bbe+        .quad   0x2b65ce72c3286108+        .quad   0x658c07f4eace2273+        .quad   0x0933f804ec38ab40+        .quad   0xa7ab69798d496476+        .quad   0x8121aadefcb5abc8+        .quad   0xa5dc12ef7b539472+        .quad   0x07fd47065e45351a++        // 2^180 * 2 * B++        .quad   0xc8583c3d258d2bcd+        .quad   0x17029a4daf60b73f+        .quad   0xfa0fc9d6416a3781+        .quad   0x1c1e5fba38b3fb23+        .quad   0x304211559ae8e7c3+        .quad   0xf281b229944882a5+        .quad   0x8a13ac2e378250e4+        .quad   0x014afa0954ba48f4+        .quad   0xcb3197001bb3666c+        .quad   0x330060524bffecb9+        .quad   0x293711991a88233c+        .quad   0x291884363d4ed364++        // 2^180 * 3 * B++        .quad   0x033c6805dc4babfa+        .quad   0x2c15bf5e5596ecc1+        .quad   0x1bc70624b59b1d3b+        .quad   0x3ede9850a19f0ec5+        .quad   0xfb9d37c3bc1ab6eb+        .quad   0x02be14534d57a240+        .quad   0xf4d73415f8a5e1f6+        .quad   0x5964f4300ccc8188+        .quad   0xe44a23152d096800+        .quad   0x5c08c55970866996+        .quad   0xdf2db60a46affb6e+        .quad   0x579155c1f856fd89++        // 2^180 * 4 * B++        .quad   0x96324edd12e0c9ef+        .quad   0x468b878df2420297+        .quad   0x199a3776a4f573be+        .quad   0x1e7fbcf18e91e92a+        .quad   0xb5f16b630817e7a6+        .quad   0x808c69233c351026+        .quad   0x324a983b54cef201+        .quad   0x53c092084a485345+        .quad   0xd2d41481f1cbafbf+        .quad   0x231d2db6716174e5+        .quad   0x0b7d7656e2a55c98+        .quad   0x3e955cd82aa495f6++        // 2^180 * 5 * B++        .quad   0xe48f535e3ed15433+        .quad   0xd075692a0d7270a3+        .quad   0x40fbd21daade6387+        .quad   0x14264887cf4495f5+        .quad   0xab39f3ef61bb3a3f+        .quad   0x8eb400652eb9193e+        .quad   0xb5de6ecc38c11f74+        .quad   0x654d7e9626f3c49f+        .quad   0xe564cfdd5c7d2ceb+        .quad   0x82eeafded737ccb9+        .quad   0x6107db62d1f9b0ab+        .quad   0x0b6baac3b4358dbb++        // 2^180 * 6 * B++        .quad   0x7ae62bcb8622fe98+        .quad   0x47762256ceb891af+        .quad   0x1a5a92bcf2e406b4+        .quad   0x7d29401784e41501+        .quad   0x204abad63700a93b+        .quad   0xbe0023d3da779373+        .quad   0xd85f0346633ab709+        .quad   0x00496dc490820412+        .quad   0x1c74b88dc27e6360+        .quad   0x074854268d14850c+        .quad   0xa145fb7b3e0dcb30+        .quad   0x10843f1b43803b23++        // 2^180 * 7 * B++        .quad   0xc5f90455376276dd+        .quad   0xce59158dd7645cd9+        .quad   0x92f65d511d366b39+        .quad   0x11574b6e526996c4+        .quad   0xd56f672de324689b+        .quad   0xd1da8aedb394a981+        .quad   0xdd7b58fe9168cfed+        .quad   0x7ce246cd4d56c1e8+        .quad   0xb8f4308e7f80be53+        .quad   0x5f3cb8cb34a9d397+        .quad   0x18a961bd33cc2b2c+        .quad   0x710045fb3a9af671++        // 2^180 * 8 * B++        .quad   0x73f93d36101b95eb+        .quad   0xfaef33794f6f4486+        .quad   0x5651735f8f15e562+        .quad   0x7fa3f19058b40da1+        .quad   0xa03fc862059d699e+        .quad   0x2370cfa19a619e69+        .quad   0xc4fe3b122f823deb+        .quad   0x1d1b056fa7f0844e+        .quad   0x1bc64631e56bf61f+        .quad   0xd379ab106e5382a3+        .quad   0x4d58c57e0540168d+        .quad   0x566256628442d8e4++        // 2^184 * 1 * B++        .quad   0xb9e499def6267ff6+        .quad   0x7772ca7b742c0843+        .quad   0x23a0153fe9a4f2b1+        .quad   0x2cdfdfecd5d05006+        .quad   0xdd499cd61ff38640+        .quad   0x29cd9bc3063625a0+        .quad   0x51e2d8023dd73dc3+        .quad   0x4a25707a203b9231+        .quad   0x2ab7668a53f6ed6a+        .quad   0x304242581dd170a1+        .quad   0x4000144c3ae20161+        .quad   0x5721896d248e49fc++        // 2^184 * 2 * B++        .quad   0x0b6e5517fd181bae+        .quad   0x9022629f2bb963b4+        .quad   0x5509bce932064625+        .quad   0x578edd74f63c13da+        .quad   0x285d5091a1d0da4e+        .quad   0x4baa6fa7b5fe3e08+        .quad   0x63e5177ce19393b3+        .quad   0x03c935afc4b030fd+        .quad   0x997276c6492b0c3d+        .quad   0x47ccc2c4dfe205fc+        .quad   0xdcd29b84dd623a3c+        .quad   0x3ec2ab590288c7a2++        // 2^184 * 3 * B++        .quad   0xa1a0d27be4d87bb9+        .quad   0xa98b4deb61391aed+        .quad   0x99a0ddd073cb9b83+        .quad   0x2dd5c25a200fcace+        .quad   0xa7213a09ae32d1cb+        .quad   0x0f2b87df40f5c2d5+        .quad   0x0baea4c6e81eab29+        .quad   0x0e1bf66c6adbac5e+        .quad   0xe2abd5e9792c887e+        .quad   0x1a020018cb926d5d+        .quad   0xbfba69cdbaae5f1e+        .quad   0x730548b35ae88f5f++        // 2^184 * 4 * B++        .quad   0xc43551a3cba8b8ee+        .quad   0x65a26f1db2115f16+        .quad   0x760f4f52ab8c3850+        .quad   0x3043443b411db8ca+        .quad   0x805b094ba1d6e334+        .quad   0xbf3ef17709353f19+        .quad   0x423f06cb0622702b+        .quad   0x585a2277d87845dd+        .quad   0xa18a5f8233d48962+        .quad   0x6698c4b5ec78257f+        .quad   0xa78e6fa5373e41ff+        .quad   0x7656278950ef981f++        // 2^184 * 5 * B++        .quad   0x38c3cf59d51fc8c0+        .quad   0x9bedd2fd0506b6f2+        .quad   0x26bf109fab570e8f+        .quad   0x3f4160a8c1b846a6+        .quad   0xe17073a3ea86cf9d+        .quad   0x3a8cfbb707155fdc+        .quad   0x4853e7fc31838a8e+        .quad   0x28bbf484b613f616+        .quad   0xf2612f5c6f136c7c+        .quad   0xafead107f6dd11be+        .quad   0x527e9ad213de6f33+        .quad   0x1e79cb358188f75d++        // 2^184 * 6 * B++        .quad   0x013436c3eef7e3f1+        .quad   0x828b6a7ffe9e10f8+        .quad   0x7ff908e5bcf9defc+        .quad   0x65d7951b3a3b3831+        .quad   0x77e953d8f5e08181+        .quad   0x84a50c44299dded9+        .quad   0xdc6c2d0c864525e5+        .quad   0x478ab52d39d1f2f4+        .quad   0x66a6a4d39252d159+        .quad   0xe5dde1bc871ac807+        .quad   0xb82c6b40a6c1c96f+        .quad   0x16d87a411a212214++        // 2^184 * 7 * B++        .quad   0xb3bd7e5a42066215+        .quad   0x879be3cd0c5a24c1+        .quad   0x57c05db1d6f994b7+        .quad   0x28f87c8165f38ca6+        .quad   0xfba4d5e2d54e0583+        .quad   0xe21fafd72ebd99fa+        .quad   0x497ac2736ee9778f+        .quad   0x1f990b577a5a6dde+        .quad   0xa3344ead1be8f7d6+        .quad   0x7d1e50ebacea798f+        .quad   0x77c6569e520de052+        .quad   0x45882fe1534d6d3e++        // 2^184 * 8 * B++        .quad   0x6669345d757983d6+        .quad   0x62b6ed1117aa11a6+        .quad   0x7ddd1857985e128f+        .quad   0x688fe5b8f626f6dd+        .quad   0xd8ac9929943c6fe4+        .quad   0xb5f9f161a38392a2+        .quad   0x2699db13bec89af3+        .quad   0x7dcf843ce405f074+        .quad   0x6c90d6484a4732c0+        .quad   0xd52143fdca563299+        .quad   0xb3be28c3915dc6e1+        .quad   0x6739687e7327191b++        // 2^188 * 1 * B++        .quad   0x9f65c5ea200814cf+        .quad   0x840536e169a31740+        .quad   0x8b0ed13925c8b4ad+        .quad   0x0080dbafe936361d+        .quad   0x8ce5aad0c9cb971f+        .quad   0x1156aaa99fd54a29+        .quad   0x41f7247015af9b78+        .quad   0x1fe8cca8420f49aa+        .quad   0x72a1848f3c0cc82a+        .quad   0x38c560c2877c9e54+        .quad   0x5004e228ce554140+        .quad   0x042418a103429d71++        // 2^188 * 2 * B++        .quad   0x899dea51abf3ff5f+        .quad   0x9b93a8672fc2d8ba+        .quad   0x2c38cb97be6ebd5c+        .quad   0x114d578497263b5d+        .quad   0x58e84c6f20816247+        .quad   0x8db2b2b6e36fd793+        .quad   0x977182561d484d85+        .quad   0x0822024f8632abd7+        .quad   0xb301bb7c6b1beca3+        .quad   0x55393f6dc6eb1375+        .quad   0x910d281097b6e4eb+        .quad   0x1ad4548d9d479ea3++        // 2^188 * 3 * B++        .quad   0xcd5a7da0389a48fd+        .quad   0xb38fa4aa9a78371e+        .quad   0xc6d9761b2cdb8e6c+        .quad   0x35cf51dbc97e1443+        .quad   0xa06fe66d0fe9fed3+        .quad   0xa8733a401c587909+        .quad   0x30d14d800df98953+        .quad   0x41ce5876c7b30258+        .quad   0x59ac3bc5d670c022+        .quad   0xeae67c109b119406+        .quad   0x9798bdf0b3782fda+        .quad   0x651e3201fd074092++        // 2^188 * 4 * B++        .quad   0xd63d8483ef30c5cf+        .quad   0x4cd4b4962361cc0c+        .quad   0xee90e500a48426ac+        .quad   0x0af51d7d18c14eeb+        .quad   0xa57ba4a01efcae9e+        .quad   0x769f4beedc308a94+        .quad   0xd1f10eeb3603cb2e+        .quad   0x4099ce5e7e441278+        .quad   0x1ac98e4f8a5121e9+        .quad   0x7dae9544dbfa2fe0+        .quad   0x8320aa0dd6430df9+        .quad   0x667282652c4a2fb5++        // 2^188 * 5 * B++        .quad   0x874621f4d86bc9ab+        .quad   0xb54c7bbe56fe6fea+        .quad   0x077a24257fadc22c+        .quad   0x1ab53be419b90d39+        .quad   0xada8b6e02946db23+        .quad   0x1c0ce51a7b253ab7+        .quad   0x8448c85a66dd485b+        .quad   0x7f1fc025d0675adf+        .quad   0xd8ee1b18319ea6aa+        .quad   0x004d88083a21f0da+        .quad   0x3bd6aa1d883a4f4b+        .quad   0x4db9a3a6dfd9fd14++        // 2^188 * 6 * B++        .quad   0x8ce7b23bb99c0755+        .quad   0x35c5d6edc4f50f7a+        .quad   0x7e1e2ed2ed9b50c3+        .quad   0x36305f16e8934da1+        .quad   0xd95b00bbcbb77c68+        .quad   0xddbc846a91f17849+        .quad   0x7cf700aebe28d9b3+        .quad   0x5ce1285c85d31f3e+        .quad   0x31b6972d98b0bde8+        .quad   0x7d920706aca6de5b+        .quad   0xe67310f8908a659f+        .quad   0x50fac2a6efdf0235++        // 2^188 * 7 * B++        .quad   0xf3d3a9f35b880f5a+        .quad   0xedec050cdb03e7c2+        .quad   0xa896981ff9f0b1a2+        .quad   0x49a4ae2bac5e34a4+        .quad   0x295b1c86f6f449bc+        .quad   0x51b2e84a1f0ab4dd+        .quad   0xc001cb30aa8e551d+        .quad   0x6a28d35944f43662+        .quad   0x28bb12ee04a740e0+        .quad   0x14313bbd9bce8174+        .quad   0x72f5b5e4e8c10c40+        .quad   0x7cbfb19936adcd5b++        // 2^188 * 8 * B++        .quad   0xa311ddc26b89792d+        .quad   0x1b30b4c6da512664+        .quad   0x0ca77b4ccf150859+        .quad   0x1de443df1b009408+        .quad   0x8e793a7acc36e6e0+        .quad   0xf9fab7a37d586eed+        .quad   0x3a4f9692bae1f4e4+        .quad   0x1c14b03eff5f447e+        .quad   0x19647bd114a85291+        .quad   0x57b76cb21034d3af+        .quad   0x6329db440f9d6dfa+        .quad   0x5ef43e586a571493++        // 2^192 * 1 * B++        .quad   0xef782014385675a6+        .quad   0xa2649f30aafda9e8+        .quad   0x4cd1eb505cdfa8cb+        .quad   0x46115aba1d4dc0b3+        .quad   0xa66dcc9dc80c1ac0+        .quad   0x97a05cf41b38a436+        .quad   0xa7ebf3be95dbd7c6+        .quad   0x7da0b8f68d7e7dab+        .quad   0xd40f1953c3b5da76+        .quad   0x1dac6f7321119e9b+        .quad   0x03cc6021feb25960+        .quad   0x5a5f887e83674b4b++        // 2^192 * 2 * B++        .quad   0x8f6301cf70a13d11+        .quad   0xcfceb815350dd0c4+        .quad   0xf70297d4a4bca47e+        .quad   0x3669b656e44d1434+        .quad   0x9e9628d3a0a643b9+        .quad   0xb5c3cb00e6c32064+        .quad   0x9b5302897c2dec32+        .quad   0x43e37ae2d5d1c70c+        .quad   0x387e3f06eda6e133+        .quad   0x67301d5199a13ac0+        .quad   0xbd5ad8f836263811+        .quad   0x6a21e6cd4fd5e9be++        // 2^192 * 3 * B++        .quad   0xf1c6170a3046e65f+        .quad   0x58712a2a00d23524+        .quad   0x69dbbd3c8c82b755+        .quad   0x586bf9f1a195ff57+        .quad   0xef4129126699b2e3+        .quad   0x71d30847708d1301+        .quad   0x325432d01182b0bd+        .quad   0x45371b07001e8b36+        .quad   0xa6db088d5ef8790b+        .quad   0x5278f0dc610937e5+        .quad   0xac0349d261a16eb8+        .quad   0x0eafb03790e52179++        // 2^192 * 4 * B++        .quad   0x960555c13748042f+        .quad   0x219a41e6820baa11+        .quad   0x1c81f73873486d0c+        .quad   0x309acc675a02c661+        .quad   0x5140805e0f75ae1d+        .quad   0xec02fbe32662cc30+        .quad   0x2cebdf1eea92396d+        .quad   0x44ae3344c5435bb3+        .quad   0x9cf289b9bba543ee+        .quad   0xf3760e9d5ac97142+        .quad   0x1d82e5c64f9360aa+        .quad   0x62d5221b7f94678f++        // 2^192 * 5 * B++        .quad   0x524c299c18d0936d+        .quad   0xc86bb56c8a0c1a0c+        .quad   0xa375052edb4a8631+        .quad   0x5c0efde4bc754562+        .quad   0x7585d4263af77a3c+        .quad   0xdfae7b11fee9144d+        .quad   0xa506708059f7193d+        .quad   0x14f29a5383922037+        .quad   0xdf717edc25b2d7f5+        .quad   0x21f970db99b53040+        .quad   0xda9234b7c3ed4c62+        .quad   0x5e72365c7bee093e++        // 2^192 * 6 * B++        .quad   0x575bfc074571217f+        .quad   0x3779675d0694d95b+        .quad   0x9a0a37bbf4191e33+        .quad   0x77f1104c47b4eabc+        .quad   0x7d9339062f08b33e+        .quad   0x5b9659e5df9f32be+        .quad   0xacff3dad1f9ebdfd+        .quad   0x70b20555cb7349b7+        .quad   0xbe5113c555112c4c+        .quad   0x6688423a9a881fcd+        .quad   0x446677855e503b47+        .quad   0x0e34398f4a06404a++        // 2^192 * 7 * B++        .quad   0xb67d22d93ecebde8+        .quad   0x09b3e84127822f07+        .quad   0x743fa61fb05b6d8d+        .quad   0x5e5405368a362372+        .quad   0x18930b093e4b1928+        .quad   0x7de3e10e73f3f640+        .quad   0xf43217da73395d6f+        .quad   0x6f8aded6ca379c3e+        .quad   0xe340123dfdb7b29a+        .quad   0x487b97e1a21ab291+        .quad   0xf9967d02fde6949e+        .quad   0x780de72ec8d3de97++        // 2^192 * 8 * B++        .quad   0x0ae28545089ae7bc+        .quad   0x388ddecf1c7f4d06+        .quad   0x38ac15510a4811b8+        .quad   0x0eb28bf671928ce4+        .quad   0x671feaf300f42772+        .quad   0x8f72eb2a2a8c41aa+        .quad   0x29a17fd797373292+        .quad   0x1defc6ad32b587a6+        .quad   0xaf5bbe1aef5195a7+        .quad   0x148c1277917b15ed+        .quad   0x2991f7fb7ae5da2e+        .quad   0x467d201bf8dd2867++        // 2^196 * 1 * B++        .quad   0x7906ee72f7bd2e6b+        .quad   0x05d270d6109abf4e+        .quad   0x8d5cfe45b941a8a4+        .quad   0x44c218671c974287+        .quad   0x745f9d56296bc318+        .quad   0x993580d4d8152e65+        .quad   0xb0e5b13f5839e9ce+        .quad   0x51fc2b28d43921c0+        .quad   0x1b8fd11795e2a98c+        .quad   0x1c4e5ee12b6b6291+        .quad   0x5b30e7107424b572+        .quad   0x6e6b9de84c4f4ac6++        // 2^196 * 2 * B++        .quad   0xdff25fce4b1de151+        .quad   0xd841c0c7e11c4025+        .quad   0x2554b3c854749c87+        .quad   0x2d292459908e0df9+        .quad   0x6b7c5f10f80cb088+        .quad   0x736b54dc56e42151+        .quad   0xc2b620a5c6ef99c4+        .quad   0x5f4c802cc3a06f42+        .quad   0x9b65c8f17d0752da+        .quad   0x881ce338c77ee800+        .quad   0xc3b514f05b62f9e3+        .quad   0x66ed5dd5bec10d48++        // 2^196 * 3 * B++        .quad   0x7d38a1c20bb2089d+        .quad   0x808334e196ccd412+        .quad   0xc4a70b8c6c97d313+        .quad   0x2eacf8bc03007f20+        .quad   0xf0adf3c9cbca047d+        .quad   0x81c3b2cbf4552f6b+        .quad   0xcfda112d44735f93+        .quad   0x1f23a0c77e20048c+        .quad   0xf235467be5bc1570+        .quad   0x03d2d9020dbab38c+        .quad   0x27529aa2fcf9e09e+        .quad   0x0840bef29d34bc50++        // 2^196 * 4 * B++        .quad   0x796dfb35dc10b287+        .quad   0x27176bcd5c7ff29d+        .quad   0x7f3d43e8c7b24905+        .quad   0x0304f5a191c54276+        .quad   0xcd54e06b7f37e4eb+        .quad   0x8cc15f87f5e96cca+        .quad   0xb8248bb0d3597dce+        .quad   0x246affa06074400c+        .quad   0x37d88e68fbe45321+        .quad   0x86097548c0d75032+        .quad   0x4e9b13ef894a0d35+        .quad   0x25a83cac5753d325++        // 2^196 * 5 * B++        .quad   0x10222f48eed8165e+        .quad   0x623fc1234b8bcf3a+        .quad   0x1e145c09c221e8f0+        .quad   0x7ccfa59fca782630+        .quad   0x9f0f66293952b6e2+        .quad   0x33db5e0e0934267b+        .quad   0xff45252bd609fedc+        .quad   0x06be10f5c506e0c9+        .quad   0x1a9615a9b62a345f+        .quad   0x22050c564a52fecc+        .quad   0xa7a2788528bc0dfe+        .quad   0x5e82770a1a1ee71d++        // 2^196 * 6 * B++        .quad   0x35425183ad896a5c+        .quad   0xe8673afbe78d52f6+        .quad   0x2c66f25f92a35f64+        .quad   0x09d04f3b3b86b102+        .quad   0xe802e80a42339c74+        .quad   0x34175166a7fffae5+        .quad   0x34865d1f1c408cae+        .quad   0x2cca982c605bc5ee+        .quad   0xfd2d5d35197dbe6e+        .quad   0x207c2eea8be4ffa3+        .quad   0x2613d8db325ae918+        .quad   0x7a325d1727741d3e++        // 2^196 * 7 * B++        .quad   0xd036b9bbd16dfde2+        .quad   0xa2055757c497a829+        .quad   0x8e6cc966a7f12667+        .quad   0x4d3b1a791239c180+        .quad   0xecd27d017e2a076a+        .quad   0xd788689f1636495e+        .quad   0x52a61af0919233e5+        .quad   0x2a479df17bb1ae64+        .quad   0x9e5eee8e33db2710+        .quad   0x189854ded6c43ca5+        .quad   0xa41c22c592718138+        .quad   0x27ad5538a43a5e9b++        // 2^196 * 8 * B++        .quad   0x2746dd4b15350d61+        .quad   0xd03fcbc8ee9521b7+        .quad   0xe86e365a138672ca+        .quad   0x510e987f7e7d89e2+        .quad   0xcb5a7d638e47077c+        .quad   0x8db7536120a1c059+        .quad   0x549e1e4d8bedfdcc+        .quad   0x080153b7503b179d+        .quad   0xdda69d930a3ed3e3+        .quad   0x3d386ef1cd60a722+        .quad   0xc817ad58bdaa4ee6+        .quad   0x23be8d554fe7372a++        // 2^200 * 1 * B++        .quad   0x95fe919a74ef4fad+        .quad   0x3a827becf6a308a2+        .quad   0x964e01d309a47b01+        .quad   0x71c43c4f5ba3c797+        .quad   0xbc1ef4bd567ae7a9+        .quad   0x3f624cb2d64498bd+        .quad   0xe41064d22c1f4ec8+        .quad   0x2ef9c5a5ba384001+        .quad   0xb6fd6df6fa9e74cd+        .quad   0xf18278bce4af267a+        .quad   0x8255b3d0f1ef990e+        .quad   0x5a758ca390c5f293++        // 2^200 * 2 * B++        .quad   0xa2b72710d9462495+        .quad   0x3aa8c6d2d57d5003+        .quad   0xe3d400bfa0b487ca+        .quad   0x2dbae244b3eb72ec+        .quad   0x8ce0918b1d61dc94+        .quad   0x8ded36469a813066+        .quad   0xd4e6a829afe8aad3+        .quad   0x0a738027f639d43f+        .quad   0x980f4a2f57ffe1cc+        .quad   0x00670d0de1839843+        .quad   0x105c3f4a49fb15fd+        .quad   0x2698ca635126a69c++        // 2^200 * 3 * B++        .quad   0xe765318832b0ba78+        .quad   0x381831f7925cff8b+        .quad   0x08a81b91a0291fcc+        .quad   0x1fb43dcc49caeb07+        .quad   0x2e3d702f5e3dd90e+        .quad   0x9e3f0918e4d25386+        .quad   0x5e773ef6024da96a+        .quad   0x3c004b0c4afa3332+        .quad   0x9aa946ac06f4b82b+        .quad   0x1ca284a5a806c4f3+        .quad   0x3ed3265fc6cd4787+        .quad   0x6b43fd01cd1fd217++        // 2^200 * 4 * B++        .quad   0xc7a75d4b4697c544+        .quad   0x15fdf848df0fffbf+        .quad   0x2868b9ebaa46785a+        .quad   0x5a68d7105b52f714+        .quad   0xb5c742583e760ef3+        .quad   0x75dc52b9ee0ab990+        .quad   0xbf1427c2072b923f+        .quad   0x73420b2d6ff0d9f0+        .quad   0xaf2cf6cb9e851e06+        .quad   0x8f593913c62238c4+        .quad   0xda8ab89699fbf373+        .quad   0x3db5632fea34bc9e++        // 2^200 * 5 * B++        .quad   0xf46eee2bf75dd9d8+        .quad   0x0d17b1f6396759a5+        .quad   0x1bf2d131499e7273+        .quad   0x04321adf49d75f13+        .quad   0x2e4990b1829825d5+        .quad   0xedeaeb873e9a8991+        .quad   0xeef03d394c704af8+        .quad   0x59197ea495df2b0e+        .quad   0x04e16019e4e55aae+        .quad   0xe77b437a7e2f92e9+        .quad   0xc7ce2dc16f159aa4+        .quad   0x45eafdc1f4d70cc0++        // 2^200 * 6 * B++        .quad   0x698401858045d72b+        .quad   0x4c22faa2cf2f0651+        .quad   0x941a36656b222dc6+        .quad   0x5a5eebc80362dade+        .quad   0xb60e4624cfccb1ed+        .quad   0x59dbc292bd5c0395+        .quad   0x31a09d1ddc0481c9+        .quad   0x3f73ceea5d56d940+        .quad   0xb7a7bfd10a4e8dc6+        .quad   0xbe57007e44c9b339+        .quad   0x60c1207f1557aefa+        .quad   0x26058891266218db++        // 2^200 * 7 * B++        .quad   0x59f704a68360ff04+        .quad   0xc3d93fde7661e6f4+        .quad   0x831b2a7312873551+        .quad   0x54ad0c2e4e615d57+        .quad   0x4c818e3cc676e542+        .quad   0x5e422c9303ceccad+        .quad   0xec07cccab4129f08+        .quad   0x0dedfa10b24443b8+        .quad   0xee3b67d5b82b522a+        .quad   0x36f163469fa5c1eb+        .quad   0xa5b4d2f26ec19fd3+        .quad   0x62ecb2baa77a9408++        // 2^200 * 8 * B++        .quad   0xe5ed795261152b3d+        .quad   0x4962357d0eddd7d1+        .quad   0x7482c8d0b96b4c71+        .quad   0x2e59f919a966d8be+        .quad   0x92072836afb62874+        .quad   0x5fcd5e8579e104a5+        .quad   0x5aad01adc630a14a+        .quad   0x61913d5075663f98+        .quad   0x0dc62d361a3231da+        .quad   0xfa47583294200270+        .quad   0x02d801513f9594ce+        .quad   0x3ddbc2a131c05d5c++        // 2^204 * 1 * B++        .quad   0x3f50a50a4ffb81ef+        .quad   0xb1e035093bf420bf+        .quad   0x9baa8e1cc6aa2cd0+        .quad   0x32239861fa237a40+        .quad   0xfb735ac2004a35d1+        .quad   0x31de0f433a6607c3+        .quad   0x7b8591bfc528d599+        .quad   0x55be9a25f5bb050c+        .quad   0x0d005acd33db3dbf+        .quad   0x0111b37c80ac35e2+        .quad   0x4892d66c6f88ebeb+        .quad   0x770eadb16508fbcd++        // 2^204 * 2 * B++        .quad   0x8451f9e05e4e89dd+        .quad   0xc06302ffbc793937+        .quad   0x5d22749556a6495c+        .quad   0x09a6755ca05603fb+        .quad   0xf1d3b681a05071b9+        .quad   0x2207659a3592ff3a+        .quad   0x5f0169297881e40e+        .quad   0x16bedd0e86ba374e+        .quad   0x5ecccc4f2c2737b5+        .quad   0x43b79e0c2dccb703+        .quad   0x33e008bc4ec43df3+        .quad   0x06c1b840f07566c0++        // 2^204 * 3 * B++        .quad   0x7688a5c6a388f877+        .quad   0x02a96c14deb2b6ac+        .quad   0x64c9f3431b8c2af8+        .quad   0x3628435554a1eed6+        .quad   0x69ee9e7f9b02805c+        .quad   0xcbff828a547d1640+        .quad   0x3d93a869b2430968+        .quad   0x46b7b8cd3fe26972+        .quad   0xe9812086fe7eebe0+        .quad   0x4cba6be72f515437+        .quad   0x1d04168b516efae9+        .quad   0x5ea1391043982cb9++        // 2^204 * 4 * B++        .quad   0x49125c9cf4702ee1+        .quad   0x4520b71f8b25b32d+        .quad   0x33193026501fef7e+        .quad   0x656d8997c8d2eb2b+        .quad   0x6f2b3be4d5d3b002+        .quad   0xafec33d96a09c880+        .quad   0x035f73a4a8bcc4cc+        .quad   0x22c5b9284662198b+        .quad   0xcb58c8fe433d8939+        .quad   0x89a0cb2e6a8d7e50+        .quad   0x79ca955309fbbe5a+        .quad   0x0c626616cd7fc106++        // 2^204 * 5 * B++        .quad   0x1ffeb80a4879b61f+        .quad   0x6396726e4ada21ed+        .quad   0x33c7b093368025ba+        .quad   0x471aa0c6f3c31788+        .quad   0x8fdfc379fbf454b1+        .quad   0x45a5a970f1a4b771+        .quad   0xac921ef7bad35915+        .quad   0x42d088dca81c2192+        .quad   0x8fda0f37a0165199+        .quad   0x0adadb77c8a0e343+        .quad   0x20fbfdfcc875e820+        .quad   0x1cf2bea80c2206e7++        // 2^204 * 6 * B++        .quad   0xc2ddf1deb36202ac+        .quad   0x92a5fe09d2e27aa5+        .quad   0x7d1648f6fc09f1d3+        .quad   0x74c2cc0513bc4959+        .quad   0x982d6e1a02c0412f+        .quad   0x90fa4c83db58e8fe+        .quad   0x01c2f5bcdcb18bc0+        .quad   0x686e0c90216abc66+        .quad   0x1fadbadba54395a7+        .quad   0xb41a02a0ae0da66a+        .quad   0xbf19f598bba37c07+        .quad   0x6a12b8acde48430d++        // 2^204 * 7 * B++        .quad   0xf8daea1f39d495d9+        .quad   0x592c190e525f1dfc+        .quad   0xdb8cbd04c9991d1b+        .quad   0x11f7fda3d88f0cb7+        .quad   0x793bdd801aaeeb5f+        .quad   0x00a2a0aac1518871+        .quad   0xe8a373a31f2136b4+        .quad   0x48aab888fc91ef19+        .quad   0x041f7e925830f40e+        .quad   0x002d6ca979661c06+        .quad   0x86dc9ff92b046a2e+        .quad   0x760360928b0493d1++        // 2^204 * 8 * B++        .quad   0x21bb41c6120cf9c6+        .quad   0xeab2aa12decda59b+        .quad   0xc1a72d020aa48b34+        .quad   0x215d4d27e87d3b68+        .quad   0xb43108e5695a0b05+        .quad   0x6cb00ee8ad37a38b+        .quad   0x5edad6eea3537381+        .quad   0x3f2602d4b6dc3224+        .quad   0xc8b247b65bcaf19c+        .quad   0x49779dc3b1b2c652+        .quad   0x89a180bbd5ece2e2+        .quad   0x13f098a3cec8e039++        // 2^208 * 1 * B++        .quad   0x9adc0ff9ce5ec54b+        .quad   0x039c2a6b8c2f130d+        .quad   0x028007c7f0f89515+        .quad   0x78968314ac04b36b+        .quad   0xf3aa57a22796bb14+        .quad   0x883abab79b07da21+        .quad   0xe54be21831a0391c+        .quad   0x5ee7fb38d83205f9+        .quad   0x538dfdcb41446a8e+        .quad   0xa5acfda9434937f9+        .quad   0x46af908d263c8c78+        .quad   0x61d0633c9bca0d09++        // 2^208 * 2 * B++        .quad   0x63744935ffdb2566+        .quad   0xc5bd6b89780b68bb+        .quad   0x6f1b3280553eec03+        .quad   0x6e965fd847aed7f5+        .quad   0xada328bcf8fc73df+        .quad   0xee84695da6f037fc+        .quad   0x637fb4db38c2a909+        .quad   0x5b23ac2df8067bdc+        .quad   0x9ad2b953ee80527b+        .quad   0xe88f19aafade6d8d+        .quad   0x0e711704150e82cf+        .quad   0x79b9bbb9dd95dedc++        // 2^208 * 3 * B++        .quad   0xebb355406a3126c2+        .quad   0xd26383a868c8c393+        .quad   0x6c0c6429e5b97a82+        .quad   0x5065f158c9fd2147+        .quad   0xd1997dae8e9f7374+        .quad   0xa032a2f8cfbb0816+        .quad   0xcd6cba126d445f0a+        .quad   0x1ba811460accb834+        .quad   0x708169fb0c429954+        .quad   0xe14600acd76ecf67+        .quad   0x2eaab98a70e645ba+        .quad   0x3981f39e58a4faf2++        // 2^208 * 4 * B++        .quad   0x18fb8a7559230a93+        .quad   0x1d168f6960e6f45d+        .quad   0x3a85a94514a93cb5+        .quad   0x38dc083705acd0fd+        .quad   0xc845dfa56de66fde+        .quad   0xe152a5002c40483a+        .quad   0xe9d2e163c7b4f632+        .quad   0x30f4452edcbc1b65+        .quad   0x856d2782c5759740+        .quad   0xfa134569f99cbecc+        .quad   0x8844fc73c0ea4e71+        .quad   0x632d9a1a593f2469++        // 2^208 * 5 * B++        .quad   0xf6bb6b15b807cba6+        .quad   0x1823c7dfbc54f0d7+        .quad   0xbb1d97036e29670b+        .quad   0x0b24f48847ed4a57+        .quad   0xbf09fd11ed0c84a7+        .quad   0x63f071810d9f693a+        .quad   0x21908c2d57cf8779+        .quad   0x3a5a7df28af64ba2+        .quad   0xdcdad4be511beac7+        .quad   0xa4538075ed26ccf2+        .quad   0xe19cff9f005f9a65+        .quad   0x34fcf74475481f63++        // 2^208 * 6 * B++        .quad   0xc197e04c789767ca+        .quad   0xb8714dcb38d9467d+        .quad   0x55de888283f95fa8+        .quad   0x3d3bdc164dfa63f7+        .quad   0xa5bb1dab78cfaa98+        .quad   0x5ceda267190b72f2+        .quad   0x9309c9110a92608e+        .quad   0x0119a3042fb374b0+        .quad   0x67a2d89ce8c2177d+        .quad   0x669da5f66895d0c1+        .quad   0xf56598e5b282a2b0+        .quad   0x56c088f1ede20a73++        // 2^208 * 7 * B++        .quad   0x336d3d1110a86e17+        .quad   0xd7f388320b75b2fa+        .quad   0xf915337625072988+        .quad   0x09674c6b99108b87+        .quad   0x581b5fac24f38f02+        .quad   0xa90be9febae30cbd+        .quad   0x9a2169028acf92f0+        .quad   0x038b7ea48359038f+        .quad   0x9f4ef82199316ff8+        .quad   0x2f49d282eaa78d4f+        .quad   0x0971a5ab5aef3174+        .quad   0x6e5e31025969eb65++        // 2^208 * 8 * B++        .quad   0xb16c62f587e593fb+        .quad   0x4999eddeca5d3e71+        .quad   0xb491c1e014cc3e6d+        .quad   0x08f5114789a8dba8+        .quad   0x3304fb0e63066222+        .quad   0xfb35068987acba3f+        .quad   0xbd1924778c1061a3+        .quad   0x3058ad43d1838620+        .quad   0x323c0ffde57663d0+        .quad   0x05c3df38a22ea610+        .quad   0xbdc78abdac994f9a+        .quad   0x26549fa4efe3dc99++        // 2^212 * 1 * B++        .quad   0x738b38d787ce8f89+        .quad   0xb62658e24179a88d+        .quad   0x30738c9cf151316d+        .quad   0x49128c7f727275c9+        .quad   0x04dbbc17f75396b9+        .quad   0x69e6a2d7d2f86746+        .quad   0xc6409d99f53eabc6+        .quad   0x606175f6332e25d2+        .quad   0x4021370ef540e7dd+        .quad   0x0910d6f5a1f1d0a5+        .quad   0x4634aacd5b06b807+        .quad   0x6a39e6356944f235++        // 2^212 * 2 * B++        .quad   0x96cd5640df90f3e7+        .quad   0x6c3a760edbfa25ea+        .quad   0x24f3ef0959e33cc4+        .quad   0x42889e7e530d2e58+        .quad   0x1da1965774049e9d+        .quad   0xfbcd6ea198fe352b+        .quad   0xb1cbcd50cc5236a6+        .quad   0x1f5ec83d3f9846e2+        .quad   0x8efb23c3328ccb75+        .quad   0xaf42a207dd876ee9+        .quad   0x20fbdadc5dfae796+        .quad   0x241e246b06bf9f51++        // 2^212 * 3 * B++        .quad   0x29e68e57ad6e98f6+        .quad   0x4c9260c80b462065+        .quad   0x3f00862ea51ebb4b+        .quad   0x5bc2c77fb38d9097+        .quad   0x7eaafc9a6280bbb8+        .quad   0x22a70f12f403d809+        .quad   0x31ce40bb1bfc8d20+        .quad   0x2bc65635e8bd53ee+        .quad   0xe8d5dc9fa96bad93+        .quad   0xe58fb17dde1947dc+        .quad   0x681532ea65185fa3+        .quad   0x1fdd6c3b034a7830++        // 2^212 * 4 * B++        .quad   0x0a64e28c55dc18fe+        .quad   0xe3df9e993399ebdd+        .quad   0x79ac432370e2e652+        .quad   0x35ff7fc33ae4cc0e+        .quad   0x9c13a6a52dd8f7a9+        .quad   0x2dbb1f8c3efdcabf+        .quad   0x961e32405e08f7b5+        .quad   0x48c8a121bbe6c9e5+        .quad   0xfc415a7c59646445+        .quad   0xd224b2d7c128b615+        .quad   0x6035c9c905fbb912+        .quad   0x42d7a91274429fab++        // 2^212 * 5 * B++        .quad   0x4e6213e3eaf72ed3+        .quad   0x6794981a43acd4e7+        .quad   0xff547cde6eb508cb+        .quad   0x6fed19dd10fcb532+        .quad   0xa9a48947933da5bc+        .quad   0x4a58920ec2e979ec+        .quad   0x96d8800013e5ac4c+        .quad   0x453692d74b48b147+        .quad   0xdd775d99a8559c6f+        .quad   0xf42a2140df003e24+        .quad   0x5223e229da928a66+        .quad   0x063f46ba6d38f22c++        // 2^212 * 6 * B++        .quad   0xd2d242895f536694+        .quad   0xca33a2c542939b2c+        .quad   0x986fada6c7ddb95c+        .quad   0x5a152c042f712d5d+        .quad   0x39843cb737346921+        .quad   0xa747fb0738c89447+        .quad   0xcb8d8031a245307e+        .quad   0x67810f8e6d82f068+        .quad   0x3eeb8fbcd2287db4+        .quad   0x72c7d3a301a03e93+        .quad   0x5473e88cbd98265a+        .quad   0x7324aa515921b403++        // 2^212 * 7 * B++        .quad   0x857942f46c3cbe8e+        .quad   0xa1d364b14730c046+        .quad   0x1c8ed914d23c41bf+        .quad   0x0838e161eef6d5d2+        .quad   0xad23f6dae82354cb+        .quad   0x6962502ab6571a6d+        .quad   0x9b651636e38e37d1+        .quad   0x5cac5005d1a3312f+        .quad   0x8cc154cce9e39904+        .quad   0x5b3a040b84de6846+        .quad   0xc4d8a61cb1be5d6e+        .quad   0x40fb897bd8861f02++        // 2^212 * 8 * B++        .quad   0x84c5aa9062de37a1+        .quad   0x421da5000d1d96e1+        .quad   0x788286306a9242d9+        .quad   0x3c5e464a690d10da+        .quad   0xe57ed8475ab10761+        .quad   0x71435e206fd13746+        .quad   0x342f824ecd025632+        .quad   0x4b16281ea8791e7b+        .quad   0xd1c101d50b813381+        .quad   0xdee60f1176ee6828+        .quad   0x0cb68893383f6409+        .quad   0x6183c565f6ff484a++        // 2^216 * 1 * B++        .quad   0x741d5a461e6bf9d6+        .quad   0x2305b3fc7777a581+        .quad   0xd45574a26474d3d9+        .quad   0x1926e1dc6401e0ff+        .quad   0xdb468549af3f666e+        .quad   0xd77fcf04f14a0ea5+        .quad   0x3df23ff7a4ba0c47+        .quad   0x3a10dfe132ce3c85+        .quad   0xe07f4e8aea17cea0+        .quad   0x2fd515463a1fc1fd+        .quad   0x175322fd31f2c0f1+        .quad   0x1fa1d01d861e5d15++        // 2^216 * 2 * B++        .quad   0xcc8055947d599832+        .quad   0x1e4656da37f15520+        .quad   0x99f6f7744e059320+        .quad   0x773563bc6a75cf33+        .quad   0x38dcac00d1df94ab+        .quad   0x2e712bddd1080de9+        .quad   0x7f13e93efdd5e262+        .quad   0x73fced18ee9a01e5+        .quad   0x06b1e90863139cb3+        .quad   0xa493da67c5a03ecd+        .quad   0x8d77cec8ad638932+        .quad   0x1f426b701b864f44++        // 2^216 * 3 * B++        .quad   0xefc9264c41911c01+        .quad   0xf1a3b7b817a22c25+        .quad   0x5875da6bf30f1447+        .quad   0x4e1af5271d31b090+        .quad   0xf17e35c891a12552+        .quad   0xb76b8153575e9c76+        .quad   0xfa83406f0d9b723e+        .quad   0x0b76bb1b3fa7e438+        .quad   0x08b8c1f97f92939b+        .quad   0xbe6771cbd444ab6e+        .quad   0x22e5646399bb8017+        .quad   0x7b6dd61eb772a955++        // 2^216 * 4 * B++        .quad   0xb7adc1e850f33d92+        .quad   0x7998fa4f608cd5cf+        .quad   0xad962dbd8dfc5bdb+        .quad   0x703e9bceaf1d2f4f+        .quad   0x5730abf9ab01d2c7+        .quad   0x16fb76dc40143b18+        .quad   0x866cbe65a0cbb281+        .quad   0x53fa9b659bff6afe+        .quad   0x6c14c8e994885455+        .quad   0x843a5d6665aed4e5+        .quad   0x181bb73ebcd65af1+        .quad   0x398d93e5c4c61f50++        // 2^216 * 5 * B++        .quad   0x1c4bd16733e248f3+        .quad   0xbd9e128715bf0a5f+        .quad   0xd43f8cf0a10b0376+        .quad   0x53b09b5ddf191b13+        .quad   0xc3877c60d2e7e3f2+        .quad   0x3b34aaa030828bb1+        .quad   0x283e26e7739ef138+        .quad   0x699c9c9002c30577+        .quad   0xf306a7235946f1cc+        .quad   0x921718b5cce5d97d+        .quad   0x28cdd24781b4e975+        .quad   0x51caf30c6fcdd907++        // 2^216 * 6 * B++        .quad   0xa60ba7427674e00a+        .quad   0x630e8570a17a7bf3+        .quad   0x3758563dcf3324cc+        .quad   0x5504aa292383fdaa+        .quad   0x737af99a18ac54c7+        .quad   0x903378dcc51cb30f+        .quad   0x2b89bc334ce10cc7+        .quad   0x12ae29c189f8e99a+        .quad   0xa99ec0cb1f0d01cf+        .quad   0x0dd1efcc3a34f7ae+        .quad   0x55ca7521d09c4e22+        .quad   0x5fd14fe958eba5ea++        // 2^216 * 7 * B++        .quad   0xb5dc2ddf2845ab2c+        .quad   0x069491b10a7fe993+        .quad   0x4daaf3d64002e346+        .quad   0x093ff26e586474d1+        .quad   0x3c42fe5ebf93cb8e+        .quad   0xbedfa85136d4565f+        .quad   0xe0f0859e884220e8+        .quad   0x7dd73f960725d128+        .quad   0xb10d24fe68059829+        .quad   0x75730672dbaf23e5+        .quad   0x1367253ab457ac29+        .quad   0x2f59bcbc86b470a4++        // 2^216 * 8 * B++        .quad   0x83847d429917135f+        .quad   0xad1b911f567d03d7+        .quad   0x7e7748d9be77aad1+        .quad   0x5458b42e2e51af4a+        .quad   0x7041d560b691c301+        .quad   0x85201b3fadd7e71e+        .quad   0x16c2e16311335585+        .quad   0x2aa55e3d010828b1+        .quad   0xed5192e60c07444f+        .quad   0x42c54e2d74421d10+        .quad   0x352b4c82fdb5c864+        .quad   0x13e9004a8a768664++        // 2^220 * 1 * B++        .quad   0xcbb5b5556c032bff+        .quad   0xdf7191b729297a3a+        .quad   0xc1ff7326aded81bb+        .quad   0x71ade8bb68be03f5+        .quad   0x1e6284c5806b467c+        .quad   0xc5f6997be75d607b+        .quad   0x8b67d958b378d262+        .quad   0x3d88d66a81cd8b70+        .quad   0x8b767a93204ed789+        .quad   0x762fcacb9fa0ae2a+        .quad   0x771febcc6dce4887+        .quad   0x343062158ff05fb3++        // 2^220 * 2 * B++        .quad   0xe05da1a7e1f5bf49+        .quad   0x26457d6dd4736092+        .quad   0x77dcb07773cc32f6+        .quad   0x0a5d94969cdd5fcd+        .quad   0xfce219072a7b31b4+        .quad   0x4d7adc75aa578016+        .quad   0x0ec276a687479324+        .quad   0x6d6d9d5d1fda4beb+        .quad   0x22b1a58ae9b08183+        .quad   0xfd95d071c15c388b+        .quad   0xa9812376850a0517+        .quad   0x33384cbabb7f335e++        // 2^220 * 3 * B++        .quad   0x3c6fa2680ca2c7b5+        .quad   0x1b5082046fb64fda+        .quad   0xeb53349c5431d6de+        .quad   0x5278b38f6b879c89+        .quad   0x33bc627a26218b8d+        .quad   0xea80b21fc7a80c61+        .quad   0x9458b12b173e9ee6+        .quad   0x076247be0e2f3059+        .quad   0x52e105f61416375a+        .quad   0xec97af3685abeba4+        .quad   0x26e6b50623a67c36+        .quad   0x5cf0e856f3d4fb01++        // 2^220 * 4 * B++        .quad   0xf6c968731ae8cab4+        .quad   0x5e20741ecb4f92c5+        .quad   0x2da53be58ccdbc3e+        .quad   0x2dddfea269970df7+        .quad   0xbeaece313db342a8+        .quad   0xcba3635b842db7ee+        .quad   0xe88c6620817f13ef+        .quad   0x1b9438aa4e76d5c6+        .quad   0x8a50777e166f031a+        .quad   0x067b39f10fb7a328+        .quad   0x1925c9a6010fbd76+        .quad   0x6df9b575cc740905++        // 2^220 * 5 * B++        .quad   0x42c1192927f6bdcf+        .quad   0x8f91917a403d61ca+        .quad   0xdc1c5a668b9e1f61+        .quad   0x1596047804ec0f8d+        .quad   0xecdfc35b48cade41+        .quad   0x6a88471fb2328270+        .quad   0x740a4a2440a01b6a+        .quad   0x471e5796003b5f29+        .quad   0xda96bbb3aced37ac+        .quad   0x7a2423b5e9208cea+        .quad   0x24cc5c3038aebae2+        .quad   0x50c356afdc5dae2f++        // 2^220 * 6 * B++        .quad   0x09dcbf4341c30318+        .quad   0xeeba061183181dce+        .quad   0xc179c0cedc1e29a1+        .quad   0x1dbf7b89073f35b0+        .quad   0xcfed9cdf1b31b964+        .quad   0xf486a9858ca51af3+        .quad   0x14897265ea8c1f84+        .quad   0x784a53dd932acc00+        .quad   0x2d99f9df14fc4920+        .quad   0x76ccb60cc4499fe5+        .quad   0xa4132cbbe5cf0003+        .quad   0x3f93d82354f000ea++        // 2^220 * 7 * B++        .quad   0x8183e7689e04ce85+        .quad   0x678fb71e04465341+        .quad   0xad92058f6688edac+        .quad   0x5da350d3532b099a+        .quad   0xeaac12d179e14978+        .quad   0xff923ff3bbebff5e+        .quad   0x4af663e40663ce27+        .quad   0x0fd381a811a5f5ff+        .quad   0xf256aceca436df54+        .quad   0x108b6168ae69d6e8+        .quad   0x20d986cb6b5d036c+        .quad   0x655957b9fee2af50++        // 2^220 * 8 * B++        .quad   0xaea8b07fa902030f+        .quad   0xf88c766af463d143+        .quad   0x15b083663c787a60+        .quad   0x08eab1148267a4a8+        .quad   0xbdc1409bd002d0ac+        .quad   0x66660245b5ccd9a6+        .quad   0x82317dc4fade85ec+        .quad   0x02fe934b6ad7df0d+        .quad   0xef5cf100cfb7ea74+        .quad   0x22897633a1cb42ac+        .quad   0xd4ce0c54cef285e2+        .quad   0x30408c048a146a55++        // 2^224 * 1 * B++        .quad   0x739d8845832fcedb+        .quad   0xfa38d6c9ae6bf863+        .quad   0x32bc0dcab74ffef7+        .quad   0x73937e8814bce45e+        .quad   0xbb2e00c9193b877f+        .quad   0xece3a890e0dc506b+        .quad   0xecf3b7c036de649f+        .quad   0x5f46040898de9e1a+        .quad   0xb9037116297bf48d+        .quad   0xa9d13b22d4f06834+        .quad   0xe19715574696bdc6+        .quad   0x2cf8a4e891d5e835++        // 2^224 * 2 * B++        .quad   0x6d93fd8707110f67+        .quad   0xdd4c09d37c38b549+        .quad   0x7cb16a4cc2736a86+        .quad   0x2049bd6e58252a09+        .quad   0x2cb5487e17d06ba2+        .quad   0x24d2381c3950196b+        .quad   0xd7659c8185978a30+        .quad   0x7a6f7f2891d6a4f6+        .quad   0x7d09fd8d6a9aef49+        .quad   0xf0ee60be5b3db90b+        .quad   0x4c21b52c519ebfd4+        .quad   0x6011aadfc545941d++        // 2^224 * 3 * B++        .quad   0x5f67926dcf95f83c+        .quad   0x7c7e856171289071+        .quad   0xd6a1e7f3998f7a5b+        .quad   0x6fc5cc1b0b62f9e0+        .quad   0x63ded0c802cbf890+        .quad   0xfbd098ca0dff6aaa+        .quad   0x624d0afdb9b6ed99+        .quad   0x69ce18b779340b1e+        .quad   0xd1ef5528b29879cb+        .quad   0xdd1aae3cd47e9092+        .quad   0x127e0442189f2352+        .quad   0x15596b3ae57101f1++        // 2^224 * 4 * B++        .quad   0x462739d23f9179a2+        .quad   0xff83123197d6ddcf+        .quad   0x1307deb553f2148a+        .quad   0x0d2237687b5f4dda+        .quad   0x09ff31167e5124ca+        .quad   0x0be4158bd9c745df+        .quad   0x292b7d227ef556e5+        .quad   0x3aa4e241afb6d138+        .quad   0x2cc138bf2a3305f5+        .quad   0x48583f8fa2e926c3+        .quad   0x083ab1a25549d2eb+        .quad   0x32fcaa6e4687a36c++        // 2^224 * 5 * B++        .quad   0x7bc56e8dc57d9af5+        .quad   0x3e0bd2ed9df0bdf2+        .quad   0xaac014de22efe4a3+        .quad   0x4627e9cefebd6a5c+        .quad   0x3207a4732787ccdf+        .quad   0x17e31908f213e3f8+        .quad   0xd5b2ecd7f60d964e+        .quad   0x746f6336c2600be9+        .quad   0x3f4af345ab6c971c+        .quad   0xe288eb729943731f+        .quad   0x33596a8a0344186d+        .quad   0x7b4917007ed66293++        // 2^224 * 6 * B++        .quad   0x2d85fb5cab84b064+        .quad   0x497810d289f3bc14+        .quad   0x476adc447b15ce0c+        .quad   0x122ba376f844fd7b+        .quad   0x54341b28dd53a2dd+        .quad   0xaa17905bdf42fc3f+        .quad   0x0ff592d94dd2f8f4+        .quad   0x1d03620fe08cd37d+        .quad   0xc20232cda2b4e554+        .quad   0x9ed0fd42115d187f+        .quad   0x2eabb4be7dd479d9+        .quad   0x02c70bf52b68ec4c++        // 2^224 * 7 * B++        .quad   0xa287ec4b5d0b2fbb+        .quad   0x415c5790074882ca+        .quad   0xe044a61ec1d0815c+        .quad   0x26334f0a409ef5e0+        .quad   0xace532bf458d72e1+        .quad   0x5be768e07cb73cb5+        .quad   0x56cf7d94ee8bbde7+        .quad   0x6b0697e3feb43a03+        .quad   0xb6c8f04adf62a3c0+        .quad   0x3ef000ef076da45d+        .quad   0x9c9cb95849f0d2a9+        .quad   0x1cc37f43441b2fae++        // 2^224 * 8 * B++        .quad   0x508f565a5cc7324f+        .quad   0xd061c4c0e506a922+        .quad   0xfb18abdb5c45ac19+        .quad   0x6c6809c10380314a+        .quad   0xd76656f1c9ceaeb9+        .quad   0x1c5b15f818e5656a+        .quad   0x26e72832844c2334+        .quad   0x3a346f772f196838+        .quad   0xd2d55112e2da6ac8+        .quad   0xe9bd0331b1e851ed+        .quad   0x960746dd8ec67262+        .quad   0x05911b9f6ef7c5d0++        // 2^228 * 1 * B++        .quad   0xe9dcd756b637ff2d+        .quad   0xec4c348fc987f0c4+        .quad   0xced59285f3fbc7b7+        .quad   0x3305354793e1ea87+        .quad   0x01c18980c5fe9f94+        .quad   0xcd656769716fd5c8+        .quad   0x816045c3d195a086+        .quad   0x6e2b7f3266cc7982+        .quad   0xcc802468f7c3568f+        .quad   0x9de9ba8219974cb3+        .quad   0xabb7229cb5b81360+        .quad   0x44e2017a6fbeba62++        // 2^228 * 2 * B++        .quad   0xc4c2a74354dab774+        .quad   0x8e5d4c3c4eaf031a+        .quad   0xb76c23d242838f17+        .quad   0x749a098f68dce4ea+        .quad   0x87f82cf3b6ca6ecd+        .quad   0x580f893e18f4a0c2+        .quad   0x058930072604e557+        .quad   0x6cab6ac256d19c1d+        .quad   0xdcdfe0a02cc1de60+        .quad   0x032665ff51c5575b+        .quad   0x2c0c32f1073abeeb+        .quad   0x6a882014cd7b8606++        // 2^228 * 3 * B++        .quad   0xa52a92fea4747fb5+        .quad   0xdc12a4491fa5ab89+        .quad   0xd82da94bb847a4ce+        .quad   0x4d77edce9512cc4e+        .quad   0xd111d17caf4feb6e+        .quad   0x050bba42b33aa4a3+        .quad   0x17514c3ceeb46c30+        .quad   0x54bedb8b1bc27d75+        .quad   0x77c8e14577e2189c+        .quad   0xa3e46f6aff99c445+        .quad   0x3144dfc86d335343+        .quad   0x3a96559e7c4216a9++        // 2^228 * 4 * B++        .quad   0x12550d37f42ad2ee+        .quad   0x8b78e00498a1fbf5+        .quad   0x5d53078233894cb2+        .quad   0x02c84e4e3e498d0c+        .quad   0x4493896880baaa52+        .quad   0x4c98afc4f285940e+        .quad   0xef4aa79ba45448b6+        .quad   0x5278c510a57aae7f+        .quad   0xa54dd074294c0b94+        .quad   0xf55d46b8df18ffb6+        .quad   0xf06fecc58dae8366+        .quad   0x588657668190d165++        // 2^228 * 5 * B++        .quad   0xd47712311aef7117+        .quad   0x50343101229e92c7+        .quad   0x7a95e1849d159b97+        .quad   0x2449959b8b5d29c9+        .quad   0xbf5834f03de25cc3+        .quad   0xb887c8aed6815496+        .quad   0x5105221a9481e892+        .quad   0x6760ed19f7723f93+        .quad   0x669ba3b7ac35e160+        .quad   0x2eccf73fba842056+        .quad   0x1aec1f17c0804f07+        .quad   0x0d96bc031856f4e7++        // 2^228 * 6 * B++        .quad   0x3318be7775c52d82+        .quad   0x4cb764b554d0aab9+        .quad   0xabcf3d27cc773d91+        .quad   0x3bf4d1848123288a+        .quad   0xb1d534b0cc7505e1+        .quad   0x32cd003416c35288+        .quad   0xcb36a5800762c29d+        .quad   0x5bfe69b9237a0bf8+        .quad   0x183eab7e78a151ab+        .quad   0xbbe990c999093763+        .quad   0xff717d6e4ac7e335+        .quad   0x4c5cddb325f39f88++        // 2^228 * 7 * B++        .quad   0xc0f6b74d6190a6eb+        .quad   0x20ea81a42db8f4e4+        .quad   0xa8bd6f7d97315760+        .quad   0x33b1d60262ac7c21+        .quad   0x57750967e7a9f902+        .quad   0x2c37fdfc4f5b467e+        .quad   0xb261663a3177ba46+        .quad   0x3a375e78dc2d532b+        .quad   0x8141e72f2d4dddea+        .quad   0xe6eafe9862c607c8+        .quad   0x23c28458573cafd0+        .quad   0x46b9476f4ff97346++        // 2^228 * 8 * B++        .quad   0x0c1ffea44f901e5c+        .quad   0x2b0b6fb72184b782+        .quad   0xe587ff910114db88+        .quad   0x37130f364785a142+        .quad   0x1215505c0d58359f+        .quad   0x2a2013c7fc28c46b+        .quad   0x24a0a1af89ea664e+        .quad   0x4400b638a1130e1f+        .quad   0x3a01b76496ed19c3+        .quad   0x31e00ab0ed327230+        .quad   0x520a885783ca15b1+        .quad   0x06aab9875accbec7++        // 2^232 * 1 * B++        .quad   0xc1339983f5df0ebb+        .quad   0xc0f3758f512c4cac+        .quad   0x2cf1130a0bb398e1+        .quad   0x6b3cecf9aa270c62+        .quad   0x5349acf3512eeaef+        .quad   0x20c141d31cc1cb49+        .quad   0x24180c07a99a688d+        .quad   0x555ef9d1c64b2d17+        .quad   0x36a770ba3b73bd08+        .quad   0x624aef08a3afbf0c+        .quad   0x5737ff98b40946f2+        .quad   0x675f4de13381749d++        // 2^232 * 2 * B++        .quad   0x0e2c52036b1782fc+        .quad   0x64816c816cad83b4+        .quad   0xd0dcbdd96964073e+        .quad   0x13d99df70164c520+        .quad   0xa12ff6d93bdab31d+        .quad   0x0725d80f9d652dfe+        .quad   0x019c4ff39abe9487+        .quad   0x60f450b882cd3c43+        .quad   0x014b5ec321e5c0ca+        .quad   0x4fcb69c9d719bfa2+        .quad   0x4e5f1c18750023a0+        .quad   0x1c06de9e55edac80++        // 2^232 * 3 * B++        .quad   0x990f7ad6a33ec4e2+        .quad   0x6608f938be2ee08e+        .quad   0x9ca143c563284515+        .quad   0x4cf38a1fec2db60d+        .quad   0xffd52b40ff6d69aa+        .quad   0x34530b18dc4049bb+        .quad   0x5e4a5c2fa34d9897+        .quad   0x78096f8e7d32ba2d+        .quad   0xa0aaaa650dfa5ce7+        .quad   0xf9c49e2a48b5478c+        .quad   0x4f09cc7d7003725b+        .quad   0x373cad3a26091abe++        // 2^232 * 4 * B++        .quad   0xb294634d82c9f57c+        .quad   0x1fcbfde124934536+        .quad   0x9e9c4db3418cdb5a+        .quad   0x0040f3d9454419fc+        .quad   0xf1bea8fb89ddbbad+        .quad   0x3bcb2cbc61aeaecb+        .quad   0x8f58a7bb1f9b8d9d+        .quad   0x21547eda5112a686+        .quad   0xdefde939fd5986d3+        .quad   0xf4272c89510a380c+        .quad   0xb72ba407bb3119b9+        .quad   0x63550a334a254df4++        // 2^232 * 5 * B++        .quad   0x6507d6edb569cf37+        .quad   0x178429b00ca52ee1+        .quad   0xea7c0090eb6bd65d+        .quad   0x3eea62c7daf78f51+        .quad   0x9bba584572547b49+        .quad   0xf305c6fae2c408e0+        .quad   0x60e8fa69c734f18d+        .quad   0x39a92bafaa7d767a+        .quad   0x9d24c713e693274e+        .quad   0x5f63857768dbd375+        .quad   0x70525560eb8ab39a+        .quad   0x68436a0665c9c4cd++        // 2^232 * 6 * B++        .quad   0xbc0235e8202f3f27+        .quad   0xc75c00e264f975b0+        .quad   0x91a4e9d5a38c2416+        .quad   0x17b6e7f68ab789f9+        .quad   0x1e56d317e820107c+        .quad   0xc5266844840ae965+        .quad   0xc1e0a1c6320ffc7a+        .quad   0x5373669c91611472+        .quad   0x5d2814ab9a0e5257+        .quad   0x908f2084c9cab3fc+        .quad   0xafcaf5885b2d1eca+        .quad   0x1cb4b5a678f87d11++        // 2^232 * 7 * B++        .quad   0xb664c06b394afc6c+        .quad   0x0c88de2498da5fb1+        .quad   0x4f8d03164bcad834+        .quad   0x330bca78de7434a2+        .quad   0x6b74aa62a2a007e7+        .quad   0xf311e0b0f071c7b1+        .quad   0x5707e438000be223+        .quad   0x2dc0fd2d82ef6eac+        .quad   0x982eff841119744e+        .quad   0xf9695e962b074724+        .quad   0xc58ac14fbfc953fb+        .quad   0x3c31be1b369f1cf5++        // 2^232 * 8 * B++        .quad   0xb0f4864d08948aee+        .quad   0x07dc19ee91ba1c6f+        .quad   0x7975cdaea6aca158+        .quad   0x330b61134262d4bb+        .quad   0xc168bc93f9cb4272+        .quad   0xaeb8711fc7cedb98+        .quad   0x7f0e52aa34ac8d7a+        .quad   0x41cec1097e7d55bb+        .quad   0xf79619d7a26d808a+        .quad   0xbb1fd49e1d9e156d+        .quad   0x73d7c36cdba1df27+        .quad   0x26b44cd91f28777d++        // 2^236 * 1 * B++        .quad   0x300a9035393aa6d8+        .quad   0x2b501131a12bb1cd+        .quad   0x7b1ff677f093c222+        .quad   0x4309c1f8cab82bad+        .quad   0xaf44842db0285f37+        .quad   0x8753189047efc8df+        .quad   0x9574e091f820979a+        .quad   0x0e378d6069615579+        .quad   0xd9fa917183075a55+        .quad   0x4bdb5ad26b009fdc+        .quad   0x7829ad2cd63def0e+        .quad   0x078fc54975fd3877++        // 2^236 * 2 * B++        .quad   0x87dfbd1428878f2d+        .quad   0x134636dd1e9421a1+        .quad   0x4f17c951257341a3+        .quad   0x5df98d4bad296cb8+        .quad   0xe2004b5bb833a98a+        .quad   0x44775dec2d4c3330+        .quad   0x3aa244067eace913+        .quad   0x272630e3d58e00a9+        .quad   0xf3678fd0ecc90b54+        .quad   0xf001459b12043599+        .quad   0x26725fbc3758b89b+        .quad   0x4325e4aa73a719ae++        // 2^236 * 3 * B++        .quad   0x657dc6ef433c3493+        .quad   0x65375e9f80dbf8c3+        .quad   0x47fd2d465b372dae+        .quad   0x4966ab79796e7947+        .quad   0xed24629acf69f59d+        .quad   0x2a4a1ccedd5abbf4+        .quad   0x3535ca1f56b2d67b+        .quad   0x5d8c68d043b1b42d+        .quad   0xee332d4de3b42b0a+        .quad   0xd84e5a2b16a4601c+        .quad   0x78243877078ba3e4+        .quad   0x77ed1eb4184ee437++        // 2^236 * 4 * B++        .quad   0xbfd4e13f201839a0+        .quad   0xaeefffe23e3df161+        .quad   0xb65b04f06b5d1fe3+        .quad   0x52e085fb2b62fbc0+        .quad   0x185d43f89e92ed1a+        .quad   0xb04a1eeafe4719c6+        .quad   0x499fbe88a6f03f4f+        .quad   0x5d8b0d2f3c859bdd+        .quad   0x124079eaa54cf2ba+        .quad   0xd72465eb001b26e7+        .quad   0x6843bcfdc97af7fd+        .quad   0x0524b42b55eacd02++        // 2^236 * 5 * B++        .quad   0xfd0d5dbee45447b0+        .quad   0x6cec351a092005ee+        .quad   0x99a47844567579cb+        .quad   0x59d242a216e7fa45+        .quad   0xbc18dcad9b829eac+        .quad   0x23ae7d28b5f579d0+        .quad   0xc346122a69384233+        .quad   0x1a6110b2e7d4ac89+        .quad   0x4f833f6ae66997ac+        .quad   0x6849762a361839a4+        .quad   0x6985dec1970ab525+        .quad   0x53045e89dcb1f546++        // 2^236 * 6 * B++        .quad   0xcb8bb346d75353db+        .quad   0xfcfcb24bae511e22+        .quad   0xcba48d40d50ae6ef+        .quad   0x26e3bae5f4f7cb5d+        .quad   0x84da3cde8d45fe12+        .quad   0xbd42c218e444e2d2+        .quad   0xa85196781f7e3598+        .quad   0x7642c93f5616e2b2+        .quad   0x2323daa74595f8e4+        .quad   0xde688c8b857abeb4+        .quad   0x3fc48e961c59326e+        .quad   0x0b2e73ca15c9b8ba++        // 2^236 * 7 * B++        .quad   0xd6bb4428c17f5026+        .quad   0x9eb27223fb5a9ca7+        .quad   0xe37ba5031919c644+        .quad   0x21ce380db59a6602+        .quad   0x0e3fbfaf79c03a55+        .quad   0x3077af054cbb5acf+        .quad   0xd5c55245db3de39f+        .quad   0x015e68c1476a4af7+        .quad   0xc1d5285220066a38+        .quad   0x95603e523570aef3+        .quad   0x832659a7226b8a4d+        .quad   0x5dd689091f8eedc9++        // 2^236 * 8 * B++        .quad   0xcbac84debfd3c856+        .quad   0x1624c348b35ff244+        .quad   0xb7f88dca5d9cad07+        .quad   0x3b0e574da2c2ebe8+        .quad   0x1d022591a5313084+        .quad   0xca2d4aaed6270872+        .quad   0x86a12b852f0bfd20+        .quad   0x56e6c439ad7da748+        .quad   0xc704ff4942bdbae6+        .quad   0x5e21ade2b2de1f79+        .quad   0xe95db3f35652fad8+        .quad   0x0822b5378f08ebc1++        // 2^240 * 1 * B++        .quad   0x51f048478f387475+        .quad   0xb25dbcf49cbecb3c+        .quad   0x9aab1244d99f2055+        .quad   0x2c709e6c1c10a5d6+        .quad   0xe1b7f29362730383+        .quad   0x4b5279ffebca8a2c+        .quad   0xdafc778abfd41314+        .quad   0x7deb10149c72610f+        .quad   0xcb62af6a8766ee7a+        .quad   0x66cbec045553cd0e+        .quad   0x588001380f0be4b5+        .quad   0x08e68e9ff62ce2ea++        // 2^240 * 2 * B++        .quad   0x34ad500a4bc130ad+        .quad   0x8d38db493d0bd49c+        .quad   0xa25c3d98500a89be+        .quad   0x2f1f3f87eeba3b09+        .quad   0x2f2d09d50ab8f2f9+        .quad   0xacb9218dc55923df+        .quad   0x4a8f342673766cb9+        .quad   0x4cb13bd738f719f5+        .quad   0xf7848c75e515b64a+        .quad   0xa59501badb4a9038+        .quad   0xc20d313f3f751b50+        .quad   0x19a1e353c0ae2ee8++        // 2^240 * 3 * B++        .quad   0x7d1c7560bafa05c3+        .quad   0xb3e1a0a0c6e55e61+        .quad   0xe3529718c0d66473+        .quad   0x41546b11c20c3486+        .quad   0xb42172cdd596bdbd+        .quad   0x93e0454398eefc40+        .quad   0x9fb15347b44109b5+        .quad   0x736bd3990266ae34+        .quad   0x85532d509334b3b4+        .quad   0x46fd114b60816573+        .quad   0xcc5f5f30425c8375+        .quad   0x412295a2b87fab5c++        // 2^240 * 4 * B++        .quad   0x19c99b88f57ed6e9+        .quad   0x5393cb266df8c825+        .quad   0x5cee3213b30ad273+        .quad   0x14e153ebb52d2e34+        .quad   0x2e655261e293eac6+        .quad   0x845a92032133acdb+        .quad   0x460975cb7900996b+        .quad   0x0760bb8d195add80+        .quad   0x413e1a17cde6818a+        .quad   0x57156da9ed69a084+        .quad   0x2cbf268f46caccb1+        .quad   0x6b34be9bc33ac5f2++        // 2^240 * 5 * B++        .quad   0xf3df2f643a78c0b2+        .quad   0x4c3e971ef22e027c+        .quad   0xec7d1c5e49c1b5a3+        .quad   0x2012c18f0922dd2d+        .quad   0x11fc69656571f2d3+        .quad   0xc6c9e845530e737a+        .quad   0xe33ae7a2d4fe5035+        .quad   0x01b9c7b62e6dd30b+        .quad   0x880b55e55ac89d29+        .quad   0x1483241f45a0a763+        .quad   0x3d36efdfc2e76c1f+        .quad   0x08af5b784e4bade8++        // 2^240 * 6 * B++        .quad   0x283499dc881f2533+        .quad   0x9d0525da779323b6+        .quad   0x897addfb673441f4+        .quad   0x32b79d71163a168d+        .quad   0xe27314d289cc2c4b+        .quad   0x4be4bd11a287178d+        .quad   0x18d528d6fa3364ce+        .quad   0x6423c1d5afd9826e+        .quad   0xcc85f8d9edfcb36a+        .quad   0x22bcc28f3746e5f9+        .quad   0xe49de338f9e5d3cd+        .quad   0x480a5efbc13e2dcc++        // 2^240 * 7 * B++        .quad   0x0b51e70b01622071+        .quad   0x06b505cf8b1dafc5+        .quad   0x2c6bb061ef5aabcd+        .quad   0x47aa27600cb7bf31+        .quad   0xb6614ce442ce221f+        .quad   0x6e199dcc4c053928+        .quad   0x663fb4a4dc1cbe03+        .quad   0x24b31d47691c8e06+        .quad   0x2a541eedc015f8c3+        .quad   0x11a4fe7e7c693f7c+        .quad   0xf0af66134ea278d6+        .quad   0x545b585d14dda094++        // 2^240 * 8 * B++        .quad   0x67bf275ea0d43a0f+        .quad   0xade68e34089beebe+        .quad   0x4289134cd479e72e+        .quad   0x0f62f9c332ba5454+        .quad   0x6204e4d0e3b321e1+        .quad   0x3baa637a28ff1e95+        .quad   0x0b0ccffd5b99bd9e+        .quad   0x4d22dc3e64c8d071+        .quad   0xfcb46589d63b5f39+        .quad   0x5cae6a3f57cbcf61+        .quad   0xfebac2d2953afa05+        .quad   0x1c0fa01a36371436++        // 2^244 * 1 * B++        .quad   0xe7547449bc7cd692+        .quad   0x0f9abeaae6f73ddf+        .quad   0x4af01ca700837e29+        .quad   0x63ab1b5d3f1bc183+        .quad   0xc11ee5e854c53fae+        .quad   0x6a0b06c12b4f3ff4+        .quad   0x33540f80e0b67a72+        .quad   0x15f18fc3cd07e3ef+        .quad   0x32750763b028f48c+        .quad   0x06020740556a065f+        .quad   0xd53bd812c3495b58+        .quad   0x08706c9b865f508d++        // 2^244 * 2 * B++        .quad   0xf37ca2ab3d343dff+        .quad   0x1a8c6a2d80abc617+        .quad   0x8e49e035d4ccffca+        .quad   0x48b46beebaa1d1b9+        .quad   0xcc991b4138b41246+        .quad   0x243b9c526f9ac26b+        .quad   0xb9ef494db7cbabbd+        .quad   0x5fba433dd082ed00+        .quad   0x9c49e355c9941ad0+        .quad   0xb9734ade74498f84+        .quad   0x41c3fed066663e5c+        .quad   0x0ecfedf8e8e710b3++        // 2^244 * 3 * B++        .quad   0x76430f9f9cd470d9+        .quad   0xb62acc9ba42f6008+        .quad   0x1898297c59adad5e+        .quad   0x7789dd2db78c5080+        .quad   0x744f7463e9403762+        .quad   0xf79a8dee8dfcc9c9+        .quad   0x163a649655e4cde3+        .quad   0x3b61788db284f435+        .quad   0xb22228190d6ef6b2+        .quad   0xa94a66b246ce4bfa+        .quad   0x46c1a77a4f0b6cc7+        .quad   0x4236ccffeb7338cf++        // 2^244 * 4 * B++        .quad   0x8497404d0d55e274+        .quad   0x6c6663d9c4ad2b53+        .quad   0xec2fb0d9ada95734+        .quad   0x2617e120cdb8f73c+        .quad   0x3bd82dbfda777df6+        .quad   0x71b177cc0b98369e+        .quad   0x1d0e8463850c3699+        .quad   0x5a71945b48e2d1f1+        .quad   0x6f203dd5405b4b42+        .quad   0x327ec60410b24509+        .quad   0x9c347230ac2a8846+        .quad   0x77de29fc11ffeb6a++        // 2^244 * 5 * B++        .quad   0xb0ac57c983b778a8+        .quad   0x53cdcca9d7fe912c+        .quad   0x61c2b854ff1f59dc+        .quad   0x3a1a2cf0f0de7dac+        .quad   0x835e138fecced2ca+        .quad   0x8c9eaf13ea963b9a+        .quad   0xc95fbfc0b2160ea6+        .quad   0x575e66f3ad877892+        .quad   0x99803a27c88fcb3a+        .quad   0x345a6789275ec0b0+        .quad   0x459789d0ff6c2be5+        .quad   0x62f882651e70a8b2++        // 2^244 * 6 * B++        .quad   0x085ae2c759ff1be4+        .quad   0x149145c93b0e40b7+        .quad   0xc467e7fa7ff27379+        .quad   0x4eeecf0ad5c73a95+        .quad   0x6d822986698a19e0+        .quad   0xdc9821e174d78a71+        .quad   0x41a85f31f6cb1f47+        .quad   0x352721c2bcda9c51+        .quad   0x48329952213fc985+        .quad   0x1087cf0d368a1746+        .quad   0x8e5261b166c15aa5+        .quad   0x2d5b2d842ed24c21++        // 2^244 * 7 * B++        .quad   0x02cfebd9ebd3ded1+        .quad   0xd45b217739021974+        .quad   0x7576f813fe30a1b7+        .quad   0x5691b6f9a34ef6c2+        .quad   0x5eb7d13d196ac533+        .quad   0x377234ecdb80be2b+        .quad   0xe144cffc7cf5ae24+        .quad   0x5226bcf9c441acec+        .quad   0x79ee6c7223e5b547+        .quad   0x6f5f50768330d679+        .quad   0xed73e1e96d8adce9+        .quad   0x27c3da1e1d8ccc03++        // 2^244 * 8 * B++        .quad   0x7eb9efb23fe24c74+        .quad   0x3e50f49f1651be01+        .quad   0x3ea732dc21858dea+        .quad   0x17377bd75bb810f9+        .quad   0x28302e71630ef9f6+        .quad   0xc2d4a2032b64cee0+        .quad   0x090820304b6292be+        .quad   0x5fca747aa82adf18+        .quad   0x232a03c35c258ea5+        .quad   0x86f23a2c6bcb0cf1+        .quad   0x3dad8d0d2e442166+        .quad   0x04a8933cab76862b++        // 2^248 * 1 * B++        .quad   0xd2c604b622943dff+        .quad   0xbc8cbece44cfb3a0+        .quad   0x5d254ff397808678+        .quad   0x0fa3614f3b1ca6bf+        .quad   0x69082b0e8c936a50+        .quad   0xf9c9a035c1dac5b6+        .quad   0x6fb73e54c4dfb634+        .quad   0x4005419b1d2bc140+        .quad   0xa003febdb9be82f0+        .quad   0x2089c1af3a44ac90+        .quad   0xf8499f911954fa8e+        .quad   0x1fba218aef40ab42++        // 2^248 * 2 * B++        .quad   0xab549448fac8f53e+        .quad   0x81f6e89a7ba63741+        .quad   0x74fd6c7d6c2b5e01+        .quad   0x392e3acaa8c86e42+        .quad   0x4f3e57043e7b0194+        .quad   0xa81d3eee08daaf7f+        .quad   0xc839c6ab99dcdef1+        .quad   0x6c535d13ff7761d5+        .quad   0x4cbd34e93e8a35af+        .quad   0x2e0781445887e816+        .quad   0x19319c76f29ab0ab+        .quad   0x25e17fe4d50ac13b++        // 2^248 * 3 * B++        .quad   0x0a289bd71e04f676+        .quad   0x208e1c52d6420f95+        .quad   0x5186d8b034691fab+        .quad   0x255751442a9fb351+        .quad   0x915f7ff576f121a7+        .quad   0xc34a32272fcd87e3+        .quad   0xccba2fde4d1be526+        .quad   0x6bba828f8969899b+        .quad   0xe2d1bc6690fe3901+        .quad   0x4cb54a18a0997ad5+        .quad   0x971d6914af8460d4+        .quad   0x559d504f7f6b7be4++        // 2^248 * 4 * B++        .quad   0xa7738378b3eb54d5+        .quad   0x1d69d366a5553c7c+        .quad   0x0a26cf62f92800ba+        .quad   0x01ab12d5807e3217+        .quad   0x9c4891e7f6d266fd+        .quad   0x0744a19b0307781b+        .quad   0x88388f1d6061e23b+        .quad   0x123ea6a3354bd50e+        .quad   0x118d189041e32d96+        .quad   0xb9ede3c2d8315848+        .quad   0x1eab4271d83245d9+        .quad   0x4a3961e2c918a154++        // 2^248 * 5 * B++        .quad   0x71dc3be0f8e6bba0+        .quad   0xd6cef8347effe30a+        .quad   0xa992425fe13a476a+        .quad   0x2cd6bce3fb1db763+        .quad   0x0327d644f3233f1e+        .quad   0x499a260e34fcf016+        .quad   0x83b5a716f2dab979+        .quad   0x68aceead9bd4111f+        .quad   0x38b4c90ef3d7c210+        .quad   0x308e6e24b7ad040c+        .quad   0x3860d9f1b7e73e23+        .quad   0x595760d5b508f597++        // 2^248 * 6 * B++        .quad   0x6129bfe104aa6397+        .quad   0x8f960008a4a7fccb+        .quad   0x3f8bc0897d909458+        .quad   0x709fa43edcb291a9+        .quad   0x882acbebfd022790+        .quad   0x89af3305c4115760+        .quad   0x65f492e37d3473f4+        .quad   0x2cb2c5df54515a2b+        .quad   0xeb0a5d8c63fd2aca+        .quad   0xd22bc1662e694eff+        .quad   0x2723f36ef8cbb03a+        .quad   0x70f029ecf0c8131f++        // 2^248 * 7 * B++        .quad   0x461307b32eed3e33+        .quad   0xae042f33a45581e7+        .quad   0xc94449d3195f0366+        .quad   0x0b7d5d8a6c314858+        .quad   0x2a6aafaa5e10b0b9+        .quad   0x78f0a370ef041aa9+        .quad   0x773efb77aa3ad61f+        .quad   0x44eca5a2a74bd9e1+        .quad   0x25d448327b95d543+        .quad   0x70d38300a3340f1d+        .quad   0xde1c531c60e1c52b+        .quad   0x272224512c7de9e4++        // 2^248 * 8 * B++        .quad   0x1abc92af49c5342e+        .quad   0xffeed811b2e6fad0+        .quad   0xefa28c8dfcc84e29+        .quad   0x11b5df18a44cc543+        .quad   0xbf7bbb8a42a975fc+        .quad   0x8c5c397796ada358+        .quad   0xe27fc76fcdedaa48+        .quad   0x19735fd7f6bc20a6+        .quad   0xe3ab90d042c84266+        .quad   0xeb848e0f7f19547e+        .quad   0x2503a1d065a497b9+        .quad   0x0fef911191df895f
+ cbits/s2n/arm/p256_montjadd.S view
@@ -0,0 +1,3165 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Point addition on NIST curve P-256 in Montgomery-Jacobian coordinates+//+//    extern void p256_montjadd(uint64_t p3[static 12], const uint64_t p1[static 12],+//                              const uint64_t p2[static 12]);+//+// Does p3 := p1 + p2 where all points are regarded as Jacobian triples with+// each coordinate in the Montgomery domain, i.e. x' = (2^256 * x) mod p_256.+// A Jacobian triple (x',y',z') represents affine point (x/z^2,y/z^3).+//+// Standard ARM ABI: X0 = p3, X1 = p1, X2 = p2+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++// This is functionally equivalent to p256_montjadd in unopt/p256_montjadd.S.+// This is the result of doing the following sequence of optimizations:+//   1. Function inlining+//   2. Eliminating redundant load/store instructions+//   3. Folding (add addr, const) + load/store+// Function inlining is done manually. The second and third optimizations are+// done by a script.++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_montjadd)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_montjadd)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_montjadd)++        .text+        .balign 4++#define NUMSIZE 32+#define NSPACE NUMSIZE*7++S2N_BN_SYMBOL(p256_montjadd):+        CFI_START++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x26)+        CFI_PUSH2(x27,x30)+        CFI_DEC_SP(NSPACE)++        mov x21, x0+        mov x22, x1+        mov x23, x2+        mov x0, sp+        ldr q19, [x22, #64]+        ldp x9, x13, [x22, #64]+        ldr q23, [x22, #80]+        ldr q0, [x22, #64]+        ldp x1, x10, [x22, #80]+        uzp2 v29.4s, v19.4s, v19.4s+        xtn v4.2s, v19.2d+        umulh x8, x9, x13+        rev64 v20.4s, v23.4s+        umull v16.2d, v19.2s, v19.2s+        umull v1.2d, v29.2s, v4.2s+        mul v20.4s, v20.4s, v0.4s+        subs x14, x9, x13+        umulh x15, x9, x1+        mov x16, v16.d[1]+        umull2 v4.2d, v19.4s, v19.4s+        mov x4, v16.d[0]+        uzp1 v17.4s, v23.4s, v0.4s+        uaddlp v19.2d, v20.4s+        lsr x7, x8, #63+        mul x11, x9, x13+        mov x12, v1.d[0]+        csetm x5, cc+        cneg x6, x14, cc+        mov x3, v4.d[1]+        mov x14, v4.d[0]+        subs x2, x10, x1+        mov x9, v1.d[1]+        cneg x17, x2, cc+        cinv x2, x5, cc+        adds x5, x4, x12, lsl #33+        extr x4, x8, x11, #63+        lsr x8, x12, #31+        uzp1 v20.4s, v0.4s, v0.4s+        shl v19.2d, v19.2d, #32+        adc x16, x16, x8+        adds x8, x14, x9, lsl #33+        lsr x14, x9, #31+        lsl x9, x5, #32+        umlal v19.2d, v20.2s, v17.2s+        adc x14, x3, x14+        adds x16, x16, x11, lsl #1+        lsr x3, x5, #32+        umulh x12, x6, x17+        adcs x4, x8, x4+        adc x11, x14, x7+        subs x8, x5, x9+        sbc x5, x5, x3+        adds x16, x16, x9+        mov x14, v19.d[0]+        mul x17, x6, x17+        adcs x3, x4, x3+        lsl x7, x16, #32+        umulh x13, x13, x10+        adcs x11, x11, x8+        lsr x8, x16, #32+        adc x5, x5, xzr+        subs x9, x16, x7+        sbc x16, x16, x8+        adds x7, x3, x7+        mov x3, v19.d[1]+        adcs x6, x11, x8+        umulh x11, x1, x10+        adcs x5, x5, x9+        eor x8, x12, x2+        adc x9, x16, xzr+        adds x16, x14, x15+        adc x15, x15, xzr+        adds x12, x16, x3+        eor x16, x17, x2+        mul x4, x1, x10+        adcs x15, x15, x13+        adc x17, x13, xzr+        adds x15, x15, x3+        adc x3, x17, xzr+        cmn x2, #0x1+        mul x17, x10, x10+        adcs x12, x12, x16+        adcs x16, x15, x8+        umulh x10, x10, x10+        adc x2, x3, x2+        adds x14, x14, x14+        adcs x12, x12, x12+        adcs x16, x16, x16+        adcs x2, x2, x2+        adc x15, xzr, xzr+        adds x14, x14, x7+        mul x3, x1, x1+        adcs x12, x12, x6+        lsr x7, x14, #32+        adcs x16, x16, x5+        lsl x5, x14, #32+        umulh x13, x1, x1+        adcs x2, x2, x9+        mov x6, #0xffffffff+        adc x15, x15, xzr+        adds x8, x4, x4+        adcs x1, x11, x11+        mov x11, #0xffffffff00000001+        adc x4, xzr, xzr+        subs x9, x14, x5+        sbc x14, x14, x7+        adds x12, x12, x5+        adcs x16, x16, x7+        lsl x5, x12, #32+        lsr x7, x12, #32+        adcs x2, x2, x9+        adcs x14, x15, x14+        adc x15, xzr, xzr+        subs x9, x12, x5+        sbc x12, x12, x7+        adds x16, x16, x5+        adcs x2, x2, x7+        adcs x14, x14, x9+        adcs x12, x15, x12+        adc x15, xzr, xzr+        adds x16, x16, x3+        adcs x2, x2, x13+        adcs x14, x14, x17+        adcs x12, x12, x10+        adc x15, x15, xzr+        adds x2, x2, x8+        adcs x14, x14, x1+        adcs x12, x12, x4+        adcs x15, x15, xzr+        adds x3, x16, #0x1+        sbcs x5, x2, x6+        sbcs x8, x14, xzr+        sbcs x11, x12, x11+        sbcs xzr, x15, xzr+        csel x19, x3, x16, cs+        csel x14, x8, x14, cs+        csel x12, x11, x12, cs+        csel x20, x5, x2, cs+        stp x14, x12, [x0, #16]+        stp x19, x20, [x0]+        ldr q19, [x23, #64]+        ldp x9, x13, [x23, #64]+        ldr q23, [x23, #80]+        ldr q0, [x23, #64]+        ldp x1, x10, [x23, #80]+        uzp2 v29.4s, v19.4s, v19.4s+        xtn v4.2s, v19.2d+        umulh x8, x9, x13+        rev64 v20.4s, v23.4s+        umull v16.2d, v19.2s, v19.2s+        umull v1.2d, v29.2s, v4.2s+        mul v20.4s, v20.4s, v0.4s+        subs x14, x9, x13+        umulh x15, x9, x1+        mov x16, v16.d[1]+        umull2 v4.2d, v19.4s, v19.4s+        mov x4, v16.d[0]+        uzp1 v17.4s, v23.4s, v0.4s+        uaddlp v19.2d, v20.4s+        lsr x7, x8, #63+        mul x11, x9, x13+        mov x12, v1.d[0]+        csetm x5, cc+        cneg x6, x14, cc+        mov x3, v4.d[1]+        mov x14, v4.d[0]+        subs x2, x10, x1+        mov x9, v1.d[1]+        cneg x17, x2, cc+        cinv x2, x5, cc+        adds x5, x4, x12, lsl #33+        extr x4, x8, x11, #63+        lsr x8, x12, #31+        uzp1 v20.4s, v0.4s, v0.4s+        shl v19.2d, v19.2d, #32+        adc x16, x16, x8+        adds x8, x14, x9, lsl #33+        lsr x14, x9, #31+        lsl x9, x5, #32+        umlal v19.2d, v20.2s, v17.2s+        adc x14, x3, x14+        adds x16, x16, x11, lsl #1+        lsr x3, x5, #32+        umulh x12, x6, x17+        adcs x4, x8, x4+        adc x11, x14, x7+        subs x8, x5, x9+        sbc x5, x5, x3+        adds x16, x16, x9+        mov x14, v19.d[0]+        mul x17, x6, x17+        adcs x3, x4, x3+        lsl x7, x16, #32+        umulh x13, x13, x10+        adcs x11, x11, x8+        lsr x8, x16, #32+        adc x5, x5, xzr+        subs x9, x16, x7+        sbc x16, x16, x8+        adds x7, x3, x7+        mov x3, v19.d[1]+        adcs x6, x11, x8+        umulh x11, x1, x10+        adcs x5, x5, x9+        eor x8, x12, x2+        adc x9, x16, xzr+        adds x16, x14, x15+        adc x15, x15, xzr+        adds x12, x16, x3+        eor x16, x17, x2+        mul x4, x1, x10+        adcs x15, x15, x13+        adc x17, x13, xzr+        adds x15, x15, x3+        adc x3, x17, xzr+        cmn x2, #0x1+        mul x17, x10, x10+        adcs x12, x12, x16+        adcs x16, x15, x8+        umulh x10, x10, x10+        adc x2, x3, x2+        adds x14, x14, x14+        adcs x12, x12, x12+        adcs x16, x16, x16+        adcs x2, x2, x2+        adc x15, xzr, xzr+        adds x14, x14, x7+        mul x3, x1, x1+        adcs x12, x12, x6+        lsr x7, x14, #32+        adcs x16, x16, x5+        lsl x5, x14, #32+        umulh x13, x1, x1+        adcs x2, x2, x9+        mov x6, #0xffffffff+        adc x15, x15, xzr+        adds x8, x4, x4+        adcs x1, x11, x11+        mov x11, #0xffffffff00000001+        adc x4, xzr, xzr+        subs x9, x14, x5+        sbc x14, x14, x7+        adds x12, x12, x5+        adcs x16, x16, x7+        lsl x5, x12, #32+        lsr x7, x12, #32+        adcs x2, x2, x9+        adcs x14, x15, x14+        adc x15, xzr, xzr+        subs x9, x12, x5+        sbc x12, x12, x7+        adds x16, x16, x5+        adcs x2, x2, x7+        adcs x14, x14, x9+        adcs x12, x15, x12+        adc x15, xzr, xzr+        adds x16, x16, x3+        adcs x2, x2, x13+        adcs x14, x14, x17+        adcs x12, x12, x10+        adc x15, x15, xzr+        adds x2, x2, x8+        adcs x14, x14, x1+        adcs x12, x12, x4+        adcs x15, x15, xzr+        adds x3, x16, #0x1+        sbcs x5, x2, x6+        sbcs x8, x14, xzr+        sbcs x11, x12, x11+        sbcs xzr, x15, xzr+        csel x16, x3, x16, cs+        csel x14, x8, x14, cs+        csel x12, x11, x12, cs+        csel x2, x5, x2, cs+        stp x14, x12, [sp, #176]+        stp x16, x2, [sp, #160]+        ldr q20, [x22, #32]+        ldp x7, x17, [x23, #64]+        ldr q0, [x23, #64]+        ldp x6, x10, [x22, #32]+        ldp x11, x15, [x23, #80]+        rev64 v16.4s, v20.4s+        subs x4, x7, x17+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x17, x10+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x11, x7+        ldr q20, [x22, #48]+        sbcs x5, x15, x17+        ngc x17, xzr+        subs x8, x11, x15+        uaddlp v27.2d, v16.4s+        umulh x4, x7, x6+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [x23, #80]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [x22, #48]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x24, x3, x13+        adcs x25, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x26, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x27, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x24+        adcs x15, x16, x25+        eor x5, x17, x4+        adcs x9, x1, x26+        eor x1, x10, x5+        adcs x16, x2, x27+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x11, x11, x13+        and x1, x1, x13+        adcs x4, x4, x1+        and x1, x12, x13+        stp x11, x4, [sp, #192]+        adcs x4, x7, xzr+        adc x1, x17, x1+        stp x4, x1, [sp, #208]+        ldr q20, [x23, #32]+        ldp x7, x17, [x22, #64]+        ldr q0, [x22, #64]+        ldp x6, x10, [x23, #32]+        ldp x11, x15, [x22, #80]+        rev64 v16.4s, v20.4s+        subs x4, x7, x17+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x17, x10+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x11, x7+        ldr q20, [x23, #48]+        sbcs x5, x15, x17+        ngc x17, xzr+        subs x8, x11, x15+        uaddlp v27.2d, v16.4s+        umulh x4, x7, x6+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [x22, #80]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [x23, #48]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x24, x3, x13+        adcs x25, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x26, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x27, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x24+        adcs x15, x16, x25+        eor x5, x17, x4+        adcs x9, x1, x26+        eor x1, x10, x5+        adcs x16, x2, x27+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x24, x11, x13+        and x1, x1, x13+        adcs x25, x4, x1+        and x1, x12, x13+        stp x24, x25, [sp, #32]+        adcs x4, x7, xzr+        adc x1, x17, x1+        stp x4, x1, [sp, #48]+        mov x1, sp+        ldr q20, [x23, #0]+        ldr q0, [x1]+        ldp x6, x10, [x23, #0]+        ldp x11, x15, [x1, #16]+        rev64 v16.4s, v20.4s+        subs x4, x19, x20+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x20, x10+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x11, x19+        ldr q20, [x23, #16]+        sbcs x5, x15, x20+        ngc x17, xzr+        subs x8, x11, x15+        uaddlp v27.2d, v16.4s+        umulh x4, x19, x6+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [x1, #16]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [x23, #16]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x19, x3, x13+        adcs x20, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x26, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x27, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x19+        adcs x15, x16, x20+        eor x5, x17, x4+        adcs x9, x1, x26+        eor x1, x10, x5+        adcs x16, x2, x27+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x11, x11, x13+        and x1, x1, x13+        adcs x4, x4, x1+        and x1, x12, x13+        stp x11, x4, [sp, #64]+        adcs x4, x7, xzr+        adc x1, x17, x1+        stp x4, x1, [sp, #80]+        ldr q20, [x22, #0]+        ldp x7, x17, [sp, #160]+        ldr q0, [sp, #160]+        ldp x6, x10, [x22, #0]+        ldp x11, x15, [sp, #176]+        rev64 v16.4s, v20.4s+        subs x4, x7, x17+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x17, x10+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x11, x7+        ldr q20, [x22, #16]+        sbcs x5, x15, x17+        ngc x17, xzr+        subs x8, x11, x15+        uaddlp v27.2d, v16.4s+        umulh x4, x7, x6+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [sp, #176]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [x22, #16]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x19, x3, x13+        adcs x20, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x26, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x27, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x19+        adcs x15, x16, x20+        eor x5, x17, x4+        adcs x9, x1, x26+        eor x1, x10, x5+        adcs x16, x2, x27+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x11, x11, x13+        and x1, x1, x13+        adcs x4, x4, x1+        and x1, x12, x13+        stp x11, x4, [sp, #128]+        adcs x4, x7, xzr+        adc x1, x17, x1+        stp x4, x1, [sp, #144]+        mov x1, sp+        ldr q20, [sp, #32]+        ldp x7, x17, [x1]+        ldr q0, [x1]+        ldp x11, x15, [x1, #16]+        rev64 v16.4s, v20.4s+        subs x4, x7, x17+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x17, x25+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x11, x7+        ldr q20, [sp, #48]+        sbcs x5, x15, x17+        ngc x17, xzr+        subs x8, x11, x15+        uaddlp v27.2d, v16.4s+        umulh x4, x7, x24+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x25, x24+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [x1, #16]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [sp, #48]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x24, x7+        sbcs x9, x25, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x19, x3, x13+        adcs x20, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x24, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x25, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x19+        adcs x15, x16, x20+        eor x5, x17, x4+        adcs x9, x1, x24+        eor x1, x10, x5+        adcs x16, x2, x25+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x19, x11, x13+        and x1, x1, x13+        adcs x20, x4, x1+        and x1, x12, x13+        adcs x4, x7, xzr+        adc x1, x17, x1+        stp x4, x1, [sp, #48]+        ldr q20, [sp, #192]+        ldp x7, x17, [sp, #160]+        ldr q0, [sp, #160]+        ldp x6, x10, [sp, #192]+        ldp x11, x15, [sp, #176]+        rev64 v16.4s, v20.4s+        subs x4, x7, x17+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x17, x10+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x11, x7+        ldr q20, [sp, #208]+        sbcs x5, x15, x17+        ngc x17, xzr+        subs x8, x11, x15+        uaddlp v27.2d, v16.4s+        umulh x4, x7, x6+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [sp, #176]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [sp, #208]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x24, x3, x13+        adcs x25, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x26, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x27, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x24+        adcs x15, x16, x25+        eor x5, x17, x4+        adcs x9, x1, x26+        eor x1, x10, x5+        adcs x16, x2, x27+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x9, x11, x13+        and x1, x1, x13+        adcs x10, x4, x1+        and x1, x12, x13+        stp x9, x10, [sp, #192]+        adcs x11, x7, xzr+        adc x12, x17, x1+        stp x11, x12, [sp, #208]+        ldp x5, x6, [sp, #64]+        ldp x4, x3, [sp, #128]+        subs x5, x5, x4+        sbcs x6, x6, x3+        ldp x7, x8, [sp, #80]+        ldp x4, x3, [sp, #144]+        sbcs x7, x7, x4+        sbcs x8, x8, x3+        csetm x3, cc+        adds x13, x5, x3+        and x4, x3, #0xffffffff+        adcs x24, x6, x4+        adcs x25, x7, xzr+        and x4, x3, #0xffffffff00000001+        adc x26, x8, x4+        stp x13, x24, [sp, #160]+        stp x25, x26, [sp, #176]+        subs x5, x19, x9+        sbcs x6, x20, x10+        ldp x7, x8, [sp, #48]+        sbcs x7, x7, x11+        sbcs x8, x8, x12+        csetm x3, cc+        adds x19, x5, x3+        and x4, x3, #0xffffffff+        adcs x20, x6, x4+        adcs x7, x7, xzr+        and x4, x3, #0xffffffff00000001+        adc x8, x8, x4+        stp x19, x20, [sp, #32]+        stp x7, x8, [sp, #48]+        ldr q19, [sp, #160]+        ldr q23, [sp, #176]+        ldr q0, [sp, #160]+        uzp2 v29.4s, v19.4s, v19.4s+        xtn v4.2s, v19.2d+        umulh x8, x13, x24+        rev64 v20.4s, v23.4s+        umull v16.2d, v19.2s, v19.2s+        umull v1.2d, v29.2s, v4.2s+        mul v20.4s, v20.4s, v0.4s+        subs x14, x13, x24+        umulh x15, x13, x25+        mov x16, v16.d[1]+        umull2 v4.2d, v19.4s, v19.4s+        mov x4, v16.d[0]+        uzp1 v17.4s, v23.4s, v0.4s+        uaddlp v19.2d, v20.4s+        lsr x7, x8, #63+        mul x11, x13, x24+        mov x12, v1.d[0]+        csetm x5, cc+        cneg x6, x14, cc+        mov x3, v4.d[1]+        mov x14, v4.d[0]+        subs x2, x26, x25+        mov x9, v1.d[1]+        cneg x17, x2, cc+        cinv x2, x5, cc+        adds x5, x4, x12, lsl #33+        extr x4, x8, x11, #63+        lsr x8, x12, #31+        uzp1 v20.4s, v0.4s, v0.4s+        shl v19.2d, v19.2d, #32+        adc x16, x16, x8+        adds x8, x14, x9, lsl #33+        lsr x14, x9, #31+        lsl x9, x5, #32+        umlal v19.2d, v20.2s, v17.2s+        adc x14, x3, x14+        adds x16, x16, x11, lsl #1+        lsr x3, x5, #32+        umulh x12, x6, x17+        adcs x4, x8, x4+        adc x11, x14, x7+        subs x8, x5, x9+        sbc x5, x5, x3+        adds x16, x16, x9+        mov x14, v19.d[0]+        mul x17, x6, x17+        adcs x3, x4, x3+        lsl x7, x16, #32+        umulh x13, x24, x26+        adcs x11, x11, x8+        lsr x8, x16, #32+        adc x5, x5, xzr+        subs x9, x16, x7+        sbc x16, x16, x8+        adds x7, x3, x7+        mov x3, v19.d[1]+        adcs x6, x11, x8+        umulh x11, x25, x26+        adcs x5, x5, x9+        eor x8, x12, x2+        adc x9, x16, xzr+        adds x16, x14, x15+        adc x15, x15, xzr+        adds x12, x16, x3+        eor x16, x17, x2+        mul x4, x25, x26+        adcs x15, x15, x13+        adc x17, x13, xzr+        adds x15, x15, x3+        adc x3, x17, xzr+        cmn x2, #0x1+        mul x17, x26, x26+        adcs x12, x12, x16+        adcs x16, x15, x8+        umulh x10, x26, x26+        adc x2, x3, x2+        adds x14, x14, x14+        adcs x12, x12, x12+        adcs x16, x16, x16+        adcs x2, x2, x2+        adc x15, xzr, xzr+        adds x14, x14, x7+        mul x3, x25, x25+        adcs x12, x12, x6+        lsr x7, x14, #32+        adcs x16, x16, x5+        lsl x5, x14, #32+        umulh x13, x25, x25+        adcs x2, x2, x9+        mov x6, #0xffffffff+        adc x15, x15, xzr+        adds x8, x4, x4+        adcs x1, x11, x11+        mov x11, #0xffffffff00000001+        adc x4, xzr, xzr+        subs x9, x14, x5+        sbc x14, x14, x7+        adds x12, x12, x5+        adcs x16, x16, x7+        lsl x5, x12, #32+        lsr x7, x12, #32+        adcs x2, x2, x9+        adcs x14, x15, x14+        adc x15, xzr, xzr+        subs x9, x12, x5+        sbc x12, x12, x7+        adds x16, x16, x5+        adcs x2, x2, x7+        adcs x14, x14, x9+        adcs x12, x15, x12+        adc x15, xzr, xzr+        adds x16, x16, x3+        adcs x2, x2, x13+        adcs x14, x14, x17+        adcs x12, x12, x10+        adc x15, x15, xzr+        adds x2, x2, x8+        adcs x14, x14, x1+        adcs x12, x12, x4+        adcs x15, x15, xzr+        adds x3, x16, #0x1+        sbcs x5, x2, x6+        sbcs x8, x14, xzr+        sbcs x11, x12, x11+        sbcs xzr, x15, xzr+        csel x24, x3, x16, cs+        csel x25, x8, x14, cs+        csel x26, x11, x12, cs+        csel x27, x5, x2, cs+        stp x25, x26, [sp, #112]+        stp x24, x27, [sp, #96]+        mov x0, sp+        ldr q19, [sp, #32]+        ldr q23, [sp, #48]+        ldr q0, [sp, #32]+        ldp x1, x10, [sp, #48]+        uzp2 v29.4s, v19.4s, v19.4s+        xtn v4.2s, v19.2d+        umulh x8, x19, x20+        rev64 v20.4s, v23.4s+        umull v16.2d, v19.2s, v19.2s+        umull v1.2d, v29.2s, v4.2s+        mul v20.4s, v20.4s, v0.4s+        subs x14, x19, x20+        umulh x15, x19, x1+        mov x16, v16.d[1]+        umull2 v4.2d, v19.4s, v19.4s+        mov x4, v16.d[0]+        uzp1 v17.4s, v23.4s, v0.4s+        uaddlp v19.2d, v20.4s+        lsr x7, x8, #63+        mul x11, x19, x20+        mov x12, v1.d[0]+        csetm x5, cc+        cneg x6, x14, cc+        mov x3, v4.d[1]+        mov x14, v4.d[0]+        subs x2, x10, x1+        mov x9, v1.d[1]+        cneg x17, x2, cc+        cinv x2, x5, cc+        adds x5, x4, x12, lsl #33+        extr x4, x8, x11, #63+        lsr x8, x12, #31+        uzp1 v20.4s, v0.4s, v0.4s+        shl v19.2d, v19.2d, #32+        adc x16, x16, x8+        adds x8, x14, x9, lsl #33+        lsr x14, x9, #31+        lsl x9, x5, #32+        umlal v19.2d, v20.2s, v17.2s+        adc x14, x3, x14+        adds x16, x16, x11, lsl #1+        lsr x3, x5, #32+        umulh x12, x6, x17+        adcs x4, x8, x4+        adc x11, x14, x7+        subs x8, x5, x9+        sbc x5, x5, x3+        adds x16, x16, x9+        mov x14, v19.d[0]+        mul x17, x6, x17+        adcs x3, x4, x3+        lsl x7, x16, #32+        umulh x13, x20, x10+        adcs x11, x11, x8+        lsr x8, x16, #32+        adc x5, x5, xzr+        subs x9, x16, x7+        sbc x16, x16, x8+        adds x7, x3, x7+        mov x3, v19.d[1]+        adcs x6, x11, x8+        umulh x11, x1, x10+        adcs x5, x5, x9+        eor x8, x12, x2+        adc x9, x16, xzr+        adds x16, x14, x15+        adc x15, x15, xzr+        adds x12, x16, x3+        eor x16, x17, x2+        mul x4, x1, x10+        adcs x15, x15, x13+        adc x17, x13, xzr+        adds x15, x15, x3+        adc x3, x17, xzr+        cmn x2, #0x1+        mul x17, x10, x10+        adcs x12, x12, x16+        adcs x16, x15, x8+        umulh x10, x10, x10+        adc x2, x3, x2+        adds x14, x14, x14+        adcs x12, x12, x12+        adcs x16, x16, x16+        adcs x2, x2, x2+        adc x15, xzr, xzr+        adds x14, x14, x7+        mul x3, x1, x1+        adcs x12, x12, x6+        lsr x7, x14, #32+        adcs x16, x16, x5+        lsl x5, x14, #32+        umulh x13, x1, x1+        adcs x2, x2, x9+        mov x6, #0xffffffff+        adc x15, x15, xzr+        adds x8, x4, x4+        adcs x1, x11, x11+        mov x11, #0xffffffff00000001+        adc x4, xzr, xzr+        subs x9, x14, x5+        sbc x14, x14, x7+        adds x12, x12, x5+        adcs x16, x16, x7+        lsl x5, x12, #32+        lsr x7, x12, #32+        adcs x2, x2, x9+        adcs x14, x15, x14+        adc x15, xzr, xzr+        subs x9, x12, x5+        sbc x12, x12, x7+        adds x16, x16, x5+        adcs x2, x2, x7+        adcs x14, x14, x9+        adcs x12, x15, x12+        adc x15, xzr, xzr+        adds x16, x16, x3+        adcs x2, x2, x13+        adcs x14, x14, x17+        adcs x12, x12, x10+        adc x15, x15, xzr+        adds x2, x2, x8+        adcs x14, x14, x1+        adcs x12, x12, x4+        adcs x15, x15, xzr+        adds x3, x16, #0x1+        sbcs x5, x2, x6+        sbcs x8, x14, xzr+        sbcs x11, x12, x11+        sbcs xzr, x15, xzr+        csel x16, x3, x16, cs+        csel x14, x8, x14, cs+        csel x12, x11, x12, cs+        csel x2, x5, x2, cs+        stp x14, x12, [x0, #16]+        stp x16, x2, [x0]+        ldr q20, [sp, #128]+        ldr q0, [sp, #96]+        ldp x6, x10, [sp, #128]+        rev64 v16.4s, v20.4s+        subs x4, x24, x27+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x27, x10+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x25, x24+        ldr q20, [sp, #144]+        sbcs x5, x26, x27+        ngc x17, xzr+        subs x8, x25, x26+        uaddlp v27.2d, v16.4s+        umulh x4, x24, x6+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [sp, #112]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [sp, #144]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x19, x3, x13+        adcs x20, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x25, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x26, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x19+        adcs x15, x16, x20+        eor x5, x17, x4+        adcs x9, x1, x25+        eor x1, x10, x5+        adcs x16, x2, x26+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x19, x11, x13+        and x1, x1, x13+        adcs x20, x4, x1+        and x1, x12, x13+        stp x19, x20, [sp, #128]+        adcs x4, x7, xzr+        adc x1, x17, x1+        stp x4, x1, [sp, #144]+        ldr q20, [sp, #64]+        ldr q0, [sp, #96]+        ldp x6, x10, [sp, #64]+        ldp x11, x15, [sp, #112]+        rev64 v16.4s, v20.4s+        subs x4, x24, x27+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x27, x10+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x11, x24+        ldr q20, [sp, #80]+        sbcs x5, x15, x27+        ngc x17, xzr+        subs x8, x11, x15+        uaddlp v27.2d, v16.4s+        umulh x4, x24, x6+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [sp, #112]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [sp, #80]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x24, x3, x13+        adcs x25, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x26, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x27, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x24+        adcs x15, x16, x25+        eor x5, x17, x4+        adcs x9, x1, x26+        eor x1, x10, x5+        adcs x16, x2, x27+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x9, x11, x13+        and x1, x1, x13+        adcs x10, x4, x1+        and x1, x12, x13+        stp x9, x10, [sp, #64]+        adcs x11, x7, xzr+        adc x12, x17, x1+        stp x11, x12, [sp, #80]+        mov x0, sp+        mov x1, sp+        ldp x5, x6, [x1]+        subs x5, x5, x19+        sbcs x6, x6, x20+        ldp x7, x8, [x1, #16]+        ldp x4, x3, [sp, #144]+        sbcs x7, x7, x4+        sbcs x8, x8, x3+        csetm x3, cc+        adds x24, x5, x3+        and x4, x3, #0xffffffff+        adcs x25, x6, x4+        adcs x7, x7, xzr+        and x4, x3, #0xffffffff00000001+        adc x8, x8, x4+        stp x7, x8, [x0, #16]+        subs x5, x9, x19+        sbcs x6, x10, x20+        ldp x4, x3, [sp, #144]+        sbcs x7, x11, x4+        sbcs x8, x12, x3+        csetm x3, cc+        adds x5, x5, x3+        and x4, x3, #0xffffffff+        adcs x6, x6, x4+        adcs x7, x7, xzr+        and x4, x3, #0xffffffff00000001+        adc x8, x8, x4+        stp x5, x6, [sp, #96]+        stp x7, x8, [sp, #112]+        ldr q20, [x22, #64]+        ldp x7, x17, [sp, #160]+        ldr q0, [sp, #160]+        ldp x6, x10, [x22, #64]+        ldp x11, x15, [sp, #176]+        rev64 v16.4s, v20.4s+        subs x4, x7, x17+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x17, x10+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x11, x7+        ldr q20, [x22, #80]+        sbcs x5, x15, x17+        ngc x17, xzr+        subs x8, x11, x15+        uaddlp v27.2d, v16.4s+        umulh x4, x7, x6+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [sp, #176]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [x22, #80]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x19, x3, x13+        adcs x20, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x26, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x27, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x19+        adcs x15, x16, x20+        eor x5, x17, x4+        adcs x9, x1, x26+        eor x1, x10, x5+        adcs x16, x2, x27+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x11, x11, x13+        and x1, x1, x13+        adcs x4, x4, x1+        and x1, x12, x13+        stp x11, x4, [sp, #160]+        adcs x19, x7, xzr+        adc x20, x17, x1+        stp x19, x20, [sp, #176]+        mov x0, sp+        mov x1, sp+        ldp x4, x3, [sp, #64]+        subs x5, x24, x4+        sbcs x6, x25, x3+        ldp x7, x8, [x1, #16]+        ldp x4, x3, [sp, #80]+        sbcs x7, x7, x4+        sbcs x8, x8, x3+        csetm x3, cc+        adds x9, x5, x3+        and x4, x3, #0xffffffff+        adcs x10, x6, x4+        adcs x11, x7, xzr+        and x4, x3, #0xffffffff00000001+        adc x3, x8, x4+        stp x9, x10, [x0]+        stp x11, x3, [x0, #16]+        ldp x5, x6, [sp, #128]+        subs x5, x5, x9+        sbcs x6, x6, x10+        ldp x7, x8, [sp, #144]+        sbcs x7, x7, x11+        sbcs x8, x8, x3+        csetm x3, cc+        adds x5, x5, x3+        and x4, x3, #0xffffffff+        adcs x6, x6, x4+        adcs x7, x7, xzr+        and x4, x3, #0xffffffff00000001+        adc x8, x8, x4+        stp x5, x6, [sp, #128]+        stp x7, x8, [sp, #144]+        ldr q20, [sp, #192]+        ldp x7, x17, [sp, #96]+        ldr q0, [sp, #96]+        ldp x6, x10, [sp, #192]+        ldp x11, x15, [sp, #112]+        rev64 v16.4s, v20.4s+        subs x4, x7, x17+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x17, x10+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x11, x7+        ldr q20, [sp, #208]+        sbcs x5, x15, x17+        ngc x17, xzr+        subs x8, x11, x15+        uaddlp v27.2d, v16.4s+        umulh x4, x7, x6+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [sp, #112]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [sp, #208]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x24, x3, x13+        adcs x25, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x26, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x27, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x24+        adcs x15, x16, x25+        eor x5, x17, x4+        adcs x9, x1, x26+        eor x1, x10, x5+        adcs x16, x2, x27+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x11, x11, x13+        and x1, x1, x13+        adcs x4, x4, x1+        and x1, x12, x13+        stp x11, x4, [sp, #96]+        adcs x4, x7, xzr+        adc x1, x17, x1+        stp x4, x1, [sp, #112]+        ldr q20, [x23, #64]+        ldp x7, x17, [sp, #160]+        ldr q0, [sp, #160]+        ldp x6, x10, [x23, #64]+        rev64 v16.4s, v20.4s+        subs x4, x7, x17+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x17, x10+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x19, x7+        ldr q20, [x23, #80]+        sbcs x5, x20, x17+        ngc x17, xzr+        subs x8, x19, x20+        uaddlp v27.2d, v16.4s+        umulh x4, x7, x6+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [sp, #176]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [x23, #80]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x19, x3, x13+        adcs x20, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x24, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x25, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x19+        adcs x15, x16, x20+        eor x5, x17, x4+        adcs x9, x1, x24+        eor x1, x10, x5+        adcs x16, x2, x25+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x19, x11, x13+        and x1, x1, x13+        adcs x20, x4, x1+        and x1, x12, x13+        stp x19, x20, [sp, #160]+        adcs x4, x7, xzr+        adc x1, x17, x1+        stp x4, x1, [sp, #176]+        ldr q20, [sp, #128]+        ldp x7, x17, [sp, #32]+        ldr q0, [sp, #32]+        ldp x6, x10, [sp, #128]+        ldp x11, x15, [sp, #48]+        rev64 v16.4s, v20.4s+        subs x4, x7, x17+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x17, x10+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x11, x7+        ldr q20, [sp, #144]+        sbcs x5, x15, x17+        ngc x17, xzr+        subs x8, x11, x15+        uaddlp v27.2d, v16.4s+        umulh x4, x7, x6+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [sp, #48]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [sp, #144]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x24, x3, x13+        adcs x25, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x26, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x27, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x24+        adcs x15, x16, x25+        eor x5, x17, x4+        adcs x9, x1, x26+        eor x1, x10, x5+        adcs x16, x2, x27+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x5, x11, x13+        and x1, x1, x13+        adcs x6, x4, x1+        and x1, x12, x13+        adcs x7, x7, xzr+        adc x9, x17, x1+        ldp x4, x3, [sp, #96]+        subs x5, x5, x4+        sbcs x6, x6, x3+        ldp x4, x3, [sp, #112]+        sbcs x7, x7, x4+        sbcs x8, x9, x3+        csetm x3, cc+        adds x15, x5, x3+        and x4, x3, #0xffffffff+        adcs x24, x6, x4+        adcs x25, x7, xzr+        and x4, x3, #0xffffffff00000001+        adc x26, x8, x4+        stp x15, x24, [sp, #128]+        stp x25, x26, [sp, #144]+        ldp x0, x1, [x22, #64]+        ldp x2, x3, [x22, #80]+        orr x12, x0, x1+        orr x13, x2, x3+        orr x12, x12, x13+        cmp x12, xzr+        cset x12, ne+        ldp x4, x5, [x23, #64]+        ldp x6, x7, [x23, #80]+        orr x13, x4, x5+        orr x14, x6, x7+        orr x13, x13, x14+        cmp x13, xzr+        cset x13, ne+        cmp x13, x12+        csel x8, x0, x19, cc+        csel x9, x1, x20, cc+        csel x8, x4, x8, hi+        csel x9, x5, x9, hi+        ldp x10, x11, [sp, #176]+        csel x10, x2, x10, cc+        csel x11, x3, x11, cc+        csel x10, x6, x10, hi+        csel x11, x7, x11, hi+        ldp x12, x13, [x22]+        ldp x0, x1, [sp]+        csel x0, x12, x0, cc+        csel x1, x13, x1, cc+        ldp x12, x13, [x23]+        csel x0, x12, x0, hi+        csel x1, x13, x1, hi+        ldp x12, x13, [x22, #16]+        ldp x2, x3, [sp, #16]+        csel x2, x12, x2, cc+        csel x3, x13, x3, cc+        ldp x12, x13, [x23, #16]+        csel x2, x12, x2, hi+        csel x3, x13, x3, hi+        ldp x12, x13, [x22, #32]+        csel x4, x12, x15, cc+        csel x5, x13, x24, cc+        ldp x12, x13, [x23, #32]+        csel x4, x12, x4, hi+        csel x5, x13, x5, hi+        ldp x12, x13, [x22, #48]+        csel x6, x12, x25, cc+        csel x7, x13, x26, cc+        ldp x12, x13, [x23, #48]+        csel x6, x12, x6, hi+        csel x7, x13, x7, hi+        stp x0, x1, [x21]+        stp x2, x3, [x21, #16]+        stp x4, x5, [x21, #32]+        stp x6, x7, [x21, #48]+        stp x8, x9, [x21, #64]+        stp x10, x11, [x21, #80]++        CFI_INC_SP(NSPACE)+        CFI_POP2(x27,x30)+        CFI_POP2(x25,x26)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_montjadd)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/p256_montjadd_alt.S view
@@ -0,0 +1,555 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Point addition on NIST curve P-256 in Montgomery-Jacobian coordinates+//+//    extern void p256_montjadd_alt(uint64_t p3[static 12],+//                                  const uint64_t p1[static 12],+//                                  const uint64_t p2[static 12]);+//+// Does p3 := p1 + p2 where all points are regarded as Jacobian triples with+// each coordinate in the Montgomery domain, i.e. x' = (2^256 * x) mod p_256.+// A Jacobian triple (x',y',z') represents affine point (x/z^2,y/z^3).+//+// Standard ARM ABI: X0 = p3, X1 = p1, X2 = p2+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_montjadd_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_montjadd_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_montjadd_alt)++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Stable homes for input arguments during main code sequence++#define input_z x15+#define input_x x16+#define input_y x17++// Pointer-offset pairs for inputs and outputs++#define x_1 input_x, #0+#define y_1 input_x, #NUMSIZE+#define z_1 input_x, #(2*NUMSIZE)++#define x_2 input_y, #0+#define y_2 input_y, #NUMSIZE+#define z_2 input_y, #(2*NUMSIZE)++#define x_3 input_z, #0+#define y_3 input_z, #NUMSIZE+#define z_3 input_z, #(2*NUMSIZE)++// Pointer-offset pairs for temporaries, with some aliasing+// #NSPACE is the total stack needed for these temporaries++#define z1sq sp, #(NUMSIZE*0)+#define ww sp, #(NUMSIZE*0)+#define resx sp, #(NUMSIZE*0)++#define yd sp, #(NUMSIZE*1)+#define y2a sp, #(NUMSIZE*1)++#define x2a sp, #(NUMSIZE*2)+#define zzx2 sp, #(NUMSIZE*2)++#define zz sp, #(NUMSIZE*3)+#define t1 sp, #(NUMSIZE*3)++#define t2 sp, #(NUMSIZE*4)+#define x1a sp, #(NUMSIZE*4)+#define zzx1 sp, #(NUMSIZE*4)+#define resy sp, #(NUMSIZE*4)++#define xd sp, #(NUMSIZE*5)+#define z2sq sp, #(NUMSIZE*5)+#define resz sp, #(NUMSIZE*5)++#define y1a sp, #(NUMSIZE*6)++#define NSPACE NUMSIZE*7++// Corresponds to bignum_montmul_p256_alt except registers++#define montmul_p256(P0,P1,P2)                  \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x7, x8, [P2] __LF                  \+        mul     x12, x3, x7 __LF                   \+        umulh   x13, x3, x7 __LF                   \+        mul     x11, x3, x8 __LF                   \+        umulh   x14, x3, x8 __LF                   \+        adds    x13, x13, x11 __LF                 \+        ldp     x9, x10, [P2+16] __LF              \+        mul     x11, x3, x9 __LF                   \+        umulh   x0, x3, x9 __LF                    \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x3, x10 __LF                  \+        umulh   x1, x3, x10 __LF                   \+        adcs    x0, x0, x11 __LF                   \+        adc     x1, x1, xzr __LF                   \+        ldp     x5, x6, [P1+16] __LF               \+        mul     x11, x4, x7 __LF                   \+        adds    x13, x13, x11 __LF                 \+        mul     x11, x4, x8 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x4, x9 __LF                   \+        adcs    x0, x0, x11 __LF                   \+        mul     x11, x4, x10 __LF                  \+        adcs    x1, x1, x11 __LF                   \+        umulh   x3, x4, x10 __LF                   \+        adc     x3, x3, xzr __LF                   \+        umulh   x11, x4, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        umulh   x11, x4, x8 __LF                   \+        adcs    x0, x0, x11 __LF                   \+        umulh   x11, x4, x9 __LF                   \+        adcs    x1, x1, x11 __LF                   \+        adc     x3, x3, xzr __LF                   \+        mul     x11, x5, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        mul     x11, x5, x8 __LF                   \+        adcs    x0, x0, x11 __LF                   \+        mul     x11, x5, x9 __LF                   \+        adcs    x1, x1, x11 __LF                   \+        mul     x11, x5, x10 __LF                  \+        adcs    x3, x3, x11 __LF                   \+        umulh   x4, x5, x10 __LF                   \+        adc     x4, x4, xzr __LF                   \+        umulh   x11, x5, x7 __LF                   \+        adds    x0, x0, x11 __LF                   \+        umulh   x11, x5, x8 __LF                   \+        adcs    x1, x1, x11 __LF                   \+        umulh   x11, x5, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        adc     x4, x4, xzr __LF                   \+        mul     x11, x6, x7 __LF                   \+        adds    x0, x0, x11 __LF                   \+        mul     x11, x6, x8 __LF                   \+        adcs    x1, x1, x11 __LF                   \+        mul     x11, x6, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        mul     x11, x6, x10 __LF                  \+        adcs    x4, x4, x11 __LF                   \+        umulh   x5, x6, x10 __LF                   \+        adc     x5, x5, xzr __LF                   \+        mov     x10, #0xffffffff00000001 __LF      \+        adds    x13, x13, x12, lsl #32 __LF        \+        lsr     x11, x12, #32 __LF                 \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x12, x10 __LF                 \+        umulh   x12, x12, x10 __LF                 \+        adcs    x0, x0, x11 __LF                   \+        adc     x12, x12, xzr __LF                 \+        umulh   x11, x6, x7 __LF                   \+        adds    x1, x1, x11 __LF                   \+        umulh   x11, x6, x8 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        umulh   x11, x6, x9 __LF                   \+        adcs    x4, x4, x11 __LF                   \+        adc     x5, x5, xzr __LF                   \+        adds    x14, x14, x13, lsl #32 __LF        \+        lsr     x11, x13, #32 __LF                 \+        adcs    x0, x0, x11 __LF                   \+        mul     x11, x13, x10 __LF                 \+        umulh   x13, x13, x10 __LF                 \+        adcs    x12, x12, x11 __LF                 \+        adc     x13, x13, xzr __LF                 \+        adds    x0, x0, x14, lsl #32 __LF          \+        lsr     x11, x14, #32 __LF                 \+        adcs    x12, x12, x11 __LF                 \+        mul     x11, x14, x10 __LF                 \+        umulh   x14, x14, x10 __LF                 \+        adcs    x13, x13, x11 __LF                 \+        adc     x14, x14, xzr __LF                 \+        adds    x12, x12, x0, lsl #32 __LF         \+        lsr     x11, x0, #32 __LF                  \+        adcs    x13, x13, x11 __LF                 \+        mul     x11, x0, x10 __LF                  \+        umulh   x0, x0, x10 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        adc     x0, x0, xzr __LF                   \+        adds    x12, x12, x1 __LF                  \+        adcs    x13, x13, x3 __LF                  \+        adcs    x14, x14, x4 __LF                  \+        adcs    x0, x0, x5 __LF                    \+        cset    x8, cs __LF                        \+        mov     x11, #0xffffffff __LF              \+        adds    x1, x12, #0x1 __LF                 \+        sbcs    x3, x13, x11 __LF                  \+        sbcs    x4, x14, xzr __LF                  \+        sbcs    x5, x0, x10 __LF                   \+        sbcs    xzr, x8, xzr __LF                  \+        csel    x12, x12, x1, cc __LF              \+        csel    x13, x13, x3, cc __LF              \+        csel    x14, x14, x4, cc __LF              \+        csel    x0, x0, x5, cc __LF                \+        stp     x12, x13, [P0] __LF                \+        stp     x14, x0, [P0+16]++// Corresponds exactly to bignum_montsqr_p256_alt++#define montsqr_p256(P0,P1)                     \+        ldp     x2, x3, [P1] __LF                  \+        mul     x9, x2, x3 __LF                    \+        umulh   x10, x2, x3 __LF                   \+        ldp     x4, x5, [P1+16] __LF               \+        mul     x11, x2, x5 __LF                   \+        umulh   x12, x2, x5 __LF                   \+        mul     x6, x2, x4 __LF                    \+        umulh   x7, x2, x4 __LF                    \+        adds    x10, x10, x6 __LF                  \+        adcs    x11, x11, x7 __LF                  \+        mul     x6, x3, x4 __LF                    \+        umulh   x7, x3, x4 __LF                    \+        adc     x7, x7, xzr __LF                   \+        adds    x11, x11, x6 __LF                  \+        mul     x13, x4, x5 __LF                   \+        umulh   x14, x4, x5 __LF                   \+        adcs    x12, x12, x7 __LF                  \+        mul     x6, x3, x5 __LF                    \+        umulh   x7, x3, x5 __LF                    \+        adc     x7, x7, xzr __LF                   \+        adds    x12, x12, x6 __LF                  \+        adcs    x13, x13, x7 __LF                  \+        adc     x14, x14, xzr __LF                 \+        adds    x9, x9, x9 __LF                    \+        adcs    x10, x10, x10 __LF                 \+        adcs    x11, x11, x11 __LF                 \+        adcs    x12, x12, x12 __LF                 \+        adcs    x13, x13, x13 __LF                 \+        adcs    x14, x14, x14 __LF                 \+        cset    x7, cs __LF                        \+        umulh   x6, x2, x2 __LF                    \+        mul     x8, x2, x2 __LF                    \+        adds    x9, x9, x6 __LF                    \+        mul     x6, x3, x3 __LF                    \+        adcs    x10, x10, x6 __LF                  \+        umulh   x6, x3, x3 __LF                    \+        adcs    x11, x11, x6 __LF                  \+        mul     x6, x4, x4 __LF                    \+        adcs    x12, x12, x6 __LF                  \+        umulh   x6, x4, x4 __LF                    \+        adcs    x13, x13, x6 __LF                  \+        mul     x6, x5, x5 __LF                    \+        adcs    x14, x14, x6 __LF                  \+        umulh   x6, x5, x5 __LF                    \+        adc     x7, x7, x6 __LF                    \+        adds    x9, x9, x8, lsl #32 __LF           \+        lsr     x3, x8, #32 __LF                   \+        adcs    x10, x10, x3 __LF                  \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x8, x3 __LF                    \+        umulh   x8, x8, x3 __LF                    \+        adcs    x11, x11, x2 __LF                  \+        adc     x8, x8, xzr __LF                   \+        adds    x10, x10, x9, lsl #32 __LF         \+        lsr     x3, x9, #32 __LF                   \+        adcs    x11, x11, x3 __LF                  \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x9, x3 __LF                    \+        umulh   x9, x9, x3 __LF                    \+        adcs    x8, x8, x2 __LF                    \+        adc     x9, x9, xzr __LF                   \+        adds    x11, x11, x10, lsl #32 __LF        \+        lsr     x3, x10, #32 __LF                  \+        adcs    x8, x8, x3 __LF                    \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x10, x3 __LF                   \+        umulh   x10, x10, x3 __LF                  \+        adcs    x9, x9, x2 __LF                    \+        adc     x10, x10, xzr __LF                 \+        adds    x8, x8, x11, lsl #32 __LF          \+        lsr     x3, x11, #32 __LF                  \+        adcs    x9, x9, x3 __LF                    \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x11, x3 __LF                   \+        umulh   x11, x11, x3 __LF                  \+        adcs    x10, x10, x2 __LF                  \+        adc     x11, x11, xzr __LF                 \+        adds    x8, x8, x12 __LF                   \+        adcs    x9, x9, x13 __LF                   \+        adcs    x10, x10, x14 __LF                 \+        adcs    x11, x11, x7 __LF                  \+        cset    x2, cs __LF                        \+        mov     x3, #0xffffffff __LF               \+        mov     x5, #0xffffffff00000001 __LF       \+        adds    x12, x8, #0x1 __LF                 \+        sbcs    x13, x9, x3 __LF                   \+        sbcs    x14, x10, xzr __LF                 \+        sbcs    x7, x11, x5 __LF                   \+        sbcs    xzr, x2, xzr __LF                  \+        csel    x8, x8, x12, cc __LF               \+        csel    x9, x9, x13, cc __LF               \+        csel    x10, x10, x14, cc __LF             \+        csel    x11, x11, x7, cc __LF              \+        stp     x8, x9, [P0] __LF                  \+        stp     x10, x11, [P0+16]++// Almost-Montgomery variant which we use when an input to other muls+// with the other argument fully reduced (which is always safe).++#define amontsqr_p256(P0,P1)                    \+        ldp     x2, x3, [P1] __LF                  \+        mul     x9, x2, x3 __LF                    \+        umulh   x10, x2, x3 __LF                   \+        ldp     x4, x5, [P1+16] __LF               \+        mul     x11, x2, x5 __LF                   \+        umulh   x12, x2, x5 __LF                   \+        mul     x6, x2, x4 __LF                    \+        umulh   x7, x2, x4 __LF                    \+        adds    x10, x10, x6 __LF                  \+        adcs    x11, x11, x7 __LF                  \+        mul     x6, x3, x4 __LF                    \+        umulh   x7, x3, x4 __LF                    \+        adc     x7, x7, xzr __LF                   \+        adds    x11, x11, x6 __LF                  \+        mul     x13, x4, x5 __LF                   \+        umulh   x14, x4, x5 __LF                   \+        adcs    x12, x12, x7 __LF                  \+        mul     x6, x3, x5 __LF                    \+        umulh   x7, x3, x5 __LF                    \+        adc     x7, x7, xzr __LF                   \+        adds    x12, x12, x6 __LF                  \+        adcs    x13, x13, x7 __LF                  \+        adc     x14, x14, xzr __LF                 \+        adds    x9, x9, x9 __LF                    \+        adcs    x10, x10, x10 __LF                 \+        adcs    x11, x11, x11 __LF                 \+        adcs    x12, x12, x12 __LF                 \+        adcs    x13, x13, x13 __LF                 \+        adcs    x14, x14, x14 __LF                 \+        cset    x7, cs __LF                        \+        umulh   x6, x2, x2 __LF                    \+        mul     x8, x2, x2 __LF                    \+        adds    x9, x9, x6 __LF                    \+        mul     x6, x3, x3 __LF                    \+        adcs    x10, x10, x6 __LF                  \+        umulh   x6, x3, x3 __LF                    \+        adcs    x11, x11, x6 __LF                  \+        mul     x6, x4, x4 __LF                    \+        adcs    x12, x12, x6 __LF                  \+        umulh   x6, x4, x4 __LF                    \+        adcs    x13, x13, x6 __LF                  \+        mul     x6, x5, x5 __LF                    \+        adcs    x14, x14, x6 __LF                  \+        umulh   x6, x5, x5 __LF                    \+        adc     x7, x7, x6 __LF                    \+        adds    x9, x9, x8, lsl #32 __LF           \+        lsr     x3, x8, #32 __LF                   \+        adcs    x10, x10, x3 __LF                  \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x8, x3 __LF                    \+        umulh   x8, x8, x3 __LF                    \+        adcs    x11, x11, x2 __LF                  \+        adc     x8, x8, xzr __LF                   \+        adds    x10, x10, x9, lsl #32 __LF         \+        lsr     x3, x9, #32 __LF                   \+        adcs    x11, x11, x3 __LF                  \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x9, x3 __LF                    \+        umulh   x9, x9, x3 __LF                    \+        adcs    x8, x8, x2 __LF                    \+        adc     x9, x9, xzr __LF                   \+        adds    x11, x11, x10, lsl #32 __LF        \+        lsr     x3, x10, #32 __LF                  \+        adcs    x8, x8, x3 __LF                    \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x10, x3 __LF                   \+        umulh   x10, x10, x3 __LF                  \+        adcs    x9, x9, x2 __LF                    \+        adc     x10, x10, xzr __LF                 \+        adds    x8, x8, x11, lsl #32 __LF          \+        lsr     x3, x11, #32 __LF                  \+        adcs    x9, x9, x3 __LF                    \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x11, x3 __LF                   \+        umulh   x11, x11, x3 __LF                  \+        adcs    x10, x10, x2 __LF                  \+        adc     x11, x11, xzr __LF                 \+        adds    x8, x8, x12 __LF                   \+        adcs    x9, x9, x13 __LF                   \+        adcs    x10, x10, x14 __LF                 \+        adcs    x11, x11, x7 __LF                  \+        mov     x2, #0xffffffffffffffff __LF       \+        csel    x2, xzr, x2, cc __LF               \+        mov     x3, #0xffffffff __LF               \+        csel    x3, xzr, x3, cc __LF               \+        mov     x5, #0xffffffff00000001 __LF       \+        csel    x5, xzr, x5, cc __LF               \+        subs    x8, x8, x2 __LF                    \+        sbcs    x9, x9, x3 __LF                    \+        sbcs    x10, x10, xzr __LF                 \+        sbc     x11, x11, x5 __LF                  \+        stp     x8, x9, [P0] __LF                  \+        stp     x10, x11, [P0+16]++// Corresponds exactly to bignum_sub_p256++#define sub_p256(P0,P1,P2)                      \+        ldp     x5, x6, [P1] __LF                  \+        ldp     x4, x3, [P2] __LF                  \+        subs    x5, x5, x4 __LF                    \+        sbcs    x6, x6, x3 __LF                    \+        ldp     x7, x8, [P1+16] __LF               \+        ldp     x4, x3, [P2+16] __LF               \+        sbcs    x7, x7, x4 __LF                    \+        sbcs    x8, x8, x3 __LF                    \+        csetm   x3, cc __LF                        \+        adds    x5, x5, x3 __LF                    \+        mov     x4, #0xffffffff __LF               \+        and     x4, x4, x3 __LF                    \+        adcs    x6, x6, x4 __LF                    \+        adcs    x7, x7, xzr __LF                   \+        mov     x4, #0xffffffff00000001 __LF       \+        and     x4, x4, x3 __LF                    \+        adc     x8, x8, x4 __LF                    \+        stp     x5, x6, [P0] __LF                  \+        stp     x7, x8, [P0+16]++S2N_BN_SYMBOL(p256_montjadd_alt):+        CFI_START++// Make room on stack for temporary variables+// Move the input arguments to stable places++        CFI_DEC_SP(NSPACE)++        mov     input_z, x0+        mov     input_x, x1+        mov     input_y, x2++// Main code, just a sequence of basic field operations+// 12 * multiply + 4 * square + 7 * subtract++        amontsqr_p256(z1sq,z_1)+        amontsqr_p256(z2sq,z_2)++        montmul_p256(y1a,z_2,y_1)+        montmul_p256(y2a,z_1,y_2)++        montmul_p256(x2a,z1sq,x_2)+        montmul_p256(x1a,z2sq,x_1)+        montmul_p256(y2a,z1sq,y2a)+        montmul_p256(y1a,z2sq,y1a)++        sub_p256(xd,x2a,x1a)+        sub_p256(yd,y2a,y1a)++        amontsqr_p256(zz,xd)+        montsqr_p256(ww,yd)++        montmul_p256(zzx1,zz,x1a)+        montmul_p256(zzx2,zz,x2a)++        sub_p256(resx,ww,zzx1)+        sub_p256(t1,zzx2,zzx1)++        montmul_p256(xd,xd,z_1)++        sub_p256(resx,resx,zzx2)++        sub_p256(t2,zzx1,resx)++        montmul_p256(t1,t1,y1a)+        montmul_p256(resz,xd,z_2)+        montmul_p256(t2,yd,t2)++        sub_p256(resy,t2,t1)++// Load in the z coordinates of the inputs to check for P1 = 0 and P2 = 0+// The condition codes get set by a comparison (P2 != 0) - (P1 != 0)+// So  "HI" <=> CF /\ ~ZF <=> P1 = 0 /\ ~(P2 = 0)+// and "LO" <=> ~CF       <=> ~(P1 = 0) /\ P2 = 0++        ldp     x0, x1, [z_1]+        ldp     x2, x3, [z_1+16]++        orr     x12, x0, x1+        orr     x13, x2, x3+        orr     x12, x12, x13+        cmp     x12, xzr+        cset    x12, ne++        ldp     x4, x5, [z_2]+        ldp     x6, x7, [z_2+16]++        orr     x13, x4, x5+        orr     x14, x6, x7+        orr     x13, x13, x14+        cmp     x13, xzr+        cset    x13, ne++        cmp     x13, x12++// Multiplex the outputs accordingly, re-using the z's in registers++        ldp     x8, x9, [resz]+        csel    x8, x0, x8, lo+        csel    x9, x1, x9, lo+        csel    x8, x4, x8, hi+        csel    x9, x5, x9, hi+        ldp     x10, x11, [resz+16]+        csel    x10, x2, x10, lo+        csel    x11, x3, x11, lo+        csel    x10, x6, x10, hi+        csel    x11, x7, x11, hi++        ldp     x12, x13, [x_1]+        ldp     x0, x1, [resx]+        csel    x0, x12, x0, lo+        csel    x1, x13, x1, lo+        ldp     x12, x13, [x_2]+        csel    x0, x12, x0, hi+        csel    x1, x13, x1, hi++        ldp     x12, x13, [x_1+16]+        ldp     x2, x3, [resx+16]+        csel    x2, x12, x2, lo+        csel    x3, x13, x3, lo+        ldp     x12, x13, [x_2+16]+        csel    x2, x12, x2, hi+        csel    x3, x13, x3, hi++        ldp     x12, x13, [y_1]+        ldp     x4, x5, [resy]+        csel    x4, x12, x4, lo+        csel    x5, x13, x5, lo+        ldp     x12, x13, [y_2]+        csel    x4, x12, x4, hi+        csel    x5, x13, x5, hi++        ldp     x12, x13, [y_1+16]+        ldp     x6, x7, [resy+16]+        csel    x6, x12, x6, lo+        csel    x7, x13, x7, lo+        ldp     x12, x13, [y_2+16]+        csel    x6, x12, x6, hi+        csel    x7, x13, x7, hi++// Finally store back the multiplexed values++        stp     x0, x1, [x_3]+        stp     x2, x3, [x_3+16]+        stp     x4, x5, [y_3]+        stp     x6, x7, [y_3+16]+        stp     x8, x9, [z_3]+        stp     x10, x11, [z_3+16]++// Restore registers and return++        CFI_INC_SP(NSPACE)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_montjadd_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/p256_montjdouble.S view
@@ -0,0 +1,1555 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Point doubling on NIST curve P-256 in Montgomery-Jacobian coordinates+//+//    extern void p256_montjdouble(uint64_t p3[static 12],+//                                 const uint64_t p1[static 12]);+//+// Does p3 := 2 * p1 where all points are regarded as Jacobian triples with+// each coordinate in the Montgomery domain, i.e. x' = (2^256 * x) mod p_256.+// A Jacobian triple (x',y',z') represents affine point (x/z^2,y/z^3).+//+// Standard ARM ABI: X0 = p3, X1 = p1+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++// This is functionally equivalent to p256_montjdouble in unopt/p256_montjdouble.S.+// This is the result of doing the following sequence of optimizations:+//   1. Function inlining+//   2. Eliminating redundant load/store instructions+//   3. Folding (add addr, const) + load/store+// Function inlining is done manually. The second and third optimizations are+// done by a script.++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_montjdouble)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_montjdouble)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_montjdouble)+        .text+        .balign 4++#define NUMSIZE 32+#define NSPACE NUMSIZE*6++S2N_BN_SYMBOL(p256_montjdouble):+        CFI_START++        CFI_DEC_SP(NSPACE+80)+        CFI_STACKSAVE2(x19,x20,NSPACE)+        CFI_STACKSAVE2(x21,x22,NSPACE+16)+        CFI_STACKSAVE2(x23,x24,NSPACE+32)+        CFI_STACKSAVE2(x25,x26,NSPACE+48)+        CFI_STACKSAVE1Z(x27,NSPACE+64)++        mov x19, x0+        mov x20, x1+        mov x0, sp+        ldr q19, [x20, #64]+        ldp x9, x13, [x20, #64]+        ldr q23, [x20, #80]+        ldr q0, [x20, #64]+        ldp x1, x10, [x20, #80]+        uzp2 v29.4s, v19.4s, v19.4s+        xtn v4.2s, v19.2d+        umulh x8, x9, x13+        rev64 v20.4s, v23.4s+        umull v16.2d, v19.2s, v19.2s+        umull v1.2d, v29.2s, v4.2s+        mul v20.4s, v20.4s, v0.4s+        subs x14, x9, x13+        umulh x15, x9, x1+        mov x16, v16.d[1]+        umull2 v4.2d, v19.4s, v19.4s+        mov x4, v16.d[0]+        uzp1 v17.4s, v23.4s, v0.4s+        uaddlp v19.2d, v20.4s+        lsr x7, x8, #63+        mul x11, x9, x13+        mov x12, v1.d[0]+        csetm x5, cc+        cneg x6, x14, cc+        mov x3, v4.d[1]+        mov x14, v4.d[0]+        subs x2, x10, x1+        mov x9, v1.d[1]+        cneg x17, x2, cc+        cinv x2, x5, cc+        adds x5, x4, x12, lsl #33+        extr x4, x8, x11, #63+        lsr x8, x12, #31+        uzp1 v20.4s, v0.4s, v0.4s+        shl v19.2d, v19.2d, #32+        adc x16, x16, x8+        adds x8, x14, x9, lsl #33+        lsr x14, x9, #31+        lsl x9, x5, #32+        umlal v19.2d, v20.2s, v17.2s+        adc x14, x3, x14+        adds x16, x16, x11, lsl #1+        lsr x3, x5, #32+        umulh x12, x6, x17+        adcs x4, x8, x4+        adc x11, x14, x7+        subs x8, x5, x9+        sbc x5, x5, x3+        adds x16, x16, x9+        mov x14, v19.d[0]+        mul x17, x6, x17+        adcs x3, x4, x3+        lsl x7, x16, #32+        umulh x13, x13, x10+        adcs x11, x11, x8+        lsr x8, x16, #32+        adc x5, x5, xzr+        subs x9, x16, x7+        sbc x16, x16, x8+        adds x7, x3, x7+        mov x3, v19.d[1]+        adcs x6, x11, x8+        umulh x11, x1, x10+        adcs x5, x5, x9+        eor x8, x12, x2+        adc x9, x16, xzr+        adds x16, x14, x15+        adc x15, x15, xzr+        adds x12, x16, x3+        eor x16, x17, x2+        mul x4, x1, x10+        adcs x15, x15, x13+        adc x17, x13, xzr+        adds x15, x15, x3+        adc x3, x17, xzr+        cmn x2, #0x1+        mul x17, x10, x10+        adcs x12, x12, x16+        adcs x16, x15, x8+        umulh x10, x10, x10+        adc x2, x3, x2+        adds x14, x14, x14+        adcs x12, x12, x12+        adcs x16, x16, x16+        adcs x2, x2, x2+        adc x15, xzr, xzr+        adds x14, x14, x7+        mul x3, x1, x1+        adcs x12, x12, x6+        lsr x7, x14, #32+        adcs x16, x16, x5+        lsl x5, x14, #32+        umulh x13, x1, x1+        adcs x2, x2, x9+        mov x6, #0xffffffff+        adc x15, x15, xzr+        adds x8, x4, x4+        adcs x1, x11, x11+        mov x11, #0xffffffff00000001+        adc x4, xzr, xzr+        subs x9, x14, x5+        sbc x14, x14, x7+        adds x12, x12, x5+        adcs x16, x16, x7+        lsl x5, x12, #32+        lsr x7, x12, #32+        adcs x2, x2, x9+        adcs x14, x15, x14+        adc x15, xzr, xzr+        subs x9, x12, x5+        sbc x12, x12, x7+        adds x16, x16, x5+        adcs x2, x2, x7+        adcs x14, x14, x9+        adcs x12, x15, x12+        adc x15, xzr, xzr+        adds x16, x16, x3+        adcs x2, x2, x13+        adcs x14, x14, x17+        adcs x12, x12, x10+        adc x15, x15, xzr+        adds x2, x2, x8+        adcs x14, x14, x1+        adcs x12, x12, x4+        adcs x15, x15, xzr+        adds x3, x16, #0x1+        sbcs x5, x2, x6+        sbcs x8, x14, xzr+        sbcs x11, x12, x11+        sbcs xzr, x15, xzr+        csel x21, x3, x16, cs+        csel x22, x8, x14, cs+        csel x23, x11, x12, cs+        csel x24, x5, x2, cs+        stp x22, x23, [x0, #16]+        stp x21, x24, [x0]+        ldr q19, [x20, #32]+        ldp x9, x13, [x20, #32]+        ldr q23, [x20, #48]+        ldr q0, [x20, #32]+        ldp x1, x10, [x20, #48]+        uzp2 v29.4s, v19.4s, v19.4s+        xtn v4.2s, v19.2d+        umulh x8, x9, x13+        rev64 v20.4s, v23.4s+        umull v16.2d, v19.2s, v19.2s+        umull v1.2d, v29.2s, v4.2s+        mul v20.4s, v20.4s, v0.4s+        subs x14, x9, x13+        umulh x15, x9, x1+        mov x16, v16.d[1]+        umull2 v4.2d, v19.4s, v19.4s+        mov x4, v16.d[0]+        uzp1 v17.4s, v23.4s, v0.4s+        uaddlp v19.2d, v20.4s+        lsr x7, x8, #63+        mul x11, x9, x13+        mov x12, v1.d[0]+        csetm x5, cc+        cneg x6, x14, cc+        mov x3, v4.d[1]+        mov x14, v4.d[0]+        subs x2, x10, x1+        mov x9, v1.d[1]+        cneg x17, x2, cc+        cinv x2, x5, cc+        adds x5, x4, x12, lsl #33+        extr x4, x8, x11, #63+        lsr x8, x12, #31+        uzp1 v20.4s, v0.4s, v0.4s+        shl v19.2d, v19.2d, #32+        adc x16, x16, x8+        adds x8, x14, x9, lsl #33+        lsr x14, x9, #31+        lsl x9, x5, #32+        umlal v19.2d, v20.2s, v17.2s+        adc x14, x3, x14+        adds x16, x16, x11, lsl #1+        lsr x3, x5, #32+        umulh x12, x6, x17+        adcs x4, x8, x4+        adc x11, x14, x7+        subs x8, x5, x9+        sbc x5, x5, x3+        adds x16, x16, x9+        mov x14, v19.d[0]+        mul x17, x6, x17+        adcs x3, x4, x3+        lsl x7, x16, #32+        umulh x13, x13, x10+        adcs x11, x11, x8+        lsr x8, x16, #32+        adc x5, x5, xzr+        subs x9, x16, x7+        sbc x16, x16, x8+        adds x7, x3, x7+        mov x3, v19.d[1]+        adcs x6, x11, x8+        umulh x11, x1, x10+        adcs x5, x5, x9+        eor x8, x12, x2+        adc x9, x16, xzr+        adds x16, x14, x15+        adc x15, x15, xzr+        adds x12, x16, x3+        eor x16, x17, x2+        mul x4, x1, x10+        adcs x15, x15, x13+        adc x17, x13, xzr+        adds x15, x15, x3+        adc x3, x17, xzr+        cmn x2, #0x1+        mul x17, x10, x10+        adcs x12, x12, x16+        adcs x16, x15, x8+        umulh x10, x10, x10+        adc x2, x3, x2+        adds x14, x14, x14+        adcs x12, x12, x12+        adcs x16, x16, x16+        adcs x2, x2, x2+        adc x15, xzr, xzr+        adds x14, x14, x7+        mul x3, x1, x1+        adcs x12, x12, x6+        lsr x7, x14, #32+        adcs x16, x16, x5+        lsl x5, x14, #32+        umulh x13, x1, x1+        adcs x2, x2, x9+        mov x6, #0xffffffff+        adc x15, x15, xzr+        adds x8, x4, x4+        adcs x1, x11, x11+        mov x11, #0xffffffff00000001+        adc x4, xzr, xzr+        subs x9, x14, x5+        sbc x14, x14, x7+        adds x12, x12, x5+        adcs x16, x16, x7+        lsl x5, x12, #32+        lsr x7, x12, #32+        adcs x2, x2, x9+        adcs x14, x15, x14+        adc x15, xzr, xzr+        subs x9, x12, x5+        sbc x12, x12, x7+        adds x16, x16, x5+        adcs x2, x2, x7+        adcs x14, x14, x9+        adcs x12, x15, x12+        adc x15, xzr, xzr+        adds x16, x16, x3+        adcs x2, x2, x13+        adcs x14, x14, x17+        adcs x12, x12, x10+        adc x15, x15, xzr+        adds x2, x2, x8+        adcs x14, x14, x1+        adcs x12, x12, x4+        adcs x15, x15, xzr+        adds x3, x16, #0x1+        sbcs x5, x2, x6+        sbcs x8, x14, xzr+        sbcs x11, x12, x11+        sbcs xzr, x15, xzr+        csel x16, x3, x16, cs+        csel x14, x8, x14, cs+        csel x12, x11, x12, cs+        csel x2, x5, x2, cs+        stp x14, x12, [sp, #48]+        stp x16, x2, [sp, #32]+        ldp x5, x6, [x20, #0]+        subs x5, x5, x21+        sbcs x6, x6, x24+        ldp x7, x8, [x20, #16]+        sbcs x7, x7, x22+        sbcs x8, x8, x23+        csetm x3, cc+        adds x10, x5, x3+        and x4, x3, #0xffffffff+        adcs x25, x6, x4+        adcs x26, x7, xzr+        and x4, x3, #0xffffffff00000001+        adc x27, x8, x4+        stp x10, x25, [sp, #96]+        stp x26, x27, [sp, #112]+        ldp x5, x6, [x20]+        adds x5, x5, x21+        adcs x6, x6, x24+        ldp x7, x8, [x20, #16]+        adcs x7, x7, x22+        adcs x8, x8, x23+        csetm x3, cs+        subs x9, x5, x3+        and x1, x3, #0xffffffff+        sbcs x5, x6, x1+        sbcs x7, x7, xzr+        and x2, x3, #0xffffffff00000001+        sbc x8, x8, x2+        stp x9, x5, [sp, #64]+        stp x7, x8, [sp, #80]+        ldr q20, [sp, #96]+        ldr q0, [sp, #64]+        rev64 v16.4s, v20.4s+        subs x4, x9, x5+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x5, x25+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x7, x9+        ldr q20, [sp, #112]+        sbcs x5, x8, x5+        ngc x17, xzr+        subs x8, x7, x8+        uaddlp v27.2d, v16.4s+        umulh x4, x9, x10+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x25, x10+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [sp, #80]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x10, x26+        sbcs x9, x25, x27+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x27, x26+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x21, x3, x13+        adcs x22, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x23, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x24, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x21+        adcs x15, x16, x22+        eor x5, x17, x4+        adcs x9, x1, x23+        eor x1, x10, x5+        adcs x16, x2, x24+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x21, x11, x13+        and x1, x1, x13+        adcs x22, x4, x1+        and x1, x12, x13+        stp x21, x22, [sp, #96]+        adcs x23, x7, xzr+        adc x24, x17, x1+        stp x23, x24, [sp, #112]+        ldp x4, x5, [x20, #32]+        ldp x8, x9, [x20, #64]+        adds x4, x4, x8+        adcs x5, x5, x9+        ldp x6, x7, [x20, #48]+        ldp x10, x11, [x20, #80]+        adcs x6, x6, x10+        adcs x7, x7, x11+        adc x3, xzr, xzr+        adds x8, x4, #0x1+        mov x9, #0xffffffff+        sbcs x9, x5, x9+        sbcs x10, x6, xzr+        mov x11, #0xffffffff00000001+        sbcs x11, x7, x11+        sbcs x3, x3, xzr+        csel x4, x4, x8, cc+        csel x5, x5, x9, cc+        csel x6, x6, x10, cc+        csel x7, x7, x11, cc+        stp x4, x5, [sp, #64]+        stp x6, x7, [sp, #80]+        ldr q20, [sp, #32]+        ldp x7, x17, [x20, #0]+        ldr q0, [x20, #0]+        ldp x6, x10, [sp, #32]+        ldp x11, x15, [x20, #16]+        rev64 v16.4s, v20.4s+        subs x4, x7, x17+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x17, x10+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x11, x7+        ldr q20, [sp, #48]+        sbcs x5, x15, x17+        ngc x17, xzr+        subs x8, x11, x15+        uaddlp v27.2d, v16.4s+        umulh x4, x7, x6+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [x20, #16]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [sp, #48]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x20, x3, x13+        adcs x25, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x26, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x27, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x20+        adcs x15, x16, x25+        eor x5, x17, x4+        adcs x9, x1, x26+        eor x1, x10, x5+        adcs x16, x2, x27+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x20, x11, x13+        and x1, x1, x13+        adcs x25, x4, x1+        and x1, x12, x13+        stp x20, x25, [sp, #128]+        adcs x4, x7, xzr+        adc x1, x17, x1+        stp x4, x1, [sp, #144]+        ldr q19, [sp, #96]+        ldr q23, [sp, #112]+        ldr q0, [sp, #96]+        uzp2 v29.4s, v19.4s, v19.4s+        xtn v4.2s, v19.2d+        umulh x8, x21, x22+        rev64 v20.4s, v23.4s+        umull v16.2d, v19.2s, v19.2s+        umull v1.2d, v29.2s, v4.2s+        mul v20.4s, v20.4s, v0.4s+        subs x14, x21, x22+        umulh x15, x21, x23+        mov x16, v16.d[1]+        umull2 v4.2d, v19.4s, v19.4s+        mov x4, v16.d[0]+        uzp1 v17.4s, v23.4s, v0.4s+        uaddlp v19.2d, v20.4s+        lsr x7, x8, #63+        mul x11, x21, x22+        mov x12, v1.d[0]+        csetm x5, cc+        cneg x6, x14, cc+        mov x3, v4.d[1]+        mov x14, v4.d[0]+        subs x2, x24, x23+        mov x9, v1.d[1]+        cneg x17, x2, cc+        cinv x2, x5, cc+        adds x5, x4, x12, lsl #33+        extr x4, x8, x11, #63+        lsr x8, x12, #31+        uzp1 v20.4s, v0.4s, v0.4s+        shl v19.2d, v19.2d, #32+        adc x16, x16, x8+        adds x8, x14, x9, lsl #33+        lsr x14, x9, #31+        lsl x9, x5, #32+        umlal v19.2d, v20.2s, v17.2s+        adc x14, x3, x14+        adds x16, x16, x11, lsl #1+        lsr x3, x5, #32+        umulh x12, x6, x17+        adcs x4, x8, x4+        adc x11, x14, x7+        subs x8, x5, x9+        sbc x5, x5, x3+        adds x16, x16, x9+        mov x14, v19.d[0]+        mul x17, x6, x17+        adcs x3, x4, x3+        lsl x7, x16, #32+        umulh x13, x22, x24+        adcs x11, x11, x8+        lsr x8, x16, #32+        adc x5, x5, xzr+        subs x9, x16, x7+        sbc x16, x16, x8+        adds x7, x3, x7+        mov x3, v19.d[1]+        adcs x6, x11, x8+        umulh x11, x23, x24+        adcs x5, x5, x9+        eor x8, x12, x2+        adc x9, x16, xzr+        adds x16, x14, x15+        adc x15, x15, xzr+        adds x12, x16, x3+        eor x16, x17, x2+        mul x4, x23, x24+        adcs x15, x15, x13+        adc x17, x13, xzr+        adds x15, x15, x3+        adc x3, x17, xzr+        cmn x2, #0x1+        mul x17, x24, x24+        adcs x12, x12, x16+        adcs x16, x15, x8+        umulh x10, x24, x24+        adc x2, x3, x2+        adds x14, x14, x14+        adcs x12, x12, x12+        adcs x16, x16, x16+        adcs x2, x2, x2+        adc x15, xzr, xzr+        adds x14, x14, x7+        mul x3, x23, x23+        adcs x12, x12, x6+        lsr x7, x14, #32+        adcs x16, x16, x5+        lsl x5, x14, #32+        umulh x13, x23, x23+        adcs x2, x2, x9+        mov x6, #0xffffffff+        adc x15, x15, xzr+        adds x8, x4, x4+        adcs x1, x11, x11+        mov x11, #0xffffffff00000001+        adc x4, xzr, xzr+        subs x9, x14, x5+        sbc x14, x14, x7+        adds x12, x12, x5+        adcs x16, x16, x7+        lsl x5, x12, #32+        lsr x7, x12, #32+        adcs x2, x2, x9+        adcs x14, x15, x14+        adc x15, xzr, xzr+        subs x9, x12, x5+        sbc x12, x12, x7+        adds x16, x16, x5+        adcs x2, x2, x7+        adcs x14, x14, x9+        adcs x12, x15, x12+        adc x15, xzr, xzr+        adds x16, x16, x3+        adcs x2, x2, x13+        adcs x14, x14, x17+        adcs x12, x12, x10+        adc x15, x15, xzr+        adds x2, x2, x8+        adcs x14, x14, x1+        adcs x12, x12, x4+        adcs x15, x15, xzr+        adds x3, x16, #0x1+        sbcs x5, x2, x6+        sbcs x8, x14, xzr+        sbcs x11, x12, x11+        sbcs xzr, x15, xzr+        csel x21, x3, x16, cs+        csel x22, x8, x14, cs+        csel x23, x11, x12, cs+        csel x24, x5, x2, cs+        ldr q19, [sp, #64]+        ldp x9, x13, [sp, #64]+        ldr q23, [sp, #80]+        ldr q0, [sp, #64]+        ldp x1, x10, [sp, #80]+        uzp2 v29.4s, v19.4s, v19.4s+        xtn v4.2s, v19.2d+        umulh x8, x9, x13+        rev64 v20.4s, v23.4s+        umull v16.2d, v19.2s, v19.2s+        umull v1.2d, v29.2s, v4.2s+        mul v20.4s, v20.4s, v0.4s+        subs x14, x9, x13+        umulh x15, x9, x1+        mov x16, v16.d[1]+        umull2 v4.2d, v19.4s, v19.4s+        mov x4, v16.d[0]+        uzp1 v17.4s, v23.4s, v0.4s+        uaddlp v19.2d, v20.4s+        lsr x7, x8, #63+        mul x11, x9, x13+        mov x12, v1.d[0]+        csetm x5, cc+        cneg x6, x14, cc+        mov x3, v4.d[1]+        mov x14, v4.d[0]+        subs x2, x10, x1+        mov x9, v1.d[1]+        cneg x17, x2, cc+        cinv x2, x5, cc+        adds x5, x4, x12, lsl #33+        extr x4, x8, x11, #63+        lsr x8, x12, #31+        uzp1 v20.4s, v0.4s, v0.4s+        shl v19.2d, v19.2d, #32+        adc x16, x16, x8+        adds x8, x14, x9, lsl #33+        lsr x14, x9, #31+        lsl x9, x5, #32+        umlal v19.2d, v20.2s, v17.2s+        adc x14, x3, x14+        adds x16, x16, x11, lsl #1+        lsr x3, x5, #32+        umulh x12, x6, x17+        adcs x4, x8, x4+        adc x11, x14, x7+        subs x8, x5, x9+        sbc x5, x5, x3+        adds x16, x16, x9+        mov x14, v19.d[0]+        mul x17, x6, x17+        adcs x3, x4, x3+        lsl x7, x16, #32+        umulh x13, x13, x10+        adcs x11, x11, x8+        lsr x8, x16, #32+        adc x5, x5, xzr+        subs x9, x16, x7+        sbc x16, x16, x8+        adds x7, x3, x7+        mov x3, v19.d[1]+        adcs x6, x11, x8+        umulh x11, x1, x10+        adcs x5, x5, x9+        eor x8, x12, x2+        adc x9, x16, xzr+        adds x16, x14, x15+        adc x15, x15, xzr+        adds x12, x16, x3+        eor x16, x17, x2+        mul x4, x1, x10+        adcs x15, x15, x13+        adc x17, x13, xzr+        adds x15, x15, x3+        adc x3, x17, xzr+        cmn x2, #0x1+        mul x17, x10, x10+        adcs x12, x12, x16+        adcs x16, x15, x8+        umulh x10, x10, x10+        adc x2, x3, x2+        adds x14, x14, x14+        adcs x12, x12, x12+        adcs x16, x16, x16+        adcs x2, x2, x2+        adc x15, xzr, xzr+        adds x14, x14, x7+        mul x3, x1, x1+        adcs x12, x12, x6+        lsr x7, x14, #32+        adcs x16, x16, x5+        lsl x5, x14, #32+        umulh x13, x1, x1+        adcs x2, x2, x9+        mov x6, #0xffffffff+        adc x15, x15, xzr+        adds x8, x4, x4+        adcs x1, x11, x11+        mov x11, #0xffffffff00000001+        adc x4, xzr, xzr+        subs x9, x14, x5+        sbc x14, x14, x7+        adds x12, x12, x5+        adcs x16, x16, x7+        lsl x5, x12, #32+        lsr x7, x12, #32+        adcs x2, x2, x9+        adcs x14, x15, x14+        adc x15, xzr, xzr+        subs x9, x12, x5+        sbc x12, x12, x7+        adds x16, x16, x5+        adcs x2, x2, x7+        adcs x14, x14, x9+        adcs x12, x15, x12+        adc x15, xzr, xzr+        adds x16, x16, x3+        adcs x2, x2, x13+        adcs x14, x14, x17+        adcs x12, x12, x10+        adc x15, x15, xzr+        adds x2, x2, x8+        adcs x14, x14, x1+        adcs x12, x12, x4+        adcs x15, x15, xzr+        adds x3, x16, #0x1+        sbcs x5, x2, x6+        sbcs x8, x14, xzr+        sbcs x11, x12, x11+        sbcs xzr, x15, xzr+        csel x13, x3, x16, cs+        csel x14, x8, x14, cs+        csel x15, x11, x12, cs+        csel x26, x5, x2, cs+        mov x1, #0x9+        mov x2, #0xffffffffffffffff+        subs x9, x2, x21+        mov x2, #0xffffffff+        sbcs x10, x2, x24+        ngcs x11, x22+        mov x2, #0xffffffff00000001+        sbc x12, x2, x23+        mul x3, x1, x9+        mul x4, x1, x10+        mul x5, x1, x11+        mul x6, x1, x12+        umulh x9, x1, x9+        umulh x10, x1, x10+        umulh x11, x1, x11+        umulh x7, x1, x12+        adds x4, x4, x9+        adcs x5, x5, x10+        adcs x6, x6, x11+        adc x7, x7, xzr+        mov x1, #0xc+        mul x8, x20, x1+        umulh x9, x20, x1+        adds x3, x3, x8+        mul x8, x25, x1+        umulh x10, x25, x1+        adcs x4, x4, x8+        ldp x11, x12, [sp, #144]+        mul x8, x11, x1+        umulh x11, x11, x1+        adcs x5, x5, x8+        mul x8, x12, x1+        umulh x12, x12, x1+        adcs x6, x6, x8+        adc x7, x7, xzr+        adds x4, x4, x9+        adcs x5, x5, x10+        adcs x6, x6, x11+        adc x7, x7, x12+        add x8, x7, #0x1+        lsl x10, x8, #32+        adds x6, x6, x10+        adc x7, x7, xzr+        neg x9, x8+        sub x10, x10, #0x1+        subs x3, x3, x9+        sbcs x4, x4, x10+        sbcs x5, x5, xzr+        sbcs x6, x6, x8+        sbc x8, x7, x8+        adds x20, x3, x8+        and x9, x8, #0xffffffff+        adcs x21, x4, x9+        adcs x22, x5, xzr+        neg x10, x9+        adc x23, x6, x10+        stp x20, x21, [sp, #160]+        stp x22, x23, [sp, #176]+        mov x2, sp+        ldp x4, x3, [x2]+        subs x5, x13, x4+        sbcs x6, x26, x3+        ldp x4, x3, [x2, #16]+        sbcs x7, x14, x4+        sbcs x8, x15, x3+        csetm x3, cc+        adds x5, x5, x3+        and x4, x3, #0xffffffff+        adcs x6, x6, x4+        adcs x7, x7, xzr+        and x4, x3, #0xffffffff00000001+        adc x8, x8, x4+        stp x5, x6, [sp, #64]+        stp x7, x8, [sp, #80]+        mov x0, sp+        ldr q19, [sp, #32]+        ldp x9, x13, [sp, #32]+        ldr q23, [sp, #48]+        ldr q0, [sp, #32]+        ldp x1, x10, [sp, #48]+        uzp2 v29.4s, v19.4s, v19.4s+        xtn v4.2s, v19.2d+        umulh x8, x9, x13+        rev64 v20.4s, v23.4s+        umull v16.2d, v19.2s, v19.2s+        umull v1.2d, v29.2s, v4.2s+        mul v20.4s, v20.4s, v0.4s+        subs x14, x9, x13+        umulh x15, x9, x1+        mov x16, v16.d[1]+        umull2 v4.2d, v19.4s, v19.4s+        mov x4, v16.d[0]+        uzp1 v17.4s, v23.4s, v0.4s+        uaddlp v19.2d, v20.4s+        lsr x7, x8, #63+        mul x11, x9, x13+        mov x12, v1.d[0]+        csetm x5, cc+        cneg x6, x14, cc+        mov x3, v4.d[1]+        mov x14, v4.d[0]+        subs x2, x10, x1+        mov x9, v1.d[1]+        cneg x17, x2, cc+        cinv x2, x5, cc+        adds x5, x4, x12, lsl #33+        extr x4, x8, x11, #63+        lsr x8, x12, #31+        uzp1 v20.4s, v0.4s, v0.4s+        shl v19.2d, v19.2d, #32+        adc x16, x16, x8+        adds x8, x14, x9, lsl #33+        lsr x14, x9, #31+        lsl x9, x5, #32+        umlal v19.2d, v20.2s, v17.2s+        adc x14, x3, x14+        adds x16, x16, x11, lsl #1+        lsr x3, x5, #32+        umulh x12, x6, x17+        adcs x4, x8, x4+        adc x11, x14, x7+        subs x8, x5, x9+        sbc x5, x5, x3+        adds x16, x16, x9+        mov x14, v19.d[0]+        mul x17, x6, x17+        adcs x3, x4, x3+        lsl x7, x16, #32+        umulh x13, x13, x10+        adcs x11, x11, x8+        lsr x8, x16, #32+        adc x5, x5, xzr+        subs x9, x16, x7+        sbc x16, x16, x8+        adds x7, x3, x7+        mov x3, v19.d[1]+        adcs x6, x11, x8+        umulh x11, x1, x10+        adcs x5, x5, x9+        eor x8, x12, x2+        adc x9, x16, xzr+        adds x16, x14, x15+        adc x15, x15, xzr+        adds x12, x16, x3+        eor x16, x17, x2+        mul x4, x1, x10+        adcs x15, x15, x13+        adc x17, x13, xzr+        adds x15, x15, x3+        adc x3, x17, xzr+        cmn x2, #0x1+        mul x17, x10, x10+        adcs x12, x12, x16+        adcs x16, x15, x8+        umulh x10, x10, x10+        adc x2, x3, x2+        adds x14, x14, x14+        adcs x12, x12, x12+        adcs x16, x16, x16+        adcs x2, x2, x2+        adc x15, xzr, xzr+        adds x14, x14, x7+        mul x3, x1, x1+        adcs x12, x12, x6+        lsr x7, x14, #32+        adcs x16, x16, x5+        lsl x5, x14, #32+        umulh x13, x1, x1+        adcs x2, x2, x9+        mov x6, #0xffffffff+        adc x15, x15, xzr+        adds x8, x4, x4+        adcs x1, x11, x11+        mov x11, #0xffffffff00000001+        adc x4, xzr, xzr+        subs x9, x14, x5+        sbc x14, x14, x7+        adds x12, x12, x5+        adcs x16, x16, x7+        lsl x5, x12, #32+        lsr x7, x12, #32+        adcs x2, x2, x9+        adcs x14, x15, x14+        adc x15, xzr, xzr+        subs x9, x12, x5+        sbc x12, x12, x7+        adds x16, x16, x5+        adcs x2, x2, x7+        adcs x14, x14, x9+        adcs x12, x15, x12+        adc x15, xzr, xzr+        adds x16, x16, x3+        adcs x2, x2, x13+        adcs x14, x14, x17+        adcs x12, x12, x10+        adc x15, x15, xzr+        adds x2, x2, x8+        adcs x14, x14, x1+        adcs x12, x12, x4+        adcs x15, x15, xzr+        adds x3, x16, #0x1+        sbcs x5, x2, x6+        sbcs x8, x14, xzr+        sbcs x11, x12, x11+        sbcs xzr, x15, xzr+        csel x24, x3, x16, cs+        csel x25, x8, x14, cs+        csel x26, x11, x12, cs+        csel x27, x5, x2, cs+        stp x25, x26, [x0, #16]+        stp x24, x27, [x0]+        ldr q20, [sp, #96]+        ldr q0, [sp, #160]+        ldp x6, x10, [sp, #96]+        rev64 v16.4s, v20.4s+        subs x4, x20, x21+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4s, v16.4s, v0.4s+        umulh x12, x21, x10+        uzp1 v28.4s, v20.4s, v0.4s+        subs x14, x22, x20+        ldr q20, [sp, #112]+        sbcs x5, x23, x21+        ngc x17, xzr+        subs x8, x22, x23+        uaddlp v27.2d, v16.4s+        umulh x4, x20, x6+        uzp1 v21.4s, v0.4s, v0.4s+        cneg x11, x8, cc+        shl v17.2d, v27.2d, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2d, v21.2s, v28.2s+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [sp, #176]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2s, v20.2d+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4s, v20.4s, v20.4s+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2s, v28.2d+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [sp, #112]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x20, x3, x13+        adcs x21, x8, x7+        umulh x8, x14, x11+        umull v21.2d, v0.2s, v1.2s+        adcs x22, x10, x12+        umull v3.2d, v0.2s, v16.2s+        adc x23, x15, xzr+        rev64 v24.4s, v20.4s+        movi v2.2d, #0xffffffff+        mul x10, x14, x11+        mul v4.4s, v24.4s, v28.4s+        subs x13, x14, x5+        uzp2 v19.4s, v28.4s, v28.4s+        csetm x15, cc+        usra v3.2d, v21.2d, #32+        mul x7, x5, x1+        umull v21.2d, v19.2s, v16.2s+        cneg x13, x13, cc+        uaddlp v5.2d, v4.4s+        subs x11, x1, x11+        and v16.16b, v3.16b, v2.16b+        umulh x5, x5, x1+        shl v24.2d, v5.2d, #32+        cneg x11, x11, cc+        umlal v16.2d, v19.2s, v1.2s+        cinv x12, x15, cc+        umlal v24.2d, v0.2s, v1.2s+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        usra v21.2d, v3.2d, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2d, v16.2d, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x20+        adcs x15, x16, x21+        eor x5, x17, x4+        adcs x9, x1, x22+        eor x1, x10, x5+        adcs x16, x2, x23+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x14, x11, x13+        and x1, x1, x13+        adcs x15, x4, x1+        and x1, x12, x13+        stp x14, x15, [sp, #96]+        adcs x13, x7, xzr+        adc x20, x17, x1+        stp x13, x20, [sp, #112]+        ldp x5, x6, [sp, #64]+        ldp x4, x3, [sp, #32]+        subs x5, x5, x4+        sbcs x6, x6, x3+        ldp x7, x8, [sp, #80]+        ldp x4, x3, [sp, #48]+        sbcs x7, x7, x4+        sbcs x8, x8, x3+        csetm x3, cc+        adds x5, x5, x3+        and x4, x3, #0xffffffff+        adcs x6, x6, x4+        adcs x7, x7, xzr+        and x4, x3, #0xffffffff00000001+        adc x8, x8, x4+        stp x5, x6, [x19, #64]+        stp x7, x8, [x19, #80]+        ldp x1, x2, [sp, #128]+        lsl x0, x1, #2+        ldp x6, x7, [sp, #160]+        subs x0, x0, x6+        extr x1, x2, x1, #62+        sbcs x1, x1, x7+        ldp x3, x4, [sp, #144]+        extr x2, x3, x2, #62+        ldp x6, x7, [sp, #176]+        sbcs x2, x2, x6+        extr x3, x4, x3, #62+        sbcs x3, x3, x7+        lsr x4, x4, #62+        sbc x4, x4, xzr+        add x5, x4, #0x1+        lsl x8, x5, #32+        negs x6, x8+        ngcs x7, xzr+        sbc x8, x8, x5+        adds x0, x0, x5+        adcs x1, x1, x6+        adcs x2, x2, x7+        adcs x3, x3, x8+        csetm x5, cc+        adds x0, x0, x5+        and x6, x5, #0xffffffff+        adcs x1, x1, x6+        adcs x2, x2, xzr+        neg x7, x6+        adc x3, x3, x7+        stp x0, x1, [x19]+        stp x2, x3, [x19, #16]+        mov x2, #0xffffffffffffffff+        subs x9, x2, x24+        mov x2, #0xffffffff+        sbcs x10, x2, x27+        ngcs x11, x25+        mov x2, #0xffffffff00000001+        sbc x12, x2, x26+        lsl x3, x9, #3+        extr x4, x10, x9, #61+        extr x5, x11, x10, #61+        extr x6, x12, x11, #61+        lsr x7, x12, #61+        mov x1, #0x3+        mul x8, x14, x1+        umulh x9, x14, x1+        adds x3, x3, x8+        mul x8, x15, x1+        umulh x10, x15, x1+        adcs x4, x4, x8+        mul x8, x13, x1+        umulh x11, x13, x1+        adcs x5, x5, x8+        mul x8, x20, x1+        umulh x12, x20, x1+        adcs x6, x6, x8+        adc x7, x7, xzr+        adds x4, x4, x9+        adcs x5, x5, x10+        adcs x6, x6, x11+        adc x7, x7, x12+        add x8, x7, #0x1+        lsl x10, x8, #32+        adds x6, x6, x10+        adc x7, x7, xzr+        neg x9, x8+        sub x10, x10, #0x1+        subs x3, x3, x9+        sbcs x4, x4, x10+        sbcs x5, x5, xzr+        sbcs x6, x6, x8+        sbc x8, x7, x8+        adds x3, x3, x8+        and x9, x8, #0xffffffff+        adcs x4, x4, x9+        adcs x5, x5, xzr+        neg x10, x9+        adc x6, x6, x10+        stp x3, x4, [x19, #32]+        stp x5, x6, [x19, #48]++        CFI_STACKLOAD1Z(x27,NSPACE+64)+        CFI_STACKLOAD2(x25,x26,NSPACE+48)+        CFI_STACKLOAD2(x23,x24,NSPACE+32)+        CFI_STACKLOAD2(x21,x22,NSPACE+16)+        CFI_STACKLOAD2(x19,x20,NSPACE)+        CFI_INC_SP((NSPACE+80))+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_montjdouble)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/p256_montjdouble_alt.S view
@@ -0,0 +1,587 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Point doubling on NIST curve P-256 in Montgomery-Jacobian coordinates+//+//    extern void p256_montjdouble_alt(uint64_t p3[static 12],+//                                     const uint64_t p1[static 12]);+//+// Does p3 := 2 * p1 where all points are regarded as Jacobian triples with+// each coordinate in the Montgomery domain, i.e. x' = (2^256 * x) mod p_256.+// A Jacobian triple (x',y',z') represents affine point (x/z^2,y/z^3).+//+// Standard ARM ABI: X0 = p3, X1 = p1+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_montjdouble_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_montjdouble_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_montjdouble_alt)+        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Stable homes for input arguments during main code sequence++#define input_z x15+#define input_x x16++// Pointer-offset pairs for inputs and outputs++#define x_1 input_x, #0+#define y_1 input_x, #NUMSIZE+#define z_1 input_x, #(2*NUMSIZE)++#define x_3 input_z, #0+#define y_3 input_z, #NUMSIZE+#define z_3 input_z, #(2*NUMSIZE)++// Pointer-offset pairs for temporaries, with some aliasing+// #NSPACE is the total stack needed for these temporaries++#define z2 sp, #(NUMSIZE*0)+#define y4 sp, #(NUMSIZE*0)++#define y2 sp, #(NUMSIZE*1)++#define t1 sp, #(NUMSIZE*2)++#define t2 sp, #(NUMSIZE*3)+#define x2p sp, #(NUMSIZE*3)+#define dx2 sp, #(NUMSIZE*3)++#define xy2 sp, #(NUMSIZE*4)++#define x4p sp, #(NUMSIZE*5)+#define d sp, #(NUMSIZE*5)++#define NSPACE NUMSIZE*6++// Corresponds exactly to bignum_montmul_p256_alt except registers++#define montmul_p256(P0,P1,P2)                  \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x7, x8, [P2] __LF                  \+        mul     x12, x3, x7 __LF                   \+        umulh   x13, x3, x7 __LF                   \+        mul     x11, x3, x8 __LF                   \+        umulh   x14, x3, x8 __LF                   \+        adds    x13, x13, x11 __LF                 \+        ldp     x9, x10, [P2+16] __LF              \+        mul     x11, x3, x9 __LF                   \+        umulh   x0, x3, x9 __LF                    \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x3, x10 __LF                  \+        umulh   x1, x3, x10 __LF                   \+        adcs    x0, x0, x11 __LF                   \+        adc     x1, x1, xzr __LF                   \+        ldp     x5, x6, [P1+16] __LF               \+        mul     x11, x4, x7 __LF                   \+        adds    x13, x13, x11 __LF                 \+        mul     x11, x4, x8 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x4, x9 __LF                   \+        adcs    x0, x0, x11 __LF                   \+        mul     x11, x4, x10 __LF                  \+        adcs    x1, x1, x11 __LF                   \+        umulh   x3, x4, x10 __LF                   \+        adc     x3, x3, xzr __LF                   \+        umulh   x11, x4, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        umulh   x11, x4, x8 __LF                   \+        adcs    x0, x0, x11 __LF                   \+        umulh   x11, x4, x9 __LF                   \+        adcs    x1, x1, x11 __LF                   \+        adc     x3, x3, xzr __LF                   \+        mul     x11, x5, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        mul     x11, x5, x8 __LF                   \+        adcs    x0, x0, x11 __LF                   \+        mul     x11, x5, x9 __LF                   \+        adcs    x1, x1, x11 __LF                   \+        mul     x11, x5, x10 __LF                  \+        adcs    x3, x3, x11 __LF                   \+        umulh   x4, x5, x10 __LF                   \+        adc     x4, x4, xzr __LF                   \+        umulh   x11, x5, x7 __LF                   \+        adds    x0, x0, x11 __LF                   \+        umulh   x11, x5, x8 __LF                   \+        adcs    x1, x1, x11 __LF                   \+        umulh   x11, x5, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        adc     x4, x4, xzr __LF                   \+        mul     x11, x6, x7 __LF                   \+        adds    x0, x0, x11 __LF                   \+        mul     x11, x6, x8 __LF                   \+        adcs    x1, x1, x11 __LF                   \+        mul     x11, x6, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        mul     x11, x6, x10 __LF                  \+        adcs    x4, x4, x11 __LF                   \+        umulh   x5, x6, x10 __LF                   \+        adc     x5, x5, xzr __LF                   \+        mov     x10, #0xffffffff00000001 __LF      \+        adds    x13, x13, x12, lsl #32 __LF        \+        lsr     x11, x12, #32 __LF                 \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x12, x10 __LF                 \+        umulh   x12, x12, x10 __LF                 \+        adcs    x0, x0, x11 __LF                   \+        adc     x12, x12, xzr __LF                 \+        umulh   x11, x6, x7 __LF                   \+        adds    x1, x1, x11 __LF                   \+        umulh   x11, x6, x8 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        umulh   x11, x6, x9 __LF                   \+        adcs    x4, x4, x11 __LF                   \+        adc     x5, x5, xzr __LF                   \+        adds    x14, x14, x13, lsl #32 __LF        \+        lsr     x11, x13, #32 __LF                 \+        adcs    x0, x0, x11 __LF                   \+        mul     x11, x13, x10 __LF                 \+        umulh   x13, x13, x10 __LF                 \+        adcs    x12, x12, x11 __LF                 \+        adc     x13, x13, xzr __LF                 \+        adds    x0, x0, x14, lsl #32 __LF          \+        lsr     x11, x14, #32 __LF                 \+        adcs    x12, x12, x11 __LF                 \+        mul     x11, x14, x10 __LF                 \+        umulh   x14, x14, x10 __LF                 \+        adcs    x13, x13, x11 __LF                 \+        adc     x14, x14, xzr __LF                 \+        adds    x12, x12, x0, lsl #32 __LF         \+        lsr     x11, x0, #32 __LF                  \+        adcs    x13, x13, x11 __LF                 \+        mul     x11, x0, x10 __LF                  \+        umulh   x0, x0, x10 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        adc     x0, x0, xzr __LF                   \+        adds    x12, x12, x1 __LF                  \+        adcs    x13, x13, x3 __LF                  \+        adcs    x14, x14, x4 __LF                  \+        adcs    x0, x0, x5 __LF                    \+        cset    x8, cs __LF                        \+        mov     x11, #0xffffffff __LF              \+        adds    x1, x12, #0x1 __LF                 \+        sbcs    x3, x13, x11 __LF                  \+        sbcs    x4, x14, xzr __LF                  \+        sbcs    x5, x0, x10 __LF                   \+        sbcs    xzr, x8, xzr __LF                  \+        csel    x12, x12, x1, cc __LF              \+        csel    x13, x13, x3, cc __LF              \+        csel    x14, x14, x4, cc __LF              \+        csel    x0, x0, x5, cc __LF                \+        stp     x12, x13, [P0] __LF                \+        stp     x14, x0, [P0+16]++// Corresponds exactly to bignum_montsqr_p256_alt++#define montsqr_p256(P0,P1)                     \+        ldp     x2, x3, [P1] __LF                  \+        mul     x9, x2, x3 __LF                    \+        umulh   x10, x2, x3 __LF                   \+        ldp     x4, x5, [P1+16] __LF               \+        mul     x11, x2, x5 __LF                   \+        umulh   x12, x2, x5 __LF                   \+        mul     x6, x2, x4 __LF                    \+        umulh   x7, x2, x4 __LF                    \+        adds    x10, x10, x6 __LF                  \+        adcs    x11, x11, x7 __LF                  \+        mul     x6, x3, x4 __LF                    \+        umulh   x7, x3, x4 __LF                    \+        adc     x7, x7, xzr __LF                   \+        adds    x11, x11, x6 __LF                  \+        mul     x13, x4, x5 __LF                   \+        umulh   x14, x4, x5 __LF                   \+        adcs    x12, x12, x7 __LF                  \+        mul     x6, x3, x5 __LF                    \+        umulh   x7, x3, x5 __LF                    \+        adc     x7, x7, xzr __LF                   \+        adds    x12, x12, x6 __LF                  \+        adcs    x13, x13, x7 __LF                  \+        adc     x14, x14, xzr __LF                 \+        adds    x9, x9, x9 __LF                    \+        adcs    x10, x10, x10 __LF                 \+        adcs    x11, x11, x11 __LF                 \+        adcs    x12, x12, x12 __LF                 \+        adcs    x13, x13, x13 __LF                 \+        adcs    x14, x14, x14 __LF                 \+        cset    x7, hs __LF                        \+        umulh   x6, x2, x2 __LF                    \+        mul     x8, x2, x2 __LF                    \+        adds    x9, x9, x6 __LF                    \+        mul     x6, x3, x3 __LF                    \+        adcs    x10, x10, x6 __LF                  \+        umulh   x6, x3, x3 __LF                    \+        adcs    x11, x11, x6 __LF                  \+        mul     x6, x4, x4 __LF                    \+        adcs    x12, x12, x6 __LF                  \+        umulh   x6, x4, x4 __LF                    \+        adcs    x13, x13, x6 __LF                  \+        mul     x6, x5, x5 __LF                    \+        adcs    x14, x14, x6 __LF                  \+        umulh   x6, x5, x5 __LF                    \+        adc     x7, x7, x6 __LF                    \+        mov     x5, #-4294967295 __LF              \+        adds    x9, x9, x8, lsl #32 __LF           \+        lsr     x3, x8, #32 __LF                   \+        adcs    x10, x10, x3 __LF                  \+        mul     x2, x8, x5 __LF                    \+        umulh   x8, x8, x5 __LF                    \+        adcs    x11, x11, x2 __LF                  \+        adc     x8, x8, xzr __LF                   \+        adds    x10, x10, x9, lsl #32 __LF         \+        lsr     x3, x9, #32 __LF                   \+        adcs    x11, x11, x3 __LF                  \+        mul     x2, x9, x5 __LF                    \+        umulh   x9, x9, x5 __LF                    \+        adcs    x8, x8, x2 __LF                    \+        adc     x9, x9, xzr __LF                   \+        adds    x11, x11, x10, lsl #32 __LF        \+        lsr     x3, x10, #32 __LF                  \+        adcs    x8, x8, x3 __LF                    \+        mul     x2, x10, x5 __LF                   \+        umulh   x10, x10, x5 __LF                  \+        adcs    x9, x9, x2 __LF                    \+        adc     x10, x10, xzr __LF                 \+        adds    x8, x8, x11, lsl #32 __LF          \+        lsr     x3, x11, #32 __LF                  \+        adcs    x9, x9, x3 __LF                    \+        mul     x2, x11, x5 __LF                   \+        umulh   x11, x11, x5 __LF                  \+        adcs    x10, x10, x2 __LF                  \+        adc     x11, x11, xzr __LF                 \+        adds    x8, x8, x12 __LF                   \+        adcs    x9, x9, x13 __LF                   \+        adcs    x10, x10, x14 __LF                 \+        adcs    x11, x11, x7 __LF                  \+        cset    x2, hs __LF                        \+        mov     x3, #4294967295 __LF               \+        adds    x12, x8, #1 __LF                   \+        sbcs    x13, x9, x3 __LF                   \+        sbcs    x14, x10, xzr __LF                 \+        sbcs    x7, x11, x5 __LF                   \+        sbcs    xzr, x2, xzr __LF                  \+        csel    x8, x8, x12, lo __LF               \+        csel    x9, x9, x13, lo __LF               \+        csel    x10, x10, x14, lo __LF             \+        csel    x11, x11, x7, lo __LF              \+        stp     x8, x9, [P0] __LF                  \+        stp     x10, x11, [P0+16]++// Corresponds exactly to bignum_sub_p256++#define sub_p256(P0,P1,P2)                      \+        ldp     x5, x6, [P1] __LF                  \+        ldp     x4, x3, [P2] __LF                  \+        subs    x5, x5, x4 __LF                    \+        sbcs    x6, x6, x3 __LF                    \+        ldp     x7, x8, [P1+16] __LF               \+        ldp     x4, x3, [P2+16] __LF               \+        sbcs    x7, x7, x4 __LF                    \+        sbcs    x8, x8, x3 __LF                    \+        csetm   x3, lo __LF                        \+        adds    x5, x5, x3 __LF                    \+        and     x4, x3, #0xffffffff __LF           \+        adcs    x6, x6, x4 __LF                    \+        adcs    x7, x7, xzr __LF                   \+        and     x4, x3, #0xffffffff00000001 __LF   \+        adc     x8, x8, x4 __LF                    \+        stp     x5, x6, [P0] __LF                  \+        stp     x7, x8, [P0+16]++// Corresponds exactly to bignum_add_p256++#define add_p256(P0,P1,P2)                      \+        ldp     x5, x6, [P1] __LF                  \+        ldp     x4, x3, [P2] __LF                  \+        adds    x5, x5, x4 __LF                    \+        adcs    x6, x6, x3 __LF                    \+        ldp     x7, x8, [P1+16] __LF               \+        ldp     x4, x3, [P2+16] __LF               \+        adcs    x7, x7, x4 __LF                    \+        adcs    x8, x8, x3 __LF                    \+        adc     x3, xzr, xzr __LF                  \+        cmn     x5, #1 __LF                        \+        mov     x4, #4294967295 __LF               \+        sbcs    xzr, x6, x4 __LF                   \+        sbcs    xzr, x7, xzr __LF                  \+        mov     x4, #-4294967295 __LF              \+        sbcs    xzr, x8, x4 __LF                   \+        adcs    x3, x3, xzr __LF                   \+        csetm   x3, ne __LF                        \+        subs    x5, x5, x3 __LF                    \+        and     x4, x3, #0xffffffff __LF           \+        sbcs    x6, x6, x4 __LF                    \+        sbcs    x7, x7, xzr __LF                   \+        and     x4, x3, #0xffffffff00000001 __LF   \+        sbc     x8, x8, x4 __LF                    \+        stp     x5, x6, [P0] __LF                  \+        stp     x7, x8, [P0+16]++// A weak version of add that only guarantees sum in 4 digits++#define weakadd_p256(P0,P1,P2)                  \+        ldp     x5, x6, [P1] __LF                  \+        ldp     x4, x3, [P2] __LF                  \+        adds    x5, x5, x4 __LF                    \+        adcs    x6, x6, x3 __LF                    \+        ldp     x7, x8, [P1+16] __LF               \+        ldp     x4, x3, [P2+16] __LF               \+        adcs    x7, x7, x4 __LF                    \+        adcs    x8, x8, x3 __LF                    \+        csetm   x3, cs __LF                        \+        subs    x5, x5, x3 __LF                    \+        and     x1, x3, #4294967295 __LF           \+        sbcs    x6, x6, x1 __LF                    \+        sbcs    x7, x7, xzr __LF                   \+        and     x2, x3, #-4294967295 __LF          \+        sbc     x8, x8, x2 __LF                    \+        stp     x5, x6, [P0] __LF                  \+        stp     x7, x8, [P0+16]++// P0 = C * P1 - D * P2 computed as D * (p_256 - P2) + C * P1+// Quotient estimation is done just as q = h + 1 as in bignum_triple_p256+// This also applies to the other functions following.++#define cmsub_p256(P0,C,P1,D,P2)                \+        mov     x1, D __LF                         \+        mov     x2, #-1 __LF                       \+        ldp     x9, x10, [P2] __LF                 \+        subs    x9, x2, x9 __LF                    \+        mov     x2, #4294967295 __LF               \+        sbcs    x10, x2, x10 __LF                  \+        ldp     x11, x12, [P2+16] __LF             \+        sbcs    x11, xzr, x11 __LF                 \+        mov     x2, #-4294967295 __LF              \+        sbc     x12, x2, x12 __LF                  \+        mul     x3, x1, x9 __LF                    \+        mul     x4, x1, x10 __LF                   \+        mul     x5, x1, x11 __LF                   \+        mul     x6, x1, x12 __LF                   \+        umulh   x9, x1, x9 __LF                    \+        umulh   x10, x1, x10 __LF                  \+        umulh   x11, x1, x11 __LF                  \+        umulh   x7, x1, x12 __LF                   \+        adds    x4, x4, x9 __LF                    \+        adcs    x5, x5, x10 __LF                   \+        adcs    x6, x6, x11 __LF                   \+        adc     x7, x7, xzr __LF                   \+        mov     x1, C __LF                         \+        ldp     x9, x10, [P1] __LF                 \+        mul     x8, x9, x1 __LF                    \+        umulh   x9, x9, x1 __LF                    \+        adds    x3, x3, x8 __LF                    \+        mul     x8, x10, x1 __LF                   \+        umulh   x10, x10, x1 __LF                  \+        adcs    x4, x4, x8 __LF                    \+        ldp     x11, x12, [P1+16] __LF             \+        mul     x8, x11, x1 __LF                   \+        umulh   x11, x11, x1 __LF                  \+        adcs    x5, x5, x8 __LF                    \+        mul     x8, x12, x1 __LF                   \+        umulh   x12, x12, x1 __LF                  \+        adcs    x6, x6, x8 __LF                    \+        adc     x7, x7, xzr __LF                   \+        adds    x4, x4, x9 __LF                    \+        adcs    x5, x5, x10 __LF                   \+        adcs    x6, x6, x11 __LF                   \+        adc     x7, x7, x12 __LF                   \+        add     x8, x7, #1 __LF                    \+        lsl     x10, x8, #32 __LF                  \+        adds    x6, x6, x10 __LF                   \+        adc     x7, x7, xzr __LF                   \+        neg     x9, x8 __LF                        \+        sub     x10, x10, #1 __LF                  \+        subs    x3, x3, x9 __LF                    \+        sbcs    x4, x4, x10 __LF                   \+        sbcs    x5, x5, xzr __LF                   \+        sbcs    x6, x6, x8 __LF                    \+        sbc     x8, x7, x8 __LF                    \+        adds    x3, x3, x8 __LF                    \+        and     x9, x8, #4294967295 __LF           \+        adcs    x4, x4, x9 __LF                    \+        adcs    x5, x5, xzr __LF                   \+        neg     x10, x9 __LF                       \+        adc     x6, x6, x10 __LF                   \+        stp     x3, x4, [P0] __LF                  \+        stp     x5, x6, [P0+16]++// P0 = 4 * P1 - P2, by direct subtraction of P2; the method+// in bignum_cmul_p256 etc. for quotient estimation still+// works when the value to be reduced is negative, as+// long as it is  > -p_256, which is the case here. The+// actual accumulation of q * p_256 is done a bit differently+// so it works for the q = 0 case.++#define cmsub41_p256(P0,P1,P2)                  \+        ldp     x1, x2, [P1] __LF                  \+        lsl     x0, x1, #2 __LF                    \+        ldp     x6, x7, [P2] __LF                  \+        subs    x0, x0, x6 __LF                    \+        extr    x1, x2, x1, #62 __LF               \+        sbcs    x1, x1, x7 __LF                    \+        ldp     x3, x4, [P1+16] __LF               \+        extr    x2, x3, x2, #62 __LF               \+        ldp     x6, x7, [P2+16] __LF               \+        sbcs    x2, x2, x6 __LF                    \+        extr    x3, x4, x3, #62 __LF               \+        sbcs    x3, x3, x7 __LF                    \+        lsr     x4, x4, #62 __LF                   \+        sbc     x4, x4, xzr __LF                   \+        add     x5, x4, #1 __LF                    \+        lsl     x8, x5, #32 __LF                   \+        subs    x6, xzr, x8 __LF                   \+        sbcs    x7, xzr, xzr __LF                  \+        sbc     x8, x8, x5 __LF                    \+        adds    x0, x0, x5 __LF                    \+        adcs    x1, x1, x6 __LF                    \+        adcs    x2, x2, x7 __LF                    \+        adcs    x3, x3, x8 __LF                    \+        csetm   x5, cc __LF                        \+        adds    x0, x0, x5 __LF                    \+        and     x6, x5, #4294967295 __LF           \+        adcs    x1, x1, x6 __LF                    \+        adcs    x2, x2, xzr __LF                   \+        neg     x7, x6 __LF                        \+        adc     x3, x3, x7 __LF                    \+        stp     x0, x1, [P0] __LF                  \+        stp     x2, x3, [P0+16]++// P0 = 3 * P1 - 8 * P2, computed as (p_256 - P2) << 3 + 3 * P1++#define cmsub38_p256(P0,P1,P2)                  \+        mov     x1, 8 __LF                         \+        mov     x2, #-1 __LF                       \+        ldp     x9, x10, [P2] __LF                 \+        subs    x9, x2, x9 __LF                    \+        mov     x2, #4294967295 __LF               \+        sbcs    x10, x2, x10 __LF                  \+        ldp     x11, x12, [P2+16] __LF             \+        sbcs    x11, xzr, x11 __LF                 \+        mov     x2, #-4294967295 __LF              \+        sbc     x12, x2, x12 __LF                  \+        lsl     x3, x9, #3 __LF                    \+        extr    x4, x10, x9, #61 __LF              \+        extr    x5, x11, x10, #61 __LF             \+        extr    x6, x12, x11, #61 __LF             \+        lsr     x7, x12, #61 __LF                  \+        mov     x1, 3 __LF                         \+        ldp     x9, x10, [P1] __LF                 \+        mul     x8, x9, x1 __LF                    \+        umulh   x9, x9, x1 __LF                    \+        adds    x3, x3, x8 __LF                    \+        mul     x8, x10, x1 __LF                   \+        umulh   x10, x10, x1 __LF                  \+        adcs    x4, x4, x8 __LF                    \+        ldp     x11, x12, [P1+16] __LF             \+        mul     x8, x11, x1 __LF                   \+        umulh   x11, x11, x1 __LF                  \+        adcs    x5, x5, x8 __LF                    \+        mul     x8, x12, x1 __LF                   \+        umulh   x12, x12, x1 __LF                  \+        adcs    x6, x6, x8 __LF                    \+        adc     x7, x7, xzr __LF                   \+        adds    x4, x4, x9 __LF                    \+        adcs    x5, x5, x10 __LF                   \+        adcs    x6, x6, x11 __LF                   \+        adc     x7, x7, x12 __LF                   \+        add     x8, x7, #1 __LF                    \+        lsl     x10, x8, #32 __LF                  \+        adds    x6, x6, x10 __LF                   \+        adc     x7, x7, xzr __LF                   \+        neg     x9, x8 __LF                        \+        sub     x10, x10, #1 __LF                  \+        subs    x3, x3, x9 __LF                    \+        sbcs    x4, x4, x10 __LF                   \+        sbcs    x5, x5, xzr __LF                   \+        sbcs    x6, x6, x8 __LF                    \+        sbc     x8, x7, x8 __LF                    \+        adds    x3, x3, x8 __LF                    \+        and     x9, x8, #4294967295 __LF           \+        adcs    x4, x4, x9 __LF                    \+        adcs    x5, x5, xzr __LF                   \+        neg     x10, x9 __LF                       \+        adc     x6, x6, x10 __LF                   \+        stp     x3, x4, [P0] __LF                  \+        stp     x5, x6, [P0+16]++S2N_BN_SYMBOL(p256_montjdouble_alt):+        CFI_START++// Make room on stack for temporary variables++        CFI_DEC_SP(NSPACE)++// Move the input arguments to stable places++        mov     input_z, x0+        mov     input_x, x1++// Main code, just a sequence of basic field operations++// z2 = z^2+// y2 = y^2++        montsqr_p256(z2,z_1)+        montsqr_p256(y2,y_1)++// x2p = x^2 - z^4 = (x + z^2) * (x - z^2)++        sub_p256(t2,x_1,z2)+        weakadd_p256(t1,x_1,z2)+        montmul_p256(x2p,t1,t2)++// t1 = y + z+// xy2 = x * y^2+// x4p = x2p^2++        add_p256(t1,y_1,z_1)+        montmul_p256(xy2,x_1,y2)+        montsqr_p256(x4p,x2p)++// t1 = (y + z)^2++        montsqr_p256(t1,t1)++// d = 12 * xy2 - 9 * x4p+// t1 = y^2 + 2 * y * z++        cmsub_p256(d,12,xy2,9,x4p)+        sub_p256(t1,t1,z2)++// y4 = y^4++        montsqr_p256(y4,y2)++// dx2 = d * x2p++        montmul_p256(dx2,d,x2p)++// z_3' = 2 * y * z++        sub_p256(z_3,t1,y2)++// x' = 4 * xy2 - d++        cmsub41_p256(x_3,xy2,d)++// y' = 3 * dx2 - 8 * y4++        cmsub38_p256(y_3,dx2,y4)++// Restore stack and return++        CFI_INC_SP(NSPACE)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_montjdouble_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/p256_montjmixadd.S view
@@ -0,0 +1,513 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Point mixed addition on NIST curve P-256 in Montgomery-Jacobian coordinates+//+//    extern void p256_montjmixadd(uint64_t p3[static 12],+//                                 const uint64_t p1[static 12],+//                                 const uint64_t p2[static 8]);+//+// Does p3 := p1 + p2 where all points are regarded as Jacobian triples with+// each coordinate in the Montgomery domain, i.e. x' = (2^256 * x) mod p_256.+// A Jacobian triple (x',y',z') represents affine point (x/z^2,y/z^3).+// The "mixed" part means that p2 only has x and y coordinates, with the+// implicit z coordinate assumed to be the identity.+//+// Standard ARM ABI: X0 = p3, X1 = p1, X2 = p2+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_montjmixadd)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_montjmixadd)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_montjmixadd)++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Stable homes for input arguments during main code sequence++#define input_z x17+#define input_x x19+#define input_y x20++// Pointer-offset pairs for inputs and outputs++#define x_1 input_x, #0+#define y_1 input_x, #NUMSIZE+#define z_1 input_x, #(2*NUMSIZE)++#define x_2 input_y, #0+#define y_2 input_y, #NUMSIZE++#define x_3 input_z, #0+#define y_3 input_z, #NUMSIZE+#define z_3 input_z, #(2*NUMSIZE)++// Pointer-offset pairs for temporaries, with some aliasing+// #NSPACE is the total stack needed for these temporaries++#define zp2 sp, #(NUMSIZE*0)+#define ww sp, #(NUMSIZE*0)+#define resx sp, #(NUMSIZE*0)++#define yd sp, #(NUMSIZE*1)+#define y2a sp, #(NUMSIZE*1)++#define x2a sp, #(NUMSIZE*2)+#define zzx2 sp, #(NUMSIZE*2)++#define zz sp, #(NUMSIZE*3)+#define t1 sp, #(NUMSIZE*3)++#define t2 sp, #(NUMSIZE*4)+#define zzx1 sp, #(NUMSIZE*4)+#define resy sp, #(NUMSIZE*4)++#define xd sp, #(NUMSIZE*5)+#define resz sp, #(NUMSIZE*5)++#define NSPACE NUMSIZE*6++// Corresponds to bignum_montmul_p256 but uses x0 in place of x17++#define montmul_p256(P0,P1,P2)                  \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x5, x6, [P1+16] __LF               \+        ldp     x7, x8, [P2] __LF                  \+        ldp     x9, x10, [P2+16] __LF              \+        mul     x11, x3, x7 __LF                   \+        mul     x13, x4, x8 __LF                   \+        umulh   x12, x3, x7 __LF                   \+        adds    x16, x11, x13 __LF                 \+        umulh   x14, x4, x8 __LF                   \+        adcs    x0, x12, x14 __LF                  \+        adcs    x14, x14, xzr __LF                 \+        adds    x12, x12, x16 __LF                 \+        adcs    x13, x13, x0 __LF                  \+        adcs    x14, x14, xzr __LF                 \+        subs    x15, x3, x4 __LF                   \+        cneg    x15, x15, lo __LF                  \+        csetm   x1, lo __LF                        \+        subs    x0, x8, x7 __LF                    \+        cneg    x0, x0, lo __LF                    \+        mul     x16, x15, x0 __LF                  \+        umulh   x0, x15, x0 __LF                   \+        cinv    x1, x1, lo __LF                    \+        eor     x16, x16, x1 __LF                  \+        eor     x0, x0, x1 __LF                    \+        cmn     x1, #1 __LF                        \+        adcs    x12, x12, x16 __LF                 \+        adcs    x13, x13, x0 __LF                  \+        adc     x14, x14, x1 __LF                  \+        lsl     x0, x11, #32 __LF                  \+        subs    x1, x11, x0 __LF                   \+        lsr     x16, x11, #32 __LF                 \+        sbc     x11, x11, x16 __LF                 \+        adds    x12, x12, x0 __LF                  \+        adcs    x13, x13, x16 __LF                 \+        adcs    x14, x14, x1 __LF                  \+        adc     x11, x11, xzr __LF                 \+        lsl     x0, x12, #32 __LF                  \+        subs    x1, x12, x0 __LF                   \+        lsr     x16, x12, #32 __LF                 \+        sbc     x12, x12, x16 __LF                 \+        adds    x13, x13, x0 __LF                  \+        adcs    x14, x14, x16 __LF                 \+        adcs    x11, x11, x1 __LF                  \+        adc     x12, x12, xzr __LF                 \+        stp     x13, x14, [P0] __LF                \+        stp     x11, x12, [P0+16] __LF             \+        mul     x11, x5, x9 __LF                   \+        mul     x13, x6, x10 __LF                  \+        umulh   x12, x5, x9 __LF                   \+        adds    x16, x11, x13 __LF                 \+        umulh   x14, x6, x10 __LF                  \+        adcs    x0, x12, x14 __LF                  \+        adcs    x14, x14, xzr __LF                 \+        adds    x12, x12, x16 __LF                 \+        adcs    x13, x13, x0 __LF                  \+        adcs    x14, x14, xzr __LF                 \+        subs    x15, x5, x6 __LF                   \+        cneg    x15, x15, lo __LF                  \+        csetm   x1, lo __LF                        \+        subs    x0, x10, x9 __LF                   \+        cneg    x0, x0, lo __LF                    \+        mul     x16, x15, x0 __LF                  \+        umulh   x0, x15, x0 __LF                   \+        cinv    x1, x1, lo __LF                    \+        eor     x16, x16, x1 __LF                  \+        eor     x0, x0, x1 __LF                    \+        cmn     x1, #1 __LF                        \+        adcs    x12, x12, x16 __LF                 \+        adcs    x13, x13, x0 __LF                  \+        adc     x14, x14, x1 __LF                  \+        subs    x3, x5, x3 __LF                    \+        sbcs    x4, x6, x4 __LF                    \+        ngc     x5, xzr __LF                       \+        cmn     x5, #1 __LF                        \+        eor     x3, x3, x5 __LF                    \+        adcs    x3, x3, xzr __LF                   \+        eor     x4, x4, x5 __LF                    \+        adcs    x4, x4, xzr __LF                   \+        subs    x7, x7, x9 __LF                    \+        sbcs    x8, x8, x10 __LF                   \+        ngc     x9, xzr __LF                       \+        cmn     x9, #1 __LF                        \+        eor     x7, x7, x9 __LF                    \+        adcs    x7, x7, xzr __LF                   \+        eor     x8, x8, x9 __LF                    \+        adcs    x8, x8, xzr __LF                   \+        eor     x10, x5, x9 __LF                   \+        ldp     x15, x1, [P0] __LF                 \+        adds    x15, x11, x15 __LF                 \+        adcs    x1, x12, x1 __LF                   \+        ldp     x5, x9, [P0+16] __LF               \+        adcs    x5, x13, x5 __LF                   \+        adcs    x9, x14, x9 __LF                   \+        adc     x2, xzr, xzr __LF                  \+        mul     x11, x3, x7 __LF                   \+        mul     x13, x4, x8 __LF                   \+        umulh   x12, x3, x7 __LF                   \+        adds    x16, x11, x13 __LF                 \+        umulh   x14, x4, x8 __LF                   \+        adcs    x0, x12, x14 __LF                  \+        adcs    x14, x14, xzr __LF                 \+        adds    x12, x12, x16 __LF                 \+        adcs    x13, x13, x0 __LF                  \+        adcs    x14, x14, xzr __LF                 \+        subs    x3, x3, x4 __LF                    \+        cneg    x3, x3, lo __LF                    \+        csetm   x4, lo __LF                        \+        subs    x0, x8, x7 __LF                    \+        cneg    x0, x0, lo __LF                    \+        mul     x16, x3, x0 __LF                   \+        umulh   x0, x3, x0 __LF                    \+        cinv    x4, x4, lo __LF                    \+        eor     x16, x16, x4 __LF                  \+        eor     x0, x0, x4 __LF                    \+        cmn     x4, #1 __LF                        \+        adcs    x12, x12, x16 __LF                 \+        adcs    x13, x13, x0 __LF                  \+        adc     x14, x14, x4 __LF                  \+        cmn     x10, #1 __LF                       \+        eor     x11, x11, x10 __LF                 \+        adcs    x11, x11, x15 __LF                 \+        eor     x12, x12, x10 __LF                 \+        adcs    x12, x12, x1 __LF                  \+        eor     x13, x13, x10 __LF                 \+        adcs    x13, x13, x5 __LF                  \+        eor     x14, x14, x10 __LF                 \+        adcs    x14, x14, x9 __LF                  \+        adcs    x3, x2, x10 __LF                   \+        adcs    x4, x10, xzr __LF                  \+        adc     x10, x10, xzr __LF                 \+        adds    x13, x13, x15 __LF                 \+        adcs    x14, x14, x1 __LF                  \+        adcs    x3, x3, x5 __LF                    \+        adcs    x4, x4, x9 __LF                    \+        adc     x10, x10, x2 __LF                  \+        lsl     x0, x11, #32 __LF                  \+        subs    x1, x11, x0 __LF                   \+        lsr     x16, x11, #32 __LF                 \+        sbc     x11, x11, x16 __LF                 \+        adds    x12, x12, x0 __LF                  \+        adcs    x13, x13, x16 __LF                 \+        adcs    x14, x14, x1 __LF                  \+        adc     x11, x11, xzr __LF                 \+        lsl     x0, x12, #32 __LF                  \+        subs    x1, x12, x0 __LF                   \+        lsr     x16, x12, #32 __LF                 \+        sbc     x12, x12, x16 __LF                 \+        adds    x13, x13, x0 __LF                  \+        adcs    x14, x14, x16 __LF                 \+        adcs    x11, x11, x1 __LF                  \+        adc     x12, x12, xzr __LF                 \+        adds    x3, x3, x11 __LF                   \+        adcs    x4, x4, x12 __LF                   \+        adc     x10, x10, xzr __LF                 \+        add     x2, x10, #1 __LF                   \+        lsl     x16, x2, #32 __LF                  \+        adds    x4, x4, x16 __LF                   \+        adc     x10, x10, xzr __LF                 \+        neg     x15, x2 __LF                       \+        sub     x16, x16, #1 __LF                  \+        subs    x13, x13, x15 __LF                 \+        sbcs    x14, x14, x16 __LF                 \+        sbcs    x3, x3, xzr __LF                   \+        sbcs    x4, x4, x2 __LF                    \+        sbcs    x7, x10, x2 __LF                   \+        adds    x13, x13, x7 __LF                  \+        mov     x10, #4294967295 __LF              \+        and     x10, x10, x7 __LF                  \+        adcs    x14, x14, x10 __LF                 \+        adcs    x3, x3, xzr __LF                   \+        mov     x10, #-4294967295 __LF             \+        and     x10, x10, x7 __LF                  \+        adc     x4, x4, x10 __LF                   \+        stp     x13, x14, [P0] __LF                \+        stp     x3, x4, [P0+16]++// Corresponds to bignum_montsqr_p256 but uses x0 in place of x17++#define montsqr_p256(P0,P1)                     \+        ldp     x2, x3, [P1] __LF                  \+        ldp     x4, x5, [P1+16] __LF               \+        umull   x15, w2, w2 __LF                   \+        lsr     x11, x2, #32 __LF                  \+        umull   x16, w11, w11 __LF                 \+        umull   x11, w2, w11 __LF                  \+        adds    x15, x15, x11, lsl #33 __LF        \+        lsr     x11, x11, #31 __LF                 \+        adc     x16, x16, x11 __LF                 \+        umull   x0, w3, w3 __LF                    \+        lsr     x11, x3, #32 __LF                  \+        umull   x1, w11, w11 __LF                  \+        umull   x11, w3, w11 __LF                  \+        mul     x12, x2, x3 __LF                   \+        umulh   x13, x2, x3 __LF                   \+        adds    x0, x0, x11, lsl #33 __LF          \+        lsr     x11, x11, #31 __LF                 \+        adc     x1, x1, x11 __LF                   \+        adds    x12, x12, x12 __LF                 \+        adcs    x13, x13, x13 __LF                 \+        adc     x1, x1, xzr __LF                   \+        adds    x16, x16, x12 __LF                 \+        adcs    x0, x0, x13 __LF                   \+        adc     x1, x1, xzr __LF                   \+        lsl     x12, x15, #32 __LF                 \+        subs    x13, x15, x12 __LF                 \+        lsr     x11, x15, #32 __LF                 \+        sbc     x15, x15, x11 __LF                 \+        adds    x16, x16, x12 __LF                 \+        adcs    x0, x0, x11 __LF                   \+        adcs    x1, x1, x13 __LF                   \+        adc     x15, x15, xzr __LF                 \+        lsl     x12, x16, #32 __LF                 \+        subs    x13, x16, x12 __LF                 \+        lsr     x11, x16, #32 __LF                 \+        sbc     x16, x16, x11 __LF                 \+        adds    x0, x0, x12 __LF                   \+        adcs    x1, x1, x11 __LF                   \+        adcs    x15, x15, x13 __LF                 \+        adc     x16, x16, xzr __LF                 \+        mul     x6, x2, x4 __LF                    \+        mul     x14, x3, x5 __LF                   \+        umulh   x8, x2, x4 __LF                    \+        subs    x10, x2, x3 __LF                   \+        cneg    x10, x10, lo __LF                  \+        csetm   x13, lo __LF                       \+        subs    x12, x5, x4 __LF                   \+        cneg    x12, x12, lo __LF                  \+        mul     x11, x10, x12 __LF                 \+        umulh   x12, x10, x12 __LF                 \+        cinv    x13, x13, lo __LF                  \+        eor     x11, x11, x13 __LF                 \+        eor     x12, x12, x13 __LF                 \+        adds    x7, x6, x8 __LF                    \+        adc     x8, x8, xzr __LF                   \+        umulh   x9, x3, x5 __LF                    \+        adds    x7, x7, x14 __LF                   \+        adcs    x8, x8, x9 __LF                    \+        adc     x9, x9, xzr __LF                   \+        adds    x8, x8, x14 __LF                   \+        adc     x9, x9, xzr __LF                   \+        cmn     x13, #1 __LF                       \+        adcs    x7, x7, x11 __LF                   \+        adcs    x8, x8, x12 __LF                   \+        adc     x9, x9, x13 __LF                   \+        adds    x6, x6, x6 __LF                    \+        adcs    x7, x7, x7 __LF                    \+        adcs    x8, x8, x8 __LF                    \+        adcs    x9, x9, x9 __LF                    \+        adc     x10, xzr, xzr __LF                 \+        adds    x6, x6, x0 __LF                    \+        adcs    x7, x7, x1 __LF                    \+        adcs    x8, x8, x15 __LF                   \+        adcs    x9, x9, x16 __LF                   \+        adc     x10, x10, xzr __LF                 \+        lsl     x12, x6, #32 __LF                  \+        subs    x13, x6, x12 __LF                  \+        lsr     x11, x6, #32 __LF                  \+        sbc     x6, x6, x11 __LF                   \+        adds    x7, x7, x12 __LF                   \+        adcs    x8, x8, x11 __LF                   \+        adcs    x9, x9, x13 __LF                   \+        adcs    x10, x10, x6 __LF                  \+        adc     x6, xzr, xzr __LF                  \+        lsl     x12, x7, #32 __LF                  \+        subs    x13, x7, x12 __LF                  \+        lsr     x11, x7, #32 __LF                  \+        sbc     x7, x7, x11 __LF                   \+        adds    x8, x8, x12 __LF                   \+        adcs    x9, x9, x11 __LF                   \+        adcs    x10, x10, x13 __LF                 \+        adcs    x6, x6, x7 __LF                    \+        adc     x7, xzr, xzr __LF                  \+        mul     x11, x4, x4 __LF                   \+        adds    x8, x8, x11 __LF                   \+        mul     x12, x5, x5 __LF                   \+        umulh   x11, x4, x4 __LF                   \+        adcs    x9, x9, x11 __LF                   \+        adcs    x10, x10, x12 __LF                 \+        umulh   x12, x5, x5 __LF                   \+        adcs    x6, x6, x12 __LF                   \+        adc     x7, x7, xzr __LF                   \+        mul     x11, x4, x5 __LF                   \+        umulh   x12, x4, x5 __LF                   \+        adds    x11, x11, x11 __LF                 \+        adcs    x12, x12, x12 __LF                 \+        adc     x13, xzr, xzr __LF                 \+        adds    x9, x9, x11 __LF                   \+        adcs    x10, x10, x12 __LF                 \+        adcs    x6, x6, x13 __LF                   \+        adcs    x7, x7, xzr __LF                   \+        mov     x11, #4294967295 __LF              \+        adds    x5, x8, #1 __LF                    \+        sbcs    x11, x9, x11 __LF                  \+        mov     x13, #-4294967295 __LF             \+        sbcs    x12, x10, xzr __LF                 \+        sbcs    x13, x6, x13 __LF                  \+        sbcs    xzr, x7, xzr __LF                  \+        csel    x8, x5, x8, hs __LF                \+        csel    x9, x11, x9, hs __LF               \+        csel    x10, x12, x10, hs __LF             \+        csel    x6, x13, x6, hs __LF               \+        stp     x8, x9, [P0] __LF                  \+        stp     x10, x6, [P0+16]++// Corresponds exactly to bignum_sub_p256++#define sub_p256(P0,P1,P2)                      \+        ldp     x5, x6, [P1] __LF                  \+        ldp     x4, x3, [P2] __LF                  \+        subs    x5, x5, x4 __LF                    \+        sbcs    x6, x6, x3 __LF                    \+        ldp     x7, x8, [P1+16] __LF               \+        ldp     x4, x3, [P2+16] __LF               \+        sbcs    x7, x7, x4 __LF                    \+        sbcs    x8, x8, x3 __LF                    \+        csetm   x3, cc __LF                        \+        adds    x5, x5, x3 __LF                    \+        mov     x4, #0xffffffff __LF               \+        and     x4, x4, x3 __LF                    \+        adcs    x6, x6, x4 __LF                    \+        adcs    x7, x7, xzr __LF                   \+        mov     x4, #0xffffffff00000001 __LF       \+        and     x4, x4, x3 __LF                    \+        adc     x8, x8, x4 __LF                    \+        stp     x5, x6, [P0] __LF                  \+        stp     x7, x8, [P0+16]++S2N_BN_SYMBOL(p256_montjmixadd):+        CFI_START++// Save regs and make room on stack for temporary variables++        CFI_PUSH2(x19,x20)+        CFI_DEC_SP(NSPACE)++// Move the input arguments to stable places++        mov     input_z, x0+        mov     input_x, x1+        mov     input_y, x2++// Main code, just a sequence of basic field operations+// 8 * multiply + 3 * square + 7 * subtract++        montsqr_p256(zp2,z_1)+        montmul_p256(y2a,z_1,y_2)++        montmul_p256(x2a,zp2,x_2)+        montmul_p256(y2a,zp2,y2a)++        sub_p256(xd,x2a,x_1)+        sub_p256(yd,y2a,y_1)++        montsqr_p256(zz,xd)+        montsqr_p256(ww,yd)++        montmul_p256(zzx1,zz,x_1)+        montmul_p256(zzx2,zz,x2a)++        sub_p256(resx,ww,zzx1)+        sub_p256(t1,zzx2,zzx1)++        montmul_p256(resz,xd,z_1)++        sub_p256(resx,resx,zzx2)++        sub_p256(t2,zzx1,resx)++        montmul_p256(t1,t1,y_1)+        montmul_p256(t2,yd,t2)++        sub_p256(resy,t2,t1)++// Test if z_1 = 0 to decide if p1 = 0 (up to projective equivalence)++        ldp     x0, x1, [z_1]+        ldp     x2, x3, [z_1+16]+        orr     x4, x0, x1+        orr     x5, x2, x3+        orr     x4, x4, x5+        cmp     x4, xzr++// Multiplex: if p1 <> 0 just copy the computed result from the staging area.+// If p1 = 0 then return the point p2 augmented with a z = 1 coordinate (in+// Montgomery form so not the simple constant 1 but rather 2^256 - p_256),+// hence giving 0 + p2 = p2 for the final result.++        ldp     x0, x1, [resx]+        ldp     x12, x13, [x_2]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x2, x3, [resx+16]+        ldp     x12, x13, [x_2+16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne++        ldp     x4, x5, [resy]+        ldp     x12, x13, [y_2]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x6, x7, [resy+16]+        ldp     x12, x13, [y_2+16]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne++        ldp     x8, x9, [resz]+        mov     x12, #0x0000000000000001+        mov     x13, #0xffffffff00000000+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x10, x11, [resz+16]+        mov     x12, #0xffffffffffffffff+        mov     x13, #0x00000000fffffffe+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne++        stp     x0, x1, [x_3]+        stp     x2, x3, [x_3+16]+        stp     x4, x5, [y_3]+        stp     x6, x7, [y_3+16]+        stp     x8, x9, [z_3]+        stp     x10, x11, [z_3+16]++// Restore registers and return++        CFI_INC_SP(NSPACE)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_montjmixadd)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/p256_montjmixadd_alt.S view
@@ -0,0 +1,517 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Point mixed addition on NIST curve P-256 in Montgomery-Jacobian coordinates+//+//    extern void p256_montjmixadd_alt(uint64_t p3[static 12],+//                                     const uint64_t p1[static 12],+//                                     const uint64_t p2[static 8]);+//+// Does p3 := p1 + p2 where all points are regarded as Jacobian triples with+// each coordinate in the Montgomery domain, i.e. x' = (2^256 * x) mod p_256.+// A Jacobian triple (x',y',z') represents affine point (x/z^2,y/z^3).+// The "mixed" part means that p2 only has x and y coordinates, with the+// implicit z coordinate assumed to be the identity.+//+// Standard ARM ABI: X0 = p3, X1 = p1, X2 = p2+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_montjmixadd_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_montjmixadd_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_montjmixadd_alt)++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Stable homes for input arguments during main code sequence++#define input_z x15+#define input_x x16+#define input_y x17++// Pointer-offset pairs for inputs and outputs++#define x_1 input_x, #0+#define y_1 input_x, #NUMSIZE+#define z_1 input_x, #(2*NUMSIZE)++#define x_2 input_y, #0+#define y_2 input_y, #NUMSIZE++#define x_3 input_z, #0+#define y_3 input_z, #NUMSIZE+#define z_3 input_z, #(2*NUMSIZE)++// Pointer-offset pairs for temporaries, with some aliasing+// #NSPACE is the total stack needed for these temporaries++#define zp2 sp, #(NUMSIZE*0)+#define ww sp, #(NUMSIZE*0)+#define resx sp, #(NUMSIZE*0)++#define yd sp, #(NUMSIZE*1)+#define y2a sp, #(NUMSIZE*1)++#define x2a sp, #(NUMSIZE*2)+#define zzx2 sp, #(NUMSIZE*2)++#define zz sp, #(NUMSIZE*3)+#define t1 sp, #(NUMSIZE*3)++#define t2 sp, #(NUMSIZE*4)+#define zzx1 sp, #(NUMSIZE*4)+#define resy sp, #(NUMSIZE*4)++#define xd sp, #(NUMSIZE*5)+#define resz sp, #(NUMSIZE*5)++#define NSPACE NUMSIZE*6++// Corresponds to bignum_montmul_p256_alt except registers++#define montmul_p256(P0,P1,P2)                  \+        ldp     x3, x4, [P1] __LF                  \+        ldp     x7, x8, [P2] __LF                  \+        mul     x12, x3, x7 __LF                   \+        umulh   x13, x3, x7 __LF                   \+        mul     x11, x3, x8 __LF                   \+        umulh   x14, x3, x8 __LF                   \+        adds    x13, x13, x11 __LF                 \+        ldp     x9, x10, [P2+16] __LF              \+        mul     x11, x3, x9 __LF                   \+        umulh   x0, x3, x9 __LF                    \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x3, x10 __LF                  \+        umulh   x1, x3, x10 __LF                   \+        adcs    x0, x0, x11 __LF                   \+        adc     x1, x1, xzr __LF                   \+        ldp     x5, x6, [P1+16] __LF               \+        mul     x11, x4, x7 __LF                   \+        adds    x13, x13, x11 __LF                 \+        mul     x11, x4, x8 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x4, x9 __LF                   \+        adcs    x0, x0, x11 __LF                   \+        mul     x11, x4, x10 __LF                  \+        adcs    x1, x1, x11 __LF                   \+        umulh   x3, x4, x10 __LF                   \+        adc     x3, x3, xzr __LF                   \+        umulh   x11, x4, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        umulh   x11, x4, x8 __LF                   \+        adcs    x0, x0, x11 __LF                   \+        umulh   x11, x4, x9 __LF                   \+        adcs    x1, x1, x11 __LF                   \+        adc     x3, x3, xzr __LF                   \+        mul     x11, x5, x7 __LF                   \+        adds    x14, x14, x11 __LF                 \+        mul     x11, x5, x8 __LF                   \+        adcs    x0, x0, x11 __LF                   \+        mul     x11, x5, x9 __LF                   \+        adcs    x1, x1, x11 __LF                   \+        mul     x11, x5, x10 __LF                  \+        adcs    x3, x3, x11 __LF                   \+        umulh   x4, x5, x10 __LF                   \+        adc     x4, x4, xzr __LF                   \+        umulh   x11, x5, x7 __LF                   \+        adds    x0, x0, x11 __LF                   \+        umulh   x11, x5, x8 __LF                   \+        adcs    x1, x1, x11 __LF                   \+        umulh   x11, x5, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        adc     x4, x4, xzr __LF                   \+        mul     x11, x6, x7 __LF                   \+        adds    x0, x0, x11 __LF                   \+        mul     x11, x6, x8 __LF                   \+        adcs    x1, x1, x11 __LF                   \+        mul     x11, x6, x9 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        mul     x11, x6, x10 __LF                  \+        adcs    x4, x4, x11 __LF                   \+        umulh   x5, x6, x10 __LF                   \+        adc     x5, x5, xzr __LF                   \+        mov     x10, #0xffffffff00000001 __LF      \+        adds    x13, x13, x12, lsl #32 __LF        \+        lsr     x11, x12, #32 __LF                 \+        adcs    x14, x14, x11 __LF                 \+        mul     x11, x12, x10 __LF                 \+        umulh   x12, x12, x10 __LF                 \+        adcs    x0, x0, x11 __LF                   \+        adc     x12, x12, xzr __LF                 \+        umulh   x11, x6, x7 __LF                   \+        adds    x1, x1, x11 __LF                   \+        umulh   x11, x6, x8 __LF                   \+        adcs    x3, x3, x11 __LF                   \+        umulh   x11, x6, x9 __LF                   \+        adcs    x4, x4, x11 __LF                   \+        adc     x5, x5, xzr __LF                   \+        adds    x14, x14, x13, lsl #32 __LF        \+        lsr     x11, x13, #32 __LF                 \+        adcs    x0, x0, x11 __LF                   \+        mul     x11, x13, x10 __LF                 \+        umulh   x13, x13, x10 __LF                 \+        adcs    x12, x12, x11 __LF                 \+        adc     x13, x13, xzr __LF                 \+        adds    x0, x0, x14, lsl #32 __LF          \+        lsr     x11, x14, #32 __LF                 \+        adcs    x12, x12, x11 __LF                 \+        mul     x11, x14, x10 __LF                 \+        umulh   x14, x14, x10 __LF                 \+        adcs    x13, x13, x11 __LF                 \+        adc     x14, x14, xzr __LF                 \+        adds    x12, x12, x0, lsl #32 __LF         \+        lsr     x11, x0, #32 __LF                  \+        adcs    x13, x13, x11 __LF                 \+        mul     x11, x0, x10 __LF                  \+        umulh   x0, x0, x10 __LF                   \+        adcs    x14, x14, x11 __LF                 \+        adc     x0, x0, xzr __LF                   \+        adds    x12, x12, x1 __LF                  \+        adcs    x13, x13, x3 __LF                  \+        adcs    x14, x14, x4 __LF                  \+        adcs    x0, x0, x5 __LF                    \+        cset    x8, cs __LF                        \+        mov     x11, #0xffffffff __LF              \+        adds    x1, x12, #0x1 __LF                 \+        sbcs    x3, x13, x11 __LF                  \+        sbcs    x4, x14, xzr __LF                  \+        sbcs    x5, x0, x10 __LF                   \+        sbcs    xzr, x8, xzr __LF                  \+        csel    x12, x12, x1, cc __LF              \+        csel    x13, x13, x3, cc __LF              \+        csel    x14, x14, x4, cc __LF              \+        csel    x0, x0, x5, cc __LF                \+        stp     x12, x13, [P0] __LF                \+        stp     x14, x0, [P0+16]++// Corresponds exactly to bignum_montsqr_p256_alt++#define montsqr_p256(P0,P1)                     \+        ldp     x2, x3, [P1] __LF                  \+        mul     x9, x2, x3 __LF                    \+        umulh   x10, x2, x3 __LF                   \+        ldp     x4, x5, [P1+16] __LF               \+        mul     x11, x2, x5 __LF                   \+        umulh   x12, x2, x5 __LF                   \+        mul     x6, x2, x4 __LF                    \+        umulh   x7, x2, x4 __LF                    \+        adds    x10, x10, x6 __LF                  \+        adcs    x11, x11, x7 __LF                  \+        mul     x6, x3, x4 __LF                    \+        umulh   x7, x3, x4 __LF                    \+        adc     x7, x7, xzr __LF                   \+        adds    x11, x11, x6 __LF                  \+        mul     x13, x4, x5 __LF                   \+        umulh   x14, x4, x5 __LF                   \+        adcs    x12, x12, x7 __LF                  \+        mul     x6, x3, x5 __LF                    \+        umulh   x7, x3, x5 __LF                    \+        adc     x7, x7, xzr __LF                   \+        adds    x12, x12, x6 __LF                  \+        adcs    x13, x13, x7 __LF                  \+        adc     x14, x14, xzr __LF                 \+        adds    x9, x9, x9 __LF                    \+        adcs    x10, x10, x10 __LF                 \+        adcs    x11, x11, x11 __LF                 \+        adcs    x12, x12, x12 __LF                 \+        adcs    x13, x13, x13 __LF                 \+        adcs    x14, x14, x14 __LF                 \+        cset    x7, cs __LF                        \+        umulh   x6, x2, x2 __LF                    \+        mul     x8, x2, x2 __LF                    \+        adds    x9, x9, x6 __LF                    \+        mul     x6, x3, x3 __LF                    \+        adcs    x10, x10, x6 __LF                  \+        umulh   x6, x3, x3 __LF                    \+        adcs    x11, x11, x6 __LF                  \+        mul     x6, x4, x4 __LF                    \+        adcs    x12, x12, x6 __LF                  \+        umulh   x6, x4, x4 __LF                    \+        adcs    x13, x13, x6 __LF                  \+        mul     x6, x5, x5 __LF                    \+        adcs    x14, x14, x6 __LF                  \+        umulh   x6, x5, x5 __LF                    \+        adc     x7, x7, x6 __LF                    \+        adds    x9, x9, x8, lsl #32 __LF           \+        lsr     x3, x8, #32 __LF                   \+        adcs    x10, x10, x3 __LF                  \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x8, x3 __LF                    \+        umulh   x8, x8, x3 __LF                    \+        adcs    x11, x11, x2 __LF                  \+        adc     x8, x8, xzr __LF                   \+        adds    x10, x10, x9, lsl #32 __LF         \+        lsr     x3, x9, #32 __LF                   \+        adcs    x11, x11, x3 __LF                  \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x9, x3 __LF                    \+        umulh   x9, x9, x3 __LF                    \+        adcs    x8, x8, x2 __LF                    \+        adc     x9, x9, xzr __LF                   \+        adds    x11, x11, x10, lsl #32 __LF        \+        lsr     x3, x10, #32 __LF                  \+        adcs    x8, x8, x3 __LF                    \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x10, x3 __LF                   \+        umulh   x10, x10, x3 __LF                  \+        adcs    x9, x9, x2 __LF                    \+        adc     x10, x10, xzr __LF                 \+        adds    x8, x8, x11, lsl #32 __LF          \+        lsr     x3, x11, #32 __LF                  \+        adcs    x9, x9, x3 __LF                    \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x11, x3 __LF                   \+        umulh   x11, x11, x3 __LF                  \+        adcs    x10, x10, x2 __LF                  \+        adc     x11, x11, xzr __LF                 \+        adds    x8, x8, x12 __LF                   \+        adcs    x9, x9, x13 __LF                   \+        adcs    x10, x10, x14 __LF                 \+        adcs    x11, x11, x7 __LF                  \+        cset    x2, cs __LF                        \+        mov     x3, #0xffffffff __LF               \+        mov     x5, #0xffffffff00000001 __LF       \+        adds    x12, x8, #0x1 __LF                 \+        sbcs    x13, x9, x3 __LF                   \+        sbcs    x14, x10, xzr __LF                 \+        sbcs    x7, x11, x5 __LF                   \+        sbcs    xzr, x2, xzr __LF                  \+        csel    x8, x8, x12, cc __LF               \+        csel    x9, x9, x13, cc __LF               \+        csel    x10, x10, x14, cc __LF             \+        csel    x11, x11, x7, cc __LF              \+        stp     x8, x9, [P0] __LF                  \+        stp     x10, x11, [P0+16]++// Almost-Montgomery variant which we use when an input to other muls+// with the other argument fully reduced (which is always safe).++#define amontsqr_p256(P0,P1)                    \+        ldp     x2, x3, [P1] __LF                  \+        mul     x9, x2, x3 __LF                    \+        umulh   x10, x2, x3 __LF                   \+        ldp     x4, x5, [P1+16] __LF               \+        mul     x11, x2, x5 __LF                   \+        umulh   x12, x2, x5 __LF                   \+        mul     x6, x2, x4 __LF                    \+        umulh   x7, x2, x4 __LF                    \+        adds    x10, x10, x6 __LF                  \+        adcs    x11, x11, x7 __LF                  \+        mul     x6, x3, x4 __LF                    \+        umulh   x7, x3, x4 __LF                    \+        adc     x7, x7, xzr __LF                   \+        adds    x11, x11, x6 __LF                  \+        mul     x13, x4, x5 __LF                   \+        umulh   x14, x4, x5 __LF                   \+        adcs    x12, x12, x7 __LF                  \+        mul     x6, x3, x5 __LF                    \+        umulh   x7, x3, x5 __LF                    \+        adc     x7, x7, xzr __LF                   \+        adds    x12, x12, x6 __LF                  \+        adcs    x13, x13, x7 __LF                  \+        adc     x14, x14, xzr __LF                 \+        adds    x9, x9, x9 __LF                    \+        adcs    x10, x10, x10 __LF                 \+        adcs    x11, x11, x11 __LF                 \+        adcs    x12, x12, x12 __LF                 \+        adcs    x13, x13, x13 __LF                 \+        adcs    x14, x14, x14 __LF                 \+        cset    x7, cs __LF                        \+        umulh   x6, x2, x2 __LF                    \+        mul     x8, x2, x2 __LF                    \+        adds    x9, x9, x6 __LF                    \+        mul     x6, x3, x3 __LF                    \+        adcs    x10, x10, x6 __LF                  \+        umulh   x6, x3, x3 __LF                    \+        adcs    x11, x11, x6 __LF                  \+        mul     x6, x4, x4 __LF                    \+        adcs    x12, x12, x6 __LF                  \+        umulh   x6, x4, x4 __LF                    \+        adcs    x13, x13, x6 __LF                  \+        mul     x6, x5, x5 __LF                    \+        adcs    x14, x14, x6 __LF                  \+        umulh   x6, x5, x5 __LF                    \+        adc     x7, x7, x6 __LF                    \+        adds    x9, x9, x8, lsl #32 __LF           \+        lsr     x3, x8, #32 __LF                   \+        adcs    x10, x10, x3 __LF                  \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x8, x3 __LF                    \+        umulh   x8, x8, x3 __LF                    \+        adcs    x11, x11, x2 __LF                  \+        adc     x8, x8, xzr __LF                   \+        adds    x10, x10, x9, lsl #32 __LF         \+        lsr     x3, x9, #32 __LF                   \+        adcs    x11, x11, x3 __LF                  \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x9, x3 __LF                    \+        umulh   x9, x9, x3 __LF                    \+        adcs    x8, x8, x2 __LF                    \+        adc     x9, x9, xzr __LF                   \+        adds    x11, x11, x10, lsl #32 __LF        \+        lsr     x3, x10, #32 __LF                  \+        adcs    x8, x8, x3 __LF                    \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x10, x3 __LF                   \+        umulh   x10, x10, x3 __LF                  \+        adcs    x9, x9, x2 __LF                    \+        adc     x10, x10, xzr __LF                 \+        adds    x8, x8, x11, lsl #32 __LF          \+        lsr     x3, x11, #32 __LF                  \+        adcs    x9, x9, x3 __LF                    \+        mov     x3, #0xffffffff00000001 __LF       \+        mul     x2, x11, x3 __LF                   \+        umulh   x11, x11, x3 __LF                  \+        adcs    x10, x10, x2 __LF                  \+        adc     x11, x11, xzr __LF                 \+        adds    x8, x8, x12 __LF                   \+        adcs    x9, x9, x13 __LF                   \+        adcs    x10, x10, x14 __LF                 \+        adcs    x11, x11, x7 __LF                  \+        mov     x2, #0xffffffffffffffff __LF       \+        csel    x2, xzr, x2, cc __LF               \+        mov     x3, #0xffffffff __LF               \+        csel    x3, xzr, x3, cc __LF               \+        mov     x5, #0xffffffff00000001 __LF       \+        csel    x5, xzr, x5, cc __LF               \+        subs    x8, x8, x2 __LF                    \+        sbcs    x9, x9, x3 __LF                    \+        sbcs    x10, x10, xzr __LF                 \+        sbc     x11, x11, x5 __LF                  \+        stp     x8, x9, [P0] __LF                  \+        stp     x10, x11, [P0+16]++// Corresponds exactly to bignum_sub_p256++#define sub_p256(P0,P1,P2)                      \+        ldp     x5, x6, [P1] __LF                  \+        ldp     x4, x3, [P2] __LF                  \+        subs    x5, x5, x4 __LF                    \+        sbcs    x6, x6, x3 __LF                    \+        ldp     x7, x8, [P1+16] __LF               \+        ldp     x4, x3, [P2+16] __LF               \+        sbcs    x7, x7, x4 __LF                    \+        sbcs    x8, x8, x3 __LF                    \+        csetm   x3, cc __LF                        \+        adds    x5, x5, x3 __LF                    \+        mov     x4, #0xffffffff __LF               \+        and     x4, x4, x3 __LF                    \+        adcs    x6, x6, x4 __LF                    \+        adcs    x7, x7, xzr __LF                   \+        mov     x4, #0xffffffff00000001 __LF       \+        and     x4, x4, x3 __LF                    \+        adc     x8, x8, x4 __LF                    \+        stp     x5, x6, [P0] __LF                  \+        stp     x7, x8, [P0+16]++S2N_BN_SYMBOL(p256_montjmixadd_alt):+        CFI_START++// Make room on stack for temporary variables+// Move the input arguments to stable places++        CFI_DEC_SP(NSPACE)++        mov     input_z, x0+        mov     input_x, x1+        mov     input_y, x2++// Main code, just a sequence of basic field operations+// 8 * multiply + 3 * square + 7 * subtract++        amontsqr_p256(zp2,z_1)+        montmul_p256(y2a,z_1,y_2)++        montmul_p256(x2a,zp2,x_2)+        montmul_p256(y2a,zp2,y2a)++        sub_p256(xd,x2a,x_1)+        sub_p256(yd,y2a,y_1)++        amontsqr_p256(zz,xd)+        montsqr_p256(ww,yd)++        montmul_p256(zzx1,zz,x_1)+        montmul_p256(zzx2,zz,x2a)++        sub_p256(resx,ww,zzx1)+        sub_p256(t1,zzx2,zzx1)++        montmul_p256(resz,xd,z_1)++        sub_p256(resx,resx,zzx2)++        sub_p256(t2,zzx1,resx)++        montmul_p256(t1,t1,y_1)+        montmul_p256(t2,yd,t2)++        sub_p256(resy,t2,t1)++// Test if z_1 = 0 to decide if p1 = 0 (up to projective equivalence)++        ldp     x0, x1, [z_1]+        ldp     x2, x3, [z_1+16]+        orr     x4, x0, x1+        orr     x5, x2, x3+        orr     x4, x4, x5+        cmp     x4, xzr++// Multiplex: if p1 <> 0 just copy the computed result from the staging area.+// If p1 = 0 then return the point p2 augmented with a z = 1 coordinate (in+// Montgomery form so not the simple constant 1 but rather 2^256 - p_256),+// hence giving 0 + p2 = p2 for the final result.++        ldp     x0, x1, [resx]+        ldp     x12, x13, [x_2]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x2, x3, [resx+16]+        ldp     x12, x13, [x_2+16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne++        ldp     x4, x5, [resy]+        ldp     x12, x13, [y_2]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x6, x7, [resy+16]+        ldp     x12, x13, [y_2+16]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne++        ldp     x8, x9, [resz]+        mov     x12, #0x0000000000000001+        mov     x13, #0xffffffff00000000+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x10, x11, [resz+16]+        mov     x12, #0xffffffffffffffff+        mov     x13, #0x00000000fffffffe+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne++        stp     x0, x1, [x_3]+        stp     x2, x3, [x_3+16]+        stp     x4, x5, [y_3]+        stp     x6, x7, [y_3+16]+        stp     x8, x9, [z_3]+        stp     x10, x11, [z_3+16]++// Restore stack and return++        CFI_INC_SP(NSPACE)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_montjmixadd_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/p256_scalarmul.S view
@@ -0,0 +1,8620 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Scalar multiplication for P-256+// Input scalar[4], point[8]; output res[8]+//+// extern void p256_scalarmul+//   (uint64_t res[static 8],+//    const uint64_t scalar[static 4],+//    const uint64_t point[static 8]);+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-256, returns the point (X,Y) = n * P. The input and output+// are affine points, and in the case of the point at infinity as+// the result, (0,0) is returned.+//+// Standard ARM ABI: X0 = res, X1 = scalar, X2 = point+// ----------------------------------------------------------------------------+++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_scalarmul)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_scalarmul)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_scalarmul)+++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Safe copies of inputs (res lasts the whole code, point not so long)+// and additional values in variables, with some aliasing++#define res x19+#define sgn x20+#define j x20+#define point x21++// Intermediate variables on the stack. The last z2, z3 values can+// safely be overlaid on the table, which is no longer needed at the end.++#define scalarb sp, #(0*NUMSIZE)+#define acc sp, #(1*NUMSIZE)+#define tabent sp, #(4*NUMSIZE)++#define tab sp, #(7*NUMSIZE)++#define z2 sp, #(7*NUMSIZE)+#define z3 sp, #(8*NUMSIZE)++#define NSPACE 31*NUMSIZE++// Loading large constants++#define movbig(nn,n3,n2,n1,n0)                                      \+        movz    nn, n0 __LF                                            \+        movk    nn, n1, lsl #16 __LF                                   \+        movk    nn, n2, lsl #32 __LF                                   \+        movk    nn, n3, lsl #48++S2N_BN_SYMBOL(p256_scalarmul):+        CFI_START++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x30)+        CFI_DEC_SP(NSPACE)++// Preserve the "res" and "point" input arguments. We load and process the+// scalar immediately so we don't bother preserving that input argument.+// Also, "point" is only needed early on and so its register gets re-used.++        mov     res, x0+        mov     point, x2++// Load the digits of group order n_256 = [x12;x13;x14;x15]++        movbig(x12, #0xf3b9, #0xcac2, #0xfc63, #0x2551)+        movbig(x13, #0xbce6, #0xfaad, #0xa717, #0x9e84)+        mov     x14, #0xffffffffffffffff+        mov     x15, #0xffffffff00000000++// First, reduce the input scalar mod n_256, i.e. conditionally subtract n_256++        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]++        subs    x6, x2, x12+        sbcs    x7, x3, x13+        sbcs    x8, x4, x14+        sbcs    x9, x5, x15++        csel    x2, x2, x6, cc+        csel    x3, x3, x7, cc+        csel    x4, x4, x8, cc+        csel    x5, x5, x9, cc++// Now if the top bit of the reduced scalar is set, negate it mod n_256,+// i.e. do n |-> n_256 - n. Remember the sign as "sgn" so we can+// correspondingly negate the point below.++        subs    x6, x12, x2+        sbcs    x7, x13, x3+        sbcs    x8, x14, x4+        sbc     x9, x15, x5++        tst     x5, #0x8000000000000000+        csel    x2, x2, x6, eq+        csel    x3, x3, x7, eq+        csel    x4, x4, x8, eq+        csel    x5, x5, x9, eq+        cset    sgn, ne++// In either case then add the recoding constant 0x08888...888 to allow+// signed digits.++        mov     x6, 0x8888888888888888+        adds    x2, x2, x6+        adcs    x3, x3, x6+        bic     x7, x6, #0xF000000000000000+        adcs    x4, x4, x6+        adc     x5, x5, x7++        stp     x2, x3, [scalarb]+        stp     x4, x5, [scalarb+16]++// Set the tab[0] table entry to Montgomery-Jacobian point = 1 * P+// The z coordinate is just the Montgomery form of the constant 1.++        add     x0, tab+        mov     x1, point+        CFI_BL(Lp256_scalarmul_local_tomont_p256)++        add     x1, point, #32+        add     x0, tab+32+        CFI_BL(Lp256_scalarmul_local_tomont_p256)++        mov     x0, #0x0000000000000001+        mov     x1, #0xffffffff00000000+        stp     x0, x1, [tab+64]+        mov     x2, #0xffffffffffffffff+        mov     x3, #0x00000000fffffffe+        stp     x2, x3, [tab+80]++// If the top bit of the scalar was set, negate (y coordinate of) the point++        ldp     x4, x5, [tab+32]+        ldp     x6, x7, [tab+48]++        mov     x0, 0xffffffffffffffff+        subs    x0, x0, x4+        mov     x1, 0x00000000ffffffff+        sbcs    x1, x1, x5+        mov     x3, 0xffffffff00000001+        sbcs    x2, xzr, x6+        sbc     x3, x3, x7++        cmp     sgn, xzr+        csel    x4, x0, x4, ne+        csel    x5, x1, x5, ne+        csel    x6, x2, x6, ne+        csel    x7, x3, x7, ne++        stp     x4, x5, [tab+32]+        stp     x6, x7, [tab+48]++// Compute and record tab[1] = 2 * p, ..., tab[7] = 8 * P++        add     x0, tab+96*1+        add     x1, tab+        CFI_BL(Lp256_scalarmul_local_p256_montjdouble)++        add     x0, tab+96*2+        add     x1, tab+96*1+        add     x2, tab+        CFI_BL(Lp256_scalarmul_local_p256_montjmixadd)++        add     x0, tab+96*3+        add     x1, tab+96*1+        CFI_BL(Lp256_scalarmul_local_p256_montjdouble)++        add     x0, tab+96*4+        add     x1, tab+96*3+        add     x2, tab+        CFI_BL(Lp256_scalarmul_local_p256_montjmixadd)++        add     x0, tab+96*5+        add     x1, tab+96*2+        CFI_BL(Lp256_scalarmul_local_p256_montjdouble)++        add     x0, tab+96*6+        add     x1, tab+96*5+        add     x2, tab+        CFI_BL(Lp256_scalarmul_local_p256_montjmixadd)++        add     x0, tab+96*7+        add     x1, tab+96*3+        CFI_BL(Lp256_scalarmul_local_p256_montjdouble)++// Initialize the accumulator as a table entry for top 4 bits (unrecoded)++        ldr     x14, [scalarb+24]+        lsr     x14, x14, #60++        mov     x0, xzr+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, xzr+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        mov     x9, xzr+        mov     x10, xzr+        mov     x11, xzr+        add     x15, tab++        .set i, 1+.rep 8+        cmp     x14, #i+        ldp     x12, x13, [x15]+        csel    x0, x12, x0, eq+        csel    x1, x13, x1, eq+        ldp     x12, x13, [x15, #16]+        csel    x2, x12, x2, eq+        csel    x3, x13, x3, eq+        ldp     x12, x13, [x15, #32]+        csel    x4, x12, x4, eq+        csel    x5, x13, x5, eq+        ldp     x12, x13, [x15, #48]+        csel    x6, x12, x6, eq+        csel    x7, x13, x7, eq+        ldp     x12, x13, [x15, #64]+        csel    x8, x12, x8, eq+        csel    x9, x13, x9, eq+        ldp     x12, x13, [x15, #80]+        csel    x10, x12, x10, eq+        csel    x11, x13, x11, eq+        add     x15, x15, #96+        .set    i, (i+1)+.endr+        stp     x0, x1, [acc]+        stp     x2, x3, [acc+16]+        stp     x4, x5, [acc+32]+        stp     x6, x7, [acc+48]+        stp     x8, x9, [acc+64]+        stp     x10, x11, [acc+80]++        mov     j, #252++// Main loop over size-4 bitfields: double 4 times then add signed digit++Lp256_scalarmul_loop:+        sub     j, j, #4++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp256_scalarmul_local_p256_montjdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp256_scalarmul_local_p256_montjdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp256_scalarmul_local_p256_montjdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp256_scalarmul_local_p256_montjdouble)++        lsr     x2, j, #6+        ldr     x14, [sp, x2, lsl #3]   // Exploits scalarb = sp exactly+        lsr     x14, x14, j+        and     x14, x14, #15++        subs    x14, x14, #8+        cset    x16, lo                 // x16 = sign of digit (1 = negative)+        cneg    x14, x14, lo            // x14 = absolute value of digit++// Conditionally select the table entry tab[i-1] = i * P in constant time++        mov     x0, xzr+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, xzr+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        mov     x9, xzr+        mov     x10, xzr+        mov     x11, xzr+        add     x15, tab+        .set i, 1+.rep 8+        cmp     x14, #i+        ldp     x12, x13, [x15]+        csel    x0, x12, x0, eq+        csel    x1, x13, x1, eq+        ldp     x12, x13, [x15, #16]+        csel    x2, x12, x2, eq+        csel    x3, x13, x3, eq+        ldp     x12, x13, [x15, #32]+        csel    x4, x12, x4, eq+        csel    x5, x13, x5, eq+        ldp     x12, x13, [x15, #48]+        csel    x6, x12, x6, eq+        csel    x7, x13, x7, eq+        ldp     x12, x13, [x15, #64]+        csel    x8, x12, x8, eq+        csel    x9, x13, x9, eq+        ldp     x12, x13, [x15, #80]+        csel    x10, x12, x10, eq+        csel    x11, x13, x11, eq+        add     x15, x15, #96+        .set    i, (i+1)+.endr++// Store it to "tabent" with the y coordinate optionally negated++        stp     x0, x1, [tabent]+        stp     x2, x3, [tabent+16]++        mov     x0, 0xffffffffffffffff+        subs    x0, x0, x4+        mov     x1, 0x00000000ffffffff+        sbcs    x1, x1, x5+        mov     x3, 0xffffffff00000001+        sbcs    x2, xzr, x6+        sbc     x3, x3, x7++        cmp     x16, xzr+        csel    x4, x0, x4, ne+        csel    x5, x1, x5, ne+        csel    x6, x2, x6, ne+        csel    x7, x3, x7, ne++        stp     x4, x5, [tabent+32]+        stp     x6, x7, [tabent+48]+        stp     x8, x9, [tabent+64]+        stp     x10, x11, [tabent+80]++        add     x0, acc+        add     x1, acc+        add     x2, tabent+        CFI_BL(Lp256_scalarmul_local_p256_montjadd)++        cbnz    j, Lp256_scalarmul_loop++// That's the end of the main loop, and we just need to translate+// back from the Jacobian representation to affine. First of all,+// let z2 = 1/z^2 and z3 = 1/z^3, both without Montgomery form++        add     x0, z2+        add     x1, acc+64+        CFI_BL(Lp256_scalarmul_local_montsqr_p256)++        add     x0, z3+        add     x2, z2+        add     x1, acc+64+        CFI_BL(Lp256_scalarmul_local_montmul_p256)++        add     x0, z2+        add     x1, z3+        CFI_BL(Lp256_scalarmul_local_demont_p256)++        add     x0, z3+        add     x1, z2+        CFI_BL(Lp256_scalarmul_local_inv_p256)++        add     x0, z2+        add     x2, z3+        add     x1, acc+64+        CFI_BL(Lp256_scalarmul_local_montmul_p256)++// Convert back from Jacobian (X,Y,Z) |-> (X/Z^2, Y/Z^3)++        add     x1, acc+        add     x2, z2+        mov     x0, res+        CFI_BL(Lp256_scalarmul_local_montmul_p256)++        add     x0, res, #32+        add     x1, acc+32+        add     x2, z3+        CFI_BL(Lp256_scalarmul_local_montmul_p256)++// Restore stack and registers and return++        CFI_INC_SP(NSPACE)+        CFI_POP2(x21,x30)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_scalarmul)++// Local copies of subroutines, complete clones at the moment++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_local_demont_p256)++Lp256_scalarmul_local_demont_p256:+        CFI_START+        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]+        lsl     x7, x2, #32+        subs    x8, x2, x7+        lsr     x6, x2, #32+        sbc     x2, x2, x6+        adds    x3, x3, x7+        adcs    x4, x4, x6+        adcs    x5, x5, x8+        adc     x2, x2, xzr+        lsl     x7, x3, #32+        subs    x8, x3, x7+        lsr     x6, x3, #32+        sbc     x3, x3, x6+        adds    x4, x4, x7+        adcs    x5, x5, x6+        adcs    x2, x2, x8+        adc     x3, x3, xzr+        lsl     x7, x4, #32+        subs    x8, x4, x7+        lsr     x6, x4, #32+        sbc     x4, x4, x6+        adds    x5, x5, x7+        adcs    x2, x2, x6+        adcs    x3, x3, x8+        adc     x4, x4, xzr+        lsl     x7, x5, #32+        subs    x8, x5, x7+        lsr     x6, x5, #32+        sbc     x5, x5, x6+        adds    x2, x2, x7+        adcs    x3, x3, x6+        adcs    x4, x4, x8+        adc     x5, x5, xzr+        stp     x2, x3, [x0]+        stp     x4, x5, [x0, #16]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_local_demont_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_local_inv_p256)++Lp256_scalarmul_local_inv_p256:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_DEC_SP(160)+        mov     x20, x0+        mov     x10, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x13, #0xffffffff00000001+        stp     x10, x11, [sp]+        stp     xzr, x13, [sp, #16]+        str     xzr, [sp, #32]+        ldp     x2, x3, [x1]+        subs    x10, x2, x10+        sbcs    x11, x3, x11+        ldp     x4, x5, [x1, #16]+        sbcs    x12, x4, xzr+        sbcs    x13, x5, x13+        csel    x2, x2, x10, cc+        csel    x3, x3, x11, cc+        csel    x4, x4, x12, cc+        csel    x5, x5, x13, cc+        stp     x2, x3, [sp, #48]+        stp     x4, x5, [sp, #64]+        str     xzr, [sp, #80]+        stp     xzr, xzr, [sp, #96]+        stp     xzr, xzr, [sp, #112]+        mov     x10, #0x4000000000000+        stp     x10, xzr, [sp, #128]+        stp     xzr, xzr, [sp, #144]+        mov     x21, #0xa+        mov     x22, #0x1+        b       Lp256_scalarmul_inv_midloop+Lp256_scalarmul_inv_loop:+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        and     x0, x12, x16+        and     x1, x13, x17+        add     x19, x0, x1+        ldr     x7, [sp]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #48]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x7, [sp, #8]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #56]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [sp]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [sp, #48]+        ldr     x7, [sp, #16]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #64]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [sp, #8]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [sp, #56]+        ldr     x7, [sp, #24]+        eor     x1, x7, x14+        ldr     x23, [sp, #32]+        eor     x3, x23, x14+        and     x3, x3, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #72]+        eor     x1, x8, x15+        ldr     x24, [sp, #80]+        eor     x0, x24, x15+        and     x0, x0, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x5, x6, #59+        str     x6, [sp, #16]+        extr    x5, x3, x5, #59+        str     x5, [sp, #24]+        asr     x3, x3, #59+        str     x3, [sp, #32]+        eor     x1, x7, x16+        eor     x5, x23, x16+        and     x5, x5, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        eor     x0, x24, x17+        and     x0, x0, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x4, x2, x4, #59+        str     x4, [sp, #64]+        extr    x2, x5, x2, #59+        str     x2, [sp, #72]+        asr     x5, x5, #59+        str     x5, [sp, #80]+        ldr     x7, [sp, #96]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #128]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #96]+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        str     x5, [sp, #128]+        adc     x3, x3, x1+        ldr     x7, [sp, #104]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #136]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #104]+        adc     x6, x6, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        str     x3, [sp, #136]+        adc     x4, x4, x1+        ldr     x7, [sp, #112]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #144]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #112]+        adc     x5, x5, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        str     x4, [sp, #144]+        adc     x2, x2, x1+        ldr     x7, [sp, #120]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #152]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldp     x0, x1, [sp, #96]+        ldr     x6, [sp, #112]+        mov     x14, #0xe000000000000000+        adds    x0, x0, x14+        sbcs    x1, x1, xzr+        mov     x11, #0x1fffffff+        adcs    x6, x6, x11+        mov     x10, #0x2000000000000000+        adcs    x5, x5, x10+        mov     x14, #0x1fffffffe0000000+        adc     x3, x3, x14+        lsl     x11, x0, #32+        subs    x14, x0, x11+        lsr     x10, x0, #32+        sbc     x0, x0, x10+        adds    x1, x1, x11+        adcs    x6, x6, x10+        adcs    x5, x5, x14+        adcs    x3, x3, x0+        mov     x14, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x10, #0xffffffff00000001+        csel    x14, x14, xzr, cs+        csel    x11, x11, xzr, cs+        csel    x10, x10, xzr, cs+        subs    x1, x1, x14+        sbcs    x6, x6, x11+        sbcs    x5, x5, xzr+        sbc     x3, x3, x10+        stp     x1, x6, [sp, #96]+        stp     x5, x3, [sp, #112]+        eor     x1, x7, x16+        and     x5, x16, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        and     x0, x17, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        ldp     x0, x1, [sp, #128]+        ldr     x3, [sp, #144]+        mov     x14, #0xe000000000000000+        adds    x0, x0, x14+        sbcs    x1, x1, xzr+        mov     x11, #0x1fffffff+        adcs    x3, x3, x11+        mov     x10, #0x2000000000000000+        adcs    x2, x2, x10+        mov     x14, #0x1fffffffe0000000+        adc     x5, x5, x14+        lsl     x11, x0, #32+        subs    x14, x0, x11+        lsr     x10, x0, #32+        sbc     x0, x0, x10+        adds    x1, x1, x11+        adcs    x3, x3, x10+        adcs    x2, x2, x14+        adcs    x5, x5, x0+        mov     x14, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x10, #0xffffffff00000001+        csel    x14, x14, xzr, cs+        csel    x11, x11, xzr, cs+        csel    x10, x10, xzr, cs+        subs    x1, x1, x14+        sbcs    x3, x3, x11+        sbcs    x2, x2, xzr+        sbc     x5, x5, x10+        stp     x1, x3, [sp, #128]+        stp     x2, x5, [sp, #144]+Lp256_scalarmul_inv_midloop:+        mov     x1, x22+        ldr     x2, [sp]+        ldr     x3, [sp, #48]+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x8, x4, #0x100, lsl #12+        sbfx    x8, x8, #21, #21+        mov     x11, #0x100000+        add     x11, x11, x11, lsl #21+        add     x9, x4, x11+        asr     x9, x9, #42+        add     x10, x5, #0x100, lsl #12+        sbfx    x10, x10, #21, #21+        add     x11, x5, x11+        asr     x11, x11, #42+        mul     x6, x8, x2+        mul     x7, x9, x3+        mul     x2, x10, x2+        mul     x3, x11, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #21, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #42+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #21, #21+        add     x15, x5, x15+        asr     x15, x15, #42+        mul     x6, x12, x2+        mul     x7, x13, x3+        mul     x2, x14, x2+        mul     x3, x15, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        mul     x2, x12, x8+        mul     x3, x12, x9+        mul     x6, x14, x8+        mul     x7, x14, x9+        madd    x8, x13, x10, x2+        madd    x9, x13, x11, x3+        madd    x16, x15, x10, x6+        madd    x17, x15, x11, x7+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #22, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #43+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #22, #21+        add     x15, x5, x15+        asr     x15, x15, #43+        mneg    x2, x12, x8+        mneg    x3, x12, x9+        mneg    x4, x14, x8+        mneg    x5, x14, x9+        msub    x10, x13, x16, x2+        msub    x11, x13, x17, x3+        msub    x12, x15, x16, x4+        msub    x13, x15, x17, x5+        mov     x22, x1+        subs    x21, x21, #0x1+        b.ne    Lp256_scalarmul_inv_loop+        ldr     x0, [sp]+        ldr     x1, [sp, #48]+        mul     x0, x0, x10+        madd    x1, x1, x11, x0+        asr     x0, x1, #63+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        eor     x14, x14, x0+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        eor     x15, x15, x0+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        eor     x16, x16, x0+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        eor     x17, x17, x0+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        ldr     x7, [sp, #96]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #128]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #96]+        adc     x2, x2, x1+        ldr     x7, [sp, #104]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #136]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #104]+        adc     x6, x6, x1+        ldr     x7, [sp, #112]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #144]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #112]+        adc     x5, x5, x1+        ldr     x7, [sp, #120]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #152]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldp     x0, x1, [sp, #96]+        ldr     x2, [sp, #112]+        mov     x14, #0xe000000000000000+        adds    x0, x0, x14+        sbcs    x1, x1, xzr+        mov     x11, #0x1fffffff+        adcs    x2, x2, x11+        mov     x10, #0x2000000000000000+        adcs    x5, x5, x10+        mov     x14, #0x1fffffffe0000000+        adc     x3, x3, x14+        lsl     x11, x0, #32+        subs    x14, x0, x11+        lsr     x10, x0, #32+        sbc     x0, x0, x10+        adds    x1, x1, x11+        adcs    x2, x2, x10+        adcs    x5, x5, x14+        adcs    x3, x3, x0+        mov     x14, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x10, #0xffffffff00000001+        csel    x14, x14, xzr, cs+        csel    x11, x11, xzr, cs+        csel    x10, x10, xzr, cs+        subs    x1, x1, x14+        sbcs    x2, x2, x11+        sbcs    x5, x5, xzr+        sbc     x3, x3, x10+        mov     x10, #0xffffffffffffffff+        subs    x10, x1, x10+        mov     x11, #0xffffffff+        sbcs    x11, x2, x11+        mov     x13, #0xffffffff00000001+        sbcs    x12, x5, xzr+        sbcs    x13, x3, x13+        csel    x10, x1, x10, cc+        csel    x11, x2, x11, cc+        csel    x12, x5, x12, cc+        csel    x13, x3, x13, cc+        stp     x10, x11, [x20]+        stp     x12, x13, [x20, #16]+        CFI_INC_SP(160)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_local_inv_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_local_montmul_p256)++Lp256_scalarmul_local_montmul_p256:+        CFI_START+        ldr q20, [x2]+        ldp x7, x17, [x1]+        ldr q0, [x1]+        ldp x6, x10, [x2]+        ldp x11, x15, [x1, #16]+        rev64 v16.4S, v20.4S+        subs x4, x7, x17+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4S, v16.4S, v0.4S+        umulh x12, x17, x10+        uzp1 v28.4S, v20.4S, v0.4S+        subs x14, x11, x7+        ldr q20, [x2, #16]+        sbcs x5, x15, x17+        ngc x17, xzr+        subs x8, x11, x15+        uaddlp v27.2D, v16.4S+        umulh x4, x7, x6+        uzp1 v21.4S, v0.4S, v0.4S+        cneg x11, x8, cc+        shl v17.2D, v27.2D, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2D, v21.2S, v28.2S+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [x1, #16]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2S, v20.2D+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4S, v20.4S, v20.4S+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2S, v28.2D+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [x2, #16]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x9, x3, x13+        adcs x3, x8, x7+        umulh x8, x14, x11+        umull v21.2D, v0.2S, v1.2S+        adcs x12, x10, x12+        umull v3.2D, v0.2S, v16.2S+        adc x15, x15, xzr+        rev64 v24.4S, v20.4S+        stp x12, x15, [x0, #16]+        movi v2.2D, #0x00000000ffffffff+        mul x10, x14, x11+        mul v4.4S, v24.4S, v28.4S+        subs x13, x14, x5+        uzp2 v19.4S, v28.4S, v28.4S+        csetm x15, cc+        usra v3.2D, v21.2D, #32+        mul x7, x5, x1+        umull v21.2D, v19.2S, v16.2S+        cneg x13, x13, cc+        uaddlp v5.2D, v4.4S+        subs x11, x1, x11+        and v16.16B, v3.16B, v2.16B+        umulh x5, x5, x1+        shl v24.2D, v5.2D, #32+        cneg x11, x11, cc+        umlal v16.2D, v19.2S, v1.2S+        cinv x12, x15, cc+        umlal v24.2D, v0.2S, v1.2S+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        stp x9, x3, [x0]+        usra v21.2D, v3.2D, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2D, v16.2D, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        ldp x15, x8, [x0]+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        ldp x9, x13, [x0, #16]+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x15+        adcs x15, x16, x8+        eor x5, x17, x4+        adcs x9, x1, x9+        eor x1, x10, x5+        adcs x16, x2, x13+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x11, x11, x13+        and x1, x1, x13+        adcs x4, x4, x1+        and x1, x12, x13+        stp x11, x4, [x0]+        adcs x4, x7, xzr+        adc x1, x17, x1+        stp x4, x1, [x0, #16]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_local_montmul_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_local_montsqr_p256)++Lp256_scalarmul_local_montsqr_p256:+        CFI_START+        ldr q19, [x1]+        ldp x9, x13, [x1]+        ldr q23, [x1, #16]+        ldr q0, [x1]+        ldp x1, x10, [x1, #16]+        uzp2 v29.4S, v19.4S, v19.4S+        xtn v4.2S, v19.2D+        umulh x8, x9, x13+        rev64 v20.4S, v23.4S+        umull v16.2D, v19.2S, v19.2S+        umull v1.2D, v29.2S, v4.2S+        mul v20.4S, v20.4S, v0.4S+        subs x14, x9, x13+        umulh x15, x9, x1+        mov x16, v16.d[1]+        umull2 v4.2D, v19.4S, v19.4S+        mov x4, v16.d[0]+        uzp1 v17.4S, v23.4S, v0.4S+        uaddlp v19.2D, v20.4S+        lsr x7, x8, #63+        mul x11, x9, x13+        mov x12, v1.d[0]+        csetm x5, cc+        cneg x6, x14, cc+        mov x3, v4.d[1]+        mov x14, v4.d[0]+        subs x2, x10, x1+        mov x9, v1.d[1]+        cneg x17, x2, cc+        cinv x2, x5, cc+        adds x5, x4, x12, lsl #33+        extr x4, x8, x11, #63+        lsr x8, x12, #31+        uzp1 v20.4S, v0.4S, v0.4S+        shl v19.2D, v19.2D, #32+        adc x16, x16, x8+        adds x8, x14, x9, lsl #33+        lsr x14, x9, #31+        lsl x9, x5, #32+        umlal v19.2D, v20.2S, v17.2S+        adc x14, x3, x14+        adds x16, x16, x11, lsl #1+        lsr x3, x5, #32+        umulh x12, x6, x17+        adcs x4, x8, x4+        adc x11, x14, x7+        subs x8, x5, x9+        sbc x5, x5, x3+        adds x16, x16, x9+        mov x14, v19.d[0]+        mul x17, x6, x17+        adcs x3, x4, x3+        lsl x7, x16, #32+        umulh x13, x13, x10+        adcs x11, x11, x8+        lsr x8, x16, #32+        adc x5, x5, xzr+        subs x9, x16, x7+        sbc x16, x16, x8+        adds x7, x3, x7+        mov x3, v19.d[1]+        adcs x6, x11, x8+        umulh x11, x1, x10+        adcs x5, x5, x9+        eor x8, x12, x2+        adc x9, x16, xzr+        adds x16, x14, x15+        adc x15, x15, xzr+        adds x12, x16, x3+        eor x16, x17, x2+        mul x4, x1, x10+        adcs x15, x15, x13+        adc x17, x13, xzr+        adds x15, x15, x3+        adc x3, x17, xzr+        cmn x2, #0x1+        mul x17, x10, x10+        adcs x12, x12, x16+        adcs x16, x15, x8+        umulh x10, x10, x10+        adc x2, x3, x2+        adds x14, x14, x14+        adcs x12, x12, x12+        adcs x16, x16, x16+        adcs x2, x2, x2+        adc x15, xzr, xzr+        adds x14, x14, x7+        mul x3, x1, x1+        adcs x12, x12, x6+        lsr x7, x14, #32+        adcs x16, x16, x5+        lsl x5, x14, #32+        umulh x13, x1, x1+        adcs x2, x2, x9+        mov x6, #0xffffffff+        adc x15, x15, xzr+        adds x8, x4, x4+        adcs x1, x11, x11+        mov x11, #0xffffffff00000001+        adc x4, xzr, xzr+        subs x9, x14, x5+        sbc x14, x14, x7+        adds x12, x12, x5+        adcs x16, x16, x7+        lsl x5, x12, #32+        lsr x7, x12, #32+        adcs x2, x2, x9+        adcs x14, x15, x14+        adc x15, xzr, xzr+        subs x9, x12, x5+        sbc x12, x12, x7+        adds x16, x16, x5+        adcs x2, x2, x7+        adcs x14, x14, x9+        adcs x12, x15, x12+        adc x15, xzr, xzr+        adds x16, x16, x3+        adcs x2, x2, x13+        adcs x14, x14, x17+        adcs x12, x12, x10+        adc x15, x15, xzr+        adds x2, x2, x8+        adcs x14, x14, x1+        adcs x12, x12, x4+        adcs x15, x15, xzr+        adds x3, x16, #0x1+        sbcs x5, x2, x6+        sbcs x8, x14, xzr+        sbcs x11, x12, x11+        sbcs xzr, x15, xzr+        csel x16, x3, x16, cs+        csel x14, x8, x14, cs+        csel x12, x11, x12, cs+        csel x2, x5, x2, cs+        stp x14, x12, [x0, #16]+        stp x16, x2, [x0]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_local_montsqr_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_local_tomont_p256)++Lp256_scalarmul_local_tomont_p256:+        CFI_START+        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]+        mov     x1, #0xffffffffffffffff+        mov     x7, #0xffffffff+        mov     x9, #0xffffffff00000001+        subs    x1, x2, x1+        sbcs    x7, x3, x7+        sbcs    x8, x4, xzr+        sbcs    x9, x5, x9+        csel    x2, x2, x1, cc+        csel    x3, x3, x7, cc+        csel    x4, x4, x8, cc+        csel    x5, x5, x9, cc+        cmp     xzr, xzr+        extr    x9, x5, x4, #32+        adcs    xzr, x4, x9+        lsr     x9, x5, #32+        adcs    x9, x5, x9+        csetm   x6, cs+        orr     x9, x9, x6+        lsl     x7, x9, #32+        lsr     x8, x9, #32+        adds    x4, x4, x7+        adc     x5, x5, x8+        negs    x6, x9+        sbcs    x7, x7, xzr+        sbc     x8, x8, xzr+        negs    x6, x6+        sbcs    x2, x2, x7+        sbcs    x3, x3, x8+        sbcs    x4, x4, x9+        sbcs    x5, x5, x9+        adds    x6, x6, x5+        mov     x7, #0xffffffff+        and     x7, x7, x5+        adcs    x2, x2, x7+        adcs    x3, x3, xzr+        mov     x7, #0xffffffff00000001+        and     x7, x7, x5+        adc     x4, x4, x7+        cmp     xzr, xzr+        extr    x9, x4, x3, #32+        adcs    xzr, x3, x9+        lsr     x9, x4, #32+        adcs    x9, x4, x9+        csetm   x5, cs+        orr     x9, x9, x5+        lsl     x7, x9, #32+        lsr     x8, x9, #32+        adds    x3, x3, x7+        adc     x4, x4, x8+        negs    x5, x9+        sbcs    x7, x7, xzr+        sbc     x8, x8, xzr+        negs    x5, x5+        sbcs    x6, x6, x7+        sbcs    x2, x2, x8+        sbcs    x3, x3, x9+        sbcs    x4, x4, x9+        adds    x5, x5, x4+        mov     x7, #0xffffffff+        and     x7, x7, x4+        adcs    x6, x6, x7+        adcs    x2, x2, xzr+        mov     x7, #0xffffffff00000001+        and     x7, x7, x4+        adc     x3, x3, x7+        cmp     xzr, xzr+        extr    x9, x3, x2, #32+        adcs    xzr, x2, x9+        lsr     x9, x3, #32+        adcs    x9, x3, x9+        csetm   x4, cs+        orr     x9, x9, x4+        lsl     x7, x9, #32+        lsr     x8, x9, #32+        adds    x2, x2, x7+        adc     x3, x3, x8+        negs    x4, x9+        sbcs    x7, x7, xzr+        sbc     x8, x8, xzr+        negs    x4, x4+        sbcs    x5, x5, x7+        sbcs    x6, x6, x8+        sbcs    x2, x2, x9+        sbcs    x3, x3, x9+        adds    x4, x4, x3+        mov     x7, #0xffffffff+        and     x7, x7, x3+        adcs    x5, x5, x7+        adcs    x6, x6, xzr+        mov     x7, #0xffffffff00000001+        and     x7, x7, x3+        adc     x2, x2, x7+        cmp     xzr, xzr+        extr    x9, x2, x6, #32+        adcs    xzr, x6, x9+        lsr     x9, x2, #32+        adcs    x9, x2, x9+        csetm   x3, cs+        orr     x9, x9, x3+        lsl     x7, x9, #32+        lsr     x8, x9, #32+        adds    x6, x6, x7+        adc     x2, x2, x8+        negs    x3, x9+        sbcs    x7, x7, xzr+        sbc     x8, x8, xzr+        negs    x3, x3+        sbcs    x4, x4, x7+        sbcs    x5, x5, x8+        sbcs    x6, x6, x9+        sbcs    x2, x2, x9+        adds    x3, x3, x2+        mov     x7, #0xffffffff+        and     x7, x7, x2+        adcs    x4, x4, x7+        adcs    x5, x5, xzr+        mov     x7, #0xffffffff00000001+        and     x7, x7, x2+        adc     x6, x6, x7+        stp     x3, x4, [x0]+        stp     x5, x6, [x0, #16]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_local_tomont_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_local_p256_montjadd)++Lp256_scalarmul_local_p256_montjadd:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x26)+        CFI_PUSH2(x27,x30)+        CFI_DEC_SP(224)+        mov     x21, x0+        mov     x22, x1+        mov     x23, x2+        mov     x0, sp+        ldr     q19, [x22, #64]+        ldp     x9, x13, [x22, #64]+        ldr     q23, [x22, #80]+        ldr     q0, [x22, #64]+        ldp     x1, x10, [x22, #80]+        uzp2    v29.4s, v19.4s, v19.4s+        xtn     v4.2s, v19.2d+        umulh   x8, x9, x13+        rev64   v20.4s, v23.4s+        umull   v16.2d, v19.2s, v19.2s+        umull   v1.2d, v29.2s, v4.2s+        mul     v20.4s, v20.4s, v0.4s+        subs    x14, x9, x13+        umulh   x15, x9, x1+        mov     x16, v16.d[1]+        umull2  v4.2d, v19.4s, v19.4s+        mov     x4, v16.d[0]+        uzp1    v17.4s, v23.4s, v0.4s+        uaddlp  v19.2d, v20.4s+        lsr     x7, x8, #63+        mul     x11, x9, x13+        mov     x12, v1.d[0]+        csetm   x5, cc  // cc = lo, ul, last+        cneg    x6, x14, cc  // cc = lo, ul, last+        mov     x3, v4.d[1]+        mov     x14, v4.d[0]+        subs    x2, x10, x1+        mov     x9, v1.d[1]+        cneg    x17, x2, cc  // cc = lo, ul, last+        cinv    x2, x5, cc  // cc = lo, ul, last+        adds    x5, x4, x12, lsl #33+        extr    x4, x8, x11, #63+        lsr     x8, x12, #31+        uzp1    v20.4s, v0.4s, v0.4s+        shl     v19.2d, v19.2d, #32+        adc     x16, x16, x8+        adds    x8, x14, x9, lsl #33+        lsr     x14, x9, #31+        lsl     x9, x5, #32+        umlal   v19.2d, v20.2s, v17.2s+        adc     x14, x3, x14+        adds    x16, x16, x11, lsl #1+        lsr     x3, x5, #32+        umulh   x12, x6, x17+        adcs    x4, x8, x4+        adc     x11, x14, x7+        subs    x8, x5, x9+        sbc     x5, x5, x3+        adds    x16, x16, x9+        mov     x14, v19.d[0]+        mul     x17, x6, x17+        adcs    x3, x4, x3+        lsl     x7, x16, #32+        umulh   x13, x13, x10+        adcs    x11, x11, x8+        lsr     x8, x16, #32+        adc     x5, x5, xzr+        subs    x9, x16, x7+        sbc     x16, x16, x8+        adds    x7, x3, x7+        mov     x3, v19.d[1]+        adcs    x6, x11, x8+        umulh   x11, x1, x10+        adcs    x5, x5, x9+        eor     x8, x12, x2+        adc     x9, x16, xzr+        adds    x16, x14, x15+        adc     x15, x15, xzr+        adds    x12, x16, x3+        eor     x16, x17, x2+        mul     x4, x1, x10+        adcs    x15, x15, x13+        adc     x17, x13, xzr+        adds    x15, x15, x3+        adc     x3, x17, xzr+        cmn     x2, #0x1+        mul     x17, x10, x10+        adcs    x12, x12, x16+        adcs    x16, x15, x8+        umulh   x10, x10, x10+        adc     x2, x3, x2+        adds    x14, x14, x14+        adcs    x12, x12, x12+        adcs    x16, x16, x16+        adcs    x2, x2, x2+        adc     x15, xzr, xzr+        adds    x14, x14, x7+        mul     x3, x1, x1+        adcs    x12, x12, x6+        lsr     x7, x14, #32+        adcs    x16, x16, x5+        lsl     x5, x14, #32+        umulh   x13, x1, x1+        adcs    x2, x2, x9+        mov     x6, #0xffffffff                 // #4294967295+        adc     x15, x15, xzr+        adds    x8, x4, x4+        adcs    x1, x11, x11+        mov     x11, #0xffffffff00000001        // #-4294967295+        adc     x4, xzr, xzr+        subs    x9, x14, x5+        sbc     x14, x14, x7+        adds    x12, x12, x5+        adcs    x16, x16, x7+        lsl     x5, x12, #32+        lsr     x7, x12, #32+        adcs    x2, x2, x9+        adcs    x14, x15, x14+        adc     x15, xzr, xzr+        subs    x9, x12, x5+        sbc     x12, x12, x7+        adds    x16, x16, x5+        adcs    x2, x2, x7+        adcs    x14, x14, x9+        adcs    x12, x15, x12+        adc     x15, xzr, xzr+        adds    x16, x16, x3+        adcs    x2, x2, x13+        adcs    x14, x14, x17+        adcs    x12, x12, x10+        adc     x15, x15, xzr+        adds    x2, x2, x8+        adcs    x14, x14, x1+        adcs    x12, x12, x4+        adcs    x15, x15, xzr+        adds    x3, x16, #0x1+        sbcs    x5, x2, x6+        sbcs    x8, x14, xzr+        sbcs    x11, x12, x11+        sbcs    xzr, x15, xzr+        csel    x19, x3, x16, cs  // cs = hs, nlast+        csel    x14, x8, x14, cs  // cs = hs, nlast+        csel    x12, x11, x12, cs  // cs = hs, nlast+        csel    x20, x5, x2, cs  // cs = hs, nlast+        stp     x14, x12, [x0, #16]+        stp     x19, x20, [x0]+        ldr     q19, [x23, #64]+        ldp     x9, x13, [x23, #64]+        ldr     q23, [x23, #80]+        ldr     q0, [x23, #64]+        ldp     x1, x10, [x23, #80]+        uzp2    v29.4s, v19.4s, v19.4s+        xtn     v4.2s, v19.2d+        umulh   x8, x9, x13+        rev64   v20.4s, v23.4s+        umull   v16.2d, v19.2s, v19.2s+        umull   v1.2d, v29.2s, v4.2s+        mul     v20.4s, v20.4s, v0.4s+        subs    x14, x9, x13+        umulh   x15, x9, x1+        mov     x16, v16.d[1]+        umull2  v4.2d, v19.4s, v19.4s+        mov     x4, v16.d[0]+        uzp1    v17.4s, v23.4s, v0.4s+        uaddlp  v19.2d, v20.4s+        lsr     x7, x8, #63+        mul     x11, x9, x13+        mov     x12, v1.d[0]+        csetm   x5, cc  // cc = lo, ul, last+        cneg    x6, x14, cc  // cc = lo, ul, last+        mov     x3, v4.d[1]+        mov     x14, v4.d[0]+        subs    x2, x10, x1+        mov     x9, v1.d[1]+        cneg    x17, x2, cc  // cc = lo, ul, last+        cinv    x2, x5, cc  // cc = lo, ul, last+        adds    x5, x4, x12, lsl #33+        extr    x4, x8, x11, #63+        lsr     x8, x12, #31+        uzp1    v20.4s, v0.4s, v0.4s+        shl     v19.2d, v19.2d, #32+        adc     x16, x16, x8+        adds    x8, x14, x9, lsl #33+        lsr     x14, x9, #31+        lsl     x9, x5, #32+        umlal   v19.2d, v20.2s, v17.2s+        adc     x14, x3, x14+        adds    x16, x16, x11, lsl #1+        lsr     x3, x5, #32+        umulh   x12, x6, x17+        adcs    x4, x8, x4+        adc     x11, x14, x7+        subs    x8, x5, x9+        sbc     x5, x5, x3+        adds    x16, x16, x9+        mov     x14, v19.d[0]+        mul     x17, x6, x17+        adcs    x3, x4, x3+        lsl     x7, x16, #32+        umulh   x13, x13, x10+        adcs    x11, x11, x8+        lsr     x8, x16, #32+        adc     x5, x5, xzr+        subs    x9, x16, x7+        sbc     x16, x16, x8+        adds    x7, x3, x7+        mov     x3, v19.d[1]+        adcs    x6, x11, x8+        umulh   x11, x1, x10+        adcs    x5, x5, x9+        eor     x8, x12, x2+        adc     x9, x16, xzr+        adds    x16, x14, x15+        adc     x15, x15, xzr+        adds    x12, x16, x3+        eor     x16, x17, x2+        mul     x4, x1, x10+        adcs    x15, x15, x13+        adc     x17, x13, xzr+        adds    x15, x15, x3+        adc     x3, x17, xzr+        cmn     x2, #0x1+        mul     x17, x10, x10+        adcs    x12, x12, x16+        adcs    x16, x15, x8+        umulh   x10, x10, x10+        adc     x2, x3, x2+        adds    x14, x14, x14+        adcs    x12, x12, x12+        adcs    x16, x16, x16+        adcs    x2, x2, x2+        adc     x15, xzr, xzr+        adds    x14, x14, x7+        mul     x3, x1, x1+        adcs    x12, x12, x6+        lsr     x7, x14, #32+        adcs    x16, x16, x5+        lsl     x5, x14, #32+        umulh   x13, x1, x1+        adcs    x2, x2, x9+        mov     x6, #0xffffffff                 // #4294967295+        adc     x15, x15, xzr+        adds    x8, x4, x4+        adcs    x1, x11, x11+        mov     x11, #0xffffffff00000001        // #-4294967295+        adc     x4, xzr, xzr+        subs    x9, x14, x5+        sbc     x14, x14, x7+        adds    x12, x12, x5+        adcs    x16, x16, x7+        lsl     x5, x12, #32+        lsr     x7, x12, #32+        adcs    x2, x2, x9+        adcs    x14, x15, x14+        adc     x15, xzr, xzr+        subs    x9, x12, x5+        sbc     x12, x12, x7+        adds    x16, x16, x5+        adcs    x2, x2, x7+        adcs    x14, x14, x9+        adcs    x12, x15, x12+        adc     x15, xzr, xzr+        adds    x16, x16, x3+        adcs    x2, x2, x13+        adcs    x14, x14, x17+        adcs    x12, x12, x10+        adc     x15, x15, xzr+        adds    x2, x2, x8+        adcs    x14, x14, x1+        adcs    x12, x12, x4+        adcs    x15, x15, xzr+        adds    x3, x16, #0x1+        sbcs    x5, x2, x6+        sbcs    x8, x14, xzr+        sbcs    x11, x12, x11+        sbcs    xzr, x15, xzr+        csel    x16, x3, x16, cs  // cs = hs, nlast+        csel    x14, x8, x14, cs  // cs = hs, nlast+        csel    x12, x11, x12, cs  // cs = hs, nlast+        csel    x2, x5, x2, cs  // cs = hs, nlast+        stp     x14, x12, [sp, #176]+        stp     x16, x2, [sp, #160]+        ldr     q20, [x22, #32]+        ldp     x7, x17, [x23, #64]+        ldr     q0, [x23, #64]+        ldp     x6, x10, [x22, #32]+        ldp     x11, x15, [x23, #80]+        rev64   v16.4s, v20.4s+        subs    x4, x7, x17+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x17, x10+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x11, x7+        ldr     q20, [x22, #48]+        sbcs    x5, x15, x17+        ngc     x17, xzr+        subs    x8, x11, x15+        uaddlp  v27.2d, v16.4s+        umulh   x4, x7, x6+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x10, x6+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [x23, #80]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        ldp     x7, x2, [x22, #48]+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x6, x7+        sbcs    x9, x10, x2+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x2, x7+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x24, x3, x13+        adcs    x25, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x26, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x27, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x24+        adcs    x15, x16, x25+        eor     x5, x17, x4+        adcs    x9, x1, x26+        eor     x1, x10, x5+        adcs    x16, x2, x27+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x11, x11, x13+        and     x1, x1, x13+        adcs    x4, x4, x1+        and     x1, x12, x13+        stp     x11, x4, [sp, #192]+        adcs    x4, x7, xzr+        adc     x1, x17, x1+        stp     x4, x1, [sp, #208]+        ldr     q20, [x23, #32]+        ldp     x7, x17, [x22, #64]+        ldr     q0, [x22, #64]+        ldp     x6, x10, [x23, #32]+        ldp     x11, x15, [x22, #80]+        rev64   v16.4s, v20.4s+        subs    x4, x7, x17+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x17, x10+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x11, x7+        ldr     q20, [x23, #48]+        sbcs    x5, x15, x17+        ngc     x17, xzr+        subs    x8, x11, x15+        uaddlp  v27.2d, v16.4s+        umulh   x4, x7, x6+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x10, x6+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [x22, #80]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        ldp     x7, x2, [x23, #48]+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x6, x7+        sbcs    x9, x10, x2+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x2, x7+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x24, x3, x13+        adcs    x25, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x26, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x27, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x24+        adcs    x15, x16, x25+        eor     x5, x17, x4+        adcs    x9, x1, x26+        eor     x1, x10, x5+        adcs    x16, x2, x27+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x24, x11, x13+        and     x1, x1, x13+        adcs    x25, x4, x1+        and     x1, x12, x13+        stp     x24, x25, [sp, #32]+        adcs    x4, x7, xzr+        adc     x1, x17, x1+        stp     x4, x1, [sp, #48]+        mov     x1, sp+        ldr     q20, [x23]+        ldr     q0, [x1]+        ldp     x6, x10, [x23]+        ldp     x11, x15, [x1, #16]+        rev64   v16.4s, v20.4s+        subs    x4, x19, x20+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x20, x10+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x11, x19+        ldr     q20, [x23, #16]+        sbcs    x5, x15, x20+        ngc     x17, xzr+        subs    x8, x11, x15+        uaddlp  v27.2d, v16.4s+        umulh   x4, x19, x6+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x10, x6+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [x1, #16]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        ldp     x7, x2, [x23, #16]+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x6, x7+        sbcs    x9, x10, x2+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x2, x7+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x19, x3, x13+        adcs    x20, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x26, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x27, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x19+        adcs    x15, x16, x20+        eor     x5, x17, x4+        adcs    x9, x1, x26+        eor     x1, x10, x5+        adcs    x16, x2, x27+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x11, x11, x13+        and     x1, x1, x13+        adcs    x4, x4, x1+        and     x1, x12, x13+        stp     x11, x4, [sp, #64]+        adcs    x4, x7, xzr+        adc     x1, x17, x1+        stp     x4, x1, [sp, #80]+        ldr     q20, [x22]+        ldp     x7, x17, [sp, #160]+        ldr     q0, [sp, #160]+        ldp     x6, x10, [x22]+        ldp     x11, x15, [sp, #176]+        rev64   v16.4s, v20.4s+        subs    x4, x7, x17+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x17, x10+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x11, x7+        ldr     q20, [x22, #16]+        sbcs    x5, x15, x17+        ngc     x17, xzr+        subs    x8, x11, x15+        uaddlp  v27.2d, v16.4s+        umulh   x4, x7, x6+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x10, x6+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [sp, #176]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        ldp     x7, x2, [x22, #16]+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x6, x7+        sbcs    x9, x10, x2+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x2, x7+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x19, x3, x13+        adcs    x20, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x26, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x27, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x19+        adcs    x15, x16, x20+        eor     x5, x17, x4+        adcs    x9, x1, x26+        eor     x1, x10, x5+        adcs    x16, x2, x27+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x11, x11, x13+        and     x1, x1, x13+        adcs    x4, x4, x1+        and     x1, x12, x13+        stp     x11, x4, [sp, #128]+        adcs    x4, x7, xzr+        adc     x1, x17, x1+        stp     x4, x1, [sp, #144]+        mov     x1, sp+        ldr     q20, [sp, #32]+        ldp     x7, x17, [x1]+        ldr     q0, [x1]+        ldp     x11, x15, [x1, #16]+        rev64   v16.4s, v20.4s+        subs    x4, x7, x17+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x17, x25+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x11, x7+        ldr     q20, [sp, #48]+        sbcs    x5, x15, x17+        ngc     x17, xzr+        subs    x8, x11, x15+        uaddlp  v27.2d, v16.4s+        umulh   x4, x7, x24+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x25, x24+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [x1, #16]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        ldp     x7, x2, [sp, #48]+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x24, x7+        sbcs    x9, x25, x2+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x2, x7+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x19, x3, x13+        adcs    x20, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x24, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x25, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x19+        adcs    x15, x16, x20+        eor     x5, x17, x4+        adcs    x9, x1, x24+        eor     x1, x10, x5+        adcs    x16, x2, x25+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x19, x11, x13+        and     x1, x1, x13+        adcs    x20, x4, x1+        and     x1, x12, x13+        adcs    x4, x7, xzr+        adc     x1, x17, x1+        stp     x4, x1, [sp, #48]+        ldr     q20, [sp, #192]+        ldp     x7, x17, [sp, #160]+        ldr     q0, [sp, #160]+        ldp     x6, x10, [sp, #192]+        ldp     x11, x15, [sp, #176]+        rev64   v16.4s, v20.4s+        subs    x4, x7, x17+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x17, x10+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x11, x7+        ldr     q20, [sp, #208]+        sbcs    x5, x15, x17+        ngc     x17, xzr+        subs    x8, x11, x15+        uaddlp  v27.2d, v16.4s+        umulh   x4, x7, x6+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x10, x6+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [sp, #176]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        ldp     x7, x2, [sp, #208]+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x6, x7+        sbcs    x9, x10, x2+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x2, x7+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x24, x3, x13+        adcs    x25, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x26, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x27, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x24+        adcs    x15, x16, x25+        eor     x5, x17, x4+        adcs    x9, x1, x26+        eor     x1, x10, x5+        adcs    x16, x2, x27+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x9, x11, x13+        and     x1, x1, x13+        adcs    x10, x4, x1+        and     x1, x12, x13+        stp     x9, x10, [sp, #192]+        adcs    x11, x7, xzr+        adc     x12, x17, x1+        stp     x11, x12, [sp, #208]+        ldp     x5, x6, [sp, #64]+        ldp     x4, x3, [sp, #128]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #80]+        ldp     x4, x3, [sp, #144]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc  // cc = lo, ul, last+        adds    x13, x5, x3+        and     x4, x3, #0xffffffff+        adcs    x24, x6, x4+        adcs    x25, x7, xzr+        and     x4, x3, #0xffffffff00000001+        adc     x26, x8, x4+        stp     x13, x24, [sp, #160]+        stp     x25, x26, [sp, #176]+        subs    x5, x19, x9+        sbcs    x6, x20, x10+        ldp     x7, x8, [sp, #48]+        sbcs    x7, x7, x11+        sbcs    x8, x8, x12+        csetm   x3, cc  // cc = lo, ul, last+        adds    x19, x5, x3+        and     x4, x3, #0xffffffff+        adcs    x20, x6, x4+        adcs    x7, x7, xzr+        and     x4, x3, #0xffffffff00000001+        adc     x8, x8, x4+        stp     x19, x20, [sp, #32]+        stp     x7, x8, [sp, #48]+        ldr     q19, [sp, #160]+        ldr     q23, [sp, #176]+        ldr     q0, [sp, #160]+        uzp2    v29.4s, v19.4s, v19.4s+        xtn     v4.2s, v19.2d+        umulh   x8, x13, x24+        rev64   v20.4s, v23.4s+        umull   v16.2d, v19.2s, v19.2s+        umull   v1.2d, v29.2s, v4.2s+        mul     v20.4s, v20.4s, v0.4s+        subs    x14, x13, x24+        umulh   x15, x13, x25+        mov     x16, v16.d[1]+        umull2  v4.2d, v19.4s, v19.4s+        mov     x4, v16.d[0]+        uzp1    v17.4s, v23.4s, v0.4s+        uaddlp  v19.2d, v20.4s+        lsr     x7, x8, #63+        mul     x11, x13, x24+        mov     x12, v1.d[0]+        csetm   x5, cc  // cc = lo, ul, last+        cneg    x6, x14, cc  // cc = lo, ul, last+        mov     x3, v4.d[1]+        mov     x14, v4.d[0]+        subs    x2, x26, x25+        mov     x9, v1.d[1]+        cneg    x17, x2, cc  // cc = lo, ul, last+        cinv    x2, x5, cc  // cc = lo, ul, last+        adds    x5, x4, x12, lsl #33+        extr    x4, x8, x11, #63+        lsr     x8, x12, #31+        uzp1    v20.4s, v0.4s, v0.4s+        shl     v19.2d, v19.2d, #32+        adc     x16, x16, x8+        adds    x8, x14, x9, lsl #33+        lsr     x14, x9, #31+        lsl     x9, x5, #32+        umlal   v19.2d, v20.2s, v17.2s+        adc     x14, x3, x14+        adds    x16, x16, x11, lsl #1+        lsr     x3, x5, #32+        umulh   x12, x6, x17+        adcs    x4, x8, x4+        adc     x11, x14, x7+        subs    x8, x5, x9+        sbc     x5, x5, x3+        adds    x16, x16, x9+        mov     x14, v19.d[0]+        mul     x17, x6, x17+        adcs    x3, x4, x3+        lsl     x7, x16, #32+        umulh   x13, x24, x26+        adcs    x11, x11, x8+        lsr     x8, x16, #32+        adc     x5, x5, xzr+        subs    x9, x16, x7+        sbc     x16, x16, x8+        adds    x7, x3, x7+        mov     x3, v19.d[1]+        adcs    x6, x11, x8+        umulh   x11, x25, x26+        adcs    x5, x5, x9+        eor     x8, x12, x2+        adc     x9, x16, xzr+        adds    x16, x14, x15+        adc     x15, x15, xzr+        adds    x12, x16, x3+        eor     x16, x17, x2+        mul     x4, x25, x26+        adcs    x15, x15, x13+        adc     x17, x13, xzr+        adds    x15, x15, x3+        adc     x3, x17, xzr+        cmn     x2, #0x1+        mul     x17, x26, x26+        adcs    x12, x12, x16+        adcs    x16, x15, x8+        umulh   x10, x26, x26+        adc     x2, x3, x2+        adds    x14, x14, x14+        adcs    x12, x12, x12+        adcs    x16, x16, x16+        adcs    x2, x2, x2+        adc     x15, xzr, xzr+        adds    x14, x14, x7+        mul     x3, x25, x25+        adcs    x12, x12, x6+        lsr     x7, x14, #32+        adcs    x16, x16, x5+        lsl     x5, x14, #32+        umulh   x13, x25, x25+        adcs    x2, x2, x9+        mov     x6, #0xffffffff                 // #4294967295+        adc     x15, x15, xzr+        adds    x8, x4, x4+        adcs    x1, x11, x11+        mov     x11, #0xffffffff00000001        // #-4294967295+        adc     x4, xzr, xzr+        subs    x9, x14, x5+        sbc     x14, x14, x7+        adds    x12, x12, x5+        adcs    x16, x16, x7+        lsl     x5, x12, #32+        lsr     x7, x12, #32+        adcs    x2, x2, x9+        adcs    x14, x15, x14+        adc     x15, xzr, xzr+        subs    x9, x12, x5+        sbc     x12, x12, x7+        adds    x16, x16, x5+        adcs    x2, x2, x7+        adcs    x14, x14, x9+        adcs    x12, x15, x12+        adc     x15, xzr, xzr+        adds    x16, x16, x3+        adcs    x2, x2, x13+        adcs    x14, x14, x17+        adcs    x12, x12, x10+        adc     x15, x15, xzr+        adds    x2, x2, x8+        adcs    x14, x14, x1+        adcs    x12, x12, x4+        adcs    x15, x15, xzr+        adds    x3, x16, #0x1+        sbcs    x5, x2, x6+        sbcs    x8, x14, xzr+        sbcs    x11, x12, x11+        sbcs    xzr, x15, xzr+        csel    x24, x3, x16, cs  // cs = hs, nlast+        csel    x25, x8, x14, cs  // cs = hs, nlast+        csel    x26, x11, x12, cs  // cs = hs, nlast+        csel    x27, x5, x2, cs  // cs = hs, nlast+        stp     x25, x26, [sp, #112]+        stp     x24, x27, [sp, #96]+        mov     x0, sp+        ldr     q19, [sp, #32]+        ldr     q23, [sp, #48]+        ldr     q0, [sp, #32]+        ldp     x1, x10, [sp, #48]+        uzp2    v29.4s, v19.4s, v19.4s+        xtn     v4.2s, v19.2d+        umulh   x8, x19, x20+        rev64   v20.4s, v23.4s+        umull   v16.2d, v19.2s, v19.2s+        umull   v1.2d, v29.2s, v4.2s+        mul     v20.4s, v20.4s, v0.4s+        subs    x14, x19, x20+        umulh   x15, x19, x1+        mov     x16, v16.d[1]+        umull2  v4.2d, v19.4s, v19.4s+        mov     x4, v16.d[0]+        uzp1    v17.4s, v23.4s, v0.4s+        uaddlp  v19.2d, v20.4s+        lsr     x7, x8, #63+        mul     x11, x19, x20+        mov     x12, v1.d[0]+        csetm   x5, cc  // cc = lo, ul, last+        cneg    x6, x14, cc  // cc = lo, ul, last+        mov     x3, v4.d[1]+        mov     x14, v4.d[0]+        subs    x2, x10, x1+        mov     x9, v1.d[1]+        cneg    x17, x2, cc  // cc = lo, ul, last+        cinv    x2, x5, cc  // cc = lo, ul, last+        adds    x5, x4, x12, lsl #33+        extr    x4, x8, x11, #63+        lsr     x8, x12, #31+        uzp1    v20.4s, v0.4s, v0.4s+        shl     v19.2d, v19.2d, #32+        adc     x16, x16, x8+        adds    x8, x14, x9, lsl #33+        lsr     x14, x9, #31+        lsl     x9, x5, #32+        umlal   v19.2d, v20.2s, v17.2s+        adc     x14, x3, x14+        adds    x16, x16, x11, lsl #1+        lsr     x3, x5, #32+        umulh   x12, x6, x17+        adcs    x4, x8, x4+        adc     x11, x14, x7+        subs    x8, x5, x9+        sbc     x5, x5, x3+        adds    x16, x16, x9+        mov     x14, v19.d[0]+        mul     x17, x6, x17+        adcs    x3, x4, x3+        lsl     x7, x16, #32+        umulh   x13, x20, x10+        adcs    x11, x11, x8+        lsr     x8, x16, #32+        adc     x5, x5, xzr+        subs    x9, x16, x7+        sbc     x16, x16, x8+        adds    x7, x3, x7+        mov     x3, v19.d[1]+        adcs    x6, x11, x8+        umulh   x11, x1, x10+        adcs    x5, x5, x9+        eor     x8, x12, x2+        adc     x9, x16, xzr+        adds    x16, x14, x15+        adc     x15, x15, xzr+        adds    x12, x16, x3+        eor     x16, x17, x2+        mul     x4, x1, x10+        adcs    x15, x15, x13+        adc     x17, x13, xzr+        adds    x15, x15, x3+        adc     x3, x17, xzr+        cmn     x2, #0x1+        mul     x17, x10, x10+        adcs    x12, x12, x16+        adcs    x16, x15, x8+        umulh   x10, x10, x10+        adc     x2, x3, x2+        adds    x14, x14, x14+        adcs    x12, x12, x12+        adcs    x16, x16, x16+        adcs    x2, x2, x2+        adc     x15, xzr, xzr+        adds    x14, x14, x7+        mul     x3, x1, x1+        adcs    x12, x12, x6+        lsr     x7, x14, #32+        adcs    x16, x16, x5+        lsl     x5, x14, #32+        umulh   x13, x1, x1+        adcs    x2, x2, x9+        mov     x6, #0xffffffff                 // #4294967295+        adc     x15, x15, xzr+        adds    x8, x4, x4+        adcs    x1, x11, x11+        mov     x11, #0xffffffff00000001        // #-4294967295+        adc     x4, xzr, xzr+        subs    x9, x14, x5+        sbc     x14, x14, x7+        adds    x12, x12, x5+        adcs    x16, x16, x7+        lsl     x5, x12, #32+        lsr     x7, x12, #32+        adcs    x2, x2, x9+        adcs    x14, x15, x14+        adc     x15, xzr, xzr+        subs    x9, x12, x5+        sbc     x12, x12, x7+        adds    x16, x16, x5+        adcs    x2, x2, x7+        adcs    x14, x14, x9+        adcs    x12, x15, x12+        adc     x15, xzr, xzr+        adds    x16, x16, x3+        adcs    x2, x2, x13+        adcs    x14, x14, x17+        adcs    x12, x12, x10+        adc     x15, x15, xzr+        adds    x2, x2, x8+        adcs    x14, x14, x1+        adcs    x12, x12, x4+        adcs    x15, x15, xzr+        adds    x3, x16, #0x1+        sbcs    x5, x2, x6+        sbcs    x8, x14, xzr+        sbcs    x11, x12, x11+        sbcs    xzr, x15, xzr+        csel    x16, x3, x16, cs  // cs = hs, nlast+        csel    x14, x8, x14, cs  // cs = hs, nlast+        csel    x12, x11, x12, cs  // cs = hs, nlast+        csel    x2, x5, x2, cs  // cs = hs, nlast+        stp     x14, x12, [x0, #16]+        stp     x16, x2, [x0]+        ldr     q20, [sp, #128]+        ldr     q0, [sp, #96]+        ldp     x6, x10, [sp, #128]+        rev64   v16.4s, v20.4s+        subs    x4, x24, x27+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x27, x10+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x25, x24+        ldr     q20, [sp, #144]+        sbcs    x5, x26, x27+        ngc     x17, xzr+        subs    x8, x25, x26+        uaddlp  v27.2d, v16.4s+        umulh   x4, x24, x6+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x10, x6+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [sp, #112]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        ldp     x7, x2, [sp, #144]+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x6, x7+        sbcs    x9, x10, x2+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x2, x7+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x19, x3, x13+        adcs    x20, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x25, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x26, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x19+        adcs    x15, x16, x20+        eor     x5, x17, x4+        adcs    x9, x1, x25+        eor     x1, x10, x5+        adcs    x16, x2, x26+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x19, x11, x13+        and     x1, x1, x13+        adcs    x20, x4, x1+        and     x1, x12, x13+        stp     x19, x20, [sp, #128]+        adcs    x4, x7, xzr+        adc     x1, x17, x1+        stp     x4, x1, [sp, #144]+        ldr     q20, [sp, #64]+        ldr     q0, [sp, #96]+        ldp     x6, x10, [sp, #64]+        ldp     x11, x15, [sp, #112]+        rev64   v16.4s, v20.4s+        subs    x4, x24, x27+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x27, x10+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x11, x24+        ldr     q20, [sp, #80]+        sbcs    x5, x15, x27+        ngc     x17, xzr+        subs    x8, x11, x15+        uaddlp  v27.2d, v16.4s+        umulh   x4, x24, x6+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x10, x6+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [sp, #112]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        ldp     x7, x2, [sp, #80]+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x6, x7+        sbcs    x9, x10, x2+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x2, x7+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x24, x3, x13+        adcs    x25, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x26, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x27, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x24+        adcs    x15, x16, x25+        eor     x5, x17, x4+        adcs    x9, x1, x26+        eor     x1, x10, x5+        adcs    x16, x2, x27+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x9, x11, x13+        and     x1, x1, x13+        adcs    x10, x4, x1+        and     x1, x12, x13+        stp     x9, x10, [sp, #64]+        adcs    x11, x7, xzr+        adc     x12, x17, x1+        stp     x11, x12, [sp, #80]+        mov     x0, sp+        mov     x1, sp+        ldp     x5, x6, [x1]+        subs    x5, x5, x19+        sbcs    x6, x6, x20+        ldp     x7, x8, [x1, #16]+        ldp     x4, x3, [sp, #144]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc  // cc = lo, ul, last+        adds    x24, x5, x3+        and     x4, x3, #0xffffffff+        adcs    x25, x6, x4+        adcs    x7, x7, xzr+        and     x4, x3, #0xffffffff00000001+        adc     x8, x8, x4+        stp     x7, x8, [x0, #16]+        subs    x5, x9, x19+        sbcs    x6, x10, x20+        ldp     x4, x3, [sp, #144]+        sbcs    x7, x11, x4+        sbcs    x8, x12, x3+        csetm   x3, cc  // cc = lo, ul, last+        adds    x5, x5, x3+        and     x4, x3, #0xffffffff+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        and     x4, x3, #0xffffffff00000001+        adc     x8, x8, x4+        stp     x5, x6, [sp, #96]+        stp     x7, x8, [sp, #112]+        ldr     q20, [x22, #64]+        ldp     x7, x17, [sp, #160]+        ldr     q0, [sp, #160]+        ldp     x6, x10, [x22, #64]+        ldp     x11, x15, [sp, #176]+        rev64   v16.4s, v20.4s+        subs    x4, x7, x17+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x17, x10+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x11, x7+        ldr     q20, [x22, #80]+        sbcs    x5, x15, x17+        ngc     x17, xzr+        subs    x8, x11, x15+        uaddlp  v27.2d, v16.4s+        umulh   x4, x7, x6+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x10, x6+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [sp, #176]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        ldp     x7, x2, [x22, #80]+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x6, x7+        sbcs    x9, x10, x2+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x2, x7+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x19, x3, x13+        adcs    x20, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x26, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x27, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x19+        adcs    x15, x16, x20+        eor     x5, x17, x4+        adcs    x9, x1, x26+        eor     x1, x10, x5+        adcs    x16, x2, x27+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x11, x11, x13+        and     x1, x1, x13+        adcs    x4, x4, x1+        and     x1, x12, x13+        stp     x11, x4, [sp, #160]+        adcs    x19, x7, xzr+        adc     x20, x17, x1+        stp     x19, x20, [sp, #176]+        mov     x0, sp+        mov     x1, sp+        ldp     x4, x3, [sp, #64]+        subs    x5, x24, x4+        sbcs    x6, x25, x3+        ldp     x7, x8, [x1, #16]+        ldp     x4, x3, [sp, #80]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc  // cc = lo, ul, last+        adds    x9, x5, x3+        and     x4, x3, #0xffffffff+        adcs    x10, x6, x4+        adcs    x11, x7, xzr+        and     x4, x3, #0xffffffff00000001+        adc     x3, x8, x4+        stp     x9, x10, [x0]+        stp     x11, x3, [x0, #16]+        ldp     x5, x6, [sp, #128]+        subs    x5, x5, x9+        sbcs    x6, x6, x10+        ldp     x7, x8, [sp, #144]+        sbcs    x7, x7, x11+        sbcs    x8, x8, x3+        csetm   x3, cc  // cc = lo, ul, last+        adds    x5, x5, x3+        and     x4, x3, #0xffffffff+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        and     x4, x3, #0xffffffff00000001+        adc     x8, x8, x4+        stp     x5, x6, [sp, #128]+        stp     x7, x8, [sp, #144]+        ldr     q20, [sp, #192]+        ldp     x7, x17, [sp, #96]+        ldr     q0, [sp, #96]+        ldp     x6, x10, [sp, #192]+        ldp     x11, x15, [sp, #112]+        rev64   v16.4s, v20.4s+        subs    x4, x7, x17+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x17, x10+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x11, x7+        ldr     q20, [sp, #208]+        sbcs    x5, x15, x17+        ngc     x17, xzr+        subs    x8, x11, x15+        uaddlp  v27.2d, v16.4s+        umulh   x4, x7, x6+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x10, x6+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [sp, #112]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        ldp     x7, x2, [sp, #208]+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x6, x7+        sbcs    x9, x10, x2+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x2, x7+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x24, x3, x13+        adcs    x25, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x26, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x27, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x24+        adcs    x15, x16, x25+        eor     x5, x17, x4+        adcs    x9, x1, x26+        eor     x1, x10, x5+        adcs    x16, x2, x27+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x11, x11, x13+        and     x1, x1, x13+        adcs    x4, x4, x1+        and     x1, x12, x13+        stp     x11, x4, [sp, #96]+        adcs    x4, x7, xzr+        adc     x1, x17, x1+        stp     x4, x1, [sp, #112]+        ldr     q20, [x23, #64]+        ldp     x7, x17, [sp, #160]+        ldr     q0, [sp, #160]+        ldp     x6, x10, [x23, #64]+        rev64   v16.4s, v20.4s+        subs    x4, x7, x17+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x17, x10+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x19, x7+        ldr     q20, [x23, #80]+        sbcs    x5, x20, x17+        ngc     x17, xzr+        subs    x8, x19, x20+        uaddlp  v27.2d, v16.4s+        umulh   x4, x7, x6+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x10, x6+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [sp, #176]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        ldp     x7, x2, [x23, #80]+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x6, x7+        sbcs    x9, x10, x2+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x2, x7+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x19, x3, x13+        adcs    x20, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x24, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x25, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x19+        adcs    x15, x16, x20+        eor     x5, x17, x4+        adcs    x9, x1, x24+        eor     x1, x10, x5+        adcs    x16, x2, x25+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x19, x11, x13+        and     x1, x1, x13+        adcs    x20, x4, x1+        and     x1, x12, x13+        stp     x19, x20, [sp, #160]+        adcs    x4, x7, xzr+        adc     x1, x17, x1+        stp     x4, x1, [sp, #176]+        ldr     q20, [sp, #128]+        ldp     x7, x17, [sp, #32]+        ldr     q0, [sp, #32]+        ldp     x6, x10, [sp, #128]+        ldp     x11, x15, [sp, #48]+        rev64   v16.4s, v20.4s+        subs    x4, x7, x17+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x17, x10+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x11, x7+        ldr     q20, [sp, #144]+        sbcs    x5, x15, x17+        ngc     x17, xzr+        subs    x8, x11, x15+        uaddlp  v27.2d, v16.4s+        umulh   x4, x7, x6+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x10, x6+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [sp, #48]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        ldp     x7, x2, [sp, #144]+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x6, x7+        sbcs    x9, x10, x2+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x2, x7+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x24, x3, x13+        adcs    x25, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x26, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x27, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x24+        adcs    x15, x16, x25+        eor     x5, x17, x4+        adcs    x9, x1, x26+        eor     x1, x10, x5+        adcs    x16, x2, x27+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x5, x11, x13+        and     x1, x1, x13+        adcs    x6, x4, x1+        and     x1, x12, x13+        adcs    x7, x7, xzr+        adc     x9, x17, x1+        ldp     x4, x3, [sp, #96]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x4, x3, [sp, #112]+        sbcs    x7, x7, x4+        sbcs    x8, x9, x3+        csetm   x3, cc  // cc = lo, ul, last+        adds    x15, x5, x3+        and     x4, x3, #0xffffffff+        adcs    x24, x6, x4+        adcs    x25, x7, xzr+        and     x4, x3, #0xffffffff00000001+        adc     x26, x8, x4+        stp     x15, x24, [sp, #128]+        stp     x25, x26, [sp, #144]+        ldp     x0, x1, [x22, #64]+        ldp     x2, x3, [x22, #80]+        orr     x12, x0, x1+        orr     x13, x2, x3+        orr     x12, x12, x13+        cmp     x12, xzr+        cset    x12, ne  // ne = any+        ldp     x4, x5, [x23, #64]+        ldp     x6, x7, [x23, #80]+        orr     x13, x4, x5+        orr     x14, x6, x7+        orr     x13, x13, x14+        cmp     x13, xzr+        cset    x13, ne  // ne = any+        cmp     x13, x12+        csel    x8, x0, x19, cc  // cc = lo, ul, last+        csel    x9, x1, x20, cc  // cc = lo, ul, last+        csel    x8, x4, x8, hi  // hi = pmore+        csel    x9, x5, x9, hi  // hi = pmore+        ldp     x10, x11, [sp, #176]+        csel    x10, x2, x10, cc  // cc = lo, ul, last+        csel    x11, x3, x11, cc  // cc = lo, ul, last+        csel    x10, x6, x10, hi  // hi = pmore+        csel    x11, x7, x11, hi  // hi = pmore+        ldp     x12, x13, [x22]+        ldp     x0, x1, [sp]+        csel    x0, x12, x0, cc  // cc = lo, ul, last+        csel    x1, x13, x1, cc  // cc = lo, ul, last+        ldp     x12, x13, [x23]+        csel    x0, x12, x0, hi  // hi = pmore+        csel    x1, x13, x1, hi  // hi = pmore+        ldp     x12, x13, [x22, #16]+        ldp     x2, x3, [sp, #16]+        csel    x2, x12, x2, cc  // cc = lo, ul, last+        csel    x3, x13, x3, cc  // cc = lo, ul, last+        ldp     x12, x13, [x23, #16]+        csel    x2, x12, x2, hi  // hi = pmore+        csel    x3, x13, x3, hi  // hi = pmore+        ldp     x12, x13, [x22, #32]+        csel    x4, x12, x15, cc  // cc = lo, ul, last+        csel    x5, x13, x24, cc  // cc = lo, ul, last+        ldp     x12, x13, [x23, #32]+        csel    x4, x12, x4, hi  // hi = pmore+        csel    x5, x13, x5, hi  // hi = pmore+        ldp     x12, x13, [x22, #48]+        csel    x6, x12, x25, cc  // cc = lo, ul, last+        csel    x7, x13, x26, cc  // cc = lo, ul, last+        ldp     x12, x13, [x23, #48]+        csel    x6, x12, x6, hi  // hi = pmore+        csel    x7, x13, x7, hi  // hi = pmore+        stp     x0, x1, [x21]+        stp     x2, x3, [x21, #16]+        stp     x4, x5, [x21, #32]+        stp     x6, x7, [x21, #48]+        stp     x8, x9, [x21, #64]+        stp     x10, x11, [x21, #80]+        CFI_INC_SP(224)+        CFI_POP2(x27,x30)+        CFI_POP2(x25,x26)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_local_p256_montjadd)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_local_p256_montjdouble)++Lp256_scalarmul_local_p256_montjdouble:+        CFI_START+        CFI_DEC_SP(272)+        stp     x19, x20, [sp, #192]+        stp     x21, x22, [sp, #208]+        stp     x23, x24, [sp, #224]+        stp     x25, x26, [sp, #240]+        stp     x27, xzr, [sp, #256]+        mov     x19, x0+        mov     x20, x1+        mov     x0, sp+        ldr     q19, [x20, #64]+        ldp     x9, x13, [x20, #64]+        ldr     q23, [x20, #80]+        ldr     q0, [x20, #64]+        ldp     x1, x10, [x20, #80]+        uzp2    v29.4s, v19.4s, v19.4s+        xtn     v4.2s, v19.2d+        umulh   x8, x9, x13+        rev64   v20.4s, v23.4s+        umull   v16.2d, v19.2s, v19.2s+        umull   v1.2d, v29.2s, v4.2s+        mul     v20.4s, v20.4s, v0.4s+        subs    x14, x9, x13+        umulh   x15, x9, x1+        mov     x16, v16.d[1]+        umull2  v4.2d, v19.4s, v19.4s+        mov     x4, v16.d[0]+        uzp1    v17.4s, v23.4s, v0.4s+        uaddlp  v19.2d, v20.4s+        lsr     x7, x8, #63+        mul     x11, x9, x13+        mov     x12, v1.d[0]+        csetm   x5, cc  // cc = lo, ul, last+        cneg    x6, x14, cc  // cc = lo, ul, last+        mov     x3, v4.d[1]+        mov     x14, v4.d[0]+        subs    x2, x10, x1+        mov     x9, v1.d[1]+        cneg    x17, x2, cc  // cc = lo, ul, last+        cinv    x2, x5, cc  // cc = lo, ul, last+        adds    x5, x4, x12, lsl #33+        extr    x4, x8, x11, #63+        lsr     x8, x12, #31+        uzp1    v20.4s, v0.4s, v0.4s+        shl     v19.2d, v19.2d, #32+        adc     x16, x16, x8+        adds    x8, x14, x9, lsl #33+        lsr     x14, x9, #31+        lsl     x9, x5, #32+        umlal   v19.2d, v20.2s, v17.2s+        adc     x14, x3, x14+        adds    x16, x16, x11, lsl #1+        lsr     x3, x5, #32+        umulh   x12, x6, x17+        adcs    x4, x8, x4+        adc     x11, x14, x7+        subs    x8, x5, x9+        sbc     x5, x5, x3+        adds    x16, x16, x9+        mov     x14, v19.d[0]+        mul     x17, x6, x17+        adcs    x3, x4, x3+        lsl     x7, x16, #32+        umulh   x13, x13, x10+        adcs    x11, x11, x8+        lsr     x8, x16, #32+        adc     x5, x5, xzr+        subs    x9, x16, x7+        sbc     x16, x16, x8+        adds    x7, x3, x7+        mov     x3, v19.d[1]+        adcs    x6, x11, x8+        umulh   x11, x1, x10+        adcs    x5, x5, x9+        eor     x8, x12, x2+        adc     x9, x16, xzr+        adds    x16, x14, x15+        adc     x15, x15, xzr+        adds    x12, x16, x3+        eor     x16, x17, x2+        mul     x4, x1, x10+        adcs    x15, x15, x13+        adc     x17, x13, xzr+        adds    x15, x15, x3+        adc     x3, x17, xzr+        cmn     x2, #0x1+        mul     x17, x10, x10+        adcs    x12, x12, x16+        adcs    x16, x15, x8+        umulh   x10, x10, x10+        adc     x2, x3, x2+        adds    x14, x14, x14+        adcs    x12, x12, x12+        adcs    x16, x16, x16+        adcs    x2, x2, x2+        adc     x15, xzr, xzr+        adds    x14, x14, x7+        mul     x3, x1, x1+        adcs    x12, x12, x6+        lsr     x7, x14, #32+        adcs    x16, x16, x5+        lsl     x5, x14, #32+        umulh   x13, x1, x1+        adcs    x2, x2, x9+        mov     x6, #0xffffffff                 // #4294967295+        adc     x15, x15, xzr+        adds    x8, x4, x4+        adcs    x1, x11, x11+        mov     x11, #0xffffffff00000001        // #-4294967295+        adc     x4, xzr, xzr+        subs    x9, x14, x5+        sbc     x14, x14, x7+        adds    x12, x12, x5+        adcs    x16, x16, x7+        lsl     x5, x12, #32+        lsr     x7, x12, #32+        adcs    x2, x2, x9+        adcs    x14, x15, x14+        adc     x15, xzr, xzr+        subs    x9, x12, x5+        sbc     x12, x12, x7+        adds    x16, x16, x5+        adcs    x2, x2, x7+        adcs    x14, x14, x9+        adcs    x12, x15, x12+        adc     x15, xzr, xzr+        adds    x16, x16, x3+        adcs    x2, x2, x13+        adcs    x14, x14, x17+        adcs    x12, x12, x10+        adc     x15, x15, xzr+        adds    x2, x2, x8+        adcs    x14, x14, x1+        adcs    x12, x12, x4+        adcs    x15, x15, xzr+        adds    x3, x16, #0x1+        sbcs    x5, x2, x6+        sbcs    x8, x14, xzr+        sbcs    x11, x12, x11+        sbcs    xzr, x15, xzr+        csel    x21, x3, x16, cs  // cs = hs, nlast+        csel    x22, x8, x14, cs  // cs = hs, nlast+        csel    x23, x11, x12, cs  // cs = hs, nlast+        csel    x24, x5, x2, cs  // cs = hs, nlast+        stp     x22, x23, [x0, #16]+        stp     x21, x24, [x0]+        ldr     q19, [x20, #32]+        ldp     x9, x13, [x20, #32]+        ldr     q23, [x20, #48]+        ldr     q0, [x20, #32]+        ldp     x1, x10, [x20, #48]+        uzp2    v29.4s, v19.4s, v19.4s+        xtn     v4.2s, v19.2d+        umulh   x8, x9, x13+        rev64   v20.4s, v23.4s+        umull   v16.2d, v19.2s, v19.2s+        umull   v1.2d, v29.2s, v4.2s+        mul     v20.4s, v20.4s, v0.4s+        subs    x14, x9, x13+        umulh   x15, x9, x1+        mov     x16, v16.d[1]+        umull2  v4.2d, v19.4s, v19.4s+        mov     x4, v16.d[0]+        uzp1    v17.4s, v23.4s, v0.4s+        uaddlp  v19.2d, v20.4s+        lsr     x7, x8, #63+        mul     x11, x9, x13+        mov     x12, v1.d[0]+        csetm   x5, cc  // cc = lo, ul, last+        cneg    x6, x14, cc  // cc = lo, ul, last+        mov     x3, v4.d[1]+        mov     x14, v4.d[0]+        subs    x2, x10, x1+        mov     x9, v1.d[1]+        cneg    x17, x2, cc  // cc = lo, ul, last+        cinv    x2, x5, cc  // cc = lo, ul, last+        adds    x5, x4, x12, lsl #33+        extr    x4, x8, x11, #63+        lsr     x8, x12, #31+        uzp1    v20.4s, v0.4s, v0.4s+        shl     v19.2d, v19.2d, #32+        adc     x16, x16, x8+        adds    x8, x14, x9, lsl #33+        lsr     x14, x9, #31+        lsl     x9, x5, #32+        umlal   v19.2d, v20.2s, v17.2s+        adc     x14, x3, x14+        adds    x16, x16, x11, lsl #1+        lsr     x3, x5, #32+        umulh   x12, x6, x17+        adcs    x4, x8, x4+        adc     x11, x14, x7+        subs    x8, x5, x9+        sbc     x5, x5, x3+        adds    x16, x16, x9+        mov     x14, v19.d[0]+        mul     x17, x6, x17+        adcs    x3, x4, x3+        lsl     x7, x16, #32+        umulh   x13, x13, x10+        adcs    x11, x11, x8+        lsr     x8, x16, #32+        adc     x5, x5, xzr+        subs    x9, x16, x7+        sbc     x16, x16, x8+        adds    x7, x3, x7+        mov     x3, v19.d[1]+        adcs    x6, x11, x8+        umulh   x11, x1, x10+        adcs    x5, x5, x9+        eor     x8, x12, x2+        adc     x9, x16, xzr+        adds    x16, x14, x15+        adc     x15, x15, xzr+        adds    x12, x16, x3+        eor     x16, x17, x2+        mul     x4, x1, x10+        adcs    x15, x15, x13+        adc     x17, x13, xzr+        adds    x15, x15, x3+        adc     x3, x17, xzr+        cmn     x2, #0x1+        mul     x17, x10, x10+        adcs    x12, x12, x16+        adcs    x16, x15, x8+        umulh   x10, x10, x10+        adc     x2, x3, x2+        adds    x14, x14, x14+        adcs    x12, x12, x12+        adcs    x16, x16, x16+        adcs    x2, x2, x2+        adc     x15, xzr, xzr+        adds    x14, x14, x7+        mul     x3, x1, x1+        adcs    x12, x12, x6+        lsr     x7, x14, #32+        adcs    x16, x16, x5+        lsl     x5, x14, #32+        umulh   x13, x1, x1+        adcs    x2, x2, x9+        mov     x6, #0xffffffff                 // #4294967295+        adc     x15, x15, xzr+        adds    x8, x4, x4+        adcs    x1, x11, x11+        mov     x11, #0xffffffff00000001        // #-4294967295+        adc     x4, xzr, xzr+        subs    x9, x14, x5+        sbc     x14, x14, x7+        adds    x12, x12, x5+        adcs    x16, x16, x7+        lsl     x5, x12, #32+        lsr     x7, x12, #32+        adcs    x2, x2, x9+        adcs    x14, x15, x14+        adc     x15, xzr, xzr+        subs    x9, x12, x5+        sbc     x12, x12, x7+        adds    x16, x16, x5+        adcs    x2, x2, x7+        adcs    x14, x14, x9+        adcs    x12, x15, x12+        adc     x15, xzr, xzr+        adds    x16, x16, x3+        adcs    x2, x2, x13+        adcs    x14, x14, x17+        adcs    x12, x12, x10+        adc     x15, x15, xzr+        adds    x2, x2, x8+        adcs    x14, x14, x1+        adcs    x12, x12, x4+        adcs    x15, x15, xzr+        adds    x3, x16, #0x1+        sbcs    x5, x2, x6+        sbcs    x8, x14, xzr+        sbcs    x11, x12, x11+        sbcs    xzr, x15, xzr+        csel    x16, x3, x16, cs  // cs = hs, nlast+        csel    x14, x8, x14, cs  // cs = hs, nlast+        csel    x12, x11, x12, cs  // cs = hs, nlast+        csel    x2, x5, x2, cs  // cs = hs, nlast+        stp     x14, x12, [sp, #48]+        stp     x16, x2, [sp, #32]+        ldp     x5, x6, [x20]+        subs    x5, x5, x21+        sbcs    x6, x6, x24+        ldp     x7, x8, [x20, #16]+        sbcs    x7, x7, x22+        sbcs    x8, x8, x23+        csetm   x3, cc  // cc = lo, ul, last+        adds    x10, x5, x3+        and     x4, x3, #0xffffffff+        adcs    x25, x6, x4+        adcs    x26, x7, xzr+        and     x4, x3, #0xffffffff00000001+        adc     x27, x8, x4+        stp     x10, x25, [sp, #96]+        stp     x26, x27, [sp, #112]+        ldp     x5, x6, [x20]+        adds    x5, x5, x21+        adcs    x6, x6, x24+        ldp     x7, x8, [x20, #16]+        adcs    x7, x7, x22+        adcs    x8, x8, x23+        csetm   x3, cs  // cs = hs, nlast+        subs    x9, x5, x3+        and     x1, x3, #0xffffffff+        sbcs    x5, x6, x1+        sbcs    x7, x7, xzr+        and     x2, x3, #0xffffffff00000001+        sbc     x8, x8, x2+        stp     x9, x5, [sp, #64]+        stp     x7, x8, [sp, #80]+        ldr     q20, [sp, #96]+        ldr     q0, [sp, #64]+        rev64   v16.4s, v20.4s+        subs    x4, x9, x5+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x5, x25+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x7, x9+        ldr     q20, [sp, #112]+        sbcs    x5, x8, x5+        ngc     x17, xzr+        subs    x8, x7, x8+        uaddlp  v27.2d, v16.4s+        umulh   x4, x9, x10+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x25, x10+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [sp, #80]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x10, x26+        sbcs    x9, x25, x27+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x27, x26+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x21, x3, x13+        adcs    x22, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x23, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x24, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x21+        adcs    x15, x16, x22+        eor     x5, x17, x4+        adcs    x9, x1, x23+        eor     x1, x10, x5+        adcs    x16, x2, x24+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x21, x11, x13+        and     x1, x1, x13+        adcs    x22, x4, x1+        and     x1, x12, x13+        stp     x21, x22, [sp, #96]+        adcs    x23, x7, xzr+        adc     x24, x17, x1+        stp     x23, x24, [sp, #112]+        ldp     x4, x5, [x20, #32]+        ldp     x8, x9, [x20, #64]+        adds    x4, x4, x8+        adcs    x5, x5, x9+        ldp     x6, x7, [x20, #48]+        ldp     x10, x11, [x20, #80]+        adcs    x6, x6, x10+        adcs    x7, x7, x11+        adc     x3, xzr, xzr+        adds    x8, x4, #0x1+        mov     x9, #0xffffffff                 // #4294967295+        sbcs    x9, x5, x9+        sbcs    x10, x6, xzr+        mov     x11, #0xffffffff00000001        // #-4294967295+        sbcs    x11, x7, x11+        sbcs    x3, x3, xzr+        csel    x4, x4, x8, cc  // cc = lo, ul, last+        csel    x5, x5, x9, cc  // cc = lo, ul, last+        csel    x6, x6, x10, cc  // cc = lo, ul, last+        csel    x7, x7, x11, cc  // cc = lo, ul, last+        stp     x4, x5, [sp, #64]+        stp     x6, x7, [sp, #80]+        ldr     q20, [sp, #32]+        ldp     x7, x17, [x20]+        ldr     q0, [x20]+        ldp     x6, x10, [sp, #32]+        ldp     x11, x15, [x20, #16]+        rev64   v16.4s, v20.4s+        subs    x4, x7, x17+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x17, x10+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x11, x7+        ldr     q20, [sp, #48]+        sbcs    x5, x15, x17+        ngc     x17, xzr+        subs    x8, x11, x15+        uaddlp  v27.2d, v16.4s+        umulh   x4, x7, x6+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x10, x6+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [x20, #16]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        ldp     x7, x2, [sp, #48]+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x6, x7+        sbcs    x9, x10, x2+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x2, x7+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x20, x3, x13+        adcs    x25, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x26, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x27, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x20+        adcs    x15, x16, x25+        eor     x5, x17, x4+        adcs    x9, x1, x26+        eor     x1, x10, x5+        adcs    x16, x2, x27+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x20, x11, x13+        and     x1, x1, x13+        adcs    x25, x4, x1+        and     x1, x12, x13+        stp     x20, x25, [sp, #128]+        adcs    x4, x7, xzr+        adc     x1, x17, x1+        stp     x4, x1, [sp, #144]+        ldr     q19, [sp, #96]+        ldr     q23, [sp, #112]+        ldr     q0, [sp, #96]+        uzp2    v29.4s, v19.4s, v19.4s+        xtn     v4.2s, v19.2d+        umulh   x8, x21, x22+        rev64   v20.4s, v23.4s+        umull   v16.2d, v19.2s, v19.2s+        umull   v1.2d, v29.2s, v4.2s+        mul     v20.4s, v20.4s, v0.4s+        subs    x14, x21, x22+        umulh   x15, x21, x23+        mov     x16, v16.d[1]+        umull2  v4.2d, v19.4s, v19.4s+        mov     x4, v16.d[0]+        uzp1    v17.4s, v23.4s, v0.4s+        uaddlp  v19.2d, v20.4s+        lsr     x7, x8, #63+        mul     x11, x21, x22+        mov     x12, v1.d[0]+        csetm   x5, cc  // cc = lo, ul, last+        cneg    x6, x14, cc  // cc = lo, ul, last+        mov     x3, v4.d[1]+        mov     x14, v4.d[0]+        subs    x2, x24, x23+        mov     x9, v1.d[1]+        cneg    x17, x2, cc  // cc = lo, ul, last+        cinv    x2, x5, cc  // cc = lo, ul, last+        adds    x5, x4, x12, lsl #33+        extr    x4, x8, x11, #63+        lsr     x8, x12, #31+        uzp1    v20.4s, v0.4s, v0.4s+        shl     v19.2d, v19.2d, #32+        adc     x16, x16, x8+        adds    x8, x14, x9, lsl #33+        lsr     x14, x9, #31+        lsl     x9, x5, #32+        umlal   v19.2d, v20.2s, v17.2s+        adc     x14, x3, x14+        adds    x16, x16, x11, lsl #1+        lsr     x3, x5, #32+        umulh   x12, x6, x17+        adcs    x4, x8, x4+        adc     x11, x14, x7+        subs    x8, x5, x9+        sbc     x5, x5, x3+        adds    x16, x16, x9+        mov     x14, v19.d[0]+        mul     x17, x6, x17+        adcs    x3, x4, x3+        lsl     x7, x16, #32+        umulh   x13, x22, x24+        adcs    x11, x11, x8+        lsr     x8, x16, #32+        adc     x5, x5, xzr+        subs    x9, x16, x7+        sbc     x16, x16, x8+        adds    x7, x3, x7+        mov     x3, v19.d[1]+        adcs    x6, x11, x8+        umulh   x11, x23, x24+        adcs    x5, x5, x9+        eor     x8, x12, x2+        adc     x9, x16, xzr+        adds    x16, x14, x15+        adc     x15, x15, xzr+        adds    x12, x16, x3+        eor     x16, x17, x2+        mul     x4, x23, x24+        adcs    x15, x15, x13+        adc     x17, x13, xzr+        adds    x15, x15, x3+        adc     x3, x17, xzr+        cmn     x2, #0x1+        mul     x17, x24, x24+        adcs    x12, x12, x16+        adcs    x16, x15, x8+        umulh   x10, x24, x24+        adc     x2, x3, x2+        adds    x14, x14, x14+        adcs    x12, x12, x12+        adcs    x16, x16, x16+        adcs    x2, x2, x2+        adc     x15, xzr, xzr+        adds    x14, x14, x7+        mul     x3, x23, x23+        adcs    x12, x12, x6+        lsr     x7, x14, #32+        adcs    x16, x16, x5+        lsl     x5, x14, #32+        umulh   x13, x23, x23+        adcs    x2, x2, x9+        mov     x6, #0xffffffff                 // #4294967295+        adc     x15, x15, xzr+        adds    x8, x4, x4+        adcs    x1, x11, x11+        mov     x11, #0xffffffff00000001        // #-4294967295+        adc     x4, xzr, xzr+        subs    x9, x14, x5+        sbc     x14, x14, x7+        adds    x12, x12, x5+        adcs    x16, x16, x7+        lsl     x5, x12, #32+        lsr     x7, x12, #32+        adcs    x2, x2, x9+        adcs    x14, x15, x14+        adc     x15, xzr, xzr+        subs    x9, x12, x5+        sbc     x12, x12, x7+        adds    x16, x16, x5+        adcs    x2, x2, x7+        adcs    x14, x14, x9+        adcs    x12, x15, x12+        adc     x15, xzr, xzr+        adds    x16, x16, x3+        adcs    x2, x2, x13+        adcs    x14, x14, x17+        adcs    x12, x12, x10+        adc     x15, x15, xzr+        adds    x2, x2, x8+        adcs    x14, x14, x1+        adcs    x12, x12, x4+        adcs    x15, x15, xzr+        adds    x3, x16, #0x1+        sbcs    x5, x2, x6+        sbcs    x8, x14, xzr+        sbcs    x11, x12, x11+        sbcs    xzr, x15, xzr+        csel    x21, x3, x16, cs  // cs = hs, nlast+        csel    x22, x8, x14, cs  // cs = hs, nlast+        csel    x23, x11, x12, cs  // cs = hs, nlast+        csel    x24, x5, x2, cs  // cs = hs, nlast+        ldr     q19, [sp, #64]+        ldp     x9, x13, [sp, #64]+        ldr     q23, [sp, #80]+        ldr     q0, [sp, #64]+        ldp     x1, x10, [sp, #80]+        uzp2    v29.4s, v19.4s, v19.4s+        xtn     v4.2s, v19.2d+        umulh   x8, x9, x13+        rev64   v20.4s, v23.4s+        umull   v16.2d, v19.2s, v19.2s+        umull   v1.2d, v29.2s, v4.2s+        mul     v20.4s, v20.4s, v0.4s+        subs    x14, x9, x13+        umulh   x15, x9, x1+        mov     x16, v16.d[1]+        umull2  v4.2d, v19.4s, v19.4s+        mov     x4, v16.d[0]+        uzp1    v17.4s, v23.4s, v0.4s+        uaddlp  v19.2d, v20.4s+        lsr     x7, x8, #63+        mul     x11, x9, x13+        mov     x12, v1.d[0]+        csetm   x5, cc  // cc = lo, ul, last+        cneg    x6, x14, cc  // cc = lo, ul, last+        mov     x3, v4.d[1]+        mov     x14, v4.d[0]+        subs    x2, x10, x1+        mov     x9, v1.d[1]+        cneg    x17, x2, cc  // cc = lo, ul, last+        cinv    x2, x5, cc  // cc = lo, ul, last+        adds    x5, x4, x12, lsl #33+        extr    x4, x8, x11, #63+        lsr     x8, x12, #31+        uzp1    v20.4s, v0.4s, v0.4s+        shl     v19.2d, v19.2d, #32+        adc     x16, x16, x8+        adds    x8, x14, x9, lsl #33+        lsr     x14, x9, #31+        lsl     x9, x5, #32+        umlal   v19.2d, v20.2s, v17.2s+        adc     x14, x3, x14+        adds    x16, x16, x11, lsl #1+        lsr     x3, x5, #32+        umulh   x12, x6, x17+        adcs    x4, x8, x4+        adc     x11, x14, x7+        subs    x8, x5, x9+        sbc     x5, x5, x3+        adds    x16, x16, x9+        mov     x14, v19.d[0]+        mul     x17, x6, x17+        adcs    x3, x4, x3+        lsl     x7, x16, #32+        umulh   x13, x13, x10+        adcs    x11, x11, x8+        lsr     x8, x16, #32+        adc     x5, x5, xzr+        subs    x9, x16, x7+        sbc     x16, x16, x8+        adds    x7, x3, x7+        mov     x3, v19.d[1]+        adcs    x6, x11, x8+        umulh   x11, x1, x10+        adcs    x5, x5, x9+        eor     x8, x12, x2+        adc     x9, x16, xzr+        adds    x16, x14, x15+        adc     x15, x15, xzr+        adds    x12, x16, x3+        eor     x16, x17, x2+        mul     x4, x1, x10+        adcs    x15, x15, x13+        adc     x17, x13, xzr+        adds    x15, x15, x3+        adc     x3, x17, xzr+        cmn     x2, #0x1+        mul     x17, x10, x10+        adcs    x12, x12, x16+        adcs    x16, x15, x8+        umulh   x10, x10, x10+        adc     x2, x3, x2+        adds    x14, x14, x14+        adcs    x12, x12, x12+        adcs    x16, x16, x16+        adcs    x2, x2, x2+        adc     x15, xzr, xzr+        adds    x14, x14, x7+        mul     x3, x1, x1+        adcs    x12, x12, x6+        lsr     x7, x14, #32+        adcs    x16, x16, x5+        lsl     x5, x14, #32+        umulh   x13, x1, x1+        adcs    x2, x2, x9+        mov     x6, #0xffffffff                 // #4294967295+        adc     x15, x15, xzr+        adds    x8, x4, x4+        adcs    x1, x11, x11+        mov     x11, #0xffffffff00000001        // #-4294967295+        adc     x4, xzr, xzr+        subs    x9, x14, x5+        sbc     x14, x14, x7+        adds    x12, x12, x5+        adcs    x16, x16, x7+        lsl     x5, x12, #32+        lsr     x7, x12, #32+        adcs    x2, x2, x9+        adcs    x14, x15, x14+        adc     x15, xzr, xzr+        subs    x9, x12, x5+        sbc     x12, x12, x7+        adds    x16, x16, x5+        adcs    x2, x2, x7+        adcs    x14, x14, x9+        adcs    x12, x15, x12+        adc     x15, xzr, xzr+        adds    x16, x16, x3+        adcs    x2, x2, x13+        adcs    x14, x14, x17+        adcs    x12, x12, x10+        adc     x15, x15, xzr+        adds    x2, x2, x8+        adcs    x14, x14, x1+        adcs    x12, x12, x4+        adcs    x15, x15, xzr+        adds    x3, x16, #0x1+        sbcs    x5, x2, x6+        sbcs    x8, x14, xzr+        sbcs    x11, x12, x11+        sbcs    xzr, x15, xzr+        csel    x13, x3, x16, cs  // cs = hs, nlast+        csel    x14, x8, x14, cs  // cs = hs, nlast+        csel    x15, x11, x12, cs  // cs = hs, nlast+        csel    x26, x5, x2, cs  // cs = hs, nlast+        mov     x1, #0x9                        // #9+        mov     x2, #0xffffffffffffffff         // #-1+        subs    x9, x2, x21+        mov     x2, #0xffffffff                 // #4294967295+        sbcs    x10, x2, x24+        ngcs    x11, x22+        mov     x2, #0xffffffff00000001         // #-4294967295+        sbc     x12, x2, x23+        mul     x3, x1, x9+        mul     x4, x1, x10+        mul     x5, x1, x11+        mul     x6, x1, x12+        umulh   x9, x1, x9+        umulh   x10, x1, x10+        umulh   x11, x1, x11+        umulh   x7, x1, x12+        adds    x4, x4, x9+        adcs    x5, x5, x10+        adcs    x6, x6, x11+        adc     x7, x7, xzr+        mov     x1, #0xc                        // #12+        mul     x8, x20, x1+        umulh   x9, x20, x1+        adds    x3, x3, x8+        mul     x8, x25, x1+        umulh   x10, x25, x1+        adcs    x4, x4, x8+        ldp     x11, x12, [sp, #144]+        mul     x8, x11, x1+        umulh   x11, x11, x1+        adcs    x5, x5, x8+        mul     x8, x12, x1+        umulh   x12, x12, x1+        adcs    x6, x6, x8+        adc     x7, x7, xzr+        adds    x4, x4, x9+        adcs    x5, x5, x10+        adcs    x6, x6, x11+        adc     x7, x7, x12+        add     x8, x7, #0x1+        lsl     x10, x8, #32+        adds    x6, x6, x10+        adc     x7, x7, xzr+        neg     x9, x8+        sub     x10, x10, #0x1+        subs    x3, x3, x9+        sbcs    x4, x4, x10+        sbcs    x5, x5, xzr+        sbcs    x6, x6, x8+        sbc     x8, x7, x8+        adds    x20, x3, x8+        and     x9, x8, #0xffffffff+        adcs    x21, x4, x9+        adcs    x22, x5, xzr+        neg     x10, x9+        adc     x23, x6, x10+        stp     x20, x21, [sp, #160]+        stp     x22, x23, [sp, #176]+        mov     x2, sp+        ldp     x4, x3, [x2]+        subs    x5, x13, x4+        sbcs    x6, x26, x3+        ldp     x4, x3, [x2, #16]+        sbcs    x7, x14, x4+        sbcs    x8, x15, x3+        csetm   x3, cc  // cc = lo, ul, last+        adds    x5, x5, x3+        and     x4, x3, #0xffffffff+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        and     x4, x3, #0xffffffff00000001+        adc     x8, x8, x4+        stp     x5, x6, [sp, #64]+        stp     x7, x8, [sp, #80]+        mov     x0, sp+        ldr     q19, [sp, #32]+        ldp     x9, x13, [sp, #32]+        ldr     q23, [sp, #48]+        ldr     q0, [sp, #32]+        ldp     x1, x10, [sp, #48]+        uzp2    v29.4s, v19.4s, v19.4s+        xtn     v4.2s, v19.2d+        umulh   x8, x9, x13+        rev64   v20.4s, v23.4s+        umull   v16.2d, v19.2s, v19.2s+        umull   v1.2d, v29.2s, v4.2s+        mul     v20.4s, v20.4s, v0.4s+        subs    x14, x9, x13+        umulh   x15, x9, x1+        mov     x16, v16.d[1]+        umull2  v4.2d, v19.4s, v19.4s+        mov     x4, v16.d[0]+        uzp1    v17.4s, v23.4s, v0.4s+        uaddlp  v19.2d, v20.4s+        lsr     x7, x8, #63+        mul     x11, x9, x13+        mov     x12, v1.d[0]+        csetm   x5, cc  // cc = lo, ul, last+        cneg    x6, x14, cc  // cc = lo, ul, last+        mov     x3, v4.d[1]+        mov     x14, v4.d[0]+        subs    x2, x10, x1+        mov     x9, v1.d[1]+        cneg    x17, x2, cc  // cc = lo, ul, last+        cinv    x2, x5, cc  // cc = lo, ul, last+        adds    x5, x4, x12, lsl #33+        extr    x4, x8, x11, #63+        lsr     x8, x12, #31+        uzp1    v20.4s, v0.4s, v0.4s+        shl     v19.2d, v19.2d, #32+        adc     x16, x16, x8+        adds    x8, x14, x9, lsl #33+        lsr     x14, x9, #31+        lsl     x9, x5, #32+        umlal   v19.2d, v20.2s, v17.2s+        adc     x14, x3, x14+        adds    x16, x16, x11, lsl #1+        lsr     x3, x5, #32+        umulh   x12, x6, x17+        adcs    x4, x8, x4+        adc     x11, x14, x7+        subs    x8, x5, x9+        sbc     x5, x5, x3+        adds    x16, x16, x9+        mov     x14, v19.d[0]+        mul     x17, x6, x17+        adcs    x3, x4, x3+        lsl     x7, x16, #32+        umulh   x13, x13, x10+        adcs    x11, x11, x8+        lsr     x8, x16, #32+        adc     x5, x5, xzr+        subs    x9, x16, x7+        sbc     x16, x16, x8+        adds    x7, x3, x7+        mov     x3, v19.d[1]+        adcs    x6, x11, x8+        umulh   x11, x1, x10+        adcs    x5, x5, x9+        eor     x8, x12, x2+        adc     x9, x16, xzr+        adds    x16, x14, x15+        adc     x15, x15, xzr+        adds    x12, x16, x3+        eor     x16, x17, x2+        mul     x4, x1, x10+        adcs    x15, x15, x13+        adc     x17, x13, xzr+        adds    x15, x15, x3+        adc     x3, x17, xzr+        cmn     x2, #0x1+        mul     x17, x10, x10+        adcs    x12, x12, x16+        adcs    x16, x15, x8+        umulh   x10, x10, x10+        adc     x2, x3, x2+        adds    x14, x14, x14+        adcs    x12, x12, x12+        adcs    x16, x16, x16+        adcs    x2, x2, x2+        adc     x15, xzr, xzr+        adds    x14, x14, x7+        mul     x3, x1, x1+        adcs    x12, x12, x6+        lsr     x7, x14, #32+        adcs    x16, x16, x5+        lsl     x5, x14, #32+        umulh   x13, x1, x1+        adcs    x2, x2, x9+        mov     x6, #0xffffffff                 // #4294967295+        adc     x15, x15, xzr+        adds    x8, x4, x4+        adcs    x1, x11, x11+        mov     x11, #0xffffffff00000001        // #-4294967295+        adc     x4, xzr, xzr+        subs    x9, x14, x5+        sbc     x14, x14, x7+        adds    x12, x12, x5+        adcs    x16, x16, x7+        lsl     x5, x12, #32+        lsr     x7, x12, #32+        adcs    x2, x2, x9+        adcs    x14, x15, x14+        adc     x15, xzr, xzr+        subs    x9, x12, x5+        sbc     x12, x12, x7+        adds    x16, x16, x5+        adcs    x2, x2, x7+        adcs    x14, x14, x9+        adcs    x12, x15, x12+        adc     x15, xzr, xzr+        adds    x16, x16, x3+        adcs    x2, x2, x13+        adcs    x14, x14, x17+        adcs    x12, x12, x10+        adc     x15, x15, xzr+        adds    x2, x2, x8+        adcs    x14, x14, x1+        adcs    x12, x12, x4+        adcs    x15, x15, xzr+        adds    x3, x16, #0x1+        sbcs    x5, x2, x6+        sbcs    x8, x14, xzr+        sbcs    x11, x12, x11+        sbcs    xzr, x15, xzr+        csel    x24, x3, x16, cs  // cs = hs, nlast+        csel    x25, x8, x14, cs  // cs = hs, nlast+        csel    x26, x11, x12, cs  // cs = hs, nlast+        csel    x27, x5, x2, cs  // cs = hs, nlast+        stp     x25, x26, [x0, #16]+        stp     x24, x27, [x0]+        ldr     q20, [sp, #96]+        ldr     q0, [sp, #160]+        ldp     x6, x10, [sp, #96]+        rev64   v16.4s, v20.4s+        subs    x4, x20, x21+        csetm   x3, cc  // cc = lo, ul, last+        cneg    x13, x4, cc  // cc = lo, ul, last+        mul     v16.4s, v16.4s, v0.4s+        umulh   x12, x21, x10+        uzp1    v28.4s, v20.4s, v0.4s+        subs    x14, x22, x20+        ldr     q20, [sp, #112]+        sbcs    x5, x23, x21+        ngc     x17, xzr+        subs    x8, x22, x23+        uaddlp  v27.2d, v16.4s+        umulh   x4, x20, x6+        uzp1    v21.4s, v0.4s, v0.4s+        cneg    x11, x8, cc  // cc = lo, ul, last+        shl     v17.2d, v27.2d, #32+        csetm   x15, cc  // cc = lo, ul, last+        subs    x9, x10, x6+        eor     x7, x14, x17+        umlal   v17.2d, v21.2s, v28.2s+        cneg    x8, x9, cc  // cc = lo, ul, last+        cinv    x9, x3, cc  // cc = lo, ul, last+        cmn     x17, #0x1+        ldr     q28, [sp, #176]+        adcs    x14, x7, xzr+        mul     x7, x13, x8+        eor     x1, x5, x17+        adcs    x5, x1, xzr+        xtn     v1.2s, v20.2d+        mov     x1, v17.d[0]+        mov     x3, v17.d[1]+        uzp2    v16.4s, v20.4s, v20.4s+        umulh   x16, x13, x8+        eor     x13, x7, x9+        adds    x8, x1, x3+        adcs    x7, x4, x12+        xtn     v0.2s, v28.2d+        adcs    x12, x12, xzr+        adds    x8, x4, x8+        adcs    x3, x3, x7+        ldp     x7, x2, [sp, #112]+        adcs    x12, x12, xzr+        cmn     x9, #0x1+        adcs    x8, x8, x13+        eor     x13, x16, x9+        adcs    x16, x3, x13+        lsl     x3, x1, #32+        adc     x13, x12, x9+        subs    x12, x6, x7+        sbcs    x9, x10, x2+        lsr     x10, x1, #32+        ngc     x4, xzr+        subs    x6, x2, x7+        cinv    x2, x15, cc  // cc = lo, ul, last+        cneg    x6, x6, cc  // cc = lo, ul, last+        subs    x7, x1, x3+        eor     x9, x9, x4+        sbc     x1, x1, x10+        adds    x15, x8, x3+        adcs    x3, x16, x10+        mul     x16, x11, x6+        adcs    x8, x13, x7+        eor     x13, x12, x4+        adc     x10, x1, xzr+        cmn     x4, #0x1+        umulh   x6, x11, x6+        adcs    x11, x13, xzr+        adcs    x1, x9, xzr+        lsl     x13, x15, #32+        subs    x12, x15, x13+        lsr     x7, x15, #32+        sbc     x15, x15, x7+        adds    x20, x3, x13+        adcs    x21, x8, x7+        umulh   x8, x14, x11+        umull   v21.2d, v0.2s, v1.2s+        adcs    x22, x10, x12+        umull   v3.2d, v0.2s, v16.2s+        adc     x23, x15, xzr+        rev64   v24.4s, v20.4s+        movi    v2.2d, #0xffffffff+        mul     x10, x14, x11+        mul     v4.4s, v24.4s, v28.4s+        subs    x13, x14, x5+        uzp2    v19.4s, v28.4s, v28.4s+        csetm   x15, cc  // cc = lo, ul, last+        usra    v3.2d, v21.2d, #32+        mul     x7, x5, x1+        umull   v21.2d, v19.2s, v16.2s+        cneg    x13, x13, cc  // cc = lo, ul, last+        uaddlp  v5.2d, v4.4s+        subs    x11, x1, x11+        and     v16.16b, v3.16b, v2.16b+        umulh   x5, x5, x1+        shl     v24.2d, v5.2d, #32+        cneg    x11, x11, cc  // cc = lo, ul, last+        umlal   v16.2d, v19.2s, v1.2s+        cinv    x12, x15, cc  // cc = lo, ul, last+        umlal   v24.2d, v0.2s, v1.2s+        adds    x15, x10, x7+        mul     x14, x13, x11+        eor     x1, x6, x2+        adcs    x6, x8, x5+        usra    v21.2d, v3.2d, #32+        adcs    x9, x5, xzr+        umulh   x11, x13, x11+        adds    x15, x8, x15+        adcs    x7, x7, x6+        eor     x8, x14, x12+        usra    v21.2d, v16.2d, #32+        adcs    x13, x9, xzr+        cmn     x12, #0x1+        mov     x9, v24.d[1]+        adcs    x14, x15, x8+        eor     x6, x11, x12+        adcs    x6, x7, x6+        mov     x5, v24.d[0]+        mov     x11, v21.d[1]+        mov     x7, v21.d[0]+        adc     x3, x13, x12+        adds    x12, x5, x9+        adcs    x13, x7, x11+        adcs    x11, x11, xzr+        adds    x12, x7, x12+        eor     x16, x16, x2+        adcs    x7, x9, x13+        adcs    x11, x11, xzr+        cmn     x2, #0x1+        adcs    x16, x12, x16+        adcs    x1, x7, x1+        adc     x2, x11, x2+        adds    x7, x5, x20+        adcs    x15, x16, x21+        eor     x5, x17, x4+        adcs    x9, x1, x22+        eor     x1, x10, x5+        adcs    x16, x2, x23+        adc     x2, xzr, xzr+        cmn     x5, #0x1+        eor     x13, x14, x5+        adcs    x14, x1, x7+        eor     x1, x6, x5+        adcs    x6, x13, x15+        adcs    x10, x1, x9+        eor     x4, x3, x5+        mov     x1, #0xffffffff                 // #4294967295+        adcs    x8, x4, x16+        lsr     x13, x14, #32+        adcs    x17, x2, x5+        adcs    x11, x5, xzr+        adc     x4, x5, xzr+        adds    x12, x10, x7+        adcs    x7, x8, x15+        adcs    x5, x17, x9+        adcs    x9, x11, x16+        lsl     x11, x14, #32+        adc     x10, x4, x2+        subs    x17, x14, x11+        sbc     x4, x14, x13+        adds    x11, x6, x11+        adcs    x12, x12, x13+        lsl     x15, x11, #32+        adcs    x17, x7, x17+        lsr     x7, x11, #32+        adc     x13, x4, xzr+        subs    x4, x11, x15+        sbc     x11, x11, x7+        adds    x8, x12, x15+        adcs    x15, x17, x7+        adcs    x4, x13, x4+        adc     x11, x11, xzr+        adds    x7, x5, x4+        adcs    x17, x9, x11+        adc     x13, x10, xzr+        add     x12, x13, #0x1+        neg     x11, x12+        lsl     x4, x12, #32+        adds    x17, x17, x4+        sub     x4, x4, #0x1+        adc     x13, x13, xzr+        subs    x11, x8, x11+        sbcs    x4, x15, x4+        sbcs    x7, x7, xzr+        sbcs    x17, x17, x12+        sbcs    x13, x13, x12+        mov     x12, #0xffffffff00000001        // #-4294967295+        adds    x14, x11, x13+        and     x1, x1, x13+        adcs    x15, x4, x1+        and     x1, x12, x13+        stp     x14, x15, [sp, #96]+        adcs    x13, x7, xzr+        adc     x20, x17, x1+        stp     x13, x20, [sp, #112]+        ldp     x5, x6, [sp, #64]+        ldp     x4, x3, [sp, #32]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #80]+        ldp     x4, x3, [sp, #48]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc  // cc = lo, ul, last+        adds    x5, x5, x3+        and     x4, x3, #0xffffffff+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        and     x4, x3, #0xffffffff00000001+        adc     x8, x8, x4+        stp     x5, x6, [x19, #64]+        stp     x7, x8, [x19, #80]+        ldp     x1, x2, [sp, #128]+        lsl     x0, x1, #2+        ldp     x6, x7, [sp, #160]+        subs    x0, x0, x6+        extr    x1, x2, x1, #62+        sbcs    x1, x1, x7+        ldp     x3, x4, [sp, #144]+        extr    x2, x3, x2, #62+        ldp     x6, x7, [sp, #176]+        sbcs    x2, x2, x6+        extr    x3, x4, x3, #62+        sbcs    x3, x3, x7+        lsr     x4, x4, #62+        sbc     x4, x4, xzr+        add     x5, x4, #0x1+        lsl     x8, x5, #32+        negs    x6, x8+        ngcs    x7, xzr+        sbc     x8, x8, x5+        adds    x0, x0, x5+        adcs    x1, x1, x6+        adcs    x2, x2, x7+        adcs    x3, x3, x8+        csetm   x5, cc  // cc = lo, ul, last+        adds    x0, x0, x5+        and     x6, x5, #0xffffffff+        adcs    x1, x1, x6+        adcs    x2, x2, xzr+        neg     x7, x6+        adc     x3, x3, x7+        stp     x0, x1, [x19]+        stp     x2, x3, [x19, #16]+        mov     x2, #0xffffffffffffffff         // #-1+        subs    x9, x2, x24+        mov     x2, #0xffffffff                 // #4294967295+        sbcs    x10, x2, x27+        ngcs    x11, x25+        mov     x2, #0xffffffff00000001         // #-4294967295+        sbc     x12, x2, x26+        lsl     x3, x9, #3+        extr    x4, x10, x9, #61+        extr    x5, x11, x10, #61+        extr    x6, x12, x11, #61+        lsr     x7, x12, #61+        mov     x1, #0x3                        // #3+        mul     x8, x14, x1+        umulh   x9, x14, x1+        adds    x3, x3, x8+        mul     x8, x15, x1+        umulh   x10, x15, x1+        adcs    x4, x4, x8+        mul     x8, x13, x1+        umulh   x11, x13, x1+        adcs    x5, x5, x8+        mul     x8, x20, x1+        umulh   x12, x20, x1+        adcs    x6, x6, x8+        adc     x7, x7, xzr+        adds    x4, x4, x9+        adcs    x5, x5, x10+        adcs    x6, x6, x11+        adc     x7, x7, x12+        add     x8, x7, #0x1+        lsl     x10, x8, #32+        adds    x6, x6, x10+        adc     x7, x7, xzr+        neg     x9, x8+        sub     x10, x10, #0x1+        subs    x3, x3, x9+        sbcs    x4, x4, x10+        sbcs    x5, x5, xzr+        sbcs    x6, x6, x8+        sbc     x8, x7, x8+        adds    x3, x3, x8+        and     x9, x8, #0xffffffff+        adcs    x4, x4, x9+        adcs    x5, x5, xzr+        neg     x10, x9+        adc     x6, x6, x10+        stp     x3, x4, [x19, #32]+        stp     x5, x6, [x19, #48]+        ldp     x27, xzr, [sp, #256]+        ldp     x25, x26, [sp, #240]+        ldp     x23, x24, [sp, #224]+        ldp     x21, x22, [sp, #208]+        ldp     x19, x20, [sp, #192]+        CFI_INC_SP(272)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_local_p256_montjdouble)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_local_p256_montjmixadd)++Lp256_scalarmul_local_p256_montjmixadd:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_DEC_SP(192)+        mov     x17, x0+        mov     x19, x1+        mov     x20, x2+        ldp     x2, x3, [x19, #64]+        ldp     x4, x5, [x19, #80]+        umull   x15, w2, w2+        lsr     x11, x2, #32+        umull   x16, w11, w11+        umull   x11, w2, w11+        adds    x15, x15, x11, lsl #33+        lsr     x11, x11, #31+        adc     x16, x16, x11+        umull   x0, w3, w3+        lsr     x11, x3, #32+        umull   x1, w11, w11+        umull   x11, w3, w11+        mul     x12, x2, x3+        umulh   x13, x2, x3+        adds    x0, x0, x11, lsl #33+        lsr     x11, x11, #31+        adc     x1, x1, x11+        adds    x12, x12, x12+        adcs    x13, x13, x13+        adc     x1, x1, xzr+        adds    x16, x16, x12+        adcs    x0, x0, x13+        adc     x1, x1, xzr+        lsl     x12, x15, #32+        subs    x13, x15, x12+        lsr     x11, x15, #32+        sbc     x15, x15, x11+        adds    x16, x16, x12+        adcs    x0, x0, x11+        adcs    x1, x1, x13+        adc     x15, x15, xzr+        lsl     x12, x16, #32+        subs    x13, x16, x12+        lsr     x11, x16, #32+        sbc     x16, x16, x11+        adds    x0, x0, x12+        adcs    x1, x1, x11+        adcs    x15, x15, x13+        adc     x16, x16, xzr+        mul     x6, x2, x4+        mul     x14, x3, x5+        umulh   x8, x2, x4+        subs    x10, x2, x3+        cneg    x10, x10, cc+        csetm   x13, cc+        subs    x12, x5, x4+        cneg    x12, x12, cc+        mul     x11, x10, x12+        umulh   x12, x10, x12+        cinv    x13, x13, cc+        eor     x11, x11, x13+        eor     x12, x12, x13+        adds    x7, x6, x8+        adc     x8, x8, xzr+        umulh   x9, x3, x5+        adds    x7, x7, x14+        adcs    x8, x8, x9+        adc     x9, x9, xzr+        adds    x8, x8, x14+        adc     x9, x9, xzr+        cmn     x13, #0x1+        adcs    x7, x7, x11+        adcs    x8, x8, x12+        adc     x9, x9, x13+        adds    x6, x6, x6+        adcs    x7, x7, x7+        adcs    x8, x8, x8+        adcs    x9, x9, x9+        adc     x10, xzr, xzr+        adds    x6, x6, x0+        adcs    x7, x7, x1+        adcs    x8, x8, x15+        adcs    x9, x9, x16+        adc     x10, x10, xzr+        lsl     x12, x6, #32+        subs    x13, x6, x12+        lsr     x11, x6, #32+        sbc     x6, x6, x11+        adds    x7, x7, x12+        adcs    x8, x8, x11+        adcs    x9, x9, x13+        adcs    x10, x10, x6+        adc     x6, xzr, xzr+        lsl     x12, x7, #32+        subs    x13, x7, x12+        lsr     x11, x7, #32+        sbc     x7, x7, x11+        adds    x8, x8, x12+        adcs    x9, x9, x11+        adcs    x10, x10, x13+        adcs    x6, x6, x7+        adc     x7, xzr, xzr+        mul     x11, x4, x4+        adds    x8, x8, x11+        mul     x12, x5, x5+        umulh   x11, x4, x4+        adcs    x9, x9, x11+        adcs    x10, x10, x12+        umulh   x12, x5, x5+        adcs    x6, x6, x12+        adc     x7, x7, xzr+        mul     x11, x4, x5+        umulh   x12, x4, x5+        adds    x11, x11, x11+        adcs    x12, x12, x12+        adc     x13, xzr, xzr+        adds    x9, x9, x11+        adcs    x10, x10, x12+        adcs    x6, x6, x13+        adcs    x7, x7, xzr+        mov     x11, #0xffffffff+        adds    x5, x8, #0x1+        sbcs    x11, x9, x11+        mov     x13, #0xffffffff00000001+        sbcs    x12, x10, xzr+        sbcs    x13, x6, x13+        sbcs    xzr, x7, xzr+        csel    x8, x5, x8, cs+        csel    x9, x11, x9, cs+        csel    x10, x12, x10, cs+        csel    x6, x13, x6, cs+        stp     x8, x9, [sp]+        stp     x10, x6, [sp, #16]+        ldp     x3, x4, [x19, #64]+        ldp     x5, x6, [x19, #80]+        ldp     x7, x8, [x20, #32]+        ldp     x9, x10, [x20, #48]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #32]+        stp     x11, x12, [sp, #48]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #32]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #48]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #32]+        stp     x3, x4, [sp, #48]+        ldp     x3, x4, [sp]+        ldp     x5, x6, [sp, #16]+        ldp     x7, x8, [x20]+        ldp     x9, x10, [x20, #16]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #64]+        stp     x11, x12, [sp, #80]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #64]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #80]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #64]+        stp     x3, x4, [sp, #80]+        ldp     x3, x4, [sp]+        ldp     x5, x6, [sp, #16]+        ldp     x7, x8, [sp, #32]+        ldp     x9, x10, [sp, #48]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #32]+        stp     x11, x12, [sp, #48]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #32]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #48]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #32]+        stp     x3, x4, [sp, #48]+        ldp     x5, x6, [sp, #64]+        ldp     x4, x3, [x19]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #80]+        ldp     x4, x3, [x19, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #160]+        stp     x7, x8, [sp, #176]+        ldp     x5, x6, [sp, #32]+        ldp     x4, x3, [x19, #32]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #48]+        ldp     x4, x3, [x19, #48]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #32]+        stp     x7, x8, [sp, #48]+        ldp     x2, x3, [sp, #160]+        ldp     x4, x5, [sp, #176]+        umull   x15, w2, w2+        lsr     x11, x2, #32+        umull   x16, w11, w11+        umull   x11, w2, w11+        adds    x15, x15, x11, lsl #33+        lsr     x11, x11, #31+        adc     x16, x16, x11+        umull   x0, w3, w3+        lsr     x11, x3, #32+        umull   x1, w11, w11+        umull   x11, w3, w11+        mul     x12, x2, x3+        umulh   x13, x2, x3+        adds    x0, x0, x11, lsl #33+        lsr     x11, x11, #31+        adc     x1, x1, x11+        adds    x12, x12, x12+        adcs    x13, x13, x13+        adc     x1, x1, xzr+        adds    x16, x16, x12+        adcs    x0, x0, x13+        adc     x1, x1, xzr+        lsl     x12, x15, #32+        subs    x13, x15, x12+        lsr     x11, x15, #32+        sbc     x15, x15, x11+        adds    x16, x16, x12+        adcs    x0, x0, x11+        adcs    x1, x1, x13+        adc     x15, x15, xzr+        lsl     x12, x16, #32+        subs    x13, x16, x12+        lsr     x11, x16, #32+        sbc     x16, x16, x11+        adds    x0, x0, x12+        adcs    x1, x1, x11+        adcs    x15, x15, x13+        adc     x16, x16, xzr+        mul     x6, x2, x4+        mul     x14, x3, x5+        umulh   x8, x2, x4+        subs    x10, x2, x3+        cneg    x10, x10, cc+        csetm   x13, cc+        subs    x12, x5, x4+        cneg    x12, x12, cc+        mul     x11, x10, x12+        umulh   x12, x10, x12+        cinv    x13, x13, cc+        eor     x11, x11, x13+        eor     x12, x12, x13+        adds    x7, x6, x8+        adc     x8, x8, xzr+        umulh   x9, x3, x5+        adds    x7, x7, x14+        adcs    x8, x8, x9+        adc     x9, x9, xzr+        adds    x8, x8, x14+        adc     x9, x9, xzr+        cmn     x13, #0x1+        adcs    x7, x7, x11+        adcs    x8, x8, x12+        adc     x9, x9, x13+        adds    x6, x6, x6+        adcs    x7, x7, x7+        adcs    x8, x8, x8+        adcs    x9, x9, x9+        adc     x10, xzr, xzr+        adds    x6, x6, x0+        adcs    x7, x7, x1+        adcs    x8, x8, x15+        adcs    x9, x9, x16+        adc     x10, x10, xzr+        lsl     x12, x6, #32+        subs    x13, x6, x12+        lsr     x11, x6, #32+        sbc     x6, x6, x11+        adds    x7, x7, x12+        adcs    x8, x8, x11+        adcs    x9, x9, x13+        adcs    x10, x10, x6+        adc     x6, xzr, xzr+        lsl     x12, x7, #32+        subs    x13, x7, x12+        lsr     x11, x7, #32+        sbc     x7, x7, x11+        adds    x8, x8, x12+        adcs    x9, x9, x11+        adcs    x10, x10, x13+        adcs    x6, x6, x7+        adc     x7, xzr, xzr+        mul     x11, x4, x4+        adds    x8, x8, x11+        mul     x12, x5, x5+        umulh   x11, x4, x4+        adcs    x9, x9, x11+        adcs    x10, x10, x12+        umulh   x12, x5, x5+        adcs    x6, x6, x12+        adc     x7, x7, xzr+        mul     x11, x4, x5+        umulh   x12, x4, x5+        adds    x11, x11, x11+        adcs    x12, x12, x12+        adc     x13, xzr, xzr+        adds    x9, x9, x11+        adcs    x10, x10, x12+        adcs    x6, x6, x13+        adcs    x7, x7, xzr+        mov     x11, #0xffffffff+        adds    x5, x8, #0x1+        sbcs    x11, x9, x11+        mov     x13, #0xffffffff00000001+        sbcs    x12, x10, xzr+        sbcs    x13, x6, x13+        sbcs    xzr, x7, xzr+        csel    x8, x5, x8, cs+        csel    x9, x11, x9, cs+        csel    x10, x12, x10, cs+        csel    x6, x13, x6, cs+        stp     x8, x9, [sp, #96]+        stp     x10, x6, [sp, #112]+        ldp     x2, x3, [sp, #32]+        ldp     x4, x5, [sp, #48]+        umull   x15, w2, w2+        lsr     x11, x2, #32+        umull   x16, w11, w11+        umull   x11, w2, w11+        adds    x15, x15, x11, lsl #33+        lsr     x11, x11, #31+        adc     x16, x16, x11+        umull   x0, w3, w3+        lsr     x11, x3, #32+        umull   x1, w11, w11+        umull   x11, w3, w11+        mul     x12, x2, x3+        umulh   x13, x2, x3+        adds    x0, x0, x11, lsl #33+        lsr     x11, x11, #31+        adc     x1, x1, x11+        adds    x12, x12, x12+        adcs    x13, x13, x13+        adc     x1, x1, xzr+        adds    x16, x16, x12+        adcs    x0, x0, x13+        adc     x1, x1, xzr+        lsl     x12, x15, #32+        subs    x13, x15, x12+        lsr     x11, x15, #32+        sbc     x15, x15, x11+        adds    x16, x16, x12+        adcs    x0, x0, x11+        adcs    x1, x1, x13+        adc     x15, x15, xzr+        lsl     x12, x16, #32+        subs    x13, x16, x12+        lsr     x11, x16, #32+        sbc     x16, x16, x11+        adds    x0, x0, x12+        adcs    x1, x1, x11+        adcs    x15, x15, x13+        adc     x16, x16, xzr+        mul     x6, x2, x4+        mul     x14, x3, x5+        umulh   x8, x2, x4+        subs    x10, x2, x3+        cneg    x10, x10, cc+        csetm   x13, cc+        subs    x12, x5, x4+        cneg    x12, x12, cc+        mul     x11, x10, x12+        umulh   x12, x10, x12+        cinv    x13, x13, cc+        eor     x11, x11, x13+        eor     x12, x12, x13+        adds    x7, x6, x8+        adc     x8, x8, xzr+        umulh   x9, x3, x5+        adds    x7, x7, x14+        adcs    x8, x8, x9+        adc     x9, x9, xzr+        adds    x8, x8, x14+        adc     x9, x9, xzr+        cmn     x13, #0x1+        adcs    x7, x7, x11+        adcs    x8, x8, x12+        adc     x9, x9, x13+        adds    x6, x6, x6+        adcs    x7, x7, x7+        adcs    x8, x8, x8+        adcs    x9, x9, x9+        adc     x10, xzr, xzr+        adds    x6, x6, x0+        adcs    x7, x7, x1+        adcs    x8, x8, x15+        adcs    x9, x9, x16+        adc     x10, x10, xzr+        lsl     x12, x6, #32+        subs    x13, x6, x12+        lsr     x11, x6, #32+        sbc     x6, x6, x11+        adds    x7, x7, x12+        adcs    x8, x8, x11+        adcs    x9, x9, x13+        adcs    x10, x10, x6+        adc     x6, xzr, xzr+        lsl     x12, x7, #32+        subs    x13, x7, x12+        lsr     x11, x7, #32+        sbc     x7, x7, x11+        adds    x8, x8, x12+        adcs    x9, x9, x11+        adcs    x10, x10, x13+        adcs    x6, x6, x7+        adc     x7, xzr, xzr+        mul     x11, x4, x4+        adds    x8, x8, x11+        mul     x12, x5, x5+        umulh   x11, x4, x4+        adcs    x9, x9, x11+        adcs    x10, x10, x12+        umulh   x12, x5, x5+        adcs    x6, x6, x12+        adc     x7, x7, xzr+        mul     x11, x4, x5+        umulh   x12, x4, x5+        adds    x11, x11, x11+        adcs    x12, x12, x12+        adc     x13, xzr, xzr+        adds    x9, x9, x11+        adcs    x10, x10, x12+        adcs    x6, x6, x13+        adcs    x7, x7, xzr+        mov     x11, #0xffffffff+        adds    x5, x8, #0x1+        sbcs    x11, x9, x11+        mov     x13, #0xffffffff00000001+        sbcs    x12, x10, xzr+        sbcs    x13, x6, x13+        sbcs    xzr, x7, xzr+        csel    x8, x5, x8, cs+        csel    x9, x11, x9, cs+        csel    x10, x12, x10, cs+        csel    x6, x13, x6, cs+        stp     x8, x9, [sp]+        stp     x10, x6, [sp, #16]+        ldp     x3, x4, [sp, #96]+        ldp     x5, x6, [sp, #112]+        ldp     x7, x8, [x19]+        ldp     x9, x10, [x19, #16]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #128]+        stp     x11, x12, [sp, #144]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #128]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #144]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #128]+        stp     x3, x4, [sp, #144]+        ldp     x3, x4, [sp, #96]+        ldp     x5, x6, [sp, #112]+        ldp     x7, x8, [sp, #64]+        ldp     x9, x10, [sp, #80]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #64]+        stp     x11, x12, [sp, #80]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #64]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #80]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #64]+        stp     x3, x4, [sp, #80]+        ldp     x5, x6, [sp]+        ldp     x4, x3, [sp, #128]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #16]+        ldp     x4, x3, [sp, #144]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp]+        stp     x7, x8, [sp, #16]+        ldp     x5, x6, [sp, #64]+        ldp     x4, x3, [sp, #128]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #80]+        ldp     x4, x3, [sp, #144]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #96]+        stp     x7, x8, [sp, #112]+        ldp     x3, x4, [sp, #160]+        ldp     x5, x6, [sp, #176]+        ldp     x7, x8, [x19, #64]+        ldp     x9, x10, [x19, #80]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #160]+        stp     x11, x12, [sp, #176]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #160]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #176]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #160]+        stp     x3, x4, [sp, #176]+        ldp     x5, x6, [sp]+        ldp     x4, x3, [sp, #64]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #16]+        ldp     x4, x3, [sp, #80]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp]+        stp     x7, x8, [sp, #16]+        ldp     x5, x6, [sp, #128]+        ldp     x4, x3, [sp]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #144]+        ldp     x4, x3, [sp, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #128]+        stp     x7, x8, [sp, #144]+        ldp     x3, x4, [sp, #96]+        ldp     x5, x6, [sp, #112]+        ldp     x7, x8, [x19, #32]+        ldp     x9, x10, [x19, #48]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #96]+        stp     x11, x12, [sp, #112]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #96]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #112]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #96]+        stp     x3, x4, [sp, #112]+        ldp     x3, x4, [sp, #32]+        ldp     x5, x6, [sp, #48]+        ldp     x7, x8, [sp, #128]+        ldp     x9, x10, [sp, #144]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #128]+        stp     x11, x12, [sp, #144]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #128]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #144]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #128]+        stp     x3, x4, [sp, #144]+        ldp     x5, x6, [sp, #128]+        ldp     x4, x3, [sp, #96]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #144]+        ldp     x4, x3, [sp, #112]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #128]+        stp     x7, x8, [sp, #144]+        ldp     x0, x1, [x19, #64]+        ldp     x2, x3, [x19, #80]+        orr     x4, x0, x1+        orr     x5, x2, x3+        orr     x4, x4, x5+        cmp     x4, xzr+        ldp     x0, x1, [sp]+        ldp     x12, x13, [x20]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x2, x3, [sp, #16]+        ldp     x12, x13, [x20, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x4, x5, [sp, #128]+        ldp     x12, x13, [x20, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x6, x7, [sp, #144]+        ldp     x12, x13, [x20, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x8, x9, [sp, #160]+        mov     x12, #0x1+        mov     x13, #0xffffffff00000000+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x10, x11, [sp, #176]+        mov     x12, #0xffffffffffffffff+        mov     x13, #0xfffffffe+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        stp     x0, x1, [x17]+        stp     x2, x3, [x17, #16]+        stp     x4, x5, [x17, #32]+        stp     x6, x7, [x17, #48]+        stp     x8, x9, [x17, #64]+        stp     x10, x11, [x17, #80]+        CFI_INC_SP(192)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_local_p256_montjmixadd)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/p256_scalarmul_alt.S view
@@ -0,0 +1,6235 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Scalar multiplication for P-256+// Input scalar[4], point[8]; output res[8]+//+// extern void p256_scalarmul_alt+//   (uint64_t res[static 8],+//    const uint64_t scalar[static 4],+//    const uint64_t point[static 8]);+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-256, returns the point (X,Y) = n * P. The input and output+// are affine points, and in the case of the point at infinity as+// the result, (0,0) is returned.+//+// Standard ARM ABI: X0 = res, X1 = scalar, X2 = point+// ----------------------------------------------------------------------------+++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_scalarmul_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_scalarmul_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_scalarmul_alt)+++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Safe copies of inputs (res lasts the whole code, point not so long)+// and additional values in variables, with some aliasing++#define res x19+#define sgn x20+#define j x20+#define point x21++// Intermediate variables on the stack. The last z2, z3 values can+// safely be overlaid on the table, which is no longer needed at the end.++#define scalarb sp, #(0*NUMSIZE)+#define acc sp, #(1*NUMSIZE)+#define tabent sp, #(4*NUMSIZE)++#define tab sp, #(7*NUMSIZE)++#define z2 sp, #(7*NUMSIZE)+#define z3 sp, #(8*NUMSIZE)++#define NSPACE 31*NUMSIZE++// Loading large constants++#define movbig(nn,n3,n2,n1,n0)                                      \+        movz    nn, n0 __LF                                            \+        movk    nn, n1, lsl #16 __LF                                   \+        movk    nn, n2, lsl #32 __LF                                   \+        movk    nn, n3, lsl #48++S2N_BN_SYMBOL(p256_scalarmul_alt):+        CFI_START++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x30)+        CFI_DEC_SP(NSPACE)++// Preserve the "res" and "point" input arguments. We load and process the+// scalar immediately so we don't bother preserving that input argument.+// Also, "point" is only needed early on and so its register gets re-used.++        mov     res, x0+        mov     point, x2++// Load the digits of group order n_256 = [x12;x13;x14;x15]++        movbig(x12, #0xf3b9, #0xcac2, #0xfc63, #0x2551)+        movbig(x13, #0xbce6, #0xfaad, #0xa717, #0x9e84)+        mov     x14, #0xffffffffffffffff+        mov     x15, #0xffffffff00000000++// First, reduce the input scalar mod n_256, i.e. conditionally subtract n_256++        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]++        subs    x6, x2, x12+        sbcs    x7, x3, x13+        sbcs    x8, x4, x14+        sbcs    x9, x5, x15++        csel    x2, x2, x6, cc+        csel    x3, x3, x7, cc+        csel    x4, x4, x8, cc+        csel    x5, x5, x9, cc++// Now if the top bit of the reduced scalar is set, negate it mod n_256,+// i.e. do n |-> n_256 - n. Remember the sign as "sgn" so we can+// correspondingly negate the point below.++        subs    x6, x12, x2+        sbcs    x7, x13, x3+        sbcs    x8, x14, x4+        sbc     x9, x15, x5++        tst     x5, #0x8000000000000000+        csel    x2, x2, x6, eq+        csel    x3, x3, x7, eq+        csel    x4, x4, x8, eq+        csel    x5, x5, x9, eq+        cset    sgn, ne++// In either case then add the recoding constant 0x08888...888 to allow+// signed digits.++        mov     x6, 0x8888888888888888+        adds    x2, x2, x6+        adcs    x3, x3, x6+        bic     x7, x6, #0xF000000000000000+        adcs    x4, x4, x6+        adc     x5, x5, x7++        stp     x2, x3, [scalarb]+        stp     x4, x5, [scalarb+16]++// Set the tab[0] table entry to Montgomery-Jacobian point = 1 * P+// The z coordinate is just the Montgomery form of the constant 1.++        add     x0, tab+        mov     x1, point+        CFI_BL(Lp256_scalarmul_alt_local_tomont_p256)++        add     x1, point, #32+        add     x0, tab+32+        CFI_BL(Lp256_scalarmul_alt_local_tomont_p256)++        mov     x0, #0x0000000000000001+        mov     x1, #0xffffffff00000000+        stp     x0, x1, [tab+64]+        mov     x2, #0xffffffffffffffff+        mov     x3, #0x00000000fffffffe+        stp     x2, x3, [tab+80]++// If the top bit of the scalar was set, negate (y coordinate of) the point++        ldp     x4, x5, [tab+32]+        ldp     x6, x7, [tab+48]++        mov     x0, 0xffffffffffffffff+        subs    x0, x0, x4+        mov     x1, 0x00000000ffffffff+        sbcs    x1, x1, x5+        mov     x3, 0xffffffff00000001+        sbcs    x2, xzr, x6+        sbc     x3, x3, x7++        cmp     sgn, xzr+        csel    x4, x0, x4, ne+        csel    x5, x1, x5, ne+        csel    x6, x2, x6, ne+        csel    x7, x3, x7, ne++        stp     x4, x5, [tab+32]+        stp     x6, x7, [tab+48]++// Compute and record tab[1] = 2 * p, ..., tab[7] = 8 * P++        add     x0, tab+96*1+        add     x1, tab+        CFI_BL(Lp256_scalarmul_alt_local_p256_montjdouble)++        add     x0, tab+96*2+        add     x1, tab+96*1+        add     x2, tab+        CFI_BL(Lp256_scalarmul_alt_local_p256_montjmixadd)++        add     x0, tab+96*3+        add     x1, tab+96*1+        CFI_BL(Lp256_scalarmul_alt_local_p256_montjdouble)++        add     x0, tab+96*4+        add     x1, tab+96*3+        add     x2, tab+        CFI_BL(Lp256_scalarmul_alt_local_p256_montjmixadd)++        add     x0, tab+96*5+        add     x1, tab+96*2+        CFI_BL(Lp256_scalarmul_alt_local_p256_montjdouble)++        add     x0, tab+96*6+        add     x1, tab+96*5+        add     x2, tab+        CFI_BL(Lp256_scalarmul_alt_local_p256_montjmixadd)++        add     x0, tab+96*7+        add     x1, tab+96*3+        CFI_BL(Lp256_scalarmul_alt_local_p256_montjdouble)++// Initialize the accumulator as a table entry for top 4 bits (unrecoded)++        ldr     x14, [scalarb+24]+        lsr     x14, x14, #60++        mov     x0, xzr+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, xzr+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        mov     x9, xzr+        mov     x10, xzr+        mov     x11, xzr+        add     x15, tab++        .set i, 1+.rep 8+        cmp     x14, #i+        ldp     x12, x13, [x15]+        csel    x0, x12, x0, eq+        csel    x1, x13, x1, eq+        ldp     x12, x13, [x15, #16]+        csel    x2, x12, x2, eq+        csel    x3, x13, x3, eq+        ldp     x12, x13, [x15, #32]+        csel    x4, x12, x4, eq+        csel    x5, x13, x5, eq+        ldp     x12, x13, [x15, #48]+        csel    x6, x12, x6, eq+        csel    x7, x13, x7, eq+        ldp     x12, x13, [x15, #64]+        csel    x8, x12, x8, eq+        csel    x9, x13, x9, eq+        ldp     x12, x13, [x15, #80]+        csel    x10, x12, x10, eq+        csel    x11, x13, x11, eq+        add     x15, x15, #96+        .set    i, (i+1)+.endr+        stp     x0, x1, [acc]+        stp     x2, x3, [acc+16]+        stp     x4, x5, [acc+32]+        stp     x6, x7, [acc+48]+        stp     x8, x9, [acc+64]+        stp     x10, x11, [acc+80]++        mov     j, #252++// Main loop over size-4 bitfields: double 4 times then add signed digit++Lp256_scalarmul_alt_loop:+        sub     j, j, #4++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp256_scalarmul_alt_local_p256_montjdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp256_scalarmul_alt_local_p256_montjdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp256_scalarmul_alt_local_p256_montjdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp256_scalarmul_alt_local_p256_montjdouble)++        lsr     x2, j, #6+        ldr     x14, [sp, x2, lsl #3]   // Exploits scalarb = sp exactly+        lsr     x14, x14, j+        and     x14, x14, #15++        subs    x14, x14, #8+        cset    x16, lo                 // x16 = sign of digit (1 = negative)+        cneg    x14, x14, lo            // x14 = absolute value of digit++// Conditionally select the table entry tab[i-1] = i * P in constant time++        mov     x0, xzr+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, xzr+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        mov     x9, xzr+        mov     x10, xzr+        mov     x11, xzr+        add     x15, tab+        .set i, 1+.rep 8+        cmp     x14, #i+        ldp     x12, x13, [x15]+        csel    x0, x12, x0, eq+        csel    x1, x13, x1, eq+        ldp     x12, x13, [x15, #16]+        csel    x2, x12, x2, eq+        csel    x3, x13, x3, eq+        ldp     x12, x13, [x15, #32]+        csel    x4, x12, x4, eq+        csel    x5, x13, x5, eq+        ldp     x12, x13, [x15, #48]+        csel    x6, x12, x6, eq+        csel    x7, x13, x7, eq+        ldp     x12, x13, [x15, #64]+        csel    x8, x12, x8, eq+        csel    x9, x13, x9, eq+        ldp     x12, x13, [x15, #80]+        csel    x10, x12, x10, eq+        csel    x11, x13, x11, eq+        add     x15, x15, #96+        .set    i, (i+1)+.endr++// Store it to "tabent" with the y coordinate optionally negated++        stp     x0, x1, [tabent]+        stp     x2, x3, [tabent+16]++        mov     x0, 0xffffffffffffffff+        subs    x0, x0, x4+        mov     x1, 0x00000000ffffffff+        sbcs    x1, x1, x5+        mov     x3, 0xffffffff00000001+        sbcs    x2, xzr, x6+        sbc     x3, x3, x7++        cmp     x16, xzr+        csel    x4, x0, x4, ne+        csel    x5, x1, x5, ne+        csel    x6, x2, x6, ne+        csel    x7, x3, x7, ne++        stp     x4, x5, [tabent+32]+        stp     x6, x7, [tabent+48]+        stp     x8, x9, [tabent+64]+        stp     x10, x11, [tabent+80]++        add     x0, acc+        add     x1, acc+        add     x2, tabent+        CFI_BL(Lp256_scalarmul_alt_local_p256_montjadd)++        cbnz    j, Lp256_scalarmul_alt_loop++// That's the end of the main loop, and we just need to translate+// back from the Jacobian representation to affine. First of all,+// let z2 = 1/z^2 and z3 = 1/z^3, both without Montgomery form++        add     x0, z2+        add     x1, acc+64+        CFI_BL(Lp256_scalarmul_alt_local_montsqr_p256)++        add     x0, z3+        add     x2, z2+        add     x1, acc+64+        CFI_BL(Lp256_scalarmul_alt_local_montmul_p256)++        add     x0, z2+        add     x1, z3+        CFI_BL(Lp256_scalarmul_alt_local_demont_p256)++        add     x0, z3+        add     x1, z2+        CFI_BL(Lp256_scalarmul_alt_local_inv_p256)++        add     x0, z2+        add     x2, z3+        add     x1, acc+64+        CFI_BL(Lp256_scalarmul_alt_local_montmul_p256)++// Convert back from Jacobian (X,Y,Z) |-> (X/Z^2, Y/Z^3)++        add     x1, acc+        add     x2, z2+        mov     x0, res+        CFI_BL(Lp256_scalarmul_alt_local_montmul_p256)++        add     x0, res, #32+        add     x1, acc+32+        add     x2, z3+        CFI_BL(Lp256_scalarmul_alt_local_montmul_p256)++// Restore stack and registers and return++        CFI_INC_SP(NSPACE)+        CFI_POP2(x21,x30)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_scalarmul_alt)++// Local copies of subroutines, complete clones at the moment++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_alt_local_demont_p256)++Lp256_scalarmul_alt_local_demont_p256:+        CFI_START+        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]+        lsl     x7, x2, #32+        subs    x8, x2, x7+        lsr     x6, x2, #32+        sbc     x2, x2, x6+        adds    x3, x3, x7+        adcs    x4, x4, x6+        adcs    x5, x5, x8+        adc     x2, x2, xzr+        lsl     x7, x3, #32+        subs    x8, x3, x7+        lsr     x6, x3, #32+        sbc     x3, x3, x6+        adds    x4, x4, x7+        adcs    x5, x5, x6+        adcs    x2, x2, x8+        adc     x3, x3, xzr+        lsl     x7, x4, #32+        subs    x8, x4, x7+        lsr     x6, x4, #32+        sbc     x4, x4, x6+        adds    x5, x5, x7+        adcs    x2, x2, x6+        adcs    x3, x3, x8+        adc     x4, x4, xzr+        lsl     x7, x5, #32+        subs    x8, x5, x7+        lsr     x6, x5, #32+        sbc     x5, x5, x6+        adds    x2, x2, x7+        adcs    x3, x3, x6+        adcs    x4, x4, x8+        adc     x5, x5, xzr+        stp     x2, x3, [x0]+        stp     x4, x5, [x0, #16]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_alt_local_demont_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_alt_local_inv_p256)++Lp256_scalarmul_alt_local_inv_p256:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_DEC_SP(160)+        mov     x20, x0+        mov     x10, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x13, #0xffffffff00000001+        stp     x10, x11, [sp]+        stp     xzr, x13, [sp, #16]+        str     xzr, [sp, #32]+        ldp     x2, x3, [x1]+        subs    x10, x2, x10+        sbcs    x11, x3, x11+        ldp     x4, x5, [x1, #16]+        sbcs    x12, x4, xzr+        sbcs    x13, x5, x13+        csel    x2, x2, x10, cc+        csel    x3, x3, x11, cc+        csel    x4, x4, x12, cc+        csel    x5, x5, x13, cc+        stp     x2, x3, [sp, #48]+        stp     x4, x5, [sp, #64]+        str     xzr, [sp, #80]+        stp     xzr, xzr, [sp, #96]+        stp     xzr, xzr, [sp, #112]+        mov     x10, #0x4000000000000+        stp     x10, xzr, [sp, #128]+        stp     xzr, xzr, [sp, #144]+        mov     x21, #0xa+        mov     x22, #0x1+        b       Lp256_scalarmul_alt_inv_midloop+Lp256_scalarmul_alt_inv_loop:+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        and     x0, x12, x16+        and     x1, x13, x17+        add     x19, x0, x1+        ldr     x7, [sp]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #48]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x7, [sp, #8]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #56]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [sp]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [sp, #48]+        ldr     x7, [sp, #16]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #64]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [sp, #8]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [sp, #56]+        ldr     x7, [sp, #24]+        eor     x1, x7, x14+        ldr     x23, [sp, #32]+        eor     x3, x23, x14+        and     x3, x3, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #72]+        eor     x1, x8, x15+        ldr     x24, [sp, #80]+        eor     x0, x24, x15+        and     x0, x0, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x5, x6, #59+        str     x6, [sp, #16]+        extr    x5, x3, x5, #59+        str     x5, [sp, #24]+        asr     x3, x3, #59+        str     x3, [sp, #32]+        eor     x1, x7, x16+        eor     x5, x23, x16+        and     x5, x5, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        eor     x0, x24, x17+        and     x0, x0, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x4, x2, x4, #59+        str     x4, [sp, #64]+        extr    x2, x5, x2, #59+        str     x2, [sp, #72]+        asr     x5, x5, #59+        str     x5, [sp, #80]+        ldr     x7, [sp, #96]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #128]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #96]+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        str     x5, [sp, #128]+        adc     x3, x3, x1+        ldr     x7, [sp, #104]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #136]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #104]+        adc     x6, x6, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        str     x3, [sp, #136]+        adc     x4, x4, x1+        ldr     x7, [sp, #112]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #144]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #112]+        adc     x5, x5, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        str     x4, [sp, #144]+        adc     x2, x2, x1+        ldr     x7, [sp, #120]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #152]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldp     x0, x1, [sp, #96]+        ldr     x6, [sp, #112]+        mov     x14, #0xe000000000000000+        adds    x0, x0, x14+        sbcs    x1, x1, xzr+        mov     x11, #0x1fffffff+        adcs    x6, x6, x11+        mov     x10, #0x2000000000000000+        adcs    x5, x5, x10+        mov     x14, #0x1fffffffe0000000+        adc     x3, x3, x14+        lsl     x11, x0, #32+        subs    x14, x0, x11+        lsr     x10, x0, #32+        sbc     x0, x0, x10+        adds    x1, x1, x11+        adcs    x6, x6, x10+        adcs    x5, x5, x14+        adcs    x3, x3, x0+        mov     x14, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x10, #0xffffffff00000001+        csel    x14, x14, xzr, cs+        csel    x11, x11, xzr, cs+        csel    x10, x10, xzr, cs+        subs    x1, x1, x14+        sbcs    x6, x6, x11+        sbcs    x5, x5, xzr+        sbc     x3, x3, x10+        stp     x1, x6, [sp, #96]+        stp     x5, x3, [sp, #112]+        eor     x1, x7, x16+        and     x5, x16, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        and     x0, x17, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        ldp     x0, x1, [sp, #128]+        ldr     x3, [sp, #144]+        mov     x14, #0xe000000000000000+        adds    x0, x0, x14+        sbcs    x1, x1, xzr+        mov     x11, #0x1fffffff+        adcs    x3, x3, x11+        mov     x10, #0x2000000000000000+        adcs    x2, x2, x10+        mov     x14, #0x1fffffffe0000000+        adc     x5, x5, x14+        lsl     x11, x0, #32+        subs    x14, x0, x11+        lsr     x10, x0, #32+        sbc     x0, x0, x10+        adds    x1, x1, x11+        adcs    x3, x3, x10+        adcs    x2, x2, x14+        adcs    x5, x5, x0+        mov     x14, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x10, #0xffffffff00000001+        csel    x14, x14, xzr, cs+        csel    x11, x11, xzr, cs+        csel    x10, x10, xzr, cs+        subs    x1, x1, x14+        sbcs    x3, x3, x11+        sbcs    x2, x2, xzr+        sbc     x5, x5, x10+        stp     x1, x3, [sp, #128]+        stp     x2, x5, [sp, #144]+Lp256_scalarmul_alt_inv_midloop:+        mov     x1, x22+        ldr     x2, [sp]+        ldr     x3, [sp, #48]+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x8, x4, #0x100, lsl #12+        sbfx    x8, x8, #21, #21+        mov     x11, #0x100000+        add     x11, x11, x11, lsl #21+        add     x9, x4, x11+        asr     x9, x9, #42+        add     x10, x5, #0x100, lsl #12+        sbfx    x10, x10, #21, #21+        add     x11, x5, x11+        asr     x11, x11, #42+        mul     x6, x8, x2+        mul     x7, x9, x3+        mul     x2, x10, x2+        mul     x3, x11, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #21, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #42+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #21, #21+        add     x15, x5, x15+        asr     x15, x15, #42+        mul     x6, x12, x2+        mul     x7, x13, x3+        mul     x2, x14, x2+        mul     x3, x15, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        mul     x2, x12, x8+        mul     x3, x12, x9+        mul     x6, x14, x8+        mul     x7, x14, x9+        madd    x8, x13, x10, x2+        madd    x9, x13, x11, x3+        madd    x16, x15, x10, x6+        madd    x17, x15, x11, x7+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #22, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #43+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #22, #21+        add     x15, x5, x15+        asr     x15, x15, #43+        mneg    x2, x12, x8+        mneg    x3, x12, x9+        mneg    x4, x14, x8+        mneg    x5, x14, x9+        msub    x10, x13, x16, x2+        msub    x11, x13, x17, x3+        msub    x12, x15, x16, x4+        msub    x13, x15, x17, x5+        mov     x22, x1+        subs    x21, x21, #0x1+        b.ne    Lp256_scalarmul_alt_inv_loop+        ldr     x0, [sp]+        ldr     x1, [sp, #48]+        mul     x0, x0, x10+        madd    x1, x1, x11, x0+        asr     x0, x1, #63+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        eor     x14, x14, x0+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        eor     x15, x15, x0+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        eor     x16, x16, x0+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        eor     x17, x17, x0+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        ldr     x7, [sp, #96]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #128]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #96]+        adc     x2, x2, x1+        ldr     x7, [sp, #104]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #136]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #104]+        adc     x6, x6, x1+        ldr     x7, [sp, #112]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #144]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #112]+        adc     x5, x5, x1+        ldr     x7, [sp, #120]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #152]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldp     x0, x1, [sp, #96]+        ldr     x2, [sp, #112]+        mov     x14, #0xe000000000000000+        adds    x0, x0, x14+        sbcs    x1, x1, xzr+        mov     x11, #0x1fffffff+        adcs    x2, x2, x11+        mov     x10, #0x2000000000000000+        adcs    x5, x5, x10+        mov     x14, #0x1fffffffe0000000+        adc     x3, x3, x14+        lsl     x11, x0, #32+        subs    x14, x0, x11+        lsr     x10, x0, #32+        sbc     x0, x0, x10+        adds    x1, x1, x11+        adcs    x2, x2, x10+        adcs    x5, x5, x14+        adcs    x3, x3, x0+        mov     x14, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x10, #0xffffffff00000001+        csel    x14, x14, xzr, cs+        csel    x11, x11, xzr, cs+        csel    x10, x10, xzr, cs+        subs    x1, x1, x14+        sbcs    x2, x2, x11+        sbcs    x5, x5, xzr+        sbc     x3, x3, x10+        mov     x10, #0xffffffffffffffff+        subs    x10, x1, x10+        mov     x11, #0xffffffff+        sbcs    x11, x2, x11+        mov     x13, #0xffffffff00000001+        sbcs    x12, x5, xzr+        sbcs    x13, x3, x13+        csel    x10, x1, x10, cc+        csel    x11, x2, x11, cc+        csel    x12, x5, x12, cc+        csel    x13, x3, x13, cc+        stp     x10, x11, [x20]+        stp     x12, x13, [x20, #16]+        CFI_INC_SP(160)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_alt_local_inv_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_alt_local_montmul_p256)++Lp256_scalarmul_alt_local_montmul_p256:+        CFI_START+        ldp     x3, x4, [x1]+        ldp     x7, x8, [x2]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x2, #16]+        mul     x11, x3, x9+        umulh   x15, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x16, x3, x10+        adcs    x15, x15, x11+        adc     x16, x16, xzr+        ldp     x5, x6, [x1, #16]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x15, x15, x11+        mul     x11, x4, x10+        adcs    x16, x16, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x15, x15, x11+        umulh   x11, x4, x9+        adcs    x16, x16, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x15, x15, x11+        mul     x11, x5, x9+        adcs    x16, x16, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x15, x15, x11+        umulh   x11, x5, x8+        adcs    x16, x16, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x15, x15, x11+        mul     x11, x6, x8+        adcs    x16, x16, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x15, x15, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x16, x16, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x15, x15, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x15, x15, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x15, lsl #32+        lsr     x11, x15, #32+        adcs    x13, x13, x11+        mul     x11, x15, x10+        umulh   x15, x15, x10+        adcs    x14, x14, x11+        adc     x15, x15, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x15, x15, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x16, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x15, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x16, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x15, x15, x5, cc+        stp     x12, x13, [x0]+        stp     x14, x15, [x0, #16]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_alt_local_montmul_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_alt_local_montsqr_p256)++Lp256_scalarmul_alt_local_montsqr_p256:+        CFI_START+        ldp     x2, x3, [x1]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [x1, #16]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        mov     x5, #0xffffffff00000001+        adds    x9, x9, x8, lsl #32+        lsr     x2, x8, #32+        adcs    x10, x10, x2+        mul     x2, x8, x5+        umulh   x8, x8, x5+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x2, x9, #32+        adcs    x11, x11, x2+        mul     x2, x9, x5+        umulh   x9, x9, x5+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x2, x10, #32+        adcs    x8, x8, x2+        mul     x2, x10, x5+        umulh   x10, x10, x5+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x2, x11, #32+        adcs    x9, x9, x2+        mul     x2, x11, x5+        umulh   x11, x11, x5+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        cset    x2, cs+        mov     x3, #0xffffffff+        adds    x12, x8, #0x1+        sbcs    x13, x9, x3+        sbcs    x14, x10, xzr+        sbcs    x7, x11, x5+        sbcs    xzr, x2, xzr+        csel    x8, x8, x12, cc+        csel    x9, x9, x13, cc+        csel    x10, x10, x14, cc+        csel    x11, x11, x7, cc+        stp     x8, x9, [x0]+        stp     x10, x11, [x0, #16]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_alt_local_montsqr_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_alt_local_tomont_p256)++Lp256_scalarmul_alt_local_tomont_p256:+        CFI_START+        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]+        mov     x1, #0xffffffffffffffff+        mov     x7, #0xffffffff+        mov     x9, #0xffffffff00000001+        subs    x1, x2, x1+        sbcs    x7, x3, x7+        sbcs    x8, x4, xzr+        sbcs    x9, x5, x9+        csel    x2, x2, x1, cc+        csel    x3, x3, x7, cc+        csel    x4, x4, x8, cc+        csel    x5, x5, x9, cc+        cmp     xzr, xzr+        extr    x9, x5, x4, #32+        adcs    xzr, x4, x9+        lsr     x9, x5, #32+        adcs    x9, x5, x9+        csetm   x6, cs+        orr     x9, x9, x6+        lsl     x7, x9, #32+        lsr     x8, x9, #32+        adds    x4, x4, x7+        adc     x5, x5, x8+        negs    x6, x9+        sbcs    x7, x7, xzr+        sbc     x8, x8, xzr+        negs    x6, x6+        sbcs    x2, x2, x7+        sbcs    x3, x3, x8+        sbcs    x4, x4, x9+        sbcs    x5, x5, x9+        adds    x6, x6, x5+        mov     x7, #0xffffffff+        and     x7, x7, x5+        adcs    x2, x2, x7+        adcs    x3, x3, xzr+        mov     x7, #0xffffffff00000001+        and     x7, x7, x5+        adc     x4, x4, x7+        cmp     xzr, xzr+        extr    x9, x4, x3, #32+        adcs    xzr, x3, x9+        lsr     x9, x4, #32+        adcs    x9, x4, x9+        csetm   x5, cs+        orr     x9, x9, x5+        lsl     x7, x9, #32+        lsr     x8, x9, #32+        adds    x3, x3, x7+        adc     x4, x4, x8+        negs    x5, x9+        sbcs    x7, x7, xzr+        sbc     x8, x8, xzr+        negs    x5, x5+        sbcs    x6, x6, x7+        sbcs    x2, x2, x8+        sbcs    x3, x3, x9+        sbcs    x4, x4, x9+        adds    x5, x5, x4+        mov     x7, #0xffffffff+        and     x7, x7, x4+        adcs    x6, x6, x7+        adcs    x2, x2, xzr+        mov     x7, #0xffffffff00000001+        and     x7, x7, x4+        adc     x3, x3, x7+        cmp     xzr, xzr+        extr    x9, x3, x2, #32+        adcs    xzr, x2, x9+        lsr     x9, x3, #32+        adcs    x9, x3, x9+        csetm   x4, cs+        orr     x9, x9, x4+        lsl     x7, x9, #32+        lsr     x8, x9, #32+        adds    x2, x2, x7+        adc     x3, x3, x8+        negs    x4, x9+        sbcs    x7, x7, xzr+        sbc     x8, x8, xzr+        negs    x4, x4+        sbcs    x5, x5, x7+        sbcs    x6, x6, x8+        sbcs    x2, x2, x9+        sbcs    x3, x3, x9+        adds    x4, x4, x3+        mov     x7, #0xffffffff+        and     x7, x7, x3+        adcs    x5, x5, x7+        adcs    x6, x6, xzr+        mov     x7, #0xffffffff00000001+        and     x7, x7, x3+        adc     x2, x2, x7+        cmp     xzr, xzr+        extr    x9, x2, x6, #32+        adcs    xzr, x6, x9+        lsr     x9, x2, #32+        adcs    x9, x2, x9+        csetm   x3, cs+        orr     x9, x9, x3+        lsl     x7, x9, #32+        lsr     x8, x9, #32+        adds    x6, x6, x7+        adc     x2, x2, x8+        negs    x3, x9+        sbcs    x7, x7, xzr+        sbc     x8, x8, xzr+        negs    x3, x3+        sbcs    x4, x4, x7+        sbcs    x5, x5, x8+        sbcs    x6, x6, x9+        sbcs    x2, x2, x9+        adds    x3, x3, x2+        mov     x7, #0xffffffff+        and     x7, x7, x2+        adcs    x4, x4, x7+        adcs    x5, x5, xzr+        mov     x7, #0xffffffff00000001+        and     x7, x7, x2+        adc     x6, x6, x7+        stp     x3, x4, [x0]+        stp     x5, x6, [x0, #16]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_alt_local_tomont_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_alt_local_p256_montjadd)++Lp256_scalarmul_alt_local_p256_montjadd:+        CFI_START+        CFI_DEC_SP(224)+        mov     x15, x0+        mov     x16, x1+        mov     x17, x2+        ldp     x2, x3, [x16, #64]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [x16, #80]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x8, x3+        umulh   x8, x8, x3+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x9, x3+        umulh   x9, x9, x3+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x10, x3+        umulh   x10, x10, x3+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x11, x3+        umulh   x11, x11, x3+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        mov     x2, #0xffffffffffffffff+        csel    x2, xzr, x2, cc+        mov     x3, #0xffffffff+        csel    x3, xzr, x3, cc+        mov     x5, #0xffffffff00000001+        csel    x5, xzr, x5, cc+        subs    x8, x8, x2+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbc     x11, x11, x5+        stp     x8, x9, [sp]+        stp     x10, x11, [sp, #16]+        ldp     x2, x3, [x17, #64]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [x17, #80]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x8, x3+        umulh   x8, x8, x3+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x9, x3+        umulh   x9, x9, x3+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x10, x3+        umulh   x10, x10, x3+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x11, x3+        umulh   x11, x11, x3+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        mov     x2, #0xffffffffffffffff+        csel    x2, xzr, x2, cc+        mov     x3, #0xffffffff+        csel    x3, xzr, x3, cc+        mov     x5, #0xffffffff00000001+        csel    x5, xzr, x5, cc+        subs    x8, x8, x2+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbc     x11, x11, x5+        stp     x8, x9, [sp, #160]+        stp     x10, x11, [sp, #176]+        ldp     x3, x4, [x17, #64]+        ldp     x7, x8, [x16, #32]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x16, #48]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [x17, #80]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #192]+        stp     x14, x0, [sp, #208]+        ldp     x3, x4, [x16, #64]+        ldp     x7, x8, [x17, #32]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x17, #48]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [x16, #80]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #32]+        stp     x14, x0, [sp, #48]+        ldp     x3, x4, [sp]+        ldp     x7, x8, [x17]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x17, #16]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #16]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #64]+        stp     x14, x0, [sp, #80]+        ldp     x3, x4, [sp, #160]+        ldp     x7, x8, [x16]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x16, #16]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #176]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #128]+        stp     x14, x0, [sp, #144]+        ldp     x3, x4, [sp]+        ldp     x7, x8, [sp, #32]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #48]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #16]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #32]+        stp     x14, x0, [sp, #48]+        ldp     x3, x4, [sp, #160]+        ldp     x7, x8, [sp, #192]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #208]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #176]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #192]+        stp     x14, x0, [sp, #208]+        ldp     x5, x6, [sp, #64]+        ldp     x4, x3, [sp, #128]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #80]+        ldp     x4, x3, [sp, #144]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #160]+        stp     x7, x8, [sp, #176]+        ldp     x5, x6, [sp, #32]+        ldp     x4, x3, [sp, #192]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #48]+        ldp     x4, x3, [sp, #208]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #32]+        stp     x7, x8, [sp, #48]+        ldp     x2, x3, [sp, #160]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [sp, #176]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x8, x3+        umulh   x8, x8, x3+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x9, x3+        umulh   x9, x9, x3+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x10, x3+        umulh   x10, x10, x3+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x11, x3+        umulh   x11, x11, x3+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        mov     x2, #0xffffffffffffffff+        csel    x2, xzr, x2, cc+        mov     x3, #0xffffffff+        csel    x3, xzr, x3, cc+        mov     x5, #0xffffffff00000001+        csel    x5, xzr, x5, cc+        subs    x8, x8, x2+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbc     x11, x11, x5+        stp     x8, x9, [sp, #96]+        stp     x10, x11, [sp, #112]+        ldp     x2, x3, [sp, #32]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [sp, #48]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x8, x3+        umulh   x8, x8, x3+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x9, x3+        umulh   x9, x9, x3+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x10, x3+        umulh   x10, x10, x3+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x11, x3+        umulh   x11, x11, x3+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        cset    x2, cs+        mov     x3, #0xffffffff+        mov     x5, #0xffffffff00000001+        adds    x12, x8, #0x1+        sbcs    x13, x9, x3+        sbcs    x14, x10, xzr+        sbcs    x7, x11, x5+        sbcs    xzr, x2, xzr+        csel    x8, x8, x12, cc+        csel    x9, x9, x13, cc+        csel    x10, x10, x14, cc+        csel    x11, x11, x7, cc+        stp     x8, x9, [sp]+        stp     x10, x11, [sp, #16]+        ldp     x3, x4, [sp, #96]+        ldp     x7, x8, [sp, #128]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #144]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #112]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #128]+        stp     x14, x0, [sp, #144]+        ldp     x3, x4, [sp, #96]+        ldp     x7, x8, [sp, #64]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #80]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #112]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #64]+        stp     x14, x0, [sp, #80]+        ldp     x5, x6, [sp]+        ldp     x4, x3, [sp, #128]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #16]+        ldp     x4, x3, [sp, #144]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp]+        stp     x7, x8, [sp, #16]+        ldp     x5, x6, [sp, #64]+        ldp     x4, x3, [sp, #128]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #80]+        ldp     x4, x3, [sp, #144]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #96]+        stp     x7, x8, [sp, #112]+        ldp     x3, x4, [sp, #160]+        ldp     x7, x8, [x16, #64]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x16, #80]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #176]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #160]+        stp     x14, x0, [sp, #176]+        ldp     x5, x6, [sp]+        ldp     x4, x3, [sp, #64]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #16]+        ldp     x4, x3, [sp, #80]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp]+        stp     x7, x8, [sp, #16]+        ldp     x5, x6, [sp, #128]+        ldp     x4, x3, [sp]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #144]+        ldp     x4, x3, [sp, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #128]+        stp     x7, x8, [sp, #144]+        ldp     x3, x4, [sp, #96]+        ldp     x7, x8, [sp, #192]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #208]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #112]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #96]+        stp     x14, x0, [sp, #112]+        ldp     x3, x4, [sp, #160]+        ldp     x7, x8, [x17, #64]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x17, #80]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #176]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #160]+        stp     x14, x0, [sp, #176]+        ldp     x3, x4, [sp, #32]+        ldp     x7, x8, [sp, #128]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #144]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #48]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #128]+        stp     x14, x0, [sp, #144]+        ldp     x5, x6, [sp, #128]+        ldp     x4, x3, [sp, #96]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #144]+        ldp     x4, x3, [sp, #112]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #128]+        stp     x7, x8, [sp, #144]+        ldp     x0, x1, [x16, #64]+        ldp     x2, x3, [x16, #80]+        orr     x12, x0, x1+        orr     x13, x2, x3+        orr     x12, x12, x13+        cmp     x12, xzr+        cset    x12, ne+        ldp     x4, x5, [x17, #64]+        ldp     x6, x7, [x17, #80]+        orr     x13, x4, x5+        orr     x14, x6, x7+        orr     x13, x13, x14+        cmp     x13, xzr+        cset    x13, ne+        cmp     x13, x12+        ldp     x8, x9, [sp, #160]+        csel    x8, x0, x8, cc+        csel    x9, x1, x9, cc+        csel    x8, x4, x8, hi+        csel    x9, x5, x9, hi+        ldp     x10, x11, [sp, #176]+        csel    x10, x2, x10, cc+        csel    x11, x3, x11, cc+        csel    x10, x6, x10, hi+        csel    x11, x7, x11, hi+        ldp     x12, x13, [x16]+        ldp     x0, x1, [sp]+        csel    x0, x12, x0, cc+        csel    x1, x13, x1, cc+        ldp     x12, x13, [x17]+        csel    x0, x12, x0, hi+        csel    x1, x13, x1, hi+        ldp     x12, x13, [x16, #16]+        ldp     x2, x3, [sp, #16]+        csel    x2, x12, x2, cc+        csel    x3, x13, x3, cc+        ldp     x12, x13, [x17, #16]+        csel    x2, x12, x2, hi+        csel    x3, x13, x3, hi+        ldp     x12, x13, [x16, #32]+        ldp     x4, x5, [sp, #128]+        csel    x4, x12, x4, cc+        csel    x5, x13, x5, cc+        ldp     x12, x13, [x17, #32]+        csel    x4, x12, x4, hi+        csel    x5, x13, x5, hi+        ldp     x12, x13, [x16, #48]+        ldp     x6, x7, [sp, #144]+        csel    x6, x12, x6, cc+        csel    x7, x13, x7, cc+        ldp     x12, x13, [x17, #48]+        csel    x6, x12, x6, hi+        csel    x7, x13, x7, hi+        stp     x0, x1, [x15]+        stp     x2, x3, [x15, #16]+        stp     x4, x5, [x15, #32]+        stp     x6, x7, [x15, #48]+        stp     x8, x9, [x15, #64]+        stp     x10, x11, [x15, #80]+        CFI_INC_SP(224)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_alt_local_p256_montjadd)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_alt_local_p256_montjdouble)++Lp256_scalarmul_alt_local_p256_montjdouble:+        CFI_START+        CFI_DEC_SP(192)+        mov     x15, x0+        mov     x16, x1+        ldp     x2, x3, [x16, #64]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [x16, #80]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        mov     x5, #0xffffffff00000001+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mul     x2, x8, x5+        umulh   x8, x8, x5+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mul     x2, x9, x5+        umulh   x9, x9, x5+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mul     x2, x10, x5+        umulh   x10, x10, x5+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mul     x2, x11, x5+        umulh   x11, x11, x5+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        cset    x2, cs+        mov     x3, #0xffffffff+        adds    x12, x8, #0x1+        sbcs    x13, x9, x3+        sbcs    x14, x10, xzr+        sbcs    x7, x11, x5+        sbcs    xzr, x2, xzr+        csel    x8, x8, x12, cc+        csel    x9, x9, x13, cc+        csel    x10, x10, x14, cc+        csel    x11, x11, x7, cc+        stp     x8, x9, [sp]+        stp     x10, x11, [sp, #16]+        ldp     x2, x3, [x16, #32]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [x16, #48]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        mov     x5, #0xffffffff00000001+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mul     x2, x8, x5+        umulh   x8, x8, x5+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mul     x2, x9, x5+        umulh   x9, x9, x5+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mul     x2, x10, x5+        umulh   x10, x10, x5+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mul     x2, x11, x5+        umulh   x11, x11, x5+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        cset    x2, cs+        mov     x3, #0xffffffff+        adds    x12, x8, #0x1+        sbcs    x13, x9, x3+        sbcs    x14, x10, xzr+        sbcs    x7, x11, x5+        sbcs    xzr, x2, xzr+        csel    x8, x8, x12, cc+        csel    x9, x9, x13, cc+        csel    x10, x10, x14, cc+        csel    x11, x11, x7, cc+        stp     x8, x9, [sp, #32]+        stp     x10, x11, [sp, #48]+        ldp     x5, x6, [x16]+        ldp     x4, x3, [sp]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [x16, #16]+        ldp     x4, x3, [sp, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        and     x4, x3, #0xffffffff+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        and     x4, x3, #0xffffffff00000001+        adc     x8, x8, x4+        stp     x5, x6, [sp, #96]+        stp     x7, x8, [sp, #112]+        ldp     x5, x6, [x16]+        ldp     x4, x3, [sp]+        adds    x5, x5, x4+        adcs    x6, x6, x3+        ldp     x7, x8, [x16, #16]+        ldp     x4, x3, [sp, #16]+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        csetm   x3, cs+        subs    x5, x5, x3+        and     x1, x3, #0xffffffff+        sbcs    x6, x6, x1+        sbcs    x7, x7, xzr+        and     x2, x3, #0xffffffff00000001+        sbc     x8, x8, x2+        stp     x5, x6, [sp, #64]+        stp     x7, x8, [sp, #80]+        ldp     x3, x4, [sp, #64]+        ldp     x7, x8, [sp, #96]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #112]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #80]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #96]+        stp     x14, x0, [sp, #112]+        ldp     x5, x6, [x16, #32]+        ldp     x4, x3, [x16, #64]+        adds    x5, x5, x4+        adcs    x6, x6, x3+        ldp     x7, x8, [x16, #48]+        ldp     x4, x3, [x16, #80]+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adc     x3, xzr, xzr+        cmn     x5, #0x1+        mov     x4, #0xffffffff+        sbcs    xzr, x6, x4+        sbcs    xzr, x7, xzr+        mov     x4, #0xffffffff00000001+        sbcs    xzr, x8, x4+        adcs    x3, x3, xzr+        csetm   x3, ne+        subs    x5, x5, x3+        and     x4, x3, #0xffffffff+        sbcs    x6, x6, x4+        sbcs    x7, x7, xzr+        and     x4, x3, #0xffffffff00000001+        sbc     x8, x8, x4+        stp     x5, x6, [sp, #64]+        stp     x7, x8, [sp, #80]+        ldp     x3, x4, [x16]+        ldp     x7, x8, [sp, #32]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #48]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [x16, #16]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #128]+        stp     x14, x0, [sp, #144]+        ldp     x2, x3, [sp, #96]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [sp, #112]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        mov     x5, #0xffffffff00000001+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mul     x2, x8, x5+        umulh   x8, x8, x5+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mul     x2, x9, x5+        umulh   x9, x9, x5+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mul     x2, x10, x5+        umulh   x10, x10, x5+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mul     x2, x11, x5+        umulh   x11, x11, x5+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        cset    x2, cs+        mov     x3, #0xffffffff+        adds    x12, x8, #0x1+        sbcs    x13, x9, x3+        sbcs    x14, x10, xzr+        sbcs    x7, x11, x5+        sbcs    xzr, x2, xzr+        csel    x8, x8, x12, cc+        csel    x9, x9, x13, cc+        csel    x10, x10, x14, cc+        csel    x11, x11, x7, cc+        stp     x8, x9, [sp, #160]+        stp     x10, x11, [sp, #176]+        ldp     x2, x3, [sp, #64]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [sp, #80]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        mov     x5, #0xffffffff00000001+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mul     x2, x8, x5+        umulh   x8, x8, x5+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mul     x2, x9, x5+        umulh   x9, x9, x5+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mul     x2, x10, x5+        umulh   x10, x10, x5+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mul     x2, x11, x5+        umulh   x11, x11, x5+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        cset    x2, cs+        mov     x3, #0xffffffff+        adds    x12, x8, #0x1+        sbcs    x13, x9, x3+        sbcs    x14, x10, xzr+        sbcs    x7, x11, x5+        sbcs    xzr, x2, xzr+        csel    x8, x8, x12, cc+        csel    x9, x9, x13, cc+        csel    x10, x10, x14, cc+        csel    x11, x11, x7, cc+        stp     x8, x9, [sp, #64]+        stp     x10, x11, [sp, #80]+        mov     x1, #0x9+        mov     x2, #0xffffffffffffffff+        ldp     x9, x10, [sp, #160]+        subs    x9, x2, x9+        mov     x2, #0xffffffff+        sbcs    x10, x2, x10+        ldp     x11, x12, [sp, #176]+        ngcs    x11, x11+        mov     x2, #0xffffffff00000001+        sbc     x12, x2, x12+        mul     x3, x1, x9+        mul     x4, x1, x10+        mul     x5, x1, x11+        mul     x6, x1, x12+        umulh   x9, x1, x9+        umulh   x10, x1, x10+        umulh   x11, x1, x11+        umulh   x7, x1, x12+        adds    x4, x4, x9+        adcs    x5, x5, x10+        adcs    x6, x6, x11+        adc     x7, x7, xzr+        mov     x1, #0xc+        ldp     x9, x10, [sp, #128]+        mul     x8, x9, x1+        umulh   x9, x9, x1+        adds    x3, x3, x8+        mul     x8, x10, x1+        umulh   x10, x10, x1+        adcs    x4, x4, x8+        ldp     x11, x12, [sp, #144]+        mul     x8, x11, x1+        umulh   x11, x11, x1+        adcs    x5, x5, x8+        mul     x8, x12, x1+        umulh   x12, x12, x1+        adcs    x6, x6, x8+        adc     x7, x7, xzr+        adds    x4, x4, x9+        adcs    x5, x5, x10+        adcs    x6, x6, x11+        adc     x7, x7, x12+        add     x8, x7, #0x1+        lsl     x10, x8, #32+        adds    x6, x6, x10+        adc     x7, x7, xzr+        neg     x9, x8+        sub     x10, x10, #0x1+        subs    x3, x3, x9+        sbcs    x4, x4, x10+        sbcs    x5, x5, xzr+        sbcs    x6, x6, x8+        sbc     x8, x7, x8+        adds    x3, x3, x8+        and     x9, x8, #0xffffffff+        adcs    x4, x4, x9+        adcs    x5, x5, xzr+        neg     x10, x9+        adc     x6, x6, x10+        stp     x3, x4, [sp, #160]+        stp     x5, x6, [sp, #176]+        ldp     x5, x6, [sp, #64]+        ldp     x4, x3, [sp]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #80]+        ldp     x4, x3, [sp, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        and     x4, x3, #0xffffffff+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        and     x4, x3, #0xffffffff00000001+        adc     x8, x8, x4+        stp     x5, x6, [sp, #64]+        stp     x7, x8, [sp, #80]+        ldp     x2, x3, [sp, #32]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [sp, #48]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        mov     x5, #0xffffffff00000001+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mul     x2, x8, x5+        umulh   x8, x8, x5+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mul     x2, x9, x5+        umulh   x9, x9, x5+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mul     x2, x10, x5+        umulh   x10, x10, x5+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mul     x2, x11, x5+        umulh   x11, x11, x5+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        cset    x2, cs+        mov     x3, #0xffffffff+        adds    x12, x8, #0x1+        sbcs    x13, x9, x3+        sbcs    x14, x10, xzr+        sbcs    x7, x11, x5+        sbcs    xzr, x2, xzr+        csel    x8, x8, x12, cc+        csel    x9, x9, x13, cc+        csel    x10, x10, x14, cc+        csel    x11, x11, x7, cc+        stp     x8, x9, [sp]+        stp     x10, x11, [sp, #16]+        ldp     x3, x4, [sp, #160]+        ldp     x7, x8, [sp, #96]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #112]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #176]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #96]+        stp     x14, x0, [sp, #112]+        ldp     x5, x6, [sp, #64]+        ldp     x4, x3, [sp, #32]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #80]+        ldp     x4, x3, [sp, #48]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        and     x4, x3, #0xffffffff+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        and     x4, x3, #0xffffffff00000001+        adc     x8, x8, x4+        stp     x5, x6, [x15, #64]+        stp     x7, x8, [x15, #80]+        ldp     x1, x2, [sp, #128]+        lsl     x0, x1, #2+        ldp     x6, x7, [sp, #160]+        subs    x0, x0, x6+        extr    x1, x2, x1, #62+        sbcs    x1, x1, x7+        ldp     x3, x4, [sp, #144]+        extr    x2, x3, x2, #62+        ldp     x6, x7, [sp, #176]+        sbcs    x2, x2, x6+        extr    x3, x4, x3, #62+        sbcs    x3, x3, x7+        lsr     x4, x4, #62+        sbc     x4, x4, xzr+        add     x5, x4, #0x1+        lsl     x8, x5, #32+        negs    x6, x8+        ngcs    x7, xzr+        sbc     x8, x8, x5+        adds    x0, x0, x5+        adcs    x1, x1, x6+        adcs    x2, x2, x7+        adcs    x3, x3, x8+        csetm   x5, cc+        adds    x0, x0, x5+        and     x6, x5, #0xffffffff+        adcs    x1, x1, x6+        adcs    x2, x2, xzr+        neg     x7, x6+        adc     x3, x3, x7+        stp     x0, x1, [x15]+        stp     x2, x3, [x15, #16]+        mov     x1, #0x8+        mov     x2, #0xffffffffffffffff+        ldp     x9, x10, [sp]+        subs    x9, x2, x9+        mov     x2, #0xffffffff+        sbcs    x10, x2, x10+        ldp     x11, x12, [sp, #16]+        ngcs    x11, x11+        mov     x2, #0xffffffff00000001+        sbc     x12, x2, x12+        lsl     x3, x9, #3+        extr    x4, x10, x9, #61+        extr    x5, x11, x10, #61+        extr    x6, x12, x11, #61+        lsr     x7, x12, #61+        mov     x1, #0x3+        ldp     x9, x10, [sp, #96]+        mul     x8, x9, x1+        umulh   x9, x9, x1+        adds    x3, x3, x8+        mul     x8, x10, x1+        umulh   x10, x10, x1+        adcs    x4, x4, x8+        ldp     x11, x12, [sp, #112]+        mul     x8, x11, x1+        umulh   x11, x11, x1+        adcs    x5, x5, x8+        mul     x8, x12, x1+        umulh   x12, x12, x1+        adcs    x6, x6, x8+        adc     x7, x7, xzr+        adds    x4, x4, x9+        adcs    x5, x5, x10+        adcs    x6, x6, x11+        adc     x7, x7, x12+        add     x8, x7, #0x1+        lsl     x10, x8, #32+        adds    x6, x6, x10+        adc     x7, x7, xzr+        neg     x9, x8+        sub     x10, x10, #0x1+        subs    x3, x3, x9+        sbcs    x4, x4, x10+        sbcs    x5, x5, xzr+        sbcs    x6, x6, x8+        sbc     x8, x7, x8+        adds    x3, x3, x8+        and     x9, x8, #0xffffffff+        adcs    x4, x4, x9+        adcs    x5, x5, xzr+        neg     x10, x9+        adc     x6, x6, x10+        stp     x3, x4, [x15, #32]+        stp     x5, x6, [x15, #48]+        CFI_INC_SP(192)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_alt_local_p256_montjdouble)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_alt_local_p256_montjmixadd)++Lp256_scalarmul_alt_local_p256_montjmixadd:+        CFI_START+        CFI_DEC_SP(192)+        mov     x15, x0+        mov     x16, x1+        mov     x17, x2+        ldp     x2, x3, [x16, #64]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [x16, #80]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x8, x3+        umulh   x8, x8, x3+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x9, x3+        umulh   x9, x9, x3+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x10, x3+        umulh   x10, x10, x3+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x11, x3+        umulh   x11, x11, x3+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        mov     x2, #0xffffffffffffffff+        csel    x2, xzr, x2, cc+        mov     x3, #0xffffffff+        csel    x3, xzr, x3, cc+        mov     x5, #0xffffffff00000001+        csel    x5, xzr, x5, cc+        subs    x8, x8, x2+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbc     x11, x11, x5+        stp     x8, x9, [sp]+        stp     x10, x11, [sp, #16]+        ldp     x3, x4, [x16, #64]+        ldp     x7, x8, [x17, #32]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x17, #48]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [x16, #80]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #32]+        stp     x14, x0, [sp, #48]+        ldp     x3, x4, [sp]+        ldp     x7, x8, [x17]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x17, #16]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #16]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #64]+        stp     x14, x0, [sp, #80]+        ldp     x3, x4, [sp]+        ldp     x7, x8, [sp, #32]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #48]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #16]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #32]+        stp     x14, x0, [sp, #48]+        ldp     x5, x6, [sp, #64]+        ldp     x4, x3, [x16]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #80]+        ldp     x4, x3, [x16, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #160]+        stp     x7, x8, [sp, #176]+        ldp     x5, x6, [sp, #32]+        ldp     x4, x3, [x16, #32]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #48]+        ldp     x4, x3, [x16, #48]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #32]+        stp     x7, x8, [sp, #48]+        ldp     x2, x3, [sp, #160]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [sp, #176]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x8, x3+        umulh   x8, x8, x3+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x9, x3+        umulh   x9, x9, x3+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x10, x3+        umulh   x10, x10, x3+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x11, x3+        umulh   x11, x11, x3+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        mov     x2, #0xffffffffffffffff+        csel    x2, xzr, x2, cc+        mov     x3, #0xffffffff+        csel    x3, xzr, x3, cc+        mov     x5, #0xffffffff00000001+        csel    x5, xzr, x5, cc+        subs    x8, x8, x2+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbc     x11, x11, x5+        stp     x8, x9, [sp, #96]+        stp     x10, x11, [sp, #112]+        ldp     x2, x3, [sp, #32]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [sp, #48]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x8, x3+        umulh   x8, x8, x3+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x9, x3+        umulh   x9, x9, x3+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x10, x3+        umulh   x10, x10, x3+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x11, x3+        umulh   x11, x11, x3+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        cset    x2, cs+        mov     x3, #0xffffffff+        mov     x5, #0xffffffff00000001+        adds    x12, x8, #0x1+        sbcs    x13, x9, x3+        sbcs    x14, x10, xzr+        sbcs    x7, x11, x5+        sbcs    xzr, x2, xzr+        csel    x8, x8, x12, cc+        csel    x9, x9, x13, cc+        csel    x10, x10, x14, cc+        csel    x11, x11, x7, cc+        stp     x8, x9, [sp]+        stp     x10, x11, [sp, #16]+        ldp     x3, x4, [sp, #96]+        ldp     x7, x8, [x16]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x16, #16]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #112]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #128]+        stp     x14, x0, [sp, #144]+        ldp     x3, x4, [sp, #96]+        ldp     x7, x8, [sp, #64]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #80]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #112]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #64]+        stp     x14, x0, [sp, #80]+        ldp     x5, x6, [sp]+        ldp     x4, x3, [sp, #128]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #16]+        ldp     x4, x3, [sp, #144]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp]+        stp     x7, x8, [sp, #16]+        ldp     x5, x6, [sp, #64]+        ldp     x4, x3, [sp, #128]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #80]+        ldp     x4, x3, [sp, #144]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #96]+        stp     x7, x8, [sp, #112]+        ldp     x3, x4, [sp, #160]+        ldp     x7, x8, [x16, #64]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x16, #80]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #176]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #160]+        stp     x14, x0, [sp, #176]+        ldp     x5, x6, [sp]+        ldp     x4, x3, [sp, #64]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #16]+        ldp     x4, x3, [sp, #80]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp]+        stp     x7, x8, [sp, #16]+        ldp     x5, x6, [sp, #128]+        ldp     x4, x3, [sp]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #144]+        ldp     x4, x3, [sp, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #128]+        stp     x7, x8, [sp, #144]+        ldp     x3, x4, [sp, #96]+        ldp     x7, x8, [x16, #32]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x16, #48]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #112]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #96]+        stp     x14, x0, [sp, #112]+        ldp     x3, x4, [sp, #32]+        ldp     x7, x8, [sp, #128]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #144]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #48]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #128]+        stp     x14, x0, [sp, #144]+        ldp     x5, x6, [sp, #128]+        ldp     x4, x3, [sp, #96]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #144]+        ldp     x4, x3, [sp, #112]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #128]+        stp     x7, x8, [sp, #144]+        ldp     x0, x1, [x16, #64]+        ldp     x2, x3, [x16, #80]+        orr     x4, x0, x1+        orr     x5, x2, x3+        orr     x4, x4, x5+        cmp     x4, xzr+        ldp     x0, x1, [sp]+        ldp     x12, x13, [x17]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x2, x3, [sp, #16]+        ldp     x12, x13, [x17, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x4, x5, [sp, #128]+        ldp     x12, x13, [x17, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x6, x7, [sp, #144]+        ldp     x12, x13, [x17, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x8, x9, [sp, #160]+        mov     x12, #0x1+        mov     x13, #0xffffffff00000000+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x10, x11, [sp, #176]+        mov     x12, #0xffffffffffffffff+        mov     x13, #0xfffffffe+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        stp     x0, x1, [x15]+        stp     x2, x3, [x15, #16]+        stp     x4, x5, [x15, #32]+        stp     x6, x7, [x15, #48]+        stp     x8, x9, [x15, #64]+        stp     x10, x11, [x15, #80]+        CFI_INC_SP(192)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_alt_local_p256_montjmixadd)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/p256_scalarmulbase.S view
@@ -0,0 +1,3782 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Scalar multiplication for precomputed point on NIST curve P-256+// Input scalar[4], blocksize, table[]; output res[8]+//+// extern void p256_scalarmulbase+//   (uint64_t res[static 8],+//    const uint64_t scalar[static 4],+//    uint64_t blocksize,+//    const uint64_t *table);+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-256, the input argument "table" is expected to be a table of+// multiples of the point P in Montgomery-affine form, with each block+// corresponding to "blocksize" bits of the scalar as follows, where+// B = 2^{blocksize-1} (e.g. B = 8 for blocksize = 4):+//+// For each i,j with blocksize * i <= 256 and 1 <= j <= B+// the multiple 2^{blocksize * i} * j * P is stored at+// tab[8 * (B * i + (j - 1))], considered as uint64_t pointers+// or tab + 64 * (B * i + (j - 1)) as byte pointers.+//+// Standard ARM ABI: X0 = res, X1 = scalar, X2 = blocksize, X3 = table+// ----------------------------------------------------------------------------+++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_scalarmulbase)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_scalarmulbase)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_scalarmulbase)+++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Safe copies of inputs and additional variables, with some aliasing++#define res x19+#define blocksize x20+#define table x21+#define i x22+#define bf x23+#define cf x24+#define j x25++// Intermediate variables on the stack. The last z2, z3 values can+// safely be overlaid on "nacc", which is no longer needed at the end.+// Uppercase syntactic variants make x86_att version simpler to generate++#define rscalar sp, #(0*NUMSIZE)+#define acc sp, #(1*NUMSIZE)+#define nacc sp, #(4*NUMSIZE)+#define tabent sp, #(7*NUMSIZE)++#define z2 sp, #(4*NUMSIZE)+#define z3 sp, #(5*NUMSIZE)++#define NSPACE 9*NUMSIZE++// Loading large constants++#define movbig(nn,n3,n2,n1,n0)                                      \+        movz    nn, n0 __LF                                            \+        movk    nn, n1, lsl #16 __LF                                   \+        movk    nn, n2, lsl #32 __LF                                   \+        movk    nn, n3, lsl #48++S2N_BN_SYMBOL(p256_scalarmulbase):+        CFI_START++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x30)+        CFI_DEC_SP(NSPACE)++// Preserve the input arguments except the scalar, since that gets absorbed+// immediately. The "table" value subsequently gets shifted up each iteration+// of the loop, while "res" and "blocksize" are static throughout.++        mov     res, x0+        mov     blocksize, x2+        mov     table, x3++// Load the digits of group order n_256 = [x15;x14;x13;x12]++        movbig(x12, #0xf3b9, #0xcac2, #0xfc63, #0x2551)+        movbig(x13, #0xbce6, #0xfaad, #0xa717, #0x9e84)+        mov     x14, #0xffffffffffffffff+        mov     x15, #0xffffffff00000000++// First, reduce the input scalar mod n_256, i.e. conditionally subtract n_256+// Store it to "rscalar" (reduced scalar)++        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]++        subs    x6, x2, x12+        sbcs    x7, x3, x13+        sbcs    x8, x4, x14+        sbcs    x9, x5, x15++        csel    x2, x2, x6, cc+        csel    x3, x3, x7, cc+        csel    x4, x4, x8, cc+        csel    x5, x5, x9, cc++        stp     x2, x3, [rscalar]+        stp     x4, x5, [rscalar+16]++// Initialize the accumulator to all zeros and the "carry flag" cf to 0++        stp     xzr, xzr, [acc]+        stp     xzr, xzr, [acc+16]+        stp     xzr, xzr, [acc+32]+        stp     xzr, xzr, [acc+48]+        stp     xzr, xzr, [acc+64]+        stp     xzr, xzr, [acc+80]+        mov     cf, xzr++// Main loop over {i >= 0 | blocksize * i <= 256}. Note the non-strict+// inequality, to allow top carry for any choices of blocksize.++        mov     i, xzr++Lp256_scalarmulbase_loop:++// The next raw bitfield is bf = bitfield(blocksize * i,blocksize) + cf,+// adding in the deferred carry cf. We then shift the whole scalar right+// by blocksize so we can keep picking bitfield(0,blocksize).++        ldp     x0, x1, [rscalar]+        ldp     x2, x3, [rscalar+16]++        mov     x4, #1+        lsl     x4, x4, blocksize+        sub     x4, x4, #1+        and     x4, x4, x0+        add     bf, x4, cf++        neg     x8, blocksize++        lsl     x5, x1, x8++        lsr     x0, x0, blocksize+        orr     x0, x0,  x5++        lsl     x6, x2, x8+        lsr     x1, x1, blocksize+        orr     x1, x1, x6++        lsl     x7, x3, x8+        lsr     x2, x2, blocksize+        orr     x2, x2, x7++        lsr     x3, x3, blocksize++        stp     x0, x1, [rscalar]+        stp     x2, x3, [rscalar+16]++// Now if bf <= B we just select entry j, unnegated and set cf = 0.+// If bf > B we set j = 2 * B - bf and negate the j'th entry, setting cf = 1.+// In either case we ultimately add bf, in the latter case with deferred+// carry as 2 * B - (2 * B - bf) = bf.++        mov     x0, #1+        lsl     x1, x0, blocksize+        lsr     x0, x1, #1++        sub     x2, x1, bf++        cmp     x0, bf+        cset    cf, cc+        csel    j, x2, bf, cc++// Load table entry j - 1 for nonzero j in constant-time style.++        mov     x16, #1+        lsl     x16, x16, blocksize+        lsr     x16, x16, #1+        mov     x17, j++Lp256_scalarmulbase_tabloop:+        ldp     x8, x9, [table]+        ldp     x10, x11, [table, #16]+        ldp     x12, x13, [table, #32]+        ldp     x14, x15, [table, #48]++        subs    x17, x17, #1+        csel    x0, x8, x0, eq+        csel    x1, x9, x1, eq+        csel    x2, x10, x2, eq+        csel    x3, x11, x3, eq+        csel    x4, x12, x4, eq+        csel    x5, x13, x5, eq+        csel    x6, x14, x6, eq+        csel    x7, x15, x7, eq++        add     table, table, #64++        sub     x16, x16, #1+        cbnz    x16, Lp256_scalarmulbase_tabloop++// Before storing back, optionally negate the y coordinate of the table entry++        stp     x0, x1, [tabent]+        stp     x2, x3, [tabent+16]++        mov     x0, 0xffffffffffffffff+        subs    x0, x0, x4+        mov     x1, 0x00000000ffffffff+        sbcs    x1, x1, x5+        mov     x3, 0xffffffff00000001+        sbcs    x2, xzr, x6+        sbc     x3, x3, x7++        cmp     cf, xzr+        csel    x4, x0, x4, ne+        csel    x5, x1, x5, ne+        csel    x6, x2, x6, ne+        csel    x7, x3, x7, ne++        stp     x4, x5, [tabent+32]+        stp     x6, x7, [tabent+48]++// Add the adjusted table point to the accumulator++        add     x0, nacc+        add     x1, acc+        add     x2, tabent+        CFI_BL(Lp256_scalarmulbase_local_p256_montjmixadd)++// However, only commit that update to the accumulator if j is nonzero,+// because the mixed addition function does not handle this case directly,+// and in any case we didn't choose the table entry appropriately.++        cmp     j, xzr+        ldp     x0, x1, [acc]+        ldp     x12, x13, [nacc]+        csel    x0, x12, x0, ne+        csel    x1, x13, x1, ne++        ldp     x2, x3, [acc+16]+        ldp     x12, x13, [nacc+16]+        csel    x2, x12, x2, ne+        csel    x3, x13, x3, ne++        ldp     x4, x5, [acc+32]+        ldp     x12, x13, [nacc+32]+        csel    x4, x12, x4, ne+        csel    x5, x13, x5, ne++        ldp     x6, x7, [acc+48]+        ldp     x12, x13, [nacc+48]+        csel    x6, x12, x6, ne+        csel    x7, x13, x7, ne++        ldp     x8, x9, [acc+64]+        ldp     x12, x13, [nacc+64]+        csel    x8, x12, x8, ne+        csel    x9, x13, x9, ne++        ldp     x10, x11, [acc+80]+        ldp     x12, x13, [nacc+80]+        csel    x10, x12, x10, ne+        csel    x11, x13, x11, ne++        stp     x0, x1, [acc]+        stp     x2, x3, [acc+16]+        stp     x4, x5, [acc+32]+        stp     x6, x7, [acc+48]+        stp     x8, x9, [acc+64]+        stp     x10, x11, [acc+80]++// Loop while blocksize * i <= 256++        add     i, i, #1+        mul     x0, blocksize, i+        cmp     x0, #257+        bcc     Lp256_scalarmulbase_loop++// That's the end of the main loop, and we just need to translate+// back from the Jacobian representation to affine. First of all,+// let z2 = 1/z^2 and z3 = 1/z^3, both without Montgomery form++        add     x0, z2+        add     x1, acc+64+        CFI_BL(Lp256_scalarmulbase_local_montsqr_p256)++        add     x0, z3+        add     x1, acc+64+        add     x2, z2+        CFI_BL(Lp256_scalarmulbase_local_montmul_p256)++        add     x0, z2+        add     x1, z3+        CFI_BL(Lp256_scalarmulbase_local_demont_p256)++        add     x0, z3+        add     x1, z2+        CFI_BL(Lp256_scalarmulbase_local_inv_p256)++        add     x0, z2+        add     x1, acc+64+        add     x2, z3+        CFI_BL(Lp256_scalarmulbase_local_montmul_p256)++// Convert back from Jacobian (X,Y,Z) |-> (X/Z^2, Y/Z^3)++        mov     x0, res+        add     x1, acc+        add     x2, z2+        CFI_BL(Lp256_scalarmulbase_local_montmul_p256)++        add     x0, res, #32+        add     x1, acc+32+        add     x2, z3+        CFI_BL(Lp256_scalarmulbase_local_montmul_p256)++// Restore stack and registers and return++        CFI_INC_SP(NSPACE)+        CFI_POP2(x25,x30)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_scalarmulbase)++// Local copies of subroutines, complete clones at the moment++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_local_demont_p256)++Lp256_scalarmulbase_local_demont_p256:+        CFI_START+        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]+        lsl     x7, x2, #32+        subs    x8, x2, x7+        lsr     x6, x2, #32+        sbc     x2, x2, x6+        adds    x3, x3, x7+        adcs    x4, x4, x6+        adcs    x5, x5, x8+        adc     x2, x2, xzr+        lsl     x7, x3, #32+        subs    x8, x3, x7+        lsr     x6, x3, #32+        sbc     x3, x3, x6+        adds    x4, x4, x7+        adcs    x5, x5, x6+        adcs    x2, x2, x8+        adc     x3, x3, xzr+        lsl     x7, x4, #32+        subs    x8, x4, x7+        lsr     x6, x4, #32+        sbc     x4, x4, x6+        adds    x5, x5, x7+        adcs    x2, x2, x6+        adcs    x3, x3, x8+        adc     x4, x4, xzr+        lsl     x7, x5, #32+        subs    x8, x5, x7+        lsr     x6, x5, #32+        sbc     x5, x5, x6+        adds    x2, x2, x7+        adcs    x3, x3, x6+        adcs    x4, x4, x8+        adc     x5, x5, xzr+        stp     x2, x3, [x0]+        stp     x4, x5, [x0, #16]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_local_demont_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_local_inv_p256)++Lp256_scalarmulbase_local_inv_p256:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_DEC_SP(160)+        mov     x20, x0+        mov     x10, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x13, #0xffffffff00000001+        stp     x10, x11, [sp]+        stp     xzr, x13, [sp, #16]+        str     xzr, [sp, #32]+        ldp     x2, x3, [x1]+        subs    x10, x2, x10+        sbcs    x11, x3, x11+        ldp     x4, x5, [x1, #16]+        sbcs    x12, x4, xzr+        sbcs    x13, x5, x13+        csel    x2, x2, x10, cc+        csel    x3, x3, x11, cc+        csel    x4, x4, x12, cc+        csel    x5, x5, x13, cc+        stp     x2, x3, [sp, #48]+        stp     x4, x5, [sp, #64]+        str     xzr, [sp, #80]+        stp     xzr, xzr, [sp, #96]+        stp     xzr, xzr, [sp, #112]+        mov     x10, #0x4000000000000+        stp     x10, xzr, [sp, #128]+        stp     xzr, xzr, [sp, #144]+        mov     x21, #0xa+        mov     x22, #0x1+        b       Lp256_scalarmulbase_inv_midloop+Lp256_scalarmulbase_inv_loop:+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        and     x0, x12, x16+        and     x1, x13, x17+        add     x19, x0, x1+        ldr     x7, [sp]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #48]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x7, [sp, #8]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #56]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [sp]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [sp, #48]+        ldr     x7, [sp, #16]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #64]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [sp, #8]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [sp, #56]+        ldr     x7, [sp, #24]+        eor     x1, x7, x14+        ldr     x23, [sp, #32]+        eor     x3, x23, x14+        and     x3, x3, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #72]+        eor     x1, x8, x15+        ldr     x24, [sp, #80]+        eor     x0, x24, x15+        and     x0, x0, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x5, x6, #59+        str     x6, [sp, #16]+        extr    x5, x3, x5, #59+        str     x5, [sp, #24]+        asr     x3, x3, #59+        str     x3, [sp, #32]+        eor     x1, x7, x16+        eor     x5, x23, x16+        and     x5, x5, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        eor     x0, x24, x17+        and     x0, x0, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x4, x2, x4, #59+        str     x4, [sp, #64]+        extr    x2, x5, x2, #59+        str     x2, [sp, #72]+        asr     x5, x5, #59+        str     x5, [sp, #80]+        ldr     x7, [sp, #96]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #128]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #96]+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        str     x5, [sp, #128]+        adc     x3, x3, x1+        ldr     x7, [sp, #104]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #136]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #104]+        adc     x6, x6, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        str     x3, [sp, #136]+        adc     x4, x4, x1+        ldr     x7, [sp, #112]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #144]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #112]+        adc     x5, x5, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        str     x4, [sp, #144]+        adc     x2, x2, x1+        ldr     x7, [sp, #120]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #152]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldp     x0, x1, [sp, #96]+        ldr     x6, [sp, #112]+        mov     x14, #0xe000000000000000+        adds    x0, x0, x14+        sbcs    x1, x1, xzr+        mov     x11, #0x1fffffff+        adcs    x6, x6, x11+        mov     x10, #0x2000000000000000+        adcs    x5, x5, x10+        mov     x14, #0x1fffffffe0000000+        adc     x3, x3, x14+        lsl     x11, x0, #32+        subs    x14, x0, x11+        lsr     x10, x0, #32+        sbc     x0, x0, x10+        adds    x1, x1, x11+        adcs    x6, x6, x10+        adcs    x5, x5, x14+        adcs    x3, x3, x0+        mov     x14, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x10, #0xffffffff00000001+        csel    x14, x14, xzr, cs+        csel    x11, x11, xzr, cs+        csel    x10, x10, xzr, cs+        subs    x1, x1, x14+        sbcs    x6, x6, x11+        sbcs    x5, x5, xzr+        sbc     x3, x3, x10+        stp     x1, x6, [sp, #96]+        stp     x5, x3, [sp, #112]+        eor     x1, x7, x16+        and     x5, x16, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        and     x0, x17, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        ldp     x0, x1, [sp, #128]+        ldr     x3, [sp, #144]+        mov     x14, #0xe000000000000000+        adds    x0, x0, x14+        sbcs    x1, x1, xzr+        mov     x11, #0x1fffffff+        adcs    x3, x3, x11+        mov     x10, #0x2000000000000000+        adcs    x2, x2, x10+        mov     x14, #0x1fffffffe0000000+        adc     x5, x5, x14+        lsl     x11, x0, #32+        subs    x14, x0, x11+        lsr     x10, x0, #32+        sbc     x0, x0, x10+        adds    x1, x1, x11+        adcs    x3, x3, x10+        adcs    x2, x2, x14+        adcs    x5, x5, x0+        mov     x14, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x10, #0xffffffff00000001+        csel    x14, x14, xzr, cs+        csel    x11, x11, xzr, cs+        csel    x10, x10, xzr, cs+        subs    x1, x1, x14+        sbcs    x3, x3, x11+        sbcs    x2, x2, xzr+        sbc     x5, x5, x10+        stp     x1, x3, [sp, #128]+        stp     x2, x5, [sp, #144]+Lp256_scalarmulbase_inv_midloop:+        mov     x1, x22+        ldr     x2, [sp]+        ldr     x3, [sp, #48]+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x8, x4, #0x100, lsl #12+        sbfx    x8, x8, #21, #21+        mov     x11, #0x100000+        add     x11, x11, x11, lsl #21+        add     x9, x4, x11+        asr     x9, x9, #42+        add     x10, x5, #0x100, lsl #12+        sbfx    x10, x10, #21, #21+        add     x11, x5, x11+        asr     x11, x11, #42+        mul     x6, x8, x2+        mul     x7, x9, x3+        mul     x2, x10, x2+        mul     x3, x11, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #21, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #42+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #21, #21+        add     x15, x5, x15+        asr     x15, x15, #42+        mul     x6, x12, x2+        mul     x7, x13, x3+        mul     x2, x14, x2+        mul     x3, x15, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        mul     x2, x12, x8+        mul     x3, x12, x9+        mul     x6, x14, x8+        mul     x7, x14, x9+        madd    x8, x13, x10, x2+        madd    x9, x13, x11, x3+        madd    x16, x15, x10, x6+        madd    x17, x15, x11, x7+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #22, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #43+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #22, #21+        add     x15, x5, x15+        asr     x15, x15, #43+        mneg    x2, x12, x8+        mneg    x3, x12, x9+        mneg    x4, x14, x8+        mneg    x5, x14, x9+        msub    x10, x13, x16, x2+        msub    x11, x13, x17, x3+        msub    x12, x15, x16, x4+        msub    x13, x15, x17, x5+        mov     x22, x1+        subs    x21, x21, #0x1+        bne     Lp256_scalarmulbase_inv_loop+        ldr     x0, [sp]+        ldr     x1, [sp, #48]+        mul     x0, x0, x10+        madd    x1, x1, x11, x0+        asr     x0, x1, #63+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        eor     x14, x14, x0+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        eor     x15, x15, x0+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        eor     x16, x16, x0+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        eor     x17, x17, x0+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        ldr     x7, [sp, #96]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #128]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #96]+        adc     x2, x2, x1+        ldr     x7, [sp, #104]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #136]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #104]+        adc     x6, x6, x1+        ldr     x7, [sp, #112]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #144]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #112]+        adc     x5, x5, x1+        ldr     x7, [sp, #120]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #152]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldp     x0, x1, [sp, #96]+        ldr     x2, [sp, #112]+        mov     x14, #0xe000000000000000+        adds    x0, x0, x14+        sbcs    x1, x1, xzr+        mov     x11, #0x1fffffff+        adcs    x2, x2, x11+        mov     x10, #0x2000000000000000+        adcs    x5, x5, x10+        mov     x14, #0x1fffffffe0000000+        adc     x3, x3, x14+        lsl     x11, x0, #32+        subs    x14, x0, x11+        lsr     x10, x0, #32+        sbc     x0, x0, x10+        adds    x1, x1, x11+        adcs    x2, x2, x10+        adcs    x5, x5, x14+        adcs    x3, x3, x0+        mov     x14, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x10, #0xffffffff00000001+        csel    x14, x14, xzr, cs+        csel    x11, x11, xzr, cs+        csel    x10, x10, xzr, cs+        subs    x1, x1, x14+        sbcs    x2, x2, x11+        sbcs    x5, x5, xzr+        sbc     x3, x3, x10+        mov     x10, #0xffffffffffffffff+        subs    x10, x1, x10+        mov     x11, #0xffffffff+        sbcs    x11, x2, x11+        mov     x13, #0xffffffff00000001+        sbcs    x12, x5, xzr+        sbcs    x13, x3, x13+        csel    x10, x1, x10, cc+        csel    x11, x2, x11, cc+        csel    x12, x5, x12, cc+        csel    x13, x3, x13, cc+        stp     x10, x11, [x20]+        stp     x12, x13, [x20, #16]+        CFI_INC_SP(160)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_local_inv_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_local_montmul_p256)++Lp256_scalarmulbase_local_montmul_p256:+        CFI_START+        ldr q20, [x2]+        ldp x7, x17, [x1]+        ldr q0, [x1]+        ldp x6, x10, [x2]+        ldp x11, x15, [x1, #16]+        rev64 v16.4S, v20.4S+        subs x4, x7, x17+        csetm x3, cc+        cneg x13, x4, cc+        mul v16.4S, v16.4S, v0.4S+        umulh x12, x17, x10+        uzp1 v28.4S, v20.4S, v0.4S+        subs x14, x11, x7+        ldr q20, [x2, #16]+        sbcs x5, x15, x17+        ngc x17, xzr+        subs x8, x11, x15+        uaddlp v27.2D, v16.4S+        umulh x4, x7, x6+        uzp1 v21.4S, v0.4S, v0.4S+        cneg x11, x8, cc+        shl v17.2D, v27.2D, #32+        csetm x15, cc+        subs x9, x10, x6+        eor x7, x14, x17+        umlal v17.2D, v21.2S, v28.2S+        cneg x8, x9, cc+        cinv x9, x3, cc+        cmn x17, #0x1+        ldr q28, [x1, #16]+        adcs x14, x7, xzr+        mul x7, x13, x8+        eor x1, x5, x17+        adcs x5, x1, xzr+        xtn v1.2S, v20.2D+        mov x1, v17.d[0]+        mov x3, v17.d[1]+        uzp2 v16.4S, v20.4S, v20.4S+        umulh x16, x13, x8+        eor x13, x7, x9+        adds x8, x1, x3+        adcs x7, x4, x12+        xtn v0.2S, v28.2D+        adcs x12, x12, xzr+        adds x8, x4, x8+        adcs x3, x3, x7+        ldp x7, x2, [x2, #16]+        adcs x12, x12, xzr+        cmn x9, #0x1+        adcs x8, x8, x13+        eor x13, x16, x9+        adcs x16, x3, x13+        lsl x3, x1, #32+        adc x13, x12, x9+        subs x12, x6, x7+        sbcs x9, x10, x2+        lsr x10, x1, #32+        ngc x4, xzr+        subs x6, x2, x7+        cinv x2, x15, cc+        cneg x6, x6, cc+        subs x7, x1, x3+        eor x9, x9, x4+        sbc x1, x1, x10+        adds x15, x8, x3+        adcs x3, x16, x10+        mul x16, x11, x6+        adcs x8, x13, x7+        eor x13, x12, x4+        adc x10, x1, xzr+        cmn x4, #0x1+        umulh x6, x11, x6+        adcs x11, x13, xzr+        adcs x1, x9, xzr+        lsl x13, x15, #32+        subs x12, x15, x13+        lsr x7, x15, #32+        sbc x15, x15, x7+        adds x9, x3, x13+        adcs x3, x8, x7+        umulh x8, x14, x11+        umull v21.2D, v0.2S, v1.2S+        adcs x12, x10, x12+        umull v3.2D, v0.2S, v16.2S+        adc x15, x15, xzr+        rev64 v24.4S, v20.4S+        stp x12, x15, [x0, #16]+        movi v2.2D, #0x00000000ffffffff+        mul x10, x14, x11+        mul v4.4S, v24.4S, v28.4S+        subs x13, x14, x5+        uzp2 v19.4S, v28.4S, v28.4S+        csetm x15, cc+        usra v3.2D, v21.2D, #32+        mul x7, x5, x1+        umull v21.2D, v19.2S, v16.2S+        cneg x13, x13, cc+        uaddlp v5.2D, v4.4S+        subs x11, x1, x11+        and v16.16B, v3.16B, v2.16B+        umulh x5, x5, x1+        shl v24.2D, v5.2D, #32+        cneg x11, x11, cc+        umlal v16.2D, v19.2S, v1.2S+        cinv x12, x15, cc+        umlal v24.2D, v0.2S, v1.2S+        adds x15, x10, x7+        mul x14, x13, x11+        eor x1, x6, x2+        adcs x6, x8, x5+        stp x9, x3, [x0]+        usra v21.2D, v3.2D, #32+        adcs x9, x5, xzr+        umulh x11, x13, x11+        adds x15, x8, x15+        adcs x7, x7, x6+        eor x8, x14, x12+        usra v21.2D, v16.2D, #32+        adcs x13, x9, xzr+        cmn x12, #0x1+        mov x9, v24.d[1]+        adcs x14, x15, x8+        eor x6, x11, x12+        adcs x6, x7, x6+        mov x5, v24.d[0]+        mov x11, v21.d[1]+        mov x7, v21.d[0]+        adc x3, x13, x12+        adds x12, x5, x9+        adcs x13, x7, x11+        ldp x15, x8, [x0]+        adcs x11, x11, xzr+        adds x12, x7, x12+        eor x16, x16, x2+        adcs x7, x9, x13+        adcs x11, x11, xzr+        cmn x2, #0x1+        ldp x9, x13, [x0, #16]+        adcs x16, x12, x16+        adcs x1, x7, x1+        adc x2, x11, x2+        adds x7, x5, x15+        adcs x15, x16, x8+        eor x5, x17, x4+        adcs x9, x1, x9+        eor x1, x10, x5+        adcs x16, x2, x13+        adc x2, xzr, xzr+        cmn x5, #0x1+        eor x13, x14, x5+        adcs x14, x1, x7+        eor x1, x6, x5+        adcs x6, x13, x15+        adcs x10, x1, x9+        eor x4, x3, x5+        mov x1, #0xffffffff+        adcs x8, x4, x16+        lsr x13, x14, #32+        adcs x17, x2, x5+        adcs x11, x5, xzr+        adc x4, x5, xzr+        adds x12, x10, x7+        adcs x7, x8, x15+        adcs x5, x17, x9+        adcs x9, x11, x16+        lsl x11, x14, #32+        adc x10, x4, x2+        subs x17, x14, x11+        sbc x4, x14, x13+        adds x11, x6, x11+        adcs x12, x12, x13+        lsl x15, x11, #32+        adcs x17, x7, x17+        lsr x7, x11, #32+        adc x13, x4, xzr+        subs x4, x11, x15+        sbc x11, x11, x7+        adds x8, x12, x15+        adcs x15, x17, x7+        adcs x4, x13, x4+        adc x11, x11, xzr+        adds x7, x5, x4+        adcs x17, x9, x11+        adc x13, x10, xzr+        add x12, x13, #0x1+        neg x11, x12+        lsl x4, x12, #32+        adds x17, x17, x4+        sub x4, x4, #0x1+        adc x13, x13, xzr+        subs x11, x8, x11+        sbcs x4, x15, x4+        sbcs x7, x7, xzr+        sbcs x17, x17, x12+        sbcs x13, x13, x12+        mov x12, #0xffffffff00000001+        adds x11, x11, x13+        and x1, x1, x13+        adcs x4, x4, x1+        and x1, x12, x13+        stp x11, x4, [x0]+        adcs x4, x7, xzr+        adc x1, x17, x1+        stp x4, x1, [x0, #16]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_local_montmul_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_local_montsqr_p256)++Lp256_scalarmulbase_local_montsqr_p256:+        CFI_START+        ldr q19, [x1]+        ldp x9, x13, [x1]+        ldr q23, [x1, #16]+        ldr q0, [x1]+        ldp x1, x10, [x1, #16]+        uzp2 v29.4S, v19.4S, v19.4S+        xtn v4.2S, v19.2D+        umulh x8, x9, x13+        rev64 v20.4S, v23.4S+        umull v16.2D, v19.2S, v19.2S+        umull v1.2D, v29.2S, v4.2S+        mul v20.4S, v20.4S, v0.4S+        subs x14, x9, x13+        umulh x15, x9, x1+        mov x16, v16.d[1]+        umull2 v4.2D, v19.4S, v19.4S+        mov x4, v16.d[0]+        uzp1 v17.4S, v23.4S, v0.4S+        uaddlp v19.2D, v20.4S+        lsr x7, x8, #63+        mul x11, x9, x13+        mov x12, v1.d[0]+        csetm x5, cc+        cneg x6, x14, cc+        mov x3, v4.d[1]+        mov x14, v4.d[0]+        subs x2, x10, x1+        mov x9, v1.d[1]+        cneg x17, x2, cc+        cinv x2, x5, cc+        adds x5, x4, x12, lsl #33+        extr x4, x8, x11, #63+        lsr x8, x12, #31+        uzp1 v20.4S, v0.4S, v0.4S+        shl v19.2D, v19.2D, #32+        adc x16, x16, x8+        adds x8, x14, x9, lsl #33+        lsr x14, x9, #31+        lsl x9, x5, #32+        umlal v19.2D, v20.2S, v17.2S+        adc x14, x3, x14+        adds x16, x16, x11, lsl #1+        lsr x3, x5, #32+        umulh x12, x6, x17+        adcs x4, x8, x4+        adc x11, x14, x7+        subs x8, x5, x9+        sbc x5, x5, x3+        adds x16, x16, x9+        mov x14, v19.d[0]+        mul x17, x6, x17+        adcs x3, x4, x3+        lsl x7, x16, #32+        umulh x13, x13, x10+        adcs x11, x11, x8+        lsr x8, x16, #32+        adc x5, x5, xzr+        subs x9, x16, x7+        sbc x16, x16, x8+        adds x7, x3, x7+        mov x3, v19.d[1]+        adcs x6, x11, x8+        umulh x11, x1, x10+        adcs x5, x5, x9+        eor x8, x12, x2+        adc x9, x16, xzr+        adds x16, x14, x15+        adc x15, x15, xzr+        adds x12, x16, x3+        eor x16, x17, x2+        mul x4, x1, x10+        adcs x15, x15, x13+        adc x17, x13, xzr+        adds x15, x15, x3+        adc x3, x17, xzr+        cmn x2, #0x1+        mul x17, x10, x10+        adcs x12, x12, x16+        adcs x16, x15, x8+        umulh x10, x10, x10+        adc x2, x3, x2+        adds x14, x14, x14+        adcs x12, x12, x12+        adcs x16, x16, x16+        adcs x2, x2, x2+        adc x15, xzr, xzr+        adds x14, x14, x7+        mul x3, x1, x1+        adcs x12, x12, x6+        lsr x7, x14, #32+        adcs x16, x16, x5+        lsl x5, x14, #32+        umulh x13, x1, x1+        adcs x2, x2, x9+        mov x6, #0xffffffff+        adc x15, x15, xzr+        adds x8, x4, x4+        adcs x1, x11, x11+        mov x11, #0xffffffff00000001+        adc x4, xzr, xzr+        subs x9, x14, x5+        sbc x14, x14, x7+        adds x12, x12, x5+        adcs x16, x16, x7+        lsl x5, x12, #32+        lsr x7, x12, #32+        adcs x2, x2, x9+        adcs x14, x15, x14+        adc x15, xzr, xzr+        subs x9, x12, x5+        sbc x12, x12, x7+        adds x16, x16, x5+        adcs x2, x2, x7+        adcs x14, x14, x9+        adcs x12, x15, x12+        adc x15, xzr, xzr+        adds x16, x16, x3+        adcs x2, x2, x13+        adcs x14, x14, x17+        adcs x12, x12, x10+        adc x15, x15, xzr+        adds x2, x2, x8+        adcs x14, x14, x1+        adcs x12, x12, x4+        adcs x15, x15, xzr+        adds x3, x16, #0x1+        sbcs x5, x2, x6+        sbcs x8, x14, xzr+        sbcs x11, x12, x11+        sbcs xzr, x15, xzr+        csel x16, x3, x16, cs+        csel x14, x8, x14, cs+        csel x12, x11, x12, cs+        csel x2, x5, x2, cs+        stp x14, x12, [x0, #16]+        stp x16, x2, [x0]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_local_montsqr_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_local_p256_montjmixadd)++Lp256_scalarmulbase_local_p256_montjmixadd:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_DEC_SP(192)+        mov     x17, x0+        mov     x19, x1+        mov     x20, x2+        ldp     x2, x3, [x19, #64]+        ldp     x4, x5, [x19, #80]+        umull   x15, w2, w2+        lsr     x11, x2, #32+        umull   x16, w11, w11+        umull   x11, w2, w11+        adds    x15, x15, x11, lsl #33+        lsr     x11, x11, #31+        adc     x16, x16, x11+        umull   x0, w3, w3+        lsr     x11, x3, #32+        umull   x1, w11, w11+        umull   x11, w3, w11+        mul     x12, x2, x3+        umulh   x13, x2, x3+        adds    x0, x0, x11, lsl #33+        lsr     x11, x11, #31+        adc     x1, x1, x11+        adds    x12, x12, x12+        adcs    x13, x13, x13+        adc     x1, x1, xzr+        adds    x16, x16, x12+        adcs    x0, x0, x13+        adc     x1, x1, xzr+        lsl     x12, x15, #32+        subs    x13, x15, x12+        lsr     x11, x15, #32+        sbc     x15, x15, x11+        adds    x16, x16, x12+        adcs    x0, x0, x11+        adcs    x1, x1, x13+        adc     x15, x15, xzr+        lsl     x12, x16, #32+        subs    x13, x16, x12+        lsr     x11, x16, #32+        sbc     x16, x16, x11+        adds    x0, x0, x12+        adcs    x1, x1, x11+        adcs    x15, x15, x13+        adc     x16, x16, xzr+        mul     x6, x2, x4+        mul     x14, x3, x5+        umulh   x8, x2, x4+        subs    x10, x2, x3+        cneg    x10, x10, cc+        csetm   x13, cc+        subs    x12, x5, x4+        cneg    x12, x12, cc+        mul     x11, x10, x12+        umulh   x12, x10, x12+        cinv    x13, x13, cc+        eor     x11, x11, x13+        eor     x12, x12, x13+        adds    x7, x6, x8+        adc     x8, x8, xzr+        umulh   x9, x3, x5+        adds    x7, x7, x14+        adcs    x8, x8, x9+        adc     x9, x9, xzr+        adds    x8, x8, x14+        adc     x9, x9, xzr+        cmn     x13, #0x1+        adcs    x7, x7, x11+        adcs    x8, x8, x12+        adc     x9, x9, x13+        adds    x6, x6, x6+        adcs    x7, x7, x7+        adcs    x8, x8, x8+        adcs    x9, x9, x9+        adc     x10, xzr, xzr+        adds    x6, x6, x0+        adcs    x7, x7, x1+        adcs    x8, x8, x15+        adcs    x9, x9, x16+        adc     x10, x10, xzr+        lsl     x12, x6, #32+        subs    x13, x6, x12+        lsr     x11, x6, #32+        sbc     x6, x6, x11+        adds    x7, x7, x12+        adcs    x8, x8, x11+        adcs    x9, x9, x13+        adcs    x10, x10, x6+        adc     x6, xzr, xzr+        lsl     x12, x7, #32+        subs    x13, x7, x12+        lsr     x11, x7, #32+        sbc     x7, x7, x11+        adds    x8, x8, x12+        adcs    x9, x9, x11+        adcs    x10, x10, x13+        adcs    x6, x6, x7+        adc     x7, xzr, xzr+        mul     x11, x4, x4+        adds    x8, x8, x11+        mul     x12, x5, x5+        umulh   x11, x4, x4+        adcs    x9, x9, x11+        adcs    x10, x10, x12+        umulh   x12, x5, x5+        adcs    x6, x6, x12+        adc     x7, x7, xzr+        mul     x11, x4, x5+        umulh   x12, x4, x5+        adds    x11, x11, x11+        adcs    x12, x12, x12+        adc     x13, xzr, xzr+        adds    x9, x9, x11+        adcs    x10, x10, x12+        adcs    x6, x6, x13+        adcs    x7, x7, xzr+        mov     x11, #0xffffffff+        adds    x5, x8, #0x1+        sbcs    x11, x9, x11+        mov     x13, #0xffffffff00000001+        sbcs    x12, x10, xzr+        sbcs    x13, x6, x13+        sbcs    xzr, x7, xzr+        csel    x8, x5, x8, cs+        csel    x9, x11, x9, cs+        csel    x10, x12, x10, cs+        csel    x6, x13, x6, cs+        stp     x8, x9, [sp]+        stp     x10, x6, [sp, #16]+        ldp     x3, x4, [x19, #64]+        ldp     x5, x6, [x19, #80]+        ldp     x7, x8, [x20, #32]+        ldp     x9, x10, [x20, #48]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #32]+        stp     x11, x12, [sp, #48]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #32]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #48]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #32]+        stp     x3, x4, [sp, #48]+        ldp     x3, x4, [sp]+        ldp     x5, x6, [sp, #16]+        ldp     x7, x8, [x20]+        ldp     x9, x10, [x20, #16]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #64]+        stp     x11, x12, [sp, #80]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #64]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #80]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #64]+        stp     x3, x4, [sp, #80]+        ldp     x3, x4, [sp]+        ldp     x5, x6, [sp, #16]+        ldp     x7, x8, [sp, #32]+        ldp     x9, x10, [sp, #48]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #32]+        stp     x11, x12, [sp, #48]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #32]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #48]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #32]+        stp     x3, x4, [sp, #48]+        ldp     x5, x6, [sp, #64]+        ldp     x4, x3, [x19]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #80]+        ldp     x4, x3, [x19, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #160]+        stp     x7, x8, [sp, #176]+        ldp     x5, x6, [sp, #32]+        ldp     x4, x3, [x19, #32]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #48]+        ldp     x4, x3, [x19, #48]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #32]+        stp     x7, x8, [sp, #48]+        ldp     x2, x3, [sp, #160]+        ldp     x4, x5, [sp, #176]+        umull   x15, w2, w2+        lsr     x11, x2, #32+        umull   x16, w11, w11+        umull   x11, w2, w11+        adds    x15, x15, x11, lsl #33+        lsr     x11, x11, #31+        adc     x16, x16, x11+        umull   x0, w3, w3+        lsr     x11, x3, #32+        umull   x1, w11, w11+        umull   x11, w3, w11+        mul     x12, x2, x3+        umulh   x13, x2, x3+        adds    x0, x0, x11, lsl #33+        lsr     x11, x11, #31+        adc     x1, x1, x11+        adds    x12, x12, x12+        adcs    x13, x13, x13+        adc     x1, x1, xzr+        adds    x16, x16, x12+        adcs    x0, x0, x13+        adc     x1, x1, xzr+        lsl     x12, x15, #32+        subs    x13, x15, x12+        lsr     x11, x15, #32+        sbc     x15, x15, x11+        adds    x16, x16, x12+        adcs    x0, x0, x11+        adcs    x1, x1, x13+        adc     x15, x15, xzr+        lsl     x12, x16, #32+        subs    x13, x16, x12+        lsr     x11, x16, #32+        sbc     x16, x16, x11+        adds    x0, x0, x12+        adcs    x1, x1, x11+        adcs    x15, x15, x13+        adc     x16, x16, xzr+        mul     x6, x2, x4+        mul     x14, x3, x5+        umulh   x8, x2, x4+        subs    x10, x2, x3+        cneg    x10, x10, cc+        csetm   x13, cc+        subs    x12, x5, x4+        cneg    x12, x12, cc+        mul     x11, x10, x12+        umulh   x12, x10, x12+        cinv    x13, x13, cc+        eor     x11, x11, x13+        eor     x12, x12, x13+        adds    x7, x6, x8+        adc     x8, x8, xzr+        umulh   x9, x3, x5+        adds    x7, x7, x14+        adcs    x8, x8, x9+        adc     x9, x9, xzr+        adds    x8, x8, x14+        adc     x9, x9, xzr+        cmn     x13, #0x1+        adcs    x7, x7, x11+        adcs    x8, x8, x12+        adc     x9, x9, x13+        adds    x6, x6, x6+        adcs    x7, x7, x7+        adcs    x8, x8, x8+        adcs    x9, x9, x9+        adc     x10, xzr, xzr+        adds    x6, x6, x0+        adcs    x7, x7, x1+        adcs    x8, x8, x15+        adcs    x9, x9, x16+        adc     x10, x10, xzr+        lsl     x12, x6, #32+        subs    x13, x6, x12+        lsr     x11, x6, #32+        sbc     x6, x6, x11+        adds    x7, x7, x12+        adcs    x8, x8, x11+        adcs    x9, x9, x13+        adcs    x10, x10, x6+        adc     x6, xzr, xzr+        lsl     x12, x7, #32+        subs    x13, x7, x12+        lsr     x11, x7, #32+        sbc     x7, x7, x11+        adds    x8, x8, x12+        adcs    x9, x9, x11+        adcs    x10, x10, x13+        adcs    x6, x6, x7+        adc     x7, xzr, xzr+        mul     x11, x4, x4+        adds    x8, x8, x11+        mul     x12, x5, x5+        umulh   x11, x4, x4+        adcs    x9, x9, x11+        adcs    x10, x10, x12+        umulh   x12, x5, x5+        adcs    x6, x6, x12+        adc     x7, x7, xzr+        mul     x11, x4, x5+        umulh   x12, x4, x5+        adds    x11, x11, x11+        adcs    x12, x12, x12+        adc     x13, xzr, xzr+        adds    x9, x9, x11+        adcs    x10, x10, x12+        adcs    x6, x6, x13+        adcs    x7, x7, xzr+        mov     x11, #0xffffffff+        adds    x5, x8, #0x1+        sbcs    x11, x9, x11+        mov     x13, #0xffffffff00000001+        sbcs    x12, x10, xzr+        sbcs    x13, x6, x13+        sbcs    xzr, x7, xzr+        csel    x8, x5, x8, cs+        csel    x9, x11, x9, cs+        csel    x10, x12, x10, cs+        csel    x6, x13, x6, cs+        stp     x8, x9, [sp, #96]+        stp     x10, x6, [sp, #112]+        ldp     x2, x3, [sp, #32]+        ldp     x4, x5, [sp, #48]+        umull   x15, w2, w2+        lsr     x11, x2, #32+        umull   x16, w11, w11+        umull   x11, w2, w11+        adds    x15, x15, x11, lsl #33+        lsr     x11, x11, #31+        adc     x16, x16, x11+        umull   x0, w3, w3+        lsr     x11, x3, #32+        umull   x1, w11, w11+        umull   x11, w3, w11+        mul     x12, x2, x3+        umulh   x13, x2, x3+        adds    x0, x0, x11, lsl #33+        lsr     x11, x11, #31+        adc     x1, x1, x11+        adds    x12, x12, x12+        adcs    x13, x13, x13+        adc     x1, x1, xzr+        adds    x16, x16, x12+        adcs    x0, x0, x13+        adc     x1, x1, xzr+        lsl     x12, x15, #32+        subs    x13, x15, x12+        lsr     x11, x15, #32+        sbc     x15, x15, x11+        adds    x16, x16, x12+        adcs    x0, x0, x11+        adcs    x1, x1, x13+        adc     x15, x15, xzr+        lsl     x12, x16, #32+        subs    x13, x16, x12+        lsr     x11, x16, #32+        sbc     x16, x16, x11+        adds    x0, x0, x12+        adcs    x1, x1, x11+        adcs    x15, x15, x13+        adc     x16, x16, xzr+        mul     x6, x2, x4+        mul     x14, x3, x5+        umulh   x8, x2, x4+        subs    x10, x2, x3+        cneg    x10, x10, cc+        csetm   x13, cc+        subs    x12, x5, x4+        cneg    x12, x12, cc+        mul     x11, x10, x12+        umulh   x12, x10, x12+        cinv    x13, x13, cc+        eor     x11, x11, x13+        eor     x12, x12, x13+        adds    x7, x6, x8+        adc     x8, x8, xzr+        umulh   x9, x3, x5+        adds    x7, x7, x14+        adcs    x8, x8, x9+        adc     x9, x9, xzr+        adds    x8, x8, x14+        adc     x9, x9, xzr+        cmn     x13, #0x1+        adcs    x7, x7, x11+        adcs    x8, x8, x12+        adc     x9, x9, x13+        adds    x6, x6, x6+        adcs    x7, x7, x7+        adcs    x8, x8, x8+        adcs    x9, x9, x9+        adc     x10, xzr, xzr+        adds    x6, x6, x0+        adcs    x7, x7, x1+        adcs    x8, x8, x15+        adcs    x9, x9, x16+        adc     x10, x10, xzr+        lsl     x12, x6, #32+        subs    x13, x6, x12+        lsr     x11, x6, #32+        sbc     x6, x6, x11+        adds    x7, x7, x12+        adcs    x8, x8, x11+        adcs    x9, x9, x13+        adcs    x10, x10, x6+        adc     x6, xzr, xzr+        lsl     x12, x7, #32+        subs    x13, x7, x12+        lsr     x11, x7, #32+        sbc     x7, x7, x11+        adds    x8, x8, x12+        adcs    x9, x9, x11+        adcs    x10, x10, x13+        adcs    x6, x6, x7+        adc     x7, xzr, xzr+        mul     x11, x4, x4+        adds    x8, x8, x11+        mul     x12, x5, x5+        umulh   x11, x4, x4+        adcs    x9, x9, x11+        adcs    x10, x10, x12+        umulh   x12, x5, x5+        adcs    x6, x6, x12+        adc     x7, x7, xzr+        mul     x11, x4, x5+        umulh   x12, x4, x5+        adds    x11, x11, x11+        adcs    x12, x12, x12+        adc     x13, xzr, xzr+        adds    x9, x9, x11+        adcs    x10, x10, x12+        adcs    x6, x6, x13+        adcs    x7, x7, xzr+        mov     x11, #0xffffffff+        adds    x5, x8, #0x1+        sbcs    x11, x9, x11+        mov     x13, #0xffffffff00000001+        sbcs    x12, x10, xzr+        sbcs    x13, x6, x13+        sbcs    xzr, x7, xzr+        csel    x8, x5, x8, cs+        csel    x9, x11, x9, cs+        csel    x10, x12, x10, cs+        csel    x6, x13, x6, cs+        stp     x8, x9, [sp]+        stp     x10, x6, [sp, #16]+        ldp     x3, x4, [sp, #96]+        ldp     x5, x6, [sp, #112]+        ldp     x7, x8, [x19]+        ldp     x9, x10, [x19, #16]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #128]+        stp     x11, x12, [sp, #144]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #128]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #144]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #128]+        stp     x3, x4, [sp, #144]+        ldp     x3, x4, [sp, #96]+        ldp     x5, x6, [sp, #112]+        ldp     x7, x8, [sp, #64]+        ldp     x9, x10, [sp, #80]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #64]+        stp     x11, x12, [sp, #80]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #64]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #80]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #64]+        stp     x3, x4, [sp, #80]+        ldp     x5, x6, [sp]+        ldp     x4, x3, [sp, #128]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #16]+        ldp     x4, x3, [sp, #144]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp]+        stp     x7, x8, [sp, #16]+        ldp     x5, x6, [sp, #64]+        ldp     x4, x3, [sp, #128]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #80]+        ldp     x4, x3, [sp, #144]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #96]+        stp     x7, x8, [sp, #112]+        ldp     x3, x4, [sp, #160]+        ldp     x5, x6, [sp, #176]+        ldp     x7, x8, [x19, #64]+        ldp     x9, x10, [x19, #80]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #160]+        stp     x11, x12, [sp, #176]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #160]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #176]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #160]+        stp     x3, x4, [sp, #176]+        ldp     x5, x6, [sp]+        ldp     x4, x3, [sp, #64]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #16]+        ldp     x4, x3, [sp, #80]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp]+        stp     x7, x8, [sp, #16]+        ldp     x5, x6, [sp, #128]+        ldp     x4, x3, [sp]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #144]+        ldp     x4, x3, [sp, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #128]+        stp     x7, x8, [sp, #144]+        ldp     x3, x4, [sp, #96]+        ldp     x5, x6, [sp, #112]+        ldp     x7, x8, [x19, #32]+        ldp     x9, x10, [x19, #48]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #96]+        stp     x11, x12, [sp, #112]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #96]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #112]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #96]+        stp     x3, x4, [sp, #112]+        ldp     x3, x4, [sp, #32]+        ldp     x5, x6, [sp, #48]+        ldp     x7, x8, [sp, #128]+        ldp     x9, x10, [sp, #144]+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x3, x4+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        stp     x13, x14, [sp, #128]+        stp     x11, x12, [sp, #144]+        mul     x11, x5, x9+        mul     x13, x6, x10+        umulh   x12, x5, x9+        adds    x16, x11, x13+        umulh   x14, x6, x10+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x15, x5, x6+        cneg    x15, x15, cc+        csetm   x1, cc+        subs    x0, x10, x9+        cneg    x0, x0, cc+        mul     x16, x15, x0+        umulh   x0, x15, x0+        cinv    x1, x1, cc+        eor     x16, x16, x1+        eor     x0, x0, x1+        cmn     x1, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x1+        subs    x3, x5, x3+        sbcs    x4, x6, x4+        ngc     x5, xzr+        cmn     x5, #0x1+        eor     x3, x3, x5+        adcs    x3, x3, xzr+        eor     x4, x4, x5+        adcs    x4, x4, xzr+        subs    x7, x7, x9+        sbcs    x8, x8, x10+        ngc     x9, xzr+        cmn     x9, #0x1+        eor     x7, x7, x9+        adcs    x7, x7, xzr+        eor     x8, x8, x9+        adcs    x8, x8, xzr+        eor     x10, x5, x9+        ldp     x15, x1, [sp, #128]+        adds    x15, x11, x15+        adcs    x1, x12, x1+        ldp     x5, x9, [sp, #144]+        adcs    x5, x13, x5+        adcs    x9, x14, x9+        adc     x2, xzr, xzr+        mul     x11, x3, x7+        mul     x13, x4, x8+        umulh   x12, x3, x7+        adds    x16, x11, x13+        umulh   x14, x4, x8+        adcs    x0, x12, x14+        adcs    x14, x14, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x0+        adcs    x14, x14, xzr+        subs    x3, x3, x4+        cneg    x3, x3, cc+        csetm   x4, cc+        subs    x0, x8, x7+        cneg    x0, x0, cc+        mul     x16, x3, x0+        umulh   x0, x3, x0+        cinv    x4, x4, cc+        eor     x16, x16, x4+        eor     x0, x0, x4+        cmn     x4, #0x1+        adcs    x12, x12, x16+        adcs    x13, x13, x0+        adc     x14, x14, x4+        cmn     x10, #0x1+        eor     x11, x11, x10+        adcs    x11, x11, x15+        eor     x12, x12, x10+        adcs    x12, x12, x1+        eor     x13, x13, x10+        adcs    x13, x13, x5+        eor     x14, x14, x10+        adcs    x14, x14, x9+        adcs    x3, x2, x10+        adcs    x4, x10, xzr+        adc     x10, x10, xzr+        adds    x13, x13, x15+        adcs    x14, x14, x1+        adcs    x3, x3, x5+        adcs    x4, x4, x9+        adc     x10, x10, x2+        lsl     x0, x11, #32+        subs    x1, x11, x0+        lsr     x16, x11, #32+        sbc     x11, x11, x16+        adds    x12, x12, x0+        adcs    x13, x13, x16+        adcs    x14, x14, x1+        adc     x11, x11, xzr+        lsl     x0, x12, #32+        subs    x1, x12, x0+        lsr     x16, x12, #32+        sbc     x12, x12, x16+        adds    x13, x13, x0+        adcs    x14, x14, x16+        adcs    x11, x11, x1+        adc     x12, x12, xzr+        adds    x3, x3, x11+        adcs    x4, x4, x12+        adc     x10, x10, xzr+        add     x2, x10, #0x1+        lsl     x16, x2, #32+        adds    x4, x4, x16+        adc     x10, x10, xzr+        neg     x15, x2+        sub     x16, x16, #0x1+        subs    x13, x13, x15+        sbcs    x14, x14, x16+        sbcs    x3, x3, xzr+        sbcs    x4, x4, x2+        sbcs    x7, x10, x2+        adds    x13, x13, x7+        mov     x10, #0xffffffff+        and     x10, x10, x7+        adcs    x14, x14, x10+        adcs    x3, x3, xzr+        mov     x10, #0xffffffff00000001+        and     x10, x10, x7+        adc     x4, x4, x10+        stp     x13, x14, [sp, #128]+        stp     x3, x4, [sp, #144]+        ldp     x5, x6, [sp, #128]+        ldp     x4, x3, [sp, #96]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #144]+        ldp     x4, x3, [sp, #112]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #128]+        stp     x7, x8, [sp, #144]+        ldp     x0, x1, [x19, #64]+        ldp     x2, x3, [x19, #80]+        orr     x4, x0, x1+        orr     x5, x2, x3+        orr     x4, x4, x5+        cmp     x4, xzr+        ldp     x0, x1, [sp]+        ldp     x12, x13, [x20]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x2, x3, [sp, #16]+        ldp     x12, x13, [x20, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x4, x5, [sp, #128]+        ldp     x12, x13, [x20, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x6, x7, [sp, #144]+        ldp     x12, x13, [x20, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x8, x9, [sp, #160]+        mov     x12, #0x1+        mov     x13, #0xffffffff00000000+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x10, x11, [sp, #176]+        mov     x12, #0xffffffffffffffff+        mov     x13, #0xfffffffe+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        stp     x0, x1, [x17]+        stp     x2, x3, [x17, #16]+        stp     x4, x5, [x17, #32]+        stp     x6, x7, [x17, #48]+        stp     x8, x9, [x17, #64]+        stp     x10, x11, [x17, #80]+        CFI_INC_SP(192)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_local_p256_montjmixadd)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/p256_scalarmulbase_alt.S view
@@ -0,0 +1,3057 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Scalar multiplication for precomputed point on NIST curve P-256+// Input scalar[4], blocksize, table[]; output res[8]+//+// extern void p256_scalarmulbase_alt+//   (uint64_t res[static 8],+//    const uint64_t scalar[static 4],+//    uint64_t blocksize,+//    const uint64_t *table);+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-256, the input argument "table" is expected to be a table of+// multiples of the point P in Montgomery-affine form, with each block+// corresponding to "blocksize" bits of the scalar as follows, where+// B = 2^{blocksize-1} (e.g. B = 8 for blocksize = 4):+//+// For each i,j with blocksize * i <= 256 and 1 <= j <= B+// the multiple 2^{blocksize * i} * j * P is stored at+// tab[8 * (B * i + (j - 1))], considered as uint64_t pointers+// or tab + 64 * (B * i + (j - 1)) as byte pointers.+//+// Standard ARM ABI: X0 = res, X1 = scalar, X2 = blocksize, X3 = table+// ----------------------------------------------------------------------------+++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_scalarmulbase_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_scalarmulbase_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_scalarmulbase_alt)+++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Safe copies of inputs and additional variables, with some aliasing++#define res x19+#define blocksize x20+#define table x21+#define i x22+#define bf x23+#define cf x24+#define j x25++// Intermediate variables on the stack. The last z2, z3 values can+// safely be overlaid on "nacc", which is no longer needed at the end.+// Uppercase syntactic variants make x86_att version simpler to generate++#define rscalar sp, #(0*NUMSIZE)+#define acc sp, #(1*NUMSIZE)+#define nacc sp, #(4*NUMSIZE)+#define tabent sp, #(7*NUMSIZE)++#define z2 sp, #(4*NUMSIZE)+#define z3 sp, #(5*NUMSIZE)++#define NSPACE 9*NUMSIZE++// Loading large constants++#define movbig(nn,n3,n2,n1,n0)                                      \+        movz    nn, n0 __LF                                            \+        movk    nn, n1, lsl #16 __LF                                   \+        movk    nn, n2, lsl #32 __LF                                   \+        movk    nn, n3, lsl #48++S2N_BN_SYMBOL(p256_scalarmulbase_alt):+        CFI_START++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x30)+        CFI_DEC_SP(NSPACE)++// Preserve the input arguments except the scalar, since that gets absorbed+// immediately. The "table" value subsequently gets shifted up each iteration+// of the loop, while "res" and "blocksize" are static throughout.++        mov     res, x0+        mov     blocksize, x2+        mov     table, x3++// Load the digits of group order n_256 = [x15;x14;x13;x12]++        movbig(x12, #0xf3b9, #0xcac2, #0xfc63, #0x2551)+        movbig(x13, #0xbce6, #0xfaad, #0xa717, #0x9e84)+        mov     x14, #0xffffffffffffffff+        mov     x15, #0xffffffff00000000++// First, reduce the input scalar mod n_256, i.e. conditionally subtract n_256+// Store it to "rscalar" (reduced scalar)++        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]++        subs    x6, x2, x12+        sbcs    x7, x3, x13+        sbcs    x8, x4, x14+        sbcs    x9, x5, x15++        csel    x2, x2, x6, cc+        csel    x3, x3, x7, cc+        csel    x4, x4, x8, cc+        csel    x5, x5, x9, cc++        stp     x2, x3, [rscalar]+        stp     x4, x5, [rscalar+16]++// Initialize the accumulator to all zeros and the "carry flag" cf to 0++        stp     xzr, xzr, [acc]+        stp     xzr, xzr, [acc+16]+        stp     xzr, xzr, [acc+32]+        stp     xzr, xzr, [acc+48]+        stp     xzr, xzr, [acc+64]+        stp     xzr, xzr, [acc+80]+        mov     cf, xzr++// Main loop over {i >= 0 | blocksize * i <= 256}. Note the non-strict+// inequality, to allow top carry for any choices of blocksize.++        mov     i, xzr++Lp256_scalarmulbase_alt_loop:++// The next raw bitfield is bf = bitfield(blocksize * i,blocksize) + cf,+// adding in the deferred carry cf. We then shift the whole scalar right+// by blocksize so we can keep picking bitfield(0,blocksize).++        ldp     x0, x1, [rscalar]+        ldp     x2, x3, [rscalar+16]++        mov     x4, #1+        lsl     x4, x4, blocksize+        sub     x4, x4, #1+        and     x4, x4, x0+        add     bf, x4, cf++        neg     x8, blocksize++        lsl     x5, x1, x8++        lsr     x0, x0, blocksize+        orr     x0, x0,  x5++        lsl     x6, x2, x8+        lsr     x1, x1, blocksize+        orr     x1, x1, x6++        lsl     x7, x3, x8+        lsr     x2, x2, blocksize+        orr     x2, x2, x7++        lsr     x3, x3, blocksize++        stp     x0, x1, [rscalar]+        stp     x2, x3, [rscalar+16]++// Now if bf <= B we just select entry j, unnegated and set cf = 0.+// If bf > B we set j = 2 * B - bf and negate the j'th entry, setting cf = 1.+// In either case we ultimately add bf, in the latter case with deferred+// carry as 2 * B - (2 * B - bf) = bf.++        mov     x0, #1+        lsl     x1, x0, blocksize+        lsr     x0, x1, #1++        sub     x2, x1, bf++        cmp     x0, bf+        cset    cf, cc+        csel    j, x2, bf, cc++// Load table entry j - 1 for nonzero j in constant-time style.++        mov     x16, #1+        lsl     x16, x16, blocksize+        lsr     x16, x16, #1+        mov     x17, j++Lp256_scalarmulbase_alt_tabloop:+        ldp     x8, x9, [table]+        ldp     x10, x11, [table, #16]+        ldp     x12, x13, [table, #32]+        ldp     x14, x15, [table, #48]++        subs    x17, x17, #1+        csel    x0, x8, x0, eq+        csel    x1, x9, x1, eq+        csel    x2, x10, x2, eq+        csel    x3, x11, x3, eq+        csel    x4, x12, x4, eq+        csel    x5, x13, x5, eq+        csel    x6, x14, x6, eq+        csel    x7, x15, x7, eq++        add     table, table, #64++        sub     x16, x16, #1+        cbnz    x16, Lp256_scalarmulbase_alt_tabloop++// Before storing back, optionally negate the y coordinate of the table entry++        stp     x0, x1, [tabent]+        stp     x2, x3, [tabent+16]++        mov     x0, 0xffffffffffffffff+        subs    x0, x0, x4+        mov     x1, 0x00000000ffffffff+        sbcs    x1, x1, x5+        mov     x3, 0xffffffff00000001+        sbcs    x2, xzr, x6+        sbc     x3, x3, x7++        cmp     cf, xzr+        csel    x4, x0, x4, ne+        csel    x5, x1, x5, ne+        csel    x6, x2, x6, ne+        csel    x7, x3, x7, ne++        stp     x4, x5, [tabent+32]+        stp     x6, x7, [tabent+48]++// Add the adjusted table point to the accumulator++        add     x0, nacc+        add     x1, acc+        add     x2, tabent+        CFI_BL(Lp256_scalarmulbase_alt_local_p256_montjmixadd)++// However, only commit that update to the accumulator if j is nonzero,+// because the mixed addition function does not handle this case directly,+// and in any case we didn't choose the table entry appropriately.++        cmp     j, xzr+        ldp     x0, x1, [acc]+        ldp     x12, x13, [nacc]+        csel    x0, x12, x0, ne+        csel    x1, x13, x1, ne++        ldp     x2, x3, [acc+16]+        ldp     x12, x13, [nacc+16]+        csel    x2, x12, x2, ne+        csel    x3, x13, x3, ne++        ldp     x4, x5, [acc+32]+        ldp     x12, x13, [nacc+32]+        csel    x4, x12, x4, ne+        csel    x5, x13, x5, ne++        ldp     x6, x7, [acc+48]+        ldp     x12, x13, [nacc+48]+        csel    x6, x12, x6, ne+        csel    x7, x13, x7, ne++        ldp     x8, x9, [acc+64]+        ldp     x12, x13, [nacc+64]+        csel    x8, x12, x8, ne+        csel    x9, x13, x9, ne++        ldp     x10, x11, [acc+80]+        ldp     x12, x13, [nacc+80]+        csel    x10, x12, x10, ne+        csel    x11, x13, x11, ne++        stp     x0, x1, [acc]+        stp     x2, x3, [acc+16]+        stp     x4, x5, [acc+32]+        stp     x6, x7, [acc+48]+        stp     x8, x9, [acc+64]+        stp     x10, x11, [acc+80]++// Loop while blocksize * i <= 256++        add     i, i, #1+        mul     x0, blocksize, i+        cmp     x0, #257+        bcc     Lp256_scalarmulbase_alt_loop++// That's the end of the main loop, and we just need to translate+// back from the Jacobian representation to affine. First of all,+// let z2 = 1/z^2 and z3 = 1/z^3, both without Montgomery form++        add     x0, z2+        add     x1, acc+64+        CFI_BL(Lp256_scalarmulbase_alt_local_montsqr_p256)++        add     x0, z3+        add     x1, acc+64+        add     x2, z2+        CFI_BL(Lp256_scalarmulbase_alt_local_montmul_p256)++        add     x0, z2+        add     x1, z3+        CFI_BL(Lp256_scalarmulbase_alt_local_demont_p256)++        add     x0, z3+        add     x1, z2+        CFI_BL(Lp256_scalarmulbase_alt_local_inv_p256)++        add     x0, z2+        add     x1, acc+64+        add     x2, z3+        CFI_BL(Lp256_scalarmulbase_alt_local_montmul_p256)++// Convert back from Jacobian (X,Y,Z) |-> (X/Z^2, Y/Z^3)++        mov     x0, res+        add     x1, acc+        add     x2, z2+        CFI_BL(Lp256_scalarmulbase_alt_local_montmul_p256)++        add     x0, res, #32+        add     x1, acc+32+        add     x2, z3+        CFI_BL(Lp256_scalarmulbase_alt_local_montmul_p256)++// Restore stack and registers and return++        CFI_INC_SP(NSPACE)+        CFI_POP2(x25,x30)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_scalarmulbase_alt)++// Local copies of subroutines, complete clones at the moment++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_alt_local_demont_p256)++Lp256_scalarmulbase_alt_local_demont_p256:+        CFI_START+        ldp     x2, x3, [x1]+        ldp     x4, x5, [x1, #16]+        lsl     x7, x2, #32+        subs    x8, x2, x7+        lsr     x6, x2, #32+        sbc     x2, x2, x6+        adds    x3, x3, x7+        adcs    x4, x4, x6+        adcs    x5, x5, x8+        adc     x2, x2, xzr+        lsl     x7, x3, #32+        subs    x8, x3, x7+        lsr     x6, x3, #32+        sbc     x3, x3, x6+        adds    x4, x4, x7+        adcs    x5, x5, x6+        adcs    x2, x2, x8+        adc     x3, x3, xzr+        lsl     x7, x4, #32+        subs    x8, x4, x7+        lsr     x6, x4, #32+        sbc     x4, x4, x6+        adds    x5, x5, x7+        adcs    x2, x2, x6+        adcs    x3, x3, x8+        adc     x4, x4, xzr+        lsl     x7, x5, #32+        subs    x8, x5, x7+        lsr     x6, x5, #32+        sbc     x5, x5, x6+        adds    x2, x2, x7+        adcs    x3, x3, x6+        adcs    x4, x4, x8+        adc     x5, x5, xzr+        stp     x2, x3, [x0]+        stp     x4, x5, [x0, #16]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_alt_local_demont_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_alt_local_inv_p256)++Lp256_scalarmulbase_alt_local_inv_p256:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_DEC_SP(160)+        mov     x20, x0+        mov     x10, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x13, #0xffffffff00000001+        stp     x10, x11, [sp]+        stp     xzr, x13, [sp, #16]+        str     xzr, [sp, #32]+        ldp     x2, x3, [x1]+        subs    x10, x2, x10+        sbcs    x11, x3, x11+        ldp     x4, x5, [x1, #16]+        sbcs    x12, x4, xzr+        sbcs    x13, x5, x13+        csel    x2, x2, x10, cc+        csel    x3, x3, x11, cc+        csel    x4, x4, x12, cc+        csel    x5, x5, x13, cc+        stp     x2, x3, [sp, #48]+        stp     x4, x5, [sp, #64]+        str     xzr, [sp, #80]+        stp     xzr, xzr, [sp, #96]+        stp     xzr, xzr, [sp, #112]+        mov     x10, #0x4000000000000+        stp     x10, xzr, [sp, #128]+        stp     xzr, xzr, [sp, #144]+        mov     x21, #0xa+        mov     x22, #0x1+        b       Lp256_scalarmulbase_alt_inv_midloop+Lp256_scalarmulbase_alt_inv_loop:+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        and     x0, x12, x16+        and     x1, x13, x17+        add     x19, x0, x1+        ldr     x7, [sp]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #48]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x7, [sp, #8]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #56]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        adc     x6, x6, x1+        extr    x4, x2, x4, #59+        str     x4, [sp]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        adc     x4, x4, x1+        extr    x5, x3, x5, #59+        str     x5, [sp, #48]+        ldr     x7, [sp, #16]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #64]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        adc     x5, x5, x1+        extr    x2, x6, x2, #59+        str     x2, [sp, #8]+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        adc     x2, x2, x1+        extr    x3, x4, x3, #59+        str     x3, [sp, #56]+        ldr     x7, [sp, #24]+        eor     x1, x7, x14+        ldr     x23, [sp, #32]+        eor     x3, x23, x14+        and     x3, x3, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #72]+        eor     x1, x8, x15+        ldr     x24, [sp, #80]+        eor     x0, x24, x15+        and     x0, x0, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        extr    x6, x5, x6, #59+        str     x6, [sp, #16]+        extr    x5, x3, x5, #59+        str     x5, [sp, #24]+        asr     x3, x3, #59+        str     x3, [sp, #32]+        eor     x1, x7, x16+        eor     x5, x23, x16+        and     x5, x5, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        eor     x0, x24, x17+        and     x0, x0, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        extr    x4, x2, x4, #59+        str     x4, [sp, #64]+        extr    x2, x5, x2, #59+        str     x2, [sp, #72]+        asr     x5, x5, #59+        str     x5, [sp, #80]+        ldr     x7, [sp, #96]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #128]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #96]+        adc     x2, x2, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x5, x19, x0+        adc     x3, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x5, x5, x0+        str     x5, [sp, #128]+        adc     x3, x3, x1+        ldr     x7, [sp, #104]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #136]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #104]+        adc     x6, x6, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x3, x3, x0+        adc     x4, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x3, x3, x0+        str     x3, [sp, #136]+        adc     x4, x4, x1+        ldr     x7, [sp, #112]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #144]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #112]+        adc     x5, x5, x1+        eor     x1, x7, x16+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x4, x4, x0+        adc     x2, xzr, x1+        eor     x1, x8, x17+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x4, x4, x0+        str     x4, [sp, #144]+        adc     x2, x2, x1+        ldr     x7, [sp, #120]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #152]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldp     x0, x1, [sp, #96]+        ldr     x6, [sp, #112]+        mov     x14, #0xe000000000000000+        adds    x0, x0, x14+        sbcs    x1, x1, xzr+        mov     x11, #0x1fffffff+        adcs    x6, x6, x11+        mov     x10, #0x2000000000000000+        adcs    x5, x5, x10+        mov     x14, #0x1fffffffe0000000+        adc     x3, x3, x14+        lsl     x11, x0, #32+        subs    x14, x0, x11+        lsr     x10, x0, #32+        sbc     x0, x0, x10+        adds    x1, x1, x11+        adcs    x6, x6, x10+        adcs    x5, x5, x14+        adcs    x3, x3, x0+        mov     x14, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x10, #0xffffffff00000001+        csel    x14, x14, xzr, cs+        csel    x11, x11, xzr, cs+        csel    x10, x10, xzr, cs+        subs    x1, x1, x14+        sbcs    x6, x6, x11+        sbcs    x5, x5, xzr+        sbc     x3, x3, x10+        stp     x1, x6, [sp, #96]+        stp     x5, x3, [sp, #112]+        eor     x1, x7, x16+        and     x5, x16, x12+        neg     x5, x5+        mul     x0, x1, x12+        umulh   x1, x1, x12+        adds    x2, x2, x0+        adc     x5, x5, x1+        eor     x1, x8, x17+        and     x0, x17, x13+        sub     x5, x5, x0+        mul     x0, x1, x13+        umulh   x1, x1, x13+        adds    x2, x2, x0+        adc     x5, x5, x1+        ldp     x0, x1, [sp, #128]+        ldr     x3, [sp, #144]+        mov     x14, #0xe000000000000000+        adds    x0, x0, x14+        sbcs    x1, x1, xzr+        mov     x11, #0x1fffffff+        adcs    x3, x3, x11+        mov     x10, #0x2000000000000000+        adcs    x2, x2, x10+        mov     x14, #0x1fffffffe0000000+        adc     x5, x5, x14+        lsl     x11, x0, #32+        subs    x14, x0, x11+        lsr     x10, x0, #32+        sbc     x0, x0, x10+        adds    x1, x1, x11+        adcs    x3, x3, x10+        adcs    x2, x2, x14+        adcs    x5, x5, x0+        mov     x14, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x10, #0xffffffff00000001+        csel    x14, x14, xzr, cs+        csel    x11, x11, xzr, cs+        csel    x10, x10, xzr, cs+        subs    x1, x1, x14+        sbcs    x3, x3, x11+        sbcs    x2, x2, xzr+        sbc     x5, x5, x10+        stp     x1, x3, [sp, #128]+        stp     x2, x5, [sp, #144]+Lp256_scalarmulbase_alt_inv_midloop:+        mov     x1, x22+        ldr     x2, [sp]+        ldr     x3, [sp, #48]+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x8, x4, #0x100, lsl #12+        sbfx    x8, x8, #21, #21+        mov     x11, #0x100000+        add     x11, x11, x11, lsl #21+        add     x9, x4, x11+        asr     x9, x9, #42+        add     x10, x5, #0x100, lsl #12+        sbfx    x10, x10, #21, #21+        add     x11, x5, x11+        asr     x11, x11, #42+        mul     x6, x8, x2+        mul     x7, x9, x3+        mul     x2, x10, x2+        mul     x3, x11, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #21, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #42+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #21, #21+        add     x15, x5, x15+        asr     x15, x15, #42+        mul     x6, x12, x2+        mul     x7, x13, x3+        mul     x2, x14, x2+        mul     x3, x15, x3+        add     x4, x6, x7+        add     x5, x2, x3+        asr     x2, x4, #20+        asr     x3, x5, #20+        and     x4, x2, #0xfffff+        orr     x4, x4, #0xfffffe0000000000+        and     x5, x3, #0xfffff+        orr     x5, x5, #0xc000000000000000+        tst     x5, #0x1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        mul     x2, x12, x8+        mul     x3, x12, x9+        mul     x6, x14, x8+        mul     x7, x14, x9+        madd    x8, x13, x10, x2+        madd    x9, x13, x11, x3+        madd    x16, x15, x10, x6+        madd    x17, x15, x11, x7+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        tst     x5, #0x2+        asr     x5, x5, #1+        csel    x6, x4, xzr, ne+        ccmp    x1, xzr, #0x8, ne+        cneg    x1, x1, ge+        cneg    x6, x6, ge+        csel    x4, x5, x4, ge+        add     x5, x5, x6+        add     x1, x1, #0x2+        asr     x5, x5, #1+        add     x12, x4, #0x100, lsl #12+        sbfx    x12, x12, #22, #21+        mov     x15, #0x100000+        add     x15, x15, x15, lsl #21+        add     x13, x4, x15+        asr     x13, x13, #43+        add     x14, x5, #0x100, lsl #12+        sbfx    x14, x14, #22, #21+        add     x15, x5, x15+        asr     x15, x15, #43+        mneg    x2, x12, x8+        mneg    x3, x12, x9+        mneg    x4, x14, x8+        mneg    x5, x14, x9+        msub    x10, x13, x16, x2+        msub    x11, x13, x17, x3+        msub    x12, x15, x16, x4+        msub    x13, x15, x17, x5+        mov     x22, x1+        subs    x21, x21, #0x1+        bne     Lp256_scalarmulbase_alt_inv_loop+        ldr     x0, [sp]+        ldr     x1, [sp, #48]+        mul     x0, x0, x10+        madd    x1, x1, x11, x0+        asr     x0, x1, #63+        cmp     x10, xzr+        csetm   x14, mi+        cneg    x10, x10, mi+        eor     x14, x14, x0+        cmp     x11, xzr+        csetm   x15, mi+        cneg    x11, x11, mi+        eor     x15, x15, x0+        cmp     x12, xzr+        csetm   x16, mi+        cneg    x12, x12, mi+        eor     x16, x16, x0+        cmp     x13, xzr+        csetm   x17, mi+        cneg    x13, x13, mi+        eor     x17, x17, x0+        and     x0, x10, x14+        and     x1, x11, x15+        add     x9, x0, x1+        ldr     x7, [sp, #96]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x4, x9, x0+        adc     x2, xzr, x1+        ldr     x8, [sp, #128]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x4, x4, x0+        str     x4, [sp, #96]+        adc     x2, x2, x1+        ldr     x7, [sp, #104]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x2, x2, x0+        adc     x6, xzr, x1+        ldr     x8, [sp, #136]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x2, x2, x0+        str     x2, [sp, #104]+        adc     x6, x6, x1+        ldr     x7, [sp, #112]+        eor     x1, x7, x14+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x6, x6, x0+        adc     x5, xzr, x1+        ldr     x8, [sp, #144]+        eor     x1, x8, x15+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x6, x6, x0+        str     x6, [sp, #112]+        adc     x5, x5, x1+        ldr     x7, [sp, #120]+        eor     x1, x7, x14+        and     x3, x14, x10+        neg     x3, x3+        mul     x0, x1, x10+        umulh   x1, x1, x10+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldr     x8, [sp, #152]+        eor     x1, x8, x15+        and     x0, x15, x11+        sub     x3, x3, x0+        mul     x0, x1, x11+        umulh   x1, x1, x11+        adds    x5, x5, x0+        adc     x3, x3, x1+        ldp     x0, x1, [sp, #96]+        ldr     x2, [sp, #112]+        mov     x14, #0xe000000000000000+        adds    x0, x0, x14+        sbcs    x1, x1, xzr+        mov     x11, #0x1fffffff+        adcs    x2, x2, x11+        mov     x10, #0x2000000000000000+        adcs    x5, x5, x10+        mov     x14, #0x1fffffffe0000000+        adc     x3, x3, x14+        lsl     x11, x0, #32+        subs    x14, x0, x11+        lsr     x10, x0, #32+        sbc     x0, x0, x10+        adds    x1, x1, x11+        adcs    x2, x2, x10+        adcs    x5, x5, x14+        adcs    x3, x3, x0+        mov     x14, #0xffffffffffffffff+        mov     x11, #0xffffffff+        mov     x10, #0xffffffff00000001+        csel    x14, x14, xzr, cs+        csel    x11, x11, xzr, cs+        csel    x10, x10, xzr, cs+        subs    x1, x1, x14+        sbcs    x2, x2, x11+        sbcs    x5, x5, xzr+        sbc     x3, x3, x10+        mov     x10, #0xffffffffffffffff+        subs    x10, x1, x10+        mov     x11, #0xffffffff+        sbcs    x11, x2, x11+        mov     x13, #0xffffffff00000001+        sbcs    x12, x5, xzr+        sbcs    x13, x3, x13+        csel    x10, x1, x10, cc+        csel    x11, x2, x11, cc+        csel    x12, x5, x12, cc+        csel    x13, x3, x13, cc+        stp     x10, x11, [x20]+        stp     x12, x13, [x20, #16]+        CFI_INC_SP(160)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_alt_local_inv_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_alt_local_montmul_p256)++Lp256_scalarmulbase_alt_local_montmul_p256:+        CFI_START+        ldp     x3, x4, [x1]+        ldp     x7, x8, [x2]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x2, #16]+        mul     x11, x3, x9+        umulh   x15, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x16, x3, x10+        adcs    x15, x15, x11+        adc     x16, x16, xzr+        ldp     x5, x6, [x1, #16]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x15, x15, x11+        mul     x11, x4, x10+        adcs    x16, x16, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x15, x15, x11+        umulh   x11, x4, x9+        adcs    x16, x16, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x15, x15, x11+        mul     x11, x5, x9+        adcs    x16, x16, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x15, x15, x11+        umulh   x11, x5, x8+        adcs    x16, x16, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x15, x15, x11+        mul     x11, x6, x8+        adcs    x16, x16, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x15, x15, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x16, x16, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x15, x15, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x15, x15, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x15, lsl #32+        lsr     x11, x15, #32+        adcs    x13, x13, x11+        mul     x11, x15, x10+        umulh   x15, x15, x10+        adcs    x14, x14, x11+        adc     x15, x15, xzr+        adds    x12, x12, x16+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x15, x15, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x16, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x15, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x16, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x15, x15, x5, cc+        stp     x12, x13, [x0]+        stp     x14, x15, [x0, #16]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_alt_local_montmul_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_alt_local_montsqr_p256)++Lp256_scalarmulbase_alt_local_montsqr_p256:+        CFI_START+        ldp     x2, x3, [x1]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [x1, #16]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        mov     x5, #0xffffffff00000001+        adds    x9, x9, x8, lsl #32+        lsr     x2, x8, #32+        adcs    x10, x10, x2+        mul     x2, x8, x5+        umulh   x8, x8, x5+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x2, x9, #32+        adcs    x11, x11, x2+        mul     x2, x9, x5+        umulh   x9, x9, x5+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x2, x10, #32+        adcs    x8, x8, x2+        mul     x2, x10, x5+        umulh   x10, x10, x5+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x2, x11, #32+        adcs    x9, x9, x2+        mul     x2, x11, x5+        umulh   x11, x11, x5+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        cset    x2, cs+        mov     x3, #0xffffffff+        adds    x12, x8, #0x1+        sbcs    x13, x9, x3+        sbcs    x14, x10, xzr+        sbcs    x7, x11, x5+        sbcs    xzr, x2, xzr+        csel    x8, x8, x12, cc+        csel    x9, x9, x13, cc+        csel    x10, x10, x14, cc+        csel    x11, x11, x7, cc+        stp     x8, x9, [x0]+        stp     x10, x11, [x0, #16]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_alt_local_montsqr_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_alt_local_p256_montjmixadd)++Lp256_scalarmulbase_alt_local_p256_montjmixadd:+        CFI_START+        CFI_DEC_SP(192)+        mov     x15, x0+        mov     x16, x1+        mov     x17, x2+        ldp     x2, x3, [x16, #64]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [x16, #80]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x8, x3+        umulh   x8, x8, x3+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x9, x3+        umulh   x9, x9, x3+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x10, x3+        umulh   x10, x10, x3+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x11, x3+        umulh   x11, x11, x3+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        mov     x2, #0xffffffffffffffff+        csel    x2, xzr, x2, cc+        mov     x3, #0xffffffff+        csel    x3, xzr, x3, cc+        mov     x5, #0xffffffff00000001+        csel    x5, xzr, x5, cc+        subs    x8, x8, x2+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbc     x11, x11, x5+        stp     x8, x9, [sp]+        stp     x10, x11, [sp, #16]+        ldp     x3, x4, [x16, #64]+        ldp     x7, x8, [x17, #32]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x17, #48]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [x16, #80]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #32]+        stp     x14, x0, [sp, #48]+        ldp     x3, x4, [sp]+        ldp     x7, x8, [x17]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x17, #16]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #16]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #64]+        stp     x14, x0, [sp, #80]+        ldp     x3, x4, [sp]+        ldp     x7, x8, [sp, #32]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #48]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #16]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #32]+        stp     x14, x0, [sp, #48]+        ldp     x5, x6, [sp, #64]+        ldp     x4, x3, [x16]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #80]+        ldp     x4, x3, [x16, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #160]+        stp     x7, x8, [sp, #176]+        ldp     x5, x6, [sp, #32]+        ldp     x4, x3, [x16, #32]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #48]+        ldp     x4, x3, [x16, #48]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #32]+        stp     x7, x8, [sp, #48]+        ldp     x2, x3, [sp, #160]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [sp, #176]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x8, x3+        umulh   x8, x8, x3+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x9, x3+        umulh   x9, x9, x3+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x10, x3+        umulh   x10, x10, x3+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x11, x3+        umulh   x11, x11, x3+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        mov     x2, #0xffffffffffffffff+        csel    x2, xzr, x2, cc+        mov     x3, #0xffffffff+        csel    x3, xzr, x3, cc+        mov     x5, #0xffffffff00000001+        csel    x5, xzr, x5, cc+        subs    x8, x8, x2+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbc     x11, x11, x5+        stp     x8, x9, [sp, #96]+        stp     x10, x11, [sp, #112]+        ldp     x2, x3, [sp, #32]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [sp, #48]+        mul     x11, x2, x5+        umulh   x12, x2, x5+        mul     x6, x2, x4+        umulh   x7, x2, x4+        adds    x10, x10, x6+        adcs    x11, x11, x7+        mul     x6, x3, x4+        umulh   x7, x3, x4+        adc     x7, x7, xzr+        adds    x11, x11, x6+        mul     x13, x4, x5+        umulh   x14, x4, x5+        adcs    x12, x12, x7+        mul     x6, x3, x5+        umulh   x7, x3, x5+        adc     x7, x7, xzr+        adds    x12, x12, x6+        adcs    x13, x13, x7+        adc     x14, x14, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        cset    x7, cs+        umulh   x6, x2, x2+        mul     x8, x2, x2+        adds    x9, x9, x6+        mul     x6, x3, x3+        adcs    x10, x10, x6+        umulh   x6, x3, x3+        adcs    x11, x11, x6+        mul     x6, x4, x4+        adcs    x12, x12, x6+        umulh   x6, x4, x4+        adcs    x13, x13, x6+        mul     x6, x5, x5+        adcs    x14, x14, x6+        umulh   x6, x5, x5+        adc     x7, x7, x6+        adds    x9, x9, x8, lsl #32+        lsr     x3, x8, #32+        adcs    x10, x10, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x8, x3+        umulh   x8, x8, x3+        adcs    x11, x11, x2+        adc     x8, x8, xzr+        adds    x10, x10, x9, lsl #32+        lsr     x3, x9, #32+        adcs    x11, x11, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x9, x3+        umulh   x9, x9, x3+        adcs    x8, x8, x2+        adc     x9, x9, xzr+        adds    x11, x11, x10, lsl #32+        lsr     x3, x10, #32+        adcs    x8, x8, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x10, x3+        umulh   x10, x10, x3+        adcs    x9, x9, x2+        adc     x10, x10, xzr+        adds    x8, x8, x11, lsl #32+        lsr     x3, x11, #32+        adcs    x9, x9, x3+        mov     x3, #0xffffffff00000001+        mul     x2, x11, x3+        umulh   x11, x11, x3+        adcs    x10, x10, x2+        adc     x11, x11, xzr+        adds    x8, x8, x12+        adcs    x9, x9, x13+        adcs    x10, x10, x14+        adcs    x11, x11, x7+        cset    x2, cs+        mov     x3, #0xffffffff+        mov     x5, #0xffffffff00000001+        adds    x12, x8, #0x1+        sbcs    x13, x9, x3+        sbcs    x14, x10, xzr+        sbcs    x7, x11, x5+        sbcs    xzr, x2, xzr+        csel    x8, x8, x12, cc+        csel    x9, x9, x13, cc+        csel    x10, x10, x14, cc+        csel    x11, x11, x7, cc+        stp     x8, x9, [sp]+        stp     x10, x11, [sp, #16]+        ldp     x3, x4, [sp, #96]+        ldp     x7, x8, [x16]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x16, #16]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #112]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #128]+        stp     x14, x0, [sp, #144]+        ldp     x3, x4, [sp, #96]+        ldp     x7, x8, [sp, #64]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #80]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #112]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #64]+        stp     x14, x0, [sp, #80]+        ldp     x5, x6, [sp]+        ldp     x4, x3, [sp, #128]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #16]+        ldp     x4, x3, [sp, #144]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp]+        stp     x7, x8, [sp, #16]+        ldp     x5, x6, [sp, #64]+        ldp     x4, x3, [sp, #128]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #80]+        ldp     x4, x3, [sp, #144]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #96]+        stp     x7, x8, [sp, #112]+        ldp     x3, x4, [sp, #160]+        ldp     x7, x8, [x16, #64]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x16, #80]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #176]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #160]+        stp     x14, x0, [sp, #176]+        ldp     x5, x6, [sp]+        ldp     x4, x3, [sp, #64]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #16]+        ldp     x4, x3, [sp, #80]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp]+        stp     x7, x8, [sp, #16]+        ldp     x5, x6, [sp, #128]+        ldp     x4, x3, [sp]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #144]+        ldp     x4, x3, [sp, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #128]+        stp     x7, x8, [sp, #144]+        ldp     x3, x4, [sp, #96]+        ldp     x7, x8, [x16, #32]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [x16, #48]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #112]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #96]+        stp     x14, x0, [sp, #112]+        ldp     x3, x4, [sp, #32]+        ldp     x7, x8, [sp, #128]+        mul     x12, x3, x7+        umulh   x13, x3, x7+        mul     x11, x3, x8+        umulh   x14, x3, x8+        adds    x13, x13, x11+        ldp     x9, x10, [sp, #144]+        mul     x11, x3, x9+        umulh   x0, x3, x9+        adcs    x14, x14, x11+        mul     x11, x3, x10+        umulh   x1, x3, x10+        adcs    x0, x0, x11+        adc     x1, x1, xzr+        ldp     x5, x6, [sp, #48]+        mul     x11, x4, x7+        adds    x13, x13, x11+        mul     x11, x4, x8+        adcs    x14, x14, x11+        mul     x11, x4, x9+        adcs    x0, x0, x11+        mul     x11, x4, x10+        adcs    x1, x1, x11+        umulh   x3, x4, x10+        adc     x3, x3, xzr+        umulh   x11, x4, x7+        adds    x14, x14, x11+        umulh   x11, x4, x8+        adcs    x0, x0, x11+        umulh   x11, x4, x9+        adcs    x1, x1, x11+        adc     x3, x3, xzr+        mul     x11, x5, x7+        adds    x14, x14, x11+        mul     x11, x5, x8+        adcs    x0, x0, x11+        mul     x11, x5, x9+        adcs    x1, x1, x11+        mul     x11, x5, x10+        adcs    x3, x3, x11+        umulh   x4, x5, x10+        adc     x4, x4, xzr+        umulh   x11, x5, x7+        adds    x0, x0, x11+        umulh   x11, x5, x8+        adcs    x1, x1, x11+        umulh   x11, x5, x9+        adcs    x3, x3, x11+        adc     x4, x4, xzr+        mul     x11, x6, x7+        adds    x0, x0, x11+        mul     x11, x6, x8+        adcs    x1, x1, x11+        mul     x11, x6, x9+        adcs    x3, x3, x11+        mul     x11, x6, x10+        adcs    x4, x4, x11+        umulh   x5, x6, x10+        adc     x5, x5, xzr+        mov     x10, #0xffffffff00000001+        adds    x13, x13, x12, lsl #32+        lsr     x11, x12, #32+        adcs    x14, x14, x11+        mul     x11, x12, x10+        umulh   x12, x12, x10+        adcs    x0, x0, x11+        adc     x12, x12, xzr+        umulh   x11, x6, x7+        adds    x1, x1, x11+        umulh   x11, x6, x8+        adcs    x3, x3, x11+        umulh   x11, x6, x9+        adcs    x4, x4, x11+        adc     x5, x5, xzr+        adds    x14, x14, x13, lsl #32+        lsr     x11, x13, #32+        adcs    x0, x0, x11+        mul     x11, x13, x10+        umulh   x13, x13, x10+        adcs    x12, x12, x11+        adc     x13, x13, xzr+        adds    x0, x0, x14, lsl #32+        lsr     x11, x14, #32+        adcs    x12, x12, x11+        mul     x11, x14, x10+        umulh   x14, x14, x10+        adcs    x13, x13, x11+        adc     x14, x14, xzr+        adds    x12, x12, x0, lsl #32+        lsr     x11, x0, #32+        adcs    x13, x13, x11+        mul     x11, x0, x10+        umulh   x0, x0, x10+        adcs    x14, x14, x11+        adc     x0, x0, xzr+        adds    x12, x12, x1+        adcs    x13, x13, x3+        adcs    x14, x14, x4+        adcs    x0, x0, x5+        cset    x8, cs+        mov     x11, #0xffffffff+        adds    x1, x12, #0x1+        sbcs    x3, x13, x11+        sbcs    x4, x14, xzr+        sbcs    x5, x0, x10+        sbcs    xzr, x8, xzr+        csel    x12, x12, x1, cc+        csel    x13, x13, x3, cc+        csel    x14, x14, x4, cc+        csel    x0, x0, x5, cc+        stp     x12, x13, [sp, #128]+        stp     x14, x0, [sp, #144]+        ldp     x5, x6, [sp, #128]+        ldp     x4, x3, [sp, #96]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #144]+        ldp     x4, x3, [sp, #112]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        csetm   x3, cc+        adds    x5, x5, x3+        mov     x4, #0xffffffff+        and     x4, x4, x3+        adcs    x6, x6, x4+        adcs    x7, x7, xzr+        mov     x4, #0xffffffff00000001+        and     x4, x4, x3+        adc     x8, x8, x4+        stp     x5, x6, [sp, #128]+        stp     x7, x8, [sp, #144]+        ldp     x0, x1, [x16, #64]+        ldp     x2, x3, [x16, #80]+        orr     x4, x0, x1+        orr     x5, x2, x3+        orr     x4, x4, x5+        cmp     x4, xzr+        ldp     x0, x1, [sp]+        ldp     x12, x13, [x17]+        csel    x0, x0, x12, ne+        csel    x1, x1, x13, ne+        ldp     x2, x3, [sp, #16]+        ldp     x12, x13, [x17, #16]+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        ldp     x4, x5, [sp, #128]+        ldp     x12, x13, [x17, #32]+        csel    x4, x4, x12, ne+        csel    x5, x5, x13, ne+        ldp     x6, x7, [sp, #144]+        ldp     x12, x13, [x17, #48]+        csel    x6, x6, x12, ne+        csel    x7, x7, x13, ne+        ldp     x8, x9, [sp, #160]+        mov     x12, #0x1+        mov     x13, #0xffffffff00000000+        csel    x8, x8, x12, ne+        csel    x9, x9, x13, ne+        ldp     x10, x11, [sp, #176]+        mov     x12, #0xffffffffffffffff+        mov     x13, #0xfffffffe+        csel    x10, x10, x12, ne+        csel    x11, x11, x13, ne+        stp     x0, x1, [x15]+        stp     x2, x3, [x15, #16]+        stp     x4, x5, [x15, #32]+        stp     x6, x7, [x15, #48]+        stp     x8, x9, [x15, #64]+        stp     x10, x11, [x15, #80]+        CFI_INC_SP(192)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_alt_local_p256_montjmixadd)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/p384_montjscalarmul.S view
@@ -0,0 +1,10004 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Montgomery-Jacobian form scalar multiplication for P-384+// Input scalar[6], point[18]; output res[18]+//+// extern void p384_montjscalarmul+//   (uint64_t res[static 18],+//    const uint64_t scalar[static 6],+//    const uint64_t point[static 18]);+//+// This function is a variant of its affine point version p384_scalarmul.+// Here, input and output points are assumed to be in Jacobian form with+// their coordinates in the Montgomery domain. Thus, if priming indicates+// Montgomery form, x' = (2^384 * x) mod p_384 etc., each point argument+// is a triple (x',y',z') representing the affine point (x/z^2,y/z^3) when+// z' is nonzero or the point at infinity (group identity) if z' = 0.+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-384, returns a representation of n * P. If the result is the+// point at infinity (either because the input point was or because the+// scalar was a multiple of p_384) then the output is guaranteed to+// represent the point at infinity, i.e. to have its z coordinate zero.+//+// Standard ARM ABI: X0 = res, X1 = scalar, X2 = point+// ----------------------------------------------------------------------------+++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p384_montjscalarmul)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p384_montjscalarmul)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p384_montjscalarmul)+++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 48+#define JACSIZE (3*NUMSIZE)++// Safe copies of input res and additional values in variables.++#define bf x22+#define sgn x23+#define j x24+#define res x25++// Intermediate variables on the stack.+// The table is 16 entries, each of size JACSIZE = 3 * NUMSIZE++#define scalarb sp, #(0*NUMSIZE)+#define acc sp, #(1*NUMSIZE)+#define tabent sp, #(4*NUMSIZE)++#define tab sp, #(7*NUMSIZE)++#define NSPACE 55*NUMSIZE++// Avoid using .rep for the sake of the BoringSSL/AWS-LC delocator,+// which doesn't accept repetitions, assembler macros etc.++#define selectblock(I)                            \+        cmp     bf, #(1*I) __LF                      \+        ldp     x20, x21, [x19] __LF                 \+        csel    x0, x20, x0, eq __LF                 \+        csel    x1, x21, x1, eq __LF                 \+        ldp     x20, x21, [x19, #16] __LF            \+        csel    x2, x20, x2, eq __LF                 \+        csel    x3, x21, x3, eq __LF                 \+        ldp     x20, x21, [x19, #32] __LF            \+        csel    x4, x20, x4, eq __LF                 \+        csel    x5, x21, x5, eq __LF                 \+        ldp     x20, x21, [x19, #48] __LF            \+        csel    x6, x20, x6, eq __LF                 \+        csel    x7, x21, x7, eq __LF                 \+        ldp     x20, x21, [x19, #64] __LF            \+        csel    x8, x20, x8, eq __LF                 \+        csel    x9, x21, x9, eq __LF                 \+        ldp     x20, x21, [x19, #80] __LF            \+        csel    x10, x20, x10, eq __LF               \+        csel    x11, x21, x11, eq __LF               \+        ldp     x20, x21, [x19, #96] __LF            \+        csel    x12, x20, x12, eq __LF               \+        csel    x13, x21, x13, eq __LF               \+        ldp     x20, x21, [x19, #112] __LF           \+        csel    x14, x20, x14, eq __LF               \+        csel    x15, x21, x15, eq __LF               \+        ldp     x20, x21, [x19, #128] __LF           \+        csel    x16, x20, x16, eq __LF               \+        csel    x17, x21, x17, eq __LF               \+        add     x19, x19, #JACSIZE++// Loading large constants++#define movbig(nn,n3,n2,n1,n0)                                      \+        movz    nn, n0 __LF                                            \+        movk    nn, n1, lsl #16 __LF                                   \+        movk    nn, n2, lsl #32 __LF                                   \+        movk    nn, n3, lsl #48++S2N_BN_SYMBOL(p384_montjscalarmul):+        CFI_START++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x30)+        CFI_DEC_SP(NSPACE)++// Preserve the "res" input argument; others get processed early.++        mov     res, x0++// Reduce the input scalar mod n_384, i.e. conditionally subtract n_384.+// Store it to "scalarb".++        ldp     x3, x4, [x1]+        movbig(x15, #0xecec, #0x196a, #0xccc5, #0x2973)+        ldp     x5, x6, [x1, #16]+        movbig(x16, #0x581a, #0x0db2, #0x48b0, #0xa77a)+        ldp     x7, x8, [x1, #32]+        movbig(x17, #0xc763, #0x4d81, #0xf437, #0x2ddf)++        subs    x9, x3, x15+        sbcs    x10, x4, x16+        sbcs    x11, x5, x17+        adcs    x12, x6, xzr+        adcs    x13, x7, xzr+        adcs    x14, x8, xzr++        csel    x3, x3, x9, cc+        csel    x4, x4, x10, cc+        csel    x5, x5, x11, cc+        csel    x6, x6, x12, cc+        csel    x7, x7, x13, cc+        csel    x8, x8, x14, cc++        stp     x3, x4, [scalarb]+        stp     x5, x6, [scalarb+16]+        stp     x7, x8, [scalarb+32]++// Set the tab[0] table entry to the input point = 1 * P++        ldp     x10, x11, [x2]+        stp     x10, x11, [tab]+        ldp     x12, x13, [x2, #16]+        stp     x12, x13, [tab+16]+        ldp     x14, x15, [x2, #32]+        stp     x14, x15, [tab+32]++        ldp     x10, x11, [x2, #48]+        stp     x10, x11, [tab+48]+        ldp     x12, x13, [x2, #64]+        stp     x12, x13, [tab+64]+        ldp     x14, x15, [x2, #80]+        stp     x14, x15, [tab+80]++        ldp     x10, x11, [x2, #96]+        stp     x10, x11, [tab+96]+        ldp     x12, x13, [x2, #112]+        stp     x12, x13, [tab+112]+        ldp     x14, x15, [x2, #128]+        stp     x14, x15, [tab+128]++// Compute and record tab[1] = 2 * p, ..., tab[15] = 16 * P++        add     x0, tab+JACSIZE*1+        add     x1, tab+        CFI_BL(Lp384_montjscalarmul_p384_montjdouble)++        add     x0, tab+JACSIZE*2+        add     x1, tab+JACSIZE*1+        add     x2, tab+        CFI_BL(Lp384_montjscalarmul_p384_montjadd)++        add     x0, tab+JACSIZE*3+        add     x1, tab+JACSIZE*1+        CFI_BL(Lp384_montjscalarmul_p384_montjdouble)++        add     x0, tab+JACSIZE*4+        add     x1, tab+JACSIZE*3+        add     x2, tab+        CFI_BL(Lp384_montjscalarmul_p384_montjadd)++        add     x0, tab+JACSIZE*5+        add     x1, tab+JACSIZE*2+        CFI_BL(Lp384_montjscalarmul_p384_montjdouble)++        add     x0, tab+JACSIZE*6+        add     x1, tab+JACSIZE*5+        add     x2, tab+        CFI_BL(Lp384_montjscalarmul_p384_montjadd)++        add     x0, tab+JACSIZE*7+        add     x1, tab+JACSIZE*3+        CFI_BL(Lp384_montjscalarmul_p384_montjdouble)++        add     x0, tab+JACSIZE*8+        add     x1, tab+JACSIZE*7+        add     x2, tab+        CFI_BL(Lp384_montjscalarmul_p384_montjadd)++        add     x0, tab+JACSIZE*9+        add     x1, tab+JACSIZE*4+        CFI_BL(Lp384_montjscalarmul_p384_montjdouble)++        add     x0, tab+JACSIZE*10+        add     x1, tab+JACSIZE*9+        add     x2, tab+        CFI_BL(Lp384_montjscalarmul_p384_montjadd)++        add     x0, tab+JACSIZE*11+        add     x1, tab+JACSIZE*5+        CFI_BL(Lp384_montjscalarmul_p384_montjdouble)++        add     x0, tab+JACSIZE*12+        add     x1, tab+JACSIZE*11+        add     x2, tab+        CFI_BL(Lp384_montjscalarmul_p384_montjadd)++        add     x0, tab+JACSIZE*13+        add     x1, tab+JACSIZE*6+        CFI_BL(Lp384_montjscalarmul_p384_montjdouble)++        add     x0, tab+JACSIZE*14+        add     x1, tab+JACSIZE*13+        add     x2, tab+        CFI_BL(Lp384_montjscalarmul_p384_montjadd)++        add     x0, tab+JACSIZE*15+        add     x1, tab+JACSIZE*7+        CFI_BL(Lp384_montjscalarmul_p384_montjdouble)++// Add the recoding constant sum_i(16 * 32^i) to the scalar to allow signed+// digits. The digits of the constant, in lowest-to-highest order, are as+// follows; they are generated dynamically since none is a simple ARM load.+//+// 0x0842108421084210+// 0x1084210842108421+// 0x2108421084210842+// 0x4210842108421084+// 0x8421084210842108+// 0x0842108421084210++        ldp     x0, x1, [scalarb]+        ldp     x2, x3, [scalarb+16]+        ldp     x4, x5, [scalarb+32]+        movbig(x8, #0x1084, #0x2108, #0x4210, #0x8421)+        adds    x0, x0, x8, lsr #1+        adcs    x1, x1, x8+        lsl     x8, x8, #1+        adcs    x2, x2, x8+        lsl     x8, x8, #1+        adcs    x3, x3, x8+        lsl     x8, x8, #1+        adcs    x4, x4, x8+        lsr     x8, x8, #4+        adcs    x5, x5, x8+        cset    x6, cs++// Record the top bitfield then shift the whole scalar left 4 bits+// to align the top of the next bitfield with the MSB (bits 379..383).++        extr    bf, x6, x5, #60+        extr    x5, x5, x4, #60+        extr    x4, x4, x3, #60+        extr    x3, x3, x2, #60+        extr    x2, x2, x1, #60+        extr    x1, x1, x0, #60+        lsl     x0, x0, #4+        stp     x0, x1, [scalarb]+        stp     x2, x3, [scalarb+16]+        stp     x4, x5, [scalarb+32]++// Initialize the accumulator to the corresponding entry using constant-time+// lookup in the table. This top digit, uniquely, is not recoded so there is+// no sign adjustment to make.++        mov     x0, xzr+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, xzr+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        mov     x9, xzr+        mov     x10, xzr+        mov     x11, xzr+        mov     x12, xzr+        mov     x13, xzr+        mov     x14, xzr+        mov     x15, xzr+        mov     x16, xzr+        mov     x17, xzr++        add     x19, tab++        selectblock(1)+        selectblock(2)+        selectblock(3)+        selectblock(4)+        selectblock(5)+        selectblock(6)+        selectblock(7)+        selectblock(8)+        selectblock(9)+        selectblock(10)+        selectblock(11)+        selectblock(12)+        selectblock(13)+        selectblock(14)+        selectblock(15)+        selectblock(16)++        stp     x0, x1, [acc]+        stp     x2, x3, [acc+16]+        stp     x4, x5, [acc+32]+        stp     x6, x7, [acc+48]+        stp     x8, x9, [acc+64]+        stp     x10, x11, [acc+80]+        stp     x12, x13, [acc+96]+        stp     x14, x15, [acc+112]+        stp     x16, x17, [acc+128]++        mov     j, #380++// Main loop over size-5 bitfields: double 5 times then add signed digit+// At each stage we shift the scalar left by 5 bits so we can simply pick+// the top 5 bits as the bitfield, saving some fiddle over indexing.++Lp384_montjscalarmul_mainloop:+        sub     j, j, #5++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp384_montjscalarmul_p384_montjdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp384_montjscalarmul_p384_montjdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp384_montjscalarmul_p384_montjdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp384_montjscalarmul_p384_montjdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp384_montjscalarmul_p384_montjdouble)++// Choose the bitfield and adjust it to sign and magnitude++        ldp     x0, x1, [scalarb]+        ldp     x2, x3, [scalarb+16]+        ldp     x4, x5, [scalarb+32]+        lsr     bf, x5, #59+        extr    x5, x5, x4, #59+        extr    x4, x4, x3, #59+        extr    x3, x3, x2, #59+        extr    x2, x2, x1, #59+        extr    x1, x1, x0, #59+        lsl     x0, x0, #5+        stp     x0, x1, [scalarb]+        stp     x2, x3, [scalarb+16]+        stp     x4, x5, [scalarb+32]++        subs    bf, bf, #16+        cset    sgn, lo                 // sgn = sign of digit (1 = negative)+        cneg    bf, bf, lo              // bf = absolute value of digit++// Conditionally select the table entry tab[i-1] = i * P in constant time++        mov     x0, xzr+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, xzr+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        mov     x9, xzr+        mov     x10, xzr+        mov     x11, xzr+        mov     x12, xzr+        mov     x13, xzr+        mov     x14, xzr+        mov     x15, xzr+        mov     x16, xzr+        mov     x17, xzr++        add     x19, tab++        selectblock(1)+        selectblock(2)+        selectblock(3)+        selectblock(4)+        selectblock(5)+        selectblock(6)+        selectblock(7)+        selectblock(8)+        selectblock(9)+        selectblock(10)+        selectblock(11)+        selectblock(12)+        selectblock(13)+        selectblock(14)+        selectblock(15)+        selectblock(16)++// Store it to "tabent" with the y coordinate optionally negated.+// This is done carefully to give coordinates < p_384 even in+// the degenerate case y = 0 (when z = 0 for points on the curve).++        stp     x0, x1, [tabent]+        stp     x2, x3, [tabent+16]+        stp     x4, x5, [tabent+32]++        stp     x12, x13, [tabent+96]+        stp     x14, x15, [tabent+112]+        stp     x16, x17, [tabent+128]++        mov     x0, #0x00000000ffffffff+        subs    x0, x0, x6+        orr     x12, x6, x7+        mov     x1, #0xffffffff00000000+        sbcs    x1, x1, x7+        orr     x13, x8, x9+        mov     x2, #0xfffffffffffffffe+        sbcs    x2, x2, x8+        orr     x14, x10, x11+        mov     x5, #0xffffffffffffffff+        sbcs    x3, x5, x9+        orr     x12, x12, x13+        sbcs    x4, x5, x10+        orr     x12, x12, x14+        sbcs    x5, x5, x11++        cmp     sgn, xzr+        ccmp    x12, xzr, #4, ne++        csel    x6, x0, x6, ne+        csel    x7, x1, x7, ne+        csel    x8, x2, x8, ne+        csel    x9, x3, x9, ne+        csel    x10, x4, x10, ne+        csel    x11, x5, x11, ne++        stp     x6, x7, [tabent+48]+        stp     x8, x9, [tabent+64]+        stp     x10, x11, [tabent+80]++// Add to the accumulator++        add     x0, acc+        add     x1, acc+        add     x2, tabent+        CFI_BL(Lp384_montjscalarmul_p384_montjadd)++        cbnz    j, Lp384_montjscalarmul_mainloop++// That's the end of the main loop, and we just need to copy the+// result in "acc" to the output.++        ldp     x0, x1, [acc]+        stp     x0, x1, [res]+        ldp     x0, x1, [acc+16]+        stp     x0, x1, [res, #16]+        ldp     x0, x1, [acc+32]+        stp     x0, x1, [res, #32]+        ldp     x0, x1, [acc+48]+        stp     x0, x1, [res, #48]+        ldp     x0, x1, [acc+64]+        stp     x0, x1, [res, #64]+        ldp     x0, x1, [acc+80]+        stp     x0, x1, [res, #80]+        ldp     x0, x1, [acc+96]+        stp     x0, x1, [res, #96]+        ldp     x0, x1, [acc+112]+        stp     x0, x1, [res, #112]+        ldp     x0, x1, [acc+128]+        stp     x0, x1, [res, #128]++// Restore stack and registers and return++        CFI_INC_SP(NSPACE)+        CFI_POP2(x25,x30)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p384_montjscalarmul)++// Local copies of subroutines, complete clones at the moment++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp384_montjscalarmul_p384_montjadd)++Lp384_montjscalarmul_p384_montjadd:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x26)+        CFI_PUSH1Z(x27)+        CFI_DEC_SP(384)+        mov     x24, x0+        mov     x25, x1+        mov     x26, x2+        mov     x0, sp+        ldr     q1, [x25, #96]+        ldp     x9, x2, [x25, #96]+        ldr     q0, [x25, #96]+        ldp     x4, x6, [x25, #112]+        rev64   v21.4s, v1.4s+        uzp2    v28.4s, v1.4s, v1.4s+        umulh   x7, x9, x2+        xtn     v17.2s, v1.2d+        mul     v27.4s, v21.4s, v0.4s+        ldr     q20, [x25, #128]+        xtn     v30.2s, v0.2d+        ldr     q1, [x25, #128]+        uzp2    v31.4s, v0.4s, v0.4s+        ldp     x5, x10, [x25, #128]+        umulh   x8, x9, x4+        uaddlp  v3.2d, v27.4s+        umull   v16.2d, v30.2s, v17.2s+        mul     x16, x9, x4+        umull   v27.2d, v30.2s, v28.2s+        shrn    v0.2s, v20.2d, #32+        xtn     v7.2s, v20.2d+        shl     v20.2d, v3.2d, #32+        umull   v3.2d, v31.2s, v28.2s+        mul     x3, x2, x4+        umlal   v20.2d, v30.2s, v17.2s+        umull   v22.2d, v7.2s, v0.2s+        usra    v27.2d, v16.2d, #32+        umulh   x11, x2, x4+        movi    v21.2d, #0xffffffff+        uzp2    v28.4s, v1.4s, v1.4s+        adds    x15, x16, x7+        and     v5.16b, v27.16b, v21.16b+        adcs    x3, x3, x8+        usra    v3.2d, v27.2d, #32+        dup     v29.2d, x6+        adcs    x16, x11, xzr+        mov     x14, v20.d[0]+        umlal   v5.2d, v31.2s, v17.2s+        mul     x8, x9, x2+        mov     x7, v20.d[1]+        shl     v19.2d, v22.2d, #33+        xtn     v25.2s, v29.2d+        rev64   v31.4s, v1.4s+        lsl     x13, x14, #32+        uzp2    v6.4s, v29.4s, v29.4s+        umlal   v19.2d, v7.2s, v7.2s+        usra    v3.2d, v5.2d, #32+        adds    x1, x8, x8+        umulh   x8, x4, x4+        add     x12, x13, x14+        mul     v17.4s, v31.4s, v29.4s+        xtn     v4.2s, v1.2d+        adcs    x14, x15, x15+        lsr     x13, x12, #32+        adcs    x15, x3, x3+        umull   v31.2d, v25.2s, v28.2s+        adcs    x11, x16, x16+        umull   v21.2d, v25.2s, v4.2s+        mov     x17, v3.d[0]+        umull   v18.2d, v6.2s, v28.2s+        adc     x16, x8, xzr+        uaddlp  v16.2d, v17.4s+        movi    v1.2d, #0xffffffff+        subs    x13, x13, x12+        usra    v31.2d, v21.2d, #32+        sbc     x8, x12, xzr+        adds    x17, x17, x1+        mul     x1, x4, x4+        shl     v28.2d, v16.2d, #32+        mov     x3, v3.d[1]+        adcs    x14, x7, x14+        extr    x7, x8, x13, #32+        adcs    x13, x3, x15+        and     v3.16b, v31.16b, v1.16b+        adcs    x11, x1, x11+        lsr     x1, x8, #32+        umlal   v3.2d, v6.2s, v4.2s+        usra    v18.2d, v31.2d, #32+        adc     x3, x16, xzr+        adds    x1, x1, x12+        umlal   v28.2d, v25.2s, v4.2s+        adc     x16, xzr, xzr+        subs    x15, x17, x7+        sbcs    x7, x14, x1+        lsl     x1, x15, #32+        sbcs    x16, x13, x16+        add     x8, x1, x15+        usra    v18.2d, v3.2d, #32+        sbcs    x14, x11, xzr+        lsr     x1, x8, #32+        sbcs    x17, x3, xzr+        sbc     x11, x12, xzr+        subs    x13, x1, x8+        umulh   x12, x4, x10+        sbc     x1, x8, xzr+        extr    x13, x1, x13, #32+        lsr     x1, x1, #32+        adds    x15, x1, x8+        adc     x1, xzr, xzr+        subs    x7, x7, x13+        sbcs    x13, x16, x15+        lsl     x3, x7, #32+        umulh   x16, x2, x5+        sbcs    x15, x14, x1+        add     x7, x3, x7+        sbcs    x3, x17, xzr+        lsr     x1, x7, #32+        sbcs    x14, x11, xzr+        sbc     x11, x8, xzr+        subs    x8, x1, x7+        sbc     x1, x7, xzr+        extr    x8, x1, x8, #32+        lsr     x1, x1, #32+        adds    x1, x1, x7+        adc     x17, xzr, xzr+        subs    x13, x13, x8+        umulh   x8, x9, x6+        sbcs    x1, x15, x1+        sbcs    x15, x3, x17+        sbcs    x3, x14, xzr+        mul     x17, x2, x5+        sbcs    x11, x11, xzr+        stp     x13, x1, [x0]+        sbc     x14, x7, xzr+        mul     x7, x4, x10+        subs    x1, x9, x2+        stp     x15, x3, [x0, #16]+        csetm   x15, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        stp     x11, x14, [x0, #32]+        mul     x14, x9, x6+        adds    x17, x8, x17+        adcs    x7, x16, x7+        adc     x13, x12, xzr+        subs    x12, x5, x6+        cneg    x3, x12, cc  // cc = lo, ul, last+        cinv    x16, x15, cc  // cc = lo, ul, last+        mul     x8, x1, x3+        umulh   x1, x1, x3+        eor     x12, x8, x16+        adds    x11, x17, x14+        adcs    x3, x7, x17+        adcs    x15, x13, x7+        adc     x8, x13, xzr+        adds    x3, x3, x14+        adcs    x15, x15, x17+        adcs    x17, x8, x7+        eor     x1, x1, x16+        adc     x13, x13, xzr+        subs    x9, x9, x4+        csetm   x8, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x4, x2, x4+        cneg    x4, x4, cc  // cc = lo, ul, last+        csetm   x7, cc  // cc = lo, ul, last+        subs    x2, x10, x6+        cinv    x8, x8, cc  // cc = lo, ul, last+        cneg    x2, x2, cc  // cc = lo, ul, last+        cmn     x16, #0x1+        adcs    x11, x11, x12+        mul     x12, x9, x2+        adcs    x3, x3, x1+        adcs    x15, x15, x16+        umulh   x9, x9, x2+        adcs    x17, x17, x16+        adc     x13, x13, x16+        subs    x1, x10, x5+        cinv    x2, x7, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        eor     x9, x9, x8+        cmn     x8, #0x1+        eor     x7, x12, x8+        mul     x12, x4, x1+        adcs    x3, x3, x7+        adcs    x7, x15, x9+        adcs    x15, x17, x8+        ldp     x9, x17, [x0, #16]+        umulh   x4, x4, x1+        adc     x8, x13, x8+        cmn     x2, #0x1+        eor     x1, x12, x2+        adcs    x1, x7, x1+        ldp     x7, x16, [x0]+        eor     x12, x4, x2+        adcs    x4, x15, x12+        ldp     x15, x12, [x0, #32]+        adc     x8, x8, x2+        adds    x13, x14, x14+        umulh   x14, x5, x10+        adcs    x2, x11, x11+        adcs    x3, x3, x3+        adcs    x1, x1, x1+        adcs    x4, x4, x4+        adcs    x11, x8, x8+        adc     x8, xzr, xzr+        adds    x13, x13, x7+        adcs    x2, x2, x16+        mul     x16, x5, x10+        adcs    x3, x3, x9+        adcs    x1, x1, x17+        umulh   x5, x5, x5+        lsl     x9, x13, #32+        add     x9, x9, x13+        adcs    x4, x4, x15+        mov     x13, v28.d[1]+        adcs    x15, x11, x12+        lsr     x7, x9, #32+        adc     x11, x8, xzr+        subs    x7, x7, x9+        umulh   x10, x10, x10+        sbc     x17, x9, xzr+        extr    x7, x17, x7, #32+        lsr     x17, x17, #32+        adds    x17, x17, x9+        adc     x12, xzr, xzr+        subs    x8, x2, x7+        sbcs    x17, x3, x17+        lsl     x7, x8, #32+        sbcs    x2, x1, x12+        add     x3, x7, x8+        sbcs    x12, x4, xzr+        lsr     x1, x3, #32+        sbcs    x7, x15, xzr+        sbc     x15, x9, xzr+        subs    x1, x1, x3+        sbc     x4, x3, xzr+        lsr     x9, x4, #32+        extr    x8, x4, x1, #32+        adds    x9, x9, x3+        adc     x4, xzr, xzr+        subs    x1, x17, x8+        lsl     x17, x1, #32+        sbcs    x8, x2, x9+        sbcs    x9, x12, x4+        add     x17, x17, x1+        mov     x1, v18.d[1]+        lsr     x2, x17, #32+        sbcs    x7, x7, xzr+        mov     x12, v18.d[0]+        sbcs    x15, x15, xzr+        sbc     x3, x3, xzr+        subs    x4, x2, x17+        sbc     x2, x17, xzr+        adds    x12, x13, x12+        adcs    x16, x16, x1+        lsr     x13, x2, #32+        extr    x1, x2, x4, #32+        adc     x2, x14, xzr+        adds    x4, x13, x17+        mul     x13, x6, x6+        adc     x14, xzr, xzr+        subs    x1, x8, x1+        sbcs    x4, x9, x4+        mov     x9, v28.d[0]+        sbcs    x7, x7, x14+        sbcs    x8, x15, xzr+        sbcs    x3, x3, xzr+        sbc     x14, x17, xzr+        adds    x17, x9, x9+        adcs    x12, x12, x12+        mov     x15, v19.d[0]+        adcs    x9, x16, x16+        umulh   x6, x6, x6+        adcs    x16, x2, x2+        adc     x2, xzr, xzr+        adds    x11, x11, x8+        adcs    x3, x3, xzr+        adcs    x14, x14, xzr+        adcs    x8, xzr, xzr+        adds    x13, x1, x13+        mov     x1, v19.d[1]+        adcs    x6, x4, x6+        mov     x4, #0xffffffff                 // #4294967295+        adcs    x15, x7, x15+        adcs    x7, x11, x5+        adcs    x1, x3, x1+        adcs    x14, x14, x10+        adc     x11, x8, xzr+        adds    x6, x6, x17+        adcs    x8, x15, x12+        adcs    x3, x7, x9+        adcs    x15, x1, x16+        mov     x16, #0xffffffff00000001        // #-4294967295+        adcs    x14, x14, x2+        mov     x2, #0x1                        // #1+        adc     x17, x11, xzr+        cmn     x13, x16+        adcs    xzr, x6, x4+        adcs    xzr, x8, x2+        adcs    xzr, x3, xzr+        adcs    xzr, x15, xzr+        adcs    xzr, x14, xzr+        adc     x1, x17, xzr+        neg     x9, x1+        and     x1, x16, x9+        adds    x11, x13, x1+        and     x13, x4, x9+        adcs    x5, x6, x13+        and     x1, x2, x9+        adcs    x7, x8, x1+        stp     x11, x5, [x0]+        adcs    x11, x3, xzr+        adcs    x2, x15, xzr+        stp     x7, x11, [x0, #16]+        adc     x17, x14, xzr+        stp     x2, x17, [x0, #32]+        ldr     q1, [x26, #96]+        ldp     x9, x2, [x26, #96]+        ldr     q0, [x26, #96]+        ldp     x4, x6, [x26, #112]+        rev64   v21.4s, v1.4s+        uzp2    v28.4s, v1.4s, v1.4s+        umulh   x7, x9, x2+        xtn     v17.2s, v1.2d+        mul     v27.4s, v21.4s, v0.4s+        ldr     q20, [x26, #128]+        xtn     v30.2s, v0.2d+        ldr     q1, [x26, #128]+        uzp2    v31.4s, v0.4s, v0.4s+        ldp     x5, x10, [x26, #128]+        umulh   x8, x9, x4+        uaddlp  v3.2d, v27.4s+        umull   v16.2d, v30.2s, v17.2s+        mul     x16, x9, x4+        umull   v27.2d, v30.2s, v28.2s+        shrn    v0.2s, v20.2d, #32+        xtn     v7.2s, v20.2d+        shl     v20.2d, v3.2d, #32+        umull   v3.2d, v31.2s, v28.2s+        mul     x3, x2, x4+        umlal   v20.2d, v30.2s, v17.2s+        umull   v22.2d, v7.2s, v0.2s+        usra    v27.2d, v16.2d, #32+        umulh   x11, x2, x4+        movi    v21.2d, #0xffffffff+        uzp2    v28.4s, v1.4s, v1.4s+        adds    x15, x16, x7+        and     v5.16b, v27.16b, v21.16b+        adcs    x3, x3, x8+        usra    v3.2d, v27.2d, #32+        dup     v29.2d, x6+        adcs    x16, x11, xzr+        mov     x14, v20.d[0]+        umlal   v5.2d, v31.2s, v17.2s+        mul     x8, x9, x2+        mov     x7, v20.d[1]+        shl     v19.2d, v22.2d, #33+        xtn     v25.2s, v29.2d+        rev64   v31.4s, v1.4s+        lsl     x13, x14, #32+        uzp2    v6.4s, v29.4s, v29.4s+        umlal   v19.2d, v7.2s, v7.2s+        usra    v3.2d, v5.2d, #32+        adds    x1, x8, x8+        umulh   x8, x4, x4+        add     x12, x13, x14+        mul     v17.4s, v31.4s, v29.4s+        xtn     v4.2s, v1.2d+        adcs    x14, x15, x15+        lsr     x13, x12, #32+        adcs    x15, x3, x3+        umull   v31.2d, v25.2s, v28.2s+        adcs    x11, x16, x16+        umull   v21.2d, v25.2s, v4.2s+        mov     x17, v3.d[0]+        umull   v18.2d, v6.2s, v28.2s+        adc     x16, x8, xzr+        uaddlp  v16.2d, v17.4s+        movi    v1.2d, #0xffffffff+        subs    x13, x13, x12+        usra    v31.2d, v21.2d, #32+        sbc     x8, x12, xzr+        adds    x17, x17, x1+        mul     x1, x4, x4+        shl     v28.2d, v16.2d, #32+        mov     x3, v3.d[1]+        adcs    x14, x7, x14+        extr    x7, x8, x13, #32+        adcs    x13, x3, x15+        and     v3.16b, v31.16b, v1.16b+        adcs    x11, x1, x11+        lsr     x1, x8, #32+        umlal   v3.2d, v6.2s, v4.2s+        usra    v18.2d, v31.2d, #32+        adc     x3, x16, xzr+        adds    x1, x1, x12+        umlal   v28.2d, v25.2s, v4.2s+        adc     x16, xzr, xzr+        subs    x15, x17, x7+        sbcs    x7, x14, x1+        lsl     x1, x15, #32+        sbcs    x16, x13, x16+        add     x8, x1, x15+        usra    v18.2d, v3.2d, #32+        sbcs    x14, x11, xzr+        lsr     x1, x8, #32+        sbcs    x17, x3, xzr+        sbc     x11, x12, xzr+        subs    x13, x1, x8+        umulh   x12, x4, x10+        sbc     x1, x8, xzr+        extr    x13, x1, x13, #32+        lsr     x1, x1, #32+        adds    x15, x1, x8+        adc     x1, xzr, xzr+        subs    x7, x7, x13+        sbcs    x13, x16, x15+        lsl     x3, x7, #32+        umulh   x16, x2, x5+        sbcs    x15, x14, x1+        add     x7, x3, x7+        sbcs    x3, x17, xzr+        lsr     x1, x7, #32+        sbcs    x14, x11, xzr+        sbc     x11, x8, xzr+        subs    x8, x1, x7+        sbc     x1, x7, xzr+        extr    x8, x1, x8, #32+        lsr     x1, x1, #32+        adds    x1, x1, x7+        adc     x17, xzr, xzr+        subs    x13, x13, x8+        umulh   x8, x9, x6+        sbcs    x1, x15, x1+        sbcs    x15, x3, x17+        sbcs    x3, x14, xzr+        mul     x17, x2, x5+        sbcs    x11, x11, xzr+        stp     x13, x1, [sp, #240]+        sbc     x14, x7, xzr+        mul     x7, x4, x10+        subs    x1, x9, x2+        stp     x15, x3, [sp, #256]+        csetm   x15, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        stp     x11, x14, [sp, #272]+        mul     x14, x9, x6+        adds    x17, x8, x17+        adcs    x7, x16, x7+        adc     x13, x12, xzr+        subs    x12, x5, x6+        cneg    x3, x12, cc  // cc = lo, ul, last+        cinv    x16, x15, cc  // cc = lo, ul, last+        mul     x8, x1, x3+        umulh   x1, x1, x3+        eor     x12, x8, x16+        adds    x11, x17, x14+        adcs    x3, x7, x17+        adcs    x15, x13, x7+        adc     x8, x13, xzr+        adds    x3, x3, x14+        adcs    x15, x15, x17+        adcs    x17, x8, x7+        eor     x1, x1, x16+        adc     x13, x13, xzr+        subs    x9, x9, x4+        csetm   x8, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x4, x2, x4+        cneg    x4, x4, cc  // cc = lo, ul, last+        csetm   x7, cc  // cc = lo, ul, last+        subs    x2, x10, x6+        cinv    x8, x8, cc  // cc = lo, ul, last+        cneg    x2, x2, cc  // cc = lo, ul, last+        cmn     x16, #0x1+        adcs    x11, x11, x12+        mul     x12, x9, x2+        adcs    x3, x3, x1+        adcs    x15, x15, x16+        umulh   x9, x9, x2+        adcs    x17, x17, x16+        adc     x13, x13, x16+        subs    x1, x10, x5+        cinv    x2, x7, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        eor     x9, x9, x8+        cmn     x8, #0x1+        eor     x7, x12, x8+        mul     x12, x4, x1+        adcs    x3, x3, x7+        adcs    x7, x15, x9+        adcs    x15, x17, x8+        ldp     x9, x17, [sp, #256]+        umulh   x4, x4, x1+        adc     x8, x13, x8+        cmn     x2, #0x1+        eor     x1, x12, x2+        adcs    x1, x7, x1+        ldp     x7, x16, [sp, #240]+        eor     x12, x4, x2+        adcs    x4, x15, x12+        ldp     x15, x12, [sp, #272]+        adc     x8, x8, x2+        adds    x13, x14, x14+        umulh   x14, x5, x10+        adcs    x2, x11, x11+        adcs    x3, x3, x3+        adcs    x1, x1, x1+        adcs    x4, x4, x4+        adcs    x11, x8, x8+        adc     x8, xzr, xzr+        adds    x13, x13, x7+        adcs    x2, x2, x16+        mul     x16, x5, x10+        adcs    x3, x3, x9+        adcs    x1, x1, x17+        umulh   x5, x5, x5+        lsl     x9, x13, #32+        add     x9, x9, x13+        adcs    x4, x4, x15+        mov     x13, v28.d[1]+        adcs    x15, x11, x12+        lsr     x7, x9, #32+        adc     x11, x8, xzr+        subs    x7, x7, x9+        umulh   x10, x10, x10+        sbc     x17, x9, xzr+        extr    x7, x17, x7, #32+        lsr     x17, x17, #32+        adds    x17, x17, x9+        adc     x12, xzr, xzr+        subs    x8, x2, x7+        sbcs    x17, x3, x17+        lsl     x7, x8, #32+        sbcs    x2, x1, x12+        add     x3, x7, x8+        sbcs    x12, x4, xzr+        lsr     x1, x3, #32+        sbcs    x7, x15, xzr+        sbc     x15, x9, xzr+        subs    x1, x1, x3+        sbc     x4, x3, xzr+        lsr     x9, x4, #32+        extr    x8, x4, x1, #32+        adds    x9, x9, x3+        adc     x4, xzr, xzr+        subs    x1, x17, x8+        lsl     x17, x1, #32+        sbcs    x8, x2, x9+        sbcs    x9, x12, x4+        add     x17, x17, x1+        mov     x1, v18.d[1]+        lsr     x2, x17, #32+        sbcs    x7, x7, xzr+        mov     x12, v18.d[0]+        sbcs    x15, x15, xzr+        sbc     x3, x3, xzr+        subs    x4, x2, x17+        sbc     x2, x17, xzr+        adds    x12, x13, x12+        adcs    x16, x16, x1+        lsr     x13, x2, #32+        extr    x1, x2, x4, #32+        adc     x2, x14, xzr+        adds    x4, x13, x17+        mul     x13, x6, x6+        adc     x14, xzr, xzr+        subs    x1, x8, x1+        sbcs    x4, x9, x4+        mov     x9, v28.d[0]+        sbcs    x7, x7, x14+        sbcs    x8, x15, xzr+        sbcs    x3, x3, xzr+        sbc     x14, x17, xzr+        adds    x17, x9, x9+        adcs    x12, x12, x12+        mov     x15, v19.d[0]+        adcs    x9, x16, x16+        umulh   x6, x6, x6+        adcs    x16, x2, x2+        adc     x2, xzr, xzr+        adds    x11, x11, x8+        adcs    x3, x3, xzr+        adcs    x14, x14, xzr+        adcs    x8, xzr, xzr+        adds    x13, x1, x13+        mov     x1, v19.d[1]+        adcs    x6, x4, x6+        mov     x4, #0xffffffff                 // #4294967295+        adcs    x15, x7, x15+        adcs    x7, x11, x5+        adcs    x1, x3, x1+        adcs    x14, x14, x10+        adc     x11, x8, xzr+        adds    x6, x6, x17+        adcs    x8, x15, x12+        adcs    x3, x7, x9+        adcs    x15, x1, x16+        mov     x16, #0xffffffff00000001        // #-4294967295+        adcs    x14, x14, x2+        mov     x2, #0x1                        // #1+        adc     x17, x11, xzr+        cmn     x13, x16+        adcs    xzr, x6, x4+        adcs    xzr, x8, x2+        adcs    xzr, x3, xzr+        adcs    xzr, x15, xzr+        adcs    xzr, x14, xzr+        adc     x1, x17, xzr+        neg     x9, x1+        and     x1, x16, x9+        adds    x11, x13, x1+        and     x13, x4, x9+        adcs    x5, x6, x13+        and     x1, x2, x9+        adcs    x7, x8, x1+        stp     x11, x5, [sp, #240]+        adcs    x11, x3, xzr+        adcs    x2, x15, xzr+        stp     x7, x11, [sp, #256]+        adc     x17, x14, xzr+        stp     x2, x17, [sp, #272]+        stp     x23, x24, [sp, #0x150]+        ldr     q3, [x26, #96]+        ldr     q25, [x25, #48]+        ldp     x13, x23, [x25, #48]+        ldp     x3, x21, [x26, #96]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [x25, #80]+        ldp     x8, x24, [x26, #112]+        subs    x6, x3, x21+        ldr     q0, [x26, #128]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [x25, #64]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [x25, #80]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [x26, #128]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x16, x4, x16+        mov     x4, v27.d[0]+        sbcs    x11, x20, x11+        sbcs    x20, x9, x12+        stp     x16, x11, [sp, #288]+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #304]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #320]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        ldp     x20, x9, [sp, #288]+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #304]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #320]+        adds    x20, x22, x20+        mul     x10, x13, x14+        adcs    x11, x11, x9+        eor     x9, x8, x21+        adcs    x21, x19, x17+        stp     x20, x11, [sp, #288]+        adcs    x12, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        stp     x21, x12, [sp, #304]+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #320]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #288]+        ldp     x21, x12, [sp, #304]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #320]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x21+        eor     x1, x22, x9+        adcs    x24, x23, x12+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x21+        adcs    x15, x17, x12+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x22, x6, x20+        eor     x3, x20, x23+        adcs    x5, x7, x3+        adcs    x14, x24, x11+        stp     x22, x5, [sp, #288]+        adcs    x5, x13, x23+        adcs    x21, x8, x23+        stp     x14, x5, [sp, #304]+        adc     x12, x15, x23+        stp     x21, x12, [sp, #320]+        ldr     q3, [x25, #96]+        ldr     q25, [x26, #48]+        ldp     x13, x23, [x26, #48]+        ldp     x3, x21, [x25, #96]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [x26, #80]+        ldp     x8, x24, [x25, #112]+        subs    x6, x3, x21+        ldr     q0, [x25, #128]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [x26, #64]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [x26, #80]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [x25, #128]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x16, x4, x16+        mov     x4, v27.d[0]+        sbcs    x11, x20, x11+        sbcs    x20, x9, x12+        stp     x16, x11, [sp, #48]+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #64]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #80]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        ldp     x20, x9, [sp, #48]+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #64]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #80]+        adds    x20, x22, x20+        mul     x10, x13, x14+        adcs    x11, x11, x9+        eor     x9, x8, x21+        adcs    x21, x19, x17+        stp     x20, x11, [sp, #48]+        adcs    x12, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        stp     x21, x12, [sp, #64]+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #80]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #48]+        ldp     x21, x12, [sp, #64]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #80]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x21+        eor     x1, x22, x9+        adcs    x24, x23, x12+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x21+        adcs    x15, x17, x12+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x22, x6, x20+        eor     x3, x20, x23+        adcs    x5, x7, x3+        adcs    x14, x24, x11+        stp     x22, x5, [sp, #48]+        adcs    x5, x13, x23+        adcs    x21, x8, x23+        stp     x14, x5, [sp, #64]+        adc     x12, x15, x23+        stp     x21, x12, [sp, #80]+        mov     x1, sp+        ldr     q3, [x1]+        ldr     q25, [x26]+        ldp     x13, x23, [x26]+        ldp     x3, x21, [x1]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [x26, #32]+        ldp     x8, x24, [x1, #16]+        subs    x6, x3, x21+        ldr     q0, [x1, #32]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [x26, #16]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [x26, #32]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [x1, #32]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x16, x4, x16+        mov     x4, v27.d[0]+        sbcs    x11, x20, x11+        sbcs    x20, x9, x12+        stp     x16, x11, [sp, #96]+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #112]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #128]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        ldp     x20, x9, [sp, #96]+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #112]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #128]+        adds    x20, x22, x20+        mul     x10, x13, x14+        adcs    x11, x11, x9+        eor     x9, x8, x21+        adcs    x21, x19, x17+        stp     x20, x11, [sp, #96]+        adcs    x12, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        stp     x21, x12, [sp, #112]+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #128]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #96]+        ldp     x21, x12, [sp, #112]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #128]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x21+        eor     x1, x22, x9+        adcs    x24, x23, x12+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x21+        adcs    x15, x17, x12+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x22, x6, x20+        eor     x3, x20, x23+        adcs    x5, x7, x3+        adcs    x14, x24, x11+        stp     x22, x5, [sp, #96]+        adcs    x5, x13, x23+        adcs    x21, x8, x23+        stp     x14, x5, [sp, #112]+        adc     x12, x15, x23+        stp     x21, x12, [sp, #128]+        ldr     q3, [sp, #240]+        ldr     q25, [x25]+        ldp     x13, x23, [x25]+        ldp     x3, x21, [sp, #240]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [x25, #32]+        ldp     x8, x24, [sp, #256]+        subs    x6, x3, x21+        ldr     q0, [sp, #272]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [x25, #16]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [x25, #32]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [sp, #272]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x16, x4, x16+        mov     x4, v27.d[0]+        sbcs    x11, x20, x11+        sbcs    x20, x9, x12+        stp     x16, x11, [sp, #192]+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #208]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #224]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        ldp     x20, x9, [sp, #192]+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #208]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #224]+        adds    x20, x22, x20+        mul     x10, x13, x14+        adcs    x11, x11, x9+        eor     x9, x8, x21+        adcs    x21, x19, x17+        stp     x20, x11, [sp, #192]+        adcs    x12, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        stp     x21, x12, [sp, #208]+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #224]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #192]+        ldp     x21, x12, [sp, #208]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #224]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x21+        eor     x1, x22, x9+        adcs    x24, x23, x12+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x21+        adcs    x15, x17, x12+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x22, x6, x20+        eor     x3, x20, x23+        adcs    x5, x7, x3+        adcs    x14, x24, x11+        stp     x22, x5, [sp, #192]+        adcs    x5, x13, x23+        adcs    x21, x8, x23+        stp     x14, x5, [sp, #208]+        adc     x12, x15, x23+        stp     x21, x12, [sp, #224]+        mov     x1, sp+        ldr     q3, [x1]+        ldr     q25, [sp, #48]+        ldp     x13, x23, [sp, #48]+        ldp     x3, x21, [x1]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [sp, #80]+        ldp     x8, x24, [x1, #16]+        subs    x6, x3, x21+        ldr     q0, [x1, #32]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [sp, #64]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [sp, #80]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [x1, #32]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x16, x4, x16+        mov     x4, v27.d[0]+        sbcs    x11, x20, x11+        sbcs    x20, x9, x12+        stp     x16, x11, [sp, #48]+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #64]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #80]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        ldp     x20, x9, [sp, #48]+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #64]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #80]+        adds    x20, x22, x20+        mul     x10, x13, x14+        adcs    x11, x11, x9+        eor     x9, x8, x21+        adcs    x21, x19, x17+        stp     x20, x11, [sp, #48]+        adcs    x12, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        stp     x21, x12, [sp, #64]+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #80]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #48]+        ldp     x21, x12, [sp, #64]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #80]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x21+        eor     x1, x22, x9+        adcs    x24, x23, x12+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x21+        adcs    x15, x17, x12+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x22, x6, x20+        eor     x3, x20, x23+        adcs    x5, x7, x3+        adcs    x14, x24, x11+        stp     x22, x5, [sp, #48]+        adcs    x5, x13, x23+        adcs    x21, x8, x23+        stp     x14, x5, [sp, #64]+        adc     x12, x15, x23+        stp     x21, x12, [sp, #80]+        ldr     q3, [sp, #240]+        ldr     q25, [sp, #288]+        ldp     x13, x23, [sp, #288]+        ldp     x3, x21, [sp, #240]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [sp, #320]+        ldp     x8, x24, [sp, #256]+        subs    x6, x3, x21+        ldr     q0, [sp, #272]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [sp, #304]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [sp, #320]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [sp, #272]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x16, x4, x16+        mov     x4, v27.d[0]+        sbcs    x11, x20, x11+        sbcs    x20, x9, x12+        stp     x16, x11, [sp, #288]+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #304]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #320]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        ldp     x20, x9, [sp, #288]+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #304]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #320]+        adds    x20, x22, x20+        mul     x10, x13, x14+        adcs    x11, x11, x9+        eor     x9, x8, x21+        adcs    x21, x19, x17+        stp     x20, x11, [sp, #288]+        adcs    x12, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        stp     x21, x12, [sp, #304]+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #320]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #288]+        ldp     x21, x12, [sp, #304]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #320]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x21+        eor     x1, x22, x9+        adcs    x24, x23, x12+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x21+        adcs    x15, x17, x12+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x22, x6, x20+        eor     x3, x20, x23+        adcs    x5, x7, x3+        adcs    x2, x24, x11+        stp     x22, x5, [sp, #288]+        adcs    x11, x13, x23+        adcs    x12, x8, x23+        stp     x2, x11, [sp, #304]+        adc     x13, x15, x23+        stp     x12, x13, [sp, #320]+        ldp     x5, x6, [sp, #96]+        ldp     x4, x3, [sp, #192]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #112]+        ldp     x4, x3, [sp, #208]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #128]+        ldp     x4, x3, [sp, #224]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, cc  // cc = lo, ul, last+        mov     x4, #0xffffffff                 // #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #0xfffffffffffffffe         // #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [sp, #240]+        stp     x7, x8, [sp, #256]+        stp     x9, x10, [sp, #272]+        ldp     x5, x6, [sp, #48]+        ldp     x4, x3, [sp, #288]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #64]+        sbcs    x7, x7, x2+        sbcs    x8, x8, x11+        ldp     x9, x10, [sp, #80]+        sbcs    x9, x9, x12+        sbcs    x10, x10, x13+        csetm   x3, cc  // cc = lo, ul, last+        mov     x4, #0xffffffff                 // #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #0xfffffffffffffffe         // #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [sp, #48]+        stp     x7, x8, [sp, #64]+        stp     x9, x10, [sp, #80]+        ldr     q1, [sp, #240]+        ldp     x9, x2, [sp, #240]+        ldr     q0, [sp, #240]+        ldp     x4, x6, [sp, #256]+        rev64   v21.4s, v1.4s+        uzp2    v28.4s, v1.4s, v1.4s+        umulh   x7, x9, x2+        xtn     v17.2s, v1.2d+        mul     v27.4s, v21.4s, v0.4s+        ldr     q20, [sp, #272]+        xtn     v30.2s, v0.2d+        ldr     q1, [sp, #272]+        uzp2    v31.4s, v0.4s, v0.4s+        ldp     x5, x10, [sp, #272]+        umulh   x8, x9, x4+        uaddlp  v3.2d, v27.4s+        umull   v16.2d, v30.2s, v17.2s+        mul     x16, x9, x4+        umull   v27.2d, v30.2s, v28.2s+        shrn    v0.2s, v20.2d, #32+        xtn     v7.2s, v20.2d+        shl     v20.2d, v3.2d, #32+        umull   v3.2d, v31.2s, v28.2s+        mul     x3, x2, x4+        umlal   v20.2d, v30.2s, v17.2s+        umull   v22.2d, v7.2s, v0.2s+        usra    v27.2d, v16.2d, #32+        umulh   x11, x2, x4+        movi    v21.2d, #0xffffffff+        uzp2    v28.4s, v1.4s, v1.4s+        adds    x15, x16, x7+        and     v5.16b, v27.16b, v21.16b+        adcs    x3, x3, x8+        usra    v3.2d, v27.2d, #32+        dup     v29.2d, x6+        adcs    x16, x11, xzr+        mov     x14, v20.d[0]+        umlal   v5.2d, v31.2s, v17.2s+        mul     x8, x9, x2+        mov     x7, v20.d[1]+        shl     v19.2d, v22.2d, #33+        xtn     v25.2s, v29.2d+        rev64   v31.4s, v1.4s+        lsl     x13, x14, #32+        uzp2    v6.4s, v29.4s, v29.4s+        umlal   v19.2d, v7.2s, v7.2s+        usra    v3.2d, v5.2d, #32+        adds    x1, x8, x8+        umulh   x8, x4, x4+        add     x12, x13, x14+        mul     v17.4s, v31.4s, v29.4s+        xtn     v4.2s, v1.2d+        adcs    x14, x15, x15+        lsr     x13, x12, #32+        adcs    x15, x3, x3+        umull   v31.2d, v25.2s, v28.2s+        adcs    x11, x16, x16+        umull   v21.2d, v25.2s, v4.2s+        mov     x17, v3.d[0]+        umull   v18.2d, v6.2s, v28.2s+        adc     x16, x8, xzr+        uaddlp  v16.2d, v17.4s+        movi    v1.2d, #0xffffffff+        subs    x13, x13, x12+        usra    v31.2d, v21.2d, #32+        sbc     x8, x12, xzr+        adds    x17, x17, x1+        mul     x1, x4, x4+        shl     v28.2d, v16.2d, #32+        mov     x3, v3.d[1]+        adcs    x14, x7, x14+        extr    x7, x8, x13, #32+        adcs    x13, x3, x15+        and     v3.16b, v31.16b, v1.16b+        adcs    x11, x1, x11+        lsr     x1, x8, #32+        umlal   v3.2d, v6.2s, v4.2s+        usra    v18.2d, v31.2d, #32+        adc     x3, x16, xzr+        adds    x1, x1, x12+        umlal   v28.2d, v25.2s, v4.2s+        adc     x16, xzr, xzr+        subs    x15, x17, x7+        sbcs    x7, x14, x1+        lsl     x1, x15, #32+        sbcs    x16, x13, x16+        add     x8, x1, x15+        usra    v18.2d, v3.2d, #32+        sbcs    x14, x11, xzr+        lsr     x1, x8, #32+        sbcs    x17, x3, xzr+        sbc     x11, x12, xzr+        subs    x13, x1, x8+        umulh   x12, x4, x10+        sbc     x1, x8, xzr+        extr    x13, x1, x13, #32+        lsr     x1, x1, #32+        adds    x15, x1, x8+        adc     x1, xzr, xzr+        subs    x7, x7, x13+        sbcs    x13, x16, x15+        lsl     x3, x7, #32+        umulh   x16, x2, x5+        sbcs    x15, x14, x1+        add     x7, x3, x7+        sbcs    x3, x17, xzr+        lsr     x1, x7, #32+        sbcs    x14, x11, xzr+        sbc     x11, x8, xzr+        subs    x8, x1, x7+        sbc     x1, x7, xzr+        extr    x8, x1, x8, #32+        lsr     x1, x1, #32+        adds    x1, x1, x7+        adc     x17, xzr, xzr+        subs    x13, x13, x8+        umulh   x8, x9, x6+        sbcs    x1, x15, x1+        sbcs    x15, x3, x17+        sbcs    x3, x14, xzr+        mul     x17, x2, x5+        sbcs    x11, x11, xzr+        stp     x13, x1, [sp, #144]+        sbc     x14, x7, xzr+        mul     x7, x4, x10+        subs    x1, x9, x2+        stp     x15, x3, [sp, #160]+        csetm   x15, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        stp     x11, x14, [sp, #176]+        mul     x14, x9, x6+        adds    x17, x8, x17+        adcs    x7, x16, x7+        adc     x13, x12, xzr+        subs    x12, x5, x6+        cneg    x3, x12, cc  // cc = lo, ul, last+        cinv    x16, x15, cc  // cc = lo, ul, last+        mul     x8, x1, x3+        umulh   x1, x1, x3+        eor     x12, x8, x16+        adds    x11, x17, x14+        adcs    x3, x7, x17+        adcs    x15, x13, x7+        adc     x8, x13, xzr+        adds    x3, x3, x14+        adcs    x15, x15, x17+        adcs    x17, x8, x7+        eor     x1, x1, x16+        adc     x13, x13, xzr+        subs    x9, x9, x4+        csetm   x8, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x4, x2, x4+        cneg    x4, x4, cc  // cc = lo, ul, last+        csetm   x7, cc  // cc = lo, ul, last+        subs    x2, x10, x6+        cinv    x8, x8, cc  // cc = lo, ul, last+        cneg    x2, x2, cc  // cc = lo, ul, last+        cmn     x16, #0x1+        adcs    x11, x11, x12+        mul     x12, x9, x2+        adcs    x3, x3, x1+        adcs    x15, x15, x16+        umulh   x9, x9, x2+        adcs    x17, x17, x16+        adc     x13, x13, x16+        subs    x1, x10, x5+        cinv    x2, x7, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        eor     x9, x9, x8+        cmn     x8, #0x1+        eor     x7, x12, x8+        mul     x12, x4, x1+        adcs    x3, x3, x7+        adcs    x7, x15, x9+        adcs    x15, x17, x8+        ldp     x9, x17, [sp, #160]+        umulh   x4, x4, x1+        adc     x8, x13, x8+        cmn     x2, #0x1+        eor     x1, x12, x2+        adcs    x1, x7, x1+        ldp     x7, x16, [sp, #144]+        eor     x12, x4, x2+        adcs    x4, x15, x12+        ldp     x15, x12, [sp, #176]+        adc     x8, x8, x2+        adds    x13, x14, x14+        umulh   x14, x5, x10+        adcs    x2, x11, x11+        adcs    x3, x3, x3+        adcs    x1, x1, x1+        adcs    x4, x4, x4+        adcs    x11, x8, x8+        adc     x8, xzr, xzr+        adds    x13, x13, x7+        adcs    x2, x2, x16+        mul     x16, x5, x10+        adcs    x3, x3, x9+        adcs    x1, x1, x17+        umulh   x5, x5, x5+        lsl     x9, x13, #32+        add     x9, x9, x13+        adcs    x4, x4, x15+        mov     x13, v28.d[1]+        adcs    x15, x11, x12+        lsr     x7, x9, #32+        adc     x11, x8, xzr+        subs    x7, x7, x9+        umulh   x10, x10, x10+        sbc     x17, x9, xzr+        extr    x7, x17, x7, #32+        lsr     x17, x17, #32+        adds    x17, x17, x9+        adc     x12, xzr, xzr+        subs    x8, x2, x7+        sbcs    x17, x3, x17+        lsl     x7, x8, #32+        sbcs    x2, x1, x12+        add     x3, x7, x8+        sbcs    x12, x4, xzr+        lsr     x1, x3, #32+        sbcs    x7, x15, xzr+        sbc     x15, x9, xzr+        subs    x1, x1, x3+        sbc     x4, x3, xzr+        lsr     x9, x4, #32+        extr    x8, x4, x1, #32+        adds    x9, x9, x3+        adc     x4, xzr, xzr+        subs    x1, x17, x8+        lsl     x17, x1, #32+        sbcs    x8, x2, x9+        sbcs    x9, x12, x4+        add     x17, x17, x1+        mov     x1, v18.d[1]+        lsr     x2, x17, #32+        sbcs    x7, x7, xzr+        mov     x12, v18.d[0]+        sbcs    x15, x15, xzr+        sbc     x3, x3, xzr+        subs    x4, x2, x17+        sbc     x2, x17, xzr+        adds    x12, x13, x12+        adcs    x16, x16, x1+        lsr     x13, x2, #32+        extr    x1, x2, x4, #32+        adc     x2, x14, xzr+        adds    x4, x13, x17+        mul     x13, x6, x6+        adc     x14, xzr, xzr+        subs    x1, x8, x1+        sbcs    x4, x9, x4+        mov     x9, v28.d[0]+        sbcs    x7, x7, x14+        sbcs    x8, x15, xzr+        sbcs    x3, x3, xzr+        sbc     x14, x17, xzr+        adds    x17, x9, x9+        adcs    x12, x12, x12+        mov     x15, v19.d[0]+        adcs    x9, x16, x16+        umulh   x6, x6, x6+        adcs    x16, x2, x2+        adc     x2, xzr, xzr+        adds    x11, x11, x8+        adcs    x3, x3, xzr+        adcs    x14, x14, xzr+        adcs    x8, xzr, xzr+        adds    x13, x1, x13+        mov     x1, v19.d[1]+        adcs    x6, x4, x6+        mov     x4, #0xffffffff                 // #4294967295+        adcs    x15, x7, x15+        adcs    x7, x11, x5+        adcs    x1, x3, x1+        adcs    x14, x14, x10+        adc     x11, x8, xzr+        adds    x6, x6, x17+        adcs    x8, x15, x12+        adcs    x3, x7, x9+        adcs    x15, x1, x16+        mov     x16, #0xffffffff00000001        // #-4294967295+        adcs    x14, x14, x2+        mov     x2, #0x1                        // #1+        adc     x17, x11, xzr+        cmn     x13, x16+        adcs    xzr, x6, x4+        adcs    xzr, x8, x2+        adcs    xzr, x3, xzr+        adcs    xzr, x15, xzr+        adcs    xzr, x14, xzr+        adc     x1, x17, xzr+        neg     x9, x1+        and     x1, x16, x9+        adds    x11, x13, x1+        and     x13, x4, x9+        adcs    x5, x6, x13+        and     x1, x2, x9+        adcs    x7, x8, x1+        stp     x11, x5, [sp, #144]+        adcs    x11, x3, xzr+        adcs    x2, x15, xzr+        stp     x7, x11, [sp, #160]+        adc     x17, x14, xzr+        stp     x2, x17, [sp, #176]+        mov     x0, sp+        ldr     q1, [sp, #48]+        ldp     x9, x2, [sp, #48]+        ldr     q0, [sp, #48]+        ldp     x4, x6, [sp, #64]+        rev64   v21.4s, v1.4s+        uzp2    v28.4s, v1.4s, v1.4s+        umulh   x7, x9, x2+        xtn     v17.2s, v1.2d+        mul     v27.4s, v21.4s, v0.4s+        ldr     q20, [sp, #80]+        xtn     v30.2s, v0.2d+        ldr     q1, [sp, #80]+        uzp2    v31.4s, v0.4s, v0.4s+        ldp     x5, x10, [sp, #80]+        umulh   x8, x9, x4+        uaddlp  v3.2d, v27.4s+        umull   v16.2d, v30.2s, v17.2s+        mul     x16, x9, x4+        umull   v27.2d, v30.2s, v28.2s+        shrn    v0.2s, v20.2d, #32+        xtn     v7.2s, v20.2d+        shl     v20.2d, v3.2d, #32+        umull   v3.2d, v31.2s, v28.2s+        mul     x3, x2, x4+        umlal   v20.2d, v30.2s, v17.2s+        umull   v22.2d, v7.2s, v0.2s+        usra    v27.2d, v16.2d, #32+        umulh   x11, x2, x4+        movi    v21.2d, #0xffffffff+        uzp2    v28.4s, v1.4s, v1.4s+        adds    x15, x16, x7+        and     v5.16b, v27.16b, v21.16b+        adcs    x3, x3, x8+        usra    v3.2d, v27.2d, #32+        dup     v29.2d, x6+        adcs    x16, x11, xzr+        mov     x14, v20.d[0]+        umlal   v5.2d, v31.2s, v17.2s+        mul     x8, x9, x2+        mov     x7, v20.d[1]+        shl     v19.2d, v22.2d, #33+        xtn     v25.2s, v29.2d+        rev64   v31.4s, v1.4s+        lsl     x13, x14, #32+        uzp2    v6.4s, v29.4s, v29.4s+        umlal   v19.2d, v7.2s, v7.2s+        usra    v3.2d, v5.2d, #32+        adds    x1, x8, x8+        umulh   x8, x4, x4+        add     x12, x13, x14+        mul     v17.4s, v31.4s, v29.4s+        xtn     v4.2s, v1.2d+        adcs    x14, x15, x15+        lsr     x13, x12, #32+        adcs    x15, x3, x3+        umull   v31.2d, v25.2s, v28.2s+        adcs    x11, x16, x16+        umull   v21.2d, v25.2s, v4.2s+        mov     x17, v3.d[0]+        umull   v18.2d, v6.2s, v28.2s+        adc     x16, x8, xzr+        uaddlp  v16.2d, v17.4s+        movi    v1.2d, #0xffffffff+        subs    x13, x13, x12+        usra    v31.2d, v21.2d, #32+        sbc     x8, x12, xzr+        adds    x17, x17, x1+        mul     x1, x4, x4+        shl     v28.2d, v16.2d, #32+        mov     x3, v3.d[1]+        adcs    x14, x7, x14+        extr    x7, x8, x13, #32+        adcs    x13, x3, x15+        and     v3.16b, v31.16b, v1.16b+        adcs    x11, x1, x11+        lsr     x1, x8, #32+        umlal   v3.2d, v6.2s, v4.2s+        usra    v18.2d, v31.2d, #32+        adc     x3, x16, xzr+        adds    x1, x1, x12+        umlal   v28.2d, v25.2s, v4.2s+        adc     x16, xzr, xzr+        subs    x15, x17, x7+        sbcs    x7, x14, x1+        lsl     x1, x15, #32+        sbcs    x16, x13, x16+        add     x8, x1, x15+        usra    v18.2d, v3.2d, #32+        sbcs    x14, x11, xzr+        lsr     x1, x8, #32+        sbcs    x17, x3, xzr+        sbc     x11, x12, xzr+        subs    x13, x1, x8+        umulh   x12, x4, x10+        sbc     x1, x8, xzr+        extr    x13, x1, x13, #32+        lsr     x1, x1, #32+        adds    x15, x1, x8+        adc     x1, xzr, xzr+        subs    x7, x7, x13+        sbcs    x13, x16, x15+        lsl     x3, x7, #32+        umulh   x16, x2, x5+        sbcs    x15, x14, x1+        add     x7, x3, x7+        sbcs    x3, x17, xzr+        lsr     x1, x7, #32+        sbcs    x14, x11, xzr+        sbc     x11, x8, xzr+        subs    x8, x1, x7+        sbc     x1, x7, xzr+        extr    x8, x1, x8, #32+        lsr     x1, x1, #32+        adds    x1, x1, x7+        adc     x17, xzr, xzr+        subs    x13, x13, x8+        umulh   x8, x9, x6+        sbcs    x1, x15, x1+        sbcs    x15, x3, x17+        sbcs    x3, x14, xzr+        mul     x17, x2, x5+        sbcs    x11, x11, xzr+        stp     x13, x1, [x0]+        sbc     x14, x7, xzr+        mul     x7, x4, x10+        subs    x1, x9, x2+        stp     x15, x3, [x0, #16]+        csetm   x15, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        stp     x11, x14, [x0, #32]+        mul     x14, x9, x6+        adds    x17, x8, x17+        adcs    x7, x16, x7+        adc     x13, x12, xzr+        subs    x12, x5, x6+        cneg    x3, x12, cc  // cc = lo, ul, last+        cinv    x16, x15, cc  // cc = lo, ul, last+        mul     x8, x1, x3+        umulh   x1, x1, x3+        eor     x12, x8, x16+        adds    x11, x17, x14+        adcs    x3, x7, x17+        adcs    x15, x13, x7+        adc     x8, x13, xzr+        adds    x3, x3, x14+        adcs    x15, x15, x17+        adcs    x17, x8, x7+        eor     x1, x1, x16+        adc     x13, x13, xzr+        subs    x9, x9, x4+        csetm   x8, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x4, x2, x4+        cneg    x4, x4, cc  // cc = lo, ul, last+        csetm   x7, cc  // cc = lo, ul, last+        subs    x2, x10, x6+        cinv    x8, x8, cc  // cc = lo, ul, last+        cneg    x2, x2, cc  // cc = lo, ul, last+        cmn     x16, #0x1+        adcs    x11, x11, x12+        mul     x12, x9, x2+        adcs    x3, x3, x1+        adcs    x15, x15, x16+        umulh   x9, x9, x2+        adcs    x17, x17, x16+        adc     x13, x13, x16+        subs    x1, x10, x5+        cinv    x2, x7, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        eor     x9, x9, x8+        cmn     x8, #0x1+        eor     x7, x12, x8+        mul     x12, x4, x1+        adcs    x3, x3, x7+        adcs    x7, x15, x9+        adcs    x15, x17, x8+        ldp     x9, x17, [x0, #16]+        umulh   x4, x4, x1+        adc     x8, x13, x8+        cmn     x2, #0x1+        eor     x1, x12, x2+        adcs    x1, x7, x1+        ldp     x7, x16, [x0]+        eor     x12, x4, x2+        adcs    x4, x15, x12+        ldp     x15, x12, [x0, #32]+        adc     x8, x8, x2+        adds    x13, x14, x14+        umulh   x14, x5, x10+        adcs    x2, x11, x11+        adcs    x3, x3, x3+        adcs    x1, x1, x1+        adcs    x4, x4, x4+        adcs    x11, x8, x8+        adc     x8, xzr, xzr+        adds    x13, x13, x7+        adcs    x2, x2, x16+        mul     x16, x5, x10+        adcs    x3, x3, x9+        adcs    x1, x1, x17+        umulh   x5, x5, x5+        lsl     x9, x13, #32+        add     x9, x9, x13+        adcs    x4, x4, x15+        mov     x13, v28.d[1]+        adcs    x15, x11, x12+        lsr     x7, x9, #32+        adc     x11, x8, xzr+        subs    x7, x7, x9+        umulh   x10, x10, x10+        sbc     x17, x9, xzr+        extr    x7, x17, x7, #32+        lsr     x17, x17, #32+        adds    x17, x17, x9+        adc     x12, xzr, xzr+        subs    x8, x2, x7+        sbcs    x17, x3, x17+        lsl     x7, x8, #32+        sbcs    x2, x1, x12+        add     x3, x7, x8+        sbcs    x12, x4, xzr+        lsr     x1, x3, #32+        sbcs    x7, x15, xzr+        sbc     x15, x9, xzr+        subs    x1, x1, x3+        sbc     x4, x3, xzr+        lsr     x9, x4, #32+        extr    x8, x4, x1, #32+        adds    x9, x9, x3+        adc     x4, xzr, xzr+        subs    x1, x17, x8+        lsl     x17, x1, #32+        sbcs    x8, x2, x9+        sbcs    x9, x12, x4+        add     x17, x17, x1+        mov     x1, v18.d[1]+        lsr     x2, x17, #32+        sbcs    x7, x7, xzr+        mov     x12, v18.d[0]+        sbcs    x15, x15, xzr+        sbc     x3, x3, xzr+        subs    x4, x2, x17+        sbc     x2, x17, xzr+        adds    x12, x13, x12+        adcs    x16, x16, x1+        lsr     x13, x2, #32+        extr    x1, x2, x4, #32+        adc     x2, x14, xzr+        adds    x4, x13, x17+        mul     x13, x6, x6+        adc     x14, xzr, xzr+        subs    x1, x8, x1+        sbcs    x4, x9, x4+        mov     x9, v28.d[0]+        sbcs    x7, x7, x14+        sbcs    x8, x15, xzr+        sbcs    x3, x3, xzr+        sbc     x14, x17, xzr+        adds    x17, x9, x9+        adcs    x12, x12, x12+        mov     x15, v19.d[0]+        adcs    x9, x16, x16+        umulh   x6, x6, x6+        adcs    x16, x2, x2+        adc     x2, xzr, xzr+        adds    x11, x11, x8+        adcs    x3, x3, xzr+        adcs    x14, x14, xzr+        adcs    x8, xzr, xzr+        adds    x13, x1, x13+        mov     x1, v19.d[1]+        adcs    x6, x4, x6+        mov     x4, #0xffffffff                 // #4294967295+        adcs    x15, x7, x15+        adcs    x7, x11, x5+        adcs    x1, x3, x1+        adcs    x14, x14, x10+        adc     x11, x8, xzr+        adds    x6, x6, x17+        adcs    x8, x15, x12+        adcs    x3, x7, x9+        adcs    x15, x1, x16+        mov     x16, #0xffffffff00000001        // #-4294967295+        adcs    x14, x14, x2+        mov     x2, #0x1                        // #1+        adc     x17, x11, xzr+        cmn     x13, x16+        adcs    xzr, x6, x4+        adcs    xzr, x8, x2+        adcs    xzr, x3, xzr+        adcs    xzr, x15, xzr+        adcs    xzr, x14, xzr+        adc     x1, x17, xzr+        neg     x9, x1+        and     x1, x16, x9+        adds    x11, x13, x1+        and     x13, x4, x9+        adcs    x5, x6, x13+        and     x1, x2, x9+        adcs    x7, x8, x1+        stp     x11, x5, [x0]+        adcs    x11, x3, xzr+        adcs    x2, x15, xzr+        stp     x7, x11, [x0, #16]+        adc     x17, x14, xzr+        stp     x2, x17, [x0, #32]+        ldr     q3, [sp, #144]+        ldr     q25, [sp, #192]+        ldp     x13, x23, [sp, #192]+        ldp     x3, x21, [sp, #144]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [sp, #224]+        ldp     x8, x24, [sp, #160]+        subs    x6, x3, x21+        ldr     q0, [sp, #176]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [sp, #208]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [sp, #224]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [sp, #176]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x16, x4, x16+        mov     x4, v27.d[0]+        sbcs    x11, x20, x11+        sbcs    x20, x9, x12+        stp     x16, x11, [sp, #192]+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #208]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #224]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        ldp     x20, x9, [sp, #192]+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #208]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #224]+        adds    x20, x22, x20+        mul     x10, x13, x14+        adcs    x11, x11, x9+        eor     x9, x8, x21+        adcs    x21, x19, x17+        stp     x20, x11, [sp, #192]+        adcs    x12, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        stp     x21, x12, [sp, #208]+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #224]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #192]+        ldp     x21, x12, [sp, #208]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #224]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x21+        eor     x1, x22, x9+        adcs    x24, x23, x12+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x21+        adcs    x15, x17, x12+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x22, x6, x20+        eor     x3, x20, x23+        adcs    x5, x7, x3+        adcs    x14, x24, x11+        stp     x22, x5, [sp, #192]+        adcs    x5, x13, x23+        adcs    x21, x8, x23+        stp     x14, x5, [sp, #208]+        adc     x12, x15, x23+        stp     x21, x12, [sp, #224]+        ldr     q3, [sp, #144]+        ldr     q25, [sp, #96]+        ldp     x13, x23, [sp, #96]+        ldp     x3, x21, [sp, #144]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [sp, #128]+        ldp     x8, x24, [sp, #160]+        subs    x6, x3, x21+        ldr     q0, [sp, #176]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [sp, #112]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [sp, #128]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [sp, #176]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x16, x4, x16+        mov     x4, v27.d[0]+        sbcs    x11, x20, x11+        sbcs    x20, x9, x12+        stp     x16, x11, [sp, #96]+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #112]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #128]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        ldp     x20, x9, [sp, #96]+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #112]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #128]+        adds    x20, x22, x20+        mul     x10, x13, x14+        adcs    x11, x11, x9+        eor     x9, x8, x21+        adcs    x21, x19, x17+        stp     x20, x11, [sp, #96]+        adcs    x12, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        stp     x21, x12, [sp, #112]+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #128]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #96]+        ldp     x21, x12, [sp, #112]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #128]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x21+        eor     x1, x22, x9+        adcs    x24, x23, x12+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x21+        adcs    x15, x17, x12+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x22, x6, x20+        eor     x3, x20, x23+        adcs    x5, x7, x3+        adcs    x2, x24, x11+        stp     x22, x5, [sp, #96]+        adcs    x11, x13, x23+        adcs    x12, x8, x23+        stp     x2, x11, [sp, #112]+        adc     x13, x15, x23+        stp     x12, x13, [sp, #128]+        mov     x0, sp+        mov     x1, sp+        ldp     x5, x6, [x1]+        ldp     x4, x3, [sp, #192]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [x1, #16]+        ldp     x4, x3, [sp, #208]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [x1, #32]+        ldp     x4, x3, [sp, #224]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, cc  // cc = lo, ul, last+        mov     x4, #0xffffffff                 // #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #0xfffffffffffffffe         // #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [x0]+        stp     x7, x8, [x0, #16]+        stp     x9, x10, [x0, #32]+        ldp     x5, x6, [sp, #96]+        ldp     x4, x3, [sp, #192]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x4, x3, [sp, #208]+        sbcs    x7, x2, x4+        sbcs    x8, x11, x3+        ldp     x4, x3, [sp, #224]+        sbcs    x9, x12, x4+        sbcs    x10, x13, x3+        csetm   x3, cc  // cc = lo, ul, last+        mov     x4, #0xffffffff                 // #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #0xfffffffffffffffe         // #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [sp, #144]+        stp     x7, x8, [sp, #160]+        stp     x9, x10, [sp, #176]+        ldr     q3, [sp, #240]+        ldr     q25, [x25, #96]+        ldp     x13, x23, [x25, #96]+        ldp     x3, x21, [sp, #240]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [x25, #128]+        ldp     x8, x24, [sp, #256]+        subs    x6, x3, x21+        ldr     q0, [sp, #272]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [x25, #112]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [x25, #128]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [sp, #272]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x16, x4, x16+        mov     x4, v27.d[0]+        sbcs    x11, x20, x11+        sbcs    x20, x9, x12+        stp     x16, x11, [sp, #240]+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #256]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #272]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        ldp     x20, x9, [sp, #240]+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #256]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #272]+        adds    x20, x22, x20+        mul     x10, x13, x14+        adcs    x11, x11, x9+        eor     x9, x8, x21+        adcs    x21, x19, x17+        stp     x20, x11, [sp, #240]+        adcs    x12, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        stp     x21, x12, [sp, #256]+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #272]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #240]+        ldp     x21, x12, [sp, #256]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #272]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x21+        eor     x1, x22, x9+        adcs    x24, x23, x12+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x21+        adcs    x15, x17, x12+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x22, x6, x20+        eor     x3, x20, x23+        adcs    x5, x7, x3+        adcs    x14, x24, x11+        stp     x22, x5, [sp, #240]+        adcs    x5, x13, x23+        adcs    x21, x8, x23+        stp     x14, x5, [sp, #256]+        adc     x12, x15, x23+        stp     x21, x12, [sp, #272]+        mov     x0, sp+        mov     x1, sp+        ldp     x5, x6, [x1]+        ldp     x4, x3, [sp, #96]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [x1, #16]+        ldp     x4, x3, [sp, #112]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [x1, #32]+        ldp     x4, x3, [sp, #128]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, cc  // cc = lo, ul, last+        mov     x4, #0xffffffff                 // #4294967295+        and     x4, x4, x3+        adds    x2, x5, x4+        eor     x4, x4, x3+        adcs    x11, x6, x4+        mov     x4, #0xfffffffffffffffe         // #-2+        and     x4, x4, x3+        adcs    x4, x7, x4+        adcs    x12, x8, x3+        adcs    x13, x9, x3+        adc     x3, x10, x3+        stp     x2, x11, [x0]+        stp     x4, x12, [x0, #16]+        stp     x13, x3, [x0, #32]+        ldp     x5, x6, [sp, #192]+        subs    x5, x5, x2+        sbcs    x6, x6, x11+        ldp     x7, x8, [sp, #208]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x12+        ldp     x9, x10, [sp, #224]+        sbcs    x9, x9, x13+        sbcs    x10, x10, x3+        csetm   x3, cc  // cc = lo, ul, last+        mov     x4, #0xffffffff                 // #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #0xfffffffffffffffe         // #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [sp, #192]+        stp     x7, x8, [sp, #208]+        stp     x9, x10, [sp, #224]+        ldr     q3, [sp, #144]+        ldr     q25, [sp, #288]+        ldp     x13, x23, [sp, #288]+        ldp     x3, x21, [sp, #144]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [sp, #320]+        ldp     x8, x24, [sp, #160]+        subs    x6, x3, x21+        ldr     q0, [sp, #176]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [sp, #304]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [sp, #320]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [sp, #176]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x16, x4, x16+        mov     x4, v27.d[0]+        sbcs    x11, x20, x11+        sbcs    x20, x9, x12+        stp     x16, x11, [sp, #144]+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #160]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #176]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        ldp     x20, x9, [sp, #144]+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #160]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #176]+        adds    x20, x22, x20+        mul     x10, x13, x14+        adcs    x11, x11, x9+        eor     x9, x8, x21+        adcs    x21, x19, x17+        stp     x20, x11, [sp, #144]+        adcs    x12, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        stp     x21, x12, [sp, #160]+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #176]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #144]+        ldp     x21, x12, [sp, #160]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #176]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x21+        eor     x1, x22, x9+        adcs    x24, x23, x12+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x21+        adcs    x15, x17, x12+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x22, x6, x20+        eor     x3, x20, x23+        adcs    x5, x7, x3+        adcs    x14, x24, x11+        stp     x22, x5, [sp, #144]+        adcs    x5, x13, x23+        adcs    x21, x8, x23+        stp     x14, x5, [sp, #160]+        adc     x12, x15, x23+        stp     x21, x12, [sp, #176]+        ldr     q3, [sp, #240]+        ldr     q25, [x26, #96]+        ldp     x13, x23, [x26, #96]+        ldp     x3, x21, [sp, #240]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [x26, #128]+        ldp     x8, x24, [sp, #256]+        subs    x6, x3, x21+        ldr     q0, [sp, #272]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [x26, #112]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [x26, #128]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [sp, #272]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x16, x4, x16+        mov     x4, v27.d[0]+        sbcs    x11, x20, x11+        sbcs    x20, x9, x12+        stp     x16, x11, [sp, #240]+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #256]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #272]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        ldp     x20, x9, [sp, #240]+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #256]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #272]+        adds    x20, x22, x20+        mul     x10, x13, x14+        adcs    x11, x11, x9+        eor     x9, x8, x21+        adcs    x21, x19, x17+        stp     x20, x11, [sp, #240]+        adcs    x12, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        stp     x21, x12, [sp, #256]+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #272]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #240]+        ldp     x21, x12, [sp, #256]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #272]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x21+        eor     x1, x22, x9+        adcs    x24, x23, x12+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x21+        adcs    x15, x17, x12+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x22, x6, x20+        eor     x3, x20, x23+        adcs    x5, x7, x3+        adcs    x14, x24, x11+        stp     x22, x5, [sp, #240]+        adcs    x5, x13, x23+        adcs    x21, x8, x23+        stp     x14, x5, [sp, #256]+        adc     x12, x15, x23+        stp     x21, x12, [sp, #272]+        ldp     x2, x27, [sp, #0x150]+        ldr     q3, [sp, #48]+        ldr     q25, [sp, #192]+        ldp     x13, x23, [sp, #192]+        ldp     x3, x21, [sp, #48]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [sp, #224]+        ldp     x8, x24, [sp, #64]+        subs    x6, x3, x21+        ldr     q0, [sp, #80]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [sp, #208]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [sp, #224]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [sp, #80]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x16, x4, x16+        mov     x4, v27.d[0]+        sbcs    x11, x20, x11+        sbcs    x20, x9, x12+        stp     x16, x11, [sp, #192]+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #208]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #224]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        ldp     x20, x9, [sp, #192]+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #208]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #224]+        adds    x20, x22, x20+        mul     x10, x13, x14+        adcs    x11, x11, x9+        eor     x9, x8, x21+        adcs    x21, x19, x17+        stp     x20, x11, [sp, #192]+        adcs    x12, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        stp     x21, x12, [sp, #208]+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #224]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #192]+        ldp     x21, x12, [sp, #208]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #224]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x21+        eor     x1, x22, x9+        adcs    x24, x23, x12+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x21+        adcs    x15, x17, x12+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x2, x6, x20+        eor     x3, x20, x23+        adcs    x6, x7, x3+        adcs    x7, x24, x11+        adcs    x9, x13, x23+        adcs    x10, x8, x23+        adc     x11, x15, x23+        ldp     x4, x3, [sp, #144]+        subs    x5, x2, x4+        sbcs    x6, x6, x3+        ldp     x4, x3, [sp, #160]+        sbcs    x7, x7, x4+        sbcs    x8, x9, x3+        ldp     x4, x3, [sp, #176]+        sbcs    x9, x10, x4+        sbcs    x10, x11, x3+        csetm   x3, cc  // cc = lo, ul, last+        mov     x4, #0xffffffff                 // #4294967295+        and     x4, x4, x3+        adds    x19, x5, x4+        eor     x4, x4, x3+        adcs    x24, x6, x4+        mov     x4, #0xfffffffffffffffe         // #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x7, x8, [sp, #208]+        stp     x9, x10, [sp, #224]+        ldp     x0, x1, [x25, #96]+        ldp     x2, x3, [x25, #112]+        ldp     x4, x5, [x25, #128]+        orr     x20, x0, x1+        orr     x21, x2, x3+        orr     x22, x4, x5+        orr     x20, x20, x21+        orr     x20, x20, x22+        cmp     x20, xzr+        cset    x20, ne  // ne = any+        ldp     x6, x7, [x26, #96]+        ldp     x8, x9, [x26, #112]+        ldp     x10, x11, [x26, #128]+        orr     x21, x6, x7+        orr     x22, x8, x9+        orr     x23, x10, x11+        orr     x21, x21, x22+        orr     x21, x21, x23+        cmp     x21, xzr+        cset    x21, ne  // ne = any+        cmp     x21, x20+        ldp     x12, x13, [sp, #240]+        csel    x12, x0, x12, cc  // cc = lo, ul, last+        csel    x13, x1, x13, cc  // cc = lo, ul, last+        csel    x12, x6, x12, hi  // hi = pmore+        csel    x13, x7, x13, hi  // hi = pmore+        ldp     x14, x15, [sp, #256]+        csel    x14, x2, x14, cc  // cc = lo, ul, last+        csel    x15, x3, x15, cc  // cc = lo, ul, last+        csel    x14, x8, x14, hi  // hi = pmore+        csel    x15, x9, x15, hi  // hi = pmore+        ldp     x16, x17, [sp, #272]+        csel    x16, x4, x16, cc  // cc = lo, ul, last+        csel    x17, x5, x17, cc  // cc = lo, ul, last+        csel    x16, x10, x16, hi  // hi = pmore+        csel    x17, x11, x17, hi  // hi = pmore+        ldp     x20, x21, [x25]+        ldp     x0, x1, [sp]+        csel    x0, x20, x0, cc  // cc = lo, ul, last+        csel    x1, x21, x1, cc  // cc = lo, ul, last+        ldp     x20, x21, [x26]+        csel    x0, x20, x0, hi  // hi = pmore+        csel    x1, x21, x1, hi  // hi = pmore+        ldp     x20, x21, [x25, #16]+        ldp     x2, x3, [sp, #16]+        csel    x2, x20, x2, cc  // cc = lo, ul, last+        csel    x3, x21, x3, cc  // cc = lo, ul, last+        ldp     x20, x21, [x26, #16]+        csel    x2, x20, x2, hi  // hi = pmore+        csel    x3, x21, x3, hi  // hi = pmore+        ldp     x20, x21, [x25, #32]+        ldp     x4, x5, [sp, #32]+        csel    x4, x20, x4, cc  // cc = lo, ul, last+        csel    x5, x21, x5, cc  // cc = lo, ul, last+        ldp     x20, x21, [x26, #32]+        csel    x4, x20, x4, hi  // hi = pmore+        csel    x5, x21, x5, hi  // hi = pmore+        ldp     x20, x21, [x25, #48]+        csel    x6, x20, x19, cc  // cc = lo, ul, last+        csel    x7, x21, x24, cc  // cc = lo, ul, last+        ldp     x20, x21, [x26, #48]+        csel    x6, x20, x6, hi  // hi = pmore+        csel    x7, x21, x7, hi  // hi = pmore+        ldp     x20, x21, [x25, #64]+        ldp     x8, x9, [sp, #208]+        csel    x8, x20, x8, cc  // cc = lo, ul, last+        csel    x9, x21, x9, cc  // cc = lo, ul, last+        ldp     x20, x21, [x26, #64]+        csel    x8, x20, x8, hi  // hi = pmore+        csel    x9, x21, x9, hi  // hi = pmore+        ldp     x20, x21, [x25, #80]+        ldp     x10, x11, [sp, #224]+        csel    x10, x20, x10, cc  // cc = lo, ul, last+        csel    x11, x21, x11, cc  // cc = lo, ul, last+        ldp     x20, x21, [x26, #80]+        csel    x10, x20, x10, hi  // hi = pmore+        csel    x11, x21, x11, hi  // hi = pmore+        stp     x0, x1, [x27]+        stp     x2, x3, [x27, #16]+        stp     x4, x5, [x27, #32]+        stp     x6, x7, [x27, #48]+        stp     x8, x9, [x27, #64]+        stp     x10, x11, [x27, #80]+        stp     x12, x13, [x27, #96]+        stp     x14, x15, [x27, #112]+        stp     x16, x17, [x27, #128]+        CFI_INC_SP(384)+        CFI_POP1Z(x27)+        CFI_POP2(x25,x26)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp384_montjscalarmul_p384_montjadd)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp384_montjscalarmul_p384_montjdouble)++Lp384_montjscalarmul_p384_montjdouble:+        CFI_START+        CFI_DEC_SP(416)+        stp     x19, x20, [sp, #336]+        stp     x21, x22, [sp, #352]+        stp     x23, x24, [sp, #368]+        stp     x25, x26, [sp, #384]+        stp     x27, xzr, [sp, #400]+        mov     x25, x0+        mov     x26, x1+        mov     x0, sp+        ldr     q1, [x26, #96]+        ldp     x9, x2, [x26, #96]+        ldr     q0, [x26, #96]+        ldp     x4, x6, [x26, #112]+        rev64   v21.4s, v1.4s+        uzp2    v28.4s, v1.4s, v1.4s+        umulh   x7, x9, x2+        xtn     v17.2s, v1.2d+        mul     v27.4s, v21.4s, v0.4s+        ldr     q20, [x26, #128]+        xtn     v30.2s, v0.2d+        ldr     q1, [x26, #128]+        uzp2    v31.4s, v0.4s, v0.4s+        ldp     x5, x10, [x26, #128]+        umulh   x8, x9, x4+        uaddlp  v3.2d, v27.4s+        umull   v16.2d, v30.2s, v17.2s+        mul     x16, x9, x4+        umull   v27.2d, v30.2s, v28.2s+        shrn    v0.2s, v20.2d, #32+        xtn     v7.2s, v20.2d+        shl     v20.2d, v3.2d, #32+        umull   v3.2d, v31.2s, v28.2s+        mul     x3, x2, x4+        umlal   v20.2d, v30.2s, v17.2s+        umull   v22.2d, v7.2s, v0.2s+        usra    v27.2d, v16.2d, #32+        umulh   x11, x2, x4+        movi    v21.2d, #0xffffffff+        uzp2    v28.4s, v1.4s, v1.4s+        adds    x15, x16, x7+        and     v5.16b, v27.16b, v21.16b+        adcs    x3, x3, x8+        usra    v3.2d, v27.2d, #32+        dup     v29.2d, x6+        adcs    x16, x11, xzr+        mov     x14, v20.d[0]+        umlal   v5.2d, v31.2s, v17.2s+        mul     x8, x9, x2+        mov     x7, v20.d[1]+        shl     v19.2d, v22.2d, #33+        xtn     v25.2s, v29.2d+        rev64   v31.4s, v1.4s+        lsl     x13, x14, #32+        uzp2    v6.4s, v29.4s, v29.4s+        umlal   v19.2d, v7.2s, v7.2s+        usra    v3.2d, v5.2d, #32+        adds    x1, x8, x8+        umulh   x8, x4, x4+        add     x12, x13, x14+        mul     v17.4s, v31.4s, v29.4s+        xtn     v4.2s, v1.2d+        adcs    x14, x15, x15+        lsr     x13, x12, #32+        adcs    x15, x3, x3+        umull   v31.2d, v25.2s, v28.2s+        adcs    x11, x16, x16+        umull   v21.2d, v25.2s, v4.2s+        mov     x17, v3.d[0]+        umull   v18.2d, v6.2s, v28.2s+        adc     x16, x8, xzr+        uaddlp  v16.2d, v17.4s+        movi    v1.2d, #0xffffffff+        subs    x13, x13, x12+        usra    v31.2d, v21.2d, #32+        sbc     x8, x12, xzr+        adds    x17, x17, x1+        mul     x1, x4, x4+        shl     v28.2d, v16.2d, #32+        mov     x3, v3.d[1]+        adcs    x14, x7, x14+        extr    x7, x8, x13, #32+        adcs    x13, x3, x15+        and     v3.16b, v31.16b, v1.16b+        adcs    x11, x1, x11+        lsr     x1, x8, #32+        umlal   v3.2d, v6.2s, v4.2s+        usra    v18.2d, v31.2d, #32+        adc     x3, x16, xzr+        adds    x1, x1, x12+        umlal   v28.2d, v25.2s, v4.2s+        adc     x16, xzr, xzr+        subs    x15, x17, x7+        sbcs    x7, x14, x1+        lsl     x1, x15, #32+        sbcs    x16, x13, x16+        add     x8, x1, x15+        usra    v18.2d, v3.2d, #32+        sbcs    x14, x11, xzr+        lsr     x1, x8, #32+        sbcs    x17, x3, xzr+        sbc     x11, x12, xzr+        subs    x13, x1, x8+        umulh   x12, x4, x10+        sbc     x1, x8, xzr+        extr    x13, x1, x13, #32+        lsr     x1, x1, #32+        adds    x15, x1, x8+        adc     x1, xzr, xzr+        subs    x7, x7, x13+        sbcs    x13, x16, x15+        lsl     x3, x7, #32+        umulh   x16, x2, x5+        sbcs    x15, x14, x1+        add     x7, x3, x7+        sbcs    x3, x17, xzr+        lsr     x1, x7, #32+        sbcs    x14, x11, xzr+        sbc     x11, x8, xzr+        subs    x8, x1, x7+        sbc     x1, x7, xzr+        extr    x8, x1, x8, #32+        lsr     x1, x1, #32+        adds    x1, x1, x7+        adc     x17, xzr, xzr+        subs    x13, x13, x8+        umulh   x8, x9, x6+        sbcs    x1, x15, x1+        sbcs    x15, x3, x17+        sbcs    x3, x14, xzr+        mul     x17, x2, x5+        sbcs    x11, x11, xzr+        stp     x13, x1, [x0]+        sbc     x14, x7, xzr+        mul     x7, x4, x10+        subs    x1, x9, x2+        stp     x15, x3, [x0, #16]+        csetm   x15, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        stp     x11, x14, [x0, #32]+        mul     x14, x9, x6+        adds    x17, x8, x17+        adcs    x7, x16, x7+        adc     x13, x12, xzr+        subs    x12, x5, x6+        cneg    x3, x12, cc  // cc = lo, ul, last+        cinv    x16, x15, cc  // cc = lo, ul, last+        mul     x8, x1, x3+        umulh   x1, x1, x3+        eor     x12, x8, x16+        adds    x11, x17, x14+        adcs    x3, x7, x17+        adcs    x15, x13, x7+        adc     x8, x13, xzr+        adds    x3, x3, x14+        adcs    x15, x15, x17+        adcs    x17, x8, x7+        eor     x1, x1, x16+        adc     x13, x13, xzr+        subs    x9, x9, x4+        csetm   x8, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x4, x2, x4+        cneg    x4, x4, cc  // cc = lo, ul, last+        csetm   x7, cc  // cc = lo, ul, last+        subs    x2, x10, x6+        cinv    x8, x8, cc  // cc = lo, ul, last+        cneg    x2, x2, cc  // cc = lo, ul, last+        cmn     x16, #0x1+        adcs    x11, x11, x12+        mul     x12, x9, x2+        adcs    x3, x3, x1+        adcs    x15, x15, x16+        umulh   x9, x9, x2+        adcs    x17, x17, x16+        adc     x13, x13, x16+        subs    x1, x10, x5+        cinv    x2, x7, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        eor     x9, x9, x8+        cmn     x8, #0x1+        eor     x7, x12, x8+        mul     x12, x4, x1+        adcs    x3, x3, x7+        adcs    x7, x15, x9+        adcs    x15, x17, x8+        ldp     x9, x17, [x0, #16]+        umulh   x4, x4, x1+        adc     x8, x13, x8+        cmn     x2, #0x1+        eor     x1, x12, x2+        adcs    x1, x7, x1+        ldp     x7, x16, [x0]+        eor     x12, x4, x2+        adcs    x4, x15, x12+        ldp     x15, x12, [x0, #32]+        adc     x8, x8, x2+        adds    x13, x14, x14+        umulh   x14, x5, x10+        adcs    x2, x11, x11+        adcs    x3, x3, x3+        adcs    x1, x1, x1+        adcs    x4, x4, x4+        adcs    x11, x8, x8+        adc     x8, xzr, xzr+        adds    x13, x13, x7+        adcs    x2, x2, x16+        mul     x16, x5, x10+        adcs    x3, x3, x9+        adcs    x1, x1, x17+        umulh   x5, x5, x5+        lsl     x9, x13, #32+        add     x9, x9, x13+        adcs    x4, x4, x15+        mov     x13, v28.d[1]+        adcs    x15, x11, x12+        lsr     x7, x9, #32+        adc     x11, x8, xzr+        subs    x7, x7, x9+        umulh   x10, x10, x10+        sbc     x17, x9, xzr+        extr    x7, x17, x7, #32+        lsr     x17, x17, #32+        adds    x17, x17, x9+        adc     x12, xzr, xzr+        subs    x8, x2, x7+        sbcs    x17, x3, x17+        lsl     x7, x8, #32+        sbcs    x2, x1, x12+        add     x3, x7, x8+        sbcs    x12, x4, xzr+        lsr     x1, x3, #32+        sbcs    x7, x15, xzr+        sbc     x15, x9, xzr+        subs    x1, x1, x3+        sbc     x4, x3, xzr+        lsr     x9, x4, #32+        extr    x8, x4, x1, #32+        adds    x9, x9, x3+        adc     x4, xzr, xzr+        subs    x1, x17, x8+        lsl     x17, x1, #32+        sbcs    x8, x2, x9+        sbcs    x9, x12, x4+        add     x17, x17, x1+        mov     x1, v18.d[1]+        lsr     x2, x17, #32+        sbcs    x7, x7, xzr+        mov     x12, v18.d[0]+        sbcs    x15, x15, xzr+        sbc     x3, x3, xzr+        subs    x4, x2, x17+        sbc     x2, x17, xzr+        adds    x12, x13, x12+        adcs    x16, x16, x1+        lsr     x13, x2, #32+        extr    x1, x2, x4, #32+        adc     x2, x14, xzr+        adds    x4, x13, x17+        mul     x13, x6, x6+        adc     x14, xzr, xzr+        subs    x1, x8, x1+        sbcs    x4, x9, x4+        mov     x9, v28.d[0]+        sbcs    x7, x7, x14+        sbcs    x8, x15, xzr+        sbcs    x3, x3, xzr+        sbc     x14, x17, xzr+        adds    x17, x9, x9+        adcs    x12, x12, x12+        mov     x15, v19.d[0]+        adcs    x9, x16, x16+        umulh   x6, x6, x6+        adcs    x16, x2, x2+        adc     x2, xzr, xzr+        adds    x11, x11, x8+        adcs    x3, x3, xzr+        adcs    x14, x14, xzr+        adcs    x8, xzr, xzr+        adds    x13, x1, x13+        mov     x1, v19.d[1]+        adcs    x6, x4, x6+        mov     x4, #0xffffffff                 // #4294967295+        adcs    x15, x7, x15+        adcs    x7, x11, x5+        adcs    x1, x3, x1+        adcs    x14, x14, x10+        adc     x11, x8, xzr+        adds    x6, x6, x17+        adcs    x8, x15, x12+        adcs    x3, x7, x9+        adcs    x15, x1, x16+        mov     x16, #0xffffffff00000001        // #-4294967295+        adcs    x14, x14, x2+        mov     x2, #0x1                        // #1+        adc     x17, x11, xzr+        cmn     x13, x16+        adcs    xzr, x6, x4+        adcs    xzr, x8, x2+        adcs    xzr, x3, xzr+        adcs    xzr, x15, xzr+        adcs    xzr, x14, xzr+        adc     x1, x17, xzr+        neg     x9, x1+        and     x1, x16, x9+        adds    x11, x13, x1+        and     x13, x4, x9+        adcs    x5, x6, x13+        and     x1, x2, x9+        adcs    x7, x8, x1+        stp     x11, x5, [x0]+        adcs    x11, x3, xzr+        adcs    x2, x15, xzr+        stp     x7, x11, [x0, #16]+        adc     x17, x14, xzr+        stp     x2, x17, [x0, #32]+        ldr     q1, [x26, #48]+        ldp     x9, x2, [x26, #48]+        ldr     q0, [x26, #48]+        ldp     x4, x6, [x26, #64]+        rev64   v21.4s, v1.4s+        uzp2    v28.4s, v1.4s, v1.4s+        umulh   x7, x9, x2+        xtn     v17.2s, v1.2d+        mul     v27.4s, v21.4s, v0.4s+        ldr     q20, [x26, #80]+        xtn     v30.2s, v0.2d+        ldr     q1, [x26, #80]+        uzp2    v31.4s, v0.4s, v0.4s+        ldp     x5, x10, [x26, #80]+        umulh   x8, x9, x4+        uaddlp  v3.2d, v27.4s+        umull   v16.2d, v30.2s, v17.2s+        mul     x16, x9, x4+        umull   v27.2d, v30.2s, v28.2s+        shrn    v0.2s, v20.2d, #32+        xtn     v7.2s, v20.2d+        shl     v20.2d, v3.2d, #32+        umull   v3.2d, v31.2s, v28.2s+        mul     x3, x2, x4+        umlal   v20.2d, v30.2s, v17.2s+        umull   v22.2d, v7.2s, v0.2s+        usra    v27.2d, v16.2d, #32+        umulh   x11, x2, x4+        movi    v21.2d, #0xffffffff+        uzp2    v28.4s, v1.4s, v1.4s+        adds    x15, x16, x7+        and     v5.16b, v27.16b, v21.16b+        adcs    x3, x3, x8+        usra    v3.2d, v27.2d, #32+        dup     v29.2d, x6+        adcs    x16, x11, xzr+        mov     x14, v20.d[0]+        umlal   v5.2d, v31.2s, v17.2s+        mul     x8, x9, x2+        mov     x7, v20.d[1]+        shl     v19.2d, v22.2d, #33+        xtn     v25.2s, v29.2d+        rev64   v31.4s, v1.4s+        lsl     x13, x14, #32+        uzp2    v6.4s, v29.4s, v29.4s+        umlal   v19.2d, v7.2s, v7.2s+        usra    v3.2d, v5.2d, #32+        adds    x1, x8, x8+        umulh   x8, x4, x4+        add     x12, x13, x14+        mul     v17.4s, v31.4s, v29.4s+        xtn     v4.2s, v1.2d+        adcs    x14, x15, x15+        lsr     x13, x12, #32+        adcs    x15, x3, x3+        umull   v31.2d, v25.2s, v28.2s+        adcs    x11, x16, x16+        umull   v21.2d, v25.2s, v4.2s+        mov     x17, v3.d[0]+        umull   v18.2d, v6.2s, v28.2s+        adc     x16, x8, xzr+        uaddlp  v16.2d, v17.4s+        movi    v1.2d, #0xffffffff+        subs    x13, x13, x12+        usra    v31.2d, v21.2d, #32+        sbc     x8, x12, xzr+        adds    x17, x17, x1+        mul     x1, x4, x4+        shl     v28.2d, v16.2d, #32+        mov     x3, v3.d[1]+        adcs    x14, x7, x14+        extr    x7, x8, x13, #32+        adcs    x13, x3, x15+        and     v3.16b, v31.16b, v1.16b+        adcs    x11, x1, x11+        lsr     x1, x8, #32+        umlal   v3.2d, v6.2s, v4.2s+        usra    v18.2d, v31.2d, #32+        adc     x3, x16, xzr+        adds    x1, x1, x12+        umlal   v28.2d, v25.2s, v4.2s+        adc     x16, xzr, xzr+        subs    x15, x17, x7+        sbcs    x7, x14, x1+        lsl     x1, x15, #32+        sbcs    x16, x13, x16+        add     x8, x1, x15+        usra    v18.2d, v3.2d, #32+        sbcs    x14, x11, xzr+        lsr     x1, x8, #32+        sbcs    x17, x3, xzr+        sbc     x11, x12, xzr+        subs    x13, x1, x8+        umulh   x12, x4, x10+        sbc     x1, x8, xzr+        extr    x13, x1, x13, #32+        lsr     x1, x1, #32+        adds    x15, x1, x8+        adc     x1, xzr, xzr+        subs    x7, x7, x13+        sbcs    x13, x16, x15+        lsl     x3, x7, #32+        umulh   x16, x2, x5+        sbcs    x15, x14, x1+        add     x7, x3, x7+        sbcs    x3, x17, xzr+        lsr     x1, x7, #32+        sbcs    x14, x11, xzr+        sbc     x11, x8, xzr+        subs    x8, x1, x7+        sbc     x1, x7, xzr+        extr    x8, x1, x8, #32+        lsr     x1, x1, #32+        adds    x1, x1, x7+        adc     x17, xzr, xzr+        subs    x13, x13, x8+        umulh   x8, x9, x6+        sbcs    x1, x15, x1+        sbcs    x15, x3, x17+        sbcs    x3, x14, xzr+        mul     x17, x2, x5+        sbcs    x11, x11, xzr+        stp     x13, x1, [sp, #48]+        sbc     x14, x7, xzr+        mul     x7, x4, x10+        subs    x1, x9, x2+        stp     x15, x3, [sp, #64]+        csetm   x15, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        stp     x11, x14, [sp, #80]+        mul     x14, x9, x6+        adds    x17, x8, x17+        adcs    x7, x16, x7+        adc     x13, x12, xzr+        subs    x12, x5, x6+        cneg    x3, x12, cc  // cc = lo, ul, last+        cinv    x16, x15, cc  // cc = lo, ul, last+        mul     x8, x1, x3+        umulh   x1, x1, x3+        eor     x12, x8, x16+        adds    x11, x17, x14+        adcs    x3, x7, x17+        adcs    x15, x13, x7+        adc     x8, x13, xzr+        adds    x3, x3, x14+        adcs    x15, x15, x17+        adcs    x17, x8, x7+        eor     x1, x1, x16+        adc     x13, x13, xzr+        subs    x9, x9, x4+        csetm   x8, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x4, x2, x4+        cneg    x4, x4, cc  // cc = lo, ul, last+        csetm   x7, cc  // cc = lo, ul, last+        subs    x2, x10, x6+        cinv    x8, x8, cc  // cc = lo, ul, last+        cneg    x2, x2, cc  // cc = lo, ul, last+        cmn     x16, #0x1+        adcs    x11, x11, x12+        mul     x12, x9, x2+        adcs    x3, x3, x1+        adcs    x15, x15, x16+        umulh   x9, x9, x2+        adcs    x17, x17, x16+        adc     x13, x13, x16+        subs    x1, x10, x5+        cinv    x2, x7, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        eor     x9, x9, x8+        cmn     x8, #0x1+        eor     x7, x12, x8+        mul     x12, x4, x1+        adcs    x3, x3, x7+        adcs    x7, x15, x9+        adcs    x15, x17, x8+        ldp     x9, x17, [sp, #64]+        umulh   x4, x4, x1+        adc     x8, x13, x8+        cmn     x2, #0x1+        eor     x1, x12, x2+        adcs    x1, x7, x1+        ldp     x7, x16, [sp, #48]+        eor     x12, x4, x2+        adcs    x4, x15, x12+        ldp     x15, x12, [sp, #80]+        adc     x8, x8, x2+        adds    x13, x14, x14+        umulh   x14, x5, x10+        adcs    x2, x11, x11+        adcs    x3, x3, x3+        adcs    x1, x1, x1+        adcs    x4, x4, x4+        adcs    x11, x8, x8+        adc     x8, xzr, xzr+        adds    x13, x13, x7+        adcs    x2, x2, x16+        mul     x16, x5, x10+        adcs    x3, x3, x9+        adcs    x1, x1, x17+        umulh   x5, x5, x5+        lsl     x9, x13, #32+        add     x9, x9, x13+        adcs    x4, x4, x15+        mov     x13, v28.d[1]+        adcs    x15, x11, x12+        lsr     x7, x9, #32+        adc     x11, x8, xzr+        subs    x7, x7, x9+        umulh   x10, x10, x10+        sbc     x17, x9, xzr+        extr    x7, x17, x7, #32+        lsr     x17, x17, #32+        adds    x17, x17, x9+        adc     x12, xzr, xzr+        subs    x8, x2, x7+        sbcs    x17, x3, x17+        lsl     x7, x8, #32+        sbcs    x2, x1, x12+        add     x3, x7, x8+        sbcs    x12, x4, xzr+        lsr     x1, x3, #32+        sbcs    x7, x15, xzr+        sbc     x15, x9, xzr+        subs    x1, x1, x3+        sbc     x4, x3, xzr+        lsr     x9, x4, #32+        extr    x8, x4, x1, #32+        adds    x9, x9, x3+        adc     x4, xzr, xzr+        subs    x1, x17, x8+        lsl     x17, x1, #32+        sbcs    x8, x2, x9+        sbcs    x9, x12, x4+        add     x17, x17, x1+        mov     x1, v18.d[1]+        lsr     x2, x17, #32+        sbcs    x7, x7, xzr+        mov     x12, v18.d[0]+        sbcs    x15, x15, xzr+        sbc     x3, x3, xzr+        subs    x4, x2, x17+        sbc     x2, x17, xzr+        adds    x12, x13, x12+        adcs    x16, x16, x1+        lsr     x13, x2, #32+        extr    x1, x2, x4, #32+        adc     x2, x14, xzr+        adds    x4, x13, x17+        mul     x13, x6, x6+        adc     x14, xzr, xzr+        subs    x1, x8, x1+        sbcs    x4, x9, x4+        mov     x9, v28.d[0]+        sbcs    x7, x7, x14+        sbcs    x8, x15, xzr+        sbcs    x3, x3, xzr+        sbc     x14, x17, xzr+        adds    x17, x9, x9+        adcs    x12, x12, x12+        mov     x15, v19.d[0]+        adcs    x9, x16, x16+        umulh   x6, x6, x6+        adcs    x16, x2, x2+        adc     x2, xzr, xzr+        adds    x11, x11, x8+        adcs    x3, x3, xzr+        adcs    x14, x14, xzr+        adcs    x8, xzr, xzr+        adds    x13, x1, x13+        mov     x1, v19.d[1]+        adcs    x6, x4, x6+        mov     x4, #0xffffffff                 // #4294967295+        adcs    x15, x7, x15+        adcs    x7, x11, x5+        adcs    x1, x3, x1+        adcs    x14, x14, x10+        adc     x11, x8, xzr+        adds    x6, x6, x17+        adcs    x8, x15, x12+        adcs    x3, x7, x9+        adcs    x15, x1, x16+        mov     x16, #0xffffffff00000001        // #-4294967295+        adcs    x14, x14, x2+        mov     x2, #0x1                        // #1+        adc     x17, x11, xzr+        cmn     x13, x16+        adcs    xzr, x6, x4+        adcs    xzr, x8, x2+        adcs    xzr, x3, xzr+        adcs    xzr, x15, xzr+        adcs    xzr, x14, xzr+        adc     x1, x17, xzr+        neg     x9, x1+        and     x1, x16, x9+        adds    x11, x13, x1+        and     x13, x4, x9+        adcs    x5, x6, x13+        and     x1, x2, x9+        adcs    x7, x8, x1+        stp     x11, x5, [sp, #48]+        adcs    x11, x3, xzr+        adcs    x2, x15, xzr+        stp     x7, x11, [sp, #64]+        adc     x17, x14, xzr+        stp     x2, x17, [sp, #80]+        ldp     x5, x6, [x26]+        ldp     x4, x3, [sp]+        adds    x5, x5, x4+        adcs    x6, x6, x3+        ldp     x7, x8, [x26, #16]+        ldp     x4, x3, [sp, #16]+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        ldp     x9, x10, [x26, #32]+        ldp     x4, x3, [sp, #32]+        adcs    x9, x9, x4+        adcs    x10, x10, x3+        csetm   x3, cs  // cs = hs, nlast+        mov     x4, #0xffffffff                 // #4294967295+        and     x4, x4, x3+        subs    x5, x5, x4+        eor     x4, x4, x3+        sbcs    x6, x6, x4+        mov     x4, #0xfffffffffffffffe         // #-2+        and     x4, x4, x3+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        sbcs    x9, x9, x3+        sbc     x10, x10, x3+        stp     x5, x6, [sp, #240]+        stp     x7, x8, [sp, #256]+        stp     x9, x10, [sp, #272]+        mov     x2, sp+        ldp     x5, x6, [x26]+        ldp     x4, x3, [x2]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [x26, #16]+        ldp     x4, x3, [x2, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [x26, #32]+        ldp     x4, x3, [x2, #32]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, cc  // cc = lo, ul, last+        mov     x4, #0xffffffff                 // #4294967295+        and     x4, x4, x3+        adds    x13, x5, x4+        eor     x4, x4, x3+        adcs    x23, x6, x4+        mov     x4, #0xfffffffffffffffe         // #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x13, x23, [sp, #192]+        stp     x7, x8, [sp, #208]+        stp     x9, x10, [sp, #224]+        ldr     q3, [sp, #240]+        ldr     q25, [sp, #192]+        ldp     x3, x21, [sp, #240]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [sp, #224]+        ldp     x8, x24, [sp, #256]+        subs    x6, x3, x21+        ldr     q0, [sp, #272]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [sp, #208]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [sp, #224]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [sp, #272]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x16, x4, x16+        mov     x4, v27.d[0]+        sbcs    x11, x20, x11+        sbcs    x20, x9, x12+        stp     x16, x11, [sp, #96]+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #112]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #128]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        ldp     x20, x9, [sp, #96]+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #112]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #128]+        adds    x20, x22, x20+        mul     x10, x13, x14+        adcs    x11, x11, x9+        eor     x9, x8, x21+        adcs    x21, x19, x17+        stp     x20, x11, [sp, #96]+        adcs    x12, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        stp     x21, x12, [sp, #112]+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #128]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #96]+        ldp     x21, x12, [sp, #112]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #128]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x21+        eor     x1, x22, x9+        adcs    x24, x23, x12+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x21+        adcs    x15, x17, x12+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x22, x6, x20+        eor     x3, x20, x23+        adcs    x5, x7, x3+        adcs    x14, x24, x11+        stp     x22, x5, [sp, #96]+        adcs    x5, x13, x23+        adcs    x21, x8, x23+        stp     x14, x5, [sp, #112]+        adc     x12, x15, x23+        stp     x21, x12, [sp, #128]+        ldp     x5, x6, [x26, #48]+        ldp     x4, x3, [x26, #96]+        adds    x5, x5, x4+        adcs    x6, x6, x3+        ldp     x7, x8, [x26, #64]+        ldp     x4, x3, [x26, #112]+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        ldp     x9, x10, [x26, #80]+        ldp     x4, x3, [x26, #128]+        adcs    x9, x9, x4+        adcs    x10, x10, x3+        adc     x3, xzr, xzr+        mov     x4, #0xffffffff                 // #4294967295+        cmp     x5, x4+        mov     x4, #0xffffffff00000000         // #-4294967296+        sbcs    xzr, x6, x4+        mov     x4, #0xfffffffffffffffe         // #-2+        sbcs    xzr, x7, x4+        adcs    xzr, x8, xzr+        adcs    xzr, x9, xzr+        adcs    xzr, x10, xzr+        adcs    x3, x3, xzr+        csetm   x3, ne  // ne = any+        mov     x4, #0xffffffff                 // #4294967295+        and     x4, x4, x3+        subs    x5, x5, x4+        eor     x4, x4, x3+        sbcs    x6, x6, x4+        mov     x4, #0xfffffffffffffffe         // #-2+        and     x4, x4, x3+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        sbcs    x9, x9, x3+        sbc     x10, x10, x3+        stp     x5, x6, [sp, #240]+        stp     x7, x8, [sp, #256]+        stp     x9, x10, [sp, #272]+        ldr     q1, [sp, #96]+        ldp     x9, x2, [sp, #96]+        ldr     q0, [sp, #96]+        ldp     x4, x6, [sp, #112]+        rev64   v21.4s, v1.4s+        uzp2    v28.4s, v1.4s, v1.4s+        umulh   x7, x9, x2+        xtn     v17.2s, v1.2d+        mul     v27.4s, v21.4s, v0.4s+        ldr     q20, [sp, #128]+        xtn     v30.2s, v0.2d+        ldr     q1, [sp, #128]+        uzp2    v31.4s, v0.4s, v0.4s+        ldp     x5, x10, [sp, #128]+        umulh   x8, x9, x4+        uaddlp  v3.2d, v27.4s+        umull   v16.2d, v30.2s, v17.2s+        mul     x16, x9, x4+        umull   v27.2d, v30.2s, v28.2s+        shrn    v0.2s, v20.2d, #32+        xtn     v7.2s, v20.2d+        shl     v20.2d, v3.2d, #32+        umull   v3.2d, v31.2s, v28.2s+        mul     x3, x2, x4+        umlal   v20.2d, v30.2s, v17.2s+        umull   v22.2d, v7.2s, v0.2s+        usra    v27.2d, v16.2d, #32+        umulh   x11, x2, x4+        movi    v21.2d, #0xffffffff+        uzp2    v28.4s, v1.4s, v1.4s+        adds    x15, x16, x7+        and     v5.16b, v27.16b, v21.16b+        adcs    x3, x3, x8+        usra    v3.2d, v27.2d, #32+        dup     v29.2d, x6+        adcs    x16, x11, xzr+        mov     x14, v20.d[0]+        umlal   v5.2d, v31.2s, v17.2s+        mul     x8, x9, x2+        mov     x7, v20.d[1]+        shl     v19.2d, v22.2d, #33+        xtn     v25.2s, v29.2d+        rev64   v31.4s, v1.4s+        lsl     x13, x14, #32+        uzp2    v6.4s, v29.4s, v29.4s+        umlal   v19.2d, v7.2s, v7.2s+        usra    v3.2d, v5.2d, #32+        adds    x1, x8, x8+        umulh   x8, x4, x4+        add     x12, x13, x14+        mul     v17.4s, v31.4s, v29.4s+        xtn     v4.2s, v1.2d+        adcs    x14, x15, x15+        lsr     x13, x12, #32+        adcs    x15, x3, x3+        umull   v31.2d, v25.2s, v28.2s+        adcs    x11, x16, x16+        umull   v21.2d, v25.2s, v4.2s+        mov     x17, v3.d[0]+        umull   v18.2d, v6.2s, v28.2s+        adc     x16, x8, xzr+        uaddlp  v16.2d, v17.4s+        movi    v1.2d, #0xffffffff+        subs    x13, x13, x12+        usra    v31.2d, v21.2d, #32+        sbc     x8, x12, xzr+        adds    x17, x17, x1+        mul     x1, x4, x4+        shl     v28.2d, v16.2d, #32+        mov     x3, v3.d[1]+        adcs    x14, x7, x14+        extr    x7, x8, x13, #32+        adcs    x13, x3, x15+        and     v3.16b, v31.16b, v1.16b+        adcs    x11, x1, x11+        lsr     x1, x8, #32+        umlal   v3.2d, v6.2s, v4.2s+        usra    v18.2d, v31.2d, #32+        adc     x3, x16, xzr+        adds    x1, x1, x12+        umlal   v28.2d, v25.2s, v4.2s+        adc     x16, xzr, xzr+        subs    x15, x17, x7+        sbcs    x7, x14, x1+        lsl     x1, x15, #32+        sbcs    x16, x13, x16+        add     x8, x1, x15+        usra    v18.2d, v3.2d, #32+        sbcs    x14, x11, xzr+        lsr     x1, x8, #32+        sbcs    x17, x3, xzr+        sbc     x11, x12, xzr+        subs    x13, x1, x8+        umulh   x12, x4, x10+        sbc     x1, x8, xzr+        extr    x13, x1, x13, #32+        lsr     x1, x1, #32+        adds    x15, x1, x8+        adc     x1, xzr, xzr+        subs    x7, x7, x13+        sbcs    x13, x16, x15+        lsl     x3, x7, #32+        umulh   x16, x2, x5+        sbcs    x15, x14, x1+        add     x7, x3, x7+        sbcs    x3, x17, xzr+        lsr     x1, x7, #32+        sbcs    x14, x11, xzr+        sbc     x11, x8, xzr+        subs    x8, x1, x7+        sbc     x1, x7, xzr+        extr    x8, x1, x8, #32+        lsr     x1, x1, #32+        adds    x1, x1, x7+        adc     x17, xzr, xzr+        subs    x13, x13, x8+        umulh   x8, x9, x6+        sbcs    x1, x15, x1+        sbcs    x15, x3, x17+        sbcs    x3, x14, xzr+        mul     x17, x2, x5+        sbcs    x11, x11, xzr+        stp     x13, x1, [sp, #288]+        sbc     x14, x7, xzr+        mul     x7, x4, x10+        subs    x1, x9, x2+        stp     x15, x3, [sp, #304]+        csetm   x15, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        stp     x11, x14, [sp, #320]+        mul     x14, x9, x6+        adds    x17, x8, x17+        adcs    x7, x16, x7+        adc     x13, x12, xzr+        subs    x12, x5, x6+        cneg    x3, x12, cc  // cc = lo, ul, last+        cinv    x16, x15, cc  // cc = lo, ul, last+        mul     x8, x1, x3+        umulh   x1, x1, x3+        eor     x12, x8, x16+        adds    x11, x17, x14+        adcs    x3, x7, x17+        adcs    x15, x13, x7+        adc     x8, x13, xzr+        adds    x3, x3, x14+        adcs    x15, x15, x17+        adcs    x17, x8, x7+        eor     x1, x1, x16+        adc     x13, x13, xzr+        subs    x9, x9, x4+        csetm   x8, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x4, x2, x4+        cneg    x4, x4, cc  // cc = lo, ul, last+        csetm   x7, cc  // cc = lo, ul, last+        subs    x2, x10, x6+        cinv    x8, x8, cc  // cc = lo, ul, last+        cneg    x2, x2, cc  // cc = lo, ul, last+        cmn     x16, #0x1+        adcs    x11, x11, x12+        mul     x12, x9, x2+        adcs    x3, x3, x1+        adcs    x15, x15, x16+        umulh   x9, x9, x2+        adcs    x17, x17, x16+        adc     x13, x13, x16+        subs    x1, x10, x5+        cinv    x2, x7, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        eor     x9, x9, x8+        cmn     x8, #0x1+        eor     x7, x12, x8+        mul     x12, x4, x1+        adcs    x3, x3, x7+        adcs    x7, x15, x9+        adcs    x15, x17, x8+        ldp     x9, x17, [sp, #304]+        umulh   x4, x4, x1+        adc     x8, x13, x8+        cmn     x2, #0x1+        eor     x1, x12, x2+        adcs    x1, x7, x1+        ldp     x7, x16, [sp, #288]+        eor     x12, x4, x2+        adcs    x4, x15, x12+        ldp     x15, x12, [sp, #320]+        adc     x8, x8, x2+        adds    x13, x14, x14+        umulh   x14, x5, x10+        adcs    x2, x11, x11+        adcs    x3, x3, x3+        adcs    x1, x1, x1+        adcs    x4, x4, x4+        adcs    x11, x8, x8+        adc     x8, xzr, xzr+        adds    x13, x13, x7+        adcs    x2, x2, x16+        mul     x16, x5, x10+        adcs    x3, x3, x9+        adcs    x1, x1, x17+        umulh   x5, x5, x5+        lsl     x9, x13, #32+        add     x9, x9, x13+        adcs    x4, x4, x15+        mov     x13, v28.d[1]+        adcs    x15, x11, x12+        lsr     x7, x9, #32+        adc     x11, x8, xzr+        subs    x7, x7, x9+        umulh   x10, x10, x10+        sbc     x17, x9, xzr+        extr    x7, x17, x7, #32+        lsr     x17, x17, #32+        adds    x17, x17, x9+        adc     x12, xzr, xzr+        subs    x8, x2, x7+        sbcs    x17, x3, x17+        lsl     x7, x8, #32+        sbcs    x2, x1, x12+        add     x3, x7, x8+        sbcs    x12, x4, xzr+        lsr     x1, x3, #32+        sbcs    x7, x15, xzr+        sbc     x15, x9, xzr+        subs    x1, x1, x3+        sbc     x4, x3, xzr+        lsr     x9, x4, #32+        extr    x8, x4, x1, #32+        adds    x9, x9, x3+        adc     x4, xzr, xzr+        subs    x1, x17, x8+        lsl     x17, x1, #32+        sbcs    x8, x2, x9+        sbcs    x9, x12, x4+        add     x17, x17, x1+        mov     x1, v18.d[1]+        lsr     x2, x17, #32+        sbcs    x7, x7, xzr+        mov     x12, v18.d[0]+        sbcs    x15, x15, xzr+        sbc     x3, x3, xzr+        subs    x4, x2, x17+        sbc     x2, x17, xzr+        adds    x12, x13, x12+        adcs    x16, x16, x1+        lsr     x13, x2, #32+        extr    x1, x2, x4, #32+        adc     x2, x14, xzr+        adds    x4, x13, x17+        mul     x13, x6, x6+        adc     x14, xzr, xzr+        subs    x1, x8, x1+        sbcs    x4, x9, x4+        mov     x9, v28.d[0]+        sbcs    x7, x7, x14+        sbcs    x8, x15, xzr+        sbcs    x3, x3, xzr+        sbc     x14, x17, xzr+        adds    x17, x9, x9+        adcs    x12, x12, x12+        mov     x15, v19.d[0]+        adcs    x9, x16, x16+        umulh   x6, x6, x6+        adcs    x16, x2, x2+        adc     x2, xzr, xzr+        adds    x11, x11, x8+        adcs    x3, x3, xzr+        adcs    x14, x14, xzr+        adcs    x8, xzr, xzr+        adds    x13, x1, x13+        mov     x1, v19.d[1]+        adcs    x6, x4, x6+        mov     x4, #0xffffffff                 // #4294967295+        adcs    x15, x7, x15+        adcs    x7, x11, x5+        adcs    x1, x3, x1+        adcs    x14, x14, x10+        adc     x11, x8, xzr+        adds    x6, x6, x17+        adcs    x8, x15, x12+        adcs    x3, x7, x9+        adcs    x15, x1, x16+        mov     x16, #0xffffffff00000001        // #-4294967295+        adcs    x14, x14, x2+        mov     x2, #0x1                        // #1+        adc     x17, x11, xzr+        cmn     x13, x16+        adcs    xzr, x6, x4+        adcs    xzr, x8, x2+        adcs    xzr, x3, xzr+        adcs    xzr, x15, xzr+        adcs    xzr, x14, xzr+        adc     x1, x17, xzr+        neg     x9, x1+        and     x1, x16, x9+        adds    x11, x13, x1+        and     x13, x4, x9+        adcs    x5, x6, x13+        and     x1, x2, x9+        adcs    x7, x8, x1+        stp     x11, x5, [sp, #288]+        adcs    x11, x3, xzr+        adcs    x2, x15, xzr+        stp     x7, x11, [sp, #304]+        adc     x17, x14, xzr+        stp     x2, x17, [sp, #320]+        ldr     q3, [x26]+        ldr     q25, [sp, #48]+        ldp     x13, x23, [sp, #48]+        ldp     x3, x21, [x26]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [sp, #80]+        ldp     x8, x24, [x26, #16]+        subs    x6, x3, x21+        ldr     q0, [x26, #32]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [sp, #64]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [sp, #80]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [x26, #32]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x26, x4, x16+        mov     x4, v27.d[0]+        sbcs    x27, x20, x11+        sbcs    x20, x9, x12+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #160]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #176]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #160]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #176]+        adds    x20, x22, x26+        mul     x10, x13, x14+        adcs    x11, x11, x27+        eor     x9, x8, x21+        adcs    x26, x19, x17+        stp     x20, x11, [sp, #144]+        adcs    x27, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #176]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #144]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #176]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x26+        eor     x1, x22, x9+        adcs    x24, x23, x27+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x26+        adcs    x15, x17, x27+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x22, x6, x20+        eor     x3, x20, x23+        adcs    x5, x7, x3+        adcs    x14, x24, x11+        stp     x22, x5, [sp, #144]+        adcs    x5, x13, x23+        adcs    x21, x8, x23+        stp     x14, x5, [sp, #160]+        adc     x12, x15, x23+        stp     x21, x12, [sp, #176]+        ldr     q1, [sp, #240]+        ldp     x9, x2, [sp, #240]+        ldr     q0, [sp, #240]+        ldp     x4, x6, [sp, #256]+        rev64   v21.4s, v1.4s+        uzp2    v28.4s, v1.4s, v1.4s+        umulh   x7, x9, x2+        xtn     v17.2s, v1.2d+        mul     v27.4s, v21.4s, v0.4s+        ldr     q20, [sp, #272]+        xtn     v30.2s, v0.2d+        ldr     q1, [sp, #272]+        uzp2    v31.4s, v0.4s, v0.4s+        ldp     x5, x10, [sp, #272]+        umulh   x8, x9, x4+        uaddlp  v3.2d, v27.4s+        umull   v16.2d, v30.2s, v17.2s+        mul     x16, x9, x4+        umull   v27.2d, v30.2s, v28.2s+        shrn    v0.2s, v20.2d, #32+        xtn     v7.2s, v20.2d+        shl     v20.2d, v3.2d, #32+        umull   v3.2d, v31.2s, v28.2s+        mul     x3, x2, x4+        umlal   v20.2d, v30.2s, v17.2s+        umull   v22.2d, v7.2s, v0.2s+        usra    v27.2d, v16.2d, #32+        umulh   x11, x2, x4+        movi    v21.2d, #0xffffffff+        uzp2    v28.4s, v1.4s, v1.4s+        adds    x15, x16, x7+        and     v5.16b, v27.16b, v21.16b+        adcs    x3, x3, x8+        usra    v3.2d, v27.2d, #32+        dup     v29.2d, x6+        adcs    x16, x11, xzr+        mov     x14, v20.d[0]+        umlal   v5.2d, v31.2s, v17.2s+        mul     x8, x9, x2+        mov     x7, v20.d[1]+        shl     v19.2d, v22.2d, #33+        xtn     v25.2s, v29.2d+        rev64   v31.4s, v1.4s+        lsl     x13, x14, #32+        uzp2    v6.4s, v29.4s, v29.4s+        umlal   v19.2d, v7.2s, v7.2s+        usra    v3.2d, v5.2d, #32+        adds    x1, x8, x8+        umulh   x8, x4, x4+        add     x12, x13, x14+        mul     v17.4s, v31.4s, v29.4s+        xtn     v4.2s, v1.2d+        adcs    x14, x15, x15+        lsr     x13, x12, #32+        adcs    x15, x3, x3+        umull   v31.2d, v25.2s, v28.2s+        adcs    x11, x16, x16+        umull   v21.2d, v25.2s, v4.2s+        mov     x17, v3.d[0]+        umull   v18.2d, v6.2s, v28.2s+        adc     x16, x8, xzr+        uaddlp  v16.2d, v17.4s+        movi    v1.2d, #0xffffffff+        subs    x13, x13, x12+        usra    v31.2d, v21.2d, #32+        sbc     x8, x12, xzr+        adds    x17, x17, x1+        mul     x1, x4, x4+        shl     v28.2d, v16.2d, #32+        mov     x3, v3.d[1]+        adcs    x14, x7, x14+        extr    x7, x8, x13, #32+        adcs    x13, x3, x15+        and     v3.16b, v31.16b, v1.16b+        adcs    x11, x1, x11+        lsr     x1, x8, #32+        umlal   v3.2d, v6.2s, v4.2s+        usra    v18.2d, v31.2d, #32+        adc     x3, x16, xzr+        adds    x1, x1, x12+        umlal   v28.2d, v25.2s, v4.2s+        adc     x16, xzr, xzr+        subs    x15, x17, x7+        sbcs    x7, x14, x1+        lsl     x1, x15, #32+        sbcs    x16, x13, x16+        add     x8, x1, x15+        usra    v18.2d, v3.2d, #32+        sbcs    x14, x11, xzr+        lsr     x1, x8, #32+        sbcs    x17, x3, xzr+        sbc     x11, x12, xzr+        subs    x13, x1, x8+        umulh   x12, x4, x10+        sbc     x1, x8, xzr+        extr    x13, x1, x13, #32+        lsr     x1, x1, #32+        adds    x15, x1, x8+        adc     x1, xzr, xzr+        subs    x7, x7, x13+        sbcs    x13, x16, x15+        lsl     x3, x7, #32+        umulh   x16, x2, x5+        sbcs    x15, x14, x1+        add     x7, x3, x7+        sbcs    x3, x17, xzr+        lsr     x1, x7, #32+        sbcs    x14, x11, xzr+        sbc     x11, x8, xzr+        subs    x8, x1, x7+        sbc     x1, x7, xzr+        extr    x8, x1, x8, #32+        lsr     x1, x1, #32+        adds    x1, x1, x7+        adc     x17, xzr, xzr+        subs    x13, x13, x8+        umulh   x8, x9, x6+        sbcs    x1, x15, x1+        sbcs    x19, x3, x17+        sbcs    x20, x14, xzr+        mul     x17, x2, x5+        sbcs    x11, x11, xzr+        stp     x13, x1, [sp, #192]+        sbc     x14, x7, xzr+        mul     x7, x4, x10+        subs    x1, x9, x2+        csetm   x15, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        stp     x11, x14, [sp, #224]+        mul     x14, x9, x6+        adds    x17, x8, x17+        adcs    x7, x16, x7+        adc     x13, x12, xzr+        subs    x12, x5, x6+        cneg    x3, x12, cc  // cc = lo, ul, last+        cinv    x16, x15, cc  // cc = lo, ul, last+        mul     x8, x1, x3+        umulh   x1, x1, x3+        eor     x12, x8, x16+        adds    x11, x17, x14+        adcs    x3, x7, x17+        adcs    x15, x13, x7+        adc     x8, x13, xzr+        adds    x3, x3, x14+        adcs    x15, x15, x17+        adcs    x17, x8, x7+        eor     x1, x1, x16+        adc     x13, x13, xzr+        subs    x9, x9, x4+        csetm   x8, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x4, x2, x4+        cneg    x4, x4, cc  // cc = lo, ul, last+        csetm   x7, cc  // cc = lo, ul, last+        subs    x2, x10, x6+        cinv    x8, x8, cc  // cc = lo, ul, last+        cneg    x2, x2, cc  // cc = lo, ul, last+        cmn     x16, #0x1+        adcs    x11, x11, x12+        mul     x12, x9, x2+        adcs    x3, x3, x1+        adcs    x15, x15, x16+        umulh   x9, x9, x2+        adcs    x17, x17, x16+        adc     x13, x13, x16+        subs    x1, x10, x5+        cinv    x2, x7, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        eor     x9, x9, x8+        cmn     x8, #0x1+        eor     x7, x12, x8+        mul     x12, x4, x1+        adcs    x3, x3, x7+        adcs    x7, x15, x9+        adcs    x15, x17, x8+        umulh   x4, x4, x1+        adc     x8, x13, x8+        cmn     x2, #0x1+        eor     x1, x12, x2+        adcs    x1, x7, x1+        ldp     x7, x16, [sp, #192]+        eor     x12, x4, x2+        adcs    x4, x15, x12+        ldp     x15, x12, [sp, #224]+        adc     x8, x8, x2+        adds    x13, x14, x14+        umulh   x14, x5, x10+        adcs    x2, x11, x11+        adcs    x3, x3, x3+        adcs    x1, x1, x1+        adcs    x4, x4, x4+        adcs    x11, x8, x8+        adc     x8, xzr, xzr+        adds    x13, x13, x7+        adcs    x2, x2, x16+        mul     x16, x5, x10+        adcs    x3, x3, x19+        adcs    x1, x1, x20+        umulh   x5, x5, x5+        lsl     x9, x13, #32+        add     x9, x9, x13+        adcs    x4, x4, x15+        mov     x13, v28.d[1]+        adcs    x15, x11, x12+        lsr     x7, x9, #32+        adc     x11, x8, xzr+        subs    x7, x7, x9+        umulh   x10, x10, x10+        sbc     x17, x9, xzr+        extr    x7, x17, x7, #32+        lsr     x17, x17, #32+        adds    x17, x17, x9+        adc     x12, xzr, xzr+        subs    x8, x2, x7+        sbcs    x17, x3, x17+        lsl     x7, x8, #32+        sbcs    x2, x1, x12+        add     x3, x7, x8+        sbcs    x12, x4, xzr+        lsr     x1, x3, #32+        sbcs    x7, x15, xzr+        sbc     x15, x9, xzr+        subs    x1, x1, x3+        sbc     x4, x3, xzr+        lsr     x9, x4, #32+        extr    x8, x4, x1, #32+        adds    x9, x9, x3+        adc     x4, xzr, xzr+        subs    x1, x17, x8+        lsl     x17, x1, #32+        sbcs    x8, x2, x9+        sbcs    x9, x12, x4+        add     x17, x17, x1+        mov     x1, v18.d[1]+        lsr     x2, x17, #32+        sbcs    x7, x7, xzr+        mov     x12, v18.d[0]+        sbcs    x15, x15, xzr+        sbc     x3, x3, xzr+        subs    x4, x2, x17+        sbc     x2, x17, xzr+        adds    x12, x13, x12+        adcs    x16, x16, x1+        lsr     x13, x2, #32+        extr    x1, x2, x4, #32+        adc     x2, x14, xzr+        adds    x4, x13, x17+        mul     x13, x6, x6+        adc     x14, xzr, xzr+        subs    x1, x8, x1+        sbcs    x4, x9, x4+        mov     x9, v28.d[0]+        sbcs    x7, x7, x14+        sbcs    x8, x15, xzr+        sbcs    x3, x3, xzr+        sbc     x14, x17, xzr+        adds    x17, x9, x9+        adcs    x12, x12, x12+        mov     x15, v19.d[0]+        adcs    x9, x16, x16+        umulh   x6, x6, x6+        adcs    x16, x2, x2+        adc     x2, xzr, xzr+        adds    x11, x11, x8+        adcs    x3, x3, xzr+        adcs    x14, x14, xzr+        adcs    x8, xzr, xzr+        adds    x13, x1, x13+        mov     x1, v19.d[1]+        adcs    x6, x4, x6+        mov     x4, #0xffffffff                 // #4294967295+        adcs    x15, x7, x15+        adcs    x7, x11, x5+        adcs    x1, x3, x1+        adcs    x14, x14, x10+        adc     x11, x8, xzr+        adds    x6, x6, x17+        adcs    x8, x15, x12+        adcs    x3, x7, x9+        adcs    x15, x1, x16+        mov     x16, #0xffffffff00000001        // #-4294967295+        adcs    x14, x14, x2+        mov     x2, #0x1                        // #1+        adc     x17, x11, xzr+        cmn     x13, x16+        adcs    xzr, x6, x4+        adcs    xzr, x8, x2+        adcs    xzr, x3, xzr+        adcs    xzr, x15, xzr+        adcs    xzr, x14, xzr+        adc     x1, x17, xzr+        neg     x9, x1+        and     x1, x16, x9+        adds    x19, x13, x1+        and     x13, x4, x9+        adcs    x20, x6, x13+        and     x1, x2, x9+        adcs    x7, x8, x1+        adcs    x11, x3, xzr+        adcs    x2, x15, xzr+        stp     x7, x11, [sp, #208]+        adc     x17, x14, xzr+        stp     x2, x17, [sp, #224]+        ldp     x0, x1, [sp, #288]+        mov     x6, #0xffffffff                 // #4294967295+        subs    x6, x6, x0+        mov     x7, #0xffffffff00000000         // #-4294967296+        sbcs    x7, x7, x1+        ldp     x0, x1, [sp, #304]+        mov     x8, #0xfffffffffffffffe         // #-2+        sbcs    x8, x8, x0+        mov     x13, #0xffffffffffffffff        // #-1+        sbcs    x9, x13, x1+        ldp     x0, x1, [sp, #320]+        sbcs    x10, x13, x0+        sbc     x11, x13, x1+        mov     x12, #0x9                       // #9+        mul     x0, x12, x6+        mul     x1, x12, x7+        mul     x2, x12, x8+        mul     x3, x12, x9+        mul     x4, x12, x10+        mul     x5, x12, x11+        umulh   x6, x12, x6+        umulh   x7, x12, x7+        umulh   x8, x12, x8+        umulh   x9, x12, x9+        umulh   x10, x12, x10+        umulh   x12, x12, x11+        adds    x1, x1, x6+        adcs    x2, x2, x7+        adcs    x3, x3, x8+        adcs    x4, x4, x9+        adcs    x5, x5, x10+        mov     x6, #0x1                        // #1+        adc     x6, x12, x6+        ldp     x8, x9, [sp, #144]+        ldp     x10, x11, [sp, #160]+        ldp     x12, x13, [sp, #176]+        mov     x14, #0xc                       // #12+        mul     x15, x14, x8+        umulh   x8, x14, x8+        adds    x0, x0, x15+        mul     x15, x14, x9+        umulh   x9, x14, x9+        adcs    x1, x1, x15+        mul     x15, x14, x10+        umulh   x10, x14, x10+        adcs    x2, x2, x15+        mul     x15, x14, x11+        umulh   x11, x14, x11+        adcs    x3, x3, x15+        mul     x15, x14, x12+        umulh   x12, x14, x12+        adcs    x4, x4, x15+        mul     x15, x14, x13+        umulh   x13, x14, x13+        adcs    x5, x5, x15+        adc     x6, x6, xzr+        adds    x1, x1, x8+        adcs    x2, x2, x9+        adcs    x3, x3, x10+        adcs    x4, x4, x11+        adcs    x5, x5, x12+        adcs    x6, x6, x13+        lsl     x7, x6, #32+        subs    x8, x6, x7+        sbc     x7, x7, xzr+        adds    x0, x0, x8+        adcs    x1, x1, x7+        adcs    x2, x2, x6+        adcs    x3, x3, xzr+        adcs    x4, x4, xzr+        adcs    x5, x5, xzr+        csetm   x6, cc  // cc = lo, ul, last+        mov     x7, #0xffffffff                 // #4294967295+        and     x7, x7, x6+        adds    x0, x0, x7+        eor     x7, x7, x6+        adcs    x1, x1, x7+        mov     x7, #0xfffffffffffffffe         // #-2+        and     x7, x7, x6+        adcs    x2, x2, x7+        adcs    x3, x3, x6+        adcs    x4, x4, x6+        adc     x5, x5, x6+        stp     x0, x1, [sp, #288]+        stp     x2, x3, [sp, #304]+        stp     x4, x5, [sp, #320]+        mov     x2, sp+        ldp     x4, x3, [x2]+        subs    x5, x19, x4+        sbcs    x6, x20, x3+        ldp     x7, x8, [sp, #208]+        ldp     x4, x3, [x2, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #224]+        ldp     x4, x3, [x2, #32]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, cc  // cc = lo, ul, last+        mov     x4, #0xffffffff                 // #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #0xfffffffffffffffe         // #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [sp, #240]+        stp     x7, x8, [sp, #256]+        stp     x9, x10, [sp, #272]+        ldr     q1, [sp, #48]+        ldp     x9, x2, [sp, #48]+        ldr     q0, [sp, #48]+        ldp     x4, x6, [sp, #64]+        rev64   v21.4s, v1.4s+        uzp2    v28.4s, v1.4s, v1.4s+        umulh   x7, x9, x2+        xtn     v17.2s, v1.2d+        mul     v27.4s, v21.4s, v0.4s+        ldr     q20, [sp, #80]+        xtn     v30.2s, v0.2d+        ldr     q1, [sp, #80]+        uzp2    v31.4s, v0.4s, v0.4s+        ldp     x5, x10, [sp, #80]+        umulh   x8, x9, x4+        uaddlp  v3.2d, v27.4s+        umull   v16.2d, v30.2s, v17.2s+        mul     x16, x9, x4+        umull   v27.2d, v30.2s, v28.2s+        shrn    v0.2s, v20.2d, #32+        xtn     v7.2s, v20.2d+        shl     v20.2d, v3.2d, #32+        umull   v3.2d, v31.2s, v28.2s+        mul     x3, x2, x4+        umlal   v20.2d, v30.2s, v17.2s+        umull   v22.2d, v7.2s, v0.2s+        usra    v27.2d, v16.2d, #32+        umulh   x11, x2, x4+        movi    v21.2d, #0xffffffff+        uzp2    v28.4s, v1.4s, v1.4s+        adds    x15, x16, x7+        and     v5.16b, v27.16b, v21.16b+        adcs    x3, x3, x8+        usra    v3.2d, v27.2d, #32+        dup     v29.2d, x6+        adcs    x16, x11, xzr+        mov     x14, v20.d[0]+        umlal   v5.2d, v31.2s, v17.2s+        mul     x8, x9, x2+        mov     x7, v20.d[1]+        shl     v19.2d, v22.2d, #33+        xtn     v25.2s, v29.2d+        rev64   v31.4s, v1.4s+        lsl     x13, x14, #32+        uzp2    v6.4s, v29.4s, v29.4s+        umlal   v19.2d, v7.2s, v7.2s+        usra    v3.2d, v5.2d, #32+        adds    x1, x8, x8+        umulh   x8, x4, x4+        add     x12, x13, x14+        mul     v17.4s, v31.4s, v29.4s+        xtn     v4.2s, v1.2d+        adcs    x14, x15, x15+        lsr     x13, x12, #32+        adcs    x15, x3, x3+        umull   v31.2d, v25.2s, v28.2s+        adcs    x11, x16, x16+        umull   v21.2d, v25.2s, v4.2s+        mov     x17, v3.d[0]+        umull   v18.2d, v6.2s, v28.2s+        adc     x16, x8, xzr+        uaddlp  v16.2d, v17.4s+        movi    v1.2d, #0xffffffff+        subs    x13, x13, x12+        usra    v31.2d, v21.2d, #32+        sbc     x8, x12, xzr+        adds    x17, x17, x1+        mul     x1, x4, x4+        shl     v28.2d, v16.2d, #32+        mov     x3, v3.d[1]+        adcs    x14, x7, x14+        extr    x7, x8, x13, #32+        adcs    x13, x3, x15+        and     v3.16b, v31.16b, v1.16b+        adcs    x11, x1, x11+        lsr     x1, x8, #32+        umlal   v3.2d, v6.2s, v4.2s+        usra    v18.2d, v31.2d, #32+        adc     x3, x16, xzr+        adds    x1, x1, x12+        umlal   v28.2d, v25.2s, v4.2s+        adc     x16, xzr, xzr+        subs    x15, x17, x7+        sbcs    x7, x14, x1+        lsl     x1, x15, #32+        sbcs    x16, x13, x16+        add     x8, x1, x15+        usra    v18.2d, v3.2d, #32+        sbcs    x14, x11, xzr+        lsr     x1, x8, #32+        sbcs    x17, x3, xzr+        sbc     x11, x12, xzr+        subs    x13, x1, x8+        umulh   x12, x4, x10+        sbc     x1, x8, xzr+        extr    x13, x1, x13, #32+        lsr     x1, x1, #32+        adds    x15, x1, x8+        adc     x1, xzr, xzr+        subs    x7, x7, x13+        sbcs    x13, x16, x15+        lsl     x3, x7, #32+        umulh   x16, x2, x5+        sbcs    x15, x14, x1+        add     x7, x3, x7+        sbcs    x3, x17, xzr+        lsr     x1, x7, #32+        sbcs    x14, x11, xzr+        sbc     x11, x8, xzr+        subs    x8, x1, x7+        sbc     x1, x7, xzr+        extr    x8, x1, x8, #32+        lsr     x1, x1, #32+        adds    x1, x1, x7+        adc     x17, xzr, xzr+        subs    x13, x13, x8+        umulh   x8, x9, x6+        sbcs    x1, x15, x1+        sbcs    x19, x3, x17+        sbcs    x20, x14, xzr+        mul     x17, x2, x5+        sbcs    x11, x11, xzr+        stp     x13, x1, [sp, #192]+        sbc     x14, x7, xzr+        mul     x7, x4, x10+        subs    x1, x9, x2+        csetm   x15, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        stp     x11, x14, [sp, #224]+        mul     x14, x9, x6+        adds    x17, x8, x17+        adcs    x7, x16, x7+        adc     x13, x12, xzr+        subs    x12, x5, x6+        cneg    x3, x12, cc  // cc = lo, ul, last+        cinv    x16, x15, cc  // cc = lo, ul, last+        mul     x8, x1, x3+        umulh   x1, x1, x3+        eor     x12, x8, x16+        adds    x11, x17, x14+        adcs    x3, x7, x17+        adcs    x15, x13, x7+        adc     x8, x13, xzr+        adds    x3, x3, x14+        adcs    x15, x15, x17+        adcs    x17, x8, x7+        eor     x1, x1, x16+        adc     x13, x13, xzr+        subs    x9, x9, x4+        csetm   x8, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x4, x2, x4+        cneg    x4, x4, cc  // cc = lo, ul, last+        csetm   x7, cc  // cc = lo, ul, last+        subs    x2, x10, x6+        cinv    x8, x8, cc  // cc = lo, ul, last+        cneg    x2, x2, cc  // cc = lo, ul, last+        cmn     x16, #0x1+        adcs    x11, x11, x12+        mul     x12, x9, x2+        adcs    x3, x3, x1+        adcs    x15, x15, x16+        umulh   x9, x9, x2+        adcs    x17, x17, x16+        adc     x13, x13, x16+        subs    x1, x10, x5+        cinv    x2, x7, cc  // cc = lo, ul, last+        cneg    x1, x1, cc  // cc = lo, ul, last+        eor     x9, x9, x8+        cmn     x8, #0x1+        eor     x7, x12, x8+        mul     x12, x4, x1+        adcs    x3, x3, x7+        adcs    x7, x15, x9+        adcs    x15, x17, x8+        umulh   x4, x4, x1+        adc     x8, x13, x8+        cmn     x2, #0x1+        eor     x1, x12, x2+        adcs    x1, x7, x1+        ldp     x7, x16, [sp, #192]+        eor     x12, x4, x2+        adcs    x4, x15, x12+        ldp     x15, x12, [sp, #224]+        adc     x8, x8, x2+        adds    x13, x14, x14+        umulh   x14, x5, x10+        adcs    x2, x11, x11+        adcs    x3, x3, x3+        adcs    x1, x1, x1+        adcs    x4, x4, x4+        adcs    x11, x8, x8+        adc     x8, xzr, xzr+        adds    x13, x13, x7+        adcs    x2, x2, x16+        mul     x16, x5, x10+        adcs    x3, x3, x19+        adcs    x1, x1, x20+        umulh   x5, x5, x5+        lsl     x9, x13, #32+        add     x9, x9, x13+        adcs    x4, x4, x15+        mov     x13, v28.d[1]+        adcs    x15, x11, x12+        lsr     x7, x9, #32+        adc     x11, x8, xzr+        subs    x7, x7, x9+        umulh   x10, x10, x10+        sbc     x17, x9, xzr+        extr    x7, x17, x7, #32+        lsr     x17, x17, #32+        adds    x17, x17, x9+        adc     x12, xzr, xzr+        subs    x8, x2, x7+        sbcs    x17, x3, x17+        lsl     x7, x8, #32+        sbcs    x2, x1, x12+        add     x3, x7, x8+        sbcs    x12, x4, xzr+        lsr     x1, x3, #32+        sbcs    x7, x15, xzr+        sbc     x15, x9, xzr+        subs    x1, x1, x3+        sbc     x4, x3, xzr+        lsr     x9, x4, #32+        extr    x8, x4, x1, #32+        adds    x9, x9, x3+        adc     x4, xzr, xzr+        subs    x1, x17, x8+        lsl     x17, x1, #32+        sbcs    x8, x2, x9+        sbcs    x9, x12, x4+        add     x17, x17, x1+        mov     x1, v18.d[1]+        lsr     x2, x17, #32+        sbcs    x7, x7, xzr+        mov     x12, v18.d[0]+        sbcs    x15, x15, xzr+        sbc     x3, x3, xzr+        subs    x4, x2, x17+        sbc     x2, x17, xzr+        adds    x12, x13, x12+        adcs    x16, x16, x1+        lsr     x13, x2, #32+        extr    x1, x2, x4, #32+        adc     x2, x14, xzr+        adds    x4, x13, x17+        mul     x13, x6, x6+        adc     x14, xzr, xzr+        subs    x1, x8, x1+        sbcs    x4, x9, x4+        mov     x9, v28.d[0]+        sbcs    x7, x7, x14+        sbcs    x8, x15, xzr+        sbcs    x3, x3, xzr+        sbc     x14, x17, xzr+        adds    x17, x9, x9+        adcs    x12, x12, x12+        mov     x15, v19.d[0]+        adcs    x9, x16, x16+        umulh   x6, x6, x6+        adcs    x16, x2, x2+        adc     x2, xzr, xzr+        adds    x11, x11, x8+        adcs    x3, x3, xzr+        adcs    x14, x14, xzr+        adcs    x8, xzr, xzr+        adds    x13, x1, x13+        mov     x1, v19.d[1]+        adcs    x6, x4, x6+        mov     x4, #0xffffffff                 // #4294967295+        adcs    x15, x7, x15+        adcs    x7, x11, x5+        adcs    x1, x3, x1+        adcs    x14, x14, x10+        adc     x11, x8, xzr+        adds    x6, x6, x17+        adcs    x8, x15, x12+        adcs    x3, x7, x9+        adcs    x15, x1, x16+        mov     x16, #0xffffffff00000001        // #-4294967295+        adcs    x14, x14, x2+        mov     x2, #0x1                        // #1+        adc     x17, x11, xzr+        cmn     x13, x16+        adcs    xzr, x6, x4+        adcs    xzr, x8, x2+        adcs    xzr, x3, xzr+        adcs    xzr, x15, xzr+        adcs    xzr, x14, xzr+        adc     x1, x17, xzr+        neg     x9, x1+        and     x1, x16, x9+        adds    x11, x13, x1+        and     x13, x4, x9+        adcs    x5, x6, x13+        and     x1, x2, x9+        adcs    x7, x8, x1+        stp     x11, x5, [sp, #192]+        adcs    x11, x3, xzr+        adcs    x2, x15, xzr+        stp     x7, x11, [sp, #208]+        adc     x17, x14, xzr+        stp     x2, x17, [sp, #224]+        ldp     x5, x6, [sp, #240]+        ldp     x4, x3, [sp, #48]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #256]+        ldp     x4, x3, [sp, #64]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #272]+        ldp     x4, x3, [sp, #80]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, cc  // cc = lo, ul, last+        mov     x4, #0xffffffff                 // #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #0xfffffffffffffffe         // #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [x25, #96]+        stp     x7, x8, [x25, #112]+        stp     x9, x10, [x25, #128]+        ldr     q3, [sp, #288]+        ldr     q25, [sp, #96]+        ldp     x13, x23, [sp, #96]+        ldp     x3, x21, [sp, #288]+        rev64   v23.4s, v25.4s+        uzp1    v17.4s, v25.4s, v3.4s+        umulh   x15, x3, x13+        mul     v6.4s, v23.4s, v3.4s+        uzp1    v3.4s, v3.4s, v3.4s+        ldr     q27, [sp, #128]+        ldp     x8, x24, [sp, #304]+        subs    x6, x3, x21+        ldr     q0, [sp, #320]+        movi    v23.2d, #0xffffffff+        csetm   x10, cc  // cc = lo, ul, last+        umulh   x19, x21, x23+        rev64   v4.4s, v27.4s+        uzp2    v25.4s, v27.4s, v27.4s+        cneg    x4, x6, cc  // cc = lo, ul, last+        subs    x7, x23, x13+        xtn     v22.2s, v0.2d+        xtn     v24.2s, v27.2d+        cneg    x20, x7, cc  // cc = lo, ul, last+        ldp     x6, x14, [sp, #112]+        mul     v27.4s, v4.4s, v0.4s+        uaddlp  v20.2d, v6.4s+        cinv    x5, x10, cc  // cc = lo, ul, last+        mul     x16, x4, x20+        uzp2    v6.4s, v0.4s, v0.4s+        umull   v21.2d, v22.2s, v25.2s+        shl     v0.2d, v20.2d, #32+        umlal   v0.2d, v3.2s, v17.2s+        mul     x22, x8, x6+        umull   v1.2d, v6.2s, v25.2s+        subs    x12, x3, x8+        umull   v20.2d, v22.2s, v24.2s+        cneg    x17, x12, cc  // cc = lo, ul, last+        umulh   x9, x8, x6+        mov     x12, v0.d[1]+        eor     x11, x16, x5+        mov     x7, v0.d[0]+        csetm   x10, cc  // cc = lo, ul, last+        usra    v21.2d, v20.2d, #32+        adds    x15, x15, x12+        adcs    x12, x19, x22+        umulh   x20, x4, x20+        adc     x19, x9, xzr+        usra    v1.2d, v21.2d, #32+        adds    x22, x15, x7+        and     v26.16b, v21.16b, v23.16b+        adcs    x16, x12, x15+        uaddlp  v25.2d, v27.4s+        adcs    x9, x19, x12+        umlal   v26.2d, v6.2s, v24.2s+        adc     x4, x19, xzr+        adds    x16, x16, x7+        shl     v27.2d, v25.2d, #32+        adcs    x9, x9, x15+        adcs    x4, x4, x12+        eor     x12, x20, x5+        adc     x15, x19, xzr+        subs    x20, x6, x13+        cneg    x20, x20, cc  // cc = lo, ul, last+        cinv    x10, x10, cc  // cc = lo, ul, last+        cmn     x5, #0x1+        mul     x19, x17, x20+        adcs    x11, x22, x11+        adcs    x12, x16, x12+        adcs    x9, x9, x5+        umulh   x17, x17, x20+        adcs    x22, x4, x5+        adc     x5, x15, x5+        subs    x16, x21, x8+        cneg    x20, x16, cc  // cc = lo, ul, last+        eor     x19, x19, x10+        csetm   x4, cc  // cc = lo, ul, last+        subs    x16, x6, x23+        cneg    x16, x16, cc  // cc = lo, ul, last+        umlal   v27.2d, v22.2s, v24.2s+        mul     x15, x20, x16+        cinv    x4, x4, cc  // cc = lo, ul, last+        cmn     x10, #0x1+        usra    v1.2d, v26.2d, #32+        adcs    x19, x12, x19+        eor     x17, x17, x10+        adcs    x9, x9, x17+        adcs    x22, x22, x10+        lsl     x12, x7, #32+        umulh   x20, x20, x16+        eor     x16, x15, x4+        ldp     x15, x17, [sp, #128]+        add     x2, x12, x7+        adc     x7, x5, x10+        ldp     x5, x10, [sp, #320]+        lsr     x1, x2, #32+        eor     x12, x20, x4+        subs    x1, x1, x2+        sbc     x20, x2, xzr+        cmn     x4, #0x1+        adcs    x9, x9, x16+        extr    x1, x20, x1, #32+        lsr     x20, x20, #32+        adcs    x22, x22, x12+        adc     x16, x7, x4+        adds    x12, x20, x2+        umulh   x7, x24, x14+        adc     x4, xzr, xzr+        subs    x1, x11, x1+        sbcs    x20, x19, x12+        sbcs    x12, x9, x4+        lsl     x9, x1, #32+        add     x1, x9, x1+        sbcs    x9, x22, xzr+        mul     x22, x24, x14+        sbcs    x16, x16, xzr+        lsr     x4, x1, #32+        sbc     x19, x2, xzr+        subs    x4, x4, x1+        sbc     x11, x1, xzr+        extr    x2, x11, x4, #32+        lsr     x4, x11, #32+        adds    x4, x4, x1+        adc     x11, xzr, xzr+        subs    x2, x20, x2+        sbcs    x4, x12, x4+        sbcs    x20, x9, x11+        lsl     x12, x2, #32+        add     x2, x12, x2+        sbcs    x9, x16, xzr+        lsr     x11, x2, #32+        sbcs    x19, x19, xzr+        sbc     x1, x1, xzr+        subs    x16, x11, x2+        sbc     x12, x2, xzr+        extr    x16, x12, x16, #32+        lsr     x12, x12, #32+        adds    x11, x12, x2+        adc     x12, xzr, xzr+        subs    x26, x4, x16+        mov     x4, v27.d[0]+        sbcs    x27, x20, x11+        sbcs    x20, x9, x12+        sbcs    x11, x19, xzr+        sbcs    x9, x1, xzr+        stp     x20, x11, [sp, #256]+        mov     x1, v1.d[0]+        sbc     x20, x2, xzr+        subs    x12, x24, x5+        mov     x11, v27.d[1]+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x2, cc  // cc = lo, ul, last+        subs    x19, x15, x14+        mov     x12, v1.d[1]+        cinv    x2, x2, cc  // cc = lo, ul, last+        cneg    x19, x19, cc  // cc = lo, ul, last+        stp     x9, x20, [sp, #272]+        mul     x9, x16, x19+        adds    x4, x7, x4+        adcs    x11, x1, x11+        adc     x1, x12, xzr+        adds    x20, x4, x22+        umulh   x19, x16, x19+        adcs    x7, x11, x4+        eor     x16, x9, x2+        adcs    x9, x1, x11+        adc     x12, x1, xzr+        adds    x7, x7, x22+        adcs    x4, x9, x4+        adcs    x9, x12, x11+        adc     x12, x1, xzr+        cmn     x2, #0x1+        eor     x1, x19, x2+        adcs    x11, x20, x16+        adcs    x19, x7, x1+        adcs    x1, x4, x2+        adcs    x20, x9, x2+        adc     x2, x12, x2+        subs    x12, x24, x10+        cneg    x16, x12, cc  // cc = lo, ul, last+        csetm   x12, cc  // cc = lo, ul, last+        subs    x9, x17, x14+        cinv    x12, x12, cc  // cc = lo, ul, last+        cneg    x9, x9, cc  // cc = lo, ul, last+        subs    x3, x24, x3+        sbcs    x21, x5, x21+        mul     x24, x16, x9+        sbcs    x4, x10, x8+        ngc     x8, xzr+        subs    x10, x5, x10+        eor     x5, x24, x12+        csetm   x7, cc  // cc = lo, ul, last+        cneg    x24, x10, cc  // cc = lo, ul, last+        subs    x10, x17, x15+        cinv    x7, x7, cc  // cc = lo, ul, last+        cneg    x10, x10, cc  // cc = lo, ul, last+        subs    x14, x13, x14+        sbcs    x15, x23, x15+        eor     x13, x21, x8+        mul     x23, x24, x10+        sbcs    x17, x6, x17+        eor     x6, x3, x8+        ngc     x21, xzr+        umulh   x9, x16, x9+        cmn     x8, #0x1+        eor     x3, x23, x7+        adcs    x23, x6, xzr+        adcs    x13, x13, xzr+        eor     x16, x4, x8+        adc     x16, x16, xzr+        eor     x4, x17, x21+        umulh   x17, x24, x10+        cmn     x21, #0x1+        eor     x24, x14, x21+        eor     x6, x15, x21+        adcs    x15, x24, xzr+        adcs    x14, x6, xzr+        adc     x6, x4, xzr+        cmn     x12, #0x1+        eor     x4, x9, x12+        adcs    x19, x19, x5+        umulh   x5, x23, x15+        adcs    x1, x1, x4+        adcs    x10, x20, x12+        eor     x4, x17, x7+        adc     x2, x2, x12+        cmn     x7, #0x1+        adcs    x12, x1, x3+        ldp     x17, x24, [sp, #256]+        mul     x1, x16, x6+        adcs    x3, x10, x4+        adc     x2, x2, x7+        ldp     x7, x4, [sp, #272]+        adds    x20, x22, x26+        mul     x10, x13, x14+        adcs    x11, x11, x27+        eor     x9, x8, x21+        adcs    x26, x19, x17+        stp     x20, x11, [sp, #240]+        adcs    x27, x12, x24+        mul     x8, x23, x15+        adcs    x3, x3, x7+        adcs    x12, x2, x4+        adc     x19, xzr, xzr+        subs    x21, x23, x16+        umulh   x2, x16, x6+        stp     x3, x12, [sp, #272]+        cneg    x3, x21, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        umulh   x11, x13, x14+        subs    x21, x13, x16+        eor     x7, x8, x9+        cneg    x17, x21, cc  // cc = lo, ul, last+        csetm   x16, cc  // cc = lo, ul, last+        subs    x21, x6, x15+        cneg    x22, x21, cc  // cc = lo, ul, last+        cinv    x21, x24, cc  // cc = lo, ul, last+        subs    x20, x23, x13+        umulh   x12, x3, x22+        cneg    x23, x20, cc  // cc = lo, ul, last+        csetm   x24, cc  // cc = lo, ul, last+        subs    x20, x14, x15+        cinv    x24, x24, cc  // cc = lo, ul, last+        mul     x22, x3, x22+        cneg    x3, x20, cc  // cc = lo, ul, last+        subs    x13, x6, x14+        cneg    x20, x13, cc  // cc = lo, ul, last+        cinv    x15, x16, cc  // cc = lo, ul, last+        adds    x13, x5, x10+        mul     x4, x23, x3+        adcs    x11, x11, x1+        adc     x14, x2, xzr+        adds    x5, x13, x8+        adcs    x16, x11, x13+        umulh   x23, x23, x3+        adcs    x3, x14, x11+        adc     x1, x14, xzr+        adds    x10, x16, x8+        adcs    x6, x3, x13+        adcs    x8, x1, x11+        umulh   x13, x17, x20+        eor     x1, x4, x24+        adc     x4, x14, xzr+        cmn     x24, #0x1+        adcs    x1, x5, x1+        eor     x16, x23, x24+        eor     x11, x1, x9+        adcs    x23, x10, x16+        eor     x2, x22, x21+        adcs    x3, x6, x24+        mul     x14, x17, x20+        eor     x17, x13, x15+        adcs    x13, x8, x24+        adc     x8, x4, x24+        cmn     x21, #0x1+        adcs    x6, x23, x2+        mov     x16, #0xfffffffffffffffe        // #-2+        eor     x20, x12, x21+        adcs    x20, x3, x20+        eor     x23, x14, x15+        adcs    x2, x13, x21+        adc     x8, x8, x21+        cmn     x15, #0x1+        ldp     x5, x4, [sp, #240]+        adcs    x22, x20, x23+        eor     x23, x22, x9+        adcs    x17, x2, x17+        adc     x22, x8, x15+        cmn     x9, #0x1+        adcs    x15, x7, x5+        ldp     x10, x14, [sp, #272]+        eor     x1, x6, x9+        lsl     x2, x15, #32+        adcs    x8, x11, x4+        adcs    x13, x1, x26+        eor     x1, x22, x9+        adcs    x24, x23, x27+        eor     x11, x17, x9+        adcs    x23, x11, x10+        adcs    x7, x1, x14+        adcs    x17, x9, x19+        adcs    x20, x9, xzr+        add     x1, x2, x15+        lsr     x3, x1, #32+        adcs    x11, x9, xzr+        adc     x9, x9, xzr+        subs    x3, x3, x1+        sbc     x6, x1, xzr+        adds    x24, x24, x5+        adcs    x4, x23, x4+        extr    x3, x6, x3, #32+        lsr     x6, x6, #32+        adcs    x21, x7, x26+        adcs    x15, x17, x27+        adcs    x7, x20, x10+        adcs    x20, x11, x14+        mov     x14, #0xffffffff                // #4294967295+        adc     x22, x9, x19+        adds    x12, x6, x1+        adc     x10, xzr, xzr+        subs    x3, x8, x3+        sbcs    x12, x13, x12+        lsl     x9, x3, #32+        add     x3, x9, x3+        sbcs    x10, x24, x10+        sbcs    x24, x4, xzr+        lsr     x9, x3, #32+        sbcs    x21, x21, xzr+        sbc     x1, x1, xzr+        subs    x9, x9, x3+        sbc     x13, x3, xzr+        extr    x9, x13, x9, #32+        lsr     x13, x13, #32+        adds    x13, x13, x3+        adc     x6, xzr, xzr+        subs    x12, x12, x9+        sbcs    x17, x10, x13+        lsl     x2, x12, #32+        sbcs    x10, x24, x6+        add     x9, x2, x12+        sbcs    x6, x21, xzr+        lsr     x5, x9, #32+        sbcs    x21, x1, xzr+        sbc     x13, x3, xzr+        subs    x8, x5, x9+        sbc     x19, x9, xzr+        lsr     x12, x19, #32+        extr    x3, x19, x8, #32+        adds    x8, x12, x9+        adc     x1, xzr, xzr+        subs    x2, x17, x3+        sbcs    x12, x10, x8+        sbcs    x5, x6, x1+        sbcs    x3, x21, xzr+        sbcs    x19, x13, xzr+        sbc     x24, x9, xzr+        adds    x23, x15, x3+        adcs    x8, x7, x19+        adcs    x11, x20, x24+        adc     x9, x22, xzr+        add     x24, x9, #0x1+        lsl     x7, x24, #32+        subs    x21, x24, x7+        sbc     x10, x7, xzr+        adds    x6, x2, x21+        adcs    x7, x12, x10+        adcs    x24, x5, x24+        adcs    x13, x23, xzr+        adcs    x8, x8, xzr+        adcs    x15, x11, xzr+        csetm   x23, cc  // cc = lo, ul, last+        and     x11, x16, x23+        and     x20, x14, x23+        adds    x22, x6, x20+        eor     x3, x20, x23+        adcs    x5, x7, x3+        adcs    x14, x24, x11+        stp     x22, x5, [sp, #240]+        adcs    x5, x13, x23+        adcs    x12, x8, x23+        stp     x14, x5, [sp, #256]+        adc     x19, x15, x23+        ldp     x1, x2, [sp, #144]+        ldp     x3, x4, [sp, #160]+        ldp     x5, x6, [sp, #176]+        lsl     x0, x1, #2+        ldp     x7, x8, [sp, #288]+        subs    x0, x0, x7+        extr    x1, x2, x1, #62+        sbcs    x1, x1, x8+        ldp     x7, x8, [sp, #304]+        extr    x2, x3, x2, #62+        sbcs    x2, x2, x7+        extr    x3, x4, x3, #62+        sbcs    x3, x3, x8+        extr    x4, x5, x4, #62+        ldp     x7, x8, [sp, #320]+        sbcs    x4, x4, x7+        extr    x5, x6, x5, #62+        sbcs    x5, x5, x8+        lsr     x6, x6, #62+        adc     x6, x6, xzr+        lsl     x7, x6, #32+        subs    x8, x6, x7+        sbc     x7, x7, xzr+        adds    x0, x0, x8+        adcs    x1, x1, x7+        adcs    x2, x2, x6+        adcs    x3, x3, xzr+        adcs    x4, x4, xzr+        adcs    x5, x5, xzr+        csetm   x8, cc  // cc = lo, ul, last+        mov     x9, #0xffffffff                 // #4294967295+        and     x9, x9, x8+        adds    x0, x0, x9+        eor     x9, x9, x8+        adcs    x1, x1, x9+        mov     x9, #0xfffffffffffffffe         // #-2+        and     x9, x9, x8+        adcs    x2, x2, x9+        adcs    x3, x3, x8+        adcs    x4, x4, x8+        adc     x5, x5, x8+        stp     x0, x1, [x25]+        stp     x2, x3, [x25, #16]+        stp     x4, x5, [x25, #32]+        ldp     x0, x1, [sp, #192]+        mov     x6, #0xffffffff                 // #4294967295+        subs    x6, x6, x0+        mov     x7, #0xffffffff00000000         // #-4294967296+        sbcs    x7, x7, x1+        ldp     x0, x1, [sp, #208]+        mov     x8, #0xfffffffffffffffe         // #-2+        sbcs    x8, x8, x0+        mov     x13, #0xffffffffffffffff        // #-1+        sbcs    x9, x13, x1+        ldp     x0, x1, [sp, #224]+        sbcs    x10, x13, x0+        sbc     x11, x13, x1+        lsl     x0, x6, #3+        extr    x1, x7, x6, #61+        extr    x2, x8, x7, #61+        extr    x3, x9, x8, #61+        extr    x4, x10, x9, #61+        extr    x5, x11, x10, #61+        lsr     x6, x11, #61+        add     x6, x6, #0x1+        ldp     x8, x9, [sp, #240]+        ldp     x10, x11, [sp, #256]+        mov     x14, #0x3                       // #3+        mul     x15, x14, x8+        umulh   x8, x14, x8+        adds    x0, x0, x15+        mul     x15, x14, x9+        umulh   x9, x14, x9+        adcs    x1, x1, x15+        mul     x15, x14, x10+        umulh   x10, x14, x10+        adcs    x2, x2, x15+        mul     x15, x14, x11+        umulh   x11, x14, x11+        adcs    x3, x3, x15+        mul     x15, x14, x12+        umulh   x12, x14, x12+        adcs    x4, x4, x15+        mul     x15, x14, x19+        umulh   x13, x14, x19+        adcs    x5, x5, x15+        adc     x6, x6, xzr+        adds    x1, x1, x8+        adcs    x2, x2, x9+        adcs    x3, x3, x10+        adcs    x4, x4, x11+        adcs    x5, x5, x12+        adcs    x6, x6, x13+        lsl     x7, x6, #32+        subs    x8, x6, x7+        sbc     x7, x7, xzr+        adds    x0, x0, x8+        adcs    x1, x1, x7+        adcs    x2, x2, x6+        adcs    x3, x3, xzr+        adcs    x4, x4, xzr+        adcs    x5, x5, xzr+        csetm   x6, cc  // cc = lo, ul, last+        mov     x7, #0xffffffff                 // #4294967295+        and     x7, x7, x6+        adds    x0, x0, x7+        eor     x7, x7, x6+        adcs    x1, x1, x7+        mov     x7, #0xfffffffffffffffe         // #-2+        and     x7, x7, x6+        adcs    x2, x2, x7+        adcs    x3, x3, x6+        adcs    x4, x4, x6+        adc     x5, x5, x6+        stp     x0, x1, [x25, #48]+        stp     x2, x3, [x25, #64]+        stp     x4, x5, [x25, #80]+        ldp     x19, x20, [sp, #336]+        ldp     x21, x22, [sp, #352]+        ldp     x23, x24, [sp, #368]+        ldp     x25, x26, [sp, #384]+        ldp     x27, xzr, [sp, #400]+        CFI_INC_SP(416)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp384_montjscalarmul_p384_montjdouble)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/p384_montjscalarmul_alt.S view
@@ -0,0 +1,7155 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Montgomery-Jacobian form scalar multiplication for P-384+// Input scalar[6], point[18]; output res[18]+//+// extern void p384_montjscalarmul_alt+//   (uint64_t res[static 18],+//    const uint64_t scalar[static 6],+//    const uint64_t point[static 18]);+//+// This function is a variant of its affine point version p384_scalarmul_alt.+// Here, input and output points are assumed to be in Jacobian form with+// their coordinates in the Montgomery domain. Thus, if priming indicates+// Montgomery form, x' = (2^384 * x) mod p_384 etc., each point argument+// is a triple (x',y',z') representing the affine point (x/z^2,y/z^3) when+// z' is nonzero or the point at infinity (group identity) if z' = 0.+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-384, returns a representation of n * P. If the result is the+// point at infinity (either because the input point was or because the+// scalar was a multiple of p_384) then the output is guaranteed to+// represent the point at infinity, i.e. to have its z coordinate zero.+//+// Standard ARM ABI: X0 = res, X1 = scalar, X2 = point+// ----------------------------------------------------------------------------+++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p384_montjscalarmul_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p384_montjscalarmul_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p384_montjscalarmul_alt)+++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 48+#define JACSIZE (3*NUMSIZE)++// Safe copies of input res and additional values in variables.++#define bf x22+#define sgn x23+#define j x24+#define res x25++// Intermediate variables on the stack.+// The table is 16 entries, each of size JACSIZE = 3 * NUMSIZE++#define scalarb sp, #(0*NUMSIZE)+#define acc sp, #(1*NUMSIZE)+#define tabent sp, #(4*NUMSIZE)++#define tab sp, #(7*NUMSIZE)++#define NSPACE 55*NUMSIZE++// Avoid using .rep for the sake of the BoringSSL/AWS-LC delocator,+// which doesn't accept repetitions, assembler macros etc.++#define selectblock(I)                            \+        cmp     bf, #(1*I) __LF                      \+        ldp     x20, x21, [x19] __LF                 \+        csel    x0, x20, x0, eq __LF                 \+        csel    x1, x21, x1, eq __LF                 \+        ldp     x20, x21, [x19, #16] __LF            \+        csel    x2, x20, x2, eq __LF                 \+        csel    x3, x21, x3, eq __LF                 \+        ldp     x20, x21, [x19, #32] __LF            \+        csel    x4, x20, x4, eq __LF                 \+        csel    x5, x21, x5, eq __LF                 \+        ldp     x20, x21, [x19, #48] __LF            \+        csel    x6, x20, x6, eq __LF                 \+        csel    x7, x21, x7, eq __LF                 \+        ldp     x20, x21, [x19, #64] __LF            \+        csel    x8, x20, x8, eq __LF                 \+        csel    x9, x21, x9, eq __LF                 \+        ldp     x20, x21, [x19, #80] __LF            \+        csel    x10, x20, x10, eq __LF               \+        csel    x11, x21, x11, eq __LF               \+        ldp     x20, x21, [x19, #96] __LF            \+        csel    x12, x20, x12, eq __LF               \+        csel    x13, x21, x13, eq __LF               \+        ldp     x20, x21, [x19, #112] __LF           \+        csel    x14, x20, x14, eq __LF               \+        csel    x15, x21, x15, eq __LF               \+        ldp     x20, x21, [x19, #128] __LF           \+        csel    x16, x20, x16, eq __LF               \+        csel    x17, x21, x17, eq __LF               \+        add     x19, x19, #JACSIZE++// Loading large constants++#define movbig(nn,n3,n2,n1,n0)                                      \+        movz    nn, n0 __LF                                            \+        movk    nn, n1, lsl #16 __LF                                   \+        movk    nn, n2, lsl #32 __LF                                   \+        movk    nn, n3, lsl #48++S2N_BN_SYMBOL(p384_montjscalarmul_alt):+        CFI_START++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x30)+        CFI_DEC_SP(NSPACE)++// Preserve the "res" input argument; others get processed early.++        mov     res, x0++// Reduce the input scalar mod n_384, i.e. conditionally subtract n_384.+// Store it to "scalarb".++        ldp     x3, x4, [x1]+        movbig(x15, #0xecec, #0x196a, #0xccc5, #0x2973)+        ldp     x5, x6, [x1, #16]+        movbig(x16, #0x581a, #0x0db2, #0x48b0, #0xa77a)+        ldp     x7, x8, [x1, #32]+        movbig(x17, #0xc763, #0x4d81, #0xf437, #0x2ddf)++        subs    x9, x3, x15+        sbcs    x10, x4, x16+        sbcs    x11, x5, x17+        adcs    x12, x6, xzr+        adcs    x13, x7, xzr+        adcs    x14, x8, xzr++        csel    x3, x3, x9, cc+        csel    x4, x4, x10, cc+        csel    x5, x5, x11, cc+        csel    x6, x6, x12, cc+        csel    x7, x7, x13, cc+        csel    x8, x8, x14, cc++        stp     x3, x4, [scalarb]+        stp     x5, x6, [scalarb+16]+        stp     x7, x8, [scalarb+32]++// Set the tab[0] table entry to the input point = 1 * P++        ldp     x10, x11, [x2]+        stp     x10, x11, [tab]+        ldp     x12, x13, [x2, #16]+        stp     x12, x13, [tab+16]+        ldp     x14, x15, [x2, #32]+        stp     x14, x15, [tab+32]++        ldp     x10, x11, [x2, #48]+        stp     x10, x11, [tab+48]+        ldp     x12, x13, [x2, #64]+        stp     x12, x13, [tab+64]+        ldp     x14, x15, [x2, #80]+        stp     x14, x15, [tab+80]++        ldp     x10, x11, [x2, #96]+        stp     x10, x11, [tab+96]+        ldp     x12, x13, [x2, #112]+        stp     x12, x13, [tab+112]+        ldp     x14, x15, [x2, #128]+        stp     x14, x15, [tab+128]++// Compute and record tab[1] = 2 * p, ..., tab[15] = 16 * P++        add     x0, tab+JACSIZE*1+        add     x1, tab+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjdouble)++        add     x0, tab+JACSIZE*2+        add     x1, tab+JACSIZE*1+        add     x2, tab+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjadd)++        add     x0, tab+JACSIZE*3+        add     x1, tab+JACSIZE*1+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjdouble)++        add     x0, tab+JACSIZE*4+        add     x1, tab+JACSIZE*3+        add     x2, tab+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjadd)++        add     x0, tab+JACSIZE*5+        add     x1, tab+JACSIZE*2+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjdouble)++        add     x0, tab+JACSIZE*6+        add     x1, tab+JACSIZE*5+        add     x2, tab+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjadd)++        add     x0, tab+JACSIZE*7+        add     x1, tab+JACSIZE*3+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjdouble)++        add     x0, tab+JACSIZE*8+        add     x1, tab+JACSIZE*7+        add     x2, tab+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjadd)++        add     x0, tab+JACSIZE*9+        add     x1, tab+JACSIZE*4+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjdouble)++        add     x0, tab+JACSIZE*10+        add     x1, tab+JACSIZE*9+        add     x2, tab+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjadd)++        add     x0, tab+JACSIZE*11+        add     x1, tab+JACSIZE*5+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjdouble)++        add     x0, tab+JACSIZE*12+        add     x1, tab+JACSIZE*11+        add     x2, tab+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjadd)++        add     x0, tab+JACSIZE*13+        add     x1, tab+JACSIZE*6+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjdouble)++        add     x0, tab+JACSIZE*14+        add     x1, tab+JACSIZE*13+        add     x2, tab+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjadd)++        add     x0, tab+JACSIZE*15+        add     x1, tab+JACSIZE*7+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjdouble)++// Add the recoding constant sum_i(16 * 32^i) to the scalar to allow signed+// digits. The digits of the constant, in lowest-to-highest order, are as+// follows; they are generated dynamically since none is a simple ARM load.+//+// 0x0842108421084210+// 0x1084210842108421+// 0x2108421084210842+// 0x4210842108421084+// 0x8421084210842108+// 0x0842108421084210++        ldp     x0, x1, [scalarb]+        ldp     x2, x3, [scalarb+16]+        ldp     x4, x5, [scalarb+32]+        movbig(x8, #0x1084, #0x2108, #0x4210, #0x8421)+        adds    x0, x0, x8, lsr #1+        adcs    x1, x1, x8+        lsl     x8, x8, #1+        adcs    x2, x2, x8+        lsl     x8, x8, #1+        adcs    x3, x3, x8+        lsl     x8, x8, #1+        adcs    x4, x4, x8+        lsr     x8, x8, #4+        adcs    x5, x5, x8+        cset    x6, cs++// Record the top bitfield then shift the whole scalar left 4 bits+// to align the top of the next bitfield with the MSB (bits 379..383).++        extr    bf, x6, x5, #60+        extr    x5, x5, x4, #60+        extr    x4, x4, x3, #60+        extr    x3, x3, x2, #60+        extr    x2, x2, x1, #60+        extr    x1, x1, x0, #60+        lsl     x0, x0, #4+        stp     x0, x1, [scalarb]+        stp     x2, x3, [scalarb+16]+        stp     x4, x5, [scalarb+32]++// Initialize the accumulator to the corresponding entry using constant-time+// lookup in the table. This top digit, uniquely, is not recoded so there is+// no sign adjustment to make.++        mov     x0, xzr+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, xzr+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        mov     x9, xzr+        mov     x10, xzr+        mov     x11, xzr+        mov     x12, xzr+        mov     x13, xzr+        mov     x14, xzr+        mov     x15, xzr+        mov     x16, xzr+        mov     x17, xzr++        add     x19, tab++        selectblock(1)+        selectblock(2)+        selectblock(3)+        selectblock(4)+        selectblock(5)+        selectblock(6)+        selectblock(7)+        selectblock(8)+        selectblock(9)+        selectblock(10)+        selectblock(11)+        selectblock(12)+        selectblock(13)+        selectblock(14)+        selectblock(15)+        selectblock(16)++        stp     x0, x1, [acc]+        stp     x2, x3, [acc+16]+        stp     x4, x5, [acc+32]+        stp     x6, x7, [acc+48]+        stp     x8, x9, [acc+64]+        stp     x10, x11, [acc+80]+        stp     x12, x13, [acc+96]+        stp     x14, x15, [acc+112]+        stp     x16, x17, [acc+128]++        mov     j, #380++// Main loop over size-5 bitfields: double 5 times then add signed digit+// At each stage we shift the scalar left by 5 bits so we can simply pick+// the top 5 bits as the bitfield, saving some fiddle over indexing.++Lp384_montjscalarmul_alt_mainloop:+        sub     j, j, #5++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjdouble)++// Choose the bitfield and adjust it to sign and magnitude++        ldp     x0, x1, [scalarb]+        ldp     x2, x3, [scalarb+16]+        ldp     x4, x5, [scalarb+32]+        lsr     bf, x5, #59+        extr    x5, x5, x4, #59+        extr    x4, x4, x3, #59+        extr    x3, x3, x2, #59+        extr    x2, x2, x1, #59+        extr    x1, x1, x0, #59+        lsl     x0, x0, #5+        stp     x0, x1, [scalarb]+        stp     x2, x3, [scalarb+16]+        stp     x4, x5, [scalarb+32]++        subs    bf, bf, #16+        cset    sgn, lo                 // sgn = sign of digit (1 = negative)+        cneg    bf, bf, lo              // bf = absolute value of digit++// Conditionally select the table entry tab[i-1] = i * P in constant time++        mov     x0, xzr+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, xzr+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        mov     x9, xzr+        mov     x10, xzr+        mov     x11, xzr+        mov     x12, xzr+        mov     x13, xzr+        mov     x14, xzr+        mov     x15, xzr+        mov     x16, xzr+        mov     x17, xzr++        add     x19, tab++        selectblock(1)+        selectblock(2)+        selectblock(3)+        selectblock(4)+        selectblock(5)+        selectblock(6)+        selectblock(7)+        selectblock(8)+        selectblock(9)+        selectblock(10)+        selectblock(11)+        selectblock(12)+        selectblock(13)+        selectblock(14)+        selectblock(15)+        selectblock(16)++// Store it to "tabent" with the y coordinate optionally negated.+// This is done carefully to give coordinates < p_384 even in+// the degenerate case y = 0 (when z = 0 for points on the curve).++        stp     x0, x1, [tabent]+        stp     x2, x3, [tabent+16]+        stp     x4, x5, [tabent+32]++        stp     x12, x13, [tabent+96]+        stp     x14, x15, [tabent+112]+        stp     x16, x17, [tabent+128]++        mov     x0, #0x00000000ffffffff+        subs    x0, x0, x6+        orr     x12, x6, x7+        mov     x1, #0xffffffff00000000+        sbcs    x1, x1, x7+        orr     x13, x8, x9+        mov     x2, #0xfffffffffffffffe+        sbcs    x2, x2, x8+        orr     x14, x10, x11+        mov     x5, #0xffffffffffffffff+        sbcs    x3, x5, x9+        orr     x12, x12, x13+        sbcs    x4, x5, x10+        orr     x12, x12, x14+        sbcs    x5, x5, x11++        cmp     sgn, xzr+        ccmp    x12, xzr, #4, ne++        csel    x6, x0, x6, ne+        csel    x7, x1, x7, ne+        csel    x8, x2, x8, ne+        csel    x9, x3, x9, ne+        csel    x10, x4, x10, ne+        csel    x11, x5, x11, ne++        stp     x6, x7, [tabent+48]+        stp     x8, x9, [tabent+64]+        stp     x10, x11, [tabent+80]++// Add to the accumulator++        add     x0, acc+        add     x1, acc+        add     x2, tabent+        CFI_BL(Lp384_montjscalarmul_alt_p384_montjadd)++        cbnz    j, Lp384_montjscalarmul_alt_mainloop++// That's the end of the main loop, and we just need to copy the+// result in "acc" to the output.++        ldp     x0, x1, [acc]+        stp     x0, x1, [res]+        ldp     x0, x1, [acc+16]+        stp     x0, x1, [res, #16]+        ldp     x0, x1, [acc+32]+        stp     x0, x1, [res, #32]+        ldp     x0, x1, [acc+48]+        stp     x0, x1, [res, #48]+        ldp     x0, x1, [acc+64]+        stp     x0, x1, [res, #64]+        ldp     x0, x1, [acc+80]+        stp     x0, x1, [res, #80]+        ldp     x0, x1, [acc+96]+        stp     x0, x1, [res, #96]+        ldp     x0, x1, [acc+112]+        stp     x0, x1, [res, #112]+        ldp     x0, x1, [acc+128]+        stp     x0, x1, [res, #128]++// Restore stack and registers and return++        CFI_INC_SP(NSPACE)+        CFI_POP2(x25,x30)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p384_montjscalarmul_alt)++// Local copies of subroutines, complete clones at the moment++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp384_montjscalarmul_alt_p384_montjadd)++Lp384_montjscalarmul_alt_p384_montjadd:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x26)+        CFI_DEC_SP(336)+        mov     x24, x0+        mov     x25, x1+        mov     x26, x2+        ldp     x2, x3, [x25, #96]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [x25, #112]+        mul     x8, x2, x4+        adds    x10, x10, x8+        mul     x11, x2, x5+        mul     x8, x3, x4+        adcs    x11, x11, x8+        umulh   x12, x2, x5+        mul     x8, x3, x5+        adcs    x12, x12, x8+        ldp     x6, x7, [x25, #128]+        mul     x13, x2, x7+        mul     x8, x3, x6+        adcs    x13, x13, x8+        umulh   x14, x2, x7+        mul     x8, x3, x7+        adcs    x14, x14, x8+        mul     x15, x5, x6+        adcs    x15, x15, xzr+        umulh   x16, x5, x6+        adc     x16, x16, xzr+        umulh   x8, x2, x4+        adds    x11, x11, x8+        umulh   x8, x3, x4+        adcs    x12, x12, x8+        umulh   x8, x3, x5+        adcs    x13, x13, x8+        umulh   x8, x3, x6+        adcs    x14, x14, x8+        umulh   x8, x3, x7+        adcs    x15, x15, x8+        adc     x16, x16, xzr+        mul     x8, x2, x6+        adds    x12, x12, x8+        mul     x8, x4, x5+        adcs    x13, x13, x8+        mul     x8, x4, x6+        adcs    x14, x14, x8+        mul     x8, x4, x7+        adcs    x15, x15, x8+        mul     x8, x5, x7+        adcs    x16, x16, x8+        mul     x17, x6, x7+        adcs    x17, x17, xzr+        umulh   x19, x6, x7+        adc     x19, x19, xzr+        umulh   x8, x2, x6+        adds    x13, x13, x8+        umulh   x8, x4, x5+        adcs    x14, x14, x8+        umulh   x8, x4, x6+        adcs    x15, x15, x8+        umulh   x8, x4, x7+        adcs    x16, x16, x8+        umulh   x8, x5, x7+        adcs    x17, x17, x8+        adc     x19, x19, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        adcs    x15, x15, x15+        adcs    x16, x16, x16+        adcs    x17, x17, x17+        adcs    x19, x19, x19+        cset    x20, hs+        umulh   x8, x2, x2+        mul     x2, x2, x2+        adds    x9, x9, x8+        mul     x8, x3, x3+        adcs    x10, x10, x8+        umulh   x8, x3, x3+        adcs    x11, x11, x8+        mul     x8, x4, x4+        adcs    x12, x12, x8+        umulh   x8, x4, x4+        adcs    x13, x13, x8+        mul     x8, x5, x5+        adcs    x14, x14, x8+        umulh   x8, x5, x5+        adcs    x15, x15, x8+        mul     x8, x6, x6+        adcs    x16, x16, x8+        umulh   x8, x6, x6+        adcs    x17, x17, x8+        mul     x8, x7, x7+        adcs    x19, x19, x8+        umulh   x8, x7, x7+        adc     x20, x20, x8+        lsl     x5, x2, #32+        add     x2, x5, x2+        mov     x5, #-4294967295+        umulh   x5, x5, x2+        mov     x4, #4294967295+        mul     x3, x4, x2+        umulh   x4, x4, x2+        adds    x5, x5, x3+        adcs    x4, x4, x2+        adc     x3, xzr, xzr+        subs    x9, x9, x5+        sbcs    x10, x10, x4+        sbcs    x11, x11, x3+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x2, x2, xzr+        lsl     x5, x9, #32+        add     x9, x5, x9+        mov     x5, #-4294967295+        umulh   x5, x5, x9+        mov     x4, #4294967295+        mul     x3, x4, x9+        umulh   x4, x4, x9+        adds    x5, x5, x3+        adcs    x4, x4, x9+        adc     x3, xzr, xzr+        subs    x10, x10, x5+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        sbcs    x13, x13, xzr+        sbcs    x2, x2, xzr+        sbc     x9, x9, xzr+        lsl     x5, x10, #32+        add     x10, x5, x10+        mov     x5, #-4294967295+        umulh   x5, x5, x10+        mov     x4, #4294967295+        mul     x3, x4, x10+        umulh   x4, x4, x10+        adds    x5, x5, x3+        adcs    x4, x4, x10+        adc     x3, xzr, xzr+        subs    x11, x11, x5+        sbcs    x12, x12, x4+        sbcs    x13, x13, x3+        sbcs    x2, x2, xzr+        sbcs    x9, x9, xzr+        sbc     x10, x10, xzr+        lsl     x5, x11, #32+        add     x11, x5, x11+        mov     x5, #-4294967295+        umulh   x5, x5, x11+        mov     x4, #4294967295+        mul     x3, x4, x11+        umulh   x4, x4, x11+        adds    x5, x5, x3+        adcs    x4, x4, x11+        adc     x3, xzr, xzr+        subs    x12, x12, x5+        sbcs    x13, x13, x4+        sbcs    x2, x2, x3+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbc     x11, x11, xzr+        lsl     x5, x12, #32+        add     x12, x5, x12+        mov     x5, #-4294967295+        umulh   x5, x5, x12+        mov     x4, #4294967295+        mul     x3, x4, x12+        umulh   x4, x4, x12+        adds    x5, x5, x3+        adcs    x4, x4, x12+        adc     x3, xzr, xzr+        subs    x13, x13, x5+        sbcs    x2, x2, x4+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbc     x12, x12, xzr+        lsl     x5, x13, #32+        add     x13, x5, x13+        mov     x5, #-4294967295+        umulh   x5, x5, x13+        mov     x4, #4294967295+        mul     x3, x4, x13+        umulh   x4, x4, x13+        adds    x5, x5, x3+        adcs    x4, x4, x13+        adc     x3, xzr, xzr+        subs    x2, x2, x5+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        adds    x2, x2, x14+        adcs    x9, x9, x15+        adcs    x10, x10, x16+        adcs    x11, x11, x17+        adcs    x12, x12, x19+        adcs    x13, x13, x20+        mov     x14, #-4294967295+        mov     x15, #4294967295+        csel    x14, x14, xzr, hs+        csel    x15, x15, xzr, hs+        cset    x16, hs+        adds    x2, x2, x14+        adcs    x9, x9, x15+        adcs    x10, x10, x16+        adcs    x11, x11, xzr+        adcs    x12, x12, xzr+        adc     x13, x13, xzr+        stp     x2, x9, [sp]+        stp     x10, x11, [sp, #16]+        stp     x12, x13, [sp, #32]+        ldp     x2, x3, [x26, #96]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [x26, #112]+        mul     x8, x2, x4+        adds    x10, x10, x8+        mul     x11, x2, x5+        mul     x8, x3, x4+        adcs    x11, x11, x8+        umulh   x12, x2, x5+        mul     x8, x3, x5+        adcs    x12, x12, x8+        ldp     x6, x7, [x26, #128]+        mul     x13, x2, x7+        mul     x8, x3, x6+        adcs    x13, x13, x8+        umulh   x14, x2, x7+        mul     x8, x3, x7+        adcs    x14, x14, x8+        mul     x15, x5, x6+        adcs    x15, x15, xzr+        umulh   x16, x5, x6+        adc     x16, x16, xzr+        umulh   x8, x2, x4+        adds    x11, x11, x8+        umulh   x8, x3, x4+        adcs    x12, x12, x8+        umulh   x8, x3, x5+        adcs    x13, x13, x8+        umulh   x8, x3, x6+        adcs    x14, x14, x8+        umulh   x8, x3, x7+        adcs    x15, x15, x8+        adc     x16, x16, xzr+        mul     x8, x2, x6+        adds    x12, x12, x8+        mul     x8, x4, x5+        adcs    x13, x13, x8+        mul     x8, x4, x6+        adcs    x14, x14, x8+        mul     x8, x4, x7+        adcs    x15, x15, x8+        mul     x8, x5, x7+        adcs    x16, x16, x8+        mul     x17, x6, x7+        adcs    x17, x17, xzr+        umulh   x19, x6, x7+        adc     x19, x19, xzr+        umulh   x8, x2, x6+        adds    x13, x13, x8+        umulh   x8, x4, x5+        adcs    x14, x14, x8+        umulh   x8, x4, x6+        adcs    x15, x15, x8+        umulh   x8, x4, x7+        adcs    x16, x16, x8+        umulh   x8, x5, x7+        adcs    x17, x17, x8+        adc     x19, x19, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        adcs    x15, x15, x15+        adcs    x16, x16, x16+        adcs    x17, x17, x17+        adcs    x19, x19, x19+        cset    x20, hs+        umulh   x8, x2, x2+        mul     x2, x2, x2+        adds    x9, x9, x8+        mul     x8, x3, x3+        adcs    x10, x10, x8+        umulh   x8, x3, x3+        adcs    x11, x11, x8+        mul     x8, x4, x4+        adcs    x12, x12, x8+        umulh   x8, x4, x4+        adcs    x13, x13, x8+        mul     x8, x5, x5+        adcs    x14, x14, x8+        umulh   x8, x5, x5+        adcs    x15, x15, x8+        mul     x8, x6, x6+        adcs    x16, x16, x8+        umulh   x8, x6, x6+        adcs    x17, x17, x8+        mul     x8, x7, x7+        adcs    x19, x19, x8+        umulh   x8, x7, x7+        adc     x20, x20, x8+        lsl     x5, x2, #32+        add     x2, x5, x2+        mov     x5, #-4294967295+        umulh   x5, x5, x2+        mov     x4, #4294967295+        mul     x3, x4, x2+        umulh   x4, x4, x2+        adds    x5, x5, x3+        adcs    x4, x4, x2+        adc     x3, xzr, xzr+        subs    x9, x9, x5+        sbcs    x10, x10, x4+        sbcs    x11, x11, x3+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x2, x2, xzr+        lsl     x5, x9, #32+        add     x9, x5, x9+        mov     x5, #-4294967295+        umulh   x5, x5, x9+        mov     x4, #4294967295+        mul     x3, x4, x9+        umulh   x4, x4, x9+        adds    x5, x5, x3+        adcs    x4, x4, x9+        adc     x3, xzr, xzr+        subs    x10, x10, x5+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        sbcs    x13, x13, xzr+        sbcs    x2, x2, xzr+        sbc     x9, x9, xzr+        lsl     x5, x10, #32+        add     x10, x5, x10+        mov     x5, #-4294967295+        umulh   x5, x5, x10+        mov     x4, #4294967295+        mul     x3, x4, x10+        umulh   x4, x4, x10+        adds    x5, x5, x3+        adcs    x4, x4, x10+        adc     x3, xzr, xzr+        subs    x11, x11, x5+        sbcs    x12, x12, x4+        sbcs    x13, x13, x3+        sbcs    x2, x2, xzr+        sbcs    x9, x9, xzr+        sbc     x10, x10, xzr+        lsl     x5, x11, #32+        add     x11, x5, x11+        mov     x5, #-4294967295+        umulh   x5, x5, x11+        mov     x4, #4294967295+        mul     x3, x4, x11+        umulh   x4, x4, x11+        adds    x5, x5, x3+        adcs    x4, x4, x11+        adc     x3, xzr, xzr+        subs    x12, x12, x5+        sbcs    x13, x13, x4+        sbcs    x2, x2, x3+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbc     x11, x11, xzr+        lsl     x5, x12, #32+        add     x12, x5, x12+        mov     x5, #-4294967295+        umulh   x5, x5, x12+        mov     x4, #4294967295+        mul     x3, x4, x12+        umulh   x4, x4, x12+        adds    x5, x5, x3+        adcs    x4, x4, x12+        adc     x3, xzr, xzr+        subs    x13, x13, x5+        sbcs    x2, x2, x4+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbc     x12, x12, xzr+        lsl     x5, x13, #32+        add     x13, x5, x13+        mov     x5, #-4294967295+        umulh   x5, x5, x13+        mov     x4, #4294967295+        mul     x3, x4, x13+        umulh   x4, x4, x13+        adds    x5, x5, x3+        adcs    x4, x4, x13+        adc     x3, xzr, xzr+        subs    x2, x2, x5+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        adds    x2, x2, x14+        adcs    x9, x9, x15+        adcs    x10, x10, x16+        adcs    x11, x11, x17+        adcs    x12, x12, x19+        adcs    x13, x13, x20+        mov     x14, #-4294967295+        mov     x15, #4294967295+        csel    x14, x14, xzr, hs+        csel    x15, x15, xzr, hs+        cset    x16, hs+        adds    x2, x2, x14+        adcs    x9, x9, x15+        adcs    x10, x10, x16+        adcs    x11, x11, xzr+        adcs    x12, x12, xzr+        adc     x13, x13, xzr+        stp     x2, x9, [sp, #240]+        stp     x10, x11, [sp, #256]+        stp     x12, x13, [sp, #272]+        ldp     x3, x4, [x26, #96]+        ldp     x5, x6, [x25, #48]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [x25, #64]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [x25, #80]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [x26, #112]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [x26, #128]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #288]+        stp     x14, x15, [sp, #304]+        stp     x16, x17, [sp, #320]+        ldp     x3, x4, [x25, #96]+        ldp     x5, x6, [x26, #48]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [x26, #64]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [x26, #80]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [x25, #112]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [x25, #128]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #48]+        stp     x14, x15, [sp, #64]+        stp     x16, x17, [sp, #80]+        ldp     x3, x4, [sp]+        ldp     x5, x6, [x26]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [x26, #16]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [x26, #32]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [sp, #16]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [sp, #32]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #96]+        stp     x14, x15, [sp, #112]+        stp     x16, x17, [sp, #128]+        ldp     x3, x4, [sp, #240]+        ldp     x5, x6, [x25]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [x25, #16]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [x25, #32]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [sp, #256]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [sp, #272]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #192]+        stp     x14, x15, [sp, #208]+        stp     x16, x17, [sp, #224]+        ldp     x3, x4, [sp]+        ldp     x5, x6, [sp, #48]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [sp, #64]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [sp, #80]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [sp, #16]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [sp, #32]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #48]+        stp     x14, x15, [sp, #64]+        stp     x16, x17, [sp, #80]+        ldp     x3, x4, [sp, #240]+        ldp     x5, x6, [sp, #288]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [sp, #304]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [sp, #320]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [sp, #256]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [sp, #272]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #288]+        stp     x14, x15, [sp, #304]+        stp     x16, x17, [sp, #320]+        ldp     x5, x6, [sp, #96]+        ldp     x4, x3, [sp, #192]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #112]+        ldp     x4, x3, [sp, #208]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #128]+        ldp     x4, x3, [sp, #224]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, lo+        mov     x4, #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [sp, #240]+        stp     x7, x8, [sp, #256]+        stp     x9, x10, [sp, #272]+        ldp     x5, x6, [sp, #48]+        ldp     x4, x3, [sp, #288]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #64]+        ldp     x4, x3, [sp, #304]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #80]+        ldp     x4, x3, [sp, #320]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, lo+        mov     x4, #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [sp, #48]+        stp     x7, x8, [sp, #64]+        stp     x9, x10, [sp, #80]+        ldp     x2, x3, [sp, #240]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [sp, #256]+        mul     x8, x2, x4+        adds    x10, x10, x8+        mul     x11, x2, x5+        mul     x8, x3, x4+        adcs    x11, x11, x8+        umulh   x12, x2, x5+        mul     x8, x3, x5+        adcs    x12, x12, x8+        ldp     x6, x7, [sp, #272]+        mul     x13, x2, x7+        mul     x8, x3, x6+        adcs    x13, x13, x8+        umulh   x14, x2, x7+        mul     x8, x3, x7+        adcs    x14, x14, x8+        mul     x15, x5, x6+        adcs    x15, x15, xzr+        umulh   x16, x5, x6+        adc     x16, x16, xzr+        umulh   x8, x2, x4+        adds    x11, x11, x8+        umulh   x8, x3, x4+        adcs    x12, x12, x8+        umulh   x8, x3, x5+        adcs    x13, x13, x8+        umulh   x8, x3, x6+        adcs    x14, x14, x8+        umulh   x8, x3, x7+        adcs    x15, x15, x8+        adc     x16, x16, xzr+        mul     x8, x2, x6+        adds    x12, x12, x8+        mul     x8, x4, x5+        adcs    x13, x13, x8+        mul     x8, x4, x6+        adcs    x14, x14, x8+        mul     x8, x4, x7+        adcs    x15, x15, x8+        mul     x8, x5, x7+        adcs    x16, x16, x8+        mul     x17, x6, x7+        adcs    x17, x17, xzr+        umulh   x19, x6, x7+        adc     x19, x19, xzr+        umulh   x8, x2, x6+        adds    x13, x13, x8+        umulh   x8, x4, x5+        adcs    x14, x14, x8+        umulh   x8, x4, x6+        adcs    x15, x15, x8+        umulh   x8, x4, x7+        adcs    x16, x16, x8+        umulh   x8, x5, x7+        adcs    x17, x17, x8+        adc     x19, x19, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        adcs    x15, x15, x15+        adcs    x16, x16, x16+        adcs    x17, x17, x17+        adcs    x19, x19, x19+        cset    x20, hs+        umulh   x8, x2, x2+        mul     x2, x2, x2+        adds    x9, x9, x8+        mul     x8, x3, x3+        adcs    x10, x10, x8+        umulh   x8, x3, x3+        adcs    x11, x11, x8+        mul     x8, x4, x4+        adcs    x12, x12, x8+        umulh   x8, x4, x4+        adcs    x13, x13, x8+        mul     x8, x5, x5+        adcs    x14, x14, x8+        umulh   x8, x5, x5+        adcs    x15, x15, x8+        mul     x8, x6, x6+        adcs    x16, x16, x8+        umulh   x8, x6, x6+        adcs    x17, x17, x8+        mul     x8, x7, x7+        adcs    x19, x19, x8+        umulh   x8, x7, x7+        adc     x20, x20, x8+        lsl     x5, x2, #32+        add     x2, x5, x2+        mov     x5, #-4294967295+        umulh   x5, x5, x2+        mov     x4, #4294967295+        mul     x3, x4, x2+        umulh   x4, x4, x2+        adds    x5, x5, x3+        adcs    x4, x4, x2+        adc     x3, xzr, xzr+        subs    x9, x9, x5+        sbcs    x10, x10, x4+        sbcs    x11, x11, x3+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x2, x2, xzr+        lsl     x5, x9, #32+        add     x9, x5, x9+        mov     x5, #-4294967295+        umulh   x5, x5, x9+        mov     x4, #4294967295+        mul     x3, x4, x9+        umulh   x4, x4, x9+        adds    x5, x5, x3+        adcs    x4, x4, x9+        adc     x3, xzr, xzr+        subs    x10, x10, x5+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        sbcs    x13, x13, xzr+        sbcs    x2, x2, xzr+        sbc     x9, x9, xzr+        lsl     x5, x10, #32+        add     x10, x5, x10+        mov     x5, #-4294967295+        umulh   x5, x5, x10+        mov     x4, #4294967295+        mul     x3, x4, x10+        umulh   x4, x4, x10+        adds    x5, x5, x3+        adcs    x4, x4, x10+        adc     x3, xzr, xzr+        subs    x11, x11, x5+        sbcs    x12, x12, x4+        sbcs    x13, x13, x3+        sbcs    x2, x2, xzr+        sbcs    x9, x9, xzr+        sbc     x10, x10, xzr+        lsl     x5, x11, #32+        add     x11, x5, x11+        mov     x5, #-4294967295+        umulh   x5, x5, x11+        mov     x4, #4294967295+        mul     x3, x4, x11+        umulh   x4, x4, x11+        adds    x5, x5, x3+        adcs    x4, x4, x11+        adc     x3, xzr, xzr+        subs    x12, x12, x5+        sbcs    x13, x13, x4+        sbcs    x2, x2, x3+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbc     x11, x11, xzr+        lsl     x5, x12, #32+        add     x12, x5, x12+        mov     x5, #-4294967295+        umulh   x5, x5, x12+        mov     x4, #4294967295+        mul     x3, x4, x12+        umulh   x4, x4, x12+        adds    x5, x5, x3+        adcs    x4, x4, x12+        adc     x3, xzr, xzr+        subs    x13, x13, x5+        sbcs    x2, x2, x4+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbc     x12, x12, xzr+        lsl     x5, x13, #32+        add     x13, x5, x13+        mov     x5, #-4294967295+        umulh   x5, x5, x13+        mov     x4, #4294967295+        mul     x3, x4, x13+        umulh   x4, x4, x13+        adds    x5, x5, x3+        adcs    x4, x4, x13+        adc     x3, xzr, xzr+        subs    x2, x2, x5+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        adds    x2, x2, x14+        adcs    x9, x9, x15+        adcs    x10, x10, x16+        adcs    x11, x11, x17+        adcs    x12, x12, x19+        adcs    x13, x13, x20+        mov     x14, #-4294967295+        mov     x15, #4294967295+        csel    x14, x14, xzr, hs+        csel    x15, x15, xzr, hs+        cset    x16, hs+        adds    x2, x2, x14+        adcs    x9, x9, x15+        adcs    x10, x10, x16+        adcs    x11, x11, xzr+        adcs    x12, x12, xzr+        adc     x13, x13, xzr+        stp     x2, x9, [sp, #144]+        stp     x10, x11, [sp, #160]+        stp     x12, x13, [sp, #176]+        ldp     x2, x3, [sp, #48]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [sp, #64]+        mul     x8, x2, x4+        adds    x10, x10, x8+        mul     x11, x2, x5+        mul     x8, x3, x4+        adcs    x11, x11, x8+        umulh   x12, x2, x5+        mul     x8, x3, x5+        adcs    x12, x12, x8+        ldp     x6, x7, [sp, #80]+        mul     x13, x2, x7+        mul     x8, x3, x6+        adcs    x13, x13, x8+        umulh   x14, x2, x7+        mul     x8, x3, x7+        adcs    x14, x14, x8+        mul     x15, x5, x6+        adcs    x15, x15, xzr+        umulh   x16, x5, x6+        adc     x16, x16, xzr+        umulh   x8, x2, x4+        adds    x11, x11, x8+        umulh   x8, x3, x4+        adcs    x12, x12, x8+        umulh   x8, x3, x5+        adcs    x13, x13, x8+        umulh   x8, x3, x6+        adcs    x14, x14, x8+        umulh   x8, x3, x7+        adcs    x15, x15, x8+        adc     x16, x16, xzr+        mul     x8, x2, x6+        adds    x12, x12, x8+        mul     x8, x4, x5+        adcs    x13, x13, x8+        mul     x8, x4, x6+        adcs    x14, x14, x8+        mul     x8, x4, x7+        adcs    x15, x15, x8+        mul     x8, x5, x7+        adcs    x16, x16, x8+        mul     x17, x6, x7+        adcs    x17, x17, xzr+        umulh   x19, x6, x7+        adc     x19, x19, xzr+        umulh   x8, x2, x6+        adds    x13, x13, x8+        umulh   x8, x4, x5+        adcs    x14, x14, x8+        umulh   x8, x4, x6+        adcs    x15, x15, x8+        umulh   x8, x4, x7+        adcs    x16, x16, x8+        umulh   x8, x5, x7+        adcs    x17, x17, x8+        adc     x19, x19, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        adcs    x15, x15, x15+        adcs    x16, x16, x16+        adcs    x17, x17, x17+        adcs    x19, x19, x19+        cset    x20, hs+        umulh   x8, x2, x2+        mul     x2, x2, x2+        adds    x9, x9, x8+        mul     x8, x3, x3+        adcs    x10, x10, x8+        umulh   x8, x3, x3+        adcs    x11, x11, x8+        mul     x8, x4, x4+        adcs    x12, x12, x8+        umulh   x8, x4, x4+        adcs    x13, x13, x8+        mul     x8, x5, x5+        adcs    x14, x14, x8+        umulh   x8, x5, x5+        adcs    x15, x15, x8+        mul     x8, x6, x6+        adcs    x16, x16, x8+        umulh   x8, x6, x6+        adcs    x17, x17, x8+        mul     x8, x7, x7+        adcs    x19, x19, x8+        umulh   x8, x7, x7+        adc     x20, x20, x8+        lsl     x5, x2, #32+        add     x2, x5, x2+        mov     x5, #-4294967295+        umulh   x5, x5, x2+        mov     x4, #4294967295+        mul     x3, x4, x2+        umulh   x4, x4, x2+        adds    x5, x5, x3+        adcs    x4, x4, x2+        adc     x3, xzr, xzr+        subs    x9, x9, x5+        sbcs    x10, x10, x4+        sbcs    x11, x11, x3+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x2, x2, xzr+        lsl     x5, x9, #32+        add     x9, x5, x9+        mov     x5, #-4294967295+        umulh   x5, x5, x9+        mov     x4, #4294967295+        mul     x3, x4, x9+        umulh   x4, x4, x9+        adds    x5, x5, x3+        adcs    x4, x4, x9+        adc     x3, xzr, xzr+        subs    x10, x10, x5+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        sbcs    x13, x13, xzr+        sbcs    x2, x2, xzr+        sbc     x9, x9, xzr+        lsl     x5, x10, #32+        add     x10, x5, x10+        mov     x5, #-4294967295+        umulh   x5, x5, x10+        mov     x4, #4294967295+        mul     x3, x4, x10+        umulh   x4, x4, x10+        adds    x5, x5, x3+        adcs    x4, x4, x10+        adc     x3, xzr, xzr+        subs    x11, x11, x5+        sbcs    x12, x12, x4+        sbcs    x13, x13, x3+        sbcs    x2, x2, xzr+        sbcs    x9, x9, xzr+        sbc     x10, x10, xzr+        lsl     x5, x11, #32+        add     x11, x5, x11+        mov     x5, #-4294967295+        umulh   x5, x5, x11+        mov     x4, #4294967295+        mul     x3, x4, x11+        umulh   x4, x4, x11+        adds    x5, x5, x3+        adcs    x4, x4, x11+        adc     x3, xzr, xzr+        subs    x12, x12, x5+        sbcs    x13, x13, x4+        sbcs    x2, x2, x3+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbc     x11, x11, xzr+        lsl     x5, x12, #32+        add     x12, x5, x12+        mov     x5, #-4294967295+        umulh   x5, x5, x12+        mov     x4, #4294967295+        mul     x3, x4, x12+        umulh   x4, x4, x12+        adds    x5, x5, x3+        adcs    x4, x4, x12+        adc     x3, xzr, xzr+        subs    x13, x13, x5+        sbcs    x2, x2, x4+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbc     x12, x12, xzr+        lsl     x5, x13, #32+        add     x13, x5, x13+        mov     x5, #-4294967295+        umulh   x5, x5, x13+        mov     x4, #4294967295+        mul     x3, x4, x13+        umulh   x4, x4, x13+        adds    x5, x5, x3+        adcs    x4, x4, x13+        adc     x3, xzr, xzr+        subs    x2, x2, x5+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        adds    x2, x2, x14+        adcs    x9, x9, x15+        adcs    x10, x10, x16+        adcs    x11, x11, x17+        adcs    x12, x12, x19+        adcs    x13, x13, x20+        adc     x6, xzr, xzr+        mov     x8, #-4294967295+        adds    x14, x2, x8+        mov     x8, #4294967295+        adcs    x15, x9, x8+        mov     x8, #1+        adcs    x16, x10, x8+        adcs    x17, x11, xzr+        adcs    x19, x12, xzr+        adcs    x20, x13, xzr+        adcs    x6, x6, xzr+        csel    x2, x2, x14, eq+        csel    x9, x9, x15, eq+        csel    x10, x10, x16, eq+        csel    x11, x11, x17, eq+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        stp     x2, x9, [sp]+        stp     x10, x11, [sp, #16]+        stp     x12, x13, [sp, #32]+        ldp     x3, x4, [sp, #144]+        ldp     x5, x6, [sp, #192]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [sp, #208]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [sp, #224]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [sp, #160]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [sp, #176]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #192]+        stp     x14, x15, [sp, #208]+        stp     x16, x17, [sp, #224]+        ldp     x3, x4, [sp, #144]+        ldp     x5, x6, [sp, #96]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [sp, #112]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [sp, #128]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [sp, #160]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [sp, #176]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #96]+        stp     x14, x15, [sp, #112]+        stp     x16, x17, [sp, #128]+        ldp     x5, x6, [sp]+        ldp     x4, x3, [sp, #192]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #16]+        ldp     x4, x3, [sp, #208]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #32]+        ldp     x4, x3, [sp, #224]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, lo+        mov     x4, #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [sp]+        stp     x7, x8, [sp, #16]+        stp     x9, x10, [sp, #32]+        ldp     x5, x6, [sp, #96]+        ldp     x4, x3, [sp, #192]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #112]+        ldp     x4, x3, [sp, #208]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #128]+        ldp     x4, x3, [sp, #224]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, lo+        mov     x4, #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [sp, #144]+        stp     x7, x8, [sp, #160]+        stp     x9, x10, [sp, #176]+        ldp     x3, x4, [sp, #240]+        ldp     x5, x6, [x25, #96]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [x25, #112]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [x25, #128]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [sp, #256]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [sp, #272]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #240]+        stp     x14, x15, [sp, #256]+        stp     x16, x17, [sp, #272]+        ldp     x5, x6, [sp]+        ldp     x4, x3, [sp, #96]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #16]+        ldp     x4, x3, [sp, #112]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #32]+        ldp     x4, x3, [sp, #128]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, lo+        mov     x4, #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [sp]+        stp     x7, x8, [sp, #16]+        stp     x9, x10, [sp, #32]+        ldp     x5, x6, [sp, #192]+        ldp     x4, x3, [sp]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #208]+        ldp     x4, x3, [sp, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #224]+        ldp     x4, x3, [sp, #32]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, lo+        mov     x4, #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [sp, #192]+        stp     x7, x8, [sp, #208]+        stp     x9, x10, [sp, #224]+        ldp     x3, x4, [sp, #144]+        ldp     x5, x6, [sp, #288]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [sp, #304]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [sp, #320]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [sp, #160]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [sp, #176]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #144]+        stp     x14, x15, [sp, #160]+        stp     x16, x17, [sp, #176]+        ldp     x3, x4, [sp, #240]+        ldp     x5, x6, [x26, #96]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [x26, #112]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [x26, #128]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [sp, #256]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [sp, #272]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #240]+        stp     x14, x15, [sp, #256]+        stp     x16, x17, [sp, #272]+        ldp     x3, x4, [sp, #48]+        ldp     x5, x6, [sp, #192]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [sp, #208]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [sp, #224]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [sp, #64]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [sp, #80]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #192]+        stp     x14, x15, [sp, #208]+        stp     x16, x17, [sp, #224]+        ldp     x5, x6, [sp, #192]+        ldp     x4, x3, [sp, #144]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #208]+        ldp     x4, x3, [sp, #160]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #224]+        ldp     x4, x3, [sp, #176]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, lo+        mov     x4, #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [sp, #192]+        stp     x7, x8, [sp, #208]+        stp     x9, x10, [sp, #224]+        ldp     x0, x1, [x25, #96]+        ldp     x2, x3, [x25, #112]+        ldp     x4, x5, [x25, #128]+        orr     x20, x0, x1+        orr     x21, x2, x3+        orr     x22, x4, x5+        orr     x20, x20, x21+        orr     x20, x20, x22+        cmp     x20, xzr+        cset    x20, ne+        ldp     x6, x7, [x26, #96]+        ldp     x8, x9, [x26, #112]+        ldp     x10, x11, [x26, #128]+        orr     x21, x6, x7+        orr     x22, x8, x9+        orr     x23, x10, x11+        orr     x21, x21, x22+        orr     x21, x21, x23+        cmp     x21, xzr+        cset    x21, ne+        cmp     x21, x20+        ldp     x12, x13, [sp, #240]+        csel    x12, x0, x12, lo+        csel    x13, x1, x13, lo+        csel    x12, x6, x12, hi+        csel    x13, x7, x13, hi+        ldp     x14, x15, [sp, #256]+        csel    x14, x2, x14, lo+        csel    x15, x3, x15, lo+        csel    x14, x8, x14, hi+        csel    x15, x9, x15, hi+        ldp     x16, x17, [sp, #272]+        csel    x16, x4, x16, lo+        csel    x17, x5, x17, lo+        csel    x16, x10, x16, hi+        csel    x17, x11, x17, hi+        ldp     x20, x21, [x25]+        ldp     x0, x1, [sp]+        csel    x0, x20, x0, lo+        csel    x1, x21, x1, lo+        ldp     x20, x21, [x26]+        csel    x0, x20, x0, hi+        csel    x1, x21, x1, hi+        ldp     x20, x21, [x25, #16]+        ldp     x2, x3, [sp, #16]+        csel    x2, x20, x2, lo+        csel    x3, x21, x3, lo+        ldp     x20, x21, [x26, #16]+        csel    x2, x20, x2, hi+        csel    x3, x21, x3, hi+        ldp     x20, x21, [x25, #32]+        ldp     x4, x5, [sp, #32]+        csel    x4, x20, x4, lo+        csel    x5, x21, x5, lo+        ldp     x20, x21, [x26, #32]+        csel    x4, x20, x4, hi+        csel    x5, x21, x5, hi+        ldp     x20, x21, [x25, #48]+        ldp     x6, x7, [sp, #192]+        csel    x6, x20, x6, lo+        csel    x7, x21, x7, lo+        ldp     x20, x21, [x26, #48]+        csel    x6, x20, x6, hi+        csel    x7, x21, x7, hi+        ldp     x20, x21, [x25, #64]+        ldp     x8, x9, [sp, #208]+        csel    x8, x20, x8, lo+        csel    x9, x21, x9, lo+        ldp     x20, x21, [x26, #64]+        csel    x8, x20, x8, hi+        csel    x9, x21, x9, hi+        ldp     x20, x21, [x25, #80]+        ldp     x10, x11, [sp, #224]+        csel    x10, x20, x10, lo+        csel    x11, x21, x11, lo+        ldp     x20, x21, [x26, #80]+        csel    x10, x20, x10, hi+        csel    x11, x21, x11, hi+        stp     x0, x1, [x24]+        stp     x2, x3, [x24, #16]+        stp     x4, x5, [x24, #32]+        stp     x6, x7, [x24, #48]+        stp     x8, x9, [x24, #64]+        stp     x10, x11, [x24, #80]+        stp     x12, x13, [x24, #96]+        stp     x14, x15, [x24, #112]+        stp     x16, x17, [x24, #128]+        CFI_INC_SP(336)+        CFI_POP2(x25,x26)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp384_montjscalarmul_alt_p384_montjadd)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp384_montjscalarmul_alt_p384_montjdouble)++Lp384_montjscalarmul_alt_p384_montjdouble:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_DEC_SP(336)+        mov     x23, x0+        mov     x24, x1+        ldp     x2, x3, [x24, #96]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [x24, #112]+        mul     x8, x2, x4+        adds    x10, x10, x8+        mul     x11, x2, x5+        mul     x8, x3, x4+        adcs    x11, x11, x8+        umulh   x12, x2, x5+        mul     x8, x3, x5+        adcs    x12, x12, x8+        ldp     x6, x7, [x24, #128]+        mul     x13, x2, x7+        mul     x8, x3, x6+        adcs    x13, x13, x8+        umulh   x14, x2, x7+        mul     x8, x3, x7+        adcs    x14, x14, x8+        mul     x15, x5, x6+        adcs    x15, x15, xzr+        umulh   x16, x5, x6+        adc     x16, x16, xzr+        umulh   x8, x2, x4+        adds    x11, x11, x8+        umulh   x8, x3, x4+        adcs    x12, x12, x8+        umulh   x8, x3, x5+        adcs    x13, x13, x8+        umulh   x8, x3, x6+        adcs    x14, x14, x8+        umulh   x8, x3, x7+        adcs    x15, x15, x8+        adc     x16, x16, xzr+        mul     x8, x2, x6+        adds    x12, x12, x8+        mul     x8, x4, x5+        adcs    x13, x13, x8+        mul     x8, x4, x6+        adcs    x14, x14, x8+        mul     x8, x4, x7+        adcs    x15, x15, x8+        mul     x8, x5, x7+        adcs    x16, x16, x8+        mul     x17, x6, x7+        adcs    x17, x17, xzr+        umulh   x19, x6, x7+        adc     x19, x19, xzr+        umulh   x8, x2, x6+        adds    x13, x13, x8+        umulh   x8, x4, x5+        adcs    x14, x14, x8+        umulh   x8, x4, x6+        adcs    x15, x15, x8+        umulh   x8, x4, x7+        adcs    x16, x16, x8+        umulh   x8, x5, x7+        adcs    x17, x17, x8+        adc     x19, x19, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        adcs    x15, x15, x15+        adcs    x16, x16, x16+        adcs    x17, x17, x17+        adcs    x19, x19, x19+        cset    x20, hs+        umulh   x8, x2, x2+        mul     x2, x2, x2+        adds    x9, x9, x8+        mul     x8, x3, x3+        adcs    x10, x10, x8+        umulh   x8, x3, x3+        adcs    x11, x11, x8+        mul     x8, x4, x4+        adcs    x12, x12, x8+        umulh   x8, x4, x4+        adcs    x13, x13, x8+        mul     x8, x5, x5+        adcs    x14, x14, x8+        umulh   x8, x5, x5+        adcs    x15, x15, x8+        mul     x8, x6, x6+        adcs    x16, x16, x8+        umulh   x8, x6, x6+        adcs    x17, x17, x8+        mul     x8, x7, x7+        adcs    x19, x19, x8+        umulh   x8, x7, x7+        adc     x20, x20, x8+        lsl     x5, x2, #32+        add     x2, x5, x2+        mov     x5, #-4294967295+        umulh   x5, x5, x2+        mov     x4, #4294967295+        mul     x3, x4, x2+        umulh   x4, x4, x2+        adds    x5, x5, x3+        adcs    x4, x4, x2+        adc     x3, xzr, xzr+        subs    x9, x9, x5+        sbcs    x10, x10, x4+        sbcs    x11, x11, x3+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x2, x2, xzr+        lsl     x5, x9, #32+        add     x9, x5, x9+        mov     x5, #-4294967295+        umulh   x5, x5, x9+        mov     x4, #4294967295+        mul     x3, x4, x9+        umulh   x4, x4, x9+        adds    x5, x5, x3+        adcs    x4, x4, x9+        adc     x3, xzr, xzr+        subs    x10, x10, x5+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        sbcs    x13, x13, xzr+        sbcs    x2, x2, xzr+        sbc     x9, x9, xzr+        lsl     x5, x10, #32+        add     x10, x5, x10+        mov     x5, #-4294967295+        umulh   x5, x5, x10+        mov     x4, #4294967295+        mul     x3, x4, x10+        umulh   x4, x4, x10+        adds    x5, x5, x3+        adcs    x4, x4, x10+        adc     x3, xzr, xzr+        subs    x11, x11, x5+        sbcs    x12, x12, x4+        sbcs    x13, x13, x3+        sbcs    x2, x2, xzr+        sbcs    x9, x9, xzr+        sbc     x10, x10, xzr+        lsl     x5, x11, #32+        add     x11, x5, x11+        mov     x5, #-4294967295+        umulh   x5, x5, x11+        mov     x4, #4294967295+        mul     x3, x4, x11+        umulh   x4, x4, x11+        adds    x5, x5, x3+        adcs    x4, x4, x11+        adc     x3, xzr, xzr+        subs    x12, x12, x5+        sbcs    x13, x13, x4+        sbcs    x2, x2, x3+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbc     x11, x11, xzr+        lsl     x5, x12, #32+        add     x12, x5, x12+        mov     x5, #-4294967295+        umulh   x5, x5, x12+        mov     x4, #4294967295+        mul     x3, x4, x12+        umulh   x4, x4, x12+        adds    x5, x5, x3+        adcs    x4, x4, x12+        adc     x3, xzr, xzr+        subs    x13, x13, x5+        sbcs    x2, x2, x4+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbc     x12, x12, xzr+        lsl     x5, x13, #32+        add     x13, x5, x13+        mov     x5, #-4294967295+        umulh   x5, x5, x13+        mov     x4, #4294967295+        mul     x3, x4, x13+        umulh   x4, x4, x13+        adds    x5, x5, x3+        adcs    x4, x4, x13+        adc     x3, xzr, xzr+        subs    x2, x2, x5+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        adds    x2, x2, x14+        adcs    x9, x9, x15+        adcs    x10, x10, x16+        adcs    x11, x11, x17+        adcs    x12, x12, x19+        adcs    x13, x13, x20+        adc     x6, xzr, xzr+        mov     x8, #-4294967295+        adds    x14, x2, x8+        mov     x8, #4294967295+        adcs    x15, x9, x8+        mov     x8, #1+        adcs    x16, x10, x8+        adcs    x17, x11, xzr+        adcs    x19, x12, xzr+        adcs    x20, x13, xzr+        adcs    x6, x6, xzr+        csel    x2, x2, x14, eq+        csel    x9, x9, x15, eq+        csel    x10, x10, x16, eq+        csel    x11, x11, x17, eq+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        stp     x2, x9, [sp]+        stp     x10, x11, [sp, #16]+        stp     x12, x13, [sp, #32]+        ldp     x2, x3, [x24, #48]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [x24, #64]+        mul     x8, x2, x4+        adds    x10, x10, x8+        mul     x11, x2, x5+        mul     x8, x3, x4+        adcs    x11, x11, x8+        umulh   x12, x2, x5+        mul     x8, x3, x5+        adcs    x12, x12, x8+        ldp     x6, x7, [x24, #80]+        mul     x13, x2, x7+        mul     x8, x3, x6+        adcs    x13, x13, x8+        umulh   x14, x2, x7+        mul     x8, x3, x7+        adcs    x14, x14, x8+        mul     x15, x5, x6+        adcs    x15, x15, xzr+        umulh   x16, x5, x6+        adc     x16, x16, xzr+        umulh   x8, x2, x4+        adds    x11, x11, x8+        umulh   x8, x3, x4+        adcs    x12, x12, x8+        umulh   x8, x3, x5+        adcs    x13, x13, x8+        umulh   x8, x3, x6+        adcs    x14, x14, x8+        umulh   x8, x3, x7+        adcs    x15, x15, x8+        adc     x16, x16, xzr+        mul     x8, x2, x6+        adds    x12, x12, x8+        mul     x8, x4, x5+        adcs    x13, x13, x8+        mul     x8, x4, x6+        adcs    x14, x14, x8+        mul     x8, x4, x7+        adcs    x15, x15, x8+        mul     x8, x5, x7+        adcs    x16, x16, x8+        mul     x17, x6, x7+        adcs    x17, x17, xzr+        umulh   x19, x6, x7+        adc     x19, x19, xzr+        umulh   x8, x2, x6+        adds    x13, x13, x8+        umulh   x8, x4, x5+        adcs    x14, x14, x8+        umulh   x8, x4, x6+        adcs    x15, x15, x8+        umulh   x8, x4, x7+        adcs    x16, x16, x8+        umulh   x8, x5, x7+        adcs    x17, x17, x8+        adc     x19, x19, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        adcs    x15, x15, x15+        adcs    x16, x16, x16+        adcs    x17, x17, x17+        adcs    x19, x19, x19+        cset    x20, hs+        umulh   x8, x2, x2+        mul     x2, x2, x2+        adds    x9, x9, x8+        mul     x8, x3, x3+        adcs    x10, x10, x8+        umulh   x8, x3, x3+        adcs    x11, x11, x8+        mul     x8, x4, x4+        adcs    x12, x12, x8+        umulh   x8, x4, x4+        adcs    x13, x13, x8+        mul     x8, x5, x5+        adcs    x14, x14, x8+        umulh   x8, x5, x5+        adcs    x15, x15, x8+        mul     x8, x6, x6+        adcs    x16, x16, x8+        umulh   x8, x6, x6+        adcs    x17, x17, x8+        mul     x8, x7, x7+        adcs    x19, x19, x8+        umulh   x8, x7, x7+        adc     x20, x20, x8+        lsl     x5, x2, #32+        add     x2, x5, x2+        mov     x5, #-4294967295+        umulh   x5, x5, x2+        mov     x4, #4294967295+        mul     x3, x4, x2+        umulh   x4, x4, x2+        adds    x5, x5, x3+        adcs    x4, x4, x2+        adc     x3, xzr, xzr+        subs    x9, x9, x5+        sbcs    x10, x10, x4+        sbcs    x11, x11, x3+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x2, x2, xzr+        lsl     x5, x9, #32+        add     x9, x5, x9+        mov     x5, #-4294967295+        umulh   x5, x5, x9+        mov     x4, #4294967295+        mul     x3, x4, x9+        umulh   x4, x4, x9+        adds    x5, x5, x3+        adcs    x4, x4, x9+        adc     x3, xzr, xzr+        subs    x10, x10, x5+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        sbcs    x13, x13, xzr+        sbcs    x2, x2, xzr+        sbc     x9, x9, xzr+        lsl     x5, x10, #32+        add     x10, x5, x10+        mov     x5, #-4294967295+        umulh   x5, x5, x10+        mov     x4, #4294967295+        mul     x3, x4, x10+        umulh   x4, x4, x10+        adds    x5, x5, x3+        adcs    x4, x4, x10+        adc     x3, xzr, xzr+        subs    x11, x11, x5+        sbcs    x12, x12, x4+        sbcs    x13, x13, x3+        sbcs    x2, x2, xzr+        sbcs    x9, x9, xzr+        sbc     x10, x10, xzr+        lsl     x5, x11, #32+        add     x11, x5, x11+        mov     x5, #-4294967295+        umulh   x5, x5, x11+        mov     x4, #4294967295+        mul     x3, x4, x11+        umulh   x4, x4, x11+        adds    x5, x5, x3+        adcs    x4, x4, x11+        adc     x3, xzr, xzr+        subs    x12, x12, x5+        sbcs    x13, x13, x4+        sbcs    x2, x2, x3+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbc     x11, x11, xzr+        lsl     x5, x12, #32+        add     x12, x5, x12+        mov     x5, #-4294967295+        umulh   x5, x5, x12+        mov     x4, #4294967295+        mul     x3, x4, x12+        umulh   x4, x4, x12+        adds    x5, x5, x3+        adcs    x4, x4, x12+        adc     x3, xzr, xzr+        subs    x13, x13, x5+        sbcs    x2, x2, x4+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbc     x12, x12, xzr+        lsl     x5, x13, #32+        add     x13, x5, x13+        mov     x5, #-4294967295+        umulh   x5, x5, x13+        mov     x4, #4294967295+        mul     x3, x4, x13+        umulh   x4, x4, x13+        adds    x5, x5, x3+        adcs    x4, x4, x13+        adc     x3, xzr, xzr+        subs    x2, x2, x5+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        adds    x2, x2, x14+        adcs    x9, x9, x15+        adcs    x10, x10, x16+        adcs    x11, x11, x17+        adcs    x12, x12, x19+        adcs    x13, x13, x20+        adc     x6, xzr, xzr+        mov     x8, #-4294967295+        adds    x14, x2, x8+        mov     x8, #4294967295+        adcs    x15, x9, x8+        mov     x8, #1+        adcs    x16, x10, x8+        adcs    x17, x11, xzr+        adcs    x19, x12, xzr+        adcs    x20, x13, xzr+        adcs    x6, x6, xzr+        csel    x2, x2, x14, eq+        csel    x9, x9, x15, eq+        csel    x10, x10, x16, eq+        csel    x11, x11, x17, eq+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        stp     x2, x9, [sp, #48]+        stp     x10, x11, [sp, #64]+        stp     x12, x13, [sp, #80]+        ldp     x5, x6, [x24]+        ldp     x4, x3, [sp]+        adds    x5, x5, x4+        adcs    x6, x6, x3+        ldp     x7, x8, [x24, #16]+        ldp     x4, x3, [sp, #16]+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        ldp     x9, x10, [x24, #32]+        ldp     x4, x3, [sp, #32]+        adcs    x9, x9, x4+        adcs    x10, x10, x3+        csetm   x3, hs+        mov     x4, #4294967295+        and     x4, x4, x3+        subs    x5, x5, x4+        eor     x4, x4, x3+        sbcs    x6, x6, x4+        mov     x4, #-2+        and     x4, x4, x3+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        sbcs    x9, x9, x3+        sbc     x10, x10, x3+        stp     x5, x6, [sp, #240]+        stp     x7, x8, [sp, #256]+        stp     x9, x10, [sp, #272]+        ldp     x5, x6, [x24]+        ldp     x4, x3, [sp]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [x24, #16]+        ldp     x4, x3, [sp, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [x24, #32]+        ldp     x4, x3, [sp, #32]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, lo+        mov     x4, #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [sp, #192]+        stp     x7, x8, [sp, #208]+        stp     x9, x10, [sp, #224]+        ldp     x3, x4, [sp, #240]+        ldp     x5, x6, [sp, #192]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [sp, #208]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [sp, #224]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [sp, #256]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [sp, #272]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #96]+        stp     x14, x15, [sp, #112]+        stp     x16, x17, [sp, #128]+        ldp     x5, x6, [x24, #48]+        ldp     x4, x3, [x24, #96]+        adds    x5, x5, x4+        adcs    x6, x6, x3+        ldp     x7, x8, [x24, #64]+        ldp     x4, x3, [x24, #112]+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        ldp     x9, x10, [x24, #80]+        ldp     x4, x3, [x24, #128]+        adcs    x9, x9, x4+        adcs    x10, x10, x3+        adc     x3, xzr, xzr+        mov     x4, #4294967295+        cmp     x5, x4+        mov     x4, #-4294967296+        sbcs    xzr, x6, x4+        mov     x4, #-2+        sbcs    xzr, x7, x4+        adcs    xzr, x8, xzr+        adcs    xzr, x9, xzr+        adcs    xzr, x10, xzr+        adcs    x3, x3, xzr+        csetm   x3, ne+        mov     x4, #4294967295+        and     x4, x4, x3+        subs    x5, x5, x4+        eor     x4, x4, x3+        sbcs    x6, x6, x4+        mov     x4, #-2+        and     x4, x4, x3+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        sbcs    x9, x9, x3+        sbc     x10, x10, x3+        stp     x5, x6, [sp, #240]+        stp     x7, x8, [sp, #256]+        stp     x9, x10, [sp, #272]+        ldp     x2, x3, [sp, #96]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [sp, #112]+        mul     x8, x2, x4+        adds    x10, x10, x8+        mul     x11, x2, x5+        mul     x8, x3, x4+        adcs    x11, x11, x8+        umulh   x12, x2, x5+        mul     x8, x3, x5+        adcs    x12, x12, x8+        ldp     x6, x7, [sp, #128]+        mul     x13, x2, x7+        mul     x8, x3, x6+        adcs    x13, x13, x8+        umulh   x14, x2, x7+        mul     x8, x3, x7+        adcs    x14, x14, x8+        mul     x15, x5, x6+        adcs    x15, x15, xzr+        umulh   x16, x5, x6+        adc     x16, x16, xzr+        umulh   x8, x2, x4+        adds    x11, x11, x8+        umulh   x8, x3, x4+        adcs    x12, x12, x8+        umulh   x8, x3, x5+        adcs    x13, x13, x8+        umulh   x8, x3, x6+        adcs    x14, x14, x8+        umulh   x8, x3, x7+        adcs    x15, x15, x8+        adc     x16, x16, xzr+        mul     x8, x2, x6+        adds    x12, x12, x8+        mul     x8, x4, x5+        adcs    x13, x13, x8+        mul     x8, x4, x6+        adcs    x14, x14, x8+        mul     x8, x4, x7+        adcs    x15, x15, x8+        mul     x8, x5, x7+        adcs    x16, x16, x8+        mul     x17, x6, x7+        adcs    x17, x17, xzr+        umulh   x19, x6, x7+        adc     x19, x19, xzr+        umulh   x8, x2, x6+        adds    x13, x13, x8+        umulh   x8, x4, x5+        adcs    x14, x14, x8+        umulh   x8, x4, x6+        adcs    x15, x15, x8+        umulh   x8, x4, x7+        adcs    x16, x16, x8+        umulh   x8, x5, x7+        adcs    x17, x17, x8+        adc     x19, x19, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        adcs    x15, x15, x15+        adcs    x16, x16, x16+        adcs    x17, x17, x17+        adcs    x19, x19, x19+        cset    x20, hs+        umulh   x8, x2, x2+        mul     x2, x2, x2+        adds    x9, x9, x8+        mul     x8, x3, x3+        adcs    x10, x10, x8+        umulh   x8, x3, x3+        adcs    x11, x11, x8+        mul     x8, x4, x4+        adcs    x12, x12, x8+        umulh   x8, x4, x4+        adcs    x13, x13, x8+        mul     x8, x5, x5+        adcs    x14, x14, x8+        umulh   x8, x5, x5+        adcs    x15, x15, x8+        mul     x8, x6, x6+        adcs    x16, x16, x8+        umulh   x8, x6, x6+        adcs    x17, x17, x8+        mul     x8, x7, x7+        adcs    x19, x19, x8+        umulh   x8, x7, x7+        adc     x20, x20, x8+        lsl     x5, x2, #32+        add     x2, x5, x2+        mov     x5, #-4294967295+        umulh   x5, x5, x2+        mov     x4, #4294967295+        mul     x3, x4, x2+        umulh   x4, x4, x2+        adds    x5, x5, x3+        adcs    x4, x4, x2+        adc     x3, xzr, xzr+        subs    x9, x9, x5+        sbcs    x10, x10, x4+        sbcs    x11, x11, x3+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x2, x2, xzr+        lsl     x5, x9, #32+        add     x9, x5, x9+        mov     x5, #-4294967295+        umulh   x5, x5, x9+        mov     x4, #4294967295+        mul     x3, x4, x9+        umulh   x4, x4, x9+        adds    x5, x5, x3+        adcs    x4, x4, x9+        adc     x3, xzr, xzr+        subs    x10, x10, x5+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        sbcs    x13, x13, xzr+        sbcs    x2, x2, xzr+        sbc     x9, x9, xzr+        lsl     x5, x10, #32+        add     x10, x5, x10+        mov     x5, #-4294967295+        umulh   x5, x5, x10+        mov     x4, #4294967295+        mul     x3, x4, x10+        umulh   x4, x4, x10+        adds    x5, x5, x3+        adcs    x4, x4, x10+        adc     x3, xzr, xzr+        subs    x11, x11, x5+        sbcs    x12, x12, x4+        sbcs    x13, x13, x3+        sbcs    x2, x2, xzr+        sbcs    x9, x9, xzr+        sbc     x10, x10, xzr+        lsl     x5, x11, #32+        add     x11, x5, x11+        mov     x5, #-4294967295+        umulh   x5, x5, x11+        mov     x4, #4294967295+        mul     x3, x4, x11+        umulh   x4, x4, x11+        adds    x5, x5, x3+        adcs    x4, x4, x11+        adc     x3, xzr, xzr+        subs    x12, x12, x5+        sbcs    x13, x13, x4+        sbcs    x2, x2, x3+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbc     x11, x11, xzr+        lsl     x5, x12, #32+        add     x12, x5, x12+        mov     x5, #-4294967295+        umulh   x5, x5, x12+        mov     x4, #4294967295+        mul     x3, x4, x12+        umulh   x4, x4, x12+        adds    x5, x5, x3+        adcs    x4, x4, x12+        adc     x3, xzr, xzr+        subs    x13, x13, x5+        sbcs    x2, x2, x4+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbc     x12, x12, xzr+        lsl     x5, x13, #32+        add     x13, x5, x13+        mov     x5, #-4294967295+        umulh   x5, x5, x13+        mov     x4, #4294967295+        mul     x3, x4, x13+        umulh   x4, x4, x13+        adds    x5, x5, x3+        adcs    x4, x4, x13+        adc     x3, xzr, xzr+        subs    x2, x2, x5+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        adds    x2, x2, x14+        adcs    x9, x9, x15+        adcs    x10, x10, x16+        adcs    x11, x11, x17+        adcs    x12, x12, x19+        adcs    x13, x13, x20+        adc     x6, xzr, xzr+        mov     x8, #-4294967295+        adds    x14, x2, x8+        mov     x8, #4294967295+        adcs    x15, x9, x8+        mov     x8, #1+        adcs    x16, x10, x8+        adcs    x17, x11, xzr+        adcs    x19, x12, xzr+        adcs    x20, x13, xzr+        adcs    x6, x6, xzr+        csel    x2, x2, x14, eq+        csel    x9, x9, x15, eq+        csel    x10, x10, x16, eq+        csel    x11, x11, x17, eq+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        stp     x2, x9, [sp, #288]+        stp     x10, x11, [sp, #304]+        stp     x12, x13, [sp, #320]+        ldp     x3, x4, [x24]+        ldp     x5, x6, [sp, #48]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [sp, #64]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [sp, #80]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [x24, #16]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [x24, #32]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #144]+        stp     x14, x15, [sp, #160]+        stp     x16, x17, [sp, #176]+        ldp     x2, x3, [sp, #240]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [sp, #256]+        mul     x8, x2, x4+        adds    x10, x10, x8+        mul     x11, x2, x5+        mul     x8, x3, x4+        adcs    x11, x11, x8+        umulh   x12, x2, x5+        mul     x8, x3, x5+        adcs    x12, x12, x8+        ldp     x6, x7, [sp, #272]+        mul     x13, x2, x7+        mul     x8, x3, x6+        adcs    x13, x13, x8+        umulh   x14, x2, x7+        mul     x8, x3, x7+        adcs    x14, x14, x8+        mul     x15, x5, x6+        adcs    x15, x15, xzr+        umulh   x16, x5, x6+        adc     x16, x16, xzr+        umulh   x8, x2, x4+        adds    x11, x11, x8+        umulh   x8, x3, x4+        adcs    x12, x12, x8+        umulh   x8, x3, x5+        adcs    x13, x13, x8+        umulh   x8, x3, x6+        adcs    x14, x14, x8+        umulh   x8, x3, x7+        adcs    x15, x15, x8+        adc     x16, x16, xzr+        mul     x8, x2, x6+        adds    x12, x12, x8+        mul     x8, x4, x5+        adcs    x13, x13, x8+        mul     x8, x4, x6+        adcs    x14, x14, x8+        mul     x8, x4, x7+        adcs    x15, x15, x8+        mul     x8, x5, x7+        adcs    x16, x16, x8+        mul     x17, x6, x7+        adcs    x17, x17, xzr+        umulh   x19, x6, x7+        adc     x19, x19, xzr+        umulh   x8, x2, x6+        adds    x13, x13, x8+        umulh   x8, x4, x5+        adcs    x14, x14, x8+        umulh   x8, x4, x6+        adcs    x15, x15, x8+        umulh   x8, x4, x7+        adcs    x16, x16, x8+        umulh   x8, x5, x7+        adcs    x17, x17, x8+        adc     x19, x19, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        adcs    x15, x15, x15+        adcs    x16, x16, x16+        adcs    x17, x17, x17+        adcs    x19, x19, x19+        cset    x20, hs+        umulh   x8, x2, x2+        mul     x2, x2, x2+        adds    x9, x9, x8+        mul     x8, x3, x3+        adcs    x10, x10, x8+        umulh   x8, x3, x3+        adcs    x11, x11, x8+        mul     x8, x4, x4+        adcs    x12, x12, x8+        umulh   x8, x4, x4+        adcs    x13, x13, x8+        mul     x8, x5, x5+        adcs    x14, x14, x8+        umulh   x8, x5, x5+        adcs    x15, x15, x8+        mul     x8, x6, x6+        adcs    x16, x16, x8+        umulh   x8, x6, x6+        adcs    x17, x17, x8+        mul     x8, x7, x7+        adcs    x19, x19, x8+        umulh   x8, x7, x7+        adc     x20, x20, x8+        lsl     x5, x2, #32+        add     x2, x5, x2+        mov     x5, #-4294967295+        umulh   x5, x5, x2+        mov     x4, #4294967295+        mul     x3, x4, x2+        umulh   x4, x4, x2+        adds    x5, x5, x3+        adcs    x4, x4, x2+        adc     x3, xzr, xzr+        subs    x9, x9, x5+        sbcs    x10, x10, x4+        sbcs    x11, x11, x3+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x2, x2, xzr+        lsl     x5, x9, #32+        add     x9, x5, x9+        mov     x5, #-4294967295+        umulh   x5, x5, x9+        mov     x4, #4294967295+        mul     x3, x4, x9+        umulh   x4, x4, x9+        adds    x5, x5, x3+        adcs    x4, x4, x9+        adc     x3, xzr, xzr+        subs    x10, x10, x5+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        sbcs    x13, x13, xzr+        sbcs    x2, x2, xzr+        sbc     x9, x9, xzr+        lsl     x5, x10, #32+        add     x10, x5, x10+        mov     x5, #-4294967295+        umulh   x5, x5, x10+        mov     x4, #4294967295+        mul     x3, x4, x10+        umulh   x4, x4, x10+        adds    x5, x5, x3+        adcs    x4, x4, x10+        adc     x3, xzr, xzr+        subs    x11, x11, x5+        sbcs    x12, x12, x4+        sbcs    x13, x13, x3+        sbcs    x2, x2, xzr+        sbcs    x9, x9, xzr+        sbc     x10, x10, xzr+        lsl     x5, x11, #32+        add     x11, x5, x11+        mov     x5, #-4294967295+        umulh   x5, x5, x11+        mov     x4, #4294967295+        mul     x3, x4, x11+        umulh   x4, x4, x11+        adds    x5, x5, x3+        adcs    x4, x4, x11+        adc     x3, xzr, xzr+        subs    x12, x12, x5+        sbcs    x13, x13, x4+        sbcs    x2, x2, x3+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbc     x11, x11, xzr+        lsl     x5, x12, #32+        add     x12, x5, x12+        mov     x5, #-4294967295+        umulh   x5, x5, x12+        mov     x4, #4294967295+        mul     x3, x4, x12+        umulh   x4, x4, x12+        adds    x5, x5, x3+        adcs    x4, x4, x12+        adc     x3, xzr, xzr+        subs    x13, x13, x5+        sbcs    x2, x2, x4+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbc     x12, x12, xzr+        lsl     x5, x13, #32+        add     x13, x5, x13+        mov     x5, #-4294967295+        umulh   x5, x5, x13+        mov     x4, #4294967295+        mul     x3, x4, x13+        umulh   x4, x4, x13+        adds    x5, x5, x3+        adcs    x4, x4, x13+        adc     x3, xzr, xzr+        subs    x2, x2, x5+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        adds    x2, x2, x14+        adcs    x9, x9, x15+        adcs    x10, x10, x16+        adcs    x11, x11, x17+        adcs    x12, x12, x19+        adcs    x13, x13, x20+        adc     x6, xzr, xzr+        mov     x8, #-4294967295+        adds    x14, x2, x8+        mov     x8, #4294967295+        adcs    x15, x9, x8+        mov     x8, #1+        adcs    x16, x10, x8+        adcs    x17, x11, xzr+        adcs    x19, x12, xzr+        adcs    x20, x13, xzr+        adcs    x6, x6, xzr+        csel    x2, x2, x14, eq+        csel    x9, x9, x15, eq+        csel    x10, x10, x16, eq+        csel    x11, x11, x17, eq+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        stp     x2, x9, [sp, #192]+        stp     x10, x11, [sp, #208]+        stp     x12, x13, [sp, #224]+        ldp     x0, x1, [sp, #288]+        mov     x6, #4294967295+        subs    x6, x6, x0+        mov     x7, #-4294967296+        sbcs    x7, x7, x1+        ldp     x0, x1, [sp, #304]+        mov     x8, #-2+        sbcs    x8, x8, x0+        mov     x13, #-1+        sbcs    x9, x13, x1+        ldp     x0, x1, [sp, #320]+        sbcs    x10, x13, x0+        sbc     x11, x13, x1+        mov     x12, #9+        mul     x0, x12, x6+        mul     x1, x12, x7+        mul     x2, x12, x8+        mul     x3, x12, x9+        mul     x4, x12, x10+        mul     x5, x12, x11+        umulh   x6, x12, x6+        umulh   x7, x12, x7+        umulh   x8, x12, x8+        umulh   x9, x12, x9+        umulh   x10, x12, x10+        umulh   x12, x12, x11+        adds    x1, x1, x6+        adcs    x2, x2, x7+        adcs    x3, x3, x8+        adcs    x4, x4, x9+        adcs    x5, x5, x10+        mov     x6, #1+        adc     x6, x12, x6+        ldp     x8, x9, [sp, #144]+        ldp     x10, x11, [sp, #160]+        ldp     x12, x13, [sp, #176]+        mov     x14, #12+        mul     x15, x14, x8+        umulh   x8, x14, x8+        adds    x0, x0, x15+        mul     x15, x14, x9+        umulh   x9, x14, x9+        adcs    x1, x1, x15+        mul     x15, x14, x10+        umulh   x10, x14, x10+        adcs    x2, x2, x15+        mul     x15, x14, x11+        umulh   x11, x14, x11+        adcs    x3, x3, x15+        mul     x15, x14, x12+        umulh   x12, x14, x12+        adcs    x4, x4, x15+        mul     x15, x14, x13+        umulh   x13, x14, x13+        adcs    x5, x5, x15+        adc     x6, x6, xzr+        adds    x1, x1, x8+        adcs    x2, x2, x9+        adcs    x3, x3, x10+        adcs    x4, x4, x11+        adcs    x5, x5, x12+        adcs    x6, x6, x13+        lsl     x7, x6, #32+        subs    x8, x6, x7+        sbc     x7, x7, xzr+        adds    x0, x0, x8+        adcs    x1, x1, x7+        adcs    x2, x2, x6+        adcs    x3, x3, xzr+        adcs    x4, x4, xzr+        adcs    x5, x5, xzr+        csetm   x6, lo+        mov     x7, #4294967295+        and     x7, x7, x6+        adds    x0, x0, x7+        eor     x7, x7, x6+        adcs    x1, x1, x7+        mov     x7, #-2+        and     x7, x7, x6+        adcs    x2, x2, x7+        adcs    x3, x3, x6+        adcs    x4, x4, x6+        adc     x5, x5, x6+        stp     x0, x1, [sp, #288]+        stp     x2, x3, [sp, #304]+        stp     x4, x5, [sp, #320]+        ldp     x5, x6, [sp, #192]+        ldp     x4, x3, [sp]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #208]+        ldp     x4, x3, [sp, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #224]+        ldp     x4, x3, [sp, #32]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, lo+        mov     x4, #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [sp, #240]+        stp     x7, x8, [sp, #256]+        stp     x9, x10, [sp, #272]+        ldp     x2, x3, [sp, #48]+        mul     x9, x2, x3+        umulh   x10, x2, x3+        ldp     x4, x5, [sp, #64]+        mul     x8, x2, x4+        adds    x10, x10, x8+        mul     x11, x2, x5+        mul     x8, x3, x4+        adcs    x11, x11, x8+        umulh   x12, x2, x5+        mul     x8, x3, x5+        adcs    x12, x12, x8+        ldp     x6, x7, [sp, #80]+        mul     x13, x2, x7+        mul     x8, x3, x6+        adcs    x13, x13, x8+        umulh   x14, x2, x7+        mul     x8, x3, x7+        adcs    x14, x14, x8+        mul     x15, x5, x6+        adcs    x15, x15, xzr+        umulh   x16, x5, x6+        adc     x16, x16, xzr+        umulh   x8, x2, x4+        adds    x11, x11, x8+        umulh   x8, x3, x4+        adcs    x12, x12, x8+        umulh   x8, x3, x5+        adcs    x13, x13, x8+        umulh   x8, x3, x6+        adcs    x14, x14, x8+        umulh   x8, x3, x7+        adcs    x15, x15, x8+        adc     x16, x16, xzr+        mul     x8, x2, x6+        adds    x12, x12, x8+        mul     x8, x4, x5+        adcs    x13, x13, x8+        mul     x8, x4, x6+        adcs    x14, x14, x8+        mul     x8, x4, x7+        adcs    x15, x15, x8+        mul     x8, x5, x7+        adcs    x16, x16, x8+        mul     x17, x6, x7+        adcs    x17, x17, xzr+        umulh   x19, x6, x7+        adc     x19, x19, xzr+        umulh   x8, x2, x6+        adds    x13, x13, x8+        umulh   x8, x4, x5+        adcs    x14, x14, x8+        umulh   x8, x4, x6+        adcs    x15, x15, x8+        umulh   x8, x4, x7+        adcs    x16, x16, x8+        umulh   x8, x5, x7+        adcs    x17, x17, x8+        adc     x19, x19, xzr+        adds    x9, x9, x9+        adcs    x10, x10, x10+        adcs    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        adcs    x15, x15, x15+        adcs    x16, x16, x16+        adcs    x17, x17, x17+        adcs    x19, x19, x19+        cset    x20, hs+        umulh   x8, x2, x2+        mul     x2, x2, x2+        adds    x9, x9, x8+        mul     x8, x3, x3+        adcs    x10, x10, x8+        umulh   x8, x3, x3+        adcs    x11, x11, x8+        mul     x8, x4, x4+        adcs    x12, x12, x8+        umulh   x8, x4, x4+        adcs    x13, x13, x8+        mul     x8, x5, x5+        adcs    x14, x14, x8+        umulh   x8, x5, x5+        adcs    x15, x15, x8+        mul     x8, x6, x6+        adcs    x16, x16, x8+        umulh   x8, x6, x6+        adcs    x17, x17, x8+        mul     x8, x7, x7+        adcs    x19, x19, x8+        umulh   x8, x7, x7+        adc     x20, x20, x8+        lsl     x5, x2, #32+        add     x2, x5, x2+        mov     x5, #-4294967295+        umulh   x5, x5, x2+        mov     x4, #4294967295+        mul     x3, x4, x2+        umulh   x4, x4, x2+        adds    x5, x5, x3+        adcs    x4, x4, x2+        adc     x3, xzr, xzr+        subs    x9, x9, x5+        sbcs    x10, x10, x4+        sbcs    x11, x11, x3+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x2, x2, xzr+        lsl     x5, x9, #32+        add     x9, x5, x9+        mov     x5, #-4294967295+        umulh   x5, x5, x9+        mov     x4, #4294967295+        mul     x3, x4, x9+        umulh   x4, x4, x9+        adds    x5, x5, x3+        adcs    x4, x4, x9+        adc     x3, xzr, xzr+        subs    x10, x10, x5+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        sbcs    x13, x13, xzr+        sbcs    x2, x2, xzr+        sbc     x9, x9, xzr+        lsl     x5, x10, #32+        add     x10, x5, x10+        mov     x5, #-4294967295+        umulh   x5, x5, x10+        mov     x4, #4294967295+        mul     x3, x4, x10+        umulh   x4, x4, x10+        adds    x5, x5, x3+        adcs    x4, x4, x10+        adc     x3, xzr, xzr+        subs    x11, x11, x5+        sbcs    x12, x12, x4+        sbcs    x13, x13, x3+        sbcs    x2, x2, xzr+        sbcs    x9, x9, xzr+        sbc     x10, x10, xzr+        lsl     x5, x11, #32+        add     x11, x5, x11+        mov     x5, #-4294967295+        umulh   x5, x5, x11+        mov     x4, #4294967295+        mul     x3, x4, x11+        umulh   x4, x4, x11+        adds    x5, x5, x3+        adcs    x4, x4, x11+        adc     x3, xzr, xzr+        subs    x12, x12, x5+        sbcs    x13, x13, x4+        sbcs    x2, x2, x3+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbc     x11, x11, xzr+        lsl     x5, x12, #32+        add     x12, x5, x12+        mov     x5, #-4294967295+        umulh   x5, x5, x12+        mov     x4, #4294967295+        mul     x3, x4, x12+        umulh   x4, x4, x12+        adds    x5, x5, x3+        adcs    x4, x4, x12+        adc     x3, xzr, xzr+        subs    x13, x13, x5+        sbcs    x2, x2, x4+        sbcs    x9, x9, x3+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbc     x12, x12, xzr+        lsl     x5, x13, #32+        add     x13, x5, x13+        mov     x5, #-4294967295+        umulh   x5, x5, x13+        mov     x4, #4294967295+        mul     x3, x4, x13+        umulh   x4, x4, x13+        adds    x5, x5, x3+        adcs    x4, x4, x13+        adc     x3, xzr, xzr+        subs    x2, x2, x5+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        adds    x2, x2, x14+        adcs    x9, x9, x15+        adcs    x10, x10, x16+        adcs    x11, x11, x17+        adcs    x12, x12, x19+        adcs    x13, x13, x20+        adc     x6, xzr, xzr+        mov     x8, #-4294967295+        adds    x14, x2, x8+        mov     x8, #4294967295+        adcs    x15, x9, x8+        mov     x8, #1+        adcs    x16, x10, x8+        adcs    x17, x11, xzr+        adcs    x19, x12, xzr+        adcs    x20, x13, xzr+        adcs    x6, x6, xzr+        csel    x2, x2, x14, eq+        csel    x9, x9, x15, eq+        csel    x10, x10, x16, eq+        csel    x11, x11, x17, eq+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        stp     x2, x9, [sp, #192]+        stp     x10, x11, [sp, #208]+        stp     x12, x13, [sp, #224]+        ldp     x5, x6, [sp, #240]+        ldp     x4, x3, [sp, #48]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #256]+        ldp     x4, x3, [sp, #64]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #272]+        ldp     x4, x3, [sp, #80]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        csetm   x3, lo+        mov     x4, #4294967295+        and     x4, x4, x3+        adds    x5, x5, x4+        eor     x4, x4, x3+        adcs    x6, x6, x4+        mov     x4, #-2+        and     x4, x4, x3+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        adcs    x9, x9, x3+        adc     x10, x10, x3+        stp     x5, x6, [x23, #96]+        stp     x7, x8, [x23, #112]+        stp     x9, x10, [x23, #128]+        ldp     x3, x4, [sp, #288]+        ldp     x5, x6, [sp, #96]+        mul     x12, x3, x5+        umulh   x13, x3, x5+        mul     x11, x3, x6+        umulh   x14, x3, x6+        adds    x13, x13, x11+        ldp     x7, x8, [sp, #112]+        mul     x11, x3, x7+        umulh   x15, x3, x7+        adcs    x14, x14, x11+        mul     x11, x3, x8+        umulh   x16, x3, x8+        adcs    x15, x15, x11+        ldp     x9, x10, [sp, #128]+        mul     x11, x3, x9+        umulh   x17, x3, x9+        adcs    x16, x16, x11+        mul     x11, x3, x10+        umulh   x19, x3, x10+        adcs    x17, x17, x11+        adc     x19, x19, xzr+        mul     x11, x4, x5+        adds    x13, x13, x11+        mul     x11, x4, x6+        adcs    x14, x14, x11+        mul     x11, x4, x7+        adcs    x15, x15, x11+        mul     x11, x4, x8+        adcs    x16, x16, x11+        mul     x11, x4, x9+        adcs    x17, x17, x11+        mul     x11, x4, x10+        adcs    x19, x19, x11+        cset    x20, hs+        umulh   x11, x4, x5+        adds    x14, x14, x11+        umulh   x11, x4, x6+        adcs    x15, x15, x11+        umulh   x11, x4, x7+        adcs    x16, x16, x11+        umulh   x11, x4, x8+        adcs    x17, x17, x11+        umulh   x11, x4, x9+        adcs    x19, x19, x11+        umulh   x11, x4, x10+        adc     x20, x20, x11+        ldp     x3, x4, [sp, #304]+        mul     x11, x3, x5+        adds    x14, x14, x11+        mul     x11, x3, x6+        adcs    x15, x15, x11+        mul     x11, x3, x7+        adcs    x16, x16, x11+        mul     x11, x3, x8+        adcs    x17, x17, x11+        mul     x11, x3, x9+        adcs    x19, x19, x11+        mul     x11, x3, x10+        adcs    x20, x20, x11+        cset    x21, hs+        umulh   x11, x3, x5+        adds    x15, x15, x11+        umulh   x11, x3, x6+        adcs    x16, x16, x11+        umulh   x11, x3, x7+        adcs    x17, x17, x11+        umulh   x11, x3, x8+        adcs    x19, x19, x11+        umulh   x11, x3, x9+        adcs    x20, x20, x11+        umulh   x11, x3, x10+        adc     x21, x21, x11+        mul     x11, x4, x5+        adds    x15, x15, x11+        mul     x11, x4, x6+        adcs    x16, x16, x11+        mul     x11, x4, x7+        adcs    x17, x17, x11+        mul     x11, x4, x8+        adcs    x19, x19, x11+        mul     x11, x4, x9+        adcs    x20, x20, x11+        mul     x11, x4, x10+        adcs    x21, x21, x11+        cset    x22, hs+        umulh   x11, x4, x5+        adds    x16, x16, x11+        umulh   x11, x4, x6+        adcs    x17, x17, x11+        umulh   x11, x4, x7+        adcs    x19, x19, x11+        umulh   x11, x4, x8+        adcs    x20, x20, x11+        umulh   x11, x4, x9+        adcs    x21, x21, x11+        umulh   x11, x4, x10+        adc     x22, x22, x11+        ldp     x3, x4, [sp, #320]+        mul     x11, x3, x5+        adds    x16, x16, x11+        mul     x11, x3, x6+        adcs    x17, x17, x11+        mul     x11, x3, x7+        adcs    x19, x19, x11+        mul     x11, x3, x8+        adcs    x20, x20, x11+        mul     x11, x3, x9+        adcs    x21, x21, x11+        mul     x11, x3, x10+        adcs    x22, x22, x11+        cset    x2, hs+        umulh   x11, x3, x5+        adds    x17, x17, x11+        umulh   x11, x3, x6+        adcs    x19, x19, x11+        umulh   x11, x3, x7+        adcs    x20, x20, x11+        umulh   x11, x3, x8+        adcs    x21, x21, x11+        umulh   x11, x3, x9+        adcs    x22, x22, x11+        umulh   x11, x3, x10+        adc     x2, x2, x11+        mul     x11, x4, x5+        adds    x17, x17, x11+        mul     x11, x4, x6+        adcs    x19, x19, x11+        mul     x11, x4, x7+        adcs    x20, x20, x11+        mul     x11, x4, x8+        adcs    x21, x21, x11+        mul     x11, x4, x9+        adcs    x22, x22, x11+        mul     x11, x4, x10+        adcs    x2, x2, x11+        cset    x1, hs+        umulh   x11, x4, x5+        adds    x19, x19, x11+        umulh   x11, x4, x6+        adcs    x20, x20, x11+        umulh   x11, x4, x7+        adcs    x21, x21, x11+        umulh   x11, x4, x8+        adcs    x22, x22, x11+        umulh   x11, x4, x9+        adcs    x2, x2, x11+        umulh   x11, x4, x10+        adc     x1, x1, x11+        lsl     x7, x12, #32+        add     x12, x7, x12+        mov     x7, #-4294967295+        umulh   x7, x7, x12+        mov     x6, #4294967295+        mul     x5, x6, x12+        umulh   x6, x6, x12+        adds    x7, x7, x5+        adcs    x6, x6, x12+        adc     x5, xzr, xzr+        subs    x13, x13, x7+        sbcs    x14, x14, x6+        sbcs    x15, x15, x5+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x12, x12, xzr+        lsl     x7, x13, #32+        add     x13, x7, x13+        mov     x7, #-4294967295+        umulh   x7, x7, x13+        mov     x6, #4294967295+        mul     x5, x6, x13+        umulh   x6, x6, x13+        adds    x7, x7, x5+        adcs    x6, x6, x13+        adc     x5, xzr, xzr+        subs    x14, x14, x7+        sbcs    x15, x15, x6+        sbcs    x16, x16, x5+        sbcs    x17, x17, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        lsl     x7, x14, #32+        add     x14, x7, x14+        mov     x7, #-4294967295+        umulh   x7, x7, x14+        mov     x6, #4294967295+        mul     x5, x6, x14+        umulh   x6, x6, x14+        adds    x7, x7, x5+        adcs    x6, x6, x14+        adc     x5, xzr, xzr+        subs    x15, x15, x7+        sbcs    x16, x16, x6+        sbcs    x17, x17, x5+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        lsl     x7, x15, #32+        add     x15, x7, x15+        mov     x7, #-4294967295+        umulh   x7, x7, x15+        mov     x6, #4294967295+        mul     x5, x6, x15+        umulh   x6, x6, x15+        adds    x7, x7, x5+        adcs    x6, x6, x15+        adc     x5, xzr, xzr+        subs    x16, x16, x7+        sbcs    x17, x17, x6+        sbcs    x12, x12, x5+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbc     x15, x15, xzr+        lsl     x7, x16, #32+        add     x16, x7, x16+        mov     x7, #-4294967295+        umulh   x7, x7, x16+        mov     x6, #4294967295+        mul     x5, x6, x16+        umulh   x6, x6, x16+        adds    x7, x7, x5+        adcs    x6, x6, x16+        adc     x5, xzr, xzr+        subs    x17, x17, x7+        sbcs    x12, x12, x6+        sbcs    x13, x13, x5+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbc     x16, x16, xzr+        lsl     x7, x17, #32+        add     x17, x7, x17+        mov     x7, #-4294967295+        umulh   x7, x7, x17+        mov     x6, #4294967295+        mul     x5, x6, x17+        umulh   x6, x6, x17+        adds    x7, x7, x5+        adcs    x6, x6, x17+        adc     x5, xzr, xzr+        subs    x12, x12, x7+        sbcs    x13, x13, x6+        sbcs    x14, x14, x5+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbc     x17, x17, xzr+        adds    x12, x12, x19+        adcs    x13, x13, x20+        adcs    x14, x14, x21+        adcs    x15, x15, x22+        adcs    x16, x16, x2+        adcs    x17, x17, x1+        adc     x10, xzr, xzr+        mov     x11, #-4294967295+        adds    x19, x12, x11+        mov     x11, #4294967295+        adcs    x20, x13, x11+        mov     x11, #1+        adcs    x21, x14, x11+        adcs    x22, x15, xzr+        adcs    x2, x16, xzr+        adcs    x1, x17, xzr+        adcs    x10, x10, xzr+        csel    x12, x12, x19, eq+        csel    x13, x13, x20, eq+        csel    x14, x14, x21, eq+        csel    x15, x15, x22, eq+        csel    x16, x16, x2, eq+        csel    x17, x17, x1, eq+        stp     x12, x13, [sp, #240]+        stp     x14, x15, [sp, #256]+        stp     x16, x17, [sp, #272]+        ldp     x1, x2, [sp, #144]+        ldp     x3, x4, [sp, #160]+        ldp     x5, x6, [sp, #176]+        lsl     x0, x1, #2+        ldp     x7, x8, [sp, #288]+        subs    x0, x0, x7+        extr    x1, x2, x1, #62+        sbcs    x1, x1, x8+        ldp     x7, x8, [sp, #304]+        extr    x2, x3, x2, #62+        sbcs    x2, x2, x7+        extr    x3, x4, x3, #62+        sbcs    x3, x3, x8+        extr    x4, x5, x4, #62+        ldp     x7, x8, [sp, #320]+        sbcs    x4, x4, x7+        extr    x5, x6, x5, #62+        sbcs    x5, x5, x8+        lsr     x6, x6, #62+        adc     x6, x6, xzr+        lsl     x7, x6, #32+        subs    x8, x6, x7+        sbc     x7, x7, xzr+        adds    x0, x0, x8+        adcs    x1, x1, x7+        adcs    x2, x2, x6+        adcs    x3, x3, xzr+        adcs    x4, x4, xzr+        adcs    x5, x5, xzr+        csetm   x8, lo+        mov     x9, #4294967295+        and     x9, x9, x8+        adds    x0, x0, x9+        eor     x9, x9, x8+        adcs    x1, x1, x9+        mov     x9, #-2+        and     x9, x9, x8+        adcs    x2, x2, x9+        adcs    x3, x3, x8+        adcs    x4, x4, x8+        adc     x5, x5, x8+        stp     x0, x1, [x23]+        stp     x2, x3, [x23, #16]+        stp     x4, x5, [x23, #32]+        ldp     x0, x1, [sp, #192]+        mov     x6, #4294967295+        subs    x6, x6, x0+        mov     x7, #-4294967296+        sbcs    x7, x7, x1+        ldp     x0, x1, [sp, #208]+        mov     x8, #-2+        sbcs    x8, x8, x0+        mov     x13, #-1+        sbcs    x9, x13, x1+        ldp     x0, x1, [sp, #224]+        sbcs    x10, x13, x0+        sbc     x11, x13, x1+        lsl     x0, x6, #3+        extr    x1, x7, x6, #61+        extr    x2, x8, x7, #61+        extr    x3, x9, x8, #61+        extr    x4, x10, x9, #61+        extr    x5, x11, x10, #61+        lsr     x6, x11, #61+        add     x6, x6, #1+        ldp     x8, x9, [sp, #240]+        ldp     x10, x11, [sp, #256]+        ldp     x12, x13, [sp, #272]+        mov     x14, #3+        mul     x15, x14, x8+        umulh   x8, x14, x8+        adds    x0, x0, x15+        mul     x15, x14, x9+        umulh   x9, x14, x9+        adcs    x1, x1, x15+        mul     x15, x14, x10+        umulh   x10, x14, x10+        adcs    x2, x2, x15+        mul     x15, x14, x11+        umulh   x11, x14, x11+        adcs    x3, x3, x15+        mul     x15, x14, x12+        umulh   x12, x14, x12+        adcs    x4, x4, x15+        mul     x15, x14, x13+        umulh   x13, x14, x13+        adcs    x5, x5, x15+        adc     x6, x6, xzr+        adds    x1, x1, x8+        adcs    x2, x2, x9+        adcs    x3, x3, x10+        adcs    x4, x4, x11+        adcs    x5, x5, x12+        adcs    x6, x6, x13+        lsl     x7, x6, #32+        subs    x8, x6, x7+        sbc     x7, x7, xzr+        adds    x0, x0, x8+        adcs    x1, x1, x7+        adcs    x2, x2, x6+        adcs    x3, x3, xzr+        adcs    x4, x4, xzr+        adcs    x5, x5, xzr+        csetm   x6, lo+        mov     x7, #4294967295+        and     x7, x7, x6+        adds    x0, x0, x7+        eor     x7, x7, x6+        adcs    x1, x1, x7+        mov     x7, #-2+        and     x7, x7, x6+        adcs    x2, x2, x7+        adcs    x3, x3, x6+        adcs    x4, x4, x6+        adc     x5, x5, x6+        stp     x0, x1, [x23, #48]+        stp     x2, x3, [x23, #64]+        stp     x4, x5, [x23, #80]+        CFI_INC_SP(336)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp384_montjscalarmul_alt_p384_montjdouble)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/p521_jscalarmul.S view
@@ -0,0 +1,2747 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Jacobian form scalar multiplication for P-521+// Input scalar[9], point[27]; output res[27]+//+// extern void p521_jscalarmul+//   (uint64_t res[static 27],+//    const uint64_t scalar[static 9],+//    const uint64_t point[static 27]);+//+// This function is a variant of its affine point version p521_scalarmul.+// Here, input and output points are assumed to be in Jacobian form with+// a triple (x,y,z) representing the affine point (x/z^2,y/z^3) when+// z is nonzero or the point at infinity (group identity) if z = 0.+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-521, returns a representation of n * P. If the result is the+// point at infinity (either because the input point was or because the+// scalar was a multiple of p_521) then the output is guaranteed to+// represent the point at infinity, i.e. to have its z coordinate zero.+//+// Standard ARM ABI: X0 = res, X1 = scalar, X2 = point+// ----------------------------------------------------------------------------+++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p521_jscalarmul)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p521_jscalarmul)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p521_jscalarmul)+++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 72+#define JACSIZE (3*NUMSIZE)++// Safe copies of input res and additional values in variables.++#define tabup x15+#define bf x16+#define sgn x17+#define j x19+#define res x20++// Intermediate variables on the stack.+// The table is 16 entries, each of size JACSIZE = 3 * NUMSIZE++#define scalarb sp, #(0*NUMSIZE)+#define acc sp, #(1*NUMSIZE)+#define tabent sp, #(4*NUMSIZE)++#define tab sp, #(7*NUMSIZE)++// Round up to maintain stack alignment++#define NSPACE 3968++#define selectblock(I)                            \+        cmp     bf, #(1*I) __LF                      \+        ldp     x10, x11, [tabup] __LF               \+        csel    x0, x10, x0, eq __LF                 \+        csel    x1, x11, x1, eq __LF                 \+        ldp     x10, x11, [tabup, #16] __LF          \+        csel    x2, x10, x2, eq __LF                 \+        csel    x3, x11, x3, eq __LF                 \+        ldp     x10, x11, [tabup, #32] __LF          \+        csel    x4, x10, x4, eq __LF                 \+        csel    x5, x11, x5, eq __LF                 \+        ldp     x10, x11, [tabup, #48] __LF          \+        csel    x6, x10, x6, eq __LF                 \+        csel    x7, x11, x7, eq __LF                 \+        ldr     x10, [tabup, #64] __LF               \+        csel    x8, x10, x8, eq __LF                 \+        add     tabup, tabup, #JACSIZE++// Loading large constants++#define movbig(nn,n3,n2,n1,n0)                                      \+        movz    nn, n0 __LF                                            \+        movk    nn, n1, lsl #16 __LF                                   \+        movk    nn, n2, lsl #32 __LF                                   \+        movk    nn, n3, lsl #48++S2N_BN_SYMBOL(p521_jscalarmul):+        CFI_START++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x30)+        CFI_DEC_SP(NSPACE)++// Preserve the "res" input argument; others get processed early.++        mov     res, x0++// Reduce the input scalar mod n_521 and store it to "scalarb".++        mov     x19, x2+        add     x0, scalarb+        CFI_BL(Lp521_jscalarmul_bignum_mod_n521_9)+        mov     x2, x19++// Set the tab[0] table entry to the input point = 1 * P, but also+// reduce all coordinates modulo p. In principle we assume reduction+// as a precondition, but this reduces the scope for surprise, e.g.+// making sure that any input with z = 0 is treated as zero, even+// if the other coordinates are not in fact reduced.++        add     x0, tab+        mov     x1, x19+        CFI_BL(Lp521_jscalarmul_bignum_mod_p521_9)++        add     x0, tab+NUMSIZE+        add     x1, x19, #NUMSIZE+        CFI_BL(Lp521_jscalarmul_bignum_mod_p521_9)++        add     x0, tab+2*NUMSIZE+        add     x1, x19, #(2*NUMSIZE)+        CFI_BL(Lp521_jscalarmul_bignum_mod_p521_9)++// If bit 520 of the scalar is set, then negate the scalar mod n_521,+// i.e. do scalar |-> n_521 - scalar, and also the point to compensate+// by negating its y coordinate. This further step is not needed by+// the indexing scheme (the top window is only a couple of bits either+// way), but is convenient to exclude a problem with the specific value+// scalar = n_521 - 18, where the last Jacobian addition is of the form+// (n_521 - 9) * P + -(9 * P) and hence is a degenerate doubling case.++        ldp     x0, x1, [scalarb]+        movbig(x10, #0xbb6f, #0xb71e, #0x9138, #0x6409)+        subs    x10, x10, x0+        movbig(x11, #0x3bb5, #0xc9b8, #0x899c, #0x47ae)+        sbcs    x11, x11, x1+        ldp     x2, x3, [scalarb+16]+        movbig(x12, #0x7fcc, #0x0148, #0xf709, #0xa5d0)+        sbcs    x12, x12, x2+        movbig(x13, #0x5186, #0x8783, #0xbf2f, #0x966b)+        sbcs    x13, x13, x3+        ldp     x4, x5, [scalarb+32]+        mov     x14, 0xfffffffffffffffa+        sbcs    x14, x14, x4+        mov     x15, 0xffffffffffffffff+        sbcs    x15, x15, x5+        ldp     x6, x7, [scalarb+48]+        mov     x16, 0xffffffffffffffff+        sbcs    x16, x16, x6+        mov     x17, 0xffffffffffffffff+        sbcs    x17, x17, x7+        ldr     x8, [scalarb+64]+        mov     x19, 0x00000000000001ff+        sbc     x19, x19, x8+        tst     x8, 0x100+        csetm   x9, ne+        csel    x0, x10, x0, ne+        csel    x1, x11, x1, ne+        csel    x2, x12, x2, ne+        csel    x3, x13, x3, ne+        csel    x4, x14, x4, ne+        csel    x5, x15, x5, ne+        csel    x6, x16, x6, ne+        csel    x7, x17, x7, ne+        csel    x8, x19, x8, ne+        stp     x0, x1, [scalarb]+        stp     x2, x3, [scalarb+16]+        stp     x4, x5, [scalarb+32]+        stp     x6, x7, [scalarb+48]+        str     x8, [scalarb+64]++        add     tabup, tab+        ldp     x0, x1, [tabup, #NUMSIZE]+        ldp     x2, x3, [tabup, #NUMSIZE+16]+        ldp     x4, x5, [tabup, #NUMSIZE+32]+        ldp     x6, x7, [tabup, #NUMSIZE+48]+        ldr     x8, [tabup, #NUMSIZE+64]+        orr     x10, x0, x1+        orr     x11, x2, x3+        orr     x12, x4, x5+        orr     x13, x6, x7+        orr     x10, x10, x11+        orr     x12, x12, x13+        orr     x12, x12, x8+        orr     x10, x10, x12+        cmp     x10, xzr+        csel    x9, x9, xzr, ne+        eor     x0, x0, x9+        eor     x1, x1, x9+        eor     x2, x2, x9+        eor     x3, x3, x9+        eor     x4, x4, x9+        eor     x5, x5, x9+        eor     x6, x6, x9+        eor     x7, x7, x9+        and     x9, x9, #0x1FF+        eor     x8, x8, x9+        stp     x0, x1, [tabup, #NUMSIZE]+        stp     x2, x3, [tabup, #NUMSIZE+16]+        stp     x4, x5, [tabup, #NUMSIZE+32]+        stp     x6, x7, [tabup, #NUMSIZE+48]+        str     x8, [tabup, #NUMSIZE+64]++// Compute and record tab[1] = 2 * p, ..., tab[15] = 16 * P++        add     x0, tab+JACSIZE*1+        add     x1, tab+        CFI_BL(Lp521_jscalarmul_jdouble)++        add     x0, tab+JACSIZE*2+        add     x1, tab+JACSIZE*1+        add     x2, tab+        CFI_BL(Lp521_jscalarmul_jadd)++        add     x0, tab+JACSIZE*3+        add     x1, tab+JACSIZE*1+        CFI_BL(Lp521_jscalarmul_jdouble)++        add     x0, tab+JACSIZE*4+        add     x1, tab+JACSIZE*3+        add     x2, tab+        CFI_BL(Lp521_jscalarmul_jadd)++        add     x0, tab+JACSIZE*5+        add     x1, tab+JACSIZE*2+        CFI_BL(Lp521_jscalarmul_jdouble)++        add     x0, tab+JACSIZE*6+        add     x1, tab+JACSIZE*5+        add     x2, tab+        CFI_BL(Lp521_jscalarmul_jadd)++        add     x0, tab+JACSIZE*7+        add     x1, tab+JACSIZE*3+        CFI_BL(Lp521_jscalarmul_jdouble)++        add     x0, tab+JACSIZE*8+        add     x1, tab+JACSIZE*7+        add     x2, tab+        CFI_BL(Lp521_jscalarmul_jadd)++        add     x0, tab+JACSIZE*9+        add     x1, tab+JACSIZE*4+        CFI_BL(Lp521_jscalarmul_jdouble)++        add     x0, tab+JACSIZE*10+        add     x1, tab+JACSIZE*9+        add     x2, tab+        CFI_BL(Lp521_jscalarmul_jadd)++        add     x0, tab+JACSIZE*11+        add     x1, tab+JACSIZE*5+        CFI_BL(Lp521_jscalarmul_jdouble)++        add     x0, tab+JACSIZE*12+        add     x1, tab+JACSIZE*11+        add     x2, tab+        CFI_BL(Lp521_jscalarmul_jadd)++        add     x0, tab+JACSIZE*13+        add     x1, tab+JACSIZE*6+        CFI_BL(Lp521_jscalarmul_jdouble)++        add     x0, tab+JACSIZE*14+        add     x1, tab+JACSIZE*13+        add     x2, tab+        CFI_BL(Lp521_jscalarmul_jadd)++        add     x0, tab+JACSIZE*15+        add     x1, tab+JACSIZE*7+        CFI_BL(Lp521_jscalarmul_jdouble)++// Add the recoding constant sum_i(16 * 32^i) to the scalar to allow signed+// digits. The digits of the constant, in lowest-to-highest order, are as+// follows; they are generated dynamically since none is a simple ARM load.+//+// 0x0842108421084210+// 0x1084210842108421+// 0x2108421084210842+// 0x4210842108421084+// 0x8421084210842108+// 0x0842108421084210+// 0x1084210842108421+// 0x2108421084210842+// 0x0000000000000084++        ldp     x0, x1, [scalarb]+        ldp     x2, x3, [scalarb+16]+        ldp     x4, x5, [scalarb+32]+        ldp     x6, x7, [scalarb+48]+        ldr     x8, [scalarb+64]++        movbig(x10, #0x1084, #0x2108, #0x4210, #0x8421)+        adds    x0, x0, x10, lsr #1+        adcs    x1, x1, x10+        lsl     x10, x10, #1+        adcs    x2, x2, x10+        lsl     x10, x10, #1+        adcs    x3, x3, x10+        lsl     x10, x10, #1+        adcs    x4, x4, x10+        lsr     x11, x10, #4+        adcs    x5, x5, x11+        lsr     x10, x10, #3+        adcs    x6, x6, x10+        lsl     x10, x10, #1+        adcs    x7, x7, x10+        lsl     x10, x10, #1+        and     x10, x10, #0xFF+        adc     x8, x8, x10++// Because of the initial reduction the top bitfield (>= bits 520) is <= 1,+// i.e. just a single bit. Record that in "bf", then shift the whole+// scalar left 56 bits to align the top of the next bitfield with the MSB+// (bits 571..575).++        lsr     bf, x8, #8+        extr    x8, x8, x7, #8+        extr    x7, x7, x6, #8+        extr    x6, x6, x5, #8+        extr    x5, x5, x4, #8+        extr    x4, x4, x3, #8+        extr    x3, x3, x2, #8+        extr    x2, x2, x1, #8+        extr    x1, x1, x0, #8+        lsl     x0, x0, #56+        stp     x0, x1, [scalarb]+        stp     x2, x3, [scalarb+16]+        stp     x4, x5, [scalarb+32]+        stp     x6, x7, [scalarb+48]+        str     x8, [scalarb+64]++// According to the top bit, initialize the accumulator to P or 0. This top+// digit, uniquely, is not recoded so there is no sign adjustment to make.+// We only really need to adjust the z coordinate to zero, but do all three.++        add     tabup, tab+        cmp     bf, xzr++        ldp     x0, x1, [tabup]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc]+        ldp     x0, x1, [tabup, #16]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+16]+        ldp     x0, x1, [tabup, #32]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+32]+        ldp     x0, x1, [tabup, #48]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+48]+        ldp     x0, x1, [tabup, #64]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+64]+        ldp     x0, x1, [tabup, #80]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+80]+        ldp     x0, x1, [tabup, #96]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+96]+        ldp     x0, x1, [tabup, #112]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+112]+        ldp     x0, x1, [tabup, #128]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+128]+        ldp     x0, x1, [tabup, #144]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+144]+        ldp     x0, x1, [tabup, #160]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+160]+        ldp     x0, x1, [tabup, #176]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+176]+        ldp     x0, x1, [tabup, #192]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+192]+        ldr     x0, [tabup, #208]+        csel    x0, x0, xzr, ne+        str     x0, [acc+208]++// Main loop over size-5 bitfields: double 5 times then add signed digit+// At each stage we shift the scalar left by 5 bits so we can simply pick+// the top 5 bits as the bitfield, saving some fiddle over indexing.++        mov     j, #520++Lp521_jscalarmul_mainloop:+        sub     j, j, #5++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp521_jscalarmul_jdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp521_jscalarmul_jdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp521_jscalarmul_jdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp521_jscalarmul_jdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp521_jscalarmul_jdouble)++// Choose the bitfield and adjust it to sign and magnitude++        ldp     x0, x1, [scalarb]+        ldp     x2, x3, [scalarb+16]+        ldp     x4, x5, [scalarb+32]+        ldp     x6, x7, [scalarb+48]+        ldr     x8, [scalarb+64]+        lsr     bf, x8, #59+        extr    x8, x8, x7, #59+        extr    x7, x7, x6, #59+        extr    x6, x6, x5, #59+        extr    x5, x5, x4, #59+        extr    x4, x4, x3, #59+        extr    x3, x3, x2, #59+        extr    x2, x2, x1, #59+        extr    x1, x1, x0, #59+        lsl     x0, x0, #5+        stp     x0, x1, [scalarb]+        stp     x2, x3, [scalarb+16]+        stp     x4, x5, [scalarb+32]+        stp     x6, x7, [scalarb+48]+        str     x8, [scalarb+64]++        subs    bf, bf, #16+        csetm   sgn, lo                 // sgn = sign of digit (1 = negative)+        cneg    bf, bf, lo              // bf = absolute value of digit++// Conditionally select the table entry tab[i-1] = i * P in constant time++        mov     x0, xzr+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, xzr+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        add     tabup, tab+        selectblock(1)+        selectblock(2)+        selectblock(3)+        selectblock(4)+        selectblock(5)+        selectblock(6)+        selectblock(7)+        selectblock(8)+        selectblock(9)+        selectblock(10)+        selectblock(11)+        selectblock(12)+        selectblock(13)+        selectblock(14)+        selectblock(15)+        selectblock(16)+        stp     x0, x1, [tabent]+        stp     x2, x3, [tabent+16]+        stp     x4, x5, [tabent+32]+        stp     x6, x7, [tabent+48]+        str     x8, [tabent+64]++        mov     x0, xzr+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, xzr+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        add     tabup, tab+2*NUMSIZE+        selectblock(1)+        selectblock(2)+        selectblock(3)+        selectblock(4)+        selectblock(5)+        selectblock(6)+        selectblock(7)+        selectblock(8)+        selectblock(9)+        selectblock(10)+        selectblock(11)+        selectblock(12)+        selectblock(13)+        selectblock(14)+        selectblock(15)+        selectblock(16)+        stp     x0, x1, [tabent+2*NUMSIZE]+        stp     x2, x3, [tabent+2*NUMSIZE+16]+        stp     x4, x5, [tabent+2*NUMSIZE+32]+        stp     x6, x7, [tabent+2*NUMSIZE+48]+        str     x8, [tabent+2*NUMSIZE+64]++        mov     x0, xzr+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, xzr+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        add     tabup, tab+NUMSIZE+        selectblock(1)+        selectblock(2)+        selectblock(3)+        selectblock(4)+        selectblock(5)+        selectblock(6)+        selectblock(7)+        selectblock(8)+        selectblock(9)+        selectblock(10)+        selectblock(11)+        selectblock(12)+        selectblock(13)+        selectblock(14)+        selectblock(15)+        selectblock(16)++// Store it to "tabent" with the y coordinate optionally negated.+// This is done carefully to give coordinates < p_521 even in+// the degenerate case y = 0 (when z = 0 for points on the curve).++        orr     x10, x0, x1+        orr     x11, x2, x3+        orr     x12, x4, x5+        orr     x13, x6, x7+        orr     x10, x10, x11+        orr     x12, x12, x13+        orr     x12, x12, x8+        orr     x10, x10, x12+        cmp     x10, xzr+        csel    sgn, sgn, xzr, ne++        eor     x0, x0, sgn+        eor     x1, x1, sgn+        eor     x2, x2, sgn+        eor     x3, x3, sgn+        eor     x4, x4, sgn+        eor     x5, x5, sgn+        eor     x6, x6, sgn+        eor     x7, x7, sgn+        and     sgn, sgn, #0x1FF+        eor     x8, x8, sgn++        stp     x0, x1, [tabent+NUMSIZE]+        stp     x2, x3, [tabent+NUMSIZE+16]+        stp     x4, x5, [tabent+NUMSIZE+32]+        stp     x6, x7, [tabent+NUMSIZE+48]+        str     x8, [tabent+NUMSIZE+64]++// Add to the accumulator++        add     x0, acc+        add     x1, acc+        add     x2, tabent+        CFI_BL(Lp521_jscalarmul_jadd)++        cbnz    j, Lp521_jscalarmul_mainloop++// That's the end of the main loop, and we just need to copy the+// result in "acc" to the output.++        ldp     x0, x1, [acc]+        stp     x0, x1, [res]+        ldp     x0, x1, [acc+16]+        stp     x0, x1, [res, #16]+        ldp     x0, x1, [acc+32]+        stp     x0, x1, [res, #32]+        ldp     x0, x1, [acc+48]+        stp     x0, x1, [res, #48]+        ldp     x0, x1, [acc+64]+        stp     x0, x1, [res, #64]+        ldp     x0, x1, [acc+80]+        stp     x0, x1, [res, #80]+        ldp     x0, x1, [acc+96]+        stp     x0, x1, [res, #96]+        ldp     x0, x1, [acc+112]+        stp     x0, x1, [res, #112]+        ldp     x0, x1, [acc+128]+        stp     x0, x1, [res, #128]+        ldp     x0, x1, [acc+144]+        stp     x0, x1, [res, #144]+        ldp     x0, x1, [acc+160]+        stp     x0, x1, [res, #160]+        ldp     x0, x1, [acc+176]+        stp     x0, x1, [res, #176]+        ldp     x0, x1, [acc+192]+        stp     x0, x1, [res, #192]+        ldr     x0, [acc+208]+        str     x0, [res, #208]++// Restore stack and registers and return++        CFI_INC_SP(NSPACE)+        CFI_POP2(x21,x30)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p521_jscalarmul)++// Local copies of subroutines, complete clones at the moment except+// that we share multiplication and squaring between the point operations.++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_bignum_mod_p521_9)++Lp521_jscalarmul_bignum_mod_p521_9:+        CFI_START+        ldr     x12, [x1, #64]+        lsr     x2, x12, #9+        cmp     xzr, xzr+        ldp     x4, x5, [x1]+        adcs    xzr, x4, x2+        adcs    xzr, x5, xzr+        ldp     x6, x7, [x1, #16]+        and     x3, x6, x7+        adcs    xzr, x3, xzr+        ldp     x8, x9, [x1, #32]+        and     x3, x8, x9+        adcs    xzr, x3, xzr+        ldp     x10, x11, [x1, #48]+        and     x3, x10, x11+        adcs    xzr, x3, xzr+        orr     x3, x12, #0xfffffffffffffe00+        adcs    x3, x3, xzr+        adcs    x4, x4, x2+        adcs    x5, x5, xzr+        adcs    x6, x6, xzr+        adcs    x7, x7, xzr+        adcs    x8, x8, xzr+        adcs    x9, x9, xzr+        adcs    x10, x10, xzr+        adcs    x11, x11, xzr+        adc     x12, x12, xzr+        and     x12, x12, #0x1ff+        stp     x4, x5, [x0]+        stp     x6, x7, [x0, #16]+        stp     x8, x9, [x0, #32]+        stp     x10, x11, [x0, #48]+        str     x12, [x0, #64]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_bignum_mod_p521_9)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_bignum_mod_n521_9)++Lp521_jscalarmul_bignum_mod_n521_9:+        CFI_START+        ldr     x14, [x1, #64]+        lsr     x15, x14, #9+        add     x15, x15, #1+        mov     x2, #39927+        movk    x2, #28359, lsl #16+        movk    x2, #18657, lsl #32+        movk    x2, #17552, lsl #48+        mul     x6, x2, x15+        mov     x3, #47185+        movk    x3, #30307, lsl #16+        movk    x3, #13895, lsl #32+        movk    x3, #50250, lsl #48+        mul     x7, x3, x15+        mov     x4, #23087+        movk    x4, #2294, lsl #16+        movk    x4, #65207, lsl #32+        movk    x4, #32819, lsl #48+        mul     x8, x4, x15+        mov     x5, #27028+        movk    x5, #16592, lsl #16+        movk    x5, #30844, lsl #32+        movk    x5, #44665, lsl #48+        mul     x9, x5, x15+        lsl     x10, x15, #2+        add     x10, x10, x15+        umulh   x13, x2, x15+        adds    x7, x7, x13+        umulh   x13, x3, x15+        adcs    x8, x8, x13+        umulh   x13, x4, x15+        adcs    x9, x9, x13+        umulh   x13, x5, x15+        adc     x10, x10, x13+        ldp     x12, x13, [x1]+        adds    x6, x6, x12+        adcs    x7, x7, x13+        ldp     x12, x13, [x1, #16]+        adcs    x8, x8, x12+        adcs    x9, x9, x13+        ldp     x13, x11, [x1, #32]+        adcs    x10, x10, x13+        adcs    x11, x11, xzr+        ldp     x12, x13, [x1, #48]+        adcs    x12, x12, xzr+        adcs    x13, x13, xzr+        orr     x14, x14, #0xfffffffffffffe00+        adcs    x14, x14, xzr+        csetm   x15, lo+        and     x2, x2, x15+        subs    x6, x6, x2+        and     x3, x3, x15+        sbcs    x7, x7, x3+        and     x4, x4, x15+        sbcs    x8, x8, x4+        and     x5, x5, x15+        sbcs    x9, x9, x5+        mov     x2, #5+        and     x2, x2, x15+        sbcs    x10, x10, x2+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        and     x14, x14, #0x1ff+        stp     x6, x7, [x0]+        stp     x8, x9, [x0, #16]+        stp     x10, x11, [x0, #32]+        stp     x12, x13, [x0, #48]+        str     x14, [x0, #64]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_bignum_mod_n521_9)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_jadd)++Lp521_jscalarmul_jadd:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x26)+        CFI_PUSH2(x27,x28)+        CFI_PUSH2(x29,x30)+        CFI_DEC_SP(576)+        mov     x26, x0+        mov     x27, x1+        mov     x28, x2+        mov     x0, sp+        add     x1, x27, #0x90+        CFI_BL(Lp521_jscalarmul_sqr_p521)+        add     x0, sp, #0x168+        add     x1, x28, #0x90+        CFI_BL(Lp521_jscalarmul_sqr_p521)+        add     x0, sp, #0x1f8+        add     x1, x28, #0x90+        add     x2, x27, #0x48+        CFI_BL(Lp521_jscalarmul_mul_p521)+        add     x0, sp, #0x48+        add     x1, x27, #0x90+        add     x2, x28, #0x48+        CFI_BL(Lp521_jscalarmul_mul_p521)+        add     x0, sp, #0x90+        mov     x1, sp+        add     x2, x28, #0x0+        CFI_BL(Lp521_jscalarmul_mul_p521)+        add     x0, sp, #0x120+        add     x1, sp, #0x168+        add     x2, x27, #0x0+        CFI_BL(Lp521_jscalarmul_mul_p521)+        add     x0, sp, #0x48+        mov     x1, sp+        add     x2, sp, #0x48+        CFI_BL(Lp521_jscalarmul_mul_p521)+        add     x0, sp, #0x1f8+        add     x1, sp, #0x168+        add     x2, sp, #0x1f8+        CFI_BL(Lp521_jscalarmul_mul_p521)+        add     x0, sp, #0x168+        add     x1, sp, #0x90+        add     x2, sp, #0x120+        CFI_BL(Lp521_jscalarmul_sub_p521)+        add     x0, sp, #0x48+        add     x1, sp, #0x48+        add     x2, sp, #0x1f8+        CFI_BL(Lp521_jscalarmul_sub_p521)+        add     x0, sp, #0xd8+        add     x1, sp, #0x168+        CFI_BL(Lp521_jscalarmul_sqr_p521)+        mov     x0, sp+        add     x1, sp, #0x48+        CFI_BL(Lp521_jscalarmul_sqr_p521)+        add     x0, sp, #0x120+        add     x1, sp, #0xd8+        add     x2, sp, #0x120+        CFI_BL(Lp521_jscalarmul_mul_p521)+        add     x0, sp, #0x90+        add     x1, sp, #0xd8+        add     x2, sp, #0x90+        CFI_BL(Lp521_jscalarmul_mul_p521)+        mov     x0, sp+        mov     x1, sp+        add     x2, sp, #0x120+        CFI_BL(Lp521_jscalarmul_sub_p521)+        add     x0, sp, #0xd8+        add     x1, sp, #0x90+        add     x2, sp, #0x120+        CFI_BL(Lp521_jscalarmul_sub_p521)+        add     x0, sp, #0x168+        add     x1, sp, #0x168+        add     x2, x27, #0x90+        CFI_BL(Lp521_jscalarmul_mul_p521)+        mov     x0, sp+        mov     x1, sp+        add     x2, sp, #0x90+        CFI_BL(Lp521_jscalarmul_sub_p521)+        add     x0, sp, #0x120+        add     x1, sp, #0x120+        mov     x2, sp+        CFI_BL(Lp521_jscalarmul_sub_p521)+        add     x0, sp, #0xd8+        add     x1, sp, #0xd8+        add     x2, sp, #0x1f8+        CFI_BL(Lp521_jscalarmul_mul_p521)+        add     x0, sp, #0x168+        add     x1, sp, #0x168+        add     x2, x28, #0x90+        CFI_BL(Lp521_jscalarmul_mul_p521)+        add     x0, sp, #0x120+        add     x1, sp, #0x48+        add     x2, sp, #0x120+        CFI_BL(Lp521_jscalarmul_mul_p521)+        add     x0, sp, #0x120+        add     x1, sp, #0x120+        add     x2, sp, #0xd8+        CFI_BL(Lp521_jscalarmul_sub_p521)+        ldp     x0, x1, [x27, #144]+        ldp     x2, x3, [x27, #160]+        ldp     x4, x5, [x27, #176]+        ldp     x6, x7, [x27, #192]+        ldr     x8, [x27, #208]+        orr     x20, x0, x1+        orr     x21, x2, x3+        orr     x22, x4, x5+        orr     x23, x6, x7+        orr     x20, x20, x21+        orr     x22, x22, x23+        orr     x20, x20, x8+        orr     x20, x20, x22+        cmp     x20, xzr+        cset    x20, ne+        ldp     x10, x11, [x28, #144]+        ldp     x12, x13, [x28, #160]+        ldp     x14, x15, [x28, #176]+        ldp     x16, x17, [x28, #192]+        ldr     x19, [x28, #208]+        orr     x21, x10, x11+        orr     x22, x12, x13+        orr     x23, x14, x15+        orr     x24, x16, x17+        orr     x21, x21, x22+        orr     x23, x23, x24+        orr     x21, x21, x19+        orr     x21, x21, x23+        csel    x0, x0, x10, ne+        csel    x1, x1, x11, ne+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        csel    x4, x4, x14, ne+        csel    x5, x5, x15, ne+        csel    x6, x6, x16, ne+        csel    x7, x7, x17, ne+        csel    x8, x8, x19, ne+        cmp     x21, xzr+        cset    x21, ne+        cmp     x21, x20+        ldp     x10, x11, [sp, #360]+        ldp     x12, x13, [sp, #376]+        ldp     x14, x15, [sp, #392]+        ldp     x16, x17, [sp, #408]+        ldr     x19, [sp, #424]+        csel    x0, x0, x10, ne+        csel    x1, x1, x11, ne+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        csel    x4, x4, x14, ne+        csel    x5, x5, x15, ne+        csel    x6, x6, x16, ne+        csel    x7, x7, x17, ne+        csel    x8, x8, x19, ne+        stp     x0, x1, [sp, #360]+        stp     x2, x3, [sp, #376]+        stp     x4, x5, [sp, #392]+        stp     x6, x7, [sp, #408]+        str     x8, [sp, #424]+        ldp     x20, x21, [x27]+        ldp     x0, x1, [sp]+        csel    x0, x20, x0, cc+        csel    x1, x21, x1, cc+        ldp     x20, x21, [x28]+        csel    x0, x20, x0, hi+        csel    x1, x21, x1, hi+        ldp     x20, x21, [x27, #16]+        ldp     x2, x3, [sp, #16]+        csel    x2, x20, x2, cc+        csel    x3, x21, x3, cc+        ldp     x20, x21, [x28, #16]+        csel    x2, x20, x2, hi+        csel    x3, x21, x3, hi+        ldp     x20, x21, [x27, #32]+        ldp     x4, x5, [sp, #32]+        csel    x4, x20, x4, cc+        csel    x5, x21, x5, cc+        ldp     x20, x21, [x28, #32]+        csel    x4, x20, x4, hi+        csel    x5, x21, x5, hi+        ldp     x20, x21, [x27, #48]+        ldp     x6, x7, [sp, #48]+        csel    x6, x20, x6, cc+        csel    x7, x21, x7, cc+        ldp     x20, x21, [x28, #48]+        csel    x6, x20, x6, hi+        csel    x7, x21, x7, hi+        ldr     x20, [x27, #64]+        ldr     x8, [sp, #64]+        csel    x8, x20, x8, cc+        ldr     x21, [x28, #64]+        csel    x8, x21, x8, hi+        ldp     x20, x21, [x27, #72]+        ldp     x10, x11, [sp, #288]+        csel    x10, x20, x10, cc+        csel    x11, x21, x11, cc+        ldp     x20, x21, [x28, #72]+        csel    x10, x20, x10, hi+        csel    x11, x21, x11, hi+        ldp     x20, x21, [x27, #88]+        ldp     x12, x13, [sp, #304]+        csel    x12, x20, x12, cc+        csel    x13, x21, x13, cc+        ldp     x20, x21, [x28, #88]+        csel    x12, x20, x12, hi+        csel    x13, x21, x13, hi+        ldp     x20, x21, [x27, #104]+        ldp     x14, x15, [sp, #320]+        csel    x14, x20, x14, cc+        csel    x15, x21, x15, cc+        ldp     x20, x21, [x28, #104]+        csel    x14, x20, x14, hi+        csel    x15, x21, x15, hi+        ldp     x20, x21, [x27, #120]+        ldp     x16, x17, [sp, #336]+        csel    x16, x20, x16, cc+        csel    x17, x21, x17, cc+        ldp     x20, x21, [x28, #120]+        csel    x16, x20, x16, hi+        csel    x17, x21, x17, hi+        ldr     x20, [x27, #136]+        ldr     x19, [sp, #352]+        csel    x19, x20, x19, cc+        ldr     x21, [x28, #136]+        csel    x19, x21, x19, hi+        stp     x0, x1, [x26]+        stp     x2, x3, [x26, #16]+        stp     x4, x5, [x26, #32]+        stp     x6, x7, [x26, #48]+        str     x8, [x26, #64]+        ldp     x0, x1, [sp, #360]+        ldp     x2, x3, [sp, #376]+        ldp     x4, x5, [sp, #392]+        ldp     x6, x7, [sp, #408]+        ldr     x8, [sp, #424]+        stp     x10, x11, [x26, #72]+        stp     x12, x13, [x26, #88]+        stp     x14, x15, [x26, #104]+        stp     x16, x17, [x26, #120]+        str     x19, [x26, #136]+        stp     x0, x1, [x26, #144]+        stp     x2, x3, [x26, #160]+        stp     x4, x5, [x26, #176]+        stp     x6, x7, [x26, #192]+        str     x8, [x26, #208]+        CFI_INC_SP(576)+        CFI_POP2(x29,x30)+        CFI_POP2(x27,x28)+        CFI_POP2(x25,x26)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_jadd)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_jdouble)++Lp521_jscalarmul_jdouble:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x26)+        CFI_PUSH2(x27,x28)+        CFI_PUSH2(x29,x30)+        CFI_DEC_SP(512)+        mov     x26, x0+        mov     x27, x1+        mov     x0, sp+        add     x1, x27, #0x90+        CFI_BL(Lp521_jscalarmul_sqr_p521)+        add     x0, sp, #0x48+        add     x1, x27, #0x48+        CFI_BL(Lp521_jscalarmul_sqr_p521)+        ldp     x5, x6, [x27]+        ldp     x4, x3, [sp]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [x27, #16]+        ldp     x4, x3, [sp, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [x27, #32]+        ldp     x4, x3, [sp, #32]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        ldp     x11, x12, [x27, #48]+        ldp     x4, x3, [sp, #48]+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        ldr     x13, [x27, #64]+        ldr     x4, [sp, #64]+        sbcs    x13, x13, x4+        sbcs    x5, x5, xzr+        sbcs    x6, x6, xzr+        sbcs    x7, x7, xzr+        sbcs    x8, x8, xzr+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        and     x13, x13, #0x1ff+        stp     x5, x6, [sp, #216]+        stp     x7, x8, [sp, #232]+        stp     x9, x10, [sp, #248]+        stp     x11, x12, [sp, #264]+        str     x13, [sp, #280]+        cmp     xzr, xzr+        ldp     x5, x6, [x27]+        ldp     x4, x3, [sp]+        adcs    x5, x5, x4+        adcs    x6, x6, x3+        ldp     x7, x8, [x27, #16]+        ldp     x4, x3, [sp, #16]+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        ldp     x9, x10, [x27, #32]+        ldp     x4, x3, [sp, #32]+        adcs    x9, x9, x4+        adcs    x10, x10, x3+        ldp     x11, x12, [x27, #48]+        ldp     x4, x3, [sp, #48]+        adcs    x11, x11, x4+        adcs    x12, x12, x3+        ldr     x13, [x27, #64]+        ldr     x4, [sp, #64]+        adc     x13, x13, x4+        subs    x4, x13, #0x200+        csetm   x4, cs+        sbcs    x5, x5, xzr+        and     x4, x4, #0x200+        sbcs    x6, x6, xzr+        sbcs    x7, x7, xzr+        sbcs    x8, x8, xzr+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, x4+        stp     x5, x6, [sp, #144]+        stp     x7, x8, [sp, #160]+        stp     x9, x10, [sp, #176]+        stp     x11, x12, [sp, #192]+        str     x13, [sp, #208]+        add     x0, sp, #0xd8+        add     x1, sp, #0x90+        add     x2, sp, #0xd8+        CFI_BL(Lp521_jscalarmul_mul_p521)+        cmp     xzr, xzr+        ldp     x5, x6, [x27, #72]+        ldp     x4, x3, [x27, #144]+        adcs    x5, x5, x4+        adcs    x6, x6, x3+        ldp     x7, x8, [x27, #88]+        ldp     x4, x3, [x27, #160]+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        ldp     x9, x10, [x27, #104]+        ldp     x4, x3, [x27, #176]+        adcs    x9, x9, x4+        adcs    x10, x10, x3+        ldp     x11, x12, [x27, #120]+        ldp     x4, x3, [x27, #192]+        adcs    x11, x11, x4+        adcs    x12, x12, x3+        ldr     x13, [x27, #136]+        ldr     x4, [x27, #208]+        adc     x13, x13, x4+        subs    x4, x13, #0x200+        csetm   x4, cs+        sbcs    x5, x5, xzr+        and     x4, x4, #0x200+        sbcs    x6, x6, xzr+        sbcs    x7, x7, xzr+        sbcs    x8, x8, xzr+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, x4+        stp     x5, x6, [sp, #144]+        stp     x7, x8, [sp, #160]+        stp     x9, x10, [sp, #176]+        stp     x11, x12, [sp, #192]+        str     x13, [sp, #208]+        add     x0, sp, #0x120+        add     x1, x27, #0x0+        add     x2, sp, #0x48+        CFI_BL(Lp521_jscalarmul_mul_p521)+        add     x0, sp, #0x168+        add     x1, sp, #0xd8+        CFI_BL(Lp521_jscalarmul_sqr_p521)+        add     x0, sp, #0x90+        add     x1, sp, #0x90+        CFI_BL(Lp521_jscalarmul_sqr_p521)+        ldp     x6, x7, [sp, #288]+        mov     x1, #0xc+        mul     x3, x1, x6+        mul     x4, x1, x7+        umulh   x6, x1, x6+        adds    x4, x4, x6+        umulh   x7, x1, x7+        ldp     x8, x9, [sp, #304]+        mul     x5, x1, x8+        mul     x6, x1, x9+        umulh   x8, x1, x8+        adcs    x5, x5, x7+        umulh   x9, x1, x9+        adcs    x6, x6, x8+        ldp     x10, x11, [sp, #320]+        mul     x7, x1, x10+        mul     x8, x1, x11+        umulh   x10, x1, x10+        adcs    x7, x7, x9+        umulh   x11, x1, x11+        adcs    x8, x8, x10+        ldp     x12, x13, [sp, #336]+        mul     x9, x1, x12+        mul     x10, x1, x13+        umulh   x12, x1, x12+        adcs    x9, x9, x11+        umulh   x13, x1, x13+        adcs    x10, x10, x12+        ldr     x14, [sp, #352]+        mul     x11, x1, x14+        adc     x11, x11, x13+        mov     x1, #0x9+        ldp     x20, x21, [sp, #360]+        mvn     x20, x20+        mul     x0, x1, x20+        umulh   x20, x1, x20+        adds    x3, x3, x0+        mvn     x21, x21+        mul     x0, x1, x21+        umulh   x21, x1, x21+        adcs    x4, x4, x0+        ldp     x22, x23, [sp, #376]+        mvn     x22, x22+        mul     x0, x1, x22+        umulh   x22, x1, x22+        adcs    x5, x5, x0+        mvn     x23, x23+        mul     x0, x1, x23+        umulh   x23, x1, x23+        adcs    x6, x6, x0+        ldp     x17, x19, [sp, #392]+        mvn     x17, x17+        mul     x0, x1, x17+        umulh   x17, x1, x17+        adcs    x7, x7, x0+        mvn     x19, x19+        mul     x0, x1, x19+        umulh   x19, x1, x19+        adcs    x8, x8, x0+        ldp     x2, x16, [sp, #408]+        mvn     x2, x2+        mul     x0, x1, x2+        umulh   x2, x1, x2+        adcs    x9, x9, x0+        mvn     x16, x16+        mul     x0, x1, x16+        umulh   x16, x1, x16+        adcs    x10, x10, x0+        ldr     x0, [sp, #424]+        eor     x0, x0, #0x1ff+        mul     x0, x1, x0+        adc     x11, x11, x0+        adds    x4, x4, x20+        adcs    x5, x5, x21+        and     x15, x4, x5+        adcs    x6, x6, x22+        and     x15, x15, x6+        adcs    x7, x7, x23+        and     x15, x15, x7+        adcs    x8, x8, x17+        and     x15, x15, x8+        adcs    x9, x9, x19+        and     x15, x15, x9+        adcs    x10, x10, x2+        and     x15, x15, x10+        adc     x11, x11, x16+        lsr     x12, x11, #9+        orr     x11, x11, #0xfffffffffffffe00+        cmp     xzr, xzr+        adcs    xzr, x3, x12+        adcs    xzr, x15, xzr+        adcs    xzr, x11, xzr+        adcs    x3, x3, x12+        adcs    x4, x4, xzr+        adcs    x5, x5, xzr+        adcs    x6, x6, xzr+        adcs    x7, x7, xzr+        adcs    x8, x8, xzr+        adcs    x9, x9, xzr+        adcs    x10, x10, xzr+        adc     x11, x11, xzr+        and     x11, x11, #0x1ff+        stp     x3, x4, [sp, #360]+        stp     x5, x6, [sp, #376]+        stp     x7, x8, [sp, #392]+        stp     x9, x10, [sp, #408]+        str     x11, [sp, #424]+        ldp     x5, x6, [sp, #144]+        ldp     x4, x3, [sp]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #160]+        ldp     x4, x3, [sp, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #176]+        ldp     x4, x3, [sp, #32]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        ldp     x11, x12, [sp, #192]+        ldp     x4, x3, [sp, #48]+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        ldr     x13, [sp, #208]+        ldr     x4, [sp, #64]+        sbcs    x13, x13, x4+        sbcs    x5, x5, xzr+        sbcs    x6, x6, xzr+        sbcs    x7, x7, xzr+        sbcs    x8, x8, xzr+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        and     x13, x13, #0x1ff+        stp     x5, x6, [sp, #144]+        stp     x7, x8, [sp, #160]+        stp     x9, x10, [sp, #176]+        stp     x11, x12, [sp, #192]+        str     x13, [sp, #208]+        mov     x0, sp+        add     x1, sp, #0x48+        CFI_BL(Lp521_jscalarmul_sqr_p521)+        add     x0, sp, #0xd8+        add     x1, sp, #0x168+        add     x2, sp, #0xd8+        CFI_BL(Lp521_jscalarmul_mul_p521)+        ldp     x5, x6, [sp, #144]+        ldp     x4, x3, [sp, #72]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #160]+        ldp     x4, x3, [sp, #88]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #176]+        ldp     x4, x3, [sp, #104]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        ldp     x11, x12, [sp, #192]+        ldp     x4, x3, [sp, #120]+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        ldr     x13, [sp, #208]+        ldr     x4, [sp, #136]+        sbcs    x13, x13, x4+        sbcs    x5, x5, xzr+        sbcs    x6, x6, xzr+        sbcs    x7, x7, xzr+        sbcs    x8, x8, xzr+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        and     x13, x13, #0x1ff+        stp     x5, x6, [x26, #144]+        stp     x7, x8, [x26, #160]+        stp     x9, x10, [x26, #176]+        stp     x11, x12, [x26, #192]+        str     x13, [x26, #208]+        ldp     x6, x7, [sp, #288]+        lsl     x3, x6, #2+        extr    x4, x7, x6, #62+        ldp     x8, x9, [sp, #304]+        extr    x5, x8, x7, #62+        extr    x6, x9, x8, #62+        ldp     x10, x11, [sp, #320]+        extr    x7, x10, x9, #62+        extr    x8, x11, x10, #62+        ldp     x12, x13, [sp, #336]+        extr    x9, x12, x11, #62+        extr    x10, x13, x12, #62+        ldr     x14, [sp, #352]+        extr    x11, x14, x13, #62+        ldp     x0, x1, [sp, #360]+        mvn     x0, x0+        adds    x3, x3, x0+        sbcs    x4, x4, x1+        ldp     x0, x1, [sp, #376]+        sbcs    x5, x5, x0+        and     x15, x4, x5+        sbcs    x6, x6, x1+        and     x15, x15, x6+        ldp     x0, x1, [sp, #392]+        sbcs    x7, x7, x0+        and     x15, x15, x7+        sbcs    x8, x8, x1+        and     x15, x15, x8+        ldp     x0, x1, [sp, #408]+        sbcs    x9, x9, x0+        and     x15, x15, x9+        sbcs    x10, x10, x1+        and     x15, x15, x10+        ldr     x0, [sp, #424]+        eor     x0, x0, #0x1ff+        adc     x11, x11, x0+        lsr     x12, x11, #9+        orr     x11, x11, #0xfffffffffffffe00+        cmp     xzr, xzr+        adcs    xzr, x3, x12+        adcs    xzr, x15, xzr+        adcs    xzr, x11, xzr+        adcs    x3, x3, x12+        adcs    x4, x4, xzr+        adcs    x5, x5, xzr+        adcs    x6, x6, xzr+        adcs    x7, x7, xzr+        adcs    x8, x8, xzr+        adcs    x9, x9, xzr+        adcs    x10, x10, xzr+        adc     x11, x11, xzr+        and     x11, x11, #0x1ff+        stp     x3, x4, [x26]+        stp     x5, x6, [x26, #16]+        stp     x7, x8, [x26, #32]+        stp     x9, x10, [x26, #48]+        str     x11, [x26, #64]+        ldp     x6, x7, [sp, #216]+        lsl     x3, x6, #1+        adds    x3, x3, x6+        extr    x4, x7, x6, #63+        adcs    x4, x4, x7+        ldp     x8, x9, [sp, #232]+        extr    x5, x8, x7, #63+        adcs    x5, x5, x8+        extr    x6, x9, x8, #63+        adcs    x6, x6, x9+        ldp     x10, x11, [sp, #248]+        extr    x7, x10, x9, #63+        adcs    x7, x7, x10+        extr    x8, x11, x10, #63+        adcs    x8, x8, x11+        ldp     x12, x13, [sp, #264]+        extr    x9, x12, x11, #63+        adcs    x9, x9, x12+        extr    x10, x13, x12, #63+        adcs    x10, x10, x13+        ldr     x14, [sp, #280]+        extr    x11, x14, x13, #63+        adc     x11, x11, x14+        ldp     x20, x21, [sp]+        mvn     x20, x20+        lsl     x0, x20, #3+        adds    x3, x3, x0+        mvn     x21, x21+        extr    x0, x21, x20, #61+        adcs    x4, x4, x0+        ldp     x22, x23, [sp, #16]+        mvn     x22, x22+        extr    x0, x22, x21, #61+        adcs    x5, x5, x0+        and     x15, x4, x5+        mvn     x23, x23+        extr    x0, x23, x22, #61+        adcs    x6, x6, x0+        and     x15, x15, x6+        ldp     x20, x21, [sp, #32]+        mvn     x20, x20+        extr    x0, x20, x23, #61+        adcs    x7, x7, x0+        and     x15, x15, x7+        mvn     x21, x21+        extr    x0, x21, x20, #61+        adcs    x8, x8, x0+        and     x15, x15, x8+        ldp     x22, x23, [sp, #48]+        mvn     x22, x22+        extr    x0, x22, x21, #61+        adcs    x9, x9, x0+        and     x15, x15, x9+        mvn     x23, x23+        extr    x0, x23, x22, #61+        adcs    x10, x10, x0+        and     x15, x15, x10+        ldr     x0, [sp, #64]+        eor     x0, x0, #0x1ff+        extr    x0, x0, x23, #61+        adc     x11, x11, x0+        lsr     x12, x11, #9+        orr     x11, x11, #0xfffffffffffffe00+        cmp     xzr, xzr+        adcs    xzr, x3, x12+        adcs    xzr, x15, xzr+        adcs    xzr, x11, xzr+        adcs    x3, x3, x12+        adcs    x4, x4, xzr+        adcs    x5, x5, xzr+        adcs    x6, x6, xzr+        adcs    x7, x7, xzr+        adcs    x8, x8, xzr+        adcs    x9, x9, xzr+        adcs    x10, x10, xzr+        adc     x11, x11, xzr+        and     x11, x11, #0x1ff+        stp     x3, x4, [x26, #72]+        stp     x5, x6, [x26, #88]+        stp     x7, x8, [x26, #104]+        stp     x9, x10, [x26, #120]+        str     x11, [x26, #136]+        CFI_INC_SP(512)+        CFI_POP2(x29,x30)+        CFI_POP2(x27,x28)+        CFI_POP2(x25,x26)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_jdouble)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_mul_p521)++Lp521_jscalarmul_mul_p521:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x26)+        CFI_DEC_SP(80)+        ldr q6, [x2]+        ldp x10, x17, [x1, #16]+        ldr q4, [x1]+        ldr q16, [x2, #32]+        ldp x5, x20, [x2, #16]+        ldr q2, [x1, #32]+        movi v31.2D, #0x00000000ffffffff+        uzp2 v17.4S, v6.4S, v6.4S+        rev64 v7.4S, v6.4S+        ldp x15, x21, [x1]+        xtn v25.2S, v6.2D+        xtn v22.2S, v4.2D+        subs x14, x10, x17+        mul v7.4S, v7.4S, v4.4S+        csetm x8, cc+        rev64 v3.4S, v16.4S+        xtn v1.2S, v16.2D+        ldp x13, x16, [x2]+        mul x26, x10, x5+        uzp2 v16.4S, v16.4S, v16.4S+        uaddlp v26.2D, v7.4S+        cneg x4, x14, cc+        subs x24, x15, x21+        xtn v5.2S, v2.2D+        mul v28.4S, v3.4S, v2.4S+        shl v26.2D, v26.2D, #32+        mul x22, x17, x20+        umull v20.2D, v22.2S, v25.2S+        uzp2 v6.4S, v4.4S, v4.4S+        umull v18.2D, v22.2S, v17.2S+        uzp2 v4.4S, v2.4S, v2.4S+        cneg x14, x24, cc+        csetm x7, cc+        umulh x11, x17, x20+        usra v18.2D, v20.2D, #32+        uaddlp v7.2D, v28.4S+        subs x19, x16, x13+        umlal v26.2D, v22.2S, v25.2S+        cneg x19, x19, cc+        shl v28.2D, v7.2D, #32+        umull v7.2D, v5.2S, v1.2S+        umull v30.2D, v5.2S, v16.2S+        cinv x6, x7, cc+        mul x25, x14, x19+        umlal v28.2D, v5.2S, v1.2S+        umull v21.2D, v6.2S, v17.2S+        umulh x14, x14, x19+        usra v30.2D, v7.2D, #32+        subs x9, x20, x5+        and v29.16B, v18.16B, v31.16B+        cinv x23, x8, cc+        mov x8, v26.d[1]+        cneg x12, x9, cc+        usra v21.2D, v18.2D, #32+        umlal v29.2D, v6.2S, v25.2S+        mul x24, x4, x12+        umull v18.2D, v4.2S, v16.2S+        movi v25.2D, #0x00000000ffffffff+        eor x9, x14, x6+        and v7.16B, v30.16B, v25.16B+        usra v21.2D, v29.2D, #32+        umulh x7, x10, x5+        usra v18.2D, v30.2D, #32+        umlal v7.2D, v4.2S, v1.2S+        mov x19, v21.d[0]+        umulh x3, x4, x12+        mov x14, v21.d[1]+        usra v18.2D, v7.2D, #32+        adds x4, x8, x19+        mov x8, v26.d[0]+        adcs x19, x26, x14+        adcs x14, x22, x7+        adc x12, x11, xzr+        adds x11, x4, x8+        adcs x26, x19, x4+        adcs x22, x14, x19+        eor x4, x24, x23+        adcs x14, x12, x14+        eor x7, x25, x6+        adc x25, xzr, x12+        eor x19, x3, x23+        adds x3, x26, x8+        adcs x24, x22, x11+        adcs x12, x14, x26+        adcs x22, x25, x22+        adcs x26, xzr, x14+        adc x14, xzr, x25+        cmn x23, #0x1+        adcs x22, x22, x4+        adcs x19, x26, x19+        adc x25, x14, x23+        subs x14, x21, x17+        cneg x23, x14, cc+        csetm x26, cc+        subs x4, x20, x16+        cneg x14, x4, cc+        cinv x4, x26, cc+        cmn x6, #0x1+        adcs x11, x11, x7+        mul x7, x23, x14+        adcs x9, x3, x9+        adcs x26, x24, x6+        umulh x3, x23, x14+        adcs x14, x12, x6+        adcs x22, x22, x6+        adcs x12, x19, x6+        extr x24, x11, x8, #55+        adc x6, x25, x6+        subs x19, x15, x17+        csetm x17, cc+        cneg x23, x19, cc+        subs x19, x20, x13+        lsl x25, x8, #9+        eor x8, x7, x4+        cneg x20, x19, cc+        umulh x7, x23, x20+        cinv x19, x17, cc+        subs x17, x15, x10+        csetm x15, cc+        stp x25, x24, [sp, #32]+        cneg x24, x17, cc+        mul x20, x23, x20+        subs x25, x5, x13+        cneg x13, x25, cc+        cinv x15, x15, cc+        mul x25, x24, x13+        subs x21, x21, x10+        csetm x23, cc+        cneg x17, x21, cc+        subs x21, x5, x16+        umulh x13, x24, x13+        cinv x10, x23, cc+        cneg x23, x21, cc+        cmn x4, #0x1+        adcs x14, x14, x8+        eor x21, x3, x4+        adcs x21, x22, x21+        eor x5, x20, x19+        adcs x24, x12, x4+        mul x12, x17, x23+        eor x8, x25, x15+        adc x25, x6, x4+        cmn x15, #0x1+        adcs x6, x9, x8+        ldp x20, x8, [x2, #48]+        eor x9, x13, x15+        adcs x4, x26, x9+        umulh x26, x17, x23+        ldp x17, x13, [x1, #48]+        adcs x9, x14, x15+        adcs x16, x21, x15+        adcs x14, x24, x15+        eor x21, x7, x19+        mul x23, x17, x20+        adc x24, x25, x15+        cmn x19, #0x1+        adcs x7, x4, x5+        adcs x9, x9, x21+        umulh x3, x13, x8+        adcs x16, x16, x19+        adcs x22, x14, x19+        eor x5, x12, x10+        adc x12, x24, x19+        cmn x10, #0x1+        adcs x19, x7, x5+        eor x14, x26, x10+        mov x7, v28.d[1]+        adcs x24, x9, x14+        extr x4, x19, x6, #55+        umulh x15, x17, x20+        mov x14, v18.d[1]+        lsr x9, x19, #55+        adcs x5, x16, x10+        mov x16, v18.d[0]+        adcs x19, x22, x10+        str x9, [sp, #64]+        extr x25, x6, x11, #55+        adc x21, x12, x10+        subs x26, x17, x13+        stp x25, x4, [sp, #48]+        stp x19, x21, [sp, #16]+        csetm x6, cc+        cneg x4, x26, cc+        mul x19, x13, x8+        subs x11, x8, x20+        stp x24, x5, [sp]+        ldp x21, x10, [x1, #32]+        cinv x12, x6, cc+        cneg x6, x11, cc+        mov x9, v28.d[0]+        umulh x25, x4, x6+        adds x22, x7, x16+        ldp x16, x5, [x2, #32]+        adcs x14, x23, x14+        adcs x11, x19, x15+        adc x24, x3, xzr+        adds x3, x22, x9+        adcs x15, x14, x22+        mul x22, x4, x6+        adcs x6, x11, x14+        adcs x4, x24, x11+        eor x14, x25, x12+        adc x26, xzr, x24+        subs x7, x21, x10+        csetm x23, cc+        cneg x19, x7, cc+        subs x24, x5, x16+        cneg x11, x24, cc+        cinv x7, x23, cc+        adds x25, x15, x9+        eor x23, x22, x12+        adcs x22, x6, x3+        mul x24, x19, x11+        adcs x15, x4, x15+        adcs x6, x26, x6+        umulh x19, x19, x11+        adcs x11, xzr, x4+        adc x26, xzr, x26+        cmn x12, #0x1+        adcs x4, x6, x23+        eor x6, x24, x7+        adcs x14, x11, x14+        adc x26, x26, x12+        subs x11, x10, x13+        cneg x12, x11, cc+        csetm x11, cc+        eor x19, x19, x7+        subs x24, x8, x5+        cinv x11, x11, cc+        cneg x24, x24, cc+        cmn x7, #0x1+        adcs x3, x3, x6+        mul x23, x12, x24+        adcs x25, x25, x19+        adcs x6, x22, x7+        umulh x19, x12, x24+        adcs x22, x15, x7+        adcs x12, x4, x7+        eor x24, x23, x11+        adcs x4, x14, x7+        adc x26, x26, x7+        eor x19, x19, x11+        subs x14, x21, x17+        cneg x7, x14, cc+        csetm x14, cc+        subs x23, x20, x16+        cinv x14, x14, cc+        cneg x23, x23, cc+        cmn x11, #0x1+        adcs x22, x22, x24+        mul x24, x7, x23+        adcs x15, x12, x19+        adcs x4, x4, x11+        adc x19, x26, x11+        umulh x26, x7, x23+        subs x7, x21, x13+        eor x11, x24, x14+        cneg x23, x7, cc+        csetm x12, cc+        subs x7, x8, x16+        cneg x7, x7, cc+        cinv x12, x12, cc+        cmn x14, #0x1+        eor x26, x26, x14+        adcs x11, x25, x11+        mul x25, x23, x7+        adcs x26, x6, x26+        adcs x6, x22, x14+        adcs x24, x15, x14+        umulh x23, x23, x7+        adcs x4, x4, x14+        adc x22, x19, x14+        eor x14, x25, x12+        eor x7, x23, x12+        cmn x12, #0x1+        adcs x14, x26, x14+        ldp x19, x25, [x2]+        ldp x15, x23, [x2, #16]+        adcs x26, x6, x7+        adcs x24, x24, x12+        adcs x7, x4, x12+        adc x4, x22, x12+        subs x19, x19, x16+        ldp x16, x22, [x1]+        sbcs x6, x25, x5+        ldp x12, x25, [x1, #16]+        sbcs x15, x15, x20+        sbcs x8, x23, x8+        csetm x23, cc+        subs x21, x21, x16+        eor x16, x19, x23+        sbcs x19, x10, x22+        eor x22, x6, x23+        eor x8, x8, x23+        sbcs x6, x17, x12+        sbcs x13, x13, x25+        csetm x12, cc+        subs x10, x10, x17+        cneg x17, x10, cc+        csetm x25, cc+        subs x5, x20, x5+        eor x10, x19, x12+        cneg x19, x5, cc+        eor x20, x15, x23+        eor x21, x21, x12+        cinv x15, x25, cc+        mul x25, x17, x19+        subs x16, x16, x23+        sbcs x5, x22, x23+        eor x6, x6, x12+        sbcs x20, x20, x23+        eor x22, x13, x12+        sbc x8, x8, x23+        subs x21, x21, x12+        umulh x19, x17, x19+        sbcs x10, x10, x12+        sbcs x17, x6, x12+        eor x6, x19, x15+        eor x19, x25, x15+        umulh x25, x17, x20+        sbc x13, x22, x12+        cmn x15, #0x1+        adcs x22, x14, x19+        adcs x19, x26, x6+        ldp x6, x26, [sp]+        adcs x14, x24, x15+        umulh x24, x21, x16+        adcs x7, x7, x15+        adc x15, x4, x15+        adds x4, x9, x6+        eor x9, x23, x12+        adcs x12, x3, x26+        stp x4, x12, [sp]+        ldp x4, x26, [sp, #16]+        umulh x12, x10, x5+        ldp x6, x23, [sp, #32]+        adcs x3, x11, x4+        mul x4, x13, x8+        adcs x26, x22, x26+        ldp x22, x11, [sp, #48]+        adcs x6, x19, x6+        stp x3, x26, [sp, #16]+        mul x26, x10, x5+        adcs x14, x14, x23+        stp x6, x14, [sp, #32]+        ldr x6, [sp, #64]+        adcs x22, x7, x22+        adcs x14, x15, x11+        mul x11, x17, x20+        adc x19, x6, xzr+        stp x22, x14, [sp, #48]+        adds x14, x26, x24+        str x19, [sp, #64]+        umulh x19, x13, x8+        adcs x7, x11, x12+        adcs x22, x4, x25+        mul x6, x21, x16+        adc x19, x19, xzr+        subs x11, x17, x13+        cneg x12, x11, cc+        csetm x11, cc+        subs x24, x8, x20+        cinv x11, x11, cc+        cneg x24, x24, cc+        adds x4, x14, x6+        adcs x14, x7, x14+        mul x3, x12, x24+        adcs x7, x22, x7+        adcs x22, x19, x22+        umulh x12, x12, x24+        adc x24, xzr, x19+        adds x19, x14, x6+        eor x3, x3, x11+        adcs x26, x7, x4+        adcs x14, x22, x14+        adcs x25, x24, x7+        adcs x23, xzr, x22+        eor x7, x12, x11+        adc x12, xzr, x24+        subs x22, x21, x10+        cneg x24, x22, cc+        csetm x22, cc+        subs x15, x5, x16+        cinv x22, x22, cc+        cneg x15, x15, cc+        cmn x11, #0x1+        adcs x3, x25, x3+        mul x25, x24, x15+        adcs x23, x23, x7+        adc x11, x12, x11+        subs x7, x10, x13+        umulh x15, x24, x15+        cneg x12, x7, cc+        csetm x7, cc+        eor x24, x25, x22+        eor x25, x15, x22+        cmn x22, #0x1+        adcs x24, x4, x24+        adcs x19, x19, x25+        adcs x15, x26, x22+        adcs x4, x14, x22+        adcs x26, x3, x22+        adcs x25, x23, x22+        adc x23, x11, x22+        subs x14, x21, x17+        cneg x3, x14, cc+        csetm x11, cc+        subs x14, x8, x5+        cneg x14, x14, cc+        cinv x7, x7, cc+        subs x13, x21, x13+        cneg x21, x13, cc+        csetm x13, cc+        mul x22, x12, x14+        subs x8, x8, x16+        cinv x13, x13, cc+        umulh x14, x12, x14+        cneg x12, x8, cc+        subs x8, x20, x16+        cneg x8, x8, cc+        cinv x16, x11, cc+        eor x22, x22, x7+        cmn x7, #0x1+        eor x14, x14, x7+        adcs x4, x4, x22+        mul x11, x3, x8+        adcs x22, x26, x14+        adcs x14, x25, x7+        eor x25, x24, x9+        adc x26, x23, x7+        umulh x7, x3, x8+        subs x17, x10, x17+        cneg x24, x17, cc+        eor x3, x11, x16+        csetm x11, cc+        subs x20, x20, x5+        cneg x5, x20, cc+        cinv x11, x11, cc+        cmn x16, #0x1+        mul x17, x21, x12+        eor x8, x7, x16+        adcs x10, x19, x3+        and x19, x9, #0x1ff+        adcs x20, x15, x8+        umulh x15, x21, x12+        eor x12, x10, x9+        eor x8, x6, x9+        adcs x6, x4, x16+        adcs x4, x22, x16+        adcs x21, x14, x16+        adc x7, x26, x16+        mul x10, x24, x5+        cmn x13, #0x1+        ldp x3, x14, [x1]+        eor x17, x17, x13+        umulh x5, x24, x5+        adcs x20, x20, x17+        eor x17, x15, x13+        adcs x16, x6, x17+        eor x22, x10, x11+        adcs x23, x4, x13+        extr x10, x14, x3, #52+        and x26, x3, #0xfffffffffffff+        adcs x24, x21, x13+        and x15, x10, #0xfffffffffffff+        adc x6, x7, x13+        cmn x11, #0x1+        adcs x17, x20, x22+        eor x4, x5, x11+        ldp x21, x10, [sp]+        adcs x7, x16, x4+        eor x16, x17, x9+        eor x13, x7, x9+        ldp x3, x17, [sp, #16]+        adcs x7, x23, x11+        eor x23, x7, x9+        ldp x5, x22, [sp, #32]+        adcs x7, x24, x11+        adc x24, x6, x11+        ldr x6, [x2, #64]+        adds x20, x8, x21+        lsl x11, x20, #9+        eor x4, x7, x9+        orr x7, x11, x19+        eor x8, x24, x9+        adcs x11, x25, x10+        mul x26, x6, x26+        ldp x19, x24, [sp, #48]+        adcs x12, x12, x3+        adcs x16, x16, x17+        adcs x9, x13, x5+        ldr x25, [sp, #64]+        extr x20, x11, x20, #55+        adcs x13, x23, x22+        adcs x4, x4, x19+        extr x23, x12, x11, #55+        adcs x8, x8, x24+        adc x11, x25, xzr+        adds x21, x9, x21+        extr x9, x16, x12, #55+        lsr x12, x16, #55+        adcs x10, x13, x10+        mul x15, x6, x15+        adcs x13, x4, x3+        ldp x16, x4, [x2]+        ldr x3, [x1, #64]+        adcs x17, x8, x17+        adcs x5, x5, x7+        adcs x20, x22, x20+        adcs x8, x19, x23+        and x22, x16, #0xfffffffffffff+        ldp x19, x7, [x1, #16]+        adcs x9, x24, x9+        extr x24, x4, x16, #52+        adc x16, x12, x25+        mul x22, x3, x22+        and x25, x24, #0xfffffffffffff+        extr x14, x19, x14, #40+        and x12, x14, #0xfffffffffffff+        extr x23, x7, x19, #28+        ldp x19, x24, [x2, #16]+        mul x14, x3, x25+        and x23, x23, #0xfffffffffffff+        add x22, x26, x22+        lsl x11, x11, #48+        lsr x26, x22, #52+        lsl x25, x22, #12+        mul x22, x6, x12+        extr x12, x19, x4, #40+        add x4, x15, x14+        mul x15, x6, x23+        add x4, x4, x26+        extr x23, x24, x19, #28+        ldp x14, x19, [x1, #32]+        and x26, x12, #0xfffffffffffff+        extr x12, x4, x25, #12+        and x25, x23, #0xfffffffffffff+        adds x21, x21, x12+        mul x12, x3, x26+        extr x23, x14, x7, #16+        and x23, x23, #0xfffffffffffff+        mul x7, x3, x25+        ldp x25, x26, [x2, #32]+        add x12, x22, x12+        extr x22, x19, x14, #56+        mul x23, x6, x23+        lsr x14, x14, #4+        extr x24, x25, x24, #16+        add x7, x15, x7+        and x15, x24, #0xfffffffffffff+        and x22, x22, #0xfffffffffffff+        lsr x24, x4, #52+        mul x15, x3, x15+        and x14, x14, #0xfffffffffffff+        add x12, x12, x24+        lsl x24, x4, #12+        lsr x4, x12, #52+        extr x24, x12, x24, #24+        adcs x10, x10, x24+        lsl x24, x12, #12+        add x12, x7, x4+        mul x22, x6, x22+        add x4, x23, x15+        extr x7, x12, x24, #36+        adcs x13, x13, x7+        lsl x15, x12, #12+        add x7, x4, x11+        lsr x24, x12, #52+        ldp x23, x11, [x2, #48]+        add x4, x7, x24+        mul x12, x6, x14+        extr x7, x26, x25, #56+        extr x14, x4, x15, #48+        and x2, x7, #0xfffffffffffff+        extr x24, x11, x23, #32+        ldp x15, x7, [x1, #48]+        and x1, x24, #0xfffffffffffff+        lsr x24, x4, #52+        mul x2, x3, x2+        extr x26, x23, x26, #44+        lsr x23, x25, #4+        and x23, x23, #0xfffffffffffff+        and x25, x26, #0xfffffffffffff+        extr x26, x7, x15, #32+        extr x19, x15, x19, #44+        mul x23, x3, x23+        and x15, x26, #0xfffffffffffff+        lsl x26, x4, #12+        and x4, x19, #0xfffffffffffff+        lsr x11, x11, #20+        mul x19, x6, x4+        adcs x17, x17, x14+        add x14, x22, x2+        add x22, x12, x23+        lsr x7, x7, #20+        add x22, x22, x24+        extr x2, x22, x26, #60+        mul x24, x3, x25+        lsr x22, x22, #52+        add x14, x14, x22+        lsl x22, x2, #8+        extr x22, x14, x22, #8+        lsl x2, x14, #12+        mul x1, x3, x1+        adcs x12, x5, x22+        mul x5, x6, x15+        and x26, x10, x13+        and x4, x26, x17+        add x23, x19, x24+        lsr x14, x14, #52+        mul x22, x3, x11+        add x11, x23, x14+        extr x25, x11, x2, #20+        lsl x19, x11, #12+        adcs x25, x20, x25+        and x14, x4, x12+        add x1, x5, x1+        and x14, x14, x25+        mul x15, x6, x7+        add x26, x15, x22+        mul x6, x6, x3+        lsr x22, x11, #52+        add x4, x1, x22+        lsr x1, x4, #52+        extr x3, x4, x19, #32+        lsl x15, x4, #12+        add x7, x26, x1+        adcs x23, x8, x3+        extr x20, x7, x15, #44+        and x3, x14, x23+        lsr x19, x7, #44+        adcs x7, x9, x20+        add x11, x6, x19+        adc x4, x16, x11+        lsr x14, x4, #9+        cmp xzr, xzr+        and x15, x3, x7+        orr x3, x4, #0xfffffffffffffe00+        adcs xzr, x21, x14+        adcs xzr, x15, xzr+        adcs xzr, x3, xzr+        adcs x11, x21, x14+        and x14, x11, #0x1ff+        adcs x1, x10, xzr+        extr x10, x1, x11, #9+        str x14, [x0, #64]+        adcs x14, x13, xzr+        extr x11, x14, x1, #9+        adcs x1, x17, xzr+        extr x4, x1, x14, #9+        stp x10, x11, [x0]+        adcs x11, x12, xzr+        extr x14, x11, x1, #9+        adcs x10, x25, xzr+        extr x11, x10, x11, #9+        stp x4, x14, [x0, #16]+        adcs x14, x23, xzr+        extr x10, x14, x10, #9+        adcs x1, x7, xzr+        stp x11, x10, [x0, #32]+        extr x14, x1, x14, #9+        adc x10, x3, xzr+        extr x26, x10, x1, #9+        stp x14, x26, [x0, #48]+        CFI_INC_SP(80)+        CFI_POP2(x25,x26)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_mul_p521)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_sqr_p521)++Lp521_jscalarmul_sqr_p521:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        ldr q23, [x1, #32]+        ldp x9, x2, [x1, #32]+        ldr q16, [x1, #32]+        ldr q20, [x1, #48]+        ldp x6, x13, [x1, #48]+        rev64 v2.4S, v23.4S+        mul x14, x9, x2+        ldr q31, [x1, #48]+        subs x22, x9, x2+        uzp2 v26.4S, v23.4S, v23.4S+        mul v30.4S, v2.4S, v16.4S+        xtn v0.2S, v20.2D+        csetm x12, cc+        xtn v21.2S, v16.2D+        xtn v23.2S, v23.2D+        umulh x10, x9, x6+        rev64 v27.4S, v31.4S+        umull v2.2D, v21.2S, v26.2S+        cneg x23, x22, cc+        uaddlp v25.2D, v30.4S+        umull v18.2D, v21.2S, v23.2S+        mul x22, x9, x6+        mul v6.4S, v27.4S, v20.4S+        uzp2 v17.4S, v20.4S, v20.4S+        shl v20.2D, v25.2D, #32+        uzp2 v27.4S, v31.4S, v31.4S+        mul x16, x2, x13+        umlal v20.2D, v21.2S, v23.2S+        usra v2.2D, v18.2D, #32+        adds x8, x22, x10+        umull v25.2D, v17.2S, v27.2S+        xtn v31.2S, v31.2D+        movi v1.2D, #0xffffffff+        adc x3, x10, xzr+        umulh x21, x2, x13+        uzp2 v21.4S, v16.4S, v16.4S+        umull v18.2D, v0.2S, v27.2S+        subs x19, x13, x6+        and v7.16B, v2.16B, v1.16B+        umull v27.2D, v0.2S, v31.2S+        cneg x20, x19, cc+        movi v30.2D, #0xffffffff+        umull v16.2D, v21.2S, v26.2S+        umlal v7.2D, v21.2S, v23.2S+        mul x19, x23, x20+        cinv x7, x12, cc+        uaddlp v6.2D, v6.4S+        eor x12, x19, x7+        adds x11, x8, x16+        umulh x10, x23, x20+        ldr q1, [x1]+        usra v16.2D, v2.2D, #32+        adcs x19, x3, x21+        shl v2.2D, v6.2D, #32+        adc x20, x21, xzr+        adds x17, x19, x16+        usra v18.2D, v27.2D, #32+        adc x19, x20, xzr+        cmn x7, #0x1+        umlal v2.2D, v0.2S, v31.2S+        umulh x16, x9, x2+        adcs x8, x11, x12+        usra v16.2D, v7.2D, #32+        ldr x12, [x1, #64]+        eor x20, x10, x7+        umulh x10, x6, x13+        mov x23, v2.d[0]+        mov x3, v2.d[1]+        adcs x21, x17, x20+        usra v25.2D, v18.2D, #32+        and v23.16B, v18.16B, v30.16B+        adc x7, x19, x7+        adds x22, x22, x22+        ldr q7, [x1, #16]+        adcs x17, x8, x8+        umlal v23.2D, v17.2S, v31.2S+        mov x19, v16.d[0]+        mul x11, x12, x12+        ldr q4, [x1]+        usra v25.2D, v23.2D, #32+        add x5, x12, x12+        adcs x15, x21, x21+        ldr q28, [x1]+        mov x12, v20.d[1]+        adcs x24, x7, x7+        mov x21, v16.d[1]+        adc x4, xzr, xzr+        adds x19, x19, x14+        ldr q18, [x1, #16]+        xtn v26.2S, v1.2D+        adcs x8, x12, x16+        adc x21, x21, xzr+        adds x7, x19, x14+        xtn v23.2S, v7.2D+        rev64 v21.4S, v28.4S+        adcs x12, x8, x16+        ldp x20, x19, [x1]+        mov x16, v25.d[1]+        xtn v22.2S, v28.2D+        adc x14, x21, xzr+        adds x8, x22, x12+        uzp2 v24.4S, v28.4S, v28.4S+        rev64 v28.4S, v18.4S+        mul x12, x6, x13+        mul v16.4S, v21.4S, v1.4S+        shrn v31.2S, v7.2D, #32+        adcs x22, x17, x14+        mov x14, v25.d[0]+        and x21, x20, #0xfffffffffffff+        umull v17.2D, v26.2S, v24.2S+        ldr q2, [x1, #32]+        adcs x17, x15, xzr+        ldr q30, [x1, #48]+        umull v7.2D, v26.2S, v22.2S+        adcs x15, x24, xzr+        ldr q0, [x1, #16]+        movi v6.2D, #0xffffffff+        adc x4, x4, xzr+        adds x14, x14, x12+        uzp1 v27.4S, v18.4S, v4.4S+        uzp2 v19.4S, v1.4S, v1.4S+        adcs x24, x3, x10+        mul x3, x5, x21+        umull v29.2D, v23.2S, v31.2S+        ldr q5, [x1]+        adc x21, x16, xzr+        adds x16, x14, x12+        extr x12, x19, x20, #52+        umull v18.2D, v19.2S, v24.2S+        adcs x24, x24, x10+        and x10, x12, #0xfffffffffffff+        ldp x14, x12, [x1, #16]+        usra v17.2D, v7.2D, #32+        adc x21, x21, xzr+        adds x23, x23, x17+        mul x17, x5, x10+        shl v21.2D, v29.2D, #33+        lsl x10, x3, #12+        lsr x1, x3, #52+        rev64 v29.4S, v2.4S+        uaddlp v25.2D, v16.4S+        add x17, x17, x1+        adcs x16, x16, x15+        extr x3, x14, x19, #40+        mov x15, v20.d[0]+        extr x10, x17, x10, #12+        and x3, x3, #0xfffffffffffff+        shl v3.2D, v25.2D, #32+        and v6.16B, v17.16B, v6.16B+        mul x1, x5, x3+        usra v18.2D, v17.2D, #32+        adcs x3, x24, x4+        extr x4, x12, x14, #28+        umlal v6.2D, v19.2S, v22.2S+        xtn v20.2S, v2.2D+        umlal v3.2D, v26.2S, v22.2S+        movi v26.2D, #0xffffffff+        lsr x24, x17, #52+        and x4, x4, #0xfffffffffffff+        uzp2 v19.4S, v2.4S, v2.4S+        add x1, x1, x24+        mul x24, x5, x4+        lsl x4, x17, #12+        xtn v24.2S, v5.2D+        extr x17, x1, x4, #24+        adc x21, x21, xzr+        umlal v21.2D, v23.2S, v23.2S+        adds x4, x15, x10+        lsl x10, x1, #12+        adcs x15, x7, x17+        mul v23.4S, v28.4S, v4.4S+        and x7, x4, #0x1ff+        lsr x17, x1, #52+        umulh x1, x19, x12+        uzp2 v17.4S, v5.4S, v5.4S+        extr x4, x15, x4, #9+        add x24, x24, x17+        mul v29.4S, v29.4S, v5.4S+        extr x17, x24, x10, #36+        extr x10, x9, x12, #16+        uzp1 v28.4S, v4.4S, v4.4S+        adcs x17, x8, x17+        and x8, x10, #0xfffffffffffff+        umull v16.2D, v24.2S, v20.2S+        extr x10, x17, x15, #9+        mul x15, x5, x8+        stp x4, x10, [x0]+        lsl x4, x24, #12+        lsr x8, x9, #4+        uaddlp v4.2D, v23.4S+        and x8, x8, #0xfffffffffffff+        umull v23.2D, v24.2S, v19.2S+        mul x8, x5, x8+        extr x10, x2, x9, #56+        lsr x24, x24, #52+        and x10, x10, #0xfffffffffffff+        add x15, x15, x24+        extr x4, x15, x4, #48+        mul x24, x5, x10+        lsr x10, x15, #52+        usra v23.2D, v16.2D, #32+        add x10, x8, x10+        shl v4.2D, v4.2D, #32+        adcs x22, x22, x4+        extr x4, x6, x2, #44+        lsl x15, x15, #12+        lsr x8, x10, #52+        extr x15, x10, x15, #60+        and x10, x4, #0xfffffffffffff+        umlal v4.2D, v28.2S, v27.2S+        add x8, x24, x8+        extr x4, x13, x6, #32+        mul x24, x5, x10+        uzp2 v16.4S, v30.4S, v30.4S+        lsl x10, x15, #8+        rev64 v28.4S, v30.4S+        and x15, x4, #0xfffffffffffff+        extr x4, x8, x10, #8+        mul x10, x5, x15+        lsl x15, x8, #12+        adcs x23, x23, x4+        lsr x4, x8, #52+        lsr x8, x13, #20+        add x4, x24, x4+        mul x8, x5, x8+        lsr x24, x4, #52+        extr x15, x4, x15, #20+        lsl x4, x4, #12+        add x10, x10, x24+        adcs x15, x16, x15+        extr x4, x10, x4, #32+        umulh x5, x20, x14+        adcs x3, x3, x4+        usra v18.2D, v6.2D, #32+        lsl x16, x10, #12+        extr x24, x15, x23, #9+        lsr x10, x10, #52+        uzp2 v27.4S, v0.4S, v0.4S+        add x8, x8, x10+        extr x10, x3, x15, #9+        extr x4, x22, x17, #9+        and v25.16B, v23.16B, v26.16B+        lsr x17, x8, #44+        extr x15, x8, x16, #44+        extr x16, x23, x22, #9+        xtn v7.2S, v30.2D+        mov x8, v4.d[0]+        stp x24, x10, [x0, #32]+        uaddlp v30.2D, v29.4S+        stp x4, x16, [x0, #16]+        umulh x24, x20, x19+        adcs x15, x21, x15+        adc x16, x11, x17+        subs x11, x20, x19+        xtn v5.2S, v0.2D+        csetm x17, cc+        extr x3, x15, x3, #9+        mov x22, v4.d[1]+        cneg x21, x11, cc+        subs x10, x12, x14+        mul v31.4S, v28.4S, v0.4S+        cneg x10, x10, cc+        cinv x11, x17, cc+        shl v4.2D, v30.2D, #32+        umull v28.2D, v5.2S, v16.2S+        extr x23, x16, x15, #9+        adds x4, x8, x5+        mul x17, x21, x10+        umull v22.2D, v5.2S, v7.2S+        adc x15, x5, xzr+        adds x4, x4, x22+        uaddlp v2.2D, v31.4S+        lsr x5, x16, #9+        adcs x16, x15, x1+        mov x15, v18.d[0]+        adc x1, x1, xzr+        umulh x10, x21, x10+        adds x22, x16, x22+        umlal v4.2D, v24.2S, v20.2S+        umull v30.2D, v27.2S, v16.2S+        stp x3, x23, [x0, #48]+        add x3, x7, x5+        adc x16, x1, xzr+        usra v28.2D, v22.2D, #32+        mul x23, x20, x19+        eor x1, x17, x11+        cmn x11, #0x1+        mov x17, v18.d[1]+        umull v18.2D, v17.2S, v19.2S+        adcs x7, x4, x1+        eor x1, x10, x11+        umlal v25.2D, v17.2S, v20.2S+        movi v16.2D, #0xffffffff+        adcs x22, x22, x1+        usra v18.2D, v23.2D, #32+        umulh x4, x14, x14+        adc x1, x16, x11+        adds x10, x8, x8+        shl v23.2D, v2.2D, #32+        str x3, [x0, #64]+        adcs x5, x7, x7+        and v16.16B, v28.16B, v16.16B+        usra v30.2D, v28.2D, #32+        adcs x7, x22, x22+        mov x21, v3.d[1]+        adcs x11, x1, x1+        umlal v16.2D, v27.2S, v7.2S+        adc x22, xzr, xzr+        adds x16, x15, x23+        mul x8, x14, x12+        umlal v23.2D, v5.2S, v7.2S+        usra v18.2D, v25.2D, #32+        umulh x15, x14, x12+        adcs x21, x21, x24+        usra v30.2D, v16.2D, #32+        adc x1, x17, xzr+        adds x3, x16, x23+        adcs x21, x21, x24+        adc x1, x1, xzr+        adds x24, x10, x21+        umulh x21, x12, x12+        adcs x16, x5, x1+        adcs x10, x7, xzr+        mov x17, v21.d[1]+        adcs x23, x11, xzr+        adc x5, x22, xzr+        adds x1, x4, x8+        adcs x22, x17, x15+        ldp x17, x4, [x0]+        mov x11, v21.d[0]+        adc x21, x21, xzr+        adds x1, x1, x8+        adcs x15, x22, x15+        adc x8, x21, xzr+        adds x22, x11, x10+        mov x21, v3.d[0]+        adcs x11, x1, x23+        ldp x1, x10, [x0, #16]+        adcs x15, x15, x5+        adc x7, x8, xzr+        adds x8, x17, x21+        mov x23, v4.d[1]+        ldp x5, x21, [x0, #32]+        adcs x17, x4, x3+        ldr x4, [x0, #64]+        mov x3, v18.d[0]+        adcs x24, x1, x24+        stp x8, x17, [x0]+        adcs x17, x10, x16+        ldp x1, x16, [x0, #48]+        adcs x5, x5, x22+        adcs x8, x21, x11+        stp x5, x8, [x0, #32]+        adcs x1, x1, x15+        mov x15, v23.d[1]+        adcs x21, x16, x7+        stp x1, x21, [x0, #48]+        adc x10, x4, xzr+        subs x7, x14, x12+        mov x16, v18.d[1]+        cneg x5, x7, cc+        csetm x4, cc+        subs x11, x13, x6+        mov x8, v23.d[0]+        cneg x7, x11, cc+        cinv x21, x4, cc+        mov x11, v30.d[0]+        adds x4, x23, x3+        mul x22, x5, x7+        mov x23, v30.d[1]+        adcs x8, x8, x16+        adcs x16, x15, x11+        adc x11, x23, xzr+        umulh x3, x5, x7+        stp x24, x17, [x0, #16]+        mov x5, v4.d[0]+        subs x15, x20, x19+        cneg x7, x15, cc+        str x10, [x0, #64]+        csetm x1, cc+        subs x24, x2, x9+        cneg x17, x24, cc+        cinv x15, x1, cc+        adds x23, x4, x5+        umulh x1, x7, x17+        adcs x24, x8, x4+        adcs x10, x16, x8+        eor x8, x22, x21+        adcs x16, x11, x16+        mul x22, x7, x17+        eor x17, x1, x15+        adc x1, xzr, x11+        adds x11, x24, x5+        eor x7, x3, x21+        adcs x3, x10, x23+        adcs x24, x16, x24+        adcs x4, x1, x10+        eor x10, x22, x15+        adcs x16, xzr, x16+        adc x1, xzr, x1+        cmn x21, #0x1+        adcs x8, x4, x8+        adcs x22, x16, x7+        adc x7, x1, x21+        subs x21, x19, x12+        csetm x4, cc+        cneg x1, x21, cc+        subs x21, x13, x2+        cinv x16, x4, cc+        cneg x4, x21, cc+        cmn x15, #0x1+        adcs x21, x23, x10+        mul x23, x1, x4+        adcs x11, x11, x17+        adcs x3, x3, x15+        umulh x1, x1, x4+        adcs x24, x24, x15+        adcs x8, x8, x15+        adcs x22, x22, x15+        eor x17, x23, x16+        adc x15, x7, x15+        subs x7, x20, x14+        cneg x7, x7, cc+        csetm x4, cc+        subs x10, x20, x12+        cneg x23, x10, cc+        csetm x10, cc+        subs x12, x6, x9+        cinv x20, x4, cc+        cneg x12, x12, cc+        cmn x16, #0x1+        eor x1, x1, x16+        adcs x17, x24, x17+        mul x4, x7, x12+        adcs x8, x8, x1+        umulh x1, x7, x12+        adcs x24, x22, x16+        adc x7, x15, x16+        subs x12, x13, x9+        cneg x12, x12, cc+        cinv x13, x10, cc+        subs x19, x19, x14+        mul x9, x23, x12+        cneg x19, x19, cc+        csetm x10, cc+        eor x16, x1, x20+        subs x22, x6, x2+        umulh x12, x23, x12+        eor x1, x4, x20+        cinv x4, x10, cc+        cneg x22, x22, cc+        cmn x20, #0x1+        adcs x15, x11, x1+        eor x6, x12, x13+        adcs x10, x3, x16+        adcs x17, x17, x20+        eor x23, x9, x13+        adcs x2, x8, x20+        mul x11, x19, x22+        adcs x24, x24, x20+        adc x7, x7, x20+        cmn x13, #0x1+        adcs x3, x10, x23+        umulh x22, x19, x22+        adcs x17, x17, x6+        eor x12, x22, x4+        extr x22, x15, x21, #63+        adcs x8, x2, x13+        extr x21, x21, x5, #63+        ldp x16, x23, [x0]+        adcs x20, x24, x13+        eor x1, x11, x4+        adc x6, x7, x13+        cmn x4, #0x1+        ldp x2, x7, [x0, #16]+        adcs x1, x3, x1+        extr x19, x1, x15, #63+        adcs x14, x17, x12+        extr x1, x14, x1, #63+        lsl x17, x5, #1+        adcs x8, x8, x4+        extr x12, x8, x14, #8+        ldp x15, x11, [x0, #32]+        adcs x9, x20, x4+        adc x3, x6, x4+        adds x16, x12, x16+        extr x6, x9, x8, #8+        ldp x14, x12, [x0, #48]+        extr x8, x3, x9, #8+        adcs x20, x6, x23+        ldr x24, [x0, #64]+        lsr x6, x3, #8+        adcs x8, x8, x2+        and x2, x1, #0x1ff+        and x1, x20, x8+        adcs x4, x6, x7+        adcs x3, x17, x15+        and x1, x1, x4+        adcs x9, x21, x11+        and x1, x1, x3+        adcs x6, x22, x14+        and x1, x1, x9+        and x21, x1, x6+        adcs x14, x19, x12+        adc x1, x24, x2+        cmp xzr, xzr+        orr x12, x1, #0xfffffffffffffe00+        lsr x1, x1, #9+        adcs xzr, x16, x1+        and x21, x21, x14+        adcs xzr, x21, xzr+        adcs xzr, x12, xzr+        adcs x21, x16, x1+        adcs x1, x20, xzr+        adcs x19, x8, xzr+        stp x21, x1, [x0]+        adcs x1, x4, xzr+        adcs x21, x3, xzr+        stp x19, x1, [x0, #16]+        adcs x1, x9, xzr+        stp x21, x1, [x0, #32]+        adcs x21, x6, xzr+        adcs x1, x14, xzr+        stp x21, x1, [x0, #48]+        adc x1, x12, xzr+        and x1, x1, #0x1ff+        str x1, [x0, #64]+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_sqr_p521)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_sub_p521)++Lp521_jscalarmul_sub_p521:+        CFI_START+        ldp     x5, x6, [x1]+        ldp     x4, x3, [x2]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [x1, #16]+        ldp     x4, x3, [x2, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [x1, #32]+        ldp     x4, x3, [x2, #32]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        ldp     x11, x12, [x1, #48]+        ldp     x4, x3, [x2, #48]+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        ldr     x13, [x1, #64]+        ldr     x4, [x2, #64]+        sbcs    x13, x13, x4+        sbcs    x5, x5, xzr+        sbcs    x6, x6, xzr+        sbcs    x7, x7, xzr+        sbcs    x8, x8, xzr+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        and     x13, x13, #0x1ff+        stp     x5, x6, [x0]+        stp     x7, x8, [x0, #16]+        stp     x9, x10, [x0, #32]+        stp     x11, x12, [x0, #48]+        str     x13, [x0, #64]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_sub_p521)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/arm/p521_jscalarmul_alt.S view
@@ -0,0 +1,2143 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Jacobian form scalar multiplication for P-521+// Input scalar[9], point[27]; output res[27]+//+// extern void p521_jscalarmul_alt+//   (uint64_t res[static 27],+//    const uint64_t scalar[static 9],+//    const uint64_t point[static 27]);+//+// This function is a variant of its affine point version p521_scalarmul.+// Here, input and output points are assumed to be in Jacobian form with+// a triple (x,y,z) representing the affine point (x/z^2,y/z^3) when+// z is nonzero or the point at infinity (group identity) if z = 0.+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-521, returns a representation of n * P. If the result is the+// point at infinity (either because the input point was or because the+// scalar was a multiple of p_521) then the output is guaranteed to+// represent the point at infinity, i.e. to have its z coordinate zero.+//+// Standard ARM ABI: X0 = res, X1 = scalar, X2 = point+// ----------------------------------------------------------------------------+++#include "_internal_s2n_bignum_arm.h"++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p521_jscalarmul_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p521_jscalarmul_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p521_jscalarmul_alt)+++        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 72+#define JACSIZE (3*NUMSIZE)++// Safe copies of input res and additional values in variables.++#define tabup x15+#define bf x16+#define sgn x17+#define j x19+#define res x20++// Intermediate variables on the stack.+// The table is 16 entries, each of size JACSIZE = 3 * NUMSIZE++#define scalarb sp, #(0*NUMSIZE)+#define acc sp, #(1*NUMSIZE)+#define tabent sp, #(4*NUMSIZE)++#define tab sp, #(7*NUMSIZE)++// Round up to maintain stack alignment++#define NSPACE 3968++#define selectblock(I)                            \+        cmp     bf, #(1*I) __LF                      \+        ldp     x10, x11, [tabup] __LF               \+        csel    x0, x10, x0, eq __LF                 \+        csel    x1, x11, x1, eq __LF                 \+        ldp     x10, x11, [tabup, #16] __LF          \+        csel    x2, x10, x2, eq __LF                 \+        csel    x3, x11, x3, eq __LF                 \+        ldp     x10, x11, [tabup, #32] __LF          \+        csel    x4, x10, x4, eq __LF                 \+        csel    x5, x11, x5, eq __LF                 \+        ldp     x10, x11, [tabup, #48] __LF          \+        csel    x6, x10, x6, eq __LF                 \+        csel    x7, x11, x7, eq __LF                 \+        ldr     x10, [tabup, #64] __LF               \+        csel    x8, x10, x8, eq __LF                 \+        add     tabup, tabup, #JACSIZE++// Loading large constants++#define movbig(nn,n3,n2,n1,n0)                                      \+        movz    nn, n0 __LF                                            \+        movk    nn, n1, lsl #16 __LF                                   \+        movk    nn, n2, lsl #32 __LF                                   \+        movk    nn, n3, lsl #48++S2N_BN_SYMBOL(p521_jscalarmul_alt):+        CFI_START++        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x30)+        CFI_DEC_SP(NSPACE)++// Preserve the "res" input argument; others get processed early.++        mov     res, x0++// Reduce the input scalar mod n_521 and store it to "scalarb".++        mov     x19, x2+        add     x0, scalarb+        CFI_BL(Lp521_jscalarmul_alt_bignum_mod_n521_9)+        mov     x2, x19++// Set the tab[0] table entry to the input point = 1 * P, but also+// reduce all coordinates modulo p. In principle we assume reduction+// as a precondition, but this reduces the scope for surprise, e.g.+// making sure that any input with z = 0 is treated as zero, even+// if the other coordinates are not in fact reduced.++        add     x0, tab+        mov     x1, x19+        CFI_BL(Lp521_jscalarmul_alt_bignum_mod_p521_9)++        add     x0, tab+NUMSIZE+        add     x1, x19, #NUMSIZE+        CFI_BL(Lp521_jscalarmul_alt_bignum_mod_p521_9)++        add     x0, tab+2*NUMSIZE+        add     x1, x19, #(2*NUMSIZE)+        CFI_BL(Lp521_jscalarmul_alt_bignum_mod_p521_9)++// If bit 520 of the scalar is set, then negate the scalar mod n_521,+// i.e. do scalar |-> n_521 - scalar, and also the point to compensate+// by negating its y coordinate. This further step is not needed by+// the indexing scheme (the top window is only a couple of bits either+// way), but is convenient to exclude a problem with the specific value+// scalar = n_521 - 18, where the last Jacobian addition is of the form+// (n_521 - 9) * P + -(9 * P) and hence is a degenerate doubling case.++        ldp     x0, x1, [scalarb]+        movbig(x10, #0xbb6f, #0xb71e, #0x9138, #0x6409)+        subs    x10, x10, x0+        movbig(x11, #0x3bb5, #0xc9b8, #0x899c, #0x47ae)+        sbcs    x11, x11, x1+        ldp     x2, x3, [scalarb+16]+        movbig(x12, #0x7fcc, #0x0148, #0xf709, #0xa5d0)+        sbcs    x12, x12, x2+        movbig(x13, #0x5186, #0x8783, #0xbf2f, #0x966b)+        sbcs    x13, x13, x3+        ldp     x4, x5, [scalarb+32]+        mov     x14, 0xfffffffffffffffa+        sbcs    x14, x14, x4+        mov     x15, 0xffffffffffffffff+        sbcs    x15, x15, x5+        ldp     x6, x7, [scalarb+48]+        mov     x16, 0xffffffffffffffff+        sbcs    x16, x16, x6+        mov     x17, 0xffffffffffffffff+        sbcs    x17, x17, x7+        ldr     x8, [scalarb+64]+        mov     x19, 0x00000000000001ff+        sbc     x19, x19, x8+        tst     x8, 0x100+        csetm   x9, ne+        csel    x0, x10, x0, ne+        csel    x1, x11, x1, ne+        csel    x2, x12, x2, ne+        csel    x3, x13, x3, ne+        csel    x4, x14, x4, ne+        csel    x5, x15, x5, ne+        csel    x6, x16, x6, ne+        csel    x7, x17, x7, ne+        csel    x8, x19, x8, ne+        stp     x0, x1, [scalarb]+        stp     x2, x3, [scalarb+16]+        stp     x4, x5, [scalarb+32]+        stp     x6, x7, [scalarb+48]+        str     x8, [scalarb+64]++        add     tabup, tab+        ldp     x0, x1, [tabup, #NUMSIZE]+        ldp     x2, x3, [tabup, #NUMSIZE+16]+        ldp     x4, x5, [tabup, #NUMSIZE+32]+        ldp     x6, x7, [tabup, #NUMSIZE+48]+        ldr     x8, [tabup, #NUMSIZE+64]+        orr     x10, x0, x1+        orr     x11, x2, x3+        orr     x12, x4, x5+        orr     x13, x6, x7+        orr     x10, x10, x11+        orr     x12, x12, x13+        orr     x12, x12, x8+        orr     x10, x10, x12+        cmp     x10, xzr+        csel    x9, x9, xzr, ne+        eor     x0, x0, x9+        eor     x1, x1, x9+        eor     x2, x2, x9+        eor     x3, x3, x9+        eor     x4, x4, x9+        eor     x5, x5, x9+        eor     x6, x6, x9+        eor     x7, x7, x9+        and     x9, x9, #0x1FF+        eor     x8, x8, x9+        stp     x0, x1, [tabup, #NUMSIZE]+        stp     x2, x3, [tabup, #NUMSIZE+16]+        stp     x4, x5, [tabup, #NUMSIZE+32]+        stp     x6, x7, [tabup, #NUMSIZE+48]+        str     x8, [tabup, #NUMSIZE+64]++// Compute and record tab[1] = 2 * p, ..., tab[15] = 16 * P++        add     x0, tab+JACSIZE*1+        add     x1, tab+        CFI_BL(Lp521_jscalarmul_alt_jdouble)++        add     x0, tab+JACSIZE*2+        add     x1, tab+JACSIZE*1+        add     x2, tab+        CFI_BL(Lp521_jscalarmul_alt_jadd)++        add     x0, tab+JACSIZE*3+        add     x1, tab+JACSIZE*1+        CFI_BL(Lp521_jscalarmul_alt_jdouble)++        add     x0, tab+JACSIZE*4+        add     x1, tab+JACSIZE*3+        add     x2, tab+        CFI_BL(Lp521_jscalarmul_alt_jadd)++        add     x0, tab+JACSIZE*5+        add     x1, tab+JACSIZE*2+        CFI_BL(Lp521_jscalarmul_alt_jdouble)++        add     x0, tab+JACSIZE*6+        add     x1, tab+JACSIZE*5+        add     x2, tab+        CFI_BL(Lp521_jscalarmul_alt_jadd)++        add     x0, tab+JACSIZE*7+        add     x1, tab+JACSIZE*3+        CFI_BL(Lp521_jscalarmul_alt_jdouble)++        add     x0, tab+JACSIZE*8+        add     x1, tab+JACSIZE*7+        add     x2, tab+        CFI_BL(Lp521_jscalarmul_alt_jadd)++        add     x0, tab+JACSIZE*9+        add     x1, tab+JACSIZE*4+        CFI_BL(Lp521_jscalarmul_alt_jdouble)++        add     x0, tab+JACSIZE*10+        add     x1, tab+JACSIZE*9+        add     x2, tab+        CFI_BL(Lp521_jscalarmul_alt_jadd)++        add     x0, tab+JACSIZE*11+        add     x1, tab+JACSIZE*5+        CFI_BL(Lp521_jscalarmul_alt_jdouble)++        add     x0, tab+JACSIZE*12+        add     x1, tab+JACSIZE*11+        add     x2, tab+        CFI_BL(Lp521_jscalarmul_alt_jadd)++        add     x0, tab+JACSIZE*13+        add     x1, tab+JACSIZE*6+        CFI_BL(Lp521_jscalarmul_alt_jdouble)++        add     x0, tab+JACSIZE*14+        add     x1, tab+JACSIZE*13+        add     x2, tab+        CFI_BL(Lp521_jscalarmul_alt_jadd)++        add     x0, tab+JACSIZE*15+        add     x1, tab+JACSIZE*7+        CFI_BL(Lp521_jscalarmul_alt_jdouble)++// Add the recoding constant sum_i(16 * 32^i) to the scalar to allow signed+// digits. The digits of the constant, in lowest-to-highest order, are as+// follows; they are generated dynamically since none is a simple ARM load.+//+// 0x0842108421084210+// 0x1084210842108421+// 0x2108421084210842+// 0x4210842108421084+// 0x8421084210842108+// 0x0842108421084210+// 0x1084210842108421+// 0x2108421084210842+// 0x0000000000000084++        ldp     x0, x1, [scalarb]+        ldp     x2, x3, [scalarb+16]+        ldp     x4, x5, [scalarb+32]+        ldp     x6, x7, [scalarb+48]+        ldr     x8, [scalarb+64]++        movbig(x10, #0x1084, #0x2108, #0x4210, #0x8421)+        adds    x0, x0, x10, lsr #1+        adcs    x1, x1, x10+        lsl     x10, x10, #1+        adcs    x2, x2, x10+        lsl     x10, x10, #1+        adcs    x3, x3, x10+        lsl     x10, x10, #1+        adcs    x4, x4, x10+        lsr     x11, x10, #4+        adcs    x5, x5, x11+        lsr     x10, x10, #3+        adcs    x6, x6, x10+        lsl     x10, x10, #1+        adcs    x7, x7, x10+        lsl     x10, x10, #1+        and     x10, x10, #0xFF+        adc     x8, x8, x10++// Because of the initial reduction the top bitfield (>= bits 520) is <= 1,+// i.e. just a single bit. Record that in "bf", then shift the whole+// scalar left 56 bits to align the top of the next bitfield with the MSB+// (bits 571..575).++        lsr     bf, x8, #8+        extr    x8, x8, x7, #8+        extr    x7, x7, x6, #8+        extr    x6, x6, x5, #8+        extr    x5, x5, x4, #8+        extr    x4, x4, x3, #8+        extr    x3, x3, x2, #8+        extr    x2, x2, x1, #8+        extr    x1, x1, x0, #8+        lsl     x0, x0, #56+        stp     x0, x1, [scalarb]+        stp     x2, x3, [scalarb+16]+        stp     x4, x5, [scalarb+32]+        stp     x6, x7, [scalarb+48]+        str     x8, [scalarb+64]++// According to the top bit, initialize the accumulator to P or 0. This top+// digit, uniquely, is not recoded so there is no sign adjustment to make.+// We only really need to adjust the z coordinate to zero, but do all three.++        add     tabup, tab+        cmp     bf, xzr++        ldp     x0, x1, [tabup]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc]+        ldp     x0, x1, [tabup, #16]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+16]+        ldp     x0, x1, [tabup, #32]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+32]+        ldp     x0, x1, [tabup, #48]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+48]+        ldp     x0, x1, [tabup, #64]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+64]+        ldp     x0, x1, [tabup, #80]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+80]+        ldp     x0, x1, [tabup, #96]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+96]+        ldp     x0, x1, [tabup, #112]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+112]+        ldp     x0, x1, [tabup, #128]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+128]+        ldp     x0, x1, [tabup, #144]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+144]+        ldp     x0, x1, [tabup, #160]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+160]+        ldp     x0, x1, [tabup, #176]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+176]+        ldp     x0, x1, [tabup, #192]+        csel    x0, x0, xzr, ne+        csel    x1, x1, xzr, ne+        stp     x0, x1, [acc+192]+        ldr     x0, [tabup, #208]+        csel    x0, x0, xzr, ne+        str     x0, [acc+208]++// Main loop over size-5 bitfields: double 5 times then add signed digit+// At each stage we shift the scalar left by 5 bits so we can simply pick+// the top 5 bits as the bitfield, saving some fiddle over indexing.++        mov     j, #520++Lp521_jscalarmul_alt_mainloop:+        sub     j, j, #5++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp521_jscalarmul_alt_jdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp521_jscalarmul_alt_jdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp521_jscalarmul_alt_jdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp521_jscalarmul_alt_jdouble)++        add     x0, acc+        add     x1, acc+        CFI_BL(Lp521_jscalarmul_alt_jdouble)++// Choose the bitfield and adjust it to sign and magnitude++        ldp     x0, x1, [scalarb]+        ldp     x2, x3, [scalarb+16]+        ldp     x4, x5, [scalarb+32]+        ldp     x6, x7, [scalarb+48]+        ldr     x8, [scalarb+64]+        lsr     bf, x8, #59+        extr    x8, x8, x7, #59+        extr    x7, x7, x6, #59+        extr    x6, x6, x5, #59+        extr    x5, x5, x4, #59+        extr    x4, x4, x3, #59+        extr    x3, x3, x2, #59+        extr    x2, x2, x1, #59+        extr    x1, x1, x0, #59+        lsl     x0, x0, #5+        stp     x0, x1, [scalarb]+        stp     x2, x3, [scalarb+16]+        stp     x4, x5, [scalarb+32]+        stp     x6, x7, [scalarb+48]+        str     x8, [scalarb+64]++        subs    bf, bf, #16+        csetm   sgn, lo                 // sgn = sign of digit (1 = negative)+        cneg    bf, bf, lo              // bf = absolute value of digit++// Conditionally select the table entry tab[i-1] = i * P in constant time++        mov     x0, xzr+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, xzr+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        add     tabup, tab+        selectblock(1)+        selectblock(2)+        selectblock(3)+        selectblock(4)+        selectblock(5)+        selectblock(6)+        selectblock(7)+        selectblock(8)+        selectblock(9)+        selectblock(10)+        selectblock(11)+        selectblock(12)+        selectblock(13)+        selectblock(14)+        selectblock(15)+        selectblock(16)+        stp     x0, x1, [tabent]+        stp     x2, x3, [tabent+16]+        stp     x4, x5, [tabent+32]+        stp     x6, x7, [tabent+48]+        str     x8, [tabent+64]++        mov     x0, xzr+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, xzr+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        add     tabup, tab+2*NUMSIZE+        selectblock(1)+        selectblock(2)+        selectblock(3)+        selectblock(4)+        selectblock(5)+        selectblock(6)+        selectblock(7)+        selectblock(8)+        selectblock(9)+        selectblock(10)+        selectblock(11)+        selectblock(12)+        selectblock(13)+        selectblock(14)+        selectblock(15)+        selectblock(16)+        stp     x0, x1, [tabent+2*NUMSIZE]+        stp     x2, x3, [tabent+2*NUMSIZE+16]+        stp     x4, x5, [tabent+2*NUMSIZE+32]+        stp     x6, x7, [tabent+2*NUMSIZE+48]+        str     x8, [tabent+2*NUMSIZE+64]++        mov     x0, xzr+        mov     x1, xzr+        mov     x2, xzr+        mov     x3, xzr+        mov     x4, xzr+        mov     x5, xzr+        mov     x6, xzr+        mov     x7, xzr+        mov     x8, xzr+        add     tabup, tab+NUMSIZE+        selectblock(1)+        selectblock(2)+        selectblock(3)+        selectblock(4)+        selectblock(5)+        selectblock(6)+        selectblock(7)+        selectblock(8)+        selectblock(9)+        selectblock(10)+        selectblock(11)+        selectblock(12)+        selectblock(13)+        selectblock(14)+        selectblock(15)+        selectblock(16)++// Store it to "tabent" with the y coordinate optionally negated.+// This is done carefully to give coordinates < p_521 even in+// the degenerate case y = 0 (when z = 0 for points on the curve).++        orr     x10, x0, x1+        orr     x11, x2, x3+        orr     x12, x4, x5+        orr     x13, x6, x7+        orr     x10, x10, x11+        orr     x12, x12, x13+        orr     x12, x12, x8+        orr     x10, x10, x12+        cmp     x10, xzr+        csel    sgn, sgn, xzr, ne++        eor     x0, x0, sgn+        eor     x1, x1, sgn+        eor     x2, x2, sgn+        eor     x3, x3, sgn+        eor     x4, x4, sgn+        eor     x5, x5, sgn+        eor     x6, x6, sgn+        eor     x7, x7, sgn+        and     sgn, sgn, #0x1FF+        eor     x8, x8, sgn++        stp     x0, x1, [tabent+NUMSIZE]+        stp     x2, x3, [tabent+NUMSIZE+16]+        stp     x4, x5, [tabent+NUMSIZE+32]+        stp     x6, x7, [tabent+NUMSIZE+48]+        str     x8, [tabent+NUMSIZE+64]++// Add to the accumulator++        add     x0, acc+        add     x1, acc+        add     x2, tabent+        CFI_BL(Lp521_jscalarmul_alt_jadd)++        cbnz    j, Lp521_jscalarmul_alt_mainloop++// That's the end of the main loop, and we just need to copy the+// result in "acc" to the output.++        ldp     x0, x1, [acc]+        stp     x0, x1, [res]+        ldp     x0, x1, [acc+16]+        stp     x0, x1, [res, #16]+        ldp     x0, x1, [acc+32]+        stp     x0, x1, [res, #32]+        ldp     x0, x1, [acc+48]+        stp     x0, x1, [res, #48]+        ldp     x0, x1, [acc+64]+        stp     x0, x1, [res, #64]+        ldp     x0, x1, [acc+80]+        stp     x0, x1, [res, #80]+        ldp     x0, x1, [acc+96]+        stp     x0, x1, [res, #96]+        ldp     x0, x1, [acc+112]+        stp     x0, x1, [res, #112]+        ldp     x0, x1, [acc+128]+        stp     x0, x1, [res, #128]+        ldp     x0, x1, [acc+144]+        stp     x0, x1, [res, #144]+        ldp     x0, x1, [acc+160]+        stp     x0, x1, [res, #160]+        ldp     x0, x1, [acc+176]+        stp     x0, x1, [res, #176]+        ldp     x0, x1, [acc+192]+        stp     x0, x1, [res, #192]+        ldr     x0, [acc+208]+        str     x0, [res, #208]++// Restore stack and registers and return++        CFI_INC_SP(NSPACE)+        CFI_POP2(x21,x30)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p521_jscalarmul_alt)++// Local copies of subroutines, complete clones at the moment except+// that we share multiplication and squaring between the point operations.++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_alt_bignum_mod_p521_9)++Lp521_jscalarmul_alt_bignum_mod_p521_9:+        CFI_START+        ldr     x12, [x1, #64]+        lsr     x2, x12, #9+        cmp     xzr, xzr+        ldp     x4, x5, [x1]+        adcs    xzr, x4, x2+        adcs    xzr, x5, xzr+        ldp     x6, x7, [x1, #16]+        and     x3, x6, x7+        adcs    xzr, x3, xzr+        ldp     x8, x9, [x1, #32]+        and     x3, x8, x9+        adcs    xzr, x3, xzr+        ldp     x10, x11, [x1, #48]+        and     x3, x10, x11+        adcs    xzr, x3, xzr+        orr     x3, x12, #0xfffffffffffffe00+        adcs    x3, x3, xzr+        adcs    x4, x4, x2+        adcs    x5, x5, xzr+        adcs    x6, x6, xzr+        adcs    x7, x7, xzr+        adcs    x8, x8, xzr+        adcs    x9, x9, xzr+        adcs    x10, x10, xzr+        adcs    x11, x11, xzr+        adc     x12, x12, xzr+        and     x12, x12, #0x1ff+        stp     x4, x5, [x0]+        stp     x6, x7, [x0, #16]+        stp     x8, x9, [x0, #32]+        stp     x10, x11, [x0, #48]+        str     x12, [x0, #64]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_alt_bignum_mod_p521_9)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_alt_bignum_mod_n521_9)++Lp521_jscalarmul_alt_bignum_mod_n521_9:+        CFI_START+        ldr     x14, [x1, #64]+        lsr     x15, x14, #9+        add     x15, x15, #1+        mov     x2, #39927+        movk    x2, #28359, lsl #16+        movk    x2, #18657, lsl #32+        movk    x2, #17552, lsl #48+        mul     x6, x2, x15+        mov     x3, #47185+        movk    x3, #30307, lsl #16+        movk    x3, #13895, lsl #32+        movk    x3, #50250, lsl #48+        mul     x7, x3, x15+        mov     x4, #23087+        movk    x4, #2294, lsl #16+        movk    x4, #65207, lsl #32+        movk    x4, #32819, lsl #48+        mul     x8, x4, x15+        mov     x5, #27028+        movk    x5, #16592, lsl #16+        movk    x5, #30844, lsl #32+        movk    x5, #44665, lsl #48+        mul     x9, x5, x15+        lsl     x10, x15, #2+        add     x10, x10, x15+        umulh   x13, x2, x15+        adds    x7, x7, x13+        umulh   x13, x3, x15+        adcs    x8, x8, x13+        umulh   x13, x4, x15+        adcs    x9, x9, x13+        umulh   x13, x5, x15+        adc     x10, x10, x13+        ldp     x12, x13, [x1]+        adds    x6, x6, x12+        adcs    x7, x7, x13+        ldp     x12, x13, [x1, #16]+        adcs    x8, x8, x12+        adcs    x9, x9, x13+        ldp     x13, x11, [x1, #32]+        adcs    x10, x10, x13+        adcs    x11, x11, xzr+        ldp     x12, x13, [x1, #48]+        adcs    x12, x12, xzr+        adcs    x13, x13, xzr+        orr     x14, x14, #0xfffffffffffffe00+        adcs    x14, x14, xzr+        csetm   x15, lo+        and     x2, x2, x15+        subs    x6, x6, x2+        and     x3, x3, x15+        sbcs    x7, x7, x3+        and     x4, x4, x15+        sbcs    x8, x8, x4+        and     x5, x5, x15+        sbcs    x9, x9, x5+        mov     x2, #5+        and     x2, x2, x15+        sbcs    x10, x10, x2+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbc     x14, x14, xzr+        and     x14, x14, #0x1ff+        stp     x6, x7, [x0]+        stp     x8, x9, [x0, #16]+        stp     x10, x11, [x0, #32]+        stp     x12, x13, [x0, #48]+        str     x14, [x0, #64]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_alt_bignum_mod_n521_9)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_alt_jadd)++Lp521_jscalarmul_alt_jadd:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x26)+        CFI_PUSH2(x27,x28)+        CFI_PUSH2(x29,x30)+        CFI_DEC_SP(576)+        mov     x27, x0+        mov     x28, x1+        mov     x29, x2+        mov     x0, sp+        add     x1, x28, #0x90+        CFI_BL(Lp521_jscalarmul_alt_sqr_p521)+        add     x0, sp, #0x168+        add     x1, x29, #0x90+        CFI_BL(Lp521_jscalarmul_alt_sqr_p521)+        add     x0, sp, #0x1f8+        add     x1, x29, #0x90+        add     x2, x28, #0x48+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        add     x0, sp, #0x48+        add     x1, x28, #0x90+        add     x2, x29, #0x48+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        add     x0, sp, #0x90+        mov     x1, sp+        add     x2, x29, #0x0+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        add     x0, sp, #0x120+        add     x1, sp, #0x168+        add     x2, x28, #0x0+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        add     x0, sp, #0x48+        mov     x1, sp+        add     x2, sp, #0x48+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        add     x0, sp, #0x1f8+        add     x1, sp, #0x168+        add     x2, sp, #0x1f8+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        add     x0, sp, #0x168+        add     x1, sp, #0x90+        add     x2, sp, #0x120+        CFI_BL(Lp521_jscalarmul_alt_sub_p521)+        add     x0, sp, #0x48+        add     x1, sp, #0x48+        add     x2, sp, #0x1f8+        CFI_BL(Lp521_jscalarmul_alt_sub_p521)+        add     x0, sp, #0xd8+        add     x1, sp, #0x168+        CFI_BL(Lp521_jscalarmul_alt_sqr_p521)+        mov     x0, sp+        add     x1, sp, #0x48+        CFI_BL(Lp521_jscalarmul_alt_sqr_p521)+        add     x0, sp, #0x120+        add     x1, sp, #0xd8+        add     x2, sp, #0x120+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        add     x0, sp, #0x90+        add     x1, sp, #0xd8+        add     x2, sp, #0x90+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        mov     x0, sp+        mov     x1, sp+        add     x2, sp, #0x120+        CFI_BL(Lp521_jscalarmul_alt_sub_p521)+        add     x0, sp, #0xd8+        add     x1, sp, #0x90+        add     x2, sp, #0x120+        CFI_BL(Lp521_jscalarmul_alt_sub_p521)+        add     x0, sp, #0x168+        add     x1, sp, #0x168+        add     x2, x28, #0x90+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        mov     x0, sp+        mov     x1, sp+        add     x2, sp, #0x90+        CFI_BL(Lp521_jscalarmul_alt_sub_p521)+        add     x0, sp, #0x120+        add     x1, sp, #0x120+        mov     x2, sp+        CFI_BL(Lp521_jscalarmul_alt_sub_p521)+        add     x0, sp, #0xd8+        add     x1, sp, #0xd8+        add     x2, sp, #0x1f8+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        add     x0, sp, #0x168+        add     x1, sp, #0x168+        add     x2, x29, #0x90+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        add     x0, sp, #0x120+        add     x1, sp, #0x48+        add     x2, sp, #0x120+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        add     x0, sp, #0x120+        add     x1, sp, #0x120+        add     x2, sp, #0xd8+        CFI_BL(Lp521_jscalarmul_alt_sub_p521)+        ldp     x0, x1, [x28, #144]+        ldp     x2, x3, [x28, #160]+        ldp     x4, x5, [x28, #176]+        ldp     x6, x7, [x28, #192]+        ldr     x8, [x28, #208]+        orr     x20, x0, x1+        orr     x21, x2, x3+        orr     x22, x4, x5+        orr     x23, x6, x7+        orr     x20, x20, x21+        orr     x22, x22, x23+        orr     x20, x20, x8+        orr     x20, x20, x22+        cmp     x20, xzr+        cset    x20, ne+        ldp     x10, x11, [x29, #144]+        ldp     x12, x13, [x29, #160]+        ldp     x14, x15, [x29, #176]+        ldp     x16, x17, [x29, #192]+        ldr     x19, [x29, #208]+        orr     x21, x10, x11+        orr     x22, x12, x13+        orr     x23, x14, x15+        orr     x24, x16, x17+        orr     x21, x21, x22+        orr     x23, x23, x24+        orr     x21, x21, x19+        orr     x21, x21, x23+        csel    x0, x0, x10, ne+        csel    x1, x1, x11, ne+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        csel    x4, x4, x14, ne+        csel    x5, x5, x15, ne+        csel    x6, x6, x16, ne+        csel    x7, x7, x17, ne+        csel    x8, x8, x19, ne+        cmp     x21, xzr+        cset    x21, ne+        cmp     x21, x20+        ldp     x10, x11, [sp, #360]+        ldp     x12, x13, [sp, #376]+        ldp     x14, x15, [sp, #392]+        ldp     x16, x17, [sp, #408]+        ldr     x19, [sp, #424]+        csel    x0, x0, x10, ne+        csel    x1, x1, x11, ne+        csel    x2, x2, x12, ne+        csel    x3, x3, x13, ne+        csel    x4, x4, x14, ne+        csel    x5, x5, x15, ne+        csel    x6, x6, x16, ne+        csel    x7, x7, x17, ne+        csel    x8, x8, x19, ne+        stp     x0, x1, [sp, #360]+        stp     x2, x3, [sp, #376]+        stp     x4, x5, [sp, #392]+        stp     x6, x7, [sp, #408]+        str     x8, [sp, #424]+        ldp     x20, x21, [x28]+        ldp     x0, x1, [sp]+        csel    x0, x20, x0, cc+        csel    x1, x21, x1, cc+        ldp     x20, x21, [x29]+        csel    x0, x20, x0, hi+        csel    x1, x21, x1, hi+        ldp     x20, x21, [x28, #16]+        ldp     x2, x3, [sp, #16]+        csel    x2, x20, x2, cc+        csel    x3, x21, x3, cc+        ldp     x20, x21, [x29, #16]+        csel    x2, x20, x2, hi+        csel    x3, x21, x3, hi+        ldp     x20, x21, [x28, #32]+        ldp     x4, x5, [sp, #32]+        csel    x4, x20, x4, cc+        csel    x5, x21, x5, cc+        ldp     x20, x21, [x29, #32]+        csel    x4, x20, x4, hi+        csel    x5, x21, x5, hi+        ldp     x20, x21, [x28, #48]+        ldp     x6, x7, [sp, #48]+        csel    x6, x20, x6, cc+        csel    x7, x21, x7, cc+        ldp     x20, x21, [x29, #48]+        csel    x6, x20, x6, hi+        csel    x7, x21, x7, hi+        ldr     x20, [x28, #64]+        ldr     x8, [sp, #64]+        csel    x8, x20, x8, cc+        ldr     x21, [x29, #64]+        csel    x8, x21, x8, hi+        ldp     x20, x21, [x28, #72]+        ldp     x10, x11, [sp, #288]+        csel    x10, x20, x10, cc+        csel    x11, x21, x11, cc+        ldp     x20, x21, [x29, #72]+        csel    x10, x20, x10, hi+        csel    x11, x21, x11, hi+        ldp     x20, x21, [x28, #88]+        ldp     x12, x13, [sp, #304]+        csel    x12, x20, x12, cc+        csel    x13, x21, x13, cc+        ldp     x20, x21, [x29, #88]+        csel    x12, x20, x12, hi+        csel    x13, x21, x13, hi+        ldp     x20, x21, [x28, #104]+        ldp     x14, x15, [sp, #320]+        csel    x14, x20, x14, cc+        csel    x15, x21, x15, cc+        ldp     x20, x21, [x29, #104]+        csel    x14, x20, x14, hi+        csel    x15, x21, x15, hi+        ldp     x20, x21, [x28, #120]+        ldp     x16, x17, [sp, #336]+        csel    x16, x20, x16, cc+        csel    x17, x21, x17, cc+        ldp     x20, x21, [x29, #120]+        csel    x16, x20, x16, hi+        csel    x17, x21, x17, hi+        ldr     x20, [x28, #136]+        ldr     x19, [sp, #352]+        csel    x19, x20, x19, cc+        ldr     x21, [x29, #136]+        csel    x19, x21, x19, hi+        stp     x0, x1, [x27]+        stp     x2, x3, [x27, #16]+        stp     x4, x5, [x27, #32]+        stp     x6, x7, [x27, #48]+        str     x8, [x27, #64]+        ldp     x0, x1, [sp, #360]+        ldp     x2, x3, [sp, #376]+        ldp     x4, x5, [sp, #392]+        ldp     x6, x7, [sp, #408]+        ldr     x8, [sp, #424]+        stp     x10, x11, [x27, #72]+        stp     x12, x13, [x27, #88]+        stp     x14, x15, [x27, #104]+        stp     x16, x17, [x27, #120]+        str     x19, [x27, #136]+        stp     x0, x1, [x27, #144]+        stp     x2, x3, [x27, #160]+        stp     x4, x5, [x27, #176]+        stp     x6, x7, [x27, #192]+        str     x8, [x27, #208]+        CFI_INC_SP(576)+        CFI_POP2(x29,x30)+        CFI_POP2(x27,x28)+        CFI_POP2(x25,x26)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_alt_jadd)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_alt_jdouble)++Lp521_jscalarmul_alt_jdouble:+        CFI_START+        CFI_PUSH2(x19,x20)+        CFI_PUSH2(x21,x22)+        CFI_PUSH2(x23,x24)+        CFI_PUSH2(x25,x26)+        CFI_PUSH2(x27,x28)+        CFI_PUSH2(x29,x30)+        CFI_DEC_SP(512)+        mov     x27, x0+        mov     x28, x1+        mov     x0, sp+        add     x1, x28, #0x90+        CFI_BL(Lp521_jscalarmul_alt_sqr_p521)+        add     x0, sp, #0x48+        add     x1, x28, #0x48+        CFI_BL(Lp521_jscalarmul_alt_sqr_p521)+        ldp     x5, x6, [x28]+        ldp     x4, x3, [sp]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [x28, #16]+        ldp     x4, x3, [sp, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [x28, #32]+        ldp     x4, x3, [sp, #32]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        ldp     x11, x12, [x28, #48]+        ldp     x4, x3, [sp, #48]+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        ldr     x13, [x28, #64]+        ldr     x4, [sp, #64]+        sbcs    x13, x13, x4+        sbcs    x5, x5, xzr+        sbcs    x6, x6, xzr+        sbcs    x7, x7, xzr+        sbcs    x8, x8, xzr+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        and     x13, x13, #0x1ff+        stp     x5, x6, [sp, #216]+        stp     x7, x8, [sp, #232]+        stp     x9, x10, [sp, #248]+        stp     x11, x12, [sp, #264]+        str     x13, [sp, #280]+        cmp     xzr, xzr+        ldp     x5, x6, [x28]+        ldp     x4, x3, [sp]+        adcs    x5, x5, x4+        adcs    x6, x6, x3+        ldp     x7, x8, [x28, #16]+        ldp     x4, x3, [sp, #16]+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        ldp     x9, x10, [x28, #32]+        ldp     x4, x3, [sp, #32]+        adcs    x9, x9, x4+        adcs    x10, x10, x3+        ldp     x11, x12, [x28, #48]+        ldp     x4, x3, [sp, #48]+        adcs    x11, x11, x4+        adcs    x12, x12, x3+        ldr     x13, [x28, #64]+        ldr     x4, [sp, #64]+        adc     x13, x13, x4+        subs    x4, x13, #0x200+        csetm   x4, cs+        sbcs    x5, x5, xzr+        and     x4, x4, #0x200+        sbcs    x6, x6, xzr+        sbcs    x7, x7, xzr+        sbcs    x8, x8, xzr+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, x4+        stp     x5, x6, [sp, #144]+        stp     x7, x8, [sp, #160]+        stp     x9, x10, [sp, #176]+        stp     x11, x12, [sp, #192]+        str     x13, [sp, #208]+        add     x0, sp, #0xd8+        add     x1, sp, #0x90+        add     x2, sp, #0xd8+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        cmp     xzr, xzr+        ldp     x5, x6, [x28, #72]+        ldp     x4, x3, [x28, #144]+        adcs    x5, x5, x4+        adcs    x6, x6, x3+        ldp     x7, x8, [x28, #88]+        ldp     x4, x3, [x28, #160]+        adcs    x7, x7, x4+        adcs    x8, x8, x3+        ldp     x9, x10, [x28, #104]+        ldp     x4, x3, [x28, #176]+        adcs    x9, x9, x4+        adcs    x10, x10, x3+        ldp     x11, x12, [x28, #120]+        ldp     x4, x3, [x28, #192]+        adcs    x11, x11, x4+        adcs    x12, x12, x3+        ldr     x13, [x28, #136]+        ldr     x4, [x28, #208]+        adc     x13, x13, x4+        subs    x4, x13, #0x200+        csetm   x4, cs+        sbcs    x5, x5, xzr+        and     x4, x4, #0x200+        sbcs    x6, x6, xzr+        sbcs    x7, x7, xzr+        sbcs    x8, x8, xzr+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, x4+        stp     x5, x6, [sp, #144]+        stp     x7, x8, [sp, #160]+        stp     x9, x10, [sp, #176]+        stp     x11, x12, [sp, #192]+        str     x13, [sp, #208]+        add     x0, sp, #0x120+        add     x1, x28, #0x0+        add     x2, sp, #0x48+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        add     x0, sp, #0x168+        add     x1, sp, #0xd8+        CFI_BL(Lp521_jscalarmul_alt_sqr_p521)+        add     x0, sp, #0x90+        add     x1, sp, #0x90+        CFI_BL(Lp521_jscalarmul_alt_sqr_p521)+        ldp     x6, x7, [sp, #288]+        mov     x1, #0xc+        mul     x3, x1, x6+        mul     x4, x1, x7+        umulh   x6, x1, x6+        adds    x4, x4, x6+        umulh   x7, x1, x7+        ldp     x8, x9, [sp, #304]+        mul     x5, x1, x8+        mul     x6, x1, x9+        umulh   x8, x1, x8+        adcs    x5, x5, x7+        umulh   x9, x1, x9+        adcs    x6, x6, x8+        ldp     x10, x11, [sp, #320]+        mul     x7, x1, x10+        mul     x8, x1, x11+        umulh   x10, x1, x10+        adcs    x7, x7, x9+        umulh   x11, x1, x11+        adcs    x8, x8, x10+        ldp     x12, x13, [sp, #336]+        mul     x9, x1, x12+        mul     x10, x1, x13+        umulh   x12, x1, x12+        adcs    x9, x9, x11+        umulh   x13, x1, x13+        adcs    x10, x10, x12+        ldr     x14, [sp, #352]+        mul     x11, x1, x14+        adc     x11, x11, x13+        mov     x1, #0x9+        ldp     x20, x21, [sp, #360]+        mvn     x20, x20+        mul     x0, x1, x20+        umulh   x20, x1, x20+        adds    x3, x3, x0+        mvn     x21, x21+        mul     x0, x1, x21+        umulh   x21, x1, x21+        adcs    x4, x4, x0+        ldp     x22, x23, [sp, #376]+        mvn     x22, x22+        mul     x0, x1, x22+        umulh   x22, x1, x22+        adcs    x5, x5, x0+        mvn     x23, x23+        mul     x0, x1, x23+        umulh   x23, x1, x23+        adcs    x6, x6, x0+        ldp     x17, x19, [sp, #392]+        mvn     x17, x17+        mul     x0, x1, x17+        umulh   x17, x1, x17+        adcs    x7, x7, x0+        mvn     x19, x19+        mul     x0, x1, x19+        umulh   x19, x1, x19+        adcs    x8, x8, x0+        ldp     x2, x16, [sp, #408]+        mvn     x2, x2+        mul     x0, x1, x2+        umulh   x2, x1, x2+        adcs    x9, x9, x0+        mvn     x16, x16+        mul     x0, x1, x16+        umulh   x16, x1, x16+        adcs    x10, x10, x0+        ldr     x0, [sp, #424]+        eor     x0, x0, #0x1ff+        mul     x0, x1, x0+        adc     x11, x11, x0+        adds    x4, x4, x20+        adcs    x5, x5, x21+        and     x15, x4, x5+        adcs    x6, x6, x22+        and     x15, x15, x6+        adcs    x7, x7, x23+        and     x15, x15, x7+        adcs    x8, x8, x17+        and     x15, x15, x8+        adcs    x9, x9, x19+        and     x15, x15, x9+        adcs    x10, x10, x2+        and     x15, x15, x10+        adc     x11, x11, x16+        lsr     x12, x11, #9+        orr     x11, x11, #0xfffffffffffffe00+        cmp     xzr, xzr+        adcs    xzr, x3, x12+        adcs    xzr, x15, xzr+        adcs    xzr, x11, xzr+        adcs    x3, x3, x12+        adcs    x4, x4, xzr+        adcs    x5, x5, xzr+        adcs    x6, x6, xzr+        adcs    x7, x7, xzr+        adcs    x8, x8, xzr+        adcs    x9, x9, xzr+        adcs    x10, x10, xzr+        adc     x11, x11, xzr+        and     x11, x11, #0x1ff+        stp     x3, x4, [sp, #360]+        stp     x5, x6, [sp, #376]+        stp     x7, x8, [sp, #392]+        stp     x9, x10, [sp, #408]+        str     x11, [sp, #424]+        ldp     x5, x6, [sp, #144]+        ldp     x4, x3, [sp]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #160]+        ldp     x4, x3, [sp, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #176]+        ldp     x4, x3, [sp, #32]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        ldp     x11, x12, [sp, #192]+        ldp     x4, x3, [sp, #48]+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        ldr     x13, [sp, #208]+        ldr     x4, [sp, #64]+        sbcs    x13, x13, x4+        sbcs    x5, x5, xzr+        sbcs    x6, x6, xzr+        sbcs    x7, x7, xzr+        sbcs    x8, x8, xzr+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        and     x13, x13, #0x1ff+        stp     x5, x6, [sp, #144]+        stp     x7, x8, [sp, #160]+        stp     x9, x10, [sp, #176]+        stp     x11, x12, [sp, #192]+        str     x13, [sp, #208]+        mov     x0, sp+        add     x1, sp, #0x48+        CFI_BL(Lp521_jscalarmul_alt_sqr_p521)+        add     x0, sp, #0xd8+        add     x1, sp, #0x168+        add     x2, sp, #0xd8+        CFI_BL(Lp521_jscalarmul_alt_mul_p521)+        ldp     x5, x6, [sp, #144]+        ldp     x4, x3, [sp, #72]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [sp, #160]+        ldp     x4, x3, [sp, #88]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [sp, #176]+        ldp     x4, x3, [sp, #104]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        ldp     x11, x12, [sp, #192]+        ldp     x4, x3, [sp, #120]+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        ldr     x13, [sp, #208]+        ldr     x4, [sp, #136]+        sbcs    x13, x13, x4+        sbcs    x5, x5, xzr+        sbcs    x6, x6, xzr+        sbcs    x7, x7, xzr+        sbcs    x8, x8, xzr+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        and     x13, x13, #0x1ff+        stp     x5, x6, [x27, #144]+        stp     x7, x8, [x27, #160]+        stp     x9, x10, [x27, #176]+        stp     x11, x12, [x27, #192]+        str     x13, [x27, #208]+        ldp     x6, x7, [sp, #288]+        lsl     x3, x6, #2+        extr    x4, x7, x6, #62+        ldp     x8, x9, [sp, #304]+        extr    x5, x8, x7, #62+        extr    x6, x9, x8, #62+        ldp     x10, x11, [sp, #320]+        extr    x7, x10, x9, #62+        extr    x8, x11, x10, #62+        ldp     x12, x13, [sp, #336]+        extr    x9, x12, x11, #62+        extr    x10, x13, x12, #62+        ldr     x14, [sp, #352]+        extr    x11, x14, x13, #62+        ldp     x0, x1, [sp, #360]+        mvn     x0, x0+        adds    x3, x3, x0+        sbcs    x4, x4, x1+        ldp     x0, x1, [sp, #376]+        sbcs    x5, x5, x0+        and     x15, x4, x5+        sbcs    x6, x6, x1+        and     x15, x15, x6+        ldp     x0, x1, [sp, #392]+        sbcs    x7, x7, x0+        and     x15, x15, x7+        sbcs    x8, x8, x1+        and     x15, x15, x8+        ldp     x0, x1, [sp, #408]+        sbcs    x9, x9, x0+        and     x15, x15, x9+        sbcs    x10, x10, x1+        and     x15, x15, x10+        ldr     x0, [sp, #424]+        eor     x0, x0, #0x1ff+        adc     x11, x11, x0+        lsr     x12, x11, #9+        orr     x11, x11, #0xfffffffffffffe00+        cmp     xzr, xzr+        adcs    xzr, x3, x12+        adcs    xzr, x15, xzr+        adcs    xzr, x11, xzr+        adcs    x3, x3, x12+        adcs    x4, x4, xzr+        adcs    x5, x5, xzr+        adcs    x6, x6, xzr+        adcs    x7, x7, xzr+        adcs    x8, x8, xzr+        adcs    x9, x9, xzr+        adcs    x10, x10, xzr+        adc     x11, x11, xzr+        and     x11, x11, #0x1ff+        stp     x3, x4, [x27]+        stp     x5, x6, [x27, #16]+        stp     x7, x8, [x27, #32]+        stp     x9, x10, [x27, #48]+        str     x11, [x27, #64]+        ldp     x6, x7, [sp, #216]+        lsl     x3, x6, #1+        adds    x3, x3, x6+        extr    x4, x7, x6, #63+        adcs    x4, x4, x7+        ldp     x8, x9, [sp, #232]+        extr    x5, x8, x7, #63+        adcs    x5, x5, x8+        extr    x6, x9, x8, #63+        adcs    x6, x6, x9+        ldp     x10, x11, [sp, #248]+        extr    x7, x10, x9, #63+        adcs    x7, x7, x10+        extr    x8, x11, x10, #63+        adcs    x8, x8, x11+        ldp     x12, x13, [sp, #264]+        extr    x9, x12, x11, #63+        adcs    x9, x9, x12+        extr    x10, x13, x12, #63+        adcs    x10, x10, x13+        ldr     x14, [sp, #280]+        extr    x11, x14, x13, #63+        adc     x11, x11, x14+        ldp     x20, x21, [sp]+        mvn     x20, x20+        lsl     x0, x20, #3+        adds    x3, x3, x0+        mvn     x21, x21+        extr    x0, x21, x20, #61+        adcs    x4, x4, x0+        ldp     x22, x23, [sp, #16]+        mvn     x22, x22+        extr    x0, x22, x21, #61+        adcs    x5, x5, x0+        and     x15, x4, x5+        mvn     x23, x23+        extr    x0, x23, x22, #61+        adcs    x6, x6, x0+        and     x15, x15, x6+        ldp     x20, x21, [sp, #32]+        mvn     x20, x20+        extr    x0, x20, x23, #61+        adcs    x7, x7, x0+        and     x15, x15, x7+        mvn     x21, x21+        extr    x0, x21, x20, #61+        adcs    x8, x8, x0+        and     x15, x15, x8+        ldp     x22, x23, [sp, #48]+        mvn     x22, x22+        extr    x0, x22, x21, #61+        adcs    x9, x9, x0+        and     x15, x15, x9+        mvn     x23, x23+        extr    x0, x23, x22, #61+        adcs    x10, x10, x0+        and     x15, x15, x10+        ldr     x0, [sp, #64]+        eor     x0, x0, #0x1ff+        extr    x0, x0, x23, #61+        adc     x11, x11, x0+        lsr     x12, x11, #9+        orr     x11, x11, #0xfffffffffffffe00+        cmp     xzr, xzr+        adcs    xzr, x3, x12+        adcs    xzr, x15, xzr+        adcs    xzr, x11, xzr+        adcs    x3, x3, x12+        adcs    x4, x4, xzr+        adcs    x5, x5, xzr+        adcs    x6, x6, xzr+        adcs    x7, x7, xzr+        adcs    x8, x8, xzr+        adcs    x9, x9, xzr+        adcs    x10, x10, xzr+        adc     x11, x11, xzr+        and     x11, x11, #0x1ff+        stp     x3, x4, [x27, #72]+        stp     x5, x6, [x27, #88]+        stp     x7, x8, [x27, #104]+        stp     x9, x10, [x27, #120]+        str     x11, [x27, #136]+        CFI_INC_SP(512)+        CFI_POP2(x29,x30)+        CFI_POP2(x27,x28)+        CFI_POP2(x25,x26)+        CFI_POP2(x23,x24)+        CFI_POP2(x21,x22)+        CFI_POP2(x19,x20)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_alt_jdouble)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_alt_mul_p521)++Lp521_jscalarmul_alt_mul_p521:+        CFI_START+        ldp     x3, x4, [x1]+        ldp     x5, x6, [x2]+        mul     x15, x3, x5+        umulh   x16, x3, x5+        mul     x14, x3, x6+        umulh   x17, x3, x6+        adds    x16, x16, x14+        ldp     x7, x8, [x2, #16]+        mul     x14, x3, x7+        umulh   x19, x3, x7+        adcs    x17, x17, x14+        mul     x14, x3, x8+        umulh   x20, x3, x8+        adcs    x19, x19, x14+        ldp     x9, x10, [x2, #32]+        mul     x14, x3, x9+        umulh   x21, x3, x9+        adcs    x20, x20, x14+        mul     x14, x3, x10+        umulh   x22, x3, x10+        adcs    x21, x21, x14+        ldp     x11, x12, [x2, #48]+        mul     x14, x3, x11+        umulh   x23, x3, x11+        adcs    x22, x22, x14+        ldr     x13, [x2, #64]+        mul     x14, x3, x12+        umulh   x24, x3, x12+        adcs    x23, x23, x14+        mul     x14, x3, x13+        umulh   x25, x3, x13+        adcs    x24, x24, x14+        adc     x25, x25, xzr+        mul     x14, x4, x5+        adds    x16, x16, x14+        mul     x14, x4, x6+        adcs    x17, x17, x14+        mul     x14, x4, x7+        adcs    x19, x19, x14+        mul     x14, x4, x8+        adcs    x20, x20, x14+        mul     x14, x4, x9+        adcs    x21, x21, x14+        mul     x14, x4, x10+        adcs    x22, x22, x14+        mul     x14, x4, x11+        adcs    x23, x23, x14+        mul     x14, x4, x12+        adcs    x24, x24, x14+        mul     x14, x4, x13+        adcs    x25, x25, x14+        cset    x26, cs+        umulh   x14, x4, x5+        adds    x17, x17, x14+        umulh   x14, x4, x6+        adcs    x19, x19, x14+        umulh   x14, x4, x7+        adcs    x20, x20, x14+        umulh   x14, x4, x8+        adcs    x21, x21, x14+        umulh   x14, x4, x9+        adcs    x22, x22, x14+        umulh   x14, x4, x10+        adcs    x23, x23, x14+        umulh   x14, x4, x11+        adcs    x24, x24, x14+        umulh   x14, x4, x12+        adcs    x25, x25, x14+        umulh   x14, x4, x13+        adc     x26, x26, x14+        stp     x15, x16, [sp, #432]+        ldp     x3, x4, [x1, #16]+        mul     x14, x3, x5+        adds    x17, x17, x14+        mul     x14, x3, x6+        adcs    x19, x19, x14+        mul     x14, x3, x7+        adcs    x20, x20, x14+        mul     x14, x3, x8+        adcs    x21, x21, x14+        mul     x14, x3, x9+        adcs    x22, x22, x14+        mul     x14, x3, x10+        adcs    x23, x23, x14+        mul     x14, x3, x11+        adcs    x24, x24, x14+        mul     x14, x3, x12+        adcs    x25, x25, x14+        mul     x14, x3, x13+        adcs    x26, x26, x14+        cset    x15, cs+        umulh   x14, x3, x5+        adds    x19, x19, x14+        umulh   x14, x3, x6+        adcs    x20, x20, x14+        umulh   x14, x3, x7+        adcs    x21, x21, x14+        umulh   x14, x3, x8+        adcs    x22, x22, x14+        umulh   x14, x3, x9+        adcs    x23, x23, x14+        umulh   x14, x3, x10+        adcs    x24, x24, x14+        umulh   x14, x3, x11+        adcs    x25, x25, x14+        umulh   x14, x3, x12+        adcs    x26, x26, x14+        umulh   x14, x3, x13+        adc     x15, x15, x14+        mul     x14, x4, x5+        adds    x19, x19, x14+        mul     x14, x4, x6+        adcs    x20, x20, x14+        mul     x14, x4, x7+        adcs    x21, x21, x14+        mul     x14, x4, x8+        adcs    x22, x22, x14+        mul     x14, x4, x9+        adcs    x23, x23, x14+        mul     x14, x4, x10+        adcs    x24, x24, x14+        mul     x14, x4, x11+        adcs    x25, x25, x14+        mul     x14, x4, x12+        adcs    x26, x26, x14+        mul     x14, x4, x13+        adcs    x15, x15, x14+        cset    x16, cs+        umulh   x14, x4, x5+        adds    x20, x20, x14+        umulh   x14, x4, x6+        adcs    x21, x21, x14+        umulh   x14, x4, x7+        adcs    x22, x22, x14+        umulh   x14, x4, x8+        adcs    x23, x23, x14+        umulh   x14, x4, x9+        adcs    x24, x24, x14+        umulh   x14, x4, x10+        adcs    x25, x25, x14+        umulh   x14, x4, x11+        adcs    x26, x26, x14+        umulh   x14, x4, x12+        adcs    x15, x15, x14+        umulh   x14, x4, x13+        adc     x16, x16, x14+        stp     x17, x19, [sp, #448]+        ldp     x3, x4, [x1, #32]+        mul     x14, x3, x5+        adds    x20, x20, x14+        mul     x14, x3, x6+        adcs    x21, x21, x14+        mul     x14, x3, x7+        adcs    x22, x22, x14+        mul     x14, x3, x8+        adcs    x23, x23, x14+        mul     x14, x3, x9+        adcs    x24, x24, x14+        mul     x14, x3, x10+        adcs    x25, x25, x14+        mul     x14, x3, x11+        adcs    x26, x26, x14+        mul     x14, x3, x12+        adcs    x15, x15, x14+        mul     x14, x3, x13+        adcs    x16, x16, x14+        cset    x17, cs+        umulh   x14, x3, x5+        adds    x21, x21, x14+        umulh   x14, x3, x6+        adcs    x22, x22, x14+        umulh   x14, x3, x7+        adcs    x23, x23, x14+        umulh   x14, x3, x8+        adcs    x24, x24, x14+        umulh   x14, x3, x9+        adcs    x25, x25, x14+        umulh   x14, x3, x10+        adcs    x26, x26, x14+        umulh   x14, x3, x11+        adcs    x15, x15, x14+        umulh   x14, x3, x12+        adcs    x16, x16, x14+        umulh   x14, x3, x13+        adc     x17, x17, x14+        mul     x14, x4, x5+        adds    x21, x21, x14+        mul     x14, x4, x6+        adcs    x22, x22, x14+        mul     x14, x4, x7+        adcs    x23, x23, x14+        mul     x14, x4, x8+        adcs    x24, x24, x14+        mul     x14, x4, x9+        adcs    x25, x25, x14+        mul     x14, x4, x10+        adcs    x26, x26, x14+        mul     x14, x4, x11+        adcs    x15, x15, x14+        mul     x14, x4, x12+        adcs    x16, x16, x14+        mul     x14, x4, x13+        adcs    x17, x17, x14+        cset    x19, cs+        umulh   x14, x4, x5+        adds    x22, x22, x14+        umulh   x14, x4, x6+        adcs    x23, x23, x14+        umulh   x14, x4, x7+        adcs    x24, x24, x14+        umulh   x14, x4, x8+        adcs    x25, x25, x14+        umulh   x14, x4, x9+        adcs    x26, x26, x14+        umulh   x14, x4, x10+        adcs    x15, x15, x14+        umulh   x14, x4, x11+        adcs    x16, x16, x14+        umulh   x14, x4, x12+        adcs    x17, x17, x14+        umulh   x14, x4, x13+        adc     x19, x19, x14+        stp     x20, x21, [sp, #464]+        ldp     x3, x4, [x1, #48]+        mul     x14, x3, x5+        adds    x22, x22, x14+        mul     x14, x3, x6+        adcs    x23, x23, x14+        mul     x14, x3, x7+        adcs    x24, x24, x14+        mul     x14, x3, x8+        adcs    x25, x25, x14+        mul     x14, x3, x9+        adcs    x26, x26, x14+        mul     x14, x3, x10+        adcs    x15, x15, x14+        mul     x14, x3, x11+        adcs    x16, x16, x14+        mul     x14, x3, x12+        adcs    x17, x17, x14+        mul     x14, x3, x13+        adcs    x19, x19, x14+        cset    x20, cs+        umulh   x14, x3, x5+        adds    x23, x23, x14+        umulh   x14, x3, x6+        adcs    x24, x24, x14+        umulh   x14, x3, x7+        adcs    x25, x25, x14+        umulh   x14, x3, x8+        adcs    x26, x26, x14+        umulh   x14, x3, x9+        adcs    x15, x15, x14+        umulh   x14, x3, x10+        adcs    x16, x16, x14+        umulh   x14, x3, x11+        adcs    x17, x17, x14+        umulh   x14, x3, x12+        adcs    x19, x19, x14+        umulh   x14, x3, x13+        adc     x20, x20, x14+        mul     x14, x4, x5+        adds    x23, x23, x14+        mul     x14, x4, x6+        adcs    x24, x24, x14+        mul     x14, x4, x7+        adcs    x25, x25, x14+        mul     x14, x4, x8+        adcs    x26, x26, x14+        mul     x14, x4, x9+        adcs    x15, x15, x14+        mul     x14, x4, x10+        adcs    x16, x16, x14+        mul     x14, x4, x11+        adcs    x17, x17, x14+        mul     x14, x4, x12+        adcs    x19, x19, x14+        mul     x14, x4, x13+        adcs    x20, x20, x14+        cset    x21, cs+        umulh   x14, x4, x5+        adds    x24, x24, x14+        umulh   x14, x4, x6+        adcs    x25, x25, x14+        umulh   x14, x4, x7+        adcs    x26, x26, x14+        umulh   x14, x4, x8+        adcs    x15, x15, x14+        umulh   x14, x4, x9+        adcs    x16, x16, x14+        umulh   x14, x4, x10+        adcs    x17, x17, x14+        umulh   x14, x4, x11+        adcs    x19, x19, x14+        umulh   x14, x4, x12+        adcs    x20, x20, x14+        umulh   x14, x4, x13+        adc     x21, x21, x14+        stp     x22, x23, [sp, #480]+        ldr     x3, [x1, #64]+        mul     x14, x3, x5+        adds    x24, x24, x14+        mul     x14, x3, x6+        adcs    x25, x25, x14+        mul     x14, x3, x7+        adcs    x26, x26, x14+        mul     x14, x3, x8+        adcs    x15, x15, x14+        mul     x14, x3, x9+        adcs    x16, x16, x14+        mul     x14, x3, x10+        adcs    x17, x17, x14+        mul     x14, x3, x11+        adcs    x19, x19, x14+        mul     x14, x3, x12+        adcs    x20, x20, x14+        mul     x14, x3, x13+        adc     x21, x21, x14+        umulh   x14, x3, x5+        adds    x25, x25, x14+        umulh   x14, x3, x6+        adcs    x26, x26, x14+        umulh   x14, x3, x7+        adcs    x15, x15, x14+        umulh   x14, x3, x8+        adcs    x16, x16, x14+        umulh   x14, x3, x9+        adcs    x17, x17, x14+        umulh   x14, x3, x10+        adcs    x19, x19, x14+        umulh   x14, x3, x11+        adcs    x20, x20, x14+        umulh   x14, x3, x12+        adc     x21, x21, x14+        cmp     xzr, xzr+        ldp     x5, x6, [sp, #432]+        extr    x14, x25, x24, #9+        adcs    x5, x5, x14+        extr    x14, x26, x25, #9+        adcs    x6, x6, x14+        ldp     x7, x8, [sp, #448]+        extr    x14, x15, x26, #9+        adcs    x7, x7, x14+        extr    x14, x16, x15, #9+        adcs    x8, x8, x14+        ldp     x9, x10, [sp, #464]+        extr    x14, x17, x16, #9+        adcs    x9, x9, x14+        extr    x14, x19, x17, #9+        adcs    x10, x10, x14+        ldp     x11, x12, [sp, #480]+        extr    x14, x20, x19, #9+        adcs    x11, x11, x14+        extr    x14, x21, x20, #9+        adcs    x12, x12, x14+        orr     x13, x24, #0xfffffffffffffe00+        lsr     x14, x21, #9+        adcs    x13, x13, x14+        sbcs    x5, x5, xzr+        sbcs    x6, x6, xzr+        sbcs    x7, x7, xzr+        sbcs    x8, x8, xzr+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbc     x13, x13, xzr+        and     x13, x13, #0x1ff+        stp     x5, x6, [x0]+        stp     x7, x8, [x0, #16]+        stp     x9, x10, [x0, #32]+        stp     x11, x12, [x0, #48]+        str     x13, [x0, #64]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_alt_mul_p521)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_alt_sqr_p521)++Lp521_jscalarmul_alt_sqr_p521:+        CFI_START+        ldp     x2, x3, [x1]+        mul     x11, x2, x3+        umulh   x12, x2, x3+        ldp     x4, x5, [x1, #16]+        mul     x10, x2, x4+        umulh   x13, x2, x4+        adds    x12, x12, x10+        ldp     x6, x7, [x1, #32]+        mul     x10, x2, x5+        umulh   x14, x2, x5+        adcs    x13, x13, x10+        ldp     x8, x9, [x1, #48]+        mul     x10, x2, x6+        umulh   x15, x2, x6+        adcs    x14, x14, x10+        mul     x10, x2, x7+        umulh   x16, x2, x7+        adcs    x15, x15, x10+        mul     x10, x2, x8+        umulh   x17, x2, x8+        adcs    x16, x16, x10+        mul     x10, x2, x9+        umulh   x19, x2, x9+        adcs    x17, x17, x10+        adc     x19, x19, xzr+        mul     x10, x3, x4+        adds    x13, x13, x10+        mul     x10, x3, x5+        adcs    x14, x14, x10+        mul     x10, x3, x6+        adcs    x15, x15, x10+        mul     x10, x3, x7+        adcs    x16, x16, x10+        mul     x10, x3, x8+        adcs    x17, x17, x10+        mul     x10, x3, x9+        adcs    x19, x19, x10+        cset    x20, cs+        umulh   x10, x3, x4+        adds    x14, x14, x10+        umulh   x10, x3, x5+        adcs    x15, x15, x10+        umulh   x10, x3, x6+        adcs    x16, x16, x10+        umulh   x10, x3, x7+        adcs    x17, x17, x10+        umulh   x10, x3, x8+        adcs    x19, x19, x10+        umulh   x10, x3, x9+        adc     x20, x20, x10+        mul     x10, x6, x7+        umulh   x21, x6, x7+        adds    x20, x20, x10+        adc     x21, x21, xzr+        mul     x10, x4, x5+        adds    x15, x15, x10+        mul     x10, x4, x6+        adcs    x16, x16, x10+        mul     x10, x4, x7+        adcs    x17, x17, x10+        mul     x10, x4, x8+        adcs    x19, x19, x10+        mul     x10, x4, x9+        adcs    x20, x20, x10+        mul     x10, x6, x8+        adcs    x21, x21, x10+        cset    x22, cs+        umulh   x10, x4, x5+        adds    x16, x16, x10+        umulh   x10, x4, x6+        adcs    x17, x17, x10+        umulh   x10, x4, x7+        adcs    x19, x19, x10+        umulh   x10, x4, x8+        adcs    x20, x20, x10+        umulh   x10, x4, x9+        adcs    x21, x21, x10+        umulh   x10, x6, x8+        adc     x22, x22, x10+        mul     x10, x7, x8+        umulh   x23, x7, x8+        adds    x22, x22, x10+        adc     x23, x23, xzr+        mul     x10, x5, x6+        adds    x17, x17, x10+        mul     x10, x5, x7+        adcs    x19, x19, x10+        mul     x10, x5, x8+        adcs    x20, x20, x10+        mul     x10, x5, x9+        adcs    x21, x21, x10+        mul     x10, x6, x9+        adcs    x22, x22, x10+        mul     x10, x7, x9+        adcs    x23, x23, x10+        cset    x24, cs+        umulh   x10, x5, x6+        adds    x19, x19, x10+        umulh   x10, x5, x7+        adcs    x20, x20, x10+        umulh   x10, x5, x8+        adcs    x21, x21, x10+        umulh   x10, x5, x9+        adcs    x22, x22, x10+        umulh   x10, x6, x9+        adcs    x23, x23, x10+        umulh   x10, x7, x9+        adc     x24, x24, x10+        mul     x10, x8, x9+        umulh   x25, x8, x9+        adds    x24, x24, x10+        adc     x25, x25, xzr+        adds    x11, x11, x11+        adcs    x12, x12, x12+        adcs    x13, x13, x13+        adcs    x14, x14, x14+        adcs    x15, x15, x15+        adcs    x16, x16, x16+        adcs    x17, x17, x17+        adcs    x19, x19, x19+        adcs    x20, x20, x20+        adcs    x21, x21, x21+        adcs    x22, x22, x22+        adcs    x23, x23, x23+        adcs    x24, x24, x24+        adcs    x25, x25, x25+        cset    x26, cs+        umulh   x10, x2, x2+        adds    x11, x11, x10+        mul     x10, x3, x3+        adcs    x12, x12, x10+        umulh   x10, x3, x3+        adcs    x13, x13, x10+        mul     x10, x4, x4+        adcs    x14, x14, x10+        umulh   x10, x4, x4+        adcs    x15, x15, x10+        mul     x10, x5, x5+        adcs    x16, x16, x10+        umulh   x10, x5, x5+        adcs    x17, x17, x10+        mul     x10, x6, x6+        adcs    x19, x19, x10+        umulh   x10, x6, x6+        adcs    x20, x20, x10+        mul     x10, x7, x7+        adcs    x21, x21, x10+        umulh   x10, x7, x7+        adcs    x22, x22, x10+        mul     x10, x8, x8+        adcs    x23, x23, x10+        umulh   x10, x8, x8+        adcs    x24, x24, x10+        mul     x10, x9, x9+        adcs    x25, x25, x10+        umulh   x10, x9, x9+        adc     x26, x26, x10+        ldr     x1, [x1, #64]+        add     x1, x1, x1+        mul     x10, x1, x2+        adds    x19, x19, x10+        umulh   x10, x1, x2+        adcs    x20, x20, x10+        mul     x10, x1, x4+        adcs    x21, x21, x10+        umulh   x10, x1, x4+        adcs    x22, x22, x10+        mul     x10, x1, x6+        adcs    x23, x23, x10+        umulh   x10, x1, x6+        adcs    x24, x24, x10+        mul     x10, x1, x8+        adcs    x25, x25, x10+        umulh   x10, x1, x8+        adcs    x26, x26, x10+        lsr     x4, x1, #1+        mul     x4, x4, x4+        adc     x4, x4, xzr+        mul     x10, x1, x3+        adds    x20, x20, x10+        umulh   x10, x1, x3+        adcs    x21, x21, x10+        mul     x10, x1, x5+        adcs    x22, x22, x10+        umulh   x10, x1, x5+        adcs    x23, x23, x10+        mul     x10, x1, x7+        adcs    x24, x24, x10+        umulh   x10, x1, x7+        adcs    x25, x25, x10+        mul     x10, x1, x9+        adcs    x26, x26, x10+        umulh   x10, x1, x9+        adc     x4, x4, x10+        mul     x2, x2, x2+        cmp     xzr, xzr+        extr    x10, x20, x19, #9+        adcs    x2, x2, x10+        extr    x10, x21, x20, #9+        adcs    x11, x11, x10+        extr    x10, x22, x21, #9+        adcs    x12, x12, x10+        extr    x10, x23, x22, #9+        adcs    x13, x13, x10+        extr    x10, x24, x23, #9+        adcs    x14, x14, x10+        extr    x10, x25, x24, #9+        adcs    x15, x15, x10+        extr    x10, x26, x25, #9+        adcs    x16, x16, x10+        extr    x10, x4, x26, #9+        adcs    x17, x17, x10+        orr     x19, x19, #0xfffffffffffffe00+        lsr     x10, x4, #9+        adcs    x19, x19, x10+        sbcs    x2, x2, xzr+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        sbcs    x14, x14, xzr+        sbcs    x15, x15, xzr+        sbcs    x16, x16, xzr+        sbcs    x17, x17, xzr+        sbc     x19, x19, xzr+        and     x19, x19, #0x1ff+        stp     x2, x11, [x0]+        stp     x12, x13, [x0, #16]+        stp     x14, x15, [x0, #32]+        stp     x16, x17, [x0, #48]+        str     x19, [x0, #64]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_alt_sqr_p521)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_alt_sub_p521)++Lp521_jscalarmul_alt_sub_p521:+        CFI_START+        ldp     x5, x6, [x1]+        ldp     x4, x3, [x2]+        subs    x5, x5, x4+        sbcs    x6, x6, x3+        ldp     x7, x8, [x1, #16]+        ldp     x4, x3, [x2, #16]+        sbcs    x7, x7, x4+        sbcs    x8, x8, x3+        ldp     x9, x10, [x1, #32]+        ldp     x4, x3, [x2, #32]+        sbcs    x9, x9, x4+        sbcs    x10, x10, x3+        ldp     x11, x12, [x1, #48]+        ldp     x4, x3, [x2, #48]+        sbcs    x11, x11, x4+        sbcs    x12, x12, x3+        ldr     x13, [x1, #64]+        ldr     x4, [x2, #64]+        sbcs    x13, x13, x4+        sbcs    x5, x5, xzr+        sbcs    x6, x6, xzr+        sbcs    x7, x7, xzr+        sbcs    x8, x8, xzr+        sbcs    x9, x9, xzr+        sbcs    x10, x10, xzr+        sbcs    x11, x11, xzr+        sbcs    x12, x12, xzr+        sbcs    x13, x13, xzr+        and     x13, x13, #0x1ff+        stp     x5, x6, [x0]+        stp     x7, x8, [x0, #16]+        stp     x9, x10, [x0, #32]+        stp     x11, x12, [x0, #48]+        str     x13, [x0, #64]+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_alt_sub_p521)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/import.sh view
@@ -0,0 +1,112 @@+#!/bin/sh+# Re-import the vendored parts of AWS's s2n-bignum.+#+# Only the files crypton calls are kept, and they are kept unmodified -- the+# dispatch that chooses between the two variants of each is in+# cbits/p256/p256_s2n.c, not in here.  Run this from cbits/s2n:+#+#     ./import.sh [commit]+#+# and commit the result together with the COMMIT line it writes, so that the+# tree always says which upstream revision it holds.+set -eu++REPO=https://github.com/awslabs/s2n-bignum+REV=${1:-main}+HERE=$(cd "$(dirname "$0")" && pwd)+TMP=$(mktemp -d)+trap 'rm -rf "$TMP"' EXIT++git clone -q "$REPO" "$TMP/s2n"+git -C "$TMP/s2n" checkout -q "$REV"++# The headers every vendored file includes.+for h in _internal_s2n_bignum_arm.h _internal_s2n_bignum_x86_att.h; do+	cp "$TMP/s2n/include/$h" "$HERE/include/$h"+done+cp "$TMP/s2n/LICENSE" "$HERE/LICENSE"++# Both variants of each routine are taken, since which one is faster is not+# the same question on the two architectures -- see README.md.  Where the+# upstream tree has no separate _alt file the plain one defines both symbols,+# so "copy it if it is there" gets the right set either way.+# x86-64 only, for the things AArch64 does not want -- see README.md.+take_x86() {+	dir=$1+	name=$2+	cp "$TMP/s2n/x86_att/$dir/$name.S" "$HERE/x86_att/$name.S"+}++take() {+	curve=$1+	name=$2+	for arch in arm x86_att; do+		for v in "" _alt; do+			src="$TMP/s2n/$arch/$curve/$name$v.S"+			if [ -f "$src" ]; then+				cp "$src" "$HERE/$arch/$name$v.S"+			fi+		done+	done+}++# P-256: variable-point scalar multiplication, affine in and out, and the+# fixed-base one, which reads a table of its own that+# cbits/p256/gen_base_table.py builds.+take p256 p256_scalarmul+take p256 p256_scalarmulbase++# The Jacobian point operations, which ECDSA verification walks itself: it+# multiplies two scalars at once, in variable time, which is allowed there+# because everything it touches is public.  See cbits/p256/p256_verify.c.+take p256 p256_montjadd+take p256 p256_montjdouble+take p256 p256_montjmixadd+take p256 bignum_tomont_p256+take p256 bignum_demont_p256+take p256 bignum_neg_p256++# P-384 and P-521 have no affine wrapper upstream, so the Montgomery and+# Jacobian conversions are built here out of these; the glue is in+# cbits/crypton_ecc_s2n.c.+take p384 p384_montjscalarmul+take p384 bignum_tomont_p384+take p384 bignum_deamont_p384+take p384 bignum_montmul_p384+take p384 bignum_montsqr_p384+take p384 bignum_montinv_p384++take p521 p521_jscalarmul+take p521 bignum_mul_p521+take p521 bignum_sqr_p521+take p521 bignum_inv_p521++# X25519, both the general one and the fixed-base one that a key is+# generated with.  The word form, which both architectures have, rather than+# the byte form that only AArch64 has: they measure the same and this way+# there is one code path.+take curve25519 curve25519_x25519+take curve25519 curve25519_x25519base++# Ed25519's base point multiplication, which signing does twice -- once for+# the nonce's point and once for the public key it derives from the secret+# key every time -- and the encoding of the result, which has one form.+take curve25519 edwards25519_scalarmulbase+take curve25519 edwards25519_encode++# Inversion modulo an odd number of any size, which is what ECDSA does once+# per signature and once per verification.  It uses no instruction beyond+# the base architecture, so there is one of it and no run-time question.+take generic bignum_modinv++# Modular exponentiation at RSA sizes.  Only x86-64: on AArch64 crypton's C+# is the faster of the two, measured, so nothing is taken for it.  The+# Karatsuba multiplications and the reduction all want ADX.+take_x86 fastmul bignum_kmul_16_32+take_x86 fastmul bignum_ksqr_16_32+take_x86 fastmul bignum_kmul_32_64+take_x86 fastmul bignum_ksqr_32_64+take_x86 fastmul bignum_emontredc_8n++git -C "$TMP/s2n" rev-parse HEAD > "$HERE/COMMIT"+echo "imported $(cat "$HERE/COMMIT")"
+ cbits/s2n/include/_internal_s2n_bignum_arm.h view
@@ -0,0 +1,103 @@+#ifdef __APPLE__+#   define S2N_BN_SYMBOL(NAME) _##NAME+#   if defined(__AARCH64EL__) || defined(__ARMEL__)+#     define __LF %%+#   else+#     define __LF ;+#   endif+#else+#   define S2N_BN_SYMBOL(name) name+#   define __LF ;+#endif++#define S2N_BN_SYM_VISIBILITY_DIRECTIVE(name) .globl S2N_BN_SYMBOL(name)++#ifdef S2N_BN_HIDE_SYMBOLS+#   ifdef __APPLE__+#      define S2N_BN_SYM_PRIVACY_DIRECTIVE(name) .private_extern S2N_BN_SYMBOL(name)+#   else+#      define S2N_BN_SYM_PRIVACY_DIRECTIVE(name) .hidden S2N_BN_SYMBOL(name)+#   endif+#else+#   define S2N_BN_SYM_PRIVACY_DIRECTIVE(name)  /* NO-OP: S2N_BN_SYM_PRIVACY_DIRECTIVE */+#endif++#ifdef __APPLE__+#   define S2N_BN_FUNCTION_TYPE_DIRECTIVE(name) /* Not used in Mach-O */+#else+#   define S2N_BN_FUNCTION_TYPE_DIRECTIVE(name) .type name, %function+#endif++#ifdef __APPLE__+#   define S2N_BN_SIZE_DIRECTIVE(name) /* Not used in Mach-O */+#else+#   define S2N_BN_SIZE_DIRECTIVE(name) .size S2N_BN_SYMBOL(name), .-S2N_BN_SYMBOL(name)+#endif++// Enable branch target identification (BTI) support unless explicitly disabled+// with -DNO_IBT, mirroring the x86 _CET_ENDBR machinery. AARCH64_VALID_CALL_TARGET+// is emitted at each entry point unconditionally by default, since BTI 'c' is in+// the hint space and so behaves as a NOP on all pre-Armv8.5-A processors. The name+// matches AWS-LC's macro, whose definition we defer to if already present, just as+// the x86 side defers to <cet.h>. Unlike CET, BTI also needs a .note.gnu.property+// section: it has GNU_PROPERTY_AARCH64_FEATURE_1_AND semantics, so one object+// without the note silently disables BTI program-wide, hence emitting it here.++#if NO_IBT+#   if defined(AARCH64_VALID_CALL_TARGET)+#     error "The s2n-bignum build option NO_IBT was configured, but AARCH64_VALID_CALL_TARGET is defined in this compilation unit. That is weird, so failing the build."+#   endif+#   define AARCH64_VALID_CALL_TARGET+#elif !defined(AARCH64_VALID_CALL_TARGET)+#   define AARCH64_VALID_CALL_TARGET hint #34 /* BTI c */+#   ifndef __APPLE__+        .pushsection .note.gnu.property, "a"+        .balign 8+        .long 4                   /* n_namesz: sizeof "GNU\0" */+        .long 0x10                /* n_descsz: 16 bytes of property data */+        .long 0x5                 /* n_type: NT_GNU_PROPERTY_TYPE_0 */+        .asciz "GNU"+        .long 0xc0000000          /* pr_type: GNU_PROPERTY_AARCH64_FEATURE_1_AND */+        .long 4                   /* pr_datasz: 4 bytes */+        .long 1                   /* pr_data: GNU_PROPERTY_AARCH64_FEATURE_1_BTI */+        .long 0                   /* pad to 8-byte alignment */+        .popsection+#   endif+#endif++// Variants of instructions including CFI (call frame information) annotations++#define CFI_START .cfi_startproc+#define CFI_RET ret __LF .cfi_endproc++#define CFI_BL(target) bl target++#define CFI_PUSH2(lo,hi) stp     lo, hi, [sp, #-16]! __LF .cfi_adjust_cfa_offset 16 __LF .cfi_rel_offset lo, 0 __LF .cfi_rel_offset hi, 8+#define CFI_PUSH1Z(reg) stp     reg, xzr, [sp, #-16]! __LF .cfi_adjust_cfa_offset 16 __LF .cfi_rel_offset reg, 0++#define CFI_POP2(lo,hi) ldp     lo, hi, [sp], #16 __LF .cfi_adjust_cfa_offset -16 __LF .cfi_restore lo __LF .cfi_restore hi+#define CFI_POP1Z(reg) ldp     reg, xzr, [sp], #16 __LF .cfi_adjust_cfa_offset -16 __LF .cfi_restore reg++#define CFI_STACKSAVE2(lo,hi,offset) stp     lo, hi, [sp, #(offset)] __LF .cfi_rel_offset lo, offset __LF .cfi_rel_offset hi, offset+8++// This is an alternative to CFI_STACKSAVE2 to work around delocator problems+// in the AWS-LC FIPS build, avoiding certain composite expressions. It is+// expected that offset8 = offset+8 as in an invocation of CFI_STACKSAVE2.+// Likewise the (offset+0) oddities in the following macros are driven by+// delocator problems.++#define CFI_STACKSAVE2X(lo,hi,offset,offset8) stp     lo, hi, [sp, #(offset+0)] __LF .cfi_rel_offset lo, offset __LF .cfi_rel_offset hi, offset8++#define CFI_STACKSAVE1Z(reg,offset) stp     reg, xzr, [sp, #(offset+0)] __LF .cfi_rel_offset reg, offset++#define CFI_STACKLOAD2(lo,hi,offset) ldp     lo, hi, [sp, #(offset+0)] __LF .cfi_restore lo __LF .cfi_restore hi+#define CFI_STACKLOAD1Z(reg,offset) ldp     reg, xzr, [sp, #(offset+0)] __LF .cfi_restore reg++// It would be better to use -(offset) not -offset, but again there seem+// to be delocator issues. We adopt a discipline of not using dangerous+// composite expressions in this macro, e.g. parenthesizing the argument+// or just using numeric constants or products where the association is+// not a problem.++#define CFI_INC_SP(offset) add     sp, sp, #(offset+0) __LF .cfi_adjust_cfa_offset -offset+#define CFI_DEC_SP(offset) sub     sp, sp, #(offset+0) __LF .cfi_adjust_cfa_offset offset
+ cbits/s2n/include/_internal_s2n_bignum_x86_att.h view
@@ -0,0 +1,68 @@+#ifdef __APPLE__+#   define S2N_BN_SYMBOL(NAME) _##NAME+#else+#   define S2N_BN_SYMBOL(name) name+#endif++#define S2N_BN_SYM_VISIBILITY_DIRECTIVE(name) .globl S2N_BN_SYMBOL(name)++#ifdef S2N_BN_HIDE_SYMBOLS+#   ifdef __APPLE__+#      define S2N_BN_SYM_PRIVACY_DIRECTIVE(name) .private_extern S2N_BN_SYMBOL(name)+#   else+#      define S2N_BN_SYM_PRIVACY_DIRECTIVE(name) .hidden S2N_BN_SYMBOL(name)+#   endif+#else+#   define S2N_BN_SYM_PRIVACY_DIRECTIVE(name)  /* NO-OP: S2N_BN_SYM_PRIVACY_DIRECTIVE */+#endif++#ifdef __APPLE__+#   define S2N_BN_FUNCTION_TYPE_DIRECTIVE(name) /* Not used in Mach-O */+#else+#   define S2N_BN_FUNCTION_TYPE_DIRECTIVE(name) .type name, %function+#endif++#ifdef __APPLE__+#   define S2N_BN_SIZE_DIRECTIVE(name) /* Not used in Mach-O */+#else+#   define S2N_BN_SIZE_DIRECTIVE(name) .size S2N_BN_SYMBOL(name), .-S2N_BN_SYMBOL(name)+#endif++// Enable indirect branch tracking support unless explicitly disabled+// with -DNO_IBT. If the platform supports CET, simply inherit this from+// the usual header. Otherwise manually define _CET_ENDBR, used at each+// x86 entry point, to be the ENDBR64 instruction, with an explicit byte+// sequence for compilers/assemblers that don't know about it. Note that+// it is safe to use ENDBR64 on all platforms, since the encoding is by+// design interpreted as a NOP on all pre-CET x86_64 processors. The only+// downside is a small increase in code size and potentially a modest+// slowdown from executing one more instruction.++#if NO_IBT+#   if defined(_CET_ENDBR)+#     error "The s2n-bignum build option NO_IBT was configured, but _CET_ENDBR is defined in this compilation unit. That is weird, so failing the build."+#   endif+#   define _CET_ENDBR+#elif defined(__CET__)+#   include <cet.h>+#elif !defined(_CET_ENDBR)+#   define _CET_ENDBR .byte 0xf3,0x0f,0x1e,0xfa+#endif++// Variants of instructions including CFI (call frame information) annotations++#define CFI_START .cfi_startproc+#define CFI_RET retq ; .cfi_endproc++#define CFI_CALL(target) callq   target++#define CFI_PUSH(reg) pushq   reg ; .cfi_adjust_cfa_offset 8 ; .cfi_rel_offset reg, 0+#define CFI_POP(reg) popq    reg ; .cfi_adjust_cfa_offset -8 ; .cfi_restore reg++#define CFI_INC_RSP(offset) addq    $offset, %rsp ; .cfi_adjust_cfa_offset -offset+#define CFI_DEC_RSP(offset) subq    $offset, %rsp ; .cfi_adjust_cfa_offset offset++#define CFI_STACKSAVE(reg,offset) mov reg, offset(%rsp) ; .cfi_rel_offset reg, offset+#define CFI_STACKLOAD(reg,offset) mov offset(%rsp), reg ; .cfi_restore reg+#define CFI_STACKSAVEU(reg,offset) movups reg, offset(%rsp) ; .cfi_rel_offset reg, offset+#define CFI_STACKLOADU(reg,offset) movups offset(%rsp), reg ; .cfi_restore reg
+ cbits/s2n/x86_att/bignum_deamont_p384.S view
@@ -0,0 +1,184 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Convert from almost-Montgomery form, z := (x / 2^384) mod p_384+// Input x[6]; output z[6]+//+//    extern void bignum_deamont_p384(uint64_t z[static 6],+//                                    const uint64_t x[static 6]);+//+// Convert a 6-digit bignum x out of its (optionally almost) Montgomery form,+// "almost" meaning any 6-digit input will work, with no range restriction.+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_deamont_p384)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_deamont_p384)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_deamont_p384)+        .text++#define z %rdi+#define x %rsi++// Additional temps in the correction phase++#define u %rax+#define v %rcx+#define w %rdx++#define vshort %ecx++// Core one-step "short" Montgomery reduction macro. Takes input in+// [d5;d4;d3;d2;d1;d0] and returns result in [d6;d5;d4;d3;d2;d1],+// adding to the existing contents of [d5;d4;d3;d2;d1;d0]. This+// is intended only for 6-word inputs as in mapping out of Montgomery,+// not for the general case of Montgomery multiplication. It is fine+// for d6 to be the same register as d0.+//+// Parms:  montreds(d6,d5,d4,d3,d2,d1,d0)+//+// We want to add (2^384 - 2^128 - 2^96 + 2^32 - 1) * w+// where w = [d0 + (d0<<32)] mod 2^64++#define montreds(d6,d5,d4,d3,d2,d1,d0)                                  \+/* Our correction multiplier is w = [d0 + (d0<<32)] mod 2^64 */         \+        movq    d0, %rdx ;                                        \+        shlq    $32, %rdx ;                                        \+        addq    d0, %rdx ;                                        \+/* Construct [%rsi;%rcx;%rax;-] = (2^384 - p_384) * w           */         \+/* We know the lowest word will cancel so we can re-use d0   */         \+/* as a temp.                                                */         \+        xorq    %rsi, %rsi ;                                       \+        movq    $0xffffffff00000001, %rax ;                        \+        mulxq   %rax, %rcx, %rax ;                                  \+        movl    $0x00000000ffffffff, %ecx ;                        \+        mulxq   %rcx, d0, %rcx ;                                   \+        adcq    d0, %rax ;                                        \+        adcq    %rdx, %rcx ;                                       \+        adcq    $0, %rsi ;                                         \+/* Now subtract that and add 2^384 * w                       */         \+        subq    %rax, d1 ;                                        \+        sbbq    %rcx, d2 ;                                        \+        sbbq    %rsi, d3 ;                                        \+        sbbq    $0, d4 ;                                          \+        sbbq    $0, d5 ;                                          \+        movq    %rdx, d6 ;                                        \+        sbbq    $0, d6++S2N_BN_SYMBOL(bignum_deamont_p384):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Save more registers to play with++        CFI_PUSH(%r12)+        CFI_PUSH(%r13)++// Set up an initial window [%r13,%r12,%r11,%r10,%r9,%r8] = x++        movq    (x), %r8+        movq    8(x), %r9+        movq    16(x), %r10+        movq    24(x), %r11+        movq    32(x), %r12+        movq    40(x), %r13++// Montgomery reduce window 0++        montreds(%r8,%r13,%r12,%r11,%r10,%r9,%r8)++// Montgomery reduce window 1++        montreds(%r9,%r8,%r13,%r12,%r11,%r10,%r9)++// Montgomery reduce window 2++        montreds(%r10,%r9,%r8,%r13,%r12,%r11,%r10)++// Montgomery reduce window 3++        montreds(%r11,%r10,%r9,%r8,%r13,%r12,%r11)++// Montgomery reduce window 4++        montreds(%r12,%r11,%r10,%r9,%r8,%r13,%r12)++// Montgomery reduce window 5++        montreds(%r13,%r12,%r11,%r10,%r9,%r8,%r13)++// Do a test addition of dd = [%r13;%r12;%r11;%r10;%r9;%r8] and+// 2^384 - p_384 = [0;0;0;1;v;u], hence setting CF iff+// dd + (2^384 - p_384) >= 2^384, hence iff dd >= p_384.++        movq    $0xffffffff00000001, u+        movl    $0x00000000ffffffff, vshort++        movq    %r8, w+        addq    u, w+        movq    %r9, w+        adcq    v, w+        movq    %r10, w+        adcq    $1, w+        movq    %r11, w+        adcq    $0, w+        movq    %r12, w+        adcq    $0, w+        movq    %r13, w+        adcq    $0, w++// Convert CF to a bitmask in w++        sbbq    w, w++// Masked addition of 2^384 - p_384, hence subtraction of p_384++        andq    w, u+        andq    w, v+        andq    $1, w++        addq   u, %r8+        adcq   v, %r9+        adcq   w, %r10+        adcq   $0, %r11+        adcq   $0, %r12+        adcq   $0, %r13++// Write back the result++        movq    %r8, (z)+        movq    %r9, 8(z)+        movq    %r10, 16(z)+        movq    %r11, 24(z)+        movq    %r12, 32(z)+        movq    %r13, 40(z)++// Restore registers and return++        CFI_POP(%r13)+        CFI_POP(%r12)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_deamont_p384)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_deamont_p384_alt.S view
@@ -0,0 +1,184 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Convert from almost-Montgomery form, z := (x / 2^384) mod p_384+// Input x[6]; output z[6]+//+//    extern void bignum_deamont_p384_alt(uint64_t z[static 6],+//                                        const uint64_t x[static 6]);+//+// Convert a 6-digit bignum x out of its (optionally almost) Montgomery form,+// "almost" meaning any 6-digit input will work, with no range restriction.+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_deamont_p384_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_deamont_p384_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_deamont_p384_alt)+        .text++#define z %rdi+#define x %rsi++// Additional temps in the correction phase++#define u %rax+#define v %rcx+#define w %rdx++#define vshort %ecx++// Core one-step "short" Montgomery reduction macro. Takes input in+// [d5;d4;d3;d2;d1;d0] and returns result in [d6;d5;d4;d3;d2;d1],+// adding to the existing [d5;d4;d3;d2;d1] and re-using d0 as a+// temporary internally, as well as %rax, %rcx and %rdx.+// It is OK for d6 and d0 to be the same register (they often are)+//+// We want to add (2^384 - 2^128 - 2^96 + 2^32 - 1) * w+// where w = [d0 + (d0<<32)] mod 2^64+//+//       montreds(d6,d5,d4,d3,d2,d1,d0)++#define montreds(d6,d5,d4,d3,d2,d1,d0)                                  \+/* Our correction multiplier is w = [d0 + (d0<<32)] mod 2^64 */         \+        movq    d0, %rcx ;                                        \+        shlq    $32, %rcx ;                                        \+        addq    d0, %rcx ;                                        \+/* Construct [%rax;%rdx;d0;-] = (2^384 - p_384) * w            */         \+/* We know the lowest word will cancel so we can re-use d0   */         \+/* and %rcx as temps.                                         */         \+        movq    $0xffffffff00000001, %rax ;                        \+        mulq    %rcx;                                            \+        movq    %rdx, d0 ;                                        \+        movq    $0x00000000ffffffff, %rax ;                        \+        mulq    %rcx;                                            \+        addq    %rax, d0 ;                                        \+        movl    $0, %eax ;                                         \+        adcq    %rcx, %rdx ;                                       \+        adcl    %eax, %eax ;                                       \+/* Now subtract that and add 2^384 * w                       */         \+        subq    d0, d1 ;                                         \+        sbbq    %rdx, d2 ;                                        \+        sbbq    %rax, d3 ;                                        \+        sbbq    $0, d4 ;                                          \+        sbbq    $0, d5 ;                                          \+        movq    %rcx, d6 ;                                        \+        sbbq    $0, d6++S2N_BN_SYMBOL(bignum_deamont_p384_alt):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Save more registers to play with++        CFI_PUSH(%r12)+        CFI_PUSH(%r13)++// Set up an initial window [%r13,%r12,%r11,%r10,%r9,%r8] = x++        movq    (x), %r8+        movq    8(x), %r9+        movq    16(x), %r10+        movq    24(x), %r11+        movq    32(x), %r12+        movq    40(x), %r13++// Montgomery reduce window 0++        montreds(%r8,%r13,%r12,%r11,%r10,%r9,%r8)++// Montgomery reduce window 1++        montreds(%r9,%r8,%r13,%r12,%r11,%r10,%r9)++// Montgomery reduce window 2++        montreds(%r10,%r9,%r8,%r13,%r12,%r11,%r10)++// Montgomery reduce window 3++        montreds(%r11,%r10,%r9,%r8,%r13,%r12,%r11)++// Montgomery reduce window 4++        montreds(%r12,%r11,%r10,%r9,%r8,%r13,%r12)++// Montgomery reduce window 5++        montreds(%r13,%r12,%r11,%r10,%r9,%r8,%r13)++// Do a test addition of dd = [%r13;%r12;%r11;%r10;%r9;%r8] and+// 2^384 - p_384 = [0;0;0;1;v;u], hence setting CF iff+// dd + (2^384 - p_384) >= 2^384, hence iff dd >= p_384.++        movq    $0xffffffff00000001, u+        movl    $0x00000000ffffffff, vshort++        movq    %r8, w+        addq    u, w+        movq    %r9, w+        adcq    v, w+        movq    %r10, w+        adcq    $1, w+        movq    %r11, w+        adcq    $0, w+        movq    %r12, w+        adcq    $0, w+        movq    %r13, w+        adcq    $0, w++// Convert CF to a bitmask in w++        sbbq    w, w++// Masked addition of 2^384 - p_384, hence subtraction of p_384++        andq    w, u+        andq    w, v+        andq    $1, w++        addq   u, %r8+        adcq   v, %r9+        adcq   w, %r10+        adcq   $0, %r11+        adcq   $0, %r12+        adcq   $0, %r13++// Write back the result++        movq    %r8, (z)+        movq    %r9, 8(z)+        movq    %r10, 16(z)+        movq    %r11, 24(z)+        movq    %r12, 32(z)+        movq    %r13, 40(z)++// Restore registers and return++        CFI_POP(%r13)+        CFI_POP(%r12)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_deamont_p384_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_demont_p256.S view
@@ -0,0 +1,116 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Convert from Montgomery form z := (x / 2^256) mod p_256, assuming x reduced+// Input x[4]; output z[4]+//+//    extern void bignum_demont_p256(uint64_t z[static 4],+//                                   const uint64_t x[static 4]);+//+// This assumes the input is < p_256 for correctness. If this is not the case,+// use the variant "bignum_deamont_p256" instead.+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_demont_p256)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_demont_p256)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_demont_p256)+        .text++#define z %rdi+#define x %rsi++// Add %rdx * m into a register-pair (high,low)+// maintaining consistent double-carrying with adcx and adox,+// using %rax and %rcx as temporaries++#define mulpadd(high,low,m)             \+        mulxq   m, %rax, %rcx ;            \+        adcxq   %rax, low ;               \+        adoxq   %rcx, high++S2N_BN_SYMBOL(bignum_demont_p256):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Save one more register to play with++        CFI_PUSH(%rbx)++// Set up an initial 4-word window [%r11,%r10,%r9,%r8] = x++        movq    (x), %r8+        movq    8(x), %r9+        movq    16(x), %r10+        movq    24(x), %r11++// Fill in two zeros to the left++        xorq    %rbx, %rbx+        xorq    %rsi, %rsi++// Montgomery reduce windows 0 and 1 together++        movq    $0x0000000100000000, %rdx+        mulpadd(%r10,%r9,%r8)+        mulpadd(%r11,%r10,%r9)+        movq    $0xffffffff00000001, %rdx+        mulpadd(%rbx,%r11,%r8)+        mulpadd(%rsi,%rbx,%r9)+        movl    $0, %r8d+        adcxq   %r8, %rsi++// Append just one more leading zero (by the above %r8 = 0 already).++        xorq    %r9, %r9++// Montgomery reduce windows 2 and 3 together++        movq    $0x0000000100000000, %rdx+        mulpadd(%rbx,%r11,%r10)+        mulpadd(%rsi,%rbx,%r11)+        movq    $0xffffffff00000001, %rdx+        mulpadd(%r8,%rsi,%r10)+        mulpadd(%r9,%r8,%r11)+        movl    $0, %r10d+        adcxq   %r10, %r9++// Since the input was assumed reduced modulo, i.e. < p, we actually know that+// 2^256 * [carries; %r9;%r8;%rsi;%rbx] is <= (p - 1) + (2^256 - 1) p+// and hence [carries; %r9;%r8;%rsi;%rbx] < p. This means in fact carries = 0+// and [%r9;%r8;%rsi;%rbx] is already our answer, without further correction.+// Write that back.++        movq    %rbx, (z)+        movq    %rsi, 8(z)+        movq    %r8, 16(z)+        movq    %r9, 24(z)++// Restore saved register and return++        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_demont_p256)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_demont_p256_alt.S view
@@ -0,0 +1,134 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Convert from Montgomery form z := (x / 2^256) mod p_256, assuming x reduced+// Input x[4]; output z[4]+//+//    extern void bignum_demont_p256_alt(uint64_t z[static 4],+//                                       const uint64_t x[static 4]);+//+// This assumes the input is < p_256 for correctness. If this is not the case,+// use the variant "bignum_deamont_p256" instead.+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_demont_p256_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_demont_p256_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_demont_p256_alt)+        .text++#define z %rdi+#define x %rsi++// Add %rdx * m into a register-pair (high,low)+// maintaining consistent double-carrying with adcx and adox,+// using %rax and %rcx as temporaries++#define mulpado(high,low,m)             \+        mulxq   m, %rax, %rcx ;            \+        adcxq   %rax, low ;               \+        adoxq   %rcx, high++// Add %rcx * m into a register-pair (high,low) maintaining consistent+// carry-catching with carry (negated, as bitmask) and using %rax and %rdx+// as temporaries++#define mulpadd(carry,high,low,m)       \+        movq    m, %rax ;                 \+        mulq    %rcx;                    \+        subq    carry, %rdx ;             \+        addq    %rax, low ;               \+        adcq    %rdx, high ;              \+        sbbq    carry, carry++// Initial version assuming no carry-in++#define mulpadi(carry,high,low,m)       \+        movq    m, %rax ;                 \+        mulq    %rcx;                    \+        addq    %rax, low ;               \+        adcq    %rdx, high ;              \+        sbbq    carry, carry++// Version with no carry in or out++#define mulpadn(high,low,m)             \+        movq    m, %rax ;                 \+        mulq    %rcx;                    \+        addq    %rax, low ;               \+        adcq    %rdx, high++S2N_BN_SYMBOL(bignum_demont_p256_alt):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Set up an initial 4-word window [%r11,%r10,%r9,%r8] = x++        movq    (x), %r8+        movq    8(x), %r9+        movq    16(x), %r10+        movq    24(x), %r11++// Load constant 2^32; %rcx toggles between this and (1 - %rcx) below++        movq    $0x0000000100000000, %rcx++// Montgomery reduce windows 0 and 1 together as [%r8;%rsi;%r11;%r10]++        mulpadi(%rsi,%r10,%r9,%r8)+        mulpadd(%rsi,%r11,%r10,%r9)+        negq    %rcx+        negq    %rsi+        incq    %rcx+        mulpadi(%r8,%rsi,%r11,%r8)+        negq    %r8+        mulpadn(%r8,%rsi,%r9)++// Montgomery reduce windows 2 and 3 together as [%r10;%r9;%r8;%rsi]++        negq    %rcx+        incq    %rcx+        mulpadi(%r9,%rsi,%r11,%r10)+        mulpadd(%r9,%r8,%rsi,%r11)+        negq    %rcx+        negq    %r9+        incq    %rcx+        mulpadi(%r10,%r9,%r8,%r10)+        negq    %r10+        mulpadn(%r10,%r9,%r11)++// Since the input was assumed reduced modulo, i.e. < p, we actually know that+// 2^256 * [carries; %r10;%r9;%r8;%rsi] is <= (p - 1) + (2^256 - 1) p+// and hence [carries; %r10;%r9;%r8;%rsi] < p. This means in fact carries = 0+// and [%r10;%r9;%r8;%rsi] is already our answer, without further correction.+// Write that back.++        movq    %rsi, (z)+        movq    %r8, 8(z)+        movq    %r9, 16(z)+        movq    %r10, 24(z)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_demont_p256_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_emontredc_8n.S view
@@ -0,0 +1,427 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Extended Montgomery reduce in 8-digit blocks, results in input-output buffer+// Inputs z[2*k], m[k], w; outputs function return (extra result bit) and z[2*k]+//+//    extern uint64_t bignum_emontredc_8n(uint64_t k, uint64_t *z, const uint64_t *m,+//                                        uint64_t w);+//+// Functionally equivalent to bignum_emontredc (see that file for more detail).+// But in general assumes that the input k is a multiple of 8.+//+// Standard x86-64 ABI: RDI = k, RSI = z, RDX = m, RCX = w, returns RAX+// Microsoft x64 ABI:   RCX = k, RDX = z, R8 = m, R9 = w, returns RAX+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_emontredc_8n)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_emontredc_8n)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_emontredc_8n)+        .text+        .balign 32++// Original input parameters are here++#define z %rsi+#define w %rcx++// This is copied in early once we stash away k++#define m %rdi++// A variable z pointer++#define zz %rbp++// Stack-based variables++#define carry  (%rsp)+#define innercount  8(%rsp)+#define outercount  16(%rsp)+#define k8m1  24(%rsp)++// -----------------------------------------------------------------------------+// Standard macros as used in pure multiplier arrays+// -----------------------------------------------------------------------------++// mulpadd i, j adds z[i] * rdx (now assumed = m[j]) into the window at i+j++.macro mulpadd arg1,arg2+        mulxq   8*\arg1(z), %rax, %rbx+.if ((\arg1 + \arg2) % 8 == 0)+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+.elseif ((\arg1 + \arg2) % 8 == 1)+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+.elseif ((\arg1 + \arg2) % 8 == 2)+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+.elseif ((\arg1 + \arg2) % 8 == 3)+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+.elseif ((\arg1 + \arg2) % 8 == 4)+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+.elseif ((\arg1 + \arg2) % 8 == 5)+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+.elseif ((\arg1 + \arg2) % 8 == 6)+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+.elseif ((\arg1 + \arg2) % 8 == 7)+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+.endif++.endm++// addrow i adds z[i] + zz[0..7] * m[j] into the window++.macro addrow arg1+        movq    8*\arg1(m), %rdx+        xorl    %eax, %eax // Get a known flag state++.if (\arg1 % 8 == 0)+        adoxq   8*\arg1(zz), %r8+.elseif (\arg1 % 8 == 1)+        adoxq   8*\arg1(zz), %r9+.elseif (\arg1 % 8 == 2)+        adoxq   8*\arg1(zz), %r10+.elseif (\arg1 % 8 == 3)+        adoxq   8*\arg1(zz), %r11+.elseif (\arg1 % 8 == 4)+        adoxq   8*\arg1(zz), %r12+.elseif (\arg1 % 8 == 5)+        adoxq   8*\arg1(zz), %r13+.elseif (\arg1 % 8 == 6)+        adoxq   8*\arg1(zz), %r14+.elseif (\arg1 % 8 == 7)+        adoxq   8*\arg1(zz), %r15+.endif++        mulpadd 0, \arg1++.if (\arg1 % 8 == 0)+        movq    %r8, 8*\arg1(zz)+        movl    $0, %r8d+.elseif (\arg1 % 8 == 1)+        movq    %r9, 8*\arg1(zz)+        movl    $0, %r9d+.elseif (\arg1 % 8 == 2)+        movq    %r10, 8*\arg1(zz)+        movl    $0, %r10d+.elseif (\arg1 % 8 == 3)+        movq    %r11, 8*\arg1(zz)+        movl    $0, %r11d+.elseif (\arg1 % 8 == 4)+        movq    %r12, 8*\arg1(zz)+        movl    $0, %r12d+.elseif (\arg1 % 8 == 5)+        movq    %r13, 8*\arg1(zz)+        movl    $0, %r13d+.elseif (\arg1 % 8 == 6)+        movq    %r14, 8*\arg1(zz)+        movl    $0, %r14d+.elseif (\arg1 % 8 == 7)+        movq    %r15, 8*\arg1(zz)+        movl    $0, %r15d+.endif++        mulpadd 1, \arg1+        mulpadd 2, \arg1+        mulpadd 3, \arg1+        mulpadd 4, \arg1+        mulpadd 5, \arg1+        mulpadd 6, \arg1+        mulpadd 7, \arg1++.if (\arg1 % 8 == 0)+        adcq    $0, %r8+.elseif (\arg1 % 8 == 1)+        adcq    $0, %r9+.elseif (\arg1 % 8 == 2)+        adcq    $0, %r10+.elseif (\arg1 % 8 == 3)+        adcq    $0, %r11+.elseif (\arg1 % 8 == 4)+        adcq    $0, %r12+.elseif (\arg1 % 8 == 5)+        adcq    $0, %r13+.elseif (\arg1 % 8 == 6)+        adcq    $0, %r14+.elseif (\arg1 % 8 == 7)+        adcq    $0, %r15+.endif+++.endm++// -----------------------------------------------------------------------------+// Anti-matter versions with z and m switched, and also not writing back the z+// words, but the inverses instead, *and* also adding in the z[0..7] at the+// beginning. The aim is to use this in Montgomery where we discover z[j]+// entries as we go along.+// -----------------------------------------------------------------------------++.macro mulpadda arg1,arg2+        mulxq   8*\arg1(m), %rax, %rbx+.if ((\arg1 + \arg2) % 8 == 0)+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+.elseif ((\arg1 + \arg2) % 8 == 1)+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+.elseif ((\arg1 + \arg2) % 8 == 2)+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+.elseif ((\arg1 + \arg2) % 8 == 3)+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+.elseif ((\arg1 + \arg2) % 8 == 4)+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+.elseif ((\arg1 + \arg2) % 8 == 5)+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+.elseif ((\arg1 + \arg2) % 8 == 6)+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+.elseif ((\arg1 + \arg2) % 8 == 7)+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+.endif++.endm++.macro adurowa arg1+        movq    w, %rdx // Get the word-level modular inverse+        xorl    %eax, %eax // Get a known flag state+.if (\arg1 % 8 == 0)+        mulxq   %r8, %rdx, %rax+.elseif (\arg1 % 8 == 1)+        mulxq   %r9, %rdx, %rax+.elseif (\arg1 % 8 == 2)+        mulxq   %r10, %rdx, %rax+.elseif (\arg1 % 8 == 3)+        mulxq   %r11, %rdx, %rax+.elseif (\arg1 % 8 == 4)+        mulxq   %r12, %rdx, %rax+.elseif (\arg1 % 8 == 5)+        mulxq   %r13, %rdx, %rax+.elseif (\arg1 % 8 == 6)+        mulxq   %r14, %rdx, %rax+.elseif (\arg1 % 8 == 7)+        mulxq   %r15, %rdx, %rax+.endif++        movq    %rdx, 8*\arg1(z) // Store multiplier word++        mulpadda 0, \arg1++        // Note that the bottom reg of the window is zero by construction+        // So it's safe just to use "mulpadda 7" here++        mulpadda 1, \arg1+        mulpadda 2, \arg1+        mulpadda 3, \arg1+        mulpadda 4, \arg1+        mulpadda 5, \arg1+        mulpadda 6, \arg1+        mulpadda 7, \arg1          // window lowest = 0 beforehand by construction++.if (\arg1 % 8 == 0)+        adcq    $0, %r8+.elseif (\arg1 % 8 == 1)+        adcq    $0, %r9+.elseif (\arg1 % 8 == 2)+        adcq    $0, %r10+.elseif (\arg1 % 8 == 3)+        adcq    $0, %r11+.elseif (\arg1 % 8 == 4)+        adcq    $0, %r12+.elseif (\arg1 % 8 == 5)+        adcq    $0, %r13+.elseif (\arg1 % 8 == 6)+        adcq    $0, %r14+.elseif (\arg1 % 8 == 7)+        adcq    $0, %r15+.endif++.endm++.macro adurowza+        movq    w, %rdx // Get the word-level modular inverse+        xorl    %eax, %eax // Get a known flag state++        movq    (z), %r8 // %r8 = zeroth word+        mulxq   %r8, %rdx, %rax // Compute multiplier word+        movq    %rdx, (z) // Store multiplier word+        movq    8(z), %r9++        mulpadda 0, 0+        movq    16(z), %r10+        mulpadda 1, 0+        movq    24(z), %r11+        mulpadda 2, 0+        movq    32(z), %r12+        mulpadda 3, 0+        movq    40(z), %r13+        mulpadda 4, 0+        movq    48(z), %r14+        mulpadda 5, 0+        movq    56(z), %r15+        mulpadda 6, 0+        mulpadda 7, 0           // r8 = 0 beforehand by construction+        adcq    $0, %r8+.endm++// -----------------------------------------------------------------------------+// Hybrid top, doing an 8 block specially then multiple additional 8 blocks+// -----------------------------------------------------------------------------++// Multiply-add: z := z + x[i...i+7] * m++.macro addrows++        adurowza+        adurowa 1+        adurowa 2+        adurowa 3+        adurowa 4+        adurowa 5+        adurowa 6+        adurowa 7++        movq    z, zz++        movq    k8m1, %rax+        testq   %rax, %rax+        jz      Lbignum_emontredc_8n_innerend+        movq    %rax, innercount+Lbignum_emontredc_8n_innerloop:+        addq    $64, zz+        addq    $64, m+        addrow 0+        addrow 1+        addrow 2+        addrow 3+        addrow 4+        addrow 5+        addrow 6+        addrow 7+        subq    $64, innercount+        jnz     Lbignum_emontredc_8n_innerloop++        movq    k8m1, %rax+Lbignum_emontredc_8n_innerend:+        subq    %rax, m++        movq    carry, %rbx+        negq    %rbx+        adcq    %r8, 64(z,%rax,1)+        adcq    %r9, 72(z,%rax,1)+        adcq    %r10, 80(z,%rax,1)+        adcq    %r11, 88(z,%rax,1)+        adcq    %r12, 96(z,%rax,1)+        adcq    %r13, 104(z,%rax,1)+        adcq    %r14, 112(z,%rax,1)+        adcq    %r15, 120(z,%rax,1)+        movl    $0, %eax+        adcq    $0, %rax+        movq    %rax, carry+.endm++// -----------------------------------------------------------------------------+// Main code.+// -----------------------------------------------------------------------------++S2N_BN_SYMBOL(bignum_emontredc_8n):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+        movq    %r9, %rcx+#endif++// Save more registers to play with++        CFI_PUSH(%rbp)+        CFI_PUSH(%rbx)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++// Pre-initialize the return value to 0 just in case of early exit below++        xorl    %eax, %eax++// Divide the input k by 8, and push k8m1 = (k/8 - 1)<<6 which is used as+// the scaled inner loop counter / pointer adjustment repeatedly. Also push+// k/8 itself which is here initializing the outer loop count.++        shrq    $3, %rdi+        jz      Lbignum_emontredc_8n_end++        leaq    -1(%rdi), %rbx+        shlq    $6, %rbx+        CFI_PUSH(%rbx)+        CFI_PUSH(%rdi)++// Make space for two more variables, and set between-stages carry to 0++        CFI_DEC_RSP(16)+        movq    $0, carry++// Copy m into its main home++        movq    %rdx, m++// Now just systematically add in the rows++Lbignum_emontredc_8n_outerloop:+        addrows+        addq    $64, z+        subq    $1, outercount+        jnz     Lbignum_emontredc_8n_outerloop++// Pop the carry-out "p", which was stored at [%rsp], put in %rax for return++        CFI_POP(%rax)++// Adjust the stack++        CFI_INC_RSP(24)++// Reset of epilog++Lbignum_emontredc_8n_end:++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbx)+        CFI_POP(%rbp)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_emontredc_8n)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_inv_p521.S view
@@ -0,0 +1,2093 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Modular inverse modulo p_521 =  2^521 - 1+// Input x[9]; output z[9]+//+// extern void bignum_inv_p521(uint64_t z[static 9],const uint64_t x[static 9]);+//+// Assuming the 9-digit input x is coprime to p_521, i.e. is not divisible+// by it, returns z < p_521 such that x * z == 1 (mod p_521). Note that+// x does not need to be reduced modulo p_521, but the output always is.+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------+#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_inv_p521)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_inv_p521)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_inv_p521)+        .text+        .balign 32++// Size in bytes of a 64-bit word++#define N 8++// Pointer-offset pairs for temporaries on stack++#define f 0(%rsp)+#define g (9*N)(%rsp)+#define u (18*N)(%rsp)+#define v (27*N)(%rsp)+#define tmp  (36*N)(%rsp)+#define tmp2  (37*N)(%rsp)+#define i  (38*N)(%rsp)+#define d  (39*N)(%rsp)++#define mat (40*N)(%rsp)++// Backup for the input pointer++#define res  (44*N)(%rsp)++// Total size to reserve on the stack++#define NSPACE 45*N++// Syntactic variants to make x86_att version simpler to generate++#define F 0+#define G (9*N)+#define U (18*N)+#define V (27*N)+#define MAT (40*N)++#define ff  (%rsp)+#define gg  (9*N)(%rsp)++// Very similar to a subroutine call to the s2n-bignum word_divstep59.+// But different in register usage and returning the final matrix as+//+// [ %r8   %r10]+// [ %r12  %r14]+//+// and also returning the matrix still negated (which doesn't matter)++#define divstep59(din,fin,gin)                                          \+        movq    din, %rsi ;                                               \+        movq    fin, %rdx ;                                               \+        movq    gin, %rcx ;                                               \+        movq    %rdx, %rbx ;                                               \+        andq    $0xfffff, %rbx ;                                           \+        movabsq $0xfffffe0000000000, %rax ;                                \+        orq     %rax, %rbx ;                                               \+        andq    $0xfffff, %rcx ;                                           \+        movabsq $0xc000000000000000, %rax ;                                \+        orq     %rax, %rcx ;                                               \+        movq    $0xfffffffffffffffe, %rax ;                                \+        xorl    %ebp, %ebp ;                                               \+        movl    $0x2, %edx ;                                               \+        movq    %rbx, %rdi ;                                               \+        movq    %rax, %r8 ;                                                \+        testq   %rsi, %rsi ;                                               \+        cmovs   %rbp, %r8 ;                                                \+        testq   $0x1, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        sarq    $1, %rcx ;                                                 \+        movl    $0x100000, %eax ;                                          \+        leaq    (%rbx,%rax), %rdx ;                                         \+        leaq    (%rcx,%rax), %rdi ;                                         \+        shlq    $0x16, %rdx ;                                              \+        shlq    $0x16, %rdi ;                                              \+        sarq    $0x2b, %rdx ;                                              \+        sarq    $0x2b, %rdi ;                                              \+        movabsq $0x20000100000, %rax ;                                     \+        leaq    (%rbx,%rax), %rbx ;                                         \+        leaq    (%rcx,%rax), %rcx ;                                         \+        sarq    $0x2a, %rbx ;                                              \+        sarq    $0x2a, %rcx ;                                              \+        movq    %rdx, MAT(%rsp) ;                                         \+        movq    %rbx, MAT+0x8(%rsp) ;                                     \+        movq    %rdi, MAT+0x10(%rsp) ;                                    \+        movq    %rcx, MAT+0x18(%rsp) ;                                    \+        movq    fin, %r12 ;                                               \+        imulq   %r12, %rdi ;                                               \+        imulq   %rdx, %r12 ;                                               \+        movq    gin, %r13 ;                                               \+        imulq   %r13, %rbx ;                                               \+        imulq   %rcx, %r13 ;                                               \+        addq    %rbx, %r12 ;                                               \+        addq    %rdi, %r13 ;                                               \+        sarq    $0x14, %r12 ;                                              \+        sarq    $0x14, %r13 ;                                              \+        movq    %r12, %rbx ;                                               \+        andq    $0xfffff, %rbx ;                                           \+        movabsq $0xfffffe0000000000, %rax ;                                \+        orq     %rax, %rbx ;                                               \+        movq    %r13, %rcx ;                                               \+        andq    $0xfffff, %rcx ;                                           \+        movabsq $0xc000000000000000, %rax ;                                \+        orq     %rax, %rcx ;                                               \+        movq    $0xfffffffffffffffe, %rax ;                                \+        movl    $0x2, %edx ;                                               \+        movq    %rbx, %rdi ;                                               \+        movq    %rax, %r8 ;                                                \+        testq   %rsi, %rsi ;                                               \+        cmovs   %rbp, %r8 ;                                                \+        testq   $0x1, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        sarq    $1, %rcx ;                                                 \+        movl    $0x100000, %eax ;                                          \+        leaq    (%rbx,%rax), %r8 ;                                          \+        leaq    (%rcx,%rax), %r10 ;                                         \+        shlq    $0x16, %r8 ;                                               \+        shlq    $0x16, %r10 ;                                              \+        sarq    $0x2b, %r8 ;                                               \+        sarq    $0x2b, %r10 ;                                              \+        movabsq $0x20000100000, %rax ;                                     \+        leaq    (%rbx,%rax), %r15 ;                                         \+        leaq    (%rcx,%rax), %r11 ;                                         \+        sarq    $0x2a, %r15 ;                                              \+        sarq    $0x2a, %r11 ;                                              \+        movq    %r13, %rbx ;                                               \+        movq    %r12, %rcx ;                                               \+        imulq   %r8, %r12 ;                                                \+        imulq   %r15, %rbx ;                                               \+        addq    %rbx, %r12 ;                                               \+        imulq   %r11, %r13 ;                                               \+        imulq   %r10, %rcx ;                                               \+        addq    %rcx, %r13 ;                                               \+        sarq    $0x14, %r12 ;                                              \+        sarq    $0x14, %r13 ;                                              \+        movq    %r12, %rbx ;                                               \+        andq    $0xfffff, %rbx ;                                           \+        movabsq $0xfffffe0000000000, %rax ;                                \+        orq     %rax, %rbx ;                                               \+        movq    %r13, %rcx ;                                               \+        andq    $0xfffff, %rcx ;                                           \+        movabsq $0xc000000000000000, %rax ;                                \+        orq     %rax, %rcx ;                                               \+        movq    MAT(%rsp), %rax ;                                         \+        imulq   %r8, %rax ;                                                \+        movq    MAT+0x10(%rsp), %rdx ;                                    \+        imulq   %r15, %rdx ;                                               \+        imulq   MAT+0x8(%rsp), %r8 ;                                      \+        imulq   MAT+0x18(%rsp), %r15 ;                                    \+        addq    %r8, %r15 ;                                                \+        leaq    (%rax,%rdx), %r9 ;                                          \+        movq    MAT(%rsp), %rax ;                                         \+        imulq   %r10, %rax ;                                               \+        movq    MAT+0x10(%rsp), %rdx ;                                    \+        imulq   %r11, %rdx ;                                               \+        imulq   MAT+0x8(%rsp), %r10 ;                                     \+        imulq   MAT+0x18(%rsp), %r11 ;                                    \+        addq    %r10, %r11 ;                                               \+        leaq    (%rax,%rdx), %r13 ;                                         \+        movq    $0xfffffffffffffffe, %rax ;                                \+        movl    $0x2, %edx ;                                               \+        movq    %rbx, %rdi ;                                               \+        movq    %rax, %r8 ;                                                \+        testq   %rsi, %rsi ;                                               \+        cmovs   %rbp, %r8 ;                                                \+        testq   $0x1, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        sarq    $1, %rcx ;                                                 \+        movl    $0x100000, %eax ;                                          \+        leaq    (%rbx,%rax), %r8 ;                                          \+        leaq    (%rcx,%rax), %r12 ;                                         \+        shlq    $0x15, %r8 ;                                               \+        shlq    $0x15, %r12 ;                                              \+        sarq    $0x2b, %r8 ;                                               \+        sarq    $0x2b, %r12 ;                                              \+        movabsq $0x20000100000, %rax ;                                     \+        leaq    (%rbx,%rax), %r10 ;                                         \+        leaq    (%rcx,%rax), %r14 ;                                         \+        sarq    $0x2b, %r10 ;                                              \+        sarq    $0x2b, %r14 ;                                              \+        movq    %r9, %rax ;                                                \+        imulq   %r8, %rax ;                                                \+        movq    %r13, %rdx ;                                               \+        imulq   %r10, %rdx ;                                               \+        imulq   %r15, %r8 ;                                                \+        imulq   %r11, %r10 ;                                               \+        addq    %r8, %r10 ;                                                \+        leaq    (%rax,%rdx), %r8 ;                                          \+        movq    %r9, %rax ;                                                \+        imulq   %r12, %rax ;                                               \+        movq    %r13, %rdx ;                                               \+        imulq   %r14, %rdx ;                                               \+        imulq   %r15, %r12 ;                                               \+        imulq   %r11, %r14 ;                                               \+        addq    %r12, %r14 ;                                               \+        leaq    (%rax,%rdx), %r12++S2N_BN_SYMBOL(bignum_inv_p521):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Save registers and make room for temporaries++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++        CFI_DEC_RSP(NSPACE)++// Save the return pointer for the end so we can overwrite %rdi later++        movq    %rdi, res++// Copy the prime p_521 = 2^521 - 1 into the f variable++        xorl    %eax, %eax+        notq    %rax+        movq    %rax, F(%rsp)+        movq    %rax, F+8(%rsp)+        movq    %rax, F+16(%rsp)+        movq    %rax, F+24(%rsp)+        movq    %rax, F+32(%rsp)+        movq    %rax, F+40(%rsp)+        movq    %rax, F+48(%rsp)+        movq    %rax, F+56(%rsp)+        movl    $0x1FF, %eax+        movq    %rax, F+64(%rsp)++// Copy the input into the g variable, but reduce it strictly mod p_521+// so that g <= f as assumed in the bound proof. This code fragment is+// very similar to bignum_mod_p521_9.++        movq    64(%rsi), %r8+        movl    $0x1FF, %ebx+        andq    %r8, %rbx+        shrq    $9, %r8++        stc+        adcq    (%rsi), %r8+        movq    8(%rsi), %r9+        adcq    $0, %r9+        movq    16(%rsi), %r10+        adcq    $0, %r10+        movq    24(%rsi), %r11+        adcq    $0, %r11+        movq    32(%rsi), %r12+        adcq    $0, %r12+        movq    40(%rsi), %r13+        adcq    $0, %r13+        movq    48(%rsi), %r14+        adcq    $0, %r14+        movq    56(%rsi), %r15+        adcq    $0, %r15+        adcq    $0, %rbx++        cmpq    $512, %rbx++        sbbq    $0, %r8+        movq    %r8, G(%rsp)+        sbbq    $0, %r9+        movq    %r9, G+8(%rsp)+        sbbq    $0, %r10+        movq    %r10, G+16(%rsp)+        sbbq    $0, %r11+        movq    %r11, G+24(%rsp)+        sbbq    $0, %r12+        movq    %r12, G+32(%rsp)+        sbbq    $0, %r13+        movq    %r13, G+40(%rsp)+        sbbq    $0, %r14+        movq    %r14, G+48(%rsp)+        sbbq    $0, %r15+        movq    %r15, G+56(%rsp)+        sbbq    $0, %rbx+        andq    $0x1FF, %rbx+        movq    %rbx, G+64(%rsp)++// Also maintain weakly reduced < 2*p_521 vector [u,v] such that+// [f,g] == x * 2^{1239-59*i} * [u,v] (mod p_521)+// starting with [p_521,x] == x * 2^{1239-59*0} * [0,2^-1239] (mod p_521)+// Note that because (2^{a+521} == 2^a) (mod p_521) we simply have+// (2^-1239 == 2^324) (mod p_521) so the constant initializer is simple.+//+// Based on the standard divstep bound, for inputs <= 2^b we need at least+// n >= (9437 * b + 1) / 4096. Since b is 521, that means 1201 iterations.+// Since we package divstep in multiples of 59 bits, we do 21 blocks of 59+// making *1239* total. (With a bit more effort we could avoid the full 59+// divsteps and use a shorter tail computation, but we keep it simple.)+// Hence, after the 21st iteration we have [f,g] == x * [u,v] and since+// |f| = 1 we get the modular inverse from u by flipping its sign with f.++        xorl    %eax, %eax+        movq    %rax, U(%rsp)+        movq    %rax, U+8(%rsp)+        movq    %rax, U+16(%rsp)+        movq    %rax, U+24(%rsp)+        movq    %rax, U+32(%rsp)+        movq    %rax, U+40(%rsp)+        movq    %rax, U+48(%rsp)+        movq    %rax, U+56(%rsp)+        movq    %rax, U+64(%rsp)++        movl    $16, %ebx+        movq    %rax, V(%rsp)+        movq    %rax, V+8(%rsp)+        movq    %rax, V+16(%rsp)+        movq    %rax, V+24(%rsp)+        movq    %rax, V+32(%rsp)+        movq    %rbx, V+40(%rsp)+        movq    %rax, V+48(%rsp)+        movq    %rax, V+56(%rsp)+        movq    %rax, V+64(%rsp)++// Start of main loop. We jump into the middle so that the divstep+// portion is common to the special 21st iteration after a uniform+// first 20.++        movq    $21, i+        movq    $1, d+        jmp     Lbignum_inv_p521_midloop++Lbignum_inv_p521_loop:++// Separate out the matrix into sign-magnitude pairs++        movq    %r8, %r9+        sarq    $63, %r9+        xorq    %r9, %r8+        subq    %r9, %r8++        movq    %r10, %r11+        sarq    $63, %r11+        xorq    %r11, %r10+        subq    %r11, %r10++        movq    %r12, %r13+        sarq    $63, %r13+        xorq    %r13, %r12+        subq    %r13, %r12++        movq    %r14, %r15+        sarq    $63, %r15+        xorq    %r15, %r14+        subq    %r15, %r14++// Adjust the initial values to allow for complement instead of negation+// This initial offset is the same for [f,g] and [u,v] compositions.+// Save it in temporary storage for the [u,v] part and do [f,g] first.++        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %rdi+        andq    %r11, %rdi+        addq    %rax, %rdi+        movq    %rdi, tmp++        movq    %r12, %rax+        andq    %r13, %rax+        movq    %r14, %rsi+        andq    %r15, %rsi+        addq    %rax, %rsi+        movq    %rsi, tmp2++// Now the computation of the updated f and g values. This maintains a+// 2-word carry between stages so we can conveniently insert the shift+// right by 59 before storing back, and not overwrite digits we need+// again of the old f and g values.+//+// Digit 0 of [f,g]++        xorl    %ebx, %ebx+        movq    F(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    G(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rdi+        adcq    %rdx, %rbx++        xorl    %ebp, %ebp+        movq    F(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    G(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp++// Digit 1 of [f,g]++        xorl    %ecx, %ecx+        movq    F+N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    G+N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $59, %rbx, %rdi+        movq    %rdi, F(%rsp)++        xorl    %edi, %edi+        movq    F+N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    G+N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $59, %rbp, %rsi+        movq    %rsi, G(%rsp)++// Digit 2 of [f,g]++        xorl    %esi, %esi+        movq    F+2*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    G+2*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $59, %rcx, %rbx+        movq    %rbx, F+N(%rsp)++        xorl    %ebx, %ebx+        movq    F+2*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    G+2*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $59, %rdi, %rbp+        movq    %rbp, G+N(%rsp)++// Digit 3 of [f,g]++        xorl    %ebp, %ebp+        movq    F+3*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    G+3*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        shrdq   $59, %rsi, %rcx+        movq    %rcx, F+2*N(%rsp)++        xorl    %ecx, %ecx+        movq    F+3*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    G+3*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $59, %rbx, %rdi+        movq    %rdi, G+2*N(%rsp)++// Digit 4 of [f,g]++        xorl    %edi, %edi+        movq    F+4*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    G+4*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $59, %rbp, %rsi+        movq    %rsi, F+3*N(%rsp)++        xorl    %esi, %esi+        movq    F+4*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    G+4*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $59, %rcx, %rbx+        movq    %rbx, G+3*N(%rsp)++// Digit 5 of [f,g]++        xorl    %ebx, %ebx+        movq    F+5*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    G+5*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $59, %rdi, %rbp+        movq    %rbp, F+4*N(%rsp)++        xorl    %ebp, %ebp+        movq    F+5*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    G+5*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        shrdq   $59, %rsi, %rcx+        movq    %rcx, G+4*N(%rsp)++// Digit 6 of [f,g]++        xorl    %ecx, %ecx+        movq    F+6*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    G+6*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $59, %rbx, %rdi+        movq    %rdi, F+5*N(%rsp)++        xorl    %edi, %edi+        movq    F+6*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    G+6*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $59, %rbp, %rsi+        movq    %rsi, G+5*N(%rsp)++// Digit 7 of [f,g]++        xorl    %esi, %esi+        movq    F+7*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    G+7*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $59, %rcx, %rbx+        movq    %rbx, F+6*N(%rsp)++        xorl    %ebx, %ebx+        movq    F+7*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    G+7*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $59, %rdi, %rbp+        movq    %rbp, G+6*N(%rsp)++// Digits 8 and 9 of [f,g]++        movq    F+8*N(%rsp), %rax+        xorq    %r9, %rax+        movq    %rax, %rbp+        sarq    $63, %rbp+        andq    %r8, %rbp+        negq    %rbp+        mulq    %r8+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    G+8*N(%rsp), %rax+        xorq    %r11, %rax+        movq    %rax, %rdx+        sarq    $63, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbp+        mulq    %r10+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        shrdq   $59, %rsi, %rcx+        movq    %rcx, F+7*N(%rsp)+        shrdq   $59, %rbp, %rsi++        movq    F+8*N(%rsp), %rax+        movq    %rsi, F+8*N(%rsp)++        xorq    %r13, %rax+        movq    %rax, %rsi+        sarq    $63, %rsi+        andq    %r12, %rsi+        negq    %rsi+        mulq    %r12+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        movq    G+8*N(%rsp), %rax+        xorq    %r15, %rax+        movq    %rax, %rdx+        sarq    $63, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rsi+        mulq    %r14+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        shrdq   $59, %rbx, %rdi+        movq    %rdi, G+7*N(%rsp)+        shrdq   $59, %rsi, %rbx+        movq    %rbx, G+8*N(%rsp)++// Get the initial carries back from storage and do the [u,v] accumulation++        movq    tmp, %rbx+        movq    tmp2, %rbp++// Digit 0 of [u,v]++        xorl    %ecx, %ecx+        movq    U(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    V(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx++        xorl    %esi, %esi+        movq    U(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, U(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    V(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, V(%rsp)++// Digit 1 of [u,v]++        xorl    %ebx, %ebx+        movq    U+N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    V+N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx++        xorl    %ebp, %ebp+        movq    U+N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, U+N(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    V+N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, V+N(%rsp)++// Digit 2 of [u,v]++        xorl    %ecx, %ecx+        movq    U+2*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    V+2*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx++        xorl    %esi, %esi+        movq    U+2*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, U+2*N(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    V+2*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, V+2*N(%rsp)++// Digit 3 of [u,v]++        xorl    %ebx, %ebx+        movq    U+3*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    V+3*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx++        xorl    %ebp, %ebp+        movq    U+3*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, U+3*N(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    V+3*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, V+3*N(%rsp)++// Digit 4 of [u,v]++        xorl    %ecx, %ecx+        movq    U+4*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    V+4*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx++        xorl    %esi, %esi+        movq    U+4*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, U+4*N(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    V+4*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, V+4*N(%rsp)++// Digit 5 of [u,v]++        xorl    %ebx, %ebx+        movq    U+5*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    V+5*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx++        xorl    %ebp, %ebp+        movq    U+5*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, U+5*N(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    V+5*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, V+5*N(%rsp)++// Digit 6 of [u,v]++        xorl    %ecx, %ecx+        movq    U+6*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    V+6*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx++        xorl    %esi, %esi+        movq    U+6*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, U+6*N(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    V+6*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, V+6*N(%rsp)++// Digit 7 of [u,v]++        xorl    %ebx, %ebx+        movq    U+7*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    V+7*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx++        xorl    %ebp, %ebp+        movq    U+7*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, U+7*N(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    V+7*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, V+7*N(%rsp)++// Digits 8 and 9 of u (top is unsigned)++        movq    U+8*N(%rsp), %rax+        xorq    %r9, %rax+        movq    %r9, %rcx+        andq    %r8, %rcx+        negq    %rcx+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    V+8*N(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rcx+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rcx, %rdx++// Modular reduction of u++        movq    %rdx, %rax+        shldq   $55, %rbx, %rdx+        sarq    $63, %rax+        addq    %rax, %rdx+        movq    %rdx, %rax+        shlq    $9, %rdx+        subq    %rdx, %rbx+        movq    %rax, %rdx+        sarq    $63, %rax+        movq    U(%rsp), %rcx+        addq    %rdx, %rcx+        movq    %rcx, U(%rsp)+        movq    U+N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, U+N(%rsp)+        movq    U+2*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, U+2*N(%rsp)+        movq    U+3*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, U+3*N(%rsp)+        movq    U+4*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, U+4*N(%rsp)+        movq    U+5*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, U+5*N(%rsp)+        movq    U+6*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, U+6*N(%rsp)+        movq    U+7*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, U+7*N(%rsp)+        adcq    %rax, %rbx++// Preload for last use of old u digit 8++        movq    U+8*N(%rsp), %rax+        movq    %rbx, U+8*N(%rsp)++// Digits 8 and 9 of v (top is unsigned)++        xorq    %r13, %rax+        movq    %r13, %rbx+        andq    %r12, %rbx+        negq    %rbx+        mulq    %r12+        addq    %rax, %rbp+        adcq    %rdx, %rbx+        movq    V+8*N(%rsp), %rax+        xorq    %r15, %rax+        movq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rbx+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rbx, %rdx++// Modular reduction of v++        movq    %rdx, %rax+        shldq   $55, %rbp, %rdx+        sarq    $63, %rax+        addq    %rax, %rdx+        movq    %rdx, %rax+        shlq    $9, %rdx+        subq    %rdx, %rbp+        movq    %rax, %rdx+        sarq    $63, %rax+        movq    V(%rsp), %rcx+        addq    %rdx, %rcx+        movq    %rcx, V(%rsp)+        movq    V+N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, V+N(%rsp)+        movq    V+2*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, V+2*N(%rsp)+        movq    V+3*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, V+3*N(%rsp)+        movq    V+4*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, V+4*N(%rsp)+        movq    V+5*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, V+5*N(%rsp)+        movq    V+6*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, V+6*N(%rsp)+        movq    V+7*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, V+7*N(%rsp)+        adcq    %rax, %rbp+        movq    %rbp, V+8*N(%rsp)++Lbignum_inv_p521_midloop:++        divstep59(d,ff,gg)+        movq    %rsi, d++// Next iteration++        decq    i+        jnz     Lbignum_inv_p521_loop++// The 21st and last iteration does not need anything except the+// u value and the sign of f; the latter can be obtained from the+// lowest word of f. So it's done differently from the main loop.+// Find the sign of the new f. For this we just need one digit+// since we know (for in-scope cases) that f is either +1 or -1.+// We don't explicitly shift right by 59 either, but looking at+// bit 63 (or any bit >= 60) of the unshifted result is enough+// to distinguish -1 from +1; this is then made into a mask.++        movq    F(%rsp), %rax+        movq    G(%rsp), %rcx+        imulq   %r8, %rax+        imulq   %r10, %rcx+        addq    %rcx, %rax+        sarq    $63, %rax++// Now separate out the matrix into sign-magnitude pairs+// and adjust each one based on the sign of f.+//+// Note that at this point we expect |f|=1 and we got its+// sign above, so then since [f,0] == x * [u,v] (mod p_521)+// we want to flip the sign of u according to that of f.++        movq    %r8, %r9+        sarq    $63, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        xorq    %rax, %r9++        movq    %r10, %r11+        sarq    $63, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        xorq    %rax, %r11++        movq    %r12, %r13+        sarq    $63, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        xorq    %rax, %r13++        movq    %r14, %r15+        sarq    $63, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        xorq    %rax, %r15++// Adjust the initial value to allow for complement instead of negation++        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %rbx+        andq    %r11, %rbx+        addq    %rax, %rbx++// Digit 0 of u++        xorl    %ecx, %ecx+        movq    U(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    V(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        movq    %rbx, U(%rsp)+        adcq    %rdx, %rcx++// Digit 1 of u++        xorl    %ebx, %ebx+        movq    U+N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    V+N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        movq    %rcx, U+N(%rsp)+        adcq    %rdx, %rbx++// Digit 2 of u++        xorl    %ecx, %ecx+        movq    U+2*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    V+2*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        movq    %rbx, U+2*N(%rsp)+        adcq    %rdx, %rcx++// Digit 3 of u++        xorl    %ebx, %ebx+        movq    U+3*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    V+3*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        movq    %rcx, U+3*N(%rsp)+        adcq    %rdx, %rbx++// Digit 4 of u++        xorl    %ecx, %ecx+        movq    U+4*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    V+4*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        movq    %rbx, U+4*N(%rsp)+        adcq    %rdx, %rcx++// Digit 5 of u++        xorl    %ebx, %ebx+        movq    U+5*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    V+5*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        movq    %rcx, U+5*N(%rsp)+        adcq    %rdx, %rbx++// Digit 6 of u++        xorl    %ecx, %ecx+        movq    U+6*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    V+6*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        movq    %rbx, U+6*N(%rsp)+        adcq    %rdx, %rcx++// Digit 7 of u++        xorl    %ebx, %ebx+        movq    U+7*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    V+7*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        movq    %rcx, U+7*N(%rsp)+        adcq    %rdx, %rbx++// Digits 8 and 9 of u (top is unsigned)++        movq    U+8*N(%rsp), %rax+        xorq    %r9, %rax+        movq    %r9, %rcx+        andq    %r8, %rcx+        negq    %rcx+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    V+8*N(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rcx+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rcx, %rdx++// Modular reduction of u++        movq    %rdx, %rax+        shldq   $55, %rbx, %rdx+        sarq    $63, %rax+        addq    %rax, %rdx+        movq    %rdx, %rax+        shlq    $9, %rdx+        subq    %rdx, %rbx+        movq    %rax, %rdx+        sarq    $63, %rax+        movq    U(%rsp), %rcx+        addq    %rdx, %rcx+        movq    %rcx, U(%rsp)+        movq    U+N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, U+N(%rsp)+        movq    U+2*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, U+2*N(%rsp)+        movq    U+3*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, U+3*N(%rsp)+        movq    U+4*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, U+4*N(%rsp)+        movq    U+5*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, U+5*N(%rsp)+        movq    U+6*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, U+6*N(%rsp)+        movq    U+7*N(%rsp), %rcx+        adcq    %rax, %rcx+        movq    %rcx, U+7*N(%rsp)+        adcq    %rax, %rbx+        movq    %rbx, U+8*N(%rsp)++// Further strict reduction ready for the output, which just means+// a conditional subtraction of p_521++        xorl    %eax, %eax+        notq    %rax+        movq    U(%rsp), %r8+        subq    %rax, %r8+        movq    U+N(%rsp), %r9+        sbbq    %rax, %r9+        movq    U+2*N(%rsp), %r10+        sbbq    %rax, %r10+        movq    U+3*N(%rsp), %r11+        sbbq    %rax, %r11+        movq    U+4*N(%rsp), %r12+        sbbq    %rax, %r12+        movq    U+5*N(%rsp), %r13+        sbbq    %rax, %r13+        movq    U+6*N(%rsp), %r14+        sbbq    %rax, %r14+        movq    U+7*N(%rsp), %r15+        sbbq    %rax, %r15+        movl    $0x1FF, %eax+        movq    U+8*N(%rsp), %rbp+        sbbq    %rax, %rbp++        cmovcq  U(%rsp), %r8+        cmovcq  U+N(%rsp), %r9+        cmovcq  U+2*N(%rsp), %r10+        cmovcq  U+3*N(%rsp), %r11+        cmovcq  U+4*N(%rsp), %r12+        cmovcq  U+5*N(%rsp), %r13+        cmovcq  U+6*N(%rsp), %r14+        cmovcq  U+7*N(%rsp), %r15+        cmovcq  U+8*N(%rsp), %rbp++// Store it back to the final output++        movq    res, %rdi+        movq    %r8, (%rdi)+        movq    %r9, N(%rdi)+        movq    %r10, 2*N(%rdi)+        movq    %r11, 3*N(%rdi)+        movq    %r12, 4*N(%rdi)+        movq    %r13, 5*N(%rdi)+        movq    %r14, 6*N(%rdi)+        movq    %r15, 7*N(%rdi)+        movq    %rbp, 8*N(%rdi)++// Restore stack and registers++        CFI_INC_RSP(NSPACE)++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_inv_p521)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/bignum_kmul_16_32.S view
@@ -0,0 +1,513 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Multiply z := x * y+// Inputs x[16], y[16]; output z[32]; temporary buffer t[>=32]+//+//    extern void bignum_kmul_16_32(uint64_t z[static 32],+//                                  const uint64_t x[static 16],+//                                  const uint64_t y[static 16],+//                                  uint64_t t[static 32]);+//+// In this x86 code the final temporary space argument t is unused, but+// it is retained in the prototype above for API consistency with ARM.+//+// Standard x86-64 ABI: RDI = z, RSI = x, RDX = y, RCX = t+// Microsoft x64 ABI:   RCX = z, RDX = x, R8 = y, R9 = t+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_kmul_16_32)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_kmul_16_32)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_kmul_16_32)+        .text++// These parameters are kept where they come in++#define z %rdi+#define x %rsi++// This one gets moved to free up %rdx for muls++#define y %rcx++// Often used for zero++#define zero %rbp+#define zeroe %ebp++// mulpadd i, j adds x[i] * rdx (now assumed = y[j]) into the window at i+j++.macro mulpadd arg1,arg2+        mulxq   8*\arg1(x), %rax, %rbx+.if ((\arg1 + \arg2) % 8 == 0)+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+.elseif ((\arg1 + \arg2) % 8 == 1)+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+.elseif ((\arg1 + \arg2) % 8 == 2)+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+.elseif ((\arg1 + \arg2) % 8 == 3)+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+.elseif ((\arg1 + \arg2) % 8 == 4)+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+.elseif ((\arg1 + \arg2) % 8 == 5)+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+.elseif ((\arg1 + \arg2) % 8 == 6)+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+.elseif ((\arg1 + \arg2) % 8 == 7)+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+.endif++.endm++// mulpade i, j adds x[i] * rdx (now assumed = y[j]) into the window at i+j+// but re-creates the top word assuming nothing to add there++.macro mulpade arg1,arg2+.if ((\arg1 + \arg2) % 8 == 0)+        mulxq   8*\arg1(x), %rax, %r9+        adcxq   %rax, %r8+        adoxq   zero, %r9+.elseif ((\arg1 + \arg2) % 8 == 1)+        mulxq   8*\arg1(x), %rax, %r10+        adcxq   %rax, %r9+        adoxq   zero, %r10+.elseif ((\arg1 + \arg2) % 8 == 2)+        mulxq   8*\arg1(x), %rax, %r11+        adcxq   %rax, %r10+        adoxq   zero, %r11+.elseif ((\arg1 + \arg2) % 8 == 3)+        mulxq   8*\arg1(x), %rax, %r12+        adcxq   %rax, %r11+        adoxq   zero, %r12+.elseif ((\arg1 + \arg2) % 8 == 4)+        mulxq   8*\arg1(x), %rax, %r13+        adcxq   %rax, %r12+        adoxq   zero, %r13+.elseif ((\arg1 + \arg2) % 8 == 5)+        mulxq   8*\arg1(x), %rax, %r14+        adcxq   %rax, %r13+        adoxq   zero, %r14+.elseif ((\arg1 + \arg2) % 8 == 6)+        mulxq   8*\arg1(x), %rax, %r15+        adcxq   %rax, %r14+        adoxq   zero, %r15+.elseif ((\arg1 + \arg2) % 8 == 7)+        mulxq   8*\arg1(x), %rax, %r8+        adcxq   %rax, %r15+        adoxq   zero, %r8+.endif++.endm++// addrow i adds z[i] + x[0..7] * y[i] into the window++.macro addrow arg1+        movq    8*\arg1(y), %rdx+        xorl    zeroe, zeroe++.if (\arg1 % 8 == 0)+        adoxq   8*\arg1(z), %r8+.elseif (\arg1 % 8 == 1)+        adoxq   8*\arg1(z), %r9+.elseif (\arg1 % 8 == 2)+        adoxq   8*\arg1(z), %r10+.elseif (\arg1 % 8 == 3)+        adoxq   8*\arg1(z), %r11+.elseif (\arg1 % 8 == 4)+        adoxq   8*\arg1(z), %r12+.elseif (\arg1 % 8 == 5)+        adoxq   8*\arg1(z), %r13+.elseif (\arg1 % 8 == 6)+        adoxq   8*\arg1(z), %r14+.elseif (\arg1 % 8 == 7)+        adoxq   8*\arg1(z), %r15+.endif++        mulpadd 0, \arg1++.if (\arg1 % 8 == 0)+        movq    %r8, 8*\arg1(z)+.elseif (\arg1 % 8 == 1)+        movq    %r9, 8*\arg1(z)+.elseif (\arg1 % 8 == 2)+        movq    %r10, 8*\arg1(z)+.elseif (\arg1 % 8 == 3)+        movq    %r11, 8*\arg1(z)+.elseif (\arg1 % 8 == 4)+        movq    %r12, 8*\arg1(z)+.elseif (\arg1 % 8 == 5)+        movq    %r13, 8*\arg1(z)+.elseif (\arg1 % 8 == 6)+        movq    %r14, 8*\arg1(z)+.elseif (\arg1 % 8 == 7)+        movq    %r15, 8*\arg1(z)+.endif++        mulpadd 1, \arg1+        mulpadd 2, \arg1+        mulpadd 3, \arg1+        mulpadd 4, \arg1+        mulpadd 5, \arg1+        mulpadd 6, \arg1+        mulpade 7, \arg1++.if (\arg1 % 8 == 0)+        adcq    zero, %r8+.elseif (\arg1 % 8 == 1)+        adcq    zero, %r9+.elseif (\arg1 % 8 == 2)+        adcq    zero, %r10+.elseif (\arg1 % 8 == 3)+        adcq    zero, %r11+.elseif (\arg1 % 8 == 4)+        adcq    zero, %r12+.elseif (\arg1 % 8 == 5)+        adcq    zero, %r13+.elseif (\arg1 % 8 == 6)+        adcq    zero, %r14+.elseif (\arg1 % 8 == 7)+        adcq    zero, %r15+.endif++.endm++// Special zero version of addrow, setting up the window from scratch++.macro addrowz+        movq    (y), %rdx+        xorl    zeroe, zeroe++        mulxq   (x), %rax, %r9+        adcq    %rax, (z)++        mulxq   8(x), %rax, %r10+        adcq    %rax, %r9++        mulxq   16(x), %rax, %r11+        adcq    %rax, %r10++        mulxq   24(x), %rax, %r12+        adcq    %rax, %r11++        mulxq   32(x), %rax, %r13+        adcq    %rax, %r12++        mulxq   40(x), %rax, %r14+        adcq    %rax, %r13++        mulxq   48(x), %rax, %r15+        adcq    %rax, %r14++        mulxq   56(x), %rax, %r8+        adcq    %rax, %r15++        adcq    zero, %r8+.endm++// This is a variant where we add the initial z[0..7] at the outset.+// This makes the initialization process a bit less wasteful. By doing+// a block of 8 we get the same effect except that we add z[0..7]+//+// adurow i adds 2^{7*64} * z[i+7] + x[0..7] * y[i] into the window++.macro adurow arg1+        movq    8*\arg1(y), %rdx+        xorl    zeroe, zeroe++        mulpadd 0, \arg1++.if (\arg1 % 8 == 0)+        movq    %r8, 8*\arg1(z)+.elseif (\arg1 % 8 == 1)+        movq    %r9, 8*\arg1(z)+.elseif (\arg1 % 8 == 2)+        movq    %r10, 8*\arg1(z)+.elseif (\arg1 % 8 == 3)+        movq    %r11, 8*\arg1(z)+.elseif (\arg1 % 8 == 4)+        movq    %r12, 8*\arg1(z)+.elseif (\arg1 % 8 == 5)+        movq    %r13, 8*\arg1(z)+.elseif (\arg1 % 8 == 6)+        movq    %r14, 8*\arg1(z)+.elseif (\arg1 % 8 == 7)+        movq    %r15, 8*\arg1(z)+.endif++        mulpadd 1, \arg1+        mulpadd 2, \arg1+        mulpadd 3, \arg1+        mulpadd 4, \arg1+        mulpadd 5, \arg1+        mulpadd 6, \arg1+        mulpade 7, \arg1++.if (\arg1 % 8 == 0)+        adcq    zero, %r8+.elseif (\arg1 % 8 == 1)+        adcq    zero, %r9+.elseif (\arg1 % 8 == 2)+        adcq    zero, %r10+.elseif (\arg1 % 8 == 3)+        adcq    zero, %r11+.elseif (\arg1 % 8 == 4)+        adcq    zero, %r12+.elseif (\arg1 % 8 == 5)+        adcq    zero, %r13+.elseif (\arg1 % 8 == 6)+        adcq    zero, %r14+.elseif (\arg1 % 8 == 7)+        adcq    zero, %r15+.endif++.endm++// Special "adurow 0" case to do first stage++.macro adurowz+        movq    (y), %rdx+        xorl    zeroe, zeroe++        movq    (z), %r8+        movq    8(z), %r9++        mulpadd 0, 0+        movq    %r8, (z)++        movq    16(z), %r10+        mulpadd 1, 0+        movq    24(z), %r11+        mulpadd 2, 0+        movq    32(z), %r12+        mulpadd 3, 0+        movq    40(z), %r13+        mulpadd 4, 0+        movq    48(z), %r14+        mulpadd 5, 0+        movq    56(z), %r15+        mulpadd 6, 0++        mulxq   56(x), %rax, %r8+        adcxq   %rax, %r15+        adoxq   zero, %r8+        adcxq   zero, %r8+.endm++// Multiply-add: z := z + x[0..7] * y++.macro addrows+        adurowz+        adurow  1+        adurow  2+        adurow  3+        adurow  4+        adurow  5+        adurow  6+        adurow  7+        addrow  8+        addrow  9+        addrow  10+        addrow  11+        addrow  12+        addrow  13+        addrow  14+        addrow  15++        movq    %r8, 128(z)+        movq    %r9, 136(z)+        movq    %r10, 144(z)+        movq    %r11, 152(z)+        movq    %r12, 160(z)+        movq    %r13, 168(z)+        movq    %r14, 176(z)+        movq    %r15, 184(z)++.endm++// mulrow i adds x[0..7] * y[i] into the window+// just like addrow but no addition of z[i]++.macro mulrow arg1+        movq    8*\arg1(y), %rdx+        xorl    zeroe, zeroe++        mulpadd 0, \arg1++.if (\arg1 % 8 == 0)+        movq    %r8, 8*\arg1(z)+.elseif (\arg1 % 8 == 1)+        movq    %r9, 8*\arg1(z)+.elseif (\arg1 % 8 == 2)+        movq    %r10, 8*\arg1(z)+.elseif (\arg1 % 8 == 3)+        movq    %r11, 8*\arg1(z)+.elseif (\arg1 % 8 == 4)+        movq    %r12, 8*\arg1(z)+.elseif (\arg1 % 8 == 5)+        movq    %r13, 8*\arg1(z)+.elseif (\arg1 % 8 == 6)+        movq    %r14, 8*\arg1(z)+.elseif (\arg1 % 8 == 7)+        movq    %r15, 8*\arg1(z)+.endif++        mulpadd 1, \arg1+        mulpadd 2, \arg1+        mulpadd 3, \arg1+        mulpadd 4, \arg1+        mulpadd 5, \arg1+        mulpadd 6, \arg1+        mulpade 7, \arg1++.if (\arg1 % 8 == 0)+        adcq    zero, %r8+.elseif (\arg1 % 8 == 1)+        adcq    zero, %r9+.elseif (\arg1 % 8 == 2)+        adcq    zero, %r10+.elseif (\arg1 % 8 == 3)+        adcq    zero, %r11+.elseif (\arg1 % 8 == 4)+        adcq    zero, %r12+.elseif (\arg1 % 8 == 5)+        adcq    zero, %r13+.elseif (\arg1 % 8 == 6)+        adcq    zero, %r14+.elseif (\arg1 % 8 == 7)+        adcq    zero, %r15+.endif+++.endm++// Special zero version of mulrow, setting up the window from scratch++.macro mulrowz+        movq    (y), %rdx+        xorl    zeroe, zeroe++        mulxq   (x), %rax, %r9+        movq    %rax, (z)++        mulxq   8(x), %rax, %r10+        adcxq    %rax, %r9++        mulxq   16(x), %rax, %r11+        adcxq   %rax, %r10++        mulxq   24(x), %rax, %r12+        adcxq   %rax, %r11++        mulxq   32(x), %rax, %r13+        adcxq   %rax, %r12++        mulxq   40(x), %rax, %r14+        adcxq   %rax, %r13++        mulxq   48(x), %rax, %r15+        adcxq   %rax, %r14++        mulxq   56(x), %rax, %r8+        adcxq   %rax, %r15++        adcq    zero, %r8+.endm++// Multiply-add: z := x[0..7] * y plus window++.macro mulrows+        mulrowz+        mulrow  1+        mulrow  2+        mulrow  3+        mulrow  4+        mulrow  5+        mulrow  6+        mulrow  7++        mulrow  8+        mulrow  9+        mulrow  10+        mulrow  11+        mulrow  12+        mulrow  13+        mulrow  14+        mulrow  15++        movq    %r8, 128(z)+        movq    %r9, 136(z)+        movq    %r10, 144(z)+        movq    %r11, 152(z)+        movq    %r12, 160(z)+        movq    %r13, 168(z)+        movq    %r14, 176(z)+        movq    %r15, 184(z)++.endm+++S2N_BN_SYMBOL(bignum_kmul_16_32):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+#endif++// Save more registers to play with++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++// Move y into its permanent home, freeing up %rdx for its special role in muls++        movq    %rdx, y++// Do the zeroth row as a pure product then the next as multiply-add++        mulrows++        addq    $64, z+        addq    $64, x+        addrows++// Restore registers and return++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_kmul_16_32)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_kmul_32_64.S view
@@ -0,0 +1,1161 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Multiply z := x * y+// Inputs x[32], y[32]; output z[64]; temporary buffer t[>=96]+//+//    extern void bignum_kmul_32_64(uint64_t z[static 64],+//                                  const uint64_t x[static 32],+//                                  const uint64_t y[static 32],+//                                  uint64_t t[static 96]);+//+// This is a Karatsuba-style function multiplying half-sized results+// internally and using temporary buffer t for intermediate results. The size+// of 96 is an overstatement for compatibility with the ARM version; it+// actually only uses 65 elements of t (64 + 1 for a stashed sign).+//+// Standard x86-64 ABI: RDI = z, RSI = x, RDX = y, RCX = t+// Microsoft x64 ABI:   RCX = z, RDX = x, R8 = y, R9 = t+// -----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_kmul_32_64)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_kmul_32_64)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_kmul_32_64)+++        .text++#define K 16++#define z %rdi+#define x %rsi+#define y %rcx++#define s %r9++// We re-use the y variable to point at t later on, when this seems clearer++#define t %rcx++S2N_BN_SYMBOL(bignum_kmul_32_64):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+        movq    %r9, %rcx+#endif++// Save callee-saved registers and also push t onto the stack; we'll+// use this space to back up both t and later z. Then move the y variable+// into its longer-term home for the first few stages.++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++        CFI_PUSH(%rcx)+        movq    %rdx, y++// Multiply the low halves++        CFI_CALL(Lbignum_kmul_32_64_local_bignum_kmul_16_32)++// Multiply the high halves++        leaq    16*K-0x40(%rdi), %rdi+        leaq    8*K-0x40(%rsi), %rsi+        leaq    8*K(%rcx), %rcx+        CFI_CALL(Lbignum_kmul_32_64_local_bignum_kmul_16_32)++// Establish %r8 as the t pointer and use the cell to back up z now++        movq    (%rsp), %r8+        subq    $16*K+0x40, %rdi+        movq    %rdi, (%rsp)++// Form |x_lo - x_hi| starting at t++        movq    -8*K-0x40(%rsi), %rax+        subq    -8*K-0x40+8*K(%rsi), %rax+        movq    %rax, (%r8)+        .set I,  1+        .rep K-1+        movq    -8*K-0x40+8*I(%rsi), %rax+        sbbq    -8*K-0x40+8*K+8*I(%rsi), %rax+        movq    %rax, 8*I(%r8)+        .set I,  (I+1)+        .endr++        movl    $0, %ebx+        sbbq    s, s // Maintain CF, set ZF for cmovs, record sign++        .set I,  0+        .rep K+        movq    8*I(%r8), %rdx+        movq    %rdx, %rax+        notq    %rdx+        cmovzq  %rax, %rdx+        adcxq   %rbx, %rdx+        movq    %rdx, 8*I(%r8)+        .set I,  (I+1)+        .endr++// Form |y_hi - y_lo| (note opposite order) starting at t[K]++        movq    -8*K+8*K(%rcx), %rax+        subq    -8*K(%rcx), %rax+        movq    %rax, 8*K(%r8)+        .set I,  1+        .rep K-1+        movq    -8*K+8*K+8*I(%rcx), %rax+        sbbq    -8*K+8*I(%rcx), %rax+        movq    %rax, 8*K+8*I(%r8)+        .set I,  (I+1)+        .endr++        movl    $0, %ebx+        sbbq    %rbp, %rbp // Maintain CF, set ZF for cmovs++        .set I,  0+        .rep K+        movq    8*K+8*I(%r8), %rdx+        movq    %rdx, %rax+        notq    %rdx+        cmovzq  %rax, %rdx+        adcxq   %rbx, %rdx+        movq    %rdx, 8*K+8*I(%r8)+        .set I,  (I+1)+        .endr++// Stash the final sign with which to add things at t[4*K]++        xorq    %rbp, s+        movq    s, 32*K(%r8)++// Multiply the absolute differences, putting the result at t[2*K]+// This has the side-effect of putting t in the "right" register %rcx+// so after the load of z, we have both z and t pointers straight.++        movq    %r8, %rcx+        leaq    8*K(%r8), %rsi+        leaq    16*K(%r8), %rdi+        CFI_CALL(Lbignum_kmul_32_64_local_bignum_kmul_16_32)+        movq    (%rsp), z++// Compose the middle parts [2,1] + [1,0] + [3,2], saving carry in %rbx.+// Put the sum at t, overwriting the absolute differences we no longer need.++        xorl    %ebx, %ebx+        .set I,  0+        .rep 2*K+        movq    8*K+8*I(z), %rax+        adcxq   8*I(z), %rax+        adoxq   16*K+8*I(z), %rax+        movq    %rax, 8*I(t)+        .set I,  (I+1)+        .endr+        adoxq   %rbx, %rbx+        adcq    $0, %rbx++// Sign-aware addition or subtraction of the complicated term.+// We double-negate it to set CF/ZF while not spoiling its+// actual form: note that we eventually adcx to it below.++        movq    32*K(t), s+        negq    s+        negq    s++        .set I,  0+        .rep 2*K+        movq    16*K+8*I(t), %rdx+        movq    %rdx, %rax+        notq    %rdx+        cmovzq  %rax, %rdx+        adcxq   8*I(t), %rdx+        movq    %rdx, 8*K+8*I(z)+        .set I,  (I+1)+        .endr++// Bump the accumulated carry. This must end up >= 0 because it's the top+// word of a value of the form ... + h * h' + l * l' - (h - l) * (h' - l') >= 0++        adcxq   s, %rbx++// Finally propagate the carry to the top part++        xorl    %eax, %eax+        addq    %rbx, 24*K(z)+        .set I,  1+        .rep K-1+        adcq    %rax, 24*K+8*I(z)+        .set I,  (I+1)+        .endr++// Restore and return. The first pop is not needed for the ABI but+// we need to adjust the stack anyway so it seems reasonable.++        CFI_POP(%rcx)++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++// Local copy of half-length subroutine. This has a slightly different+// interface, expecting y argument in %rcx directly, and not doing any+// save-restore of the other registers. It naturally moves z and x on by+// 0x40, which we compensate for when it is called by adjusting offsets.++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lbignum_kmul_32_64_local_bignum_kmul_16_32)++Lbignum_kmul_32_64_local_bignum_kmul_16_32:+        CFI_START+        movq   (%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %r9+        movq   %rax, (%rdi)+        mulxq  0x8(%rsi), %rax, %r10+        adcxq  %rax, %r9+        mulxq  0x10(%rsi), %rax, %r11+        adcxq  %rax, %r10+        mulxq  0x18(%rsi), %rax, %r12+        adcxq  %rax, %r11+        mulxq  0x20(%rsi), %rax, %r13+        adcxq  %rax, %r12+        mulxq  0x28(%rsi), %rax, %r14+        adcxq  %rax, %r13+        mulxq  0x30(%rsi), %rax, %r15+        adcxq  %rax, %r14+        mulxq  0x38(%rsi), %rax, %r8+        adcxq  %rax, %r15+        adcq   %rbp, %r8+        movq   0x8(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        movq   %r9, 0x8(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x38(%rsi), %rax, %r9+        adcxq  %rax, %r8+        adoxq  %rbp, %r9+        adcq   %rbp, %r9+        movq   0x10(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        movq   %r10, 0x10(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x38(%rsi), %rax, %r10+        adcxq  %rax, %r9+        adoxq  %rbp, %r10+        adcq   %rbp, %r10+        movq   0x18(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        movq   %r11, 0x18(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x38(%rsi), %rax, %r11+        adcxq  %rax, %r10+        adoxq  %rbp, %r11+        adcq   %rbp, %r11+        movq   0x20(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        movq   %r12, 0x20(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x38(%rsi), %rax, %r12+        adcxq  %rax, %r11+        adoxq  %rbp, %r12+        adcq   %rbp, %r12+        movq   0x28(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        movq   %r13, 0x28(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x38(%rsi), %rax, %r13+        adcxq  %rax, %r12+        adoxq  %rbp, %r13+        adcq   %rbp, %r13+        movq   0x30(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        movq   %r14, 0x30(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x38(%rsi), %rax, %r14+        adcxq  %rax, %r13+        adoxq  %rbp, %r14+        adcq   %rbp, %r14+        movq   0x38(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        movq   %r15, 0x38(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x38(%rsi), %rax, %r15+        adcxq  %rax, %r14+        adoxq  %rbp, %r15+        adcq   %rbp, %r15+        movq   0x40(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        movq   %r8, 0x40(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x38(%rsi), %rax, %r8+        adcxq  %rax, %r15+        adoxq  %rbp, %r8+        adcq   %rbp, %r8+        movq   0x48(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        movq   %r9, 0x48(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x38(%rsi), %rax, %r9+        adcxq  %rax, %r8+        adoxq  %rbp, %r9+        adcq   %rbp, %r9+        movq   0x50(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        movq   %r10, 0x50(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x38(%rsi), %rax, %r10+        adcxq  %rax, %r9+        adoxq  %rbp, %r10+        adcq   %rbp, %r10+        movq   0x58(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        movq   %r11, 0x58(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x38(%rsi), %rax, %r11+        adcxq  %rax, %r10+        adoxq  %rbp, %r11+        adcq   %rbp, %r11+        movq   0x60(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        movq   %r12, 0x60(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x38(%rsi), %rax, %r12+        adcxq  %rax, %r11+        adoxq  %rbp, %r12+        adcq   %rbp, %r12+        movq   0x68(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        movq   %r13, 0x68(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x38(%rsi), %rax, %r13+        adcxq  %rax, %r12+        adoxq  %rbp, %r13+        adcq   %rbp, %r13+        movq   0x70(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        movq   %r14, 0x70(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x38(%rsi), %rax, %r14+        adcxq  %rax, %r13+        adoxq  %rbp, %r14+        adcq   %rbp, %r14+        movq   0x78(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        movq   %r15, 0x78(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x38(%rsi), %rax, %r15+        adcxq  %rax, %r14+        adoxq  %rbp, %r15+        adcq   %rbp, %r15+        movq   %r8, 0x80(%rdi)+        movq   %r9, 0x88(%rdi)+        movq   %r10, 0x90(%rdi)+        movq   %r11, 0x98(%rdi)+        movq   %r12, 0xa0(%rdi)+        movq   %r13, 0xa8(%rdi)+        movq   %r14, 0xb0(%rdi)+        movq   %r15, 0xb8(%rdi)+        addq   $0x40, %rdi+        addq   $0x40, %rsi+        movq   (%rcx), %rdx+        xorl   %ebp, %ebp+        movq   (%rdi), %r8+        movq   0x8(%rdi), %r9+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        movq   %r8, (%rdi)+        movq   0x10(%rdi), %r10+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        movq   0x18(%rdi), %r11+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        movq   0x20(%rdi), %r12+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        movq   0x28(%rdi), %r13+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        movq   0x30(%rdi), %r14+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        movq   0x38(%rdi), %r15+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x38(%rsi), %rax, %r8+        adcxq  %rax, %r15+        adoxq  %rbp, %r8+        adcxq  %rbp, %r8+        movq   0x8(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        movq   %r9, 0x8(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x38(%rsi), %rax, %r9+        adcxq  %rax, %r8+        adoxq  %rbp, %r9+        adcq   %rbp, %r9+        movq   0x10(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        movq   %r10, 0x10(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x38(%rsi), %rax, %r10+        adcxq  %rax, %r9+        adoxq  %rbp, %r10+        adcq   %rbp, %r10+        movq   0x18(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        movq   %r11, 0x18(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x38(%rsi), %rax, %r11+        adcxq  %rax, %r10+        adoxq  %rbp, %r11+        adcq   %rbp, %r11+        movq   0x20(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        movq   %r12, 0x20(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x38(%rsi), %rax, %r12+        adcxq  %rax, %r11+        adoxq  %rbp, %r12+        adcq   %rbp, %r12+        movq   0x28(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        movq   %r13, 0x28(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x38(%rsi), %rax, %r13+        adcxq  %rax, %r12+        adoxq  %rbp, %r13+        adcq   %rbp, %r13+        movq   0x30(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        movq   %r14, 0x30(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x38(%rsi), %rax, %r14+        adcxq  %rax, %r13+        adoxq  %rbp, %r14+        adcq   %rbp, %r14+        movq   0x38(%rcx), %rdx+        xorl   %ebp, %ebp+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        movq   %r15, 0x38(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x38(%rsi), %rax, %r15+        adcxq  %rax, %r14+        adoxq  %rbp, %r15+        adcq   %rbp, %r15+        movq   0x40(%rcx), %rdx+        xorl   %ebp, %ebp+        adoxq  0x40(%rdi), %r8+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        movq   %r8, 0x40(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x38(%rsi), %rax, %r8+        adcxq  %rax, %r15+        adoxq  %rbp, %r8+        adcq   %rbp, %r8+        movq   0x48(%rcx), %rdx+        xorl   %ebp, %ebp+        adoxq  0x48(%rdi), %r9+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        movq   %r9, 0x48(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x38(%rsi), %rax, %r9+        adcxq  %rax, %r8+        adoxq  %rbp, %r9+        adcq   %rbp, %r9+        movq   0x50(%rcx), %rdx+        xorl   %ebp, %ebp+        adoxq  0x50(%rdi), %r10+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        movq   %r10, 0x50(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x38(%rsi), %rax, %r10+        adcxq  %rax, %r9+        adoxq  %rbp, %r10+        adcq   %rbp, %r10+        movq   0x58(%rcx), %rdx+        xorl   %ebp, %ebp+        adoxq  0x58(%rdi), %r11+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        movq   %r11, 0x58(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x38(%rsi), %rax, %r11+        adcxq  %rax, %r10+        adoxq  %rbp, %r11+        adcq   %rbp, %r11+        movq   0x60(%rcx), %rdx+        xorl   %ebp, %ebp+        adoxq  0x60(%rdi), %r12+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        movq   %r12, 0x60(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x38(%rsi), %rax, %r12+        adcxq  %rax, %r11+        adoxq  %rbp, %r12+        adcq   %rbp, %r12+        movq   0x68(%rcx), %rdx+        xorl   %ebp, %ebp+        adoxq  0x68(%rdi), %r13+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        movq   %r13, 0x68(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x38(%rsi), %rax, %r13+        adcxq  %rax, %r12+        adoxq  %rbp, %r13+        adcq   %rbp, %r13+        movq   0x70(%rcx), %rdx+        xorl   %ebp, %ebp+        adoxq  0x70(%rdi), %r14+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        movq   %r14, 0x70(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x38(%rsi), %rax, %r14+        adcxq  %rax, %r13+        adoxq  %rbp, %r14+        adcq   %rbp, %r14+        movq   0x78(%rcx), %rdx+        xorl   %ebp, %ebp+        adoxq  0x78(%rdi), %r15+        mulxq  (%rsi), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        movq   %r15, 0x78(%rdi)+        mulxq  0x8(%rsi), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x10(%rsi), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x18(%rsi), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x20(%rsi), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x28(%rsi), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x30(%rsi), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x38(%rsi), %rax, %r15+        adcxq  %rax, %r14+        adoxq  %rbp, %r15+        adcq   %rbp, %r15+        movq   %r8, 0x80(%rdi)+        movq   %r9, 0x88(%rdi)+        movq   %r10, 0x90(%rdi)+        movq   %r11, 0x98(%rdi)+        movq   %r12, 0xa0(%rdi)+        movq   %r13, 0xa8(%rdi)+        movq   %r14, 0xb0(%rdi)+        movq   %r15, 0xb8(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lbignum_kmul_32_64_local_bignum_kmul_16_32)++S2N_BN_SIZE_DIRECTIVE(bignum_kmul_32_64)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_ksqr_16_32.S view
@@ -0,0 +1,545 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Square, z := x^2+// Input x[16]; output z[32]; temporary buffer t[>=24]+//+//    extern void bignum_ksqr_16_32(uint64_t z[static 32],+//                                  const uint64_t x[static 16],+//                                  uint64_t t[static 24]);+//+// In this x86 code the final temporary space argument t is unused, but+// it is retained in the prototype above for API consistency with ARM.+//+// Standard x86-64 ABI: RDI = z, RSI = x, RDX = t+// Microsoft x64 ABI:   RCX = z, RDX = x, R8 = t+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_ksqr_16_32)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_ksqr_16_32)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_ksqr_16_32)+        .text++#define z %rdi+#define x %rsi++// A zero register++#define zero %rbp+#define zeroe %ebp++// ------------------------------------------------------------------------+// mulpadd i, j adds rdx * x[i] into the window  at the i+j point+// ------------------------------------------------------------------------++.macro mulpadd arg1,arg2+        mulxq   8*\arg1(x), %rax, %rcx+.if ((\arg1 + \arg2) % 8 == 0)+        adcxq   %rax, %r8+        adoxq   %rcx, %r9+.elseif ((\arg1 + \arg2) % 8 == 1)+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+.elseif ((\arg1 + \arg2) % 8 == 2)+        adcxq   %rax, %r10+        adoxq   %rcx, %r11+.elseif ((\arg1 + \arg2) % 8 == 3)+        adcxq   %rax, %r11+        adoxq   %rcx, %r12+.elseif ((\arg1 + \arg2) % 8 == 4)+        adcxq   %rax, %r12+        adoxq   %rcx, %r13+.elseif ((\arg1 + \arg2) % 8 == 5)+        adcxq   %rax, %r13+        adoxq   %rcx, %r14+.elseif ((\arg1 + \arg2) % 8 == 6)+        adcxq   %rax, %r14+        adoxq   %rcx, %r15+.elseif ((\arg1 + \arg2) % 8 == 7)+        adcxq   %rax, %r15+        adoxq   %rcx, %r8+.endif++.endm++// ------------------------------------------------------------------------+// mulpade i, j adds rdx * x[i] into the window at i+j+// but re-creates the top word assuming nothing to add there+// ------------------------------------------------------------------------++.macro mulpade arg1,arg2+.if ((\arg1 + \arg2) % 8 == 0)+        mulxq   8*\arg1(x), %rax, %r9+        adcxq   %rax, %r8+        adoxq   zero, %r9+.elseif ((\arg1 + \arg2) % 8 == 1)+        mulxq   8*\arg1(x), %rax, %r10+        adcxq   %rax, %r9+        adoxq   zero, %r10+.elseif ((\arg1 + \arg2) % 8 == 2)+        mulxq   8*\arg1(x), %rax, %r11+        adcxq   %rax, %r10+        adoxq   zero, %r11+.elseif ((\arg1 + \arg2) % 8 == 3)+        mulxq   8*\arg1(x), %rax, %r12+        adcxq   %rax, %r11+        adoxq   zero, %r12+.elseif ((\arg1 + \arg2) % 8 == 4)+        mulxq   8*\arg1(x), %rax, %r13+        adcxq   %rax, %r12+        adoxq   zero, %r13+.elseif ((\arg1 + \arg2) % 8 == 5)+        mulxq   8*\arg1(x), %rax, %r14+        adcxq   %rax, %r13+        adoxq   zero, %r14+.elseif ((\arg1 + \arg2) % 8 == 6)+        mulxq   8*\arg1(x), %rax, %r15+        adcxq   %rax, %r14+        adoxq   zero, %r15+.elseif ((\arg1 + \arg2) % 8 == 7)+        mulxq   8*\arg1(x), %rax, %r8+        adcxq   %rax, %r15+        adoxq   zero, %r8+.endif++.endm++// ------------------------------------------------------------------------+// addrow i,j adds z[i+j] + x[i..i+7] * x[j] into the window+// ------------------------------------------------------------------------++.macro addrow arg1,arg2+        movq    8*\arg2(x), %rdx+        xorl    zeroe, zeroe // Get a known flag state and give a zero reg++.if ((\arg1 + \arg2) % 8 == 0)+        adoxq   8*(\arg1+\arg2)(z), %r8+.elseif ((\arg1 + \arg2) % 8 == 1)+        adoxq   8*(\arg1+\arg2)(z), %r9+.elseif ((\arg1 + \arg2) % 8 == 2)+        adoxq   8*(\arg1+\arg2)(z), %r10+.elseif ((\arg1 + \arg2) % 8 == 3)+        adoxq   8*(\arg1+\arg2)(z), %r11+.elseif ((\arg1 + \arg2) % 8 == 4)+        adoxq   8*(\arg1+\arg2)(z), %r12+.elseif ((\arg1 + \arg2) % 8 == 5)+        adoxq   8*(\arg1+\arg2)(z), %r13+.elseif ((\arg1 + \arg2) % 8 == 6)+        adoxq   8*(\arg1+\arg2)(z), %r14+.elseif ((\arg1 + \arg2) % 8 == 7)+        adoxq   8*(\arg1+\arg2)(z), %r15+.endif++        mulpadd \arg1, \arg2++.if ((\arg1 + \arg2) % 8 == 0)+        movq    %r8, 8*(\arg1+\arg2)(z)+.elseif ((\arg1 + \arg2) % 8 == 1)+        movq    %r9, 8*(\arg1+\arg2)(z)+.elseif ((\arg1 + \arg2) % 8 == 2)+        movq    %r10, 8*(\arg1+\arg2)(z)+.elseif ((\arg1 + \arg2) % 8 == 3)+        movq    %r11, 8*(\arg1+\arg2)(z)+.elseif ((\arg1 + \arg2) % 8 == 4)+        movq    %r12, 8*(\arg1+\arg2)(z)+.elseif ((\arg1 + \arg2) % 8 == 5)+        movq    %r13, 8*(\arg1+\arg2)(z)+.elseif ((\arg1 + \arg2) % 8 == 6)+        movq    %r14, 8*(\arg1+\arg2)(z)+.elseif ((\arg1 + \arg2) % 8 == 7)+        movq    %r15, 8*(\arg1+\arg2)(z)+.endif++        mulpadd (\arg1+1), \arg2+        mulpadd (\arg1+2), \arg2+        mulpadd (\arg1+3), \arg2+        mulpadd (\arg1+4), \arg2+        mulpadd (\arg1+5), \arg2+        mulpade (\arg1+6), \arg2+        mulpade (\arg1+7), \arg2++.if ((\arg1 + \arg2) % 8 == 0)+        adcxq   zero, %r8+.elseif ((\arg1 + \arg2) % 8 == 1)+        adcxq   zero, %r9+.elseif ((\arg1 + \arg2) % 8 == 2)+        adcxq   zero, %r10+.elseif ((\arg1 + \arg2) % 8 == 3)+        adcxq   zero, %r11+.elseif ((\arg1 + \arg2) % 8 == 4)+        adcxq   zero, %r12+.elseif ((\arg1 + \arg2) % 8 == 5)+        adcxq   zero, %r13+.elseif ((\arg1 + \arg2) % 8 == 6)+        adcxq   zero, %r14+.elseif ((\arg1 + \arg2) % 8 == 7)+        adcxq   zero, %r15+.endif+++.endm+++// ------------------------------------------------------------------------+// Adds off-diagonal part of x[i..i+7]^2 into the window, writes 0..7 back+// ------------------------------------------------------------------------++.macro sqr arg1++        xorl    zeroe, zeroe++// Set up the initial window++        movq    16*\arg1+8(z), %r9+        movq    16*\arg1+16(z), %r10+        movq    16*\arg1+24(z), %r11+        movq    16*\arg1+32(z), %r12+        movq    16*\arg1+40(z), %r13+        movq    16*\arg1+48(z), %r14+        movq    16*\arg1+56(z), %r15++// Add in the first diagonal [%r8..%r10] + 2 wb = 10 + 20 + 30 + 40 + 50 + 60 + 70++        movq    8*\arg1(x), %rdx+        mulpadd (\arg1+1), (\arg1+0)+        movq    %r9, 16*\arg1+8(z)+        mulpadd (\arg1+2), (\arg1+0)+        movq    %r10, 16*\arg1+16(z)+        mulpadd (\arg1+3), (\arg1+0)+        mulpadd (\arg1+4), (\arg1+0)+        mulpadd (\arg1+5), (\arg1+0)+        mulpadd (\arg1+6), (\arg1+0)+        mulpade (\arg1+7), (\arg1+0)+        adcxq   zero, %r8++// Add in the next diagonal = 21 + 31 + 41 + 51 + 61 + 71 + 54++        xorl    zeroe, zeroe+        movq    8*\arg1+8(x), %rdx+        mulpadd (\arg1+2), (\arg1+1)+        movq    %r11, 16*\arg1+24(z)+        mulpadd (\arg1+3), (\arg1+1)+        movq    %r12, 16*\arg1+32(z)+        mulpadd (\arg1+4), (\arg1+1)+        mulpadd (\arg1+5), (\arg1+1)+        mulpadd (\arg1+6), (\arg1+1)+        mulpade (\arg1+7), (\arg1+1)+        movq    8*\arg1+32(x), %rdx+        mulpade (\arg1+5), (\arg1+4)+        adcxq   zero, %r10++// And the next one = 32 + 42 + 52 + 62 + 72 + 64 + 65++        xorl    zeroe, zeroe+        movq    8*\arg1+16(x), %rdx+        mulpadd (\arg1+3), (\arg1+2)+        movq    %r13, 16*\arg1+40(z)+        mulpadd (\arg1+4), (\arg1+2)+        movq    %r14, 16*\arg1+48(z)+        mulpadd (\arg1+5), (\arg1+2)+        mulpadd (\arg1+6), (\arg1+2)+        mulpadd (\arg1+7), (\arg1+2)+        movq    8*\arg1+48(x), %rdx+        mulpade (\arg1+4), (\arg1+6)+        mulpade (\arg1+5), (\arg1+6)+        adcxq   zero, %r12++// And the final one = 43 + 53 + 63 + 73 + 74 + 75 + 76++        xorl    zeroe, zeroe+        movq    8*\arg1+24(x), %rdx+        mulpadd (\arg1+4), (\arg1+3)+        movq    %r15, 16*\arg1+56(z)+        mulpadd (\arg1+5), (\arg1+3)+        mulpadd (\arg1+6), (\arg1+3)+        mulpadd (\arg1+7), (\arg1+3)+        movq    8*\arg1+56(x), %rdx+        mulpadd (\arg1+4), (\arg1+7)+        mulpade (\arg1+5), (\arg1+7)+        mulpade (\arg1+6), (\arg1+7)+        adcxq   zero, %r14+.endm++// ------------------------------------------------------------------------+// Multiply-add: z := z + x[i...i+7] * x+// ------------------------------------------------------------------------++.macro addrows arg1++        sqr \arg1++        .set I,  (\arg1+8)+.rep (8-\arg1)+        addrow \arg1, I+        .set I,  (I+1)+.endr++        movq    %r8, 8*(16+\arg1)(z)+        movq    %r9, 8*(17+\arg1)(z)+        movq    %r10, 8*(18+\arg1)(z)+        movq    %r11, 8*(19+\arg1)(z)+        movq    %r12, 8*(20+\arg1)(z)+        movq    %r13, 8*(21+\arg1)(z)+        movq    %r14, 8*(22+\arg1)(z)+.endm+++// ------------------------------------------------------------------------+// mulrow i,j adds x[i..i+7] * x[j] into the window+// just like addrow but no addition of z[i+j]+// ------------------------------------------------------------------------++.macro mulrow arg1,arg2+        movq    8*\arg2(x), %rdx+        xorl    zeroe, zeroe // Get a known flag state and give a zero reg++        mulpadd \arg1, \arg2++.if ((\arg1 + \arg2) % 8 == 0)+        movq    %r8, 8*(\arg1+\arg2)(z)+.elseif ((\arg1 + \arg2) % 8 == 1)+        movq    %r9, 8*(\arg1+\arg2)(z)+.elseif ((\arg1 + \arg2) % 8 == 2)+        movq    %r10, 8*(\arg1+\arg2)(z)+.elseif ((\arg1 + \arg2) % 8 == 3)+        movq    %r11, 8*(\arg1+\arg2)(z)+.elseif ((\arg1 + \arg2) % 8 == 4)+        movq    %r12, 8*(\arg1+\arg2)(z)+.elseif ((\arg1 + \arg2) % 8 == 5)+        movq    %r13, 8*(\arg1+\arg2)(z)+.elseif ((\arg1 + \arg2) % 8 == 6)+        movq    %r14, 8*(\arg1+\arg2)(z)+.elseif ((\arg1 + \arg2) % 8 == 7)+        movq    %r15, 8*(\arg1+\arg2)(z)+.endif++        mulpadd (\arg1+1), \arg2+        mulpadd (\arg1+2), \arg2+        mulpadd (\arg1+3), \arg2+        mulpadd (\arg1+4), \arg2+        mulpadd (\arg1+5), \arg2+.if ((\arg1 + \arg2) % 8 == 0)+        mulpade (\arg1+6), \arg2+.else+        mulpadd (\arg1+6), \arg2+.endif++        mulpade (\arg1+7), \arg2++.if ((\arg1 + \arg2) % 8 == 0)+        adcxq   zero, %r8+.elseif ((\arg1 + \arg2) % 8 == 1)+        adcxq   zero, %r9+.elseif ((\arg1 + \arg2) % 8 == 2)+        adcxq   zero, %r10+.elseif ((\arg1 + \arg2) % 8 == 3)+        adcxq   zero, %r11+.elseif ((\arg1 + \arg2) % 8 == 4)+        adcxq   zero, %r12+.elseif ((\arg1 + \arg2) % 8 == 5)+        adcxq   zero, %r13+.elseif ((\arg1 + \arg2) % 8 == 6)+        adcxq   zero, %r14+.elseif ((\arg1 + \arg2) % 8 == 7)+        adcxq   zero, %r15+.endif+++.endm++// ------------------------------------------------------------------------+// Compute off-diagonal part of x[0..7]^2, write back 1..7 elements and+// set up the high part in the standard register window. DOES NOT WRITE z[0]!+// ------------------------------------------------------------------------++.macro sqrz++        xorl    zeroe, zeroe++// Set initial window [%r8..%r10] + 2 wb = 10 + 20 + 30 + 40 + 50 + 60 + 70++        movq    (x), %rdx+        mulxq   8(x), %r9, %rax+        movq    %r9, 8(z)+        mulxq   16(x), %r10, %rcx+        adcxq   %rax, %r10+        movq    %r10, 16(z)+        mulxq   24(x), %r11, %rax+        adcxq   %rcx, %r11+        mulxq   32(x), %r12, %rcx+        adcxq   %rax, %r12+        mulxq   40(x), %r13, %rax+        adcxq   %rcx, %r13+        mulxq   48(x), %r14, %rcx+        adcxq   %rax, %r14+        mulxq   56(x), %r15, %r8+        adcxq   %rcx, %r15+        adcxq   zero, %r8++// Add in the next diagonal = 21 + 31 + 41 + 51 + 61 + 71 + 54++        xorl    zeroe, zeroe+        movq    8(x), %rdx+        mulpadd 2, 1+        movq    %r11, 24(z)+        mulpadd 3, 1+        movq    %r12, 32(z)+        mulpadd 4, 1+        mulpadd 5, 1+        mulpadd 6, 1+        mulpade 7, 1+        movq    32(x), %rdx+        mulpade 5, 4+        adcxq   zero, %r10++// And the next one = 32 + 42 + 52 + 62 + 72 + 64 + 65++        xorl    zeroe, zeroe+        movq    16(x), %rdx+        mulpadd 3, 2+        movq    %r13, 40(z)+        mulpadd 4, 2+        movq    %r14, 48(z)+        mulpadd 5, 2+        mulpadd 6, 2+        mulpadd 7, 2+        movq    48(x), %rdx+        mulpade 4, 6+        mulpade 5, 6+        adcxq   zero, %r12++// And the final one = 43 + 53 + 63 + 73 + 74 + 75 + 76++        xorl    zeroe, zeroe+        movq    24(x), %rdx+        mulpadd 4, 3+        movq    %r15, 56(z)+        mulpadd 5, 3+        mulpadd 6, 3+        mulpadd 7, 3+        movq    56(x), %rdx+        mulpadd 4, 7+        mulpade 5, 7+        mulpade 6, 7+        adcxq   zero, %r14+.endm++// ------------------------------------------------------------------------+// Multiply-add: z := x[0...7] * x off-diagonal elements+// ------------------------------------------------------------------------++.macro mulrows+        sqrz++        .set I,  8+.rep 8+        mulrow 0, I+        .set I,  (I+1)+.endr++        movq    %r8, 128(z)+        movq    %r9, 136(z)+        movq    %r10, 144(z)+        movq    %r11, 152(z)+        movq    %r12, 160(z)+        movq    %r13, 168(z)+        movq    %r14, 176(z)+        movq    %r15, 184(z)+.endm++// ------------------------------------------------------------------------+// The actual code+// ------------------------------------------------------------------------++++S2N_BN_SYMBOL(bignum_ksqr_16_32):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Save more registers to play with++        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++// Now just systematically add in the rows to get all off-diagonal elements++        mulrows+        addrows 8++// Double and add the diagonal elements. Note that z[0] was never written above++        xorl    zeroe, zeroe+        movq    (x), %rdx+        mulxq   %rdx, %rax, %rcx+        movq    %rax, (z)++        movq    8(z), %rdx+        adcxq   %rdx, %rdx+        adoxq   %rcx, %rdx+        movq    %rdx, 8(z)++        .set I,  1+.rep 14+        movq    8*I(x), %rdx+        mulxq   %rdx, %rax, %rcx++        movq    8*(2*I)(z), %rdx+        adcxq   %rdx, %rdx+        adoxq   %rax, %rdx+        movq    %rdx, 8*(2*I)(z)++        movq    8*(2*I+1)(z), %rdx+        adcxq   %rdx, %rdx+        adoxq   %rcx, %rdx+        movq    %rdx, 8*(2*I+1)(z)+        .set I,  (I+1)+.endr++        movq    8*I(x), %rdx+        mulxq   %rdx, %rax, %rcx++        movq    8*(2*I)(z), %rdx+        adcxq   %rdx, %rdx+        adoxq   %rax, %rdx+        movq    %rdx, 8*(2*I)(z)++        adcxq   zero, %rcx+        adoxq   zero, %rcx+        movq    %rcx, 8*(2*I+1)(z)+        .set I,  (I+1)+++// Restore registers and return++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_ksqr_16_32)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_ksqr_32_64.S view
@@ -0,0 +1,809 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Square, z := x^2+// Input x[32]; output z[64]; temporary buffer t[>=72]+//+//    extern void bignum_ksqr_32_64(uint64_t z[static 64],+//                                  const uint64_t x[static 32],+//                                  uint64_t t[static 72]);+//+// This is a Karatsuba-style function squaring half-sized results+// and using temporary buffer t for intermediate results. The size of 72+// is an overstatement for compatibility with the ARM version; it actually+// only uses 65 elements of t (64 + 1 for a suspended carry).+//+// Standard x86-64 ABI: RDI = z, RSI = x, RDX = t+// Microsoft x64 ABI:   RCX = z, RDX = x, R8 = t+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_ksqr_32_64)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_ksqr_32_64)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_ksqr_32_64)++        .text++#define K 16++#define z %rdi+#define x %rsi+#define t %rcx++S2N_BN_SYMBOL(bignum_ksqr_32_64):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+#endif++// Save callee-preserved registers once and for all at the outset+// Later we further reshuffle the input arguments to avoid extra saves++        CFI_PUSH(%rbp)+        CFI_PUSH(%rbx)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++// Move the temp space pointer since we need %rdx for multiplications++        movq    %rdx, t++// Square the low half++        CFI_CALL(Lbignum_ksqr_32_64_local_bignum_sqr_16_32)++// Square the high half; from here on x and z are modified++        leaq    8*K(x), x // input at x+8*K+        leaq    16*K(z), z // result at z+16*K+        CFI_CALL(Lbignum_ksqr_32_64_local_bignum_sqr_16_32)++// Form |x_lo - x_hi|, stored at t++        movq    -8*K(x), %rax+        subq    (x), %rax+        movq    %rax, (t)+        .set I,  1+        .rep K-1+        movq    -8*K+8*I(x), %rax+        sbbq    8*I(x), %rax+        movq    %rax, 8*I(t)+        .set I,  (I+1)+        .endr++        movl    $0, %ebx+        sbbq    %rax, %rax // Maintain CF, set ZF for cmovs++        .set I,  0+        .rep K+        movq    8*I(t), %rdx+        movq    %rdx, %rax+        notq    %rdx+        adcxq   %rbx, %rdx+        cmovzq  %rax, %rdx+        movq    %rdx, 8*I(t)+        .set I,  (I+1)+        .endr++// Compose the middle parts [2,1] + [1,0] + [3,2]+// Put the low half of this at t[K] and the top half in place at z[2*K]; a+// fully in-place version is awkward with the otherwise beneficial double+// carry chain. Stash the carry suspended from the 3k position at the end of+// the temp buffer t[4*K].++        xorl    %edx, %edx+        .set I,  0+        .rep K+        movq    -16*K+8*K+8*I(z), %rax+        adcxq   -16*K+8*I(z), %rax+        adoxq   -16*K+16*K+8*I(z), %rax+        movq    %rax, 8*K+8*I(t)+        .set I,  (I+1)+        .endr++        .rep K+        movq    -16*K+8*K+8*I(z), %rax+        adcxq   -16*K+8*I(z), %rax+        adoxq   -16*K+16*K+8*I(z), %rax+        movq    %rax, -16*K+8*K+8*I(z)+        .set I,  (I+1)+        .endr++        adoxq   %rdx, %rdx+        adcq    $0, %rdx+        movq    %rdx, 32*K(t)++// Square the absolute difference, putting the result M at t[2*K].+// This involves another shuffle so now t' = z_orig and x' = t_orig+// while z' points within the temp buffer to the product M itself++        movq    t, x+        leaq    -16*K(z), t+        leaq    16*K(x), z+        CFI_CALL(Lbignum_ksqr_32_64_local_bignum_sqr_16_32)++// Subtract M, pausing at the 3k position to bump down accumulated carry.+// The carry cannot go negative since it's the top word of a value+// of the form ... + h^2 + l^2 - (h - l)^2 >= 0++        movq    8*K(x), %rax+        subq    (z), %rax+        movq    %rax, 8*K(t)++        .set I,  1++        .rep (K-1)+        movq    8*K+8*I(x), %rax+        sbbq    8*I(z), %rax+        movq    %rax, 8*K+8*I(t)+        .set I,  (I+1)+        .endr++        .rep K+        movq    8*K+8*I(t), %rax+        sbbq    8*I(z), %rax+        movq    %rax, 8*K+8*I(t)+        .set I,  (I+1)+        .endr++        movq    32*K(x), %rdx+        sbbq    $0, %rdx++// Finally propagate the carry to the top quarter++        xorl    %eax, %eax+        addq    %rdx, 24*K(t)+        .set I,  1+        .rep K-1+        adcq    %rax, 24*K+8*I(t)+        .set I,  (I+1)+        .endr++// Restore registers and return++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbx)+        CFI_POP(%rbp)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++// Local copy of the half-length subroutine++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lbignum_ksqr_32_64_local_bignum_sqr_16_32)++Lbignum_ksqr_32_64_local_bignum_sqr_16_32:+        CFI_START+        xorl   %ebp, %ebp+        movq   (x), %rdx+        mulxq  0x8(x), %r9, %rax+        movq   %r9, 0x8(z)+        mulxq  0x10(x), %r10, %rbx+        adcxq  %rax, %r10+        movq   %r10, 0x10(z)+        mulxq  0x18(x), %r11, %rax+        adcxq  %rbx, %r11+        mulxq  0x20(x), %r12, %rbx+        adcxq  %rax, %r12+        mulxq  0x28(x), %r13, %rax+        adcxq  %rbx, %r13+        mulxq  0x30(x), %r14, %rbx+        adcxq  %rax, %r14+        mulxq  0x38(x), %r15, %r8+        adcxq  %rbx, %r15+        adcxq  %rbp, %r8+        xorl   %ebp, %ebp+        movq   0x8(x), %rdx+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        movq   %r11, 0x18(z)+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        movq   %r12, 0x20(z)+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x38(x), %rax, %r9+        adcxq  %rax, %r8+        adoxq  %rbp, %r9+        movq   0x20(x), %rdx+        mulxq  0x28(x), %rax, %r10+        adcxq  %rax, %r9+        adoxq  %rbp, %r10+        adcxq  %rbp, %r10+        xorl   %ebp, %ebp+        movq   0x10(x), %rdx+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        movq   %r13, 0x28(z)+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        movq   %r14, 0x30(z)+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x38(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        movq   0x30(x), %rdx+        mulxq  0x20(x), %rax, %r11+        adcxq  %rax, %r10+        adoxq  %rbp, %r11+        mulxq  0x28(x), %rax, %r12+        adcxq  %rax, %r11+        adoxq  %rbp, %r12+        adcxq  %rbp, %r12+        xorl   %ebp, %ebp+        movq   0x18(x), %rdx+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        movq   %r15, 0x38(z)+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x38(x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        movq   0x38(x), %rdx+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x28(x), %rax, %r13+        adcxq  %rax, %r12+        adoxq  %rbp, %r13+        mulxq  0x30(x), %rax, %r14+        adcxq  %rax, %r13+        adoxq  %rbp, %r14+        adcxq  %rbp, %r14+        movq   0x40(x), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        movq   %r8, 0x40(z)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x30(x), %rax, %r15+        adcxq  %rax, %r14+        adoxq  %rbp, %r15+        mulxq  0x38(x), %rax, %r8+        adcxq  %rax, %r15+        adoxq  %rbp, %r8+        adcxq  %rbp, %r8+        movq   0x48(x), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        movq   %r9, 0x48(z)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x38(x), %rax, %r9+        adcxq  %rax, %r8+        adoxq  %rbp, %r9+        adcxq  %rbp, %r9+        movq   0x50(x), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        movq   %r10, 0x50(z)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x38(x), %rax, %r10+        adcxq  %rax, %r9+        adoxq  %rbp, %r10+        adcxq  %rbp, %r10+        movq   0x58(x), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        movq   %r11, 0x58(z)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x38(x), %rax, %r11+        adcxq  %rax, %r10+        adoxq  %rbp, %r11+        adcxq  %rbp, %r11+        movq   0x60(x), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        movq   %r12, 0x60(z)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x38(x), %rax, %r12+        adcxq  %rax, %r11+        adoxq  %rbp, %r12+        adcxq  %rbp, %r12+        movq   0x68(x), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        movq   %r13, 0x68(z)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x38(x), %rax, %r13+        adcxq  %rax, %r12+        adoxq  %rbp, %r13+        adcxq  %rbp, %r13+        movq   0x70(x), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        movq   %r14, 0x70(z)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x38(x), %rax, %r14+        adcxq  %rax, %r13+        adoxq  %rbp, %r14+        adcxq  %rbp, %r14+        movq   0x78(x), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        movq   %r15, 0x78(z)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x38(x), %rax, %r15+        adcxq  %rax, %r14+        adoxq  %rbp, %r15+        adcxq  %rbp, %r15+        movq   %r8, 0x80(z)+        movq   %r9, 0x88(z)+        movq   %r10, 0x90(z)+        movq   %r11, 0x98(z)+        movq   %r12, 0xa0(z)+        movq   %r13, 0xa8(z)+        movq   %r14, 0xb0(z)+        movq   %r15, 0xb8(z)+        xorl   %ebp, %ebp+        movq   0x88(z), %r9+        movq   0x90(z), %r10+        movq   0x98(z), %r11+        movq   0xa0(z), %r12+        movq   0xa8(z), %r13+        movq   0xb0(z), %r14+        movq   0xb8(z), %r15+        movq   0x40(x), %rdx+        mulxq  0x48(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        movq   %r9, 0x88(z)+        mulxq  0x50(x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        movq   %r10, 0x90(z)+        mulxq  0x58(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x60(x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x68(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x70(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x78(x), %rax, %r8+        adcxq  %rax, %r15+        adoxq  %rbp, %r8+        adcxq  %rbp, %r8+        xorl   %ebp, %ebp+        movq   0x48(x), %rdx+        mulxq  0x50(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        movq   %r11, 0x98(z)+        mulxq  0x58(x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        movq   %r12, 0xa0(z)+        mulxq  0x60(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x68(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x70(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x78(x), %rax, %r9+        adcxq  %rax, %r8+        adoxq  %rbp, %r9+        movq   0x60(x), %rdx+        mulxq  0x68(x), %rax, %r10+        adcxq  %rax, %r9+        adoxq  %rbp, %r10+        adcxq  %rbp, %r10+        xorl   %ebp, %ebp+        movq   0x50(x), %rdx+        mulxq  0x58(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        movq   %r13, 0xa8(z)+        mulxq  0x60(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        movq   %r14, 0xb0(z)+        mulxq  0x68(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x70(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x78(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        movq   0x70(x), %rdx+        mulxq  0x60(x), %rax, %r11+        adcxq  %rax, %r10+        adoxq  %rbp, %r11+        mulxq  0x68(x), %rax, %r12+        adcxq  %rax, %r11+        adoxq  %rbp, %r12+        adcxq  %rbp, %r12+        xorl   %ebp, %ebp+        movq   0x58(x), %rdx+        mulxq  0x60(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        movq   %r15, 0xb8(z)+        mulxq  0x68(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x70(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x78(x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        movq   0x78(x), %rdx+        mulxq  0x60(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x68(x), %rax, %r13+        adcxq  %rax, %r12+        adoxq  %rbp, %r13+        mulxq  0x70(x), %rax, %r14+        adcxq  %rax, %r13+        adoxq  %rbp, %r14+        adcxq  %rbp, %r14+        movq   %r8, 0xc0(z)+        movq   %r9, 0xc8(z)+        movq   %r10, 0xd0(z)+        movq   %r11, 0xd8(z)+        movq   %r12, 0xe0(z)+        movq   %r13, 0xe8(z)+        movq   %r14, 0xf0(z)+        xorl   %ebp, %ebp+        movq   (x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   %rax, (z)+        movq   0x8(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0x8(z)+        movq   0x8(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0x10(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0x10(z)+        movq   0x18(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0x18(z)+        movq   0x10(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0x20(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0x20(z)+        movq   0x28(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0x28(z)+        movq   0x18(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0x30(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0x30(z)+        movq   0x38(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0x38(z)+        movq   0x20(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0x40(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0x40(z)+        movq   0x48(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0x48(z)+        movq   0x28(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0x50(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0x50(z)+        movq   0x58(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0x58(z)+        movq   0x30(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0x60(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0x60(z)+        movq   0x68(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0x68(z)+        movq   0x38(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0x70(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0x70(z)+        movq   0x78(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0x78(z)+        movq   0x40(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0x80(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0x80(z)+        movq   0x88(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0x88(z)+        movq   0x48(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0x90(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0x90(z)+        movq   0x98(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0x98(z)+        movq   0x50(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0xa0(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0xa0(z)+        movq   0xa8(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0xa8(z)+        movq   0x58(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0xb0(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0xb0(z)+        movq   0xb8(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0xb8(z)+        movq   0x60(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0xc0(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0xc0(z)+        movq   0xc8(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0xc8(z)+        movq   0x68(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0xd0(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0xd0(z)+        movq   0xd8(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0xd8(z)+        movq   0x70(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0xe0(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0xe0(z)+        movq   0xe8(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rbx, %rdx+        movq   %rdx, 0xe8(z)+        movq   0x78(x), %rdx+        mulxq  %rdx, %rax, %rbx+        movq   0xf0(z), %rdx+        adcxq  %rdx, %rdx+        adoxq  %rax, %rdx+        movq   %rdx, 0xf0(z)+        adcxq  %rbp, %rbx+        adoxq  %rbp, %rbx+        movq   %rbx, 0xf8(z)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lbignum_ksqr_32_64_local_bignum_sqr_16_32)++S2N_BN_SIZE_DIRECTIVE(bignum_ksqr_32_64)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_modinv.S view
@@ -0,0 +1,714 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Invert modulo m, z = (1/a) mod b, assuming b is an odd number > 1, coprime a+// Inputs a[k], b[k]; output z[k]; temporary buffer t[>=3*k]+//+//    extern void bignum_modinv(uint64_t k, uint64_t *z, const uint64_t *a,+//                              const uint64_t *b, uint64_t *t);+//+// k-digit (digit=64 bits) "z := a^-1 mod b" (modular inverse of a modulo b)+// using t as a temporary buffer (t at least 3*k words = 24*k bytes), and+// assuming that a and b are coprime *and* that b is an odd number > 1.+//+// Standard x86-64 ABI: RDI = k, RSI = z, RDX = a, RCX = b, R8 = t+// Microsoft x64 ABI:   RCX = k, RDX = z, R8 = a, R9 = b, [RSP+40] = t+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_modinv)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_modinv)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_modinv)+        .text+        .balign 32++// We get CHUNKSIZE bits per outer iteration, 64 minus a few for proxy errors++#define CHUNKSIZE 58++// These variables are so fundamental we keep them consistently in registers.+// k actually stays where it was at the beginning, while l gets set up  later++#define k %rdi+#define l %r13++// These are kept on the stack since there aren't enough registers++#define mat_mm      (%rsp)+#define mat_mn      8(%rsp)+#define mat_nm      16(%rsp)+#define mat_nn      24(%rsp)+#define t           32(%rsp)+// Modular inverse+#define v           40(%rsp)+// We reconstruct n as m + 8*k as needed+#define m           48(%rsp)+#define w           56(%rsp)+#define z           64(%rsp)+// Original b pointer, not b the temp+#define bm          72(%rsp)++#define STACKVARSIZE 80++// These get set to m/n or w/z during the cross-multiplications etc.+// Otherwise they can be used as additional temporaries++#define p1 %r8+#define p2 %r15++// These are shorthands for common temporary registers++#define a %rax+#define b %rbx+#define c %rcx+#define d %rdx+#define i %r9++// Temporaries for the top proxy selection part++#define c1        %r10+#define c2        %r11+#define h1        %r12+#define h2        %rbp+#define l1        %r14+#define l2        %rsi++// Re-use for the actual proxies; m_hi = h1 and n_hi = h2 are assumed++#define m_hi    %r12+#define n_hi    %rbp+#define m_lo    %r14+#define n_lo    %rsi++// Re-use for the matrix entries in the inner loop, though they+// get spilled to the corresponding memory locations mat_...++#define m_m     %r10+#define m_n     %r11+#define n_m     %rcx+#define n_n     %rdx++#define ashort %eax+#define ishort %r9d+#define m_mshort %r10d+#define m_nshort %r11d+#define n_mshort %ecx+#define n_nshort %edx++S2N_BN_SYMBOL(bignum_modinv):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+        movq    %r9, %rcx+        movq    56(%rsp), %r8+#endif++// Save all required registers and make room on stack for all the above vars++        CFI_PUSH(%rbp)+        CFI_PUSH(%rbx)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(STACKVARSIZE)++// If k = 0 then do nothing (this is out of scope anyway)++        testq   k, k+        jz      Lbignum_modinv_end++// Set up the additional two buffers m and n beyond w in temp space+// and record all pointers m, n, w and z in stack-based variables++        movq    %rsi, z+        movq    %r8, w+        movq    %rcx, bm+        leaq    (%r8,k,8), %r10+        movq    %r10, m+        leaq    (%r10,k,8), p2++// Initialize the main buffers with their starting values:+// m = a, n = b, w = b (to be tweaked to b - 1) and z = 0++        xorq    %r11, %r11+        xorq    i, i+Lbignum_modinv_copyloop:+        movq    (%rdx,i,8), a+        movq    (%rcx,i,8), b+        movq    a, (%r10,i,8)+        movq    b, (p2,i,8)+        movq    b, (%r8,i,8)+        movq    %r11, (%rsi,i,8)+        incq    i+        cmpq    k, i+        jc      Lbignum_modinv_copyloop++// Tweak down w to b - 1 (this crude approach is safe as b needs to be odd+// for it to be in scope). We have then established the congruence invariant:+//+//   a * w == -m (mod b)+//   a * z == n (mod b)+//+// This, with the bounds w <= b and z <= b, is maintained round the outer loop++        movq    (%r8), a+        movq    a, b+        decq    b+        movq    b, (%r8)++// Compute v = negated modular inverse of b mod 2^64, reusing a from above+// This is used for Montgomery reduction operations each time round the loop++        movq    a, h2+        movq    a, h1+        shlq    $2, h2+        subq    h2, h1+        xorq    $2, h1++        movq    h1, h2+        imulq   a, h2+        movl    $2, ashort+        addq    h2, a+        addq    $1, h2++        imulq   a, h1++        imulq   h2, h2+        movl    $1, ashort+        addq    h2, a+        imulq   a, h1++        imulq   h2, h2+        movl    $1, ashort+        addq    h2, a+        imulq   a, h1++        imulq   h2, h2+        movl    $1, ashort+        addq    h2, a+        imulq   a, h1++        movq    h1, v++// Set up the outer loop count of 128 * k+// The invariant is that m * n < 2^t at all times.++        movq    k, a+        shlq    $7, a+        movq    a, t++// Start of the main outer loop iterated t / CHUNKSIZE times++Lbignum_modinv_outerloop:++// We need only bother with sharper l = min k (ceil(t/64)) digits+// for the computations on m and n (but we still need k for w and z).+// Either both m and n fit in l digits, or m has become zero and so+// nothing happens in the loop anyway and this makes no difference.++        movq    t, l+        addq    $63, l+        shrq    $6, l+        cmpq    k, l+        cmovncq k, l++// Select upper and lower proxies for both m and n to drive the inner+// loop. The lower proxies are simply the lowest digits themselves,+// m_lo = m[0] and n_lo = n[0], while the upper proxies are bitfields+// of the two inputs selected so their top bit (63) aligns with the+// most significant bit of *either* of the two inputs.++        xorq    h1, h1 // Previous high and low for m+        xorq    l1, l1+        xorq    h2, h2 // Previous high and low for n+        xorq    l2, l2+        xorq    c2, c2 // Mask flag: previous word of one was nonzero+        // and in this case h1 and h2 are those words++        movq    m, p1+        leaq    (p1,k,8), p2+        xorq    i, i+Lbignum_modinv_toploop:+        movq    (p1,i,8), b+        movq    (p2,i,8), c+        movq    c2, c1+        andq    h1, c1+        andq    h2, c2+        movq    b, a+        orq     c, a+        negq    a+        cmovcq  c1, l1+        cmovcq  c2, l2+        cmovcq  b, h1+        cmovcq  c, h2+        sbbq    c2, c2+        incq    i+        cmpq    l, i+        jc      Lbignum_modinv_toploop++        movq    h1, a+        orq     h2, a+        bsrq    a, c+        xorq    $63, c+        shldq   %cl, l1, h1+        shldq   %cl, l2, h2++// m_lo = m[0], n_lo = n[0];++        movq    (p1), %rax+        movq    %rax, m_lo++        movq    (p2), %rax+        movq    %rax, n_lo++// Now the inner loop, with i as loop counter from CHUNKSIZE down.+// This records a matrix of updates to apply to the initial+// values of m and n with, at stage j:+//+//     sgn * m' = (m_m * m - m_n * n) / 2^j+//    -sgn * n' = (n_m * m - n_n * n) / 2^j+//+// where "sgn" is either +1 or -1, and we lose track of which except+// that both instance above are the same. This throwing away the sign+// costs nothing (since we have to correct in general anyway because+// of the proxied comparison) and makes things a bit simpler. But it+// is simply the parity of the number of times the first condition,+// used as the swapping criterion, fires in this loop.++        movl    $1, m_mshort+        movl    $0, m_nshort+        movl    $0, n_mshort+        movl    $1, n_nshort+        movl    $CHUNKSIZE, ishort++// Stash more variables over the inner loop to free up regs++        movq    k, mat_mn+        movq    l, mat_nm+        movq    p1, mat_mm+        movq    p2, mat_nn++// Conceptually in the inner loop we follow these steps:+//+// * If m_lo is odd and m_hi < n_hi, then swap the four pairs+//    (m_hi,n_hi); (m_lo,n_lo); (m_m,n_m); (m_n,n_n)+//+// * Now, if m_lo is odd (old or new, doesn't matter as initial n_lo is odd)+//    m_hi := m_hi - n_hi, m_lo := m_lo - n_lo+//    m_m  := m_m + n_m, m_n := m_n + n_n+//+// * Halve and double them+//     m_hi := m_hi / 2, m_lo := m_lo / 2+//     n_m := n_m * 2, n_n := n_n * 2+//+// The actual computation computes updates before actually swapping and+// then corrects as needed.++Lbignum_modinv_innerloop:++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorq    p1, p1+        xorq    p2, p2+        btq     $0, m_lo++        cmovcq  n_hi, %rax+        cmovcq  n_lo, %rbx+        cmovcq  n_m, p1+        cmovcq  n_n, p2++        movq    m_lo, l+        subq    %rbx, m_lo+        subq    l, %rbx+        movq    m_hi, k+        subq    %rax, k+        cmovcq  m_hi, n_hi+        leaq    -1(k), m_hi+        cmovcq  %rbx, m_lo+        cmovcq  l, n_lo+        notq    m_hi+        cmovcq  m_m, n_m+        cmovcq  m_n, n_n+        cmovncq k, m_hi++        shrq    $1, m_lo+        addq    p1, m_m+        addq    p2, m_n+        shrq    $1, m_hi+        addq    n_m, n_m+        addq    n_n, n_n++// End of the inner for-loop++        decq    i+        jnz     Lbignum_modinv_innerloop++// Unstash the temporary variables++        movq    mat_mn, k+        movq    mat_nm, l+        movq    mat_mm, p1+        movq    mat_nn, p2++// Put the matrix entries in memory since we're out of registers+// We pull them out repeatedly in the next loop++        movq    m_m, mat_mm+        movq    m_n, mat_mn+        movq    n_m, mat_nm+        movq    n_n, mat_nn++// Apply the update to w and z, using addition in this case, and also take+// the chance to shift an additional 6 = 64-CHUNKSIZE bits to be ready for a+// Montgomery multiplication. Because we know that m_m + m_n <= 2^58 and+// w, z <= b < 2^{64k}, we know that both of these fit in k+1 words.+// We do this before the m-n update to allow us to play with c1 and c2 here.+//+//    l1::w = 2^6 * (m_m * w + m_n * z)+//    l2::z = 2^6 * (n_m * w + n_n * z)+//+// with c1 and c2 recording previous words for the shifting part++        movq    w, p1+        movq    z, p2+        xorq    l1, l1+        xorq    l2, l2+        xorq    c1, c1+        xorq    c2, c2+        xorq    i, i+Lbignum_modinv_congloop:++        movq    (p1,i,8), c+        movq    mat_mm, a+        mulq    c+        addq    a, l1+        adcq    $0, d+        movq    d, h1 // Now h1::l1 := m_m * w + l1_in++        movq    mat_nm, a+        mulq    c+        addq    a, l2+        adcq    $0, d+        movq    d, h2 // Now h2::l2 := n_m * w + l2_in++        movq    (p2,i,8), c+        movq    mat_mn, a+        mulq    c+        addq    a, l1+        adcq    d, h1 // h1::l1 := m_m * w + m_n * z + l1_in+        shrdq   $CHUNKSIZE, l1, c1+        movq    c1, (p1,i,8)+        movq    l1, c1+        movq    h1, l1++        movq    mat_nn, a+        mulq    c+        addq    a, l2+        adcq    d, h2 // h2::l2 := n_m * w + n_n * z + l2_in+        shrdq   $CHUNKSIZE, l2, c2+        movq    c2, (p2,i,8)+        movq    l2, c2+        movq    h2, l2++        incq    i+        cmpq    k, i+        jc      Lbignum_modinv_congloop++        shldq   $64-CHUNKSIZE, c1, l1+        shldq   $64-CHUNKSIZE, c2, l2++// Do a Montgomery reduction of l1::w++        movq    bm, p2++        movq    (p1), b+        movq    v, h1+        imulq   b, h1+        movq    (p2), a+        mulq    h1+        addq    b, a // Will be zero but want the carry+        movq    %rdx, c1+        movl    $1, ishort+        movq    k, c+        decq    c+        jz      Lbignum_modinv_wmontend++Lbignum_modinv_wmontloop:+        adcq    (p1,i,8), c1+        sbbq    b, b+        movq    (p2,i,8), a+        mulq    h1+        subq    b, %rdx+        addq    c1, a+        movq    a, -8(p1,i,8)+        movq    %rdx, c1+        incq    i+        decq    c+        jnz     Lbignum_modinv_wmontloop++Lbignum_modinv_wmontend:+        adcq    l1, c1+        movq    c1, -8(p1,k,8)+        sbbq    c1, c1+        negq    c1++        movq    k, c+        xorq    i, i+Lbignum_modinv_wcmploop:+        movq    (p1,i,8), a+        sbbq    (p2,i,8), a+        incq    i+        decq    c+        jnz     Lbignum_modinv_wcmploop+        sbbq    $0, c1+        sbbq    c1, c1+        notq    c1++        xorq    c, c+        xorq    i, i+Lbignum_modinv_wcorrloop:+        movq    (p1,i,8), a+        movq    (p2,i,8), b+        andq    c1, b+        negq    c+        sbbq    b, a+        sbbq    c, c+        movq    a, (p1,i,8)+        incq    i+        cmpq    k, i+        jc      Lbignum_modinv_wcorrloop++// Do a Montgomery reduction of l2::z++        movq    z, p1++        movq    (p1), b+        movq    v, h2+        imulq   b, h2+        movq    (p2), a+        mulq    h2+        addq    b, a // Will be zero but want the carry+        movq    %rdx, c2+        movl    $1, ishort+        movq    k, c+        decq    c+        jz      Lbignum_modinv_zmontend++Lbignum_modinv_zmontloop:+        adcq    (p1,i,8), c2+        sbbq    b, b+        movq    (p2,i,8), a+        mulq    h2+        subq    b, %rdx+        addq    c2, a+        movq    a, -8(p1,i,8)+        movq    %rdx, c2+        incq    i+        decq    c+        jnz     Lbignum_modinv_zmontloop++Lbignum_modinv_zmontend:+        adcq    l2, c2+        movq    c2, -8(p1,k,8)+        sbbq    c2, c2+        negq    c2++        movq    k, c+        xorq    i, i+Lbignum_modinv_zcmploop:+        movq    (p1,i,8), a+        sbbq    (p2,i,8), a+        incq    i+        decq    c+        jnz     Lbignum_modinv_zcmploop+        sbbq    $0, c2+        sbbq    c2, c2+        notq    c2++        xorq    c, c+        xorq    i, i+Lbignum_modinv_zcorrloop:+        movq    (p1,i,8), a+        movq    (p2,i,8), b+        andq    c2, b+        negq    c+        sbbq    b, a+        sbbq    c, c+        movq    a, (p1,i,8)+        incq    i+        cmpq    k, i+        jc      Lbignum_modinv_zcorrloop++// Now actually compute the updates to m and n corresponding to the matrix,+// and correct the signs if they have gone negative. First we compute the+// (k+1)-sized updates with the following invariant (here h1 and h2 are in+// fact carry bitmasks, either 0 or -1):+//+//    h1::l1::m = m_m * m - m_n * n+//    h2::l2::n = n_m * m - n_n * n++        movq    m, p1+        leaq    (p1,k,8), p2+        xorq    i, i+        xorq    h1, h1+        xorq    l1, l1+        xorq    h2, h2+        xorq    l2, l2+Lbignum_modinv_crossloop:++        movq    (p1,i,8), c+        movq    mat_mm, a+        mulq    c+        addq    a, l1+        adcq    $0, d+        movq    d, c1 // Now c1::l1 is +ve part 1++        movq    mat_nm, a+        mulq    c+        addq    a, l2+        adcq    $0, d+        movq    d, c2 // Now c2::l2 is +ve part 2++        movq    (p2,i,8), c+        movq    mat_mn, a+        mulq    c+        subq    h1, d // Now d::a is -ve part 1++        subq    a, l1+        sbbq    d, c1+        sbbq    h1, h1+        movq    l1, (p1,i,8)+        movq    c1, l1++        movq    mat_nn, a+        mulq    c+        subq    h2, d // Now d::a is -ve part 2++        subq    a, l2+        sbbq    d, c2+        sbbq    h2, h2+        movq    l2, (p2,i,8)+        movq    c2, l2++        incq    i+        cmpq    l, i+        jc      Lbignum_modinv_crossloop++// Now fix the signs of m and n if they have gone negative++        xorq    i, i+        movq    h1, c1 // carry-in coded up as well+        movq    h2, c2 // carry-in coded up as well+        xorq    h1, l1 // for the Lbignum_modinv_end digit+        xorq    h2, l2 // for the Lbignum_modinv_end digit+Lbignum_modinv_optnegloop:+        movq    (p1,i,8), a+        xorq    h1, a+        negq    c1+        adcq    $0, a+        sbbq    c1, c1+        movq    a, (p1,i,8)+        movq    (p2,i,8), a+        xorq    h2, a+        negq    c2+        adcq    $0, a+        sbbq    c2, c2+        movq    a, (p2,i,8)+        incq    i+        cmpq    l, i+        jc      Lbignum_modinv_optnegloop+        subq    c1, l1+        subq    c2, l2++// Now shift them right CHUNKSIZE bits++        movq    l, i+Lbignum_modinv_shiftloop:+        movq    -8(p1,i,8), a+        movq    a, c1+        shrdq   $CHUNKSIZE, l1, a+        movq    a, -8(p1,i,8)+        movq    c1, l1+        movq    -8(p2,i,8), a+        movq    a, c2+        shrdq   $CHUNKSIZE, l2, a+        movq    a, -8(p2,i,8)+        movq    c2, l2+        decq    i+        jnz     Lbignum_modinv_shiftloop++// Finally, use the signs h1 and h2 to do optional modular negations of+// w and z respectively, flipping h2 to make signs work. We don't make+// any checks for zero values, but we certainly retain w <= b and z <= b.+// This is enough for the Montgomery step in the next iteration to give+// strict reduction w < b amd z < b, and anyway when we terminate we+// could not have z = b since it violates the coprimality assumption for+// in-scope cases.++        notq    h2+        movq    bm, c+        movq    w, p1+        movq    z, p2+        movq    h1, c1+        movq    h2, c2+        xorq    i, i+Lbignum_modinv_fliploop:+        movq    h2, d+        movq    (c,i,8), a+        andq    a, d+        andq    h1, a+        movq    (p1,i,8), b+        xorq    h1, b+        negq    c1+        adcq    b, a+        sbbq    c1, c1+        movq    a, (p1,i,8)+        movq    (p2,i,8), b+        xorq    h2, b+        negq    c2+        adcq    b, d+        sbbq    c2, c2+        movq    d, (p2,i,8)+        incq    i+        cmpq    k, i+        jc      Lbignum_modinv_fliploop++// End of main loop. We can stop if t' <= 0 since then m * n < 2^0, which+// since n is odd and m and n are coprime (in the in-scope cases) means+// m = 0, n = 1 and hence from the congruence invariant a * z == 1 (mod b).+// Moreover we do in fact need to maintain strictly t > 0 in the main loop,+// or the computation of the optimized digit bound l could collapse to 0.++        subq    $CHUNKSIZE, t+        jnbe    Lbignum_modinv_outerloop++Lbignum_modinv_end:+        CFI_INC_RSP(STACKVARSIZE)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbx)+        CFI_POP(%rbp)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_modinv)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_montinv_p384.S view
@@ -0,0 +1,1834 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Montgomery inverse modulo p_384 = 2^384 - 2^128 - 2^96 + 2^32 - 1+// Input x[6]; output z[6]+//+// extern void bignum_montinv_p384(uint64_t z[static 6],+//                                 const uint64_t x[static 6]);+//+// If the 6-digit input x is coprime to p_384, i.e. is not divisible+// by it, returns z < p_384 such that x * z == 2^768 (mod p_384). This+// is effectively "Montgomery inverse" because if we consider x and z as+// Montgomery forms of X and Z, i.e. x == 2^384 * X and z == 2^384 * Z+// (both mod p_384) then X * Z == 1 (mod p_384). That is, this function+// gives the analog of the modular inverse bignum_inv_p384 but with both+// input and output in the Montgomery domain. Note that x does not need+// to be reduced modulo p_384, but the output always is. If the input+// is divisible (i.e. is 0 or p_384), then there can be no solution to+// the congruence x * z == 2^768 (mod p_384), and z = 0 is returned.+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------+#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_montinv_p384)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_montinv_p384)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_montinv_p384)+        .text+        .balign 32++// Size in bytes of a 64-bit word++#define N 8++// Pointer-offset pairs for temporaries on stack+// The u and v variables are 6 words each as expected, but the f and g+// variables are 8 words each -- they need to have at least one extra+// word for a sign word, and to preserve alignment we "round up" to 8.+// In fact, we currently keep an extra word in u and v as well.++#define f 0(%rsp)+#define g (8*N)(%rsp)+#define u (16*N)(%rsp)+#define v (24*N)(%rsp)+#define tmp  (32*N)(%rsp)+#define tmp2  (33*N)(%rsp)+#define i  (34*N)(%rsp)+#define d  (35*N)(%rsp)++#define mat (36*N)(%rsp)++// Backup for the input pointer++#define res  (40*N)(%rsp)++// Total size to reserve on the stack++#define NSPACE 42*N++// Syntactic variants to make x86_att version simpler to generate++#define F 0+#define G (8*N)+#define U (16*N)+#define V (24*N)+#define MAT (36*N)++#define ff  (%rsp)+#define gg  (8*N)(%rsp)++// ---------------------------------------------------------------------------+// Core signed almost-Montgomery reduction macro from P[6..0] to P[5..0].+// ---------------------------------------------------------------------------++#define amontred(P)                                                     \+/* We only know the input is -2^444 < x < 2^444. To do traditional  */  \+/* unsigned Montgomery reduction, start by adding 2^61 * p_384.     */  \+        movq    $0xe000000000000000, %r8 ;                             \+        xorl    %eax, %eax ;                                           \+        addq    P, %r8 ;                                            \+        movq    $0x000000001fffffff, %r9 ;                             \+        leaq    -1(%rax), %rax ;                                       \+        adcq    N+P, %r9 ;                                          \+        movq    $0xdfffffffe0000000, %r10 ;                            \+        adcq    2*N+P, %r10 ;                                       \+        movq    3*N+P, %r11 ;                                       \+        adcq    %rax, %r11 ;                                           \+        movq    4*N+P, %r12 ;                                       \+        adcq    %rax, %r12 ;                                           \+        movq    5*N+P, %r13 ;                                       \+        adcq    %rax, %r13 ;                                           \+        movq    $0x1fffffffffffffff, %r14 ;                            \+        adcq    6*N+P, %r14 ;                                       \+/* Correction multiplier is %rbx = w = [d0 + (d0<<32)] mod 2^64 */   \+        movq    %r8, %rbx ;                                            \+        shlq    $32, %rbx ;                                            \+        addq    %r8, %rbx ;                                            \+/* Construct [%rbp;%rdx;%rax;-] = (2^384 - p_384) * w */               \+/* We know lowest word will cancel so can re-use %r8 as a temp */    \+        xorl    %ebp, %ebp ;                                           \+        movq    $0xffffffff00000001, %rax ;                            \+        mulq    %rbx;                                                \+        movq    %rdx, %r8 ;                                            \+        movq    $0x00000000ffffffff, %rax ;                            \+        mulq    %rbx;                                                \+        addq    %r8, %rax ;                                            \+        adcq    %rbx, %rdx ;                                           \+        adcl    %ebp, %ebp ;                                           \+/*  Now subtract that and add 2^384 * w, catching carry in %rax  */  \+        subq    %rax, %r9 ;                                            \+        sbbq    %rdx, %r10 ;                                           \+        sbbq    %rbp, %r11 ;                                           \+        sbbq    $0, %r12 ;                                             \+        sbbq    $0, %r13 ;                                             \+        sbbq    $0, %r14 ;                                             \+        sbbq    %rax, %rax ;                                           \+        addq    %rbx, %r14 ;                                           \+        adcq    $0, %rax ;                                             \+/* Now if top is nonzero we subtract p_384 (almost-Montgomery) */   \+        negq    %rax;                                                \+        movq    $0x00000000ffffffff, %rbx ;                            \+        andq    %rax, %rbx ;                                           \+        movq    $0xffffffff00000000, %rcx ;                            \+        andq    %rax, %rcx ;                                           \+        movq    $0xfffffffffffffffe, %rdx ;                            \+        andq    %rax, %rdx ;                                           \+        subq    %rbx, %r9 ;                                            \+        movq    %r9, P ;                                            \+        sbbq    %rcx, %r10 ;                                           \+        movq    %r10, N+P ;                                         \+        sbbq    %rdx, %r11 ;                                           \+        movq    %r11, 2*N+P ;                                       \+        sbbq    %rax, %r12 ;                                           \+        movq    %r12, 3*N+P ;                                       \+        sbbq    %rax, %r13 ;                                           \+        movq    %r13, 4*N+P ;                                       \+        sbbq    %rax, %r14 ;                                           \+        movq    %r14, 5*N+P++// Very similar to a subroutine call to the s2n-bignum word_divstep59.+// But different in register usage and returning the final matrix as+//+// [ %r8   %r10]+// [ %r12  %r14]+//+// and also returning the matrix still negated (which doesn't matter)++#define divstep59(din,fin,gin)                                          \+        movq    din, %rsi ;                                               \+        movq    fin, %rdx ;                                               \+        movq    gin, %rcx ;                                               \+        movq    %rdx, %rbx ;                                               \+        andq    $0xfffff, %rbx ;                                           \+        movabsq $0xfffffe0000000000, %rax ;                                \+        orq     %rax, %rbx ;                                               \+        andq    $0xfffff, %rcx ;                                           \+        movabsq $0xc000000000000000, %rax ;                                \+        orq     %rax, %rcx ;                                               \+        movq    $0xfffffffffffffffe, %rax ;                                \+        xorl    %ebp, %ebp ;                                               \+        movl    $0x2, %edx ;                                               \+        movq    %rbx, %rdi ;                                               \+        movq    %rax, %r8 ;                                                \+        testq   %rsi, %rsi ;                                               \+        cmovs   %rbp, %r8 ;                                                \+        testq   $0x1, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        sarq    $1, %rcx ;                                                 \+        movl    $0x100000, %eax ;                                          \+        leaq    (%rbx,%rax), %rdx ;                                         \+        leaq    (%rcx,%rax), %rdi ;                                         \+        shlq    $0x16, %rdx ;                                              \+        shlq    $0x16, %rdi ;                                              \+        sarq    $0x2b, %rdx ;                                              \+        sarq    $0x2b, %rdi ;                                              \+        movabsq $0x20000100000, %rax ;                                     \+        leaq    (%rbx,%rax), %rbx ;                                         \+        leaq    (%rcx,%rax), %rcx ;                                         \+        sarq    $0x2a, %rbx ;                                              \+        sarq    $0x2a, %rcx ;                                              \+        movq    %rdx, MAT(%rsp) ;                                         \+        movq    %rbx, MAT+0x8(%rsp) ;                                     \+        movq    %rdi, MAT+0x10(%rsp) ;                                    \+        movq    %rcx, MAT+0x18(%rsp) ;                                    \+        movq    fin, %r12 ;                                               \+        imulq   %r12, %rdi ;                                               \+        imulq   %rdx, %r12 ;                                               \+        movq    gin, %r13 ;                                               \+        imulq   %r13, %rbx ;                                               \+        imulq   %rcx, %r13 ;                                               \+        addq    %rbx, %r12 ;                                               \+        addq    %rdi, %r13 ;                                               \+        sarq    $0x14, %r12 ;                                              \+        sarq    $0x14, %r13 ;                                              \+        movq    %r12, %rbx ;                                               \+        andq    $0xfffff, %rbx ;                                           \+        movabsq $0xfffffe0000000000, %rax ;                                \+        orq     %rax, %rbx ;                                               \+        movq    %r13, %rcx ;                                               \+        andq    $0xfffff, %rcx ;                                           \+        movabsq $0xc000000000000000, %rax ;                                \+        orq     %rax, %rcx ;                                               \+        movq    $0xfffffffffffffffe, %rax ;                                \+        movl    $0x2, %edx ;                                               \+        movq    %rbx, %rdi ;                                               \+        movq    %rax, %r8 ;                                                \+        testq   %rsi, %rsi ;                                               \+        cmovs   %rbp, %r8 ;                                                \+        testq   $0x1, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        sarq    $1, %rcx ;                                                 \+        movl    $0x100000, %eax ;                                          \+        leaq    (%rbx,%rax), %r8 ;                                          \+        leaq    (%rcx,%rax), %r10 ;                                         \+        shlq    $0x16, %r8 ;                                               \+        shlq    $0x16, %r10 ;                                              \+        sarq    $0x2b, %r8 ;                                               \+        sarq    $0x2b, %r10 ;                                              \+        movabsq $0x20000100000, %rax ;                                     \+        leaq    (%rbx,%rax), %r15 ;                                         \+        leaq    (%rcx,%rax), %r11 ;                                         \+        sarq    $0x2a, %r15 ;                                              \+        sarq    $0x2a, %r11 ;                                              \+        movq    %r13, %rbx ;                                               \+        movq    %r12, %rcx ;                                               \+        imulq   %r8, %r12 ;                                                \+        imulq   %r15, %rbx ;                                               \+        addq    %rbx, %r12 ;                                               \+        imulq   %r11, %r13 ;                                               \+        imulq   %r10, %rcx ;                                               \+        addq    %rcx, %r13 ;                                               \+        sarq    $0x14, %r12 ;                                              \+        sarq    $0x14, %r13 ;                                              \+        movq    %r12, %rbx ;                                               \+        andq    $0xfffff, %rbx ;                                           \+        movabsq $0xfffffe0000000000, %rax ;                                \+        orq     %rax, %rbx ;                                               \+        movq    %r13, %rcx ;                                               \+        andq    $0xfffff, %rcx ;                                           \+        movabsq $0xc000000000000000, %rax ;                                \+        orq     %rax, %rcx ;                                               \+        movq    MAT(%rsp), %rax ;                                         \+        imulq   %r8, %rax ;                                                \+        movq    MAT+0x10(%rsp), %rdx ;                                    \+        imulq   %r15, %rdx ;                                               \+        imulq   MAT+0x8(%rsp), %r8 ;                                      \+        imulq   MAT+0x18(%rsp), %r15 ;                                    \+        addq    %r8, %r15 ;                                                \+        leaq    (%rax,%rdx), %r9 ;                                          \+        movq    MAT(%rsp), %rax ;                                         \+        imulq   %r10, %rax ;                                               \+        movq    MAT+0x10(%rsp), %rdx ;                                    \+        imulq   %r11, %rdx ;                                               \+        imulq   MAT+0x8(%rsp), %r10 ;                                     \+        imulq   MAT+0x18(%rsp), %r11 ;                                    \+        addq    %r10, %r11 ;                                               \+        leaq    (%rax,%rdx), %r13 ;                                         \+        movq    $0xfffffffffffffffe, %rax ;                                \+        movl    $0x2, %edx ;                                               \+        movq    %rbx, %rdi ;                                               \+        movq    %rax, %r8 ;                                                \+        testq   %rsi, %rsi ;                                               \+        cmovs   %rbp, %r8 ;                                                \+        testq   $0x1, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        cmovs   %rbp, %r8 ;                                                \+        movq    %rbx, %rdi ;                                               \+        testq   %rdx, %rcx ;                                               \+        cmoveq  %rbp, %r8 ;                                                \+        cmoveq  %rbp, %rdi ;                                               \+        sarq    $1, %rcx ;                                                 \+        xorq    %r8, %rdi ;                                                \+        xorq    %r8, %rsi ;                                                \+        btq     $0x3f, %r8 ;                                               \+        cmovbq  %rcx, %rbx ;                                               \+        movq    %rax, %r8 ;                                                \+        subq    %rax, %rsi ;                                               \+        leaq    (%rcx,%rdi), %rcx ;                                         \+        sarq    $1, %rcx ;                                                 \+        movl    $0x100000, %eax ;                                          \+        leaq    (%rbx,%rax), %r8 ;                                          \+        leaq    (%rcx,%rax), %r12 ;                                         \+        shlq    $0x15, %r8 ;                                               \+        shlq    $0x15, %r12 ;                                              \+        sarq    $0x2b, %r8 ;                                               \+        sarq    $0x2b, %r12 ;                                              \+        movabsq $0x20000100000, %rax ;                                     \+        leaq    (%rbx,%rax), %r10 ;                                         \+        leaq    (%rcx,%rax), %r14 ;                                         \+        sarq    $0x2b, %r10 ;                                              \+        sarq    $0x2b, %r14 ;                                              \+        movq    %r9, %rax ;                                                \+        imulq   %r8, %rax ;                                                \+        movq    %r13, %rdx ;                                               \+        imulq   %r10, %rdx ;                                               \+        imulq   %r15, %r8 ;                                                \+        imulq   %r11, %r10 ;                                               \+        addq    %r8, %r10 ;                                                \+        leaq    (%rax,%rdx), %r8 ;                                          \+        movq    %r9, %rax ;                                                \+        imulq   %r12, %rax ;                                               \+        movq    %r13, %rdx ;                                               \+        imulq   %r14, %rdx ;                                               \+        imulq   %r15, %r12 ;                                               \+        imulq   %r11, %r14 ;                                               \+        addq    %r12, %r14 ;                                               \+        leaq    (%rax,%rdx), %r12++S2N_BN_SYMBOL(bignum_montinv_p384):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Save registers and make room for temporaries++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++        CFI_DEC_RSP(NSPACE)++// Save the return pointer for the end so we can overwrite %rdi later++        movq    %rdi, res++// Copy the constant p_384 into f including the 7th zero digit++        movl    $0xffffffff, %eax+        movq    %rax, F(%rsp)+        movq    %rax, %rbx+        notq    %rbx+        movq    %rbx, F+N(%rsp)+        xorl    %ebp, %ebp+        leaq    -2(%rbp), %rcx+        movq    %rcx, F+2*N(%rsp)+        leaq    -1(%rbp), %rdx+        movq    %rdx, F+3*N(%rsp)+        movq    %rdx, F+4*N(%rsp)+        movq    %rdx, F+5*N(%rsp)+        movq    %rbp, F+6*N(%rsp)++// Copy input but to g, reduced mod p_384 so that g <= f as assumed+// in the divstep bound proof.++        movq    (%rsi), %r8+        subq    %rax, %r8+        movq    N(%rsi), %r9+        sbbq    %rbx, %r9+        movq    2*N(%rsi), %r10+        sbbq    %rcx, %r10+        movq    3*N(%rsi), %r11+        sbbq    %rdx, %r11+        movq    4*N(%rsi), %r12+        sbbq    %rdx, %r12+        movq    5*N(%rsi), %r13+        sbbq    %rdx, %r13++        cmovcq  (%rsi), %r8+        cmovcq  N(%rsi), %r9+        cmovcq  2*N(%rsi), %r10+        cmovcq  3*N(%rsi), %r11+        cmovcq  4*N(%rsi), %r12+        cmovcq  5*N(%rsi), %r13++        movq    %r8, G(%rsp)+        movq    %r9, G+N(%rsp)+        movq    %r10, G+2*N(%rsp)+        movq    %r11, G+3*N(%rsp)+        movq    %r12, G+4*N(%rsp)+        movq    %r13, G+5*N(%rsp)+        movq    %rbp, G+6*N(%rsp)++// Also maintain reduced < 2^384 vector [u,v] such that+// [f,g] == x * 2^{5*i-843} * [u,v] (mod p_384)+// starting with [p_384,x] == x * 2^{5*0-843} * [0,2^843] (mod p_384)+// The weird-looking 5*i modifications come in because we are doing+// 64-bit word-sized Montgomery reductions at each stage, which is+// 5 bits more than the 59-bit requirement to keep things stable.+// After the 15th and last iteration and sign adjustment, when+// f == 1 for in-scope cases, we have x * 2^{75-843} * u == 1, i.e.+// x * u == 2^768 as required.++        xorl    %eax, %eax+        movq    %rax, U(%rsp)+        movq    %rax, U+N(%rsp)+        movq    %rax, U+2*N(%rsp)+        movq    %rax, U+3*N(%rsp)+        movq    %rax, U+4*N(%rsp)+        movq    %rax, U+5*N(%rsp)++// The starting constant 2^843 mod p_384 is+// 0x0000000000000800:00001000000007ff:fffff00000000000+//  :00001000000007ff:fffff00000000800:0000000000000000+// where colons separate 64-bit subwords, least significant at the right.+// These are constructed dynamically to reduce large constant loads.++        movq    %rax, V(%rsp)+        movq    $0xfffff00000000800, %rcx+        movq    %rcx, V+N(%rsp)+        movq    $0x00001000000007ff, %rdx+        movq    %rdx, V+2*N(%rsp)+        btr     $11, %rcx+        movq    %rcx, V+3*N(%rsp)+        movq    %rdx, V+4*N(%rsp)+        bts     $11, %rax+        movq    %rax, V+5*N(%rsp)++// Start of main loop. We jump into the middle so that the divstep+// portion is common to the special fifteenth iteration after a uniform+// first 14.++        movq    $15, i+        movq    $1, d+        jmp     Lbignum_montinv_p384_midloop++Lbignum_montinv_p384_loop:++// Separate out the matrix into sign-magnitude pairs++        movq    %r8, %r9+        sarq    $63, %r9+        xorq    %r9, %r8+        subq    %r9, %r8++        movq    %r10, %r11+        sarq    $63, %r11+        xorq    %r11, %r10+        subq    %r11, %r10++        movq    %r12, %r13+        sarq    $63, %r13+        xorq    %r13, %r12+        subq    %r13, %r12++        movq    %r14, %r15+        sarq    $63, %r15+        xorq    %r15, %r14+        subq    %r15, %r14++// Adjust the initial values to allow for complement instead of negation+// This initial offset is the same for [f,g] and [u,v] compositions.+// Save it in temporary storage for the [u,v] part and do [f,g] first.++        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %rdi+        andq    %r11, %rdi+        addq    %rax, %rdi+        movq    %rdi, tmp++        movq    %r12, %rax+        andq    %r13, %rax+        movq    %r14, %rsi+        andq    %r15, %rsi+        addq    %rax, %rsi+        movq    %rsi, tmp2++// Now the computation of the updated f and g values. This maintains a+// 2-word carry between stages so we can conveniently insert the shift+// right by 59 before storing back, and not overwrite digits we need+// again of the old f and g values.+//+// Digit 0 of [f,g]++        xorl    %ebx, %ebx+        movq    F(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    G(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rdi+        adcq    %rdx, %rbx++        xorl    %ebp, %ebp+        movq    F(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    G(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp++// Digit 1 of [f,g]++        xorl    %ecx, %ecx+        movq    F+N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    G+N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $59, %rbx, %rdi+        movq    %rdi, F(%rsp)++        xorl    %edi, %edi+        movq    F+N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    G+N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $59, %rbp, %rsi+        movq    %rsi, G(%rsp)++// Digit 2 of [f,g]++        xorl    %esi, %esi+        movq    F+2*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    G+2*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $59, %rcx, %rbx+        movq    %rbx, F+N(%rsp)++        xorl    %ebx, %ebx+        movq    F+2*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    G+2*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $59, %rdi, %rbp+        movq    %rbp, G+N(%rsp)++// Digit 3 of [f,g]++        xorl    %ebp, %ebp+        movq    F+3*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    G+3*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        shrdq   $59, %rsi, %rcx+        movq    %rcx, F+2*N(%rsp)++        xorl    %ecx, %ecx+        movq    F+3*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    G+3*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $59, %rbx, %rdi+        movq    %rdi, G+2*N(%rsp)++// Digit 4 of [f,g]++        xorl    %edi, %edi+        movq    F+4*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    G+4*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $59, %rbp, %rsi+        movq    %rsi, F+3*N(%rsp)++        xorl    %esi, %esi+        movq    F+4*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    G+4*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $59, %rcx, %rbx+        movq    %rbx, G+3*N(%rsp)++// Digits 5 and 6 of [f,g]++        movq    F+5*N(%rsp), %rax+        xorq    %r9, %rax+        movq    F+6*N(%rsp), %rbx+        xorq    %r9, %rbx+        andq    %r8, %rbx+        negq    %rbx+        mulq    %r8+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    G+5*N(%rsp), %rax+        xorq    %r11, %rax+        movq    G+6*N(%rsp), %rdx+        xorq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbx+        mulq    %r10+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $59, %rdi, %rbp+        movq    %rbp, F+4*N(%rsp)+        shrdq   $59, %rbx, %rdi+        sarq    $59, %rbx++        movq    F+5*N(%rsp), %rax+        movq    %rdi, F+5*N(%rsp)++        movq    F+6*N(%rsp), %rdi+        movq    %rbx, F+6*N(%rsp)++        xorq    %r13, %rax+        xorq    %r13, %rdi+        andq    %r12, %rdi+        negq    %rdi+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rdi+        movq    G+5*N(%rsp), %rax+        xorq    %r15, %rax+        movq    G+6*N(%rsp), %rdx+        xorq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rdi+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rdi+        shrdq   $59, %rsi, %rcx+        movq    %rcx, G+4*N(%rsp)+        shrdq   $59, %rdi, %rsi+        movq    %rsi, G+5*N(%rsp)+        sarq    $59, %rdi+        movq    %rdi, G+6*N(%rsp)++// Get the initial carries back from storage and do the [u,v] accumulation++        movq    tmp, %rbx+        movq    tmp2, %rbp++// Digit 0 of [u,v]++        xorl    %ecx, %ecx+        movq    U(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    V(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx++        xorl    %esi, %esi+        movq    U(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, U(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    V(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, V(%rsp)++// Digit 1 of [u,v]++        xorl    %ebx, %ebx+        movq    U+N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    V+N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx++        xorl    %ebp, %ebp+        movq    U+N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, U+N(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    V+N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, V+N(%rsp)++// Digit 2 of [u,v]++        xorl    %ecx, %ecx+        movq    U+2*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    V+2*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx++        xorl    %esi, %esi+        movq    U+2*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, U+2*N(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    V+2*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, V+2*N(%rsp)++// Digit 3 of [u,v]++        xorl    %ebx, %ebx+        movq    U+3*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    V+3*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx++        xorl    %ebp, %ebp+        movq    U+3*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, U+3*N(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    V+3*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, V+3*N(%rsp)++// Digit 4 of [u,v]++        xorl    %ecx, %ecx+        movq    U+4*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    V+4*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx++        xorl    %esi, %esi+        movq    U+4*N(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, U+4*N(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    V+4*N(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, V+4*N(%rsp)++// Digits 5 and 6 of u (top is unsigned)++        movq    U+5*N(%rsp), %rax+        xorq    %r9, %rax+        movq    %r9, %rbx+        andq    %r8, %rbx+        negq    %rbx+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    V+5*N(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbx+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rbx, %rdx++// Preload for last use of old u digit 3++        movq    U+5*N(%rsp), %rax+        movq    %rcx, U+5*N(%rsp)+        movq    %rdx, U+6*N(%rsp)++// Digits 5 and 6 of v (top is unsigned)++        xorq    %r13, %rax+        movq    %r13, %rcx+        andq    %r12, %rcx+        negq    %rcx+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rcx+        movq    V+5*N(%rsp), %rax+        xorq    %r15, %rax+        movq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rcx+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rcx, %rdx+        movq    %rsi, V+5*N(%rsp)+        movq    %rdx, V+6*N(%rsp)++// Montgomery reduction of u++        amontred(u)++// Montgomery reduction of v++        amontred(v)++Lbignum_montinv_p384_midloop:++        divstep59(d,ff,gg)+        movq    %rsi, d++// Next iteration++        decq    i+        jnz     Lbignum_montinv_p384_loop++// The 15th and last iteration does not need anything except the+// u value and the sign of f; the latter can be obtained from the+// lowest word of f. So it's done differently from the main loop.+// Find the sign of the new f. For this we just need one digit+// since we know (for in-scope cases) that f is either +1 or -1.+// We don't explicitly shift right by 59 either, but looking at+// bit 63 (or any bit >= 60) of the unshifted result is enough+// to distinguish -1 from +1; this is then made into a mask.++        movq    F(%rsp), %rax+        movq    G(%rsp), %rcx+        imulq   %r8, %rax+        imulq   %r10, %rcx+        addq    %rcx, %rax+        sarq    $63, %rax++// Now separate out the matrix into sign-magnitude pairs+// and adjust each one based on the sign of f.+//+// Note that at this point we expect |f|=1 and we got its+// sign above, so then since [f,0] == x * 2^{-768} [u,v] (mod p_384)+// we want to flip the sign of u according to that of f.++        movq    %r8, %r9+        sarq    $63, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        xorq    %rax, %r9++        movq    %r10, %r11+        sarq    $63, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        xorq    %rax, %r11++        movq    %r12, %r13+        sarq    $63, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        xorq    %rax, %r13++        movq    %r14, %r15+        sarq    $63, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        xorq    %rax, %r15++// Adjust the initial value to allow for complement instead of negation++        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %r12+        andq    %r11, %r12+        addq    %rax, %r12++// Digit 0 of [u]++        xorl    %r13d, %r13d+        movq    U(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    V(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r12+        movq    %r12, U(%rsp)+        adcq    %rdx, %r13++// Digit 1 of [u]++        xorl    %r14d, %r14d+        movq    U+N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r13+        adcq    %rdx, %r14+        movq    V+N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r13+        movq    %r13, U+N(%rsp)+        adcq    %rdx, %r14++// Digit 2 of [u]++        xorl    %r15d, %r15d+        movq    U+2*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    V+2*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r14+        movq    %r14, U+2*N(%rsp)+        adcq    %rdx, %r15++// Digit 3 of [u]++        xorl    %r14d, %r14d+        movq    U+3*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r15+        adcq    %rdx, %r14+        movq    V+3*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r15+        movq    %r15, U+3*N(%rsp)+        adcq    %rdx, %r14++// Digit 4 of [u]++        xorl    %r15d, %r15d+        movq    U+4*N(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    V+4*N(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r14+        movq    %r14, U+4*N(%rsp)+        adcq    %rdx, %r15++// Digits 5 and 6 of u (top is unsigned)++        movq    U+5*N(%rsp), %rax+        xorq    %r9, %rax+        andq    %r8, %r9+        negq    %r9+        mulq    %r8+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    V+5*N(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %r9+        mulq    %r10+        addq    %rax, %r15+        movq    %r15, U+5*N(%rsp)+        adcq    %rdx, %r9+        movq    %r9, U+6*N(%rsp)++// Montgomery reduce u++        amontred(u)++// Perform final strict reduction mod p_384 and copy to output++        movl    $0xffffffff, %eax+        movq    %rax, %rbx+        notq    %rbx+        xorl    %ebp, %ebp+        leaq    -2(%rbp), %rcx+        leaq    -1(%rbp), %rdx++        movq    U(%rsp), %r8+        subq    %rax, %r8+        movq    U+N(%rsp), %r9+        sbbq    %rbx, %r9+        movq    U+2*N(%rsp), %r10+        sbbq    %rcx, %r10+        movq    U+3*N(%rsp), %r11+        sbbq    %rdx, %r11+        movq    U+4*N(%rsp), %r12+        sbbq    %rdx, %r12+        movq    U+5*N(%rsp), %r13+        sbbq    %rdx, %r13++        cmovcq  U(%rsp), %r8+        cmovcq  U+N(%rsp), %r9+        cmovcq  U+2*N(%rsp), %r10+        cmovcq  U+3*N(%rsp), %r11+        cmovcq  U+4*N(%rsp), %r12+        cmovcq  U+5*N(%rsp), %r13++        movq    res, %rdi+        movq    %r8, (%rdi)+        movq    %r9, N(%rdi)+        movq    %r10, 2*N(%rdi)+        movq    %r11, 3*N(%rdi)+        movq    %r12, 4*N(%rdi)+        movq    %r13, 5*N(%rdi)++// Restore stack and registers++        CFI_INC_RSP(NSPACE)++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_montinv_p384)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/bignum_montmul_p384.S view
@@ -0,0 +1,291 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Montgomery multiply, z := (x * y / 2^384) mod p_384+// Inputs x[6], y[6]; output z[6]+//+//    extern void bignum_montmul_p384(uint64_t z[static 6],+//                                    const uint64_t x[static 6],+//                                    const uint64_t y[static 6]);+//+// Does z := (2^{-384} * x * y) mod p_384, assuming that the inputs x and y+// satisfy x * y <= 2^384 * p_384 (in particular this is true if we are in+// the "usual" case x < p_384 and y < p_384).+//+// Standard x86-64 ABI: RDI = z, RSI = x, RDX = y+// Microsoft x64 ABI:   RCX = z, RDX = x, R8 = y+// -----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_montmul_p384)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_montmul_p384)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_montmul_p384)+        .text++#define z %rdi+#define x %rsi++// We move the y argument here so we can use %rdx for multipliers++#define y %rcx++// Some temp registers for the last correction stage++#define d %rax+#define u %rdx+#define v %rcx+#define w %rbx++// Add %rdx * m into a register-pair (high,low)+// maintaining consistent double-carrying with adcx and adox,+// using %rax and %rbx as temporaries++#define mulpadd(high,low,m)             \+        mulxq   m, %rax, %rbx ;            \+        adcxq   %rax, low ;               \+        adoxq   %rbx, high++// Core one-step Montgomery reduction macro. Takes input in+// [d7;d6;d5;d4;d3;d2;d1;d0] and returns result in [d7;d6;d5;d4;d3;d2;d1],+// adding to the existing contents, re-using d0 as a temporary internally+//+// We want to add (2^384 - 2^128 - 2^96 + 2^32 - 1) * w+// where w = [d0 + (d0<<32)] mod 2^64+//+//       montredc(d7,d6,d5,d4,d3,d2,d1,d0)+//+// This particular variant, with its mix of addition and subtraction+// at the top, is not intended to maintain a coherent carry or borrow out.+// It is assumed the final result would fit in [d7;d6;d5;d4;d3;d2;d1].+// which is always the case here as the top word is even always in {0,1}++#define montredc(d7,d6,d5,d4,d3,d2,d1,d0)                               \+/* Our correction multiplier is w = [d0 + (d0<<32)] mod 2^64 */         \+        movq    d0, %rdx ;                                        \+        shlq    $32, %rdx ;                                        \+        addq    d0, %rdx ;                                        \+/* Construct [%rbp;%rbx;%rax;-] = (2^384 - p_384) * w */                   \+/* We know the lowest word will cancel so we can re-use d0 as a temp */ \+        xorl    %ebp, %ebp ;                                       \+        movq    $0xffffffff00000001, %rax ;                        \+        mulxq   %rax, %rbx, %rax ;                                  \+        movl    $0x00000000ffffffff, %ebx ;                        \+        mulxq   %rbx, d0, %rbx ;                                   \+        adcq    d0, %rax ;                                        \+        adcq    %rdx, %rbx ;                                       \+        adcl    %ebp, %ebp ;                                       \+/*  Now subtract that and add 2^384 * w */                              \+        subq    %rax, d1 ;                                        \+        sbbq    %rbx, d2 ;                                        \+        sbbq    %rbp, d3 ;                                        \+        sbbq    $0, d4 ;                                          \+        sbbq    $0, d5 ;                                          \+        sbbq    $0, %rdx ;                                         \+        addq    %rdx, d6 ;                                        \+        adcq    $0, d7++S2N_BN_SYMBOL(bignum_montmul_p384):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+#endif++// Save more registers to play with++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++// Copy y into a safe register to start with++        movq    %rdx, y++// Do row 0 computation, which is a bit different:+// set up initial window [%r14,%r13,%r12,%r11,%r10,%r9,%r8] = y[0] * x+// Unlike later, we only need a single carry chain++        movq    (y), %rdx+        xorl    %r15d, %r15d+        mulxq   (x), %r8, %r9+        mulxq   8(x), %rbx, %r10+        addq    %rbx, %r9+        mulxq   16(x), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   24(x), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   32(x), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   40(x), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14++// Montgomery reduce the zeroth window++        montredc(%r15, %r14,%r13,%r12,%r11,%r10,%r9,%r8)++// Add row 1++        movq    8(y), %rdx+        xorl    %r8d, %r8d+        mulpadd(%r10,%r9,(x))+        mulpadd(%r11,%r10, 8(x))+        mulpadd(%r12,%r11,16(x))+        mulpadd(%r13,%r12,24(x))+        mulpadd(%r14,%r13,32(x))+        adoxq   %r8, %r15+        mulxq   40(x), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8++// Montgomery reduce window 1++        montredc(%r8, %r15,%r14,%r13,%r12,%r11,%r10,%r9)++// Add row 2++        movq    16(y), %rdx+        xorl    %r9d, %r9d+        mulpadd(%r11,%r10,(x))+        mulpadd(%r12,%r11,8(x))+        mulpadd(%r13,%r12,16(x))+        mulpadd(%r14,%r13,24(x))+        mulpadd(%r15,%r14,32(x))+        adoxq   %r9, %r8+        mulxq   40(x), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9++// Montgomery reduce window 2++        montredc(%r9, %r8,%r15,%r14,%r13,%r12,%r11,%r10)++// Add row 3++        movq    24(y), %rdx+        xorl    %r10d, %r10d+        mulpadd(%r12,%r11,(x))+        mulpadd(%r13,%r12,8(x))+        mulpadd(%r14,%r13,16(x))+        mulpadd(%r15,%r14,24(x))+        mulpadd(%r8,%r15,32(x))+        adoxq   %r10, %r9+        mulxq   40(x), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10++// Montgomery reduce window 3++        montredc(%r10, %r9,%r8,%r15,%r14,%r13,%r12,%r11)++// Add row 4++        movq    32(y), %rdx+        xorl    %r11d, %r11d+        mulpadd(%r13,%r12,(x))+        mulpadd(%r14,%r13,8(x))+        mulpadd(%r15,%r14,16(x))+        mulpadd(%r8,%r15,24(x))+        mulpadd(%r9,%r8,32(x))+        adoxq   %r11, %r10+        mulxq   40(x), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11++// Montgomery reduce window 4++        montredc(%r11, %r10,%r9,%r8,%r15,%r14,%r13,%r12)++// Add row 5++        movq    40(y), %rdx+        xorl    %r12d, %r12d+        mulpadd(%r14,%r13,(x))+        mulpadd(%r15,%r14,8(x))+        mulpadd(%r8,%r15,16(x))+        mulpadd(%r9,%r8,24(x))+        mulpadd(%r10,%r9,32(x))+        adoxq   %r12, %r11+        mulxq   40(x), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12++// Montgomery reduce window 5++        montredc(%r12, %r11,%r10,%r9,%r8,%r15,%r14,%r13)++// We now have a pre-reduced 7-word form z = [%r12; %r11;%r10;%r9;%r8;%r15;%r14]+// Next, accumulate in different registers z - p_384, or more precisely+//+//   [%r12; %r13;%rbp;%rdx;%rcx;%rbx;%rax] = z + (2^384 - p_384)++        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d++        movq    $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0x00000000ffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x0000000000000001, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0, %r12++// ~ZF <=> %r12 >= 1 <=> z + (2^384 - p_384) >= 2^384 <=> z >= p_384, which+// determines whether to use the further reduced argument or the original z.++        cmovnzq %rax, %r14+        cmovnzq %rbx, %r15+        cmovnzq %rcx, %r8+        cmovnzq %rdx, %r9+        cmovnzq %rbp, %r10+        cmovnzq %r13, %r11++// Write back the result++        movq    %r14, (z)+        movq    %r15, 8(z)+        movq    %r8, 16(z)+        movq    %r9, 24(z)+        movq    %r10, 32(z)+        movq    %r11, 40(z)++// Restore registers and return++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_montmul_p384)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_montmul_p384_alt.S view
@@ -0,0 +1,316 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Montgomery multiply, z := (x * y / 2^384) mod p_384+// Inputs x[6], y[6]; output z[6]+//+//    extern void bignum_montmul_p384_alt(uint64_t z[static 6],+//                                        const uint64_t x[static 6],+//                                        const uint64_t y[static 6]);+//+// Does z := (2^{-384} * x * y) mod p_384, assuming that the inputs x and y+// satisfy x * y <= 2^384 * p_384 (in particular this is true if we are in+// the "usual" case x < p_384 and y < p_384).+//+// Standard x86-64 ABI: RDI = z, RSI = x, RDX = y+// Microsoft x64 ABI:   RCX = z, RDX = x, R8 = y+// -----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_montmul_p384_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_montmul_p384_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_montmul_p384_alt)+        .text++#define z %rdi+#define x %rsi++// We move the y argument here so we can use %rdx for multipliers++#define y %rcx++// Some temp registers for the last correction stage++#define d %rax+#define u %rdx+#define v %rcx+#define w %rbx++// Add %rbx * m into a register-pair (high,low) maintaining consistent+// carry-catching with carry (negated, as bitmask) and using %rax and %rdx+// as temporaries++#define mulpadd(carry,high,low,m)       \+        movq    m, %rax ;                 \+        mulq    %rbx;                    \+        subq    carry, %rdx ;             \+        addq    %rax, low ;               \+        adcq    %rdx, high ;              \+        sbbq    carry, carry++// Initial version assuming no carry-in++#define mulpadi(carry,high,low,m)       \+        movq    m, %rax ;                 \+        mulq    %rbx;                    \+        addq    %rax, low ;               \+        adcq    %rdx, high ;              \+        sbbq    carry, carry++// End version not catching the top carry-out++#define mulpade(carry,high,low,m)       \+        movq    m, %rax ;                 \+        mulq    %rbx;                    \+        subq    carry, %rdx ;             \+        addq    %rax, low ;               \+        adcq    %rdx, high++// Core one-step Montgomery reduction macro. Takes input in+// [d7;d6;d5;d4;d3;d2;d1;d0] and returns result in [d7;d6;d5;d4;d3;d2;d1],+// adding to the existing contents, re-using d0 as a temporary internally+//+// We want to add (2^384 - 2^128 - 2^96 + 2^32 - 1) * w+// where w = [d0 + (d0<<32)] mod 2^64+//+//       montredc(d7,d6,d5,d4,d3,d2,d1,d0)+//+// This particular variant, with its mix of addition and subtraction+// at the top, is not intended to maintain a coherent carry or borrow out.+// It is assumed the final result would fit in [d7;d6;d5;d4;d3;d2;d1].+// which is always the case here as the top word is even always in {0,1}++#define montredc(d7,d6,d5,d4,d3,d2,d1,d0)                               \+/* Our correction multiplier is w = [d0 + (d0<<32)] mod 2^64 */         \+        movq    d0, %rbx ;                                        \+        shlq    $32, %rbx ;                                        \+        addq    d0, %rbx ;                                        \+/* Construct [%rbp;%rdx;%rax;-] = (2^384 - p_384) * w */                   \+/* We know the lowest word will cancel so we can re-use d0 as a temp */ \+        xorl    %ebp, %ebp ;                                       \+        movq    $0xffffffff00000001, %rax ;                        \+        mulq    %rbx;                                            \+        movq    %rdx, d0 ;                                        \+        movq    $0x00000000ffffffff, %rax ;                        \+        mulq    %rbx;                                            \+        addq    d0, %rax ;                                        \+        adcq    %rbx, %rdx ;                                       \+        adcl    %ebp, %ebp ;                                       \+/*  Now subtract that and add 2^384 * w */                              \+        subq    %rax, d1 ;                                        \+        sbbq    %rdx, d2 ;                                        \+        sbbq    %rbp, d3 ;                                        \+        sbbq    $0, d4 ;                                          \+        sbbq    $0, d5 ;                                          \+        sbbq    $0, %rbx ;                                         \+        addq    %rbx, d6 ;                                        \+        adcq    $0, d7++S2N_BN_SYMBOL(bignum_montmul_p384_alt):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+#endif++// Save more registers to play with++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++// Copy y into a safe register to start with++        movq    %rdx, y++// Do row 0 computation, which is a bit different:+// set up initial window [%r14,%r13,%r12,%r11,%r10,%r9,%r8] = y[0] * x+// Unlike later, we only need a single carry chain++        movq    (y), %rbx+        movq    (x), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9++        movq    8(x), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10++        movq    16(x), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11++        movq    24(x), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12++        movq    32(x), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13++        movq    40(x), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14++        xorl    %r15d, %r15d++// Montgomery reduce the zeroth window++        montredc(%r15, %r14,%r13,%r12,%r11,%r10,%r9,%r8)++// Add row 1++        movq    8(y), %rbx+        mulpadi(%r8,%r10,%r9,(x))+        mulpadd(%r8,%r11,%r10,8(x))+        mulpadd(%r8,%r12,%r11,16(x))+        mulpadd(%r8,%r13,%r12,24(x))+        mulpadd(%r8,%r14,%r13,32(x))+        mulpadd(%r8,%r15,%r14,40(x))+        negq    %r8++// Montgomery reduce window 1++        montredc(%r8, %r15,%r14,%r13,%r12,%r11,%r10,%r9)++// Add row 2++        movq    16(y), %rbx+        mulpadi(%r9,%r11,%r10,(x))+        mulpadd(%r9,%r12,%r11,8(x))+        mulpadd(%r9,%r13,%r12,16(x))+        mulpadd(%r9,%r14,%r13,24(x))+        mulpadd(%r9,%r15,%r14,32(x))+        mulpadd(%r9,%r8,%r15,40(x))+        negq    %r9++// Montgomery reduce window 2++        montredc(%r9, %r8,%r15,%r14,%r13,%r12,%r11,%r10)++// Add row 3++        movq    24(y), %rbx+        mulpadi(%r10,%r12,%r11,(x))+        mulpadd(%r10,%r13,%r12,8(x))+        mulpadd(%r10,%r14,%r13,16(x))+        mulpadd(%r10,%r15,%r14,24(x))+        mulpadd(%r10,%r8,%r15,32(x))+        mulpadd(%r10,%r9,%r8,40(x))+        negq    %r10++// Montgomery reduce window 3++        montredc(%r10, %r9,%r8,%r15,%r14,%r13,%r12,%r11)++// Add row 4++        movq    32(y), %rbx+        mulpadi(%r11,%r13,%r12,(x))+        mulpadd(%r11,%r14,%r13,8(x))+        mulpadd(%r11,%r15,%r14,16(x))+        mulpadd(%r11,%r8,%r15,24(x))+        mulpadd(%r11,%r9,%r8,32(x))+        mulpadd(%r11,%r10,%r9,40(x))+        negq    %r11++// Montgomery reduce window 4++        montredc(%r11, %r10,%r9,%r8,%r15,%r14,%r13,%r12)++// Add row 5++        movq    40(y), %rbx+        mulpadi(%r12,%r14,%r13,(x))+        mulpadd(%r12,%r15,%r14,8(x))+        mulpadd(%r12,%r8,%r15,16(x))+        mulpadd(%r12,%r9,%r8,24(x))+        mulpadd(%r12,%r10,%r9,32(x))+        mulpadd(%r12,%r11,%r10,40(x))+        negq    %r12++// Montgomery reduce window 5++        montredc(%r12, %r11,%r10,%r9,%r8,%r15,%r14,%r13)++// We now have a pre-reduced 7-word form z = [%r12; %r11;%r10;%r9;%r8;%r15;%r14]+// Next, accumulate in different registers z - p_384, or more precisely+//+//   [%r12; %r13;%rbp;%rdx;%rcx;%rbx;%rax] = z + (2^384 - p_384)++        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d++        movq    $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0x00000000ffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x0000000000000001, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0, %r12++// ~ZF <=> %r12 >= 1 <=> z + (2^384 - p_384) >= 2^384 <=> z >= p_384, which+// determines whether to use the further reduced argument or the original z.++        cmovnzq %rax, %r14+        cmovnzq %rbx, %r15+        cmovnzq %rcx, %r8+        cmovnzq %rdx, %r9+        cmovnzq %rbp, %r10+        cmovnzq %r13, %r11++// Write back the result++        movq    %r14, (z)+        movq    %r15, 8(z)+        movq    %r8, 16(z)+        movq    %r9, 24(z)+        movq    %r10, 32(z)+        movq    %r11, 40(z)++// Restore registers and return++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_montmul_p384_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_montsqr_p384.S view
@@ -0,0 +1,294 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Montgomery square, z := (x^2 / 2^384) mod p_384+// Input x[6]; output z[6]+//+//    extern void bignum_montsqr_p384(uint64_t z[static 6],+//                                    const uint64_t x[static 6]);+//+// Does z := (x^2 / 2^384) mod p_384, assuming x^2 <= 2^384 * p_384, which is+// guaranteed in particular if x < p_384 initially (the "intended" case).+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_montsqr_p384)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_montsqr_p384)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_montsqr_p384)+        .text++#define z %rdi+#define x %rsi++// Some temp registers for the last correction stage++#define d %rax+#define u %rdx+#define v %r10+#define w %r11++// A zero register, very often++#define zero %rbp+#define zeroe %ebp++// Add %rdx * m into a register-pair (high,low)+// maintaining consistent double-carrying with adcx and adox,+// using %rax and %rbx as temporaries++#define mulpadd(high,low,m)             \+        mulxq   m, %rax, %rbx ;            \+        adcxq   %rax, low ;               \+        adoxq   %rbx, high++// Core one-step "short" Montgomery reduction macro. Takes input in+// [d5;d4;d3;d2;d1;d0] and returns result in [d6;d5;d4;d3;d2;d1],+// adding to the existing [d5;d4;d3;d2;d1] and re-using d0 as a+// temporary internally, as well as %rax, %rbx and %rdx.+// It is OK for d6 and d0 to be the same register (they often are)+//+// We want to add (2^384 - 2^128 - 2^96 + 2^32 - 1) * w+// where w = [d0 + (d0<<32)] mod 2^64+//+//       montreds(d6,d5,d4,d3,d2,d1,d0)++#define montreds(d6,d5,d4,d3,d2,d1,d0)                                  \+/* Our correction multiplier is w = [d0 + (d0<<32)] mod 2^64 */         \+        movq    d0, %rdx ;                                        \+        shlq    $32, %rdx ;                                        \+        addq    d0, %rdx ;                                        \+/* Construct [%rbx;d0;%rax;-] = (2^384 - p_384) * w            */         \+/* We know the lowest word will cancel so we can re-use d0   */         \+/* and %rbx as temps.                                         */         \+        movq    $0xffffffff00000001, %rax ;                        \+        mulxq   %rax, d0, %rax ;                                   \+        movl    $0x00000000ffffffff, %ebx ;                        \+        mulxq   %rbx, %rbx, d0 ;                                   \+        addq    %rbx, %rax ;                                       \+        adcq    %rdx, d0 ;                                        \+        movl    $0, %ebx ;                                         \+        adcq    %rbx, %rbx ;                                       \+/* Now subtract that and add 2^384 * w                       */         \+        subq    %rax, d1 ;                                        \+        sbbq    d0, d2 ;                                         \+        sbbq    %rbx, d3 ;                                        \+        sbbq    $0, d4 ;                                          \+        sbbq    $0, d5 ;                                          \+        movq    %rdx, d6 ;                                        \+        sbbq    $0, d6++S2N_BN_SYMBOL(bignum_montsqr_p384):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Save more registers to play with++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++// Set up an initial window [%rcx;%r15;...%r9] = [34;05;03;01]+// Note that we are using %rcx as the first step past the rotating window++        movq    (x), %rdx+        mulxq   8(x), %r9, %r10+        mulxq   24(x), %r11, %r12+        mulxq   40(x), %r13, %r14+        movq    24(x), %rdx+        mulxq   32(x), %r15, %rcx++// Clear our zero register, and also initialize the flags for the carry chain++        xorl    zeroe, zeroe++// Chain in the addition of 02 + 12 + 13 + 14 + 15 to that window+// (no carry-out possible)++        movq    16(x), %rdx+        mulpadd(%r11,%r10,(x))+        mulpadd(%r12,%r11,8(x))+        movq    8(x), %rdx+        mulpadd(%r13,%r12,24(x))+        mulpadd(%r14,%r13,32(x))+        mulpadd(%r15,%r14,40(x))+        adcxq   zero, %r15+        adoxq   zero, %rcx+        adcq    zero, %rcx++// Again zero out the flags. Actually they are already cleared but it may+// help decouple these in the OOO engine not to wait for the chain above++        xorl    zeroe, zeroe++// Now chain in the 04 + 23 + 24 + 25 + 35 + 45 terms+// We are running out of registers in our rotating window, so we start+// using %rbx (and hence need care with using mulpadd after this). Thus+// our result so far is in [%rbp;%rbx;%rcx;%r15;...%r9]++        movq    32(x), %rdx+        mulpadd(%r13,%r12,(x))+        movq    16(x), %rdx+        mulpadd(%r14,%r13,24(x))+        mulpadd(%r15,%r14,32(x))+        mulxq   40(x), %rax, %rdx+        adcxq   %rax, %r15+        adoxq   %rdx, %rcx++// First set up the last couple of spots in our window, [%rbp;%rbx] = 45+// then add the last other term 35++        movq    40(x), %rdx+        mulxq   32(x), %rbx, %rbp+        mulxq   24(x), %rax, %rdx+        adcxq   %rax, %rcx+        adoxq   %rdx, %rbx+        movl    $0, %eax+        adcxq   %rax, %rbx+        adoxq   %rax, %rbp+        adcq    %rax, %rbp++// Just for a clear fresh start for the flags; we don't use the zero++        xorq    %rax, %rax++// Double and add to the 00 + 11 + 22 + 33 + 44 + 55 terms+// For one glorious moment the entire squaring result is all in the+// register file as [%rsi;%rbp;%rbx;%rcx;%r15;...;%r8]+// (since we've now finished with x we can re-use %rsi)++        movq    (x), %rdx+        mulxq   (x), %r8, %rax+        adcxq   %r9, %r9+        adoxq   %rax, %r9+        movq    8(x), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    16(x), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    24(x), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %r15, %r15+        adoxq   %rdx, %r15+        movq    32(x), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %rcx, %rcx+        adoxq   %rax, %rcx+        adcxq   %rbx, %rbx+        adoxq   %rdx, %rbx+        movq    40(x), %rdx+        mulxq   %rdx, %rax, %rsi+        adcxq   %rbp, %rbp+        adoxq   %rax, %rbp+        movl    $0, %eax+        adcxq   %rax, %rsi+        adoxq   %rax, %rsi++// We need just *one* more register as a temp for the Montgomery steps.+// Since we are writing to the z buffer anyway, make use of that to stash %rbx.++        movq    %rbx, (z)++// Montgomery reduce the %r13,...,%r8 window 6 times++        montreds(%r8,%r13,%r12,%r11,%r10,%r9,%r8)+        montreds(%r9,%r8,%r13,%r12,%r11,%r10,%r9)+        montreds(%r10,%r9,%r8,%r13,%r12,%r11,%r10)+        montreds(%r11,%r10,%r9,%r8,%r13,%r12,%r11)+        montreds(%r12,%r11,%r10,%r9,%r8,%r13,%r12)+        montreds(%r13,%r12,%r11,%r10,%r9,%r8,%r13)++// Now we can safely restore %rbx before accumulating++        movq    (z), %rbx++        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rsi+        movl    $0, %r8d+        adcq    %r8, %r8++// We now have a pre-reduced 7-word form z = [%r8; %rsi;%rbp;%rbx;%rcx;%r15;%r14]+// Next, accumulate in different registers z - p_384, or more precisely+//+//   [%r8; %r13;%r12;%r11;%r10;%r9;%rax] = z + (2^384 - p_384)++        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movq    $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0x00000000ffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x0000000000000001, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rsi, %r13+        adcq    $0, %r8++// ~ZF <=> %r12 >= 1 <=> z + (2^384 - p_384) >= 2^384 <=> z >= p_384, which+// determines whether to use the further reduced argument or the original z.++        cmovnzq %rax, %r14+        cmovnzq %r9, %r15+        cmovnzq %r10, %rcx+        cmovnzq %r11, %rbx+        cmovnzq %r12, %rbp+        cmovnzq %r13, %rsi++// Write back the result++        movq    %r14, (z)+        movq    %r15, 8(z)+        movq    %rcx, 16(z)+        movq    %rbx, 24(z)+        movq    %rbp, 32(z)+        movq    %rsi, 40(z)++// Restore registers and return++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_montsqr_p384)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_montsqr_p384_alt.S view
@@ -0,0 +1,339 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Montgomery square, z := (x^2 / 2^384) mod p_384+// Input x[6]; output z[6]+//+//    extern void bignum_montsqr_p384_alt(uint64_t z[static 6],+//                                        const uint64_t x[static 6]);+//+// Does z := (x^2 / 2^384) mod p_384, assuming x^2 <= 2^384 * p_384, which is+// guaranteed in particular if x < p_384 initially (the "intended" case).+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_montsqr_p384_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_montsqr_p384_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_montsqr_p384_alt)+        .text++#define z %rdi+#define x %rsi++// Some temp registers for the last correction stage++#define d %rax+#define u %rdx+#define v %r10+#define w %r11++// A zero register, very often++#define zero %rbp+#define zeroe %ebp++// Add %rbx * m into a register-pair (high,low) maintaining consistent+// carry-catching with carry (negated, as bitmask) and using %rax and %rdx+// as temporaries++#define mulpadd(carry,high,low,m)       \+        movq    m, %rax ;                 \+        mulq    %rbx;                    \+        subq    carry, %rdx ;             \+        addq    %rax, low ;               \+        adcq    %rdx, high ;              \+        sbbq    carry, carry++// Initial version assuming no carry-in++#define mulpadi(carry,high,low,m)       \+        movq    m, %rax ;                 \+        mulq    %rbx;                    \+        addq    %rax, low ;               \+        adcq    %rdx, high ;              \+        sbbq    carry, carry++// End version not catching the top carry-out++#define mulpade(carry,high,low,m)       \+        movq    m, %rax ;                 \+        mulq    %rbx;                    \+        subq    carry, %rdx ;             \+        addq    %rax, low ;               \+        adcq    %rdx, high++// Core one-step "short" Montgomery reduction macro. Takes input in+// [d5;d4;d3;d2;d1;d0] and returns result in [d6;d5;d4;d3;d2;d1],+// adding to the existing [d5;d4;d3;d2;d1] and re-using d0 as a+// temporary internally, as well as %rax, %rbx and %rdx.+// It is OK for d6 and d0 to be the same register (they often are)+//+// We want to add (2^384 - 2^128 - 2^96 + 2^32 - 1) * w+// where w = [d0 + (d0<<32)] mod 2^64+//+//       montreds(d6,d5,d4,d3,d2,d1,d0)++#define montreds(d6,d5,d4,d3,d2,d1,d0)                                  \+/* Our correction multiplier is w = [d0 + (d0<<32)] mod 2^64 */         \+        movq    d0, %rbx ;                                        \+        shlq    $32, %rbx ;                                        \+        addq    d0, %rbx ;                                        \+/* Construct [%rax;%rdx;d0;-] = (2^384 - p_384) * w            */         \+/* We know the lowest word will cancel so we can re-use d0   */         \+/* and %rbx as temps.                                         */         \+        movq    $0xffffffff00000001, %rax ;                        \+        mulq    %rbx;                                            \+        movq    %rdx, d0 ;                                        \+        movq    $0x00000000ffffffff, %rax ;                        \+        mulq    %rbx;                                            \+        addq    %rax, d0 ;                                        \+        movl    $0, %eax ;                                         \+        adcq    %rbx, %rdx ;                                       \+        adcl    %eax, %eax ;                                       \+/* Now subtract that and add 2^384 * w                       */         \+        subq    d0, d1 ;                                         \+        sbbq    %rdx, d2 ;                                        \+        sbbq    %rax, d3 ;                                        \+        sbbq    $0, d4 ;                                          \+        sbbq    $0, d5 ;                                          \+        movq    %rbx, d6 ;                                        \+        sbbq    $0, d6++S2N_BN_SYMBOL(bignum_montsqr_p384_alt):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Save more registers to play with++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++// Set up an initial window [%rcx;%r15;...%r9] = [34;05;03;01]+// Note that we are using %rcx as the first step past the rotating window++        movq    (x), %rbx+        movq    8(x), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10++        movq    24(x), %rax+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12++        movq    40(x), %rax+        mulq    %rbx+        movq    %rax, %r13+        movq    %rdx, %r14++        movq    24(x), %rax+        mulq     32(x)+        movq    %rax, %r15+        movq    %rdx, %rcx++// Chain in the addition of 02 + 12 + 13 + 14 + 15 to that window+// (no carry-out possible)++        movq    16(x), %rbx+        mulpadi(%rbp,%r11,%r10,(x))+        mulpadd(%rbp,%r12,%r11,8(x))+        movq    8(x), %rbx+        mulpadd(%rbp,%r13,%r12,24(x))+        mulpadd(%rbp,%r14,%r13,32(x))+        mulpade(%rbp,%r15,%r14,40(x))+        adcq    $0, %rcx++// Now chain in the 04 + 23 + 24 + 25 + 35 + 45 terms+// We are running out of registers in our rotating window, so we start+// using %rbx (and hence need care with using mulpadd after this). Thus+// our result so far is in [%rbp;%rbx;%rcx;%r15;...%r9]++        movq    32(x), %rbx+        mulpadi(%rbp,%r13,%r12,(x))+        movq    16(x), %rbx+        mulpadd(%rbp,%r14,%r13,24(x))+        mulpadd(%rbp,%r15,%r14,32(x))+        mulpadd(%rbp,%rcx,%r15,40(x))++        xorl    %ebx, %ebx+        movq    24(x), %rax+        mulq     40(x)+        subq    %rbp, %rdx+        xorl    %ebp, %ebp+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        movq    32(x), %rax+        mulq     40(x)+        addq    %rax, %rbx+        adcq    %rdx, %rbp++// Double the window as [%r8;%rbp;%rbx;%rcx;%r15;...%r9]++        xorl    %r8d, %r8d+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %r15, %r15+        adcq    %rcx, %rcx+        adcq    %rbx, %rbx+        adcq    %rbp, %rbp+        adcl    %r8d, %r8d++// Add the doubled window to the 00 + 11 + 22 + 33 + 44 + 55 terms+// For one glorious moment the entire squaring result is all in the+// register file as [%rsi;%rbp;%rbx;%rcx;%r15;...;%r8]+// (since we've now finished with x we can re-use %rsi). But since+// we are so close to running out of registers, we do a bit of+// reshuffling and temporary storage in the output buffer.++        movq    (x), %rax+        mulq    %rax+        movq    %r8, (z)+        movq    %rax, %r8+        movq    8(x), %rax+        movq    %rbp, 8(z)+        addq    %rdx, %r9+        sbbq    %rbp, %rbp++        mulq    %rax+        negq    %rbp+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp++        movq    16(x), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp++        movq    24(x), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp++        movq    32(x), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %rcx+        adcq    %rdx, %rbx+        sbbq    %rbp, %rbp++        movq    40(x), %rax+        mulq    %rax+        negq    %rbp+        adcq    8(z), %rax+        adcq    (z), %rdx+        movq    %rax, %rbp+        movq    %rdx, %rsi++// We need just *one* more register as a temp for the Montgomery steps.+// Since we are writing to the z buffer anyway, make use of that again+// to stash %rbx.++        movq    %rbx, (z)++// Montgomery reduce the %r13,...,%r8 window 6 times++        montreds(%r8,%r13,%r12,%r11,%r10,%r9,%r8)+        montreds(%r9,%r8,%r13,%r12,%r11,%r10,%r9)+        montreds(%r10,%r9,%r8,%r13,%r12,%r11,%r10)+        montreds(%r11,%r10,%r9,%r8,%r13,%r12,%r11)+        montreds(%r12,%r11,%r10,%r9,%r8,%r13,%r12)+        montreds(%r13,%r12,%r11,%r10,%r9,%r8,%r13)++// Now we can safely restore %rbx before accumulating++        movq    (z), %rbx++        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rsi+        movl    $0, %r8d+        adcq    %r8, %r8++// We now have a pre-reduced 7-word form z = [%r8; %rsi;%rbp;%rbx;%rcx;%r15;%r14]+// Next, accumulate in different registers z - p_384, or more precisely+//+//   [%r8; %r13;%r12;%r11;%r10;%r9;%rax] = z + (2^384 - p_384)++        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movq    $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0x00000000ffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x0000000000000001, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rsi, %r13+        adcq    $0, %r8++// ~ZF <=> %r12 >= 1 <=> z + (2^384 - p_384) >= 2^384 <=> z >= p_384, which+// determines whether to use the further reduced argument or the original z.++        cmovnzq %rax, %r14+        cmovnzq %r9, %r15+        cmovnzq %r10, %rcx+        cmovnzq %r11, %rbx+        cmovnzq %r12, %rbp+        cmovnzq %r13, %rsi++// Write back the result++        movq    %r14, (z)+        movq    %r15, 8(z)+        movq    %rcx, 16(z)+        movq    %rbx, 24(z)+        movq    %rbp, 32(z)+        movq    %rsi, 40(z)++// Restore registers and return++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_montsqr_p384_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_mul_p521.S view
@@ -0,0 +1,394 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Multiply modulo p_521, z := (x * y) mod p_521, assuming x and y reduced+// Inputs x[9], y[9]; output z[9]+//+//    extern void bignum_mul_p521(uint64_t z[static 9], const uint64_t x[static 9],+//                                const uint64_t y[static 9]);+//+// Standard x86-64 ABI: RDI = z, RSI = x, RDX = y+// Microsoft x64 ABI:   RCX = z, RDX = x, R8 = y+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_mul_p521)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_mul_p521)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_mul_p521)+        .text++#define z %rdi+#define x %rsi++// Copied in++#define y %rcx++// mulpadd (high,low,x) adds rdx * x to a register-pair (high,low)+// maintaining consistent double-carrying with adcx and adox,+// using %rax and %rbx as temporaries.++#define mulpadd(high,low,x)             \+        mulxq   x, %rax, %rbx ;            \+        adcxq   %rax, low ;               \+        adoxq   %rbx, high++S2N_BN_SYMBOL(bignum_mul_p521):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+#endif++// Save more registers to play with and make temporary space on stack++        CFI_PUSH(%rbp)+        CFI_PUSH(%rbx)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(64)++// Copy y into a safe register to start with++        movq    %rdx, y++// Clone of the main body of bignum_8_16, writing back the low 8 words to+// the temporary buffer on the stack and keeping the top half in %r15,...,%r8++        xorl   %ebp, %ebp+        movq   (y), %rdx+        mulxq  (x), %r8, %r9+        movq   %r8, (%rsp)+        mulxq  0x8(x), %rbx, %r10+        adcq   %rbx, %r9+        mulxq  0x10(x), %rbx, %r11+        adcq   %rbx, %r10+        mulxq  0x18(x), %rbx, %r12+        adcq   %rbx, %r11+        mulxq  0x20(x), %rbx, %r13+        adcq   %rbx, %r12+        mulxq  0x28(x), %rbx, %r14+        adcq   %rbx, %r13+        mulxq  0x30(x), %rbx, %r15+        adcq   %rbx, %r14+        mulxq  0x38(x), %rbx, %r8+        adcq   %rbx, %r15+        adcq   %rbp, %r8+        movq   0x8(y), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        movq   %r9, 0x8(%rsp)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x38(x), %rax, %r9+        adcxq  %rax, %r8+        adoxq  %rbp, %r9+        adcq   %rbp, %r9+        movq   0x10(y), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        movq   %r10, 0x10(%rsp)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x38(x), %rax, %r10+        adcxq  %rax, %r9+        adoxq  %rbp, %r10+        adcq   %rbp, %r10+        movq   0x18(y), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        movq   %r11, 0x18(%rsp)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x38(x), %rax, %r11+        adcxq  %rax, %r10+        adoxq  %rbp, %r11+        adcq   %rbp, %r11+        movq   0x20(y), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        movq   %r12, 0x20(%rsp)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x38(x), %rax, %r12+        adcxq  %rax, %r11+        adoxq  %rbp, %r12+        adcq   %rbp, %r12+        movq   0x28(y), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        movq   %r13, 0x28(%rsp)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x38(x), %rax, %r13+        adcxq  %rax, %r12+        adoxq  %rbp, %r13+        adcq   %rbp, %r13+        movq   0x30(y), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r14+        adoxq  %rbx, %r15+        movq   %r14, 0x30(%rsp)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x38(x), %rax, %r14+        adcxq  %rax, %r13+        adoxq  %rbp, %r14+        adcq   %rbp, %r14+        movq   0x38(y), %rdx+        xorl   %ebp, %ebp+        mulxq  (x), %rax, %rbx+        adcxq  %rax, %r15+        adoxq  %rbx, %r8+        movq   %r15, 0x38(%rsp)+        mulxq  0x8(x), %rax, %rbx+        adcxq  %rax, %r8+        adoxq  %rbx, %r9+        mulxq  0x10(x), %rax, %rbx+        adcxq  %rax, %r9+        adoxq  %rbx, %r10+        mulxq  0x18(x), %rax, %rbx+        adcxq  %rax, %r10+        adoxq  %rbx, %r11+        mulxq  0x20(x), %rax, %rbx+        adcxq  %rax, %r11+        adoxq  %rbx, %r12+        mulxq  0x28(x), %rax, %rbx+        adcxq  %rax, %r12+        adoxq  %rbx, %r13+        mulxq  0x30(x), %rax, %rbx+        adcxq  %rax, %r13+        adoxq  %rbx, %r14+        mulxq  0x38(x), %rax, %r15+        adcxq  %rax, %r14+        adoxq  %rbp, %r15+        adcq   %rbp, %r15++// Accumulate x[8] * y[0..7], extending the window to %rbp,%r15,...,%r8++        movq    64(x), %rdx+        xorl    %ebp, %ebp+        mulpadd(%r9,%r8,(y))+        mulpadd(%r10,%r9,8(y))+        mulpadd(%r11,%r10,16(y))+        mulpadd(%r12,%r11,24(y))+        mulpadd(%r13,%r12,32(y))+        mulpadd(%r14,%r13,40(y))+        mulpadd(%r15,%r14,48(y))+        mulxq   56(y), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbp, %rbx+        adcq    %rbx, %rbp++// Accumulate y[8] * x[0..8] within this extended window %rbp,%r15,...,%r8++        movq    64(y), %rdx+        xorl    %eax, %eax+        mulpadd(%r9,%r8,(x))+        mulpadd(%r10,%r9,8(x))+        mulpadd(%r11,%r10,16(x))+        mulpadd(%r12,%r11,24(x))+        mulpadd(%r13,%r12,32(x))+        mulpadd(%r14,%r13,40(x))+        mulpadd(%r15,%r14,48(x))+        mulxq   56(x), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %rbp+        mulxq   64(x), %rax, %rbx+        adcq    %rax, %rbp++// Rotate the upper portion right 9 bits since 2^512 == 2^-9 (mod p_521)+// Let rotated result %rbp,%r15,%r14,...,%r8 be h (high) and %rsp[0..7] be l (low)++        movq    %r8, %rax+        andq    $0x1FF, %rax+        shrdq   $9, %r9, %r8+        shrdq   $9, %r10, %r9+        shrdq   $9, %r11, %r10+        shrdq   $9, %r12, %r11+        shrdq   $9, %r13, %r12+        shrdq   $9, %r14, %r13+        shrdq   $9, %r15, %r14+        shrdq   $9, %rbp, %r15+        shrq    $9, %rbp+        addq    %rax, %rbp++// Force carry-in then add to get s = h + l + 1+// but actually add all 1s in the top 53 bits to get simple carry out++        stc+        adcq    (%rsp), %r8+        adcq    8(%rsp), %r9+        adcq    16(%rsp), %r10+        adcq    24(%rsp), %r11+        adcq    32(%rsp), %r12+        adcq    40(%rsp), %r13+        adcq    48(%rsp), %r14+        adcq    56(%rsp), %r15+        adcq    $~0x1FF, %rbp++// Now CF is set <=> h + l + 1 >= 2^521 <=> h + l >= p_521,+// in which case the lower 521 bits are already right. Otherwise if+// CF is clear, we want to subtract 1. Hence subtract the complement+// of the carry flag then mask the top word, which scrubs the+// padding in either case. Write digits back as they are created.++        cmc+        sbbq    $0, %r8+        movq    %r8, (z)+        sbbq    $0, %r9+        movq    %r9, 8(z)+        sbbq    $0, %r10+        movq    %r10, 16(z)+        sbbq    $0, %r11+        movq    %r11, 24(z)+        sbbq    $0, %r12+        movq    %r12, 32(z)+        sbbq    $0, %r13+        movq    %r13, 40(z)+        sbbq    $0, %r14+        movq    %r14, 48(z)+        sbbq    $0, %r15+        movq    %r15, 56(z)+        sbbq    $0, %rbp+        andq    $0x1FF, %rbp+        movq    %rbp, 64(z)++// Restore registers and return++        CFI_INC_RSP(64)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbx)+        CFI_POP(%rbp)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_mul_p521)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_mul_p521_alt.S view
@@ -0,0 +1,321 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Multiply modulo p_521, z := (x * y) mod p_521, assuming x and y reduced+// Inputs x[9], y[9]; output z[9]+//+//    extern void bignum_mul_p521_alt(uint64_t z[static 9],+//                                    const uint64_t x[static 9],+//                                    const uint64_t y[static 9]);+//+// Standard x86-64 ABI: RDI = z, RSI = x, RDX = y+// Microsoft x64 ABI:   RCX = z, RDX = x, R8 = y+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_mul_p521_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_mul_p521_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_mul_p521_alt)+        .text++#define z %rdi+#define x %rsi++// This is moved from %rdx to free it for muls++#define y %rcx++// Macro for the key "multiply and add to (c,h,l)" step++#define combadd(c,h,l,numa,numb)                \+        movq    numa, %rax ;                      \+        mulq     numb;                 \+        addq    %rax, l ;                         \+        adcq    %rdx, h ;                         \+        adcq    $0, c++// A minutely shorter form for when c = 0 initially++#define combadz(c,h,l,numa,numb)                \+        movq    numa, %rax ;                      \+        mulq     numb;                 \+        addq    %rax, l ;                         \+        adcq    %rdx, h ;                         \+        adcq    c, c++// A short form where we don't expect a top carry++#define combads(h,l,numa,numb)                  \+        movq    numa, %rax ;                      \+        mulq     numb;                 \+        addq    %rax, l ;                         \+        adcq    %rdx, h++S2N_BN_SYMBOL(bignum_mul_p521_alt):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+#endif++// Make more registers available and make temporary space on stack++        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(72)++// Copy y into a safe register to start with++        movq    %rdx, y++// Start doing a conventional columnwise multiplication,+// temporarily storing the lower 9 digits to the stack.+// Start with result term 0++        movq    (x), %rax+        mulq     (y)++        movq    %rax, (%rsp)+        movq    %rdx, %r9+        xorq    %r10, %r10++// Result term 1++        xorq    %r11, %r11+        combads(%r10,%r9,(x),8(y))+        combadz(%r11,%r10,%r9,8(x),(y))+        movq    %r9, 8(%rsp)++// Result term 2++        xorq    %r12, %r12+        combadz(%r12,%r11,%r10,(x),16(y))+        combadd(%r12,%r11,%r10,8(x),8(y))+        combadd(%r12,%r11,%r10,16(x),(y))+        movq    %r10, 16(%rsp)++// Result term 3++        xorq    %r13, %r13+        combadz(%r13,%r12,%r11,(x),24(y))+        combadd(%r13,%r12,%r11,8(x),16(y))+        combadd(%r13,%r12,%r11,16(x),8(y))+        combadd(%r13,%r12,%r11,24(x),(y))+        movq    %r11, 24(%rsp)++// Result term 4++        xorq    %r14, %r14+        combadz(%r14,%r13,%r12,(x),32(y))+        combadd(%r14,%r13,%r12,8(x),24(y))+        combadd(%r14,%r13,%r12,16(x),16(y))+        combadd(%r14,%r13,%r12,24(x),8(y))+        combadd(%r14,%r13,%r12,32(x),(y))+        movq    %r12, 32(%rsp)++// Result term 5++        xorq    %r15, %r15+        combadz(%r15,%r14,%r13,(x),40(y))+        combadd(%r15,%r14,%r13,8(x),32(y))+        combadd(%r15,%r14,%r13,16(x),24(y))+        combadd(%r15,%r14,%r13,24(x),16(y))+        combadd(%r15,%r14,%r13,32(x),8(y))+        combadd(%r15,%r14,%r13,40(x),(y))+        movq    %r13, 40(%rsp)++// Result term 6++        xorq    %r8, %r8+        combadz(%r8,%r15,%r14,(x),48(y))+        combadd(%r8,%r15,%r14,8(x),40(y))+        combadd(%r8,%r15,%r14,16(x),32(y))+        combadd(%r8,%r15,%r14,24(x),24(y))+        combadd(%r8,%r15,%r14,32(x),16(y))+        combadd(%r8,%r15,%r14,40(x),8(y))+        combadd(%r8,%r15,%r14,48(x),(y))+        movq    %r14, 48(%rsp)++// Result term 7++        xorq    %r9, %r9+        combadz(%r9,%r8,%r15,(x),56(y))+        combadd(%r9,%r8,%r15,8(x),48(y))+        combadd(%r9,%r8,%r15,16(x),40(y))+        combadd(%r9,%r8,%r15,24(x),32(y))+        combadd(%r9,%r8,%r15,32(x),24(y))+        combadd(%r9,%r8,%r15,40(x),16(y))+        combadd(%r9,%r8,%r15,48(x),8(y))+        combadd(%r9,%r8,%r15,56(x),(y))+        movq    %r15, 56(%rsp)++// Result term 8++        xorq    %r10, %r10+        combadz(%r10,%r9,%r8,(x),64(y))+        combadd(%r10,%r9,%r8,8(x),56(y))+        combadd(%r10,%r9,%r8,16(x),48(y))+        combadd(%r10,%r9,%r8,24(x),40(y))+        combadd(%r10,%r9,%r8,32(x),32(y))+        combadd(%r10,%r9,%r8,40(x),24(y))+        combadd(%r10,%r9,%r8,48(x),16(y))+        combadd(%r10,%r9,%r8,56(x),8(y))+        combadd(%r10,%r9,%r8,64(x),(y))+        movq    %r8, 64(%rsp)++// At this point we suspend writing back results and collect them+// in a register window. Next is result term 9++        xorq    %r11, %r11+        combadz(%r11,%r10,%r9,8(x),64(y))+        combadd(%r11,%r10,%r9,16(x),56(y))+        combadd(%r11,%r10,%r9,24(x),48(y))+        combadd(%r11,%r10,%r9,32(x),40(y))+        combadd(%r11,%r10,%r9,40(x),32(y))+        combadd(%r11,%r10,%r9,48(x),24(y))+        combadd(%r11,%r10,%r9,56(x),16(y))+        combadd(%r11,%r10,%r9,64(x),8(y))++// Result term 10++        xorq    %r12, %r12+        combadz(%r12,%r11,%r10,16(x),64(y))+        combadd(%r12,%r11,%r10,24(x),56(y))+        combadd(%r12,%r11,%r10,32(x),48(y))+        combadd(%r12,%r11,%r10,40(x),40(y))+        combadd(%r12,%r11,%r10,48(x),32(y))+        combadd(%r12,%r11,%r10,56(x),24(y))+        combadd(%r12,%r11,%r10,64(x),16(y))++// Result term 11++        xorq    %r13, %r13+        combadz(%r13,%r12,%r11,24(x),64(y))+        combadd(%r13,%r12,%r11,32(x),56(y))+        combadd(%r13,%r12,%r11,40(x),48(y))+        combadd(%r13,%r12,%r11,48(x),40(y))+        combadd(%r13,%r12,%r11,56(x),32(y))+        combadd(%r13,%r12,%r11,64(x),24(y))++// Result term 12++        xorq    %r14, %r14+        combadz(%r14,%r13,%r12,32(x),64(y))+        combadd(%r14,%r13,%r12,40(x),56(y))+        combadd(%r14,%r13,%r12,48(x),48(y))+        combadd(%r14,%r13,%r12,56(x),40(y))+        combadd(%r14,%r13,%r12,64(x),32(y))++// Result term 13++        xorq    %r15, %r15+        combadz(%r15,%r14,%r13,40(x),64(y))+        combadd(%r15,%r14,%r13,48(x),56(y))+        combadd(%r15,%r14,%r13,56(x),48(y))+        combadd(%r15,%r14,%r13,64(x),40(y))++// Result term 14++        xorq    %r8, %r8+        combadz(%r8,%r15,%r14,48(x),64(y))+        combadd(%r8,%r15,%r14,56(x),56(y))+        combadd(%r8,%r15,%r14,64(x),48(y))++// Result term 15++        combads(%r8,%r15,56(x),64(y))+        combads(%r8,%r15,64(x),56(y))++// Result term 16++        movq    64(x), %rax+        imulq   64(y), %rax+        addq    %r8, %rax++// Now the upper portion is [%rax;%r15;%r14;%r13;%r12;%r11;%r10;%r9;[%rsp+64]].+// Rotate the upper portion right 9 bits since 2^512 == 2^-9 (mod p_521)+// Let rotated result %rdx,%r15,%r14,...,%r8 be h (high) and %rsp[0..7] be l (low)++        movq    64(%rsp), %r8+        movq    %r8, %rdx+        andq    $0x1FF, %rdx+        shrdq   $9, %r9, %r8+        shrdq   $9, %r10, %r9+        shrdq   $9, %r11, %r10+        shrdq   $9, %r12, %r11+        shrdq   $9, %r13, %r12+        shrdq   $9, %r14, %r13+        shrdq   $9, %r15, %r14+        shrdq   $9, %rax, %r15+        shrq    $9, %rax+        addq    %rax, %rdx++// Force carry-in then add to get s = h + l + 1+// but actually add all 1s in the top 53 bits to get simple carry out++        stc+        adcq    (%rsp), %r8+        adcq    8(%rsp), %r9+        adcq    16(%rsp), %r10+        adcq    24(%rsp), %r11+        adcq    32(%rsp), %r12+        adcq    40(%rsp), %r13+        adcq    48(%rsp), %r14+        adcq    56(%rsp), %r15+        adcq    $~0x1FF, %rdx++// Now CF is set <=> h + l + 1 >= 2^521 <=> h + l >= p_521,+// in which case the lower 521 bits are already right. Otherwise if+// CF is clear, we want to subtract 1. Hence subtract the complement+// of the carry flag then mask the top word, which scrubs the+// padding in either case. Write digits back as they are created.++        cmc+        sbbq    $0, %r8+        movq    %r8, (z)+        sbbq    $0, %r9+        movq    %r9, 8(z)+        sbbq    $0, %r10+        movq    %r10, 16(z)+        sbbq    $0, %r11+        movq    %r11, 24(z)+        sbbq    $0, %r12+        movq    %r12, 32(z)+        sbbq    $0, %r13+        movq    %r13, 40(z)+        sbbq    $0, %r14+        movq    %r14, 48(z)+        sbbq    $0, %r15+        movq    %r15, 56(z)+        sbbq    $0, %rdx+        andq    $0x1FF, %rdx+        movq    %rdx, 64(z)++// Restore registers and return++        CFI_INC_RSP(72)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_mul_p521_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_neg_p256.S view
@@ -0,0 +1,97 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Negate modulo p_256, z := (-x) mod p_256, assuming x reduced+// Input x[4]; output z[4]+//+//    extern void bignum_neg_p256(uint64_t z[static 4], const uint64_t x[static 4]);+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_neg_p256)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_neg_p256)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_neg_p256)+        .text++#define z %rdi+#define x %rsi++#define q %rdx++#define d0 %rax+#define d1 %rcx+#define d2 %r8+#define d3 %r9++#define n1 %r10+#define n3 %r11++#define d0short %eax+#define n1short %r10d++S2N_BN_SYMBOL(bignum_neg_p256):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Load the input digits as [d3;d2;d1;d0] and also set a bitmask q+// for the input being nonzero, so that we avoid doing -0 = p_256+// and hence maintain strict modular reduction++        movq    (x), d0+        movq    8(x), d1+        movq    d0, n1+        orq     d1, n1+        movq    16(x), d2+        movq    24(x), d3+        movq    d2, n3+        orq     d3, n3+        orq     n1, n3+        negq    n3+        sbbq    q, q++// Load the non-trivial words of p_256 = [n3;0;n1;-1] and mask them with q++        movl    $0x00000000ffffffff, n1short+        movq    $0xffffffff00000001, n3+        andq    q, n1+        andq    q, n3++// Do the subtraction, getting it as [n3;d0;n1;q] to avoid moves++        subq    d0, q+        movl    $0, d0short+        sbbq    d1, n1+        sbbq    d2, d0+        sbbq    d3, n3++// Write back++        movq    q, (z)+        movq    n1, 8(z)+        movq    d0, 16(z)+        movq    n3, 24(z)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_neg_p256)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_sqr_p521.S view
@@ -0,0 +1,302 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Square modulo p_521, z := (x^2) mod p_521, assuming x reduced+// Input x[9]; output z[9]+//+//    extern void bignum_sqr_p521(uint64_t z[static 9], const uint64_t x[static 9]);+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_sqr_p521)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_sqr_p521)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_sqr_p521)+        .text++#define z %rdi+#define x %rsi++// A zero register++#define zero %rbp+#define zeroe %ebp++// mulpadd(high,low,i) adds %rdx * x[i] to a register-pair (high,low)+// maintaining consistent double-carrying with adcx and adox,+// using %rax and %rcx as temporaries.++#define mulpadd(high,low,I)             \+        mulxq   I(x), %rax, %rcx ;        \+        adcxq   %rax, low ;               \+        adoxq   %rcx, high++// mulpade(high,low,i) adds %rdx * x[i] to a register-pair (high,low)+// maintaining consistent double-carrying with adcx and adox,+// using %rax as a temporary, assuming high created from scratch+// and that zero has value zero.++#define mulpade(high,low,I)             \+        mulxq   I(x), %rax, high ;       \+        adcxq   %rax, low ;               \+        adoxq   zero, high++S2N_BN_SYMBOL(bignum_sqr_p521):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Save more registers to play with and make temporary space on stack++        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(64)++// Do a basic 8x8 squaring stashing in %rsp[0..7] but keeping the+// top half in the usual rotating register window %r15,...,%r8. Except+// for the lack of full writeback this is the same as bignum_sqr_8_16.++        xorl    zeroe, zeroe++        movq    (x), %rdx+        mulxq   8(x), %r9, %rax+        movq    %r9, 8(%rsp)+        mulxq   16(x), %r10, %rcx+        adcxq   %rax, %r10+        movq    %r10, 16(%rsp)+        mulxq   24(x), %r11, %rax+        adcxq   %rcx, %r11+        mulxq   32(x), %r12, %rcx+        adcxq   %rax, %r12+        mulxq   40(x), %r13, %rax+        adcxq   %rcx, %r13+        mulxq   48(x), %r14, %rcx+        adcxq   %rax, %r14+        mulxq   56(x), %r15, %r8+        adcxq   %rcx, %r15+        adcxq   zero, %r8++        xorl    zeroe, zeroe+        movq    8(x), %rdx+        mulpadd(%r12,%r11,16)+        movq    %r11, 24(%rsp)+        mulpadd(%r13,%r12,24)+        movq    %r12, 32(%rsp)+        mulpadd(%r14,%r13,32)+        mulpadd(%r15,%r14,40)+        mulpadd(%r8,%r15,48)+        mulpade(%r9,%r8,56)+        movq    32(x), %rdx+        mulpade(%r10,%r9,40)+        adcxq   zero, %r10++        xorl    zeroe, zeroe+        movq    16(x), %rdx+        mulpadd(%r14,%r13,24)+        movq    %r13, 40(%rsp)+        mulpadd(%r15,%r14,32)+        movq    %r14, 48(%rsp)+        mulpadd(%r8,%r15,40)+        mulpadd(%r9,%r8,48)+        mulpadd(%r10,%r9,56)+        movq    48(x), %rdx+        mulpade(%r11,%r10,32)+        mulpade(%r12,%r11,40)+        adcxq   zero, %r12++        xorl    zeroe, zeroe+        movq    24(x), %rdx+        mulpadd(%r8,%r15,32)+        movq    %r15, 56(%rsp)+        mulpadd(%r9,%r8,40)+        mulpadd(%r10,%r9,48)+        mulpadd(%r11,%r10,56)+        movq    56(x), %rdx+        mulpadd(%r12,%r11,32)+        mulpade(%r13,%r12,40)+        mulpade(%r14,%r13,48)+        adcxq   zero, %r14++        xorl    zeroe, zeroe+        movq    (x), %rdx+        mulxq   %rdx, %rax, %rcx+        movq    %rax, (%rsp)+        movq    8(%rsp), %rax+        adcxq   %rax, %rax+        adoxq   %rcx, %rax+        movq    %rax, 8(%rsp)++        movq    16(%rsp), %rax+        movq    8(x), %rdx+        mulxq   %rdx, %rdx, %rcx+        adcxq   %rax, %rax+        adoxq   %rdx, %rax+        movq    %rax, 16(%rsp)+        movq    24(%rsp), %rax+        adcxq   %rax, %rax+        adoxq   %rcx, %rax+        movq    %rax, 24(%rsp)++        movq    32(%rsp), %rax+        movq    16(x), %rdx+        mulxq   %rdx, %rdx, %rcx+        adcxq   %rax, %rax+        adoxq   %rdx, %rax+        movq    %rax, 32(%rsp)+        movq    40(%rsp), %rax+        adcxq   %rax, %rax+        adoxq   %rcx, %rax+        movq    %rax, 40(%rsp)++        movq    48(%rsp), %rax+        movq    24(x), %rdx+        mulxq   %rdx, %rdx, %rcx+        adcxq   %rax, %rax+        adoxq   %rdx, %rax+        movq    %rax, 48(%rsp)+        movq    56(%rsp), %rax+        adcxq   %rax, %rax+        adoxq   %rcx, %rax+        movq    %rax, 56(%rsp)++        movq    32(x), %rdx+        mulxq   %rdx, %rdx, %rcx+        adcxq   %r8, %r8+        adoxq   %rdx, %r8+        adcxq   %r9, %r9+        adoxq   %rcx, %r9++        movq    40(x), %rdx+        mulxq   %rdx, %rdx, %rcx+        adcxq   %r10, %r10+        adoxq   %rdx, %r10+        adcxq   %r11, %r11+        adoxq   %rcx, %r11++        movq    48(x), %rdx+        mulxq   %rdx, %rdx, %rcx+        adcxq   %r12, %r12+        adoxq   %rdx, %r12+        adcxq   %r13, %r13+        adoxq   %rcx, %r13++        movq    56(x), %rdx+        mulxq   %rdx, %rdx, %r15+        adcxq   %r14, %r14+        adoxq   %rdx, %r14+        adcxq   zero, %r15+        adoxq   zero, %r15++// Augment the high part with the contribution from the top little word C.+// If we write the input as 2^512 * C + x then we are otherwise just doing+// x^2, so we need to add to the high part 2^512 * C^2 + (2 * C) * x.+// The initial doubling add of C also clears the CF and OF flags as desired.+// We extend the window now to the 9-element %rbp,%r15,%r14,...,%r8.++        movq    64(x), %rdx+        movq    %rdx, %rbp+        imulq   %rbp, %rbp+        addq    %rdx, %rdx+        mulpadd(%r9,%r8,0)+        mulpadd(%r10,%r9,8)+        mulpadd(%r11,%r10,16)+        mulpadd(%r12,%r11,24)+        mulpadd(%r13,%r12,32)+        mulpadd(%r14,%r13,40)+        mulpadd(%r15,%r14,48)+        mulxq  56(x), %rax, %rcx+        adcxq  %rax, %r15+        adoxq  %rcx, %rbp+        adcq   $0, %rbp++// Rotate the upper portion right 9 bits since 2^512 == 2^-9 (mod p_521)+// Let rotated result %rbp,%r15,%r14,...,%r8 be h (high) and %rsp[0..7] be l (low)++        movq    %r8, %rax+        andq    $0x1FF, %rax+        shrdq   $9, %r9, %r8+        shrdq   $9, %r10, %r9+        shrdq   $9, %r11, %r10+        shrdq   $9, %r12, %r11+        shrdq   $9, %r13, %r12+        shrdq   $9, %r14, %r13+        shrdq   $9, %r15, %r14+        shrdq   $9, %rbp, %r15+        shrq    $9, %rbp+        addq    %rax, %rbp++// Force carry-in then add to get s = h + l + 1+// but actually add all 1s in the top 53 bits to get simple carry out++        stc+        adcq    (%rsp), %r8+        adcq    8(%rsp), %r9+        adcq    16(%rsp), %r10+        adcq    24(%rsp), %r11+        adcq    32(%rsp), %r12+        adcq    40(%rsp), %r13+        adcq    48(%rsp), %r14+        adcq    56(%rsp), %r15+        adcq    $~0x1FF, %rbp++// Now CF is set <=> h + l + 1 >= 2^521 <=> h + l >= p_521,+// in which case the lower 521 bits are already right. Otherwise if+// CF is clear, we want to subtract 1. Hence subtract the complement+// of the carry flag then mask the top word, which scrubs the+// padding in either case. Write digits back as they are created.++        cmc+        sbbq    $0, %r8+        movq    %r8, (z)+        sbbq    $0, %r9+        movq    %r9, 8(z)+        sbbq    $0, %r10+        movq    %r10, 16(z)+        sbbq    $0, %r11+        movq    %r11, 24(z)+        sbbq    $0, %r12+        movq    %r12, 32(z)+        sbbq    $0, %r13+        movq    %r13, 40(z)+        sbbq    $0, %r14+        movq    %r14, 48(z)+        sbbq    $0, %r15+        movq    %r15, 56(z)+        sbbq    $0, %rbp+        andq    $0x1FF, %rbp+        movq    %rbp, 64(z)++// Restore registers and return++        CFI_INC_RSP(64)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_sqr_p521)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_sqr_p521_alt.S view
@@ -0,0 +1,315 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Square modulo p_521, z := (x^2) mod p_521, assuming x reduced+// Input x[9]; output z[9]+//+//    extern void bignum_sqr_p521_alt(uint64_t z[static 9],+//                                    const uint64_t x[static 9]);+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_sqr_p521_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_sqr_p521_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_sqr_p521_alt)+        .text++// Input arguments++#define z %rdi+#define x %rsi++// Macro for the key "multiply and add to (c,h,l)" step++#define combadd(c,h,l,numa,numb)                \+        movq    numa, %rax ;                      \+        mulq     numb;                 \+        addq    %rax, l ;                         \+        adcq    %rdx, h ;                         \+        adcq    $0, c++// Set up initial window (c,h,l) = numa * numb++#define combaddz(c,h,l,numa,numb)               \+        movq    numa, %rax ;                      \+        mulq     numb;                 \+        xorq    c, c ;                           \+        movq    %rax, l ;                         \+        movq    %rdx, h++// Doubling step (c,h,l) = 2 * (c,hh,ll) + (0,h,l)++#define doubladd(c,h,l,hh,ll)                   \+        addq    ll, ll ;                         \+        adcq    hh, hh ;                         \+        adcq    c, c ;                           \+        addq    ll, l ;                          \+        adcq    hh, h ;                          \+        adcq    $0, c++// Square term incorporation (c,h,l) += numba^2++#define combadd1(c,h,l,numa)                    \+        movq    numa, %rax ;                      \+        mulq    %rax;                            \+        addq    %rax, l ;                         \+        adcq    %rdx, h ;                         \+        adcq    $0, c++// A short form where we don't expect a top carry++#define combads(h,l,numa)                       \+        movq    numa, %rax ;                      \+        mulq    %rax;                            \+        addq    %rax, l ;                         \+        adcq    %rdx, h++// A version doubling directly before adding, for single non-square terms++#define combadd2(c,h,l,numa,numb)               \+        movq    numa, %rax ;                      \+        mulq     numb;                 \+        addq    %rax, %rax ;                       \+        adcq    %rdx, %rdx ;                       \+        adcq    $0, c ;                           \+        addq    %rax, l ;                         \+        adcq    %rdx, h ;                         \+        adcq    $0, c++S2N_BN_SYMBOL(bignum_sqr_p521_alt):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Make more registers available and make temporary space on stack++        CFI_PUSH(%rbx)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(72)++// Start doing a conventional columnwise squaring,+// temporarily storing the lower 9 digits on the stack.+// Start with result term 0++        movq    (x), %rax+        mulq    %rax++        movq    %rax, (%rsp)+        movq    %rdx, %r9+        xorq    %r10, %r10++// Result term 1++        xorq    %r11, %r11+        combadd2(%r11,%r10,%r9,(x),8(x))+        movq    %r9, 8(%rsp)++// Result term 2++        xorq    %r12, %r12+        combadd1(%r12,%r11,%r10,8(x))+        combadd2(%r12,%r11,%r10,(x),16(x))+        movq    %r10, 16(%rsp)++// Result term 3++        combaddz(%r13,%rcx,%rbx,(x),24(x))+        combadd(%r13,%rcx,%rbx,8(x),16(x))+        doubladd(%r13,%r12,%r11,%rcx,%rbx)+        movq    %r11, 24(%rsp)++// Result term 4++        combaddz(%r14,%rcx,%rbx,(x),32(x))+        combadd(%r14,%rcx,%rbx,8(x),24(x))+        doubladd(%r14,%r13,%r12,%rcx,%rbx)+        combadd1(%r14,%r13,%r12,16(x))+        movq    %r12, 32(%rsp)++// Result term 5++        combaddz(%r15,%rcx,%rbx,(x),40(x))+        combadd(%r15,%rcx,%rbx,8(x),32(x))+        combadd(%r15,%rcx,%rbx,16(x),24(x))+        doubladd(%r15,%r14,%r13,%rcx,%rbx)+        movq    %r13, 40(%rsp)++// Result term 6++        combaddz(%r8,%rcx,%rbx,(x),48(x))+        combadd(%r8,%rcx,%rbx,8(x),40(x))+        combadd(%r8,%rcx,%rbx,16(x),32(x))+        doubladd(%r8,%r15,%r14,%rcx,%rbx)+        combadd1(%r8,%r15,%r14,24(x))+        movq    %r14, 48(%rsp)++// Result term 7++        combaddz(%r9,%rcx,%rbx,(x),56(x))+        combadd(%r9,%rcx,%rbx,8(x),48(x))+        combadd(%r9,%rcx,%rbx,16(x),40(x))+        combadd(%r9,%rcx,%rbx,24(x),32(x))+        doubladd(%r9,%r8,%r15,%rcx,%rbx)+        movq    %r15, 56(%rsp)++// Result term 8++        combaddz(%r10,%rcx,%rbx,(x),64(x))+        combadd(%r10,%rcx,%rbx,8(x),56(x))+        combadd(%r10,%rcx,%rbx,16(x),48(x))+        combadd(%r10,%rcx,%rbx,24(x),40(x))+        doubladd(%r10,%r9,%r8,%rcx,%rbx)+        combadd1(%r10,%r9,%r8,32(x))+        movq    %r8, 64(%rsp)++// We now stop writing back and keep remaining results in a register window.+// Continue with result term 9++        combaddz(%r11,%rcx,%rbx,8(x),64(x))+        combadd(%r11,%rcx,%rbx,16(x),56(x))+        combadd(%r11,%rcx,%rbx,24(x),48(x))+        combadd(%r11,%rcx,%rbx,32(x),40(x))+        doubladd(%r11,%r10,%r9,%rcx,%rbx)++// Result term 10++        combaddz(%r12,%rcx,%rbx,16(x),64(x))+        combadd(%r12,%rcx,%rbx,24(x),56(x))+        combadd(%r12,%rcx,%rbx,32(x),48(x))+        doubladd(%r12,%r11,%r10,%rcx,%rbx)+        combadd1(%r12,%r11,%r10,40(x))++// Result term 11++        combaddz(%r13,%rcx,%rbx,24(x),64(x))+        combadd(%r13,%rcx,%rbx,32(x),56(x))+        combadd(%r13,%rcx,%rbx,40(x),48(x))+        doubladd(%r13,%r12,%r11,%rcx,%rbx)++// Result term 12++        combaddz(%r14,%rcx,%rbx,32(x),64(x))+        combadd(%r14,%rcx,%rbx,40(x),56(x))+        doubladd(%r14,%r13,%r12,%rcx,%rbx)+        combadd1(%r14,%r13,%r12,48(x))++// Result term 13++        combaddz(%r15,%rcx,%rbx,40(x),64(x))+        combadd(%r15,%rcx,%rbx,48(x),56(x))+        doubladd(%r15,%r14,%r13,%rcx,%rbx);++// Result term 14++        xorq    %r8, %r8+        combadd1(%r8,%r15,%r14,56(x))+        combadd2(%r8,%r15,%r14,48(x),64(x))++// Result term 15++        movq    56(x), %rax+        mulq     64(x)+        addq    %rax, %rax+        adcq    %rdx, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8++// Result term 16++        movq    64(x), %rax+        imulq   %rax, %rax+        addq    %r8, %rax++// Now the upper portion is [%rax;%r15;%r14;%r13;%r12;%r11;%r10;%r9;[%rsp+64]].+// Rotate the upper portion right 9 bits since 2^512 == 2^-9 (mod p_521)+// Let rotated result %rdx,%r15,%r14,...,%r8 be h (high) and %rsp[0..7] be l (low)++        movq    64(%rsp), %r8+        movq    %r8, %rdx+        andq    $0x1FF, %rdx+        shrdq   $9, %r9, %r8+        shrdq   $9, %r10, %r9+        shrdq   $9, %r11, %r10+        shrdq   $9, %r12, %r11+        shrdq   $9, %r13, %r12+        shrdq   $9, %r14, %r13+        shrdq   $9, %r15, %r14+        shrdq   $9, %rax, %r15+        shrq    $9, %rax+        addq    %rax, %rdx++// Force carry-in then add to get s = h + l + 1+// but actually add all 1s in the top 53 bits to get simple carry out++        stc+        adcq    (%rsp), %r8+        adcq    8(%rsp), %r9+        adcq    16(%rsp), %r10+        adcq    24(%rsp), %r11+        adcq    32(%rsp), %r12+        adcq    40(%rsp), %r13+        adcq    48(%rsp), %r14+        adcq    56(%rsp), %r15+        adcq    $~0x1FF, %rdx++// Now CF is set <=> h + l + 1 >= 2^521 <=> h + l >= p_521,+// in which case the lower 521 bits are already right. Otherwise if+// CF is clear, we want to subtract 1. Hence subtract the complement+// of the carry flag then mask the top word, which scrubs the+// padding in either case. Write digits back as they are created.++        cmc+        sbbq    $0, %r8+        movq    %r8, (z)+        sbbq    $0, %r9+        movq    %r9, 8(z)+        sbbq    $0, %r10+        movq    %r10, 16(z)+        sbbq    $0, %r11+        movq    %r11, 24(z)+        sbbq    $0, %r12+        movq    %r12, 32(z)+        sbbq    $0, %r13+        movq    %r13, 40(z)+        sbbq    $0, %r14+        movq    %r14, 48(z)+        sbbq    $0, %r15+        movq    %r15, 56(z)+        sbbq    $0, %rdx+        andq    $0x1FF, %rdx+        movq    %rdx, 64(z)++// Restore registers and return++        CFI_INC_RSP(72)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbx)+#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_sqr_p521_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_tomont_p256.S view
@@ -0,0 +1,195 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Convert to Montgomery form z := (2^256 * x) mod p_256+// Input x[4]; output z[4]+//+//    extern void bignum_tomont_p256(uint64_t z[static 4],+//                                   const uint64_t x[static 4]);+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_tomont_p256)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_tomont_p256)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_tomont_p256)+        .text++#define z %rdi+#define x %rsi++// Some temp registers for the last correction stage++#define d %rax+#define u %rdx+#define v %rcx++#define dshort %eax+#define ushort %edx++// Add %rdx * m into a register-pair (high,low)+// maintaining consistent double-carrying with adcx and adox,+// using %rax and %rbx as temporaries++#define mulpadd(high,low,m)             \+        mulxq   m, %rax, %rcx ;            \+        adcxq   %rax, low ;               \+        adoxq   %rcx, high++S2N_BN_SYMBOL(bignum_tomont_p256):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// We are essentially just doing a Montgomery multiplication of x and the+// precomputed constant y = 2^512 mod p, so the code is almost the same+// modulo a few registers and the change from loading y[i] to using constants.+// Because there is no y pointer to keep, we use one register less.++        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++// Do row 0 computation, which is a bit different:+// set up initial window [%r12,%r11,%r10,%r9,%r8] = y[0] * x+// Unlike later, we only need a single carry chain++        xorq    %r13, %r13+        movl    $0x0000000000000003, %edx+        mulxq   (x), %r8, %r9+        mulxq   8(x), %rcx, %r10+        adcxq   %rcx, %r9+        mulxq   16(x), %rcx, %r11+        adcxq   %rcx, %r10+        mulxq   24(x), %rcx, %r12+        adcxq   %rcx, %r11+        adcxq   %r13, %r12++// Add row 1++        movq    $0xfffffffbffffffff, %rdx+        xorq    %r14, %r14+        mulpadd(%r10,%r9,(x))+        mulpadd(%r11,%r10,8(x))+        mulpadd(%r12,%r11,16(x))+        mulpadd(%r13,%r12,24(x))+        adcq   %r14, %r13++// Montgomery reduce windows 0 and 1 together++        xorq    %r15, %r15+        movq    $0x0000000100000000, %rdx+        mulpadd(%r10,%r9,%r8)+        mulpadd(%r11,%r10,%r9)+        movq    $0xffffffff00000001, %rdx+        mulpadd(%r12,%r11,%r8)+        mulpadd(%r13,%r12,%r9)+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcxq   %r15, %r14++// Add row 2++        movq    $0xfffffffffffffffe, %rdx+        xorq    %r8, %r8+        mulpadd(%r11,%r10,(x))+        mulpadd(%r12,%r11,8(x))+        mulpadd(%r13,%r12,16(x))+        mulpadd(%r14,%r13,24(x))+        adcxq   %r8, %r14+        adoxq   %r8, %r15+        adcxq   %r8, %r15++// Add row 3++        movq    $0x00000004fffffffd, %rdx+        xorq    %r9, %r9+        mulpadd(%r12,%r11,(x))+        mulpadd(%r13,%r12,8(x))+        mulpadd(%r14,%r13,16(x))+        mulpadd(%r15,%r14,24(x))+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcxq   %r9, %r8++// Montgomery reduce windows 2 and 3 together++        xorq    %r9, %r9+        movq    $0x0000000100000000, %rdx+        mulpadd(%r12,%r11,%r10)+        mulpadd(%r13,%r12,%r11)+        movq    $0xffffffff00000001, %rdx+        mulpadd(%r14,%r13,%r10)+        mulpadd(%r15,%r14,%r11)+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcxq   %r9, %r8++// We now have a pre-reduced 5-word form [%r8; %r15;%r14;%r13;%r12]+// Load non-trivial digits of p_256 = [v; 0; u; -1]++        movl    $0x00000000ffffffff, ushort+        movq    $0xffffffff00000001, v++// Now do the subtraction (0,p_256-1) - (%r8,%r15,%r14,%r13,%r12) to get the carry++        movq    $-2, d+        subq    %r12, d+        movq    u, d+        sbbq    %r13, d+        movl    $0, dshort+        sbbq    %r14, d+        movq    v, d+        sbbq    %r15, d++// This last comparison in the chain will actually even set the mask+// for us, so we don't need to separately create it from the carry.+// This means p_256 - 1 < (c,d1,d0,d5,d4), i.e. we are so far >= p_256++        movl    $0, dshort+        sbbq    %r8, d+        andq    d, u+        andq    d, v++// Do a masked subtraction of p_256 and write back++        subq    d, %r12+        sbbq    u, %r13+        sbbq    $0, %r14+        sbbq    v, %r15++        movq    %r12, (z)+        movq    %r13, 8(z)+        movq    %r14, 16(z)+        movq    %r15, 24(z)++// Restore registers and return++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_tomont_p256)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_tomont_p256_alt.S view
@@ -0,0 +1,203 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Convert to Montgomery form z := (2^256 * x) mod p_256+// Input x[4]; output z[4]+//+//    extern void bignum_tomont_p256_alt(uint64_t z[static 4],+//                                       const uint64_t x[static 4]);+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_tomont_p256_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_tomont_p256_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_tomont_p256_alt)+        .text++#define z %rdi+#define x %rsi++// Add %rcx * m into a register-pair (high,low) maintaining consistent+// carry-catching with carry (negated, as bitmask) and using %rax and %rdx+// as temporaries++#define mulpadd(carry,high,low,m)       \+        movq    m, %rax ;                 \+        mulq    %rcx;                    \+        subq    carry, %rdx ;             \+        addq    %rax, low ;               \+        adcq    %rdx, high ;              \+        sbbq    carry, carry++// Initial version assuming no carry-in++#define mulpadi(carry,high,low,m)       \+        movq    m, %rax ;                 \+        mulq    %rcx;                    \+        addq    %rax, low ;               \+        adcq    %rdx, high ;              \+        sbbq    carry, carry++// End version not catching the top carry-out++#define mulpade(carry,high,low,m)       \+        movq    m, %rax ;                 \+        mulq    %rcx;                    \+        subq    carry, %rdx ;             \+        addq    %rax, low ;               \+        adcq    %rdx, high++S2N_BN_SYMBOL(bignum_tomont_p256_alt):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Save more registers to play with++        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++// Do row 0 computation, which is a bit different:+// set up initial window [%r12,%r11,%r10,%r9,%r8] = y[0] * x+// Unlike later, we only need a single carry chain++        movl    $0x0000000000000003, %ecx+        movq    (x), %rax+        mulq    %rcx+        movq    %rax, %r8+        movq    %rdx, %r9++        movq    8(x), %rax+        mulq    %rcx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10++        movq    16(x), %rax+        mulq    %rcx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11++        movq    24(x), %rax+        mulq    %rcx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12++// Add row 1++        movq    $0xfffffffbffffffff, %rcx+        xorl    %r13d, %r13d+        mulpadi(%r14,%r10,%r9,(x))+        mulpadd(%r14,%r11,%r10,8(x))+        mulpadd(%r14,%r12,%r11,16(x))+        mulpade(%r14,%r13,%r12,24(x))++// Montgomery reduce windows 0 and 1 together++        xorl    %r14d, %r14d+        movq    $0x0000000100000000, %rcx+        mulpadi(%r15,%r10,%r9,%r8)+        mulpadd(%r15,%r11,%r10,%r9)+        notq    %rcx+        leaq    2(%rcx), %rcx+        mulpadd(%r15,%r12,%r11,%r8)+        mulpade(%r15,%r13,%r12,%r9)+        adcq    %r14, %r14++// Add row 2++        movq    $0xfffffffffffffffe, %rcx+        xorl    %r15d, %r15d+        mulpadi(%r8,%r11,%r10,(x))+        mulpadd(%r8,%r12,%r11,8(x))+        mulpadd(%r8,%r13,%r12,16(x))+        mulpade(%r8,%r14,%r13,24(x))+        adcq    %r15, %r15++// Add row 3++        movq    $0x00000004fffffffd, %rcx+        xorl    %r8d, %r8d+        mulpadi(%r9,%r12,%r11,(x))+        mulpadd(%r9,%r13,%r12,8(x))+        mulpadd(%r9,%r14,%r13,16(x))+        mulpade(%r9,%r15,%r14,24(x))+        adcq    %r8, %r8++// Montgomery reduce windows 2 and 3 together++        movq    $0x0000000100000000, %rcx+        mulpadi(%r9,%r12,%r11,%r10)+        mulpadd(%r9,%r13,%r12,%r11)+        notq    %rcx+        leaq    2(%rcx), %rcx+        mulpadd(%r9,%r14,%r13,%r10)+        mulpadd(%r9,%r15,%r14,%r11)+        subq    %r9, %r8++// We now have a pre-reduced 5-word form [%r8; %r15;%r14;%r13;%r12]+// Load [%rax;%r11;%r9;%rcx;%rdx] = 2^320 - p_256, re-using earlier numbers a bit+// Do [%rax;%r11;%r9;%rcx;%rdx] = [%r8;%r15;%r14;%r13;%r12] + (2^320 - p_256)++        xorl    %edx, %edx+        leaq    -1(%rdx), %r9+        incq    %rdx+        addq    %r12, %rdx+        decq    %rcx+        adcq    %r13, %rcx+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0x00000000fffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax++// Now carry is set if r + (2^320 - p_256) >= 2^320, i.e. r >= p_256+// where r is the pre-reduced form. So conditionally select the+// output accordingly.++        cmovcq  %rdx, %r12+        cmovcq  %rcx, %r13+        cmovcq  %r9, %r14+        cmovcq  %r11, %r15++// Write back reduced value++        movq    %r12, (z)+        movq    %r13, 8(z)+        movq    %r14, 16(z)+        movq    %r15, 24(z)++// Restore registers and return++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_tomont_p256_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_tomont_p384.S view
@@ -0,0 +1,295 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Convert to Montgomery form z := (2^384 * x) mod p_384+// Input x[6]; output z[6]+//+//    extern void bignum_tomont_p384(uint64_t z[static 6],+//                                   const uint64_t x[static 6]);+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_tomont_p384)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_tomont_p384)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_tomont_p384)+        .text++#define z %rdi+#define x %rsi++// Fairly consistently used as a zero register++#define zero %rbp++// Some temp registers for the last correction stage++#define d %rax+#define u %rdx+#define v %rcx+#define w %rsi++#define vshort %ecx+#define wshort %esi++// Add %rdx * m into a register-pair (high,low)+// maintaining consistent double-carrying with adcx and adox,+// using %rax and %rcx as temporaries++#define mulpadd(high,low,m)             \+        mulxq   m, %rax, %rcx ;            \+        adcxq   %rax, low ;               \+        adoxq   %rcx, high++// Core one-step Montgomery reduction macro. Takes input in+// [d7;d6;d5;d4;d3;d2;d1;d0] and returns result in [d7;d6;d5;d4;d3;d2;d1],+// adding to the existing contents, re-using d0 as a temporary internally+//+// We want to add (2^384 - 2^128 - 2^96 + 2^32 - 1) * w+// where w = [d0 + (d0<<32)] mod 2^64+//+//       montredc(d7,d6,d5,d4,d3,d2,d1,d0)+//+// This particular variant, with its mix of addition and subtraction+// at the top, is not intended to maintain a coherent carry or borrow out.+// It is assumed the final result would fit in [d7;d6;d5;d4;d3;d2;d1].+// which is always the case here as the top word is even always in {0,1}++#define montredc(d7,d6,d5,d4,d3,d2,d1,d0)                               \+/* Our correction multiplier is w = [d0 + (d0<<32)] mod 2^64 */         \+        movq    d0, %rdx ;                                        \+        shlq    $32, %rdx ;                                        \+        addq    d0, %rdx ;                                        \+/* Construct [%rbp;%rcx;%rax;-] = (2^384 - p_384) * w */                   \+/* We know the lowest word will cancel so we can re-use d0 as a temp */ \+        xorl    %ebp, %ebp ;                                       \+        movq    $0xffffffff00000001, %rax ;                        \+        mulxq   %rax, %rcx, %rax ;                                  \+        movl    $0x00000000ffffffff, %ecx ;                        \+        mulxq   %rcx, d0, %rcx ;                                   \+        adcq    d0, %rax ;                                        \+        adcq    %rdx, %rcx ;                                       \+        adcl    %ebp, %ebp ;                                       \+/*  Now subtract that and add 2^384 * w */                              \+        subq    %rax, d1 ;                                        \+        sbbq    %rcx, d2 ;                                        \+        sbbq    %rbp, d3 ;                                        \+        sbbq    $0, d4 ;                                          \+        sbbq    $0, d5 ;                                          \+        sbbq    $0, %rdx ;                                         \+        addq    %rdx, d6 ;                                        \+        adcq    $0, d7++S2N_BN_SYMBOL(bignum_tomont_p384):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// We are essentially just doing a Montgomery multiplication of x and the+// precomputed constant y = 2^768 mod p, so the code is almost the same+// modulo a few registers and the change from loading y[i] to using constants,+// plus the easy digits y[4] = 1 and y[5] = 0 being treated specially.+// Because there is no y pointer to keep, we use one register less.++        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++// Do row 0 computation, which is a bit different:+// set up initial window [%r14,%r13,%r12,%r11,%r10,%r9,%r8] = y[0] * x+// Unlike later, we only need a single carry chain++        movq    $0xfffffffe00000001, %rdx+        mulxq   (x), %r8, %r9+        mulxq   8(x), %rcx, %r10+        addq    %rcx, %r9+        mulxq   16(x), %rcx, %r11+        adcq    %rcx, %r10+        mulxq   24(x), %rcx, %r12+        adcq    %rcx, %r11+        mulxq   32(x), %rcx, %r13+        adcq    %rcx, %r12+        mulxq   40(x), %rcx, %r14+        adcq    %rcx, %r13+        adcq    $0, %r14++// Montgomery reduce the zeroth window++        xorq    %r15, %r15+        montredc(%r15, %r14,%r13,%r12,%r11,%r10,%r9,%r8)++// Add row 1++        xorq    zero, zero+        movq    $0x0000000200000000, %rdx+        xorq    %r8, %r8+        mulpadd(%r10,%r9,(x))+        mulpadd(%r11,%r10,8(x))+        mulpadd(%r12,%r11,16(x))+        mulpadd(%r13,%r12,24(x))+        mulpadd(%r14,%r13,32(x))+        mulpadd(%r15,%r14,40(x))+        adcxq   zero, %r15+        adoxq   zero, %r8+        adcxq   zero, %r8++// Montgomery reduce window 1++        montredc(%r8, %r15,%r14,%r13,%r12,%r11,%r10,%r9)++// Add row 2++        xorq    zero, zero+        movq    $0xfffffffe00000000, %rdx+        xorq    %r9, %r9+        mulpadd(%r11,%r10,(x))+        mulpadd(%r12,%r11,8(x))+        mulpadd(%r13,%r12,16(x))+        mulpadd(%r14,%r13,24(x))+        mulpadd(%r15,%r14,32(x))+        mulpadd(%r8,%r15,40(x))+        adcxq   zero, %r8+        adoxq   zero, %r9+        adcxq   zero, %r9++// Montgomery reduce window 2++        montredc(%r9, %r8,%r15,%r14,%r13,%r12,%r11,%r10)++// Add row 3++        xorq    zero, zero+        movq    $0x0000000200000000, %rdx+        xorq    %r10, %r10+        mulpadd(%r12,%r11,(x))+        mulpadd(%r13,%r12,8(x))+        mulpadd(%r14,%r13,16(x))+        mulpadd(%r15,%r14,24(x))+        mulpadd(%r8,%r15,32(x))+        mulpadd(%r9,%r8,40(x))+        adcxq   zero, %r9+        adoxq   zero, %r10+        adcxq   zero, %r10++// Montgomery reduce window 3++        montredc(%r10, %r9,%r8,%r15,%r14,%r13,%r12,%r11)++// Add row 4. The multiplier y[4] = 1, so we just add x to the window+// while extending it with one more digit, initially this carry++        xorq    %r11, %r11+        addq    (x), %r12+        adcq    8(x), %r13+        adcq    16(x), %r14+        adcq    24(x), %r15+        adcq    32(x), %r8+        adcq    40(x), %r9+        adcq    $0, %r10+        adcq    $0, %r11++// Montgomery reduce window 4++        montredc(%r11, %r10,%r9,%r8,%r15,%r14,%r13,%r12)++// Add row 5, The multiplier y[5] = 0, so this is trivial: all we do is+// bring down another zero digit into the window.++        xorq    %r12, %r12++// Montgomery reduce window 5++        montredc(%r12, %r11,%r10,%r9,%r8,%r15,%r14,%r13)++// We now have a pre-reduced 7-word form [%r12;%r11;%r10;%r9;%r8;%r15;%r14]++// We know, writing B = 2^{6*64} that the full implicit result is+// B^2 c <= z + (B - 1) * p < B * p + (B - 1) * p < 2 * B * p,+// so the top half is certainly < 2 * p. If c = 1 already, we know+// subtracting p will give the reduced modulus. But now we do a+// comparison to catch cases where the residue is >= p.+// First set [0;0;0;w;v;u] = 2^384 - p_384++        movq    $0xffffffff00000001, u+        movl    $0x00000000ffffffff, vshort+        movl    $0x0000000000000001, wshort++// Let dd = [%r11;%r10;%r9;%r8;%r15;%r14] be the topless 6-word intermediate result.+// Set CF if the addition dd + (2^384 - p_384) >= 2^384, hence iff dd >= p_384.++        movq    %r14, d+        addq    u, d+        movq    %r15, d+        adcq    v, d+        movq    %r8, d+        adcq    w, d+        movq    %r9, d+        adcq    $0, d+        movq    %r10, d+        adcq    $0, d+        movq    %r11, d+        adcq    $0, d++// Now just add this new carry into the existing %r12. It's easy to see they+// can't both be 1 by our range assumptions, so this gives us a {0,1} flag++        adcq    $0, %r12++// Now convert it into a bitmask++        negq    %r12++// Masked addition of 2^384 - p_384, hence subtraction of p_384++        andq    %r12, u+        andq    %r12, v+        andq    %r12, w++        addq   u, %r14+        adcq   v, %r15+        adcq   w, %r8+        adcq   $0, %r9+        adcq   $0, %r10+        adcq   $0, %r11++// Write back the result++        movq    %r14, (z)+        movq    %r15, 8(z)+        movq    %r8, 16(z)+        movq    %r9, 24(z)+        movq    %r10, 32(z)+        movq    %r11, 40(z)++// Restore registers and return++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_tomont_p384)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/bignum_tomont_p384_alt.S view
@@ -0,0 +1,324 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Convert to Montgomery form z := (2^384 * x) mod p_384+// Input x[6]; output z[6]+//+//    extern void bignum_tomont_p384_alt(uint64_t z[static 6],+//                                       const uint64_t x[static 6]);+//+// Standard x86-64 ABI: RDI = z, RSI = x+// Microsoft x64 ABI:   RCX = z, RDX = x+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(bignum_tomont_p384_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(bignum_tomont_p384_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(bignum_tomont_p384_alt)+        .text++#define z %rdi+#define x %rsi++// Some temp registers for the last correction stage++#define d %rax+#define u %rdx+#define v %rcx+#define w %rsi++#define vshort %ecx+#define wshort %esi++// Add %rbx * m into a register-pair (high,low) maintaining consistent+// carry-catching with carry (negated, as bitmask) and using %rax and %rdx+// as temporaries++#define mulpadd(carry,high,low,m)       \+        movq    m, %rax ;                 \+        mulq    %rbx;                    \+        subq    carry, %rdx ;             \+        addq    %rax, low ;               \+        adcq    %rdx, high ;              \+        sbbq    carry, carry++// Initial version assuming no carry-in++#define mulpadi(carry,high,low,m)       \+        movq    m, %rax ;                 \+        mulq    %rbx;                    \+        addq    %rax, low ;               \+        adcq    %rdx, high ;              \+        sbbq    carry, carry++// End version not catching the top carry-out++#define mulpade(carry,high,low,m)       \+        movq    m, %rax ;                 \+        mulq    %rbx;                    \+        subq    carry, %rdx ;             \+        addq    %rax, low ;               \+        adcq    %rdx, high++// Core one-step Montgomery reduction macro. Takes input in+// [d7;d6;d5;d4;d3;d2;d1;d0] and returns result in [d7;d6;d5;d4;d3;d2;d1],+// adding to the existing contents, re-using d0 as a temporary internally+//+// We want to add (2^384 - 2^128 - 2^96 + 2^32 - 1) * w+// where w = [d0 + (d0<<32)] mod 2^64+//+//       montredc(d7,d6,d5,d4,d3,d2,d1,d0)+//+// This particular variant, with its mix of addition and subtraction+// at the top, is not intended to maintain a coherent carry or borrow out.+// It is assumed the final result would fit in [d7;d6;d5;d4;d3;d2;d1].+// which is always the case here as the top word is even always in {0,1}++#define montredc(d7,d6,d5,d4,d3,d2,d1,d0)                               \+/* Our correction multiplier is w = [d0 + (d0<<32)] mod 2^64 */         \+        movq    d0, %rbx ;                                        \+        shlq    $32, %rbx ;                                        \+        addq    d0, %rbx ;                                        \+/* Construct [%rcx;%rdx;%rax;-] = (2^384 - p_384) * w */                   \+/* We know the lowest word will cancel so we can re-use d0 as a temp */ \+        xorl    %ecx, %ecx ;                                       \+        movq    $0xffffffff00000001, %rax ;                        \+        mulq    %rbx;                                            \+        movq    %rdx, d0 ;                                        \+        movq    $0x00000000ffffffff, %rax ;                        \+        mulq    %rbx;                                            \+        addq    d0, %rax ;                                        \+        adcq    %rbx, %rdx ;                                       \+        adcl    %ecx, %ecx ;                                       \+/*  Now subtract that and add 2^384 * w */                              \+        subq    %rax, d1 ;                                        \+        sbbq    %rdx, d2 ;                                        \+        sbbq    %rcx, d3 ;                                        \+        sbbq    $0, d4 ;                                          \+        sbbq    $0, d5 ;                                          \+        sbbq    $0, %rbx ;                                         \+        addq    %rbx, d6 ;                                        \+        adcq    $0, d7++S2N_BN_SYMBOL(bignum_tomont_p384_alt):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// We are essentially just doing a Montgomery multiplication of x and the+// precomputed constant y = 2^768 mod p, so the code is almost the same+// modulo a few registers and the change from loading y[i] to using constants,+// plus the easy digits y[4] = 1 and y[5] = 0 being treated specially.+// Because there is no y pointer to keep, we use one register less.++        CFI_PUSH(%rbx)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++// Do row 0 computation, which is a bit different:+// set up initial window [%r14,%r13,%r12,%r11,%r10,%r9,%r8] = y[0] * x+// Unlike later, we only need a single carry chain++        movq    $0xfffffffe00000001, %rbx+        movq    (x), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9++        movq    8(x), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10++        movq    16(x), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11++        movq    24(x), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12++        movq    32(x), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13++        movq    40(x), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14++        xorl    %r15d, %r15d++// Montgomery reduce the zeroth window++        montredc(%r15, %r14,%r13,%r12,%r11,%r10,%r9,%r8)++// Add row 1++        movq    $0x0000000200000000, %rbx+        mulpadi(%r8,%r10,%r9,(x))+        mulpadd(%r8,%r11,%r10,8(x))+        mulpadd(%r8,%r12,%r11,16(x))+        mulpadd(%r8,%r13,%r12,24(x))+        mulpadd(%r8,%r14,%r13,32(x))+        mulpadd(%r8,%r15,%r14,40(x))+        negq    %r8++// Montgomery reduce window 1++        montredc(%r8, %r15,%r14,%r13,%r12,%r11,%r10,%r9)++// Add row 2++        movq    $0xfffffffe00000000, %rbx+        mulpadi(%r9,%r11,%r10,(x))+        mulpadd(%r9,%r12,%r11,8(x))+        mulpadd(%r9,%r13,%r12,16(x))+        mulpadd(%r9,%r14,%r13,24(x))+        mulpadd(%r9,%r15,%r14,32(x))+        mulpadd(%r9,%r8,%r15,40(x))+        negq    %r9++// Montgomery reduce window 2++        montredc(%r9, %r8,%r15,%r14,%r13,%r12,%r11,%r10)++// Add row 3++        movq    $0x0000000200000000, %rbx+        mulpadi(%r10,%r12,%r11,(x))+        mulpadd(%r10,%r13,%r12,8(x))+        mulpadd(%r10,%r14,%r13,16(x))+        mulpadd(%r10,%r15,%r14,24(x))+        mulpadd(%r10,%r8,%r15,32(x))+        mulpadd(%r10,%r9,%r8,40(x))+        negq    %r10++// Montgomery reduce window 3++        montredc(%r10, %r9,%r8,%r15,%r14,%r13,%r12,%r11)++// Add row 4. The multiplier y[4] = 1, so we just add x to the window+// while extending it with one more digit, initially this carry++        xorq    %r11, %r11+        addq    (x), %r12+        adcq    8(x), %r13+        adcq    16(x), %r14+        adcq    24(x), %r15+        adcq    32(x), %r8+        adcq    40(x), %r9+        adcq    %r11, %r10+        adcq    %r11, %r11++// Montgomery reduce window 4++        montredc(%r11, %r10,%r9,%r8,%r15,%r14,%r13,%r12)++// Add row 5, The multiplier y[5] = 0, so this is trivial: all we do is+// bring down another zero digit into the window.++        xorq    %r12, %r12++// Montgomery reduce window 5++        montredc(%r12, %r11,%r10,%r9,%r8,%r15,%r14,%r13)++// We now have a pre-reduced 7-word form [%r12;%r11;%r10;%r9;%r8;%r15;%r14]++// We know, writing B = 2^{6*64} that the full implicit result is+// B^2 c <= z + (B - 1) * p < B * p + (B - 1) * p < 2 * B * p,+// so the top half is certainly < 2 * p. If c = 1 already, we know+// subtracting p will give the reduced modulus. But now we do a+// comparison to catch cases where the residue is >= p.+// First set [0;0;0;w;v;u] = 2^384 - p_384++        movq    $0xffffffff00000001, u+        movl    $0x00000000ffffffff, vshort+        movl    $0x0000000000000001, wshort++// Let dd = [%r11;%r10;%r9;%r8;%r15;%r14] be the topless 6-word intermediate result.+// Set CF if the addition dd + (2^384 - p_384) >= 2^384, hence iff dd >= p_384.++        movq    %r14, d+        addq    u, d+        movq    %r15, d+        adcq    v, d+        movq    %r8, d+        adcq    w, d+        movq    %r9, d+        adcq    $0, d+        movq    %r10, d+        adcq    $0, d+        movq    %r11, d+        adcq    $0, d++// Now just add this new carry into the existing %r12. It's easy to see they+// can't both be 1 by our range assumptions, so this gives us a {0,1} flag++        adcq    $0, %r12++// Now convert it into a bitmask++        negq    %r12++// Masked addition of 2^384 - p_384, hence subtraction of p_384++        andq    %r12, u+        andq    %r12, v+        andq    %r12, w++        addq   u, %r14+        adcq   v, %r15+        adcq   w, %r8+        adcq   $0, %r9+        adcq   $0, %r10+        adcq   $0, %r11++// Write back the result++        movq    %r14, (z)+        movq    %r15, 8(z)+        movq    %r8, 16(z)+        movq    %r9, 24(z)+        movq    %r10, 32(z)+        movq    %r11, 40(z)++// Restore registers and return++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(bignum_tomont_p384_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/curve25519_x25519.S view
@@ -0,0 +1,2189 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// The x25519 function for curve25519+// Inputs scalar[4], point[4]; output res[4]+//+// extern void curve25519_x25519+//   (uint64_t res[static 4],const uint64_t scalar[static 4],+//    const uint64_t point[static 4]);+//+// The function has a second prototype considering the arguments as arrays+// of bytes rather than 64-bit words. The underlying code is the same, since+// the x86 platform is little-endian.+//+// extern void curve25519_x25519_byte+//   (uint8_t res[static 32],const uint8_t scalar[static 32],+//    const uint8_t point[static 32]);+//+// Given a scalar n and the X coordinate of an input point P = (X,Y) on+// curve25519 (Y can live in any extension field of characteristic 2^255-19),+// this returns the X coordinate of n * P = (X, Y), or 0 when n * P is the+// point at infinity. Both n and X inputs are first slightly modified/mangled+// as specified in the relevant RFC (https://www.rfc-editor.org/rfc/rfc7748);+// in particular the lower three bits of n are set to zero. Does not implement+// the zero-check specified in Section 6.1.+//+// Standard x86-64 ABI: RDI = res, RSI = scalar, RDX = point+// Microsoft x64 ABI:   RCX = res, RDX = scalar, R8 = point+// ----------------------------------------------------------------------------+#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(curve25519_x25519)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(curve25519_x25519)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(curve25519_x25519)+        S2N_BN_SYM_VISIBILITY_DIRECTIVE(curve25519_x25519_byte)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(curve25519_x25519_byte)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(curve25519_x25519_byte)+        .text+        .balign 32++// Size of individual field elements++#define NUMSIZE 32++// Stable homes for the input result argument during the whole body+// and other variables that are only needed prior to the modular inverse.++#define res  12*NUMSIZE(%rsp)+#define i  12*NUMSIZE+8(%rsp)+#define swap  12*NUMSIZE+16(%rsp)++// Pointers to result x coord to be written, assuming the base "res"+// has been loaded into %rbp++#define resx 0(%rbp)++// Pointer-offset pairs for temporaries on stack with some aliasing.+// Both dmsn and dnsm need space for >= 5 digits, and we allocate 8++#define scalar (0*NUMSIZE)(%rsp)++#define pointx (1*NUMSIZE)(%rsp)++#define dm (2*NUMSIZE)(%rsp)++#define zm (3*NUMSIZE)(%rsp)+#define sm (3*NUMSIZE)(%rsp)+#define dpro (3*NUMSIZE)(%rsp)++#define sn (4*NUMSIZE)(%rsp)++#define dn (5*NUMSIZE)(%rsp)+#define e (5*NUMSIZE)(%rsp)++#define dmsn (6*NUMSIZE)(%rsp)+#define p (6*NUMSIZE)(%rsp)+#define zn (7*NUMSIZE)(%rsp)++#define xm (8*NUMSIZE)(%rsp)+#define dnsm (8*NUMSIZE)(%rsp)+#define spro (8*NUMSIZE)(%rsp)++#define xn (10*NUMSIZE)(%rsp)+#define s (10*NUMSIZE)(%rsp)++#define d (11*NUMSIZE)(%rsp)++// Total size to reserve on the stack+// This includes space for the 3 other variables above+// and rounds up to a multiple of 32++#define NSPACE 13*NUMSIZE++// Macro wrapping up the basic field operation bignum_mul_p25519, only+// trivially different from a pure function call to that subroutine.++#define mul_p25519(P0,P1,P2)                    \+        xorl   %edi, %edi ;                        \+        movq   P2, %rdx ;                       \+        mulxq  P1, %r8, %r9 ;                    \+        mulxq  0x8+P1, %rax, %r10 ;              \+        addq   %rax, %r9 ;                         \+        mulxq  0x10+P1, %rax, %r11 ;             \+        adcq   %rax, %r10 ;                        \+        mulxq  0x18+P1, %rax, %r12 ;             \+        adcq   %rax, %r11 ;                        \+        adcq   %rdi, %r12 ;                        \+        xorl   %edi, %edi ;                        \+        movq   0x8+P2, %rdx ;                   \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rbx, %r10 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x18+P1, %rax, %r13 ;             \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rdi, %r13 ;                        \+        adcxq  %rdi, %r13 ;                        \+        xorl   %edi, %edi ;                        \+        movq   0x10+P2, %rdx ;                  \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rbx, %r13 ;                        \+        mulxq  0x18+P1, %rax, %r14 ;             \+        adcxq  %rax, %r13 ;                        \+        adoxq  %rdi, %r14 ;                        \+        adcxq  %rdi, %r14 ;                        \+        xorl   %edi, %edi ;                        \+        movq   0x18+P2, %rdx ;                  \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rbx, %r13 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r13 ;                        \+        adoxq  %rbx, %r14 ;                        \+        mulxq  0x18+P1, %rax, %r15 ;             \+        adcxq  %rax, %r14 ;                        \+        adoxq  %rdi, %r15 ;                        \+        adcxq  %rdi, %r15 ;                        \+        movl   $0x26, %edx ;                       \+        xorl   %edi, %edi ;                        \+        mulxq  %r12, %rax, %rbx ;                   \+        adcxq  %rax, %r8 ;                         \+        adoxq  %rbx, %r9 ;                         \+        mulxq  %r13, %rax, %rbx ;                   \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rbx, %r10 ;                        \+        mulxq  %r14, %rax, %rbx ;                   \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  %r15, %rax, %r12 ;                   \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rdi, %r12 ;                        \+        adcxq  %rdi, %r12 ;                        \+        shldq  $0x1, %r11, %r12 ;                   \+        movl   $0x13, %edx ;                       \+        incq   %r12;                             \+        bts    $63, %r11 ;                         \+        mulxq  %r12, %rax, %rbx ;                   \+        addq   %rax, %r8 ;                         \+        adcq   %rbx, %r9 ;                         \+        adcq   %rdi, %r10 ;                        \+        adcq   %rdi, %r11 ;                        \+        sbbq   %rax, %rax ;                        \+        notq   %rax;                             \+        andq   %rdx, %rax ;                        \+        subq   %rax, %r8 ;                         \+        sbbq   %rdi, %r9 ;                         \+        sbbq   %rdi, %r10 ;                        \+        sbbq   %rdi, %r11 ;                        \+        btr    $63, %r11 ;                         \+        movq   %r8, P0 ;                        \+        movq   %r9, 0x8+P0 ;                    \+        movq   %r10, 0x10+P0 ;                  \+        movq   %r11, 0x18+P0++// A version of multiplication that only guarantees output < 2 * p_25519.+// This basically skips the +1 and final correction in quotient estimation.++#define mul_4(P0,P1,P2)                         \+        xorl   %ecx, %ecx ;                        \+        movq   P2, %rdx ;                       \+        mulxq  P1, %r8, %r9 ;                    \+        mulxq  0x8+P1, %rax, %r10 ;              \+        addq   %rax, %r9 ;                         \+        mulxq  0x10+P1, %rax, %r11 ;             \+        adcq   %rax, %r10 ;                        \+        mulxq  0x18+P1, %rax, %r12 ;             \+        adcq   %rax, %r11 ;                        \+        adcq   %rcx, %r12 ;                        \+        xorl   %ecx, %ecx ;                        \+        movq   0x8+P2, %rdx ;                   \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rbx, %r10 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x18+P1, %rax, %r13 ;             \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rcx, %r13 ;                        \+        adcxq  %rcx, %r13 ;                        \+        xorl   %ecx, %ecx ;                        \+        movq   0x10+P2, %rdx ;                  \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rbx, %r13 ;                        \+        mulxq  0x18+P1, %rax, %r14 ;             \+        adcxq  %rax, %r13 ;                        \+        adoxq  %rcx, %r14 ;                        \+        adcxq  %rcx, %r14 ;                        \+        xorl   %ecx, %ecx ;                        \+        movq   0x18+P2, %rdx ;                  \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rbx, %r13 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r13 ;                        \+        adoxq  %rbx, %r14 ;                        \+        mulxq  0x18+P1, %rax, %r15 ;             \+        adcxq  %rax, %r14 ;                        \+        adoxq  %rcx, %r15 ;                        \+        adcxq  %rcx, %r15 ;                        \+        movl   $0x26, %edx ;                       \+        xorl   %ecx, %ecx ;                        \+        mulxq  %r12, %rax, %rbx ;                   \+        adcxq  %rax, %r8 ;                         \+        adoxq  %rbx, %r9 ;                         \+        mulxq  %r13, %rax, %rbx ;                   \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rbx, %r10 ;                        \+        mulxq  %r14, %rax, %rbx ;                   \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  %r15, %rax, %r12 ;                   \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rcx, %r12 ;                        \+        adcxq  %rcx, %r12 ;                        \+        shldq  $0x1, %r11, %r12 ;                   \+        btr    $0x3f, %r11 ;                       \+        movl   $0x13, %edx ;                       \+        imulq  %r12, %rdx ;                        \+        addq   %rdx, %r8 ;                         \+        adcq   %rcx, %r9 ;                         \+        adcq   %rcx, %r10 ;                        \+        adcq   %rcx, %r11 ;                        \+        movq   %r8, P0 ;                        \+        movq   %r9, 0x8+P0 ;                    \+        movq   %r10, 0x10+P0 ;                  \+        movq   %r11, 0x18+P0++// Multiplication just giving a 5-digit result (actually < 39 * p_25519)+// by not doing anything beyond the first stage of reduction++#define mul_5(P0,P1,P2)                         \+        xorl   %edi, %edi ;                        \+        movq   P2, %rdx ;                       \+        mulxq  P1, %r8, %r9 ;                    \+        mulxq  0x8+P1, %rax, %r10 ;              \+        addq   %rax, %r9 ;                         \+        mulxq  0x10+P1, %rax, %r11 ;             \+        adcq   %rax, %r10 ;                        \+        mulxq  0x18+P1, %rax, %r12 ;             \+        adcq   %rax, %r11 ;                        \+        adcq   %rdi, %r12 ;                        \+        xorl   %edi, %edi ;                        \+        movq   0x8+P2, %rdx ;                   \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rbx, %r10 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x18+P1, %rax, %r13 ;             \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rdi, %r13 ;                        \+        adcxq  %rdi, %r13 ;                        \+        xorl   %edi, %edi ;                        \+        movq   0x10+P2, %rdx ;                  \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rbx, %r13 ;                        \+        mulxq  0x18+P1, %rax, %r14 ;             \+        adcxq  %rax, %r13 ;                        \+        adoxq  %rdi, %r14 ;                        \+        adcxq  %rdi, %r14 ;                        \+        xorl   %edi, %edi ;                        \+        movq   0x18+P2, %rdx ;                  \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rbx, %r13 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r13 ;                        \+        adoxq  %rbx, %r14 ;                        \+        mulxq  0x18+P1, %rax, %r15 ;             \+        adcxq  %rax, %r14 ;                        \+        adoxq  %rdi, %r15 ;                        \+        adcxq  %rdi, %r15 ;                        \+        movl   $0x26, %edx ;                       \+        xorl   %edi, %edi ;                        \+        mulxq  %r12, %rax, %rbx ;                   \+        adcxq  %rax, %r8 ;                         \+        adoxq  %rbx, %r9 ;                         \+        mulxq  %r13, %rax, %rbx ;                   \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rbx, %r10 ;                        \+        mulxq  %r14, %rax, %rbx ;                   \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  %r15, %rax, %r12 ;                   \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rdi, %r12 ;                        \+        adcxq  %rdi, %r12 ;                        \+        movq   %r8, P0 ;                        \+        movq   %r9, 0x8+P0 ;                    \+        movq   %r10, 0x10+P0 ;                  \+        movq   %r11, 0x18+P0 ;                  \+        movq   %r12, 0x20+P0++// Squaring just giving a result < 2 * p_25519, which is done by+// basically skipping the +1 in the quotient estimate and the final+// optional correction.++#define sqr_4(P0,P1)                            \+        movq   P1, %rdx ;                       \+        mulxq  %rdx, %r8, %r15 ;                    \+        mulxq  0x8+P1, %r9, %r10 ;               \+        mulxq  0x18+P1, %r11, %r12 ;             \+        movq   0x10+P1, %rdx ;                  \+        mulxq  0x18+P1, %r13, %r14 ;             \+        xorl   %ebx, %ebx ;                        \+        mulxq  P1, %rax, %rcx ;                  \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rcx, %r11 ;                        \+        mulxq  0x8+P1, %rax, %rcx ;              \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rcx, %r12 ;                        \+        movq   0x18+P1, %rdx ;                  \+        mulxq  0x8+P1, %rax, %rcx ;              \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rcx, %r13 ;                        \+        adcxq  %rbx, %r13 ;                        \+        adoxq  %rbx, %r14 ;                        \+        adcq   %rbx, %r14 ;                        \+        xorl   %ebx, %ebx ;                        \+        adcxq  %r9, %r9 ;                          \+        adoxq  %r15, %r9 ;                         \+        movq   0x8+P1, %rdx ;                   \+        mulxq  %rdx, %rax, %rdx ;                   \+        adcxq  %r10, %r10 ;                        \+        adoxq  %rax, %r10 ;                        \+        adcxq  %r11, %r11 ;                        \+        adoxq  %rdx, %r11 ;                        \+        movq   0x10+P1, %rdx ;                  \+        mulxq  %rdx, %rax, %rdx ;                   \+        adcxq  %r12, %r12 ;                        \+        adoxq  %rax, %r12 ;                        \+        adcxq  %r13, %r13 ;                        \+        adoxq  %rdx, %r13 ;                        \+        movq   0x18+P1, %rdx ;                  \+        mulxq  %rdx, %rax, %r15 ;                   \+        adcxq  %r14, %r14 ;                        \+        adoxq  %rax, %r14 ;                        \+        adcxq  %rbx, %r15 ;                        \+        adoxq  %rbx, %r15 ;                        \+        movl   $0x26, %edx ;                       \+        xorl   %ebx, %ebx ;                        \+        mulxq  %r12, %rax, %rcx ;                   \+        adcxq  %rax, %r8 ;                         \+        adoxq  %rcx, %r9 ;                         \+        mulxq  %r13, %rax, %rcx ;                   \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rcx, %r10 ;                        \+        mulxq  %r14, %rax, %rcx ;                   \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rcx, %r11 ;                        \+        mulxq  %r15, %rax, %r12 ;                   \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        adcxq  %rbx, %r12 ;                        \+        shldq  $0x1, %r11, %r12 ;                   \+        btr    $0x3f, %r11 ;                       \+        movl   $0x13, %edx ;                       \+        imulq  %r12, %rdx ;                        \+        addq   %rdx, %r8 ;                         \+        adcq   %rbx, %r9 ;                         \+        adcq   %rbx, %r10 ;                        \+        adcq   %rbx, %r11 ;                        \+        movq   %r8, P0 ;                        \+        movq   %r9, 0x8+P0 ;                    \+        movq   %r10, 0x10+P0 ;                  \+        movq   %r11, 0x18+P0++// Add 5-digit inputs and normalize to 4 digits++#define add5_4(P0,P1,P2)                        \+        movq    P1, %r8 ;                       \+        addq    P2, %r8 ;                       \+        movq    8+P1, %r9 ;                     \+        adcq    8+P2, %r9 ;                     \+        movq    16+P1, %r10 ;                   \+        adcq    16+P2, %r10 ;                   \+        movq    24+P1, %r11 ;                   \+        adcq    24+P2, %r11 ;                   \+        movq    32+P1, %r12 ;                   \+        adcq    32+P2, %r12 ;                   \+        xorl    %ebx, %ebx ;                       \+        shldq  $0x1, %r11, %r12 ;                   \+        btr    $0x3f, %r11 ;                       \+        movl   $0x13, %edx ;                       \+        imulq  %r12, %rdx ;                        \+        addq   %rdx, %r8 ;                         \+        adcq   %rbx, %r9 ;                         \+        adcq   %rbx, %r10 ;                        \+        adcq   %rbx, %r11 ;                        \+        movq   %r8, P0 ;                        \+        movq   %r9, 0x8+P0 ;                    \+        movq   %r10, 0x10+P0 ;                  \+        movq   %r11, 0x18+P0++// Modular addition with double modulus 2 * p_25519 = 2^256 - 38.+// This only ensures that the result fits in 4 digits, not that it is reduced+// even w.r.t. double modulus. The result is always correct modulo provided+// the sum of the inputs is < 2^256 + 2^256 - 38, so in particular provided+// at least one of them is reduced double modulo.++#define add_twice4(P0,P1,P2)                    \+        movq    P1, %r8 ;                       \+        xorl    %ecx, %ecx ;                       \+        addq    P2, %r8 ;                       \+        movq    0x8+P1, %r9 ;                   \+        adcq    0x8+P2, %r9 ;                   \+        movq    0x10+P1, %r10 ;                 \+        adcq    0x10+P2, %r10 ;                 \+        movq    0x18+P1, %r11 ;                 \+        adcq    0x18+P2, %r11 ;                 \+        movl    $38, %eax ;                        \+        cmovncq %rcx, %rax ;                       \+        addq    %rax, %r8 ;                        \+        adcq    %rcx, %r9 ;                        \+        adcq    %rcx, %r10 ;                       \+        adcq    %rcx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++// Modular subtraction with double modulus 2 * p_25519 = 2^256 - 38++#define sub_twice4(P0,P1,P2)                    \+        movq    P1, %r8 ;                       \+        xorl    %ebx, %ebx ;                       \+        subq    P2, %r8 ;                       \+        movq    8+P1, %r9 ;                     \+        sbbq    8+P2, %r9 ;                     \+        movl    $38, %ecx ;                        \+        movq    16+P1, %r10 ;                   \+        sbbq    16+P2, %r10 ;                   \+        movq    24+P1, %rax ;                   \+        sbbq    24+P2, %rax ;                   \+        cmovncq %rbx, %rcx ;                       \+        subq    %rcx, %r8 ;                        \+        sbbq    %rbx, %r9 ;                        \+        sbbq    %rbx, %r10 ;                       \+        sbbq    %rbx, %rax ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 8+P0 ;                     \+        movq    %r10, 16+P0 ;                   \+        movq    %rax, 24+P0++// 5-digit subtraction with upward bias to make it positive, adding+// 1000 * (2^255 - 19) = 2^256 * 500 - 19000, then normalizing to 4 digits++#define sub5_4(P0,P1,P2)                        \+        movq    P1, %r8 ;                       \+        subq    P2, %r8 ;                       \+        movq    8+P1, %r9 ;                     \+        sbbq    8+P2, %r9 ;                     \+        movq    16+P1, %r10 ;                   \+        sbbq    16+P2, %r10 ;                   \+        movq    24+P1, %r11 ;                   \+        sbbq    24+P2, %r11 ;                   \+        movq    32+P1, %r12 ;                   \+        sbbq    32+P2, %r12 ;                   \+        xorl    %ebx, %ebx ;                       \+        subq    $19000, %r8 ;                      \+        sbbq    %rbx, %r9 ;                        \+        sbbq    %rbx, %r10 ;                       \+        sbbq    %rbx, %r11 ;                       \+        sbbq    %rbx, %r12 ;                       \+        addq    $500, %r12 ;                       \+        shldq  $0x1, %r11, %r12 ;                   \+        btr    $0x3f, %r11 ;                       \+        movl   $0x13, %edx ;                       \+        imulq  %r12, %rdx ;                        \+        addq   %rdx, %r8 ;                         \+        adcq   %rbx, %r9 ;                         \+        adcq   %rbx, %r10 ;                        \+        adcq   %rbx, %r11 ;                        \+        movq   %r8, P0 ;                        \+        movq   %r9, 0x8+P0 ;                    \+        movq   %r10, 0x10+P0 ;                  \+        movq   %r11, 0x18+P0++// Combined z = c * x + y with reduction only < 2 * p_25519+// It is assumed that 19 * (c * x + y) < 2^60 * 2^256 so we+// don't need a high mul in the final part.++#define cmadd_4(P0,C1,P2,P3)                    \+        movq    P3, %r8 ;                       \+        movq    8+P3, %r9 ;                     \+        movq    16+P3, %r10 ;                   \+        movq    24+P3, %r11 ;                   \+        xorl    %edi, %edi ;                       \+        movq    $C1, %rdx ;                        \+        mulxq   P2, %rax, %rbx ;                 \+        adcxq   %rax, %r8 ;                        \+        adoxq   %rbx, %r9 ;                        \+        mulxq   8+P2, %rax, %rbx ;               \+        adcxq   %rax, %r9 ;                        \+        adoxq   %rbx, %r10 ;                       \+        mulxq   16+P2, %rax, %rbx ;              \+        adcxq   %rax, %r10 ;                       \+        adoxq   %rbx, %r11 ;                       \+        mulxq   24+P2, %rax, %rbx ;              \+        adcxq   %rax, %r11 ;                       \+        adoxq   %rdi, %rbx ;                       \+        adcxq   %rdi, %rbx ;                       \+        shldq   $0x1, %r11, %rbx ;                  \+        btr     $63, %r11 ;                        \+        movl    $0x13, %edx ;                      \+        imulq   %rdx, %rbx ;                       \+        addq    %rbx, %r8 ;                        \+        adcq    %rdi, %r9 ;                        \+        adcq    %rdi, %r10 ;                       \+        adcq    %rdi, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++// Multiplex: z := if NZ then x else y++#define mux_4(P0,P1,P2)                         \+        movq    P1, %rax ;                      \+        movq    P2, %rcx ;                      \+        cmovzq  %rcx, %rax ;                       \+        movq    %rax, P0 ;                      \+        movq    8+P1, %rax ;                    \+        movq    8+P2, %rcx ;                    \+        cmovzq  %rcx, %rax ;                       \+        movq    %rax, 8+P0 ;                    \+        movq    16+P1, %rax ;                   \+        movq    16+P2, %rcx ;                   \+        cmovzq  %rcx, %rax ;                       \+        movq    %rax, 16+P0 ;                   \+        movq    24+P1, %rax ;                   \+        movq    24+P2, %rcx ;                   \+        cmovzq  %rcx, %rax ;                       \+        movq    %rax, 24+P0++S2N_BN_SYMBOL(curve25519_x25519):+S2N_BN_SYMBOL(curve25519_x25519_byte):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+#endif++// Save registers, make room for temps, preserve input arguments.++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(NSPACE)++// Move the output pointer to a stable place++        movq    %rdi, res++// Copy the inputs to the local variables with minimal mangling:+//+//  - The scalar is in principle turned into 01xxx...xxx000 but+//    in the structure below the special handling of these bits is+//    explicit in the main computation; the scalar is just copied.+//+//  - The point x coord is reduced mod 2^255 by masking off the+//    top bit. In the main loop we only need reduction < 2 * p_25519.++        movq    (%rsi), %rax+        movq    %rax, (%rsp)+        movq    8(%rsi), %rax+        movq    %rax, 8(%rsp)+        movq    16(%rsi), %rax+        movq    %rax, 16(%rsp)+        movq    24(%rsi), %rax+        movq    %rax, 24(%rsp)++        movq    (%rdx), %r8+        movq    8(%rdx), %r9+        movq    16(%rdx), %r10+        movq    24(%rdx), %r11+        btr     $63, %r11+        movq    %r8, 32(%rsp)+        movq    %r9, 40(%rsp)+        movq    %r10, 48(%rsp)+        movq    %r11, 56(%rsp)++// Initialize with explicit doubling in order to handle set bit 254.+// Set swap = 1 and (xm,zm) = (x,1) then double as (xn,zn) = 2 * (x,1).+// We use the fact that the point x coordinate is still in registers.+// Since zm = 1 we could do the doubling with an operation count of+// 2 * S + M instead of 2 * S + 2 * M, but it doesn't seem worth+// the slight complication arising from a different linear combination.++        movl    $1, %eax+        movq    %rax, swap+        movq    %r8, 256(%rsp)+        movq    %rax, 96(%rsp)+        xorl    %eax, %eax+        movq    %r9, 264(%rsp)+        movq    %rax, 104(%rsp)+        movq    %r10, 272(%rsp)+        movq    %rax, 112(%rsp)+        movq    %r11, 280(%rsp)+        movq    %rax, 120(%rsp)++        sub_twice4(d,xm,zm)+        add_twice4(s,xm,zm)+        sqr_4(d,d)+        sqr_4(s,s)+        sub_twice4(p,s,d)+        cmadd_4(e,0x1db42,p,d)+        mul_4(xn,s,d)+        mul_4(zn,p,e)++// The main loop over unmodified bits from i = 253, ..., i = 3 (inclusive).+// This is a classic Montgomery ladder, with the main coordinates only+// reduced mod 2 * p_25519, some intermediate results even more loosely.++        movl    $253, %eax+        movq    %rax, i++Lcurve25519_x25519_scalarloop:++// sm = xm + zm; sn = xn + zn; dm = xm - zm; dn = xn - zn++        sub_twice4(dm,xm,zm)+        add_twice4(sn,xn,zn)+        sub_twice4(dn,xn,zn)+        add_twice4(sm,xm,zm)++// DOUBLING: mux d = xt - zt and s = xt + zt for appropriate choice of (xt,zt)++        movq    i, %rdx+        movq    %rdx, %rcx+        shrq    $6, %rdx+        movq    (%rsp,%rdx,8), %rdx+        shrq    %cl, %rdx+        andq    $1, %rdx+        cmpq    swap, %rdx+        movq    %rdx, swap+        mux_4(d,dm,dn)+        mux_4(s,sm,sn)++// ADDING: dmsn = dm * sn; dnsm = sm * dn++        mul_5(dnsm,sm,dn)+        mul_5(dmsn,sn,dm)++// DOUBLING: d = (xt - zt)^2++        sqr_4(d,d)++// ADDING: dpro = (dmsn - dnsm)^2, spro = (dmsn + dnsm)^2+// DOUBLING: s = (xt + zt)^2++        sub5_4(dpro,dmsn,dnsm)+        add5_4(spro,dmsn,dnsm)+        sqr_4(s,s)+        sqr_4(dpro,dpro)++// DOUBLING: p = 4 * xt * zt = s - d++        sub_twice4(p,s,d)++// ADDING: xm' = (dmsn + dnsm)^2++        sqr_4(xm,spro)++// DOUBLING: e = 121666 * p + d++        cmadd_4(e,0x1db42,p,d)++// DOUBLING: xn' = (xt + zt)^2 * (xt - zt)^2 = s * d++        mul_4(xn,s,d)++// DOUBLING: zn' = (4 * xt * zt) * ((xt - zt)^2 + 121666 * (4 * xt * zt))+//               = p * (d + 121666 * p)++        mul_4(zn,p,e)++// ADDING: zm' = x * (dmsn - dnsm)^2++        mul_4(zm,dpro,pointx)++// Loop down as far as 3 (inclusive)++        movq    i, %rax+        subq    $1, %rax+        movq    %rax, i+        cmpq    $3, %rax+        jnc     Lcurve25519_x25519_scalarloop++// Multiplex directly into (xn,zn) then do three pure doubling steps;+// this accounts for the implicit zeroing of the three lowest bits+// of the scalar.++        movq    swap, %rdx+        testq   %rdx, %rdx+        mux_4(xn,xm,xn)+        mux_4(zn,zm,zn)++        sub_twice4(d,xn,zn)+        add_twice4(s,xn,zn)+        sqr_4(d,d)+        sqr_4(s,s)+        sub_twice4(p,s,d)+        cmadd_4(e,0x1db42,p,d)+        mul_4(xn,s,d)+        mul_4(zn,p,e)++        sub_twice4(d,xn,zn)+        add_twice4(s,xn,zn)+        sqr_4(d,d)+        sqr_4(s,s)+        sub_twice4(p,s,d)+        cmadd_4(e,0x1db42,p,d)+        mul_4(xn,s,d)+        mul_4(zn,p,e)++        sub_twice4(d,xn,zn)+        add_twice4(s,xn,zn)+        sqr_4(d,d)+        sqr_4(s,s)+        sub_twice4(p,s,d)+        cmadd_4(e,0x1db42,p,d)+        mul_4(xn,s,d)+        mul_4(zn,p,e)++// The projective result of the scalar multiplication is now (xn,zn).+// Prepare to call the modular inverse function to get zn' = 1/zn++        leaq    224(%rsp), %rdi+        leaq    224(%rsp), %rsi++// Inline copy of bignum_inv_p25519, identical except for stripping out+// the prologue and epilogue saving and restoring registers and making+// and reclaiming room on the stack. For more details and explanations see+// "x86/curve25519/bignum_inv_p25519.S". Note that the stack it uses for+// its own temporaries is 208 bytes, so it has no effect on variables+// that are needed in the rest of our computation here: res, xn and zn.++        movq    %rdi, 0xc0(%rsp)+        xorl    %eax, %eax+        leaq    -0x13(%rax), %rcx+        notq    %rax+        movq    %rcx, (%rsp)+        movq    %rax, 0x8(%rsp)+        movq    %rax, 0x10(%rsp)+        btr     $0x3f, %rax+        movq    %rax, 0x18(%rsp)+        movq    (%rsi), %rdx+        movq    0x8(%rsi), %rcx+        movq    0x10(%rsi), %r8+        movq    0x18(%rsi), %r9+        movl    $0x1, %eax+        xorl    %r10d, %r10d+        bts     $0x3f, %r9+        adcq    %r10, %rax+        imulq   $0x13, %rax, %rax+        addq    %rax, %rdx+        adcq    %r10, %rcx+        adcq    %r10, %r8+        adcq    %r10, %r9+        movl    $0x13, %eax+        cmovbq  %r10, %rax+        subq    %rax, %rdx+        sbbq    %r10, %rcx+        sbbq    %r10, %r8+        sbbq    %r10, %r9+        btr     $0x3f, %r9+        movq    %rdx, 0x20(%rsp)+        movq    %rcx, 0x28(%rsp)+        movq    %r8, 0x30(%rsp)+        movq    %r9, 0x38(%rsp)+        xorl    %eax, %eax+        movq    %rax, 0x40(%rsp)+        movq    %rax, 0x48(%rsp)+        movq    %rax, 0x50(%rsp)+        movq    %rax, 0x58(%rsp)+        movabsq $0xa0f99e2375022099, %rax+        movq    %rax, 0x60(%rsp)+        movabsq $0xa8c68f3f1d132595, %rax+        movq    %rax, 0x68(%rsp)+        movabsq $0x6c6c893805ac5242, %rax+        movq    %rax, 0x70(%rsp)+        movabsq $0x276508b241770615, %rax+        movq    %rax, 0x78(%rsp)+        movq    $0xa,  0x90(%rsp)+        movq    $0x1,  0x98(%rsp)+        jmp     Lcurve25519_x25519_midloop+Lcurve25519_x25519_inverseloop:+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %rdi+        andq    %r11, %rdi+        addq    %rax, %rdi+        movq    %rdi, 0x80(%rsp)+        movq    %r12, %rax+        andq    %r13, %rax+        movq    %r14, %rsi+        andq    %r15, %rsi+        addq    %rax, %rsi+        movq    %rsi, 0x88(%rsp)+        xorl    %ebx, %ebx+        movq    (%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x20(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    (%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x20(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        xorl    %ecx, %ecx+        movq    0x8(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x28(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, (%rsp)+        xorl    %edi, %edi+        movq    0x8(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    0x28(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $0x3b, %rbp, %rsi+        movq    %rsi, 0x20(%rsp)+        xorl    %esi, %esi+        movq    0x10(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    0x30(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rcx, %rbx+        movq    %rbx, 0x8(%rsp)+        xorl    %ebx, %ebx+        movq    0x10(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x30(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $0x3b, %rdi, %rbp+        movq    %rbp, 0x28(%rsp)+        movq    0x18(%rsp), %rax+        xorq    %r9, %rax+        movq    %rax, %rbp+        sarq    $0x3f, %rbp+        andq    %r8, %rbp+        negq    %rbp+        mulq    %r8+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x38(%rsp), %rax+        xorq    %r11, %rax+        movq    %rax, %rdx+        sarq    $0x3f, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbp+        mulq    %r10+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        shrdq   $0x3b, %rsi, %rcx+        movq    %rcx, 0x10(%rsp)+        shrdq   $0x3b, %rbp, %rsi+        movq    0x18(%rsp), %rax+        movq    %rsi, 0x18(%rsp)+        xorq    %r13, %rax+        movq    %rax, %rsi+        sarq    $0x3f, %rsi+        andq    %r12, %rsi+        negq    %rsi+        mulq    %r12+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        movq    0x38(%rsp), %rax+        xorq    %r15, %rax+        movq    %rax, %rdx+        sarq    $0x3f, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rsi+        mulq    %r14+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, 0x30(%rsp)+        shrdq   $0x3b, %rsi, %rbx+        movq    %rbx, 0x38(%rsp)+        movq    0x80(%rsp), %rbx+        movq    0x88(%rsp), %rbp+        xorl    %ecx, %ecx+        movq    0x40(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x60(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x40(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x40(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x60(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x60(%rsp)+        xorl    %ebx, %ebx+        movq    0x48(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x68(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    0x48(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, 0x48(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x68(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, 0x68(%rsp)+        xorl    %ecx, %ecx+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x70(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x50(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x50(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x70(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x70(%rsp)+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        movq    %r9, %rbx+        andq    %r8, %rbx+        negq    %rbx+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbx+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rbx, %rdx+        movq    %rdx, %rbx+        shldq   $0x1, %rcx, %rdx+        sarq    $0x3f, %rbx+        addq    %rbx, %rdx+        movl    $0x13, %eax+        imulq   %rdx+        movq    0x40(%rsp), %r8+        addq    %rax, %r8+        movq    %r8, 0x40(%rsp)+        movq    0x48(%rsp), %r8+        adcq    %rdx, %r8+        movq    %r8, 0x48(%rsp)+        movq    0x50(%rsp), %r8+        adcq    %rbx, %r8+        movq    %r8, 0x50(%rsp)+        adcq    %rbx, %rcx+        shlq    $0x3f, %rax+        addq    %rax, %rcx+        movq    0x58(%rsp), %rax+        movq    %rcx, 0x58(%rsp)+        xorq    %r13, %rax+        movq    %r13, %rcx+        andq    %r12, %rcx+        negq    %rcx+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rcx+        movq    0x78(%rsp), %rax+        xorq    %r15, %rax+        movq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rcx+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rcx, %rdx+        movq    %rdx, %rcx+        shldq   $0x1, %rsi, %rdx+        sarq    $0x3f, %rcx+        movl    $0x13, %eax+        addq    %rcx, %rdx+        imulq   %rdx+        movq    0x60(%rsp), %r8+        addq    %rax, %r8+        movq    %r8, 0x60(%rsp)+        movq    0x68(%rsp), %r8+        adcq    %rdx, %r8+        movq    %r8, 0x68(%rsp)+        movq    0x70(%rsp), %r8+        adcq    %rcx, %r8+        movq    %r8, 0x70(%rsp)+        adcq    %rcx, %rsi+        shlq    $0x3f, %rax+        addq    %rax, %rsi+        movq    %rsi, 0x78(%rsp)+Lcurve25519_x25519_midloop:+        movq    0x98(%rsp), %rsi+        movq    (%rsp), %rdx+        movq    0x20(%rsp), %rcx+        movq    %rdx, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        xorl    %ebp, %ebp+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %rdx+        leaq    (%rcx,%rax), %rdi+        shlq    $0x16, %rdx+        shlq    $0x16, %rdi+        sarq    $0x2b, %rdx+        sarq    $0x2b, %rdi+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %rbx+        leaq    (%rcx,%rax), %rcx+        sarq    $0x2a, %rbx+        sarq    $0x2a, %rcx+        movq    %rdx, 0xa0(%rsp)+        movq    %rbx, 0xa8(%rsp)+        movq    %rdi, 0xb0(%rsp)+        movq    %rcx, 0xb8(%rsp)+        movq    (%rsp), %r12+        imulq   %r12, %rdi+        imulq   %rdx, %r12+        movq    0x20(%rsp), %r13+        imulq   %r13, %rbx+        imulq   %rcx, %r13+        addq    %rbx, %r12+        addq    %rdi, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r10+        shlq    $0x16, %r8+        shlq    $0x16, %r10+        sarq    $0x2b, %r8+        sarq    $0x2b, %r10+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r15+        leaq    (%rcx,%rax), %r11+        sarq    $0x2a, %r15+        sarq    $0x2a, %r11+        movq    %r13, %rbx+        movq    %r12, %rcx+        imulq   %r8, %r12+        imulq   %r15, %rbx+        addq    %rbx, %r12+        imulq   %r11, %r13+        imulq   %r10, %rcx+        addq    %rcx, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    0xa0(%rsp), %rax+        imulq   %r8, %rax+        movq    0xb0(%rsp), %rdx+        imulq   %r15, %rdx+        imulq   0xa8(%rsp), %r8+        imulq   0xb8(%rsp), %r15+        addq    %r8, %r15+        leaq    (%rax,%rdx), %r9+        movq    0xa0(%rsp), %rax+        imulq   %r10, %rax+        movq    0xb0(%rsp), %rdx+        imulq   %r11, %rdx+        imulq   0xa8(%rsp), %r10+        imulq   0xb8(%rsp), %r11+        addq    %r10, %r11+        leaq    (%rax,%rdx), %r13+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r12+        shlq    $0x15, %r8+        shlq    $0x15, %r12+        sarq    $0x2b, %r8+        sarq    $0x2b, %r12+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r10+        leaq    (%rcx,%rax), %r14+        sarq    $0x2b, %r10+        sarq    $0x2b, %r14+        movq    %r9, %rax+        imulq   %r8, %rax+        movq    %r13, %rdx+        imulq   %r10, %rdx+        imulq   %r15, %r8+        imulq   %r11, %r10+        addq    %r8, %r10+        leaq    (%rax,%rdx), %r8+        movq    %r9, %rax+        imulq   %r12, %rax+        movq    %r13, %rdx+        imulq   %r14, %rdx+        imulq   %r15, %r12+        imulq   %r11, %r14+        addq    %r12, %r14+        leaq    (%rax,%rdx), %r12+        movq    %rsi, 0x98(%rsp)+        decq     0x90(%rsp)+        jne     Lcurve25519_x25519_inverseloop+        movq    (%rsp), %rax+        movq    0x20(%rsp), %rcx+        imulq   %r8, %rax+        imulq   %r10, %rcx+        addq    %rcx, %rax+        sarq    $0x3f, %rax+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        xorq    %rax, %r9+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        xorq    %rax, %r11+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        xorq    %rax, %r13+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        xorq    %rax, %r15+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %r12+        andq    %r11, %r12+        addq    %rax, %r12+        xorl    %r13d, %r13d+        movq    0x40(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x60(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movq    0x48(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r13+        adcq    %rdx, %r14+        movq    0x68(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x70(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        andq    %r8, %r9+        negq    %r9+        mulq    %r8+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %r9+        mulq    %r10+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    %r9, %rax+        shldq   $0x1, %r15, %rax+        sarq    $0x3f, %r9+        movl    $0x13, %ebx+        leaq    0x1(%rax,%r9,1), %rax+        imulq   %rbx+        xorl    %ebp, %ebp+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r9, %r14+        adcq    %r9, %r15+        shlq    $0x3f, %rax+        addq    %rax, %r15+        cmovns  %rbp, %rbx+        subq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    %rbp, %r14+        sbbq    %rbp, %r15+        btr     $0x3f, %r15+        movq    0xc0(%rsp), %rdi+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)++// Now the result is xn * (1/zn), fully reduced modulo p.+// Note that in the degenerate case zn = 0 (mod p_25519), the+// modular inverse code above will produce 1/zn = 0, giving+// the correct overall X25519 result of zero for the point at+// infinity.++        movq    res, %rbp+        mul_p25519(resx,xn,zn)++// Restore stack and registers++        CFI_INC_RSP(NSPACE)++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(curve25519_x25519)+S2N_BN_SIZE_DIRECTIVE(curve25519_x25519_byte)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/curve25519_x25519_alt.S view
@@ -0,0 +1,2350 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// The x25519 function for curve25519+// Inputs scalar[4], point[4]; output res[4]+//+// extern void curve25519_x25519_alt+//   (uint64_t res[static 4],const uint64_t scalar[static 4],+//    const uint64_t point[static 4]);+//+// The function has a second prototype considering the arguments as arrays+// of bytes rather than 64-bit words. The underlying code is the same, since+// the x86 platform is little-endian.+//+// extern void curve25519_x25519_byte_alt+//   (uint8_t res[static 32],const uint8_t scalar[static 32],+//    const uint8_t point[static 32]);+//+// Given a scalar n and the X coordinate of an input point P = (X,Y) on+// curve25519 (Y can live in any extension field of characteristic 2^255-19),+// this returns the X coordinate of n * P = (X, Y), or 0 when n * P is the+// point at infinity. Both n and X inputs are first slightly modified/mangled+// as specified in the relevant RFC (https://www.rfc-editor.org/rfc/rfc7748);+// in particular the lower three bits of n are set to zero. Does not implement+// the zero-check specified in Section 6.1.+//+// Standard x86-64 ABI: RDI = res, RSI = scalar, RDX = point+// Microsoft x64 ABI:   RCX = res, RDX = scalar, R8 = point+// ----------------------------------------------------------------------------+#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(curve25519_x25519_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(curve25519_x25519_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(curve25519_x25519_alt)+        S2N_BN_SYM_VISIBILITY_DIRECTIVE(curve25519_x25519_byte_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(curve25519_x25519_byte_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(curve25519_x25519_byte_alt)+        .text+        .balign 32++// Size of individual field elements++#define NUMSIZE 32++// Stable homes for the input result argument during the whole body+// and other variables that are only needed prior to the modular inverse.++#define res  12*NUMSIZE(%rsp)+#define i  12*NUMSIZE+8(%rsp)+#define swap  12*NUMSIZE+16(%rsp)++// Pointers to result x coord to be written, assuming the base "res"+// has been loaded into %rbp++#define resx 0(%rbp)++// Pointer-offset pairs for temporaries on stack with some aliasing.+// Both dmsn and dnsm need space for >= 5 digits, and we allocate 8++#define scalar (0*NUMSIZE)(%rsp)++#define pointx (1*NUMSIZE)(%rsp)++#define dm (2*NUMSIZE)(%rsp)++#define zm (3*NUMSIZE)(%rsp)+#define sm (3*NUMSIZE)(%rsp)+#define dpro (3*NUMSIZE)(%rsp)++#define sn (4*NUMSIZE)(%rsp)++#define dn (5*NUMSIZE)(%rsp)+#define e (5*NUMSIZE)(%rsp)++#define dmsn (6*NUMSIZE)(%rsp)+#define p (6*NUMSIZE)(%rsp)+#define zn (7*NUMSIZE)(%rsp)++#define xm (8*NUMSIZE)(%rsp)+#define dnsm (8*NUMSIZE)(%rsp)+#define spro (8*NUMSIZE)(%rsp)++#define xn (10*NUMSIZE)(%rsp)+#define s (10*NUMSIZE)(%rsp)++#define d (11*NUMSIZE)(%rsp)++// Total size to reserve on the stack+// This includes space for the 3 other variables above+// and rounds up to a multiple of 32++#define NSPACE 13*NUMSIZE++// Macro wrapping up the basic field operation bignum_mul_p25519_alt, only+// trivially different from a pure function call to that subroutine.++#define mul_p25519(P0,P1,P2)                    \+        movq    P1, %rax ;                      \+        mulq     P2;                 \+        movq    %rax, %r8 ;                         \+        movq    %rdx, %r9 ;                         \+        xorq    %r10, %r10 ;                        \+        xorq    %r11, %r11 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x8+P2;             \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     P2;                 \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        adcq    $0x0, %r11 ;                        \+        xorq    %r12, %r12 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x10+P2;            \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    %r12, %r12 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x8+P2;             \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     P2;                 \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        xorq    %r13, %r13 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x18+P2;            \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    %r13, %r13 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x10+P2;            \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x8+P2;             \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     P2;                 \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        xorq    %r14, %r14 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x18+P2;            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    %r14, %r14 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x10+P2;            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x8+P2;             \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        xorq    %r15, %r15 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x18+P2;            \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    %r15, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x10+P2;            \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    $0x0, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x18+P2;            \+        addq    %rax, %r14 ;                        \+        adcq    %rdx, %r15 ;                        \+        movl    $0x26, %esi ;                       \+        movq    %r12, %rax ;                        \+        mulq    %rsi;                            \+        addq    %rax, %r8 ;                         \+        adcq    %rdx, %r9 ;                         \+        sbbq    %rcx, %rcx ;                        \+        movq    %r13, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r14, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r15, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        xorq    %rcx, %rcx ;                        \+        addq    %rax, %r11 ;                        \+        movq    %rdx, %r12 ;                        \+        adcq    %rcx, %r12 ;                        \+        shldq   $0x1, %r11, %r12 ;                    \+        leaq    0x1(%r12), %rax ;                  \+        movl    $0x13, %esi ;                       \+        bts     $63, %r11 ;                         \+        imulq   %rsi, %rax ;                        \+        addq    %rax, %r8 ;                         \+        adcq    %rcx, %r9 ;                         \+        adcq    %rcx, %r10 ;                        \+        adcq    %rcx, %r11 ;                        \+        sbbq    %rax, %rax ;                        \+        notq    %rax;                            \+        andq    %rsi, %rax ;                        \+        subq    %rax, %r8 ;                         \+        sbbq    %rcx, %r9 ;                         \+        sbbq    %rcx, %r10 ;                        \+        sbbq    %rcx, %r11 ;                        \+        btr     $63, %r11 ;                         \+        movq    %r8, P0 ;                        \+        movq    %r9, 0x8+P0 ;                    \+        movq    %r10, 0x10+P0 ;                  \+        movq    %r11, 0x18+P0++// A version of multiplication that only guarantees output < 2 * p_25519.+// This basically skips the +1 and final correction in quotient estimation.++#define mul_4(P0,P1,P2)                         \+        movq    P1, %rax ;                      \+        mulq     P2;                 \+        movq    %rax, %r8 ;                         \+        movq    %rdx, %r9 ;                         \+        xorq    %r10, %r10 ;                        \+        xorq    %r11, %r11 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x8+P2;             \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     P2;                 \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        adcq    $0x0, %r11 ;                        \+        xorq    %r12, %r12 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x10+P2;            \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    %r12, %r12 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x8+P2;             \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     P2;                 \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        xorq    %r13, %r13 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x18+P2;            \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    %r13, %r13 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x10+P2;            \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x8+P2;             \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     P2;                 \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        xorq    %r14, %r14 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x18+P2;            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    %r14, %r14 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x10+P2;            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x8+P2;             \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        xorq    %r15, %r15 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x18+P2;            \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    %r15, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x10+P2;            \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    $0x0, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x18+P2;            \+        addq    %rax, %r14 ;                        \+        adcq    %rdx, %r15 ;                        \+        movl    $0x26, %esi ;                       \+        movq    %r12, %rax ;                        \+        mulq    %rsi;                            \+        addq    %rax, %r8 ;                         \+        adcq    %rdx, %r9 ;                         \+        sbbq    %rcx, %rcx ;                        \+        movq    %r13, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r14, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r15, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        xorq    %rcx, %rcx ;                        \+        addq    %rax, %r11 ;                        \+        movq    %rdx, %r12 ;                        \+        adcq    %rcx, %r12 ;                        \+        shldq   $0x1, %r11, %r12 ;                    \+        btr     $0x3f, %r11 ;                      \+        movl    $0x13, %edx ;                      \+        imulq   %r12, %rdx ;                       \+        addq    %rdx, %r8 ;                        \+        adcq    %rcx, %r9 ;                        \+        adcq    %rcx, %r10 ;                       \+        adcq    %rcx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++// Multiplication just giving a 5-digit result (actually < 39 * p_25519)+// by not doing anything beyond the first stage of reduction++#define mul_5(P0,P1,P2)                         \+        movq    P1, %rax ;                      \+        mulq     P2;                 \+        movq    %rax, %r8 ;                         \+        movq    %rdx, %r9 ;                         \+        xorq    %r10, %r10 ;                        \+        xorq    %r11, %r11 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x8+P2;             \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     P2;                 \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        adcq    $0x0, %r11 ;                        \+        xorq    %r12, %r12 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x10+P2;            \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    %r12, %r12 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x8+P2;             \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     P2;                 \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        xorq    %r13, %r13 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x18+P2;            \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    %r13, %r13 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x10+P2;            \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x8+P2;             \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     P2;                 \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        xorq    %r14, %r14 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x18+P2;            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    %r14, %r14 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x10+P2;            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x8+P2;             \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        xorq    %r15, %r15 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x18+P2;            \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    %r15, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x10+P2;            \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    $0x0, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x18+P2;            \+        addq    %rax, %r14 ;                        \+        adcq    %rdx, %r15 ;                        \+        movl    $0x26, %esi ;                       \+        movq    %r12, %rax ;                        \+        mulq    %rsi;                            \+        addq    %rax, %r8 ;                         \+        adcq    %rdx, %r9 ;                         \+        sbbq    %rcx, %rcx ;                        \+        movq    %r13, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r14, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r15, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        xorq    %rcx, %rcx ;                        \+        addq    %rax, %r11 ;                        \+        movq    %rdx, %r12 ;                        \+        adcq    %rcx, %r12 ;                        \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0 ;                 \+        movq    %r12, 0x20+P0++// Squaring just giving a result < 2 * p_25519, which is done by+// basically skipping the +1 in the quotient estimate and the final+// optional correction.++#define sqr_4(P0,P1)                            \+        movq    P1, %rax ;                      \+        mulq    %rax;                            \+        movq    %rax, %r8 ;                         \+        movq    %rdx, %r9 ;                         \+        xorq    %r10, %r10 ;                        \+        xorq    %r11, %r11 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x8+P1;             \+        addq    %rax, %rax ;                        \+        adcq    %rdx, %rdx ;                        \+        adcq    $0x0, %r11 ;                        \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        adcq    $0x0, %r11 ;                        \+        xorq    %r12, %r12 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq    %rax;                            \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x10+P1;            \+        addq    %rax, %rax ;                        \+        adcq    %rdx, %rdx ;                        \+        adcq    $0x0, %r12 ;                        \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        xorq    %r13, %r13 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x18+P1;            \+        addq    %rax, %rax ;                        \+        adcq    %rdx, %rdx ;                        \+        adcq    $0x0, %r13 ;                        \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x10+P1;            \+        addq    %rax, %rax ;                        \+        adcq    %rdx, %rdx ;                        \+        adcq    $0x0, %r13 ;                        \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        xorq    %r14, %r14 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x18+P1;            \+        addq    %rax, %rax ;                        \+        adcq    %rdx, %rdx ;                        \+        adcq    $0x0, %r14 ;                        \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq    %rax;                            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        xorq    %r15, %r15 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x18+P1;            \+        addq    %rax, %rax ;                        \+        adcq    %rdx, %rdx ;                        \+        adcq    $0x0, %r15 ;                        \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    $0x0, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq    %rax;                            \+        addq    %rax, %r14 ;                        \+        adcq    %rdx, %r15 ;                        \+        movl    $0x26, %esi ;                       \+        movq    %r12, %rax ;                        \+        mulq    %rsi;                            \+        addq    %rax, %r8 ;                         \+        adcq    %rdx, %r9 ;                         \+        sbbq    %rcx, %rcx ;                        \+        movq    %r13, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r14, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r15, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        xorq    %rcx, %rcx ;                        \+        addq    %rax, %r11 ;                        \+        movq    %rdx, %r12 ;                        \+        adcq    %rcx, %r12 ;                        \+        shldq   $0x1, %r11, %r12 ;                  \+        btr     $0x3f, %r11 ;                      \+        movl    $0x13, %edx ;                      \+        imulq   %r12, %rdx ;                       \+        addq    %rdx, %r8 ;                        \+        adcq    %rcx, %r9 ;                        \+        adcq    %rcx, %r10 ;                       \+        adcq    %rcx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++// Add 5-digit inputs and normalize to 4 digits++#define add5_4(P0,P1,P2)                        \+        movq    P1, %r8 ;                       \+        addq    P2, %r8 ;                       \+        movq    8+P1, %r9 ;                     \+        adcq    8+P2, %r9 ;                     \+        movq    16+P1, %r10 ;                   \+        adcq    16+P2, %r10 ;                   \+        movq    24+P1, %r11 ;                   \+        adcq    24+P2, %r11 ;                   \+        movq    32+P1, %r12 ;                   \+        adcq    32+P2, %r12 ;                   \+        xorl    %ebx, %ebx ;                       \+        shldq  $0x1, %r11, %r12 ;                   \+        btr    $0x3f, %r11 ;                       \+        movl   $0x13, %edx ;                       \+        imulq  %r12, %rdx ;                        \+        addq   %rdx, %r8 ;                         \+        adcq   %rbx, %r9 ;                         \+        adcq   %rbx, %r10 ;                        \+        adcq   %rbx, %r11 ;                        \+        movq   %r8, P0 ;                        \+        movq   %r9, 0x8+P0 ;                    \+        movq   %r10, 0x10+P0 ;                  \+        movq   %r11, 0x18+P0++// Modular addition with double modulus 2 * p_25519 = 2^256 - 38.+// This only ensures that the result fits in 4 digits, not that it is reduced+// even w.r.t. double modulus. The result is always correct modulo provided+// the sum of the inputs is < 2^256 + 2^256 - 38, so in particular provided+// at least one of them is reduced double modulo.++#define add_twice4(P0,P1,P2)                    \+        movq    P1, %r8 ;                       \+        xorl    %ecx, %ecx ;                       \+        addq    P2, %r8 ;                       \+        movq    0x8+P1, %r9 ;                   \+        adcq    0x8+P2, %r9 ;                   \+        movq    0x10+P1, %r10 ;                 \+        adcq    0x10+P2, %r10 ;                 \+        movq    0x18+P1, %r11 ;                 \+        adcq    0x18+P2, %r11 ;                 \+        movl    $38, %eax ;                        \+        cmovncq %rcx, %rax ;                       \+        addq    %rax, %r8 ;                        \+        adcq    %rcx, %r9 ;                        \+        adcq    %rcx, %r10 ;                       \+        adcq    %rcx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++// Modular subtraction with double modulus 2 * p_25519 = 2^256 - 38++#define sub_twice4(P0,P1,P2)                    \+        movq    P1, %r8 ;                       \+        xorl    %ebx, %ebx ;                       \+        subq    P2, %r8 ;                       \+        movq    8+P1, %r9 ;                     \+        sbbq    8+P2, %r9 ;                     \+        movl    $38, %ecx ;                        \+        movq    16+P1, %r10 ;                   \+        sbbq    16+P2, %r10 ;                   \+        movq    24+P1, %rax ;                   \+        sbbq    24+P2, %rax ;                   \+        cmovncq %rbx, %rcx ;                       \+        subq    %rcx, %r8 ;                        \+        sbbq    %rbx, %r9 ;                        \+        sbbq    %rbx, %r10 ;                       \+        sbbq    %rbx, %rax ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 8+P0 ;                     \+        movq    %r10, 16+P0 ;                   \+        movq    %rax, 24+P0++// 5-digit subtraction with upward bias to make it positive, adding+// 1000 * (2^255 - 19) = 2^256 * 500 - 19000, then normalizing to 4 digits++#define sub5_4(P0,P1,P2)                        \+        movq    P1, %r8 ;                       \+        subq    P2, %r8 ;                       \+        movq    8+P1, %r9 ;                     \+        sbbq    8+P2, %r9 ;                     \+        movq    16+P1, %r10 ;                   \+        sbbq    16+P2, %r10 ;                   \+        movq    24+P1, %r11 ;                   \+        sbbq    24+P2, %r11 ;                   \+        movq    32+P1, %r12 ;                   \+        sbbq    32+P2, %r12 ;                   \+        xorl    %ebx, %ebx ;                       \+        subq    $19000, %r8 ;                      \+        sbbq    %rbx, %r9 ;                        \+        sbbq    %rbx, %r10 ;                       \+        sbbq    %rbx, %r11 ;                       \+        sbbq    %rbx, %r12 ;                       \+        addq    $500, %r12 ;                       \+        shldq   $0x1, %r11, %r12 ;                  \+        btr     $0x3f, %r11 ;                      \+        movl    $0x13, %edx ;                      \+        imulq   %r12, %rdx ;                       \+        addq    %rdx, %r8 ;                        \+        adcq    %rbx, %r9 ;                        \+        adcq    %rbx, %r10 ;                       \+        adcq    %rbx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++// Combined z = c * x + y with reduction only < 2 * p_25519+// It is assumed that 19 * (c * x + y) < 2^60 * 2^256 so we+// don't need a high mul in the final part.++#define cmadd_4(P0,C1,P2,P3)                    \+        movq    $C1, %rsi ;                         \+        movq    P2, %rax ;                       \+        mulq    %rsi;                            \+        movq    %rax, %r8 ;                         \+        movq    %rdx, %r9 ;                         \+        movq    0x8+P2, %rax ;                   \+        xorq    %r10, %r10 ;                        \+        mulq    %rsi;                            \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        movq    0x10+P2, %rax ;                  \+        mulq    %rsi;                            \+        addq    %rax, %r10 ;                        \+        adcq    $0x0, %rdx ;                        \+        movq    0x18+P2, %rax ;                  \+        movq    %rdx, %r11 ;                        \+        mulq    %rsi;                            \+        xorl    %esi, %esi ;                        \+        addq    %rax, %r11 ;                        \+        adcq    %rsi, %rdx ;                        \+        addq    P3, %r8 ;                        \+        adcq    0x8+P3, %r9 ;                    \+        adcq    0x10+P3, %r10 ;                  \+        adcq    0x18+P3, %r11 ;                  \+        adcq    %rsi, %rdx ;                        \+        shldq   $0x1, %r11, %rdx ;                  \+        btr     $63, %r11 ;                        \+        movl    $0x13, %ebx ;                      \+        imulq   %rbx, %rdx ;                       \+        addq    %rdx, %r8 ;                        \+        adcq    %rsi, %r9 ;                        \+        adcq    %rsi, %r10 ;                       \+        adcq    %rsi, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++// Multiplex: z := if NZ then x else y++#define mux_4(P0,P1,P2)                         \+        movq    P1, %rax ;                      \+        movq    P2, %rcx ;                      \+        cmovzq  %rcx, %rax ;                       \+        movq    %rax, P0 ;                      \+        movq    8+P1, %rax ;                    \+        movq    8+P2, %rcx ;                    \+        cmovzq  %rcx, %rax ;                       \+        movq    %rax, 8+P0 ;                    \+        movq    16+P1, %rax ;                   \+        movq    16+P2, %rcx ;                   \+        cmovzq  %rcx, %rax ;                       \+        movq    %rax, 16+P0 ;                   \+        movq    24+P1, %rax ;                   \+        movq    24+P2, %rcx ;                   \+        cmovzq  %rcx, %rax ;                       \+        movq    %rax, 24+P0++S2N_BN_SYMBOL(curve25519_x25519_alt):+S2N_BN_SYMBOL(curve25519_x25519_byte_alt):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+#endif++// Save registers, make room for temps, preserve input arguments.++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(NSPACE)++// Move the output pointer to a stable place++        movq    %rdi, res++// Copy the inputs to the local variables with minimal mangling:+//+//  - The scalar is in principle turned into 01xxx...xxx000 but+//    in the structure below the special handling of these bits is+//    explicit in the main computation; the scalar is just copied.+//+//  - The point x coord is reduced mod 2^255 by masking off the+//    top bit. In the main loop we only need reduction < 2 * p_25519.++        movq    (%rsi), %rax+        movq    %rax, (%rsp)+        movq    8(%rsi), %rax+        movq    %rax, 8(%rsp)+        movq    16(%rsi), %rax+        movq    %rax, 16(%rsp)+        movq    24(%rsi), %rax+        movq    %rax, 24(%rsp)++        movq    (%rdx), %r8+        movq    8(%rdx), %r9+        movq    16(%rdx), %r10+        movq    24(%rdx), %r11+        btr     $63, %r11+        movq    %r8, 32(%rsp)+        movq    %r9, 40(%rsp)+        movq    %r10, 48(%rsp)+        movq    %r11, 56(%rsp)++// Initialize with explicit doubling in order to handle set bit 254.+// Set swap = 1 and (xm,zm) = (x,1) then double as (xn,zn) = 2 * (x,1).+// We use the fact that the point x coordinate is still in registers.+// Since zm = 1 we could do the doubling with an operation count of+// 2 * S + M instead of 2 * S + 2 * M, but it doesn't seem worth+// the slight complication arising from a different linear combination.++        movl    $1, %eax+        movq    %rax, swap+        movq    %r8, 256(%rsp)+        movq    %rax, 96(%rsp)+        xorl    %eax, %eax+        movq    %r9, 264(%rsp)+        movq    %rax, 104(%rsp)+        movq    %r10, 272(%rsp)+        movq    %rax, 112(%rsp)+        movq    %r11, 280(%rsp)+        movq    %rax, 120(%rsp)++        sub_twice4(d,xm,zm)+        add_twice4(s,xm,zm)+        sqr_4(d,d)+        sqr_4(s,s)+        sub_twice4(p,s,d)+        cmadd_4(e,0x1db42,p,d)+        mul_4(xn,s,d)+        mul_4(zn,p,e)++// The main loop over unmodified bits from i = 253, ..., i = 3 (inclusive).+// This is a classic Montgomery ladder, with the main coordinates only+// reduced mod 2 * p_25519, some intermediate results even more loosely.++        movl    $253, %eax+        movq    %rax, i++Lcurve25519_x25519_alt_scalarloop:++// sm = xm + zm; sn = xn + zn; dm = xm - zm; dn = xn - zn++        sub_twice4(dm,xm,zm)+        add_twice4(sn,xn,zn)+        sub_twice4(dn,xn,zn)+        add_twice4(sm,xm,zm)++// DOUBLING: mux d = xt - zt and s = xt + zt for appropriate choice of (xt,zt)++        movq    i, %rdx+        movq    %rdx, %rcx+        shrq    $6, %rdx+        movq    (%rsp,%rdx,8), %rdx+        shrq    %cl, %rdx+        andq    $1, %rdx+        cmpq    swap, %rdx+        movq    %rdx, swap+        mux_4(d,dm,dn)+        mux_4(s,sm,sn)++// ADDING: dmsn = dm * sn; dnsm = sm * dn++        mul_5(dnsm,sm,dn)+        mul_5(dmsn,sn,dm)++// DOUBLING: d = (xt - zt)^2++        sqr_4(d,d)++// ADDING: dpro = (dmsn - dnsm)^2, spro = (dmsn + dnsm)^2+// DOUBLING: s = (xt + zt)^2++        sub5_4(dpro,dmsn,dnsm)+        add5_4(spro,dmsn,dnsm)+        sqr_4(s,s)+        sqr_4(dpro,dpro)++// DOUBLING: p = 4 * xt * zt = s - d++        sub_twice4(p,s,d)++// ADDING: xm' = (dmsn + dnsm)^2++        sqr_4(xm,spro)++// DOUBLING: e = 121666 * p + d++        cmadd_4(e,0x1db42,p,d)++// DOUBLING: xn' = (xt + zt)^2 * (xt - zt)^2 = s * d++        mul_4(xn,s,d)++// DOUBLING: zn' = (4 * xt * zt) * ((xt - zt)^2 + 121666 * (4 * xt * zt))+//               = p * (d + 121666 * p)++        mul_4(zn,p,e)++// ADDING: zm' = x * (dmsn - dnsm)^2++        mul_4(zm,dpro,pointx)++// Loop down as far as 3 (inclusive)++        movq    i, %rax+        subq    $1, %rax+        movq    %rax, i+        cmpq    $3, %rax+        jnc     Lcurve25519_x25519_alt_scalarloop++// Multiplex directly into (xn,zn) then do three pure doubling steps;+// this accounts for the implicit zeroing of the three lowest bits+// of the scalar.++        movq    swap, %rdx+        testq   %rdx, %rdx+        mux_4(xn,xm,xn)+        mux_4(zn,zm,zn)++        sub_twice4(d,xn,zn)+        add_twice4(s,xn,zn)+        sqr_4(d,d)+        sqr_4(s,s)+        sub_twice4(p,s,d)+        cmadd_4(e,0x1db42,p,d)+        mul_4(xn,s,d)+        mul_4(zn,p,e)++        sub_twice4(d,xn,zn)+        add_twice4(s,xn,zn)+        sqr_4(d,d)+        sqr_4(s,s)+        sub_twice4(p,s,d)+        cmadd_4(e,0x1db42,p,d)+        mul_4(xn,s,d)+        mul_4(zn,p,e)++        sub_twice4(d,xn,zn)+        add_twice4(s,xn,zn)+        sqr_4(d,d)+        sqr_4(s,s)+        sub_twice4(p,s,d)+        cmadd_4(e,0x1db42,p,d)+        mul_4(xn,s,d)+        mul_4(zn,p,e)++// The projective result of the scalar multiplication is now (xn,zn).+// Prepare to call the modular inverse function to get zn' = 1/zn++        leaq    224(%rsp), %rdi+        leaq    224(%rsp), %rsi++// Inline copy of bignum_inv_p25519, identical except for stripping out+// the prologue and epilogue saving and restoring registers and making+// and reclaiming room on the stack. For more details and explanations see+// "x86/curve25519/bignum_inv_p25519.S". Note that the stack it uses for+// its own temporaries is 208 bytes, so it has no effect on variables+// that are needed in the rest of our computation here: res, xn and zn.++        movq    %rdi, 0xc0(%rsp)+        xorl    %eax, %eax+        leaq    -0x13(%rax), %rcx+        notq    %rax+        movq    %rcx, (%rsp)+        movq    %rax, 0x8(%rsp)+        movq    %rax, 0x10(%rsp)+        btr     $0x3f, %rax+        movq    %rax, 0x18(%rsp)+        movq    (%rsi), %rdx+        movq    0x8(%rsi), %rcx+        movq    0x10(%rsi), %r8+        movq    0x18(%rsi), %r9+        movl    $0x1, %eax+        xorl    %r10d, %r10d+        bts     $0x3f, %r9+        adcq    %r10, %rax+        imulq   $0x13, %rax, %rax+        addq    %rax, %rdx+        adcq    %r10, %rcx+        adcq    %r10, %r8+        adcq    %r10, %r9+        movl    $0x13, %eax+        cmovbq  %r10, %rax+        subq    %rax, %rdx+        sbbq    %r10, %rcx+        sbbq    %r10, %r8+        sbbq    %r10, %r9+        btr     $0x3f, %r9+        movq    %rdx, 0x20(%rsp)+        movq    %rcx, 0x28(%rsp)+        movq    %r8, 0x30(%rsp)+        movq    %r9, 0x38(%rsp)+        xorl    %eax, %eax+        movq    %rax, 0x40(%rsp)+        movq    %rax, 0x48(%rsp)+        movq    %rax, 0x50(%rsp)+        movq    %rax, 0x58(%rsp)+        movabsq $0xa0f99e2375022099, %rax+        movq    %rax, 0x60(%rsp)+        movabsq $0xa8c68f3f1d132595, %rax+        movq    %rax, 0x68(%rsp)+        movabsq $0x6c6c893805ac5242, %rax+        movq    %rax, 0x70(%rsp)+        movabsq $0x276508b241770615, %rax+        movq    %rax, 0x78(%rsp)+        movq    $0xa,  0x90(%rsp)+        movq    $0x1,  0x98(%rsp)+        jmp     Lcurve25519_x25519_alt_midloop+Lcurve25519_x25519_alt_inverseloop:+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %rdi+        andq    %r11, %rdi+        addq    %rax, %rdi+        movq    %rdi, 0x80(%rsp)+        movq    %r12, %rax+        andq    %r13, %rax+        movq    %r14, %rsi+        andq    %r15, %rsi+        addq    %rax, %rsi+        movq    %rsi, 0x88(%rsp)+        xorl    %ebx, %ebx+        movq    (%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x20(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    (%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x20(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        xorl    %ecx, %ecx+        movq    0x8(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x28(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, (%rsp)+        xorl    %edi, %edi+        movq    0x8(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    0x28(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $0x3b, %rbp, %rsi+        movq    %rsi, 0x20(%rsp)+        xorl    %esi, %esi+        movq    0x10(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    0x30(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rcx, %rbx+        movq    %rbx, 0x8(%rsp)+        xorl    %ebx, %ebx+        movq    0x10(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x30(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $0x3b, %rdi, %rbp+        movq    %rbp, 0x28(%rsp)+        movq    0x18(%rsp), %rax+        xorq    %r9, %rax+        movq    %rax, %rbp+        sarq    $0x3f, %rbp+        andq    %r8, %rbp+        negq    %rbp+        mulq    %r8+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x38(%rsp), %rax+        xorq    %r11, %rax+        movq    %rax, %rdx+        sarq    $0x3f, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbp+        mulq    %r10+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        shrdq   $0x3b, %rsi, %rcx+        movq    %rcx, 0x10(%rsp)+        shrdq   $0x3b, %rbp, %rsi+        movq    0x18(%rsp), %rax+        movq    %rsi, 0x18(%rsp)+        xorq    %r13, %rax+        movq    %rax, %rsi+        sarq    $0x3f, %rsi+        andq    %r12, %rsi+        negq    %rsi+        mulq    %r12+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        movq    0x38(%rsp), %rax+        xorq    %r15, %rax+        movq    %rax, %rdx+        sarq    $0x3f, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rsi+        mulq    %r14+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, 0x30(%rsp)+        shrdq   $0x3b, %rsi, %rbx+        movq    %rbx, 0x38(%rsp)+        movq    0x80(%rsp), %rbx+        movq    0x88(%rsp), %rbp+        xorl    %ecx, %ecx+        movq    0x40(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x60(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x40(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x40(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x60(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x60(%rsp)+        xorl    %ebx, %ebx+        movq    0x48(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x68(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    0x48(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, 0x48(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x68(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, 0x68(%rsp)+        xorl    %ecx, %ecx+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x70(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x50(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x50(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x70(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x70(%rsp)+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        movq    %r9, %rbx+        andq    %r8, %rbx+        negq    %rbx+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbx+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rbx, %rdx+        movq    %rdx, %rbx+        shldq   $0x1, %rcx, %rdx+        sarq    $0x3f, %rbx+        addq    %rbx, %rdx+        movl    $0x13, %eax+        imulq   %rdx+        movq    0x40(%rsp), %r8+        addq    %rax, %r8+        movq    %r8, 0x40(%rsp)+        movq    0x48(%rsp), %r8+        adcq    %rdx, %r8+        movq    %r8, 0x48(%rsp)+        movq    0x50(%rsp), %r8+        adcq    %rbx, %r8+        movq    %r8, 0x50(%rsp)+        adcq    %rbx, %rcx+        shlq    $0x3f, %rax+        addq    %rax, %rcx+        movq    0x58(%rsp), %rax+        movq    %rcx, 0x58(%rsp)+        xorq    %r13, %rax+        movq    %r13, %rcx+        andq    %r12, %rcx+        negq    %rcx+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rcx+        movq    0x78(%rsp), %rax+        xorq    %r15, %rax+        movq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rcx+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rcx, %rdx+        movq    %rdx, %rcx+        shldq   $0x1, %rsi, %rdx+        sarq    $0x3f, %rcx+        movl    $0x13, %eax+        addq    %rcx, %rdx+        imulq   %rdx+        movq    0x60(%rsp), %r8+        addq    %rax, %r8+        movq    %r8, 0x60(%rsp)+        movq    0x68(%rsp), %r8+        adcq    %rdx, %r8+        movq    %r8, 0x68(%rsp)+        movq    0x70(%rsp), %r8+        adcq    %rcx, %r8+        movq    %r8, 0x70(%rsp)+        adcq    %rcx, %rsi+        shlq    $0x3f, %rax+        addq    %rax, %rsi+        movq    %rsi, 0x78(%rsp)+Lcurve25519_x25519_alt_midloop:+        movq    0x98(%rsp), %rsi+        movq    (%rsp), %rdx+        movq    0x20(%rsp), %rcx+        movq    %rdx, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        xorl    %ebp, %ebp+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %rdx+        leaq    (%rcx,%rax), %rdi+        shlq    $0x16, %rdx+        shlq    $0x16, %rdi+        sarq    $0x2b, %rdx+        sarq    $0x2b, %rdi+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %rbx+        leaq    (%rcx,%rax), %rcx+        sarq    $0x2a, %rbx+        sarq    $0x2a, %rcx+        movq    %rdx, 0xa0(%rsp)+        movq    %rbx, 0xa8(%rsp)+        movq    %rdi, 0xb0(%rsp)+        movq    %rcx, 0xb8(%rsp)+        movq    (%rsp), %r12+        imulq   %r12, %rdi+        imulq   %rdx, %r12+        movq    0x20(%rsp), %r13+        imulq   %r13, %rbx+        imulq   %rcx, %r13+        addq    %rbx, %r12+        addq    %rdi, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r10+        shlq    $0x16, %r8+        shlq    $0x16, %r10+        sarq    $0x2b, %r8+        sarq    $0x2b, %r10+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r15+        leaq    (%rcx,%rax), %r11+        sarq    $0x2a, %r15+        sarq    $0x2a, %r11+        movq    %r13, %rbx+        movq    %r12, %rcx+        imulq   %r8, %r12+        imulq   %r15, %rbx+        addq    %rbx, %r12+        imulq   %r11, %r13+        imulq   %r10, %rcx+        addq    %rcx, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    0xa0(%rsp), %rax+        imulq   %r8, %rax+        movq    0xb0(%rsp), %rdx+        imulq   %r15, %rdx+        imulq   0xa8(%rsp), %r8+        imulq   0xb8(%rsp), %r15+        addq    %r8, %r15+        leaq    (%rax,%rdx), %r9+        movq    0xa0(%rsp), %rax+        imulq   %r10, %rax+        movq    0xb0(%rsp), %rdx+        imulq   %r11, %rdx+        imulq   0xa8(%rsp), %r10+        imulq   0xb8(%rsp), %r11+        addq    %r10, %r11+        leaq    (%rax,%rdx), %r13+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r12+        shlq    $0x15, %r8+        shlq    $0x15, %r12+        sarq    $0x2b, %r8+        sarq    $0x2b, %r12+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r10+        leaq    (%rcx,%rax), %r14+        sarq    $0x2b, %r10+        sarq    $0x2b, %r14+        movq    %r9, %rax+        imulq   %r8, %rax+        movq    %r13, %rdx+        imulq   %r10, %rdx+        imulq   %r15, %r8+        imulq   %r11, %r10+        addq    %r8, %r10+        leaq    (%rax,%rdx), %r8+        movq    %r9, %rax+        imulq   %r12, %rax+        movq    %r13, %rdx+        imulq   %r14, %rdx+        imulq   %r15, %r12+        imulq   %r11, %r14+        addq    %r12, %r14+        leaq    (%rax,%rdx), %r12+        movq    %rsi, 0x98(%rsp)+        decq     0x90(%rsp)+        jne     Lcurve25519_x25519_alt_inverseloop+        movq    (%rsp), %rax+        movq    0x20(%rsp), %rcx+        imulq   %r8, %rax+        imulq   %r10, %rcx+        addq    %rcx, %rax+        sarq    $0x3f, %rax+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        xorq    %rax, %r9+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        xorq    %rax, %r11+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        xorq    %rax, %r13+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        xorq    %rax, %r15+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %r12+        andq    %r11, %r12+        addq    %rax, %r12+        xorl    %r13d, %r13d+        movq    0x40(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x60(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movq    0x48(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r13+        adcq    %rdx, %r14+        movq    0x68(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x70(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        andq    %r8, %r9+        negq    %r9+        mulq    %r8+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %r9+        mulq    %r10+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    %r9, %rax+        shldq   $0x1, %r15, %rax+        sarq    $0x3f, %r9+        movl    $0x13, %ebx+        leaq    0x1(%rax,%r9,1), %rax+        imulq   %rbx+        xorl    %ebp, %ebp+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r9, %r14+        adcq    %r9, %r15+        shlq    $0x3f, %rax+        addq    %rax, %r15+        cmovns  %rbp, %rbx+        subq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    %rbp, %r14+        sbbq    %rbp, %r15+        btr     $0x3f, %r15+        movq    0xc0(%rsp), %rdi+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)++// Now the result is xn * (1/zn), fully reduced modulo p.+// Note that in the degenerate case zn = 0 (mod p_25519), the+// modular inverse code above will produce 1/zn = 0, giving+// the correct overall X25519 result of zero for the point at+// infinity.++        movq    res, %rbp+        mul_p25519(resx,xn,zn)++// Restore stack and registers++        CFI_INC_RSP(NSPACE)++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(curve25519_x25519_alt)+S2N_BN_SIZE_DIRECTIVE(curve25519_x25519_byte_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/curve25519_x25519base.S view
@@ -0,0 +1,9890 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// The x25519 function for curve25519 on base element 9+// Input scalar[4]; output res[4]+//+// extern void curve25519_x25519base+//   (uint64_t res[static 4], const uint64_t scalar[static 4]);+//+// The function has a second prototype considering the arguments as arrays+// of bytes rather than 64-bit words. The underlying code is the same, since+// the x86 platform is little-endian.+//+// extern void curve25519_x25519base_byte+//   (uint8_t res[static 32],const uint8_t scalar[static 32]);+//+// Given a scalar n, returns the X coordinate of n * G where G = (9,...) is+// the standard generator. The scalar is first slightly modified/mangled+// as specified in the relevant RFC (https://www.rfc-editor.org/rfc/rfc7748).+//+// Standard x86-64 ABI: RDI = res, RSI = scalar+// Microsoft x64 ABI:   RCX = res, RDX = scalar+// ----------------------------------------------------------------------------+#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(curve25519_x25519base)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(curve25519_x25519base)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(curve25519_x25519base)+        S2N_BN_SYM_VISIBILITY_DIRECTIVE(curve25519_x25519base_byte)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(curve25519_x25519base_byte)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(curve25519_x25519base_byte)++        .text+        .balign 32++// Size of individual field elements++#define NUMSIZE 32++// Pointer-offset pairs for result and temporaries on stack with some aliasing.+// The result "resx" assumes the "res" pointer has been preloaded into %rbp.++#define resx (0*NUMSIZE)(%rbp)++#define scalar (0*NUMSIZE)(%rsp)++#define tabent (1*NUMSIZE)(%rsp)+#define ymx_2 (1*NUMSIZE)(%rsp)+#define xpy_2 (2*NUMSIZE)(%rsp)+#define kxy_2 (3*NUMSIZE)(%rsp)++#define acc (4*NUMSIZE)(%rsp)+#define x_1 (4*NUMSIZE)(%rsp)+#define y_1 (5*NUMSIZE)(%rsp)+#define z_1 (6*NUMSIZE)(%rsp)+#define w_1 (7*NUMSIZE)(%rsp)+#define x_3 (4*NUMSIZE)(%rsp)+#define y_3 (5*NUMSIZE)(%rsp)+#define z_3 (6*NUMSIZE)(%rsp)+#define w_3 (7*NUMSIZE)(%rsp)++#define tmpspace (8*NUMSIZE)(%rsp)+#define t0 (8*NUMSIZE)(%rsp)+#define t1 (9*NUMSIZE)(%rsp)+#define t2 (10*NUMSIZE)(%rsp)+#define t3 (11*NUMSIZE)(%rsp)+#define t4 (12*NUMSIZE)(%rsp)+#define t5 (13*NUMSIZE)(%rsp)++// Stable homes for the input result pointer, and other variables++#define res  14*NUMSIZE(%rsp)++#define i  14*NUMSIZE+8(%rsp)++#define bias  14*NUMSIZE+16(%rsp)++#define bf  14*NUMSIZE+24(%rsp)+#define ix  14*NUMSIZE+24(%rsp)++#define tab  15*NUMSIZE(%rsp)++// Total size to reserve on the stack++#define NSPACE 488++// Macro wrapping up the basic field multiplication, only trivially+// different from a pure function call to bignum_mul_p25519.++#define mul_p25519(P0,P1,P2)                    \+        xorl   %esi, %esi ;                        \+        movq   P2, %rdx ;                       \+        mulxq  P1, %r8, %r9 ;                    \+        mulxq  0x8+P1, %rax, %r10 ;              \+        addq   %rax, %r9 ;                         \+        mulxq  0x10+P1, %rax, %r11 ;             \+        adcq   %rax, %r10 ;                        \+        mulxq  0x18+P1, %rax, %r12 ;             \+        adcq   %rax, %r11 ;                        \+        adcq   %rsi, %r12 ;                        \+        xorl   %esi, %esi ;                        \+        movq   0x8+P2, %rdx ;                   \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rbx, %r10 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x18+P1, %rax, %r13 ;             \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rsi, %r13 ;                        \+        adcxq  %rsi, %r13 ;                        \+        xorl   %esi, %esi ;                        \+        movq   0x10+P2, %rdx ;                  \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rbx, %r13 ;                        \+        mulxq  0x18+P1, %rax, %r14 ;             \+        adcxq  %rax, %r13 ;                        \+        adoxq  %rsi, %r14 ;                        \+        adcxq  %rsi, %r14 ;                        \+        xorl   %esi, %esi ;                        \+        movq   0x18+P2, %rdx ;                  \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rbx, %r13 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r13 ;                        \+        adoxq  %rbx, %r14 ;                        \+        mulxq  0x18+P1, %rax, %r15 ;             \+        adcxq  %rax, %r14 ;                        \+        adoxq  %rsi, %r15 ;                        \+        adcxq  %rsi, %r15 ;                        \+        movl   $0x26, %edx ;                       \+        xorl   %esi, %esi ;                        \+        mulxq  %r12, %rax, %rbx ;                   \+        adcxq  %rax, %r8 ;                         \+        adoxq  %rbx, %r9 ;                         \+        mulxq  %r13, %rax, %rbx ;                   \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rbx, %r10 ;                        \+        mulxq  %r14, %rax, %rbx ;                   \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  %r15, %rax, %r12 ;                   \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rsi, %r12 ;                        \+        adcxq  %rsi, %r12 ;                        \+        shldq  $0x1, %r11, %r12 ;                   \+        movl   $0x13, %edx ;                       \+        incq   %r12;                             \+        bts    $63, %r11 ;                         \+        mulxq  %r12, %rax, %rbx ;                   \+        addq   %rax, %r8 ;                         \+        adcq   %rbx, %r9 ;                         \+        adcq   %rsi, %r10 ;                        \+        adcq   %rsi, %r11 ;                        \+        sbbq   %rax, %rax ;                        \+        notq   %rax;                             \+        andq   %rdx, %rax ;                        \+        subq   %rax, %r8 ;                         \+        sbbq   %rsi, %r9 ;                         \+        sbbq   %rsi, %r10 ;                        \+        sbbq   %rsi, %r11 ;                        \+        btr    $63, %r11 ;                         \+        movq   %r8, P0 ;                        \+        movq   %r9, 0x8+P0 ;                    \+        movq   %r10, 0x10+P0 ;                  \+        movq   %r11, 0x18+P0++// A version of multiplication that only guarantees output < 2 * p_25519.+// This basically skips the +1 and final correction in quotient estimation.++#define mul_4(P0,P1,P2)                         \+        xorl   %ecx, %ecx ;                        \+        movq   P2, %rdx ;                       \+        mulxq  P1, %r8, %r9 ;                    \+        mulxq  0x8+P1, %rax, %r10 ;              \+        addq   %rax, %r9 ;                         \+        mulxq  0x10+P1, %rax, %r11 ;             \+        adcq   %rax, %r10 ;                        \+        mulxq  0x18+P1, %rax, %r12 ;             \+        adcq   %rax, %r11 ;                        \+        adcq   %rcx, %r12 ;                        \+        xorl   %ecx, %ecx ;                        \+        movq   0x8+P2, %rdx ;                   \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rbx, %r10 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x18+P1, %rax, %r13 ;             \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rcx, %r13 ;                        \+        adcxq  %rcx, %r13 ;                        \+        xorl   %ecx, %ecx ;                        \+        movq   0x10+P2, %rdx ;                  \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rbx, %r13 ;                        \+        mulxq  0x18+P1, %rax, %r14 ;             \+        adcxq  %rax, %r13 ;                        \+        adoxq  %rcx, %r14 ;                        \+        adcxq  %rcx, %r14 ;                        \+        xorl   %ecx, %ecx ;                        \+        movq   0x18+P2, %rdx ;                  \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rbx, %r13 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r13 ;                        \+        adoxq  %rbx, %r14 ;                        \+        mulxq  0x18+P1, %rax, %r15 ;             \+        adcxq  %rax, %r14 ;                        \+        adoxq  %rcx, %r15 ;                        \+        adcxq  %rcx, %r15 ;                        \+        movl   $0x26, %edx ;                       \+        xorl   %ecx, %ecx ;                        \+        mulxq  %r12, %rax, %rbx ;                   \+        adcxq  %rax, %r8 ;                         \+        adoxq  %rbx, %r9 ;                         \+        mulxq  %r13, %rax, %rbx ;                   \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rbx, %r10 ;                        \+        mulxq  %r14, %rax, %rbx ;                   \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  %r15, %rax, %r12 ;                   \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rcx, %r12 ;                        \+        adcxq  %rcx, %r12 ;                        \+        shldq  $0x1, %r11, %r12 ;                   \+        btr    $0x3f, %r11 ;                       \+        movl   $0x13, %edx ;                       \+        imulq  %r12, %rdx ;                        \+        addq   %rdx, %r8 ;                         \+        adcq   %rcx, %r9 ;                         \+        adcq   %rcx, %r10 ;                        \+        adcq   %rcx, %r11 ;                        \+        movq   %r8, P0 ;                        \+        movq   %r9, 0x8+P0 ;                    \+        movq   %r10, 0x10+P0 ;                  \+        movq   %r11, 0x18+P0++// Modular subtraction with double modulus 2 * p_25519 = 2^256 - 38++#define sub_twice4(P0,P1,P2)                    \+        movq    P1, %r8 ;                       \+        xorl    %ebx, %ebx ;                       \+        subq    P2, %r8 ;                       \+        movq    8+P1, %r9 ;                     \+        sbbq    8+P2, %r9 ;                     \+        movl    $38, %ecx ;                        \+        movq    16+P1, %r10 ;                   \+        sbbq    16+P2, %r10 ;                   \+        movq    24+P1, %rax ;                   \+        sbbq    24+P2, %rax ;                   \+        cmovncq %rbx, %rcx ;                       \+        subq    %rcx, %r8 ;                        \+        sbbq    %rbx, %r9 ;                        \+        sbbq    %rbx, %r10 ;                       \+        sbbq    %rbx, %rax ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 8+P0 ;                     \+        movq    %r10, 16+P0 ;                   \+        movq    %rax, 24+P0++// Modular addition and doubling with double modulus 2 * p_25519 = 2^256 - 38.+// This only ensures that the result fits in 4 digits, not that it is reduced+// even w.r.t. double modulus. The result is always correct modulo provided+// the sum of the inputs is < 2^256 + 2^256 - 38, so in particular provided+// at least one of them is reduced double modulo.++#define add_twice4(P0,P1,P2)                    \+        movq    P1, %r8 ;                       \+        xorl    %ecx, %ecx ;                       \+        addq    P2, %r8 ;                       \+        movq    0x8+P1, %r9 ;                   \+        adcq    0x8+P2, %r9 ;                   \+        movq    0x10+P1, %r10 ;                 \+        adcq    0x10+P2, %r10 ;                 \+        movq    0x18+P1, %r11 ;                 \+        adcq    0x18+P2, %r11 ;                 \+        movl    $38, %eax ;                        \+        cmovncq %rcx, %rax ;                       \+        addq    %rax, %r8 ;                        \+        adcq    %rcx, %r9 ;                        \+        adcq    %rcx, %r10 ;                       \+        adcq    %rcx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++#define double_twice4(P0,P1)                    \+        movq    P1, %r8 ;                       \+        xorl    %ecx, %ecx ;                       \+        addq    %r8, %r8 ;                         \+        movq    0x8+P1, %r9 ;                   \+        adcq    %r9, %r9 ;                         \+        movq    0x10+P1, %r10 ;                 \+        adcq    %r10, %r10 ;                       \+        movq    0x18+P1, %r11 ;                 \+        adcq    %r11, %r11 ;                       \+        movl    $38, %eax ;                        \+        cmovncq %rcx, %rax ;                       \+        addq    %rax, %r8 ;                        \+        adcq    %rcx, %r9 ;                        \+        adcq    %rcx, %r10 ;                       \+        adcq    %rcx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++S2N_BN_SYMBOL(curve25519_x25519base):+S2N_BN_SYMBOL(curve25519_x25519base_byte):+        CFI_START+        _CET_ENDBR++// In this case the Windows form literally makes a subroutine call.+// This avoids hassle arising from keeping code and data together.++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        CFI_CALL(Lcurve25519_x25519base_standard)+        CFI_POP(%rsi)+        CFI_POP(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(curve25519_x25519base)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lcurve25519_x25519base_standard)++Lcurve25519_x25519base_standard:+        CFI_START+#endif++// Save registers, make room for temps, preserve input arguments.++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(NSPACE)++// Move the output pointer to a stable place++        movq    %rdi, res++// Copy the input scalar to its local variable while mangling it.+// In principle the mangling is into 01xxx...xxx000, but actually+// we only clear the top two bits so 00xxx...xxxxxx. The additional+// 2^254 * G is taken care of by the starting value for the addition+// chain below, while we never look at the three low bits at all.++        movq    (%rsi), %rax+        movq    %rax, (%rsp)+        movq    8(%rsi), %rax+        movq    %rax, 8(%rsp)+        movq    16(%rsi), %rax+        movq    %rax, 16(%rsp)+        movq    $0x3fffffffffffffff, %rax+        andq    24(%rsi), %rax+        movq    %rax, 24(%rsp)++// The main part of the computation is on the edwards25519 curve in+// extended-projective coordinates (X,Y,Z,T), representing a point+// (x,y) via x = X/Z, y = Y/Z and x * y = T/Z (so X * Y = T * Z).+// Only at the very end do we translate back to curve25519. So G+// below means the generator within edwards25519 corresponding to+// (9,...) for curve25519, via the standard isomorphism.+//+// Initialize accumulator "acc" to either (2^254 + 8) * G or just 2^254 * G+// depending on bit 3 of the scalar, the only nonzero bit of the bottom 4.+// Thus, we have effectively dealt with bits 0, 1, 2, 3, 254 and 255.++        movq    (%rsp), %rax+        andq    $8, %rax++        leaq    S2N_BN_SYMBOL(curve25519_x25519base_constant)(%rip), %r10+        leaq    8*12(%r10), %r11++        movq    (%r10), %rax+        movq    (%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*16(%rsp)++        movq    8*1(%r10), %rax+        movq    8*1(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*17(%rsp)++        movq    8*2(%r10), %rax+        movq    8*2(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*18(%rsp)++        movq    8*3(%r10), %rax+        movq    8*3(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*19(%rsp)++        movq    8*4(%r10), %rax+        movq    8*4(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*20(%rsp)++        movq    8*5(%r10), %rax+        movq    8*5(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*21(%rsp)++        movq    8*6(%r10), %rax+        movq    8*6(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*22(%rsp)++        movq    8*7(%r10), %rax+        movq    8*7(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*23(%rsp)++        movl    $1, %eax+        movq    %rax, 8*24(%rsp)+        movl    $0, %eax+        movq    %rax, 8*25(%rsp)+        movq    %rax, 8*26(%rsp)+        movq    %rax, 8*27(%rsp)++        movq    8*8(%r10), %rax+        movq    8*8(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*28(%rsp)++        movq    8*9(%r10), %rax+        movq    8*9(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*29(%rsp)++        movq    8*10(%r10), %rax+        movq    8*10(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*30(%rsp)++        movq    8*11(%r10), %rax+        movq    8*11(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*31(%rsp)++// The counter "i" tracks the bit position for which the scalar has+// already been absorbed, starting at 4 and going up in chunks of 4.+//+// The pointer "tab" points at the current block of the table for+// multiples (2^i * j) * G at the current bit position i; 1 <= j <= 8.+//+// The bias is always either 0 and 1 and needs to be added to the+// partially processed scalar implicitly. This is used to absorb 4 bits+// of scalar per iteration from 3-bit table indexing by exploiting+// negation: (16 * h + l) * G = (16 * (h + 1) - (16 - l)) * G is used+// when l >= 9. Note that we can't have any bias left over at the+// end because of the clearing of bit 255 of the scalar, meaning the+// l >= 9 case cannot arise on the last iteration.++        movq    $4, i+        leaq    8*24(%r10), %rax+        movq    %rax, tab+        movq    $0, bias++// Start of the main loop, repeated 63 times for i = 4, 8, ..., 252++Lcurve25519_x25519base_scalarloop:++// Look at the next 4-bit field "bf", adding the previous bias as well.+// Choose the table index "ix" as bf when bf <= 8 and 16 - bf for bf >= 9,+// setting the bias to 1 for the next iteration in the latter case.++        movq    i, %rax+        movq    %rax, %rcx+        shrq    $6, %rax+        movq    (%rsp,%rax,8), %rax // Exploiting scalar = sp exactly+        shrq    %cl, %rax+        andq    $15, %rax+        addq    bias, %rax+        movq    %rax, bf++        cmpq    $9, bf+        sbbq    %rax, %rax+        incq    %rax+        movq    %rax, bias++        movq    $16, %rdi+        subq    bf, %rdi+        cmpq    $0, bias+        cmovzq  bf, %rdi+        movq    %rdi, ix++// Perform constant-time lookup in the table to get element number "ix".+// The table entry for the affine point (x,y) is actually a triple+// (y - x,x + y,2 * d * x * y) to precompute parts of the addition.+// Note that "ix" can be 0, so we set up the appropriate identity first.++        movl    $1, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        movl    $1, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        xorl    %r13d, %r13d+        xorl    %r14d, %r14d+        xorl    %r15d, %r15d++        movq    tab, %rbp++        cmpq    $1, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $2, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $3, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $4, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $5, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $6, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $7, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $8, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15++        addq    $96, %rbp+        movq    %rbp, tab++// We now have the triple from the table in registers as follows+//+//      [%rdx;%rcx;%rbx;%rax] = y - x+//      [%r11;%r10;%r9;%r8] = x + y+//      [%r15;%r14;%r13;%r12] = 2 * d * x * y+//+// In case bias = 1 we need to negate this. For Edwards curves+// -(x,y) = (-x,y), i.e. we need to negate the x coordinate.+// In this processed encoding, that amounts to swapping the+// first two fields and negating the third.+//+// The optional negation here also pretends bias = 0 whenever+// ix = 0 so that it doesn't need to handle the case of zero+// inputs, since no non-trivial table entries are zero. Note+// that in the zero case the whole negation is trivial, and+// so indeed is the swapping.++        cmpq    $0, bias++        movq    %rax, %rsi+        cmovnzq %r8, %rsi+        cmovnzq %rax, %r8+        movq    %rsi, 32(%rsp)+        movq    %r8, 64(%rsp)++        movq    %rbx, %rsi+        cmovnzq %r9, %rsi+        cmovnzq %rbx, %r9+        movq    %rsi, 40(%rsp)+        movq    %r9, 72(%rsp)++        movq    %rcx, %rsi+        cmovnzq %r10, %rsi+        cmovnzq %rcx, %r10+        movq    %rsi, 48(%rsp)+        movq    %r10, 80(%rsp)++        movq    %rdx, %rsi+        cmovnzq %r11, %rsi+        cmovnzq %rdx, %r11+        movq    %rsi, 56(%rsp)+        movq    %r11, 88(%rsp)++        movq    $-19, %rax+        movq    $-1, %rbx+        movq    $-1, %rcx+        movq    $0x7fffffffffffffff, %rdx+        subq    %r12, %rax+        sbbq    %r13, %rbx+        sbbq    %r14, %rcx+        sbbq    %r15, %rdx++        movq    ix, %r8+        movq    bias, %r9+        testq   %r8, %r8+        cmovzq  %r8, %r9+        testq   %r9, %r9++        cmovzq  %r12, %rax+        cmovzq  %r13, %rbx+        cmovzq  %r14, %rcx+        cmovzq  %r15, %rdx+        movq    %rax, 96(%rsp)+        movq    %rbx, 104(%rsp)+        movq    %rcx, 112(%rsp)+        movq    %rdx, 120(%rsp)++// Extended-projective and precomputed mixed addition.+// This is effectively the same as calling the standalone+// function edwards25519_pepadd(acc,acc,tabent), but we+// only retain slightly weaker normalization < 2 * p_25519+// throughout the inner loop, so the computation is+// slightly different, and faster overall.++        double_twice4(t0,z_1)+        sub_twice4(t1,y_1,x_1)+        add_twice4(t2,y_1,x_1)+        mul_4(t3,w_1,kxy_2)+        mul_4(t1,t1,ymx_2)+        mul_4(t2,t2,xpy_2)+        sub_twice4(t4,t0,t3)+        add_twice4(t0,t0,t3)+        sub_twice4(t5,t2,t1)+        add_twice4(t1,t2,t1)+        mul_4(z_3,t4,t0)+        mul_4(x_3,t5,t4)+        mul_4(y_3,t0,t1)+        mul_4(w_3,t5,t1)++// End of the main loop; move on by 4 bits.++        addq    $4, i+        cmpq    $256, i+        jc      Lcurve25519_x25519base_scalarloop++// Now we need to translate from Edwards curve edwards25519 back+// to the Montgomery form curve25519. The mapping in the affine+// representations is+//+// (x,y) |-> ((1 + y) / (1 - y), c * (1 + y) / ((1 - y) * x))+//+// For x25519, we only need the x coordinate, and we compute this as+//+// (1 + y) / (1 - y) = (x + x * y) / (x - x * y)+//                   = (X/Z + T/Z) / (X/Z - T/Z)+//                   = (X + T) / (X - T)+//                   = (X + T) * inverse(X - T)+//+// We could equally well use (Z + Y) / (Z - Y), but the above has the+// same cost, and it more explicitly forces zero output whenever X = 0,+// regardless of how the modular inverse behaves on zero inputs. In+// the present setting (base point 9, mangled scalar) that doesn't+// really matter anyway since X = 0 never arises, but it seems a+// little bit tidier. Note that both Edwards point (0,1) which maps to+// the Montgomery point at infinity, and Edwards (0,-1) which maps to+// Montgomery (0,0) [this is the 2-torsion point] are both by definition+// mapped to 0 by the X coordinate mapping used to define curve25519.+//+// First the addition and subtraction:++        add_twice4(t1,x_3,w_3)+        sub_twice4(t2,x_3,w_3)++// Prepare to call the modular inverse function to get t0 = 1/t2+// Note that this works for the weakly normalized z_3 equally well.+// The non-coprime case z_3 == 0 (mod p_25519) cannot arise anyway.++        leaq    256(%rsp), %rdi+        leaq    320(%rsp), %rsi++// Inline copy of bignum_inv_p25519, identical except for stripping out+// the prologue and epilogue saving and restoring registers and making+// and reclaiming room on the stack. For more details and explanations see+// "x86/curve25519/bignum_inv_p25519.S". Note that the stack it uses for+// its own temporaries is 208 bytes, so it has no effect on variables+// that are needed in the rest of our computation here: res, t0, t1, t2.++        movq    %rdi, 0xc0(%rsp)+        xorl    %eax, %eax+        leaq    -0x13(%rax), %rcx+        notq    %rax+        movq    %rcx, (%rsp)+        movq    %rax, 0x8(%rsp)+        movq    %rax, 0x10(%rsp)+        btr     $0x3f, %rax+        movq    %rax, 0x18(%rsp)+        movq    (%rsi), %rdx+        movq    0x8(%rsi), %rcx+        movq    0x10(%rsi), %r8+        movq    0x18(%rsi), %r9+        movl    $0x1, %eax+        xorl    %r10d, %r10d+        bts     $0x3f, %r9+        adcq    %r10, %rax+        imulq   $0x13, %rax, %rax+        addq    %rax, %rdx+        adcq    %r10, %rcx+        adcq    %r10, %r8+        adcq    %r10, %r9+        movl    $0x13, %eax+        cmovbq  %r10, %rax+        subq    %rax, %rdx+        sbbq    %r10, %rcx+        sbbq    %r10, %r8+        sbbq    %r10, %r9+        btr     $0x3f, %r9+        movq    %rdx, 0x20(%rsp)+        movq    %rcx, 0x28(%rsp)+        movq    %r8, 0x30(%rsp)+        movq    %r9, 0x38(%rsp)+        xorl    %eax, %eax+        movq    %rax, 0x40(%rsp)+        movq    %rax, 0x48(%rsp)+        movq    %rax, 0x50(%rsp)+        movq    %rax, 0x58(%rsp)+        movabsq $0xa0f99e2375022099, %rax+        movq    %rax, 0x60(%rsp)+        movabsq $0xa8c68f3f1d132595, %rax+        movq    %rax, 0x68(%rsp)+        movabsq $0x6c6c893805ac5242, %rax+        movq    %rax, 0x70(%rsp)+        movabsq $0x276508b241770615, %rax+        movq    %rax, 0x78(%rsp)+        movq    $0xa,  0x90(%rsp)+        movq    $0x1,  0x98(%rsp)+        jmp     Lcurve25519_x25519base_midloop+Lcurve25519_x25519base_inverseloop:+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %rdi+        andq    %r11, %rdi+        addq    %rax, %rdi+        movq    %rdi, 0x80(%rsp)+        movq    %r12, %rax+        andq    %r13, %rax+        movq    %r14, %rsi+        andq    %r15, %rsi+        addq    %rax, %rsi+        movq    %rsi, 0x88(%rsp)+        xorl    %ebx, %ebx+        movq    (%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x20(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    (%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x20(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        xorl    %ecx, %ecx+        movq    0x8(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x28(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, (%rsp)+        xorl    %edi, %edi+        movq    0x8(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    0x28(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $0x3b, %rbp, %rsi+        movq    %rsi, 0x20(%rsp)+        xorl    %esi, %esi+        movq    0x10(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    0x30(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rcx, %rbx+        movq    %rbx, 0x8(%rsp)+        xorl    %ebx, %ebx+        movq    0x10(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x30(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $0x3b, %rdi, %rbp+        movq    %rbp, 0x28(%rsp)+        movq    0x18(%rsp), %rax+        xorq    %r9, %rax+        movq    %rax, %rbp+        sarq    $0x3f, %rbp+        andq    %r8, %rbp+        negq    %rbp+        mulq    %r8+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x38(%rsp), %rax+        xorq    %r11, %rax+        movq    %rax, %rdx+        sarq    $0x3f, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbp+        mulq    %r10+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        shrdq   $0x3b, %rsi, %rcx+        movq    %rcx, 0x10(%rsp)+        shrdq   $0x3b, %rbp, %rsi+        movq    0x18(%rsp), %rax+        movq    %rsi, 0x18(%rsp)+        xorq    %r13, %rax+        movq    %rax, %rsi+        sarq    $0x3f, %rsi+        andq    %r12, %rsi+        negq    %rsi+        mulq    %r12+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        movq    0x38(%rsp), %rax+        xorq    %r15, %rax+        movq    %rax, %rdx+        sarq    $0x3f, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rsi+        mulq    %r14+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, 0x30(%rsp)+        shrdq   $0x3b, %rsi, %rbx+        movq    %rbx, 0x38(%rsp)+        movq    0x80(%rsp), %rbx+        movq    0x88(%rsp), %rbp+        xorl    %ecx, %ecx+        movq    0x40(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x60(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x40(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x40(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x60(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x60(%rsp)+        xorl    %ebx, %ebx+        movq    0x48(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x68(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    0x48(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, 0x48(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x68(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, 0x68(%rsp)+        xorl    %ecx, %ecx+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x70(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x50(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x50(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x70(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x70(%rsp)+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        movq    %r9, %rbx+        andq    %r8, %rbx+        negq    %rbx+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbx+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rbx, %rdx+        movq    %rdx, %rbx+        shldq   $0x1, %rcx, %rdx+        sarq    $0x3f, %rbx+        addq    %rbx, %rdx+        movl    $0x13, %eax+        imulq   %rdx+        movq    0x40(%rsp), %r8+        addq    %rax, %r8+        movq    %r8, 0x40(%rsp)+        movq    0x48(%rsp), %r8+        adcq    %rdx, %r8+        movq    %r8, 0x48(%rsp)+        movq    0x50(%rsp), %r8+        adcq    %rbx, %r8+        movq    %r8, 0x50(%rsp)+        adcq    %rbx, %rcx+        shlq    $0x3f, %rax+        addq    %rax, %rcx+        movq    0x58(%rsp), %rax+        movq    %rcx, 0x58(%rsp)+        xorq    %r13, %rax+        movq    %r13, %rcx+        andq    %r12, %rcx+        negq    %rcx+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rcx+        movq    0x78(%rsp), %rax+        xorq    %r15, %rax+        movq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rcx+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rcx, %rdx+        movq    %rdx, %rcx+        shldq   $0x1, %rsi, %rdx+        sarq    $0x3f, %rcx+        movl    $0x13, %eax+        addq    %rcx, %rdx+        imulq   %rdx+        movq    0x60(%rsp), %r8+        addq    %rax, %r8+        movq    %r8, 0x60(%rsp)+        movq    0x68(%rsp), %r8+        adcq    %rdx, %r8+        movq    %r8, 0x68(%rsp)+        movq    0x70(%rsp), %r8+        adcq    %rcx, %r8+        movq    %r8, 0x70(%rsp)+        adcq    %rcx, %rsi+        shlq    $0x3f, %rax+        addq    %rax, %rsi+        movq    %rsi, 0x78(%rsp)+Lcurve25519_x25519base_midloop:+        movq    0x98(%rsp), %rsi+        movq    (%rsp), %rdx+        movq    0x20(%rsp), %rcx+        movq    %rdx, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        xorl    %ebp, %ebp+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %rdx+        leaq    (%rcx,%rax), %rdi+        shlq    $0x16, %rdx+        shlq    $0x16, %rdi+        sarq    $0x2b, %rdx+        sarq    $0x2b, %rdi+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %rbx+        leaq    (%rcx,%rax), %rcx+        sarq    $0x2a, %rbx+        sarq    $0x2a, %rcx+        movq    %rdx, 0xa0(%rsp)+        movq    %rbx, 0xa8(%rsp)+        movq    %rdi, 0xb0(%rsp)+        movq    %rcx, 0xb8(%rsp)+        movq    (%rsp), %r12+        imulq   %r12, %rdi+        imulq   %rdx, %r12+        movq    0x20(%rsp), %r13+        imulq   %r13, %rbx+        imulq   %rcx, %r13+        addq    %rbx, %r12+        addq    %rdi, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r10+        shlq    $0x16, %r8+        shlq    $0x16, %r10+        sarq    $0x2b, %r8+        sarq    $0x2b, %r10+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r15+        leaq    (%rcx,%rax), %r11+        sarq    $0x2a, %r15+        sarq    $0x2a, %r11+        movq    %r13, %rbx+        movq    %r12, %rcx+        imulq   %r8, %r12+        imulq   %r15, %rbx+        addq    %rbx, %r12+        imulq   %r11, %r13+        imulq   %r10, %rcx+        addq    %rcx, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    0xa0(%rsp), %rax+        imulq   %r8, %rax+        movq    0xb0(%rsp), %rdx+        imulq   %r15, %rdx+        imulq   0xa8(%rsp), %r8+        imulq   0xb8(%rsp), %r15+        addq    %r8, %r15+        leaq    (%rax,%rdx), %r9+        movq    0xa0(%rsp), %rax+        imulq   %r10, %rax+        movq    0xb0(%rsp), %rdx+        imulq   %r11, %rdx+        imulq   0xa8(%rsp), %r10+        imulq   0xb8(%rsp), %r11+        addq    %r10, %r11+        leaq    (%rax,%rdx), %r13+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r12+        shlq    $0x15, %r8+        shlq    $0x15, %r12+        sarq    $0x2b, %r8+        sarq    $0x2b, %r12+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r10+        leaq    (%rcx,%rax), %r14+        sarq    $0x2b, %r10+        sarq    $0x2b, %r14+        movq    %r9, %rax+        imulq   %r8, %rax+        movq    %r13, %rdx+        imulq   %r10, %rdx+        imulq   %r15, %r8+        imulq   %r11, %r10+        addq    %r8, %r10+        leaq    (%rax,%rdx), %r8+        movq    %r9, %rax+        imulq   %r12, %rax+        movq    %r13, %rdx+        imulq   %r14, %rdx+        imulq   %r15, %r12+        imulq   %r11, %r14+        addq    %r12, %r14+        leaq    (%rax,%rdx), %r12+        movq    %rsi, 0x98(%rsp)+        decq     0x90(%rsp)+        jne     Lcurve25519_x25519base_inverseloop+        movq    (%rsp), %rax+        movq    0x20(%rsp), %rcx+        imulq   %r8, %rax+        imulq   %r10, %rcx+        addq    %rcx, %rax+        sarq    $0x3f, %rax+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        xorq    %rax, %r9+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        xorq    %rax, %r11+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        xorq    %rax, %r13+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        xorq    %rax, %r15+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %r12+        andq    %r11, %r12+        addq    %rax, %r12+        xorl    %r13d, %r13d+        movq    0x40(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x60(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movq    0x48(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r13+        adcq    %rdx, %r14+        movq    0x68(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x70(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        andq    %r8, %r9+        negq    %r9+        mulq    %r8+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %r9+        mulq    %r10+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    %r9, %rax+        shldq   $0x1, %r15, %rax+        sarq    $0x3f, %r9+        movl    $0x13, %ebx+        leaq    0x1(%rax,%r9,1), %rax+        imulq   %rbx+        xorl    %ebp, %ebp+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r9, %r14+        adcq    %r9, %r15+        shlq    $0x3f, %rax+        addq    %rax, %r15+        cmovns  %rbp, %rbx+        subq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    %rbp, %r14+        sbbq    %rbp, %r15+        btr     $0x3f, %r15+        movq    0xc0(%rsp), %rdi+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)++// The final result is (X + T) / (X - T)+// This is the only operation in the whole computation that+// fully reduces modulo p_25519 since now we want the canonical+// answer as output.++        movq    res, %rbp+        mul_p25519(resx,t1,t0)++// Restore stack and registers++        CFI_INC_RSP(NSPACE)++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++#if WINDOWS_ABI+S2N_BN_SIZE_DIRECTIVE(Lcurve25519_x25519base_standard)+#else+S2N_BN_SIZE_DIRECTIVE(curve25519_x25519base)+#endif++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif++// ****************************************************************************+// The precomputed data (all read-only).+// ****************************************************************************++#if defined(__ELF__)+.section .rodata+.type S2N_BN_SYMBOL(curve25519_x25519base_constant), %object+.size S2N_BN_SYMBOL(curve25519_x25519base_constant), 48576+#elif defined(__APPLE__)+.const_data+#endif++S2N_BN_SYMBOL(curve25519_x25519base_constant):++// 2^254 * G and (2^254 + 8) * G in extended-projective coordinates+// but with z = 1 assumed and hence left out, so they are (X,Y,T) only.++        .quad   0x251037f7cf4e861d+        .quad   0x10ede0fb19fb128f+        .quad   0x96c033b175f5e2c8+        .quad   0x055f070d6c15fb0d++        .quad   0x7c52af2c97473e69+        .quad   0x022f82391bad8378+        .quad   0x9991e1b02adb476f+        .quad   0x511144a03a99b855++        .quad   0x5fafc3b88ff2e4ae+        .quad   0x855e4ff0de1230ff+        .quad   0x72e302a348492870+        .quad   0x1253c19e53dbe1bc+++        .quad   0x331d086e0d9abcaa+        .quad   0x1e23c96d311a10c9+        .quad   0x96d0f95e58c13478+        .quad   0x2f72f7384fcfcc59++        .quad   0x39a6cd1cfd7d87c9+        .quad   0x9867a0abd8ae153a+        .quad   0xa49d2a5f35986745+        .quad   0x57012940cdfe82e1++        .quad   0x5046a6532ec5544a+        .quad   0x6d674004739ff6c9+        .quad   0x9bbaa44b234a70e3+        .quad   0x5e6d8901138cf386++// Precomputed table of multiples of generator for edwards25519+// all in precomputed extended-projective (y-x,x+y,2*d*x*y) triples.++        // 2^4 * 1 * G++        .quad   0x7ec851ca553e2df3+        .quad   0xa71284cba64878b3+        .quad   0xe6b5e4193288d1e7+        .quad   0x4cf210ec5a9a8883+        .quad   0x322d04a52d9021f6+        .quad   0xb9c19f3375c6bf9c+        .quad   0x587a3a4342d20b09+        .quad   0x143b1cf8aa64fe61+        .quad   0x9f867c7d968acaab+        .quad   0x5f54258e27092729+        .quad   0xd0a7d34bea180975+        .quad   0x21b546a3374126e1++        // 2^4 * 2 * G++        .quad   0xa94ff858a2888343+        .quad   0xce0ed4565313ed3c+        .quad   0xf55c3dcfb5bf34fa+        .quad   0x0a653ca5c9eab371+        .quad   0x490a7a45d185218f+        .quad   0x9a15377846049335+        .quad   0x0060ea09cc31e1f6+        .quad   0x7e041577f86ee965+        .quad   0x66b2a496ce5b67f3+        .quad   0xff5492d8bd569796+        .quad   0x503cec294a592cd0+        .quad   0x566943650813acb2++        // 2^4 * 3 * G++        .quad   0xb818db0c26620798+        .quad   0x5d5c31d9606e354a+        .quad   0x0982fa4f00a8cdc7+        .quad   0x17e12bcd4653e2d4+        .quad   0x5672f9eb1dabb69d+        .quad   0xba70b535afe853fc+        .quad   0x47ac0f752796d66d+        .quad   0x32a5351794117275+        .quad   0xd3a644a6df648437+        .quad   0x703b6559880fbfdd+        .quad   0xcb852540ad3a1aa5+        .quad   0x0900b3f78e4c6468++        // 2^4 * 4 * G++        .quad   0x0a851b9f679d651b+        .quad   0xe108cb61033342f2+        .quad   0xd601f57fe88b30a3+        .quad   0x371f3acaed2dd714+        .quad   0xed280fbec816ad31+        .quad   0x52d9595bd8e6efe3+        .quad   0x0fe71772f6c623f5+        .quad   0x4314030b051e293c+        .quad   0xd560005efbf0bcad+        .quad   0x8eb70f2ed1870c5e+        .quad   0x201f9033d084e6a0+        .quad   0x4c3a5ae1ce7b6670++        // 2^4 * 5 * G++        .quad   0x4138a434dcb8fa95+        .quad   0x870cf67d6c96840b+        .quad   0xde388574297be82c+        .quad   0x7c814db27262a55a+        .quad   0xbaf875e4c93da0dd+        .quad   0xb93282a771b9294d+        .quad   0x80d63fb7f4c6c460+        .quad   0x6de9c73dea66c181+        .quad   0x478904d5a04df8f2+        .quad   0xfafbae4ab10142d3+        .quad   0xf6c8ac63555d0998+        .quad   0x5aac4a412f90b104++        // 2^4 * 6 * G++        .quad   0xc64f326b3ac92908+        .quad   0x5551b282e663e1e0+        .quad   0x476b35f54a1a4b83+        .quad   0x1b9da3fe189f68c2+        .quad   0x603a0d0abd7f5134+        .quad   0x8089c932e1d3ae46+        .quad   0xdf2591398798bd63+        .quad   0x1c145cd274ba0235+        .quad   0x32e8386475f3d743+        .quad   0x365b8baf6ae5d9ef+        .quad   0x825238b6385b681e+        .quad   0x234929c1167d65e1++        // 2^4 * 7 * G++        .quad   0x984decaba077ade8+        .quad   0x383f77ad19eb389d+        .quad   0xc7ec6b7e2954d794+        .quad   0x59c77b3aeb7c3a7a+        .quad   0x48145cc21d099fcf+        .quad   0x4535c192cc28d7e5+        .quad   0x80e7c1e548247e01+        .quad   0x4a5f28743b2973ee+        .quad   0xd3add725225ccf62+        .quad   0x911a3381b2152c5d+        .quad   0xd8b39fad5b08f87d+        .quad   0x6f05606b4799fe3b++        // 2^4 * 8 * G++        .quad   0x9ffe9e92177ba962+        .quad   0x98aee71d0de5cae1+        .quad   0x3ff4ae942d831044+        .quad   0x714de12e58533ac8+        .quad   0x5b433149f91b6483+        .quad   0xadb5dc655a2cbf62+        .quad   0x87fa8412632827b3+        .quad   0x60895e91ab49f8d8+        .quad   0xe9ecf2ed0cf86c18+        .quad   0xb46d06120735dfd4+        .quad   0xbc9da09804b96be7+        .quad   0x73e2e62fd96dc26b++        // 2^8 * 1 * G++        .quad   0xed5b635449aa515e+        .quad   0xa865c49f0bc6823a+        .quad   0x850c1fe95b42d1c4+        .quad   0x30d76d6f03d315b9+        .quad   0x2eccdd0e632f9c1d+        .quad   0x51d0b69676893115+        .quad   0x52dfb76ba8637a58+        .quad   0x6dd37d49a00eef39+        .quad   0x6c4444172106e4c7+        .quad   0xfb53d680928d7f69+        .quad   0xb4739ea4694d3f26+        .quad   0x10c697112e864bb0++        // 2^8 * 2 * G++        .quad   0x6493c4277dbe5fde+        .quad   0x265d4fad19ad7ea2+        .quad   0x0e00dfc846304590+        .quad   0x25e61cabed66fe09+        .quad   0x0ca62aa08358c805+        .quad   0x6a3d4ae37a204247+        .quad   0x7464d3a63b11eddc+        .quad   0x03bf9baf550806ef+        .quad   0x3f13e128cc586604+        .quad   0x6f5873ecb459747e+        .quad   0xa0b63dedcc1268f5+        .quad   0x566d78634586e22c++        // 2^8 * 3 * G++        .quad   0x1637a49f9cc10834+        .quad   0xbc8e56d5a89bc451+        .quad   0x1cb5ec0f7f7fd2db+        .quad   0x33975bca5ecc35d9+        .quad   0xa1054285c65a2fd0+        .quad   0x6c64112af31667c3+        .quad   0x680ae240731aee58+        .quad   0x14fba5f34793b22a+        .quad   0x3cd746166985f7d4+        .quad   0x593e5e84c9c80057+        .quad   0x2fc3f2b67b61131e+        .quad   0x14829cea83fc526c++        // 2^8 * 4 * G++        .quad   0xff437b8497dd95c2+        .quad   0x6c744e30aa4eb5a7+        .quad   0x9e0c5d613c85e88b+        .quad   0x2fd9c71e5f758173+        .quad   0x21e70b2f4e71ecb8+        .quad   0xe656ddb940a477e3+        .quad   0xbf6556cece1d4f80+        .quad   0x05fc3bc4535d7b7e+        .quad   0x24b8b3ae52afdedd+        .quad   0x3495638ced3b30cf+        .quad   0x33a4bc83a9be8195+        .quad   0x373767475c651f04++        // 2^8 * 5 * G++        .quad   0x2fba99fd40d1add9+        .quad   0xb307166f96f4d027+        .quad   0x4363f05215f03bae+        .quad   0x1fbea56c3b18f999+        .quad   0x634095cb14246590+        .quad   0xef12144016c15535+        .quad   0x9e38140c8910bc60+        .quad   0x6bf5905730907c8c+        .quad   0x0fa778f1e1415b8a+        .quad   0x06409ff7bac3a77e+        .quad   0x6f52d7b89aa29a50+        .quad   0x02521cf67a635a56++        // 2^8 * 6 * G++        .quad   0x513fee0b0a9d5294+        .quad   0x8f98e75c0fdf5a66+        .quad   0xd4618688bfe107ce+        .quad   0x3fa00a7e71382ced+        .quad   0xb1146720772f5ee4+        .quad   0xe8f894b196079ace+        .quad   0x4af8224d00ac824a+        .quad   0x001753d9f7cd6cc4+        .quad   0x3c69232d963ddb34+        .quad   0x1dde87dab4973858+        .quad   0xaad7d1f9a091f285+        .quad   0x12b5fe2fa048edb6++        // 2^8 * 7 * G++        .quad   0x71f0fbc496fce34d+        .quad   0x73b9826badf35bed+        .quad   0xd2047261ff28c561+        .quad   0x749b76f96fb1206f+        .quad   0xdf2b7c26ad6f1e92+        .quad   0x4b66d323504b8913+        .quad   0x8c409dc0751c8bc3+        .quad   0x6f7e93c20796c7b8+        .quad   0x1f5af604aea6ae05+        .quad   0xc12351f1bee49c99+        .quad   0x61a808b5eeff6b66+        .quad   0x0fcec10f01e02151++        // 2^8 * 8 * G++        .quad   0x644d58a649fe1e44+        .quad   0x21fcaea231ad777e+        .quad   0x02441c5a887fd0d2+        .quad   0x4901aa7183c511f3+        .quad   0x3df2d29dc4244e45+        .quad   0x2b020e7493d8de0a+        .quad   0x6cc8067e820c214d+        .quad   0x413779166feab90a+        .quad   0x08b1b7548c1af8f0+        .quad   0xce0f7a7c246299b4+        .quad   0xf760b0f91e06d939+        .quad   0x41bb887b726d1213++        // 2^12 * 1 * G++        .quad   0x9267806c567c49d8+        .quad   0x066d04ccca791e6a+        .quad   0xa69f5645e3cc394b+        .quad   0x5c95b686a0788cd2+        .quad   0x97d980e0aa39f7d2+        .quad   0x35d0384252c6b51c+        .quad   0x7d43f49307cd55aa+        .quad   0x56bd36cfb78ac362+        .quad   0x2ac519c10d14a954+        .quad   0xeaf474b494b5fa90+        .quad   0xe6af8382a9f87a5a+        .quad   0x0dea6db1879be094++        // 2^12 * 2 * G++        .quad   0xaa66bf547344e5ab+        .quad   0xda1258888f1b4309+        .quad   0x5e87d2b3fd564b2f+        .quad   0x5b2c78885483b1dd+        .quad   0x15baeb74d6a8797a+        .quad   0x7ef55cf1fac41732+        .quad   0x29001f5a3c8b05c5+        .quad   0x0ad7cc8752eaccfb+        .quad   0x52151362793408cf+        .quad   0xeb0f170319963d94+        .quad   0xa833b2fa883d9466+        .quad   0x093a7fa775003c78++        // 2^12 * 3 * G++        .quad   0xe5107de63a16d7be+        .quad   0xa377ffdc9af332cf+        .quad   0x70d5bf18440b677f+        .quad   0x6a252b19a4a31403+        .quad   0xb8e9604460a91286+        .quad   0x7f3fd8047778d3de+        .quad   0x67d01e31bf8a5e2d+        .quad   0x7b038a06c27b653e+        .quad   0x9ed919d5d36990f3+        .quad   0x5213aebbdb4eb9f2+        .quad   0xc708ea054cb99135+        .quad   0x58ded57f72260e56++        // 2^12 * 4 * G++        .quad   0x78e79dade9413d77+        .quad   0xf257f9d59729e67d+        .quad   0x59db910ee37aa7e6+        .quad   0x6aa11b5bbb9e039c+        .quad   0xda6d53265b0fd48b+        .quad   0x8960823193bfa988+        .quad   0xd78ac93261d57e28+        .quad   0x79f2942d3a5c8143+        .quad   0x97da2f25b6c88de9+        .quad   0x251ba7eaacf20169+        .quad   0x09b44f87ef4eb4e4+        .quad   0x7d90ab1bbc6a7da5++        // 2^12 * 5 * G++        .quad   0x9acca683a7016bfe+        .quad   0x90505f4df2c50b6d+        .quad   0x6b610d5fcce435aa+        .quad   0x19a10d446198ff96+        .quad   0x1a07a3f496b3c397+        .quad   0x11ceaa188f4e2532+        .quad   0x7d9498d5a7751bf0+        .quad   0x19ed161f508dd8a0+        .quad   0x560a2cd687dce6ca+        .quad   0x7f3568c48664cf4d+        .quad   0x8741e95222803a38+        .quad   0x483bdab1595653fc++        // 2^12 * 6 * G++        .quad   0xfa780f148734fa49+        .quad   0x106f0b70360534e0+        .quad   0x2210776fe3e307bd+        .quad   0x3286c109dde6a0fe+        .quad   0xd6cf4d0ab4da80f6+        .quad   0x82483e45f8307fe0+        .quad   0x05005269ae6f9da4+        .quad   0x1c7052909cf7877a+        .quad   0x32ee7de2874e98d4+        .quad   0x14c362e9b97e0c60+        .quad   0x5781dcde6a60a38a+        .quad   0x217dd5eaaa7aa840++        // 2^12 * 7 * G++        .quad   0x9db7c4d0248e1eb0+        .quad   0xe07697e14d74bf52+        .quad   0x1e6a9b173c562354+        .quad   0x7fa7c21f795a4965+        .quad   0x8bdf1fb9be8c0ec8+        .quad   0x00bae7f8e30a0282+        .quad   0x4963991dad6c4f6c+        .quad   0x07058a6e5df6f60a+        .quad   0xe9eb02c4db31f67f+        .quad   0xed25fd8910bcfb2b+        .quad   0x46c8131f5c5cddb4+        .quad   0x33b21c13a0cb9bce++        // 2^12 * 8 * G++        .quad   0x360692f8087d8e31+        .quad   0xf4dcc637d27163f7+        .quad   0x25a4e62065ea5963+        .quad   0x659bf72e5ac160d9+        .quad   0x9aafb9b05ee38c5b+        .quad   0xbf9d2d4e071a13c7+        .quad   0x8eee6e6de933290a+        .quad   0x1c3bab17ae109717+        .quad   0x1c9ab216c7cab7b0+        .quad   0x7d65d37407bbc3cc+        .quad   0x52744750504a58d5+        .quad   0x09f2606b131a2990++        // 2^16 * 1 * G++        .quad   0x40e87d44744346be+        .quad   0x1d48dad415b52b25+        .quad   0x7c3a8a18a13b603e+        .quad   0x4eb728c12fcdbdf7+        .quad   0x7e234c597c6691ae+        .quad   0x64889d3d0a85b4c8+        .quad   0xdae2c90c354afae7+        .quad   0x0a871e070c6a9e1d+        .quad   0x3301b5994bbc8989+        .quad   0x736bae3a5bdd4260+        .quad   0x0d61ade219d59e3c+        .quad   0x3ee7300f2685d464++        // 2^16 * 2 * G++        .quad   0xf5d255e49e7dd6b7+        .quad   0x8016115c610b1eac+        .quad   0x3c99975d92e187ca+        .quad   0x13815762979125c2+        .quad   0x43fa7947841e7518+        .quad   0xe5c6fa59639c46d7+        .quad   0xa1065e1de3052b74+        .quad   0x7d47c6a2cfb89030+        .quad   0x3fdad0148ef0d6e0+        .quad   0x9d3e749a91546f3c+        .quad   0x71ec621026bb8157+        .quad   0x148cf58d34c9ec80++        // 2^16 * 3 * G++        .quad   0x46a492f67934f027+        .quad   0x469984bef6840aa9+        .quad   0x5ca1bc2a89611854+        .quad   0x3ff2fa1ebd5dbbd4+        .quad   0xe2572f7d9ae4756d+        .quad   0x56c345bb88f3487f+        .quad   0x9fd10b6d6960a88d+        .quad   0x278febad4eaea1b9+        .quad   0xb1aa681f8c933966+        .quad   0x8c21949c20290c98+        .quad   0x39115291219d3c52+        .quad   0x4104dd02fe9c677b++        // 2^16 * 4 * G++        .quad   0x72b2bf5e1124422a+        .quad   0xa1fa0c3398a33ab5+        .quad   0x94cb6101fa52b666+        .quad   0x2c863b00afaf53d5+        .quad   0x81214e06db096ab8+        .quad   0x21a8b6c90ce44f35+        .quad   0x6524c12a409e2af5+        .quad   0x0165b5a48efca481+        .quad   0xf190a474a0846a76+        .quad   0x12eff984cd2f7cc0+        .quad   0x695e290658aa2b8f+        .quad   0x591b67d9bffec8b8++        // 2^16 * 5 * G++        .quad   0x312f0d1c80b49bfa+        .quad   0x5979515eabf3ec8a+        .quad   0x727033c09ef01c88+        .quad   0x3de02ec7ca8f7bcb+        .quad   0x99b9b3719f18b55d+        .quad   0xe465e5faa18c641e+        .quad   0x61081136c29f05ed+        .quad   0x489b4f867030128b+        .quad   0xd232102d3aeb92ef+        .quad   0xe16253b46116a861+        .quad   0x3d7eabe7190baa24+        .quad   0x49f5fbba496cbebf++        // 2^16 * 6 * G++        .quad   0x30949a108a5bcfd4+        .quad   0xdc40dd70bc6473eb+        .quad   0x92c294c1307c0d1c+        .quad   0x5604a86dcbfa6e74+        .quad   0x155d628c1e9c572e+        .quad   0x8a4d86acc5884741+        .quad   0x91a352f6515763eb+        .quad   0x06a1a6c28867515b+        .quad   0x7288d1d47c1764b6+        .quad   0x72541140e0418b51+        .quad   0x9f031a6018acf6d1+        .quad   0x20989e89fe2742c6++        // 2^16 * 7 * G++        .quad   0x499777fd3a2dcc7f+        .quad   0x32857c2ca54fd892+        .quad   0xa279d864d207e3a0+        .quad   0x0403ed1d0ca67e29+        .quad   0x1674278b85eaec2e+        .quad   0x5621dc077acb2bdf+        .quad   0x640a4c1661cbf45a+        .quad   0x730b9950f70595d3+        .quad   0xc94b2d35874ec552+        .quad   0xc5e6c8cf98246f8d+        .quad   0xf7cb46fa16c035ce+        .quad   0x5bd7454308303dcc++        // 2^16 * 8 * G++        .quad   0x7f9ad19528b24cc2+        .quad   0x7f6b54656335c181+        .quad   0x66b8b66e4fc07236+        .quad   0x133a78007380ad83+        .quad   0x85c4932115e7792a+        .quad   0xc64c89a2bdcdddc9+        .quad   0x9d1e3da8ada3d762+        .quad   0x5bb7db123067f82c+        .quad   0x0961f467c6ca62be+        .quad   0x04ec21d6211952ee+        .quad   0x182360779bd54770+        .quad   0x740dca6d58f0e0d2++        // 2^20 * 1 * G++        .quad   0x50b70bf5d3f0af0b+        .quad   0x4feaf48ae32e71f7+        .quad   0x60e84ed3a55bbd34+        .quad   0x00ed489b3f50d1ed+        .quad   0x3906c72aed261ae5+        .quad   0x9ab68fd988e100f7+        .quad   0xf5e9059af3360197+        .quad   0x0e53dc78bf2b6d47+        .quad   0xb90829bf7971877a+        .quad   0x5e4444636d17e631+        .quad   0x4d05c52e18276893+        .quad   0x27632d9a5a4a4af5++        // 2^20 * 2 * G++        .quad   0xd11ff05154b260ce+        .quad   0xd86dc38e72f95270+        .quad   0x601fcd0d267cc138+        .quad   0x2b67916429e90ccd+        .quad   0xa98285d187eaffdb+        .quad   0xa5b4fbbbd8d0a864+        .quad   0xb658f27f022663f7+        .quad   0x3bbc2b22d99ce282+        .quad   0xb917c952583c0a58+        .quad   0x653ff9b80fe4c6f3+        .quad   0x9b0da7d7bcdf3c0c+        .quad   0x43a0eeb6ab54d60e++        // 2^20 * 3 * G++        .quad   0x396966a46d4a5487+        .quad   0xf811a18aac2bb3ba+        .quad   0x66e4685b5628b26b+        .quad   0x70a477029d929b92+        .quad   0x3ac6322357875fe8+        .quad   0xd9d4f4ecf5fbcb8f+        .quad   0x8dee8493382bb620+        .quad   0x50c5eaa14c799fdc+        .quad   0xdd0edc8bd6f2fb3c+        .quad   0x54c63aa79cc7b7a0+        .quad   0xae0b032b2c8d9f1a+        .quad   0x6f9ce107602967fb++        // 2^20 * 4 * G++        .quad   0xad1054b1cde1c22a+        .quad   0xc4a8e90248eb32df+        .quad   0x5f3e7b33accdc0ea+        .quad   0x72364713fc79963e+        .quad   0x139693063520e0b5+        .quad   0x437fcf7c88ea03fe+        .quad   0xf7d4c40bd3c959bc+        .quad   0x699154d1f893ded9+        .quad   0x315d5c75b4b27526+        .quad   0xcccb842d0236daa5+        .quad   0x22f0c8a3345fee8e+        .quad   0x73975a617d39dbed++        // 2^20 * 5 * G++        .quad   0xe4024df96375da10+        .quad   0x78d3251a1830c870+        .quad   0x902b1948658cd91c+        .quad   0x7e18b10b29b7438a+        .quad   0x6f37f392f4433e46+        .quad   0x0e19b9a11f566b18+        .quad   0x220fb78a1fd1d662+        .quad   0x362a4258a381c94d+        .quad   0x9071d9132b6beb2f+        .quad   0x0f26e9ad28418247+        .quad   0xeab91ec9bdec925d+        .quad   0x4be65bc8f48af2de++        // 2^20 * 6 * G++        .quad   0x78487feba36e7028+        .quad   0x5f3f13001dd8ce34+        .quad   0x934fb12d4b30c489+        .quad   0x056c244d397f0a2b+        .quad   0x1d50fba257c26234+        .quad   0x7bd4823adeb0678b+        .quad   0xc2b0dc6ea6538af5+        .quad   0x5665eec6351da73e+        .quad   0xdb3ee00943bfb210+        .quad   0x4972018720800ac2+        .quad   0x26ab5d6173bd8667+        .quad   0x20b209c2ab204938++        // 2^20 * 7 * G++        .quad   0x549e342ac07fb34b+        .quad   0x02d8220821373d93+        .quad   0xbc262d70acd1f567+        .quad   0x7a92c9fdfbcac784+        .quad   0x1fcca94516bd3289+        .quad   0x448d65aa41420428+        .quad   0x59c3b7b216a55d62+        .quad   0x49992cc64e612cd8+        .quad   0x65bd1bea70f801de+        .quad   0x1befb7c0fe49e28a+        .quad   0xa86306cdb1b2ae4a+        .quad   0x3b7ac0cd265c2a09++        // 2^20 * 8 * G++        .quad   0x822bee438c01bcec+        .quad   0x530cb525c0fbc73b+        .quad   0x48519034c1953fe9+        .quad   0x265cc261e09a0f5b+        .quad   0xf0d54e4f22ed39a7+        .quad   0xa2aae91e5608150a+        .quad   0xf421b2e9eddae875+        .quad   0x31bc531d6b7de992+        .quad   0xdf3d134da980f971+        .quad   0x7a4fb8d1221a22a7+        .quad   0x3df7d42035aad6d8+        .quad   0x2a14edcc6a1a125e++        // 2^24 * 1 * G++        .quad   0xdf48ee0752cfce4e+        .quad   0xc3fffaf306ec08b7+        .quad   0x05710b2ab95459c4+        .quad   0x161d25fa963ea38d+        .quad   0x231a8c570478433c+        .quad   0xb7b5270ec281439d+        .quad   0xdbaa99eae3d9079f+        .quad   0x2c03f5256c2b03d9+        .quad   0x790f18757b53a47d+        .quad   0x307b0130cf0c5879+        .quad   0x31903d77257ef7f9+        .quad   0x699468bdbd96bbaf++        // 2^24 * 2 * G++        .quad   0xbd1f2f46f4dafecf+        .quad   0x7cef0114a47fd6f7+        .quad   0xd31ffdda4a47b37f+        .quad   0x525219a473905785+        .quad   0xd8dd3de66aa91948+        .quad   0x485064c22fc0d2cc+        .quad   0x9b48246634fdea2f+        .quad   0x293e1c4e6c4a2e3a+        .quad   0x376e134b925112e1+        .quad   0x703778b5dca15da0+        .quad   0xb04589af461c3111+        .quad   0x5b605c447f032823++        // 2^24 * 3 * G++        .quad   0xb965805920c47c89+        .quad   0xe7f0100c923b8fcc+        .quad   0x0001256502e2ef77+        .quad   0x24a76dcea8aeb3ee+        .quad   0x3be9fec6f0e7f04c+        .quad   0x866a579e75e34962+        .quad   0x5542ef161e1de61a+        .quad   0x2f12fef4cc5abdd5+        .quad   0x0a4522b2dfc0c740+        .quad   0x10d06e7f40c9a407+        .quad   0xc6cf144178cff668+        .quad   0x5e607b2518a43790++        // 2^24 * 4 * G++        .quad   0x58b31d8f6cdf1818+        .quad   0x35cfa74fc36258a2+        .quad   0xe1b3ff4f66e61d6e+        .quad   0x5067acab6ccdd5f7+        .quad   0xa02c431ca596cf14+        .quad   0xe3c42d40aed3e400+        .quad   0xd24526802e0f26db+        .quad   0x201f33139e457068+        .quad   0xfd527f6b08039d51+        .quad   0x18b14964017c0006+        .quad   0xd5220eb02e25a4a8+        .quad   0x397cba8862460375++        // 2^24 * 5 * G++        .quad   0x30c13093f05959b2+        .quad   0xe23aa18de9a97976+        .quad   0x222fd491721d5e26+        .quad   0x2339d320766e6c3a+        .quad   0x7815c3fbc81379e7+        .quad   0xa6619420dde12af1+        .quad   0xffa9c0f885a8fdd5+        .quad   0x771b4022c1e1c252+        .quad   0xd87dd986513a2fa7+        .quad   0xf5ac9b71f9d4cf08+        .quad   0xd06bc31b1ea283b3+        .quad   0x331a189219971a76++        // 2^24 * 6 * G++        .quad   0xf5166f45fb4f80c6+        .quad   0x9c36c7de61c775cf+        .quad   0xe3d4e81b9041d91c+        .quad   0x31167c6b83bdfe21+        .quad   0x26512f3a9d7572af+        .quad   0x5bcbe28868074a9e+        .quad   0x84edc1c11180f7c4+        .quad   0x1ac9619ff649a67b+        .quad   0xf22b3842524b1068+        .quad   0x5068343bee9ce987+        .quad   0xfc9d71844a6250c8+        .quad   0x612436341f08b111++        // 2^24 * 7 * G++        .quad   0xd99d41db874e898d+        .quad   0x09fea5f16c07dc20+        .quad   0x793d2c67d00f9bbc+        .quad   0x46ebe2309e5eff40+        .quad   0x8b6349e31a2d2638+        .quad   0x9ddfb7009bd3fd35+        .quad   0x7f8bf1b8a3a06ba4+        .quad   0x1522aa3178d90445+        .quad   0x2c382f5369614938+        .quad   0xdafe409ab72d6d10+        .quad   0xe8c83391b646f227+        .quad   0x45fe70f50524306c++        // 2^24 * 8 * G++        .quad   0xda4875a6960c0b8c+        .quad   0x5b68d076ef0e2f20+        .quad   0x07fb51cf3d0b8fd4+        .quad   0x428d1623a0e392d4+        .quad   0x62f24920c8951491+        .quad   0x05f007c83f630ca2+        .quad   0x6fbb45d2f5c9d4b8+        .quad   0x16619f6db57a2245+        .quad   0x084f4a4401a308fd+        .quad   0xa82219c376a5caac+        .quad   0xdeb8de4643d1bc7d+        .quad   0x1d81592d60bd38c6++        // 2^28 * 1 * G++        .quad   0xd833d7beec2a4c38+        .quad   0x2c9162830acc20ed+        .quad   0xe93a47aa92df7581+        .quad   0x702d67a3333c4a81+        .quad   0x3a4a369a2f89c8a1+        .quad   0x63137a1d7c8de80d+        .quad   0xbcac008a78eda015+        .quad   0x2cb8b3a5b483b03f+        .quad   0x36e417cbcb1b90a1+        .quad   0x33b3ddaa7f11794e+        .quad   0x3f510808885bc607+        .quad   0x24141dc0e6a8020d++        // 2^28 * 2 * G++        .quad   0x59f73c773fefee9d+        .quad   0xb3f1ef89c1cf989d+        .quad   0xe35dfb42e02e545f+        .quad   0x5766120b47a1b47c+        .quad   0x91925dccbd83157d+        .quad   0x3ca1205322cc8094+        .quad   0x28e57f183f90d6e4+        .quad   0x1a4714cede2e767b+        .quad   0xdb20ba0fb8b6b7ff+        .quad   0xb732c3b677511fa1+        .quad   0xa92b51c099f02d89+        .quad   0x4f3875ad489ca5f1++        // 2^28 * 3 * G++        .quad   0xc7fc762f4932ab22+        .quad   0x7ac0edf72f4c3c1b+        .quad   0x5f6b55aa9aa895e8+        .quad   0x3680274dad0a0081+        .quad   0x79ed13f6ee73eec0+        .quad   0xa5c6526d69110bb1+        .quad   0xe48928c38603860c+        .quad   0x722a1446fd7059f5+        .quad   0xd0959fe9a8cf8819+        .quad   0xd0a995508475a99c+        .quad   0x6eac173320b09cc5+        .quad   0x628ecf04331b1095++        // 2^28 * 4 * G++        .quad   0x98bcb118a9d0ddbc+        .quad   0xee449e3408b4802b+        .quad   0x87089226b8a6b104+        .quad   0x685f349a45c7915d+        .quad   0x9b41acf85c74ccf1+        .quad   0xb673318108265251+        .quad   0x99c92aed11adb147+        .quad   0x7a47d70d34ecb40f+        .quad   0x60a0c4cbcc43a4f5+        .quad   0x775c66ca3677bea9+        .quad   0xa17aa1752ff8f5ed+        .quad   0x11ded9020e01fdc0++        // 2^28 * 5 * G++        .quad   0x890e7809caefe704+        .quad   0x8728296de30e8c6c+        .quad   0x4c5cd2a392aeb1c9+        .quad   0x194263d15771531f+        .quad   0x471f95b03bea93b7+        .quad   0x0552d7d43313abd3+        .quad   0xbd9370e2e17e3f7b+        .quad   0x7b120f1db20e5bec+        .quad   0x17d2fb3d86502d7a+        .quad   0xb564d84450a69352+        .quad   0x7da962c8a60ed75d+        .quad   0x00d0f85b318736aa++        // 2^28 * 6 * G++        .quad   0x978b142e777c84fd+        .quad   0xf402644705a8c062+        .quad   0xa67ad51be7e612c7+        .quad   0x2f7b459698dd6a33+        .quad   0xa6753c1efd7621c1+        .quad   0x69c0b4a7445671f5+        .quad   0x971f527405b23c11+        .quad   0x387bc74851a8c7cd+        .quad   0x81894b4d4a52a9a8+        .quad   0xadd93e12f6b8832f+        .quad   0x184d8548b61bd638+        .quad   0x3f1c62dbd6c9f6cd++        // 2^28 * 7 * G++        .quad   0x2e8f1f0091910c1f+        .quad   0xa4df4fe0bff2e12c+        .quad   0x60c6560aee927438+        .quad   0x6338283facefc8fa+        .quad   0x3fad3e40148f693d+        .quad   0x052656e194eb9a72+        .quad   0x2f4dcbfd184f4e2f+        .quad   0x406f8db1c482e18b+        .quad   0x9e630d2c7f191ee4+        .quad   0x4fbf8301bc3ff670+        .quad   0x787d8e4e7afb73c4+        .quad   0x50d83d5be8f58fa5++        // 2^28 * 8 * G++        .quad   0x85683916c11a1897+        .quad   0x2d69a4efe506d008+        .quad   0x39af1378f664bd01+        .quad   0x65942131361517c6+        .quad   0xc0accf90b4d3b66d+        .quad   0xa7059de561732e60+        .quad   0x033d1f7870c6b0ba+        .quad   0x584161cd26d946e4+        .quad   0xbbf2b1a072d27ca2+        .quad   0xbf393c59fbdec704+        .quad   0xe98dbbcee262b81e+        .quad   0x02eebd0b3029b589++        // 2^32 * 1 * G++        .quad   0x61368756a60dac5f+        .quad   0x17e02f6aebabdc57+        .quad   0x7f193f2d4cce0f7d+        .quad   0x20234a7789ecdcf0+        .quad   0x8765b69f7b85c5e8+        .quad   0x6ff0678bd168bab2+        .quad   0x3a70e77c1d330f9b+        .quad   0x3a5f6d51b0af8e7c+        .quad   0x76d20db67178b252+        .quad   0x071c34f9d51ed160+        .quad   0xf62a4a20b3e41170+        .quad   0x7cd682353cffe366++        // 2^32 * 2 * G++        .quad   0x0be1a45bd887fab6+        .quad   0x2a846a32ba403b6e+        .quad   0xd9921012e96e6000+        .quad   0x2838c8863bdc0943+        .quad   0xa665cd6068acf4f3+        .quad   0x42d92d183cd7e3d3+        .quad   0x5759389d336025d9+        .quad   0x3ef0253b2b2cd8ff+        .quad   0xd16bb0cf4a465030+        .quad   0xfa496b4115c577ab+        .quad   0x82cfae8af4ab419d+        .quad   0x21dcb8a606a82812++        // 2^32 * 3 * G++        .quad   0x5c6004468c9d9fc8+        .quad   0x2540096ed42aa3cb+        .quad   0x125b4d4c12ee2f9c+        .quad   0x0bc3d08194a31dab+        .quad   0x9a8d00fabe7731ba+        .quad   0x8203607e629e1889+        .quad   0xb2cc023743f3d97f+        .quad   0x5d840dbf6c6f678b+        .quad   0x706e380d309fe18b+        .quad   0x6eb02da6b9e165c7+        .quad   0x57bbba997dae20ab+        .quad   0x3a4276232ac196dd++        // 2^32 * 4 * G++        .quad   0x4b42432c8a7084fa+        .quad   0x898a19e3dfb9e545+        .quad   0xbe9f00219c58e45d+        .quad   0x1ff177cea16debd1+        .quad   0x3bf8c172db447ecb+        .quad   0x5fcfc41fc6282dbd+        .quad   0x80acffc075aa15fe+        .quad   0x0770c9e824e1a9f9+        .quad   0xcf61d99a45b5b5fd+        .quad   0x860984e91b3a7924+        .quad   0xe7300919303e3e89+        .quad   0x39f264fd41500b1e++        // 2^32 * 5 * G++        .quad   0xa7ad3417dbe7e29c+        .quad   0xbd94376a2b9c139c+        .quad   0xa0e91b8e93597ba9+        .quad   0x1712d73468889840+        .quad   0xd19b4aabfe097be1+        .quad   0xa46dfce1dfe01929+        .quad   0xc3c908942ca6f1ff+        .quad   0x65c621272c35f14e+        .quad   0xe72b89f8ce3193dd+        .quad   0x4d103356a125c0bb+        .quad   0x0419a93d2e1cfe83+        .quad   0x22f9800ab19ce272++        // 2^32 * 6 * G++        .quad   0x605a368a3e9ef8cb+        .quad   0xe3e9c022a5504715+        .quad   0x553d48b05f24248f+        .quad   0x13f416cd647626e5+        .quad   0x42029fdd9a6efdac+        .quad   0xb912cebe34a54941+        .quad   0x640f64b987bdf37b+        .quad   0x4171a4d38598cab4+        .quad   0xfa2758aa99c94c8c+        .quad   0x23006f6fb000b807+        .quad   0xfbd291ddadda5392+        .quad   0x508214fa574bd1ab++        // 2^32 * 7 * G++        .quad   0xc20269153ed6fe4b+        .quad   0xa65a6739511d77c4+        .quad   0xcbde26462c14af94+        .quad   0x22f960ec6faba74b+        .quad   0x461a15bb53d003d6+        .quad   0xb2102888bcf3c965+        .quad   0x27c576756c683a5a+        .quad   0x3a7758a4c86cb447+        .quad   0x548111f693ae5076+        .quad   0x1dae21df1dfd54a6+        .quad   0x12248c90f3115e65+        .quad   0x5d9fd15f8de7f494++        // 2^32 * 8 * G++        .quad   0x031408d36d63727f+        .quad   0x6a379aefd7c7b533+        .quad   0xa9e18fc5ccaee24b+        .quad   0x332f35914f8fbed3+        .quad   0x3f244d2aeed7521e+        .quad   0x8e3a9028432e9615+        .quad   0xe164ba772e9c16d4+        .quad   0x3bc187fa47eb98d8+        .quad   0x6d470115ea86c20c+        .quad   0x998ab7cb6c46d125+        .quad   0xd77832b53a660188+        .quad   0x450d81ce906fba03++        // 2^36 * 1 * G++        .quad   0xf8ae4d2ad8453902+        .quad   0x7018058ee8db2d1d+        .quad   0xaab3995fc7d2c11e+        .quad   0x53b16d2324ccca79+        .quad   0x23264d66b2cae0b5+        .quad   0x7dbaed33ebca6576+        .quad   0x030ebed6f0d24ac8+        .quad   0x2a887f78f7635510+        .quad   0x2a23b9e75c012d4f+        .quad   0x0c974651cae1f2ea+        .quad   0x2fb63273675d70ca+        .quad   0x0ba7250b864403f5++        // 2^36 * 2 * G++        .quad   0xbb0d18fd029c6421+        .quad   0xbc2d142189298f02+        .quad   0x8347f8e68b250e96+        .quad   0x7b9f2fe8032d71c9+        .quad   0xdd63589386f86d9c+        .quad   0x61699176e13a85a4+        .quad   0x2e5111954eaa7d57+        .quad   0x32c21b57fb60bdfb+        .quad   0xd87823cd319e0780+        .quad   0xefc4cfc1897775c5+        .quad   0x4854fb129a0ab3f7+        .quad   0x12c49d417238c371++        // 2^36 * 3 * G++        .quad   0x0950b533ffe83769+        .quad   0x21861c1d8e1d6bd1+        .quad   0xf022d8381302e510+        .quad   0x2509200c6391cab4+        .quad   0x09b3a01783799542+        .quad   0x626dd08faad5ee3f+        .quad   0xba00bceeeb70149f+        .quad   0x1421b246a0a444c9+        .quad   0x4aa43a8e8c24a7c7+        .quad   0x04c1f540d8f05ef5+        .quad   0xadba5e0c0b3eb9dc+        .quad   0x2ab5504448a49ce3++        // 2^36 * 4 * G++        .quad   0x2ed227266f0f5dec+        .quad   0x9824ee415ed50824+        .quad   0x807bec7c9468d415+        .quad   0x7093bae1b521e23f+        .quad   0xdc07ac631c5d3afa+        .quad   0x58615171f9df8c6c+        .quad   0x72a079d89d73e2b0+        .quad   0x7301f4ceb4eae15d+        .quad   0x6409e759d6722c41+        .quad   0xa674e1cf72bf729b+        .quad   0xbc0a24eb3c21e569+        .quad   0x390167d24ebacb23++        // 2^36 * 5 * G++        .quad   0x27f58e3bba353f1c+        .quad   0x4c47764dbf6a4361+        .quad   0xafbbc4e56e562650+        .quad   0x07db2ee6aae1a45d+        .quad   0xd7bb054ba2f2120b+        .quad   0xe2b9ceaeb10589b7+        .quad   0x3fe8bac8f3c0edbe+        .quad   0x4cbd40767112cb69+        .quad   0x0b603cc029c58176+        .quad   0x5988e3825cb15d61+        .quad   0x2bb61413dcf0ad8d+        .quad   0x7b8eec6c74183287++        // 2^36 * 6 * G++        .quad   0xe4ca40782cd27cb0+        .quad   0xdaf9c323fbe967bd+        .quad   0xb29bd34a8ad41e9e+        .quad   0x72810497626ede4d+        .quad   0x32fee570fc386b73+        .quad   0xda8b0141da3a8cc7+        .quad   0x975ffd0ac8968359+        .quad   0x6ee809a1b132a855+        .quad   0x9444bb31fcfd863a+        .quad   0x2fe3690a3e4e48c5+        .quad   0xdc29c867d088fa25+        .quad   0x13bd1e38d173292e++        // 2^36 * 7 * G++        .quad   0xd32b4cd8696149b5+        .quad   0xe55937d781d8aab7+        .quad   0x0bcb2127ae122b94+        .quad   0x41e86fcfb14099b0+        .quad   0x223fb5cf1dfac521+        .quad   0x325c25316f554450+        .quad   0x030b98d7659177ac+        .quad   0x1ed018b64f88a4bd+        .quad   0x3630dfa1b802a6b0+        .quad   0x880f874742ad3bd5+        .quad   0x0af90d6ceec5a4d4+        .quad   0x746a247a37cdc5d9++        // 2^36 * 8 * G++        .quad   0xd531b8bd2b7b9af6+        .quad   0x5005093537fc5b51+        .quad   0x232fcf25c593546d+        .quad   0x20a365142bb40f49+        .quad   0x6eccd85278d941ed+        .quad   0x2254ae83d22f7843+        .quad   0xc522d02e7bbfcdb7+        .quad   0x681e3351bff0e4e2+        .quad   0x8b64b59d83034f45+        .quad   0x2f8b71f21fa20efb+        .quad   0x69249495ba6550e4+        .quad   0x539ef98e45d5472b++        // 2^40 * 1 * G++        .quad   0x6e7bb6a1a6205275+        .quad   0xaa4f21d7413c8e83+        .quad   0x6f56d155e88f5cb2+        .quad   0x2de25d4ba6345be1+        .quad   0xd074d8961cae743f+        .quad   0xf86d18f5ee1c63ed+        .quad   0x97bdc55be7f4ed29+        .quad   0x4cbad279663ab108+        .quad   0x80d19024a0d71fcd+        .quad   0xc525c20afb288af8+        .quad   0xb1a3974b5f3a6419+        .quad   0x7d7fbcefe2007233++        // 2^40 * 2 * G++        .quad   0xfaef1e6a266b2801+        .quad   0x866c68c4d5739f16+        .quad   0xf68a2fbc1b03762c+        .quad   0x5975435e87b75a8d+        .quad   0xcd7c5dc5f3c29094+        .quad   0xc781a29a2a9105ab+        .quad   0x80c61d36421c3058+        .quad   0x4f9cd196dcd8d4d7+        .quad   0x199297d86a7b3768+        .quad   0xd0d058241ad17a63+        .quad   0xba029cad5c1c0c17+        .quad   0x7ccdd084387a0307++        // 2^40 * 3 * G++        .quad   0xdca6422c6d260417+        .quad   0xae153d50948240bd+        .quad   0xa9c0c1b4fb68c677+        .quad   0x428bd0ed61d0cf53+        .quad   0x9b0c84186760cc93+        .quad   0xcdae007a1ab32a99+        .quad   0xa88dec86620bda18+        .quad   0x3593ca848190ca44+        .quad   0x9213189a5e849aa7+        .quad   0xd4d8c33565d8facd+        .quad   0x8c52545b53fdbbd1+        .quad   0x27398308da2d63e6++        // 2^40 * 4 * G++        .quad   0x42c38d28435ed413+        .quad   0xbd50f3603278ccc9+        .quad   0xbb07ab1a79da03ef+        .quad   0x269597aebe8c3355+        .quad   0xb9a10e4c0a702453+        .quad   0x0fa25866d57d1bde+        .quad   0xffb9d9b5cd27daf7+        .quad   0x572c2945492c33fd+        .quad   0xc77fc745d6cd30be+        .quad   0xe4dfe8d3e3baaefb+        .quad   0xa22c8830aa5dda0c+        .quad   0x7f985498c05bca80++        // 2^40 * 5 * G++        .quad   0x3849ce889f0be117+        .quad   0x8005ad1b7b54a288+        .quad   0x3da3c39f23fc921c+        .quad   0x76c2ec470a31f304+        .quad   0xd35615520fbf6363+        .quad   0x08045a45cf4dfba6+        .quad   0xeec24fbc873fa0c2+        .quad   0x30f2653cd69b12e7+        .quad   0x8a08c938aac10c85+        .quad   0x46179b60db276bcb+        .quad   0xa920c01e0e6fac70+        .quad   0x2f1273f1596473da++        // 2^40 * 6 * G++        .quad   0x4739fc7c8ae01e11+        .quad   0xfd5274904a6aab9f+        .quad   0x41d98a8287728f2e+        .quad   0x5d9e572ad85b69f2+        .quad   0x30488bd755a70bc0+        .quad   0x06d6b5a4f1d442e7+        .quad   0xead1a69ebc596162+        .quad   0x38ac1997edc5f784+        .quad   0x0666b517a751b13b+        .quad   0x747d06867e9b858c+        .quad   0xacacc011454dde49+        .quad   0x22dfcd9cbfe9e69c++        // 2^40 * 7 * G++        .quad   0x8ddbd2e0c30d0cd9+        .quad   0xad8e665facbb4333+        .quad   0x8f6b258c322a961f+        .quad   0x6b2916c05448c1c7+        .quad   0x56ec59b4103be0a1+        .quad   0x2ee3baecd259f969+        .quad   0x797cb29413f5cd32+        .quad   0x0fe9877824cde472+        .quad   0x7edb34d10aba913b+        .quad   0x4ea3cd822e6dac0e+        .quad   0x66083dff6578f815+        .quad   0x4c303f307ff00a17++        // 2^40 * 8 * G++        .quad   0xd30a3bd617b28c85+        .quad   0xc5d377b739773bea+        .quad   0xc6c6e78c1e6a5cbf+        .quad   0x0d61b8f78b2ab7c4+        .quad   0x29fc03580dd94500+        .quad   0xecd27aa46fbbec93+        .quad   0x130a155fc2e2a7f8+        .quad   0x416b151ab706a1d5+        .quad   0x56a8d7efe9c136b0+        .quad   0xbd07e5cd58e44b20+        .quad   0xafe62fda1b57e0ab+        .quad   0x191a2af74277e8d2++        // 2^44 * 1 * G++        .quad   0xd550095bab6f4985+        .quad   0x04f4cd5b4fbfaf1a+        .quad   0x9d8e2ed12a0c7540+        .quad   0x2bc24e04b2212286+        .quad   0x09d4b60b2fe09a14+        .quad   0xc384f0afdbb1747e+        .quad   0x58e2ea8978b5fd6e+        .quad   0x519ef577b5e09b0a+        .quad   0x1863d7d91124cca9+        .quad   0x7ac08145b88a708e+        .quad   0x2bcd7309857031f5+        .quad   0x62337a6e8ab8fae5++        // 2^44 * 2 * G++        .quad   0x4bcef17f06ffca16+        .quad   0xde06e1db692ae16a+        .quad   0x0753702d614f42b0+        .quad   0x5f6041b45b9212d0+        .quad   0xd1ab324e1b3a1273+        .quad   0x18947cf181055340+        .quad   0x3b5d9567a98c196e+        .quad   0x7fa00425802e1e68+        .quad   0x7d531574028c2705+        .quad   0x80317d69db0d75fe+        .quad   0x30fface8ef8c8ddd+        .quad   0x7e9de97bb6c3e998++        // 2^44 * 3 * G++        .quad   0x1558967b9e6585a3+        .quad   0x97c99ce098e98b92+        .quad   0x10af149b6eb3adad+        .quad   0x42181fe8f4d38cfa+        .quad   0xf004be62a24d40dd+        .quad   0xba0659910452d41f+        .quad   0x81c45ee162a44234+        .quad   0x4cb829d8a22266ef+        .quad   0x1dbcaa8407b86681+        .quad   0x081f001e8b26753b+        .quad   0x3cd7ce6a84048e81+        .quad   0x78af11633f25f22c++        // 2^44 * 4 * G++        .quad   0x8416ebd40b50babc+        .quad   0x1508722628208bee+        .quad   0xa3148fafb9c1c36d+        .quad   0x0d07daacd32d7d5d+        .quad   0x3241c00e7d65318c+        .quad   0xe6bee5dcd0e86de7+        .quad   0x118b2dc2fbc08c26+        .quad   0x680d04a7fc603dc3+        .quad   0xf9c2414a695aa3eb+        .quad   0xdaa42c4c05a68f21+        .quad   0x7c6c23987f93963e+        .quad   0x210e8cd30c3954e3++        // 2^44 * 5 * G++        .quad   0xac4201f210a71c06+        .quad   0x6a65e0aef3bfb021+        .quad   0xbc42c35c393632f7+        .quad   0x56ea8db1865f0742+        .quad   0x2b50f16137fe6c26+        .quad   0xe102bcd856e404d8+        .quad   0x12b0f1414c561f6b+        .quad   0x51b17bc8d028ec91+        .quad   0xfff5fb4bcf535119+        .quad   0xf4989d79df1108a0+        .quad   0xbdfcea659a3ba325+        .quad   0x18a11f1174d1a6f2++        // 2^44 * 6 * G++        .quad   0x407375ab3f6bba29+        .quad   0x9ec3b6d8991e482e+        .quad   0x99c80e82e55f92e9+        .quad   0x307c13b6fb0c0ae1+        .quad   0xfbd63cdad27a5f2c+        .quad   0xf00fc4bc8aa106d7+        .quad   0x53fb5c1a8e64a430+        .quad   0x04eaabe50c1a2e85+        .quad   0x24751021cb8ab5e7+        .quad   0xfc2344495c5010eb+        .quad   0x5f1e717b4e5610a1+        .quad   0x44da5f18c2710cd5++        // 2^44 * 7 * G++        .quad   0x033cc55ff1b82eb5+        .quad   0xb15ae36d411cae52+        .quad   0xba40b6198ffbacd3+        .quad   0x768edce1532e861f+        .quad   0x9156fe6b89d8eacc+        .quad   0xe6b79451e23126a1+        .quad   0xbd7463d93944eb4e+        .quad   0x726373f6767203ae+        .quad   0xe305ca72eb7ef68a+        .quad   0x662cf31f70eadb23+        .quad   0x18f026fdb4c45b68+        .quad   0x513b5384b5d2ecbd++        // 2^44 * 8 * G++        .quad   0x46d46280c729989e+        .quad   0x4b93fbd05368a5dd+        .quad   0x63df3f81d1765a89+        .quad   0x34cebd64b9a0a223+        .quad   0x5e2702878af34ceb+        .quad   0x900b0409b946d6ae+        .quad   0x6512ebf7dabd8512+        .quad   0x61d9b76988258f81+        .quad   0xa6c5a71349b7d94b+        .quad   0xa3f3d15823eb9446+        .quad   0x0416fbd277484834+        .quad   0x69d45e6f2c70812f++        // 2^48 * 1 * G++        .quad   0xce16f74bc53c1431+        .quad   0x2b9725ce2072edde+        .quad   0xb8b9c36fb5b23ee7+        .quad   0x7e2e0e450b5cc908+        .quad   0x9fe62b434f460efb+        .quad   0xded303d4a63607d6+        .quad   0xf052210eb7a0da24+        .quad   0x237e7dbe00545b93+        .quad   0x013575ed6701b430+        .quad   0x231094e69f0bfd10+        .quad   0x75320f1583e47f22+        .quad   0x71afa699b11155e3++        // 2^48 * 2 * G++        .quad   0x65ce6f9b3953b61d+        .quad   0xc65839eaafa141e6+        .quad   0x0f435ffda9f759fe+        .quad   0x021142e9c2b1c28e+        .quad   0xea423c1c473b50d6+        .quad   0x51e87a1f3b38ef10+        .quad   0x9b84bf5fb2c9be95+        .quad   0x00731fbc78f89a1c+        .quad   0xe430c71848f81880+        .quad   0xbf960c225ecec119+        .quad   0xb6dae0836bba15e3+        .quad   0x4c4d6f3347e15808++        // 2^48 * 3 * G++        .quad   0x18f7eccfc17d1fc9+        .quad   0x6c75f5a651403c14+        .quad   0xdbde712bf7ee0cdf+        .quad   0x193fddaaa7e47a22+        .quad   0x2f0cddfc988f1970+        .quad   0x6b916227b0b9f51b+        .quad   0x6ec7b6c4779176be+        .quad   0x38bf9500a88f9fa8+        .quad   0x1fd2c93c37e8876f+        .quad   0xa2f61e5a18d1462c+        .quad   0x5080f58239241276+        .quad   0x6a6fb99ebf0d4969++        // 2^48 * 4 * G++        .quad   0x6a46c1bb560855eb+        .quad   0x2416bb38f893f09d+        .quad   0xd71d11378f71acc1+        .quad   0x75f76914a31896ea+        .quad   0xeeb122b5b6e423c6+        .quad   0x939d7010f286ff8e+        .quad   0x90a92a831dcf5d8c+        .quad   0x136fda9f42c5eb10+        .quad   0xf94cdfb1a305bdd1+        .quad   0x0f364b9d9ff82c08+        .quad   0x2a87d8a5c3bb588a+        .quad   0x022183510be8dcba++        // 2^48 * 5 * G++        .quad   0x4af766385ead2d14+        .quad   0xa08ed880ca7c5830+        .quad   0x0d13a6e610211e3d+        .quad   0x6a071ce17b806c03+        .quad   0x9d5a710143307a7f+        .quad   0xb063de9ec47da45f+        .quad   0x22bbfe52be927ad3+        .quad   0x1387c441fd40426c+        .quad   0xb5d3c3d187978af8+        .quad   0x722b5a3d7f0e4413+        .quad   0x0d7b4848bb477ca0+        .quad   0x3171b26aaf1edc92++        // 2^48 * 6 * G++        .quad   0xa92f319097564ca8+        .quad   0xff7bb84c2275e119+        .quad   0x4f55fe37a4875150+        .quad   0x221fd4873cf0835a+        .quad   0xa60db7d8b28a47d1+        .quad   0xa6bf14d61770a4f1+        .quad   0xd4a1f89353ddbd58+        .quad   0x6c514a63344243e9+        .quad   0x2322204f3a156341+        .quad   0xfb73e0e9ba0a032d+        .quad   0xfce0dd4c410f030e+        .quad   0x48daa596fb924aaa++        // 2^48 * 7 * G++        .quad   0x6eca8e665ca59cc7+        .quad   0xa847254b2e38aca0+        .quad   0x31afc708d21e17ce+        .quad   0x676dd6fccad84af7+        .quad   0x14f61d5dc84c9793+        .quad   0x9941f9e3ef418206+        .quad   0xcdf5b88f346277ac+        .quad   0x58c837fa0e8a79a9+        .quad   0x0cf9688596fc9058+        .quad   0x1ddcbbf37b56a01b+        .quad   0xdcc2e77d4935d66a+        .quad   0x1c4f73f2c6a57f0a++        // 2^48 * 8 * G++        .quad   0x0e7a4fbd305fa0bb+        .quad   0x829d4ce054c663ad+        .quad   0xf421c3832fe33848+        .quad   0x795ac80d1bf64c42+        .quad   0xb36e706efc7c3484+        .quad   0x73dfc9b4c3c1cf61+        .quad   0xeb1d79c9781cc7e5+        .quad   0x70459adb7daf675c+        .quad   0x1b91db4991b42bb3+        .quad   0x572696234b02dcca+        .quad   0x9fdf9ee51f8c78dc+        .quad   0x5fe162848ce21fd3++        // 2^52 * 1 * G++        .quad   0xe2790aae4d077c41+        .quad   0x8b938270db7469a3+        .quad   0x6eb632dc8abd16a2+        .quad   0x720814ecaa064b72+        .quad   0x315c29c795115389+        .quad   0xd7e0e507862f74ce+        .quad   0x0c4a762185927432+        .quad   0x72de6c984a25a1e4+        .quad   0xae9ab553bf6aa310+        .quad   0x050a50a9806d6e1b+        .quad   0x92bb7403adff5139+        .quad   0x0394d27645be618b++        // 2^52 * 2 * G++        .quad   0x4d572251857eedf4+        .quad   0xe3724edde19e93c5+        .quad   0x8a71420e0b797035+        .quad   0x3b3c833687abe743+        .quad   0xf5396425b23545a4+        .quad   0x15a7a27e98fbb296+        .quad   0xab6c52bc636fdd86+        .quad   0x79d995a8419334ee+        .quad   0xcd8a8ea61195dd75+        .quad   0xa504d8a81dd9a82f+        .quad   0x540dca81a35879b6+        .quad   0x60dd16a379c86a8a++        // 2^52 * 3 * G++        .quad   0x35a2c8487381e559+        .quad   0x596ffea6d78082cb+        .quad   0xcb9771ebdba7b653+        .quad   0x5a08b5019b4da685+        .quad   0x3501d6f8153e47b8+        .quad   0xb7a9675414a2f60c+        .quad   0x112ee8b6455d9523+        .quad   0x4e62a3c18112ea8a+        .quad   0xc8d4ac04516ab786+        .quad   0x595af3215295b23d+        .quad   0xd6edd234db0230c1+        .quad   0x0929efe8825b41cc++        // 2^52 * 4 * G++        .quad   0x5f0601d1cbd0f2d3+        .quad   0x736e412f6132bb7f+        .quad   0x83604432238dde87+        .quad   0x1e3a5272f5c0753c+        .quad   0x8b3172b7ad56651d+        .quad   0x01581b7a3fabd717+        .quad   0x2dc94df6424df6e4+        .quad   0x30376e5d2c29284f+        .quad   0xd2918da78159a59c+        .quad   0x6bdc1cd93f0713f3+        .quad   0x565f7a934acd6590+        .quad   0x53daacec4cb4c128++        // 2^52 * 5 * G++        .quad   0x4ca73bd79cc8a7d6+        .quad   0x4d4a738f47e9a9b2+        .quad   0xf4cbf12942f5fe00+        .quad   0x01a13ff9bdbf0752+        .quad   0x99852bc3852cfdb0+        .quad   0x2cc12e9559d6ed0b+        .quad   0x70f9e2bf9b5ac27b+        .quad   0x4f3b8c117959ae99+        .quad   0x55b6c9c82ff26412+        .quad   0x1ac4a8c91fb667a8+        .quad   0xd527bfcfeb778bf2+        .quad   0x303337da7012a3be++        // 2^52 * 6 * G++        .quad   0x955422228c1c9d7c+        .quad   0x01fac1371a9b340f+        .quad   0x7e8d9177925b48d7+        .quad   0x53f8ad5661b3e31b+        .quad   0x976d3ccbfad2fdd1+        .quad   0xcb88839737a640a8+        .quad   0x2ff00c1d6734cb25+        .quad   0x269ff4dc789c2d2b+        .quad   0x0c003fbdc08d678d+        .quad   0x4d982fa37ead2b17+        .quad   0xc07e6bcdb2e582f1+        .quad   0x296c7291df412a44++        // 2^52 * 7 * G++        .quad   0x7903de2b33daf397+        .quad   0xd0ff0619c9a624b3+        .quad   0x8a1d252b555b3e18+        .quad   0x2b6d581c52e0b7c0+        .quad   0xdfb23205dab8b59e+        .quad   0x465aeaa0c8092250+        .quad   0xd133c1189a725d18+        .quad   0x2327370261f117d1+        .quad   0x3d0543d3623e7986+        .quad   0x679414c2c278a354+        .quad   0xae43f0cc726196f6+        .quad   0x7836c41f8245eaba++        // 2^52 * 8 * G++        .quad   0xe7a254db49e95a81+        .quad   0x5192d5d008b0ad73+        .quad   0x4d20e5b1d00afc07+        .quad   0x5d55f8012cf25f38+        .quad   0xca651e848011937c+        .quad   0xc6b0c46e6ef41a28+        .quad   0xb7021ba75f3f8d52+        .quad   0x119dff99ead7b9fd+        .quad   0x43eadfcbf4b31d4d+        .quad   0xc6503f7411148892+        .quad   0xfeee68c5060d3b17+        .quad   0x329293b3dd4a0ac8++        // 2^56 * 1 * G++        .quad   0x4e59214fe194961a+        .quad   0x49be7dc70d71cd4f+        .quad   0x9300cfd23b50f22d+        .quad   0x4789d446fc917232+        .quad   0x2879852d5d7cb208+        .quad   0xb8dedd70687df2e7+        .quad   0xdc0bffab21687891+        .quad   0x2b44c043677daa35+        .quad   0x1a1c87ab074eb78e+        .quad   0xfac6d18e99daf467+        .quad   0x3eacbbcd484f9067+        .quad   0x60c52eef2bb9a4e4++        // 2^56 * 2 * G++        .quad   0x0b5d89bc3bfd8bf1+        .quad   0xb06b9237c9f3551a+        .quad   0x0e4c16b0d53028f5+        .quad   0x10bc9c312ccfcaab+        .quad   0x702bc5c27cae6d11+        .quad   0x44c7699b54a48cab+        .quad   0xefbc4056ba492eb2+        .quad   0x70d77248d9b6676d+        .quad   0xaa8ae84b3ec2a05b+        .quad   0x98699ef4ed1781e0+        .quad   0x794513e4708e85d1+        .quad   0x63755bd3a976f413++        // 2^56 * 3 * G++        .quad   0xb55fa03e2ad10853+        .quad   0x356f75909ee63569+        .quad   0x9ff9f1fdbe69b890+        .quad   0x0d8cc1c48bc16f84+        .quad   0x3dc7101897f1acb7+        .quad   0x5dda7d5ec165bbd8+        .quad   0x508e5b9c0fa1020f+        .quad   0x2763751737c52a56+        .quad   0x029402d36eb419a9+        .quad   0xf0b44e7e77b460a5+        .quad   0xcfa86230d43c4956+        .quad   0x70c2dd8a7ad166e7++        // 2^56 * 4 * G++        .quad   0x656194509f6fec0e+        .quad   0xee2e7ea946c6518d+        .quad   0x9733c1f367e09b5c+        .quad   0x2e0fac6363948495+        .quad   0x91d4967db8ed7e13+        .quad   0x74252f0ad776817a+        .quad   0xe40982e00d852564+        .quad   0x32b8613816a53ce5+        .quad   0x79e7f7bee448cd64+        .quad   0x6ac83a67087886d0+        .quad   0xf89fd4d9a0e4db2e+        .quad   0x4179215c735a4f41++        // 2^56 * 5 * G++        .quad   0x8c7094e7d7dced2a+        .quad   0x97fb8ac347d39c70+        .quad   0xe13be033a906d902+        .quad   0x700344a30cd99d76+        .quad   0xe4ae33b9286bcd34+        .quad   0xb7ef7eb6559dd6dc+        .quad   0x278b141fb3d38e1f+        .quad   0x31fa85662241c286+        .quad   0xaf826c422e3622f4+        .quad   0xc12029879833502d+        .quad   0x9bc1b7e12b389123+        .quad   0x24bb2312a9952489++        // 2^56 * 6 * G++        .quad   0xb1a8ed1732de67c3+        .quad   0x3cb49418461b4948+        .quad   0x8ebd434376cfbcd2+        .quad   0x0fee3e871e188008+        .quad   0x41f80c2af5f85c6b+        .quad   0x687284c304fa6794+        .quad   0x8945df99a3ba1bad+        .quad   0x0d1d2af9ffeb5d16+        .quad   0xa9da8aa132621edf+        .quad   0x30b822a159226579+        .quad   0x4004197ba79ac193+        .quad   0x16acd79718531d76++        // 2^56 * 7 * G++        .quad   0x72df72af2d9b1d3d+        .quad   0x63462a36a432245a+        .quad   0x3ecea07916b39637+        .quad   0x123e0ef6b9302309+        .quad   0xc959c6c57887b6ad+        .quad   0x94e19ead5f90feba+        .quad   0x16e24e62a342f504+        .quad   0x164ed34b18161700+        .quad   0x487ed94c192fe69a+        .quad   0x61ae2cea3a911513+        .quad   0x877bf6d3b9a4de27+        .quad   0x78da0fc61073f3eb++        // 2^56 * 8 * G++        .quad   0x5bf15d28e52bc66a+        .quad   0x2c47e31870f01a8e+        .quad   0x2419afbc06c28bdd+        .quad   0x2d25deeb256b173a+        .quad   0xa29f80f1680c3a94+        .quad   0x71f77e151ae9e7e6+        .quad   0x1100f15848017973+        .quad   0x054aa4b316b38ddd+        .quad   0xdfc8468d19267cb8+        .quad   0x0b28789c66e54daf+        .quad   0x2aeb1d2a666eec17+        .quad   0x134610a6ab7da760++        // 2^60 * 1 * G++        .quad   0xcaf55ec27c59b23f+        .quad   0x99aeed3e154d04f2+        .quad   0x68441d72e14141f4+        .quad   0x140345133932a0a2+        .quad   0xd91430e0dc028c3c+        .quad   0x0eb955a85217c771+        .quad   0x4b09e1ed2c99a1fa+        .quad   0x42881af2bd6a743c+        .quad   0x7bfec69aab5cad3d+        .quad   0xc23e8cd34cb2cfad+        .quad   0x685dd14bfb37d6a2+        .quad   0x0ad6d64415677a18++        // 2^60 * 2 * G++        .quad   0x781a439e417becb5+        .quad   0x4ac5938cd10e0266+        .quad   0x5da385110692ac24+        .quad   0x11b065a2ade31233+        .quad   0x7914892847927e9f+        .quad   0x33dad6ef370aa877+        .quad   0x1f8f24fa11122703+        .quad   0x5265ac2f2adf9592+        .quad   0x405fdd309afcb346+        .quad   0xd9723d4428e63f54+        .quad   0x94c01df05f65aaae+        .quad   0x43e4dc3ae14c0809++        // 2^60 * 3 * G++        .quad   0xbc12c7f1a938a517+        .quad   0x473028ab3180b2e1+        .quad   0x3f78571efbcd254a+        .quad   0x74e534426ff6f90f+        .quad   0xea6f7ac3adc2c6a3+        .quad   0xd0e928f6e9717c94+        .quad   0xe2d379ead645eaf5+        .quad   0x46dd8785c51ffbbe+        .quad   0x709801be375c8898+        .quad   0x4b06dab5e3fd8348+        .quad   0x75880ced27230714+        .quad   0x2b09468fdd2f4c42++        // 2^60 * 4 * G++        .quad   0x97c749eeb701cb96+        .quad   0x83f438d4b6a369c3+        .quad   0x62962b8b9a402cd9+        .quad   0x6976c7509888df7b+        .quad   0x5b97946582ffa02a+        .quad   0xda096a51fea8f549+        .quad   0xa06351375f77af9b+        .quad   0x1bcfde61201d1e76+        .quad   0x4a4a5490246a59a2+        .quad   0xd63ebddee87fdd90+        .quad   0xd9437c670d2371fa+        .quad   0x69e87308d30f8ed6++        // 2^60 * 5 * G++        .quad   0x435a8bb15656beb0+        .quad   0xf8fac9ba4f4d5bca+        .quad   0xb9b278c41548c075+        .quad   0x3eb0ef76e892b622+        .quad   0x0f80bf028bc80303+        .quad   0x6aae16b37a18cefb+        .quad   0xdd47ea47d72cd6a3+        .quad   0x61943588f4ed39aa+        .quad   0xd26e5c3e91039f85+        .quad   0xc0e9e77df6f33aa9+        .quad   0xe8968c5570066a93+        .quad   0x3c34d1881faaaddd++        // 2^60 * 6 * G++        .quad   0x3f9d2b5ea09f9ec0+        .quad   0x1dab3b6fb623a890+        .quad   0xa09ba3ea72d926c4+        .quad   0x374193513fd8b36d+        .quad   0xbd5b0b8f2fffe0d9+        .quad   0x6aa254103ed24fb9+        .quad   0x2ac7d7bcb26821c4+        .quad   0x605b394b60dca36a+        .quad   0xb4e856e45a9d1ed2+        .quad   0xefe848766c97a9a2+        .quad   0xb104cf641e5eee7d+        .quad   0x2f50b81c88a71c8f++        // 2^60 * 7 * G++        .quad   0x31723c61fc6811bb+        .quad   0x9cb450486211800f+        .quad   0x768933d347995753+        .quad   0x3491a53502752fcd+        .quad   0x2b552ca0a7da522a+        .quad   0x3230b336449b0250+        .quad   0xf2c4c5bca4b99fb9+        .quad   0x7b2c674958074a22+        .quad   0xd55165883ed28cdf+        .quad   0x12d84fd2d362de39+        .quad   0x0a874ad3e3378e4f+        .quad   0x000d2b1f7c763e74++        // 2^60 * 8 * G++        .quad   0x3d420811d06d4a67+        .quad   0xbefc048590e0ffe3+        .quad   0xf870c6b7bd487bde+        .quad   0x6e2a7316319afa28+        .quad   0x9624778c3e94a8ab+        .quad   0x0ad6f3cee9a78bec+        .quad   0x948ac7810d743c4f+        .quad   0x76627935aaecfccc+        .quad   0x56a8ac24d6d59a9f+        .quad   0xc8db753e3096f006+        .quad   0x477f41e68f4c5299+        .quad   0x588d851cf6c86114++        // 2^64 * 1 * G++        .quad   0x51138ec78df6b0fe+        .quad   0x5397da89e575f51b+        .quad   0x09207a1d717af1b9+        .quad   0x2102fdba2b20d650+        .quad   0xcd2a65e777d1f515+        .quad   0x548991878faa60f1+        .quad   0xb1b73bbcdabc06e5+        .quad   0x654878cba97cc9fb+        .quad   0x969ee405055ce6a1+        .quad   0x36bca7681251ad29+        .quad   0x3a1af517aa7da415+        .quad   0x0ad725db29ecb2ba++        // 2^64 * 2 * G++        .quad   0xdc4267b1834e2457+        .quad   0xb67544b570ce1bc5+        .quad   0x1af07a0bf7d15ed7+        .quad   0x4aefcffb71a03650+        .quad   0xfec7bc0c9b056f85+        .quad   0x537d5268e7f5ffd7+        .quad   0x77afc6624312aefa+        .quad   0x4f675f5302399fd9+        .quad   0xc32d36360415171e+        .quad   0xcd2bef118998483b+        .quad   0x870a6eadd0945110+        .quad   0x0bccbb72a2a86561++        // 2^64 * 3 * G++        .quad   0x185e962feab1a9c8+        .quad   0x86e7e63565147dcd+        .quad   0xb092e031bb5b6df2+        .quad   0x4024f0ab59d6b73e+        .quad   0x186d5e4c50fe1296+        .quad   0xe0397b82fee89f7e+        .quad   0x3bc7f6c5507031b0+        .quad   0x6678fd69108f37c2+        .quad   0x1586fa31636863c2+        .quad   0x07f68c48572d33f2+        .quad   0x4f73cc9f789eaefc+        .quad   0x2d42e2108ead4701++        // 2^64 * 4 * G++        .quad   0x97f5131594dfd29b+        .quad   0x6155985d313f4c6a+        .quad   0xeba13f0708455010+        .quad   0x676b2608b8d2d322+        .quad   0x21717b0d0f537593+        .quad   0x914e690b131e064c+        .quad   0x1bb687ae752ae09f+        .quad   0x420bf3a79b423c6e+        .quad   0x8138ba651c5b2b47+        .quad   0x8671b6ec311b1b80+        .quad   0x7bff0cb1bc3135b0+        .quad   0x745d2ffa9c0cf1e0++        // 2^64 * 5 * G++        .quad   0xbf525a1e2bc9c8bd+        .quad   0xea5b260826479d81+        .quad   0xd511c70edf0155db+        .quad   0x1ae23ceb960cf5d0+        .quad   0x6036df5721d34e6a+        .quad   0xb1db8827997bb3d0+        .quad   0xd3c209c3c8756afa+        .quad   0x06e15be54c1dc839+        .quad   0x5b725d871932994a+        .quad   0x32351cb5ceb1dab0+        .quad   0x7dc41549dab7ca05+        .quad   0x58ded861278ec1f7++        // 2^64 * 6 * G++        .quad   0xd8173793f266c55c+        .quad   0xc8c976c5cc454e49+        .quad   0x5ce382f8bc26c3a8+        .quad   0x2ff39de85485f6f9+        .quad   0x2dfb5ba8b6c2c9a8+        .quad   0x48eeef8ef52c598c+        .quad   0x33809107f12d1573+        .quad   0x08ba696b531d5bd8+        .quad   0x77ed3eeec3efc57a+        .quad   0x04e05517d4ff4811+        .quad   0xea3d7a3ff1a671cb+        .quad   0x120633b4947cfe54++        // 2^64 * 7 * G++        .quad   0x0b94987891610042+        .quad   0x4ee7b13cecebfae8+        .quad   0x70be739594f0a4c0+        .quad   0x35d30a99b4d59185+        .quad   0x82bd31474912100a+        .quad   0xde237b6d7e6fbe06+        .quad   0xe11e761911ea79c6+        .quad   0x07433be3cb393bde+        .quad   0xff7944c05ce997f4+        .quad   0x575d3de4b05c51a3+        .quad   0x583381fd5a76847c+        .quad   0x2d873ede7af6da9f++        // 2^64 * 8 * G++        .quad   0x157a316443373409+        .quad   0xfab8b7eef4aa81d9+        .quad   0xb093fee6f5a64806+        .quad   0x2e773654707fa7b6+        .quad   0xaa6202e14e5df981+        .quad   0xa20d59175015e1f5+        .quad   0x18a275d3bae21d6c+        .quad   0x0543618a01600253+        .quad   0x0deabdf4974c23c1+        .quad   0xaa6f0a259dce4693+        .quad   0x04202cb8a29aba2c+        .quad   0x4b1443362d07960d++        // 2^68 * 1 * G++        .quad   0x47b837f753242cec+        .quad   0x256dc48cc04212f2+        .quad   0xe222fbfbe1d928c5+        .quad   0x48ea295bad8a2c07+        .quad   0x299b1c3f57c5715e+        .quad   0x96cb929e6b686d90+        .quad   0x3004806447235ab3+        .quad   0x2c435c24a44d9fe1+        .quad   0x0607c97c80f8833f+        .quad   0x0e851578ca25ec5b+        .quad   0x54f7450b161ebb6f+        .quad   0x7bcb4792a0def80e++        // 2^68 * 2 * G++        .quad   0x8487e3d02bc73659+        .quad   0x4baf8445059979df+        .quad   0xd17c975adcad6fbf+        .quad   0x57369f0bdefc96b6+        .quad   0x1cecd0a0045224c2+        .quad   0x757f1b1b69e53952+        .quad   0x775b7a925289f681+        .quad   0x1b6cc62016736148+        .quad   0xf1a9990175638698+        .quad   0x353dd1beeeaa60d3+        .quad   0x849471334c9ba488+        .quad   0x63fa6e6843ade311++        // 2^68 * 3 * G++        .quad   0xd15c20536597c168+        .quad   0x9f73740098d28789+        .quad   0x18aee7f13257ba1f+        .quad   0x3418bfda07346f14+        .quad   0x2195becdd24b5eb7+        .quad   0x5e41f18cc0cd44f9+        .quad   0xdf28074441ca9ede+        .quad   0x07073b98f35b7d67+        .quad   0xd03c676c4ce530d4+        .quad   0x0b64c0473b5df9f4+        .quad   0x065cef8b19b3a31e+        .quad   0x3084d661533102c9++        // 2^68 * 4 * G++        .quad   0xe1f6b79ebf8469ad+        .quad   0x15801004e2663135+        .quad   0x9a498330af74181b+        .quad   0x3ba2504f049b673c+        .quad   0x9a6ce876760321fd+        .quad   0x7fe2b5109eb63ad8+        .quad   0x00e7d4ae8ac80592+        .quad   0x73d86b7abb6f723a+        .quad   0x0b52b5606dba5ab6+        .quad   0xa9134f0fbbb1edab+        .quad   0x30a9520d9b04a635+        .quad   0x6813b8f37973e5db++        // 2^68 * 5 * G++        .quad   0x9854b054334127c1+        .quad   0x105d047882fbff25+        .quad   0xdb49f7f944186f4f+        .quad   0x1768e838bed0b900+        .quad   0xf194ca56f3157e29+        .quad   0x136d35705ef528a5+        .quad   0xdd4cef778b0599bc+        .quad   0x7d5472af24f833ed+        .quad   0xd0ef874daf33da47+        .quad   0x00d3be5db6e339f9+        .quad   0x3f2a8a2f9c9ceece+        .quad   0x5d1aeb792352435a++        // 2^68 * 6 * G++        .quad   0xf59e6bb319cd63ca+        .quad   0x670c159221d06839+        .quad   0xb06d565b2150cab6+        .quad   0x20fb199d104f12a3+        .quad   0x12c7bfaeb61ba775+        .quad   0xb84e621fe263bffd+        .quad   0x0b47a5c35c840dcf+        .quad   0x7e83be0bccaf8634+        .quad   0x61943dee6d99c120+        .quad   0x86101f2e460b9fe0+        .quad   0x6bb2f1518ee8598d+        .quad   0x76b76289fcc475cc++        // 2^68 * 7 * G++        .quad   0x791b4cc1756286fa+        .quad   0xdbced317d74a157c+        .quad   0x7e732421ea72bde6+        .quad   0x01fe18491131c8e9+        .quad   0x4245f1a1522ec0b3+        .quad   0x558785b22a75656d+        .quad   0x1d485a2548a1b3c0+        .quad   0x60959eccd58fe09f+        .quad   0x3ebfeb7ba8ed7a09+        .quad   0x49fdc2bbe502789c+        .quad   0x44ebce5d3c119428+        .quad   0x35e1eb55be947f4a++        // 2^68 * 8 * G++        .quad   0xdbdae701c5738dd3+        .quad   0xf9c6f635b26f1bee+        .quad   0x61e96a8042f15ef4+        .quad   0x3aa1d11faf60a4d8+        .quad   0x14fd6dfa726ccc74+        .quad   0x3b084cfe2f53b965+        .quad   0xf33ae4f552a2c8b4+        .quad   0x59aab07a0d40166a+        .quad   0x77bcec4c925eac25+        .quad   0x1848718460137738+        .quad   0x5b374337fea9f451+        .quad   0x1865e78ec8e6aa46++        // 2^72 * 1 * G++        .quad   0xccc4b7c7b66e1f7a+        .quad   0x44157e25f50c2f7e+        .quad   0x3ef06dfc713eaf1c+        .quad   0x582f446752da63f7+        .quad   0x967c54e91c529ccb+        .quad   0x30f6269264c635fb+        .quad   0x2747aff478121965+        .quad   0x17038418eaf66f5c+        .quad   0xc6317bd320324ce4+        .quad   0xa81042e8a4488bc4+        .quad   0xb21ef18b4e5a1364+        .quad   0x0c2a1c4bcda28dc9++        // 2^72 * 2 * G++        .quad   0xd24dc7d06f1f0447+        .quad   0xb2269e3edb87c059+        .quad   0xd15b0272fbb2d28f+        .quad   0x7c558bd1c6f64877+        .quad   0xedc4814869bd6945+        .quad   0x0d6d907dbe1c8d22+        .quad   0xc63bd212d55cc5ab+        .quad   0x5a6a9b30a314dc83+        .quad   0xd0ec1524d396463d+        .quad   0x12bb628ac35a24f0+        .quad   0xa50c3a791cbc5fa4+        .quad   0x0404a5ca0afbafc3++        // 2^72 * 3 * G++        .quad   0x8c1f40070aa743d6+        .quad   0xccbad0cb5b265ee8+        .quad   0x574b046b668fd2de+        .quad   0x46395bfdcadd9633+        .quad   0x62bc9e1b2a416fd1+        .quad   0xb5c6f728e350598b+        .quad   0x04343fd83d5d6967+        .quad   0x39527516e7f8ee98+        .quad   0x117fdb2d1a5d9a9c+        .quad   0x9c7745bcd1005c2a+        .quad   0xefd4bef154d56fea+        .quad   0x76579a29e822d016++        // 2^72 * 4 * G++        .quad   0x45b68e7e49c02a17+        .quad   0x23cd51a2bca9a37f+        .quad   0x3ed65f11ec224c1b+        .quad   0x43a384dc9e05bdb1+        .quad   0x333cb51352b434f2+        .quad   0xd832284993de80e1+        .quad   0xb5512887750d35ce+        .quad   0x02c514bb2a2777c1+        .quad   0x684bd5da8bf1b645+        .quad   0xfb8bd37ef6b54b53+        .quad   0x313916d7a9b0d253+        .quad   0x1160920961548059++        // 2^72 * 5 * G++        .quad   0xb44d166929dacfaa+        .quad   0xda529f4c8413598f+        .quad   0xe9ef63ca453d5559+        .quad   0x351e125bc5698e0b+        .quad   0x7a385616369b4dcd+        .quad   0x75c02ca7655c3563+        .quad   0x7dc21bf9d4f18021+        .quad   0x2f637d7491e6e042+        .quad   0xd4b49b461af67bbe+        .quad   0xd603037ac8ab8961+        .quad   0x71dee19ff9a699fb+        .quad   0x7f182d06e7ce2a9a++        // 2^72 * 6 * G++        .quad   0x7a7c8e64ab0168ec+        .quad   0xcb5a4a5515edc543+        .quad   0x095519d347cd0eda+        .quad   0x67d4ac8c343e93b0+        .quad   0x09454b728e217522+        .quad   0xaa58e8f4d484b8d8+        .quad   0xd358254d7f46903c+        .quad   0x44acc043241c5217+        .quad   0x1c7d6bbb4f7a5777+        .quad   0x8b35fed4918313e1+        .quad   0x4adca1c6c96b4684+        .quad   0x556d1c8312ad71bd++        // 2^72 * 7 * G++        .quad   0x17ef40e30c8d3982+        .quad   0x31f7073e15a3fa34+        .quad   0x4f21f3cb0773646e+        .quad   0x746c6c6d1d824eff+        .quad   0x81f06756b11be821+        .quad   0x0faff82310a3f3dd+        .quad   0xf8b2d0556a99465d+        .quad   0x097abe38cc8c7f05+        .quad   0x0c49c9877ea52da4+        .quad   0x4c4369559bdc1d43+        .quad   0x022c3809f7ccebd2+        .quad   0x577e14a34bee84bd++        // 2^72 * 8 * G++        .quad   0xf0e268ac61a73b0a+        .quad   0xf2fafa103791a5f5+        .quad   0xc1e13e826b6d00e9+        .quad   0x60fa7ee96fd78f42+        .quad   0x94fecebebd4dd72b+        .quad   0xf46a4fda060f2211+        .quad   0x124a5977c0c8d1ff+        .quad   0x705304b8fb009295+        .quad   0xb63d1d354d296ec6+        .quad   0xf3c3053e5fad31d8+        .quad   0x670b958cb4bd42ec+        .quad   0x21398e0ca16353fd++        // 2^76 * 1 * G++        .quad   0x216ab2ca8da7d2ef+        .quad   0x366ad9dd99f42827+        .quad   0xae64b9004fdd3c75+        .quad   0x403a395b53909e62+        .quad   0x86c5fc16861b7e9a+        .quad   0xf6a330476a27c451+        .quad   0x01667267a1e93597+        .quad   0x05ffb9cd6082dfeb+        .quad   0xa617fa9ff53f6139+        .quad   0x60f2b5e513e66cb6+        .quad   0xd7a8beefb3448aa4+        .quad   0x7a2932856f5ea192++        // 2^76 * 2 * G++        .quad   0x0b39d761b02de888+        .quad   0x5f550e7ed2414e1f+        .quad   0xa6bfa45822e1a940+        .quad   0x050a2f7dfd447b99+        .quad   0xb89c444879639302+        .quad   0x4ae4f19350c67f2c+        .quad   0xf0b35da8c81af9c6+        .quad   0x39d0003546871017+        .quad   0x437c3b33a650db77+        .quad   0x6bafe81dbac52bb2+        .quad   0xfe99402d2db7d318+        .quad   0x2b5b7eec372ba6ce++        // 2^76 * 3 * G++        .quad   0xb3bc4bbd83f50eef+        .quad   0x508f0c998c927866+        .quad   0x43e76587c8b7e66e+        .quad   0x0f7655a3a47f98d9+        .quad   0xa694404d613ac8f4+        .quad   0x500c3c2bfa97e72c+        .quad   0x874104d21fcec210+        .quad   0x1b205fb38604a8ee+        .quad   0x55ecad37d24b133c+        .quad   0x441e147d6038c90b+        .quad   0x656683a1d62c6fee+        .quad   0x0157d5dc87e0ecae++        // 2^76 * 4 * G++        .quad   0xf2a7af510354c13d+        .quad   0xd7a0b145aa372b60+        .quad   0x2869b96a05a3d470+        .quad   0x6528e42d82460173+        .quad   0x95265514d71eb524+        .quad   0xe603d8815df14593+        .quad   0x147cdf410d4de6b7+        .quad   0x5293b1730437c850+        .quad   0x23d0e0814bccf226+        .quad   0x92c745cd8196fb93+        .quad   0x8b61796c59541e5b+        .quad   0x40a44df0c021f978++        // 2^76 * 5 * G++        .quad   0xdaa869894f20ea6a+        .quad   0xea14a3d14c620618+        .quad   0x6001fccb090bf8be+        .quad   0x35f4e822947e9cf0+        .quad   0x86c96e514bc5d095+        .quad   0xf20d4098fca6804a+        .quad   0x27363d89c826ea5d+        .quad   0x39ca36565719cacf+        .quad   0x97506f2f6f87b75c+        .quad   0xc624aea0034ae070+        .quad   0x1ec856e3aad34dd6+        .quad   0x055b0be0e440e58f++        // 2^76 * 6 * G++        .quad   0x6469a17d89735d12+        .quad   0xdb6f27d5e662b9f1+        .quad   0x9fcba3286a395681+        .quad   0x363b8004d269af25+        .quad   0x4d12a04b6ea33da2+        .quad   0x57cf4c15e36126dd+        .quad   0x90ec9675ee44d967+        .quad   0x64ca348d2a985aac+        .quad   0x99588e19e4c4912d+        .quad   0xefcc3b4e1ca5ce6b+        .quad   0x4522ea60fa5b98d5+        .quad   0x7064bbab1de4a819++        // 2^76 * 7 * G++        .quad   0xb919e1515a770641+        .quad   0xa9a2e2c74e7f8039+        .quad   0x7527250b3df23109+        .quad   0x756a7330ac27b78b+        .quad   0xa290c06142542129+        .quad   0xf2e2c2aebe8d5b90+        .quad   0xcf2458db76abfe1b+        .quad   0x02157ade83d626bf+        .quad   0x3e46972a1b9a038b+        .quad   0x2e4ee66a7ee03fb4+        .quad   0x81a248776edbb4ca+        .quad   0x1a944ee88ecd0563++        // 2^76 * 8 * G++        .quad   0xd5a91d1151039372+        .quad   0x2ed377b799ca26de+        .quad   0xa17202acfd366b6b+        .quad   0x0730291bd6901995+        .quad   0xbb40a859182362d6+        .quad   0xb99f55778a4d1abb+        .quad   0x8d18b427758559f6+        .quad   0x26c20fe74d26235a+        .quad   0x648d1d9fe9cc22f5+        .quad   0x66bc561928dd577c+        .quad   0x47d3ed21652439d1+        .quad   0x49d271acedaf8b49++        // 2^80 * 1 * G++        .quad   0x89f5058a382b33f3+        .quad   0x5ae2ba0bad48c0b4+        .quad   0x8f93b503a53db36e+        .quad   0x5aa3ed9d95a232e6+        .quad   0x2798aaf9b4b75601+        .quad   0x5eac72135c8dad72+        .quad   0xd2ceaa6161b7a023+        .quad   0x1bbfb284e98f7d4e+        .quad   0x656777e9c7d96561+        .quad   0xcb2b125472c78036+        .quad   0x65053299d9506eee+        .quad   0x4a07e14e5e8957cc++        // 2^80 * 2 * G++        .quad   0x4ee412cb980df999+        .quad   0xa315d76f3c6ec771+        .quad   0xbba5edde925c77fd+        .quad   0x3f0bac391d313402+        .quad   0x240b58cdc477a49b+        .quad   0xfd38dade6447f017+        .quad   0x19928d32a7c86aad+        .quad   0x50af7aed84afa081+        .quad   0x6e4fde0115f65be5+        .quad   0x29982621216109b2+        .quad   0x780205810badd6d9+        .quad   0x1921a316baebd006++        // 2^80 * 3 * G++        .quad   0x89422f7edfb870fc+        .quad   0x2c296beb4f76b3bd+        .quad   0x0738f1d436c24df7+        .quad   0x6458df41e273aeb0+        .quad   0xd75aad9ad9f3c18b+        .quad   0x566a0eef60b1c19c+        .quad   0x3e9a0bac255c0ed9+        .quad   0x7b049deca062c7f5+        .quad   0xdccbe37a35444483+        .quad   0x758879330fedbe93+        .quad   0x786004c312c5dd87+        .quad   0x6093dccbc2950e64++        // 2^80 * 4 * G++        .quad   0x1ff39a8585e0706d+        .quad   0x36d0a5d8b3e73933+        .quad   0x43b9f2e1718f453b+        .quad   0x57d1ea084827a97c+        .quad   0x6bdeeebe6084034b+        .quad   0x3199c2b6780fb854+        .quad   0x973376abb62d0695+        .quad   0x6e3180c98b647d90+        .quad   0xee7ab6e7a128b071+        .quad   0xa4c1596d93a88baa+        .quad   0xf7b4de82b2216130+        .quad   0x363e999ddd97bd18++        // 2^80 * 5 * G++        .quad   0x96a843c135ee1fc4+        .quad   0x976eb35508e4c8cf+        .quad   0xb42f6801b58cd330+        .quad   0x48ee9b78693a052b+        .quad   0x2f1848dce24baec6+        .quad   0x769b7255babcaf60+        .quad   0x90cb3c6e3cefe931+        .quad   0x231f979bc6f9b355+        .quad   0x5c31de4bcc2af3c6+        .quad   0xb04bb030fe208d1f+        .quad   0xb78d7009c14fb466+        .quad   0x079bfa9b08792413++        // 2^80 * 6 * G++        .quad   0xe3903a51da300df4+        .quad   0x843964233da95ab0+        .quad   0xed3cf12d0b356480+        .quad   0x038c77f684817194+        .quad   0xf3c9ed80a2d54245+        .quad   0x0aa08b7877f63952+        .quad   0xd76dac63d1085475+        .quad   0x1ef4fb159470636b+        .quad   0x854e5ee65b167bec+        .quad   0x59590a4296d0cdc2+        .quad   0x72b2df3498102199+        .quad   0x575ee92a4a0bff56++        // 2^80 * 7 * G++        .quad   0xd4c080908a182fcf+        .quad   0x30e170c299489dbd+        .quad   0x05babd5752f733de+        .quad   0x43d4e7112cd3fd00+        .quad   0x5d46bc450aa4d801+        .quad   0xc3af1227a533b9d8+        .quad   0x389e3b262b8906c2+        .quad   0x200a1e7e382f581b+        .quad   0x518db967eaf93ac5+        .quad   0x71bc989b056652c0+        .quad   0xfe2b85d9567197f5+        .quad   0x050eca52651e4e38++        // 2^80 * 8 * G++        .quad   0xc3431ade453f0c9c+        .quad   0xe9f5045eff703b9b+        .quad   0xfcd97ac9ed847b3d+        .quad   0x4b0ee6c21c58f4c6+        .quad   0x97ac397660e668ea+        .quad   0x9b19bbfe153ab497+        .quad   0x4cb179b534eca79f+        .quad   0x6151c09fa131ae57+        .quad   0x3af55c0dfdf05d96+        .quad   0xdd262ee02ab4ee7a+        .quad   0x11b2bb8712171709+        .quad   0x1fef24fa800f030b++        // 2^84 * 1 * G++        .quad   0xb496123a6b6c6609+        .quad   0xa750fe8580ab5938+        .quad   0xf471bf39b7c27a5f+        .quad   0x507903ce77ac193c+        .quad   0xff91a66a90166220+        .quad   0xf22552ae5bf1e009+        .quad   0x7dff85d87f90df7c+        .quad   0x4f620ffe0c736fb9+        .quad   0x62f90d65dfde3e34+        .quad   0xcf28c592b9fa5fad+        .quad   0x99c86ef9c6164510+        .quad   0x25d448044a256c84++        // 2^84 * 2 * G++        .quad   0xbd68230ec7e9b16f+        .quad   0x0eb1b9c1c1c5795d+        .quad   0x7943c8c495b6b1ff+        .quad   0x2f9faf620bbacf5e+        .quad   0x2c7c4415c9022b55+        .quad   0x56a0d241812eb1fe+        .quad   0xf02ea1c9d7b65e0d+        .quad   0x4180512fd5323b26+        .quad   0xa4ff3e698a48a5db+        .quad   0xba6a3806bd95403b+        .quad   0x9f7ce1af47d5b65d+        .quad   0x15e087e55939d2fb++        // 2^84 * 3 * G++        .quad   0x12207543745c1496+        .quad   0xdaff3cfdda38610c+        .quad   0xe4e797272c71c34f+        .quad   0x39c07b1934bdede9+        .quad   0x8894186efb963f38+        .quad   0x48a00e80dc639bd5+        .quad   0xa4e8092be96c1c99+        .quad   0x5a097d54ca573661+        .quad   0x2d45892b17c9e755+        .quad   0xd033fd7289308df8+        .quad   0x6c2fe9d9525b8bd9+        .quad   0x2edbecf1c11cc079++        // 2^84 * 4 * G++        .quad   0x1616a4e3c715a0d2+        .quad   0x53623cb0f8341d4d+        .quad   0x96ef5329c7e899cb+        .quad   0x3d4e8dbba668baa6+        .quad   0xee0f0fddd087a25f+        .quad   0x9c7531555c3e34ee+        .quad   0x660c572e8fab3ab5+        .quad   0x0854fc44544cd3b2+        .quad   0x61eba0c555edad19+        .quad   0x24b533fef0a83de6+        .quad   0x3b77042883baa5f8+        .quad   0x678f82b898a47e8d++        // 2^84 * 5 * G++        .quad   0xb1491d0bd6900c54+        .quad   0x3539722c9d132636+        .quad   0x4db928920b362bc9+        .quad   0x4d7cd1fea68b69df+        .quad   0x1e09d94057775696+        .quad   0xeed1265c3cd951db+        .quad   0xfa9dac2b20bce16f+        .quad   0x0f7f76e0e8d089f4+        .quad   0x36d9ebc5d485b00c+        .quad   0xa2596492e4adb365+        .quad   0xc1659480c2119ccd+        .quad   0x45306349186e0d5f++        // 2^84 * 6 * G++        .quad   0x94ddd0c1a6cdff1d+        .quad   0x55f6f115e84213ae+        .quad   0x6c935f85992fcf6a+        .quad   0x067ee0f54a37f16f+        .quad   0x96a414ec2b072491+        .quad   0x1bb2218127a7b65b+        .quad   0x6d2849596e8a4af0+        .quad   0x65f3b08ccd27765f+        .quad   0xecb29fff199801f7+        .quad   0x9d361d1fa2a0f72f+        .quad   0x25f11d2375fd2f49+        .quad   0x124cefe80fe10fe2++        // 2^84 * 7 * G++        .quad   0x4c126cf9d18df255+        .quad   0xc1d471e9147a63b6+        .quad   0x2c6d3c73f3c93b5f+        .quad   0x6be3a6a2e3ff86a2+        .quad   0x1518e85b31b16489+        .quad   0x8faadcb7db710bfb+        .quad   0x39b0bdf4a14ae239+        .quad   0x05f4cbea503d20c1+        .quad   0xce040e9ec04145bc+        .quad   0xc71ff4e208f6834c+        .quad   0xbd546e8dab8847a3+        .quad   0x64666aa0a4d2aba5++        // 2^84 * 8 * G++        .quad   0x6841435a7c06d912+        .quad   0xca123c21bb3f830b+        .quad   0xd4b37b27b1cbe278+        .quad   0x1d753b84c76f5046+        .quad   0xb0c53bf73337e94c+        .quad   0x7cb5697e11e14f15+        .quad   0x4b84abac1930c750+        .quad   0x28dd4abfe0640468+        .quad   0x7dc0b64c44cb9f44+        .quad   0x18a3e1ace3925dbf+        .quad   0x7a3034862d0457c4+        .quad   0x4c498bf78a0c892e++        // 2^88 * 1 * G++        .quad   0x37d653fb1aa73196+        .quad   0x0f9495303fd76418+        .quad   0xad200b09fb3a17b2+        .quad   0x544d49292fc8613e+        .quad   0x22d2aff530976b86+        .quad   0x8d90b806c2d24604+        .quad   0xdca1896c4de5bae5+        .quad   0x28005fe6c8340c17+        .quad   0x6aefba9f34528688+        .quad   0x5c1bff9425107da1+        .quad   0xf75bbbcd66d94b36+        .quad   0x72e472930f316dfa++        // 2^88 * 2 * G++        .quad   0x2695208c9781084f+        .quad   0xb1502a0b23450ee1+        .quad   0xfd9daea603efde02+        .quad   0x5a9d2e8c2733a34c+        .quad   0x07f3f635d32a7627+        .quad   0x7aaa4d865f6566f0+        .quad   0x3c85e79728d04450+        .quad   0x1fee7f000fe06438+        .quad   0x765305da03dbf7e5+        .quad   0xa4daf2491434cdbd+        .quad   0x7b4ad5cdd24a88ec+        .quad   0x00f94051ee040543++        // 2^88 * 3 * G++        .quad   0x8d356b23c3d330b2+        .quad   0xf21c8b9bb0471b06+        .quad   0xb36c316c6e42b83c+        .quad   0x07d79c7e8beab10d+        .quad   0xd7ef93bb07af9753+        .quad   0x583ed0cf3db766a7+        .quad   0xce6998bf6e0b1ec5+        .quad   0x47b7ffd25dd40452+        .quad   0x87fbfb9cbc08dd12+        .quad   0x8a066b3ae1eec29b+        .quad   0x0d57242bdb1fc1bf+        .quad   0x1c3520a35ea64bb6++        // 2^88 * 4 * G++        .quad   0x80d253a6bccba34a+        .quad   0x3e61c3a13838219b+        .quad   0x90c3b6019882e396+        .quad   0x1c3d05775d0ee66f+        .quad   0xcda86f40216bc059+        .quad   0x1fbb231d12bcd87e+        .quad   0xb4956a9e17c70990+        .quad   0x38750c3b66d12e55+        .quad   0x692ef1409422e51a+        .quad   0xcbc0c73c2b5df671+        .quad   0x21014fe7744ce029+        .quad   0x0621e2c7d330487c++        // 2^88 * 5 * G++        .quad   0xaf9860cc8259838d+        .quad   0x90ea48c1c69f9adc+        .quad   0x6526483765581e30+        .quad   0x0007d6097bd3a5bc+        .quad   0xb7ae1796b0dbf0f3+        .quad   0x54dfafb9e17ce196+        .quad   0x25923071e9aaa3b4+        .quad   0x5d8e589ca1002e9d+        .quad   0xc0bf1d950842a94b+        .quad   0xb2d3c363588f2e3e+        .quad   0x0a961438bb51e2ef+        .quad   0x1583d7783c1cbf86++        // 2^88 * 6 * G++        .quad   0xeceea2ef5da27ae1+        .quad   0x597c3a1455670174+        .quad   0xc9a62a126609167a+        .quad   0x252a5f2e81ed8f70+        .quad   0x90034704cc9d28c7+        .quad   0x1d1b679ef72cc58f+        .quad   0x16e12b5fbe5b8726+        .quad   0x4958064e83c5580a+        .quad   0x0d2894265066e80d+        .quad   0xfcc3f785307c8c6b+        .quad   0x1b53da780c1112fd+        .quad   0x079c170bd843b388++        // 2^88 * 7 * G++        .quad   0x0506ece464fa6fff+        .quad   0xbee3431e6205e523+        .quad   0x3579422451b8ea42+        .quad   0x6dec05e34ac9fb00+        .quad   0xcdd6cd50c0d5d056+        .quad   0x9af7686dbb03573b+        .quad   0x3ca6723ff3c3ef48+        .quad   0x6768c0d7317b8acc+        .quad   0x94b625e5f155c1b3+        .quad   0x417bf3a7997b7b91+        .quad   0xc22cbddc6d6b2600+        .quad   0x51445e14ddcd52f4++        // 2^88 * 8 * G++        .quad   0x57502b4b3b144951+        .quad   0x8e67ff6b444bbcb3+        .quad   0xb8bd6927166385db+        .quad   0x13186f31e39295c8+        .quad   0x893147ab2bbea455+        .quad   0x8c53a24f92079129+        .quad   0x4b49f948be30f7a7+        .quad   0x12e990086e4fd43d+        .quad   0xf10c96b37fdfbb2e+        .quad   0x9f9a935e121ceaf9+        .quad   0xdf1136c43a5b983f+        .quad   0x77b2e3f05d3e99af++        // 2^92 * 1 * G++        .quad   0xfd0d75879cf12657+        .quad   0xe82fef94e53a0e29+        .quad   0xcc34a7f05bbb4be7+        .quad   0x0b251172a50c38a2+        .quad   0x9532f48fcc5cd29b+        .quad   0x2ba851bea3ce3671+        .quad   0x32dacaa051122941+        .quad   0x478d99d9350004f2+        .quad   0x1d5ad94890bb02c0+        .quad   0x50e208b10ec25115+        .quad   0xa26a22894ef21702+        .quad   0x4dc923343b524805++        // 2^92 * 2 * G++        .quad   0xe3828c400f8086b6+        .quad   0x3f77e6f7979f0dc8+        .quad   0x7ef6de304df42cb4+        .quad   0x5265797cb6abd784+        .quad   0x3ad3e3ebf36c4975+        .quad   0xd75d25a537862125+        .quad   0xe873943da025a516+        .quad   0x6bbc7cb4c411c847+        .quad   0x3c6f9cd1d4a50d56+        .quad   0xb6244077c6feab7e+        .quad   0x6ff9bf483580972e+        .quad   0x00375883b332acfb++        // 2^92 * 3 * G++        .quad   0x0001b2cd28cb0940+        .quad   0x63fb51a06f1c24c9+        .quad   0xb5ad8691dcd5ca31+        .quad   0x67238dbd8c450660+        .quad   0xc98bec856c75c99c+        .quad   0xe44184c000e33cf4+        .quad   0x0a676b9bba907634+        .quad   0x669e2cb571f379d7+        .quad   0xcb116b73a49bd308+        .quad   0x025aad6b2392729e+        .quad   0xb4793efa3f55d9b1+        .quad   0x72a1056140678bb9++        // 2^92 * 4 * G++        .quad   0xa2b6812b1cc9249d+        .quad   0x62866eee21211f58+        .quad   0x2cb5c5b85df10ece+        .quad   0x03a6b259e263ae00+        .quad   0x0d8d2909e2e505b6+        .quad   0x98ca78abc0291230+        .quad   0x77ef5569a9b12327+        .quad   0x7c77897b81439b47+        .quad   0xf1c1b5e2de331cb5+        .quad   0x5a9f5d8e15fca420+        .quad   0x9fa438f17bd932b1+        .quad   0x2a381bf01c6146e7++        // 2^92 * 5 * G++        .quad   0xac9b9879cfc811c1+        .quad   0x8b7d29813756e567+        .quad   0x50da4e607c70edfc+        .quad   0x5dbca62f884400b6+        .quad   0xf7c0be32b534166f+        .quad   0x27e6ca6419cf70d4+        .quad   0x934df7d7a957a759+        .quad   0x5701461dabdec2aa+        .quad   0x2c6747402c915c25+        .quad   0x1bdcd1a80b0d340a+        .quad   0x5e5601bd07b43f5f+        .quad   0x2555b4e05539a242++        // 2^92 * 6 * G++        .quad   0x6fc09f5266ddd216+        .quad   0xdce560a7c8e37048+        .quad   0xec65939da2df62fd+        .quad   0x7a869ae7e52ed192+        .quad   0x78409b1d87e463d4+        .quad   0xad4da95acdfb639d+        .quad   0xec28773755259b9c+        .quad   0x69c806e9c31230ab+        .quad   0x7b48f57414bb3f22+        .quad   0x68c7cee4aedccc88+        .quad   0xed2f936179ed80be+        .quad   0x25d70b885f77bc4b++        // 2^92 * 7 * G++        .quad   0x4151c3d9762bf4de+        .quad   0x083f435f2745d82b+        .quad   0x29775a2e0d23ddd5+        .quad   0x138e3a6269a5db24+        .quad   0x98459d29bb1ae4d4+        .quad   0x56b9c4c739f954ec+        .quad   0x832743f6c29b4b3e+        .quad   0x21ea8e2798b6878a+        .quad   0x87bef4b46a5a7b9c+        .quad   0xd2299d1b5fc1d062+        .quad   0x82409818dd321648+        .quad   0x5c5abeb1e5a2e03d++        // 2^92 * 8 * G++        .quad   0x14722af4b73c2ddb+        .quad   0xbc470c5f5a05060d+        .quad   0x00943eac2581b02e+        .quad   0x0e434b3b1f499c8f+        .quad   0x02cde6de1306a233+        .quad   0x7b5a52a2116f8ec7+        .quad   0xe1c681f4c1163b5b+        .quad   0x241d350660d32643+        .quad   0x6be4404d0ebc52c7+        .quad   0xae46233bb1a791f5+        .quad   0x2aec170ed25db42b+        .quad   0x1d8dfd966645d694++        // 2^96 * 1 * G++        .quad   0x296fa9c59c2ec4de+        .quad   0xbc8b61bf4f84f3cb+        .quad   0x1c7706d917a8f908+        .quad   0x63b795fc7ad3255d+        .quad   0xd598639c12ddb0a4+        .quad   0xa5d19f30c024866b+        .quad   0xd17c2f0358fce460+        .quad   0x07a195152e095e8a+        .quad   0xa8368f02389e5fc8+        .quad   0x90433b02cf8de43b+        .quad   0xafa1fd5dc5412643+        .quad   0x3e8fe83d032f0137++        // 2^96 * 2 * G++        .quad   0x2f8b15b90570a294+        .quad   0x94f2427067084549+        .quad   0xde1c5ae161bbfd84+        .quad   0x75ba3b797fac4007+        .quad   0x08704c8de8efd13c+        .quad   0xdfc51a8e33e03731+        .quad   0xa59d5da51260cde3+        .quad   0x22d60899a6258c86+        .quad   0x6239dbc070cdd196+        .quad   0x60fe8a8b6c7d8a9a+        .quad   0xb38847bceb401260+        .quad   0x0904d07b87779e5e++        // 2^96 * 3 * G++        .quad   0xb4ce1fd4ddba919c+        .quad   0xcf31db3ec74c8daa+        .quad   0x2c63cc63ad86cc51+        .quad   0x43e2143fbc1dde07+        .quad   0xf4322d6648f940b9+        .quad   0x06952f0cbd2d0c39+        .quad   0x167697ada081f931+        .quad   0x6240aacebaf72a6c+        .quad   0xf834749c5ba295a0+        .quad   0xd6947c5bca37d25a+        .quad   0x66f13ba7e7c9316a+        .quad   0x56bdaf238db40cac++        // 2^96 * 4 * G++        .quad   0x362ab9e3f53533eb+        .quad   0x338568d56eb93d40+        .quad   0x9e0e14521d5a5572+        .quad   0x1d24a86d83741318+        .quad   0x1310d36cc19d3bb2+        .quad   0x062a6bb7622386b9+        .quad   0x7c9b8591d7a14f5c+        .quad   0x03aa31507e1e5754+        .quad   0xf4ec7648ffd4ce1f+        .quad   0xe045eaf054ac8c1c+        .quad   0x88d225821d09357c+        .quad   0x43b261dc9aeb4859++        // 2^96 * 5 * G++        .quad   0xe55b1e1988bb79bb+        .quad   0xa09ed07dc17a359d+        .quad   0xb02c2ee2603dea33+        .quad   0x326055cf5b276bc2+        .quad   0x19513d8b6c951364+        .quad   0x94fe7126000bf47b+        .quad   0x028d10ddd54f9567+        .quad   0x02b4d5e242940964+        .quad   0xb4a155cb28d18df2+        .quad   0xeacc4646186ce508+        .quad   0xc49cf4936c824389+        .quad   0x27a6c809ae5d3410++        // 2^96 * 6 * G++        .quad   0x8ba6ebcd1f0db188+        .quad   0x37d3d73a675a5be8+        .quad   0xf22edfa315f5585a+        .quad   0x2cb67174ff60a17e+        .quad   0xcd2c270ac43d6954+        .quad   0xdd4a3e576a66cab2+        .quad   0x79fa592469d7036c+        .quad   0x221503603d8c2599+        .quad   0x59eecdf9390be1d0+        .quad   0xa9422044728ce3f1+        .quad   0x82891c667a94f0f4+        .quad   0x7b1df4b73890f436++        // 2^96 * 7 * G++        .quad   0xe492f2e0b3b2a224+        .quad   0x7c6c9e062b551160+        .quad   0x15eb8fe20d7f7b0e+        .quad   0x61fcef2658fc5992+        .quad   0x5f2e221807f8f58c+        .quad   0xe3555c9fd49409d4+        .quad   0xb2aaa88d1fb6a630+        .quad   0x68698245d352e03d+        .quad   0xdbb15d852a18187a+        .quad   0xf3e4aad386ddacd7+        .quad   0x44bae2810ff6c482+        .quad   0x46cf4c473daf01cf++        // 2^96 * 8 * G++        .quad   0x426525ed9ec4e5f9+        .quad   0x0e5eda0116903303+        .quad   0x72b1a7f2cbe5cadc+        .quad   0x29387bcd14eb5f40+        .quad   0x213c6ea7f1498140+        .quad   0x7c1e7ef8392b4854+        .quad   0x2488c38c5629ceba+        .quad   0x1065aae50d8cc5bb+        .quad   0x1c2c4525df200d57+        .quad   0x5c3b2dd6bfca674a+        .quad   0x0a07e7b1e1834030+        .quad   0x69a198e64f1ce716++        // 2^100 * 1 * G++        .quad   0x7afcd613efa9d697+        .quad   0x0cc45aa41c067959+        .quad   0xa56fe104c1fada96+        .quad   0x3a73b70472e40365+        .quad   0x7b26e56b9e2d4734+        .quad   0xc4c7132b81c61675+        .quad   0xef5c9525ec9cde7f+        .quad   0x39c80b16e71743ad+        .quad   0x0f196e0d1b826c68+        .quad   0xf71ff0e24960e3db+        .quad   0x6113167023b7436c+        .quad   0x0cf0ea5877da7282++        // 2^100 * 2 * G++        .quad   0x196c80a4ddd4ccbd+        .quad   0x22e6f55d95f2dd9d+        .quad   0xc75e33c740d6c71b+        .quad   0x7bb51279cb3c042f+        .quad   0xe332ced43ba6945a+        .quad   0xde0b1361e881c05d+        .quad   0x1ad40f095e67ed3b+        .quad   0x5da8acdab8c63d5d+        .quad   0xc4b6664a3a70159f+        .quad   0x76194f0f0a904e14+        .quad   0xa5614c39a4096c13+        .quad   0x6cd0ff50979feced++        // 2^100 * 3 * G++        .quad   0xc0e067e78f4428ac+        .quad   0x14835ab0a61135e3+        .quad   0xf21d14f338062935+        .quad   0x6390a4c8df04849c+        .quad   0x7fecfabdb04ba18e+        .quad   0xd0fc7bfc3bddbcf7+        .quad   0xa41d486e057a131c+        .quad   0x641a4391f2223a61+        .quad   0xc5c6b95aa606a8db+        .quad   0x914b7f9eb06825f1+        .quad   0x2a731f6b44fc9eff+        .quad   0x30ddf38562705cfc++        // 2^100 * 4 * G++        .quad   0x4e3dcbdad1bff7f9+        .quad   0xc9118e8220645717+        .quad   0xbacccebc0f189d56+        .quad   0x1b4822e9d4467668+        .quad   0x33bef2bd68bcd52c+        .quad   0xc649dbb069482ef2+        .quad   0xb5b6ee0c41cb1aee+        .quad   0x5c294d270212a7e5+        .quad   0xab360a7f25563781+        .quad   0x2512228a480f7958+        .quad   0xc75d05276114b4e3+        .quad   0x222d9625d976fe2a++        // 2^100 * 5 * G++        .quad   0x1c717f85b372ace1+        .quad   0x81930e694638bf18+        .quad   0x239cad056bc08b58+        .quad   0x0b34271c87f8fff4+        .quad   0x0f94be7e0a344f85+        .quad   0xeb2faa8c87f22c38+        .quad   0x9ce1e75e4ee16f0f+        .quad   0x43e64e5418a08dea+        .quad   0x8155e2521a35ce63+        .quad   0xbe100d4df912028e+        .quad   0xbff80bf8a57ddcec+        .quad   0x57342dc96d6bc6e4++        // 2^100 * 6 * G++        .quad   0xefeef065c8ce5998+        .quad   0xbf029510b5cbeaa2+        .quad   0x8c64a10620b7c458+        .quad   0x35134fb231c24855+        .quad   0xf3c3bcb71e707bf6+        .quad   0x351d9b8c7291a762+        .quad   0x00502e6edad69a33+        .quad   0x522f521f1ec8807f+        .quad   0x272c1f46f9a3902b+        .quad   0xc91ba3b799657bcc+        .quad   0xae614b304f8a1c0e+        .quad   0x7afcaad70b99017b++        // 2^100 * 7 * G++        .quad   0xc25ded54a4b8be41+        .quad   0x902d13e11bb0e2dd+        .quad   0x41f43233cde82ab2+        .quad   0x1085faa5c3aae7cb+        .quad   0xa88141ecef842b6b+        .quad   0x55e7b14797abe6c5+        .quad   0x8c748f9703784ffe+        .quad   0x5b50a1f7afcd00b7+        .quad   0x9b840f66f1361315+        .quad   0x18462242701003e9+        .quad   0x65ed45fae4a25080+        .quad   0x0a2862393fda7320++        // 2^100 * 8 * G++        .quad   0x46ab13c8347cbc9d+        .quad   0x3849e8d499c12383+        .quad   0x4cea314087d64ac9+        .quad   0x1f354134b1a29ee7+        .quad   0x960e737b6ecb9d17+        .quad   0xfaf24948d67ceae1+        .quad   0x37e7a9b4d55e1b89+        .quad   0x5cb7173cb46c59eb+        .quad   0x4a89e68b82b7abf0+        .quad   0xf41cd9279ba6b7b9+        .quad   0x16e6c210e18d876f+        .quad   0x7cacdb0f7f1b09c6++        // 2^104 * 1 * G++        .quad   0x9062b2e0d91a78bc+        .quad   0x47c9889cc8509667+        .quad   0x9df54a66405070b8+        .quad   0x7369e6a92493a1bf+        .quad   0xe1014434dcc5caed+        .quad   0x47ed5d963c84fb33+        .quad   0x70019576ed86a0e7+        .quad   0x25b2697bd267f9e4+        .quad   0x9d673ffb13986864+        .quad   0x3ca5fbd9415dc7b8+        .quad   0xe04ecc3bdf273b5e+        .quad   0x1420683db54e4cd2++        // 2^104 * 2 * G++        .quad   0xb478bd1e249dd197+        .quad   0x620c35005e58c102+        .quad   0xfb02d32fccbaac5c+        .quad   0x60b63bebf508a72d+        .quad   0x34eebb6fc1cc5ad0+        .quad   0x6a1b0ce99646ac8b+        .quad   0xd3b0da49a66bde53+        .quad   0x31e83b4161d081c1+        .quad   0x97e8c7129e062b4f+        .quad   0x49e48f4f29320ad8+        .quad   0x5bece14b6f18683f+        .quad   0x55cf1eb62d550317++        // 2^104 * 3 * G++        .quad   0x5879101065c23d58+        .quad   0x8b9d086d5094819c+        .quad   0xe2402fa912c55fa7+        .quad   0x669a6564570891d4+        .quad   0x3076b5e37df58c52+        .quad   0xd73ab9dde799cc36+        .quad   0xbd831ce34913ee20+        .quad   0x1a56fbaa62ba0133+        .quad   0x943e6b505c9dc9ec+        .quad   0x302557bba77c371a+        .quad   0x9873ae5641347651+        .quad   0x13c4836799c58a5c++        // 2^104 * 4 * G++        .quad   0x423a5d465ab3e1b9+        .quad   0xfc13c187c7f13f61+        .quad   0x19f83664ecb5b9b6+        .quad   0x66f80c93a637b607+        .quad   0xc4dcfb6a5d8bd080+        .quad   0xdeebc4ec571a4842+        .quad   0xd4b2e883b8e55365+        .quad   0x50bdc87dc8e5b827+        .quad   0x606d37836edfe111+        .quad   0x32353e15f011abd9+        .quad   0x64b03ac325b73b96+        .quad   0x1dd56444725fd5ae++        // 2^104 * 5 * G++        .quad   0x8fa47ff83362127d+        .quad   0xbc9f6ac471cd7c15+        .quad   0x6e71454349220c8b+        .quad   0x0e645912219f732e+        .quad   0xc297e60008bac89a+        .quad   0x7d4cea11eae1c3e0+        .quad   0xf3e38be19fe7977c+        .quad   0x3a3a450f63a305cd+        .quad   0x078f2f31d8394627+        .quad   0x389d3183de94a510+        .quad   0xd1e36c6d17996f80+        .quad   0x318c8d9393a9a87b++        // 2^104 * 6 * G++        .quad   0xf2745d032afffe19+        .quad   0x0c9f3c497f24db66+        .quad   0xbc98d3e3ba8598ef+        .quad   0x224c7c679a1d5314+        .quad   0x5d669e29ab1dd398+        .quad   0xfc921658342d9e3b+        .quad   0x55851dfdf35973cd+        .quad   0x509a41c325950af6+        .quad   0xbdc06edca6f925e9+        .quad   0x793ef3f4641b1f33+        .quad   0x82ec12809d833e89+        .quad   0x05bff02328a11389++        // 2^104 * 7 * G++        .quad   0x3632137023cae00b+        .quad   0x544acf0ad1accf59+        .quad   0x96741049d21a1c88+        .quad   0x780b8cc3fa2a44a7+        .quad   0x6881a0dd0dc512e4+        .quad   0x4fe70dc844a5fafe+        .quad   0x1f748e6b8f4a5240+        .quad   0x576277cdee01a3ea+        .quad   0x1ef38abc234f305f+        .quad   0x9a577fbd1405de08+        .quad   0x5e82a51434e62a0d+        .quad   0x5ff418726271b7a1++        // 2^104 * 8 * G++        .quad   0x398e080c1789db9d+        .quad   0xa7602025f3e778f5+        .quad   0xfa98894c06bd035d+        .quad   0x106a03dc25a966be+        .quad   0xe5db47e813b69540+        .quad   0xf35d2a3b432610e1+        .quad   0xac1f26e938781276+        .quad   0x29d4db8ca0a0cb69+        .quad   0xd9ad0aaf333353d0+        .quad   0x38669da5acd309e5+        .quad   0x3c57658ac888f7f0+        .quad   0x4ab38a51052cbefa++        // 2^108 * 1 * G++        .quad   0xdfdacbee4324c0e9+        .quad   0x054442883f955bb7+        .quad   0xdef7aaa8ea31609f+        .quad   0x68aee70642287cff+        .quad   0xf68fe2e8809de054+        .quad   0xe3bc096a9c82bad1+        .quad   0x076353d40aadbf45+        .quad   0x7b9b1fb5dea1959e+        .quad   0xf01cc8f17471cc0c+        .quad   0x95242e37579082bb+        .quad   0x27776093d3e46b5f+        .quad   0x2d13d55a28bd85fb++        // 2^108 * 2 * G++        .quad   0xfac5d2065b35b8da+        .quad   0xa8da8a9a85624bb7+        .quad   0xccd2ca913d21cd0f+        .quad   0x6b8341ee8bf90d58+        .quad   0xbf019cce7aee7a52+        .quad   0xa8ded2b6e454ead3+        .quad   0x3c619f0b87a8bb19+        .quad   0x3619b5d7560916d8+        .quad   0x3579f26b0282c4b2+        .quad   0x64d592f24fafefae+        .quad   0xb7cded7b28c8c7c0+        .quad   0x6a927b6b7173a8d7++        // 2^108 * 3 * G++        .quad   0x1f6db24f986e4656+        .quad   0x1021c02ed1e9105b+        .quad   0xf8ff3fff2cc0a375+        .quad   0x1d2a6bf8c6c82592+        .quad   0x8d7040863ece88eb+        .quad   0xf0e307a980eec08c+        .quad   0xac2250610d788fda+        .quad   0x056d92a43a0d478d+        .quad   0x1b05a196fc3da5a1+        .quad   0x77d7a8c243b59ed0+        .quad   0x06da3d6297d17918+        .quad   0x66fbb494f12353f7++        // 2^108 * 4 * G++        .quad   0x751a50b9d85c0fb8+        .quad   0xd1afdc258bcf097b+        .quad   0x2f16a6a38309a969+        .quad   0x14ddff9ee5b00659+        .quad   0xd6d70996f12309d6+        .quad   0xdbfb2385e9c3d539+        .quad   0x46d602b0f7552411+        .quad   0x270a0b0557843e0c+        .quad   0x61ff0640a7862bcc+        .quad   0x81cac09a5f11abfe+        .quad   0x9047830455d12abb+        .quad   0x19a4bde1945ae873++        // 2^108 * 5 * G++        .quad   0x9b9f26f520a6200a+        .quad   0x64804443cf13eaf8+        .quad   0x8a63673f8631edd3+        .quad   0x72bbbce11ed39dc1+        .quad   0x40c709dec076c49f+        .quad   0x657bfaf27f3e53f6+        .quad   0x40662331eca042c4+        .quad   0x14b375487eb4df04+        .quad   0xae853c94ab66dc47+        .quad   0xeb62343edf762d6e+        .quad   0xf08e0e186fb2f7d1+        .quad   0x4f0b1c02700ab37a++        // 2^108 * 6 * G++        .quad   0xe1706787d81951fa+        .quad   0xa10a2c8eb290c77b+        .quad   0xe7382fa03ed66773+        .quad   0x0a4d84710bcc4b54+        .quad   0x79fd21ccc1b2e23f+        .quad   0x4ae7c281453df52a+        .quad   0xc8172ec9d151486b+        .quad   0x68abe9443e0a7534+        .quad   0xda12c6c407831dcb+        .quad   0x0da230d74d5c510d+        .quad   0x4ab1531e6bd404e1+        .quad   0x4106b166bcf440ef++        // 2^108 * 7 * G++        .quad   0x02e57a421cd23668+        .quad   0x4ad9fb5d0eaef6fd+        .quad   0x954e6727b1244480+        .quad   0x7f792f9d2699f331+        .quad   0xa485ccd539e4ecf2+        .quad   0x5aa3f3ad0555bab5+        .quad   0x145e3439937df82d+        .quad   0x1238b51e1214283f+        .quad   0x0b886b925fd4d924+        .quad   0x60906f7a3626a80d+        .quad   0xecd367b4b98abd12+        .quad   0x2876beb1def344cf++        // 2^108 * 8 * G++        .quad   0xdc84e93563144691+        .quad   0x632fe8a0d61f23f4+        .quad   0x4caa800612a9a8d5+        .quad   0x48f9dbfa0e9918d3+        .quad   0xd594b3333a8a85f8+        .quad   0x4ea37689e78d7d58+        .quad   0x73bf9f455e8e351f+        .quad   0x5507d7d2bc41ebb4+        .quad   0x1ceb2903299572fc+        .quad   0x7c8ccaa29502d0ee+        .quad   0x91bfa43411cce67b+        .quad   0x5784481964a831e7++        // 2^112 * 1 * G++        .quad   0xda7c2b256768d593+        .quad   0x98c1c0574422ca13+        .quad   0xf1a80bd5ca0ace1d+        .quad   0x29cdd1adc088a690+        .quad   0xd6cfd1ef5fddc09c+        .quad   0xe82b3efdf7575dce+        .quad   0x25d56b5d201634c2+        .quad   0x3041c6bb04ed2b9b+        .quad   0x0ff2f2f9d956e148+        .quad   0xade797759f356b2e+        .quad   0x1a4698bb5f6c025c+        .quad   0x104bbd6814049a7b++        // 2^112 * 2 * G++        .quad   0x51f0fd3168f1ed67+        .quad   0x2c811dcdd86f3bc2+        .quad   0x44dc5c4304d2f2de+        .quad   0x5be8cc57092a7149+        .quad   0xa95d9a5fd67ff163+        .quad   0xe92be69d4cc75681+        .quad   0xb7f8024cde20f257+        .quad   0x204f2a20fb072df5+        .quad   0xc8143b3d30ebb079+        .quad   0x7589155abd652e30+        .quad   0x653c3c318f6d5c31+        .quad   0x2570fb17c279161f++        // 2^112 * 3 * G++        .quad   0x3efa367f2cb61575+        .quad   0xf5f96f761cd6026c+        .quad   0xe8c7142a65b52562+        .quad   0x3dcb65ea53030acd+        .quad   0x192ea9550bb8245a+        .quad   0xc8e6fba88f9050d1+        .quad   0x7986ea2d88a4c935+        .quad   0x241c5f91de018668+        .quad   0x28d8172940de6caa+        .quad   0x8fbf2cf022d9733a+        .quad   0x16d7fcdd235b01d1+        .quad   0x08420edd5fcdf0e5++        // 2^112 * 4 * G++        .quad   0xcdff20ab8362fa4a+        .quad   0x57e118d4e21a3e6e+        .quad   0xe3179617fc39e62b+        .quad   0x0d9a53efbc1769fd+        .quad   0x0358c34e04f410ce+        .quad   0xb6135b5a276e0685+        .quad   0x5d9670c7ebb91521+        .quad   0x04d654f321db889c+        .quad   0x5e7dc116ddbdb5d5+        .quad   0x2954deb68da5dd2d+        .quad   0x1cb608173334a292+        .quad   0x4a7a4f2618991ad7++        // 2^112 * 5 * G++        .quad   0xf4a718025fb15f95+        .quad   0x3df65f346b5c1b8f+        .quad   0xcdfcf08500e01112+        .quad   0x11b50c4cddd31848+        .quad   0x24c3b291af372a4b+        .quad   0x93da8270718147f2+        .quad   0xdd84856486899ef2+        .quad   0x4a96314223e0ee33+        .quad   0xa6e8274408a4ffd6+        .quad   0x738e177e9c1576d9+        .quad   0x773348b63d02b3f2+        .quad   0x4f4bce4dce6bcc51++        // 2^112 * 6 * G++        .quad   0xa71fce5ae2242584+        .quad   0x26ea725692f58a9e+        .quad   0xd21a09d71cea3cf4+        .quad   0x73fcdd14b71c01e6+        .quad   0x30e2616ec49d0b6f+        .quad   0xe456718fcaec2317+        .quad   0x48eb409bf26b4fa6+        .quad   0x3042cee561595f37+        .quad   0x427e7079449bac41+        .quad   0x855ae36dbce2310a+        .quad   0x4cae76215f841a7c+        .quad   0x389e740c9a9ce1d6++        // 2^112 * 7 * G++        .quad   0x64fcb3ae34dcb9ce+        .quad   0x97500323e348d0ad+        .quad   0x45b3f07d62c6381b+        .quad   0x61545379465a6788+        .quad   0xc9bd78f6570eac28+        .quad   0xe55b0b3227919ce1+        .quad   0x65fc3eaba19b91ed+        .quad   0x25c425e5d6263690+        .quad   0x3f3e06a6f1d7de6e+        .quad   0x3ef976278e062308+        .quad   0x8c14f6264e8a6c77+        .quad   0x6539a08915484759++        // 2^112 * 8 * G++        .quad   0xe9d21f74c3d2f773+        .quad   0xc150544125c46845+        .quad   0x624e5ce8f9b99e33+        .quad   0x11c5e4aac5cd186c+        .quad   0xddc4dbd414bb4a19+        .quad   0x19b2bc3c98424f8e+        .quad   0x48a89fd736ca7169+        .quad   0x0f65320ef019bd90+        .quad   0xd486d1b1cafde0c6+        .quad   0x4f3fe6e3163b5181+        .quad   0x59a8af0dfaf2939a+        .quad   0x4cabc7bdec33072a++        // 2^116 * 1 * G++        .quad   0x16faa8fb532f7428+        .quad   0xdbd42ea046a4e272+        .quad   0x5337653b8b9ea480+        .quad   0x4065947223973f03+        .quad   0xf7c0a19c1a54a044+        .quad   0x4a1c5e2477bd9fbb+        .quad   0xa6e3ca115af22972+        .quad   0x1819bb953f2e9e0d+        .quad   0x498fbb795e042e84+        .quad   0x7d0dd89a7698b714+        .quad   0x8bfb0ba427fe6295+        .quad   0x36ba82e721200524++        // 2^116 * 2 * G++        .quad   0xd60ecbb74245ec41+        .quad   0xfd9be89e34348716+        .quad   0xc9240afee42284de+        .quad   0x4472f648d0531db4+        .quad   0xc8d69d0a57274ed5+        .quad   0x45ba803260804b17+        .quad   0xdf3cda102255dfac+        .quad   0x77d221232709b339+        .quad   0x498a6d7064ad94d8+        .quad   0xa5b5c8fd9af62263+        .quad   0x8ca8ed0545c141f4+        .quad   0x2c63bec3662d358c++        // 2^116 * 3 * G++        .quad   0x7fe60d8bea787955+        .quad   0xb9dc117eb5f401b7+        .quad   0x91c7c09a19355cce+        .quad   0x22692ef59442bedf+        .quad   0x9a518b3a8586f8bf+        .quad   0x9ee71af6cbb196f0+        .quad   0xaa0625e6a2385cf2+        .quad   0x1deb2176ddd7c8d1+        .quad   0x8563d19a2066cf6c+        .quad   0x401bfd8c4dcc7cd7+        .quad   0xd976a6becd0d8f62+        .quad   0x67cfd773a278b05e++        // 2^116 * 4 * G++        .quad   0x8dec31faef3ee475+        .quad   0x99dbff8a9e22fd92+        .quad   0x512d11594e26cab1+        .quad   0x0cde561eec4310b9+        .quad   0x2d5fa9855a4e586a+        .quad   0x65f8f7a449beab7e+        .quad   0xaa074dddf21d33d3+        .quad   0x185cba721bcb9dee+        .quad   0x93869da3f4e3cb41+        .quad   0xbf0392f540f7977e+        .quad   0x026204fcd0463b83+        .quad   0x3ec91a769eec6eed++        // 2^116 * 5 * G++        .quad   0x1e9df75bf78166ad+        .quad   0x4dfda838eb0cd7af+        .quad   0xba002ed8c1eaf988+        .quad   0x13fedb3e11f33cfc+        .quad   0x0fad2fb7b0a3402f+        .quad   0x46615ecbfb69f4a8+        .quad   0xf745bcc8c5f8eaa6+        .quad   0x7a5fa8794a94e896+        .quad   0x52958faa13cd67a1+        .quad   0x965ee0818bdbb517+        .quad   0x16e58daa2e8845b3+        .quad   0x357d397d5499da8f++        // 2^116 * 6 * G++        .quad   0x1ebfa05fb0bace6c+        .quad   0xc934620c1caf9a1e+        .quad   0xcc771cc41d82b61a+        .quad   0x2d94a16aa5f74fec+        .quad   0x481dacb4194bfbf8+        .quad   0x4d77e3f1bae58299+        .quad   0x1ef4612e7d1372a0+        .quad   0x3a8d867e70ff69e1+        .quad   0x6f58cd5d55aff958+        .quad   0xba3eaa5c75567721+        .quad   0x75c123999165227d+        .quad   0x69be1343c2f2b35e++        // 2^116 * 7 * G++        .quad   0x0e091d5ee197c92a+        .quad   0x4f51019f2945119f+        .quad   0x143679b9f034e99c+        .quad   0x7d88112e4d24c696+        .quad   0x82bbbdac684b8de3+        .quad   0xa2f4c7d03fca0718+        .quad   0x337f92fbe096aaa8+        .quad   0x200d4d8c63587376+        .quad   0x208aed4b4893b32b+        .quad   0x3efbf23ebe59b964+        .quad   0xd762deb0dba5e507+        .quad   0x69607bd681bd9d94++        // 2^116 * 8 * G++        .quad   0xf6be021068de1ce1+        .quad   0xe8d518e70edcbc1f+        .quad   0xe3effdd01b5505a5+        .quad   0x35f63353d3ec3fd0+        .quad   0x3b7f3bd49323a902+        .quad   0x7c21b5566b2c6e53+        .quad   0xe5ba8ff53a7852a7+        .quad   0x28bc77a5838ece00+        .quad   0x63ba78a8e25d8036+        .quad   0x63651e0094333490+        .quad   0x48d82f20288ce532+        .quad   0x3a31abfa36b57524++        // 2^120 * 1 * G++        .quad   0x239e9624089c0a2e+        .quad   0xc748c4c03afe4738+        .quad   0x17dbed2a764fa12a+        .quad   0x639b93f0321c8582+        .quad   0xc08f788f3f78d289+        .quad   0xfe30a72ca1404d9f+        .quad   0xf2778bfccf65cc9d+        .quad   0x7ee498165acb2021+        .quad   0x7bd508e39111a1c3+        .quad   0x2b2b90d480907489+        .quad   0xe7d2aec2ae72fd19+        .quad   0x0edf493c85b602a6++        // 2^120 * 2 * G++        .quad   0xaecc8158599b5a68+        .quad   0xea574f0febade20e+        .quad   0x4fe41d7422b67f07+        .quad   0x403b92e3019d4fb4+        .quad   0x6767c4d284764113+        .quad   0xa090403ff7f5f835+        .quad   0x1c8fcffacae6bede+        .quad   0x04c00c54d1dfa369+        .quad   0x4dc22f818b465cf8+        .quad   0x71a0f35a1480eff8+        .quad   0xaee8bfad04c7d657+        .quad   0x355bb12ab26176f4++        // 2^120 * 3 * G++        .quad   0xa71e64cc7493bbf4+        .quad   0xe5bd84d9eca3b0c3+        .quad   0x0a6bc50cfa05e785+        .quad   0x0f9b8132182ec312+        .quad   0xa301dac75a8c7318+        .quad   0xed90039db3ceaa11+        .quad   0x6f077cbf3bae3f2d+        .quad   0x7518eaf8e052ad8e+        .quad   0xa48859c41b7f6c32+        .quad   0x0f2d60bcf4383298+        .quad   0x1815a929c9b1d1d9+        .quad   0x47c3871bbb1755c4++        // 2^120 * 4 * G++        .quad   0x5144539771ec4f48+        .quad   0xf805b17dc98c5d6e+        .quad   0xf762c11a47c3c66b+        .quad   0x00b89b85764699dc+        .quad   0xfbe65d50c85066b0+        .quad   0x62ecc4b0b3a299b0+        .quad   0xe53754ea441ae8e0+        .quad   0x08fea02ce8d48d5f+        .quad   0x824ddd7668deead0+        .quad   0xc86445204b685d23+        .quad   0xb514cfcd5d89d665+        .quad   0x473829a74f75d537++        // 2^120 * 5 * G++        .quad   0x82d2da754679c418+        .quad   0xe63bd7d8b2618df0+        .quad   0x355eef24ac47eb0a+        .quad   0x2078684c4833c6b4+        .quad   0x23d9533aad3902c9+        .quad   0x64c2ddceef03588f+        .quad   0x15257390cfe12fb4+        .quad   0x6c668b4d44e4d390+        .quad   0x3b48cf217a78820c+        .quad   0xf76a0ab281273e97+        .quad   0xa96c65a78c8eed7b+        .quad   0x7411a6054f8a433f++        // 2^120 * 6 * G++        .quad   0x4d659d32b99dc86d+        .quad   0x044cdc75603af115+        .quad   0xb34c712cdcc2e488+        .quad   0x7c136574fb8134ff+        .quad   0x579ae53d18b175b4+        .quad   0x68713159f392a102+        .quad   0x8455ecba1eef35f5+        .quad   0x1ec9a872458c398f+        .quad   0xb8e6a4d400a2509b+        .quad   0x9b81d7020bc882b4+        .quad   0x57e7cc9bf1957561+        .quad   0x3add88a5c7cd6460++        // 2^120 * 7 * G++        .quad   0xab895770b635dcf2+        .quad   0x02dfef6cf66c1fbc+        .quad   0x85530268beb6d187+        .quad   0x249929fccc879e74+        .quad   0x85c298d459393046+        .quad   0x8f7e35985ff659ec+        .quad   0x1d2ca22af2f66e3a+        .quad   0x61ba1131a406a720+        .quad   0xa3d0a0f116959029+        .quad   0x023b6b6cba7ebd89+        .quad   0x7bf15a3e26783307+        .quad   0x5620310cbbd8ece7++        // 2^120 * 8 * G++        .quad   0x528993434934d643+        .quad   0xb9dbf806a51222f5+        .quad   0x8f6d878fc3f41c22+        .quad   0x37676a2a4d9d9730+        .quad   0x6646b5f477e285d6+        .quad   0x40e8ff676c8f6193+        .quad   0xa6ec7311abb594dd+        .quad   0x7ec846f3658cec4d+        .quad   0x9b5e8f3f1da22ec7+        .quad   0x130f1d776c01cd13+        .quad   0x214c8fcfa2989fb8+        .quad   0x6daaf723399b9dd5++        // 2^124 * 1 * G++        .quad   0x591e4a5610628564+        .quad   0x2a4bb87ca8b4df34+        .quad   0xde2a2572e7a38e43+        .quad   0x3cbdabd9fee5046e+        .quad   0x81aebbdd2cd13070+        .quad   0x962e4325f85a0e9e+        .quad   0xde9391aacadffecb+        .quad   0x53177fda52c230e6+        .quad   0xa7bc970650b9de79+        .quad   0x3d12a7fbc301b59b+        .quad   0x02652e68d36ae38c+        .quad   0x79d739835a6199dc++        // 2^124 * 2 * G++        .quad   0xd9354df64131c1bd+        .quad   0x758094a186ec5822+        .quad   0x4464ee12e459f3c2+        .quad   0x6c11fce4cb133282+        .quad   0x21c9d9920d591737+        .quad   0x9bea41d2e9b46cd6+        .quad   0xe20e84200d89bfca+        .quad   0x79d99f946eae5ff8+        .quad   0xf17b483568673205+        .quad   0x387deae83caad96c+        .quad   0x61b471fd56ffe386+        .quad   0x31741195b745a599++        // 2^124 * 3 * G++        .quad   0xe8d10190b77a360b+        .quad   0x99b983209995e702+        .quad   0xbd4fdff8fa0247aa+        .quad   0x2772e344e0d36a87+        .quad   0x17f8ba683b02a047+        .quad   0x50212096feefb6c8+        .quad   0x70139be21556cbe2+        .quad   0x203e44a11d98915b+        .quad   0xd6863eba37b9e39f+        .quad   0x105bc169723b5a23+        .quad   0x104f6459a65c0762+        .quad   0x567951295b4d38d4++        // 2^124 * 4 * G++        .quad   0x535fd60613037524+        .quad   0xe210adf6b0fbc26a+        .quad   0xac8d0a9b23e990ae+        .quad   0x47204d08d72fdbf9+        .quad   0x07242eb30d4b497f+        .quad   0x1ef96306b9bccc87+        .quad   0x37950934d8116f45+        .quad   0x05468d6201405b04+        .quad   0x00f565a9f93267de+        .quad   0xcecfd78dc0d58e8a+        .quad   0xa215e2dcf318e28e+        .quad   0x4599ee919b633352++        // 2^124 * 5 * G++        .quad   0xd3c220ca70e0e76b+        .quad   0xb12bea58ea9f3094+        .quad   0x294ddec8c3271282+        .quad   0x0c3539e1a1d1d028+        .quad   0xac746d6b861ae579+        .quad   0x31ab0650f6aea9dc+        .quad   0x241d661140256d4c+        .quad   0x2f485e853d21a5de+        .quad   0x329744839c0833f3+        .quad   0x6fe6257fd2abc484+        .quad   0x5327d1814b358817+        .quad   0x65712585893fe9bc++        // 2^124 * 6 * G++        .quad   0x9c102fb732a61161+        .quad   0xe48e10dd34d520a8+        .quad   0x365c63546f9a9176+        .quad   0x32f6fe4c046f6006+        .quad   0x81c29f1bd708ee3f+        .quad   0xddcb5a05ae6407d0+        .quad   0x97aec1d7d2a3eba7+        .quad   0x1590521a91d50831+        .quad   0x40a3a11ec7910acc+        .quad   0x9013dff8f16d27ae+        .quad   0x1a9720d8abb195d4+        .quad   0x1bb9fe452ea98463++        // 2^124 * 7 * G++        .quad   0xe9d1d950b3d54f9e+        .quad   0x2d5f9cbee00d33c1+        .quad   0x51c2c656a04fc6ac+        .quad   0x65c091ee3c1cbcc9+        .quad   0xcf5e6c95cc36747c+        .quad   0x294201536b0bc30d+        .quad   0x453ac67cee797af0+        .quad   0x5eae6ab32a8bb3c9+        .quad   0x7083661114f118ea+        .quad   0x2b37b87b94349cad+        .quad   0x7273f51cb4e99f40+        .quad   0x78a2a95823d75698++        // 2^124 * 8 * G++        .quad   0xa2b072e95c8c2ace+        .quad   0x69cffc96651e9c4b+        .quad   0x44328ef842e7b42b+        .quad   0x5dd996c122aadeb3+        .quad   0xb4f23c425ef83207+        .quad   0xabf894d3c9a934b5+        .quad   0xd0708c1339fd87f7+        .quad   0x1876789117166130+        .quad   0x925b5ef0670c507c+        .quad   0x819bc842b93c33bf+        .quad   0x10792e9a70dd003f+        .quad   0x59ad4b7a6e28dc74++        // 2^128 * 1 * G++        .quad   0x5f3a7562eb3dbe47+        .quad   0xf7ea38548ebda0b8+        .quad   0x00c3e53145747299+        .quad   0x1304e9e71627d551+        .quad   0x583b04bfacad8ea2+        .quad   0x29b743e8148be884+        .quad   0x2b1e583b0810c5db+        .quad   0x2b5449e58eb3bbaa+        .quad   0x789814d26adc9cfe+        .quad   0x3c1bab3f8b48dd0b+        .quad   0xda0fe1fff979c60a+        .quad   0x4468de2d7c2dd693++        // 2^128 * 2 * G++        .quad   0x51bb355e9419469e+        .quad   0x33e6dc4c23ddc754+        .quad   0x93a5b6d6447f9962+        .quad   0x6cce7c6ffb44bd63+        .quad   0x4b9ad8c6f86307ce+        .quad   0x21113531435d0c28+        .quad   0xd4a866c5657a772c+        .quad   0x5da6427e63247352+        .quad   0x1a94c688deac22ca+        .quad   0xb9066ef7bbae1ff8+        .quad   0x88ad8c388d59580f+        .quad   0x58f29abfe79f2ca8++        // 2^128 * 3 * G++        .quad   0xe90ecfab8de73e68+        .quad   0x54036f9f377e76a5+        .quad   0xf0495b0bbe015982+        .quad   0x577629c4a7f41e36+        .quad   0x4b5a64bf710ecdf6+        .quad   0xb14ce538462c293c+        .quad   0x3643d056d50b3ab9+        .quad   0x6af93724185b4870+        .quad   0x3220024509c6a888+        .quad   0xd2e036134b558973+        .quad   0x83e236233c33289f+        .quad   0x701f25bb0caec18f++        // 2^128 * 4 * G++        .quad   0xc3a8b0f8e4616ced+        .quad   0xf700660e9e25a87d+        .quad   0x61e3061ff4bca59c+        .quad   0x2e0c92bfbdc40be9+        .quad   0x9d18f6d97cbec113+        .quad   0x844a06e674bfdbe4+        .quad   0x20f5b522ac4e60d6+        .quad   0x720a5bc050955e51+        .quad   0x0c3f09439b805a35+        .quad   0xe84e8b376242abfc+        .quad   0x691417f35c229346+        .quad   0x0e9b9cbb144ef0ec++        // 2^128 * 5 * G++        .quad   0xfbbad48ffb5720ad+        .quad   0xee81916bdbf90d0e+        .quad   0xd4813152635543bf+        .quad   0x221104eb3f337bd8+        .quad   0x8dee9bd55db1beee+        .quad   0xc9c3ab370a723fb9+        .quad   0x44a8f1bf1c68d791+        .quad   0x366d44191cfd3cde+        .quad   0x9e3c1743f2bc8c14+        .quad   0x2eda26fcb5856c3b+        .quad   0xccb82f0e68a7fb97+        .quad   0x4167a4e6bc593244++        // 2^128 * 6 * G++        .quad   0x643b9d2876f62700+        .quad   0x5d1d9d400e7668eb+        .quad   0x1b4b430321fc0684+        .quad   0x7938bb7e2255246a+        .quad   0xc2be2665f8ce8fee+        .quad   0xe967ff14e880d62c+        .quad   0xf12e6e7e2f364eee+        .quad   0x34b33370cb7ed2f6+        .quad   0xcdc591ee8681d6cc+        .quad   0xce02109ced85a753+        .quad   0xed7485c158808883+        .quad   0x1176fc6e2dfe65e4++        // 2^128 * 7 * G++        .quad   0xb4af6cd05b9c619b+        .quad   0x2ddfc9f4b2a58480+        .quad   0x3d4fa502ebe94dc4+        .quad   0x08fc3a4c677d5f34+        .quad   0xdb90e28949770eb8+        .quad   0x98fbcc2aacf440a3+        .quad   0x21354ffeded7879b+        .quad   0x1f6a3e54f26906b6+        .quad   0x60a4c199d30734ea+        .quad   0x40c085b631165cd6+        .quad   0xe2333e23f7598295+        .quad   0x4f2fad0116b900d1++        // 2^128 * 8 * G++        .quad   0x44beb24194ae4e54+        .quad   0x5f541c511857ef6c+        .quad   0xa61e6b2d368d0498+        .quad   0x445484a4972ef7ab+        .quad   0x962cd91db73bb638+        .quad   0xe60577aafc129c08+        .quad   0x6f619b39f3b61689+        .quad   0x3451995f2944ee81+        .quad   0x9152fcd09fea7d7c+        .quad   0x4a816c94b0935cf6+        .quad   0x258e9aaa47285c40+        .quad   0x10b89ca6042893b7++        // 2^132 * 1 * G++        .quad   0x9b2a426e3b646025+        .quad   0x32127190385ce4cf+        .quad   0xa25cffc2dd6dea45+        .quad   0x06409010bea8de75+        .quad   0xd67cded679d34aa0+        .quad   0xcc0b9ec0cc4db39f+        .quad   0xa535a456e35d190f+        .quad   0x2e05d9eaf61f6fef+        .quad   0xc447901ad61beb59+        .quad   0x661f19bce5dc880a+        .quad   0x24685482b7ca6827+        .quad   0x293c778cefe07f26++        // 2^132 * 2 * G++        .quad   0x86809e7007069096+        .quad   0xaad75b15e4e50189+        .quad   0x07f35715a21a0147+        .quad   0x0487f3f112815d5e+        .quad   0x16c795d6a11ff200+        .quad   0xcb70d0e2b15815c9+        .quad   0x89f293209b5395b5+        .quad   0x50b8c2d031e47b4f+        .quad   0x48350c08068a4962+        .quad   0x6ffdd05351092c9a+        .quad   0x17af4f4aaf6fc8dd+        .quad   0x4b0553b53cdba58b++        // 2^132 * 3 * G++        .quad   0x9c65fcbe1b32ff79+        .quad   0xeb75ea9f03b50f9b+        .quad   0xfced2a6c6c07e606+        .quad   0x35106cd551717908+        .quad   0xbf05211b27c152d4+        .quad   0x5ec26849bd1af639+        .quad   0x5e0b2caa8e6fab98+        .quad   0x054c8bdd50bd0840+        .quad   0x38a0b12f1dcf073d+        .quad   0x4b60a8a3b7f6a276+        .quad   0xfed5ac25d3404f9a+        .quad   0x72e82d5e5505c229++        // 2^132 * 4 * G++        .quad   0x6b0b697ff0d844c8+        .quad   0xbb12f85cd979cb49+        .quad   0xd2a541c6c1da0f1f+        .quad   0x7b7c242958ce7211+        .quad   0x00d9cdfd69771d02+        .quad   0x410276cd6cfbf17e+        .quad   0x4c45306c1cb12ec7+        .quad   0x2857bf1627500861+        .quad   0x9f21903f0101689e+        .quad   0xd779dfd3bf861005+        .quad   0xa122ee5f3deb0f1b+        .quad   0x510df84b485a00d4++        // 2^132 * 5 * G++        .quad   0xa54133bb9277a1fa+        .quad   0x74ec3b6263991237+        .quad   0x1a3c54dc35d2f15a+        .quad   0x2d347144e482ba3a+        .quad   0x24b3c887c70ac15e+        .quad   0xb0f3a557fb81b732+        .quad   0x9b2cde2fe578cc1b+        .quad   0x4cf7ed0703b54f8e+        .quad   0x6bd47c6598fbee0f+        .quad   0x9e4733e2ab55be2d+        .quad   0x1093f624127610c5+        .quad   0x4e05e26ad0a1eaa4++        // 2^132 * 6 * G++        .quad   0xda9b6b624b531f20+        .quad   0x429a760e77509abb+        .quad   0xdbe9f522e823cb80+        .quad   0x618f1856880c8f82+        .quad   0x1833c773e18fe6c0+        .quad   0xe3c4711ad3c87265+        .quad   0x3bfd3c4f0116b283+        .quad   0x1955875eb4cd4db8+        .quad   0x6da6de8f0e399799+        .quad   0x7ad61aa440fda178+        .quad   0xb32cd8105e3563dd+        .quad   0x15f6beae2ae340ae++        // 2^132 * 7 * G++        .quad   0x862bcb0c31ec3a62+        .quad   0x810e2b451138f3c2+        .quad   0x788ec4b839dac2a4+        .quad   0x28f76867ae2a9281+        .quad   0xba9a0f7b9245e215+        .quad   0xf368612dd98c0dbb+        .quad   0x2e84e4cbf220b020+        .quad   0x6ba92fe962d90eda+        .quad   0x3e4df9655884e2aa+        .quad   0xbd62fbdbdbd465a5+        .quad   0xd7596caa0de9e524+        .quad   0x6e8042ccb2b1b3d7++        // 2^132 * 8 * G++        .quad   0xf10d3c29ce28ca6e+        .quad   0xbad34540fcb6093d+        .quad   0xe7426ed7a2ea2d3f+        .quad   0x08af9d4e4ff298b9+        .quad   0x1530653616521f7e+        .quad   0x660d06b896203dba+        .quad   0x2d3989bc545f0879+        .quad   0x4b5303af78ebd7b0+        .quad   0x72f8a6c3bebcbde8+        .quad   0x4f0fca4adc3a8e89+        .quad   0x6fa9d4e8c7bfdf7a+        .quad   0x0dcf2d679b624eb7++        // 2^136 * 1 * G++        .quad   0x3d5947499718289c+        .quad   0x12ebf8c524533f26+        .quad   0x0262bfcb14c3ef15+        .quad   0x20b878d577b7518e+        .quad   0x753941be5a45f06e+        .quad   0xd07caeed6d9c5f65+        .quad   0x11776b9c72ff51b6+        .quad   0x17d2d1d9ef0d4da9+        .quad   0x27f2af18073f3e6a+        .quad   0xfd3fe519d7521069+        .quad   0x22e3b72c3ca60022+        .quad   0x72214f63cc65c6a7++        // 2^136 * 2 * G++        .quad   0xb4e37f405307a693+        .quad   0xaba714d72f336795+        .quad   0xd6fbd0a773761099+        .quad   0x5fdf48c58171cbc9+        .quad   0x1d9db7b9f43b29c9+        .quad   0xd605824a4f518f75+        .quad   0xf2c072bd312f9dc4+        .quad   0x1f24ac855a1545b0+        .quad   0x24d608328e9505aa+        .quad   0x4748c1d10c1420ee+        .quad   0xc7ffe45c06fb25a2+        .quad   0x00ba739e2ae395e6++        // 2^136 * 3 * G++        .quad   0x592e98de5c8790d6+        .quad   0xe5bfb7d345c2a2df+        .quad   0x115a3b60f9b49922+        .quad   0x03283a3e67ad78f3+        .quad   0xae4426f5ea88bb26+        .quad   0x360679d984973bfb+        .quad   0x5c9f030c26694e50+        .quad   0x72297de7d518d226+        .quad   0x48241dc7be0cb939+        .quad   0x32f19b4d8b633080+        .quad   0xd3dfc90d02289308+        .quad   0x05e1296846271945++        // 2^136 * 4 * G++        .quad   0xba82eeb32d9c495a+        .quad   0xceefc8fcf12bb97c+        .quad   0xb02dabae93b5d1e0+        .quad   0x39c00c9c13698d9b+        .quad   0xadbfbbc8242c4550+        .quad   0xbcc80cecd03081d9+        .quad   0x843566a6f5c8df92+        .quad   0x78cf25d38258ce4c+        .quad   0x15ae6b8e31489d68+        .quad   0xaa851cab9c2bf087+        .quad   0xc9a75a97f04efa05+        .quad   0x006b52076b3ff832++        // 2^136 * 5 * G++        .quad   0x29e0cfe19d95781c+        .quad   0xb681df18966310e2+        .quad   0x57df39d370516b39+        .quad   0x4d57e3443bc76122+        .quad   0xf5cb7e16b9ce082d+        .quad   0x3407f14c417abc29+        .quad   0xd4b36bce2bf4a7ab+        .quad   0x7de2e9561a9f75ce+        .quad   0xde70d4f4b6a55ecb+        .quad   0x4801527f5d85db99+        .quad   0xdbc9c440d3ee9a81+        .quad   0x6b2a90af1a6029ed++        // 2^136 * 6 * G++        .quad   0x6923f4fc9ae61e97+        .quad   0x5735281de03f5fd1+        .quad   0xa764ae43e6edd12d+        .quad   0x5fd8f4e9d12d3e4a+        .quad   0x77ebf3245bb2d80a+        .quad   0xd8301b472fb9079b+        .quad   0xc647e6f24cee7333+        .quad   0x465812c8276c2109+        .quad   0x4d43beb22a1062d9+        .quad   0x7065fb753831dc16+        .quad   0x180d4a7bde2968d7+        .quad   0x05b32c2b1cb16790++        // 2^136 * 7 * G++        .quad   0xc8c05eccd24da8fd+        .quad   0xa1cf1aac05dfef83+        .quad   0xdbbeeff27df9cd61+        .quad   0x3b5556a37b471e99+        .quad   0xf7fca42c7ad58195+        .quad   0x3214286e4333f3cc+        .quad   0xb6c29d0d340b979d+        .quad   0x31771a48567307e1+        .quad   0x32b0c524e14dd482+        .quad   0xedb351541a2ba4b6+        .quad   0xa3d16048282b5af3+        .quad   0x4fc079d27a7336eb++        // 2^136 * 8 * G++        .quad   0x51c938b089bf2f7f+        .quad   0x2497bd6502dfe9a7+        .quad   0xffffc09c7880e453+        .quad   0x124567cecaf98e92+        .quad   0xdc348b440c86c50d+        .quad   0x1337cbc9cc94e651+        .quad   0x6422f74d643e3cb9+        .quad   0x241170c2bae3cd08+        .quad   0x3ff9ab860ac473b4+        .quad   0xf0911dee0113e435+        .quad   0x4ae75060ebc6c4af+        .quad   0x3f8612966c87000d++        // 2^140 * 1 * G++        .quad   0x0c9c5303f7957be4+        .quad   0xa3c31a20e085c145+        .quad   0xb0721d71d0850050+        .quad   0x0aba390eab0bf2da+        .quad   0x529fdffe638c7bf3+        .quad   0xdf2b9e60388b4995+        .quad   0xe027b34f1bad0249+        .quad   0x7bc92fc9b9fa74ed+        .quad   0x9f97ef2e801ad9f9+        .quad   0x83697d5479afda3a+        .quad   0xe906b3ffbd596b50+        .quad   0x02672b37dd3fb8e0++        // 2^140 * 2 * G++        .quad   0x48b2ca8b260885e4+        .quad   0xa4286bec82b34c1c+        .quad   0x937e1a2617f58f74+        .quad   0x741d1fcbab2ca2a5+        .quad   0xee9ba729398ca7f5+        .quad   0xeb9ca6257a4849db+        .quad   0x29eb29ce7ec544e1+        .quad   0x232ca21ef736e2c8+        .quad   0xbf61423d253fcb17+        .quad   0x08803ceafa39eb14+        .quad   0xf18602df9851c7af+        .quad   0x0400f3a049e3414b++        // 2^140 * 3 * G++        .quad   0xabce0476ba61c55b+        .quad   0x36a3d6d7c4d39716+        .quad   0x6eb259d5e8d82d09+        .quad   0x0c9176e984d756fb+        .quad   0x2efba412a06e7b06+        .quad   0x146785452c8d2560+        .quad   0xdf9713ebd67a91c7+        .quad   0x32830ac7157eadf3+        .quad   0x0e782a7ab73769e8+        .quad   0x04a05d7875b18e2c+        .quad   0x29525226ebcceae1+        .quad   0x0d794f8383eba820++        // 2^140 * 4 * G++        .quad   0xff35f5cb9e1516f4+        .quad   0xee805bcf648aae45+        .quad   0xf0d73c2bb93a9ef3+        .quad   0x097b0bf22092a6c2+        .quad   0x7be44ce7a7a2e1ac+        .quad   0x411fd93efad1b8b7+        .quad   0x1734a1d70d5f7c9b+        .quad   0x0d6592233127db16+        .quad   0xc48bab1521a9d733+        .quad   0xa6c2eaead61abb25+        .quad   0x625c6c1cc6cb4305+        .quad   0x7fc90fea93eb3a67++        // 2^140 * 5 * G++        .quad   0x0408f1fe1f5c5926+        .quad   0x1a8f2f5e3b258bf4+        .quad   0x40a951a2fdc71669+        .quad   0x6598ee93c98b577e+        .quad   0xc527deb59c7cb23d+        .quad   0x955391695328404e+        .quad   0xd64392817ccf2c7a+        .quad   0x6ce97dabf7d8fa11+        .quad   0x25b5a8e50ef7c48f+        .quad   0xeb6034116f2ce532+        .quad   0xc5e75173e53de537+        .quad   0x73119fa08c12bb03++        // 2^140 * 6 * G++        .quad   0xed30129453f1a4cb+        .quad   0xbce621c9c8f53787+        .quad   0xfacb2b1338bee7b9+        .quad   0x3025798a9ea8428c+        .quad   0x7845b94d21f4774d+        .quad   0xbf62f16c7897b727+        .quad   0x671857c03c56522b+        .quad   0x3cd6a85295621212+        .quad   0x3fecde923aeca999+        .quad   0xbdaa5b0062e8c12f+        .quad   0x67b99dfc96988ade+        .quad   0x3f52c02852661036++        // 2^140 * 7 * G++        .quad   0xffeaa48e2a1351c6+        .quad   0x28624754fa7f53d7+        .quad   0x0b5ba9e57582ddf1+        .quad   0x60c0104ba696ac59+        .quad   0x9258bf99eec416c6+        .quad   0xac8a5017a9d2f671+        .quad   0x629549ab16dea4ab+        .quad   0x05d0e85c99091569+        .quad   0x051de020de9cbe97+        .quad   0xfa07fc56b50bcf74+        .quad   0x378cec9f0f11df65+        .quad   0x36853c69ab96de4d++        // 2^140 * 8 * G++        .quad   0x36d9b8de78f39b2d+        .quad   0x7f42ed71a847b9ec+        .quad   0x241cd1d679bd3fde+        .quad   0x6a704fec92fbce6b+        .quad   0x4433c0b0fac5e7be+        .quad   0x724bae854c08dcbe+        .quad   0xf1f24cc446978f9b+        .quad   0x4a0aff6d62825fc8+        .quad   0xe917fb9e61095301+        .quad   0xc102df9402a092f8+        .quad   0xbf09e2f5fa66190b+        .quad   0x681109bee0dcfe37++        // 2^144 * 1 * G++        .quad   0x559a0cc9782a0dde+        .quad   0x551dcdb2ea718385+        .quad   0x7f62865b31ef238c+        .quad   0x504aa7767973613d+        .quad   0x9c18fcfa36048d13+        .quad   0x29159db373899ddd+        .quad   0xdc9f350b9f92d0aa+        .quad   0x26f57eee878a19d4+        .quad   0x0cab2cd55687efb1+        .quad   0x5180d162247af17b+        .quad   0x85c15a344f5a2467+        .quad   0x4041943d9dba3069++        // 2^144 * 2 * G++        .quad   0xc3c0eeba43ebcc96+        .quad   0x8d749c9c26ea9caf+        .quad   0xd9fa95ee1c77ccc6+        .quad   0x1420a1d97684340f+        .quad   0x4b217743a26caadd+        .quad   0x47a6b424648ab7ce+        .quad   0xcb1d4f7a03fbc9e3+        .quad   0x12d931429800d019+        .quad   0x00c67799d337594f+        .quad   0x5e3c5140b23aa47b+        .quad   0x44182854e35ff395+        .quad   0x1b4f92314359a012++        // 2^144 * 3 * G++        .quad   0x3e5c109d89150951+        .quad   0x39cefa912de9696a+        .quad   0x20eae43f975f3020+        .quad   0x239b572a7f132dae+        .quad   0x33cf3030a49866b1+        .quad   0x251f73d2215f4859+        .quad   0xab82aa4051def4f6+        .quad   0x5ff191d56f9a23f6+        .quad   0x819ed433ac2d9068+        .quad   0x2883ab795fc98523+        .quad   0xef4572805593eb3d+        .quad   0x020c526a758f36cb++        // 2^144 * 4 * G++        .quad   0x779834f89ed8dbbc+        .quad   0xc8f2aaf9dc7ca46c+        .quad   0xa9524cdca3e1b074+        .quad   0x02aacc4615313877+        .quad   0xe931ef59f042cc89+        .quad   0x2c589c9d8e124bb6+        .quad   0xadc8e18aaec75997+        .quad   0x452cfe0a5602c50c+        .quad   0x86a0f7a0647877df+        .quad   0xbbc464270e607c9f+        .quad   0xab17ea25f1fb11c9+        .quad   0x4cfb7d7b304b877b++        // 2^144 * 5 * G++        .quad   0x72b43d6cb89b75fe+        .quad   0x54c694d99c6adc80+        .quad   0xb8c3aa373ee34c9f+        .quad   0x14b4622b39075364+        .quad   0xe28699c29789ef12+        .quad   0x2b6ecd71df57190d+        .quad   0xc343c857ecc970d0+        .quad   0x5b1d4cbc434d3ac5+        .quad   0xb6fb2615cc0a9f26+        .quad   0x3a4f0e2bb88dcce5+        .quad   0x1301498b3369a705+        .quad   0x2f98f71258592dd1++        // 2^144 * 6 * G++        .quad   0x0c94a74cb50f9e56+        .quad   0x5b1ff4a98e8e1320+        .quad   0x9a2acc2182300f67+        .quad   0x3a6ae249d806aaf9+        .quad   0x2e12ae444f54a701+        .quad   0xfcfe3ef0a9cbd7de+        .quad   0xcebf890d75835de0+        .quad   0x1d8062e9e7614554+        .quad   0x657ada85a9907c5a+        .quad   0x1a0ea8b591b90f62+        .quad   0x8d0e1dfbdf34b4e9+        .quad   0x298b8ce8aef25ff3++        // 2^144 * 7 * G++        .quad   0x2a927953eff70cb2+        .quad   0x4b89c92a79157076+        .quad   0x9418457a30a7cf6a+        .quad   0x34b8a8404d5ce485+        .quad   0x837a72ea0a2165de+        .quad   0x3fab07b40bcf79f6+        .quad   0x521636c77738ae70+        .quad   0x6ba6271803a7d7dc+        .quad   0xc26eecb583693335+        .quad   0xd5a813df63b5fefd+        .quad   0xa293aa9aa4b22573+        .quad   0x71d62bdd465e1c6a++        // 2^144 * 8 * G++        .quad   0x6533cc28d378df80+        .quad   0xf6db43790a0fa4b4+        .quad   0xe3645ff9f701da5a+        .quad   0x74d5f317f3172ba4+        .quad   0xcd2db5dab1f75ef5+        .quad   0xd77f95cf16b065f5+        .quad   0x14571fea3f49f085+        .quad   0x1c333621262b2b3d+        .quad   0xa86fe55467d9ca81+        .quad   0x398b7c752b298c37+        .quad   0xda6d0892e3ac623b+        .quad   0x4aebcc4547e9d98c++        // 2^148 * 1 * G++        .quad   0x53175a7205d21a77+        .quad   0xb0c04422d3b934d4+        .quad   0xadd9f24bdd5deadc+        .quad   0x074f46e69f10ff8c+        .quad   0x0de9b204a059a445+        .quad   0xe15cb4aa4b17ad0f+        .quad   0xe1bbec521f79c557+        .quad   0x2633f1b9d071081b+        .quad   0xc1fb4177018b9910+        .quad   0xa6ea20dc6c0fe140+        .quad   0xd661f3e74354c6ff+        .quad   0x5ecb72e6f1a3407a++        // 2^148 * 2 * G++        .quad   0xa515a31b2259fb4e+        .quad   0x0960f3972bcac52f+        .quad   0xedb52fec8d3454cb+        .quad   0x382e2720c476c019+        .quad   0xfeeae106e8e86997+        .quad   0x9863337f98d09383+        .quad   0x9470480eaa06ebef+        .quad   0x038b6898d4c5c2d0+        .quad   0xf391c51d8ace50a6+        .quad   0x3142d0b9ae2d2948+        .quad   0xdb4d5a1a7f24ca80+        .quad   0x21aeba8b59250ea8++        // 2^148 * 3 * G++        .quad   0x24f13b34cf405530+        .quad   0x3c44ea4a43088af7+        .quad   0x5dd5c5170006a482+        .quad   0x118eb8f8890b086d+        .quad   0x53853600f0087f23+        .quad   0x4c461879da7d5784+        .quad   0x6af303deb41f6860+        .quad   0x0a3c16c5c27c18ed+        .quad   0x17e49c17cc947f3d+        .quad   0xccc6eda6aac1d27b+        .quad   0xdf6092ceb0f08e56+        .quad   0x4909b3e22c67c36b++        // 2^148 * 4 * G++        .quad   0x9c9c85ea63fe2e89+        .quad   0xbe1baf910e9412ec+        .quad   0x8f7baa8a86fbfe7b+        .quad   0x0fb17f9fef968b6c+        .quad   0x59a16676706ff64e+        .quad   0x10b953dd0d86a53d+        .quad   0x5848e1e6ce5c0b96+        .quad   0x2d8b78e712780c68+        .quad   0x79d5c62eafc3902b+        .quad   0x773a215289e80728+        .quad   0xc38ae640e10120b9+        .quad   0x09ae23717b2b1a6d++        // 2^148 * 5 * G++        .quad   0xbb6a192a4e4d083c+        .quad   0x34ace0630029e192+        .quad   0x98245a59aafabaeb+        .quad   0x6d9c8a9ada97faac+        .quad   0x10ab8fa1ad32b1d0+        .quad   0xe9aced1be2778b24+        .quad   0xa8856bc0373de90f+        .quad   0x66f35ddddda53996+        .quad   0xd27d9afb24997323+        .quad   0x1bb7e07ef6f01d2e+        .quad   0x2ba7472df52ecc7f+        .quad   0x03019b4f646f9dc8++        // 2^148 * 6 * G++        .quad   0x04a186b5565345cd+        .quad   0xeee76610bcc4116a+        .quad   0x689c73b478fb2a45+        .quad   0x387dcbff65697512+        .quad   0xaf09b214e6b3dc6b+        .quad   0x3f7573b5ad7d2f65+        .quad   0xd019d988100a23b0+        .quad   0x392b63a58b5c35f7+        .quad   0x4093addc9c07c205+        .quad   0xc565be15f532c37e+        .quad   0x63dbecfd1583402a+        .quad   0x61722b4aef2e032e++        // 2^148 * 7 * G++        .quad   0x0012aafeecbd47af+        .quad   0x55a266fb1cd46309+        .quad   0xf203eb680967c72c+        .quad   0x39633944ca3c1429+        .quad   0xd6b07a5581cb0e3c+        .quad   0x290ff006d9444969+        .quad   0x08680b6a16dcda1f+        .quad   0x5568d2b75a06de59+        .quad   0x8d0cb88c1b37cfe1+        .quad   0x05b6a5a3053818f3+        .quad   0xf2e9bc04b787d959+        .quad   0x6beba1249add7f64++        // 2^148 * 8 * G++        .quad   0x1d06005ca5b1b143+        .quad   0x6d4c6bb87fd1cda2+        .quad   0x6ef5967653fcffe7+        .quad   0x097c29e8c1ce1ea5+        .quad   0x5c3cecb943f5a53b+        .quad   0x9cc9a61d06c08df2+        .quad   0xcfba639a85895447+        .quad   0x5a845ae80df09fd5+        .quad   0x4ce97dbe5deb94ca+        .quad   0x38d0a4388c709c48+        .quad   0xc43eced4a169d097+        .quad   0x0a1249fff7e587c3++        // 2^152 * 1 * G++        .quad   0x12f0071b276d01c9+        .quad   0xe7b8bac586c48c70+        .quad   0x5308129b71d6fba9+        .quad   0x5d88fbf95a3db792+        .quad   0x0b408d9e7354b610+        .quad   0x806b32535ba85b6e+        .quad   0xdbe63a034a58a207+        .quad   0x173bd9ddc9a1df2c+        .quad   0x2b500f1efe5872df+        .quad   0x58d6582ed43918c1+        .quad   0xe6ed278ec9673ae0+        .quad   0x06e1cd13b19ea319++        // 2^152 * 2 * G++        .quad   0x40d0ad516f166f23+        .quad   0x118e32931fab6abe+        .quad   0x3fe35e14a04d088e+        .quad   0x3080603526e16266+        .quad   0x472baf629e5b0353+        .quad   0x3baa0b90278d0447+        .quad   0x0c785f469643bf27+        .quad   0x7f3a6a1a8d837b13+        .quad   0xf7e644395d3d800b+        .quad   0x95a8d555c901edf6+        .quad   0x68cd7830592c6339+        .quad   0x30d0fded2e51307e++        // 2^152 * 3 * G++        .quad   0xe0594d1af21233b3+        .quad   0x1bdbe78ef0cc4d9c+        .quad   0x6965187f8f499a77+        .quad   0x0a9214202c099868+        .quad   0x9cb4971e68b84750+        .quad   0xa09572296664bbcf+        .quad   0x5c8de72672fa412b+        .quad   0x4615084351c589d9+        .quad   0xbc9019c0aeb9a02e+        .quad   0x55c7110d16034cae+        .quad   0x0e6df501659932ec+        .quad   0x3bca0d2895ca5dfe++        // 2^152 * 4 * G++        .quad   0x40f031bc3c5d62a4+        .quad   0x19fc8b3ecff07a60+        .quad   0x98183da2130fb545+        .quad   0x5631deddae8f13cd+        .quad   0x9c688eb69ecc01bf+        .quad   0xf0bc83ada644896f+        .quad   0xca2d955f5f7a9fe2+        .quad   0x4ea8b4038df28241+        .quad   0x2aed460af1cad202+        .quad   0x46305305a48cee83+        .quad   0x9121774549f11a5f+        .quad   0x24ce0930542ca463++        // 2^152 * 5 * G++        .quad   0x1fe890f5fd06c106+        .quad   0xb5c468355d8810f2+        .quad   0x827808fe6e8caf3e+        .quad   0x41d4e3c28a06d74b+        .quad   0x3fcfa155fdf30b85+        .quad   0xd2f7168e36372ea4+        .quad   0xb2e064de6492f844+        .quad   0x549928a7324f4280+        .quad   0xf26e32a763ee1a2e+        .quad   0xae91e4b7d25ffdea+        .quad   0xbc3bd33bd17f4d69+        .quad   0x491b66dec0dcff6a++        // 2^152 * 6 * G++        .quad   0x98f5b13dc7ea32a7+        .quad   0xe3d5f8cc7e16db98+        .quad   0xac0abf52cbf8d947+        .quad   0x08f338d0c85ee4ac+        .quad   0x75f04a8ed0da64a1+        .quad   0xed222caf67e2284b+        .quad   0x8234a3791f7b7ba4+        .quad   0x4cf6b8b0b7018b67+        .quad   0xc383a821991a73bd+        .quad   0xab27bc01df320c7a+        .quad   0xc13d331b84777063+        .quad   0x530d4a82eb078a99++        // 2^152 * 7 * G++        .quad   0x004c3630e1f94825+        .quad   0x7e2d78268cab535a+        .quad   0xc7482323cc84ff8b+        .quad   0x65ea753f101770b9+        .quad   0x6d6973456c9abf9e+        .quad   0x257fb2fc4900a880+        .quad   0x2bacf412c8cfb850+        .quad   0x0db3e7e00cbfbd5b+        .quad   0x3d66fc3ee2096363+        .quad   0x81d62c7f61b5cb6b+        .quad   0x0fbe044213443b1a+        .quad   0x02a4ec1921e1a1db++        // 2^152 * 8 * G++        .quad   0x5ce6259a3b24b8a2+        .quad   0xb8577acc45afa0b8+        .quad   0xcccbe6e88ba07037+        .quad   0x3d143c51127809bf+        .quad   0xf5c86162f1cf795f+        .quad   0x118c861926ee57f2+        .quad   0x172124851c063578+        .quad   0x36d12b5dec067fcf+        .quad   0x126d279179154557+        .quad   0xd5e48f5cfc783a0a+        .quad   0x36bdb6e8df179bac+        .quad   0x2ef517885ba82859++        // 2^156 * 1 * G++        .quad   0x88bd438cd11e0d4a+        .quad   0x30cb610d43ccf308+        .quad   0xe09a0e3791937bcc+        .quad   0x4559135b25b1720c+        .quad   0x1ea436837c6da1e9+        .quad   0xf9c189af1fb9bdbe+        .quad   0x303001fcce5dd155+        .quad   0x28a7c99ebc57be52+        .quad   0xb8fd9399e8d19e9d+        .quad   0x908191cb962423ff+        .quad   0xb2b948d747c742a3+        .quad   0x37f33226d7fb44c4++        // 2^156 * 2 * G++        .quad   0x0dae8767b55f6e08+        .quad   0x4a43b3b35b203a02+        .quad   0xe3725a6e80af8c79+        .quad   0x0f7a7fd1705fa7a3+        .quad   0x33912553c821b11d+        .quad   0x66ed42c241e301df+        .quad   0x066fcc11104222fd+        .quad   0x307a3b41c192168f+        .quad   0x8eeb5d076eb55ce0+        .quad   0x2fc536bfaa0d925a+        .quad   0xbe81830fdcb6c6e8+        .quad   0x556c7045827baf52++        // 2^156 * 3 * G++        .quad   0x8e2b517302e9d8b7+        .quad   0xe3e52269248714e8+        .quad   0xbd4fbd774ca960b5+        .quad   0x6f4b4199c5ecada9+        .quad   0xb94b90022bf44406+        .quad   0xabd4237eff90b534+        .quad   0x7600a960faf86d3a+        .quad   0x2f45abdac2322ee3+        .quad   0x61af4912c8ef8a6a+        .quad   0xe58fa4fe43fb6e5e+        .quad   0xb5afcc5d6fd427cf+        .quad   0x6a5393281e1e11eb++        // 2^156 * 4 * G++        .quad   0xf3da5139a5d1ee89+        .quad   0x8145457cff936988+        .quad   0x3f622fed00e188c4+        .quad   0x0f513815db8b5a3d+        .quad   0x0fff04fe149443cf+        .quad   0x53cac6d9865cddd7+        .quad   0x31385b03531ed1b7+        .quad   0x5846a27cacd1039d+        .quad   0x4ff5cdac1eb08717+        .quad   0x67e8b29590f2e9bc+        .quad   0x44093b5e237afa99+        .quad   0x0d414bed8708b8b2++        // 2^156 * 5 * G++        .quad   0xcfb68265fd0e75f6+        .quad   0xe45b3e28bb90e707+        .quad   0x7242a8de9ff92c7a+        .quad   0x685b3201933202dd+        .quad   0x81886a92294ac9e8+        .quad   0x23162b45d55547be+        .quad   0x94cfbc4403715983+        .quad   0x50eb8fdb134bc401+        .quad   0xc0b73ec6d6b330cd+        .quad   0x84e44807132faff1+        .quad   0x732b7352c4a5dee1+        .quad   0x5d7c7cf1aa7cd2d2++        // 2^156 * 6 * G++        .quad   0xaf3b46bf7a4aafa2+        .quad   0xb78705ec4d40d411+        .quad   0x114f0c6aca7c15e3+        .quad   0x3f364faaa9489d4d+        .quad   0x33d1013e9b73a562+        .quad   0x925cef5748ec26e1+        .quad   0xa7fce614dd468058+        .quad   0x78b0fad41e9aa438+        .quad   0xbf56a431ed05b488+        .quad   0xa533e66c9c495c7e+        .quad   0xe8652baf87f3651a+        .quad   0x0241800059d66c33++        // 2^156 * 7 * G++        .quad   0xceb077fea37a5be4+        .quad   0xdb642f02e5a5eeb7+        .quad   0xc2e6d0c5471270b8+        .quad   0x4771b65538e4529c+        .quad   0x28350c7dcf38ea01+        .quad   0x7c6cdbc0b2917ab6+        .quad   0xace7cfbe857082f7+        .quad   0x4d2845aba2d9a1e0+        .quad   0xbb537fe0447070de+        .quad   0xcba744436dd557df+        .quad   0xd3b5a3473600dbcb+        .quad   0x4aeabbe6f9ffd7f8++        // 2^156 * 8 * G++        .quad   0x4630119e40d8f78c+        .quad   0xa01a9bc53c710e11+        .quad   0x486d2b258910dd79+        .quad   0x1e6c47b3db0324e5+        .quad   0x6a2134bcc4a9c8f2+        .quad   0xfbf8fd1c8ace2e37+        .quad   0x000ae3049911a0ba+        .quad   0x046e3a616bc89b9e+        .quad   0x14e65442f03906be+        .quad   0x4a019d54e362be2a+        .quad   0x68ccdfec8dc230c7+        .quad   0x7cfb7e3faf6b861c++        // 2^160 * 1 * G++        .quad   0x4637974e8c58aedc+        .quad   0xb9ef22fbabf041a4+        .quad   0xe185d956e980718a+        .quad   0x2f1b78fab143a8a6+        .quad   0x96eebffb305b2f51+        .quad   0xd3f938ad889596b8+        .quad   0xf0f52dc746d5dd25+        .quad   0x57968290bb3a0095+        .quad   0xf71ab8430a20e101+        .quad   0xf393658d24f0ec47+        .quad   0xcf7509a86ee2eed1+        .quad   0x7dc43e35dc2aa3e1++        // 2^160 * 2 * G++        .quad   0x85966665887dd9c3+        .quad   0xc90f9b314bb05355+        .quad   0xc6e08df8ef2079b1+        .quad   0x7ef72016758cc12f+        .quad   0x5a782a5c273e9718+        .quad   0x3576c6995e4efd94+        .quad   0x0f2ed8051f237d3e+        .quad   0x044fb81d82d50a99+        .quad   0xc1df18c5a907e3d9+        .quad   0x57b3371dce4c6359+        .quad   0xca704534b201bb49+        .quad   0x7f79823f9c30dd2e++        // 2^160 * 3 * G++        .quad   0x8334d239a3b513e8+        .quad   0xc13670d4b91fa8d8+        .quad   0x12b54136f590bd33+        .quad   0x0a4e0373d784d9b4+        .quad   0x6a9c1ff068f587ba+        .quad   0x0827894e0050c8de+        .quad   0x3cbf99557ded5be7+        .quad   0x64a9b0431c06d6f0+        .quad   0x2eb3d6a15b7d2919+        .quad   0xb0b4f6a0d53a8235+        .quad   0x7156ce4389a45d47+        .quad   0x071a7d0ace18346c++        // 2^160 * 4 * G++        .quad   0xd3072daac887ba0b+        .quad   0x01262905bfa562ee+        .quad   0xcf543002c0ef768b+        .quad   0x2c3bcc7146ea7e9c+        .quad   0xcc0c355220e14431+        .quad   0x0d65950709b15141+        .quad   0x9af5621b209d5f36+        .quad   0x7c69bcf7617755d3+        .quad   0x07f0d7eb04e8295f+        .quad   0x10db18252f50f37d+        .quad   0xe951a9a3171798d7+        .quad   0x6f5a9a7322aca51d++        // 2^160 * 5 * G++        .quad   0x8ba1000c2f41c6c5+        .quad   0xc49f79c10cfefb9b+        .quad   0x4efa47703cc51c9f+        .quad   0x494e21a2e147afca+        .quad   0xe729d4eba3d944be+        .quad   0x8d9e09408078af9e+        .quad   0x4525567a47869c03+        .quad   0x02ab9680ee8d3b24+        .quad   0xefa48a85dde50d9a+        .quad   0x219a224e0fb9a249+        .quad   0xfa091f1dd91ef6d9+        .quad   0x6b5d76cbea46bb34++        // 2^160 * 6 * G++        .quad   0x8857556cec0cd994+        .quad   0x6472dc6f5cd01dba+        .quad   0xaf0169148f42b477+        .quad   0x0ae333f685277354+        .quad   0xe0f941171e782522+        .quad   0xf1e6ae74036936d3+        .quad   0x408b3ea2d0fcc746+        .quad   0x16fb869c03dd313e+        .quad   0x288e199733b60962+        .quad   0x24fc72b4d8abe133+        .quad   0x4811f7ed0991d03e+        .quad   0x3f81e38b8f70d075++        // 2^160 * 7 * G++        .quad   0x7f910fcc7ed9affe+        .quad   0x545cb8a12465874b+        .quad   0xa8397ed24b0c4704+        .quad   0x50510fc104f50993+        .quad   0x0adb7f355f17c824+        .quad   0x74b923c3d74299a4+        .quad   0xd57c3e8bcbf8eaf7+        .quad   0x0ad3e2d34cdedc3d+        .quad   0x6f0c0fc5336e249d+        .quad   0x745ede19c331cfd9+        .quad   0xf2d6fd0009eefe1c+        .quad   0x127c158bf0fa1ebe++        // 2^160 * 8 * G++        .quad   0xf6197c422e9879a2+        .quad   0xa44addd452ca3647+        .quad   0x9b413fc14b4eaccb+        .quad   0x354ef87d07ef4f68+        .quad   0xdea28fc4ae51b974+        .quad   0x1d9973d3744dfe96+        .quad   0x6240680b873848a8+        .quad   0x4ed82479d167df95+        .quad   0xfee3b52260c5d975+        .quad   0x50352efceb41b0b8+        .quad   0x8808ac30a9f6653c+        .quad   0x302d92d20539236d++        // 2^164 * 1 * G++        .quad   0x4c59023fcb3efb7c+        .quad   0x6c2fcb99c63c2a94+        .quad   0xba4190e2c3c7e084+        .quad   0x0e545daea51874d9+        .quad   0x957b8b8b0df53c30+        .quad   0x2a1c770a8e60f098+        .quad   0xbbc7a670345796de+        .quad   0x22a48f9a90c99bc9+        .quad   0x6b7dc0dc8d3fac58+        .quad   0x5497cd6ce6e42bfd+        .quad   0x542f7d1bf400d305+        .quad   0x4159f47f048d9136++        // 2^164 * 2 * G++        .quad   0x20ad660839e31e32+        .quad   0xf81e1bd58405be50+        .quad   0xf8064056f4dabc69+        .quad   0x14d23dd4ce71b975+        .quad   0x748515a8bbd24839+        .quad   0x77128347afb02b55+        .quad   0x50ba2ac649a2a17f+        .quad   0x060525513ad730f1+        .quad   0xf2398e098aa27f82+        .quad   0x6d7982bb89a1b024+        .quad   0xfa694084214dd24c+        .quad   0x71ab966fa32301c3++        // 2^164 * 3 * G++        .quad   0x2dcbd8e34ded02fc+        .quad   0x1151f3ec596f22aa+        .quad   0xbca255434e0328da+        .quad   0x35768fbe92411b22+        .quad   0xb1088a0702809955+        .quad   0x43b273ea0b43c391+        .quad   0xca9b67aefe0686ed+        .quad   0x605eecbf8335f4ed+        .quad   0x83200a656c340431+        .quad   0x9fcd71678ee59c2f+        .quad   0x75d4613f71300f8a+        .quad   0x7a912faf60f542f9++        // 2^164 * 4 * G++        .quad   0xb204585e5edc1a43+        .quad   0x9f0e16ee5897c73c+        .quad   0x5b82c0ae4e70483c+        .quad   0x624a170e2bddf9be+        .quad   0x253f4f8dfa2d5597+        .quad   0x25e49c405477130c+        .quad   0x00c052e5996b1102+        .quad   0x33cb966e33bb6c4a+        .quad   0x597028047f116909+        .quad   0x828ac41c1e564467+        .quad   0x70417dbde6217387+        .quad   0x721627aefbac4384++        // 2^164 * 5 * G++        .quad   0x97d03bc38736add5+        .quad   0x2f1422afc532b130+        .quad   0x3aa68a057101bbc4+        .quad   0x4c946cf7e74f9fa7+        .quad   0xfd3097bc410b2f22+        .quad   0xf1a05da7b5cfa844+        .quad   0x61289a1def57ca74+        .quad   0x245ea199bb821902+        .quad   0xaedca66978d477f8+        .quad   0x1898ba3c29117fe1+        .quad   0xcf73f983720cbd58+        .quad   0x67da12e6b8b56351++        // 2^164 * 6 * G++        .quad   0x7067e187b4bd6e07+        .quad   0x6e8f0203c7d1fe74+        .quad   0x93c6aa2f38c85a30+        .quad   0x76297d1f3d75a78a+        .quad   0x2b7ef3d38ec8308c+        .quad   0x828fd7ec71eb94ab+        .quad   0x807c3b36c5062abd+        .quad   0x0cb64cb831a94141+        .quad   0x3030fc33534c6378+        .quad   0xb9635c5ce541e861+        .quad   0x15d9a9bed9b2c728+        .quad   0x49233ea3f3775dcb++        // 2^164 * 7 * G++        .quad   0x629398fa8dbffc3a+        .quad   0xe12fe52dd54db455+        .quad   0xf3be11dfdaf25295+        .quad   0x628b140dce5e7b51+        .quad   0x7b3985fe1c9f249b+        .quad   0x4fd6b2d5a1233293+        .quad   0xceb345941adf4d62+        .quad   0x6987ff6f542de50c+        .quad   0x47e241428f83753c+        .quad   0x6317bebc866af997+        .quad   0xdabb5b433d1a9829+        .quad   0x074d8d245287fb2d++        // 2^164 * 8 * G++        .quad   0x8337d9cd440bfc31+        .quad   0x729d2ca1af318fd7+        .quad   0xa040a4a4772c2070+        .quad   0x46002ef03a7349be+        .quad   0x481875c6c0e31488+        .quad   0x219429b2e22034b4+        .quad   0x7223c98a31283b65+        .quad   0x3420d60b342277f9+        .quad   0xfaa23adeaffe65f7+        .quad   0x78261ed45be0764c+        .quad   0x441c0a1e2f164403+        .quad   0x5aea8e567a87d395++        // 2^168 * 1 * G++        .quad   0x7813c1a2bca4283d+        .quad   0xed62f091a1863dd9+        .quad   0xaec7bcb8c268fa86+        .quad   0x10e5d3b76f1cae4c+        .quad   0x2dbc6fb6e4e0f177+        .quad   0x04e1bf29a4bd6a93+        .quad   0x5e1966d4787af6e8+        .quad   0x0edc5f5eb426d060+        .quad   0x5453bfd653da8e67+        .quad   0xe9dc1eec24a9f641+        .quad   0xbf87263b03578a23+        .quad   0x45b46c51361cba72++        // 2^168 * 2 * G++        .quad   0xa9402abf314f7fa1+        .quad   0xe257f1dc8e8cf450+        .quad   0x1dbbd54b23a8be84+        .quad   0x2177bfa36dcb713b+        .quad   0xce9d4ddd8a7fe3e4+        .quad   0xab13645676620e30+        .quad   0x4b594f7bb30e9958+        .quad   0x5c1c0aef321229df+        .quad   0x37081bbcfa79db8f+        .quad   0x6048811ec25f59b3+        .quad   0x087a76659c832487+        .quad   0x4ae619387d8ab5bb++        // 2^168 * 3 * G++        .quad   0x8ddbf6aa5344a32e+        .quad   0x7d88eab4b41b4078+        .quad   0x5eb0eb974a130d60+        .quad   0x1a00d91b17bf3e03+        .quad   0x61117e44985bfb83+        .quad   0xfce0462a71963136+        .quad   0x83ac3448d425904b+        .quad   0x75685abe5ba43d64+        .quad   0x6e960933eb61f2b2+        .quad   0x543d0fa8c9ff4952+        .quad   0xdf7275107af66569+        .quad   0x135529b623b0e6aa++        // 2^168 * 4 * G++        .quad   0x18f0dbd7add1d518+        .quad   0x979f7888cfc11f11+        .quad   0x8732e1f07114759b+        .quad   0x79b5b81a65ca3a01+        .quad   0xf5c716bce22e83fe+        .quad   0xb42beb19e80985c1+        .quad   0xec9da63714254aae+        .quad   0x5972ea051590a613+        .quad   0x0fd4ac20dc8f7811+        .quad   0x9a9ad294ac4d4fa8+        .quad   0xc01b2d64b3360434+        .quad   0x4f7e9c95905f3bdb++        // 2^168 * 5 * G++        .quad   0x62674bbc5781302e+        .quad   0xd8520f3989addc0f+        .quad   0x8c2999ae53fbd9c6+        .quad   0x31993ad92e638e4c+        .quad   0x71c8443d355299fe+        .quad   0x8bcd3b1cdbebead7+        .quad   0x8092499ef1a49466+        .quad   0x1942eec4a144adc8+        .quad   0x7dac5319ae234992+        .quad   0x2c1b3d910cea3e92+        .quad   0x553ce494253c1122+        .quad   0x2a0a65314ef9ca75++        // 2^168 * 6 * G++        .quad   0x2db7937ff7f927c2+        .quad   0xdb741f0617d0a635+        .quad   0x5982f3a21155af76+        .quad   0x4cf6e218647c2ded+        .quad   0xcf361acd3c1c793a+        .quad   0x2f9ebcac5a35bc3b+        .quad   0x60e860e9a8cda6ab+        .quad   0x055dc39b6dea1a13+        .quad   0xb119227cc28d5bb6+        .quad   0x07e24ebc774dffab+        .quad   0xa83c78cee4a32c89+        .quad   0x121a307710aa24b6++        // 2^168 * 7 * G++        .quad   0xe4db5d5e9f034a97+        .quad   0xe153fc093034bc2d+        .quad   0x460546919551d3b1+        .quad   0x333fc76c7a40e52d+        .quad   0xd659713ec77483c9+        .quad   0x88bfe077b82b96af+        .quad   0x289e28231097bcd3+        .quad   0x527bb94a6ced3a9b+        .quad   0x563d992a995b482e+        .quad   0x3405d07c6e383801+        .quad   0x485035de2f64d8e5+        .quad   0x6b89069b20a7a9f7++        // 2^168 * 8 * G++        .quad   0x812aa0416270220d+        .quad   0x995a89faf9245b4e+        .quad   0xffadc4ce5072ef05+        .quad   0x23bc2103aa73eb73+        .quad   0x4082fa8cb5c7db77+        .quad   0x068686f8c734c155+        .quad   0x29e6c8d9f6e7a57e+        .quad   0x0473d308a7639bcf+        .quad   0xcaee792603589e05+        .quad   0x2b4b421246dcc492+        .quad   0x02a1ef74e601a94f+        .quad   0x102f73bfde04341a++        // 2^172 * 1 * G++        .quad   0xb5a2d50c7ec20d3e+        .quad   0xc64bdd6ea0c97263+        .quad   0x56e89052c1ff734d+        .quad   0x4929c6f72b2ffaba+        .quad   0x358ecba293a36247+        .quad   0xaf8f9862b268fd65+        .quad   0x412f7e9968a01c89+        .quad   0x5786f312cd754524+        .quad   0x337788ffca14032c+        .quad   0xf3921028447f1ee3+        .quad   0x8b14071f231bccad+        .quad   0x4c817b4bf2344783++        // 2^172 * 2 * G++        .quad   0x0ff853852871b96e+        .quad   0xe13e9fab60c3f1bb+        .quad   0xeefd595325344402+        .quad   0x0a37c37075b7744b+        .quad   0x413ba057a40b4484+        .quad   0xba4c2e1a4f5f6a43+        .quad   0x614ba0a5aee1d61c+        .quad   0x78a1531a8b05dc53+        .quad   0x6cbdf1703ad0562b+        .quad   0x8ecf4830c92521a3+        .quad   0xdaebd303fd8424e7+        .quad   0x72ad82a42e5ec56f++        // 2^172 * 3 * G++        .quad   0x3f9e8e35bafb65f6+        .quad   0x39d69ec8f27293a1+        .quad   0x6cb8cd958cf6a3d0+        .quad   0x1734778173adae6d+        .quad   0xc368939167024bc3+        .quad   0x8e69d16d49502fda+        .quad   0xfcf2ec3ce45f4b29+        .quad   0x065f669ea3b4cbc4+        .quad   0x8a00aec75532db4d+        .quad   0xb869a4e443e31bb1+        .quad   0x4a0f8552d3a7f515+        .quad   0x19adeb7c303d7c08++        // 2^172 * 4 * G++        .quad   0xc720cb6153ead9a3+        .quad   0x55b2c97f512b636e+        .quad   0xb1e35b5fd40290b1+        .quad   0x2fd9ccf13b530ee2+        .quad   0x9d05ba7d43c31794+        .quad   0x2470c8ff93322526+        .quad   0x8323dec816197438+        .quad   0x2852709881569b53+        .quad   0x07bd475b47f796b8+        .quad   0xd2c7b013542c8f54+        .quad   0x2dbd23f43b24f87e+        .quad   0x6551afd77b0901d6++        // 2^172 * 5 * G++        .quad   0x4546baaf54aac27f+        .quad   0xf6f66fecb2a45a28+        .quad   0x582d1b5b562bcfe8+        .quad   0x44b123f3920f785f+        .quad   0x68a24ce3a1d5c9ac+        .quad   0xbb77a33d10ff6461+        .quad   0x0f86ce4425d3166e+        .quad   0x56507c0950b9623b+        .quad   0x1206f0b7d1713e63+        .quad   0x353fe3d915bafc74+        .quad   0x194ceb970ad9d94d+        .quad   0x62fadd7cf9d03ad3++        // 2^172 * 6 * G++        .quad   0xc6b5967b5598a074+        .quad   0x5efe91ce8e493e25+        .quad   0xd4b72c4549280888+        .quad   0x20ef1149a26740c2+        .quad   0x3cd7bc61e7ce4594+        .quad   0xcd6b35a9b7dd267e+        .quad   0xa080abc84366ef27+        .quad   0x6ec7c46f59c79711+        .quad   0x2f07ad636f09a8a2+        .quad   0x8697e6ce24205e7d+        .quad   0xc0aefc05ee35a139+        .quad   0x15e80958b5f9d897++        // 2^172 * 7 * G++        .quad   0x25a5ef7d0c3e235b+        .quad   0x6c39c17fbe134ee7+        .quad   0xc774e1342dc5c327+        .quad   0x021354b892021f39+        .quad   0x4dd1ed355bb061c4+        .quad   0x42dc0cef941c0700+        .quad   0x61305dc1fd86340e+        .quad   0x56b2cc930e55a443+        .quad   0x1df79da6a6bfc5a2+        .quad   0x02f3a2749fde4369+        .quad   0xb323d9f2cda390a7+        .quad   0x7be0847b8774d363++        // 2^172 * 8 * G++        .quad   0x8c99cc5a8b3f55c3+        .quad   0x0611d7253fded2a0+        .quad   0xed2995ff36b70a36+        .quad   0x1f699a54d78a2619+        .quad   0x1466f5af5307fa11+        .quad   0x817fcc7ded6c0af2+        .quad   0x0a6de44ec3a4a3fb+        .quad   0x74071475bc927d0b+        .quad   0xe77292f373e7ea8a+        .quad   0x296537d2cb045a31+        .quad   0x1bd0653ed3274fde+        .quad   0x2f9a2c4476bd2966++        // 2^176 * 1 * G++        .quad   0xeb18b9ab7f5745c6+        .quad   0x023a8aee5787c690+        .quad   0xb72712da2df7afa9+        .quad   0x36597d25ea5c013d+        .quad   0xa2b4dae0b5511c9a+        .quad   0x7ac860292bffff06+        .quad   0x981f375df5504234+        .quad   0x3f6bd725da4ea12d+        .quad   0x734d8d7b106058ac+        .quad   0xd940579e6fc6905f+        .quad   0x6466f8f99202932d+        .quad   0x7b7ecc19da60d6d0++        // 2^176 * 2 * G++        .quad   0x78c2373c695c690d+        .quad   0xdd252e660642906e+        .quad   0x951d44444ae12bd2+        .quad   0x4235ad7601743956+        .quad   0x6dae4a51a77cfa9b+        .quad   0x82263654e7a38650+        .quad   0x09bbffcd8f2d82db+        .quad   0x03bedc661bf5caba+        .quad   0x6258cb0d078975f5+        .quad   0x492942549189f298+        .quad   0xa0cab423e2e36ee4+        .quad   0x0e7ce2b0cdf066a1++        // 2^176 * 3 * G++        .quad   0xc494643ac48c85a3+        .quad   0xfd361df43c6139ad+        .quad   0x09db17dd3ae94d48+        .quad   0x666e0a5d8fb4674a+        .quad   0xfea6fedfd94b70f9+        .quad   0xf130c051c1fcba2d+        .quad   0x4882d47e7f2fab89+        .quad   0x615256138aeceeb5+        .quad   0x2abbf64e4870cb0d+        .quad   0xcd65bcf0aa458b6b+        .quad   0x9abe4eba75e8985d+        .quad   0x7f0bc810d514dee4++        // 2^176 * 4 * G++        .quad   0xb9006ba426f4136f+        .quad   0x8d67369e57e03035+        .quad   0xcbc8dfd94f463c28+        .quad   0x0d1f8dbcf8eedbf5+        .quad   0x83ac9dad737213a0+        .quad   0x9ff6f8ba2ef72e98+        .quad   0x311e2edd43ec6957+        .quad   0x1d3a907ddec5ab75+        .quad   0xba1693313ed081dc+        .quad   0x29329fad851b3480+        .quad   0x0128013c030321cb+        .quad   0x00011b44a31bfde3++        // 2^176 * 5 * G++        .quad   0x3fdfa06c3fc66c0c+        .quad   0x5d40e38e4dd60dd2+        .quad   0x7ae38b38268e4d71+        .quad   0x3ac48d916e8357e1+        .quad   0x16561f696a0aa75c+        .quad   0xc1bf725c5852bd6a+        .quad   0x11a8dd7f9a7966ad+        .quad   0x63d988a2d2851026+        .quad   0x00120753afbd232e+        .quad   0xe92bceb8fdd8f683+        .quad   0xf81669b384e72b91+        .quad   0x33fad52b2368a066++        // 2^176 * 6 * G++        .quad   0x540649c6c5e41e16+        .quad   0x0af86430333f7735+        .quad   0xb2acfcd2f305e746+        .quad   0x16c0f429a256dca7+        .quad   0x8d2cc8d0c422cfe8+        .quad   0x072b4f7b05a13acb+        .quad   0xa3feb6e6ecf6a56f+        .quad   0x3cc355ccb90a71e2+        .quad   0xe9b69443903e9131+        .quad   0xb8a494cb7a5637ce+        .quad   0xc87cd1a4baba9244+        .quad   0x631eaf426bae7568++        // 2^176 * 7 * G++        .quad   0xb3e90410da66fe9f+        .quad   0x85dd4b526c16e5a6+        .quad   0xbc3d97611ef9bf83+        .quad   0x5599648b1ea919b5+        .quad   0x47d975b9a3700de8+        .quad   0x7280c5fbe2f80552+        .quad   0x53658f2732e45de1+        .quad   0x431f2c7f665f80b5+        .quad   0xd6026344858f7b19+        .quad   0x14ab352fa1ea514a+        .quad   0x8900441a2090a9d7+        .quad   0x7b04715f91253b26++        // 2^176 * 8 * G++        .quad   0x83edbd28acf6ae43+        .quad   0x86357c8b7d5c7ab4+        .quad   0xc0404769b7eb2c44+        .quad   0x59b37bf5c2f6583f+        .quad   0xb376c280c4e6bac6+        .quad   0x970ed3dd6d1d9b0b+        .quad   0xb09a9558450bf944+        .quad   0x48d0acfa57cde223+        .quad   0xb60f26e47dabe671+        .quad   0xf1d1a197622f3a37+        .quad   0x4208ce7ee9960394+        .quad   0x16234191336d3bdb++        // 2^180 * 1 * G++        .quad   0xf19aeac733a63aef+        .quad   0x2c7fba5d4442454e+        .quad   0x5da87aa04795e441+        .quad   0x413051e1a4e0b0f5+        .quad   0x852dd1fd3d578bbe+        .quad   0x2b65ce72c3286108+        .quad   0x658c07f4eace2273+        .quad   0x0933f804ec38ab40+        .quad   0xa7ab69798d496476+        .quad   0x8121aadefcb5abc8+        .quad   0xa5dc12ef7b539472+        .quad   0x07fd47065e45351a++        // 2^180 * 2 * G++        .quad   0xc8583c3d258d2bcd+        .quad   0x17029a4daf60b73f+        .quad   0xfa0fc9d6416a3781+        .quad   0x1c1e5fba38b3fb23+        .quad   0x304211559ae8e7c3+        .quad   0xf281b229944882a5+        .quad   0x8a13ac2e378250e4+        .quad   0x014afa0954ba48f4+        .quad   0xcb3197001bb3666c+        .quad   0x330060524bffecb9+        .quad   0x293711991a88233c+        .quad   0x291884363d4ed364++        // 2^180 * 3 * G++        .quad   0x033c6805dc4babfa+        .quad   0x2c15bf5e5596ecc1+        .quad   0x1bc70624b59b1d3b+        .quad   0x3ede9850a19f0ec5+        .quad   0xfb9d37c3bc1ab6eb+        .quad   0x02be14534d57a240+        .quad   0xf4d73415f8a5e1f6+        .quad   0x5964f4300ccc8188+        .quad   0xe44a23152d096800+        .quad   0x5c08c55970866996+        .quad   0xdf2db60a46affb6e+        .quad   0x579155c1f856fd89++        // 2^180 * 4 * G++        .quad   0x96324edd12e0c9ef+        .quad   0x468b878df2420297+        .quad   0x199a3776a4f573be+        .quad   0x1e7fbcf18e91e92a+        .quad   0xb5f16b630817e7a6+        .quad   0x808c69233c351026+        .quad   0x324a983b54cef201+        .quad   0x53c092084a485345+        .quad   0xd2d41481f1cbafbf+        .quad   0x231d2db6716174e5+        .quad   0x0b7d7656e2a55c98+        .quad   0x3e955cd82aa495f6++        // 2^180 * 5 * G++        .quad   0xe48f535e3ed15433+        .quad   0xd075692a0d7270a3+        .quad   0x40fbd21daade6387+        .quad   0x14264887cf4495f5+        .quad   0xab39f3ef61bb3a3f+        .quad   0x8eb400652eb9193e+        .quad   0xb5de6ecc38c11f74+        .quad   0x654d7e9626f3c49f+        .quad   0xe564cfdd5c7d2ceb+        .quad   0x82eeafded737ccb9+        .quad   0x6107db62d1f9b0ab+        .quad   0x0b6baac3b4358dbb++        // 2^180 * 6 * G++        .quad   0x7ae62bcb8622fe98+        .quad   0x47762256ceb891af+        .quad   0x1a5a92bcf2e406b4+        .quad   0x7d29401784e41501+        .quad   0x204abad63700a93b+        .quad   0xbe0023d3da779373+        .quad   0xd85f0346633ab709+        .quad   0x00496dc490820412+        .quad   0x1c74b88dc27e6360+        .quad   0x074854268d14850c+        .quad   0xa145fb7b3e0dcb30+        .quad   0x10843f1b43803b23++        // 2^180 * 7 * G++        .quad   0xc5f90455376276dd+        .quad   0xce59158dd7645cd9+        .quad   0x92f65d511d366b39+        .quad   0x11574b6e526996c4+        .quad   0xd56f672de324689b+        .quad   0xd1da8aedb394a981+        .quad   0xdd7b58fe9168cfed+        .quad   0x7ce246cd4d56c1e8+        .quad   0xb8f4308e7f80be53+        .quad   0x5f3cb8cb34a9d397+        .quad   0x18a961bd33cc2b2c+        .quad   0x710045fb3a9af671++        // 2^180 * 8 * G++        .quad   0x73f93d36101b95eb+        .quad   0xfaef33794f6f4486+        .quad   0x5651735f8f15e562+        .quad   0x7fa3f19058b40da1+        .quad   0xa03fc862059d699e+        .quad   0x2370cfa19a619e69+        .quad   0xc4fe3b122f823deb+        .quad   0x1d1b056fa7f0844e+        .quad   0x1bc64631e56bf61f+        .quad   0xd379ab106e5382a3+        .quad   0x4d58c57e0540168d+        .quad   0x566256628442d8e4++        // 2^184 * 1 * G++        .quad   0xb9e499def6267ff6+        .quad   0x7772ca7b742c0843+        .quad   0x23a0153fe9a4f2b1+        .quad   0x2cdfdfecd5d05006+        .quad   0xdd499cd61ff38640+        .quad   0x29cd9bc3063625a0+        .quad   0x51e2d8023dd73dc3+        .quad   0x4a25707a203b9231+        .quad   0x2ab7668a53f6ed6a+        .quad   0x304242581dd170a1+        .quad   0x4000144c3ae20161+        .quad   0x5721896d248e49fc++        // 2^184 * 2 * G++        .quad   0x0b6e5517fd181bae+        .quad   0x9022629f2bb963b4+        .quad   0x5509bce932064625+        .quad   0x578edd74f63c13da+        .quad   0x285d5091a1d0da4e+        .quad   0x4baa6fa7b5fe3e08+        .quad   0x63e5177ce19393b3+        .quad   0x03c935afc4b030fd+        .quad   0x997276c6492b0c3d+        .quad   0x47ccc2c4dfe205fc+        .quad   0xdcd29b84dd623a3c+        .quad   0x3ec2ab590288c7a2++        // 2^184 * 3 * G++        .quad   0xa1a0d27be4d87bb9+        .quad   0xa98b4deb61391aed+        .quad   0x99a0ddd073cb9b83+        .quad   0x2dd5c25a200fcace+        .quad   0xa7213a09ae32d1cb+        .quad   0x0f2b87df40f5c2d5+        .quad   0x0baea4c6e81eab29+        .quad   0x0e1bf66c6adbac5e+        .quad   0xe2abd5e9792c887e+        .quad   0x1a020018cb926d5d+        .quad   0xbfba69cdbaae5f1e+        .quad   0x730548b35ae88f5f++        // 2^184 * 4 * G++        .quad   0xc43551a3cba8b8ee+        .quad   0x65a26f1db2115f16+        .quad   0x760f4f52ab8c3850+        .quad   0x3043443b411db8ca+        .quad   0x805b094ba1d6e334+        .quad   0xbf3ef17709353f19+        .quad   0x423f06cb0622702b+        .quad   0x585a2277d87845dd+        .quad   0xa18a5f8233d48962+        .quad   0x6698c4b5ec78257f+        .quad   0xa78e6fa5373e41ff+        .quad   0x7656278950ef981f++        // 2^184 * 5 * G++        .quad   0x38c3cf59d51fc8c0+        .quad   0x9bedd2fd0506b6f2+        .quad   0x26bf109fab570e8f+        .quad   0x3f4160a8c1b846a6+        .quad   0xe17073a3ea86cf9d+        .quad   0x3a8cfbb707155fdc+        .quad   0x4853e7fc31838a8e+        .quad   0x28bbf484b613f616+        .quad   0xf2612f5c6f136c7c+        .quad   0xafead107f6dd11be+        .quad   0x527e9ad213de6f33+        .quad   0x1e79cb358188f75d++        // 2^184 * 6 * G++        .quad   0x013436c3eef7e3f1+        .quad   0x828b6a7ffe9e10f8+        .quad   0x7ff908e5bcf9defc+        .quad   0x65d7951b3a3b3831+        .quad   0x77e953d8f5e08181+        .quad   0x84a50c44299dded9+        .quad   0xdc6c2d0c864525e5+        .quad   0x478ab52d39d1f2f4+        .quad   0x66a6a4d39252d159+        .quad   0xe5dde1bc871ac807+        .quad   0xb82c6b40a6c1c96f+        .quad   0x16d87a411a212214++        // 2^184 * 7 * G++        .quad   0xb3bd7e5a42066215+        .quad   0x879be3cd0c5a24c1+        .quad   0x57c05db1d6f994b7+        .quad   0x28f87c8165f38ca6+        .quad   0xfba4d5e2d54e0583+        .quad   0xe21fafd72ebd99fa+        .quad   0x497ac2736ee9778f+        .quad   0x1f990b577a5a6dde+        .quad   0xa3344ead1be8f7d6+        .quad   0x7d1e50ebacea798f+        .quad   0x77c6569e520de052+        .quad   0x45882fe1534d6d3e++        // 2^184 * 8 * G++        .quad   0x6669345d757983d6+        .quad   0x62b6ed1117aa11a6+        .quad   0x7ddd1857985e128f+        .quad   0x688fe5b8f626f6dd+        .quad   0xd8ac9929943c6fe4+        .quad   0xb5f9f161a38392a2+        .quad   0x2699db13bec89af3+        .quad   0x7dcf843ce405f074+        .quad   0x6c90d6484a4732c0+        .quad   0xd52143fdca563299+        .quad   0xb3be28c3915dc6e1+        .quad   0x6739687e7327191b++        // 2^188 * 1 * G++        .quad   0x9f65c5ea200814cf+        .quad   0x840536e169a31740+        .quad   0x8b0ed13925c8b4ad+        .quad   0x0080dbafe936361d+        .quad   0x8ce5aad0c9cb971f+        .quad   0x1156aaa99fd54a29+        .quad   0x41f7247015af9b78+        .quad   0x1fe8cca8420f49aa+        .quad   0x72a1848f3c0cc82a+        .quad   0x38c560c2877c9e54+        .quad   0x5004e228ce554140+        .quad   0x042418a103429d71++        // 2^188 * 2 * G++        .quad   0x899dea51abf3ff5f+        .quad   0x9b93a8672fc2d8ba+        .quad   0x2c38cb97be6ebd5c+        .quad   0x114d578497263b5d+        .quad   0x58e84c6f20816247+        .quad   0x8db2b2b6e36fd793+        .quad   0x977182561d484d85+        .quad   0x0822024f8632abd7+        .quad   0xb301bb7c6b1beca3+        .quad   0x55393f6dc6eb1375+        .quad   0x910d281097b6e4eb+        .quad   0x1ad4548d9d479ea3++        // 2^188 * 3 * G++        .quad   0xcd5a7da0389a48fd+        .quad   0xb38fa4aa9a78371e+        .quad   0xc6d9761b2cdb8e6c+        .quad   0x35cf51dbc97e1443+        .quad   0xa06fe66d0fe9fed3+        .quad   0xa8733a401c587909+        .quad   0x30d14d800df98953+        .quad   0x41ce5876c7b30258+        .quad   0x59ac3bc5d670c022+        .quad   0xeae67c109b119406+        .quad   0x9798bdf0b3782fda+        .quad   0x651e3201fd074092++        // 2^188 * 4 * G++        .quad   0xd63d8483ef30c5cf+        .quad   0x4cd4b4962361cc0c+        .quad   0xee90e500a48426ac+        .quad   0x0af51d7d18c14eeb+        .quad   0xa57ba4a01efcae9e+        .quad   0x769f4beedc308a94+        .quad   0xd1f10eeb3603cb2e+        .quad   0x4099ce5e7e441278+        .quad   0x1ac98e4f8a5121e9+        .quad   0x7dae9544dbfa2fe0+        .quad   0x8320aa0dd6430df9+        .quad   0x667282652c4a2fb5++        // 2^188 * 5 * G++        .quad   0x874621f4d86bc9ab+        .quad   0xb54c7bbe56fe6fea+        .quad   0x077a24257fadc22c+        .quad   0x1ab53be419b90d39+        .quad   0xada8b6e02946db23+        .quad   0x1c0ce51a7b253ab7+        .quad   0x8448c85a66dd485b+        .quad   0x7f1fc025d0675adf+        .quad   0xd8ee1b18319ea6aa+        .quad   0x004d88083a21f0da+        .quad   0x3bd6aa1d883a4f4b+        .quad   0x4db9a3a6dfd9fd14++        // 2^188 * 6 * G++        .quad   0x8ce7b23bb99c0755+        .quad   0x35c5d6edc4f50f7a+        .quad   0x7e1e2ed2ed9b50c3+        .quad   0x36305f16e8934da1+        .quad   0xd95b00bbcbb77c68+        .quad   0xddbc846a91f17849+        .quad   0x7cf700aebe28d9b3+        .quad   0x5ce1285c85d31f3e+        .quad   0x31b6972d98b0bde8+        .quad   0x7d920706aca6de5b+        .quad   0xe67310f8908a659f+        .quad   0x50fac2a6efdf0235++        // 2^188 * 7 * G++        .quad   0xf3d3a9f35b880f5a+        .quad   0xedec050cdb03e7c2+        .quad   0xa896981ff9f0b1a2+        .quad   0x49a4ae2bac5e34a4+        .quad   0x295b1c86f6f449bc+        .quad   0x51b2e84a1f0ab4dd+        .quad   0xc001cb30aa8e551d+        .quad   0x6a28d35944f43662+        .quad   0x28bb12ee04a740e0+        .quad   0x14313bbd9bce8174+        .quad   0x72f5b5e4e8c10c40+        .quad   0x7cbfb19936adcd5b++        // 2^188 * 8 * G++        .quad   0xa311ddc26b89792d+        .quad   0x1b30b4c6da512664+        .quad   0x0ca77b4ccf150859+        .quad   0x1de443df1b009408+        .quad   0x8e793a7acc36e6e0+        .quad   0xf9fab7a37d586eed+        .quad   0x3a4f9692bae1f4e4+        .quad   0x1c14b03eff5f447e+        .quad   0x19647bd114a85291+        .quad   0x57b76cb21034d3af+        .quad   0x6329db440f9d6dfa+        .quad   0x5ef43e586a571493++        // 2^192 * 1 * G++        .quad   0xef782014385675a6+        .quad   0xa2649f30aafda9e8+        .quad   0x4cd1eb505cdfa8cb+        .quad   0x46115aba1d4dc0b3+        .quad   0xa66dcc9dc80c1ac0+        .quad   0x97a05cf41b38a436+        .quad   0xa7ebf3be95dbd7c6+        .quad   0x7da0b8f68d7e7dab+        .quad   0xd40f1953c3b5da76+        .quad   0x1dac6f7321119e9b+        .quad   0x03cc6021feb25960+        .quad   0x5a5f887e83674b4b++        // 2^192 * 2 * G++        .quad   0x8f6301cf70a13d11+        .quad   0xcfceb815350dd0c4+        .quad   0xf70297d4a4bca47e+        .quad   0x3669b656e44d1434+        .quad   0x9e9628d3a0a643b9+        .quad   0xb5c3cb00e6c32064+        .quad   0x9b5302897c2dec32+        .quad   0x43e37ae2d5d1c70c+        .quad   0x387e3f06eda6e133+        .quad   0x67301d5199a13ac0+        .quad   0xbd5ad8f836263811+        .quad   0x6a21e6cd4fd5e9be++        // 2^192 * 3 * G++        .quad   0xf1c6170a3046e65f+        .quad   0x58712a2a00d23524+        .quad   0x69dbbd3c8c82b755+        .quad   0x586bf9f1a195ff57+        .quad   0xef4129126699b2e3+        .quad   0x71d30847708d1301+        .quad   0x325432d01182b0bd+        .quad   0x45371b07001e8b36+        .quad   0xa6db088d5ef8790b+        .quad   0x5278f0dc610937e5+        .quad   0xac0349d261a16eb8+        .quad   0x0eafb03790e52179++        // 2^192 * 4 * G++        .quad   0x960555c13748042f+        .quad   0x219a41e6820baa11+        .quad   0x1c81f73873486d0c+        .quad   0x309acc675a02c661+        .quad   0x5140805e0f75ae1d+        .quad   0xec02fbe32662cc30+        .quad   0x2cebdf1eea92396d+        .quad   0x44ae3344c5435bb3+        .quad   0x9cf289b9bba543ee+        .quad   0xf3760e9d5ac97142+        .quad   0x1d82e5c64f9360aa+        .quad   0x62d5221b7f94678f++        // 2^192 * 5 * G++        .quad   0x524c299c18d0936d+        .quad   0xc86bb56c8a0c1a0c+        .quad   0xa375052edb4a8631+        .quad   0x5c0efde4bc754562+        .quad   0x7585d4263af77a3c+        .quad   0xdfae7b11fee9144d+        .quad   0xa506708059f7193d+        .quad   0x14f29a5383922037+        .quad   0xdf717edc25b2d7f5+        .quad   0x21f970db99b53040+        .quad   0xda9234b7c3ed4c62+        .quad   0x5e72365c7bee093e++        // 2^192 * 6 * G++        .quad   0x575bfc074571217f+        .quad   0x3779675d0694d95b+        .quad   0x9a0a37bbf4191e33+        .quad   0x77f1104c47b4eabc+        .quad   0x7d9339062f08b33e+        .quad   0x5b9659e5df9f32be+        .quad   0xacff3dad1f9ebdfd+        .quad   0x70b20555cb7349b7+        .quad   0xbe5113c555112c4c+        .quad   0x6688423a9a881fcd+        .quad   0x446677855e503b47+        .quad   0x0e34398f4a06404a++        // 2^192 * 7 * G++        .quad   0xb67d22d93ecebde8+        .quad   0x09b3e84127822f07+        .quad   0x743fa61fb05b6d8d+        .quad   0x5e5405368a362372+        .quad   0x18930b093e4b1928+        .quad   0x7de3e10e73f3f640+        .quad   0xf43217da73395d6f+        .quad   0x6f8aded6ca379c3e+        .quad   0xe340123dfdb7b29a+        .quad   0x487b97e1a21ab291+        .quad   0xf9967d02fde6949e+        .quad   0x780de72ec8d3de97++        // 2^192 * 8 * G++        .quad   0x0ae28545089ae7bc+        .quad   0x388ddecf1c7f4d06+        .quad   0x38ac15510a4811b8+        .quad   0x0eb28bf671928ce4+        .quad   0x671feaf300f42772+        .quad   0x8f72eb2a2a8c41aa+        .quad   0x29a17fd797373292+        .quad   0x1defc6ad32b587a6+        .quad   0xaf5bbe1aef5195a7+        .quad   0x148c1277917b15ed+        .quad   0x2991f7fb7ae5da2e+        .quad   0x467d201bf8dd2867++        // 2^196 * 1 * G++        .quad   0x7906ee72f7bd2e6b+        .quad   0x05d270d6109abf4e+        .quad   0x8d5cfe45b941a8a4+        .quad   0x44c218671c974287+        .quad   0x745f9d56296bc318+        .quad   0x993580d4d8152e65+        .quad   0xb0e5b13f5839e9ce+        .quad   0x51fc2b28d43921c0+        .quad   0x1b8fd11795e2a98c+        .quad   0x1c4e5ee12b6b6291+        .quad   0x5b30e7107424b572+        .quad   0x6e6b9de84c4f4ac6++        // 2^196 * 2 * G++        .quad   0xdff25fce4b1de151+        .quad   0xd841c0c7e11c4025+        .quad   0x2554b3c854749c87+        .quad   0x2d292459908e0df9+        .quad   0x6b7c5f10f80cb088+        .quad   0x736b54dc56e42151+        .quad   0xc2b620a5c6ef99c4+        .quad   0x5f4c802cc3a06f42+        .quad   0x9b65c8f17d0752da+        .quad   0x881ce338c77ee800+        .quad   0xc3b514f05b62f9e3+        .quad   0x66ed5dd5bec10d48++        // 2^196 * 3 * G++        .quad   0x7d38a1c20bb2089d+        .quad   0x808334e196ccd412+        .quad   0xc4a70b8c6c97d313+        .quad   0x2eacf8bc03007f20+        .quad   0xf0adf3c9cbca047d+        .quad   0x81c3b2cbf4552f6b+        .quad   0xcfda112d44735f93+        .quad   0x1f23a0c77e20048c+        .quad   0xf235467be5bc1570+        .quad   0x03d2d9020dbab38c+        .quad   0x27529aa2fcf9e09e+        .quad   0x0840bef29d34bc50++        // 2^196 * 4 * G++        .quad   0x796dfb35dc10b287+        .quad   0x27176bcd5c7ff29d+        .quad   0x7f3d43e8c7b24905+        .quad   0x0304f5a191c54276+        .quad   0xcd54e06b7f37e4eb+        .quad   0x8cc15f87f5e96cca+        .quad   0xb8248bb0d3597dce+        .quad   0x246affa06074400c+        .quad   0x37d88e68fbe45321+        .quad   0x86097548c0d75032+        .quad   0x4e9b13ef894a0d35+        .quad   0x25a83cac5753d325++        // 2^196 * 5 * G++        .quad   0x10222f48eed8165e+        .quad   0x623fc1234b8bcf3a+        .quad   0x1e145c09c221e8f0+        .quad   0x7ccfa59fca782630+        .quad   0x9f0f66293952b6e2+        .quad   0x33db5e0e0934267b+        .quad   0xff45252bd609fedc+        .quad   0x06be10f5c506e0c9+        .quad   0x1a9615a9b62a345f+        .quad   0x22050c564a52fecc+        .quad   0xa7a2788528bc0dfe+        .quad   0x5e82770a1a1ee71d++        // 2^196 * 6 * G++        .quad   0x35425183ad896a5c+        .quad   0xe8673afbe78d52f6+        .quad   0x2c66f25f92a35f64+        .quad   0x09d04f3b3b86b102+        .quad   0xe802e80a42339c74+        .quad   0x34175166a7fffae5+        .quad   0x34865d1f1c408cae+        .quad   0x2cca982c605bc5ee+        .quad   0xfd2d5d35197dbe6e+        .quad   0x207c2eea8be4ffa3+        .quad   0x2613d8db325ae918+        .quad   0x7a325d1727741d3e++        // 2^196 * 7 * G++        .quad   0xd036b9bbd16dfde2+        .quad   0xa2055757c497a829+        .quad   0x8e6cc966a7f12667+        .quad   0x4d3b1a791239c180+        .quad   0xecd27d017e2a076a+        .quad   0xd788689f1636495e+        .quad   0x52a61af0919233e5+        .quad   0x2a479df17bb1ae64+        .quad   0x9e5eee8e33db2710+        .quad   0x189854ded6c43ca5+        .quad   0xa41c22c592718138+        .quad   0x27ad5538a43a5e9b++        // 2^196 * 8 * G++        .quad   0x2746dd4b15350d61+        .quad   0xd03fcbc8ee9521b7+        .quad   0xe86e365a138672ca+        .quad   0x510e987f7e7d89e2+        .quad   0xcb5a7d638e47077c+        .quad   0x8db7536120a1c059+        .quad   0x549e1e4d8bedfdcc+        .quad   0x080153b7503b179d+        .quad   0xdda69d930a3ed3e3+        .quad   0x3d386ef1cd60a722+        .quad   0xc817ad58bdaa4ee6+        .quad   0x23be8d554fe7372a++        // 2^200 * 1 * G++        .quad   0x95fe919a74ef4fad+        .quad   0x3a827becf6a308a2+        .quad   0x964e01d309a47b01+        .quad   0x71c43c4f5ba3c797+        .quad   0xbc1ef4bd567ae7a9+        .quad   0x3f624cb2d64498bd+        .quad   0xe41064d22c1f4ec8+        .quad   0x2ef9c5a5ba384001+        .quad   0xb6fd6df6fa9e74cd+        .quad   0xf18278bce4af267a+        .quad   0x8255b3d0f1ef990e+        .quad   0x5a758ca390c5f293++        // 2^200 * 2 * G++        .quad   0xa2b72710d9462495+        .quad   0x3aa8c6d2d57d5003+        .quad   0xe3d400bfa0b487ca+        .quad   0x2dbae244b3eb72ec+        .quad   0x8ce0918b1d61dc94+        .quad   0x8ded36469a813066+        .quad   0xd4e6a829afe8aad3+        .quad   0x0a738027f639d43f+        .quad   0x980f4a2f57ffe1cc+        .quad   0x00670d0de1839843+        .quad   0x105c3f4a49fb15fd+        .quad   0x2698ca635126a69c++        // 2^200 * 3 * G++        .quad   0xe765318832b0ba78+        .quad   0x381831f7925cff8b+        .quad   0x08a81b91a0291fcc+        .quad   0x1fb43dcc49caeb07+        .quad   0x2e3d702f5e3dd90e+        .quad   0x9e3f0918e4d25386+        .quad   0x5e773ef6024da96a+        .quad   0x3c004b0c4afa3332+        .quad   0x9aa946ac06f4b82b+        .quad   0x1ca284a5a806c4f3+        .quad   0x3ed3265fc6cd4787+        .quad   0x6b43fd01cd1fd217++        // 2^200 * 4 * G++        .quad   0xc7a75d4b4697c544+        .quad   0x15fdf848df0fffbf+        .quad   0x2868b9ebaa46785a+        .quad   0x5a68d7105b52f714+        .quad   0xb5c742583e760ef3+        .quad   0x75dc52b9ee0ab990+        .quad   0xbf1427c2072b923f+        .quad   0x73420b2d6ff0d9f0+        .quad   0xaf2cf6cb9e851e06+        .quad   0x8f593913c62238c4+        .quad   0xda8ab89699fbf373+        .quad   0x3db5632fea34bc9e++        // 2^200 * 5 * G++        .quad   0xf46eee2bf75dd9d8+        .quad   0x0d17b1f6396759a5+        .quad   0x1bf2d131499e7273+        .quad   0x04321adf49d75f13+        .quad   0x2e4990b1829825d5+        .quad   0xedeaeb873e9a8991+        .quad   0xeef03d394c704af8+        .quad   0x59197ea495df2b0e+        .quad   0x04e16019e4e55aae+        .quad   0xe77b437a7e2f92e9+        .quad   0xc7ce2dc16f159aa4+        .quad   0x45eafdc1f4d70cc0++        // 2^200 * 6 * G++        .quad   0x698401858045d72b+        .quad   0x4c22faa2cf2f0651+        .quad   0x941a36656b222dc6+        .quad   0x5a5eebc80362dade+        .quad   0xb60e4624cfccb1ed+        .quad   0x59dbc292bd5c0395+        .quad   0x31a09d1ddc0481c9+        .quad   0x3f73ceea5d56d940+        .quad   0xb7a7bfd10a4e8dc6+        .quad   0xbe57007e44c9b339+        .quad   0x60c1207f1557aefa+        .quad   0x26058891266218db++        // 2^200 * 7 * G++        .quad   0x59f704a68360ff04+        .quad   0xc3d93fde7661e6f4+        .quad   0x831b2a7312873551+        .quad   0x54ad0c2e4e615d57+        .quad   0x4c818e3cc676e542+        .quad   0x5e422c9303ceccad+        .quad   0xec07cccab4129f08+        .quad   0x0dedfa10b24443b8+        .quad   0xee3b67d5b82b522a+        .quad   0x36f163469fa5c1eb+        .quad   0xa5b4d2f26ec19fd3+        .quad   0x62ecb2baa77a9408++        // 2^200 * 8 * G++        .quad   0xe5ed795261152b3d+        .quad   0x4962357d0eddd7d1+        .quad   0x7482c8d0b96b4c71+        .quad   0x2e59f919a966d8be+        .quad   0x92072836afb62874+        .quad   0x5fcd5e8579e104a5+        .quad   0x5aad01adc630a14a+        .quad   0x61913d5075663f98+        .quad   0x0dc62d361a3231da+        .quad   0xfa47583294200270+        .quad   0x02d801513f9594ce+        .quad   0x3ddbc2a131c05d5c++        // 2^204 * 1 * G++        .quad   0x3f50a50a4ffb81ef+        .quad   0xb1e035093bf420bf+        .quad   0x9baa8e1cc6aa2cd0+        .quad   0x32239861fa237a40+        .quad   0xfb735ac2004a35d1+        .quad   0x31de0f433a6607c3+        .quad   0x7b8591bfc528d599+        .quad   0x55be9a25f5bb050c+        .quad   0x0d005acd33db3dbf+        .quad   0x0111b37c80ac35e2+        .quad   0x4892d66c6f88ebeb+        .quad   0x770eadb16508fbcd++        // 2^204 * 2 * G++        .quad   0x8451f9e05e4e89dd+        .quad   0xc06302ffbc793937+        .quad   0x5d22749556a6495c+        .quad   0x09a6755ca05603fb+        .quad   0xf1d3b681a05071b9+        .quad   0x2207659a3592ff3a+        .quad   0x5f0169297881e40e+        .quad   0x16bedd0e86ba374e+        .quad   0x5ecccc4f2c2737b5+        .quad   0x43b79e0c2dccb703+        .quad   0x33e008bc4ec43df3+        .quad   0x06c1b840f07566c0++        // 2^204 * 3 * G++        .quad   0x7688a5c6a388f877+        .quad   0x02a96c14deb2b6ac+        .quad   0x64c9f3431b8c2af8+        .quad   0x3628435554a1eed6+        .quad   0x69ee9e7f9b02805c+        .quad   0xcbff828a547d1640+        .quad   0x3d93a869b2430968+        .quad   0x46b7b8cd3fe26972+        .quad   0xe9812086fe7eebe0+        .quad   0x4cba6be72f515437+        .quad   0x1d04168b516efae9+        .quad   0x5ea1391043982cb9++        // 2^204 * 4 * G++        .quad   0x49125c9cf4702ee1+        .quad   0x4520b71f8b25b32d+        .quad   0x33193026501fef7e+        .quad   0x656d8997c8d2eb2b+        .quad   0x6f2b3be4d5d3b002+        .quad   0xafec33d96a09c880+        .quad   0x035f73a4a8bcc4cc+        .quad   0x22c5b9284662198b+        .quad   0xcb58c8fe433d8939+        .quad   0x89a0cb2e6a8d7e50+        .quad   0x79ca955309fbbe5a+        .quad   0x0c626616cd7fc106++        // 2^204 * 5 * G++        .quad   0x1ffeb80a4879b61f+        .quad   0x6396726e4ada21ed+        .quad   0x33c7b093368025ba+        .quad   0x471aa0c6f3c31788+        .quad   0x8fdfc379fbf454b1+        .quad   0x45a5a970f1a4b771+        .quad   0xac921ef7bad35915+        .quad   0x42d088dca81c2192+        .quad   0x8fda0f37a0165199+        .quad   0x0adadb77c8a0e343+        .quad   0x20fbfdfcc875e820+        .quad   0x1cf2bea80c2206e7++        // 2^204 * 6 * G++        .quad   0xc2ddf1deb36202ac+        .quad   0x92a5fe09d2e27aa5+        .quad   0x7d1648f6fc09f1d3+        .quad   0x74c2cc0513bc4959+        .quad   0x982d6e1a02c0412f+        .quad   0x90fa4c83db58e8fe+        .quad   0x01c2f5bcdcb18bc0+        .quad   0x686e0c90216abc66+        .quad   0x1fadbadba54395a7+        .quad   0xb41a02a0ae0da66a+        .quad   0xbf19f598bba37c07+        .quad   0x6a12b8acde48430d++        // 2^204 * 7 * G++        .quad   0xf8daea1f39d495d9+        .quad   0x592c190e525f1dfc+        .quad   0xdb8cbd04c9991d1b+        .quad   0x11f7fda3d88f0cb7+        .quad   0x793bdd801aaeeb5f+        .quad   0x00a2a0aac1518871+        .quad   0xe8a373a31f2136b4+        .quad   0x48aab888fc91ef19+        .quad   0x041f7e925830f40e+        .quad   0x002d6ca979661c06+        .quad   0x86dc9ff92b046a2e+        .quad   0x760360928b0493d1++        // 2^204 * 8 * G++        .quad   0x21bb41c6120cf9c6+        .quad   0xeab2aa12decda59b+        .quad   0xc1a72d020aa48b34+        .quad   0x215d4d27e87d3b68+        .quad   0xb43108e5695a0b05+        .quad   0x6cb00ee8ad37a38b+        .quad   0x5edad6eea3537381+        .quad   0x3f2602d4b6dc3224+        .quad   0xc8b247b65bcaf19c+        .quad   0x49779dc3b1b2c652+        .quad   0x89a180bbd5ece2e2+        .quad   0x13f098a3cec8e039++        // 2^208 * 1 * G++        .quad   0x9adc0ff9ce5ec54b+        .quad   0x039c2a6b8c2f130d+        .quad   0x028007c7f0f89515+        .quad   0x78968314ac04b36b+        .quad   0xf3aa57a22796bb14+        .quad   0x883abab79b07da21+        .quad   0xe54be21831a0391c+        .quad   0x5ee7fb38d83205f9+        .quad   0x538dfdcb41446a8e+        .quad   0xa5acfda9434937f9+        .quad   0x46af908d263c8c78+        .quad   0x61d0633c9bca0d09++        // 2^208 * 2 * G++        .quad   0x63744935ffdb2566+        .quad   0xc5bd6b89780b68bb+        .quad   0x6f1b3280553eec03+        .quad   0x6e965fd847aed7f5+        .quad   0xada328bcf8fc73df+        .quad   0xee84695da6f037fc+        .quad   0x637fb4db38c2a909+        .quad   0x5b23ac2df8067bdc+        .quad   0x9ad2b953ee80527b+        .quad   0xe88f19aafade6d8d+        .quad   0x0e711704150e82cf+        .quad   0x79b9bbb9dd95dedc++        // 2^208 * 3 * G++        .quad   0xebb355406a3126c2+        .quad   0xd26383a868c8c393+        .quad   0x6c0c6429e5b97a82+        .quad   0x5065f158c9fd2147+        .quad   0xd1997dae8e9f7374+        .quad   0xa032a2f8cfbb0816+        .quad   0xcd6cba126d445f0a+        .quad   0x1ba811460accb834+        .quad   0x708169fb0c429954+        .quad   0xe14600acd76ecf67+        .quad   0x2eaab98a70e645ba+        .quad   0x3981f39e58a4faf2++        // 2^208 * 4 * G++        .quad   0x18fb8a7559230a93+        .quad   0x1d168f6960e6f45d+        .quad   0x3a85a94514a93cb5+        .quad   0x38dc083705acd0fd+        .quad   0xc845dfa56de66fde+        .quad   0xe152a5002c40483a+        .quad   0xe9d2e163c7b4f632+        .quad   0x30f4452edcbc1b65+        .quad   0x856d2782c5759740+        .quad   0xfa134569f99cbecc+        .quad   0x8844fc73c0ea4e71+        .quad   0x632d9a1a593f2469++        // 2^208 * 5 * G++        .quad   0xf6bb6b15b807cba6+        .quad   0x1823c7dfbc54f0d7+        .quad   0xbb1d97036e29670b+        .quad   0x0b24f48847ed4a57+        .quad   0xbf09fd11ed0c84a7+        .quad   0x63f071810d9f693a+        .quad   0x21908c2d57cf8779+        .quad   0x3a5a7df28af64ba2+        .quad   0xdcdad4be511beac7+        .quad   0xa4538075ed26ccf2+        .quad   0xe19cff9f005f9a65+        .quad   0x34fcf74475481f63++        // 2^208 * 6 * G++        .quad   0xc197e04c789767ca+        .quad   0xb8714dcb38d9467d+        .quad   0x55de888283f95fa8+        .quad   0x3d3bdc164dfa63f7+        .quad   0xa5bb1dab78cfaa98+        .quad   0x5ceda267190b72f2+        .quad   0x9309c9110a92608e+        .quad   0x0119a3042fb374b0+        .quad   0x67a2d89ce8c2177d+        .quad   0x669da5f66895d0c1+        .quad   0xf56598e5b282a2b0+        .quad   0x56c088f1ede20a73++        // 2^208 * 7 * G++        .quad   0x336d3d1110a86e17+        .quad   0xd7f388320b75b2fa+        .quad   0xf915337625072988+        .quad   0x09674c6b99108b87+        .quad   0x581b5fac24f38f02+        .quad   0xa90be9febae30cbd+        .quad   0x9a2169028acf92f0+        .quad   0x038b7ea48359038f+        .quad   0x9f4ef82199316ff8+        .quad   0x2f49d282eaa78d4f+        .quad   0x0971a5ab5aef3174+        .quad   0x6e5e31025969eb65++        // 2^208 * 8 * G++        .quad   0xb16c62f587e593fb+        .quad   0x4999eddeca5d3e71+        .quad   0xb491c1e014cc3e6d+        .quad   0x08f5114789a8dba8+        .quad   0x3304fb0e63066222+        .quad   0xfb35068987acba3f+        .quad   0xbd1924778c1061a3+        .quad   0x3058ad43d1838620+        .quad   0x323c0ffde57663d0+        .quad   0x05c3df38a22ea610+        .quad   0xbdc78abdac994f9a+        .quad   0x26549fa4efe3dc99++        // 2^212 * 1 * G++        .quad   0x738b38d787ce8f89+        .quad   0xb62658e24179a88d+        .quad   0x30738c9cf151316d+        .quad   0x49128c7f727275c9+        .quad   0x04dbbc17f75396b9+        .quad   0x69e6a2d7d2f86746+        .quad   0xc6409d99f53eabc6+        .quad   0x606175f6332e25d2+        .quad   0x4021370ef540e7dd+        .quad   0x0910d6f5a1f1d0a5+        .quad   0x4634aacd5b06b807+        .quad   0x6a39e6356944f235++        // 2^212 * 2 * G++        .quad   0x96cd5640df90f3e7+        .quad   0x6c3a760edbfa25ea+        .quad   0x24f3ef0959e33cc4+        .quad   0x42889e7e530d2e58+        .quad   0x1da1965774049e9d+        .quad   0xfbcd6ea198fe352b+        .quad   0xb1cbcd50cc5236a6+        .quad   0x1f5ec83d3f9846e2+        .quad   0x8efb23c3328ccb75+        .quad   0xaf42a207dd876ee9+        .quad   0x20fbdadc5dfae796+        .quad   0x241e246b06bf9f51++        // 2^212 * 3 * G++        .quad   0x29e68e57ad6e98f6+        .quad   0x4c9260c80b462065+        .quad   0x3f00862ea51ebb4b+        .quad   0x5bc2c77fb38d9097+        .quad   0x7eaafc9a6280bbb8+        .quad   0x22a70f12f403d809+        .quad   0x31ce40bb1bfc8d20+        .quad   0x2bc65635e8bd53ee+        .quad   0xe8d5dc9fa96bad93+        .quad   0xe58fb17dde1947dc+        .quad   0x681532ea65185fa3+        .quad   0x1fdd6c3b034a7830++        // 2^212 * 4 * G++        .quad   0x0a64e28c55dc18fe+        .quad   0xe3df9e993399ebdd+        .quad   0x79ac432370e2e652+        .quad   0x35ff7fc33ae4cc0e+        .quad   0x9c13a6a52dd8f7a9+        .quad   0x2dbb1f8c3efdcabf+        .quad   0x961e32405e08f7b5+        .quad   0x48c8a121bbe6c9e5+        .quad   0xfc415a7c59646445+        .quad   0xd224b2d7c128b615+        .quad   0x6035c9c905fbb912+        .quad   0x42d7a91274429fab++        // 2^212 * 5 * G++        .quad   0x4e6213e3eaf72ed3+        .quad   0x6794981a43acd4e7+        .quad   0xff547cde6eb508cb+        .quad   0x6fed19dd10fcb532+        .quad   0xa9a48947933da5bc+        .quad   0x4a58920ec2e979ec+        .quad   0x96d8800013e5ac4c+        .quad   0x453692d74b48b147+        .quad   0xdd775d99a8559c6f+        .quad   0xf42a2140df003e24+        .quad   0x5223e229da928a66+        .quad   0x063f46ba6d38f22c++        // 2^212 * 6 * G++        .quad   0xd2d242895f536694+        .quad   0xca33a2c542939b2c+        .quad   0x986fada6c7ddb95c+        .quad   0x5a152c042f712d5d+        .quad   0x39843cb737346921+        .quad   0xa747fb0738c89447+        .quad   0xcb8d8031a245307e+        .quad   0x67810f8e6d82f068+        .quad   0x3eeb8fbcd2287db4+        .quad   0x72c7d3a301a03e93+        .quad   0x5473e88cbd98265a+        .quad   0x7324aa515921b403++        // 2^212 * 7 * G++        .quad   0x857942f46c3cbe8e+        .quad   0xa1d364b14730c046+        .quad   0x1c8ed914d23c41bf+        .quad   0x0838e161eef6d5d2+        .quad   0xad23f6dae82354cb+        .quad   0x6962502ab6571a6d+        .quad   0x9b651636e38e37d1+        .quad   0x5cac5005d1a3312f+        .quad   0x8cc154cce9e39904+        .quad   0x5b3a040b84de6846+        .quad   0xc4d8a61cb1be5d6e+        .quad   0x40fb897bd8861f02++        // 2^212 * 8 * G++        .quad   0x84c5aa9062de37a1+        .quad   0x421da5000d1d96e1+        .quad   0x788286306a9242d9+        .quad   0x3c5e464a690d10da+        .quad   0xe57ed8475ab10761+        .quad   0x71435e206fd13746+        .quad   0x342f824ecd025632+        .quad   0x4b16281ea8791e7b+        .quad   0xd1c101d50b813381+        .quad   0xdee60f1176ee6828+        .quad   0x0cb68893383f6409+        .quad   0x6183c565f6ff484a++        // 2^216 * 1 * G++        .quad   0x741d5a461e6bf9d6+        .quad   0x2305b3fc7777a581+        .quad   0xd45574a26474d3d9+        .quad   0x1926e1dc6401e0ff+        .quad   0xdb468549af3f666e+        .quad   0xd77fcf04f14a0ea5+        .quad   0x3df23ff7a4ba0c47+        .quad   0x3a10dfe132ce3c85+        .quad   0xe07f4e8aea17cea0+        .quad   0x2fd515463a1fc1fd+        .quad   0x175322fd31f2c0f1+        .quad   0x1fa1d01d861e5d15++        // 2^216 * 2 * G++        .quad   0xcc8055947d599832+        .quad   0x1e4656da37f15520+        .quad   0x99f6f7744e059320+        .quad   0x773563bc6a75cf33+        .quad   0x38dcac00d1df94ab+        .quad   0x2e712bddd1080de9+        .quad   0x7f13e93efdd5e262+        .quad   0x73fced18ee9a01e5+        .quad   0x06b1e90863139cb3+        .quad   0xa493da67c5a03ecd+        .quad   0x8d77cec8ad638932+        .quad   0x1f426b701b864f44++        // 2^216 * 3 * G++        .quad   0xefc9264c41911c01+        .quad   0xf1a3b7b817a22c25+        .quad   0x5875da6bf30f1447+        .quad   0x4e1af5271d31b090+        .quad   0xf17e35c891a12552+        .quad   0xb76b8153575e9c76+        .quad   0xfa83406f0d9b723e+        .quad   0x0b76bb1b3fa7e438+        .quad   0x08b8c1f97f92939b+        .quad   0xbe6771cbd444ab6e+        .quad   0x22e5646399bb8017+        .quad   0x7b6dd61eb772a955++        // 2^216 * 4 * G++        .quad   0xb7adc1e850f33d92+        .quad   0x7998fa4f608cd5cf+        .quad   0xad962dbd8dfc5bdb+        .quad   0x703e9bceaf1d2f4f+        .quad   0x5730abf9ab01d2c7+        .quad   0x16fb76dc40143b18+        .quad   0x866cbe65a0cbb281+        .quad   0x53fa9b659bff6afe+        .quad   0x6c14c8e994885455+        .quad   0x843a5d6665aed4e5+        .quad   0x181bb73ebcd65af1+        .quad   0x398d93e5c4c61f50++        // 2^216 * 5 * G++        .quad   0x1c4bd16733e248f3+        .quad   0xbd9e128715bf0a5f+        .quad   0xd43f8cf0a10b0376+        .quad   0x53b09b5ddf191b13+        .quad   0xc3877c60d2e7e3f2+        .quad   0x3b34aaa030828bb1+        .quad   0x283e26e7739ef138+        .quad   0x699c9c9002c30577+        .quad   0xf306a7235946f1cc+        .quad   0x921718b5cce5d97d+        .quad   0x28cdd24781b4e975+        .quad   0x51caf30c6fcdd907++        // 2^216 * 6 * G++        .quad   0xa60ba7427674e00a+        .quad   0x630e8570a17a7bf3+        .quad   0x3758563dcf3324cc+        .quad   0x5504aa292383fdaa+        .quad   0x737af99a18ac54c7+        .quad   0x903378dcc51cb30f+        .quad   0x2b89bc334ce10cc7+        .quad   0x12ae29c189f8e99a+        .quad   0xa99ec0cb1f0d01cf+        .quad   0x0dd1efcc3a34f7ae+        .quad   0x55ca7521d09c4e22+        .quad   0x5fd14fe958eba5ea++        // 2^216 * 7 * G++        .quad   0xb5dc2ddf2845ab2c+        .quad   0x069491b10a7fe993+        .quad   0x4daaf3d64002e346+        .quad   0x093ff26e586474d1+        .quad   0x3c42fe5ebf93cb8e+        .quad   0xbedfa85136d4565f+        .quad   0xe0f0859e884220e8+        .quad   0x7dd73f960725d128+        .quad   0xb10d24fe68059829+        .quad   0x75730672dbaf23e5+        .quad   0x1367253ab457ac29+        .quad   0x2f59bcbc86b470a4++        // 2^216 * 8 * G++        .quad   0x83847d429917135f+        .quad   0xad1b911f567d03d7+        .quad   0x7e7748d9be77aad1+        .quad   0x5458b42e2e51af4a+        .quad   0x7041d560b691c301+        .quad   0x85201b3fadd7e71e+        .quad   0x16c2e16311335585+        .quad   0x2aa55e3d010828b1+        .quad   0xed5192e60c07444f+        .quad   0x42c54e2d74421d10+        .quad   0x352b4c82fdb5c864+        .quad   0x13e9004a8a768664++        // 2^220 * 1 * G++        .quad   0xcbb5b5556c032bff+        .quad   0xdf7191b729297a3a+        .quad   0xc1ff7326aded81bb+        .quad   0x71ade8bb68be03f5+        .quad   0x1e6284c5806b467c+        .quad   0xc5f6997be75d607b+        .quad   0x8b67d958b378d262+        .quad   0x3d88d66a81cd8b70+        .quad   0x8b767a93204ed789+        .quad   0x762fcacb9fa0ae2a+        .quad   0x771febcc6dce4887+        .quad   0x343062158ff05fb3++        // 2^220 * 2 * G++        .quad   0xe05da1a7e1f5bf49+        .quad   0x26457d6dd4736092+        .quad   0x77dcb07773cc32f6+        .quad   0x0a5d94969cdd5fcd+        .quad   0xfce219072a7b31b4+        .quad   0x4d7adc75aa578016+        .quad   0x0ec276a687479324+        .quad   0x6d6d9d5d1fda4beb+        .quad   0x22b1a58ae9b08183+        .quad   0xfd95d071c15c388b+        .quad   0xa9812376850a0517+        .quad   0x33384cbabb7f335e++        // 2^220 * 3 * G++        .quad   0x3c6fa2680ca2c7b5+        .quad   0x1b5082046fb64fda+        .quad   0xeb53349c5431d6de+        .quad   0x5278b38f6b879c89+        .quad   0x33bc627a26218b8d+        .quad   0xea80b21fc7a80c61+        .quad   0x9458b12b173e9ee6+        .quad   0x076247be0e2f3059+        .quad   0x52e105f61416375a+        .quad   0xec97af3685abeba4+        .quad   0x26e6b50623a67c36+        .quad   0x5cf0e856f3d4fb01++        // 2^220 * 4 * G++        .quad   0xf6c968731ae8cab4+        .quad   0x5e20741ecb4f92c5+        .quad   0x2da53be58ccdbc3e+        .quad   0x2dddfea269970df7+        .quad   0xbeaece313db342a8+        .quad   0xcba3635b842db7ee+        .quad   0xe88c6620817f13ef+        .quad   0x1b9438aa4e76d5c6+        .quad   0x8a50777e166f031a+        .quad   0x067b39f10fb7a328+        .quad   0x1925c9a6010fbd76+        .quad   0x6df9b575cc740905++        // 2^220 * 5 * G++        .quad   0x42c1192927f6bdcf+        .quad   0x8f91917a403d61ca+        .quad   0xdc1c5a668b9e1f61+        .quad   0x1596047804ec0f8d+        .quad   0xecdfc35b48cade41+        .quad   0x6a88471fb2328270+        .quad   0x740a4a2440a01b6a+        .quad   0x471e5796003b5f29+        .quad   0xda96bbb3aced37ac+        .quad   0x7a2423b5e9208cea+        .quad   0x24cc5c3038aebae2+        .quad   0x50c356afdc5dae2f++        // 2^220 * 6 * G++        .quad   0x09dcbf4341c30318+        .quad   0xeeba061183181dce+        .quad   0xc179c0cedc1e29a1+        .quad   0x1dbf7b89073f35b0+        .quad   0xcfed9cdf1b31b964+        .quad   0xf486a9858ca51af3+        .quad   0x14897265ea8c1f84+        .quad   0x784a53dd932acc00+        .quad   0x2d99f9df14fc4920+        .quad   0x76ccb60cc4499fe5+        .quad   0xa4132cbbe5cf0003+        .quad   0x3f93d82354f000ea++        // 2^220 * 7 * G++        .quad   0x8183e7689e04ce85+        .quad   0x678fb71e04465341+        .quad   0xad92058f6688edac+        .quad   0x5da350d3532b099a+        .quad   0xeaac12d179e14978+        .quad   0xff923ff3bbebff5e+        .quad   0x4af663e40663ce27+        .quad   0x0fd381a811a5f5ff+        .quad   0xf256aceca436df54+        .quad   0x108b6168ae69d6e8+        .quad   0x20d986cb6b5d036c+        .quad   0x655957b9fee2af50++        // 2^220 * 8 * G++        .quad   0xaea8b07fa902030f+        .quad   0xf88c766af463d143+        .quad   0x15b083663c787a60+        .quad   0x08eab1148267a4a8+        .quad   0xbdc1409bd002d0ac+        .quad   0x66660245b5ccd9a6+        .quad   0x82317dc4fade85ec+        .quad   0x02fe934b6ad7df0d+        .quad   0xef5cf100cfb7ea74+        .quad   0x22897633a1cb42ac+        .quad   0xd4ce0c54cef285e2+        .quad   0x30408c048a146a55++        // 2^224 * 1 * G++        .quad   0x739d8845832fcedb+        .quad   0xfa38d6c9ae6bf863+        .quad   0x32bc0dcab74ffef7+        .quad   0x73937e8814bce45e+        .quad   0xbb2e00c9193b877f+        .quad   0xece3a890e0dc506b+        .quad   0xecf3b7c036de649f+        .quad   0x5f46040898de9e1a+        .quad   0xb9037116297bf48d+        .quad   0xa9d13b22d4f06834+        .quad   0xe19715574696bdc6+        .quad   0x2cf8a4e891d5e835++        // 2^224 * 2 * G++        .quad   0x6d93fd8707110f67+        .quad   0xdd4c09d37c38b549+        .quad   0x7cb16a4cc2736a86+        .quad   0x2049bd6e58252a09+        .quad   0x2cb5487e17d06ba2+        .quad   0x24d2381c3950196b+        .quad   0xd7659c8185978a30+        .quad   0x7a6f7f2891d6a4f6+        .quad   0x7d09fd8d6a9aef49+        .quad   0xf0ee60be5b3db90b+        .quad   0x4c21b52c519ebfd4+        .quad   0x6011aadfc545941d++        // 2^224 * 3 * G++        .quad   0x5f67926dcf95f83c+        .quad   0x7c7e856171289071+        .quad   0xd6a1e7f3998f7a5b+        .quad   0x6fc5cc1b0b62f9e0+        .quad   0x63ded0c802cbf890+        .quad   0xfbd098ca0dff6aaa+        .quad   0x624d0afdb9b6ed99+        .quad   0x69ce18b779340b1e+        .quad   0xd1ef5528b29879cb+        .quad   0xdd1aae3cd47e9092+        .quad   0x127e0442189f2352+        .quad   0x15596b3ae57101f1++        // 2^224 * 4 * G++        .quad   0x462739d23f9179a2+        .quad   0xff83123197d6ddcf+        .quad   0x1307deb553f2148a+        .quad   0x0d2237687b5f4dda+        .quad   0x09ff31167e5124ca+        .quad   0x0be4158bd9c745df+        .quad   0x292b7d227ef556e5+        .quad   0x3aa4e241afb6d138+        .quad   0x2cc138bf2a3305f5+        .quad   0x48583f8fa2e926c3+        .quad   0x083ab1a25549d2eb+        .quad   0x32fcaa6e4687a36c++        // 2^224 * 5 * G++        .quad   0x7bc56e8dc57d9af5+        .quad   0x3e0bd2ed9df0bdf2+        .quad   0xaac014de22efe4a3+        .quad   0x4627e9cefebd6a5c+        .quad   0x3207a4732787ccdf+        .quad   0x17e31908f213e3f8+        .quad   0xd5b2ecd7f60d964e+        .quad   0x746f6336c2600be9+        .quad   0x3f4af345ab6c971c+        .quad   0xe288eb729943731f+        .quad   0x33596a8a0344186d+        .quad   0x7b4917007ed66293++        // 2^224 * 6 * G++        .quad   0x2d85fb5cab84b064+        .quad   0x497810d289f3bc14+        .quad   0x476adc447b15ce0c+        .quad   0x122ba376f844fd7b+        .quad   0x54341b28dd53a2dd+        .quad   0xaa17905bdf42fc3f+        .quad   0x0ff592d94dd2f8f4+        .quad   0x1d03620fe08cd37d+        .quad   0xc20232cda2b4e554+        .quad   0x9ed0fd42115d187f+        .quad   0x2eabb4be7dd479d9+        .quad   0x02c70bf52b68ec4c++        // 2^224 * 7 * G++        .quad   0xa287ec4b5d0b2fbb+        .quad   0x415c5790074882ca+        .quad   0xe044a61ec1d0815c+        .quad   0x26334f0a409ef5e0+        .quad   0xace532bf458d72e1+        .quad   0x5be768e07cb73cb5+        .quad   0x56cf7d94ee8bbde7+        .quad   0x6b0697e3feb43a03+        .quad   0xb6c8f04adf62a3c0+        .quad   0x3ef000ef076da45d+        .quad   0x9c9cb95849f0d2a9+        .quad   0x1cc37f43441b2fae++        // 2^224 * 8 * G++        .quad   0x508f565a5cc7324f+        .quad   0xd061c4c0e506a922+        .quad   0xfb18abdb5c45ac19+        .quad   0x6c6809c10380314a+        .quad   0xd76656f1c9ceaeb9+        .quad   0x1c5b15f818e5656a+        .quad   0x26e72832844c2334+        .quad   0x3a346f772f196838+        .quad   0xd2d55112e2da6ac8+        .quad   0xe9bd0331b1e851ed+        .quad   0x960746dd8ec67262+        .quad   0x05911b9f6ef7c5d0++        // 2^228 * 1 * G++        .quad   0xe9dcd756b637ff2d+        .quad   0xec4c348fc987f0c4+        .quad   0xced59285f3fbc7b7+        .quad   0x3305354793e1ea87+        .quad   0x01c18980c5fe9f94+        .quad   0xcd656769716fd5c8+        .quad   0x816045c3d195a086+        .quad   0x6e2b7f3266cc7982+        .quad   0xcc802468f7c3568f+        .quad   0x9de9ba8219974cb3+        .quad   0xabb7229cb5b81360+        .quad   0x44e2017a6fbeba62++        // 2^228 * 2 * G++        .quad   0xc4c2a74354dab774+        .quad   0x8e5d4c3c4eaf031a+        .quad   0xb76c23d242838f17+        .quad   0x749a098f68dce4ea+        .quad   0x87f82cf3b6ca6ecd+        .quad   0x580f893e18f4a0c2+        .quad   0x058930072604e557+        .quad   0x6cab6ac256d19c1d+        .quad   0xdcdfe0a02cc1de60+        .quad   0x032665ff51c5575b+        .quad   0x2c0c32f1073abeeb+        .quad   0x6a882014cd7b8606++        // 2^228 * 3 * G++        .quad   0xa52a92fea4747fb5+        .quad   0xdc12a4491fa5ab89+        .quad   0xd82da94bb847a4ce+        .quad   0x4d77edce9512cc4e+        .quad   0xd111d17caf4feb6e+        .quad   0x050bba42b33aa4a3+        .quad   0x17514c3ceeb46c30+        .quad   0x54bedb8b1bc27d75+        .quad   0x77c8e14577e2189c+        .quad   0xa3e46f6aff99c445+        .quad   0x3144dfc86d335343+        .quad   0x3a96559e7c4216a9++        // 2^228 * 4 * G++        .quad   0x12550d37f42ad2ee+        .quad   0x8b78e00498a1fbf5+        .quad   0x5d53078233894cb2+        .quad   0x02c84e4e3e498d0c+        .quad   0x4493896880baaa52+        .quad   0x4c98afc4f285940e+        .quad   0xef4aa79ba45448b6+        .quad   0x5278c510a57aae7f+        .quad   0xa54dd074294c0b94+        .quad   0xf55d46b8df18ffb6+        .quad   0xf06fecc58dae8366+        .quad   0x588657668190d165++        // 2^228 * 5 * G++        .quad   0xd47712311aef7117+        .quad   0x50343101229e92c7+        .quad   0x7a95e1849d159b97+        .quad   0x2449959b8b5d29c9+        .quad   0xbf5834f03de25cc3+        .quad   0xb887c8aed6815496+        .quad   0x5105221a9481e892+        .quad   0x6760ed19f7723f93+        .quad   0x669ba3b7ac35e160+        .quad   0x2eccf73fba842056+        .quad   0x1aec1f17c0804f07+        .quad   0x0d96bc031856f4e7++        // 2^228 * 6 * G++        .quad   0x3318be7775c52d82+        .quad   0x4cb764b554d0aab9+        .quad   0xabcf3d27cc773d91+        .quad   0x3bf4d1848123288a+        .quad   0xb1d534b0cc7505e1+        .quad   0x32cd003416c35288+        .quad   0xcb36a5800762c29d+        .quad   0x5bfe69b9237a0bf8+        .quad   0x183eab7e78a151ab+        .quad   0xbbe990c999093763+        .quad   0xff717d6e4ac7e335+        .quad   0x4c5cddb325f39f88++        // 2^228 * 7 * G++        .quad   0xc0f6b74d6190a6eb+        .quad   0x20ea81a42db8f4e4+        .quad   0xa8bd6f7d97315760+        .quad   0x33b1d60262ac7c21+        .quad   0x57750967e7a9f902+        .quad   0x2c37fdfc4f5b467e+        .quad   0xb261663a3177ba46+        .quad   0x3a375e78dc2d532b+        .quad   0x8141e72f2d4dddea+        .quad   0xe6eafe9862c607c8+        .quad   0x23c28458573cafd0+        .quad   0x46b9476f4ff97346++        // 2^228 * 8 * G++        .quad   0x0c1ffea44f901e5c+        .quad   0x2b0b6fb72184b782+        .quad   0xe587ff910114db88+        .quad   0x37130f364785a142+        .quad   0x1215505c0d58359f+        .quad   0x2a2013c7fc28c46b+        .quad   0x24a0a1af89ea664e+        .quad   0x4400b638a1130e1f+        .quad   0x3a01b76496ed19c3+        .quad   0x31e00ab0ed327230+        .quad   0x520a885783ca15b1+        .quad   0x06aab9875accbec7++        // 2^232 * 1 * G++        .quad   0xc1339983f5df0ebb+        .quad   0xc0f3758f512c4cac+        .quad   0x2cf1130a0bb398e1+        .quad   0x6b3cecf9aa270c62+        .quad   0x5349acf3512eeaef+        .quad   0x20c141d31cc1cb49+        .quad   0x24180c07a99a688d+        .quad   0x555ef9d1c64b2d17+        .quad   0x36a770ba3b73bd08+        .quad   0x624aef08a3afbf0c+        .quad   0x5737ff98b40946f2+        .quad   0x675f4de13381749d++        // 2^232 * 2 * G++        .quad   0x0e2c52036b1782fc+        .quad   0x64816c816cad83b4+        .quad   0xd0dcbdd96964073e+        .quad   0x13d99df70164c520+        .quad   0xa12ff6d93bdab31d+        .quad   0x0725d80f9d652dfe+        .quad   0x019c4ff39abe9487+        .quad   0x60f450b882cd3c43+        .quad   0x014b5ec321e5c0ca+        .quad   0x4fcb69c9d719bfa2+        .quad   0x4e5f1c18750023a0+        .quad   0x1c06de9e55edac80++        // 2^232 * 3 * G++        .quad   0x990f7ad6a33ec4e2+        .quad   0x6608f938be2ee08e+        .quad   0x9ca143c563284515+        .quad   0x4cf38a1fec2db60d+        .quad   0xffd52b40ff6d69aa+        .quad   0x34530b18dc4049bb+        .quad   0x5e4a5c2fa34d9897+        .quad   0x78096f8e7d32ba2d+        .quad   0xa0aaaa650dfa5ce7+        .quad   0xf9c49e2a48b5478c+        .quad   0x4f09cc7d7003725b+        .quad   0x373cad3a26091abe++        // 2^232 * 4 * G++        .quad   0xb294634d82c9f57c+        .quad   0x1fcbfde124934536+        .quad   0x9e9c4db3418cdb5a+        .quad   0x0040f3d9454419fc+        .quad   0xf1bea8fb89ddbbad+        .quad   0x3bcb2cbc61aeaecb+        .quad   0x8f58a7bb1f9b8d9d+        .quad   0x21547eda5112a686+        .quad   0xdefde939fd5986d3+        .quad   0xf4272c89510a380c+        .quad   0xb72ba407bb3119b9+        .quad   0x63550a334a254df4++        // 2^232 * 5 * G++        .quad   0x6507d6edb569cf37+        .quad   0x178429b00ca52ee1+        .quad   0xea7c0090eb6bd65d+        .quad   0x3eea62c7daf78f51+        .quad   0x9bba584572547b49+        .quad   0xf305c6fae2c408e0+        .quad   0x60e8fa69c734f18d+        .quad   0x39a92bafaa7d767a+        .quad   0x9d24c713e693274e+        .quad   0x5f63857768dbd375+        .quad   0x70525560eb8ab39a+        .quad   0x68436a0665c9c4cd++        // 2^232 * 6 * G++        .quad   0xbc0235e8202f3f27+        .quad   0xc75c00e264f975b0+        .quad   0x91a4e9d5a38c2416+        .quad   0x17b6e7f68ab789f9+        .quad   0x1e56d317e820107c+        .quad   0xc5266844840ae965+        .quad   0xc1e0a1c6320ffc7a+        .quad   0x5373669c91611472+        .quad   0x5d2814ab9a0e5257+        .quad   0x908f2084c9cab3fc+        .quad   0xafcaf5885b2d1eca+        .quad   0x1cb4b5a678f87d11++        // 2^232 * 7 * G++        .quad   0xb664c06b394afc6c+        .quad   0x0c88de2498da5fb1+        .quad   0x4f8d03164bcad834+        .quad   0x330bca78de7434a2+        .quad   0x6b74aa62a2a007e7+        .quad   0xf311e0b0f071c7b1+        .quad   0x5707e438000be223+        .quad   0x2dc0fd2d82ef6eac+        .quad   0x982eff841119744e+        .quad   0xf9695e962b074724+        .quad   0xc58ac14fbfc953fb+        .quad   0x3c31be1b369f1cf5++        // 2^232 * 8 * G++        .quad   0xb0f4864d08948aee+        .quad   0x07dc19ee91ba1c6f+        .quad   0x7975cdaea6aca158+        .quad   0x330b61134262d4bb+        .quad   0xc168bc93f9cb4272+        .quad   0xaeb8711fc7cedb98+        .quad   0x7f0e52aa34ac8d7a+        .quad   0x41cec1097e7d55bb+        .quad   0xf79619d7a26d808a+        .quad   0xbb1fd49e1d9e156d+        .quad   0x73d7c36cdba1df27+        .quad   0x26b44cd91f28777d++        // 2^236 * 1 * G++        .quad   0x300a9035393aa6d8+        .quad   0x2b501131a12bb1cd+        .quad   0x7b1ff677f093c222+        .quad   0x4309c1f8cab82bad+        .quad   0xaf44842db0285f37+        .quad   0x8753189047efc8df+        .quad   0x9574e091f820979a+        .quad   0x0e378d6069615579+        .quad   0xd9fa917183075a55+        .quad   0x4bdb5ad26b009fdc+        .quad   0x7829ad2cd63def0e+        .quad   0x078fc54975fd3877++        // 2^236 * 2 * G++        .quad   0x87dfbd1428878f2d+        .quad   0x134636dd1e9421a1+        .quad   0x4f17c951257341a3+        .quad   0x5df98d4bad296cb8+        .quad   0xe2004b5bb833a98a+        .quad   0x44775dec2d4c3330+        .quad   0x3aa244067eace913+        .quad   0x272630e3d58e00a9+        .quad   0xf3678fd0ecc90b54+        .quad   0xf001459b12043599+        .quad   0x26725fbc3758b89b+        .quad   0x4325e4aa73a719ae++        // 2^236 * 3 * G++        .quad   0x657dc6ef433c3493+        .quad   0x65375e9f80dbf8c3+        .quad   0x47fd2d465b372dae+        .quad   0x4966ab79796e7947+        .quad   0xed24629acf69f59d+        .quad   0x2a4a1ccedd5abbf4+        .quad   0x3535ca1f56b2d67b+        .quad   0x5d8c68d043b1b42d+        .quad   0xee332d4de3b42b0a+        .quad   0xd84e5a2b16a4601c+        .quad   0x78243877078ba3e4+        .quad   0x77ed1eb4184ee437++        // 2^236 * 4 * G++        .quad   0xbfd4e13f201839a0+        .quad   0xaeefffe23e3df161+        .quad   0xb65b04f06b5d1fe3+        .quad   0x52e085fb2b62fbc0+        .quad   0x185d43f89e92ed1a+        .quad   0xb04a1eeafe4719c6+        .quad   0x499fbe88a6f03f4f+        .quad   0x5d8b0d2f3c859bdd+        .quad   0x124079eaa54cf2ba+        .quad   0xd72465eb001b26e7+        .quad   0x6843bcfdc97af7fd+        .quad   0x0524b42b55eacd02++        // 2^236 * 5 * G++        .quad   0xfd0d5dbee45447b0+        .quad   0x6cec351a092005ee+        .quad   0x99a47844567579cb+        .quad   0x59d242a216e7fa45+        .quad   0xbc18dcad9b829eac+        .quad   0x23ae7d28b5f579d0+        .quad   0xc346122a69384233+        .quad   0x1a6110b2e7d4ac89+        .quad   0x4f833f6ae66997ac+        .quad   0x6849762a361839a4+        .quad   0x6985dec1970ab525+        .quad   0x53045e89dcb1f546++        // 2^236 * 6 * G++        .quad   0xcb8bb346d75353db+        .quad   0xfcfcb24bae511e22+        .quad   0xcba48d40d50ae6ef+        .quad   0x26e3bae5f4f7cb5d+        .quad   0x84da3cde8d45fe12+        .quad   0xbd42c218e444e2d2+        .quad   0xa85196781f7e3598+        .quad   0x7642c93f5616e2b2+        .quad   0x2323daa74595f8e4+        .quad   0xde688c8b857abeb4+        .quad   0x3fc48e961c59326e+        .quad   0x0b2e73ca15c9b8ba++        // 2^236 * 7 * G++        .quad   0xd6bb4428c17f5026+        .quad   0x9eb27223fb5a9ca7+        .quad   0xe37ba5031919c644+        .quad   0x21ce380db59a6602+        .quad   0x0e3fbfaf79c03a55+        .quad   0x3077af054cbb5acf+        .quad   0xd5c55245db3de39f+        .quad   0x015e68c1476a4af7+        .quad   0xc1d5285220066a38+        .quad   0x95603e523570aef3+        .quad   0x832659a7226b8a4d+        .quad   0x5dd689091f8eedc9++        // 2^236 * 8 * G++        .quad   0xcbac84debfd3c856+        .quad   0x1624c348b35ff244+        .quad   0xb7f88dca5d9cad07+        .quad   0x3b0e574da2c2ebe8+        .quad   0x1d022591a5313084+        .quad   0xca2d4aaed6270872+        .quad   0x86a12b852f0bfd20+        .quad   0x56e6c439ad7da748+        .quad   0xc704ff4942bdbae6+        .quad   0x5e21ade2b2de1f79+        .quad   0xe95db3f35652fad8+        .quad   0x0822b5378f08ebc1++        // 2^240 * 1 * G++        .quad   0x51f048478f387475+        .quad   0xb25dbcf49cbecb3c+        .quad   0x9aab1244d99f2055+        .quad   0x2c709e6c1c10a5d6+        .quad   0xe1b7f29362730383+        .quad   0x4b5279ffebca8a2c+        .quad   0xdafc778abfd41314+        .quad   0x7deb10149c72610f+        .quad   0xcb62af6a8766ee7a+        .quad   0x66cbec045553cd0e+        .quad   0x588001380f0be4b5+        .quad   0x08e68e9ff62ce2ea++        // 2^240 * 2 * G++        .quad   0x34ad500a4bc130ad+        .quad   0x8d38db493d0bd49c+        .quad   0xa25c3d98500a89be+        .quad   0x2f1f3f87eeba3b09+        .quad   0x2f2d09d50ab8f2f9+        .quad   0xacb9218dc55923df+        .quad   0x4a8f342673766cb9+        .quad   0x4cb13bd738f719f5+        .quad   0xf7848c75e515b64a+        .quad   0xa59501badb4a9038+        .quad   0xc20d313f3f751b50+        .quad   0x19a1e353c0ae2ee8++        // 2^240 * 3 * G++        .quad   0x7d1c7560bafa05c3+        .quad   0xb3e1a0a0c6e55e61+        .quad   0xe3529718c0d66473+        .quad   0x41546b11c20c3486+        .quad   0xb42172cdd596bdbd+        .quad   0x93e0454398eefc40+        .quad   0x9fb15347b44109b5+        .quad   0x736bd3990266ae34+        .quad   0x85532d509334b3b4+        .quad   0x46fd114b60816573+        .quad   0xcc5f5f30425c8375+        .quad   0x412295a2b87fab5c++        // 2^240 * 4 * G++        .quad   0x19c99b88f57ed6e9+        .quad   0x5393cb266df8c825+        .quad   0x5cee3213b30ad273+        .quad   0x14e153ebb52d2e34+        .quad   0x2e655261e293eac6+        .quad   0x845a92032133acdb+        .quad   0x460975cb7900996b+        .quad   0x0760bb8d195add80+        .quad   0x413e1a17cde6818a+        .quad   0x57156da9ed69a084+        .quad   0x2cbf268f46caccb1+        .quad   0x6b34be9bc33ac5f2++        // 2^240 * 5 * G++        .quad   0xf3df2f643a78c0b2+        .quad   0x4c3e971ef22e027c+        .quad   0xec7d1c5e49c1b5a3+        .quad   0x2012c18f0922dd2d+        .quad   0x11fc69656571f2d3+        .quad   0xc6c9e845530e737a+        .quad   0xe33ae7a2d4fe5035+        .quad   0x01b9c7b62e6dd30b+        .quad   0x880b55e55ac89d29+        .quad   0x1483241f45a0a763+        .quad   0x3d36efdfc2e76c1f+        .quad   0x08af5b784e4bade8++        // 2^240 * 6 * G++        .quad   0x283499dc881f2533+        .quad   0x9d0525da779323b6+        .quad   0x897addfb673441f4+        .quad   0x32b79d71163a168d+        .quad   0xe27314d289cc2c4b+        .quad   0x4be4bd11a287178d+        .quad   0x18d528d6fa3364ce+        .quad   0x6423c1d5afd9826e+        .quad   0xcc85f8d9edfcb36a+        .quad   0x22bcc28f3746e5f9+        .quad   0xe49de338f9e5d3cd+        .quad   0x480a5efbc13e2dcc++        // 2^240 * 7 * G++        .quad   0x0b51e70b01622071+        .quad   0x06b505cf8b1dafc5+        .quad   0x2c6bb061ef5aabcd+        .quad   0x47aa27600cb7bf31+        .quad   0xb6614ce442ce221f+        .quad   0x6e199dcc4c053928+        .quad   0x663fb4a4dc1cbe03+        .quad   0x24b31d47691c8e06+        .quad   0x2a541eedc015f8c3+        .quad   0x11a4fe7e7c693f7c+        .quad   0xf0af66134ea278d6+        .quad   0x545b585d14dda094++        // 2^240 * 8 * G++        .quad   0x67bf275ea0d43a0f+        .quad   0xade68e34089beebe+        .quad   0x4289134cd479e72e+        .quad   0x0f62f9c332ba5454+        .quad   0x6204e4d0e3b321e1+        .quad   0x3baa637a28ff1e95+        .quad   0x0b0ccffd5b99bd9e+        .quad   0x4d22dc3e64c8d071+        .quad   0xfcb46589d63b5f39+        .quad   0x5cae6a3f57cbcf61+        .quad   0xfebac2d2953afa05+        .quad   0x1c0fa01a36371436++        // 2^244 * 1 * G++        .quad   0xe7547449bc7cd692+        .quad   0x0f9abeaae6f73ddf+        .quad   0x4af01ca700837e29+        .quad   0x63ab1b5d3f1bc183+        .quad   0xc11ee5e854c53fae+        .quad   0x6a0b06c12b4f3ff4+        .quad   0x33540f80e0b67a72+        .quad   0x15f18fc3cd07e3ef+        .quad   0x32750763b028f48c+        .quad   0x06020740556a065f+        .quad   0xd53bd812c3495b58+        .quad   0x08706c9b865f508d++        // 2^244 * 2 * G++        .quad   0xf37ca2ab3d343dff+        .quad   0x1a8c6a2d80abc617+        .quad   0x8e49e035d4ccffca+        .quad   0x48b46beebaa1d1b9+        .quad   0xcc991b4138b41246+        .quad   0x243b9c526f9ac26b+        .quad   0xb9ef494db7cbabbd+        .quad   0x5fba433dd082ed00+        .quad   0x9c49e355c9941ad0+        .quad   0xb9734ade74498f84+        .quad   0x41c3fed066663e5c+        .quad   0x0ecfedf8e8e710b3++        // 2^244 * 3 * G++        .quad   0x76430f9f9cd470d9+        .quad   0xb62acc9ba42f6008+        .quad   0x1898297c59adad5e+        .quad   0x7789dd2db78c5080+        .quad   0x744f7463e9403762+        .quad   0xf79a8dee8dfcc9c9+        .quad   0x163a649655e4cde3+        .quad   0x3b61788db284f435+        .quad   0xb22228190d6ef6b2+        .quad   0xa94a66b246ce4bfa+        .quad   0x46c1a77a4f0b6cc7+        .quad   0x4236ccffeb7338cf++        // 2^244 * 4 * G++        .quad   0x8497404d0d55e274+        .quad   0x6c6663d9c4ad2b53+        .quad   0xec2fb0d9ada95734+        .quad   0x2617e120cdb8f73c+        .quad   0x3bd82dbfda777df6+        .quad   0x71b177cc0b98369e+        .quad   0x1d0e8463850c3699+        .quad   0x5a71945b48e2d1f1+        .quad   0x6f203dd5405b4b42+        .quad   0x327ec60410b24509+        .quad   0x9c347230ac2a8846+        .quad   0x77de29fc11ffeb6a++        // 2^244 * 5 * G++        .quad   0xb0ac57c983b778a8+        .quad   0x53cdcca9d7fe912c+        .quad   0x61c2b854ff1f59dc+        .quad   0x3a1a2cf0f0de7dac+        .quad   0x835e138fecced2ca+        .quad   0x8c9eaf13ea963b9a+        .quad   0xc95fbfc0b2160ea6+        .quad   0x575e66f3ad877892+        .quad   0x99803a27c88fcb3a+        .quad   0x345a6789275ec0b0+        .quad   0x459789d0ff6c2be5+        .quad   0x62f882651e70a8b2++        // 2^244 * 6 * G++        .quad   0x085ae2c759ff1be4+        .quad   0x149145c93b0e40b7+        .quad   0xc467e7fa7ff27379+        .quad   0x4eeecf0ad5c73a95+        .quad   0x6d822986698a19e0+        .quad   0xdc9821e174d78a71+        .quad   0x41a85f31f6cb1f47+        .quad   0x352721c2bcda9c51+        .quad   0x48329952213fc985+        .quad   0x1087cf0d368a1746+        .quad   0x8e5261b166c15aa5+        .quad   0x2d5b2d842ed24c21++        // 2^244 * 7 * G++        .quad   0x02cfebd9ebd3ded1+        .quad   0xd45b217739021974+        .quad   0x7576f813fe30a1b7+        .quad   0x5691b6f9a34ef6c2+        .quad   0x5eb7d13d196ac533+        .quad   0x377234ecdb80be2b+        .quad   0xe144cffc7cf5ae24+        .quad   0x5226bcf9c441acec+        .quad   0x79ee6c7223e5b547+        .quad   0x6f5f50768330d679+        .quad   0xed73e1e96d8adce9+        .quad   0x27c3da1e1d8ccc03++        // 2^244 * 8 * G++        .quad   0x7eb9efb23fe24c74+        .quad   0x3e50f49f1651be01+        .quad   0x3ea732dc21858dea+        .quad   0x17377bd75bb810f9+        .quad   0x28302e71630ef9f6+        .quad   0xc2d4a2032b64cee0+        .quad   0x090820304b6292be+        .quad   0x5fca747aa82adf18+        .quad   0x232a03c35c258ea5+        .quad   0x86f23a2c6bcb0cf1+        .quad   0x3dad8d0d2e442166+        .quad   0x04a8933cab76862b++        // 2^248 * 1 * G++        .quad   0xd2c604b622943dff+        .quad   0xbc8cbece44cfb3a0+        .quad   0x5d254ff397808678+        .quad   0x0fa3614f3b1ca6bf+        .quad   0x69082b0e8c936a50+        .quad   0xf9c9a035c1dac5b6+        .quad   0x6fb73e54c4dfb634+        .quad   0x4005419b1d2bc140+        .quad   0xa003febdb9be82f0+        .quad   0x2089c1af3a44ac90+        .quad   0xf8499f911954fa8e+        .quad   0x1fba218aef40ab42++        // 2^248 * 2 * G++        .quad   0xab549448fac8f53e+        .quad   0x81f6e89a7ba63741+        .quad   0x74fd6c7d6c2b5e01+        .quad   0x392e3acaa8c86e42+        .quad   0x4f3e57043e7b0194+        .quad   0xa81d3eee08daaf7f+        .quad   0xc839c6ab99dcdef1+        .quad   0x6c535d13ff7761d5+        .quad   0x4cbd34e93e8a35af+        .quad   0x2e0781445887e816+        .quad   0x19319c76f29ab0ab+        .quad   0x25e17fe4d50ac13b++        // 2^248 * 3 * G++        .quad   0x0a289bd71e04f676+        .quad   0x208e1c52d6420f95+        .quad   0x5186d8b034691fab+        .quad   0x255751442a9fb351+        .quad   0x915f7ff576f121a7+        .quad   0xc34a32272fcd87e3+        .quad   0xccba2fde4d1be526+        .quad   0x6bba828f8969899b+        .quad   0xe2d1bc6690fe3901+        .quad   0x4cb54a18a0997ad5+        .quad   0x971d6914af8460d4+        .quad   0x559d504f7f6b7be4++        // 2^248 * 4 * G++        .quad   0xa7738378b3eb54d5+        .quad   0x1d69d366a5553c7c+        .quad   0x0a26cf62f92800ba+        .quad   0x01ab12d5807e3217+        .quad   0x9c4891e7f6d266fd+        .quad   0x0744a19b0307781b+        .quad   0x88388f1d6061e23b+        .quad   0x123ea6a3354bd50e+        .quad   0x118d189041e32d96+        .quad   0xb9ede3c2d8315848+        .quad   0x1eab4271d83245d9+        .quad   0x4a3961e2c918a154++        // 2^248 * 5 * G++        .quad   0x71dc3be0f8e6bba0+        .quad   0xd6cef8347effe30a+        .quad   0xa992425fe13a476a+        .quad   0x2cd6bce3fb1db763+        .quad   0x0327d644f3233f1e+        .quad   0x499a260e34fcf016+        .quad   0x83b5a716f2dab979+        .quad   0x68aceead9bd4111f+        .quad   0x38b4c90ef3d7c210+        .quad   0x308e6e24b7ad040c+        .quad   0x3860d9f1b7e73e23+        .quad   0x595760d5b508f597++        // 2^248 * 6 * G++        .quad   0x6129bfe104aa6397+        .quad   0x8f960008a4a7fccb+        .quad   0x3f8bc0897d909458+        .quad   0x709fa43edcb291a9+        .quad   0x882acbebfd022790+        .quad   0x89af3305c4115760+        .quad   0x65f492e37d3473f4+        .quad   0x2cb2c5df54515a2b+        .quad   0xeb0a5d8c63fd2aca+        .quad   0xd22bc1662e694eff+        .quad   0x2723f36ef8cbb03a+        .quad   0x70f029ecf0c8131f++        // 2^248 * 7 * G++        .quad   0x461307b32eed3e33+        .quad   0xae042f33a45581e7+        .quad   0xc94449d3195f0366+        .quad   0x0b7d5d8a6c314858+        .quad   0x2a6aafaa5e10b0b9+        .quad   0x78f0a370ef041aa9+        .quad   0x773efb77aa3ad61f+        .quad   0x44eca5a2a74bd9e1+        .quad   0x25d448327b95d543+        .quad   0x70d38300a3340f1d+        .quad   0xde1c531c60e1c52b+        .quad   0x272224512c7de9e4++        // 2^248 * 8 * G++        .quad   0x1abc92af49c5342e+        .quad   0xffeed811b2e6fad0+        .quad   0xefa28c8dfcc84e29+        .quad   0x11b5df18a44cc543+        .quad   0xbf7bbb8a42a975fc+        .quad   0x8c5c397796ada358+        .quad   0xe27fc76fcdedaa48+        .quad   0x19735fd7f6bc20a6+        .quad   0xe3ab90d042c84266+        .quad   0xeb848e0f7f19547e+        .quad   0x2503a1d065a497b9+        .quad   0x0fef911191df895f++        // 2^252 * 1 * G++        .quad   0xb1507ca1ab1c6eb9+        .quad   0xbd448f3e16b687b3+        .quad   0x3455fb7f2c7a91ab+        .quad   0x7579229e2f2adec1+        .quad   0x6ab5dcb85b1c16b7+        .quad   0x94c0fce83c7b27a5+        .quad   0xa4b11c1a735517be+        .quad   0x499238d0ba0eafaa+        .quad   0xecf46e527aba8b57+        .quad   0x15a08c478bd1647b+        .quad   0x7af1c6a65f706fef+        .quad   0x6345fa78f03a30d5++        // 2^252 * 2 * G++        .quad   0xdf02f95f1015e7a1+        .quad   0x790ec41da9b40263+        .quad   0x4d3a0ea133ea1107+        .quad   0x54f70be7e33af8c9+        .quad   0x93d3cbe9bdd8f0a4+        .quad   0xdb152c1bfd177302+        .quad   0x7dbddc6d7f17a875+        .quad   0x3e1a71cc8f426efe+        .quad   0xc83ca3e390babd62+        .quad   0x80ede3670291c833+        .quad   0xc88038ccd37900c4+        .quad   0x2c5fc0231ec31fa1++        // 2^252 * 3 * G++        .quad   0xfeba911717038b4f+        .quad   0xe5123721c9deef81+        .quad   0x1c97e4e75d0d8834+        .quad   0x68afae7a23dc3bc6+        .quad   0xc422e4d102456e65+        .quad   0x87414ac1cad47b91+        .quad   0x1592e2bba2b6ffdd+        .quad   0x75d9d2bff5c2100f+        .quad   0x5bd9b4763626e81c+        .quad   0x89966936bca02edd+        .quad   0x0a41193d61f077b3+        .quad   0x3097a24200ce5471++        // 2^252 * 4 * G++        .quad   0x57427734c7f8b84c+        .quad   0xf141a13e01b270e9+        .quad   0x02d1adfeb4e564a6+        .quad   0x4bb23d92ce83bd48+        .quad   0xa162e7246695c486+        .quad   0x131d633435a89607+        .quad   0x30521561a0d12a37+        .quad   0x56704bada6afb363+        .quad   0xaf6c4aa752f912b9+        .quad   0x5e665f6cd86770c8+        .quad   0x4c35ac83a3c8cd58+        .quad   0x2b7a29c010a58a7e++        // 2^252 * 5 * G++        .quad   0xc4007f77d0c1cec3+        .quad   0x8d1020b6bac492f8+        .quad   0x32ec29d57e69daaf+        .quad   0x599408759d95fce0+        .quad   0x33810a23bf00086e+        .quad   0xafce925ee736ff7c+        .quad   0x3d60e670e24922d4+        .quad   0x11ce9e714f96061b+        .quad   0x219ef713d815bac1+        .quad   0xf141465d485be25c+        .quad   0x6d5447cc4e513c51+        .quad   0x174926be5ef44393++        // 2^252 * 6 * G++        .quad   0xb5deb2f9fc5bd5bb+        .quad   0x92daa72ae1d810e1+        .quad   0xafc4cfdcb72a1c59+        .quad   0x497d78813fc22a24+        .quad   0x3ef5d41593ea022e+        .quad   0x5cbcc1a20ed0eed6+        .quad   0x8fd24ecf07382c8c+        .quad   0x6fa42ead06d8e1ad+        .quad   0xe276824a1f73371f+        .quad   0x7f7cf01c4f5b6736+        .quad   0x7e201fe304fa46e7+        .quad   0x785a36a357808c96++        // 2^252 * 7 * G++        .quad   0x825fbdfd63014d2b+        .quad   0xc852369c6ca7578b+        .quad   0x5b2fcd285c0b5df0+        .quad   0x12ab214c58048c8f+        .quad   0x070442985d517bc3+        .quad   0x6acd56c7ae653678+        .quad   0x00a27983985a7763+        .quad   0x5167effae512662b+        .quad   0xbd4ea9e10f53c4b6+        .quad   0x1673dc5f8ac91a14+        .quad   0xa8f81a4e2acc1aba+        .quad   0x33a92a7924332a25++        // 2^252 * 8 * G++        .quad   0x9dd1f49927996c02+        .quad   0x0cb3b058e04d1752+        .quad   0x1f7e88967fd02c3e+        .quad   0x2f964268cb8b3eb1+        .quad   0x7ba95ba0218f2ada+        .quad   0xcff42287330fb9ca+        .quad   0xdada496d56c6d907+        .quad   0x5380c296f4beee54+        .quad   0x9d4f270466898d0a+        .quad   0x3d0987990aff3f7a+        .quad   0xd09ef36267daba45+        .quad   0x7761455e7b1c669c
+ cbits/s2n/x86_att/curve25519_x25519base_alt.S view
@@ -0,0 +1,9965 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// The x25519 function for curve25519 on base element 9+// Input scalar[4]; output res[4]+//+// extern void curve25519_x25519base_alt+//   (uint64_t res[static 4],const uint64_t scalar[static 4]);+//+// The function has a second prototype considering the arguments as arrays+// of bytes rather than 64-bit words. The underlying code is the same, since+// the x86 platform is little-endian.+//+// extern void curve25519_x25519base_byte_alt+//   (uint8_t res[static 32],const uint8_t scalar[static 32]);+//+// Given a scalar n, returns the X coordinate of n * G where G = (9,...) is+// the standard generator. The scalar is first slightly modified/mangled+// as specified in the relevant RFC (https://www.rfc-editor.org/rfc/rfc7748).+//+// Standard x86-64 ABI: RDI = res, RSI = scalar+// Microsoft x64 ABI:   RCX = res, RDX = scalar+// ----------------------------------------------------------------------------+#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(curve25519_x25519base_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(curve25519_x25519base_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(curve25519_x25519base_alt)+        S2N_BN_SYM_VISIBILITY_DIRECTIVE(curve25519_x25519base_byte_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(curve25519_x25519base_byte_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(curve25519_x25519base_byte_alt)++        .text+        .balign 32++// Size of individual field elements++#define NUMSIZE 32++// Pointer-offset pairs for result and temporaries on stack with some aliasing.+// The result "resx" assumes the "res" pointer has been preloaded into %rbp.++#define resx (0*NUMSIZE)(%rbp)++#define scalar (0*NUMSIZE)(%rsp)++#define tabent (1*NUMSIZE)(%rsp)+#define ymx_2 (1*NUMSIZE)(%rsp)+#define xpy_2 (2*NUMSIZE)(%rsp)+#define kxy_2 (3*NUMSIZE)(%rsp)++#define acc (4*NUMSIZE)(%rsp)+#define x_1 (4*NUMSIZE)(%rsp)+#define y_1 (5*NUMSIZE)(%rsp)+#define z_1 (6*NUMSIZE)(%rsp)+#define w_1 (7*NUMSIZE)(%rsp)+#define x_3 (4*NUMSIZE)(%rsp)+#define y_3 (5*NUMSIZE)(%rsp)+#define z_3 (6*NUMSIZE)(%rsp)+#define w_3 (7*NUMSIZE)(%rsp)++#define tmpspace (8*NUMSIZE)(%rsp)+#define t0 (8*NUMSIZE)(%rsp)+#define t1 (9*NUMSIZE)(%rsp)+#define t2 (10*NUMSIZE)(%rsp)+#define t3 (11*NUMSIZE)(%rsp)+#define t4 (12*NUMSIZE)(%rsp)+#define t5 (13*NUMSIZE)(%rsp)++// Stable homes for the input result pointer, and other variables++#define res  14*NUMSIZE(%rsp)++#define i  14*NUMSIZE+8(%rsp)++#define bias  14*NUMSIZE+16(%rsp)++#define bf  14*NUMSIZE+24(%rsp)+#define ix  14*NUMSIZE+24(%rsp)++#define tab  15*NUMSIZE(%rsp)++// Total size to reserve on the stack++#define NSPACE 488++// Macro wrapping up the basic field multiplication, only trivially+// different from a pure function call to bignum_mul_p25519_alt.++#define mul_p25519(P0,P1,P2)                    \+        movq    P1, %rax ;                      \+        mulq     P2;                 \+        movq    %rax, %r8 ;                         \+        movq    %rdx, %r9 ;                         \+        xorq    %r10, %r10 ;                        \+        xorq    %r11, %r11 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x8+P2;             \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     P2;                 \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        adcq    $0x0, %r11 ;                        \+        xorq    %r12, %r12 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x10+P2;            \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    %r12, %r12 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x8+P2;             \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     P2;                 \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        xorq    %r13, %r13 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x18+P2;            \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    %r13, %r13 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x10+P2;            \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x8+P2;             \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     P2;                 \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        xorq    %r14, %r14 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x18+P2;            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    %r14, %r14 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x10+P2;            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x8+P2;             \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        xorq    %r15, %r15 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x18+P2;            \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    %r15, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x10+P2;            \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    $0x0, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x18+P2;            \+        addq    %rax, %r14 ;                        \+        adcq    %rdx, %r15 ;                        \+        movl    $0x26, %esi ;                       \+        movq    %r12, %rax ;                        \+        mulq    %rsi;                            \+        addq    %rax, %r8 ;                         \+        adcq    %rdx, %r9 ;                         \+        sbbq    %rcx, %rcx ;                        \+        movq    %r13, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r14, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r15, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        xorq    %rcx, %rcx ;                        \+        addq    %rax, %r11 ;                        \+        movq    %rdx, %r12 ;                        \+        adcq    %rcx, %r12 ;                        \+        shldq   $0x1, %r11, %r12 ;                    \+        leaq    0x1(%r12), %rax ;                  \+        movl    $0x13, %esi ;                       \+        bts     $63, %r11 ;                         \+        imulq   %rsi, %rax ;                        \+        addq    %rax, %r8 ;                         \+        adcq    %rcx, %r9 ;                         \+        adcq    %rcx, %r10 ;                        \+        adcq    %rcx, %r11 ;                        \+        sbbq    %rax, %rax ;                        \+        notq    %rax;                            \+        andq    %rsi, %rax ;                        \+        subq    %rax, %r8 ;                         \+        sbbq    %rcx, %r9 ;                         \+        sbbq    %rcx, %r10 ;                        \+        sbbq    %rcx, %r11 ;                        \+        btr     $63, %r11 ;                         \+        movq    %r8, P0 ;                        \+        movq    %r9, 0x8+P0 ;                    \+        movq    %r10, 0x10+P0 ;                  \+        movq    %r11, 0x18+P0++// A version of multiplication that only guarantees output < 2 * p_25519.+// This basically skips the +1 and final correction in quotient estimation.++#define mul_4(P0,P1,P2)                         \+        movq    P1, %rax ;                      \+        mulq     P2;                 \+        movq    %rax, %r8 ;                         \+        movq    %rdx, %r9 ;                         \+        xorq    %r10, %r10 ;                        \+        xorq    %r11, %r11 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x8+P2;             \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     P2;                 \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        adcq    $0x0, %r11 ;                        \+        xorq    %r12, %r12 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x10+P2;            \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    %r12, %r12 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x8+P2;             \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     P2;                 \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        xorq    %r13, %r13 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x18+P2;            \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    %r13, %r13 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x10+P2;            \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x8+P2;             \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     P2;                 \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        xorq    %r14, %r14 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x18+P2;            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    %r14, %r14 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x10+P2;            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x8+P2;             \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        xorq    %r15, %r15 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x18+P2;            \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    %r15, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x10+P2;            \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    $0x0, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x18+P2;            \+        addq    %rax, %r14 ;                        \+        adcq    %rdx, %r15 ;                        \+        movl    $0x26, %ebx ;                       \+        movq    %r12, %rax ;                        \+        mulq    %rbx;                            \+        addq    %rax, %r8 ;                         \+        adcq    %rdx, %r9 ;                         \+        sbbq    %rcx, %rcx ;                        \+        movq    %r13, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r14, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r15, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                        \+        xorq    %rcx, %rcx ;                        \+        addq    %rax, %r11 ;                        \+        movq    %rdx, %r12 ;                        \+        adcq    %rcx, %r12 ;                        \+        shldq   $0x1, %r11, %r12 ;                    \+        btr     $0x3f, %r11 ;                      \+        movl    $0x13, %edx ;                      \+        imulq   %r12, %rdx ;                       \+        addq    %rdx, %r8 ;                        \+        adcq    %rcx, %r9 ;                        \+        adcq    %rcx, %r10 ;                       \+        adcq    %rcx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++// Modular subtraction with double modulus 2 * p_25519 = 2^256 - 38++#define sub_twice4(P0,P1,P2)                    \+        movq    P1, %r8 ;                       \+        xorl    %ebx, %ebx ;                       \+        subq    P2, %r8 ;                       \+        movq    8+P1, %r9 ;                     \+        sbbq    8+P2, %r9 ;                     \+        movl    $38, %ecx ;                        \+        movq    16+P1, %r10 ;                   \+        sbbq    16+P2, %r10 ;                   \+        movq    24+P1, %rax ;                   \+        sbbq    24+P2, %rax ;                   \+        cmovncq %rbx, %rcx ;                       \+        subq    %rcx, %r8 ;                        \+        sbbq    %rbx, %r9 ;                        \+        sbbq    %rbx, %r10 ;                       \+        sbbq    %rbx, %rax ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 8+P0 ;                     \+        movq    %r10, 16+P0 ;                   \+        movq    %rax, 24+P0++// Modular addition and doubling with double modulus 2 * p_25519 = 2^256 - 38.+// This only ensures that the result fits in 4 digits, not that it is reduced+// even w.r.t. double modulus. The result is always correct modulo provided+// the sum of the inputs is < 2^256 + 2^256 - 38, so in particular provided+// at least one of them is reduced double modulo.++#define add_twice4(P0,P1,P2)                    \+        movq    P1, %r8 ;                       \+        xorl    %ecx, %ecx ;                       \+        addq    P2, %r8 ;                       \+        movq    0x8+P1, %r9 ;                   \+        adcq    0x8+P2, %r9 ;                   \+        movq    0x10+P1, %r10 ;                 \+        adcq    0x10+P2, %r10 ;                 \+        movq    0x18+P1, %r11 ;                 \+        adcq    0x18+P2, %r11 ;                 \+        movl    $38, %eax ;                        \+        cmovncq %rcx, %rax ;                       \+        addq    %rax, %r8 ;                        \+        adcq    %rcx, %r9 ;                        \+        adcq    %rcx, %r10 ;                       \+        adcq    %rcx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++#define double_twice4(P0,P1)                    \+        movq    P1, %r8 ;                       \+        xorl    %ecx, %ecx ;                       \+        addq    %r8, %r8 ;                         \+        movq    0x8+P1, %r9 ;                   \+        adcq    %r9, %r9 ;                         \+        movq    0x10+P1, %r10 ;                 \+        adcq    %r10, %r10 ;                       \+        movq    0x18+P1, %r11 ;                 \+        adcq    %r11, %r11 ;                       \+        movl    $38, %eax ;                        \+        cmovncq %rcx, %rax ;                       \+        addq    %rax, %r8 ;                        \+        adcq    %rcx, %r9 ;                        \+        adcq    %rcx, %r10 ;                       \+        adcq    %rcx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++S2N_BN_SYMBOL(curve25519_x25519base_alt):+S2N_BN_SYMBOL(curve25519_x25519base_byte_alt):+        CFI_START+        _CET_ENDBR++// In this case the Windows form literally makes a subroutine call.+// This avoids hassle arising from keeping code and data together.++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        CFI_CALL(Lcurve25519_x25519base_alt_standard)+        CFI_POP(%rsi)+        CFI_POP(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(curve25519_x25519base_alt)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lcurve25519_x25519base_alt_standard)++Lcurve25519_x25519base_alt_standard:+        CFI_START+#endif++// Save registers, make room for temps, preserve input arguments.++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(NSPACE)++// Move the output pointer to a stable place++        movq    %rdi, res++// Copy the input scalar to its local variable while mangling it.+// In principle the mangling is into 01xxx...xxx000, but actually+// we only clear the top two bits so 00xxx...xxxxxx. The additional+// 2^254 * G is taken care of by the starting value for the addition+// chain below, while we never look at the three low bits at all.++        movq    (%rsi), %rax+        movq    %rax, (%rsp)+        movq    8(%rsi), %rax+        movq    %rax, 8(%rsp)+        movq    16(%rsi), %rax+        movq    %rax, 16(%rsp)+        movq    $0x3fffffffffffffff, %rax+        andq    24(%rsi), %rax+        movq    %rax, 24(%rsp)++// The main part of the computation is on the edwards25519 curve in+// extended-projective coordinates (X,Y,Z,T), representing a point+// (x,y) via x = X/Z, y = Y/Z and x * y = T/Z (so X * Y = T * Z).+// Only at the very end do we translate back to curve25519. So G+// below means the generator within edwards25519 corresponding to+// (9,...) for curve25519, via the standard isomorphism.+//+// Initialize accumulator "acc" to either (2^254 + 8) * G or just 2^254 * G+// depending on bit 3 of the scalar, the only nonzero bit of the bottom 4.+// Thus, we have effectively dealt with bits 0, 1, 2, 3, 254 and 255.++        movq    (%rsp), %rax+        andq    $8, %rax++        leaq    S2N_BN_SYMBOL(curve25519_x25519base_alt_constant)(%rip), %r10+        leaq    8*12(%r10), %r11++        movq    (%r10), %rax+        movq    (%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*16(%rsp)++        movq    8*1(%r10), %rax+        movq    8*1(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*17(%rsp)++        movq    8*2(%r10), %rax+        movq    8*2(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*18(%rsp)++        movq    8*3(%r10), %rax+        movq    8*3(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*19(%rsp)++        movq    8*4(%r10), %rax+        movq    8*4(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*20(%rsp)++        movq    8*5(%r10), %rax+        movq    8*5(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*21(%rsp)++        movq    8*6(%r10), %rax+        movq    8*6(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*22(%rsp)++        movq    8*7(%r10), %rax+        movq    8*7(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*23(%rsp)++        movl    $1, %eax+        movq    %rax, 8*24(%rsp)+        movl    $0, %eax+        movq    %rax, 8*25(%rsp)+        movq    %rax, 8*26(%rsp)+        movq    %rax, 8*27(%rsp)++        movq    8*8(%r10), %rax+        movq    8*8(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*28(%rsp)++        movq    8*9(%r10), %rax+        movq    8*9(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*29(%rsp)++        movq    8*10(%r10), %rax+        movq    8*10(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*30(%rsp)++        movq    8*11(%r10), %rax+        movq    8*11(%r11), %rcx+        cmovnzq %rcx, %rax+        movq    %rax, 8*31(%rsp)++// The counter "i" tracks the bit position for which the scalar has+// already been absorbed, starting at 4 and going up in chunks of 4.+//+// The pointer "tab" points at the current block of the table for+// multiples (2^i * j) * G at the current bit position i; 1 <= j <= 8.+//+// The bias is always either 0 and 1 and needs to be added to the+// partially processed scalar implicitly. This is used to absorb 4 bits+// of scalar per iteration from 3-bit table indexing by exploiting+// negation: (16 * h + l) * G = (16 * (h + 1) - (16 - l)) * G is used+// when l >= 9. Note that we can't have any bias left over at the+// end because of the clearing of bit 255 of the scalar, meaning the+// l >= 9 case cannot arise on the last iteration.++        movq    $4, i+        leaq    8*24(%r10), %rax+        movq    %rax, tab+        movq    $0, bias++// Start of the main loop, repeated 63 times for i = 4, 8, ..., 252++Lcurve25519_x25519base_alt_scalarloop:++// Look at the next 4-bit field "bf", adding the previous bias as well.+// Choose the table index "ix" as bf when bf <= 8 and 16 - bf for bf >= 9,+// setting the bias to 1 for the next iteration in the latter case.++        movq    i, %rax+        movq    %rax, %rcx+        shrq    $6, %rax+        movq    (%rsp,%rax,8), %rax // Exploiting scalar = sp exactly+        shrq    %cl, %rax+        andq    $15, %rax+        addq    bias, %rax+        movq    %rax, bf++        cmpq    $9, bf+        sbbq    %rax, %rax+        incq    %rax+        movq    %rax, bias++        movq    $16, %rdi+        subq    bf, %rdi+        cmpq    $0, bias+        cmovzq  bf, %rdi+        movq    %rdi, ix++// Perform constant-time lookup in the table to get element number "ix".+// The table entry for the affine point (x,y) is actually a triple+// (y - x,x + y,2 * d * x * y) to precompute parts of the addition.+// Note that "ix" can be 0, so we set up the appropriate identity first.++        movl    $1, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        movl    $1, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        xorl    %r13d, %r13d+        xorl    %r14d, %r14d+        xorl    %r15d, %r15d++        movq    tab, %rbp++        cmpq    $1, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $2, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $3, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $4, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $5, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $6, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $7, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $8, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15++        addq    $96, %rbp+        movq    %rbp, tab++// We now have the triple from the table in registers as follows+//+//      [%rdx;%rcx;%rbx;%rax] = y - x+//      [%r11;%r10;%r9;%r8] = x + y+//      [%r15;%r14;%r13;%r12] = 2 * d * x * y+//+// In case bias = 1 we need to negate this. For Edwards curves+// -(x,y) = (-x,y), i.e. we need to negate the x coordinate.+// In this processed encoding, that amounts to swapping the+// first two fields and negating the third.+//+// The optional negation here also pretends bias = 0 whenever+// ix = 0 so that it doesn't need to handle the case of zero+// inputs, since no non-trivial table entries are zero. Note+// that in the zero case the whole negation is trivial, and+// so indeed is the swapping.++        cmpq    $0, bias++        movq    %rax, %rsi+        cmovnzq %r8, %rsi+        cmovnzq %rax, %r8+        movq    %rsi, 32(%rsp)+        movq    %r8, 64(%rsp)++        movq    %rbx, %rsi+        cmovnzq %r9, %rsi+        cmovnzq %rbx, %r9+        movq    %rsi, 40(%rsp)+        movq    %r9, 72(%rsp)++        movq    %rcx, %rsi+        cmovnzq %r10, %rsi+        cmovnzq %rcx, %r10+        movq    %rsi, 48(%rsp)+        movq    %r10, 80(%rsp)++        movq    %rdx, %rsi+        cmovnzq %r11, %rsi+        cmovnzq %rdx, %r11+        movq    %rsi, 56(%rsp)+        movq    %r11, 88(%rsp)++        movq    $-19, %rax+        movq    $-1, %rbx+        movq    $-1, %rcx+        movq    $0x7fffffffffffffff, %rdx+        subq    %r12, %rax+        sbbq    %r13, %rbx+        sbbq    %r14, %rcx+        sbbq    %r15, %rdx++        movq    ix, %r8+        movq    bias, %r9+        testq   %r8, %r8+        cmovzq  %r8, %r9+        testq   %r9, %r9++        cmovzq  %r12, %rax+        cmovzq  %r13, %rbx+        cmovzq  %r14, %rcx+        cmovzq  %r15, %rdx+        movq    %rax, 96(%rsp)+        movq    %rbx, 104(%rsp)+        movq    %rcx, 112(%rsp)+        movq    %rdx, 120(%rsp)++// Extended-projective and precomputed mixed addition.+// This is effectively the same as calling the standalone+// function edwards25519_pepadd_alt(acc,acc,tabent), but we+// only retain slightly weaker normalization < 2 * p_25519+// throughout the inner loop, so the computation is+// slightly different, and faster overall.++        double_twice4(t0,z_1)+        sub_twice4(t1,y_1,x_1)+        add_twice4(t2,y_1,x_1)+        mul_4(t3,w_1,kxy_2)+        mul_4(t1,t1,ymx_2)+        mul_4(t2,t2,xpy_2)+        sub_twice4(t4,t0,t3)+        add_twice4(t0,t0,t3)+        sub_twice4(t5,t2,t1)+        add_twice4(t1,t2,t1)+        mul_4(z_3,t4,t0)+        mul_4(x_3,t5,t4)+        mul_4(y_3,t0,t1)+        mul_4(w_3,t5,t1)++// End of the main loop; move on by 4 bits.++        addq    $4, i+        cmpq    $256, i+        jc      Lcurve25519_x25519base_alt_scalarloop++// Now we need to translate from Edwards curve edwards25519 back+// to the Montgomery form curve25519. The mapping in the affine+// representations is+//+// (x,y) |-> ((1 + y) / (1 - y), c * (1 + y) / ((1 - y) * x))+//+// For x25519, we only need the x coordinate, and we compute this as+//+// (1 + y) / (1 - y) = (x + x * y) / (x - x * y)+//                   = (X/Z + T/Z) / (X/Z - T/Z)+//                   = (X + T) / (X - T)+//                   = (X + T) * inverse(X - T)+//+// We could equally well use (Z + Y) / (Z - Y), but the above has the+// same cost, and it more explicitly forces zero output whenever X = 0,+// regardless of how the modular inverse behaves on zero inputs. In+// the present setting (base point 9, mangled scalar) that doesn't+// really matter anyway since X = 0 never arises, but it seems a+// little bit tidier. Note that both Edwards point (0,1) which maps to+// the Montgomery point at infinity, and Edwards (0,-1) which maps to+// Montgomery (0,0) [this is the 2-torsion point] are both by definition+// mapped to 0 by the X coordinate mapping used to define curve25519.+//+// First the addition and subtraction:++        add_twice4(t1,x_3,w_3)+        sub_twice4(t2,x_3,w_3)++// Prepare to call the modular inverse function to get t0 = 1/t2+// Note that this works for the weakly normalized z_3 equally well.+// The non-coprime case z_3 == 0 (mod p_25519) cannot arise anyway.++        leaq    256(%rsp), %rdi+        leaq    320(%rsp), %rsi++// Inline copy of bignum_inv_p25519, identical except for stripping out+// the prologue and epilogue saving and restoring registers and making+// and reclaiming room on the stack. For more details and explanations see+// "x86/curve25519/bignum_inv_p25519.S". Note that the stack it uses for+// its own temporaries is 208 bytes, so it has no effect on variables+// that are needed in the rest of our computation here: res, t0, t1, t2.++        movq    %rdi, 0xc0(%rsp)+        xorl    %eax, %eax+        leaq    -0x13(%rax), %rcx+        notq    %rax+        movq    %rcx, (%rsp)+        movq    %rax, 0x8(%rsp)+        movq    %rax, 0x10(%rsp)+        btr     $0x3f, %rax+        movq    %rax, 0x18(%rsp)+        movq    (%rsi), %rdx+        movq    0x8(%rsi), %rcx+        movq    0x10(%rsi), %r8+        movq    0x18(%rsi), %r9+        movl    $0x1, %eax+        xorl    %r10d, %r10d+        bts     $0x3f, %r9+        adcq    %r10, %rax+        imulq   $0x13, %rax, %rax+        addq    %rax, %rdx+        adcq    %r10, %rcx+        adcq    %r10, %r8+        adcq    %r10, %r9+        movl    $0x13, %eax+        cmovbq  %r10, %rax+        subq    %rax, %rdx+        sbbq    %r10, %rcx+        sbbq    %r10, %r8+        sbbq    %r10, %r9+        btr     $0x3f, %r9+        movq    %rdx, 0x20(%rsp)+        movq    %rcx, 0x28(%rsp)+        movq    %r8, 0x30(%rsp)+        movq    %r9, 0x38(%rsp)+        xorl    %eax, %eax+        movq    %rax, 0x40(%rsp)+        movq    %rax, 0x48(%rsp)+        movq    %rax, 0x50(%rsp)+        movq    %rax, 0x58(%rsp)+        movabsq $0xa0f99e2375022099, %rax+        movq    %rax, 0x60(%rsp)+        movabsq $0xa8c68f3f1d132595, %rax+        movq    %rax, 0x68(%rsp)+        movabsq $0x6c6c893805ac5242, %rax+        movq    %rax, 0x70(%rsp)+        movabsq $0x276508b241770615, %rax+        movq    %rax, 0x78(%rsp)+        movq    $0xa,  0x90(%rsp)+        movq    $0x1,  0x98(%rsp)+        jmp     Lcurve25519_x25519base_alt_midloop+Lcurve25519_x25519base_alt_inverseloop:+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %rdi+        andq    %r11, %rdi+        addq    %rax, %rdi+        movq    %rdi, 0x80(%rsp)+        movq    %r12, %rax+        andq    %r13, %rax+        movq    %r14, %rsi+        andq    %r15, %rsi+        addq    %rax, %rsi+        movq    %rsi, 0x88(%rsp)+        xorl    %ebx, %ebx+        movq    (%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x20(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    (%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x20(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        xorl    %ecx, %ecx+        movq    0x8(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x28(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, (%rsp)+        xorl    %edi, %edi+        movq    0x8(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    0x28(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $0x3b, %rbp, %rsi+        movq    %rsi, 0x20(%rsp)+        xorl    %esi, %esi+        movq    0x10(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    0x30(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rcx, %rbx+        movq    %rbx, 0x8(%rsp)+        xorl    %ebx, %ebx+        movq    0x10(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x30(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $0x3b, %rdi, %rbp+        movq    %rbp, 0x28(%rsp)+        movq    0x18(%rsp), %rax+        xorq    %r9, %rax+        movq    %rax, %rbp+        sarq    $0x3f, %rbp+        andq    %r8, %rbp+        negq    %rbp+        mulq    %r8+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x38(%rsp), %rax+        xorq    %r11, %rax+        movq    %rax, %rdx+        sarq    $0x3f, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbp+        mulq    %r10+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        shrdq   $0x3b, %rsi, %rcx+        movq    %rcx, 0x10(%rsp)+        shrdq   $0x3b, %rbp, %rsi+        movq    0x18(%rsp), %rax+        movq    %rsi, 0x18(%rsp)+        xorq    %r13, %rax+        movq    %rax, %rsi+        sarq    $0x3f, %rsi+        andq    %r12, %rsi+        negq    %rsi+        mulq    %r12+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        movq    0x38(%rsp), %rax+        xorq    %r15, %rax+        movq    %rax, %rdx+        sarq    $0x3f, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rsi+        mulq    %r14+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, 0x30(%rsp)+        shrdq   $0x3b, %rsi, %rbx+        movq    %rbx, 0x38(%rsp)+        movq    0x80(%rsp), %rbx+        movq    0x88(%rsp), %rbp+        xorl    %ecx, %ecx+        movq    0x40(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x60(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x40(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x40(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x60(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x60(%rsp)+        xorl    %ebx, %ebx+        movq    0x48(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x68(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    0x48(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, 0x48(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x68(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, 0x68(%rsp)+        xorl    %ecx, %ecx+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x70(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x50(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x50(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x70(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x70(%rsp)+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        movq    %r9, %rbx+        andq    %r8, %rbx+        negq    %rbx+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbx+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rbx, %rdx+        movq    %rdx, %rbx+        shldq   $0x1, %rcx, %rdx+        sarq    $0x3f, %rbx+        addq    %rbx, %rdx+        movl    $0x13, %eax+        imulq   %rdx+        movq    0x40(%rsp), %r8+        addq    %rax, %r8+        movq    %r8, 0x40(%rsp)+        movq    0x48(%rsp), %r8+        adcq    %rdx, %r8+        movq    %r8, 0x48(%rsp)+        movq    0x50(%rsp), %r8+        adcq    %rbx, %r8+        movq    %r8, 0x50(%rsp)+        adcq    %rbx, %rcx+        shlq    $0x3f, %rax+        addq    %rax, %rcx+        movq    0x58(%rsp), %rax+        movq    %rcx, 0x58(%rsp)+        xorq    %r13, %rax+        movq    %r13, %rcx+        andq    %r12, %rcx+        negq    %rcx+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rcx+        movq    0x78(%rsp), %rax+        xorq    %r15, %rax+        movq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rcx+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rcx, %rdx+        movq    %rdx, %rcx+        shldq   $0x1, %rsi, %rdx+        sarq    $0x3f, %rcx+        movl    $0x13, %eax+        addq    %rcx, %rdx+        imulq   %rdx+        movq    0x60(%rsp), %r8+        addq    %rax, %r8+        movq    %r8, 0x60(%rsp)+        movq    0x68(%rsp), %r8+        adcq    %rdx, %r8+        movq    %r8, 0x68(%rsp)+        movq    0x70(%rsp), %r8+        adcq    %rcx, %r8+        movq    %r8, 0x70(%rsp)+        adcq    %rcx, %rsi+        shlq    $0x3f, %rax+        addq    %rax, %rsi+        movq    %rsi, 0x78(%rsp)+Lcurve25519_x25519base_alt_midloop:+        movq    0x98(%rsp), %rsi+        movq    (%rsp), %rdx+        movq    0x20(%rsp), %rcx+        movq    %rdx, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        xorl    %ebp, %ebp+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %rdx+        leaq    (%rcx,%rax), %rdi+        shlq    $0x16, %rdx+        shlq    $0x16, %rdi+        sarq    $0x2b, %rdx+        sarq    $0x2b, %rdi+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %rbx+        leaq    (%rcx,%rax), %rcx+        sarq    $0x2a, %rbx+        sarq    $0x2a, %rcx+        movq    %rdx, 0xa0(%rsp)+        movq    %rbx, 0xa8(%rsp)+        movq    %rdi, 0xb0(%rsp)+        movq    %rcx, 0xb8(%rsp)+        movq    (%rsp), %r12+        imulq   %r12, %rdi+        imulq   %rdx, %r12+        movq    0x20(%rsp), %r13+        imulq   %r13, %rbx+        imulq   %rcx, %r13+        addq    %rbx, %r12+        addq    %rdi, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r10+        shlq    $0x16, %r8+        shlq    $0x16, %r10+        sarq    $0x2b, %r8+        sarq    $0x2b, %r10+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r15+        leaq    (%rcx,%rax), %r11+        sarq    $0x2a, %r15+        sarq    $0x2a, %r11+        movq    %r13, %rbx+        movq    %r12, %rcx+        imulq   %r8, %r12+        imulq   %r15, %rbx+        addq    %rbx, %r12+        imulq   %r11, %r13+        imulq   %r10, %rcx+        addq    %rcx, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    0xa0(%rsp), %rax+        imulq   %r8, %rax+        movq    0xb0(%rsp), %rdx+        imulq   %r15, %rdx+        imulq   0xa8(%rsp), %r8+        imulq   0xb8(%rsp), %r15+        addq    %r8, %r15+        leaq    (%rax,%rdx), %r9+        movq    0xa0(%rsp), %rax+        imulq   %r10, %rax+        movq    0xb0(%rsp), %rdx+        imulq   %r11, %rdx+        imulq   0xa8(%rsp), %r10+        imulq   0xb8(%rsp), %r11+        addq    %r10, %r11+        leaq    (%rax,%rdx), %r13+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r12+        shlq    $0x15, %r8+        shlq    $0x15, %r12+        sarq    $0x2b, %r8+        sarq    $0x2b, %r12+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r10+        leaq    (%rcx,%rax), %r14+        sarq    $0x2b, %r10+        sarq    $0x2b, %r14+        movq    %r9, %rax+        imulq   %r8, %rax+        movq    %r13, %rdx+        imulq   %r10, %rdx+        imulq   %r15, %r8+        imulq   %r11, %r10+        addq    %r8, %r10+        leaq    (%rax,%rdx), %r8+        movq    %r9, %rax+        imulq   %r12, %rax+        movq    %r13, %rdx+        imulq   %r14, %rdx+        imulq   %r15, %r12+        imulq   %r11, %r14+        addq    %r12, %r14+        leaq    (%rax,%rdx), %r12+        movq    %rsi, 0x98(%rsp)+        decq     0x90(%rsp)+        jne     Lcurve25519_x25519base_alt_inverseloop+        movq    (%rsp), %rax+        movq    0x20(%rsp), %rcx+        imulq   %r8, %rax+        imulq   %r10, %rcx+        addq    %rcx, %rax+        sarq    $0x3f, %rax+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        xorq    %rax, %r9+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        xorq    %rax, %r11+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        xorq    %rax, %r13+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        xorq    %rax, %r15+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %r12+        andq    %r11, %r12+        addq    %rax, %r12+        xorl    %r13d, %r13d+        movq    0x40(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x60(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movq    0x48(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r13+        adcq    %rdx, %r14+        movq    0x68(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x70(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        andq    %r8, %r9+        negq    %r9+        mulq    %r8+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %r9+        mulq    %r10+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    %r9, %rax+        shldq   $0x1, %r15, %rax+        sarq    $0x3f, %r9+        movl    $0x13, %ebx+        leaq    0x1(%rax,%r9,1), %rax+        imulq   %rbx+        xorl    %ebp, %ebp+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r9, %r14+        adcq    %r9, %r15+        shlq    $0x3f, %rax+        addq    %rax, %r15+        cmovns  %rbp, %rbx+        subq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    %rbp, %r14+        sbbq    %rbp, %r15+        btr     $0x3f, %r15+        movq    0xc0(%rsp), %rdi+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)++// The final result is (X + T) / (X - T)+// This is the only operation in the whole computation that+// fully reduces modulo p_25519 since now we want the canonical+// answer as output.++        movq    res, %rbp+        mul_p25519(resx,t1,t0)++// Restore stack and registers++        CFI_INC_RSP(NSPACE)++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++#if WINDOWS_ABI+S2N_BN_SIZE_DIRECTIVE(Lcurve25519_x25519base_alt_standard)+#else+S2N_BN_SIZE_DIRECTIVE(curve25519_x25519base_alt)+#endif++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif++// ****************************************************************************+// The precomputed data (all read-only).+// ****************************************************************************++#if defined(__ELF__)+.section .rodata+.type S2N_BN_SYMBOL(curve25519_x25519base_alt_constant), %object+.size S2N_BN_SYMBOL(curve25519_x25519base_alt_constant), 48576+#elif defined(__APPLE__)+.const_data+#endif++S2N_BN_SYMBOL(curve25519_x25519base_alt_constant):++// 2^254 * G and (2^254 + 8) * G in extended-projective coordinates+// but with z = 1 assumed and hence left out, so they are (X,Y,T) only.++        .quad   0x251037f7cf4e861d+        .quad   0x10ede0fb19fb128f+        .quad   0x96c033b175f5e2c8+        .quad   0x055f070d6c15fb0d++        .quad   0x7c52af2c97473e69+        .quad   0x022f82391bad8378+        .quad   0x9991e1b02adb476f+        .quad   0x511144a03a99b855++        .quad   0x5fafc3b88ff2e4ae+        .quad   0x855e4ff0de1230ff+        .quad   0x72e302a348492870+        .quad   0x1253c19e53dbe1bc++        .quad   0x331d086e0d9abcaa+        .quad   0x1e23c96d311a10c9+        .quad   0x96d0f95e58c13478+        .quad   0x2f72f7384fcfcc59++        .quad   0x39a6cd1cfd7d87c9+        .quad   0x9867a0abd8ae153a+        .quad   0xa49d2a5f35986745+        .quad   0x57012940cdfe82e1++        .quad   0x5046a6532ec5544a+        .quad   0x6d674004739ff6c9+        .quad   0x9bbaa44b234a70e3+        .quad   0x5e6d8901138cf386++// Precomputed table of multiples of generator for edwards25519+// all in precomputed extended-projective (y-x,x+y,2*d*x*y) triples.++        // 2^4 * 1 * G++        .quad   0x7ec851ca553e2df3+        .quad   0xa71284cba64878b3+        .quad   0xe6b5e4193288d1e7+        .quad   0x4cf210ec5a9a8883+        .quad   0x322d04a52d9021f6+        .quad   0xb9c19f3375c6bf9c+        .quad   0x587a3a4342d20b09+        .quad   0x143b1cf8aa64fe61+        .quad   0x9f867c7d968acaab+        .quad   0x5f54258e27092729+        .quad   0xd0a7d34bea180975+        .quad   0x21b546a3374126e1++        // 2^4 * 2 * G++        .quad   0xa94ff858a2888343+        .quad   0xce0ed4565313ed3c+        .quad   0xf55c3dcfb5bf34fa+        .quad   0x0a653ca5c9eab371+        .quad   0x490a7a45d185218f+        .quad   0x9a15377846049335+        .quad   0x0060ea09cc31e1f6+        .quad   0x7e041577f86ee965+        .quad   0x66b2a496ce5b67f3+        .quad   0xff5492d8bd569796+        .quad   0x503cec294a592cd0+        .quad   0x566943650813acb2++        // 2^4 * 3 * G++        .quad   0xb818db0c26620798+        .quad   0x5d5c31d9606e354a+        .quad   0x0982fa4f00a8cdc7+        .quad   0x17e12bcd4653e2d4+        .quad   0x5672f9eb1dabb69d+        .quad   0xba70b535afe853fc+        .quad   0x47ac0f752796d66d+        .quad   0x32a5351794117275+        .quad   0xd3a644a6df648437+        .quad   0x703b6559880fbfdd+        .quad   0xcb852540ad3a1aa5+        .quad   0x0900b3f78e4c6468++        // 2^4 * 4 * G++        .quad   0x0a851b9f679d651b+        .quad   0xe108cb61033342f2+        .quad   0xd601f57fe88b30a3+        .quad   0x371f3acaed2dd714+        .quad   0xed280fbec816ad31+        .quad   0x52d9595bd8e6efe3+        .quad   0x0fe71772f6c623f5+        .quad   0x4314030b051e293c+        .quad   0xd560005efbf0bcad+        .quad   0x8eb70f2ed1870c5e+        .quad   0x201f9033d084e6a0+        .quad   0x4c3a5ae1ce7b6670++        // 2^4 * 5 * G++        .quad   0x4138a434dcb8fa95+        .quad   0x870cf67d6c96840b+        .quad   0xde388574297be82c+        .quad   0x7c814db27262a55a+        .quad   0xbaf875e4c93da0dd+        .quad   0xb93282a771b9294d+        .quad   0x80d63fb7f4c6c460+        .quad   0x6de9c73dea66c181+        .quad   0x478904d5a04df8f2+        .quad   0xfafbae4ab10142d3+        .quad   0xf6c8ac63555d0998+        .quad   0x5aac4a412f90b104++        // 2^4 * 6 * G++        .quad   0xc64f326b3ac92908+        .quad   0x5551b282e663e1e0+        .quad   0x476b35f54a1a4b83+        .quad   0x1b9da3fe189f68c2+        .quad   0x603a0d0abd7f5134+        .quad   0x8089c932e1d3ae46+        .quad   0xdf2591398798bd63+        .quad   0x1c145cd274ba0235+        .quad   0x32e8386475f3d743+        .quad   0x365b8baf6ae5d9ef+        .quad   0x825238b6385b681e+        .quad   0x234929c1167d65e1++        // 2^4 * 7 * G++        .quad   0x984decaba077ade8+        .quad   0x383f77ad19eb389d+        .quad   0xc7ec6b7e2954d794+        .quad   0x59c77b3aeb7c3a7a+        .quad   0x48145cc21d099fcf+        .quad   0x4535c192cc28d7e5+        .quad   0x80e7c1e548247e01+        .quad   0x4a5f28743b2973ee+        .quad   0xd3add725225ccf62+        .quad   0x911a3381b2152c5d+        .quad   0xd8b39fad5b08f87d+        .quad   0x6f05606b4799fe3b++        // 2^4 * 8 * G++        .quad   0x9ffe9e92177ba962+        .quad   0x98aee71d0de5cae1+        .quad   0x3ff4ae942d831044+        .quad   0x714de12e58533ac8+        .quad   0x5b433149f91b6483+        .quad   0xadb5dc655a2cbf62+        .quad   0x87fa8412632827b3+        .quad   0x60895e91ab49f8d8+        .quad   0xe9ecf2ed0cf86c18+        .quad   0xb46d06120735dfd4+        .quad   0xbc9da09804b96be7+        .quad   0x73e2e62fd96dc26b++        // 2^8 * 1 * G++        .quad   0xed5b635449aa515e+        .quad   0xa865c49f0bc6823a+        .quad   0x850c1fe95b42d1c4+        .quad   0x30d76d6f03d315b9+        .quad   0x2eccdd0e632f9c1d+        .quad   0x51d0b69676893115+        .quad   0x52dfb76ba8637a58+        .quad   0x6dd37d49a00eef39+        .quad   0x6c4444172106e4c7+        .quad   0xfb53d680928d7f69+        .quad   0xb4739ea4694d3f26+        .quad   0x10c697112e864bb0++        // 2^8 * 2 * G++        .quad   0x6493c4277dbe5fde+        .quad   0x265d4fad19ad7ea2+        .quad   0x0e00dfc846304590+        .quad   0x25e61cabed66fe09+        .quad   0x0ca62aa08358c805+        .quad   0x6a3d4ae37a204247+        .quad   0x7464d3a63b11eddc+        .quad   0x03bf9baf550806ef+        .quad   0x3f13e128cc586604+        .quad   0x6f5873ecb459747e+        .quad   0xa0b63dedcc1268f5+        .quad   0x566d78634586e22c++        // 2^8 * 3 * G++        .quad   0x1637a49f9cc10834+        .quad   0xbc8e56d5a89bc451+        .quad   0x1cb5ec0f7f7fd2db+        .quad   0x33975bca5ecc35d9+        .quad   0xa1054285c65a2fd0+        .quad   0x6c64112af31667c3+        .quad   0x680ae240731aee58+        .quad   0x14fba5f34793b22a+        .quad   0x3cd746166985f7d4+        .quad   0x593e5e84c9c80057+        .quad   0x2fc3f2b67b61131e+        .quad   0x14829cea83fc526c++        // 2^8 * 4 * G++        .quad   0xff437b8497dd95c2+        .quad   0x6c744e30aa4eb5a7+        .quad   0x9e0c5d613c85e88b+        .quad   0x2fd9c71e5f758173+        .quad   0x21e70b2f4e71ecb8+        .quad   0xe656ddb940a477e3+        .quad   0xbf6556cece1d4f80+        .quad   0x05fc3bc4535d7b7e+        .quad   0x24b8b3ae52afdedd+        .quad   0x3495638ced3b30cf+        .quad   0x33a4bc83a9be8195+        .quad   0x373767475c651f04++        // 2^8 * 5 * G++        .quad   0x2fba99fd40d1add9+        .quad   0xb307166f96f4d027+        .quad   0x4363f05215f03bae+        .quad   0x1fbea56c3b18f999+        .quad   0x634095cb14246590+        .quad   0xef12144016c15535+        .quad   0x9e38140c8910bc60+        .quad   0x6bf5905730907c8c+        .quad   0x0fa778f1e1415b8a+        .quad   0x06409ff7bac3a77e+        .quad   0x6f52d7b89aa29a50+        .quad   0x02521cf67a635a56++        // 2^8 * 6 * G++        .quad   0x513fee0b0a9d5294+        .quad   0x8f98e75c0fdf5a66+        .quad   0xd4618688bfe107ce+        .quad   0x3fa00a7e71382ced+        .quad   0xb1146720772f5ee4+        .quad   0xe8f894b196079ace+        .quad   0x4af8224d00ac824a+        .quad   0x001753d9f7cd6cc4+        .quad   0x3c69232d963ddb34+        .quad   0x1dde87dab4973858+        .quad   0xaad7d1f9a091f285+        .quad   0x12b5fe2fa048edb6++        // 2^8 * 7 * G++        .quad   0x71f0fbc496fce34d+        .quad   0x73b9826badf35bed+        .quad   0xd2047261ff28c561+        .quad   0x749b76f96fb1206f+        .quad   0xdf2b7c26ad6f1e92+        .quad   0x4b66d323504b8913+        .quad   0x8c409dc0751c8bc3+        .quad   0x6f7e93c20796c7b8+        .quad   0x1f5af604aea6ae05+        .quad   0xc12351f1bee49c99+        .quad   0x61a808b5eeff6b66+        .quad   0x0fcec10f01e02151++        // 2^8 * 8 * G++        .quad   0x644d58a649fe1e44+        .quad   0x21fcaea231ad777e+        .quad   0x02441c5a887fd0d2+        .quad   0x4901aa7183c511f3+        .quad   0x3df2d29dc4244e45+        .quad   0x2b020e7493d8de0a+        .quad   0x6cc8067e820c214d+        .quad   0x413779166feab90a+        .quad   0x08b1b7548c1af8f0+        .quad   0xce0f7a7c246299b4+        .quad   0xf760b0f91e06d939+        .quad   0x41bb887b726d1213++        // 2^12 * 1 * G++        .quad   0x9267806c567c49d8+        .quad   0x066d04ccca791e6a+        .quad   0xa69f5645e3cc394b+        .quad   0x5c95b686a0788cd2+        .quad   0x97d980e0aa39f7d2+        .quad   0x35d0384252c6b51c+        .quad   0x7d43f49307cd55aa+        .quad   0x56bd36cfb78ac362+        .quad   0x2ac519c10d14a954+        .quad   0xeaf474b494b5fa90+        .quad   0xe6af8382a9f87a5a+        .quad   0x0dea6db1879be094++        // 2^12 * 2 * G++        .quad   0xaa66bf547344e5ab+        .quad   0xda1258888f1b4309+        .quad   0x5e87d2b3fd564b2f+        .quad   0x5b2c78885483b1dd+        .quad   0x15baeb74d6a8797a+        .quad   0x7ef55cf1fac41732+        .quad   0x29001f5a3c8b05c5+        .quad   0x0ad7cc8752eaccfb+        .quad   0x52151362793408cf+        .quad   0xeb0f170319963d94+        .quad   0xa833b2fa883d9466+        .quad   0x093a7fa775003c78++        // 2^12 * 3 * G++        .quad   0xe5107de63a16d7be+        .quad   0xa377ffdc9af332cf+        .quad   0x70d5bf18440b677f+        .quad   0x6a252b19a4a31403+        .quad   0xb8e9604460a91286+        .quad   0x7f3fd8047778d3de+        .quad   0x67d01e31bf8a5e2d+        .quad   0x7b038a06c27b653e+        .quad   0x9ed919d5d36990f3+        .quad   0x5213aebbdb4eb9f2+        .quad   0xc708ea054cb99135+        .quad   0x58ded57f72260e56++        // 2^12 * 4 * G++        .quad   0x78e79dade9413d77+        .quad   0xf257f9d59729e67d+        .quad   0x59db910ee37aa7e6+        .quad   0x6aa11b5bbb9e039c+        .quad   0xda6d53265b0fd48b+        .quad   0x8960823193bfa988+        .quad   0xd78ac93261d57e28+        .quad   0x79f2942d3a5c8143+        .quad   0x97da2f25b6c88de9+        .quad   0x251ba7eaacf20169+        .quad   0x09b44f87ef4eb4e4+        .quad   0x7d90ab1bbc6a7da5++        // 2^12 * 5 * G++        .quad   0x9acca683a7016bfe+        .quad   0x90505f4df2c50b6d+        .quad   0x6b610d5fcce435aa+        .quad   0x19a10d446198ff96+        .quad   0x1a07a3f496b3c397+        .quad   0x11ceaa188f4e2532+        .quad   0x7d9498d5a7751bf0+        .quad   0x19ed161f508dd8a0+        .quad   0x560a2cd687dce6ca+        .quad   0x7f3568c48664cf4d+        .quad   0x8741e95222803a38+        .quad   0x483bdab1595653fc++        // 2^12 * 6 * G++        .quad   0xfa780f148734fa49+        .quad   0x106f0b70360534e0+        .quad   0x2210776fe3e307bd+        .quad   0x3286c109dde6a0fe+        .quad   0xd6cf4d0ab4da80f6+        .quad   0x82483e45f8307fe0+        .quad   0x05005269ae6f9da4+        .quad   0x1c7052909cf7877a+        .quad   0x32ee7de2874e98d4+        .quad   0x14c362e9b97e0c60+        .quad   0x5781dcde6a60a38a+        .quad   0x217dd5eaaa7aa840++        // 2^12 * 7 * G++        .quad   0x9db7c4d0248e1eb0+        .quad   0xe07697e14d74bf52+        .quad   0x1e6a9b173c562354+        .quad   0x7fa7c21f795a4965+        .quad   0x8bdf1fb9be8c0ec8+        .quad   0x00bae7f8e30a0282+        .quad   0x4963991dad6c4f6c+        .quad   0x07058a6e5df6f60a+        .quad   0xe9eb02c4db31f67f+        .quad   0xed25fd8910bcfb2b+        .quad   0x46c8131f5c5cddb4+        .quad   0x33b21c13a0cb9bce++        // 2^12 * 8 * G++        .quad   0x360692f8087d8e31+        .quad   0xf4dcc637d27163f7+        .quad   0x25a4e62065ea5963+        .quad   0x659bf72e5ac160d9+        .quad   0x9aafb9b05ee38c5b+        .quad   0xbf9d2d4e071a13c7+        .quad   0x8eee6e6de933290a+        .quad   0x1c3bab17ae109717+        .quad   0x1c9ab216c7cab7b0+        .quad   0x7d65d37407bbc3cc+        .quad   0x52744750504a58d5+        .quad   0x09f2606b131a2990++        // 2^16 * 1 * G++        .quad   0x40e87d44744346be+        .quad   0x1d48dad415b52b25+        .quad   0x7c3a8a18a13b603e+        .quad   0x4eb728c12fcdbdf7+        .quad   0x7e234c597c6691ae+        .quad   0x64889d3d0a85b4c8+        .quad   0xdae2c90c354afae7+        .quad   0x0a871e070c6a9e1d+        .quad   0x3301b5994bbc8989+        .quad   0x736bae3a5bdd4260+        .quad   0x0d61ade219d59e3c+        .quad   0x3ee7300f2685d464++        // 2^16 * 2 * G++        .quad   0xf5d255e49e7dd6b7+        .quad   0x8016115c610b1eac+        .quad   0x3c99975d92e187ca+        .quad   0x13815762979125c2+        .quad   0x43fa7947841e7518+        .quad   0xe5c6fa59639c46d7+        .quad   0xa1065e1de3052b74+        .quad   0x7d47c6a2cfb89030+        .quad   0x3fdad0148ef0d6e0+        .quad   0x9d3e749a91546f3c+        .quad   0x71ec621026bb8157+        .quad   0x148cf58d34c9ec80++        // 2^16 * 3 * G++        .quad   0x46a492f67934f027+        .quad   0x469984bef6840aa9+        .quad   0x5ca1bc2a89611854+        .quad   0x3ff2fa1ebd5dbbd4+        .quad   0xe2572f7d9ae4756d+        .quad   0x56c345bb88f3487f+        .quad   0x9fd10b6d6960a88d+        .quad   0x278febad4eaea1b9+        .quad   0xb1aa681f8c933966+        .quad   0x8c21949c20290c98+        .quad   0x39115291219d3c52+        .quad   0x4104dd02fe9c677b++        // 2^16 * 4 * G++        .quad   0x72b2bf5e1124422a+        .quad   0xa1fa0c3398a33ab5+        .quad   0x94cb6101fa52b666+        .quad   0x2c863b00afaf53d5+        .quad   0x81214e06db096ab8+        .quad   0x21a8b6c90ce44f35+        .quad   0x6524c12a409e2af5+        .quad   0x0165b5a48efca481+        .quad   0xf190a474a0846a76+        .quad   0x12eff984cd2f7cc0+        .quad   0x695e290658aa2b8f+        .quad   0x591b67d9bffec8b8++        // 2^16 * 5 * G++        .quad   0x312f0d1c80b49bfa+        .quad   0x5979515eabf3ec8a+        .quad   0x727033c09ef01c88+        .quad   0x3de02ec7ca8f7bcb+        .quad   0x99b9b3719f18b55d+        .quad   0xe465e5faa18c641e+        .quad   0x61081136c29f05ed+        .quad   0x489b4f867030128b+        .quad   0xd232102d3aeb92ef+        .quad   0xe16253b46116a861+        .quad   0x3d7eabe7190baa24+        .quad   0x49f5fbba496cbebf++        // 2^16 * 6 * G++        .quad   0x30949a108a5bcfd4+        .quad   0xdc40dd70bc6473eb+        .quad   0x92c294c1307c0d1c+        .quad   0x5604a86dcbfa6e74+        .quad   0x155d628c1e9c572e+        .quad   0x8a4d86acc5884741+        .quad   0x91a352f6515763eb+        .quad   0x06a1a6c28867515b+        .quad   0x7288d1d47c1764b6+        .quad   0x72541140e0418b51+        .quad   0x9f031a6018acf6d1+        .quad   0x20989e89fe2742c6++        // 2^16 * 7 * G++        .quad   0x499777fd3a2dcc7f+        .quad   0x32857c2ca54fd892+        .quad   0xa279d864d207e3a0+        .quad   0x0403ed1d0ca67e29+        .quad   0x1674278b85eaec2e+        .quad   0x5621dc077acb2bdf+        .quad   0x640a4c1661cbf45a+        .quad   0x730b9950f70595d3+        .quad   0xc94b2d35874ec552+        .quad   0xc5e6c8cf98246f8d+        .quad   0xf7cb46fa16c035ce+        .quad   0x5bd7454308303dcc++        // 2^16 * 8 * G++        .quad   0x7f9ad19528b24cc2+        .quad   0x7f6b54656335c181+        .quad   0x66b8b66e4fc07236+        .quad   0x133a78007380ad83+        .quad   0x85c4932115e7792a+        .quad   0xc64c89a2bdcdddc9+        .quad   0x9d1e3da8ada3d762+        .quad   0x5bb7db123067f82c+        .quad   0x0961f467c6ca62be+        .quad   0x04ec21d6211952ee+        .quad   0x182360779bd54770+        .quad   0x740dca6d58f0e0d2++        // 2^20 * 1 * G++        .quad   0x50b70bf5d3f0af0b+        .quad   0x4feaf48ae32e71f7+        .quad   0x60e84ed3a55bbd34+        .quad   0x00ed489b3f50d1ed+        .quad   0x3906c72aed261ae5+        .quad   0x9ab68fd988e100f7+        .quad   0xf5e9059af3360197+        .quad   0x0e53dc78bf2b6d47+        .quad   0xb90829bf7971877a+        .quad   0x5e4444636d17e631+        .quad   0x4d05c52e18276893+        .quad   0x27632d9a5a4a4af5++        // 2^20 * 2 * G++        .quad   0xd11ff05154b260ce+        .quad   0xd86dc38e72f95270+        .quad   0x601fcd0d267cc138+        .quad   0x2b67916429e90ccd+        .quad   0xa98285d187eaffdb+        .quad   0xa5b4fbbbd8d0a864+        .quad   0xb658f27f022663f7+        .quad   0x3bbc2b22d99ce282+        .quad   0xb917c952583c0a58+        .quad   0x653ff9b80fe4c6f3+        .quad   0x9b0da7d7bcdf3c0c+        .quad   0x43a0eeb6ab54d60e++        // 2^20 * 3 * G++        .quad   0x396966a46d4a5487+        .quad   0xf811a18aac2bb3ba+        .quad   0x66e4685b5628b26b+        .quad   0x70a477029d929b92+        .quad   0x3ac6322357875fe8+        .quad   0xd9d4f4ecf5fbcb8f+        .quad   0x8dee8493382bb620+        .quad   0x50c5eaa14c799fdc+        .quad   0xdd0edc8bd6f2fb3c+        .quad   0x54c63aa79cc7b7a0+        .quad   0xae0b032b2c8d9f1a+        .quad   0x6f9ce107602967fb++        // 2^20 * 4 * G++        .quad   0xad1054b1cde1c22a+        .quad   0xc4a8e90248eb32df+        .quad   0x5f3e7b33accdc0ea+        .quad   0x72364713fc79963e+        .quad   0x139693063520e0b5+        .quad   0x437fcf7c88ea03fe+        .quad   0xf7d4c40bd3c959bc+        .quad   0x699154d1f893ded9+        .quad   0x315d5c75b4b27526+        .quad   0xcccb842d0236daa5+        .quad   0x22f0c8a3345fee8e+        .quad   0x73975a617d39dbed++        // 2^20 * 5 * G++        .quad   0xe4024df96375da10+        .quad   0x78d3251a1830c870+        .quad   0x902b1948658cd91c+        .quad   0x7e18b10b29b7438a+        .quad   0x6f37f392f4433e46+        .quad   0x0e19b9a11f566b18+        .quad   0x220fb78a1fd1d662+        .quad   0x362a4258a381c94d+        .quad   0x9071d9132b6beb2f+        .quad   0x0f26e9ad28418247+        .quad   0xeab91ec9bdec925d+        .quad   0x4be65bc8f48af2de++        // 2^20 * 6 * G++        .quad   0x78487feba36e7028+        .quad   0x5f3f13001dd8ce34+        .quad   0x934fb12d4b30c489+        .quad   0x056c244d397f0a2b+        .quad   0x1d50fba257c26234+        .quad   0x7bd4823adeb0678b+        .quad   0xc2b0dc6ea6538af5+        .quad   0x5665eec6351da73e+        .quad   0xdb3ee00943bfb210+        .quad   0x4972018720800ac2+        .quad   0x26ab5d6173bd8667+        .quad   0x20b209c2ab204938++        // 2^20 * 7 * G++        .quad   0x549e342ac07fb34b+        .quad   0x02d8220821373d93+        .quad   0xbc262d70acd1f567+        .quad   0x7a92c9fdfbcac784+        .quad   0x1fcca94516bd3289+        .quad   0x448d65aa41420428+        .quad   0x59c3b7b216a55d62+        .quad   0x49992cc64e612cd8+        .quad   0x65bd1bea70f801de+        .quad   0x1befb7c0fe49e28a+        .quad   0xa86306cdb1b2ae4a+        .quad   0x3b7ac0cd265c2a09++        // 2^20 * 8 * G++        .quad   0x822bee438c01bcec+        .quad   0x530cb525c0fbc73b+        .quad   0x48519034c1953fe9+        .quad   0x265cc261e09a0f5b+        .quad   0xf0d54e4f22ed39a7+        .quad   0xa2aae91e5608150a+        .quad   0xf421b2e9eddae875+        .quad   0x31bc531d6b7de992+        .quad   0xdf3d134da980f971+        .quad   0x7a4fb8d1221a22a7+        .quad   0x3df7d42035aad6d8+        .quad   0x2a14edcc6a1a125e++        // 2^24 * 1 * G++        .quad   0xdf48ee0752cfce4e+        .quad   0xc3fffaf306ec08b7+        .quad   0x05710b2ab95459c4+        .quad   0x161d25fa963ea38d+        .quad   0x231a8c570478433c+        .quad   0xb7b5270ec281439d+        .quad   0xdbaa99eae3d9079f+        .quad   0x2c03f5256c2b03d9+        .quad   0x790f18757b53a47d+        .quad   0x307b0130cf0c5879+        .quad   0x31903d77257ef7f9+        .quad   0x699468bdbd96bbaf++        // 2^24 * 2 * G++        .quad   0xbd1f2f46f4dafecf+        .quad   0x7cef0114a47fd6f7+        .quad   0xd31ffdda4a47b37f+        .quad   0x525219a473905785+        .quad   0xd8dd3de66aa91948+        .quad   0x485064c22fc0d2cc+        .quad   0x9b48246634fdea2f+        .quad   0x293e1c4e6c4a2e3a+        .quad   0x376e134b925112e1+        .quad   0x703778b5dca15da0+        .quad   0xb04589af461c3111+        .quad   0x5b605c447f032823++        // 2^24 * 3 * G++        .quad   0xb965805920c47c89+        .quad   0xe7f0100c923b8fcc+        .quad   0x0001256502e2ef77+        .quad   0x24a76dcea8aeb3ee+        .quad   0x3be9fec6f0e7f04c+        .quad   0x866a579e75e34962+        .quad   0x5542ef161e1de61a+        .quad   0x2f12fef4cc5abdd5+        .quad   0x0a4522b2dfc0c740+        .quad   0x10d06e7f40c9a407+        .quad   0xc6cf144178cff668+        .quad   0x5e607b2518a43790++        // 2^24 * 4 * G++        .quad   0x58b31d8f6cdf1818+        .quad   0x35cfa74fc36258a2+        .quad   0xe1b3ff4f66e61d6e+        .quad   0x5067acab6ccdd5f7+        .quad   0xa02c431ca596cf14+        .quad   0xe3c42d40aed3e400+        .quad   0xd24526802e0f26db+        .quad   0x201f33139e457068+        .quad   0xfd527f6b08039d51+        .quad   0x18b14964017c0006+        .quad   0xd5220eb02e25a4a8+        .quad   0x397cba8862460375++        // 2^24 * 5 * G++        .quad   0x30c13093f05959b2+        .quad   0xe23aa18de9a97976+        .quad   0x222fd491721d5e26+        .quad   0x2339d320766e6c3a+        .quad   0x7815c3fbc81379e7+        .quad   0xa6619420dde12af1+        .quad   0xffa9c0f885a8fdd5+        .quad   0x771b4022c1e1c252+        .quad   0xd87dd986513a2fa7+        .quad   0xf5ac9b71f9d4cf08+        .quad   0xd06bc31b1ea283b3+        .quad   0x331a189219971a76++        // 2^24 * 6 * G++        .quad   0xf5166f45fb4f80c6+        .quad   0x9c36c7de61c775cf+        .quad   0xe3d4e81b9041d91c+        .quad   0x31167c6b83bdfe21+        .quad   0x26512f3a9d7572af+        .quad   0x5bcbe28868074a9e+        .quad   0x84edc1c11180f7c4+        .quad   0x1ac9619ff649a67b+        .quad   0xf22b3842524b1068+        .quad   0x5068343bee9ce987+        .quad   0xfc9d71844a6250c8+        .quad   0x612436341f08b111++        // 2^24 * 7 * G++        .quad   0xd99d41db874e898d+        .quad   0x09fea5f16c07dc20+        .quad   0x793d2c67d00f9bbc+        .quad   0x46ebe2309e5eff40+        .quad   0x8b6349e31a2d2638+        .quad   0x9ddfb7009bd3fd35+        .quad   0x7f8bf1b8a3a06ba4+        .quad   0x1522aa3178d90445+        .quad   0x2c382f5369614938+        .quad   0xdafe409ab72d6d10+        .quad   0xe8c83391b646f227+        .quad   0x45fe70f50524306c++        // 2^24 * 8 * G++        .quad   0xda4875a6960c0b8c+        .quad   0x5b68d076ef0e2f20+        .quad   0x07fb51cf3d0b8fd4+        .quad   0x428d1623a0e392d4+        .quad   0x62f24920c8951491+        .quad   0x05f007c83f630ca2+        .quad   0x6fbb45d2f5c9d4b8+        .quad   0x16619f6db57a2245+        .quad   0x084f4a4401a308fd+        .quad   0xa82219c376a5caac+        .quad   0xdeb8de4643d1bc7d+        .quad   0x1d81592d60bd38c6++        // 2^28 * 1 * G++        .quad   0xd833d7beec2a4c38+        .quad   0x2c9162830acc20ed+        .quad   0xe93a47aa92df7581+        .quad   0x702d67a3333c4a81+        .quad   0x3a4a369a2f89c8a1+        .quad   0x63137a1d7c8de80d+        .quad   0xbcac008a78eda015+        .quad   0x2cb8b3a5b483b03f+        .quad   0x36e417cbcb1b90a1+        .quad   0x33b3ddaa7f11794e+        .quad   0x3f510808885bc607+        .quad   0x24141dc0e6a8020d++        // 2^28 * 2 * G++        .quad   0x59f73c773fefee9d+        .quad   0xb3f1ef89c1cf989d+        .quad   0xe35dfb42e02e545f+        .quad   0x5766120b47a1b47c+        .quad   0x91925dccbd83157d+        .quad   0x3ca1205322cc8094+        .quad   0x28e57f183f90d6e4+        .quad   0x1a4714cede2e767b+        .quad   0xdb20ba0fb8b6b7ff+        .quad   0xb732c3b677511fa1+        .quad   0xa92b51c099f02d89+        .quad   0x4f3875ad489ca5f1++        // 2^28 * 3 * G++        .quad   0xc7fc762f4932ab22+        .quad   0x7ac0edf72f4c3c1b+        .quad   0x5f6b55aa9aa895e8+        .quad   0x3680274dad0a0081+        .quad   0x79ed13f6ee73eec0+        .quad   0xa5c6526d69110bb1+        .quad   0xe48928c38603860c+        .quad   0x722a1446fd7059f5+        .quad   0xd0959fe9a8cf8819+        .quad   0xd0a995508475a99c+        .quad   0x6eac173320b09cc5+        .quad   0x628ecf04331b1095++        // 2^28 * 4 * G++        .quad   0x98bcb118a9d0ddbc+        .quad   0xee449e3408b4802b+        .quad   0x87089226b8a6b104+        .quad   0x685f349a45c7915d+        .quad   0x9b41acf85c74ccf1+        .quad   0xb673318108265251+        .quad   0x99c92aed11adb147+        .quad   0x7a47d70d34ecb40f+        .quad   0x60a0c4cbcc43a4f5+        .quad   0x775c66ca3677bea9+        .quad   0xa17aa1752ff8f5ed+        .quad   0x11ded9020e01fdc0++        // 2^28 * 5 * G++        .quad   0x890e7809caefe704+        .quad   0x8728296de30e8c6c+        .quad   0x4c5cd2a392aeb1c9+        .quad   0x194263d15771531f+        .quad   0x471f95b03bea93b7+        .quad   0x0552d7d43313abd3+        .quad   0xbd9370e2e17e3f7b+        .quad   0x7b120f1db20e5bec+        .quad   0x17d2fb3d86502d7a+        .quad   0xb564d84450a69352+        .quad   0x7da962c8a60ed75d+        .quad   0x00d0f85b318736aa++        // 2^28 * 6 * G++        .quad   0x978b142e777c84fd+        .quad   0xf402644705a8c062+        .quad   0xa67ad51be7e612c7+        .quad   0x2f7b459698dd6a33+        .quad   0xa6753c1efd7621c1+        .quad   0x69c0b4a7445671f5+        .quad   0x971f527405b23c11+        .quad   0x387bc74851a8c7cd+        .quad   0x81894b4d4a52a9a8+        .quad   0xadd93e12f6b8832f+        .quad   0x184d8548b61bd638+        .quad   0x3f1c62dbd6c9f6cd++        // 2^28 * 7 * G++        .quad   0x2e8f1f0091910c1f+        .quad   0xa4df4fe0bff2e12c+        .quad   0x60c6560aee927438+        .quad   0x6338283facefc8fa+        .quad   0x3fad3e40148f693d+        .quad   0x052656e194eb9a72+        .quad   0x2f4dcbfd184f4e2f+        .quad   0x406f8db1c482e18b+        .quad   0x9e630d2c7f191ee4+        .quad   0x4fbf8301bc3ff670+        .quad   0x787d8e4e7afb73c4+        .quad   0x50d83d5be8f58fa5++        // 2^28 * 8 * G++        .quad   0x85683916c11a1897+        .quad   0x2d69a4efe506d008+        .quad   0x39af1378f664bd01+        .quad   0x65942131361517c6+        .quad   0xc0accf90b4d3b66d+        .quad   0xa7059de561732e60+        .quad   0x033d1f7870c6b0ba+        .quad   0x584161cd26d946e4+        .quad   0xbbf2b1a072d27ca2+        .quad   0xbf393c59fbdec704+        .quad   0xe98dbbcee262b81e+        .quad   0x02eebd0b3029b589++        // 2^32 * 1 * G++        .quad   0x61368756a60dac5f+        .quad   0x17e02f6aebabdc57+        .quad   0x7f193f2d4cce0f7d+        .quad   0x20234a7789ecdcf0+        .quad   0x8765b69f7b85c5e8+        .quad   0x6ff0678bd168bab2+        .quad   0x3a70e77c1d330f9b+        .quad   0x3a5f6d51b0af8e7c+        .quad   0x76d20db67178b252+        .quad   0x071c34f9d51ed160+        .quad   0xf62a4a20b3e41170+        .quad   0x7cd682353cffe366++        // 2^32 * 2 * G++        .quad   0x0be1a45bd887fab6+        .quad   0x2a846a32ba403b6e+        .quad   0xd9921012e96e6000+        .quad   0x2838c8863bdc0943+        .quad   0xa665cd6068acf4f3+        .quad   0x42d92d183cd7e3d3+        .quad   0x5759389d336025d9+        .quad   0x3ef0253b2b2cd8ff+        .quad   0xd16bb0cf4a465030+        .quad   0xfa496b4115c577ab+        .quad   0x82cfae8af4ab419d+        .quad   0x21dcb8a606a82812++        // 2^32 * 3 * G++        .quad   0x5c6004468c9d9fc8+        .quad   0x2540096ed42aa3cb+        .quad   0x125b4d4c12ee2f9c+        .quad   0x0bc3d08194a31dab+        .quad   0x9a8d00fabe7731ba+        .quad   0x8203607e629e1889+        .quad   0xb2cc023743f3d97f+        .quad   0x5d840dbf6c6f678b+        .quad   0x706e380d309fe18b+        .quad   0x6eb02da6b9e165c7+        .quad   0x57bbba997dae20ab+        .quad   0x3a4276232ac196dd++        // 2^32 * 4 * G++        .quad   0x4b42432c8a7084fa+        .quad   0x898a19e3dfb9e545+        .quad   0xbe9f00219c58e45d+        .quad   0x1ff177cea16debd1+        .quad   0x3bf8c172db447ecb+        .quad   0x5fcfc41fc6282dbd+        .quad   0x80acffc075aa15fe+        .quad   0x0770c9e824e1a9f9+        .quad   0xcf61d99a45b5b5fd+        .quad   0x860984e91b3a7924+        .quad   0xe7300919303e3e89+        .quad   0x39f264fd41500b1e++        // 2^32 * 5 * G++        .quad   0xa7ad3417dbe7e29c+        .quad   0xbd94376a2b9c139c+        .quad   0xa0e91b8e93597ba9+        .quad   0x1712d73468889840+        .quad   0xd19b4aabfe097be1+        .quad   0xa46dfce1dfe01929+        .quad   0xc3c908942ca6f1ff+        .quad   0x65c621272c35f14e+        .quad   0xe72b89f8ce3193dd+        .quad   0x4d103356a125c0bb+        .quad   0x0419a93d2e1cfe83+        .quad   0x22f9800ab19ce272++        // 2^32 * 6 * G++        .quad   0x605a368a3e9ef8cb+        .quad   0xe3e9c022a5504715+        .quad   0x553d48b05f24248f+        .quad   0x13f416cd647626e5+        .quad   0x42029fdd9a6efdac+        .quad   0xb912cebe34a54941+        .quad   0x640f64b987bdf37b+        .quad   0x4171a4d38598cab4+        .quad   0xfa2758aa99c94c8c+        .quad   0x23006f6fb000b807+        .quad   0xfbd291ddadda5392+        .quad   0x508214fa574bd1ab++        // 2^32 * 7 * G++        .quad   0xc20269153ed6fe4b+        .quad   0xa65a6739511d77c4+        .quad   0xcbde26462c14af94+        .quad   0x22f960ec6faba74b+        .quad   0x461a15bb53d003d6+        .quad   0xb2102888bcf3c965+        .quad   0x27c576756c683a5a+        .quad   0x3a7758a4c86cb447+        .quad   0x548111f693ae5076+        .quad   0x1dae21df1dfd54a6+        .quad   0x12248c90f3115e65+        .quad   0x5d9fd15f8de7f494++        // 2^32 * 8 * G++        .quad   0x031408d36d63727f+        .quad   0x6a379aefd7c7b533+        .quad   0xa9e18fc5ccaee24b+        .quad   0x332f35914f8fbed3+        .quad   0x3f244d2aeed7521e+        .quad   0x8e3a9028432e9615+        .quad   0xe164ba772e9c16d4+        .quad   0x3bc187fa47eb98d8+        .quad   0x6d470115ea86c20c+        .quad   0x998ab7cb6c46d125+        .quad   0xd77832b53a660188+        .quad   0x450d81ce906fba03++        // 2^36 * 1 * G++        .quad   0xf8ae4d2ad8453902+        .quad   0x7018058ee8db2d1d+        .quad   0xaab3995fc7d2c11e+        .quad   0x53b16d2324ccca79+        .quad   0x23264d66b2cae0b5+        .quad   0x7dbaed33ebca6576+        .quad   0x030ebed6f0d24ac8+        .quad   0x2a887f78f7635510+        .quad   0x2a23b9e75c012d4f+        .quad   0x0c974651cae1f2ea+        .quad   0x2fb63273675d70ca+        .quad   0x0ba7250b864403f5++        // 2^36 * 2 * G++        .quad   0xbb0d18fd029c6421+        .quad   0xbc2d142189298f02+        .quad   0x8347f8e68b250e96+        .quad   0x7b9f2fe8032d71c9+        .quad   0xdd63589386f86d9c+        .quad   0x61699176e13a85a4+        .quad   0x2e5111954eaa7d57+        .quad   0x32c21b57fb60bdfb+        .quad   0xd87823cd319e0780+        .quad   0xefc4cfc1897775c5+        .quad   0x4854fb129a0ab3f7+        .quad   0x12c49d417238c371++        // 2^36 * 3 * G++        .quad   0x0950b533ffe83769+        .quad   0x21861c1d8e1d6bd1+        .quad   0xf022d8381302e510+        .quad   0x2509200c6391cab4+        .quad   0x09b3a01783799542+        .quad   0x626dd08faad5ee3f+        .quad   0xba00bceeeb70149f+        .quad   0x1421b246a0a444c9+        .quad   0x4aa43a8e8c24a7c7+        .quad   0x04c1f540d8f05ef5+        .quad   0xadba5e0c0b3eb9dc+        .quad   0x2ab5504448a49ce3++        // 2^36 * 4 * G++        .quad   0x2ed227266f0f5dec+        .quad   0x9824ee415ed50824+        .quad   0x807bec7c9468d415+        .quad   0x7093bae1b521e23f+        .quad   0xdc07ac631c5d3afa+        .quad   0x58615171f9df8c6c+        .quad   0x72a079d89d73e2b0+        .quad   0x7301f4ceb4eae15d+        .quad   0x6409e759d6722c41+        .quad   0xa674e1cf72bf729b+        .quad   0xbc0a24eb3c21e569+        .quad   0x390167d24ebacb23++        // 2^36 * 5 * G++        .quad   0x27f58e3bba353f1c+        .quad   0x4c47764dbf6a4361+        .quad   0xafbbc4e56e562650+        .quad   0x07db2ee6aae1a45d+        .quad   0xd7bb054ba2f2120b+        .quad   0xe2b9ceaeb10589b7+        .quad   0x3fe8bac8f3c0edbe+        .quad   0x4cbd40767112cb69+        .quad   0x0b603cc029c58176+        .quad   0x5988e3825cb15d61+        .quad   0x2bb61413dcf0ad8d+        .quad   0x7b8eec6c74183287++        // 2^36 * 6 * G++        .quad   0xe4ca40782cd27cb0+        .quad   0xdaf9c323fbe967bd+        .quad   0xb29bd34a8ad41e9e+        .quad   0x72810497626ede4d+        .quad   0x32fee570fc386b73+        .quad   0xda8b0141da3a8cc7+        .quad   0x975ffd0ac8968359+        .quad   0x6ee809a1b132a855+        .quad   0x9444bb31fcfd863a+        .quad   0x2fe3690a3e4e48c5+        .quad   0xdc29c867d088fa25+        .quad   0x13bd1e38d173292e++        // 2^36 * 7 * G++        .quad   0xd32b4cd8696149b5+        .quad   0xe55937d781d8aab7+        .quad   0x0bcb2127ae122b94+        .quad   0x41e86fcfb14099b0+        .quad   0x223fb5cf1dfac521+        .quad   0x325c25316f554450+        .quad   0x030b98d7659177ac+        .quad   0x1ed018b64f88a4bd+        .quad   0x3630dfa1b802a6b0+        .quad   0x880f874742ad3bd5+        .quad   0x0af90d6ceec5a4d4+        .quad   0x746a247a37cdc5d9++        // 2^36 * 8 * G++        .quad   0xd531b8bd2b7b9af6+        .quad   0x5005093537fc5b51+        .quad   0x232fcf25c593546d+        .quad   0x20a365142bb40f49+        .quad   0x6eccd85278d941ed+        .quad   0x2254ae83d22f7843+        .quad   0xc522d02e7bbfcdb7+        .quad   0x681e3351bff0e4e2+        .quad   0x8b64b59d83034f45+        .quad   0x2f8b71f21fa20efb+        .quad   0x69249495ba6550e4+        .quad   0x539ef98e45d5472b++        // 2^40 * 1 * G++        .quad   0x6e7bb6a1a6205275+        .quad   0xaa4f21d7413c8e83+        .quad   0x6f56d155e88f5cb2+        .quad   0x2de25d4ba6345be1+        .quad   0xd074d8961cae743f+        .quad   0xf86d18f5ee1c63ed+        .quad   0x97bdc55be7f4ed29+        .quad   0x4cbad279663ab108+        .quad   0x80d19024a0d71fcd+        .quad   0xc525c20afb288af8+        .quad   0xb1a3974b5f3a6419+        .quad   0x7d7fbcefe2007233++        // 2^40 * 2 * G++        .quad   0xfaef1e6a266b2801+        .quad   0x866c68c4d5739f16+        .quad   0xf68a2fbc1b03762c+        .quad   0x5975435e87b75a8d+        .quad   0xcd7c5dc5f3c29094+        .quad   0xc781a29a2a9105ab+        .quad   0x80c61d36421c3058+        .quad   0x4f9cd196dcd8d4d7+        .quad   0x199297d86a7b3768+        .quad   0xd0d058241ad17a63+        .quad   0xba029cad5c1c0c17+        .quad   0x7ccdd084387a0307++        // 2^40 * 3 * G++        .quad   0xdca6422c6d260417+        .quad   0xae153d50948240bd+        .quad   0xa9c0c1b4fb68c677+        .quad   0x428bd0ed61d0cf53+        .quad   0x9b0c84186760cc93+        .quad   0xcdae007a1ab32a99+        .quad   0xa88dec86620bda18+        .quad   0x3593ca848190ca44+        .quad   0x9213189a5e849aa7+        .quad   0xd4d8c33565d8facd+        .quad   0x8c52545b53fdbbd1+        .quad   0x27398308da2d63e6++        // 2^40 * 4 * G++        .quad   0x42c38d28435ed413+        .quad   0xbd50f3603278ccc9+        .quad   0xbb07ab1a79da03ef+        .quad   0x269597aebe8c3355+        .quad   0xb9a10e4c0a702453+        .quad   0x0fa25866d57d1bde+        .quad   0xffb9d9b5cd27daf7+        .quad   0x572c2945492c33fd+        .quad   0xc77fc745d6cd30be+        .quad   0xe4dfe8d3e3baaefb+        .quad   0xa22c8830aa5dda0c+        .quad   0x7f985498c05bca80++        // 2^40 * 5 * G++        .quad   0x3849ce889f0be117+        .quad   0x8005ad1b7b54a288+        .quad   0x3da3c39f23fc921c+        .quad   0x76c2ec470a31f304+        .quad   0xd35615520fbf6363+        .quad   0x08045a45cf4dfba6+        .quad   0xeec24fbc873fa0c2+        .quad   0x30f2653cd69b12e7+        .quad   0x8a08c938aac10c85+        .quad   0x46179b60db276bcb+        .quad   0xa920c01e0e6fac70+        .quad   0x2f1273f1596473da++        // 2^40 * 6 * G++        .quad   0x4739fc7c8ae01e11+        .quad   0xfd5274904a6aab9f+        .quad   0x41d98a8287728f2e+        .quad   0x5d9e572ad85b69f2+        .quad   0x30488bd755a70bc0+        .quad   0x06d6b5a4f1d442e7+        .quad   0xead1a69ebc596162+        .quad   0x38ac1997edc5f784+        .quad   0x0666b517a751b13b+        .quad   0x747d06867e9b858c+        .quad   0xacacc011454dde49+        .quad   0x22dfcd9cbfe9e69c++        // 2^40 * 7 * G++        .quad   0x8ddbd2e0c30d0cd9+        .quad   0xad8e665facbb4333+        .quad   0x8f6b258c322a961f+        .quad   0x6b2916c05448c1c7+        .quad   0x56ec59b4103be0a1+        .quad   0x2ee3baecd259f969+        .quad   0x797cb29413f5cd32+        .quad   0x0fe9877824cde472+        .quad   0x7edb34d10aba913b+        .quad   0x4ea3cd822e6dac0e+        .quad   0x66083dff6578f815+        .quad   0x4c303f307ff00a17++        // 2^40 * 8 * G++        .quad   0xd30a3bd617b28c85+        .quad   0xc5d377b739773bea+        .quad   0xc6c6e78c1e6a5cbf+        .quad   0x0d61b8f78b2ab7c4+        .quad   0x29fc03580dd94500+        .quad   0xecd27aa46fbbec93+        .quad   0x130a155fc2e2a7f8+        .quad   0x416b151ab706a1d5+        .quad   0x56a8d7efe9c136b0+        .quad   0xbd07e5cd58e44b20+        .quad   0xafe62fda1b57e0ab+        .quad   0x191a2af74277e8d2++        // 2^44 * 1 * G++        .quad   0xd550095bab6f4985+        .quad   0x04f4cd5b4fbfaf1a+        .quad   0x9d8e2ed12a0c7540+        .quad   0x2bc24e04b2212286+        .quad   0x09d4b60b2fe09a14+        .quad   0xc384f0afdbb1747e+        .quad   0x58e2ea8978b5fd6e+        .quad   0x519ef577b5e09b0a+        .quad   0x1863d7d91124cca9+        .quad   0x7ac08145b88a708e+        .quad   0x2bcd7309857031f5+        .quad   0x62337a6e8ab8fae5++        // 2^44 * 2 * G++        .quad   0x4bcef17f06ffca16+        .quad   0xde06e1db692ae16a+        .quad   0x0753702d614f42b0+        .quad   0x5f6041b45b9212d0+        .quad   0xd1ab324e1b3a1273+        .quad   0x18947cf181055340+        .quad   0x3b5d9567a98c196e+        .quad   0x7fa00425802e1e68+        .quad   0x7d531574028c2705+        .quad   0x80317d69db0d75fe+        .quad   0x30fface8ef8c8ddd+        .quad   0x7e9de97bb6c3e998++        // 2^44 * 3 * G++        .quad   0x1558967b9e6585a3+        .quad   0x97c99ce098e98b92+        .quad   0x10af149b6eb3adad+        .quad   0x42181fe8f4d38cfa+        .quad   0xf004be62a24d40dd+        .quad   0xba0659910452d41f+        .quad   0x81c45ee162a44234+        .quad   0x4cb829d8a22266ef+        .quad   0x1dbcaa8407b86681+        .quad   0x081f001e8b26753b+        .quad   0x3cd7ce6a84048e81+        .quad   0x78af11633f25f22c++        // 2^44 * 4 * G++        .quad   0x8416ebd40b50babc+        .quad   0x1508722628208bee+        .quad   0xa3148fafb9c1c36d+        .quad   0x0d07daacd32d7d5d+        .quad   0x3241c00e7d65318c+        .quad   0xe6bee5dcd0e86de7+        .quad   0x118b2dc2fbc08c26+        .quad   0x680d04a7fc603dc3+        .quad   0xf9c2414a695aa3eb+        .quad   0xdaa42c4c05a68f21+        .quad   0x7c6c23987f93963e+        .quad   0x210e8cd30c3954e3++        // 2^44 * 5 * G++        .quad   0xac4201f210a71c06+        .quad   0x6a65e0aef3bfb021+        .quad   0xbc42c35c393632f7+        .quad   0x56ea8db1865f0742+        .quad   0x2b50f16137fe6c26+        .quad   0xe102bcd856e404d8+        .quad   0x12b0f1414c561f6b+        .quad   0x51b17bc8d028ec91+        .quad   0xfff5fb4bcf535119+        .quad   0xf4989d79df1108a0+        .quad   0xbdfcea659a3ba325+        .quad   0x18a11f1174d1a6f2++        // 2^44 * 6 * G++        .quad   0x407375ab3f6bba29+        .quad   0x9ec3b6d8991e482e+        .quad   0x99c80e82e55f92e9+        .quad   0x307c13b6fb0c0ae1+        .quad   0xfbd63cdad27a5f2c+        .quad   0xf00fc4bc8aa106d7+        .quad   0x53fb5c1a8e64a430+        .quad   0x04eaabe50c1a2e85+        .quad   0x24751021cb8ab5e7+        .quad   0xfc2344495c5010eb+        .quad   0x5f1e717b4e5610a1+        .quad   0x44da5f18c2710cd5++        // 2^44 * 7 * G++        .quad   0x033cc55ff1b82eb5+        .quad   0xb15ae36d411cae52+        .quad   0xba40b6198ffbacd3+        .quad   0x768edce1532e861f+        .quad   0x9156fe6b89d8eacc+        .quad   0xe6b79451e23126a1+        .quad   0xbd7463d93944eb4e+        .quad   0x726373f6767203ae+        .quad   0xe305ca72eb7ef68a+        .quad   0x662cf31f70eadb23+        .quad   0x18f026fdb4c45b68+        .quad   0x513b5384b5d2ecbd++        // 2^44 * 8 * G++        .quad   0x46d46280c729989e+        .quad   0x4b93fbd05368a5dd+        .quad   0x63df3f81d1765a89+        .quad   0x34cebd64b9a0a223+        .quad   0x5e2702878af34ceb+        .quad   0x900b0409b946d6ae+        .quad   0x6512ebf7dabd8512+        .quad   0x61d9b76988258f81+        .quad   0xa6c5a71349b7d94b+        .quad   0xa3f3d15823eb9446+        .quad   0x0416fbd277484834+        .quad   0x69d45e6f2c70812f++        // 2^48 * 1 * G++        .quad   0xce16f74bc53c1431+        .quad   0x2b9725ce2072edde+        .quad   0xb8b9c36fb5b23ee7+        .quad   0x7e2e0e450b5cc908+        .quad   0x9fe62b434f460efb+        .quad   0xded303d4a63607d6+        .quad   0xf052210eb7a0da24+        .quad   0x237e7dbe00545b93+        .quad   0x013575ed6701b430+        .quad   0x231094e69f0bfd10+        .quad   0x75320f1583e47f22+        .quad   0x71afa699b11155e3++        // 2^48 * 2 * G++        .quad   0x65ce6f9b3953b61d+        .quad   0xc65839eaafa141e6+        .quad   0x0f435ffda9f759fe+        .quad   0x021142e9c2b1c28e+        .quad   0xea423c1c473b50d6+        .quad   0x51e87a1f3b38ef10+        .quad   0x9b84bf5fb2c9be95+        .quad   0x00731fbc78f89a1c+        .quad   0xe430c71848f81880+        .quad   0xbf960c225ecec119+        .quad   0xb6dae0836bba15e3+        .quad   0x4c4d6f3347e15808++        // 2^48 * 3 * G++        .quad   0x18f7eccfc17d1fc9+        .quad   0x6c75f5a651403c14+        .quad   0xdbde712bf7ee0cdf+        .quad   0x193fddaaa7e47a22+        .quad   0x2f0cddfc988f1970+        .quad   0x6b916227b0b9f51b+        .quad   0x6ec7b6c4779176be+        .quad   0x38bf9500a88f9fa8+        .quad   0x1fd2c93c37e8876f+        .quad   0xa2f61e5a18d1462c+        .quad   0x5080f58239241276+        .quad   0x6a6fb99ebf0d4969++        // 2^48 * 4 * G++        .quad   0x6a46c1bb560855eb+        .quad   0x2416bb38f893f09d+        .quad   0xd71d11378f71acc1+        .quad   0x75f76914a31896ea+        .quad   0xeeb122b5b6e423c6+        .quad   0x939d7010f286ff8e+        .quad   0x90a92a831dcf5d8c+        .quad   0x136fda9f42c5eb10+        .quad   0xf94cdfb1a305bdd1+        .quad   0x0f364b9d9ff82c08+        .quad   0x2a87d8a5c3bb588a+        .quad   0x022183510be8dcba++        // 2^48 * 5 * G++        .quad   0x4af766385ead2d14+        .quad   0xa08ed880ca7c5830+        .quad   0x0d13a6e610211e3d+        .quad   0x6a071ce17b806c03+        .quad   0x9d5a710143307a7f+        .quad   0xb063de9ec47da45f+        .quad   0x22bbfe52be927ad3+        .quad   0x1387c441fd40426c+        .quad   0xb5d3c3d187978af8+        .quad   0x722b5a3d7f0e4413+        .quad   0x0d7b4848bb477ca0+        .quad   0x3171b26aaf1edc92++        // 2^48 * 6 * G++        .quad   0xa92f319097564ca8+        .quad   0xff7bb84c2275e119+        .quad   0x4f55fe37a4875150+        .quad   0x221fd4873cf0835a+        .quad   0xa60db7d8b28a47d1+        .quad   0xa6bf14d61770a4f1+        .quad   0xd4a1f89353ddbd58+        .quad   0x6c514a63344243e9+        .quad   0x2322204f3a156341+        .quad   0xfb73e0e9ba0a032d+        .quad   0xfce0dd4c410f030e+        .quad   0x48daa596fb924aaa++        // 2^48 * 7 * G++        .quad   0x6eca8e665ca59cc7+        .quad   0xa847254b2e38aca0+        .quad   0x31afc708d21e17ce+        .quad   0x676dd6fccad84af7+        .quad   0x14f61d5dc84c9793+        .quad   0x9941f9e3ef418206+        .quad   0xcdf5b88f346277ac+        .quad   0x58c837fa0e8a79a9+        .quad   0x0cf9688596fc9058+        .quad   0x1ddcbbf37b56a01b+        .quad   0xdcc2e77d4935d66a+        .quad   0x1c4f73f2c6a57f0a++        // 2^48 * 8 * G++        .quad   0x0e7a4fbd305fa0bb+        .quad   0x829d4ce054c663ad+        .quad   0xf421c3832fe33848+        .quad   0x795ac80d1bf64c42+        .quad   0xb36e706efc7c3484+        .quad   0x73dfc9b4c3c1cf61+        .quad   0xeb1d79c9781cc7e5+        .quad   0x70459adb7daf675c+        .quad   0x1b91db4991b42bb3+        .quad   0x572696234b02dcca+        .quad   0x9fdf9ee51f8c78dc+        .quad   0x5fe162848ce21fd3++        // 2^52 * 1 * G++        .quad   0xe2790aae4d077c41+        .quad   0x8b938270db7469a3+        .quad   0x6eb632dc8abd16a2+        .quad   0x720814ecaa064b72+        .quad   0x315c29c795115389+        .quad   0xd7e0e507862f74ce+        .quad   0x0c4a762185927432+        .quad   0x72de6c984a25a1e4+        .quad   0xae9ab553bf6aa310+        .quad   0x050a50a9806d6e1b+        .quad   0x92bb7403adff5139+        .quad   0x0394d27645be618b++        // 2^52 * 2 * G++        .quad   0x4d572251857eedf4+        .quad   0xe3724edde19e93c5+        .quad   0x8a71420e0b797035+        .quad   0x3b3c833687abe743+        .quad   0xf5396425b23545a4+        .quad   0x15a7a27e98fbb296+        .quad   0xab6c52bc636fdd86+        .quad   0x79d995a8419334ee+        .quad   0xcd8a8ea61195dd75+        .quad   0xa504d8a81dd9a82f+        .quad   0x540dca81a35879b6+        .quad   0x60dd16a379c86a8a++        // 2^52 * 3 * G++        .quad   0x35a2c8487381e559+        .quad   0x596ffea6d78082cb+        .quad   0xcb9771ebdba7b653+        .quad   0x5a08b5019b4da685+        .quad   0x3501d6f8153e47b8+        .quad   0xb7a9675414a2f60c+        .quad   0x112ee8b6455d9523+        .quad   0x4e62a3c18112ea8a+        .quad   0xc8d4ac04516ab786+        .quad   0x595af3215295b23d+        .quad   0xd6edd234db0230c1+        .quad   0x0929efe8825b41cc++        // 2^52 * 4 * G++        .quad   0x5f0601d1cbd0f2d3+        .quad   0x736e412f6132bb7f+        .quad   0x83604432238dde87+        .quad   0x1e3a5272f5c0753c+        .quad   0x8b3172b7ad56651d+        .quad   0x01581b7a3fabd717+        .quad   0x2dc94df6424df6e4+        .quad   0x30376e5d2c29284f+        .quad   0xd2918da78159a59c+        .quad   0x6bdc1cd93f0713f3+        .quad   0x565f7a934acd6590+        .quad   0x53daacec4cb4c128++        // 2^52 * 5 * G++        .quad   0x4ca73bd79cc8a7d6+        .quad   0x4d4a738f47e9a9b2+        .quad   0xf4cbf12942f5fe00+        .quad   0x01a13ff9bdbf0752+        .quad   0x99852bc3852cfdb0+        .quad   0x2cc12e9559d6ed0b+        .quad   0x70f9e2bf9b5ac27b+        .quad   0x4f3b8c117959ae99+        .quad   0x55b6c9c82ff26412+        .quad   0x1ac4a8c91fb667a8+        .quad   0xd527bfcfeb778bf2+        .quad   0x303337da7012a3be++        // 2^52 * 6 * G++        .quad   0x955422228c1c9d7c+        .quad   0x01fac1371a9b340f+        .quad   0x7e8d9177925b48d7+        .quad   0x53f8ad5661b3e31b+        .quad   0x976d3ccbfad2fdd1+        .quad   0xcb88839737a640a8+        .quad   0x2ff00c1d6734cb25+        .quad   0x269ff4dc789c2d2b+        .quad   0x0c003fbdc08d678d+        .quad   0x4d982fa37ead2b17+        .quad   0xc07e6bcdb2e582f1+        .quad   0x296c7291df412a44++        // 2^52 * 7 * G++        .quad   0x7903de2b33daf397+        .quad   0xd0ff0619c9a624b3+        .quad   0x8a1d252b555b3e18+        .quad   0x2b6d581c52e0b7c0+        .quad   0xdfb23205dab8b59e+        .quad   0x465aeaa0c8092250+        .quad   0xd133c1189a725d18+        .quad   0x2327370261f117d1+        .quad   0x3d0543d3623e7986+        .quad   0x679414c2c278a354+        .quad   0xae43f0cc726196f6+        .quad   0x7836c41f8245eaba++        // 2^52 * 8 * G++        .quad   0xe7a254db49e95a81+        .quad   0x5192d5d008b0ad73+        .quad   0x4d20e5b1d00afc07+        .quad   0x5d55f8012cf25f38+        .quad   0xca651e848011937c+        .quad   0xc6b0c46e6ef41a28+        .quad   0xb7021ba75f3f8d52+        .quad   0x119dff99ead7b9fd+        .quad   0x43eadfcbf4b31d4d+        .quad   0xc6503f7411148892+        .quad   0xfeee68c5060d3b17+        .quad   0x329293b3dd4a0ac8++        // 2^56 * 1 * G++        .quad   0x4e59214fe194961a+        .quad   0x49be7dc70d71cd4f+        .quad   0x9300cfd23b50f22d+        .quad   0x4789d446fc917232+        .quad   0x2879852d5d7cb208+        .quad   0xb8dedd70687df2e7+        .quad   0xdc0bffab21687891+        .quad   0x2b44c043677daa35+        .quad   0x1a1c87ab074eb78e+        .quad   0xfac6d18e99daf467+        .quad   0x3eacbbcd484f9067+        .quad   0x60c52eef2bb9a4e4++        // 2^56 * 2 * G++        .quad   0x0b5d89bc3bfd8bf1+        .quad   0xb06b9237c9f3551a+        .quad   0x0e4c16b0d53028f5+        .quad   0x10bc9c312ccfcaab+        .quad   0x702bc5c27cae6d11+        .quad   0x44c7699b54a48cab+        .quad   0xefbc4056ba492eb2+        .quad   0x70d77248d9b6676d+        .quad   0xaa8ae84b3ec2a05b+        .quad   0x98699ef4ed1781e0+        .quad   0x794513e4708e85d1+        .quad   0x63755bd3a976f413++        // 2^56 * 3 * G++        .quad   0xb55fa03e2ad10853+        .quad   0x356f75909ee63569+        .quad   0x9ff9f1fdbe69b890+        .quad   0x0d8cc1c48bc16f84+        .quad   0x3dc7101897f1acb7+        .quad   0x5dda7d5ec165bbd8+        .quad   0x508e5b9c0fa1020f+        .quad   0x2763751737c52a56+        .quad   0x029402d36eb419a9+        .quad   0xf0b44e7e77b460a5+        .quad   0xcfa86230d43c4956+        .quad   0x70c2dd8a7ad166e7++        // 2^56 * 4 * G++        .quad   0x656194509f6fec0e+        .quad   0xee2e7ea946c6518d+        .quad   0x9733c1f367e09b5c+        .quad   0x2e0fac6363948495+        .quad   0x91d4967db8ed7e13+        .quad   0x74252f0ad776817a+        .quad   0xe40982e00d852564+        .quad   0x32b8613816a53ce5+        .quad   0x79e7f7bee448cd64+        .quad   0x6ac83a67087886d0+        .quad   0xf89fd4d9a0e4db2e+        .quad   0x4179215c735a4f41++        // 2^56 * 5 * G++        .quad   0x8c7094e7d7dced2a+        .quad   0x97fb8ac347d39c70+        .quad   0xe13be033a906d902+        .quad   0x700344a30cd99d76+        .quad   0xe4ae33b9286bcd34+        .quad   0xb7ef7eb6559dd6dc+        .quad   0x278b141fb3d38e1f+        .quad   0x31fa85662241c286+        .quad   0xaf826c422e3622f4+        .quad   0xc12029879833502d+        .quad   0x9bc1b7e12b389123+        .quad   0x24bb2312a9952489++        // 2^56 * 6 * G++        .quad   0xb1a8ed1732de67c3+        .quad   0x3cb49418461b4948+        .quad   0x8ebd434376cfbcd2+        .quad   0x0fee3e871e188008+        .quad   0x41f80c2af5f85c6b+        .quad   0x687284c304fa6794+        .quad   0x8945df99a3ba1bad+        .quad   0x0d1d2af9ffeb5d16+        .quad   0xa9da8aa132621edf+        .quad   0x30b822a159226579+        .quad   0x4004197ba79ac193+        .quad   0x16acd79718531d76++        // 2^56 * 7 * G++        .quad   0x72df72af2d9b1d3d+        .quad   0x63462a36a432245a+        .quad   0x3ecea07916b39637+        .quad   0x123e0ef6b9302309+        .quad   0xc959c6c57887b6ad+        .quad   0x94e19ead5f90feba+        .quad   0x16e24e62a342f504+        .quad   0x164ed34b18161700+        .quad   0x487ed94c192fe69a+        .quad   0x61ae2cea3a911513+        .quad   0x877bf6d3b9a4de27+        .quad   0x78da0fc61073f3eb++        // 2^56 * 8 * G++        .quad   0x5bf15d28e52bc66a+        .quad   0x2c47e31870f01a8e+        .quad   0x2419afbc06c28bdd+        .quad   0x2d25deeb256b173a+        .quad   0xa29f80f1680c3a94+        .quad   0x71f77e151ae9e7e6+        .quad   0x1100f15848017973+        .quad   0x054aa4b316b38ddd+        .quad   0xdfc8468d19267cb8+        .quad   0x0b28789c66e54daf+        .quad   0x2aeb1d2a666eec17+        .quad   0x134610a6ab7da760++        // 2^60 * 1 * G++        .quad   0xcaf55ec27c59b23f+        .quad   0x99aeed3e154d04f2+        .quad   0x68441d72e14141f4+        .quad   0x140345133932a0a2+        .quad   0xd91430e0dc028c3c+        .quad   0x0eb955a85217c771+        .quad   0x4b09e1ed2c99a1fa+        .quad   0x42881af2bd6a743c+        .quad   0x7bfec69aab5cad3d+        .quad   0xc23e8cd34cb2cfad+        .quad   0x685dd14bfb37d6a2+        .quad   0x0ad6d64415677a18++        // 2^60 * 2 * G++        .quad   0x781a439e417becb5+        .quad   0x4ac5938cd10e0266+        .quad   0x5da385110692ac24+        .quad   0x11b065a2ade31233+        .quad   0x7914892847927e9f+        .quad   0x33dad6ef370aa877+        .quad   0x1f8f24fa11122703+        .quad   0x5265ac2f2adf9592+        .quad   0x405fdd309afcb346+        .quad   0xd9723d4428e63f54+        .quad   0x94c01df05f65aaae+        .quad   0x43e4dc3ae14c0809++        // 2^60 * 3 * G++        .quad   0xbc12c7f1a938a517+        .quad   0x473028ab3180b2e1+        .quad   0x3f78571efbcd254a+        .quad   0x74e534426ff6f90f+        .quad   0xea6f7ac3adc2c6a3+        .quad   0xd0e928f6e9717c94+        .quad   0xe2d379ead645eaf5+        .quad   0x46dd8785c51ffbbe+        .quad   0x709801be375c8898+        .quad   0x4b06dab5e3fd8348+        .quad   0x75880ced27230714+        .quad   0x2b09468fdd2f4c42++        // 2^60 * 4 * G++        .quad   0x97c749eeb701cb96+        .quad   0x83f438d4b6a369c3+        .quad   0x62962b8b9a402cd9+        .quad   0x6976c7509888df7b+        .quad   0x5b97946582ffa02a+        .quad   0xda096a51fea8f549+        .quad   0xa06351375f77af9b+        .quad   0x1bcfde61201d1e76+        .quad   0x4a4a5490246a59a2+        .quad   0xd63ebddee87fdd90+        .quad   0xd9437c670d2371fa+        .quad   0x69e87308d30f8ed6++        // 2^60 * 5 * G++        .quad   0x435a8bb15656beb0+        .quad   0xf8fac9ba4f4d5bca+        .quad   0xb9b278c41548c075+        .quad   0x3eb0ef76e892b622+        .quad   0x0f80bf028bc80303+        .quad   0x6aae16b37a18cefb+        .quad   0xdd47ea47d72cd6a3+        .quad   0x61943588f4ed39aa+        .quad   0xd26e5c3e91039f85+        .quad   0xc0e9e77df6f33aa9+        .quad   0xe8968c5570066a93+        .quad   0x3c34d1881faaaddd++        // 2^60 * 6 * G++        .quad   0x3f9d2b5ea09f9ec0+        .quad   0x1dab3b6fb623a890+        .quad   0xa09ba3ea72d926c4+        .quad   0x374193513fd8b36d+        .quad   0xbd5b0b8f2fffe0d9+        .quad   0x6aa254103ed24fb9+        .quad   0x2ac7d7bcb26821c4+        .quad   0x605b394b60dca36a+        .quad   0xb4e856e45a9d1ed2+        .quad   0xefe848766c97a9a2+        .quad   0xb104cf641e5eee7d+        .quad   0x2f50b81c88a71c8f++        // 2^60 * 7 * G++        .quad   0x31723c61fc6811bb+        .quad   0x9cb450486211800f+        .quad   0x768933d347995753+        .quad   0x3491a53502752fcd+        .quad   0x2b552ca0a7da522a+        .quad   0x3230b336449b0250+        .quad   0xf2c4c5bca4b99fb9+        .quad   0x7b2c674958074a22+        .quad   0xd55165883ed28cdf+        .quad   0x12d84fd2d362de39+        .quad   0x0a874ad3e3378e4f+        .quad   0x000d2b1f7c763e74++        // 2^60 * 8 * G++        .quad   0x3d420811d06d4a67+        .quad   0xbefc048590e0ffe3+        .quad   0xf870c6b7bd487bde+        .quad   0x6e2a7316319afa28+        .quad   0x9624778c3e94a8ab+        .quad   0x0ad6f3cee9a78bec+        .quad   0x948ac7810d743c4f+        .quad   0x76627935aaecfccc+        .quad   0x56a8ac24d6d59a9f+        .quad   0xc8db753e3096f006+        .quad   0x477f41e68f4c5299+        .quad   0x588d851cf6c86114++        // 2^64 * 1 * G++        .quad   0x51138ec78df6b0fe+        .quad   0x5397da89e575f51b+        .quad   0x09207a1d717af1b9+        .quad   0x2102fdba2b20d650+        .quad   0xcd2a65e777d1f515+        .quad   0x548991878faa60f1+        .quad   0xb1b73bbcdabc06e5+        .quad   0x654878cba97cc9fb+        .quad   0x969ee405055ce6a1+        .quad   0x36bca7681251ad29+        .quad   0x3a1af517aa7da415+        .quad   0x0ad725db29ecb2ba++        // 2^64 * 2 * G++        .quad   0xdc4267b1834e2457+        .quad   0xb67544b570ce1bc5+        .quad   0x1af07a0bf7d15ed7+        .quad   0x4aefcffb71a03650+        .quad   0xfec7bc0c9b056f85+        .quad   0x537d5268e7f5ffd7+        .quad   0x77afc6624312aefa+        .quad   0x4f675f5302399fd9+        .quad   0xc32d36360415171e+        .quad   0xcd2bef118998483b+        .quad   0x870a6eadd0945110+        .quad   0x0bccbb72a2a86561++        // 2^64 * 3 * G++        .quad   0x185e962feab1a9c8+        .quad   0x86e7e63565147dcd+        .quad   0xb092e031bb5b6df2+        .quad   0x4024f0ab59d6b73e+        .quad   0x186d5e4c50fe1296+        .quad   0xe0397b82fee89f7e+        .quad   0x3bc7f6c5507031b0+        .quad   0x6678fd69108f37c2+        .quad   0x1586fa31636863c2+        .quad   0x07f68c48572d33f2+        .quad   0x4f73cc9f789eaefc+        .quad   0x2d42e2108ead4701++        // 2^64 * 4 * G++        .quad   0x97f5131594dfd29b+        .quad   0x6155985d313f4c6a+        .quad   0xeba13f0708455010+        .quad   0x676b2608b8d2d322+        .quad   0x21717b0d0f537593+        .quad   0x914e690b131e064c+        .quad   0x1bb687ae752ae09f+        .quad   0x420bf3a79b423c6e+        .quad   0x8138ba651c5b2b47+        .quad   0x8671b6ec311b1b80+        .quad   0x7bff0cb1bc3135b0+        .quad   0x745d2ffa9c0cf1e0++        // 2^64 * 5 * G++        .quad   0xbf525a1e2bc9c8bd+        .quad   0xea5b260826479d81+        .quad   0xd511c70edf0155db+        .quad   0x1ae23ceb960cf5d0+        .quad   0x6036df5721d34e6a+        .quad   0xb1db8827997bb3d0+        .quad   0xd3c209c3c8756afa+        .quad   0x06e15be54c1dc839+        .quad   0x5b725d871932994a+        .quad   0x32351cb5ceb1dab0+        .quad   0x7dc41549dab7ca05+        .quad   0x58ded861278ec1f7++        // 2^64 * 6 * G++        .quad   0xd8173793f266c55c+        .quad   0xc8c976c5cc454e49+        .quad   0x5ce382f8bc26c3a8+        .quad   0x2ff39de85485f6f9+        .quad   0x2dfb5ba8b6c2c9a8+        .quad   0x48eeef8ef52c598c+        .quad   0x33809107f12d1573+        .quad   0x08ba696b531d5bd8+        .quad   0x77ed3eeec3efc57a+        .quad   0x04e05517d4ff4811+        .quad   0xea3d7a3ff1a671cb+        .quad   0x120633b4947cfe54++        // 2^64 * 7 * G++        .quad   0x0b94987891610042+        .quad   0x4ee7b13cecebfae8+        .quad   0x70be739594f0a4c0+        .quad   0x35d30a99b4d59185+        .quad   0x82bd31474912100a+        .quad   0xde237b6d7e6fbe06+        .quad   0xe11e761911ea79c6+        .quad   0x07433be3cb393bde+        .quad   0xff7944c05ce997f4+        .quad   0x575d3de4b05c51a3+        .quad   0x583381fd5a76847c+        .quad   0x2d873ede7af6da9f++        // 2^64 * 8 * G++        .quad   0x157a316443373409+        .quad   0xfab8b7eef4aa81d9+        .quad   0xb093fee6f5a64806+        .quad   0x2e773654707fa7b6+        .quad   0xaa6202e14e5df981+        .quad   0xa20d59175015e1f5+        .quad   0x18a275d3bae21d6c+        .quad   0x0543618a01600253+        .quad   0x0deabdf4974c23c1+        .quad   0xaa6f0a259dce4693+        .quad   0x04202cb8a29aba2c+        .quad   0x4b1443362d07960d++        // 2^68 * 1 * G++        .quad   0x47b837f753242cec+        .quad   0x256dc48cc04212f2+        .quad   0xe222fbfbe1d928c5+        .quad   0x48ea295bad8a2c07+        .quad   0x299b1c3f57c5715e+        .quad   0x96cb929e6b686d90+        .quad   0x3004806447235ab3+        .quad   0x2c435c24a44d9fe1+        .quad   0x0607c97c80f8833f+        .quad   0x0e851578ca25ec5b+        .quad   0x54f7450b161ebb6f+        .quad   0x7bcb4792a0def80e++        // 2^68 * 2 * G++        .quad   0x8487e3d02bc73659+        .quad   0x4baf8445059979df+        .quad   0xd17c975adcad6fbf+        .quad   0x57369f0bdefc96b6+        .quad   0x1cecd0a0045224c2+        .quad   0x757f1b1b69e53952+        .quad   0x775b7a925289f681+        .quad   0x1b6cc62016736148+        .quad   0xf1a9990175638698+        .quad   0x353dd1beeeaa60d3+        .quad   0x849471334c9ba488+        .quad   0x63fa6e6843ade311++        // 2^68 * 3 * G++        .quad   0xd15c20536597c168+        .quad   0x9f73740098d28789+        .quad   0x18aee7f13257ba1f+        .quad   0x3418bfda07346f14+        .quad   0x2195becdd24b5eb7+        .quad   0x5e41f18cc0cd44f9+        .quad   0xdf28074441ca9ede+        .quad   0x07073b98f35b7d67+        .quad   0xd03c676c4ce530d4+        .quad   0x0b64c0473b5df9f4+        .quad   0x065cef8b19b3a31e+        .quad   0x3084d661533102c9++        // 2^68 * 4 * G++        .quad   0xe1f6b79ebf8469ad+        .quad   0x15801004e2663135+        .quad   0x9a498330af74181b+        .quad   0x3ba2504f049b673c+        .quad   0x9a6ce876760321fd+        .quad   0x7fe2b5109eb63ad8+        .quad   0x00e7d4ae8ac80592+        .quad   0x73d86b7abb6f723a+        .quad   0x0b52b5606dba5ab6+        .quad   0xa9134f0fbbb1edab+        .quad   0x30a9520d9b04a635+        .quad   0x6813b8f37973e5db++        // 2^68 * 5 * G++        .quad   0x9854b054334127c1+        .quad   0x105d047882fbff25+        .quad   0xdb49f7f944186f4f+        .quad   0x1768e838bed0b900+        .quad   0xf194ca56f3157e29+        .quad   0x136d35705ef528a5+        .quad   0xdd4cef778b0599bc+        .quad   0x7d5472af24f833ed+        .quad   0xd0ef874daf33da47+        .quad   0x00d3be5db6e339f9+        .quad   0x3f2a8a2f9c9ceece+        .quad   0x5d1aeb792352435a++        // 2^68 * 6 * G++        .quad   0xf59e6bb319cd63ca+        .quad   0x670c159221d06839+        .quad   0xb06d565b2150cab6+        .quad   0x20fb199d104f12a3+        .quad   0x12c7bfaeb61ba775+        .quad   0xb84e621fe263bffd+        .quad   0x0b47a5c35c840dcf+        .quad   0x7e83be0bccaf8634+        .quad   0x61943dee6d99c120+        .quad   0x86101f2e460b9fe0+        .quad   0x6bb2f1518ee8598d+        .quad   0x76b76289fcc475cc++        // 2^68 * 7 * G++        .quad   0x791b4cc1756286fa+        .quad   0xdbced317d74a157c+        .quad   0x7e732421ea72bde6+        .quad   0x01fe18491131c8e9+        .quad   0x4245f1a1522ec0b3+        .quad   0x558785b22a75656d+        .quad   0x1d485a2548a1b3c0+        .quad   0x60959eccd58fe09f+        .quad   0x3ebfeb7ba8ed7a09+        .quad   0x49fdc2bbe502789c+        .quad   0x44ebce5d3c119428+        .quad   0x35e1eb55be947f4a++        // 2^68 * 8 * G++        .quad   0xdbdae701c5738dd3+        .quad   0xf9c6f635b26f1bee+        .quad   0x61e96a8042f15ef4+        .quad   0x3aa1d11faf60a4d8+        .quad   0x14fd6dfa726ccc74+        .quad   0x3b084cfe2f53b965+        .quad   0xf33ae4f552a2c8b4+        .quad   0x59aab07a0d40166a+        .quad   0x77bcec4c925eac25+        .quad   0x1848718460137738+        .quad   0x5b374337fea9f451+        .quad   0x1865e78ec8e6aa46++        // 2^72 * 1 * G++        .quad   0xccc4b7c7b66e1f7a+        .quad   0x44157e25f50c2f7e+        .quad   0x3ef06dfc713eaf1c+        .quad   0x582f446752da63f7+        .quad   0x967c54e91c529ccb+        .quad   0x30f6269264c635fb+        .quad   0x2747aff478121965+        .quad   0x17038418eaf66f5c+        .quad   0xc6317bd320324ce4+        .quad   0xa81042e8a4488bc4+        .quad   0xb21ef18b4e5a1364+        .quad   0x0c2a1c4bcda28dc9++        // 2^72 * 2 * G++        .quad   0xd24dc7d06f1f0447+        .quad   0xb2269e3edb87c059+        .quad   0xd15b0272fbb2d28f+        .quad   0x7c558bd1c6f64877+        .quad   0xedc4814869bd6945+        .quad   0x0d6d907dbe1c8d22+        .quad   0xc63bd212d55cc5ab+        .quad   0x5a6a9b30a314dc83+        .quad   0xd0ec1524d396463d+        .quad   0x12bb628ac35a24f0+        .quad   0xa50c3a791cbc5fa4+        .quad   0x0404a5ca0afbafc3++        // 2^72 * 3 * G++        .quad   0x8c1f40070aa743d6+        .quad   0xccbad0cb5b265ee8+        .quad   0x574b046b668fd2de+        .quad   0x46395bfdcadd9633+        .quad   0x62bc9e1b2a416fd1+        .quad   0xb5c6f728e350598b+        .quad   0x04343fd83d5d6967+        .quad   0x39527516e7f8ee98+        .quad   0x117fdb2d1a5d9a9c+        .quad   0x9c7745bcd1005c2a+        .quad   0xefd4bef154d56fea+        .quad   0x76579a29e822d016++        // 2^72 * 4 * G++        .quad   0x45b68e7e49c02a17+        .quad   0x23cd51a2bca9a37f+        .quad   0x3ed65f11ec224c1b+        .quad   0x43a384dc9e05bdb1+        .quad   0x333cb51352b434f2+        .quad   0xd832284993de80e1+        .quad   0xb5512887750d35ce+        .quad   0x02c514bb2a2777c1+        .quad   0x684bd5da8bf1b645+        .quad   0xfb8bd37ef6b54b53+        .quad   0x313916d7a9b0d253+        .quad   0x1160920961548059++        // 2^72 * 5 * G++        .quad   0xb44d166929dacfaa+        .quad   0xda529f4c8413598f+        .quad   0xe9ef63ca453d5559+        .quad   0x351e125bc5698e0b+        .quad   0x7a385616369b4dcd+        .quad   0x75c02ca7655c3563+        .quad   0x7dc21bf9d4f18021+        .quad   0x2f637d7491e6e042+        .quad   0xd4b49b461af67bbe+        .quad   0xd603037ac8ab8961+        .quad   0x71dee19ff9a699fb+        .quad   0x7f182d06e7ce2a9a++        // 2^72 * 6 * G++        .quad   0x7a7c8e64ab0168ec+        .quad   0xcb5a4a5515edc543+        .quad   0x095519d347cd0eda+        .quad   0x67d4ac8c343e93b0+        .quad   0x09454b728e217522+        .quad   0xaa58e8f4d484b8d8+        .quad   0xd358254d7f46903c+        .quad   0x44acc043241c5217+        .quad   0x1c7d6bbb4f7a5777+        .quad   0x8b35fed4918313e1+        .quad   0x4adca1c6c96b4684+        .quad   0x556d1c8312ad71bd++        // 2^72 * 7 * G++        .quad   0x17ef40e30c8d3982+        .quad   0x31f7073e15a3fa34+        .quad   0x4f21f3cb0773646e+        .quad   0x746c6c6d1d824eff+        .quad   0x81f06756b11be821+        .quad   0x0faff82310a3f3dd+        .quad   0xf8b2d0556a99465d+        .quad   0x097abe38cc8c7f05+        .quad   0x0c49c9877ea52da4+        .quad   0x4c4369559bdc1d43+        .quad   0x022c3809f7ccebd2+        .quad   0x577e14a34bee84bd++        // 2^72 * 8 * G++        .quad   0xf0e268ac61a73b0a+        .quad   0xf2fafa103791a5f5+        .quad   0xc1e13e826b6d00e9+        .quad   0x60fa7ee96fd78f42+        .quad   0x94fecebebd4dd72b+        .quad   0xf46a4fda060f2211+        .quad   0x124a5977c0c8d1ff+        .quad   0x705304b8fb009295+        .quad   0xb63d1d354d296ec6+        .quad   0xf3c3053e5fad31d8+        .quad   0x670b958cb4bd42ec+        .quad   0x21398e0ca16353fd++        // 2^76 * 1 * G++        .quad   0x216ab2ca8da7d2ef+        .quad   0x366ad9dd99f42827+        .quad   0xae64b9004fdd3c75+        .quad   0x403a395b53909e62+        .quad   0x86c5fc16861b7e9a+        .quad   0xf6a330476a27c451+        .quad   0x01667267a1e93597+        .quad   0x05ffb9cd6082dfeb+        .quad   0xa617fa9ff53f6139+        .quad   0x60f2b5e513e66cb6+        .quad   0xd7a8beefb3448aa4+        .quad   0x7a2932856f5ea192++        // 2^76 * 2 * G++        .quad   0x0b39d761b02de888+        .quad   0x5f550e7ed2414e1f+        .quad   0xa6bfa45822e1a940+        .quad   0x050a2f7dfd447b99+        .quad   0xb89c444879639302+        .quad   0x4ae4f19350c67f2c+        .quad   0xf0b35da8c81af9c6+        .quad   0x39d0003546871017+        .quad   0x437c3b33a650db77+        .quad   0x6bafe81dbac52bb2+        .quad   0xfe99402d2db7d318+        .quad   0x2b5b7eec372ba6ce++        // 2^76 * 3 * G++        .quad   0xb3bc4bbd83f50eef+        .quad   0x508f0c998c927866+        .quad   0x43e76587c8b7e66e+        .quad   0x0f7655a3a47f98d9+        .quad   0xa694404d613ac8f4+        .quad   0x500c3c2bfa97e72c+        .quad   0x874104d21fcec210+        .quad   0x1b205fb38604a8ee+        .quad   0x55ecad37d24b133c+        .quad   0x441e147d6038c90b+        .quad   0x656683a1d62c6fee+        .quad   0x0157d5dc87e0ecae++        // 2^76 * 4 * G++        .quad   0xf2a7af510354c13d+        .quad   0xd7a0b145aa372b60+        .quad   0x2869b96a05a3d470+        .quad   0x6528e42d82460173+        .quad   0x95265514d71eb524+        .quad   0xe603d8815df14593+        .quad   0x147cdf410d4de6b7+        .quad   0x5293b1730437c850+        .quad   0x23d0e0814bccf226+        .quad   0x92c745cd8196fb93+        .quad   0x8b61796c59541e5b+        .quad   0x40a44df0c021f978++        // 2^76 * 5 * G++        .quad   0xdaa869894f20ea6a+        .quad   0xea14a3d14c620618+        .quad   0x6001fccb090bf8be+        .quad   0x35f4e822947e9cf0+        .quad   0x86c96e514bc5d095+        .quad   0xf20d4098fca6804a+        .quad   0x27363d89c826ea5d+        .quad   0x39ca36565719cacf+        .quad   0x97506f2f6f87b75c+        .quad   0xc624aea0034ae070+        .quad   0x1ec856e3aad34dd6+        .quad   0x055b0be0e440e58f++        // 2^76 * 6 * G++        .quad   0x6469a17d89735d12+        .quad   0xdb6f27d5e662b9f1+        .quad   0x9fcba3286a395681+        .quad   0x363b8004d269af25+        .quad   0x4d12a04b6ea33da2+        .quad   0x57cf4c15e36126dd+        .quad   0x90ec9675ee44d967+        .quad   0x64ca348d2a985aac+        .quad   0x99588e19e4c4912d+        .quad   0xefcc3b4e1ca5ce6b+        .quad   0x4522ea60fa5b98d5+        .quad   0x7064bbab1de4a819++        // 2^76 * 7 * G++        .quad   0xb919e1515a770641+        .quad   0xa9a2e2c74e7f8039+        .quad   0x7527250b3df23109+        .quad   0x756a7330ac27b78b+        .quad   0xa290c06142542129+        .quad   0xf2e2c2aebe8d5b90+        .quad   0xcf2458db76abfe1b+        .quad   0x02157ade83d626bf+        .quad   0x3e46972a1b9a038b+        .quad   0x2e4ee66a7ee03fb4+        .quad   0x81a248776edbb4ca+        .quad   0x1a944ee88ecd0563++        // 2^76 * 8 * G++        .quad   0xd5a91d1151039372+        .quad   0x2ed377b799ca26de+        .quad   0xa17202acfd366b6b+        .quad   0x0730291bd6901995+        .quad   0xbb40a859182362d6+        .quad   0xb99f55778a4d1abb+        .quad   0x8d18b427758559f6+        .quad   0x26c20fe74d26235a+        .quad   0x648d1d9fe9cc22f5+        .quad   0x66bc561928dd577c+        .quad   0x47d3ed21652439d1+        .quad   0x49d271acedaf8b49++        // 2^80 * 1 * G++        .quad   0x89f5058a382b33f3+        .quad   0x5ae2ba0bad48c0b4+        .quad   0x8f93b503a53db36e+        .quad   0x5aa3ed9d95a232e6+        .quad   0x2798aaf9b4b75601+        .quad   0x5eac72135c8dad72+        .quad   0xd2ceaa6161b7a023+        .quad   0x1bbfb284e98f7d4e+        .quad   0x656777e9c7d96561+        .quad   0xcb2b125472c78036+        .quad   0x65053299d9506eee+        .quad   0x4a07e14e5e8957cc++        // 2^80 * 2 * G++        .quad   0x4ee412cb980df999+        .quad   0xa315d76f3c6ec771+        .quad   0xbba5edde925c77fd+        .quad   0x3f0bac391d313402+        .quad   0x240b58cdc477a49b+        .quad   0xfd38dade6447f017+        .quad   0x19928d32a7c86aad+        .quad   0x50af7aed84afa081+        .quad   0x6e4fde0115f65be5+        .quad   0x29982621216109b2+        .quad   0x780205810badd6d9+        .quad   0x1921a316baebd006++        // 2^80 * 3 * G++        .quad   0x89422f7edfb870fc+        .quad   0x2c296beb4f76b3bd+        .quad   0x0738f1d436c24df7+        .quad   0x6458df41e273aeb0+        .quad   0xd75aad9ad9f3c18b+        .quad   0x566a0eef60b1c19c+        .quad   0x3e9a0bac255c0ed9+        .quad   0x7b049deca062c7f5+        .quad   0xdccbe37a35444483+        .quad   0x758879330fedbe93+        .quad   0x786004c312c5dd87+        .quad   0x6093dccbc2950e64++        // 2^80 * 4 * G++        .quad   0x1ff39a8585e0706d+        .quad   0x36d0a5d8b3e73933+        .quad   0x43b9f2e1718f453b+        .quad   0x57d1ea084827a97c+        .quad   0x6bdeeebe6084034b+        .quad   0x3199c2b6780fb854+        .quad   0x973376abb62d0695+        .quad   0x6e3180c98b647d90+        .quad   0xee7ab6e7a128b071+        .quad   0xa4c1596d93a88baa+        .quad   0xf7b4de82b2216130+        .quad   0x363e999ddd97bd18++        // 2^80 * 5 * G++        .quad   0x96a843c135ee1fc4+        .quad   0x976eb35508e4c8cf+        .quad   0xb42f6801b58cd330+        .quad   0x48ee9b78693a052b+        .quad   0x2f1848dce24baec6+        .quad   0x769b7255babcaf60+        .quad   0x90cb3c6e3cefe931+        .quad   0x231f979bc6f9b355+        .quad   0x5c31de4bcc2af3c6+        .quad   0xb04bb030fe208d1f+        .quad   0xb78d7009c14fb466+        .quad   0x079bfa9b08792413++        // 2^80 * 6 * G++        .quad   0xe3903a51da300df4+        .quad   0x843964233da95ab0+        .quad   0xed3cf12d0b356480+        .quad   0x038c77f684817194+        .quad   0xf3c9ed80a2d54245+        .quad   0x0aa08b7877f63952+        .quad   0xd76dac63d1085475+        .quad   0x1ef4fb159470636b+        .quad   0x854e5ee65b167bec+        .quad   0x59590a4296d0cdc2+        .quad   0x72b2df3498102199+        .quad   0x575ee92a4a0bff56++        // 2^80 * 7 * G++        .quad   0xd4c080908a182fcf+        .quad   0x30e170c299489dbd+        .quad   0x05babd5752f733de+        .quad   0x43d4e7112cd3fd00+        .quad   0x5d46bc450aa4d801+        .quad   0xc3af1227a533b9d8+        .quad   0x389e3b262b8906c2+        .quad   0x200a1e7e382f581b+        .quad   0x518db967eaf93ac5+        .quad   0x71bc989b056652c0+        .quad   0xfe2b85d9567197f5+        .quad   0x050eca52651e4e38++        // 2^80 * 8 * G++        .quad   0xc3431ade453f0c9c+        .quad   0xe9f5045eff703b9b+        .quad   0xfcd97ac9ed847b3d+        .quad   0x4b0ee6c21c58f4c6+        .quad   0x97ac397660e668ea+        .quad   0x9b19bbfe153ab497+        .quad   0x4cb179b534eca79f+        .quad   0x6151c09fa131ae57+        .quad   0x3af55c0dfdf05d96+        .quad   0xdd262ee02ab4ee7a+        .quad   0x11b2bb8712171709+        .quad   0x1fef24fa800f030b++        // 2^84 * 1 * G++        .quad   0xb496123a6b6c6609+        .quad   0xa750fe8580ab5938+        .quad   0xf471bf39b7c27a5f+        .quad   0x507903ce77ac193c+        .quad   0xff91a66a90166220+        .quad   0xf22552ae5bf1e009+        .quad   0x7dff85d87f90df7c+        .quad   0x4f620ffe0c736fb9+        .quad   0x62f90d65dfde3e34+        .quad   0xcf28c592b9fa5fad+        .quad   0x99c86ef9c6164510+        .quad   0x25d448044a256c84++        // 2^84 * 2 * G++        .quad   0xbd68230ec7e9b16f+        .quad   0x0eb1b9c1c1c5795d+        .quad   0x7943c8c495b6b1ff+        .quad   0x2f9faf620bbacf5e+        .quad   0x2c7c4415c9022b55+        .quad   0x56a0d241812eb1fe+        .quad   0xf02ea1c9d7b65e0d+        .quad   0x4180512fd5323b26+        .quad   0xa4ff3e698a48a5db+        .quad   0xba6a3806bd95403b+        .quad   0x9f7ce1af47d5b65d+        .quad   0x15e087e55939d2fb++        // 2^84 * 3 * G++        .quad   0x12207543745c1496+        .quad   0xdaff3cfdda38610c+        .quad   0xe4e797272c71c34f+        .quad   0x39c07b1934bdede9+        .quad   0x8894186efb963f38+        .quad   0x48a00e80dc639bd5+        .quad   0xa4e8092be96c1c99+        .quad   0x5a097d54ca573661+        .quad   0x2d45892b17c9e755+        .quad   0xd033fd7289308df8+        .quad   0x6c2fe9d9525b8bd9+        .quad   0x2edbecf1c11cc079++        // 2^84 * 4 * G++        .quad   0x1616a4e3c715a0d2+        .quad   0x53623cb0f8341d4d+        .quad   0x96ef5329c7e899cb+        .quad   0x3d4e8dbba668baa6+        .quad   0xee0f0fddd087a25f+        .quad   0x9c7531555c3e34ee+        .quad   0x660c572e8fab3ab5+        .quad   0x0854fc44544cd3b2+        .quad   0x61eba0c555edad19+        .quad   0x24b533fef0a83de6+        .quad   0x3b77042883baa5f8+        .quad   0x678f82b898a47e8d++        // 2^84 * 5 * G++        .quad   0xb1491d0bd6900c54+        .quad   0x3539722c9d132636+        .quad   0x4db928920b362bc9+        .quad   0x4d7cd1fea68b69df+        .quad   0x1e09d94057775696+        .quad   0xeed1265c3cd951db+        .quad   0xfa9dac2b20bce16f+        .quad   0x0f7f76e0e8d089f4+        .quad   0x36d9ebc5d485b00c+        .quad   0xa2596492e4adb365+        .quad   0xc1659480c2119ccd+        .quad   0x45306349186e0d5f++        // 2^84 * 6 * G++        .quad   0x94ddd0c1a6cdff1d+        .quad   0x55f6f115e84213ae+        .quad   0x6c935f85992fcf6a+        .quad   0x067ee0f54a37f16f+        .quad   0x96a414ec2b072491+        .quad   0x1bb2218127a7b65b+        .quad   0x6d2849596e8a4af0+        .quad   0x65f3b08ccd27765f+        .quad   0xecb29fff199801f7+        .quad   0x9d361d1fa2a0f72f+        .quad   0x25f11d2375fd2f49+        .quad   0x124cefe80fe10fe2++        // 2^84 * 7 * G++        .quad   0x4c126cf9d18df255+        .quad   0xc1d471e9147a63b6+        .quad   0x2c6d3c73f3c93b5f+        .quad   0x6be3a6a2e3ff86a2+        .quad   0x1518e85b31b16489+        .quad   0x8faadcb7db710bfb+        .quad   0x39b0bdf4a14ae239+        .quad   0x05f4cbea503d20c1+        .quad   0xce040e9ec04145bc+        .quad   0xc71ff4e208f6834c+        .quad   0xbd546e8dab8847a3+        .quad   0x64666aa0a4d2aba5++        // 2^84 * 8 * G++        .quad   0x6841435a7c06d912+        .quad   0xca123c21bb3f830b+        .quad   0xd4b37b27b1cbe278+        .quad   0x1d753b84c76f5046+        .quad   0xb0c53bf73337e94c+        .quad   0x7cb5697e11e14f15+        .quad   0x4b84abac1930c750+        .quad   0x28dd4abfe0640468+        .quad   0x7dc0b64c44cb9f44+        .quad   0x18a3e1ace3925dbf+        .quad   0x7a3034862d0457c4+        .quad   0x4c498bf78a0c892e++        // 2^88 * 1 * G++        .quad   0x37d653fb1aa73196+        .quad   0x0f9495303fd76418+        .quad   0xad200b09fb3a17b2+        .quad   0x544d49292fc8613e+        .quad   0x22d2aff530976b86+        .quad   0x8d90b806c2d24604+        .quad   0xdca1896c4de5bae5+        .quad   0x28005fe6c8340c17+        .quad   0x6aefba9f34528688+        .quad   0x5c1bff9425107da1+        .quad   0xf75bbbcd66d94b36+        .quad   0x72e472930f316dfa++        // 2^88 * 2 * G++        .quad   0x2695208c9781084f+        .quad   0xb1502a0b23450ee1+        .quad   0xfd9daea603efde02+        .quad   0x5a9d2e8c2733a34c+        .quad   0x07f3f635d32a7627+        .quad   0x7aaa4d865f6566f0+        .quad   0x3c85e79728d04450+        .quad   0x1fee7f000fe06438+        .quad   0x765305da03dbf7e5+        .quad   0xa4daf2491434cdbd+        .quad   0x7b4ad5cdd24a88ec+        .quad   0x00f94051ee040543++        // 2^88 * 3 * G++        .quad   0x8d356b23c3d330b2+        .quad   0xf21c8b9bb0471b06+        .quad   0xb36c316c6e42b83c+        .quad   0x07d79c7e8beab10d+        .quad   0xd7ef93bb07af9753+        .quad   0x583ed0cf3db766a7+        .quad   0xce6998bf6e0b1ec5+        .quad   0x47b7ffd25dd40452+        .quad   0x87fbfb9cbc08dd12+        .quad   0x8a066b3ae1eec29b+        .quad   0x0d57242bdb1fc1bf+        .quad   0x1c3520a35ea64bb6++        // 2^88 * 4 * G++        .quad   0x80d253a6bccba34a+        .quad   0x3e61c3a13838219b+        .quad   0x90c3b6019882e396+        .quad   0x1c3d05775d0ee66f+        .quad   0xcda86f40216bc059+        .quad   0x1fbb231d12bcd87e+        .quad   0xb4956a9e17c70990+        .quad   0x38750c3b66d12e55+        .quad   0x692ef1409422e51a+        .quad   0xcbc0c73c2b5df671+        .quad   0x21014fe7744ce029+        .quad   0x0621e2c7d330487c++        // 2^88 * 5 * G++        .quad   0xaf9860cc8259838d+        .quad   0x90ea48c1c69f9adc+        .quad   0x6526483765581e30+        .quad   0x0007d6097bd3a5bc+        .quad   0xb7ae1796b0dbf0f3+        .quad   0x54dfafb9e17ce196+        .quad   0x25923071e9aaa3b4+        .quad   0x5d8e589ca1002e9d+        .quad   0xc0bf1d950842a94b+        .quad   0xb2d3c363588f2e3e+        .quad   0x0a961438bb51e2ef+        .quad   0x1583d7783c1cbf86++        // 2^88 * 6 * G++        .quad   0xeceea2ef5da27ae1+        .quad   0x597c3a1455670174+        .quad   0xc9a62a126609167a+        .quad   0x252a5f2e81ed8f70+        .quad   0x90034704cc9d28c7+        .quad   0x1d1b679ef72cc58f+        .quad   0x16e12b5fbe5b8726+        .quad   0x4958064e83c5580a+        .quad   0x0d2894265066e80d+        .quad   0xfcc3f785307c8c6b+        .quad   0x1b53da780c1112fd+        .quad   0x079c170bd843b388++        // 2^88 * 7 * G++        .quad   0x0506ece464fa6fff+        .quad   0xbee3431e6205e523+        .quad   0x3579422451b8ea42+        .quad   0x6dec05e34ac9fb00+        .quad   0xcdd6cd50c0d5d056+        .quad   0x9af7686dbb03573b+        .quad   0x3ca6723ff3c3ef48+        .quad   0x6768c0d7317b8acc+        .quad   0x94b625e5f155c1b3+        .quad   0x417bf3a7997b7b91+        .quad   0xc22cbddc6d6b2600+        .quad   0x51445e14ddcd52f4++        // 2^88 * 8 * G++        .quad   0x57502b4b3b144951+        .quad   0x8e67ff6b444bbcb3+        .quad   0xb8bd6927166385db+        .quad   0x13186f31e39295c8+        .quad   0x893147ab2bbea455+        .quad   0x8c53a24f92079129+        .quad   0x4b49f948be30f7a7+        .quad   0x12e990086e4fd43d+        .quad   0xf10c96b37fdfbb2e+        .quad   0x9f9a935e121ceaf9+        .quad   0xdf1136c43a5b983f+        .quad   0x77b2e3f05d3e99af++        // 2^92 * 1 * G++        .quad   0xfd0d75879cf12657+        .quad   0xe82fef94e53a0e29+        .quad   0xcc34a7f05bbb4be7+        .quad   0x0b251172a50c38a2+        .quad   0x9532f48fcc5cd29b+        .quad   0x2ba851bea3ce3671+        .quad   0x32dacaa051122941+        .quad   0x478d99d9350004f2+        .quad   0x1d5ad94890bb02c0+        .quad   0x50e208b10ec25115+        .quad   0xa26a22894ef21702+        .quad   0x4dc923343b524805++        // 2^92 * 2 * G++        .quad   0xe3828c400f8086b6+        .quad   0x3f77e6f7979f0dc8+        .quad   0x7ef6de304df42cb4+        .quad   0x5265797cb6abd784+        .quad   0x3ad3e3ebf36c4975+        .quad   0xd75d25a537862125+        .quad   0xe873943da025a516+        .quad   0x6bbc7cb4c411c847+        .quad   0x3c6f9cd1d4a50d56+        .quad   0xb6244077c6feab7e+        .quad   0x6ff9bf483580972e+        .quad   0x00375883b332acfb++        // 2^92 * 3 * G++        .quad   0x0001b2cd28cb0940+        .quad   0x63fb51a06f1c24c9+        .quad   0xb5ad8691dcd5ca31+        .quad   0x67238dbd8c450660+        .quad   0xc98bec856c75c99c+        .quad   0xe44184c000e33cf4+        .quad   0x0a676b9bba907634+        .quad   0x669e2cb571f379d7+        .quad   0xcb116b73a49bd308+        .quad   0x025aad6b2392729e+        .quad   0xb4793efa3f55d9b1+        .quad   0x72a1056140678bb9++        // 2^92 * 4 * G++        .quad   0xa2b6812b1cc9249d+        .quad   0x62866eee21211f58+        .quad   0x2cb5c5b85df10ece+        .quad   0x03a6b259e263ae00+        .quad   0x0d8d2909e2e505b6+        .quad   0x98ca78abc0291230+        .quad   0x77ef5569a9b12327+        .quad   0x7c77897b81439b47+        .quad   0xf1c1b5e2de331cb5+        .quad   0x5a9f5d8e15fca420+        .quad   0x9fa438f17bd932b1+        .quad   0x2a381bf01c6146e7++        // 2^92 * 5 * G++        .quad   0xac9b9879cfc811c1+        .quad   0x8b7d29813756e567+        .quad   0x50da4e607c70edfc+        .quad   0x5dbca62f884400b6+        .quad   0xf7c0be32b534166f+        .quad   0x27e6ca6419cf70d4+        .quad   0x934df7d7a957a759+        .quad   0x5701461dabdec2aa+        .quad   0x2c6747402c915c25+        .quad   0x1bdcd1a80b0d340a+        .quad   0x5e5601bd07b43f5f+        .quad   0x2555b4e05539a242++        // 2^92 * 6 * G++        .quad   0x6fc09f5266ddd216+        .quad   0xdce560a7c8e37048+        .quad   0xec65939da2df62fd+        .quad   0x7a869ae7e52ed192+        .quad   0x78409b1d87e463d4+        .quad   0xad4da95acdfb639d+        .quad   0xec28773755259b9c+        .quad   0x69c806e9c31230ab+        .quad   0x7b48f57414bb3f22+        .quad   0x68c7cee4aedccc88+        .quad   0xed2f936179ed80be+        .quad   0x25d70b885f77bc4b++        // 2^92 * 7 * G++        .quad   0x4151c3d9762bf4de+        .quad   0x083f435f2745d82b+        .quad   0x29775a2e0d23ddd5+        .quad   0x138e3a6269a5db24+        .quad   0x98459d29bb1ae4d4+        .quad   0x56b9c4c739f954ec+        .quad   0x832743f6c29b4b3e+        .quad   0x21ea8e2798b6878a+        .quad   0x87bef4b46a5a7b9c+        .quad   0xd2299d1b5fc1d062+        .quad   0x82409818dd321648+        .quad   0x5c5abeb1e5a2e03d++        // 2^92 * 8 * G++        .quad   0x14722af4b73c2ddb+        .quad   0xbc470c5f5a05060d+        .quad   0x00943eac2581b02e+        .quad   0x0e434b3b1f499c8f+        .quad   0x02cde6de1306a233+        .quad   0x7b5a52a2116f8ec7+        .quad   0xe1c681f4c1163b5b+        .quad   0x241d350660d32643+        .quad   0x6be4404d0ebc52c7+        .quad   0xae46233bb1a791f5+        .quad   0x2aec170ed25db42b+        .quad   0x1d8dfd966645d694++        // 2^96 * 1 * G++        .quad   0x296fa9c59c2ec4de+        .quad   0xbc8b61bf4f84f3cb+        .quad   0x1c7706d917a8f908+        .quad   0x63b795fc7ad3255d+        .quad   0xd598639c12ddb0a4+        .quad   0xa5d19f30c024866b+        .quad   0xd17c2f0358fce460+        .quad   0x07a195152e095e8a+        .quad   0xa8368f02389e5fc8+        .quad   0x90433b02cf8de43b+        .quad   0xafa1fd5dc5412643+        .quad   0x3e8fe83d032f0137++        // 2^96 * 2 * G++        .quad   0x2f8b15b90570a294+        .quad   0x94f2427067084549+        .quad   0xde1c5ae161bbfd84+        .quad   0x75ba3b797fac4007+        .quad   0x08704c8de8efd13c+        .quad   0xdfc51a8e33e03731+        .quad   0xa59d5da51260cde3+        .quad   0x22d60899a6258c86+        .quad   0x6239dbc070cdd196+        .quad   0x60fe8a8b6c7d8a9a+        .quad   0xb38847bceb401260+        .quad   0x0904d07b87779e5e++        // 2^96 * 3 * G++        .quad   0xb4ce1fd4ddba919c+        .quad   0xcf31db3ec74c8daa+        .quad   0x2c63cc63ad86cc51+        .quad   0x43e2143fbc1dde07+        .quad   0xf4322d6648f940b9+        .quad   0x06952f0cbd2d0c39+        .quad   0x167697ada081f931+        .quad   0x6240aacebaf72a6c+        .quad   0xf834749c5ba295a0+        .quad   0xd6947c5bca37d25a+        .quad   0x66f13ba7e7c9316a+        .quad   0x56bdaf238db40cac++        // 2^96 * 4 * G++        .quad   0x362ab9e3f53533eb+        .quad   0x338568d56eb93d40+        .quad   0x9e0e14521d5a5572+        .quad   0x1d24a86d83741318+        .quad   0x1310d36cc19d3bb2+        .quad   0x062a6bb7622386b9+        .quad   0x7c9b8591d7a14f5c+        .quad   0x03aa31507e1e5754+        .quad   0xf4ec7648ffd4ce1f+        .quad   0xe045eaf054ac8c1c+        .quad   0x88d225821d09357c+        .quad   0x43b261dc9aeb4859++        // 2^96 * 5 * G++        .quad   0xe55b1e1988bb79bb+        .quad   0xa09ed07dc17a359d+        .quad   0xb02c2ee2603dea33+        .quad   0x326055cf5b276bc2+        .quad   0x19513d8b6c951364+        .quad   0x94fe7126000bf47b+        .quad   0x028d10ddd54f9567+        .quad   0x02b4d5e242940964+        .quad   0xb4a155cb28d18df2+        .quad   0xeacc4646186ce508+        .quad   0xc49cf4936c824389+        .quad   0x27a6c809ae5d3410++        // 2^96 * 6 * G++        .quad   0x8ba6ebcd1f0db188+        .quad   0x37d3d73a675a5be8+        .quad   0xf22edfa315f5585a+        .quad   0x2cb67174ff60a17e+        .quad   0xcd2c270ac43d6954+        .quad   0xdd4a3e576a66cab2+        .quad   0x79fa592469d7036c+        .quad   0x221503603d8c2599+        .quad   0x59eecdf9390be1d0+        .quad   0xa9422044728ce3f1+        .quad   0x82891c667a94f0f4+        .quad   0x7b1df4b73890f436++        // 2^96 * 7 * G++        .quad   0xe492f2e0b3b2a224+        .quad   0x7c6c9e062b551160+        .quad   0x15eb8fe20d7f7b0e+        .quad   0x61fcef2658fc5992+        .quad   0x5f2e221807f8f58c+        .quad   0xe3555c9fd49409d4+        .quad   0xb2aaa88d1fb6a630+        .quad   0x68698245d352e03d+        .quad   0xdbb15d852a18187a+        .quad   0xf3e4aad386ddacd7+        .quad   0x44bae2810ff6c482+        .quad   0x46cf4c473daf01cf++        // 2^96 * 8 * G++        .quad   0x426525ed9ec4e5f9+        .quad   0x0e5eda0116903303+        .quad   0x72b1a7f2cbe5cadc+        .quad   0x29387bcd14eb5f40+        .quad   0x213c6ea7f1498140+        .quad   0x7c1e7ef8392b4854+        .quad   0x2488c38c5629ceba+        .quad   0x1065aae50d8cc5bb+        .quad   0x1c2c4525df200d57+        .quad   0x5c3b2dd6bfca674a+        .quad   0x0a07e7b1e1834030+        .quad   0x69a198e64f1ce716++        // 2^100 * 1 * G++        .quad   0x7afcd613efa9d697+        .quad   0x0cc45aa41c067959+        .quad   0xa56fe104c1fada96+        .quad   0x3a73b70472e40365+        .quad   0x7b26e56b9e2d4734+        .quad   0xc4c7132b81c61675+        .quad   0xef5c9525ec9cde7f+        .quad   0x39c80b16e71743ad+        .quad   0x0f196e0d1b826c68+        .quad   0xf71ff0e24960e3db+        .quad   0x6113167023b7436c+        .quad   0x0cf0ea5877da7282++        // 2^100 * 2 * G++        .quad   0x196c80a4ddd4ccbd+        .quad   0x22e6f55d95f2dd9d+        .quad   0xc75e33c740d6c71b+        .quad   0x7bb51279cb3c042f+        .quad   0xe332ced43ba6945a+        .quad   0xde0b1361e881c05d+        .quad   0x1ad40f095e67ed3b+        .quad   0x5da8acdab8c63d5d+        .quad   0xc4b6664a3a70159f+        .quad   0x76194f0f0a904e14+        .quad   0xa5614c39a4096c13+        .quad   0x6cd0ff50979feced++        // 2^100 * 3 * G++        .quad   0xc0e067e78f4428ac+        .quad   0x14835ab0a61135e3+        .quad   0xf21d14f338062935+        .quad   0x6390a4c8df04849c+        .quad   0x7fecfabdb04ba18e+        .quad   0xd0fc7bfc3bddbcf7+        .quad   0xa41d486e057a131c+        .quad   0x641a4391f2223a61+        .quad   0xc5c6b95aa606a8db+        .quad   0x914b7f9eb06825f1+        .quad   0x2a731f6b44fc9eff+        .quad   0x30ddf38562705cfc++        // 2^100 * 4 * G++        .quad   0x4e3dcbdad1bff7f9+        .quad   0xc9118e8220645717+        .quad   0xbacccebc0f189d56+        .quad   0x1b4822e9d4467668+        .quad   0x33bef2bd68bcd52c+        .quad   0xc649dbb069482ef2+        .quad   0xb5b6ee0c41cb1aee+        .quad   0x5c294d270212a7e5+        .quad   0xab360a7f25563781+        .quad   0x2512228a480f7958+        .quad   0xc75d05276114b4e3+        .quad   0x222d9625d976fe2a++        // 2^100 * 5 * G++        .quad   0x1c717f85b372ace1+        .quad   0x81930e694638bf18+        .quad   0x239cad056bc08b58+        .quad   0x0b34271c87f8fff4+        .quad   0x0f94be7e0a344f85+        .quad   0xeb2faa8c87f22c38+        .quad   0x9ce1e75e4ee16f0f+        .quad   0x43e64e5418a08dea+        .quad   0x8155e2521a35ce63+        .quad   0xbe100d4df912028e+        .quad   0xbff80bf8a57ddcec+        .quad   0x57342dc96d6bc6e4++        // 2^100 * 6 * G++        .quad   0xefeef065c8ce5998+        .quad   0xbf029510b5cbeaa2+        .quad   0x8c64a10620b7c458+        .quad   0x35134fb231c24855+        .quad   0xf3c3bcb71e707bf6+        .quad   0x351d9b8c7291a762+        .quad   0x00502e6edad69a33+        .quad   0x522f521f1ec8807f+        .quad   0x272c1f46f9a3902b+        .quad   0xc91ba3b799657bcc+        .quad   0xae614b304f8a1c0e+        .quad   0x7afcaad70b99017b++        // 2^100 * 7 * G++        .quad   0xc25ded54a4b8be41+        .quad   0x902d13e11bb0e2dd+        .quad   0x41f43233cde82ab2+        .quad   0x1085faa5c3aae7cb+        .quad   0xa88141ecef842b6b+        .quad   0x55e7b14797abe6c5+        .quad   0x8c748f9703784ffe+        .quad   0x5b50a1f7afcd00b7+        .quad   0x9b840f66f1361315+        .quad   0x18462242701003e9+        .quad   0x65ed45fae4a25080+        .quad   0x0a2862393fda7320++        // 2^100 * 8 * G++        .quad   0x46ab13c8347cbc9d+        .quad   0x3849e8d499c12383+        .quad   0x4cea314087d64ac9+        .quad   0x1f354134b1a29ee7+        .quad   0x960e737b6ecb9d17+        .quad   0xfaf24948d67ceae1+        .quad   0x37e7a9b4d55e1b89+        .quad   0x5cb7173cb46c59eb+        .quad   0x4a89e68b82b7abf0+        .quad   0xf41cd9279ba6b7b9+        .quad   0x16e6c210e18d876f+        .quad   0x7cacdb0f7f1b09c6++        // 2^104 * 1 * G++        .quad   0x9062b2e0d91a78bc+        .quad   0x47c9889cc8509667+        .quad   0x9df54a66405070b8+        .quad   0x7369e6a92493a1bf+        .quad   0xe1014434dcc5caed+        .quad   0x47ed5d963c84fb33+        .quad   0x70019576ed86a0e7+        .quad   0x25b2697bd267f9e4+        .quad   0x9d673ffb13986864+        .quad   0x3ca5fbd9415dc7b8+        .quad   0xe04ecc3bdf273b5e+        .quad   0x1420683db54e4cd2++        // 2^104 * 2 * G++        .quad   0xb478bd1e249dd197+        .quad   0x620c35005e58c102+        .quad   0xfb02d32fccbaac5c+        .quad   0x60b63bebf508a72d+        .quad   0x34eebb6fc1cc5ad0+        .quad   0x6a1b0ce99646ac8b+        .quad   0xd3b0da49a66bde53+        .quad   0x31e83b4161d081c1+        .quad   0x97e8c7129e062b4f+        .quad   0x49e48f4f29320ad8+        .quad   0x5bece14b6f18683f+        .quad   0x55cf1eb62d550317++        // 2^104 * 3 * G++        .quad   0x5879101065c23d58+        .quad   0x8b9d086d5094819c+        .quad   0xe2402fa912c55fa7+        .quad   0x669a6564570891d4+        .quad   0x3076b5e37df58c52+        .quad   0xd73ab9dde799cc36+        .quad   0xbd831ce34913ee20+        .quad   0x1a56fbaa62ba0133+        .quad   0x943e6b505c9dc9ec+        .quad   0x302557bba77c371a+        .quad   0x9873ae5641347651+        .quad   0x13c4836799c58a5c++        // 2^104 * 4 * G++        .quad   0x423a5d465ab3e1b9+        .quad   0xfc13c187c7f13f61+        .quad   0x19f83664ecb5b9b6+        .quad   0x66f80c93a637b607+        .quad   0xc4dcfb6a5d8bd080+        .quad   0xdeebc4ec571a4842+        .quad   0xd4b2e883b8e55365+        .quad   0x50bdc87dc8e5b827+        .quad   0x606d37836edfe111+        .quad   0x32353e15f011abd9+        .quad   0x64b03ac325b73b96+        .quad   0x1dd56444725fd5ae++        // 2^104 * 5 * G++        .quad   0x8fa47ff83362127d+        .quad   0xbc9f6ac471cd7c15+        .quad   0x6e71454349220c8b+        .quad   0x0e645912219f732e+        .quad   0xc297e60008bac89a+        .quad   0x7d4cea11eae1c3e0+        .quad   0xf3e38be19fe7977c+        .quad   0x3a3a450f63a305cd+        .quad   0x078f2f31d8394627+        .quad   0x389d3183de94a510+        .quad   0xd1e36c6d17996f80+        .quad   0x318c8d9393a9a87b++        // 2^104 * 6 * G++        .quad   0xf2745d032afffe19+        .quad   0x0c9f3c497f24db66+        .quad   0xbc98d3e3ba8598ef+        .quad   0x224c7c679a1d5314+        .quad   0x5d669e29ab1dd398+        .quad   0xfc921658342d9e3b+        .quad   0x55851dfdf35973cd+        .quad   0x509a41c325950af6+        .quad   0xbdc06edca6f925e9+        .quad   0x793ef3f4641b1f33+        .quad   0x82ec12809d833e89+        .quad   0x05bff02328a11389++        // 2^104 * 7 * G++        .quad   0x3632137023cae00b+        .quad   0x544acf0ad1accf59+        .quad   0x96741049d21a1c88+        .quad   0x780b8cc3fa2a44a7+        .quad   0x6881a0dd0dc512e4+        .quad   0x4fe70dc844a5fafe+        .quad   0x1f748e6b8f4a5240+        .quad   0x576277cdee01a3ea+        .quad   0x1ef38abc234f305f+        .quad   0x9a577fbd1405de08+        .quad   0x5e82a51434e62a0d+        .quad   0x5ff418726271b7a1++        // 2^104 * 8 * G++        .quad   0x398e080c1789db9d+        .quad   0xa7602025f3e778f5+        .quad   0xfa98894c06bd035d+        .quad   0x106a03dc25a966be+        .quad   0xe5db47e813b69540+        .quad   0xf35d2a3b432610e1+        .quad   0xac1f26e938781276+        .quad   0x29d4db8ca0a0cb69+        .quad   0xd9ad0aaf333353d0+        .quad   0x38669da5acd309e5+        .quad   0x3c57658ac888f7f0+        .quad   0x4ab38a51052cbefa++        // 2^108 * 1 * G++        .quad   0xdfdacbee4324c0e9+        .quad   0x054442883f955bb7+        .quad   0xdef7aaa8ea31609f+        .quad   0x68aee70642287cff+        .quad   0xf68fe2e8809de054+        .quad   0xe3bc096a9c82bad1+        .quad   0x076353d40aadbf45+        .quad   0x7b9b1fb5dea1959e+        .quad   0xf01cc8f17471cc0c+        .quad   0x95242e37579082bb+        .quad   0x27776093d3e46b5f+        .quad   0x2d13d55a28bd85fb++        // 2^108 * 2 * G++        .quad   0xfac5d2065b35b8da+        .quad   0xa8da8a9a85624bb7+        .quad   0xccd2ca913d21cd0f+        .quad   0x6b8341ee8bf90d58+        .quad   0xbf019cce7aee7a52+        .quad   0xa8ded2b6e454ead3+        .quad   0x3c619f0b87a8bb19+        .quad   0x3619b5d7560916d8+        .quad   0x3579f26b0282c4b2+        .quad   0x64d592f24fafefae+        .quad   0xb7cded7b28c8c7c0+        .quad   0x6a927b6b7173a8d7++        // 2^108 * 3 * G++        .quad   0x1f6db24f986e4656+        .quad   0x1021c02ed1e9105b+        .quad   0xf8ff3fff2cc0a375+        .quad   0x1d2a6bf8c6c82592+        .quad   0x8d7040863ece88eb+        .quad   0xf0e307a980eec08c+        .quad   0xac2250610d788fda+        .quad   0x056d92a43a0d478d+        .quad   0x1b05a196fc3da5a1+        .quad   0x77d7a8c243b59ed0+        .quad   0x06da3d6297d17918+        .quad   0x66fbb494f12353f7++        // 2^108 * 4 * G++        .quad   0x751a50b9d85c0fb8+        .quad   0xd1afdc258bcf097b+        .quad   0x2f16a6a38309a969+        .quad   0x14ddff9ee5b00659+        .quad   0xd6d70996f12309d6+        .quad   0xdbfb2385e9c3d539+        .quad   0x46d602b0f7552411+        .quad   0x270a0b0557843e0c+        .quad   0x61ff0640a7862bcc+        .quad   0x81cac09a5f11abfe+        .quad   0x9047830455d12abb+        .quad   0x19a4bde1945ae873++        // 2^108 * 5 * G++        .quad   0x9b9f26f520a6200a+        .quad   0x64804443cf13eaf8+        .quad   0x8a63673f8631edd3+        .quad   0x72bbbce11ed39dc1+        .quad   0x40c709dec076c49f+        .quad   0x657bfaf27f3e53f6+        .quad   0x40662331eca042c4+        .quad   0x14b375487eb4df04+        .quad   0xae853c94ab66dc47+        .quad   0xeb62343edf762d6e+        .quad   0xf08e0e186fb2f7d1+        .quad   0x4f0b1c02700ab37a++        // 2^108 * 6 * G++        .quad   0xe1706787d81951fa+        .quad   0xa10a2c8eb290c77b+        .quad   0xe7382fa03ed66773+        .quad   0x0a4d84710bcc4b54+        .quad   0x79fd21ccc1b2e23f+        .quad   0x4ae7c281453df52a+        .quad   0xc8172ec9d151486b+        .quad   0x68abe9443e0a7534+        .quad   0xda12c6c407831dcb+        .quad   0x0da230d74d5c510d+        .quad   0x4ab1531e6bd404e1+        .quad   0x4106b166bcf440ef++        // 2^108 * 7 * G++        .quad   0x02e57a421cd23668+        .quad   0x4ad9fb5d0eaef6fd+        .quad   0x954e6727b1244480+        .quad   0x7f792f9d2699f331+        .quad   0xa485ccd539e4ecf2+        .quad   0x5aa3f3ad0555bab5+        .quad   0x145e3439937df82d+        .quad   0x1238b51e1214283f+        .quad   0x0b886b925fd4d924+        .quad   0x60906f7a3626a80d+        .quad   0xecd367b4b98abd12+        .quad   0x2876beb1def344cf++        // 2^108 * 8 * G++        .quad   0xdc84e93563144691+        .quad   0x632fe8a0d61f23f4+        .quad   0x4caa800612a9a8d5+        .quad   0x48f9dbfa0e9918d3+        .quad   0xd594b3333a8a85f8+        .quad   0x4ea37689e78d7d58+        .quad   0x73bf9f455e8e351f+        .quad   0x5507d7d2bc41ebb4+        .quad   0x1ceb2903299572fc+        .quad   0x7c8ccaa29502d0ee+        .quad   0x91bfa43411cce67b+        .quad   0x5784481964a831e7++        // 2^112 * 1 * G++        .quad   0xda7c2b256768d593+        .quad   0x98c1c0574422ca13+        .quad   0xf1a80bd5ca0ace1d+        .quad   0x29cdd1adc088a690+        .quad   0xd6cfd1ef5fddc09c+        .quad   0xe82b3efdf7575dce+        .quad   0x25d56b5d201634c2+        .quad   0x3041c6bb04ed2b9b+        .quad   0x0ff2f2f9d956e148+        .quad   0xade797759f356b2e+        .quad   0x1a4698bb5f6c025c+        .quad   0x104bbd6814049a7b++        // 2^112 * 2 * G++        .quad   0x51f0fd3168f1ed67+        .quad   0x2c811dcdd86f3bc2+        .quad   0x44dc5c4304d2f2de+        .quad   0x5be8cc57092a7149+        .quad   0xa95d9a5fd67ff163+        .quad   0xe92be69d4cc75681+        .quad   0xb7f8024cde20f257+        .quad   0x204f2a20fb072df5+        .quad   0xc8143b3d30ebb079+        .quad   0x7589155abd652e30+        .quad   0x653c3c318f6d5c31+        .quad   0x2570fb17c279161f++        // 2^112 * 3 * G++        .quad   0x3efa367f2cb61575+        .quad   0xf5f96f761cd6026c+        .quad   0xe8c7142a65b52562+        .quad   0x3dcb65ea53030acd+        .quad   0x192ea9550bb8245a+        .quad   0xc8e6fba88f9050d1+        .quad   0x7986ea2d88a4c935+        .quad   0x241c5f91de018668+        .quad   0x28d8172940de6caa+        .quad   0x8fbf2cf022d9733a+        .quad   0x16d7fcdd235b01d1+        .quad   0x08420edd5fcdf0e5++        // 2^112 * 4 * G++        .quad   0xcdff20ab8362fa4a+        .quad   0x57e118d4e21a3e6e+        .quad   0xe3179617fc39e62b+        .quad   0x0d9a53efbc1769fd+        .quad   0x0358c34e04f410ce+        .quad   0xb6135b5a276e0685+        .quad   0x5d9670c7ebb91521+        .quad   0x04d654f321db889c+        .quad   0x5e7dc116ddbdb5d5+        .quad   0x2954deb68da5dd2d+        .quad   0x1cb608173334a292+        .quad   0x4a7a4f2618991ad7++        // 2^112 * 5 * G++        .quad   0xf4a718025fb15f95+        .quad   0x3df65f346b5c1b8f+        .quad   0xcdfcf08500e01112+        .quad   0x11b50c4cddd31848+        .quad   0x24c3b291af372a4b+        .quad   0x93da8270718147f2+        .quad   0xdd84856486899ef2+        .quad   0x4a96314223e0ee33+        .quad   0xa6e8274408a4ffd6+        .quad   0x738e177e9c1576d9+        .quad   0x773348b63d02b3f2+        .quad   0x4f4bce4dce6bcc51++        // 2^112 * 6 * G++        .quad   0xa71fce5ae2242584+        .quad   0x26ea725692f58a9e+        .quad   0xd21a09d71cea3cf4+        .quad   0x73fcdd14b71c01e6+        .quad   0x30e2616ec49d0b6f+        .quad   0xe456718fcaec2317+        .quad   0x48eb409bf26b4fa6+        .quad   0x3042cee561595f37+        .quad   0x427e7079449bac41+        .quad   0x855ae36dbce2310a+        .quad   0x4cae76215f841a7c+        .quad   0x389e740c9a9ce1d6++        // 2^112 * 7 * G++        .quad   0x64fcb3ae34dcb9ce+        .quad   0x97500323e348d0ad+        .quad   0x45b3f07d62c6381b+        .quad   0x61545379465a6788+        .quad   0xc9bd78f6570eac28+        .quad   0xe55b0b3227919ce1+        .quad   0x65fc3eaba19b91ed+        .quad   0x25c425e5d6263690+        .quad   0x3f3e06a6f1d7de6e+        .quad   0x3ef976278e062308+        .quad   0x8c14f6264e8a6c77+        .quad   0x6539a08915484759++        // 2^112 * 8 * G++        .quad   0xe9d21f74c3d2f773+        .quad   0xc150544125c46845+        .quad   0x624e5ce8f9b99e33+        .quad   0x11c5e4aac5cd186c+        .quad   0xddc4dbd414bb4a19+        .quad   0x19b2bc3c98424f8e+        .quad   0x48a89fd736ca7169+        .quad   0x0f65320ef019bd90+        .quad   0xd486d1b1cafde0c6+        .quad   0x4f3fe6e3163b5181+        .quad   0x59a8af0dfaf2939a+        .quad   0x4cabc7bdec33072a++        // 2^116 * 1 * G++        .quad   0x16faa8fb532f7428+        .quad   0xdbd42ea046a4e272+        .quad   0x5337653b8b9ea480+        .quad   0x4065947223973f03+        .quad   0xf7c0a19c1a54a044+        .quad   0x4a1c5e2477bd9fbb+        .quad   0xa6e3ca115af22972+        .quad   0x1819bb953f2e9e0d+        .quad   0x498fbb795e042e84+        .quad   0x7d0dd89a7698b714+        .quad   0x8bfb0ba427fe6295+        .quad   0x36ba82e721200524++        // 2^116 * 2 * G++        .quad   0xd60ecbb74245ec41+        .quad   0xfd9be89e34348716+        .quad   0xc9240afee42284de+        .quad   0x4472f648d0531db4+        .quad   0xc8d69d0a57274ed5+        .quad   0x45ba803260804b17+        .quad   0xdf3cda102255dfac+        .quad   0x77d221232709b339+        .quad   0x498a6d7064ad94d8+        .quad   0xa5b5c8fd9af62263+        .quad   0x8ca8ed0545c141f4+        .quad   0x2c63bec3662d358c++        // 2^116 * 3 * G++        .quad   0x7fe60d8bea787955+        .quad   0xb9dc117eb5f401b7+        .quad   0x91c7c09a19355cce+        .quad   0x22692ef59442bedf+        .quad   0x9a518b3a8586f8bf+        .quad   0x9ee71af6cbb196f0+        .quad   0xaa0625e6a2385cf2+        .quad   0x1deb2176ddd7c8d1+        .quad   0x8563d19a2066cf6c+        .quad   0x401bfd8c4dcc7cd7+        .quad   0xd976a6becd0d8f62+        .quad   0x67cfd773a278b05e++        // 2^116 * 4 * G++        .quad   0x8dec31faef3ee475+        .quad   0x99dbff8a9e22fd92+        .quad   0x512d11594e26cab1+        .quad   0x0cde561eec4310b9+        .quad   0x2d5fa9855a4e586a+        .quad   0x65f8f7a449beab7e+        .quad   0xaa074dddf21d33d3+        .quad   0x185cba721bcb9dee+        .quad   0x93869da3f4e3cb41+        .quad   0xbf0392f540f7977e+        .quad   0x026204fcd0463b83+        .quad   0x3ec91a769eec6eed++        // 2^116 * 5 * G++        .quad   0x1e9df75bf78166ad+        .quad   0x4dfda838eb0cd7af+        .quad   0xba002ed8c1eaf988+        .quad   0x13fedb3e11f33cfc+        .quad   0x0fad2fb7b0a3402f+        .quad   0x46615ecbfb69f4a8+        .quad   0xf745bcc8c5f8eaa6+        .quad   0x7a5fa8794a94e896+        .quad   0x52958faa13cd67a1+        .quad   0x965ee0818bdbb517+        .quad   0x16e58daa2e8845b3+        .quad   0x357d397d5499da8f++        // 2^116 * 6 * G++        .quad   0x1ebfa05fb0bace6c+        .quad   0xc934620c1caf9a1e+        .quad   0xcc771cc41d82b61a+        .quad   0x2d94a16aa5f74fec+        .quad   0x481dacb4194bfbf8+        .quad   0x4d77e3f1bae58299+        .quad   0x1ef4612e7d1372a0+        .quad   0x3a8d867e70ff69e1+        .quad   0x6f58cd5d55aff958+        .quad   0xba3eaa5c75567721+        .quad   0x75c123999165227d+        .quad   0x69be1343c2f2b35e++        // 2^116 * 7 * G++        .quad   0x0e091d5ee197c92a+        .quad   0x4f51019f2945119f+        .quad   0x143679b9f034e99c+        .quad   0x7d88112e4d24c696+        .quad   0x82bbbdac684b8de3+        .quad   0xa2f4c7d03fca0718+        .quad   0x337f92fbe096aaa8+        .quad   0x200d4d8c63587376+        .quad   0x208aed4b4893b32b+        .quad   0x3efbf23ebe59b964+        .quad   0xd762deb0dba5e507+        .quad   0x69607bd681bd9d94++        // 2^116 * 8 * G++        .quad   0xf6be021068de1ce1+        .quad   0xe8d518e70edcbc1f+        .quad   0xe3effdd01b5505a5+        .quad   0x35f63353d3ec3fd0+        .quad   0x3b7f3bd49323a902+        .quad   0x7c21b5566b2c6e53+        .quad   0xe5ba8ff53a7852a7+        .quad   0x28bc77a5838ece00+        .quad   0x63ba78a8e25d8036+        .quad   0x63651e0094333490+        .quad   0x48d82f20288ce532+        .quad   0x3a31abfa36b57524++        // 2^120 * 1 * G++        .quad   0x239e9624089c0a2e+        .quad   0xc748c4c03afe4738+        .quad   0x17dbed2a764fa12a+        .quad   0x639b93f0321c8582+        .quad   0xc08f788f3f78d289+        .quad   0xfe30a72ca1404d9f+        .quad   0xf2778bfccf65cc9d+        .quad   0x7ee498165acb2021+        .quad   0x7bd508e39111a1c3+        .quad   0x2b2b90d480907489+        .quad   0xe7d2aec2ae72fd19+        .quad   0x0edf493c85b602a6++        // 2^120 * 2 * G++        .quad   0xaecc8158599b5a68+        .quad   0xea574f0febade20e+        .quad   0x4fe41d7422b67f07+        .quad   0x403b92e3019d4fb4+        .quad   0x6767c4d284764113+        .quad   0xa090403ff7f5f835+        .quad   0x1c8fcffacae6bede+        .quad   0x04c00c54d1dfa369+        .quad   0x4dc22f818b465cf8+        .quad   0x71a0f35a1480eff8+        .quad   0xaee8bfad04c7d657+        .quad   0x355bb12ab26176f4++        // 2^120 * 3 * G++        .quad   0xa71e64cc7493bbf4+        .quad   0xe5bd84d9eca3b0c3+        .quad   0x0a6bc50cfa05e785+        .quad   0x0f9b8132182ec312+        .quad   0xa301dac75a8c7318+        .quad   0xed90039db3ceaa11+        .quad   0x6f077cbf3bae3f2d+        .quad   0x7518eaf8e052ad8e+        .quad   0xa48859c41b7f6c32+        .quad   0x0f2d60bcf4383298+        .quad   0x1815a929c9b1d1d9+        .quad   0x47c3871bbb1755c4++        // 2^120 * 4 * G++        .quad   0x5144539771ec4f48+        .quad   0xf805b17dc98c5d6e+        .quad   0xf762c11a47c3c66b+        .quad   0x00b89b85764699dc+        .quad   0xfbe65d50c85066b0+        .quad   0x62ecc4b0b3a299b0+        .quad   0xe53754ea441ae8e0+        .quad   0x08fea02ce8d48d5f+        .quad   0x824ddd7668deead0+        .quad   0xc86445204b685d23+        .quad   0xb514cfcd5d89d665+        .quad   0x473829a74f75d537++        // 2^120 * 5 * G++        .quad   0x82d2da754679c418+        .quad   0xe63bd7d8b2618df0+        .quad   0x355eef24ac47eb0a+        .quad   0x2078684c4833c6b4+        .quad   0x23d9533aad3902c9+        .quad   0x64c2ddceef03588f+        .quad   0x15257390cfe12fb4+        .quad   0x6c668b4d44e4d390+        .quad   0x3b48cf217a78820c+        .quad   0xf76a0ab281273e97+        .quad   0xa96c65a78c8eed7b+        .quad   0x7411a6054f8a433f++        // 2^120 * 6 * G++        .quad   0x4d659d32b99dc86d+        .quad   0x044cdc75603af115+        .quad   0xb34c712cdcc2e488+        .quad   0x7c136574fb8134ff+        .quad   0x579ae53d18b175b4+        .quad   0x68713159f392a102+        .quad   0x8455ecba1eef35f5+        .quad   0x1ec9a872458c398f+        .quad   0xb8e6a4d400a2509b+        .quad   0x9b81d7020bc882b4+        .quad   0x57e7cc9bf1957561+        .quad   0x3add88a5c7cd6460++        // 2^120 * 7 * G++        .quad   0xab895770b635dcf2+        .quad   0x02dfef6cf66c1fbc+        .quad   0x85530268beb6d187+        .quad   0x249929fccc879e74+        .quad   0x85c298d459393046+        .quad   0x8f7e35985ff659ec+        .quad   0x1d2ca22af2f66e3a+        .quad   0x61ba1131a406a720+        .quad   0xa3d0a0f116959029+        .quad   0x023b6b6cba7ebd89+        .quad   0x7bf15a3e26783307+        .quad   0x5620310cbbd8ece7++        // 2^120 * 8 * G++        .quad   0x528993434934d643+        .quad   0xb9dbf806a51222f5+        .quad   0x8f6d878fc3f41c22+        .quad   0x37676a2a4d9d9730+        .quad   0x6646b5f477e285d6+        .quad   0x40e8ff676c8f6193+        .quad   0xa6ec7311abb594dd+        .quad   0x7ec846f3658cec4d+        .quad   0x9b5e8f3f1da22ec7+        .quad   0x130f1d776c01cd13+        .quad   0x214c8fcfa2989fb8+        .quad   0x6daaf723399b9dd5++        // 2^124 * 1 * G++        .quad   0x591e4a5610628564+        .quad   0x2a4bb87ca8b4df34+        .quad   0xde2a2572e7a38e43+        .quad   0x3cbdabd9fee5046e+        .quad   0x81aebbdd2cd13070+        .quad   0x962e4325f85a0e9e+        .quad   0xde9391aacadffecb+        .quad   0x53177fda52c230e6+        .quad   0xa7bc970650b9de79+        .quad   0x3d12a7fbc301b59b+        .quad   0x02652e68d36ae38c+        .quad   0x79d739835a6199dc++        // 2^124 * 2 * G++        .quad   0xd9354df64131c1bd+        .quad   0x758094a186ec5822+        .quad   0x4464ee12e459f3c2+        .quad   0x6c11fce4cb133282+        .quad   0x21c9d9920d591737+        .quad   0x9bea41d2e9b46cd6+        .quad   0xe20e84200d89bfca+        .quad   0x79d99f946eae5ff8+        .quad   0xf17b483568673205+        .quad   0x387deae83caad96c+        .quad   0x61b471fd56ffe386+        .quad   0x31741195b745a599++        // 2^124 * 3 * G++        .quad   0xe8d10190b77a360b+        .quad   0x99b983209995e702+        .quad   0xbd4fdff8fa0247aa+        .quad   0x2772e344e0d36a87+        .quad   0x17f8ba683b02a047+        .quad   0x50212096feefb6c8+        .quad   0x70139be21556cbe2+        .quad   0x203e44a11d98915b+        .quad   0xd6863eba37b9e39f+        .quad   0x105bc169723b5a23+        .quad   0x104f6459a65c0762+        .quad   0x567951295b4d38d4++        // 2^124 * 4 * G++        .quad   0x535fd60613037524+        .quad   0xe210adf6b0fbc26a+        .quad   0xac8d0a9b23e990ae+        .quad   0x47204d08d72fdbf9+        .quad   0x07242eb30d4b497f+        .quad   0x1ef96306b9bccc87+        .quad   0x37950934d8116f45+        .quad   0x05468d6201405b04+        .quad   0x00f565a9f93267de+        .quad   0xcecfd78dc0d58e8a+        .quad   0xa215e2dcf318e28e+        .quad   0x4599ee919b633352++        // 2^124 * 5 * G++        .quad   0xd3c220ca70e0e76b+        .quad   0xb12bea58ea9f3094+        .quad   0x294ddec8c3271282+        .quad   0x0c3539e1a1d1d028+        .quad   0xac746d6b861ae579+        .quad   0x31ab0650f6aea9dc+        .quad   0x241d661140256d4c+        .quad   0x2f485e853d21a5de+        .quad   0x329744839c0833f3+        .quad   0x6fe6257fd2abc484+        .quad   0x5327d1814b358817+        .quad   0x65712585893fe9bc++        // 2^124 * 6 * G++        .quad   0x9c102fb732a61161+        .quad   0xe48e10dd34d520a8+        .quad   0x365c63546f9a9176+        .quad   0x32f6fe4c046f6006+        .quad   0x81c29f1bd708ee3f+        .quad   0xddcb5a05ae6407d0+        .quad   0x97aec1d7d2a3eba7+        .quad   0x1590521a91d50831+        .quad   0x40a3a11ec7910acc+        .quad   0x9013dff8f16d27ae+        .quad   0x1a9720d8abb195d4+        .quad   0x1bb9fe452ea98463++        // 2^124 * 7 * G++        .quad   0xe9d1d950b3d54f9e+        .quad   0x2d5f9cbee00d33c1+        .quad   0x51c2c656a04fc6ac+        .quad   0x65c091ee3c1cbcc9+        .quad   0xcf5e6c95cc36747c+        .quad   0x294201536b0bc30d+        .quad   0x453ac67cee797af0+        .quad   0x5eae6ab32a8bb3c9+        .quad   0x7083661114f118ea+        .quad   0x2b37b87b94349cad+        .quad   0x7273f51cb4e99f40+        .quad   0x78a2a95823d75698++        // 2^124 * 8 * G++        .quad   0xa2b072e95c8c2ace+        .quad   0x69cffc96651e9c4b+        .quad   0x44328ef842e7b42b+        .quad   0x5dd996c122aadeb3+        .quad   0xb4f23c425ef83207+        .quad   0xabf894d3c9a934b5+        .quad   0xd0708c1339fd87f7+        .quad   0x1876789117166130+        .quad   0x925b5ef0670c507c+        .quad   0x819bc842b93c33bf+        .quad   0x10792e9a70dd003f+        .quad   0x59ad4b7a6e28dc74++        // 2^128 * 1 * G++        .quad   0x5f3a7562eb3dbe47+        .quad   0xf7ea38548ebda0b8+        .quad   0x00c3e53145747299+        .quad   0x1304e9e71627d551+        .quad   0x583b04bfacad8ea2+        .quad   0x29b743e8148be884+        .quad   0x2b1e583b0810c5db+        .quad   0x2b5449e58eb3bbaa+        .quad   0x789814d26adc9cfe+        .quad   0x3c1bab3f8b48dd0b+        .quad   0xda0fe1fff979c60a+        .quad   0x4468de2d7c2dd693++        // 2^128 * 2 * G++        .quad   0x51bb355e9419469e+        .quad   0x33e6dc4c23ddc754+        .quad   0x93a5b6d6447f9962+        .quad   0x6cce7c6ffb44bd63+        .quad   0x4b9ad8c6f86307ce+        .quad   0x21113531435d0c28+        .quad   0xd4a866c5657a772c+        .quad   0x5da6427e63247352+        .quad   0x1a94c688deac22ca+        .quad   0xb9066ef7bbae1ff8+        .quad   0x88ad8c388d59580f+        .quad   0x58f29abfe79f2ca8++        // 2^128 * 3 * G++        .quad   0xe90ecfab8de73e68+        .quad   0x54036f9f377e76a5+        .quad   0xf0495b0bbe015982+        .quad   0x577629c4a7f41e36+        .quad   0x4b5a64bf710ecdf6+        .quad   0xb14ce538462c293c+        .quad   0x3643d056d50b3ab9+        .quad   0x6af93724185b4870+        .quad   0x3220024509c6a888+        .quad   0xd2e036134b558973+        .quad   0x83e236233c33289f+        .quad   0x701f25bb0caec18f++        // 2^128 * 4 * G++        .quad   0xc3a8b0f8e4616ced+        .quad   0xf700660e9e25a87d+        .quad   0x61e3061ff4bca59c+        .quad   0x2e0c92bfbdc40be9+        .quad   0x9d18f6d97cbec113+        .quad   0x844a06e674bfdbe4+        .quad   0x20f5b522ac4e60d6+        .quad   0x720a5bc050955e51+        .quad   0x0c3f09439b805a35+        .quad   0xe84e8b376242abfc+        .quad   0x691417f35c229346+        .quad   0x0e9b9cbb144ef0ec++        // 2^128 * 5 * G++        .quad   0xfbbad48ffb5720ad+        .quad   0xee81916bdbf90d0e+        .quad   0xd4813152635543bf+        .quad   0x221104eb3f337bd8+        .quad   0x8dee9bd55db1beee+        .quad   0xc9c3ab370a723fb9+        .quad   0x44a8f1bf1c68d791+        .quad   0x366d44191cfd3cde+        .quad   0x9e3c1743f2bc8c14+        .quad   0x2eda26fcb5856c3b+        .quad   0xccb82f0e68a7fb97+        .quad   0x4167a4e6bc593244++        // 2^128 * 6 * G++        .quad   0x643b9d2876f62700+        .quad   0x5d1d9d400e7668eb+        .quad   0x1b4b430321fc0684+        .quad   0x7938bb7e2255246a+        .quad   0xc2be2665f8ce8fee+        .quad   0xe967ff14e880d62c+        .quad   0xf12e6e7e2f364eee+        .quad   0x34b33370cb7ed2f6+        .quad   0xcdc591ee8681d6cc+        .quad   0xce02109ced85a753+        .quad   0xed7485c158808883+        .quad   0x1176fc6e2dfe65e4++        // 2^128 * 7 * G++        .quad   0xb4af6cd05b9c619b+        .quad   0x2ddfc9f4b2a58480+        .quad   0x3d4fa502ebe94dc4+        .quad   0x08fc3a4c677d5f34+        .quad   0xdb90e28949770eb8+        .quad   0x98fbcc2aacf440a3+        .quad   0x21354ffeded7879b+        .quad   0x1f6a3e54f26906b6+        .quad   0x60a4c199d30734ea+        .quad   0x40c085b631165cd6+        .quad   0xe2333e23f7598295+        .quad   0x4f2fad0116b900d1++        // 2^128 * 8 * G++        .quad   0x44beb24194ae4e54+        .quad   0x5f541c511857ef6c+        .quad   0xa61e6b2d368d0498+        .quad   0x445484a4972ef7ab+        .quad   0x962cd91db73bb638+        .quad   0xe60577aafc129c08+        .quad   0x6f619b39f3b61689+        .quad   0x3451995f2944ee81+        .quad   0x9152fcd09fea7d7c+        .quad   0x4a816c94b0935cf6+        .quad   0x258e9aaa47285c40+        .quad   0x10b89ca6042893b7++        // 2^132 * 1 * G++        .quad   0x9b2a426e3b646025+        .quad   0x32127190385ce4cf+        .quad   0xa25cffc2dd6dea45+        .quad   0x06409010bea8de75+        .quad   0xd67cded679d34aa0+        .quad   0xcc0b9ec0cc4db39f+        .quad   0xa535a456e35d190f+        .quad   0x2e05d9eaf61f6fef+        .quad   0xc447901ad61beb59+        .quad   0x661f19bce5dc880a+        .quad   0x24685482b7ca6827+        .quad   0x293c778cefe07f26++        // 2^132 * 2 * G++        .quad   0x86809e7007069096+        .quad   0xaad75b15e4e50189+        .quad   0x07f35715a21a0147+        .quad   0x0487f3f112815d5e+        .quad   0x16c795d6a11ff200+        .quad   0xcb70d0e2b15815c9+        .quad   0x89f293209b5395b5+        .quad   0x50b8c2d031e47b4f+        .quad   0x48350c08068a4962+        .quad   0x6ffdd05351092c9a+        .quad   0x17af4f4aaf6fc8dd+        .quad   0x4b0553b53cdba58b++        // 2^132 * 3 * G++        .quad   0x9c65fcbe1b32ff79+        .quad   0xeb75ea9f03b50f9b+        .quad   0xfced2a6c6c07e606+        .quad   0x35106cd551717908+        .quad   0xbf05211b27c152d4+        .quad   0x5ec26849bd1af639+        .quad   0x5e0b2caa8e6fab98+        .quad   0x054c8bdd50bd0840+        .quad   0x38a0b12f1dcf073d+        .quad   0x4b60a8a3b7f6a276+        .quad   0xfed5ac25d3404f9a+        .quad   0x72e82d5e5505c229++        // 2^132 * 4 * G++        .quad   0x6b0b697ff0d844c8+        .quad   0xbb12f85cd979cb49+        .quad   0xd2a541c6c1da0f1f+        .quad   0x7b7c242958ce7211+        .quad   0x00d9cdfd69771d02+        .quad   0x410276cd6cfbf17e+        .quad   0x4c45306c1cb12ec7+        .quad   0x2857bf1627500861+        .quad   0x9f21903f0101689e+        .quad   0xd779dfd3bf861005+        .quad   0xa122ee5f3deb0f1b+        .quad   0x510df84b485a00d4++        // 2^132 * 5 * G++        .quad   0xa54133bb9277a1fa+        .quad   0x74ec3b6263991237+        .quad   0x1a3c54dc35d2f15a+        .quad   0x2d347144e482ba3a+        .quad   0x24b3c887c70ac15e+        .quad   0xb0f3a557fb81b732+        .quad   0x9b2cde2fe578cc1b+        .quad   0x4cf7ed0703b54f8e+        .quad   0x6bd47c6598fbee0f+        .quad   0x9e4733e2ab55be2d+        .quad   0x1093f624127610c5+        .quad   0x4e05e26ad0a1eaa4++        // 2^132 * 6 * G++        .quad   0xda9b6b624b531f20+        .quad   0x429a760e77509abb+        .quad   0xdbe9f522e823cb80+        .quad   0x618f1856880c8f82+        .quad   0x1833c773e18fe6c0+        .quad   0xe3c4711ad3c87265+        .quad   0x3bfd3c4f0116b283+        .quad   0x1955875eb4cd4db8+        .quad   0x6da6de8f0e399799+        .quad   0x7ad61aa440fda178+        .quad   0xb32cd8105e3563dd+        .quad   0x15f6beae2ae340ae++        // 2^132 * 7 * G++        .quad   0x862bcb0c31ec3a62+        .quad   0x810e2b451138f3c2+        .quad   0x788ec4b839dac2a4+        .quad   0x28f76867ae2a9281+        .quad   0xba9a0f7b9245e215+        .quad   0xf368612dd98c0dbb+        .quad   0x2e84e4cbf220b020+        .quad   0x6ba92fe962d90eda+        .quad   0x3e4df9655884e2aa+        .quad   0xbd62fbdbdbd465a5+        .quad   0xd7596caa0de9e524+        .quad   0x6e8042ccb2b1b3d7++        // 2^132 * 8 * G++        .quad   0xf10d3c29ce28ca6e+        .quad   0xbad34540fcb6093d+        .quad   0xe7426ed7a2ea2d3f+        .quad   0x08af9d4e4ff298b9+        .quad   0x1530653616521f7e+        .quad   0x660d06b896203dba+        .quad   0x2d3989bc545f0879+        .quad   0x4b5303af78ebd7b0+        .quad   0x72f8a6c3bebcbde8+        .quad   0x4f0fca4adc3a8e89+        .quad   0x6fa9d4e8c7bfdf7a+        .quad   0x0dcf2d679b624eb7++        // 2^136 * 1 * G++        .quad   0x3d5947499718289c+        .quad   0x12ebf8c524533f26+        .quad   0x0262bfcb14c3ef15+        .quad   0x20b878d577b7518e+        .quad   0x753941be5a45f06e+        .quad   0xd07caeed6d9c5f65+        .quad   0x11776b9c72ff51b6+        .quad   0x17d2d1d9ef0d4da9+        .quad   0x27f2af18073f3e6a+        .quad   0xfd3fe519d7521069+        .quad   0x22e3b72c3ca60022+        .quad   0x72214f63cc65c6a7++        // 2^136 * 2 * G++        .quad   0xb4e37f405307a693+        .quad   0xaba714d72f336795+        .quad   0xd6fbd0a773761099+        .quad   0x5fdf48c58171cbc9+        .quad   0x1d9db7b9f43b29c9+        .quad   0xd605824a4f518f75+        .quad   0xf2c072bd312f9dc4+        .quad   0x1f24ac855a1545b0+        .quad   0x24d608328e9505aa+        .quad   0x4748c1d10c1420ee+        .quad   0xc7ffe45c06fb25a2+        .quad   0x00ba739e2ae395e6++        // 2^136 * 3 * G++        .quad   0x592e98de5c8790d6+        .quad   0xe5bfb7d345c2a2df+        .quad   0x115a3b60f9b49922+        .quad   0x03283a3e67ad78f3+        .quad   0xae4426f5ea88bb26+        .quad   0x360679d984973bfb+        .quad   0x5c9f030c26694e50+        .quad   0x72297de7d518d226+        .quad   0x48241dc7be0cb939+        .quad   0x32f19b4d8b633080+        .quad   0xd3dfc90d02289308+        .quad   0x05e1296846271945++        // 2^136 * 4 * G++        .quad   0xba82eeb32d9c495a+        .quad   0xceefc8fcf12bb97c+        .quad   0xb02dabae93b5d1e0+        .quad   0x39c00c9c13698d9b+        .quad   0xadbfbbc8242c4550+        .quad   0xbcc80cecd03081d9+        .quad   0x843566a6f5c8df92+        .quad   0x78cf25d38258ce4c+        .quad   0x15ae6b8e31489d68+        .quad   0xaa851cab9c2bf087+        .quad   0xc9a75a97f04efa05+        .quad   0x006b52076b3ff832++        // 2^136 * 5 * G++        .quad   0x29e0cfe19d95781c+        .quad   0xb681df18966310e2+        .quad   0x57df39d370516b39+        .quad   0x4d57e3443bc76122+        .quad   0xf5cb7e16b9ce082d+        .quad   0x3407f14c417abc29+        .quad   0xd4b36bce2bf4a7ab+        .quad   0x7de2e9561a9f75ce+        .quad   0xde70d4f4b6a55ecb+        .quad   0x4801527f5d85db99+        .quad   0xdbc9c440d3ee9a81+        .quad   0x6b2a90af1a6029ed++        // 2^136 * 6 * G++        .quad   0x6923f4fc9ae61e97+        .quad   0x5735281de03f5fd1+        .quad   0xa764ae43e6edd12d+        .quad   0x5fd8f4e9d12d3e4a+        .quad   0x77ebf3245bb2d80a+        .quad   0xd8301b472fb9079b+        .quad   0xc647e6f24cee7333+        .quad   0x465812c8276c2109+        .quad   0x4d43beb22a1062d9+        .quad   0x7065fb753831dc16+        .quad   0x180d4a7bde2968d7+        .quad   0x05b32c2b1cb16790++        // 2^136 * 7 * G++        .quad   0xc8c05eccd24da8fd+        .quad   0xa1cf1aac05dfef83+        .quad   0xdbbeeff27df9cd61+        .quad   0x3b5556a37b471e99+        .quad   0xf7fca42c7ad58195+        .quad   0x3214286e4333f3cc+        .quad   0xb6c29d0d340b979d+        .quad   0x31771a48567307e1+        .quad   0x32b0c524e14dd482+        .quad   0xedb351541a2ba4b6+        .quad   0xa3d16048282b5af3+        .quad   0x4fc079d27a7336eb++        // 2^136 * 8 * G++        .quad   0x51c938b089bf2f7f+        .quad   0x2497bd6502dfe9a7+        .quad   0xffffc09c7880e453+        .quad   0x124567cecaf98e92+        .quad   0xdc348b440c86c50d+        .quad   0x1337cbc9cc94e651+        .quad   0x6422f74d643e3cb9+        .quad   0x241170c2bae3cd08+        .quad   0x3ff9ab860ac473b4+        .quad   0xf0911dee0113e435+        .quad   0x4ae75060ebc6c4af+        .quad   0x3f8612966c87000d++        // 2^140 * 1 * G++        .quad   0x0c9c5303f7957be4+        .quad   0xa3c31a20e085c145+        .quad   0xb0721d71d0850050+        .quad   0x0aba390eab0bf2da+        .quad   0x529fdffe638c7bf3+        .quad   0xdf2b9e60388b4995+        .quad   0xe027b34f1bad0249+        .quad   0x7bc92fc9b9fa74ed+        .quad   0x9f97ef2e801ad9f9+        .quad   0x83697d5479afda3a+        .quad   0xe906b3ffbd596b50+        .quad   0x02672b37dd3fb8e0++        // 2^140 * 2 * G++        .quad   0x48b2ca8b260885e4+        .quad   0xa4286bec82b34c1c+        .quad   0x937e1a2617f58f74+        .quad   0x741d1fcbab2ca2a5+        .quad   0xee9ba729398ca7f5+        .quad   0xeb9ca6257a4849db+        .quad   0x29eb29ce7ec544e1+        .quad   0x232ca21ef736e2c8+        .quad   0xbf61423d253fcb17+        .quad   0x08803ceafa39eb14+        .quad   0xf18602df9851c7af+        .quad   0x0400f3a049e3414b++        // 2^140 * 3 * G++        .quad   0xabce0476ba61c55b+        .quad   0x36a3d6d7c4d39716+        .quad   0x6eb259d5e8d82d09+        .quad   0x0c9176e984d756fb+        .quad   0x2efba412a06e7b06+        .quad   0x146785452c8d2560+        .quad   0xdf9713ebd67a91c7+        .quad   0x32830ac7157eadf3+        .quad   0x0e782a7ab73769e8+        .quad   0x04a05d7875b18e2c+        .quad   0x29525226ebcceae1+        .quad   0x0d794f8383eba820++        // 2^140 * 4 * G++        .quad   0xff35f5cb9e1516f4+        .quad   0xee805bcf648aae45+        .quad   0xf0d73c2bb93a9ef3+        .quad   0x097b0bf22092a6c2+        .quad   0x7be44ce7a7a2e1ac+        .quad   0x411fd93efad1b8b7+        .quad   0x1734a1d70d5f7c9b+        .quad   0x0d6592233127db16+        .quad   0xc48bab1521a9d733+        .quad   0xa6c2eaead61abb25+        .quad   0x625c6c1cc6cb4305+        .quad   0x7fc90fea93eb3a67++        // 2^140 * 5 * G++        .quad   0x0408f1fe1f5c5926+        .quad   0x1a8f2f5e3b258bf4+        .quad   0x40a951a2fdc71669+        .quad   0x6598ee93c98b577e+        .quad   0xc527deb59c7cb23d+        .quad   0x955391695328404e+        .quad   0xd64392817ccf2c7a+        .quad   0x6ce97dabf7d8fa11+        .quad   0x25b5a8e50ef7c48f+        .quad   0xeb6034116f2ce532+        .quad   0xc5e75173e53de537+        .quad   0x73119fa08c12bb03++        // 2^140 * 6 * G++        .quad   0xed30129453f1a4cb+        .quad   0xbce621c9c8f53787+        .quad   0xfacb2b1338bee7b9+        .quad   0x3025798a9ea8428c+        .quad   0x7845b94d21f4774d+        .quad   0xbf62f16c7897b727+        .quad   0x671857c03c56522b+        .quad   0x3cd6a85295621212+        .quad   0x3fecde923aeca999+        .quad   0xbdaa5b0062e8c12f+        .quad   0x67b99dfc96988ade+        .quad   0x3f52c02852661036++        // 2^140 * 7 * G++        .quad   0xffeaa48e2a1351c6+        .quad   0x28624754fa7f53d7+        .quad   0x0b5ba9e57582ddf1+        .quad   0x60c0104ba696ac59+        .quad   0x9258bf99eec416c6+        .quad   0xac8a5017a9d2f671+        .quad   0x629549ab16dea4ab+        .quad   0x05d0e85c99091569+        .quad   0x051de020de9cbe97+        .quad   0xfa07fc56b50bcf74+        .quad   0x378cec9f0f11df65+        .quad   0x36853c69ab96de4d++        // 2^140 * 8 * G++        .quad   0x36d9b8de78f39b2d+        .quad   0x7f42ed71a847b9ec+        .quad   0x241cd1d679bd3fde+        .quad   0x6a704fec92fbce6b+        .quad   0x4433c0b0fac5e7be+        .quad   0x724bae854c08dcbe+        .quad   0xf1f24cc446978f9b+        .quad   0x4a0aff6d62825fc8+        .quad   0xe917fb9e61095301+        .quad   0xc102df9402a092f8+        .quad   0xbf09e2f5fa66190b+        .quad   0x681109bee0dcfe37++        // 2^144 * 1 * G++        .quad   0x559a0cc9782a0dde+        .quad   0x551dcdb2ea718385+        .quad   0x7f62865b31ef238c+        .quad   0x504aa7767973613d+        .quad   0x9c18fcfa36048d13+        .quad   0x29159db373899ddd+        .quad   0xdc9f350b9f92d0aa+        .quad   0x26f57eee878a19d4+        .quad   0x0cab2cd55687efb1+        .quad   0x5180d162247af17b+        .quad   0x85c15a344f5a2467+        .quad   0x4041943d9dba3069++        // 2^144 * 2 * G++        .quad   0xc3c0eeba43ebcc96+        .quad   0x8d749c9c26ea9caf+        .quad   0xd9fa95ee1c77ccc6+        .quad   0x1420a1d97684340f+        .quad   0x4b217743a26caadd+        .quad   0x47a6b424648ab7ce+        .quad   0xcb1d4f7a03fbc9e3+        .quad   0x12d931429800d019+        .quad   0x00c67799d337594f+        .quad   0x5e3c5140b23aa47b+        .quad   0x44182854e35ff395+        .quad   0x1b4f92314359a012++        // 2^144 * 3 * G++        .quad   0x3e5c109d89150951+        .quad   0x39cefa912de9696a+        .quad   0x20eae43f975f3020+        .quad   0x239b572a7f132dae+        .quad   0x33cf3030a49866b1+        .quad   0x251f73d2215f4859+        .quad   0xab82aa4051def4f6+        .quad   0x5ff191d56f9a23f6+        .quad   0x819ed433ac2d9068+        .quad   0x2883ab795fc98523+        .quad   0xef4572805593eb3d+        .quad   0x020c526a758f36cb++        // 2^144 * 4 * G++        .quad   0x779834f89ed8dbbc+        .quad   0xc8f2aaf9dc7ca46c+        .quad   0xa9524cdca3e1b074+        .quad   0x02aacc4615313877+        .quad   0xe931ef59f042cc89+        .quad   0x2c589c9d8e124bb6+        .quad   0xadc8e18aaec75997+        .quad   0x452cfe0a5602c50c+        .quad   0x86a0f7a0647877df+        .quad   0xbbc464270e607c9f+        .quad   0xab17ea25f1fb11c9+        .quad   0x4cfb7d7b304b877b++        // 2^144 * 5 * G++        .quad   0x72b43d6cb89b75fe+        .quad   0x54c694d99c6adc80+        .quad   0xb8c3aa373ee34c9f+        .quad   0x14b4622b39075364+        .quad   0xe28699c29789ef12+        .quad   0x2b6ecd71df57190d+        .quad   0xc343c857ecc970d0+        .quad   0x5b1d4cbc434d3ac5+        .quad   0xb6fb2615cc0a9f26+        .quad   0x3a4f0e2bb88dcce5+        .quad   0x1301498b3369a705+        .quad   0x2f98f71258592dd1++        // 2^144 * 6 * G++        .quad   0x0c94a74cb50f9e56+        .quad   0x5b1ff4a98e8e1320+        .quad   0x9a2acc2182300f67+        .quad   0x3a6ae249d806aaf9+        .quad   0x2e12ae444f54a701+        .quad   0xfcfe3ef0a9cbd7de+        .quad   0xcebf890d75835de0+        .quad   0x1d8062e9e7614554+        .quad   0x657ada85a9907c5a+        .quad   0x1a0ea8b591b90f62+        .quad   0x8d0e1dfbdf34b4e9+        .quad   0x298b8ce8aef25ff3++        // 2^144 * 7 * G++        .quad   0x2a927953eff70cb2+        .quad   0x4b89c92a79157076+        .quad   0x9418457a30a7cf6a+        .quad   0x34b8a8404d5ce485+        .quad   0x837a72ea0a2165de+        .quad   0x3fab07b40bcf79f6+        .quad   0x521636c77738ae70+        .quad   0x6ba6271803a7d7dc+        .quad   0xc26eecb583693335+        .quad   0xd5a813df63b5fefd+        .quad   0xa293aa9aa4b22573+        .quad   0x71d62bdd465e1c6a++        // 2^144 * 8 * G++        .quad   0x6533cc28d378df80+        .quad   0xf6db43790a0fa4b4+        .quad   0xe3645ff9f701da5a+        .quad   0x74d5f317f3172ba4+        .quad   0xcd2db5dab1f75ef5+        .quad   0xd77f95cf16b065f5+        .quad   0x14571fea3f49f085+        .quad   0x1c333621262b2b3d+        .quad   0xa86fe55467d9ca81+        .quad   0x398b7c752b298c37+        .quad   0xda6d0892e3ac623b+        .quad   0x4aebcc4547e9d98c++        // 2^148 * 1 * G++        .quad   0x53175a7205d21a77+        .quad   0xb0c04422d3b934d4+        .quad   0xadd9f24bdd5deadc+        .quad   0x074f46e69f10ff8c+        .quad   0x0de9b204a059a445+        .quad   0xe15cb4aa4b17ad0f+        .quad   0xe1bbec521f79c557+        .quad   0x2633f1b9d071081b+        .quad   0xc1fb4177018b9910+        .quad   0xa6ea20dc6c0fe140+        .quad   0xd661f3e74354c6ff+        .quad   0x5ecb72e6f1a3407a++        // 2^148 * 2 * G++        .quad   0xa515a31b2259fb4e+        .quad   0x0960f3972bcac52f+        .quad   0xedb52fec8d3454cb+        .quad   0x382e2720c476c019+        .quad   0xfeeae106e8e86997+        .quad   0x9863337f98d09383+        .quad   0x9470480eaa06ebef+        .quad   0x038b6898d4c5c2d0+        .quad   0xf391c51d8ace50a6+        .quad   0x3142d0b9ae2d2948+        .quad   0xdb4d5a1a7f24ca80+        .quad   0x21aeba8b59250ea8++        // 2^148 * 3 * G++        .quad   0x24f13b34cf405530+        .quad   0x3c44ea4a43088af7+        .quad   0x5dd5c5170006a482+        .quad   0x118eb8f8890b086d+        .quad   0x53853600f0087f23+        .quad   0x4c461879da7d5784+        .quad   0x6af303deb41f6860+        .quad   0x0a3c16c5c27c18ed+        .quad   0x17e49c17cc947f3d+        .quad   0xccc6eda6aac1d27b+        .quad   0xdf6092ceb0f08e56+        .quad   0x4909b3e22c67c36b++        // 2^148 * 4 * G++        .quad   0x9c9c85ea63fe2e89+        .quad   0xbe1baf910e9412ec+        .quad   0x8f7baa8a86fbfe7b+        .quad   0x0fb17f9fef968b6c+        .quad   0x59a16676706ff64e+        .quad   0x10b953dd0d86a53d+        .quad   0x5848e1e6ce5c0b96+        .quad   0x2d8b78e712780c68+        .quad   0x79d5c62eafc3902b+        .quad   0x773a215289e80728+        .quad   0xc38ae640e10120b9+        .quad   0x09ae23717b2b1a6d++        // 2^148 * 5 * G++        .quad   0xbb6a192a4e4d083c+        .quad   0x34ace0630029e192+        .quad   0x98245a59aafabaeb+        .quad   0x6d9c8a9ada97faac+        .quad   0x10ab8fa1ad32b1d0+        .quad   0xe9aced1be2778b24+        .quad   0xa8856bc0373de90f+        .quad   0x66f35ddddda53996+        .quad   0xd27d9afb24997323+        .quad   0x1bb7e07ef6f01d2e+        .quad   0x2ba7472df52ecc7f+        .quad   0x03019b4f646f9dc8++        // 2^148 * 6 * G++        .quad   0x04a186b5565345cd+        .quad   0xeee76610bcc4116a+        .quad   0x689c73b478fb2a45+        .quad   0x387dcbff65697512+        .quad   0xaf09b214e6b3dc6b+        .quad   0x3f7573b5ad7d2f65+        .quad   0xd019d988100a23b0+        .quad   0x392b63a58b5c35f7+        .quad   0x4093addc9c07c205+        .quad   0xc565be15f532c37e+        .quad   0x63dbecfd1583402a+        .quad   0x61722b4aef2e032e++        // 2^148 * 7 * G++        .quad   0x0012aafeecbd47af+        .quad   0x55a266fb1cd46309+        .quad   0xf203eb680967c72c+        .quad   0x39633944ca3c1429+        .quad   0xd6b07a5581cb0e3c+        .quad   0x290ff006d9444969+        .quad   0x08680b6a16dcda1f+        .quad   0x5568d2b75a06de59+        .quad   0x8d0cb88c1b37cfe1+        .quad   0x05b6a5a3053818f3+        .quad   0xf2e9bc04b787d959+        .quad   0x6beba1249add7f64++        // 2^148 * 8 * G++        .quad   0x1d06005ca5b1b143+        .quad   0x6d4c6bb87fd1cda2+        .quad   0x6ef5967653fcffe7+        .quad   0x097c29e8c1ce1ea5+        .quad   0x5c3cecb943f5a53b+        .quad   0x9cc9a61d06c08df2+        .quad   0xcfba639a85895447+        .quad   0x5a845ae80df09fd5+        .quad   0x4ce97dbe5deb94ca+        .quad   0x38d0a4388c709c48+        .quad   0xc43eced4a169d097+        .quad   0x0a1249fff7e587c3++        // 2^152 * 1 * G++        .quad   0x12f0071b276d01c9+        .quad   0xe7b8bac586c48c70+        .quad   0x5308129b71d6fba9+        .quad   0x5d88fbf95a3db792+        .quad   0x0b408d9e7354b610+        .quad   0x806b32535ba85b6e+        .quad   0xdbe63a034a58a207+        .quad   0x173bd9ddc9a1df2c+        .quad   0x2b500f1efe5872df+        .quad   0x58d6582ed43918c1+        .quad   0xe6ed278ec9673ae0+        .quad   0x06e1cd13b19ea319++        // 2^152 * 2 * G++        .quad   0x40d0ad516f166f23+        .quad   0x118e32931fab6abe+        .quad   0x3fe35e14a04d088e+        .quad   0x3080603526e16266+        .quad   0x472baf629e5b0353+        .quad   0x3baa0b90278d0447+        .quad   0x0c785f469643bf27+        .quad   0x7f3a6a1a8d837b13+        .quad   0xf7e644395d3d800b+        .quad   0x95a8d555c901edf6+        .quad   0x68cd7830592c6339+        .quad   0x30d0fded2e51307e++        // 2^152 * 3 * G++        .quad   0xe0594d1af21233b3+        .quad   0x1bdbe78ef0cc4d9c+        .quad   0x6965187f8f499a77+        .quad   0x0a9214202c099868+        .quad   0x9cb4971e68b84750+        .quad   0xa09572296664bbcf+        .quad   0x5c8de72672fa412b+        .quad   0x4615084351c589d9+        .quad   0xbc9019c0aeb9a02e+        .quad   0x55c7110d16034cae+        .quad   0x0e6df501659932ec+        .quad   0x3bca0d2895ca5dfe++        // 2^152 * 4 * G++        .quad   0x40f031bc3c5d62a4+        .quad   0x19fc8b3ecff07a60+        .quad   0x98183da2130fb545+        .quad   0x5631deddae8f13cd+        .quad   0x9c688eb69ecc01bf+        .quad   0xf0bc83ada644896f+        .quad   0xca2d955f5f7a9fe2+        .quad   0x4ea8b4038df28241+        .quad   0x2aed460af1cad202+        .quad   0x46305305a48cee83+        .quad   0x9121774549f11a5f+        .quad   0x24ce0930542ca463++        // 2^152 * 5 * G++        .quad   0x1fe890f5fd06c106+        .quad   0xb5c468355d8810f2+        .quad   0x827808fe6e8caf3e+        .quad   0x41d4e3c28a06d74b+        .quad   0x3fcfa155fdf30b85+        .quad   0xd2f7168e36372ea4+        .quad   0xb2e064de6492f844+        .quad   0x549928a7324f4280+        .quad   0xf26e32a763ee1a2e+        .quad   0xae91e4b7d25ffdea+        .quad   0xbc3bd33bd17f4d69+        .quad   0x491b66dec0dcff6a++        // 2^152 * 6 * G++        .quad   0x98f5b13dc7ea32a7+        .quad   0xe3d5f8cc7e16db98+        .quad   0xac0abf52cbf8d947+        .quad   0x08f338d0c85ee4ac+        .quad   0x75f04a8ed0da64a1+        .quad   0xed222caf67e2284b+        .quad   0x8234a3791f7b7ba4+        .quad   0x4cf6b8b0b7018b67+        .quad   0xc383a821991a73bd+        .quad   0xab27bc01df320c7a+        .quad   0xc13d331b84777063+        .quad   0x530d4a82eb078a99++        // 2^152 * 7 * G++        .quad   0x004c3630e1f94825+        .quad   0x7e2d78268cab535a+        .quad   0xc7482323cc84ff8b+        .quad   0x65ea753f101770b9+        .quad   0x6d6973456c9abf9e+        .quad   0x257fb2fc4900a880+        .quad   0x2bacf412c8cfb850+        .quad   0x0db3e7e00cbfbd5b+        .quad   0x3d66fc3ee2096363+        .quad   0x81d62c7f61b5cb6b+        .quad   0x0fbe044213443b1a+        .quad   0x02a4ec1921e1a1db++        // 2^152 * 8 * G++        .quad   0x5ce6259a3b24b8a2+        .quad   0xb8577acc45afa0b8+        .quad   0xcccbe6e88ba07037+        .quad   0x3d143c51127809bf+        .quad   0xf5c86162f1cf795f+        .quad   0x118c861926ee57f2+        .quad   0x172124851c063578+        .quad   0x36d12b5dec067fcf+        .quad   0x126d279179154557+        .quad   0xd5e48f5cfc783a0a+        .quad   0x36bdb6e8df179bac+        .quad   0x2ef517885ba82859++        // 2^156 * 1 * G++        .quad   0x88bd438cd11e0d4a+        .quad   0x30cb610d43ccf308+        .quad   0xe09a0e3791937bcc+        .quad   0x4559135b25b1720c+        .quad   0x1ea436837c6da1e9+        .quad   0xf9c189af1fb9bdbe+        .quad   0x303001fcce5dd155+        .quad   0x28a7c99ebc57be52+        .quad   0xb8fd9399e8d19e9d+        .quad   0x908191cb962423ff+        .quad   0xb2b948d747c742a3+        .quad   0x37f33226d7fb44c4++        // 2^156 * 2 * G++        .quad   0x0dae8767b55f6e08+        .quad   0x4a43b3b35b203a02+        .quad   0xe3725a6e80af8c79+        .quad   0x0f7a7fd1705fa7a3+        .quad   0x33912553c821b11d+        .quad   0x66ed42c241e301df+        .quad   0x066fcc11104222fd+        .quad   0x307a3b41c192168f+        .quad   0x8eeb5d076eb55ce0+        .quad   0x2fc536bfaa0d925a+        .quad   0xbe81830fdcb6c6e8+        .quad   0x556c7045827baf52++        // 2^156 * 3 * G++        .quad   0x8e2b517302e9d8b7+        .quad   0xe3e52269248714e8+        .quad   0xbd4fbd774ca960b5+        .quad   0x6f4b4199c5ecada9+        .quad   0xb94b90022bf44406+        .quad   0xabd4237eff90b534+        .quad   0x7600a960faf86d3a+        .quad   0x2f45abdac2322ee3+        .quad   0x61af4912c8ef8a6a+        .quad   0xe58fa4fe43fb6e5e+        .quad   0xb5afcc5d6fd427cf+        .quad   0x6a5393281e1e11eb++        // 2^156 * 4 * G++        .quad   0xf3da5139a5d1ee89+        .quad   0x8145457cff936988+        .quad   0x3f622fed00e188c4+        .quad   0x0f513815db8b5a3d+        .quad   0x0fff04fe149443cf+        .quad   0x53cac6d9865cddd7+        .quad   0x31385b03531ed1b7+        .quad   0x5846a27cacd1039d+        .quad   0x4ff5cdac1eb08717+        .quad   0x67e8b29590f2e9bc+        .quad   0x44093b5e237afa99+        .quad   0x0d414bed8708b8b2++        // 2^156 * 5 * G++        .quad   0xcfb68265fd0e75f6+        .quad   0xe45b3e28bb90e707+        .quad   0x7242a8de9ff92c7a+        .quad   0x685b3201933202dd+        .quad   0x81886a92294ac9e8+        .quad   0x23162b45d55547be+        .quad   0x94cfbc4403715983+        .quad   0x50eb8fdb134bc401+        .quad   0xc0b73ec6d6b330cd+        .quad   0x84e44807132faff1+        .quad   0x732b7352c4a5dee1+        .quad   0x5d7c7cf1aa7cd2d2++        // 2^156 * 6 * G++        .quad   0xaf3b46bf7a4aafa2+        .quad   0xb78705ec4d40d411+        .quad   0x114f0c6aca7c15e3+        .quad   0x3f364faaa9489d4d+        .quad   0x33d1013e9b73a562+        .quad   0x925cef5748ec26e1+        .quad   0xa7fce614dd468058+        .quad   0x78b0fad41e9aa438+        .quad   0xbf56a431ed05b488+        .quad   0xa533e66c9c495c7e+        .quad   0xe8652baf87f3651a+        .quad   0x0241800059d66c33++        // 2^156 * 7 * G++        .quad   0xceb077fea37a5be4+        .quad   0xdb642f02e5a5eeb7+        .quad   0xc2e6d0c5471270b8+        .quad   0x4771b65538e4529c+        .quad   0x28350c7dcf38ea01+        .quad   0x7c6cdbc0b2917ab6+        .quad   0xace7cfbe857082f7+        .quad   0x4d2845aba2d9a1e0+        .quad   0xbb537fe0447070de+        .quad   0xcba744436dd557df+        .quad   0xd3b5a3473600dbcb+        .quad   0x4aeabbe6f9ffd7f8++        // 2^156 * 8 * G++        .quad   0x4630119e40d8f78c+        .quad   0xa01a9bc53c710e11+        .quad   0x486d2b258910dd79+        .quad   0x1e6c47b3db0324e5+        .quad   0x6a2134bcc4a9c8f2+        .quad   0xfbf8fd1c8ace2e37+        .quad   0x000ae3049911a0ba+        .quad   0x046e3a616bc89b9e+        .quad   0x14e65442f03906be+        .quad   0x4a019d54e362be2a+        .quad   0x68ccdfec8dc230c7+        .quad   0x7cfb7e3faf6b861c++        // 2^160 * 1 * G++        .quad   0x4637974e8c58aedc+        .quad   0xb9ef22fbabf041a4+        .quad   0xe185d956e980718a+        .quad   0x2f1b78fab143a8a6+        .quad   0x96eebffb305b2f51+        .quad   0xd3f938ad889596b8+        .quad   0xf0f52dc746d5dd25+        .quad   0x57968290bb3a0095+        .quad   0xf71ab8430a20e101+        .quad   0xf393658d24f0ec47+        .quad   0xcf7509a86ee2eed1+        .quad   0x7dc43e35dc2aa3e1++        // 2^160 * 2 * G++        .quad   0x85966665887dd9c3+        .quad   0xc90f9b314bb05355+        .quad   0xc6e08df8ef2079b1+        .quad   0x7ef72016758cc12f+        .quad   0x5a782a5c273e9718+        .quad   0x3576c6995e4efd94+        .quad   0x0f2ed8051f237d3e+        .quad   0x044fb81d82d50a99+        .quad   0xc1df18c5a907e3d9+        .quad   0x57b3371dce4c6359+        .quad   0xca704534b201bb49+        .quad   0x7f79823f9c30dd2e++        // 2^160 * 3 * G++        .quad   0x8334d239a3b513e8+        .quad   0xc13670d4b91fa8d8+        .quad   0x12b54136f590bd33+        .quad   0x0a4e0373d784d9b4+        .quad   0x6a9c1ff068f587ba+        .quad   0x0827894e0050c8de+        .quad   0x3cbf99557ded5be7+        .quad   0x64a9b0431c06d6f0+        .quad   0x2eb3d6a15b7d2919+        .quad   0xb0b4f6a0d53a8235+        .quad   0x7156ce4389a45d47+        .quad   0x071a7d0ace18346c++        // 2^160 * 4 * G++        .quad   0xd3072daac887ba0b+        .quad   0x01262905bfa562ee+        .quad   0xcf543002c0ef768b+        .quad   0x2c3bcc7146ea7e9c+        .quad   0xcc0c355220e14431+        .quad   0x0d65950709b15141+        .quad   0x9af5621b209d5f36+        .quad   0x7c69bcf7617755d3+        .quad   0x07f0d7eb04e8295f+        .quad   0x10db18252f50f37d+        .quad   0xe951a9a3171798d7+        .quad   0x6f5a9a7322aca51d++        // 2^160 * 5 * G++        .quad   0x8ba1000c2f41c6c5+        .quad   0xc49f79c10cfefb9b+        .quad   0x4efa47703cc51c9f+        .quad   0x494e21a2e147afca+        .quad   0xe729d4eba3d944be+        .quad   0x8d9e09408078af9e+        .quad   0x4525567a47869c03+        .quad   0x02ab9680ee8d3b24+        .quad   0xefa48a85dde50d9a+        .quad   0x219a224e0fb9a249+        .quad   0xfa091f1dd91ef6d9+        .quad   0x6b5d76cbea46bb34++        // 2^160 * 6 * G++        .quad   0x8857556cec0cd994+        .quad   0x6472dc6f5cd01dba+        .quad   0xaf0169148f42b477+        .quad   0x0ae333f685277354+        .quad   0xe0f941171e782522+        .quad   0xf1e6ae74036936d3+        .quad   0x408b3ea2d0fcc746+        .quad   0x16fb869c03dd313e+        .quad   0x288e199733b60962+        .quad   0x24fc72b4d8abe133+        .quad   0x4811f7ed0991d03e+        .quad   0x3f81e38b8f70d075++        // 2^160 * 7 * G++        .quad   0x7f910fcc7ed9affe+        .quad   0x545cb8a12465874b+        .quad   0xa8397ed24b0c4704+        .quad   0x50510fc104f50993+        .quad   0x0adb7f355f17c824+        .quad   0x74b923c3d74299a4+        .quad   0xd57c3e8bcbf8eaf7+        .quad   0x0ad3e2d34cdedc3d+        .quad   0x6f0c0fc5336e249d+        .quad   0x745ede19c331cfd9+        .quad   0xf2d6fd0009eefe1c+        .quad   0x127c158bf0fa1ebe++        // 2^160 * 8 * G++        .quad   0xf6197c422e9879a2+        .quad   0xa44addd452ca3647+        .quad   0x9b413fc14b4eaccb+        .quad   0x354ef87d07ef4f68+        .quad   0xdea28fc4ae51b974+        .quad   0x1d9973d3744dfe96+        .quad   0x6240680b873848a8+        .quad   0x4ed82479d167df95+        .quad   0xfee3b52260c5d975+        .quad   0x50352efceb41b0b8+        .quad   0x8808ac30a9f6653c+        .quad   0x302d92d20539236d++        // 2^164 * 1 * G++        .quad   0x4c59023fcb3efb7c+        .quad   0x6c2fcb99c63c2a94+        .quad   0xba4190e2c3c7e084+        .quad   0x0e545daea51874d9+        .quad   0x957b8b8b0df53c30+        .quad   0x2a1c770a8e60f098+        .quad   0xbbc7a670345796de+        .quad   0x22a48f9a90c99bc9+        .quad   0x6b7dc0dc8d3fac58+        .quad   0x5497cd6ce6e42bfd+        .quad   0x542f7d1bf400d305+        .quad   0x4159f47f048d9136++        // 2^164 * 2 * G++        .quad   0x20ad660839e31e32+        .quad   0xf81e1bd58405be50+        .quad   0xf8064056f4dabc69+        .quad   0x14d23dd4ce71b975+        .quad   0x748515a8bbd24839+        .quad   0x77128347afb02b55+        .quad   0x50ba2ac649a2a17f+        .quad   0x060525513ad730f1+        .quad   0xf2398e098aa27f82+        .quad   0x6d7982bb89a1b024+        .quad   0xfa694084214dd24c+        .quad   0x71ab966fa32301c3++        // 2^164 * 3 * G++        .quad   0x2dcbd8e34ded02fc+        .quad   0x1151f3ec596f22aa+        .quad   0xbca255434e0328da+        .quad   0x35768fbe92411b22+        .quad   0xb1088a0702809955+        .quad   0x43b273ea0b43c391+        .quad   0xca9b67aefe0686ed+        .quad   0x605eecbf8335f4ed+        .quad   0x83200a656c340431+        .quad   0x9fcd71678ee59c2f+        .quad   0x75d4613f71300f8a+        .quad   0x7a912faf60f542f9++        // 2^164 * 4 * G++        .quad   0xb204585e5edc1a43+        .quad   0x9f0e16ee5897c73c+        .quad   0x5b82c0ae4e70483c+        .quad   0x624a170e2bddf9be+        .quad   0x253f4f8dfa2d5597+        .quad   0x25e49c405477130c+        .quad   0x00c052e5996b1102+        .quad   0x33cb966e33bb6c4a+        .quad   0x597028047f116909+        .quad   0x828ac41c1e564467+        .quad   0x70417dbde6217387+        .quad   0x721627aefbac4384++        // 2^164 * 5 * G++        .quad   0x97d03bc38736add5+        .quad   0x2f1422afc532b130+        .quad   0x3aa68a057101bbc4+        .quad   0x4c946cf7e74f9fa7+        .quad   0xfd3097bc410b2f22+        .quad   0xf1a05da7b5cfa844+        .quad   0x61289a1def57ca74+        .quad   0x245ea199bb821902+        .quad   0xaedca66978d477f8+        .quad   0x1898ba3c29117fe1+        .quad   0xcf73f983720cbd58+        .quad   0x67da12e6b8b56351++        // 2^164 * 6 * G++        .quad   0x7067e187b4bd6e07+        .quad   0x6e8f0203c7d1fe74+        .quad   0x93c6aa2f38c85a30+        .quad   0x76297d1f3d75a78a+        .quad   0x2b7ef3d38ec8308c+        .quad   0x828fd7ec71eb94ab+        .quad   0x807c3b36c5062abd+        .quad   0x0cb64cb831a94141+        .quad   0x3030fc33534c6378+        .quad   0xb9635c5ce541e861+        .quad   0x15d9a9bed9b2c728+        .quad   0x49233ea3f3775dcb++        // 2^164 * 7 * G++        .quad   0x629398fa8dbffc3a+        .quad   0xe12fe52dd54db455+        .quad   0xf3be11dfdaf25295+        .quad   0x628b140dce5e7b51+        .quad   0x7b3985fe1c9f249b+        .quad   0x4fd6b2d5a1233293+        .quad   0xceb345941adf4d62+        .quad   0x6987ff6f542de50c+        .quad   0x47e241428f83753c+        .quad   0x6317bebc866af997+        .quad   0xdabb5b433d1a9829+        .quad   0x074d8d245287fb2d++        // 2^164 * 8 * G++        .quad   0x8337d9cd440bfc31+        .quad   0x729d2ca1af318fd7+        .quad   0xa040a4a4772c2070+        .quad   0x46002ef03a7349be+        .quad   0x481875c6c0e31488+        .quad   0x219429b2e22034b4+        .quad   0x7223c98a31283b65+        .quad   0x3420d60b342277f9+        .quad   0xfaa23adeaffe65f7+        .quad   0x78261ed45be0764c+        .quad   0x441c0a1e2f164403+        .quad   0x5aea8e567a87d395++        // 2^168 * 1 * G++        .quad   0x7813c1a2bca4283d+        .quad   0xed62f091a1863dd9+        .quad   0xaec7bcb8c268fa86+        .quad   0x10e5d3b76f1cae4c+        .quad   0x2dbc6fb6e4e0f177+        .quad   0x04e1bf29a4bd6a93+        .quad   0x5e1966d4787af6e8+        .quad   0x0edc5f5eb426d060+        .quad   0x5453bfd653da8e67+        .quad   0xe9dc1eec24a9f641+        .quad   0xbf87263b03578a23+        .quad   0x45b46c51361cba72++        // 2^168 * 2 * G++        .quad   0xa9402abf314f7fa1+        .quad   0xe257f1dc8e8cf450+        .quad   0x1dbbd54b23a8be84+        .quad   0x2177bfa36dcb713b+        .quad   0xce9d4ddd8a7fe3e4+        .quad   0xab13645676620e30+        .quad   0x4b594f7bb30e9958+        .quad   0x5c1c0aef321229df+        .quad   0x37081bbcfa79db8f+        .quad   0x6048811ec25f59b3+        .quad   0x087a76659c832487+        .quad   0x4ae619387d8ab5bb++        // 2^168 * 3 * G++        .quad   0x8ddbf6aa5344a32e+        .quad   0x7d88eab4b41b4078+        .quad   0x5eb0eb974a130d60+        .quad   0x1a00d91b17bf3e03+        .quad   0x61117e44985bfb83+        .quad   0xfce0462a71963136+        .quad   0x83ac3448d425904b+        .quad   0x75685abe5ba43d64+        .quad   0x6e960933eb61f2b2+        .quad   0x543d0fa8c9ff4952+        .quad   0xdf7275107af66569+        .quad   0x135529b623b0e6aa++        // 2^168 * 4 * G++        .quad   0x18f0dbd7add1d518+        .quad   0x979f7888cfc11f11+        .quad   0x8732e1f07114759b+        .quad   0x79b5b81a65ca3a01+        .quad   0xf5c716bce22e83fe+        .quad   0xb42beb19e80985c1+        .quad   0xec9da63714254aae+        .quad   0x5972ea051590a613+        .quad   0x0fd4ac20dc8f7811+        .quad   0x9a9ad294ac4d4fa8+        .quad   0xc01b2d64b3360434+        .quad   0x4f7e9c95905f3bdb++        // 2^168 * 5 * G++        .quad   0x62674bbc5781302e+        .quad   0xd8520f3989addc0f+        .quad   0x8c2999ae53fbd9c6+        .quad   0x31993ad92e638e4c+        .quad   0x71c8443d355299fe+        .quad   0x8bcd3b1cdbebead7+        .quad   0x8092499ef1a49466+        .quad   0x1942eec4a144adc8+        .quad   0x7dac5319ae234992+        .quad   0x2c1b3d910cea3e92+        .quad   0x553ce494253c1122+        .quad   0x2a0a65314ef9ca75++        // 2^168 * 6 * G++        .quad   0x2db7937ff7f927c2+        .quad   0xdb741f0617d0a635+        .quad   0x5982f3a21155af76+        .quad   0x4cf6e218647c2ded+        .quad   0xcf361acd3c1c793a+        .quad   0x2f9ebcac5a35bc3b+        .quad   0x60e860e9a8cda6ab+        .quad   0x055dc39b6dea1a13+        .quad   0xb119227cc28d5bb6+        .quad   0x07e24ebc774dffab+        .quad   0xa83c78cee4a32c89+        .quad   0x121a307710aa24b6++        // 2^168 * 7 * G++        .quad   0xe4db5d5e9f034a97+        .quad   0xe153fc093034bc2d+        .quad   0x460546919551d3b1+        .quad   0x333fc76c7a40e52d+        .quad   0xd659713ec77483c9+        .quad   0x88bfe077b82b96af+        .quad   0x289e28231097bcd3+        .quad   0x527bb94a6ced3a9b+        .quad   0x563d992a995b482e+        .quad   0x3405d07c6e383801+        .quad   0x485035de2f64d8e5+        .quad   0x6b89069b20a7a9f7++        // 2^168 * 8 * G++        .quad   0x812aa0416270220d+        .quad   0x995a89faf9245b4e+        .quad   0xffadc4ce5072ef05+        .quad   0x23bc2103aa73eb73+        .quad   0x4082fa8cb5c7db77+        .quad   0x068686f8c734c155+        .quad   0x29e6c8d9f6e7a57e+        .quad   0x0473d308a7639bcf+        .quad   0xcaee792603589e05+        .quad   0x2b4b421246dcc492+        .quad   0x02a1ef74e601a94f+        .quad   0x102f73bfde04341a++        // 2^172 * 1 * G++        .quad   0xb5a2d50c7ec20d3e+        .quad   0xc64bdd6ea0c97263+        .quad   0x56e89052c1ff734d+        .quad   0x4929c6f72b2ffaba+        .quad   0x358ecba293a36247+        .quad   0xaf8f9862b268fd65+        .quad   0x412f7e9968a01c89+        .quad   0x5786f312cd754524+        .quad   0x337788ffca14032c+        .quad   0xf3921028447f1ee3+        .quad   0x8b14071f231bccad+        .quad   0x4c817b4bf2344783++        // 2^172 * 2 * G++        .quad   0x0ff853852871b96e+        .quad   0xe13e9fab60c3f1bb+        .quad   0xeefd595325344402+        .quad   0x0a37c37075b7744b+        .quad   0x413ba057a40b4484+        .quad   0xba4c2e1a4f5f6a43+        .quad   0x614ba0a5aee1d61c+        .quad   0x78a1531a8b05dc53+        .quad   0x6cbdf1703ad0562b+        .quad   0x8ecf4830c92521a3+        .quad   0xdaebd303fd8424e7+        .quad   0x72ad82a42e5ec56f++        // 2^172 * 3 * G++        .quad   0x3f9e8e35bafb65f6+        .quad   0x39d69ec8f27293a1+        .quad   0x6cb8cd958cf6a3d0+        .quad   0x1734778173adae6d+        .quad   0xc368939167024bc3+        .quad   0x8e69d16d49502fda+        .quad   0xfcf2ec3ce45f4b29+        .quad   0x065f669ea3b4cbc4+        .quad   0x8a00aec75532db4d+        .quad   0xb869a4e443e31bb1+        .quad   0x4a0f8552d3a7f515+        .quad   0x19adeb7c303d7c08++        // 2^172 * 4 * G++        .quad   0xc720cb6153ead9a3+        .quad   0x55b2c97f512b636e+        .quad   0xb1e35b5fd40290b1+        .quad   0x2fd9ccf13b530ee2+        .quad   0x9d05ba7d43c31794+        .quad   0x2470c8ff93322526+        .quad   0x8323dec816197438+        .quad   0x2852709881569b53+        .quad   0x07bd475b47f796b8+        .quad   0xd2c7b013542c8f54+        .quad   0x2dbd23f43b24f87e+        .quad   0x6551afd77b0901d6++        // 2^172 * 5 * G++        .quad   0x4546baaf54aac27f+        .quad   0xf6f66fecb2a45a28+        .quad   0x582d1b5b562bcfe8+        .quad   0x44b123f3920f785f+        .quad   0x68a24ce3a1d5c9ac+        .quad   0xbb77a33d10ff6461+        .quad   0x0f86ce4425d3166e+        .quad   0x56507c0950b9623b+        .quad   0x1206f0b7d1713e63+        .quad   0x353fe3d915bafc74+        .quad   0x194ceb970ad9d94d+        .quad   0x62fadd7cf9d03ad3++        // 2^172 * 6 * G++        .quad   0xc6b5967b5598a074+        .quad   0x5efe91ce8e493e25+        .quad   0xd4b72c4549280888+        .quad   0x20ef1149a26740c2+        .quad   0x3cd7bc61e7ce4594+        .quad   0xcd6b35a9b7dd267e+        .quad   0xa080abc84366ef27+        .quad   0x6ec7c46f59c79711+        .quad   0x2f07ad636f09a8a2+        .quad   0x8697e6ce24205e7d+        .quad   0xc0aefc05ee35a139+        .quad   0x15e80958b5f9d897++        // 2^172 * 7 * G++        .quad   0x25a5ef7d0c3e235b+        .quad   0x6c39c17fbe134ee7+        .quad   0xc774e1342dc5c327+        .quad   0x021354b892021f39+        .quad   0x4dd1ed355bb061c4+        .quad   0x42dc0cef941c0700+        .quad   0x61305dc1fd86340e+        .quad   0x56b2cc930e55a443+        .quad   0x1df79da6a6bfc5a2+        .quad   0x02f3a2749fde4369+        .quad   0xb323d9f2cda390a7+        .quad   0x7be0847b8774d363++        // 2^172 * 8 * G++        .quad   0x8c99cc5a8b3f55c3+        .quad   0x0611d7253fded2a0+        .quad   0xed2995ff36b70a36+        .quad   0x1f699a54d78a2619+        .quad   0x1466f5af5307fa11+        .quad   0x817fcc7ded6c0af2+        .quad   0x0a6de44ec3a4a3fb+        .quad   0x74071475bc927d0b+        .quad   0xe77292f373e7ea8a+        .quad   0x296537d2cb045a31+        .quad   0x1bd0653ed3274fde+        .quad   0x2f9a2c4476bd2966++        // 2^176 * 1 * G++        .quad   0xeb18b9ab7f5745c6+        .quad   0x023a8aee5787c690+        .quad   0xb72712da2df7afa9+        .quad   0x36597d25ea5c013d+        .quad   0xa2b4dae0b5511c9a+        .quad   0x7ac860292bffff06+        .quad   0x981f375df5504234+        .quad   0x3f6bd725da4ea12d+        .quad   0x734d8d7b106058ac+        .quad   0xd940579e6fc6905f+        .quad   0x6466f8f99202932d+        .quad   0x7b7ecc19da60d6d0++        // 2^176 * 2 * G++        .quad   0x78c2373c695c690d+        .quad   0xdd252e660642906e+        .quad   0x951d44444ae12bd2+        .quad   0x4235ad7601743956+        .quad   0x6dae4a51a77cfa9b+        .quad   0x82263654e7a38650+        .quad   0x09bbffcd8f2d82db+        .quad   0x03bedc661bf5caba+        .quad   0x6258cb0d078975f5+        .quad   0x492942549189f298+        .quad   0xa0cab423e2e36ee4+        .quad   0x0e7ce2b0cdf066a1++        // 2^176 * 3 * G++        .quad   0xc494643ac48c85a3+        .quad   0xfd361df43c6139ad+        .quad   0x09db17dd3ae94d48+        .quad   0x666e0a5d8fb4674a+        .quad   0xfea6fedfd94b70f9+        .quad   0xf130c051c1fcba2d+        .quad   0x4882d47e7f2fab89+        .quad   0x615256138aeceeb5+        .quad   0x2abbf64e4870cb0d+        .quad   0xcd65bcf0aa458b6b+        .quad   0x9abe4eba75e8985d+        .quad   0x7f0bc810d514dee4++        // 2^176 * 4 * G++        .quad   0xb9006ba426f4136f+        .quad   0x8d67369e57e03035+        .quad   0xcbc8dfd94f463c28+        .quad   0x0d1f8dbcf8eedbf5+        .quad   0x83ac9dad737213a0+        .quad   0x9ff6f8ba2ef72e98+        .quad   0x311e2edd43ec6957+        .quad   0x1d3a907ddec5ab75+        .quad   0xba1693313ed081dc+        .quad   0x29329fad851b3480+        .quad   0x0128013c030321cb+        .quad   0x00011b44a31bfde3++        // 2^176 * 5 * G++        .quad   0x3fdfa06c3fc66c0c+        .quad   0x5d40e38e4dd60dd2+        .quad   0x7ae38b38268e4d71+        .quad   0x3ac48d916e8357e1+        .quad   0x16561f696a0aa75c+        .quad   0xc1bf725c5852bd6a+        .quad   0x11a8dd7f9a7966ad+        .quad   0x63d988a2d2851026+        .quad   0x00120753afbd232e+        .quad   0xe92bceb8fdd8f683+        .quad   0xf81669b384e72b91+        .quad   0x33fad52b2368a066++        // 2^176 * 6 * G++        .quad   0x540649c6c5e41e16+        .quad   0x0af86430333f7735+        .quad   0xb2acfcd2f305e746+        .quad   0x16c0f429a256dca7+        .quad   0x8d2cc8d0c422cfe8+        .quad   0x072b4f7b05a13acb+        .quad   0xa3feb6e6ecf6a56f+        .quad   0x3cc355ccb90a71e2+        .quad   0xe9b69443903e9131+        .quad   0xb8a494cb7a5637ce+        .quad   0xc87cd1a4baba9244+        .quad   0x631eaf426bae7568++        // 2^176 * 7 * G++        .quad   0xb3e90410da66fe9f+        .quad   0x85dd4b526c16e5a6+        .quad   0xbc3d97611ef9bf83+        .quad   0x5599648b1ea919b5+        .quad   0x47d975b9a3700de8+        .quad   0x7280c5fbe2f80552+        .quad   0x53658f2732e45de1+        .quad   0x431f2c7f665f80b5+        .quad   0xd6026344858f7b19+        .quad   0x14ab352fa1ea514a+        .quad   0x8900441a2090a9d7+        .quad   0x7b04715f91253b26++        // 2^176 * 8 * G++        .quad   0x83edbd28acf6ae43+        .quad   0x86357c8b7d5c7ab4+        .quad   0xc0404769b7eb2c44+        .quad   0x59b37bf5c2f6583f+        .quad   0xb376c280c4e6bac6+        .quad   0x970ed3dd6d1d9b0b+        .quad   0xb09a9558450bf944+        .quad   0x48d0acfa57cde223+        .quad   0xb60f26e47dabe671+        .quad   0xf1d1a197622f3a37+        .quad   0x4208ce7ee9960394+        .quad   0x16234191336d3bdb++        // 2^180 * 1 * G++        .quad   0xf19aeac733a63aef+        .quad   0x2c7fba5d4442454e+        .quad   0x5da87aa04795e441+        .quad   0x413051e1a4e0b0f5+        .quad   0x852dd1fd3d578bbe+        .quad   0x2b65ce72c3286108+        .quad   0x658c07f4eace2273+        .quad   0x0933f804ec38ab40+        .quad   0xa7ab69798d496476+        .quad   0x8121aadefcb5abc8+        .quad   0xa5dc12ef7b539472+        .quad   0x07fd47065e45351a++        // 2^180 * 2 * G++        .quad   0xc8583c3d258d2bcd+        .quad   0x17029a4daf60b73f+        .quad   0xfa0fc9d6416a3781+        .quad   0x1c1e5fba38b3fb23+        .quad   0x304211559ae8e7c3+        .quad   0xf281b229944882a5+        .quad   0x8a13ac2e378250e4+        .quad   0x014afa0954ba48f4+        .quad   0xcb3197001bb3666c+        .quad   0x330060524bffecb9+        .quad   0x293711991a88233c+        .quad   0x291884363d4ed364++        // 2^180 * 3 * G++        .quad   0x033c6805dc4babfa+        .quad   0x2c15bf5e5596ecc1+        .quad   0x1bc70624b59b1d3b+        .quad   0x3ede9850a19f0ec5+        .quad   0xfb9d37c3bc1ab6eb+        .quad   0x02be14534d57a240+        .quad   0xf4d73415f8a5e1f6+        .quad   0x5964f4300ccc8188+        .quad   0xe44a23152d096800+        .quad   0x5c08c55970866996+        .quad   0xdf2db60a46affb6e+        .quad   0x579155c1f856fd89++        // 2^180 * 4 * G++        .quad   0x96324edd12e0c9ef+        .quad   0x468b878df2420297+        .quad   0x199a3776a4f573be+        .quad   0x1e7fbcf18e91e92a+        .quad   0xb5f16b630817e7a6+        .quad   0x808c69233c351026+        .quad   0x324a983b54cef201+        .quad   0x53c092084a485345+        .quad   0xd2d41481f1cbafbf+        .quad   0x231d2db6716174e5+        .quad   0x0b7d7656e2a55c98+        .quad   0x3e955cd82aa495f6++        // 2^180 * 5 * G++        .quad   0xe48f535e3ed15433+        .quad   0xd075692a0d7270a3+        .quad   0x40fbd21daade6387+        .quad   0x14264887cf4495f5+        .quad   0xab39f3ef61bb3a3f+        .quad   0x8eb400652eb9193e+        .quad   0xb5de6ecc38c11f74+        .quad   0x654d7e9626f3c49f+        .quad   0xe564cfdd5c7d2ceb+        .quad   0x82eeafded737ccb9+        .quad   0x6107db62d1f9b0ab+        .quad   0x0b6baac3b4358dbb++        // 2^180 * 6 * G++        .quad   0x7ae62bcb8622fe98+        .quad   0x47762256ceb891af+        .quad   0x1a5a92bcf2e406b4+        .quad   0x7d29401784e41501+        .quad   0x204abad63700a93b+        .quad   0xbe0023d3da779373+        .quad   0xd85f0346633ab709+        .quad   0x00496dc490820412+        .quad   0x1c74b88dc27e6360+        .quad   0x074854268d14850c+        .quad   0xa145fb7b3e0dcb30+        .quad   0x10843f1b43803b23++        // 2^180 * 7 * G++        .quad   0xc5f90455376276dd+        .quad   0xce59158dd7645cd9+        .quad   0x92f65d511d366b39+        .quad   0x11574b6e526996c4+        .quad   0xd56f672de324689b+        .quad   0xd1da8aedb394a981+        .quad   0xdd7b58fe9168cfed+        .quad   0x7ce246cd4d56c1e8+        .quad   0xb8f4308e7f80be53+        .quad   0x5f3cb8cb34a9d397+        .quad   0x18a961bd33cc2b2c+        .quad   0x710045fb3a9af671++        // 2^180 * 8 * G++        .quad   0x73f93d36101b95eb+        .quad   0xfaef33794f6f4486+        .quad   0x5651735f8f15e562+        .quad   0x7fa3f19058b40da1+        .quad   0xa03fc862059d699e+        .quad   0x2370cfa19a619e69+        .quad   0xc4fe3b122f823deb+        .quad   0x1d1b056fa7f0844e+        .quad   0x1bc64631e56bf61f+        .quad   0xd379ab106e5382a3+        .quad   0x4d58c57e0540168d+        .quad   0x566256628442d8e4++        // 2^184 * 1 * G++        .quad   0xb9e499def6267ff6+        .quad   0x7772ca7b742c0843+        .quad   0x23a0153fe9a4f2b1+        .quad   0x2cdfdfecd5d05006+        .quad   0xdd499cd61ff38640+        .quad   0x29cd9bc3063625a0+        .quad   0x51e2d8023dd73dc3+        .quad   0x4a25707a203b9231+        .quad   0x2ab7668a53f6ed6a+        .quad   0x304242581dd170a1+        .quad   0x4000144c3ae20161+        .quad   0x5721896d248e49fc++        // 2^184 * 2 * G++        .quad   0x0b6e5517fd181bae+        .quad   0x9022629f2bb963b4+        .quad   0x5509bce932064625+        .quad   0x578edd74f63c13da+        .quad   0x285d5091a1d0da4e+        .quad   0x4baa6fa7b5fe3e08+        .quad   0x63e5177ce19393b3+        .quad   0x03c935afc4b030fd+        .quad   0x997276c6492b0c3d+        .quad   0x47ccc2c4dfe205fc+        .quad   0xdcd29b84dd623a3c+        .quad   0x3ec2ab590288c7a2++        // 2^184 * 3 * G++        .quad   0xa1a0d27be4d87bb9+        .quad   0xa98b4deb61391aed+        .quad   0x99a0ddd073cb9b83+        .quad   0x2dd5c25a200fcace+        .quad   0xa7213a09ae32d1cb+        .quad   0x0f2b87df40f5c2d5+        .quad   0x0baea4c6e81eab29+        .quad   0x0e1bf66c6adbac5e+        .quad   0xe2abd5e9792c887e+        .quad   0x1a020018cb926d5d+        .quad   0xbfba69cdbaae5f1e+        .quad   0x730548b35ae88f5f++        // 2^184 * 4 * G++        .quad   0xc43551a3cba8b8ee+        .quad   0x65a26f1db2115f16+        .quad   0x760f4f52ab8c3850+        .quad   0x3043443b411db8ca+        .quad   0x805b094ba1d6e334+        .quad   0xbf3ef17709353f19+        .quad   0x423f06cb0622702b+        .quad   0x585a2277d87845dd+        .quad   0xa18a5f8233d48962+        .quad   0x6698c4b5ec78257f+        .quad   0xa78e6fa5373e41ff+        .quad   0x7656278950ef981f++        // 2^184 * 5 * G++        .quad   0x38c3cf59d51fc8c0+        .quad   0x9bedd2fd0506b6f2+        .quad   0x26bf109fab570e8f+        .quad   0x3f4160a8c1b846a6+        .quad   0xe17073a3ea86cf9d+        .quad   0x3a8cfbb707155fdc+        .quad   0x4853e7fc31838a8e+        .quad   0x28bbf484b613f616+        .quad   0xf2612f5c6f136c7c+        .quad   0xafead107f6dd11be+        .quad   0x527e9ad213de6f33+        .quad   0x1e79cb358188f75d++        // 2^184 * 6 * G++        .quad   0x013436c3eef7e3f1+        .quad   0x828b6a7ffe9e10f8+        .quad   0x7ff908e5bcf9defc+        .quad   0x65d7951b3a3b3831+        .quad   0x77e953d8f5e08181+        .quad   0x84a50c44299dded9+        .quad   0xdc6c2d0c864525e5+        .quad   0x478ab52d39d1f2f4+        .quad   0x66a6a4d39252d159+        .quad   0xe5dde1bc871ac807+        .quad   0xb82c6b40a6c1c96f+        .quad   0x16d87a411a212214++        // 2^184 * 7 * G++        .quad   0xb3bd7e5a42066215+        .quad   0x879be3cd0c5a24c1+        .quad   0x57c05db1d6f994b7+        .quad   0x28f87c8165f38ca6+        .quad   0xfba4d5e2d54e0583+        .quad   0xe21fafd72ebd99fa+        .quad   0x497ac2736ee9778f+        .quad   0x1f990b577a5a6dde+        .quad   0xa3344ead1be8f7d6+        .quad   0x7d1e50ebacea798f+        .quad   0x77c6569e520de052+        .quad   0x45882fe1534d6d3e++        // 2^184 * 8 * G++        .quad   0x6669345d757983d6+        .quad   0x62b6ed1117aa11a6+        .quad   0x7ddd1857985e128f+        .quad   0x688fe5b8f626f6dd+        .quad   0xd8ac9929943c6fe4+        .quad   0xb5f9f161a38392a2+        .quad   0x2699db13bec89af3+        .quad   0x7dcf843ce405f074+        .quad   0x6c90d6484a4732c0+        .quad   0xd52143fdca563299+        .quad   0xb3be28c3915dc6e1+        .quad   0x6739687e7327191b++        // 2^188 * 1 * G++        .quad   0x9f65c5ea200814cf+        .quad   0x840536e169a31740+        .quad   0x8b0ed13925c8b4ad+        .quad   0x0080dbafe936361d+        .quad   0x8ce5aad0c9cb971f+        .quad   0x1156aaa99fd54a29+        .quad   0x41f7247015af9b78+        .quad   0x1fe8cca8420f49aa+        .quad   0x72a1848f3c0cc82a+        .quad   0x38c560c2877c9e54+        .quad   0x5004e228ce554140+        .quad   0x042418a103429d71++        // 2^188 * 2 * G++        .quad   0x899dea51abf3ff5f+        .quad   0x9b93a8672fc2d8ba+        .quad   0x2c38cb97be6ebd5c+        .quad   0x114d578497263b5d+        .quad   0x58e84c6f20816247+        .quad   0x8db2b2b6e36fd793+        .quad   0x977182561d484d85+        .quad   0x0822024f8632abd7+        .quad   0xb301bb7c6b1beca3+        .quad   0x55393f6dc6eb1375+        .quad   0x910d281097b6e4eb+        .quad   0x1ad4548d9d479ea3++        // 2^188 * 3 * G++        .quad   0xcd5a7da0389a48fd+        .quad   0xb38fa4aa9a78371e+        .quad   0xc6d9761b2cdb8e6c+        .quad   0x35cf51dbc97e1443+        .quad   0xa06fe66d0fe9fed3+        .quad   0xa8733a401c587909+        .quad   0x30d14d800df98953+        .quad   0x41ce5876c7b30258+        .quad   0x59ac3bc5d670c022+        .quad   0xeae67c109b119406+        .quad   0x9798bdf0b3782fda+        .quad   0x651e3201fd074092++        // 2^188 * 4 * G++        .quad   0xd63d8483ef30c5cf+        .quad   0x4cd4b4962361cc0c+        .quad   0xee90e500a48426ac+        .quad   0x0af51d7d18c14eeb+        .quad   0xa57ba4a01efcae9e+        .quad   0x769f4beedc308a94+        .quad   0xd1f10eeb3603cb2e+        .quad   0x4099ce5e7e441278+        .quad   0x1ac98e4f8a5121e9+        .quad   0x7dae9544dbfa2fe0+        .quad   0x8320aa0dd6430df9+        .quad   0x667282652c4a2fb5++        // 2^188 * 5 * G++        .quad   0x874621f4d86bc9ab+        .quad   0xb54c7bbe56fe6fea+        .quad   0x077a24257fadc22c+        .quad   0x1ab53be419b90d39+        .quad   0xada8b6e02946db23+        .quad   0x1c0ce51a7b253ab7+        .quad   0x8448c85a66dd485b+        .quad   0x7f1fc025d0675adf+        .quad   0xd8ee1b18319ea6aa+        .quad   0x004d88083a21f0da+        .quad   0x3bd6aa1d883a4f4b+        .quad   0x4db9a3a6dfd9fd14++        // 2^188 * 6 * G++        .quad   0x8ce7b23bb99c0755+        .quad   0x35c5d6edc4f50f7a+        .quad   0x7e1e2ed2ed9b50c3+        .quad   0x36305f16e8934da1+        .quad   0xd95b00bbcbb77c68+        .quad   0xddbc846a91f17849+        .quad   0x7cf700aebe28d9b3+        .quad   0x5ce1285c85d31f3e+        .quad   0x31b6972d98b0bde8+        .quad   0x7d920706aca6de5b+        .quad   0xe67310f8908a659f+        .quad   0x50fac2a6efdf0235++        // 2^188 * 7 * G++        .quad   0xf3d3a9f35b880f5a+        .quad   0xedec050cdb03e7c2+        .quad   0xa896981ff9f0b1a2+        .quad   0x49a4ae2bac5e34a4+        .quad   0x295b1c86f6f449bc+        .quad   0x51b2e84a1f0ab4dd+        .quad   0xc001cb30aa8e551d+        .quad   0x6a28d35944f43662+        .quad   0x28bb12ee04a740e0+        .quad   0x14313bbd9bce8174+        .quad   0x72f5b5e4e8c10c40+        .quad   0x7cbfb19936adcd5b++        // 2^188 * 8 * G++        .quad   0xa311ddc26b89792d+        .quad   0x1b30b4c6da512664+        .quad   0x0ca77b4ccf150859+        .quad   0x1de443df1b009408+        .quad   0x8e793a7acc36e6e0+        .quad   0xf9fab7a37d586eed+        .quad   0x3a4f9692bae1f4e4+        .quad   0x1c14b03eff5f447e+        .quad   0x19647bd114a85291+        .quad   0x57b76cb21034d3af+        .quad   0x6329db440f9d6dfa+        .quad   0x5ef43e586a571493++        // 2^192 * 1 * G++        .quad   0xef782014385675a6+        .quad   0xa2649f30aafda9e8+        .quad   0x4cd1eb505cdfa8cb+        .quad   0x46115aba1d4dc0b3+        .quad   0xa66dcc9dc80c1ac0+        .quad   0x97a05cf41b38a436+        .quad   0xa7ebf3be95dbd7c6+        .quad   0x7da0b8f68d7e7dab+        .quad   0xd40f1953c3b5da76+        .quad   0x1dac6f7321119e9b+        .quad   0x03cc6021feb25960+        .quad   0x5a5f887e83674b4b++        // 2^192 * 2 * G++        .quad   0x8f6301cf70a13d11+        .quad   0xcfceb815350dd0c4+        .quad   0xf70297d4a4bca47e+        .quad   0x3669b656e44d1434+        .quad   0x9e9628d3a0a643b9+        .quad   0xb5c3cb00e6c32064+        .quad   0x9b5302897c2dec32+        .quad   0x43e37ae2d5d1c70c+        .quad   0x387e3f06eda6e133+        .quad   0x67301d5199a13ac0+        .quad   0xbd5ad8f836263811+        .quad   0x6a21e6cd4fd5e9be++        // 2^192 * 3 * G++        .quad   0xf1c6170a3046e65f+        .quad   0x58712a2a00d23524+        .quad   0x69dbbd3c8c82b755+        .quad   0x586bf9f1a195ff57+        .quad   0xef4129126699b2e3+        .quad   0x71d30847708d1301+        .quad   0x325432d01182b0bd+        .quad   0x45371b07001e8b36+        .quad   0xa6db088d5ef8790b+        .quad   0x5278f0dc610937e5+        .quad   0xac0349d261a16eb8+        .quad   0x0eafb03790e52179++        // 2^192 * 4 * G++        .quad   0x960555c13748042f+        .quad   0x219a41e6820baa11+        .quad   0x1c81f73873486d0c+        .quad   0x309acc675a02c661+        .quad   0x5140805e0f75ae1d+        .quad   0xec02fbe32662cc30+        .quad   0x2cebdf1eea92396d+        .quad   0x44ae3344c5435bb3+        .quad   0x9cf289b9bba543ee+        .quad   0xf3760e9d5ac97142+        .quad   0x1d82e5c64f9360aa+        .quad   0x62d5221b7f94678f++        // 2^192 * 5 * G++        .quad   0x524c299c18d0936d+        .quad   0xc86bb56c8a0c1a0c+        .quad   0xa375052edb4a8631+        .quad   0x5c0efde4bc754562+        .quad   0x7585d4263af77a3c+        .quad   0xdfae7b11fee9144d+        .quad   0xa506708059f7193d+        .quad   0x14f29a5383922037+        .quad   0xdf717edc25b2d7f5+        .quad   0x21f970db99b53040+        .quad   0xda9234b7c3ed4c62+        .quad   0x5e72365c7bee093e++        // 2^192 * 6 * G++        .quad   0x575bfc074571217f+        .quad   0x3779675d0694d95b+        .quad   0x9a0a37bbf4191e33+        .quad   0x77f1104c47b4eabc+        .quad   0x7d9339062f08b33e+        .quad   0x5b9659e5df9f32be+        .quad   0xacff3dad1f9ebdfd+        .quad   0x70b20555cb7349b7+        .quad   0xbe5113c555112c4c+        .quad   0x6688423a9a881fcd+        .quad   0x446677855e503b47+        .quad   0x0e34398f4a06404a++        // 2^192 * 7 * G++        .quad   0xb67d22d93ecebde8+        .quad   0x09b3e84127822f07+        .quad   0x743fa61fb05b6d8d+        .quad   0x5e5405368a362372+        .quad   0x18930b093e4b1928+        .quad   0x7de3e10e73f3f640+        .quad   0xf43217da73395d6f+        .quad   0x6f8aded6ca379c3e+        .quad   0xe340123dfdb7b29a+        .quad   0x487b97e1a21ab291+        .quad   0xf9967d02fde6949e+        .quad   0x780de72ec8d3de97++        // 2^192 * 8 * G++        .quad   0x0ae28545089ae7bc+        .quad   0x388ddecf1c7f4d06+        .quad   0x38ac15510a4811b8+        .quad   0x0eb28bf671928ce4+        .quad   0x671feaf300f42772+        .quad   0x8f72eb2a2a8c41aa+        .quad   0x29a17fd797373292+        .quad   0x1defc6ad32b587a6+        .quad   0xaf5bbe1aef5195a7+        .quad   0x148c1277917b15ed+        .quad   0x2991f7fb7ae5da2e+        .quad   0x467d201bf8dd2867++        // 2^196 * 1 * G++        .quad   0x7906ee72f7bd2e6b+        .quad   0x05d270d6109abf4e+        .quad   0x8d5cfe45b941a8a4+        .quad   0x44c218671c974287+        .quad   0x745f9d56296bc318+        .quad   0x993580d4d8152e65+        .quad   0xb0e5b13f5839e9ce+        .quad   0x51fc2b28d43921c0+        .quad   0x1b8fd11795e2a98c+        .quad   0x1c4e5ee12b6b6291+        .quad   0x5b30e7107424b572+        .quad   0x6e6b9de84c4f4ac6++        // 2^196 * 2 * G++        .quad   0xdff25fce4b1de151+        .quad   0xd841c0c7e11c4025+        .quad   0x2554b3c854749c87+        .quad   0x2d292459908e0df9+        .quad   0x6b7c5f10f80cb088+        .quad   0x736b54dc56e42151+        .quad   0xc2b620a5c6ef99c4+        .quad   0x5f4c802cc3a06f42+        .quad   0x9b65c8f17d0752da+        .quad   0x881ce338c77ee800+        .quad   0xc3b514f05b62f9e3+        .quad   0x66ed5dd5bec10d48++        // 2^196 * 3 * G++        .quad   0x7d38a1c20bb2089d+        .quad   0x808334e196ccd412+        .quad   0xc4a70b8c6c97d313+        .quad   0x2eacf8bc03007f20+        .quad   0xf0adf3c9cbca047d+        .quad   0x81c3b2cbf4552f6b+        .quad   0xcfda112d44735f93+        .quad   0x1f23a0c77e20048c+        .quad   0xf235467be5bc1570+        .quad   0x03d2d9020dbab38c+        .quad   0x27529aa2fcf9e09e+        .quad   0x0840bef29d34bc50++        // 2^196 * 4 * G++        .quad   0x796dfb35dc10b287+        .quad   0x27176bcd5c7ff29d+        .quad   0x7f3d43e8c7b24905+        .quad   0x0304f5a191c54276+        .quad   0xcd54e06b7f37e4eb+        .quad   0x8cc15f87f5e96cca+        .quad   0xb8248bb0d3597dce+        .quad   0x246affa06074400c+        .quad   0x37d88e68fbe45321+        .quad   0x86097548c0d75032+        .quad   0x4e9b13ef894a0d35+        .quad   0x25a83cac5753d325++        // 2^196 * 5 * G++        .quad   0x10222f48eed8165e+        .quad   0x623fc1234b8bcf3a+        .quad   0x1e145c09c221e8f0+        .quad   0x7ccfa59fca782630+        .quad   0x9f0f66293952b6e2+        .quad   0x33db5e0e0934267b+        .quad   0xff45252bd609fedc+        .quad   0x06be10f5c506e0c9+        .quad   0x1a9615a9b62a345f+        .quad   0x22050c564a52fecc+        .quad   0xa7a2788528bc0dfe+        .quad   0x5e82770a1a1ee71d++        // 2^196 * 6 * G++        .quad   0x35425183ad896a5c+        .quad   0xe8673afbe78d52f6+        .quad   0x2c66f25f92a35f64+        .quad   0x09d04f3b3b86b102+        .quad   0xe802e80a42339c74+        .quad   0x34175166a7fffae5+        .quad   0x34865d1f1c408cae+        .quad   0x2cca982c605bc5ee+        .quad   0xfd2d5d35197dbe6e+        .quad   0x207c2eea8be4ffa3+        .quad   0x2613d8db325ae918+        .quad   0x7a325d1727741d3e++        // 2^196 * 7 * G++        .quad   0xd036b9bbd16dfde2+        .quad   0xa2055757c497a829+        .quad   0x8e6cc966a7f12667+        .quad   0x4d3b1a791239c180+        .quad   0xecd27d017e2a076a+        .quad   0xd788689f1636495e+        .quad   0x52a61af0919233e5+        .quad   0x2a479df17bb1ae64+        .quad   0x9e5eee8e33db2710+        .quad   0x189854ded6c43ca5+        .quad   0xa41c22c592718138+        .quad   0x27ad5538a43a5e9b++        // 2^196 * 8 * G++        .quad   0x2746dd4b15350d61+        .quad   0xd03fcbc8ee9521b7+        .quad   0xe86e365a138672ca+        .quad   0x510e987f7e7d89e2+        .quad   0xcb5a7d638e47077c+        .quad   0x8db7536120a1c059+        .quad   0x549e1e4d8bedfdcc+        .quad   0x080153b7503b179d+        .quad   0xdda69d930a3ed3e3+        .quad   0x3d386ef1cd60a722+        .quad   0xc817ad58bdaa4ee6+        .quad   0x23be8d554fe7372a++        // 2^200 * 1 * G++        .quad   0x95fe919a74ef4fad+        .quad   0x3a827becf6a308a2+        .quad   0x964e01d309a47b01+        .quad   0x71c43c4f5ba3c797+        .quad   0xbc1ef4bd567ae7a9+        .quad   0x3f624cb2d64498bd+        .quad   0xe41064d22c1f4ec8+        .quad   0x2ef9c5a5ba384001+        .quad   0xb6fd6df6fa9e74cd+        .quad   0xf18278bce4af267a+        .quad   0x8255b3d0f1ef990e+        .quad   0x5a758ca390c5f293++        // 2^200 * 2 * G++        .quad   0xa2b72710d9462495+        .quad   0x3aa8c6d2d57d5003+        .quad   0xe3d400bfa0b487ca+        .quad   0x2dbae244b3eb72ec+        .quad   0x8ce0918b1d61dc94+        .quad   0x8ded36469a813066+        .quad   0xd4e6a829afe8aad3+        .quad   0x0a738027f639d43f+        .quad   0x980f4a2f57ffe1cc+        .quad   0x00670d0de1839843+        .quad   0x105c3f4a49fb15fd+        .quad   0x2698ca635126a69c++        // 2^200 * 3 * G++        .quad   0xe765318832b0ba78+        .quad   0x381831f7925cff8b+        .quad   0x08a81b91a0291fcc+        .quad   0x1fb43dcc49caeb07+        .quad   0x2e3d702f5e3dd90e+        .quad   0x9e3f0918e4d25386+        .quad   0x5e773ef6024da96a+        .quad   0x3c004b0c4afa3332+        .quad   0x9aa946ac06f4b82b+        .quad   0x1ca284a5a806c4f3+        .quad   0x3ed3265fc6cd4787+        .quad   0x6b43fd01cd1fd217++        // 2^200 * 4 * G++        .quad   0xc7a75d4b4697c544+        .quad   0x15fdf848df0fffbf+        .quad   0x2868b9ebaa46785a+        .quad   0x5a68d7105b52f714+        .quad   0xb5c742583e760ef3+        .quad   0x75dc52b9ee0ab990+        .quad   0xbf1427c2072b923f+        .quad   0x73420b2d6ff0d9f0+        .quad   0xaf2cf6cb9e851e06+        .quad   0x8f593913c62238c4+        .quad   0xda8ab89699fbf373+        .quad   0x3db5632fea34bc9e++        // 2^200 * 5 * G++        .quad   0xf46eee2bf75dd9d8+        .quad   0x0d17b1f6396759a5+        .quad   0x1bf2d131499e7273+        .quad   0x04321adf49d75f13+        .quad   0x2e4990b1829825d5+        .quad   0xedeaeb873e9a8991+        .quad   0xeef03d394c704af8+        .quad   0x59197ea495df2b0e+        .quad   0x04e16019e4e55aae+        .quad   0xe77b437a7e2f92e9+        .quad   0xc7ce2dc16f159aa4+        .quad   0x45eafdc1f4d70cc0++        // 2^200 * 6 * G++        .quad   0x698401858045d72b+        .quad   0x4c22faa2cf2f0651+        .quad   0x941a36656b222dc6+        .quad   0x5a5eebc80362dade+        .quad   0xb60e4624cfccb1ed+        .quad   0x59dbc292bd5c0395+        .quad   0x31a09d1ddc0481c9+        .quad   0x3f73ceea5d56d940+        .quad   0xb7a7bfd10a4e8dc6+        .quad   0xbe57007e44c9b339+        .quad   0x60c1207f1557aefa+        .quad   0x26058891266218db++        // 2^200 * 7 * G++        .quad   0x59f704a68360ff04+        .quad   0xc3d93fde7661e6f4+        .quad   0x831b2a7312873551+        .quad   0x54ad0c2e4e615d57+        .quad   0x4c818e3cc676e542+        .quad   0x5e422c9303ceccad+        .quad   0xec07cccab4129f08+        .quad   0x0dedfa10b24443b8+        .quad   0xee3b67d5b82b522a+        .quad   0x36f163469fa5c1eb+        .quad   0xa5b4d2f26ec19fd3+        .quad   0x62ecb2baa77a9408++        // 2^200 * 8 * G++        .quad   0xe5ed795261152b3d+        .quad   0x4962357d0eddd7d1+        .quad   0x7482c8d0b96b4c71+        .quad   0x2e59f919a966d8be+        .quad   0x92072836afb62874+        .quad   0x5fcd5e8579e104a5+        .quad   0x5aad01adc630a14a+        .quad   0x61913d5075663f98+        .quad   0x0dc62d361a3231da+        .quad   0xfa47583294200270+        .quad   0x02d801513f9594ce+        .quad   0x3ddbc2a131c05d5c++        // 2^204 * 1 * G++        .quad   0x3f50a50a4ffb81ef+        .quad   0xb1e035093bf420bf+        .quad   0x9baa8e1cc6aa2cd0+        .quad   0x32239861fa237a40+        .quad   0xfb735ac2004a35d1+        .quad   0x31de0f433a6607c3+        .quad   0x7b8591bfc528d599+        .quad   0x55be9a25f5bb050c+        .quad   0x0d005acd33db3dbf+        .quad   0x0111b37c80ac35e2+        .quad   0x4892d66c6f88ebeb+        .quad   0x770eadb16508fbcd++        // 2^204 * 2 * G++        .quad   0x8451f9e05e4e89dd+        .quad   0xc06302ffbc793937+        .quad   0x5d22749556a6495c+        .quad   0x09a6755ca05603fb+        .quad   0xf1d3b681a05071b9+        .quad   0x2207659a3592ff3a+        .quad   0x5f0169297881e40e+        .quad   0x16bedd0e86ba374e+        .quad   0x5ecccc4f2c2737b5+        .quad   0x43b79e0c2dccb703+        .quad   0x33e008bc4ec43df3+        .quad   0x06c1b840f07566c0++        // 2^204 * 3 * G++        .quad   0x7688a5c6a388f877+        .quad   0x02a96c14deb2b6ac+        .quad   0x64c9f3431b8c2af8+        .quad   0x3628435554a1eed6+        .quad   0x69ee9e7f9b02805c+        .quad   0xcbff828a547d1640+        .quad   0x3d93a869b2430968+        .quad   0x46b7b8cd3fe26972+        .quad   0xe9812086fe7eebe0+        .quad   0x4cba6be72f515437+        .quad   0x1d04168b516efae9+        .quad   0x5ea1391043982cb9++        // 2^204 * 4 * G++        .quad   0x49125c9cf4702ee1+        .quad   0x4520b71f8b25b32d+        .quad   0x33193026501fef7e+        .quad   0x656d8997c8d2eb2b+        .quad   0x6f2b3be4d5d3b002+        .quad   0xafec33d96a09c880+        .quad   0x035f73a4a8bcc4cc+        .quad   0x22c5b9284662198b+        .quad   0xcb58c8fe433d8939+        .quad   0x89a0cb2e6a8d7e50+        .quad   0x79ca955309fbbe5a+        .quad   0x0c626616cd7fc106++        // 2^204 * 5 * G++        .quad   0x1ffeb80a4879b61f+        .quad   0x6396726e4ada21ed+        .quad   0x33c7b093368025ba+        .quad   0x471aa0c6f3c31788+        .quad   0x8fdfc379fbf454b1+        .quad   0x45a5a970f1a4b771+        .quad   0xac921ef7bad35915+        .quad   0x42d088dca81c2192+        .quad   0x8fda0f37a0165199+        .quad   0x0adadb77c8a0e343+        .quad   0x20fbfdfcc875e820+        .quad   0x1cf2bea80c2206e7++        // 2^204 * 6 * G++        .quad   0xc2ddf1deb36202ac+        .quad   0x92a5fe09d2e27aa5+        .quad   0x7d1648f6fc09f1d3+        .quad   0x74c2cc0513bc4959+        .quad   0x982d6e1a02c0412f+        .quad   0x90fa4c83db58e8fe+        .quad   0x01c2f5bcdcb18bc0+        .quad   0x686e0c90216abc66+        .quad   0x1fadbadba54395a7+        .quad   0xb41a02a0ae0da66a+        .quad   0xbf19f598bba37c07+        .quad   0x6a12b8acde48430d++        // 2^204 * 7 * G++        .quad   0xf8daea1f39d495d9+        .quad   0x592c190e525f1dfc+        .quad   0xdb8cbd04c9991d1b+        .quad   0x11f7fda3d88f0cb7+        .quad   0x793bdd801aaeeb5f+        .quad   0x00a2a0aac1518871+        .quad   0xe8a373a31f2136b4+        .quad   0x48aab888fc91ef19+        .quad   0x041f7e925830f40e+        .quad   0x002d6ca979661c06+        .quad   0x86dc9ff92b046a2e+        .quad   0x760360928b0493d1++        // 2^204 * 8 * G++        .quad   0x21bb41c6120cf9c6+        .quad   0xeab2aa12decda59b+        .quad   0xc1a72d020aa48b34+        .quad   0x215d4d27e87d3b68+        .quad   0xb43108e5695a0b05+        .quad   0x6cb00ee8ad37a38b+        .quad   0x5edad6eea3537381+        .quad   0x3f2602d4b6dc3224+        .quad   0xc8b247b65bcaf19c+        .quad   0x49779dc3b1b2c652+        .quad   0x89a180bbd5ece2e2+        .quad   0x13f098a3cec8e039++        // 2^208 * 1 * G++        .quad   0x9adc0ff9ce5ec54b+        .quad   0x039c2a6b8c2f130d+        .quad   0x028007c7f0f89515+        .quad   0x78968314ac04b36b+        .quad   0xf3aa57a22796bb14+        .quad   0x883abab79b07da21+        .quad   0xe54be21831a0391c+        .quad   0x5ee7fb38d83205f9+        .quad   0x538dfdcb41446a8e+        .quad   0xa5acfda9434937f9+        .quad   0x46af908d263c8c78+        .quad   0x61d0633c9bca0d09++        // 2^208 * 2 * G++        .quad   0x63744935ffdb2566+        .quad   0xc5bd6b89780b68bb+        .quad   0x6f1b3280553eec03+        .quad   0x6e965fd847aed7f5+        .quad   0xada328bcf8fc73df+        .quad   0xee84695da6f037fc+        .quad   0x637fb4db38c2a909+        .quad   0x5b23ac2df8067bdc+        .quad   0x9ad2b953ee80527b+        .quad   0xe88f19aafade6d8d+        .quad   0x0e711704150e82cf+        .quad   0x79b9bbb9dd95dedc++        // 2^208 * 3 * G++        .quad   0xebb355406a3126c2+        .quad   0xd26383a868c8c393+        .quad   0x6c0c6429e5b97a82+        .quad   0x5065f158c9fd2147+        .quad   0xd1997dae8e9f7374+        .quad   0xa032a2f8cfbb0816+        .quad   0xcd6cba126d445f0a+        .quad   0x1ba811460accb834+        .quad   0x708169fb0c429954+        .quad   0xe14600acd76ecf67+        .quad   0x2eaab98a70e645ba+        .quad   0x3981f39e58a4faf2++        // 2^208 * 4 * G++        .quad   0x18fb8a7559230a93+        .quad   0x1d168f6960e6f45d+        .quad   0x3a85a94514a93cb5+        .quad   0x38dc083705acd0fd+        .quad   0xc845dfa56de66fde+        .quad   0xe152a5002c40483a+        .quad   0xe9d2e163c7b4f632+        .quad   0x30f4452edcbc1b65+        .quad   0x856d2782c5759740+        .quad   0xfa134569f99cbecc+        .quad   0x8844fc73c0ea4e71+        .quad   0x632d9a1a593f2469++        // 2^208 * 5 * G++        .quad   0xf6bb6b15b807cba6+        .quad   0x1823c7dfbc54f0d7+        .quad   0xbb1d97036e29670b+        .quad   0x0b24f48847ed4a57+        .quad   0xbf09fd11ed0c84a7+        .quad   0x63f071810d9f693a+        .quad   0x21908c2d57cf8779+        .quad   0x3a5a7df28af64ba2+        .quad   0xdcdad4be511beac7+        .quad   0xa4538075ed26ccf2+        .quad   0xe19cff9f005f9a65+        .quad   0x34fcf74475481f63++        // 2^208 * 6 * G++        .quad   0xc197e04c789767ca+        .quad   0xb8714dcb38d9467d+        .quad   0x55de888283f95fa8+        .quad   0x3d3bdc164dfa63f7+        .quad   0xa5bb1dab78cfaa98+        .quad   0x5ceda267190b72f2+        .quad   0x9309c9110a92608e+        .quad   0x0119a3042fb374b0+        .quad   0x67a2d89ce8c2177d+        .quad   0x669da5f66895d0c1+        .quad   0xf56598e5b282a2b0+        .quad   0x56c088f1ede20a73++        // 2^208 * 7 * G++        .quad   0x336d3d1110a86e17+        .quad   0xd7f388320b75b2fa+        .quad   0xf915337625072988+        .quad   0x09674c6b99108b87+        .quad   0x581b5fac24f38f02+        .quad   0xa90be9febae30cbd+        .quad   0x9a2169028acf92f0+        .quad   0x038b7ea48359038f+        .quad   0x9f4ef82199316ff8+        .quad   0x2f49d282eaa78d4f+        .quad   0x0971a5ab5aef3174+        .quad   0x6e5e31025969eb65++        // 2^208 * 8 * G++        .quad   0xb16c62f587e593fb+        .quad   0x4999eddeca5d3e71+        .quad   0xb491c1e014cc3e6d+        .quad   0x08f5114789a8dba8+        .quad   0x3304fb0e63066222+        .quad   0xfb35068987acba3f+        .quad   0xbd1924778c1061a3+        .quad   0x3058ad43d1838620+        .quad   0x323c0ffde57663d0+        .quad   0x05c3df38a22ea610+        .quad   0xbdc78abdac994f9a+        .quad   0x26549fa4efe3dc99++        // 2^212 * 1 * G++        .quad   0x738b38d787ce8f89+        .quad   0xb62658e24179a88d+        .quad   0x30738c9cf151316d+        .quad   0x49128c7f727275c9+        .quad   0x04dbbc17f75396b9+        .quad   0x69e6a2d7d2f86746+        .quad   0xc6409d99f53eabc6+        .quad   0x606175f6332e25d2+        .quad   0x4021370ef540e7dd+        .quad   0x0910d6f5a1f1d0a5+        .quad   0x4634aacd5b06b807+        .quad   0x6a39e6356944f235++        // 2^212 * 2 * G++        .quad   0x96cd5640df90f3e7+        .quad   0x6c3a760edbfa25ea+        .quad   0x24f3ef0959e33cc4+        .quad   0x42889e7e530d2e58+        .quad   0x1da1965774049e9d+        .quad   0xfbcd6ea198fe352b+        .quad   0xb1cbcd50cc5236a6+        .quad   0x1f5ec83d3f9846e2+        .quad   0x8efb23c3328ccb75+        .quad   0xaf42a207dd876ee9+        .quad   0x20fbdadc5dfae796+        .quad   0x241e246b06bf9f51++        // 2^212 * 3 * G++        .quad   0x29e68e57ad6e98f6+        .quad   0x4c9260c80b462065+        .quad   0x3f00862ea51ebb4b+        .quad   0x5bc2c77fb38d9097+        .quad   0x7eaafc9a6280bbb8+        .quad   0x22a70f12f403d809+        .quad   0x31ce40bb1bfc8d20+        .quad   0x2bc65635e8bd53ee+        .quad   0xe8d5dc9fa96bad93+        .quad   0xe58fb17dde1947dc+        .quad   0x681532ea65185fa3+        .quad   0x1fdd6c3b034a7830++        // 2^212 * 4 * G++        .quad   0x0a64e28c55dc18fe+        .quad   0xe3df9e993399ebdd+        .quad   0x79ac432370e2e652+        .quad   0x35ff7fc33ae4cc0e+        .quad   0x9c13a6a52dd8f7a9+        .quad   0x2dbb1f8c3efdcabf+        .quad   0x961e32405e08f7b5+        .quad   0x48c8a121bbe6c9e5+        .quad   0xfc415a7c59646445+        .quad   0xd224b2d7c128b615+        .quad   0x6035c9c905fbb912+        .quad   0x42d7a91274429fab++        // 2^212 * 5 * G++        .quad   0x4e6213e3eaf72ed3+        .quad   0x6794981a43acd4e7+        .quad   0xff547cde6eb508cb+        .quad   0x6fed19dd10fcb532+        .quad   0xa9a48947933da5bc+        .quad   0x4a58920ec2e979ec+        .quad   0x96d8800013e5ac4c+        .quad   0x453692d74b48b147+        .quad   0xdd775d99a8559c6f+        .quad   0xf42a2140df003e24+        .quad   0x5223e229da928a66+        .quad   0x063f46ba6d38f22c++        // 2^212 * 6 * G++        .quad   0xd2d242895f536694+        .quad   0xca33a2c542939b2c+        .quad   0x986fada6c7ddb95c+        .quad   0x5a152c042f712d5d+        .quad   0x39843cb737346921+        .quad   0xa747fb0738c89447+        .quad   0xcb8d8031a245307e+        .quad   0x67810f8e6d82f068+        .quad   0x3eeb8fbcd2287db4+        .quad   0x72c7d3a301a03e93+        .quad   0x5473e88cbd98265a+        .quad   0x7324aa515921b403++        // 2^212 * 7 * G++        .quad   0x857942f46c3cbe8e+        .quad   0xa1d364b14730c046+        .quad   0x1c8ed914d23c41bf+        .quad   0x0838e161eef6d5d2+        .quad   0xad23f6dae82354cb+        .quad   0x6962502ab6571a6d+        .quad   0x9b651636e38e37d1+        .quad   0x5cac5005d1a3312f+        .quad   0x8cc154cce9e39904+        .quad   0x5b3a040b84de6846+        .quad   0xc4d8a61cb1be5d6e+        .quad   0x40fb897bd8861f02++        // 2^212 * 8 * G++        .quad   0x84c5aa9062de37a1+        .quad   0x421da5000d1d96e1+        .quad   0x788286306a9242d9+        .quad   0x3c5e464a690d10da+        .quad   0xe57ed8475ab10761+        .quad   0x71435e206fd13746+        .quad   0x342f824ecd025632+        .quad   0x4b16281ea8791e7b+        .quad   0xd1c101d50b813381+        .quad   0xdee60f1176ee6828+        .quad   0x0cb68893383f6409+        .quad   0x6183c565f6ff484a++        // 2^216 * 1 * G++        .quad   0x741d5a461e6bf9d6+        .quad   0x2305b3fc7777a581+        .quad   0xd45574a26474d3d9+        .quad   0x1926e1dc6401e0ff+        .quad   0xdb468549af3f666e+        .quad   0xd77fcf04f14a0ea5+        .quad   0x3df23ff7a4ba0c47+        .quad   0x3a10dfe132ce3c85+        .quad   0xe07f4e8aea17cea0+        .quad   0x2fd515463a1fc1fd+        .quad   0x175322fd31f2c0f1+        .quad   0x1fa1d01d861e5d15++        // 2^216 * 2 * G++        .quad   0xcc8055947d599832+        .quad   0x1e4656da37f15520+        .quad   0x99f6f7744e059320+        .quad   0x773563bc6a75cf33+        .quad   0x38dcac00d1df94ab+        .quad   0x2e712bddd1080de9+        .quad   0x7f13e93efdd5e262+        .quad   0x73fced18ee9a01e5+        .quad   0x06b1e90863139cb3+        .quad   0xa493da67c5a03ecd+        .quad   0x8d77cec8ad638932+        .quad   0x1f426b701b864f44++        // 2^216 * 3 * G++        .quad   0xefc9264c41911c01+        .quad   0xf1a3b7b817a22c25+        .quad   0x5875da6bf30f1447+        .quad   0x4e1af5271d31b090+        .quad   0xf17e35c891a12552+        .quad   0xb76b8153575e9c76+        .quad   0xfa83406f0d9b723e+        .quad   0x0b76bb1b3fa7e438+        .quad   0x08b8c1f97f92939b+        .quad   0xbe6771cbd444ab6e+        .quad   0x22e5646399bb8017+        .quad   0x7b6dd61eb772a955++        // 2^216 * 4 * G++        .quad   0xb7adc1e850f33d92+        .quad   0x7998fa4f608cd5cf+        .quad   0xad962dbd8dfc5bdb+        .quad   0x703e9bceaf1d2f4f+        .quad   0x5730abf9ab01d2c7+        .quad   0x16fb76dc40143b18+        .quad   0x866cbe65a0cbb281+        .quad   0x53fa9b659bff6afe+        .quad   0x6c14c8e994885455+        .quad   0x843a5d6665aed4e5+        .quad   0x181bb73ebcd65af1+        .quad   0x398d93e5c4c61f50++        // 2^216 * 5 * G++        .quad   0x1c4bd16733e248f3+        .quad   0xbd9e128715bf0a5f+        .quad   0xd43f8cf0a10b0376+        .quad   0x53b09b5ddf191b13+        .quad   0xc3877c60d2e7e3f2+        .quad   0x3b34aaa030828bb1+        .quad   0x283e26e7739ef138+        .quad   0x699c9c9002c30577+        .quad   0xf306a7235946f1cc+        .quad   0x921718b5cce5d97d+        .quad   0x28cdd24781b4e975+        .quad   0x51caf30c6fcdd907++        // 2^216 * 6 * G++        .quad   0xa60ba7427674e00a+        .quad   0x630e8570a17a7bf3+        .quad   0x3758563dcf3324cc+        .quad   0x5504aa292383fdaa+        .quad   0x737af99a18ac54c7+        .quad   0x903378dcc51cb30f+        .quad   0x2b89bc334ce10cc7+        .quad   0x12ae29c189f8e99a+        .quad   0xa99ec0cb1f0d01cf+        .quad   0x0dd1efcc3a34f7ae+        .quad   0x55ca7521d09c4e22+        .quad   0x5fd14fe958eba5ea++        // 2^216 * 7 * G++        .quad   0xb5dc2ddf2845ab2c+        .quad   0x069491b10a7fe993+        .quad   0x4daaf3d64002e346+        .quad   0x093ff26e586474d1+        .quad   0x3c42fe5ebf93cb8e+        .quad   0xbedfa85136d4565f+        .quad   0xe0f0859e884220e8+        .quad   0x7dd73f960725d128+        .quad   0xb10d24fe68059829+        .quad   0x75730672dbaf23e5+        .quad   0x1367253ab457ac29+        .quad   0x2f59bcbc86b470a4++        // 2^216 * 8 * G++        .quad   0x83847d429917135f+        .quad   0xad1b911f567d03d7+        .quad   0x7e7748d9be77aad1+        .quad   0x5458b42e2e51af4a+        .quad   0x7041d560b691c301+        .quad   0x85201b3fadd7e71e+        .quad   0x16c2e16311335585+        .quad   0x2aa55e3d010828b1+        .quad   0xed5192e60c07444f+        .quad   0x42c54e2d74421d10+        .quad   0x352b4c82fdb5c864+        .quad   0x13e9004a8a768664++        // 2^220 * 1 * G++        .quad   0xcbb5b5556c032bff+        .quad   0xdf7191b729297a3a+        .quad   0xc1ff7326aded81bb+        .quad   0x71ade8bb68be03f5+        .quad   0x1e6284c5806b467c+        .quad   0xc5f6997be75d607b+        .quad   0x8b67d958b378d262+        .quad   0x3d88d66a81cd8b70+        .quad   0x8b767a93204ed789+        .quad   0x762fcacb9fa0ae2a+        .quad   0x771febcc6dce4887+        .quad   0x343062158ff05fb3++        // 2^220 * 2 * G++        .quad   0xe05da1a7e1f5bf49+        .quad   0x26457d6dd4736092+        .quad   0x77dcb07773cc32f6+        .quad   0x0a5d94969cdd5fcd+        .quad   0xfce219072a7b31b4+        .quad   0x4d7adc75aa578016+        .quad   0x0ec276a687479324+        .quad   0x6d6d9d5d1fda4beb+        .quad   0x22b1a58ae9b08183+        .quad   0xfd95d071c15c388b+        .quad   0xa9812376850a0517+        .quad   0x33384cbabb7f335e++        // 2^220 * 3 * G++        .quad   0x3c6fa2680ca2c7b5+        .quad   0x1b5082046fb64fda+        .quad   0xeb53349c5431d6de+        .quad   0x5278b38f6b879c89+        .quad   0x33bc627a26218b8d+        .quad   0xea80b21fc7a80c61+        .quad   0x9458b12b173e9ee6+        .quad   0x076247be0e2f3059+        .quad   0x52e105f61416375a+        .quad   0xec97af3685abeba4+        .quad   0x26e6b50623a67c36+        .quad   0x5cf0e856f3d4fb01++        // 2^220 * 4 * G++        .quad   0xf6c968731ae8cab4+        .quad   0x5e20741ecb4f92c5+        .quad   0x2da53be58ccdbc3e+        .quad   0x2dddfea269970df7+        .quad   0xbeaece313db342a8+        .quad   0xcba3635b842db7ee+        .quad   0xe88c6620817f13ef+        .quad   0x1b9438aa4e76d5c6+        .quad   0x8a50777e166f031a+        .quad   0x067b39f10fb7a328+        .quad   0x1925c9a6010fbd76+        .quad   0x6df9b575cc740905++        // 2^220 * 5 * G++        .quad   0x42c1192927f6bdcf+        .quad   0x8f91917a403d61ca+        .quad   0xdc1c5a668b9e1f61+        .quad   0x1596047804ec0f8d+        .quad   0xecdfc35b48cade41+        .quad   0x6a88471fb2328270+        .quad   0x740a4a2440a01b6a+        .quad   0x471e5796003b5f29+        .quad   0xda96bbb3aced37ac+        .quad   0x7a2423b5e9208cea+        .quad   0x24cc5c3038aebae2+        .quad   0x50c356afdc5dae2f++        // 2^220 * 6 * G++        .quad   0x09dcbf4341c30318+        .quad   0xeeba061183181dce+        .quad   0xc179c0cedc1e29a1+        .quad   0x1dbf7b89073f35b0+        .quad   0xcfed9cdf1b31b964+        .quad   0xf486a9858ca51af3+        .quad   0x14897265ea8c1f84+        .quad   0x784a53dd932acc00+        .quad   0x2d99f9df14fc4920+        .quad   0x76ccb60cc4499fe5+        .quad   0xa4132cbbe5cf0003+        .quad   0x3f93d82354f000ea++        // 2^220 * 7 * G++        .quad   0x8183e7689e04ce85+        .quad   0x678fb71e04465341+        .quad   0xad92058f6688edac+        .quad   0x5da350d3532b099a+        .quad   0xeaac12d179e14978+        .quad   0xff923ff3bbebff5e+        .quad   0x4af663e40663ce27+        .quad   0x0fd381a811a5f5ff+        .quad   0xf256aceca436df54+        .quad   0x108b6168ae69d6e8+        .quad   0x20d986cb6b5d036c+        .quad   0x655957b9fee2af50++        // 2^220 * 8 * G++        .quad   0xaea8b07fa902030f+        .quad   0xf88c766af463d143+        .quad   0x15b083663c787a60+        .quad   0x08eab1148267a4a8+        .quad   0xbdc1409bd002d0ac+        .quad   0x66660245b5ccd9a6+        .quad   0x82317dc4fade85ec+        .quad   0x02fe934b6ad7df0d+        .quad   0xef5cf100cfb7ea74+        .quad   0x22897633a1cb42ac+        .quad   0xd4ce0c54cef285e2+        .quad   0x30408c048a146a55++        // 2^224 * 1 * G++        .quad   0x739d8845832fcedb+        .quad   0xfa38d6c9ae6bf863+        .quad   0x32bc0dcab74ffef7+        .quad   0x73937e8814bce45e+        .quad   0xbb2e00c9193b877f+        .quad   0xece3a890e0dc506b+        .quad   0xecf3b7c036de649f+        .quad   0x5f46040898de9e1a+        .quad   0xb9037116297bf48d+        .quad   0xa9d13b22d4f06834+        .quad   0xe19715574696bdc6+        .quad   0x2cf8a4e891d5e835++        // 2^224 * 2 * G++        .quad   0x6d93fd8707110f67+        .quad   0xdd4c09d37c38b549+        .quad   0x7cb16a4cc2736a86+        .quad   0x2049bd6e58252a09+        .quad   0x2cb5487e17d06ba2+        .quad   0x24d2381c3950196b+        .quad   0xd7659c8185978a30+        .quad   0x7a6f7f2891d6a4f6+        .quad   0x7d09fd8d6a9aef49+        .quad   0xf0ee60be5b3db90b+        .quad   0x4c21b52c519ebfd4+        .quad   0x6011aadfc545941d++        // 2^224 * 3 * G++        .quad   0x5f67926dcf95f83c+        .quad   0x7c7e856171289071+        .quad   0xd6a1e7f3998f7a5b+        .quad   0x6fc5cc1b0b62f9e0+        .quad   0x63ded0c802cbf890+        .quad   0xfbd098ca0dff6aaa+        .quad   0x624d0afdb9b6ed99+        .quad   0x69ce18b779340b1e+        .quad   0xd1ef5528b29879cb+        .quad   0xdd1aae3cd47e9092+        .quad   0x127e0442189f2352+        .quad   0x15596b3ae57101f1++        // 2^224 * 4 * G++        .quad   0x462739d23f9179a2+        .quad   0xff83123197d6ddcf+        .quad   0x1307deb553f2148a+        .quad   0x0d2237687b5f4dda+        .quad   0x09ff31167e5124ca+        .quad   0x0be4158bd9c745df+        .quad   0x292b7d227ef556e5+        .quad   0x3aa4e241afb6d138+        .quad   0x2cc138bf2a3305f5+        .quad   0x48583f8fa2e926c3+        .quad   0x083ab1a25549d2eb+        .quad   0x32fcaa6e4687a36c++        // 2^224 * 5 * G++        .quad   0x7bc56e8dc57d9af5+        .quad   0x3e0bd2ed9df0bdf2+        .quad   0xaac014de22efe4a3+        .quad   0x4627e9cefebd6a5c+        .quad   0x3207a4732787ccdf+        .quad   0x17e31908f213e3f8+        .quad   0xd5b2ecd7f60d964e+        .quad   0x746f6336c2600be9+        .quad   0x3f4af345ab6c971c+        .quad   0xe288eb729943731f+        .quad   0x33596a8a0344186d+        .quad   0x7b4917007ed66293++        // 2^224 * 6 * G++        .quad   0x2d85fb5cab84b064+        .quad   0x497810d289f3bc14+        .quad   0x476adc447b15ce0c+        .quad   0x122ba376f844fd7b+        .quad   0x54341b28dd53a2dd+        .quad   0xaa17905bdf42fc3f+        .quad   0x0ff592d94dd2f8f4+        .quad   0x1d03620fe08cd37d+        .quad   0xc20232cda2b4e554+        .quad   0x9ed0fd42115d187f+        .quad   0x2eabb4be7dd479d9+        .quad   0x02c70bf52b68ec4c++        // 2^224 * 7 * G++        .quad   0xa287ec4b5d0b2fbb+        .quad   0x415c5790074882ca+        .quad   0xe044a61ec1d0815c+        .quad   0x26334f0a409ef5e0+        .quad   0xace532bf458d72e1+        .quad   0x5be768e07cb73cb5+        .quad   0x56cf7d94ee8bbde7+        .quad   0x6b0697e3feb43a03+        .quad   0xb6c8f04adf62a3c0+        .quad   0x3ef000ef076da45d+        .quad   0x9c9cb95849f0d2a9+        .quad   0x1cc37f43441b2fae++        // 2^224 * 8 * G++        .quad   0x508f565a5cc7324f+        .quad   0xd061c4c0e506a922+        .quad   0xfb18abdb5c45ac19+        .quad   0x6c6809c10380314a+        .quad   0xd76656f1c9ceaeb9+        .quad   0x1c5b15f818e5656a+        .quad   0x26e72832844c2334+        .quad   0x3a346f772f196838+        .quad   0xd2d55112e2da6ac8+        .quad   0xe9bd0331b1e851ed+        .quad   0x960746dd8ec67262+        .quad   0x05911b9f6ef7c5d0++        // 2^228 * 1 * G++        .quad   0xe9dcd756b637ff2d+        .quad   0xec4c348fc987f0c4+        .quad   0xced59285f3fbc7b7+        .quad   0x3305354793e1ea87+        .quad   0x01c18980c5fe9f94+        .quad   0xcd656769716fd5c8+        .quad   0x816045c3d195a086+        .quad   0x6e2b7f3266cc7982+        .quad   0xcc802468f7c3568f+        .quad   0x9de9ba8219974cb3+        .quad   0xabb7229cb5b81360+        .quad   0x44e2017a6fbeba62++        // 2^228 * 2 * G++        .quad   0xc4c2a74354dab774+        .quad   0x8e5d4c3c4eaf031a+        .quad   0xb76c23d242838f17+        .quad   0x749a098f68dce4ea+        .quad   0x87f82cf3b6ca6ecd+        .quad   0x580f893e18f4a0c2+        .quad   0x058930072604e557+        .quad   0x6cab6ac256d19c1d+        .quad   0xdcdfe0a02cc1de60+        .quad   0x032665ff51c5575b+        .quad   0x2c0c32f1073abeeb+        .quad   0x6a882014cd7b8606++        // 2^228 * 3 * G++        .quad   0xa52a92fea4747fb5+        .quad   0xdc12a4491fa5ab89+        .quad   0xd82da94bb847a4ce+        .quad   0x4d77edce9512cc4e+        .quad   0xd111d17caf4feb6e+        .quad   0x050bba42b33aa4a3+        .quad   0x17514c3ceeb46c30+        .quad   0x54bedb8b1bc27d75+        .quad   0x77c8e14577e2189c+        .quad   0xa3e46f6aff99c445+        .quad   0x3144dfc86d335343+        .quad   0x3a96559e7c4216a9++        // 2^228 * 4 * G++        .quad   0x12550d37f42ad2ee+        .quad   0x8b78e00498a1fbf5+        .quad   0x5d53078233894cb2+        .quad   0x02c84e4e3e498d0c+        .quad   0x4493896880baaa52+        .quad   0x4c98afc4f285940e+        .quad   0xef4aa79ba45448b6+        .quad   0x5278c510a57aae7f+        .quad   0xa54dd074294c0b94+        .quad   0xf55d46b8df18ffb6+        .quad   0xf06fecc58dae8366+        .quad   0x588657668190d165++        // 2^228 * 5 * G++        .quad   0xd47712311aef7117+        .quad   0x50343101229e92c7+        .quad   0x7a95e1849d159b97+        .quad   0x2449959b8b5d29c9+        .quad   0xbf5834f03de25cc3+        .quad   0xb887c8aed6815496+        .quad   0x5105221a9481e892+        .quad   0x6760ed19f7723f93+        .quad   0x669ba3b7ac35e160+        .quad   0x2eccf73fba842056+        .quad   0x1aec1f17c0804f07+        .quad   0x0d96bc031856f4e7++        // 2^228 * 6 * G++        .quad   0x3318be7775c52d82+        .quad   0x4cb764b554d0aab9+        .quad   0xabcf3d27cc773d91+        .quad   0x3bf4d1848123288a+        .quad   0xb1d534b0cc7505e1+        .quad   0x32cd003416c35288+        .quad   0xcb36a5800762c29d+        .quad   0x5bfe69b9237a0bf8+        .quad   0x183eab7e78a151ab+        .quad   0xbbe990c999093763+        .quad   0xff717d6e4ac7e335+        .quad   0x4c5cddb325f39f88++        // 2^228 * 7 * G++        .quad   0xc0f6b74d6190a6eb+        .quad   0x20ea81a42db8f4e4+        .quad   0xa8bd6f7d97315760+        .quad   0x33b1d60262ac7c21+        .quad   0x57750967e7a9f902+        .quad   0x2c37fdfc4f5b467e+        .quad   0xb261663a3177ba46+        .quad   0x3a375e78dc2d532b+        .quad   0x8141e72f2d4dddea+        .quad   0xe6eafe9862c607c8+        .quad   0x23c28458573cafd0+        .quad   0x46b9476f4ff97346++        // 2^228 * 8 * G++        .quad   0x0c1ffea44f901e5c+        .quad   0x2b0b6fb72184b782+        .quad   0xe587ff910114db88+        .quad   0x37130f364785a142+        .quad   0x1215505c0d58359f+        .quad   0x2a2013c7fc28c46b+        .quad   0x24a0a1af89ea664e+        .quad   0x4400b638a1130e1f+        .quad   0x3a01b76496ed19c3+        .quad   0x31e00ab0ed327230+        .quad   0x520a885783ca15b1+        .quad   0x06aab9875accbec7++        // 2^232 * 1 * G++        .quad   0xc1339983f5df0ebb+        .quad   0xc0f3758f512c4cac+        .quad   0x2cf1130a0bb398e1+        .quad   0x6b3cecf9aa270c62+        .quad   0x5349acf3512eeaef+        .quad   0x20c141d31cc1cb49+        .quad   0x24180c07a99a688d+        .quad   0x555ef9d1c64b2d17+        .quad   0x36a770ba3b73bd08+        .quad   0x624aef08a3afbf0c+        .quad   0x5737ff98b40946f2+        .quad   0x675f4de13381749d++        // 2^232 * 2 * G++        .quad   0x0e2c52036b1782fc+        .quad   0x64816c816cad83b4+        .quad   0xd0dcbdd96964073e+        .quad   0x13d99df70164c520+        .quad   0xa12ff6d93bdab31d+        .quad   0x0725d80f9d652dfe+        .quad   0x019c4ff39abe9487+        .quad   0x60f450b882cd3c43+        .quad   0x014b5ec321e5c0ca+        .quad   0x4fcb69c9d719bfa2+        .quad   0x4e5f1c18750023a0+        .quad   0x1c06de9e55edac80++        // 2^232 * 3 * G++        .quad   0x990f7ad6a33ec4e2+        .quad   0x6608f938be2ee08e+        .quad   0x9ca143c563284515+        .quad   0x4cf38a1fec2db60d+        .quad   0xffd52b40ff6d69aa+        .quad   0x34530b18dc4049bb+        .quad   0x5e4a5c2fa34d9897+        .quad   0x78096f8e7d32ba2d+        .quad   0xa0aaaa650dfa5ce7+        .quad   0xf9c49e2a48b5478c+        .quad   0x4f09cc7d7003725b+        .quad   0x373cad3a26091abe++        // 2^232 * 4 * G++        .quad   0xb294634d82c9f57c+        .quad   0x1fcbfde124934536+        .quad   0x9e9c4db3418cdb5a+        .quad   0x0040f3d9454419fc+        .quad   0xf1bea8fb89ddbbad+        .quad   0x3bcb2cbc61aeaecb+        .quad   0x8f58a7bb1f9b8d9d+        .quad   0x21547eda5112a686+        .quad   0xdefde939fd5986d3+        .quad   0xf4272c89510a380c+        .quad   0xb72ba407bb3119b9+        .quad   0x63550a334a254df4++        // 2^232 * 5 * G++        .quad   0x6507d6edb569cf37+        .quad   0x178429b00ca52ee1+        .quad   0xea7c0090eb6bd65d+        .quad   0x3eea62c7daf78f51+        .quad   0x9bba584572547b49+        .quad   0xf305c6fae2c408e0+        .quad   0x60e8fa69c734f18d+        .quad   0x39a92bafaa7d767a+        .quad   0x9d24c713e693274e+        .quad   0x5f63857768dbd375+        .quad   0x70525560eb8ab39a+        .quad   0x68436a0665c9c4cd++        // 2^232 * 6 * G++        .quad   0xbc0235e8202f3f27+        .quad   0xc75c00e264f975b0+        .quad   0x91a4e9d5a38c2416+        .quad   0x17b6e7f68ab789f9+        .quad   0x1e56d317e820107c+        .quad   0xc5266844840ae965+        .quad   0xc1e0a1c6320ffc7a+        .quad   0x5373669c91611472+        .quad   0x5d2814ab9a0e5257+        .quad   0x908f2084c9cab3fc+        .quad   0xafcaf5885b2d1eca+        .quad   0x1cb4b5a678f87d11++        // 2^232 * 7 * G++        .quad   0xb664c06b394afc6c+        .quad   0x0c88de2498da5fb1+        .quad   0x4f8d03164bcad834+        .quad   0x330bca78de7434a2+        .quad   0x6b74aa62a2a007e7+        .quad   0xf311e0b0f071c7b1+        .quad   0x5707e438000be223+        .quad   0x2dc0fd2d82ef6eac+        .quad   0x982eff841119744e+        .quad   0xf9695e962b074724+        .quad   0xc58ac14fbfc953fb+        .quad   0x3c31be1b369f1cf5++        // 2^232 * 8 * G++        .quad   0xb0f4864d08948aee+        .quad   0x07dc19ee91ba1c6f+        .quad   0x7975cdaea6aca158+        .quad   0x330b61134262d4bb+        .quad   0xc168bc93f9cb4272+        .quad   0xaeb8711fc7cedb98+        .quad   0x7f0e52aa34ac8d7a+        .quad   0x41cec1097e7d55bb+        .quad   0xf79619d7a26d808a+        .quad   0xbb1fd49e1d9e156d+        .quad   0x73d7c36cdba1df27+        .quad   0x26b44cd91f28777d++        // 2^236 * 1 * G++        .quad   0x300a9035393aa6d8+        .quad   0x2b501131a12bb1cd+        .quad   0x7b1ff677f093c222+        .quad   0x4309c1f8cab82bad+        .quad   0xaf44842db0285f37+        .quad   0x8753189047efc8df+        .quad   0x9574e091f820979a+        .quad   0x0e378d6069615579+        .quad   0xd9fa917183075a55+        .quad   0x4bdb5ad26b009fdc+        .quad   0x7829ad2cd63def0e+        .quad   0x078fc54975fd3877++        // 2^236 * 2 * G++        .quad   0x87dfbd1428878f2d+        .quad   0x134636dd1e9421a1+        .quad   0x4f17c951257341a3+        .quad   0x5df98d4bad296cb8+        .quad   0xe2004b5bb833a98a+        .quad   0x44775dec2d4c3330+        .quad   0x3aa244067eace913+        .quad   0x272630e3d58e00a9+        .quad   0xf3678fd0ecc90b54+        .quad   0xf001459b12043599+        .quad   0x26725fbc3758b89b+        .quad   0x4325e4aa73a719ae++        // 2^236 * 3 * G++        .quad   0x657dc6ef433c3493+        .quad   0x65375e9f80dbf8c3+        .quad   0x47fd2d465b372dae+        .quad   0x4966ab79796e7947+        .quad   0xed24629acf69f59d+        .quad   0x2a4a1ccedd5abbf4+        .quad   0x3535ca1f56b2d67b+        .quad   0x5d8c68d043b1b42d+        .quad   0xee332d4de3b42b0a+        .quad   0xd84e5a2b16a4601c+        .quad   0x78243877078ba3e4+        .quad   0x77ed1eb4184ee437++        // 2^236 * 4 * G++        .quad   0xbfd4e13f201839a0+        .quad   0xaeefffe23e3df161+        .quad   0xb65b04f06b5d1fe3+        .quad   0x52e085fb2b62fbc0+        .quad   0x185d43f89e92ed1a+        .quad   0xb04a1eeafe4719c6+        .quad   0x499fbe88a6f03f4f+        .quad   0x5d8b0d2f3c859bdd+        .quad   0x124079eaa54cf2ba+        .quad   0xd72465eb001b26e7+        .quad   0x6843bcfdc97af7fd+        .quad   0x0524b42b55eacd02++        // 2^236 * 5 * G++        .quad   0xfd0d5dbee45447b0+        .quad   0x6cec351a092005ee+        .quad   0x99a47844567579cb+        .quad   0x59d242a216e7fa45+        .quad   0xbc18dcad9b829eac+        .quad   0x23ae7d28b5f579d0+        .quad   0xc346122a69384233+        .quad   0x1a6110b2e7d4ac89+        .quad   0x4f833f6ae66997ac+        .quad   0x6849762a361839a4+        .quad   0x6985dec1970ab525+        .quad   0x53045e89dcb1f546++        // 2^236 * 6 * G++        .quad   0xcb8bb346d75353db+        .quad   0xfcfcb24bae511e22+        .quad   0xcba48d40d50ae6ef+        .quad   0x26e3bae5f4f7cb5d+        .quad   0x84da3cde8d45fe12+        .quad   0xbd42c218e444e2d2+        .quad   0xa85196781f7e3598+        .quad   0x7642c93f5616e2b2+        .quad   0x2323daa74595f8e4+        .quad   0xde688c8b857abeb4+        .quad   0x3fc48e961c59326e+        .quad   0x0b2e73ca15c9b8ba++        // 2^236 * 7 * G++        .quad   0xd6bb4428c17f5026+        .quad   0x9eb27223fb5a9ca7+        .quad   0xe37ba5031919c644+        .quad   0x21ce380db59a6602+        .quad   0x0e3fbfaf79c03a55+        .quad   0x3077af054cbb5acf+        .quad   0xd5c55245db3de39f+        .quad   0x015e68c1476a4af7+        .quad   0xc1d5285220066a38+        .quad   0x95603e523570aef3+        .quad   0x832659a7226b8a4d+        .quad   0x5dd689091f8eedc9++        // 2^236 * 8 * G++        .quad   0xcbac84debfd3c856+        .quad   0x1624c348b35ff244+        .quad   0xb7f88dca5d9cad07+        .quad   0x3b0e574da2c2ebe8+        .quad   0x1d022591a5313084+        .quad   0xca2d4aaed6270872+        .quad   0x86a12b852f0bfd20+        .quad   0x56e6c439ad7da748+        .quad   0xc704ff4942bdbae6+        .quad   0x5e21ade2b2de1f79+        .quad   0xe95db3f35652fad8+        .quad   0x0822b5378f08ebc1++        // 2^240 * 1 * G++        .quad   0x51f048478f387475+        .quad   0xb25dbcf49cbecb3c+        .quad   0x9aab1244d99f2055+        .quad   0x2c709e6c1c10a5d6+        .quad   0xe1b7f29362730383+        .quad   0x4b5279ffebca8a2c+        .quad   0xdafc778abfd41314+        .quad   0x7deb10149c72610f+        .quad   0xcb62af6a8766ee7a+        .quad   0x66cbec045553cd0e+        .quad   0x588001380f0be4b5+        .quad   0x08e68e9ff62ce2ea++        // 2^240 * 2 * G++        .quad   0x34ad500a4bc130ad+        .quad   0x8d38db493d0bd49c+        .quad   0xa25c3d98500a89be+        .quad   0x2f1f3f87eeba3b09+        .quad   0x2f2d09d50ab8f2f9+        .quad   0xacb9218dc55923df+        .quad   0x4a8f342673766cb9+        .quad   0x4cb13bd738f719f5+        .quad   0xf7848c75e515b64a+        .quad   0xa59501badb4a9038+        .quad   0xc20d313f3f751b50+        .quad   0x19a1e353c0ae2ee8++        // 2^240 * 3 * G++        .quad   0x7d1c7560bafa05c3+        .quad   0xb3e1a0a0c6e55e61+        .quad   0xe3529718c0d66473+        .quad   0x41546b11c20c3486+        .quad   0xb42172cdd596bdbd+        .quad   0x93e0454398eefc40+        .quad   0x9fb15347b44109b5+        .quad   0x736bd3990266ae34+        .quad   0x85532d509334b3b4+        .quad   0x46fd114b60816573+        .quad   0xcc5f5f30425c8375+        .quad   0x412295a2b87fab5c++        // 2^240 * 4 * G++        .quad   0x19c99b88f57ed6e9+        .quad   0x5393cb266df8c825+        .quad   0x5cee3213b30ad273+        .quad   0x14e153ebb52d2e34+        .quad   0x2e655261e293eac6+        .quad   0x845a92032133acdb+        .quad   0x460975cb7900996b+        .quad   0x0760bb8d195add80+        .quad   0x413e1a17cde6818a+        .quad   0x57156da9ed69a084+        .quad   0x2cbf268f46caccb1+        .quad   0x6b34be9bc33ac5f2++        // 2^240 * 5 * G++        .quad   0xf3df2f643a78c0b2+        .quad   0x4c3e971ef22e027c+        .quad   0xec7d1c5e49c1b5a3+        .quad   0x2012c18f0922dd2d+        .quad   0x11fc69656571f2d3+        .quad   0xc6c9e845530e737a+        .quad   0xe33ae7a2d4fe5035+        .quad   0x01b9c7b62e6dd30b+        .quad   0x880b55e55ac89d29+        .quad   0x1483241f45a0a763+        .quad   0x3d36efdfc2e76c1f+        .quad   0x08af5b784e4bade8++        // 2^240 * 6 * G++        .quad   0x283499dc881f2533+        .quad   0x9d0525da779323b6+        .quad   0x897addfb673441f4+        .quad   0x32b79d71163a168d+        .quad   0xe27314d289cc2c4b+        .quad   0x4be4bd11a287178d+        .quad   0x18d528d6fa3364ce+        .quad   0x6423c1d5afd9826e+        .quad   0xcc85f8d9edfcb36a+        .quad   0x22bcc28f3746e5f9+        .quad   0xe49de338f9e5d3cd+        .quad   0x480a5efbc13e2dcc++        // 2^240 * 7 * G++        .quad   0x0b51e70b01622071+        .quad   0x06b505cf8b1dafc5+        .quad   0x2c6bb061ef5aabcd+        .quad   0x47aa27600cb7bf31+        .quad   0xb6614ce442ce221f+        .quad   0x6e199dcc4c053928+        .quad   0x663fb4a4dc1cbe03+        .quad   0x24b31d47691c8e06+        .quad   0x2a541eedc015f8c3+        .quad   0x11a4fe7e7c693f7c+        .quad   0xf0af66134ea278d6+        .quad   0x545b585d14dda094++        // 2^240 * 8 * G++        .quad   0x67bf275ea0d43a0f+        .quad   0xade68e34089beebe+        .quad   0x4289134cd479e72e+        .quad   0x0f62f9c332ba5454+        .quad   0x6204e4d0e3b321e1+        .quad   0x3baa637a28ff1e95+        .quad   0x0b0ccffd5b99bd9e+        .quad   0x4d22dc3e64c8d071+        .quad   0xfcb46589d63b5f39+        .quad   0x5cae6a3f57cbcf61+        .quad   0xfebac2d2953afa05+        .quad   0x1c0fa01a36371436++        // 2^244 * 1 * G++        .quad   0xe7547449bc7cd692+        .quad   0x0f9abeaae6f73ddf+        .quad   0x4af01ca700837e29+        .quad   0x63ab1b5d3f1bc183+        .quad   0xc11ee5e854c53fae+        .quad   0x6a0b06c12b4f3ff4+        .quad   0x33540f80e0b67a72+        .quad   0x15f18fc3cd07e3ef+        .quad   0x32750763b028f48c+        .quad   0x06020740556a065f+        .quad   0xd53bd812c3495b58+        .quad   0x08706c9b865f508d++        // 2^244 * 2 * G++        .quad   0xf37ca2ab3d343dff+        .quad   0x1a8c6a2d80abc617+        .quad   0x8e49e035d4ccffca+        .quad   0x48b46beebaa1d1b9+        .quad   0xcc991b4138b41246+        .quad   0x243b9c526f9ac26b+        .quad   0xb9ef494db7cbabbd+        .quad   0x5fba433dd082ed00+        .quad   0x9c49e355c9941ad0+        .quad   0xb9734ade74498f84+        .quad   0x41c3fed066663e5c+        .quad   0x0ecfedf8e8e710b3++        // 2^244 * 3 * G++        .quad   0x76430f9f9cd470d9+        .quad   0xb62acc9ba42f6008+        .quad   0x1898297c59adad5e+        .quad   0x7789dd2db78c5080+        .quad   0x744f7463e9403762+        .quad   0xf79a8dee8dfcc9c9+        .quad   0x163a649655e4cde3+        .quad   0x3b61788db284f435+        .quad   0xb22228190d6ef6b2+        .quad   0xa94a66b246ce4bfa+        .quad   0x46c1a77a4f0b6cc7+        .quad   0x4236ccffeb7338cf++        // 2^244 * 4 * G++        .quad   0x8497404d0d55e274+        .quad   0x6c6663d9c4ad2b53+        .quad   0xec2fb0d9ada95734+        .quad   0x2617e120cdb8f73c+        .quad   0x3bd82dbfda777df6+        .quad   0x71b177cc0b98369e+        .quad   0x1d0e8463850c3699+        .quad   0x5a71945b48e2d1f1+        .quad   0x6f203dd5405b4b42+        .quad   0x327ec60410b24509+        .quad   0x9c347230ac2a8846+        .quad   0x77de29fc11ffeb6a++        // 2^244 * 5 * G++        .quad   0xb0ac57c983b778a8+        .quad   0x53cdcca9d7fe912c+        .quad   0x61c2b854ff1f59dc+        .quad   0x3a1a2cf0f0de7dac+        .quad   0x835e138fecced2ca+        .quad   0x8c9eaf13ea963b9a+        .quad   0xc95fbfc0b2160ea6+        .quad   0x575e66f3ad877892+        .quad   0x99803a27c88fcb3a+        .quad   0x345a6789275ec0b0+        .quad   0x459789d0ff6c2be5+        .quad   0x62f882651e70a8b2++        // 2^244 * 6 * G++        .quad   0x085ae2c759ff1be4+        .quad   0x149145c93b0e40b7+        .quad   0xc467e7fa7ff27379+        .quad   0x4eeecf0ad5c73a95+        .quad   0x6d822986698a19e0+        .quad   0xdc9821e174d78a71+        .quad   0x41a85f31f6cb1f47+        .quad   0x352721c2bcda9c51+        .quad   0x48329952213fc985+        .quad   0x1087cf0d368a1746+        .quad   0x8e5261b166c15aa5+        .quad   0x2d5b2d842ed24c21++        // 2^244 * 7 * G++        .quad   0x02cfebd9ebd3ded1+        .quad   0xd45b217739021974+        .quad   0x7576f813fe30a1b7+        .quad   0x5691b6f9a34ef6c2+        .quad   0x5eb7d13d196ac533+        .quad   0x377234ecdb80be2b+        .quad   0xe144cffc7cf5ae24+        .quad   0x5226bcf9c441acec+        .quad   0x79ee6c7223e5b547+        .quad   0x6f5f50768330d679+        .quad   0xed73e1e96d8adce9+        .quad   0x27c3da1e1d8ccc03++        // 2^244 * 8 * G++        .quad   0x7eb9efb23fe24c74+        .quad   0x3e50f49f1651be01+        .quad   0x3ea732dc21858dea+        .quad   0x17377bd75bb810f9+        .quad   0x28302e71630ef9f6+        .quad   0xc2d4a2032b64cee0+        .quad   0x090820304b6292be+        .quad   0x5fca747aa82adf18+        .quad   0x232a03c35c258ea5+        .quad   0x86f23a2c6bcb0cf1+        .quad   0x3dad8d0d2e442166+        .quad   0x04a8933cab76862b++        // 2^248 * 1 * G++        .quad   0xd2c604b622943dff+        .quad   0xbc8cbece44cfb3a0+        .quad   0x5d254ff397808678+        .quad   0x0fa3614f3b1ca6bf+        .quad   0x69082b0e8c936a50+        .quad   0xf9c9a035c1dac5b6+        .quad   0x6fb73e54c4dfb634+        .quad   0x4005419b1d2bc140+        .quad   0xa003febdb9be82f0+        .quad   0x2089c1af3a44ac90+        .quad   0xf8499f911954fa8e+        .quad   0x1fba218aef40ab42++        // 2^248 * 2 * G++        .quad   0xab549448fac8f53e+        .quad   0x81f6e89a7ba63741+        .quad   0x74fd6c7d6c2b5e01+        .quad   0x392e3acaa8c86e42+        .quad   0x4f3e57043e7b0194+        .quad   0xa81d3eee08daaf7f+        .quad   0xc839c6ab99dcdef1+        .quad   0x6c535d13ff7761d5+        .quad   0x4cbd34e93e8a35af+        .quad   0x2e0781445887e816+        .quad   0x19319c76f29ab0ab+        .quad   0x25e17fe4d50ac13b++        // 2^248 * 3 * G++        .quad   0x0a289bd71e04f676+        .quad   0x208e1c52d6420f95+        .quad   0x5186d8b034691fab+        .quad   0x255751442a9fb351+        .quad   0x915f7ff576f121a7+        .quad   0xc34a32272fcd87e3+        .quad   0xccba2fde4d1be526+        .quad   0x6bba828f8969899b+        .quad   0xe2d1bc6690fe3901+        .quad   0x4cb54a18a0997ad5+        .quad   0x971d6914af8460d4+        .quad   0x559d504f7f6b7be4++        // 2^248 * 4 * G++        .quad   0xa7738378b3eb54d5+        .quad   0x1d69d366a5553c7c+        .quad   0x0a26cf62f92800ba+        .quad   0x01ab12d5807e3217+        .quad   0x9c4891e7f6d266fd+        .quad   0x0744a19b0307781b+        .quad   0x88388f1d6061e23b+        .quad   0x123ea6a3354bd50e+        .quad   0x118d189041e32d96+        .quad   0xb9ede3c2d8315848+        .quad   0x1eab4271d83245d9+        .quad   0x4a3961e2c918a154++        // 2^248 * 5 * G++        .quad   0x71dc3be0f8e6bba0+        .quad   0xd6cef8347effe30a+        .quad   0xa992425fe13a476a+        .quad   0x2cd6bce3fb1db763+        .quad   0x0327d644f3233f1e+        .quad   0x499a260e34fcf016+        .quad   0x83b5a716f2dab979+        .quad   0x68aceead9bd4111f+        .quad   0x38b4c90ef3d7c210+        .quad   0x308e6e24b7ad040c+        .quad   0x3860d9f1b7e73e23+        .quad   0x595760d5b508f597++        // 2^248 * 6 * G++        .quad   0x6129bfe104aa6397+        .quad   0x8f960008a4a7fccb+        .quad   0x3f8bc0897d909458+        .quad   0x709fa43edcb291a9+        .quad   0x882acbebfd022790+        .quad   0x89af3305c4115760+        .quad   0x65f492e37d3473f4+        .quad   0x2cb2c5df54515a2b+        .quad   0xeb0a5d8c63fd2aca+        .quad   0xd22bc1662e694eff+        .quad   0x2723f36ef8cbb03a+        .quad   0x70f029ecf0c8131f++        // 2^248 * 7 * G++        .quad   0x461307b32eed3e33+        .quad   0xae042f33a45581e7+        .quad   0xc94449d3195f0366+        .quad   0x0b7d5d8a6c314858+        .quad   0x2a6aafaa5e10b0b9+        .quad   0x78f0a370ef041aa9+        .quad   0x773efb77aa3ad61f+        .quad   0x44eca5a2a74bd9e1+        .quad   0x25d448327b95d543+        .quad   0x70d38300a3340f1d+        .quad   0xde1c531c60e1c52b+        .quad   0x272224512c7de9e4++        // 2^248 * 8 * G++        .quad   0x1abc92af49c5342e+        .quad   0xffeed811b2e6fad0+        .quad   0xefa28c8dfcc84e29+        .quad   0x11b5df18a44cc543+        .quad   0xbf7bbb8a42a975fc+        .quad   0x8c5c397796ada358+        .quad   0xe27fc76fcdedaa48+        .quad   0x19735fd7f6bc20a6+        .quad   0xe3ab90d042c84266+        .quad   0xeb848e0f7f19547e+        .quad   0x2503a1d065a497b9+        .quad   0x0fef911191df895f++        // 2^252 * 1 * G++        .quad   0xb1507ca1ab1c6eb9+        .quad   0xbd448f3e16b687b3+        .quad   0x3455fb7f2c7a91ab+        .quad   0x7579229e2f2adec1+        .quad   0x6ab5dcb85b1c16b7+        .quad   0x94c0fce83c7b27a5+        .quad   0xa4b11c1a735517be+        .quad   0x499238d0ba0eafaa+        .quad   0xecf46e527aba8b57+        .quad   0x15a08c478bd1647b+        .quad   0x7af1c6a65f706fef+        .quad   0x6345fa78f03a30d5++        // 2^252 * 2 * G++        .quad   0xdf02f95f1015e7a1+        .quad   0x790ec41da9b40263+        .quad   0x4d3a0ea133ea1107+        .quad   0x54f70be7e33af8c9+        .quad   0x93d3cbe9bdd8f0a4+        .quad   0xdb152c1bfd177302+        .quad   0x7dbddc6d7f17a875+        .quad   0x3e1a71cc8f426efe+        .quad   0xc83ca3e390babd62+        .quad   0x80ede3670291c833+        .quad   0xc88038ccd37900c4+        .quad   0x2c5fc0231ec31fa1++        // 2^252 * 3 * G++        .quad   0xfeba911717038b4f+        .quad   0xe5123721c9deef81+        .quad   0x1c97e4e75d0d8834+        .quad   0x68afae7a23dc3bc6+        .quad   0xc422e4d102456e65+        .quad   0x87414ac1cad47b91+        .quad   0x1592e2bba2b6ffdd+        .quad   0x75d9d2bff5c2100f+        .quad   0x5bd9b4763626e81c+        .quad   0x89966936bca02edd+        .quad   0x0a41193d61f077b3+        .quad   0x3097a24200ce5471++        // 2^252 * 4 * G++        .quad   0x57427734c7f8b84c+        .quad   0xf141a13e01b270e9+        .quad   0x02d1adfeb4e564a6+        .quad   0x4bb23d92ce83bd48+        .quad   0xa162e7246695c486+        .quad   0x131d633435a89607+        .quad   0x30521561a0d12a37+        .quad   0x56704bada6afb363+        .quad   0xaf6c4aa752f912b9+        .quad   0x5e665f6cd86770c8+        .quad   0x4c35ac83a3c8cd58+        .quad   0x2b7a29c010a58a7e++        // 2^252 * 5 * G++        .quad   0xc4007f77d0c1cec3+        .quad   0x8d1020b6bac492f8+        .quad   0x32ec29d57e69daaf+        .quad   0x599408759d95fce0+        .quad   0x33810a23bf00086e+        .quad   0xafce925ee736ff7c+        .quad   0x3d60e670e24922d4+        .quad   0x11ce9e714f96061b+        .quad   0x219ef713d815bac1+        .quad   0xf141465d485be25c+        .quad   0x6d5447cc4e513c51+        .quad   0x174926be5ef44393++        // 2^252 * 6 * G++        .quad   0xb5deb2f9fc5bd5bb+        .quad   0x92daa72ae1d810e1+        .quad   0xafc4cfdcb72a1c59+        .quad   0x497d78813fc22a24+        .quad   0x3ef5d41593ea022e+        .quad   0x5cbcc1a20ed0eed6+        .quad   0x8fd24ecf07382c8c+        .quad   0x6fa42ead06d8e1ad+        .quad   0xe276824a1f73371f+        .quad   0x7f7cf01c4f5b6736+        .quad   0x7e201fe304fa46e7+        .quad   0x785a36a357808c96++        // 2^252 * 7 * G++        .quad   0x825fbdfd63014d2b+        .quad   0xc852369c6ca7578b+        .quad   0x5b2fcd285c0b5df0+        .quad   0x12ab214c58048c8f+        .quad   0x070442985d517bc3+        .quad   0x6acd56c7ae653678+        .quad   0x00a27983985a7763+        .quad   0x5167effae512662b+        .quad   0xbd4ea9e10f53c4b6+        .quad   0x1673dc5f8ac91a14+        .quad   0xa8f81a4e2acc1aba+        .quad   0x33a92a7924332a25++        // 2^252 * 8 * G++        .quad   0x9dd1f49927996c02+        .quad   0x0cb3b058e04d1752+        .quad   0x1f7e88967fd02c3e+        .quad   0x2f964268cb8b3eb1+        .quad   0x7ba95ba0218f2ada+        .quad   0xcff42287330fb9ca+        .quad   0xdada496d56c6d907+        .quad   0x5380c296f4beee54+        .quad   0x9d4f270466898d0a+        .quad   0x3d0987990aff3f7a+        .quad   0xd09ef36267daba45+        .quad   0x7761455e7b1c669c
+ cbits/s2n/x86_att/edwards25519_encode.S view
@@ -0,0 +1,86 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Encode edwards25519 point into compressed form as 256-bit number+// Input p[8]; output z[32] (bytes)+//+//    extern void edwards25519_encode(uint8_t z[static 32],+//                                    const uint64_t p[static 8]);+//+// This assumes that the input buffer p points to a pair of 256-bit+// numbers x (at p) and y (at p+4) representing a point (x,y) on the+// edwards25519 curve. It is assumed that both x and y are < p_25519+// but there is no checking of this, nor of the fact that (x,y) is+// in fact on the curve.+//+// The output in z is a little-endian array of bytes corresponding to+// the standard compressed encoding of a point as 2^255 * x_0 + y+// where x_0 is the least significant bit of x.+// See "https://datatracker.ietf.org/doc/html/rfc8032#section-5.1.2"+// In this implementation, y is simply truncated to 255 bits, but if+// it is reduced mod p_25519 as expected this does not affect values.+//+// Standard x86-64 ABI: RDI = z, RSI = p+// Microsoft x64 ABI:   RCX = z, RDX = p+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(edwards25519_encode)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(edwards25519_encode)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(edwards25519_encode)+        .text++#define z %rdi+#define p %rsi+#define y0 %rax+#define y1 %rcx+#define y2 %rdx+#define y3 %r8+#define xb %r9++S2N_BN_SYMBOL(edwards25519_encode):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Load lowest word of x coordinate in xb and full y as [y3;y2;y1;y0].++        movq    (p), xb+        movq    32(p), y0+        movq    40(p), y1+        movq    48(p), y2+        movq    56(p), y3++// Compute the encoded form, making the LSB of x the MSB of the encoding++        btr     $63, y3+        shlq    $63, xb+        orq     xb, y3++// Store back (by the word, since x86 is little-endian anyway)++        movq    y0, (z)+        movq    y1, 8(z)+        movq    y2, 16(z)+        movq    y3, 24(z)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(edwards25519_encode)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack,"",%progbits+#endif
+ cbits/s2n/x86_att/edwards25519_scalarmulbase.S view
@@ -0,0 +1,9926 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Scalar multiplication for the edwards25519 standard basepoint+// Input scalar[4]; output res[8]+//+// extern void edwards25519_scalarmulbase+//   (uint64_t res[static 8],const uint64_t scalar[static 4]);+//+// Given a scalar n, returns point (X,Y) = n * B where B = (...,4/5) is+// the standard basepoint for the edwards25519 (Ed25519) curve.+//+// Standard x86-64 ABI: RDI = res, RSI = scalar+// Microsoft x64 ABI:   RCX = res, RDX = scalar+// ----------------------------------------------------------------------------+#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(edwards25519_scalarmulbase)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(edwards25519_scalarmulbase)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(edwards25519_scalarmulbase)++        .text+        .balign 32++// Size of individual field elements++#define NUMSIZE 32++// Pointer-offset pairs for result and temporaries on stack with some aliasing.+// The result "resx" assumes the "res" pointer has been preloaded into %rbp.++#define resx (0*NUMSIZE)(%rbp)+#define resy (1*NUMSIZE)(%rbp)++#define scalar (0*NUMSIZE)(%rsp)++#define tabent (1*NUMSIZE)(%rsp)+#define ymx_2 (1*NUMSIZE)(%rsp)+#define xpy_2 (2*NUMSIZE)(%rsp)+#define kxy_2 (3*NUMSIZE)(%rsp)++#define t0 (4*NUMSIZE)(%rsp)+#define t1 (5*NUMSIZE)(%rsp)+#define t2 (6*NUMSIZE)(%rsp)+#define t3 (7*NUMSIZE)(%rsp)+#define t4 (8*NUMSIZE)(%rsp)+#define t5 (9*NUMSIZE)(%rsp)++#define acc (10*NUMSIZE)(%rsp)+#define x_1 (10*NUMSIZE)(%rsp)+#define y_1 (11*NUMSIZE)(%rsp)+#define z_1 (12*NUMSIZE)(%rsp)+#define w_1 (13*NUMSIZE)(%rsp)+#define x_3 (10*NUMSIZE)(%rsp)+#define y_3 (11*NUMSIZE)(%rsp)+#define z_3 (12*NUMSIZE)(%rsp)+#define w_3 (13*NUMSIZE)(%rsp)++// Stable homes for the input result pointer, and other variables++#define res  14*NUMSIZE(%rsp)++#define i  14*NUMSIZE+8(%rsp)++#define bias  14*NUMSIZE+16(%rsp)++#define bf  14*NUMSIZE+24(%rsp)+#define ix  14*NUMSIZE+24(%rsp)++#define tab  15*NUMSIZE(%rsp)++// Total size to reserve on the stack++#define NSPACE 488++// Syntactic variants to make x86_att version simpler to generate++#define SCALAR 0+#define TABENT (1*NUMSIZE)+#define ACC (10*NUMSIZE)+#define X3 (10*NUMSIZE)+#define Z3 (12*NUMSIZE)+#define W3 (13*NUMSIZE)++// Macro wrapping up the basic field multiplication, only trivially+// different from a pure function call to bignum_mul_p25519.++#define mul_p25519(P0,P1,P2)                    \+        xorl   %esi, %esi ;                        \+        movq   P2, %rdx ;                       \+        mulxq  P1, %r8, %r9 ;                    \+        mulxq  0x8+P1, %rax, %r10 ;              \+        addq   %rax, %r9 ;                         \+        mulxq  0x10+P1, %rax, %r11 ;             \+        adcq   %rax, %r10 ;                        \+        mulxq  0x18+P1, %rax, %r12 ;             \+        adcq   %rax, %r11 ;                        \+        adcq   %rsi, %r12 ;                        \+        xorl   %esi, %esi ;                        \+        movq   0x8+P2, %rdx ;                   \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rbx, %r10 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x18+P1, %rax, %r13 ;             \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rsi, %r13 ;                        \+        adcxq  %rsi, %r13 ;                        \+        xorl   %esi, %esi ;                        \+        movq   0x10+P2, %rdx ;                  \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rbx, %r13 ;                        \+        mulxq  0x18+P1, %rax, %r14 ;             \+        adcxq  %rax, %r13 ;                        \+        adoxq  %rsi, %r14 ;                        \+        adcxq  %rsi, %r14 ;                        \+        xorl   %esi, %esi ;                        \+        movq   0x18+P2, %rdx ;                  \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rbx, %r13 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r13 ;                        \+        adoxq  %rbx, %r14 ;                        \+        mulxq  0x18+P1, %rax, %r15 ;             \+        adcxq  %rax, %r14 ;                        \+        adoxq  %rsi, %r15 ;                        \+        adcxq  %rsi, %r15 ;                        \+        movl   $0x26, %edx ;                       \+        xorl   %esi, %esi ;                        \+        mulxq  %r12, %rax, %rbx ;                   \+        adcxq  %rax, %r8 ;                         \+        adoxq  %rbx, %r9 ;                         \+        mulxq  %r13, %rax, %rbx ;                   \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rbx, %r10 ;                        \+        mulxq  %r14, %rax, %rbx ;                   \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  %r15, %rax, %r12 ;                   \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rsi, %r12 ;                        \+        adcxq  %rsi, %r12 ;                        \+        shldq  $0x1, %r11, %r12 ;                   \+        movl   $0x13, %edx ;                       \+        incq   %r12;                             \+        bts    $63, %r11 ;                         \+        mulxq  %r12, %rax, %rbx ;                   \+        addq   %rax, %r8 ;                         \+        adcq   %rbx, %r9 ;                         \+        adcq   %rsi, %r10 ;                        \+        adcq   %rsi, %r11 ;                        \+        sbbq   %rax, %rax ;                        \+        notq   %rax;                             \+        andq   %rdx, %rax ;                        \+        subq   %rax, %r8 ;                         \+        sbbq   %rsi, %r9 ;                         \+        sbbq   %rsi, %r10 ;                        \+        sbbq   %rsi, %r11 ;                        \+        btr    $63, %r11 ;                         \+        movq   %r8, P0 ;                        \+        movq   %r9, 0x8+P0 ;                    \+        movq   %r10, 0x10+P0 ;                  \+        movq   %r11, 0x18+P0++// A version of multiplication that only guarantees output < 2 * p_25519.+// This basically skips the +1 and final correction in quotient estimation.++#define mul_4(P0,P1,P2)                         \+        xorl   %ecx, %ecx ;                        \+        movq   P2, %rdx ;                       \+        mulxq  P1, %r8, %r9 ;                    \+        mulxq  0x8+P1, %rax, %r10 ;              \+        addq   %rax, %r9 ;                         \+        mulxq  0x10+P1, %rax, %r11 ;             \+        adcq   %rax, %r10 ;                        \+        mulxq  0x18+P1, %rax, %r12 ;             \+        adcq   %rax, %r11 ;                        \+        adcq   %rcx, %r12 ;                        \+        xorl   %ecx, %ecx ;                        \+        movq   0x8+P2, %rdx ;                   \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rbx, %r10 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x18+P1, %rax, %r13 ;             \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rcx, %r13 ;                        \+        adcxq  %rcx, %r13 ;                        \+        xorl   %ecx, %ecx ;                        \+        movq   0x10+P2, %rdx ;                  \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rbx, %r13 ;                        \+        mulxq  0x18+P1, %rax, %r14 ;             \+        adcxq  %rax, %r13 ;                        \+        adoxq  %rcx, %r14 ;                        \+        adcxq  %rcx, %r14 ;                        \+        xorl   %ecx, %ecx ;                        \+        movq   0x18+P2, %rdx ;                  \+        mulxq  P1, %rax, %rbx ;                  \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rbx, %r12 ;                        \+        mulxq  0x8+P1, %rax, %rbx ;              \+        adcxq  %rax, %r12 ;                        \+        adoxq  %rbx, %r13 ;                        \+        mulxq  0x10+P1, %rax, %rbx ;             \+        adcxq  %rax, %r13 ;                        \+        adoxq  %rbx, %r14 ;                        \+        mulxq  0x18+P1, %rax, %r15 ;             \+        adcxq  %rax, %r14 ;                        \+        adoxq  %rcx, %r15 ;                        \+        adcxq  %rcx, %r15 ;                        \+        movl   $0x26, %edx ;                       \+        xorl   %ecx, %ecx ;                        \+        mulxq  %r12, %rax, %rbx ;                   \+        adcxq  %rax, %r8 ;                         \+        adoxq  %rbx, %r9 ;                         \+        mulxq  %r13, %rax, %rbx ;                   \+        adcxq  %rax, %r9 ;                         \+        adoxq  %rbx, %r10 ;                        \+        mulxq  %r14, %rax, %rbx ;                   \+        adcxq  %rax, %r10 ;                        \+        adoxq  %rbx, %r11 ;                        \+        mulxq  %r15, %rax, %r12 ;                   \+        adcxq  %rax, %r11 ;                        \+        adoxq  %rcx, %r12 ;                        \+        adcxq  %rcx, %r12 ;                        \+        shldq  $0x1, %r11, %r12 ;                   \+        btr    $0x3f, %r11 ;                       \+        movl   $0x13, %edx ;                       \+        imulq  %r12, %rdx ;                        \+        addq   %rdx, %r8 ;                         \+        adcq   %rcx, %r9 ;                         \+        adcq   %rcx, %r10 ;                        \+        adcq   %rcx, %r11 ;                        \+        movq   %r8, P0 ;                        \+        movq   %r9, 0x8+P0 ;                    \+        movq   %r10, 0x10+P0 ;                  \+        movq   %r11, 0x18+P0++// Modular subtraction with double modulus 2 * p_25519 = 2^256 - 38++#define sub_twice4(P0,P1,P2)                    \+        movq    P1, %r8 ;                       \+        xorl    %ebx, %ebx ;                       \+        subq    P2, %r8 ;                       \+        movq    8+P1, %r9 ;                     \+        sbbq    8+P2, %r9 ;                     \+        movl    $38, %ecx ;                        \+        movq    16+P1, %r10 ;                   \+        sbbq    16+P2, %r10 ;                   \+        movq    24+P1, %rax ;                   \+        sbbq    24+P2, %rax ;                   \+        cmovncq %rbx, %rcx ;                       \+        subq    %rcx, %r8 ;                        \+        sbbq    %rbx, %r9 ;                        \+        sbbq    %rbx, %r10 ;                       \+        sbbq    %rbx, %rax ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 8+P0 ;                     \+        movq    %r10, 16+P0 ;                   \+        movq    %rax, 24+P0++// Modular addition and doubling with double modulus 2 * p_25519 = 2^256 - 38.+// This only ensures that the result fits in 4 digits, not that it is reduced+// even w.r.t. double modulus. The result is always correct modulo provided+// the sum of the inputs is < 2^256 + 2^256 - 38, so in particular provided+// at least one of them is reduced double modulo.++#define add_twice4(P0,P1,P2)                    \+        movq    P1, %r8 ;                       \+        xorl    %ecx, %ecx ;                       \+        addq    P2, %r8 ;                       \+        movq    0x8+P1, %r9 ;                   \+        adcq    0x8+P2, %r9 ;                   \+        movq    0x10+P1, %r10 ;                 \+        adcq    0x10+P2, %r10 ;                 \+        movq    0x18+P1, %r11 ;                 \+        adcq    0x18+P2, %r11 ;                 \+        movl    $38, %eax ;                        \+        cmovncq %rcx, %rax ;                       \+        addq    %rax, %r8 ;                        \+        adcq    %rcx, %r9 ;                        \+        adcq    %rcx, %r10 ;                       \+        adcq    %rcx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++#define double_twice4(P0,P1)                    \+        movq    P1, %r8 ;                       \+        xorl    %ecx, %ecx ;                       \+        addq    %r8, %r8 ;                         \+        movq    0x8+P1, %r9 ;                   \+        adcq    %r9, %r9 ;                         \+        movq    0x10+P1, %r10 ;                 \+        adcq    %r10, %r10 ;                       \+        movq    0x18+P1, %r11 ;                 \+        adcq    %r11, %r11 ;                       \+        movl    $38, %eax ;                        \+        cmovncq %rcx, %rax ;                       \+        addq    %rax, %r8 ;                        \+        adcq    %rcx, %r9 ;                        \+        adcq    %rcx, %r10 ;                       \+        adcq    %rcx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++S2N_BN_SYMBOL(edwards25519_scalarmulbase):+        CFI_START+        _CET_ENDBR++// In this case the Windows form literally makes a subroutine call.+// This avoids hassle arising from keeping code and data together.++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        CFI_CALL(Ledwards25519_scalarmulbase_standard)+        CFI_POP(%rsi)+        CFI_POP(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(edwards25519_scalarmulbase)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Ledwards25519_scalarmulbase_standard)++Ledwards25519_scalarmulbase_standard:+        CFI_START+#endif++// Save registers, make room for temps, preserve input arguments.++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(NSPACE)++// Move the output pointer to a stable place++        movq    %rdi, res++// Copy the input scalar x to its local variable while reducing it+// modulo 2^252 + m where m = 27742317777372353535851937790883648493;+// this is the order of the basepoint so this doesn't change the result.+// First do q = floor(x/2^252) and x' = x - q * (2^252 + m), which gives+// an initial result -15 * m <= x' < 2^252++        movq    (%rsi), %r8+        movq    8(%rsi), %r9+        movq    16(%rsi), %r10+        movq    24(%rsi), %r11++        movq    %r11, %rcx+        shrq    $60, %rcx++        movq    $0x5812631a5cf5d3ed, %rax+        mulq    %rcx+        movq    %rax, %r12+        movq    %rdx, %r13+        movq    $0x14def9dea2f79cd6, %rax+        mulq    %rcx+        addq    %rax, %r13+        adcq    $0, %rdx+        shlq    $60, %rcx++        subq    %r12, %r8+        sbbq    %r13, %r9+        sbbq    %rdx, %r10+        sbbq    %rcx, %r11++// If x' < 0 then just directly negate it; this makes sure the+// reduced argument is strictly 0 <= x' < 2^252, but now we need+// to record (done via bit 255 of the reduced scalar, which is+// ignored in the main loop) when we negated so we can flip+// the end result to compensate.++        sbbq    %rax, %rax++        xorq    %rax, %r8+        xorq    %rax, %r9+        xorq    %rax, %r10+        xorq    %rax, %r11++        negq    %rax+        adcq    $0, %r8+        adcq    $0, %r9+        adcq    $0, %r10+        adcq    $0, %r11++        shlq    $63, %rax+        orq     %rax, %r11++// And before we store the scalar, test and reset bit 251 to+// initialize the main loop just below.++        movq    %r8, SCALAR(%rsp)+        movq    %r9, SCALAR+8(%rsp)+        movq    %r10, SCALAR+16(%rsp)+        btr     $59, %r11+        movq    %r11, SCALAR+24(%rsp)++// The main part of the computation is in extended-projective coordinates+// (X,Y,Z,T), representing an affine point on the edwards25519 curve+// (x,y) via x = X/Z, y = Y/Z and x * y = T/Z (so X * Y = T * Z).+// In comments B means the standard basepoint (x,4/5) =+// (0x216....f25d51a,0x6666..666658).+//+// Initialize accumulator "acc" to either 0 or 2^251 * B depending on+// bit 251 of the (reduced) scalar. That leaves bits 0..250 to handle.++        leaq    S2N_BN_SYMBOL(edwards25519_scalarmulbase_constant)(%rip), %r10+        leaq    8*12(%r10), %r11++        movq    (%r10), %rax+        movq    (%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC(%rsp)++        movq    8*1(%r10), %rax+        movq    8*1(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+8(%rsp)++        movq    8*2(%r10), %rax+        movq    8*2(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+16(%rsp)++        movq    8*3(%r10), %rax+        movq    8*3(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+24(%rsp)++        movq    8*4(%r10), %rax+        movq    8*4(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+32(%rsp)++        movq    8*5(%r10), %rax+        movq    8*5(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+40(%rsp)++        movq    8*6(%r10), %rax+        movq    8*6(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+48(%rsp)++        movq    8*7(%r10), %rax+        movq    8*7(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+56(%rsp)++        movl    $1, %eax+        movq    %rax, ACC+64(%rsp)+        movl    $0, %eax+        movq    %rax, ACC+72(%rsp)+        movq    %rax, ACC+80(%rsp)+        movq    %rax, ACC+88(%rsp)++        movq    8*8(%r10), %rax+        movq    8*8(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+96(%rsp)++        movq    8*9(%r10), %rax+        movq    8*9(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+104(%rsp)++        movq    8*10(%r10), %rax+        movq    8*10(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+112(%rsp)++        movq    8*11(%r10), %rax+        movq    8*11(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+120(%rsp)++// The counter "i" tracks the bit position for which the scalar has+// already been absorbed, starting at 0 and going up in chunks of 4.+//+// The pointer "tab" points at the current block of the table for+// multiples (2^i * j) * B at the current bit position i; 1 <= j <= 8.+//+// The bias is always either 0 and 1 and needs to be added to the+// partially processed scalar implicitly. This is used to absorb 4 bits+// of scalar per iteration from 3-bit table indexing by exploiting+// negation: (16 * h + l) * B = (16 * (h + 1) - (16 - l)) * B is used+// when l >= 9. Note that we can't have any bias left over at the+// end because we made sure bit 251 is clear in the reduced scalar.++        movq    $0, i+        leaq    8*24(%r10), %rax+        movq    %rax, tab+        movq    $0, bias++// Start of the main loop, repeated 63 times for i = 4, 8, ..., 252++Ledwards25519_scalarmulbase_scalarloop:++// Look at the next 4-bit field "bf", adding the previous bias as well.+// Choose the table index "ix" as bf when bf <= 8 and 16 - bf for bf >= 9,+// setting the bias to 1 for the next iteration in the latter case.++        movq    i, %rax+        movq    %rax, %rcx+        shrq    $6, %rax+        movq    (%rsp,%rax,8), %rax // Exploiting scalar = sp exactly+        shrq    %cl, %rax+        andq    $15, %rax+        addq    bias, %rax+        movq    %rax, bf++        cmpq    $9, bf+        sbbq    %rax, %rax+        incq    %rax+        movq    %rax, bias++        movq    $16, %rdi+        subq    bf, %rdi+        cmpq    $0, bias+        cmovzq  bf, %rdi+        movq    %rdi, ix++// Perform constant-time lookup in the table to get element number "ix".+// The table entry for the affine point (x,y) is actually a triple+// (y - x,x + y,2 * d * x * y) to precompute parts of the addition.+// Note that "ix" can be 0, so we set up the appropriate identity first.++        movl    $1, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        movl    $1, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        xorl    %r13d, %r13d+        xorl    %r14d, %r14d+        xorl    %r15d, %r15d++        movq    tab, %rbp++        cmpq    $1, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $2, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $3, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $4, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $5, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $6, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $7, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $8, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15++        addq    $96, %rbp+        movq    %rbp, tab++// We now have the triple from the table in registers as follows+//+//      [%rdx;%rcx;%rbx;%rax] = y - x+//      [%r11;%r10;%r9;%r8] = x + y+//      [%r15;%r14;%r13;%r12] = 2 * d * x * y+//+// In case bias = 1 we need to negate this. For Edwards curves+// -(x,y) = (-x,y), i.e. we need to negate the x coordinate.+// In this processed encoding, that amounts to swapping the+// first two fields and negating the third.+//+// The optional negation here also pretends bias = 0 whenever+// ix = 0 so that it doesn't need to handle the case of zero+// inputs, since no non-trivial table entries are zero. Note+// that in the zero case the whole negation is trivial, and+// so indeed is the swapping.++        cmpq    $0, bias++        movq    %rax, %rsi+        cmovnzq %r8, %rsi+        cmovnzq %rax, %r8+        movq    %rsi, TABENT(%rsp)+        movq    %r8, TABENT+32(%rsp)++        movq    %rbx, %rsi+        cmovnzq %r9, %rsi+        cmovnzq %rbx, %r9+        movq    %rsi, TABENT+8(%rsp)+        movq    %r9, TABENT+40(%rsp)++        movq    %rcx, %rsi+        cmovnzq %r10, %rsi+        cmovnzq %rcx, %r10+        movq    %rsi, TABENT+16(%rsp)+        movq    %r10, TABENT+48(%rsp)++        movq    %rdx, %rsi+        cmovnzq %r11, %rsi+        cmovnzq %rdx, %r11+        movq    %rsi, TABENT+24(%rsp)+        movq    %r11, TABENT+56(%rsp)++        movq    $-19, %rax+        movq    $-1, %rbx+        movq    $-1, %rcx+        movq    $0x7fffffffffffffff, %rdx+        subq    %r12, %rax+        sbbq    %r13, %rbx+        sbbq    %r14, %rcx+        sbbq    %r15, %rdx++        movq    ix, %r8+        movq    bias, %r9+        testq   %r8, %r8+        cmovzq  %r8, %r9+        testq   %r9, %r9++        cmovzq  %r12, %rax+        cmovzq  %r13, %rbx+        cmovzq  %r14, %rcx+        cmovzq  %r15, %rdx+        movq    %rax, TABENT+64(%rsp)+        movq    %rbx, TABENT+72(%rsp)+        movq    %rcx, TABENT+80(%rsp)+        movq    %rdx, TABENT+88(%rsp)++// Extended-projective and precomputed mixed addition.+// This is effectively the same as calling the standalone+// function edwards25519_pepadd(acc,acc,tabent), but we+// only retain slightly weaker normalization < 2 * p_25519+// throughout the inner loop, so the computation is+// slightly different, and faster overall.++        double_twice4(t0,z_1)+        sub_twice4(t1,y_1,x_1)+        add_twice4(t2,y_1,x_1)+        mul_4(t3,w_1,kxy_2)+        mul_4(t1,t1,ymx_2)+        mul_4(t2,t2,xpy_2)+        sub_twice4(t4,t0,t3)+        add_twice4(t0,t0,t3)+        sub_twice4(t5,t2,t1)+        add_twice4(t1,t2,t1)+        mul_4(z_3,t4,t0)+        mul_4(x_3,t5,t4)+        mul_4(y_3,t0,t1)+        mul_4(w_3,t5,t1)++// End of the main loop; move on by 4 bits.++        addq    $4, i+        cmpq    $252, i+        jc      Ledwards25519_scalarmulbase_scalarloop++// Insert the optional negation of the projective X coordinate, and+// so by extension the final affine x coordinate x = X/Z and thus+// the point P = (x,y). We only know X < 2 * p_25519, so we do the+// negation as 2 * p_25519 - X to keep it nonnegative. From this+// point on we don't need any normalization of the coordinates+// except for making sure that they fit in 4 digits.++        movq    X3(%rsp), %r8+        movq    X3+8(%rsp), %r9+        movq    X3+16(%rsp), %r10+        movq    X3+24(%rsp), %r11+        movq    $0xffffffffffffffda, %r12+        subq    %r8, %r12+        movq    $0xffffffffffffffff, %r13+        sbbq    %r9, %r13+        movq    $0xffffffffffffffff, %r14+        sbbq    %r10, %r14+        movq    $0xffffffffffffffff, %r15+        sbbq    %r11, %r15+        movq    SCALAR+24(%rsp), %rax+        btq     $63, %rax+        cmovcq  %r12, %r8+        cmovcq  %r13, %r9+        cmovcq  %r14, %r10+        cmovcq  %r15, %r11+        movq    %r8, X3(%rsp)+        movq    %r9, X3+8(%rsp)+        movq    %r10, X3+16(%rsp)+        movq    %r11, X3+24(%rsp)++// Now we need to map out of the extended-projective representation+// (X,Y,Z,W) back to the affine form (x,y) = (X/Z,Y/Z). This means+// first calling the modular inverse to get w_3 = 1/z_3.++        leaq    W3(%rsp), %rdi+        leaq    Z3(%rsp), %rsi++// Inline copy of bignum_inv_p25519, identical except for stripping out+// the prologue and epilogue saving and restoring registers and making+// and reclaiming room on the stack. For more details and explanations see+// "x86/curve25519/bignum_inv_p25519.S". Note that the stack it uses for+// its own temporaries is 208 bytes, so it has no effect on variables+// that are needed in the rest of our computation here: res, x_3, y_3,+// z_3 and w_3.++        movq    %rdi, 0xc0(%rsp)+        xorl    %eax, %eax+        leaq    -0x13(%rax), %rcx+        notq    %rax+        movq    %rcx, (%rsp)+        movq    %rax, 0x8(%rsp)+        movq    %rax, 0x10(%rsp)+        btr     $0x3f, %rax+        movq    %rax, 0x18(%rsp)+        movq    (%rsi), %rdx+        movq    0x8(%rsi), %rcx+        movq    0x10(%rsi), %r8+        movq    0x18(%rsi), %r9+        movl    $0x1, %eax+        xorl    %r10d, %r10d+        bts     $0x3f, %r9+        adcq    %r10, %rax+        imulq   $0x13, %rax, %rax+        addq    %rax, %rdx+        adcq    %r10, %rcx+        adcq    %r10, %r8+        adcq    %r10, %r9+        movl    $0x13, %eax+        cmovbq  %r10, %rax+        subq    %rax, %rdx+        sbbq    %r10, %rcx+        sbbq    %r10, %r8+        sbbq    %r10, %r9+        btr     $0x3f, %r9+        movq    %rdx, 0x20(%rsp)+        movq    %rcx, 0x28(%rsp)+        movq    %r8, 0x30(%rsp)+        movq    %r9, 0x38(%rsp)+        xorl    %eax, %eax+        movq    %rax, 0x40(%rsp)+        movq    %rax, 0x48(%rsp)+        movq    %rax, 0x50(%rsp)+        movq    %rax, 0x58(%rsp)+        movabsq $0xa0f99e2375022099, %rax+        movq    %rax, 0x60(%rsp)+        movabsq $0xa8c68f3f1d132595, %rax+        movq    %rax, 0x68(%rsp)+        movabsq $0x6c6c893805ac5242, %rax+        movq    %rax, 0x70(%rsp)+        movabsq $0x276508b241770615, %rax+        movq    %rax, 0x78(%rsp)+        movq    $0xa,  0x90(%rsp)+        movq    $0x1,  0x98(%rsp)+        jmp     Ledwards25519_scalarmulbase_midloop+Ledwards25519_scalarmulbase_inverseloop:+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %rdi+        andq    %r11, %rdi+        addq    %rax, %rdi+        movq    %rdi, 0x80(%rsp)+        movq    %r12, %rax+        andq    %r13, %rax+        movq    %r14, %rsi+        andq    %r15, %rsi+        addq    %rax, %rsi+        movq    %rsi, 0x88(%rsp)+        xorl    %ebx, %ebx+        movq    (%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x20(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    (%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x20(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        xorl    %ecx, %ecx+        movq    0x8(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x28(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, (%rsp)+        xorl    %edi, %edi+        movq    0x8(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    0x28(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $0x3b, %rbp, %rsi+        movq    %rsi, 0x20(%rsp)+        xorl    %esi, %esi+        movq    0x10(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    0x30(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rcx, %rbx+        movq    %rbx, 0x8(%rsp)+        xorl    %ebx, %ebx+        movq    0x10(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x30(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $0x3b, %rdi, %rbp+        movq    %rbp, 0x28(%rsp)+        movq    0x18(%rsp), %rax+        xorq    %r9, %rax+        movq    %rax, %rbp+        sarq    $0x3f, %rbp+        andq    %r8, %rbp+        negq    %rbp+        mulq    %r8+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x38(%rsp), %rax+        xorq    %r11, %rax+        movq    %rax, %rdx+        sarq    $0x3f, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbp+        mulq    %r10+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        shrdq   $0x3b, %rsi, %rcx+        movq    %rcx, 0x10(%rsp)+        shrdq   $0x3b, %rbp, %rsi+        movq    0x18(%rsp), %rax+        movq    %rsi, 0x18(%rsp)+        xorq    %r13, %rax+        movq    %rax, %rsi+        sarq    $0x3f, %rsi+        andq    %r12, %rsi+        negq    %rsi+        mulq    %r12+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        movq    0x38(%rsp), %rax+        xorq    %r15, %rax+        movq    %rax, %rdx+        sarq    $0x3f, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rsi+        mulq    %r14+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, 0x30(%rsp)+        shrdq   $0x3b, %rsi, %rbx+        movq    %rbx, 0x38(%rsp)+        movq    0x80(%rsp), %rbx+        movq    0x88(%rsp), %rbp+        xorl    %ecx, %ecx+        movq    0x40(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x60(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x40(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x40(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x60(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x60(%rsp)+        xorl    %ebx, %ebx+        movq    0x48(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x68(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    0x48(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, 0x48(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x68(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, 0x68(%rsp)+        xorl    %ecx, %ecx+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x70(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x50(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x50(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x70(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x70(%rsp)+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        movq    %r9, %rbx+        andq    %r8, %rbx+        negq    %rbx+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbx+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rbx, %rdx+        movq    %rdx, %rbx+        shldq   $0x1, %rcx, %rdx+        sarq    $0x3f, %rbx+        addq    %rbx, %rdx+        movl    $0x13, %eax+        imulq   %rdx+        movq    0x40(%rsp), %r8+        addq    %rax, %r8+        movq    %r8, 0x40(%rsp)+        movq    0x48(%rsp), %r8+        adcq    %rdx, %r8+        movq    %r8, 0x48(%rsp)+        movq    0x50(%rsp), %r8+        adcq    %rbx, %r8+        movq    %r8, 0x50(%rsp)+        adcq    %rbx, %rcx+        shlq    $0x3f, %rax+        addq    %rax, %rcx+        movq    0x58(%rsp), %rax+        movq    %rcx, 0x58(%rsp)+        xorq    %r13, %rax+        movq    %r13, %rcx+        andq    %r12, %rcx+        negq    %rcx+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rcx+        movq    0x78(%rsp), %rax+        xorq    %r15, %rax+        movq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rcx+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rcx, %rdx+        movq    %rdx, %rcx+        shldq   $0x1, %rsi, %rdx+        sarq    $0x3f, %rcx+        movl    $0x13, %eax+        addq    %rcx, %rdx+        imulq   %rdx+        movq    0x60(%rsp), %r8+        addq    %rax, %r8+        movq    %r8, 0x60(%rsp)+        movq    0x68(%rsp), %r8+        adcq    %rdx, %r8+        movq    %r8, 0x68(%rsp)+        movq    0x70(%rsp), %r8+        adcq    %rcx, %r8+        movq    %r8, 0x70(%rsp)+        adcq    %rcx, %rsi+        shlq    $0x3f, %rax+        addq    %rax, %rsi+        movq    %rsi, 0x78(%rsp)+Ledwards25519_scalarmulbase_midloop:+        movq    0x98(%rsp), %rsi+        movq    (%rsp), %rdx+        movq    0x20(%rsp), %rcx+        movq    %rdx, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        xorl    %ebp, %ebp+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %rdx+        leaq    (%rcx,%rax), %rdi+        shlq    $0x16, %rdx+        shlq    $0x16, %rdi+        sarq    $0x2b, %rdx+        sarq    $0x2b, %rdi+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %rbx+        leaq    (%rcx,%rax), %rcx+        sarq    $0x2a, %rbx+        sarq    $0x2a, %rcx+        movq    %rdx, 0xa0(%rsp)+        movq    %rbx, 0xa8(%rsp)+        movq    %rdi, 0xb0(%rsp)+        movq    %rcx, 0xb8(%rsp)+        movq    (%rsp), %r12+        imulq   %r12, %rdi+        imulq   %rdx, %r12+        movq    0x20(%rsp), %r13+        imulq   %r13, %rbx+        imulq   %rcx, %r13+        addq    %rbx, %r12+        addq    %rdi, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r10+        shlq    $0x16, %r8+        shlq    $0x16, %r10+        sarq    $0x2b, %r8+        sarq    $0x2b, %r10+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r15+        leaq    (%rcx,%rax), %r11+        sarq    $0x2a, %r15+        sarq    $0x2a, %r11+        movq    %r13, %rbx+        movq    %r12, %rcx+        imulq   %r8, %r12+        imulq   %r15, %rbx+        addq    %rbx, %r12+        imulq   %r11, %r13+        imulq   %r10, %rcx+        addq    %rcx, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    0xa0(%rsp), %rax+        imulq   %r8, %rax+        movq    0xb0(%rsp), %rdx+        imulq   %r15, %rdx+        imulq   0xa8(%rsp), %r8+        imulq   0xb8(%rsp), %r15+        addq    %r8, %r15+        leaq    (%rax,%rdx), %r9+        movq    0xa0(%rsp), %rax+        imulq   %r10, %rax+        movq    0xb0(%rsp), %rdx+        imulq   %r11, %rdx+        imulq   0xa8(%rsp), %r10+        imulq   0xb8(%rsp), %r11+        addq    %r10, %r11+        leaq    (%rax,%rdx), %r13+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r12+        shlq    $0x15, %r8+        shlq    $0x15, %r12+        sarq    $0x2b, %r8+        sarq    $0x2b, %r12+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r10+        leaq    (%rcx,%rax), %r14+        sarq    $0x2b, %r10+        sarq    $0x2b, %r14+        movq    %r9, %rax+        imulq   %r8, %rax+        movq    %r13, %rdx+        imulq   %r10, %rdx+        imulq   %r15, %r8+        imulq   %r11, %r10+        addq    %r8, %r10+        leaq    (%rax,%rdx), %r8+        movq    %r9, %rax+        imulq   %r12, %rax+        movq    %r13, %rdx+        imulq   %r14, %rdx+        imulq   %r15, %r12+        imulq   %r11, %r14+        addq    %r12, %r14+        leaq    (%rax,%rdx), %r12+        movq    %rsi, 0x98(%rsp)+        decq     0x90(%rsp)+        jne     Ledwards25519_scalarmulbase_inverseloop+        movq    (%rsp), %rax+        movq    0x20(%rsp), %rcx+        imulq   %r8, %rax+        imulq   %r10, %rcx+        addq    %rcx, %rax+        sarq    $0x3f, %rax+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        xorq    %rax, %r9+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        xorq    %rax, %r11+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        xorq    %rax, %r13+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        xorq    %rax, %r15+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %r12+        andq    %r11, %r12+        addq    %rax, %r12+        xorl    %r13d, %r13d+        movq    0x40(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x60(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movq    0x48(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r13+        adcq    %rdx, %r14+        movq    0x68(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x70(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        andq    %r8, %r9+        negq    %r9+        mulq    %r8+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %r9+        mulq    %r10+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    %r9, %rax+        shldq   $0x1, %r15, %rax+        sarq    $0x3f, %r9+        movl    $0x13, %ebx+        leaq    0x1(%rax,%r9,1), %rax+        imulq   %rbx+        xorl    %ebp, %ebp+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r9, %r14+        adcq    %r9, %r15+        shlq    $0x3f, %rax+        addq    %rax, %r15+        cmovns  %rbp, %rbx+        subq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    %rbp, %r14+        sbbq    %rbp, %r15+        btr     $0x3f, %r15+        movq    0xc0(%rsp), %rdi+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)++// The final result is x = X * inv(Z), y = Y * inv(Z).+// These are the only operations in the whole computation that+// fully reduce modulo p_25519 since now we want the canonical+// answer as output.++        movq    res, %rbp+        mul_p25519(resx,x_3,w_3)+        mul_p25519(resy,y_3,w_3)++// Restore stack and registers++        CFI_INC_RSP(NSPACE)++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++#if WINDOWS_ABI+S2N_BN_SIZE_DIRECTIVE(Ledwards25519_scalarmulbase_standard)+#else+S2N_BN_SIZE_DIRECTIVE(edwards25519_scalarmulbase)+#endif++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif++// ****************************************************************************+// The precomputed data (all read-only).+// ****************************************************************************++#if defined(__ELF__)+.section .rodata+.type S2N_BN_SYMBOL(edwards25519_scalarmulbase_constant), %object+.size S2N_BN_SYMBOL(edwards25519_scalarmulbase_constant), 48576+#elif defined(__APPLE__)+.const_data+#endif++S2N_BN_SYMBOL(edwards25519_scalarmulbase_constant):++// 0 * B = 0 and 2^251 * B in extended-projective coordinates+// but with Z = 1 assumed and hence left out, so they are (X,Y,T) only.++        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000++        .quad   0x0000000000000001+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000++        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000++        .quad   0x525f946d7c7220e7+        .quad   0x4636b0b2f1e35444+        .quad   0x796e9d70e892ae0f+        .quad   0x03dec05fa937adb1+        .quad   0x6d1c271cc6375515+        .quad   0x462588c4a4ca4f14+        .quad   0x691129fee55afc39+        .quad   0x15949f784d8472f5+        .quad   0xbd89e510afad0049+        .quad   0x4d1f08c073b9860e+        .quad   0x07716e8b2d00af9d+        .quad   0x70d685f68f859714++// Precomputed table of multiples of generator for edwards25519+// all in precomputed extended-projective (y-x,x+y,2*d*x*y) triples.++        // 2^0 * 1 * G++        .quad   0x9d103905d740913e+        .quad   0xfd399f05d140beb3+        .quad   0xa5c18434688f8a09+        .quad   0x44fd2f9298f81267+        .quad   0x2fbc93c6f58c3b85+        .quad   0xcf932dc6fb8c0e19+        .quad   0x270b4898643d42c2+        .quad   0x07cf9d3a33d4ba65+        .quad   0xabc91205877aaa68+        .quad   0x26d9e823ccaac49e+        .quad   0x5a1b7dcbdd43598c+        .quad   0x6f117b689f0c65a8++        // 2^0 * 2 * G++        .quad   0x8a99a56042b4d5a8+        .quad   0x8f2b810c4e60acf6+        .quad   0xe09e236bb16e37aa+        .quad   0x6bb595a669c92555+        .quad   0x9224e7fc933c71d7+        .quad   0x9f469d967a0ff5b5+        .quad   0x5aa69a65e1d60702+        .quad   0x590c063fa87d2e2e+        .quad   0x43faa8b3a59b7a5f+        .quad   0x36c16bdd5d9acf78+        .quad   0x500fa0840b3d6a31+        .quad   0x701af5b13ea50b73++        // 2^0 * 3 * G++        .quad   0x56611fe8a4fcd265+        .quad   0x3bd353fde5c1ba7d+        .quad   0x8131f31a214bd6bd+        .quad   0x2ab91587555bda62+        .quad   0xaf25b0a84cee9730+        .quad   0x025a8430e8864b8a+        .quad   0xc11b50029f016732+        .quad   0x7a164e1b9a80f8f4+        .quad   0x14ae933f0dd0d889+        .quad   0x589423221c35da62+        .quad   0xd170e5458cf2db4c+        .quad   0x5a2826af12b9b4c6++        // 2^0 * 4 * G++        .quad   0x95fe050a056818bf+        .quad   0x327e89715660faa9+        .quad   0xc3e8e3cd06a05073+        .quad   0x27933f4c7445a49a+        .quad   0x287351b98efc099f+        .quad   0x6765c6f47dfd2538+        .quad   0xca348d3dfb0a9265+        .quad   0x680e910321e58727+        .quad   0x5a13fbe9c476ff09+        .quad   0x6e9e39457b5cc172+        .quad   0x5ddbdcf9102b4494+        .quad   0x7f9d0cbf63553e2b++        // 2^0 * 5 * G++        .quad   0x7f9182c3a447d6ba+        .quad   0xd50014d14b2729b7+        .quad   0xe33cf11cb864a087+        .quad   0x154a7e73eb1b55f3+        .quad   0xa212bc4408a5bb33+        .quad   0x8d5048c3c75eed02+        .quad   0xdd1beb0c5abfec44+        .quad   0x2945ccf146e206eb+        .quad   0xbcbbdbf1812a8285+        .quad   0x270e0807d0bdd1fc+        .quad   0xb41b670b1bbda72d+        .quad   0x43aabe696b3bb69a++        // 2^0 * 6 * G++        .quad   0x499806b67b7d8ca4+        .quad   0x575be28427d22739+        .quad   0xbb085ce7204553b9+        .quad   0x38b64c41ae417884+        .quad   0x3a0ceeeb77157131+        .quad   0x9b27158900c8af88+        .quad   0x8065b668da59a736+        .quad   0x51e57bb6a2cc38bd+        .quad   0x85ac326702ea4b71+        .quad   0xbe70e00341a1bb01+        .quad   0x53e4a24b083bc144+        .quad   0x10b8e91a9f0d61e3++        // 2^0 * 7 * G++        .quad   0xba6f2c9aaa3221b1+        .quad   0x6ca021533bba23a7+        .quad   0x9dea764f92192c3a+        .quad   0x1d6edd5d2e5317e0+        .quad   0x6b1a5cd0944ea3bf+        .quad   0x7470353ab39dc0d2+        .quad   0x71b2528228542e49+        .quad   0x461bea69283c927e+        .quad   0xf1836dc801b8b3a2+        .quad   0xb3035f47053ea49a+        .quad   0x529c41ba5877adf3+        .quad   0x7a9fbb1c6a0f90a7++        // 2^0 * 8 * G++        .quad   0xe2a75dedf39234d9+        .quad   0x963d7680e1b558f9+        .quad   0x2c2741ac6e3c23fb+        .quad   0x3a9024a1320e01c3+        .quad   0x59b7596604dd3e8f+        .quad   0x6cb30377e288702c+        .quad   0xb1339c665ed9c323+        .quad   0x0915e76061bce52f+        .quad   0xe7c1f5d9c9a2911a+        .quad   0xb8a371788bcca7d7+        .quad   0x636412190eb62a32+        .quad   0x26907c5c2ecc4e95++        // 2^4 * 1 * G++        .quad   0x7ec851ca553e2df3+        .quad   0xa71284cba64878b3+        .quad   0xe6b5e4193288d1e7+        .quad   0x4cf210ec5a9a8883+        .quad   0x322d04a52d9021f6+        .quad   0xb9c19f3375c6bf9c+        .quad   0x587a3a4342d20b09+        .quad   0x143b1cf8aa64fe61+        .quad   0x9f867c7d968acaab+        .quad   0x5f54258e27092729+        .quad   0xd0a7d34bea180975+        .quad   0x21b546a3374126e1++        // 2^4 * 2 * G++        .quad   0xa94ff858a2888343+        .quad   0xce0ed4565313ed3c+        .quad   0xf55c3dcfb5bf34fa+        .quad   0x0a653ca5c9eab371+        .quad   0x490a7a45d185218f+        .quad   0x9a15377846049335+        .quad   0x0060ea09cc31e1f6+        .quad   0x7e041577f86ee965+        .quad   0x66b2a496ce5b67f3+        .quad   0xff5492d8bd569796+        .quad   0x503cec294a592cd0+        .quad   0x566943650813acb2++        // 2^4 * 3 * G++        .quad   0xb818db0c26620798+        .quad   0x5d5c31d9606e354a+        .quad   0x0982fa4f00a8cdc7+        .quad   0x17e12bcd4653e2d4+        .quad   0x5672f9eb1dabb69d+        .quad   0xba70b535afe853fc+        .quad   0x47ac0f752796d66d+        .quad   0x32a5351794117275+        .quad   0xd3a644a6df648437+        .quad   0x703b6559880fbfdd+        .quad   0xcb852540ad3a1aa5+        .quad   0x0900b3f78e4c6468++        // 2^4 * 4 * G++        .quad   0x0a851b9f679d651b+        .quad   0xe108cb61033342f2+        .quad   0xd601f57fe88b30a3+        .quad   0x371f3acaed2dd714+        .quad   0xed280fbec816ad31+        .quad   0x52d9595bd8e6efe3+        .quad   0x0fe71772f6c623f5+        .quad   0x4314030b051e293c+        .quad   0xd560005efbf0bcad+        .quad   0x8eb70f2ed1870c5e+        .quad   0x201f9033d084e6a0+        .quad   0x4c3a5ae1ce7b6670++        // 2^4 * 5 * G++        .quad   0x4138a434dcb8fa95+        .quad   0x870cf67d6c96840b+        .quad   0xde388574297be82c+        .quad   0x7c814db27262a55a+        .quad   0xbaf875e4c93da0dd+        .quad   0xb93282a771b9294d+        .quad   0x80d63fb7f4c6c460+        .quad   0x6de9c73dea66c181+        .quad   0x478904d5a04df8f2+        .quad   0xfafbae4ab10142d3+        .quad   0xf6c8ac63555d0998+        .quad   0x5aac4a412f90b104++        // 2^4 * 6 * G++        .quad   0xc64f326b3ac92908+        .quad   0x5551b282e663e1e0+        .quad   0x476b35f54a1a4b83+        .quad   0x1b9da3fe189f68c2+        .quad   0x603a0d0abd7f5134+        .quad   0x8089c932e1d3ae46+        .quad   0xdf2591398798bd63+        .quad   0x1c145cd274ba0235+        .quad   0x32e8386475f3d743+        .quad   0x365b8baf6ae5d9ef+        .quad   0x825238b6385b681e+        .quad   0x234929c1167d65e1++        // 2^4 * 7 * G++        .quad   0x984decaba077ade8+        .quad   0x383f77ad19eb389d+        .quad   0xc7ec6b7e2954d794+        .quad   0x59c77b3aeb7c3a7a+        .quad   0x48145cc21d099fcf+        .quad   0x4535c192cc28d7e5+        .quad   0x80e7c1e548247e01+        .quad   0x4a5f28743b2973ee+        .quad   0xd3add725225ccf62+        .quad   0x911a3381b2152c5d+        .quad   0xd8b39fad5b08f87d+        .quad   0x6f05606b4799fe3b++        // 2^4 * 8 * G++        .quad   0x9ffe9e92177ba962+        .quad   0x98aee71d0de5cae1+        .quad   0x3ff4ae942d831044+        .quad   0x714de12e58533ac8+        .quad   0x5b433149f91b6483+        .quad   0xadb5dc655a2cbf62+        .quad   0x87fa8412632827b3+        .quad   0x60895e91ab49f8d8+        .quad   0xe9ecf2ed0cf86c18+        .quad   0xb46d06120735dfd4+        .quad   0xbc9da09804b96be7+        .quad   0x73e2e62fd96dc26b++        // 2^8 * 1 * G++        .quad   0xed5b635449aa515e+        .quad   0xa865c49f0bc6823a+        .quad   0x850c1fe95b42d1c4+        .quad   0x30d76d6f03d315b9+        .quad   0x2eccdd0e632f9c1d+        .quad   0x51d0b69676893115+        .quad   0x52dfb76ba8637a58+        .quad   0x6dd37d49a00eef39+        .quad   0x6c4444172106e4c7+        .quad   0xfb53d680928d7f69+        .quad   0xb4739ea4694d3f26+        .quad   0x10c697112e864bb0++        // 2^8 * 2 * G++        .quad   0x6493c4277dbe5fde+        .quad   0x265d4fad19ad7ea2+        .quad   0x0e00dfc846304590+        .quad   0x25e61cabed66fe09+        .quad   0x0ca62aa08358c805+        .quad   0x6a3d4ae37a204247+        .quad   0x7464d3a63b11eddc+        .quad   0x03bf9baf550806ef+        .quad   0x3f13e128cc586604+        .quad   0x6f5873ecb459747e+        .quad   0xa0b63dedcc1268f5+        .quad   0x566d78634586e22c++        // 2^8 * 3 * G++        .quad   0x1637a49f9cc10834+        .quad   0xbc8e56d5a89bc451+        .quad   0x1cb5ec0f7f7fd2db+        .quad   0x33975bca5ecc35d9+        .quad   0xa1054285c65a2fd0+        .quad   0x6c64112af31667c3+        .quad   0x680ae240731aee58+        .quad   0x14fba5f34793b22a+        .quad   0x3cd746166985f7d4+        .quad   0x593e5e84c9c80057+        .quad   0x2fc3f2b67b61131e+        .quad   0x14829cea83fc526c++        // 2^8 * 4 * G++        .quad   0xff437b8497dd95c2+        .quad   0x6c744e30aa4eb5a7+        .quad   0x9e0c5d613c85e88b+        .quad   0x2fd9c71e5f758173+        .quad   0x21e70b2f4e71ecb8+        .quad   0xe656ddb940a477e3+        .quad   0xbf6556cece1d4f80+        .quad   0x05fc3bc4535d7b7e+        .quad   0x24b8b3ae52afdedd+        .quad   0x3495638ced3b30cf+        .quad   0x33a4bc83a9be8195+        .quad   0x373767475c651f04++        // 2^8 * 5 * G++        .quad   0x2fba99fd40d1add9+        .quad   0xb307166f96f4d027+        .quad   0x4363f05215f03bae+        .quad   0x1fbea56c3b18f999+        .quad   0x634095cb14246590+        .quad   0xef12144016c15535+        .quad   0x9e38140c8910bc60+        .quad   0x6bf5905730907c8c+        .quad   0x0fa778f1e1415b8a+        .quad   0x06409ff7bac3a77e+        .quad   0x6f52d7b89aa29a50+        .quad   0x02521cf67a635a56++        // 2^8 * 6 * G++        .quad   0x513fee0b0a9d5294+        .quad   0x8f98e75c0fdf5a66+        .quad   0xd4618688bfe107ce+        .quad   0x3fa00a7e71382ced+        .quad   0xb1146720772f5ee4+        .quad   0xe8f894b196079ace+        .quad   0x4af8224d00ac824a+        .quad   0x001753d9f7cd6cc4+        .quad   0x3c69232d963ddb34+        .quad   0x1dde87dab4973858+        .quad   0xaad7d1f9a091f285+        .quad   0x12b5fe2fa048edb6++        // 2^8 * 7 * G++        .quad   0x71f0fbc496fce34d+        .quad   0x73b9826badf35bed+        .quad   0xd2047261ff28c561+        .quad   0x749b76f96fb1206f+        .quad   0xdf2b7c26ad6f1e92+        .quad   0x4b66d323504b8913+        .quad   0x8c409dc0751c8bc3+        .quad   0x6f7e93c20796c7b8+        .quad   0x1f5af604aea6ae05+        .quad   0xc12351f1bee49c99+        .quad   0x61a808b5eeff6b66+        .quad   0x0fcec10f01e02151++        // 2^8 * 8 * G++        .quad   0x644d58a649fe1e44+        .quad   0x21fcaea231ad777e+        .quad   0x02441c5a887fd0d2+        .quad   0x4901aa7183c511f3+        .quad   0x3df2d29dc4244e45+        .quad   0x2b020e7493d8de0a+        .quad   0x6cc8067e820c214d+        .quad   0x413779166feab90a+        .quad   0x08b1b7548c1af8f0+        .quad   0xce0f7a7c246299b4+        .quad   0xf760b0f91e06d939+        .quad   0x41bb887b726d1213++        // 2^12 * 1 * G++        .quad   0x9267806c567c49d8+        .quad   0x066d04ccca791e6a+        .quad   0xa69f5645e3cc394b+        .quad   0x5c95b686a0788cd2+        .quad   0x97d980e0aa39f7d2+        .quad   0x35d0384252c6b51c+        .quad   0x7d43f49307cd55aa+        .quad   0x56bd36cfb78ac362+        .quad   0x2ac519c10d14a954+        .quad   0xeaf474b494b5fa90+        .quad   0xe6af8382a9f87a5a+        .quad   0x0dea6db1879be094++        // 2^12 * 2 * G++        .quad   0xaa66bf547344e5ab+        .quad   0xda1258888f1b4309+        .quad   0x5e87d2b3fd564b2f+        .quad   0x5b2c78885483b1dd+        .quad   0x15baeb74d6a8797a+        .quad   0x7ef55cf1fac41732+        .quad   0x29001f5a3c8b05c5+        .quad   0x0ad7cc8752eaccfb+        .quad   0x52151362793408cf+        .quad   0xeb0f170319963d94+        .quad   0xa833b2fa883d9466+        .quad   0x093a7fa775003c78++        // 2^12 * 3 * G++        .quad   0xe5107de63a16d7be+        .quad   0xa377ffdc9af332cf+        .quad   0x70d5bf18440b677f+        .quad   0x6a252b19a4a31403+        .quad   0xb8e9604460a91286+        .quad   0x7f3fd8047778d3de+        .quad   0x67d01e31bf8a5e2d+        .quad   0x7b038a06c27b653e+        .quad   0x9ed919d5d36990f3+        .quad   0x5213aebbdb4eb9f2+        .quad   0xc708ea054cb99135+        .quad   0x58ded57f72260e56++        // 2^12 * 4 * G++        .quad   0x78e79dade9413d77+        .quad   0xf257f9d59729e67d+        .quad   0x59db910ee37aa7e6+        .quad   0x6aa11b5bbb9e039c+        .quad   0xda6d53265b0fd48b+        .quad   0x8960823193bfa988+        .quad   0xd78ac93261d57e28+        .quad   0x79f2942d3a5c8143+        .quad   0x97da2f25b6c88de9+        .quad   0x251ba7eaacf20169+        .quad   0x09b44f87ef4eb4e4+        .quad   0x7d90ab1bbc6a7da5++        // 2^12 * 5 * G++        .quad   0x9acca683a7016bfe+        .quad   0x90505f4df2c50b6d+        .quad   0x6b610d5fcce435aa+        .quad   0x19a10d446198ff96+        .quad   0x1a07a3f496b3c397+        .quad   0x11ceaa188f4e2532+        .quad   0x7d9498d5a7751bf0+        .quad   0x19ed161f508dd8a0+        .quad   0x560a2cd687dce6ca+        .quad   0x7f3568c48664cf4d+        .quad   0x8741e95222803a38+        .quad   0x483bdab1595653fc++        // 2^12 * 6 * G++        .quad   0xfa780f148734fa49+        .quad   0x106f0b70360534e0+        .quad   0x2210776fe3e307bd+        .quad   0x3286c109dde6a0fe+        .quad   0xd6cf4d0ab4da80f6+        .quad   0x82483e45f8307fe0+        .quad   0x05005269ae6f9da4+        .quad   0x1c7052909cf7877a+        .quad   0x32ee7de2874e98d4+        .quad   0x14c362e9b97e0c60+        .quad   0x5781dcde6a60a38a+        .quad   0x217dd5eaaa7aa840++        // 2^12 * 7 * G++        .quad   0x9db7c4d0248e1eb0+        .quad   0xe07697e14d74bf52+        .quad   0x1e6a9b173c562354+        .quad   0x7fa7c21f795a4965+        .quad   0x8bdf1fb9be8c0ec8+        .quad   0x00bae7f8e30a0282+        .quad   0x4963991dad6c4f6c+        .quad   0x07058a6e5df6f60a+        .quad   0xe9eb02c4db31f67f+        .quad   0xed25fd8910bcfb2b+        .quad   0x46c8131f5c5cddb4+        .quad   0x33b21c13a0cb9bce++        // 2^12 * 8 * G++        .quad   0x360692f8087d8e31+        .quad   0xf4dcc637d27163f7+        .quad   0x25a4e62065ea5963+        .quad   0x659bf72e5ac160d9+        .quad   0x9aafb9b05ee38c5b+        .quad   0xbf9d2d4e071a13c7+        .quad   0x8eee6e6de933290a+        .quad   0x1c3bab17ae109717+        .quad   0x1c9ab216c7cab7b0+        .quad   0x7d65d37407bbc3cc+        .quad   0x52744750504a58d5+        .quad   0x09f2606b131a2990++        // 2^16 * 1 * G++        .quad   0x40e87d44744346be+        .quad   0x1d48dad415b52b25+        .quad   0x7c3a8a18a13b603e+        .quad   0x4eb728c12fcdbdf7+        .quad   0x7e234c597c6691ae+        .quad   0x64889d3d0a85b4c8+        .quad   0xdae2c90c354afae7+        .quad   0x0a871e070c6a9e1d+        .quad   0x3301b5994bbc8989+        .quad   0x736bae3a5bdd4260+        .quad   0x0d61ade219d59e3c+        .quad   0x3ee7300f2685d464++        // 2^16 * 2 * G++        .quad   0xf5d255e49e7dd6b7+        .quad   0x8016115c610b1eac+        .quad   0x3c99975d92e187ca+        .quad   0x13815762979125c2+        .quad   0x43fa7947841e7518+        .quad   0xe5c6fa59639c46d7+        .quad   0xa1065e1de3052b74+        .quad   0x7d47c6a2cfb89030+        .quad   0x3fdad0148ef0d6e0+        .quad   0x9d3e749a91546f3c+        .quad   0x71ec621026bb8157+        .quad   0x148cf58d34c9ec80++        // 2^16 * 3 * G++        .quad   0x46a492f67934f027+        .quad   0x469984bef6840aa9+        .quad   0x5ca1bc2a89611854+        .quad   0x3ff2fa1ebd5dbbd4+        .quad   0xe2572f7d9ae4756d+        .quad   0x56c345bb88f3487f+        .quad   0x9fd10b6d6960a88d+        .quad   0x278febad4eaea1b9+        .quad   0xb1aa681f8c933966+        .quad   0x8c21949c20290c98+        .quad   0x39115291219d3c52+        .quad   0x4104dd02fe9c677b++        // 2^16 * 4 * G++        .quad   0x72b2bf5e1124422a+        .quad   0xa1fa0c3398a33ab5+        .quad   0x94cb6101fa52b666+        .quad   0x2c863b00afaf53d5+        .quad   0x81214e06db096ab8+        .quad   0x21a8b6c90ce44f35+        .quad   0x6524c12a409e2af5+        .quad   0x0165b5a48efca481+        .quad   0xf190a474a0846a76+        .quad   0x12eff984cd2f7cc0+        .quad   0x695e290658aa2b8f+        .quad   0x591b67d9bffec8b8++        // 2^16 * 5 * G++        .quad   0x312f0d1c80b49bfa+        .quad   0x5979515eabf3ec8a+        .quad   0x727033c09ef01c88+        .quad   0x3de02ec7ca8f7bcb+        .quad   0x99b9b3719f18b55d+        .quad   0xe465e5faa18c641e+        .quad   0x61081136c29f05ed+        .quad   0x489b4f867030128b+        .quad   0xd232102d3aeb92ef+        .quad   0xe16253b46116a861+        .quad   0x3d7eabe7190baa24+        .quad   0x49f5fbba496cbebf++        // 2^16 * 6 * G++        .quad   0x30949a108a5bcfd4+        .quad   0xdc40dd70bc6473eb+        .quad   0x92c294c1307c0d1c+        .quad   0x5604a86dcbfa6e74+        .quad   0x155d628c1e9c572e+        .quad   0x8a4d86acc5884741+        .quad   0x91a352f6515763eb+        .quad   0x06a1a6c28867515b+        .quad   0x7288d1d47c1764b6+        .quad   0x72541140e0418b51+        .quad   0x9f031a6018acf6d1+        .quad   0x20989e89fe2742c6++        // 2^16 * 7 * G++        .quad   0x499777fd3a2dcc7f+        .quad   0x32857c2ca54fd892+        .quad   0xa279d864d207e3a0+        .quad   0x0403ed1d0ca67e29+        .quad   0x1674278b85eaec2e+        .quad   0x5621dc077acb2bdf+        .quad   0x640a4c1661cbf45a+        .quad   0x730b9950f70595d3+        .quad   0xc94b2d35874ec552+        .quad   0xc5e6c8cf98246f8d+        .quad   0xf7cb46fa16c035ce+        .quad   0x5bd7454308303dcc++        // 2^16 * 8 * G++        .quad   0x7f9ad19528b24cc2+        .quad   0x7f6b54656335c181+        .quad   0x66b8b66e4fc07236+        .quad   0x133a78007380ad83+        .quad   0x85c4932115e7792a+        .quad   0xc64c89a2bdcdddc9+        .quad   0x9d1e3da8ada3d762+        .quad   0x5bb7db123067f82c+        .quad   0x0961f467c6ca62be+        .quad   0x04ec21d6211952ee+        .quad   0x182360779bd54770+        .quad   0x740dca6d58f0e0d2++        // 2^20 * 1 * G++        .quad   0x50b70bf5d3f0af0b+        .quad   0x4feaf48ae32e71f7+        .quad   0x60e84ed3a55bbd34+        .quad   0x00ed489b3f50d1ed+        .quad   0x3906c72aed261ae5+        .quad   0x9ab68fd988e100f7+        .quad   0xf5e9059af3360197+        .quad   0x0e53dc78bf2b6d47+        .quad   0xb90829bf7971877a+        .quad   0x5e4444636d17e631+        .quad   0x4d05c52e18276893+        .quad   0x27632d9a5a4a4af5++        // 2^20 * 2 * G++        .quad   0xd11ff05154b260ce+        .quad   0xd86dc38e72f95270+        .quad   0x601fcd0d267cc138+        .quad   0x2b67916429e90ccd+        .quad   0xa98285d187eaffdb+        .quad   0xa5b4fbbbd8d0a864+        .quad   0xb658f27f022663f7+        .quad   0x3bbc2b22d99ce282+        .quad   0xb917c952583c0a58+        .quad   0x653ff9b80fe4c6f3+        .quad   0x9b0da7d7bcdf3c0c+        .quad   0x43a0eeb6ab54d60e++        // 2^20 * 3 * G++        .quad   0x396966a46d4a5487+        .quad   0xf811a18aac2bb3ba+        .quad   0x66e4685b5628b26b+        .quad   0x70a477029d929b92+        .quad   0x3ac6322357875fe8+        .quad   0xd9d4f4ecf5fbcb8f+        .quad   0x8dee8493382bb620+        .quad   0x50c5eaa14c799fdc+        .quad   0xdd0edc8bd6f2fb3c+        .quad   0x54c63aa79cc7b7a0+        .quad   0xae0b032b2c8d9f1a+        .quad   0x6f9ce107602967fb++        // 2^20 * 4 * G++        .quad   0xad1054b1cde1c22a+        .quad   0xc4a8e90248eb32df+        .quad   0x5f3e7b33accdc0ea+        .quad   0x72364713fc79963e+        .quad   0x139693063520e0b5+        .quad   0x437fcf7c88ea03fe+        .quad   0xf7d4c40bd3c959bc+        .quad   0x699154d1f893ded9+        .quad   0x315d5c75b4b27526+        .quad   0xcccb842d0236daa5+        .quad   0x22f0c8a3345fee8e+        .quad   0x73975a617d39dbed++        // 2^20 * 5 * G++        .quad   0xe4024df96375da10+        .quad   0x78d3251a1830c870+        .quad   0x902b1948658cd91c+        .quad   0x7e18b10b29b7438a+        .quad   0x6f37f392f4433e46+        .quad   0x0e19b9a11f566b18+        .quad   0x220fb78a1fd1d662+        .quad   0x362a4258a381c94d+        .quad   0x9071d9132b6beb2f+        .quad   0x0f26e9ad28418247+        .quad   0xeab91ec9bdec925d+        .quad   0x4be65bc8f48af2de++        // 2^20 * 6 * G++        .quad   0x78487feba36e7028+        .quad   0x5f3f13001dd8ce34+        .quad   0x934fb12d4b30c489+        .quad   0x056c244d397f0a2b+        .quad   0x1d50fba257c26234+        .quad   0x7bd4823adeb0678b+        .quad   0xc2b0dc6ea6538af5+        .quad   0x5665eec6351da73e+        .quad   0xdb3ee00943bfb210+        .quad   0x4972018720800ac2+        .quad   0x26ab5d6173bd8667+        .quad   0x20b209c2ab204938++        // 2^20 * 7 * G++        .quad   0x549e342ac07fb34b+        .quad   0x02d8220821373d93+        .quad   0xbc262d70acd1f567+        .quad   0x7a92c9fdfbcac784+        .quad   0x1fcca94516bd3289+        .quad   0x448d65aa41420428+        .quad   0x59c3b7b216a55d62+        .quad   0x49992cc64e612cd8+        .quad   0x65bd1bea70f801de+        .quad   0x1befb7c0fe49e28a+        .quad   0xa86306cdb1b2ae4a+        .quad   0x3b7ac0cd265c2a09++        // 2^20 * 8 * G++        .quad   0x822bee438c01bcec+        .quad   0x530cb525c0fbc73b+        .quad   0x48519034c1953fe9+        .quad   0x265cc261e09a0f5b+        .quad   0xf0d54e4f22ed39a7+        .quad   0xa2aae91e5608150a+        .quad   0xf421b2e9eddae875+        .quad   0x31bc531d6b7de992+        .quad   0xdf3d134da980f971+        .quad   0x7a4fb8d1221a22a7+        .quad   0x3df7d42035aad6d8+        .quad   0x2a14edcc6a1a125e++        // 2^24 * 1 * G++        .quad   0xdf48ee0752cfce4e+        .quad   0xc3fffaf306ec08b7+        .quad   0x05710b2ab95459c4+        .quad   0x161d25fa963ea38d+        .quad   0x231a8c570478433c+        .quad   0xb7b5270ec281439d+        .quad   0xdbaa99eae3d9079f+        .quad   0x2c03f5256c2b03d9+        .quad   0x790f18757b53a47d+        .quad   0x307b0130cf0c5879+        .quad   0x31903d77257ef7f9+        .quad   0x699468bdbd96bbaf++        // 2^24 * 2 * G++        .quad   0xbd1f2f46f4dafecf+        .quad   0x7cef0114a47fd6f7+        .quad   0xd31ffdda4a47b37f+        .quad   0x525219a473905785+        .quad   0xd8dd3de66aa91948+        .quad   0x485064c22fc0d2cc+        .quad   0x9b48246634fdea2f+        .quad   0x293e1c4e6c4a2e3a+        .quad   0x376e134b925112e1+        .quad   0x703778b5dca15da0+        .quad   0xb04589af461c3111+        .quad   0x5b605c447f032823++        // 2^24 * 3 * G++        .quad   0xb965805920c47c89+        .quad   0xe7f0100c923b8fcc+        .quad   0x0001256502e2ef77+        .quad   0x24a76dcea8aeb3ee+        .quad   0x3be9fec6f0e7f04c+        .quad   0x866a579e75e34962+        .quad   0x5542ef161e1de61a+        .quad   0x2f12fef4cc5abdd5+        .quad   0x0a4522b2dfc0c740+        .quad   0x10d06e7f40c9a407+        .quad   0xc6cf144178cff668+        .quad   0x5e607b2518a43790++        // 2^24 * 4 * G++        .quad   0x58b31d8f6cdf1818+        .quad   0x35cfa74fc36258a2+        .quad   0xe1b3ff4f66e61d6e+        .quad   0x5067acab6ccdd5f7+        .quad   0xa02c431ca596cf14+        .quad   0xe3c42d40aed3e400+        .quad   0xd24526802e0f26db+        .quad   0x201f33139e457068+        .quad   0xfd527f6b08039d51+        .quad   0x18b14964017c0006+        .quad   0xd5220eb02e25a4a8+        .quad   0x397cba8862460375++        // 2^24 * 5 * G++        .quad   0x30c13093f05959b2+        .quad   0xe23aa18de9a97976+        .quad   0x222fd491721d5e26+        .quad   0x2339d320766e6c3a+        .quad   0x7815c3fbc81379e7+        .quad   0xa6619420dde12af1+        .quad   0xffa9c0f885a8fdd5+        .quad   0x771b4022c1e1c252+        .quad   0xd87dd986513a2fa7+        .quad   0xf5ac9b71f9d4cf08+        .quad   0xd06bc31b1ea283b3+        .quad   0x331a189219971a76++        // 2^24 * 6 * G++        .quad   0xf5166f45fb4f80c6+        .quad   0x9c36c7de61c775cf+        .quad   0xe3d4e81b9041d91c+        .quad   0x31167c6b83bdfe21+        .quad   0x26512f3a9d7572af+        .quad   0x5bcbe28868074a9e+        .quad   0x84edc1c11180f7c4+        .quad   0x1ac9619ff649a67b+        .quad   0xf22b3842524b1068+        .quad   0x5068343bee9ce987+        .quad   0xfc9d71844a6250c8+        .quad   0x612436341f08b111++        // 2^24 * 7 * G++        .quad   0xd99d41db874e898d+        .quad   0x09fea5f16c07dc20+        .quad   0x793d2c67d00f9bbc+        .quad   0x46ebe2309e5eff40+        .quad   0x8b6349e31a2d2638+        .quad   0x9ddfb7009bd3fd35+        .quad   0x7f8bf1b8a3a06ba4+        .quad   0x1522aa3178d90445+        .quad   0x2c382f5369614938+        .quad   0xdafe409ab72d6d10+        .quad   0xe8c83391b646f227+        .quad   0x45fe70f50524306c++        // 2^24 * 8 * G++        .quad   0xda4875a6960c0b8c+        .quad   0x5b68d076ef0e2f20+        .quad   0x07fb51cf3d0b8fd4+        .quad   0x428d1623a0e392d4+        .quad   0x62f24920c8951491+        .quad   0x05f007c83f630ca2+        .quad   0x6fbb45d2f5c9d4b8+        .quad   0x16619f6db57a2245+        .quad   0x084f4a4401a308fd+        .quad   0xa82219c376a5caac+        .quad   0xdeb8de4643d1bc7d+        .quad   0x1d81592d60bd38c6++        // 2^28 * 1 * G++        .quad   0xd833d7beec2a4c38+        .quad   0x2c9162830acc20ed+        .quad   0xe93a47aa92df7581+        .quad   0x702d67a3333c4a81+        .quad   0x3a4a369a2f89c8a1+        .quad   0x63137a1d7c8de80d+        .quad   0xbcac008a78eda015+        .quad   0x2cb8b3a5b483b03f+        .quad   0x36e417cbcb1b90a1+        .quad   0x33b3ddaa7f11794e+        .quad   0x3f510808885bc607+        .quad   0x24141dc0e6a8020d++        // 2^28 * 2 * G++        .quad   0x59f73c773fefee9d+        .quad   0xb3f1ef89c1cf989d+        .quad   0xe35dfb42e02e545f+        .quad   0x5766120b47a1b47c+        .quad   0x91925dccbd83157d+        .quad   0x3ca1205322cc8094+        .quad   0x28e57f183f90d6e4+        .quad   0x1a4714cede2e767b+        .quad   0xdb20ba0fb8b6b7ff+        .quad   0xb732c3b677511fa1+        .quad   0xa92b51c099f02d89+        .quad   0x4f3875ad489ca5f1++        // 2^28 * 3 * G++        .quad   0xc7fc762f4932ab22+        .quad   0x7ac0edf72f4c3c1b+        .quad   0x5f6b55aa9aa895e8+        .quad   0x3680274dad0a0081+        .quad   0x79ed13f6ee73eec0+        .quad   0xa5c6526d69110bb1+        .quad   0xe48928c38603860c+        .quad   0x722a1446fd7059f5+        .quad   0xd0959fe9a8cf8819+        .quad   0xd0a995508475a99c+        .quad   0x6eac173320b09cc5+        .quad   0x628ecf04331b1095++        // 2^28 * 4 * G++        .quad   0x98bcb118a9d0ddbc+        .quad   0xee449e3408b4802b+        .quad   0x87089226b8a6b104+        .quad   0x685f349a45c7915d+        .quad   0x9b41acf85c74ccf1+        .quad   0xb673318108265251+        .quad   0x99c92aed11adb147+        .quad   0x7a47d70d34ecb40f+        .quad   0x60a0c4cbcc43a4f5+        .quad   0x775c66ca3677bea9+        .quad   0xa17aa1752ff8f5ed+        .quad   0x11ded9020e01fdc0++        // 2^28 * 5 * G++        .quad   0x890e7809caefe704+        .quad   0x8728296de30e8c6c+        .quad   0x4c5cd2a392aeb1c9+        .quad   0x194263d15771531f+        .quad   0x471f95b03bea93b7+        .quad   0x0552d7d43313abd3+        .quad   0xbd9370e2e17e3f7b+        .quad   0x7b120f1db20e5bec+        .quad   0x17d2fb3d86502d7a+        .quad   0xb564d84450a69352+        .quad   0x7da962c8a60ed75d+        .quad   0x00d0f85b318736aa++        // 2^28 * 6 * G++        .quad   0x978b142e777c84fd+        .quad   0xf402644705a8c062+        .quad   0xa67ad51be7e612c7+        .quad   0x2f7b459698dd6a33+        .quad   0xa6753c1efd7621c1+        .quad   0x69c0b4a7445671f5+        .quad   0x971f527405b23c11+        .quad   0x387bc74851a8c7cd+        .quad   0x81894b4d4a52a9a8+        .quad   0xadd93e12f6b8832f+        .quad   0x184d8548b61bd638+        .quad   0x3f1c62dbd6c9f6cd++        // 2^28 * 7 * G++        .quad   0x2e8f1f0091910c1f+        .quad   0xa4df4fe0bff2e12c+        .quad   0x60c6560aee927438+        .quad   0x6338283facefc8fa+        .quad   0x3fad3e40148f693d+        .quad   0x052656e194eb9a72+        .quad   0x2f4dcbfd184f4e2f+        .quad   0x406f8db1c482e18b+        .quad   0x9e630d2c7f191ee4+        .quad   0x4fbf8301bc3ff670+        .quad   0x787d8e4e7afb73c4+        .quad   0x50d83d5be8f58fa5++        // 2^28 * 8 * G++        .quad   0x85683916c11a1897+        .quad   0x2d69a4efe506d008+        .quad   0x39af1378f664bd01+        .quad   0x65942131361517c6+        .quad   0xc0accf90b4d3b66d+        .quad   0xa7059de561732e60+        .quad   0x033d1f7870c6b0ba+        .quad   0x584161cd26d946e4+        .quad   0xbbf2b1a072d27ca2+        .quad   0xbf393c59fbdec704+        .quad   0xe98dbbcee262b81e+        .quad   0x02eebd0b3029b589++        // 2^32 * 1 * G++        .quad   0x61368756a60dac5f+        .quad   0x17e02f6aebabdc57+        .quad   0x7f193f2d4cce0f7d+        .quad   0x20234a7789ecdcf0+        .quad   0x8765b69f7b85c5e8+        .quad   0x6ff0678bd168bab2+        .quad   0x3a70e77c1d330f9b+        .quad   0x3a5f6d51b0af8e7c+        .quad   0x76d20db67178b252+        .quad   0x071c34f9d51ed160+        .quad   0xf62a4a20b3e41170+        .quad   0x7cd682353cffe366++        // 2^32 * 2 * G++        .quad   0x0be1a45bd887fab6+        .quad   0x2a846a32ba403b6e+        .quad   0xd9921012e96e6000+        .quad   0x2838c8863bdc0943+        .quad   0xa665cd6068acf4f3+        .quad   0x42d92d183cd7e3d3+        .quad   0x5759389d336025d9+        .quad   0x3ef0253b2b2cd8ff+        .quad   0xd16bb0cf4a465030+        .quad   0xfa496b4115c577ab+        .quad   0x82cfae8af4ab419d+        .quad   0x21dcb8a606a82812++        // 2^32 * 3 * G++        .quad   0x5c6004468c9d9fc8+        .quad   0x2540096ed42aa3cb+        .quad   0x125b4d4c12ee2f9c+        .quad   0x0bc3d08194a31dab+        .quad   0x9a8d00fabe7731ba+        .quad   0x8203607e629e1889+        .quad   0xb2cc023743f3d97f+        .quad   0x5d840dbf6c6f678b+        .quad   0x706e380d309fe18b+        .quad   0x6eb02da6b9e165c7+        .quad   0x57bbba997dae20ab+        .quad   0x3a4276232ac196dd++        // 2^32 * 4 * G++        .quad   0x4b42432c8a7084fa+        .quad   0x898a19e3dfb9e545+        .quad   0xbe9f00219c58e45d+        .quad   0x1ff177cea16debd1+        .quad   0x3bf8c172db447ecb+        .quad   0x5fcfc41fc6282dbd+        .quad   0x80acffc075aa15fe+        .quad   0x0770c9e824e1a9f9+        .quad   0xcf61d99a45b5b5fd+        .quad   0x860984e91b3a7924+        .quad   0xe7300919303e3e89+        .quad   0x39f264fd41500b1e++        // 2^32 * 5 * G++        .quad   0xa7ad3417dbe7e29c+        .quad   0xbd94376a2b9c139c+        .quad   0xa0e91b8e93597ba9+        .quad   0x1712d73468889840+        .quad   0xd19b4aabfe097be1+        .quad   0xa46dfce1dfe01929+        .quad   0xc3c908942ca6f1ff+        .quad   0x65c621272c35f14e+        .quad   0xe72b89f8ce3193dd+        .quad   0x4d103356a125c0bb+        .quad   0x0419a93d2e1cfe83+        .quad   0x22f9800ab19ce272++        // 2^32 * 6 * G++        .quad   0x605a368a3e9ef8cb+        .quad   0xe3e9c022a5504715+        .quad   0x553d48b05f24248f+        .quad   0x13f416cd647626e5+        .quad   0x42029fdd9a6efdac+        .quad   0xb912cebe34a54941+        .quad   0x640f64b987bdf37b+        .quad   0x4171a4d38598cab4+        .quad   0xfa2758aa99c94c8c+        .quad   0x23006f6fb000b807+        .quad   0xfbd291ddadda5392+        .quad   0x508214fa574bd1ab++        // 2^32 * 7 * G++        .quad   0xc20269153ed6fe4b+        .quad   0xa65a6739511d77c4+        .quad   0xcbde26462c14af94+        .quad   0x22f960ec6faba74b+        .quad   0x461a15bb53d003d6+        .quad   0xb2102888bcf3c965+        .quad   0x27c576756c683a5a+        .quad   0x3a7758a4c86cb447+        .quad   0x548111f693ae5076+        .quad   0x1dae21df1dfd54a6+        .quad   0x12248c90f3115e65+        .quad   0x5d9fd15f8de7f494++        // 2^32 * 8 * G++        .quad   0x031408d36d63727f+        .quad   0x6a379aefd7c7b533+        .quad   0xa9e18fc5ccaee24b+        .quad   0x332f35914f8fbed3+        .quad   0x3f244d2aeed7521e+        .quad   0x8e3a9028432e9615+        .quad   0xe164ba772e9c16d4+        .quad   0x3bc187fa47eb98d8+        .quad   0x6d470115ea86c20c+        .quad   0x998ab7cb6c46d125+        .quad   0xd77832b53a660188+        .quad   0x450d81ce906fba03++        // 2^36 * 1 * G++        .quad   0xf8ae4d2ad8453902+        .quad   0x7018058ee8db2d1d+        .quad   0xaab3995fc7d2c11e+        .quad   0x53b16d2324ccca79+        .quad   0x23264d66b2cae0b5+        .quad   0x7dbaed33ebca6576+        .quad   0x030ebed6f0d24ac8+        .quad   0x2a887f78f7635510+        .quad   0x2a23b9e75c012d4f+        .quad   0x0c974651cae1f2ea+        .quad   0x2fb63273675d70ca+        .quad   0x0ba7250b864403f5++        // 2^36 * 2 * G++        .quad   0xbb0d18fd029c6421+        .quad   0xbc2d142189298f02+        .quad   0x8347f8e68b250e96+        .quad   0x7b9f2fe8032d71c9+        .quad   0xdd63589386f86d9c+        .quad   0x61699176e13a85a4+        .quad   0x2e5111954eaa7d57+        .quad   0x32c21b57fb60bdfb+        .quad   0xd87823cd319e0780+        .quad   0xefc4cfc1897775c5+        .quad   0x4854fb129a0ab3f7+        .quad   0x12c49d417238c371++        // 2^36 * 3 * G++        .quad   0x0950b533ffe83769+        .quad   0x21861c1d8e1d6bd1+        .quad   0xf022d8381302e510+        .quad   0x2509200c6391cab4+        .quad   0x09b3a01783799542+        .quad   0x626dd08faad5ee3f+        .quad   0xba00bceeeb70149f+        .quad   0x1421b246a0a444c9+        .quad   0x4aa43a8e8c24a7c7+        .quad   0x04c1f540d8f05ef5+        .quad   0xadba5e0c0b3eb9dc+        .quad   0x2ab5504448a49ce3++        // 2^36 * 4 * G++        .quad   0x2ed227266f0f5dec+        .quad   0x9824ee415ed50824+        .quad   0x807bec7c9468d415+        .quad   0x7093bae1b521e23f+        .quad   0xdc07ac631c5d3afa+        .quad   0x58615171f9df8c6c+        .quad   0x72a079d89d73e2b0+        .quad   0x7301f4ceb4eae15d+        .quad   0x6409e759d6722c41+        .quad   0xa674e1cf72bf729b+        .quad   0xbc0a24eb3c21e569+        .quad   0x390167d24ebacb23++        // 2^36 * 5 * G++        .quad   0x27f58e3bba353f1c+        .quad   0x4c47764dbf6a4361+        .quad   0xafbbc4e56e562650+        .quad   0x07db2ee6aae1a45d+        .quad   0xd7bb054ba2f2120b+        .quad   0xe2b9ceaeb10589b7+        .quad   0x3fe8bac8f3c0edbe+        .quad   0x4cbd40767112cb69+        .quad   0x0b603cc029c58176+        .quad   0x5988e3825cb15d61+        .quad   0x2bb61413dcf0ad8d+        .quad   0x7b8eec6c74183287++        // 2^36 * 6 * G++        .quad   0xe4ca40782cd27cb0+        .quad   0xdaf9c323fbe967bd+        .quad   0xb29bd34a8ad41e9e+        .quad   0x72810497626ede4d+        .quad   0x32fee570fc386b73+        .quad   0xda8b0141da3a8cc7+        .quad   0x975ffd0ac8968359+        .quad   0x6ee809a1b132a855+        .quad   0x9444bb31fcfd863a+        .quad   0x2fe3690a3e4e48c5+        .quad   0xdc29c867d088fa25+        .quad   0x13bd1e38d173292e++        // 2^36 * 7 * G++        .quad   0xd32b4cd8696149b5+        .quad   0xe55937d781d8aab7+        .quad   0x0bcb2127ae122b94+        .quad   0x41e86fcfb14099b0+        .quad   0x223fb5cf1dfac521+        .quad   0x325c25316f554450+        .quad   0x030b98d7659177ac+        .quad   0x1ed018b64f88a4bd+        .quad   0x3630dfa1b802a6b0+        .quad   0x880f874742ad3bd5+        .quad   0x0af90d6ceec5a4d4+        .quad   0x746a247a37cdc5d9++        // 2^36 * 8 * G++        .quad   0xd531b8bd2b7b9af6+        .quad   0x5005093537fc5b51+        .quad   0x232fcf25c593546d+        .quad   0x20a365142bb40f49+        .quad   0x6eccd85278d941ed+        .quad   0x2254ae83d22f7843+        .quad   0xc522d02e7bbfcdb7+        .quad   0x681e3351bff0e4e2+        .quad   0x8b64b59d83034f45+        .quad   0x2f8b71f21fa20efb+        .quad   0x69249495ba6550e4+        .quad   0x539ef98e45d5472b++        // 2^40 * 1 * G++        .quad   0x6e7bb6a1a6205275+        .quad   0xaa4f21d7413c8e83+        .quad   0x6f56d155e88f5cb2+        .quad   0x2de25d4ba6345be1+        .quad   0xd074d8961cae743f+        .quad   0xf86d18f5ee1c63ed+        .quad   0x97bdc55be7f4ed29+        .quad   0x4cbad279663ab108+        .quad   0x80d19024a0d71fcd+        .quad   0xc525c20afb288af8+        .quad   0xb1a3974b5f3a6419+        .quad   0x7d7fbcefe2007233++        // 2^40 * 2 * G++        .quad   0xfaef1e6a266b2801+        .quad   0x866c68c4d5739f16+        .quad   0xf68a2fbc1b03762c+        .quad   0x5975435e87b75a8d+        .quad   0xcd7c5dc5f3c29094+        .quad   0xc781a29a2a9105ab+        .quad   0x80c61d36421c3058+        .quad   0x4f9cd196dcd8d4d7+        .quad   0x199297d86a7b3768+        .quad   0xd0d058241ad17a63+        .quad   0xba029cad5c1c0c17+        .quad   0x7ccdd084387a0307++        // 2^40 * 3 * G++        .quad   0xdca6422c6d260417+        .quad   0xae153d50948240bd+        .quad   0xa9c0c1b4fb68c677+        .quad   0x428bd0ed61d0cf53+        .quad   0x9b0c84186760cc93+        .quad   0xcdae007a1ab32a99+        .quad   0xa88dec86620bda18+        .quad   0x3593ca848190ca44+        .quad   0x9213189a5e849aa7+        .quad   0xd4d8c33565d8facd+        .quad   0x8c52545b53fdbbd1+        .quad   0x27398308da2d63e6++        // 2^40 * 4 * G++        .quad   0x42c38d28435ed413+        .quad   0xbd50f3603278ccc9+        .quad   0xbb07ab1a79da03ef+        .quad   0x269597aebe8c3355+        .quad   0xb9a10e4c0a702453+        .quad   0x0fa25866d57d1bde+        .quad   0xffb9d9b5cd27daf7+        .quad   0x572c2945492c33fd+        .quad   0xc77fc745d6cd30be+        .quad   0xe4dfe8d3e3baaefb+        .quad   0xa22c8830aa5dda0c+        .quad   0x7f985498c05bca80++        // 2^40 * 5 * G++        .quad   0x3849ce889f0be117+        .quad   0x8005ad1b7b54a288+        .quad   0x3da3c39f23fc921c+        .quad   0x76c2ec470a31f304+        .quad   0xd35615520fbf6363+        .quad   0x08045a45cf4dfba6+        .quad   0xeec24fbc873fa0c2+        .quad   0x30f2653cd69b12e7+        .quad   0x8a08c938aac10c85+        .quad   0x46179b60db276bcb+        .quad   0xa920c01e0e6fac70+        .quad   0x2f1273f1596473da++        // 2^40 * 6 * G++        .quad   0x4739fc7c8ae01e11+        .quad   0xfd5274904a6aab9f+        .quad   0x41d98a8287728f2e+        .quad   0x5d9e572ad85b69f2+        .quad   0x30488bd755a70bc0+        .quad   0x06d6b5a4f1d442e7+        .quad   0xead1a69ebc596162+        .quad   0x38ac1997edc5f784+        .quad   0x0666b517a751b13b+        .quad   0x747d06867e9b858c+        .quad   0xacacc011454dde49+        .quad   0x22dfcd9cbfe9e69c++        // 2^40 * 7 * G++        .quad   0x8ddbd2e0c30d0cd9+        .quad   0xad8e665facbb4333+        .quad   0x8f6b258c322a961f+        .quad   0x6b2916c05448c1c7+        .quad   0x56ec59b4103be0a1+        .quad   0x2ee3baecd259f969+        .quad   0x797cb29413f5cd32+        .quad   0x0fe9877824cde472+        .quad   0x7edb34d10aba913b+        .quad   0x4ea3cd822e6dac0e+        .quad   0x66083dff6578f815+        .quad   0x4c303f307ff00a17++        // 2^40 * 8 * G++        .quad   0xd30a3bd617b28c85+        .quad   0xc5d377b739773bea+        .quad   0xc6c6e78c1e6a5cbf+        .quad   0x0d61b8f78b2ab7c4+        .quad   0x29fc03580dd94500+        .quad   0xecd27aa46fbbec93+        .quad   0x130a155fc2e2a7f8+        .quad   0x416b151ab706a1d5+        .quad   0x56a8d7efe9c136b0+        .quad   0xbd07e5cd58e44b20+        .quad   0xafe62fda1b57e0ab+        .quad   0x191a2af74277e8d2++        // 2^44 * 1 * G++        .quad   0xd550095bab6f4985+        .quad   0x04f4cd5b4fbfaf1a+        .quad   0x9d8e2ed12a0c7540+        .quad   0x2bc24e04b2212286+        .quad   0x09d4b60b2fe09a14+        .quad   0xc384f0afdbb1747e+        .quad   0x58e2ea8978b5fd6e+        .quad   0x519ef577b5e09b0a+        .quad   0x1863d7d91124cca9+        .quad   0x7ac08145b88a708e+        .quad   0x2bcd7309857031f5+        .quad   0x62337a6e8ab8fae5++        // 2^44 * 2 * G++        .quad   0x4bcef17f06ffca16+        .quad   0xde06e1db692ae16a+        .quad   0x0753702d614f42b0+        .quad   0x5f6041b45b9212d0+        .quad   0xd1ab324e1b3a1273+        .quad   0x18947cf181055340+        .quad   0x3b5d9567a98c196e+        .quad   0x7fa00425802e1e68+        .quad   0x7d531574028c2705+        .quad   0x80317d69db0d75fe+        .quad   0x30fface8ef8c8ddd+        .quad   0x7e9de97bb6c3e998++        // 2^44 * 3 * G++        .quad   0x1558967b9e6585a3+        .quad   0x97c99ce098e98b92+        .quad   0x10af149b6eb3adad+        .quad   0x42181fe8f4d38cfa+        .quad   0xf004be62a24d40dd+        .quad   0xba0659910452d41f+        .quad   0x81c45ee162a44234+        .quad   0x4cb829d8a22266ef+        .quad   0x1dbcaa8407b86681+        .quad   0x081f001e8b26753b+        .quad   0x3cd7ce6a84048e81+        .quad   0x78af11633f25f22c++        // 2^44 * 4 * G++        .quad   0x8416ebd40b50babc+        .quad   0x1508722628208bee+        .quad   0xa3148fafb9c1c36d+        .quad   0x0d07daacd32d7d5d+        .quad   0x3241c00e7d65318c+        .quad   0xe6bee5dcd0e86de7+        .quad   0x118b2dc2fbc08c26+        .quad   0x680d04a7fc603dc3+        .quad   0xf9c2414a695aa3eb+        .quad   0xdaa42c4c05a68f21+        .quad   0x7c6c23987f93963e+        .quad   0x210e8cd30c3954e3++        // 2^44 * 5 * G++        .quad   0xac4201f210a71c06+        .quad   0x6a65e0aef3bfb021+        .quad   0xbc42c35c393632f7+        .quad   0x56ea8db1865f0742+        .quad   0x2b50f16137fe6c26+        .quad   0xe102bcd856e404d8+        .quad   0x12b0f1414c561f6b+        .quad   0x51b17bc8d028ec91+        .quad   0xfff5fb4bcf535119+        .quad   0xf4989d79df1108a0+        .quad   0xbdfcea659a3ba325+        .quad   0x18a11f1174d1a6f2++        // 2^44 * 6 * G++        .quad   0x407375ab3f6bba29+        .quad   0x9ec3b6d8991e482e+        .quad   0x99c80e82e55f92e9+        .quad   0x307c13b6fb0c0ae1+        .quad   0xfbd63cdad27a5f2c+        .quad   0xf00fc4bc8aa106d7+        .quad   0x53fb5c1a8e64a430+        .quad   0x04eaabe50c1a2e85+        .quad   0x24751021cb8ab5e7+        .quad   0xfc2344495c5010eb+        .quad   0x5f1e717b4e5610a1+        .quad   0x44da5f18c2710cd5++        // 2^44 * 7 * G++        .quad   0x033cc55ff1b82eb5+        .quad   0xb15ae36d411cae52+        .quad   0xba40b6198ffbacd3+        .quad   0x768edce1532e861f+        .quad   0x9156fe6b89d8eacc+        .quad   0xe6b79451e23126a1+        .quad   0xbd7463d93944eb4e+        .quad   0x726373f6767203ae+        .quad   0xe305ca72eb7ef68a+        .quad   0x662cf31f70eadb23+        .quad   0x18f026fdb4c45b68+        .quad   0x513b5384b5d2ecbd++        // 2^44 * 8 * G++        .quad   0x46d46280c729989e+        .quad   0x4b93fbd05368a5dd+        .quad   0x63df3f81d1765a89+        .quad   0x34cebd64b9a0a223+        .quad   0x5e2702878af34ceb+        .quad   0x900b0409b946d6ae+        .quad   0x6512ebf7dabd8512+        .quad   0x61d9b76988258f81+        .quad   0xa6c5a71349b7d94b+        .quad   0xa3f3d15823eb9446+        .quad   0x0416fbd277484834+        .quad   0x69d45e6f2c70812f++        // 2^48 * 1 * G++        .quad   0xce16f74bc53c1431+        .quad   0x2b9725ce2072edde+        .quad   0xb8b9c36fb5b23ee7+        .quad   0x7e2e0e450b5cc908+        .quad   0x9fe62b434f460efb+        .quad   0xded303d4a63607d6+        .quad   0xf052210eb7a0da24+        .quad   0x237e7dbe00545b93+        .quad   0x013575ed6701b430+        .quad   0x231094e69f0bfd10+        .quad   0x75320f1583e47f22+        .quad   0x71afa699b11155e3++        // 2^48 * 2 * G++        .quad   0x65ce6f9b3953b61d+        .quad   0xc65839eaafa141e6+        .quad   0x0f435ffda9f759fe+        .quad   0x021142e9c2b1c28e+        .quad   0xea423c1c473b50d6+        .quad   0x51e87a1f3b38ef10+        .quad   0x9b84bf5fb2c9be95+        .quad   0x00731fbc78f89a1c+        .quad   0xe430c71848f81880+        .quad   0xbf960c225ecec119+        .quad   0xb6dae0836bba15e3+        .quad   0x4c4d6f3347e15808++        // 2^48 * 3 * G++        .quad   0x18f7eccfc17d1fc9+        .quad   0x6c75f5a651403c14+        .quad   0xdbde712bf7ee0cdf+        .quad   0x193fddaaa7e47a22+        .quad   0x2f0cddfc988f1970+        .quad   0x6b916227b0b9f51b+        .quad   0x6ec7b6c4779176be+        .quad   0x38bf9500a88f9fa8+        .quad   0x1fd2c93c37e8876f+        .quad   0xa2f61e5a18d1462c+        .quad   0x5080f58239241276+        .quad   0x6a6fb99ebf0d4969++        // 2^48 * 4 * G++        .quad   0x6a46c1bb560855eb+        .quad   0x2416bb38f893f09d+        .quad   0xd71d11378f71acc1+        .quad   0x75f76914a31896ea+        .quad   0xeeb122b5b6e423c6+        .quad   0x939d7010f286ff8e+        .quad   0x90a92a831dcf5d8c+        .quad   0x136fda9f42c5eb10+        .quad   0xf94cdfb1a305bdd1+        .quad   0x0f364b9d9ff82c08+        .quad   0x2a87d8a5c3bb588a+        .quad   0x022183510be8dcba++        // 2^48 * 5 * G++        .quad   0x4af766385ead2d14+        .quad   0xa08ed880ca7c5830+        .quad   0x0d13a6e610211e3d+        .quad   0x6a071ce17b806c03+        .quad   0x9d5a710143307a7f+        .quad   0xb063de9ec47da45f+        .quad   0x22bbfe52be927ad3+        .quad   0x1387c441fd40426c+        .quad   0xb5d3c3d187978af8+        .quad   0x722b5a3d7f0e4413+        .quad   0x0d7b4848bb477ca0+        .quad   0x3171b26aaf1edc92++        // 2^48 * 6 * G++        .quad   0xa92f319097564ca8+        .quad   0xff7bb84c2275e119+        .quad   0x4f55fe37a4875150+        .quad   0x221fd4873cf0835a+        .quad   0xa60db7d8b28a47d1+        .quad   0xa6bf14d61770a4f1+        .quad   0xd4a1f89353ddbd58+        .quad   0x6c514a63344243e9+        .quad   0x2322204f3a156341+        .quad   0xfb73e0e9ba0a032d+        .quad   0xfce0dd4c410f030e+        .quad   0x48daa596fb924aaa++        // 2^48 * 7 * G++        .quad   0x6eca8e665ca59cc7+        .quad   0xa847254b2e38aca0+        .quad   0x31afc708d21e17ce+        .quad   0x676dd6fccad84af7+        .quad   0x14f61d5dc84c9793+        .quad   0x9941f9e3ef418206+        .quad   0xcdf5b88f346277ac+        .quad   0x58c837fa0e8a79a9+        .quad   0x0cf9688596fc9058+        .quad   0x1ddcbbf37b56a01b+        .quad   0xdcc2e77d4935d66a+        .quad   0x1c4f73f2c6a57f0a++        // 2^48 * 8 * G++        .quad   0x0e7a4fbd305fa0bb+        .quad   0x829d4ce054c663ad+        .quad   0xf421c3832fe33848+        .quad   0x795ac80d1bf64c42+        .quad   0xb36e706efc7c3484+        .quad   0x73dfc9b4c3c1cf61+        .quad   0xeb1d79c9781cc7e5+        .quad   0x70459adb7daf675c+        .quad   0x1b91db4991b42bb3+        .quad   0x572696234b02dcca+        .quad   0x9fdf9ee51f8c78dc+        .quad   0x5fe162848ce21fd3++        // 2^52 * 1 * G++        .quad   0xe2790aae4d077c41+        .quad   0x8b938270db7469a3+        .quad   0x6eb632dc8abd16a2+        .quad   0x720814ecaa064b72+        .quad   0x315c29c795115389+        .quad   0xd7e0e507862f74ce+        .quad   0x0c4a762185927432+        .quad   0x72de6c984a25a1e4+        .quad   0xae9ab553bf6aa310+        .quad   0x050a50a9806d6e1b+        .quad   0x92bb7403adff5139+        .quad   0x0394d27645be618b++        // 2^52 * 2 * G++        .quad   0x4d572251857eedf4+        .quad   0xe3724edde19e93c5+        .quad   0x8a71420e0b797035+        .quad   0x3b3c833687abe743+        .quad   0xf5396425b23545a4+        .quad   0x15a7a27e98fbb296+        .quad   0xab6c52bc636fdd86+        .quad   0x79d995a8419334ee+        .quad   0xcd8a8ea61195dd75+        .quad   0xa504d8a81dd9a82f+        .quad   0x540dca81a35879b6+        .quad   0x60dd16a379c86a8a++        // 2^52 * 3 * G++        .quad   0x35a2c8487381e559+        .quad   0x596ffea6d78082cb+        .quad   0xcb9771ebdba7b653+        .quad   0x5a08b5019b4da685+        .quad   0x3501d6f8153e47b8+        .quad   0xb7a9675414a2f60c+        .quad   0x112ee8b6455d9523+        .quad   0x4e62a3c18112ea8a+        .quad   0xc8d4ac04516ab786+        .quad   0x595af3215295b23d+        .quad   0xd6edd234db0230c1+        .quad   0x0929efe8825b41cc++        // 2^52 * 4 * G++        .quad   0x5f0601d1cbd0f2d3+        .quad   0x736e412f6132bb7f+        .quad   0x83604432238dde87+        .quad   0x1e3a5272f5c0753c+        .quad   0x8b3172b7ad56651d+        .quad   0x01581b7a3fabd717+        .quad   0x2dc94df6424df6e4+        .quad   0x30376e5d2c29284f+        .quad   0xd2918da78159a59c+        .quad   0x6bdc1cd93f0713f3+        .quad   0x565f7a934acd6590+        .quad   0x53daacec4cb4c128++        // 2^52 * 5 * G++        .quad   0x4ca73bd79cc8a7d6+        .quad   0x4d4a738f47e9a9b2+        .quad   0xf4cbf12942f5fe00+        .quad   0x01a13ff9bdbf0752+        .quad   0x99852bc3852cfdb0+        .quad   0x2cc12e9559d6ed0b+        .quad   0x70f9e2bf9b5ac27b+        .quad   0x4f3b8c117959ae99+        .quad   0x55b6c9c82ff26412+        .quad   0x1ac4a8c91fb667a8+        .quad   0xd527bfcfeb778bf2+        .quad   0x303337da7012a3be++        // 2^52 * 6 * G++        .quad   0x955422228c1c9d7c+        .quad   0x01fac1371a9b340f+        .quad   0x7e8d9177925b48d7+        .quad   0x53f8ad5661b3e31b+        .quad   0x976d3ccbfad2fdd1+        .quad   0xcb88839737a640a8+        .quad   0x2ff00c1d6734cb25+        .quad   0x269ff4dc789c2d2b+        .quad   0x0c003fbdc08d678d+        .quad   0x4d982fa37ead2b17+        .quad   0xc07e6bcdb2e582f1+        .quad   0x296c7291df412a44++        // 2^52 * 7 * G++        .quad   0x7903de2b33daf397+        .quad   0xd0ff0619c9a624b3+        .quad   0x8a1d252b555b3e18+        .quad   0x2b6d581c52e0b7c0+        .quad   0xdfb23205dab8b59e+        .quad   0x465aeaa0c8092250+        .quad   0xd133c1189a725d18+        .quad   0x2327370261f117d1+        .quad   0x3d0543d3623e7986+        .quad   0x679414c2c278a354+        .quad   0xae43f0cc726196f6+        .quad   0x7836c41f8245eaba++        // 2^52 * 8 * G++        .quad   0xe7a254db49e95a81+        .quad   0x5192d5d008b0ad73+        .quad   0x4d20e5b1d00afc07+        .quad   0x5d55f8012cf25f38+        .quad   0xca651e848011937c+        .quad   0xc6b0c46e6ef41a28+        .quad   0xb7021ba75f3f8d52+        .quad   0x119dff99ead7b9fd+        .quad   0x43eadfcbf4b31d4d+        .quad   0xc6503f7411148892+        .quad   0xfeee68c5060d3b17+        .quad   0x329293b3dd4a0ac8++        // 2^56 * 1 * G++        .quad   0x4e59214fe194961a+        .quad   0x49be7dc70d71cd4f+        .quad   0x9300cfd23b50f22d+        .quad   0x4789d446fc917232+        .quad   0x2879852d5d7cb208+        .quad   0xb8dedd70687df2e7+        .quad   0xdc0bffab21687891+        .quad   0x2b44c043677daa35+        .quad   0x1a1c87ab074eb78e+        .quad   0xfac6d18e99daf467+        .quad   0x3eacbbcd484f9067+        .quad   0x60c52eef2bb9a4e4++        // 2^56 * 2 * G++        .quad   0x0b5d89bc3bfd8bf1+        .quad   0xb06b9237c9f3551a+        .quad   0x0e4c16b0d53028f5+        .quad   0x10bc9c312ccfcaab+        .quad   0x702bc5c27cae6d11+        .quad   0x44c7699b54a48cab+        .quad   0xefbc4056ba492eb2+        .quad   0x70d77248d9b6676d+        .quad   0xaa8ae84b3ec2a05b+        .quad   0x98699ef4ed1781e0+        .quad   0x794513e4708e85d1+        .quad   0x63755bd3a976f413++        // 2^56 * 3 * G++        .quad   0xb55fa03e2ad10853+        .quad   0x356f75909ee63569+        .quad   0x9ff9f1fdbe69b890+        .quad   0x0d8cc1c48bc16f84+        .quad   0x3dc7101897f1acb7+        .quad   0x5dda7d5ec165bbd8+        .quad   0x508e5b9c0fa1020f+        .quad   0x2763751737c52a56+        .quad   0x029402d36eb419a9+        .quad   0xf0b44e7e77b460a5+        .quad   0xcfa86230d43c4956+        .quad   0x70c2dd8a7ad166e7++        // 2^56 * 4 * G++        .quad   0x656194509f6fec0e+        .quad   0xee2e7ea946c6518d+        .quad   0x9733c1f367e09b5c+        .quad   0x2e0fac6363948495+        .quad   0x91d4967db8ed7e13+        .quad   0x74252f0ad776817a+        .quad   0xe40982e00d852564+        .quad   0x32b8613816a53ce5+        .quad   0x79e7f7bee448cd64+        .quad   0x6ac83a67087886d0+        .quad   0xf89fd4d9a0e4db2e+        .quad   0x4179215c735a4f41++        // 2^56 * 5 * G++        .quad   0x8c7094e7d7dced2a+        .quad   0x97fb8ac347d39c70+        .quad   0xe13be033a906d902+        .quad   0x700344a30cd99d76+        .quad   0xe4ae33b9286bcd34+        .quad   0xb7ef7eb6559dd6dc+        .quad   0x278b141fb3d38e1f+        .quad   0x31fa85662241c286+        .quad   0xaf826c422e3622f4+        .quad   0xc12029879833502d+        .quad   0x9bc1b7e12b389123+        .quad   0x24bb2312a9952489++        // 2^56 * 6 * G++        .quad   0xb1a8ed1732de67c3+        .quad   0x3cb49418461b4948+        .quad   0x8ebd434376cfbcd2+        .quad   0x0fee3e871e188008+        .quad   0x41f80c2af5f85c6b+        .quad   0x687284c304fa6794+        .quad   0x8945df99a3ba1bad+        .quad   0x0d1d2af9ffeb5d16+        .quad   0xa9da8aa132621edf+        .quad   0x30b822a159226579+        .quad   0x4004197ba79ac193+        .quad   0x16acd79718531d76++        // 2^56 * 7 * G++        .quad   0x72df72af2d9b1d3d+        .quad   0x63462a36a432245a+        .quad   0x3ecea07916b39637+        .quad   0x123e0ef6b9302309+        .quad   0xc959c6c57887b6ad+        .quad   0x94e19ead5f90feba+        .quad   0x16e24e62a342f504+        .quad   0x164ed34b18161700+        .quad   0x487ed94c192fe69a+        .quad   0x61ae2cea3a911513+        .quad   0x877bf6d3b9a4de27+        .quad   0x78da0fc61073f3eb++        // 2^56 * 8 * G++        .quad   0x5bf15d28e52bc66a+        .quad   0x2c47e31870f01a8e+        .quad   0x2419afbc06c28bdd+        .quad   0x2d25deeb256b173a+        .quad   0xa29f80f1680c3a94+        .quad   0x71f77e151ae9e7e6+        .quad   0x1100f15848017973+        .quad   0x054aa4b316b38ddd+        .quad   0xdfc8468d19267cb8+        .quad   0x0b28789c66e54daf+        .quad   0x2aeb1d2a666eec17+        .quad   0x134610a6ab7da760++        // 2^60 * 1 * G++        .quad   0xcaf55ec27c59b23f+        .quad   0x99aeed3e154d04f2+        .quad   0x68441d72e14141f4+        .quad   0x140345133932a0a2+        .quad   0xd91430e0dc028c3c+        .quad   0x0eb955a85217c771+        .quad   0x4b09e1ed2c99a1fa+        .quad   0x42881af2bd6a743c+        .quad   0x7bfec69aab5cad3d+        .quad   0xc23e8cd34cb2cfad+        .quad   0x685dd14bfb37d6a2+        .quad   0x0ad6d64415677a18++        // 2^60 * 2 * G++        .quad   0x781a439e417becb5+        .quad   0x4ac5938cd10e0266+        .quad   0x5da385110692ac24+        .quad   0x11b065a2ade31233+        .quad   0x7914892847927e9f+        .quad   0x33dad6ef370aa877+        .quad   0x1f8f24fa11122703+        .quad   0x5265ac2f2adf9592+        .quad   0x405fdd309afcb346+        .quad   0xd9723d4428e63f54+        .quad   0x94c01df05f65aaae+        .quad   0x43e4dc3ae14c0809++        // 2^60 * 3 * G++        .quad   0xbc12c7f1a938a517+        .quad   0x473028ab3180b2e1+        .quad   0x3f78571efbcd254a+        .quad   0x74e534426ff6f90f+        .quad   0xea6f7ac3adc2c6a3+        .quad   0xd0e928f6e9717c94+        .quad   0xe2d379ead645eaf5+        .quad   0x46dd8785c51ffbbe+        .quad   0x709801be375c8898+        .quad   0x4b06dab5e3fd8348+        .quad   0x75880ced27230714+        .quad   0x2b09468fdd2f4c42++        // 2^60 * 4 * G++        .quad   0x97c749eeb701cb96+        .quad   0x83f438d4b6a369c3+        .quad   0x62962b8b9a402cd9+        .quad   0x6976c7509888df7b+        .quad   0x5b97946582ffa02a+        .quad   0xda096a51fea8f549+        .quad   0xa06351375f77af9b+        .quad   0x1bcfde61201d1e76+        .quad   0x4a4a5490246a59a2+        .quad   0xd63ebddee87fdd90+        .quad   0xd9437c670d2371fa+        .quad   0x69e87308d30f8ed6++        // 2^60 * 5 * G++        .quad   0x435a8bb15656beb0+        .quad   0xf8fac9ba4f4d5bca+        .quad   0xb9b278c41548c075+        .quad   0x3eb0ef76e892b622+        .quad   0x0f80bf028bc80303+        .quad   0x6aae16b37a18cefb+        .quad   0xdd47ea47d72cd6a3+        .quad   0x61943588f4ed39aa+        .quad   0xd26e5c3e91039f85+        .quad   0xc0e9e77df6f33aa9+        .quad   0xe8968c5570066a93+        .quad   0x3c34d1881faaaddd++        // 2^60 * 6 * G++        .quad   0x3f9d2b5ea09f9ec0+        .quad   0x1dab3b6fb623a890+        .quad   0xa09ba3ea72d926c4+        .quad   0x374193513fd8b36d+        .quad   0xbd5b0b8f2fffe0d9+        .quad   0x6aa254103ed24fb9+        .quad   0x2ac7d7bcb26821c4+        .quad   0x605b394b60dca36a+        .quad   0xb4e856e45a9d1ed2+        .quad   0xefe848766c97a9a2+        .quad   0xb104cf641e5eee7d+        .quad   0x2f50b81c88a71c8f++        // 2^60 * 7 * G++        .quad   0x31723c61fc6811bb+        .quad   0x9cb450486211800f+        .quad   0x768933d347995753+        .quad   0x3491a53502752fcd+        .quad   0x2b552ca0a7da522a+        .quad   0x3230b336449b0250+        .quad   0xf2c4c5bca4b99fb9+        .quad   0x7b2c674958074a22+        .quad   0xd55165883ed28cdf+        .quad   0x12d84fd2d362de39+        .quad   0x0a874ad3e3378e4f+        .quad   0x000d2b1f7c763e74++        // 2^60 * 8 * G++        .quad   0x3d420811d06d4a67+        .quad   0xbefc048590e0ffe3+        .quad   0xf870c6b7bd487bde+        .quad   0x6e2a7316319afa28+        .quad   0x9624778c3e94a8ab+        .quad   0x0ad6f3cee9a78bec+        .quad   0x948ac7810d743c4f+        .quad   0x76627935aaecfccc+        .quad   0x56a8ac24d6d59a9f+        .quad   0xc8db753e3096f006+        .quad   0x477f41e68f4c5299+        .quad   0x588d851cf6c86114++        // 2^64 * 1 * G++        .quad   0x51138ec78df6b0fe+        .quad   0x5397da89e575f51b+        .quad   0x09207a1d717af1b9+        .quad   0x2102fdba2b20d650+        .quad   0xcd2a65e777d1f515+        .quad   0x548991878faa60f1+        .quad   0xb1b73bbcdabc06e5+        .quad   0x654878cba97cc9fb+        .quad   0x969ee405055ce6a1+        .quad   0x36bca7681251ad29+        .quad   0x3a1af517aa7da415+        .quad   0x0ad725db29ecb2ba++        // 2^64 * 2 * G++        .quad   0xdc4267b1834e2457+        .quad   0xb67544b570ce1bc5+        .quad   0x1af07a0bf7d15ed7+        .quad   0x4aefcffb71a03650+        .quad   0xfec7bc0c9b056f85+        .quad   0x537d5268e7f5ffd7+        .quad   0x77afc6624312aefa+        .quad   0x4f675f5302399fd9+        .quad   0xc32d36360415171e+        .quad   0xcd2bef118998483b+        .quad   0x870a6eadd0945110+        .quad   0x0bccbb72a2a86561++        // 2^64 * 3 * G++        .quad   0x185e962feab1a9c8+        .quad   0x86e7e63565147dcd+        .quad   0xb092e031bb5b6df2+        .quad   0x4024f0ab59d6b73e+        .quad   0x186d5e4c50fe1296+        .quad   0xe0397b82fee89f7e+        .quad   0x3bc7f6c5507031b0+        .quad   0x6678fd69108f37c2+        .quad   0x1586fa31636863c2+        .quad   0x07f68c48572d33f2+        .quad   0x4f73cc9f789eaefc+        .quad   0x2d42e2108ead4701++        // 2^64 * 4 * G++        .quad   0x97f5131594dfd29b+        .quad   0x6155985d313f4c6a+        .quad   0xeba13f0708455010+        .quad   0x676b2608b8d2d322+        .quad   0x21717b0d0f537593+        .quad   0x914e690b131e064c+        .quad   0x1bb687ae752ae09f+        .quad   0x420bf3a79b423c6e+        .quad   0x8138ba651c5b2b47+        .quad   0x8671b6ec311b1b80+        .quad   0x7bff0cb1bc3135b0+        .quad   0x745d2ffa9c0cf1e0++        // 2^64 * 5 * G++        .quad   0xbf525a1e2bc9c8bd+        .quad   0xea5b260826479d81+        .quad   0xd511c70edf0155db+        .quad   0x1ae23ceb960cf5d0+        .quad   0x6036df5721d34e6a+        .quad   0xb1db8827997bb3d0+        .quad   0xd3c209c3c8756afa+        .quad   0x06e15be54c1dc839+        .quad   0x5b725d871932994a+        .quad   0x32351cb5ceb1dab0+        .quad   0x7dc41549dab7ca05+        .quad   0x58ded861278ec1f7++        // 2^64 * 6 * G++        .quad   0xd8173793f266c55c+        .quad   0xc8c976c5cc454e49+        .quad   0x5ce382f8bc26c3a8+        .quad   0x2ff39de85485f6f9+        .quad   0x2dfb5ba8b6c2c9a8+        .quad   0x48eeef8ef52c598c+        .quad   0x33809107f12d1573+        .quad   0x08ba696b531d5bd8+        .quad   0x77ed3eeec3efc57a+        .quad   0x04e05517d4ff4811+        .quad   0xea3d7a3ff1a671cb+        .quad   0x120633b4947cfe54++        // 2^64 * 7 * G++        .quad   0x0b94987891610042+        .quad   0x4ee7b13cecebfae8+        .quad   0x70be739594f0a4c0+        .quad   0x35d30a99b4d59185+        .quad   0x82bd31474912100a+        .quad   0xde237b6d7e6fbe06+        .quad   0xe11e761911ea79c6+        .quad   0x07433be3cb393bde+        .quad   0xff7944c05ce997f4+        .quad   0x575d3de4b05c51a3+        .quad   0x583381fd5a76847c+        .quad   0x2d873ede7af6da9f++        // 2^64 * 8 * G++        .quad   0x157a316443373409+        .quad   0xfab8b7eef4aa81d9+        .quad   0xb093fee6f5a64806+        .quad   0x2e773654707fa7b6+        .quad   0xaa6202e14e5df981+        .quad   0xa20d59175015e1f5+        .quad   0x18a275d3bae21d6c+        .quad   0x0543618a01600253+        .quad   0x0deabdf4974c23c1+        .quad   0xaa6f0a259dce4693+        .quad   0x04202cb8a29aba2c+        .quad   0x4b1443362d07960d++        // 2^68 * 1 * G++        .quad   0x47b837f753242cec+        .quad   0x256dc48cc04212f2+        .quad   0xe222fbfbe1d928c5+        .quad   0x48ea295bad8a2c07+        .quad   0x299b1c3f57c5715e+        .quad   0x96cb929e6b686d90+        .quad   0x3004806447235ab3+        .quad   0x2c435c24a44d9fe1+        .quad   0x0607c97c80f8833f+        .quad   0x0e851578ca25ec5b+        .quad   0x54f7450b161ebb6f+        .quad   0x7bcb4792a0def80e++        // 2^68 * 2 * G++        .quad   0x8487e3d02bc73659+        .quad   0x4baf8445059979df+        .quad   0xd17c975adcad6fbf+        .quad   0x57369f0bdefc96b6+        .quad   0x1cecd0a0045224c2+        .quad   0x757f1b1b69e53952+        .quad   0x775b7a925289f681+        .quad   0x1b6cc62016736148+        .quad   0xf1a9990175638698+        .quad   0x353dd1beeeaa60d3+        .quad   0x849471334c9ba488+        .quad   0x63fa6e6843ade311++        // 2^68 * 3 * G++        .quad   0xd15c20536597c168+        .quad   0x9f73740098d28789+        .quad   0x18aee7f13257ba1f+        .quad   0x3418bfda07346f14+        .quad   0x2195becdd24b5eb7+        .quad   0x5e41f18cc0cd44f9+        .quad   0xdf28074441ca9ede+        .quad   0x07073b98f35b7d67+        .quad   0xd03c676c4ce530d4+        .quad   0x0b64c0473b5df9f4+        .quad   0x065cef8b19b3a31e+        .quad   0x3084d661533102c9++        // 2^68 * 4 * G++        .quad   0xe1f6b79ebf8469ad+        .quad   0x15801004e2663135+        .quad   0x9a498330af74181b+        .quad   0x3ba2504f049b673c+        .quad   0x9a6ce876760321fd+        .quad   0x7fe2b5109eb63ad8+        .quad   0x00e7d4ae8ac80592+        .quad   0x73d86b7abb6f723a+        .quad   0x0b52b5606dba5ab6+        .quad   0xa9134f0fbbb1edab+        .quad   0x30a9520d9b04a635+        .quad   0x6813b8f37973e5db++        // 2^68 * 5 * G++        .quad   0x9854b054334127c1+        .quad   0x105d047882fbff25+        .quad   0xdb49f7f944186f4f+        .quad   0x1768e838bed0b900+        .quad   0xf194ca56f3157e29+        .quad   0x136d35705ef528a5+        .quad   0xdd4cef778b0599bc+        .quad   0x7d5472af24f833ed+        .quad   0xd0ef874daf33da47+        .quad   0x00d3be5db6e339f9+        .quad   0x3f2a8a2f9c9ceece+        .quad   0x5d1aeb792352435a++        // 2^68 * 6 * G++        .quad   0xf59e6bb319cd63ca+        .quad   0x670c159221d06839+        .quad   0xb06d565b2150cab6+        .quad   0x20fb199d104f12a3+        .quad   0x12c7bfaeb61ba775+        .quad   0xb84e621fe263bffd+        .quad   0x0b47a5c35c840dcf+        .quad   0x7e83be0bccaf8634+        .quad   0x61943dee6d99c120+        .quad   0x86101f2e460b9fe0+        .quad   0x6bb2f1518ee8598d+        .quad   0x76b76289fcc475cc++        // 2^68 * 7 * G++        .quad   0x791b4cc1756286fa+        .quad   0xdbced317d74a157c+        .quad   0x7e732421ea72bde6+        .quad   0x01fe18491131c8e9+        .quad   0x4245f1a1522ec0b3+        .quad   0x558785b22a75656d+        .quad   0x1d485a2548a1b3c0+        .quad   0x60959eccd58fe09f+        .quad   0x3ebfeb7ba8ed7a09+        .quad   0x49fdc2bbe502789c+        .quad   0x44ebce5d3c119428+        .quad   0x35e1eb55be947f4a++        // 2^68 * 8 * G++        .quad   0xdbdae701c5738dd3+        .quad   0xf9c6f635b26f1bee+        .quad   0x61e96a8042f15ef4+        .quad   0x3aa1d11faf60a4d8+        .quad   0x14fd6dfa726ccc74+        .quad   0x3b084cfe2f53b965+        .quad   0xf33ae4f552a2c8b4+        .quad   0x59aab07a0d40166a+        .quad   0x77bcec4c925eac25+        .quad   0x1848718460137738+        .quad   0x5b374337fea9f451+        .quad   0x1865e78ec8e6aa46++        // 2^72 * 1 * G++        .quad   0xccc4b7c7b66e1f7a+        .quad   0x44157e25f50c2f7e+        .quad   0x3ef06dfc713eaf1c+        .quad   0x582f446752da63f7+        .quad   0x967c54e91c529ccb+        .quad   0x30f6269264c635fb+        .quad   0x2747aff478121965+        .quad   0x17038418eaf66f5c+        .quad   0xc6317bd320324ce4+        .quad   0xa81042e8a4488bc4+        .quad   0xb21ef18b4e5a1364+        .quad   0x0c2a1c4bcda28dc9++        // 2^72 * 2 * G++        .quad   0xd24dc7d06f1f0447+        .quad   0xb2269e3edb87c059+        .quad   0xd15b0272fbb2d28f+        .quad   0x7c558bd1c6f64877+        .quad   0xedc4814869bd6945+        .quad   0x0d6d907dbe1c8d22+        .quad   0xc63bd212d55cc5ab+        .quad   0x5a6a9b30a314dc83+        .quad   0xd0ec1524d396463d+        .quad   0x12bb628ac35a24f0+        .quad   0xa50c3a791cbc5fa4+        .quad   0x0404a5ca0afbafc3++        // 2^72 * 3 * G++        .quad   0x8c1f40070aa743d6+        .quad   0xccbad0cb5b265ee8+        .quad   0x574b046b668fd2de+        .quad   0x46395bfdcadd9633+        .quad   0x62bc9e1b2a416fd1+        .quad   0xb5c6f728e350598b+        .quad   0x04343fd83d5d6967+        .quad   0x39527516e7f8ee98+        .quad   0x117fdb2d1a5d9a9c+        .quad   0x9c7745bcd1005c2a+        .quad   0xefd4bef154d56fea+        .quad   0x76579a29e822d016++        // 2^72 * 4 * G++        .quad   0x45b68e7e49c02a17+        .quad   0x23cd51a2bca9a37f+        .quad   0x3ed65f11ec224c1b+        .quad   0x43a384dc9e05bdb1+        .quad   0x333cb51352b434f2+        .quad   0xd832284993de80e1+        .quad   0xb5512887750d35ce+        .quad   0x02c514bb2a2777c1+        .quad   0x684bd5da8bf1b645+        .quad   0xfb8bd37ef6b54b53+        .quad   0x313916d7a9b0d253+        .quad   0x1160920961548059++        // 2^72 * 5 * G++        .quad   0xb44d166929dacfaa+        .quad   0xda529f4c8413598f+        .quad   0xe9ef63ca453d5559+        .quad   0x351e125bc5698e0b+        .quad   0x7a385616369b4dcd+        .quad   0x75c02ca7655c3563+        .quad   0x7dc21bf9d4f18021+        .quad   0x2f637d7491e6e042+        .quad   0xd4b49b461af67bbe+        .quad   0xd603037ac8ab8961+        .quad   0x71dee19ff9a699fb+        .quad   0x7f182d06e7ce2a9a++        // 2^72 * 6 * G++        .quad   0x7a7c8e64ab0168ec+        .quad   0xcb5a4a5515edc543+        .quad   0x095519d347cd0eda+        .quad   0x67d4ac8c343e93b0+        .quad   0x09454b728e217522+        .quad   0xaa58e8f4d484b8d8+        .quad   0xd358254d7f46903c+        .quad   0x44acc043241c5217+        .quad   0x1c7d6bbb4f7a5777+        .quad   0x8b35fed4918313e1+        .quad   0x4adca1c6c96b4684+        .quad   0x556d1c8312ad71bd++        // 2^72 * 7 * G++        .quad   0x17ef40e30c8d3982+        .quad   0x31f7073e15a3fa34+        .quad   0x4f21f3cb0773646e+        .quad   0x746c6c6d1d824eff+        .quad   0x81f06756b11be821+        .quad   0x0faff82310a3f3dd+        .quad   0xf8b2d0556a99465d+        .quad   0x097abe38cc8c7f05+        .quad   0x0c49c9877ea52da4+        .quad   0x4c4369559bdc1d43+        .quad   0x022c3809f7ccebd2+        .quad   0x577e14a34bee84bd++        // 2^72 * 8 * G++        .quad   0xf0e268ac61a73b0a+        .quad   0xf2fafa103791a5f5+        .quad   0xc1e13e826b6d00e9+        .quad   0x60fa7ee96fd78f42+        .quad   0x94fecebebd4dd72b+        .quad   0xf46a4fda060f2211+        .quad   0x124a5977c0c8d1ff+        .quad   0x705304b8fb009295+        .quad   0xb63d1d354d296ec6+        .quad   0xf3c3053e5fad31d8+        .quad   0x670b958cb4bd42ec+        .quad   0x21398e0ca16353fd++        // 2^76 * 1 * G++        .quad   0x216ab2ca8da7d2ef+        .quad   0x366ad9dd99f42827+        .quad   0xae64b9004fdd3c75+        .quad   0x403a395b53909e62+        .quad   0x86c5fc16861b7e9a+        .quad   0xf6a330476a27c451+        .quad   0x01667267a1e93597+        .quad   0x05ffb9cd6082dfeb+        .quad   0xa617fa9ff53f6139+        .quad   0x60f2b5e513e66cb6+        .quad   0xd7a8beefb3448aa4+        .quad   0x7a2932856f5ea192++        // 2^76 * 2 * G++        .quad   0x0b39d761b02de888+        .quad   0x5f550e7ed2414e1f+        .quad   0xa6bfa45822e1a940+        .quad   0x050a2f7dfd447b99+        .quad   0xb89c444879639302+        .quad   0x4ae4f19350c67f2c+        .quad   0xf0b35da8c81af9c6+        .quad   0x39d0003546871017+        .quad   0x437c3b33a650db77+        .quad   0x6bafe81dbac52bb2+        .quad   0xfe99402d2db7d318+        .quad   0x2b5b7eec372ba6ce++        // 2^76 * 3 * G++        .quad   0xb3bc4bbd83f50eef+        .quad   0x508f0c998c927866+        .quad   0x43e76587c8b7e66e+        .quad   0x0f7655a3a47f98d9+        .quad   0xa694404d613ac8f4+        .quad   0x500c3c2bfa97e72c+        .quad   0x874104d21fcec210+        .quad   0x1b205fb38604a8ee+        .quad   0x55ecad37d24b133c+        .quad   0x441e147d6038c90b+        .quad   0x656683a1d62c6fee+        .quad   0x0157d5dc87e0ecae++        // 2^76 * 4 * G++        .quad   0xf2a7af510354c13d+        .quad   0xd7a0b145aa372b60+        .quad   0x2869b96a05a3d470+        .quad   0x6528e42d82460173+        .quad   0x95265514d71eb524+        .quad   0xe603d8815df14593+        .quad   0x147cdf410d4de6b7+        .quad   0x5293b1730437c850+        .quad   0x23d0e0814bccf226+        .quad   0x92c745cd8196fb93+        .quad   0x8b61796c59541e5b+        .quad   0x40a44df0c021f978++        // 2^76 * 5 * G++        .quad   0xdaa869894f20ea6a+        .quad   0xea14a3d14c620618+        .quad   0x6001fccb090bf8be+        .quad   0x35f4e822947e9cf0+        .quad   0x86c96e514bc5d095+        .quad   0xf20d4098fca6804a+        .quad   0x27363d89c826ea5d+        .quad   0x39ca36565719cacf+        .quad   0x97506f2f6f87b75c+        .quad   0xc624aea0034ae070+        .quad   0x1ec856e3aad34dd6+        .quad   0x055b0be0e440e58f++        // 2^76 * 6 * G++        .quad   0x6469a17d89735d12+        .quad   0xdb6f27d5e662b9f1+        .quad   0x9fcba3286a395681+        .quad   0x363b8004d269af25+        .quad   0x4d12a04b6ea33da2+        .quad   0x57cf4c15e36126dd+        .quad   0x90ec9675ee44d967+        .quad   0x64ca348d2a985aac+        .quad   0x99588e19e4c4912d+        .quad   0xefcc3b4e1ca5ce6b+        .quad   0x4522ea60fa5b98d5+        .quad   0x7064bbab1de4a819++        // 2^76 * 7 * G++        .quad   0xb919e1515a770641+        .quad   0xa9a2e2c74e7f8039+        .quad   0x7527250b3df23109+        .quad   0x756a7330ac27b78b+        .quad   0xa290c06142542129+        .quad   0xf2e2c2aebe8d5b90+        .quad   0xcf2458db76abfe1b+        .quad   0x02157ade83d626bf+        .quad   0x3e46972a1b9a038b+        .quad   0x2e4ee66a7ee03fb4+        .quad   0x81a248776edbb4ca+        .quad   0x1a944ee88ecd0563++        // 2^76 * 8 * G++        .quad   0xd5a91d1151039372+        .quad   0x2ed377b799ca26de+        .quad   0xa17202acfd366b6b+        .quad   0x0730291bd6901995+        .quad   0xbb40a859182362d6+        .quad   0xb99f55778a4d1abb+        .quad   0x8d18b427758559f6+        .quad   0x26c20fe74d26235a+        .quad   0x648d1d9fe9cc22f5+        .quad   0x66bc561928dd577c+        .quad   0x47d3ed21652439d1+        .quad   0x49d271acedaf8b49++        // 2^80 * 1 * G++        .quad   0x89f5058a382b33f3+        .quad   0x5ae2ba0bad48c0b4+        .quad   0x8f93b503a53db36e+        .quad   0x5aa3ed9d95a232e6+        .quad   0x2798aaf9b4b75601+        .quad   0x5eac72135c8dad72+        .quad   0xd2ceaa6161b7a023+        .quad   0x1bbfb284e98f7d4e+        .quad   0x656777e9c7d96561+        .quad   0xcb2b125472c78036+        .quad   0x65053299d9506eee+        .quad   0x4a07e14e5e8957cc++        // 2^80 * 2 * G++        .quad   0x4ee412cb980df999+        .quad   0xa315d76f3c6ec771+        .quad   0xbba5edde925c77fd+        .quad   0x3f0bac391d313402+        .quad   0x240b58cdc477a49b+        .quad   0xfd38dade6447f017+        .quad   0x19928d32a7c86aad+        .quad   0x50af7aed84afa081+        .quad   0x6e4fde0115f65be5+        .quad   0x29982621216109b2+        .quad   0x780205810badd6d9+        .quad   0x1921a316baebd006++        // 2^80 * 3 * G++        .quad   0x89422f7edfb870fc+        .quad   0x2c296beb4f76b3bd+        .quad   0x0738f1d436c24df7+        .quad   0x6458df41e273aeb0+        .quad   0xd75aad9ad9f3c18b+        .quad   0x566a0eef60b1c19c+        .quad   0x3e9a0bac255c0ed9+        .quad   0x7b049deca062c7f5+        .quad   0xdccbe37a35444483+        .quad   0x758879330fedbe93+        .quad   0x786004c312c5dd87+        .quad   0x6093dccbc2950e64++        // 2^80 * 4 * G++        .quad   0x1ff39a8585e0706d+        .quad   0x36d0a5d8b3e73933+        .quad   0x43b9f2e1718f453b+        .quad   0x57d1ea084827a97c+        .quad   0x6bdeeebe6084034b+        .quad   0x3199c2b6780fb854+        .quad   0x973376abb62d0695+        .quad   0x6e3180c98b647d90+        .quad   0xee7ab6e7a128b071+        .quad   0xa4c1596d93a88baa+        .quad   0xf7b4de82b2216130+        .quad   0x363e999ddd97bd18++        // 2^80 * 5 * G++        .quad   0x96a843c135ee1fc4+        .quad   0x976eb35508e4c8cf+        .quad   0xb42f6801b58cd330+        .quad   0x48ee9b78693a052b+        .quad   0x2f1848dce24baec6+        .quad   0x769b7255babcaf60+        .quad   0x90cb3c6e3cefe931+        .quad   0x231f979bc6f9b355+        .quad   0x5c31de4bcc2af3c6+        .quad   0xb04bb030fe208d1f+        .quad   0xb78d7009c14fb466+        .quad   0x079bfa9b08792413++        // 2^80 * 6 * G++        .quad   0xe3903a51da300df4+        .quad   0x843964233da95ab0+        .quad   0xed3cf12d0b356480+        .quad   0x038c77f684817194+        .quad   0xf3c9ed80a2d54245+        .quad   0x0aa08b7877f63952+        .quad   0xd76dac63d1085475+        .quad   0x1ef4fb159470636b+        .quad   0x854e5ee65b167bec+        .quad   0x59590a4296d0cdc2+        .quad   0x72b2df3498102199+        .quad   0x575ee92a4a0bff56++        // 2^80 * 7 * G++        .quad   0xd4c080908a182fcf+        .quad   0x30e170c299489dbd+        .quad   0x05babd5752f733de+        .quad   0x43d4e7112cd3fd00+        .quad   0x5d46bc450aa4d801+        .quad   0xc3af1227a533b9d8+        .quad   0x389e3b262b8906c2+        .quad   0x200a1e7e382f581b+        .quad   0x518db967eaf93ac5+        .quad   0x71bc989b056652c0+        .quad   0xfe2b85d9567197f5+        .quad   0x050eca52651e4e38++        // 2^80 * 8 * G++        .quad   0xc3431ade453f0c9c+        .quad   0xe9f5045eff703b9b+        .quad   0xfcd97ac9ed847b3d+        .quad   0x4b0ee6c21c58f4c6+        .quad   0x97ac397660e668ea+        .quad   0x9b19bbfe153ab497+        .quad   0x4cb179b534eca79f+        .quad   0x6151c09fa131ae57+        .quad   0x3af55c0dfdf05d96+        .quad   0xdd262ee02ab4ee7a+        .quad   0x11b2bb8712171709+        .quad   0x1fef24fa800f030b++        // 2^84 * 1 * G++        .quad   0xb496123a6b6c6609+        .quad   0xa750fe8580ab5938+        .quad   0xf471bf39b7c27a5f+        .quad   0x507903ce77ac193c+        .quad   0xff91a66a90166220+        .quad   0xf22552ae5bf1e009+        .quad   0x7dff85d87f90df7c+        .quad   0x4f620ffe0c736fb9+        .quad   0x62f90d65dfde3e34+        .quad   0xcf28c592b9fa5fad+        .quad   0x99c86ef9c6164510+        .quad   0x25d448044a256c84++        // 2^84 * 2 * G++        .quad   0xbd68230ec7e9b16f+        .quad   0x0eb1b9c1c1c5795d+        .quad   0x7943c8c495b6b1ff+        .quad   0x2f9faf620bbacf5e+        .quad   0x2c7c4415c9022b55+        .quad   0x56a0d241812eb1fe+        .quad   0xf02ea1c9d7b65e0d+        .quad   0x4180512fd5323b26+        .quad   0xa4ff3e698a48a5db+        .quad   0xba6a3806bd95403b+        .quad   0x9f7ce1af47d5b65d+        .quad   0x15e087e55939d2fb++        // 2^84 * 3 * G++        .quad   0x12207543745c1496+        .quad   0xdaff3cfdda38610c+        .quad   0xe4e797272c71c34f+        .quad   0x39c07b1934bdede9+        .quad   0x8894186efb963f38+        .quad   0x48a00e80dc639bd5+        .quad   0xa4e8092be96c1c99+        .quad   0x5a097d54ca573661+        .quad   0x2d45892b17c9e755+        .quad   0xd033fd7289308df8+        .quad   0x6c2fe9d9525b8bd9+        .quad   0x2edbecf1c11cc079++        // 2^84 * 4 * G++        .quad   0x1616a4e3c715a0d2+        .quad   0x53623cb0f8341d4d+        .quad   0x96ef5329c7e899cb+        .quad   0x3d4e8dbba668baa6+        .quad   0xee0f0fddd087a25f+        .quad   0x9c7531555c3e34ee+        .quad   0x660c572e8fab3ab5+        .quad   0x0854fc44544cd3b2+        .quad   0x61eba0c555edad19+        .quad   0x24b533fef0a83de6+        .quad   0x3b77042883baa5f8+        .quad   0x678f82b898a47e8d++        // 2^84 * 5 * G++        .quad   0xb1491d0bd6900c54+        .quad   0x3539722c9d132636+        .quad   0x4db928920b362bc9+        .quad   0x4d7cd1fea68b69df+        .quad   0x1e09d94057775696+        .quad   0xeed1265c3cd951db+        .quad   0xfa9dac2b20bce16f+        .quad   0x0f7f76e0e8d089f4+        .quad   0x36d9ebc5d485b00c+        .quad   0xa2596492e4adb365+        .quad   0xc1659480c2119ccd+        .quad   0x45306349186e0d5f++        // 2^84 * 6 * G++        .quad   0x94ddd0c1a6cdff1d+        .quad   0x55f6f115e84213ae+        .quad   0x6c935f85992fcf6a+        .quad   0x067ee0f54a37f16f+        .quad   0x96a414ec2b072491+        .quad   0x1bb2218127a7b65b+        .quad   0x6d2849596e8a4af0+        .quad   0x65f3b08ccd27765f+        .quad   0xecb29fff199801f7+        .quad   0x9d361d1fa2a0f72f+        .quad   0x25f11d2375fd2f49+        .quad   0x124cefe80fe10fe2++        // 2^84 * 7 * G++        .quad   0x4c126cf9d18df255+        .quad   0xc1d471e9147a63b6+        .quad   0x2c6d3c73f3c93b5f+        .quad   0x6be3a6a2e3ff86a2+        .quad   0x1518e85b31b16489+        .quad   0x8faadcb7db710bfb+        .quad   0x39b0bdf4a14ae239+        .quad   0x05f4cbea503d20c1+        .quad   0xce040e9ec04145bc+        .quad   0xc71ff4e208f6834c+        .quad   0xbd546e8dab8847a3+        .quad   0x64666aa0a4d2aba5++        // 2^84 * 8 * G++        .quad   0x6841435a7c06d912+        .quad   0xca123c21bb3f830b+        .quad   0xd4b37b27b1cbe278+        .quad   0x1d753b84c76f5046+        .quad   0xb0c53bf73337e94c+        .quad   0x7cb5697e11e14f15+        .quad   0x4b84abac1930c750+        .quad   0x28dd4abfe0640468+        .quad   0x7dc0b64c44cb9f44+        .quad   0x18a3e1ace3925dbf+        .quad   0x7a3034862d0457c4+        .quad   0x4c498bf78a0c892e++        // 2^88 * 1 * G++        .quad   0x37d653fb1aa73196+        .quad   0x0f9495303fd76418+        .quad   0xad200b09fb3a17b2+        .quad   0x544d49292fc8613e+        .quad   0x22d2aff530976b86+        .quad   0x8d90b806c2d24604+        .quad   0xdca1896c4de5bae5+        .quad   0x28005fe6c8340c17+        .quad   0x6aefba9f34528688+        .quad   0x5c1bff9425107da1+        .quad   0xf75bbbcd66d94b36+        .quad   0x72e472930f316dfa++        // 2^88 * 2 * G++        .quad   0x2695208c9781084f+        .quad   0xb1502a0b23450ee1+        .quad   0xfd9daea603efde02+        .quad   0x5a9d2e8c2733a34c+        .quad   0x07f3f635d32a7627+        .quad   0x7aaa4d865f6566f0+        .quad   0x3c85e79728d04450+        .quad   0x1fee7f000fe06438+        .quad   0x765305da03dbf7e5+        .quad   0xa4daf2491434cdbd+        .quad   0x7b4ad5cdd24a88ec+        .quad   0x00f94051ee040543++        // 2^88 * 3 * G++        .quad   0x8d356b23c3d330b2+        .quad   0xf21c8b9bb0471b06+        .quad   0xb36c316c6e42b83c+        .quad   0x07d79c7e8beab10d+        .quad   0xd7ef93bb07af9753+        .quad   0x583ed0cf3db766a7+        .quad   0xce6998bf6e0b1ec5+        .quad   0x47b7ffd25dd40452+        .quad   0x87fbfb9cbc08dd12+        .quad   0x8a066b3ae1eec29b+        .quad   0x0d57242bdb1fc1bf+        .quad   0x1c3520a35ea64bb6++        // 2^88 * 4 * G++        .quad   0x80d253a6bccba34a+        .quad   0x3e61c3a13838219b+        .quad   0x90c3b6019882e396+        .quad   0x1c3d05775d0ee66f+        .quad   0xcda86f40216bc059+        .quad   0x1fbb231d12bcd87e+        .quad   0xb4956a9e17c70990+        .quad   0x38750c3b66d12e55+        .quad   0x692ef1409422e51a+        .quad   0xcbc0c73c2b5df671+        .quad   0x21014fe7744ce029+        .quad   0x0621e2c7d330487c++        // 2^88 * 5 * G++        .quad   0xaf9860cc8259838d+        .quad   0x90ea48c1c69f9adc+        .quad   0x6526483765581e30+        .quad   0x0007d6097bd3a5bc+        .quad   0xb7ae1796b0dbf0f3+        .quad   0x54dfafb9e17ce196+        .quad   0x25923071e9aaa3b4+        .quad   0x5d8e589ca1002e9d+        .quad   0xc0bf1d950842a94b+        .quad   0xb2d3c363588f2e3e+        .quad   0x0a961438bb51e2ef+        .quad   0x1583d7783c1cbf86++        // 2^88 * 6 * G++        .quad   0xeceea2ef5da27ae1+        .quad   0x597c3a1455670174+        .quad   0xc9a62a126609167a+        .quad   0x252a5f2e81ed8f70+        .quad   0x90034704cc9d28c7+        .quad   0x1d1b679ef72cc58f+        .quad   0x16e12b5fbe5b8726+        .quad   0x4958064e83c5580a+        .quad   0x0d2894265066e80d+        .quad   0xfcc3f785307c8c6b+        .quad   0x1b53da780c1112fd+        .quad   0x079c170bd843b388++        // 2^88 * 7 * G++        .quad   0x0506ece464fa6fff+        .quad   0xbee3431e6205e523+        .quad   0x3579422451b8ea42+        .quad   0x6dec05e34ac9fb00+        .quad   0xcdd6cd50c0d5d056+        .quad   0x9af7686dbb03573b+        .quad   0x3ca6723ff3c3ef48+        .quad   0x6768c0d7317b8acc+        .quad   0x94b625e5f155c1b3+        .quad   0x417bf3a7997b7b91+        .quad   0xc22cbddc6d6b2600+        .quad   0x51445e14ddcd52f4++        // 2^88 * 8 * G++        .quad   0x57502b4b3b144951+        .quad   0x8e67ff6b444bbcb3+        .quad   0xb8bd6927166385db+        .quad   0x13186f31e39295c8+        .quad   0x893147ab2bbea455+        .quad   0x8c53a24f92079129+        .quad   0x4b49f948be30f7a7+        .quad   0x12e990086e4fd43d+        .quad   0xf10c96b37fdfbb2e+        .quad   0x9f9a935e121ceaf9+        .quad   0xdf1136c43a5b983f+        .quad   0x77b2e3f05d3e99af++        // 2^92 * 1 * G++        .quad   0xfd0d75879cf12657+        .quad   0xe82fef94e53a0e29+        .quad   0xcc34a7f05bbb4be7+        .quad   0x0b251172a50c38a2+        .quad   0x9532f48fcc5cd29b+        .quad   0x2ba851bea3ce3671+        .quad   0x32dacaa051122941+        .quad   0x478d99d9350004f2+        .quad   0x1d5ad94890bb02c0+        .quad   0x50e208b10ec25115+        .quad   0xa26a22894ef21702+        .quad   0x4dc923343b524805++        // 2^92 * 2 * G++        .quad   0xe3828c400f8086b6+        .quad   0x3f77e6f7979f0dc8+        .quad   0x7ef6de304df42cb4+        .quad   0x5265797cb6abd784+        .quad   0x3ad3e3ebf36c4975+        .quad   0xd75d25a537862125+        .quad   0xe873943da025a516+        .quad   0x6bbc7cb4c411c847+        .quad   0x3c6f9cd1d4a50d56+        .quad   0xb6244077c6feab7e+        .quad   0x6ff9bf483580972e+        .quad   0x00375883b332acfb++        // 2^92 * 3 * G++        .quad   0x0001b2cd28cb0940+        .quad   0x63fb51a06f1c24c9+        .quad   0xb5ad8691dcd5ca31+        .quad   0x67238dbd8c450660+        .quad   0xc98bec856c75c99c+        .quad   0xe44184c000e33cf4+        .quad   0x0a676b9bba907634+        .quad   0x669e2cb571f379d7+        .quad   0xcb116b73a49bd308+        .quad   0x025aad6b2392729e+        .quad   0xb4793efa3f55d9b1+        .quad   0x72a1056140678bb9++        // 2^92 * 4 * G++        .quad   0xa2b6812b1cc9249d+        .quad   0x62866eee21211f58+        .quad   0x2cb5c5b85df10ece+        .quad   0x03a6b259e263ae00+        .quad   0x0d8d2909e2e505b6+        .quad   0x98ca78abc0291230+        .quad   0x77ef5569a9b12327+        .quad   0x7c77897b81439b47+        .quad   0xf1c1b5e2de331cb5+        .quad   0x5a9f5d8e15fca420+        .quad   0x9fa438f17bd932b1+        .quad   0x2a381bf01c6146e7++        // 2^92 * 5 * G++        .quad   0xac9b9879cfc811c1+        .quad   0x8b7d29813756e567+        .quad   0x50da4e607c70edfc+        .quad   0x5dbca62f884400b6+        .quad   0xf7c0be32b534166f+        .quad   0x27e6ca6419cf70d4+        .quad   0x934df7d7a957a759+        .quad   0x5701461dabdec2aa+        .quad   0x2c6747402c915c25+        .quad   0x1bdcd1a80b0d340a+        .quad   0x5e5601bd07b43f5f+        .quad   0x2555b4e05539a242++        // 2^92 * 6 * G++        .quad   0x6fc09f5266ddd216+        .quad   0xdce560a7c8e37048+        .quad   0xec65939da2df62fd+        .quad   0x7a869ae7e52ed192+        .quad   0x78409b1d87e463d4+        .quad   0xad4da95acdfb639d+        .quad   0xec28773755259b9c+        .quad   0x69c806e9c31230ab+        .quad   0x7b48f57414bb3f22+        .quad   0x68c7cee4aedccc88+        .quad   0xed2f936179ed80be+        .quad   0x25d70b885f77bc4b++        // 2^92 * 7 * G++        .quad   0x4151c3d9762bf4de+        .quad   0x083f435f2745d82b+        .quad   0x29775a2e0d23ddd5+        .quad   0x138e3a6269a5db24+        .quad   0x98459d29bb1ae4d4+        .quad   0x56b9c4c739f954ec+        .quad   0x832743f6c29b4b3e+        .quad   0x21ea8e2798b6878a+        .quad   0x87bef4b46a5a7b9c+        .quad   0xd2299d1b5fc1d062+        .quad   0x82409818dd321648+        .quad   0x5c5abeb1e5a2e03d++        // 2^92 * 8 * G++        .quad   0x14722af4b73c2ddb+        .quad   0xbc470c5f5a05060d+        .quad   0x00943eac2581b02e+        .quad   0x0e434b3b1f499c8f+        .quad   0x02cde6de1306a233+        .quad   0x7b5a52a2116f8ec7+        .quad   0xe1c681f4c1163b5b+        .quad   0x241d350660d32643+        .quad   0x6be4404d0ebc52c7+        .quad   0xae46233bb1a791f5+        .quad   0x2aec170ed25db42b+        .quad   0x1d8dfd966645d694++        // 2^96 * 1 * G++        .quad   0x296fa9c59c2ec4de+        .quad   0xbc8b61bf4f84f3cb+        .quad   0x1c7706d917a8f908+        .quad   0x63b795fc7ad3255d+        .quad   0xd598639c12ddb0a4+        .quad   0xa5d19f30c024866b+        .quad   0xd17c2f0358fce460+        .quad   0x07a195152e095e8a+        .quad   0xa8368f02389e5fc8+        .quad   0x90433b02cf8de43b+        .quad   0xafa1fd5dc5412643+        .quad   0x3e8fe83d032f0137++        // 2^96 * 2 * G++        .quad   0x2f8b15b90570a294+        .quad   0x94f2427067084549+        .quad   0xde1c5ae161bbfd84+        .quad   0x75ba3b797fac4007+        .quad   0x08704c8de8efd13c+        .quad   0xdfc51a8e33e03731+        .quad   0xa59d5da51260cde3+        .quad   0x22d60899a6258c86+        .quad   0x6239dbc070cdd196+        .quad   0x60fe8a8b6c7d8a9a+        .quad   0xb38847bceb401260+        .quad   0x0904d07b87779e5e++        // 2^96 * 3 * G++        .quad   0xb4ce1fd4ddba919c+        .quad   0xcf31db3ec74c8daa+        .quad   0x2c63cc63ad86cc51+        .quad   0x43e2143fbc1dde07+        .quad   0xf4322d6648f940b9+        .quad   0x06952f0cbd2d0c39+        .quad   0x167697ada081f931+        .quad   0x6240aacebaf72a6c+        .quad   0xf834749c5ba295a0+        .quad   0xd6947c5bca37d25a+        .quad   0x66f13ba7e7c9316a+        .quad   0x56bdaf238db40cac++        // 2^96 * 4 * G++        .quad   0x362ab9e3f53533eb+        .quad   0x338568d56eb93d40+        .quad   0x9e0e14521d5a5572+        .quad   0x1d24a86d83741318+        .quad   0x1310d36cc19d3bb2+        .quad   0x062a6bb7622386b9+        .quad   0x7c9b8591d7a14f5c+        .quad   0x03aa31507e1e5754+        .quad   0xf4ec7648ffd4ce1f+        .quad   0xe045eaf054ac8c1c+        .quad   0x88d225821d09357c+        .quad   0x43b261dc9aeb4859++        // 2^96 * 5 * G++        .quad   0xe55b1e1988bb79bb+        .quad   0xa09ed07dc17a359d+        .quad   0xb02c2ee2603dea33+        .quad   0x326055cf5b276bc2+        .quad   0x19513d8b6c951364+        .quad   0x94fe7126000bf47b+        .quad   0x028d10ddd54f9567+        .quad   0x02b4d5e242940964+        .quad   0xb4a155cb28d18df2+        .quad   0xeacc4646186ce508+        .quad   0xc49cf4936c824389+        .quad   0x27a6c809ae5d3410++        // 2^96 * 6 * G++        .quad   0x8ba6ebcd1f0db188+        .quad   0x37d3d73a675a5be8+        .quad   0xf22edfa315f5585a+        .quad   0x2cb67174ff60a17e+        .quad   0xcd2c270ac43d6954+        .quad   0xdd4a3e576a66cab2+        .quad   0x79fa592469d7036c+        .quad   0x221503603d8c2599+        .quad   0x59eecdf9390be1d0+        .quad   0xa9422044728ce3f1+        .quad   0x82891c667a94f0f4+        .quad   0x7b1df4b73890f436++        // 2^96 * 7 * G++        .quad   0xe492f2e0b3b2a224+        .quad   0x7c6c9e062b551160+        .quad   0x15eb8fe20d7f7b0e+        .quad   0x61fcef2658fc5992+        .quad   0x5f2e221807f8f58c+        .quad   0xe3555c9fd49409d4+        .quad   0xb2aaa88d1fb6a630+        .quad   0x68698245d352e03d+        .quad   0xdbb15d852a18187a+        .quad   0xf3e4aad386ddacd7+        .quad   0x44bae2810ff6c482+        .quad   0x46cf4c473daf01cf++        // 2^96 * 8 * G++        .quad   0x426525ed9ec4e5f9+        .quad   0x0e5eda0116903303+        .quad   0x72b1a7f2cbe5cadc+        .quad   0x29387bcd14eb5f40+        .quad   0x213c6ea7f1498140+        .quad   0x7c1e7ef8392b4854+        .quad   0x2488c38c5629ceba+        .quad   0x1065aae50d8cc5bb+        .quad   0x1c2c4525df200d57+        .quad   0x5c3b2dd6bfca674a+        .quad   0x0a07e7b1e1834030+        .quad   0x69a198e64f1ce716++        // 2^100 * 1 * G++        .quad   0x7afcd613efa9d697+        .quad   0x0cc45aa41c067959+        .quad   0xa56fe104c1fada96+        .quad   0x3a73b70472e40365+        .quad   0x7b26e56b9e2d4734+        .quad   0xc4c7132b81c61675+        .quad   0xef5c9525ec9cde7f+        .quad   0x39c80b16e71743ad+        .quad   0x0f196e0d1b826c68+        .quad   0xf71ff0e24960e3db+        .quad   0x6113167023b7436c+        .quad   0x0cf0ea5877da7282++        // 2^100 * 2 * G++        .quad   0x196c80a4ddd4ccbd+        .quad   0x22e6f55d95f2dd9d+        .quad   0xc75e33c740d6c71b+        .quad   0x7bb51279cb3c042f+        .quad   0xe332ced43ba6945a+        .quad   0xde0b1361e881c05d+        .quad   0x1ad40f095e67ed3b+        .quad   0x5da8acdab8c63d5d+        .quad   0xc4b6664a3a70159f+        .quad   0x76194f0f0a904e14+        .quad   0xa5614c39a4096c13+        .quad   0x6cd0ff50979feced++        // 2^100 * 3 * G++        .quad   0xc0e067e78f4428ac+        .quad   0x14835ab0a61135e3+        .quad   0xf21d14f338062935+        .quad   0x6390a4c8df04849c+        .quad   0x7fecfabdb04ba18e+        .quad   0xd0fc7bfc3bddbcf7+        .quad   0xa41d486e057a131c+        .quad   0x641a4391f2223a61+        .quad   0xc5c6b95aa606a8db+        .quad   0x914b7f9eb06825f1+        .quad   0x2a731f6b44fc9eff+        .quad   0x30ddf38562705cfc++        // 2^100 * 4 * G++        .quad   0x4e3dcbdad1bff7f9+        .quad   0xc9118e8220645717+        .quad   0xbacccebc0f189d56+        .quad   0x1b4822e9d4467668+        .quad   0x33bef2bd68bcd52c+        .quad   0xc649dbb069482ef2+        .quad   0xb5b6ee0c41cb1aee+        .quad   0x5c294d270212a7e5+        .quad   0xab360a7f25563781+        .quad   0x2512228a480f7958+        .quad   0xc75d05276114b4e3+        .quad   0x222d9625d976fe2a++        // 2^100 * 5 * G++        .quad   0x1c717f85b372ace1+        .quad   0x81930e694638bf18+        .quad   0x239cad056bc08b58+        .quad   0x0b34271c87f8fff4+        .quad   0x0f94be7e0a344f85+        .quad   0xeb2faa8c87f22c38+        .quad   0x9ce1e75e4ee16f0f+        .quad   0x43e64e5418a08dea+        .quad   0x8155e2521a35ce63+        .quad   0xbe100d4df912028e+        .quad   0xbff80bf8a57ddcec+        .quad   0x57342dc96d6bc6e4++        // 2^100 * 6 * G++        .quad   0xefeef065c8ce5998+        .quad   0xbf029510b5cbeaa2+        .quad   0x8c64a10620b7c458+        .quad   0x35134fb231c24855+        .quad   0xf3c3bcb71e707bf6+        .quad   0x351d9b8c7291a762+        .quad   0x00502e6edad69a33+        .quad   0x522f521f1ec8807f+        .quad   0x272c1f46f9a3902b+        .quad   0xc91ba3b799657bcc+        .quad   0xae614b304f8a1c0e+        .quad   0x7afcaad70b99017b++        // 2^100 * 7 * G++        .quad   0xc25ded54a4b8be41+        .quad   0x902d13e11bb0e2dd+        .quad   0x41f43233cde82ab2+        .quad   0x1085faa5c3aae7cb+        .quad   0xa88141ecef842b6b+        .quad   0x55e7b14797abe6c5+        .quad   0x8c748f9703784ffe+        .quad   0x5b50a1f7afcd00b7+        .quad   0x9b840f66f1361315+        .quad   0x18462242701003e9+        .quad   0x65ed45fae4a25080+        .quad   0x0a2862393fda7320++        // 2^100 * 8 * G++        .quad   0x46ab13c8347cbc9d+        .quad   0x3849e8d499c12383+        .quad   0x4cea314087d64ac9+        .quad   0x1f354134b1a29ee7+        .quad   0x960e737b6ecb9d17+        .quad   0xfaf24948d67ceae1+        .quad   0x37e7a9b4d55e1b89+        .quad   0x5cb7173cb46c59eb+        .quad   0x4a89e68b82b7abf0+        .quad   0xf41cd9279ba6b7b9+        .quad   0x16e6c210e18d876f+        .quad   0x7cacdb0f7f1b09c6++        // 2^104 * 1 * G++        .quad   0x9062b2e0d91a78bc+        .quad   0x47c9889cc8509667+        .quad   0x9df54a66405070b8+        .quad   0x7369e6a92493a1bf+        .quad   0xe1014434dcc5caed+        .quad   0x47ed5d963c84fb33+        .quad   0x70019576ed86a0e7+        .quad   0x25b2697bd267f9e4+        .quad   0x9d673ffb13986864+        .quad   0x3ca5fbd9415dc7b8+        .quad   0xe04ecc3bdf273b5e+        .quad   0x1420683db54e4cd2++        // 2^104 * 2 * G++        .quad   0xb478bd1e249dd197+        .quad   0x620c35005e58c102+        .quad   0xfb02d32fccbaac5c+        .quad   0x60b63bebf508a72d+        .quad   0x34eebb6fc1cc5ad0+        .quad   0x6a1b0ce99646ac8b+        .quad   0xd3b0da49a66bde53+        .quad   0x31e83b4161d081c1+        .quad   0x97e8c7129e062b4f+        .quad   0x49e48f4f29320ad8+        .quad   0x5bece14b6f18683f+        .quad   0x55cf1eb62d550317++        // 2^104 * 3 * G++        .quad   0x5879101065c23d58+        .quad   0x8b9d086d5094819c+        .quad   0xe2402fa912c55fa7+        .quad   0x669a6564570891d4+        .quad   0x3076b5e37df58c52+        .quad   0xd73ab9dde799cc36+        .quad   0xbd831ce34913ee20+        .quad   0x1a56fbaa62ba0133+        .quad   0x943e6b505c9dc9ec+        .quad   0x302557bba77c371a+        .quad   0x9873ae5641347651+        .quad   0x13c4836799c58a5c++        // 2^104 * 4 * G++        .quad   0x423a5d465ab3e1b9+        .quad   0xfc13c187c7f13f61+        .quad   0x19f83664ecb5b9b6+        .quad   0x66f80c93a637b607+        .quad   0xc4dcfb6a5d8bd080+        .quad   0xdeebc4ec571a4842+        .quad   0xd4b2e883b8e55365+        .quad   0x50bdc87dc8e5b827+        .quad   0x606d37836edfe111+        .quad   0x32353e15f011abd9+        .quad   0x64b03ac325b73b96+        .quad   0x1dd56444725fd5ae++        // 2^104 * 5 * G++        .quad   0x8fa47ff83362127d+        .quad   0xbc9f6ac471cd7c15+        .quad   0x6e71454349220c8b+        .quad   0x0e645912219f732e+        .quad   0xc297e60008bac89a+        .quad   0x7d4cea11eae1c3e0+        .quad   0xf3e38be19fe7977c+        .quad   0x3a3a450f63a305cd+        .quad   0x078f2f31d8394627+        .quad   0x389d3183de94a510+        .quad   0xd1e36c6d17996f80+        .quad   0x318c8d9393a9a87b++        // 2^104 * 6 * G++        .quad   0xf2745d032afffe19+        .quad   0x0c9f3c497f24db66+        .quad   0xbc98d3e3ba8598ef+        .quad   0x224c7c679a1d5314+        .quad   0x5d669e29ab1dd398+        .quad   0xfc921658342d9e3b+        .quad   0x55851dfdf35973cd+        .quad   0x509a41c325950af6+        .quad   0xbdc06edca6f925e9+        .quad   0x793ef3f4641b1f33+        .quad   0x82ec12809d833e89+        .quad   0x05bff02328a11389++        // 2^104 * 7 * G++        .quad   0x3632137023cae00b+        .quad   0x544acf0ad1accf59+        .quad   0x96741049d21a1c88+        .quad   0x780b8cc3fa2a44a7+        .quad   0x6881a0dd0dc512e4+        .quad   0x4fe70dc844a5fafe+        .quad   0x1f748e6b8f4a5240+        .quad   0x576277cdee01a3ea+        .quad   0x1ef38abc234f305f+        .quad   0x9a577fbd1405de08+        .quad   0x5e82a51434e62a0d+        .quad   0x5ff418726271b7a1++        // 2^104 * 8 * G++        .quad   0x398e080c1789db9d+        .quad   0xa7602025f3e778f5+        .quad   0xfa98894c06bd035d+        .quad   0x106a03dc25a966be+        .quad   0xe5db47e813b69540+        .quad   0xf35d2a3b432610e1+        .quad   0xac1f26e938781276+        .quad   0x29d4db8ca0a0cb69+        .quad   0xd9ad0aaf333353d0+        .quad   0x38669da5acd309e5+        .quad   0x3c57658ac888f7f0+        .quad   0x4ab38a51052cbefa++        // 2^108 * 1 * G++        .quad   0xdfdacbee4324c0e9+        .quad   0x054442883f955bb7+        .quad   0xdef7aaa8ea31609f+        .quad   0x68aee70642287cff+        .quad   0xf68fe2e8809de054+        .quad   0xe3bc096a9c82bad1+        .quad   0x076353d40aadbf45+        .quad   0x7b9b1fb5dea1959e+        .quad   0xf01cc8f17471cc0c+        .quad   0x95242e37579082bb+        .quad   0x27776093d3e46b5f+        .quad   0x2d13d55a28bd85fb++        // 2^108 * 2 * G++        .quad   0xfac5d2065b35b8da+        .quad   0xa8da8a9a85624bb7+        .quad   0xccd2ca913d21cd0f+        .quad   0x6b8341ee8bf90d58+        .quad   0xbf019cce7aee7a52+        .quad   0xa8ded2b6e454ead3+        .quad   0x3c619f0b87a8bb19+        .quad   0x3619b5d7560916d8+        .quad   0x3579f26b0282c4b2+        .quad   0x64d592f24fafefae+        .quad   0xb7cded7b28c8c7c0+        .quad   0x6a927b6b7173a8d7++        // 2^108 * 3 * G++        .quad   0x1f6db24f986e4656+        .quad   0x1021c02ed1e9105b+        .quad   0xf8ff3fff2cc0a375+        .quad   0x1d2a6bf8c6c82592+        .quad   0x8d7040863ece88eb+        .quad   0xf0e307a980eec08c+        .quad   0xac2250610d788fda+        .quad   0x056d92a43a0d478d+        .quad   0x1b05a196fc3da5a1+        .quad   0x77d7a8c243b59ed0+        .quad   0x06da3d6297d17918+        .quad   0x66fbb494f12353f7++        // 2^108 * 4 * G++        .quad   0x751a50b9d85c0fb8+        .quad   0xd1afdc258bcf097b+        .quad   0x2f16a6a38309a969+        .quad   0x14ddff9ee5b00659+        .quad   0xd6d70996f12309d6+        .quad   0xdbfb2385e9c3d539+        .quad   0x46d602b0f7552411+        .quad   0x270a0b0557843e0c+        .quad   0x61ff0640a7862bcc+        .quad   0x81cac09a5f11abfe+        .quad   0x9047830455d12abb+        .quad   0x19a4bde1945ae873++        // 2^108 * 5 * G++        .quad   0x9b9f26f520a6200a+        .quad   0x64804443cf13eaf8+        .quad   0x8a63673f8631edd3+        .quad   0x72bbbce11ed39dc1+        .quad   0x40c709dec076c49f+        .quad   0x657bfaf27f3e53f6+        .quad   0x40662331eca042c4+        .quad   0x14b375487eb4df04+        .quad   0xae853c94ab66dc47+        .quad   0xeb62343edf762d6e+        .quad   0xf08e0e186fb2f7d1+        .quad   0x4f0b1c02700ab37a++        // 2^108 * 6 * G++        .quad   0xe1706787d81951fa+        .quad   0xa10a2c8eb290c77b+        .quad   0xe7382fa03ed66773+        .quad   0x0a4d84710bcc4b54+        .quad   0x79fd21ccc1b2e23f+        .quad   0x4ae7c281453df52a+        .quad   0xc8172ec9d151486b+        .quad   0x68abe9443e0a7534+        .quad   0xda12c6c407831dcb+        .quad   0x0da230d74d5c510d+        .quad   0x4ab1531e6bd404e1+        .quad   0x4106b166bcf440ef++        // 2^108 * 7 * G++        .quad   0x02e57a421cd23668+        .quad   0x4ad9fb5d0eaef6fd+        .quad   0x954e6727b1244480+        .quad   0x7f792f9d2699f331+        .quad   0xa485ccd539e4ecf2+        .quad   0x5aa3f3ad0555bab5+        .quad   0x145e3439937df82d+        .quad   0x1238b51e1214283f+        .quad   0x0b886b925fd4d924+        .quad   0x60906f7a3626a80d+        .quad   0xecd367b4b98abd12+        .quad   0x2876beb1def344cf++        // 2^108 * 8 * G++        .quad   0xdc84e93563144691+        .quad   0x632fe8a0d61f23f4+        .quad   0x4caa800612a9a8d5+        .quad   0x48f9dbfa0e9918d3+        .quad   0xd594b3333a8a85f8+        .quad   0x4ea37689e78d7d58+        .quad   0x73bf9f455e8e351f+        .quad   0x5507d7d2bc41ebb4+        .quad   0x1ceb2903299572fc+        .quad   0x7c8ccaa29502d0ee+        .quad   0x91bfa43411cce67b+        .quad   0x5784481964a831e7++        // 2^112 * 1 * G++        .quad   0xda7c2b256768d593+        .quad   0x98c1c0574422ca13+        .quad   0xf1a80bd5ca0ace1d+        .quad   0x29cdd1adc088a690+        .quad   0xd6cfd1ef5fddc09c+        .quad   0xe82b3efdf7575dce+        .quad   0x25d56b5d201634c2+        .quad   0x3041c6bb04ed2b9b+        .quad   0x0ff2f2f9d956e148+        .quad   0xade797759f356b2e+        .quad   0x1a4698bb5f6c025c+        .quad   0x104bbd6814049a7b++        // 2^112 * 2 * G++        .quad   0x51f0fd3168f1ed67+        .quad   0x2c811dcdd86f3bc2+        .quad   0x44dc5c4304d2f2de+        .quad   0x5be8cc57092a7149+        .quad   0xa95d9a5fd67ff163+        .quad   0xe92be69d4cc75681+        .quad   0xb7f8024cde20f257+        .quad   0x204f2a20fb072df5+        .quad   0xc8143b3d30ebb079+        .quad   0x7589155abd652e30+        .quad   0x653c3c318f6d5c31+        .quad   0x2570fb17c279161f++        // 2^112 * 3 * G++        .quad   0x3efa367f2cb61575+        .quad   0xf5f96f761cd6026c+        .quad   0xe8c7142a65b52562+        .quad   0x3dcb65ea53030acd+        .quad   0x192ea9550bb8245a+        .quad   0xc8e6fba88f9050d1+        .quad   0x7986ea2d88a4c935+        .quad   0x241c5f91de018668+        .quad   0x28d8172940de6caa+        .quad   0x8fbf2cf022d9733a+        .quad   0x16d7fcdd235b01d1+        .quad   0x08420edd5fcdf0e5++        // 2^112 * 4 * G++        .quad   0xcdff20ab8362fa4a+        .quad   0x57e118d4e21a3e6e+        .quad   0xe3179617fc39e62b+        .quad   0x0d9a53efbc1769fd+        .quad   0x0358c34e04f410ce+        .quad   0xb6135b5a276e0685+        .quad   0x5d9670c7ebb91521+        .quad   0x04d654f321db889c+        .quad   0x5e7dc116ddbdb5d5+        .quad   0x2954deb68da5dd2d+        .quad   0x1cb608173334a292+        .quad   0x4a7a4f2618991ad7++        // 2^112 * 5 * G++        .quad   0xf4a718025fb15f95+        .quad   0x3df65f346b5c1b8f+        .quad   0xcdfcf08500e01112+        .quad   0x11b50c4cddd31848+        .quad   0x24c3b291af372a4b+        .quad   0x93da8270718147f2+        .quad   0xdd84856486899ef2+        .quad   0x4a96314223e0ee33+        .quad   0xa6e8274408a4ffd6+        .quad   0x738e177e9c1576d9+        .quad   0x773348b63d02b3f2+        .quad   0x4f4bce4dce6bcc51++        // 2^112 * 6 * G++        .quad   0xa71fce5ae2242584+        .quad   0x26ea725692f58a9e+        .quad   0xd21a09d71cea3cf4+        .quad   0x73fcdd14b71c01e6+        .quad   0x30e2616ec49d0b6f+        .quad   0xe456718fcaec2317+        .quad   0x48eb409bf26b4fa6+        .quad   0x3042cee561595f37+        .quad   0x427e7079449bac41+        .quad   0x855ae36dbce2310a+        .quad   0x4cae76215f841a7c+        .quad   0x389e740c9a9ce1d6++        // 2^112 * 7 * G++        .quad   0x64fcb3ae34dcb9ce+        .quad   0x97500323e348d0ad+        .quad   0x45b3f07d62c6381b+        .quad   0x61545379465a6788+        .quad   0xc9bd78f6570eac28+        .quad   0xe55b0b3227919ce1+        .quad   0x65fc3eaba19b91ed+        .quad   0x25c425e5d6263690+        .quad   0x3f3e06a6f1d7de6e+        .quad   0x3ef976278e062308+        .quad   0x8c14f6264e8a6c77+        .quad   0x6539a08915484759++        // 2^112 * 8 * G++        .quad   0xe9d21f74c3d2f773+        .quad   0xc150544125c46845+        .quad   0x624e5ce8f9b99e33+        .quad   0x11c5e4aac5cd186c+        .quad   0xddc4dbd414bb4a19+        .quad   0x19b2bc3c98424f8e+        .quad   0x48a89fd736ca7169+        .quad   0x0f65320ef019bd90+        .quad   0xd486d1b1cafde0c6+        .quad   0x4f3fe6e3163b5181+        .quad   0x59a8af0dfaf2939a+        .quad   0x4cabc7bdec33072a++        // 2^116 * 1 * G++        .quad   0x16faa8fb532f7428+        .quad   0xdbd42ea046a4e272+        .quad   0x5337653b8b9ea480+        .quad   0x4065947223973f03+        .quad   0xf7c0a19c1a54a044+        .quad   0x4a1c5e2477bd9fbb+        .quad   0xa6e3ca115af22972+        .quad   0x1819bb953f2e9e0d+        .quad   0x498fbb795e042e84+        .quad   0x7d0dd89a7698b714+        .quad   0x8bfb0ba427fe6295+        .quad   0x36ba82e721200524++        // 2^116 * 2 * G++        .quad   0xd60ecbb74245ec41+        .quad   0xfd9be89e34348716+        .quad   0xc9240afee42284de+        .quad   0x4472f648d0531db4+        .quad   0xc8d69d0a57274ed5+        .quad   0x45ba803260804b17+        .quad   0xdf3cda102255dfac+        .quad   0x77d221232709b339+        .quad   0x498a6d7064ad94d8+        .quad   0xa5b5c8fd9af62263+        .quad   0x8ca8ed0545c141f4+        .quad   0x2c63bec3662d358c++        // 2^116 * 3 * G++        .quad   0x7fe60d8bea787955+        .quad   0xb9dc117eb5f401b7+        .quad   0x91c7c09a19355cce+        .quad   0x22692ef59442bedf+        .quad   0x9a518b3a8586f8bf+        .quad   0x9ee71af6cbb196f0+        .quad   0xaa0625e6a2385cf2+        .quad   0x1deb2176ddd7c8d1+        .quad   0x8563d19a2066cf6c+        .quad   0x401bfd8c4dcc7cd7+        .quad   0xd976a6becd0d8f62+        .quad   0x67cfd773a278b05e++        // 2^116 * 4 * G++        .quad   0x8dec31faef3ee475+        .quad   0x99dbff8a9e22fd92+        .quad   0x512d11594e26cab1+        .quad   0x0cde561eec4310b9+        .quad   0x2d5fa9855a4e586a+        .quad   0x65f8f7a449beab7e+        .quad   0xaa074dddf21d33d3+        .quad   0x185cba721bcb9dee+        .quad   0x93869da3f4e3cb41+        .quad   0xbf0392f540f7977e+        .quad   0x026204fcd0463b83+        .quad   0x3ec91a769eec6eed++        // 2^116 * 5 * G++        .quad   0x1e9df75bf78166ad+        .quad   0x4dfda838eb0cd7af+        .quad   0xba002ed8c1eaf988+        .quad   0x13fedb3e11f33cfc+        .quad   0x0fad2fb7b0a3402f+        .quad   0x46615ecbfb69f4a8+        .quad   0xf745bcc8c5f8eaa6+        .quad   0x7a5fa8794a94e896+        .quad   0x52958faa13cd67a1+        .quad   0x965ee0818bdbb517+        .quad   0x16e58daa2e8845b3+        .quad   0x357d397d5499da8f++        // 2^116 * 6 * G++        .quad   0x1ebfa05fb0bace6c+        .quad   0xc934620c1caf9a1e+        .quad   0xcc771cc41d82b61a+        .quad   0x2d94a16aa5f74fec+        .quad   0x481dacb4194bfbf8+        .quad   0x4d77e3f1bae58299+        .quad   0x1ef4612e7d1372a0+        .quad   0x3a8d867e70ff69e1+        .quad   0x6f58cd5d55aff958+        .quad   0xba3eaa5c75567721+        .quad   0x75c123999165227d+        .quad   0x69be1343c2f2b35e++        // 2^116 * 7 * G++        .quad   0x0e091d5ee197c92a+        .quad   0x4f51019f2945119f+        .quad   0x143679b9f034e99c+        .quad   0x7d88112e4d24c696+        .quad   0x82bbbdac684b8de3+        .quad   0xa2f4c7d03fca0718+        .quad   0x337f92fbe096aaa8+        .quad   0x200d4d8c63587376+        .quad   0x208aed4b4893b32b+        .quad   0x3efbf23ebe59b964+        .quad   0xd762deb0dba5e507+        .quad   0x69607bd681bd9d94++        // 2^116 * 8 * G++        .quad   0xf6be021068de1ce1+        .quad   0xe8d518e70edcbc1f+        .quad   0xe3effdd01b5505a5+        .quad   0x35f63353d3ec3fd0+        .quad   0x3b7f3bd49323a902+        .quad   0x7c21b5566b2c6e53+        .quad   0xe5ba8ff53a7852a7+        .quad   0x28bc77a5838ece00+        .quad   0x63ba78a8e25d8036+        .quad   0x63651e0094333490+        .quad   0x48d82f20288ce532+        .quad   0x3a31abfa36b57524++        // 2^120 * 1 * G++        .quad   0x239e9624089c0a2e+        .quad   0xc748c4c03afe4738+        .quad   0x17dbed2a764fa12a+        .quad   0x639b93f0321c8582+        .quad   0xc08f788f3f78d289+        .quad   0xfe30a72ca1404d9f+        .quad   0xf2778bfccf65cc9d+        .quad   0x7ee498165acb2021+        .quad   0x7bd508e39111a1c3+        .quad   0x2b2b90d480907489+        .quad   0xe7d2aec2ae72fd19+        .quad   0x0edf493c85b602a6++        // 2^120 * 2 * G++        .quad   0xaecc8158599b5a68+        .quad   0xea574f0febade20e+        .quad   0x4fe41d7422b67f07+        .quad   0x403b92e3019d4fb4+        .quad   0x6767c4d284764113+        .quad   0xa090403ff7f5f835+        .quad   0x1c8fcffacae6bede+        .quad   0x04c00c54d1dfa369+        .quad   0x4dc22f818b465cf8+        .quad   0x71a0f35a1480eff8+        .quad   0xaee8bfad04c7d657+        .quad   0x355bb12ab26176f4++        // 2^120 * 3 * G++        .quad   0xa71e64cc7493bbf4+        .quad   0xe5bd84d9eca3b0c3+        .quad   0x0a6bc50cfa05e785+        .quad   0x0f9b8132182ec312+        .quad   0xa301dac75a8c7318+        .quad   0xed90039db3ceaa11+        .quad   0x6f077cbf3bae3f2d+        .quad   0x7518eaf8e052ad8e+        .quad   0xa48859c41b7f6c32+        .quad   0x0f2d60bcf4383298+        .quad   0x1815a929c9b1d1d9+        .quad   0x47c3871bbb1755c4++        // 2^120 * 4 * G++        .quad   0x5144539771ec4f48+        .quad   0xf805b17dc98c5d6e+        .quad   0xf762c11a47c3c66b+        .quad   0x00b89b85764699dc+        .quad   0xfbe65d50c85066b0+        .quad   0x62ecc4b0b3a299b0+        .quad   0xe53754ea441ae8e0+        .quad   0x08fea02ce8d48d5f+        .quad   0x824ddd7668deead0+        .quad   0xc86445204b685d23+        .quad   0xb514cfcd5d89d665+        .quad   0x473829a74f75d537++        // 2^120 * 5 * G++        .quad   0x82d2da754679c418+        .quad   0xe63bd7d8b2618df0+        .quad   0x355eef24ac47eb0a+        .quad   0x2078684c4833c6b4+        .quad   0x23d9533aad3902c9+        .quad   0x64c2ddceef03588f+        .quad   0x15257390cfe12fb4+        .quad   0x6c668b4d44e4d390+        .quad   0x3b48cf217a78820c+        .quad   0xf76a0ab281273e97+        .quad   0xa96c65a78c8eed7b+        .quad   0x7411a6054f8a433f++        // 2^120 * 6 * G++        .quad   0x4d659d32b99dc86d+        .quad   0x044cdc75603af115+        .quad   0xb34c712cdcc2e488+        .quad   0x7c136574fb8134ff+        .quad   0x579ae53d18b175b4+        .quad   0x68713159f392a102+        .quad   0x8455ecba1eef35f5+        .quad   0x1ec9a872458c398f+        .quad   0xb8e6a4d400a2509b+        .quad   0x9b81d7020bc882b4+        .quad   0x57e7cc9bf1957561+        .quad   0x3add88a5c7cd6460++        // 2^120 * 7 * G++        .quad   0xab895770b635dcf2+        .quad   0x02dfef6cf66c1fbc+        .quad   0x85530268beb6d187+        .quad   0x249929fccc879e74+        .quad   0x85c298d459393046+        .quad   0x8f7e35985ff659ec+        .quad   0x1d2ca22af2f66e3a+        .quad   0x61ba1131a406a720+        .quad   0xa3d0a0f116959029+        .quad   0x023b6b6cba7ebd89+        .quad   0x7bf15a3e26783307+        .quad   0x5620310cbbd8ece7++        // 2^120 * 8 * G++        .quad   0x528993434934d643+        .quad   0xb9dbf806a51222f5+        .quad   0x8f6d878fc3f41c22+        .quad   0x37676a2a4d9d9730+        .quad   0x6646b5f477e285d6+        .quad   0x40e8ff676c8f6193+        .quad   0xa6ec7311abb594dd+        .quad   0x7ec846f3658cec4d+        .quad   0x9b5e8f3f1da22ec7+        .quad   0x130f1d776c01cd13+        .quad   0x214c8fcfa2989fb8+        .quad   0x6daaf723399b9dd5++        // 2^124 * 1 * G++        .quad   0x591e4a5610628564+        .quad   0x2a4bb87ca8b4df34+        .quad   0xde2a2572e7a38e43+        .quad   0x3cbdabd9fee5046e+        .quad   0x81aebbdd2cd13070+        .quad   0x962e4325f85a0e9e+        .quad   0xde9391aacadffecb+        .quad   0x53177fda52c230e6+        .quad   0xa7bc970650b9de79+        .quad   0x3d12a7fbc301b59b+        .quad   0x02652e68d36ae38c+        .quad   0x79d739835a6199dc++        // 2^124 * 2 * G++        .quad   0xd9354df64131c1bd+        .quad   0x758094a186ec5822+        .quad   0x4464ee12e459f3c2+        .quad   0x6c11fce4cb133282+        .quad   0x21c9d9920d591737+        .quad   0x9bea41d2e9b46cd6+        .quad   0xe20e84200d89bfca+        .quad   0x79d99f946eae5ff8+        .quad   0xf17b483568673205+        .quad   0x387deae83caad96c+        .quad   0x61b471fd56ffe386+        .quad   0x31741195b745a599++        // 2^124 * 3 * G++        .quad   0xe8d10190b77a360b+        .quad   0x99b983209995e702+        .quad   0xbd4fdff8fa0247aa+        .quad   0x2772e344e0d36a87+        .quad   0x17f8ba683b02a047+        .quad   0x50212096feefb6c8+        .quad   0x70139be21556cbe2+        .quad   0x203e44a11d98915b+        .quad   0xd6863eba37b9e39f+        .quad   0x105bc169723b5a23+        .quad   0x104f6459a65c0762+        .quad   0x567951295b4d38d4++        // 2^124 * 4 * G++        .quad   0x535fd60613037524+        .quad   0xe210adf6b0fbc26a+        .quad   0xac8d0a9b23e990ae+        .quad   0x47204d08d72fdbf9+        .quad   0x07242eb30d4b497f+        .quad   0x1ef96306b9bccc87+        .quad   0x37950934d8116f45+        .quad   0x05468d6201405b04+        .quad   0x00f565a9f93267de+        .quad   0xcecfd78dc0d58e8a+        .quad   0xa215e2dcf318e28e+        .quad   0x4599ee919b633352++        // 2^124 * 5 * G++        .quad   0xd3c220ca70e0e76b+        .quad   0xb12bea58ea9f3094+        .quad   0x294ddec8c3271282+        .quad   0x0c3539e1a1d1d028+        .quad   0xac746d6b861ae579+        .quad   0x31ab0650f6aea9dc+        .quad   0x241d661140256d4c+        .quad   0x2f485e853d21a5de+        .quad   0x329744839c0833f3+        .quad   0x6fe6257fd2abc484+        .quad   0x5327d1814b358817+        .quad   0x65712585893fe9bc++        // 2^124 * 6 * G++        .quad   0x9c102fb732a61161+        .quad   0xe48e10dd34d520a8+        .quad   0x365c63546f9a9176+        .quad   0x32f6fe4c046f6006+        .quad   0x81c29f1bd708ee3f+        .quad   0xddcb5a05ae6407d0+        .quad   0x97aec1d7d2a3eba7+        .quad   0x1590521a91d50831+        .quad   0x40a3a11ec7910acc+        .quad   0x9013dff8f16d27ae+        .quad   0x1a9720d8abb195d4+        .quad   0x1bb9fe452ea98463++        // 2^124 * 7 * G++        .quad   0xe9d1d950b3d54f9e+        .quad   0x2d5f9cbee00d33c1+        .quad   0x51c2c656a04fc6ac+        .quad   0x65c091ee3c1cbcc9+        .quad   0xcf5e6c95cc36747c+        .quad   0x294201536b0bc30d+        .quad   0x453ac67cee797af0+        .quad   0x5eae6ab32a8bb3c9+        .quad   0x7083661114f118ea+        .quad   0x2b37b87b94349cad+        .quad   0x7273f51cb4e99f40+        .quad   0x78a2a95823d75698++        // 2^124 * 8 * G++        .quad   0xa2b072e95c8c2ace+        .quad   0x69cffc96651e9c4b+        .quad   0x44328ef842e7b42b+        .quad   0x5dd996c122aadeb3+        .quad   0xb4f23c425ef83207+        .quad   0xabf894d3c9a934b5+        .quad   0xd0708c1339fd87f7+        .quad   0x1876789117166130+        .quad   0x925b5ef0670c507c+        .quad   0x819bc842b93c33bf+        .quad   0x10792e9a70dd003f+        .quad   0x59ad4b7a6e28dc74++        // 2^128 * 1 * G++        .quad   0x5f3a7562eb3dbe47+        .quad   0xf7ea38548ebda0b8+        .quad   0x00c3e53145747299+        .quad   0x1304e9e71627d551+        .quad   0x583b04bfacad8ea2+        .quad   0x29b743e8148be884+        .quad   0x2b1e583b0810c5db+        .quad   0x2b5449e58eb3bbaa+        .quad   0x789814d26adc9cfe+        .quad   0x3c1bab3f8b48dd0b+        .quad   0xda0fe1fff979c60a+        .quad   0x4468de2d7c2dd693++        // 2^128 * 2 * G++        .quad   0x51bb355e9419469e+        .quad   0x33e6dc4c23ddc754+        .quad   0x93a5b6d6447f9962+        .quad   0x6cce7c6ffb44bd63+        .quad   0x4b9ad8c6f86307ce+        .quad   0x21113531435d0c28+        .quad   0xd4a866c5657a772c+        .quad   0x5da6427e63247352+        .quad   0x1a94c688deac22ca+        .quad   0xb9066ef7bbae1ff8+        .quad   0x88ad8c388d59580f+        .quad   0x58f29abfe79f2ca8++        // 2^128 * 3 * G++        .quad   0xe90ecfab8de73e68+        .quad   0x54036f9f377e76a5+        .quad   0xf0495b0bbe015982+        .quad   0x577629c4a7f41e36+        .quad   0x4b5a64bf710ecdf6+        .quad   0xb14ce538462c293c+        .quad   0x3643d056d50b3ab9+        .quad   0x6af93724185b4870+        .quad   0x3220024509c6a888+        .quad   0xd2e036134b558973+        .quad   0x83e236233c33289f+        .quad   0x701f25bb0caec18f++        // 2^128 * 4 * G++        .quad   0xc3a8b0f8e4616ced+        .quad   0xf700660e9e25a87d+        .quad   0x61e3061ff4bca59c+        .quad   0x2e0c92bfbdc40be9+        .quad   0x9d18f6d97cbec113+        .quad   0x844a06e674bfdbe4+        .quad   0x20f5b522ac4e60d6+        .quad   0x720a5bc050955e51+        .quad   0x0c3f09439b805a35+        .quad   0xe84e8b376242abfc+        .quad   0x691417f35c229346+        .quad   0x0e9b9cbb144ef0ec++        // 2^128 * 5 * G++        .quad   0xfbbad48ffb5720ad+        .quad   0xee81916bdbf90d0e+        .quad   0xd4813152635543bf+        .quad   0x221104eb3f337bd8+        .quad   0x8dee9bd55db1beee+        .quad   0xc9c3ab370a723fb9+        .quad   0x44a8f1bf1c68d791+        .quad   0x366d44191cfd3cde+        .quad   0x9e3c1743f2bc8c14+        .quad   0x2eda26fcb5856c3b+        .quad   0xccb82f0e68a7fb97+        .quad   0x4167a4e6bc593244++        // 2^128 * 6 * G++        .quad   0x643b9d2876f62700+        .quad   0x5d1d9d400e7668eb+        .quad   0x1b4b430321fc0684+        .quad   0x7938bb7e2255246a+        .quad   0xc2be2665f8ce8fee+        .quad   0xe967ff14e880d62c+        .quad   0xf12e6e7e2f364eee+        .quad   0x34b33370cb7ed2f6+        .quad   0xcdc591ee8681d6cc+        .quad   0xce02109ced85a753+        .quad   0xed7485c158808883+        .quad   0x1176fc6e2dfe65e4++        // 2^128 * 7 * G++        .quad   0xb4af6cd05b9c619b+        .quad   0x2ddfc9f4b2a58480+        .quad   0x3d4fa502ebe94dc4+        .quad   0x08fc3a4c677d5f34+        .quad   0xdb90e28949770eb8+        .quad   0x98fbcc2aacf440a3+        .quad   0x21354ffeded7879b+        .quad   0x1f6a3e54f26906b6+        .quad   0x60a4c199d30734ea+        .quad   0x40c085b631165cd6+        .quad   0xe2333e23f7598295+        .quad   0x4f2fad0116b900d1++        // 2^128 * 8 * G++        .quad   0x44beb24194ae4e54+        .quad   0x5f541c511857ef6c+        .quad   0xa61e6b2d368d0498+        .quad   0x445484a4972ef7ab+        .quad   0x962cd91db73bb638+        .quad   0xe60577aafc129c08+        .quad   0x6f619b39f3b61689+        .quad   0x3451995f2944ee81+        .quad   0x9152fcd09fea7d7c+        .quad   0x4a816c94b0935cf6+        .quad   0x258e9aaa47285c40+        .quad   0x10b89ca6042893b7++        // 2^132 * 1 * G++        .quad   0x9b2a426e3b646025+        .quad   0x32127190385ce4cf+        .quad   0xa25cffc2dd6dea45+        .quad   0x06409010bea8de75+        .quad   0xd67cded679d34aa0+        .quad   0xcc0b9ec0cc4db39f+        .quad   0xa535a456e35d190f+        .quad   0x2e05d9eaf61f6fef+        .quad   0xc447901ad61beb59+        .quad   0x661f19bce5dc880a+        .quad   0x24685482b7ca6827+        .quad   0x293c778cefe07f26++        // 2^132 * 2 * G++        .quad   0x86809e7007069096+        .quad   0xaad75b15e4e50189+        .quad   0x07f35715a21a0147+        .quad   0x0487f3f112815d5e+        .quad   0x16c795d6a11ff200+        .quad   0xcb70d0e2b15815c9+        .quad   0x89f293209b5395b5+        .quad   0x50b8c2d031e47b4f+        .quad   0x48350c08068a4962+        .quad   0x6ffdd05351092c9a+        .quad   0x17af4f4aaf6fc8dd+        .quad   0x4b0553b53cdba58b++        // 2^132 * 3 * G++        .quad   0x9c65fcbe1b32ff79+        .quad   0xeb75ea9f03b50f9b+        .quad   0xfced2a6c6c07e606+        .quad   0x35106cd551717908+        .quad   0xbf05211b27c152d4+        .quad   0x5ec26849bd1af639+        .quad   0x5e0b2caa8e6fab98+        .quad   0x054c8bdd50bd0840+        .quad   0x38a0b12f1dcf073d+        .quad   0x4b60a8a3b7f6a276+        .quad   0xfed5ac25d3404f9a+        .quad   0x72e82d5e5505c229++        // 2^132 * 4 * G++        .quad   0x6b0b697ff0d844c8+        .quad   0xbb12f85cd979cb49+        .quad   0xd2a541c6c1da0f1f+        .quad   0x7b7c242958ce7211+        .quad   0x00d9cdfd69771d02+        .quad   0x410276cd6cfbf17e+        .quad   0x4c45306c1cb12ec7+        .quad   0x2857bf1627500861+        .quad   0x9f21903f0101689e+        .quad   0xd779dfd3bf861005+        .quad   0xa122ee5f3deb0f1b+        .quad   0x510df84b485a00d4++        // 2^132 * 5 * G++        .quad   0xa54133bb9277a1fa+        .quad   0x74ec3b6263991237+        .quad   0x1a3c54dc35d2f15a+        .quad   0x2d347144e482ba3a+        .quad   0x24b3c887c70ac15e+        .quad   0xb0f3a557fb81b732+        .quad   0x9b2cde2fe578cc1b+        .quad   0x4cf7ed0703b54f8e+        .quad   0x6bd47c6598fbee0f+        .quad   0x9e4733e2ab55be2d+        .quad   0x1093f624127610c5+        .quad   0x4e05e26ad0a1eaa4++        // 2^132 * 6 * G++        .quad   0xda9b6b624b531f20+        .quad   0x429a760e77509abb+        .quad   0xdbe9f522e823cb80+        .quad   0x618f1856880c8f82+        .quad   0x1833c773e18fe6c0+        .quad   0xe3c4711ad3c87265+        .quad   0x3bfd3c4f0116b283+        .quad   0x1955875eb4cd4db8+        .quad   0x6da6de8f0e399799+        .quad   0x7ad61aa440fda178+        .quad   0xb32cd8105e3563dd+        .quad   0x15f6beae2ae340ae++        // 2^132 * 7 * G++        .quad   0x862bcb0c31ec3a62+        .quad   0x810e2b451138f3c2+        .quad   0x788ec4b839dac2a4+        .quad   0x28f76867ae2a9281+        .quad   0xba9a0f7b9245e215+        .quad   0xf368612dd98c0dbb+        .quad   0x2e84e4cbf220b020+        .quad   0x6ba92fe962d90eda+        .quad   0x3e4df9655884e2aa+        .quad   0xbd62fbdbdbd465a5+        .quad   0xd7596caa0de9e524+        .quad   0x6e8042ccb2b1b3d7++        // 2^132 * 8 * G++        .quad   0xf10d3c29ce28ca6e+        .quad   0xbad34540fcb6093d+        .quad   0xe7426ed7a2ea2d3f+        .quad   0x08af9d4e4ff298b9+        .quad   0x1530653616521f7e+        .quad   0x660d06b896203dba+        .quad   0x2d3989bc545f0879+        .quad   0x4b5303af78ebd7b0+        .quad   0x72f8a6c3bebcbde8+        .quad   0x4f0fca4adc3a8e89+        .quad   0x6fa9d4e8c7bfdf7a+        .quad   0x0dcf2d679b624eb7++        // 2^136 * 1 * G++        .quad   0x3d5947499718289c+        .quad   0x12ebf8c524533f26+        .quad   0x0262bfcb14c3ef15+        .quad   0x20b878d577b7518e+        .quad   0x753941be5a45f06e+        .quad   0xd07caeed6d9c5f65+        .quad   0x11776b9c72ff51b6+        .quad   0x17d2d1d9ef0d4da9+        .quad   0x27f2af18073f3e6a+        .quad   0xfd3fe519d7521069+        .quad   0x22e3b72c3ca60022+        .quad   0x72214f63cc65c6a7++        // 2^136 * 2 * G++        .quad   0xb4e37f405307a693+        .quad   0xaba714d72f336795+        .quad   0xd6fbd0a773761099+        .quad   0x5fdf48c58171cbc9+        .quad   0x1d9db7b9f43b29c9+        .quad   0xd605824a4f518f75+        .quad   0xf2c072bd312f9dc4+        .quad   0x1f24ac855a1545b0+        .quad   0x24d608328e9505aa+        .quad   0x4748c1d10c1420ee+        .quad   0xc7ffe45c06fb25a2+        .quad   0x00ba739e2ae395e6++        // 2^136 * 3 * G++        .quad   0x592e98de5c8790d6+        .quad   0xe5bfb7d345c2a2df+        .quad   0x115a3b60f9b49922+        .quad   0x03283a3e67ad78f3+        .quad   0xae4426f5ea88bb26+        .quad   0x360679d984973bfb+        .quad   0x5c9f030c26694e50+        .quad   0x72297de7d518d226+        .quad   0x48241dc7be0cb939+        .quad   0x32f19b4d8b633080+        .quad   0xd3dfc90d02289308+        .quad   0x05e1296846271945++        // 2^136 * 4 * G++        .quad   0xba82eeb32d9c495a+        .quad   0xceefc8fcf12bb97c+        .quad   0xb02dabae93b5d1e0+        .quad   0x39c00c9c13698d9b+        .quad   0xadbfbbc8242c4550+        .quad   0xbcc80cecd03081d9+        .quad   0x843566a6f5c8df92+        .quad   0x78cf25d38258ce4c+        .quad   0x15ae6b8e31489d68+        .quad   0xaa851cab9c2bf087+        .quad   0xc9a75a97f04efa05+        .quad   0x006b52076b3ff832++        // 2^136 * 5 * G++        .quad   0x29e0cfe19d95781c+        .quad   0xb681df18966310e2+        .quad   0x57df39d370516b39+        .quad   0x4d57e3443bc76122+        .quad   0xf5cb7e16b9ce082d+        .quad   0x3407f14c417abc29+        .quad   0xd4b36bce2bf4a7ab+        .quad   0x7de2e9561a9f75ce+        .quad   0xde70d4f4b6a55ecb+        .quad   0x4801527f5d85db99+        .quad   0xdbc9c440d3ee9a81+        .quad   0x6b2a90af1a6029ed++        // 2^136 * 6 * G++        .quad   0x6923f4fc9ae61e97+        .quad   0x5735281de03f5fd1+        .quad   0xa764ae43e6edd12d+        .quad   0x5fd8f4e9d12d3e4a+        .quad   0x77ebf3245bb2d80a+        .quad   0xd8301b472fb9079b+        .quad   0xc647e6f24cee7333+        .quad   0x465812c8276c2109+        .quad   0x4d43beb22a1062d9+        .quad   0x7065fb753831dc16+        .quad   0x180d4a7bde2968d7+        .quad   0x05b32c2b1cb16790++        // 2^136 * 7 * G++        .quad   0xc8c05eccd24da8fd+        .quad   0xa1cf1aac05dfef83+        .quad   0xdbbeeff27df9cd61+        .quad   0x3b5556a37b471e99+        .quad   0xf7fca42c7ad58195+        .quad   0x3214286e4333f3cc+        .quad   0xb6c29d0d340b979d+        .quad   0x31771a48567307e1+        .quad   0x32b0c524e14dd482+        .quad   0xedb351541a2ba4b6+        .quad   0xa3d16048282b5af3+        .quad   0x4fc079d27a7336eb++        // 2^136 * 8 * G++        .quad   0x51c938b089bf2f7f+        .quad   0x2497bd6502dfe9a7+        .quad   0xffffc09c7880e453+        .quad   0x124567cecaf98e92+        .quad   0xdc348b440c86c50d+        .quad   0x1337cbc9cc94e651+        .quad   0x6422f74d643e3cb9+        .quad   0x241170c2bae3cd08+        .quad   0x3ff9ab860ac473b4+        .quad   0xf0911dee0113e435+        .quad   0x4ae75060ebc6c4af+        .quad   0x3f8612966c87000d++        // 2^140 * 1 * G++        .quad   0x0c9c5303f7957be4+        .quad   0xa3c31a20e085c145+        .quad   0xb0721d71d0850050+        .quad   0x0aba390eab0bf2da+        .quad   0x529fdffe638c7bf3+        .quad   0xdf2b9e60388b4995+        .quad   0xe027b34f1bad0249+        .quad   0x7bc92fc9b9fa74ed+        .quad   0x9f97ef2e801ad9f9+        .quad   0x83697d5479afda3a+        .quad   0xe906b3ffbd596b50+        .quad   0x02672b37dd3fb8e0++        // 2^140 * 2 * G++        .quad   0x48b2ca8b260885e4+        .quad   0xa4286bec82b34c1c+        .quad   0x937e1a2617f58f74+        .quad   0x741d1fcbab2ca2a5+        .quad   0xee9ba729398ca7f5+        .quad   0xeb9ca6257a4849db+        .quad   0x29eb29ce7ec544e1+        .quad   0x232ca21ef736e2c8+        .quad   0xbf61423d253fcb17+        .quad   0x08803ceafa39eb14+        .quad   0xf18602df9851c7af+        .quad   0x0400f3a049e3414b++        // 2^140 * 3 * G++        .quad   0xabce0476ba61c55b+        .quad   0x36a3d6d7c4d39716+        .quad   0x6eb259d5e8d82d09+        .quad   0x0c9176e984d756fb+        .quad   0x2efba412a06e7b06+        .quad   0x146785452c8d2560+        .quad   0xdf9713ebd67a91c7+        .quad   0x32830ac7157eadf3+        .quad   0x0e782a7ab73769e8+        .quad   0x04a05d7875b18e2c+        .quad   0x29525226ebcceae1+        .quad   0x0d794f8383eba820++        // 2^140 * 4 * G++        .quad   0xff35f5cb9e1516f4+        .quad   0xee805bcf648aae45+        .quad   0xf0d73c2bb93a9ef3+        .quad   0x097b0bf22092a6c2+        .quad   0x7be44ce7a7a2e1ac+        .quad   0x411fd93efad1b8b7+        .quad   0x1734a1d70d5f7c9b+        .quad   0x0d6592233127db16+        .quad   0xc48bab1521a9d733+        .quad   0xa6c2eaead61abb25+        .quad   0x625c6c1cc6cb4305+        .quad   0x7fc90fea93eb3a67++        // 2^140 * 5 * G++        .quad   0x0408f1fe1f5c5926+        .quad   0x1a8f2f5e3b258bf4+        .quad   0x40a951a2fdc71669+        .quad   0x6598ee93c98b577e+        .quad   0xc527deb59c7cb23d+        .quad   0x955391695328404e+        .quad   0xd64392817ccf2c7a+        .quad   0x6ce97dabf7d8fa11+        .quad   0x25b5a8e50ef7c48f+        .quad   0xeb6034116f2ce532+        .quad   0xc5e75173e53de537+        .quad   0x73119fa08c12bb03++        // 2^140 * 6 * G++        .quad   0xed30129453f1a4cb+        .quad   0xbce621c9c8f53787+        .quad   0xfacb2b1338bee7b9+        .quad   0x3025798a9ea8428c+        .quad   0x7845b94d21f4774d+        .quad   0xbf62f16c7897b727+        .quad   0x671857c03c56522b+        .quad   0x3cd6a85295621212+        .quad   0x3fecde923aeca999+        .quad   0xbdaa5b0062e8c12f+        .quad   0x67b99dfc96988ade+        .quad   0x3f52c02852661036++        // 2^140 * 7 * G++        .quad   0xffeaa48e2a1351c6+        .quad   0x28624754fa7f53d7+        .quad   0x0b5ba9e57582ddf1+        .quad   0x60c0104ba696ac59+        .quad   0x9258bf99eec416c6+        .quad   0xac8a5017a9d2f671+        .quad   0x629549ab16dea4ab+        .quad   0x05d0e85c99091569+        .quad   0x051de020de9cbe97+        .quad   0xfa07fc56b50bcf74+        .quad   0x378cec9f0f11df65+        .quad   0x36853c69ab96de4d++        // 2^140 * 8 * G++        .quad   0x36d9b8de78f39b2d+        .quad   0x7f42ed71a847b9ec+        .quad   0x241cd1d679bd3fde+        .quad   0x6a704fec92fbce6b+        .quad   0x4433c0b0fac5e7be+        .quad   0x724bae854c08dcbe+        .quad   0xf1f24cc446978f9b+        .quad   0x4a0aff6d62825fc8+        .quad   0xe917fb9e61095301+        .quad   0xc102df9402a092f8+        .quad   0xbf09e2f5fa66190b+        .quad   0x681109bee0dcfe37++        // 2^144 * 1 * G++        .quad   0x559a0cc9782a0dde+        .quad   0x551dcdb2ea718385+        .quad   0x7f62865b31ef238c+        .quad   0x504aa7767973613d+        .quad   0x9c18fcfa36048d13+        .quad   0x29159db373899ddd+        .quad   0xdc9f350b9f92d0aa+        .quad   0x26f57eee878a19d4+        .quad   0x0cab2cd55687efb1+        .quad   0x5180d162247af17b+        .quad   0x85c15a344f5a2467+        .quad   0x4041943d9dba3069++        // 2^144 * 2 * G++        .quad   0xc3c0eeba43ebcc96+        .quad   0x8d749c9c26ea9caf+        .quad   0xd9fa95ee1c77ccc6+        .quad   0x1420a1d97684340f+        .quad   0x4b217743a26caadd+        .quad   0x47a6b424648ab7ce+        .quad   0xcb1d4f7a03fbc9e3+        .quad   0x12d931429800d019+        .quad   0x00c67799d337594f+        .quad   0x5e3c5140b23aa47b+        .quad   0x44182854e35ff395+        .quad   0x1b4f92314359a012++        // 2^144 * 3 * G++        .quad   0x3e5c109d89150951+        .quad   0x39cefa912de9696a+        .quad   0x20eae43f975f3020+        .quad   0x239b572a7f132dae+        .quad   0x33cf3030a49866b1+        .quad   0x251f73d2215f4859+        .quad   0xab82aa4051def4f6+        .quad   0x5ff191d56f9a23f6+        .quad   0x819ed433ac2d9068+        .quad   0x2883ab795fc98523+        .quad   0xef4572805593eb3d+        .quad   0x020c526a758f36cb++        // 2^144 * 4 * G++        .quad   0x779834f89ed8dbbc+        .quad   0xc8f2aaf9dc7ca46c+        .quad   0xa9524cdca3e1b074+        .quad   0x02aacc4615313877+        .quad   0xe931ef59f042cc89+        .quad   0x2c589c9d8e124bb6+        .quad   0xadc8e18aaec75997+        .quad   0x452cfe0a5602c50c+        .quad   0x86a0f7a0647877df+        .quad   0xbbc464270e607c9f+        .quad   0xab17ea25f1fb11c9+        .quad   0x4cfb7d7b304b877b++        // 2^144 * 5 * G++        .quad   0x72b43d6cb89b75fe+        .quad   0x54c694d99c6adc80+        .quad   0xb8c3aa373ee34c9f+        .quad   0x14b4622b39075364+        .quad   0xe28699c29789ef12+        .quad   0x2b6ecd71df57190d+        .quad   0xc343c857ecc970d0+        .quad   0x5b1d4cbc434d3ac5+        .quad   0xb6fb2615cc0a9f26+        .quad   0x3a4f0e2bb88dcce5+        .quad   0x1301498b3369a705+        .quad   0x2f98f71258592dd1++        // 2^144 * 6 * G++        .quad   0x0c94a74cb50f9e56+        .quad   0x5b1ff4a98e8e1320+        .quad   0x9a2acc2182300f67+        .quad   0x3a6ae249d806aaf9+        .quad   0x2e12ae444f54a701+        .quad   0xfcfe3ef0a9cbd7de+        .quad   0xcebf890d75835de0+        .quad   0x1d8062e9e7614554+        .quad   0x657ada85a9907c5a+        .quad   0x1a0ea8b591b90f62+        .quad   0x8d0e1dfbdf34b4e9+        .quad   0x298b8ce8aef25ff3++        // 2^144 * 7 * G++        .quad   0x2a927953eff70cb2+        .quad   0x4b89c92a79157076+        .quad   0x9418457a30a7cf6a+        .quad   0x34b8a8404d5ce485+        .quad   0x837a72ea0a2165de+        .quad   0x3fab07b40bcf79f6+        .quad   0x521636c77738ae70+        .quad   0x6ba6271803a7d7dc+        .quad   0xc26eecb583693335+        .quad   0xd5a813df63b5fefd+        .quad   0xa293aa9aa4b22573+        .quad   0x71d62bdd465e1c6a++        // 2^144 * 8 * G++        .quad   0x6533cc28d378df80+        .quad   0xf6db43790a0fa4b4+        .quad   0xe3645ff9f701da5a+        .quad   0x74d5f317f3172ba4+        .quad   0xcd2db5dab1f75ef5+        .quad   0xd77f95cf16b065f5+        .quad   0x14571fea3f49f085+        .quad   0x1c333621262b2b3d+        .quad   0xa86fe55467d9ca81+        .quad   0x398b7c752b298c37+        .quad   0xda6d0892e3ac623b+        .quad   0x4aebcc4547e9d98c++        // 2^148 * 1 * G++        .quad   0x53175a7205d21a77+        .quad   0xb0c04422d3b934d4+        .quad   0xadd9f24bdd5deadc+        .quad   0x074f46e69f10ff8c+        .quad   0x0de9b204a059a445+        .quad   0xe15cb4aa4b17ad0f+        .quad   0xe1bbec521f79c557+        .quad   0x2633f1b9d071081b+        .quad   0xc1fb4177018b9910+        .quad   0xa6ea20dc6c0fe140+        .quad   0xd661f3e74354c6ff+        .quad   0x5ecb72e6f1a3407a++        // 2^148 * 2 * G++        .quad   0xa515a31b2259fb4e+        .quad   0x0960f3972bcac52f+        .quad   0xedb52fec8d3454cb+        .quad   0x382e2720c476c019+        .quad   0xfeeae106e8e86997+        .quad   0x9863337f98d09383+        .quad   0x9470480eaa06ebef+        .quad   0x038b6898d4c5c2d0+        .quad   0xf391c51d8ace50a6+        .quad   0x3142d0b9ae2d2948+        .quad   0xdb4d5a1a7f24ca80+        .quad   0x21aeba8b59250ea8++        // 2^148 * 3 * G++        .quad   0x24f13b34cf405530+        .quad   0x3c44ea4a43088af7+        .quad   0x5dd5c5170006a482+        .quad   0x118eb8f8890b086d+        .quad   0x53853600f0087f23+        .quad   0x4c461879da7d5784+        .quad   0x6af303deb41f6860+        .quad   0x0a3c16c5c27c18ed+        .quad   0x17e49c17cc947f3d+        .quad   0xccc6eda6aac1d27b+        .quad   0xdf6092ceb0f08e56+        .quad   0x4909b3e22c67c36b++        // 2^148 * 4 * G++        .quad   0x9c9c85ea63fe2e89+        .quad   0xbe1baf910e9412ec+        .quad   0x8f7baa8a86fbfe7b+        .quad   0x0fb17f9fef968b6c+        .quad   0x59a16676706ff64e+        .quad   0x10b953dd0d86a53d+        .quad   0x5848e1e6ce5c0b96+        .quad   0x2d8b78e712780c68+        .quad   0x79d5c62eafc3902b+        .quad   0x773a215289e80728+        .quad   0xc38ae640e10120b9+        .quad   0x09ae23717b2b1a6d++        // 2^148 * 5 * G++        .quad   0xbb6a192a4e4d083c+        .quad   0x34ace0630029e192+        .quad   0x98245a59aafabaeb+        .quad   0x6d9c8a9ada97faac+        .quad   0x10ab8fa1ad32b1d0+        .quad   0xe9aced1be2778b24+        .quad   0xa8856bc0373de90f+        .quad   0x66f35ddddda53996+        .quad   0xd27d9afb24997323+        .quad   0x1bb7e07ef6f01d2e+        .quad   0x2ba7472df52ecc7f+        .quad   0x03019b4f646f9dc8++        // 2^148 * 6 * G++        .quad   0x04a186b5565345cd+        .quad   0xeee76610bcc4116a+        .quad   0x689c73b478fb2a45+        .quad   0x387dcbff65697512+        .quad   0xaf09b214e6b3dc6b+        .quad   0x3f7573b5ad7d2f65+        .quad   0xd019d988100a23b0+        .quad   0x392b63a58b5c35f7+        .quad   0x4093addc9c07c205+        .quad   0xc565be15f532c37e+        .quad   0x63dbecfd1583402a+        .quad   0x61722b4aef2e032e++        // 2^148 * 7 * G++        .quad   0x0012aafeecbd47af+        .quad   0x55a266fb1cd46309+        .quad   0xf203eb680967c72c+        .quad   0x39633944ca3c1429+        .quad   0xd6b07a5581cb0e3c+        .quad   0x290ff006d9444969+        .quad   0x08680b6a16dcda1f+        .quad   0x5568d2b75a06de59+        .quad   0x8d0cb88c1b37cfe1+        .quad   0x05b6a5a3053818f3+        .quad   0xf2e9bc04b787d959+        .quad   0x6beba1249add7f64++        // 2^148 * 8 * G++        .quad   0x1d06005ca5b1b143+        .quad   0x6d4c6bb87fd1cda2+        .quad   0x6ef5967653fcffe7+        .quad   0x097c29e8c1ce1ea5+        .quad   0x5c3cecb943f5a53b+        .quad   0x9cc9a61d06c08df2+        .quad   0xcfba639a85895447+        .quad   0x5a845ae80df09fd5+        .quad   0x4ce97dbe5deb94ca+        .quad   0x38d0a4388c709c48+        .quad   0xc43eced4a169d097+        .quad   0x0a1249fff7e587c3++        // 2^152 * 1 * G++        .quad   0x12f0071b276d01c9+        .quad   0xe7b8bac586c48c70+        .quad   0x5308129b71d6fba9+        .quad   0x5d88fbf95a3db792+        .quad   0x0b408d9e7354b610+        .quad   0x806b32535ba85b6e+        .quad   0xdbe63a034a58a207+        .quad   0x173bd9ddc9a1df2c+        .quad   0x2b500f1efe5872df+        .quad   0x58d6582ed43918c1+        .quad   0xe6ed278ec9673ae0+        .quad   0x06e1cd13b19ea319++        // 2^152 * 2 * G++        .quad   0x40d0ad516f166f23+        .quad   0x118e32931fab6abe+        .quad   0x3fe35e14a04d088e+        .quad   0x3080603526e16266+        .quad   0x472baf629e5b0353+        .quad   0x3baa0b90278d0447+        .quad   0x0c785f469643bf27+        .quad   0x7f3a6a1a8d837b13+        .quad   0xf7e644395d3d800b+        .quad   0x95a8d555c901edf6+        .quad   0x68cd7830592c6339+        .quad   0x30d0fded2e51307e++        // 2^152 * 3 * G++        .quad   0xe0594d1af21233b3+        .quad   0x1bdbe78ef0cc4d9c+        .quad   0x6965187f8f499a77+        .quad   0x0a9214202c099868+        .quad   0x9cb4971e68b84750+        .quad   0xa09572296664bbcf+        .quad   0x5c8de72672fa412b+        .quad   0x4615084351c589d9+        .quad   0xbc9019c0aeb9a02e+        .quad   0x55c7110d16034cae+        .quad   0x0e6df501659932ec+        .quad   0x3bca0d2895ca5dfe++        // 2^152 * 4 * G++        .quad   0x40f031bc3c5d62a4+        .quad   0x19fc8b3ecff07a60+        .quad   0x98183da2130fb545+        .quad   0x5631deddae8f13cd+        .quad   0x9c688eb69ecc01bf+        .quad   0xf0bc83ada644896f+        .quad   0xca2d955f5f7a9fe2+        .quad   0x4ea8b4038df28241+        .quad   0x2aed460af1cad202+        .quad   0x46305305a48cee83+        .quad   0x9121774549f11a5f+        .quad   0x24ce0930542ca463++        // 2^152 * 5 * G++        .quad   0x1fe890f5fd06c106+        .quad   0xb5c468355d8810f2+        .quad   0x827808fe6e8caf3e+        .quad   0x41d4e3c28a06d74b+        .quad   0x3fcfa155fdf30b85+        .quad   0xd2f7168e36372ea4+        .quad   0xb2e064de6492f844+        .quad   0x549928a7324f4280+        .quad   0xf26e32a763ee1a2e+        .quad   0xae91e4b7d25ffdea+        .quad   0xbc3bd33bd17f4d69+        .quad   0x491b66dec0dcff6a++        // 2^152 * 6 * G++        .quad   0x98f5b13dc7ea32a7+        .quad   0xe3d5f8cc7e16db98+        .quad   0xac0abf52cbf8d947+        .quad   0x08f338d0c85ee4ac+        .quad   0x75f04a8ed0da64a1+        .quad   0xed222caf67e2284b+        .quad   0x8234a3791f7b7ba4+        .quad   0x4cf6b8b0b7018b67+        .quad   0xc383a821991a73bd+        .quad   0xab27bc01df320c7a+        .quad   0xc13d331b84777063+        .quad   0x530d4a82eb078a99++        // 2^152 * 7 * G++        .quad   0x004c3630e1f94825+        .quad   0x7e2d78268cab535a+        .quad   0xc7482323cc84ff8b+        .quad   0x65ea753f101770b9+        .quad   0x6d6973456c9abf9e+        .quad   0x257fb2fc4900a880+        .quad   0x2bacf412c8cfb850+        .quad   0x0db3e7e00cbfbd5b+        .quad   0x3d66fc3ee2096363+        .quad   0x81d62c7f61b5cb6b+        .quad   0x0fbe044213443b1a+        .quad   0x02a4ec1921e1a1db++        // 2^152 * 8 * G++        .quad   0x5ce6259a3b24b8a2+        .quad   0xb8577acc45afa0b8+        .quad   0xcccbe6e88ba07037+        .quad   0x3d143c51127809bf+        .quad   0xf5c86162f1cf795f+        .quad   0x118c861926ee57f2+        .quad   0x172124851c063578+        .quad   0x36d12b5dec067fcf+        .quad   0x126d279179154557+        .quad   0xd5e48f5cfc783a0a+        .quad   0x36bdb6e8df179bac+        .quad   0x2ef517885ba82859++        // 2^156 * 1 * G++        .quad   0x88bd438cd11e0d4a+        .quad   0x30cb610d43ccf308+        .quad   0xe09a0e3791937bcc+        .quad   0x4559135b25b1720c+        .quad   0x1ea436837c6da1e9+        .quad   0xf9c189af1fb9bdbe+        .quad   0x303001fcce5dd155+        .quad   0x28a7c99ebc57be52+        .quad   0xb8fd9399e8d19e9d+        .quad   0x908191cb962423ff+        .quad   0xb2b948d747c742a3+        .quad   0x37f33226d7fb44c4++        // 2^156 * 2 * G++        .quad   0x0dae8767b55f6e08+        .quad   0x4a43b3b35b203a02+        .quad   0xe3725a6e80af8c79+        .quad   0x0f7a7fd1705fa7a3+        .quad   0x33912553c821b11d+        .quad   0x66ed42c241e301df+        .quad   0x066fcc11104222fd+        .quad   0x307a3b41c192168f+        .quad   0x8eeb5d076eb55ce0+        .quad   0x2fc536bfaa0d925a+        .quad   0xbe81830fdcb6c6e8+        .quad   0x556c7045827baf52++        // 2^156 * 3 * G++        .quad   0x8e2b517302e9d8b7+        .quad   0xe3e52269248714e8+        .quad   0xbd4fbd774ca960b5+        .quad   0x6f4b4199c5ecada9+        .quad   0xb94b90022bf44406+        .quad   0xabd4237eff90b534+        .quad   0x7600a960faf86d3a+        .quad   0x2f45abdac2322ee3+        .quad   0x61af4912c8ef8a6a+        .quad   0xe58fa4fe43fb6e5e+        .quad   0xb5afcc5d6fd427cf+        .quad   0x6a5393281e1e11eb++        // 2^156 * 4 * G++        .quad   0xf3da5139a5d1ee89+        .quad   0x8145457cff936988+        .quad   0x3f622fed00e188c4+        .quad   0x0f513815db8b5a3d+        .quad   0x0fff04fe149443cf+        .quad   0x53cac6d9865cddd7+        .quad   0x31385b03531ed1b7+        .quad   0x5846a27cacd1039d+        .quad   0x4ff5cdac1eb08717+        .quad   0x67e8b29590f2e9bc+        .quad   0x44093b5e237afa99+        .quad   0x0d414bed8708b8b2++        // 2^156 * 5 * G++        .quad   0xcfb68265fd0e75f6+        .quad   0xe45b3e28bb90e707+        .quad   0x7242a8de9ff92c7a+        .quad   0x685b3201933202dd+        .quad   0x81886a92294ac9e8+        .quad   0x23162b45d55547be+        .quad   0x94cfbc4403715983+        .quad   0x50eb8fdb134bc401+        .quad   0xc0b73ec6d6b330cd+        .quad   0x84e44807132faff1+        .quad   0x732b7352c4a5dee1+        .quad   0x5d7c7cf1aa7cd2d2++        // 2^156 * 6 * G++        .quad   0xaf3b46bf7a4aafa2+        .quad   0xb78705ec4d40d411+        .quad   0x114f0c6aca7c15e3+        .quad   0x3f364faaa9489d4d+        .quad   0x33d1013e9b73a562+        .quad   0x925cef5748ec26e1+        .quad   0xa7fce614dd468058+        .quad   0x78b0fad41e9aa438+        .quad   0xbf56a431ed05b488+        .quad   0xa533e66c9c495c7e+        .quad   0xe8652baf87f3651a+        .quad   0x0241800059d66c33++        // 2^156 * 7 * G++        .quad   0xceb077fea37a5be4+        .quad   0xdb642f02e5a5eeb7+        .quad   0xc2e6d0c5471270b8+        .quad   0x4771b65538e4529c+        .quad   0x28350c7dcf38ea01+        .quad   0x7c6cdbc0b2917ab6+        .quad   0xace7cfbe857082f7+        .quad   0x4d2845aba2d9a1e0+        .quad   0xbb537fe0447070de+        .quad   0xcba744436dd557df+        .quad   0xd3b5a3473600dbcb+        .quad   0x4aeabbe6f9ffd7f8++        // 2^156 * 8 * G++        .quad   0x4630119e40d8f78c+        .quad   0xa01a9bc53c710e11+        .quad   0x486d2b258910dd79+        .quad   0x1e6c47b3db0324e5+        .quad   0x6a2134bcc4a9c8f2+        .quad   0xfbf8fd1c8ace2e37+        .quad   0x000ae3049911a0ba+        .quad   0x046e3a616bc89b9e+        .quad   0x14e65442f03906be+        .quad   0x4a019d54e362be2a+        .quad   0x68ccdfec8dc230c7+        .quad   0x7cfb7e3faf6b861c++        // 2^160 * 1 * G++        .quad   0x4637974e8c58aedc+        .quad   0xb9ef22fbabf041a4+        .quad   0xe185d956e980718a+        .quad   0x2f1b78fab143a8a6+        .quad   0x96eebffb305b2f51+        .quad   0xd3f938ad889596b8+        .quad   0xf0f52dc746d5dd25+        .quad   0x57968290bb3a0095+        .quad   0xf71ab8430a20e101+        .quad   0xf393658d24f0ec47+        .quad   0xcf7509a86ee2eed1+        .quad   0x7dc43e35dc2aa3e1++        // 2^160 * 2 * G++        .quad   0x85966665887dd9c3+        .quad   0xc90f9b314bb05355+        .quad   0xc6e08df8ef2079b1+        .quad   0x7ef72016758cc12f+        .quad   0x5a782a5c273e9718+        .quad   0x3576c6995e4efd94+        .quad   0x0f2ed8051f237d3e+        .quad   0x044fb81d82d50a99+        .quad   0xc1df18c5a907e3d9+        .quad   0x57b3371dce4c6359+        .quad   0xca704534b201bb49+        .quad   0x7f79823f9c30dd2e++        // 2^160 * 3 * G++        .quad   0x8334d239a3b513e8+        .quad   0xc13670d4b91fa8d8+        .quad   0x12b54136f590bd33+        .quad   0x0a4e0373d784d9b4+        .quad   0x6a9c1ff068f587ba+        .quad   0x0827894e0050c8de+        .quad   0x3cbf99557ded5be7+        .quad   0x64a9b0431c06d6f0+        .quad   0x2eb3d6a15b7d2919+        .quad   0xb0b4f6a0d53a8235+        .quad   0x7156ce4389a45d47+        .quad   0x071a7d0ace18346c++        // 2^160 * 4 * G++        .quad   0xd3072daac887ba0b+        .quad   0x01262905bfa562ee+        .quad   0xcf543002c0ef768b+        .quad   0x2c3bcc7146ea7e9c+        .quad   0xcc0c355220e14431+        .quad   0x0d65950709b15141+        .quad   0x9af5621b209d5f36+        .quad   0x7c69bcf7617755d3+        .quad   0x07f0d7eb04e8295f+        .quad   0x10db18252f50f37d+        .quad   0xe951a9a3171798d7+        .quad   0x6f5a9a7322aca51d++        // 2^160 * 5 * G++        .quad   0x8ba1000c2f41c6c5+        .quad   0xc49f79c10cfefb9b+        .quad   0x4efa47703cc51c9f+        .quad   0x494e21a2e147afca+        .quad   0xe729d4eba3d944be+        .quad   0x8d9e09408078af9e+        .quad   0x4525567a47869c03+        .quad   0x02ab9680ee8d3b24+        .quad   0xefa48a85dde50d9a+        .quad   0x219a224e0fb9a249+        .quad   0xfa091f1dd91ef6d9+        .quad   0x6b5d76cbea46bb34++        // 2^160 * 6 * G++        .quad   0x8857556cec0cd994+        .quad   0x6472dc6f5cd01dba+        .quad   0xaf0169148f42b477+        .quad   0x0ae333f685277354+        .quad   0xe0f941171e782522+        .quad   0xf1e6ae74036936d3+        .quad   0x408b3ea2d0fcc746+        .quad   0x16fb869c03dd313e+        .quad   0x288e199733b60962+        .quad   0x24fc72b4d8abe133+        .quad   0x4811f7ed0991d03e+        .quad   0x3f81e38b8f70d075++        // 2^160 * 7 * G++        .quad   0x7f910fcc7ed9affe+        .quad   0x545cb8a12465874b+        .quad   0xa8397ed24b0c4704+        .quad   0x50510fc104f50993+        .quad   0x0adb7f355f17c824+        .quad   0x74b923c3d74299a4+        .quad   0xd57c3e8bcbf8eaf7+        .quad   0x0ad3e2d34cdedc3d+        .quad   0x6f0c0fc5336e249d+        .quad   0x745ede19c331cfd9+        .quad   0xf2d6fd0009eefe1c+        .quad   0x127c158bf0fa1ebe++        // 2^160 * 8 * G++        .quad   0xf6197c422e9879a2+        .quad   0xa44addd452ca3647+        .quad   0x9b413fc14b4eaccb+        .quad   0x354ef87d07ef4f68+        .quad   0xdea28fc4ae51b974+        .quad   0x1d9973d3744dfe96+        .quad   0x6240680b873848a8+        .quad   0x4ed82479d167df95+        .quad   0xfee3b52260c5d975+        .quad   0x50352efceb41b0b8+        .quad   0x8808ac30a9f6653c+        .quad   0x302d92d20539236d++        // 2^164 * 1 * G++        .quad   0x4c59023fcb3efb7c+        .quad   0x6c2fcb99c63c2a94+        .quad   0xba4190e2c3c7e084+        .quad   0x0e545daea51874d9+        .quad   0x957b8b8b0df53c30+        .quad   0x2a1c770a8e60f098+        .quad   0xbbc7a670345796de+        .quad   0x22a48f9a90c99bc9+        .quad   0x6b7dc0dc8d3fac58+        .quad   0x5497cd6ce6e42bfd+        .quad   0x542f7d1bf400d305+        .quad   0x4159f47f048d9136++        // 2^164 * 2 * G++        .quad   0x20ad660839e31e32+        .quad   0xf81e1bd58405be50+        .quad   0xf8064056f4dabc69+        .quad   0x14d23dd4ce71b975+        .quad   0x748515a8bbd24839+        .quad   0x77128347afb02b55+        .quad   0x50ba2ac649a2a17f+        .quad   0x060525513ad730f1+        .quad   0xf2398e098aa27f82+        .quad   0x6d7982bb89a1b024+        .quad   0xfa694084214dd24c+        .quad   0x71ab966fa32301c3++        // 2^164 * 3 * G++        .quad   0x2dcbd8e34ded02fc+        .quad   0x1151f3ec596f22aa+        .quad   0xbca255434e0328da+        .quad   0x35768fbe92411b22+        .quad   0xb1088a0702809955+        .quad   0x43b273ea0b43c391+        .quad   0xca9b67aefe0686ed+        .quad   0x605eecbf8335f4ed+        .quad   0x83200a656c340431+        .quad   0x9fcd71678ee59c2f+        .quad   0x75d4613f71300f8a+        .quad   0x7a912faf60f542f9++        // 2^164 * 4 * G++        .quad   0xb204585e5edc1a43+        .quad   0x9f0e16ee5897c73c+        .quad   0x5b82c0ae4e70483c+        .quad   0x624a170e2bddf9be+        .quad   0x253f4f8dfa2d5597+        .quad   0x25e49c405477130c+        .quad   0x00c052e5996b1102+        .quad   0x33cb966e33bb6c4a+        .quad   0x597028047f116909+        .quad   0x828ac41c1e564467+        .quad   0x70417dbde6217387+        .quad   0x721627aefbac4384++        // 2^164 * 5 * G++        .quad   0x97d03bc38736add5+        .quad   0x2f1422afc532b130+        .quad   0x3aa68a057101bbc4+        .quad   0x4c946cf7e74f9fa7+        .quad   0xfd3097bc410b2f22+        .quad   0xf1a05da7b5cfa844+        .quad   0x61289a1def57ca74+        .quad   0x245ea199bb821902+        .quad   0xaedca66978d477f8+        .quad   0x1898ba3c29117fe1+        .quad   0xcf73f983720cbd58+        .quad   0x67da12e6b8b56351++        // 2^164 * 6 * G++        .quad   0x7067e187b4bd6e07+        .quad   0x6e8f0203c7d1fe74+        .quad   0x93c6aa2f38c85a30+        .quad   0x76297d1f3d75a78a+        .quad   0x2b7ef3d38ec8308c+        .quad   0x828fd7ec71eb94ab+        .quad   0x807c3b36c5062abd+        .quad   0x0cb64cb831a94141+        .quad   0x3030fc33534c6378+        .quad   0xb9635c5ce541e861+        .quad   0x15d9a9bed9b2c728+        .quad   0x49233ea3f3775dcb++        // 2^164 * 7 * G++        .quad   0x629398fa8dbffc3a+        .quad   0xe12fe52dd54db455+        .quad   0xf3be11dfdaf25295+        .quad   0x628b140dce5e7b51+        .quad   0x7b3985fe1c9f249b+        .quad   0x4fd6b2d5a1233293+        .quad   0xceb345941adf4d62+        .quad   0x6987ff6f542de50c+        .quad   0x47e241428f83753c+        .quad   0x6317bebc866af997+        .quad   0xdabb5b433d1a9829+        .quad   0x074d8d245287fb2d++        // 2^164 * 8 * G++        .quad   0x8337d9cd440bfc31+        .quad   0x729d2ca1af318fd7+        .quad   0xa040a4a4772c2070+        .quad   0x46002ef03a7349be+        .quad   0x481875c6c0e31488+        .quad   0x219429b2e22034b4+        .quad   0x7223c98a31283b65+        .quad   0x3420d60b342277f9+        .quad   0xfaa23adeaffe65f7+        .quad   0x78261ed45be0764c+        .quad   0x441c0a1e2f164403+        .quad   0x5aea8e567a87d395++        // 2^168 * 1 * G++        .quad   0x7813c1a2bca4283d+        .quad   0xed62f091a1863dd9+        .quad   0xaec7bcb8c268fa86+        .quad   0x10e5d3b76f1cae4c+        .quad   0x2dbc6fb6e4e0f177+        .quad   0x04e1bf29a4bd6a93+        .quad   0x5e1966d4787af6e8+        .quad   0x0edc5f5eb426d060+        .quad   0x5453bfd653da8e67+        .quad   0xe9dc1eec24a9f641+        .quad   0xbf87263b03578a23+        .quad   0x45b46c51361cba72++        // 2^168 * 2 * G++        .quad   0xa9402abf314f7fa1+        .quad   0xe257f1dc8e8cf450+        .quad   0x1dbbd54b23a8be84+        .quad   0x2177bfa36dcb713b+        .quad   0xce9d4ddd8a7fe3e4+        .quad   0xab13645676620e30+        .quad   0x4b594f7bb30e9958+        .quad   0x5c1c0aef321229df+        .quad   0x37081bbcfa79db8f+        .quad   0x6048811ec25f59b3+        .quad   0x087a76659c832487+        .quad   0x4ae619387d8ab5bb++        // 2^168 * 3 * G++        .quad   0x8ddbf6aa5344a32e+        .quad   0x7d88eab4b41b4078+        .quad   0x5eb0eb974a130d60+        .quad   0x1a00d91b17bf3e03+        .quad   0x61117e44985bfb83+        .quad   0xfce0462a71963136+        .quad   0x83ac3448d425904b+        .quad   0x75685abe5ba43d64+        .quad   0x6e960933eb61f2b2+        .quad   0x543d0fa8c9ff4952+        .quad   0xdf7275107af66569+        .quad   0x135529b623b0e6aa++        // 2^168 * 4 * G++        .quad   0x18f0dbd7add1d518+        .quad   0x979f7888cfc11f11+        .quad   0x8732e1f07114759b+        .quad   0x79b5b81a65ca3a01+        .quad   0xf5c716bce22e83fe+        .quad   0xb42beb19e80985c1+        .quad   0xec9da63714254aae+        .quad   0x5972ea051590a613+        .quad   0x0fd4ac20dc8f7811+        .quad   0x9a9ad294ac4d4fa8+        .quad   0xc01b2d64b3360434+        .quad   0x4f7e9c95905f3bdb++        // 2^168 * 5 * G++        .quad   0x62674bbc5781302e+        .quad   0xd8520f3989addc0f+        .quad   0x8c2999ae53fbd9c6+        .quad   0x31993ad92e638e4c+        .quad   0x71c8443d355299fe+        .quad   0x8bcd3b1cdbebead7+        .quad   0x8092499ef1a49466+        .quad   0x1942eec4a144adc8+        .quad   0x7dac5319ae234992+        .quad   0x2c1b3d910cea3e92+        .quad   0x553ce494253c1122+        .quad   0x2a0a65314ef9ca75++        // 2^168 * 6 * G++        .quad   0x2db7937ff7f927c2+        .quad   0xdb741f0617d0a635+        .quad   0x5982f3a21155af76+        .quad   0x4cf6e218647c2ded+        .quad   0xcf361acd3c1c793a+        .quad   0x2f9ebcac5a35bc3b+        .quad   0x60e860e9a8cda6ab+        .quad   0x055dc39b6dea1a13+        .quad   0xb119227cc28d5bb6+        .quad   0x07e24ebc774dffab+        .quad   0xa83c78cee4a32c89+        .quad   0x121a307710aa24b6++        // 2^168 * 7 * G++        .quad   0xe4db5d5e9f034a97+        .quad   0xe153fc093034bc2d+        .quad   0x460546919551d3b1+        .quad   0x333fc76c7a40e52d+        .quad   0xd659713ec77483c9+        .quad   0x88bfe077b82b96af+        .quad   0x289e28231097bcd3+        .quad   0x527bb94a6ced3a9b+        .quad   0x563d992a995b482e+        .quad   0x3405d07c6e383801+        .quad   0x485035de2f64d8e5+        .quad   0x6b89069b20a7a9f7++        // 2^168 * 8 * G++        .quad   0x812aa0416270220d+        .quad   0x995a89faf9245b4e+        .quad   0xffadc4ce5072ef05+        .quad   0x23bc2103aa73eb73+        .quad   0x4082fa8cb5c7db77+        .quad   0x068686f8c734c155+        .quad   0x29e6c8d9f6e7a57e+        .quad   0x0473d308a7639bcf+        .quad   0xcaee792603589e05+        .quad   0x2b4b421246dcc492+        .quad   0x02a1ef74e601a94f+        .quad   0x102f73bfde04341a++        // 2^172 * 1 * G++        .quad   0xb5a2d50c7ec20d3e+        .quad   0xc64bdd6ea0c97263+        .quad   0x56e89052c1ff734d+        .quad   0x4929c6f72b2ffaba+        .quad   0x358ecba293a36247+        .quad   0xaf8f9862b268fd65+        .quad   0x412f7e9968a01c89+        .quad   0x5786f312cd754524+        .quad   0x337788ffca14032c+        .quad   0xf3921028447f1ee3+        .quad   0x8b14071f231bccad+        .quad   0x4c817b4bf2344783++        // 2^172 * 2 * G++        .quad   0x0ff853852871b96e+        .quad   0xe13e9fab60c3f1bb+        .quad   0xeefd595325344402+        .quad   0x0a37c37075b7744b+        .quad   0x413ba057a40b4484+        .quad   0xba4c2e1a4f5f6a43+        .quad   0x614ba0a5aee1d61c+        .quad   0x78a1531a8b05dc53+        .quad   0x6cbdf1703ad0562b+        .quad   0x8ecf4830c92521a3+        .quad   0xdaebd303fd8424e7+        .quad   0x72ad82a42e5ec56f++        // 2^172 * 3 * G++        .quad   0x3f9e8e35bafb65f6+        .quad   0x39d69ec8f27293a1+        .quad   0x6cb8cd958cf6a3d0+        .quad   0x1734778173adae6d+        .quad   0xc368939167024bc3+        .quad   0x8e69d16d49502fda+        .quad   0xfcf2ec3ce45f4b29+        .quad   0x065f669ea3b4cbc4+        .quad   0x8a00aec75532db4d+        .quad   0xb869a4e443e31bb1+        .quad   0x4a0f8552d3a7f515+        .quad   0x19adeb7c303d7c08++        // 2^172 * 4 * G++        .quad   0xc720cb6153ead9a3+        .quad   0x55b2c97f512b636e+        .quad   0xb1e35b5fd40290b1+        .quad   0x2fd9ccf13b530ee2+        .quad   0x9d05ba7d43c31794+        .quad   0x2470c8ff93322526+        .quad   0x8323dec816197438+        .quad   0x2852709881569b53+        .quad   0x07bd475b47f796b8+        .quad   0xd2c7b013542c8f54+        .quad   0x2dbd23f43b24f87e+        .quad   0x6551afd77b0901d6++        // 2^172 * 5 * G++        .quad   0x4546baaf54aac27f+        .quad   0xf6f66fecb2a45a28+        .quad   0x582d1b5b562bcfe8+        .quad   0x44b123f3920f785f+        .quad   0x68a24ce3a1d5c9ac+        .quad   0xbb77a33d10ff6461+        .quad   0x0f86ce4425d3166e+        .quad   0x56507c0950b9623b+        .quad   0x1206f0b7d1713e63+        .quad   0x353fe3d915bafc74+        .quad   0x194ceb970ad9d94d+        .quad   0x62fadd7cf9d03ad3++        // 2^172 * 6 * G++        .quad   0xc6b5967b5598a074+        .quad   0x5efe91ce8e493e25+        .quad   0xd4b72c4549280888+        .quad   0x20ef1149a26740c2+        .quad   0x3cd7bc61e7ce4594+        .quad   0xcd6b35a9b7dd267e+        .quad   0xa080abc84366ef27+        .quad   0x6ec7c46f59c79711+        .quad   0x2f07ad636f09a8a2+        .quad   0x8697e6ce24205e7d+        .quad   0xc0aefc05ee35a139+        .quad   0x15e80958b5f9d897++        // 2^172 * 7 * G++        .quad   0x25a5ef7d0c3e235b+        .quad   0x6c39c17fbe134ee7+        .quad   0xc774e1342dc5c327+        .quad   0x021354b892021f39+        .quad   0x4dd1ed355bb061c4+        .quad   0x42dc0cef941c0700+        .quad   0x61305dc1fd86340e+        .quad   0x56b2cc930e55a443+        .quad   0x1df79da6a6bfc5a2+        .quad   0x02f3a2749fde4369+        .quad   0xb323d9f2cda390a7+        .quad   0x7be0847b8774d363++        // 2^172 * 8 * G++        .quad   0x8c99cc5a8b3f55c3+        .quad   0x0611d7253fded2a0+        .quad   0xed2995ff36b70a36+        .quad   0x1f699a54d78a2619+        .quad   0x1466f5af5307fa11+        .quad   0x817fcc7ded6c0af2+        .quad   0x0a6de44ec3a4a3fb+        .quad   0x74071475bc927d0b+        .quad   0xe77292f373e7ea8a+        .quad   0x296537d2cb045a31+        .quad   0x1bd0653ed3274fde+        .quad   0x2f9a2c4476bd2966++        // 2^176 * 1 * G++        .quad   0xeb18b9ab7f5745c6+        .quad   0x023a8aee5787c690+        .quad   0xb72712da2df7afa9+        .quad   0x36597d25ea5c013d+        .quad   0xa2b4dae0b5511c9a+        .quad   0x7ac860292bffff06+        .quad   0x981f375df5504234+        .quad   0x3f6bd725da4ea12d+        .quad   0x734d8d7b106058ac+        .quad   0xd940579e6fc6905f+        .quad   0x6466f8f99202932d+        .quad   0x7b7ecc19da60d6d0++        // 2^176 * 2 * G++        .quad   0x78c2373c695c690d+        .quad   0xdd252e660642906e+        .quad   0x951d44444ae12bd2+        .quad   0x4235ad7601743956+        .quad   0x6dae4a51a77cfa9b+        .quad   0x82263654e7a38650+        .quad   0x09bbffcd8f2d82db+        .quad   0x03bedc661bf5caba+        .quad   0x6258cb0d078975f5+        .quad   0x492942549189f298+        .quad   0xa0cab423e2e36ee4+        .quad   0x0e7ce2b0cdf066a1++        // 2^176 * 3 * G++        .quad   0xc494643ac48c85a3+        .quad   0xfd361df43c6139ad+        .quad   0x09db17dd3ae94d48+        .quad   0x666e0a5d8fb4674a+        .quad   0xfea6fedfd94b70f9+        .quad   0xf130c051c1fcba2d+        .quad   0x4882d47e7f2fab89+        .quad   0x615256138aeceeb5+        .quad   0x2abbf64e4870cb0d+        .quad   0xcd65bcf0aa458b6b+        .quad   0x9abe4eba75e8985d+        .quad   0x7f0bc810d514dee4++        // 2^176 * 4 * G++        .quad   0xb9006ba426f4136f+        .quad   0x8d67369e57e03035+        .quad   0xcbc8dfd94f463c28+        .quad   0x0d1f8dbcf8eedbf5+        .quad   0x83ac9dad737213a0+        .quad   0x9ff6f8ba2ef72e98+        .quad   0x311e2edd43ec6957+        .quad   0x1d3a907ddec5ab75+        .quad   0xba1693313ed081dc+        .quad   0x29329fad851b3480+        .quad   0x0128013c030321cb+        .quad   0x00011b44a31bfde3++        // 2^176 * 5 * G++        .quad   0x3fdfa06c3fc66c0c+        .quad   0x5d40e38e4dd60dd2+        .quad   0x7ae38b38268e4d71+        .quad   0x3ac48d916e8357e1+        .quad   0x16561f696a0aa75c+        .quad   0xc1bf725c5852bd6a+        .quad   0x11a8dd7f9a7966ad+        .quad   0x63d988a2d2851026+        .quad   0x00120753afbd232e+        .quad   0xe92bceb8fdd8f683+        .quad   0xf81669b384e72b91+        .quad   0x33fad52b2368a066++        // 2^176 * 6 * G++        .quad   0x540649c6c5e41e16+        .quad   0x0af86430333f7735+        .quad   0xb2acfcd2f305e746+        .quad   0x16c0f429a256dca7+        .quad   0x8d2cc8d0c422cfe8+        .quad   0x072b4f7b05a13acb+        .quad   0xa3feb6e6ecf6a56f+        .quad   0x3cc355ccb90a71e2+        .quad   0xe9b69443903e9131+        .quad   0xb8a494cb7a5637ce+        .quad   0xc87cd1a4baba9244+        .quad   0x631eaf426bae7568++        // 2^176 * 7 * G++        .quad   0xb3e90410da66fe9f+        .quad   0x85dd4b526c16e5a6+        .quad   0xbc3d97611ef9bf83+        .quad   0x5599648b1ea919b5+        .quad   0x47d975b9a3700de8+        .quad   0x7280c5fbe2f80552+        .quad   0x53658f2732e45de1+        .quad   0x431f2c7f665f80b5+        .quad   0xd6026344858f7b19+        .quad   0x14ab352fa1ea514a+        .quad   0x8900441a2090a9d7+        .quad   0x7b04715f91253b26++        // 2^176 * 8 * G++        .quad   0x83edbd28acf6ae43+        .quad   0x86357c8b7d5c7ab4+        .quad   0xc0404769b7eb2c44+        .quad   0x59b37bf5c2f6583f+        .quad   0xb376c280c4e6bac6+        .quad   0x970ed3dd6d1d9b0b+        .quad   0xb09a9558450bf944+        .quad   0x48d0acfa57cde223+        .quad   0xb60f26e47dabe671+        .quad   0xf1d1a197622f3a37+        .quad   0x4208ce7ee9960394+        .quad   0x16234191336d3bdb++        // 2^180 * 1 * G++        .quad   0xf19aeac733a63aef+        .quad   0x2c7fba5d4442454e+        .quad   0x5da87aa04795e441+        .quad   0x413051e1a4e0b0f5+        .quad   0x852dd1fd3d578bbe+        .quad   0x2b65ce72c3286108+        .quad   0x658c07f4eace2273+        .quad   0x0933f804ec38ab40+        .quad   0xa7ab69798d496476+        .quad   0x8121aadefcb5abc8+        .quad   0xa5dc12ef7b539472+        .quad   0x07fd47065e45351a++        // 2^180 * 2 * G++        .quad   0xc8583c3d258d2bcd+        .quad   0x17029a4daf60b73f+        .quad   0xfa0fc9d6416a3781+        .quad   0x1c1e5fba38b3fb23+        .quad   0x304211559ae8e7c3+        .quad   0xf281b229944882a5+        .quad   0x8a13ac2e378250e4+        .quad   0x014afa0954ba48f4+        .quad   0xcb3197001bb3666c+        .quad   0x330060524bffecb9+        .quad   0x293711991a88233c+        .quad   0x291884363d4ed364++        // 2^180 * 3 * G++        .quad   0x033c6805dc4babfa+        .quad   0x2c15bf5e5596ecc1+        .quad   0x1bc70624b59b1d3b+        .quad   0x3ede9850a19f0ec5+        .quad   0xfb9d37c3bc1ab6eb+        .quad   0x02be14534d57a240+        .quad   0xf4d73415f8a5e1f6+        .quad   0x5964f4300ccc8188+        .quad   0xe44a23152d096800+        .quad   0x5c08c55970866996+        .quad   0xdf2db60a46affb6e+        .quad   0x579155c1f856fd89++        // 2^180 * 4 * G++        .quad   0x96324edd12e0c9ef+        .quad   0x468b878df2420297+        .quad   0x199a3776a4f573be+        .quad   0x1e7fbcf18e91e92a+        .quad   0xb5f16b630817e7a6+        .quad   0x808c69233c351026+        .quad   0x324a983b54cef201+        .quad   0x53c092084a485345+        .quad   0xd2d41481f1cbafbf+        .quad   0x231d2db6716174e5+        .quad   0x0b7d7656e2a55c98+        .quad   0x3e955cd82aa495f6++        // 2^180 * 5 * G++        .quad   0xe48f535e3ed15433+        .quad   0xd075692a0d7270a3+        .quad   0x40fbd21daade6387+        .quad   0x14264887cf4495f5+        .quad   0xab39f3ef61bb3a3f+        .quad   0x8eb400652eb9193e+        .quad   0xb5de6ecc38c11f74+        .quad   0x654d7e9626f3c49f+        .quad   0xe564cfdd5c7d2ceb+        .quad   0x82eeafded737ccb9+        .quad   0x6107db62d1f9b0ab+        .quad   0x0b6baac3b4358dbb++        // 2^180 * 6 * G++        .quad   0x7ae62bcb8622fe98+        .quad   0x47762256ceb891af+        .quad   0x1a5a92bcf2e406b4+        .quad   0x7d29401784e41501+        .quad   0x204abad63700a93b+        .quad   0xbe0023d3da779373+        .quad   0xd85f0346633ab709+        .quad   0x00496dc490820412+        .quad   0x1c74b88dc27e6360+        .quad   0x074854268d14850c+        .quad   0xa145fb7b3e0dcb30+        .quad   0x10843f1b43803b23++        // 2^180 * 7 * G++        .quad   0xc5f90455376276dd+        .quad   0xce59158dd7645cd9+        .quad   0x92f65d511d366b39+        .quad   0x11574b6e526996c4+        .quad   0xd56f672de324689b+        .quad   0xd1da8aedb394a981+        .quad   0xdd7b58fe9168cfed+        .quad   0x7ce246cd4d56c1e8+        .quad   0xb8f4308e7f80be53+        .quad   0x5f3cb8cb34a9d397+        .quad   0x18a961bd33cc2b2c+        .quad   0x710045fb3a9af671++        // 2^180 * 8 * G++        .quad   0x73f93d36101b95eb+        .quad   0xfaef33794f6f4486+        .quad   0x5651735f8f15e562+        .quad   0x7fa3f19058b40da1+        .quad   0xa03fc862059d699e+        .quad   0x2370cfa19a619e69+        .quad   0xc4fe3b122f823deb+        .quad   0x1d1b056fa7f0844e+        .quad   0x1bc64631e56bf61f+        .quad   0xd379ab106e5382a3+        .quad   0x4d58c57e0540168d+        .quad   0x566256628442d8e4++        // 2^184 * 1 * G++        .quad   0xb9e499def6267ff6+        .quad   0x7772ca7b742c0843+        .quad   0x23a0153fe9a4f2b1+        .quad   0x2cdfdfecd5d05006+        .quad   0xdd499cd61ff38640+        .quad   0x29cd9bc3063625a0+        .quad   0x51e2d8023dd73dc3+        .quad   0x4a25707a203b9231+        .quad   0x2ab7668a53f6ed6a+        .quad   0x304242581dd170a1+        .quad   0x4000144c3ae20161+        .quad   0x5721896d248e49fc++        // 2^184 * 2 * G++        .quad   0x0b6e5517fd181bae+        .quad   0x9022629f2bb963b4+        .quad   0x5509bce932064625+        .quad   0x578edd74f63c13da+        .quad   0x285d5091a1d0da4e+        .quad   0x4baa6fa7b5fe3e08+        .quad   0x63e5177ce19393b3+        .quad   0x03c935afc4b030fd+        .quad   0x997276c6492b0c3d+        .quad   0x47ccc2c4dfe205fc+        .quad   0xdcd29b84dd623a3c+        .quad   0x3ec2ab590288c7a2++        // 2^184 * 3 * G++        .quad   0xa1a0d27be4d87bb9+        .quad   0xa98b4deb61391aed+        .quad   0x99a0ddd073cb9b83+        .quad   0x2dd5c25a200fcace+        .quad   0xa7213a09ae32d1cb+        .quad   0x0f2b87df40f5c2d5+        .quad   0x0baea4c6e81eab29+        .quad   0x0e1bf66c6adbac5e+        .quad   0xe2abd5e9792c887e+        .quad   0x1a020018cb926d5d+        .quad   0xbfba69cdbaae5f1e+        .quad   0x730548b35ae88f5f++        // 2^184 * 4 * G++        .quad   0xc43551a3cba8b8ee+        .quad   0x65a26f1db2115f16+        .quad   0x760f4f52ab8c3850+        .quad   0x3043443b411db8ca+        .quad   0x805b094ba1d6e334+        .quad   0xbf3ef17709353f19+        .quad   0x423f06cb0622702b+        .quad   0x585a2277d87845dd+        .quad   0xa18a5f8233d48962+        .quad   0x6698c4b5ec78257f+        .quad   0xa78e6fa5373e41ff+        .quad   0x7656278950ef981f++        // 2^184 * 5 * G++        .quad   0x38c3cf59d51fc8c0+        .quad   0x9bedd2fd0506b6f2+        .quad   0x26bf109fab570e8f+        .quad   0x3f4160a8c1b846a6+        .quad   0xe17073a3ea86cf9d+        .quad   0x3a8cfbb707155fdc+        .quad   0x4853e7fc31838a8e+        .quad   0x28bbf484b613f616+        .quad   0xf2612f5c6f136c7c+        .quad   0xafead107f6dd11be+        .quad   0x527e9ad213de6f33+        .quad   0x1e79cb358188f75d++        // 2^184 * 6 * G++        .quad   0x013436c3eef7e3f1+        .quad   0x828b6a7ffe9e10f8+        .quad   0x7ff908e5bcf9defc+        .quad   0x65d7951b3a3b3831+        .quad   0x77e953d8f5e08181+        .quad   0x84a50c44299dded9+        .quad   0xdc6c2d0c864525e5+        .quad   0x478ab52d39d1f2f4+        .quad   0x66a6a4d39252d159+        .quad   0xe5dde1bc871ac807+        .quad   0xb82c6b40a6c1c96f+        .quad   0x16d87a411a212214++        // 2^184 * 7 * G++        .quad   0xb3bd7e5a42066215+        .quad   0x879be3cd0c5a24c1+        .quad   0x57c05db1d6f994b7+        .quad   0x28f87c8165f38ca6+        .quad   0xfba4d5e2d54e0583+        .quad   0xe21fafd72ebd99fa+        .quad   0x497ac2736ee9778f+        .quad   0x1f990b577a5a6dde+        .quad   0xa3344ead1be8f7d6+        .quad   0x7d1e50ebacea798f+        .quad   0x77c6569e520de052+        .quad   0x45882fe1534d6d3e++        // 2^184 * 8 * G++        .quad   0x6669345d757983d6+        .quad   0x62b6ed1117aa11a6+        .quad   0x7ddd1857985e128f+        .quad   0x688fe5b8f626f6dd+        .quad   0xd8ac9929943c6fe4+        .quad   0xb5f9f161a38392a2+        .quad   0x2699db13bec89af3+        .quad   0x7dcf843ce405f074+        .quad   0x6c90d6484a4732c0+        .quad   0xd52143fdca563299+        .quad   0xb3be28c3915dc6e1+        .quad   0x6739687e7327191b++        // 2^188 * 1 * G++        .quad   0x9f65c5ea200814cf+        .quad   0x840536e169a31740+        .quad   0x8b0ed13925c8b4ad+        .quad   0x0080dbafe936361d+        .quad   0x8ce5aad0c9cb971f+        .quad   0x1156aaa99fd54a29+        .quad   0x41f7247015af9b78+        .quad   0x1fe8cca8420f49aa+        .quad   0x72a1848f3c0cc82a+        .quad   0x38c560c2877c9e54+        .quad   0x5004e228ce554140+        .quad   0x042418a103429d71++        // 2^188 * 2 * G++        .quad   0x899dea51abf3ff5f+        .quad   0x9b93a8672fc2d8ba+        .quad   0x2c38cb97be6ebd5c+        .quad   0x114d578497263b5d+        .quad   0x58e84c6f20816247+        .quad   0x8db2b2b6e36fd793+        .quad   0x977182561d484d85+        .quad   0x0822024f8632abd7+        .quad   0xb301bb7c6b1beca3+        .quad   0x55393f6dc6eb1375+        .quad   0x910d281097b6e4eb+        .quad   0x1ad4548d9d479ea3++        // 2^188 * 3 * G++        .quad   0xcd5a7da0389a48fd+        .quad   0xb38fa4aa9a78371e+        .quad   0xc6d9761b2cdb8e6c+        .quad   0x35cf51dbc97e1443+        .quad   0xa06fe66d0fe9fed3+        .quad   0xa8733a401c587909+        .quad   0x30d14d800df98953+        .quad   0x41ce5876c7b30258+        .quad   0x59ac3bc5d670c022+        .quad   0xeae67c109b119406+        .quad   0x9798bdf0b3782fda+        .quad   0x651e3201fd074092++        // 2^188 * 4 * G++        .quad   0xd63d8483ef30c5cf+        .quad   0x4cd4b4962361cc0c+        .quad   0xee90e500a48426ac+        .quad   0x0af51d7d18c14eeb+        .quad   0xa57ba4a01efcae9e+        .quad   0x769f4beedc308a94+        .quad   0xd1f10eeb3603cb2e+        .quad   0x4099ce5e7e441278+        .quad   0x1ac98e4f8a5121e9+        .quad   0x7dae9544dbfa2fe0+        .quad   0x8320aa0dd6430df9+        .quad   0x667282652c4a2fb5++        // 2^188 * 5 * G++        .quad   0x874621f4d86bc9ab+        .quad   0xb54c7bbe56fe6fea+        .quad   0x077a24257fadc22c+        .quad   0x1ab53be419b90d39+        .quad   0xada8b6e02946db23+        .quad   0x1c0ce51a7b253ab7+        .quad   0x8448c85a66dd485b+        .quad   0x7f1fc025d0675adf+        .quad   0xd8ee1b18319ea6aa+        .quad   0x004d88083a21f0da+        .quad   0x3bd6aa1d883a4f4b+        .quad   0x4db9a3a6dfd9fd14++        // 2^188 * 6 * G++        .quad   0x8ce7b23bb99c0755+        .quad   0x35c5d6edc4f50f7a+        .quad   0x7e1e2ed2ed9b50c3+        .quad   0x36305f16e8934da1+        .quad   0xd95b00bbcbb77c68+        .quad   0xddbc846a91f17849+        .quad   0x7cf700aebe28d9b3+        .quad   0x5ce1285c85d31f3e+        .quad   0x31b6972d98b0bde8+        .quad   0x7d920706aca6de5b+        .quad   0xe67310f8908a659f+        .quad   0x50fac2a6efdf0235++        // 2^188 * 7 * G++        .quad   0xf3d3a9f35b880f5a+        .quad   0xedec050cdb03e7c2+        .quad   0xa896981ff9f0b1a2+        .quad   0x49a4ae2bac5e34a4+        .quad   0x295b1c86f6f449bc+        .quad   0x51b2e84a1f0ab4dd+        .quad   0xc001cb30aa8e551d+        .quad   0x6a28d35944f43662+        .quad   0x28bb12ee04a740e0+        .quad   0x14313bbd9bce8174+        .quad   0x72f5b5e4e8c10c40+        .quad   0x7cbfb19936adcd5b++        // 2^188 * 8 * G++        .quad   0xa311ddc26b89792d+        .quad   0x1b30b4c6da512664+        .quad   0x0ca77b4ccf150859+        .quad   0x1de443df1b009408+        .quad   0x8e793a7acc36e6e0+        .quad   0xf9fab7a37d586eed+        .quad   0x3a4f9692bae1f4e4+        .quad   0x1c14b03eff5f447e+        .quad   0x19647bd114a85291+        .quad   0x57b76cb21034d3af+        .quad   0x6329db440f9d6dfa+        .quad   0x5ef43e586a571493++        // 2^192 * 1 * G++        .quad   0xef782014385675a6+        .quad   0xa2649f30aafda9e8+        .quad   0x4cd1eb505cdfa8cb+        .quad   0x46115aba1d4dc0b3+        .quad   0xa66dcc9dc80c1ac0+        .quad   0x97a05cf41b38a436+        .quad   0xa7ebf3be95dbd7c6+        .quad   0x7da0b8f68d7e7dab+        .quad   0xd40f1953c3b5da76+        .quad   0x1dac6f7321119e9b+        .quad   0x03cc6021feb25960+        .quad   0x5a5f887e83674b4b++        // 2^192 * 2 * G++        .quad   0x8f6301cf70a13d11+        .quad   0xcfceb815350dd0c4+        .quad   0xf70297d4a4bca47e+        .quad   0x3669b656e44d1434+        .quad   0x9e9628d3a0a643b9+        .quad   0xb5c3cb00e6c32064+        .quad   0x9b5302897c2dec32+        .quad   0x43e37ae2d5d1c70c+        .quad   0x387e3f06eda6e133+        .quad   0x67301d5199a13ac0+        .quad   0xbd5ad8f836263811+        .quad   0x6a21e6cd4fd5e9be++        // 2^192 * 3 * G++        .quad   0xf1c6170a3046e65f+        .quad   0x58712a2a00d23524+        .quad   0x69dbbd3c8c82b755+        .quad   0x586bf9f1a195ff57+        .quad   0xef4129126699b2e3+        .quad   0x71d30847708d1301+        .quad   0x325432d01182b0bd+        .quad   0x45371b07001e8b36+        .quad   0xa6db088d5ef8790b+        .quad   0x5278f0dc610937e5+        .quad   0xac0349d261a16eb8+        .quad   0x0eafb03790e52179++        // 2^192 * 4 * G++        .quad   0x960555c13748042f+        .quad   0x219a41e6820baa11+        .quad   0x1c81f73873486d0c+        .quad   0x309acc675a02c661+        .quad   0x5140805e0f75ae1d+        .quad   0xec02fbe32662cc30+        .quad   0x2cebdf1eea92396d+        .quad   0x44ae3344c5435bb3+        .quad   0x9cf289b9bba543ee+        .quad   0xf3760e9d5ac97142+        .quad   0x1d82e5c64f9360aa+        .quad   0x62d5221b7f94678f++        // 2^192 * 5 * G++        .quad   0x524c299c18d0936d+        .quad   0xc86bb56c8a0c1a0c+        .quad   0xa375052edb4a8631+        .quad   0x5c0efde4bc754562+        .quad   0x7585d4263af77a3c+        .quad   0xdfae7b11fee9144d+        .quad   0xa506708059f7193d+        .quad   0x14f29a5383922037+        .quad   0xdf717edc25b2d7f5+        .quad   0x21f970db99b53040+        .quad   0xda9234b7c3ed4c62+        .quad   0x5e72365c7bee093e++        // 2^192 * 6 * G++        .quad   0x575bfc074571217f+        .quad   0x3779675d0694d95b+        .quad   0x9a0a37bbf4191e33+        .quad   0x77f1104c47b4eabc+        .quad   0x7d9339062f08b33e+        .quad   0x5b9659e5df9f32be+        .quad   0xacff3dad1f9ebdfd+        .quad   0x70b20555cb7349b7+        .quad   0xbe5113c555112c4c+        .quad   0x6688423a9a881fcd+        .quad   0x446677855e503b47+        .quad   0x0e34398f4a06404a++        // 2^192 * 7 * G++        .quad   0xb67d22d93ecebde8+        .quad   0x09b3e84127822f07+        .quad   0x743fa61fb05b6d8d+        .quad   0x5e5405368a362372+        .quad   0x18930b093e4b1928+        .quad   0x7de3e10e73f3f640+        .quad   0xf43217da73395d6f+        .quad   0x6f8aded6ca379c3e+        .quad   0xe340123dfdb7b29a+        .quad   0x487b97e1a21ab291+        .quad   0xf9967d02fde6949e+        .quad   0x780de72ec8d3de97++        // 2^192 * 8 * G++        .quad   0x0ae28545089ae7bc+        .quad   0x388ddecf1c7f4d06+        .quad   0x38ac15510a4811b8+        .quad   0x0eb28bf671928ce4+        .quad   0x671feaf300f42772+        .quad   0x8f72eb2a2a8c41aa+        .quad   0x29a17fd797373292+        .quad   0x1defc6ad32b587a6+        .quad   0xaf5bbe1aef5195a7+        .quad   0x148c1277917b15ed+        .quad   0x2991f7fb7ae5da2e+        .quad   0x467d201bf8dd2867++        // 2^196 * 1 * G++        .quad   0x7906ee72f7bd2e6b+        .quad   0x05d270d6109abf4e+        .quad   0x8d5cfe45b941a8a4+        .quad   0x44c218671c974287+        .quad   0x745f9d56296bc318+        .quad   0x993580d4d8152e65+        .quad   0xb0e5b13f5839e9ce+        .quad   0x51fc2b28d43921c0+        .quad   0x1b8fd11795e2a98c+        .quad   0x1c4e5ee12b6b6291+        .quad   0x5b30e7107424b572+        .quad   0x6e6b9de84c4f4ac6++        // 2^196 * 2 * G++        .quad   0xdff25fce4b1de151+        .quad   0xd841c0c7e11c4025+        .quad   0x2554b3c854749c87+        .quad   0x2d292459908e0df9+        .quad   0x6b7c5f10f80cb088+        .quad   0x736b54dc56e42151+        .quad   0xc2b620a5c6ef99c4+        .quad   0x5f4c802cc3a06f42+        .quad   0x9b65c8f17d0752da+        .quad   0x881ce338c77ee800+        .quad   0xc3b514f05b62f9e3+        .quad   0x66ed5dd5bec10d48++        // 2^196 * 3 * G++        .quad   0x7d38a1c20bb2089d+        .quad   0x808334e196ccd412+        .quad   0xc4a70b8c6c97d313+        .quad   0x2eacf8bc03007f20+        .quad   0xf0adf3c9cbca047d+        .quad   0x81c3b2cbf4552f6b+        .quad   0xcfda112d44735f93+        .quad   0x1f23a0c77e20048c+        .quad   0xf235467be5bc1570+        .quad   0x03d2d9020dbab38c+        .quad   0x27529aa2fcf9e09e+        .quad   0x0840bef29d34bc50++        // 2^196 * 4 * G++        .quad   0x796dfb35dc10b287+        .quad   0x27176bcd5c7ff29d+        .quad   0x7f3d43e8c7b24905+        .quad   0x0304f5a191c54276+        .quad   0xcd54e06b7f37e4eb+        .quad   0x8cc15f87f5e96cca+        .quad   0xb8248bb0d3597dce+        .quad   0x246affa06074400c+        .quad   0x37d88e68fbe45321+        .quad   0x86097548c0d75032+        .quad   0x4e9b13ef894a0d35+        .quad   0x25a83cac5753d325++        // 2^196 * 5 * G++        .quad   0x10222f48eed8165e+        .quad   0x623fc1234b8bcf3a+        .quad   0x1e145c09c221e8f0+        .quad   0x7ccfa59fca782630+        .quad   0x9f0f66293952b6e2+        .quad   0x33db5e0e0934267b+        .quad   0xff45252bd609fedc+        .quad   0x06be10f5c506e0c9+        .quad   0x1a9615a9b62a345f+        .quad   0x22050c564a52fecc+        .quad   0xa7a2788528bc0dfe+        .quad   0x5e82770a1a1ee71d++        // 2^196 * 6 * G++        .quad   0x35425183ad896a5c+        .quad   0xe8673afbe78d52f6+        .quad   0x2c66f25f92a35f64+        .quad   0x09d04f3b3b86b102+        .quad   0xe802e80a42339c74+        .quad   0x34175166a7fffae5+        .quad   0x34865d1f1c408cae+        .quad   0x2cca982c605bc5ee+        .quad   0xfd2d5d35197dbe6e+        .quad   0x207c2eea8be4ffa3+        .quad   0x2613d8db325ae918+        .quad   0x7a325d1727741d3e++        // 2^196 * 7 * G++        .quad   0xd036b9bbd16dfde2+        .quad   0xa2055757c497a829+        .quad   0x8e6cc966a7f12667+        .quad   0x4d3b1a791239c180+        .quad   0xecd27d017e2a076a+        .quad   0xd788689f1636495e+        .quad   0x52a61af0919233e5+        .quad   0x2a479df17bb1ae64+        .quad   0x9e5eee8e33db2710+        .quad   0x189854ded6c43ca5+        .quad   0xa41c22c592718138+        .quad   0x27ad5538a43a5e9b++        // 2^196 * 8 * G++        .quad   0x2746dd4b15350d61+        .quad   0xd03fcbc8ee9521b7+        .quad   0xe86e365a138672ca+        .quad   0x510e987f7e7d89e2+        .quad   0xcb5a7d638e47077c+        .quad   0x8db7536120a1c059+        .quad   0x549e1e4d8bedfdcc+        .quad   0x080153b7503b179d+        .quad   0xdda69d930a3ed3e3+        .quad   0x3d386ef1cd60a722+        .quad   0xc817ad58bdaa4ee6+        .quad   0x23be8d554fe7372a++        // 2^200 * 1 * G++        .quad   0x95fe919a74ef4fad+        .quad   0x3a827becf6a308a2+        .quad   0x964e01d309a47b01+        .quad   0x71c43c4f5ba3c797+        .quad   0xbc1ef4bd567ae7a9+        .quad   0x3f624cb2d64498bd+        .quad   0xe41064d22c1f4ec8+        .quad   0x2ef9c5a5ba384001+        .quad   0xb6fd6df6fa9e74cd+        .quad   0xf18278bce4af267a+        .quad   0x8255b3d0f1ef990e+        .quad   0x5a758ca390c5f293++        // 2^200 * 2 * G++        .quad   0xa2b72710d9462495+        .quad   0x3aa8c6d2d57d5003+        .quad   0xe3d400bfa0b487ca+        .quad   0x2dbae244b3eb72ec+        .quad   0x8ce0918b1d61dc94+        .quad   0x8ded36469a813066+        .quad   0xd4e6a829afe8aad3+        .quad   0x0a738027f639d43f+        .quad   0x980f4a2f57ffe1cc+        .quad   0x00670d0de1839843+        .quad   0x105c3f4a49fb15fd+        .quad   0x2698ca635126a69c++        // 2^200 * 3 * G++        .quad   0xe765318832b0ba78+        .quad   0x381831f7925cff8b+        .quad   0x08a81b91a0291fcc+        .quad   0x1fb43dcc49caeb07+        .quad   0x2e3d702f5e3dd90e+        .quad   0x9e3f0918e4d25386+        .quad   0x5e773ef6024da96a+        .quad   0x3c004b0c4afa3332+        .quad   0x9aa946ac06f4b82b+        .quad   0x1ca284a5a806c4f3+        .quad   0x3ed3265fc6cd4787+        .quad   0x6b43fd01cd1fd217++        // 2^200 * 4 * G++        .quad   0xc7a75d4b4697c544+        .quad   0x15fdf848df0fffbf+        .quad   0x2868b9ebaa46785a+        .quad   0x5a68d7105b52f714+        .quad   0xb5c742583e760ef3+        .quad   0x75dc52b9ee0ab990+        .quad   0xbf1427c2072b923f+        .quad   0x73420b2d6ff0d9f0+        .quad   0xaf2cf6cb9e851e06+        .quad   0x8f593913c62238c4+        .quad   0xda8ab89699fbf373+        .quad   0x3db5632fea34bc9e++        // 2^200 * 5 * G++        .quad   0xf46eee2bf75dd9d8+        .quad   0x0d17b1f6396759a5+        .quad   0x1bf2d131499e7273+        .quad   0x04321adf49d75f13+        .quad   0x2e4990b1829825d5+        .quad   0xedeaeb873e9a8991+        .quad   0xeef03d394c704af8+        .quad   0x59197ea495df2b0e+        .quad   0x04e16019e4e55aae+        .quad   0xe77b437a7e2f92e9+        .quad   0xc7ce2dc16f159aa4+        .quad   0x45eafdc1f4d70cc0++        // 2^200 * 6 * G++        .quad   0x698401858045d72b+        .quad   0x4c22faa2cf2f0651+        .quad   0x941a36656b222dc6+        .quad   0x5a5eebc80362dade+        .quad   0xb60e4624cfccb1ed+        .quad   0x59dbc292bd5c0395+        .quad   0x31a09d1ddc0481c9+        .quad   0x3f73ceea5d56d940+        .quad   0xb7a7bfd10a4e8dc6+        .quad   0xbe57007e44c9b339+        .quad   0x60c1207f1557aefa+        .quad   0x26058891266218db++        // 2^200 * 7 * G++        .quad   0x59f704a68360ff04+        .quad   0xc3d93fde7661e6f4+        .quad   0x831b2a7312873551+        .quad   0x54ad0c2e4e615d57+        .quad   0x4c818e3cc676e542+        .quad   0x5e422c9303ceccad+        .quad   0xec07cccab4129f08+        .quad   0x0dedfa10b24443b8+        .quad   0xee3b67d5b82b522a+        .quad   0x36f163469fa5c1eb+        .quad   0xa5b4d2f26ec19fd3+        .quad   0x62ecb2baa77a9408++        // 2^200 * 8 * G++        .quad   0xe5ed795261152b3d+        .quad   0x4962357d0eddd7d1+        .quad   0x7482c8d0b96b4c71+        .quad   0x2e59f919a966d8be+        .quad   0x92072836afb62874+        .quad   0x5fcd5e8579e104a5+        .quad   0x5aad01adc630a14a+        .quad   0x61913d5075663f98+        .quad   0x0dc62d361a3231da+        .quad   0xfa47583294200270+        .quad   0x02d801513f9594ce+        .quad   0x3ddbc2a131c05d5c++        // 2^204 * 1 * G++        .quad   0x3f50a50a4ffb81ef+        .quad   0xb1e035093bf420bf+        .quad   0x9baa8e1cc6aa2cd0+        .quad   0x32239861fa237a40+        .quad   0xfb735ac2004a35d1+        .quad   0x31de0f433a6607c3+        .quad   0x7b8591bfc528d599+        .quad   0x55be9a25f5bb050c+        .quad   0x0d005acd33db3dbf+        .quad   0x0111b37c80ac35e2+        .quad   0x4892d66c6f88ebeb+        .quad   0x770eadb16508fbcd++        // 2^204 * 2 * G++        .quad   0x8451f9e05e4e89dd+        .quad   0xc06302ffbc793937+        .quad   0x5d22749556a6495c+        .quad   0x09a6755ca05603fb+        .quad   0xf1d3b681a05071b9+        .quad   0x2207659a3592ff3a+        .quad   0x5f0169297881e40e+        .quad   0x16bedd0e86ba374e+        .quad   0x5ecccc4f2c2737b5+        .quad   0x43b79e0c2dccb703+        .quad   0x33e008bc4ec43df3+        .quad   0x06c1b840f07566c0++        // 2^204 * 3 * G++        .quad   0x7688a5c6a388f877+        .quad   0x02a96c14deb2b6ac+        .quad   0x64c9f3431b8c2af8+        .quad   0x3628435554a1eed6+        .quad   0x69ee9e7f9b02805c+        .quad   0xcbff828a547d1640+        .quad   0x3d93a869b2430968+        .quad   0x46b7b8cd3fe26972+        .quad   0xe9812086fe7eebe0+        .quad   0x4cba6be72f515437+        .quad   0x1d04168b516efae9+        .quad   0x5ea1391043982cb9++        // 2^204 * 4 * G++        .quad   0x49125c9cf4702ee1+        .quad   0x4520b71f8b25b32d+        .quad   0x33193026501fef7e+        .quad   0x656d8997c8d2eb2b+        .quad   0x6f2b3be4d5d3b002+        .quad   0xafec33d96a09c880+        .quad   0x035f73a4a8bcc4cc+        .quad   0x22c5b9284662198b+        .quad   0xcb58c8fe433d8939+        .quad   0x89a0cb2e6a8d7e50+        .quad   0x79ca955309fbbe5a+        .quad   0x0c626616cd7fc106++        // 2^204 * 5 * G++        .quad   0x1ffeb80a4879b61f+        .quad   0x6396726e4ada21ed+        .quad   0x33c7b093368025ba+        .quad   0x471aa0c6f3c31788+        .quad   0x8fdfc379fbf454b1+        .quad   0x45a5a970f1a4b771+        .quad   0xac921ef7bad35915+        .quad   0x42d088dca81c2192+        .quad   0x8fda0f37a0165199+        .quad   0x0adadb77c8a0e343+        .quad   0x20fbfdfcc875e820+        .quad   0x1cf2bea80c2206e7++        // 2^204 * 6 * G++        .quad   0xc2ddf1deb36202ac+        .quad   0x92a5fe09d2e27aa5+        .quad   0x7d1648f6fc09f1d3+        .quad   0x74c2cc0513bc4959+        .quad   0x982d6e1a02c0412f+        .quad   0x90fa4c83db58e8fe+        .quad   0x01c2f5bcdcb18bc0+        .quad   0x686e0c90216abc66+        .quad   0x1fadbadba54395a7+        .quad   0xb41a02a0ae0da66a+        .quad   0xbf19f598bba37c07+        .quad   0x6a12b8acde48430d++        // 2^204 * 7 * G++        .quad   0xf8daea1f39d495d9+        .quad   0x592c190e525f1dfc+        .quad   0xdb8cbd04c9991d1b+        .quad   0x11f7fda3d88f0cb7+        .quad   0x793bdd801aaeeb5f+        .quad   0x00a2a0aac1518871+        .quad   0xe8a373a31f2136b4+        .quad   0x48aab888fc91ef19+        .quad   0x041f7e925830f40e+        .quad   0x002d6ca979661c06+        .quad   0x86dc9ff92b046a2e+        .quad   0x760360928b0493d1++        // 2^204 * 8 * G++        .quad   0x21bb41c6120cf9c6+        .quad   0xeab2aa12decda59b+        .quad   0xc1a72d020aa48b34+        .quad   0x215d4d27e87d3b68+        .quad   0xb43108e5695a0b05+        .quad   0x6cb00ee8ad37a38b+        .quad   0x5edad6eea3537381+        .quad   0x3f2602d4b6dc3224+        .quad   0xc8b247b65bcaf19c+        .quad   0x49779dc3b1b2c652+        .quad   0x89a180bbd5ece2e2+        .quad   0x13f098a3cec8e039++        // 2^208 * 1 * G++        .quad   0x9adc0ff9ce5ec54b+        .quad   0x039c2a6b8c2f130d+        .quad   0x028007c7f0f89515+        .quad   0x78968314ac04b36b+        .quad   0xf3aa57a22796bb14+        .quad   0x883abab79b07da21+        .quad   0xe54be21831a0391c+        .quad   0x5ee7fb38d83205f9+        .quad   0x538dfdcb41446a8e+        .quad   0xa5acfda9434937f9+        .quad   0x46af908d263c8c78+        .quad   0x61d0633c9bca0d09++        // 2^208 * 2 * G++        .quad   0x63744935ffdb2566+        .quad   0xc5bd6b89780b68bb+        .quad   0x6f1b3280553eec03+        .quad   0x6e965fd847aed7f5+        .quad   0xada328bcf8fc73df+        .quad   0xee84695da6f037fc+        .quad   0x637fb4db38c2a909+        .quad   0x5b23ac2df8067bdc+        .quad   0x9ad2b953ee80527b+        .quad   0xe88f19aafade6d8d+        .quad   0x0e711704150e82cf+        .quad   0x79b9bbb9dd95dedc++        // 2^208 * 3 * G++        .quad   0xebb355406a3126c2+        .quad   0xd26383a868c8c393+        .quad   0x6c0c6429e5b97a82+        .quad   0x5065f158c9fd2147+        .quad   0xd1997dae8e9f7374+        .quad   0xa032a2f8cfbb0816+        .quad   0xcd6cba126d445f0a+        .quad   0x1ba811460accb834+        .quad   0x708169fb0c429954+        .quad   0xe14600acd76ecf67+        .quad   0x2eaab98a70e645ba+        .quad   0x3981f39e58a4faf2++        // 2^208 * 4 * G++        .quad   0x18fb8a7559230a93+        .quad   0x1d168f6960e6f45d+        .quad   0x3a85a94514a93cb5+        .quad   0x38dc083705acd0fd+        .quad   0xc845dfa56de66fde+        .quad   0xe152a5002c40483a+        .quad   0xe9d2e163c7b4f632+        .quad   0x30f4452edcbc1b65+        .quad   0x856d2782c5759740+        .quad   0xfa134569f99cbecc+        .quad   0x8844fc73c0ea4e71+        .quad   0x632d9a1a593f2469++        // 2^208 * 5 * G++        .quad   0xf6bb6b15b807cba6+        .quad   0x1823c7dfbc54f0d7+        .quad   0xbb1d97036e29670b+        .quad   0x0b24f48847ed4a57+        .quad   0xbf09fd11ed0c84a7+        .quad   0x63f071810d9f693a+        .quad   0x21908c2d57cf8779+        .quad   0x3a5a7df28af64ba2+        .quad   0xdcdad4be511beac7+        .quad   0xa4538075ed26ccf2+        .quad   0xe19cff9f005f9a65+        .quad   0x34fcf74475481f63++        // 2^208 * 6 * G++        .quad   0xc197e04c789767ca+        .quad   0xb8714dcb38d9467d+        .quad   0x55de888283f95fa8+        .quad   0x3d3bdc164dfa63f7+        .quad   0xa5bb1dab78cfaa98+        .quad   0x5ceda267190b72f2+        .quad   0x9309c9110a92608e+        .quad   0x0119a3042fb374b0+        .quad   0x67a2d89ce8c2177d+        .quad   0x669da5f66895d0c1+        .quad   0xf56598e5b282a2b0+        .quad   0x56c088f1ede20a73++        // 2^208 * 7 * G++        .quad   0x336d3d1110a86e17+        .quad   0xd7f388320b75b2fa+        .quad   0xf915337625072988+        .quad   0x09674c6b99108b87+        .quad   0x581b5fac24f38f02+        .quad   0xa90be9febae30cbd+        .quad   0x9a2169028acf92f0+        .quad   0x038b7ea48359038f+        .quad   0x9f4ef82199316ff8+        .quad   0x2f49d282eaa78d4f+        .quad   0x0971a5ab5aef3174+        .quad   0x6e5e31025969eb65++        // 2^208 * 8 * G++        .quad   0xb16c62f587e593fb+        .quad   0x4999eddeca5d3e71+        .quad   0xb491c1e014cc3e6d+        .quad   0x08f5114789a8dba8+        .quad   0x3304fb0e63066222+        .quad   0xfb35068987acba3f+        .quad   0xbd1924778c1061a3+        .quad   0x3058ad43d1838620+        .quad   0x323c0ffde57663d0+        .quad   0x05c3df38a22ea610+        .quad   0xbdc78abdac994f9a+        .quad   0x26549fa4efe3dc99++        // 2^212 * 1 * G++        .quad   0x738b38d787ce8f89+        .quad   0xb62658e24179a88d+        .quad   0x30738c9cf151316d+        .quad   0x49128c7f727275c9+        .quad   0x04dbbc17f75396b9+        .quad   0x69e6a2d7d2f86746+        .quad   0xc6409d99f53eabc6+        .quad   0x606175f6332e25d2+        .quad   0x4021370ef540e7dd+        .quad   0x0910d6f5a1f1d0a5+        .quad   0x4634aacd5b06b807+        .quad   0x6a39e6356944f235++        // 2^212 * 2 * G++        .quad   0x96cd5640df90f3e7+        .quad   0x6c3a760edbfa25ea+        .quad   0x24f3ef0959e33cc4+        .quad   0x42889e7e530d2e58+        .quad   0x1da1965774049e9d+        .quad   0xfbcd6ea198fe352b+        .quad   0xb1cbcd50cc5236a6+        .quad   0x1f5ec83d3f9846e2+        .quad   0x8efb23c3328ccb75+        .quad   0xaf42a207dd876ee9+        .quad   0x20fbdadc5dfae796+        .quad   0x241e246b06bf9f51++        // 2^212 * 3 * G++        .quad   0x29e68e57ad6e98f6+        .quad   0x4c9260c80b462065+        .quad   0x3f00862ea51ebb4b+        .quad   0x5bc2c77fb38d9097+        .quad   0x7eaafc9a6280bbb8+        .quad   0x22a70f12f403d809+        .quad   0x31ce40bb1bfc8d20+        .quad   0x2bc65635e8bd53ee+        .quad   0xe8d5dc9fa96bad93+        .quad   0xe58fb17dde1947dc+        .quad   0x681532ea65185fa3+        .quad   0x1fdd6c3b034a7830++        // 2^212 * 4 * G++        .quad   0x0a64e28c55dc18fe+        .quad   0xe3df9e993399ebdd+        .quad   0x79ac432370e2e652+        .quad   0x35ff7fc33ae4cc0e+        .quad   0x9c13a6a52dd8f7a9+        .quad   0x2dbb1f8c3efdcabf+        .quad   0x961e32405e08f7b5+        .quad   0x48c8a121bbe6c9e5+        .quad   0xfc415a7c59646445+        .quad   0xd224b2d7c128b615+        .quad   0x6035c9c905fbb912+        .quad   0x42d7a91274429fab++        // 2^212 * 5 * G++        .quad   0x4e6213e3eaf72ed3+        .quad   0x6794981a43acd4e7+        .quad   0xff547cde6eb508cb+        .quad   0x6fed19dd10fcb532+        .quad   0xa9a48947933da5bc+        .quad   0x4a58920ec2e979ec+        .quad   0x96d8800013e5ac4c+        .quad   0x453692d74b48b147+        .quad   0xdd775d99a8559c6f+        .quad   0xf42a2140df003e24+        .quad   0x5223e229da928a66+        .quad   0x063f46ba6d38f22c++        // 2^212 * 6 * G++        .quad   0xd2d242895f536694+        .quad   0xca33a2c542939b2c+        .quad   0x986fada6c7ddb95c+        .quad   0x5a152c042f712d5d+        .quad   0x39843cb737346921+        .quad   0xa747fb0738c89447+        .quad   0xcb8d8031a245307e+        .quad   0x67810f8e6d82f068+        .quad   0x3eeb8fbcd2287db4+        .quad   0x72c7d3a301a03e93+        .quad   0x5473e88cbd98265a+        .quad   0x7324aa515921b403++        // 2^212 * 7 * G++        .quad   0x857942f46c3cbe8e+        .quad   0xa1d364b14730c046+        .quad   0x1c8ed914d23c41bf+        .quad   0x0838e161eef6d5d2+        .quad   0xad23f6dae82354cb+        .quad   0x6962502ab6571a6d+        .quad   0x9b651636e38e37d1+        .quad   0x5cac5005d1a3312f+        .quad   0x8cc154cce9e39904+        .quad   0x5b3a040b84de6846+        .quad   0xc4d8a61cb1be5d6e+        .quad   0x40fb897bd8861f02++        // 2^212 * 8 * G++        .quad   0x84c5aa9062de37a1+        .quad   0x421da5000d1d96e1+        .quad   0x788286306a9242d9+        .quad   0x3c5e464a690d10da+        .quad   0xe57ed8475ab10761+        .quad   0x71435e206fd13746+        .quad   0x342f824ecd025632+        .quad   0x4b16281ea8791e7b+        .quad   0xd1c101d50b813381+        .quad   0xdee60f1176ee6828+        .quad   0x0cb68893383f6409+        .quad   0x6183c565f6ff484a++        // 2^216 * 1 * G++        .quad   0x741d5a461e6bf9d6+        .quad   0x2305b3fc7777a581+        .quad   0xd45574a26474d3d9+        .quad   0x1926e1dc6401e0ff+        .quad   0xdb468549af3f666e+        .quad   0xd77fcf04f14a0ea5+        .quad   0x3df23ff7a4ba0c47+        .quad   0x3a10dfe132ce3c85+        .quad   0xe07f4e8aea17cea0+        .quad   0x2fd515463a1fc1fd+        .quad   0x175322fd31f2c0f1+        .quad   0x1fa1d01d861e5d15++        // 2^216 * 2 * G++        .quad   0xcc8055947d599832+        .quad   0x1e4656da37f15520+        .quad   0x99f6f7744e059320+        .quad   0x773563bc6a75cf33+        .quad   0x38dcac00d1df94ab+        .quad   0x2e712bddd1080de9+        .quad   0x7f13e93efdd5e262+        .quad   0x73fced18ee9a01e5+        .quad   0x06b1e90863139cb3+        .quad   0xa493da67c5a03ecd+        .quad   0x8d77cec8ad638932+        .quad   0x1f426b701b864f44++        // 2^216 * 3 * G++        .quad   0xefc9264c41911c01+        .quad   0xf1a3b7b817a22c25+        .quad   0x5875da6bf30f1447+        .quad   0x4e1af5271d31b090+        .quad   0xf17e35c891a12552+        .quad   0xb76b8153575e9c76+        .quad   0xfa83406f0d9b723e+        .quad   0x0b76bb1b3fa7e438+        .quad   0x08b8c1f97f92939b+        .quad   0xbe6771cbd444ab6e+        .quad   0x22e5646399bb8017+        .quad   0x7b6dd61eb772a955++        // 2^216 * 4 * G++        .quad   0xb7adc1e850f33d92+        .quad   0x7998fa4f608cd5cf+        .quad   0xad962dbd8dfc5bdb+        .quad   0x703e9bceaf1d2f4f+        .quad   0x5730abf9ab01d2c7+        .quad   0x16fb76dc40143b18+        .quad   0x866cbe65a0cbb281+        .quad   0x53fa9b659bff6afe+        .quad   0x6c14c8e994885455+        .quad   0x843a5d6665aed4e5+        .quad   0x181bb73ebcd65af1+        .quad   0x398d93e5c4c61f50++        // 2^216 * 5 * G++        .quad   0x1c4bd16733e248f3+        .quad   0xbd9e128715bf0a5f+        .quad   0xd43f8cf0a10b0376+        .quad   0x53b09b5ddf191b13+        .quad   0xc3877c60d2e7e3f2+        .quad   0x3b34aaa030828bb1+        .quad   0x283e26e7739ef138+        .quad   0x699c9c9002c30577+        .quad   0xf306a7235946f1cc+        .quad   0x921718b5cce5d97d+        .quad   0x28cdd24781b4e975+        .quad   0x51caf30c6fcdd907++        // 2^216 * 6 * G++        .quad   0xa60ba7427674e00a+        .quad   0x630e8570a17a7bf3+        .quad   0x3758563dcf3324cc+        .quad   0x5504aa292383fdaa+        .quad   0x737af99a18ac54c7+        .quad   0x903378dcc51cb30f+        .quad   0x2b89bc334ce10cc7+        .quad   0x12ae29c189f8e99a+        .quad   0xa99ec0cb1f0d01cf+        .quad   0x0dd1efcc3a34f7ae+        .quad   0x55ca7521d09c4e22+        .quad   0x5fd14fe958eba5ea++        // 2^216 * 7 * G++        .quad   0xb5dc2ddf2845ab2c+        .quad   0x069491b10a7fe993+        .quad   0x4daaf3d64002e346+        .quad   0x093ff26e586474d1+        .quad   0x3c42fe5ebf93cb8e+        .quad   0xbedfa85136d4565f+        .quad   0xe0f0859e884220e8+        .quad   0x7dd73f960725d128+        .quad   0xb10d24fe68059829+        .quad   0x75730672dbaf23e5+        .quad   0x1367253ab457ac29+        .quad   0x2f59bcbc86b470a4++        // 2^216 * 8 * G++        .quad   0x83847d429917135f+        .quad   0xad1b911f567d03d7+        .quad   0x7e7748d9be77aad1+        .quad   0x5458b42e2e51af4a+        .quad   0x7041d560b691c301+        .quad   0x85201b3fadd7e71e+        .quad   0x16c2e16311335585+        .quad   0x2aa55e3d010828b1+        .quad   0xed5192e60c07444f+        .quad   0x42c54e2d74421d10+        .quad   0x352b4c82fdb5c864+        .quad   0x13e9004a8a768664++        // 2^220 * 1 * G++        .quad   0xcbb5b5556c032bff+        .quad   0xdf7191b729297a3a+        .quad   0xc1ff7326aded81bb+        .quad   0x71ade8bb68be03f5+        .quad   0x1e6284c5806b467c+        .quad   0xc5f6997be75d607b+        .quad   0x8b67d958b378d262+        .quad   0x3d88d66a81cd8b70+        .quad   0x8b767a93204ed789+        .quad   0x762fcacb9fa0ae2a+        .quad   0x771febcc6dce4887+        .quad   0x343062158ff05fb3++        // 2^220 * 2 * G++        .quad   0xe05da1a7e1f5bf49+        .quad   0x26457d6dd4736092+        .quad   0x77dcb07773cc32f6+        .quad   0x0a5d94969cdd5fcd+        .quad   0xfce219072a7b31b4+        .quad   0x4d7adc75aa578016+        .quad   0x0ec276a687479324+        .quad   0x6d6d9d5d1fda4beb+        .quad   0x22b1a58ae9b08183+        .quad   0xfd95d071c15c388b+        .quad   0xa9812376850a0517+        .quad   0x33384cbabb7f335e++        // 2^220 * 3 * G++        .quad   0x3c6fa2680ca2c7b5+        .quad   0x1b5082046fb64fda+        .quad   0xeb53349c5431d6de+        .quad   0x5278b38f6b879c89+        .quad   0x33bc627a26218b8d+        .quad   0xea80b21fc7a80c61+        .quad   0x9458b12b173e9ee6+        .quad   0x076247be0e2f3059+        .quad   0x52e105f61416375a+        .quad   0xec97af3685abeba4+        .quad   0x26e6b50623a67c36+        .quad   0x5cf0e856f3d4fb01++        // 2^220 * 4 * G++        .quad   0xf6c968731ae8cab4+        .quad   0x5e20741ecb4f92c5+        .quad   0x2da53be58ccdbc3e+        .quad   0x2dddfea269970df7+        .quad   0xbeaece313db342a8+        .quad   0xcba3635b842db7ee+        .quad   0xe88c6620817f13ef+        .quad   0x1b9438aa4e76d5c6+        .quad   0x8a50777e166f031a+        .quad   0x067b39f10fb7a328+        .quad   0x1925c9a6010fbd76+        .quad   0x6df9b575cc740905++        // 2^220 * 5 * G++        .quad   0x42c1192927f6bdcf+        .quad   0x8f91917a403d61ca+        .quad   0xdc1c5a668b9e1f61+        .quad   0x1596047804ec0f8d+        .quad   0xecdfc35b48cade41+        .quad   0x6a88471fb2328270+        .quad   0x740a4a2440a01b6a+        .quad   0x471e5796003b5f29+        .quad   0xda96bbb3aced37ac+        .quad   0x7a2423b5e9208cea+        .quad   0x24cc5c3038aebae2+        .quad   0x50c356afdc5dae2f++        // 2^220 * 6 * G++        .quad   0x09dcbf4341c30318+        .quad   0xeeba061183181dce+        .quad   0xc179c0cedc1e29a1+        .quad   0x1dbf7b89073f35b0+        .quad   0xcfed9cdf1b31b964+        .quad   0xf486a9858ca51af3+        .quad   0x14897265ea8c1f84+        .quad   0x784a53dd932acc00+        .quad   0x2d99f9df14fc4920+        .quad   0x76ccb60cc4499fe5+        .quad   0xa4132cbbe5cf0003+        .quad   0x3f93d82354f000ea++        // 2^220 * 7 * G++        .quad   0x8183e7689e04ce85+        .quad   0x678fb71e04465341+        .quad   0xad92058f6688edac+        .quad   0x5da350d3532b099a+        .quad   0xeaac12d179e14978+        .quad   0xff923ff3bbebff5e+        .quad   0x4af663e40663ce27+        .quad   0x0fd381a811a5f5ff+        .quad   0xf256aceca436df54+        .quad   0x108b6168ae69d6e8+        .quad   0x20d986cb6b5d036c+        .quad   0x655957b9fee2af50++        // 2^220 * 8 * G++        .quad   0xaea8b07fa902030f+        .quad   0xf88c766af463d143+        .quad   0x15b083663c787a60+        .quad   0x08eab1148267a4a8+        .quad   0xbdc1409bd002d0ac+        .quad   0x66660245b5ccd9a6+        .quad   0x82317dc4fade85ec+        .quad   0x02fe934b6ad7df0d+        .quad   0xef5cf100cfb7ea74+        .quad   0x22897633a1cb42ac+        .quad   0xd4ce0c54cef285e2+        .quad   0x30408c048a146a55++        // 2^224 * 1 * G++        .quad   0x739d8845832fcedb+        .quad   0xfa38d6c9ae6bf863+        .quad   0x32bc0dcab74ffef7+        .quad   0x73937e8814bce45e+        .quad   0xbb2e00c9193b877f+        .quad   0xece3a890e0dc506b+        .quad   0xecf3b7c036de649f+        .quad   0x5f46040898de9e1a+        .quad   0xb9037116297bf48d+        .quad   0xa9d13b22d4f06834+        .quad   0xe19715574696bdc6+        .quad   0x2cf8a4e891d5e835++        // 2^224 * 2 * G++        .quad   0x6d93fd8707110f67+        .quad   0xdd4c09d37c38b549+        .quad   0x7cb16a4cc2736a86+        .quad   0x2049bd6e58252a09+        .quad   0x2cb5487e17d06ba2+        .quad   0x24d2381c3950196b+        .quad   0xd7659c8185978a30+        .quad   0x7a6f7f2891d6a4f6+        .quad   0x7d09fd8d6a9aef49+        .quad   0xf0ee60be5b3db90b+        .quad   0x4c21b52c519ebfd4+        .quad   0x6011aadfc545941d++        // 2^224 * 3 * G++        .quad   0x5f67926dcf95f83c+        .quad   0x7c7e856171289071+        .quad   0xd6a1e7f3998f7a5b+        .quad   0x6fc5cc1b0b62f9e0+        .quad   0x63ded0c802cbf890+        .quad   0xfbd098ca0dff6aaa+        .quad   0x624d0afdb9b6ed99+        .quad   0x69ce18b779340b1e+        .quad   0xd1ef5528b29879cb+        .quad   0xdd1aae3cd47e9092+        .quad   0x127e0442189f2352+        .quad   0x15596b3ae57101f1++        // 2^224 * 4 * G++        .quad   0x462739d23f9179a2+        .quad   0xff83123197d6ddcf+        .quad   0x1307deb553f2148a+        .quad   0x0d2237687b5f4dda+        .quad   0x09ff31167e5124ca+        .quad   0x0be4158bd9c745df+        .quad   0x292b7d227ef556e5+        .quad   0x3aa4e241afb6d138+        .quad   0x2cc138bf2a3305f5+        .quad   0x48583f8fa2e926c3+        .quad   0x083ab1a25549d2eb+        .quad   0x32fcaa6e4687a36c++        // 2^224 * 5 * G++        .quad   0x7bc56e8dc57d9af5+        .quad   0x3e0bd2ed9df0bdf2+        .quad   0xaac014de22efe4a3+        .quad   0x4627e9cefebd6a5c+        .quad   0x3207a4732787ccdf+        .quad   0x17e31908f213e3f8+        .quad   0xd5b2ecd7f60d964e+        .quad   0x746f6336c2600be9+        .quad   0x3f4af345ab6c971c+        .quad   0xe288eb729943731f+        .quad   0x33596a8a0344186d+        .quad   0x7b4917007ed66293++        // 2^224 * 6 * G++        .quad   0x2d85fb5cab84b064+        .quad   0x497810d289f3bc14+        .quad   0x476adc447b15ce0c+        .quad   0x122ba376f844fd7b+        .quad   0x54341b28dd53a2dd+        .quad   0xaa17905bdf42fc3f+        .quad   0x0ff592d94dd2f8f4+        .quad   0x1d03620fe08cd37d+        .quad   0xc20232cda2b4e554+        .quad   0x9ed0fd42115d187f+        .quad   0x2eabb4be7dd479d9+        .quad   0x02c70bf52b68ec4c++        // 2^224 * 7 * G++        .quad   0xa287ec4b5d0b2fbb+        .quad   0x415c5790074882ca+        .quad   0xe044a61ec1d0815c+        .quad   0x26334f0a409ef5e0+        .quad   0xace532bf458d72e1+        .quad   0x5be768e07cb73cb5+        .quad   0x56cf7d94ee8bbde7+        .quad   0x6b0697e3feb43a03+        .quad   0xb6c8f04adf62a3c0+        .quad   0x3ef000ef076da45d+        .quad   0x9c9cb95849f0d2a9+        .quad   0x1cc37f43441b2fae++        // 2^224 * 8 * G++        .quad   0x508f565a5cc7324f+        .quad   0xd061c4c0e506a922+        .quad   0xfb18abdb5c45ac19+        .quad   0x6c6809c10380314a+        .quad   0xd76656f1c9ceaeb9+        .quad   0x1c5b15f818e5656a+        .quad   0x26e72832844c2334+        .quad   0x3a346f772f196838+        .quad   0xd2d55112e2da6ac8+        .quad   0xe9bd0331b1e851ed+        .quad   0x960746dd8ec67262+        .quad   0x05911b9f6ef7c5d0++        // 2^228 * 1 * G++        .quad   0xe9dcd756b637ff2d+        .quad   0xec4c348fc987f0c4+        .quad   0xced59285f3fbc7b7+        .quad   0x3305354793e1ea87+        .quad   0x01c18980c5fe9f94+        .quad   0xcd656769716fd5c8+        .quad   0x816045c3d195a086+        .quad   0x6e2b7f3266cc7982+        .quad   0xcc802468f7c3568f+        .quad   0x9de9ba8219974cb3+        .quad   0xabb7229cb5b81360+        .quad   0x44e2017a6fbeba62++        // 2^228 * 2 * G++        .quad   0xc4c2a74354dab774+        .quad   0x8e5d4c3c4eaf031a+        .quad   0xb76c23d242838f17+        .quad   0x749a098f68dce4ea+        .quad   0x87f82cf3b6ca6ecd+        .quad   0x580f893e18f4a0c2+        .quad   0x058930072604e557+        .quad   0x6cab6ac256d19c1d+        .quad   0xdcdfe0a02cc1de60+        .quad   0x032665ff51c5575b+        .quad   0x2c0c32f1073abeeb+        .quad   0x6a882014cd7b8606++        // 2^228 * 3 * G++        .quad   0xa52a92fea4747fb5+        .quad   0xdc12a4491fa5ab89+        .quad   0xd82da94bb847a4ce+        .quad   0x4d77edce9512cc4e+        .quad   0xd111d17caf4feb6e+        .quad   0x050bba42b33aa4a3+        .quad   0x17514c3ceeb46c30+        .quad   0x54bedb8b1bc27d75+        .quad   0x77c8e14577e2189c+        .quad   0xa3e46f6aff99c445+        .quad   0x3144dfc86d335343+        .quad   0x3a96559e7c4216a9++        // 2^228 * 4 * G++        .quad   0x12550d37f42ad2ee+        .quad   0x8b78e00498a1fbf5+        .quad   0x5d53078233894cb2+        .quad   0x02c84e4e3e498d0c+        .quad   0x4493896880baaa52+        .quad   0x4c98afc4f285940e+        .quad   0xef4aa79ba45448b6+        .quad   0x5278c510a57aae7f+        .quad   0xa54dd074294c0b94+        .quad   0xf55d46b8df18ffb6+        .quad   0xf06fecc58dae8366+        .quad   0x588657668190d165++        // 2^228 * 5 * G++        .quad   0xd47712311aef7117+        .quad   0x50343101229e92c7+        .quad   0x7a95e1849d159b97+        .quad   0x2449959b8b5d29c9+        .quad   0xbf5834f03de25cc3+        .quad   0xb887c8aed6815496+        .quad   0x5105221a9481e892+        .quad   0x6760ed19f7723f93+        .quad   0x669ba3b7ac35e160+        .quad   0x2eccf73fba842056+        .quad   0x1aec1f17c0804f07+        .quad   0x0d96bc031856f4e7++        // 2^228 * 6 * G++        .quad   0x3318be7775c52d82+        .quad   0x4cb764b554d0aab9+        .quad   0xabcf3d27cc773d91+        .quad   0x3bf4d1848123288a+        .quad   0xb1d534b0cc7505e1+        .quad   0x32cd003416c35288+        .quad   0xcb36a5800762c29d+        .quad   0x5bfe69b9237a0bf8+        .quad   0x183eab7e78a151ab+        .quad   0xbbe990c999093763+        .quad   0xff717d6e4ac7e335+        .quad   0x4c5cddb325f39f88++        // 2^228 * 7 * G++        .quad   0xc0f6b74d6190a6eb+        .quad   0x20ea81a42db8f4e4+        .quad   0xa8bd6f7d97315760+        .quad   0x33b1d60262ac7c21+        .quad   0x57750967e7a9f902+        .quad   0x2c37fdfc4f5b467e+        .quad   0xb261663a3177ba46+        .quad   0x3a375e78dc2d532b+        .quad   0x8141e72f2d4dddea+        .quad   0xe6eafe9862c607c8+        .quad   0x23c28458573cafd0+        .quad   0x46b9476f4ff97346++        // 2^228 * 8 * G++        .quad   0x0c1ffea44f901e5c+        .quad   0x2b0b6fb72184b782+        .quad   0xe587ff910114db88+        .quad   0x37130f364785a142+        .quad   0x1215505c0d58359f+        .quad   0x2a2013c7fc28c46b+        .quad   0x24a0a1af89ea664e+        .quad   0x4400b638a1130e1f+        .quad   0x3a01b76496ed19c3+        .quad   0x31e00ab0ed327230+        .quad   0x520a885783ca15b1+        .quad   0x06aab9875accbec7++        // 2^232 * 1 * G++        .quad   0xc1339983f5df0ebb+        .quad   0xc0f3758f512c4cac+        .quad   0x2cf1130a0bb398e1+        .quad   0x6b3cecf9aa270c62+        .quad   0x5349acf3512eeaef+        .quad   0x20c141d31cc1cb49+        .quad   0x24180c07a99a688d+        .quad   0x555ef9d1c64b2d17+        .quad   0x36a770ba3b73bd08+        .quad   0x624aef08a3afbf0c+        .quad   0x5737ff98b40946f2+        .quad   0x675f4de13381749d++        // 2^232 * 2 * G++        .quad   0x0e2c52036b1782fc+        .quad   0x64816c816cad83b4+        .quad   0xd0dcbdd96964073e+        .quad   0x13d99df70164c520+        .quad   0xa12ff6d93bdab31d+        .quad   0x0725d80f9d652dfe+        .quad   0x019c4ff39abe9487+        .quad   0x60f450b882cd3c43+        .quad   0x014b5ec321e5c0ca+        .quad   0x4fcb69c9d719bfa2+        .quad   0x4e5f1c18750023a0+        .quad   0x1c06de9e55edac80++        // 2^232 * 3 * G++        .quad   0x990f7ad6a33ec4e2+        .quad   0x6608f938be2ee08e+        .quad   0x9ca143c563284515+        .quad   0x4cf38a1fec2db60d+        .quad   0xffd52b40ff6d69aa+        .quad   0x34530b18dc4049bb+        .quad   0x5e4a5c2fa34d9897+        .quad   0x78096f8e7d32ba2d+        .quad   0xa0aaaa650dfa5ce7+        .quad   0xf9c49e2a48b5478c+        .quad   0x4f09cc7d7003725b+        .quad   0x373cad3a26091abe++        // 2^232 * 4 * G++        .quad   0xb294634d82c9f57c+        .quad   0x1fcbfde124934536+        .quad   0x9e9c4db3418cdb5a+        .quad   0x0040f3d9454419fc+        .quad   0xf1bea8fb89ddbbad+        .quad   0x3bcb2cbc61aeaecb+        .quad   0x8f58a7bb1f9b8d9d+        .quad   0x21547eda5112a686+        .quad   0xdefde939fd5986d3+        .quad   0xf4272c89510a380c+        .quad   0xb72ba407bb3119b9+        .quad   0x63550a334a254df4++        // 2^232 * 5 * G++        .quad   0x6507d6edb569cf37+        .quad   0x178429b00ca52ee1+        .quad   0xea7c0090eb6bd65d+        .quad   0x3eea62c7daf78f51+        .quad   0x9bba584572547b49+        .quad   0xf305c6fae2c408e0+        .quad   0x60e8fa69c734f18d+        .quad   0x39a92bafaa7d767a+        .quad   0x9d24c713e693274e+        .quad   0x5f63857768dbd375+        .quad   0x70525560eb8ab39a+        .quad   0x68436a0665c9c4cd++        // 2^232 * 6 * G++        .quad   0xbc0235e8202f3f27+        .quad   0xc75c00e264f975b0+        .quad   0x91a4e9d5a38c2416+        .quad   0x17b6e7f68ab789f9+        .quad   0x1e56d317e820107c+        .quad   0xc5266844840ae965+        .quad   0xc1e0a1c6320ffc7a+        .quad   0x5373669c91611472+        .quad   0x5d2814ab9a0e5257+        .quad   0x908f2084c9cab3fc+        .quad   0xafcaf5885b2d1eca+        .quad   0x1cb4b5a678f87d11++        // 2^232 * 7 * G++        .quad   0xb664c06b394afc6c+        .quad   0x0c88de2498da5fb1+        .quad   0x4f8d03164bcad834+        .quad   0x330bca78de7434a2+        .quad   0x6b74aa62a2a007e7+        .quad   0xf311e0b0f071c7b1+        .quad   0x5707e438000be223+        .quad   0x2dc0fd2d82ef6eac+        .quad   0x982eff841119744e+        .quad   0xf9695e962b074724+        .quad   0xc58ac14fbfc953fb+        .quad   0x3c31be1b369f1cf5++        // 2^232 * 8 * G++        .quad   0xb0f4864d08948aee+        .quad   0x07dc19ee91ba1c6f+        .quad   0x7975cdaea6aca158+        .quad   0x330b61134262d4bb+        .quad   0xc168bc93f9cb4272+        .quad   0xaeb8711fc7cedb98+        .quad   0x7f0e52aa34ac8d7a+        .quad   0x41cec1097e7d55bb+        .quad   0xf79619d7a26d808a+        .quad   0xbb1fd49e1d9e156d+        .quad   0x73d7c36cdba1df27+        .quad   0x26b44cd91f28777d++        // 2^236 * 1 * G++        .quad   0x300a9035393aa6d8+        .quad   0x2b501131a12bb1cd+        .quad   0x7b1ff677f093c222+        .quad   0x4309c1f8cab82bad+        .quad   0xaf44842db0285f37+        .quad   0x8753189047efc8df+        .quad   0x9574e091f820979a+        .quad   0x0e378d6069615579+        .quad   0xd9fa917183075a55+        .quad   0x4bdb5ad26b009fdc+        .quad   0x7829ad2cd63def0e+        .quad   0x078fc54975fd3877++        // 2^236 * 2 * G++        .quad   0x87dfbd1428878f2d+        .quad   0x134636dd1e9421a1+        .quad   0x4f17c951257341a3+        .quad   0x5df98d4bad296cb8+        .quad   0xe2004b5bb833a98a+        .quad   0x44775dec2d4c3330+        .quad   0x3aa244067eace913+        .quad   0x272630e3d58e00a9+        .quad   0xf3678fd0ecc90b54+        .quad   0xf001459b12043599+        .quad   0x26725fbc3758b89b+        .quad   0x4325e4aa73a719ae++        // 2^236 * 3 * G++        .quad   0x657dc6ef433c3493+        .quad   0x65375e9f80dbf8c3+        .quad   0x47fd2d465b372dae+        .quad   0x4966ab79796e7947+        .quad   0xed24629acf69f59d+        .quad   0x2a4a1ccedd5abbf4+        .quad   0x3535ca1f56b2d67b+        .quad   0x5d8c68d043b1b42d+        .quad   0xee332d4de3b42b0a+        .quad   0xd84e5a2b16a4601c+        .quad   0x78243877078ba3e4+        .quad   0x77ed1eb4184ee437++        // 2^236 * 4 * G++        .quad   0xbfd4e13f201839a0+        .quad   0xaeefffe23e3df161+        .quad   0xb65b04f06b5d1fe3+        .quad   0x52e085fb2b62fbc0+        .quad   0x185d43f89e92ed1a+        .quad   0xb04a1eeafe4719c6+        .quad   0x499fbe88a6f03f4f+        .quad   0x5d8b0d2f3c859bdd+        .quad   0x124079eaa54cf2ba+        .quad   0xd72465eb001b26e7+        .quad   0x6843bcfdc97af7fd+        .quad   0x0524b42b55eacd02++        // 2^236 * 5 * G++        .quad   0xfd0d5dbee45447b0+        .quad   0x6cec351a092005ee+        .quad   0x99a47844567579cb+        .quad   0x59d242a216e7fa45+        .quad   0xbc18dcad9b829eac+        .quad   0x23ae7d28b5f579d0+        .quad   0xc346122a69384233+        .quad   0x1a6110b2e7d4ac89+        .quad   0x4f833f6ae66997ac+        .quad   0x6849762a361839a4+        .quad   0x6985dec1970ab525+        .quad   0x53045e89dcb1f546++        // 2^236 * 6 * G++        .quad   0xcb8bb346d75353db+        .quad   0xfcfcb24bae511e22+        .quad   0xcba48d40d50ae6ef+        .quad   0x26e3bae5f4f7cb5d+        .quad   0x84da3cde8d45fe12+        .quad   0xbd42c218e444e2d2+        .quad   0xa85196781f7e3598+        .quad   0x7642c93f5616e2b2+        .quad   0x2323daa74595f8e4+        .quad   0xde688c8b857abeb4+        .quad   0x3fc48e961c59326e+        .quad   0x0b2e73ca15c9b8ba++        // 2^236 * 7 * G++        .quad   0xd6bb4428c17f5026+        .quad   0x9eb27223fb5a9ca7+        .quad   0xe37ba5031919c644+        .quad   0x21ce380db59a6602+        .quad   0x0e3fbfaf79c03a55+        .quad   0x3077af054cbb5acf+        .quad   0xd5c55245db3de39f+        .quad   0x015e68c1476a4af7+        .quad   0xc1d5285220066a38+        .quad   0x95603e523570aef3+        .quad   0x832659a7226b8a4d+        .quad   0x5dd689091f8eedc9++        // 2^236 * 8 * G++        .quad   0xcbac84debfd3c856+        .quad   0x1624c348b35ff244+        .quad   0xb7f88dca5d9cad07+        .quad   0x3b0e574da2c2ebe8+        .quad   0x1d022591a5313084+        .quad   0xca2d4aaed6270872+        .quad   0x86a12b852f0bfd20+        .quad   0x56e6c439ad7da748+        .quad   0xc704ff4942bdbae6+        .quad   0x5e21ade2b2de1f79+        .quad   0xe95db3f35652fad8+        .quad   0x0822b5378f08ebc1++        // 2^240 * 1 * G++        .quad   0x51f048478f387475+        .quad   0xb25dbcf49cbecb3c+        .quad   0x9aab1244d99f2055+        .quad   0x2c709e6c1c10a5d6+        .quad   0xe1b7f29362730383+        .quad   0x4b5279ffebca8a2c+        .quad   0xdafc778abfd41314+        .quad   0x7deb10149c72610f+        .quad   0xcb62af6a8766ee7a+        .quad   0x66cbec045553cd0e+        .quad   0x588001380f0be4b5+        .quad   0x08e68e9ff62ce2ea++        // 2^240 * 2 * G++        .quad   0x34ad500a4bc130ad+        .quad   0x8d38db493d0bd49c+        .quad   0xa25c3d98500a89be+        .quad   0x2f1f3f87eeba3b09+        .quad   0x2f2d09d50ab8f2f9+        .quad   0xacb9218dc55923df+        .quad   0x4a8f342673766cb9+        .quad   0x4cb13bd738f719f5+        .quad   0xf7848c75e515b64a+        .quad   0xa59501badb4a9038+        .quad   0xc20d313f3f751b50+        .quad   0x19a1e353c0ae2ee8++        // 2^240 * 3 * G++        .quad   0x7d1c7560bafa05c3+        .quad   0xb3e1a0a0c6e55e61+        .quad   0xe3529718c0d66473+        .quad   0x41546b11c20c3486+        .quad   0xb42172cdd596bdbd+        .quad   0x93e0454398eefc40+        .quad   0x9fb15347b44109b5+        .quad   0x736bd3990266ae34+        .quad   0x85532d509334b3b4+        .quad   0x46fd114b60816573+        .quad   0xcc5f5f30425c8375+        .quad   0x412295a2b87fab5c++        // 2^240 * 4 * G++        .quad   0x19c99b88f57ed6e9+        .quad   0x5393cb266df8c825+        .quad   0x5cee3213b30ad273+        .quad   0x14e153ebb52d2e34+        .quad   0x2e655261e293eac6+        .quad   0x845a92032133acdb+        .quad   0x460975cb7900996b+        .quad   0x0760bb8d195add80+        .quad   0x413e1a17cde6818a+        .quad   0x57156da9ed69a084+        .quad   0x2cbf268f46caccb1+        .quad   0x6b34be9bc33ac5f2++        // 2^240 * 5 * G++        .quad   0xf3df2f643a78c0b2+        .quad   0x4c3e971ef22e027c+        .quad   0xec7d1c5e49c1b5a3+        .quad   0x2012c18f0922dd2d+        .quad   0x11fc69656571f2d3+        .quad   0xc6c9e845530e737a+        .quad   0xe33ae7a2d4fe5035+        .quad   0x01b9c7b62e6dd30b+        .quad   0x880b55e55ac89d29+        .quad   0x1483241f45a0a763+        .quad   0x3d36efdfc2e76c1f+        .quad   0x08af5b784e4bade8++        // 2^240 * 6 * G++        .quad   0x283499dc881f2533+        .quad   0x9d0525da779323b6+        .quad   0x897addfb673441f4+        .quad   0x32b79d71163a168d+        .quad   0xe27314d289cc2c4b+        .quad   0x4be4bd11a287178d+        .quad   0x18d528d6fa3364ce+        .quad   0x6423c1d5afd9826e+        .quad   0xcc85f8d9edfcb36a+        .quad   0x22bcc28f3746e5f9+        .quad   0xe49de338f9e5d3cd+        .quad   0x480a5efbc13e2dcc++        // 2^240 * 7 * G++        .quad   0x0b51e70b01622071+        .quad   0x06b505cf8b1dafc5+        .quad   0x2c6bb061ef5aabcd+        .quad   0x47aa27600cb7bf31+        .quad   0xb6614ce442ce221f+        .quad   0x6e199dcc4c053928+        .quad   0x663fb4a4dc1cbe03+        .quad   0x24b31d47691c8e06+        .quad   0x2a541eedc015f8c3+        .quad   0x11a4fe7e7c693f7c+        .quad   0xf0af66134ea278d6+        .quad   0x545b585d14dda094++        // 2^240 * 8 * G++        .quad   0x67bf275ea0d43a0f+        .quad   0xade68e34089beebe+        .quad   0x4289134cd479e72e+        .quad   0x0f62f9c332ba5454+        .quad   0x6204e4d0e3b321e1+        .quad   0x3baa637a28ff1e95+        .quad   0x0b0ccffd5b99bd9e+        .quad   0x4d22dc3e64c8d071+        .quad   0xfcb46589d63b5f39+        .quad   0x5cae6a3f57cbcf61+        .quad   0xfebac2d2953afa05+        .quad   0x1c0fa01a36371436++        // 2^244 * 1 * G++        .quad   0xe7547449bc7cd692+        .quad   0x0f9abeaae6f73ddf+        .quad   0x4af01ca700837e29+        .quad   0x63ab1b5d3f1bc183+        .quad   0xc11ee5e854c53fae+        .quad   0x6a0b06c12b4f3ff4+        .quad   0x33540f80e0b67a72+        .quad   0x15f18fc3cd07e3ef+        .quad   0x32750763b028f48c+        .quad   0x06020740556a065f+        .quad   0xd53bd812c3495b58+        .quad   0x08706c9b865f508d++        // 2^244 * 2 * G++        .quad   0xf37ca2ab3d343dff+        .quad   0x1a8c6a2d80abc617+        .quad   0x8e49e035d4ccffca+        .quad   0x48b46beebaa1d1b9+        .quad   0xcc991b4138b41246+        .quad   0x243b9c526f9ac26b+        .quad   0xb9ef494db7cbabbd+        .quad   0x5fba433dd082ed00+        .quad   0x9c49e355c9941ad0+        .quad   0xb9734ade74498f84+        .quad   0x41c3fed066663e5c+        .quad   0x0ecfedf8e8e710b3++        // 2^244 * 3 * G++        .quad   0x76430f9f9cd470d9+        .quad   0xb62acc9ba42f6008+        .quad   0x1898297c59adad5e+        .quad   0x7789dd2db78c5080+        .quad   0x744f7463e9403762+        .quad   0xf79a8dee8dfcc9c9+        .quad   0x163a649655e4cde3+        .quad   0x3b61788db284f435+        .quad   0xb22228190d6ef6b2+        .quad   0xa94a66b246ce4bfa+        .quad   0x46c1a77a4f0b6cc7+        .quad   0x4236ccffeb7338cf++        // 2^244 * 4 * G++        .quad   0x8497404d0d55e274+        .quad   0x6c6663d9c4ad2b53+        .quad   0xec2fb0d9ada95734+        .quad   0x2617e120cdb8f73c+        .quad   0x3bd82dbfda777df6+        .quad   0x71b177cc0b98369e+        .quad   0x1d0e8463850c3699+        .quad   0x5a71945b48e2d1f1+        .quad   0x6f203dd5405b4b42+        .quad   0x327ec60410b24509+        .quad   0x9c347230ac2a8846+        .quad   0x77de29fc11ffeb6a++        // 2^244 * 5 * G++        .quad   0xb0ac57c983b778a8+        .quad   0x53cdcca9d7fe912c+        .quad   0x61c2b854ff1f59dc+        .quad   0x3a1a2cf0f0de7dac+        .quad   0x835e138fecced2ca+        .quad   0x8c9eaf13ea963b9a+        .quad   0xc95fbfc0b2160ea6+        .quad   0x575e66f3ad877892+        .quad   0x99803a27c88fcb3a+        .quad   0x345a6789275ec0b0+        .quad   0x459789d0ff6c2be5+        .quad   0x62f882651e70a8b2++        // 2^244 * 6 * G++        .quad   0x085ae2c759ff1be4+        .quad   0x149145c93b0e40b7+        .quad   0xc467e7fa7ff27379+        .quad   0x4eeecf0ad5c73a95+        .quad   0x6d822986698a19e0+        .quad   0xdc9821e174d78a71+        .quad   0x41a85f31f6cb1f47+        .quad   0x352721c2bcda9c51+        .quad   0x48329952213fc985+        .quad   0x1087cf0d368a1746+        .quad   0x8e5261b166c15aa5+        .quad   0x2d5b2d842ed24c21++        // 2^244 * 7 * G++        .quad   0x02cfebd9ebd3ded1+        .quad   0xd45b217739021974+        .quad   0x7576f813fe30a1b7+        .quad   0x5691b6f9a34ef6c2+        .quad   0x5eb7d13d196ac533+        .quad   0x377234ecdb80be2b+        .quad   0xe144cffc7cf5ae24+        .quad   0x5226bcf9c441acec+        .quad   0x79ee6c7223e5b547+        .quad   0x6f5f50768330d679+        .quad   0xed73e1e96d8adce9+        .quad   0x27c3da1e1d8ccc03++        // 2^244 * 8 * G++        .quad   0x7eb9efb23fe24c74+        .quad   0x3e50f49f1651be01+        .quad   0x3ea732dc21858dea+        .quad   0x17377bd75bb810f9+        .quad   0x28302e71630ef9f6+        .quad   0xc2d4a2032b64cee0+        .quad   0x090820304b6292be+        .quad   0x5fca747aa82adf18+        .quad   0x232a03c35c258ea5+        .quad   0x86f23a2c6bcb0cf1+        .quad   0x3dad8d0d2e442166+        .quad   0x04a8933cab76862b++        // 2^248 * 1 * G++        .quad   0xd2c604b622943dff+        .quad   0xbc8cbece44cfb3a0+        .quad   0x5d254ff397808678+        .quad   0x0fa3614f3b1ca6bf+        .quad   0x69082b0e8c936a50+        .quad   0xf9c9a035c1dac5b6+        .quad   0x6fb73e54c4dfb634+        .quad   0x4005419b1d2bc140+        .quad   0xa003febdb9be82f0+        .quad   0x2089c1af3a44ac90+        .quad   0xf8499f911954fa8e+        .quad   0x1fba218aef40ab42++        // 2^248 * 2 * G++        .quad   0xab549448fac8f53e+        .quad   0x81f6e89a7ba63741+        .quad   0x74fd6c7d6c2b5e01+        .quad   0x392e3acaa8c86e42+        .quad   0x4f3e57043e7b0194+        .quad   0xa81d3eee08daaf7f+        .quad   0xc839c6ab99dcdef1+        .quad   0x6c535d13ff7761d5+        .quad   0x4cbd34e93e8a35af+        .quad   0x2e0781445887e816+        .quad   0x19319c76f29ab0ab+        .quad   0x25e17fe4d50ac13b++        // 2^248 * 3 * G++        .quad   0x0a289bd71e04f676+        .quad   0x208e1c52d6420f95+        .quad   0x5186d8b034691fab+        .quad   0x255751442a9fb351+        .quad   0x915f7ff576f121a7+        .quad   0xc34a32272fcd87e3+        .quad   0xccba2fde4d1be526+        .quad   0x6bba828f8969899b+        .quad   0xe2d1bc6690fe3901+        .quad   0x4cb54a18a0997ad5+        .quad   0x971d6914af8460d4+        .quad   0x559d504f7f6b7be4++        // 2^248 * 4 * G++        .quad   0xa7738378b3eb54d5+        .quad   0x1d69d366a5553c7c+        .quad   0x0a26cf62f92800ba+        .quad   0x01ab12d5807e3217+        .quad   0x9c4891e7f6d266fd+        .quad   0x0744a19b0307781b+        .quad   0x88388f1d6061e23b+        .quad   0x123ea6a3354bd50e+        .quad   0x118d189041e32d96+        .quad   0xb9ede3c2d8315848+        .quad   0x1eab4271d83245d9+        .quad   0x4a3961e2c918a154++        // 2^248 * 5 * G++        .quad   0x71dc3be0f8e6bba0+        .quad   0xd6cef8347effe30a+        .quad   0xa992425fe13a476a+        .quad   0x2cd6bce3fb1db763+        .quad   0x0327d644f3233f1e+        .quad   0x499a260e34fcf016+        .quad   0x83b5a716f2dab979+        .quad   0x68aceead9bd4111f+        .quad   0x38b4c90ef3d7c210+        .quad   0x308e6e24b7ad040c+        .quad   0x3860d9f1b7e73e23+        .quad   0x595760d5b508f597++        // 2^248 * 6 * G++        .quad   0x6129bfe104aa6397+        .quad   0x8f960008a4a7fccb+        .quad   0x3f8bc0897d909458+        .quad   0x709fa43edcb291a9+        .quad   0x882acbebfd022790+        .quad   0x89af3305c4115760+        .quad   0x65f492e37d3473f4+        .quad   0x2cb2c5df54515a2b+        .quad   0xeb0a5d8c63fd2aca+        .quad   0xd22bc1662e694eff+        .quad   0x2723f36ef8cbb03a+        .quad   0x70f029ecf0c8131f++        // 2^248 * 7 * G++        .quad   0x461307b32eed3e33+        .quad   0xae042f33a45581e7+        .quad   0xc94449d3195f0366+        .quad   0x0b7d5d8a6c314858+        .quad   0x2a6aafaa5e10b0b9+        .quad   0x78f0a370ef041aa9+        .quad   0x773efb77aa3ad61f+        .quad   0x44eca5a2a74bd9e1+        .quad   0x25d448327b95d543+        .quad   0x70d38300a3340f1d+        .quad   0xde1c531c60e1c52b+        .quad   0x272224512c7de9e4++        // 2^248 * 8 * G++        .quad   0x1abc92af49c5342e+        .quad   0xffeed811b2e6fad0+        .quad   0xefa28c8dfcc84e29+        .quad   0x11b5df18a44cc543+        .quad   0xbf7bbb8a42a975fc+        .quad   0x8c5c397796ada358+        .quad   0xe27fc76fcdedaa48+        .quad   0x19735fd7f6bc20a6+        .quad   0xe3ab90d042c84266+        .quad   0xeb848e0f7f19547e+        .quad   0x2503a1d065a497b9+        .quad   0x0fef911191df895f
+ cbits/s2n/x86_att/edwards25519_scalarmulbase_alt.S view
@@ -0,0 +1,10002 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Scalar multiplication for the edwards25519 standard basepoint+// Input scalar[4]; output res[8]+//+// extern void edwards25519_scalarmulbase_alt+//   (uint64_t res[static 8],const uint64_t scalar[static 4]);+//+// Given a scalar n, returns point (X,Y) = n * B where B = (...,4/5) is+// the standard basepoint for the edwards25519 (Ed25519) curve.+//+// Standard x86-64 ABI: RDI = res, RSI = scalar+// Microsoft x64 ABI:   RCX = res, RDX = scalar+// ----------------------------------------------------------------------------+#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(edwards25519_scalarmulbase_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(edwards25519_scalarmulbase_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(edwards25519_scalarmulbase_alt)++        .text+        .balign 32++// Size of individual field elements++#define NUMSIZE 32++// Pointer-offset pairs for result and temporaries on stack with some aliasing.+// The result "resx" assumes the "res" pointer has been preloaded into %rbp.++#define resx (0*NUMSIZE)(%rbp)+#define resy (1*NUMSIZE)(%rbp)++#define scalar (0*NUMSIZE)(%rsp)++#define tabent (1*NUMSIZE)(%rsp)+#define ymx_2 (1*NUMSIZE)(%rsp)+#define xpy_2 (2*NUMSIZE)(%rsp)+#define kxy_2 (3*NUMSIZE)(%rsp)++#define t0 (4*NUMSIZE)(%rsp)+#define t1 (5*NUMSIZE)(%rsp)+#define t2 (6*NUMSIZE)(%rsp)+#define t3 (7*NUMSIZE)(%rsp)+#define t4 (8*NUMSIZE)(%rsp)+#define t5 (9*NUMSIZE)(%rsp)++#define acc (10*NUMSIZE)(%rsp)+#define x_1 (10*NUMSIZE)(%rsp)+#define y_1 (11*NUMSIZE)(%rsp)+#define z_1 (12*NUMSIZE)(%rsp)+#define w_1 (13*NUMSIZE)(%rsp)+#define x_3 (10*NUMSIZE)(%rsp)+#define y_3 (11*NUMSIZE)(%rsp)+#define z_3 (12*NUMSIZE)(%rsp)+#define w_3 (13*NUMSIZE)(%rsp)++// Stable homes for the input result pointer, and other variables++#define res  14*NUMSIZE(%rsp)++#define i  14*NUMSIZE+8(%rsp)++#define bias  14*NUMSIZE+16(%rsp)++#define bf  14*NUMSIZE+24(%rsp)+#define ix  14*NUMSIZE+24(%rsp)++#define tab  15*NUMSIZE(%rsp)++// Total size to reserve on the stack++#define NSPACE 488++// Syntactic variants to make x86_att version simpler to generate++#define SCALAR 0+#define TABENT (1*NUMSIZE)+#define ACC (10*NUMSIZE)+#define X3 (10*NUMSIZE)+#define Z3 (12*NUMSIZE)+#define W3 (13*NUMSIZE)++// Macro wrapping up the basic field multiplication, only trivially+// different from a pure function call to bignum_mul_p25519_alt.++#define mul_p25519(P0,P1,P2)                    \+        movq    P1, %rax ;                      \+        mulq     P2;                 \+        movq    %rax, %r8 ;                         \+        movq    %rdx, %r9 ;                         \+        xorq    %r10, %r10 ;                        \+        xorq    %r11, %r11 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x8+P2;             \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     P2;                 \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        adcq    $0x0, %r11 ;                        \+        xorq    %r12, %r12 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x10+P2;            \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    %r12, %r12 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x8+P2;             \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     P2;                 \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        xorq    %r13, %r13 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x18+P2;            \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    %r13, %r13 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x10+P2;            \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x8+P2;             \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     P2;                 \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        xorq    %r14, %r14 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x18+P2;            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    %r14, %r14 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x10+P2;            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x8+P2;             \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        xorq    %r15, %r15 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x18+P2;            \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    %r15, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x10+P2;            \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    $0x0, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x18+P2;            \+        addq    %rax, %r14 ;                        \+        adcq    %rdx, %r15 ;                        \+        movl    $0x26, %esi ;                       \+        movq    %r12, %rax ;                        \+        mulq    %rsi;                            \+        addq    %rax, %r8 ;                         \+        adcq    %rdx, %r9 ;                         \+        sbbq    %rcx, %rcx ;                        \+        movq    %r13, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r14, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r15, %rax ;                        \+        mulq    %rsi;                            \+        subq    %rcx, %rdx ;                        \+        xorq    %rcx, %rcx ;                        \+        addq    %rax, %r11 ;                        \+        movq    %rdx, %r12 ;                        \+        adcq    %rcx, %r12 ;                        \+        shldq   $0x1, %r11, %r12 ;                    \+        leaq    0x1(%r12), %rax ;                  \+        movl    $0x13, %esi ;                       \+        bts     $63, %r11 ;                         \+        imulq   %rsi, %rax ;                        \+        addq    %rax, %r8 ;                         \+        adcq    %rcx, %r9 ;                         \+        adcq    %rcx, %r10 ;                        \+        adcq    %rcx, %r11 ;                        \+        sbbq    %rax, %rax ;                        \+        notq    %rax;                            \+        andq    %rsi, %rax ;                        \+        subq    %rax, %r8 ;                         \+        sbbq    %rcx, %r9 ;                         \+        sbbq    %rcx, %r10 ;                        \+        sbbq    %rcx, %r11 ;                        \+        btr     $63, %r11 ;                         \+        movq    %r8, P0 ;                        \+        movq    %r9, 0x8+P0 ;                    \+        movq    %r10, 0x10+P0 ;                  \+        movq    %r11, 0x18+P0++// A version of multiplication that only guarantees output < 2 * p_25519.+// This basically skips the +1 and final correction in quotient estimation.++#define mul_4(P0,P1,P2)                         \+        movq    P1, %rax ;                      \+        mulq     P2;                 \+        movq    %rax, %r8 ;                         \+        movq    %rdx, %r9 ;                         \+        xorq    %r10, %r10 ;                        \+        xorq    %r11, %r11 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x8+P2;             \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     P2;                 \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        adcq    $0x0, %r11 ;                        \+        xorq    %r12, %r12 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x10+P2;            \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    %r12, %r12 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x8+P2;             \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     P2;                 \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        adcq    $0x0, %r12 ;                        \+        xorq    %r13, %r13 ;                        \+        movq    P1, %rax ;                      \+        mulq     0x18+P2;            \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    %r13, %r13 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x10+P2;            \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x8+P2;             \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     P2;                 \+        addq    %rax, %r11 ;                        \+        adcq    %rdx, %r12 ;                        \+        adcq    $0x0, %r13 ;                        \+        xorq    %r14, %r14 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq     0x18+P2;            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    %r14, %r14 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x10+P2;            \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x8+P2;             \+        addq    %rax, %r12 ;                        \+        adcq    %rdx, %r13 ;                        \+        adcq    $0x0, %r14 ;                        \+        xorq    %r15, %r15 ;                        \+        movq    0x10+P1, %rax ;                 \+        mulq     0x18+P2;            \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    %r15, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x10+P2;            \+        addq    %rax, %r13 ;                        \+        adcq    %rdx, %r14 ;                        \+        adcq    $0x0, %r15 ;                        \+        movq    0x18+P1, %rax ;                 \+        mulq     0x18+P2;            \+        addq    %rax, %r14 ;                        \+        adcq    %rdx, %r15 ;                        \+        movl    $0x26, %ebx ;                       \+        movq    %r12, %rax ;                        \+        mulq    %rbx;                            \+        addq    %rax, %r8 ;                         \+        adcq    %rdx, %r9 ;                         \+        sbbq    %rcx, %rcx ;                        \+        movq    %r13, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r9 ;                         \+        adcq    %rdx, %r10 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r14, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                        \+        addq    %rax, %r10 ;                        \+        adcq    %rdx, %r11 ;                        \+        sbbq    %rcx, %rcx ;                        \+        movq    %r15, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                        \+        xorq    %rcx, %rcx ;                        \+        addq    %rax, %r11 ;                        \+        movq    %rdx, %r12 ;                        \+        adcq    %rcx, %r12 ;                        \+        shldq   $0x1, %r11, %r12 ;                    \+        btr     $0x3f, %r11 ;                      \+        movl    $0x13, %edx ;                      \+        imulq   %r12, %rdx ;                       \+        addq    %rdx, %r8 ;                        \+        adcq    %rcx, %r9 ;                        \+        adcq    %rcx, %r10 ;                       \+        adcq    %rcx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++// Modular subtraction with double modulus 2 * p_25519 = 2^256 - 38++#define sub_twice4(P0,P1,P2)                    \+        movq    P1, %r8 ;                       \+        xorl    %ebx, %ebx ;                       \+        subq    P2, %r8 ;                       \+        movq    8+P1, %r9 ;                     \+        sbbq    8+P2, %r9 ;                     \+        movl    $38, %ecx ;                        \+        movq    16+P1, %r10 ;                   \+        sbbq    16+P2, %r10 ;                   \+        movq    24+P1, %rax ;                   \+        sbbq    24+P2, %rax ;                   \+        cmovncq %rbx, %rcx ;                       \+        subq    %rcx, %r8 ;                        \+        sbbq    %rbx, %r9 ;                        \+        sbbq    %rbx, %r10 ;                       \+        sbbq    %rbx, %rax ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 8+P0 ;                     \+        movq    %r10, 16+P0 ;                   \+        movq    %rax, 24+P0++// Modular addition and doubling with double modulus 2 * p_25519 = 2^256 - 38.+// This only ensures that the result fits in 4 digits, not that it is reduced+// even w.r.t. double modulus. The result is always correct modulo provided+// the sum of the inputs is < 2^256 + 2^256 - 38, so in particular provided+// at least one of them is reduced double modulo.++#define add_twice4(P0,P1,P2)                    \+        movq    P1, %r8 ;                       \+        xorl    %ecx, %ecx ;                       \+        addq    P2, %r8 ;                       \+        movq    0x8+P1, %r9 ;                   \+        adcq    0x8+P2, %r9 ;                   \+        movq    0x10+P1, %r10 ;                 \+        adcq    0x10+P2, %r10 ;                 \+        movq    0x18+P1, %r11 ;                 \+        adcq    0x18+P2, %r11 ;                 \+        movl    $38, %eax ;                        \+        cmovncq %rcx, %rax ;                       \+        addq    %rax, %r8 ;                        \+        adcq    %rcx, %r9 ;                        \+        adcq    %rcx, %r10 ;                       \+        adcq    %rcx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++#define double_twice4(P0,P1)                    \+        movq    P1, %r8 ;                       \+        xorl    %ecx, %ecx ;                       \+        addq    %r8, %r8 ;                         \+        movq    0x8+P1, %r9 ;                   \+        adcq    %r9, %r9 ;                         \+        movq    0x10+P1, %r10 ;                 \+        adcq    %r10, %r10 ;                       \+        movq    0x18+P1, %r11 ;                 \+        adcq    %r11, %r11 ;                       \+        movl    $38, %eax ;                        \+        cmovncq %rcx, %rax ;                       \+        addq    %rax, %r8 ;                        \+        adcq    %rcx, %r9 ;                        \+        adcq    %rcx, %r10 ;                       \+        adcq    %rcx, %r11 ;                       \+        movq    %r8, P0 ;                       \+        movq    %r9, 0x8+P0 ;                   \+        movq    %r10, 0x10+P0 ;                 \+        movq    %r11, 0x18+P0++S2N_BN_SYMBOL(edwards25519_scalarmulbase_alt):+        CFI_START+        _CET_ENDBR++// In this case the Windows form literally makes a subroutine call.+// This avoids hassle arising from keeping code and data together.++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        CFI_CALL(Ledwards25519_scalarmulbase_alt_standard)+        CFI_POP(%rsi)+        CFI_POP(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(edwards25519_scalarmulbase_alt)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Ledwards25519_scalarmulbase_alt_standard)++Ledwards25519_scalarmulbase_alt_standard:+        CFI_START+#endif++// Save registers, make room for temps, preserve input arguments.++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(NSPACE)++// Move the output pointer to a stable place++        movq    %rdi, res++// Copy the input scalar x to its local variable while reducing it+// modulo 2^252 + m where m = 27742317777372353535851937790883648493;+// this is the order of the basepoint so this doesn't change the result.+// First do q = floor(x/2^252) and x' = x - q * (2^252 + m), which gives+// an initial result -15 * m <= x' < 2^252++        movq    (%rsi), %r8+        movq    8(%rsi), %r9+        movq    16(%rsi), %r10+        movq    24(%rsi), %r11++        movq    %r11, %rcx+        shrq    $60, %rcx++        movq    $0x5812631a5cf5d3ed, %rax+        mulq    %rcx+        movq    %rax, %r12+        movq    %rdx, %r13+        movq    $0x14def9dea2f79cd6, %rax+        mulq    %rcx+        addq    %rax, %r13+        adcq    $0, %rdx+        shlq    $60, %rcx++        subq    %r12, %r8+        sbbq    %r13, %r9+        sbbq    %rdx, %r10+        sbbq    %rcx, %r11++// If x' < 0 then just directly negate it; this makes sure the+// reduced argument is strictly 0 <= x' < 2^252, but now we need+// to record (done via bit 255 of the reduced scalar, which is+// ignored in the main loop) when we negated so we can flip+// the end result to compensate.++        sbbq    %rax, %rax++        xorq    %rax, %r8+        xorq    %rax, %r9+        xorq    %rax, %r10+        xorq    %rax, %r11++        negq    %rax+        adcq    $0, %r8+        adcq    $0, %r9+        adcq    $0, %r10+        adcq    $0, %r11++        shlq    $63, %rax+        orq     %rax, %r11++// And before we store the scalar, test and reset bit 251 to+// initialize the main loop just below.++        movq    %r8, SCALAR(%rsp)+        movq    %r9, SCALAR+8(%rsp)+        movq    %r10, SCALAR+16(%rsp)+        btr     $59, %r11+        movq    %r11, SCALAR+24(%rsp)++// The main part of the computation is in extended-projective coordinates+// (X,Y,Z,T), representing an affine point on the edwards25519 curve+// (x,y) via x = X/Z, y = Y/Z and x * y = T/Z (so X * Y = T * Z).+// In comments B means the standard basepoint (x,4/5) =+// (0x216....f25d51a,0x6666..666658).+//+// Initialize accumulator "acc" to either 0 or 2^251 * B depending on+// bit 251 of the (reduced) scalar. That leaves bits 0..250 to handle.++        leaq    S2N_BN_SYMBOL(edwards25519_scalarmulbase_alt_constant)(%rip), %r10+        leaq    8*12(%r10), %r11++        movq    (%r10), %rax+        movq    (%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC(%rsp)++        movq    8*1(%r10), %rax+        movq    8*1(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+8(%rsp)++        movq    8*2(%r10), %rax+        movq    8*2(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+16(%rsp)++        movq    8*3(%r10), %rax+        movq    8*3(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+24(%rsp)++        movq    8*4(%r10), %rax+        movq    8*4(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+32(%rsp)++        movq    8*5(%r10), %rax+        movq    8*5(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+40(%rsp)++        movq    8*6(%r10), %rax+        movq    8*6(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+48(%rsp)++        movq    8*7(%r10), %rax+        movq    8*7(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+56(%rsp)++        movl    $1, %eax+        movq    %rax, ACC+64(%rsp)+        movl    $0, %eax+        movq    %rax, ACC+72(%rsp)+        movq    %rax, ACC+80(%rsp)+        movq    %rax, ACC+88(%rsp)++        movq    8*8(%r10), %rax+        movq    8*8(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+96(%rsp)++        movq    8*9(%r10), %rax+        movq    8*9(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+104(%rsp)++        movq    8*10(%r10), %rax+        movq    8*10(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+112(%rsp)++        movq    8*11(%r10), %rax+        movq    8*11(%r11), %rcx+        cmovcq  %rcx, %rax+        movq    %rax, ACC+120(%rsp)++// The counter "i" tracks the bit position for which the scalar has+// already been absorbed, starting at 0 and going up in chunks of 4.+//+// The pointer "tab" points at the current block of the table for+// multiples (2^i * j) * B at the current bit position i; 1 <= j <= 8.+//+// The bias is always either 0 and 1 and needs to be added to the+// partially processed scalar implicitly. This is used to absorb 4 bits+// of scalar per iteration from 3-bit table indexing by exploiting+// negation: (16 * h + l) * B = (16 * (h + 1) - (16 - l)) * B is used+// when l >= 9. Note that we can't have any bias left over at the+// end because we made sure bit 251 is clear in the reduced scalar.++        movq    $0, i+        leaq    8*24(%r10), %rax+        movq    %rax, tab+        movq    $0, bias++// Start of the main loop, repeated 63 times for i = 4, 8, ..., 252++Ledwards25519_scalarmulbase_alt_scalarloop:++// Look at the next 4-bit field "bf", adding the previous bias as well.+// Choose the table index "ix" as bf when bf <= 8 and 16 - bf for bf >= 9,+// setting the bias to 1 for the next iteration in the latter case.++        movq    i, %rax+        movq    %rax, %rcx+        shrq    $6, %rax+        movq    (%rsp,%rax,8), %rax // Exploiting scalar = sp exactly+        shrq    %cl, %rax+        andq    $15, %rax+        addq    bias, %rax+        movq    %rax, bf++        cmpq    $9, bf+        sbbq    %rax, %rax+        incq    %rax+        movq    %rax, bias++        movq    $16, %rdi+        subq    bf, %rdi+        cmpq    $0, bias+        cmovzq  bf, %rdi+        movq    %rdi, ix++// Perform constant-time lookup in the table to get element number "ix".+// The table entry for the affine point (x,y) is actually a triple+// (y - x,x + y,2 * d * x * y) to precompute parts of the addition.+// Note that "ix" can be 0, so we set up the appropriate identity first.++        movl    $1, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        movl    $1, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        xorl    %r13d, %r13d+        xorl    %r14d, %r14d+        xorl    %r15d, %r15d++        movq    tab, %rbp++        cmpq    $1, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $2, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $3, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $4, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $5, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $6, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $7, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15+        addq    $96, %rbp++        cmpq    $8, ix+        movq    (%rbp), %rsi+        cmovzq  %rsi, %rax+        movq    8(%rbp), %rsi+        cmovzq  %rsi, %rbx+        movq    16(%rbp), %rsi+        cmovzq  %rsi, %rcx+        movq    24(%rbp), %rsi+        cmovzq  %rsi, %rdx+        movq    32(%rbp), %rsi+        cmovzq  %rsi, %r8+        movq    40(%rbp), %rsi+        cmovzq  %rsi, %r9+        movq    48(%rbp), %rsi+        cmovzq  %rsi, %r10+        movq    56(%rbp), %rsi+        cmovzq  %rsi, %r11+        movq    64(%rbp), %rsi+        cmovzq  %rsi, %r12+        movq    72(%rbp), %rsi+        cmovzq  %rsi, %r13+        movq    80(%rbp), %rsi+        cmovzq  %rsi, %r14+        movq    88(%rbp), %rsi+        cmovzq  %rsi, %r15++        addq    $96, %rbp+        movq    %rbp, tab++// We now have the triple from the table in registers as follows+//+//      [%rdx;%rcx;%rbx;%rax] = y - x+//      [%r11;%r10;%r9;%r8] = x + y+//      [%r15;%r14;%r13;%r12] = 2 * d * x * y+//+// In case bias = 1 we need to negate this. For Edwards curves+// -(x,y) = (-x,y), i.e. we need to negate the x coordinate.+// In this processed encoding, that amounts to swapping the+// first two fields and negating the third.+//+// The optional negation here also pretends bias = 0 whenever+// ix = 0 so that it doesn't need to handle the case of zero+// inputs, since no non-trivial table entries are zero. Note+// that in the zero case the whole negation is trivial, and+// so indeed is the swapping.++        cmpq    $0, bias++        movq    %rax, %rsi+        cmovnzq %r8, %rsi+        cmovnzq %rax, %r8+        movq    %rsi, TABENT(%rsp)+        movq    %r8, TABENT+32(%rsp)++        movq    %rbx, %rsi+        cmovnzq %r9, %rsi+        cmovnzq %rbx, %r9+        movq    %rsi, TABENT+8(%rsp)+        movq    %r9, TABENT+40(%rsp)++        movq    %rcx, %rsi+        cmovnzq %r10, %rsi+        cmovnzq %rcx, %r10+        movq    %rsi, TABENT+16(%rsp)+        movq    %r10, TABENT+48(%rsp)++        movq    %rdx, %rsi+        cmovnzq %r11, %rsi+        cmovnzq %rdx, %r11+        movq    %rsi, TABENT+24(%rsp)+        movq    %r11, TABENT+56(%rsp)++        movq    $-19, %rax+        movq    $-1, %rbx+        movq    $-1, %rcx+        movq    $0x7fffffffffffffff, %rdx+        subq    %r12, %rax+        sbbq    %r13, %rbx+        sbbq    %r14, %rcx+        sbbq    %r15, %rdx++        movq    ix, %r8+        movq    bias, %r9+        testq   %r8, %r8+        cmovzq  %r8, %r9+        testq   %r9, %r9++        cmovzq  %r12, %rax+        cmovzq  %r13, %rbx+        cmovzq  %r14, %rcx+        cmovzq  %r15, %rdx+        movq    %rax, TABENT+64(%rsp)+        movq    %rbx, TABENT+72(%rsp)+        movq    %rcx, TABENT+80(%rsp)+        movq    %rdx, TABENT+88(%rsp)++// Extended-projective and precomputed mixed addition.+// This is effectively the same as calling the standalone+// function edwards25519_pepadd(acc,acc,tabent), but we+// only retain slightly weaker normalization < 2 * p_25519+// throughout the inner loop, so the computation is+// slightly different, and faster overall.++        double_twice4(t0,z_1)+        sub_twice4(t1,y_1,x_1)+        add_twice4(t2,y_1,x_1)+        mul_4(t3,w_1,kxy_2)+        mul_4(t1,t1,ymx_2)+        mul_4(t2,t2,xpy_2)+        sub_twice4(t4,t0,t3)+        add_twice4(t0,t0,t3)+        sub_twice4(t5,t2,t1)+        add_twice4(t1,t2,t1)+        mul_4(z_3,t4,t0)+        mul_4(x_3,t5,t4)+        mul_4(y_3,t0,t1)+        mul_4(w_3,t5,t1)++// End of the main loop; move on by 4 bits.++        addq    $4, i+        cmpq    $252, i+        jc      Ledwards25519_scalarmulbase_alt_scalarloop++// Insert the optional negation of the projective X coordinate, and+// so by extension the final affine x coordinate x = X/Z and thus+// the point P = (x,y). We only know X < 2 * p_25519, so we do the+// negation as 2 * p_25519 - X to keep it nonnegative. From this+// point on we don't need any normalization of the coordinates+// except for making sure that they fit in 4 digits.++        movq    X3(%rsp), %r8+        movq    X3+8(%rsp), %r9+        movq    X3+16(%rsp), %r10+        movq    X3+24(%rsp), %r11+        movq    $0xffffffffffffffda, %r12+        subq    %r8, %r12+        movq    $0xffffffffffffffff, %r13+        sbbq    %r9, %r13+        movq    $0xffffffffffffffff, %r14+        sbbq    %r10, %r14+        movq    $0xffffffffffffffff, %r15+        sbbq    %r11, %r15+        movq    SCALAR+24(%rsp), %rax+        btq     $63, %rax+        cmovcq  %r12, %r8+        cmovcq  %r13, %r9+        cmovcq  %r14, %r10+        cmovcq  %r15, %r11+        movq    %r8, X3(%rsp)+        movq    %r9, X3+8(%rsp)+        movq    %r10, X3+16(%rsp)+        movq    %r11, X3+24(%rsp)++// Now we need to map out of the extended-projective representation+// (X,Y,Z,W) back to the affine form (x,y) = (X/Z,Y/Z). This means+// first calling the modular inverse to get w_3 = 1/z_3.++        leaq    W3(%rsp), %rdi+        leaq    Z3(%rsp), %rsi++// Inline copy of bignum_inv_p25519, identical except for stripping out+// the prologue and epilogue saving and restoring registers and making+// and reclaiming room on the stack. For more details and explanations see+// "x86/curve25519/bignum_inv_p25519.S". Note that the stack it uses for+// its own temporaries is 208 bytes, so it has no effect on variables+// that are needed in the rest of our computation here: res, x_3, y_3,+// z_3 and w_3.++        movq    %rdi, 0xc0(%rsp)+        xorl    %eax, %eax+        leaq    -0x13(%rax), %rcx+        notq    %rax+        movq    %rcx, (%rsp)+        movq    %rax, 0x8(%rsp)+        movq    %rax, 0x10(%rsp)+        btr     $0x3f, %rax+        movq    %rax, 0x18(%rsp)+        movq    (%rsi), %rdx+        movq    0x8(%rsi), %rcx+        movq    0x10(%rsi), %r8+        movq    0x18(%rsi), %r9+        movl    $0x1, %eax+        xorl    %r10d, %r10d+        bts     $0x3f, %r9+        adcq    %r10, %rax+        imulq   $0x13, %rax, %rax+        addq    %rax, %rdx+        adcq    %r10, %rcx+        adcq    %r10, %r8+        adcq    %r10, %r9+        movl    $0x13, %eax+        cmovbq  %r10, %rax+        subq    %rax, %rdx+        sbbq    %r10, %rcx+        sbbq    %r10, %r8+        sbbq    %r10, %r9+        btr     $0x3f, %r9+        movq    %rdx, 0x20(%rsp)+        movq    %rcx, 0x28(%rsp)+        movq    %r8, 0x30(%rsp)+        movq    %r9, 0x38(%rsp)+        xorl    %eax, %eax+        movq    %rax, 0x40(%rsp)+        movq    %rax, 0x48(%rsp)+        movq    %rax, 0x50(%rsp)+        movq    %rax, 0x58(%rsp)+        movabsq $0xa0f99e2375022099, %rax+        movq    %rax, 0x60(%rsp)+        movabsq $0xa8c68f3f1d132595, %rax+        movq    %rax, 0x68(%rsp)+        movabsq $0x6c6c893805ac5242, %rax+        movq    %rax, 0x70(%rsp)+        movabsq $0x276508b241770615, %rax+        movq    %rax, 0x78(%rsp)+        movq    $0xa,  0x90(%rsp)+        movq    $0x1,  0x98(%rsp)+        jmp     Ledwards25519_scalarmulbase_alt_midloop+Ledwards25519_scalarmulbase_alt_inverseloop:+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %rdi+        andq    %r11, %rdi+        addq    %rax, %rdi+        movq    %rdi, 0x80(%rsp)+        movq    %r12, %rax+        andq    %r13, %rax+        movq    %r14, %rsi+        andq    %r15, %rsi+        addq    %rax, %rsi+        movq    %rsi, 0x88(%rsp)+        xorl    %ebx, %ebx+        movq    (%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x20(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    (%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x20(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        xorl    %ecx, %ecx+        movq    0x8(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x28(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, (%rsp)+        xorl    %edi, %edi+        movq    0x8(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    0x28(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $0x3b, %rbp, %rsi+        movq    %rsi, 0x20(%rsp)+        xorl    %esi, %esi+        movq    0x10(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    0x30(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rcx, %rbx+        movq    %rbx, 0x8(%rsp)+        xorl    %ebx, %ebx+        movq    0x10(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x30(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $0x3b, %rdi, %rbp+        movq    %rbp, 0x28(%rsp)+        movq    0x18(%rsp), %rax+        xorq    %r9, %rax+        movq    %rax, %rbp+        sarq    $0x3f, %rbp+        andq    %r8, %rbp+        negq    %rbp+        mulq    %r8+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x38(%rsp), %rax+        xorq    %r11, %rax+        movq    %rax, %rdx+        sarq    $0x3f, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbp+        mulq    %r10+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        shrdq   $0x3b, %rsi, %rcx+        movq    %rcx, 0x10(%rsp)+        shrdq   $0x3b, %rbp, %rsi+        movq    0x18(%rsp), %rax+        movq    %rsi, 0x18(%rsp)+        xorq    %r13, %rax+        movq    %rax, %rsi+        sarq    $0x3f, %rsi+        andq    %r12, %rsi+        negq    %rsi+        mulq    %r12+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        movq    0x38(%rsp), %rax+        xorq    %r15, %rax+        movq    %rax, %rdx+        sarq    $0x3f, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rsi+        mulq    %r14+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, 0x30(%rsp)+        shrdq   $0x3b, %rsi, %rbx+        movq    %rbx, 0x38(%rsp)+        movq    0x80(%rsp), %rbx+        movq    0x88(%rsp), %rbp+        xorl    %ecx, %ecx+        movq    0x40(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x60(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x40(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x40(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x60(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x60(%rsp)+        xorl    %ebx, %ebx+        movq    0x48(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x68(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    0x48(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, 0x48(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x68(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, 0x68(%rsp)+        xorl    %ecx, %ecx+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x70(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x50(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x50(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x70(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x70(%rsp)+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        movq    %r9, %rbx+        andq    %r8, %rbx+        negq    %rbx+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbx+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rbx, %rdx+        movq    %rdx, %rbx+        shldq   $0x1, %rcx, %rdx+        sarq    $0x3f, %rbx+        addq    %rbx, %rdx+        movl    $0x13, %eax+        imulq   %rdx+        movq    0x40(%rsp), %r8+        addq    %rax, %r8+        movq    %r8, 0x40(%rsp)+        movq    0x48(%rsp), %r8+        adcq    %rdx, %r8+        movq    %r8, 0x48(%rsp)+        movq    0x50(%rsp), %r8+        adcq    %rbx, %r8+        movq    %r8, 0x50(%rsp)+        adcq    %rbx, %rcx+        shlq    $0x3f, %rax+        addq    %rax, %rcx+        movq    0x58(%rsp), %rax+        movq    %rcx, 0x58(%rsp)+        xorq    %r13, %rax+        movq    %r13, %rcx+        andq    %r12, %rcx+        negq    %rcx+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rcx+        movq    0x78(%rsp), %rax+        xorq    %r15, %rax+        movq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rcx+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rcx, %rdx+        movq    %rdx, %rcx+        shldq   $0x1, %rsi, %rdx+        sarq    $0x3f, %rcx+        movl    $0x13, %eax+        addq    %rcx, %rdx+        imulq   %rdx+        movq    0x60(%rsp), %r8+        addq    %rax, %r8+        movq    %r8, 0x60(%rsp)+        movq    0x68(%rsp), %r8+        adcq    %rdx, %r8+        movq    %r8, 0x68(%rsp)+        movq    0x70(%rsp), %r8+        adcq    %rcx, %r8+        movq    %r8, 0x70(%rsp)+        adcq    %rcx, %rsi+        shlq    $0x3f, %rax+        addq    %rax, %rsi+        movq    %rsi, 0x78(%rsp)+Ledwards25519_scalarmulbase_alt_midloop:+        movq    0x98(%rsp), %rsi+        movq    (%rsp), %rdx+        movq    0x20(%rsp), %rcx+        movq    %rdx, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        xorl    %ebp, %ebp+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %rdx+        leaq    (%rcx,%rax), %rdi+        shlq    $0x16, %rdx+        shlq    $0x16, %rdi+        sarq    $0x2b, %rdx+        sarq    $0x2b, %rdi+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %rbx+        leaq    (%rcx,%rax), %rcx+        sarq    $0x2a, %rbx+        sarq    $0x2a, %rcx+        movq    %rdx, 0xa0(%rsp)+        movq    %rbx, 0xa8(%rsp)+        movq    %rdi, 0xb0(%rsp)+        movq    %rcx, 0xb8(%rsp)+        movq    (%rsp), %r12+        imulq   %r12, %rdi+        imulq   %rdx, %r12+        movq    0x20(%rsp), %r13+        imulq   %r13, %rbx+        imulq   %rcx, %r13+        addq    %rbx, %r12+        addq    %rdi, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r10+        shlq    $0x16, %r8+        shlq    $0x16, %r10+        sarq    $0x2b, %r8+        sarq    $0x2b, %r10+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r15+        leaq    (%rcx,%rax), %r11+        sarq    $0x2a, %r15+        sarq    $0x2a, %r11+        movq    %r13, %rbx+        movq    %r12, %rcx+        imulq   %r8, %r12+        imulq   %r15, %rbx+        addq    %rbx, %r12+        imulq   %r11, %r13+        imulq   %r10, %rcx+        addq    %rcx, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    0xa0(%rsp), %rax+        imulq   %r8, %rax+        movq    0xb0(%rsp), %rdx+        imulq   %r15, %rdx+        imulq   0xa8(%rsp), %r8+        imulq   0xb8(%rsp), %r15+        addq    %r8, %r15+        leaq    (%rax,%rdx), %r9+        movq    0xa0(%rsp), %rax+        imulq   %r10, %rax+        movq    0xb0(%rsp), %rdx+        imulq   %r11, %rdx+        imulq   0xa8(%rsp), %r10+        imulq   0xb8(%rsp), %r11+        addq    %r10, %r11+        leaq    (%rax,%rdx), %r13+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r12+        shlq    $0x15, %r8+        shlq    $0x15, %r12+        sarq    $0x2b, %r8+        sarq    $0x2b, %r12+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r10+        leaq    (%rcx,%rax), %r14+        sarq    $0x2b, %r10+        sarq    $0x2b, %r14+        movq    %r9, %rax+        imulq   %r8, %rax+        movq    %r13, %rdx+        imulq   %r10, %rdx+        imulq   %r15, %r8+        imulq   %r11, %r10+        addq    %r8, %r10+        leaq    (%rax,%rdx), %r8+        movq    %r9, %rax+        imulq   %r12, %rax+        movq    %r13, %rdx+        imulq   %r14, %rdx+        imulq   %r15, %r12+        imulq   %r11, %r14+        addq    %r12, %r14+        leaq    (%rax,%rdx), %r12+        movq    %rsi, 0x98(%rsp)+        decq     0x90(%rsp)+        jne     Ledwards25519_scalarmulbase_alt_inverseloop+        movq    (%rsp), %rax+        movq    0x20(%rsp), %rcx+        imulq   %r8, %rax+        imulq   %r10, %rcx+        addq    %rcx, %rax+        sarq    $0x3f, %rax+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        xorq    %rax, %r9+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        xorq    %rax, %r11+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        xorq    %rax, %r13+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        xorq    %rax, %r15+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %r12+        andq    %r11, %r12+        addq    %rax, %r12+        xorl    %r13d, %r13d+        movq    0x40(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x60(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movq    0x48(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r13+        adcq    %rdx, %r14+        movq    0x68(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x70(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        andq    %r8, %r9+        negq    %r9+        mulq    %r8+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %r9+        mulq    %r10+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    %r9, %rax+        shldq   $0x1, %r15, %rax+        sarq    $0x3f, %r9+        movl    $0x13, %ebx+        leaq    0x1(%rax,%r9,1), %rax+        imulq   %rbx+        xorl    %ebp, %ebp+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r9, %r14+        adcq    %r9, %r15+        shlq    $0x3f, %rax+        addq    %rax, %r15+        cmovns  %rbp, %rbx+        subq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    %rbp, %r14+        sbbq    %rbp, %r15+        btr     $0x3f, %r15+        movq    0xc0(%rsp), %rdi+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)++// The final result is x = X * inv(Z), y = Y * inv(Z).+// These are the only operations in the whole computation that+// fully reduce modulo p_25519 since now we want the canonical+// answer as output.++        movq    res, %rbp+        mul_p25519(resx,x_3,w_3)+        mul_p25519(resy,y_3,w_3)++// Restore stack and registers++        CFI_INC_RSP(NSPACE)++        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++#if WINDOWS_ABI+S2N_BN_SIZE_DIRECTIVE(Ledwards25519_scalarmulbase_alt_standard)+#else+S2N_BN_SIZE_DIRECTIVE(edwards25519_scalarmulbase_alt)+#endif++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif++// ****************************************************************************+// The precomputed data (all read-only).+// ****************************************************************************++#if defined(__ELF__)+.section .rodata+.type S2N_BN_SYMBOL(edwards25519_scalarmulbase_alt_constant), %object+.size S2N_BN_SYMBOL(edwards25519_scalarmulbase_alt_constant), 48576+#elif defined(__APPLE__)+.const_data+#endif++S2N_BN_SYMBOL(edwards25519_scalarmulbase_alt_constant):++// 0 * B = 0 and 2^251 * B in extended-projective coordinates+// but with Z = 1 assumed and hence left out, so they are (X,Y,T) only.++        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000++        .quad   0x0000000000000001+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000++        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000+        .quad   0x0000000000000000++        .quad   0x525f946d7c7220e7+        .quad   0x4636b0b2f1e35444+        .quad   0x796e9d70e892ae0f+        .quad   0x03dec05fa937adb1+        .quad   0x6d1c271cc6375515+        .quad   0x462588c4a4ca4f14+        .quad   0x691129fee55afc39+        .quad   0x15949f784d8472f5+        .quad   0xbd89e510afad0049+        .quad   0x4d1f08c073b9860e+        .quad   0x07716e8b2d00af9d+        .quad   0x70d685f68f859714++// Precomputed table of multiples of generator for edwards25519+// all in precomputed extended-projective (y-x,x+y,2*d*x*y) triples.++        // 2^0 * 1 * G++        .quad   0x9d103905d740913e+        .quad   0xfd399f05d140beb3+        .quad   0xa5c18434688f8a09+        .quad   0x44fd2f9298f81267+        .quad   0x2fbc93c6f58c3b85+        .quad   0xcf932dc6fb8c0e19+        .quad   0x270b4898643d42c2+        .quad   0x07cf9d3a33d4ba65+        .quad   0xabc91205877aaa68+        .quad   0x26d9e823ccaac49e+        .quad   0x5a1b7dcbdd43598c+        .quad   0x6f117b689f0c65a8++        // 2^0 * 2 * G++        .quad   0x8a99a56042b4d5a8+        .quad   0x8f2b810c4e60acf6+        .quad   0xe09e236bb16e37aa+        .quad   0x6bb595a669c92555+        .quad   0x9224e7fc933c71d7+        .quad   0x9f469d967a0ff5b5+        .quad   0x5aa69a65e1d60702+        .quad   0x590c063fa87d2e2e+        .quad   0x43faa8b3a59b7a5f+        .quad   0x36c16bdd5d9acf78+        .quad   0x500fa0840b3d6a31+        .quad   0x701af5b13ea50b73++        // 2^0 * 3 * G++        .quad   0x56611fe8a4fcd265+        .quad   0x3bd353fde5c1ba7d+        .quad   0x8131f31a214bd6bd+        .quad   0x2ab91587555bda62+        .quad   0xaf25b0a84cee9730+        .quad   0x025a8430e8864b8a+        .quad   0xc11b50029f016732+        .quad   0x7a164e1b9a80f8f4+        .quad   0x14ae933f0dd0d889+        .quad   0x589423221c35da62+        .quad   0xd170e5458cf2db4c+        .quad   0x5a2826af12b9b4c6++        // 2^0 * 4 * G++        .quad   0x95fe050a056818bf+        .quad   0x327e89715660faa9+        .quad   0xc3e8e3cd06a05073+        .quad   0x27933f4c7445a49a+        .quad   0x287351b98efc099f+        .quad   0x6765c6f47dfd2538+        .quad   0xca348d3dfb0a9265+        .quad   0x680e910321e58727+        .quad   0x5a13fbe9c476ff09+        .quad   0x6e9e39457b5cc172+        .quad   0x5ddbdcf9102b4494+        .quad   0x7f9d0cbf63553e2b++        // 2^0 * 5 * G++        .quad   0x7f9182c3a447d6ba+        .quad   0xd50014d14b2729b7+        .quad   0xe33cf11cb864a087+        .quad   0x154a7e73eb1b55f3+        .quad   0xa212bc4408a5bb33+        .quad   0x8d5048c3c75eed02+        .quad   0xdd1beb0c5abfec44+        .quad   0x2945ccf146e206eb+        .quad   0xbcbbdbf1812a8285+        .quad   0x270e0807d0bdd1fc+        .quad   0xb41b670b1bbda72d+        .quad   0x43aabe696b3bb69a++        // 2^0 * 6 * G++        .quad   0x499806b67b7d8ca4+        .quad   0x575be28427d22739+        .quad   0xbb085ce7204553b9+        .quad   0x38b64c41ae417884+        .quad   0x3a0ceeeb77157131+        .quad   0x9b27158900c8af88+        .quad   0x8065b668da59a736+        .quad   0x51e57bb6a2cc38bd+        .quad   0x85ac326702ea4b71+        .quad   0xbe70e00341a1bb01+        .quad   0x53e4a24b083bc144+        .quad   0x10b8e91a9f0d61e3++        // 2^0 * 7 * G++        .quad   0xba6f2c9aaa3221b1+        .quad   0x6ca021533bba23a7+        .quad   0x9dea764f92192c3a+        .quad   0x1d6edd5d2e5317e0+        .quad   0x6b1a5cd0944ea3bf+        .quad   0x7470353ab39dc0d2+        .quad   0x71b2528228542e49+        .quad   0x461bea69283c927e+        .quad   0xf1836dc801b8b3a2+        .quad   0xb3035f47053ea49a+        .quad   0x529c41ba5877adf3+        .quad   0x7a9fbb1c6a0f90a7++        // 2^0 * 8 * G++        .quad   0xe2a75dedf39234d9+        .quad   0x963d7680e1b558f9+        .quad   0x2c2741ac6e3c23fb+        .quad   0x3a9024a1320e01c3+        .quad   0x59b7596604dd3e8f+        .quad   0x6cb30377e288702c+        .quad   0xb1339c665ed9c323+        .quad   0x0915e76061bce52f+        .quad   0xe7c1f5d9c9a2911a+        .quad   0xb8a371788bcca7d7+        .quad   0x636412190eb62a32+        .quad   0x26907c5c2ecc4e95++        // 2^4 * 1 * G++        .quad   0x7ec851ca553e2df3+        .quad   0xa71284cba64878b3+        .quad   0xe6b5e4193288d1e7+        .quad   0x4cf210ec5a9a8883+        .quad   0x322d04a52d9021f6+        .quad   0xb9c19f3375c6bf9c+        .quad   0x587a3a4342d20b09+        .quad   0x143b1cf8aa64fe61+        .quad   0x9f867c7d968acaab+        .quad   0x5f54258e27092729+        .quad   0xd0a7d34bea180975+        .quad   0x21b546a3374126e1++        // 2^4 * 2 * G++        .quad   0xa94ff858a2888343+        .quad   0xce0ed4565313ed3c+        .quad   0xf55c3dcfb5bf34fa+        .quad   0x0a653ca5c9eab371+        .quad   0x490a7a45d185218f+        .quad   0x9a15377846049335+        .quad   0x0060ea09cc31e1f6+        .quad   0x7e041577f86ee965+        .quad   0x66b2a496ce5b67f3+        .quad   0xff5492d8bd569796+        .quad   0x503cec294a592cd0+        .quad   0x566943650813acb2++        // 2^4 * 3 * G++        .quad   0xb818db0c26620798+        .quad   0x5d5c31d9606e354a+        .quad   0x0982fa4f00a8cdc7+        .quad   0x17e12bcd4653e2d4+        .quad   0x5672f9eb1dabb69d+        .quad   0xba70b535afe853fc+        .quad   0x47ac0f752796d66d+        .quad   0x32a5351794117275+        .quad   0xd3a644a6df648437+        .quad   0x703b6559880fbfdd+        .quad   0xcb852540ad3a1aa5+        .quad   0x0900b3f78e4c6468++        // 2^4 * 4 * G++        .quad   0x0a851b9f679d651b+        .quad   0xe108cb61033342f2+        .quad   0xd601f57fe88b30a3+        .quad   0x371f3acaed2dd714+        .quad   0xed280fbec816ad31+        .quad   0x52d9595bd8e6efe3+        .quad   0x0fe71772f6c623f5+        .quad   0x4314030b051e293c+        .quad   0xd560005efbf0bcad+        .quad   0x8eb70f2ed1870c5e+        .quad   0x201f9033d084e6a0+        .quad   0x4c3a5ae1ce7b6670++        // 2^4 * 5 * G++        .quad   0x4138a434dcb8fa95+        .quad   0x870cf67d6c96840b+        .quad   0xde388574297be82c+        .quad   0x7c814db27262a55a+        .quad   0xbaf875e4c93da0dd+        .quad   0xb93282a771b9294d+        .quad   0x80d63fb7f4c6c460+        .quad   0x6de9c73dea66c181+        .quad   0x478904d5a04df8f2+        .quad   0xfafbae4ab10142d3+        .quad   0xf6c8ac63555d0998+        .quad   0x5aac4a412f90b104++        // 2^4 * 6 * G++        .quad   0xc64f326b3ac92908+        .quad   0x5551b282e663e1e0+        .quad   0x476b35f54a1a4b83+        .quad   0x1b9da3fe189f68c2+        .quad   0x603a0d0abd7f5134+        .quad   0x8089c932e1d3ae46+        .quad   0xdf2591398798bd63+        .quad   0x1c145cd274ba0235+        .quad   0x32e8386475f3d743+        .quad   0x365b8baf6ae5d9ef+        .quad   0x825238b6385b681e+        .quad   0x234929c1167d65e1++        // 2^4 * 7 * G++        .quad   0x984decaba077ade8+        .quad   0x383f77ad19eb389d+        .quad   0xc7ec6b7e2954d794+        .quad   0x59c77b3aeb7c3a7a+        .quad   0x48145cc21d099fcf+        .quad   0x4535c192cc28d7e5+        .quad   0x80e7c1e548247e01+        .quad   0x4a5f28743b2973ee+        .quad   0xd3add725225ccf62+        .quad   0x911a3381b2152c5d+        .quad   0xd8b39fad5b08f87d+        .quad   0x6f05606b4799fe3b++        // 2^4 * 8 * G++        .quad   0x9ffe9e92177ba962+        .quad   0x98aee71d0de5cae1+        .quad   0x3ff4ae942d831044+        .quad   0x714de12e58533ac8+        .quad   0x5b433149f91b6483+        .quad   0xadb5dc655a2cbf62+        .quad   0x87fa8412632827b3+        .quad   0x60895e91ab49f8d8+        .quad   0xe9ecf2ed0cf86c18+        .quad   0xb46d06120735dfd4+        .quad   0xbc9da09804b96be7+        .quad   0x73e2e62fd96dc26b++        // 2^8 * 1 * G++        .quad   0xed5b635449aa515e+        .quad   0xa865c49f0bc6823a+        .quad   0x850c1fe95b42d1c4+        .quad   0x30d76d6f03d315b9+        .quad   0x2eccdd0e632f9c1d+        .quad   0x51d0b69676893115+        .quad   0x52dfb76ba8637a58+        .quad   0x6dd37d49a00eef39+        .quad   0x6c4444172106e4c7+        .quad   0xfb53d680928d7f69+        .quad   0xb4739ea4694d3f26+        .quad   0x10c697112e864bb0++        // 2^8 * 2 * G++        .quad   0x6493c4277dbe5fde+        .quad   0x265d4fad19ad7ea2+        .quad   0x0e00dfc846304590+        .quad   0x25e61cabed66fe09+        .quad   0x0ca62aa08358c805+        .quad   0x6a3d4ae37a204247+        .quad   0x7464d3a63b11eddc+        .quad   0x03bf9baf550806ef+        .quad   0x3f13e128cc586604+        .quad   0x6f5873ecb459747e+        .quad   0xa0b63dedcc1268f5+        .quad   0x566d78634586e22c++        // 2^8 * 3 * G++        .quad   0x1637a49f9cc10834+        .quad   0xbc8e56d5a89bc451+        .quad   0x1cb5ec0f7f7fd2db+        .quad   0x33975bca5ecc35d9+        .quad   0xa1054285c65a2fd0+        .quad   0x6c64112af31667c3+        .quad   0x680ae240731aee58+        .quad   0x14fba5f34793b22a+        .quad   0x3cd746166985f7d4+        .quad   0x593e5e84c9c80057+        .quad   0x2fc3f2b67b61131e+        .quad   0x14829cea83fc526c++        // 2^8 * 4 * G++        .quad   0xff437b8497dd95c2+        .quad   0x6c744e30aa4eb5a7+        .quad   0x9e0c5d613c85e88b+        .quad   0x2fd9c71e5f758173+        .quad   0x21e70b2f4e71ecb8+        .quad   0xe656ddb940a477e3+        .quad   0xbf6556cece1d4f80+        .quad   0x05fc3bc4535d7b7e+        .quad   0x24b8b3ae52afdedd+        .quad   0x3495638ced3b30cf+        .quad   0x33a4bc83a9be8195+        .quad   0x373767475c651f04++        // 2^8 * 5 * G++        .quad   0x2fba99fd40d1add9+        .quad   0xb307166f96f4d027+        .quad   0x4363f05215f03bae+        .quad   0x1fbea56c3b18f999+        .quad   0x634095cb14246590+        .quad   0xef12144016c15535+        .quad   0x9e38140c8910bc60+        .quad   0x6bf5905730907c8c+        .quad   0x0fa778f1e1415b8a+        .quad   0x06409ff7bac3a77e+        .quad   0x6f52d7b89aa29a50+        .quad   0x02521cf67a635a56++        // 2^8 * 6 * G++        .quad   0x513fee0b0a9d5294+        .quad   0x8f98e75c0fdf5a66+        .quad   0xd4618688bfe107ce+        .quad   0x3fa00a7e71382ced+        .quad   0xb1146720772f5ee4+        .quad   0xe8f894b196079ace+        .quad   0x4af8224d00ac824a+        .quad   0x001753d9f7cd6cc4+        .quad   0x3c69232d963ddb34+        .quad   0x1dde87dab4973858+        .quad   0xaad7d1f9a091f285+        .quad   0x12b5fe2fa048edb6++        // 2^8 * 7 * G++        .quad   0x71f0fbc496fce34d+        .quad   0x73b9826badf35bed+        .quad   0xd2047261ff28c561+        .quad   0x749b76f96fb1206f+        .quad   0xdf2b7c26ad6f1e92+        .quad   0x4b66d323504b8913+        .quad   0x8c409dc0751c8bc3+        .quad   0x6f7e93c20796c7b8+        .quad   0x1f5af604aea6ae05+        .quad   0xc12351f1bee49c99+        .quad   0x61a808b5eeff6b66+        .quad   0x0fcec10f01e02151++        // 2^8 * 8 * G++        .quad   0x644d58a649fe1e44+        .quad   0x21fcaea231ad777e+        .quad   0x02441c5a887fd0d2+        .quad   0x4901aa7183c511f3+        .quad   0x3df2d29dc4244e45+        .quad   0x2b020e7493d8de0a+        .quad   0x6cc8067e820c214d+        .quad   0x413779166feab90a+        .quad   0x08b1b7548c1af8f0+        .quad   0xce0f7a7c246299b4+        .quad   0xf760b0f91e06d939+        .quad   0x41bb887b726d1213++        // 2^12 * 1 * G++        .quad   0x9267806c567c49d8+        .quad   0x066d04ccca791e6a+        .quad   0xa69f5645e3cc394b+        .quad   0x5c95b686a0788cd2+        .quad   0x97d980e0aa39f7d2+        .quad   0x35d0384252c6b51c+        .quad   0x7d43f49307cd55aa+        .quad   0x56bd36cfb78ac362+        .quad   0x2ac519c10d14a954+        .quad   0xeaf474b494b5fa90+        .quad   0xe6af8382a9f87a5a+        .quad   0x0dea6db1879be094++        // 2^12 * 2 * G++        .quad   0xaa66bf547344e5ab+        .quad   0xda1258888f1b4309+        .quad   0x5e87d2b3fd564b2f+        .quad   0x5b2c78885483b1dd+        .quad   0x15baeb74d6a8797a+        .quad   0x7ef55cf1fac41732+        .quad   0x29001f5a3c8b05c5+        .quad   0x0ad7cc8752eaccfb+        .quad   0x52151362793408cf+        .quad   0xeb0f170319963d94+        .quad   0xa833b2fa883d9466+        .quad   0x093a7fa775003c78++        // 2^12 * 3 * G++        .quad   0xe5107de63a16d7be+        .quad   0xa377ffdc9af332cf+        .quad   0x70d5bf18440b677f+        .quad   0x6a252b19a4a31403+        .quad   0xb8e9604460a91286+        .quad   0x7f3fd8047778d3de+        .quad   0x67d01e31bf8a5e2d+        .quad   0x7b038a06c27b653e+        .quad   0x9ed919d5d36990f3+        .quad   0x5213aebbdb4eb9f2+        .quad   0xc708ea054cb99135+        .quad   0x58ded57f72260e56++        // 2^12 * 4 * G++        .quad   0x78e79dade9413d77+        .quad   0xf257f9d59729e67d+        .quad   0x59db910ee37aa7e6+        .quad   0x6aa11b5bbb9e039c+        .quad   0xda6d53265b0fd48b+        .quad   0x8960823193bfa988+        .quad   0xd78ac93261d57e28+        .quad   0x79f2942d3a5c8143+        .quad   0x97da2f25b6c88de9+        .quad   0x251ba7eaacf20169+        .quad   0x09b44f87ef4eb4e4+        .quad   0x7d90ab1bbc6a7da5++        // 2^12 * 5 * G++        .quad   0x9acca683a7016bfe+        .quad   0x90505f4df2c50b6d+        .quad   0x6b610d5fcce435aa+        .quad   0x19a10d446198ff96+        .quad   0x1a07a3f496b3c397+        .quad   0x11ceaa188f4e2532+        .quad   0x7d9498d5a7751bf0+        .quad   0x19ed161f508dd8a0+        .quad   0x560a2cd687dce6ca+        .quad   0x7f3568c48664cf4d+        .quad   0x8741e95222803a38+        .quad   0x483bdab1595653fc++        // 2^12 * 6 * G++        .quad   0xfa780f148734fa49+        .quad   0x106f0b70360534e0+        .quad   0x2210776fe3e307bd+        .quad   0x3286c109dde6a0fe+        .quad   0xd6cf4d0ab4da80f6+        .quad   0x82483e45f8307fe0+        .quad   0x05005269ae6f9da4+        .quad   0x1c7052909cf7877a+        .quad   0x32ee7de2874e98d4+        .quad   0x14c362e9b97e0c60+        .quad   0x5781dcde6a60a38a+        .quad   0x217dd5eaaa7aa840++        // 2^12 * 7 * G++        .quad   0x9db7c4d0248e1eb0+        .quad   0xe07697e14d74bf52+        .quad   0x1e6a9b173c562354+        .quad   0x7fa7c21f795a4965+        .quad   0x8bdf1fb9be8c0ec8+        .quad   0x00bae7f8e30a0282+        .quad   0x4963991dad6c4f6c+        .quad   0x07058a6e5df6f60a+        .quad   0xe9eb02c4db31f67f+        .quad   0xed25fd8910bcfb2b+        .quad   0x46c8131f5c5cddb4+        .quad   0x33b21c13a0cb9bce++        // 2^12 * 8 * G++        .quad   0x360692f8087d8e31+        .quad   0xf4dcc637d27163f7+        .quad   0x25a4e62065ea5963+        .quad   0x659bf72e5ac160d9+        .quad   0x9aafb9b05ee38c5b+        .quad   0xbf9d2d4e071a13c7+        .quad   0x8eee6e6de933290a+        .quad   0x1c3bab17ae109717+        .quad   0x1c9ab216c7cab7b0+        .quad   0x7d65d37407bbc3cc+        .quad   0x52744750504a58d5+        .quad   0x09f2606b131a2990++        // 2^16 * 1 * G++        .quad   0x40e87d44744346be+        .quad   0x1d48dad415b52b25+        .quad   0x7c3a8a18a13b603e+        .quad   0x4eb728c12fcdbdf7+        .quad   0x7e234c597c6691ae+        .quad   0x64889d3d0a85b4c8+        .quad   0xdae2c90c354afae7+        .quad   0x0a871e070c6a9e1d+        .quad   0x3301b5994bbc8989+        .quad   0x736bae3a5bdd4260+        .quad   0x0d61ade219d59e3c+        .quad   0x3ee7300f2685d464++        // 2^16 * 2 * G++        .quad   0xf5d255e49e7dd6b7+        .quad   0x8016115c610b1eac+        .quad   0x3c99975d92e187ca+        .quad   0x13815762979125c2+        .quad   0x43fa7947841e7518+        .quad   0xe5c6fa59639c46d7+        .quad   0xa1065e1de3052b74+        .quad   0x7d47c6a2cfb89030+        .quad   0x3fdad0148ef0d6e0+        .quad   0x9d3e749a91546f3c+        .quad   0x71ec621026bb8157+        .quad   0x148cf58d34c9ec80++        // 2^16 * 3 * G++        .quad   0x46a492f67934f027+        .quad   0x469984bef6840aa9+        .quad   0x5ca1bc2a89611854+        .quad   0x3ff2fa1ebd5dbbd4+        .quad   0xe2572f7d9ae4756d+        .quad   0x56c345bb88f3487f+        .quad   0x9fd10b6d6960a88d+        .quad   0x278febad4eaea1b9+        .quad   0xb1aa681f8c933966+        .quad   0x8c21949c20290c98+        .quad   0x39115291219d3c52+        .quad   0x4104dd02fe9c677b++        // 2^16 * 4 * G++        .quad   0x72b2bf5e1124422a+        .quad   0xa1fa0c3398a33ab5+        .quad   0x94cb6101fa52b666+        .quad   0x2c863b00afaf53d5+        .quad   0x81214e06db096ab8+        .quad   0x21a8b6c90ce44f35+        .quad   0x6524c12a409e2af5+        .quad   0x0165b5a48efca481+        .quad   0xf190a474a0846a76+        .quad   0x12eff984cd2f7cc0+        .quad   0x695e290658aa2b8f+        .quad   0x591b67d9bffec8b8++        // 2^16 * 5 * G++        .quad   0x312f0d1c80b49bfa+        .quad   0x5979515eabf3ec8a+        .quad   0x727033c09ef01c88+        .quad   0x3de02ec7ca8f7bcb+        .quad   0x99b9b3719f18b55d+        .quad   0xe465e5faa18c641e+        .quad   0x61081136c29f05ed+        .quad   0x489b4f867030128b+        .quad   0xd232102d3aeb92ef+        .quad   0xe16253b46116a861+        .quad   0x3d7eabe7190baa24+        .quad   0x49f5fbba496cbebf++        // 2^16 * 6 * G++        .quad   0x30949a108a5bcfd4+        .quad   0xdc40dd70bc6473eb+        .quad   0x92c294c1307c0d1c+        .quad   0x5604a86dcbfa6e74+        .quad   0x155d628c1e9c572e+        .quad   0x8a4d86acc5884741+        .quad   0x91a352f6515763eb+        .quad   0x06a1a6c28867515b+        .quad   0x7288d1d47c1764b6+        .quad   0x72541140e0418b51+        .quad   0x9f031a6018acf6d1+        .quad   0x20989e89fe2742c6++        // 2^16 * 7 * G++        .quad   0x499777fd3a2dcc7f+        .quad   0x32857c2ca54fd892+        .quad   0xa279d864d207e3a0+        .quad   0x0403ed1d0ca67e29+        .quad   0x1674278b85eaec2e+        .quad   0x5621dc077acb2bdf+        .quad   0x640a4c1661cbf45a+        .quad   0x730b9950f70595d3+        .quad   0xc94b2d35874ec552+        .quad   0xc5e6c8cf98246f8d+        .quad   0xf7cb46fa16c035ce+        .quad   0x5bd7454308303dcc++        // 2^16 * 8 * G++        .quad   0x7f9ad19528b24cc2+        .quad   0x7f6b54656335c181+        .quad   0x66b8b66e4fc07236+        .quad   0x133a78007380ad83+        .quad   0x85c4932115e7792a+        .quad   0xc64c89a2bdcdddc9+        .quad   0x9d1e3da8ada3d762+        .quad   0x5bb7db123067f82c+        .quad   0x0961f467c6ca62be+        .quad   0x04ec21d6211952ee+        .quad   0x182360779bd54770+        .quad   0x740dca6d58f0e0d2++        // 2^20 * 1 * G++        .quad   0x50b70bf5d3f0af0b+        .quad   0x4feaf48ae32e71f7+        .quad   0x60e84ed3a55bbd34+        .quad   0x00ed489b3f50d1ed+        .quad   0x3906c72aed261ae5+        .quad   0x9ab68fd988e100f7+        .quad   0xf5e9059af3360197+        .quad   0x0e53dc78bf2b6d47+        .quad   0xb90829bf7971877a+        .quad   0x5e4444636d17e631+        .quad   0x4d05c52e18276893+        .quad   0x27632d9a5a4a4af5++        // 2^20 * 2 * G++        .quad   0xd11ff05154b260ce+        .quad   0xd86dc38e72f95270+        .quad   0x601fcd0d267cc138+        .quad   0x2b67916429e90ccd+        .quad   0xa98285d187eaffdb+        .quad   0xa5b4fbbbd8d0a864+        .quad   0xb658f27f022663f7+        .quad   0x3bbc2b22d99ce282+        .quad   0xb917c952583c0a58+        .quad   0x653ff9b80fe4c6f3+        .quad   0x9b0da7d7bcdf3c0c+        .quad   0x43a0eeb6ab54d60e++        // 2^20 * 3 * G++        .quad   0x396966a46d4a5487+        .quad   0xf811a18aac2bb3ba+        .quad   0x66e4685b5628b26b+        .quad   0x70a477029d929b92+        .quad   0x3ac6322357875fe8+        .quad   0xd9d4f4ecf5fbcb8f+        .quad   0x8dee8493382bb620+        .quad   0x50c5eaa14c799fdc+        .quad   0xdd0edc8bd6f2fb3c+        .quad   0x54c63aa79cc7b7a0+        .quad   0xae0b032b2c8d9f1a+        .quad   0x6f9ce107602967fb++        // 2^20 * 4 * G++        .quad   0xad1054b1cde1c22a+        .quad   0xc4a8e90248eb32df+        .quad   0x5f3e7b33accdc0ea+        .quad   0x72364713fc79963e+        .quad   0x139693063520e0b5+        .quad   0x437fcf7c88ea03fe+        .quad   0xf7d4c40bd3c959bc+        .quad   0x699154d1f893ded9+        .quad   0x315d5c75b4b27526+        .quad   0xcccb842d0236daa5+        .quad   0x22f0c8a3345fee8e+        .quad   0x73975a617d39dbed++        // 2^20 * 5 * G++        .quad   0xe4024df96375da10+        .quad   0x78d3251a1830c870+        .quad   0x902b1948658cd91c+        .quad   0x7e18b10b29b7438a+        .quad   0x6f37f392f4433e46+        .quad   0x0e19b9a11f566b18+        .quad   0x220fb78a1fd1d662+        .quad   0x362a4258a381c94d+        .quad   0x9071d9132b6beb2f+        .quad   0x0f26e9ad28418247+        .quad   0xeab91ec9bdec925d+        .quad   0x4be65bc8f48af2de++        // 2^20 * 6 * G++        .quad   0x78487feba36e7028+        .quad   0x5f3f13001dd8ce34+        .quad   0x934fb12d4b30c489+        .quad   0x056c244d397f0a2b+        .quad   0x1d50fba257c26234+        .quad   0x7bd4823adeb0678b+        .quad   0xc2b0dc6ea6538af5+        .quad   0x5665eec6351da73e+        .quad   0xdb3ee00943bfb210+        .quad   0x4972018720800ac2+        .quad   0x26ab5d6173bd8667+        .quad   0x20b209c2ab204938++        // 2^20 * 7 * G++        .quad   0x549e342ac07fb34b+        .quad   0x02d8220821373d93+        .quad   0xbc262d70acd1f567+        .quad   0x7a92c9fdfbcac784+        .quad   0x1fcca94516bd3289+        .quad   0x448d65aa41420428+        .quad   0x59c3b7b216a55d62+        .quad   0x49992cc64e612cd8+        .quad   0x65bd1bea70f801de+        .quad   0x1befb7c0fe49e28a+        .quad   0xa86306cdb1b2ae4a+        .quad   0x3b7ac0cd265c2a09++        // 2^20 * 8 * G++        .quad   0x822bee438c01bcec+        .quad   0x530cb525c0fbc73b+        .quad   0x48519034c1953fe9+        .quad   0x265cc261e09a0f5b+        .quad   0xf0d54e4f22ed39a7+        .quad   0xa2aae91e5608150a+        .quad   0xf421b2e9eddae875+        .quad   0x31bc531d6b7de992+        .quad   0xdf3d134da980f971+        .quad   0x7a4fb8d1221a22a7+        .quad   0x3df7d42035aad6d8+        .quad   0x2a14edcc6a1a125e++        // 2^24 * 1 * G++        .quad   0xdf48ee0752cfce4e+        .quad   0xc3fffaf306ec08b7+        .quad   0x05710b2ab95459c4+        .quad   0x161d25fa963ea38d+        .quad   0x231a8c570478433c+        .quad   0xb7b5270ec281439d+        .quad   0xdbaa99eae3d9079f+        .quad   0x2c03f5256c2b03d9+        .quad   0x790f18757b53a47d+        .quad   0x307b0130cf0c5879+        .quad   0x31903d77257ef7f9+        .quad   0x699468bdbd96bbaf++        // 2^24 * 2 * G++        .quad   0xbd1f2f46f4dafecf+        .quad   0x7cef0114a47fd6f7+        .quad   0xd31ffdda4a47b37f+        .quad   0x525219a473905785+        .quad   0xd8dd3de66aa91948+        .quad   0x485064c22fc0d2cc+        .quad   0x9b48246634fdea2f+        .quad   0x293e1c4e6c4a2e3a+        .quad   0x376e134b925112e1+        .quad   0x703778b5dca15da0+        .quad   0xb04589af461c3111+        .quad   0x5b605c447f032823++        // 2^24 * 3 * G++        .quad   0xb965805920c47c89+        .quad   0xe7f0100c923b8fcc+        .quad   0x0001256502e2ef77+        .quad   0x24a76dcea8aeb3ee+        .quad   0x3be9fec6f0e7f04c+        .quad   0x866a579e75e34962+        .quad   0x5542ef161e1de61a+        .quad   0x2f12fef4cc5abdd5+        .quad   0x0a4522b2dfc0c740+        .quad   0x10d06e7f40c9a407+        .quad   0xc6cf144178cff668+        .quad   0x5e607b2518a43790++        // 2^24 * 4 * G++        .quad   0x58b31d8f6cdf1818+        .quad   0x35cfa74fc36258a2+        .quad   0xe1b3ff4f66e61d6e+        .quad   0x5067acab6ccdd5f7+        .quad   0xa02c431ca596cf14+        .quad   0xe3c42d40aed3e400+        .quad   0xd24526802e0f26db+        .quad   0x201f33139e457068+        .quad   0xfd527f6b08039d51+        .quad   0x18b14964017c0006+        .quad   0xd5220eb02e25a4a8+        .quad   0x397cba8862460375++        // 2^24 * 5 * G++        .quad   0x30c13093f05959b2+        .quad   0xe23aa18de9a97976+        .quad   0x222fd491721d5e26+        .quad   0x2339d320766e6c3a+        .quad   0x7815c3fbc81379e7+        .quad   0xa6619420dde12af1+        .quad   0xffa9c0f885a8fdd5+        .quad   0x771b4022c1e1c252+        .quad   0xd87dd986513a2fa7+        .quad   0xf5ac9b71f9d4cf08+        .quad   0xd06bc31b1ea283b3+        .quad   0x331a189219971a76++        // 2^24 * 6 * G++        .quad   0xf5166f45fb4f80c6+        .quad   0x9c36c7de61c775cf+        .quad   0xe3d4e81b9041d91c+        .quad   0x31167c6b83bdfe21+        .quad   0x26512f3a9d7572af+        .quad   0x5bcbe28868074a9e+        .quad   0x84edc1c11180f7c4+        .quad   0x1ac9619ff649a67b+        .quad   0xf22b3842524b1068+        .quad   0x5068343bee9ce987+        .quad   0xfc9d71844a6250c8+        .quad   0x612436341f08b111++        // 2^24 * 7 * G++        .quad   0xd99d41db874e898d+        .quad   0x09fea5f16c07dc20+        .quad   0x793d2c67d00f9bbc+        .quad   0x46ebe2309e5eff40+        .quad   0x8b6349e31a2d2638+        .quad   0x9ddfb7009bd3fd35+        .quad   0x7f8bf1b8a3a06ba4+        .quad   0x1522aa3178d90445+        .quad   0x2c382f5369614938+        .quad   0xdafe409ab72d6d10+        .quad   0xe8c83391b646f227+        .quad   0x45fe70f50524306c++        // 2^24 * 8 * G++        .quad   0xda4875a6960c0b8c+        .quad   0x5b68d076ef0e2f20+        .quad   0x07fb51cf3d0b8fd4+        .quad   0x428d1623a0e392d4+        .quad   0x62f24920c8951491+        .quad   0x05f007c83f630ca2+        .quad   0x6fbb45d2f5c9d4b8+        .quad   0x16619f6db57a2245+        .quad   0x084f4a4401a308fd+        .quad   0xa82219c376a5caac+        .quad   0xdeb8de4643d1bc7d+        .quad   0x1d81592d60bd38c6++        // 2^28 * 1 * G++        .quad   0xd833d7beec2a4c38+        .quad   0x2c9162830acc20ed+        .quad   0xe93a47aa92df7581+        .quad   0x702d67a3333c4a81+        .quad   0x3a4a369a2f89c8a1+        .quad   0x63137a1d7c8de80d+        .quad   0xbcac008a78eda015+        .quad   0x2cb8b3a5b483b03f+        .quad   0x36e417cbcb1b90a1+        .quad   0x33b3ddaa7f11794e+        .quad   0x3f510808885bc607+        .quad   0x24141dc0e6a8020d++        // 2^28 * 2 * G++        .quad   0x59f73c773fefee9d+        .quad   0xb3f1ef89c1cf989d+        .quad   0xe35dfb42e02e545f+        .quad   0x5766120b47a1b47c+        .quad   0x91925dccbd83157d+        .quad   0x3ca1205322cc8094+        .quad   0x28e57f183f90d6e4+        .quad   0x1a4714cede2e767b+        .quad   0xdb20ba0fb8b6b7ff+        .quad   0xb732c3b677511fa1+        .quad   0xa92b51c099f02d89+        .quad   0x4f3875ad489ca5f1++        // 2^28 * 3 * G++        .quad   0xc7fc762f4932ab22+        .quad   0x7ac0edf72f4c3c1b+        .quad   0x5f6b55aa9aa895e8+        .quad   0x3680274dad0a0081+        .quad   0x79ed13f6ee73eec0+        .quad   0xa5c6526d69110bb1+        .quad   0xe48928c38603860c+        .quad   0x722a1446fd7059f5+        .quad   0xd0959fe9a8cf8819+        .quad   0xd0a995508475a99c+        .quad   0x6eac173320b09cc5+        .quad   0x628ecf04331b1095++        // 2^28 * 4 * G++        .quad   0x98bcb118a9d0ddbc+        .quad   0xee449e3408b4802b+        .quad   0x87089226b8a6b104+        .quad   0x685f349a45c7915d+        .quad   0x9b41acf85c74ccf1+        .quad   0xb673318108265251+        .quad   0x99c92aed11adb147+        .quad   0x7a47d70d34ecb40f+        .quad   0x60a0c4cbcc43a4f5+        .quad   0x775c66ca3677bea9+        .quad   0xa17aa1752ff8f5ed+        .quad   0x11ded9020e01fdc0++        // 2^28 * 5 * G++        .quad   0x890e7809caefe704+        .quad   0x8728296de30e8c6c+        .quad   0x4c5cd2a392aeb1c9+        .quad   0x194263d15771531f+        .quad   0x471f95b03bea93b7+        .quad   0x0552d7d43313abd3+        .quad   0xbd9370e2e17e3f7b+        .quad   0x7b120f1db20e5bec+        .quad   0x17d2fb3d86502d7a+        .quad   0xb564d84450a69352+        .quad   0x7da962c8a60ed75d+        .quad   0x00d0f85b318736aa++        // 2^28 * 6 * G++        .quad   0x978b142e777c84fd+        .quad   0xf402644705a8c062+        .quad   0xa67ad51be7e612c7+        .quad   0x2f7b459698dd6a33+        .quad   0xa6753c1efd7621c1+        .quad   0x69c0b4a7445671f5+        .quad   0x971f527405b23c11+        .quad   0x387bc74851a8c7cd+        .quad   0x81894b4d4a52a9a8+        .quad   0xadd93e12f6b8832f+        .quad   0x184d8548b61bd638+        .quad   0x3f1c62dbd6c9f6cd++        // 2^28 * 7 * G++        .quad   0x2e8f1f0091910c1f+        .quad   0xa4df4fe0bff2e12c+        .quad   0x60c6560aee927438+        .quad   0x6338283facefc8fa+        .quad   0x3fad3e40148f693d+        .quad   0x052656e194eb9a72+        .quad   0x2f4dcbfd184f4e2f+        .quad   0x406f8db1c482e18b+        .quad   0x9e630d2c7f191ee4+        .quad   0x4fbf8301bc3ff670+        .quad   0x787d8e4e7afb73c4+        .quad   0x50d83d5be8f58fa5++        // 2^28 * 8 * G++        .quad   0x85683916c11a1897+        .quad   0x2d69a4efe506d008+        .quad   0x39af1378f664bd01+        .quad   0x65942131361517c6+        .quad   0xc0accf90b4d3b66d+        .quad   0xa7059de561732e60+        .quad   0x033d1f7870c6b0ba+        .quad   0x584161cd26d946e4+        .quad   0xbbf2b1a072d27ca2+        .quad   0xbf393c59fbdec704+        .quad   0xe98dbbcee262b81e+        .quad   0x02eebd0b3029b589++        // 2^32 * 1 * G++        .quad   0x61368756a60dac5f+        .quad   0x17e02f6aebabdc57+        .quad   0x7f193f2d4cce0f7d+        .quad   0x20234a7789ecdcf0+        .quad   0x8765b69f7b85c5e8+        .quad   0x6ff0678bd168bab2+        .quad   0x3a70e77c1d330f9b+        .quad   0x3a5f6d51b0af8e7c+        .quad   0x76d20db67178b252+        .quad   0x071c34f9d51ed160+        .quad   0xf62a4a20b3e41170+        .quad   0x7cd682353cffe366++        // 2^32 * 2 * G++        .quad   0x0be1a45bd887fab6+        .quad   0x2a846a32ba403b6e+        .quad   0xd9921012e96e6000+        .quad   0x2838c8863bdc0943+        .quad   0xa665cd6068acf4f3+        .quad   0x42d92d183cd7e3d3+        .quad   0x5759389d336025d9+        .quad   0x3ef0253b2b2cd8ff+        .quad   0xd16bb0cf4a465030+        .quad   0xfa496b4115c577ab+        .quad   0x82cfae8af4ab419d+        .quad   0x21dcb8a606a82812++        // 2^32 * 3 * G++        .quad   0x5c6004468c9d9fc8+        .quad   0x2540096ed42aa3cb+        .quad   0x125b4d4c12ee2f9c+        .quad   0x0bc3d08194a31dab+        .quad   0x9a8d00fabe7731ba+        .quad   0x8203607e629e1889+        .quad   0xb2cc023743f3d97f+        .quad   0x5d840dbf6c6f678b+        .quad   0x706e380d309fe18b+        .quad   0x6eb02da6b9e165c7+        .quad   0x57bbba997dae20ab+        .quad   0x3a4276232ac196dd++        // 2^32 * 4 * G++        .quad   0x4b42432c8a7084fa+        .quad   0x898a19e3dfb9e545+        .quad   0xbe9f00219c58e45d+        .quad   0x1ff177cea16debd1+        .quad   0x3bf8c172db447ecb+        .quad   0x5fcfc41fc6282dbd+        .quad   0x80acffc075aa15fe+        .quad   0x0770c9e824e1a9f9+        .quad   0xcf61d99a45b5b5fd+        .quad   0x860984e91b3a7924+        .quad   0xe7300919303e3e89+        .quad   0x39f264fd41500b1e++        // 2^32 * 5 * G++        .quad   0xa7ad3417dbe7e29c+        .quad   0xbd94376a2b9c139c+        .quad   0xa0e91b8e93597ba9+        .quad   0x1712d73468889840+        .quad   0xd19b4aabfe097be1+        .quad   0xa46dfce1dfe01929+        .quad   0xc3c908942ca6f1ff+        .quad   0x65c621272c35f14e+        .quad   0xe72b89f8ce3193dd+        .quad   0x4d103356a125c0bb+        .quad   0x0419a93d2e1cfe83+        .quad   0x22f9800ab19ce272++        // 2^32 * 6 * G++        .quad   0x605a368a3e9ef8cb+        .quad   0xe3e9c022a5504715+        .quad   0x553d48b05f24248f+        .quad   0x13f416cd647626e5+        .quad   0x42029fdd9a6efdac+        .quad   0xb912cebe34a54941+        .quad   0x640f64b987bdf37b+        .quad   0x4171a4d38598cab4+        .quad   0xfa2758aa99c94c8c+        .quad   0x23006f6fb000b807+        .quad   0xfbd291ddadda5392+        .quad   0x508214fa574bd1ab++        // 2^32 * 7 * G++        .quad   0xc20269153ed6fe4b+        .quad   0xa65a6739511d77c4+        .quad   0xcbde26462c14af94+        .quad   0x22f960ec6faba74b+        .quad   0x461a15bb53d003d6+        .quad   0xb2102888bcf3c965+        .quad   0x27c576756c683a5a+        .quad   0x3a7758a4c86cb447+        .quad   0x548111f693ae5076+        .quad   0x1dae21df1dfd54a6+        .quad   0x12248c90f3115e65+        .quad   0x5d9fd15f8de7f494++        // 2^32 * 8 * G++        .quad   0x031408d36d63727f+        .quad   0x6a379aefd7c7b533+        .quad   0xa9e18fc5ccaee24b+        .quad   0x332f35914f8fbed3+        .quad   0x3f244d2aeed7521e+        .quad   0x8e3a9028432e9615+        .quad   0xe164ba772e9c16d4+        .quad   0x3bc187fa47eb98d8+        .quad   0x6d470115ea86c20c+        .quad   0x998ab7cb6c46d125+        .quad   0xd77832b53a660188+        .quad   0x450d81ce906fba03++        // 2^36 * 1 * G++        .quad   0xf8ae4d2ad8453902+        .quad   0x7018058ee8db2d1d+        .quad   0xaab3995fc7d2c11e+        .quad   0x53b16d2324ccca79+        .quad   0x23264d66b2cae0b5+        .quad   0x7dbaed33ebca6576+        .quad   0x030ebed6f0d24ac8+        .quad   0x2a887f78f7635510+        .quad   0x2a23b9e75c012d4f+        .quad   0x0c974651cae1f2ea+        .quad   0x2fb63273675d70ca+        .quad   0x0ba7250b864403f5++        // 2^36 * 2 * G++        .quad   0xbb0d18fd029c6421+        .quad   0xbc2d142189298f02+        .quad   0x8347f8e68b250e96+        .quad   0x7b9f2fe8032d71c9+        .quad   0xdd63589386f86d9c+        .quad   0x61699176e13a85a4+        .quad   0x2e5111954eaa7d57+        .quad   0x32c21b57fb60bdfb+        .quad   0xd87823cd319e0780+        .quad   0xefc4cfc1897775c5+        .quad   0x4854fb129a0ab3f7+        .quad   0x12c49d417238c371++        // 2^36 * 3 * G++        .quad   0x0950b533ffe83769+        .quad   0x21861c1d8e1d6bd1+        .quad   0xf022d8381302e510+        .quad   0x2509200c6391cab4+        .quad   0x09b3a01783799542+        .quad   0x626dd08faad5ee3f+        .quad   0xba00bceeeb70149f+        .quad   0x1421b246a0a444c9+        .quad   0x4aa43a8e8c24a7c7+        .quad   0x04c1f540d8f05ef5+        .quad   0xadba5e0c0b3eb9dc+        .quad   0x2ab5504448a49ce3++        // 2^36 * 4 * G++        .quad   0x2ed227266f0f5dec+        .quad   0x9824ee415ed50824+        .quad   0x807bec7c9468d415+        .quad   0x7093bae1b521e23f+        .quad   0xdc07ac631c5d3afa+        .quad   0x58615171f9df8c6c+        .quad   0x72a079d89d73e2b0+        .quad   0x7301f4ceb4eae15d+        .quad   0x6409e759d6722c41+        .quad   0xa674e1cf72bf729b+        .quad   0xbc0a24eb3c21e569+        .quad   0x390167d24ebacb23++        // 2^36 * 5 * G++        .quad   0x27f58e3bba353f1c+        .quad   0x4c47764dbf6a4361+        .quad   0xafbbc4e56e562650+        .quad   0x07db2ee6aae1a45d+        .quad   0xd7bb054ba2f2120b+        .quad   0xe2b9ceaeb10589b7+        .quad   0x3fe8bac8f3c0edbe+        .quad   0x4cbd40767112cb69+        .quad   0x0b603cc029c58176+        .quad   0x5988e3825cb15d61+        .quad   0x2bb61413dcf0ad8d+        .quad   0x7b8eec6c74183287++        // 2^36 * 6 * G++        .quad   0xe4ca40782cd27cb0+        .quad   0xdaf9c323fbe967bd+        .quad   0xb29bd34a8ad41e9e+        .quad   0x72810497626ede4d+        .quad   0x32fee570fc386b73+        .quad   0xda8b0141da3a8cc7+        .quad   0x975ffd0ac8968359+        .quad   0x6ee809a1b132a855+        .quad   0x9444bb31fcfd863a+        .quad   0x2fe3690a3e4e48c5+        .quad   0xdc29c867d088fa25+        .quad   0x13bd1e38d173292e++        // 2^36 * 7 * G++        .quad   0xd32b4cd8696149b5+        .quad   0xe55937d781d8aab7+        .quad   0x0bcb2127ae122b94+        .quad   0x41e86fcfb14099b0+        .quad   0x223fb5cf1dfac521+        .quad   0x325c25316f554450+        .quad   0x030b98d7659177ac+        .quad   0x1ed018b64f88a4bd+        .quad   0x3630dfa1b802a6b0+        .quad   0x880f874742ad3bd5+        .quad   0x0af90d6ceec5a4d4+        .quad   0x746a247a37cdc5d9++        // 2^36 * 8 * G++        .quad   0xd531b8bd2b7b9af6+        .quad   0x5005093537fc5b51+        .quad   0x232fcf25c593546d+        .quad   0x20a365142bb40f49+        .quad   0x6eccd85278d941ed+        .quad   0x2254ae83d22f7843+        .quad   0xc522d02e7bbfcdb7+        .quad   0x681e3351bff0e4e2+        .quad   0x8b64b59d83034f45+        .quad   0x2f8b71f21fa20efb+        .quad   0x69249495ba6550e4+        .quad   0x539ef98e45d5472b++        // 2^40 * 1 * G++        .quad   0x6e7bb6a1a6205275+        .quad   0xaa4f21d7413c8e83+        .quad   0x6f56d155e88f5cb2+        .quad   0x2de25d4ba6345be1+        .quad   0xd074d8961cae743f+        .quad   0xf86d18f5ee1c63ed+        .quad   0x97bdc55be7f4ed29+        .quad   0x4cbad279663ab108+        .quad   0x80d19024a0d71fcd+        .quad   0xc525c20afb288af8+        .quad   0xb1a3974b5f3a6419+        .quad   0x7d7fbcefe2007233++        // 2^40 * 2 * G++        .quad   0xfaef1e6a266b2801+        .quad   0x866c68c4d5739f16+        .quad   0xf68a2fbc1b03762c+        .quad   0x5975435e87b75a8d+        .quad   0xcd7c5dc5f3c29094+        .quad   0xc781a29a2a9105ab+        .quad   0x80c61d36421c3058+        .quad   0x4f9cd196dcd8d4d7+        .quad   0x199297d86a7b3768+        .quad   0xd0d058241ad17a63+        .quad   0xba029cad5c1c0c17+        .quad   0x7ccdd084387a0307++        // 2^40 * 3 * G++        .quad   0xdca6422c6d260417+        .quad   0xae153d50948240bd+        .quad   0xa9c0c1b4fb68c677+        .quad   0x428bd0ed61d0cf53+        .quad   0x9b0c84186760cc93+        .quad   0xcdae007a1ab32a99+        .quad   0xa88dec86620bda18+        .quad   0x3593ca848190ca44+        .quad   0x9213189a5e849aa7+        .quad   0xd4d8c33565d8facd+        .quad   0x8c52545b53fdbbd1+        .quad   0x27398308da2d63e6++        // 2^40 * 4 * G++        .quad   0x42c38d28435ed413+        .quad   0xbd50f3603278ccc9+        .quad   0xbb07ab1a79da03ef+        .quad   0x269597aebe8c3355+        .quad   0xb9a10e4c0a702453+        .quad   0x0fa25866d57d1bde+        .quad   0xffb9d9b5cd27daf7+        .quad   0x572c2945492c33fd+        .quad   0xc77fc745d6cd30be+        .quad   0xe4dfe8d3e3baaefb+        .quad   0xa22c8830aa5dda0c+        .quad   0x7f985498c05bca80++        // 2^40 * 5 * G++        .quad   0x3849ce889f0be117+        .quad   0x8005ad1b7b54a288+        .quad   0x3da3c39f23fc921c+        .quad   0x76c2ec470a31f304+        .quad   0xd35615520fbf6363+        .quad   0x08045a45cf4dfba6+        .quad   0xeec24fbc873fa0c2+        .quad   0x30f2653cd69b12e7+        .quad   0x8a08c938aac10c85+        .quad   0x46179b60db276bcb+        .quad   0xa920c01e0e6fac70+        .quad   0x2f1273f1596473da++        // 2^40 * 6 * G++        .quad   0x4739fc7c8ae01e11+        .quad   0xfd5274904a6aab9f+        .quad   0x41d98a8287728f2e+        .quad   0x5d9e572ad85b69f2+        .quad   0x30488bd755a70bc0+        .quad   0x06d6b5a4f1d442e7+        .quad   0xead1a69ebc596162+        .quad   0x38ac1997edc5f784+        .quad   0x0666b517a751b13b+        .quad   0x747d06867e9b858c+        .quad   0xacacc011454dde49+        .quad   0x22dfcd9cbfe9e69c++        // 2^40 * 7 * G++        .quad   0x8ddbd2e0c30d0cd9+        .quad   0xad8e665facbb4333+        .quad   0x8f6b258c322a961f+        .quad   0x6b2916c05448c1c7+        .quad   0x56ec59b4103be0a1+        .quad   0x2ee3baecd259f969+        .quad   0x797cb29413f5cd32+        .quad   0x0fe9877824cde472+        .quad   0x7edb34d10aba913b+        .quad   0x4ea3cd822e6dac0e+        .quad   0x66083dff6578f815+        .quad   0x4c303f307ff00a17++        // 2^40 * 8 * G++        .quad   0xd30a3bd617b28c85+        .quad   0xc5d377b739773bea+        .quad   0xc6c6e78c1e6a5cbf+        .quad   0x0d61b8f78b2ab7c4+        .quad   0x29fc03580dd94500+        .quad   0xecd27aa46fbbec93+        .quad   0x130a155fc2e2a7f8+        .quad   0x416b151ab706a1d5+        .quad   0x56a8d7efe9c136b0+        .quad   0xbd07e5cd58e44b20+        .quad   0xafe62fda1b57e0ab+        .quad   0x191a2af74277e8d2++        // 2^44 * 1 * G++        .quad   0xd550095bab6f4985+        .quad   0x04f4cd5b4fbfaf1a+        .quad   0x9d8e2ed12a0c7540+        .quad   0x2bc24e04b2212286+        .quad   0x09d4b60b2fe09a14+        .quad   0xc384f0afdbb1747e+        .quad   0x58e2ea8978b5fd6e+        .quad   0x519ef577b5e09b0a+        .quad   0x1863d7d91124cca9+        .quad   0x7ac08145b88a708e+        .quad   0x2bcd7309857031f5+        .quad   0x62337a6e8ab8fae5++        // 2^44 * 2 * G++        .quad   0x4bcef17f06ffca16+        .quad   0xde06e1db692ae16a+        .quad   0x0753702d614f42b0+        .quad   0x5f6041b45b9212d0+        .quad   0xd1ab324e1b3a1273+        .quad   0x18947cf181055340+        .quad   0x3b5d9567a98c196e+        .quad   0x7fa00425802e1e68+        .quad   0x7d531574028c2705+        .quad   0x80317d69db0d75fe+        .quad   0x30fface8ef8c8ddd+        .quad   0x7e9de97bb6c3e998++        // 2^44 * 3 * G++        .quad   0x1558967b9e6585a3+        .quad   0x97c99ce098e98b92+        .quad   0x10af149b6eb3adad+        .quad   0x42181fe8f4d38cfa+        .quad   0xf004be62a24d40dd+        .quad   0xba0659910452d41f+        .quad   0x81c45ee162a44234+        .quad   0x4cb829d8a22266ef+        .quad   0x1dbcaa8407b86681+        .quad   0x081f001e8b26753b+        .quad   0x3cd7ce6a84048e81+        .quad   0x78af11633f25f22c++        // 2^44 * 4 * G++        .quad   0x8416ebd40b50babc+        .quad   0x1508722628208bee+        .quad   0xa3148fafb9c1c36d+        .quad   0x0d07daacd32d7d5d+        .quad   0x3241c00e7d65318c+        .quad   0xe6bee5dcd0e86de7+        .quad   0x118b2dc2fbc08c26+        .quad   0x680d04a7fc603dc3+        .quad   0xf9c2414a695aa3eb+        .quad   0xdaa42c4c05a68f21+        .quad   0x7c6c23987f93963e+        .quad   0x210e8cd30c3954e3++        // 2^44 * 5 * G++        .quad   0xac4201f210a71c06+        .quad   0x6a65e0aef3bfb021+        .quad   0xbc42c35c393632f7+        .quad   0x56ea8db1865f0742+        .quad   0x2b50f16137fe6c26+        .quad   0xe102bcd856e404d8+        .quad   0x12b0f1414c561f6b+        .quad   0x51b17bc8d028ec91+        .quad   0xfff5fb4bcf535119+        .quad   0xf4989d79df1108a0+        .quad   0xbdfcea659a3ba325+        .quad   0x18a11f1174d1a6f2++        // 2^44 * 6 * G++        .quad   0x407375ab3f6bba29+        .quad   0x9ec3b6d8991e482e+        .quad   0x99c80e82e55f92e9+        .quad   0x307c13b6fb0c0ae1+        .quad   0xfbd63cdad27a5f2c+        .quad   0xf00fc4bc8aa106d7+        .quad   0x53fb5c1a8e64a430+        .quad   0x04eaabe50c1a2e85+        .quad   0x24751021cb8ab5e7+        .quad   0xfc2344495c5010eb+        .quad   0x5f1e717b4e5610a1+        .quad   0x44da5f18c2710cd5++        // 2^44 * 7 * G++        .quad   0x033cc55ff1b82eb5+        .quad   0xb15ae36d411cae52+        .quad   0xba40b6198ffbacd3+        .quad   0x768edce1532e861f+        .quad   0x9156fe6b89d8eacc+        .quad   0xe6b79451e23126a1+        .quad   0xbd7463d93944eb4e+        .quad   0x726373f6767203ae+        .quad   0xe305ca72eb7ef68a+        .quad   0x662cf31f70eadb23+        .quad   0x18f026fdb4c45b68+        .quad   0x513b5384b5d2ecbd++        // 2^44 * 8 * G++        .quad   0x46d46280c729989e+        .quad   0x4b93fbd05368a5dd+        .quad   0x63df3f81d1765a89+        .quad   0x34cebd64b9a0a223+        .quad   0x5e2702878af34ceb+        .quad   0x900b0409b946d6ae+        .quad   0x6512ebf7dabd8512+        .quad   0x61d9b76988258f81+        .quad   0xa6c5a71349b7d94b+        .quad   0xa3f3d15823eb9446+        .quad   0x0416fbd277484834+        .quad   0x69d45e6f2c70812f++        // 2^48 * 1 * G++        .quad   0xce16f74bc53c1431+        .quad   0x2b9725ce2072edde+        .quad   0xb8b9c36fb5b23ee7+        .quad   0x7e2e0e450b5cc908+        .quad   0x9fe62b434f460efb+        .quad   0xded303d4a63607d6+        .quad   0xf052210eb7a0da24+        .quad   0x237e7dbe00545b93+        .quad   0x013575ed6701b430+        .quad   0x231094e69f0bfd10+        .quad   0x75320f1583e47f22+        .quad   0x71afa699b11155e3++        // 2^48 * 2 * G++        .quad   0x65ce6f9b3953b61d+        .quad   0xc65839eaafa141e6+        .quad   0x0f435ffda9f759fe+        .quad   0x021142e9c2b1c28e+        .quad   0xea423c1c473b50d6+        .quad   0x51e87a1f3b38ef10+        .quad   0x9b84bf5fb2c9be95+        .quad   0x00731fbc78f89a1c+        .quad   0xe430c71848f81880+        .quad   0xbf960c225ecec119+        .quad   0xb6dae0836bba15e3+        .quad   0x4c4d6f3347e15808++        // 2^48 * 3 * G++        .quad   0x18f7eccfc17d1fc9+        .quad   0x6c75f5a651403c14+        .quad   0xdbde712bf7ee0cdf+        .quad   0x193fddaaa7e47a22+        .quad   0x2f0cddfc988f1970+        .quad   0x6b916227b0b9f51b+        .quad   0x6ec7b6c4779176be+        .quad   0x38bf9500a88f9fa8+        .quad   0x1fd2c93c37e8876f+        .quad   0xa2f61e5a18d1462c+        .quad   0x5080f58239241276+        .quad   0x6a6fb99ebf0d4969++        // 2^48 * 4 * G++        .quad   0x6a46c1bb560855eb+        .quad   0x2416bb38f893f09d+        .quad   0xd71d11378f71acc1+        .quad   0x75f76914a31896ea+        .quad   0xeeb122b5b6e423c6+        .quad   0x939d7010f286ff8e+        .quad   0x90a92a831dcf5d8c+        .quad   0x136fda9f42c5eb10+        .quad   0xf94cdfb1a305bdd1+        .quad   0x0f364b9d9ff82c08+        .quad   0x2a87d8a5c3bb588a+        .quad   0x022183510be8dcba++        // 2^48 * 5 * G++        .quad   0x4af766385ead2d14+        .quad   0xa08ed880ca7c5830+        .quad   0x0d13a6e610211e3d+        .quad   0x6a071ce17b806c03+        .quad   0x9d5a710143307a7f+        .quad   0xb063de9ec47da45f+        .quad   0x22bbfe52be927ad3+        .quad   0x1387c441fd40426c+        .quad   0xb5d3c3d187978af8+        .quad   0x722b5a3d7f0e4413+        .quad   0x0d7b4848bb477ca0+        .quad   0x3171b26aaf1edc92++        // 2^48 * 6 * G++        .quad   0xa92f319097564ca8+        .quad   0xff7bb84c2275e119+        .quad   0x4f55fe37a4875150+        .quad   0x221fd4873cf0835a+        .quad   0xa60db7d8b28a47d1+        .quad   0xa6bf14d61770a4f1+        .quad   0xd4a1f89353ddbd58+        .quad   0x6c514a63344243e9+        .quad   0x2322204f3a156341+        .quad   0xfb73e0e9ba0a032d+        .quad   0xfce0dd4c410f030e+        .quad   0x48daa596fb924aaa++        // 2^48 * 7 * G++        .quad   0x6eca8e665ca59cc7+        .quad   0xa847254b2e38aca0+        .quad   0x31afc708d21e17ce+        .quad   0x676dd6fccad84af7+        .quad   0x14f61d5dc84c9793+        .quad   0x9941f9e3ef418206+        .quad   0xcdf5b88f346277ac+        .quad   0x58c837fa0e8a79a9+        .quad   0x0cf9688596fc9058+        .quad   0x1ddcbbf37b56a01b+        .quad   0xdcc2e77d4935d66a+        .quad   0x1c4f73f2c6a57f0a++        // 2^48 * 8 * G++        .quad   0x0e7a4fbd305fa0bb+        .quad   0x829d4ce054c663ad+        .quad   0xf421c3832fe33848+        .quad   0x795ac80d1bf64c42+        .quad   0xb36e706efc7c3484+        .quad   0x73dfc9b4c3c1cf61+        .quad   0xeb1d79c9781cc7e5+        .quad   0x70459adb7daf675c+        .quad   0x1b91db4991b42bb3+        .quad   0x572696234b02dcca+        .quad   0x9fdf9ee51f8c78dc+        .quad   0x5fe162848ce21fd3++        // 2^52 * 1 * G++        .quad   0xe2790aae4d077c41+        .quad   0x8b938270db7469a3+        .quad   0x6eb632dc8abd16a2+        .quad   0x720814ecaa064b72+        .quad   0x315c29c795115389+        .quad   0xd7e0e507862f74ce+        .quad   0x0c4a762185927432+        .quad   0x72de6c984a25a1e4+        .quad   0xae9ab553bf6aa310+        .quad   0x050a50a9806d6e1b+        .quad   0x92bb7403adff5139+        .quad   0x0394d27645be618b++        // 2^52 * 2 * G++        .quad   0x4d572251857eedf4+        .quad   0xe3724edde19e93c5+        .quad   0x8a71420e0b797035+        .quad   0x3b3c833687abe743+        .quad   0xf5396425b23545a4+        .quad   0x15a7a27e98fbb296+        .quad   0xab6c52bc636fdd86+        .quad   0x79d995a8419334ee+        .quad   0xcd8a8ea61195dd75+        .quad   0xa504d8a81dd9a82f+        .quad   0x540dca81a35879b6+        .quad   0x60dd16a379c86a8a++        // 2^52 * 3 * G++        .quad   0x35a2c8487381e559+        .quad   0x596ffea6d78082cb+        .quad   0xcb9771ebdba7b653+        .quad   0x5a08b5019b4da685+        .quad   0x3501d6f8153e47b8+        .quad   0xb7a9675414a2f60c+        .quad   0x112ee8b6455d9523+        .quad   0x4e62a3c18112ea8a+        .quad   0xc8d4ac04516ab786+        .quad   0x595af3215295b23d+        .quad   0xd6edd234db0230c1+        .quad   0x0929efe8825b41cc++        // 2^52 * 4 * G++        .quad   0x5f0601d1cbd0f2d3+        .quad   0x736e412f6132bb7f+        .quad   0x83604432238dde87+        .quad   0x1e3a5272f5c0753c+        .quad   0x8b3172b7ad56651d+        .quad   0x01581b7a3fabd717+        .quad   0x2dc94df6424df6e4+        .quad   0x30376e5d2c29284f+        .quad   0xd2918da78159a59c+        .quad   0x6bdc1cd93f0713f3+        .quad   0x565f7a934acd6590+        .quad   0x53daacec4cb4c128++        // 2^52 * 5 * G++        .quad   0x4ca73bd79cc8a7d6+        .quad   0x4d4a738f47e9a9b2+        .quad   0xf4cbf12942f5fe00+        .quad   0x01a13ff9bdbf0752+        .quad   0x99852bc3852cfdb0+        .quad   0x2cc12e9559d6ed0b+        .quad   0x70f9e2bf9b5ac27b+        .quad   0x4f3b8c117959ae99+        .quad   0x55b6c9c82ff26412+        .quad   0x1ac4a8c91fb667a8+        .quad   0xd527bfcfeb778bf2+        .quad   0x303337da7012a3be++        // 2^52 * 6 * G++        .quad   0x955422228c1c9d7c+        .quad   0x01fac1371a9b340f+        .quad   0x7e8d9177925b48d7+        .quad   0x53f8ad5661b3e31b+        .quad   0x976d3ccbfad2fdd1+        .quad   0xcb88839737a640a8+        .quad   0x2ff00c1d6734cb25+        .quad   0x269ff4dc789c2d2b+        .quad   0x0c003fbdc08d678d+        .quad   0x4d982fa37ead2b17+        .quad   0xc07e6bcdb2e582f1+        .quad   0x296c7291df412a44++        // 2^52 * 7 * G++        .quad   0x7903de2b33daf397+        .quad   0xd0ff0619c9a624b3+        .quad   0x8a1d252b555b3e18+        .quad   0x2b6d581c52e0b7c0+        .quad   0xdfb23205dab8b59e+        .quad   0x465aeaa0c8092250+        .quad   0xd133c1189a725d18+        .quad   0x2327370261f117d1+        .quad   0x3d0543d3623e7986+        .quad   0x679414c2c278a354+        .quad   0xae43f0cc726196f6+        .quad   0x7836c41f8245eaba++        // 2^52 * 8 * G++        .quad   0xe7a254db49e95a81+        .quad   0x5192d5d008b0ad73+        .quad   0x4d20e5b1d00afc07+        .quad   0x5d55f8012cf25f38+        .quad   0xca651e848011937c+        .quad   0xc6b0c46e6ef41a28+        .quad   0xb7021ba75f3f8d52+        .quad   0x119dff99ead7b9fd+        .quad   0x43eadfcbf4b31d4d+        .quad   0xc6503f7411148892+        .quad   0xfeee68c5060d3b17+        .quad   0x329293b3dd4a0ac8++        // 2^56 * 1 * G++        .quad   0x4e59214fe194961a+        .quad   0x49be7dc70d71cd4f+        .quad   0x9300cfd23b50f22d+        .quad   0x4789d446fc917232+        .quad   0x2879852d5d7cb208+        .quad   0xb8dedd70687df2e7+        .quad   0xdc0bffab21687891+        .quad   0x2b44c043677daa35+        .quad   0x1a1c87ab074eb78e+        .quad   0xfac6d18e99daf467+        .quad   0x3eacbbcd484f9067+        .quad   0x60c52eef2bb9a4e4++        // 2^56 * 2 * G++        .quad   0x0b5d89bc3bfd8bf1+        .quad   0xb06b9237c9f3551a+        .quad   0x0e4c16b0d53028f5+        .quad   0x10bc9c312ccfcaab+        .quad   0x702bc5c27cae6d11+        .quad   0x44c7699b54a48cab+        .quad   0xefbc4056ba492eb2+        .quad   0x70d77248d9b6676d+        .quad   0xaa8ae84b3ec2a05b+        .quad   0x98699ef4ed1781e0+        .quad   0x794513e4708e85d1+        .quad   0x63755bd3a976f413++        // 2^56 * 3 * G++        .quad   0xb55fa03e2ad10853+        .quad   0x356f75909ee63569+        .quad   0x9ff9f1fdbe69b890+        .quad   0x0d8cc1c48bc16f84+        .quad   0x3dc7101897f1acb7+        .quad   0x5dda7d5ec165bbd8+        .quad   0x508e5b9c0fa1020f+        .quad   0x2763751737c52a56+        .quad   0x029402d36eb419a9+        .quad   0xf0b44e7e77b460a5+        .quad   0xcfa86230d43c4956+        .quad   0x70c2dd8a7ad166e7++        // 2^56 * 4 * G++        .quad   0x656194509f6fec0e+        .quad   0xee2e7ea946c6518d+        .quad   0x9733c1f367e09b5c+        .quad   0x2e0fac6363948495+        .quad   0x91d4967db8ed7e13+        .quad   0x74252f0ad776817a+        .quad   0xe40982e00d852564+        .quad   0x32b8613816a53ce5+        .quad   0x79e7f7bee448cd64+        .quad   0x6ac83a67087886d0+        .quad   0xf89fd4d9a0e4db2e+        .quad   0x4179215c735a4f41++        // 2^56 * 5 * G++        .quad   0x8c7094e7d7dced2a+        .quad   0x97fb8ac347d39c70+        .quad   0xe13be033a906d902+        .quad   0x700344a30cd99d76+        .quad   0xe4ae33b9286bcd34+        .quad   0xb7ef7eb6559dd6dc+        .quad   0x278b141fb3d38e1f+        .quad   0x31fa85662241c286+        .quad   0xaf826c422e3622f4+        .quad   0xc12029879833502d+        .quad   0x9bc1b7e12b389123+        .quad   0x24bb2312a9952489++        // 2^56 * 6 * G++        .quad   0xb1a8ed1732de67c3+        .quad   0x3cb49418461b4948+        .quad   0x8ebd434376cfbcd2+        .quad   0x0fee3e871e188008+        .quad   0x41f80c2af5f85c6b+        .quad   0x687284c304fa6794+        .quad   0x8945df99a3ba1bad+        .quad   0x0d1d2af9ffeb5d16+        .quad   0xa9da8aa132621edf+        .quad   0x30b822a159226579+        .quad   0x4004197ba79ac193+        .quad   0x16acd79718531d76++        // 2^56 * 7 * G++        .quad   0x72df72af2d9b1d3d+        .quad   0x63462a36a432245a+        .quad   0x3ecea07916b39637+        .quad   0x123e0ef6b9302309+        .quad   0xc959c6c57887b6ad+        .quad   0x94e19ead5f90feba+        .quad   0x16e24e62a342f504+        .quad   0x164ed34b18161700+        .quad   0x487ed94c192fe69a+        .quad   0x61ae2cea3a911513+        .quad   0x877bf6d3b9a4de27+        .quad   0x78da0fc61073f3eb++        // 2^56 * 8 * G++        .quad   0x5bf15d28e52bc66a+        .quad   0x2c47e31870f01a8e+        .quad   0x2419afbc06c28bdd+        .quad   0x2d25deeb256b173a+        .quad   0xa29f80f1680c3a94+        .quad   0x71f77e151ae9e7e6+        .quad   0x1100f15848017973+        .quad   0x054aa4b316b38ddd+        .quad   0xdfc8468d19267cb8+        .quad   0x0b28789c66e54daf+        .quad   0x2aeb1d2a666eec17+        .quad   0x134610a6ab7da760++        // 2^60 * 1 * G++        .quad   0xcaf55ec27c59b23f+        .quad   0x99aeed3e154d04f2+        .quad   0x68441d72e14141f4+        .quad   0x140345133932a0a2+        .quad   0xd91430e0dc028c3c+        .quad   0x0eb955a85217c771+        .quad   0x4b09e1ed2c99a1fa+        .quad   0x42881af2bd6a743c+        .quad   0x7bfec69aab5cad3d+        .quad   0xc23e8cd34cb2cfad+        .quad   0x685dd14bfb37d6a2+        .quad   0x0ad6d64415677a18++        // 2^60 * 2 * G++        .quad   0x781a439e417becb5+        .quad   0x4ac5938cd10e0266+        .quad   0x5da385110692ac24+        .quad   0x11b065a2ade31233+        .quad   0x7914892847927e9f+        .quad   0x33dad6ef370aa877+        .quad   0x1f8f24fa11122703+        .quad   0x5265ac2f2adf9592+        .quad   0x405fdd309afcb346+        .quad   0xd9723d4428e63f54+        .quad   0x94c01df05f65aaae+        .quad   0x43e4dc3ae14c0809++        // 2^60 * 3 * G++        .quad   0xbc12c7f1a938a517+        .quad   0x473028ab3180b2e1+        .quad   0x3f78571efbcd254a+        .quad   0x74e534426ff6f90f+        .quad   0xea6f7ac3adc2c6a3+        .quad   0xd0e928f6e9717c94+        .quad   0xe2d379ead645eaf5+        .quad   0x46dd8785c51ffbbe+        .quad   0x709801be375c8898+        .quad   0x4b06dab5e3fd8348+        .quad   0x75880ced27230714+        .quad   0x2b09468fdd2f4c42++        // 2^60 * 4 * G++        .quad   0x97c749eeb701cb96+        .quad   0x83f438d4b6a369c3+        .quad   0x62962b8b9a402cd9+        .quad   0x6976c7509888df7b+        .quad   0x5b97946582ffa02a+        .quad   0xda096a51fea8f549+        .quad   0xa06351375f77af9b+        .quad   0x1bcfde61201d1e76+        .quad   0x4a4a5490246a59a2+        .quad   0xd63ebddee87fdd90+        .quad   0xd9437c670d2371fa+        .quad   0x69e87308d30f8ed6++        // 2^60 * 5 * G++        .quad   0x435a8bb15656beb0+        .quad   0xf8fac9ba4f4d5bca+        .quad   0xb9b278c41548c075+        .quad   0x3eb0ef76e892b622+        .quad   0x0f80bf028bc80303+        .quad   0x6aae16b37a18cefb+        .quad   0xdd47ea47d72cd6a3+        .quad   0x61943588f4ed39aa+        .quad   0xd26e5c3e91039f85+        .quad   0xc0e9e77df6f33aa9+        .quad   0xe8968c5570066a93+        .quad   0x3c34d1881faaaddd++        // 2^60 * 6 * G++        .quad   0x3f9d2b5ea09f9ec0+        .quad   0x1dab3b6fb623a890+        .quad   0xa09ba3ea72d926c4+        .quad   0x374193513fd8b36d+        .quad   0xbd5b0b8f2fffe0d9+        .quad   0x6aa254103ed24fb9+        .quad   0x2ac7d7bcb26821c4+        .quad   0x605b394b60dca36a+        .quad   0xb4e856e45a9d1ed2+        .quad   0xefe848766c97a9a2+        .quad   0xb104cf641e5eee7d+        .quad   0x2f50b81c88a71c8f++        // 2^60 * 7 * G++        .quad   0x31723c61fc6811bb+        .quad   0x9cb450486211800f+        .quad   0x768933d347995753+        .quad   0x3491a53502752fcd+        .quad   0x2b552ca0a7da522a+        .quad   0x3230b336449b0250+        .quad   0xf2c4c5bca4b99fb9+        .quad   0x7b2c674958074a22+        .quad   0xd55165883ed28cdf+        .quad   0x12d84fd2d362de39+        .quad   0x0a874ad3e3378e4f+        .quad   0x000d2b1f7c763e74++        // 2^60 * 8 * G++        .quad   0x3d420811d06d4a67+        .quad   0xbefc048590e0ffe3+        .quad   0xf870c6b7bd487bde+        .quad   0x6e2a7316319afa28+        .quad   0x9624778c3e94a8ab+        .quad   0x0ad6f3cee9a78bec+        .quad   0x948ac7810d743c4f+        .quad   0x76627935aaecfccc+        .quad   0x56a8ac24d6d59a9f+        .quad   0xc8db753e3096f006+        .quad   0x477f41e68f4c5299+        .quad   0x588d851cf6c86114++        // 2^64 * 1 * G++        .quad   0x51138ec78df6b0fe+        .quad   0x5397da89e575f51b+        .quad   0x09207a1d717af1b9+        .quad   0x2102fdba2b20d650+        .quad   0xcd2a65e777d1f515+        .quad   0x548991878faa60f1+        .quad   0xb1b73bbcdabc06e5+        .quad   0x654878cba97cc9fb+        .quad   0x969ee405055ce6a1+        .quad   0x36bca7681251ad29+        .quad   0x3a1af517aa7da415+        .quad   0x0ad725db29ecb2ba++        // 2^64 * 2 * G++        .quad   0xdc4267b1834e2457+        .quad   0xb67544b570ce1bc5+        .quad   0x1af07a0bf7d15ed7+        .quad   0x4aefcffb71a03650+        .quad   0xfec7bc0c9b056f85+        .quad   0x537d5268e7f5ffd7+        .quad   0x77afc6624312aefa+        .quad   0x4f675f5302399fd9+        .quad   0xc32d36360415171e+        .quad   0xcd2bef118998483b+        .quad   0x870a6eadd0945110+        .quad   0x0bccbb72a2a86561++        // 2^64 * 3 * G++        .quad   0x185e962feab1a9c8+        .quad   0x86e7e63565147dcd+        .quad   0xb092e031bb5b6df2+        .quad   0x4024f0ab59d6b73e+        .quad   0x186d5e4c50fe1296+        .quad   0xe0397b82fee89f7e+        .quad   0x3bc7f6c5507031b0+        .quad   0x6678fd69108f37c2+        .quad   0x1586fa31636863c2+        .quad   0x07f68c48572d33f2+        .quad   0x4f73cc9f789eaefc+        .quad   0x2d42e2108ead4701++        // 2^64 * 4 * G++        .quad   0x97f5131594dfd29b+        .quad   0x6155985d313f4c6a+        .quad   0xeba13f0708455010+        .quad   0x676b2608b8d2d322+        .quad   0x21717b0d0f537593+        .quad   0x914e690b131e064c+        .quad   0x1bb687ae752ae09f+        .quad   0x420bf3a79b423c6e+        .quad   0x8138ba651c5b2b47+        .quad   0x8671b6ec311b1b80+        .quad   0x7bff0cb1bc3135b0+        .quad   0x745d2ffa9c0cf1e0++        // 2^64 * 5 * G++        .quad   0xbf525a1e2bc9c8bd+        .quad   0xea5b260826479d81+        .quad   0xd511c70edf0155db+        .quad   0x1ae23ceb960cf5d0+        .quad   0x6036df5721d34e6a+        .quad   0xb1db8827997bb3d0+        .quad   0xd3c209c3c8756afa+        .quad   0x06e15be54c1dc839+        .quad   0x5b725d871932994a+        .quad   0x32351cb5ceb1dab0+        .quad   0x7dc41549dab7ca05+        .quad   0x58ded861278ec1f7++        // 2^64 * 6 * G++        .quad   0xd8173793f266c55c+        .quad   0xc8c976c5cc454e49+        .quad   0x5ce382f8bc26c3a8+        .quad   0x2ff39de85485f6f9+        .quad   0x2dfb5ba8b6c2c9a8+        .quad   0x48eeef8ef52c598c+        .quad   0x33809107f12d1573+        .quad   0x08ba696b531d5bd8+        .quad   0x77ed3eeec3efc57a+        .quad   0x04e05517d4ff4811+        .quad   0xea3d7a3ff1a671cb+        .quad   0x120633b4947cfe54++        // 2^64 * 7 * G++        .quad   0x0b94987891610042+        .quad   0x4ee7b13cecebfae8+        .quad   0x70be739594f0a4c0+        .quad   0x35d30a99b4d59185+        .quad   0x82bd31474912100a+        .quad   0xde237b6d7e6fbe06+        .quad   0xe11e761911ea79c6+        .quad   0x07433be3cb393bde+        .quad   0xff7944c05ce997f4+        .quad   0x575d3de4b05c51a3+        .quad   0x583381fd5a76847c+        .quad   0x2d873ede7af6da9f++        // 2^64 * 8 * G++        .quad   0x157a316443373409+        .quad   0xfab8b7eef4aa81d9+        .quad   0xb093fee6f5a64806+        .quad   0x2e773654707fa7b6+        .quad   0xaa6202e14e5df981+        .quad   0xa20d59175015e1f5+        .quad   0x18a275d3bae21d6c+        .quad   0x0543618a01600253+        .quad   0x0deabdf4974c23c1+        .quad   0xaa6f0a259dce4693+        .quad   0x04202cb8a29aba2c+        .quad   0x4b1443362d07960d++        // 2^68 * 1 * G++        .quad   0x47b837f753242cec+        .quad   0x256dc48cc04212f2+        .quad   0xe222fbfbe1d928c5+        .quad   0x48ea295bad8a2c07+        .quad   0x299b1c3f57c5715e+        .quad   0x96cb929e6b686d90+        .quad   0x3004806447235ab3+        .quad   0x2c435c24a44d9fe1+        .quad   0x0607c97c80f8833f+        .quad   0x0e851578ca25ec5b+        .quad   0x54f7450b161ebb6f+        .quad   0x7bcb4792a0def80e++        // 2^68 * 2 * G++        .quad   0x8487e3d02bc73659+        .quad   0x4baf8445059979df+        .quad   0xd17c975adcad6fbf+        .quad   0x57369f0bdefc96b6+        .quad   0x1cecd0a0045224c2+        .quad   0x757f1b1b69e53952+        .quad   0x775b7a925289f681+        .quad   0x1b6cc62016736148+        .quad   0xf1a9990175638698+        .quad   0x353dd1beeeaa60d3+        .quad   0x849471334c9ba488+        .quad   0x63fa6e6843ade311++        // 2^68 * 3 * G++        .quad   0xd15c20536597c168+        .quad   0x9f73740098d28789+        .quad   0x18aee7f13257ba1f+        .quad   0x3418bfda07346f14+        .quad   0x2195becdd24b5eb7+        .quad   0x5e41f18cc0cd44f9+        .quad   0xdf28074441ca9ede+        .quad   0x07073b98f35b7d67+        .quad   0xd03c676c4ce530d4+        .quad   0x0b64c0473b5df9f4+        .quad   0x065cef8b19b3a31e+        .quad   0x3084d661533102c9++        // 2^68 * 4 * G++        .quad   0xe1f6b79ebf8469ad+        .quad   0x15801004e2663135+        .quad   0x9a498330af74181b+        .quad   0x3ba2504f049b673c+        .quad   0x9a6ce876760321fd+        .quad   0x7fe2b5109eb63ad8+        .quad   0x00e7d4ae8ac80592+        .quad   0x73d86b7abb6f723a+        .quad   0x0b52b5606dba5ab6+        .quad   0xa9134f0fbbb1edab+        .quad   0x30a9520d9b04a635+        .quad   0x6813b8f37973e5db++        // 2^68 * 5 * G++        .quad   0x9854b054334127c1+        .quad   0x105d047882fbff25+        .quad   0xdb49f7f944186f4f+        .quad   0x1768e838bed0b900+        .quad   0xf194ca56f3157e29+        .quad   0x136d35705ef528a5+        .quad   0xdd4cef778b0599bc+        .quad   0x7d5472af24f833ed+        .quad   0xd0ef874daf33da47+        .quad   0x00d3be5db6e339f9+        .quad   0x3f2a8a2f9c9ceece+        .quad   0x5d1aeb792352435a++        // 2^68 * 6 * G++        .quad   0xf59e6bb319cd63ca+        .quad   0x670c159221d06839+        .quad   0xb06d565b2150cab6+        .quad   0x20fb199d104f12a3+        .quad   0x12c7bfaeb61ba775+        .quad   0xb84e621fe263bffd+        .quad   0x0b47a5c35c840dcf+        .quad   0x7e83be0bccaf8634+        .quad   0x61943dee6d99c120+        .quad   0x86101f2e460b9fe0+        .quad   0x6bb2f1518ee8598d+        .quad   0x76b76289fcc475cc++        // 2^68 * 7 * G++        .quad   0x791b4cc1756286fa+        .quad   0xdbced317d74a157c+        .quad   0x7e732421ea72bde6+        .quad   0x01fe18491131c8e9+        .quad   0x4245f1a1522ec0b3+        .quad   0x558785b22a75656d+        .quad   0x1d485a2548a1b3c0+        .quad   0x60959eccd58fe09f+        .quad   0x3ebfeb7ba8ed7a09+        .quad   0x49fdc2bbe502789c+        .quad   0x44ebce5d3c119428+        .quad   0x35e1eb55be947f4a++        // 2^68 * 8 * G++        .quad   0xdbdae701c5738dd3+        .quad   0xf9c6f635b26f1bee+        .quad   0x61e96a8042f15ef4+        .quad   0x3aa1d11faf60a4d8+        .quad   0x14fd6dfa726ccc74+        .quad   0x3b084cfe2f53b965+        .quad   0xf33ae4f552a2c8b4+        .quad   0x59aab07a0d40166a+        .quad   0x77bcec4c925eac25+        .quad   0x1848718460137738+        .quad   0x5b374337fea9f451+        .quad   0x1865e78ec8e6aa46++        // 2^72 * 1 * G++        .quad   0xccc4b7c7b66e1f7a+        .quad   0x44157e25f50c2f7e+        .quad   0x3ef06dfc713eaf1c+        .quad   0x582f446752da63f7+        .quad   0x967c54e91c529ccb+        .quad   0x30f6269264c635fb+        .quad   0x2747aff478121965+        .quad   0x17038418eaf66f5c+        .quad   0xc6317bd320324ce4+        .quad   0xa81042e8a4488bc4+        .quad   0xb21ef18b4e5a1364+        .quad   0x0c2a1c4bcda28dc9++        // 2^72 * 2 * G++        .quad   0xd24dc7d06f1f0447+        .quad   0xb2269e3edb87c059+        .quad   0xd15b0272fbb2d28f+        .quad   0x7c558bd1c6f64877+        .quad   0xedc4814869bd6945+        .quad   0x0d6d907dbe1c8d22+        .quad   0xc63bd212d55cc5ab+        .quad   0x5a6a9b30a314dc83+        .quad   0xd0ec1524d396463d+        .quad   0x12bb628ac35a24f0+        .quad   0xa50c3a791cbc5fa4+        .quad   0x0404a5ca0afbafc3++        // 2^72 * 3 * G++        .quad   0x8c1f40070aa743d6+        .quad   0xccbad0cb5b265ee8+        .quad   0x574b046b668fd2de+        .quad   0x46395bfdcadd9633+        .quad   0x62bc9e1b2a416fd1+        .quad   0xb5c6f728e350598b+        .quad   0x04343fd83d5d6967+        .quad   0x39527516e7f8ee98+        .quad   0x117fdb2d1a5d9a9c+        .quad   0x9c7745bcd1005c2a+        .quad   0xefd4bef154d56fea+        .quad   0x76579a29e822d016++        // 2^72 * 4 * G++        .quad   0x45b68e7e49c02a17+        .quad   0x23cd51a2bca9a37f+        .quad   0x3ed65f11ec224c1b+        .quad   0x43a384dc9e05bdb1+        .quad   0x333cb51352b434f2+        .quad   0xd832284993de80e1+        .quad   0xb5512887750d35ce+        .quad   0x02c514bb2a2777c1+        .quad   0x684bd5da8bf1b645+        .quad   0xfb8bd37ef6b54b53+        .quad   0x313916d7a9b0d253+        .quad   0x1160920961548059++        // 2^72 * 5 * G++        .quad   0xb44d166929dacfaa+        .quad   0xda529f4c8413598f+        .quad   0xe9ef63ca453d5559+        .quad   0x351e125bc5698e0b+        .quad   0x7a385616369b4dcd+        .quad   0x75c02ca7655c3563+        .quad   0x7dc21bf9d4f18021+        .quad   0x2f637d7491e6e042+        .quad   0xd4b49b461af67bbe+        .quad   0xd603037ac8ab8961+        .quad   0x71dee19ff9a699fb+        .quad   0x7f182d06e7ce2a9a++        // 2^72 * 6 * G++        .quad   0x7a7c8e64ab0168ec+        .quad   0xcb5a4a5515edc543+        .quad   0x095519d347cd0eda+        .quad   0x67d4ac8c343e93b0+        .quad   0x09454b728e217522+        .quad   0xaa58e8f4d484b8d8+        .quad   0xd358254d7f46903c+        .quad   0x44acc043241c5217+        .quad   0x1c7d6bbb4f7a5777+        .quad   0x8b35fed4918313e1+        .quad   0x4adca1c6c96b4684+        .quad   0x556d1c8312ad71bd++        // 2^72 * 7 * G++        .quad   0x17ef40e30c8d3982+        .quad   0x31f7073e15a3fa34+        .quad   0x4f21f3cb0773646e+        .quad   0x746c6c6d1d824eff+        .quad   0x81f06756b11be821+        .quad   0x0faff82310a3f3dd+        .quad   0xf8b2d0556a99465d+        .quad   0x097abe38cc8c7f05+        .quad   0x0c49c9877ea52da4+        .quad   0x4c4369559bdc1d43+        .quad   0x022c3809f7ccebd2+        .quad   0x577e14a34bee84bd++        // 2^72 * 8 * G++        .quad   0xf0e268ac61a73b0a+        .quad   0xf2fafa103791a5f5+        .quad   0xc1e13e826b6d00e9+        .quad   0x60fa7ee96fd78f42+        .quad   0x94fecebebd4dd72b+        .quad   0xf46a4fda060f2211+        .quad   0x124a5977c0c8d1ff+        .quad   0x705304b8fb009295+        .quad   0xb63d1d354d296ec6+        .quad   0xf3c3053e5fad31d8+        .quad   0x670b958cb4bd42ec+        .quad   0x21398e0ca16353fd++        // 2^76 * 1 * G++        .quad   0x216ab2ca8da7d2ef+        .quad   0x366ad9dd99f42827+        .quad   0xae64b9004fdd3c75+        .quad   0x403a395b53909e62+        .quad   0x86c5fc16861b7e9a+        .quad   0xf6a330476a27c451+        .quad   0x01667267a1e93597+        .quad   0x05ffb9cd6082dfeb+        .quad   0xa617fa9ff53f6139+        .quad   0x60f2b5e513e66cb6+        .quad   0xd7a8beefb3448aa4+        .quad   0x7a2932856f5ea192++        // 2^76 * 2 * G++        .quad   0x0b39d761b02de888+        .quad   0x5f550e7ed2414e1f+        .quad   0xa6bfa45822e1a940+        .quad   0x050a2f7dfd447b99+        .quad   0xb89c444879639302+        .quad   0x4ae4f19350c67f2c+        .quad   0xf0b35da8c81af9c6+        .quad   0x39d0003546871017+        .quad   0x437c3b33a650db77+        .quad   0x6bafe81dbac52bb2+        .quad   0xfe99402d2db7d318+        .quad   0x2b5b7eec372ba6ce++        // 2^76 * 3 * G++        .quad   0xb3bc4bbd83f50eef+        .quad   0x508f0c998c927866+        .quad   0x43e76587c8b7e66e+        .quad   0x0f7655a3a47f98d9+        .quad   0xa694404d613ac8f4+        .quad   0x500c3c2bfa97e72c+        .quad   0x874104d21fcec210+        .quad   0x1b205fb38604a8ee+        .quad   0x55ecad37d24b133c+        .quad   0x441e147d6038c90b+        .quad   0x656683a1d62c6fee+        .quad   0x0157d5dc87e0ecae++        // 2^76 * 4 * G++        .quad   0xf2a7af510354c13d+        .quad   0xd7a0b145aa372b60+        .quad   0x2869b96a05a3d470+        .quad   0x6528e42d82460173+        .quad   0x95265514d71eb524+        .quad   0xe603d8815df14593+        .quad   0x147cdf410d4de6b7+        .quad   0x5293b1730437c850+        .quad   0x23d0e0814bccf226+        .quad   0x92c745cd8196fb93+        .quad   0x8b61796c59541e5b+        .quad   0x40a44df0c021f978++        // 2^76 * 5 * G++        .quad   0xdaa869894f20ea6a+        .quad   0xea14a3d14c620618+        .quad   0x6001fccb090bf8be+        .quad   0x35f4e822947e9cf0+        .quad   0x86c96e514bc5d095+        .quad   0xf20d4098fca6804a+        .quad   0x27363d89c826ea5d+        .quad   0x39ca36565719cacf+        .quad   0x97506f2f6f87b75c+        .quad   0xc624aea0034ae070+        .quad   0x1ec856e3aad34dd6+        .quad   0x055b0be0e440e58f++        // 2^76 * 6 * G++        .quad   0x6469a17d89735d12+        .quad   0xdb6f27d5e662b9f1+        .quad   0x9fcba3286a395681+        .quad   0x363b8004d269af25+        .quad   0x4d12a04b6ea33da2+        .quad   0x57cf4c15e36126dd+        .quad   0x90ec9675ee44d967+        .quad   0x64ca348d2a985aac+        .quad   0x99588e19e4c4912d+        .quad   0xefcc3b4e1ca5ce6b+        .quad   0x4522ea60fa5b98d5+        .quad   0x7064bbab1de4a819++        // 2^76 * 7 * G++        .quad   0xb919e1515a770641+        .quad   0xa9a2e2c74e7f8039+        .quad   0x7527250b3df23109+        .quad   0x756a7330ac27b78b+        .quad   0xa290c06142542129+        .quad   0xf2e2c2aebe8d5b90+        .quad   0xcf2458db76abfe1b+        .quad   0x02157ade83d626bf+        .quad   0x3e46972a1b9a038b+        .quad   0x2e4ee66a7ee03fb4+        .quad   0x81a248776edbb4ca+        .quad   0x1a944ee88ecd0563++        // 2^76 * 8 * G++        .quad   0xd5a91d1151039372+        .quad   0x2ed377b799ca26de+        .quad   0xa17202acfd366b6b+        .quad   0x0730291bd6901995+        .quad   0xbb40a859182362d6+        .quad   0xb99f55778a4d1abb+        .quad   0x8d18b427758559f6+        .quad   0x26c20fe74d26235a+        .quad   0x648d1d9fe9cc22f5+        .quad   0x66bc561928dd577c+        .quad   0x47d3ed21652439d1+        .quad   0x49d271acedaf8b49++        // 2^80 * 1 * G++        .quad   0x89f5058a382b33f3+        .quad   0x5ae2ba0bad48c0b4+        .quad   0x8f93b503a53db36e+        .quad   0x5aa3ed9d95a232e6+        .quad   0x2798aaf9b4b75601+        .quad   0x5eac72135c8dad72+        .quad   0xd2ceaa6161b7a023+        .quad   0x1bbfb284e98f7d4e+        .quad   0x656777e9c7d96561+        .quad   0xcb2b125472c78036+        .quad   0x65053299d9506eee+        .quad   0x4a07e14e5e8957cc++        // 2^80 * 2 * G++        .quad   0x4ee412cb980df999+        .quad   0xa315d76f3c6ec771+        .quad   0xbba5edde925c77fd+        .quad   0x3f0bac391d313402+        .quad   0x240b58cdc477a49b+        .quad   0xfd38dade6447f017+        .quad   0x19928d32a7c86aad+        .quad   0x50af7aed84afa081+        .quad   0x6e4fde0115f65be5+        .quad   0x29982621216109b2+        .quad   0x780205810badd6d9+        .quad   0x1921a316baebd006++        // 2^80 * 3 * G++        .quad   0x89422f7edfb870fc+        .quad   0x2c296beb4f76b3bd+        .quad   0x0738f1d436c24df7+        .quad   0x6458df41e273aeb0+        .quad   0xd75aad9ad9f3c18b+        .quad   0x566a0eef60b1c19c+        .quad   0x3e9a0bac255c0ed9+        .quad   0x7b049deca062c7f5+        .quad   0xdccbe37a35444483+        .quad   0x758879330fedbe93+        .quad   0x786004c312c5dd87+        .quad   0x6093dccbc2950e64++        // 2^80 * 4 * G++        .quad   0x1ff39a8585e0706d+        .quad   0x36d0a5d8b3e73933+        .quad   0x43b9f2e1718f453b+        .quad   0x57d1ea084827a97c+        .quad   0x6bdeeebe6084034b+        .quad   0x3199c2b6780fb854+        .quad   0x973376abb62d0695+        .quad   0x6e3180c98b647d90+        .quad   0xee7ab6e7a128b071+        .quad   0xa4c1596d93a88baa+        .quad   0xf7b4de82b2216130+        .quad   0x363e999ddd97bd18++        // 2^80 * 5 * G++        .quad   0x96a843c135ee1fc4+        .quad   0x976eb35508e4c8cf+        .quad   0xb42f6801b58cd330+        .quad   0x48ee9b78693a052b+        .quad   0x2f1848dce24baec6+        .quad   0x769b7255babcaf60+        .quad   0x90cb3c6e3cefe931+        .quad   0x231f979bc6f9b355+        .quad   0x5c31de4bcc2af3c6+        .quad   0xb04bb030fe208d1f+        .quad   0xb78d7009c14fb466+        .quad   0x079bfa9b08792413++        // 2^80 * 6 * G++        .quad   0xe3903a51da300df4+        .quad   0x843964233da95ab0+        .quad   0xed3cf12d0b356480+        .quad   0x038c77f684817194+        .quad   0xf3c9ed80a2d54245+        .quad   0x0aa08b7877f63952+        .quad   0xd76dac63d1085475+        .quad   0x1ef4fb159470636b+        .quad   0x854e5ee65b167bec+        .quad   0x59590a4296d0cdc2+        .quad   0x72b2df3498102199+        .quad   0x575ee92a4a0bff56++        // 2^80 * 7 * G++        .quad   0xd4c080908a182fcf+        .quad   0x30e170c299489dbd+        .quad   0x05babd5752f733de+        .quad   0x43d4e7112cd3fd00+        .quad   0x5d46bc450aa4d801+        .quad   0xc3af1227a533b9d8+        .quad   0x389e3b262b8906c2+        .quad   0x200a1e7e382f581b+        .quad   0x518db967eaf93ac5+        .quad   0x71bc989b056652c0+        .quad   0xfe2b85d9567197f5+        .quad   0x050eca52651e4e38++        // 2^80 * 8 * G++        .quad   0xc3431ade453f0c9c+        .quad   0xe9f5045eff703b9b+        .quad   0xfcd97ac9ed847b3d+        .quad   0x4b0ee6c21c58f4c6+        .quad   0x97ac397660e668ea+        .quad   0x9b19bbfe153ab497+        .quad   0x4cb179b534eca79f+        .quad   0x6151c09fa131ae57+        .quad   0x3af55c0dfdf05d96+        .quad   0xdd262ee02ab4ee7a+        .quad   0x11b2bb8712171709+        .quad   0x1fef24fa800f030b++        // 2^84 * 1 * G++        .quad   0xb496123a6b6c6609+        .quad   0xa750fe8580ab5938+        .quad   0xf471bf39b7c27a5f+        .quad   0x507903ce77ac193c+        .quad   0xff91a66a90166220+        .quad   0xf22552ae5bf1e009+        .quad   0x7dff85d87f90df7c+        .quad   0x4f620ffe0c736fb9+        .quad   0x62f90d65dfde3e34+        .quad   0xcf28c592b9fa5fad+        .quad   0x99c86ef9c6164510+        .quad   0x25d448044a256c84++        // 2^84 * 2 * G++        .quad   0xbd68230ec7e9b16f+        .quad   0x0eb1b9c1c1c5795d+        .quad   0x7943c8c495b6b1ff+        .quad   0x2f9faf620bbacf5e+        .quad   0x2c7c4415c9022b55+        .quad   0x56a0d241812eb1fe+        .quad   0xf02ea1c9d7b65e0d+        .quad   0x4180512fd5323b26+        .quad   0xa4ff3e698a48a5db+        .quad   0xba6a3806bd95403b+        .quad   0x9f7ce1af47d5b65d+        .quad   0x15e087e55939d2fb++        // 2^84 * 3 * G++        .quad   0x12207543745c1496+        .quad   0xdaff3cfdda38610c+        .quad   0xe4e797272c71c34f+        .quad   0x39c07b1934bdede9+        .quad   0x8894186efb963f38+        .quad   0x48a00e80dc639bd5+        .quad   0xa4e8092be96c1c99+        .quad   0x5a097d54ca573661+        .quad   0x2d45892b17c9e755+        .quad   0xd033fd7289308df8+        .quad   0x6c2fe9d9525b8bd9+        .quad   0x2edbecf1c11cc079++        // 2^84 * 4 * G++        .quad   0x1616a4e3c715a0d2+        .quad   0x53623cb0f8341d4d+        .quad   0x96ef5329c7e899cb+        .quad   0x3d4e8dbba668baa6+        .quad   0xee0f0fddd087a25f+        .quad   0x9c7531555c3e34ee+        .quad   0x660c572e8fab3ab5+        .quad   0x0854fc44544cd3b2+        .quad   0x61eba0c555edad19+        .quad   0x24b533fef0a83de6+        .quad   0x3b77042883baa5f8+        .quad   0x678f82b898a47e8d++        // 2^84 * 5 * G++        .quad   0xb1491d0bd6900c54+        .quad   0x3539722c9d132636+        .quad   0x4db928920b362bc9+        .quad   0x4d7cd1fea68b69df+        .quad   0x1e09d94057775696+        .quad   0xeed1265c3cd951db+        .quad   0xfa9dac2b20bce16f+        .quad   0x0f7f76e0e8d089f4+        .quad   0x36d9ebc5d485b00c+        .quad   0xa2596492e4adb365+        .quad   0xc1659480c2119ccd+        .quad   0x45306349186e0d5f++        // 2^84 * 6 * G++        .quad   0x94ddd0c1a6cdff1d+        .quad   0x55f6f115e84213ae+        .quad   0x6c935f85992fcf6a+        .quad   0x067ee0f54a37f16f+        .quad   0x96a414ec2b072491+        .quad   0x1bb2218127a7b65b+        .quad   0x6d2849596e8a4af0+        .quad   0x65f3b08ccd27765f+        .quad   0xecb29fff199801f7+        .quad   0x9d361d1fa2a0f72f+        .quad   0x25f11d2375fd2f49+        .quad   0x124cefe80fe10fe2++        // 2^84 * 7 * G++        .quad   0x4c126cf9d18df255+        .quad   0xc1d471e9147a63b6+        .quad   0x2c6d3c73f3c93b5f+        .quad   0x6be3a6a2e3ff86a2+        .quad   0x1518e85b31b16489+        .quad   0x8faadcb7db710bfb+        .quad   0x39b0bdf4a14ae239+        .quad   0x05f4cbea503d20c1+        .quad   0xce040e9ec04145bc+        .quad   0xc71ff4e208f6834c+        .quad   0xbd546e8dab8847a3+        .quad   0x64666aa0a4d2aba5++        // 2^84 * 8 * G++        .quad   0x6841435a7c06d912+        .quad   0xca123c21bb3f830b+        .quad   0xd4b37b27b1cbe278+        .quad   0x1d753b84c76f5046+        .quad   0xb0c53bf73337e94c+        .quad   0x7cb5697e11e14f15+        .quad   0x4b84abac1930c750+        .quad   0x28dd4abfe0640468+        .quad   0x7dc0b64c44cb9f44+        .quad   0x18a3e1ace3925dbf+        .quad   0x7a3034862d0457c4+        .quad   0x4c498bf78a0c892e++        // 2^88 * 1 * G++        .quad   0x37d653fb1aa73196+        .quad   0x0f9495303fd76418+        .quad   0xad200b09fb3a17b2+        .quad   0x544d49292fc8613e+        .quad   0x22d2aff530976b86+        .quad   0x8d90b806c2d24604+        .quad   0xdca1896c4de5bae5+        .quad   0x28005fe6c8340c17+        .quad   0x6aefba9f34528688+        .quad   0x5c1bff9425107da1+        .quad   0xf75bbbcd66d94b36+        .quad   0x72e472930f316dfa++        // 2^88 * 2 * G++        .quad   0x2695208c9781084f+        .quad   0xb1502a0b23450ee1+        .quad   0xfd9daea603efde02+        .quad   0x5a9d2e8c2733a34c+        .quad   0x07f3f635d32a7627+        .quad   0x7aaa4d865f6566f0+        .quad   0x3c85e79728d04450+        .quad   0x1fee7f000fe06438+        .quad   0x765305da03dbf7e5+        .quad   0xa4daf2491434cdbd+        .quad   0x7b4ad5cdd24a88ec+        .quad   0x00f94051ee040543++        // 2^88 * 3 * G++        .quad   0x8d356b23c3d330b2+        .quad   0xf21c8b9bb0471b06+        .quad   0xb36c316c6e42b83c+        .quad   0x07d79c7e8beab10d+        .quad   0xd7ef93bb07af9753+        .quad   0x583ed0cf3db766a7+        .quad   0xce6998bf6e0b1ec5+        .quad   0x47b7ffd25dd40452+        .quad   0x87fbfb9cbc08dd12+        .quad   0x8a066b3ae1eec29b+        .quad   0x0d57242bdb1fc1bf+        .quad   0x1c3520a35ea64bb6++        // 2^88 * 4 * G++        .quad   0x80d253a6bccba34a+        .quad   0x3e61c3a13838219b+        .quad   0x90c3b6019882e396+        .quad   0x1c3d05775d0ee66f+        .quad   0xcda86f40216bc059+        .quad   0x1fbb231d12bcd87e+        .quad   0xb4956a9e17c70990+        .quad   0x38750c3b66d12e55+        .quad   0x692ef1409422e51a+        .quad   0xcbc0c73c2b5df671+        .quad   0x21014fe7744ce029+        .quad   0x0621e2c7d330487c++        // 2^88 * 5 * G++        .quad   0xaf9860cc8259838d+        .quad   0x90ea48c1c69f9adc+        .quad   0x6526483765581e30+        .quad   0x0007d6097bd3a5bc+        .quad   0xb7ae1796b0dbf0f3+        .quad   0x54dfafb9e17ce196+        .quad   0x25923071e9aaa3b4+        .quad   0x5d8e589ca1002e9d+        .quad   0xc0bf1d950842a94b+        .quad   0xb2d3c363588f2e3e+        .quad   0x0a961438bb51e2ef+        .quad   0x1583d7783c1cbf86++        // 2^88 * 6 * G++        .quad   0xeceea2ef5da27ae1+        .quad   0x597c3a1455670174+        .quad   0xc9a62a126609167a+        .quad   0x252a5f2e81ed8f70+        .quad   0x90034704cc9d28c7+        .quad   0x1d1b679ef72cc58f+        .quad   0x16e12b5fbe5b8726+        .quad   0x4958064e83c5580a+        .quad   0x0d2894265066e80d+        .quad   0xfcc3f785307c8c6b+        .quad   0x1b53da780c1112fd+        .quad   0x079c170bd843b388++        // 2^88 * 7 * G++        .quad   0x0506ece464fa6fff+        .quad   0xbee3431e6205e523+        .quad   0x3579422451b8ea42+        .quad   0x6dec05e34ac9fb00+        .quad   0xcdd6cd50c0d5d056+        .quad   0x9af7686dbb03573b+        .quad   0x3ca6723ff3c3ef48+        .quad   0x6768c0d7317b8acc+        .quad   0x94b625e5f155c1b3+        .quad   0x417bf3a7997b7b91+        .quad   0xc22cbddc6d6b2600+        .quad   0x51445e14ddcd52f4++        // 2^88 * 8 * G++        .quad   0x57502b4b3b144951+        .quad   0x8e67ff6b444bbcb3+        .quad   0xb8bd6927166385db+        .quad   0x13186f31e39295c8+        .quad   0x893147ab2bbea455+        .quad   0x8c53a24f92079129+        .quad   0x4b49f948be30f7a7+        .quad   0x12e990086e4fd43d+        .quad   0xf10c96b37fdfbb2e+        .quad   0x9f9a935e121ceaf9+        .quad   0xdf1136c43a5b983f+        .quad   0x77b2e3f05d3e99af++        // 2^92 * 1 * G++        .quad   0xfd0d75879cf12657+        .quad   0xe82fef94e53a0e29+        .quad   0xcc34a7f05bbb4be7+        .quad   0x0b251172a50c38a2+        .quad   0x9532f48fcc5cd29b+        .quad   0x2ba851bea3ce3671+        .quad   0x32dacaa051122941+        .quad   0x478d99d9350004f2+        .quad   0x1d5ad94890bb02c0+        .quad   0x50e208b10ec25115+        .quad   0xa26a22894ef21702+        .quad   0x4dc923343b524805++        // 2^92 * 2 * G++        .quad   0xe3828c400f8086b6+        .quad   0x3f77e6f7979f0dc8+        .quad   0x7ef6de304df42cb4+        .quad   0x5265797cb6abd784+        .quad   0x3ad3e3ebf36c4975+        .quad   0xd75d25a537862125+        .quad   0xe873943da025a516+        .quad   0x6bbc7cb4c411c847+        .quad   0x3c6f9cd1d4a50d56+        .quad   0xb6244077c6feab7e+        .quad   0x6ff9bf483580972e+        .quad   0x00375883b332acfb++        // 2^92 * 3 * G++        .quad   0x0001b2cd28cb0940+        .quad   0x63fb51a06f1c24c9+        .quad   0xb5ad8691dcd5ca31+        .quad   0x67238dbd8c450660+        .quad   0xc98bec856c75c99c+        .quad   0xe44184c000e33cf4+        .quad   0x0a676b9bba907634+        .quad   0x669e2cb571f379d7+        .quad   0xcb116b73a49bd308+        .quad   0x025aad6b2392729e+        .quad   0xb4793efa3f55d9b1+        .quad   0x72a1056140678bb9++        // 2^92 * 4 * G++        .quad   0xa2b6812b1cc9249d+        .quad   0x62866eee21211f58+        .quad   0x2cb5c5b85df10ece+        .quad   0x03a6b259e263ae00+        .quad   0x0d8d2909e2e505b6+        .quad   0x98ca78abc0291230+        .quad   0x77ef5569a9b12327+        .quad   0x7c77897b81439b47+        .quad   0xf1c1b5e2de331cb5+        .quad   0x5a9f5d8e15fca420+        .quad   0x9fa438f17bd932b1+        .quad   0x2a381bf01c6146e7++        // 2^92 * 5 * G++        .quad   0xac9b9879cfc811c1+        .quad   0x8b7d29813756e567+        .quad   0x50da4e607c70edfc+        .quad   0x5dbca62f884400b6+        .quad   0xf7c0be32b534166f+        .quad   0x27e6ca6419cf70d4+        .quad   0x934df7d7a957a759+        .quad   0x5701461dabdec2aa+        .quad   0x2c6747402c915c25+        .quad   0x1bdcd1a80b0d340a+        .quad   0x5e5601bd07b43f5f+        .quad   0x2555b4e05539a242++        // 2^92 * 6 * G++        .quad   0x6fc09f5266ddd216+        .quad   0xdce560a7c8e37048+        .quad   0xec65939da2df62fd+        .quad   0x7a869ae7e52ed192+        .quad   0x78409b1d87e463d4+        .quad   0xad4da95acdfb639d+        .quad   0xec28773755259b9c+        .quad   0x69c806e9c31230ab+        .quad   0x7b48f57414bb3f22+        .quad   0x68c7cee4aedccc88+        .quad   0xed2f936179ed80be+        .quad   0x25d70b885f77bc4b++        // 2^92 * 7 * G++        .quad   0x4151c3d9762bf4de+        .quad   0x083f435f2745d82b+        .quad   0x29775a2e0d23ddd5+        .quad   0x138e3a6269a5db24+        .quad   0x98459d29bb1ae4d4+        .quad   0x56b9c4c739f954ec+        .quad   0x832743f6c29b4b3e+        .quad   0x21ea8e2798b6878a+        .quad   0x87bef4b46a5a7b9c+        .quad   0xd2299d1b5fc1d062+        .quad   0x82409818dd321648+        .quad   0x5c5abeb1e5a2e03d++        // 2^92 * 8 * G++        .quad   0x14722af4b73c2ddb+        .quad   0xbc470c5f5a05060d+        .quad   0x00943eac2581b02e+        .quad   0x0e434b3b1f499c8f+        .quad   0x02cde6de1306a233+        .quad   0x7b5a52a2116f8ec7+        .quad   0xe1c681f4c1163b5b+        .quad   0x241d350660d32643+        .quad   0x6be4404d0ebc52c7+        .quad   0xae46233bb1a791f5+        .quad   0x2aec170ed25db42b+        .quad   0x1d8dfd966645d694++        // 2^96 * 1 * G++        .quad   0x296fa9c59c2ec4de+        .quad   0xbc8b61bf4f84f3cb+        .quad   0x1c7706d917a8f908+        .quad   0x63b795fc7ad3255d+        .quad   0xd598639c12ddb0a4+        .quad   0xa5d19f30c024866b+        .quad   0xd17c2f0358fce460+        .quad   0x07a195152e095e8a+        .quad   0xa8368f02389e5fc8+        .quad   0x90433b02cf8de43b+        .quad   0xafa1fd5dc5412643+        .quad   0x3e8fe83d032f0137++        // 2^96 * 2 * G++        .quad   0x2f8b15b90570a294+        .quad   0x94f2427067084549+        .quad   0xde1c5ae161bbfd84+        .quad   0x75ba3b797fac4007+        .quad   0x08704c8de8efd13c+        .quad   0xdfc51a8e33e03731+        .quad   0xa59d5da51260cde3+        .quad   0x22d60899a6258c86+        .quad   0x6239dbc070cdd196+        .quad   0x60fe8a8b6c7d8a9a+        .quad   0xb38847bceb401260+        .quad   0x0904d07b87779e5e++        // 2^96 * 3 * G++        .quad   0xb4ce1fd4ddba919c+        .quad   0xcf31db3ec74c8daa+        .quad   0x2c63cc63ad86cc51+        .quad   0x43e2143fbc1dde07+        .quad   0xf4322d6648f940b9+        .quad   0x06952f0cbd2d0c39+        .quad   0x167697ada081f931+        .quad   0x6240aacebaf72a6c+        .quad   0xf834749c5ba295a0+        .quad   0xd6947c5bca37d25a+        .quad   0x66f13ba7e7c9316a+        .quad   0x56bdaf238db40cac++        // 2^96 * 4 * G++        .quad   0x362ab9e3f53533eb+        .quad   0x338568d56eb93d40+        .quad   0x9e0e14521d5a5572+        .quad   0x1d24a86d83741318+        .quad   0x1310d36cc19d3bb2+        .quad   0x062a6bb7622386b9+        .quad   0x7c9b8591d7a14f5c+        .quad   0x03aa31507e1e5754+        .quad   0xf4ec7648ffd4ce1f+        .quad   0xe045eaf054ac8c1c+        .quad   0x88d225821d09357c+        .quad   0x43b261dc9aeb4859++        // 2^96 * 5 * G++        .quad   0xe55b1e1988bb79bb+        .quad   0xa09ed07dc17a359d+        .quad   0xb02c2ee2603dea33+        .quad   0x326055cf5b276bc2+        .quad   0x19513d8b6c951364+        .quad   0x94fe7126000bf47b+        .quad   0x028d10ddd54f9567+        .quad   0x02b4d5e242940964+        .quad   0xb4a155cb28d18df2+        .quad   0xeacc4646186ce508+        .quad   0xc49cf4936c824389+        .quad   0x27a6c809ae5d3410++        // 2^96 * 6 * G++        .quad   0x8ba6ebcd1f0db188+        .quad   0x37d3d73a675a5be8+        .quad   0xf22edfa315f5585a+        .quad   0x2cb67174ff60a17e+        .quad   0xcd2c270ac43d6954+        .quad   0xdd4a3e576a66cab2+        .quad   0x79fa592469d7036c+        .quad   0x221503603d8c2599+        .quad   0x59eecdf9390be1d0+        .quad   0xa9422044728ce3f1+        .quad   0x82891c667a94f0f4+        .quad   0x7b1df4b73890f436++        // 2^96 * 7 * G++        .quad   0xe492f2e0b3b2a224+        .quad   0x7c6c9e062b551160+        .quad   0x15eb8fe20d7f7b0e+        .quad   0x61fcef2658fc5992+        .quad   0x5f2e221807f8f58c+        .quad   0xe3555c9fd49409d4+        .quad   0xb2aaa88d1fb6a630+        .quad   0x68698245d352e03d+        .quad   0xdbb15d852a18187a+        .quad   0xf3e4aad386ddacd7+        .quad   0x44bae2810ff6c482+        .quad   0x46cf4c473daf01cf++        // 2^96 * 8 * G++        .quad   0x426525ed9ec4e5f9+        .quad   0x0e5eda0116903303+        .quad   0x72b1a7f2cbe5cadc+        .quad   0x29387bcd14eb5f40+        .quad   0x213c6ea7f1498140+        .quad   0x7c1e7ef8392b4854+        .quad   0x2488c38c5629ceba+        .quad   0x1065aae50d8cc5bb+        .quad   0x1c2c4525df200d57+        .quad   0x5c3b2dd6bfca674a+        .quad   0x0a07e7b1e1834030+        .quad   0x69a198e64f1ce716++        // 2^100 * 1 * G++        .quad   0x7afcd613efa9d697+        .quad   0x0cc45aa41c067959+        .quad   0xa56fe104c1fada96+        .quad   0x3a73b70472e40365+        .quad   0x7b26e56b9e2d4734+        .quad   0xc4c7132b81c61675+        .quad   0xef5c9525ec9cde7f+        .quad   0x39c80b16e71743ad+        .quad   0x0f196e0d1b826c68+        .quad   0xf71ff0e24960e3db+        .quad   0x6113167023b7436c+        .quad   0x0cf0ea5877da7282++        // 2^100 * 2 * G++        .quad   0x196c80a4ddd4ccbd+        .quad   0x22e6f55d95f2dd9d+        .quad   0xc75e33c740d6c71b+        .quad   0x7bb51279cb3c042f+        .quad   0xe332ced43ba6945a+        .quad   0xde0b1361e881c05d+        .quad   0x1ad40f095e67ed3b+        .quad   0x5da8acdab8c63d5d+        .quad   0xc4b6664a3a70159f+        .quad   0x76194f0f0a904e14+        .quad   0xa5614c39a4096c13+        .quad   0x6cd0ff50979feced++        // 2^100 * 3 * G++        .quad   0xc0e067e78f4428ac+        .quad   0x14835ab0a61135e3+        .quad   0xf21d14f338062935+        .quad   0x6390a4c8df04849c+        .quad   0x7fecfabdb04ba18e+        .quad   0xd0fc7bfc3bddbcf7+        .quad   0xa41d486e057a131c+        .quad   0x641a4391f2223a61+        .quad   0xc5c6b95aa606a8db+        .quad   0x914b7f9eb06825f1+        .quad   0x2a731f6b44fc9eff+        .quad   0x30ddf38562705cfc++        // 2^100 * 4 * G++        .quad   0x4e3dcbdad1bff7f9+        .quad   0xc9118e8220645717+        .quad   0xbacccebc0f189d56+        .quad   0x1b4822e9d4467668+        .quad   0x33bef2bd68bcd52c+        .quad   0xc649dbb069482ef2+        .quad   0xb5b6ee0c41cb1aee+        .quad   0x5c294d270212a7e5+        .quad   0xab360a7f25563781+        .quad   0x2512228a480f7958+        .quad   0xc75d05276114b4e3+        .quad   0x222d9625d976fe2a++        // 2^100 * 5 * G++        .quad   0x1c717f85b372ace1+        .quad   0x81930e694638bf18+        .quad   0x239cad056bc08b58+        .quad   0x0b34271c87f8fff4+        .quad   0x0f94be7e0a344f85+        .quad   0xeb2faa8c87f22c38+        .quad   0x9ce1e75e4ee16f0f+        .quad   0x43e64e5418a08dea+        .quad   0x8155e2521a35ce63+        .quad   0xbe100d4df912028e+        .quad   0xbff80bf8a57ddcec+        .quad   0x57342dc96d6bc6e4++        // 2^100 * 6 * G++        .quad   0xefeef065c8ce5998+        .quad   0xbf029510b5cbeaa2+        .quad   0x8c64a10620b7c458+        .quad   0x35134fb231c24855+        .quad   0xf3c3bcb71e707bf6+        .quad   0x351d9b8c7291a762+        .quad   0x00502e6edad69a33+        .quad   0x522f521f1ec8807f+        .quad   0x272c1f46f9a3902b+        .quad   0xc91ba3b799657bcc+        .quad   0xae614b304f8a1c0e+        .quad   0x7afcaad70b99017b++        // 2^100 * 7 * G++        .quad   0xc25ded54a4b8be41+        .quad   0x902d13e11bb0e2dd+        .quad   0x41f43233cde82ab2+        .quad   0x1085faa5c3aae7cb+        .quad   0xa88141ecef842b6b+        .quad   0x55e7b14797abe6c5+        .quad   0x8c748f9703784ffe+        .quad   0x5b50a1f7afcd00b7+        .quad   0x9b840f66f1361315+        .quad   0x18462242701003e9+        .quad   0x65ed45fae4a25080+        .quad   0x0a2862393fda7320++        // 2^100 * 8 * G++        .quad   0x46ab13c8347cbc9d+        .quad   0x3849e8d499c12383+        .quad   0x4cea314087d64ac9+        .quad   0x1f354134b1a29ee7+        .quad   0x960e737b6ecb9d17+        .quad   0xfaf24948d67ceae1+        .quad   0x37e7a9b4d55e1b89+        .quad   0x5cb7173cb46c59eb+        .quad   0x4a89e68b82b7abf0+        .quad   0xf41cd9279ba6b7b9+        .quad   0x16e6c210e18d876f+        .quad   0x7cacdb0f7f1b09c6++        // 2^104 * 1 * G++        .quad   0x9062b2e0d91a78bc+        .quad   0x47c9889cc8509667+        .quad   0x9df54a66405070b8+        .quad   0x7369e6a92493a1bf+        .quad   0xe1014434dcc5caed+        .quad   0x47ed5d963c84fb33+        .quad   0x70019576ed86a0e7+        .quad   0x25b2697bd267f9e4+        .quad   0x9d673ffb13986864+        .quad   0x3ca5fbd9415dc7b8+        .quad   0xe04ecc3bdf273b5e+        .quad   0x1420683db54e4cd2++        // 2^104 * 2 * G++        .quad   0xb478bd1e249dd197+        .quad   0x620c35005e58c102+        .quad   0xfb02d32fccbaac5c+        .quad   0x60b63bebf508a72d+        .quad   0x34eebb6fc1cc5ad0+        .quad   0x6a1b0ce99646ac8b+        .quad   0xd3b0da49a66bde53+        .quad   0x31e83b4161d081c1+        .quad   0x97e8c7129e062b4f+        .quad   0x49e48f4f29320ad8+        .quad   0x5bece14b6f18683f+        .quad   0x55cf1eb62d550317++        // 2^104 * 3 * G++        .quad   0x5879101065c23d58+        .quad   0x8b9d086d5094819c+        .quad   0xe2402fa912c55fa7+        .quad   0x669a6564570891d4+        .quad   0x3076b5e37df58c52+        .quad   0xd73ab9dde799cc36+        .quad   0xbd831ce34913ee20+        .quad   0x1a56fbaa62ba0133+        .quad   0x943e6b505c9dc9ec+        .quad   0x302557bba77c371a+        .quad   0x9873ae5641347651+        .quad   0x13c4836799c58a5c++        // 2^104 * 4 * G++        .quad   0x423a5d465ab3e1b9+        .quad   0xfc13c187c7f13f61+        .quad   0x19f83664ecb5b9b6+        .quad   0x66f80c93a637b607+        .quad   0xc4dcfb6a5d8bd080+        .quad   0xdeebc4ec571a4842+        .quad   0xd4b2e883b8e55365+        .quad   0x50bdc87dc8e5b827+        .quad   0x606d37836edfe111+        .quad   0x32353e15f011abd9+        .quad   0x64b03ac325b73b96+        .quad   0x1dd56444725fd5ae++        // 2^104 * 5 * G++        .quad   0x8fa47ff83362127d+        .quad   0xbc9f6ac471cd7c15+        .quad   0x6e71454349220c8b+        .quad   0x0e645912219f732e+        .quad   0xc297e60008bac89a+        .quad   0x7d4cea11eae1c3e0+        .quad   0xf3e38be19fe7977c+        .quad   0x3a3a450f63a305cd+        .quad   0x078f2f31d8394627+        .quad   0x389d3183de94a510+        .quad   0xd1e36c6d17996f80+        .quad   0x318c8d9393a9a87b++        // 2^104 * 6 * G++        .quad   0xf2745d032afffe19+        .quad   0x0c9f3c497f24db66+        .quad   0xbc98d3e3ba8598ef+        .quad   0x224c7c679a1d5314+        .quad   0x5d669e29ab1dd398+        .quad   0xfc921658342d9e3b+        .quad   0x55851dfdf35973cd+        .quad   0x509a41c325950af6+        .quad   0xbdc06edca6f925e9+        .quad   0x793ef3f4641b1f33+        .quad   0x82ec12809d833e89+        .quad   0x05bff02328a11389++        // 2^104 * 7 * G++        .quad   0x3632137023cae00b+        .quad   0x544acf0ad1accf59+        .quad   0x96741049d21a1c88+        .quad   0x780b8cc3fa2a44a7+        .quad   0x6881a0dd0dc512e4+        .quad   0x4fe70dc844a5fafe+        .quad   0x1f748e6b8f4a5240+        .quad   0x576277cdee01a3ea+        .quad   0x1ef38abc234f305f+        .quad   0x9a577fbd1405de08+        .quad   0x5e82a51434e62a0d+        .quad   0x5ff418726271b7a1++        // 2^104 * 8 * G++        .quad   0x398e080c1789db9d+        .quad   0xa7602025f3e778f5+        .quad   0xfa98894c06bd035d+        .quad   0x106a03dc25a966be+        .quad   0xe5db47e813b69540+        .quad   0xf35d2a3b432610e1+        .quad   0xac1f26e938781276+        .quad   0x29d4db8ca0a0cb69+        .quad   0xd9ad0aaf333353d0+        .quad   0x38669da5acd309e5+        .quad   0x3c57658ac888f7f0+        .quad   0x4ab38a51052cbefa++        // 2^108 * 1 * G++        .quad   0xdfdacbee4324c0e9+        .quad   0x054442883f955bb7+        .quad   0xdef7aaa8ea31609f+        .quad   0x68aee70642287cff+        .quad   0xf68fe2e8809de054+        .quad   0xe3bc096a9c82bad1+        .quad   0x076353d40aadbf45+        .quad   0x7b9b1fb5dea1959e+        .quad   0xf01cc8f17471cc0c+        .quad   0x95242e37579082bb+        .quad   0x27776093d3e46b5f+        .quad   0x2d13d55a28bd85fb++        // 2^108 * 2 * G++        .quad   0xfac5d2065b35b8da+        .quad   0xa8da8a9a85624bb7+        .quad   0xccd2ca913d21cd0f+        .quad   0x6b8341ee8bf90d58+        .quad   0xbf019cce7aee7a52+        .quad   0xa8ded2b6e454ead3+        .quad   0x3c619f0b87a8bb19+        .quad   0x3619b5d7560916d8+        .quad   0x3579f26b0282c4b2+        .quad   0x64d592f24fafefae+        .quad   0xb7cded7b28c8c7c0+        .quad   0x6a927b6b7173a8d7++        // 2^108 * 3 * G++        .quad   0x1f6db24f986e4656+        .quad   0x1021c02ed1e9105b+        .quad   0xf8ff3fff2cc0a375+        .quad   0x1d2a6bf8c6c82592+        .quad   0x8d7040863ece88eb+        .quad   0xf0e307a980eec08c+        .quad   0xac2250610d788fda+        .quad   0x056d92a43a0d478d+        .quad   0x1b05a196fc3da5a1+        .quad   0x77d7a8c243b59ed0+        .quad   0x06da3d6297d17918+        .quad   0x66fbb494f12353f7++        // 2^108 * 4 * G++        .quad   0x751a50b9d85c0fb8+        .quad   0xd1afdc258bcf097b+        .quad   0x2f16a6a38309a969+        .quad   0x14ddff9ee5b00659+        .quad   0xd6d70996f12309d6+        .quad   0xdbfb2385e9c3d539+        .quad   0x46d602b0f7552411+        .quad   0x270a0b0557843e0c+        .quad   0x61ff0640a7862bcc+        .quad   0x81cac09a5f11abfe+        .quad   0x9047830455d12abb+        .quad   0x19a4bde1945ae873++        // 2^108 * 5 * G++        .quad   0x9b9f26f520a6200a+        .quad   0x64804443cf13eaf8+        .quad   0x8a63673f8631edd3+        .quad   0x72bbbce11ed39dc1+        .quad   0x40c709dec076c49f+        .quad   0x657bfaf27f3e53f6+        .quad   0x40662331eca042c4+        .quad   0x14b375487eb4df04+        .quad   0xae853c94ab66dc47+        .quad   0xeb62343edf762d6e+        .quad   0xf08e0e186fb2f7d1+        .quad   0x4f0b1c02700ab37a++        // 2^108 * 6 * G++        .quad   0xe1706787d81951fa+        .quad   0xa10a2c8eb290c77b+        .quad   0xe7382fa03ed66773+        .quad   0x0a4d84710bcc4b54+        .quad   0x79fd21ccc1b2e23f+        .quad   0x4ae7c281453df52a+        .quad   0xc8172ec9d151486b+        .quad   0x68abe9443e0a7534+        .quad   0xda12c6c407831dcb+        .quad   0x0da230d74d5c510d+        .quad   0x4ab1531e6bd404e1+        .quad   0x4106b166bcf440ef++        // 2^108 * 7 * G++        .quad   0x02e57a421cd23668+        .quad   0x4ad9fb5d0eaef6fd+        .quad   0x954e6727b1244480+        .quad   0x7f792f9d2699f331+        .quad   0xa485ccd539e4ecf2+        .quad   0x5aa3f3ad0555bab5+        .quad   0x145e3439937df82d+        .quad   0x1238b51e1214283f+        .quad   0x0b886b925fd4d924+        .quad   0x60906f7a3626a80d+        .quad   0xecd367b4b98abd12+        .quad   0x2876beb1def344cf++        // 2^108 * 8 * G++        .quad   0xdc84e93563144691+        .quad   0x632fe8a0d61f23f4+        .quad   0x4caa800612a9a8d5+        .quad   0x48f9dbfa0e9918d3+        .quad   0xd594b3333a8a85f8+        .quad   0x4ea37689e78d7d58+        .quad   0x73bf9f455e8e351f+        .quad   0x5507d7d2bc41ebb4+        .quad   0x1ceb2903299572fc+        .quad   0x7c8ccaa29502d0ee+        .quad   0x91bfa43411cce67b+        .quad   0x5784481964a831e7++        // 2^112 * 1 * G++        .quad   0xda7c2b256768d593+        .quad   0x98c1c0574422ca13+        .quad   0xf1a80bd5ca0ace1d+        .quad   0x29cdd1adc088a690+        .quad   0xd6cfd1ef5fddc09c+        .quad   0xe82b3efdf7575dce+        .quad   0x25d56b5d201634c2+        .quad   0x3041c6bb04ed2b9b+        .quad   0x0ff2f2f9d956e148+        .quad   0xade797759f356b2e+        .quad   0x1a4698bb5f6c025c+        .quad   0x104bbd6814049a7b++        // 2^112 * 2 * G++        .quad   0x51f0fd3168f1ed67+        .quad   0x2c811dcdd86f3bc2+        .quad   0x44dc5c4304d2f2de+        .quad   0x5be8cc57092a7149+        .quad   0xa95d9a5fd67ff163+        .quad   0xe92be69d4cc75681+        .quad   0xb7f8024cde20f257+        .quad   0x204f2a20fb072df5+        .quad   0xc8143b3d30ebb079+        .quad   0x7589155abd652e30+        .quad   0x653c3c318f6d5c31+        .quad   0x2570fb17c279161f++        // 2^112 * 3 * G++        .quad   0x3efa367f2cb61575+        .quad   0xf5f96f761cd6026c+        .quad   0xe8c7142a65b52562+        .quad   0x3dcb65ea53030acd+        .quad   0x192ea9550bb8245a+        .quad   0xc8e6fba88f9050d1+        .quad   0x7986ea2d88a4c935+        .quad   0x241c5f91de018668+        .quad   0x28d8172940de6caa+        .quad   0x8fbf2cf022d9733a+        .quad   0x16d7fcdd235b01d1+        .quad   0x08420edd5fcdf0e5++        // 2^112 * 4 * G++        .quad   0xcdff20ab8362fa4a+        .quad   0x57e118d4e21a3e6e+        .quad   0xe3179617fc39e62b+        .quad   0x0d9a53efbc1769fd+        .quad   0x0358c34e04f410ce+        .quad   0xb6135b5a276e0685+        .quad   0x5d9670c7ebb91521+        .quad   0x04d654f321db889c+        .quad   0x5e7dc116ddbdb5d5+        .quad   0x2954deb68da5dd2d+        .quad   0x1cb608173334a292+        .quad   0x4a7a4f2618991ad7++        // 2^112 * 5 * G++        .quad   0xf4a718025fb15f95+        .quad   0x3df65f346b5c1b8f+        .quad   0xcdfcf08500e01112+        .quad   0x11b50c4cddd31848+        .quad   0x24c3b291af372a4b+        .quad   0x93da8270718147f2+        .quad   0xdd84856486899ef2+        .quad   0x4a96314223e0ee33+        .quad   0xa6e8274408a4ffd6+        .quad   0x738e177e9c1576d9+        .quad   0x773348b63d02b3f2+        .quad   0x4f4bce4dce6bcc51++        // 2^112 * 6 * G++        .quad   0xa71fce5ae2242584+        .quad   0x26ea725692f58a9e+        .quad   0xd21a09d71cea3cf4+        .quad   0x73fcdd14b71c01e6+        .quad   0x30e2616ec49d0b6f+        .quad   0xe456718fcaec2317+        .quad   0x48eb409bf26b4fa6+        .quad   0x3042cee561595f37+        .quad   0x427e7079449bac41+        .quad   0x855ae36dbce2310a+        .quad   0x4cae76215f841a7c+        .quad   0x389e740c9a9ce1d6++        // 2^112 * 7 * G++        .quad   0x64fcb3ae34dcb9ce+        .quad   0x97500323e348d0ad+        .quad   0x45b3f07d62c6381b+        .quad   0x61545379465a6788+        .quad   0xc9bd78f6570eac28+        .quad   0xe55b0b3227919ce1+        .quad   0x65fc3eaba19b91ed+        .quad   0x25c425e5d6263690+        .quad   0x3f3e06a6f1d7de6e+        .quad   0x3ef976278e062308+        .quad   0x8c14f6264e8a6c77+        .quad   0x6539a08915484759++        // 2^112 * 8 * G++        .quad   0xe9d21f74c3d2f773+        .quad   0xc150544125c46845+        .quad   0x624e5ce8f9b99e33+        .quad   0x11c5e4aac5cd186c+        .quad   0xddc4dbd414bb4a19+        .quad   0x19b2bc3c98424f8e+        .quad   0x48a89fd736ca7169+        .quad   0x0f65320ef019bd90+        .quad   0xd486d1b1cafde0c6+        .quad   0x4f3fe6e3163b5181+        .quad   0x59a8af0dfaf2939a+        .quad   0x4cabc7bdec33072a++        // 2^116 * 1 * G++        .quad   0x16faa8fb532f7428+        .quad   0xdbd42ea046a4e272+        .quad   0x5337653b8b9ea480+        .quad   0x4065947223973f03+        .quad   0xf7c0a19c1a54a044+        .quad   0x4a1c5e2477bd9fbb+        .quad   0xa6e3ca115af22972+        .quad   0x1819bb953f2e9e0d+        .quad   0x498fbb795e042e84+        .quad   0x7d0dd89a7698b714+        .quad   0x8bfb0ba427fe6295+        .quad   0x36ba82e721200524++        // 2^116 * 2 * G++        .quad   0xd60ecbb74245ec41+        .quad   0xfd9be89e34348716+        .quad   0xc9240afee42284de+        .quad   0x4472f648d0531db4+        .quad   0xc8d69d0a57274ed5+        .quad   0x45ba803260804b17+        .quad   0xdf3cda102255dfac+        .quad   0x77d221232709b339+        .quad   0x498a6d7064ad94d8+        .quad   0xa5b5c8fd9af62263+        .quad   0x8ca8ed0545c141f4+        .quad   0x2c63bec3662d358c++        // 2^116 * 3 * G++        .quad   0x7fe60d8bea787955+        .quad   0xb9dc117eb5f401b7+        .quad   0x91c7c09a19355cce+        .quad   0x22692ef59442bedf+        .quad   0x9a518b3a8586f8bf+        .quad   0x9ee71af6cbb196f0+        .quad   0xaa0625e6a2385cf2+        .quad   0x1deb2176ddd7c8d1+        .quad   0x8563d19a2066cf6c+        .quad   0x401bfd8c4dcc7cd7+        .quad   0xd976a6becd0d8f62+        .quad   0x67cfd773a278b05e++        // 2^116 * 4 * G++        .quad   0x8dec31faef3ee475+        .quad   0x99dbff8a9e22fd92+        .quad   0x512d11594e26cab1+        .quad   0x0cde561eec4310b9+        .quad   0x2d5fa9855a4e586a+        .quad   0x65f8f7a449beab7e+        .quad   0xaa074dddf21d33d3+        .quad   0x185cba721bcb9dee+        .quad   0x93869da3f4e3cb41+        .quad   0xbf0392f540f7977e+        .quad   0x026204fcd0463b83+        .quad   0x3ec91a769eec6eed++        // 2^116 * 5 * G++        .quad   0x1e9df75bf78166ad+        .quad   0x4dfda838eb0cd7af+        .quad   0xba002ed8c1eaf988+        .quad   0x13fedb3e11f33cfc+        .quad   0x0fad2fb7b0a3402f+        .quad   0x46615ecbfb69f4a8+        .quad   0xf745bcc8c5f8eaa6+        .quad   0x7a5fa8794a94e896+        .quad   0x52958faa13cd67a1+        .quad   0x965ee0818bdbb517+        .quad   0x16e58daa2e8845b3+        .quad   0x357d397d5499da8f++        // 2^116 * 6 * G++        .quad   0x1ebfa05fb0bace6c+        .quad   0xc934620c1caf9a1e+        .quad   0xcc771cc41d82b61a+        .quad   0x2d94a16aa5f74fec+        .quad   0x481dacb4194bfbf8+        .quad   0x4d77e3f1bae58299+        .quad   0x1ef4612e7d1372a0+        .quad   0x3a8d867e70ff69e1+        .quad   0x6f58cd5d55aff958+        .quad   0xba3eaa5c75567721+        .quad   0x75c123999165227d+        .quad   0x69be1343c2f2b35e++        // 2^116 * 7 * G++        .quad   0x0e091d5ee197c92a+        .quad   0x4f51019f2945119f+        .quad   0x143679b9f034e99c+        .quad   0x7d88112e4d24c696+        .quad   0x82bbbdac684b8de3+        .quad   0xa2f4c7d03fca0718+        .quad   0x337f92fbe096aaa8+        .quad   0x200d4d8c63587376+        .quad   0x208aed4b4893b32b+        .quad   0x3efbf23ebe59b964+        .quad   0xd762deb0dba5e507+        .quad   0x69607bd681bd9d94++        // 2^116 * 8 * G++        .quad   0xf6be021068de1ce1+        .quad   0xe8d518e70edcbc1f+        .quad   0xe3effdd01b5505a5+        .quad   0x35f63353d3ec3fd0+        .quad   0x3b7f3bd49323a902+        .quad   0x7c21b5566b2c6e53+        .quad   0xe5ba8ff53a7852a7+        .quad   0x28bc77a5838ece00+        .quad   0x63ba78a8e25d8036+        .quad   0x63651e0094333490+        .quad   0x48d82f20288ce532+        .quad   0x3a31abfa36b57524++        // 2^120 * 1 * G++        .quad   0x239e9624089c0a2e+        .quad   0xc748c4c03afe4738+        .quad   0x17dbed2a764fa12a+        .quad   0x639b93f0321c8582+        .quad   0xc08f788f3f78d289+        .quad   0xfe30a72ca1404d9f+        .quad   0xf2778bfccf65cc9d+        .quad   0x7ee498165acb2021+        .quad   0x7bd508e39111a1c3+        .quad   0x2b2b90d480907489+        .quad   0xe7d2aec2ae72fd19+        .quad   0x0edf493c85b602a6++        // 2^120 * 2 * G++        .quad   0xaecc8158599b5a68+        .quad   0xea574f0febade20e+        .quad   0x4fe41d7422b67f07+        .quad   0x403b92e3019d4fb4+        .quad   0x6767c4d284764113+        .quad   0xa090403ff7f5f835+        .quad   0x1c8fcffacae6bede+        .quad   0x04c00c54d1dfa369+        .quad   0x4dc22f818b465cf8+        .quad   0x71a0f35a1480eff8+        .quad   0xaee8bfad04c7d657+        .quad   0x355bb12ab26176f4++        // 2^120 * 3 * G++        .quad   0xa71e64cc7493bbf4+        .quad   0xe5bd84d9eca3b0c3+        .quad   0x0a6bc50cfa05e785+        .quad   0x0f9b8132182ec312+        .quad   0xa301dac75a8c7318+        .quad   0xed90039db3ceaa11+        .quad   0x6f077cbf3bae3f2d+        .quad   0x7518eaf8e052ad8e+        .quad   0xa48859c41b7f6c32+        .quad   0x0f2d60bcf4383298+        .quad   0x1815a929c9b1d1d9+        .quad   0x47c3871bbb1755c4++        // 2^120 * 4 * G++        .quad   0x5144539771ec4f48+        .quad   0xf805b17dc98c5d6e+        .quad   0xf762c11a47c3c66b+        .quad   0x00b89b85764699dc+        .quad   0xfbe65d50c85066b0+        .quad   0x62ecc4b0b3a299b0+        .quad   0xe53754ea441ae8e0+        .quad   0x08fea02ce8d48d5f+        .quad   0x824ddd7668deead0+        .quad   0xc86445204b685d23+        .quad   0xb514cfcd5d89d665+        .quad   0x473829a74f75d537++        // 2^120 * 5 * G++        .quad   0x82d2da754679c418+        .quad   0xe63bd7d8b2618df0+        .quad   0x355eef24ac47eb0a+        .quad   0x2078684c4833c6b4+        .quad   0x23d9533aad3902c9+        .quad   0x64c2ddceef03588f+        .quad   0x15257390cfe12fb4+        .quad   0x6c668b4d44e4d390+        .quad   0x3b48cf217a78820c+        .quad   0xf76a0ab281273e97+        .quad   0xa96c65a78c8eed7b+        .quad   0x7411a6054f8a433f++        // 2^120 * 6 * G++        .quad   0x4d659d32b99dc86d+        .quad   0x044cdc75603af115+        .quad   0xb34c712cdcc2e488+        .quad   0x7c136574fb8134ff+        .quad   0x579ae53d18b175b4+        .quad   0x68713159f392a102+        .quad   0x8455ecba1eef35f5+        .quad   0x1ec9a872458c398f+        .quad   0xb8e6a4d400a2509b+        .quad   0x9b81d7020bc882b4+        .quad   0x57e7cc9bf1957561+        .quad   0x3add88a5c7cd6460++        // 2^120 * 7 * G++        .quad   0xab895770b635dcf2+        .quad   0x02dfef6cf66c1fbc+        .quad   0x85530268beb6d187+        .quad   0x249929fccc879e74+        .quad   0x85c298d459393046+        .quad   0x8f7e35985ff659ec+        .quad   0x1d2ca22af2f66e3a+        .quad   0x61ba1131a406a720+        .quad   0xa3d0a0f116959029+        .quad   0x023b6b6cba7ebd89+        .quad   0x7bf15a3e26783307+        .quad   0x5620310cbbd8ece7++        // 2^120 * 8 * G++        .quad   0x528993434934d643+        .quad   0xb9dbf806a51222f5+        .quad   0x8f6d878fc3f41c22+        .quad   0x37676a2a4d9d9730+        .quad   0x6646b5f477e285d6+        .quad   0x40e8ff676c8f6193+        .quad   0xa6ec7311abb594dd+        .quad   0x7ec846f3658cec4d+        .quad   0x9b5e8f3f1da22ec7+        .quad   0x130f1d776c01cd13+        .quad   0x214c8fcfa2989fb8+        .quad   0x6daaf723399b9dd5++        // 2^124 * 1 * G++        .quad   0x591e4a5610628564+        .quad   0x2a4bb87ca8b4df34+        .quad   0xde2a2572e7a38e43+        .quad   0x3cbdabd9fee5046e+        .quad   0x81aebbdd2cd13070+        .quad   0x962e4325f85a0e9e+        .quad   0xde9391aacadffecb+        .quad   0x53177fda52c230e6+        .quad   0xa7bc970650b9de79+        .quad   0x3d12a7fbc301b59b+        .quad   0x02652e68d36ae38c+        .quad   0x79d739835a6199dc++        // 2^124 * 2 * G++        .quad   0xd9354df64131c1bd+        .quad   0x758094a186ec5822+        .quad   0x4464ee12e459f3c2+        .quad   0x6c11fce4cb133282+        .quad   0x21c9d9920d591737+        .quad   0x9bea41d2e9b46cd6+        .quad   0xe20e84200d89bfca+        .quad   0x79d99f946eae5ff8+        .quad   0xf17b483568673205+        .quad   0x387deae83caad96c+        .quad   0x61b471fd56ffe386+        .quad   0x31741195b745a599++        // 2^124 * 3 * G++        .quad   0xe8d10190b77a360b+        .quad   0x99b983209995e702+        .quad   0xbd4fdff8fa0247aa+        .quad   0x2772e344e0d36a87+        .quad   0x17f8ba683b02a047+        .quad   0x50212096feefb6c8+        .quad   0x70139be21556cbe2+        .quad   0x203e44a11d98915b+        .quad   0xd6863eba37b9e39f+        .quad   0x105bc169723b5a23+        .quad   0x104f6459a65c0762+        .quad   0x567951295b4d38d4++        // 2^124 * 4 * G++        .quad   0x535fd60613037524+        .quad   0xe210adf6b0fbc26a+        .quad   0xac8d0a9b23e990ae+        .quad   0x47204d08d72fdbf9+        .quad   0x07242eb30d4b497f+        .quad   0x1ef96306b9bccc87+        .quad   0x37950934d8116f45+        .quad   0x05468d6201405b04+        .quad   0x00f565a9f93267de+        .quad   0xcecfd78dc0d58e8a+        .quad   0xa215e2dcf318e28e+        .quad   0x4599ee919b633352++        // 2^124 * 5 * G++        .quad   0xd3c220ca70e0e76b+        .quad   0xb12bea58ea9f3094+        .quad   0x294ddec8c3271282+        .quad   0x0c3539e1a1d1d028+        .quad   0xac746d6b861ae579+        .quad   0x31ab0650f6aea9dc+        .quad   0x241d661140256d4c+        .quad   0x2f485e853d21a5de+        .quad   0x329744839c0833f3+        .quad   0x6fe6257fd2abc484+        .quad   0x5327d1814b358817+        .quad   0x65712585893fe9bc++        // 2^124 * 6 * G++        .quad   0x9c102fb732a61161+        .quad   0xe48e10dd34d520a8+        .quad   0x365c63546f9a9176+        .quad   0x32f6fe4c046f6006+        .quad   0x81c29f1bd708ee3f+        .quad   0xddcb5a05ae6407d0+        .quad   0x97aec1d7d2a3eba7+        .quad   0x1590521a91d50831+        .quad   0x40a3a11ec7910acc+        .quad   0x9013dff8f16d27ae+        .quad   0x1a9720d8abb195d4+        .quad   0x1bb9fe452ea98463++        // 2^124 * 7 * G++        .quad   0xe9d1d950b3d54f9e+        .quad   0x2d5f9cbee00d33c1+        .quad   0x51c2c656a04fc6ac+        .quad   0x65c091ee3c1cbcc9+        .quad   0xcf5e6c95cc36747c+        .quad   0x294201536b0bc30d+        .quad   0x453ac67cee797af0+        .quad   0x5eae6ab32a8bb3c9+        .quad   0x7083661114f118ea+        .quad   0x2b37b87b94349cad+        .quad   0x7273f51cb4e99f40+        .quad   0x78a2a95823d75698++        // 2^124 * 8 * G++        .quad   0xa2b072e95c8c2ace+        .quad   0x69cffc96651e9c4b+        .quad   0x44328ef842e7b42b+        .quad   0x5dd996c122aadeb3+        .quad   0xb4f23c425ef83207+        .quad   0xabf894d3c9a934b5+        .quad   0xd0708c1339fd87f7+        .quad   0x1876789117166130+        .quad   0x925b5ef0670c507c+        .quad   0x819bc842b93c33bf+        .quad   0x10792e9a70dd003f+        .quad   0x59ad4b7a6e28dc74++        // 2^128 * 1 * G++        .quad   0x5f3a7562eb3dbe47+        .quad   0xf7ea38548ebda0b8+        .quad   0x00c3e53145747299+        .quad   0x1304e9e71627d551+        .quad   0x583b04bfacad8ea2+        .quad   0x29b743e8148be884+        .quad   0x2b1e583b0810c5db+        .quad   0x2b5449e58eb3bbaa+        .quad   0x789814d26adc9cfe+        .quad   0x3c1bab3f8b48dd0b+        .quad   0xda0fe1fff979c60a+        .quad   0x4468de2d7c2dd693++        // 2^128 * 2 * G++        .quad   0x51bb355e9419469e+        .quad   0x33e6dc4c23ddc754+        .quad   0x93a5b6d6447f9962+        .quad   0x6cce7c6ffb44bd63+        .quad   0x4b9ad8c6f86307ce+        .quad   0x21113531435d0c28+        .quad   0xd4a866c5657a772c+        .quad   0x5da6427e63247352+        .quad   0x1a94c688deac22ca+        .quad   0xb9066ef7bbae1ff8+        .quad   0x88ad8c388d59580f+        .quad   0x58f29abfe79f2ca8++        // 2^128 * 3 * G++        .quad   0xe90ecfab8de73e68+        .quad   0x54036f9f377e76a5+        .quad   0xf0495b0bbe015982+        .quad   0x577629c4a7f41e36+        .quad   0x4b5a64bf710ecdf6+        .quad   0xb14ce538462c293c+        .quad   0x3643d056d50b3ab9+        .quad   0x6af93724185b4870+        .quad   0x3220024509c6a888+        .quad   0xd2e036134b558973+        .quad   0x83e236233c33289f+        .quad   0x701f25bb0caec18f++        // 2^128 * 4 * G++        .quad   0xc3a8b0f8e4616ced+        .quad   0xf700660e9e25a87d+        .quad   0x61e3061ff4bca59c+        .quad   0x2e0c92bfbdc40be9+        .quad   0x9d18f6d97cbec113+        .quad   0x844a06e674bfdbe4+        .quad   0x20f5b522ac4e60d6+        .quad   0x720a5bc050955e51+        .quad   0x0c3f09439b805a35+        .quad   0xe84e8b376242abfc+        .quad   0x691417f35c229346+        .quad   0x0e9b9cbb144ef0ec++        // 2^128 * 5 * G++        .quad   0xfbbad48ffb5720ad+        .quad   0xee81916bdbf90d0e+        .quad   0xd4813152635543bf+        .quad   0x221104eb3f337bd8+        .quad   0x8dee9bd55db1beee+        .quad   0xc9c3ab370a723fb9+        .quad   0x44a8f1bf1c68d791+        .quad   0x366d44191cfd3cde+        .quad   0x9e3c1743f2bc8c14+        .quad   0x2eda26fcb5856c3b+        .quad   0xccb82f0e68a7fb97+        .quad   0x4167a4e6bc593244++        // 2^128 * 6 * G++        .quad   0x643b9d2876f62700+        .quad   0x5d1d9d400e7668eb+        .quad   0x1b4b430321fc0684+        .quad   0x7938bb7e2255246a+        .quad   0xc2be2665f8ce8fee+        .quad   0xe967ff14e880d62c+        .quad   0xf12e6e7e2f364eee+        .quad   0x34b33370cb7ed2f6+        .quad   0xcdc591ee8681d6cc+        .quad   0xce02109ced85a753+        .quad   0xed7485c158808883+        .quad   0x1176fc6e2dfe65e4++        // 2^128 * 7 * G++        .quad   0xb4af6cd05b9c619b+        .quad   0x2ddfc9f4b2a58480+        .quad   0x3d4fa502ebe94dc4+        .quad   0x08fc3a4c677d5f34+        .quad   0xdb90e28949770eb8+        .quad   0x98fbcc2aacf440a3+        .quad   0x21354ffeded7879b+        .quad   0x1f6a3e54f26906b6+        .quad   0x60a4c199d30734ea+        .quad   0x40c085b631165cd6+        .quad   0xe2333e23f7598295+        .quad   0x4f2fad0116b900d1++        // 2^128 * 8 * G++        .quad   0x44beb24194ae4e54+        .quad   0x5f541c511857ef6c+        .quad   0xa61e6b2d368d0498+        .quad   0x445484a4972ef7ab+        .quad   0x962cd91db73bb638+        .quad   0xe60577aafc129c08+        .quad   0x6f619b39f3b61689+        .quad   0x3451995f2944ee81+        .quad   0x9152fcd09fea7d7c+        .quad   0x4a816c94b0935cf6+        .quad   0x258e9aaa47285c40+        .quad   0x10b89ca6042893b7++        // 2^132 * 1 * G++        .quad   0x9b2a426e3b646025+        .quad   0x32127190385ce4cf+        .quad   0xa25cffc2dd6dea45+        .quad   0x06409010bea8de75+        .quad   0xd67cded679d34aa0+        .quad   0xcc0b9ec0cc4db39f+        .quad   0xa535a456e35d190f+        .quad   0x2e05d9eaf61f6fef+        .quad   0xc447901ad61beb59+        .quad   0x661f19bce5dc880a+        .quad   0x24685482b7ca6827+        .quad   0x293c778cefe07f26++        // 2^132 * 2 * G++        .quad   0x86809e7007069096+        .quad   0xaad75b15e4e50189+        .quad   0x07f35715a21a0147+        .quad   0x0487f3f112815d5e+        .quad   0x16c795d6a11ff200+        .quad   0xcb70d0e2b15815c9+        .quad   0x89f293209b5395b5+        .quad   0x50b8c2d031e47b4f+        .quad   0x48350c08068a4962+        .quad   0x6ffdd05351092c9a+        .quad   0x17af4f4aaf6fc8dd+        .quad   0x4b0553b53cdba58b++        // 2^132 * 3 * G++        .quad   0x9c65fcbe1b32ff79+        .quad   0xeb75ea9f03b50f9b+        .quad   0xfced2a6c6c07e606+        .quad   0x35106cd551717908+        .quad   0xbf05211b27c152d4+        .quad   0x5ec26849bd1af639+        .quad   0x5e0b2caa8e6fab98+        .quad   0x054c8bdd50bd0840+        .quad   0x38a0b12f1dcf073d+        .quad   0x4b60a8a3b7f6a276+        .quad   0xfed5ac25d3404f9a+        .quad   0x72e82d5e5505c229++        // 2^132 * 4 * G++        .quad   0x6b0b697ff0d844c8+        .quad   0xbb12f85cd979cb49+        .quad   0xd2a541c6c1da0f1f+        .quad   0x7b7c242958ce7211+        .quad   0x00d9cdfd69771d02+        .quad   0x410276cd6cfbf17e+        .quad   0x4c45306c1cb12ec7+        .quad   0x2857bf1627500861+        .quad   0x9f21903f0101689e+        .quad   0xd779dfd3bf861005+        .quad   0xa122ee5f3deb0f1b+        .quad   0x510df84b485a00d4++        // 2^132 * 5 * G++        .quad   0xa54133bb9277a1fa+        .quad   0x74ec3b6263991237+        .quad   0x1a3c54dc35d2f15a+        .quad   0x2d347144e482ba3a+        .quad   0x24b3c887c70ac15e+        .quad   0xb0f3a557fb81b732+        .quad   0x9b2cde2fe578cc1b+        .quad   0x4cf7ed0703b54f8e+        .quad   0x6bd47c6598fbee0f+        .quad   0x9e4733e2ab55be2d+        .quad   0x1093f624127610c5+        .quad   0x4e05e26ad0a1eaa4++        // 2^132 * 6 * G++        .quad   0xda9b6b624b531f20+        .quad   0x429a760e77509abb+        .quad   0xdbe9f522e823cb80+        .quad   0x618f1856880c8f82+        .quad   0x1833c773e18fe6c0+        .quad   0xe3c4711ad3c87265+        .quad   0x3bfd3c4f0116b283+        .quad   0x1955875eb4cd4db8+        .quad   0x6da6de8f0e399799+        .quad   0x7ad61aa440fda178+        .quad   0xb32cd8105e3563dd+        .quad   0x15f6beae2ae340ae++        // 2^132 * 7 * G++        .quad   0x862bcb0c31ec3a62+        .quad   0x810e2b451138f3c2+        .quad   0x788ec4b839dac2a4+        .quad   0x28f76867ae2a9281+        .quad   0xba9a0f7b9245e215+        .quad   0xf368612dd98c0dbb+        .quad   0x2e84e4cbf220b020+        .quad   0x6ba92fe962d90eda+        .quad   0x3e4df9655884e2aa+        .quad   0xbd62fbdbdbd465a5+        .quad   0xd7596caa0de9e524+        .quad   0x6e8042ccb2b1b3d7++        // 2^132 * 8 * G++        .quad   0xf10d3c29ce28ca6e+        .quad   0xbad34540fcb6093d+        .quad   0xe7426ed7a2ea2d3f+        .quad   0x08af9d4e4ff298b9+        .quad   0x1530653616521f7e+        .quad   0x660d06b896203dba+        .quad   0x2d3989bc545f0879+        .quad   0x4b5303af78ebd7b0+        .quad   0x72f8a6c3bebcbde8+        .quad   0x4f0fca4adc3a8e89+        .quad   0x6fa9d4e8c7bfdf7a+        .quad   0x0dcf2d679b624eb7++        // 2^136 * 1 * G++        .quad   0x3d5947499718289c+        .quad   0x12ebf8c524533f26+        .quad   0x0262bfcb14c3ef15+        .quad   0x20b878d577b7518e+        .quad   0x753941be5a45f06e+        .quad   0xd07caeed6d9c5f65+        .quad   0x11776b9c72ff51b6+        .quad   0x17d2d1d9ef0d4da9+        .quad   0x27f2af18073f3e6a+        .quad   0xfd3fe519d7521069+        .quad   0x22e3b72c3ca60022+        .quad   0x72214f63cc65c6a7++        // 2^136 * 2 * G++        .quad   0xb4e37f405307a693+        .quad   0xaba714d72f336795+        .quad   0xd6fbd0a773761099+        .quad   0x5fdf48c58171cbc9+        .quad   0x1d9db7b9f43b29c9+        .quad   0xd605824a4f518f75+        .quad   0xf2c072bd312f9dc4+        .quad   0x1f24ac855a1545b0+        .quad   0x24d608328e9505aa+        .quad   0x4748c1d10c1420ee+        .quad   0xc7ffe45c06fb25a2+        .quad   0x00ba739e2ae395e6++        // 2^136 * 3 * G++        .quad   0x592e98de5c8790d6+        .quad   0xe5bfb7d345c2a2df+        .quad   0x115a3b60f9b49922+        .quad   0x03283a3e67ad78f3+        .quad   0xae4426f5ea88bb26+        .quad   0x360679d984973bfb+        .quad   0x5c9f030c26694e50+        .quad   0x72297de7d518d226+        .quad   0x48241dc7be0cb939+        .quad   0x32f19b4d8b633080+        .quad   0xd3dfc90d02289308+        .quad   0x05e1296846271945++        // 2^136 * 4 * G++        .quad   0xba82eeb32d9c495a+        .quad   0xceefc8fcf12bb97c+        .quad   0xb02dabae93b5d1e0+        .quad   0x39c00c9c13698d9b+        .quad   0xadbfbbc8242c4550+        .quad   0xbcc80cecd03081d9+        .quad   0x843566a6f5c8df92+        .quad   0x78cf25d38258ce4c+        .quad   0x15ae6b8e31489d68+        .quad   0xaa851cab9c2bf087+        .quad   0xc9a75a97f04efa05+        .quad   0x006b52076b3ff832++        // 2^136 * 5 * G++        .quad   0x29e0cfe19d95781c+        .quad   0xb681df18966310e2+        .quad   0x57df39d370516b39+        .quad   0x4d57e3443bc76122+        .quad   0xf5cb7e16b9ce082d+        .quad   0x3407f14c417abc29+        .quad   0xd4b36bce2bf4a7ab+        .quad   0x7de2e9561a9f75ce+        .quad   0xde70d4f4b6a55ecb+        .quad   0x4801527f5d85db99+        .quad   0xdbc9c440d3ee9a81+        .quad   0x6b2a90af1a6029ed++        // 2^136 * 6 * G++        .quad   0x6923f4fc9ae61e97+        .quad   0x5735281de03f5fd1+        .quad   0xa764ae43e6edd12d+        .quad   0x5fd8f4e9d12d3e4a+        .quad   0x77ebf3245bb2d80a+        .quad   0xd8301b472fb9079b+        .quad   0xc647e6f24cee7333+        .quad   0x465812c8276c2109+        .quad   0x4d43beb22a1062d9+        .quad   0x7065fb753831dc16+        .quad   0x180d4a7bde2968d7+        .quad   0x05b32c2b1cb16790++        // 2^136 * 7 * G++        .quad   0xc8c05eccd24da8fd+        .quad   0xa1cf1aac05dfef83+        .quad   0xdbbeeff27df9cd61+        .quad   0x3b5556a37b471e99+        .quad   0xf7fca42c7ad58195+        .quad   0x3214286e4333f3cc+        .quad   0xb6c29d0d340b979d+        .quad   0x31771a48567307e1+        .quad   0x32b0c524e14dd482+        .quad   0xedb351541a2ba4b6+        .quad   0xa3d16048282b5af3+        .quad   0x4fc079d27a7336eb++        // 2^136 * 8 * G++        .quad   0x51c938b089bf2f7f+        .quad   0x2497bd6502dfe9a7+        .quad   0xffffc09c7880e453+        .quad   0x124567cecaf98e92+        .quad   0xdc348b440c86c50d+        .quad   0x1337cbc9cc94e651+        .quad   0x6422f74d643e3cb9+        .quad   0x241170c2bae3cd08+        .quad   0x3ff9ab860ac473b4+        .quad   0xf0911dee0113e435+        .quad   0x4ae75060ebc6c4af+        .quad   0x3f8612966c87000d++        // 2^140 * 1 * G++        .quad   0x0c9c5303f7957be4+        .quad   0xa3c31a20e085c145+        .quad   0xb0721d71d0850050+        .quad   0x0aba390eab0bf2da+        .quad   0x529fdffe638c7bf3+        .quad   0xdf2b9e60388b4995+        .quad   0xe027b34f1bad0249+        .quad   0x7bc92fc9b9fa74ed+        .quad   0x9f97ef2e801ad9f9+        .quad   0x83697d5479afda3a+        .quad   0xe906b3ffbd596b50+        .quad   0x02672b37dd3fb8e0++        // 2^140 * 2 * G++        .quad   0x48b2ca8b260885e4+        .quad   0xa4286bec82b34c1c+        .quad   0x937e1a2617f58f74+        .quad   0x741d1fcbab2ca2a5+        .quad   0xee9ba729398ca7f5+        .quad   0xeb9ca6257a4849db+        .quad   0x29eb29ce7ec544e1+        .quad   0x232ca21ef736e2c8+        .quad   0xbf61423d253fcb17+        .quad   0x08803ceafa39eb14+        .quad   0xf18602df9851c7af+        .quad   0x0400f3a049e3414b++        // 2^140 * 3 * G++        .quad   0xabce0476ba61c55b+        .quad   0x36a3d6d7c4d39716+        .quad   0x6eb259d5e8d82d09+        .quad   0x0c9176e984d756fb+        .quad   0x2efba412a06e7b06+        .quad   0x146785452c8d2560+        .quad   0xdf9713ebd67a91c7+        .quad   0x32830ac7157eadf3+        .quad   0x0e782a7ab73769e8+        .quad   0x04a05d7875b18e2c+        .quad   0x29525226ebcceae1+        .quad   0x0d794f8383eba820++        // 2^140 * 4 * G++        .quad   0xff35f5cb9e1516f4+        .quad   0xee805bcf648aae45+        .quad   0xf0d73c2bb93a9ef3+        .quad   0x097b0bf22092a6c2+        .quad   0x7be44ce7a7a2e1ac+        .quad   0x411fd93efad1b8b7+        .quad   0x1734a1d70d5f7c9b+        .quad   0x0d6592233127db16+        .quad   0xc48bab1521a9d733+        .quad   0xa6c2eaead61abb25+        .quad   0x625c6c1cc6cb4305+        .quad   0x7fc90fea93eb3a67++        // 2^140 * 5 * G++        .quad   0x0408f1fe1f5c5926+        .quad   0x1a8f2f5e3b258bf4+        .quad   0x40a951a2fdc71669+        .quad   0x6598ee93c98b577e+        .quad   0xc527deb59c7cb23d+        .quad   0x955391695328404e+        .quad   0xd64392817ccf2c7a+        .quad   0x6ce97dabf7d8fa11+        .quad   0x25b5a8e50ef7c48f+        .quad   0xeb6034116f2ce532+        .quad   0xc5e75173e53de537+        .quad   0x73119fa08c12bb03++        // 2^140 * 6 * G++        .quad   0xed30129453f1a4cb+        .quad   0xbce621c9c8f53787+        .quad   0xfacb2b1338bee7b9+        .quad   0x3025798a9ea8428c+        .quad   0x7845b94d21f4774d+        .quad   0xbf62f16c7897b727+        .quad   0x671857c03c56522b+        .quad   0x3cd6a85295621212+        .quad   0x3fecde923aeca999+        .quad   0xbdaa5b0062e8c12f+        .quad   0x67b99dfc96988ade+        .quad   0x3f52c02852661036++        // 2^140 * 7 * G++        .quad   0xffeaa48e2a1351c6+        .quad   0x28624754fa7f53d7+        .quad   0x0b5ba9e57582ddf1+        .quad   0x60c0104ba696ac59+        .quad   0x9258bf99eec416c6+        .quad   0xac8a5017a9d2f671+        .quad   0x629549ab16dea4ab+        .quad   0x05d0e85c99091569+        .quad   0x051de020de9cbe97+        .quad   0xfa07fc56b50bcf74+        .quad   0x378cec9f0f11df65+        .quad   0x36853c69ab96de4d++        // 2^140 * 8 * G++        .quad   0x36d9b8de78f39b2d+        .quad   0x7f42ed71a847b9ec+        .quad   0x241cd1d679bd3fde+        .quad   0x6a704fec92fbce6b+        .quad   0x4433c0b0fac5e7be+        .quad   0x724bae854c08dcbe+        .quad   0xf1f24cc446978f9b+        .quad   0x4a0aff6d62825fc8+        .quad   0xe917fb9e61095301+        .quad   0xc102df9402a092f8+        .quad   0xbf09e2f5fa66190b+        .quad   0x681109bee0dcfe37++        // 2^144 * 1 * G++        .quad   0x559a0cc9782a0dde+        .quad   0x551dcdb2ea718385+        .quad   0x7f62865b31ef238c+        .quad   0x504aa7767973613d+        .quad   0x9c18fcfa36048d13+        .quad   0x29159db373899ddd+        .quad   0xdc9f350b9f92d0aa+        .quad   0x26f57eee878a19d4+        .quad   0x0cab2cd55687efb1+        .quad   0x5180d162247af17b+        .quad   0x85c15a344f5a2467+        .quad   0x4041943d9dba3069++        // 2^144 * 2 * G++        .quad   0xc3c0eeba43ebcc96+        .quad   0x8d749c9c26ea9caf+        .quad   0xd9fa95ee1c77ccc6+        .quad   0x1420a1d97684340f+        .quad   0x4b217743a26caadd+        .quad   0x47a6b424648ab7ce+        .quad   0xcb1d4f7a03fbc9e3+        .quad   0x12d931429800d019+        .quad   0x00c67799d337594f+        .quad   0x5e3c5140b23aa47b+        .quad   0x44182854e35ff395+        .quad   0x1b4f92314359a012++        // 2^144 * 3 * G++        .quad   0x3e5c109d89150951+        .quad   0x39cefa912de9696a+        .quad   0x20eae43f975f3020+        .quad   0x239b572a7f132dae+        .quad   0x33cf3030a49866b1+        .quad   0x251f73d2215f4859+        .quad   0xab82aa4051def4f6+        .quad   0x5ff191d56f9a23f6+        .quad   0x819ed433ac2d9068+        .quad   0x2883ab795fc98523+        .quad   0xef4572805593eb3d+        .quad   0x020c526a758f36cb++        // 2^144 * 4 * G++        .quad   0x779834f89ed8dbbc+        .quad   0xc8f2aaf9dc7ca46c+        .quad   0xa9524cdca3e1b074+        .quad   0x02aacc4615313877+        .quad   0xe931ef59f042cc89+        .quad   0x2c589c9d8e124bb6+        .quad   0xadc8e18aaec75997+        .quad   0x452cfe0a5602c50c+        .quad   0x86a0f7a0647877df+        .quad   0xbbc464270e607c9f+        .quad   0xab17ea25f1fb11c9+        .quad   0x4cfb7d7b304b877b++        // 2^144 * 5 * G++        .quad   0x72b43d6cb89b75fe+        .quad   0x54c694d99c6adc80+        .quad   0xb8c3aa373ee34c9f+        .quad   0x14b4622b39075364+        .quad   0xe28699c29789ef12+        .quad   0x2b6ecd71df57190d+        .quad   0xc343c857ecc970d0+        .quad   0x5b1d4cbc434d3ac5+        .quad   0xb6fb2615cc0a9f26+        .quad   0x3a4f0e2bb88dcce5+        .quad   0x1301498b3369a705+        .quad   0x2f98f71258592dd1++        // 2^144 * 6 * G++        .quad   0x0c94a74cb50f9e56+        .quad   0x5b1ff4a98e8e1320+        .quad   0x9a2acc2182300f67+        .quad   0x3a6ae249d806aaf9+        .quad   0x2e12ae444f54a701+        .quad   0xfcfe3ef0a9cbd7de+        .quad   0xcebf890d75835de0+        .quad   0x1d8062e9e7614554+        .quad   0x657ada85a9907c5a+        .quad   0x1a0ea8b591b90f62+        .quad   0x8d0e1dfbdf34b4e9+        .quad   0x298b8ce8aef25ff3++        // 2^144 * 7 * G++        .quad   0x2a927953eff70cb2+        .quad   0x4b89c92a79157076+        .quad   0x9418457a30a7cf6a+        .quad   0x34b8a8404d5ce485+        .quad   0x837a72ea0a2165de+        .quad   0x3fab07b40bcf79f6+        .quad   0x521636c77738ae70+        .quad   0x6ba6271803a7d7dc+        .quad   0xc26eecb583693335+        .quad   0xd5a813df63b5fefd+        .quad   0xa293aa9aa4b22573+        .quad   0x71d62bdd465e1c6a++        // 2^144 * 8 * G++        .quad   0x6533cc28d378df80+        .quad   0xf6db43790a0fa4b4+        .quad   0xe3645ff9f701da5a+        .quad   0x74d5f317f3172ba4+        .quad   0xcd2db5dab1f75ef5+        .quad   0xd77f95cf16b065f5+        .quad   0x14571fea3f49f085+        .quad   0x1c333621262b2b3d+        .quad   0xa86fe55467d9ca81+        .quad   0x398b7c752b298c37+        .quad   0xda6d0892e3ac623b+        .quad   0x4aebcc4547e9d98c++        // 2^148 * 1 * G++        .quad   0x53175a7205d21a77+        .quad   0xb0c04422d3b934d4+        .quad   0xadd9f24bdd5deadc+        .quad   0x074f46e69f10ff8c+        .quad   0x0de9b204a059a445+        .quad   0xe15cb4aa4b17ad0f+        .quad   0xe1bbec521f79c557+        .quad   0x2633f1b9d071081b+        .quad   0xc1fb4177018b9910+        .quad   0xa6ea20dc6c0fe140+        .quad   0xd661f3e74354c6ff+        .quad   0x5ecb72e6f1a3407a++        // 2^148 * 2 * G++        .quad   0xa515a31b2259fb4e+        .quad   0x0960f3972bcac52f+        .quad   0xedb52fec8d3454cb+        .quad   0x382e2720c476c019+        .quad   0xfeeae106e8e86997+        .quad   0x9863337f98d09383+        .quad   0x9470480eaa06ebef+        .quad   0x038b6898d4c5c2d0+        .quad   0xf391c51d8ace50a6+        .quad   0x3142d0b9ae2d2948+        .quad   0xdb4d5a1a7f24ca80+        .quad   0x21aeba8b59250ea8++        // 2^148 * 3 * G++        .quad   0x24f13b34cf405530+        .quad   0x3c44ea4a43088af7+        .quad   0x5dd5c5170006a482+        .quad   0x118eb8f8890b086d+        .quad   0x53853600f0087f23+        .quad   0x4c461879da7d5784+        .quad   0x6af303deb41f6860+        .quad   0x0a3c16c5c27c18ed+        .quad   0x17e49c17cc947f3d+        .quad   0xccc6eda6aac1d27b+        .quad   0xdf6092ceb0f08e56+        .quad   0x4909b3e22c67c36b++        // 2^148 * 4 * G++        .quad   0x9c9c85ea63fe2e89+        .quad   0xbe1baf910e9412ec+        .quad   0x8f7baa8a86fbfe7b+        .quad   0x0fb17f9fef968b6c+        .quad   0x59a16676706ff64e+        .quad   0x10b953dd0d86a53d+        .quad   0x5848e1e6ce5c0b96+        .quad   0x2d8b78e712780c68+        .quad   0x79d5c62eafc3902b+        .quad   0x773a215289e80728+        .quad   0xc38ae640e10120b9+        .quad   0x09ae23717b2b1a6d++        // 2^148 * 5 * G++        .quad   0xbb6a192a4e4d083c+        .quad   0x34ace0630029e192+        .quad   0x98245a59aafabaeb+        .quad   0x6d9c8a9ada97faac+        .quad   0x10ab8fa1ad32b1d0+        .quad   0xe9aced1be2778b24+        .quad   0xa8856bc0373de90f+        .quad   0x66f35ddddda53996+        .quad   0xd27d9afb24997323+        .quad   0x1bb7e07ef6f01d2e+        .quad   0x2ba7472df52ecc7f+        .quad   0x03019b4f646f9dc8++        // 2^148 * 6 * G++        .quad   0x04a186b5565345cd+        .quad   0xeee76610bcc4116a+        .quad   0x689c73b478fb2a45+        .quad   0x387dcbff65697512+        .quad   0xaf09b214e6b3dc6b+        .quad   0x3f7573b5ad7d2f65+        .quad   0xd019d988100a23b0+        .quad   0x392b63a58b5c35f7+        .quad   0x4093addc9c07c205+        .quad   0xc565be15f532c37e+        .quad   0x63dbecfd1583402a+        .quad   0x61722b4aef2e032e++        // 2^148 * 7 * G++        .quad   0x0012aafeecbd47af+        .quad   0x55a266fb1cd46309+        .quad   0xf203eb680967c72c+        .quad   0x39633944ca3c1429+        .quad   0xd6b07a5581cb0e3c+        .quad   0x290ff006d9444969+        .quad   0x08680b6a16dcda1f+        .quad   0x5568d2b75a06de59+        .quad   0x8d0cb88c1b37cfe1+        .quad   0x05b6a5a3053818f3+        .quad   0xf2e9bc04b787d959+        .quad   0x6beba1249add7f64++        // 2^148 * 8 * G++        .quad   0x1d06005ca5b1b143+        .quad   0x6d4c6bb87fd1cda2+        .quad   0x6ef5967653fcffe7+        .quad   0x097c29e8c1ce1ea5+        .quad   0x5c3cecb943f5a53b+        .quad   0x9cc9a61d06c08df2+        .quad   0xcfba639a85895447+        .quad   0x5a845ae80df09fd5+        .quad   0x4ce97dbe5deb94ca+        .quad   0x38d0a4388c709c48+        .quad   0xc43eced4a169d097+        .quad   0x0a1249fff7e587c3++        // 2^152 * 1 * G++        .quad   0x12f0071b276d01c9+        .quad   0xe7b8bac586c48c70+        .quad   0x5308129b71d6fba9+        .quad   0x5d88fbf95a3db792+        .quad   0x0b408d9e7354b610+        .quad   0x806b32535ba85b6e+        .quad   0xdbe63a034a58a207+        .quad   0x173bd9ddc9a1df2c+        .quad   0x2b500f1efe5872df+        .quad   0x58d6582ed43918c1+        .quad   0xe6ed278ec9673ae0+        .quad   0x06e1cd13b19ea319++        // 2^152 * 2 * G++        .quad   0x40d0ad516f166f23+        .quad   0x118e32931fab6abe+        .quad   0x3fe35e14a04d088e+        .quad   0x3080603526e16266+        .quad   0x472baf629e5b0353+        .quad   0x3baa0b90278d0447+        .quad   0x0c785f469643bf27+        .quad   0x7f3a6a1a8d837b13+        .quad   0xf7e644395d3d800b+        .quad   0x95a8d555c901edf6+        .quad   0x68cd7830592c6339+        .quad   0x30d0fded2e51307e++        // 2^152 * 3 * G++        .quad   0xe0594d1af21233b3+        .quad   0x1bdbe78ef0cc4d9c+        .quad   0x6965187f8f499a77+        .quad   0x0a9214202c099868+        .quad   0x9cb4971e68b84750+        .quad   0xa09572296664bbcf+        .quad   0x5c8de72672fa412b+        .quad   0x4615084351c589d9+        .quad   0xbc9019c0aeb9a02e+        .quad   0x55c7110d16034cae+        .quad   0x0e6df501659932ec+        .quad   0x3bca0d2895ca5dfe++        // 2^152 * 4 * G++        .quad   0x40f031bc3c5d62a4+        .quad   0x19fc8b3ecff07a60+        .quad   0x98183da2130fb545+        .quad   0x5631deddae8f13cd+        .quad   0x9c688eb69ecc01bf+        .quad   0xf0bc83ada644896f+        .quad   0xca2d955f5f7a9fe2+        .quad   0x4ea8b4038df28241+        .quad   0x2aed460af1cad202+        .quad   0x46305305a48cee83+        .quad   0x9121774549f11a5f+        .quad   0x24ce0930542ca463++        // 2^152 * 5 * G++        .quad   0x1fe890f5fd06c106+        .quad   0xb5c468355d8810f2+        .quad   0x827808fe6e8caf3e+        .quad   0x41d4e3c28a06d74b+        .quad   0x3fcfa155fdf30b85+        .quad   0xd2f7168e36372ea4+        .quad   0xb2e064de6492f844+        .quad   0x549928a7324f4280+        .quad   0xf26e32a763ee1a2e+        .quad   0xae91e4b7d25ffdea+        .quad   0xbc3bd33bd17f4d69+        .quad   0x491b66dec0dcff6a++        // 2^152 * 6 * G++        .quad   0x98f5b13dc7ea32a7+        .quad   0xe3d5f8cc7e16db98+        .quad   0xac0abf52cbf8d947+        .quad   0x08f338d0c85ee4ac+        .quad   0x75f04a8ed0da64a1+        .quad   0xed222caf67e2284b+        .quad   0x8234a3791f7b7ba4+        .quad   0x4cf6b8b0b7018b67+        .quad   0xc383a821991a73bd+        .quad   0xab27bc01df320c7a+        .quad   0xc13d331b84777063+        .quad   0x530d4a82eb078a99++        // 2^152 * 7 * G++        .quad   0x004c3630e1f94825+        .quad   0x7e2d78268cab535a+        .quad   0xc7482323cc84ff8b+        .quad   0x65ea753f101770b9+        .quad   0x6d6973456c9abf9e+        .quad   0x257fb2fc4900a880+        .quad   0x2bacf412c8cfb850+        .quad   0x0db3e7e00cbfbd5b+        .quad   0x3d66fc3ee2096363+        .quad   0x81d62c7f61b5cb6b+        .quad   0x0fbe044213443b1a+        .quad   0x02a4ec1921e1a1db++        // 2^152 * 8 * G++        .quad   0x5ce6259a3b24b8a2+        .quad   0xb8577acc45afa0b8+        .quad   0xcccbe6e88ba07037+        .quad   0x3d143c51127809bf+        .quad   0xf5c86162f1cf795f+        .quad   0x118c861926ee57f2+        .quad   0x172124851c063578+        .quad   0x36d12b5dec067fcf+        .quad   0x126d279179154557+        .quad   0xd5e48f5cfc783a0a+        .quad   0x36bdb6e8df179bac+        .quad   0x2ef517885ba82859++        // 2^156 * 1 * G++        .quad   0x88bd438cd11e0d4a+        .quad   0x30cb610d43ccf308+        .quad   0xe09a0e3791937bcc+        .quad   0x4559135b25b1720c+        .quad   0x1ea436837c6da1e9+        .quad   0xf9c189af1fb9bdbe+        .quad   0x303001fcce5dd155+        .quad   0x28a7c99ebc57be52+        .quad   0xb8fd9399e8d19e9d+        .quad   0x908191cb962423ff+        .quad   0xb2b948d747c742a3+        .quad   0x37f33226d7fb44c4++        // 2^156 * 2 * G++        .quad   0x0dae8767b55f6e08+        .quad   0x4a43b3b35b203a02+        .quad   0xe3725a6e80af8c79+        .quad   0x0f7a7fd1705fa7a3+        .quad   0x33912553c821b11d+        .quad   0x66ed42c241e301df+        .quad   0x066fcc11104222fd+        .quad   0x307a3b41c192168f+        .quad   0x8eeb5d076eb55ce0+        .quad   0x2fc536bfaa0d925a+        .quad   0xbe81830fdcb6c6e8+        .quad   0x556c7045827baf52++        // 2^156 * 3 * G++        .quad   0x8e2b517302e9d8b7+        .quad   0xe3e52269248714e8+        .quad   0xbd4fbd774ca960b5+        .quad   0x6f4b4199c5ecada9+        .quad   0xb94b90022bf44406+        .quad   0xabd4237eff90b534+        .quad   0x7600a960faf86d3a+        .quad   0x2f45abdac2322ee3+        .quad   0x61af4912c8ef8a6a+        .quad   0xe58fa4fe43fb6e5e+        .quad   0xb5afcc5d6fd427cf+        .quad   0x6a5393281e1e11eb++        // 2^156 * 4 * G++        .quad   0xf3da5139a5d1ee89+        .quad   0x8145457cff936988+        .quad   0x3f622fed00e188c4+        .quad   0x0f513815db8b5a3d+        .quad   0x0fff04fe149443cf+        .quad   0x53cac6d9865cddd7+        .quad   0x31385b03531ed1b7+        .quad   0x5846a27cacd1039d+        .quad   0x4ff5cdac1eb08717+        .quad   0x67e8b29590f2e9bc+        .quad   0x44093b5e237afa99+        .quad   0x0d414bed8708b8b2++        // 2^156 * 5 * G++        .quad   0xcfb68265fd0e75f6+        .quad   0xe45b3e28bb90e707+        .quad   0x7242a8de9ff92c7a+        .quad   0x685b3201933202dd+        .quad   0x81886a92294ac9e8+        .quad   0x23162b45d55547be+        .quad   0x94cfbc4403715983+        .quad   0x50eb8fdb134bc401+        .quad   0xc0b73ec6d6b330cd+        .quad   0x84e44807132faff1+        .quad   0x732b7352c4a5dee1+        .quad   0x5d7c7cf1aa7cd2d2++        // 2^156 * 6 * G++        .quad   0xaf3b46bf7a4aafa2+        .quad   0xb78705ec4d40d411+        .quad   0x114f0c6aca7c15e3+        .quad   0x3f364faaa9489d4d+        .quad   0x33d1013e9b73a562+        .quad   0x925cef5748ec26e1+        .quad   0xa7fce614dd468058+        .quad   0x78b0fad41e9aa438+        .quad   0xbf56a431ed05b488+        .quad   0xa533e66c9c495c7e+        .quad   0xe8652baf87f3651a+        .quad   0x0241800059d66c33++        // 2^156 * 7 * G++        .quad   0xceb077fea37a5be4+        .quad   0xdb642f02e5a5eeb7+        .quad   0xc2e6d0c5471270b8+        .quad   0x4771b65538e4529c+        .quad   0x28350c7dcf38ea01+        .quad   0x7c6cdbc0b2917ab6+        .quad   0xace7cfbe857082f7+        .quad   0x4d2845aba2d9a1e0+        .quad   0xbb537fe0447070de+        .quad   0xcba744436dd557df+        .quad   0xd3b5a3473600dbcb+        .quad   0x4aeabbe6f9ffd7f8++        // 2^156 * 8 * G++        .quad   0x4630119e40d8f78c+        .quad   0xa01a9bc53c710e11+        .quad   0x486d2b258910dd79+        .quad   0x1e6c47b3db0324e5+        .quad   0x6a2134bcc4a9c8f2+        .quad   0xfbf8fd1c8ace2e37+        .quad   0x000ae3049911a0ba+        .quad   0x046e3a616bc89b9e+        .quad   0x14e65442f03906be+        .quad   0x4a019d54e362be2a+        .quad   0x68ccdfec8dc230c7+        .quad   0x7cfb7e3faf6b861c++        // 2^160 * 1 * G++        .quad   0x4637974e8c58aedc+        .quad   0xb9ef22fbabf041a4+        .quad   0xe185d956e980718a+        .quad   0x2f1b78fab143a8a6+        .quad   0x96eebffb305b2f51+        .quad   0xd3f938ad889596b8+        .quad   0xf0f52dc746d5dd25+        .quad   0x57968290bb3a0095+        .quad   0xf71ab8430a20e101+        .quad   0xf393658d24f0ec47+        .quad   0xcf7509a86ee2eed1+        .quad   0x7dc43e35dc2aa3e1++        // 2^160 * 2 * G++        .quad   0x85966665887dd9c3+        .quad   0xc90f9b314bb05355+        .quad   0xc6e08df8ef2079b1+        .quad   0x7ef72016758cc12f+        .quad   0x5a782a5c273e9718+        .quad   0x3576c6995e4efd94+        .quad   0x0f2ed8051f237d3e+        .quad   0x044fb81d82d50a99+        .quad   0xc1df18c5a907e3d9+        .quad   0x57b3371dce4c6359+        .quad   0xca704534b201bb49+        .quad   0x7f79823f9c30dd2e++        // 2^160 * 3 * G++        .quad   0x8334d239a3b513e8+        .quad   0xc13670d4b91fa8d8+        .quad   0x12b54136f590bd33+        .quad   0x0a4e0373d784d9b4+        .quad   0x6a9c1ff068f587ba+        .quad   0x0827894e0050c8de+        .quad   0x3cbf99557ded5be7+        .quad   0x64a9b0431c06d6f0+        .quad   0x2eb3d6a15b7d2919+        .quad   0xb0b4f6a0d53a8235+        .quad   0x7156ce4389a45d47+        .quad   0x071a7d0ace18346c++        // 2^160 * 4 * G++        .quad   0xd3072daac887ba0b+        .quad   0x01262905bfa562ee+        .quad   0xcf543002c0ef768b+        .quad   0x2c3bcc7146ea7e9c+        .quad   0xcc0c355220e14431+        .quad   0x0d65950709b15141+        .quad   0x9af5621b209d5f36+        .quad   0x7c69bcf7617755d3+        .quad   0x07f0d7eb04e8295f+        .quad   0x10db18252f50f37d+        .quad   0xe951a9a3171798d7+        .quad   0x6f5a9a7322aca51d++        // 2^160 * 5 * G++        .quad   0x8ba1000c2f41c6c5+        .quad   0xc49f79c10cfefb9b+        .quad   0x4efa47703cc51c9f+        .quad   0x494e21a2e147afca+        .quad   0xe729d4eba3d944be+        .quad   0x8d9e09408078af9e+        .quad   0x4525567a47869c03+        .quad   0x02ab9680ee8d3b24+        .quad   0xefa48a85dde50d9a+        .quad   0x219a224e0fb9a249+        .quad   0xfa091f1dd91ef6d9+        .quad   0x6b5d76cbea46bb34++        // 2^160 * 6 * G++        .quad   0x8857556cec0cd994+        .quad   0x6472dc6f5cd01dba+        .quad   0xaf0169148f42b477+        .quad   0x0ae333f685277354+        .quad   0xe0f941171e782522+        .quad   0xf1e6ae74036936d3+        .quad   0x408b3ea2d0fcc746+        .quad   0x16fb869c03dd313e+        .quad   0x288e199733b60962+        .quad   0x24fc72b4d8abe133+        .quad   0x4811f7ed0991d03e+        .quad   0x3f81e38b8f70d075++        // 2^160 * 7 * G++        .quad   0x7f910fcc7ed9affe+        .quad   0x545cb8a12465874b+        .quad   0xa8397ed24b0c4704+        .quad   0x50510fc104f50993+        .quad   0x0adb7f355f17c824+        .quad   0x74b923c3d74299a4+        .quad   0xd57c3e8bcbf8eaf7+        .quad   0x0ad3e2d34cdedc3d+        .quad   0x6f0c0fc5336e249d+        .quad   0x745ede19c331cfd9+        .quad   0xf2d6fd0009eefe1c+        .quad   0x127c158bf0fa1ebe++        // 2^160 * 8 * G++        .quad   0xf6197c422e9879a2+        .quad   0xa44addd452ca3647+        .quad   0x9b413fc14b4eaccb+        .quad   0x354ef87d07ef4f68+        .quad   0xdea28fc4ae51b974+        .quad   0x1d9973d3744dfe96+        .quad   0x6240680b873848a8+        .quad   0x4ed82479d167df95+        .quad   0xfee3b52260c5d975+        .quad   0x50352efceb41b0b8+        .quad   0x8808ac30a9f6653c+        .quad   0x302d92d20539236d++        // 2^164 * 1 * G++        .quad   0x4c59023fcb3efb7c+        .quad   0x6c2fcb99c63c2a94+        .quad   0xba4190e2c3c7e084+        .quad   0x0e545daea51874d9+        .quad   0x957b8b8b0df53c30+        .quad   0x2a1c770a8e60f098+        .quad   0xbbc7a670345796de+        .quad   0x22a48f9a90c99bc9+        .quad   0x6b7dc0dc8d3fac58+        .quad   0x5497cd6ce6e42bfd+        .quad   0x542f7d1bf400d305+        .quad   0x4159f47f048d9136++        // 2^164 * 2 * G++        .quad   0x20ad660839e31e32+        .quad   0xf81e1bd58405be50+        .quad   0xf8064056f4dabc69+        .quad   0x14d23dd4ce71b975+        .quad   0x748515a8bbd24839+        .quad   0x77128347afb02b55+        .quad   0x50ba2ac649a2a17f+        .quad   0x060525513ad730f1+        .quad   0xf2398e098aa27f82+        .quad   0x6d7982bb89a1b024+        .quad   0xfa694084214dd24c+        .quad   0x71ab966fa32301c3++        // 2^164 * 3 * G++        .quad   0x2dcbd8e34ded02fc+        .quad   0x1151f3ec596f22aa+        .quad   0xbca255434e0328da+        .quad   0x35768fbe92411b22+        .quad   0xb1088a0702809955+        .quad   0x43b273ea0b43c391+        .quad   0xca9b67aefe0686ed+        .quad   0x605eecbf8335f4ed+        .quad   0x83200a656c340431+        .quad   0x9fcd71678ee59c2f+        .quad   0x75d4613f71300f8a+        .quad   0x7a912faf60f542f9++        // 2^164 * 4 * G++        .quad   0xb204585e5edc1a43+        .quad   0x9f0e16ee5897c73c+        .quad   0x5b82c0ae4e70483c+        .quad   0x624a170e2bddf9be+        .quad   0x253f4f8dfa2d5597+        .quad   0x25e49c405477130c+        .quad   0x00c052e5996b1102+        .quad   0x33cb966e33bb6c4a+        .quad   0x597028047f116909+        .quad   0x828ac41c1e564467+        .quad   0x70417dbde6217387+        .quad   0x721627aefbac4384++        // 2^164 * 5 * G++        .quad   0x97d03bc38736add5+        .quad   0x2f1422afc532b130+        .quad   0x3aa68a057101bbc4+        .quad   0x4c946cf7e74f9fa7+        .quad   0xfd3097bc410b2f22+        .quad   0xf1a05da7b5cfa844+        .quad   0x61289a1def57ca74+        .quad   0x245ea199bb821902+        .quad   0xaedca66978d477f8+        .quad   0x1898ba3c29117fe1+        .quad   0xcf73f983720cbd58+        .quad   0x67da12e6b8b56351++        // 2^164 * 6 * G++        .quad   0x7067e187b4bd6e07+        .quad   0x6e8f0203c7d1fe74+        .quad   0x93c6aa2f38c85a30+        .quad   0x76297d1f3d75a78a+        .quad   0x2b7ef3d38ec8308c+        .quad   0x828fd7ec71eb94ab+        .quad   0x807c3b36c5062abd+        .quad   0x0cb64cb831a94141+        .quad   0x3030fc33534c6378+        .quad   0xb9635c5ce541e861+        .quad   0x15d9a9bed9b2c728+        .quad   0x49233ea3f3775dcb++        // 2^164 * 7 * G++        .quad   0x629398fa8dbffc3a+        .quad   0xe12fe52dd54db455+        .quad   0xf3be11dfdaf25295+        .quad   0x628b140dce5e7b51+        .quad   0x7b3985fe1c9f249b+        .quad   0x4fd6b2d5a1233293+        .quad   0xceb345941adf4d62+        .quad   0x6987ff6f542de50c+        .quad   0x47e241428f83753c+        .quad   0x6317bebc866af997+        .quad   0xdabb5b433d1a9829+        .quad   0x074d8d245287fb2d++        // 2^164 * 8 * G++        .quad   0x8337d9cd440bfc31+        .quad   0x729d2ca1af318fd7+        .quad   0xa040a4a4772c2070+        .quad   0x46002ef03a7349be+        .quad   0x481875c6c0e31488+        .quad   0x219429b2e22034b4+        .quad   0x7223c98a31283b65+        .quad   0x3420d60b342277f9+        .quad   0xfaa23adeaffe65f7+        .quad   0x78261ed45be0764c+        .quad   0x441c0a1e2f164403+        .quad   0x5aea8e567a87d395++        // 2^168 * 1 * G++        .quad   0x7813c1a2bca4283d+        .quad   0xed62f091a1863dd9+        .quad   0xaec7bcb8c268fa86+        .quad   0x10e5d3b76f1cae4c+        .quad   0x2dbc6fb6e4e0f177+        .quad   0x04e1bf29a4bd6a93+        .quad   0x5e1966d4787af6e8+        .quad   0x0edc5f5eb426d060+        .quad   0x5453bfd653da8e67+        .quad   0xe9dc1eec24a9f641+        .quad   0xbf87263b03578a23+        .quad   0x45b46c51361cba72++        // 2^168 * 2 * G++        .quad   0xa9402abf314f7fa1+        .quad   0xe257f1dc8e8cf450+        .quad   0x1dbbd54b23a8be84+        .quad   0x2177bfa36dcb713b+        .quad   0xce9d4ddd8a7fe3e4+        .quad   0xab13645676620e30+        .quad   0x4b594f7bb30e9958+        .quad   0x5c1c0aef321229df+        .quad   0x37081bbcfa79db8f+        .quad   0x6048811ec25f59b3+        .quad   0x087a76659c832487+        .quad   0x4ae619387d8ab5bb++        // 2^168 * 3 * G++        .quad   0x8ddbf6aa5344a32e+        .quad   0x7d88eab4b41b4078+        .quad   0x5eb0eb974a130d60+        .quad   0x1a00d91b17bf3e03+        .quad   0x61117e44985bfb83+        .quad   0xfce0462a71963136+        .quad   0x83ac3448d425904b+        .quad   0x75685abe5ba43d64+        .quad   0x6e960933eb61f2b2+        .quad   0x543d0fa8c9ff4952+        .quad   0xdf7275107af66569+        .quad   0x135529b623b0e6aa++        // 2^168 * 4 * G++        .quad   0x18f0dbd7add1d518+        .quad   0x979f7888cfc11f11+        .quad   0x8732e1f07114759b+        .quad   0x79b5b81a65ca3a01+        .quad   0xf5c716bce22e83fe+        .quad   0xb42beb19e80985c1+        .quad   0xec9da63714254aae+        .quad   0x5972ea051590a613+        .quad   0x0fd4ac20dc8f7811+        .quad   0x9a9ad294ac4d4fa8+        .quad   0xc01b2d64b3360434+        .quad   0x4f7e9c95905f3bdb++        // 2^168 * 5 * G++        .quad   0x62674bbc5781302e+        .quad   0xd8520f3989addc0f+        .quad   0x8c2999ae53fbd9c6+        .quad   0x31993ad92e638e4c+        .quad   0x71c8443d355299fe+        .quad   0x8bcd3b1cdbebead7+        .quad   0x8092499ef1a49466+        .quad   0x1942eec4a144adc8+        .quad   0x7dac5319ae234992+        .quad   0x2c1b3d910cea3e92+        .quad   0x553ce494253c1122+        .quad   0x2a0a65314ef9ca75++        // 2^168 * 6 * G++        .quad   0x2db7937ff7f927c2+        .quad   0xdb741f0617d0a635+        .quad   0x5982f3a21155af76+        .quad   0x4cf6e218647c2ded+        .quad   0xcf361acd3c1c793a+        .quad   0x2f9ebcac5a35bc3b+        .quad   0x60e860e9a8cda6ab+        .quad   0x055dc39b6dea1a13+        .quad   0xb119227cc28d5bb6+        .quad   0x07e24ebc774dffab+        .quad   0xa83c78cee4a32c89+        .quad   0x121a307710aa24b6++        // 2^168 * 7 * G++        .quad   0xe4db5d5e9f034a97+        .quad   0xe153fc093034bc2d+        .quad   0x460546919551d3b1+        .quad   0x333fc76c7a40e52d+        .quad   0xd659713ec77483c9+        .quad   0x88bfe077b82b96af+        .quad   0x289e28231097bcd3+        .quad   0x527bb94a6ced3a9b+        .quad   0x563d992a995b482e+        .quad   0x3405d07c6e383801+        .quad   0x485035de2f64d8e5+        .quad   0x6b89069b20a7a9f7++        // 2^168 * 8 * G++        .quad   0x812aa0416270220d+        .quad   0x995a89faf9245b4e+        .quad   0xffadc4ce5072ef05+        .quad   0x23bc2103aa73eb73+        .quad   0x4082fa8cb5c7db77+        .quad   0x068686f8c734c155+        .quad   0x29e6c8d9f6e7a57e+        .quad   0x0473d308a7639bcf+        .quad   0xcaee792603589e05+        .quad   0x2b4b421246dcc492+        .quad   0x02a1ef74e601a94f+        .quad   0x102f73bfde04341a++        // 2^172 * 1 * G++        .quad   0xb5a2d50c7ec20d3e+        .quad   0xc64bdd6ea0c97263+        .quad   0x56e89052c1ff734d+        .quad   0x4929c6f72b2ffaba+        .quad   0x358ecba293a36247+        .quad   0xaf8f9862b268fd65+        .quad   0x412f7e9968a01c89+        .quad   0x5786f312cd754524+        .quad   0x337788ffca14032c+        .quad   0xf3921028447f1ee3+        .quad   0x8b14071f231bccad+        .quad   0x4c817b4bf2344783++        // 2^172 * 2 * G++        .quad   0x0ff853852871b96e+        .quad   0xe13e9fab60c3f1bb+        .quad   0xeefd595325344402+        .quad   0x0a37c37075b7744b+        .quad   0x413ba057a40b4484+        .quad   0xba4c2e1a4f5f6a43+        .quad   0x614ba0a5aee1d61c+        .quad   0x78a1531a8b05dc53+        .quad   0x6cbdf1703ad0562b+        .quad   0x8ecf4830c92521a3+        .quad   0xdaebd303fd8424e7+        .quad   0x72ad82a42e5ec56f++        // 2^172 * 3 * G++        .quad   0x3f9e8e35bafb65f6+        .quad   0x39d69ec8f27293a1+        .quad   0x6cb8cd958cf6a3d0+        .quad   0x1734778173adae6d+        .quad   0xc368939167024bc3+        .quad   0x8e69d16d49502fda+        .quad   0xfcf2ec3ce45f4b29+        .quad   0x065f669ea3b4cbc4+        .quad   0x8a00aec75532db4d+        .quad   0xb869a4e443e31bb1+        .quad   0x4a0f8552d3a7f515+        .quad   0x19adeb7c303d7c08++        // 2^172 * 4 * G++        .quad   0xc720cb6153ead9a3+        .quad   0x55b2c97f512b636e+        .quad   0xb1e35b5fd40290b1+        .quad   0x2fd9ccf13b530ee2+        .quad   0x9d05ba7d43c31794+        .quad   0x2470c8ff93322526+        .quad   0x8323dec816197438+        .quad   0x2852709881569b53+        .quad   0x07bd475b47f796b8+        .quad   0xd2c7b013542c8f54+        .quad   0x2dbd23f43b24f87e+        .quad   0x6551afd77b0901d6++        // 2^172 * 5 * G++        .quad   0x4546baaf54aac27f+        .quad   0xf6f66fecb2a45a28+        .quad   0x582d1b5b562bcfe8+        .quad   0x44b123f3920f785f+        .quad   0x68a24ce3a1d5c9ac+        .quad   0xbb77a33d10ff6461+        .quad   0x0f86ce4425d3166e+        .quad   0x56507c0950b9623b+        .quad   0x1206f0b7d1713e63+        .quad   0x353fe3d915bafc74+        .quad   0x194ceb970ad9d94d+        .quad   0x62fadd7cf9d03ad3++        // 2^172 * 6 * G++        .quad   0xc6b5967b5598a074+        .quad   0x5efe91ce8e493e25+        .quad   0xd4b72c4549280888+        .quad   0x20ef1149a26740c2+        .quad   0x3cd7bc61e7ce4594+        .quad   0xcd6b35a9b7dd267e+        .quad   0xa080abc84366ef27+        .quad   0x6ec7c46f59c79711+        .quad   0x2f07ad636f09a8a2+        .quad   0x8697e6ce24205e7d+        .quad   0xc0aefc05ee35a139+        .quad   0x15e80958b5f9d897++        // 2^172 * 7 * G++        .quad   0x25a5ef7d0c3e235b+        .quad   0x6c39c17fbe134ee7+        .quad   0xc774e1342dc5c327+        .quad   0x021354b892021f39+        .quad   0x4dd1ed355bb061c4+        .quad   0x42dc0cef941c0700+        .quad   0x61305dc1fd86340e+        .quad   0x56b2cc930e55a443+        .quad   0x1df79da6a6bfc5a2+        .quad   0x02f3a2749fde4369+        .quad   0xb323d9f2cda390a7+        .quad   0x7be0847b8774d363++        // 2^172 * 8 * G++        .quad   0x8c99cc5a8b3f55c3+        .quad   0x0611d7253fded2a0+        .quad   0xed2995ff36b70a36+        .quad   0x1f699a54d78a2619+        .quad   0x1466f5af5307fa11+        .quad   0x817fcc7ded6c0af2+        .quad   0x0a6de44ec3a4a3fb+        .quad   0x74071475bc927d0b+        .quad   0xe77292f373e7ea8a+        .quad   0x296537d2cb045a31+        .quad   0x1bd0653ed3274fde+        .quad   0x2f9a2c4476bd2966++        // 2^176 * 1 * G++        .quad   0xeb18b9ab7f5745c6+        .quad   0x023a8aee5787c690+        .quad   0xb72712da2df7afa9+        .quad   0x36597d25ea5c013d+        .quad   0xa2b4dae0b5511c9a+        .quad   0x7ac860292bffff06+        .quad   0x981f375df5504234+        .quad   0x3f6bd725da4ea12d+        .quad   0x734d8d7b106058ac+        .quad   0xd940579e6fc6905f+        .quad   0x6466f8f99202932d+        .quad   0x7b7ecc19da60d6d0++        // 2^176 * 2 * G++        .quad   0x78c2373c695c690d+        .quad   0xdd252e660642906e+        .quad   0x951d44444ae12bd2+        .quad   0x4235ad7601743956+        .quad   0x6dae4a51a77cfa9b+        .quad   0x82263654e7a38650+        .quad   0x09bbffcd8f2d82db+        .quad   0x03bedc661bf5caba+        .quad   0x6258cb0d078975f5+        .quad   0x492942549189f298+        .quad   0xa0cab423e2e36ee4+        .quad   0x0e7ce2b0cdf066a1++        // 2^176 * 3 * G++        .quad   0xc494643ac48c85a3+        .quad   0xfd361df43c6139ad+        .quad   0x09db17dd3ae94d48+        .quad   0x666e0a5d8fb4674a+        .quad   0xfea6fedfd94b70f9+        .quad   0xf130c051c1fcba2d+        .quad   0x4882d47e7f2fab89+        .quad   0x615256138aeceeb5+        .quad   0x2abbf64e4870cb0d+        .quad   0xcd65bcf0aa458b6b+        .quad   0x9abe4eba75e8985d+        .quad   0x7f0bc810d514dee4++        // 2^176 * 4 * G++        .quad   0xb9006ba426f4136f+        .quad   0x8d67369e57e03035+        .quad   0xcbc8dfd94f463c28+        .quad   0x0d1f8dbcf8eedbf5+        .quad   0x83ac9dad737213a0+        .quad   0x9ff6f8ba2ef72e98+        .quad   0x311e2edd43ec6957+        .quad   0x1d3a907ddec5ab75+        .quad   0xba1693313ed081dc+        .quad   0x29329fad851b3480+        .quad   0x0128013c030321cb+        .quad   0x00011b44a31bfde3++        // 2^176 * 5 * G++        .quad   0x3fdfa06c3fc66c0c+        .quad   0x5d40e38e4dd60dd2+        .quad   0x7ae38b38268e4d71+        .quad   0x3ac48d916e8357e1+        .quad   0x16561f696a0aa75c+        .quad   0xc1bf725c5852bd6a+        .quad   0x11a8dd7f9a7966ad+        .quad   0x63d988a2d2851026+        .quad   0x00120753afbd232e+        .quad   0xe92bceb8fdd8f683+        .quad   0xf81669b384e72b91+        .quad   0x33fad52b2368a066++        // 2^176 * 6 * G++        .quad   0x540649c6c5e41e16+        .quad   0x0af86430333f7735+        .quad   0xb2acfcd2f305e746+        .quad   0x16c0f429a256dca7+        .quad   0x8d2cc8d0c422cfe8+        .quad   0x072b4f7b05a13acb+        .quad   0xa3feb6e6ecf6a56f+        .quad   0x3cc355ccb90a71e2+        .quad   0xe9b69443903e9131+        .quad   0xb8a494cb7a5637ce+        .quad   0xc87cd1a4baba9244+        .quad   0x631eaf426bae7568++        // 2^176 * 7 * G++        .quad   0xb3e90410da66fe9f+        .quad   0x85dd4b526c16e5a6+        .quad   0xbc3d97611ef9bf83+        .quad   0x5599648b1ea919b5+        .quad   0x47d975b9a3700de8+        .quad   0x7280c5fbe2f80552+        .quad   0x53658f2732e45de1+        .quad   0x431f2c7f665f80b5+        .quad   0xd6026344858f7b19+        .quad   0x14ab352fa1ea514a+        .quad   0x8900441a2090a9d7+        .quad   0x7b04715f91253b26++        // 2^176 * 8 * G++        .quad   0x83edbd28acf6ae43+        .quad   0x86357c8b7d5c7ab4+        .quad   0xc0404769b7eb2c44+        .quad   0x59b37bf5c2f6583f+        .quad   0xb376c280c4e6bac6+        .quad   0x970ed3dd6d1d9b0b+        .quad   0xb09a9558450bf944+        .quad   0x48d0acfa57cde223+        .quad   0xb60f26e47dabe671+        .quad   0xf1d1a197622f3a37+        .quad   0x4208ce7ee9960394+        .quad   0x16234191336d3bdb++        // 2^180 * 1 * G++        .quad   0xf19aeac733a63aef+        .quad   0x2c7fba5d4442454e+        .quad   0x5da87aa04795e441+        .quad   0x413051e1a4e0b0f5+        .quad   0x852dd1fd3d578bbe+        .quad   0x2b65ce72c3286108+        .quad   0x658c07f4eace2273+        .quad   0x0933f804ec38ab40+        .quad   0xa7ab69798d496476+        .quad   0x8121aadefcb5abc8+        .quad   0xa5dc12ef7b539472+        .quad   0x07fd47065e45351a++        // 2^180 * 2 * G++        .quad   0xc8583c3d258d2bcd+        .quad   0x17029a4daf60b73f+        .quad   0xfa0fc9d6416a3781+        .quad   0x1c1e5fba38b3fb23+        .quad   0x304211559ae8e7c3+        .quad   0xf281b229944882a5+        .quad   0x8a13ac2e378250e4+        .quad   0x014afa0954ba48f4+        .quad   0xcb3197001bb3666c+        .quad   0x330060524bffecb9+        .quad   0x293711991a88233c+        .quad   0x291884363d4ed364++        // 2^180 * 3 * G++        .quad   0x033c6805dc4babfa+        .quad   0x2c15bf5e5596ecc1+        .quad   0x1bc70624b59b1d3b+        .quad   0x3ede9850a19f0ec5+        .quad   0xfb9d37c3bc1ab6eb+        .quad   0x02be14534d57a240+        .quad   0xf4d73415f8a5e1f6+        .quad   0x5964f4300ccc8188+        .quad   0xe44a23152d096800+        .quad   0x5c08c55970866996+        .quad   0xdf2db60a46affb6e+        .quad   0x579155c1f856fd89++        // 2^180 * 4 * G++        .quad   0x96324edd12e0c9ef+        .quad   0x468b878df2420297+        .quad   0x199a3776a4f573be+        .quad   0x1e7fbcf18e91e92a+        .quad   0xb5f16b630817e7a6+        .quad   0x808c69233c351026+        .quad   0x324a983b54cef201+        .quad   0x53c092084a485345+        .quad   0xd2d41481f1cbafbf+        .quad   0x231d2db6716174e5+        .quad   0x0b7d7656e2a55c98+        .quad   0x3e955cd82aa495f6++        // 2^180 * 5 * G++        .quad   0xe48f535e3ed15433+        .quad   0xd075692a0d7270a3+        .quad   0x40fbd21daade6387+        .quad   0x14264887cf4495f5+        .quad   0xab39f3ef61bb3a3f+        .quad   0x8eb400652eb9193e+        .quad   0xb5de6ecc38c11f74+        .quad   0x654d7e9626f3c49f+        .quad   0xe564cfdd5c7d2ceb+        .quad   0x82eeafded737ccb9+        .quad   0x6107db62d1f9b0ab+        .quad   0x0b6baac3b4358dbb++        // 2^180 * 6 * G++        .quad   0x7ae62bcb8622fe98+        .quad   0x47762256ceb891af+        .quad   0x1a5a92bcf2e406b4+        .quad   0x7d29401784e41501+        .quad   0x204abad63700a93b+        .quad   0xbe0023d3da779373+        .quad   0xd85f0346633ab709+        .quad   0x00496dc490820412+        .quad   0x1c74b88dc27e6360+        .quad   0x074854268d14850c+        .quad   0xa145fb7b3e0dcb30+        .quad   0x10843f1b43803b23++        // 2^180 * 7 * G++        .quad   0xc5f90455376276dd+        .quad   0xce59158dd7645cd9+        .quad   0x92f65d511d366b39+        .quad   0x11574b6e526996c4+        .quad   0xd56f672de324689b+        .quad   0xd1da8aedb394a981+        .quad   0xdd7b58fe9168cfed+        .quad   0x7ce246cd4d56c1e8+        .quad   0xb8f4308e7f80be53+        .quad   0x5f3cb8cb34a9d397+        .quad   0x18a961bd33cc2b2c+        .quad   0x710045fb3a9af671++        // 2^180 * 8 * G++        .quad   0x73f93d36101b95eb+        .quad   0xfaef33794f6f4486+        .quad   0x5651735f8f15e562+        .quad   0x7fa3f19058b40da1+        .quad   0xa03fc862059d699e+        .quad   0x2370cfa19a619e69+        .quad   0xc4fe3b122f823deb+        .quad   0x1d1b056fa7f0844e+        .quad   0x1bc64631e56bf61f+        .quad   0xd379ab106e5382a3+        .quad   0x4d58c57e0540168d+        .quad   0x566256628442d8e4++        // 2^184 * 1 * G++        .quad   0xb9e499def6267ff6+        .quad   0x7772ca7b742c0843+        .quad   0x23a0153fe9a4f2b1+        .quad   0x2cdfdfecd5d05006+        .quad   0xdd499cd61ff38640+        .quad   0x29cd9bc3063625a0+        .quad   0x51e2d8023dd73dc3+        .quad   0x4a25707a203b9231+        .quad   0x2ab7668a53f6ed6a+        .quad   0x304242581dd170a1+        .quad   0x4000144c3ae20161+        .quad   0x5721896d248e49fc++        // 2^184 * 2 * G++        .quad   0x0b6e5517fd181bae+        .quad   0x9022629f2bb963b4+        .quad   0x5509bce932064625+        .quad   0x578edd74f63c13da+        .quad   0x285d5091a1d0da4e+        .quad   0x4baa6fa7b5fe3e08+        .quad   0x63e5177ce19393b3+        .quad   0x03c935afc4b030fd+        .quad   0x997276c6492b0c3d+        .quad   0x47ccc2c4dfe205fc+        .quad   0xdcd29b84dd623a3c+        .quad   0x3ec2ab590288c7a2++        // 2^184 * 3 * G++        .quad   0xa1a0d27be4d87bb9+        .quad   0xa98b4deb61391aed+        .quad   0x99a0ddd073cb9b83+        .quad   0x2dd5c25a200fcace+        .quad   0xa7213a09ae32d1cb+        .quad   0x0f2b87df40f5c2d5+        .quad   0x0baea4c6e81eab29+        .quad   0x0e1bf66c6adbac5e+        .quad   0xe2abd5e9792c887e+        .quad   0x1a020018cb926d5d+        .quad   0xbfba69cdbaae5f1e+        .quad   0x730548b35ae88f5f++        // 2^184 * 4 * G++        .quad   0xc43551a3cba8b8ee+        .quad   0x65a26f1db2115f16+        .quad   0x760f4f52ab8c3850+        .quad   0x3043443b411db8ca+        .quad   0x805b094ba1d6e334+        .quad   0xbf3ef17709353f19+        .quad   0x423f06cb0622702b+        .quad   0x585a2277d87845dd+        .quad   0xa18a5f8233d48962+        .quad   0x6698c4b5ec78257f+        .quad   0xa78e6fa5373e41ff+        .quad   0x7656278950ef981f++        // 2^184 * 5 * G++        .quad   0x38c3cf59d51fc8c0+        .quad   0x9bedd2fd0506b6f2+        .quad   0x26bf109fab570e8f+        .quad   0x3f4160a8c1b846a6+        .quad   0xe17073a3ea86cf9d+        .quad   0x3a8cfbb707155fdc+        .quad   0x4853e7fc31838a8e+        .quad   0x28bbf484b613f616+        .quad   0xf2612f5c6f136c7c+        .quad   0xafead107f6dd11be+        .quad   0x527e9ad213de6f33+        .quad   0x1e79cb358188f75d++        // 2^184 * 6 * G++        .quad   0x013436c3eef7e3f1+        .quad   0x828b6a7ffe9e10f8+        .quad   0x7ff908e5bcf9defc+        .quad   0x65d7951b3a3b3831+        .quad   0x77e953d8f5e08181+        .quad   0x84a50c44299dded9+        .quad   0xdc6c2d0c864525e5+        .quad   0x478ab52d39d1f2f4+        .quad   0x66a6a4d39252d159+        .quad   0xe5dde1bc871ac807+        .quad   0xb82c6b40a6c1c96f+        .quad   0x16d87a411a212214++        // 2^184 * 7 * G++        .quad   0xb3bd7e5a42066215+        .quad   0x879be3cd0c5a24c1+        .quad   0x57c05db1d6f994b7+        .quad   0x28f87c8165f38ca6+        .quad   0xfba4d5e2d54e0583+        .quad   0xe21fafd72ebd99fa+        .quad   0x497ac2736ee9778f+        .quad   0x1f990b577a5a6dde+        .quad   0xa3344ead1be8f7d6+        .quad   0x7d1e50ebacea798f+        .quad   0x77c6569e520de052+        .quad   0x45882fe1534d6d3e++        // 2^184 * 8 * G++        .quad   0x6669345d757983d6+        .quad   0x62b6ed1117aa11a6+        .quad   0x7ddd1857985e128f+        .quad   0x688fe5b8f626f6dd+        .quad   0xd8ac9929943c6fe4+        .quad   0xb5f9f161a38392a2+        .quad   0x2699db13bec89af3+        .quad   0x7dcf843ce405f074+        .quad   0x6c90d6484a4732c0+        .quad   0xd52143fdca563299+        .quad   0xb3be28c3915dc6e1+        .quad   0x6739687e7327191b++        // 2^188 * 1 * G++        .quad   0x9f65c5ea200814cf+        .quad   0x840536e169a31740+        .quad   0x8b0ed13925c8b4ad+        .quad   0x0080dbafe936361d+        .quad   0x8ce5aad0c9cb971f+        .quad   0x1156aaa99fd54a29+        .quad   0x41f7247015af9b78+        .quad   0x1fe8cca8420f49aa+        .quad   0x72a1848f3c0cc82a+        .quad   0x38c560c2877c9e54+        .quad   0x5004e228ce554140+        .quad   0x042418a103429d71++        // 2^188 * 2 * G++        .quad   0x899dea51abf3ff5f+        .quad   0x9b93a8672fc2d8ba+        .quad   0x2c38cb97be6ebd5c+        .quad   0x114d578497263b5d+        .quad   0x58e84c6f20816247+        .quad   0x8db2b2b6e36fd793+        .quad   0x977182561d484d85+        .quad   0x0822024f8632abd7+        .quad   0xb301bb7c6b1beca3+        .quad   0x55393f6dc6eb1375+        .quad   0x910d281097b6e4eb+        .quad   0x1ad4548d9d479ea3++        // 2^188 * 3 * G++        .quad   0xcd5a7da0389a48fd+        .quad   0xb38fa4aa9a78371e+        .quad   0xc6d9761b2cdb8e6c+        .quad   0x35cf51dbc97e1443+        .quad   0xa06fe66d0fe9fed3+        .quad   0xa8733a401c587909+        .quad   0x30d14d800df98953+        .quad   0x41ce5876c7b30258+        .quad   0x59ac3bc5d670c022+        .quad   0xeae67c109b119406+        .quad   0x9798bdf0b3782fda+        .quad   0x651e3201fd074092++        // 2^188 * 4 * G++        .quad   0xd63d8483ef30c5cf+        .quad   0x4cd4b4962361cc0c+        .quad   0xee90e500a48426ac+        .quad   0x0af51d7d18c14eeb+        .quad   0xa57ba4a01efcae9e+        .quad   0x769f4beedc308a94+        .quad   0xd1f10eeb3603cb2e+        .quad   0x4099ce5e7e441278+        .quad   0x1ac98e4f8a5121e9+        .quad   0x7dae9544dbfa2fe0+        .quad   0x8320aa0dd6430df9+        .quad   0x667282652c4a2fb5++        // 2^188 * 5 * G++        .quad   0x874621f4d86bc9ab+        .quad   0xb54c7bbe56fe6fea+        .quad   0x077a24257fadc22c+        .quad   0x1ab53be419b90d39+        .quad   0xada8b6e02946db23+        .quad   0x1c0ce51a7b253ab7+        .quad   0x8448c85a66dd485b+        .quad   0x7f1fc025d0675adf+        .quad   0xd8ee1b18319ea6aa+        .quad   0x004d88083a21f0da+        .quad   0x3bd6aa1d883a4f4b+        .quad   0x4db9a3a6dfd9fd14++        // 2^188 * 6 * G++        .quad   0x8ce7b23bb99c0755+        .quad   0x35c5d6edc4f50f7a+        .quad   0x7e1e2ed2ed9b50c3+        .quad   0x36305f16e8934da1+        .quad   0xd95b00bbcbb77c68+        .quad   0xddbc846a91f17849+        .quad   0x7cf700aebe28d9b3+        .quad   0x5ce1285c85d31f3e+        .quad   0x31b6972d98b0bde8+        .quad   0x7d920706aca6de5b+        .quad   0xe67310f8908a659f+        .quad   0x50fac2a6efdf0235++        // 2^188 * 7 * G++        .quad   0xf3d3a9f35b880f5a+        .quad   0xedec050cdb03e7c2+        .quad   0xa896981ff9f0b1a2+        .quad   0x49a4ae2bac5e34a4+        .quad   0x295b1c86f6f449bc+        .quad   0x51b2e84a1f0ab4dd+        .quad   0xc001cb30aa8e551d+        .quad   0x6a28d35944f43662+        .quad   0x28bb12ee04a740e0+        .quad   0x14313bbd9bce8174+        .quad   0x72f5b5e4e8c10c40+        .quad   0x7cbfb19936adcd5b++        // 2^188 * 8 * G++        .quad   0xa311ddc26b89792d+        .quad   0x1b30b4c6da512664+        .quad   0x0ca77b4ccf150859+        .quad   0x1de443df1b009408+        .quad   0x8e793a7acc36e6e0+        .quad   0xf9fab7a37d586eed+        .quad   0x3a4f9692bae1f4e4+        .quad   0x1c14b03eff5f447e+        .quad   0x19647bd114a85291+        .quad   0x57b76cb21034d3af+        .quad   0x6329db440f9d6dfa+        .quad   0x5ef43e586a571493++        // 2^192 * 1 * G++        .quad   0xef782014385675a6+        .quad   0xa2649f30aafda9e8+        .quad   0x4cd1eb505cdfa8cb+        .quad   0x46115aba1d4dc0b3+        .quad   0xa66dcc9dc80c1ac0+        .quad   0x97a05cf41b38a436+        .quad   0xa7ebf3be95dbd7c6+        .quad   0x7da0b8f68d7e7dab+        .quad   0xd40f1953c3b5da76+        .quad   0x1dac6f7321119e9b+        .quad   0x03cc6021feb25960+        .quad   0x5a5f887e83674b4b++        // 2^192 * 2 * G++        .quad   0x8f6301cf70a13d11+        .quad   0xcfceb815350dd0c4+        .quad   0xf70297d4a4bca47e+        .quad   0x3669b656e44d1434+        .quad   0x9e9628d3a0a643b9+        .quad   0xb5c3cb00e6c32064+        .quad   0x9b5302897c2dec32+        .quad   0x43e37ae2d5d1c70c+        .quad   0x387e3f06eda6e133+        .quad   0x67301d5199a13ac0+        .quad   0xbd5ad8f836263811+        .quad   0x6a21e6cd4fd5e9be++        // 2^192 * 3 * G++        .quad   0xf1c6170a3046e65f+        .quad   0x58712a2a00d23524+        .quad   0x69dbbd3c8c82b755+        .quad   0x586bf9f1a195ff57+        .quad   0xef4129126699b2e3+        .quad   0x71d30847708d1301+        .quad   0x325432d01182b0bd+        .quad   0x45371b07001e8b36+        .quad   0xa6db088d5ef8790b+        .quad   0x5278f0dc610937e5+        .quad   0xac0349d261a16eb8+        .quad   0x0eafb03790e52179++        // 2^192 * 4 * G++        .quad   0x960555c13748042f+        .quad   0x219a41e6820baa11+        .quad   0x1c81f73873486d0c+        .quad   0x309acc675a02c661+        .quad   0x5140805e0f75ae1d+        .quad   0xec02fbe32662cc30+        .quad   0x2cebdf1eea92396d+        .quad   0x44ae3344c5435bb3+        .quad   0x9cf289b9bba543ee+        .quad   0xf3760e9d5ac97142+        .quad   0x1d82e5c64f9360aa+        .quad   0x62d5221b7f94678f++        // 2^192 * 5 * G++        .quad   0x524c299c18d0936d+        .quad   0xc86bb56c8a0c1a0c+        .quad   0xa375052edb4a8631+        .quad   0x5c0efde4bc754562+        .quad   0x7585d4263af77a3c+        .quad   0xdfae7b11fee9144d+        .quad   0xa506708059f7193d+        .quad   0x14f29a5383922037+        .quad   0xdf717edc25b2d7f5+        .quad   0x21f970db99b53040+        .quad   0xda9234b7c3ed4c62+        .quad   0x5e72365c7bee093e++        // 2^192 * 6 * G++        .quad   0x575bfc074571217f+        .quad   0x3779675d0694d95b+        .quad   0x9a0a37bbf4191e33+        .quad   0x77f1104c47b4eabc+        .quad   0x7d9339062f08b33e+        .quad   0x5b9659e5df9f32be+        .quad   0xacff3dad1f9ebdfd+        .quad   0x70b20555cb7349b7+        .quad   0xbe5113c555112c4c+        .quad   0x6688423a9a881fcd+        .quad   0x446677855e503b47+        .quad   0x0e34398f4a06404a++        // 2^192 * 7 * G++        .quad   0xb67d22d93ecebde8+        .quad   0x09b3e84127822f07+        .quad   0x743fa61fb05b6d8d+        .quad   0x5e5405368a362372+        .quad   0x18930b093e4b1928+        .quad   0x7de3e10e73f3f640+        .quad   0xf43217da73395d6f+        .quad   0x6f8aded6ca379c3e+        .quad   0xe340123dfdb7b29a+        .quad   0x487b97e1a21ab291+        .quad   0xf9967d02fde6949e+        .quad   0x780de72ec8d3de97++        // 2^192 * 8 * G++        .quad   0x0ae28545089ae7bc+        .quad   0x388ddecf1c7f4d06+        .quad   0x38ac15510a4811b8+        .quad   0x0eb28bf671928ce4+        .quad   0x671feaf300f42772+        .quad   0x8f72eb2a2a8c41aa+        .quad   0x29a17fd797373292+        .quad   0x1defc6ad32b587a6+        .quad   0xaf5bbe1aef5195a7+        .quad   0x148c1277917b15ed+        .quad   0x2991f7fb7ae5da2e+        .quad   0x467d201bf8dd2867++        // 2^196 * 1 * G++        .quad   0x7906ee72f7bd2e6b+        .quad   0x05d270d6109abf4e+        .quad   0x8d5cfe45b941a8a4+        .quad   0x44c218671c974287+        .quad   0x745f9d56296bc318+        .quad   0x993580d4d8152e65+        .quad   0xb0e5b13f5839e9ce+        .quad   0x51fc2b28d43921c0+        .quad   0x1b8fd11795e2a98c+        .quad   0x1c4e5ee12b6b6291+        .quad   0x5b30e7107424b572+        .quad   0x6e6b9de84c4f4ac6++        // 2^196 * 2 * G++        .quad   0xdff25fce4b1de151+        .quad   0xd841c0c7e11c4025+        .quad   0x2554b3c854749c87+        .quad   0x2d292459908e0df9+        .quad   0x6b7c5f10f80cb088+        .quad   0x736b54dc56e42151+        .quad   0xc2b620a5c6ef99c4+        .quad   0x5f4c802cc3a06f42+        .quad   0x9b65c8f17d0752da+        .quad   0x881ce338c77ee800+        .quad   0xc3b514f05b62f9e3+        .quad   0x66ed5dd5bec10d48++        // 2^196 * 3 * G++        .quad   0x7d38a1c20bb2089d+        .quad   0x808334e196ccd412+        .quad   0xc4a70b8c6c97d313+        .quad   0x2eacf8bc03007f20+        .quad   0xf0adf3c9cbca047d+        .quad   0x81c3b2cbf4552f6b+        .quad   0xcfda112d44735f93+        .quad   0x1f23a0c77e20048c+        .quad   0xf235467be5bc1570+        .quad   0x03d2d9020dbab38c+        .quad   0x27529aa2fcf9e09e+        .quad   0x0840bef29d34bc50++        // 2^196 * 4 * G++        .quad   0x796dfb35dc10b287+        .quad   0x27176bcd5c7ff29d+        .quad   0x7f3d43e8c7b24905+        .quad   0x0304f5a191c54276+        .quad   0xcd54e06b7f37e4eb+        .quad   0x8cc15f87f5e96cca+        .quad   0xb8248bb0d3597dce+        .quad   0x246affa06074400c+        .quad   0x37d88e68fbe45321+        .quad   0x86097548c0d75032+        .quad   0x4e9b13ef894a0d35+        .quad   0x25a83cac5753d325++        // 2^196 * 5 * G++        .quad   0x10222f48eed8165e+        .quad   0x623fc1234b8bcf3a+        .quad   0x1e145c09c221e8f0+        .quad   0x7ccfa59fca782630+        .quad   0x9f0f66293952b6e2+        .quad   0x33db5e0e0934267b+        .quad   0xff45252bd609fedc+        .quad   0x06be10f5c506e0c9+        .quad   0x1a9615a9b62a345f+        .quad   0x22050c564a52fecc+        .quad   0xa7a2788528bc0dfe+        .quad   0x5e82770a1a1ee71d++        // 2^196 * 6 * G++        .quad   0x35425183ad896a5c+        .quad   0xe8673afbe78d52f6+        .quad   0x2c66f25f92a35f64+        .quad   0x09d04f3b3b86b102+        .quad   0xe802e80a42339c74+        .quad   0x34175166a7fffae5+        .quad   0x34865d1f1c408cae+        .quad   0x2cca982c605bc5ee+        .quad   0xfd2d5d35197dbe6e+        .quad   0x207c2eea8be4ffa3+        .quad   0x2613d8db325ae918+        .quad   0x7a325d1727741d3e++        // 2^196 * 7 * G++        .quad   0xd036b9bbd16dfde2+        .quad   0xa2055757c497a829+        .quad   0x8e6cc966a7f12667+        .quad   0x4d3b1a791239c180+        .quad   0xecd27d017e2a076a+        .quad   0xd788689f1636495e+        .quad   0x52a61af0919233e5+        .quad   0x2a479df17bb1ae64+        .quad   0x9e5eee8e33db2710+        .quad   0x189854ded6c43ca5+        .quad   0xa41c22c592718138+        .quad   0x27ad5538a43a5e9b++        // 2^196 * 8 * G++        .quad   0x2746dd4b15350d61+        .quad   0xd03fcbc8ee9521b7+        .quad   0xe86e365a138672ca+        .quad   0x510e987f7e7d89e2+        .quad   0xcb5a7d638e47077c+        .quad   0x8db7536120a1c059+        .quad   0x549e1e4d8bedfdcc+        .quad   0x080153b7503b179d+        .quad   0xdda69d930a3ed3e3+        .quad   0x3d386ef1cd60a722+        .quad   0xc817ad58bdaa4ee6+        .quad   0x23be8d554fe7372a++        // 2^200 * 1 * G++        .quad   0x95fe919a74ef4fad+        .quad   0x3a827becf6a308a2+        .quad   0x964e01d309a47b01+        .quad   0x71c43c4f5ba3c797+        .quad   0xbc1ef4bd567ae7a9+        .quad   0x3f624cb2d64498bd+        .quad   0xe41064d22c1f4ec8+        .quad   0x2ef9c5a5ba384001+        .quad   0xb6fd6df6fa9e74cd+        .quad   0xf18278bce4af267a+        .quad   0x8255b3d0f1ef990e+        .quad   0x5a758ca390c5f293++        // 2^200 * 2 * G++        .quad   0xa2b72710d9462495+        .quad   0x3aa8c6d2d57d5003+        .quad   0xe3d400bfa0b487ca+        .quad   0x2dbae244b3eb72ec+        .quad   0x8ce0918b1d61dc94+        .quad   0x8ded36469a813066+        .quad   0xd4e6a829afe8aad3+        .quad   0x0a738027f639d43f+        .quad   0x980f4a2f57ffe1cc+        .quad   0x00670d0de1839843+        .quad   0x105c3f4a49fb15fd+        .quad   0x2698ca635126a69c++        // 2^200 * 3 * G++        .quad   0xe765318832b0ba78+        .quad   0x381831f7925cff8b+        .quad   0x08a81b91a0291fcc+        .quad   0x1fb43dcc49caeb07+        .quad   0x2e3d702f5e3dd90e+        .quad   0x9e3f0918e4d25386+        .quad   0x5e773ef6024da96a+        .quad   0x3c004b0c4afa3332+        .quad   0x9aa946ac06f4b82b+        .quad   0x1ca284a5a806c4f3+        .quad   0x3ed3265fc6cd4787+        .quad   0x6b43fd01cd1fd217++        // 2^200 * 4 * G++        .quad   0xc7a75d4b4697c544+        .quad   0x15fdf848df0fffbf+        .quad   0x2868b9ebaa46785a+        .quad   0x5a68d7105b52f714+        .quad   0xb5c742583e760ef3+        .quad   0x75dc52b9ee0ab990+        .quad   0xbf1427c2072b923f+        .quad   0x73420b2d6ff0d9f0+        .quad   0xaf2cf6cb9e851e06+        .quad   0x8f593913c62238c4+        .quad   0xda8ab89699fbf373+        .quad   0x3db5632fea34bc9e++        // 2^200 * 5 * G++        .quad   0xf46eee2bf75dd9d8+        .quad   0x0d17b1f6396759a5+        .quad   0x1bf2d131499e7273+        .quad   0x04321adf49d75f13+        .quad   0x2e4990b1829825d5+        .quad   0xedeaeb873e9a8991+        .quad   0xeef03d394c704af8+        .quad   0x59197ea495df2b0e+        .quad   0x04e16019e4e55aae+        .quad   0xe77b437a7e2f92e9+        .quad   0xc7ce2dc16f159aa4+        .quad   0x45eafdc1f4d70cc0++        // 2^200 * 6 * G++        .quad   0x698401858045d72b+        .quad   0x4c22faa2cf2f0651+        .quad   0x941a36656b222dc6+        .quad   0x5a5eebc80362dade+        .quad   0xb60e4624cfccb1ed+        .quad   0x59dbc292bd5c0395+        .quad   0x31a09d1ddc0481c9+        .quad   0x3f73ceea5d56d940+        .quad   0xb7a7bfd10a4e8dc6+        .quad   0xbe57007e44c9b339+        .quad   0x60c1207f1557aefa+        .quad   0x26058891266218db++        // 2^200 * 7 * G++        .quad   0x59f704a68360ff04+        .quad   0xc3d93fde7661e6f4+        .quad   0x831b2a7312873551+        .quad   0x54ad0c2e4e615d57+        .quad   0x4c818e3cc676e542+        .quad   0x5e422c9303ceccad+        .quad   0xec07cccab4129f08+        .quad   0x0dedfa10b24443b8+        .quad   0xee3b67d5b82b522a+        .quad   0x36f163469fa5c1eb+        .quad   0xa5b4d2f26ec19fd3+        .quad   0x62ecb2baa77a9408++        // 2^200 * 8 * G++        .quad   0xe5ed795261152b3d+        .quad   0x4962357d0eddd7d1+        .quad   0x7482c8d0b96b4c71+        .quad   0x2e59f919a966d8be+        .quad   0x92072836afb62874+        .quad   0x5fcd5e8579e104a5+        .quad   0x5aad01adc630a14a+        .quad   0x61913d5075663f98+        .quad   0x0dc62d361a3231da+        .quad   0xfa47583294200270+        .quad   0x02d801513f9594ce+        .quad   0x3ddbc2a131c05d5c++        // 2^204 * 1 * G++        .quad   0x3f50a50a4ffb81ef+        .quad   0xb1e035093bf420bf+        .quad   0x9baa8e1cc6aa2cd0+        .quad   0x32239861fa237a40+        .quad   0xfb735ac2004a35d1+        .quad   0x31de0f433a6607c3+        .quad   0x7b8591bfc528d599+        .quad   0x55be9a25f5bb050c+        .quad   0x0d005acd33db3dbf+        .quad   0x0111b37c80ac35e2+        .quad   0x4892d66c6f88ebeb+        .quad   0x770eadb16508fbcd++        // 2^204 * 2 * G++        .quad   0x8451f9e05e4e89dd+        .quad   0xc06302ffbc793937+        .quad   0x5d22749556a6495c+        .quad   0x09a6755ca05603fb+        .quad   0xf1d3b681a05071b9+        .quad   0x2207659a3592ff3a+        .quad   0x5f0169297881e40e+        .quad   0x16bedd0e86ba374e+        .quad   0x5ecccc4f2c2737b5+        .quad   0x43b79e0c2dccb703+        .quad   0x33e008bc4ec43df3+        .quad   0x06c1b840f07566c0++        // 2^204 * 3 * G++        .quad   0x7688a5c6a388f877+        .quad   0x02a96c14deb2b6ac+        .quad   0x64c9f3431b8c2af8+        .quad   0x3628435554a1eed6+        .quad   0x69ee9e7f9b02805c+        .quad   0xcbff828a547d1640+        .quad   0x3d93a869b2430968+        .quad   0x46b7b8cd3fe26972+        .quad   0xe9812086fe7eebe0+        .quad   0x4cba6be72f515437+        .quad   0x1d04168b516efae9+        .quad   0x5ea1391043982cb9++        // 2^204 * 4 * G++        .quad   0x49125c9cf4702ee1+        .quad   0x4520b71f8b25b32d+        .quad   0x33193026501fef7e+        .quad   0x656d8997c8d2eb2b+        .quad   0x6f2b3be4d5d3b002+        .quad   0xafec33d96a09c880+        .quad   0x035f73a4a8bcc4cc+        .quad   0x22c5b9284662198b+        .quad   0xcb58c8fe433d8939+        .quad   0x89a0cb2e6a8d7e50+        .quad   0x79ca955309fbbe5a+        .quad   0x0c626616cd7fc106++        // 2^204 * 5 * G++        .quad   0x1ffeb80a4879b61f+        .quad   0x6396726e4ada21ed+        .quad   0x33c7b093368025ba+        .quad   0x471aa0c6f3c31788+        .quad   0x8fdfc379fbf454b1+        .quad   0x45a5a970f1a4b771+        .quad   0xac921ef7bad35915+        .quad   0x42d088dca81c2192+        .quad   0x8fda0f37a0165199+        .quad   0x0adadb77c8a0e343+        .quad   0x20fbfdfcc875e820+        .quad   0x1cf2bea80c2206e7++        // 2^204 * 6 * G++        .quad   0xc2ddf1deb36202ac+        .quad   0x92a5fe09d2e27aa5+        .quad   0x7d1648f6fc09f1d3+        .quad   0x74c2cc0513bc4959+        .quad   0x982d6e1a02c0412f+        .quad   0x90fa4c83db58e8fe+        .quad   0x01c2f5bcdcb18bc0+        .quad   0x686e0c90216abc66+        .quad   0x1fadbadba54395a7+        .quad   0xb41a02a0ae0da66a+        .quad   0xbf19f598bba37c07+        .quad   0x6a12b8acde48430d++        // 2^204 * 7 * G++        .quad   0xf8daea1f39d495d9+        .quad   0x592c190e525f1dfc+        .quad   0xdb8cbd04c9991d1b+        .quad   0x11f7fda3d88f0cb7+        .quad   0x793bdd801aaeeb5f+        .quad   0x00a2a0aac1518871+        .quad   0xe8a373a31f2136b4+        .quad   0x48aab888fc91ef19+        .quad   0x041f7e925830f40e+        .quad   0x002d6ca979661c06+        .quad   0x86dc9ff92b046a2e+        .quad   0x760360928b0493d1++        // 2^204 * 8 * G++        .quad   0x21bb41c6120cf9c6+        .quad   0xeab2aa12decda59b+        .quad   0xc1a72d020aa48b34+        .quad   0x215d4d27e87d3b68+        .quad   0xb43108e5695a0b05+        .quad   0x6cb00ee8ad37a38b+        .quad   0x5edad6eea3537381+        .quad   0x3f2602d4b6dc3224+        .quad   0xc8b247b65bcaf19c+        .quad   0x49779dc3b1b2c652+        .quad   0x89a180bbd5ece2e2+        .quad   0x13f098a3cec8e039++        // 2^208 * 1 * G++        .quad   0x9adc0ff9ce5ec54b+        .quad   0x039c2a6b8c2f130d+        .quad   0x028007c7f0f89515+        .quad   0x78968314ac04b36b+        .quad   0xf3aa57a22796bb14+        .quad   0x883abab79b07da21+        .quad   0xe54be21831a0391c+        .quad   0x5ee7fb38d83205f9+        .quad   0x538dfdcb41446a8e+        .quad   0xa5acfda9434937f9+        .quad   0x46af908d263c8c78+        .quad   0x61d0633c9bca0d09++        // 2^208 * 2 * G++        .quad   0x63744935ffdb2566+        .quad   0xc5bd6b89780b68bb+        .quad   0x6f1b3280553eec03+        .quad   0x6e965fd847aed7f5+        .quad   0xada328bcf8fc73df+        .quad   0xee84695da6f037fc+        .quad   0x637fb4db38c2a909+        .quad   0x5b23ac2df8067bdc+        .quad   0x9ad2b953ee80527b+        .quad   0xe88f19aafade6d8d+        .quad   0x0e711704150e82cf+        .quad   0x79b9bbb9dd95dedc++        // 2^208 * 3 * G++        .quad   0xebb355406a3126c2+        .quad   0xd26383a868c8c393+        .quad   0x6c0c6429e5b97a82+        .quad   0x5065f158c9fd2147+        .quad   0xd1997dae8e9f7374+        .quad   0xa032a2f8cfbb0816+        .quad   0xcd6cba126d445f0a+        .quad   0x1ba811460accb834+        .quad   0x708169fb0c429954+        .quad   0xe14600acd76ecf67+        .quad   0x2eaab98a70e645ba+        .quad   0x3981f39e58a4faf2++        // 2^208 * 4 * G++        .quad   0x18fb8a7559230a93+        .quad   0x1d168f6960e6f45d+        .quad   0x3a85a94514a93cb5+        .quad   0x38dc083705acd0fd+        .quad   0xc845dfa56de66fde+        .quad   0xe152a5002c40483a+        .quad   0xe9d2e163c7b4f632+        .quad   0x30f4452edcbc1b65+        .quad   0x856d2782c5759740+        .quad   0xfa134569f99cbecc+        .quad   0x8844fc73c0ea4e71+        .quad   0x632d9a1a593f2469++        // 2^208 * 5 * G++        .quad   0xf6bb6b15b807cba6+        .quad   0x1823c7dfbc54f0d7+        .quad   0xbb1d97036e29670b+        .quad   0x0b24f48847ed4a57+        .quad   0xbf09fd11ed0c84a7+        .quad   0x63f071810d9f693a+        .quad   0x21908c2d57cf8779+        .quad   0x3a5a7df28af64ba2+        .quad   0xdcdad4be511beac7+        .quad   0xa4538075ed26ccf2+        .quad   0xe19cff9f005f9a65+        .quad   0x34fcf74475481f63++        // 2^208 * 6 * G++        .quad   0xc197e04c789767ca+        .quad   0xb8714dcb38d9467d+        .quad   0x55de888283f95fa8+        .quad   0x3d3bdc164dfa63f7+        .quad   0xa5bb1dab78cfaa98+        .quad   0x5ceda267190b72f2+        .quad   0x9309c9110a92608e+        .quad   0x0119a3042fb374b0+        .quad   0x67a2d89ce8c2177d+        .quad   0x669da5f66895d0c1+        .quad   0xf56598e5b282a2b0+        .quad   0x56c088f1ede20a73++        // 2^208 * 7 * G++        .quad   0x336d3d1110a86e17+        .quad   0xd7f388320b75b2fa+        .quad   0xf915337625072988+        .quad   0x09674c6b99108b87+        .quad   0x581b5fac24f38f02+        .quad   0xa90be9febae30cbd+        .quad   0x9a2169028acf92f0+        .quad   0x038b7ea48359038f+        .quad   0x9f4ef82199316ff8+        .quad   0x2f49d282eaa78d4f+        .quad   0x0971a5ab5aef3174+        .quad   0x6e5e31025969eb65++        // 2^208 * 8 * G++        .quad   0xb16c62f587e593fb+        .quad   0x4999eddeca5d3e71+        .quad   0xb491c1e014cc3e6d+        .quad   0x08f5114789a8dba8+        .quad   0x3304fb0e63066222+        .quad   0xfb35068987acba3f+        .quad   0xbd1924778c1061a3+        .quad   0x3058ad43d1838620+        .quad   0x323c0ffde57663d0+        .quad   0x05c3df38a22ea610+        .quad   0xbdc78abdac994f9a+        .quad   0x26549fa4efe3dc99++        // 2^212 * 1 * G++        .quad   0x738b38d787ce8f89+        .quad   0xb62658e24179a88d+        .quad   0x30738c9cf151316d+        .quad   0x49128c7f727275c9+        .quad   0x04dbbc17f75396b9+        .quad   0x69e6a2d7d2f86746+        .quad   0xc6409d99f53eabc6+        .quad   0x606175f6332e25d2+        .quad   0x4021370ef540e7dd+        .quad   0x0910d6f5a1f1d0a5+        .quad   0x4634aacd5b06b807+        .quad   0x6a39e6356944f235++        // 2^212 * 2 * G++        .quad   0x96cd5640df90f3e7+        .quad   0x6c3a760edbfa25ea+        .quad   0x24f3ef0959e33cc4+        .quad   0x42889e7e530d2e58+        .quad   0x1da1965774049e9d+        .quad   0xfbcd6ea198fe352b+        .quad   0xb1cbcd50cc5236a6+        .quad   0x1f5ec83d3f9846e2+        .quad   0x8efb23c3328ccb75+        .quad   0xaf42a207dd876ee9+        .quad   0x20fbdadc5dfae796+        .quad   0x241e246b06bf9f51++        // 2^212 * 3 * G++        .quad   0x29e68e57ad6e98f6+        .quad   0x4c9260c80b462065+        .quad   0x3f00862ea51ebb4b+        .quad   0x5bc2c77fb38d9097+        .quad   0x7eaafc9a6280bbb8+        .quad   0x22a70f12f403d809+        .quad   0x31ce40bb1bfc8d20+        .quad   0x2bc65635e8bd53ee+        .quad   0xe8d5dc9fa96bad93+        .quad   0xe58fb17dde1947dc+        .quad   0x681532ea65185fa3+        .quad   0x1fdd6c3b034a7830++        // 2^212 * 4 * G++        .quad   0x0a64e28c55dc18fe+        .quad   0xe3df9e993399ebdd+        .quad   0x79ac432370e2e652+        .quad   0x35ff7fc33ae4cc0e+        .quad   0x9c13a6a52dd8f7a9+        .quad   0x2dbb1f8c3efdcabf+        .quad   0x961e32405e08f7b5+        .quad   0x48c8a121bbe6c9e5+        .quad   0xfc415a7c59646445+        .quad   0xd224b2d7c128b615+        .quad   0x6035c9c905fbb912+        .quad   0x42d7a91274429fab++        // 2^212 * 5 * G++        .quad   0x4e6213e3eaf72ed3+        .quad   0x6794981a43acd4e7+        .quad   0xff547cde6eb508cb+        .quad   0x6fed19dd10fcb532+        .quad   0xa9a48947933da5bc+        .quad   0x4a58920ec2e979ec+        .quad   0x96d8800013e5ac4c+        .quad   0x453692d74b48b147+        .quad   0xdd775d99a8559c6f+        .quad   0xf42a2140df003e24+        .quad   0x5223e229da928a66+        .quad   0x063f46ba6d38f22c++        // 2^212 * 6 * G++        .quad   0xd2d242895f536694+        .quad   0xca33a2c542939b2c+        .quad   0x986fada6c7ddb95c+        .quad   0x5a152c042f712d5d+        .quad   0x39843cb737346921+        .quad   0xa747fb0738c89447+        .quad   0xcb8d8031a245307e+        .quad   0x67810f8e6d82f068+        .quad   0x3eeb8fbcd2287db4+        .quad   0x72c7d3a301a03e93+        .quad   0x5473e88cbd98265a+        .quad   0x7324aa515921b403++        // 2^212 * 7 * G++        .quad   0x857942f46c3cbe8e+        .quad   0xa1d364b14730c046+        .quad   0x1c8ed914d23c41bf+        .quad   0x0838e161eef6d5d2+        .quad   0xad23f6dae82354cb+        .quad   0x6962502ab6571a6d+        .quad   0x9b651636e38e37d1+        .quad   0x5cac5005d1a3312f+        .quad   0x8cc154cce9e39904+        .quad   0x5b3a040b84de6846+        .quad   0xc4d8a61cb1be5d6e+        .quad   0x40fb897bd8861f02++        // 2^212 * 8 * G++        .quad   0x84c5aa9062de37a1+        .quad   0x421da5000d1d96e1+        .quad   0x788286306a9242d9+        .quad   0x3c5e464a690d10da+        .quad   0xe57ed8475ab10761+        .quad   0x71435e206fd13746+        .quad   0x342f824ecd025632+        .quad   0x4b16281ea8791e7b+        .quad   0xd1c101d50b813381+        .quad   0xdee60f1176ee6828+        .quad   0x0cb68893383f6409+        .quad   0x6183c565f6ff484a++        // 2^216 * 1 * G++        .quad   0x741d5a461e6bf9d6+        .quad   0x2305b3fc7777a581+        .quad   0xd45574a26474d3d9+        .quad   0x1926e1dc6401e0ff+        .quad   0xdb468549af3f666e+        .quad   0xd77fcf04f14a0ea5+        .quad   0x3df23ff7a4ba0c47+        .quad   0x3a10dfe132ce3c85+        .quad   0xe07f4e8aea17cea0+        .quad   0x2fd515463a1fc1fd+        .quad   0x175322fd31f2c0f1+        .quad   0x1fa1d01d861e5d15++        // 2^216 * 2 * G++        .quad   0xcc8055947d599832+        .quad   0x1e4656da37f15520+        .quad   0x99f6f7744e059320+        .quad   0x773563bc6a75cf33+        .quad   0x38dcac00d1df94ab+        .quad   0x2e712bddd1080de9+        .quad   0x7f13e93efdd5e262+        .quad   0x73fced18ee9a01e5+        .quad   0x06b1e90863139cb3+        .quad   0xa493da67c5a03ecd+        .quad   0x8d77cec8ad638932+        .quad   0x1f426b701b864f44++        // 2^216 * 3 * G++        .quad   0xefc9264c41911c01+        .quad   0xf1a3b7b817a22c25+        .quad   0x5875da6bf30f1447+        .quad   0x4e1af5271d31b090+        .quad   0xf17e35c891a12552+        .quad   0xb76b8153575e9c76+        .quad   0xfa83406f0d9b723e+        .quad   0x0b76bb1b3fa7e438+        .quad   0x08b8c1f97f92939b+        .quad   0xbe6771cbd444ab6e+        .quad   0x22e5646399bb8017+        .quad   0x7b6dd61eb772a955++        // 2^216 * 4 * G++        .quad   0xb7adc1e850f33d92+        .quad   0x7998fa4f608cd5cf+        .quad   0xad962dbd8dfc5bdb+        .quad   0x703e9bceaf1d2f4f+        .quad   0x5730abf9ab01d2c7+        .quad   0x16fb76dc40143b18+        .quad   0x866cbe65a0cbb281+        .quad   0x53fa9b659bff6afe+        .quad   0x6c14c8e994885455+        .quad   0x843a5d6665aed4e5+        .quad   0x181bb73ebcd65af1+        .quad   0x398d93e5c4c61f50++        // 2^216 * 5 * G++        .quad   0x1c4bd16733e248f3+        .quad   0xbd9e128715bf0a5f+        .quad   0xd43f8cf0a10b0376+        .quad   0x53b09b5ddf191b13+        .quad   0xc3877c60d2e7e3f2+        .quad   0x3b34aaa030828bb1+        .quad   0x283e26e7739ef138+        .quad   0x699c9c9002c30577+        .quad   0xf306a7235946f1cc+        .quad   0x921718b5cce5d97d+        .quad   0x28cdd24781b4e975+        .quad   0x51caf30c6fcdd907++        // 2^216 * 6 * G++        .quad   0xa60ba7427674e00a+        .quad   0x630e8570a17a7bf3+        .quad   0x3758563dcf3324cc+        .quad   0x5504aa292383fdaa+        .quad   0x737af99a18ac54c7+        .quad   0x903378dcc51cb30f+        .quad   0x2b89bc334ce10cc7+        .quad   0x12ae29c189f8e99a+        .quad   0xa99ec0cb1f0d01cf+        .quad   0x0dd1efcc3a34f7ae+        .quad   0x55ca7521d09c4e22+        .quad   0x5fd14fe958eba5ea++        // 2^216 * 7 * G++        .quad   0xb5dc2ddf2845ab2c+        .quad   0x069491b10a7fe993+        .quad   0x4daaf3d64002e346+        .quad   0x093ff26e586474d1+        .quad   0x3c42fe5ebf93cb8e+        .quad   0xbedfa85136d4565f+        .quad   0xe0f0859e884220e8+        .quad   0x7dd73f960725d128+        .quad   0xb10d24fe68059829+        .quad   0x75730672dbaf23e5+        .quad   0x1367253ab457ac29+        .quad   0x2f59bcbc86b470a4++        // 2^216 * 8 * G++        .quad   0x83847d429917135f+        .quad   0xad1b911f567d03d7+        .quad   0x7e7748d9be77aad1+        .quad   0x5458b42e2e51af4a+        .quad   0x7041d560b691c301+        .quad   0x85201b3fadd7e71e+        .quad   0x16c2e16311335585+        .quad   0x2aa55e3d010828b1+        .quad   0xed5192e60c07444f+        .quad   0x42c54e2d74421d10+        .quad   0x352b4c82fdb5c864+        .quad   0x13e9004a8a768664++        // 2^220 * 1 * G++        .quad   0xcbb5b5556c032bff+        .quad   0xdf7191b729297a3a+        .quad   0xc1ff7326aded81bb+        .quad   0x71ade8bb68be03f5+        .quad   0x1e6284c5806b467c+        .quad   0xc5f6997be75d607b+        .quad   0x8b67d958b378d262+        .quad   0x3d88d66a81cd8b70+        .quad   0x8b767a93204ed789+        .quad   0x762fcacb9fa0ae2a+        .quad   0x771febcc6dce4887+        .quad   0x343062158ff05fb3++        // 2^220 * 2 * G++        .quad   0xe05da1a7e1f5bf49+        .quad   0x26457d6dd4736092+        .quad   0x77dcb07773cc32f6+        .quad   0x0a5d94969cdd5fcd+        .quad   0xfce219072a7b31b4+        .quad   0x4d7adc75aa578016+        .quad   0x0ec276a687479324+        .quad   0x6d6d9d5d1fda4beb+        .quad   0x22b1a58ae9b08183+        .quad   0xfd95d071c15c388b+        .quad   0xa9812376850a0517+        .quad   0x33384cbabb7f335e++        // 2^220 * 3 * G++        .quad   0x3c6fa2680ca2c7b5+        .quad   0x1b5082046fb64fda+        .quad   0xeb53349c5431d6de+        .quad   0x5278b38f6b879c89+        .quad   0x33bc627a26218b8d+        .quad   0xea80b21fc7a80c61+        .quad   0x9458b12b173e9ee6+        .quad   0x076247be0e2f3059+        .quad   0x52e105f61416375a+        .quad   0xec97af3685abeba4+        .quad   0x26e6b50623a67c36+        .quad   0x5cf0e856f3d4fb01++        // 2^220 * 4 * G++        .quad   0xf6c968731ae8cab4+        .quad   0x5e20741ecb4f92c5+        .quad   0x2da53be58ccdbc3e+        .quad   0x2dddfea269970df7+        .quad   0xbeaece313db342a8+        .quad   0xcba3635b842db7ee+        .quad   0xe88c6620817f13ef+        .quad   0x1b9438aa4e76d5c6+        .quad   0x8a50777e166f031a+        .quad   0x067b39f10fb7a328+        .quad   0x1925c9a6010fbd76+        .quad   0x6df9b575cc740905++        // 2^220 * 5 * G++        .quad   0x42c1192927f6bdcf+        .quad   0x8f91917a403d61ca+        .quad   0xdc1c5a668b9e1f61+        .quad   0x1596047804ec0f8d+        .quad   0xecdfc35b48cade41+        .quad   0x6a88471fb2328270+        .quad   0x740a4a2440a01b6a+        .quad   0x471e5796003b5f29+        .quad   0xda96bbb3aced37ac+        .quad   0x7a2423b5e9208cea+        .quad   0x24cc5c3038aebae2+        .quad   0x50c356afdc5dae2f++        // 2^220 * 6 * G++        .quad   0x09dcbf4341c30318+        .quad   0xeeba061183181dce+        .quad   0xc179c0cedc1e29a1+        .quad   0x1dbf7b89073f35b0+        .quad   0xcfed9cdf1b31b964+        .quad   0xf486a9858ca51af3+        .quad   0x14897265ea8c1f84+        .quad   0x784a53dd932acc00+        .quad   0x2d99f9df14fc4920+        .quad   0x76ccb60cc4499fe5+        .quad   0xa4132cbbe5cf0003+        .quad   0x3f93d82354f000ea++        // 2^220 * 7 * G++        .quad   0x8183e7689e04ce85+        .quad   0x678fb71e04465341+        .quad   0xad92058f6688edac+        .quad   0x5da350d3532b099a+        .quad   0xeaac12d179e14978+        .quad   0xff923ff3bbebff5e+        .quad   0x4af663e40663ce27+        .quad   0x0fd381a811a5f5ff+        .quad   0xf256aceca436df54+        .quad   0x108b6168ae69d6e8+        .quad   0x20d986cb6b5d036c+        .quad   0x655957b9fee2af50++        // 2^220 * 8 * G++        .quad   0xaea8b07fa902030f+        .quad   0xf88c766af463d143+        .quad   0x15b083663c787a60+        .quad   0x08eab1148267a4a8+        .quad   0xbdc1409bd002d0ac+        .quad   0x66660245b5ccd9a6+        .quad   0x82317dc4fade85ec+        .quad   0x02fe934b6ad7df0d+        .quad   0xef5cf100cfb7ea74+        .quad   0x22897633a1cb42ac+        .quad   0xd4ce0c54cef285e2+        .quad   0x30408c048a146a55++        // 2^224 * 1 * G++        .quad   0x739d8845832fcedb+        .quad   0xfa38d6c9ae6bf863+        .quad   0x32bc0dcab74ffef7+        .quad   0x73937e8814bce45e+        .quad   0xbb2e00c9193b877f+        .quad   0xece3a890e0dc506b+        .quad   0xecf3b7c036de649f+        .quad   0x5f46040898de9e1a+        .quad   0xb9037116297bf48d+        .quad   0xa9d13b22d4f06834+        .quad   0xe19715574696bdc6+        .quad   0x2cf8a4e891d5e835++        // 2^224 * 2 * G++        .quad   0x6d93fd8707110f67+        .quad   0xdd4c09d37c38b549+        .quad   0x7cb16a4cc2736a86+        .quad   0x2049bd6e58252a09+        .quad   0x2cb5487e17d06ba2+        .quad   0x24d2381c3950196b+        .quad   0xd7659c8185978a30+        .quad   0x7a6f7f2891d6a4f6+        .quad   0x7d09fd8d6a9aef49+        .quad   0xf0ee60be5b3db90b+        .quad   0x4c21b52c519ebfd4+        .quad   0x6011aadfc545941d++        // 2^224 * 3 * G++        .quad   0x5f67926dcf95f83c+        .quad   0x7c7e856171289071+        .quad   0xd6a1e7f3998f7a5b+        .quad   0x6fc5cc1b0b62f9e0+        .quad   0x63ded0c802cbf890+        .quad   0xfbd098ca0dff6aaa+        .quad   0x624d0afdb9b6ed99+        .quad   0x69ce18b779340b1e+        .quad   0xd1ef5528b29879cb+        .quad   0xdd1aae3cd47e9092+        .quad   0x127e0442189f2352+        .quad   0x15596b3ae57101f1++        // 2^224 * 4 * G++        .quad   0x462739d23f9179a2+        .quad   0xff83123197d6ddcf+        .quad   0x1307deb553f2148a+        .quad   0x0d2237687b5f4dda+        .quad   0x09ff31167e5124ca+        .quad   0x0be4158bd9c745df+        .quad   0x292b7d227ef556e5+        .quad   0x3aa4e241afb6d138+        .quad   0x2cc138bf2a3305f5+        .quad   0x48583f8fa2e926c3+        .quad   0x083ab1a25549d2eb+        .quad   0x32fcaa6e4687a36c++        // 2^224 * 5 * G++        .quad   0x7bc56e8dc57d9af5+        .quad   0x3e0bd2ed9df0bdf2+        .quad   0xaac014de22efe4a3+        .quad   0x4627e9cefebd6a5c+        .quad   0x3207a4732787ccdf+        .quad   0x17e31908f213e3f8+        .quad   0xd5b2ecd7f60d964e+        .quad   0x746f6336c2600be9+        .quad   0x3f4af345ab6c971c+        .quad   0xe288eb729943731f+        .quad   0x33596a8a0344186d+        .quad   0x7b4917007ed66293++        // 2^224 * 6 * G++        .quad   0x2d85fb5cab84b064+        .quad   0x497810d289f3bc14+        .quad   0x476adc447b15ce0c+        .quad   0x122ba376f844fd7b+        .quad   0x54341b28dd53a2dd+        .quad   0xaa17905bdf42fc3f+        .quad   0x0ff592d94dd2f8f4+        .quad   0x1d03620fe08cd37d+        .quad   0xc20232cda2b4e554+        .quad   0x9ed0fd42115d187f+        .quad   0x2eabb4be7dd479d9+        .quad   0x02c70bf52b68ec4c++        // 2^224 * 7 * G++        .quad   0xa287ec4b5d0b2fbb+        .quad   0x415c5790074882ca+        .quad   0xe044a61ec1d0815c+        .quad   0x26334f0a409ef5e0+        .quad   0xace532bf458d72e1+        .quad   0x5be768e07cb73cb5+        .quad   0x56cf7d94ee8bbde7+        .quad   0x6b0697e3feb43a03+        .quad   0xb6c8f04adf62a3c0+        .quad   0x3ef000ef076da45d+        .quad   0x9c9cb95849f0d2a9+        .quad   0x1cc37f43441b2fae++        // 2^224 * 8 * G++        .quad   0x508f565a5cc7324f+        .quad   0xd061c4c0e506a922+        .quad   0xfb18abdb5c45ac19+        .quad   0x6c6809c10380314a+        .quad   0xd76656f1c9ceaeb9+        .quad   0x1c5b15f818e5656a+        .quad   0x26e72832844c2334+        .quad   0x3a346f772f196838+        .quad   0xd2d55112e2da6ac8+        .quad   0xe9bd0331b1e851ed+        .quad   0x960746dd8ec67262+        .quad   0x05911b9f6ef7c5d0++        // 2^228 * 1 * G++        .quad   0xe9dcd756b637ff2d+        .quad   0xec4c348fc987f0c4+        .quad   0xced59285f3fbc7b7+        .quad   0x3305354793e1ea87+        .quad   0x01c18980c5fe9f94+        .quad   0xcd656769716fd5c8+        .quad   0x816045c3d195a086+        .quad   0x6e2b7f3266cc7982+        .quad   0xcc802468f7c3568f+        .quad   0x9de9ba8219974cb3+        .quad   0xabb7229cb5b81360+        .quad   0x44e2017a6fbeba62++        // 2^228 * 2 * G++        .quad   0xc4c2a74354dab774+        .quad   0x8e5d4c3c4eaf031a+        .quad   0xb76c23d242838f17+        .quad   0x749a098f68dce4ea+        .quad   0x87f82cf3b6ca6ecd+        .quad   0x580f893e18f4a0c2+        .quad   0x058930072604e557+        .quad   0x6cab6ac256d19c1d+        .quad   0xdcdfe0a02cc1de60+        .quad   0x032665ff51c5575b+        .quad   0x2c0c32f1073abeeb+        .quad   0x6a882014cd7b8606++        // 2^228 * 3 * G++        .quad   0xa52a92fea4747fb5+        .quad   0xdc12a4491fa5ab89+        .quad   0xd82da94bb847a4ce+        .quad   0x4d77edce9512cc4e+        .quad   0xd111d17caf4feb6e+        .quad   0x050bba42b33aa4a3+        .quad   0x17514c3ceeb46c30+        .quad   0x54bedb8b1bc27d75+        .quad   0x77c8e14577e2189c+        .quad   0xa3e46f6aff99c445+        .quad   0x3144dfc86d335343+        .quad   0x3a96559e7c4216a9++        // 2^228 * 4 * G++        .quad   0x12550d37f42ad2ee+        .quad   0x8b78e00498a1fbf5+        .quad   0x5d53078233894cb2+        .quad   0x02c84e4e3e498d0c+        .quad   0x4493896880baaa52+        .quad   0x4c98afc4f285940e+        .quad   0xef4aa79ba45448b6+        .quad   0x5278c510a57aae7f+        .quad   0xa54dd074294c0b94+        .quad   0xf55d46b8df18ffb6+        .quad   0xf06fecc58dae8366+        .quad   0x588657668190d165++        // 2^228 * 5 * G++        .quad   0xd47712311aef7117+        .quad   0x50343101229e92c7+        .quad   0x7a95e1849d159b97+        .quad   0x2449959b8b5d29c9+        .quad   0xbf5834f03de25cc3+        .quad   0xb887c8aed6815496+        .quad   0x5105221a9481e892+        .quad   0x6760ed19f7723f93+        .quad   0x669ba3b7ac35e160+        .quad   0x2eccf73fba842056+        .quad   0x1aec1f17c0804f07+        .quad   0x0d96bc031856f4e7++        // 2^228 * 6 * G++        .quad   0x3318be7775c52d82+        .quad   0x4cb764b554d0aab9+        .quad   0xabcf3d27cc773d91+        .quad   0x3bf4d1848123288a+        .quad   0xb1d534b0cc7505e1+        .quad   0x32cd003416c35288+        .quad   0xcb36a5800762c29d+        .quad   0x5bfe69b9237a0bf8+        .quad   0x183eab7e78a151ab+        .quad   0xbbe990c999093763+        .quad   0xff717d6e4ac7e335+        .quad   0x4c5cddb325f39f88++        // 2^228 * 7 * G++        .quad   0xc0f6b74d6190a6eb+        .quad   0x20ea81a42db8f4e4+        .quad   0xa8bd6f7d97315760+        .quad   0x33b1d60262ac7c21+        .quad   0x57750967e7a9f902+        .quad   0x2c37fdfc4f5b467e+        .quad   0xb261663a3177ba46+        .quad   0x3a375e78dc2d532b+        .quad   0x8141e72f2d4dddea+        .quad   0xe6eafe9862c607c8+        .quad   0x23c28458573cafd0+        .quad   0x46b9476f4ff97346++        // 2^228 * 8 * G++        .quad   0x0c1ffea44f901e5c+        .quad   0x2b0b6fb72184b782+        .quad   0xe587ff910114db88+        .quad   0x37130f364785a142+        .quad   0x1215505c0d58359f+        .quad   0x2a2013c7fc28c46b+        .quad   0x24a0a1af89ea664e+        .quad   0x4400b638a1130e1f+        .quad   0x3a01b76496ed19c3+        .quad   0x31e00ab0ed327230+        .quad   0x520a885783ca15b1+        .quad   0x06aab9875accbec7++        // 2^232 * 1 * G++        .quad   0xc1339983f5df0ebb+        .quad   0xc0f3758f512c4cac+        .quad   0x2cf1130a0bb398e1+        .quad   0x6b3cecf9aa270c62+        .quad   0x5349acf3512eeaef+        .quad   0x20c141d31cc1cb49+        .quad   0x24180c07a99a688d+        .quad   0x555ef9d1c64b2d17+        .quad   0x36a770ba3b73bd08+        .quad   0x624aef08a3afbf0c+        .quad   0x5737ff98b40946f2+        .quad   0x675f4de13381749d++        // 2^232 * 2 * G++        .quad   0x0e2c52036b1782fc+        .quad   0x64816c816cad83b4+        .quad   0xd0dcbdd96964073e+        .quad   0x13d99df70164c520+        .quad   0xa12ff6d93bdab31d+        .quad   0x0725d80f9d652dfe+        .quad   0x019c4ff39abe9487+        .quad   0x60f450b882cd3c43+        .quad   0x014b5ec321e5c0ca+        .quad   0x4fcb69c9d719bfa2+        .quad   0x4e5f1c18750023a0+        .quad   0x1c06de9e55edac80++        // 2^232 * 3 * G++        .quad   0x990f7ad6a33ec4e2+        .quad   0x6608f938be2ee08e+        .quad   0x9ca143c563284515+        .quad   0x4cf38a1fec2db60d+        .quad   0xffd52b40ff6d69aa+        .quad   0x34530b18dc4049bb+        .quad   0x5e4a5c2fa34d9897+        .quad   0x78096f8e7d32ba2d+        .quad   0xa0aaaa650dfa5ce7+        .quad   0xf9c49e2a48b5478c+        .quad   0x4f09cc7d7003725b+        .quad   0x373cad3a26091abe++        // 2^232 * 4 * G++        .quad   0xb294634d82c9f57c+        .quad   0x1fcbfde124934536+        .quad   0x9e9c4db3418cdb5a+        .quad   0x0040f3d9454419fc+        .quad   0xf1bea8fb89ddbbad+        .quad   0x3bcb2cbc61aeaecb+        .quad   0x8f58a7bb1f9b8d9d+        .quad   0x21547eda5112a686+        .quad   0xdefde939fd5986d3+        .quad   0xf4272c89510a380c+        .quad   0xb72ba407bb3119b9+        .quad   0x63550a334a254df4++        // 2^232 * 5 * G++        .quad   0x6507d6edb569cf37+        .quad   0x178429b00ca52ee1+        .quad   0xea7c0090eb6bd65d+        .quad   0x3eea62c7daf78f51+        .quad   0x9bba584572547b49+        .quad   0xf305c6fae2c408e0+        .quad   0x60e8fa69c734f18d+        .quad   0x39a92bafaa7d767a+        .quad   0x9d24c713e693274e+        .quad   0x5f63857768dbd375+        .quad   0x70525560eb8ab39a+        .quad   0x68436a0665c9c4cd++        // 2^232 * 6 * G++        .quad   0xbc0235e8202f3f27+        .quad   0xc75c00e264f975b0+        .quad   0x91a4e9d5a38c2416+        .quad   0x17b6e7f68ab789f9+        .quad   0x1e56d317e820107c+        .quad   0xc5266844840ae965+        .quad   0xc1e0a1c6320ffc7a+        .quad   0x5373669c91611472+        .quad   0x5d2814ab9a0e5257+        .quad   0x908f2084c9cab3fc+        .quad   0xafcaf5885b2d1eca+        .quad   0x1cb4b5a678f87d11++        // 2^232 * 7 * G++        .quad   0xb664c06b394afc6c+        .quad   0x0c88de2498da5fb1+        .quad   0x4f8d03164bcad834+        .quad   0x330bca78de7434a2+        .quad   0x6b74aa62a2a007e7+        .quad   0xf311e0b0f071c7b1+        .quad   0x5707e438000be223+        .quad   0x2dc0fd2d82ef6eac+        .quad   0x982eff841119744e+        .quad   0xf9695e962b074724+        .quad   0xc58ac14fbfc953fb+        .quad   0x3c31be1b369f1cf5++        // 2^232 * 8 * G++        .quad   0xb0f4864d08948aee+        .quad   0x07dc19ee91ba1c6f+        .quad   0x7975cdaea6aca158+        .quad   0x330b61134262d4bb+        .quad   0xc168bc93f9cb4272+        .quad   0xaeb8711fc7cedb98+        .quad   0x7f0e52aa34ac8d7a+        .quad   0x41cec1097e7d55bb+        .quad   0xf79619d7a26d808a+        .quad   0xbb1fd49e1d9e156d+        .quad   0x73d7c36cdba1df27+        .quad   0x26b44cd91f28777d++        // 2^236 * 1 * G++        .quad   0x300a9035393aa6d8+        .quad   0x2b501131a12bb1cd+        .quad   0x7b1ff677f093c222+        .quad   0x4309c1f8cab82bad+        .quad   0xaf44842db0285f37+        .quad   0x8753189047efc8df+        .quad   0x9574e091f820979a+        .quad   0x0e378d6069615579+        .quad   0xd9fa917183075a55+        .quad   0x4bdb5ad26b009fdc+        .quad   0x7829ad2cd63def0e+        .quad   0x078fc54975fd3877++        // 2^236 * 2 * G++        .quad   0x87dfbd1428878f2d+        .quad   0x134636dd1e9421a1+        .quad   0x4f17c951257341a3+        .quad   0x5df98d4bad296cb8+        .quad   0xe2004b5bb833a98a+        .quad   0x44775dec2d4c3330+        .quad   0x3aa244067eace913+        .quad   0x272630e3d58e00a9+        .quad   0xf3678fd0ecc90b54+        .quad   0xf001459b12043599+        .quad   0x26725fbc3758b89b+        .quad   0x4325e4aa73a719ae++        // 2^236 * 3 * G++        .quad   0x657dc6ef433c3493+        .quad   0x65375e9f80dbf8c3+        .quad   0x47fd2d465b372dae+        .quad   0x4966ab79796e7947+        .quad   0xed24629acf69f59d+        .quad   0x2a4a1ccedd5abbf4+        .quad   0x3535ca1f56b2d67b+        .quad   0x5d8c68d043b1b42d+        .quad   0xee332d4de3b42b0a+        .quad   0xd84e5a2b16a4601c+        .quad   0x78243877078ba3e4+        .quad   0x77ed1eb4184ee437++        // 2^236 * 4 * G++        .quad   0xbfd4e13f201839a0+        .quad   0xaeefffe23e3df161+        .quad   0xb65b04f06b5d1fe3+        .quad   0x52e085fb2b62fbc0+        .quad   0x185d43f89e92ed1a+        .quad   0xb04a1eeafe4719c6+        .quad   0x499fbe88a6f03f4f+        .quad   0x5d8b0d2f3c859bdd+        .quad   0x124079eaa54cf2ba+        .quad   0xd72465eb001b26e7+        .quad   0x6843bcfdc97af7fd+        .quad   0x0524b42b55eacd02++        // 2^236 * 5 * G++        .quad   0xfd0d5dbee45447b0+        .quad   0x6cec351a092005ee+        .quad   0x99a47844567579cb+        .quad   0x59d242a216e7fa45+        .quad   0xbc18dcad9b829eac+        .quad   0x23ae7d28b5f579d0+        .quad   0xc346122a69384233+        .quad   0x1a6110b2e7d4ac89+        .quad   0x4f833f6ae66997ac+        .quad   0x6849762a361839a4+        .quad   0x6985dec1970ab525+        .quad   0x53045e89dcb1f546++        // 2^236 * 6 * G++        .quad   0xcb8bb346d75353db+        .quad   0xfcfcb24bae511e22+        .quad   0xcba48d40d50ae6ef+        .quad   0x26e3bae5f4f7cb5d+        .quad   0x84da3cde8d45fe12+        .quad   0xbd42c218e444e2d2+        .quad   0xa85196781f7e3598+        .quad   0x7642c93f5616e2b2+        .quad   0x2323daa74595f8e4+        .quad   0xde688c8b857abeb4+        .quad   0x3fc48e961c59326e+        .quad   0x0b2e73ca15c9b8ba++        // 2^236 * 7 * G++        .quad   0xd6bb4428c17f5026+        .quad   0x9eb27223fb5a9ca7+        .quad   0xe37ba5031919c644+        .quad   0x21ce380db59a6602+        .quad   0x0e3fbfaf79c03a55+        .quad   0x3077af054cbb5acf+        .quad   0xd5c55245db3de39f+        .quad   0x015e68c1476a4af7+        .quad   0xc1d5285220066a38+        .quad   0x95603e523570aef3+        .quad   0x832659a7226b8a4d+        .quad   0x5dd689091f8eedc9++        // 2^236 * 8 * G++        .quad   0xcbac84debfd3c856+        .quad   0x1624c348b35ff244+        .quad   0xb7f88dca5d9cad07+        .quad   0x3b0e574da2c2ebe8+        .quad   0x1d022591a5313084+        .quad   0xca2d4aaed6270872+        .quad   0x86a12b852f0bfd20+        .quad   0x56e6c439ad7da748+        .quad   0xc704ff4942bdbae6+        .quad   0x5e21ade2b2de1f79+        .quad   0xe95db3f35652fad8+        .quad   0x0822b5378f08ebc1++        // 2^240 * 1 * G++        .quad   0x51f048478f387475+        .quad   0xb25dbcf49cbecb3c+        .quad   0x9aab1244d99f2055+        .quad   0x2c709e6c1c10a5d6+        .quad   0xe1b7f29362730383+        .quad   0x4b5279ffebca8a2c+        .quad   0xdafc778abfd41314+        .quad   0x7deb10149c72610f+        .quad   0xcb62af6a8766ee7a+        .quad   0x66cbec045553cd0e+        .quad   0x588001380f0be4b5+        .quad   0x08e68e9ff62ce2ea++        // 2^240 * 2 * G++        .quad   0x34ad500a4bc130ad+        .quad   0x8d38db493d0bd49c+        .quad   0xa25c3d98500a89be+        .quad   0x2f1f3f87eeba3b09+        .quad   0x2f2d09d50ab8f2f9+        .quad   0xacb9218dc55923df+        .quad   0x4a8f342673766cb9+        .quad   0x4cb13bd738f719f5+        .quad   0xf7848c75e515b64a+        .quad   0xa59501badb4a9038+        .quad   0xc20d313f3f751b50+        .quad   0x19a1e353c0ae2ee8++        // 2^240 * 3 * G++        .quad   0x7d1c7560bafa05c3+        .quad   0xb3e1a0a0c6e55e61+        .quad   0xe3529718c0d66473+        .quad   0x41546b11c20c3486+        .quad   0xb42172cdd596bdbd+        .quad   0x93e0454398eefc40+        .quad   0x9fb15347b44109b5+        .quad   0x736bd3990266ae34+        .quad   0x85532d509334b3b4+        .quad   0x46fd114b60816573+        .quad   0xcc5f5f30425c8375+        .quad   0x412295a2b87fab5c++        // 2^240 * 4 * G++        .quad   0x19c99b88f57ed6e9+        .quad   0x5393cb266df8c825+        .quad   0x5cee3213b30ad273+        .quad   0x14e153ebb52d2e34+        .quad   0x2e655261e293eac6+        .quad   0x845a92032133acdb+        .quad   0x460975cb7900996b+        .quad   0x0760bb8d195add80+        .quad   0x413e1a17cde6818a+        .quad   0x57156da9ed69a084+        .quad   0x2cbf268f46caccb1+        .quad   0x6b34be9bc33ac5f2++        // 2^240 * 5 * G++        .quad   0xf3df2f643a78c0b2+        .quad   0x4c3e971ef22e027c+        .quad   0xec7d1c5e49c1b5a3+        .quad   0x2012c18f0922dd2d+        .quad   0x11fc69656571f2d3+        .quad   0xc6c9e845530e737a+        .quad   0xe33ae7a2d4fe5035+        .quad   0x01b9c7b62e6dd30b+        .quad   0x880b55e55ac89d29+        .quad   0x1483241f45a0a763+        .quad   0x3d36efdfc2e76c1f+        .quad   0x08af5b784e4bade8++        // 2^240 * 6 * G++        .quad   0x283499dc881f2533+        .quad   0x9d0525da779323b6+        .quad   0x897addfb673441f4+        .quad   0x32b79d71163a168d+        .quad   0xe27314d289cc2c4b+        .quad   0x4be4bd11a287178d+        .quad   0x18d528d6fa3364ce+        .quad   0x6423c1d5afd9826e+        .quad   0xcc85f8d9edfcb36a+        .quad   0x22bcc28f3746e5f9+        .quad   0xe49de338f9e5d3cd+        .quad   0x480a5efbc13e2dcc++        // 2^240 * 7 * G++        .quad   0x0b51e70b01622071+        .quad   0x06b505cf8b1dafc5+        .quad   0x2c6bb061ef5aabcd+        .quad   0x47aa27600cb7bf31+        .quad   0xb6614ce442ce221f+        .quad   0x6e199dcc4c053928+        .quad   0x663fb4a4dc1cbe03+        .quad   0x24b31d47691c8e06+        .quad   0x2a541eedc015f8c3+        .quad   0x11a4fe7e7c693f7c+        .quad   0xf0af66134ea278d6+        .quad   0x545b585d14dda094++        // 2^240 * 8 * G++        .quad   0x67bf275ea0d43a0f+        .quad   0xade68e34089beebe+        .quad   0x4289134cd479e72e+        .quad   0x0f62f9c332ba5454+        .quad   0x6204e4d0e3b321e1+        .quad   0x3baa637a28ff1e95+        .quad   0x0b0ccffd5b99bd9e+        .quad   0x4d22dc3e64c8d071+        .quad   0xfcb46589d63b5f39+        .quad   0x5cae6a3f57cbcf61+        .quad   0xfebac2d2953afa05+        .quad   0x1c0fa01a36371436++        // 2^244 * 1 * G++        .quad   0xe7547449bc7cd692+        .quad   0x0f9abeaae6f73ddf+        .quad   0x4af01ca700837e29+        .quad   0x63ab1b5d3f1bc183+        .quad   0xc11ee5e854c53fae+        .quad   0x6a0b06c12b4f3ff4+        .quad   0x33540f80e0b67a72+        .quad   0x15f18fc3cd07e3ef+        .quad   0x32750763b028f48c+        .quad   0x06020740556a065f+        .quad   0xd53bd812c3495b58+        .quad   0x08706c9b865f508d++        // 2^244 * 2 * G++        .quad   0xf37ca2ab3d343dff+        .quad   0x1a8c6a2d80abc617+        .quad   0x8e49e035d4ccffca+        .quad   0x48b46beebaa1d1b9+        .quad   0xcc991b4138b41246+        .quad   0x243b9c526f9ac26b+        .quad   0xb9ef494db7cbabbd+        .quad   0x5fba433dd082ed00+        .quad   0x9c49e355c9941ad0+        .quad   0xb9734ade74498f84+        .quad   0x41c3fed066663e5c+        .quad   0x0ecfedf8e8e710b3++        // 2^244 * 3 * G++        .quad   0x76430f9f9cd470d9+        .quad   0xb62acc9ba42f6008+        .quad   0x1898297c59adad5e+        .quad   0x7789dd2db78c5080+        .quad   0x744f7463e9403762+        .quad   0xf79a8dee8dfcc9c9+        .quad   0x163a649655e4cde3+        .quad   0x3b61788db284f435+        .quad   0xb22228190d6ef6b2+        .quad   0xa94a66b246ce4bfa+        .quad   0x46c1a77a4f0b6cc7+        .quad   0x4236ccffeb7338cf++        // 2^244 * 4 * G++        .quad   0x8497404d0d55e274+        .quad   0x6c6663d9c4ad2b53+        .quad   0xec2fb0d9ada95734+        .quad   0x2617e120cdb8f73c+        .quad   0x3bd82dbfda777df6+        .quad   0x71b177cc0b98369e+        .quad   0x1d0e8463850c3699+        .quad   0x5a71945b48e2d1f1+        .quad   0x6f203dd5405b4b42+        .quad   0x327ec60410b24509+        .quad   0x9c347230ac2a8846+        .quad   0x77de29fc11ffeb6a++        // 2^244 * 5 * G++        .quad   0xb0ac57c983b778a8+        .quad   0x53cdcca9d7fe912c+        .quad   0x61c2b854ff1f59dc+        .quad   0x3a1a2cf0f0de7dac+        .quad   0x835e138fecced2ca+        .quad   0x8c9eaf13ea963b9a+        .quad   0xc95fbfc0b2160ea6+        .quad   0x575e66f3ad877892+        .quad   0x99803a27c88fcb3a+        .quad   0x345a6789275ec0b0+        .quad   0x459789d0ff6c2be5+        .quad   0x62f882651e70a8b2++        // 2^244 * 6 * G++        .quad   0x085ae2c759ff1be4+        .quad   0x149145c93b0e40b7+        .quad   0xc467e7fa7ff27379+        .quad   0x4eeecf0ad5c73a95+        .quad   0x6d822986698a19e0+        .quad   0xdc9821e174d78a71+        .quad   0x41a85f31f6cb1f47+        .quad   0x352721c2bcda9c51+        .quad   0x48329952213fc985+        .quad   0x1087cf0d368a1746+        .quad   0x8e5261b166c15aa5+        .quad   0x2d5b2d842ed24c21++        // 2^244 * 7 * G++        .quad   0x02cfebd9ebd3ded1+        .quad   0xd45b217739021974+        .quad   0x7576f813fe30a1b7+        .quad   0x5691b6f9a34ef6c2+        .quad   0x5eb7d13d196ac533+        .quad   0x377234ecdb80be2b+        .quad   0xe144cffc7cf5ae24+        .quad   0x5226bcf9c441acec+        .quad   0x79ee6c7223e5b547+        .quad   0x6f5f50768330d679+        .quad   0xed73e1e96d8adce9+        .quad   0x27c3da1e1d8ccc03++        // 2^244 * 8 * G++        .quad   0x7eb9efb23fe24c74+        .quad   0x3e50f49f1651be01+        .quad   0x3ea732dc21858dea+        .quad   0x17377bd75bb810f9+        .quad   0x28302e71630ef9f6+        .quad   0xc2d4a2032b64cee0+        .quad   0x090820304b6292be+        .quad   0x5fca747aa82adf18+        .quad   0x232a03c35c258ea5+        .quad   0x86f23a2c6bcb0cf1+        .quad   0x3dad8d0d2e442166+        .quad   0x04a8933cab76862b++        // 2^248 * 1 * G++        .quad   0xd2c604b622943dff+        .quad   0xbc8cbece44cfb3a0+        .quad   0x5d254ff397808678+        .quad   0x0fa3614f3b1ca6bf+        .quad   0x69082b0e8c936a50+        .quad   0xf9c9a035c1dac5b6+        .quad   0x6fb73e54c4dfb634+        .quad   0x4005419b1d2bc140+        .quad   0xa003febdb9be82f0+        .quad   0x2089c1af3a44ac90+        .quad   0xf8499f911954fa8e+        .quad   0x1fba218aef40ab42++        // 2^248 * 2 * G++        .quad   0xab549448fac8f53e+        .quad   0x81f6e89a7ba63741+        .quad   0x74fd6c7d6c2b5e01+        .quad   0x392e3acaa8c86e42+        .quad   0x4f3e57043e7b0194+        .quad   0xa81d3eee08daaf7f+        .quad   0xc839c6ab99dcdef1+        .quad   0x6c535d13ff7761d5+        .quad   0x4cbd34e93e8a35af+        .quad   0x2e0781445887e816+        .quad   0x19319c76f29ab0ab+        .quad   0x25e17fe4d50ac13b++        // 2^248 * 3 * G++        .quad   0x0a289bd71e04f676+        .quad   0x208e1c52d6420f95+        .quad   0x5186d8b034691fab+        .quad   0x255751442a9fb351+        .quad   0x915f7ff576f121a7+        .quad   0xc34a32272fcd87e3+        .quad   0xccba2fde4d1be526+        .quad   0x6bba828f8969899b+        .quad   0xe2d1bc6690fe3901+        .quad   0x4cb54a18a0997ad5+        .quad   0x971d6914af8460d4+        .quad   0x559d504f7f6b7be4++        // 2^248 * 4 * G++        .quad   0xa7738378b3eb54d5+        .quad   0x1d69d366a5553c7c+        .quad   0x0a26cf62f92800ba+        .quad   0x01ab12d5807e3217+        .quad   0x9c4891e7f6d266fd+        .quad   0x0744a19b0307781b+        .quad   0x88388f1d6061e23b+        .quad   0x123ea6a3354bd50e+        .quad   0x118d189041e32d96+        .quad   0xb9ede3c2d8315848+        .quad   0x1eab4271d83245d9+        .quad   0x4a3961e2c918a154++        // 2^248 * 5 * G++        .quad   0x71dc3be0f8e6bba0+        .quad   0xd6cef8347effe30a+        .quad   0xa992425fe13a476a+        .quad   0x2cd6bce3fb1db763+        .quad   0x0327d644f3233f1e+        .quad   0x499a260e34fcf016+        .quad   0x83b5a716f2dab979+        .quad   0x68aceead9bd4111f+        .quad   0x38b4c90ef3d7c210+        .quad   0x308e6e24b7ad040c+        .quad   0x3860d9f1b7e73e23+        .quad   0x595760d5b508f597++        // 2^248 * 6 * G++        .quad   0x6129bfe104aa6397+        .quad   0x8f960008a4a7fccb+        .quad   0x3f8bc0897d909458+        .quad   0x709fa43edcb291a9+        .quad   0x882acbebfd022790+        .quad   0x89af3305c4115760+        .quad   0x65f492e37d3473f4+        .quad   0x2cb2c5df54515a2b+        .quad   0xeb0a5d8c63fd2aca+        .quad   0xd22bc1662e694eff+        .quad   0x2723f36ef8cbb03a+        .quad   0x70f029ecf0c8131f++        // 2^248 * 7 * G++        .quad   0x461307b32eed3e33+        .quad   0xae042f33a45581e7+        .quad   0xc94449d3195f0366+        .quad   0x0b7d5d8a6c314858+        .quad   0x2a6aafaa5e10b0b9+        .quad   0x78f0a370ef041aa9+        .quad   0x773efb77aa3ad61f+        .quad   0x44eca5a2a74bd9e1+        .quad   0x25d448327b95d543+        .quad   0x70d38300a3340f1d+        .quad   0xde1c531c60e1c52b+        .quad   0x272224512c7de9e4++        // 2^248 * 8 * G++        .quad   0x1abc92af49c5342e+        .quad   0xffeed811b2e6fad0+        .quad   0xefa28c8dfcc84e29+        .quad   0x11b5df18a44cc543+        .quad   0xbf7bbb8a42a975fc+        .quad   0x8c5c397796ada358+        .quad   0xe27fc76fcdedaa48+        .quad   0x19735fd7f6bc20a6+        .quad   0xe3ab90d042c84266+        .quad   0xeb848e0f7f19547e+        .quad   0x2503a1d065a497b9+        .quad   0x0fef911191df895f
+ cbits/s2n/x86_att/p256_montjadd.S view
@@ -0,0 +1,593 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Point addition on NIST curve P-256 in Montgomery-Jacobian coordinates+//+//    extern void p256_montjadd(uint64_t p3[static 12], const uint64_t p1[static 12],+//                              const uint64_t p2[static 12]);+//+// Does p3 := p1 + p2 where all points are regarded as Jacobian triples with+// each coordinate in the Montgomery domain, i.e. x' = (2^256 * x) mod p_256.+// A Jacobian triple (x',y',z') represents affine point (x/z^2,y/z^3).+//+// Standard x86-64 ABI: RDI = p3, RSI = p1, RDX = p2+// Microsoft x64 ABI:   RCX = p3, RDX = p1, R8 = p2+// ----------------------------------------------------------------------------+#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_montjadd)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_montjadd)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_montjadd)+        .text++// Size of individual field elements++#define NUMSIZE 32++// Pointer-offset pairs for inputs and outputs+// These assume %rdi = p3, %rsi = p1 and %rbp = p2,+// which needs to be set up explicitly before use.+// The first two hold initially, and the second is+// set up by copying the initial %rdx input to %rbp.+// Thereafter, no code macro modifies any of them.++#define x_1 0(%rsi)+#define y_1 NUMSIZE(%rsi)+#define z_1 (2*NUMSIZE)(%rsi)++#define x_2 0(%rbp)+#define y_2 NUMSIZE(%rbp)+#define z_2 (2*NUMSIZE)(%rbp)++#define x_3 0(%rdi)+#define y_3 NUMSIZE(%rdi)+#define z_3 (2*NUMSIZE)(%rdi)++// Pointer-offset pairs for temporaries, with some aliasing+// NSPACE is the total stack needed for these temporaries++#define z1sq (NUMSIZE*0)(%rsp)+#define ww (NUMSIZE*0)(%rsp)+#define resx (NUMSIZE*0)(%rsp)++#define yd (NUMSIZE*1)(%rsp)+#define y2a (NUMSIZE*1)(%rsp)++#define x2a (NUMSIZE*2)(%rsp)+#define zzx2 (NUMSIZE*2)(%rsp)++#define zz (NUMSIZE*3)(%rsp)+#define t1 (NUMSIZE*3)(%rsp)++#define t2 (NUMSIZE*4)(%rsp)+#define x1a (NUMSIZE*4)(%rsp)+#define zzx1 (NUMSIZE*4)(%rsp)+#define resy (NUMSIZE*4)(%rsp)++#define xd (NUMSIZE*5)(%rsp)+#define z2sq (NUMSIZE*5)(%rsp)+#define resz (NUMSIZE*5)(%rsp)++#define y1a (NUMSIZE*6)(%rsp)++#define NSPACE NUMSIZE*7++// Corresponds exactly to bignum_montmul_p256++#define montmul_p256(P0,P1,P2)                  \+        xorl    %r13d, %r13d ;                      \+        movq    P2, %rdx ;                       \+        mulxq   P1, %r8, %r9 ;                     \+        mulxq   0x8+P1, %rbx, %r10 ;               \+        adcq    %rbx, %r9 ;                         \+        mulxq   0x10+P1, %rbx, %r11 ;              \+        adcq    %rbx, %r10 ;                        \+        mulxq   0x18+P1, %rbx, %r12 ;              \+        adcq    %rbx, %r11 ;                        \+        adcq    %r13, %r12 ;                        \+        movq    0x8+P2, %rdx ;                   \+        xorl    %r14d, %r14d ;                      \+        mulxq   P1, %rax, %rbx ;                   \+        adcxq   %rax, %r9 ;                         \+        adoxq   %rbx, %r10 ;                        \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        mulxq   0x10+P1, %rax, %rbx ;              \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   0x18+P1, %rax, %rbx ;              \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcq    %r14, %r13 ;                        \+        xorl    %r15d, %r15d ;                      \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r9 ;                         \+        adoxq   %rbx, %r10 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        notq    %rdx;                            \+        leaq    0x2(%rdx), %rdx ;                  \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcxq   %r15, %r13 ;                        \+        adoxq   %r15, %r14 ;                        \+        adcq    %r15, %r14 ;                        \+        movq    0x10+P2, %rdx ;                  \+        xorl    %r8d, %r8d ;                        \+        mulxq   P1, %rax, %rbx ;                   \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   0x10+P1, %rax, %rbx ;              \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adoxq   %r8, %r14 ;                         \+        mulxq   0x18+P1, %rax, %rbx ;              \+        adcq    %rax, %r13 ;                        \+        adcq    %rbx, %r14 ;                        \+        adcq    %r8, %r15 ;                         \+        movq    0x18+P2, %rdx ;                  \+        xorl    %r9d, %r9d ;                        \+        mulxq   P1, %rax, %rbx ;                   \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        mulxq   0x10+P1, %rax, %rbx ;              \+        adcxq   %rax, %r13 ;                        \+        adoxq   %rbx, %r14 ;                        \+        adoxq   %r9, %r15 ;                         \+        mulxq   0x18+P1, %rax, %rbx ;              \+        adcq    %rax, %r14 ;                        \+        adcq    %rbx, %r15 ;                        \+        adcq    %r9, %r8 ;                          \+        xorl    %r9d, %r9d ;                        \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        notq    %rdx;                            \+        leaq    0x2(%rdx), %rdx ;                  \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r13 ;                        \+        adoxq   %rbx, %r14 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r14 ;                        \+        adoxq   %rbx, %r15 ;                        \+        adcxq   %r9, %r15 ;                         \+        adoxq   %r9, %r8 ;                          \+        adcq    %r9, %r8 ;                          \+        movl    $0x1, %ecx ;                        \+        addq    %r12, %rcx ;                        \+        decq    %rdx;                            \+        adcq    %r13, %rdx ;                        \+        decq    %r9;                             \+        movq    %r9, %rax ;                         \+        adcq    %r14, %r9 ;                         \+        movl    $0xfffffffe, %r11d ;                \+        adcq    %r15, %r11 ;                        \+        adcq    %r8, %rax ;                         \+        cmovbq  %rcx, %r12 ;                        \+        cmovbq  %rdx, %r13 ;                        \+        cmovbq  %r9, %r14 ;                         \+        cmovbq  %r11, %r15 ;                        \+        movq    %r12, P0 ;                       \+        movq    %r13, 0x8+P0 ;                   \+        movq    %r14, 0x10+P0 ;                  \+        movq    %r15, 0x18+P0++// Corresponds exactly to bignum_montsqr_p256 except for+// register tweaks to avoid modifying %rbp.++#define montsqr_p256(P0,P1)                     \+        movq    P1, %rdx ;                       \+        mulxq   %rdx, %r8, %r15 ;                     \+        mulxq   0x8+P1, %r9, %r10 ;                \+        mulxq   0x18+P1, %r11, %r12 ;              \+        movq    0x10+P1, %rdx ;                  \+        mulxq   0x18+P1, %r13, %r14 ;              \+        xorl    %ecx, %ecx ;                        \+        mulxq   P1, %rax, %rbx ;                   \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        movq    0x18+P1, %rdx ;                  \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcxq   %rcx, %r13 ;                        \+        adoxq   %rcx, %r14 ;                        \+        adcq    %rcx, %r14 ;                        \+        xorl    %ecx, %ecx ;                        \+        adcxq   %r9, %r9 ;                          \+        adoxq   %r15, %r9 ;                         \+        movq    0x8+P1, %rdx ;                   \+        mulxq   %rdx, %rax, %rdx ;                    \+        adcxq   %r10, %r10 ;                        \+        adoxq   %rax, %r10 ;                        \+        adcxq   %r11, %r11 ;                        \+        adoxq   %rdx, %r11 ;                        \+        movq    0x10+P1, %rdx ;                  \+        mulxq   %rdx, %rax, %rdx ;                    \+        adcxq   %r12, %r12 ;                        \+        adoxq   %rax, %r12 ;                        \+        adcxq   %r13, %r13 ;                        \+        adoxq   %rdx, %r13 ;                        \+        movq    0x18+P1, %rdx ;                  \+        mulxq   %rdx, %rax, %r15 ;                    \+        adcxq   %r14, %r14 ;                        \+        adoxq   %rax, %r14 ;                        \+        adcxq   %rcx, %r15 ;                        \+        adoxq   %rcx, %r15 ;                        \+        xorl    %ecx, %ecx ;                        \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r9 ;                         \+        adoxq   %rbx, %r10 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        movabsq $0xffffffff00000001, %rdx ;         \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcxq   %rcx, %r13 ;                        \+        movl    %ecx, %r9d ;                        \+        adoxq   %rcx, %r9 ;                         \+        adcxq   %rcx, %r9 ;                         \+        addq    %r9, %r14 ;                         \+        adcq    %rcx, %r15 ;                        \+        movl    %ecx, %r8d ;                        \+        adcq    %rcx, %r8 ;                         \+        xorl    %ecx, %ecx ;                        \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        movabsq $0xffffffff00000001, %rdx ;         \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r13 ;                        \+        adoxq   %rbx, %r14 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r14 ;                        \+        adoxq   %rbx, %r15 ;                        \+        adcxq   %rcx, %r15 ;                        \+        adoxq   %rcx, %r8 ;                         \+        adcq    %rcx, %r8 ;                         \+        movl    $0x1, %ebx ;                        \+        addq    %r12, %rbx ;                        \+        leaq    -0x1(%rdx), %rdx ;                  \+        adcq    %r13, %rdx ;                        \+        leaq    -0x1(%rcx), %rcx ;                  \+        movq    %rcx, %rax ;                        \+        adcq    %r14, %rcx ;                        \+        movl    $0xfffffffe, %r11d ;                \+        adcq    %r15, %r11 ;                        \+        adcq    %r8, %rax ;                         \+        cmovbq  %rbx, %r12 ;                        \+        cmovbq  %rdx, %r13 ;                        \+        cmovbq  %rcx, %r14 ;                        \+        cmovbq  %r11, %r15 ;                        \+        movq    %r12, P0 ;                       \+        movq    %r13, 0x8+P0 ;                   \+        movq    %r14, 0x10+P0 ;                  \+        movq    %r15, 0x18+P0++// Almost-Montgomery variant which we use when an input to other muls+// with the other argument fully reduced (which is always safe).+// Again, the basic squaring code is tweaked to avoid modifying %rbp.++#define amontsqr_p256(P0,P1)                    \+        movq    P1, %rdx ;                       \+        mulxq   %rdx, %r8, %r15 ;                     \+        mulxq   0x8+P1, %r9, %r10 ;                \+        mulxq   0x18+P1, %r11, %r12 ;              \+        movq    0x10+P1, %rdx ;                  \+        mulxq   0x18+P1, %r13, %r14 ;              \+        xorl    %ecx, %ecx ;                        \+        mulxq   P1, %rax, %rbx ;                   \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        movq    0x18+P1, %rdx ;                  \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcxq   %rcx, %r13 ;                        \+        adoxq   %rcx, %r14 ;                        \+        adcq    %rcx, %r14 ;                        \+        xorl    %ecx, %ecx ;                        \+        adcxq   %r9, %r9 ;                          \+        adoxq   %r15, %r9 ;                         \+        movq    0x8+P1, %rdx ;                   \+        mulxq   %rdx, %rax, %rdx ;                    \+        adcxq   %r10, %r10 ;                        \+        adoxq   %rax, %r10 ;                        \+        adcxq   %r11, %r11 ;                        \+        adoxq   %rdx, %r11 ;                        \+        movq    0x10+P1, %rdx ;                  \+        mulxq   %rdx, %rax, %rdx ;                    \+        adcxq   %r12, %r12 ;                        \+        adoxq   %rax, %r12 ;                        \+        adcxq   %r13, %r13 ;                        \+        adoxq   %rdx, %r13 ;                        \+        movq    0x18+P1, %rdx ;                  \+        mulxq   %rdx, %rax, %r15 ;                    \+        adcxq   %r14, %r14 ;                        \+        adoxq   %rax, %r14 ;                        \+        adcxq   %rcx, %r15 ;                        \+        adoxq   %rcx, %r15 ;                        \+        xorl    %ecx, %ecx ;                        \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r9 ;                         \+        adoxq   %rbx, %r10 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        movabsq $0xffffffff00000001, %rdx ;         \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcxq   %rcx, %r13 ;                        \+        movl    %ecx, %r9d ;                        \+        adoxq   %rcx, %r9 ;                         \+        adcxq   %rcx, %r9 ;                         \+        addq    %r9, %r14 ;                         \+        adcq    %rcx, %r15 ;                        \+        movl    %ecx, %r8d ;                        \+        adcq    %rcx, %r8 ;                         \+        xorl    %ecx, %ecx ;                        \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        movabsq $0xffffffff00000001, %rdx ;         \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r13 ;                        \+        adoxq   %rbx, %r14 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r14 ;                        \+        adoxq   %rbx, %r15 ;                        \+        adcxq   %rcx, %r15 ;                        \+        adoxq   %rcx, %r8 ;                         \+        adcq    %rcx, %r8 ;                         \+        movl    $0x1, %r8d ;                        \+        leaq    -0x1(%rdx), %rdx ;                  \+        leaq    -0x1(%rcx), %rax ;                  \+        movl    $0xfffffffe, %r11d ;                \+        cmovzq  %rcx, %r8 ;                         \+        cmovzq  %rcx, %rdx ;                        \+        cmovzq  %rcx, %rax ;                        \+        cmovzq  %rcx, %r11 ;                        \+        addq    %r8, %r12 ;                         \+        adcq    %rdx, %r13 ;                        \+        adcq    %rax, %r14 ;                        \+        adcq    %r11, %r15 ;                        \+        movq    %r12, P0 ;                       \+        movq    %r13, 0x8+P0 ;                   \+        movq    %r14, 0x10+P0 ;                  \+        movq    %r15, 0x18+P0++// Corresponds exactly to bignum_sub_p256++#define sub_p256(P0,P1,P2)                      \+        movq    P1, %rax ;                       \+        subq    P2, %rax ;                       \+        movq    0x8+P1, %rcx ;                   \+        sbbq    0x8+P2, %rcx ;                   \+        movq    0x10+P1, %r8 ;                   \+        sbbq    0x10+P2, %r8 ;                   \+        movq    0x18+P1, %r9 ;                   \+        sbbq    0x18+P2, %r9 ;                   \+        movl    $0xffffffff, %r10d ;                \+        sbbq    %r11, %r11 ;                        \+        xorq    %rdx, %rdx ;                        \+        andq    %r11, %r10 ;                        \+        subq    %r10, %rdx ;                        \+        addq    %r11, %rax ;                        \+        movq    %rax, P0 ;                       \+        adcq    %r10, %rcx ;                        \+        movq    %rcx, 0x8+P0 ;                   \+        adcq    $0x0, %r8 ;                         \+        movq    %r8, 0x10+P0 ;                   \+        adcq    %rdx, %r9 ;                         \+        movq    %r9, 0x18+P0++// Additional macros to help with final multiplexing++#define load4(r0,r1,r2,r3,P)                    \+        movq    P, r0 ;                        \+        movq    8+P, r1 ;                      \+        movq    16+P, r2 ;                     \+        movq    24+P, r3++#define store4(P,r0,r1,r2,r3)                   \+        movq    r0, P ;                        \+        movq    r1, 8+P ;                      \+        movq    r2, 16+P ;                     \+        movq    r3, 24+P++#define czload4(r0,r1,r2,r3,P)                  \+        cmovzq  P, r0 ;                        \+        cmovzq  8+P, r1 ;                      \+        cmovzq  16+P, r2 ;                     \+        cmovzq  24+P, r3++#define muxload4(r0,r1,r2,r3,P0,P1,P2)          \+        movq    P0, r0 ;                       \+        cmovbq  P1, r0 ;                       \+        cmovnbe P2, r0 ;                       \+        movq    8+P0, r1 ;                     \+        cmovbq  8+P1, r1 ;                     \+        cmovnbe 8+P2, r1 ;                     \+        movq    16+P0, r2 ;                    \+        cmovbq  16+P1, r2 ;                    \+        cmovnbe 16+P2, r2 ;                    \+        movq    24+P0, r3 ;                    \+        cmovbq  24+P1, r3 ;                    \+        cmovnbe 24+P2, r3++S2N_BN_SYMBOL(p256_montjadd):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+#endif++// Save registers and make room on stack for temporary variables+// Put the input y in %rbp where it lasts as long as it's needed.++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++        CFI_DEC_RSP(NSPACE)++        movq    %rdx, %rbp++// Main code, just a sequence of basic field operations+// 12 * multiply + 4 * square + 7 * subtract++        amontsqr_p256(z1sq,z_1)+        amontsqr_p256(z2sq,z_2)++        montmul_p256(y1a,z_2,y_1)+        montmul_p256(y2a,z_1,y_2)++        montmul_p256(x2a,z1sq,x_2)+        montmul_p256(x1a,z2sq,x_1)+        montmul_p256(y2a,z1sq,y2a)+        montmul_p256(y1a,z2sq,y1a)++        sub_p256(xd,x2a,x1a)+        sub_p256(yd,y2a,y1a)++        amontsqr_p256(zz,xd)+        montsqr_p256(ww,yd)++        montmul_p256(zzx1,zz,x1a)+        montmul_p256(zzx2,zz,x2a)++        sub_p256(resx,ww,zzx1)+        sub_p256(t1,zzx2,zzx1)++        montmul_p256(xd,xd,z_1)++        sub_p256(resx,resx,zzx2)++        sub_p256(t2,zzx1,resx)++        montmul_p256(t1,t1,y1a)++        montmul_p256(resz,xd,z_2)+        montmul_p256(t2,yd,t2)++        sub_p256(resy,t2,t1)++// Load in the z coordinates of the inputs to check for P1 = 0 and P2 = 0+// The condition codes get set by a comparison (P2 != 0) - (P1 != 0)+// So "NBE" <=> ~(CF \/ ZF) <=> P1 = 0 /\ ~(P2 = 0)+// and "B"  <=> CF          <=> ~(P1 = 0) /\ P2 = 0+// and "Z"  <=> ZF          <=> (P1 = 0 <=> P2 = 0)++        load4(%r8,%r9,%r10,%r11,z_1)++        movq    %r8, %rax+        movq    %r9, %rdx+        orq     %r10, %rax+        orq     %r11, %rdx+        orq     %rdx, %rax+        negq    %rax+        sbbq    %rax, %rax++        load4(%r12,%r13,%r14,%r15,z_2)++        movq    %r12, %rbx+        movq    %r13, %rdx+        orq     %r14, %rbx+        orq     %r15, %rdx+        orq     %rdx, %rbx+        negq    %rbx+        sbbq    %rbx, %rbx++        cmpq    %rax, %rbx++// Multiplex the outputs accordingly, re-using the z's in registers++        cmovbq  %r8, %r12+        cmovbq  %r9, %r13+        cmovbq  %r10, %r14+        cmovbq  %r11, %r15++        czload4(%r12,%r13,%r14,%r15,resz)++        muxload4(%rax,%rbx,%rcx,%rdx,resx,x_1,x_2)+        muxload4(%r8,%r9,%r10,%r11,resy,y_1,y_2)++// Finally store back the multiplexed values++        store4(x_3,%rax,%rbx,%rcx,%rdx)+        store4(y_3,%r8,%r9,%r10,%r11)+        store4(z_3,%r12,%r13,%r14,%r15)++// Restore stack and registers++        CFI_INC_RSP(NSPACE)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_montjadd)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/p256_montjadd_alt.S view
@@ -0,0 +1,579 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Point addition on NIST curve P-256 in Montgomery-Jacobian coordinates+//+//    extern void p256_montjadd_alt(uint64_t p3[static 12],+//                                  const uint64_t p1[static 12],+//                                  const uint64_t p2[static 12]);+//+// Does p3 := p1 + p2 where all points are regarded as Jacobian triples with+// each coordinate in the Montgomery domain, i.e. x' = (2^256 * x) mod p_256.+// A Jacobian triple (x',y',z') represents affine point (x/z^2,y/z^3).+//+// Standard x86-64 ABI: RDI = p3, RSI = p1, RDX = p2+// Microsoft x64 ABI:   RCX = p3, RDX = p1, R8 = p2+// ----------------------------------------------------------------------------+#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_montjadd_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_montjadd_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_montjadd_alt)+        .text++// Size of individual field elements++#define NUMSIZE 32++// Pointer-offset pairs for inputs and outputs+// These assume %rdi = p3, %rsi = p1 and %rbp = p2,+// which needs to be set up explicitly before use.+// The first two hold initially, and the second is+// set up by copying the initial %rdx input to %rbp.+// Thereafter, no code macro modifies any of them.++#define x_1 0(%rsi)+#define y_1 NUMSIZE(%rsi)+#define z_1 (2*NUMSIZE)(%rsi)++#define x_2 0(%rbp)+#define y_2 NUMSIZE(%rbp)+#define z_2 (2*NUMSIZE)(%rbp)++#define x_3 0(%rdi)+#define y_3 NUMSIZE(%rdi)+#define z_3 (2*NUMSIZE)(%rdi)++// Pointer-offset pairs for temporaries, with some aliasing+// NSPACE is the total stack needed for these temporaries++#define z1sq (NUMSIZE*0)(%rsp)+#define ww (NUMSIZE*0)(%rsp)+#define resx (NUMSIZE*0)(%rsp)++#define yd (NUMSIZE*1)(%rsp)+#define y2a (NUMSIZE*1)(%rsp)++#define x2a (NUMSIZE*2)(%rsp)+#define zzx2 (NUMSIZE*2)(%rsp)++#define zz (NUMSIZE*3)(%rsp)+#define t1 (NUMSIZE*3)(%rsp)++#define t2 (NUMSIZE*4)(%rsp)+#define x1a (NUMSIZE*4)(%rsp)+#define zzx1 (NUMSIZE*4)(%rsp)+#define resy (NUMSIZE*4)(%rsp)++#define xd (NUMSIZE*5)(%rsp)+#define z2sq (NUMSIZE*5)(%rsp)+#define resz (NUMSIZE*5)(%rsp)++#define y1a (NUMSIZE*6)(%rsp)++#define NSPACE NUMSIZE*7++// Corresponds exactly to bignum_montmul_p256_alt++#define montmul_p256(P0,P1,P2)                  \+        movq    P2, %rbx ;                      \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        movq    %rax, %r8 ;                        \+        movq    %rdx, %r9 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        xorl    %r10d, %r10d ;                     \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        movq    0x10+P1, %rax ;                 \+        mulq    %rbx;                            \+        xorl    %r11d, %r11d ;                     \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        movq    0x18+P1, %rax ;                 \+        mulq    %rbx;                            \+        xorl    %r12d, %r12d ;                     \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        movq    0x8+P2, %rbx ;                  \+        xorl    %r13d, %r13d ;                     \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        sbbq    %r14, %r14 ;                       \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %r14, %rdx ;                       \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %r14, %r14 ;                       \+        movq    0x10+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r14, %rdx ;                       \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %r14, %r14 ;                       \+        movq    0x18+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r14, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        xorl    %r14d, %r14d ;                     \+        movq    $0x100000000, %rbx ;               \+        movq    %r8, %rax ;                        \+        mulq    %rbx;                            \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        sbbq    %r15, %r15 ;                       \+        movq    %r9, %rax ;                        \+        mulq    %rbx;                            \+        subq    %r15, %rdx ;                       \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %r15, %r15 ;                       \+        notq    %rbx;                            \+        leaq    0x2(%rbx), %rbx ;                 \+        movq    %r8, %rax ;                        \+        mulq    %rbx;                            \+        subq    %r15, %rdx ;                       \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %r15, %r15 ;                       \+        movq    %r9, %rax ;                        \+        mulq    %rbx;                            \+        subq    %r15, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        adcq    %r14, %r14 ;                       \+        movq    0x10+P2, %rbx ;                 \+        xorl    %r15d, %r15d ;                     \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %r8, %r8 ;                         \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %r8, %rdx ;                        \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %r8, %r8 ;                         \+        movq    0x10+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r8, %rdx ;                        \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %r8, %r8 ;                         \+        movq    0x18+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r8, %rdx ;                        \+        addq    %rax, %r13 ;                       \+        adcq    %rdx, %r14 ;                       \+        adcq    %r15, %r15 ;                       \+        movq    0x18+P2, %rbx ;                 \+        xorl    %r8d, %r8d ;                       \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %r9, %r9 ;                         \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %r9, %rdx ;                        \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %r9, %r9 ;                         \+        movq    0x10+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r9, %rdx ;                        \+        addq    %rax, %r13 ;                       \+        adcq    %rdx, %r14 ;                       \+        sbbq    %r9, %r9 ;                         \+        movq    0x18+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r9, %rdx ;                        \+        addq    %rax, %r14 ;                       \+        adcq    %rdx, %r15 ;                       \+        adcq    %r8, %r8 ;                         \+        xorl    %r9d, %r9d ;                       \+        movq    $0x100000000, %rbx ;               \+        movq    %r10, %rax ;                       \+        mulq    %rbx;                            \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    %r11, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %rcx, %rcx ;                       \+        notq    %rbx;                            \+        leaq    0x2(%rbx), %rbx ;                 \+        movq    %r10, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r13 ;                       \+        adcq    %rdx, %r14 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    %r11, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r14 ;                       \+        adcq    %rdx, %r15 ;                       \+        adcq    %r9, %r8 ;                         \+        movl    $0x1, %ecx ;                       \+        addq    %r12, %rcx ;                       \+        decq    %rbx;                            \+        adcq    %r13, %rbx ;                       \+        decq    %r9;                             \+        movq    %r9, %rax ;                        \+        adcq    %r14, %r9 ;                        \+        movl    $0xfffffffe, %r11d ;               \+        adcq    %r15, %r11 ;                       \+        adcq    %r8, %rax ;                        \+        cmovbq  %rcx, %r12 ;                       \+        cmovbq  %rbx, %r13 ;                       \+        cmovbq  %r9, %r14 ;                        \+        cmovbq  %r11, %r15 ;                       \+        movq    %r12, P0 ;                      \+        movq    %r13, 0x8+P0 ;                  \+        movq    %r14, 0x10+P0 ;                 \+        movq    %r15, 0x18+P0++// Corresponds exactly to bignum_montsqr_p256_alt++#define montsqr_p256(P0,P1)                     \+        movq    P1, %rax ;                      \+        movq    %rax, %rbx ;                       \+        mulq    %rax;                            \+        movq    %rax, %r8 ;                        \+        movq    %rdx, %r15 ;                       \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        movq    %rax, %r9 ;                        \+        movq    %rdx, %r10 ;                       \+        movq    0x18+P1, %rax ;                 \+        movq    %rax, %r13 ;                       \+        mulq    %rbx;                            \+        movq    %rax, %r11 ;                       \+        movq    %rdx, %r12 ;                       \+        movq    0x10+P1, %rax ;                 \+        movq    %rax, %rbx ;                       \+        mulq    %r13;                            \+        movq    %rax, %r13 ;                       \+        movq    %rdx, %r14 ;                       \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    0x18+P1, %rbx ;                 \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        adcq    $0x0, %r14 ;                       \+        xorl    %ecx, %ecx ;                       \+        addq    %r9, %r9 ;                         \+        adcq    %r10, %r10 ;                       \+        adcq    %r11, %r11 ;                       \+        adcq    %r12, %r12 ;                       \+        adcq    %r13, %r13 ;                       \+        adcq    %r14, %r14 ;                       \+        adcq    %rcx, %rcx ;                       \+        movq    0x8+P1, %rax ;                  \+        mulq    %rax;                            \+        addq    %r15, %r9 ;                        \+        adcq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %r15, %r15 ;                       \+        movq    0x10+P1, %rax ;                 \+        mulq    %rax;                            \+        negq    %r15;                            \+        adcq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %r15, %r15 ;                       \+        movq    0x18+P1, %rax ;                 \+        mulq    %rax;                            \+        negq    %r15;                            \+        adcq    %rax, %r14 ;                       \+        adcq    %rcx, %rdx ;                       \+        movq    %rdx, %r15 ;                       \+        movq    $0x100000000, %rbx ;               \+        movq    %r8, %rax ;                        \+        mulq    %rbx;                            \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    %r9, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %rcx, %rcx ;                       \+        notq    %rbx;                            \+        leaq    0x2(%rbx), %rbx ;                 \+        movq    %r8, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %rcx, %rcx ;                       \+        xorl    %r8d, %r8d ;                       \+        movq    %r9, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        adcq    %r8, %r14 ;                        \+        adcq    %r8, %r15 ;                        \+        adcq    %r8, %r8 ;                         \+        movq    $0x100000000, %rbx ;               \+        movq    %r10, %rax ;                       \+        mulq    %rbx;                            \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    %r11, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %rcx, %rcx ;                       \+        notq    %rbx;                            \+        leaq    0x2(%rbx), %rbx ;                 \+        movq    %r10, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r13 ;                       \+        adcq    %rdx, %r14 ;                       \+        sbbq    %rcx, %rcx ;                       \+        xorl    %r9d, %r9d ;                       \+        movq    %r11, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r14 ;                       \+        adcq    %rdx, %r15 ;                       \+        adcq    %r9, %r8 ;                         \+        movl    $0x1, %ecx ;                       \+        addq    %r12, %rcx ;                       \+        leaq    -0x1(%rbx), %rbx ;                 \+        adcq    %r13, %rbx ;                       \+        leaq    -0x1(%r9), %r9 ;                   \+        movq    %r9, %rax ;                        \+        adcq    %r14, %r9 ;                        \+        movl    $0xfffffffe, %r11d ;               \+        adcq    %r15, %r11 ;                       \+        adcq    %r8, %rax ;                        \+        cmovbq  %rcx, %r12 ;                       \+        cmovbq  %rbx, %r13 ;                       \+        cmovbq  %r9, %r14 ;                        \+        cmovbq  %r11, %r15 ;                       \+        movq    %r12, P0 ;                      \+        movq    %r13, 0x8+P0 ;                  \+        movq    %r14, 0x10+P0 ;                 \+        movq    %r15, 0x18+P0++// Corresponds exactly to bignum_sub_p256++#define sub_p256(P0,P1,P2)                      \+        movq    P1, %rax ;                       \+        subq    P2, %rax ;                       \+        movq    0x8+P1, %rcx ;                   \+        sbbq    0x8+P2, %rcx ;                   \+        movq    0x10+P1, %r8 ;                   \+        sbbq    0x10+P2, %r8 ;                   \+        movq    0x18+P1, %r9 ;                   \+        sbbq    0x18+P2, %r9 ;                   \+        movl    $0xffffffff, %r10d ;                \+        sbbq    %r11, %r11 ;                        \+        xorq    %rdx, %rdx ;                        \+        andq    %r11, %r10 ;                        \+        subq    %r10, %rdx ;                        \+        addq    %r11, %rax ;                        \+        movq    %rax, P0 ;                       \+        adcq    %r10, %rcx ;                        \+        movq    %rcx, 0x8+P0 ;                   \+        adcq    $0x0, %r8 ;                         \+        movq    %r8, 0x10+P0 ;                   \+        adcq    %rdx, %r9 ;                         \+        movq    %r9, 0x18+P0++// Additional macros to help with final multiplexing++#define load4(r0,r1,r2,r3,P)                    \+        movq    P, r0 ;                        \+        movq    8+P, r1 ;                      \+        movq    16+P, r2 ;                     \+        movq    24+P, r3++#define store4(P,r0,r1,r2,r3)                   \+        movq    r0, P ;                        \+        movq    r1, 8+P ;                      \+        movq    r2, 16+P ;                     \+        movq    r3, 24+P++#define czload4(r0,r1,r2,r3,P)                  \+        cmovzq  P, r0 ;                        \+        cmovzq  8+P, r1 ;                      \+        cmovzq  16+P, r2 ;                     \+        cmovzq  24+P, r3++#define muxload4(r0,r1,r2,r3,P0,P1,P2)          \+        movq    P0, r0 ;                       \+        cmovbq  P1, r0 ;                       \+        cmovnbe P2, r0 ;                       \+        movq    8+P0, r1 ;                     \+        cmovbq  8+P1, r1 ;                     \+        cmovnbe 8+P2, r1 ;                     \+        movq    16+P0, r2 ;                    \+        cmovbq  16+P1, r2 ;                    \+        cmovnbe 16+P2, r2 ;                    \+        movq    24+P0, r3 ;                    \+        cmovbq  24+P1, r3 ;                    \+        cmovnbe 24+P2, r3++S2N_BN_SYMBOL(p256_montjadd_alt):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+#endif++// Save registers and make room on stack for temporary variables+// Put the input y in %rbp where it lasts as long as it's needed.++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++        CFI_DEC_RSP(NSPACE)++        movq    %rdx, %rbp++// Main code, just a sequence of basic field operations+// 12 * multiply + 4 * square + 7 * subtract++        montsqr_p256(z1sq,z_1)+        montsqr_p256(z2sq,z_2)++        montmul_p256(y1a,z_2,y_1)+        montmul_p256(y2a,z_1,y_2)++        montmul_p256(x2a,z1sq,x_2)+        montmul_p256(x1a,z2sq,x_1)+        montmul_p256(y2a,z1sq,y2a)+        montmul_p256(y1a,z2sq,y1a)++        sub_p256(xd,x2a,x1a)+        sub_p256(yd,y2a,y1a)++        montsqr_p256(zz,xd)+        montsqr_p256(ww,yd)++        montmul_p256(zzx1,zz,x1a)+        montmul_p256(zzx2,zz,x2a)++        sub_p256(resx,ww,zzx1)+        sub_p256(t1,zzx2,zzx1)++        montmul_p256(xd,xd,z_1)++        sub_p256(resx,resx,zzx2)++        sub_p256(t2,zzx1,resx)++        montmul_p256(t1,t1,y1a)++        montmul_p256(resz,xd,z_2)+        montmul_p256(t2,yd,t2)++        sub_p256(resy,t2,t1)++// Load in the z coordinates of the inputs to check for P1 = 0 and P2 = 0+// The condition codes get set by a comparison (P2 != 0) - (P1 != 0)+// So "NBE" <=> ~(CF \/ ZF) <=> P1 = 0 /\ ~(P2 = 0)+// and "B"  <=> CF          <=> ~(P1 = 0) /\ P2 = 0+// and "Z"  <=> ZF          <=> (P1 = 0 <=> P2 = 0)++        load4(%r8,%r9,%r10,%r11,z_1)++        movq    %r8, %rax+        movq    %r9, %rdx+        orq     %r10, %rax+        orq     %r11, %rdx+        orq     %rdx, %rax+        negq    %rax+        sbbq    %rax, %rax++        load4(%r12,%r13,%r14,%r15,z_2)++        movq    %r12, %rbx+        movq    %r13, %rdx+        orq     %r14, %rbx+        orq     %r15, %rdx+        orq     %rdx, %rbx+        negq    %rbx+        sbbq    %rbx, %rbx++        cmpq    %rax, %rbx++// Multiplex the outputs accordingly, re-using the z's in registers++        cmovbq  %r8, %r12+        cmovbq  %r9, %r13+        cmovbq  %r10, %r14+        cmovbq  %r11, %r15++        czload4(%r12,%r13,%r14,%r15,resz)++        muxload4(%rax,%rbx,%rcx,%rdx,resx,x_1,x_2)+        muxload4(%r8,%r9,%r10,%r11,resy,y_1,y_2)++// Finally store back the multiplexed values++        store4(x_3,%rax,%rbx,%rcx,%rdx)+        store4(y_3,%r8,%r9,%r10,%r11)+        store4(z_3,%r12,%r13,%r14,%r15)++// Restore stack and registers++        CFI_INC_RSP(NSPACE)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_montjadd_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/p256_montjdouble.S view
@@ -0,0 +1,634 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Point doubling on NIST curve P-256 in Montgomery-Jacobian coordinates+//+//    extern void p256_montjdouble(uint64_t p3[static 12],+//                                 const uint64_t p1[static 12]);+//+// Does p3 := 2 * p1 where all points are regarded as Jacobian triples with+// each coordinate in the Montgomery domain, i.e. x' = (2^256 * x) mod p_256.+// A Jacobian triple (x',y',z') represents affine point (x/z^2,y/z^3).+//+// Standard x86-64 ABI: RDI = p3, RSI = p1+// Microsoft x64 ABI:   RCX = p3, RDX = p1+// ----------------------------------------------------------------------------+#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_montjdouble)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_montjdouble)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_montjdouble)+        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Pointer-offset pairs for inputs and outputs+// These assume %rdi = p3, %rsi = p1, which is true when the+// arguments come in initially and is not disturbed throughout.++#define x_1 0(%rsi)+#define y_1 NUMSIZE(%rsi)+#define z_1 (2*NUMSIZE)(%rsi)++#define x_3 0(%rdi)+#define y_3 NUMSIZE(%rdi)+#define z_3 (2*NUMSIZE)(%rdi)++// Pointer-offset pairs for temporaries, with some aliasing+// NSPACE is the total stack needed for these temporaries++#define z2 (NUMSIZE*0)(%rsp)+#define y4 (NUMSIZE*0)(%rsp)++#define y2 (NUMSIZE*1)(%rsp)++#define t1 (NUMSIZE*2)(%rsp)++#define t2 (NUMSIZE*3)(%rsp)+#define x2p (NUMSIZE*3)(%rsp)+#define dx2 (NUMSIZE*3)(%rsp)++#define xy2 (NUMSIZE*4)(%rsp)++#define x4p (NUMSIZE*5)(%rsp)+#define d (NUMSIZE*5)(%rsp)++#define NSPACE NUMSIZE*6++// Corresponds exactly to bignum_montmul_p256++#define montmul_p256(P0,P1,P2)                  \+        xorl    %r13d, %r13d ;                      \+        movq    P2, %rdx ;                       \+        mulxq   P1, %r8, %r9 ;                     \+        mulxq   0x8+P1, %rbx, %r10 ;               \+        adcq    %rbx, %r9 ;                         \+        mulxq   0x10+P1, %rbx, %r11 ;              \+        adcq    %rbx, %r10 ;                        \+        mulxq   0x18+P1, %rbx, %r12 ;              \+        adcq    %rbx, %r11 ;                        \+        adcq    %r13, %r12 ;                        \+        movq    0x8+P2, %rdx ;                   \+        xorl    %r14d, %r14d ;                      \+        mulxq   P1, %rax, %rbx ;                   \+        adcxq   %rax, %r9 ;                         \+        adoxq   %rbx, %r10 ;                        \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        mulxq   0x10+P1, %rax, %rbx ;              \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   0x18+P1, %rax, %rbx ;              \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcq    %r14, %r13 ;                        \+        xorl    %r15d, %r15d ;                      \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r9 ;                         \+        adoxq   %rbx, %r10 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        notq    %rdx;                            \+        leaq    0x2(%rdx), %rdx ;                  \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcxq   %r15, %r13 ;                        \+        adoxq   %r15, %r14 ;                        \+        adcq    %r15, %r14 ;                        \+        movq    0x10+P2, %rdx ;                  \+        xorl    %r8d, %r8d ;                        \+        mulxq   P1, %rax, %rbx ;                   \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   0x10+P1, %rax, %rbx ;              \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adoxq   %r8, %r14 ;                         \+        mulxq   0x18+P1, %rax, %rbx ;              \+        adcq    %rax, %r13 ;                        \+        adcq    %rbx, %r14 ;                        \+        adcq    %r8, %r15 ;                         \+        movq    0x18+P2, %rdx ;                  \+        xorl    %r9d, %r9d ;                        \+        mulxq   P1, %rax, %rbx ;                   \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        mulxq   0x10+P1, %rax, %rbx ;              \+        adcxq   %rax, %r13 ;                        \+        adoxq   %rbx, %r14 ;                        \+        adoxq   %r9, %r15 ;                         \+        mulxq   0x18+P1, %rax, %rbx ;              \+        adcq    %rax, %r14 ;                        \+        adcq    %rbx, %r15 ;                        \+        adcq    %r9, %r8 ;                          \+        xorl    %r9d, %r9d ;                        \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        notq    %rdx;                            \+        leaq    0x2(%rdx), %rdx ;                  \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r13 ;                        \+        adoxq   %rbx, %r14 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r14 ;                        \+        adoxq   %rbx, %r15 ;                        \+        adcxq   %r9, %r15 ;                         \+        adoxq   %r9, %r8 ;                          \+        adcq    %r9, %r8 ;                          \+        movl    $0x1, %ecx ;                        \+        addq    %r12, %rcx ;                        \+        decq    %rdx;                            \+        adcq    %r13, %rdx ;                        \+        decq    %r9;                             \+        movq    %r9, %rax ;                         \+        adcq    %r14, %r9 ;                         \+        movl    $0xfffffffe, %r11d ;                \+        adcq    %r15, %r11 ;                        \+        adcq    %r8, %rax ;                         \+        cmovbq  %rcx, %r12 ;                        \+        cmovbq  %rdx, %r13 ;                        \+        cmovbq  %r9, %r14 ;                         \+        cmovbq  %r11, %r15 ;                        \+        movq    %r12, P0 ;                       \+        movq    %r13, 0x8+P0 ;                   \+        movq    %r14, 0x10+P0 ;                  \+        movq    %r15, 0x18+P0++// Corresponds exactly to bignum_montsqr_p256++#define montsqr_p256(P0,P1)                     \+        movq    P1, %rdx ;                       \+        mulxq   %rdx, %r8, %r15 ;                     \+        mulxq   0x8+P1, %r9, %r10 ;                \+        mulxq   0x18+P1, %r11, %r12 ;              \+        movq    0x10+P1, %rdx ;                  \+        mulxq   0x18+P1, %r13, %r14 ;              \+        xorl    %ebp, %ebp ;                        \+        mulxq   P1, %rax, %rbx ;                   \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        movq    0x18+P1, %rdx ;                  \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcxq   %rbp, %r13 ;                        \+        adoxq   %rbp, %r14 ;                        \+        adcq    %rbp, %r14 ;                        \+        xorl    %ebp, %ebp ;                        \+        adcxq   %r9, %r9 ;                          \+        adoxq   %r15, %r9 ;                         \+        movq    0x8+P1, %rdx ;                   \+        mulxq   %rdx, %rax, %rdx ;                    \+        adcxq   %r10, %r10 ;                        \+        adoxq   %rax, %r10 ;                        \+        adcxq   %r11, %r11 ;                        \+        adoxq   %rdx, %r11 ;                        \+        movq    0x10+P1, %rdx ;                  \+        mulxq   %rdx, %rax, %rdx ;                    \+        adcxq   %r12, %r12 ;                        \+        adoxq   %rax, %r12 ;                        \+        adcxq   %r13, %r13 ;                        \+        adoxq   %rdx, %r13 ;                        \+        movq    0x18+P1, %rdx ;                  \+        mulxq   %rdx, %rax, %r15 ;                    \+        adcxq   %r14, %r14 ;                        \+        adoxq   %rax, %r14 ;                        \+        adcxq   %rbp, %r15 ;                        \+        adoxq   %rbp, %r15 ;                        \+        xorl    %ebp, %ebp ;                        \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r9 ;                         \+        adoxq   %rbx, %r10 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        movabsq $0xffffffff00000001, %rdx ;         \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcxq   %rbp, %r13 ;                        \+        movl    %ebp, %r9d ;                        \+        adoxq   %rbp, %r9 ;                         \+        adcxq   %rbp, %r9 ;                         \+        addq    %r9, %r14 ;                         \+        adcq    %rbp, %r15 ;                        \+        movl    %ebp, %r8d ;                        \+        adcq    %rbp, %r8 ;                         \+        xorl    %ebp, %ebp ;                        \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        movabsq $0xffffffff00000001, %rdx ;         \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r13 ;                        \+        adoxq   %rbx, %r14 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r14 ;                        \+        adoxq   %rbx, %r15 ;                        \+        adcxq   %rbp, %r15 ;                        \+        adoxq   %rbp, %r8 ;                         \+        adcq    %rbp, %r8 ;                         \+        movl    $0x1, %ecx ;                        \+        addq    %r12, %rcx ;                        \+        leaq    -0x1(%rdx), %rdx ;                  \+        adcq    %r13, %rdx ;                        \+        leaq    -0x1(%rbp), %rbp ;                  \+        movq    %rbp, %rax ;                        \+        adcq    %r14, %rbp ;                        \+        movl    $0xfffffffe, %r11d ;                \+        adcq    %r15, %r11 ;                        \+        adcq    %r8, %rax ;                         \+        cmovbq  %rcx, %r12 ;                        \+        cmovbq  %rdx, %r13 ;                        \+        cmovbq  %rbp, %r14 ;                        \+        cmovbq  %r11, %r15 ;                        \+        movq    %r12, P0 ;                       \+        movq    %r13, 0x8+P0 ;                   \+        movq    %r14, 0x10+P0 ;                  \+        movq    %r15, 0x18+P0++// Corresponds exactly to bignum_sub_p256++#define sub_p256(P0,P1,P2)                      \+        movq    P1, %rax ;                       \+        subq    P2, %rax ;                       \+        movq    0x8+P1, %rcx ;                   \+        sbbq    0x8+P2, %rcx ;                   \+        movq    0x10+P1, %r8 ;                   \+        sbbq    0x10+P2, %r8 ;                   \+        movq    0x18+P1, %r9 ;                   \+        sbbq    0x18+P2, %r9 ;                   \+        movl    $0xffffffff, %r10d ;                \+        sbbq    %r11, %r11 ;                        \+        xorq    %rdx, %rdx ;                        \+        andq    %r11, %r10 ;                        \+        subq    %r10, %rdx ;                        \+        addq    %r11, %rax ;                        \+        movq    %rax, P0 ;                       \+        adcq    %r10, %rcx ;                        \+        movq    %rcx, 0x8+P0 ;                   \+        adcq    $0x0, %r8 ;                         \+        movq    %r8, 0x10+P0 ;                   \+        adcq    %rdx, %r9 ;                         \+        movq    %r9, 0x18+P0++// Corresponds exactly to bignum_add_p256++#define add_p256(P0,P1,P2)                      \+        xorq    %r11, %r11 ;                        \+        movq    P1, %rax ;                       \+        addq    P2, %rax ;                       \+        movq    0x8+P1, %rcx ;                   \+        adcq    0x8+P2, %rcx ;                   \+        movq    0x10+P1, %r8 ;                   \+        adcq    0x10+P2, %r8 ;                   \+        movq    0x18+P1, %r9 ;                   \+        adcq    0x18+P2, %r9 ;                   \+        adcq    %r11, %r11 ;                        \+        subq    $0xffffffffffffffff, %rax ;         \+        movl    $0xffffffff, %r10d ;                \+        sbbq    %r10, %rcx ;                        \+        sbbq    $0x0, %r8 ;                         \+        movq    $0xffffffff00000001, %rdx ;         \+        sbbq    %rdx, %r9 ;                         \+        sbbq    $0x0, %r11 ;                        \+        andq    %r11, %r10 ;                        \+        andq    %r11, %rdx ;                        \+        addq    %r11, %rax ;                        \+        movq    %rax, P0 ;                       \+        adcq    %r10, %rcx ;                        \+        movq    %rcx, 0x8+P0 ;                   \+        adcq    $0x0, %r8 ;                         \+        movq    %r8, 0x10+P0 ;                   \+        adcq    %rdx, %r9 ;                         \+        movq    %r9, 0x18+P0++// A weak version of add that only guarantees sum in 4 digits++#define weakadd_p256(P0,P1,P2)                  \+        movq    P1, %rax ;                       \+        addq    P2, %rax ;                       \+        movq    0x8+P1, %rcx ;                   \+        adcq    0x8+P2, %rcx ;                   \+        movq    0x10+P1, %r8 ;                   \+        adcq    0x10+P2, %r8 ;                   \+        movq    0x18+P1, %r9 ;                   \+        adcq    0x18+P2, %r9 ;                   \+        movl    $0xffffffff, %r10d ;                \+        sbbq    %r11, %r11 ;                        \+        xorq    %rdx, %rdx ;                        \+        andq    %r11, %r10 ;                        \+        subq    %r10, %rdx ;                        \+        subq    %r11, %rax ;                        \+        movq    %rax, P0 ;                       \+        sbbq    %r10, %rcx ;                        \+        movq    %rcx, 0x8+P0 ;                   \+        sbbq    $0x0, %r8 ;                         \+        movq    %r8, 0x10+P0 ;                   \+        sbbq    %rdx, %r9 ;                         \+        movq    %r9, 0x18+P0++// P0 = C * P1 - D * P2  computed as d * (p_256 - P2) + c * P1+// Quotient estimation is done just as q = h + 1 as in bignum_triple_p256+// This also applies to the other functions following.++#define cmsub_p256(P0,C,P1,D,P2)                \+        /* First (%r11;%r10;%r9;%r8) = p_256 - P2 */ \+        movq    $0xffffffffffffffff, %r8 ;          \+        xorl    %r10d, %r10d ;                      \+        subq    P2, %r8 ;                        \+        movq    $0x00000000ffffffff, %r9 ;          \+        sbbq    0x8+P2, %r9 ;                    \+        sbbq    0x10+P2, %r10 ;                  \+        movq    $0xffffffff00000001, %r11 ;         \+        sbbq    0x18+P2, %r11 ;                  \+        /* (%r12;%r11;%r10;%r9;%r8) = D * (p_256 - P2) */  \+        xorl    %r12d, %r12d ;                      \+        movq    $D, %rdx ;                         \+        mulxq   %r8, %r8, %rax ;                    \+        mulxq   %r9, %r9, %rcx ;                    \+        addq    %rax, %r9 ;                        \+        mulxq   %r10, %r10, %rax ;                  \+        adcq    %rcx, %r10 ;                       \+        mulxq   %r11, %r11, %rcx ;                  \+        adcq    %rax, %r11 ;                       \+        adcq    %rcx, %r12 ;                       \+        /* (%rdx;%r11;%r10;%r9;%r8) = 2^256 + C * P1 + D * (p_256 - P2) */ \+        movq    $C, %rdx ;                         \+        xorl    %eax, %eax ;                       \+        mulxq   P1, %rax, %rcx ;                 \+        adcxq   %rax, %r8 ;                        \+        adoxq   %rcx, %r9 ;                        \+        mulxq   0x8+P1, %rax, %rcx ;             \+        adcxq   %rax, %r9 ;                        \+        adoxq   %rcx, %r10 ;                       \+        mulxq   0x10+P1, %rax, %rcx ;            \+        adcxq   %rax, %r10 ;                       \+        adoxq   %rcx, %r11 ;                       \+        mulxq   0x18+P1, %rax, %rdx ;            \+        adcxq   %rax, %r11 ;                       \+        adoxq   %r12, %rdx ;                       \+        adcq    $1, %rdx ;                         \+        /* Now the tail for modular reduction from tripling */ \+        addq    %rdx, %r8 ;                         \+        movq    $0x100000000, %rax ;                \+        mulxq   %rax, %rax, %rcx ;                    \+        sbbq    $0x0, %rax ;                        \+        sbbq    $0x0, %rcx ;                        \+        subq    %rax, %r9 ;                         \+        sbbq    %rcx, %r10 ;                        \+        movq    $0xffffffff00000001, %rax ;         \+        mulxq   %rax, %rax, %rcx ;                    \+        sbbq    %rax, %r11 ;                        \+        sbbq    %rcx, %rdx ;                        \+        decq    %rdx;                            \+        movl    $0xffffffff, %eax ;                 \+        andq    %rdx, %rax ;                        \+        xorl    %ecx, %ecx ;                        \+        subq    %rax, %rcx ;                        \+        addq    %rdx, %r8 ;                         \+        movq    %r8, P0 ;                        \+        adcq    %rax, %r9 ;                         \+        movq    %r9, 0x8+P0 ;                    \+        adcq    $0x0, %r10 ;                        \+        movq    %r10, 0x10+P0 ;                  \+        adcq    %rcx, %r11 ;                        \+        movq    %r11, 0x18+P0++// P0 = 3 * P1 - 8 * P2, computed as (p_256 - P2) << 3 + 3 * P1++#define cmsub38_p256(P0,P1,P2)                  \+        /* First (%r11;%r10;%r9;%r8) = p_256 - P2 */ \+        movq    $0xffffffffffffffff, %r8 ;          \+        xorl    %r10d, %r10d ;                      \+        subq    P2, %r8 ;                        \+        movq    $0x00000000ffffffff, %r9 ;          \+        sbbq    0x8+P2, %r9 ;                    \+        sbbq    0x10+P2, %r10 ;                  \+        movq    $0xffffffff00000001, %r11 ;         \+        sbbq    0x18+P2, %r11 ;                  \+        /* (%r12;%r11;%r10;%r9;%r8) = (p_256 - P2) << 3 */  \+        movq    %r11, %r12 ;                       \+        shldq   $3, %r10, %r11 ;                    \+        shldq   $3, %r9, %r10 ;                     \+        shldq   $3, %r8, %r9 ;                      \+        shlq    $3, %r8 ;                          \+        shrq    $61, %r12 ;                        \+        /* (%rdx;%r11;%r10;%r9;%r8) = 2^256 + 3 * P1 + 8 * (p_256 - P2) */ \+        movq    $3, %rdx ;                         \+        xorl    %eax, %eax ;                       \+        mulxq   P1, %rax, %rcx ;                 \+        adcxq   %rax, %r8 ;                        \+        adoxq   %rcx, %r9 ;                        \+        mulxq   0x8+P1, %rax, %rcx ;             \+        adcxq   %rax, %r9 ;                        \+        adoxq   %rcx, %r10 ;                       \+        mulxq   0x10+P1, %rax, %rcx ;            \+        adcxq   %rax, %r10 ;                       \+        adoxq   %rcx, %r11 ;                       \+        mulxq   0x18+P1, %rax, %rdx ;            \+        adcxq   %rax, %r11 ;                       \+        adoxq   %r12, %rdx ;                       \+        adcq    $1, %rdx ;                         \+        /* Now the tail for modular reduction from tripling */ \+        addq    %rdx, %r8 ;                         \+        movq    $0x100000000, %rax ;                \+        mulxq   %rax, %rax, %rcx ;                    \+        sbbq    $0x0, %rax ;                        \+        sbbq    $0x0, %rcx ;                        \+        subq    %rax, %r9 ;                         \+        sbbq    %rcx, %r10 ;                        \+        movq    $0xffffffff00000001, %rax ;         \+        mulxq   %rax, %rax, %rcx ;                    \+        sbbq    %rax, %r11 ;                        \+        sbbq    %rcx, %rdx ;                        \+        decq    %rdx;                            \+        movl    $0xffffffff, %eax ;                 \+        andq    %rdx, %rax ;                        \+        xorl    %ecx, %ecx ;                        \+        subq    %rax, %rcx ;                        \+        addq    %rdx, %r8 ;                         \+        movq    %r8, P0 ;                        \+        adcq    %rax, %r9 ;                         \+        movq    %r9, 0x8+P0 ;                    \+        adcq    $0x0, %r10 ;                        \+        movq    %r10, 0x10+P0 ;                  \+        adcq    %rcx, %r11 ;                        \+        movq    %r11, 0x18+P0++// P0 = 4 * P1 - P2, by direct subtraction of P2,+// since the quotient estimate still works safely+// for initial value > -p_256++#define cmsub41_p256(P0,P1,P2)                  \+        movq    0x18+P1, %r11 ;                  \+        movq    %r11, %rdx ;                       \+        movq    0x10+P1, %r10 ;                  \+        shldq   $2, %r10, %r11 ;                    \+        movq    0x8+P1, %r9 ;                    \+        shldq   $2, %r9, %r10 ;                     \+        movq    P1, %r8 ;                        \+        shldq   $2, %r8, %r9 ;                      \+        shlq    $2, %r8 ;                          \+        shrq    $62, %rdx ;                        \+        addq    $1, %rdx ;                         \+        subq    P2, %r8 ;                       \+        sbbq    0x8+P2, %r9 ;                   \+        sbbq    0x10+P2, %r10 ;                 \+        sbbq    0x18+P2, %r11 ;                 \+        sbbq    $0, %rdx ;                         \+        /* Now the tail for modular reduction from tripling */ \+        addq    %rdx, %r8 ;                         \+        movq    $0x100000000, %rax ;                \+        mulxq   %rax, %rax, %rcx ;                    \+        sbbq    $0x0, %rax ;                        \+        sbbq    $0x0, %rcx ;                        \+        subq    %rax, %r9 ;                         \+        sbbq    %rcx, %r10 ;                        \+        movq    $0xffffffff00000001, %rax ;         \+        mulxq   %rax, %rax, %rcx ;                    \+        sbbq    %rax, %r11 ;                        \+        sbbq    %rcx, %rdx ;                        \+        decq    %rdx;                            \+        movl    $0xffffffff, %eax ;                 \+        andq    %rdx, %rax ;                        \+        xorl    %ecx, %ecx ;                        \+        subq    %rax, %rcx ;                        \+        addq    %rdx, %r8 ;                         \+        movq    %r8, P0 ;                        \+        adcq    %rax, %r9 ;                         \+        movq    %r9, 0x8+P0 ;                    \+        adcq    $0x0, %r10 ;                        \+        movq    %r10, 0x10+P0 ;                  \+        adcq    %rcx, %r11 ;                        \+        movq    %r11, 0x18+P0++S2N_BN_SYMBOL(p256_montjdouble):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Save registers and make room on stack for temporary variables++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++        CFI_DEC_RSP(NSPACE)++// Main code, just a sequence of basic field operations++// z2 = z^2+// y2 = y^2++        montsqr_p256(z2,z_1)+        montsqr_p256(y2,y_1)++// x2p = x^2 - z^4 = (x + z^2) * (x - z^2)++        sub_p256(t2,x_1,z2)+        weakadd_p256(t1,x_1,z2)+        montmul_p256(x2p,t1,t2)++// t1 = y + z+// xy2 = x * y^2+// x4p = x2p^2++        add_p256(t1,y_1,z_1)+        montmul_p256(xy2,x_1,y2)+        montsqr_p256(x4p,x2p)++// t1 = (y + z)^2++        montsqr_p256(t1,t1)++// d = 12 * xy2 - 9 * x4p+// t1 = y^2 + 2 * y * z++        cmsub_p256(d,12,xy2,9,x4p)+        sub_p256(t1,t1,z2)++// y4 = y^4++        montsqr_p256(y4,y2)++// dx2 = d * x2p++        montmul_p256(dx2,d,x2p)++// z_3' = 2 * y * z++        sub_p256(z_3,t1,y2)++// x' = 4 * xy2 - d++        cmsub41_p256(x_3,xy2,d)++// y' = 3 * dx2 - 8 * y4++        cmsub38_p256(y_3,dx2,y4)++// Restore stack and registers++        CFI_INC_RSP(NSPACE)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_montjdouble)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/p256_montjdouble_alt.S view
@@ -0,0 +1,747 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Point doubling on NIST curve P-256 in Montgomery-Jacobian coordinates+//+//    extern void p256_montjdouble_alt(uint64_t p3[static 12],+//                                     const uint64_t p1[static 12]);+//+// Does p3 := 2 * p1 where all points are regarded as Jacobian triples with+// each coordinate in the Montgomery domain, i.e. x' = (2^256 * x) mod p_256.+// A Jacobian triple (x',y',z') represents affine point (x/z^2,y/z^3).+//+// Standard x86-64 ABI: RDI = p3, RSI = p1+// Microsoft x64 ABI:   RCX = p3, RDX = p1+// ----------------------------------------------------------------------------+#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_montjdouble_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_montjdouble_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_montjdouble_alt)+        .text+        .balign 4++// Size of individual field elements++#define NUMSIZE 32++// Pointer-offset pairs for inputs and outputs+// These assume %rdi = p3, %rsi = p1, which is true when the+// arguments come in initially and is not disturbed throughout.++#define x_1 0(%rsi)+#define y_1 NUMSIZE(%rsi)+#define z_1 (2*NUMSIZE)(%rsi)++#define x_3 0(%rdi)+#define y_3 NUMSIZE(%rdi)+#define z_3 (2*NUMSIZE)(%rdi)++// Pointer-offset pairs for temporaries, with some aliasing+// NSPACE is the total stack needed for these temporaries++#define z2 (NUMSIZE*0)(%rsp)+#define y4 (NUMSIZE*0)(%rsp)++#define y2 (NUMSIZE*1)(%rsp)++#define t1 (NUMSIZE*2)(%rsp)++#define t2 (NUMSIZE*3)(%rsp)+#define x2p (NUMSIZE*3)(%rsp)+#define dx2 (NUMSIZE*3)(%rsp)++#define xy2 (NUMSIZE*4)(%rsp)++#define x4p (NUMSIZE*5)(%rsp)+#define d (NUMSIZE*5)(%rsp)++#define NSPACE NUMSIZE*6++// Corresponds exactly to bignum_montmul_p256_alt++#define montmul_p256(P0,P1,P2)                  \+        movq    P2, %rbx ;                      \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        movq    %rax, %r8 ;                        \+        movq    %rdx, %r9 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        xorl    %r10d, %r10d ;                     \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        movq    0x10+P1, %rax ;                 \+        mulq    %rbx;                            \+        xorl    %r11d, %r11d ;                     \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        movq    0x18+P1, %rax ;                 \+        mulq    %rbx;                            \+        xorl    %r12d, %r12d ;                     \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        movq    0x8+P2, %rbx ;                  \+        xorl    %r13d, %r13d ;                     \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        sbbq    %r14, %r14 ;                       \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %r14, %rdx ;                       \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %r14, %r14 ;                       \+        movq    0x10+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r14, %rdx ;                       \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %r14, %r14 ;                       \+        movq    0x18+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r14, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        xorl    %r14d, %r14d ;                     \+        movq    $0x100000000, %rbx ;               \+        movq    %r8, %rax ;                        \+        mulq    %rbx;                            \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        sbbq    %r15, %r15 ;                       \+        movq    %r9, %rax ;                        \+        mulq    %rbx;                            \+        subq    %r15, %rdx ;                       \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %r15, %r15 ;                       \+        notq    %rbx;                            \+        leaq    0x2(%rbx), %rbx ;                 \+        movq    %r8, %rax ;                        \+        mulq    %rbx;                            \+        subq    %r15, %rdx ;                       \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %r15, %r15 ;                       \+        movq    %r9, %rax ;                        \+        mulq    %rbx;                            \+        subq    %r15, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        adcq    %r14, %r14 ;                       \+        movq    0x10+P2, %rbx ;                 \+        xorl    %r15d, %r15d ;                     \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %r8, %r8 ;                         \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %r8, %rdx ;                        \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %r8, %r8 ;                         \+        movq    0x10+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r8, %rdx ;                        \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %r8, %r8 ;                         \+        movq    0x18+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r8, %rdx ;                        \+        addq    %rax, %r13 ;                       \+        adcq    %rdx, %r14 ;                       \+        adcq    %r15, %r15 ;                       \+        movq    0x18+P2, %rbx ;                 \+        xorl    %r8d, %r8d ;                       \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %r9, %r9 ;                         \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %r9, %rdx ;                        \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %r9, %r9 ;                         \+        movq    0x10+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r9, %rdx ;                        \+        addq    %rax, %r13 ;                       \+        adcq    %rdx, %r14 ;                       \+        sbbq    %r9, %r9 ;                         \+        movq    0x18+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r9, %rdx ;                        \+        addq    %rax, %r14 ;                       \+        adcq    %rdx, %r15 ;                       \+        adcq    %r8, %r8 ;                         \+        xorl    %r9d, %r9d ;                       \+        movq    $0x100000000, %rbx ;               \+        movq    %r10, %rax ;                       \+        mulq    %rbx;                            \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    %r11, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %rcx, %rcx ;                       \+        notq    %rbx;                            \+        leaq    0x2(%rbx), %rbx ;                 \+        movq    %r10, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r13 ;                       \+        adcq    %rdx, %r14 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    %r11, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r14 ;                       \+        adcq    %rdx, %r15 ;                       \+        adcq    %r9, %r8 ;                         \+        movl    $0x1, %ecx ;                       \+        addq    %r12, %rcx ;                       \+        decq    %rbx;                            \+        adcq    %r13, %rbx ;                       \+        decq    %r9;                             \+        movq    %r9, %rax ;                        \+        adcq    %r14, %r9 ;                        \+        movl    $0xfffffffe, %r11d ;               \+        adcq    %r15, %r11 ;                       \+        adcq    %r8, %rax ;                        \+        cmovbq  %rcx, %r12 ;                       \+        cmovbq  %rbx, %r13 ;                       \+        cmovbq  %r9, %r14 ;                        \+        cmovbq  %r11, %r15 ;                       \+        movq    %r12, P0 ;                      \+        movq    %r13, 0x8+P0 ;                  \+        movq    %r14, 0x10+P0 ;                 \+        movq    %r15, 0x18+P0++// Corresponds exactly to bignum_montsqr_p256_alt++#define montsqr_p256(P0,P1)                     \+        movq    P1, %rax ;                      \+        movq    %rax, %rbx ;                       \+        mulq    %rax;                            \+        movq    %rax, %r8 ;                        \+        movq    %rdx, %r15 ;                       \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        movq    %rax, %r9 ;                        \+        movq    %rdx, %r10 ;                       \+        movq    0x18+P1, %rax ;                 \+        movq    %rax, %r13 ;                       \+        mulq    %rbx;                            \+        movq    %rax, %r11 ;                       \+        movq    %rdx, %r12 ;                       \+        movq    0x10+P1, %rax ;                 \+        movq    %rax, %rbx ;                       \+        mulq    %r13;                            \+        movq    %rax, %r13 ;                       \+        movq    %rdx, %r14 ;                       \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    0x18+P1, %rbx ;                 \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        adcq    $0x0, %r14 ;                       \+        xorl    %ecx, %ecx ;                       \+        addq    %r9, %r9 ;                         \+        adcq    %r10, %r10 ;                       \+        adcq    %r11, %r11 ;                       \+        adcq    %r12, %r12 ;                       \+        adcq    %r13, %r13 ;                       \+        adcq    %r14, %r14 ;                       \+        adcq    %rcx, %rcx ;                       \+        movq    0x8+P1, %rax ;                  \+        mulq    %rax;                            \+        addq    %r15, %r9 ;                        \+        adcq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %r15, %r15 ;                       \+        movq    0x10+P1, %rax ;                 \+        mulq    %rax;                            \+        negq    %r15;                            \+        adcq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %r15, %r15 ;                       \+        movq    0x18+P1, %rax ;                 \+        mulq    %rax;                            \+        negq    %r15;                            \+        adcq    %rax, %r14 ;                       \+        adcq    %rcx, %rdx ;                       \+        movq    %rdx, %r15 ;                       \+        movq    $0x100000000, %rbx ;               \+        movq    %r8, %rax ;                        \+        mulq    %rbx;                            \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    %r9, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %rcx, %rcx ;                       \+        notq    %rbx;                            \+        leaq    0x2(%rbx), %rbx ;                 \+        movq    %r8, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %rcx, %rcx ;                       \+        xorl    %r8d, %r8d ;                       \+        movq    %r9, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        adcq    %r8, %r14 ;                        \+        adcq    %r8, %r15 ;                        \+        adcq    %r8, %r8 ;                         \+        movq    $0x100000000, %rbx ;               \+        movq    %r10, %rax ;                       \+        mulq    %rbx;                            \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    %r11, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %rcx, %rcx ;                       \+        notq    %rbx;                            \+        leaq    0x2(%rbx), %rbx ;                 \+        movq    %r10, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r13 ;                       \+        adcq    %rdx, %r14 ;                       \+        sbbq    %rcx, %rcx ;                       \+        xorl    %r9d, %r9d ;                       \+        movq    %r11, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r14 ;                       \+        adcq    %rdx, %r15 ;                       \+        adcq    %r9, %r8 ;                         \+        movl    $0x1, %ecx ;                       \+        addq    %r12, %rcx ;                       \+        leaq    -0x1(%rbx), %rbx ;                 \+        adcq    %r13, %rbx ;                       \+        leaq    -0x1(%r9), %r9 ;                   \+        movq    %r9, %rax ;                        \+        adcq    %r14, %r9 ;                        \+        movl    $0xfffffffe, %r11d ;               \+        adcq    %r15, %r11 ;                       \+        adcq    %r8, %rax ;                        \+        cmovbq  %rcx, %r12 ;                       \+        cmovbq  %rbx, %r13 ;                       \+        cmovbq  %r9, %r14 ;                        \+        cmovbq  %r11, %r15 ;                       \+        movq    %r12, P0 ;                      \+        movq    %r13, 0x8+P0 ;                  \+        movq    %r14, 0x10+P0 ;                 \+        movq    %r15, 0x18+P0++// Corresponds exactly to bignum_sub_p256++#define sub_p256(P0,P1,P2)                      \+        movq    P1, %rax ;                       \+        subq    P2, %rax ;                       \+        movq    0x8+P1, %rcx ;                   \+        sbbq    0x8+P2, %rcx ;                   \+        movq    0x10+P1, %r8 ;                   \+        sbbq    0x10+P2, %r8 ;                   \+        movq    0x18+P1, %r9 ;                   \+        sbbq    0x18+P2, %r9 ;                   \+        movl    $0xffffffff, %r10d ;                \+        sbbq    %r11, %r11 ;                        \+        xorq    %rdx, %rdx ;                        \+        andq    %r11, %r10 ;                        \+        subq    %r10, %rdx ;                        \+        addq    %r11, %rax ;                        \+        movq    %rax, P0 ;                       \+        adcq    %r10, %rcx ;                        \+        movq    %rcx, 0x8+P0 ;                   \+        adcq    $0x0, %r8 ;                         \+        movq    %r8, 0x10+P0 ;                   \+        adcq    %rdx, %r9 ;                         \+        movq    %r9, 0x18+P0++// Corresponds exactly to bignum_add_p256++#define add_p256(P0,P1,P2)                      \+        xorq    %r11, %r11 ;                        \+        movq    P1, %rax ;                       \+        addq    P2, %rax ;                       \+        movq    0x8+P1, %rcx ;                   \+        adcq    0x8+P2, %rcx ;                   \+        movq    0x10+P1, %r8 ;                   \+        adcq    0x10+P2, %r8 ;                   \+        movq    0x18+P1, %r9 ;                   \+        adcq    0x18+P2, %r9 ;                   \+        adcq    %r11, %r11 ;                        \+        subq    $0xffffffffffffffff, %rax ;         \+        movl    $0xffffffff, %r10d ;                \+        sbbq    %r10, %rcx ;                        \+        sbbq    $0x0, %r8 ;                         \+        movq    $0xffffffff00000001, %rdx ;         \+        sbbq    %rdx, %r9 ;                         \+        sbbq    $0x0, %r11 ;                        \+        andq    %r11, %r10 ;                        \+        andq    %r11, %rdx ;                        \+        addq    %r11, %rax ;                        \+        movq    %rax, P0 ;                       \+        adcq    %r10, %rcx ;                        \+        movq    %rcx, 0x8+P0 ;                   \+        adcq    $0x0, %r8 ;                         \+        movq    %r8, 0x10+P0 ;                   \+        adcq    %rdx, %r9 ;                         \+        movq    %r9, 0x18+P0++// A weak version of add that only guarantees sum in 4 digits++#define weakadd_p256(P0,P1,P2)                  \+        movq    P1, %rax ;                       \+        addq    P2, %rax ;                       \+        movq    0x8+P1, %rcx ;                   \+        adcq    0x8+P2, %rcx ;                   \+        movq    0x10+P1, %r8 ;                   \+        adcq    0x10+P2, %r8 ;                   \+        movq    0x18+P1, %r9 ;                   \+        adcq    0x18+P2, %r9 ;                   \+        movl    $0xffffffff, %r10d ;                \+        sbbq    %r11, %r11 ;                        \+        xorq    %rdx, %rdx ;                        \+        andq    %r11, %r10 ;                        \+        subq    %r10, %rdx ;                        \+        subq    %r11, %rax ;                        \+        movq    %rax, P0 ;                       \+        sbbq    %r10, %rcx ;                        \+        movq    %rcx, 0x8+P0 ;                   \+        sbbq    $0x0, %r8 ;                         \+        movq    %r8, 0x10+P0 ;                   \+        sbbq    %rdx, %r9 ;                         \+        movq    %r9, 0x18+P0++// P0 = C * P1 - D * P2  computed as d * (p_256 - P2) + c * P1+// Quotient estimation is done just as q = h + 1 as in bignum_triple_p256_alt.+// This also applies to the other functions following.++#define cmsub_p256(P0,C,P1,D,P2)                \+        /* First (%r12;%r11;%r10;%r9) = p_256 - P2 */ \+        movq    $0xffffffffffffffff, %r9 ;          \+        xorl    %r11d, %r11d ;                      \+        subq    P2, %r9 ;                        \+        movq    $0x00000000ffffffff, %r10 ;         \+        sbbq    0x8+P2, %r10 ;                   \+        sbbq    0x10+P2, %r11 ;                  \+        movq    $0xffffffff00000001, %r12 ;         \+        sbbq    0x18+P2, %r12 ;                  \+        /* (%r12;%r11;%r10;%r9;%r8) = D * (p_256 - P2) */  \+        movq    $D, %rcx ;                         \+        movq    %r9, %rax ;                        \+        mulq    %rcx;                            \+        movq    %rax, %r8 ;                        \+        movq    %rdx, %r9 ;                        \+        movq    %r10, %rax ;                       \+        xorl    %r10d, %r10d ;                     \+        mulq    %rcx;                            \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        movq    %r11, %rax ;                       \+        xorl    %r11d, %r11d ;                     \+        mulq    %rcx;                            \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        movq    %r12, %rax ;                       \+        xorl    %r12d, %r12d ;                     \+        mulq    %rcx;                            \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        /* (%rcx;%r11;%r10;%r9;%r8) = 2^256 + C * P1 + D * (p_256 - P2) */ \+        movl    $C, %ecx ;                         \+        movq    P1, %rax ;                      \+        mulq    %rcx;                            \+        addq    %rax, %r8 ;                        \+        adcq    %rdx, %r9 ;                        \+        sbbq    %rbx, %rbx ;                       \+        movq    0x8+P1, %rax ;                  \+        mulq    %rcx;                            \+        subq    %rbx, %rdx ;                       \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        sbbq    %rbx, %rbx ;                       \+        movq    0x10+P1, %rax ;                 \+        mulq    %rcx;                            \+        subq    %rbx, %rdx ;                       \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %rbx, %rbx ;                       \+        movq    0x18+P1, %rax ;                 \+        mulq    %rcx;                            \+        subq    %rbx, %rdx ;                       \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        leaq    1(%r12), %rcx ;                   \+        /* Now the tail for modular reduction from tripling */ \+        movq    $0xffffffff00000001, %rax ;        \+        mulq    %rcx;                            \+        movq    %rcx, %rbx ;                       \+        shlq    $0x20, %rbx ;                      \+        addq    %rcx, %r8 ;                        \+        sbbq    $0x0, %rbx ;                       \+        subq    %rbx, %r9 ;                        \+        sbbq    $0x0, %r10 ;                       \+        sbbq    %rax, %r11 ;                       \+        sbbq    %rdx, %rcx ;                       \+        decq    %rcx;                            \+        movl    $0xffffffff, %eax ;                \+        andq    %rcx, %rax ;                       \+        xorl    %edx, %edx ;                       \+        subq    %rax, %rdx ;                       \+        addq    %rcx, %r8 ;                        \+        movq    %r8, P0 ;                       \+        adcq    %rax, %r9 ;                        \+        movq    %r9, 0x8+P0 ;                   \+        adcq    $0x0, %r10 ;                       \+        movq    %r10, 0x10+P0 ;                 \+        adcq    %rdx, %r11 ;                       \+        movq    %r11, 0x18+P0++// P0 = 3 * P1 - 8 * P2, computed as (p_256 - P2) << 3 + 3 * P1++#define cmsub38_p256(P0,P1,P2)                  \+        /* First (%r11;%r10;%r9;%r8) = p_256 - P2 */ \+        movq    $0xffffffffffffffff, %r8 ;          \+        xorl    %r10d, %r10d ;                      \+        subq    P2, %r8 ;                        \+        movq    $0x00000000ffffffff, %r9 ;          \+        sbbq    0x8+P2, %r9 ;                    \+        sbbq    0x10+P2, %r10 ;                  \+        movq    $0xffffffff00000001, %r11 ;         \+        sbbq    0x18+P2, %r11 ;                  \+        /* (%r12;%r11;%r10;%r9;%r8) = (p_256 - P2) << 3 */  \+        movq    %r11, %r12 ;                       \+        shldq   $3, %r10, %r11 ;                    \+        shldq   $3, %r9, %r10 ;                     \+        shldq   $3, %r8, %r9 ;                      \+        shlq    $3, %r8 ;                          \+        shrq    $61, %r12 ;                        \+        /* (%rcx;%r11;%r10;%r9;%r8) = 2^256 + 3 * P1 + 8 * (p_256 - P2) */ \+        movl    $3, %ecx ;                         \+        movq    P1, %rax ;                      \+        mulq    %rcx;                            \+        addq    %rax, %r8 ;                        \+        adcq    %rdx, %r9 ;                        \+        sbbq    %rbx, %rbx ;                       \+        movq    0x8+P1, %rax ;                  \+        mulq    %rcx;                            \+        subq    %rbx, %rdx ;                       \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        sbbq    %rbx, %rbx ;                       \+        movq    0x10+P1, %rax ;                 \+        mulq    %rcx;                            \+        subq    %rbx, %rdx ;                       \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %rbx, %rbx ;                       \+        movq    0x18+P1, %rax ;                 \+        mulq    %rcx;                            \+        subq    %rbx, %rdx ;                       \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        leaq    1(%r12), %rcx ;                   \+        /* Now the tail for modular reduction from tripling */ \+        movq    $0xffffffff00000001, %rax ;        \+        mulq    %rcx;                            \+        movq    %rcx, %rbx ;                       \+        shlq    $0x20, %rbx ;                      \+        addq    %rcx, %r8 ;                        \+        sbbq    $0x0, %rbx ;                       \+        subq    %rbx, %r9 ;                        \+        sbbq    $0x0, %r10 ;                       \+        sbbq    %rax, %r11 ;                       \+        sbbq    %rdx, %rcx ;                       \+        decq    %rcx;                            \+        movl    $0xffffffff, %eax ;                \+        andq    %rcx, %rax ;                       \+        xorl    %edx, %edx ;                       \+        subq    %rax, %rdx ;                       \+        addq    %rcx, %r8 ;                        \+        movq    %r8, P0 ;                       \+        adcq    %rax, %r9 ;                        \+        movq    %r9, 0x8+P0 ;                   \+        adcq    $0x0, %r10 ;                       \+        movq    %r10, 0x10+P0 ;                 \+        adcq    %rdx, %r11 ;                       \+        movq    %r11, 0x18+P0++// P0 = 4 * P1 - P2, by direct subtraction of P2,+// since the quotient estimate still works safely+// for initial value > -p_256++#define cmsub41_p256(P0,P1,P2)                  \+        movq    0x18+P1, %r11 ;                  \+        movq    %r11, %rcx ;                       \+        movq    0x10+P1, %r10 ;                  \+        shldq   $2, %r10, %r11 ;                    \+        movq    0x8+P1, %r9 ;                    \+        shldq   $2, %r9, %r10 ;                     \+        movq    P1, %r8 ;                        \+        shldq   $2, %r8, %r9 ;                      \+        shlq    $2, %r8 ;                          \+        shrq    $62, %rcx ;                        \+        addq    $1, %rcx ;                         \+        subq    P2, %r8 ;                       \+        sbbq    0x8+P2, %r9 ;                   \+        sbbq    0x10+P2, %r10 ;                 \+        sbbq    0x18+P2, %r11 ;                 \+        sbbq    $0, %rcx ;                         \+        /* Now the tail for modular reduction from tripling */ \+        movq    $0xffffffff00000001, %rax ;        \+        mulq    %rcx;                            \+        movq    %rcx, %rbx ;                       \+        shlq    $0x20, %rbx ;                      \+        addq    %rcx, %r8 ;                        \+        sbbq    $0x0, %rbx ;                       \+        subq    %rbx, %r9 ;                        \+        sbbq    $0x0, %r10 ;                       \+        sbbq    %rax, %r11 ;                       \+        sbbq    %rdx, %rcx ;                       \+        decq    %rcx;                            \+        movl    $0xffffffff, %eax ;                \+        andq    %rcx, %rax ;                       \+        xorl    %edx, %edx ;                       \+        subq    %rax, %rdx ;                       \+        addq    %rcx, %r8 ;                        \+        movq    %r8, P0 ;                       \+        adcq    %rax, %r9 ;                        \+        movq    %r9, 0x8+P0 ;                   \+        adcq    $0x0, %r10 ;                       \+        movq    %r10, 0x10+P0 ;                 \+        adcq    %rdx, %r11 ;                       \+        movq    %r11, 0x18+P0++S2N_BN_SYMBOL(p256_montjdouble_alt):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+#endif++// Save registers and make room on stack for temporary variables++        CFI_PUSH(%rbx)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++        CFI_DEC_RSP(NSPACE)++// Main code, just a sequence of basic field operations++// z2 = z^2+// y2 = y^2++        montsqr_p256(z2,z_1)+        montsqr_p256(y2,y_1)++// x2p = x^2 - z^4 = (x + z^2) * (x - z^2)++        sub_p256(t2,x_1,z2)+        weakadd_p256(t1,x_1,z2)+        montmul_p256(x2p,t1,t2)++// t1 = y + z+// xy2 = x * y^2+// x4p = x2p^2++        add_p256(t1,y_1,z_1)+        montmul_p256(xy2,x_1,y2)+        montsqr_p256(x4p,x2p)++// t1 = (y + z)^2++        montsqr_p256(t1,t1)++// d = 12 * xy2 - 9 * x4p+// t1 = y^2 + 2 * y * z++        cmsub_p256(d,12,xy2,9,x4p)+        sub_p256(t1,t1,z2)++// y4 = y^4++        montsqr_p256(y4,y2)++// dx2 = d * x2p++        montmul_p256(dx2,d,x2p)++// z_3' = 2 * y * z++        sub_p256(z_3,t1,y2)++// x' = 4 * xy2 - d++        cmsub41_p256(x_3,xy2,d)++// y' = 3 * dx2 - 8 * y4++        cmsub38_p256(y_3,dx2,y4)++// Restore stack and registers++        CFI_INC_RSP(NSPACE)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_montjdouble_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/p256_montjmixadd.S view
@@ -0,0 +1,567 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Point mixed addition on NIST curve P-256 in Montgomery-Jacobian coordinates+//+//    extern void p256_montjmixadd(uint64_t p3[static 12],+//                                 const uint64_t p1[static 12],+//                                 const uint64_t p2[static 8]);+//+// Does p3 := p1 + p2 where all points are regarded as Jacobian triples with+// each coordinate in the Montgomery domain, i.e. x' = (2^256 * x) mod p_256.+// A Jacobian triple (x',y',z') represents affine point (x/z^2,y/z^3).+// The "mixed" part means that p2 only has x and y coordinates, with the+// implicit z coordinate assumed to be the identity.+//+// Standard x86-64 ABI: RDI = p3, RSI = p1, RDX = p2+// Microsoft x64 ABI:   RCX = p3, RDX = p1, R8 = p2+// ----------------------------------------------------------------------------+#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_montjmixadd)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_montjmixadd)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_montjmixadd)+        .text++// Size of individual field elements++#define NUMSIZE 32++// Pointer-offset pairs for inputs and outputs.+// These assume %rdi = p3, %rsi = p1 and %rbp = p2,+// which needs to be set up explicitly before use.+// The first two hold initially, and the second is+// set up by copying the initial %rdx input to %rbp.+// Thereafter, no code macro modifies any of them.++#define x_1 0(%rsi)+#define y_1 NUMSIZE(%rsi)+#define z_1 (2*NUMSIZE)(%rsi)++#define x_2 0(%rbp)+#define y_2 NUMSIZE(%rbp)++#define x_3 0(%rdi)+#define y_3 NUMSIZE(%rdi)+#define z_3 (2*NUMSIZE)(%rdi)++// Pointer-offset pairs for temporaries, with some aliasing.+// NSPACE is the total stack needed for all temporaries.++#define zp2 (NUMSIZE*0)(%rsp)+#define ww (NUMSIZE*0)(%rsp)+#define resx (NUMSIZE*0)(%rsp)++#define yd (NUMSIZE*1)(%rsp)+#define y2a (NUMSIZE*1)(%rsp)++#define x2a (NUMSIZE*2)(%rsp)+#define zzx2 (NUMSIZE*2)(%rsp)++#define zz (NUMSIZE*3)(%rsp)+#define t1 (NUMSIZE*3)(%rsp)++#define t2 (NUMSIZE*4)(%rsp)+#define zzx1 (NUMSIZE*4)(%rsp)+#define resy (NUMSIZE*4)(%rsp)++#define xd (NUMSIZE*5)(%rsp)+#define resz (NUMSIZE*5)(%rsp)++#define NSPACE NUMSIZE*6++// Corresponds exactly to bignum_montmul_p256++#define montmul_p256(P0,P1,P2)                  \+        xorl    %r13d, %r13d ;                      \+        movq    P2, %rdx ;                       \+        mulxq   P1, %r8, %r9 ;                     \+        mulxq   0x8+P1, %rbx, %r10 ;               \+        adcq    %rbx, %r9 ;                         \+        mulxq   0x10+P1, %rbx, %r11 ;              \+        adcq    %rbx, %r10 ;                        \+        mulxq   0x18+P1, %rbx, %r12 ;              \+        adcq    %rbx, %r11 ;                        \+        adcq    %r13, %r12 ;                        \+        movq    0x8+P2, %rdx ;                   \+        xorl    %r14d, %r14d ;                      \+        mulxq   P1, %rax, %rbx ;                   \+        adcxq   %rax, %r9 ;                         \+        adoxq   %rbx, %r10 ;                        \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        mulxq   0x10+P1, %rax, %rbx ;              \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   0x18+P1, %rax, %rbx ;              \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcq    %r14, %r13 ;                        \+        xorl    %r15d, %r15d ;                      \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r9 ;                         \+        adoxq   %rbx, %r10 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        notq    %rdx;                            \+        leaq    0x2(%rdx), %rdx ;                  \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcxq   %r15, %r13 ;                        \+        adoxq   %r15, %r14 ;                        \+        adcq    %r15, %r14 ;                        \+        movq    0x10+P2, %rdx ;                  \+        xorl    %r8d, %r8d ;                        \+        mulxq   P1, %rax, %rbx ;                   \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   0x10+P1, %rax, %rbx ;              \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adoxq   %r8, %r14 ;                         \+        mulxq   0x18+P1, %rax, %rbx ;              \+        adcq    %rax, %r13 ;                        \+        adcq    %rbx, %r14 ;                        \+        adcq    %r8, %r15 ;                         \+        movq    0x18+P2, %rdx ;                  \+        xorl    %r9d, %r9d ;                        \+        mulxq   P1, %rax, %rbx ;                   \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        mulxq   0x10+P1, %rax, %rbx ;              \+        adcxq   %rax, %r13 ;                        \+        adoxq   %rbx, %r14 ;                        \+        adoxq   %r9, %r15 ;                         \+        mulxq   0x18+P1, %rax, %rbx ;              \+        adcq    %rax, %r14 ;                        \+        adcq    %rbx, %r15 ;                        \+        adcq    %r9, %r8 ;                          \+        xorl    %r9d, %r9d ;                        \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        notq    %rdx;                            \+        leaq    0x2(%rdx), %rdx ;                  \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r13 ;                        \+        adoxq   %rbx, %r14 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r14 ;                        \+        adoxq   %rbx, %r15 ;                        \+        adcxq   %r9, %r15 ;                         \+        adoxq   %r9, %r8 ;                          \+        adcq    %r9, %r8 ;                          \+        movl    $0x1, %ecx ;                        \+        addq    %r12, %rcx ;                        \+        decq    %rdx;                            \+        adcq    %r13, %rdx ;                        \+        decq    %r9;                             \+        movq    %r9, %rax ;                         \+        adcq    %r14, %r9 ;                         \+        movl    $0xfffffffe, %r11d ;                \+        adcq    %r15, %r11 ;                        \+        adcq    %r8, %rax ;                         \+        cmovbq  %rcx, %r12 ;                        \+        cmovbq  %rdx, %r13 ;                        \+        cmovbq  %r9, %r14 ;                         \+        cmovbq  %r11, %r15 ;                        \+        movq    %r12, P0 ;                       \+        movq    %r13, 0x8+P0 ;                   \+        movq    %r14, 0x10+P0 ;                  \+        movq    %r15, 0x18+P0++// Corresponds exactly to bignum_montsqr_p256 except for+// register tweaks to avoid modifying %rbp.++#define montsqr_p256(P0,P1)                     \+        movq    P1, %rdx ;                       \+        mulxq   %rdx, %r8, %r15 ;                     \+        mulxq   0x8+P1, %r9, %r10 ;                \+        mulxq   0x18+P1, %r11, %r12 ;              \+        movq    0x10+P1, %rdx ;                  \+        mulxq   0x18+P1, %r13, %r14 ;              \+        xorl    %ecx, %ecx ;                        \+        mulxq   P1, %rax, %rbx ;                   \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        movq    0x18+P1, %rdx ;                  \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcxq   %rcx, %r13 ;                        \+        adoxq   %rcx, %r14 ;                        \+        adcq    %rcx, %r14 ;                        \+        xorl    %ecx, %ecx ;                        \+        adcxq   %r9, %r9 ;                          \+        adoxq   %r15, %r9 ;                         \+        movq    0x8+P1, %rdx ;                   \+        mulxq   %rdx, %rax, %rdx ;                    \+        adcxq   %r10, %r10 ;                        \+        adoxq   %rax, %r10 ;                        \+        adcxq   %r11, %r11 ;                        \+        adoxq   %rdx, %r11 ;                        \+        movq    0x10+P1, %rdx ;                  \+        mulxq   %rdx, %rax, %rdx ;                    \+        adcxq   %r12, %r12 ;                        \+        adoxq   %rax, %r12 ;                        \+        adcxq   %r13, %r13 ;                        \+        adoxq   %rdx, %r13 ;                        \+        movq    0x18+P1, %rdx ;                  \+        mulxq   %rdx, %rax, %r15 ;                    \+        adcxq   %r14, %r14 ;                        \+        adoxq   %rax, %r14 ;                        \+        adcxq   %rcx, %r15 ;                        \+        adoxq   %rcx, %r15 ;                        \+        xorl    %ecx, %ecx ;                        \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r9 ;                         \+        adoxq   %rbx, %r10 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        movabsq $0xffffffff00000001, %rdx ;         \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcxq   %rcx, %r13 ;                        \+        movl    %ecx, %r9d ;                        \+        adoxq   %rcx, %r9 ;                         \+        adcxq   %rcx, %r9 ;                         \+        addq    %r9, %r14 ;                         \+        adcq    %rcx, %r15 ;                        \+        movl    %ecx, %r8d ;                        \+        adcq    %rcx, %r8 ;                         \+        xorl    %ecx, %ecx ;                        \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        movabsq $0xffffffff00000001, %rdx ;         \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r13 ;                        \+        adoxq   %rbx, %r14 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r14 ;                        \+        adoxq   %rbx, %r15 ;                        \+        adcxq   %rcx, %r15 ;                        \+        adoxq   %rcx, %r8 ;                         \+        adcq    %rcx, %r8 ;                         \+        movl    $0x1, %ebx ;                        \+        addq    %r12, %rbx ;                        \+        leaq    -0x1(%rdx), %rdx ;                  \+        adcq    %r13, %rdx ;                        \+        leaq    -0x1(%rcx), %rcx ;                  \+        movq    %rcx, %rax ;                        \+        adcq    %r14, %rcx ;                        \+        movl    $0xfffffffe, %r11d ;                \+        adcq    %r15, %r11 ;                        \+        adcq    %r8, %rax ;                         \+        cmovbq  %rbx, %r12 ;                        \+        cmovbq  %rdx, %r13 ;                        \+        cmovbq  %rcx, %r14 ;                        \+        cmovbq  %r11, %r15 ;                        \+        movq    %r12, P0 ;                       \+        movq    %r13, 0x8+P0 ;                   \+        movq    %r14, 0x10+P0 ;                  \+        movq    %r15, 0x18+P0++// Almost-Montgomery variant which we use when an input to other muls+// with the other argument fully reduced (which is always safe).+// Again, the basic squaring code is tweaked to avoid modifying %rbp.++#define amontsqr_p256(P0,P1)                    \+        movq    P1, %rdx ;                       \+        mulxq   %rdx, %r8, %r15 ;                     \+        mulxq   0x8+P1, %r9, %r10 ;                \+        mulxq   0x18+P1, %r11, %r12 ;              \+        movq    0x10+P1, %rdx ;                  \+        mulxq   0x18+P1, %r13, %r14 ;              \+        xorl    %ecx, %ecx ;                        \+        mulxq   P1, %rax, %rbx ;                   \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        movq    0x18+P1, %rdx ;                  \+        mulxq   0x8+P1, %rax, %rbx ;               \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcxq   %rcx, %r13 ;                        \+        adoxq   %rcx, %r14 ;                        \+        adcq    %rcx, %r14 ;                        \+        xorl    %ecx, %ecx ;                        \+        adcxq   %r9, %r9 ;                          \+        adoxq   %r15, %r9 ;                         \+        movq    0x8+P1, %rdx ;                   \+        mulxq   %rdx, %rax, %rdx ;                    \+        adcxq   %r10, %r10 ;                        \+        adoxq   %rax, %r10 ;                        \+        adcxq   %r11, %r11 ;                        \+        adoxq   %rdx, %r11 ;                        \+        movq    0x10+P1, %rdx ;                  \+        mulxq   %rdx, %rax, %rdx ;                    \+        adcxq   %r12, %r12 ;                        \+        adoxq   %rax, %r12 ;                        \+        adcxq   %r13, %r13 ;                        \+        adoxq   %rdx, %r13 ;                        \+        movq    0x18+P1, %rdx ;                  \+        mulxq   %rdx, %rax, %r15 ;                    \+        adcxq   %r14, %r14 ;                        \+        adoxq   %rax, %r14 ;                        \+        adcxq   %rcx, %r15 ;                        \+        adoxq   %rcx, %r15 ;                        \+        xorl    %ecx, %ecx ;                        \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r9 ;                         \+        adoxq   %rbx, %r10 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r10 ;                        \+        adoxq   %rbx, %r11 ;                        \+        movabsq $0xffffffff00000001, %rdx ;         \+        mulxq   %r8, %rax, %rbx ;                     \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r9, %rax, %rbx ;                     \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        adcxq   %rcx, %r13 ;                        \+        movl    %ecx, %r9d ;                        \+        adoxq   %rcx, %r9 ;                         \+        adcxq   %rcx, %r9 ;                         \+        addq    %r9, %r14 ;                         \+        adcq    %rcx, %r15 ;                        \+        movl    %ecx, %r8d ;                        \+        adcq    %rcx, %r8 ;                         \+        xorl    %ecx, %ecx ;                        \+        movabsq $0x100000000, %rdx ;                \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r11 ;                        \+        adoxq   %rbx, %r12 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r12 ;                        \+        adoxq   %rbx, %r13 ;                        \+        movabsq $0xffffffff00000001, %rdx ;         \+        mulxq   %r10, %rax, %rbx ;                    \+        adcxq   %rax, %r13 ;                        \+        adoxq   %rbx, %r14 ;                        \+        mulxq   %r11, %rax, %rbx ;                    \+        adcxq   %rax, %r14 ;                        \+        adoxq   %rbx, %r15 ;                        \+        adcxq   %rcx, %r15 ;                        \+        adoxq   %rcx, %r8 ;                         \+        adcq    %rcx, %r8 ;                         \+        movl    $0x1, %r8d ;                        \+        leaq    -0x1(%rdx), %rdx ;                  \+        leaq    -0x1(%rcx), %rax ;                  \+        movl    $0xfffffffe, %r11d ;                \+        cmovzq  %rcx, %r8 ;                         \+        cmovzq  %rcx, %rdx ;                        \+        cmovzq  %rcx, %rax ;                        \+        cmovzq  %rcx, %r11 ;                        \+        addq    %r8, %r12 ;                         \+        adcq    %rdx, %r13 ;                        \+        adcq    %rax, %r14 ;                        \+        adcq    %r11, %r15 ;                        \+        movq    %r12, P0 ;                       \+        movq    %r13, 0x8+P0 ;                   \+        movq    %r14, 0x10+P0 ;                  \+        movq    %r15, 0x18+P0++// Corresponds exactly to bignum_sub_p256++#define sub_p256(P0,P1,P2)                      \+        movq    P1, %rax ;                       \+        subq    P2, %rax ;                       \+        movq    0x8+P1, %rcx ;                   \+        sbbq    0x8+P2, %rcx ;                   \+        movq    0x10+P1, %r8 ;                   \+        sbbq    0x10+P2, %r8 ;                   \+        movq    0x18+P1, %r9 ;                   \+        sbbq    0x18+P2, %r9 ;                   \+        movl    $0xffffffff, %r10d ;                \+        sbbq    %r11, %r11 ;                        \+        xorq    %rdx, %rdx ;                        \+        andq    %r11, %r10 ;                        \+        subq    %r10, %rdx ;                        \+        addq    %r11, %rax ;                        \+        movq    %rax, P0 ;                       \+        adcq    %r10, %rcx ;                        \+        movq    %rcx, 0x8+P0 ;                   \+        adcq    $0x0, %r8 ;                         \+        movq    %r8, 0x10+P0 ;                   \+        adcq    %rdx, %r9 ;                         \+        movq    %r9, 0x18+P0++// Additional macros to help with final multiplexing++#define testzero4(P)                            \+        movq    P, %rax ;                       \+        movq    8+P, %rdx ;                     \+        orq     16+P, %rax ;                    \+        orq     24+P, %rdx ;                    \+        orq     %rdx, %rax++#define mux4(r0,r1,r2,r3,PNE,PEQ)               \+        movq    PNE, r0 ;                      \+        movq    PEQ, %rax ;                     \+        cmovzq  %rax, r0 ;                        \+        movq    8+PNE, r1 ;                    \+        movq    8+PEQ, %rax ;                   \+        cmovzq  %rax, r1 ;                        \+        movq    16+PNE, r2 ;                   \+        movq    16+PEQ, %rax ;                  \+        cmovzq  %rax, r2 ;                        \+        movq    24+PNE, r3 ;                   \+        movq    24+PEQ, %rax ;                  \+        cmovzq  %rax, r3++#define load4(r0,r1,r2,r3,P)                    \+        movq    P, r0 ;                        \+        movq    8+P, r1 ;                      \+        movq    16+P, r2 ;                     \+        movq    24+P, r3++#define store4(P,r0,r1,r2,r3)                   \+        movq    r0, P ;                        \+        movq    r1, 8+P ;                      \+        movq    r2, 16+P ;                     \+        movq    r3, 24+P++S2N_BN_SYMBOL(p256_montjmixadd):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+#endif++// Save registers and make room on stack for temporary variables+// Put the input y in %rbp where it lasts as long as it's needed.++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++        CFI_DEC_RSP(NSPACE)++        movq    %rdx, %rbp++// Main code, just a sequence of basic field operations+// 8 * multiply + 3 * square + 7 * subtract++        amontsqr_p256(zp2,z_1)++        montmul_p256(y2a,z_1,y_2)+        montmul_p256(x2a,zp2,x_2)+        montmul_p256(y2a,zp2,y2a)++        sub_p256(xd,x2a,x_1)++        sub_p256(yd,y2a,y_1)++        amontsqr_p256(zz,xd)+        montsqr_p256(ww,yd)++        montmul_p256(zzx1,zz,x_1)+        montmul_p256(zzx2,zz,x2a)++        sub_p256(resx,ww,zzx1)+        sub_p256(t1,zzx2,zzx1)++        montmul_p256(resz,xd,z_1)++        sub_p256(resx,resx,zzx2)++        sub_p256(t2,zzx1,resx)++        montmul_p256(t1,t1,y_1)+        montmul_p256(t2,yd,t2)++        sub_p256(resy,t2,t1)++// Test if z_1 = 0 to decide if p1 = 0 (up to projective equivalence)++        testzero4(z_1)++// Multiplex: if p1 <> 0 just copy the computed result from the staging area.+// If p1 = 0 then return the point p2 augmented with a z = 1 coordinate (in+// Montgomery form so not the simple constant 1 but rather 2^256 - p_256),+// hence giving 0 + p2 = p2 for the final result.++        mux4(%r8,%r9,%r10,%r11,resx,x_2)+        mux4(%r12,%r13,%r14,%r15,resy,y_2)++        store4(x_3,%r8,%r9,%r10,%r11)+        store4(y_3,%r12,%r13,%r14,%r15)++        load4(%r8,%r9,%r10,%r11,resz)+        movl    $1, %eax+        cmovzq  %rax, %r8+        movq    $0xffffffff00000000, %rax+        cmovzq  %rax, %r9+        movq    $0xffffffffffffffff, %rax+        cmovzq  %rax, %r10+        movl    $0x00000000fffffffe, %eax+        cmovzq  %rax, %r11++        store4(z_3,%r8,%r9,%r10,%r11)++// Restore stack and registers++        CFI_INC_RSP(NSPACE)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_montjmixadd)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/p256_montjmixadd_alt.S view
@@ -0,0 +1,552 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Point mixed addition on NIST curve P-256 in Montgomery-Jacobian coordinates+//+//    extern void p256_montjmixadd_alt(uint64_t p3[static 12],+//                                     const uint64_t p1[static 12],+//                                     const uint64_t p2[static 8]);+//+// Does p3 := p1 + p2 where all points are regarded as Jacobian triples with+// each coordinate in the Montgomery domain, i.e. x' = (2^256 * x) mod p_256.+// A Jacobian triple (x',y',z') represents affine point (x/z^2,y/z^3).+// The "mixed" part means that p2 only has x and y coordinates, with the+// implicit z coordinate assumed to be the identity.+//+// Standard x86-64 ABI: RDI = p3, RSI = p1, RDX = p2+// Microsoft x64 ABI:   RCX = p3, RDX = p1, R8 = p2+// ----------------------------------------------------------------------------+#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_montjmixadd_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_montjmixadd_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_montjmixadd_alt)+        .text++// Size of individual field elements++#define NUMSIZE 32++// Pointer-offset pairs for inputs and outputs.+// These assume %rdi = p3, %rsi = p1 and %rbp = p2,+// which needs to be set up explicitly before use.+// The first two hold initially, and the second is+// set up by copying the initial %rdx input to %rbp.+// Thereafter, no code macro modifies any of them.++#define x_1 0(%rsi)+#define y_1 NUMSIZE(%rsi)+#define z_1 (2*NUMSIZE)(%rsi)++#define x_2 0(%rbp)+#define y_2 NUMSIZE(%rbp)++#define x_3 0(%rdi)+#define y_3 NUMSIZE(%rdi)+#define z_3 (2*NUMSIZE)(%rdi)++// Pointer-offset pairs for temporaries, with some aliasing.+// NSPACE is the total stack needed for all temporaries.++#define zp2 (NUMSIZE*0)(%rsp)+#define ww (NUMSIZE*0)(%rsp)+#define resx (NUMSIZE*0)(%rsp)++#define yd (NUMSIZE*1)(%rsp)+#define y2a (NUMSIZE*1)(%rsp)++#define x2a (NUMSIZE*2)(%rsp)+#define zzx2 (NUMSIZE*2)(%rsp)++#define zz (NUMSIZE*3)(%rsp)+#define t1 (NUMSIZE*3)(%rsp)++#define t2 (NUMSIZE*4)(%rsp)+#define zzx1 (NUMSIZE*4)(%rsp)+#define resy (NUMSIZE*4)(%rsp)++#define xd (NUMSIZE*5)(%rsp)+#define resz (NUMSIZE*5)(%rsp)++#define NSPACE NUMSIZE*6++// Corresponds exactly to bignum_montmul_p256_alt++#define montmul_p256(P0,P1,P2)                  \+        movq    P2, %rbx ;                      \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        movq    %rax, %r8 ;                        \+        movq    %rdx, %r9 ;                        \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        xorl    %r10d, %r10d ;                     \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        movq    0x10+P1, %rax ;                 \+        mulq    %rbx;                            \+        xorl    %r11d, %r11d ;                     \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        movq    0x18+P1, %rax ;                 \+        mulq    %rbx;                            \+        xorl    %r12d, %r12d ;                     \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        movq    0x8+P2, %rbx ;                  \+        xorl    %r13d, %r13d ;                     \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        sbbq    %r14, %r14 ;                       \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %r14, %rdx ;                       \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %r14, %r14 ;                       \+        movq    0x10+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r14, %rdx ;                       \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %r14, %r14 ;                       \+        movq    0x18+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r14, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        xorl    %r14d, %r14d ;                     \+        movq    $0x100000000, %rbx ;               \+        movq    %r8, %rax ;                        \+        mulq    %rbx;                            \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        sbbq    %r15, %r15 ;                       \+        movq    %r9, %rax ;                        \+        mulq    %rbx;                            \+        subq    %r15, %rdx ;                       \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %r15, %r15 ;                       \+        notq    %rbx;                            \+        leaq    0x2(%rbx), %rbx ;                 \+        movq    %r8, %rax ;                        \+        mulq    %rbx;                            \+        subq    %r15, %rdx ;                       \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %r15, %r15 ;                       \+        movq    %r9, %rax ;                        \+        mulq    %rbx;                            \+        subq    %r15, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        adcq    %r14, %r14 ;                       \+        movq    0x10+P2, %rbx ;                 \+        xorl    %r15d, %r15d ;                     \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %r8, %r8 ;                         \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %r8, %rdx ;                        \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %r8, %r8 ;                         \+        movq    0x10+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r8, %rdx ;                        \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %r8, %r8 ;                         \+        movq    0x18+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r8, %rdx ;                        \+        addq    %rax, %r13 ;                       \+        adcq    %rdx, %r14 ;                       \+        adcq    %r15, %r15 ;                       \+        movq    0x18+P2, %rbx ;                 \+        xorl    %r8d, %r8d ;                       \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %r9, %r9 ;                         \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %r9, %rdx ;                        \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %r9, %r9 ;                         \+        movq    0x10+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r9, %rdx ;                        \+        addq    %rax, %r13 ;                       \+        adcq    %rdx, %r14 ;                       \+        sbbq    %r9, %r9 ;                         \+        movq    0x18+P1, %rax ;                 \+        mulq    %rbx;                            \+        subq    %r9, %rdx ;                        \+        addq    %rax, %r14 ;                       \+        adcq    %rdx, %r15 ;                       \+        adcq    %r8, %r8 ;                         \+        xorl    %r9d, %r9d ;                       \+        movq    $0x100000000, %rbx ;               \+        movq    %r10, %rax ;                       \+        mulq    %rbx;                            \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    %r11, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %rcx, %rcx ;                       \+        notq    %rbx;                            \+        leaq    0x2(%rbx), %rbx ;                 \+        movq    %r10, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r13 ;                       \+        adcq    %rdx, %r14 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    %r11, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r14 ;                       \+        adcq    %rdx, %r15 ;                       \+        adcq    %r9, %r8 ;                         \+        movl    $0x1, %ecx ;                       \+        addq    %r12, %rcx ;                       \+        decq    %rbx;                            \+        adcq    %r13, %rbx ;                       \+        decq    %r9;                             \+        movq    %r9, %rax ;                        \+        adcq    %r14, %r9 ;                        \+        movl    $0xfffffffe, %r11d ;               \+        adcq    %r15, %r11 ;                       \+        adcq    %r8, %rax ;                        \+        cmovbq  %rcx, %r12 ;                       \+        cmovbq  %rbx, %r13 ;                       \+        cmovbq  %r9, %r14 ;                        \+        cmovbq  %r11, %r15 ;                       \+        movq    %r12, P0 ;                      \+        movq    %r13, 0x8+P0 ;                  \+        movq    %r14, 0x10+P0 ;                 \+        movq    %r15, 0x18+P0++// Corresponds exactly to bignum_montsqr_p256_alt++#define montsqr_p256(P0,P1)                     \+        movq    P1, %rax ;                      \+        movq    %rax, %rbx ;                       \+        mulq    %rax;                            \+        movq    %rax, %r8 ;                        \+        movq    %rdx, %r15 ;                       \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        movq    %rax, %r9 ;                        \+        movq    %rdx, %r10 ;                       \+        movq    0x18+P1, %rax ;                 \+        movq    %rax, %r13 ;                       \+        mulq    %rbx;                            \+        movq    %rax, %r11 ;                       \+        movq    %rdx, %r12 ;                       \+        movq    0x10+P1, %rax ;                 \+        movq    %rax, %rbx ;                       \+        mulq    %r13;                            \+        movq    %rax, %r13 ;                       \+        movq    %rdx, %r14 ;                       \+        movq    P1, %rax ;                      \+        mulq    %rbx;                            \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    0x18+P1, %rbx ;                 \+        movq    0x8+P1, %rax ;                  \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        adcq    $0x0, %r14 ;                       \+        xorl    %ecx, %ecx ;                       \+        addq    %r9, %r9 ;                         \+        adcq    %r10, %r10 ;                       \+        adcq    %r11, %r11 ;                       \+        adcq    %r12, %r12 ;                       \+        adcq    %r13, %r13 ;                       \+        adcq    %r14, %r14 ;                       \+        adcq    %rcx, %rcx ;                       \+        movq    0x8+P1, %rax ;                  \+        mulq    %rax;                            \+        addq    %r15, %r9 ;                        \+        adcq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %r15, %r15 ;                       \+        movq    0x10+P1, %rax ;                 \+        mulq    %rax;                            \+        negq    %r15;                            \+        adcq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %r15, %r15 ;                       \+        movq    0x18+P1, %rax ;                 \+        mulq    %rax;                            \+        negq    %r15;                            \+        adcq    %rax, %r14 ;                       \+        adcq    %rcx, %rdx ;                       \+        movq    %rdx, %r15 ;                       \+        movq    $0x100000000, %rbx ;               \+        movq    %r8, %rax ;                        \+        mulq    %rbx;                            \+        addq    %rax, %r9 ;                        \+        adcq    %rdx, %r10 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    %r9, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r10 ;                       \+        adcq    %rdx, %r11 ;                       \+        sbbq    %rcx, %rcx ;                       \+        notq    %rbx;                            \+        leaq    0x2(%rbx), %rbx ;                 \+        movq    %r8, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %rcx, %rcx ;                       \+        xorl    %r8d, %r8d ;                       \+        movq    %r9, %rax ;                        \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        adcq    %r8, %r14 ;                        \+        adcq    %r8, %r15 ;                        \+        adcq    %r8, %r8 ;                         \+        movq    $0x100000000, %rbx ;               \+        movq    %r10, %rax ;                       \+        mulq    %rbx;                            \+        addq    %rax, %r11 ;                       \+        adcq    %rdx, %r12 ;                       \+        sbbq    %rcx, %rcx ;                       \+        movq    %r11, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r12 ;                       \+        adcq    %rdx, %r13 ;                       \+        sbbq    %rcx, %rcx ;                       \+        notq    %rbx;                            \+        leaq    0x2(%rbx), %rbx ;                 \+        movq    %r10, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r13 ;                       \+        adcq    %rdx, %r14 ;                       \+        sbbq    %rcx, %rcx ;                       \+        xorl    %r9d, %r9d ;                       \+        movq    %r11, %rax ;                       \+        mulq    %rbx;                            \+        subq    %rcx, %rdx ;                       \+        addq    %rax, %r14 ;                       \+        adcq    %rdx, %r15 ;                       \+        adcq    %r9, %r8 ;                         \+        movl    $0x1, %ecx ;                       \+        addq    %r12, %rcx ;                       \+        leaq    -0x1(%rbx), %rbx ;                 \+        adcq    %r13, %rbx ;                       \+        leaq    -0x1(%r9), %r9 ;                   \+        movq    %r9, %rax ;                        \+        adcq    %r14, %r9 ;                        \+        movl    $0xfffffffe, %r11d ;               \+        adcq    %r15, %r11 ;                       \+        adcq    %r8, %rax ;                        \+        cmovbq  %rcx, %r12 ;                       \+        cmovbq  %rbx, %r13 ;                       \+        cmovbq  %r9, %r14 ;                        \+        cmovbq  %r11, %r15 ;                       \+        movq    %r12, P0 ;                      \+        movq    %r13, 0x8+P0 ;                  \+        movq    %r14, 0x10+P0 ;                 \+        movq    %r15, 0x18+P0++// Corresponds exactly to bignum_sub_p256++#define sub_p256(P0,P1,P2)                      \+        movq    P1, %rax ;                       \+        subq    P2, %rax ;                       \+        movq    0x8+P1, %rcx ;                   \+        sbbq    0x8+P2, %rcx ;                   \+        movq    0x10+P1, %r8 ;                   \+        sbbq    0x10+P2, %r8 ;                   \+        movq    0x18+P1, %r9 ;                   \+        sbbq    0x18+P2, %r9 ;                   \+        movl    $0xffffffff, %r10d ;                \+        sbbq    %r11, %r11 ;                        \+        xorq    %rdx, %rdx ;                        \+        andq    %r11, %r10 ;                        \+        subq    %r10, %rdx ;                        \+        addq    %r11, %rax ;                        \+        movq    %rax, P0 ;                       \+        adcq    %r10, %rcx ;                        \+        movq    %rcx, 0x8+P0 ;                   \+        adcq    $0x0, %r8 ;                         \+        movq    %r8, 0x10+P0 ;                   \+        adcq    %rdx, %r9 ;                         \+        movq    %r9, 0x18+P0++// Additional macros to help with final multiplexing++#define testzero4(P)                            \+        movq    P, %rax ;                       \+        movq    8+P, %rdx ;                     \+        orq     16+P, %rax ;                    \+        orq     24+P, %rdx ;                    \+        orq     %rdx, %rax++#define mux4(r0,r1,r2,r3,PNE,PEQ)               \+        movq    PNE, r0 ;                      \+        movq    PEQ, %rax ;                     \+        cmovzq  %rax, r0 ;                        \+        movq    8+PNE, r1 ;                    \+        movq    8+PEQ, %rax ;                   \+        cmovzq  %rax, r1 ;                        \+        movq    16+PNE, r2 ;                   \+        movq    16+PEQ, %rax ;                  \+        cmovzq  %rax, r2 ;                        \+        movq    24+PNE, r3 ;                   \+        movq    24+PEQ, %rax ;                  \+        cmovzq  %rax, r3++#define load4(r0,r1,r2,r3,P)                    \+        movq    P, r0 ;                        \+        movq    8+P, r1 ;                      \+        movq    16+P, r2 ;                     \+        movq    24+P, r3++#define store4(P,r0,r1,r2,r3)                   \+        movq    r0, P ;                        \+        movq    r1, 8+P ;                      \+        movq    r2, 16+P ;                     \+        movq    r3, 24+P++S2N_BN_SYMBOL(p256_montjmixadd_alt):+        CFI_START+        _CET_ENDBR++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+#endif++// Save registers and make room on stack for temporary variables+// Put the input y in %rbp where it lasts as long as it's needed.++        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)++        CFI_DEC_RSP(NSPACE)++        movq    %rdx, %rbp++// Main code, just a sequence of basic field operations+// 8 * multiply + 3 * square + 7 * subtract++        montsqr_p256(zp2,z_1)++        montmul_p256(y2a,z_1,y_2)+        montmul_p256(x2a,zp2,x_2)+        montmul_p256(y2a,zp2,y2a)++        sub_p256(xd,x2a,x_1)++        sub_p256(yd,y2a,y_1)++        montsqr_p256(zz,xd)+        montsqr_p256(ww,yd)++        montmul_p256(zzx1,zz,x_1)+        montmul_p256(zzx2,zz,x2a)++        sub_p256(resx,ww,zzx1)+        sub_p256(t1,zzx2,zzx1)++        montmul_p256(resz,xd,z_1)++        sub_p256(resx,resx,zzx2)++        sub_p256(t2,zzx1,resx)++        montmul_p256(t1,t1,y_1)+        montmul_p256(t2,yd,t2)++        sub_p256(resy,t2,t1)++// Test if z_1 = 0 to decide if p1 = 0 (up to projective equivalence)++        testzero4(z_1)++// Multiplex: if p1 <> 0 just copy the computed result from the staging area.+// If p1 = 0 then return the point p2 augmented with a z = 1 coordinate (in+// Montgomery form so not the simple constant 1 but rather 2^256 - p_256),+// hence giving 0 + p2 = p2 for the final result.++        mux4(%r8,%r9,%r10,%r11,resx,x_2)+        mux4(%r12,%r13,%r14,%r15,resy,y_2)++        store4(x_3,%r8,%r9,%r10,%r11)+        store4(y_3,%r12,%r13,%r14,%r15)++        load4(%r8,%r9,%r10,%r11,resz)+        movl    $1, %eax+        cmovzq  %rax, %r8+        movq    $0xffffffff00000000, %rax+        cmovzq  %rax, %r9+        movq    $0xffffffffffffffff, %rax+        cmovzq  %rax, %r10+        movl    $0x00000000fffffffe, %eax+        cmovzq  %rax, %r11++        store4(z_3,%r8,%r9,%r10,%r11)++// Restore stack and registers++        CFI_INC_RSP(NSPACE)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)++#if WINDOWS_ABI+        CFI_POP(%rsi)+        CFI_POP(%rdi)+#endif+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_montjmixadd_alt)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/p256_scalarmul.S view
@@ -0,0 +1,6823 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Scalar multiplication for P-256+// Input scalar[4], point[8]; output res[8]+//+// extern void p256_scalarmul+//   (uint64_t res[static 8],const uint64_t scalar[static 4],+//     const uint64_t point[static 8]);+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-256, returns the point (X,Y) = n * P. The input and output+// are affine points, and in the case of the point at infinity as+// the result, (0,0) is returned.+//+// Standard x86-64 ABI: RDI = res, RSI = scalar, RDX = point+// Microsoft x64 ABI:   RCX = res, RDX = scalar, R8 = point+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_scalarmul)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_scalarmul)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_scalarmul)++        .text+        .balign 32++// Size of individual field elements++#define NUMSIZE 32++// Intermediate variables on the stack. The last z2, z3 values can+// safely be overlaid on the table, which is no longer needed at the end.+// Uppercase syntactic variants make x86_att version simpler to generate++#define SCALARB (0*NUMSIZE)+#define scalarb (0*NUMSIZE)(%rsp)+#define ACC (1*NUMSIZE)+#define acc (1*NUMSIZE)(%rsp)+#define TABENT (4*NUMSIZE)+#define tabent (4*NUMSIZE)(%rsp)++#define TAB (7*NUMSIZE)+#define tab (7*NUMSIZE)(%rsp)++#define Z2 (7*NUMSIZE)+#define z2 (7*NUMSIZE)(%rsp)+#define Z3 (8*NUMSIZE)+#define z3 (8*NUMSIZE)(%rsp)++#define res (31*NUMSIZE)(%rsp)++#define NSPACE 32*NUMSIZE++S2N_BN_SYMBOL(p256_scalarmul):+        CFI_START+        _CET_ENDBR++// The Windows version literally calls the standard ABI version.+// This simplifies the proofs since subroutine offsets are fixed.++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+        CFI_CALL(Lp256_scalarmul_standard)+        CFI_POP(%rsi)+        CFI_POP(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_scalarmul)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_standard)++Lp256_scalarmul_standard:+        CFI_START+#endif++// Real start of the standard ABI code.++        CFI_PUSH(%r15)+        CFI_PUSH(%r14)+        CFI_PUSH(%r13)+        CFI_PUSH(%r12)+        CFI_PUSH(%rbp)+        CFI_PUSH(%rbx)++        CFI_DEC_RSP(NSPACE)++// Preserve the "res" and "point" input arguments. We load and process the+// scalar immediately so we don't bother preserving that input argument.+// Also, "point" is only needed early on and so its register gets re-used.++        movq    %rdx, %rbx+        movq    %rdi, res++// Load the digits of group order n_256 = [%r15;%r14;%r13;%r12]++        movq    $0xf3b9cac2fc632551, %r12+        movq    $0xbce6faada7179e84, %r13+        movq    $0xffffffffffffffff, %r14+        movq    $0xffffffff00000000, %r15++// First, reduce the input scalar mod n_256, i.e. conditionally subtract n_256++        movq    (%rsi), %r8+        subq    %r12, %r8+        movq    8(%rsi), %r9+        sbbq    %r13, %r9+        movq    16(%rsi), %r10+        sbbq    %r14, %r10+        movq    24(%rsi), %r11+        sbbq    %r15, %r11++        cmovcq  (%rsi), %r8+        cmovcq  8(%rsi), %r9+        cmovcq  16(%rsi), %r10+        cmovcq  24(%rsi), %r11++// Now if the top bit of the reduced scalar is set, negate it mod n_256,+// i.e. do n |-> n_256 - n. Remember the sign in %rbp so we can+// correspondingly negate the point below.++        subq    %r8, %r12+        sbbq    %r9, %r13+        sbbq    %r10, %r14+        sbbq    %r11, %r15++        movq    %r11, %rbp+        shrq    $63, %rbp+        cmovnzq %r12, %r8+        cmovnzq %r13, %r9+        cmovnzq %r14, %r10+        cmovnzq %r15, %r11++// In either case then add the recoding constant 0x08888...888 to allow+// signed digits.++        movq    $0x8888888888888888, %rax+        addq    %rax, %r8+        adcq    %rax, %r9+        adcq    %rax, %r10+        adcq    %rax, %r11+        btc     $63, %r11++        movq    %r8, SCALARB(%rsp)+        movq    %r9, SCALARB+8(%rsp)+        movq    %r10, SCALARB+16(%rsp)+        movq    %r11, SCALARB+24(%rsp)++// Set the tab[0] table entry to Montgomery-Jacobian point = 1 * P+// The z coordinate is just the Montgomery form of the constant 1.++        leaq    TAB(%rsp), %rdi+        movq    %rbx, %rsi+        CFI_CALL(Lp256_scalarmul_local_tomont_p256)++        leaq    32(%rbx), %rsi+        leaq    TAB+32(%rsp), %rdi+        CFI_CALL(Lp256_scalarmul_local_tomont_p256)++        movl    $1, %eax+        movq    %rax, TAB+64(%rsp)+        movq    $0xffffffff00000000, %rdx+        movq    %rdx, TAB+72(%rsp)+        subq    $2, %rax+        movq    %rax, TAB+80(%rsp)+        movq    $0x00000000fffffffe, %rax+        movq    %rax, TAB+88(%rsp)++// If the top bit of the scalar was set, negate (y coordinate of) the point++        movq    TAB+32(%rsp), %r12+        movq    TAB+40(%rsp), %r13+        movq    TAB+48(%rsp), %r14+        movq    TAB+56(%rsp), %r15++        xorl    %r10d, %r10d+        leaq    -1(%r10), %r8+        movq    $0x00000000ffffffff, %r11+        movq    %r11, %r9+        negq    %r11++        subq    %r12, %r8+        sbbq    %r13, %r9+        sbbq    %r14, %r10+        sbbq    %r15, %r11++        testq   %rbp, %rbp+        cmovzq  %r12, %r8+        cmovzq  %r13, %r9+        cmovzq  %r14, %r10+        cmovzq  %r15, %r11++        movq    %r8, TAB+32(%rsp)+        movq    %r9, TAB+40(%rsp)+        movq    %r10, TAB+48(%rsp)+        movq    %r11, TAB+56(%rsp)++// Compute and record tab[1] = 2 * p, ..., tab[7] = 8 * P++        leaq    TAB+96*1(%rsp), %rdi+        leaq    TAB(%rsp), %rsi+        CFI_CALL(Lp256_scalarmul_local_p256_montjdouble)++        leaq    TAB+96*2(%rsp), %rdi+        leaq    TAB+96*1(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp256_scalarmul_local_p256_montjmixadd)++        leaq    TAB+96*3(%rsp), %rdi+        leaq    TAB+96*1(%rsp), %rsi+        CFI_CALL(Lp256_scalarmul_local_p256_montjdouble)++        leaq    TAB+96*4(%rsp), %rdi+        leaq    TAB+96*3(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp256_scalarmul_local_p256_montjmixadd)++        leaq    TAB+96*5(%rsp), %rdi+        leaq    TAB+96*2(%rsp), %rsi+        CFI_CALL(Lp256_scalarmul_local_p256_montjdouble)++        leaq    TAB+96*6(%rsp), %rdi+        leaq    TAB+96*5(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp256_scalarmul_local_p256_montjmixadd)++        leaq    TAB+96*7(%rsp), %rdi+        leaq    TAB+96*3(%rsp), %rsi+        CFI_CALL(Lp256_scalarmul_local_p256_montjdouble)++// Set up accumulator as table entry for top 4 bits (constant-time indexing)++        movq    SCALARB+24(%rsp), %rdi+        shrq    $60, %rdi++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        xorl    %r13d, %r13d+        xorl    %r14d, %r14d+        xorl    %r15d, %r15d++        .set I, 1+.rep 8+        cmpq    $I, %rdi++        cmovzq  TAB+96*(I-1)(%rsp), %rax+        cmovzq  TAB+96*(I-1)+8(%rsp), %rbx+        cmovzq  TAB+96*(I-1)+16(%rsp), %rcx+        cmovzq  TAB+96*(I-1)+24(%rsp), %rdx+        cmovzq  TAB+96*(I-1)+32(%rsp), %r8+        cmovzq  TAB+96*(I-1)+40(%rsp), %r9+        cmovzq  TAB+96*(I-1)+48(%rsp), %r10+        cmovzq  TAB+96*(I-1)+56(%rsp), %r11+        cmovzq  TAB+96*(I-1)+64(%rsp), %r12+        cmovzq  TAB+96*(I-1)+72(%rsp), %r13+        cmovzq  TAB+96*(I-1)+80(%rsp), %r14+        cmovzq  TAB+96*(I-1)+88(%rsp), %r15+        .set    I, (I+1)+.endr+        movq     %rax, ACC(%rsp)+        movq     %rbx, ACC+8(%rsp)+        movq     %rcx, ACC+16(%rsp)+        movq     %rdx, ACC+24(%rsp)+        movq     %r8, ACC+32(%rsp)+        movq     %r9, ACC+40(%rsp)+        movq     %r10, ACC+48(%rsp)+        movq     %r11, ACC+56(%rsp)+        movq     %r12, ACC+64(%rsp)+        movq     %r13, ACC+72(%rsp)+        movq     %r14, ACC+80(%rsp)+        movq     %r15, ACC+88(%rsp)++// Main loop over size-4 bitfield++        movl    $252, %ebp++Lp256_scalarmul_loop:+        subq    $4, %rbp++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp256_scalarmul_local_p256_montjdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp256_scalarmul_local_p256_montjdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp256_scalarmul_local_p256_montjdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp256_scalarmul_local_p256_montjdouble)++        movq    %rbp, %rax+        shrq    $6, %rax+        movq    (%rsp,%rax,8), %rdi+        movq    %rbp, %rcx+        shrq    %cl, %rdi+        andq    $15, %rdi++        subq    $8, %rdi+        sbbq    %rsi, %rsi // %rsi = sign of digit (-1 = negative)+        xorq    %rsi, %rdi+        subq    %rsi, %rdi // %rdi = absolute value of digit++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        xorl    %r13d, %r13d+        xorl    %r14d, %r14d+        xorl    %r15d, %r15d++        .set I, 1+.rep 8+        cmpq    $I, %rdi++        cmovzq  TAB+96*(I-1)(%rsp), %rax+        cmovzq  TAB+96*(I-1)+8(%rsp), %rbx+        cmovzq  TAB+96*(I-1)+16(%rsp), %rcx+        cmovzq  TAB+96*(I-1)+24(%rsp), %rdx+        cmovzq  TAB+96*(I-1)+32(%rsp), %r8+        cmovzq  TAB+96*(I-1)+40(%rsp), %r9+        cmovzq  TAB+96*(I-1)+48(%rsp), %r10+        cmovzq  TAB+96*(I-1)+56(%rsp), %r11+        cmovzq  TAB+96*(I-1)+64(%rsp), %r12+        cmovzq  TAB+96*(I-1)+72(%rsp), %r13+        cmovzq  TAB+96*(I-1)+80(%rsp), %r14+        cmovzq  TAB+96*(I-1)+88(%rsp), %r15+        .set    I, (I+1)+.endr++        movq     %r12, TABENT+64(%rsp)+        movq     %r13, TABENT+72(%rsp)+        movq     %r14, TABENT+80(%rsp)+        movq     %r15, TABENT+88(%rsp)++        xorl    %r14d, %r14d+        leaq    -1(%r14), %r12+        movq    $0x00000000ffffffff, %r15+        movq    %r15, %r13+        negq    %r15++        subq    %r8, %r12+        sbbq    %r9, %r13+        sbbq    %r10, %r14+        sbbq    %r11, %r15++        testq    %rsi, %rsi+        cmovnzq  %r12, %r8+        cmovnzq  %r13, %r9+        cmovnzq  %r14, %r10+        cmovnzq  %r15, %r11++        movq     %rax, TABENT(%rsp)+        movq     %rbx, TABENT+8(%rsp)+        movq     %rcx, TABENT+16(%rsp)+        movq     %rdx, TABENT+24(%rsp)++        movq     %r8, TABENT+32(%rsp)+        movq     %r9, TABENT+40(%rsp)+        movq     %r10, TABENT+48(%rsp)+        movq     %r11, TABENT+56(%rsp)++        leaq    TABENT(%rsp), %rdx+        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp256_scalarmul_local_p256_montjadd)++        testq   %rbp, %rbp+        jne     Lp256_scalarmul_loop++// Let z2 = 1/z^2 and z3 = 1/z^3, both without Montgomery form++        leaq    Z2(%rsp), %rdi+        leaq    ACC+64(%rsp), %rsi+        CFI_CALL(Lp256_scalarmul_local_montsqr_p256)++        leaq    Z3(%rsp), %rdi+        leaq    ACC+64(%rsp), %rsi+        leaq    Z2(%rsp), %rdx+        CFI_CALL(Lp256_scalarmul_local_montmul_p256)++        leaq    Z2(%rsp), %rdi+        leaq    Z3(%rsp), %rsi+        CFI_CALL(Lp256_scalarmul_local_demont_p256)++        leaq    Z3(%rsp), %rdi+        leaq    Z2(%rsp), %rsi+        CFI_CALL(Lp256_scalarmul_local_inv_p256)++        leaq    Z2(%rsp), %rdi+        leaq    ACC+64(%rsp), %rsi+        leaq    Z3(%rsp), %rdx+        CFI_CALL(Lp256_scalarmul_local_montmul_p256)++// Convert back from Jacobian (X, Y, Z) |-> (X/Z^2, Y/Z^3)++        movq    res, %rdi+        leaq    ACC(%rsp), %rsi+        leaq    Z2(%rsp), %rdx+        movq    %rdi, %rbx+        CFI_CALL(Lp256_scalarmul_local_montmul_p256)++        leaq    32(%rbx), %rdi+        leaq    ACC+32(%rsp), %rsi+        leaq    Z3(%rsp), %rdx+        CFI_CALL(Lp256_scalarmul_local_montmul_p256)++// Restore stack and registers and return++        CFI_INC_RSP(NSPACE)+        CFI_POP(%rbx)+        CFI_POP(%rbp)+        CFI_POP(%r12)+        CFI_POP(%r13)+        CFI_POP(%r14)+        CFI_POP(%r15)+        CFI_RET++#if WINDOWS_ABI+S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_standard)+#else+S2N_BN_SIZE_DIRECTIVE(p256_scalarmul)+#endif++// Local copies of subroutines, complete clones at the moment++Lp256_scalarmul_local_demont_p256:+        CFI_START+        CFI_PUSH(%rbx)+        movq    (%rsi), %r8+        movq    0x8(%rsi), %r9+        movq    0x10(%rsi), %r10+        movq    0x18(%rsi), %r11+        xorq    %rbx, %rbx+        xorq    %rsi, %rsi+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rcx+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+        mulxq   %r9, %rax, %rcx+        adcxq   %rax, %r10+        adoxq   %rcx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rcx+        adcxq   %rax, %r11+        adoxq   %rcx, %rbx+        mulxq   %r9, %rax, %rcx+        adcxq   %rax, %rbx+        adoxq   %rcx, %rsi+        movl    $0x0, %r8d+        adcxq   %r8, %rsi+        xorq    %r9, %r9+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rcx+        adcxq   %rax, %r11+        adoxq   %rcx, %rbx+        mulxq   %r11, %rax, %rcx+        adcxq   %rax, %rbx+        adoxq   %rcx, %rsi+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rcx+        adcxq   %rax, %rsi+        adoxq   %rcx, %r8+        mulxq   %r11, %rax, %rcx+        adcxq   %rax, %r8+        adoxq   %rcx, %r9+        movl    $0x0, %r10d+        adcxq   %r10, %r9+        movq    %rbx, (%rdi)+        movq    %rsi, 0x8(%rdi)+        movq    %r8, 0x10(%rdi)+        movq    %r9, 0x18(%rdi)+        CFI_POP(%rbx)+        CFI_RET++Lp256_scalarmul_local_inv_p256:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(240)+        movq    %rdi, 0xe0(%rsp)+        xorl    %ecx, %ecx+        movl    $0xffffffff, %edx+        movq    %rdx, %rbx+        leaq    -0x1(%rcx), %rax+        negq    %rdx+        movq    %rax, (%rsp)+        movq    %rbx, 0x8(%rsp)+        movq    %rcx, 0x10(%rsp)+        movq    %rdx, 0x18(%rsp)+        movq    %rcx, 0x20(%rsp)+        movq    (%rsi), %r8+        movq    0x8(%rsi), %r9+        movq    0x10(%rsi), %r10+        movq    0x18(%rsi), %r11+        leaq    0x1(%rcx), %rax+        addq    %r8, %rax+        leaq    -0x1(%rdx), %rbx+        adcq    %r9, %rbx+        notq    %rcx+        adcq    %r10, %rcx+        notq    %rdx+        adcq    %r11, %rdx+        cmovaeq %r8, %rax+        cmovaeq %r9, %rbx+        cmovaeq %r10, %rcx+        cmovaeq %r11, %rdx+        movq    %rax, 0x28(%rsp)+        movq    %rbx, 0x30(%rsp)+        movq    %rcx, 0x38(%rsp)+        movq    %rdx, 0x40(%rsp)+        xorl    %eax, %eax+        movq    %rax, 0x48(%rsp)+        xorl    %eax, %eax+        movq    %rax, 0x50(%rsp)+        movq    %rax, 0x58(%rsp)+        movq    %rax, 0x60(%rsp)+        movq    %rax, 0x68(%rsp)+        movq    $0x4000000000000, %rcx+        movq    %rcx, 0x78(%rsp)+        movq    %rax, 0x80(%rsp)+        movq    %rax, 0x88(%rsp)+        movq    %rax, 0x90(%rsp)+        movq    $0xa,  0xb0(%rsp)+        movq    $0x1,  0xb8(%rsp)+        jmp     Lp256_scalarmul_inv_midloop+Lp256_scalarmul_inv_loop:+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %rdi+        andq    %r11, %rdi+        addq    %rax, %rdi+        movq    %rdi, 0xa0(%rsp)+        movq    %r12, %rax+        andq    %r13, %rax+        movq    %r14, %rsi+        andq    %r15, %rsi+        addq    %rax, %rsi+        movq    %rsi, 0xa8(%rsp)+        xorl    %ebx, %ebx+        movq    (%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x28(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    (%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x28(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        xorl    %ecx, %ecx+        movq    0x8(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x30(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, (%rsp)+        xorl    %edi, %edi+        movq    0x8(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    0x30(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $0x3b, %rbp, %rsi+        movq    %rsi, 0x28(%rsp)+        xorl    %esi, %esi+        movq    0x10(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    0x38(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rcx, %rbx+        movq    %rbx, 0x8(%rsp)+        xorl    %ebx, %ebx+        movq    0x10(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x38(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $0x3b, %rdi, %rbp+        movq    %rbp, 0x30(%rsp)+        movq    0x18(%rsp), %rax+        xorq    %r9, %rax+        movq    0x20(%rsp), %rbp+        xorq    %r9, %rbp+        andq    %r8, %rbp+        negq    %rbp+        mulq    %r8+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x40(%rsp), %rax+        xorq    %r11, %rax+        movq    0x48(%rsp), %rdx+        xorq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbp+        mulq    %r10+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        shrdq   $0x3b, %rsi, %rcx+        movq    %rcx, 0x10(%rsp)+        shrdq   $0x3b, %rbp, %rsi+        sarq    $0x3b, %rbp+        movq    0x18(%rsp), %rax+        movq    %rsi, 0x18(%rsp)+        movq    0x20(%rsp), %rsi+        movq    %rbp, 0x20(%rsp)+        xorq    %r13, %rax+        xorq    %r13, %rsi+        andq    %r12, %rsi+        negq    %rsi+        mulq    %r12+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        movq    0x40(%rsp), %rax+        xorq    %r15, %rax+        movq    0x48(%rsp), %rdx+        xorq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rsi+        mulq    %r14+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, 0x38(%rsp)+        shrdq   $0x3b, %rsi, %rbx+        movq    %rbx, 0x40(%rsp)+        sarq    $0x3b, %rsi+        movq    %rsi, 0x48(%rsp)+        movq    0xa0(%rsp), %rbx+        movq    0xa8(%rsp), %rbp+        xorl    %ecx, %ecx+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x50(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x50(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x78(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x78(%rsp)+        xorl    %ebx, %ebx+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x80(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    0x58(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, 0x58(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x80(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, 0x80(%rsp)+        xorl    %ecx, %ecx+        movq    0x60(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x88(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x60(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x60(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x88(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x88(%rsp)+        movq    0x68(%rsp), %rax+        xorq    %r9, %rax+        movq    %r9, %rbx+        andq    %r8, %rbx+        negq    %rbx+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x90(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbx+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rbx, %rdx+        movq    0x68(%rsp), %rax+        movq    %rcx, 0x68(%rsp)+        movq    %rdx, 0x70(%rsp)+        xorq    %r13, %rax+        movq    %r13, %rcx+        andq    %r12, %rcx+        negq    %rcx+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rcx+        movq    0x90(%rsp), %rax+        xorq    %r15, %rax+        movq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rcx+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rcx, %rdx+        movq    %rsi, 0x90(%rsp)+        movq    %rdx, 0x98(%rsp)+        movq    $0xe000000000000000, %r8+        addq    0x50(%rsp), %r8+        movq    $0xffffffffffffffff, %r9+        adcq    0x58(%rsp), %r9+        movq    $0x1fffffff, %r10+        adcq    0x60(%rsp), %r10+        movq    $0x2000000000000000, %r11+        adcq    0x68(%rsp), %r11+        movq    $0x1fffffffe0000000, %r12+        adcq    0x70(%rsp), %r12+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        movq    $0xffffffff00000001, %rax+        mulq    %r8+        shrq    $0x20, %r8+        addq    %rbx, %r9+        adcq    %r8, %r10+        adcq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rax, %rax+        movl    $0xffffffff, %ebx+        andq    %rax, %rbx+        movq    $0xffffffff00000001, %rdx+        andq    %rax, %rdx+        subq    %rax, %r9+        movq    %r9, 0x50(%rsp)+        sbbq    %rbx, %r10+        movq    %r10, 0x58(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x60(%rsp)+        sbbq    %rdx, %r12+        movq    %r12, 0x68(%rsp)+        movq    $0xe000000000000000, %r8+        addq    0x78(%rsp), %r8+        movq    $0xffffffffffffffff, %r9+        adcq    0x80(%rsp), %r9+        movq    $0x1fffffff, %r10+        adcq    0x88(%rsp), %r10+        movq    $0x2000000000000000, %r11+        adcq    0x90(%rsp), %r11+        movq    $0x1fffffffe0000000, %r12+        adcq    0x98(%rsp), %r12+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        movq    $0xffffffff00000001, %rax+        mulq    %r8+        shrq    $0x20, %r8+        addq    %rbx, %r9+        adcq    %r8, %r10+        adcq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rax, %rax+        movl    $0xffffffff, %ebx+        andq    %rax, %rbx+        movq    $0xffffffff00000001, %rdx+        andq    %rax, %rdx+        subq    %rax, %r9+        movq    %r9, 0x78(%rsp)+        sbbq    %rbx, %r10+        movq    %r10, 0x80(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x88(%rsp)+        sbbq    %rdx, %r12+        movq    %r12, 0x90(%rsp)+Lp256_scalarmul_inv_midloop:+        movq    0xb8(%rsp), %rsi+        movq    (%rsp), %rdx+        movq    0x28(%rsp), %rcx+        movq    %rdx, %rbx+        andq    $0xfffff, %rbx+        movq    $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        andq    $0xfffff, %rcx+        movq    $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        xorl    %ebp, %ebp+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %rdx+        leaq    (%rcx,%rax), %rdi+        shlq    $0x16, %rdx+        shlq    $0x16, %rdi+        sarq    $0x2b, %rdx+        sarq    $0x2b, %rdi+        movq    $0x20000100000, %rax+        leaq    (%rbx,%rax), %rbx+        leaq    (%rcx,%rax), %rcx+        sarq    $0x2a, %rbx+        sarq    $0x2a, %rcx+        movq    %rdx, 0xc0(%rsp)+        movq    %rbx, 0xc8(%rsp)+        movq    %rdi, 0xd0(%rsp)+        movq    %rcx, 0xd8(%rsp)+        movq    (%rsp), %r12+        imulq   %r12, %rdi+        imulq   %rdx, %r12+        movq    0x28(%rsp), %r13+        imulq   %r13, %rbx+        imulq   %rcx, %r13+        addq    %rbx, %r12+        addq    %rdi, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movq    $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movq    $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r10+        shlq    $0x16, %r8+        shlq    $0x16, %r10+        sarq    $0x2b, %r8+        sarq    $0x2b, %r10+        movq    $0x20000100000, %rax+        leaq    (%rbx,%rax), %r15+        leaq    (%rcx,%rax), %r11+        sarq    $0x2a, %r15+        sarq    $0x2a, %r11+        movq    %r13, %rbx+        movq    %r12, %rcx+        imulq   %r8, %r12+        imulq   %r15, %rbx+        addq    %rbx, %r12+        imulq   %r11, %r13+        imulq   %r10, %rcx+        addq    %rcx, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movq    $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movq    $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    0xc0(%rsp), %rax+        imulq   %r8, %rax+        movq    0xd0(%rsp), %rdx+        imulq   %r15, %rdx+        imulq   0xc8(%rsp), %r8+        imulq   0xd8(%rsp), %r15+        addq    %r8, %r15+        leaq    (%rax,%rdx), %r9+        movq    0xc0(%rsp), %rax+        imulq   %r10, %rax+        movq    0xd0(%rsp), %rdx+        imulq   %r11, %rdx+        imulq   0xc8(%rsp), %r10+        imulq   0xd8(%rsp), %r11+        addq    %r10, %r11+        leaq    (%rax,%rdx), %r13+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r12+        shlq    $0x15, %r8+        shlq    $0x15, %r12+        sarq    $0x2b, %r8+        sarq    $0x2b, %r12+        movq    $0x20000100000, %rax+        leaq    (%rbx,%rax), %r10+        leaq    (%rcx,%rax), %r14+        sarq    $0x2b, %r10+        sarq    $0x2b, %r14+        movq    %r9, %rax+        imulq   %r8, %rax+        movq    %r13, %rdx+        imulq   %r10, %rdx+        imulq   %r15, %r8+        imulq   %r11, %r10+        addq    %r8, %r10+        leaq    (%rax,%rdx), %r8+        movq    %r9, %rax+        imulq   %r12, %rax+        movq    %r13, %rdx+        imulq   %r14, %rdx+        imulq   %r15, %r12+        imulq   %r11, %r14+        addq    %r12, %r14+        leaq    (%rax,%rdx), %r12+        movq    %rsi, 0xb8(%rsp)+        decq     0xb0(%rsp)+        jne     Lp256_scalarmul_inv_loop+        movq    (%rsp), %rax+        movq    0x28(%rsp), %rcx+        imulq   %r8, %rax+        imulq   %r10, %rcx+        addq    %rcx, %rax+        sarq    $0x3f, %rax+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        xorq    %rax, %r9+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        xorq    %rax, %r11+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        xorq    %rax, %r13+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        xorq    %rax, %r15+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %r12+        andq    %r11, %r12+        addq    %rax, %r12+        xorl    %r13d, %r13d+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r13+        adcq    %rdx, %r14+        movq    0x80(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    0x60(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x88(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x68(%rsp), %rax+        xorq    %r9, %rax+        andq    %r8, %r9+        negq    %r9+        mulq    %r8+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    0x90(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %r9+        mulq    %r10+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    %r12, 0x50(%rsp)+        movq    %r13, 0x58(%rsp)+        movq    %r14, 0x60(%rsp)+        movq    %r15, 0x68(%rsp)+        movq    %r9, 0x70(%rsp)+        movq    $0xe000000000000000, %r8+        addq    0x50(%rsp), %r8+        movq    $0xffffffffffffffff, %r9+        adcq    0x58(%rsp), %r9+        movq    $0x1fffffff, %r10+        adcq    0x60(%rsp), %r10+        movq    $0x2000000000000000, %r11+        adcq    0x68(%rsp), %r11+        movq    $0x1fffffffe0000000, %r12+        adcq    0x70(%rsp), %r12+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        movq    $0xffffffff00000001, %rax+        mulq    %r8+        shrq    $0x20, %r8+        addq    %rbx, %r9+        adcq    %r8, %r10+        adcq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rax, %rax+        movl    $0xffffffff, %ebx+        andq    %rax, %rbx+        movq    $0xffffffff00000001, %rdx+        andq    %rax, %rdx+        subq    %rax, %r9+        movq    %r9, 0x50(%rsp)+        sbbq    %rbx, %r10+        movq    %r10, 0x58(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x60(%rsp)+        sbbq    %rdx, %r12+        movq    %r12, 0x68(%rsp)+        movq    0x50(%rsp), %r8+        movq    0x58(%rsp), %r9+        movq    0x60(%rsp), %r10+        movq    0x68(%rsp), %r11+        movl    $0x1, %eax+        movl    $0xffffffff, %ebx+        leaq    -0x2(%rax), %rcx+        leaq    -0x1(%rbx), %rdx+        notq    %rbx+        addq    %r8, %rax+        adcq    %r9, %rbx+        adcq    %r10, %rcx+        adcq    %r11, %rdx+        cmovaeq %r8, %rax+        cmovaeq %r9, %rbx+        cmovaeq %r10, %rcx+        cmovaeq %r11, %rdx+        movq    0xe0(%rsp), %rdi+        movq    %rax, (%rdi)+        movq    %rbx, 0x8(%rdi)+        movq    %rcx, 0x10(%rdi)+        movq    %rdx, 0x18(%rdi)+        CFI_INC_RSP(240)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++Lp256_scalarmul_local_montmul_p256:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        movq    %rdx, %rcx+        xorl    %r13d, %r13d+        movq    (%rcx), %rdx+        mulxq   (%rsi), %r8, %r9+        mulxq   0x8(%rsi), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x10(%rsi), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x18(%rsi), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x8(%rcx), %rdx+        xorl    %r14d, %r14d+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x10(%rcx), %rdx+        xorl    %r8d, %r8d+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x18(%rsi), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x18(%rcx), %rdx+        xorl    %r9d, %r9d+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x18(%rsi), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbx)+        CFI_RET++Lp256_scalarmul_local_montsqr_p256:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        movq    (%rsi), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0x8(%rsi), %r9, %r10+        mulxq   0x18(%rsi), %r11, %r12+        movq    0x10(%rsi), %rdx+        mulxq   0x18(%rsi), %r13, %r14+        xorl    %ebp, %ebp+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x18(%rsi), %rdx+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rbp, %r13+        adoxq   %rbp, %r14+        adcq    %rbp, %r14+        xorl    %ebp, %ebp+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0x8(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x10(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x18(%rsi), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rbp, %r15+        adoxq   %rbp, %r15+        xorl    %ebp, %ebp+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rbp, %r13+        movl    %ebp, %r9d+        adoxq   %rbp, %r9+        adcxq   %rbp, %r9+        addq    %r9, %r14+        adcq    %rbp, %r15+        movl    %ebp, %r8d+        adcq    %rbp, %r8+        xorl    %ebp, %ebp+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %r8+        adcq    %rbp, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rdx), %rdx+        adcq    %r13, %rdx+        leaq    -0x1(%rbp), %rbp+        movq    %rbp, %rax+        adcq    %r14, %rbp+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %rbp, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++Lp256_scalarmul_local_tomont_p256:+        CFI_START+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        xorq    %r13, %r13+        movl    $0x3, %edx+        mulxq   (%rsi), %r8, %r9+        mulxq   0x8(%rsi), %rcx, %r10+        adcxq   %rcx, %r9+        mulxq   0x10(%rsi), %rcx, %r11+        adcxq   %rcx, %r10+        mulxq   0x18(%rsi), %rcx, %r12+        adcxq   %rcx, %r11+        adcxq   %r13, %r12+        movq    $0xfffffffbffffffff, %rdx+        xorq    %r14, %r14+        mulxq   (%rsi), %rax, %rcx+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+        mulxq   0x8(%rsi), %rax, %rcx+        adcxq   %rax, %r10+        adoxq   %rcx, %r11+        mulxq   0x10(%rsi), %rax, %rcx+        adcxq   %rax, %r11+        adoxq   %rcx, %r12+        mulxq   0x18(%rsi), %rax, %rcx+        adcxq   %rax, %r12+        adoxq   %rcx, %r13+        adcq    %r14, %r13+        xorq    %r15, %r15+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rcx+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+        mulxq   %r9, %rax, %rcx+        adcxq   %rax, %r10+        adoxq   %rcx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rcx+        adcxq   %rax, %r11+        adoxq   %rcx, %r12+        mulxq   %r9, %rax, %rcx+        adcxq   %rax, %r12+        adoxq   %rcx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcxq   %r15, %r14+        movq    $0xfffffffffffffffe, %rdx+        xorq    %r8, %r8+        mulxq   (%rsi), %rax, %rcx+        adcxq   %rax, %r10+        adoxq   %rcx, %r11+        mulxq   0x8(%rsi), %rax, %rcx+        adcxq   %rax, %r11+        adoxq   %rcx, %r12+        mulxq   0x10(%rsi), %rax, %rcx+        adcxq   %rax, %r12+        adoxq   %rcx, %r13+        mulxq   0x18(%rsi), %rax, %rcx+        adcxq   %rax, %r13+        adoxq   %rcx, %r14+        adcxq   %r8, %r14+        adoxq   %r8, %r15+        adcxq   %r8, %r15+        movq    $0x4fffffffd, %rdx+        xorq    %r9, %r9+        mulxq   (%rsi), %rax, %rcx+        adcxq   %rax, %r11+        adoxq   %rcx, %r12+        mulxq   0x8(%rsi), %rax, %rcx+        adcxq   %rax, %r12+        adoxq   %rcx, %r13+        mulxq   0x10(%rsi), %rax, %rcx+        adcxq   %rax, %r13+        adoxq   %rcx, %r14+        mulxq   0x18(%rsi), %rax, %rcx+        adcxq   %rax, %r14+        adoxq   %rcx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcxq   %r9, %r8+        xorq    %r9, %r9+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rcx+        adcxq   %rax, %r11+        adoxq   %rcx, %r12+        mulxq   %r11, %rax, %rcx+        adcxq   %rax, %r12+        adoxq   %rcx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rcx+        adcxq   %rax, %r13+        adoxq   %rcx, %r14+        mulxq   %r11, %rax, %rcx+        adcxq   %rax, %r14+        adoxq   %rcx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcxq   %r9, %r8+        movl    $0xffffffff, %edx+        movq    $0xffffffff00000001, %rcx+        movq    $0xfffffffffffffffe, %rax+        subq    %r12, %rax+        movq    %rdx, %rax+        sbbq    %r13, %rax+        movl    $0x0, %eax+        sbbq    %r14, %rax+        movq    %rcx, %rax+        sbbq    %r15, %rax+        movl    $0x0, %eax+        sbbq    %r8, %rax+        andq    %rax, %rdx+        andq    %rax, %rcx+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    $0x0, %r14+        sbbq    %rcx, %r15+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_RET++Lp256_scalarmul_local_p256_montjadd:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(224)+        movq    %rdx, %rbp+        movq    0x40(%rsi), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0x48(%rsi), %r9, %r10+        mulxq   0x58(%rsi), %r11, %r12+        movq    0x50(%rsi), %rdx+        mulxq   0x58(%rsi), %r13, %r14+        xorl    %ecx, %ecx+        mulxq   0x40(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x58(%rsi), %rdx+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        adoxq   %rcx, %r14+        adcq    %rcx, %r14+        xorl    %ecx, %ecx+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0x48(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x50(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x58(%rsi), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rcx, %r15+        adoxq   %rcx, %r15+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        movl    %ecx, %r9d+        adoxq   %rcx, %r9+        adcxq   %rcx, %r9+        addq    %r9, %r14+        adcq    %rcx, %r15+        movl    %ecx, %r8d+        adcq    %rcx, %r8+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rcx, %r15+        adoxq   %rcx, %r8+        adcq    %rcx, %r8+        movl    $0x1, %r8d+        leaq    -0x1(%rdx), %rdx+        leaq    -0x1(%rcx), %rax+        movl    $0xfffffffe, %r11d+        cmoveq  %rcx, %r8+        cmoveq  %rcx, %rdx+        cmoveq  %rcx, %rax+        cmoveq  %rcx, %r11+        addq    %r8, %r12+        adcq    %rdx, %r13+        adcq    %rax, %r14+        adcq    %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        movq    0x40(%rbp), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0x48(%rbp), %r9, %r10+        mulxq   0x58(%rbp), %r11, %r12+        movq    0x50(%rbp), %rdx+        mulxq   0x58(%rbp), %r13, %r14+        xorl    %ecx, %ecx+        mulxq   0x40(%rbp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x48(%rbp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x58(%rbp), %rdx+        mulxq   0x48(%rbp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        adoxq   %rcx, %r14+        adcq    %rcx, %r14+        xorl    %ecx, %ecx+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0x48(%rbp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x50(%rbp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x58(%rbp), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rcx, %r15+        adoxq   %rcx, %r15+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        movl    %ecx, %r9d+        adoxq   %rcx, %r9+        adcxq   %rcx, %r9+        addq    %r9, %r14+        adcq    %rcx, %r15+        movl    %ecx, %r8d+        adcq    %rcx, %r8+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rcx, %r15+        adoxq   %rcx, %r8+        adcq    %rcx, %r8+        movl    $0x1, %r8d+        leaq    -0x1(%rdx), %rdx+        leaq    -0x1(%rcx), %rax+        movl    $0xfffffffe, %r11d+        cmoveq  %rcx, %r8+        cmoveq  %rcx, %rdx+        cmoveq  %rcx, %rax+        cmoveq  %rcx, %r11+        addq    %r8, %r12+        adcq    %rdx, %r13+        adcq    %rax, %r14+        adcq    %r11, %r15+        movq    %r12, 0xa0(%rsp)+        movq    %r13, 0xa8(%rsp)+        movq    %r14, 0xb0(%rsp)+        movq    %r15, 0xb8(%rsp)+        xorl    %r13d, %r13d+        movq    0x20(%rsi), %rdx+        mulxq   0x40(%rbp), %r8, %r9+        mulxq   0x48(%rbp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x50(%rbp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x58(%rbp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x28(%rsi), %rdx+        xorl    %r14d, %r14d+        mulxq   0x40(%rbp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x48(%rbp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x50(%rbp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x58(%rbp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x30(%rsi), %rdx+        xorl    %r8d, %r8d+        mulxq   0x40(%rbp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x48(%rbp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x50(%rbp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x58(%rbp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x38(%rsi), %rdx+        xorl    %r9d, %r9d+        mulxq   0x40(%rbp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x48(%rbp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x50(%rbp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x58(%rbp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xc0(%rsp)+        movq    %r13, 0xc8(%rsp)+        movq    %r14, 0xd0(%rsp)+        movq    %r15, 0xd8(%rsp)+        xorl    %r13d, %r13d+        movq    0x20(%rbp), %rdx+        mulxq   0x40(%rsi), %r8, %r9+        mulxq   0x48(%rsi), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x50(%rsi), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x58(%rsi), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x28(%rbp), %rdx+        xorl    %r14d, %r14d+        mulxq   0x40(%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x50(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x58(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x30(%rbp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x40(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x50(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x58(%rsi), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x38(%rbp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x40(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x50(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x58(%rsi), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x20(%rsp)+        movq    %r13, 0x28(%rsp)+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        xorl    %r13d, %r13d+        movq    0x0(%rbp), %rdx+        mulxq   (%rsp), %r8, %r9+        mulxq   0x8(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x10(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x18(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x8(%rbp), %rdx+        xorl    %r14d, %r14d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x10(%rbp), %rdx+        xorl    %r8d, %r8d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x18(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x18(%rbp), %rdx+        xorl    %r9d, %r9d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x18(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x40(%rsp)+        movq    %r13, 0x48(%rsp)+        movq    %r14, 0x50(%rsp)+        movq    %r15, 0x58(%rsp)+        xorl    %r13d, %r13d+        movq    (%rsi), %rdx+        mulxq   0xa0(%rsp), %r8, %r9+        mulxq   0xa8(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0xb0(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0xb8(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x8(%rsi), %rdx+        xorl    %r14d, %r14d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x10(%rsi), %rdx+        xorl    %r8d, %r8d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x18(%rsi), %rdx+        xorl    %r9d, %r9d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        xorl    %r13d, %r13d+        movq    0x20(%rsp), %rdx+        mulxq   (%rsp), %r8, %r9+        mulxq   0x8(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x10(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x18(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x28(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x30(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x18(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x38(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x18(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x20(%rsp)+        movq    %r13, 0x28(%rsp)+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        xorl    %r13d, %r13d+        movq    0xc0(%rsp), %rdx+        mulxq   0xa0(%rsp), %r8, %r9+        mulxq   0xa8(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0xb0(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0xb8(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0xc8(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0xd0(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0xd8(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xc0(%rsp)+        movq    %r13, 0xc8(%rsp)+        movq    %r14, 0xd0(%rsp)+        movq    %r15, 0xd8(%rsp)+        movq    0x40(%rsp), %rax+        subq    0x80(%rsp), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x88(%rsp), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x90(%rsp), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x98(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0xa0(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0xa8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0xb0(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0xb8(%rsp)+        movq    0x20(%rsp), %rax+        subq    0xc0(%rsp), %rax+        movq    0x28(%rsp), %rcx+        sbbq    0xc8(%rsp), %rcx+        movq    0x30(%rsp), %r8+        sbbq    0xd0(%rsp), %r8+        movq    0x38(%rsp), %r9+        sbbq    0xd8(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x20(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x28(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x30(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x38(%rsp)+        movq    0xa0(%rsp), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0xa8(%rsp), %r9, %r10+        mulxq   0xb8(%rsp), %r11, %r12+        movq    0xb0(%rsp), %rdx+        mulxq   0xb8(%rsp), %r13, %r14+        xorl    %ecx, %ecx+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0xb8(%rsp), %rdx+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        adoxq   %rcx, %r14+        adcq    %rcx, %r14+        xorl    %ecx, %ecx+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0xa8(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0xb0(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0xb8(%rsp), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rcx, %r15+        adoxq   %rcx, %r15+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        movl    %ecx, %r9d+        adoxq   %rcx, %r9+        adcxq   %rcx, %r9+        addq    %r9, %r14+        adcq    %rcx, %r15+        movl    %ecx, %r8d+        adcq    %rcx, %r8+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rcx, %r15+        adoxq   %rcx, %r8+        adcq    %rcx, %r8+        movl    $0x1, %r8d+        leaq    -0x1(%rdx), %rdx+        leaq    -0x1(%rcx), %rax+        movl    $0xfffffffe, %r11d+        cmoveq  %rcx, %r8+        cmoveq  %rcx, %rdx+        cmoveq  %rcx, %rax+        cmoveq  %rcx, %r11+        addq    %r8, %r12+        adcq    %rdx, %r13+        adcq    %rax, %r14+        adcq    %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        movq    0x20(%rsp), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0x28(%rsp), %r9, %r10+        mulxq   0x38(%rsp), %r11, %r12+        movq    0x30(%rsp), %rdx+        mulxq   0x38(%rsp), %r13, %r14+        xorl    %ecx, %ecx+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x38(%rsp), %rdx+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        adoxq   %rcx, %r14+        adcq    %rcx, %r14+        xorl    %ecx, %ecx+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0x28(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x30(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x38(%rsp), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rcx, %r15+        adoxq   %rcx, %r15+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        movl    %ecx, %r9d+        adoxq   %rcx, %r9+        adcxq   %rcx, %r9+        addq    %r9, %r14+        adcq    %rcx, %r15+        movl    %ecx, %r8d+        adcq    %rcx, %r8+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rcx, %r15+        adoxq   %rcx, %r8+        adcq    %rcx, %r8+        movl    $0x1, %ebx+        addq    %r12, %rbx+        leaq    -0x1(%rdx), %rdx+        adcq    %r13, %rdx+        leaq    -0x1(%rcx), %rcx+        movq    %rcx, %rax+        adcq    %r14, %rcx+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rbx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %rcx, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        xorl    %r13d, %r13d+        movq    0x80(%rsp), %rdx+        mulxq   0x60(%rsp), %r8, %r9+        mulxq   0x68(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x70(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x78(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x88(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x78(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x90(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x98(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        xorl    %r13d, %r13d+        movq    0x40(%rsp), %rdx+        mulxq   0x60(%rsp), %r8, %r9+        mulxq   0x68(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x70(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x78(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x48(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x78(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x50(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x58(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x40(%rsp)+        movq    %r13, 0x48(%rsp)+        movq    %r14, 0x50(%rsp)+        movq    %r15, 0x58(%rsp)+        movq    (%rsp), %rax+        subq    0x80(%rsp), %rax+        movq    0x8(%rsp), %rcx+        sbbq    0x88(%rsp), %rcx+        movq    0x10(%rsp), %r8+        sbbq    0x90(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x98(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, (%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x18(%rsp)+        movq    0x40(%rsp), %rax+        subq    0x80(%rsp), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x88(%rsp), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x90(%rsp), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x98(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x60(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x68(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x70(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x78(%rsp)+        xorl    %r13d, %r13d+        movq    0x40(%rsi), %rdx+        mulxq   0xa0(%rsp), %r8, %r9+        mulxq   0xa8(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0xb0(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0xb8(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x48(%rsi), %rdx+        xorl    %r14d, %r14d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x50(%rsi), %rdx+        xorl    %r8d, %r8d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x58(%rsi), %rdx+        xorl    %r9d, %r9d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xa0(%rsp)+        movq    %r13, 0xa8(%rsp)+        movq    %r14, 0xb0(%rsp)+        movq    %r15, 0xb8(%rsp)+        movq    (%rsp), %rax+        subq    0x40(%rsp), %rax+        movq    0x8(%rsp), %rcx+        sbbq    0x48(%rsp), %rcx+        movq    0x10(%rsp), %r8+        sbbq    0x50(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x58(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, (%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x18(%rsp)+        movq    0x80(%rsp), %rax+        subq    (%rsp), %rax+        movq    0x88(%rsp), %rcx+        sbbq    0x8(%rsp), %rcx+        movq    0x90(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x98(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x80(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x88(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x90(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x98(%rsp)+        xorl    %r13d, %r13d+        movq    0xc0(%rsp), %rdx+        mulxq   0x60(%rsp), %r8, %r9+        mulxq   0x68(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x70(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x78(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0xc8(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x78(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0xd0(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0xd8(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        xorl    %r13d, %r13d+        movq    0x40(%rbp), %rdx+        mulxq   0xa0(%rsp), %r8, %r9+        mulxq   0xa8(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0xb0(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0xb8(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x48(%rbp), %rdx+        xorl    %r14d, %r14d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x50(%rbp), %rdx+        xorl    %r8d, %r8d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x58(%rbp), %rdx+        xorl    %r9d, %r9d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xa0(%rsp)+        movq    %r13, 0xa8(%rsp)+        movq    %r14, 0xb0(%rsp)+        movq    %r15, 0xb8(%rsp)+        xorl    %r13d, %r13d+        movq    0x80(%rsp), %rdx+        mulxq   0x20(%rsp), %r8, %r9+        mulxq   0x28(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x30(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x38(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x88(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x38(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x90(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x38(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x98(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x38(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    0x80(%rsp), %rax+        subq    0x60(%rsp), %rax+        movq    0x88(%rsp), %rcx+        sbbq    0x68(%rsp), %rcx+        movq    0x90(%rsp), %r8+        sbbq    0x70(%rsp), %r8+        movq    0x98(%rsp), %r9+        sbbq    0x78(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x80(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x88(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x90(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x98(%rsp)+        movq    0x40(%rsi), %r8+        movq    0x48(%rsi), %r9+        movq    0x50(%rsi), %r10+        movq    0x58(%rsi), %r11+        movq    %r8, %rax+        movq    %r9, %rdx+        orq     %r10, %rax+        orq     %r11, %rdx+        orq     %rdx, %rax+        negq    %rax+        sbbq    %rax, %rax+        movq    0x40(%rbp), %r12+        movq    0x48(%rbp), %r13+        movq    0x50(%rbp), %r14+        movq    0x58(%rbp), %r15+        movq    %r12, %rbx+        movq    %r13, %rdx+        orq     %r14, %rbx+        orq     %r15, %rdx+        orq     %rdx, %rbx+        negq    %rbx+        sbbq    %rbx, %rbx+        cmpq    %rax, %rbx+        cmovbq  %r8, %r12+        cmovbq  %r9, %r13+        cmovbq  %r10, %r14+        cmovbq  %r11, %r15+        cmoveq  0xa0(%rsp), %r12+        cmoveq  0xa8(%rsp), %r13+        cmoveq  0xb0(%rsp), %r14+        cmoveq  0xb8(%rsp), %r15+        movq    (%rsp), %rax+        cmovbq  (%rsi), %rax+        cmova   0x0(%rbp), %rax+        movq    0x8(%rsp), %rbx+        cmovbq  0x8(%rsi), %rbx+        cmova   0x8(%rbp), %rbx+        movq    0x10(%rsp), %rcx+        cmovbq  0x10(%rsi), %rcx+        cmova   0x10(%rbp), %rcx+        movq    0x18(%rsp), %rdx+        cmovbq  0x18(%rsi), %rdx+        cmova   0x18(%rbp), %rdx+        movq    0x80(%rsp), %r8+        cmovbq  0x20(%rsi), %r8+        cmova   0x20(%rbp), %r8+        movq    0x88(%rsp), %r9+        cmovbq  0x28(%rsi), %r9+        cmova   0x28(%rbp), %r9+        movq    0x90(%rsp), %r10+        cmovbq  0x30(%rsi), %r10+        cmova   0x30(%rbp), %r10+        movq    0x98(%rsp), %r11+        cmovbq  0x38(%rsi), %r11+        cmova   0x38(%rbp), %r11+        movq    %rax, (%rdi)+        movq    %rbx, 0x8(%rdi)+        movq    %rcx, 0x10(%rdi)+        movq    %rdx, 0x18(%rdi)+        movq    %r8, 0x20(%rdi)+        movq    %r9, 0x28(%rdi)+        movq    %r10, 0x30(%rdi)+        movq    %r11, 0x38(%rdi)+        movq    %r12, 0x40(%rdi)+        movq    %r13, 0x48(%rdi)+        movq    %r14, 0x50(%rdi)+        movq    %r15, 0x58(%rdi)+        CFI_INC_RSP(224)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++Lp256_scalarmul_local_p256_montjdouble:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(192)+        movq    0x40(%rsi), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0x48(%rsi), %r9, %r10+        mulxq   0x58(%rsi), %r11, %r12+        movq    0x50(%rsi), %rdx+        mulxq   0x58(%rsi), %r13, %r14+        xorl    %ebp, %ebp+        mulxq   0x40(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x58(%rsi), %rdx+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rbp, %r13+        adoxq   %rbp, %r14+        adcq    %rbp, %r14+        xorl    %ebp, %ebp+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0x48(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x50(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x58(%rsi), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rbp, %r15+        adoxq   %rbp, %r15+        xorl    %ebp, %ebp+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rbp, %r13+        movl    %ebp, %r9d+        adoxq   %rbp, %r9+        adcxq   %rbp, %r9+        addq    %r9, %r14+        adcq    %rbp, %r15+        movl    %ebp, %r8d+        adcq    %rbp, %r8+        xorl    %ebp, %ebp+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %r8+        adcq    %rbp, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rdx), %rdx+        adcq    %r13, %rdx+        leaq    -0x1(%rbp), %rbp+        movq    %rbp, %rax+        adcq    %r14, %rbp+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %rbp, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        movq    0x20(%rsi), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0x28(%rsi), %r9, %r10+        mulxq   0x38(%rsi), %r11, %r12+        movq    0x30(%rsi), %rdx+        mulxq   0x38(%rsi), %r13, %r14+        xorl    %ebp, %ebp+        mulxq   0x20(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x28(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x38(%rsi), %rdx+        mulxq   0x28(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rbp, %r13+        adoxq   %rbp, %r14+        adcq    %rbp, %r14+        xorl    %ebp, %ebp+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0x28(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x30(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x38(%rsi), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rbp, %r15+        adoxq   %rbp, %r15+        xorl    %ebp, %ebp+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rbp, %r13+        movl    %ebp, %r9d+        adoxq   %rbp, %r9+        adcxq   %rbp, %r9+        addq    %r9, %r14+        adcq    %rbp, %r15+        movl    %ebp, %r8d+        adcq    %rbp, %r8+        xorl    %ebp, %ebp+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %r8+        adcq    %rbp, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rdx), %rdx+        adcq    %r13, %rdx+        leaq    -0x1(%rbp), %rbp+        movq    %rbp, %rax+        adcq    %r14, %rbp+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %rbp, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x20(%rsp)+        movq    %r13, 0x28(%rsp)+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    (%rsi), %rax+        subq    (%rsp), %rax+        movq    0x8(%rsi), %rcx+        sbbq    0x8(%rsp), %rcx+        movq    0x10(%rsi), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x18(%rsi), %r9+        sbbq    0x18(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x60(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x68(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x70(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x78(%rsp)+        movq    (%rsi), %rax+        addq    (%rsp), %rax+        movq    0x8(%rsi), %rcx+        adcq    0x8(%rsp), %rcx+        movq    0x10(%rsi), %r8+        adcq    0x10(%rsp), %r8+        movq    0x18(%rsi), %r9+        adcq    0x18(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        subq    %r11, %rax+        movq    %rax, 0x40(%rsp)+        sbbq    %r10, %rcx+        movq    %rcx, 0x48(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x50(%rsp)+        sbbq    %rdx, %r9+        movq    %r9, 0x58(%rsp)+        xorl    %r13d, %r13d+        movq    0x60(%rsp), %rdx+        mulxq   0x40(%rsp), %r8, %r9+        mulxq   0x48(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x50(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x58(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x68(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   0x40(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x48(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x50(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x58(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x70(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x40(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x48(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x50(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x58(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x78(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x40(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x48(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x50(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x58(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        xorq    %r11, %r11+        movq    0x20(%rsi), %rax+        addq    0x40(%rsi), %rax+        movq    0x28(%rsi), %rcx+        adcq    0x48(%rsi), %rcx+        movq    0x30(%rsi), %r8+        adcq    0x50(%rsi), %r8+        movq    0x38(%rsi), %r9+        adcq    0x58(%rsi), %r9+        adcq    %r11, %r11+        subq    $0xffffffffffffffff, %rax+        movl    $0xffffffff, %r10d+        sbbq    %r10, %rcx+        sbbq    $0x0, %r8+        movq    $0xffffffff00000001, %rdx+        sbbq    %rdx, %r9+        sbbq    $0x0, %r11+        andq    %r11, %r10+        andq    %r11, %rdx+        addq    %r11, %rax+        movq    %rax, 0x40(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x48(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x50(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x58(%rsp)+        xorl    %r13d, %r13d+        movq    0x20(%rsp), %rdx+        mulxq   (%rsi), %r8, %r9+        mulxq   0x8(%rsi), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x10(%rsi), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x18(%rsi), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x28(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x30(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x18(%rsi), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x38(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x18(%rsi), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    0x60(%rsp), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0x68(%rsp), %r9, %r10+        mulxq   0x78(%rsp), %r11, %r12+        movq    0x70(%rsp), %rdx+        mulxq   0x78(%rsp), %r13, %r14+        xorl    %ebp, %ebp+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x78(%rsp), %rdx+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rbp, %r13+        adoxq   %rbp, %r14+        adcq    %rbp, %r14+        xorl    %ebp, %ebp+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0x68(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x70(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x78(%rsp), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rbp, %r15+        adoxq   %rbp, %r15+        xorl    %ebp, %ebp+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rbp, %r13+        movl    %ebp, %r9d+        adoxq   %rbp, %r9+        adcxq   %rbp, %r9+        addq    %r9, %r14+        adcq    %rbp, %r15+        movl    %ebp, %r8d+        adcq    %rbp, %r8+        xorl    %ebp, %ebp+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %r8+        adcq    %rbp, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rdx), %rdx+        adcq    %r13, %rdx+        leaq    -0x1(%rbp), %rbp+        movq    %rbp, %rax+        adcq    %r14, %rbp+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %rbp, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xa0(%rsp)+        movq    %r13, 0xa8(%rsp)+        movq    %r14, 0xb0(%rsp)+        movq    %r15, 0xb8(%rsp)+        movq    0x40(%rsp), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0x48(%rsp), %r9, %r10+        mulxq   0x58(%rsp), %r11, %r12+        movq    0x50(%rsp), %rdx+        mulxq   0x58(%rsp), %r13, %r14+        xorl    %ebp, %ebp+        mulxq   0x40(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x48(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x58(%rsp), %rdx+        mulxq   0x48(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rbp, %r13+        adoxq   %rbp, %r14+        adcq    %rbp, %r14+        xorl    %ebp, %ebp+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0x48(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x50(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x58(%rsp), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rbp, %r15+        adoxq   %rbp, %r15+        xorl    %ebp, %ebp+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rbp, %r13+        movl    %ebp, %r9d+        adoxq   %rbp, %r9+        adcxq   %rbp, %r9+        addq    %r9, %r14+        adcq    %rbp, %r15+        movl    %ebp, %r8d+        adcq    %rbp, %r8+        xorl    %ebp, %ebp+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %r8+        adcq    %rbp, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rdx), %rdx+        adcq    %r13, %rdx+        leaq    -0x1(%rbp), %rbp+        movq    %rbp, %rax+        adcq    %r14, %rbp+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %rbp, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x40(%rsp)+        movq    %r13, 0x48(%rsp)+        movq    %r14, 0x50(%rsp)+        movq    %r15, 0x58(%rsp)+        movq    $0xffffffffffffffff, %r8+        xorl    %r10d, %r10d+        subq    0xa0(%rsp), %r8+        movq    $0xffffffff, %r9+        sbbq    0xa8(%rsp), %r9+        sbbq    0xb0(%rsp), %r10+        movq    $0xffffffff00000001, %r11+        sbbq    0xb8(%rsp), %r11+        xorl    %r12d, %r12d+        movq    $0x9, %rdx+        mulxq   %r8, %r8, %rax+        mulxq   %r9, %r9, %rcx+        addq    %rax, %r9+        mulxq   %r10, %r10, %rax+        adcq    %rcx, %r10+        mulxq   %r11, %r11, %rcx+        adcq    %rax, %r11+        adcq    %rcx, %r12+        movq    $0xc, %rdx+        xorl    %eax, %eax+        mulxq   0x80(%rsp), %rax, %rcx+        adcxq   %rax, %r8+        adoxq   %rcx, %r9+        mulxq   0x88(%rsp), %rax, %rcx+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+        mulxq   0x90(%rsp), %rax, %rcx+        adcxq   %rax, %r10+        adoxq   %rcx, %r11+        mulxq   0x98(%rsp), %rax, %rdx+        adcxq   %rax, %r11+        adoxq   %r12, %rdx+        adcq    $0x1, %rdx+        addq    %rdx, %r8+        movq    $0x100000000, %rax+        mulxq   %rax, %rax, %rcx+        sbbq    $0x0, %rax+        sbbq    $0x0, %rcx+        subq    %rax, %r9+        sbbq    %rcx, %r10+        movq    $0xffffffff00000001, %rax+        mulxq   %rax, %rax, %rcx+        sbbq    %rax, %r11+        sbbq    %rcx, %rdx+        decq    %rdx+        movl    $0xffffffff, %eax+        andq    %rdx, %rax+        xorl    %ecx, %ecx+        subq    %rax, %rcx+        addq    %rdx, %r8+        movq    %r8, 0xa0(%rsp)+        adcq    %rax, %r9+        movq    %r9, 0xa8(%rsp)+        adcq    $0x0, %r10+        movq    %r10, 0xb0(%rsp)+        adcq    %rcx, %r11+        movq    %r11, 0xb8(%rsp)+        movq    0x40(%rsp), %rax+        subq    (%rsp), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x8(%rsp), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x40(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x48(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x50(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x58(%rsp)+        movq    0x20(%rsp), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0x28(%rsp), %r9, %r10+        mulxq   0x38(%rsp), %r11, %r12+        movq    0x30(%rsp), %rdx+        mulxq   0x38(%rsp), %r13, %r14+        xorl    %ebp, %ebp+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x38(%rsp), %rdx+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rbp, %r13+        adoxq   %rbp, %r14+        adcq    %rbp, %r14+        xorl    %ebp, %ebp+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0x28(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x30(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x38(%rsp), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rbp, %r15+        adoxq   %rbp, %r15+        xorl    %ebp, %ebp+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rbp, %r13+        movl    %ebp, %r9d+        adoxq   %rbp, %r9+        adcxq   %rbp, %r9+        addq    %r9, %r14+        adcq    %rbp, %r15+        movl    %ebp, %r8d+        adcq    %rbp, %r8+        xorl    %ebp, %ebp+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %r8+        adcq    %rbp, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rdx), %rdx+        adcq    %r13, %rdx+        leaq    -0x1(%rbp), %rbp+        movq    %rbp, %rax+        adcq    %r14, %rbp+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %rbp, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        xorl    %r13d, %r13d+        movq    0x60(%rsp), %rdx+        mulxq   0xa0(%rsp), %r8, %r9+        mulxq   0xa8(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0xb0(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0xb8(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x68(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x70(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x78(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        movq    0x40(%rsp), %rax+        subq    0x20(%rsp), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x28(%rsp), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x30(%rsp), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x38(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x40(%rdi)+        adcq    %r10, %rcx+        movq    %rcx, 0x48(%rdi)+        adcq    $0x0, %r8+        movq    %r8, 0x50(%rdi)+        adcq    %rdx, %r9+        movq    %r9, 0x58(%rdi)+        movq    0x98(%rsp), %r11+        movq    %r11, %rdx+        movq    0x90(%rsp), %r10+        shldq   $0x2, %r10, %r11+        movq    0x88(%rsp), %r9+        shldq   $0x2, %r9, %r10+        movq    0x80(%rsp), %r8+        shldq   $0x2, %r8, %r9+        shlq    $0x2, %r8+        shrq    $0x3e, %rdx+        addq    $0x1, %rdx+        subq    0xa0(%rsp), %r8+        sbbq    0xa8(%rsp), %r9+        sbbq    0xb0(%rsp), %r10+        sbbq    0xb8(%rsp), %r11+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        movq    $0x100000000, %rax+        mulxq   %rax, %rax, %rcx+        sbbq    $0x0, %rax+        sbbq    $0x0, %rcx+        subq    %rax, %r9+        sbbq    %rcx, %r10+        movq    $0xffffffff00000001, %rax+        mulxq   %rax, %rax, %rcx+        sbbq    %rax, %r11+        sbbq    %rcx, %rdx+        decq    %rdx+        movl    $0xffffffff, %eax+        andq    %rdx, %rax+        xorl    %ecx, %ecx+        subq    %rax, %rcx+        addq    %rdx, %r8+        movq    %r8, (%rdi)+        adcq    %rax, %r9+        movq    %r9, 0x8(%rdi)+        adcq    $0x0, %r10+        movq    %r10, 0x10(%rdi)+        adcq    %rcx, %r11+        movq    %r11, 0x18(%rdi)+        movq    $0xffffffffffffffff, %r8+        xorl    %r10d, %r10d+        subq    (%rsp), %r8+        movq    $0xffffffff, %r9+        sbbq    0x8(%rsp), %r9+        sbbq    0x10(%rsp), %r10+        movq    $0xffffffff00000001, %r11+        sbbq    0x18(%rsp), %r11+        movq    %r11, %r12+        shldq   $0x3, %r10, %r11+        shldq   $0x3, %r9, %r10+        shldq   $0x3, %r8, %r9+        shlq    $0x3, %r8+        shrq    $0x3d, %r12+        movq    $0x3, %rdx+        xorl    %eax, %eax+        mulxq   0x60(%rsp), %rax, %rcx+        adcxq   %rax, %r8+        adoxq   %rcx, %r9+        mulxq   0x68(%rsp), %rax, %rcx+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+        mulxq   0x70(%rsp), %rax, %rcx+        adcxq   %rax, %r10+        adoxq   %rcx, %r11+        mulxq   0x78(%rsp), %rax, %rdx+        adcxq   %rax, %r11+        adoxq   %r12, %rdx+        adcq    $0x1, %rdx+        addq    %rdx, %r8+        movq    $0x100000000, %rax+        mulxq   %rax, %rax, %rcx+        sbbq    $0x0, %rax+        sbbq    $0x0, %rcx+        subq    %rax, %r9+        sbbq    %rcx, %r10+        movq    $0xffffffff00000001, %rax+        mulxq   %rax, %rax, %rcx+        sbbq    %rax, %r11+        sbbq    %rcx, %rdx+        decq    %rdx+        movl    $0xffffffff, %eax+        andq    %rdx, %rax+        xorl    %ecx, %ecx+        subq    %rax, %rcx+        addq    %rdx, %r8+        movq    %r8, 0x20(%rdi)+        adcq    %rax, %r9+        movq    %r9, 0x28(%rdi)+        adcq    $0x0, %r10+        movq    %r10, 0x30(%rdi)+        adcq    %rcx, %r11+        movq    %r11, 0x38(%rdi)+        CFI_INC_RSP(192)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++Lp256_scalarmul_local_p256_montjmixadd:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(192)+        movq    %rdx, %rbp+        movq    0x40(%rsi), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0x48(%rsi), %r9, %r10+        mulxq   0x58(%rsi), %r11, %r12+        movq    0x50(%rsi), %rdx+        mulxq   0x58(%rsi), %r13, %r14+        xorl    %ecx, %ecx+        mulxq   0x40(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x58(%rsi), %rdx+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        adoxq   %rcx, %r14+        adcq    %rcx, %r14+        xorl    %ecx, %ecx+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0x48(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x50(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x58(%rsi), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rcx, %r15+        adoxq   %rcx, %r15+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        movl    %ecx, %r9d+        adoxq   %rcx, %r9+        adcxq   %rcx, %r9+        addq    %r9, %r14+        adcq    %rcx, %r15+        movl    %ecx, %r8d+        adcq    %rcx, %r8+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rcx, %r15+        adoxq   %rcx, %r8+        adcq    %rcx, %r8+        movl    $0x1, %r8d+        leaq    -0x1(%rdx), %rdx+        leaq    -0x1(%rcx), %rax+        movl    $0xfffffffe, %r11d+        cmoveq  %rcx, %r8+        cmoveq  %rcx, %rdx+        cmoveq  %rcx, %rax+        cmoveq  %rcx, %r11+        addq    %r8, %r12+        adcq    %rdx, %r13+        adcq    %rax, %r14+        adcq    %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        xorl    %r13d, %r13d+        movq    0x20(%rbp), %rdx+        mulxq   0x40(%rsi), %r8, %r9+        mulxq   0x48(%rsi), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x50(%rsi), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x58(%rsi), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x28(%rbp), %rdx+        xorl    %r14d, %r14d+        mulxq   0x40(%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x50(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x58(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x30(%rbp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x40(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x50(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x58(%rsi), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x38(%rbp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x40(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x50(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x58(%rsi), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x20(%rsp)+        movq    %r13, 0x28(%rsp)+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        xorl    %r13d, %r13d+        movq    0x0(%rbp), %rdx+        mulxq   (%rsp), %r8, %r9+        mulxq   0x8(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x10(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x18(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x8(%rbp), %rdx+        xorl    %r14d, %r14d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x10(%rbp), %rdx+        xorl    %r8d, %r8d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x18(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x18(%rbp), %rdx+        xorl    %r9d, %r9d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x18(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x40(%rsp)+        movq    %r13, 0x48(%rsp)+        movq    %r14, 0x50(%rsp)+        movq    %r15, 0x58(%rsp)+        xorl    %r13d, %r13d+        movq    0x20(%rsp), %rdx+        mulxq   (%rsp), %r8, %r9+        mulxq   0x8(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x10(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x18(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x28(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x30(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x18(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x38(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x18(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x20(%rsp)+        movq    %r13, 0x28(%rsp)+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    0x40(%rsp), %rax+        subq    (%rsi), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x8(%rsi), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x10(%rsi), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x18(%rsi), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0xa0(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0xa8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0xb0(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0xb8(%rsp)+        movq    0x20(%rsp), %rax+        subq    0x20(%rsi), %rax+        movq    0x28(%rsp), %rcx+        sbbq    0x28(%rsi), %rcx+        movq    0x30(%rsp), %r8+        sbbq    0x30(%rsi), %r8+        movq    0x38(%rsp), %r9+        sbbq    0x38(%rsi), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x20(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x28(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x30(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x38(%rsp)+        movq    0xa0(%rsp), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0xa8(%rsp), %r9, %r10+        mulxq   0xb8(%rsp), %r11, %r12+        movq    0xb0(%rsp), %rdx+        mulxq   0xb8(%rsp), %r13, %r14+        xorl    %ecx, %ecx+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0xb8(%rsp), %rdx+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        adoxq   %rcx, %r14+        adcq    %rcx, %r14+        xorl    %ecx, %ecx+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0xa8(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0xb0(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0xb8(%rsp), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rcx, %r15+        adoxq   %rcx, %r15+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        movl    %ecx, %r9d+        adoxq   %rcx, %r9+        adcxq   %rcx, %r9+        addq    %r9, %r14+        adcq    %rcx, %r15+        movl    %ecx, %r8d+        adcq    %rcx, %r8+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rcx, %r15+        adoxq   %rcx, %r8+        adcq    %rcx, %r8+        movl    $0x1, %r8d+        leaq    -0x1(%rdx), %rdx+        leaq    -0x1(%rcx), %rax+        movl    $0xfffffffe, %r11d+        cmoveq  %rcx, %r8+        cmoveq  %rcx, %rdx+        cmoveq  %rcx, %rax+        cmoveq  %rcx, %r11+        addq    %r8, %r12+        adcq    %rdx, %r13+        adcq    %rax, %r14+        adcq    %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        movq    0x20(%rsp), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0x28(%rsp), %r9, %r10+        mulxq   0x38(%rsp), %r11, %r12+        movq    0x30(%rsp), %rdx+        mulxq   0x38(%rsp), %r13, %r14+        xorl    %ecx, %ecx+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x38(%rsp), %rdx+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        adoxq   %rcx, %r14+        adcq    %rcx, %r14+        xorl    %ecx, %ecx+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0x28(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x30(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x38(%rsp), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rcx, %r15+        adoxq   %rcx, %r15+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        movl    %ecx, %r9d+        adoxq   %rcx, %r9+        adcxq   %rcx, %r9+        addq    %r9, %r14+        adcq    %rcx, %r15+        movl    %ecx, %r8d+        adcq    %rcx, %r8+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rcx, %r15+        adoxq   %rcx, %r8+        adcq    %rcx, %r8+        movl    $0x1, %ebx+        addq    %r12, %rbx+        leaq    -0x1(%rdx), %rdx+        adcq    %r13, %rdx+        leaq    -0x1(%rcx), %rcx+        movq    %rcx, %rax+        adcq    %r14, %rcx+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rbx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %rcx, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        xorl    %r13d, %r13d+        movq    (%rsi), %rdx+        mulxq   0x60(%rsp), %r8, %r9+        mulxq   0x68(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x70(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x78(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x8(%rsi), %rdx+        xorl    %r14d, %r14d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x78(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x10(%rsi), %rdx+        xorl    %r8d, %r8d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x18(%rsi), %rdx+        xorl    %r9d, %r9d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        xorl    %r13d, %r13d+        movq    0x40(%rsp), %rdx+        mulxq   0x60(%rsp), %r8, %r9+        mulxq   0x68(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x70(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x78(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x48(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x78(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x50(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x58(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x40(%rsp)+        movq    %r13, 0x48(%rsp)+        movq    %r14, 0x50(%rsp)+        movq    %r15, 0x58(%rsp)+        movq    (%rsp), %rax+        subq    0x80(%rsp), %rax+        movq    0x8(%rsp), %rcx+        sbbq    0x88(%rsp), %rcx+        movq    0x10(%rsp), %r8+        sbbq    0x90(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x98(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, (%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x18(%rsp)+        movq    0x40(%rsp), %rax+        subq    0x80(%rsp), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x88(%rsp), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x90(%rsp), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x98(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x60(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x68(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x70(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x78(%rsp)+        xorl    %r13d, %r13d+        movq    0x40(%rsi), %rdx+        mulxq   0xa0(%rsp), %r8, %r9+        mulxq   0xa8(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0xb0(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0xb8(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x48(%rsi), %rdx+        xorl    %r14d, %r14d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x50(%rsi), %rdx+        xorl    %r8d, %r8d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x58(%rsi), %rdx+        xorl    %r9d, %r9d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xa0(%rsp)+        movq    %r13, 0xa8(%rsp)+        movq    %r14, 0xb0(%rsp)+        movq    %r15, 0xb8(%rsp)+        movq    (%rsp), %rax+        subq    0x40(%rsp), %rax+        movq    0x8(%rsp), %rcx+        sbbq    0x48(%rsp), %rcx+        movq    0x10(%rsp), %r8+        sbbq    0x50(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x58(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, (%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x18(%rsp)+        movq    0x80(%rsp), %rax+        subq    (%rsp), %rax+        movq    0x88(%rsp), %rcx+        sbbq    0x8(%rsp), %rcx+        movq    0x90(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x98(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x80(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x88(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x90(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x98(%rsp)+        xorl    %r13d, %r13d+        movq    0x20(%rsi), %rdx+        mulxq   0x60(%rsp), %r8, %r9+        mulxq   0x68(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x70(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x78(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x28(%rsi), %rdx+        xorl    %r14d, %r14d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x78(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x30(%rsi), %rdx+        xorl    %r8d, %r8d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x38(%rsi), %rdx+        xorl    %r9d, %r9d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        xorl    %r13d, %r13d+        movq    0x80(%rsp), %rdx+        mulxq   0x20(%rsp), %r8, %r9+        mulxq   0x28(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x30(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x38(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x88(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x38(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x90(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x38(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x98(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x38(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    0x80(%rsp), %rax+        subq    0x60(%rsp), %rax+        movq    0x88(%rsp), %rcx+        sbbq    0x68(%rsp), %rcx+        movq    0x90(%rsp), %r8+        sbbq    0x70(%rsp), %r8+        movq    0x98(%rsp), %r9+        sbbq    0x78(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x80(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x88(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x90(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x98(%rsp)+        movq    0x40(%rsi), %rax+        movq    0x48(%rsi), %rdx+        orq     0x50(%rsi), %rax+        orq     0x58(%rsi), %rdx+        orq     %rdx, %rax+        movq    (%rsp), %r8+        movq    0x0(%rbp), %rax+        cmoveq  %rax, %r8+        movq    0x8(%rsp), %r9+        movq    0x8(%rbp), %rax+        cmoveq  %rax, %r9+        movq    0x10(%rsp), %r10+        movq    0x10(%rbp), %rax+        cmoveq  %rax, %r10+        movq    0x18(%rsp), %r11+        movq    0x18(%rbp), %rax+        cmoveq  %rax, %r11+        movq    0x80(%rsp), %r12+        movq    0x20(%rbp), %rax+        cmoveq  %rax, %r12+        movq    0x88(%rsp), %r13+        movq    0x28(%rbp), %rax+        cmoveq  %rax, %r13+        movq    0x90(%rsp), %r14+        movq    0x30(%rbp), %rax+        cmoveq  %rax, %r14+        movq    0x98(%rsp), %r15+        movq    0x38(%rbp), %rax+        cmoveq  %rax, %r15+        movq    %r8, (%rdi)+        movq    %r9, 0x8(%rdi)+        movq    %r10, 0x10(%rdi)+        movq    %r11, 0x18(%rdi)+        movq    %r12, 0x20(%rdi)+        movq    %r13, 0x28(%rdi)+        movq    %r14, 0x30(%rdi)+        movq    %r15, 0x38(%rdi)+        movq    0xa0(%rsp), %r8+        movq    0xa8(%rsp), %r9+        movq    0xb0(%rsp), %r10+        movq    0xb8(%rsp), %r11+        movl    $0x1, %eax+        cmoveq  %rax, %r8+        movq    $0xffffffff00000000, %rax+        cmoveq  %rax, %r9+        movq    $0xffffffffffffffff, %rax+        cmoveq  %rax, %r10+        movl    $0xfffffffe, %eax+        cmoveq  %rax, %r11+        movq    %r8, 0x40(%rdi)+        movq    %r9, 0x48(%rdi)+        movq    %r10, 0x50(%rdi)+        movq    %r11, 0x58(%rdi)+        CFI_INC_RSP(192)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/p256_scalarmul_alt.S view
@@ -0,0 +1,8672 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Scalar multiplication for P-256+// Input scalar[4], point[8]; output res[8]+//+// extern void p256_scalarmul_alt+//   (uint64_t res[static 8],const uint64_t scalar[static 4],+//     const uint64_t point[static 8]);+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-256, returns the point (X,Y) = n * P. The input and output+// are affine points, and in the case of the point at infinity as+// the result, (0,0) is returned.+//+// Standard x86-64 ABI: RDI = res, RSI = scalar, RDX = point+// Microsoft x64 ABI:   RCX = res, RDX = scalar, R8 = point+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_scalarmul_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_scalarmul_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_scalarmul_alt)++        .text+        .balign 32++// Size of individual field elements++#define NUMSIZE 32++// Intermediate variables on the stack. The last z2, z3 values can+// safely be overlaid on the table, which is no longer needed at the end.+// Uppercase syntactic variants make x86_att version simpler to generate++#define SCALARB (0*NUMSIZE)+#define scalarb (0*NUMSIZE)(%rsp)+#define ACC (1*NUMSIZE)+#define acc (1*NUMSIZE)(%rsp)+#define TABENT (4*NUMSIZE)+#define tabent (4*NUMSIZE)(%rsp)++#define TAB (7*NUMSIZE)+#define tab (7*NUMSIZE)(%rsp)++#define Z2 (7*NUMSIZE)+#define z2 (7*NUMSIZE)(%rsp)+#define Z3 (8*NUMSIZE)+#define z3 (8*NUMSIZE)(%rsp)++#define res (31*NUMSIZE)(%rsp)++#define NSPACE 32*NUMSIZE++S2N_BN_SYMBOL(p256_scalarmul_alt):+        CFI_START+        _CET_ENDBR++// The Windows version literally calls the standard ABI version.+// This simplifies the proofs since subroutine offsets are fixed.++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+        CFI_CALL(Lp256_scalarmul_alt_standard)+        CFI_POP(%rsi)+        CFI_POP(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_scalarmul_alt)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmul_alt_standard)++Lp256_scalarmul_alt_standard:+        CFI_START+#endif++// Real start of the standard ABI code.++        CFI_PUSH(%r15)+        CFI_PUSH(%r14)+        CFI_PUSH(%r13)+        CFI_PUSH(%r12)+        CFI_PUSH(%rbp)+        CFI_PUSH(%rbx)++        CFI_DEC_RSP(NSPACE)++// Preserve the "res" and "point" input arguments. We load and process the+// scalar immediately so we don't bother preserving that input argument.+// Also, "point" is only needed early on and so its register gets re-used.++        movq    %rdx, %rbx+        movq    %rdi, res++// Load the digits of group order n_256 = [%r15;%r14;%r13;%r12]++        movq    $0xf3b9cac2fc632551, %r12+        movq    $0xbce6faada7179e84, %r13+        movq    $0xffffffffffffffff, %r14+        movq    $0xffffffff00000000, %r15++// First, reduce the input scalar mod n_256, i.e. conditionally subtract n_256++        movq    (%rsi), %r8+        subq    %r12, %r8+        movq    8(%rsi), %r9+        sbbq    %r13, %r9+        movq    16(%rsi), %r10+        sbbq    %r14, %r10+        movq    24(%rsi), %r11+        sbbq    %r15, %r11++        cmovcq  (%rsi), %r8+        cmovcq  8(%rsi), %r9+        cmovcq  16(%rsi), %r10+        cmovcq  24(%rsi), %r11++// Now if the top bit of the reduced scalar is set, negate it mod n_256,+// i.e. do n |-> n_256 - n. Remember the sign in %rbp so we can+// correspondingly negate the point below.++        subq    %r8, %r12+        sbbq    %r9, %r13+        sbbq    %r10, %r14+        sbbq    %r11, %r15++        movq    %r11, %rbp+        shrq    $63, %rbp+        cmovnzq %r12, %r8+        cmovnzq %r13, %r9+        cmovnzq %r14, %r10+        cmovnzq %r15, %r11++// In either case then add the recoding constant 0x08888...888 to allow+// signed digits.++        movq    $0x8888888888888888, %rax+        addq    %rax, %r8+        adcq    %rax, %r9+        adcq    %rax, %r10+        adcq    %rax, %r11+        btc     $63, %r11++        movq    %r8, SCALARB(%rsp)+        movq    %r9, SCALARB+8(%rsp)+        movq    %r10, SCALARB+16(%rsp)+        movq    %r11, SCALARB+24(%rsp)++// Set the tab[0] table entry to Montgomery-Jacobian point = 1 * P+// The z coordinate is just the Montgomery form of the constant 1.++        leaq    TAB(%rsp), %rdi+        movq    %rbx, %rsi+        CFI_CALL(Lp256_scalarmul_alt_local_tomont_p256)++        leaq    32(%rbx), %rsi+        leaq    TAB+32(%rsp), %rdi+        CFI_CALL(Lp256_scalarmul_alt_local_tomont_p256)++        movl    $1, %eax+        movq    %rax, TAB+64(%rsp)+        movq    $0xffffffff00000000, %rdx+        movq    %rdx, TAB+72(%rsp)+        subq    $2, %rax+        movq    %rax, TAB+80(%rsp)+        movq    $0x00000000fffffffe, %rax+        movq    %rax, TAB+88(%rsp)++// If the top bit of the scalar was set, negate (y coordinate of) the point++        movq    TAB+32(%rsp), %r12+        movq    TAB+40(%rsp), %r13+        movq    TAB+48(%rsp), %r14+        movq    TAB+56(%rsp), %r15++        xorl    %r10d, %r10d+        leaq    -1(%r10), %r8+        movq    $0x00000000ffffffff, %r11+        movq    %r11, %r9+        negq    %r11++        subq    %r12, %r8+        sbbq    %r13, %r9+        sbbq    %r14, %r10+        sbbq    %r15, %r11++        testq   %rbp, %rbp+        cmovzq  %r12, %r8+        cmovzq  %r13, %r9+        cmovzq  %r14, %r10+        cmovzq  %r15, %r11++        movq    %r8, TAB+32(%rsp)+        movq    %r9, TAB+40(%rsp)+        movq    %r10, TAB+48(%rsp)+        movq    %r11, TAB+56(%rsp)++// Compute and record tab[1] = 2 * p, ..., tab[7] = 8 * P++        leaq    TAB+96*1(%rsp), %rdi+        leaq    TAB(%rsp), %rsi+        CFI_CALL(Lp256_scalarmul_alt_local_p256_montjdouble)++        leaq    TAB+96*2(%rsp), %rdi+        leaq    TAB+96*1(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp256_scalarmul_alt_local_p256_montjmixadd)++        leaq    TAB+96*3(%rsp), %rdi+        leaq    TAB+96*1(%rsp), %rsi+        CFI_CALL(Lp256_scalarmul_alt_local_p256_montjdouble)++        leaq    TAB+96*4(%rsp), %rdi+        leaq    TAB+96*3(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp256_scalarmul_alt_local_p256_montjmixadd)++        leaq    TAB+96*5(%rsp), %rdi+        leaq    TAB+96*2(%rsp), %rsi+        CFI_CALL(Lp256_scalarmul_alt_local_p256_montjdouble)++        leaq    TAB+96*6(%rsp), %rdi+        leaq    TAB+96*5(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp256_scalarmul_alt_local_p256_montjmixadd)++        leaq    TAB+96*7(%rsp), %rdi+        leaq    TAB+96*3(%rsp), %rsi+        CFI_CALL(Lp256_scalarmul_alt_local_p256_montjdouble)++// Set up accumulator as table entry for top 4 bits (constant-time indexing)++        movq    SCALARB+24(%rsp), %rdi+        shrq    $60, %rdi++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        xorl    %r13d, %r13d+        xorl    %r14d, %r14d+        xorl    %r15d, %r15d++        .set I, 1+.rep 8+        cmpq    $I, %rdi++        cmovzq  TAB+96*(I-1)(%rsp), %rax+        cmovzq  TAB+96*(I-1)+8(%rsp), %rbx+        cmovzq  TAB+96*(I-1)+16(%rsp), %rcx+        cmovzq  TAB+96*(I-1)+24(%rsp), %rdx+        cmovzq  TAB+96*(I-1)+32(%rsp), %r8+        cmovzq  TAB+96*(I-1)+40(%rsp), %r9+        cmovzq  TAB+96*(I-1)+48(%rsp), %r10+        cmovzq  TAB+96*(I-1)+56(%rsp), %r11+        cmovzq  TAB+96*(I-1)+64(%rsp), %r12+        cmovzq  TAB+96*(I-1)+72(%rsp), %r13+        cmovzq  TAB+96*(I-1)+80(%rsp), %r14+        cmovzq  TAB+96*(I-1)+88(%rsp), %r15+        .set    I, (I+1)+.endr+        movq     %rax, ACC(%rsp)+        movq     %rbx, ACC+8(%rsp)+        movq     %rcx, ACC+16(%rsp)+        movq     %rdx, ACC+24(%rsp)+        movq     %r8, ACC+32(%rsp)+        movq     %r9, ACC+40(%rsp)+        movq     %r10, ACC+48(%rsp)+        movq     %r11, ACC+56(%rsp)+        movq     %r12, ACC+64(%rsp)+        movq     %r13, ACC+72(%rsp)+        movq     %r14, ACC+80(%rsp)+        movq     %r15, ACC+88(%rsp)++// Main loop over size-4 bitfield++        movl    $252, %ebp++Lp256_scalarmul_alt_loop:+        subq    $4, %rbp++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp256_scalarmul_alt_local_p256_montjdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp256_scalarmul_alt_local_p256_montjdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp256_scalarmul_alt_local_p256_montjdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp256_scalarmul_alt_local_p256_montjdouble)++        movq    %rbp, %rax+        shrq    $6, %rax+        movq    (%rsp,%rax,8), %rdi+        movq    %rbp, %rcx+        shrq    %cl, %rdi+        andq    $15, %rdi++        subq    $8, %rdi+        sbbq    %rsi, %rsi // %rsi = sign of digit (-1 = negative)+        xorq    %rsi, %rdi+        subq    %rsi, %rdi // %rdi = absolute value of digit++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        xorl    %r13d, %r13d+        xorl    %r14d, %r14d+        xorl    %r15d, %r15d++        .set I, 1+.rep 8+        cmpq    $I, %rdi++        cmovzq  TAB+96*(I-1)(%rsp), %rax+        cmovzq  TAB+96*(I-1)+8(%rsp), %rbx+        cmovzq  TAB+96*(I-1)+16(%rsp), %rcx+        cmovzq  TAB+96*(I-1)+24(%rsp), %rdx+        cmovzq  TAB+96*(I-1)+32(%rsp), %r8+        cmovzq  TAB+96*(I-1)+40(%rsp), %r9+        cmovzq  TAB+96*(I-1)+48(%rsp), %r10+        cmovzq  TAB+96*(I-1)+56(%rsp), %r11+        cmovzq  TAB+96*(I-1)+64(%rsp), %r12+        cmovzq  TAB+96*(I-1)+72(%rsp), %r13+        cmovzq  TAB+96*(I-1)+80(%rsp), %r14+        cmovzq  TAB+96*(I-1)+88(%rsp), %r15+        .set    I, (I+1)+.endr++        movq     %r12, TABENT+64(%rsp)+        movq     %r13, TABENT+72(%rsp)+        movq     %r14, TABENT+80(%rsp)+        movq     %r15, TABENT+88(%rsp)++        xorl    %r14d, %r14d+        leaq    -1(%r14), %r12+        movq    $0x00000000ffffffff, %r15+        movq    %r15, %r13+        negq    %r15++        subq    %r8, %r12+        sbbq    %r9, %r13+        sbbq    %r10, %r14+        sbbq    %r11, %r15++        testq    %rsi, %rsi+        cmovnzq  %r12, %r8+        cmovnzq  %r13, %r9+        cmovnzq  %r14, %r10+        cmovnzq  %r15, %r11++        movq     %rax, TABENT(%rsp)+        movq     %rbx, TABENT+8(%rsp)+        movq     %rcx, TABENT+16(%rsp)+        movq     %rdx, TABENT+24(%rsp)++        movq     %r8, TABENT+32(%rsp)+        movq     %r9, TABENT+40(%rsp)+        movq     %r10, TABENT+48(%rsp)+        movq     %r11, TABENT+56(%rsp)++        leaq    TABENT(%rsp), %rdx+        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp256_scalarmul_alt_local_p256_montjadd)++        testq   %rbp, %rbp+        jne     Lp256_scalarmul_alt_loop++// Let z2 = 1/z^2 and z3 = 1/z^3, both without Montgomery form++        leaq    Z2(%rsp), %rdi+        leaq    ACC+64(%rsp), %rsi+        CFI_CALL(Lp256_scalarmul_alt_local_montsqr_p256)++        leaq    Z3(%rsp), %rdi+        leaq    ACC+64(%rsp), %rsi+        leaq    Z2(%rsp), %rdx+        CFI_CALL(Lp256_scalarmul_alt_local_montmul_p256)++        leaq    Z2(%rsp), %rdi+        leaq    Z3(%rsp), %rsi+        CFI_CALL(Lp256_scalarmul_alt_local_demont_p256)++        leaq    Z3(%rsp), %rdi+        leaq    Z2(%rsp), %rsi+        CFI_CALL(Lp256_scalarmul_alt_local_inv_p256)++        leaq    Z2(%rsp), %rdi+        leaq    ACC+64(%rsp), %rsi+        leaq    Z3(%rsp), %rdx+        CFI_CALL(Lp256_scalarmul_alt_local_montmul_p256)++// Convert back from Jacobian (X, Y, Z) |-> (X/Z^2, Y/Z^3)++        movq    res, %rdi+        leaq    ACC(%rsp), %rsi+        leaq    Z2(%rsp), %rdx+        movq    %rdi, %rbx+        CFI_CALL(Lp256_scalarmul_alt_local_montmul_p256)++        leaq    32(%rbx), %rdi+        leaq    ACC+32(%rsp), %rsi+        leaq    Z3(%rsp), %rdx+        CFI_CALL(Lp256_scalarmul_alt_local_montmul_p256)++// Restore stack and registers and return++        CFI_INC_RSP(NSPACE)+        CFI_POP(%rbx)+        CFI_POP(%rbp)+        CFI_POP(%r12)+        CFI_POP(%r13)+        CFI_POP(%r14)+        CFI_POP(%r15)+        CFI_RET++#if WINDOWS_ABI+S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmul_alt_standard)+#else+S2N_BN_SIZE_DIRECTIVE(p256_scalarmul_alt)+#endif++// Local copies of subroutines, complete clones at the moment++Lp256_scalarmul_alt_local_demont_p256:+        CFI_START+        movq    (%rsi), %r8+        movq    0x8(%rsi), %r9+        movq    0x10(%rsi), %r10+        movq    0x18(%rsi), %r11+        movabsq $0x100000000, %rcx+        movq    %r8, %rax+        mulq    %rcx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rsi, %rsi+        movq    %r9, %rax+        mulq    %rcx+        subq    %rsi, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rsi, %rsi+        negq    %rcx+        negq    %rsi+        incq    %rcx+        movq    %r8, %rax+        mulq    %rcx+        addq    %rax, %r11+        adcq    %rdx, %rsi+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rax+        mulq    %rcx+        addq    %rax, %rsi+        adcq    %rdx, %r8+        negq    %rcx+        incq    %rcx+        movq    %r10, %rax+        mulq    %rcx+        addq    %rax, %r11+        adcq    %rdx, %rsi+        sbbq    %r9, %r9+        movq    %r11, %rax+        mulq    %rcx+        subq    %r9, %rdx+        addq    %rax, %rsi+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %rcx+        negq    %r9+        incq    %rcx+        movq    %r10, %rax+        mulq    %rcx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rax+        mulq    %rcx+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    %rsi, (%rdi)+        movq    %r8, 0x8(%rdi)+        movq    %r9, 0x10(%rdi)+        movq    %r10, 0x18(%rdi)+        CFI_RET++Lp256_scalarmul_alt_local_inv_p256:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(240)+        movq    %rdi, 0xe0(%rsp)+        xorl    %ecx, %ecx+        movl    $0xffffffff, %edx+        movq    %rdx, %rbx+        leaq    -0x1(%rcx), %rax+        negq    %rdx+        movq    %rax, (%rsp)+        movq    %rbx, 0x8(%rsp)+        movq    %rcx, 0x10(%rsp)+        movq    %rdx, 0x18(%rsp)+        movq    %rcx, 0x20(%rsp)+        movq    (%rsi), %r8+        movq    0x8(%rsi), %r9+        movq    0x10(%rsi), %r10+        movq    0x18(%rsi), %r11+        leaq    0x1(%rcx), %rax+        addq    %r8, %rax+        leaq    -0x1(%rdx), %rbx+        adcq    %r9, %rbx+        notq    %rcx+        adcq    %r10, %rcx+        notq    %rdx+        adcq    %r11, %rdx+        cmovaeq %r8, %rax+        cmovaeq %r9, %rbx+        cmovaeq %r10, %rcx+        cmovaeq %r11, %rdx+        movq    %rax, 0x28(%rsp)+        movq    %rbx, 0x30(%rsp)+        movq    %rcx, 0x38(%rsp)+        movq    %rdx, 0x40(%rsp)+        xorl    %eax, %eax+        movq    %rax, 0x48(%rsp)+        xorl    %eax, %eax+        movq    %rax, 0x50(%rsp)+        movq    %rax, 0x58(%rsp)+        movq    %rax, 0x60(%rsp)+        movq    %rax, 0x68(%rsp)+        movabsq $0x4000000000000, %rcx+        movq    %rcx, 0x78(%rsp)+        movq    %rax, 0x80(%rsp)+        movq    %rax, 0x88(%rsp)+        movq    %rax, 0x90(%rsp)+        movq    $0xa,  0xb0(%rsp)+        movq    $0x1,  0xb8(%rsp)+        jmp     Lp256_scalarmul_alt_inv_midloop+Lp256_scalarmul_alt_inv_loop:+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %rdi+        andq    %r11, %rdi+        addq    %rax, %rdi+        movq    %rdi, 0xa0(%rsp)+        movq    %r12, %rax+        andq    %r13, %rax+        movq    %r14, %rsi+        andq    %r15, %rsi+        addq    %rax, %rsi+        movq    %rsi, 0xa8(%rsp)+        xorl    %ebx, %ebx+        movq    (%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x28(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    (%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x28(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        xorl    %ecx, %ecx+        movq    0x8(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x30(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, (%rsp)+        xorl    %edi, %edi+        movq    0x8(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    0x30(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $0x3b, %rbp, %rsi+        movq    %rsi, 0x28(%rsp)+        xorl    %esi, %esi+        movq    0x10(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    0x38(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rcx, %rbx+        movq    %rbx, 0x8(%rsp)+        xorl    %ebx, %ebx+        movq    0x10(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x38(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $0x3b, %rdi, %rbp+        movq    %rbp, 0x30(%rsp)+        movq    0x18(%rsp), %rax+        xorq    %r9, %rax+        movq    0x20(%rsp), %rbp+        xorq    %r9, %rbp+        andq    %r8, %rbp+        negq    %rbp+        mulq    %r8+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x40(%rsp), %rax+        xorq    %r11, %rax+        movq    0x48(%rsp), %rdx+        xorq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbp+        mulq    %r10+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        shrdq   $0x3b, %rsi, %rcx+        movq    %rcx, 0x10(%rsp)+        shrdq   $0x3b, %rbp, %rsi+        sarq    $0x3b, %rbp+        movq    0x18(%rsp), %rax+        movq    %rsi, 0x18(%rsp)+        movq    0x20(%rsp), %rsi+        movq    %rbp, 0x20(%rsp)+        xorq    %r13, %rax+        xorq    %r13, %rsi+        andq    %r12, %rsi+        negq    %rsi+        mulq    %r12+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        movq    0x40(%rsp), %rax+        xorq    %r15, %rax+        movq    0x48(%rsp), %rdx+        xorq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rsi+        mulq    %r14+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, 0x38(%rsp)+        shrdq   $0x3b, %rsi, %rbx+        movq    %rbx, 0x40(%rsp)+        sarq    $0x3b, %rsi+        movq    %rsi, 0x48(%rsp)+        movq    0xa0(%rsp), %rbx+        movq    0xa8(%rsp), %rbp+        xorl    %ecx, %ecx+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x50(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x50(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x78(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x78(%rsp)+        xorl    %ebx, %ebx+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x80(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    0x58(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, 0x58(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x80(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, 0x80(%rsp)+        xorl    %ecx, %ecx+        movq    0x60(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x88(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x60(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x60(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x88(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x88(%rsp)+        movq    0x68(%rsp), %rax+        xorq    %r9, %rax+        movq    %r9, %rbx+        andq    %r8, %rbx+        negq    %rbx+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x90(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbx+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rbx, %rdx+        movq    0x68(%rsp), %rax+        movq    %rcx, 0x68(%rsp)+        movq    %rdx, 0x70(%rsp)+        xorq    %r13, %rax+        movq    %r13, %rcx+        andq    %r12, %rcx+        negq    %rcx+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rcx+        movq    0x90(%rsp), %rax+        xorq    %r15, %rax+        movq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rcx+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rcx, %rdx+        movq    %rsi, 0x90(%rsp)+        movq    %rdx, 0x98(%rsp)+        movabsq $0xe000000000000000, %r8+        addq    0x50(%rsp), %r8+        movq    $0xffffffffffffffff, %r9+        adcq    0x58(%rsp), %r9+        movq    $0x1fffffff, %r10+        adcq    0x60(%rsp), %r10+        movabsq $0x2000000000000000, %r11+        adcq    0x68(%rsp), %r11+        movabsq $0x1fffffffe0000000, %r12+        adcq    0x70(%rsp), %r12+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %r8+        shrq    $0x20, %r8+        addq    %rbx, %r9+        adcq    %r8, %r10+        adcq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rax, %rax+        movl    $0xffffffff, %ebx+        andq    %rax, %rbx+        movabsq $0xffffffff00000001, %rdx+        andq    %rax, %rdx+        subq    %rax, %r9+        movq    %r9, 0x50(%rsp)+        sbbq    %rbx, %r10+        movq    %r10, 0x58(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x60(%rsp)+        sbbq    %rdx, %r12+        movq    %r12, 0x68(%rsp)+        movabsq $0xe000000000000000, %r8+        addq    0x78(%rsp), %r8+        movq    $0xffffffffffffffff, %r9+        adcq    0x80(%rsp), %r9+        movq    $0x1fffffff, %r10+        adcq    0x88(%rsp), %r10+        movabsq $0x2000000000000000, %r11+        adcq    0x90(%rsp), %r11+        movabsq $0x1fffffffe0000000, %r12+        adcq    0x98(%rsp), %r12+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %r8+        shrq    $0x20, %r8+        addq    %rbx, %r9+        adcq    %r8, %r10+        adcq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rax, %rax+        movl    $0xffffffff, %ebx+        andq    %rax, %rbx+        movabsq $0xffffffff00000001, %rdx+        andq    %rax, %rdx+        subq    %rax, %r9+        movq    %r9, 0x78(%rsp)+        sbbq    %rbx, %r10+        movq    %r10, 0x80(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x88(%rsp)+        sbbq    %rdx, %r12+        movq    %r12, 0x90(%rsp)+Lp256_scalarmul_alt_inv_midloop:+        movq    0xb8(%rsp), %rsi+        movq    (%rsp), %rdx+        movq    0x28(%rsp), %rcx+        movq    %rdx, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        xorl    %ebp, %ebp+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %rdx+        leaq    (%rcx,%rax), %rdi+        shlq    $0x16, %rdx+        shlq    $0x16, %rdi+        sarq    $0x2b, %rdx+        sarq    $0x2b, %rdi+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %rbx+        leaq    (%rcx,%rax), %rcx+        sarq    $0x2a, %rbx+        sarq    $0x2a, %rcx+        movq    %rdx, 0xc0(%rsp)+        movq    %rbx, 0xc8(%rsp)+        movq    %rdi, 0xd0(%rsp)+        movq    %rcx, 0xd8(%rsp)+        movq    (%rsp), %r12+        imulq   %r12, %rdi+        imulq   %rdx, %r12+        movq    0x28(%rsp), %r13+        imulq   %r13, %rbx+        imulq   %rcx, %r13+        addq    %rbx, %r12+        addq    %rdi, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r10+        shlq    $0x16, %r8+        shlq    $0x16, %r10+        sarq    $0x2b, %r8+        sarq    $0x2b, %r10+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r15+        leaq    (%rcx,%rax), %r11+        sarq    $0x2a, %r15+        sarq    $0x2a, %r11+        movq    %r13, %rbx+        movq    %r12, %rcx+        imulq   %r8, %r12+        imulq   %r15, %rbx+        addq    %rbx, %r12+        imulq   %r11, %r13+        imulq   %r10, %rcx+        addq    %rcx, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    0xc0(%rsp), %rax+        imulq   %r8, %rax+        movq    0xd0(%rsp), %rdx+        imulq   %r15, %rdx+        imulq   0xc8(%rsp), %r8+        imulq   0xd8(%rsp), %r15+        addq    %r8, %r15+        leaq    (%rax,%rdx), %r9+        movq    0xc0(%rsp), %rax+        imulq   %r10, %rax+        movq    0xd0(%rsp), %rdx+        imulq   %r11, %rdx+        imulq   0xc8(%rsp), %r10+        imulq   0xd8(%rsp), %r11+        addq    %r10, %r11+        leaq    (%rax,%rdx), %r13+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r12+        shlq    $0x15, %r8+        shlq    $0x15, %r12+        sarq    $0x2b, %r8+        sarq    $0x2b, %r12+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r10+        leaq    (%rcx,%rax), %r14+        sarq    $0x2b, %r10+        sarq    $0x2b, %r14+        movq    %r9, %rax+        imulq   %r8, %rax+        movq    %r13, %rdx+        imulq   %r10, %rdx+        imulq   %r15, %r8+        imulq   %r11, %r10+        addq    %r8, %r10+        leaq    (%rax,%rdx), %r8+        movq    %r9, %rax+        imulq   %r12, %rax+        movq    %r13, %rdx+        imulq   %r14, %rdx+        imulq   %r15, %r12+        imulq   %r11, %r14+        addq    %r12, %r14+        leaq    (%rax,%rdx), %r12+        movq    %rsi, 0xb8(%rsp)+        decq     0xb0(%rsp)+        jne     Lp256_scalarmul_alt_inv_loop+        movq    (%rsp), %rax+        movq    0x28(%rsp), %rcx+        imulq   %r8, %rax+        imulq   %r10, %rcx+        addq    %rcx, %rax+        sarq    $0x3f, %rax+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        xorq    %rax, %r9+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        xorq    %rax, %r11+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        xorq    %rax, %r13+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        xorq    %rax, %r15+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %r12+        andq    %r11, %r12+        addq    %rax, %r12+        xorl    %r13d, %r13d+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r13+        adcq    %rdx, %r14+        movq    0x80(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    0x60(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x88(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x68(%rsp), %rax+        xorq    %r9, %rax+        andq    %r8, %r9+        negq    %r9+        mulq    %r8+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    0x90(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %r9+        mulq    %r10+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    %r12, 0x50(%rsp)+        movq    %r13, 0x58(%rsp)+        movq    %r14, 0x60(%rsp)+        movq    %r15, 0x68(%rsp)+        movq    %r9, 0x70(%rsp)+        movabsq $0xe000000000000000, %r8+        addq    0x50(%rsp), %r8+        movq    $0xffffffffffffffff, %r9+        adcq    0x58(%rsp), %r9+        movq    $0x1fffffff, %r10+        adcq    0x60(%rsp), %r10+        movabsq $0x2000000000000000, %r11+        adcq    0x68(%rsp), %r11+        movabsq $0x1fffffffe0000000, %r12+        adcq    0x70(%rsp), %r12+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %r8+        shrq    $0x20, %r8+        addq    %rbx, %r9+        adcq    %r8, %r10+        adcq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rax, %rax+        movl    $0xffffffff, %ebx+        andq    %rax, %rbx+        movabsq $0xffffffff00000001, %rdx+        andq    %rax, %rdx+        subq    %rax, %r9+        movq    %r9, 0x50(%rsp)+        sbbq    %rbx, %r10+        movq    %r10, 0x58(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x60(%rsp)+        sbbq    %rdx, %r12+        movq    %r12, 0x68(%rsp)+        movq    0x50(%rsp), %r8+        movq    0x58(%rsp), %r9+        movq    0x60(%rsp), %r10+        movq    0x68(%rsp), %r11+        movl    $0x1, %eax+        movl    $0xffffffff, %ebx+        leaq    -0x2(%rax), %rcx+        leaq    -0x1(%rbx), %rdx+        notq    %rbx+        addq    %r8, %rax+        adcq    %r9, %rbx+        adcq    %r10, %rcx+        adcq    %r11, %rdx+        cmovaeq %r8, %rax+        cmovaeq %r9, %rbx+        cmovaeq %r10, %rcx+        cmovaeq %r11, %rdx+        movq    0xe0(%rsp), %rdi+        movq    %rax, (%rdi)+        movq    %rbx, 0x8(%rdi)+        movq    %rcx, 0x10(%rdi)+        movq    %rdx, 0x18(%rdi)+        CFI_INC_RSP(240)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++Lp256_scalarmul_alt_local_montmul_p256:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        movq    %rdx, %rcx+        movq    (%rcx), %rbx+        movq    (%rsi), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x8(%rsi), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x10(%rsi), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x18(%rsi), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x8(%rcx), %rbx+        xorl    %r13d, %r13d+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x10(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x18(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x10(%rcx), %rbx+        xorl    %r15d, %r15d+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x10(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x18(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x18(%rcx), %rbx+        xorl    %r8d, %r8d+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x10(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x18(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbx)+        CFI_RET++Lp256_scalarmul_alt_local_montsqr_p256:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        movq    (%rsi), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0x8(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x18(%rsi), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x10(%rsi), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0x18(%rsi), %rbx+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0x8(%rsi), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0x10(%rsi), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0x18(%rsi), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbx)+        CFI_RET+++Lp256_scalarmul_alt_local_tomont_p256:+        CFI_START+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        movl    $0x3, %ecx+        movq    (%rsi), %rax+        mulq    %rcx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x8(%rsi), %rax+        mulq    %rcx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x10(%rsi), %rax+        mulq    %rcx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x18(%rsi), %rax+        mulq    %rcx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movabsq $0xfffffffbffffffff, %rcx+        xorl    %r13d, %r13d+        movq    (%rsi), %rax+        mulq    %rcx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x8(%rsi), %rax+        mulq    %rcx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x10(%rsi), %rax+        mulq    %rcx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x18(%rsi), %rax+        mulq    %rcx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rcx+        movq    %r8, %rax+        mulq    %rcx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rcx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rcx+        leaq    0x2(%rcx), %rcx+        movq    %r8, %rax+        mulq    %rcx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rcx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    $0xfffffffffffffffe, %rcx+        xorl    %r15d, %r15d+        movq    (%rsi), %rax+        mulq    %rcx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x8(%rsi), %rax+        mulq    %rcx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x10(%rsi), %rax+        mulq    %rcx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x18(%rsi), %rax+        mulq    %rcx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movabsq $0x4fffffffd, %rcx+        xorl    %r8d, %r8d+        movq    (%rsi), %rax+        mulq    %rcx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x8(%rsi), %rax+        mulq    %rcx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x10(%rsi), %rax+        mulq    %rcx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x18(%rsi), %rax+        mulq    %rcx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rcx+        movq    %r10, %rax+        mulq    %rcx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    %r11, %rax+        mulq    %rcx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        notq    %rcx+        leaq    0x2(%rcx), %rcx+        movq    %r10, %rax+        mulq    %rcx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    %r11, %rax+        mulq    %rcx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        subq    %r9, %r8+        xorl    %edx, %edx+        leaq    -0x1(%rdx), %r9+        incq    %rdx+        addq    %r12, %rdx+        decq    %rcx+        adcq    %r13, %rcx+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rdx, %r12+        cmovbq  %rcx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_RET++Lp256_scalarmul_alt_local_p256_montjadd:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(224)+        movq    %rdx, %rbp+        movq    0x40(%rsi), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0x48(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x58(%rsi), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x50(%rsi), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x40(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0x58(%rsi), %rbx+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0x48(%rsi), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0x50(%rsi), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0x58(%rsi), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        movq    0x40(%rbp), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0x48(%rbp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x58(%rbp), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x50(%rbp), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x40(%rbp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0x48(%rbp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0x58(%rbp), %rbx+        movq    0x48(%rbp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0x48(%rbp), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0x50(%rbp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0x58(%rbp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xa0(%rsp)+        movq    %r13, 0xa8(%rsp)+        movq    %r14, 0xb0(%rsp)+        movq    %r15, 0xb8(%rsp)+        movq    0x20(%rsi), %rbx+        movq    0x40(%rbp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x48(%rbp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x50(%rbp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x58(%rbp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x28(%rsi), %rbx+        xorl    %r13d, %r13d+        movq    0x40(%rbp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x48(%rbp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x50(%rbp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x58(%rbp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x30(%rsi), %rbx+        xorl    %r15d, %r15d+        movq    0x40(%rbp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x48(%rbp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x50(%rbp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x58(%rbp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x38(%rsi), %rbx+        xorl    %r8d, %r8d+        movq    0x40(%rbp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x48(%rbp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x50(%rbp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x58(%rbp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xc0(%rsp)+        movq    %r13, 0xc8(%rsp)+        movq    %r14, 0xd0(%rsp)+        movq    %r15, 0xd8(%rsp)+        movq    0x20(%rbp), %rbx+        movq    0x40(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x48(%rsi), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x50(%rsi), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x58(%rsi), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x28(%rbp), %rbx+        xorl    %r13d, %r13d+        movq    0x40(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x50(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x58(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x30(%rbp), %rbx+        xorl    %r15d, %r15d+        movq    0x40(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x50(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x58(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x38(%rbp), %rbx+        xorl    %r8d, %r8d+        movq    0x40(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x50(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x58(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x20(%rsp)+        movq    %r13, 0x28(%rsp)+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    0x0(%rbp), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x10(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x18(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x8(%rbp), %rbx+        xorl    %r13d, %r13d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x10(%rbp), %rbx+        xorl    %r15d, %r15d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x18(%rbp), %rbx+        xorl    %r8d, %r8d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x40(%rsp)+        movq    %r13, 0x48(%rsp)+        movq    %r14, 0x50(%rsp)+        movq    %r15, 0x58(%rsp)+        movq    (%rsi), %rbx+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x8(%rsi), %rbx+        xorl    %r13d, %r13d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x10(%rsi), %rbx+        xorl    %r15d, %r15d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x18(%rsi), %rbx+        xorl    %r8d, %r8d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    0x20(%rsp), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x10(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x18(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x28(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x30(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x38(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x20(%rsp)+        movq    %r13, 0x28(%rsp)+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    0xc0(%rsp), %rbx+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0xc8(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0xd0(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0xd8(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xc0(%rsp)+        movq    %r13, 0xc8(%rsp)+        movq    %r14, 0xd0(%rsp)+        movq    %r15, 0xd8(%rsp)+        movq    0x40(%rsp), %rax+        subq    0x80(%rsp), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x88(%rsp), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x90(%rsp), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x98(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0xa0(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0xa8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0xb0(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0xb8(%rsp)+        movq    0x20(%rsp), %rax+        subq    0xc0(%rsp), %rax+        movq    0x28(%rsp), %rcx+        sbbq    0xc8(%rsp), %rcx+        movq    0x30(%rsp), %r8+        sbbq    0xd0(%rsp), %r8+        movq    0x38(%rsp), %r9+        sbbq    0xd8(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x20(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x28(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x30(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x38(%rsp)+        movq    0xa0(%rsp), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0xb8(%rsp), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0xb0(%rsp), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0xb8(%rsp), %rbx+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0xa8(%rsp), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0xb0(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0xb8(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        movq    0x20(%rsp), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0x28(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x38(%rsp), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x30(%rsp), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x20(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0x38(%rsp), %rbx+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0x28(%rsp), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0x30(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0x38(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        movq    0x80(%rsp), %rbx+        movq    0x60(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x70(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x78(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x88(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x90(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x98(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    0x40(%rsp), %rbx+        movq    0x60(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x70(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x78(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x48(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x50(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x58(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x40(%rsp)+        movq    %r13, 0x48(%rsp)+        movq    %r14, 0x50(%rsp)+        movq    %r15, 0x58(%rsp)+        movq    (%rsp), %rax+        subq    0x80(%rsp), %rax+        movq    0x8(%rsp), %rcx+        sbbq    0x88(%rsp), %rcx+        movq    0x10(%rsp), %r8+        sbbq    0x90(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x98(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, (%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x18(%rsp)+        movq    0x40(%rsp), %rax+        subq    0x80(%rsp), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x88(%rsp), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x90(%rsp), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x98(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x60(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x68(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x70(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x78(%rsp)+        movq    0x40(%rsi), %rbx+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x48(%rsi), %rbx+        xorl    %r13d, %r13d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x50(%rsi), %rbx+        xorl    %r15d, %r15d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x58(%rsi), %rbx+        xorl    %r8d, %r8d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xa0(%rsp)+        movq    %r13, 0xa8(%rsp)+        movq    %r14, 0xb0(%rsp)+        movq    %r15, 0xb8(%rsp)+        movq    (%rsp), %rax+        subq    0x40(%rsp), %rax+        movq    0x8(%rsp), %rcx+        sbbq    0x48(%rsp), %rcx+        movq    0x10(%rsp), %r8+        sbbq    0x50(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x58(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, (%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x18(%rsp)+        movq    0x80(%rsp), %rax+        subq    (%rsp), %rax+        movq    0x88(%rsp), %rcx+        sbbq    0x8(%rsp), %rcx+        movq    0x90(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x98(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x80(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x88(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x90(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x98(%rsp)+        movq    0xc0(%rsp), %rbx+        movq    0x60(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x70(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x78(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0xc8(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0xd0(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0xd8(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        movq    0x40(%rbp), %rbx+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x48(%rbp), %rbx+        xorl    %r13d, %r13d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x50(%rbp), %rbx+        xorl    %r15d, %r15d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x58(%rbp), %rbx+        xorl    %r8d, %r8d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xa0(%rsp)+        movq    %r13, 0xa8(%rsp)+        movq    %r14, 0xb0(%rsp)+        movq    %r15, 0xb8(%rsp)+        movq    0x80(%rsp), %rbx+        movq    0x20(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x28(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x30(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x38(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x88(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    0x20(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x30(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x90(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    0x20(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x30(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x98(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    0x20(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x30(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    0x80(%rsp), %rax+        subq    0x60(%rsp), %rax+        movq    0x88(%rsp), %rcx+        sbbq    0x68(%rsp), %rcx+        movq    0x90(%rsp), %r8+        sbbq    0x70(%rsp), %r8+        movq    0x98(%rsp), %r9+        sbbq    0x78(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x80(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x88(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x90(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x98(%rsp)+        movq    0x40(%rsi), %r8+        movq    0x48(%rsi), %r9+        movq    0x50(%rsi), %r10+        movq    0x58(%rsi), %r11+        movq    %r8, %rax+        movq    %r9, %rdx+        orq     %r10, %rax+        orq     %r11, %rdx+        orq     %rdx, %rax+        negq    %rax+        sbbq    %rax, %rax+        movq    0x40(%rbp), %r12+        movq    0x48(%rbp), %r13+        movq    0x50(%rbp), %r14+        movq    0x58(%rbp), %r15+        movq    %r12, %rbx+        movq    %r13, %rdx+        orq     %r14, %rbx+        orq     %r15, %rdx+        orq     %rdx, %rbx+        negq    %rbx+        sbbq    %rbx, %rbx+        cmpq    %rax, %rbx+        cmovbq  %r8, %r12+        cmovbq  %r9, %r13+        cmovbq  %r10, %r14+        cmovbq  %r11, %r15+        cmoveq  0xa0(%rsp), %r12+        cmoveq  0xa8(%rsp), %r13+        cmoveq  0xb0(%rsp), %r14+        cmoveq  0xb8(%rsp), %r15+        movq    (%rsp), %rax+        cmovbq  (%rsi), %rax+        cmova   0x0(%rbp), %rax+        movq    0x8(%rsp), %rbx+        cmovbq  0x8(%rsi), %rbx+        cmova   0x8(%rbp), %rbx+        movq    0x10(%rsp), %rcx+        cmovbq  0x10(%rsi), %rcx+        cmova   0x10(%rbp), %rcx+        movq    0x18(%rsp), %rdx+        cmovbq  0x18(%rsi), %rdx+        cmova   0x18(%rbp), %rdx+        movq    0x80(%rsp), %r8+        cmovbq  0x20(%rsi), %r8+        cmova   0x20(%rbp), %r8+        movq    0x88(%rsp), %r9+        cmovbq  0x28(%rsi), %r9+        cmova   0x28(%rbp), %r9+        movq    0x90(%rsp), %r10+        cmovbq  0x30(%rsi), %r10+        cmova   0x30(%rbp), %r10+        movq    0x98(%rsp), %r11+        cmovbq  0x38(%rsi), %r11+        cmova   0x38(%rbp), %r11+        movq    %rax, (%rdi)+        movq    %rbx, 0x8(%rdi)+        movq    %rcx, 0x10(%rdi)+        movq    %rdx, 0x18(%rdi)+        movq    %r8, 0x20(%rdi)+        movq    %r9, 0x28(%rdi)+        movq    %r10, 0x30(%rdi)+        movq    %r11, 0x38(%rdi)+        movq    %r12, 0x40(%rdi)+        movq    %r13, 0x48(%rdi)+        movq    %r14, 0x50(%rdi)+        movq    %r15, 0x58(%rdi)+        CFI_INC_RSP(224)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++Lp256_scalarmul_alt_local_p256_montjdouble:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(192)+        movq    0x40(%rsi), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0x48(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x58(%rsi), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x50(%rsi), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x40(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0x58(%rsi), %rbx+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0x48(%rsi), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0x50(%rsi), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0x58(%rsi), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        movq    0x20(%rsi), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0x28(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x38(%rsi), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x30(%rsi), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x20(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0x28(%rsi), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0x38(%rsi), %rbx+        movq    0x28(%rsi), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0x28(%rsi), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0x30(%rsi), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0x38(%rsi), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x20(%rsp)+        movq    %r13, 0x28(%rsp)+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    (%rsi), %rax+        subq    (%rsp), %rax+        movq    0x8(%rsi), %rcx+        sbbq    0x8(%rsp), %rcx+        movq    0x10(%rsi), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x18(%rsi), %r9+        sbbq    0x18(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x60(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x68(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x70(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x78(%rsp)+        movq    (%rsi), %rax+        addq    (%rsp), %rax+        movq    0x8(%rsi), %rcx+        adcq    0x8(%rsp), %rcx+        movq    0x10(%rsi), %r8+        adcq    0x10(%rsp), %r8+        movq    0x18(%rsi), %r9+        adcq    0x18(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        subq    %r11, %rax+        movq    %rax, 0x40(%rsp)+        sbbq    %r10, %rcx+        movq    %rcx, 0x48(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x50(%rsp)+        sbbq    %rdx, %r9+        movq    %r9, 0x58(%rsp)+        movq    0x60(%rsp), %rbx+        movq    0x40(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x48(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x50(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x58(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x68(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    0x40(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x48(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x50(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x58(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x70(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    0x40(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x48(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x50(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x58(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x78(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    0x40(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x48(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x50(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x58(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        xorq    %r11, %r11+        movq    0x20(%rsi), %rax+        addq    0x40(%rsi), %rax+        movq    0x28(%rsi), %rcx+        adcq    0x48(%rsi), %rcx+        movq    0x30(%rsi), %r8+        adcq    0x50(%rsi), %r8+        movq    0x38(%rsi), %r9+        adcq    0x58(%rsi), %r9+        adcq    %r11, %r11+        subq    $0xffffffffffffffff, %rax+        movl    $0xffffffff, %r10d+        sbbq    %r10, %rcx+        sbbq    $0x0, %r8+        movabsq $0xffffffff00000001, %rdx+        sbbq    %rdx, %r9+        sbbq    $0x0, %r11+        andq    %r11, %r10+        andq    %r11, %rdx+        addq    %r11, %rax+        movq    %rax, 0x40(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x48(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x50(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x58(%rsp)+        movq    0x20(%rsp), %rbx+        movq    (%rsi), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x8(%rsi), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x10(%rsi), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x18(%rsi), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x28(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x10(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x18(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x30(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x10(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x18(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x38(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x10(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x18(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    0x60(%rsp), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0x68(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x78(%rsp), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x70(%rsp), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0x78(%rsp), %rbx+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0x68(%rsp), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0x70(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0x78(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xa0(%rsp)+        movq    %r13, 0xa8(%rsp)+        movq    %r14, 0xb0(%rsp)+        movq    %r15, 0xb8(%rsp)+        movq    0x40(%rsp), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0x48(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x58(%rsp), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x50(%rsp), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x40(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0x48(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0x58(%rsp), %rbx+        movq    0x48(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0x48(%rsp), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0x50(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0x58(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x40(%rsp)+        movq    %r13, 0x48(%rsp)+        movq    %r14, 0x50(%rsp)+        movq    %r15, 0x58(%rsp)+        movq    $0xffffffffffffffff, %r9+        xorl    %r11d, %r11d+        subq    0xa0(%rsp), %r9+        movabsq $0xffffffff, %r10+        sbbq    0xa8(%rsp), %r10+        sbbq    0xb0(%rsp), %r11+        movabsq $0xffffffff00000001, %r12+        sbbq    0xb8(%rsp), %r12+        movq    $0x9, %rcx+        movq    %r9, %rax+        mulq    %rcx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    %r10, %rax+        xorl    %r10d, %r10d+        mulq    %rcx+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    %r11, %rax+        xorl    %r11d, %r11d+        mulq    %rcx+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    %r12, %rax+        xorl    %r12d, %r12d+        mulq    %rcx+        addq    %rax, %r11+        adcq    %rdx, %r12+        movl    $0xc, %ecx+        movq    0x80(%rsp), %rax+        mulq    %rcx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %rbx, %rbx+        movq    0x88(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rbx, %rbx+        movq    0x90(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbx, %rbx+        movq    0x98(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        leaq    0x1(%r12), %rcx+        movabsq $0xffffffff00000001, %rax+        mulq    %rcx+        movq    %rcx, %rbx+        shlq    $0x20, %rbx+        addq    %rcx, %r8+        sbbq    $0x0, %rbx+        subq    %rbx, %r9+        sbbq    $0x0, %r10+        sbbq    %rax, %r11+        sbbq    %rdx, %rcx+        decq    %rcx+        movl    $0xffffffff, %eax+        andq    %rcx, %rax+        xorl    %edx, %edx+        subq    %rax, %rdx+        addq    %rcx, %r8+        movq    %r8, 0xa0(%rsp)+        adcq    %rax, %r9+        movq    %r9, 0xa8(%rsp)+        adcq    $0x0, %r10+        movq    %r10, 0xb0(%rsp)+        adcq    %rdx, %r11+        movq    %r11, 0xb8(%rsp)+        movq    0x40(%rsp), %rax+        subq    (%rsp), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x8(%rsp), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x40(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x48(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x50(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x58(%rsp)+        movq    0x20(%rsp), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0x28(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x38(%rsp), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x30(%rsp), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x20(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0x38(%rsp), %rbx+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0x28(%rsp), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0x30(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0x38(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        movq    0x60(%rsp), %rbx+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x68(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x70(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x78(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        movq    0x40(%rsp), %rax+        subq    0x20(%rsp), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x28(%rsp), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x30(%rsp), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x38(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x40(%rdi)+        adcq    %r10, %rcx+        movq    %rcx, 0x48(%rdi)+        adcq    $0x0, %r8+        movq    %r8, 0x50(%rdi)+        adcq    %rdx, %r9+        movq    %r9, 0x58(%rdi)+        movq    0x98(%rsp), %r11+        movq    %r11, %rcx+        movq    0x90(%rsp), %r10+        shldq   $0x2, %r10, %r11+        movq    0x88(%rsp), %r9+        shldq   $0x2, %r9, %r10+        movq    0x80(%rsp), %r8+        shldq   $0x2, %r8, %r9+        shlq    $0x2, %r8+        shrq    $0x3e, %rcx+        addq    $0x1, %rcx+        subq    0xa0(%rsp), %r8+        sbbq    0xa8(%rsp), %r9+        sbbq    0xb0(%rsp), %r10+        sbbq    0xb8(%rsp), %r11+        sbbq    $0x0, %rcx+        movabsq $0xffffffff00000001, %rax+        mulq    %rcx+        movq    %rcx, %rbx+        shlq    $0x20, %rbx+        addq    %rcx, %r8+        sbbq    $0x0, %rbx+        subq    %rbx, %r9+        sbbq    $0x0, %r10+        sbbq    %rax, %r11+        sbbq    %rdx, %rcx+        decq    %rcx+        movl    $0xffffffff, %eax+        andq    %rcx, %rax+        xorl    %edx, %edx+        subq    %rax, %rdx+        addq    %rcx, %r8+        movq    %r8, (%rdi)+        adcq    %rax, %r9+        movq    %r9, 0x8(%rdi)+        adcq    $0x0, %r10+        movq    %r10, 0x10(%rdi)+        adcq    %rdx, %r11+        movq    %r11, 0x18(%rdi)+        movq    $0xffffffffffffffff, %r8+        xorl    %r10d, %r10d+        subq    (%rsp), %r8+        movabsq $0xffffffff, %r9+        sbbq    0x8(%rsp), %r9+        sbbq    0x10(%rsp), %r10+        movabsq $0xffffffff00000001, %r11+        sbbq    0x18(%rsp), %r11+        movq    %r11, %r12+        shldq   $0x3, %r10, %r11+        shldq   $0x3, %r9, %r10+        shldq   $0x3, %r8, %r9+        shlq    $0x3, %r8+        shrq    $0x3d, %r12+        movl    $0x3, %ecx+        movq    0x60(%rsp), %rax+        mulq    %rcx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %rbx, %rbx+        movq    0x68(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rbx, %rbx+        movq    0x70(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbx, %rbx+        movq    0x78(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        leaq    0x1(%r12), %rcx+        movabsq $0xffffffff00000001, %rax+        mulq    %rcx+        movq    %rcx, %rbx+        shlq    $0x20, %rbx+        addq    %rcx, %r8+        sbbq    $0x0, %rbx+        subq    %rbx, %r9+        sbbq    $0x0, %r10+        sbbq    %rax, %r11+        sbbq    %rdx, %rcx+        decq    %rcx+        movl    $0xffffffff, %eax+        andq    %rcx, %rax+        xorl    %edx, %edx+        subq    %rax, %rdx+        addq    %rcx, %r8+        movq    %r8, 0x20(%rdi)+        adcq    %rax, %r9+        movq    %r9, 0x28(%rdi)+        adcq    $0x0, %r10+        movq    %r10, 0x30(%rdi)+        adcq    %rdx, %r11+        movq    %r11, 0x38(%rdi)+        CFI_INC_RSP(192)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbx)+        CFI_RET++Lp256_scalarmul_alt_local_p256_montjmixadd:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(192)+        movq    %rdx, %rbp+        movq    0x40(%rsi), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0x48(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x58(%rsi), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x50(%rsi), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x40(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0x58(%rsi), %rbx+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0x48(%rsi), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0x50(%rsi), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0x58(%rsi), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        movq    0x20(%rbp), %rbx+        movq    0x40(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x48(%rsi), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x50(%rsi), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x58(%rsi), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x28(%rbp), %rbx+        xorl    %r13d, %r13d+        movq    0x40(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x50(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x58(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x30(%rbp), %rbx+        xorl    %r15d, %r15d+        movq    0x40(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x50(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x58(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x38(%rbp), %rbx+        xorl    %r8d, %r8d+        movq    0x40(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x50(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x58(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x20(%rsp)+        movq    %r13, 0x28(%rsp)+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    0x0(%rbp), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x10(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x18(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x8(%rbp), %rbx+        xorl    %r13d, %r13d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x10(%rbp), %rbx+        xorl    %r15d, %r15d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x18(%rbp), %rbx+        xorl    %r8d, %r8d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x40(%rsp)+        movq    %r13, 0x48(%rsp)+        movq    %r14, 0x50(%rsp)+        movq    %r15, 0x58(%rsp)+        movq    0x20(%rsp), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x10(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x18(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x28(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x30(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x38(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x20(%rsp)+        movq    %r13, 0x28(%rsp)+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    0x40(%rsp), %rax+        subq    (%rsi), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x8(%rsi), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x10(%rsi), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x18(%rsi), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0xa0(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0xa8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0xb0(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0xb8(%rsp)+        movq    0x20(%rsp), %rax+        subq    0x20(%rsi), %rax+        movq    0x28(%rsp), %rcx+        sbbq    0x28(%rsi), %rcx+        movq    0x30(%rsp), %r8+        sbbq    0x30(%rsi), %r8+        movq    0x38(%rsp), %r9+        sbbq    0x38(%rsi), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x20(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x28(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x30(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x38(%rsp)+        movq    0xa0(%rsp), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0xb8(%rsp), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0xb0(%rsp), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0xb8(%rsp), %rbx+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0xa8(%rsp), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0xb0(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0xb8(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        movq    0x20(%rsp), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0x28(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x38(%rsp), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x30(%rsp), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x20(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0x38(%rsp), %rbx+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0x28(%rsp), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0x30(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0x38(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        movq    (%rsi), %rbx+        movq    0x60(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x70(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x78(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x8(%rsi), %rbx+        xorl    %r13d, %r13d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x10(%rsi), %rbx+        xorl    %r15d, %r15d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x18(%rsi), %rbx+        xorl    %r8d, %r8d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    0x40(%rsp), %rbx+        movq    0x60(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x70(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x78(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x48(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x50(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x58(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x40(%rsp)+        movq    %r13, 0x48(%rsp)+        movq    %r14, 0x50(%rsp)+        movq    %r15, 0x58(%rsp)+        movq    (%rsp), %rax+        subq    0x80(%rsp), %rax+        movq    0x8(%rsp), %rcx+        sbbq    0x88(%rsp), %rcx+        movq    0x10(%rsp), %r8+        sbbq    0x90(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x98(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, (%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x18(%rsp)+        movq    0x40(%rsp), %rax+        subq    0x80(%rsp), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x88(%rsp), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x90(%rsp), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x98(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x60(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x68(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x70(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x78(%rsp)+        movq    0x40(%rsi), %rbx+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x48(%rsi), %rbx+        xorl    %r13d, %r13d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x50(%rsi), %rbx+        xorl    %r15d, %r15d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x58(%rsi), %rbx+        xorl    %r8d, %r8d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xa0(%rsp)+        movq    %r13, 0xa8(%rsp)+        movq    %r14, 0xb0(%rsp)+        movq    %r15, 0xb8(%rsp)+        movq    (%rsp), %rax+        subq    0x40(%rsp), %rax+        movq    0x8(%rsp), %rcx+        sbbq    0x48(%rsp), %rcx+        movq    0x10(%rsp), %r8+        sbbq    0x50(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x58(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, (%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x18(%rsp)+        movq    0x80(%rsp), %rax+        subq    (%rsp), %rax+        movq    0x88(%rsp), %rcx+        sbbq    0x8(%rsp), %rcx+        movq    0x90(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x98(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x80(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x88(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x90(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x98(%rsp)+        movq    0x20(%rsi), %rbx+        movq    0x60(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x70(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x78(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x28(%rsi), %rbx+        xorl    %r13d, %r13d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x30(%rsi), %rbx+        xorl    %r15d, %r15d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x38(%rsi), %rbx+        xorl    %r8d, %r8d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        movq    0x80(%rsp), %rbx+        movq    0x20(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x28(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x30(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x38(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x88(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    0x20(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x30(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x90(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    0x20(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x30(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x98(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    0x20(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x30(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    0x80(%rsp), %rax+        subq    0x60(%rsp), %rax+        movq    0x88(%rsp), %rcx+        sbbq    0x68(%rsp), %rcx+        movq    0x90(%rsp), %r8+        sbbq    0x70(%rsp), %r8+        movq    0x98(%rsp), %r9+        sbbq    0x78(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x80(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x88(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x90(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x98(%rsp)+        movq    0x40(%rsi), %rax+        movq    0x48(%rsi), %rdx+        orq     0x50(%rsi), %rax+        orq     0x58(%rsi), %rdx+        orq     %rdx, %rax+        movq    (%rsp), %r8+        movq    0x0(%rbp), %rax+        cmoveq  %rax, %r8+        movq    0x8(%rsp), %r9+        movq    0x8(%rbp), %rax+        cmoveq  %rax, %r9+        movq    0x10(%rsp), %r10+        movq    0x10(%rbp), %rax+        cmoveq  %rax, %r10+        movq    0x18(%rsp), %r11+        movq    0x18(%rbp), %rax+        cmoveq  %rax, %r11+        movq    0x80(%rsp), %r12+        movq    0x20(%rbp), %rax+        cmoveq  %rax, %r12+        movq    0x88(%rsp), %r13+        movq    0x28(%rbp), %rax+        cmoveq  %rax, %r13+        movq    0x90(%rsp), %r14+        movq    0x30(%rbp), %rax+        cmoveq  %rax, %r14+        movq    0x98(%rsp), %r15+        movq    0x38(%rbp), %rax+        cmoveq  %rax, %r15+        movq    %r8, (%rdi)+        movq    %r9, 0x8(%rdi)+        movq    %r10, 0x10(%rdi)+        movq    %r11, 0x18(%rdi)+        movq    %r12, 0x20(%rdi)+        movq    %r13, 0x28(%rdi)+        movq    %r14, 0x30(%rdi)+        movq    %r15, 0x38(%rdi)+        movq    0xa0(%rsp), %r8+        movq    0xa8(%rsp), %r9+        movq    0xb0(%rsp), %r10+        movq    0xb8(%rsp), %r11+        movl    $0x1, %eax+        cmoveq  %rax, %r8+        movabsq $0xffffffff00000000, %rax+        cmoveq  %rax, %r9+        movq    $0xffffffffffffffff, %rax+        cmoveq  %rax, %r10+        movl    $0xfffffffe, %eax+        cmoveq  %rax, %r11+        movq    %r8, 0x40(%rdi)+        movq    %r9, 0x48(%rdi)+        movq    %r10, 0x50(%rdi)+        movq    %r11, 0x58(%rdi)+        CFI_INC_RSP(192)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/p256_scalarmulbase.S view
@@ -0,0 +1,3571 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Scalar multiplication for precomputed point on NIST curve P-256+// Input scalar[4], blocksize, table[]; output res[8]+//+// extern void p256_scalarmulbase+//   (uint64_t res[static 8],+//    const uint64_t scalar[static 4],+//    uint64_t blocksize,+//    const uint64_t *table);+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-256, the input argument "table" is expected to be a table of+// multiples of the point P in Montgomery-affine form, with each block+// corresponding to "blocksize" bits of the scalar as follows, where+// B = 2^{blocksize-1} (e.g. B = 8 for blocksize = 4):+//+// For each i,j with blocksize * i <= 256 and 1 <= j <= B+// the multiple 2^{blocksize * i} * j * P is stored at+// tab[8 * (B * i + (j - 1))], considered as uint64_t pointers+// or tab + 64 * (B * i + (j - 1)) as byte pointers.+//+// Standard x86-64 ABI: RDI = res, RSI = scalar, RDX = blocksize, RCX = table+// Microsoft x64 ABI:   RCX = res, RDX = scalar, R8 = blocksize, R9 = table+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_scalarmulbase)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_scalarmulbase)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_scalarmulbase)+++        .text+        .balign 32++// Size of individual field elements++#define NUMSIZE 32++// Intermediate variables on the stack. The last z2, z3 values can+// safely be overlaid on "nacc", which is no longer needed at the end.+// Uppercase syntactic variants make x86_att version simpler to generate++#define RSCALAR (0*NUMSIZE)+#define ACC (1*NUMSIZE)+#define NACC (4*NUMSIZE)+#define TABENT (7*NUMSIZE)+#define Z2 (4*NUMSIZE)+#define Z3 (5*NUMSIZE)++#define rscalar RSCALAR(%rsp)+#define acc ACC(%rsp)+#define nacc NACC(%rsp)+#define tabent TABENT(%rsp)++#define z2 Z2(%rsp)+#define z3 Z3(%rsp)++#define res (9*NUMSIZE)(%rsp)+#define blocksize (9*NUMSIZE+8)(%rsp)+#define table (9*NUMSIZE+16)(%rsp)+#define i (9*NUMSIZE+24)(%rsp)+#define bf (9*NUMSIZE+32)(%rsp)+#define cf (9*NUMSIZE+40)(%rsp)+#define j (9*NUMSIZE+48)(%rsp)++#define NSPACE 11*NUMSIZE++S2N_BN_SYMBOL(p256_scalarmulbase):+        CFI_START+        _CET_ENDBR++// The Windows version literally calls the standard ABI version.+// This simplifies the proofs since subroutine offsets are fixed.++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+        movq    %r9, %rcx+        CFI_CALL(Lp256_scalarmulbase_standard)+        CFI_POP(%rsi)+        CFI_POP(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_scalarmulbase)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_standard)++Lp256_scalarmulbase_standard:+        CFI_START+#endif++// Real start of the standard ABI code.++        CFI_PUSH(%r15)+        CFI_PUSH(%r14)+        CFI_PUSH(%r13)+        CFI_PUSH(%r12)+        CFI_PUSH(%rbp)+        CFI_PUSH(%rbx)++        CFI_DEC_RSP(NSPACE)++// Preserve the input arguments except the scalar, since that gets absorbed+// immediately. The "table" value subsequently gets shifted up each iteration+// of the loop, while "res" and "blocksize" are static throughout.++        movq    %rdi, res+        movq    %rdx, blocksize+        movq    %rcx, table++// Load the digits of group order n_256 = [%r15;%r14;%r13;%r12]++        movq    $0xf3b9cac2fc632551, %r12+        movq    $0xbce6faada7179e84, %r13+        movq    $0xffffffffffffffff, %r14+        movq    $0xffffffff00000000, %r15++// First, reduce the input scalar mod n_256, i.e. conditionally subtract n_256+// Store it to "rscalar" (reduced scalar)++        movq    (%rsi), %r8+        subq    %r12, %r8+        movq    8(%rsi), %r9+        sbbq    %r13, %r9+        movq    16(%rsi), %r10+        sbbq    %r14, %r10+        movq    24(%rsi), %r11+        sbbq    %r15, %r11++        cmovcq  (%rsi), %r8+        cmovcq  8(%rsi), %r9+        cmovcq  16(%rsi), %r10+        cmovcq  24(%rsi), %r11++        movq    %r8, RSCALAR(%rsp)+        movq    %r9, RSCALAR+8(%rsp)+        movq    %r10, RSCALAR+16(%rsp)+        movq    %r11, RSCALAR+24(%rsp)++// Initialize the accumulator to all zeros and the "carry flag" cf to 0++        xorl    %eax, %eax++        movq    %rax, ACC(%rsp)+        movq    %rax, ACC+8(%rsp)+        movq    %rax, ACC+16(%rsp)+        movq    %rax, ACC+24(%rsp)+        movq    %rax, ACC+32(%rsp)+        movq    %rax, ACC+40(%rsp)+        movq    %rax, ACC+48(%rsp)+        movq    %rax, ACC+56(%rsp)+        movq    %rax, ACC+64(%rsp)+        movq    %rax, ACC+72(%rsp)+        movq    %rax, ACC+80(%rsp)+        movq    %rax, ACC+88(%rsp)++        movq    %rax, cf++// Main loop over {i >= 0 | blocksize * i <= 256}. Note the non-strict+// inequality, to allow top carry for any choices of blocksize.++        movq    %rax, i++Lp256_scalarmulbase_loop:++// The next raw bitfield is bf = bitfield(blocksize * i,blocksize) + cf,+// adding in the deferred carry cf. We then shift the whole scalar right+// by blocksize so we can keep picking bitfield(0,blocksize).++        movq    RSCALAR(%rsp), %r8+        movq    RSCALAR+8(%rsp), %r9+        movq    RSCALAR+16(%rsp), %r10+        movq    RSCALAR+24(%rsp), %r11++        movq    blocksize, %rcx+        movl    $1, %eax+        shlq    %cl, %rax+        decq    %rax+        andq    %r8, %rax++        shrdq   %cl, %r9, %r8+        shrdq   %cl, %r10, %r9+        shrdq   %cl, %r11, %r10+        shrq    %cl, %r11++        addq    cf, %rax+        movq    %rax, bf++        movq   %r8, RSCALAR(%rsp)+        movq   %r9, RSCALAR+8(%rsp)+        movq   %r10, RSCALAR+16(%rsp)+        movq   %r11, RSCALAR+24(%rsp)++// Now if bf <= B we just select entry j, unnegated and set cf = 0.+// If bf > B we set j = 2 * B - bf and negate the j'th entry, setting cf = 1.+// In either case we ultimately add bf, in the latter case with deferred+// carry as 2 * B - (2 * B - bf) = bf.++        movl    $1, %eax+        movq    blocksize, %rcx+        shlq    %cl, %rax+        movq    %rax, %rbx+        shrq    $1, %rax++        subq    bf, %rbx+        cmpq    bf, %rax++        cmovncq bf, %rbx+        sbbq    %rax, %rax+        movq    %rbx, j+        negq    %rax+        movq    %rax, cf++// Load table entry j - 1 for nonzero j in constant-time style.++        movq    blocksize, %rcx+        decq    %rcx+        movl    $1, %esi+        shlq    %cl, %rsi+        movq    j, %r12+        movq    table, %rbp++Lp256_scalarmulbase_tabloop:+        subq    $1, %r12+        cmovzq  (%rbp), %rax+        cmovzq  8(%rbp), %rbx+        cmovzq  16(%rbp), %rcx+        cmovzq  24(%rbp), %rdx+        cmovzq  32(%rbp), %r8+        cmovzq  40(%rbp), %r9+        cmovzq  48(%rbp), %r10+        cmovzq  56(%rbp), %r11++        addq    $64, %rbp+        decq    %rsi+        jnz     Lp256_scalarmulbase_tabloop++        movq    %rbp, table++// Before storing back, optionally negate the y coordinate of the table entry++        xorl    %r14d, %r14d+        leaq    -1(%r14), %r12+        movq    $0x00000000ffffffff, %r15+        movq    %r15, %r13+        negq    %r15++        subq    %r8, %r12+        sbbq    %r9, %r13+        sbbq    %r10, %r14+        sbbq    %r11, %r15++        movq    %rax, TABENT(%rsp)+        movq    %rbx, TABENT+8(%rsp)+        movq    %rcx, TABENT+16(%rsp)+        movq    %rdx, TABENT+24(%rsp)++        movq    cf, %rax+        testq   %rax, %rax+        cmovnzq %r12, %r8+        cmovnzq %r13, %r9+        cmovnzq %r14, %r10+        cmovnzq %r15, %r11++        movq    %r8, TABENT+32(%rsp)+        movq    %r9, TABENT+40(%rsp)+        movq    %r10, TABENT+48(%rsp)+        movq    %r11, TABENT+56(%rsp)++// Add the adjusted table point to the accumulator++        leaq    NACC(%rsp), %rdi+        leaq    ACC(%rsp), %rsi+        leaq    TABENT(%rsp), %rdx+        CFI_CALL(Lp256_scalarmulbase_local_p256_montjmixadd)++// However, only commit that update to the accumulator if j is nonzero,+// because the mixed addition function does not handle this case directly,+// and in any case we didn't choose the table entry appropriately.++        movq    j, %rax+        testq   %rax, %rax++        movq    ACC(%rsp), %rax+        cmovnzq NACC(%rsp), %rax+        movq    %rax, ACC(%rsp)++        movq    ACC+8(%rsp), %rax+        cmovnzq NACC+8(%rsp), %rax+        movq    %rax, ACC+8(%rsp)++        movq    ACC+16(%rsp), %rax+        cmovnzq NACC+16(%rsp), %rax+        movq    %rax, ACC+16(%rsp)++        movq    ACC+24(%rsp), %rax+        cmovnzq NACC+24(%rsp), %rax+        movq    %rax, ACC+24(%rsp)++        movq    ACC+32(%rsp), %rax+        cmovnzq NACC+32(%rsp), %rax+        movq    %rax, ACC+32(%rsp)++        movq    ACC+40(%rsp), %rax+        cmovnzq NACC+40(%rsp), %rax+        movq    %rax, ACC+40(%rsp)++        movq    ACC+48(%rsp), %rax+        cmovnzq NACC+48(%rsp), %rax+        movq    %rax, ACC+48(%rsp)++        movq    ACC+56(%rsp), %rax+        cmovnzq NACC+56(%rsp), %rax+        movq    %rax, ACC+56(%rsp)++        movq    ACC+64(%rsp), %rax+        cmovnzq NACC+64(%rsp), %rax+        movq    %rax, ACC+64(%rsp)++        movq    ACC+72(%rsp), %rax+        cmovnzq NACC+72(%rsp), %rax+        movq    %rax, ACC+72(%rsp)++        movq    ACC+80(%rsp), %rax+        cmovnzq NACC+80(%rsp), %rax+        movq    %rax, ACC+80(%rsp)++        movq    ACC+88(%rsp), %rax+        cmovnzq NACC+88(%rsp), %rax+        movq    %rax, ACC+88(%rsp)++// Loop while blocksize * i <= 256++        movq    i, %rax+        incq    %rax+        movq    %rax, i++        imulq   blocksize, %rax+        cmpq    $257, %rax+        jc      Lp256_scalarmulbase_loop++// That's the end of the main loop, and we just need to translate+// back from the Jacobian representation to affine. First of all,+// let z2 = 1/z^2 and z3 = 1/z^3, both without Montgomery form++        leaq    Z2(%rsp), %rdi+        leaq    ACC+64(%rsp), %rsi+        CFI_CALL(Lp256_scalarmulbase_local_montsqr_p256)++        leaq    Z3(%rsp), %rdi+        leaq    ACC+64(%rsp), %rsi+        leaq    Z2(%rsp), %rdx+        CFI_CALL(Lp256_scalarmulbase_local_montmul_p256)++        leaq    Z2(%rsp), %rdi+        leaq    Z3(%rsp), %rsi+        CFI_CALL(Lp256_scalarmulbase_local_demont_p256)++        leaq    Z3(%rsp), %rdi+        leaq    Z2(%rsp), %rsi+        CFI_CALL(Lp256_scalarmulbase_local_inv_p256)++        leaq    Z2(%rsp), %rdi+        leaq    ACC+64(%rsp), %rsi+        leaq    Z3(%rsp), %rdx+        CFI_CALL(Lp256_scalarmulbase_local_montmul_p256)++// Convert back from Jacobian (X, Y, Z) |-> (X/Z^2, Y/Z^3)++        movq    res, %rdi+        leaq    ACC(%rsp), %rsi+        leaq    Z2(%rsp), %rdx+        movq    %rdi, %rbx+        CFI_CALL(Lp256_scalarmulbase_local_montmul_p256)++        leaq    32(%rbx), %rdi+        leaq    ACC+32(%rsp), %rsi+        leaq    Z3(%rsp), %rdx+        CFI_CALL(Lp256_scalarmulbase_local_montmul_p256)++// Restore stack and registers and return++        CFI_INC_RSP(NSPACE)+        CFI_POP(%rbx)+        CFI_POP(%rbp)+        CFI_POP(%r12)+        CFI_POP(%r13)+        CFI_POP(%r14)+        CFI_POP(%r15)+        CFI_RET++#if WINDOWS_ABI+S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_standard)+#else+S2N_BN_SIZE_DIRECTIVE(p256_scalarmulbase)+#endif++// Local copies of subroutines, complete clones at the moment++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_local_demont_p256)++Lp256_scalarmulbase_local_demont_p256:+        CFI_START+        CFI_PUSH(%rbx)+        movq    (%rsi), %r8+        movq    0x8(%rsi), %r9+        movq    0x10(%rsi), %r10+        movq    0x18(%rsi), %r11+        xorq    %rbx, %rbx+        xorq    %rsi, %rsi+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rcx+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+        mulxq   %r9, %rax, %rcx+        adcxq   %rax, %r10+        adoxq   %rcx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rcx+        adcxq   %rax, %r11+        adoxq   %rcx, %rbx+        mulxq   %r9, %rax, %rcx+        adcxq   %rax, %rbx+        adoxq   %rcx, %rsi+        movl    $0x0, %r8d+        adcxq   %r8, %rsi+        xorq    %r9, %r9+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rcx+        adcxq   %rax, %r11+        adoxq   %rcx, %rbx+        mulxq   %r11, %rax, %rcx+        adcxq   %rax, %rbx+        adoxq   %rcx, %rsi+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rcx+        adcxq   %rax, %rsi+        adoxq   %rcx, %r8+        mulxq   %r11, %rax, %rcx+        adcxq   %rax, %r8+        adoxq   %rcx, %r9+        movl    $0x0, %r10d+        adcxq   %r10, %r9+        movq    %rbx, (%rdi)+        movq    %rsi, 0x8(%rdi)+        movq    %r8, 0x10(%rdi)+        movq    %r9, 0x18(%rdi)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_local_demont_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_local_inv_p256)++Lp256_scalarmulbase_local_inv_p256:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(240)+        movq    %rdi, 0xe0(%rsp)+        xorl    %ecx, %ecx+        movl    $0xffffffff, %edx+        movq    %rdx, %rbx+        leaq    -0x1(%rcx), %rax+        negq    %rdx+        movq    %rax, (%rsp)+        movq    %rbx, 0x8(%rsp)+        movq    %rcx, 0x10(%rsp)+        movq    %rdx, 0x18(%rsp)+        movq    %rcx, 0x20(%rsp)+        movq    (%rsi), %r8+        movq    0x8(%rsi), %r9+        movq    0x10(%rsi), %r10+        movq    0x18(%rsi), %r11+        leaq    0x1(%rcx), %rax+        addq    %r8, %rax+        leaq    -0x1(%rdx), %rbx+        adcq    %r9, %rbx+        notq    %rcx+        adcq    %r10, %rcx+        notq    %rdx+        adcq    %r11, %rdx+        cmovaeq %r8, %rax+        cmovaeq %r9, %rbx+        cmovaeq %r10, %rcx+        cmovaeq %r11, %rdx+        movq    %rax, 0x28(%rsp)+        movq    %rbx, 0x30(%rsp)+        movq    %rcx, 0x38(%rsp)+        movq    %rdx, 0x40(%rsp)+        xorl    %eax, %eax+        movq    %rax, 0x48(%rsp)+        xorl    %eax, %eax+        movq    %rax, 0x50(%rsp)+        movq    %rax, 0x58(%rsp)+        movq    %rax, 0x60(%rsp)+        movq    %rax, 0x68(%rsp)+        movq    $0x4000000000000, %rcx+        movq    %rcx, 0x78(%rsp)+        movq    %rax, 0x80(%rsp)+        movq    %rax, 0x88(%rsp)+        movq    %rax, 0x90(%rsp)+        movq    $0xa,  0xb0(%rsp)+        movq    $0x1,  0xb8(%rsp)+        jmp     Lp256_scalarmulbase_inv_midloop+Lp256_scalarmulbase_inv_loop:+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %rdi+        andq    %r11, %rdi+        addq    %rax, %rdi+        movq    %rdi, 0xa0(%rsp)+        movq    %r12, %rax+        andq    %r13, %rax+        movq    %r14, %rsi+        andq    %r15, %rsi+        addq    %rax, %rsi+        movq    %rsi, 0xa8(%rsp)+        xorl    %ebx, %ebx+        movq    (%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x28(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    (%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x28(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        xorl    %ecx, %ecx+        movq    0x8(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x30(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, (%rsp)+        xorl    %edi, %edi+        movq    0x8(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    0x30(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $0x3b, %rbp, %rsi+        movq    %rsi, 0x28(%rsp)+        xorl    %esi, %esi+        movq    0x10(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    0x38(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rcx, %rbx+        movq    %rbx, 0x8(%rsp)+        xorl    %ebx, %ebx+        movq    0x10(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x38(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $0x3b, %rdi, %rbp+        movq    %rbp, 0x30(%rsp)+        movq    0x18(%rsp), %rax+        xorq    %r9, %rax+        movq    0x20(%rsp), %rbp+        xorq    %r9, %rbp+        andq    %r8, %rbp+        negq    %rbp+        mulq    %r8+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x40(%rsp), %rax+        xorq    %r11, %rax+        movq    0x48(%rsp), %rdx+        xorq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbp+        mulq    %r10+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        shrdq   $0x3b, %rsi, %rcx+        movq    %rcx, 0x10(%rsp)+        shrdq   $0x3b, %rbp, %rsi+        sarq    $0x3b, %rbp+        movq    0x18(%rsp), %rax+        movq    %rsi, 0x18(%rsp)+        movq    0x20(%rsp), %rsi+        movq    %rbp, 0x20(%rsp)+        xorq    %r13, %rax+        xorq    %r13, %rsi+        andq    %r12, %rsi+        negq    %rsi+        mulq    %r12+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        movq    0x40(%rsp), %rax+        xorq    %r15, %rax+        movq    0x48(%rsp), %rdx+        xorq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rsi+        mulq    %r14+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, 0x38(%rsp)+        shrdq   $0x3b, %rsi, %rbx+        movq    %rbx, 0x40(%rsp)+        sarq    $0x3b, %rsi+        movq    %rsi, 0x48(%rsp)+        movq    0xa0(%rsp), %rbx+        movq    0xa8(%rsp), %rbp+        xorl    %ecx, %ecx+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x50(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x50(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x78(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x78(%rsp)+        xorl    %ebx, %ebx+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x80(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    0x58(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, 0x58(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x80(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, 0x80(%rsp)+        xorl    %ecx, %ecx+        movq    0x60(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x88(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x60(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x60(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x88(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x88(%rsp)+        movq    0x68(%rsp), %rax+        xorq    %r9, %rax+        movq    %r9, %rbx+        andq    %r8, %rbx+        negq    %rbx+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x90(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbx+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rbx, %rdx+        movq    0x68(%rsp), %rax+        movq    %rcx, 0x68(%rsp)+        movq    %rdx, 0x70(%rsp)+        xorq    %r13, %rax+        movq    %r13, %rcx+        andq    %r12, %rcx+        negq    %rcx+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rcx+        movq    0x90(%rsp), %rax+        xorq    %r15, %rax+        movq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rcx+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rcx, %rdx+        movq    %rsi, 0x90(%rsp)+        movq    %rdx, 0x98(%rsp)+        movq    $0xe000000000000000, %r8+        addq    0x50(%rsp), %r8+        movq    $0xffffffffffffffff, %r9+        adcq    0x58(%rsp), %r9+        movq    $0x1fffffff, %r10+        adcq    0x60(%rsp), %r10+        movq    $0x2000000000000000, %r11+        adcq    0x68(%rsp), %r11+        movq    $0x1fffffffe0000000, %r12+        adcq    0x70(%rsp), %r12+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        movq    $0xffffffff00000001, %rax+        mulq    %r8+        shrq    $0x20, %r8+        addq    %rbx, %r9+        adcq    %r8, %r10+        adcq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rax, %rax+        movl    $0xffffffff, %ebx+        andq    %rax, %rbx+        movq    $0xffffffff00000001, %rdx+        andq    %rax, %rdx+        subq    %rax, %r9+        movq    %r9, 0x50(%rsp)+        sbbq    %rbx, %r10+        movq    %r10, 0x58(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x60(%rsp)+        sbbq    %rdx, %r12+        movq    %r12, 0x68(%rsp)+        movq    $0xe000000000000000, %r8+        addq    0x78(%rsp), %r8+        movq    $0xffffffffffffffff, %r9+        adcq    0x80(%rsp), %r9+        movq    $0x1fffffff, %r10+        adcq    0x88(%rsp), %r10+        movq    $0x2000000000000000, %r11+        adcq    0x90(%rsp), %r11+        movq    $0x1fffffffe0000000, %r12+        adcq    0x98(%rsp), %r12+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        movq    $0xffffffff00000001, %rax+        mulq    %r8+        shrq    $0x20, %r8+        addq    %rbx, %r9+        adcq    %r8, %r10+        adcq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rax, %rax+        movl    $0xffffffff, %ebx+        andq    %rax, %rbx+        movq    $0xffffffff00000001, %rdx+        andq    %rax, %rdx+        subq    %rax, %r9+        movq    %r9, 0x78(%rsp)+        sbbq    %rbx, %r10+        movq    %r10, 0x80(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x88(%rsp)+        sbbq    %rdx, %r12+        movq    %r12, 0x90(%rsp)+Lp256_scalarmulbase_inv_midloop:+        movq    0xb8(%rsp), %rsi+        movq    (%rsp), %rdx+        movq    0x28(%rsp), %rcx+        movq    %rdx, %rbx+        andq    $0xfffff, %rbx+        movq    $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        andq    $0xfffff, %rcx+        movq    $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        xorl    %ebp, %ebp+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %rdx+        leaq    (%rcx,%rax), %rdi+        shlq    $0x16, %rdx+        shlq    $0x16, %rdi+        sarq    $0x2b, %rdx+        sarq    $0x2b, %rdi+        movq    $0x20000100000, %rax+        leaq    (%rbx,%rax), %rbx+        leaq    (%rcx,%rax), %rcx+        sarq    $0x2a, %rbx+        sarq    $0x2a, %rcx+        movq    %rdx, 0xc0(%rsp)+        movq    %rbx, 0xc8(%rsp)+        movq    %rdi, 0xd0(%rsp)+        movq    %rcx, 0xd8(%rsp)+        movq    (%rsp), %r12+        imulq   %r12, %rdi+        imulq   %rdx, %r12+        movq    0x28(%rsp), %r13+        imulq   %r13, %rbx+        imulq   %rcx, %r13+        addq    %rbx, %r12+        addq    %rdi, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movq    $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movq    $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r10+        shlq    $0x16, %r8+        shlq    $0x16, %r10+        sarq    $0x2b, %r8+        sarq    $0x2b, %r10+        movq    $0x20000100000, %rax+        leaq    (%rbx,%rax), %r15+        leaq    (%rcx,%rax), %r11+        sarq    $0x2a, %r15+        sarq    $0x2a, %r11+        movq    %r13, %rbx+        movq    %r12, %rcx+        imulq   %r8, %r12+        imulq   %r15, %rbx+        addq    %rbx, %r12+        imulq   %r11, %r13+        imulq   %r10, %rcx+        addq    %rcx, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movq    $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movq    $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    0xc0(%rsp), %rax+        imulq   %r8, %rax+        movq    0xd0(%rsp), %rdx+        imulq   %r15, %rdx+        imulq   0xc8(%rsp), %r8+        imulq   0xd8(%rsp), %r15+        addq    %r8, %r15+        leaq    (%rax,%rdx), %r9+        movq    0xc0(%rsp), %rax+        imulq   %r10, %rax+        movq    0xd0(%rsp), %rdx+        imulq   %r11, %rdx+        imulq   0xc8(%rsp), %r10+        imulq   0xd8(%rsp), %r11+        addq    %r10, %r11+        leaq    (%rax,%rdx), %r13+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r12+        shlq    $0x15, %r8+        shlq    $0x15, %r12+        sarq    $0x2b, %r8+        sarq    $0x2b, %r12+        movq    $0x20000100000, %rax+        leaq    (%rbx,%rax), %r10+        leaq    (%rcx,%rax), %r14+        sarq    $0x2b, %r10+        sarq    $0x2b, %r14+        movq    %r9, %rax+        imulq   %r8, %rax+        movq    %r13, %rdx+        imulq   %r10, %rdx+        imulq   %r15, %r8+        imulq   %r11, %r10+        addq    %r8, %r10+        leaq    (%rax,%rdx), %r8+        movq    %r9, %rax+        imulq   %r12, %rax+        movq    %r13, %rdx+        imulq   %r14, %rdx+        imulq   %r15, %r12+        imulq   %r11, %r14+        addq    %r12, %r14+        leaq    (%rax,%rdx), %r12+        movq    %rsi, 0xb8(%rsp)+        decq     0xb0(%rsp)+        jne     Lp256_scalarmulbase_inv_loop+        movq    (%rsp), %rax+        movq    0x28(%rsp), %rcx+        imulq   %r8, %rax+        imulq   %r10, %rcx+        addq    %rcx, %rax+        sarq    $0x3f, %rax+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        xorq    %rax, %r9+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        xorq    %rax, %r11+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        xorq    %rax, %r13+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        xorq    %rax, %r15+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %r12+        andq    %r11, %r12+        addq    %rax, %r12+        xorl    %r13d, %r13d+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r13+        adcq    %rdx, %r14+        movq    0x80(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    0x60(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x88(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x68(%rsp), %rax+        xorq    %r9, %rax+        andq    %r8, %r9+        negq    %r9+        mulq    %r8+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    0x90(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %r9+        mulq    %r10+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    %r12, 0x50(%rsp)+        movq    %r13, 0x58(%rsp)+        movq    %r14, 0x60(%rsp)+        movq    %r15, 0x68(%rsp)+        movq    %r9, 0x70(%rsp)+        movq    $0xe000000000000000, %r8+        addq    0x50(%rsp), %r8+        movq    $0xffffffffffffffff, %r9+        adcq    0x58(%rsp), %r9+        movq    $0x1fffffff, %r10+        adcq    0x60(%rsp), %r10+        movq    $0x2000000000000000, %r11+        adcq    0x68(%rsp), %r11+        movq    $0x1fffffffe0000000, %r12+        adcq    0x70(%rsp), %r12+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        movq    $0xffffffff00000001, %rax+        mulq    %r8+        shrq    $0x20, %r8+        addq    %rbx, %r9+        adcq    %r8, %r10+        adcq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rax, %rax+        movl    $0xffffffff, %ebx+        andq    %rax, %rbx+        movq    $0xffffffff00000001, %rdx+        andq    %rax, %rdx+        subq    %rax, %r9+        movq    %r9, 0x50(%rsp)+        sbbq    %rbx, %r10+        movq    %r10, 0x58(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x60(%rsp)+        sbbq    %rdx, %r12+        movq    %r12, 0x68(%rsp)+        movq    0x50(%rsp), %r8+        movq    0x58(%rsp), %r9+        movq    0x60(%rsp), %r10+        movq    0x68(%rsp), %r11+        movl    $0x1, %eax+        movl    $0xffffffff, %ebx+        leaq    -0x2(%rax), %rcx+        leaq    -0x1(%rbx), %rdx+        notq    %rbx+        addq    %r8, %rax+        adcq    %r9, %rbx+        adcq    %r10, %rcx+        adcq    %r11, %rdx+        cmovaeq %r8, %rax+        cmovaeq %r9, %rbx+        cmovaeq %r10, %rcx+        cmovaeq %r11, %rdx+        movq    0xe0(%rsp), %rdi+        movq    %rax, (%rdi)+        movq    %rbx, 0x8(%rdi)+        movq    %rcx, 0x10(%rdi)+        movq    %rdx, 0x18(%rdi)+        CFI_INC_RSP(240)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_local_inv_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_local_montmul_p256)++Lp256_scalarmulbase_local_montmul_p256:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        movq    %rdx, %rcx+        xorl    %r13d, %r13d+        movq    (%rcx), %rdx+        mulxq   (%rsi), %r8, %r9+        mulxq   0x8(%rsi), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x10(%rsi), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x18(%rsi), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x8(%rcx), %rdx+        xorl    %r14d, %r14d+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x10(%rcx), %rdx+        xorl    %r8d, %r8d+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x18(%rsi), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x18(%rcx), %rdx+        xorl    %r9d, %r9d+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x18(%rsi), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_local_montmul_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_local_montsqr_p256)++Lp256_scalarmulbase_local_montsqr_p256:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        movq    (%rsi), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0x8(%rsi), %r9, %r10+        mulxq   0x18(%rsi), %r11, %r12+        movq    0x10(%rsi), %rdx+        mulxq   0x18(%rsi), %r13, %r14+        xorl    %ebp, %ebp+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x18(%rsi), %rdx+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rbp, %r13+        adoxq   %rbp, %r14+        adcq    %rbp, %r14+        xorl    %ebp, %ebp+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0x8(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x10(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x18(%rsi), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rbp, %r15+        adoxq   %rbp, %r15+        xorl    %ebp, %ebp+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rbp, %r13+        movl    %ebp, %r9d+        adoxq   %rbp, %r9+        adcxq   %rbp, %r9+        addq    %r9, %r14+        adcq    %rbp, %r15+        movl    %ebp, %r8d+        adcq    %rbp, %r8+        xorl    %ebp, %ebp+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %r8+        adcq    %rbp, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rdx), %rdx+        adcq    %r13, %rdx+        leaq    -0x1(%rbp), %rbp+        movq    %rbp, %rax+        adcq    %r14, %rbp+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %rbp, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_local_montsqr_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_local_p256_montjmixadd)++Lp256_scalarmulbase_local_p256_montjmixadd:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(192)+        movq    %rdx, %rbp+        movq    0x40(%rsi), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0x48(%rsi), %r9, %r10+        mulxq   0x58(%rsi), %r11, %r12+        movq    0x50(%rsi), %rdx+        mulxq   0x58(%rsi), %r13, %r14+        xorl    %ecx, %ecx+        mulxq   0x40(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x58(%rsi), %rdx+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        adoxq   %rcx, %r14+        adcq    %rcx, %r14+        xorl    %ecx, %ecx+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0x48(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x50(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x58(%rsi), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rcx, %r15+        adoxq   %rcx, %r15+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        movl    %ecx, %r9d+        adoxq   %rcx, %r9+        adcxq   %rcx, %r9+        addq    %r9, %r14+        adcq    %rcx, %r15+        movl    %ecx, %r8d+        adcq    %rcx, %r8+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rcx, %r15+        adoxq   %rcx, %r8+        adcq    %rcx, %r8+        movl    $0x1, %r8d+        leaq    -0x1(%rdx), %rdx+        leaq    -0x1(%rcx), %rax+        movl    $0xfffffffe, %r11d+        cmoveq  %rcx, %r8+        cmoveq  %rcx, %rdx+        cmoveq  %rcx, %rax+        cmoveq  %rcx, %r11+        addq    %r8, %r12+        adcq    %rdx, %r13+        adcq    %rax, %r14+        adcq    %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        xorl    %r13d, %r13d+        movq    0x20(%rbp), %rdx+        mulxq   0x40(%rsi), %r8, %r9+        mulxq   0x48(%rsi), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x50(%rsi), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x58(%rsi), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x28(%rbp), %rdx+        xorl    %r14d, %r14d+        mulxq   0x40(%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x50(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x58(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x30(%rbp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x40(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x50(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x58(%rsi), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x38(%rbp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x40(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x50(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x58(%rsi), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x20(%rsp)+        movq    %r13, 0x28(%rsp)+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        xorl    %r13d, %r13d+        movq    0x0(%rbp), %rdx+        mulxq   (%rsp), %r8, %r9+        mulxq   0x8(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x10(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x18(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x8(%rbp), %rdx+        xorl    %r14d, %r14d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x10(%rbp), %rdx+        xorl    %r8d, %r8d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x18(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x18(%rbp), %rdx+        xorl    %r9d, %r9d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x18(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x40(%rsp)+        movq    %r13, 0x48(%rsp)+        movq    %r14, 0x50(%rsp)+        movq    %r15, 0x58(%rsp)+        xorl    %r13d, %r13d+        movq    0x20(%rsp), %rdx+        mulxq   (%rsp), %r8, %r9+        mulxq   0x8(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x10(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x18(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x28(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x30(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x18(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x38(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x18(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x20(%rsp)+        movq    %r13, 0x28(%rsp)+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    0x40(%rsp), %rax+        subq    (%rsi), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x8(%rsi), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x10(%rsi), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x18(%rsi), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0xa0(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0xa8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0xb0(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0xb8(%rsp)+        movq    0x20(%rsp), %rax+        subq    0x20(%rsi), %rax+        movq    0x28(%rsp), %rcx+        sbbq    0x28(%rsi), %rcx+        movq    0x30(%rsp), %r8+        sbbq    0x30(%rsi), %r8+        movq    0x38(%rsp), %r9+        sbbq    0x38(%rsi), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x20(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x28(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x30(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x38(%rsp)+        movq    0xa0(%rsp), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0xa8(%rsp), %r9, %r10+        mulxq   0xb8(%rsp), %r11, %r12+        movq    0xb0(%rsp), %rdx+        mulxq   0xb8(%rsp), %r13, %r14+        xorl    %ecx, %ecx+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0xb8(%rsp), %rdx+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        adoxq   %rcx, %r14+        adcq    %rcx, %r14+        xorl    %ecx, %ecx+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0xa8(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0xb0(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0xb8(%rsp), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rcx, %r15+        adoxq   %rcx, %r15+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        movl    %ecx, %r9d+        adoxq   %rcx, %r9+        adcxq   %rcx, %r9+        addq    %r9, %r14+        adcq    %rcx, %r15+        movl    %ecx, %r8d+        adcq    %rcx, %r8+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rcx, %r15+        adoxq   %rcx, %r8+        adcq    %rcx, %r8+        movl    $0x1, %r8d+        leaq    -0x1(%rdx), %rdx+        leaq    -0x1(%rcx), %rax+        movl    $0xfffffffe, %r11d+        cmoveq  %rcx, %r8+        cmoveq  %rcx, %rdx+        cmoveq  %rcx, %rax+        cmoveq  %rcx, %r11+        addq    %r8, %r12+        adcq    %rdx, %r13+        adcq    %rax, %r14+        adcq    %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        movq    0x20(%rsp), %rdx+        mulxq   %rdx, %r8, %r15+        mulxq   0x28(%rsp), %r9, %r10+        mulxq   0x38(%rsp), %r11, %r12+        movq    0x30(%rsp), %rdx+        mulxq   0x38(%rsp), %r13, %r14+        xorl    %ecx, %ecx+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x38(%rsp), %rdx+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        adoxq   %rcx, %r14+        adcq    %rcx, %r14+        xorl    %ecx, %ecx+        adcxq   %r9, %r9+        adoxq   %r15, %r9+        movq    0x28(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x30(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x38(%rsp), %rdx+        mulxq   %rdx, %rax, %r15+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %rcx, %r15+        adoxq   %rcx, %r15+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    $0xffffffff00000001, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %rcx, %r13+        movl    %ecx, %r9d+        adoxq   %rcx, %r9+        adcxq   %rcx, %r9+        addq    %r9, %r14+        adcq    %rcx, %r15+        movl    %ecx, %r8d+        adcq    %rcx, %r8+        xorl    %ecx, %ecx+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    $0xffffffff00000001, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rcx, %r15+        adoxq   %rcx, %r8+        adcq    %rcx, %r8+        movl    $0x1, %ebx+        addq    %r12, %rbx+        leaq    -0x1(%rdx), %rdx+        adcq    %r13, %rdx+        leaq    -0x1(%rcx), %rcx+        movq    %rcx, %rax+        adcq    %r14, %rcx+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rbx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %rcx, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        xorl    %r13d, %r13d+        movq    (%rsi), %rdx+        mulxq   0x60(%rsp), %r8, %r9+        mulxq   0x68(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x70(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x78(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x8(%rsi), %rdx+        xorl    %r14d, %r14d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x78(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x10(%rsi), %rdx+        xorl    %r8d, %r8d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x18(%rsi), %rdx+        xorl    %r9d, %r9d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        xorl    %r13d, %r13d+        movq    0x40(%rsp), %rdx+        mulxq   0x60(%rsp), %r8, %r9+        mulxq   0x68(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x70(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x78(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x48(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x78(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x50(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x58(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x40(%rsp)+        movq    %r13, 0x48(%rsp)+        movq    %r14, 0x50(%rsp)+        movq    %r15, 0x58(%rsp)+        movq    (%rsp), %rax+        subq    0x80(%rsp), %rax+        movq    0x8(%rsp), %rcx+        sbbq    0x88(%rsp), %rcx+        movq    0x10(%rsp), %r8+        sbbq    0x90(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x98(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, (%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x18(%rsp)+        movq    0x40(%rsp), %rax+        subq    0x80(%rsp), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x88(%rsp), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x90(%rsp), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x98(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x60(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x68(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x70(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x78(%rsp)+        xorl    %r13d, %r13d+        movq    0x40(%rsi), %rdx+        mulxq   0xa0(%rsp), %r8, %r9+        mulxq   0xa8(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0xb0(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0xb8(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x48(%rsi), %rdx+        xorl    %r14d, %r14d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x50(%rsi), %rdx+        xorl    %r8d, %r8d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x58(%rsi), %rdx+        xorl    %r9d, %r9d+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xa0(%rsp)+        movq    %r13, 0xa8(%rsp)+        movq    %r14, 0xb0(%rsp)+        movq    %r15, 0xb8(%rsp)+        movq    (%rsp), %rax+        subq    0x40(%rsp), %rax+        movq    0x8(%rsp), %rcx+        sbbq    0x48(%rsp), %rcx+        movq    0x10(%rsp), %r8+        sbbq    0x50(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x58(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, (%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x18(%rsp)+        movq    0x80(%rsp), %rax+        subq    (%rsp), %rax+        movq    0x88(%rsp), %rcx+        sbbq    0x8(%rsp), %rcx+        movq    0x90(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x98(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x80(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x88(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x90(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x98(%rsp)+        xorl    %r13d, %r13d+        movq    0x20(%rsi), %rdx+        mulxq   0x60(%rsp), %r8, %r9+        mulxq   0x68(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x70(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x78(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x28(%rsi), %rdx+        xorl    %r14d, %r14d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x78(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x30(%rsi), %rdx+        xorl    %r8d, %r8d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x38(%rsi), %rdx+        xorl    %r9d, %r9d+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x70(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x78(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        xorl    %r13d, %r13d+        movq    0x80(%rsp), %rdx+        mulxq   0x20(%rsp), %r8, %r9+        mulxq   0x28(%rsp), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x30(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x38(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        adcq    %r13, %r12+        movq    0x88(%rsp), %rdx+        xorl    %r14d, %r14d+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x38(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcq    %r14, %r13+        xorl    %r15d, %r15d+        movq    $0x100000000, %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r8, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r9, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adcxq   %r15, %r13+        adoxq   %r15, %r14+        adcq    %r15, %r14+        movq    0x90(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        adoxq   %r8, %r14+        mulxq   0x38(%rsp), %rax, %rbx+        adcq    %rax, %r13+        adcq    %rbx, %r14+        adcq    %r8, %r15+        movq    0x98(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x28(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r9, %r15+        mulxq   0x38(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r9, %r8+        xorl    %r9d, %r9d+        movq    $0x100000000, %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        notq    %rdx+        leaq    0x2(%rdx), %rdx+        mulxq   %r10, %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   %r11, %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %r9, %r15+        adoxq   %r9, %r8+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rdx+        adcq    %r13, %rdx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rdx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    0x80(%rsp), %rax+        subq    0x60(%rsp), %rax+        movq    0x88(%rsp), %rcx+        sbbq    0x68(%rsp), %rcx+        movq    0x90(%rsp), %r8+        sbbq    0x70(%rsp), %r8+        movq    0x98(%rsp), %r9+        sbbq    0x78(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x80(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x88(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x90(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x98(%rsp)+        movq    0x40(%rsi), %rax+        movq    0x48(%rsi), %rdx+        orq     0x50(%rsi), %rax+        orq     0x58(%rsi), %rdx+        orq     %rdx, %rax+        movq    (%rsp), %r8+        movq    0x0(%rbp), %rax+        cmoveq  %rax, %r8+        movq    0x8(%rsp), %r9+        movq    0x8(%rbp), %rax+        cmoveq  %rax, %r9+        movq    0x10(%rsp), %r10+        movq    0x10(%rbp), %rax+        cmoveq  %rax, %r10+        movq    0x18(%rsp), %r11+        movq    0x18(%rbp), %rax+        cmoveq  %rax, %r11+        movq    0x80(%rsp), %r12+        movq    0x20(%rbp), %rax+        cmoveq  %rax, %r12+        movq    0x88(%rsp), %r13+        movq    0x28(%rbp), %rax+        cmoveq  %rax, %r13+        movq    0x90(%rsp), %r14+        movq    0x30(%rbp), %rax+        cmoveq  %rax, %r14+        movq    0x98(%rsp), %r15+        movq    0x38(%rbp), %rax+        cmoveq  %rax, %r15+        movq    %r8, (%rdi)+        movq    %r9, 0x8(%rdi)+        movq    %r10, 0x10(%rdi)+        movq    %r11, 0x18(%rdi)+        movq    %r12, 0x20(%rdi)+        movq    %r13, 0x28(%rdi)+        movq    %r14, 0x30(%rdi)+        movq    %r15, 0x38(%rdi)+        movq    0xa0(%rsp), %r8+        movq    0xa8(%rsp), %r9+        movq    0xb0(%rsp), %r10+        movq    0xb8(%rsp), %r11+        movl    $0x1, %eax+        cmoveq  %rax, %r8+        movq    $0xffffffff00000000, %rax+        cmoveq  %rax, %r9+        movq    $0xffffffffffffffff, %rax+        cmoveq  %rax, %r10+        movl    $0xfffffffe, %eax+        cmoveq  %rax, %r11+        movq    %r8, 0x40(%rdi)+        movq    %r9, 0x48(%rdi)+        movq    %r10, 0x50(%rdi)+        movq    %r11, 0x58(%rdi)+        CFI_INC_RSP(192)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_local_p256_montjmixadd)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/p256_scalarmulbase_alt.S view
@@ -0,0 +1,4212 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Scalar multiplication for precomputed point on NIST curve P-256+// Input scalar[4], blocksize, table[]; output res[8]+//+// extern void p256_scalarmulbase_alt+//   (uint64_t res[static 8],+//    const uint64_t scalar[static 4],+//    uint64_t blocksize,+//    const uint64_t *table);+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-256, the input argument "table" is expected to be a table of+// multiples of the point P in Montgomery-affine form, with each block+// corresponding to "blocksize" bits of the scalar as follows, where+// B = 2^{blocksize-1} (e.g. B = 8 for blocksize = 4):+//+// For each i,j with blocksize * i <= 256 and 1 <= j <= B+// the multiple 2^{blocksize * i} * j * P is stored at+// tab[8 * (B * i + (j - 1))], considered as uint64_t pointers+// or tab + 64 * (B * i + (j - 1)) as byte pointers.+//+// Standard x86-64 ABI: RDI = res, RSI = scalar, RDX = blocksize, RCX = table+// Microsoft x64 ABI:   RCX = res, RDX = scalar, R8 = blocksize, R9 = table+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p256_scalarmulbase_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p256_scalarmulbase_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p256_scalarmulbase_alt)+++        .text+        .balign 32++// Size of individual field elements++#define NUMSIZE 32++// Intermediate variables on the stack. The last z2, z3 values can+// safely be overlaid on "nacc", which is no longer needed at the end.+// Uppercase syntactic variants make x86_att version simpler to generate++#define RSCALAR (0*NUMSIZE)+#define ACC (1*NUMSIZE)+#define NACC (4*NUMSIZE)+#define TABENT (7*NUMSIZE)+#define Z2 (4*NUMSIZE)+#define Z3 (5*NUMSIZE)++#define rscalar RSCALAR(%rsp)+#define acc ACC(%rsp)+#define nacc NACC(%rsp)+#define tabent TABENT(%rsp)++#define z2 Z2(%rsp)+#define z3 Z3(%rsp)++#define res (9*NUMSIZE)(%rsp)+#define blocksize (9*NUMSIZE+8)(%rsp)+#define table (9*NUMSIZE+16)(%rsp)+#define i (9*NUMSIZE+24)(%rsp)+#define bf (9*NUMSIZE+32)(%rsp)+#define cf (9*NUMSIZE+40)(%rsp)+#define j (9*NUMSIZE+48)(%rsp)++#define NSPACE 11*NUMSIZE++S2N_BN_SYMBOL(p256_scalarmulbase_alt):+        CFI_START+        _CET_ENDBR++// The Windows version literally calls the standard ABI version.+// This simplifies the proofs since subroutine offsets are fixed.++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+        movq    %r9, %rcx+        CFI_CALL(Lp256_scalarmulbase_alt_standard)+        CFI_POP(%rsi)+        CFI_POP(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p256_scalarmulbase_alt)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_alt_standard)++Lp256_scalarmulbase_alt_standard:+        CFI_START+#endif++// Real start of the standard ABI code.++        CFI_PUSH(%r15)+        CFI_PUSH(%r14)+        CFI_PUSH(%r13)+        CFI_PUSH(%r12)+        CFI_PUSH(%rbp)+        CFI_PUSH(%rbx)++        CFI_DEC_RSP(NSPACE)++// Preserve the input arguments except the scalar, since that gets absorbed+// immediately. The "table" value subsequently gets shifted up each iteration+// of the loop, while "res" and "blocksize" are static throughout.++        movq    %rdi, res+        movq    %rdx, blocksize+        movq    %rcx, table++// Load the digits of group order n_256 = [%r15;%r14;%r13;%r12]++        movq    $0xf3b9cac2fc632551, %r12+        movq    $0xbce6faada7179e84, %r13+        movq    $0xffffffffffffffff, %r14+        movq    $0xffffffff00000000, %r15++// First, reduce the input scalar mod n_256, i.e. conditionally subtract n_256+// Store it to "rscalar" (reduced scalar)++        movq    (%rsi), %r8+        subq    %r12, %r8+        movq    8(%rsi), %r9+        sbbq    %r13, %r9+        movq    16(%rsi), %r10+        sbbq    %r14, %r10+        movq    24(%rsi), %r11+        sbbq    %r15, %r11++        cmovcq  (%rsi), %r8+        cmovcq  8(%rsi), %r9+        cmovcq  16(%rsi), %r10+        cmovcq  24(%rsi), %r11++        movq    %r8, RSCALAR(%rsp)+        movq    %r9, RSCALAR+8(%rsp)+        movq    %r10, RSCALAR+16(%rsp)+        movq    %r11, RSCALAR+24(%rsp)++// Initialize the accumulator to all zeros and the "carry flag" cf to 0++        xorl    %eax, %eax++        movq    %rax, ACC(%rsp)+        movq    %rax, ACC+8(%rsp)+        movq    %rax, ACC+16(%rsp)+        movq    %rax, ACC+24(%rsp)+        movq    %rax, ACC+32(%rsp)+        movq    %rax, ACC+40(%rsp)+        movq    %rax, ACC+48(%rsp)+        movq    %rax, ACC+56(%rsp)+        movq    %rax, ACC+64(%rsp)+        movq    %rax, ACC+72(%rsp)+        movq    %rax, ACC+80(%rsp)+        movq    %rax, ACC+88(%rsp)++        movq    %rax, cf++// Main loop over {i >= 0 | blocksize * i <= 256}. Note the non-strict+// inequality, to allow top carry for any choices of blocksize.++        movq    %rax, i++Lp256_scalarmulbase_alt_loop:++// The next raw bitfield is bf = bitfield(blocksize * i,blocksize) + cf,+// adding in the deferred carry cf. We then shift the whole scalar right+// by blocksize so we can keep picking bitfield(0,blocksize).++        movq    RSCALAR(%rsp), %r8+        movq    RSCALAR+8(%rsp), %r9+        movq    RSCALAR+16(%rsp), %r10+        movq    RSCALAR+24(%rsp), %r11++        movq    blocksize, %rcx+        movl    $1, %eax+        shlq    %cl, %rax+        decq    %rax+        andq    %r8, %rax++        shrdq   %cl, %r9, %r8+        shrdq   %cl, %r10, %r9+        shrdq   %cl, %r11, %r10+        shrq    %cl, %r11++        addq    cf, %rax+        movq    %rax, bf++        movq   %r8, RSCALAR(%rsp)+        movq   %r9, RSCALAR+8(%rsp)+        movq   %r10, RSCALAR+16(%rsp)+        movq   %r11, RSCALAR+24(%rsp)++// Now if bf <= B we just select entry j, unnegated and set cf = 0.+// If bf > B we set j = 2 * B - bf and negate the j'th entry, setting cf = 1.+// In either case we ultimately add bf, in the latter case with deferred+// carry as 2 * B - (2 * B - bf) = bf.++        movl    $1, %eax+        movq    blocksize, %rcx+        shlq    %cl, %rax+        movq    %rax, %rbx+        shrq    $1, %rax++        subq    bf, %rbx+        cmpq    bf, %rax++        cmovncq bf, %rbx+        sbbq    %rax, %rax+        movq    %rbx, j+        negq    %rax+        movq    %rax, cf++// Load table entry j - 1 for nonzero j in constant-time style.++        movq    blocksize, %rcx+        decq    %rcx+        movl    $1, %esi+        shlq    %cl, %rsi+        movq    j, %r12+        movq    table, %rbp++Lp256_scalarmulbase_alt_tabloop:+        subq    $1, %r12+        cmovzq  (%rbp), %rax+        cmovzq  8(%rbp), %rbx+        cmovzq  16(%rbp), %rcx+        cmovzq  24(%rbp), %rdx+        cmovzq  32(%rbp), %r8+        cmovzq  40(%rbp), %r9+        cmovzq  48(%rbp), %r10+        cmovzq  56(%rbp), %r11++        addq    $64, %rbp+        decq    %rsi+        jnz     Lp256_scalarmulbase_alt_tabloop++        movq    %rbp, table++// Before storing back, optionally negate the y coordinate of the table entry++        xorl    %r14d, %r14d+        leaq    -1(%r14), %r12+        movq    $0x00000000ffffffff, %r15+        movq    %r15, %r13+        negq    %r15++        subq    %r8, %r12+        sbbq    %r9, %r13+        sbbq    %r10, %r14+        sbbq    %r11, %r15++        movq    %rax, TABENT(%rsp)+        movq    %rbx, TABENT+8(%rsp)+        movq    %rcx, TABENT+16(%rsp)+        movq    %rdx, TABENT+24(%rsp)++        movq    cf, %rax+        testq   %rax, %rax+        cmovnzq %r12, %r8+        cmovnzq %r13, %r9+        cmovnzq %r14, %r10+        cmovnzq %r15, %r11++        movq    %r8, TABENT+32(%rsp)+        movq    %r9, TABENT+40(%rsp)+        movq    %r10, TABENT+48(%rsp)+        movq    %r11, TABENT+56(%rsp)++// Add the adjusted table point to the accumulator++        leaq    NACC(%rsp), %rdi+        leaq    ACC(%rsp), %rsi+        leaq    TABENT(%rsp), %rdx+        CFI_CALL(Lp256_scalarmulbase_alt_local_p256_montjmixadd)++// However, only commit that update to the accumulator if j is nonzero,+// because the mixed addition function does not handle this case directly,+// and in any case we didn't choose the table entry appropriately.++        movq    j, %rax+        testq   %rax, %rax++        movq    ACC(%rsp), %rax+        cmovnzq NACC(%rsp), %rax+        movq    %rax, ACC(%rsp)++        movq    ACC+8(%rsp), %rax+        cmovnzq NACC+8(%rsp), %rax+        movq    %rax, ACC+8(%rsp)++        movq    ACC+16(%rsp), %rax+        cmovnzq NACC+16(%rsp), %rax+        movq    %rax, ACC+16(%rsp)++        movq    ACC+24(%rsp), %rax+        cmovnzq NACC+24(%rsp), %rax+        movq    %rax, ACC+24(%rsp)++        movq    ACC+32(%rsp), %rax+        cmovnzq NACC+32(%rsp), %rax+        movq    %rax, ACC+32(%rsp)++        movq    ACC+40(%rsp), %rax+        cmovnzq NACC+40(%rsp), %rax+        movq    %rax, ACC+40(%rsp)++        movq    ACC+48(%rsp), %rax+        cmovnzq NACC+48(%rsp), %rax+        movq    %rax, ACC+48(%rsp)++        movq    ACC+56(%rsp), %rax+        cmovnzq NACC+56(%rsp), %rax+        movq    %rax, ACC+56(%rsp)++        movq    ACC+64(%rsp), %rax+        cmovnzq NACC+64(%rsp), %rax+        movq    %rax, ACC+64(%rsp)++        movq    ACC+72(%rsp), %rax+        cmovnzq NACC+72(%rsp), %rax+        movq    %rax, ACC+72(%rsp)++        movq    ACC+80(%rsp), %rax+        cmovnzq NACC+80(%rsp), %rax+        movq    %rax, ACC+80(%rsp)++        movq    ACC+88(%rsp), %rax+        cmovnzq NACC+88(%rsp), %rax+        movq    %rax, ACC+88(%rsp)++// Loop while blocksize * i <= 256++        movq    i, %rax+        incq    %rax+        movq    %rax, i++        imulq   blocksize, %rax+        cmpq    $257, %rax+        jc      Lp256_scalarmulbase_alt_loop++// That's the end of the main loop, and we just need to translate+// back from the Jacobian representation to affine. First of all,+// let z2 = 1/z^2 and z3 = 1/z^3, both without Montgomery form++        leaq    Z2(%rsp), %rdi+        leaq    ACC+64(%rsp), %rsi+        CFI_CALL(Lp256_scalarmulbase_alt_local_montsqr_p256)++        leaq    Z3(%rsp), %rdi+        leaq    ACC+64(%rsp), %rsi+        leaq    Z2(%rsp), %rdx+        CFI_CALL(Lp256_scalarmulbase_alt_local_montmul_p256)++        leaq    Z2(%rsp), %rdi+        leaq    Z3(%rsp), %rsi+        CFI_CALL(Lp256_scalarmulbase_alt_local_demont_p256)++        leaq    Z3(%rsp), %rdi+        leaq    Z2(%rsp), %rsi+        CFI_CALL(Lp256_scalarmulbase_alt_local_inv_p256)++        leaq    Z2(%rsp), %rdi+        leaq    ACC+64(%rsp), %rsi+        leaq    Z3(%rsp), %rdx+        CFI_CALL(Lp256_scalarmulbase_alt_local_montmul_p256)++// Convert back from Jacobian (X, Y, Z) |-> (X/Z^2, Y/Z^3)++        movq    res, %rdi+        leaq    ACC(%rsp), %rsi+        leaq    Z2(%rsp), %rdx+        movq    %rdi, %rbx+        CFI_CALL(Lp256_scalarmulbase_alt_local_montmul_p256)++        leaq    32(%rbx), %rdi+        leaq    ACC+32(%rsp), %rsi+        leaq    Z3(%rsp), %rdx+        CFI_CALL(Lp256_scalarmulbase_alt_local_montmul_p256)++// Restore stack and registers and return++        CFI_INC_RSP(NSPACE)+        CFI_POP(%rbx)+        CFI_POP(%rbp)+        CFI_POP(%r12)+        CFI_POP(%r13)+        CFI_POP(%r14)+        CFI_POP(%r15)+        CFI_RET++#if WINDOWS_ABI+S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_alt_standard)+#else+S2N_BN_SIZE_DIRECTIVE(p256_scalarmulbase_alt)+#endif++// Local copies of subroutines, complete clones at the moment++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_alt_local_demont_p256)++Lp256_scalarmulbase_alt_local_demont_p256:+        CFI_START+        movq    (%rsi), %r8+        movq    0x8(%rsi), %r9+        movq    0x10(%rsi), %r10+        movq    0x18(%rsi), %r11+        movabsq $0x100000000, %rcx+        movq    %r8, %rax+        mulq    %rcx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rsi, %rsi+        movq    %r9, %rax+        mulq    %rcx+        subq    %rsi, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rsi, %rsi+        negq    %rcx+        negq    %rsi+        incq    %rcx+        movq    %r8, %rax+        mulq    %rcx+        addq    %rax, %r11+        adcq    %rdx, %rsi+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rax+        mulq    %rcx+        addq    %rax, %rsi+        adcq    %rdx, %r8+        negq    %rcx+        incq    %rcx+        movq    %r10, %rax+        mulq    %rcx+        addq    %rax, %r11+        adcq    %rdx, %rsi+        sbbq    %r9, %r9+        movq    %r11, %rax+        mulq    %rcx+        subq    %r9, %rdx+        addq    %rax, %rsi+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %rcx+        negq    %r9+        incq    %rcx+        movq    %r10, %rax+        mulq    %rcx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rax+        mulq    %rcx+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    %rsi, (%rdi)+        movq    %r8, 0x8(%rdi)+        movq    %r9, 0x10(%rdi)+        movq    %r10, 0x18(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_alt_local_demont_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_alt_local_inv_p256)++Lp256_scalarmulbase_alt_local_inv_p256:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(240)+        movq    %rdi, 0xe0(%rsp)+        xorl    %ecx, %ecx+        movl    $0xffffffff, %edx+        movq    %rdx, %rbx+        leaq    -0x1(%rcx), %rax+        negq    %rdx+        movq    %rax, (%rsp)+        movq    %rbx, 0x8(%rsp)+        movq    %rcx, 0x10(%rsp)+        movq    %rdx, 0x18(%rsp)+        movq    %rcx, 0x20(%rsp)+        movq    (%rsi), %r8+        movq    0x8(%rsi), %r9+        movq    0x10(%rsi), %r10+        movq    0x18(%rsi), %r11+        leaq    0x1(%rcx), %rax+        addq    %r8, %rax+        leaq    -0x1(%rdx), %rbx+        adcq    %r9, %rbx+        notq    %rcx+        adcq    %r10, %rcx+        notq    %rdx+        adcq    %r11, %rdx+        cmovaeq %r8, %rax+        cmovaeq %r9, %rbx+        cmovaeq %r10, %rcx+        cmovaeq %r11, %rdx+        movq    %rax, 0x28(%rsp)+        movq    %rbx, 0x30(%rsp)+        movq    %rcx, 0x38(%rsp)+        movq    %rdx, 0x40(%rsp)+        xorl    %eax, %eax+        movq    %rax, 0x48(%rsp)+        xorl    %eax, %eax+        movq    %rax, 0x50(%rsp)+        movq    %rax, 0x58(%rsp)+        movq    %rax, 0x60(%rsp)+        movq    %rax, 0x68(%rsp)+        movabsq $0x4000000000000, %rcx+        movq    %rcx, 0x78(%rsp)+        movq    %rax, 0x80(%rsp)+        movq    %rax, 0x88(%rsp)+        movq    %rax, 0x90(%rsp)+        movq    $0xa,  0xb0(%rsp)+        movq    $0x1,  0xb8(%rsp)+        jmp     Lp256_scalarmulbase_alt_inv_midloop+Lp256_scalarmulbase_alt_inv_loop:+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %rdi+        andq    %r11, %rdi+        addq    %rax, %rdi+        movq    %rdi, 0xa0(%rsp)+        movq    %r12, %rax+        andq    %r13, %rax+        movq    %r14, %rsi+        andq    %r15, %rsi+        addq    %rax, %rsi+        movq    %rsi, 0xa8(%rsp)+        xorl    %ebx, %ebx+        movq    (%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x28(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    (%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x28(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        xorl    %ecx, %ecx+        movq    0x8(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x30(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, (%rsp)+        xorl    %edi, %edi+        movq    0x8(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        movq    0x30(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rdi+        shrdq   $0x3b, %rbp, %rsi+        movq    %rsi, 0x28(%rsp)+        xorl    %esi, %esi+        movq    0x10(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        movq    0x38(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rcx, %rbx+        movq    %rbx, 0x8(%rsp)+        xorl    %ebx, %ebx+        movq    0x10(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        movq    0x38(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rdi+        adcq    %rdx, %rbx+        shrdq   $0x3b, %rdi, %rbp+        movq    %rbp, 0x30(%rsp)+        movq    0x18(%rsp), %rax+        xorq    %r9, %rax+        movq    0x20(%rsp), %rbp+        xorq    %r9, %rbp+        andq    %r8, %rbp+        negq    %rbp+        mulq    %r8+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x40(%rsp), %rax+        xorq    %r11, %rax+        movq    0x48(%rsp), %rdx+        xorq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbp+        mulq    %r10+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        shrdq   $0x3b, %rsi, %rcx+        movq    %rcx, 0x10(%rsp)+        shrdq   $0x3b, %rbp, %rsi+        sarq    $0x3b, %rbp+        movq    0x18(%rsp), %rax+        movq    %rsi, 0x18(%rsp)+        movq    0x20(%rsp), %rsi+        movq    %rbp, 0x20(%rsp)+        xorq    %r13, %rax+        xorq    %r13, %rsi+        andq    %r12, %rsi+        negq    %rsi+        mulq    %r12+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        movq    0x40(%rsp), %rax+        xorq    %r15, %rax+        movq    0x48(%rsp), %rdx+        xorq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rsi+        mulq    %r14+        addq    %rax, %rbx+        adcq    %rdx, %rsi+        shrdq   $0x3b, %rbx, %rdi+        movq    %rdi, 0x38(%rsp)+        shrdq   $0x3b, %rsi, %rbx+        movq    %rbx, 0x40(%rsp)+        sarq    $0x3b, %rsi+        movq    %rsi, 0x48(%rsp)+        movq    0xa0(%rsp), %rbx+        movq    0xa8(%rsp), %rbp+        xorl    %ecx, %ecx+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x50(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x50(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x78(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x78(%rsp)+        xorl    %ebx, %ebx+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x80(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        xorl    %ebp, %ebp+        movq    0x58(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rcx, 0x58(%rsp)+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    0x80(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rdx, %rbp+        movq    %rsi, 0x80(%rsp)+        xorl    %ecx, %ecx+        movq    0x60(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        movq    0x88(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        xorl    %esi, %esi+        movq    0x60(%rsp), %rax+        xorq    %r13, %rax+        mulq    %r12+        movq    %rbx, 0x60(%rsp)+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    0x88(%rsp), %rax+        xorq    %r15, %rax+        mulq    %r14+        addq    %rax, %rbp+        adcq    %rdx, %rsi+        movq    %rbp, 0x88(%rsp)+        movq    0x68(%rsp), %rax+        xorq    %r9, %rax+        movq    %r9, %rbx+        andq    %r8, %rbx+        negq    %rbx+        mulq    %r8+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        movq    0x90(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %rbx+        mulq    %r10+        addq    %rax, %rcx+        adcq    %rbx, %rdx+        movq    0x68(%rsp), %rax+        movq    %rcx, 0x68(%rsp)+        movq    %rdx, 0x70(%rsp)+        xorq    %r13, %rax+        movq    %r13, %rcx+        andq    %r12, %rcx+        negq    %rcx+        mulq    %r12+        addq    %rax, %rsi+        adcq    %rdx, %rcx+        movq    0x90(%rsp), %rax+        xorq    %r15, %rax+        movq    %r15, %rdx+        andq    %r14, %rdx+        subq    %rdx, %rcx+        mulq    %r14+        addq    %rax, %rsi+        adcq    %rcx, %rdx+        movq    %rsi, 0x90(%rsp)+        movq    %rdx, 0x98(%rsp)+        movabsq $0xe000000000000000, %r8+        addq    0x50(%rsp), %r8+        movq    $0xffffffffffffffff, %r9+        adcq    0x58(%rsp), %r9+        movq    $0x1fffffff, %r10+        adcq    0x60(%rsp), %r10+        movabsq $0x2000000000000000, %r11+        adcq    0x68(%rsp), %r11+        movabsq $0x1fffffffe0000000, %r12+        adcq    0x70(%rsp), %r12+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %r8+        shrq    $0x20, %r8+        addq    %rbx, %r9+        adcq    %r8, %r10+        adcq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rax, %rax+        movl    $0xffffffff, %ebx+        andq    %rax, %rbx+        movabsq $0xffffffff00000001, %rdx+        andq    %rax, %rdx+        subq    %rax, %r9+        movq    %r9, 0x50(%rsp)+        sbbq    %rbx, %r10+        movq    %r10, 0x58(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x60(%rsp)+        sbbq    %rdx, %r12+        movq    %r12, 0x68(%rsp)+        movabsq $0xe000000000000000, %r8+        addq    0x78(%rsp), %r8+        movq    $0xffffffffffffffff, %r9+        adcq    0x80(%rsp), %r9+        movq    $0x1fffffff, %r10+        adcq    0x88(%rsp), %r10+        movabsq $0x2000000000000000, %r11+        adcq    0x90(%rsp), %r11+        movabsq $0x1fffffffe0000000, %r12+        adcq    0x98(%rsp), %r12+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %r8+        shrq    $0x20, %r8+        addq    %rbx, %r9+        adcq    %r8, %r10+        adcq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rax, %rax+        movl    $0xffffffff, %ebx+        andq    %rax, %rbx+        movabsq $0xffffffff00000001, %rdx+        andq    %rax, %rdx+        subq    %rax, %r9+        movq    %r9, 0x78(%rsp)+        sbbq    %rbx, %r10+        movq    %r10, 0x80(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x88(%rsp)+        sbbq    %rdx, %r12+        movq    %r12, 0x90(%rsp)+Lp256_scalarmulbase_alt_inv_midloop:+        movq    0xb8(%rsp), %rsi+        movq    (%rsp), %rdx+        movq    0x28(%rsp), %rcx+        movq    %rdx, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        xorl    %ebp, %ebp+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %rdx+        leaq    (%rcx,%rax), %rdi+        shlq    $0x16, %rdx+        shlq    $0x16, %rdi+        sarq    $0x2b, %rdx+        sarq    $0x2b, %rdi+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %rbx+        leaq    (%rcx,%rax), %rcx+        sarq    $0x2a, %rbx+        sarq    $0x2a, %rcx+        movq    %rdx, 0xc0(%rsp)+        movq    %rbx, 0xc8(%rsp)+        movq    %rdi, 0xd0(%rsp)+        movq    %rcx, 0xd8(%rsp)+        movq    (%rsp), %r12+        imulq   %r12, %rdi+        imulq   %rdx, %r12+        movq    0x28(%rsp), %r13+        imulq   %r13, %rbx+        imulq   %rcx, %r13+        addq    %rbx, %r12+        addq    %rdi, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r10+        shlq    $0x16, %r8+        shlq    $0x16, %r10+        sarq    $0x2b, %r8+        sarq    $0x2b, %r10+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r15+        leaq    (%rcx,%rax), %r11+        sarq    $0x2a, %r15+        sarq    $0x2a, %r11+        movq    %r13, %rbx+        movq    %r12, %rcx+        imulq   %r8, %r12+        imulq   %r15, %rbx+        addq    %rbx, %r12+        imulq   %r11, %r13+        imulq   %r10, %rcx+        addq    %rcx, %r13+        sarq    $0x14, %r12+        sarq    $0x14, %r13+        movq    %r12, %rbx+        andq    $0xfffff, %rbx+        movabsq $0xfffffe0000000000, %rax+        orq     %rax, %rbx+        movq    %r13, %rcx+        andq    $0xfffff, %rcx+        movabsq $0xc000000000000000, %rax+        orq     %rax, %rcx+        movq    0xc0(%rsp), %rax+        imulq   %r8, %rax+        movq    0xd0(%rsp), %rdx+        imulq   %r15, %rdx+        imulq   0xc8(%rsp), %r8+        imulq   0xd8(%rsp), %r15+        addq    %r8, %r15+        leaq    (%rax,%rdx), %r9+        movq    0xc0(%rsp), %rax+        imulq   %r10, %rax+        movq    0xd0(%rsp), %rdx+        imulq   %r11, %rdx+        imulq   0xc8(%rsp), %r10+        imulq   0xd8(%rsp), %r11+        addq    %r10, %r11+        leaq    (%rax,%rdx), %r13+        movq    $0xfffffffffffffffe, %rax+        movl    $0x2, %edx+        movq    %rbx, %rdi+        movq    %rax, %r8+        testq   %rsi, %rsi+        cmovs   %rbp, %r8+        testq   $0x1, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        cmovs   %rbp, %r8+        movq    %rbx, %rdi+        testq   %rdx, %rcx+        cmoveq  %rbp, %r8+        cmoveq  %rbp, %rdi+        sarq    $1, %rcx+        xorq    %r8, %rdi+        xorq    %r8, %rsi+        btq     $0x3f, %r8+        cmovbq  %rcx, %rbx+        movq    %rax, %r8+        subq    %rax, %rsi+        leaq    (%rcx,%rdi), %rcx+        sarq    $1, %rcx+        movl    $0x100000, %eax+        leaq    (%rbx,%rax), %r8+        leaq    (%rcx,%rax), %r12+        shlq    $0x15, %r8+        shlq    $0x15, %r12+        sarq    $0x2b, %r8+        sarq    $0x2b, %r12+        movabsq $0x20000100000, %rax+        leaq    (%rbx,%rax), %r10+        leaq    (%rcx,%rax), %r14+        sarq    $0x2b, %r10+        sarq    $0x2b, %r14+        movq    %r9, %rax+        imulq   %r8, %rax+        movq    %r13, %rdx+        imulq   %r10, %rdx+        imulq   %r15, %r8+        imulq   %r11, %r10+        addq    %r8, %r10+        leaq    (%rax,%rdx), %r8+        movq    %r9, %rax+        imulq   %r12, %rax+        movq    %r13, %rdx+        imulq   %r14, %rdx+        imulq   %r15, %r12+        imulq   %r11, %r14+        addq    %r12, %r14+        leaq    (%rax,%rdx), %r12+        movq    %rsi, 0xb8(%rsp)+        decq     0xb0(%rsp)+        jne     Lp256_scalarmulbase_alt_inv_loop+        movq    (%rsp), %rax+        movq    0x28(%rsp), %rcx+        imulq   %r8, %rax+        imulq   %r10, %rcx+        addq    %rcx, %rax+        sarq    $0x3f, %rax+        movq    %r8, %r9+        sarq    $0x3f, %r9+        xorq    %r9, %r8+        subq    %r9, %r8+        xorq    %rax, %r9+        movq    %r10, %r11+        sarq    $0x3f, %r11+        xorq    %r11, %r10+        subq    %r11, %r10+        xorq    %rax, %r11+        movq    %r12, %r13+        sarq    $0x3f, %r13+        xorq    %r13, %r12+        subq    %r13, %r12+        xorq    %rax, %r13+        movq    %r14, %r15+        sarq    $0x3f, %r15+        xorq    %r15, %r14+        subq    %r15, %r14+        xorq    %rax, %r15+        movq    %r8, %rax+        andq    %r9, %rax+        movq    %r10, %r12+        andq    %r11, %r12+        addq    %rax, %r12+        xorl    %r13d, %r13d+        movq    0x50(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x78(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movq    0x58(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r13+        adcq    %rdx, %r14+        movq    0x80(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    0x60(%rsp), %rax+        xorq    %r9, %rax+        mulq    %r8+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x88(%rsp), %rax+        xorq    %r11, %rax+        mulq    %r10+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x68(%rsp), %rax+        xorq    %r9, %rax+        andq    %r8, %r9+        negq    %r9+        mulq    %r8+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    0x90(%rsp), %rax+        xorq    %r11, %rax+        movq    %r11, %rdx+        andq    %r10, %rdx+        subq    %rdx, %r9+        mulq    %r10+        addq    %rax, %r15+        adcq    %rdx, %r9+        movq    %r12, 0x50(%rsp)+        movq    %r13, 0x58(%rsp)+        movq    %r14, 0x60(%rsp)+        movq    %r15, 0x68(%rsp)+        movq    %r9, 0x70(%rsp)+        movabsq $0xe000000000000000, %r8+        addq    0x50(%rsp), %r8+        movq    $0xffffffffffffffff, %r9+        adcq    0x58(%rsp), %r9+        movq    $0x1fffffff, %r10+        adcq    0x60(%rsp), %r10+        movabsq $0x2000000000000000, %r11+        adcq    0x68(%rsp), %r11+        movabsq $0x1fffffffe0000000, %r12+        adcq    0x70(%rsp), %r12+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %r8+        shrq    $0x20, %r8+        addq    %rbx, %r9+        adcq    %r8, %r10+        adcq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rax, %rax+        movl    $0xffffffff, %ebx+        andq    %rax, %rbx+        movabsq $0xffffffff00000001, %rdx+        andq    %rax, %rdx+        subq    %rax, %r9+        movq    %r9, 0x50(%rsp)+        sbbq    %rbx, %r10+        movq    %r10, 0x58(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x60(%rsp)+        sbbq    %rdx, %r12+        movq    %r12, 0x68(%rsp)+        movq    0x50(%rsp), %r8+        movq    0x58(%rsp), %r9+        movq    0x60(%rsp), %r10+        movq    0x68(%rsp), %r11+        movl    $0x1, %eax+        movl    $0xffffffff, %ebx+        leaq    -0x2(%rax), %rcx+        leaq    -0x1(%rbx), %rdx+        notq    %rbx+        addq    %r8, %rax+        adcq    %r9, %rbx+        adcq    %r10, %rcx+        adcq    %r11, %rdx+        cmovaeq %r8, %rax+        cmovaeq %r9, %rbx+        cmovaeq %r10, %rcx+        cmovaeq %r11, %rdx+        movq    0xe0(%rsp), %rdi+        movq    %rax, (%rdi)+        movq    %rbx, 0x8(%rdi)+        movq    %rcx, 0x10(%rdi)+        movq    %rdx, 0x18(%rdi)+        CFI_INC_RSP(240)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_alt_local_inv_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_alt_local_montmul_p256)++Lp256_scalarmulbase_alt_local_montmul_p256:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        movq    %rdx, %rcx+        movq    (%rcx), %rbx+        movq    (%rsi), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x8(%rsi), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x10(%rsi), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x18(%rsi), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x8(%rcx), %rbx+        xorl    %r13d, %r13d+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x10(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x18(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x10(%rcx), %rbx+        xorl    %r15d, %r15d+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x10(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x18(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x18(%rcx), %rbx+        xorl    %r8d, %r8d+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x10(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x18(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_alt_local_montmul_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_alt_local_montsqr_p256)++Lp256_scalarmulbase_alt_local_montsqr_p256:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        movq    (%rsi), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0x8(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x18(%rsi), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x10(%rsi), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0x18(%rsi), %rbx+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0x8(%rsi), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0x10(%rsi), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0x18(%rsi), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rdi)+        movq    %r13, 0x8(%rdi)+        movq    %r14, 0x10(%rdi)+        movq    %r15, 0x18(%rdi)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_alt_local_montsqr_p256)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp256_scalarmulbase_alt_local_p256_montjmixadd)++Lp256_scalarmulbase_alt_local_p256_montjmixadd:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(192)+        movq    %rdx, %rbp+        movq    0x40(%rsi), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0x48(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x58(%rsi), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x50(%rsi), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x40(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0x58(%rsi), %rbx+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0x48(%rsi), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0x50(%rsi), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0x58(%rsi), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        movq    0x20(%rbp), %rbx+        movq    0x40(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x48(%rsi), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x50(%rsi), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x58(%rsi), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x28(%rbp), %rbx+        xorl    %r13d, %r13d+        movq    0x40(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x50(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x58(%rsi), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x30(%rbp), %rbx+        xorl    %r15d, %r15d+        movq    0x40(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x50(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x58(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x38(%rbp), %rbx+        xorl    %r8d, %r8d+        movq    0x40(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x50(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x58(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x20(%rsp)+        movq    %r13, 0x28(%rsp)+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    0x0(%rbp), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x10(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x18(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x8(%rbp), %rbx+        xorl    %r13d, %r13d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x10(%rbp), %rbx+        xorl    %r15d, %r15d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x18(%rbp), %rbx+        xorl    %r8d, %r8d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x40(%rsp)+        movq    %r13, 0x48(%rsp)+        movq    %r14, 0x50(%rsp)+        movq    %r15, 0x58(%rsp)+        movq    0x20(%rsp), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x10(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x18(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x28(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x30(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x38(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x20(%rsp)+        movq    %r13, 0x28(%rsp)+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    0x40(%rsp), %rax+        subq    (%rsi), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x8(%rsi), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x10(%rsi), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x18(%rsi), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0xa0(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0xa8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0xb0(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0xb8(%rsp)+        movq    0x20(%rsp), %rax+        subq    0x20(%rsi), %rax+        movq    0x28(%rsp), %rcx+        sbbq    0x28(%rsi), %rcx+        movq    0x30(%rsp), %r8+        sbbq    0x30(%rsi), %r8+        movq    0x38(%rsp), %r9+        sbbq    0x38(%rsi), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x20(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x28(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x30(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x38(%rsp)+        movq    0xa0(%rsp), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0xb8(%rsp), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0xb0(%rsp), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0xb8(%rsp), %rbx+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0xa8(%rsp), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0xb0(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0xb8(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        movq    0x20(%rsp), %rax+        movq    %rax, %rbx+        mulq    %rax+        movq    %rax, %r8+        movq    %rdx, %r15+        movq    0x28(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x38(%rsp), %rax+        movq    %rax, %r13+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x30(%rsp), %rax+        movq    %rax, %rbx+        mulq    %r13+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x20(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    0x38(%rsp), %rbx+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorl    %ecx, %ecx+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %rcx, %rcx+        movq    0x28(%rsp), %rax+        mulq    %rax+        addq    %r15, %r9+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        movq    0x30(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r15, %r15+        movq    0x38(%rsp), %rax+        mulq    %rax+        negq    %r15+        adcq    %rax, %r14+        adcq    %rcx, %rdx+        movq    %rdx, %r15+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rcx, %rcx+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        xorl    %r8d, %r8d+        movq    %r9, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r8, %r14+        adcq    %r8, %r15+        adcq    %r8, %r8+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        xorl    %r9d, %r9d+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        leaq    -0x1(%rbx), %rbx+        adcq    %r13, %rbx+        leaq    -0x1(%r9), %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, (%rsp)+        movq    %r13, 0x8(%rsp)+        movq    %r14, 0x10(%rsp)+        movq    %r15, 0x18(%rsp)+        movq    (%rsi), %rbx+        movq    0x60(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x70(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x78(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x8(%rsi), %rbx+        xorl    %r13d, %r13d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x10(%rsi), %rbx+        xorl    %r15d, %r15d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x18(%rsi), %rbx+        xorl    %r8d, %r8d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    0x40(%rsp), %rbx+        movq    0x60(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x70(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x78(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x48(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x50(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x58(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x40(%rsp)+        movq    %r13, 0x48(%rsp)+        movq    %r14, 0x50(%rsp)+        movq    %r15, 0x58(%rsp)+        movq    (%rsp), %rax+        subq    0x80(%rsp), %rax+        movq    0x8(%rsp), %rcx+        sbbq    0x88(%rsp), %rcx+        movq    0x10(%rsp), %r8+        sbbq    0x90(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x98(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, (%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x18(%rsp)+        movq    0x40(%rsp), %rax+        subq    0x80(%rsp), %rax+        movq    0x48(%rsp), %rcx+        sbbq    0x88(%rsp), %rcx+        movq    0x50(%rsp), %r8+        sbbq    0x90(%rsp), %r8+        movq    0x58(%rsp), %r9+        sbbq    0x98(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x60(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x68(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x70(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x78(%rsp)+        movq    0x40(%rsi), %rbx+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x48(%rsi), %rbx+        xorl    %r13d, %r13d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x50(%rsi), %rbx+        xorl    %r15d, %r15d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x58(%rsi), %rbx+        xorl    %r8d, %r8d+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0xa0(%rsp)+        movq    %r13, 0xa8(%rsp)+        movq    %r14, 0xb0(%rsp)+        movq    %r15, 0xb8(%rsp)+        movq    (%rsp), %rax+        subq    0x40(%rsp), %rax+        movq    0x8(%rsp), %rcx+        sbbq    0x48(%rsp), %rcx+        movq    0x10(%rsp), %r8+        sbbq    0x50(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x58(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, (%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x8(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x18(%rsp)+        movq    0x80(%rsp), %rax+        subq    (%rsp), %rax+        movq    0x88(%rsp), %rcx+        sbbq    0x8(%rsp), %rcx+        movq    0x90(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x98(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x80(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x88(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x90(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x98(%rsp)+        movq    0x20(%rsi), %rbx+        movq    0x60(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x70(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x78(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x28(%rsi), %rbx+        xorl    %r13d, %r13d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x30(%rsi), %rbx+        xorl    %r15d, %r15d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x38(%rsi), %rbx+        xorl    %r8d, %r8d+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x70(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x60(%rsp)+        movq    %r13, 0x68(%rsp)+        movq    %r14, 0x70(%rsp)+        movq    %r15, 0x78(%rsp)+        movq    0x80(%rsp), %rbx+        movq    0x20(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x28(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x30(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x38(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x88(%rsp), %rbx+        xorl    %r13d, %r13d+        movq    0x20(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r14, %r14+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r14, %r14+        movq    0x30(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r14, %r14+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %r14, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        xorl    %r14d, %r14d+        movabsq $0x100000000, %rbx+        movq    %r8, %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r15, %r15+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r8, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r15, %r15+        movq    %r9, %rax+        mulq    %rbx+        subq    %r15, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x90(%rsp), %rbx+        xorl    %r15d, %r15d+        movq    0x20(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x30(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x98(%rsp), %rbx+        xorl    %r8d, %r8d+        movq    0x20(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x30(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        xorl    %r9d, %r9d+        movabsq $0x100000000, %rbx+        movq    %r10, %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rcx, %rcx+        notq    %rbx+        leaq    0x2(%rbx), %rbx+        movq    %r10, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rcx, %rcx+        movq    %r11, %rax+        mulq    %rbx+        subq    %rcx, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r9, %r8+        movl    $0x1, %ecx+        addq    %r12, %rcx+        decq    %rbx+        adcq    %r13, %rbx+        decq    %r9+        movq    %r9, %rax+        adcq    %r14, %r9+        movl    $0xfffffffe, %r11d+        adcq    %r15, %r11+        adcq    %r8, %rax+        cmovbq  %rcx, %r12+        cmovbq  %rbx, %r13+        cmovbq  %r9, %r14+        cmovbq  %r11, %r15+        movq    %r12, 0x80(%rsp)+        movq    %r13, 0x88(%rsp)+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    0x80(%rsp), %rax+        subq    0x60(%rsp), %rax+        movq    0x88(%rsp), %rcx+        sbbq    0x68(%rsp), %rcx+        movq    0x90(%rsp), %r8+        sbbq    0x70(%rsp), %r8+        movq    0x98(%rsp), %r9+        sbbq    0x78(%rsp), %r9+        movl    $0xffffffff, %r10d+        sbbq    %r11, %r11+        xorq    %rdx, %rdx+        andq    %r11, %r10+        subq    %r10, %rdx+        addq    %r11, %rax+        movq    %rax, 0x80(%rsp)+        adcq    %r10, %rcx+        movq    %rcx, 0x88(%rsp)+        adcq    $0x0, %r8+        movq    %r8, 0x90(%rsp)+        adcq    %rdx, %r9+        movq    %r9, 0x98(%rsp)+        movq    0x40(%rsi), %rax+        movq    0x48(%rsi), %rdx+        orq     0x50(%rsi), %rax+        orq     0x58(%rsi), %rdx+        orq     %rdx, %rax+        movq    (%rsp), %r8+        movq    0x0(%rbp), %rax+        cmoveq  %rax, %r8+        movq    0x8(%rsp), %r9+        movq    0x8(%rbp), %rax+        cmoveq  %rax, %r9+        movq    0x10(%rsp), %r10+        movq    0x10(%rbp), %rax+        cmoveq  %rax, %r10+        movq    0x18(%rsp), %r11+        movq    0x18(%rbp), %rax+        cmoveq  %rax, %r11+        movq    0x80(%rsp), %r12+        movq    0x20(%rbp), %rax+        cmoveq  %rax, %r12+        movq    0x88(%rsp), %r13+        movq    0x28(%rbp), %rax+        cmoveq  %rax, %r13+        movq    0x90(%rsp), %r14+        movq    0x30(%rbp), %rax+        cmoveq  %rax, %r14+        movq    0x98(%rsp), %r15+        movq    0x38(%rbp), %rax+        cmoveq  %rax, %r15+        movq    %r8, (%rdi)+        movq    %r9, 0x8(%rdi)+        movq    %r10, 0x10(%rdi)+        movq    %r11, 0x18(%rdi)+        movq    %r12, 0x20(%rdi)+        movq    %r13, 0x28(%rdi)+        movq    %r14, 0x30(%rdi)+        movq    %r15, 0x38(%rdi)+        movq    0xa0(%rsp), %r8+        movq    0xa8(%rsp), %r9+        movq    0xb0(%rsp), %r10+        movq    0xb8(%rsp), %r11+        movl    $0x1, %eax+        cmoveq  %rax, %r8+        movabsq $0xffffffff00000000, %rax+        cmoveq  %rax, %r9+        movq    $0xffffffffffffffff, %rax+        cmoveq  %rax, %r10+        movl    $0xfffffffe, %eax+        cmoveq  %rax, %r11+        movq    %r8, 0x40(%rdi)+        movq    %r9, 0x48(%rdi)+        movq    %r10, 0x50(%rdi)+        movq    %r11, 0x58(%rdi)+        CFI_INC_RSP(192)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp256_scalarmulbase_alt_local_p256_montjmixadd)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/p384_montjscalarmul.S view
@@ -0,0 +1,7418 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Montgomery-Jacobian form scalar multiplication for P-384+// Input scalar[6], point[18]; output res[18]+//+// extern void p384_montjscalarmul+//   (uint64_t res[static 18],+//    const uint64_t scalar[static 6],+//    const uint64_t point[static 18]);+//+// This function is a variant of its affine point version p384_scalarmul.+// Here, input and output points are assumed to be in Jacobian form with+// their coordinates in the Montgomery domain. Thus, if priming indicates+// Montgomery form, x' = (2^384 * x) mod p_384 etc., each point argument+// is a triple (x',y',z') representing the affine point (x/z^2,y/z^3) when+// z' is nonzero or the point at infinity (group identity) if z' = 0.+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-384, returns a representation of n * P. If the result is the+// point at infinity (either because the input point was or because the+// scalar was a multiple of p_384) then the output is guaranteed to+// represent the point at infinity, i.e. to have its z coordinate zero.+//+// Standard x86-64 ABI: RDI = res, RSI = scalar, RDX = point+// Microsoft x64 ABI:   RCX = res, RDX = scalar, R8 = point+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p384_montjscalarmul)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p384_montjscalarmul)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p384_montjscalarmul)+++        .text+        .balign 32++// Size of individual field elements++#define NUMSIZE 48+#define JACSIZE (3*NUMSIZE)++// Intermediate variables on the stack.+// The table is 16 entries, each of size JACSIZE = 3 * NUMSIZE+// Uppercase syntactic variants make x86_att version simpler to generate.++#define SCALARB (0*NUMSIZE)+#define scalarb (0*NUMSIZE)(%rsp)+#define ACC (1*NUMSIZE)+#define acc (1*NUMSIZE)(%rsp)+#define TABENT (4*NUMSIZE)+#define tabent (4*NUMSIZE)(%rsp)++#define TAB (7*NUMSIZE)+#define tab (7*NUMSIZE)(%rsp)++#define res (55*NUMSIZE)(%rsp)++#define NSPACE 56*NUMSIZE++// Avoid using .rep for the sake of the BoringSSL/AWS-LC delocator,+// which doesn't accept repetitions, assembler macros etc.++#define selectblock_xz(I)                         \+        cmpq    $I, %rdi ;                           \+        cmovzq  TAB+JACSIZE*(I-1)(%rsp), %rax ;     \+        cmovzq  TAB+JACSIZE*(I-1)+8(%rsp), %rbx ;   \+        cmovzq  TAB+JACSIZE*(I-1)+16(%rsp), %rcx ;  \+        cmovzq  TAB+JACSIZE*(I-1)+24(%rsp), %rdx ;  \+        cmovzq  TAB+JACSIZE*(I-1)+32(%rsp), %r8 ;   \+        cmovzq  TAB+JACSIZE*(I-1)+40(%rsp), %r9 ;   \+        cmovzq  TAB+JACSIZE*(I-1)+96(%rsp), %r10 ;  \+        cmovzq  TAB+JACSIZE*(I-1)+104(%rsp), %r11 ; \+        cmovzq  TAB+JACSIZE*(I-1)+112(%rsp), %r12 ; \+        cmovzq  TAB+JACSIZE*(I-1)+120(%rsp), %r13 ; \+        cmovzq  TAB+JACSIZE*(I-1)+128(%rsp), %r14 ; \+        cmovzq  TAB+JACSIZE*(I-1)+136(%rsp), %r15++#define selectblock_y(I)                          \+        cmpq    $I, %rdi ;                           \+        cmovzq  TAB+JACSIZE*(I-1)+48(%rsp), %rax ;  \+        cmovzq  TAB+JACSIZE*(I-1)+56(%rsp), %rbx ;  \+        cmovzq  TAB+JACSIZE*(I-1)+64(%rsp), %rcx ;  \+        cmovzq  TAB+JACSIZE*(I-1)+72(%rsp), %rdx ;  \+        cmovzq  TAB+JACSIZE*(I-1)+80(%rsp), %r8 ;   \+        cmovzq  TAB+JACSIZE*(I-1)+88(%rsp), %r9++S2N_BN_SYMBOL(p384_montjscalarmul):+        CFI_START+        _CET_ENDBR++// The Windows version literally calls the standard ABI version.+// This simplifies the proofs since subroutine offsets are fixed.++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+        CFI_CALL(Lp384_montjscalarmul_standard)+        CFI_POP(%rsi)+        CFI_POP(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p384_montjscalarmul)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp384_montjscalarmul_standard)++Lp384_montjscalarmul_standard:+        CFI_START+#endif++// Real start of the standard ABI code.++        CFI_PUSH(%r15)+        CFI_PUSH(%r14)+        CFI_PUSH(%r13)+        CFI_PUSH(%r12)+        CFI_PUSH(%rbp)+        CFI_PUSH(%rbx)++        CFI_DEC_RSP(NSPACE)++// Preserve the "res" input argument; others get processed early.++        movq    %rdi, res++// Reduce the input scalar mod n_384, i.e. conditionally subtract n_384.+// Store it to "scalarb".++        movq    (%rsi), %r8+        movq    $0xecec196accc52973, %rax+        subq    %rax, %r8+        movq    8(%rsi), %r9+        movq    $0x581a0db248b0a77a, %rax+        sbbq    %rax, %r9+        movq    16(%rsi), %r10+        movq    $0xc7634d81f4372ddf, %rax+        sbbq    %rax, %r10+        movq    24(%rsi), %r11+        movq    $0xffffffffffffffff, %rax+        sbbq    %rax, %r11+        movq    32(%rsi), %r12+        sbbq    %rax, %r12+        movq    40(%rsi), %r13+        sbbq    %rax, %r13++        cmovcq  (%rsi), %r8+        cmovcq  8(%rsi), %r9+        cmovcq  16(%rsi), %r10+        cmovcq  24(%rsi), %r11+        cmovcq  32(%rsi), %r12+        cmovcq  40(%rsi), %r13++        movq    %r8, SCALARB(%rsp)+        movq    %r9, SCALARB+8(%rsp)+        movq    %r10, SCALARB+16(%rsp)+        movq    %r11, SCALARB+24(%rsp)+        movq    %r12, SCALARB+32(%rsp)+        movq    %r13, SCALARB+40(%rsp)++// Set the tab[0] table entry to the input point = 1 * P++        movq    (%rdx), %rax+        movq    %rax, TAB(%rsp)+        movq    8(%rdx), %rax+        movq    %rax, TAB+8(%rsp)+        movq    16(%rdx), %rax+        movq    %rax, TAB+16(%rsp)+        movq    24(%rdx), %rax+        movq    %rax, TAB+24(%rsp)+        movq    32(%rdx), %rax+        movq    %rax, TAB+32(%rsp)+        movq    40(%rdx), %rax+        movq    %rax, TAB+40(%rsp)++        movq    48(%rdx), %rax+        movq    %rax, TAB+48(%rsp)+        movq    56(%rdx), %rax+        movq    %rax, TAB+56(%rsp)+        movq    64(%rdx), %rax+        movq    %rax, TAB+64(%rsp)+        movq    72(%rdx), %rax+        movq    %rax, TAB+72(%rsp)+        movq    80(%rdx), %rax+        movq    %rax, TAB+80(%rsp)+        movq    88(%rdx), %rax+        movq    %rax, TAB+88(%rsp)++        movq    96(%rdx), %rax+        movq    %rax, TAB+96(%rsp)+        movq    104(%rdx), %rax+        movq    %rax, TAB+104(%rsp)+        movq    112(%rdx), %rax+        movq    %rax, TAB+112(%rsp)+        movq    120(%rdx), %rax+        movq    %rax, TAB+120(%rsp)+        movq    128(%rdx), %rax+        movq    %rax, TAB+128(%rsp)+        movq    136(%rdx), %rax+        movq    %rax, TAB+136(%rsp)++// Compute and record tab[1] = 2 * p, ..., tab[15] = 16 * P++        leaq    TAB+JACSIZE*1(%rsp), %rdi+        leaq    TAB(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_p384_montjdouble)++        leaq    TAB+JACSIZE*2(%rsp), %rdi+        leaq    TAB+JACSIZE*1(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp384_montjscalarmul_p384_montjadd)++        leaq    TAB+JACSIZE*3(%rsp), %rdi+        leaq    TAB+JACSIZE*1(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_p384_montjdouble)++        leaq    TAB+JACSIZE*4(%rsp), %rdi+        leaq    TAB+JACSIZE*3(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp384_montjscalarmul_p384_montjadd)++        leaq    TAB+JACSIZE*5(%rsp), %rdi+        leaq    TAB+JACSIZE*2(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_p384_montjdouble)++        leaq    TAB+JACSIZE*6(%rsp), %rdi+        leaq    TAB+JACSIZE*5(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp384_montjscalarmul_p384_montjadd)++        leaq    TAB+JACSIZE*7(%rsp), %rdi+        leaq    TAB+JACSIZE*3(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_p384_montjdouble)++        leaq    TAB+JACSIZE*8(%rsp), %rdi+        leaq    TAB+JACSIZE*7(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp384_montjscalarmul_p384_montjadd)++        leaq    TAB+JACSIZE*9(%rsp), %rdi+        leaq    TAB+JACSIZE*4(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_p384_montjdouble)++        leaq    TAB+JACSIZE*10(%rsp), %rdi+        leaq    TAB+JACSIZE*9(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp384_montjscalarmul_p384_montjadd)++        leaq    TAB+JACSIZE*11(%rsp), %rdi+        leaq    TAB+JACSIZE*5(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_p384_montjdouble)++        leaq    TAB+JACSIZE*12(%rsp), %rdi+        leaq    TAB+JACSIZE*11(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp384_montjscalarmul_p384_montjadd)++        leaq    TAB+JACSIZE*13(%rsp), %rdi+        leaq    TAB+JACSIZE*6(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_p384_montjdouble)++        leaq    TAB+JACSIZE*14(%rsp), %rdi+        leaq    TAB+JACSIZE*13(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp384_montjscalarmul_p384_montjadd)++        leaq    TAB+JACSIZE*15(%rsp), %rdi+        leaq    TAB+JACSIZE*7(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_p384_montjdouble)++// Add the recoding constant sum_i(16 * 32^i) to the scalar to allow signed+// digits. The digits of the constant, in lowest-to-highest order, are as+// follows; they are generated dynamically to use fewer large constant loads.+//+// 0x0842108421084210+// 0x1084210842108421+// 0x2108421084210842+// 0x4210842108421084+// 0x8421084210842108+// 0x0842108421084210++        movq    $0x1084210842108421, %rax+        movq    %rax, %rcx+        shrq    $1, %rax+        movq    SCALARB(%rsp), %r8+        addq    %rax, %r8+        movq    SCALARB+8(%rsp), %r9+        adcq    %rcx, %r9+        leaq    (%rcx,%rcx), %rcx+        movq    SCALARB+16(%rsp), %r10+        adcq    %rcx, %r10+        leaq    (%rcx,%rcx), %rcx+        movq    SCALARB+24(%rsp), %r11+        adcq    %rcx, %r11+        leaq    (%rcx,%rcx), %rcx+        movq    SCALARB+32(%rsp), %r12+        adcq    %rcx, %r12+        movq    SCALARB+40(%rsp), %r13+        adcq    %rax, %r13+        sbbq    %rdi, %rdi+        negq    %rdi++// Record the top bitfield in %rdi then shift the whole scalar left 4 bits+// to align the top of the next bitfield with the MSB (bits 379..383).++        shldq   $4, %r13, %rdi+        shldq   $4, %r12, %r13+        shldq   $4, %r11, %r12+        shldq   $4, %r10, %r11+        shldq   $4, %r9, %r10+        shldq   $4, %r8, %r9+        shlq    $4, %r8++        movq    %r8, SCALARB(%rsp)+        movq    %r9, SCALARB+8(%rsp)+        movq    %r10, SCALARB+16(%rsp)+        movq    %r11, SCALARB+24(%rsp)+        movq    %r12, SCALARB+32(%rsp)+        movq    %r13, SCALARB+40(%rsp)++// Initialize the accumulator to the corresponding entry using constant-time+// lookup in the table. This top digit, uniquely, is not recoded so there is+// no sign adjustment to make. On the x86 integer side we don't have enough+// registers to hold all the fields; this could be better done with SIMD+// registers anyway. So we do x and z coordinates in one sweep, y in another+// (this is a rehearsal for below where we might need to negate the y).++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        xorl    %r13d, %r13d+        xorl    %r14d, %r14d+        xorl    %r15d, %r15d++        selectblock_xz(1)+        selectblock_xz(2)+        selectblock_xz(3)+        selectblock_xz(4)+        selectblock_xz(5)+        selectblock_xz(6)+        selectblock_xz(7)+        selectblock_xz(8)+        selectblock_xz(9)+        selectblock_xz(10)+        selectblock_xz(11)+        selectblock_xz(12)+        selectblock_xz(13)+        selectblock_xz(14)+        selectblock_xz(15)+        selectblock_xz(16)++        movq     %rax, ACC(%rsp)+        movq     %rbx, ACC+8(%rsp)+        movq     %rcx, ACC+16(%rsp)+        movq     %rdx, ACC+24(%rsp)+        movq     %r8, ACC+32(%rsp)+        movq     %r9, ACC+40(%rsp)+        movq     %r10, ACC+96(%rsp)+        movq     %r11, ACC+104(%rsp)+        movq     %r12, ACC+112(%rsp)+        movq     %r13, ACC+120(%rsp)+        movq     %r14, ACC+128(%rsp)+        movq     %r15, ACC+136(%rsp)++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d++        selectblock_y(1)+        selectblock_y(2)+        selectblock_y(3)+        selectblock_y(4)+        selectblock_y(5)+        selectblock_y(6)+        selectblock_y(7)+        selectblock_y(8)+        selectblock_y(9)+        selectblock_y(10)+        selectblock_y(11)+        selectblock_y(12)+        selectblock_y(13)+        selectblock_y(14)+        selectblock_y(15)+        selectblock_y(16)++        movq     %rax, ACC+48(%rsp)+        movq     %rbx, ACC+56(%rsp)+        movq     %rcx, ACC+64(%rsp)+        movq     %rdx, ACC+72(%rsp)+        movq     %r8, ACC+80(%rsp)+        movq     %r9, ACC+88(%rsp)++// Main loop over size-5 bitfields: double 5 times then add signed digit+// At each stage we shift the scalar left by 5 bits so we can simply pick+// the top 5 bits as the bitfield, saving some fiddle over indexing.++        movl    $380, %ebp++Lp384_montjscalarmul_mainloop:+        subq    $5, %rbp++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp384_montjscalarmul_p384_montjdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp384_montjscalarmul_p384_montjdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp384_montjscalarmul_p384_montjdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp384_montjscalarmul_p384_montjdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp384_montjscalarmul_p384_montjdouble)++// Choose the bitfield and adjust it to sign and magnitude++        movq    SCALARB(%rsp), %r8+        movq    SCALARB+8(%rsp), %r9+        movq    SCALARB+16(%rsp), %r10+        movq    SCALARB+24(%rsp), %r11+        movq    SCALARB+32(%rsp), %r12+        movq    SCALARB+40(%rsp), %r13++        movq    %r13, %rdi+        shrq    $59, %rdi+        shldq   $5, %r12, %r13+        shldq   $5, %r11, %r12+        shldq   $5, %r10, %r11+        shldq   $5, %r9, %r10+        shldq   $5, %r8, %r9+        shlq    $5, %r8++        movq    %r8, SCALARB(%rsp)+        movq    %r9, SCALARB+8(%rsp)+        movq    %r10, SCALARB+16(%rsp)+        movq    %r11, SCALARB+24(%rsp)+        movq    %r12, SCALARB+32(%rsp)+        movq    %r13, SCALARB+40(%rsp)++        subq    $16, %rdi+        sbbq    %rsi, %rsi // %rsi = sign of digit (-1 = negative)+        xorq    %rsi, %rdi+        subq    %rsi, %rdi // %rdi = absolute value of digit++// Conditionally select the table entry tab[i-1] = i * P in constant time+// Again, this is done in two sweeps, first doing x and z then y.++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        xorl    %r13d, %r13d+        xorl    %r14d, %r14d+        xorl    %r15d, %r15d++        selectblock_xz(1)+        selectblock_xz(2)+        selectblock_xz(3)+        selectblock_xz(4)+        selectblock_xz(5)+        selectblock_xz(6)+        selectblock_xz(7)+        selectblock_xz(8)+        selectblock_xz(9)+        selectblock_xz(10)+        selectblock_xz(11)+        selectblock_xz(12)+        selectblock_xz(13)+        selectblock_xz(14)+        selectblock_xz(15)+        selectblock_xz(16)++        movq     %rax, TABENT(%rsp)+        movq     %rbx, TABENT+8(%rsp)+        movq     %rcx, TABENT+16(%rsp)+        movq     %rdx, TABENT+24(%rsp)+        movq     %r8, TABENT+32(%rsp)+        movq     %r9, TABENT+40(%rsp)+        movq     %r10, TABENT+96(%rsp)+        movq     %r11, TABENT+104(%rsp)+        movq     %r12, TABENT+112(%rsp)+        movq     %r13, TABENT+120(%rsp)+        movq     %r14, TABENT+128(%rsp)+        movq     %r15, TABENT+136(%rsp)++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d++        selectblock_y(1)+        selectblock_y(2)+        selectblock_y(3)+        selectblock_y(4)+        selectblock_y(5)+        selectblock_y(6)+        selectblock_y(7)+        selectblock_y(8)+        selectblock_y(9)+        selectblock_y(10)+        selectblock_y(11)+        selectblock_y(12)+        selectblock_y(13)+        selectblock_y(14)+        selectblock_y(15)+        selectblock_y(16)++// Store it to "tabent" with the y coordinate optionally negated.+// This is done carefully to give coordinates < p_384 even in+// the degenerate case y = 0 (when z = 0 for points on the curve).+// The digits of the prime p_384 are generated dynamically from+// the zeroth via not/lea to reduce the number of constant loads.++        movq    %rax, %r10+        orq     %rbx, %r10+        movq    %rcx, %r11+        orq     %rdx, %r11+        movq    %r8, %r12+        orq     %r9, %r12+        orq     %r11, %r10+        orq     %r12, %r10+        cmovzq  %r10, %rsi++        movl    $0xffffffff, %r10d+        movq    %r10, %r11+        notq    %r11+        leaq    (%r10,%r11), %r13+        subq    %rax, %r10+        leaq    -1(%r13), %r12+        sbbq    %rbx, %r11+        movq    %r13, %r14+        sbbq    %rcx, %r12+        sbbq    %rdx, %r13+        movq    %r14, %r15+        sbbq    %r8, %r14+        sbbq    %r9, %r15++        testq   %rsi, %rsi+        cmovnzq  %r10, %rax+        cmovnzq  %r11, %rbx+        cmovnzq  %r12, %rcx+        cmovnzq  %r13, %rdx+        cmovnzq  %r14, %r8+        cmovnzq  %r15, %r9++        movq    %rax, TABENT+48(%rsp)+        movq    %rbx, TABENT+56(%rsp)+        movq    %rcx, TABENT+64(%rsp)+        movq    %rdx, TABENT+72(%rsp)+        movq    %r8, TABENT+80(%rsp)+        movq    %r9, TABENT+88(%rsp)++// Add to the accumulator++        leaq    TABENT(%rsp), %rdx+        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp384_montjscalarmul_p384_montjadd)++        testq   %rbp, %rbp+        jne     Lp384_montjscalarmul_mainloop++// That's the end of the main loop, and we just need to copy the+// result in "acc" to the output.++        movq    res, %rdi+        movq    ACC(%rsp), %rax+        movq    %rax, (%rdi)+        movq    ACC+8(%rsp), %rax+        movq    %rax, 8(%rdi)+        movq    ACC+16(%rsp), %rax+        movq    %rax, 16(%rdi)+        movq    ACC+24(%rsp), %rax+        movq    %rax, 24(%rdi)+        movq    ACC+32(%rsp), %rax+        movq    %rax, 32(%rdi)+        movq    ACC+40(%rsp), %rax+        movq    %rax, 40(%rdi)+        movq    ACC+48(%rsp), %rax+        movq    %rax, 48(%rdi)+        movq    ACC+56(%rsp), %rax+        movq    %rax, 56(%rdi)+        movq    ACC+64(%rsp), %rax+        movq    %rax, 64(%rdi)+        movq    ACC+72(%rsp), %rax+        movq    %rax, 72(%rdi)+        movq    ACC+80(%rsp), %rax+        movq    %rax, 80(%rdi)+        movq    ACC+88(%rsp), %rax+        movq    %rax, 88(%rdi)+        movq    ACC+96(%rsp), %rax+        movq    %rax, 96(%rdi)+        movq    ACC+104(%rsp), %rax+        movq    %rax, 104(%rdi)+        movq    ACC+112(%rsp), %rax+        movq    %rax, 112(%rdi)+        movq    ACC+120(%rsp), %rax+        movq    %rax, 120(%rdi)+        movq    ACC+128(%rsp), %rax+        movq    %rax, 128(%rdi)+        movq    ACC+136(%rsp), %rax+        movq    %rax, 136(%rdi)++// Restore stack and registers and return++        CFI_INC_RSP(NSPACE)+        CFI_POP(%rbx)+        CFI_POP(%rbp)+        CFI_POP(%r12)+        CFI_POP(%r13)+        CFI_POP(%r14)+        CFI_POP(%r15)+        CFI_RET++#if WINDOWS_ABI+S2N_BN_SIZE_DIRECTIVE(Lp384_montjscalarmul_standard)+#else+S2N_BN_SIZE_DIRECTIVE(p384_montjscalarmul)+#endif++// Local copies of subroutines, complete clones at the moment++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp384_montjscalarmul_p384_montjadd)++Lp384_montjscalarmul_p384_montjadd:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(352)+        movq    %rsi, 0x150(%rsp)+        movq    %rdx, 0x158(%rsp)+        movq    0x60(%rsi), %rdx+        mulxq   0x68(%rsi), %r9, %r10+        mulxq   0x78(%rsi), %r11, %r12+        mulxq   0x88(%rsi), %r13, %r14+        movq    0x78(%rsi), %rdx+        mulxq   0x80(%rsi), %r15, %rcx+        xorl    %ebp, %ebp+        movq    0x70(%rsi), %rdx+        mulxq   0x60(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x68(%rsi), %rdx+        mulxq   0x78(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x80(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x88(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %rcx+        adcq    %rbp, %rcx+        xorl    %ebp, %ebp+        movq    0x80(%rsi), %rdx+        mulxq   0x60(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    0x70(%rsi), %rdx+        mulxq   0x78(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x80(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x88(%rsi), %rax, %rdx+        adcxq   %rax, %r15+        adoxq   %rdx, %rcx+        movq    0x88(%rsi), %rdx+        mulxq   0x80(%rsi), %rbx, %rbp+        mulxq   0x78(%rsi), %rax, %rdx+        adcxq   %rax, %rcx+        adoxq   %rdx, %rbx+        movl    $0x0, %eax+        adcxq   %rax, %rbx+        adoxq   %rax, %rbp+        adcq    %rax, %rbp+        xorq    %rax, %rax+        movq    0x60(%rsi), %rdx+        mulxq   0x60(%rsi), %r8, %rax+        adcxq   %r9, %r9+        adoxq   %rax, %r9+        movq    0x68(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x70(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x78(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %r15, %r15+        adoxq   %rdx, %r15+        movq    0x80(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %rcx, %rcx+        adoxq   %rax, %rcx+        adcxq   %rbx, %rbx+        adoxq   %rdx, %rbx+        movq    0x88(%rsi), %rdx+        mulxq   %rdx, %rax, %rsi+        adcxq   %rbp, %rbp+        adoxq   %rax, %rbp+        movl    $0x0, %eax+        adcxq   %rax, %rsi+        adoxq   %rax, %rsi+        movq    %rbx, (%rsp)+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r8, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r8+        addq    %rbx, %rax+        adcq    %rdx, %r8+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r9+        sbbq    %r8, %r10+        sbbq    %rbx, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rdx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r9, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r9+        addq    %rbx, %rax+        adcq    %rdx, %r9+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r10+        sbbq    %r9, %r11+        sbbq    %rbx, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rdx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r10, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r10+        addq    %rbx, %rax+        adcq    %rdx, %r10+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r11+        sbbq    %r10, %r12+        sbbq    %rbx, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rdx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r11, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r11+        addq    %rbx, %rax+        adcq    %rdx, %r11+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r12+        sbbq    %r11, %r13+        sbbq    %rbx, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rdx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r12, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r12+        addq    %rbx, %rax+        adcq    %rdx, %r12+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r13+        sbbq    %r12, %r8+        sbbq    %rbx, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rdx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r13, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r13+        addq    %rbx, %rax+        adcq    %rdx, %r13+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r8+        sbbq    %r13, %r9+        sbbq    %rbx, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rdx, %r13+        sbbq    $0x0, %r13+        movq    (%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rsi+        movl    $0x0, %r8d+        movabsq $0xffffffff00000001, %rax+        movl    $0xffffffff, %r9d+        movl    $0x1, %r10d+        cmovaeq %r8, %rax+        cmovaeq %r8, %r9+        cmovaeq %r8, %r10+        addq    %rax, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r8, %rbx+        adcq    %r8, %rbp+        adcq    %r8, %rsi+        movq    %r14, (%rsp)+        movq    %r15, 0x8(%rsp)+        movq    %rcx, 0x10(%rsp)+        movq    %rbx, 0x18(%rsp)+        movq    %rbp, 0x20(%rsp)+        movq    %rsi, 0x28(%rsp)+        movq    0x158(%rsp), %rsi+        movq    0x60(%rsi), %rdx+        mulxq   0x68(%rsi), %r9, %r10+        mulxq   0x78(%rsi), %r11, %r12+        mulxq   0x88(%rsi), %r13, %r14+        movq    0x78(%rsi), %rdx+        mulxq   0x80(%rsi), %r15, %rcx+        xorl    %ebp, %ebp+        movq    0x70(%rsi), %rdx+        mulxq   0x60(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x68(%rsi), %rdx+        mulxq   0x78(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x80(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x88(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %rcx+        adcq    %rbp, %rcx+        xorl    %ebp, %ebp+        movq    0x80(%rsi), %rdx+        mulxq   0x60(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    0x70(%rsi), %rdx+        mulxq   0x78(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x80(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x88(%rsi), %rax, %rdx+        adcxq   %rax, %r15+        adoxq   %rdx, %rcx+        movq    0x88(%rsi), %rdx+        mulxq   0x80(%rsi), %rbx, %rbp+        mulxq   0x78(%rsi), %rax, %rdx+        adcxq   %rax, %rcx+        adoxq   %rdx, %rbx+        movl    $0x0, %eax+        adcxq   %rax, %rbx+        adoxq   %rax, %rbp+        adcq    %rax, %rbp+        xorq    %rax, %rax+        movq    0x60(%rsi), %rdx+        mulxq   0x60(%rsi), %r8, %rax+        adcxq   %r9, %r9+        adoxq   %rax, %r9+        movq    0x68(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x70(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x78(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %r15, %r15+        adoxq   %rdx, %r15+        movq    0x80(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %rcx, %rcx+        adoxq   %rax, %rcx+        adcxq   %rbx, %rbx+        adoxq   %rdx, %rbx+        movq    0x88(%rsi), %rdx+        mulxq   %rdx, %rax, %rsi+        adcxq   %rbp, %rbp+        adoxq   %rax, %rbp+        movl    $0x0, %eax+        adcxq   %rax, %rsi+        adoxq   %rax, %rsi+        movq    %rbx, 0xf0(%rsp)+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r8, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r8+        addq    %rbx, %rax+        adcq    %rdx, %r8+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r9+        sbbq    %r8, %r10+        sbbq    %rbx, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rdx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r9, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r9+        addq    %rbx, %rax+        adcq    %rdx, %r9+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r10+        sbbq    %r9, %r11+        sbbq    %rbx, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rdx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r10, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r10+        addq    %rbx, %rax+        adcq    %rdx, %r10+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r11+        sbbq    %r10, %r12+        sbbq    %rbx, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rdx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r11, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r11+        addq    %rbx, %rax+        adcq    %rdx, %r11+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r12+        sbbq    %r11, %r13+        sbbq    %rbx, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rdx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r12, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r12+        addq    %rbx, %rax+        adcq    %rdx, %r12+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r13+        sbbq    %r12, %r8+        sbbq    %rbx, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rdx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r13, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r13+        addq    %rbx, %rax+        adcq    %rdx, %r13+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r8+        sbbq    %r13, %r9+        sbbq    %rbx, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rdx, %r13+        sbbq    $0x0, %r13+        movq    0xf0(%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rsi+        movl    $0x0, %r8d+        movabsq $0xffffffff00000001, %rax+        movl    $0xffffffff, %r9d+        movl    $0x1, %r10d+        cmovaeq %r8, %rax+        cmovaeq %r8, %r9+        cmovaeq %r8, %r10+        addq    %rax, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r8, %rbx+        adcq    %r8, %rbp+        adcq    %r8, %rsi+        movq    %r14, 0xf0(%rsp)+        movq    %r15, 0xf8(%rsp)+        movq    %rcx, 0x100(%rsp)+        movq    %rbx, 0x108(%rsp)+        movq    %rbp, 0x110(%rsp)+        movq    %rsi, 0x118(%rsp)+        movq    0x150(%rsp), %rsi+        movq    0x158(%rsp), %rcx+        movq    0x30(%rsi), %rdx+        xorl    %r15d, %r15d+        mulxq   0x60(%rcx), %r8, %r9+        mulxq   0x68(%rcx), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0x70(%rcx), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x78(%rcx), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0x80(%rcx), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0x88(%rcx), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x38(%rsi), %rdx+        xorl    %r8d, %r8d+        mulxq   0x60(%rcx), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x68(%rcx), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x70(%rcx), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x78(%rcx), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x80(%rcx), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0x88(%rcx), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x40(%rsi), %rdx+        xorl    %r9d, %r9d+        mulxq   0x60(%rcx), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rcx), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x70(%rcx), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x78(%rcx), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x80(%rcx), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0x88(%rcx), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x48(%rsi), %rdx+        xorl    %r10d, %r10d+        mulxq   0x60(%rcx), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x68(%rcx), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x70(%rcx), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x78(%rcx), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x80(%rcx), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0x88(%rcx), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x50(%rsi), %rdx+        xorl    %r11d, %r11d+        mulxq   0x60(%rcx), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x68(%rcx), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x70(%rcx), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x78(%rcx), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x80(%rcx), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0x88(%rcx), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x58(%rsi), %rdx+        xorl    %r12d, %r12d+        mulxq   0x60(%rcx), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x68(%rcx), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x70(%rcx), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x78(%rcx), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x80(%rcx), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0x88(%rcx), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x120(%rsp)+        movq    %r15, 0x128(%rsp)+        movq    %r8, 0x130(%rsp)+        movq    %r9, 0x138(%rsp)+        movq    %r10, 0x140(%rsp)+        movq    %r11, 0x148(%rsp)+        movq    0x150(%rsp), %rsi+        movq    0x158(%rsp), %rcx+        movq    0x30(%rcx), %rdx+        xorl    %r15d, %r15d+        mulxq   0x60(%rsi), %r8, %r9+        mulxq   0x68(%rsi), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0x70(%rsi), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x78(%rsi), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0x80(%rsi), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0x88(%rsi), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x38(%rcx), %rdx+        xorl    %r8d, %r8d+        mulxq   0x60(%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x68(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x70(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x78(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x80(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0x88(%rsi), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x40(%rcx), %rdx+        xorl    %r9d, %r9d+        mulxq   0x60(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x70(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x78(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x80(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0x88(%rsi), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x48(%rcx), %rdx+        xorl    %r10d, %r10d+        mulxq   0x60(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x68(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x70(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x78(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x80(%rsi), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0x88(%rsi), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x50(%rcx), %rdx+        xorl    %r11d, %r11d+        mulxq   0x60(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x68(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x70(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x78(%rsi), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x80(%rsi), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0x88(%rsi), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x58(%rcx), %rdx+        xorl    %r12d, %r12d+        mulxq   0x60(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x68(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x70(%rsi), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x78(%rsi), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x80(%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0x88(%rsi), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    %r8, 0x40(%rsp)+        movq    %r9, 0x48(%rsp)+        movq    %r10, 0x50(%rsp)+        movq    %r11, 0x58(%rsp)+        movq    0x158(%rsp), %rcx+        movq    (%rcx), %rdx+        xorl    %r15d, %r15d+        mulxq   (%rsp), %r8, %r9+        mulxq   0x8(%rsp), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0x10(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x18(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0x20(%rsp), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0x28(%rsp), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x8(%rcx), %rdx+        xorl    %r8d, %r8d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0x28(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x10(%rcx), %rdx+        xorl    %r9d, %r9d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0x28(%rsp), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x18(%rcx), %rdx+        xorl    %r10d, %r10d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0x28(%rsp), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x20(%rcx), %rdx+        xorl    %r11d, %r11d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0x28(%rsp), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x28(%rcx), %rdx+        xorl    %r12d, %r12d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0x28(%rsp), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x60(%rsp)+        movq    %r15, 0x68(%rsp)+        movq    %r8, 0x70(%rsp)+        movq    %r9, 0x78(%rsp)+        movq    %r10, 0x80(%rsp)+        movq    %r11, 0x88(%rsp)+        movq    0x150(%rsp), %rsi+        movq    (%rsi), %rdx+        xorl    %r15d, %r15d+        mulxq   0xf0(%rsp), %r8, %r9+        mulxq   0xf8(%rsp), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0x100(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x108(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0x110(%rsp), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0x118(%rsp), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x8(%rsi), %rdx+        xorl    %r8d, %r8d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x10(%rsi), %rdx+        xorl    %r9d, %r9d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x18(%rsi), %rdx+        xorl    %r10d, %r10d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x20(%rsi), %rdx+        xorl    %r11d, %r11d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x28(%rsi), %rdx+        xorl    %r12d, %r12d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0xc0(%rsp)+        movq    %r15, 0xc8(%rsp)+        movq    %r8, 0xd0(%rsp)+        movq    %r9, 0xd8(%rsp)+        movq    %r10, 0xe0(%rsp)+        movq    %r11, 0xe8(%rsp)+        movq    0x30(%rsp), %rdx+        xorl    %r15d, %r15d+        mulxq   (%rsp), %r8, %r9+        mulxq   0x8(%rsp), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0x10(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x18(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0x20(%rsp), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0x28(%rsp), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x38(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0x28(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x40(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0x28(%rsp), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x48(%rsp), %rdx+        xorl    %r10d, %r10d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0x28(%rsp), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x50(%rsp), %rdx+        xorl    %r11d, %r11d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0x28(%rsp), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x58(%rsp), %rdx+        xorl    %r12d, %r12d+        mulxq   (%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x8(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x10(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x18(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x20(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0x28(%rsp), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    %r8, 0x40(%rsp)+        movq    %r9, 0x48(%rsp)+        movq    %r10, 0x50(%rsp)+        movq    %r11, 0x58(%rsp)+        movq    0x120(%rsp), %rdx+        xorl    %r15d, %r15d+        mulxq   0xf0(%rsp), %r8, %r9+        mulxq   0xf8(%rsp), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0x100(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x108(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0x110(%rsp), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0x118(%rsp), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x128(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x130(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x138(%rsp), %rdx+        xorl    %r10d, %r10d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x140(%rsp), %rdx+        xorl    %r11d, %r11d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x148(%rsp), %rdx+        xorl    %r12d, %r12d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x120(%rsp)+        movq    %r15, 0x128(%rsp)+        movq    %r8, 0x130(%rsp)+        movq    %r9, 0x138(%rsp)+        movq    %r10, 0x140(%rsp)+        movq    %r11, 0x148(%rsp)+        movq    0x60(%rsp), %rax+        subq    0xc0(%rsp), %rax+        movq    0x68(%rsp), %rdx+        sbbq    0xc8(%rsp), %rdx+        movq    0x70(%rsp), %r8+        sbbq    0xd0(%rsp), %r8+        movq    0x78(%rsp), %r9+        sbbq    0xd8(%rsp), %r9+        movq    0x80(%rsp), %r10+        sbbq    0xe0(%rsp), %r10+        movq    0x88(%rsp), %r11+        sbbq    0xe8(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %esi+        andq    %rsi, %rcx+        xorq    %rsi, %rsi+        subq    %rcx, %rsi+        subq    %rsi, %rax+        movq    %rax, 0xf0(%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0xf8(%rsp)+        sbbq    %rax, %rax+        andq    %rsi, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0x100(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x108(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x110(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x118(%rsp)+        movq    0x30(%rsp), %rax+        subq    0x120(%rsp), %rax+        movq    0x38(%rsp), %rdx+        sbbq    0x128(%rsp), %rdx+        movq    0x40(%rsp), %r8+        sbbq    0x130(%rsp), %r8+        movq    0x48(%rsp), %r9+        sbbq    0x138(%rsp), %r9+        movq    0x50(%rsp), %r10+        sbbq    0x140(%rsp), %r10+        movq    0x58(%rsp), %r11+        sbbq    0x148(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %esi+        andq    %rsi, %rcx+        xorq    %rsi, %rsi+        subq    %rcx, %rsi+        subq    %rsi, %rax+        movq    %rax, 0x30(%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0x38(%rsp)+        sbbq    %rax, %rax+        andq    %rsi, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0x40(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x48(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x50(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x58(%rsp)+        movq    0xf0(%rsp), %rdx+        mulxq   0xf8(%rsp), %r9, %r10+        mulxq   0x108(%rsp), %r11, %r12+        mulxq   0x118(%rsp), %r13, %r14+        movq    0x108(%rsp), %rdx+        mulxq   0x110(%rsp), %r15, %rcx+        xorl    %ebp, %ebp+        movq    0x100(%rsp), %rdx+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0xf8(%rsp), %rdx+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x118(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %rcx+        adcq    %rbp, %rcx+        xorl    %ebp, %ebp+        movq    0x110(%rsp), %rdx+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    0x100(%rsp), %rdx+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x118(%rsp), %rax, %rdx+        adcxq   %rax, %r15+        adoxq   %rdx, %rcx+        movq    0x118(%rsp), %rdx+        mulxq   0x110(%rsp), %rbx, %rbp+        mulxq   0x108(%rsp), %rax, %rdx+        adcxq   %rax, %rcx+        adoxq   %rdx, %rbx+        movl    $0x0, %eax+        adcxq   %rax, %rbx+        adoxq   %rax, %rbp+        adcq    %rax, %rbp+        xorq    %rax, %rax+        movq    0xf0(%rsp), %rdx+        mulxq   0xf0(%rsp), %r8, %rax+        adcxq   %r9, %r9+        adoxq   %rax, %r9+        movq    0xf8(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x100(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x108(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %r15, %r15+        adoxq   %rdx, %r15+        movq    0x110(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %rcx, %rcx+        adoxq   %rax, %rcx+        adcxq   %rbx, %rbx+        adoxq   %rdx, %rbx+        movq    0x118(%rsp), %rdx+        mulxq   %rdx, %rax, %rsi+        adcxq   %rbp, %rbp+        adoxq   %rax, %rbp+        movl    $0x0, %eax+        adcxq   %rax, %rsi+        adoxq   %rax, %rsi+        movq    %rbx, 0x90(%rsp)+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r8, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r8+        addq    %rbx, %rax+        adcq    %rdx, %r8+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r9+        sbbq    %r8, %r10+        sbbq    %rbx, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rdx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r9, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r9+        addq    %rbx, %rax+        adcq    %rdx, %r9+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r10+        sbbq    %r9, %r11+        sbbq    %rbx, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rdx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r10, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r10+        addq    %rbx, %rax+        adcq    %rdx, %r10+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r11+        sbbq    %r10, %r12+        sbbq    %rbx, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rdx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r11, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r11+        addq    %rbx, %rax+        adcq    %rdx, %r11+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r12+        sbbq    %r11, %r13+        sbbq    %rbx, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rdx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r12, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r12+        addq    %rbx, %rax+        adcq    %rdx, %r12+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r13+        sbbq    %r12, %r8+        sbbq    %rbx, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rdx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r13, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r13+        addq    %rbx, %rax+        adcq    %rdx, %r13+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r8+        sbbq    %r13, %r9+        sbbq    %rbx, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rdx, %r13+        sbbq    $0x0, %r13+        movq    0x90(%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rsi+        movl    $0x0, %r8d+        movabsq $0xffffffff00000001, %rax+        movl    $0xffffffff, %r9d+        movl    $0x1, %r10d+        cmovaeq %r8, %rax+        cmovaeq %r8, %r9+        cmovaeq %r8, %r10+        addq    %rax, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r8, %rbx+        adcq    %r8, %rbp+        adcq    %r8, %rsi+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    %rcx, 0xa0(%rsp)+        movq    %rbx, 0xa8(%rsp)+        movq    %rbp, 0xb0(%rsp)+        movq    %rsi, 0xb8(%rsp)+        movq    0x30(%rsp), %rdx+        mulxq   0x38(%rsp), %r9, %r10+        mulxq   0x48(%rsp), %r11, %r12+        mulxq   0x58(%rsp), %r13, %r14+        movq    0x48(%rsp), %rdx+        mulxq   0x50(%rsp), %r15, %rcx+        xorl    %ebp, %ebp+        movq    0x40(%rsp), %rdx+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x38(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x38(%rsp), %rdx+        mulxq   0x48(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x50(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x58(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %rcx+        adcq    %rbp, %rcx+        xorl    %ebp, %ebp+        movq    0x50(%rsp), %rdx+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    0x40(%rsp), %rdx+        mulxq   0x48(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x50(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x58(%rsp), %rax, %rdx+        adcxq   %rax, %r15+        adoxq   %rdx, %rcx+        movq    0x58(%rsp), %rdx+        mulxq   0x50(%rsp), %rbx, %rbp+        mulxq   0x48(%rsp), %rax, %rdx+        adcxq   %rax, %rcx+        adoxq   %rdx, %rbx+        movl    $0x0, %eax+        adcxq   %rax, %rbx+        adoxq   %rax, %rbp+        adcq    %rax, %rbp+        xorq    %rax, %rax+        movq    0x30(%rsp), %rdx+        mulxq   0x30(%rsp), %r8, %rax+        adcxq   %r9, %r9+        adoxq   %rax, %r9+        movq    0x38(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x40(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x48(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %r15, %r15+        adoxq   %rdx, %r15+        movq    0x50(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %rcx, %rcx+        adoxq   %rax, %rcx+        adcxq   %rbx, %rbx+        adoxq   %rdx, %rbx+        movq    0x58(%rsp), %rdx+        mulxq   %rdx, %rax, %rsi+        adcxq   %rbp, %rbp+        adoxq   %rax, %rbp+        movl    $0x0, %eax+        adcxq   %rax, %rsi+        adoxq   %rax, %rsi+        movq    %rbx, (%rsp)+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r8, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r8+        addq    %rbx, %rax+        adcq    %rdx, %r8+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r9+        sbbq    %r8, %r10+        sbbq    %rbx, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rdx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r9, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r9+        addq    %rbx, %rax+        adcq    %rdx, %r9+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r10+        sbbq    %r9, %r11+        sbbq    %rbx, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rdx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r10, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r10+        addq    %rbx, %rax+        adcq    %rdx, %r10+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r11+        sbbq    %r10, %r12+        sbbq    %rbx, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rdx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r11, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r11+        addq    %rbx, %rax+        adcq    %rdx, %r11+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r12+        sbbq    %r11, %r13+        sbbq    %rbx, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rdx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r12, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r12+        addq    %rbx, %rax+        adcq    %rdx, %r12+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r13+        sbbq    %r12, %r8+        sbbq    %rbx, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rdx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r13, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r13+        addq    %rbx, %rax+        adcq    %rdx, %r13+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r8+        sbbq    %r13, %r9+        sbbq    %rbx, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rdx, %r13+        sbbq    $0x0, %r13+        movq    (%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rsi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rsi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rsi+        movq    %r14, (%rsp)+        movq    %r15, 0x8(%rsp)+        movq    %rcx, 0x10(%rsp)+        movq    %rbx, 0x18(%rsp)+        movq    %rbp, 0x20(%rsp)+        movq    %rsi, 0x28(%rsp)+        movq    0xc0(%rsp), %rdx+        xorl    %r15d, %r15d+        mulxq   0x90(%rsp), %r8, %r9+        mulxq   0x98(%rsp), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0xa0(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0xa8(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0xb0(%rsp), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0xb8(%rsp), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0xc8(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0xd0(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0xd8(%rsp), %rdx+        xorl    %r10d, %r10d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0xe0(%rsp), %rdx+        xorl    %r11d, %r11d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0xe8(%rsp), %rdx+        xorl    %r12d, %r12d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0xc0(%rsp)+        movq    %r15, 0xc8(%rsp)+        movq    %r8, 0xd0(%rsp)+        movq    %r9, 0xd8(%rsp)+        movq    %r10, 0xe0(%rsp)+        movq    %r11, 0xe8(%rsp)+        movq    0x60(%rsp), %rdx+        xorl    %r15d, %r15d+        mulxq   0x90(%rsp), %r8, %r9+        mulxq   0x98(%rsp), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0xa0(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0xa8(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0xb0(%rsp), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0xb8(%rsp), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x68(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x70(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x78(%rsp), %rdx+        xorl    %r10d, %r10d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x80(%rsp), %rdx+        xorl    %r11d, %r11d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x88(%rsp), %rdx+        xorl    %r12d, %r12d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x60(%rsp)+        movq    %r15, 0x68(%rsp)+        movq    %r8, 0x70(%rsp)+        movq    %r9, 0x78(%rsp)+        movq    %r10, 0x80(%rsp)+        movq    %r11, 0x88(%rsp)+        movq    (%rsp), %rax+        subq    0xc0(%rsp), %rax+        movq    0x8(%rsp), %rdx+        sbbq    0xc8(%rsp), %rdx+        movq    0x10(%rsp), %r8+        sbbq    0xd0(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0xd8(%rsp), %r9+        movq    0x20(%rsp), %r10+        sbbq    0xe0(%rsp), %r10+        movq    0x28(%rsp), %r11+        sbbq    0xe8(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %esi+        andq    %rsi, %rcx+        xorq    %rsi, %rsi+        subq    %rcx, %rsi+        subq    %rsi, %rax+        movq    %rax, (%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0x8(%rsp)+        sbbq    %rax, %rax+        andq    %rsi, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0x10(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x18(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x20(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x28(%rsp)+        movq    0x60(%rsp), %rax+        subq    0xc0(%rsp), %rax+        movq    0x68(%rsp), %rdx+        sbbq    0xc8(%rsp), %rdx+        movq    0x70(%rsp), %r8+        sbbq    0xd0(%rsp), %r8+        movq    0x78(%rsp), %r9+        sbbq    0xd8(%rsp), %r9+        movq    0x80(%rsp), %r10+        sbbq    0xe0(%rsp), %r10+        movq    0x88(%rsp), %r11+        sbbq    0xe8(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %esi+        andq    %rsi, %rcx+        xorq    %rsi, %rsi+        subq    %rcx, %rsi+        subq    %rsi, %rax+        movq    %rax, 0x90(%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0x98(%rsp)+        sbbq    %rax, %rax+        andq    %rsi, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0xa0(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0xa8(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0xb0(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0xb8(%rsp)+        movq    0x150(%rsp), %rsi+        movq    0x60(%rsi), %rdx+        xorl    %r15d, %r15d+        mulxq   0xf0(%rsp), %r8, %r9+        mulxq   0xf8(%rsp), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0x100(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x108(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0x110(%rsp), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0x118(%rsp), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x68(%rsi), %rdx+        xorl    %r8d, %r8d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x70(%rsi), %rdx+        xorl    %r9d, %r9d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x78(%rsi), %rdx+        xorl    %r10d, %r10d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x80(%rsi), %rdx+        xorl    %r11d, %r11d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x88(%rsi), %rdx+        xorl    %r12d, %r12d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0xf0(%rsp)+        movq    %r15, 0xf8(%rsp)+        movq    %r8, 0x100(%rsp)+        movq    %r9, 0x108(%rsp)+        movq    %r10, 0x110(%rsp)+        movq    %r11, 0x118(%rsp)+        movq    (%rsp), %rax+        subq    0x60(%rsp), %rax+        movq    0x8(%rsp), %rdx+        sbbq    0x68(%rsp), %rdx+        movq    0x10(%rsp), %r8+        sbbq    0x70(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x78(%rsp), %r9+        movq    0x20(%rsp), %r10+        sbbq    0x80(%rsp), %r10+        movq    0x28(%rsp), %r11+        sbbq    0x88(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %esi+        andq    %rsi, %rcx+        xorq    %rsi, %rsi+        subq    %rcx, %rsi+        subq    %rsi, %rax+        movq    %rax, (%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0x8(%rsp)+        sbbq    %rax, %rax+        andq    %rsi, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0x10(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x18(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x20(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x28(%rsp)+        movq    0xc0(%rsp), %rax+        subq    (%rsp), %rax+        movq    0xc8(%rsp), %rdx+        sbbq    0x8(%rsp), %rdx+        movq    0xd0(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0xd8(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movq    0xe0(%rsp), %r10+        sbbq    0x20(%rsp), %r10+        movq    0xe8(%rsp), %r11+        sbbq    0x28(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %esi+        andq    %rsi, %rcx+        xorq    %rsi, %rsi+        subq    %rcx, %rsi+        subq    %rsi, %rax+        movq    %rax, 0xc0(%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0xc8(%rsp)+        sbbq    %rax, %rax+        andq    %rsi, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0xd0(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0xd8(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0xe0(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0xe8(%rsp)+        movq    0x120(%rsp), %rdx+        xorl    %r15d, %r15d+        mulxq   0x90(%rsp), %r8, %r9+        mulxq   0x98(%rsp), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0xa0(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0xa8(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0xb0(%rsp), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0xb8(%rsp), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x128(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x130(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x138(%rsp), %rdx+        xorl    %r10d, %r10d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x140(%rsp), %rdx+        xorl    %r11d, %r11d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x148(%rsp), %rdx+        xorl    %r12d, %r12d+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0xb8(%rsp), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    %r8, 0xa0(%rsp)+        movq    %r9, 0xa8(%rsp)+        movq    %r10, 0xb0(%rsp)+        movq    %r11, 0xb8(%rsp)+        movq    0x158(%rsp), %rcx+        movq    0x60(%rcx), %rdx+        xorl    %r15d, %r15d+        mulxq   0xf0(%rsp), %r8, %r9+        mulxq   0xf8(%rsp), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0x100(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x108(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0x110(%rsp), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0x118(%rsp), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x68(%rcx), %rdx+        xorl    %r8d, %r8d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x70(%rcx), %rdx+        xorl    %r9d, %r9d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x78(%rcx), %rdx+        xorl    %r10d, %r10d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x80(%rcx), %rdx+        xorl    %r11d, %r11d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x88(%rcx), %rdx+        xorl    %r12d, %r12d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0xf0(%rsp)+        movq    %r15, 0xf8(%rsp)+        movq    %r8, 0x100(%rsp)+        movq    %r9, 0x108(%rsp)+        movq    %r10, 0x110(%rsp)+        movq    %r11, 0x118(%rsp)+        movq    0xc0(%rsp), %rdx+        xorl    %r15d, %r15d+        mulxq   0x30(%rsp), %r8, %r9+        mulxq   0x38(%rsp), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0x40(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x48(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0x50(%rsp), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0x58(%rsp), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0xc8(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x38(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x40(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x48(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x50(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0x58(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0xd0(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x38(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x40(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x48(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x50(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0x58(%rsp), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0xd8(%rsp), %rdx+        xorl    %r10d, %r10d+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x38(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x40(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x48(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x50(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0x58(%rsp), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0xe0(%rsp), %rdx+        xorl    %r11d, %r11d+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x38(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x40(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x48(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x50(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0x58(%rsp), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0xe8(%rsp), %rdx+        xorl    %r12d, %r12d+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x38(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x40(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x48(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x50(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0x58(%rsp), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0xc0(%rsp)+        movq    %r15, 0xc8(%rsp)+        movq    %r8, 0xd0(%rsp)+        movq    %r9, 0xd8(%rsp)+        movq    %r10, 0xe0(%rsp)+        movq    %r11, 0xe8(%rsp)+        movq    0xc0(%rsp), %rax+        subq    0x90(%rsp), %rax+        movq    0xc8(%rsp), %rdx+        sbbq    0x98(%rsp), %rdx+        movq    0xd0(%rsp), %r8+        sbbq    0xa0(%rsp), %r8+        movq    0xd8(%rsp), %r9+        sbbq    0xa8(%rsp), %r9+        movq    0xe0(%rsp), %r10+        sbbq    0xb0(%rsp), %r10+        movq    0xe8(%rsp), %r11+        sbbq    0xb8(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %esi+        andq    %rsi, %rcx+        xorq    %rsi, %rsi+        subq    %rcx, %rsi+        subq    %rsi, %rax+        movq    %rax, 0xc0(%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0xc8(%rsp)+        sbbq    %rax, %rax+        andq    %rsi, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0xd0(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0xd8(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0xe0(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0xe8(%rsp)+        movq    0x158(%rsp), %rcx+        movq    0x60(%rcx), %r8+        movq    0x68(%rcx), %r9+        movq    0x70(%rcx), %r10+        movq    0x78(%rcx), %r11+        movq    0x80(%rcx), %rbx+        movq    0x88(%rcx), %rbp+        movq    %r8, %rax+        movq    %r9, %rdx+        orq     %r10, %rax+        orq     %r11, %rdx+        orq     %rbx, %rax+        orq     %rbp, %rdx+        orq     %rdx, %rax+        negq    %rax+        sbbq    %rax, %rax+        movq    0x150(%rsp), %rsi+        movq    0x60(%rsi), %r12+        movq    0x68(%rsi), %r13+        movq    0x70(%rsi), %r14+        movq    0x78(%rsi), %r15+        movq    0x80(%rsi), %rdx+        movq    0x88(%rsi), %rcx+        cmoveq  %r12, %r8+        cmoveq  %r13, %r9+        cmoveq  %r14, %r10+        cmoveq  %r15, %r11+        cmoveq  %rdx, %rbx+        cmoveq  %rcx, %rbp+        orq     %r13, %r12+        orq     %r15, %r14+        orq     %rcx, %rdx+        orq     %r14, %r12+        orq     %r12, %rdx+        negq    %rdx+        sbbq    %rdx, %rdx+        cmpq    %rdx, %rax+        cmoveq  0xf0(%rsp), %r8+        cmoveq  0xf8(%rsp), %r9+        cmoveq  0x100(%rsp), %r10+        cmoveq  0x108(%rsp), %r11+        cmoveq  0x110(%rsp), %rbx+        cmoveq  0x118(%rsp), %rbp+        movq    %r8, 0xf0(%rsp)+        movq    %r9, 0xf8(%rsp)+        movq    %r10, 0x100(%rsp)+        movq    %r11, 0x108(%rsp)+        movq    %rbx, 0x110(%rsp)+        movq    %rbp, 0x118(%rsp)+        movq    0x158(%rsp), %rcx+        movq    0x150(%rsp), %rsi+        movq    (%rsp), %r8+        cmovbq  (%rsi), %r8+        cmova   (%rcx), %r8+        movq    0x8(%rsp), %r9+        cmovbq  0x8(%rsi), %r9+        cmova   0x8(%rcx), %r9+        movq    0x10(%rsp), %r10+        cmovbq  0x10(%rsi), %r10+        cmova   0x10(%rcx), %r10+        movq    0x18(%rsp), %r11+        cmovbq  0x18(%rsi), %r11+        cmova   0x18(%rcx), %r11+        movq    0x20(%rsp), %rbx+        cmovbq  0x20(%rsi), %rbx+        cmova   0x20(%rcx), %rbx+        movq    0x28(%rsp), %rbp+        cmovbq  0x28(%rsi), %rbp+        cmova   0x28(%rcx), %rbp+        movq    0xc0(%rsp), %r12+        cmovbq  0x30(%rsi), %r12+        cmova   0x30(%rcx), %r12+        movq    0xc8(%rsp), %r13+        cmovbq  0x38(%rsi), %r13+        cmova   0x38(%rcx), %r13+        movq    0xd0(%rsp), %r14+        cmovbq  0x40(%rsi), %r14+        cmova   0x40(%rcx), %r14+        movq    0xd8(%rsp), %r15+        cmovbq  0x48(%rsi), %r15+        cmova   0x48(%rcx), %r15+        movq    0xe0(%rsp), %rdx+        cmovbq  0x50(%rsi), %rdx+        cmova   0x50(%rcx), %rdx+        movq    0xe8(%rsp), %rax+        cmovbq  0x58(%rsi), %rax+        cmova   0x58(%rcx), %rax+        movq    %r8, (%rdi)+        movq    %r9, 0x8(%rdi)+        movq    %r10, 0x10(%rdi)+        movq    %r11, 0x18(%rdi)+        movq    %rbx, 0x20(%rdi)+        movq    %rbp, 0x28(%rdi)+        movq    0xf0(%rsp), %r8+        movq    0xf8(%rsp), %r9+        movq    0x100(%rsp), %r10+        movq    0x108(%rsp), %r11+        movq    0x110(%rsp), %rbx+        movq    0x118(%rsp), %rbp+        movq    %r12, 0x30(%rdi)+        movq    %r13, 0x38(%rdi)+        movq    %r14, 0x40(%rdi)+        movq    %r15, 0x48(%rdi)+        movq    %rdx, 0x50(%rdi)+        movq    %rax, 0x58(%rdi)+        movq    %r8, 0x60(%rdi)+        movq    %r9, 0x68(%rdi)+        movq    %r10, 0x70(%rdi)+        movq    %r11, 0x78(%rdi)+        movq    %rbx, 0x80(%rdi)+        movq    %rbp, 0x88(%rdi)+        CFI_INC_RSP(352)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp384_montjscalarmul_p384_montjadd)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp384_montjscalarmul_p384_montjdouble)++Lp384_montjscalarmul_p384_montjdouble:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(344)+        movq    %rdi, 0x150(%rsp)+        movq    0x60(%rsi), %rdx+        mulxq   0x68(%rsi), %r9, %r10+        mulxq   0x78(%rsi), %r11, %r12+        mulxq   0x88(%rsi), %r13, %r14+        movq    0x78(%rsi), %rdx+        mulxq   0x80(%rsi), %r15, %rcx+        xorl    %ebp, %ebp+        movq    0x70(%rsi), %rdx+        mulxq   0x60(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x68(%rsi), %rdx+        mulxq   0x78(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x80(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x88(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %rcx+        adcq    %rbp, %rcx+        xorl    %ebp, %ebp+        movq    0x80(%rsi), %rdx+        mulxq   0x60(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    0x70(%rsi), %rdx+        mulxq   0x78(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x80(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x88(%rsi), %rax, %rdx+        adcxq   %rax, %r15+        adoxq   %rdx, %rcx+        movq    0x88(%rsi), %rdx+        mulxq   0x80(%rsi), %rbx, %rbp+        mulxq   0x78(%rsi), %rax, %rdx+        adcxq   %rax, %rcx+        adoxq   %rdx, %rbx+        movl    $0x0, %eax+        adcxq   %rax, %rbx+        adoxq   %rax, %rbp+        adcq    %rax, %rbp+        xorq    %rax, %rax+        movq    0x60(%rsi), %rdx+        mulxq   0x60(%rsi), %r8, %rax+        adcxq   %r9, %r9+        adoxq   %rax, %r9+        movq    0x68(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x70(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x78(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %r15, %r15+        adoxq   %rdx, %r15+        movq    0x80(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %rcx, %rcx+        adoxq   %rax, %rcx+        adcxq   %rbx, %rbx+        adoxq   %rdx, %rbx+        movq    0x88(%rsi), %rdx+        mulxq   %rdx, %rax, %rdi+        adcxq   %rbp, %rbp+        adoxq   %rax, %rbp+        movl    $0x0, %eax+        adcxq   %rax, %rdi+        adoxq   %rax, %rdi+        movq    %rbx, (%rsp)+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r8, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r8+        addq    %rbx, %rax+        adcq    %rdx, %r8+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r9+        sbbq    %r8, %r10+        sbbq    %rbx, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rdx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r9, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r9+        addq    %rbx, %rax+        adcq    %rdx, %r9+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r10+        sbbq    %r9, %r11+        sbbq    %rbx, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rdx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r10, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r10+        addq    %rbx, %rax+        adcq    %rdx, %r10+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r11+        sbbq    %r10, %r12+        sbbq    %rbx, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rdx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r11, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r11+        addq    %rbx, %rax+        adcq    %rdx, %r11+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r12+        sbbq    %r11, %r13+        sbbq    %rbx, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rdx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r12, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r12+        addq    %rbx, %rax+        adcq    %rdx, %r12+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r13+        sbbq    %r12, %r8+        sbbq    %rbx, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rdx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r13, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r13+        addq    %rbx, %rax+        adcq    %rdx, %r13+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r8+        sbbq    %r13, %r9+        sbbq    %rbx, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rdx, %r13+        sbbq    $0x0, %r13+        movq    (%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rdi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rdi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rdi+        movq    %r14, (%rsp)+        movq    %r15, 0x8(%rsp)+        movq    %rcx, 0x10(%rsp)+        movq    %rbx, 0x18(%rsp)+        movq    %rbp, 0x20(%rsp)+        movq    %rdi, 0x28(%rsp)+        movq    0x30(%rsi), %rdx+        mulxq   0x38(%rsi), %r9, %r10+        mulxq   0x48(%rsi), %r11, %r12+        mulxq   0x58(%rsi), %r13, %r14+        movq    0x48(%rsi), %rdx+        mulxq   0x50(%rsi), %r15, %rcx+        xorl    %ebp, %ebp+        movq    0x40(%rsi), %rdx+        mulxq   0x30(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x38(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x38(%rsi), %rdx+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x50(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x58(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %rcx+        adcq    %rbp, %rcx+        xorl    %ebp, %ebp+        movq    0x50(%rsi), %rdx+        mulxq   0x30(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    0x40(%rsi), %rdx+        mulxq   0x48(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x50(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x58(%rsi), %rax, %rdx+        adcxq   %rax, %r15+        adoxq   %rdx, %rcx+        movq    0x58(%rsi), %rdx+        mulxq   0x50(%rsi), %rbx, %rbp+        mulxq   0x48(%rsi), %rax, %rdx+        adcxq   %rax, %rcx+        adoxq   %rdx, %rbx+        movl    $0x0, %eax+        adcxq   %rax, %rbx+        adoxq   %rax, %rbp+        adcq    %rax, %rbp+        xorq    %rax, %rax+        movq    0x30(%rsi), %rdx+        mulxq   0x30(%rsi), %r8, %rax+        adcxq   %r9, %r9+        adoxq   %rax, %r9+        movq    0x38(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x40(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x48(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %r15, %r15+        adoxq   %rdx, %r15+        movq    0x50(%rsi), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %rcx, %rcx+        adoxq   %rax, %rcx+        adcxq   %rbx, %rbx+        adoxq   %rdx, %rbx+        movq    0x58(%rsi), %rdx+        mulxq   %rdx, %rax, %rdi+        adcxq   %rbp, %rbp+        adoxq   %rax, %rbp+        movl    $0x0, %eax+        adcxq   %rax, %rdi+        adoxq   %rax, %rdi+        movq    %rbx, 0x30(%rsp)+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r8, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r8+        addq    %rbx, %rax+        adcq    %rdx, %r8+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r9+        sbbq    %r8, %r10+        sbbq    %rbx, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rdx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r9, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r9+        addq    %rbx, %rax+        adcq    %rdx, %r9+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r10+        sbbq    %r9, %r11+        sbbq    %rbx, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rdx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r10, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r10+        addq    %rbx, %rax+        adcq    %rdx, %r10+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r11+        sbbq    %r10, %r12+        sbbq    %rbx, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rdx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r11, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r11+        addq    %rbx, %rax+        adcq    %rdx, %r11+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r12+        sbbq    %r11, %r13+        sbbq    %rbx, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rdx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r12, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r12+        addq    %rbx, %rax+        adcq    %rdx, %r12+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r13+        sbbq    %r12, %r8+        sbbq    %rbx, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rdx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r13, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r13+        addq    %rbx, %rax+        adcq    %rdx, %r13+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r8+        sbbq    %r13, %r9+        sbbq    %rbx, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rdx, %r13+        sbbq    $0x0, %r13+        movq    0x30(%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rdi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rdi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rdi+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    %rcx, 0x40(%rsp)+        movq    %rbx, 0x48(%rsp)+        movq    %rbp, 0x50(%rsp)+        movq    %rdi, 0x58(%rsp)+        movq    (%rsi), %rax+        addq    (%rsp), %rax+        movq    0x8(%rsi), %rcx+        adcq    0x8(%rsp), %rcx+        movq    0x10(%rsi), %r8+        adcq    0x10(%rsp), %r8+        movq    0x18(%rsi), %r9+        adcq    0x18(%rsp), %r9+        movq    0x20(%rsi), %r10+        adcq    0x20(%rsp), %r10+        movq    0x28(%rsi), %r11+        adcq    0x28(%rsp), %r11+        sbbq    %rdx, %rdx+        movl    $0x1, %ebx+        andq    %rdx, %rbx+        movl    $0xffffffff, %ebp+        andq    %rbp, %rdx+        xorq    %rbp, %rbp+        subq    %rdx, %rbp+        addq    %rbp, %rax+        movq    %rax, 0xf0(%rsp)+        adcq    %rdx, %rcx+        movq    %rcx, 0xf8(%rsp)+        adcq    %rbx, %r8+        movq    %r8, 0x100(%rsp)+        adcq    $0x0, %r9+        movq    %r9, 0x108(%rsp)+        adcq    $0x0, %r10+        movq    %r10, 0x110(%rsp)+        adcq    $0x0, %r11+        movq    %r11, 0x118(%rsp)+        movq    (%rsi), %rax+        subq    (%rsp), %rax+        movq    0x8(%rsi), %rdx+        sbbq    0x8(%rsp), %rdx+        movq    0x10(%rsi), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x18(%rsi), %r9+        sbbq    0x18(%rsp), %r9+        movq    0x20(%rsi), %r10+        sbbq    0x20(%rsp), %r10+        movq    0x28(%rsi), %r11+        sbbq    0x28(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %ebx+        andq    %rbx, %rcx+        xorq    %rbx, %rbx+        subq    %rcx, %rbx+        subq    %rbx, %rax+        movq    %rax, 0xc0(%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0xc8(%rsp)+        sbbq    %rax, %rax+        andq    %rbx, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0xd0(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0xd8(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0xe0(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0xe8(%rsp)+        movq    0xc0(%rsp), %rdx+        xorl    %r15d, %r15d+        mulxq   0xf0(%rsp), %r8, %r9+        mulxq   0xf8(%rsp), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0x100(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x108(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0x110(%rsp), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0x118(%rsp), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0xc8(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0xd0(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0xd8(%rsp), %rdx+        xorl    %r10d, %r10d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0xe0(%rsp), %rdx+        xorl    %r11d, %r11d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0xe8(%rsp), %rdx+        xorl    %r12d, %r12d+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0x118(%rsp), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x60(%rsp)+        movq    %r15, 0x68(%rsp)+        movq    %r8, 0x70(%rsp)+        movq    %r9, 0x78(%rsp)+        movq    %r10, 0x80(%rsp)+        movq    %r11, 0x88(%rsp)+        movq    0x30(%rsi), %rax+        addq    0x60(%rsi), %rax+        movq    0x38(%rsi), %rcx+        adcq    0x68(%rsi), %rcx+        movq    0x40(%rsi), %r8+        adcq    0x70(%rsi), %r8+        movq    0x48(%rsi), %r9+        adcq    0x78(%rsi), %r9+        movq    0x50(%rsi), %r10+        adcq    0x80(%rsi), %r10+        movq    0x58(%rsi), %r11+        adcq    0x88(%rsi), %r11+        movl    $0x0, %edx+        adcq    %rdx, %rdx+        movabsq $0xffffffff00000001, %rbp+        addq    %rbp, %rax+        movl    $0xffffffff, %ebp+        adcq    %rbp, %rcx+        adcq    $0x1, %r8+        adcq    $0x0, %r9+        adcq    $0x0, %r10+        adcq    $0x0, %r11+        adcq    $0xffffffffffffffff, %rdx+        movl    $0x1, %ebx+        andq    %rdx, %rbx+        andq    %rbp, %rdx+        xorq    %rbp, %rbp+        subq    %rdx, %rbp+        subq    %rbp, %rax+        movq    %rax, 0xf0(%rsp)+        sbbq    %rdx, %rcx+        movq    %rcx, 0xf8(%rsp)+        sbbq    %rbx, %r8+        movq    %r8, 0x100(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x108(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x110(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x118(%rsp)+        movq    0x60(%rsp), %rdx+        mulxq   0x68(%rsp), %r9, %r10+        mulxq   0x78(%rsp), %r11, %r12+        mulxq   0x88(%rsp), %r13, %r14+        movq    0x78(%rsp), %rdx+        mulxq   0x80(%rsp), %r15, %rcx+        xorl    %ebp, %ebp+        movq    0x70(%rsp), %rdx+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x68(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x68(%rsp), %rdx+        mulxq   0x78(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x80(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x88(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %rcx+        adcq    %rbp, %rcx+        xorl    %ebp, %ebp+        movq    0x80(%rsp), %rdx+        mulxq   0x60(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    0x70(%rsp), %rdx+        mulxq   0x78(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x80(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x88(%rsp), %rax, %rdx+        adcxq   %rax, %r15+        adoxq   %rdx, %rcx+        movq    0x88(%rsp), %rdx+        mulxq   0x80(%rsp), %rbx, %rbp+        mulxq   0x78(%rsp), %rax, %rdx+        adcxq   %rax, %rcx+        adoxq   %rdx, %rbx+        movl    $0x0, %eax+        adcxq   %rax, %rbx+        adoxq   %rax, %rbp+        adcq    %rax, %rbp+        xorq    %rax, %rax+        movq    0x60(%rsp), %rdx+        mulxq   0x60(%rsp), %r8, %rax+        adcxq   %r9, %r9+        adoxq   %rax, %r9+        movq    0x68(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x70(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x78(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %r15, %r15+        adoxq   %rdx, %r15+        movq    0x80(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %rcx, %rcx+        adoxq   %rax, %rcx+        adcxq   %rbx, %rbx+        adoxq   %rdx, %rbx+        movq    0x88(%rsp), %rdx+        mulxq   %rdx, %rax, %rdi+        adcxq   %rbp, %rbp+        adoxq   %rax, %rbp+        movl    $0x0, %eax+        adcxq   %rax, %rdi+        adoxq   %rax, %rdi+        movq    %rbx, 0x120(%rsp)+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r8, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r8+        addq    %rbx, %rax+        adcq    %rdx, %r8+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r9+        sbbq    %r8, %r10+        sbbq    %rbx, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rdx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r9, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r9+        addq    %rbx, %rax+        adcq    %rdx, %r9+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r10+        sbbq    %r9, %r11+        sbbq    %rbx, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rdx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r10, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r10+        addq    %rbx, %rax+        adcq    %rdx, %r10+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r11+        sbbq    %r10, %r12+        sbbq    %rbx, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rdx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r11, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r11+        addq    %rbx, %rax+        adcq    %rdx, %r11+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r12+        sbbq    %r11, %r13+        sbbq    %rbx, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rdx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r12, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r12+        addq    %rbx, %rax+        adcq    %rdx, %r12+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r13+        sbbq    %r12, %r8+        sbbq    %rbx, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rdx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r13, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r13+        addq    %rbx, %rax+        adcq    %rdx, %r13+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r8+        sbbq    %r13, %r9+        sbbq    %rbx, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rdx, %r13+        sbbq    $0x0, %r13+        movq    0x120(%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rdi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rdi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rdi+        movq    %r14, 0x120(%rsp)+        movq    %r15, 0x128(%rsp)+        movq    %rcx, 0x130(%rsp)+        movq    %rbx, 0x138(%rsp)+        movq    %rbp, 0x140(%rsp)+        movq    %rdi, 0x148(%rsp)+        movq    0x30(%rsp), %rdx+        xorl    %r15d, %r15d+        mulxq   (%rsi), %r8, %r9+        mulxq   0x8(%rsi), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0x10(%rsi), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x18(%rsi), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0x20(%rsi), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0x28(%rsi), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x38(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x20(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0x28(%rsi), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x40(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x20(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0x28(%rsi), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x48(%rsp), %rdx+        xorl    %r10d, %r10d+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x20(%rsi), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0x28(%rsi), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x50(%rsp), %rdx+        xorl    %r11d, %r11d+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x20(%rsi), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0x28(%rsi), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x58(%rsp), %rdx+        xorl    %r12d, %r12d+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x20(%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0x28(%rsi), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    %r8, 0xa0(%rsp)+        movq    %r9, 0xa8(%rsp)+        movq    %r10, 0xb0(%rsp)+        movq    %r11, 0xb8(%rsp)+        movq    0xf0(%rsp), %rdx+        mulxq   0xf8(%rsp), %r9, %r10+        mulxq   0x108(%rsp), %r11, %r12+        mulxq   0x118(%rsp), %r13, %r14+        movq    0x108(%rsp), %rdx+        mulxq   0x110(%rsp), %r15, %rcx+        xorl    %ebp, %ebp+        movq    0x100(%rsp), %rdx+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0xf8(%rsp), %rdx+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x118(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %rcx+        adcq    %rbp, %rcx+        xorl    %ebp, %ebp+        movq    0x110(%rsp), %rdx+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    0x100(%rsp), %rdx+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x118(%rsp), %rax, %rdx+        adcxq   %rax, %r15+        adoxq   %rdx, %rcx+        movq    0x118(%rsp), %rdx+        mulxq   0x110(%rsp), %rbx, %rbp+        mulxq   0x108(%rsp), %rax, %rdx+        adcxq   %rax, %rcx+        adoxq   %rdx, %rbx+        movl    $0x0, %eax+        adcxq   %rax, %rbx+        adoxq   %rax, %rbp+        adcq    %rax, %rbp+        xorq    %rax, %rax+        movq    0xf0(%rsp), %rdx+        mulxq   0xf0(%rsp), %r8, %rax+        adcxq   %r9, %r9+        adoxq   %rax, %r9+        movq    0xf8(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x100(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x108(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %r15, %r15+        adoxq   %rdx, %r15+        movq    0x110(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %rcx, %rcx+        adoxq   %rax, %rcx+        adcxq   %rbx, %rbx+        adoxq   %rdx, %rbx+        movq    0x118(%rsp), %rdx+        mulxq   %rdx, %rax, %rdi+        adcxq   %rbp, %rbp+        adoxq   %rax, %rbp+        movl    $0x0, %eax+        adcxq   %rax, %rdi+        adoxq   %rax, %rdi+        movq    %rbx, 0xc0(%rsp)+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r8, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r8+        addq    %rbx, %rax+        adcq    %rdx, %r8+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r9+        sbbq    %r8, %r10+        sbbq    %rbx, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rdx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r9, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r9+        addq    %rbx, %rax+        adcq    %rdx, %r9+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r10+        sbbq    %r9, %r11+        sbbq    %rbx, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rdx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r10, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r10+        addq    %rbx, %rax+        adcq    %rdx, %r10+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r11+        sbbq    %r10, %r12+        sbbq    %rbx, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rdx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r11, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r11+        addq    %rbx, %rax+        adcq    %rdx, %r11+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r12+        sbbq    %r11, %r13+        sbbq    %rbx, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rdx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r12, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r12+        addq    %rbx, %rax+        adcq    %rdx, %r12+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r13+        sbbq    %r12, %r8+        sbbq    %rbx, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rdx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r13, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r13+        addq    %rbx, %rax+        adcq    %rdx, %r13+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r8+        sbbq    %r13, %r9+        sbbq    %rbx, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rdx, %r13+        sbbq    $0x0, %r13+        movq    0xc0(%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rdi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rdi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rdi+        movq    %r14, 0xc0(%rsp)+        movq    %r15, 0xc8(%rsp)+        movq    %rcx, 0xd0(%rsp)+        movq    %rbx, 0xd8(%rsp)+        movq    %rbp, 0xe0(%rsp)+        movq    %rdi, 0xe8(%rsp)+        movabsq $0xffffffff, %r8+        subq    0x120(%rsp), %r8+        movabsq $0xffffffff00000000, %r9+        sbbq    0x128(%rsp), %r9+        movq    $0xfffffffffffffffe, %r10+        sbbq    0x130(%rsp), %r10+        movq    $0xffffffffffffffff, %r11+        sbbq    0x138(%rsp), %r11+        movq    $0xffffffffffffffff, %r12+        sbbq    0x140(%rsp), %r12+        movq    $0xffffffffffffffff, %r13+        sbbq    0x148(%rsp), %r13+        movq    $0x9, %rdx+        mulxq   %r8, %r8, %rax+        mulxq   %r9, %r9, %rcx+        addq    %rax, %r9+        mulxq   %r10, %r10, %rax+        adcq    %rcx, %r10+        mulxq   %r11, %r11, %rcx+        adcq    %rax, %r11+        mulxq   %r12, %r12, %rax+        adcq    %rcx, %r12+        mulxq   %r13, %r13, %r14+        adcq    %rax, %r13+        adcq    $0x1, %r14+        xorl    %ecx, %ecx+        movq    $0xc, %rdx+        mulxq   0x90(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x98(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0xa0(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0xa8(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0xb0(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0xb8(%rsp), %rax, %rdx+        adcxq   %rax, %r13+        adoxq   %r14, %rdx+        adcxq   %rcx, %rdx+        xorq    %rcx, %rcx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rax, %rcx+        adcxq   %rax, %r8+        adoxq   %rcx, %r9+        movl    $0xffffffff, %eax+        mulxq   %rax, %rax, %rcx+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+        adcxq   %rdx, %r10+        movl    $0x0, %eax+        movl    $0x0, %ecx+        adoxq   %rax, %rax+        adcq    %rax, %r11+        adcq    %rcx, %r12+        adcq    %rcx, %r13+        adcq    %rcx, %rcx+        subq    $0x1, %rcx+        movl    $0xffffffff, %edx+        xorq    %rax, %rax+        andq    %rcx, %rdx+        subq    %rdx, %rax+        andq    $0x1, %rcx+        subq    %rax, %r8+        movq    %r8, 0x120(%rsp)+        sbbq    %rdx, %r9+        movq    %r9, 0x128(%rsp)+        sbbq    %rcx, %r10+        movq    %r10, 0x130(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x138(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x140(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x148(%rsp)+        movq    0xc0(%rsp), %rax+        subq    (%rsp), %rax+        movq    0xc8(%rsp), %rdx+        sbbq    0x8(%rsp), %rdx+        movq    0xd0(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0xd8(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movq    0xe0(%rsp), %r10+        sbbq    0x20(%rsp), %r10+        movq    0xe8(%rsp), %r11+        sbbq    0x28(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %ebx+        andq    %rbx, %rcx+        xorq    %rbx, %rbx+        subq    %rcx, %rbx+        subq    %rbx, %rax+        movq    %rax, 0xf0(%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0xf8(%rsp)+        sbbq    %rax, %rax+        andq    %rbx, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0x100(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x108(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x110(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x118(%rsp)+        movq    0x30(%rsp), %rdx+        mulxq   0x38(%rsp), %r9, %r10+        mulxq   0x48(%rsp), %r11, %r12+        mulxq   0x58(%rsp), %r13, %r14+        movq    0x48(%rsp), %rdx+        mulxq   0x50(%rsp), %r15, %rcx+        xorl    %ebp, %ebp+        movq    0x40(%rsp), %rdx+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x38(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    0x38(%rsp), %rdx+        mulxq   0x48(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x50(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x58(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adcxq   %rbp, %r15+        adoxq   %rbp, %rcx+        adcq    %rbp, %rcx+        xorl    %ebp, %ebp+        movq    0x50(%rsp), %rdx+        mulxq   0x30(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    0x40(%rsp), %rdx+        mulxq   0x48(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x50(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x58(%rsp), %rax, %rdx+        adcxq   %rax, %r15+        adoxq   %rdx, %rcx+        movq    0x58(%rsp), %rdx+        mulxq   0x50(%rsp), %rbx, %rbp+        mulxq   0x48(%rsp), %rax, %rdx+        adcxq   %rax, %rcx+        adoxq   %rdx, %rbx+        movl    $0x0, %eax+        adcxq   %rax, %rbx+        adoxq   %rax, %rbp+        adcq    %rax, %rbp+        xorq    %rax, %rax+        movq    0x30(%rsp), %rdx+        mulxq   0x30(%rsp), %r8, %rax+        adcxq   %r9, %r9+        adoxq   %rax, %r9+        movq    0x38(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r10, %r10+        adoxq   %rax, %r10+        adcxq   %r11, %r11+        adoxq   %rdx, %r11+        movq    0x40(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r12, %r12+        adoxq   %rax, %r12+        adcxq   %r13, %r13+        adoxq   %rdx, %r13+        movq    0x48(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %r14, %r14+        adoxq   %rax, %r14+        adcxq   %r15, %r15+        adoxq   %rdx, %r15+        movq    0x50(%rsp), %rdx+        mulxq   %rdx, %rax, %rdx+        adcxq   %rcx, %rcx+        adoxq   %rax, %rcx+        adcxq   %rbx, %rbx+        adoxq   %rdx, %rbx+        movq    0x58(%rsp), %rdx+        mulxq   %rdx, %rax, %rdi+        adcxq   %rbp, %rbp+        adoxq   %rax, %rbp+        movl    $0x0, %eax+        adcxq   %rax, %rdi+        adoxq   %rax, %rdi+        movq    %rbx, 0xc0(%rsp)+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r8, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r8+        addq    %rbx, %rax+        adcq    %rdx, %r8+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r9+        sbbq    %r8, %r10+        sbbq    %rbx, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rdx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r9, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r9+        addq    %rbx, %rax+        adcq    %rdx, %r9+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r10+        sbbq    %r9, %r11+        sbbq    %rbx, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rdx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r10, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r10+        addq    %rbx, %rax+        adcq    %rdx, %r10+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r11+        sbbq    %r10, %r12+        sbbq    %rbx, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rdx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r11, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r11+        addq    %rbx, %rax+        adcq    %rdx, %r11+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r12+        sbbq    %r11, %r13+        sbbq    %rbx, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rdx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r12, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r12+        addq    %rbx, %rax+        adcq    %rdx, %r12+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r13+        sbbq    %r12, %r8+        sbbq    %rbx, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rdx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %r13, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %rbx, %r13+        addq    %rbx, %rax+        adcq    %rdx, %r13+        movl    $0x0, %ebx+        adcq    %rbx, %rbx+        subq    %rax, %r8+        sbbq    %r13, %r9+        sbbq    %rbx, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rdx, %r13+        sbbq    $0x0, %r13+        movq    0xc0(%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rdi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rdi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rdi+        movq    %r14, 0xc0(%rsp)+        movq    %r15, 0xc8(%rsp)+        movq    %rcx, 0xd0(%rsp)+        movq    %rbx, 0xd8(%rsp)+        movq    %rbp, 0xe0(%rsp)+        movq    %rdi, 0xe8(%rsp)+        movq    0x150(%rsp), %rdi+        movq    0xf0(%rsp), %rax+        subq    0x30(%rsp), %rax+        movq    0xf8(%rsp), %rdx+        sbbq    0x38(%rsp), %rdx+        movq    0x100(%rsp), %r8+        sbbq    0x40(%rsp), %r8+        movq    0x108(%rsp), %r9+        sbbq    0x48(%rsp), %r9+        movq    0x110(%rsp), %r10+        sbbq    0x50(%rsp), %r10+        movq    0x118(%rsp), %r11+        sbbq    0x58(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %ebx+        andq    %rbx, %rcx+        xorq    %rbx, %rbx+        subq    %rcx, %rbx+        subq    %rbx, %rax+        movq    %rax, 0x60(%rdi)+        sbbq    %rcx, %rdx+        movq    %rdx, 0x68(%rdi)+        sbbq    %rax, %rax+        andq    %rbx, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0x70(%rdi)+        sbbq    $0x0, %r9+        movq    %r9, 0x78(%rdi)+        sbbq    $0x0, %r10+        movq    %r10, 0x80(%rdi)+        sbbq    $0x0, %r11+        movq    %r11, 0x88(%rdi)+        movq    0x60(%rsp), %rdx+        xorl    %r15d, %r15d+        mulxq   0x120(%rsp), %r8, %r9+        mulxq   0x128(%rsp), %rbx, %r10+        addq    %rbx, %r9+        mulxq   0x130(%rsp), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x138(%rsp), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0x140(%rsp), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0x148(%rsp), %rbx, %r14+        adcq    %rbx, %r13+        adcq    %r15, %r14+        movq    %r8, %rdx+        shlq    $0x20, %rdx+        addq    %r8, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r8, %rbx+        adcq    %r8, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rbx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rdx+        addq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x68(%rsp), %rdx+        xorl    %r8d, %r8d+        mulxq   0x120(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x128(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x130(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x138(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x140(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        adoxq   %r8, %r15+        mulxq   0x148(%rsp), %rax, %rbx+        adcq    %rax, %r14+        adcq    %rbx, %r15+        adcq    %r8, %r8+        movq    %r9, %rdx+        shlq    $0x20, %rdx+        addq    %r9, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r9, %rbx+        adcq    %r9, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rbx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rdx+        addq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x70(%rsp), %rdx+        xorl    %r9d, %r9d+        mulxq   0x120(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x128(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x130(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x138(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x140(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        adoxq   %r9, %r8+        mulxq   0x148(%rsp), %rax, %rbx+        adcq    %rax, %r15+        adcq    %rbx, %r8+        adcq    %r9, %r9+        movq    %r10, %rdx+        shlq    $0x20, %rdx+        addq    %r10, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r10, %rbx+        adcq    %r10, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rbx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rdx+        addq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x78(%rsp), %rdx+        xorl    %r10d, %r10d+        mulxq   0x120(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x128(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x130(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x138(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x140(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        adoxq   %r10, %r9+        mulxq   0x148(%rsp), %rax, %rbx+        adcq    %rax, %r8+        adcq    %rbx, %r9+        adcq    %r10, %r10+        movq    %r11, %rdx+        shlq    $0x20, %rdx+        addq    %r11, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r11, %rbx+        adcq    %r11, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rbx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rdx+        addq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x80(%rsp), %rdx+        xorl    %r11d, %r11d+        mulxq   0x120(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x128(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x130(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x138(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x140(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        adoxq   %r11, %r10+        mulxq   0x148(%rsp), %rax, %rbx+        adcq    %rax, %r9+        adcq    %rbx, %r10+        adcq    %r11, %r11+        movq    %r12, %rdx+        shlq    $0x20, %rdx+        addq    %r12, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r12, %rbx+        adcq    %r12, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rbx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rdx+        addq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x88(%rsp), %rdx+        xorl    %r12d, %r12d+        mulxq   0x120(%rsp), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x128(%rsp), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x130(%rsp), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x138(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x140(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        adoxq   %r12, %r11+        mulxq   0x148(%rsp), %rax, %rbx+        adcq    %rax, %r10+        adcq    %rbx, %r11+        adcq    %r12, %r12+        movq    %r13, %rdx+        shlq    $0x20, %rdx+        addq    %r13, %rdx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rbx, %rax+        movl    $0xffffffff, %ebx+        mulxq   %rbx, %r13, %rbx+        adcq    %r13, %rax+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rbx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rdx+        addq    %rdx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0xf0(%rsp)+        movq    %r15, 0xf8(%rsp)+        movq    %r8, 0x100(%rsp)+        movq    %r9, 0x108(%rsp)+        movq    %r10, 0x110(%rsp)+        movq    %r11, 0x118(%rsp)+        movq    0xb8(%rsp), %rdx+        movq    %rdx, %r13+        shrq    $0x3e, %rdx+        movq    0xb0(%rsp), %r12+        shldq   $0x2, %r12, %r13+        movq    0xa8(%rsp), %r11+        shldq   $0x2, %r11, %r12+        movq    0xa0(%rsp), %r10+        shldq   $0x2, %r10, %r11+        movq    0x98(%rsp), %r9+        shldq   $0x2, %r9, %r10+        movq    0x90(%rsp), %r8+        shldq   $0x2, %r8, %r9+        shlq    $0x2, %r8+        addq    $0x1, %rdx+        subq    0x120(%rsp), %r8+        sbbq    0x128(%rsp), %r9+        sbbq    0x130(%rsp), %r10+        sbbq    0x138(%rsp), %r11+        sbbq    0x140(%rsp), %r12+        sbbq    0x148(%rsp), %r13+        sbbq    $0x0, %rdx+        xorq    %rcx, %rcx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rax, %rcx+        adcxq   %rax, %r8+        adoxq   %rcx, %r9+        movl    $0xffffffff, %eax+        mulxq   %rax, %rax, %rcx+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+        adcxq   %rdx, %r10+        movl    $0x0, %eax+        movl    $0x0, %ecx+        adoxq   %rax, %rax+        adcq    %rax, %r11+        adcq    %rcx, %r12+        adcq    %rcx, %r13+        adcq    %rcx, %rcx+        subq    $0x1, %rcx+        movl    $0xffffffff, %edx+        xorq    %rax, %rax+        andq    %rcx, %rdx+        subq    %rdx, %rax+        andq    $0x1, %rcx+        subq    %rax, %r8+        movq    %r8, (%rdi)+        sbbq    %rdx, %r9+        movq    %r9, 0x8(%rdi)+        sbbq    %rcx, %r10+        movq    %r10, 0x10(%rdi)+        sbbq    $0x0, %r11+        movq    %r11, 0x18(%rdi)+        sbbq    $0x0, %r12+        movq    %r12, 0x20(%rdi)+        sbbq    $0x0, %r13+        movq    %r13, 0x28(%rdi)+        movabsq $0xffffffff, %r8+        subq    0xc0(%rsp), %r8+        movabsq $0xffffffff00000000, %r9+        sbbq    0xc8(%rsp), %r9+        movq    $0xfffffffffffffffe, %r10+        sbbq    0xd0(%rsp), %r10+        movq    $0xffffffffffffffff, %r11+        sbbq    0xd8(%rsp), %r11+        movq    $0xffffffffffffffff, %r12+        sbbq    0xe0(%rsp), %r12+        movq    $0xffffffffffffffff, %r13+        sbbq    0xe8(%rsp), %r13+        movq    %r13, %r14+        shrq    $0x3d, %r14+        shldq   $0x3, %r12, %r13+        shldq   $0x3, %r11, %r12+        shldq   $0x3, %r10, %r11+        shldq   $0x3, %r9, %r10+        shldq   $0x3, %r8, %r9+        shlq    $0x3, %r8+        addq    $0x1, %r14+        xorl    %ecx, %ecx+        movq    $0x3, %rdx+        mulxq   0xf0(%rsp), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0xf8(%rsp), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x100(%rsp), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x108(%rsp), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x110(%rsp), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x118(%rsp), %rax, %rdx+        adcxq   %rax, %r13+        adoxq   %r14, %rdx+        adcxq   %rcx, %rdx+        xorq    %rcx, %rcx+        movabsq $0xffffffff00000001, %rax+        mulxq   %rax, %rax, %rcx+        adcxq   %rax, %r8+        adoxq   %rcx, %r9+        movl    $0xffffffff, %eax+        mulxq   %rax, %rax, %rcx+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+        adcxq   %rdx, %r10+        movl    $0x0, %eax+        movl    $0x0, %ecx+        adoxq   %rax, %rax+        adcq    %rax, %r11+        adcq    %rcx, %r12+        adcq    %rcx, %r13+        adcq    %rcx, %rcx+        subq    $0x1, %rcx+        movl    $0xffffffff, %edx+        xorq    %rax, %rax+        andq    %rcx, %rdx+        subq    %rdx, %rax+        andq    $0x1, %rcx+        subq    %rax, %r8+        movq    %r8, 0x30(%rdi)+        sbbq    %rdx, %r9+        movq    %r9, 0x38(%rdi)+        sbbq    %rcx, %r10+        movq    %r10, 0x40(%rdi)+        sbbq    $0x0, %r11+        movq    %r11, 0x48(%rdi)+        sbbq    $0x0, %r12+        movq    %r12, 0x50(%rdi)+        sbbq    $0x0, %r13+        movq    %r13, 0x58(%rdi)+        CFI_INC_RSP(344)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp384_montjscalarmul_p384_montjdouble)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/p384_montjscalarmul_alt.S view
@@ -0,0 +1,9440 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Montgomery-Jacobian form scalar multiplication for P-384+// Input scalar[6], point[18]; output res[18]+//+// extern void p384_montjscalarmul_alt+//   (uint64_t res[static 18],+//    const uint64_t scalar[static 6],+//    const uint64_t point[static 18]);+//+// This function is a variant of its affine point version p384_scalarmul.+// Here, input and output points are assumed to be in Jacobian form with+// their coordinates in the Montgomery domain. Thus, if priming indicates+// Montgomery form, x' = (2^384 * x) mod p_384 etc., each point argument+// is a triple (x',y',z') representing the affine point (x/z^2,y/z^3) when+// z' is nonzero or the point at infinity (group identity) if z' = 0.+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-384, returns a representation of n * P. If the result is the+// point at infinity (either because the input point was or because the+// scalar was a multiple of p_384) then the output is guaranteed to+// represent the point at infinity, i.e. to have its z coordinate zero.+//+// Standard x86-64 ABI: RDI = res, RSI = scalar, RDX = point+// Microsoft x64 ABI:   RCX = res, RDX = scalar, R8 = point+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p384_montjscalarmul_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p384_montjscalarmul_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p384_montjscalarmul_alt)+++        .text+        .balign 32++// Size of individual field elements++#define NUMSIZE 48+#define JACSIZE (3*NUMSIZE)++// Intermediate variables on the stack.+// The table is 16 entries, each of size JACSIZE = 3 * NUMSIZE+// Uppercase syntactic variants make x86_att version simpler to generate.++#define SCALARB (0*NUMSIZE)+#define scalarb (0*NUMSIZE)(%rsp)+#define ACC (1*NUMSIZE)+#define acc (1*NUMSIZE)(%rsp)+#define TABENT (4*NUMSIZE)+#define tabent (4*NUMSIZE)(%rsp)++#define TAB (7*NUMSIZE)+#define tab (7*NUMSIZE)(%rsp)++#define res (55*NUMSIZE)(%rsp)++#define NSPACE 56*NUMSIZE++// Avoid using .rep for the sake of the BoringSSL/AWS-LC delocator,+// which doesn't accept repetitions, assembler macros etc.++#define selectblock_xz(I)                         \+        cmpq    $I, %rdi ;                           \+        cmovzq  TAB+JACSIZE*(I-1)(%rsp), %rax ;     \+        cmovzq  TAB+JACSIZE*(I-1)+8(%rsp), %rbx ;   \+        cmovzq  TAB+JACSIZE*(I-1)+16(%rsp), %rcx ;  \+        cmovzq  TAB+JACSIZE*(I-1)+24(%rsp), %rdx ;  \+        cmovzq  TAB+JACSIZE*(I-1)+32(%rsp), %r8 ;   \+        cmovzq  TAB+JACSIZE*(I-1)+40(%rsp), %r9 ;   \+        cmovzq  TAB+JACSIZE*(I-1)+96(%rsp), %r10 ;  \+        cmovzq  TAB+JACSIZE*(I-1)+104(%rsp), %r11 ; \+        cmovzq  TAB+JACSIZE*(I-1)+112(%rsp), %r12 ; \+        cmovzq  TAB+JACSIZE*(I-1)+120(%rsp), %r13 ; \+        cmovzq  TAB+JACSIZE*(I-1)+128(%rsp), %r14 ; \+        cmovzq  TAB+JACSIZE*(I-1)+136(%rsp), %r15++#define selectblock_y(I)                          \+        cmpq    $I, %rdi ;                           \+        cmovzq  TAB+JACSIZE*(I-1)+48(%rsp), %rax ;  \+        cmovzq  TAB+JACSIZE*(I-1)+56(%rsp), %rbx ;  \+        cmovzq  TAB+JACSIZE*(I-1)+64(%rsp), %rcx ;  \+        cmovzq  TAB+JACSIZE*(I-1)+72(%rsp), %rdx ;  \+        cmovzq  TAB+JACSIZE*(I-1)+80(%rsp), %r8 ;   \+        cmovzq  TAB+JACSIZE*(I-1)+88(%rsp), %r9++S2N_BN_SYMBOL(p384_montjscalarmul_alt):+        CFI_START+        _CET_ENDBR++// The Windows version literally calls the standard ABI version.+// This simplifies the proofs since subroutine offsets are fixed.++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+        CFI_CALL(Lp384_montjscalarmul_alt_standard)+        CFI_POP(%rsi)+        CFI_POP(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p384_montjscalarmul_alt)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp384_montjscalarmul_alt_standard)++Lp384_montjscalarmul_alt_standard:+        CFI_START+#endif++// Real start of the standard ABI code.++        CFI_PUSH(%r15)+        CFI_PUSH(%r14)+        CFI_PUSH(%r13)+        CFI_PUSH(%r12)+        CFI_PUSH(%rbp)+        CFI_PUSH(%rbx)++        CFI_DEC_RSP(NSPACE)++// Preserve the "res" input argument; others get processed early.++        movq    %rdi, res++// Reduce the input scalar mod n_384, i.e. conditionally subtract n_384.+// Store it to "scalarb".++        movq    (%rsi), %r8+        movq    $0xecec196accc52973, %rax+        subq    %rax, %r8+        movq    8(%rsi), %r9+        movq    $0x581a0db248b0a77a, %rax+        sbbq    %rax, %r9+        movq    16(%rsi), %r10+        movq    $0xc7634d81f4372ddf, %rax+        sbbq    %rax, %r10+        movq    24(%rsi), %r11+        movq    $0xffffffffffffffff, %rax+        sbbq    %rax, %r11+        movq    32(%rsi), %r12+        sbbq    %rax, %r12+        movq    40(%rsi), %r13+        sbbq    %rax, %r13++        cmovcq  (%rsi), %r8+        cmovcq  8(%rsi), %r9+        cmovcq  16(%rsi), %r10+        cmovcq  24(%rsi), %r11+        cmovcq  32(%rsi), %r12+        cmovcq  40(%rsi), %r13++        movq    %r8, SCALARB(%rsp)+        movq    %r9, SCALARB+8(%rsp)+        movq    %r10, SCALARB+16(%rsp)+        movq    %r11, SCALARB+24(%rsp)+        movq    %r12, SCALARB+32(%rsp)+        movq    %r13, SCALARB+40(%rsp)++// Set the tab[0] table entry to the input point = 1 * P++        movq    (%rdx), %rax+        movq    %rax, TAB(%rsp)+        movq    8(%rdx), %rax+        movq    %rax, TAB+8(%rsp)+        movq    16(%rdx), %rax+        movq    %rax, TAB+16(%rsp)+        movq    24(%rdx), %rax+        movq    %rax, TAB+24(%rsp)+        movq    32(%rdx), %rax+        movq    %rax, TAB+32(%rsp)+        movq    40(%rdx), %rax+        movq    %rax, TAB+40(%rsp)++        movq    48(%rdx), %rax+        movq    %rax, TAB+48(%rsp)+        movq    56(%rdx), %rax+        movq    %rax, TAB+56(%rsp)+        movq    64(%rdx), %rax+        movq    %rax, TAB+64(%rsp)+        movq    72(%rdx), %rax+        movq    %rax, TAB+72(%rsp)+        movq    80(%rdx), %rax+        movq    %rax, TAB+80(%rsp)+        movq    88(%rdx), %rax+        movq    %rax, TAB+88(%rsp)++        movq    96(%rdx), %rax+        movq    %rax, TAB+96(%rsp)+        movq    104(%rdx), %rax+        movq    %rax, TAB+104(%rsp)+        movq    112(%rdx), %rax+        movq    %rax, TAB+112(%rsp)+        movq    120(%rdx), %rax+        movq    %rax, TAB+120(%rsp)+        movq    128(%rdx), %rax+        movq    %rax, TAB+128(%rsp)+        movq    136(%rdx), %rax+        movq    %rax, TAB+136(%rsp)++// Compute and record tab[1] = 2 * p, ..., tab[15] = 16 * P++        leaq    TAB+JACSIZE*1(%rsp), %rdi+        leaq    TAB(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjdouble)++        leaq    TAB+JACSIZE*2(%rsp), %rdi+        leaq    TAB+JACSIZE*1(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjadd)++        leaq    TAB+JACSIZE*3(%rsp), %rdi+        leaq    TAB+JACSIZE*1(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjdouble)++        leaq    TAB+JACSIZE*4(%rsp), %rdi+        leaq    TAB+JACSIZE*3(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjadd)++        leaq    TAB+JACSIZE*5(%rsp), %rdi+        leaq    TAB+JACSIZE*2(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjdouble)++        leaq    TAB+JACSIZE*6(%rsp), %rdi+        leaq    TAB+JACSIZE*5(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjadd)++        leaq    TAB+JACSIZE*7(%rsp), %rdi+        leaq    TAB+JACSIZE*3(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjdouble)++        leaq    TAB+JACSIZE*8(%rsp), %rdi+        leaq    TAB+JACSIZE*7(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjadd)++        leaq    TAB+JACSIZE*9(%rsp), %rdi+        leaq    TAB+JACSIZE*4(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjdouble)++        leaq    TAB+JACSIZE*10(%rsp), %rdi+        leaq    TAB+JACSIZE*9(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjadd)++        leaq    TAB+JACSIZE*11(%rsp), %rdi+        leaq    TAB+JACSIZE*5(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjdouble)++        leaq    TAB+JACSIZE*12(%rsp), %rdi+        leaq    TAB+JACSIZE*11(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjadd)++        leaq    TAB+JACSIZE*13(%rsp), %rdi+        leaq    TAB+JACSIZE*6(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjdouble)++        leaq    TAB+JACSIZE*14(%rsp), %rdi+        leaq    TAB+JACSIZE*13(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjadd)++        leaq    TAB+JACSIZE*15(%rsp), %rdi+        leaq    TAB+JACSIZE*7(%rsp), %rsi+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjdouble)++// Add the recoding constant sum_i(16 * 32^i) to the scalar to allow signed+// digits. The digits of the constant, in lowest-to-highest order, are as+// follows; they are generated dynamically to use fewer large constant loads.+//+// 0x0842108421084210+// 0x1084210842108421+// 0x2108421084210842+// 0x4210842108421084+// 0x8421084210842108+// 0x0842108421084210++        movq    $0x1084210842108421, %rax+        movq    %rax, %rcx+        shrq    $1, %rax+        movq    SCALARB(%rsp), %r8+        addq    %rax, %r8+        movq    SCALARB+8(%rsp), %r9+        adcq    %rcx, %r9+        leaq    (%rcx,%rcx), %rcx+        movq    SCALARB+16(%rsp), %r10+        adcq    %rcx, %r10+        leaq    (%rcx,%rcx), %rcx+        movq    SCALARB+24(%rsp), %r11+        adcq    %rcx, %r11+        leaq    (%rcx,%rcx), %rcx+        movq    SCALARB+32(%rsp), %r12+        adcq    %rcx, %r12+        movq    SCALARB+40(%rsp), %r13+        adcq    %rax, %r13+        sbbq    %rdi, %rdi+        negq    %rdi++// Record the top bitfield in %rdi then shift the whole scalar left 4 bits+// to align the top of the next bitfield with the MSB (bits 379..383).++        shldq   $4, %r13, %rdi+        shldq   $4, %r12, %r13+        shldq   $4, %r11, %r12+        shldq   $4, %r10, %r11+        shldq   $4, %r9, %r10+        shldq   $4, %r8, %r9+        shlq    $4, %r8++        movq    %r8, SCALARB(%rsp)+        movq    %r9, SCALARB+8(%rsp)+        movq    %r10, SCALARB+16(%rsp)+        movq    %r11, SCALARB+24(%rsp)+        movq    %r12, SCALARB+32(%rsp)+        movq    %r13, SCALARB+40(%rsp)++// Initialize the accumulator to the corresponding entry using constant-time+// lookup in the table. This top digit, uniquely, is not recoded so there is+// no sign adjustment to make. On the x86 integer side we don't have enough+// registers to hold all the fields; this could be better done with SIMD+// registers anyway. So we do x and z coordinates in one sweep, y in another+// (this is a rehearsal for below where we might need to negate the y).++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        xorl    %r13d, %r13d+        xorl    %r14d, %r14d+        xorl    %r15d, %r15d++        selectblock_xz(1)+        selectblock_xz(2)+        selectblock_xz(3)+        selectblock_xz(4)+        selectblock_xz(5)+        selectblock_xz(6)+        selectblock_xz(7)+        selectblock_xz(8)+        selectblock_xz(9)+        selectblock_xz(10)+        selectblock_xz(11)+        selectblock_xz(12)+        selectblock_xz(13)+        selectblock_xz(14)+        selectblock_xz(15)+        selectblock_xz(16)++        movq     %rax, ACC(%rsp)+        movq     %rbx, ACC+8(%rsp)+        movq     %rcx, ACC+16(%rsp)+        movq     %rdx, ACC+24(%rsp)+        movq     %r8, ACC+32(%rsp)+        movq     %r9, ACC+40(%rsp)+        movq     %r10, ACC+96(%rsp)+        movq     %r11, ACC+104(%rsp)+        movq     %r12, ACC+112(%rsp)+        movq     %r13, ACC+120(%rsp)+        movq     %r14, ACC+128(%rsp)+        movq     %r15, ACC+136(%rsp)++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d++        selectblock_y(1)+        selectblock_y(2)+        selectblock_y(3)+        selectblock_y(4)+        selectblock_y(5)+        selectblock_y(6)+        selectblock_y(7)+        selectblock_y(8)+        selectblock_y(9)+        selectblock_y(10)+        selectblock_y(11)+        selectblock_y(12)+        selectblock_y(13)+        selectblock_y(14)+        selectblock_y(15)+        selectblock_y(16)++        movq     %rax, ACC+48(%rsp)+        movq     %rbx, ACC+56(%rsp)+        movq     %rcx, ACC+64(%rsp)+        movq     %rdx, ACC+72(%rsp)+        movq     %r8, ACC+80(%rsp)+        movq     %r9, ACC+88(%rsp)++// Main loop over size-5 bitfields: double 5 times then add signed digit+// At each stage we shift the scalar left by 5 bits so we can simply pick+// the top 5 bits as the bitfield, saving some fiddle over indexing.++        movl    $380, %ebp++Lp384_montjscalarmul_alt_mainloop:+        subq    $5, %rbp++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjdouble)++// Choose the bitfield and adjust it to sign and magnitude++        movq    SCALARB(%rsp), %r8+        movq    SCALARB+8(%rsp), %r9+        movq    SCALARB+16(%rsp), %r10+        movq    SCALARB+24(%rsp), %r11+        movq    SCALARB+32(%rsp), %r12+        movq    SCALARB+40(%rsp), %r13++        movq    %r13, %rdi+        shrq    $59, %rdi+        shldq   $5, %r12, %r13+        shldq   $5, %r11, %r12+        shldq   $5, %r10, %r11+        shldq   $5, %r9, %r10+        shldq   $5, %r8, %r9+        shlq    $5, %r8++        movq    %r8, SCALARB(%rsp)+        movq    %r9, SCALARB+8(%rsp)+        movq    %r10, SCALARB+16(%rsp)+        movq    %r11, SCALARB+24(%rsp)+        movq    %r12, SCALARB+32(%rsp)+        movq    %r13, SCALARB+40(%rsp)++        subq    $16, %rdi+        sbbq    %rsi, %rsi // %rsi = sign of digit (-1 = negative)+        xorq    %rsi, %rdi+        subq    %rsi, %rdi // %rdi = absolute value of digit++// Conditionally select the table entry tab[i-1] = i * P in constant time+// Again, this is done in two sweeps, first doing x and z then y.++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        xorl    %r13d, %r13d+        xorl    %r14d, %r14d+        xorl    %r15d, %r15d++selectblock_xz(1)+        selectblock_xz(2)+        selectblock_xz(3)+        selectblock_xz(4)+        selectblock_xz(5)+        selectblock_xz(6)+        selectblock_xz(7)+        selectblock_xz(8)+        selectblock_xz(9)+        selectblock_xz(10)+        selectblock_xz(11)+        selectblock_xz(12)+        selectblock_xz(13)+        selectblock_xz(14)+        selectblock_xz(15)+        selectblock_xz(16)++        movq     %rax, TABENT(%rsp)+        movq     %rbx, TABENT+8(%rsp)+        movq     %rcx, TABENT+16(%rsp)+        movq     %rdx, TABENT+24(%rsp)+        movq     %r8, TABENT+32(%rsp)+        movq     %r9, TABENT+40(%rsp)+        movq     %r10, TABENT+96(%rsp)+        movq     %r11, TABENT+104(%rsp)+        movq     %r12, TABENT+112(%rsp)+        movq     %r13, TABENT+120(%rsp)+        movq     %r14, TABENT+128(%rsp)+        movq     %r15, TABENT+136(%rsp)++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d++        selectblock_y(1)+        selectblock_y(2)+        selectblock_y(3)+        selectblock_y(4)+        selectblock_y(5)+        selectblock_y(6)+        selectblock_y(7)+        selectblock_y(8)+        selectblock_y(9)+        selectblock_y(10)+        selectblock_y(11)+        selectblock_y(12)+        selectblock_y(13)+        selectblock_y(14)+        selectblock_y(15)+        selectblock_y(16)++// Store it to "tabent" with the y coordinate optionally negated.+// This is done carefully to give coordinates < p_384 even in+// the degenerate case y = 0 (when z = 0 for points on the curve).+// The digits of the prime p_384 are generated dynamically from+// the zeroth via not/lea to reduce the number of constant loads.++        movq    %rax, %r10+        orq     %rbx, %r10+        movq    %rcx, %r11+        orq     %rdx, %r11+        movq    %r8, %r12+        orq     %r9, %r12+        orq     %r11, %r10+        orq     %r12, %r10+        cmovzq  %r10, %rsi++        movl    $0xffffffff, %r10d+        movq    %r10, %r11+        notq    %r11+        leaq    (%r10,%r11), %r13+        subq    %rax, %r10+        leaq    -1(%r13), %r12+        sbbq    %rbx, %r11+        movq    %r13, %r14+        sbbq    %rcx, %r12+        sbbq    %rdx, %r13+        movq    %r14, %r15+        sbbq    %r8, %r14+        sbbq    %r9, %r15++        testq   %rsi, %rsi+        cmovnzq  %r10, %rax+        cmovnzq  %r11, %rbx+        cmovnzq  %r12, %rcx+        cmovnzq  %r13, %rdx+        cmovnzq  %r14, %r8+        cmovnzq  %r15, %r9++        movq    %rax, TABENT+48(%rsp)+        movq    %rbx, TABENT+56(%rsp)+        movq    %rcx, TABENT+64(%rsp)+        movq    %rdx, TABENT+72(%rsp)+        movq    %r8, TABENT+80(%rsp)+        movq    %r9, TABENT+88(%rsp)++// Add to the accumulator++        leaq    TABENT(%rsp), %rdx+        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp384_montjscalarmul_alt_p384_montjadd)++        testq   %rbp, %rbp+        jne     Lp384_montjscalarmul_alt_mainloop++// That's the end of the main loop, and we just need to copy the+// result in "acc" to the output.++        movq    res, %rdi+        movq    ACC(%rsp), %rax+        movq    %rax, (%rdi)+        movq    ACC+8(%rsp), %rax+        movq    %rax, 8(%rdi)+        movq    ACC+16(%rsp), %rax+        movq    %rax, 16(%rdi)+        movq    ACC+24(%rsp), %rax+        movq    %rax, 24(%rdi)+        movq    ACC+32(%rsp), %rax+        movq    %rax, 32(%rdi)+        movq    ACC+40(%rsp), %rax+        movq    %rax, 40(%rdi)+        movq    ACC+48(%rsp), %rax+        movq    %rax, 48(%rdi)+        movq    ACC+56(%rsp), %rax+        movq    %rax, 56(%rdi)+        movq    ACC+64(%rsp), %rax+        movq    %rax, 64(%rdi)+        movq    ACC+72(%rsp), %rax+        movq    %rax, 72(%rdi)+        movq    ACC+80(%rsp), %rax+        movq    %rax, 80(%rdi)+        movq    ACC+88(%rsp), %rax+        movq    %rax, 88(%rdi)+        movq    ACC+96(%rsp), %rax+        movq    %rax, 96(%rdi)+        movq    ACC+104(%rsp), %rax+        movq    %rax, 104(%rdi)+        movq    ACC+112(%rsp), %rax+        movq    %rax, 112(%rdi)+        movq    ACC+120(%rsp), %rax+        movq    %rax, 120(%rdi)+        movq    ACC+128(%rsp), %rax+        movq    %rax, 128(%rdi)+        movq    ACC+136(%rsp), %rax+        movq    %rax, 136(%rdi)++// Restore stack and registers and return++        CFI_INC_RSP(NSPACE)+        CFI_POP(%rbx)+        CFI_POP(%rbp)+        CFI_POP(%r12)+        CFI_POP(%r13)+        CFI_POP(%r14)+        CFI_POP(%r15)+        CFI_RET++#if WINDOWS_ABI+S2N_BN_SIZE_DIRECTIVE(Lp384_montjscalarmul_alt_standard)+#else+S2N_BN_SIZE_DIRECTIVE(p384_montjscalarmul_alt)+#endif++// Local copies of subroutines, complete clones at the moment++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp384_montjscalarmul_alt_p384_montjadd)++Lp384_montjscalarmul_alt_p384_montjadd:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(352)+        movq    %rsi, 0x150(%rsp)+        movq    %rdx, 0x158(%rsp)+        movq    0x60(%rsi), %rbx+        movq    0x68(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x78(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x88(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x78(%rsi), %rax+        mulq     0x80(%rsi)+        movq    %rax, %r15+        movq    %rdx, %rcx+        movq    0x70(%rsi), %rbx+        movq    0x60(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x68(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rbp, %rbp+        movq    0x68(%rsi), %rbx+        movq    0x78(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x80(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x88(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %rcx+        movq    0x80(%rsi), %rbx+        movq    0x60(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x70(%rsi), %rbx+        movq    0x78(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x80(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x88(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r15+        adcq    %rdx, %rcx+        sbbq    %rbp, %rbp+        xorl    %ebx, %ebx+        movq    0x78(%rsi), %rax+        mulq     0x88(%rsi)+        subq    %rbp, %rdx+        xorl    %ebp, %ebp+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        movq    0x80(%rsi), %rax+        mulq     0x88(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rbp+        xorl    %r8d, %r8d+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %r15, %r15+        adcq    %rcx, %rcx+        adcq    %rbx, %rbx+        adcq    %rbp, %rbp+        adcl    %r8d, %r8d+        movq    0x60(%rsi), %rax+        mulq    %rax+        movq    %r8, (%rsp)+        movq    %rax, %r8+        movq    0x68(%rsi), %rax+        movq    %rbp, 0x8(%rsp)+        addq    %rdx, %r9+        sbbq    %rbp, %rbp+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x70(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x78(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x80(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %rcx+        adcq    %rdx, %rbx+        sbbq    %rbp, %rbp+        movq    0x88(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    0x8(%rsp), %rax+        adcq    (%rsp), %rdx+        movq    %rax, %rbp+        movq    %rdx, %rsi+        movq    %rbx, (%rsp)+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r8+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r8, %r9+        sbbq    %rdx, %r10+        sbbq    %rax, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rbx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r9+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r9, %r10+        sbbq    %rdx, %r11+        sbbq    %rax, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rbx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r10+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r10, %r11+        sbbq    %rdx, %r12+        sbbq    %rax, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rbx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r11+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r11, %r12+        sbbq    %rdx, %r13+        sbbq    %rax, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rbx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r12+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r12, %r13+        sbbq    %rdx, %r8+        sbbq    %rax, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rbx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r13+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r13, %r8+        sbbq    %rdx, %r9+        sbbq    %rax, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rbx, %r13+        sbbq    $0x0, %r13+        movq    (%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rsi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rsi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rsi+        movq    %r14, (%rsp)+        movq    %r15, 0x8(%rsp)+        movq    %rcx, 0x10(%rsp)+        movq    %rbx, 0x18(%rsp)+        movq    %rbp, 0x20(%rsp)+        movq    %rsi, 0x28(%rsp)+        movq    0x158(%rsp), %rsi+        movq    0x60(%rsi), %rbx+        movq    0x68(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x78(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x88(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x78(%rsi), %rax+        mulq     0x80(%rsi)+        movq    %rax, %r15+        movq    %rdx, %rcx+        movq    0x70(%rsi), %rbx+        movq    0x60(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x68(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rbp, %rbp+        movq    0x68(%rsi), %rbx+        movq    0x78(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x80(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x88(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %rcx+        movq    0x80(%rsi), %rbx+        movq    0x60(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x70(%rsi), %rbx+        movq    0x78(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x80(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x88(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r15+        adcq    %rdx, %rcx+        sbbq    %rbp, %rbp+        xorl    %ebx, %ebx+        movq    0x78(%rsi), %rax+        mulq     0x88(%rsi)+        subq    %rbp, %rdx+        xorl    %ebp, %ebp+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        movq    0x80(%rsi), %rax+        mulq     0x88(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rbp+        xorl    %r8d, %r8d+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %r15, %r15+        adcq    %rcx, %rcx+        adcq    %rbx, %rbx+        adcq    %rbp, %rbp+        adcl    %r8d, %r8d+        movq    0x60(%rsi), %rax+        mulq    %rax+        movq    %r8, 0xf0(%rsp)+        movq    %rax, %r8+        movq    0x68(%rsi), %rax+        movq    %rbp, 0xf8(%rsp)+        addq    %rdx, %r9+        sbbq    %rbp, %rbp+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x70(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x78(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x80(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %rcx+        adcq    %rdx, %rbx+        sbbq    %rbp, %rbp+        movq    0x88(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    0xf8(%rsp), %rax+        adcq    0xf0(%rsp), %rdx+        movq    %rax, %rbp+        movq    %rdx, %rsi+        movq    %rbx, 0xf0(%rsp)+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r8+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r8, %r9+        sbbq    %rdx, %r10+        sbbq    %rax, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rbx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r9+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r9, %r10+        sbbq    %rdx, %r11+        sbbq    %rax, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rbx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r10+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r10, %r11+        sbbq    %rdx, %r12+        sbbq    %rax, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rbx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r11+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r11, %r12+        sbbq    %rdx, %r13+        sbbq    %rax, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rbx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r12+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r12, %r13+        sbbq    %rdx, %r8+        sbbq    %rax, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rbx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r13+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r13, %r8+        sbbq    %rdx, %r9+        sbbq    %rax, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rbx, %r13+        sbbq    $0x0, %r13+        movq    0xf0(%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rsi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rsi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rsi+        movq    %r14, 0xf0(%rsp)+        movq    %r15, 0xf8(%rsp)+        movq    %rcx, 0x100(%rsp)+        movq    %rbx, 0x108(%rsp)+        movq    %rbp, 0x110(%rsp)+        movq    %rsi, 0x118(%rsp)+        movq    0x150(%rsp), %rsi+        movq    0x158(%rsp), %rcx+        movq    0x30(%rsi), %rbx+        movq    0x60(%rcx), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x68(%rcx), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x70(%rcx), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x78(%rcx), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x80(%rcx), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x88(%rcx), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0x38(%rsi), %rbx+        movq    0x60(%rcx), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0x68(%rcx), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x70(%rcx), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x78(%rcx), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x80(%rcx), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0x88(%rcx), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0x40(%rsi), %rbx+        movq    0x60(%rcx), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0x68(%rcx), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x70(%rcx), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x78(%rcx), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x80(%rcx), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0x88(%rcx), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0x48(%rsi), %rbx+        movq    0x60(%rcx), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0x68(%rcx), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0x70(%rcx), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0x78(%rcx), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0x80(%rcx), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0x88(%rcx), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0x50(%rsi), %rbx+        movq    0x60(%rcx), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0x68(%rcx), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0x70(%rcx), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0x78(%rcx), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0x80(%rcx), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0x88(%rcx), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0x58(%rsi), %rbx+        movq    0x60(%rcx), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0x68(%rcx), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0x70(%rcx), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0x78(%rcx), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0x80(%rcx), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0x88(%rcx), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x120(%rsp)+        movq    %r15, 0x128(%rsp)+        movq    %r8, 0x130(%rsp)+        movq    %r9, 0x138(%rsp)+        movq    %r10, 0x140(%rsp)+        movq    %r11, 0x148(%rsp)+        movq    0x150(%rsp), %rsi+        movq    0x158(%rsp), %rcx+        movq    0x30(%rcx), %rbx+        movq    0x60(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x68(%rsi), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x70(%rsi), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x78(%rsi), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x80(%rsi), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x88(%rsi), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0x38(%rcx), %rbx+        movq    0x60(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0x68(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x70(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x78(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x80(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0x88(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0x40(%rcx), %rbx+        movq    0x60(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0x68(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x70(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x78(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x80(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0x88(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0x48(%rcx), %rbx+        movq    0x60(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0x68(%rsi), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0x70(%rsi), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0x78(%rsi), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0x80(%rsi), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0x88(%rsi), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0x50(%rcx), %rbx+        movq    0x60(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0x68(%rsi), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0x70(%rsi), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0x78(%rsi), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0x80(%rsi), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0x88(%rsi), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0x58(%rcx), %rbx+        movq    0x60(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0x68(%rsi), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0x70(%rsi), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0x78(%rsi), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0x80(%rsi), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0x88(%rsi), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    %r8, 0x40(%rsp)+        movq    %r9, 0x48(%rsp)+        movq    %r10, 0x50(%rsp)+        movq    %r11, 0x58(%rsp)+        movq    0x158(%rsp), %rcx+        movq    (%rcx), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x10(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x18(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x20(%rsp), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x28(%rsp), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0x8(%rcx), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x20(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0x10(%rcx), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x20(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0x18(%rcx), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0x20(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0x20(%rcx), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0x20(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0x28(%rcx), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0x20(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x60(%rsp)+        movq    %r15, 0x68(%rsp)+        movq    %r8, 0x70(%rsp)+        movq    %r9, 0x78(%rsp)+        movq    %r10, 0x80(%rsp)+        movq    %r11, 0x88(%rsp)+        movq    0x150(%rsp), %rsi+        movq    (%rsi), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x100(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x108(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x110(%rsp), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x118(%rsp), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0x8(%rsi), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0x10(%rsi), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0x18(%rsi), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0x20(%rsi), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0x28(%rsi), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0xc0(%rsp)+        movq    %r15, 0xc8(%rsp)+        movq    %r8, 0xd0(%rsp)+        movq    %r9, 0xd8(%rsp)+        movq    %r10, 0xe0(%rsp)+        movq    %r11, 0xe8(%rsp)+        movq    0x30(%rsp), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x10(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x18(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x20(%rsp), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x28(%rsp), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0x38(%rsp), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x20(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0x40(%rsp), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x20(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0x48(%rsp), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0x20(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0x50(%rsp), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0x20(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0x58(%rsp), %rbx+        movq    (%rsp), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0x8(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0x10(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0x18(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0x20(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0x28(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    %r8, 0x40(%rsp)+        movq    %r9, 0x48(%rsp)+        movq    %r10, 0x50(%rsp)+        movq    %r11, 0x58(%rsp)+        movq    0x120(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x100(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x108(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x110(%rsp), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x118(%rsp), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0x128(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0x130(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0x138(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0x140(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0x148(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x120(%rsp)+        movq    %r15, 0x128(%rsp)+        movq    %r8, 0x130(%rsp)+        movq    %r9, 0x138(%rsp)+        movq    %r10, 0x140(%rsp)+        movq    %r11, 0x148(%rsp)+        movq    0x60(%rsp), %rax+        subq    0xc0(%rsp), %rax+        movq    0x68(%rsp), %rdx+        sbbq    0xc8(%rsp), %rdx+        movq    0x70(%rsp), %r8+        sbbq    0xd0(%rsp), %r8+        movq    0x78(%rsp), %r9+        sbbq    0xd8(%rsp), %r9+        movq    0x80(%rsp), %r10+        sbbq    0xe0(%rsp), %r10+        movq    0x88(%rsp), %r11+        sbbq    0xe8(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %esi+        andq    %rsi, %rcx+        xorq    %rsi, %rsi+        subq    %rcx, %rsi+        subq    %rsi, %rax+        movq    %rax, 0xf0(%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0xf8(%rsp)+        sbbq    %rax, %rax+        andq    %rsi, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0x100(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x108(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x110(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x118(%rsp)+        movq    0x30(%rsp), %rax+        subq    0x120(%rsp), %rax+        movq    0x38(%rsp), %rdx+        sbbq    0x128(%rsp), %rdx+        movq    0x40(%rsp), %r8+        sbbq    0x130(%rsp), %r8+        movq    0x48(%rsp), %r9+        sbbq    0x138(%rsp), %r9+        movq    0x50(%rsp), %r10+        sbbq    0x140(%rsp), %r10+        movq    0x58(%rsp), %r11+        sbbq    0x148(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %esi+        andq    %rsi, %rcx+        xorq    %rsi, %rsi+        subq    %rcx, %rsi+        subq    %rsi, %rax+        movq    %rax, 0x30(%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0x38(%rsp)+        sbbq    %rax, %rax+        andq    %rsi, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0x40(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x48(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x50(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x58(%rsp)+        movq    0xf0(%rsp), %rbx+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x108(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x118(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x108(%rsp), %rax+        mulq     0x110(%rsp)+        movq    %rax, %r15+        movq    %rdx, %rcx+        movq    0x100(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rbp, %rbp+        movq    0xf8(%rsp), %rbx+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %rcx+        movq    0x110(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x100(%rsp), %rbx+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r15+        adcq    %rdx, %rcx+        sbbq    %rbp, %rbp+        xorl    %ebx, %ebx+        movq    0x108(%rsp), %rax+        mulq     0x118(%rsp)+        subq    %rbp, %rdx+        xorl    %ebp, %ebp+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        movq    0x110(%rsp), %rax+        mulq     0x118(%rsp)+        addq    %rax, %rbx+        adcq    %rdx, %rbp+        xorl    %r8d, %r8d+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %r15, %r15+        adcq    %rcx, %rcx+        adcq    %rbx, %rbx+        adcq    %rbp, %rbp+        adcl    %r8d, %r8d+        movq    0xf0(%rsp), %rax+        mulq    %rax+        movq    %r8, 0x90(%rsp)+        movq    %rax, %r8+        movq    0xf8(%rsp), %rax+        movq    %rbp, 0x98(%rsp)+        addq    %rdx, %r9+        sbbq    %rbp, %rbp+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x100(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x108(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x110(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %rcx+        adcq    %rdx, %rbx+        sbbq    %rbp, %rbp+        movq    0x118(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    0x98(%rsp), %rax+        adcq    0x90(%rsp), %rdx+        movq    %rax, %rbp+        movq    %rdx, %rsi+        movq    %rbx, 0x90(%rsp)+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r8+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r8, %r9+        sbbq    %rdx, %r10+        sbbq    %rax, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rbx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r9+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r9, %r10+        sbbq    %rdx, %r11+        sbbq    %rax, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rbx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r10+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r10, %r11+        sbbq    %rdx, %r12+        sbbq    %rax, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rbx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r11+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r11, %r12+        sbbq    %rdx, %r13+        sbbq    %rax, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rbx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r12+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r12, %r13+        sbbq    %rdx, %r8+        sbbq    %rax, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rbx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r13+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r13, %r8+        sbbq    %rdx, %r9+        sbbq    %rax, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rbx, %r13+        sbbq    $0x0, %r13+        movq    0x90(%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rsi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rsi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rsi+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    %rcx, 0xa0(%rsp)+        movq    %rbx, 0xa8(%rsp)+        movq    %rbp, 0xb0(%rsp)+        movq    %rsi, 0xb8(%rsp)+        movq    0x30(%rsp), %rbx+        movq    0x38(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x48(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x58(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x48(%rsp), %rax+        mulq     0x50(%rsp)+        movq    %rax, %r15+        movq    %rdx, %rcx+        movq    0x40(%rsp), %rbx+        movq    0x30(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rbp, %rbp+        movq    0x38(%rsp), %rbx+        movq    0x48(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x50(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x58(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %rcx+        movq    0x50(%rsp), %rbx+        movq    0x30(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x40(%rsp), %rbx+        movq    0x48(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x50(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x58(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r15+        adcq    %rdx, %rcx+        sbbq    %rbp, %rbp+        xorl    %ebx, %ebx+        movq    0x48(%rsp), %rax+        mulq     0x58(%rsp)+        subq    %rbp, %rdx+        xorl    %ebp, %ebp+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        movq    0x50(%rsp), %rax+        mulq     0x58(%rsp)+        addq    %rax, %rbx+        adcq    %rdx, %rbp+        xorl    %r8d, %r8d+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %r15, %r15+        adcq    %rcx, %rcx+        adcq    %rbx, %rbx+        adcq    %rbp, %rbp+        adcl    %r8d, %r8d+        movq    0x30(%rsp), %rax+        mulq    %rax+        movq    %r8, (%rsp)+        movq    %rax, %r8+        movq    0x38(%rsp), %rax+        movq    %rbp, 0x8(%rsp)+        addq    %rdx, %r9+        sbbq    %rbp, %rbp+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x40(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x48(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x50(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %rcx+        adcq    %rdx, %rbx+        sbbq    %rbp, %rbp+        movq    0x58(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    0x8(%rsp), %rax+        adcq    (%rsp), %rdx+        movq    %rax, %rbp+        movq    %rdx, %rsi+        movq    %rbx, (%rsp)+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r8+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r8, %r9+        sbbq    %rdx, %r10+        sbbq    %rax, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rbx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r9+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r9, %r10+        sbbq    %rdx, %r11+        sbbq    %rax, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rbx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r10+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r10, %r11+        sbbq    %rdx, %r12+        sbbq    %rax, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rbx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r11+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r11, %r12+        sbbq    %rdx, %r13+        sbbq    %rax, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rbx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r12+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r12, %r13+        sbbq    %rdx, %r8+        sbbq    %rax, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rbx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r13+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r13, %r8+        sbbq    %rdx, %r9+        sbbq    %rax, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rbx, %r13+        sbbq    $0x0, %r13+        movq    (%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rsi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rsi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rsi+        movq    %r14, (%rsp)+        movq    %r15, 0x8(%rsp)+        movq    %rcx, 0x10(%rsp)+        movq    %rbx, 0x18(%rsp)+        movq    %rbp, 0x20(%rsp)+        movq    %rsi, 0x28(%rsp)+        movq    0xc0(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x98(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0xc8(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0xd0(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0xd8(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0xe0(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0xe8(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0xc0(%rsp)+        movq    %r15, 0xc8(%rsp)+        movq    %r8, 0xd0(%rsp)+        movq    %r9, 0xd8(%rsp)+        movq    %r10, 0xe0(%rsp)+        movq    %r11, 0xe8(%rsp)+        movq    0x60(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x98(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0x68(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0x70(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0x78(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0x80(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0x88(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x60(%rsp)+        movq    %r15, 0x68(%rsp)+        movq    %r8, 0x70(%rsp)+        movq    %r9, 0x78(%rsp)+        movq    %r10, 0x80(%rsp)+        movq    %r11, 0x88(%rsp)+        movq    (%rsp), %rax+        subq    0xc0(%rsp), %rax+        movq    0x8(%rsp), %rdx+        sbbq    0xc8(%rsp), %rdx+        movq    0x10(%rsp), %r8+        sbbq    0xd0(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0xd8(%rsp), %r9+        movq    0x20(%rsp), %r10+        sbbq    0xe0(%rsp), %r10+        movq    0x28(%rsp), %r11+        sbbq    0xe8(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %esi+        andq    %rsi, %rcx+        xorq    %rsi, %rsi+        subq    %rcx, %rsi+        subq    %rsi, %rax+        movq    %rax, (%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0x8(%rsp)+        sbbq    %rax, %rax+        andq    %rsi, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0x10(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x18(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x20(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x28(%rsp)+        movq    0x60(%rsp), %rax+        subq    0xc0(%rsp), %rax+        movq    0x68(%rsp), %rdx+        sbbq    0xc8(%rsp), %rdx+        movq    0x70(%rsp), %r8+        sbbq    0xd0(%rsp), %r8+        movq    0x78(%rsp), %r9+        sbbq    0xd8(%rsp), %r9+        movq    0x80(%rsp), %r10+        sbbq    0xe0(%rsp), %r10+        movq    0x88(%rsp), %r11+        sbbq    0xe8(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %esi+        andq    %rsi, %rcx+        xorq    %rsi, %rsi+        subq    %rcx, %rsi+        subq    %rsi, %rax+        movq    %rax, 0x90(%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0x98(%rsp)+        sbbq    %rax, %rax+        andq    %rsi, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0xa0(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0xa8(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0xb0(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0xb8(%rsp)+        movq    0x150(%rsp), %rsi+        movq    0x60(%rsi), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x100(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x108(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x110(%rsp), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x118(%rsp), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0x68(%rsi), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0x70(%rsi), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0x78(%rsi), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0x80(%rsi), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0x88(%rsi), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0xf0(%rsp)+        movq    %r15, 0xf8(%rsp)+        movq    %r8, 0x100(%rsp)+        movq    %r9, 0x108(%rsp)+        movq    %r10, 0x110(%rsp)+        movq    %r11, 0x118(%rsp)+        movq    (%rsp), %rax+        subq    0x60(%rsp), %rax+        movq    0x8(%rsp), %rdx+        sbbq    0x68(%rsp), %rdx+        movq    0x10(%rsp), %r8+        sbbq    0x70(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x78(%rsp), %r9+        movq    0x20(%rsp), %r10+        sbbq    0x80(%rsp), %r10+        movq    0x28(%rsp), %r11+        sbbq    0x88(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %esi+        andq    %rsi, %rcx+        xorq    %rsi, %rsi+        subq    %rcx, %rsi+        subq    %rsi, %rax+        movq    %rax, (%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0x8(%rsp)+        sbbq    %rax, %rax+        andq    %rsi, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0x10(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x18(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x20(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x28(%rsp)+        movq    0xc0(%rsp), %rax+        subq    (%rsp), %rax+        movq    0xc8(%rsp), %rdx+        sbbq    0x8(%rsp), %rdx+        movq    0xd0(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0xd8(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movq    0xe0(%rsp), %r10+        sbbq    0x20(%rsp), %r10+        movq    0xe8(%rsp), %r11+        sbbq    0x28(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %esi+        andq    %rsi, %rcx+        xorq    %rsi, %rsi+        subq    %rcx, %rsi+        subq    %rsi, %rax+        movq    %rax, 0xc0(%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0xc8(%rsp)+        sbbq    %rax, %rax+        andq    %rsi, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0xd0(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0xd8(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0xe0(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0xe8(%rsp)+        movq    0x120(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x98(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0x128(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0x130(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0x138(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0x140(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0x148(%rsp), %rbx+        movq    0x90(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0x98(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0xa0(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0xa8(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0xb0(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0xb8(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    %r8, 0xa0(%rsp)+        movq    %r9, 0xa8(%rsp)+        movq    %r10, 0xb0(%rsp)+        movq    %r11, 0xb8(%rsp)+        movq    0x158(%rsp), %rcx+        movq    0x60(%rcx), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x100(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x108(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x110(%rsp), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x118(%rsp), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0x68(%rcx), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0x70(%rcx), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0x78(%rcx), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0x80(%rcx), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0x88(%rcx), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0xf0(%rsp)+        movq    %r15, 0xf8(%rsp)+        movq    %r8, 0x100(%rsp)+        movq    %r9, 0x108(%rsp)+        movq    %r10, 0x110(%rsp)+        movq    %r11, 0x118(%rsp)+        movq    0xc0(%rsp), %rbx+        movq    0x30(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x38(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x40(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x48(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x50(%rsp), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x58(%rsp), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0xc8(%rsp), %rbx+        movq    0x30(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x40(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x48(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x50(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0x58(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0xd0(%rsp), %rbx+        movq    0x30(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x40(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x48(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x50(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0x58(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0xd8(%rsp), %rbx+        movq    0x30(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0x40(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0x48(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0x50(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0x58(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0xe0(%rsp), %rbx+        movq    0x30(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0x40(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0x48(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0x50(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0x58(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0xe8(%rsp), %rbx+        movq    0x30(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0x40(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0x48(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0x50(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0x58(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0xc0(%rsp)+        movq    %r15, 0xc8(%rsp)+        movq    %r8, 0xd0(%rsp)+        movq    %r9, 0xd8(%rsp)+        movq    %r10, 0xe0(%rsp)+        movq    %r11, 0xe8(%rsp)+        movq    0xc0(%rsp), %rax+        subq    0x90(%rsp), %rax+        movq    0xc8(%rsp), %rdx+        sbbq    0x98(%rsp), %rdx+        movq    0xd0(%rsp), %r8+        sbbq    0xa0(%rsp), %r8+        movq    0xd8(%rsp), %r9+        sbbq    0xa8(%rsp), %r9+        movq    0xe0(%rsp), %r10+        sbbq    0xb0(%rsp), %r10+        movq    0xe8(%rsp), %r11+        sbbq    0xb8(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %esi+        andq    %rsi, %rcx+        xorq    %rsi, %rsi+        subq    %rcx, %rsi+        subq    %rsi, %rax+        movq    %rax, 0xc0(%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0xc8(%rsp)+        sbbq    %rax, %rax+        andq    %rsi, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0xd0(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0xd8(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0xe0(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0xe8(%rsp)+        movq    0x158(%rsp), %rcx+        movq    0x60(%rcx), %r8+        movq    0x68(%rcx), %r9+        movq    0x70(%rcx), %r10+        movq    0x78(%rcx), %r11+        movq    0x80(%rcx), %rbx+        movq    0x88(%rcx), %rbp+        movq    %r8, %rax+        movq    %r9, %rdx+        orq     %r10, %rax+        orq     %r11, %rdx+        orq     %rbx, %rax+        orq     %rbp, %rdx+        orq     %rdx, %rax+        negq    %rax+        sbbq    %rax, %rax+        movq    0x150(%rsp), %rsi+        movq    0x60(%rsi), %r12+        movq    0x68(%rsi), %r13+        movq    0x70(%rsi), %r14+        movq    0x78(%rsi), %r15+        movq    0x80(%rsi), %rdx+        movq    0x88(%rsi), %rcx+        cmoveq  %r12, %r8+        cmoveq  %r13, %r9+        cmoveq  %r14, %r10+        cmoveq  %r15, %r11+        cmoveq  %rdx, %rbx+        cmoveq  %rcx, %rbp+        orq     %r13, %r12+        orq     %r15, %r14+        orq     %rcx, %rdx+        orq     %r14, %r12+        orq     %r12, %rdx+        negq    %rdx+        sbbq    %rdx, %rdx+        cmpq    %rdx, %rax+        cmoveq  0xf0(%rsp), %r8+        cmoveq  0xf8(%rsp), %r9+        cmoveq  0x100(%rsp), %r10+        cmoveq  0x108(%rsp), %r11+        cmoveq  0x110(%rsp), %rbx+        cmoveq  0x118(%rsp), %rbp+        movq    %r8, 0xf0(%rsp)+        movq    %r9, 0xf8(%rsp)+        movq    %r10, 0x100(%rsp)+        movq    %r11, 0x108(%rsp)+        movq    %rbx, 0x110(%rsp)+        movq    %rbp, 0x118(%rsp)+        movq    0x158(%rsp), %rcx+        movq    0x150(%rsp), %rsi+        movq    (%rsp), %r8+        cmovbq  (%rsi), %r8+        cmova   (%rcx), %r8+        movq    0x8(%rsp), %r9+        cmovbq  0x8(%rsi), %r9+        cmova   0x8(%rcx), %r9+        movq    0x10(%rsp), %r10+        cmovbq  0x10(%rsi), %r10+        cmova   0x10(%rcx), %r10+        movq    0x18(%rsp), %r11+        cmovbq  0x18(%rsi), %r11+        cmova   0x18(%rcx), %r11+        movq    0x20(%rsp), %rbx+        cmovbq  0x20(%rsi), %rbx+        cmova   0x20(%rcx), %rbx+        movq    0x28(%rsp), %rbp+        cmovbq  0x28(%rsi), %rbp+        cmova   0x28(%rcx), %rbp+        movq    0xc0(%rsp), %r12+        cmovbq  0x30(%rsi), %r12+        cmova   0x30(%rcx), %r12+        movq    0xc8(%rsp), %r13+        cmovbq  0x38(%rsi), %r13+        cmova   0x38(%rcx), %r13+        movq    0xd0(%rsp), %r14+        cmovbq  0x40(%rsi), %r14+        cmova   0x40(%rcx), %r14+        movq    0xd8(%rsp), %r15+        cmovbq  0x48(%rsi), %r15+        cmova   0x48(%rcx), %r15+        movq    0xe0(%rsp), %rdx+        cmovbq  0x50(%rsi), %rdx+        cmova   0x50(%rcx), %rdx+        movq    0xe8(%rsp), %rax+        cmovbq  0x58(%rsi), %rax+        cmova   0x58(%rcx), %rax+        movq    %r8, (%rdi)+        movq    %r9, 0x8(%rdi)+        movq    %r10, 0x10(%rdi)+        movq    %r11, 0x18(%rdi)+        movq    %rbx, 0x20(%rdi)+        movq    %rbp, 0x28(%rdi)+        movq    0xf0(%rsp), %r8+        movq    0xf8(%rsp), %r9+        movq    0x100(%rsp), %r10+        movq    0x108(%rsp), %r11+        movq    0x110(%rsp), %rbx+        movq    0x118(%rsp), %rbp+        movq    %r12, 0x30(%rdi)+        movq    %r13, 0x38(%rdi)+        movq    %r14, 0x40(%rdi)+        movq    %r15, 0x48(%rdi)+        movq    %rdx, 0x50(%rdi)+        movq    %rax, 0x58(%rdi)+        movq    %r8, 0x60(%rdi)+        movq    %r9, 0x68(%rdi)+        movq    %r10, 0x70(%rdi)+        movq    %r11, 0x78(%rdi)+        movq    %rbx, 0x80(%rdi)+        movq    %rbp, 0x88(%rdi)+        CFI_INC_RSP(352)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp384_montjscalarmul_alt_p384_montjadd)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp384_montjscalarmul_alt_p384_montjdouble)++Lp384_montjscalarmul_alt_p384_montjdouble:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(344)+        movq    %rdi, 0x150(%rsp)+        movq    0x60(%rsi), %rbx+        movq    0x68(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x78(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x88(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x78(%rsi), %rax+        mulq     0x80(%rsi)+        movq    %rax, %r15+        movq    %rdx, %rcx+        movq    0x70(%rsi), %rbx+        movq    0x60(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x68(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rbp, %rbp+        movq    0x68(%rsi), %rbx+        movq    0x78(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x80(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x88(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %rcx+        movq    0x80(%rsi), %rbx+        movq    0x60(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x70(%rsi), %rbx+        movq    0x78(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x80(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x88(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r15+        adcq    %rdx, %rcx+        sbbq    %rbp, %rbp+        xorl    %ebx, %ebx+        movq    0x78(%rsi), %rax+        mulq     0x88(%rsi)+        subq    %rbp, %rdx+        xorl    %ebp, %ebp+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        movq    0x80(%rsi), %rax+        mulq     0x88(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rbp+        xorl    %r8d, %r8d+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %r15, %r15+        adcq    %rcx, %rcx+        adcq    %rbx, %rbx+        adcq    %rbp, %rbp+        adcl    %r8d, %r8d+        movq    0x60(%rsi), %rax+        mulq    %rax+        movq    %r8, (%rsp)+        movq    %rax, %r8+        movq    0x68(%rsi), %rax+        movq    %rbp, 0x8(%rsp)+        addq    %rdx, %r9+        sbbq    %rbp, %rbp+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x70(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x78(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x80(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %rcx+        adcq    %rdx, %rbx+        sbbq    %rbp, %rbp+        movq    0x88(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    0x8(%rsp), %rax+        adcq    (%rsp), %rdx+        movq    %rax, %rbp+        movq    %rdx, %rdi+        movq    %rbx, (%rsp)+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r8+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r8, %r9+        sbbq    %rdx, %r10+        sbbq    %rax, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rbx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r9+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r9, %r10+        sbbq    %rdx, %r11+        sbbq    %rax, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rbx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r10+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r10, %r11+        sbbq    %rdx, %r12+        sbbq    %rax, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rbx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r11+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r11, %r12+        sbbq    %rdx, %r13+        sbbq    %rax, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rbx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r12+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r12, %r13+        sbbq    %rdx, %r8+        sbbq    %rax, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rbx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r13+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r13, %r8+        sbbq    %rdx, %r9+        sbbq    %rax, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rbx, %r13+        sbbq    $0x0, %r13+        movq    (%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rdi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rdi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rdi+        movq    %r14, (%rsp)+        movq    %r15, 0x8(%rsp)+        movq    %rcx, 0x10(%rsp)+        movq    %rbx, 0x18(%rsp)+        movq    %rbp, 0x20(%rsp)+        movq    %rdi, 0x28(%rsp)+        movq    0x30(%rsi), %rbx+        movq    0x38(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x48(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x58(%rsi), %rax+        mulq    %rbx+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x48(%rsi), %rax+        mulq     0x50(%rsi)+        movq    %rax, %r15+        movq    %rdx, %rcx+        movq    0x40(%rsi), %rbx+        movq    0x30(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x38(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rbp, %rbp+        movq    0x38(%rsi), %rbx+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x50(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x58(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %rcx+        movq    0x50(%rsi), %rbx+        movq    0x30(%rsi), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x40(%rsi), %rbx+        movq    0x48(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x50(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x58(%rsi), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r15+        adcq    %rdx, %rcx+        sbbq    %rbp, %rbp+        xorl    %ebx, %ebx+        movq    0x48(%rsi), %rax+        mulq     0x58(%rsi)+        subq    %rbp, %rdx+        xorl    %ebp, %ebp+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        movq    0x50(%rsi), %rax+        mulq     0x58(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rbp+        xorl    %r8d, %r8d+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %r15, %r15+        adcq    %rcx, %rcx+        adcq    %rbx, %rbx+        adcq    %rbp, %rbp+        adcl    %r8d, %r8d+        movq    0x30(%rsi), %rax+        mulq    %rax+        movq    %r8, 0x30(%rsp)+        movq    %rax, %r8+        movq    0x38(%rsi), %rax+        movq    %rbp, 0x38(%rsp)+        addq    %rdx, %r9+        sbbq    %rbp, %rbp+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x40(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x48(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x50(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %rcx+        adcq    %rdx, %rbx+        sbbq    %rbp, %rbp+        movq    0x58(%rsi), %rax+        mulq    %rax+        negq    %rbp+        adcq    0x38(%rsp), %rax+        adcq    0x30(%rsp), %rdx+        movq    %rax, %rbp+        movq    %rdx, %rdi+        movq    %rbx, 0x30(%rsp)+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r8+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r8, %r9+        sbbq    %rdx, %r10+        sbbq    %rax, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rbx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r9+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r9, %r10+        sbbq    %rdx, %r11+        sbbq    %rax, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rbx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r10+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r10, %r11+        sbbq    %rdx, %r12+        sbbq    %rax, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rbx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r11+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r11, %r12+        sbbq    %rdx, %r13+        sbbq    %rax, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rbx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r12+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r12, %r13+        sbbq    %rdx, %r8+        sbbq    %rax, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rbx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r13+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r13, %r8+        sbbq    %rdx, %r9+        sbbq    %rax, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rbx, %r13+        sbbq    $0x0, %r13+        movq    0x30(%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rdi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rdi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rdi+        movq    %r14, 0x30(%rsp)+        movq    %r15, 0x38(%rsp)+        movq    %rcx, 0x40(%rsp)+        movq    %rbx, 0x48(%rsp)+        movq    %rbp, 0x50(%rsp)+        movq    %rdi, 0x58(%rsp)+        movq    (%rsi), %rax+        addq    (%rsp), %rax+        movq    0x8(%rsi), %rcx+        adcq    0x8(%rsp), %rcx+        movq    0x10(%rsi), %r8+        adcq    0x10(%rsp), %r8+        movq    0x18(%rsi), %r9+        adcq    0x18(%rsp), %r9+        movq    0x20(%rsi), %r10+        adcq    0x20(%rsp), %r10+        movq    0x28(%rsi), %r11+        adcq    0x28(%rsp), %r11+        sbbq    %rdx, %rdx+        movl    $0x1, %ebx+        andq    %rdx, %rbx+        movl    $0xffffffff, %ebp+        andq    %rbp, %rdx+        xorq    %rbp, %rbp+        subq    %rdx, %rbp+        addq    %rbp, %rax+        movq    %rax, 0xf0(%rsp)+        adcq    %rdx, %rcx+        movq    %rcx, 0xf8(%rsp)+        adcq    %rbx, %r8+        movq    %r8, 0x100(%rsp)+        adcq    $0x0, %r9+        movq    %r9, 0x108(%rsp)+        adcq    $0x0, %r10+        movq    %r10, 0x110(%rsp)+        adcq    $0x0, %r11+        movq    %r11, 0x118(%rsp)+        movq    (%rsi), %rax+        subq    (%rsp), %rax+        movq    0x8(%rsi), %rdx+        sbbq    0x8(%rsp), %rdx+        movq    0x10(%rsi), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x18(%rsi), %r9+        sbbq    0x18(%rsp), %r9+        movq    0x20(%rsi), %r10+        sbbq    0x20(%rsp), %r10+        movq    0x28(%rsi), %r11+        sbbq    0x28(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %ebx+        andq    %rbx, %rcx+        xorq    %rbx, %rbx+        subq    %rcx, %rbx+        subq    %rbx, %rax+        movq    %rax, 0xc0(%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0xc8(%rsp)+        sbbq    %rax, %rax+        andq    %rbx, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0xd0(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0xd8(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0xe0(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0xe8(%rsp)+        movq    0xc0(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x100(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x108(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x110(%rsp), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x118(%rsp), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0xc8(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0xd0(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0xd8(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0xe0(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0xe8(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0x100(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x60(%rsp)+        movq    %r15, 0x68(%rsp)+        movq    %r8, 0x70(%rsp)+        movq    %r9, 0x78(%rsp)+        movq    %r10, 0x80(%rsp)+        movq    %r11, 0x88(%rsp)+        movq    0x30(%rsi), %rax+        addq    0x60(%rsi), %rax+        movq    0x38(%rsi), %rcx+        adcq    0x68(%rsi), %rcx+        movq    0x40(%rsi), %r8+        adcq    0x70(%rsi), %r8+        movq    0x48(%rsi), %r9+        adcq    0x78(%rsi), %r9+        movq    0x50(%rsi), %r10+        adcq    0x80(%rsi), %r10+        movq    0x58(%rsi), %r11+        adcq    0x88(%rsi), %r11+        movl    $0x0, %edx+        adcq    %rdx, %rdx+        movabsq $0xffffffff00000001, %rbp+        addq    %rbp, %rax+        movl    $0xffffffff, %ebp+        adcq    %rbp, %rcx+        adcq    $0x1, %r8+        adcq    $0x0, %r9+        adcq    $0x0, %r10+        adcq    $0x0, %r11+        adcq    $0xffffffffffffffff, %rdx+        movl    $0x1, %ebx+        andq    %rdx, %rbx+        andq    %rbp, %rdx+        xorq    %rbp, %rbp+        subq    %rdx, %rbp+        subq    %rbp, %rax+        movq    %rax, 0xf0(%rsp)+        sbbq    %rdx, %rcx+        movq    %rcx, 0xf8(%rsp)+        sbbq    %rbx, %r8+        movq    %r8, 0x100(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x108(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x110(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x118(%rsp)+        movq    0x60(%rsp), %rbx+        movq    0x68(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x78(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x88(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x78(%rsp), %rax+        mulq     0x80(%rsp)+        movq    %rax, %r15+        movq    %rdx, %rcx+        movq    0x70(%rsp), %rbx+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x68(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rbp, %rbp+        movq    0x68(%rsp), %rbx+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x80(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x88(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %rcx+        movq    0x80(%rsp), %rbx+        movq    0x60(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x70(%rsp), %rbx+        movq    0x78(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x80(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x88(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r15+        adcq    %rdx, %rcx+        sbbq    %rbp, %rbp+        xorl    %ebx, %ebx+        movq    0x78(%rsp), %rax+        mulq     0x88(%rsp)+        subq    %rbp, %rdx+        xorl    %ebp, %ebp+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        movq    0x80(%rsp), %rax+        mulq     0x88(%rsp)+        addq    %rax, %rbx+        adcq    %rdx, %rbp+        xorl    %r8d, %r8d+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %r15, %r15+        adcq    %rcx, %rcx+        adcq    %rbx, %rbx+        adcq    %rbp, %rbp+        adcl    %r8d, %r8d+        movq    0x60(%rsp), %rax+        mulq    %rax+        movq    %r8, 0x120(%rsp)+        movq    %rax, %r8+        movq    0x68(%rsp), %rax+        movq    %rbp, 0x128(%rsp)+        addq    %rdx, %r9+        sbbq    %rbp, %rbp+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x70(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x78(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x80(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %rcx+        adcq    %rdx, %rbx+        sbbq    %rbp, %rbp+        movq    0x88(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    0x128(%rsp), %rax+        adcq    0x120(%rsp), %rdx+        movq    %rax, %rbp+        movq    %rdx, %rdi+        movq    %rbx, 0x120(%rsp)+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r8+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r8, %r9+        sbbq    %rdx, %r10+        sbbq    %rax, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rbx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r9+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r9, %r10+        sbbq    %rdx, %r11+        sbbq    %rax, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rbx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r10+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r10, %r11+        sbbq    %rdx, %r12+        sbbq    %rax, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rbx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r11+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r11, %r12+        sbbq    %rdx, %r13+        sbbq    %rax, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rbx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r12+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r12, %r13+        sbbq    %rdx, %r8+        sbbq    %rax, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rbx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r13+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r13, %r8+        sbbq    %rdx, %r9+        sbbq    %rax, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rbx, %r13+        sbbq    $0x0, %r13+        movq    0x120(%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rdi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rdi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rdi+        movq    %r14, 0x120(%rsp)+        movq    %r15, 0x128(%rsp)+        movq    %rcx, 0x130(%rsp)+        movq    %rbx, 0x138(%rsp)+        movq    %rbp, 0x140(%rsp)+        movq    %rdi, 0x148(%rsp)+        movq    0x30(%rsp), %rbx+        movq    (%rsi), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x8(%rsi), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x10(%rsi), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x18(%rsi), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x20(%rsi), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x28(%rsi), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0x38(%rsp), %rbx+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x10(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x18(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x20(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0x28(%rsi), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0x40(%rsp), %rbx+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x10(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x18(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x20(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0x28(%rsi), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0x48(%rsp), %rbx+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0x10(%rsi), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0x18(%rsi), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0x20(%rsi), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0x28(%rsi), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0x50(%rsp), %rbx+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0x10(%rsi), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0x18(%rsi), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0x20(%rsi), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0x28(%rsi), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0x58(%rsp), %rbx+        movq    (%rsi), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0x8(%rsi), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0x10(%rsi), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0x18(%rsi), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0x20(%rsi), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0x28(%rsi), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0x90(%rsp)+        movq    %r15, 0x98(%rsp)+        movq    %r8, 0xa0(%rsp)+        movq    %r9, 0xa8(%rsp)+        movq    %r10, 0xb0(%rsp)+        movq    %r11, 0xb8(%rsp)+        movq    0xf0(%rsp), %rbx+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x108(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x118(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x108(%rsp), %rax+        mulq     0x110(%rsp)+        movq    %rax, %r15+        movq    %rdx, %rcx+        movq    0x100(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0xf8(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rbp, %rbp+        movq    0xf8(%rsp), %rbx+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %rcx+        movq    0x110(%rsp), %rbx+        movq    0xf0(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x100(%rsp), %rbx+        movq    0x108(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x110(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x118(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r15+        adcq    %rdx, %rcx+        sbbq    %rbp, %rbp+        xorl    %ebx, %ebx+        movq    0x108(%rsp), %rax+        mulq     0x118(%rsp)+        subq    %rbp, %rdx+        xorl    %ebp, %ebp+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        movq    0x110(%rsp), %rax+        mulq     0x118(%rsp)+        addq    %rax, %rbx+        adcq    %rdx, %rbp+        xorl    %r8d, %r8d+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %r15, %r15+        adcq    %rcx, %rcx+        adcq    %rbx, %rbx+        adcq    %rbp, %rbp+        adcl    %r8d, %r8d+        movq    0xf0(%rsp), %rax+        mulq    %rax+        movq    %r8, 0xc0(%rsp)+        movq    %rax, %r8+        movq    0xf8(%rsp), %rax+        movq    %rbp, 0xc8(%rsp)+        addq    %rdx, %r9+        sbbq    %rbp, %rbp+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x100(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x108(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x110(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %rcx+        adcq    %rdx, %rbx+        sbbq    %rbp, %rbp+        movq    0x118(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    0xc8(%rsp), %rax+        adcq    0xc0(%rsp), %rdx+        movq    %rax, %rbp+        movq    %rdx, %rdi+        movq    %rbx, 0xc0(%rsp)+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r8+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r8, %r9+        sbbq    %rdx, %r10+        sbbq    %rax, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rbx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r9+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r9, %r10+        sbbq    %rdx, %r11+        sbbq    %rax, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rbx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r10+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r10, %r11+        sbbq    %rdx, %r12+        sbbq    %rax, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rbx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r11+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r11, %r12+        sbbq    %rdx, %r13+        sbbq    %rax, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rbx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r12+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r12, %r13+        sbbq    %rdx, %r8+        sbbq    %rax, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rbx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r13+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r13, %r8+        sbbq    %rdx, %r9+        sbbq    %rax, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rbx, %r13+        sbbq    $0x0, %r13+        movq    0xc0(%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rdi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rdi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rdi+        movq    %r14, 0xc0(%rsp)+        movq    %r15, 0xc8(%rsp)+        movq    %rcx, 0xd0(%rsp)+        movq    %rbx, 0xd8(%rsp)+        movq    %rbp, 0xe0(%rsp)+        movq    %rdi, 0xe8(%rsp)+        movabsq $0xffffffff, %r9+        subq    0x120(%rsp), %r9+        movabsq $0xffffffff00000000, %r10+        sbbq    0x128(%rsp), %r10+        movq    $0xfffffffffffffffe, %r11+        sbbq    0x130(%rsp), %r11+        movq    $0xffffffffffffffff, %r12+        sbbq    0x138(%rsp), %r12+        movq    $0xffffffffffffffff, %r13+        sbbq    0x140(%rsp), %r13+        movq    $0xffffffffffffffff, %r14+        sbbq    0x148(%rsp), %r14+        movq    $0x9, %rcx+        movq    %r9, %rax+        mulq    %rcx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    %r10, %rax+        xorl    %r10d, %r10d+        mulq    %rcx+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    %r11, %rax+        xorl    %r11d, %r11d+        mulq    %rcx+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    %r12, %rax+        xorl    %r12d, %r12d+        mulq    %rcx+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    %r13, %rax+        xorl    %r13d, %r13d+        mulq    %rcx+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    %r14, %rax+        movl    $0x1, %r14d+        mulq    %rcx+        addq    %rax, %r13+        adcq    %rdx, %r14+        movl    $0xc, %ecx+        movq    0x90(%rsp), %rax+        mulq    %rcx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %rbx, %rbx+        movq    0x98(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rbx, %rbx+        movq    0xa0(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbx, %rbx+        movq    0xa8(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rbx, %rbx+        movq    0xb0(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbx, %rbx+        movq    0xb8(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        movabsq $0xffffffff00000001, %rax+        mulq    %r14+        addq    %rax, %r8+        adcq    %rdx, %r9+        adcq    %r14, %r10+        movq    %r14, %rax+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %edx+        negq    %rcx+        mulq    %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        adcq    %rcx, %r11+        adcq    $0x0, %r12+        adcq    $0x0, %r13+        sbbq    %rcx, %rcx+        notq    %rcx+        movl    $0xffffffff, %edx+        xorq    %rax, %rax+        andq    %rcx, %rdx+        subq    %rdx, %rax+        andq    $0x1, %rcx+        subq    %rax, %r8+        movq    %r8, 0x120(%rsp)+        sbbq    %rdx, %r9+        movq    %r9, 0x128(%rsp)+        sbbq    %rcx, %r10+        movq    %r10, 0x130(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x138(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x140(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x148(%rsp)+        movq    0xc0(%rsp), %rax+        subq    (%rsp), %rax+        movq    0xc8(%rsp), %rdx+        sbbq    0x8(%rsp), %rdx+        movq    0xd0(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0xd8(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movq    0xe0(%rsp), %r10+        sbbq    0x20(%rsp), %r10+        movq    0xe8(%rsp), %r11+        sbbq    0x28(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %ebx+        andq    %rbx, %rcx+        xorq    %rbx, %rbx+        subq    %rcx, %rbx+        subq    %rbx, %rax+        movq    %rax, 0xf0(%rsp)+        sbbq    %rcx, %rdx+        movq    %rdx, 0xf8(%rsp)+        sbbq    %rax, %rax+        andq    %rbx, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0x100(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x108(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x110(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x118(%rsp)+        movq    0x30(%rsp), %rbx+        movq    0x38(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r9+        movq    %rdx, %r10+        movq    0x48(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r11+        movq    %rdx, %r12+        movq    0x58(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r13+        movq    %rdx, %r14+        movq    0x48(%rsp), %rax+        mulq     0x50(%rsp)+        movq    %rax, %r15+        movq    %rdx, %rcx+        movq    0x40(%rsp), %rbx+        movq    0x30(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x38(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rbp, %rbp+        movq    0x38(%rsp), %rbx+        movq    0x48(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x50(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x58(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %rcx+        movq    0x50(%rsp), %rbx+        movq    0x30(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x40(%rsp), %rbx+        movq    0x48(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x50(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x58(%rsp), %rax+        mulq    %rbx+        subq    %rbp, %rdx+        addq    %rax, %r15+        adcq    %rdx, %rcx+        sbbq    %rbp, %rbp+        xorl    %ebx, %ebx+        movq    0x48(%rsp), %rax+        mulq     0x58(%rsp)+        subq    %rbp, %rdx+        xorl    %ebp, %ebp+        addq    %rax, %rcx+        adcq    %rdx, %rbx+        adcl    %ebp, %ebp+        movq    0x50(%rsp), %rax+        mulq     0x58(%rsp)+        addq    %rax, %rbx+        adcq    %rdx, %rbp+        xorl    %r8d, %r8d+        addq    %r9, %r9+        adcq    %r10, %r10+        adcq    %r11, %r11+        adcq    %r12, %r12+        adcq    %r13, %r13+        adcq    %r14, %r14+        adcq    %r15, %r15+        adcq    %rcx, %rcx+        adcq    %rbx, %rbx+        adcq    %rbp, %rbp+        adcl    %r8d, %r8d+        movq    0x30(%rsp), %rax+        mulq    %rax+        movq    %r8, 0xc0(%rsp)+        movq    %rax, %r8+        movq    0x38(%rsp), %rax+        movq    %rbp, 0xc8(%rsp)+        addq    %rdx, %r9+        sbbq    %rbp, %rbp+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x40(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x48(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x50(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    %rax, %rcx+        adcq    %rdx, %rbx+        sbbq    %rbp, %rbp+        movq    0x58(%rsp), %rax+        mulq    %rax+        negq    %rbp+        adcq    0xc8(%rsp), %rax+        adcq    0xc0(%rsp), %rdx+        movq    %rax, %rbp+        movq    %rdx, %rdi+        movq    %rbx, 0xc0(%rsp)+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r8+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r8, %r9+        sbbq    %rdx, %r10+        sbbq    %rax, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        movq    %rbx, %r8+        sbbq    $0x0, %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r9+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r9, %r10+        sbbq    %rdx, %r11+        sbbq    %rax, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r8+        movq    %rbx, %r9+        sbbq    $0x0, %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r10+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r10, %r11+        sbbq    %rdx, %r12+        sbbq    %rax, %r13+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        movq    %rbx, %r10+        sbbq    $0x0, %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r11+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r11, %r12+        sbbq    %rdx, %r13+        sbbq    %rax, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        movq    %rbx, %r11+        sbbq    $0x0, %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r12+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r12, %r13+        sbbq    %rdx, %r8+        sbbq    %rax, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %r11+        movq    %rbx, %r12+        sbbq    $0x0, %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %rax, %r13+        movl    $0x0, %eax+        adcq    %rbx, %rdx+        adcl    %eax, %eax+        subq    %r13, %r8+        sbbq    %rdx, %r9+        sbbq    %rax, %r10+        sbbq    $0x0, %r11+        sbbq    $0x0, %r12+        movq    %rbx, %r13+        sbbq    $0x0, %r13+        movq    0xc0(%rsp), %rbx+        addq    %r8, %r14+        adcq    %r9, %r15+        adcq    %r10, %rcx+        adcq    %r11, %rbx+        adcq    %r12, %rbp+        adcq    %r13, %rdi+        movl    $0x0, %r8d+        adcq    %r8, %r8+        xorq    %r11, %r11+        xorq    %r12, %r12+        xorq    %r13, %r13+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %r9d+        adcq    %r15, %r9+        movl    $0x1, %r10d+        adcq    %rcx, %r10+        adcq    %rbx, %r11+        adcq    %rbp, %r12+        adcq    %rdi, %r13+        adcq    $0x0, %r8+        cmovneq %rax, %r14+        cmovneq %r9, %r15+        cmovneq %r10, %rcx+        cmovneq %r11, %rbx+        cmovneq %r12, %rbp+        cmovneq %r13, %rdi+        movq    %r14, 0xc0(%rsp)+        movq    %r15, 0xc8(%rsp)+        movq    %rcx, 0xd0(%rsp)+        movq    %rbx, 0xd8(%rsp)+        movq    %rbp, 0xe0(%rsp)+        movq    %rdi, 0xe8(%rsp)+        movq    0x150(%rsp), %rdi+        movq    0xf0(%rsp), %rax+        subq    0x30(%rsp), %rax+        movq    0xf8(%rsp), %rdx+        sbbq    0x38(%rsp), %rdx+        movq    0x100(%rsp), %r8+        sbbq    0x40(%rsp), %r8+        movq    0x108(%rsp), %r9+        sbbq    0x48(%rsp), %r9+        movq    0x110(%rsp), %r10+        sbbq    0x50(%rsp), %r10+        movq    0x118(%rsp), %r11+        sbbq    0x58(%rsp), %r11+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %ebx+        andq    %rbx, %rcx+        xorq    %rbx, %rbx+        subq    %rcx, %rbx+        subq    %rbx, %rax+        movq    %rax, 0x60(%rdi)+        sbbq    %rcx, %rdx+        movq    %rdx, 0x68(%rdi)+        sbbq    %rax, %rax+        andq    %rbx, %rcx+        negq    %rax+        sbbq    %rcx, %r8+        movq    %r8, 0x70(%rdi)+        sbbq    $0x0, %r9+        movq    %r9, 0x78(%rdi)+        sbbq    $0x0, %r10+        movq    %r10, 0x80(%rdi)+        sbbq    $0x0, %r11+        movq    %r11, 0x88(%rdi)+        movq    0x60(%rsp), %rbx+        movq    0x120(%rsp), %rax+        mulq    %rbx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x128(%rsp), %rax+        mulq    %rbx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x130(%rsp), %rax+        mulq    %rbx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x138(%rsp), %rax+        mulq    %rbx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x140(%rsp), %rax+        mulq    %rbx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x148(%rsp), %rax+        mulq    %rbx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        xorl    %r15d, %r15d+        movq    %r8, %rbx+        shlq    $0x20, %rbx+        addq    %r8, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r8+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r8, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r9+        sbbq    %rdx, %r10+        sbbq    %rbp, %r11+        sbbq    $0x0, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %rbx+        addq    %rbx, %r14+        adcq    $0x0, %r15+        movq    0x68(%rsp), %rbx+        movq    0x120(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r8, %r8+        movq    0x128(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r8, %r8+        movq    0x130(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r8, %r8+        movq    0x138(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r8, %r8+        movq    0x140(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r8, %r8+        movq    0x148(%rsp), %rax+        mulq    %rbx+        subq    %r8, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r8, %r8+        negq    %r8+        movq    %r9, %rbx+        shlq    $0x20, %rbx+        addq    %r9, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r9+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r9, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r10+        sbbq    %rdx, %r11+        sbbq    %rbp, %r12+        sbbq    $0x0, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %rbx+        addq    %rbx, %r15+        adcq    $0x0, %r8+        movq    0x70(%rsp), %rbx+        movq    0x120(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r9, %r9+        movq    0x128(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r9, %r9+        movq    0x130(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r9, %r9+        movq    0x138(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r9, %r9+        movq    0x140(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r9, %r9+        movq    0x148(%rsp), %rax+        mulq    %rbx+        subq    %r9, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r9, %r9+        negq    %r9+        movq    %r10, %rbx+        shlq    $0x20, %rbx+        addq    %r10, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r10+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r10, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r11+        sbbq    %rdx, %r12+        sbbq    %rbp, %r13+        sbbq    $0x0, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %rbx+        addq    %rbx, %r8+        adcq    $0x0, %r9+        movq    0x78(%rsp), %rbx+        movq    0x120(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %r10, %r10+        movq    0x128(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r10, %r10+        movq    0x130(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r10, %r10+        movq    0x138(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r10, %r10+        movq    0x140(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r10, %r10+        movq    0x148(%rsp), %rax+        mulq    %rbx+        subq    %r10, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r10, %r10+        negq    %r10+        movq    %r11, %rbx+        shlq    $0x20, %rbx+        addq    %r11, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r11+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r11, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r12+        sbbq    %rdx, %r13+        sbbq    %rbp, %r14+        sbbq    $0x0, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %rbx+        addq    %rbx, %r9+        adcq    $0x0, %r10+        movq    0x80(%rsp), %rbx+        movq    0x120(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %r11, %r11+        movq    0x128(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r11, %r11+        movq    0x130(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r11, %r11+        movq    0x138(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r11, %r11+        movq    0x140(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r11, %r11+        movq    0x148(%rsp), %rax+        mulq    %rbx+        subq    %r11, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r11, %r11+        negq    %r11+        movq    %r12, %rbx+        shlq    $0x20, %rbx+        addq    %r12, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r12+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r12, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r13+        sbbq    %rdx, %r14+        sbbq    %rbp, %r15+        sbbq    $0x0, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %rbx+        addq    %rbx, %r10+        adcq    $0x0, %r11+        movq    0x88(%rsp), %rbx+        movq    0x120(%rsp), %rax+        mulq    %rbx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %r12, %r12+        movq    0x128(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %r12, %r12+        movq    0x130(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        sbbq    %r12, %r12+        movq    0x138(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %r12, %r12+        movq    0x140(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %r12, %r12+        movq    0x148(%rsp), %rax+        mulq    %rbx+        subq    %r12, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %r12, %r12+        negq    %r12+        movq    %r13, %rbx+        shlq    $0x20, %rbx+        addq    %r13, %rbx+        xorl    %ebp, %ebp+        movabsq $0xffffffff00000001, %rax+        mulq    %rbx+        movq    %rdx, %r13+        movabsq $0xffffffff, %rax+        mulq    %rbx+        addq    %r13, %rax+        adcq    %rbx, %rdx+        adcl    %ebp, %ebp+        subq    %rax, %r14+        sbbq    %rdx, %r15+        sbbq    %rbp, %r8+        sbbq    $0x0, %r9+        sbbq    $0x0, %r10+        sbbq    $0x0, %rbx+        addq    %rbx, %r11+        adcq    $0x0, %r12+        xorl    %edx, %edx+        xorl    %ebp, %ebp+        xorl    %r13d, %r13d+        movabsq $0xffffffff00000001, %rax+        addq    %r14, %rax+        movl    $0xffffffff, %ebx+        adcq    %r15, %rbx+        movl    $0x1, %ecx+        adcq    %r8, %rcx+        adcq    %r9, %rdx+        adcq    %r10, %rbp+        adcq    %r11, %r13+        adcq    $0x0, %r12+        cmovneq %rax, %r14+        cmovneq %rbx, %r15+        cmovneq %rcx, %r8+        cmovneq %rdx, %r9+        cmovneq %rbp, %r10+        cmovneq %r13, %r11+        movq    %r14, 0xf0(%rsp)+        movq    %r15, 0xf8(%rsp)+        movq    %r8, 0x100(%rsp)+        movq    %r9, 0x108(%rsp)+        movq    %r10, 0x110(%rsp)+        movq    %r11, 0x118(%rsp)+        movq    0xb8(%rsp), %rcx+        movq    %rcx, %r13+        shrq    $0x3e, %rcx+        movq    0xb0(%rsp), %r12+        shldq   $0x2, %r12, %r13+        movq    0xa8(%rsp), %r11+        shldq   $0x2, %r11, %r12+        movq    0xa0(%rsp), %r10+        shldq   $0x2, %r10, %r11+        movq    0x98(%rsp), %r9+        shldq   $0x2, %r9, %r10+        movq    0x90(%rsp), %r8+        shldq   $0x2, %r8, %r9+        shlq    $0x2, %r8+        addq    $0x1, %rcx+        subq    0x120(%rsp), %r8+        sbbq    0x128(%rsp), %r9+        sbbq    0x130(%rsp), %r10+        sbbq    0x138(%rsp), %r11+        sbbq    0x140(%rsp), %r12+        sbbq    0x148(%rsp), %r13+        sbbq    $0x0, %rcx+        movabsq $0xffffffff00000001, %rax+        mulq    %rcx+        addq    %rax, %r8+        adcq    %rdx, %r9+        adcq    %rcx, %r10+        movq    %rcx, %rax+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %edx+        negq    %rcx+        mulq    %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        adcq    %rcx, %r11+        adcq    $0x0, %r12+        adcq    $0x0, %r13+        sbbq    %rcx, %rcx+        notq    %rcx+        movl    $0xffffffff, %edx+        xorq    %rax, %rax+        andq    %rcx, %rdx+        subq    %rdx, %rax+        andq    $0x1, %rcx+        subq    %rax, %r8+        movq    %r8, (%rdi)+        sbbq    %rdx, %r9+        movq    %r9, 0x8(%rdi)+        sbbq    %rcx, %r10+        movq    %r10, 0x10(%rdi)+        sbbq    $0x0, %r11+        movq    %r11, 0x18(%rdi)+        sbbq    $0x0, %r12+        movq    %r12, 0x20(%rdi)+        sbbq    $0x0, %r13+        movq    %r13, 0x28(%rdi)+        movabsq $0xffffffff, %r8+        subq    0xc0(%rsp), %r8+        movabsq $0xffffffff00000000, %r9+        sbbq    0xc8(%rsp), %r9+        movq    $0xfffffffffffffffe, %r10+        sbbq    0xd0(%rsp), %r10+        movq    $0xffffffffffffffff, %r11+        sbbq    0xd8(%rsp), %r11+        movq    $0xffffffffffffffff, %r12+        sbbq    0xe0(%rsp), %r12+        movq    $0xffffffffffffffff, %r13+        sbbq    0xe8(%rsp), %r13+        movq    %r13, %r14+        shrq    $0x3d, %r14+        shldq   $0x3, %r12, %r13+        shldq   $0x3, %r11, %r12+        shldq   $0x3, %r10, %r11+        shldq   $0x3, %r9, %r10+        shldq   $0x3, %r8, %r9+        shlq    $0x3, %r8+        addq    $0x1, %r14+        movl    $0x3, %ecx+        movq    0xf0(%rsp), %rax+        mulq    %rcx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %rbx, %rbx+        movq    0xf8(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rbx, %rbx+        movq    0x100(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbx, %rbx+        movq    0x108(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rbx, %rbx+        movq    0x110(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbx, %rbx+        movq    0x118(%rsp), %rax+        mulq    %rcx+        subq    %rbx, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        movabsq $0xffffffff00000001, %rax+        mulq    %r14+        addq    %rax, %r8+        adcq    %rdx, %r9+        adcq    %r14, %r10+        movq    %r14, %rax+        sbbq    %rcx, %rcx+        movl    $0xffffffff, %edx+        negq    %rcx+        mulq    %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        adcq    %rcx, %r11+        adcq    $0x0, %r12+        adcq    $0x0, %r13+        sbbq    %rcx, %rcx+        notq    %rcx+        movl    $0xffffffff, %edx+        xorq    %rax, %rax+        andq    %rcx, %rdx+        subq    %rdx, %rax+        andq    $0x1, %rcx+        subq    %rax, %r8+        movq    %r8, 0x30(%rdi)+        sbbq    %rdx, %r9+        movq    %r9, 0x38(%rdi)+        sbbq    %rcx, %r10+        movq    %r10, 0x40(%rdi)+        sbbq    $0x0, %r11+        movq    %r11, 0x48(%rdi)+        sbbq    $0x0, %r12+        movq    %r12, 0x50(%rdi)+        sbbq    $0x0, %r13+        movq    %r13, 0x58(%rdi)+        CFI_INC_RSP(344)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp384_montjscalarmul_alt_p384_montjdouble)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/p521_jscalarmul.S view
@@ -0,0 +1,2505 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Jacobian form scalar multiplication for P-521+// Input scalar[9], point[27]; output res[27]+//+// extern void p521_jscalarmul+//   (uint64_t res[static 27],+//    const uint64_t scalar[static 9],+//    const uint64_t point[static 27]);+//+// This function is a variant of its affine point version p521_scalarmul.+// Here, input and output points are assumed to be in Jacobian form with+// a triple (x,y,z) representing the affine point (x/z^2,y/z^3) when+// z is nonzero or the point at infinity (group identity) if z = 0.+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-521, returns a representation of n * P. If the result is the+// point at infinity (either because the input point was or because the+// scalar was a multiple of p_521) then the output is guaranteed to+// represent the point at infinity, i.e. to have its z coordinate zero.+//+// Standard x86-64 ABI: RDI = res, RSI = scalar, RDX = point+// Microsoft x64 ABI:   RCX = res, RDX = scalar, R8 = point+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p521_jscalarmul)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p521_jscalarmul)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p521_jscalarmul)+++        .text+        .balign 32++// Size of individual field elements++#define NUMSIZE 72+#define JACSIZE (3*NUMSIZE)++// Intermediate variables on the stack.+// The table is 16 entries, each of size JACSIZE = 3 * NUMSIZE+// Uppercase syntactic variants make x86_att version simpler to generate.++#define SCALARB (0*NUMSIZE)+#define scalarb (0*NUMSIZE)(%rsp)+#define ACC (1*NUMSIZE)+#define acc (1*NUMSIZE)(%rsp)+#define TABENT (4*NUMSIZE)+#define tabent (4*NUMSIZE)(%rsp)++#define TAB (7*NUMSIZE)+#define tab (7*NUMSIZE)(%rsp)++#define res (55*NUMSIZE)(%rsp)++#define NSPACE 56*NUMSIZE++// Avoid using .rep for the sake of the BoringSSL/AWS-LC delocator,+// which doesn't accept repetitions, assembler macros etc.++#define selectblock(I,C)                                        \+        cmpq    $I, %rdi ;                                         \+        cmovzq  TAB+JACSIZE*(I-1)+C*NUMSIZE(%rsp), %rax ;         \+        cmovzq  TAB+JACSIZE*(I-1)+8+C*NUMSIZE(%rsp), %rbx ;       \+        cmovzq  TAB+JACSIZE*(I-1)+16+C*NUMSIZE(%rsp), %rcx ;      \+        cmovzq  TAB+JACSIZE*(I-1)+24+C*NUMSIZE(%rsp), %rdx ;      \+        cmovzq  TAB+JACSIZE*(I-1)+32+C*NUMSIZE(%rsp), %r8 ;       \+        cmovzq  TAB+JACSIZE*(I-1)+40+C*NUMSIZE(%rsp), %r9 ;       \+        cmovzq  TAB+JACSIZE*(I-1)+48+C*NUMSIZE(%rsp), %r10 ;      \+        cmovzq  TAB+JACSIZE*(I-1)+56+C*NUMSIZE(%rsp), %r11 ;      \+        cmovzq  TAB+JACSIZE*(I-1)+64+C*NUMSIZE(%rsp), %r12++S2N_BN_SYMBOL(p521_jscalarmul):+        CFI_START+        _CET_ENDBR++// The Windows version literally calls the standard ABI version.+// This simplifies the proofs since subroutine offsets are fixed.++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+        CFI_CALL(Lp521_jscalarmul_standard)+        CFI_POP(%rsi)+        CFI_POP(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p521_jscalarmul)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_standard)++Lp521_jscalarmul_standard:+        CFI_START+#endif++// Real start of the standard ABI code.++        CFI_PUSH(%r15)+        CFI_PUSH(%r14)+        CFI_PUSH(%r13)+        CFI_PUSH(%r12)+        CFI_PUSH(%rbp)+        CFI_PUSH(%rbx)++        CFI_DEC_RSP(NSPACE)++// Preserve the "res" input argument; others get processed early.++        movq    %rdi, res++// Reduce the input scalar mod n_521 and store it to "scalarb".++        movq    %rdx, %rbx+        leaq    SCALARB(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_bignum_mod_n521_9)++// Set the tab[0] table entry to the input point = 1 * P, but also+// reduce all coordinates modulo p. In principle we assume reduction+// as a precondition, but this reduces the scope for surprise, e.g.+// making sure that any input with z = 0 is treated as zero, even+// if the other coordinates are not in fact reduced.++        leaq    TAB(%rsp), %rdi+        movq    %rbx, %rsi+        CFI_CALL(Lp521_jscalarmul_bignum_mod_p521_9)++        leaq    TAB+NUMSIZE(%rsp), %rdi+        leaq    NUMSIZE(%rbx), %rsi+        CFI_CALL(Lp521_jscalarmul_bignum_mod_p521_9)++        leaq    TAB+2*NUMSIZE(%rsp), %rdi+        leaq    2*NUMSIZE(%rbx), %rsi+        CFI_CALL(Lp521_jscalarmul_bignum_mod_p521_9)++// If bit 520 of the scalar is set, then negate the scalar mod n_521,+// i.e. do scalar |-> n_521 - scalar, and also the point to compensate+// by negating its y coordinate. This further step is not needed by+// the indexing scheme (the top window is only a couple of bits either+// way), but is convenient to exclude a problem with the specific value+// scalar = n_521 - 18, where the last Jacobian addition is of the form+// (n_521 - 9) * P + -(9 * P) and hence is a degenerate doubling case.++        xorl    %eax, %eax+        notq    %rax+        movq    $0xbb6fb71e91386409, %r8+        subq    SCALARB(%rsp), %r8+        movq    $0x3bb5c9b8899c47ae, %r9+        sbbq    SCALARB+8(%rsp), %r9+        movq    $0x7fcc0148f709a5d0, %r10+        sbbq    SCALARB+16(%rsp), %r10+        movq    $0x51868783bf2f966b, %r11+        sbbq    SCALARB+24(%rsp), %r11+        leaq    -5(%rax), %r12+        sbbq    SCALARB+32(%rsp), %r12+        movq    %rax, %r13+        sbbq    SCALARB+40(%rsp), %r13+        movq    %rax, %r14+        sbbq    SCALARB+48(%rsp), %r14+        movq    %rax, %r15+        sbbq    SCALARB+56(%rsp), %r15+        movq    $0x1ff, %rax+        movq    SCALARB+64(%rsp), %rcx+        sbbq    %rcx, %rax++        btq     $8, %rcx+        sbbq    %rcx, %rcx++        cmovncq SCALARB(%rsp), %r8+        cmovncq SCALARB+8(%rsp), %r9+        cmovncq SCALARB+16(%rsp), %r10+        cmovncq SCALARB+24(%rsp), %r11+        cmovncq SCALARB+32(%rsp), %r12+        cmovncq SCALARB+40(%rsp), %r13+        cmovncq SCALARB+48(%rsp), %r14+        cmovncq SCALARB+56(%rsp), %r15+        cmovncq SCALARB+64(%rsp), %rax++        movq    %r8, SCALARB(%rsp)+        movq    %r9, SCALARB+8(%rsp)+        movq    %r10, SCALARB+16(%rsp)+        movq    %r11, SCALARB+24(%rsp)+        movq    %r12, SCALARB+32(%rsp)+        movq    %r13, SCALARB+40(%rsp)+        movq    %r14, SCALARB+48(%rsp)+        movq    %r15, SCALARB+56(%rsp)+        movq    %rax, SCALARB+64(%rsp)++        movq    TAB+NUMSIZE(%rsp), %r8+        movq    TAB+NUMSIZE+8(%rsp), %r9+        movq    TAB+NUMSIZE+16(%rsp), %r10+        movq    TAB+NUMSIZE+24(%rsp), %r11+        movq    TAB+NUMSIZE+32(%rsp), %r12+        movq    TAB+NUMSIZE+40(%rsp), %r13+        movq    TAB+NUMSIZE+48(%rsp), %r14+        movq    TAB+NUMSIZE+56(%rsp), %r15+        movq    TAB+NUMSIZE+64(%rsp), %rax++        movq    %r8, %rbx+        movq    %r12, %rbp+        orq     %r9, %rbx+        orq     %r13, %rbp+        orq     %r10, %rbx+        orq     %r14, %rbp+        orq     %r11, %rbx+        orq     %r15, %rbp+        orq     %rbp, %rbx+        orq     %rax, %rbx+        cmovzq  %rbx, %rcx++        xorq    %rcx, %r8+        xorq    %rcx, %r9+        xorq    %rcx, %r10+        xorq    %rcx, %r11+        xorq    %rcx, %r12+        xorq    %rcx, %r13+        xorq    %rcx, %r14+        xorq    %rcx, %r15+        andq    $0x1FF, %rcx+        xorq    %rcx, %rax++        movq    %r8, TAB+NUMSIZE(%rsp)+        movq    %r9, TAB+NUMSIZE+8(%rsp)+        movq    %r10, TAB+NUMSIZE+16(%rsp)+        movq    %r11, TAB+NUMSIZE+24(%rsp)+        movq    %r12, TAB+NUMSIZE+32(%rsp)+        movq    %r13, TAB+NUMSIZE+40(%rsp)+        movq    %r14, TAB+NUMSIZE+48(%rsp)+        movq    %r15, TAB+NUMSIZE+56(%rsp)+        movq    %rax, TAB+NUMSIZE+64(%rsp)++// Compute and record tab[1] = 2 * p, ..., tab[15] = 16 * P++        leaq    TAB+JACSIZE*1(%rsp), %rdi+        leaq    TAB(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_jdouble)++        leaq    TAB+JACSIZE*2(%rsp), %rdi+        leaq    TAB+JACSIZE*1(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp521_jscalarmul_jadd)++        leaq    TAB+JACSIZE*3(%rsp), %rdi+        leaq    TAB+JACSIZE*1(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_jdouble)++        leaq    TAB+JACSIZE*4(%rsp), %rdi+        leaq    TAB+JACSIZE*3(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp521_jscalarmul_jadd)++        leaq    TAB+JACSIZE*5(%rsp), %rdi+        leaq    TAB+JACSIZE*2(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_jdouble)++        leaq    TAB+JACSIZE*6(%rsp), %rdi+        leaq    TAB+JACSIZE*5(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp521_jscalarmul_jadd)++        leaq    TAB+JACSIZE*7(%rsp), %rdi+        leaq    TAB+JACSIZE*3(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_jdouble)++        leaq    TAB+JACSIZE*8(%rsp), %rdi+        leaq    TAB+JACSIZE*7(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp521_jscalarmul_jadd)++        leaq    TAB+JACSIZE*9(%rsp), %rdi+        leaq    TAB+JACSIZE*4(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_jdouble)++        leaq    TAB+JACSIZE*10(%rsp), %rdi+        leaq    TAB+JACSIZE*9(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp521_jscalarmul_jadd)++        leaq    TAB+JACSIZE*11(%rsp), %rdi+        leaq    TAB+JACSIZE*5(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_jdouble)++        leaq    TAB+JACSIZE*12(%rsp), %rdi+        leaq    TAB+JACSIZE*11(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp521_jscalarmul_jadd)++        leaq    TAB+JACSIZE*13(%rsp), %rdi+        leaq    TAB+JACSIZE*6(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_jdouble)++        leaq    TAB+JACSIZE*14(%rsp), %rdi+        leaq    TAB+JACSIZE*13(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp521_jscalarmul_jadd)++        leaq    TAB+JACSIZE*15(%rsp), %rdi+        leaq    TAB+JACSIZE*7(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_jdouble)++// Add the recoding constant sum_i(16 * 32^i) to the scalar to allow signed+// digits. The digits of the constant, in lowest-to-highest order, are as+// follows; they are generated dynamically to use fewer large constant loads.+//+// 0x0842108421084210 %rax+// 0x1084210842108421 %rbx+// 0x2108421084210842 %rbx<<1+// 0x4210842108421084 %rbx<<2+// 0x8421084210842108 %rbx<<3+// 0x0842108421084210 %rax+// 0x1084210842108421 %rbx+// 0x2108421084210842 %rbx<<1+// 0x0000000000000084++        movq    $0x1084210842108421, %rax+        movq    %rax, %rbx+        shrq    $1, %rax+        movq    SCALARB(%rsp), %r8+        addq    %rax, %r8+        movq    SCALARB+8(%rsp), %r9+        adcq    %rbx, %r9+        leaq    (%rbx,%rbx), %rcx+        movq    SCALARB+16(%rsp), %r10+        adcq    %rcx, %r10+        leaq    (%rcx,%rcx), %rcx+        movq    SCALARB+24(%rsp), %r11+        adcq    %rcx, %r11+        leaq    (%rcx,%rcx), %rcx+        movq    SCALARB+32(%rsp), %r12+        adcq    %rcx, %r12+        movq    SCALARB+40(%rsp), %r13+        adcq    %rax, %r13+        movq    SCALARB+48(%rsp), %r14+        adcq    %rbx, %r14+        movq    SCALARB+56(%rsp), %r15+        leaq    (%rbx,%rbx), %rcx+        adcq    %rcx, %r15+        movq    SCALARB+64(%rsp), %rax+        adcq    $0x84, %rax++// Because of the initial reduction the top bitfield (>= bits 520) is <= 1,+// i.e. just a single bit. Record that in %rdi, then shift the whole+// scalar left 56 bits to align the top of the next bitfield with the MSB+// (bits 571..575).++        movq    %rax, %rdi+        shrq    $8, %rdi+        shldq   $56, %r15, %rax+        shldq   $56, %r14, %r15+        shldq   $56, %r13, %r14+        shldq   $56, %r12, %r13+        shldq   $56, %r11, %r12+        shldq   $56, %r10, %r11+        shldq   $56, %r9, %r10+        shldq   $56, %r8, %r9+        shlq    $56, %r8++        movq    %r8, SCALARB(%rsp)+        movq    %r9, SCALARB+8(%rsp)+        movq    %r10, SCALARB+16(%rsp)+        movq    %r11, SCALARB+24(%rsp)+        movq    %r12, SCALARB+32(%rsp)+        movq    %r13, SCALARB+40(%rsp)+        movq    %r14, SCALARB+48(%rsp)+        movq    %r15, SCALARB+56(%rsp)+        movq    %rax, SCALARB+64(%rsp)++// According to the top bit, initialize the accumulator to P or 0. This top+// digit, uniquely, is not recoded so there is no sign adjustment to make.+// We only really need to adjust the z coordinate to zero, but do all three.++        xorl    %ecx, %ecx+        testq   %rdi, %rdi++        movq    TAB(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC(%rsp)+        movq    TAB+8(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+8(%rsp)+        movq    TAB+16(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+16(%rsp)+        movq    TAB+24(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+24(%rsp)+        movq    TAB+32(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+32(%rsp)+        movq    TAB+40(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+40(%rsp)+        movq    TAB+48(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+48(%rsp)+        movq    TAB+56(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+56(%rsp)+        movq    TAB+64(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+64(%rsp)+        movq    TAB+72(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+72(%rsp)+        movq    TAB+80(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+80(%rsp)+        movq    TAB+88(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+88(%rsp)+        movq    TAB+96(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+96(%rsp)+        movq    TAB+104(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+104(%rsp)+        movq    TAB+112(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+112(%rsp)+        movq    TAB+120(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+120(%rsp)+        movq    TAB+128(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+128(%rsp)+        movq    TAB+136(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+136(%rsp)+        movq    TAB+144(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+144(%rsp)+        movq    TAB+152(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+152(%rsp)+        movq    TAB+160(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+160(%rsp)+        movq    TAB+168(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+168(%rsp)+        movq    TAB+176(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+176(%rsp)+        movq    TAB+184(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+184(%rsp)+        movq    TAB+192(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+192(%rsp)+        movq    TAB+200(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+200(%rsp)+        movq    TAB+208(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+208(%rsp)++// Main loop over size-5 bitfields: double 5 times then add signed digit+// At each stage we shift the scalar left by 5 bits so we can simply pick+// the top 5 bits as the bitfield, saving some fiddle over indexing.++        movl    $520, %ebp++Lp521_jscalarmul_mainloop:+        subq    $5, %rbp++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_jdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_jdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_jdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_jdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_jdouble)++// Choose the bitfield and adjust it to sign and magnitude++        movq    SCALARB(%rsp), %r8+        movq    SCALARB+8(%rsp), %r9+        movq    SCALARB+16(%rsp), %r10+        movq    SCALARB+24(%rsp), %r11+        movq    SCALARB+32(%rsp), %r12+        movq    SCALARB+40(%rsp), %r13+        movq    SCALARB+48(%rsp), %r14+        movq    SCALARB+56(%rsp), %r15+        movq    SCALARB+64(%rsp), %rax+++        movq    %rax, %rdi+        shrq    $59, %rdi++        shldq   $5, %r15, %rax+        shldq   $5, %r14, %r15+        shldq   $5, %r13, %r14+        shldq   $5, %r12, %r13+        shldq   $5, %r11, %r12+        shldq   $5, %r10, %r11+        shldq   $5, %r9, %r10+        shldq   $5, %r8, %r9+        shlq    $5, %r8++        movq    %r8, SCALARB(%rsp)+        movq    %r9, SCALARB+8(%rsp)+        movq    %r10, SCALARB+16(%rsp)+        movq    %r11, SCALARB+24(%rsp)+        movq    %r12, SCALARB+32(%rsp)+        movq    %r13, SCALARB+40(%rsp)+        movq    %r14, SCALARB+48(%rsp)+        movq    %r15, SCALARB+56(%rsp)+        movq    %rax, SCALARB+64(%rsp)++        subq    $16, %rdi+        sbbq    %rsi, %rsi // %rsi = sign of digit (-1 = negative)+        xorq    %rsi, %rdi+        subq    %rsi, %rdi // %rdi = absolute value of digit++// Conditionally select the table entry tab[i-1] = i * P in constant time+// Again, this is done in separate sweeps per coordinate, doing y last.++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        selectblock(1,0)+        selectblock(2,0)+        selectblock(3,0)+        selectblock(4,0)+        selectblock(5,0)+        selectblock(6,0)+        selectblock(7,0)+        selectblock(8,0)+        selectblock(9,0)+        selectblock(10,0)+        selectblock(11,0)+        selectblock(12,0)+        selectblock(13,0)+        selectblock(14,0)+        selectblock(15,0)+        selectblock(16,0)+        movq    %rax, TABENT(%rsp)+        movq    %rbx, TABENT+8(%rsp)+        movq    %rcx, TABENT+16(%rsp)+        movq    %rdx, TABENT+24(%rsp)+        movq    %r8, TABENT+32(%rsp)+        movq    %r9, TABENT+40(%rsp)+        movq    %r10, TABENT+48(%rsp)+        movq    %r11, TABENT+56(%rsp)+        movq    %r12, TABENT+64(%rsp)++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        selectblock(1,2)+        selectblock(2,2)+        selectblock(3,2)+        selectblock(4,2)+        selectblock(5,2)+        selectblock(6,2)+        selectblock(7,2)+        selectblock(8,2)+        selectblock(9,2)+        selectblock(10,2)+        selectblock(11,2)+        selectblock(12,2)+        selectblock(13,2)+        selectblock(14,2)+        selectblock(15,2)+        selectblock(16,2)+        movq    %rax, TABENT+2*NUMSIZE(%rsp)+        movq    %rbx, TABENT+2*NUMSIZE+8(%rsp)+        movq    %rcx, TABENT+2*NUMSIZE+16(%rsp)+        movq    %rdx, TABENT+2*NUMSIZE+24(%rsp)+        movq    %r8, TABENT+2*NUMSIZE+32(%rsp)+        movq    %r9, TABENT+2*NUMSIZE+40(%rsp)+        movq    %r10, TABENT+2*NUMSIZE+48(%rsp)+        movq    %r11, TABENT+2*NUMSIZE+56(%rsp)+        movq    %r12, TABENT+2*NUMSIZE+64(%rsp)++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        selectblock(1,1)+        selectblock(2,1)+        selectblock(3,1)+        selectblock(4,1)+        selectblock(5,1)+        selectblock(6,1)+        selectblock(7,1)+        selectblock(8,1)+        selectblock(9,1)+        selectblock(10,1)+        selectblock(11,1)+        selectblock(12,1)+        selectblock(13,1)+        selectblock(14,1)+        selectblock(15,1)+        selectblock(16,1)++// Store it to "tabent" with the y coordinate optionally negated.+// This is done carefully to give coordinates < p_521 even in+// the degenerate case y = 0 (when z = 0 for points on the curve).++        movq    %rax, %r13+        orq     %rbx, %r13+        movq    %rcx, %r14+        orq     %rdx, %r14+        movq    %r8, %r15+        orq     %r9, %r15+        movq    %r10, %rdi+        orq     %r11, %rdi+        orq     %r14, %r13+        orq     %rdi, %r15+        orq     %r12, %r15+        orq     %r15, %r13+        cmovzq  %r13, %rsi++        xorq    %rsi, %rax+        xorq    %rsi, %rbx+        xorq    %rsi, %rcx+        xorq    %rsi, %rdx+        xorq    %rsi, %r8+        xorq    %rsi, %r9+        xorq    %rsi, %r10+        xorq    %rsi, %r11+        andq    $0x1FF, %rsi+        xorq    %rsi, %r12++        movq    %rax, TABENT+NUMSIZE(%rsp)+        movq    %rbx, TABENT+NUMSIZE+8(%rsp)+        movq    %rcx, TABENT+NUMSIZE+16(%rsp)+        movq    %rdx, TABENT+NUMSIZE+24(%rsp)+        movq    %r8, TABENT+NUMSIZE+32(%rsp)+        movq    %r9, TABENT+NUMSIZE+40(%rsp)+        movq    %r10, TABENT+NUMSIZE+48(%rsp)+        movq    %r11, TABENT+NUMSIZE+56(%rsp)+        movq    %r12, TABENT+NUMSIZE+64(%rsp)++// Add to the accumulator++        leaq    TABENT(%rsp), %rdx+        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_jadd)++        testq   %rbp, %rbp+        jne     Lp521_jscalarmul_mainloop++// That's the end of the main loop, and we just need to copy the+// result in "acc" to the output.++        movq    res, %rdi+        movq    ACC(%rsp), %rax+        movq    %rax, (%rdi)+        movq    ACC+8(%rsp), %rax+        movq    %rax, 8(%rdi)+        movq    ACC+16(%rsp), %rax+        movq    %rax, 16(%rdi)+        movq    ACC+24(%rsp), %rax+        movq    %rax, 24(%rdi)+        movq    ACC+32(%rsp), %rax+        movq    %rax, 32(%rdi)+        movq    ACC+40(%rsp), %rax+        movq    %rax, 40(%rdi)+        movq    ACC+48(%rsp), %rax+        movq    %rax, 48(%rdi)+        movq    ACC+56(%rsp), %rax+        movq    %rax, 56(%rdi)+        movq    ACC+64(%rsp), %rax+        movq    %rax, 64(%rdi)+        movq    ACC+72(%rsp), %rax+        movq    %rax, 72(%rdi)+        movq    ACC+80(%rsp), %rax+        movq    %rax, 80(%rdi)+        movq    ACC+88(%rsp), %rax+        movq    %rax, 88(%rdi)+        movq    ACC+96(%rsp), %rax+        movq    %rax, 96(%rdi)+        movq    ACC+104(%rsp), %rax+        movq    %rax, 104(%rdi)+        movq    ACC+112(%rsp), %rax+        movq    %rax, 112(%rdi)+        movq    ACC+120(%rsp), %rax+        movq    %rax, 120(%rdi)+        movq    ACC+128(%rsp), %rax+        movq    %rax, 128(%rdi)+        movq    ACC+136(%rsp), %rax+        movq    %rax, 136(%rdi)+        movq    ACC+144(%rsp), %rax+        movq    %rax, 144(%rdi)+        movq    ACC+152(%rsp), %rax+        movq    %rax, 152(%rdi)+        movq    ACC+160(%rsp), %rax+        movq    %rax, 160(%rdi)+        movq    ACC+168(%rsp), %rax+        movq    %rax, 168(%rdi)+        movq    ACC+176(%rsp), %rax+        movq    %rax, 176(%rdi)+        movq    ACC+184(%rsp), %rax+        movq    %rax, 184(%rdi)+        movq    ACC+192(%rsp), %rax+        movq    %rax, 192(%rdi)+        movq    ACC+200(%rsp), %rax+        movq    %rax, 200(%rdi)+        movq    ACC+208(%rsp), %rax+        movq    %rax, 208(%rdi)++// Restore stack and registers and return++        CFI_INC_RSP(NSPACE)+        CFI_POP(%rbx)+        CFI_POP(%rbp)+        CFI_POP(%r12)+        CFI_POP(%r13)+        CFI_POP(%r14)+        CFI_POP(%r15)+        CFI_RET++#if WINDOWS_ABI+S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_standard)+#else+S2N_BN_SIZE_DIRECTIVE(p521_jscalarmul)+#endif++// Local copies of subroutines, complete clones at the moment++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_bignum_mod_p521_9)++Lp521_jscalarmul_bignum_mod_p521_9:+        CFI_START+        CFI_PUSH(%rbx)+        movq    0x40(%rsi), %rax+        movl    $0x1ff, %edx+        andq    %rax, %rdx+        shrq    $0x9, %rax+        stc+        adcq    (%rsi), %rax+        movq    0x8(%rsi), %rcx+        adcq    $0x0, %rcx+        movq    0x10(%rsi), %r8+        adcq    $0x0, %r8+        movq    0x18(%rsi), %r9+        adcq    $0x0, %r9+        movq    0x20(%rsi), %r10+        adcq    $0x0, %r10+        movq    0x28(%rsi), %r11+        adcq    $0x0, %r11+        movq    0x30(%rsi), %rbx+        adcq    $0x0, %rbx+        movq    0x38(%rsi), %rsi+        adcq    $0x0, %rsi+        adcq    $0x0, %rdx+        cmpq    $0x200, %rdx+        sbbq    $0x0, %rax+        movq    %rax, (%rdi)+        sbbq    $0x0, %rcx+        movq    %rcx, 0x8(%rdi)+        sbbq    $0x0, %r8+        movq    %r8, 0x10(%rdi)+        sbbq    $0x0, %r9+        movq    %r9, 0x18(%rdi)+        sbbq    $0x0, %r10+        movq    %r10, 0x20(%rdi)+        sbbq    $0x0, %r11+        movq    %r11, 0x28(%rdi)+        sbbq    $0x0, %rbx+        movq    %rbx, 0x30(%rdi)+        sbbq    $0x0, %rsi+        movq    %rsi, 0x38(%rdi)+        sbbq    $0x0, %rdx+        andq    $0x1ff, %rdx+        movq    %rdx, 0x40(%rdi)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_bignum_mod_p521_9)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_bignum_mod_n521_9)++Lp521_jscalarmul_bignum_mod_n521_9:+        CFI_START+        movq    0x40(%rsi), %rdx+        movq    $0xfffffffffffffe00, %rax+        orq     %rdx, %rax+        movq    %rax, 0x40(%rdi)+        shrq    $0x9, %rdx+        addq    $0x1, %rdx+        movq    $0x449048e16ec79bf7, %r9+        mulxq   %r9, %rax, %rcx+        adcxq   (%rsi), %rax+        movq    %rax, (%rdi)+        movq    $0xc44a36477663b851, %r10+        mulxq   %r10, %rax, %r8+        adcxq   0x8(%rsi), %rax+        adoxq   %rcx, %rax+        movq    %rax, 0x8(%rdi)+        movq    $0x8033feb708f65a2f, %r11+        mulxq   %r11, %rax, %rcx+        adcxq   0x10(%rsi), %rax+        adoxq   %r8, %rax+        movq    %rax, 0x10(%rdi)+        movq    $0xae79787c40d06994, %rax+        mulxq   %rax, %rax, %r8+        adcxq   0x18(%rsi), %rax+        adoxq   %rcx, %rax+        movq    %rax, 0x18(%rdi)+        movl    $0x5, %eax+        mulxq   %rax, %rax, %rcx+        adcxq   0x20(%rsi), %rax+        adoxq   %r8, %rax+        movq    %rax, 0x20(%rdi)+        movq    %rcx, %rax+        adoxq   %rcx, %rcx+        adcq    0x28(%rsi), %rcx+        movq    %rcx, 0x28(%rdi)+        movq    0x30(%rsi), %rcx+        adcq    %rax, %rcx+        movq    %rcx, 0x30(%rdi)+        movq    0x38(%rsi), %rcx+        adcq    %rax, %rcx+        movq    %rcx, 0x38(%rdi)+        movq    0x40(%rdi), %rcx+        adcq    %rax, %rcx+        cmc+        sbbq    %rdx, %rdx+        andq    %rdx, %r9+        andq    %rdx, %r10+        andq    %rdx, %r11+        movq    $0xae79787c40d06994, %r8+        andq    %rdx, %r8+        andl    $0x5, %edx+        subq    %r9, (%rdi)+        sbbq    %r10, 0x8(%rdi)+        sbbq    %r11, 0x10(%rdi)+        sbbq    %r8, 0x18(%rdi)+        sbbq    %rdx, 0x20(%rdi)+        sbbq    %rax, 0x28(%rdi)+        sbbq    %rax, 0x30(%rdi)+        sbbq    %rax, 0x38(%rdi)+        sbbl    %eax, %ecx+        andl    $0x1ff, %ecx+        movq    %rcx, 0x40(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_bignum_mod_n521_9)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_jadd)++Lp521_jscalarmul_jadd:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(528)+        movq    %rdi, 0x1f8(%rsp)+        movq    %rsi, 0x200(%rsp)+        movq    %rdx, 0x208(%rsp)+        movq    0x200(%rsp), %rsi+        leaq    0x90(%rsi), %rsi+        leaq    (%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_sqr_p521)+        movq    0x208(%rsp), %rdi+        leaq    0x90(%rdi), %rsi+        leaq    0x168(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_sqr_p521)+        movq    0x200(%rsp), %rsi+        movq    0x208(%rsp), %rdi+        leaq    0x48(%rsi), %rdx+        leaq    0x90(%rdi), %rsi+        leaq    0x1b0(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        movq    0x200(%rsp), %rsi+        movq    0x208(%rsp), %rdi+        leaq    0x48(%rdi), %rdx+        leaq    0x90(%rsi), %rsi+        leaq    0x48(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        movq    0x208(%rsp), %rdi+        leaq    (%rdi), %rdx+        leaq    (%rsp), %rsi+        leaq    0x90(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        movq    0x200(%rsp), %rsi+        leaq    (%rsi), %rdx+        leaq    0x168(%rsp), %rsi+        leaq    0x120(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        leaq    0x48(%rsp), %rdx+        leaq    (%rsp), %rsi+        leaq    0x48(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        leaq    0x1b0(%rsp), %rdx+        leaq    0x168(%rsp), %rsi+        leaq    0x1b0(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        movq    0x90(%rsp), %rax+        subq    0x120(%rsp), %rax+        movq    0x98(%rsp), %rdx+        sbbq    0x128(%rsp), %rdx+        movq    0xa0(%rsp), %r8+        sbbq    0x130(%rsp), %r8+        movq    0xa8(%rsp), %r9+        sbbq    0x138(%rsp), %r9+        movq    0xb0(%rsp), %r10+        sbbq    0x140(%rsp), %r10+        movq    0xb8(%rsp), %r11+        sbbq    0x148(%rsp), %r11+        movq    0xc0(%rsp), %r12+        sbbq    0x150(%rsp), %r12+        movq    0xc8(%rsp), %r13+        sbbq    0x158(%rsp), %r13+        movq    0xd0(%rsp), %r14+        sbbq    0x160(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0x168(%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x170(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x178(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x180(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x188(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x190(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x198(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x1a0(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x1a8(%rsp)+        movq    0x48(%rsp), %rax+        subq    0x1b0(%rsp), %rax+        movq    0x50(%rsp), %rdx+        sbbq    0x1b8(%rsp), %rdx+        movq    0x58(%rsp), %r8+        sbbq    0x1c0(%rsp), %r8+        movq    0x60(%rsp), %r9+        sbbq    0x1c8(%rsp), %r9+        movq    0x68(%rsp), %r10+        sbbq    0x1d0(%rsp), %r10+        movq    0x70(%rsp), %r11+        sbbq    0x1d8(%rsp), %r11+        movq    0x78(%rsp), %r12+        sbbq    0x1e0(%rsp), %r12+        movq    0x80(%rsp), %r13+        sbbq    0x1e8(%rsp), %r13+        movq    0x88(%rsp), %r14+        sbbq    0x1f0(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0x48(%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x50(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x58(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x60(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x68(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x70(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x78(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x80(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x88(%rsp)+        leaq    0x168(%rsp), %rsi+        leaq    0xd8(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_sqr_p521)+        leaq    0x48(%rsp), %rsi+        leaq    (%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_sqr_p521)+        leaq    0x120(%rsp), %rdx+        leaq    0xd8(%rsp), %rsi+        leaq    0x120(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        leaq    0x90(%rsp), %rdx+        leaq    0xd8(%rsp), %rsi+        leaq    0x90(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        movq    (%rsp), %rax+        subq    0x120(%rsp), %rax+        movq    0x8(%rsp), %rdx+        sbbq    0x128(%rsp), %rdx+        movq    0x10(%rsp), %r8+        sbbq    0x130(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x138(%rsp), %r9+        movq    0x20(%rsp), %r10+        sbbq    0x140(%rsp), %r10+        movq    0x28(%rsp), %r11+        sbbq    0x148(%rsp), %r11+        movq    0x30(%rsp), %r12+        sbbq    0x150(%rsp), %r12+        movq    0x38(%rsp), %r13+        sbbq    0x158(%rsp), %r13+        movq    0x40(%rsp), %r14+        sbbq    0x160(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, (%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x8(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x18(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x20(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x28(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x30(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x38(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x40(%rsp)+        movq    0x90(%rsp), %rax+        subq    0x120(%rsp), %rax+        movq    0x98(%rsp), %rdx+        sbbq    0x128(%rsp), %rdx+        movq    0xa0(%rsp), %r8+        sbbq    0x130(%rsp), %r8+        movq    0xa8(%rsp), %r9+        sbbq    0x138(%rsp), %r9+        movq    0xb0(%rsp), %r10+        sbbq    0x140(%rsp), %r10+        movq    0xb8(%rsp), %r11+        sbbq    0x148(%rsp), %r11+        movq    0xc0(%rsp), %r12+        sbbq    0x150(%rsp), %r12+        movq    0xc8(%rsp), %r13+        sbbq    0x158(%rsp), %r13+        movq    0xd0(%rsp), %r14+        sbbq    0x160(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0xd8(%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0xe0(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0xe8(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0xf0(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0xf8(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x100(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x108(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x110(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x118(%rsp)+        movq    0x200(%rsp), %rsi+        leaq    0x90(%rsi), %rdx+        leaq    0x168(%rsp), %rsi+        leaq    0x168(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        movq    (%rsp), %rax+        subq    0x90(%rsp), %rax+        movq    0x8(%rsp), %rdx+        sbbq    0x98(%rsp), %rdx+        movq    0x10(%rsp), %r8+        sbbq    0xa0(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0xa8(%rsp), %r9+        movq    0x20(%rsp), %r10+        sbbq    0xb0(%rsp), %r10+        movq    0x28(%rsp), %r11+        sbbq    0xb8(%rsp), %r11+        movq    0x30(%rsp), %r12+        sbbq    0xc0(%rsp), %r12+        movq    0x38(%rsp), %r13+        sbbq    0xc8(%rsp), %r13+        movq    0x40(%rsp), %r14+        sbbq    0xd0(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, (%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x8(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x18(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x20(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x28(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x30(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x38(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x40(%rsp)+        movq    0x120(%rsp), %rax+        subq    (%rsp), %rax+        movq    0x128(%rsp), %rdx+        sbbq    0x8(%rsp), %rdx+        movq    0x130(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x138(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movq    0x140(%rsp), %r10+        sbbq    0x20(%rsp), %r10+        movq    0x148(%rsp), %r11+        sbbq    0x28(%rsp), %r11+        movq    0x150(%rsp), %r12+        sbbq    0x30(%rsp), %r12+        movq    0x158(%rsp), %r13+        sbbq    0x38(%rsp), %r13+        movq    0x160(%rsp), %r14+        sbbq    0x40(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0x120(%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x128(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x130(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x138(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x140(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x148(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x150(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x158(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x160(%rsp)+        leaq    0x1b0(%rsp), %rdx+        leaq    0xd8(%rsp), %rsi+        leaq    0xd8(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        movq    0x208(%rsp), %rdi+        leaq    0x90(%rdi), %rdx+        leaq    0x168(%rsp), %rsi+        leaq    0x168(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        leaq    0x120(%rsp), %rdx+        leaq    0x48(%rsp), %rsi+        leaq    0x120(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        movq    0x120(%rsp), %rax+        subq    0xd8(%rsp), %rax+        movq    0x128(%rsp), %rdx+        sbbq    0xe0(%rsp), %rdx+        movq    0x130(%rsp), %r8+        sbbq    0xe8(%rsp), %r8+        movq    0x138(%rsp), %r9+        sbbq    0xf0(%rsp), %r9+        movq    0x140(%rsp), %r10+        sbbq    0xf8(%rsp), %r10+        movq    0x148(%rsp), %r11+        sbbq    0x100(%rsp), %r11+        movq    0x150(%rsp), %r12+        sbbq    0x108(%rsp), %r12+        movq    0x158(%rsp), %r13+        sbbq    0x110(%rsp), %r13+        movq    0x160(%rsp), %r14+        sbbq    0x118(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0x120(%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x128(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x130(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x138(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x140(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x148(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x150(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x158(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x160(%rsp)+        movq    0x200(%rsp), %rsi+        movq    0x90(%rsi), %r8+        movq    0x98(%rsi), %r9+        movq    0xa0(%rsi), %r10+        movq    0xa8(%rsi), %r11+        movq    0xb0(%rsi), %r12+        movq    0xb8(%rsi), %r13+        movq    0xc0(%rsi), %r14+        movq    0xc8(%rsi), %r15+        movq    0xd0(%rsi), %rbp+        orq     %r9, %r8+        orq     %r11, %r10+        orq     %r13, %r12+        orq     %r15, %r14+        orq     %r10, %r8+        orq     %r14, %r12+        orq     %rbp, %r8+        orq     %r12, %r8+        negq    %r8+        sbbq    %rax, %rax+        movq    0x208(%rsp), %rdi+        movq    0x90(%rdi), %r8+        movq    0x98(%rdi), %r9+        movq    0xa0(%rdi), %r10+        movq    0xa8(%rdi), %r11+        movq    0xb0(%rdi), %r12+        movq    0xb8(%rdi), %r13+        movq    0xc0(%rdi), %r14+        movq    0xc8(%rdi), %r15+        movq    0xd0(%rdi), %rbp+        orq     %r9, %r8+        orq     %r11, %r10+        orq     %r13, %r12+        orq     %r15, %r14+        orq     %r10, %r8+        orq     %r14, %r12+        orq     %rbp, %r8+        orq     %r12, %r8+        negq    %r8+        sbbq    %rdx, %rdx+        cmpq    %rax, %rdx+        movq    0x120(%rsp), %r8+        cmovbq  0x48(%rsi), %r8+        cmova   0x48(%rdi), %r8+        movq    0x128(%rsp), %r9+        cmovbq  0x50(%rsi), %r9+        cmova   0x50(%rdi), %r9+        movq    0x130(%rsp), %r10+        cmovbq  0x58(%rsi), %r10+        cmova   0x58(%rdi), %r10+        movq    0x138(%rsp), %r11+        cmovbq  0x60(%rsi), %r11+        cmova   0x60(%rdi), %r11+        movq    0x140(%rsp), %r12+        cmovbq  0x68(%rsi), %r12+        cmova   0x68(%rdi), %r12+        movq    0x148(%rsp), %r13+        cmovbq  0x70(%rsi), %r13+        cmova   0x70(%rdi), %r13+        movq    0x150(%rsp), %r14+        cmovbq  0x78(%rsi), %r14+        cmova   0x78(%rdi), %r14+        movq    0x158(%rsp), %r15+        cmovbq  0x80(%rsi), %r15+        cmova   0x80(%rdi), %r15+        movq    0x160(%rsp), %rbp+        cmovbq  0x88(%rsi), %rbp+        cmova   0x88(%rdi), %rbp+        movq    %r8, 0x120(%rsp)+        movq    %r9, 0x128(%rsp)+        movq    %r10, 0x130(%rsp)+        movq    %r11, 0x138(%rsp)+        movq    %r12, 0x140(%rsp)+        movq    %r13, 0x148(%rsp)+        movq    %r14, 0x150(%rsp)+        movq    %r15, 0x158(%rsp)+        movq    %rbp, 0x160(%rsp)+        movq    0x168(%rsp), %r8+        cmovbq  0x90(%rsi), %r8+        cmova   0x90(%rdi), %r8+        movq    0x170(%rsp), %r9+        cmovbq  0x98(%rsi), %r9+        cmova   0x98(%rdi), %r9+        movq    0x178(%rsp), %r10+        cmovbq  0xa0(%rsi), %r10+        cmova   0xa0(%rdi), %r10+        movq    0x180(%rsp), %r11+        cmovbq  0xa8(%rsi), %r11+        cmova   0xa8(%rdi), %r11+        movq    0x188(%rsp), %r12+        cmovbq  0xb0(%rsi), %r12+        cmova   0xb0(%rdi), %r12+        movq    0x190(%rsp), %r13+        cmovbq  0xb8(%rsi), %r13+        cmova   0xb8(%rdi), %r13+        movq    0x198(%rsp), %r14+        cmovbq  0xc0(%rsi), %r14+        cmova   0xc0(%rdi), %r14+        movq    0x1a0(%rsp), %r15+        cmovbq  0xc8(%rsi), %r15+        cmova   0xc8(%rdi), %r15+        movq    0x1a8(%rsp), %rbp+        cmovbq  0xd0(%rsi), %rbp+        cmova   0xd0(%rdi), %rbp+        movq    %r8, 0x168(%rsp)+        movq    %r9, 0x170(%rsp)+        movq    %r10, 0x178(%rsp)+        movq    %r11, 0x180(%rsp)+        movq    %r12, 0x188(%rsp)+        movq    %r13, 0x190(%rsp)+        movq    %r14, 0x198(%rsp)+        movq    %r15, 0x1a0(%rsp)+        movq    %rbp, 0x1a8(%rsp)+        movq    (%rsp), %r8+        cmovbq  (%rsi), %r8+        cmova   (%rdi), %r8+        movq    0x8(%rsp), %r9+        cmovbq  0x8(%rsi), %r9+        cmova   0x8(%rdi), %r9+        movq    0x10(%rsp), %r10+        cmovbq  0x10(%rsi), %r10+        cmova   0x10(%rdi), %r10+        movq    0x18(%rsp), %r11+        cmovbq  0x18(%rsi), %r11+        cmova   0x18(%rdi), %r11+        movq    0x20(%rsp), %r12+        cmovbq  0x20(%rsi), %r12+        cmova   0x20(%rdi), %r12+        movq    0x28(%rsp), %r13+        cmovbq  0x28(%rsi), %r13+        cmova   0x28(%rdi), %r13+        movq    0x30(%rsp), %r14+        cmovbq  0x30(%rsi), %r14+        cmova   0x30(%rdi), %r14+        movq    0x38(%rsp), %r15+        cmovbq  0x38(%rsi), %r15+        cmova   0x38(%rdi), %r15+        movq    0x40(%rsp), %rbp+        cmovbq  0x40(%rsi), %rbp+        cmova   0x40(%rdi), %rbp+        movq    0x1f8(%rsp), %rdi+        movq    %r8, (%rdi)+        movq    %r9, 0x8(%rdi)+        movq    %r10, 0x10(%rdi)+        movq    %r11, 0x18(%rdi)+        movq    %r12, 0x20(%rdi)+        movq    %r13, 0x28(%rdi)+        movq    %r14, 0x30(%rdi)+        movq    %r15, 0x38(%rdi)+        movq    %rbp, 0x40(%rdi)+        movq    0x120(%rsp), %rax+        movq    %rax, 0x48(%rdi)+        movq    0x128(%rsp), %rax+        movq    %rax, 0x50(%rdi)+        movq    0x130(%rsp), %rax+        movq    %rax, 0x58(%rdi)+        movq    0x138(%rsp), %rax+        movq    %rax, 0x60(%rdi)+        movq    0x140(%rsp), %rax+        movq    %rax, 0x68(%rdi)+        movq    0x148(%rsp), %rax+        movq    %rax, 0x70(%rdi)+        movq    0x150(%rsp), %rax+        movq    %rax, 0x78(%rdi)+        movq    0x158(%rsp), %rax+        movq    %rax, 0x80(%rdi)+        movq    0x160(%rsp), %rax+        movq    %rax, 0x88(%rdi)+        movq    0x168(%rsp), %rax+        movq    %rax, 0x90(%rdi)+        movq    0x170(%rsp), %rax+        movq    %rax, 0x98(%rdi)+        movq    0x178(%rsp), %rax+        movq    %rax, 0xa0(%rdi)+        movq    0x180(%rsp), %rax+        movq    %rax, 0xa8(%rdi)+        movq    0x188(%rsp), %rax+        movq    %rax, 0xb0(%rdi)+        movq    0x190(%rsp), %rax+        movq    %rax, 0xb8(%rdi)+        movq    0x198(%rsp), %rax+        movq    %rax, 0xc0(%rdi)+        movq    0x1a0(%rsp), %rax+        movq    %rax, 0xc8(%rdi)+        movq    0x1a8(%rsp), %rax+        movq    %rax, 0xd0(%rdi)+        CFI_INC_RSP(528)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_jadd)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_jdouble)++Lp521_jscalarmul_jdouble:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(520)+        movq    %rdi, 0x1f8(%rsp)+        movq    %rsi, 0x200(%rsp)+        movq    0x200(%rsp), %rdi+        leaq    0x90(%rdi), %rsi+        leaq    (%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_sqr_p521)+        movq    0x200(%rsp), %rdi+        leaq    0x48(%rdi), %rsi+        leaq    0x48(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_sqr_p521)+        movq    0x200(%rsp), %rdi+        stc+        movq    (%rdi), %rax+        adcq    (%rsp), %rax+        movq    0x8(%rdi), %rbx+        adcq    0x8(%rsp), %rbx+        movq    0x10(%rdi), %r8+        adcq    0x10(%rsp), %r8+        movq    0x18(%rdi), %r9+        adcq    0x18(%rsp), %r9+        movq    0x20(%rdi), %r10+        adcq    0x20(%rsp), %r10+        movq    0x28(%rdi), %r11+        adcq    0x28(%rsp), %r11+        movq    0x30(%rdi), %r12+        adcq    0x30(%rsp), %r12+        movq    0x38(%rdi), %r13+        adcq    0x38(%rsp), %r13+        movq    0x40(%rdi), %r14+        adcq    0x40(%rsp), %r14+        movq    $0x200, %rdx+        andq    %r14, %rdx+        cmpq    $0x200, %rdx+        sbbq    $0x0, %rax+        movq    %rax, 0x168(%rsp)+        sbbq    $0x0, %rbx+        movq    %rbx, 0x170(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x178(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x180(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x188(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x190(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x198(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x1a0(%rsp)+        sbbq    %rdx, %r14+        movq    %r14, 0x1a8(%rsp)+        movq    0x200(%rsp), %rdi+        movq    (%rdi), %rax+        subq    (%rsp), %rax+        movq    0x8(%rdi), %rdx+        sbbq    0x8(%rsp), %rdx+        movq    0x10(%rdi), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x18(%rdi), %r9+        sbbq    0x18(%rsp), %r9+        movq    0x20(%rdi), %r10+        sbbq    0x20(%rsp), %r10+        movq    0x28(%rdi), %r11+        sbbq    0x28(%rsp), %r11+        movq    0x30(%rdi), %r12+        sbbq    0x30(%rsp), %r12+        movq    0x38(%rdi), %r13+        sbbq    0x38(%rsp), %r13+        movq    0x40(%rdi), %r14+        sbbq    0x40(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0x120(%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x128(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x130(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x138(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x140(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x148(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x150(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x158(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x160(%rsp)+        leaq    0x120(%rsp), %rdx+        leaq    0x168(%rsp), %rsi+        leaq    0x90(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        movq    0x200(%rsp), %rdi+        stc+        movq    0x48(%rdi), %rax+        adcq    0x90(%rdi), %rax+        movq    0x50(%rdi), %rbx+        adcq    0x98(%rdi), %rbx+        movq    0x58(%rdi), %r8+        adcq    0xa0(%rdi), %r8+        movq    0x60(%rdi), %r9+        adcq    0xa8(%rdi), %r9+        movq    0x68(%rdi), %r10+        adcq    0xb0(%rdi), %r10+        movq    0x70(%rdi), %r11+        adcq    0xb8(%rdi), %r11+        movq    0x78(%rdi), %r12+        adcq    0xc0(%rdi), %r12+        movq    0x80(%rdi), %r13+        adcq    0xc8(%rdi), %r13+        movq    0x88(%rdi), %r14+        adcq    0xd0(%rdi), %r14+        movq    $0x200, %rdx+        andq    %r14, %rdx+        cmpq    $0x200, %rdx+        sbbq    $0x0, %rax+        movq    %rax, 0x168(%rsp)+        sbbq    $0x0, %rbx+        movq    %rbx, 0x170(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x178(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x180(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x188(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x190(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x198(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x1a0(%rsp)+        sbbq    %rdx, %r14+        movq    %r14, 0x1a8(%rsp)+        leaq    0x90(%rsp), %rsi+        leaq    0x1b0(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_sqr_p521)+        movq    0x200(%rsp), %rdi+        leaq    0x48(%rsp), %rdx+        leaq    (%rdi), %rsi+        leaq    0xd8(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        leaq    0x168(%rsp), %rsi+        leaq    0x120(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_sqr_p521)+        movq    $0x9, %rdx+        movq    0x1f0(%rsp), %rbx+        xorq    $0x1ff, %rbx+        movq    0x1b0(%rsp), %rax+        notq    %rax+        mulxq   %rax, %r8, %r9+        movq    0x1b8(%rsp), %rax+        notq    %rax+        mulxq   %rax, %rax, %r10+        addq    %rax, %r9+        movq    0x1c0(%rsp), %rax+        notq    %rax+        mulxq   %rax, %rax, %r11+        adcq    %rax, %r10+        movq    0x1c8(%rsp), %rax+        notq    %rax+        mulxq   %rax, %rax, %r12+        adcq    %rax, %r11+        movq    0x1d0(%rsp), %rax+        notq    %rax+        mulxq   %rax, %rax, %r13+        adcq    %rax, %r12+        movq    0x1d8(%rsp), %rax+        notq    %rax+        mulxq   %rax, %rax, %r14+        adcq    %rax, %r13+        movq    0x1e0(%rsp), %rax+        notq    %rax+        mulxq   %rax, %rax, %r15+        adcq    %rax, %r14+        movq    0x1e8(%rsp), %rax+        notq    %rax+        mulxq   %rax, %rax, %rcx+        adcq    %rax, %r15+        mulxq   %rbx, %rbx, %rax+        adcq    %rcx, %rbx+        xorl    %eax, %eax+        movq    $0xc, %rdx+        mulxq   0xd8(%rsp), %rax, %rcx+        adcxq   %rax, %r8+        adoxq   %rcx, %r9+        mulxq   0xe0(%rsp), %rax, %rcx+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+        mulxq   0xe8(%rsp), %rax, %rcx+        adcxq   %rax, %r10+        adoxq   %rcx, %r11+        mulxq   0xf0(%rsp), %rax, %rcx+        adcxq   %rax, %r11+        adoxq   %rcx, %r12+        mulxq   0xf8(%rsp), %rax, %rcx+        adcxq   %rax, %r12+        adoxq   %rcx, %r13+        mulxq   0x100(%rsp), %rax, %rcx+        adcxq   %rax, %r13+        adoxq   %rcx, %r14+        mulxq   0x108(%rsp), %rax, %rcx+        adcxq   %rax, %r14+        adoxq   %rcx, %r15+        mulxq   0x110(%rsp), %rax, %rcx+        adcxq   %rax, %r15+        adoxq   %rcx, %rbx+        mulxq   0x118(%rsp), %rax, %rcx+        adcxq   %rax, %rbx+        movq    %r9, %rax+        andq    %r10, %rax+        andq    %r11, %rax+        andq    %r12, %rax+        andq    %r13, %rax+        andq    %r14, %rax+        andq    %r15, %rax+        movq    %rbx, %rdx+        shrq    $0x9, %rdx+        orq     $0xfffffffffffffe00, %rbx+        leaq    0x1(%rdx), %rcx+        addq    %r8, %rcx+        movl    $0x0, %ecx+        adcq    %rcx, %rax+        movq    %rbx, %rax+        adcq    %rcx, %rax+        adcq    %rdx, %r8+        movq    %r8, 0x1b0(%rsp)+        adcq    %rcx, %r9+        movq    %r9, 0x1b8(%rsp)+        adcq    %rcx, %r10+        movq    %r10, 0x1c0(%rsp)+        adcq    %rcx, %r11+        movq    %r11, 0x1c8(%rsp)+        adcq    %rcx, %r12+        movq    %r12, 0x1d0(%rsp)+        adcq    %rcx, %r13+        movq    %r13, 0x1d8(%rsp)+        adcq    %rcx, %r14+        movq    %r14, 0x1e0(%rsp)+        adcq    %rcx, %r15+        movq    %r15, 0x1e8(%rsp)+        adcq    %rcx, %rbx+        andq    $0x1ff, %rbx+        movq    %rbx, 0x1f0(%rsp)+        movq    0x120(%rsp), %rax+        subq    (%rsp), %rax+        movq    0x128(%rsp), %rdx+        sbbq    0x8(%rsp), %rdx+        movq    0x130(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x138(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movq    0x140(%rsp), %r10+        sbbq    0x20(%rsp), %r10+        movq    0x148(%rsp), %r11+        sbbq    0x28(%rsp), %r11+        movq    0x150(%rsp), %r12+        sbbq    0x30(%rsp), %r12+        movq    0x158(%rsp), %r13+        sbbq    0x38(%rsp), %r13+        movq    0x160(%rsp), %r14+        sbbq    0x40(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0x168(%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x170(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x178(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x180(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x188(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x190(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x198(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x1a0(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x1a8(%rsp)+        leaq    0x48(%rsp), %rsi+        leaq    0x120(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_sqr_p521)+        movq    0x1f8(%rsp), %rdi+        movq    0x168(%rsp), %rax+        subq    0x48(%rsp), %rax+        movq    0x170(%rsp), %rdx+        sbbq    0x50(%rsp), %rdx+        movq    0x178(%rsp), %r8+        sbbq    0x58(%rsp), %r8+        movq    0x180(%rsp), %r9+        sbbq    0x60(%rsp), %r9+        movq    0x188(%rsp), %r10+        sbbq    0x68(%rsp), %r10+        movq    0x190(%rsp), %r11+        sbbq    0x70(%rsp), %r11+        movq    0x198(%rsp), %r12+        sbbq    0x78(%rsp), %r12+        movq    0x1a0(%rsp), %r13+        sbbq    0x80(%rsp), %r13+        movq    0x1a8(%rsp), %r14+        sbbq    0x88(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0x90(%rdi)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x98(%rdi)+        sbbq    $0x0, %r8+        movq    %r8, 0xa0(%rdi)+        sbbq    $0x0, %r9+        movq    %r9, 0xa8(%rdi)+        sbbq    $0x0, %r10+        movq    %r10, 0xb0(%rdi)+        sbbq    $0x0, %r11+        movq    %r11, 0xb8(%rdi)+        sbbq    $0x0, %r12+        movq    %r12, 0xc0(%rdi)+        sbbq    $0x0, %r13+        movq    %r13, 0xc8(%rdi)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0xd0(%rdi)+        leaq    0x90(%rsp), %rdx+        leaq    0x1b0(%rsp), %rsi+        leaq    0x168(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_mul_p521)+        movq    0x1f8(%rsp), %rdi+        movq    0x118(%rsp), %rbx+        movq    0x110(%rsp), %r15+        shldq   $0x2, %r15, %rbx+        movq    0x108(%rsp), %r14+        shldq   $0x2, %r14, %r15+        movq    0x100(%rsp), %r13+        shldq   $0x2, %r13, %r14+        movq    0xf8(%rsp), %r12+        shldq   $0x2, %r12, %r13+        movq    0xf0(%rsp), %r11+        shldq   $0x2, %r11, %r12+        movq    0xe8(%rsp), %r10+        shldq   $0x2, %r10, %r11+        movq    0xe0(%rsp), %r9+        shldq   $0x2, %r9, %r10+        movq    0xd8(%rsp), %r8+        shldq   $0x2, %r8, %r9+        shlq    $0x2, %r8+        movq    0x1f0(%rsp), %rcx+        xorq    $0x1ff, %rcx+        movq    0x1b0(%rsp), %rax+        notq    %rax+        addq    %rax, %r8+        movq    0x1b8(%rsp), %rax+        notq    %rax+        adcq    %rax, %r9+        movq    0x1c0(%rsp), %rax+        notq    %rax+        adcq    %rax, %r10+        movq    0x1c8(%rsp), %rax+        notq    %rax+        adcq    %rax, %r11+        movq    0x1d0(%rsp), %rax+        notq    %rax+        adcq    %rax, %r12+        movq    0x1d8(%rsp), %rax+        notq    %rax+        adcq    %rax, %r13+        movq    0x1e0(%rsp), %rax+        notq    %rax+        adcq    %rax, %r14+        movq    0x1e8(%rsp), %rax+        notq    %rax+        adcq    %rax, %r15+        adcq    %rcx, %rbx+        movq    %r9, %rax+        andq    %r10, %rax+        andq    %r11, %rax+        andq    %r12, %rax+        andq    %r13, %rax+        andq    %r14, %rax+        andq    %r15, %rax+        movq    %rbx, %rdx+        shrq    $0x9, %rdx+        orq     $0xfffffffffffffe00, %rbx+        leaq    0x1(%rdx), %rcx+        addq    %r8, %rcx+        movl    $0x0, %ecx+        adcq    %rcx, %rax+        movq    %rbx, %rax+        adcq    %rcx, %rax+        adcq    %rdx, %r8+        movq    %r8, (%rdi)+        adcq    %rcx, %r9+        movq    %r9, 0x8(%rdi)+        adcq    %rcx, %r10+        movq    %r10, 0x10(%rdi)+        adcq    %rcx, %r11+        movq    %r11, 0x18(%rdi)+        adcq    %rcx, %r12+        movq    %r12, 0x20(%rdi)+        adcq    %rcx, %r13+        movq    %r13, 0x28(%rdi)+        adcq    %rcx, %r14+        movq    %r14, 0x30(%rdi)+        adcq    %rcx, %r15+        movq    %r15, 0x38(%rdi)+        adcq    %rcx, %rbx+        andq    $0x1ff, %rbx+        movq    %rbx, 0x40(%rdi)+        movq    0x1f8(%rsp), %rdi+        movq    0x160(%rsp), %rbx+        xorq    $0x1ff, %rbx+        movq    0x158(%rsp), %r15+        notq    %r15+        shldq   $0x3, %r15, %rbx+        movq    0x150(%rsp), %r14+        notq    %r14+        shldq   $0x3, %r14, %r15+        movq    0x148(%rsp), %r13+        notq    %r13+        shldq   $0x3, %r13, %r14+        movq    0x140(%rsp), %r12+        notq    %r12+        shldq   $0x3, %r12, %r13+        movq    0x138(%rsp), %r11+        notq    %r11+        shldq   $0x3, %r11, %r12+        movq    0x130(%rsp), %r10+        notq    %r10+        shldq   $0x3, %r10, %r11+        movq    0x128(%rsp), %r9+        notq    %r9+        shldq   $0x3, %r9, %r10+        movq    0x120(%rsp), %r8+        notq    %r8+        shldq   $0x3, %r8, %r9+        shlq    $0x3, %r8+        movq    $0x3, %rdx+        xorl    %eax, %eax+        mulxq   0x168(%rsp), %rax, %rcx+        adcxq   %rax, %r8+        adoxq   %rcx, %r9+        mulxq   0x170(%rsp), %rax, %rcx+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+        mulxq   0x178(%rsp), %rax, %rcx+        adcxq   %rax, %r10+        adoxq   %rcx, %r11+        mulxq   0x180(%rsp), %rax, %rcx+        adcxq   %rax, %r11+        adoxq   %rcx, %r12+        mulxq   0x188(%rsp), %rax, %rcx+        adcxq   %rax, %r12+        adoxq   %rcx, %r13+        mulxq   0x190(%rsp), %rax, %rcx+        adcxq   %rax, %r13+        adoxq   %rcx, %r14+        mulxq   0x198(%rsp), %rax, %rcx+        adcxq   %rax, %r14+        adoxq   %rcx, %r15+        mulxq   0x1a0(%rsp), %rax, %rcx+        adcxq   %rax, %r15+        adoxq   %rcx, %rbx+        mulxq   0x1a8(%rsp), %rax, %rcx+        adcxq   %rax, %rbx+        movq    %r9, %rax+        andq    %r10, %rax+        andq    %r11, %rax+        andq    %r12, %rax+        andq    %r13, %rax+        andq    %r14, %rax+        andq    %r15, %rax+        movq    %rbx, %rdx+        shrq    $0x9, %rdx+        orq     $0xfffffffffffffe00, %rbx+        leaq    0x1(%rdx), %rcx+        addq    %r8, %rcx+        movl    $0x0, %ecx+        adcq    %rcx, %rax+        movq    %rbx, %rax+        adcq    %rcx, %rax+        adcq    %rdx, %r8+        movq    %r8, 0x48(%rdi)+        adcq    %rcx, %r9+        movq    %r9, 0x50(%rdi)+        adcq    %rcx, %r10+        movq    %r10, 0x58(%rdi)+        adcq    %rcx, %r11+        movq    %r11, 0x60(%rdi)+        adcq    %rcx, %r12+        movq    %r12, 0x68(%rdi)+        adcq    %rcx, %r13+        movq    %r13, 0x70(%rdi)+        adcq    %rcx, %r14+        movq    %r14, 0x78(%rdi)+        adcq    %rcx, %r15+        movq    %r15, 0x80(%rdi)+        adcq    %rcx, %rbx+        andq    $0x1ff, %rbx+        movq    %rbx, 0x88(%rdi)+        CFI_INC_RSP(520)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_jdouble)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_mul_p521)++Lp521_jscalarmul_mul_p521:+        CFI_START+        CFI_DEC_RSP(64)+        movq    %rdx, %rcx+        xorl    %ebp, %ebp+        movq    (%rcx), %rdx+        mulxq   (%rsi), %r8, %r9+        movq    %r8, (%rsp)+        mulxq   0x8(%rsi), %rbx, %r10+        adcq    %rbx, %r9+        mulxq   0x10(%rsi), %rbx, %r11+        adcq    %rbx, %r10+        mulxq   0x18(%rsi), %rbx, %r12+        adcq    %rbx, %r11+        mulxq   0x20(%rsi), %rbx, %r13+        adcq    %rbx, %r12+        mulxq   0x28(%rsi), %rbx, %r14+        adcq    %rbx, %r13+        mulxq   0x30(%rsi), %rbx, %r15+        adcq    %rbx, %r14+        mulxq   0x38(%rsi), %rbx, %r8+        adcq    %rbx, %r15+        adcq    %rbp, %r8+        movq    0x8(%rcx), %rdx+        xorl    %ebp, %ebp+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        movq    %r9, 0x8(%rsp)+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x20(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x28(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x30(%rsi), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x38(%rsi), %rax, %r9+        adcxq   %rax, %r8+        adoxq   %rbp, %r9+        adcq    %rbp, %r9+        movq    0x10(%rcx), %rdx+        xorl    %ebp, %ebp+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        movq    %r10, 0x10(%rsp)+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x20(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x28(%rsi), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x30(%rsi), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x38(%rsi), %rax, %r10+        adcxq   %rax, %r9+        adoxq   %rbp, %r10+        adcq    %rbp, %r10+        movq    0x18(%rcx), %rdx+        xorl    %ebp, %ebp+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        movq    %r11, 0x18(%rsp)+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x20(%rsi), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x28(%rsi), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x30(%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x38(%rsi), %rax, %r11+        adcxq   %rax, %r10+        adoxq   %rbp, %r11+        adcq    %rbp, %r11+        movq    0x20(%rcx), %rdx+        xorl    %ebp, %ebp+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        movq    %r12, 0x20(%rsp)+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x20(%rsi), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x28(%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x30(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x38(%rsi), %rax, %r12+        adcxq   %rax, %r11+        adoxq   %rbp, %r12+        adcq    %rbp, %r12+        movq    0x28(%rcx), %rdx+        xorl    %ebp, %ebp+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        movq    %r13, 0x28(%rsp)+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x20(%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x28(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x30(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x38(%rsi), %rax, %r13+        adcxq   %rax, %r12+        adoxq   %rbp, %r13+        adcq    %rbp, %r13+        movq    0x30(%rcx), %rdx+        xorl    %ebp, %ebp+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        movq    %r14, 0x30(%rsp)+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x20(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x28(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x30(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x38(%rsi), %rax, %r14+        adcxq   %rax, %r13+        adoxq   %rbp, %r14+        adcq    %rbp, %r14+        movq    0x38(%rcx), %rdx+        xorl    %ebp, %ebp+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %r8+        movq    %r15, 0x38(%rsp)+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x20(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x28(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x30(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x38(%rsi), %rax, %r15+        adcxq   %rax, %r14+        adoxq   %rbp, %r15+        adcq    %rbp, %r15+        movq    0x40(%rsi), %rdx+        xorl    %ebp, %ebp+        mulxq   (%rcx), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x8(%rcx), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x10(%rcx), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x18(%rcx), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x20(%rcx), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x28(%rcx), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x30(%rcx), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x38(%rcx), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbp, %rbx+        adcq    %rbx, %rbp+        movq    0x40(%rcx), %rdx+        xorl    %eax, %eax+        mulxq   (%rsi), %rax, %rbx+        adcxq   %rax, %r8+        adoxq   %rbx, %r9+        mulxq   0x8(%rsi), %rax, %rbx+        adcxq   %rax, %r9+        adoxq   %rbx, %r10+        mulxq   0x10(%rsi), %rax, %rbx+        adcxq   %rax, %r10+        adoxq   %rbx, %r11+        mulxq   0x18(%rsi), %rax, %rbx+        adcxq   %rax, %r11+        adoxq   %rbx, %r12+        mulxq   0x20(%rsi), %rax, %rbx+        adcxq   %rax, %r12+        adoxq   %rbx, %r13+        mulxq   0x28(%rsi), %rax, %rbx+        adcxq   %rax, %r13+        adoxq   %rbx, %r14+        mulxq   0x30(%rsi), %rax, %rbx+        adcxq   %rax, %r14+        adoxq   %rbx, %r15+        mulxq   0x38(%rsi), %rax, %rbx+        adcxq   %rax, %r15+        adoxq   %rbx, %rbp+        mulxq   0x40(%rsi), %rax, %rbx+        adcq    %rax, %rbp+        movq    %r8, %rax+        andq    $0x1ff, %rax+        shrdq   $0x9, %r9, %r8+        shrdq   $0x9, %r10, %r9+        shrdq   $0x9, %r11, %r10+        shrdq   $0x9, %r12, %r11+        shrdq   $0x9, %r13, %r12+        shrdq   $0x9, %r14, %r13+        shrdq   $0x9, %r15, %r14+        shrdq   $0x9, %rbp, %r15+        shrq    $0x9, %rbp+        addq    %rax, %rbp+        stc+        adcq    (%rsp), %r8+        adcq    0x8(%rsp), %r9+        adcq    0x10(%rsp), %r10+        adcq    0x18(%rsp), %r11+        adcq    0x20(%rsp), %r12+        adcq    0x28(%rsp), %r13+        adcq    0x30(%rsp), %r14+        adcq    0x38(%rsp), %r15+        adcq    $0xfffffffffffffe00, %rbp+        cmc+        sbbq    $0x0, %r8+        movq    %r8, (%rdi)+        sbbq    $0x0, %r9+        movq    %r9, 0x8(%rdi)+        sbbq    $0x0, %r10+        movq    %r10, 0x10(%rdi)+        sbbq    $0x0, %r11+        movq    %r11, 0x18(%rdi)+        sbbq    $0x0, %r12+        movq    %r12, 0x20(%rdi)+        sbbq    $0x0, %r13+        movq    %r13, 0x28(%rdi)+        sbbq    $0x0, %r14+        movq    %r14, 0x30(%rdi)+        sbbq    $0x0, %r15+        movq    %r15, 0x38(%rdi)+        sbbq    $0x0, %rbp+        andq    $0x1ff, %rbp+        movq    %rbp, 0x40(%rdi)+        CFI_INC_RSP(64)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_mul_p521)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_sqr_p521)++Lp521_jscalarmul_sqr_p521:+        CFI_START+        CFI_DEC_RSP(64)+        xorl    %ebp, %ebp+        movq    (%rsi), %rdx+        mulxq   0x8(%rsi), %r9, %rax+        movq    %r9, 0x8(%rsp)+        mulxq   0x10(%rsi), %r10, %rcx+        adcxq   %rax, %r10+        movq    %r10, 0x10(%rsp)+        mulxq   0x18(%rsi), %r11, %rax+        adcxq   %rcx, %r11+        mulxq   0x20(%rsi), %r12, %rcx+        adcxq   %rax, %r12+        mulxq   0x28(%rsi), %r13, %rax+        adcxq   %rcx, %r13+        mulxq   0x30(%rsi), %r14, %rcx+        adcxq   %rax, %r14+        mulxq   0x38(%rsi), %r15, %r8+        adcxq   %rcx, %r15+        adcxq   %rbp, %r8+        xorl    %ebp, %ebp+        movq    0x8(%rsi), %rdx+        mulxq   0x10(%rsi), %rax, %rcx+        adcxq   %rax, %r11+        adoxq   %rcx, %r12+        movq    %r11, 0x18(%rsp)+        mulxq   0x18(%rsi), %rax, %rcx+        adcxq   %rax, %r12+        adoxq   %rcx, %r13+        movq    %r12, 0x20(%rsp)+        mulxq   0x20(%rsi), %rax, %rcx+        adcxq   %rax, %r13+        adoxq   %rcx, %r14+        mulxq   0x28(%rsi), %rax, %rcx+        adcxq   %rax, %r14+        adoxq   %rcx, %r15+        mulxq   0x30(%rsi), %rax, %rcx+        adcxq   %rax, %r15+        adoxq   %rcx, %r8+        mulxq   0x38(%rsi), %rax, %r9+        adcxq   %rax, %r8+        adoxq   %rbp, %r9+        movq    0x20(%rsi), %rdx+        mulxq   0x28(%rsi), %rax, %r10+        adcxq   %rax, %r9+        adoxq   %rbp, %r10+        adcxq   %rbp, %r10+        xorl    %ebp, %ebp+        movq    0x10(%rsi), %rdx+        mulxq   0x18(%rsi), %rax, %rcx+        adcxq   %rax, %r13+        adoxq   %rcx, %r14+        movq    %r13, 0x28(%rsp)+        mulxq   0x20(%rsi), %rax, %rcx+        adcxq   %rax, %r14+        adoxq   %rcx, %r15+        movq    %r14, 0x30(%rsp)+        mulxq   0x28(%rsi), %rax, %rcx+        adcxq   %rax, %r15+        adoxq   %rcx, %r8+        mulxq   0x30(%rsi), %rax, %rcx+        adcxq   %rax, %r8+        adoxq   %rcx, %r9+        mulxq   0x38(%rsi), %rax, %rcx+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+        movq    0x30(%rsi), %rdx+        mulxq   0x20(%rsi), %rax, %r11+        adcxq   %rax, %r10+        adoxq   %rbp, %r11+        mulxq   0x28(%rsi), %rax, %r12+        adcxq   %rax, %r11+        adoxq   %rbp, %r12+        adcxq   %rbp, %r12+        xorl    %ebp, %ebp+        movq    0x18(%rsi), %rdx+        mulxq   0x20(%rsi), %rax, %rcx+        adcxq   %rax, %r15+        adoxq   %rcx, %r8+        movq    %r15, 0x38(%rsp)+        mulxq   0x28(%rsi), %rax, %rcx+        adcxq   %rax, %r8+        adoxq   %rcx, %r9+        mulxq   0x30(%rsi), %rax, %rcx+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+        mulxq   0x38(%rsi), %rax, %rcx+        adcxq   %rax, %r10+        adoxq   %rcx, %r11+        movq    0x38(%rsi), %rdx+        mulxq   0x20(%rsi), %rax, %rcx+        adcxq   %rax, %r11+        adoxq   %rcx, %r12+        mulxq   0x28(%rsi), %rax, %r13+        adcxq   %rax, %r12+        adoxq   %rbp, %r13+        mulxq   0x30(%rsi), %rax, %r14+        adcxq   %rax, %r13+        adoxq   %rbp, %r14+        adcxq   %rbp, %r14+        xorl    %ebp, %ebp+        movq    (%rsi), %rdx+        mulxq   %rdx, %rax, %rcx+        movq    %rax, (%rsp)+        movq    0x8(%rsp), %rax+        adcxq   %rax, %rax+        adoxq   %rcx, %rax+        movq    %rax, 0x8(%rsp)+        movq    0x10(%rsp), %rax+        movq    0x8(%rsi), %rdx+        mulxq   %rdx, %rdx, %rcx+        adcxq   %rax, %rax+        adoxq   %rdx, %rax+        movq    %rax, 0x10(%rsp)+        movq    0x18(%rsp), %rax+        adcxq   %rax, %rax+        adoxq   %rcx, %rax+        movq    %rax, 0x18(%rsp)+        movq    0x20(%rsp), %rax+        movq    0x10(%rsi), %rdx+        mulxq   %rdx, %rdx, %rcx+        adcxq   %rax, %rax+        adoxq   %rdx, %rax+        movq    %rax, 0x20(%rsp)+        movq    0x28(%rsp), %rax+        adcxq   %rax, %rax+        adoxq   %rcx, %rax+        movq    %rax, 0x28(%rsp)+        movq    0x30(%rsp), %rax+        movq    0x18(%rsi), %rdx+        mulxq   %rdx, %rdx, %rcx+        adcxq   %rax, %rax+        adoxq   %rdx, %rax+        movq    %rax, 0x30(%rsp)+        movq    0x38(%rsp), %rax+        adcxq   %rax, %rax+        adoxq   %rcx, %rax+        movq    %rax, 0x38(%rsp)+        movq    0x20(%rsi), %rdx+        mulxq   %rdx, %rdx, %rcx+        adcxq   %r8, %r8+        adoxq   %rdx, %r8+        adcxq   %r9, %r9+        adoxq   %rcx, %r9+        movq    0x28(%rsi), %rdx+        mulxq   %rdx, %rdx, %rcx+        adcxq   %r10, %r10+        adoxq   %rdx, %r10+        adcxq   %r11, %r11+        adoxq   %rcx, %r11+        movq    0x30(%rsi), %rdx+        mulxq   %rdx, %rdx, %rcx+        adcxq   %r12, %r12+        adoxq   %rdx, %r12+        adcxq   %r13, %r13+        adoxq   %rcx, %r13+        movq    0x38(%rsi), %rdx+        mulxq   %rdx, %rdx, %r15+        adcxq   %r14, %r14+        adoxq   %rdx, %r14+        adcxq   %rbp, %r15+        adoxq   %rbp, %r15+        movq    0x40(%rsi), %rdx+        movq    %rdx, %rbp+        imulq   %rbp, %rbp+        addq    %rdx, %rdx+        mulxq   (%rsi), %rax, %rcx+        adcxq   %rax, %r8+        adoxq   %rcx, %r9+        mulxq   0x8(%rsi), %rax, %rcx+        adcxq   %rax, %r9+        adoxq   %rcx, %r10+        mulxq   0x10(%rsi), %rax, %rcx+        adcxq   %rax, %r10+        adoxq   %rcx, %r11+        mulxq   0x18(%rsi), %rax, %rcx+        adcxq   %rax, %r11+        adoxq   %rcx, %r12+        mulxq   0x20(%rsi), %rax, %rcx+        adcxq   %rax, %r12+        adoxq   %rcx, %r13+        mulxq   0x28(%rsi), %rax, %rcx+        adcxq   %rax, %r13+        adoxq   %rcx, %r14+        mulxq   0x30(%rsi), %rax, %rcx+        adcxq   %rax, %r14+        adoxq   %rcx, %r15+        mulxq   0x38(%rsi), %rax, %rcx+        adcxq   %rax, %r15+        adoxq   %rcx, %rbp+        adcq    $0x0, %rbp+        movq    %r8, %rax+        andq    $0x1ff, %rax+        shrdq   $0x9, %r9, %r8+        shrdq   $0x9, %r10, %r9+        shrdq   $0x9, %r11, %r10+        shrdq   $0x9, %r12, %r11+        shrdq   $0x9, %r13, %r12+        shrdq   $0x9, %r14, %r13+        shrdq   $0x9, %r15, %r14+        shrdq   $0x9, %rbp, %r15+        shrq    $0x9, %rbp+        addq    %rax, %rbp+        stc+        adcq    (%rsp), %r8+        adcq    0x8(%rsp), %r9+        adcq    0x10(%rsp), %r10+        adcq    0x18(%rsp), %r11+        adcq    0x20(%rsp), %r12+        adcq    0x28(%rsp), %r13+        adcq    0x30(%rsp), %r14+        adcq    0x38(%rsp), %r15+        adcq    $0xfffffffffffffe00, %rbp+        cmc+        sbbq    $0x0, %r8+        movq    %r8, (%rdi)+        sbbq    $0x0, %r9+        movq    %r9, 0x8(%rdi)+        sbbq    $0x0, %r10+        movq    %r10, 0x10(%rdi)+        sbbq    $0x0, %r11+        movq    %r11, 0x18(%rdi)+        sbbq    $0x0, %r12+        movq    %r12, 0x20(%rdi)+        sbbq    $0x0, %r13+        movq    %r13, 0x28(%rdi)+        sbbq    $0x0, %r14+        movq    %r14, 0x30(%rdi)+        sbbq    $0x0, %r15+        movq    %r15, 0x38(%rdi)+        sbbq    $0x0, %rbp+        andq    $0x1ff, %rbp+        movq    %rbp, 0x40(%rdi)+        CFI_INC_RSP(64)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_sqr_p521)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/s2n/x86_att/p521_jscalarmul_alt.S view
@@ -0,0 +1,2850 @@+// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.+// SPDX-License-Identifier: Apache-2.0 OR ISC OR MIT-0++// ----------------------------------------------------------------------------+// Jacobian form scalar multiplication for P-521+// Input scalar[9], point[27]; output res[27]+//+// extern void p521_jscalarmul_alt+//   (uint64_t res[static 27],+//    const uint64_t scalar[static 9],+//    const uint64_t point[static 27]);+//+// This function is a variant of its affine point version p521_scalarmul.+// Here, input and output points are assumed to be in Jacobian form with+// a triple (x,y,z) representing the affine point (x/z^2,y/z^3) when+// z is nonzero or the point at infinity (group identity) if z = 0.+//+// Given scalar = n and point = P, assumed to be on the NIST elliptic+// curve P-521, returns a representation of n * P. If the result is the+// point at infinity (either because the input point was or because the+// scalar was a multiple of p_521) then the output is guaranteed to+// represent the point at infinity, i.e. to have its z coordinate zero.+//+// Standard x86-64 ABI: RDI = res, RSI = scalar, RDX = point+// Microsoft x64 ABI:   RCX = res, RDX = scalar, R8 = point+// ----------------------------------------------------------------------------++#include "_internal_s2n_bignum_x86_att.h"+++        S2N_BN_SYM_VISIBILITY_DIRECTIVE(p521_jscalarmul_alt)+        S2N_BN_FUNCTION_TYPE_DIRECTIVE(p521_jscalarmul_alt)+        S2N_BN_SYM_PRIVACY_DIRECTIVE(p521_jscalarmul_alt)+++        .text+        .balign 32++// Size of individual field elements++#define NUMSIZE 72+#define JACSIZE (3*NUMSIZE)++// Intermediate variables on the stack.+// The table is 16 entries, each of size JACSIZE = 3 * NUMSIZE+// Uppercase syntactic variants make x86_att version simpler to generate.++#define SCALARB (0*NUMSIZE)+#define scalarb (0*NUMSIZE)(%rsp)+#define ACC (1*NUMSIZE)+#define acc (1*NUMSIZE)(%rsp)+#define TABENT (4*NUMSIZE)+#define tabent (4*NUMSIZE)(%rsp)++#define TAB (7*NUMSIZE)+#define tab (7*NUMSIZE)(%rsp)++#define res (55*NUMSIZE)(%rsp)++#define NSPACE 56*NUMSIZE++// Avoid using .rep for the sake of the BoringSSL/AWS-LC delocator,+// which doesn't accept repetitions, assembler macros etc.++#define selectblock(I,C)                                        \+        cmpq    $I, %rdi ;                                         \+        cmovzq  TAB+JACSIZE*(I-1)+C*NUMSIZE(%rsp), %rax ;         \+        cmovzq  TAB+JACSIZE*(I-1)+8+C*NUMSIZE(%rsp), %rbx ;       \+        cmovzq  TAB+JACSIZE*(I-1)+16+C*NUMSIZE(%rsp), %rcx ;      \+        cmovzq  TAB+JACSIZE*(I-1)+24+C*NUMSIZE(%rsp), %rdx ;      \+        cmovzq  TAB+JACSIZE*(I-1)+32+C*NUMSIZE(%rsp), %r8 ;       \+        cmovzq  TAB+JACSIZE*(I-1)+40+C*NUMSIZE(%rsp), %r9 ;       \+        cmovzq  TAB+JACSIZE*(I-1)+48+C*NUMSIZE(%rsp), %r10 ;      \+        cmovzq  TAB+JACSIZE*(I-1)+56+C*NUMSIZE(%rsp), %r11 ;      \+        cmovzq  TAB+JACSIZE*(I-1)+64+C*NUMSIZE(%rsp), %r12++S2N_BN_SYMBOL(p521_jscalarmul_alt):+        CFI_START+        _CET_ENDBR++// The Windows version literally calls the standard ABI version.+// This simplifies the proofs since subroutine offsets are fixed.++#if WINDOWS_ABI+        CFI_PUSH(%rdi)+        CFI_PUSH(%rsi)+        movq    %rcx, %rdi+        movq    %rdx, %rsi+        movq    %r8, %rdx+        CFI_CALL(Lp521_jscalarmul_alt_standard)+        CFI_POP(%rsi)+        CFI_POP(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(p521_jscalarmul_alt)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_alt_standard)++Lp521_jscalarmul_alt_standard:+        CFI_START+#endif++// Real start of the standard ABI code.++        CFI_PUSH(%r15)+        CFI_PUSH(%r14)+        CFI_PUSH(%r13)+        CFI_PUSH(%r12)+        CFI_PUSH(%rbp)+        CFI_PUSH(%rbx)++        CFI_DEC_RSP(NSPACE)++// Preserve the "res" input argument; others get processed early.++        movq    %rdi, res++// Reduce the input scalar mod n_521 and store it to "scalarb".++        movq    %rdx, %rbx+        leaq    SCALARB(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_bignum_mod_n521_9)++// Set the tab[0] table entry to the input point = 1 * P, but also+// reduce all coordinates modulo p. In principle we assume reduction+// as a precondition, but this reduces the scope for surprise, e.g.+// making sure that any input with z = 0 is treated as zero, even+// if the other coordinates are not in fact reduced.++        leaq    TAB(%rsp), %rdi+        movq    %rbx, %rsi+        CFI_CALL(Lp521_jscalarmul_alt_bignum_mod_p521_9)++        leaq    TAB+NUMSIZE(%rsp), %rdi+        leaq    NUMSIZE(%rbx), %rsi+        CFI_CALL(Lp521_jscalarmul_alt_bignum_mod_p521_9)++        leaq    TAB+2*NUMSIZE(%rsp), %rdi+        leaq    2*NUMSIZE(%rbx), %rsi+        CFI_CALL(Lp521_jscalarmul_alt_bignum_mod_p521_9)++// If bit 520 of the scalar is set, then negate the scalar mod n_521,+// i.e. do scalar |-> n_521 - scalar, and also the point to compensate+// by negating its y coordinate. This further step is not needed by+// the indexing scheme (the top window is only a couple of bits either+// way), but is convenient to exclude a problem with the specific value+// scalar = n_521 - 18, where the last Jacobian addition is of the form+// (n_521 - 9) * P + -(9 * P) and hence is a degenerate doubling case.++        xorl    %eax, %eax+        notq    %rax+        movq    $0xbb6fb71e91386409, %r8+        subq    SCALARB(%rsp), %r8+        movq    $0x3bb5c9b8899c47ae, %r9+        sbbq    SCALARB+8(%rsp), %r9+        movq    $0x7fcc0148f709a5d0, %r10+        sbbq    SCALARB+16(%rsp), %r10+        movq    $0x51868783bf2f966b, %r11+        sbbq    SCALARB+24(%rsp), %r11+        leaq    -5(%rax), %r12+        sbbq    SCALARB+32(%rsp), %r12+        movq    %rax, %r13+        sbbq    SCALARB+40(%rsp), %r13+        movq    %rax, %r14+        sbbq    SCALARB+48(%rsp), %r14+        movq    %rax, %r15+        sbbq    SCALARB+56(%rsp), %r15+        movq    $0x1ff, %rax+        movq    SCALARB+64(%rsp), %rcx+        sbbq    %rcx, %rax++        btq     $8, %rcx+        sbbq    %rcx, %rcx++        cmovncq SCALARB(%rsp), %r8+        cmovncq SCALARB+8(%rsp), %r9+        cmovncq SCALARB+16(%rsp), %r10+        cmovncq SCALARB+24(%rsp), %r11+        cmovncq SCALARB+32(%rsp), %r12+        cmovncq SCALARB+40(%rsp), %r13+        cmovncq SCALARB+48(%rsp), %r14+        cmovncq SCALARB+56(%rsp), %r15+        cmovncq SCALARB+64(%rsp), %rax++        movq    %r8, SCALARB(%rsp)+        movq    %r9, SCALARB+8(%rsp)+        movq    %r10, SCALARB+16(%rsp)+        movq    %r11, SCALARB+24(%rsp)+        movq    %r12, SCALARB+32(%rsp)+        movq    %r13, SCALARB+40(%rsp)+        movq    %r14, SCALARB+48(%rsp)+        movq    %r15, SCALARB+56(%rsp)+        movq    %rax, SCALARB+64(%rsp)++        movq    TAB+NUMSIZE(%rsp), %r8+        movq    TAB+NUMSIZE+8(%rsp), %r9+        movq    TAB+NUMSIZE+16(%rsp), %r10+        movq    TAB+NUMSIZE+24(%rsp), %r11+        movq    TAB+NUMSIZE+32(%rsp), %r12+        movq    TAB+NUMSIZE+40(%rsp), %r13+        movq    TAB+NUMSIZE+48(%rsp), %r14+        movq    TAB+NUMSIZE+56(%rsp), %r15+        movq    TAB+NUMSIZE+64(%rsp), %rax++        movq    %r8, %rbx+        movq    %r12, %rbp+        orq     %r9, %rbx+        orq     %r13, %rbp+        orq     %r10, %rbx+        orq     %r14, %rbp+        orq     %r11, %rbx+        orq     %r15, %rbp+        orq     %rbp, %rbx+        orq     %rax, %rbx+        cmovzq  %rbx, %rcx++        xorq    %rcx, %r8+        xorq    %rcx, %r9+        xorq    %rcx, %r10+        xorq    %rcx, %r11+        xorq    %rcx, %r12+        xorq    %rcx, %r13+        xorq    %rcx, %r14+        xorq    %rcx, %r15+        andq    $0x1FF, %rcx+        xorq    %rcx, %rax++        movq    %r8, TAB+NUMSIZE(%rsp)+        movq    %r9, TAB+NUMSIZE+8(%rsp)+        movq    %r10, TAB+NUMSIZE+16(%rsp)+        movq    %r11, TAB+NUMSIZE+24(%rsp)+        movq    %r12, TAB+NUMSIZE+32(%rsp)+        movq    %r13, TAB+NUMSIZE+40(%rsp)+        movq    %r14, TAB+NUMSIZE+48(%rsp)+        movq    %r15, TAB+NUMSIZE+56(%rsp)+        movq    %rax, TAB+NUMSIZE+64(%rsp)++// Compute and record tab[1] = 2 * p, ..., tab[15] = 16 * P++        leaq    TAB+JACSIZE*1(%rsp), %rdi+        leaq    TAB(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_alt_jdouble)++        leaq    TAB+JACSIZE*2(%rsp), %rdi+        leaq    TAB+JACSIZE*1(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp521_jscalarmul_alt_jadd)++        leaq    TAB+JACSIZE*3(%rsp), %rdi+        leaq    TAB+JACSIZE*1(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_alt_jdouble)++        leaq    TAB+JACSIZE*4(%rsp), %rdi+        leaq    TAB+JACSIZE*3(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp521_jscalarmul_alt_jadd)++        leaq    TAB+JACSIZE*5(%rsp), %rdi+        leaq    TAB+JACSIZE*2(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_alt_jdouble)++        leaq    TAB+JACSIZE*6(%rsp), %rdi+        leaq    TAB+JACSIZE*5(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp521_jscalarmul_alt_jadd)++        leaq    TAB+JACSIZE*7(%rsp), %rdi+        leaq    TAB+JACSIZE*3(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_alt_jdouble)++        leaq    TAB+JACSIZE*8(%rsp), %rdi+        leaq    TAB+JACSIZE*7(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp521_jscalarmul_alt_jadd)++        leaq    TAB+JACSIZE*9(%rsp), %rdi+        leaq    TAB+JACSIZE*4(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_alt_jdouble)++        leaq    TAB+JACSIZE*10(%rsp), %rdi+        leaq    TAB+JACSIZE*9(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp521_jscalarmul_alt_jadd)++        leaq    TAB+JACSIZE*11(%rsp), %rdi+        leaq    TAB+JACSIZE*5(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_alt_jdouble)++        leaq    TAB+JACSIZE*12(%rsp), %rdi+        leaq    TAB+JACSIZE*11(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp521_jscalarmul_alt_jadd)++        leaq    TAB+JACSIZE*13(%rsp), %rdi+        leaq    TAB+JACSIZE*6(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_alt_jdouble)++        leaq    TAB+JACSIZE*14(%rsp), %rdi+        leaq    TAB+JACSIZE*13(%rsp), %rsi+        leaq    TAB(%rsp), %rdx+        CFI_CALL(Lp521_jscalarmul_alt_jadd)++        leaq    TAB+JACSIZE*15(%rsp), %rdi+        leaq    TAB+JACSIZE*7(%rsp), %rsi+        CFI_CALL(Lp521_jscalarmul_alt_jdouble)++// Add the recoding constant sum_i(16 * 32^i) to the scalar to allow signed+// digits. The digits of the constant, in lowest-to-highest order, are as+// follows; they are generated dynamically to use fewer large constant loads.+//+// 0x0842108421084210 %rax+// 0x1084210842108421 %rbx+// 0x2108421084210842 %rbx<<1+// 0x4210842108421084 %rbx<<2+// 0x8421084210842108 %rbx<<3+// 0x0842108421084210 %rax+// 0x1084210842108421 %rbx+// 0x2108421084210842 %rbx<<1+// 0x0000000000000084++        movq    $0x1084210842108421, %rax+        movq    %rax, %rbx+        shrq    $1, %rax+        movq    SCALARB(%rsp), %r8+        addq    %rax, %r8+        movq    SCALARB+8(%rsp), %r9+        adcq    %rbx, %r9+        leaq    (%rbx,%rbx), %rcx+        movq    SCALARB+16(%rsp), %r10+        adcq    %rcx, %r10+        leaq    (%rcx,%rcx), %rcx+        movq    SCALARB+24(%rsp), %r11+        adcq    %rcx, %r11+        leaq    (%rcx,%rcx), %rcx+        movq    SCALARB+32(%rsp), %r12+        adcq    %rcx, %r12+        movq    SCALARB+40(%rsp), %r13+        adcq    %rax, %r13+        movq    SCALARB+48(%rsp), %r14+        adcq    %rbx, %r14+        movq    SCALARB+56(%rsp), %r15+        leaq    (%rbx,%rbx), %rcx+        adcq    %rcx, %r15+        movq    SCALARB+64(%rsp), %rax+        adcq    $0x84, %rax++// Because of the initial reduction the top bitfield (>= bits 520) is <= 1,+// i.e. just a single bit. Record that in %rdi, then shift the whole+// scalar left 56 bits to align the top of the next bitfield with the MSB+// (bits 571..575).++        movq    %rax, %rdi+        shrq    $8, %rdi+        shldq   $56, %r15, %rax+        shldq   $56, %r14, %r15+        shldq   $56, %r13, %r14+        shldq   $56, %r12, %r13+        shldq   $56, %r11, %r12+        shldq   $56, %r10, %r11+        shldq   $56, %r9, %r10+        shldq   $56, %r8, %r9+        shlq    $56, %r8++        movq    %r8, SCALARB(%rsp)+        movq    %r9, SCALARB+8(%rsp)+        movq    %r10, SCALARB+16(%rsp)+        movq    %r11, SCALARB+24(%rsp)+        movq    %r12, SCALARB+32(%rsp)+        movq    %r13, SCALARB+40(%rsp)+        movq    %r14, SCALARB+48(%rsp)+        movq    %r15, SCALARB+56(%rsp)+        movq    %rax, SCALARB+64(%rsp)++// According to the top bit, initialize the accumulator to P or 0. This top+// digit, uniquely, is not recoded so there is no sign adjustment to make.+// We only really need to adjust the z coordinate to zero, but do all three.++        xorl    %ecx, %ecx+        testq   %rdi, %rdi++        movq    TAB(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC(%rsp)+        movq    TAB+8(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+8(%rsp)+        movq    TAB+16(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+16(%rsp)+        movq    TAB+24(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+24(%rsp)+        movq    TAB+32(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+32(%rsp)+        movq    TAB+40(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+40(%rsp)+        movq    TAB+48(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+48(%rsp)+        movq    TAB+56(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+56(%rsp)+        movq    TAB+64(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+64(%rsp)+        movq    TAB+72(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+72(%rsp)+        movq    TAB+80(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+80(%rsp)+        movq    TAB+88(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+88(%rsp)+        movq    TAB+96(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+96(%rsp)+        movq    TAB+104(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+104(%rsp)+        movq    TAB+112(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+112(%rsp)+        movq    TAB+120(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+120(%rsp)+        movq    TAB+128(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+128(%rsp)+        movq    TAB+136(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+136(%rsp)+        movq    TAB+144(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+144(%rsp)+        movq    TAB+152(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+152(%rsp)+        movq    TAB+160(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+160(%rsp)+        movq    TAB+168(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+168(%rsp)+        movq    TAB+176(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+176(%rsp)+        movq    TAB+184(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+184(%rsp)+        movq    TAB+192(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+192(%rsp)+        movq    TAB+200(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+200(%rsp)+        movq    TAB+208(%rsp), %rax+        cmovzq  %rcx, %rax+        movq    %rax, ACC+208(%rsp)++// Main loop over size-5 bitfields: double 5 times then add signed digit+// At each stage we shift the scalar left by 5 bits so we can simply pick+// the top 5 bits as the bitfield, saving some fiddle over indexing.++        movl    $520, %ebp++Lp521_jscalarmul_alt_mainloop:+        subq    $5, %rbp++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_jdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_jdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_jdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_jdouble)++        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_jdouble)++// Choose the bitfield and adjust it to sign and magnitude++        movq    SCALARB(%rsp), %r8+        movq    SCALARB+8(%rsp), %r9+        movq    SCALARB+16(%rsp), %r10+        movq    SCALARB+24(%rsp), %r11+        movq    SCALARB+32(%rsp), %r12+        movq    SCALARB+40(%rsp), %r13+        movq    SCALARB+48(%rsp), %r14+        movq    SCALARB+56(%rsp), %r15+        movq    SCALARB+64(%rsp), %rax+++        movq    %rax, %rdi+        shrq    $59, %rdi++        shldq   $5, %r15, %rax+        shldq   $5, %r14, %r15+        shldq   $5, %r13, %r14+        shldq   $5, %r12, %r13+        shldq   $5, %r11, %r12+        shldq   $5, %r10, %r11+        shldq   $5, %r9, %r10+        shldq   $5, %r8, %r9+        shlq    $5, %r8++        movq    %r8, SCALARB(%rsp)+        movq    %r9, SCALARB+8(%rsp)+        movq    %r10, SCALARB+16(%rsp)+        movq    %r11, SCALARB+24(%rsp)+        movq    %r12, SCALARB+32(%rsp)+        movq    %r13, SCALARB+40(%rsp)+        movq    %r14, SCALARB+48(%rsp)+        movq    %r15, SCALARB+56(%rsp)+        movq    %rax, SCALARB+64(%rsp)++        subq    $16, %rdi+        sbbq    %rsi, %rsi // %rsi = sign of digit (-1 = negative)+        xorq    %rsi, %rdi+        subq    %rsi, %rdi // %rdi = absolute value of digit++// Conditionally select the table entry tab[i-1] = i * P in constant time+// Again, this is done in separate sweeps per coordinate, doing y last.++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        selectblock(1,0)+        selectblock(2,0)+        selectblock(3,0)+        selectblock(4,0)+        selectblock(5,0)+        selectblock(6,0)+        selectblock(7,0)+        selectblock(8,0)+        selectblock(9,0)+        selectblock(10,0)+        selectblock(11,0)+        selectblock(12,0)+        selectblock(13,0)+        selectblock(14,0)+        selectblock(15,0)+        selectblock(16,0)+        movq    %rax, TABENT(%rsp)+        movq    %rbx, TABENT+8(%rsp)+        movq    %rcx, TABENT+16(%rsp)+        movq    %rdx, TABENT+24(%rsp)+        movq    %r8, TABENT+32(%rsp)+        movq    %r9, TABENT+40(%rsp)+        movq    %r10, TABENT+48(%rsp)+        movq    %r11, TABENT+56(%rsp)+        movq    %r12, TABENT+64(%rsp)++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        selectblock(1,2)+        selectblock(2,2)+        selectblock(3,2)+        selectblock(4,2)+        selectblock(5,2)+        selectblock(6,2)+        selectblock(7,2)+        selectblock(8,2)+        selectblock(9,2)+        selectblock(10,2)+        selectblock(11,2)+        selectblock(12,2)+        selectblock(13,2)+        selectblock(14,2)+        selectblock(15,2)+        selectblock(16,2)+        movq    %rax, TABENT+2*NUMSIZE(%rsp)+        movq    %rbx, TABENT+2*NUMSIZE+8(%rsp)+        movq    %rcx, TABENT+2*NUMSIZE+16(%rsp)+        movq    %rdx, TABENT+2*NUMSIZE+24(%rsp)+        movq    %r8, TABENT+2*NUMSIZE+32(%rsp)+        movq    %r9, TABENT+2*NUMSIZE+40(%rsp)+        movq    %r10, TABENT+2*NUMSIZE+48(%rsp)+        movq    %r11, TABENT+2*NUMSIZE+56(%rsp)+        movq    %r12, TABENT+2*NUMSIZE+64(%rsp)++        xorl    %eax, %eax+        xorl    %ebx, %ebx+        xorl    %ecx, %ecx+        xorl    %edx, %edx+        xorl    %r8d, %r8d+        xorl    %r9d, %r9d+        xorl    %r10d, %r10d+        xorl    %r11d, %r11d+        xorl    %r12d, %r12d+        selectblock(1,1)+        selectblock(2,1)+        selectblock(3,1)+        selectblock(4,1)+        selectblock(5,1)+        selectblock(6,1)+        selectblock(7,1)+        selectblock(8,1)+        selectblock(9,1)+        selectblock(10,1)+        selectblock(11,1)+        selectblock(12,1)+        selectblock(13,1)+        selectblock(14,1)+        selectblock(15,1)+        selectblock(16,1)++// Store it to "tabent" with the y coordinate optionally negated.+// This is done carefully to give coordinates < p_521 even in+// the degenerate case y = 0 (when z = 0 for points on the curve).++        movq    %rax, %r13+        orq     %rbx, %r13+        movq    %rcx, %r14+        orq     %rdx, %r14+        movq    %r8, %r15+        orq     %r9, %r15+        movq    %r10, %rdi+        orq     %r11, %rdi+        orq     %r14, %r13+        orq     %rdi, %r15+        orq     %r12, %r15+        orq     %r15, %r13+        cmovzq  %r13, %rsi++        xorq    %rsi, %rax+        xorq    %rsi, %rbx+        xorq    %rsi, %rcx+        xorq    %rsi, %rdx+        xorq    %rsi, %r8+        xorq    %rsi, %r9+        xorq    %rsi, %r10+        xorq    %rsi, %r11+        andq    $0x1FF, %rsi+        xorq    %rsi, %r12++        movq    %rax, TABENT+NUMSIZE(%rsp)+        movq    %rbx, TABENT+NUMSIZE+8(%rsp)+        movq    %rcx, TABENT+NUMSIZE+16(%rsp)+        movq    %rdx, TABENT+NUMSIZE+24(%rsp)+        movq    %r8, TABENT+NUMSIZE+32(%rsp)+        movq    %r9, TABENT+NUMSIZE+40(%rsp)+        movq    %r10, TABENT+NUMSIZE+48(%rsp)+        movq    %r11, TABENT+NUMSIZE+56(%rsp)+        movq    %r12, TABENT+NUMSIZE+64(%rsp)++// Add to the accumulator++        leaq    TABENT(%rsp), %rdx+        leaq    ACC(%rsp), %rsi+        leaq    ACC(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_jadd)++        testq   %rbp, %rbp+        jne     Lp521_jscalarmul_alt_mainloop++// That's the end of the main loop, and we just need to copy the+// result in "acc" to the output.++        movq    res, %rdi+        movq    ACC(%rsp), %rax+        movq    %rax, (%rdi)+        movq    ACC+8(%rsp), %rax+        movq    %rax, 8(%rdi)+        movq    ACC+16(%rsp), %rax+        movq    %rax, 16(%rdi)+        movq    ACC+24(%rsp), %rax+        movq    %rax, 24(%rdi)+        movq    ACC+32(%rsp), %rax+        movq    %rax, 32(%rdi)+        movq    ACC+40(%rsp), %rax+        movq    %rax, 40(%rdi)+        movq    ACC+48(%rsp), %rax+        movq    %rax, 48(%rdi)+        movq    ACC+56(%rsp), %rax+        movq    %rax, 56(%rdi)+        movq    ACC+64(%rsp), %rax+        movq    %rax, 64(%rdi)+        movq    ACC+72(%rsp), %rax+        movq    %rax, 72(%rdi)+        movq    ACC+80(%rsp), %rax+        movq    %rax, 80(%rdi)+        movq    ACC+88(%rsp), %rax+        movq    %rax, 88(%rdi)+        movq    ACC+96(%rsp), %rax+        movq    %rax, 96(%rdi)+        movq    ACC+104(%rsp), %rax+        movq    %rax, 104(%rdi)+        movq    ACC+112(%rsp), %rax+        movq    %rax, 112(%rdi)+        movq    ACC+120(%rsp), %rax+        movq    %rax, 120(%rdi)+        movq    ACC+128(%rsp), %rax+        movq    %rax, 128(%rdi)+        movq    ACC+136(%rsp), %rax+        movq    %rax, 136(%rdi)+        movq    ACC+144(%rsp), %rax+        movq    %rax, 144(%rdi)+        movq    ACC+152(%rsp), %rax+        movq    %rax, 152(%rdi)+        movq    ACC+160(%rsp), %rax+        movq    %rax, 160(%rdi)+        movq    ACC+168(%rsp), %rax+        movq    %rax, 168(%rdi)+        movq    ACC+176(%rsp), %rax+        movq    %rax, 176(%rdi)+        movq    ACC+184(%rsp), %rax+        movq    %rax, 184(%rdi)+        movq    ACC+192(%rsp), %rax+        movq    %rax, 192(%rdi)+        movq    ACC+200(%rsp), %rax+        movq    %rax, 200(%rdi)+        movq    ACC+208(%rsp), %rax+        movq    %rax, 208(%rdi)++// Restore stack and registers and return++        CFI_INC_RSP(NSPACE)+        CFI_POP(%rbx)+        CFI_POP(%rbp)+        CFI_POP(%r12)+        CFI_POP(%r13)+        CFI_POP(%r14)+        CFI_POP(%r15)+        CFI_RET++#if WINDOWS_ABI+S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_alt_standard)+#else+S2N_BN_SIZE_DIRECTIVE(p521_jscalarmul_alt)+#endif++// Local copies of subroutines, complete clones at the moment++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_alt_bignum_mod_p521_9)++Lp521_jscalarmul_alt_bignum_mod_p521_9:+        CFI_START+        CFI_PUSH(%rbx)+        movq    0x40(%rsi), %rax+        movl    $0x1ff, %edx+        andq    %rax, %rdx+        shrq    $0x9, %rax+        stc+        adcq    (%rsi), %rax+        movq    0x8(%rsi), %rcx+        adcq    $0x0, %rcx+        movq    0x10(%rsi), %r8+        adcq    $0x0, %r8+        movq    0x18(%rsi), %r9+        adcq    $0x0, %r9+        movq    0x20(%rsi), %r10+        adcq    $0x0, %r10+        movq    0x28(%rsi), %r11+        adcq    $0x0, %r11+        movq    0x30(%rsi), %rbx+        adcq    $0x0, %rbx+        movq    0x38(%rsi), %rsi+        adcq    $0x0, %rsi+        adcq    $0x0, %rdx+        cmpq    $0x200, %rdx+        sbbq    $0x0, %rax+        movq    %rax, (%rdi)+        sbbq    $0x0, %rcx+        movq    %rcx, 0x8(%rdi)+        sbbq    $0x0, %r8+        movq    %r8, 0x10(%rdi)+        sbbq    $0x0, %r9+        movq    %r9, 0x18(%rdi)+        sbbq    $0x0, %r10+        movq    %r10, 0x20(%rdi)+        sbbq    $0x0, %r11+        movq    %r11, 0x28(%rdi)+        sbbq    $0x0, %rbx+        movq    %rbx, 0x30(%rdi)+        sbbq    $0x0, %rsi+        movq    %rsi, 0x38(%rdi)+        sbbq    $0x0, %rdx+        andq    $0x1ff, %rdx+        movq    %rdx, 0x40(%rdi)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_alt_bignum_mod_p521_9)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_alt_bignum_mod_n521_9)++Lp521_jscalarmul_alt_bignum_mod_n521_9:+        CFI_START+        movq    0x40(%rsi), %rcx+        movq    $0xfffffffffffffe00, %rax+        orq     %rcx, %rax+        movq    %rax, 0x40(%rdi)+        shrq    $0x9, %rcx+        addq    $0x1, %rcx+        movq    $0x449048e16ec79bf7, %rax+        mulq    %rcx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    $0xc44a36477663b851, %rax+        mulq    %rcx+        xorq    %r10, %r10+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    $0x8033feb708f65a2f, %rax+        mulq    %rcx+        xorq    %r11, %r11+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    $0xae79787c40d06994, %rax+        mulq    %rcx+        imulq   $0x5, %rcx, %rcx+        addq    %rax, %r11+        adcq    %rdx, %rcx+        sbbq    %rdx, %rdx+        negq    %rdx+        xorl    %eax, %eax+        addq    (%rsi), %r8+        movq    %r8, (%rdi)+        adcq    0x8(%rsi), %r9+        movq    %r9, 0x8(%rdi)+        adcq    0x10(%rsi), %r10+        movq    %r10, 0x10(%rdi)+        adcq    0x18(%rsi), %r11+        movq    %r11, 0x18(%rdi)+        adcq    0x20(%rsi), %rcx+        movq    %rcx, 0x20(%rdi)+        adcq    0x28(%rsi), %rdx+        movq    %rdx, 0x28(%rdi)+        movq    0x30(%rsi), %rdx+        adcq    %rax, %rdx+        movq    %rdx, 0x30(%rdi)+        movq    0x38(%rsi), %rdx+        adcq    %rax, %rdx+        movq    %rdx, 0x38(%rdi)+        movq    0x40(%rdi), %rcx+        adcq    %rax, %rcx+        cmc+        sbbq    %rdx, %rdx+        movq    $0x449048e16ec79bf7, %r8+        andq    %rdx, %r8+        movq    $0xc44a36477663b851, %r9+        andq    %rdx, %r9+        movq    $0x8033feb708f65a2f, %r10+        andq    %rdx, %r10+        movq    $0xae79787c40d06994, %r11+        andq    %rdx, %r11+        andq    $0x5, %rdx+        subq    %r8, (%rdi)+        sbbq    %r9, 0x8(%rdi)+        sbbq    %r10, 0x10(%rdi)+        sbbq    %r11, 0x18(%rdi)+        sbbq    %rdx, 0x20(%rdi)+        sbbq    %rax, 0x28(%rdi)+        sbbq    %rax, 0x30(%rdi)+        sbbq    %rax, 0x38(%rdi)+        sbbl    %eax, %ecx+        andl    $0x1ff, %ecx+        movq    %rcx, 0x40(%rdi)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_alt_bignum_mod_n521_9)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_alt_jadd)++Lp521_jscalarmul_alt_jadd:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(528)+        movq    %rdi, 0x1f8(%rsp)+        movq    %rsi, 0x200(%rsp)+        movq    %rdx, 0x208(%rsp)+        movq    0x200(%rsp), %rsi+        leaq    0x90(%rsi), %rsi+        leaq    (%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_sqr_p521)+        movq    0x208(%rsp), %rdi+        leaq    0x90(%rdi), %rsi+        leaq    0x168(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_sqr_p521)+        movq    0x200(%rsp), %rsi+        movq    0x208(%rsp), %rdi+        leaq    0x48(%rsi), %rdx+        leaq    0x90(%rdi), %rsi+        leaq    0x1b0(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        movq    0x200(%rsp), %rsi+        movq    0x208(%rsp), %rdi+        leaq    0x48(%rdi), %rdx+        leaq    0x90(%rsi), %rsi+        leaq    0x48(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        movq    0x208(%rsp), %rdi+        leaq    (%rdi), %rdx+        leaq    (%rsp), %rsi+        leaq    0x90(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        movq    0x200(%rsp), %rsi+        leaq    (%rsi), %rdx+        leaq    0x168(%rsp), %rsi+        leaq    0x120(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        leaq    0x48(%rsp), %rdx+        leaq    (%rsp), %rsi+        leaq    0x48(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        leaq    0x1b0(%rsp), %rdx+        leaq    0x168(%rsp), %rsi+        leaq    0x1b0(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        movq    0x90(%rsp), %rax+        subq    0x120(%rsp), %rax+        movq    0x98(%rsp), %rdx+        sbbq    0x128(%rsp), %rdx+        movq    0xa0(%rsp), %r8+        sbbq    0x130(%rsp), %r8+        movq    0xa8(%rsp), %r9+        sbbq    0x138(%rsp), %r9+        movq    0xb0(%rsp), %r10+        sbbq    0x140(%rsp), %r10+        movq    0xb8(%rsp), %r11+        sbbq    0x148(%rsp), %r11+        movq    0xc0(%rsp), %r12+        sbbq    0x150(%rsp), %r12+        movq    0xc8(%rsp), %r13+        sbbq    0x158(%rsp), %r13+        movq    0xd0(%rsp), %r14+        sbbq    0x160(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0x168(%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x170(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x178(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x180(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x188(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x190(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x198(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x1a0(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x1a8(%rsp)+        movq    0x48(%rsp), %rax+        subq    0x1b0(%rsp), %rax+        movq    0x50(%rsp), %rdx+        sbbq    0x1b8(%rsp), %rdx+        movq    0x58(%rsp), %r8+        sbbq    0x1c0(%rsp), %r8+        movq    0x60(%rsp), %r9+        sbbq    0x1c8(%rsp), %r9+        movq    0x68(%rsp), %r10+        sbbq    0x1d0(%rsp), %r10+        movq    0x70(%rsp), %r11+        sbbq    0x1d8(%rsp), %r11+        movq    0x78(%rsp), %r12+        sbbq    0x1e0(%rsp), %r12+        movq    0x80(%rsp), %r13+        sbbq    0x1e8(%rsp), %r13+        movq    0x88(%rsp), %r14+        sbbq    0x1f0(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0x48(%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x50(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x58(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x60(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x68(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x70(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x78(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x80(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x88(%rsp)+        leaq    0x168(%rsp), %rsi+        leaq    0xd8(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_sqr_p521)+        leaq    0x48(%rsp), %rsi+        leaq    (%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_sqr_p521)+        leaq    0x120(%rsp), %rdx+        leaq    0xd8(%rsp), %rsi+        leaq    0x120(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        leaq    0x90(%rsp), %rdx+        leaq    0xd8(%rsp), %rsi+        leaq    0x90(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        movq    (%rsp), %rax+        subq    0x120(%rsp), %rax+        movq    0x8(%rsp), %rdx+        sbbq    0x128(%rsp), %rdx+        movq    0x10(%rsp), %r8+        sbbq    0x130(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0x138(%rsp), %r9+        movq    0x20(%rsp), %r10+        sbbq    0x140(%rsp), %r10+        movq    0x28(%rsp), %r11+        sbbq    0x148(%rsp), %r11+        movq    0x30(%rsp), %r12+        sbbq    0x150(%rsp), %r12+        movq    0x38(%rsp), %r13+        sbbq    0x158(%rsp), %r13+        movq    0x40(%rsp), %r14+        sbbq    0x160(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, (%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x8(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x18(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x20(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x28(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x30(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x38(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x40(%rsp)+        movq    0x90(%rsp), %rax+        subq    0x120(%rsp), %rax+        movq    0x98(%rsp), %rdx+        sbbq    0x128(%rsp), %rdx+        movq    0xa0(%rsp), %r8+        sbbq    0x130(%rsp), %r8+        movq    0xa8(%rsp), %r9+        sbbq    0x138(%rsp), %r9+        movq    0xb0(%rsp), %r10+        sbbq    0x140(%rsp), %r10+        movq    0xb8(%rsp), %r11+        sbbq    0x148(%rsp), %r11+        movq    0xc0(%rsp), %r12+        sbbq    0x150(%rsp), %r12+        movq    0xc8(%rsp), %r13+        sbbq    0x158(%rsp), %r13+        movq    0xd0(%rsp), %r14+        sbbq    0x160(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0xd8(%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0xe0(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0xe8(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0xf0(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0xf8(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x100(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x108(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x110(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x118(%rsp)+        movq    0x200(%rsp), %rsi+        leaq    0x90(%rsi), %rdx+        leaq    0x168(%rsp), %rsi+        leaq    0x168(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        movq    (%rsp), %rax+        subq    0x90(%rsp), %rax+        movq    0x8(%rsp), %rdx+        sbbq    0x98(%rsp), %rdx+        movq    0x10(%rsp), %r8+        sbbq    0xa0(%rsp), %r8+        movq    0x18(%rsp), %r9+        sbbq    0xa8(%rsp), %r9+        movq    0x20(%rsp), %r10+        sbbq    0xb0(%rsp), %r10+        movq    0x28(%rsp), %r11+        sbbq    0xb8(%rsp), %r11+        movq    0x30(%rsp), %r12+        sbbq    0xc0(%rsp), %r12+        movq    0x38(%rsp), %r13+        sbbq    0xc8(%rsp), %r13+        movq    0x40(%rsp), %r14+        sbbq    0xd0(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, (%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x8(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x10(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x18(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x20(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x28(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x30(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x38(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x40(%rsp)+        movq    0x120(%rsp), %rax+        subq    (%rsp), %rax+        movq    0x128(%rsp), %rdx+        sbbq    0x8(%rsp), %rdx+        movq    0x130(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x138(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movq    0x140(%rsp), %r10+        sbbq    0x20(%rsp), %r10+        movq    0x148(%rsp), %r11+        sbbq    0x28(%rsp), %r11+        movq    0x150(%rsp), %r12+        sbbq    0x30(%rsp), %r12+        movq    0x158(%rsp), %r13+        sbbq    0x38(%rsp), %r13+        movq    0x160(%rsp), %r14+        sbbq    0x40(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0x120(%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x128(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x130(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x138(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x140(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x148(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x150(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x158(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x160(%rsp)+        leaq    0x1b0(%rsp), %rdx+        leaq    0xd8(%rsp), %rsi+        leaq    0xd8(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        movq    0x208(%rsp), %rdi+        leaq    0x90(%rdi), %rdx+        leaq    0x168(%rsp), %rsi+        leaq    0x168(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        leaq    0x120(%rsp), %rdx+        leaq    0x48(%rsp), %rsi+        leaq    0x120(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        movq    0x120(%rsp), %rax+        subq    0xd8(%rsp), %rax+        movq    0x128(%rsp), %rdx+        sbbq    0xe0(%rsp), %rdx+        movq    0x130(%rsp), %r8+        sbbq    0xe8(%rsp), %r8+        movq    0x138(%rsp), %r9+        sbbq    0xf0(%rsp), %r9+        movq    0x140(%rsp), %r10+        sbbq    0xf8(%rsp), %r10+        movq    0x148(%rsp), %r11+        sbbq    0x100(%rsp), %r11+        movq    0x150(%rsp), %r12+        sbbq    0x108(%rsp), %r12+        movq    0x158(%rsp), %r13+        sbbq    0x110(%rsp), %r13+        movq    0x160(%rsp), %r14+        sbbq    0x118(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0x120(%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x128(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x130(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x138(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x140(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x148(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x150(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x158(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x160(%rsp)+        movq    0x200(%rsp), %rsi+        movq    0x90(%rsi), %r8+        movq    0x98(%rsi), %r9+        movq    0xa0(%rsi), %r10+        movq    0xa8(%rsi), %r11+        movq    0xb0(%rsi), %r12+        movq    0xb8(%rsi), %r13+        movq    0xc0(%rsi), %r14+        movq    0xc8(%rsi), %r15+        movq    0xd0(%rsi), %rbp+        orq     %r9, %r8+        orq     %r11, %r10+        orq     %r13, %r12+        orq     %r15, %r14+        orq     %r10, %r8+        orq     %r14, %r12+        orq     %rbp, %r8+        orq     %r12, %r8+        negq    %r8+        sbbq    %rax, %rax+        movq    0x208(%rsp), %rdi+        movq    0x90(%rdi), %r8+        movq    0x98(%rdi), %r9+        movq    0xa0(%rdi), %r10+        movq    0xa8(%rdi), %r11+        movq    0xb0(%rdi), %r12+        movq    0xb8(%rdi), %r13+        movq    0xc0(%rdi), %r14+        movq    0xc8(%rdi), %r15+        movq    0xd0(%rdi), %rbp+        orq     %r9, %r8+        orq     %r11, %r10+        orq     %r13, %r12+        orq     %r15, %r14+        orq     %r10, %r8+        orq     %r14, %r12+        orq     %rbp, %r8+        orq     %r12, %r8+        negq    %r8+        sbbq    %rdx, %rdx+        cmpq    %rax, %rdx+        movq    0x120(%rsp), %r8+        cmovbq  0x48(%rsi), %r8+        cmova   0x48(%rdi), %r8+        movq    0x128(%rsp), %r9+        cmovbq  0x50(%rsi), %r9+        cmova   0x50(%rdi), %r9+        movq    0x130(%rsp), %r10+        cmovbq  0x58(%rsi), %r10+        cmova   0x58(%rdi), %r10+        movq    0x138(%rsp), %r11+        cmovbq  0x60(%rsi), %r11+        cmova   0x60(%rdi), %r11+        movq    0x140(%rsp), %r12+        cmovbq  0x68(%rsi), %r12+        cmova   0x68(%rdi), %r12+        movq    0x148(%rsp), %r13+        cmovbq  0x70(%rsi), %r13+        cmova   0x70(%rdi), %r13+        movq    0x150(%rsp), %r14+        cmovbq  0x78(%rsi), %r14+        cmova   0x78(%rdi), %r14+        movq    0x158(%rsp), %r15+        cmovbq  0x80(%rsi), %r15+        cmova   0x80(%rdi), %r15+        movq    0x160(%rsp), %rbp+        cmovbq  0x88(%rsi), %rbp+        cmova   0x88(%rdi), %rbp+        movq    %r8, 0x120(%rsp)+        movq    %r9, 0x128(%rsp)+        movq    %r10, 0x130(%rsp)+        movq    %r11, 0x138(%rsp)+        movq    %r12, 0x140(%rsp)+        movq    %r13, 0x148(%rsp)+        movq    %r14, 0x150(%rsp)+        movq    %r15, 0x158(%rsp)+        movq    %rbp, 0x160(%rsp)+        movq    0x168(%rsp), %r8+        cmovbq  0x90(%rsi), %r8+        cmova   0x90(%rdi), %r8+        movq    0x170(%rsp), %r9+        cmovbq  0x98(%rsi), %r9+        cmova   0x98(%rdi), %r9+        movq    0x178(%rsp), %r10+        cmovbq  0xa0(%rsi), %r10+        cmova   0xa0(%rdi), %r10+        movq    0x180(%rsp), %r11+        cmovbq  0xa8(%rsi), %r11+        cmova   0xa8(%rdi), %r11+        movq    0x188(%rsp), %r12+        cmovbq  0xb0(%rsi), %r12+        cmova   0xb0(%rdi), %r12+        movq    0x190(%rsp), %r13+        cmovbq  0xb8(%rsi), %r13+        cmova   0xb8(%rdi), %r13+        movq    0x198(%rsp), %r14+        cmovbq  0xc0(%rsi), %r14+        cmova   0xc0(%rdi), %r14+        movq    0x1a0(%rsp), %r15+        cmovbq  0xc8(%rsi), %r15+        cmova   0xc8(%rdi), %r15+        movq    0x1a8(%rsp), %rbp+        cmovbq  0xd0(%rsi), %rbp+        cmova   0xd0(%rdi), %rbp+        movq    %r8, 0x168(%rsp)+        movq    %r9, 0x170(%rsp)+        movq    %r10, 0x178(%rsp)+        movq    %r11, 0x180(%rsp)+        movq    %r12, 0x188(%rsp)+        movq    %r13, 0x190(%rsp)+        movq    %r14, 0x198(%rsp)+        movq    %r15, 0x1a0(%rsp)+        movq    %rbp, 0x1a8(%rsp)+        movq    (%rsp), %r8+        cmovbq  (%rsi), %r8+        cmova   (%rdi), %r8+        movq    0x8(%rsp), %r9+        cmovbq  0x8(%rsi), %r9+        cmova   0x8(%rdi), %r9+        movq    0x10(%rsp), %r10+        cmovbq  0x10(%rsi), %r10+        cmova   0x10(%rdi), %r10+        movq    0x18(%rsp), %r11+        cmovbq  0x18(%rsi), %r11+        cmova   0x18(%rdi), %r11+        movq    0x20(%rsp), %r12+        cmovbq  0x20(%rsi), %r12+        cmova   0x20(%rdi), %r12+        movq    0x28(%rsp), %r13+        cmovbq  0x28(%rsi), %r13+        cmova   0x28(%rdi), %r13+        movq    0x30(%rsp), %r14+        cmovbq  0x30(%rsi), %r14+        cmova   0x30(%rdi), %r14+        movq    0x38(%rsp), %r15+        cmovbq  0x38(%rsi), %r15+        cmova   0x38(%rdi), %r15+        movq    0x40(%rsp), %rbp+        cmovbq  0x40(%rsi), %rbp+        cmova   0x40(%rdi), %rbp+        movq    0x1f8(%rsp), %rdi+        movq    %r8, (%rdi)+        movq    %r9, 0x8(%rdi)+        movq    %r10, 0x10(%rdi)+        movq    %r11, 0x18(%rdi)+        movq    %r12, 0x20(%rdi)+        movq    %r13, 0x28(%rdi)+        movq    %r14, 0x30(%rdi)+        movq    %r15, 0x38(%rdi)+        movq    %rbp, 0x40(%rdi)+        movq    0x120(%rsp), %rax+        movq    %rax, 0x48(%rdi)+        movq    0x128(%rsp), %rax+        movq    %rax, 0x50(%rdi)+        movq    0x130(%rsp), %rax+        movq    %rax, 0x58(%rdi)+        movq    0x138(%rsp), %rax+        movq    %rax, 0x60(%rdi)+        movq    0x140(%rsp), %rax+        movq    %rax, 0x68(%rdi)+        movq    0x148(%rsp), %rax+        movq    %rax, 0x70(%rdi)+        movq    0x150(%rsp), %rax+        movq    %rax, 0x78(%rdi)+        movq    0x158(%rsp), %rax+        movq    %rax, 0x80(%rdi)+        movq    0x160(%rsp), %rax+        movq    %rax, 0x88(%rdi)+        movq    0x168(%rsp), %rax+        movq    %rax, 0x90(%rdi)+        movq    0x170(%rsp), %rax+        movq    %rax, 0x98(%rdi)+        movq    0x178(%rsp), %rax+        movq    %rax, 0xa0(%rdi)+        movq    0x180(%rsp), %rax+        movq    %rax, 0xa8(%rdi)+        movq    0x188(%rsp), %rax+        movq    %rax, 0xb0(%rdi)+        movq    0x190(%rsp), %rax+        movq    %rax, 0xb8(%rdi)+        movq    0x198(%rsp), %rax+        movq    %rax, 0xc0(%rdi)+        movq    0x1a0(%rsp), %rax+        movq    %rax, 0xc8(%rdi)+        movq    0x1a8(%rsp), %rax+        movq    %rax, 0xd0(%rdi)+        CFI_INC_RSP(528)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_alt_jadd)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_alt_jdouble)++Lp521_jscalarmul_alt_jdouble:+        CFI_START+        CFI_PUSH(%rbx)+        CFI_PUSH(%rbp)+        CFI_PUSH(%r12)+        CFI_PUSH(%r13)+        CFI_PUSH(%r14)+        CFI_PUSH(%r15)+        CFI_DEC_RSP(520)+        movq    %rdi, 0x1f8(%rsp)+        movq    %rsi, 0x200(%rsp)+        movq    0x200(%rsp), %rdi+        leaq    0x90(%rdi), %rsi+        leaq    (%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_sqr_p521)+        movq    0x200(%rsp), %rdi+        leaq    0x48(%rdi), %rsi+        leaq    0x48(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_sqr_p521)+        movq    0x200(%rsp), %rdi+        stc+        movq    (%rdi), %rax+        adcq    (%rsp), %rax+        movq    0x8(%rdi), %rbx+        adcq    0x8(%rsp), %rbx+        movq    0x10(%rdi), %r8+        adcq    0x10(%rsp), %r8+        movq    0x18(%rdi), %r9+        adcq    0x18(%rsp), %r9+        movq    0x20(%rdi), %r10+        adcq    0x20(%rsp), %r10+        movq    0x28(%rdi), %r11+        adcq    0x28(%rsp), %r11+        movq    0x30(%rdi), %r12+        adcq    0x30(%rsp), %r12+        movq    0x38(%rdi), %r13+        adcq    0x38(%rsp), %r13+        movq    0x40(%rdi), %r14+        adcq    0x40(%rsp), %r14+        movq    $0x200, %rdx+        andq    %r14, %rdx+        cmpq    $0x200, %rdx+        sbbq    $0x0, %rax+        movq    %rax, 0x168(%rsp)+        sbbq    $0x0, %rbx+        movq    %rbx, 0x170(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x178(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x180(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x188(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x190(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x198(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x1a0(%rsp)+        sbbq    %rdx, %r14+        movq    %r14, 0x1a8(%rsp)+        movq    0x200(%rsp), %rdi+        movq    (%rdi), %rax+        subq    (%rsp), %rax+        movq    0x8(%rdi), %rdx+        sbbq    0x8(%rsp), %rdx+        movq    0x10(%rdi), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x18(%rdi), %r9+        sbbq    0x18(%rsp), %r9+        movq    0x20(%rdi), %r10+        sbbq    0x20(%rsp), %r10+        movq    0x28(%rdi), %r11+        sbbq    0x28(%rsp), %r11+        movq    0x30(%rdi), %r12+        sbbq    0x30(%rsp), %r12+        movq    0x38(%rdi), %r13+        sbbq    0x38(%rsp), %r13+        movq    0x40(%rdi), %r14+        sbbq    0x40(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0x120(%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x128(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x130(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x138(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x140(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x148(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x150(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x158(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x160(%rsp)+        leaq    0x120(%rsp), %rdx+        leaq    0x168(%rsp), %rsi+        leaq    0x90(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        movq    0x200(%rsp), %rdi+        stc+        movq    0x48(%rdi), %rax+        adcq    0x90(%rdi), %rax+        movq    0x50(%rdi), %rbx+        adcq    0x98(%rdi), %rbx+        movq    0x58(%rdi), %r8+        adcq    0xa0(%rdi), %r8+        movq    0x60(%rdi), %r9+        adcq    0xa8(%rdi), %r9+        movq    0x68(%rdi), %r10+        adcq    0xb0(%rdi), %r10+        movq    0x70(%rdi), %r11+        adcq    0xb8(%rdi), %r11+        movq    0x78(%rdi), %r12+        adcq    0xc0(%rdi), %r12+        movq    0x80(%rdi), %r13+        adcq    0xc8(%rdi), %r13+        movq    0x88(%rdi), %r14+        adcq    0xd0(%rdi), %r14+        movq    $0x200, %rdx+        andq    %r14, %rdx+        cmpq    $0x200, %rdx+        sbbq    $0x0, %rax+        movq    %rax, 0x168(%rsp)+        sbbq    $0x0, %rbx+        movq    %rbx, 0x170(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x178(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x180(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x188(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x190(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x198(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x1a0(%rsp)+        sbbq    %rdx, %r14+        movq    %r14, 0x1a8(%rsp)+        leaq    0x90(%rsp), %rsi+        leaq    0x1b0(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_sqr_p521)+        movq    0x200(%rsp), %rdi+        leaq    0x48(%rsp), %rdx+        leaq    (%rdi), %rsi+        leaq    0xd8(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        leaq    0x168(%rsp), %rsi+        leaq    0x120(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_sqr_p521)+        movq    $0x9, %rcx+        movq    0x1b0(%rsp), %rax+        notq    %rax+        mulq    %rcx+        movq    %rax, %r8+        movq    %rdx, %r9+        movq    0x1b8(%rsp), %rax+        notq    %rax+        mulq    %rcx+        xorl    %r10d, %r10d+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x1c0(%rsp), %rax+        notq    %rax+        mulq    %rcx+        xorl    %r11d, %r11d+        addq    %rax, %r10+        adcq    %rdx, %r11+        movq    0x1c8(%rsp), %rax+        notq    %rax+        mulq    %rcx+        xorl    %r12d, %r12d+        addq    %rax, %r11+        adcq    %rdx, %r12+        movq    0x1d0(%rsp), %rax+        notq    %rax+        mulq    %rcx+        xorl    %r13d, %r13d+        addq    %rax, %r12+        adcq    %rdx, %r13+        movq    0x1d8(%rsp), %rax+        notq    %rax+        mulq    %rcx+        xorl    %r14d, %r14d+        addq    %rax, %r13+        adcq    %rdx, %r14+        movq    0x1e0(%rsp), %rax+        notq    %rax+        mulq    %rcx+        xorl    %r15d, %r15d+        addq    %rax, %r14+        adcq    %rdx, %r15+        movq    0x1e8(%rsp), %rax+        notq    %rax+        mulq    %rcx+        xorl    %ebx, %ebx+        addq    %rax, %r15+        adcq    %rdx, %rbx+        movq    0x1f0(%rsp), %rax+        xorq    $0x1ff, %rax+        imulq   %rcx, %rax+        addq    %rax, %rbx+        xorl    %eax, %eax+        movl    $0xc, %ecx+        movq    0xd8(%rsp), %rax+        mulq    %rcx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %rbp, %rbp+        movq    0xe0(%rsp), %rax+        mulq    %rcx+        subq    %rbp, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rbp, %rbp+        movq    0xe8(%rsp), %rax+        mulq    %rcx+        subq    %rbp, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0xf0(%rsp), %rax+        mulq    %rcx+        subq    %rbp, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rbp, %rbp+        movq    0xf8(%rsp), %rax+        mulq    %rcx+        subq    %rbp, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x100(%rsp), %rax+        mulq    %rcx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x108(%rsp), %rax+        mulq    %rcx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x110(%rsp), %rax+        mulq    %rcx+        subq    %rbp, %rdx+        addq    %rax, %r15+        adcq    %rdx, %rbx+        movq    0x118(%rsp), %rax+        imulq   %rcx, %rax+        addq    %rax, %rbx+        movq    %r9, %rax+        andq    %r10, %rax+        andq    %r11, %rax+        andq    %r12, %rax+        andq    %r13, %rax+        andq    %r14, %rax+        andq    %r15, %rax+        movq    %rbx, %rdx+        shrq    $0x9, %rdx+        orq     $0xfffffffffffffe00, %rbx+        leaq    0x1(%rdx), %rcx+        addq    %r8, %rcx+        movl    $0x0, %ecx+        adcq    %rcx, %rax+        movq    %rbx, %rax+        adcq    %rcx, %rax+        adcq    %rdx, %r8+        movq    %r8, 0x1b0(%rsp)+        adcq    %rcx, %r9+        movq    %r9, 0x1b8(%rsp)+        adcq    %rcx, %r10+        movq    %r10, 0x1c0(%rsp)+        adcq    %rcx, %r11+        movq    %r11, 0x1c8(%rsp)+        adcq    %rcx, %r12+        movq    %r12, 0x1d0(%rsp)+        adcq    %rcx, %r13+        movq    %r13, 0x1d8(%rsp)+        adcq    %rcx, %r14+        movq    %r14, 0x1e0(%rsp)+        adcq    %rcx, %r15+        movq    %r15, 0x1e8(%rsp)+        adcq    %rcx, %rbx+        andq    $0x1ff, %rbx+        movq    %rbx, 0x1f0(%rsp)+        movq    0x120(%rsp), %rax+        subq    (%rsp), %rax+        movq    0x128(%rsp), %rdx+        sbbq    0x8(%rsp), %rdx+        movq    0x130(%rsp), %r8+        sbbq    0x10(%rsp), %r8+        movq    0x138(%rsp), %r9+        sbbq    0x18(%rsp), %r9+        movq    0x140(%rsp), %r10+        sbbq    0x20(%rsp), %r10+        movq    0x148(%rsp), %r11+        sbbq    0x28(%rsp), %r11+        movq    0x150(%rsp), %r12+        sbbq    0x30(%rsp), %r12+        movq    0x158(%rsp), %r13+        sbbq    0x38(%rsp), %r13+        movq    0x160(%rsp), %r14+        sbbq    0x40(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0x168(%rsp)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x170(%rsp)+        sbbq    $0x0, %r8+        movq    %r8, 0x178(%rsp)+        sbbq    $0x0, %r9+        movq    %r9, 0x180(%rsp)+        sbbq    $0x0, %r10+        movq    %r10, 0x188(%rsp)+        sbbq    $0x0, %r11+        movq    %r11, 0x190(%rsp)+        sbbq    $0x0, %r12+        movq    %r12, 0x198(%rsp)+        sbbq    $0x0, %r13+        movq    %r13, 0x1a0(%rsp)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0x1a8(%rsp)+        leaq    0x48(%rsp), %rsi+        leaq    0x120(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_sqr_p521)+        movq    0x1f8(%rsp), %rdi+        movq    0x168(%rsp), %rax+        subq    0x48(%rsp), %rax+        movq    0x170(%rsp), %rdx+        sbbq    0x50(%rsp), %rdx+        movq    0x178(%rsp), %r8+        sbbq    0x58(%rsp), %r8+        movq    0x180(%rsp), %r9+        sbbq    0x60(%rsp), %r9+        movq    0x188(%rsp), %r10+        sbbq    0x68(%rsp), %r10+        movq    0x190(%rsp), %r11+        sbbq    0x70(%rsp), %r11+        movq    0x198(%rsp), %r12+        sbbq    0x78(%rsp), %r12+        movq    0x1a0(%rsp), %r13+        sbbq    0x80(%rsp), %r13+        movq    0x1a8(%rsp), %r14+        sbbq    0x88(%rsp), %r14+        sbbq    $0x0, %rax+        movq    %rax, 0x90(%rdi)+        sbbq    $0x0, %rdx+        movq    %rdx, 0x98(%rdi)+        sbbq    $0x0, %r8+        movq    %r8, 0xa0(%rdi)+        sbbq    $0x0, %r9+        movq    %r9, 0xa8(%rdi)+        sbbq    $0x0, %r10+        movq    %r10, 0xb0(%rdi)+        sbbq    $0x0, %r11+        movq    %r11, 0xb8(%rdi)+        sbbq    $0x0, %r12+        movq    %r12, 0xc0(%rdi)+        sbbq    $0x0, %r13+        movq    %r13, 0xc8(%rdi)+        sbbq    $0x0, %r14+        andq    $0x1ff, %r14+        movq    %r14, 0xd0(%rdi)+        leaq    0x90(%rsp), %rdx+        leaq    0x1b0(%rsp), %rsi+        leaq    0x168(%rsp), %rdi+        CFI_CALL(Lp521_jscalarmul_alt_mul_p521)+        movq    0x1f8(%rsp), %rdi+        movq    0x118(%rsp), %rbx+        movq    0x110(%rsp), %r15+        shldq   $0x2, %r15, %rbx+        movq    0x108(%rsp), %r14+        shldq   $0x2, %r14, %r15+        movq    0x100(%rsp), %r13+        shldq   $0x2, %r13, %r14+        movq    0xf8(%rsp), %r12+        shldq   $0x2, %r12, %r13+        movq    0xf0(%rsp), %r11+        shldq   $0x2, %r11, %r12+        movq    0xe8(%rsp), %r10+        shldq   $0x2, %r10, %r11+        movq    0xe0(%rsp), %r9+        shldq   $0x2, %r9, %r10+        movq    0xd8(%rsp), %r8+        shldq   $0x2, %r8, %r9+        shlq    $0x2, %r8+        movq    0x1f0(%rsp), %rcx+        xorq    $0x1ff, %rcx+        movq    0x1b0(%rsp), %rax+        notq    %rax+        addq    %rax, %r8+        movq    0x1b8(%rsp), %rax+        notq    %rax+        adcq    %rax, %r9+        movq    0x1c0(%rsp), %rax+        notq    %rax+        adcq    %rax, %r10+        movq    0x1c8(%rsp), %rax+        notq    %rax+        adcq    %rax, %r11+        movq    0x1d0(%rsp), %rax+        notq    %rax+        adcq    %rax, %r12+        movq    0x1d8(%rsp), %rax+        notq    %rax+        adcq    %rax, %r13+        movq    0x1e0(%rsp), %rax+        notq    %rax+        adcq    %rax, %r14+        movq    0x1e8(%rsp), %rax+        notq    %rax+        adcq    %rax, %r15+        adcq    %rcx, %rbx+        movq    %r9, %rax+        andq    %r10, %rax+        andq    %r11, %rax+        andq    %r12, %rax+        andq    %r13, %rax+        andq    %r14, %rax+        andq    %r15, %rax+        movq    %rbx, %rdx+        shrq    $0x9, %rdx+        orq     $0xfffffffffffffe00, %rbx+        leaq    0x1(%rdx), %rcx+        addq    %r8, %rcx+        movl    $0x0, %ecx+        adcq    %rcx, %rax+        movq    %rbx, %rax+        adcq    %rcx, %rax+        adcq    %rdx, %r8+        movq    %r8, (%rdi)+        adcq    %rcx, %r9+        movq    %r9, 0x8(%rdi)+        adcq    %rcx, %r10+        movq    %r10, 0x10(%rdi)+        adcq    %rcx, %r11+        movq    %r11, 0x18(%rdi)+        adcq    %rcx, %r12+        movq    %r12, 0x20(%rdi)+        adcq    %rcx, %r13+        movq    %r13, 0x28(%rdi)+        adcq    %rcx, %r14+        movq    %r14, 0x30(%rdi)+        adcq    %rcx, %r15+        movq    %r15, 0x38(%rdi)+        adcq    %rcx, %rbx+        andq    $0x1ff, %rbx+        movq    %rbx, 0x40(%rdi)+        movq    0x1f8(%rsp), %rdi+        movq    0x160(%rsp), %rbx+        xorq    $0x1ff, %rbx+        movq    0x158(%rsp), %r15+        notq    %r15+        shldq   $0x3, %r15, %rbx+        movq    0x150(%rsp), %r14+        notq    %r14+        shldq   $0x3, %r14, %r15+        movq    0x148(%rsp), %r13+        notq    %r13+        shldq   $0x3, %r13, %r14+        movq    0x140(%rsp), %r12+        notq    %r12+        shldq   $0x3, %r12, %r13+        movq    0x138(%rsp), %r11+        notq    %r11+        shldq   $0x3, %r11, %r12+        movq    0x130(%rsp), %r10+        notq    %r10+        shldq   $0x3, %r10, %r11+        movq    0x128(%rsp), %r9+        notq    %r9+        shldq   $0x3, %r9, %r10+        movq    0x120(%rsp), %r8+        notq    %r8+        shldq   $0x3, %r8, %r9+        shlq    $0x3, %r8+        movl    $0x3, %ecx+        movq    0x168(%rsp), %rax+        mulq    %rcx+        addq    %rax, %r8+        adcq    %rdx, %r9+        sbbq    %rbp, %rbp+        movq    0x170(%rsp), %rax+        mulq    %rcx+        subq    %rbp, %rdx+        addq    %rax, %r9+        adcq    %rdx, %r10+        sbbq    %rbp, %rbp+        movq    0x178(%rsp), %rax+        mulq    %rcx+        subq    %rbp, %rdx+        addq    %rax, %r10+        adcq    %rdx, %r11+        sbbq    %rbp, %rbp+        movq    0x180(%rsp), %rax+        mulq    %rcx+        subq    %rbp, %rdx+        addq    %rax, %r11+        adcq    %rdx, %r12+        sbbq    %rbp, %rbp+        movq    0x188(%rsp), %rax+        mulq    %rcx+        subq    %rbp, %rdx+        addq    %rax, %r12+        adcq    %rdx, %r13+        sbbq    %rbp, %rbp+        movq    0x190(%rsp), %rax+        mulq    %rcx+        subq    %rbp, %rdx+        addq    %rax, %r13+        adcq    %rdx, %r14+        sbbq    %rbp, %rbp+        movq    0x198(%rsp), %rax+        mulq    %rcx+        subq    %rbp, %rdx+        addq    %rax, %r14+        adcq    %rdx, %r15+        sbbq    %rbp, %rbp+        movq    0x1a0(%rsp), %rax+        mulq    %rcx+        subq    %rbp, %rdx+        addq    %rax, %r15+        adcq    %rdx, %rbx+        movq    0x1a8(%rsp), %rax+        imulq   %rcx, %rax+        addq    %rax, %rbx+        movq    %r9, %rax+        andq    %r10, %rax+        andq    %r11, %rax+        andq    %r12, %rax+        andq    %r13, %rax+        andq    %r14, %rax+        andq    %r15, %rax+        movq    %rbx, %rdx+        shrq    $0x9, %rdx+        orq     $0xfffffffffffffe00, %rbx+        leaq    0x1(%rdx), %rcx+        addq    %r8, %rcx+        movl    $0x0, %ecx+        adcq    %rcx, %rax+        movq    %rbx, %rax+        adcq    %rcx, %rax+        adcq    %rdx, %r8+        movq    %r8, 0x48(%rdi)+        adcq    %rcx, %r9+        movq    %r9, 0x50(%rdi)+        adcq    %rcx, %r10+        movq    %r10, 0x58(%rdi)+        adcq    %rcx, %r11+        movq    %r11, 0x60(%rdi)+        adcq    %rcx, %r12+        movq    %r12, 0x68(%rdi)+        adcq    %rcx, %r13+        movq    %r13, 0x70(%rdi)+        adcq    %rcx, %r14+        movq    %r14, 0x78(%rdi)+        adcq    %rcx, %r15+        movq    %r15, 0x80(%rdi)+        adcq    %rcx, %rbx+        andq    $0x1ff, %rbx+        movq    %rbx, 0x88(%rdi)+        CFI_INC_RSP(520)+        CFI_POP(%r15)+        CFI_POP(%r14)+        CFI_POP(%r13)+        CFI_POP(%r12)+        CFI_POP(%rbp)+        CFI_POP(%rbx)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_alt_jdouble)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_alt_mul_p521)++Lp521_jscalarmul_alt_mul_p521:+        CFI_START+        CFI_DEC_RSP(72)+        movq    %rdx, %rcx+        movq    (%rsi), %rax+        mulq     (%rcx)+        movq    %rax, (%rsp)+        movq    %rdx, %r9+        xorq    %r10, %r10+        xorq    %r11, %r11+        movq    (%rsi), %rax+        mulq     0x8(%rcx)+        addq    %rax, %r9+        adcq    %rdx, %r10+        movq    0x8(%rsi), %rax+        mulq     (%rcx)+        addq    %rax, %r9+        adcq    %rdx, %r10+        adcq    %r11, %r11+        movq    %r9, 0x8(%rsp)+        xorq    %r12, %r12+        movq    (%rsi), %rax+        mulq     0x10(%rcx)+        addq    %rax, %r10+        adcq    %rdx, %r11+        adcq    %r12, %r12+        movq    0x8(%rsi), %rax+        mulq     0x8(%rcx)+        addq    %rax, %r10+        adcq    %rdx, %r11+        adcq    $0x0, %r12+        movq    0x10(%rsi), %rax+        mulq     (%rcx)+        addq    %rax, %r10+        adcq    %rdx, %r11+        adcq    $0x0, %r12+        movq    %r10, 0x10(%rsp)+        xorq    %r13, %r13+        movq    (%rsi), %rax+        mulq     0x18(%rcx)+        addq    %rax, %r11+        adcq    %rdx, %r12+        adcq    %r13, %r13+        movq    0x8(%rsi), %rax+        mulq     0x10(%rcx)+        addq    %rax, %r11+        adcq    %rdx, %r12+        adcq    $0x0, %r13+        movq    0x10(%rsi), %rax+        mulq     0x8(%rcx)+        addq    %rax, %r11+        adcq    %rdx, %r12+        adcq    $0x0, %r13+        movq    0x18(%rsi), %rax+        mulq     (%rcx)+        addq    %rax, %r11+        adcq    %rdx, %r12+        adcq    $0x0, %r13+        movq    %r11, 0x18(%rsp)+        xorq    %r14, %r14+        movq    (%rsi), %rax+        mulq     0x20(%rcx)+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x8(%rsi), %rax+        mulq     0x18(%rcx)+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        movq    0x10(%rsi), %rax+        mulq     0x10(%rcx)+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        movq    0x18(%rsi), %rax+        mulq     0x8(%rcx)+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        movq    0x20(%rsi), %rax+        mulq     (%rcx)+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        movq    %r12, 0x20(%rsp)+        xorq    %r15, %r15+        movq    (%rsi), %rax+        mulq     0x28(%rcx)+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x8(%rsi), %rax+        mulq     0x20(%rcx)+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x10(%rsi), %rax+        mulq     0x18(%rcx)+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x18(%rsi), %rax+        mulq     0x10(%rcx)+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x20(%rsi), %rax+        mulq     0x8(%rcx)+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x28(%rsi), %rax+        mulq     (%rcx)+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    $0x0, %r15+        movq    %r13, 0x28(%rsp)+        xorq    %r8, %r8+        movq    (%rsi), %rax+        mulq     0x30(%rcx)+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        movq    0x8(%rsi), %rax+        mulq     0x28(%rcx)+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x10(%rsi), %rax+        mulq     0x20(%rcx)+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x18(%rsi), %rax+        mulq     0x18(%rcx)+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x20(%rsi), %rax+        mulq     0x10(%rcx)+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x28(%rsi), %rax+        mulq     0x8(%rcx)+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x30(%rsi), %rax+        mulq     (%rcx)+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %r8+        movq    %r14, 0x30(%rsp)+        xorq    %r9, %r9+        movq    (%rsi), %rax+        mulq     0x38(%rcx)+        addq    %rax, %r15+        adcq    %rdx, %r8+        adcq    %r9, %r9+        movq    0x8(%rsi), %rax+        mulq     0x30(%rcx)+        addq    %rax, %r15+        adcq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x10(%rsi), %rax+        mulq     0x28(%rcx)+        addq    %rax, %r15+        adcq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x18(%rsi), %rax+        mulq     0x20(%rcx)+        addq    %rax, %r15+        adcq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x20(%rsi), %rax+        mulq     0x18(%rcx)+        addq    %rax, %r15+        adcq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x28(%rsi), %rax+        mulq     0x10(%rcx)+        addq    %rax, %r15+        adcq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x30(%rsi), %rax+        mulq     0x8(%rcx)+        addq    %rax, %r15+        adcq    %rdx, %r8+        adcq    $0x0, %r9+        movq    0x38(%rsi), %rax+        mulq     (%rcx)+        addq    %rax, %r15+        adcq    %rdx, %r8+        adcq    $0x0, %r9+        movq    %r15, 0x38(%rsp)+        xorq    %r10, %r10+        movq    (%rsi), %rax+        mulq     0x40(%rcx)+        addq    %rax, %r8+        adcq    %rdx, %r9+        adcq    %r10, %r10+        movq    0x8(%rsi), %rax+        mulq     0x38(%rcx)+        addq    %rax, %r8+        adcq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x10(%rsi), %rax+        mulq     0x30(%rcx)+        addq    %rax, %r8+        adcq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x18(%rsi), %rax+        mulq     0x28(%rcx)+        addq    %rax, %r8+        adcq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x20(%rsi), %rax+        mulq     0x20(%rcx)+        addq    %rax, %r8+        adcq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x28(%rsi), %rax+        mulq     0x18(%rcx)+        addq    %rax, %r8+        adcq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x30(%rsi), %rax+        mulq     0x10(%rcx)+        addq    %rax, %r8+        adcq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x38(%rsi), %rax+        mulq     0x8(%rcx)+        addq    %rax, %r8+        adcq    %rdx, %r9+        adcq    $0x0, %r10+        movq    0x40(%rsi), %rax+        mulq     (%rcx)+        addq    %rax, %r8+        adcq    %rdx, %r9+        adcq    $0x0, %r10+        movq    %r8, 0x40(%rsp)+        xorq    %r11, %r11+        movq    0x8(%rsi), %rax+        mulq     0x40(%rcx)+        addq    %rax, %r9+        adcq    %rdx, %r10+        adcq    %r11, %r11+        movq    0x10(%rsi), %rax+        mulq     0x38(%rcx)+        addq    %rax, %r9+        adcq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x18(%rsi), %rax+        mulq     0x30(%rcx)+        addq    %rax, %r9+        adcq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x20(%rsi), %rax+        mulq     0x28(%rcx)+        addq    %rax, %r9+        adcq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x28(%rsi), %rax+        mulq     0x20(%rcx)+        addq    %rax, %r9+        adcq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x30(%rsi), %rax+        mulq     0x18(%rcx)+        addq    %rax, %r9+        adcq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x38(%rsi), %rax+        mulq     0x10(%rcx)+        addq    %rax, %r9+        adcq    %rdx, %r10+        adcq    $0x0, %r11+        movq    0x40(%rsi), %rax+        mulq     0x8(%rcx)+        addq    %rax, %r9+        adcq    %rdx, %r10+        adcq    $0x0, %r11+        xorq    %r12, %r12+        movq    0x10(%rsi), %rax+        mulq     0x40(%rcx)+        addq    %rax, %r10+        adcq    %rdx, %r11+        adcq    %r12, %r12+        movq    0x18(%rsi), %rax+        mulq     0x38(%rcx)+        addq    %rax, %r10+        adcq    %rdx, %r11+        adcq    $0x0, %r12+        movq    0x20(%rsi), %rax+        mulq     0x30(%rcx)+        addq    %rax, %r10+        adcq    %rdx, %r11+        adcq    $0x0, %r12+        movq    0x28(%rsi), %rax+        mulq     0x28(%rcx)+        addq    %rax, %r10+        adcq    %rdx, %r11+        adcq    $0x0, %r12+        movq    0x30(%rsi), %rax+        mulq     0x20(%rcx)+        addq    %rax, %r10+        adcq    %rdx, %r11+        adcq    $0x0, %r12+        movq    0x38(%rsi), %rax+        mulq     0x18(%rcx)+        addq    %rax, %r10+        adcq    %rdx, %r11+        adcq    $0x0, %r12+        movq    0x40(%rsi), %rax+        mulq     0x10(%rcx)+        addq    %rax, %r10+        adcq    %rdx, %r11+        adcq    $0x0, %r12+        xorq    %r13, %r13+        movq    0x18(%rsi), %rax+        mulq     0x40(%rcx)+        addq    %rax, %r11+        adcq    %rdx, %r12+        adcq    %r13, %r13+        movq    0x20(%rsi), %rax+        mulq     0x38(%rcx)+        addq    %rax, %r11+        adcq    %rdx, %r12+        adcq    $0x0, %r13+        movq    0x28(%rsi), %rax+        mulq     0x30(%rcx)+        addq    %rax, %r11+        adcq    %rdx, %r12+        adcq    $0x0, %r13+        movq    0x30(%rsi), %rax+        mulq     0x28(%rcx)+        addq    %rax, %r11+        adcq    %rdx, %r12+        adcq    $0x0, %r13+        movq    0x38(%rsi), %rax+        mulq     0x20(%rcx)+        addq    %rax, %r11+        adcq    %rdx, %r12+        adcq    $0x0, %r13+        movq    0x40(%rsi), %rax+        mulq     0x18(%rcx)+        addq    %rax, %r11+        adcq    %rdx, %r12+        adcq    $0x0, %r13+        xorq    %r14, %r14+        movq    0x20(%rsi), %rax+        mulq     0x40(%rcx)+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    %r14, %r14+        movq    0x28(%rsi), %rax+        mulq     0x38(%rcx)+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        movq    0x30(%rsi), %rax+        mulq     0x30(%rcx)+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        movq    0x38(%rsi), %rax+        mulq     0x28(%rcx)+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        movq    0x40(%rsi), %rax+        mulq     0x20(%rcx)+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        xorq    %r15, %r15+        movq    0x28(%rsi), %rax+        mulq     0x40(%rcx)+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    %r15, %r15+        movq    0x30(%rsi), %rax+        mulq     0x38(%rcx)+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x38(%rsi), %rax+        mulq     0x30(%rcx)+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    $0x0, %r15+        movq    0x40(%rsi), %rax+        mulq     0x28(%rcx)+        addq    %rax, %r13+        adcq    %rdx, %r14+        adcq    $0x0, %r15+        xorq    %r8, %r8+        movq    0x30(%rsi), %rax+        mulq     0x40(%rcx)+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    %r8, %r8+        movq    0x38(%rsi), %rax+        mulq     0x38(%rcx)+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x40(%rsi), %rax+        mulq     0x30(%rcx)+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x38(%rsi), %rax+        mulq     0x40(%rcx)+        addq    %rax, %r15+        adcq    %rdx, %r8+        movq    0x40(%rsi), %rax+        mulq     0x38(%rcx)+        addq    %rax, %r15+        adcq    %rdx, %r8+        movq    0x40(%rsi), %rax+        imulq   0x40(%rcx), %rax+        addq    %r8, %rax+        movq    0x40(%rsp), %r8+        movq    %r8, %rdx+        andq    $0x1ff, %rdx+        shrdq   $0x9, %r9, %r8+        shrdq   $0x9, %r10, %r9+        shrdq   $0x9, %r11, %r10+        shrdq   $0x9, %r12, %r11+        shrdq   $0x9, %r13, %r12+        shrdq   $0x9, %r14, %r13+        shrdq   $0x9, %r15, %r14+        shrdq   $0x9, %rax, %r15+        shrq    $0x9, %rax+        addq    %rax, %rdx+        stc+        adcq    (%rsp), %r8+        adcq    0x8(%rsp), %r9+        adcq    0x10(%rsp), %r10+        adcq    0x18(%rsp), %r11+        adcq    0x20(%rsp), %r12+        adcq    0x28(%rsp), %r13+        adcq    0x30(%rsp), %r14+        adcq    0x38(%rsp), %r15+        adcq    $0xfffffffffffffe00, %rdx+        cmc+        sbbq    $0x0, %r8+        movq    %r8, (%rdi)+        sbbq    $0x0, %r9+        movq    %r9, 0x8(%rdi)+        sbbq    $0x0, %r10+        movq    %r10, 0x10(%rdi)+        sbbq    $0x0, %r11+        movq    %r11, 0x18(%rdi)+        sbbq    $0x0, %r12+        movq    %r12, 0x20(%rdi)+        sbbq    $0x0, %r13+        movq    %r13, 0x28(%rdi)+        sbbq    $0x0, %r14+        movq    %r14, 0x30(%rdi)+        sbbq    $0x0, %r15+        movq    %r15, 0x38(%rdi)+        sbbq    $0x0, %rdx+        andq    $0x1ff, %rdx+        movq    %rdx, 0x40(%rdi)+        CFI_INC_RSP(72)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_alt_mul_p521)++S2N_BN_FUNCTION_TYPE_DIRECTIVE(Lp521_jscalarmul_alt_sqr_p521)++Lp521_jscalarmul_alt_sqr_p521:+        CFI_START+        CFI_DEC_RSP(72)+        movq    (%rsi), %rax+        mulq    %rax+        movq    %rax, (%rsp)+        movq    %rdx, %r9+        xorq    %r10, %r10+        xorq    %r11, %r11+        movq    (%rsi), %rax+        mulq     0x8(%rsi)+        addq    %rax, %rax+        adcq    %rdx, %rdx+        adcq    $0x0, %r11+        addq    %rax, %r9+        adcq    %rdx, %r10+        adcq    $0x0, %r11+        movq    %r9, 0x8(%rsp)+        xorq    %r12, %r12+        movq    0x8(%rsi), %rax+        mulq    %rax+        addq    %rax, %r10+        adcq    %rdx, %r11+        adcq    $0x0, %r12+        movq    (%rsi), %rax+        mulq     0x10(%rsi)+        addq    %rax, %rax+        adcq    %rdx, %rdx+        adcq    $0x0, %r12+        addq    %rax, %r10+        adcq    %rdx, %r11+        adcq    $0x0, %r12+        movq    %r10, 0x10(%rsp)+        movq    (%rsi), %rax+        mulq     0x18(%rsi)+        xorq    %r13, %r13+        movq    %rax, %rbx+        movq    %rdx, %rcx+        movq    0x8(%rsi), %rax+        mulq     0x10(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r13+        addq    %rbx, %rbx+        adcq    %rcx, %rcx+        adcq    %r13, %r13+        addq    %rbx, %r11+        adcq    %rcx, %r12+        adcq    $0x0, %r13+        movq    %r11, 0x18(%rsp)+        movq    (%rsi), %rax+        mulq     0x20(%rsi)+        xorq    %r14, %r14+        movq    %rax, %rbx+        movq    %rdx, %rcx+        movq    0x8(%rsi), %rax+        mulq     0x18(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r14+        addq    %rbx, %rbx+        adcq    %rcx, %rcx+        adcq    %r14, %r14+        addq    %rbx, %r12+        adcq    %rcx, %r13+        adcq    $0x0, %r14+        movq    0x10(%rsi), %rax+        mulq    %rax+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        movq    %r12, 0x20(%rsp)+        movq    (%rsi), %rax+        mulq     0x28(%rsi)+        xorq    %r15, %r15+        movq    %rax, %rbx+        movq    %rdx, %rcx+        movq    0x8(%rsi), %rax+        mulq     0x20(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r15+        movq    0x10(%rsi), %rax+        mulq     0x18(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r15+        addq    %rbx, %rbx+        adcq    %rcx, %rcx+        adcq    %r15, %r15+        addq    %rbx, %r13+        adcq    %rcx, %r14+        adcq    $0x0, %r15+        movq    %r13, 0x28(%rsp)+        movq    (%rsi), %rax+        mulq     0x30(%rsi)+        xorq    %r8, %r8+        movq    %rax, %rbx+        movq    %rdx, %rcx+        movq    0x8(%rsi), %rax+        mulq     0x28(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r8+        movq    0x10(%rsi), %rax+        mulq     0x20(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r8+        addq    %rbx, %rbx+        adcq    %rcx, %rcx+        adcq    %r8, %r8+        addq    %rbx, %r14+        adcq    %rcx, %r15+        adcq    $0x0, %r8+        movq    0x18(%rsi), %rax+        mulq    %rax+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %r8+        movq    %r14, 0x30(%rsp)+        movq    (%rsi), %rax+        mulq     0x38(%rsi)+        xorq    %r9, %r9+        movq    %rax, %rbx+        movq    %rdx, %rcx+        movq    0x8(%rsi), %rax+        mulq     0x30(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r9+        movq    0x10(%rsi), %rax+        mulq     0x28(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r9+        movq    0x18(%rsi), %rax+        mulq     0x20(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r9+        addq    %rbx, %rbx+        adcq    %rcx, %rcx+        adcq    %r9, %r9+        addq    %rbx, %r15+        adcq    %rcx, %r8+        adcq    $0x0, %r9+        movq    %r15, 0x38(%rsp)+        movq    (%rsi), %rax+        mulq     0x40(%rsi)+        xorq    %r10, %r10+        movq    %rax, %rbx+        movq    %rdx, %rcx+        movq    0x8(%rsi), %rax+        mulq     0x38(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r10+        movq    0x10(%rsi), %rax+        mulq     0x30(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r10+        movq    0x18(%rsi), %rax+        mulq     0x28(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r10+        addq    %rbx, %rbx+        adcq    %rcx, %rcx+        adcq    %r10, %r10+        addq    %rbx, %r8+        adcq    %rcx, %r9+        adcq    $0x0, %r10+        movq    0x20(%rsi), %rax+        mulq    %rax+        addq    %rax, %r8+        adcq    %rdx, %r9+        adcq    $0x0, %r10+        movq    %r8, 0x40(%rsp)+        movq    0x8(%rsi), %rax+        mulq     0x40(%rsi)+        xorq    %r11, %r11+        movq    %rax, %rbx+        movq    %rdx, %rcx+        movq    0x10(%rsi), %rax+        mulq     0x38(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r11+        movq    0x18(%rsi), %rax+        mulq     0x30(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r11+        movq    0x20(%rsi), %rax+        mulq     0x28(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r11+        addq    %rbx, %rbx+        adcq    %rcx, %rcx+        adcq    %r11, %r11+        addq    %rbx, %r9+        adcq    %rcx, %r10+        adcq    $0x0, %r11+        movq    0x10(%rsi), %rax+        mulq     0x40(%rsi)+        xorq    %r12, %r12+        movq    %rax, %rbx+        movq    %rdx, %rcx+        movq    0x18(%rsi), %rax+        mulq     0x38(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r12+        movq    0x20(%rsi), %rax+        mulq     0x30(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r12+        addq    %rbx, %rbx+        adcq    %rcx, %rcx+        adcq    %r12, %r12+        addq    %rbx, %r10+        adcq    %rcx, %r11+        adcq    $0x0, %r12+        movq    0x28(%rsi), %rax+        mulq    %rax+        addq    %rax, %r10+        adcq    %rdx, %r11+        adcq    $0x0, %r12+        movq    0x18(%rsi), %rax+        mulq     0x40(%rsi)+        xorq    %r13, %r13+        movq    %rax, %rbx+        movq    %rdx, %rcx+        movq    0x20(%rsi), %rax+        mulq     0x38(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r13+        movq    0x28(%rsi), %rax+        mulq     0x30(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r13+        addq    %rbx, %rbx+        adcq    %rcx, %rcx+        adcq    %r13, %r13+        addq    %rbx, %r11+        adcq    %rcx, %r12+        adcq    $0x0, %r13+        movq    0x20(%rsi), %rax+        mulq     0x40(%rsi)+        xorq    %r14, %r14+        movq    %rax, %rbx+        movq    %rdx, %rcx+        movq    0x28(%rsi), %rax+        mulq     0x38(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r14+        addq    %rbx, %rbx+        adcq    %rcx, %rcx+        adcq    %r14, %r14+        addq    %rbx, %r12+        adcq    %rcx, %r13+        adcq    $0x0, %r14+        movq    0x30(%rsi), %rax+        mulq    %rax+        addq    %rax, %r12+        adcq    %rdx, %r13+        adcq    $0x0, %r14+        movq    0x28(%rsi), %rax+        mulq     0x40(%rsi)+        xorq    %r15, %r15+        movq    %rax, %rbx+        movq    %rdx, %rcx+        movq    0x30(%rsi), %rax+        mulq     0x38(%rsi)+        addq    %rax, %rbx+        adcq    %rdx, %rcx+        adcq    $0x0, %r15+        addq    %rbx, %rbx+        adcq    %rcx, %rcx+        adcq    %r15, %r15+        addq    %rbx, %r13+        adcq    %rcx, %r14+        adcq    $0x0, %r15+        xorq    %r8, %r8+        movq    0x38(%rsi), %rax+        mulq    %rax+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x30(%rsi), %rax+        mulq     0x40(%rsi)+        addq    %rax, %rax+        adcq    %rdx, %rdx+        adcq    $0x0, %r8+        addq    %rax, %r14+        adcq    %rdx, %r15+        adcq    $0x0, %r8+        movq    0x38(%rsi), %rax+        mulq     0x40(%rsi)+        addq    %rax, %rax+        adcq    %rdx, %rdx+        addq    %rax, %r15+        adcq    %rdx, %r8+        movq    0x40(%rsi), %rax+        imulq   %rax, %rax+        addq    %r8, %rax+        movq    0x40(%rsp), %r8+        movq    %r8, %rdx+        andq    $0x1ff, %rdx+        shrdq   $0x9, %r9, %r8+        shrdq   $0x9, %r10, %r9+        shrdq   $0x9, %r11, %r10+        shrdq   $0x9, %r12, %r11+        shrdq   $0x9, %r13, %r12+        shrdq   $0x9, %r14, %r13+        shrdq   $0x9, %r15, %r14+        shrdq   $0x9, %rax, %r15+        shrq    $0x9, %rax+        addq    %rax, %rdx+        stc+        adcq    (%rsp), %r8+        adcq    0x8(%rsp), %r9+        adcq    0x10(%rsp), %r10+        adcq    0x18(%rsp), %r11+        adcq    0x20(%rsp), %r12+        adcq    0x28(%rsp), %r13+        adcq    0x30(%rsp), %r14+        adcq    0x38(%rsp), %r15+        adcq    $0xfffffffffffffe00, %rdx+        cmc+        sbbq    $0x0, %r8+        movq    %r8, (%rdi)+        sbbq    $0x0, %r9+        movq    %r9, 0x8(%rdi)+        sbbq    $0x0, %r10+        movq    %r10, 0x10(%rdi)+        sbbq    $0x0, %r11+        movq    %r11, 0x18(%rdi)+        sbbq    $0x0, %r12+        movq    %r12, 0x20(%rdi)+        sbbq    $0x0, %r13+        movq    %r13, 0x28(%rdi)+        sbbq    $0x0, %r14+        movq    %r14, 0x30(%rdi)+        sbbq    $0x0, %r15+        movq    %r15, 0x38(%rdi)+        sbbq    $0x0, %rdx+        andq    $0x1ff, %rdx+        movq    %rdx, 0x40(%rdi)+        CFI_INC_RSP(72)+        CFI_RET++S2N_BN_SIZE_DIRECTIVE(Lp521_jscalarmul_alt_sqr_p521)++#if defined(__linux__) && defined(__ELF__)+.section .note.GNU-stack, "", %progbits+#endif
+ cbits/sha1_armv8.c view
@@ -0,0 +1,169 @@+/*+ * SHA-1 using the ARMv8-A cryptographic extensions.+ *+ * crypton_sha1.c computes the compression function a round at a time in plain+ * C.  AArch64 has instructions for it -- SHA1C, SHA1P, SHA1M, SHA1H, SHA1SU0+ * and SHA1SU1 -- which do four rounds at a time and most of the message+ * schedule alongside.  They come with the SHA-256 ones this tree already uses,+ * under the same optional feature, so anything that has those has these.+ *+ * SHA-1 is not a hash to choose today, but it is still what a number of+ * protocols and file formats ask for.+ */++#include <stdint.h>+#include <arm_neon.h>+#if defined(__linux__)+#include <sys/auxv.h>+#include <asm/hwcap.h>+#endif++/*+ * The instructions are an extension, so a translation unit compiled for+ * baseline ARMv8-A may not use them; see sha256_armv8.c for the whole of that+ * argument.+ */+#include "crypton_armv8_target.h"++/*+ * A group of four rounds, and the schedule that goes with it.+ *+ * SHA1C, SHA1P and SHA1M each do four rounds with one of the three round+ * functions -- choose, parity and majority -- taking the four state words in+ * a register, E in a general one, and the four message words with their round+ * constant already added.  SHA1H is the rotation of A by thirty that carries+ * E from one group to the next.+ *+ * The schedule is the exclusive or of four earlier words rotated left by one.+ * SHA1SU0 does the three terms that reach furthest back and SHA1SU1 the last+ * one, together with the rotation and the dependency inside the group.+ */+#define GROUP(f, ecur, enext, wk_cur, wk_next, kk, w0, w1, w2, w3)           \+	do {                                                                 \+		enext = vsha1h_u32(vgetq_lane_u32(abcd, 0));                 \+		abcd = f(abcd, ecur, wk_cur);                                \+		wk_next = vaddq_u32(w2, kk);                                 \+		w0 = vsha1su0q_u32(w0, w1, w2);                              \+		w3 = vsha1su1q_u32(w3, w2);                                  \+	} while (0)++/*+ * One 64-byte block.  `state` is the five words of chaining value in host+ * order, `buf` the block as it arrived, which SHA-1 reads big-endian.+ */+CRYPTON_TARGET_ARMV8_CRYPTO+void crypton_sha1_armv8_do_chunks(uint32_t state[5], const uint8_t *data,+                                  uint32_t blocks)+{+	const uint32x4_t k0 = vdupq_n_u32(0x5a827999);+	const uint32x4_t k1 = vdupq_n_u32(0x6ed9eba1);+	const uint32x4_t k2 = vdupq_n_u32(0x8f1bbcdc);+	const uint32x4_t k3 = vdupq_n_u32(0xca62c1d6);+	uint32x4_t abcd, abcd_prev;+	uint32x4_t m0, m1, m2, m3;+	uint32x4_t wk0, wk1;+	uint32_t e0, e1, e_prev;++	abcd = vld1q_u32(state);+	e0 = state[4];++	for (; blocks > 0; blocks--, data += 64) {+	const uint8_t *buf = data;++	abcd_prev = abcd;+	e_prev = e0;++	m0 = vreinterpretq_u32_u8(vrev32q_u8(vld1q_u8(buf)));+	m1 = vreinterpretq_u32_u8(vrev32q_u8(vld1q_u8(buf + 16)));+	m2 = vreinterpretq_u32_u8(vrev32q_u8(vld1q_u8(buf + 32)));+	m3 = vreinterpretq_u32_u8(vrev32q_u8(vld1q_u8(buf + 48)));++	wk0 = vaddq_u32(m0, k0);+	wk1 = vaddq_u32(m1, k0);++	/* rounds 0 to 15, where the schedule has less to do each group until+	 * it is running a whole group ahead */+	e1 = vsha1h_u32(vgetq_lane_u32(abcd, 0));+	abcd = vsha1cq_u32(abcd, e0, wk0);+	wk0 = vaddq_u32(m2, k0);+	m0 = vsha1su0q_u32(m0, m1, m2);++	e0 = vsha1h_u32(vgetq_lane_u32(abcd, 0));+	abcd = vsha1cq_u32(abcd, e1, wk1);+	wk1 = vaddq_u32(m3, k0);+	m1 = vsha1su0q_u32(m1, m2, m3);+	m0 = vsha1su1q_u32(m0, m3);++	e1 = vsha1h_u32(vgetq_lane_u32(abcd, 0));+	abcd = vsha1cq_u32(abcd, e0, wk0);+	wk0 = vaddq_u32(m0, k0);+	m2 = vsha1su0q_u32(m2, m3, m0);+	m1 = vsha1su1q_u32(m1, m0);++	e0 = vsha1h_u32(vgetq_lane_u32(abcd, 0));+	abcd = vsha1cq_u32(abcd, e1, wk1);+	wk1 = vaddq_u32(m1, k1);+	m3 = vsha1su0q_u32(m3, m0, m1);+	m2 = vsha1su1q_u32(m2, m1);++	/* rounds 16 to 19, still the choose function, and then twenty of each+	 * of the others; the message registers come back to the same roles+	 * every fourth group */+	GROUP(vsha1cq_u32, e0, e1, wk0, wk0, k1, m0, m1, m2, m3);+	GROUP(vsha1pq_u32, e1, e0, wk1, wk1, k1, m1, m2, m3, m0);+	GROUP(vsha1pq_u32, e0, e1, wk0, wk0, k1, m2, m3, m0, m1);+	GROUP(vsha1pq_u32, e1, e0, wk1, wk1, k1, m3, m0, m1, m2);+	GROUP(vsha1pq_u32, e0, e1, wk0, wk0, k2, m0, m1, m2, m3);+	GROUP(vsha1pq_u32, e1, e0, wk1, wk1, k2, m1, m2, m3, m0);+	GROUP(vsha1mq_u32, e0, e1, wk0, wk0, k2, m2, m3, m0, m1);+	GROUP(vsha1mq_u32, e1, e0, wk1, wk1, k2, m3, m0, m1, m2);+	GROUP(vsha1mq_u32, e0, e1, wk0, wk0, k2, m0, m1, m2, m3);+	GROUP(vsha1mq_u32, e1, e0, wk1, wk1, k3, m1, m2, m3, m0);+	GROUP(vsha1mq_u32, e0, e1, wk0, wk0, k3, m2, m3, m0, m1);+	GROUP(vsha1pq_u32, e1, e0, wk1, wk1, k3, m3, m0, m1, m2);++	/* rounds 64 to 79, where the schedule runs out a piece at a time */+	e1 = vsha1h_u32(vgetq_lane_u32(abcd, 0));+	abcd = vsha1pq_u32(abcd, e0, wk0);+	wk0 = vaddq_u32(m2, k3);+	m3 = vsha1su1q_u32(m3, m2);++	e0 = vsha1h_u32(vgetq_lane_u32(abcd, 0));+	abcd = vsha1pq_u32(abcd, e1, wk1);+	wk1 = vaddq_u32(m3, k3);++	e1 = vsha1h_u32(vgetq_lane_u32(abcd, 0));+	abcd = vsha1pq_u32(abcd, e0, wk0);++	e0 = vsha1h_u32(vgetq_lane_u32(abcd, 0));+	abcd = vsha1pq_u32(abcd, e1, wk1);++	abcd = vaddq_u32(abcd, abcd_prev);+	e0 += e_prev;+	}++	vst1q_u32(state, abcd);+	state[4] = e0;+}++/* the one-block form, for the partial block a message ends with */+void crypton_sha1_armv8_do_chunk(uint32_t state[5], const uint8_t buf[64])+{+	crypton_sha1_armv8_do_chunks(state, (const uint8_t *) buf, 1);+}++/*+ * The SHA-1 instructions are optional in ARMv8.0, and arrive with the SHA-256+ * ones.  They are always there on Apple silicon; elsewhere the kernel reports+ * them.+ */+int crypton_sha1_armv8_available(void)+{+#if defined(__APPLE__)+	return 1;+#elif defined(__linux__)+	return (getauxval(AT_HWCAP) & HWCAP_SHA1) != 0;+#else+	return 0;+#endif+}
+ cbits/sha1_x86.c view
@@ -0,0 +1,174 @@+/*+ * SHA-1 using the Intel SHA extensions.+ *+ * crypton_sha1.c computes the compression function a round at a time in plain+ * C.  The same extension that carries SHA256RNDS2 carries four instructions+ * for this one -- SHA1RNDS4, SHA1NEXTE, SHA1MSG1 and SHA1MSG2 -- which do four+ * rounds at a time and most of the message schedule alongside.+ *+ * SHA-1 is not a hash to choose today, but it is still what a number of+ * protocols and file formats ask for, and the instructions are already there+ * on any processor that has the SHA-256 ones.+ */++#include <stdint.h>+#include <immintrin.h>+#include "crypton_cpu.h"++/*+ * The instructions are an extension, so a translation unit compiled for the+ * x86-64 baseline may not use them; see cbits/sha256_x86.c for the whole of+ * that argument.  SSE4.1 and SSSE3 come along for the same reasons there.+ */+#ifdef WITH_TARGET_ATTRIBUTES+#define TARGET_X86_SHA __attribute__((target("sha,sse4.1,ssse3")))+#else+#define TARGET_X86_SHA+#endif++/*+ * A group of four rounds, and the schedule that goes with it.+ *+ * SHA1RNDS4 takes the four state words in one register -- A in the top lane,+ * which is why both the state and each block are loaded reversed -- and the+ * four message words with E already added into the first, which is what+ * SHA1NEXTE produces from the state as it stood four rounds ago.  The round+ * function and constant come from the immediate: 0 for rounds 0 to 19, then+ * one per twenty.+ *+ * The schedule is the exclusive or of four earlier words rotated left by one.+ * SHA1MSG1 does the part that reaches furthest back, the exclusive or with+ * the word eight before is an ordinary one, and SHA1MSG2 does the last part+ * together with the rotation and the dependency inside the group of four.+ */+#define GROUP(imm, ecur, enext, w0, w1, w2, w3)                              \+	do {                                                                 \+		ecur = _mm_sha1nexte_epu32(ecur, w0);                        \+		enext = abcd;                                                \+		w1 = _mm_sha1msg2_epu32(w1, w0);                             \+		abcd = _mm_sha1rnds4_epu32(abcd, ecur, imm);                 \+		w3 = _mm_sha1msg1_epu32(w3, w0);                             \+		w2 = _mm_xor_si128(w2, w0);                                  \+	} while (0)++/* the same without the part of the schedule that has run out */+#define GROUP_NOMSG1(imm, ecur, enext, w0, w1, w2)                           \+	do {                                                                 \+		ecur = _mm_sha1nexte_epu32(ecur, w0);                        \+		enext = abcd;                                                \+		w1 = _mm_sha1msg2_epu32(w1, w0);                             \+		abcd = _mm_sha1rnds4_epu32(abcd, ecur, imm);                 \+		w2 = _mm_xor_si128(w2, w0);                                  \+	} while (0)++#define GROUP_MSG2(imm, ecur, enext, w0, w1)                                 \+	do {                                                                 \+		ecur = _mm_sha1nexte_epu32(ecur, w0);                        \+		enext = abcd;                                                \+		w1 = _mm_sha1msg2_epu32(w1, w0);                             \+		abcd = _mm_sha1rnds4_epu32(abcd, ecur, imm);                 \+	} while (0)++#define GROUP_ROUNDS(imm, ecur, enext, w0)                                   \+	do {                                                                 \+		ecur = _mm_sha1nexte_epu32(ecur, w0);                        \+		enext = abcd;                                                \+		abcd = _mm_sha1rnds4_epu32(abcd, ecur, imm);                 \+	} while (0)++/*+ * Any number of 64-byte blocks.  `state` is the five words of chaining+ * value in host order, `data` the blocks as they arrived, which SHA-1+ * reads big-endian.+ *+ * The state stays in registers from one block to the next.  Taking them a+ * block at a time, which is what this did, spends a load, a store and two+ * shuffles either side of every block putting state back where it already+ * was, and against the hundred-odd cycles a block costs with these+ * instructions that is worth having.+ */+TARGET_X86_SHA+void crypton_sha1_x86_do_chunks(uint32_t state[5], const uint8_t *data,+                                uint32_t blocks)+{+	/* the whole register reversed, which byte-swaps each word and puts+	 * the first of them in the top lane */+	const __m128i bswap = _mm_setr_epi8(15, 14, 13, 12, 11, 10, 9, 8,+	                                    7, 6, 5, 4, 3, 2, 1, 0);+	__m128i abcd, e0, e1, abcd_prev, e_prev;+	__m128i m0, m1, m2, m3;++	abcd = _mm_shuffle_epi32(_mm_loadu_si128((const __m128i *) state), 0x1b);+	e0 = _mm_set_epi32((int) state[4], 0, 0, 0);++	for (; blocks > 0; blocks--, data += 64) {+	const uint32_t *buf = (const uint32_t *) data;++	abcd_prev = abcd;+	e_prev = e0;++	m0 = _mm_shuffle_epi8(_mm_loadu_si128((const __m128i *) buf), bswap);+	m1 = _mm_shuffle_epi8(_mm_loadu_si128((const __m128i *) (buf + 4)), bswap);+	m2 = _mm_shuffle_epi8(_mm_loadu_si128((const __m128i *) (buf + 8)), bswap);+	m3 = _mm_shuffle_epi8(_mm_loadu_si128((const __m128i *) (buf + 12)), bswap);++	/* rounds 0 to 15, where the schedule has nothing to extend yet: the+	 * first group takes E by an ordinary addition rather than SHA1NEXTE,+	 * there being no state from four rounds ago */+	e0 = _mm_add_epi32(e0, m0);+	e1 = abcd;+	abcd = _mm_sha1rnds4_epu32(abcd, e0, 0);++	e1 = _mm_sha1nexte_epu32(e1, m1);+	e0 = abcd;+	abcd = _mm_sha1rnds4_epu32(abcd, e1, 0);+	m0 = _mm_sha1msg1_epu32(m0, m1);++	e0 = _mm_sha1nexte_epu32(e0, m2);+	e1 = abcd;+	abcd = _mm_sha1rnds4_epu32(abcd, e0, 0);+	m1 = _mm_sha1msg1_epu32(m1, m2);+	m0 = _mm_xor_si128(m0, m2);++	GROUP(0, e1, e0, m3, m0, m1, m2);++	/* rounds 16 to 63, where every group both hashes and schedules; the+	 * four message registers come back to the same roles every fourth+	 * group, and the round function changes every twentieth round */+	GROUP(0, e0, e1, m0, m1, m2, m3);+	GROUP(1, e1, e0, m1, m2, m3, m0);+	GROUP(1, e0, e1, m2, m3, m0, m1);+	GROUP(1, e1, e0, m3, m0, m1, m2);+	GROUP(1, e0, e1, m0, m1, m2, m3);+	GROUP(1, e1, e0, m1, m2, m3, m0);+	GROUP(2, e0, e1, m2, m3, m0, m1);+	GROUP(2, e1, e0, m3, m0, m1, m2);+	GROUP(2, e0, e1, m0, m1, m2, m3);+	GROUP(2, e1, e0, m1, m2, m3, m0);+	GROUP(2, e0, e1, m2, m3, m0, m1);+	GROUP(3, e1, e0, m3, m0, m1, m2);++	/* rounds 64 to 79, where the schedule runs out a piece at a time.  The+	 * first of these still extends: the part of the last four words that+	 * reaches sixteen back is taken here, three groups before they are+	 * finished */+	GROUP(3, e0, e1, m0, m1, m2, m3);+	GROUP_NOMSG1(3, e1, e0, m1, m2, m3);+	GROUP_MSG2(3, e0, e1, m2, m3);+	GROUP_ROUNDS(3, e1, e0, m3);++	/* and the chaining value, E through the same instruction that would+	 * have carried it into a fifth round */+	e0 = _mm_sha1nexte_epu32(e0, e_prev);+	abcd = _mm_add_epi32(abcd, abcd_prev);+	}++	_mm_storeu_si128((__m128i *) state, _mm_shuffle_epi32(abcd, 0x1b));+	state[4] = (uint32_t) _mm_extract_epi32(e0, 3);+}++/* the one-block form, for the partial block a message ends with */+void crypton_sha1_x86_do_chunk(uint32_t state[5], const uint8_t buf[64])+{+	crypton_sha1_x86_do_chunks(state, (const uint8_t *) buf, 1);+}
+ cbits/sha256_armv8.c view
@@ -0,0 +1,141 @@+/*+ * SHA-256 using the ARMv8-A cryptographic extensions.+ *+ * crypton_sha256.c computes the compression function a round at a time in+ * plain C.  AArch64 has instructions for it -- SHA256H, SHA256H2, SHA256SU0+ * and SHA256SU1 -- which do four rounds at a time and compute the message+ * schedule alongside.  This provides that version; crypton_sha256.c picks+ * between the two at runtime.+ *+ * SHA-224 shares the compression function, so it comes along for free.+ */++#include <stdint.h>+#include <arm_neon.h>+#if defined(__linux__)+#include <sys/auxv.h>+#include <asm/hwcap.h>+#endif++/*+ * The SHA-2 instructions are an extension, so a translation unit compiled for+ * baseline ARMv8-A may not use them.  Mark the function that does, the way+ * cbits/aes/x86ni.h marks its x86 counterparts, rather than raising+ * -march for every file in the library: the flag use_target_attributes picks+ * between the two, and with it set -- which is the default -- nothing else+ * enables the extensions, so without these the file does not compile at all on+ * a toolchain whose baseline lacks them.  Apple's does not lack them, which is+ * why only Linux noticed.+ *+ * "+crypto" rather than "crypto": GCC rejects the latter.+ */+#include "crypton_armv8_target.h"++static const uint32_t K[64] = {+	0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5,+	0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5,+	0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3,+	0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174,+	0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc,+	0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da,+	0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7,+	0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967,+	0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13,+	0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85,+	0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3,+	0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070,+	0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5,+	0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3,+	0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208,+	0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2,+};++/*+ * One 64-byte block.  `state` is the eight words of chaining value in host+ * order, `buf` the block as it arrived, which SHA-256 reads big-endian.+ */+CRYPTON_TARGET_ARMV8_CRYPTO+void crypton_sha256_armv8_do_chunk(uint32_t state[8], const uint8_t buf[64])+{+	uint32x4_t abcd, efgh, abcd_prev, efgh_prev, abcd_save, tmp;+	uint32x4_t m0, m1, m2, m3;+	int i;++	abcd_prev = abcd = vld1q_u32(state);+	efgh_prev = efgh = vld1q_u32(state + 4);++	m0 = vreinterpretq_u32_u8(vrev32q_u8(vld1q_u8(buf)));+	m1 = vreinterpretq_u32_u8(vrev32q_u8(vld1q_u8(buf + 16)));+	m2 = vreinterpretq_u32_u8(vrev32q_u8(vld1q_u8(buf + 32)));+	m3 = vreinterpretq_u32_u8(vrev32q_u8(vld1q_u8(buf + 48)));++	/* twelve groups of four rounds that also extend the schedule ... */+	for (i = 0; i < 48; i += 16) {+		uint32x4_t n0, n1, n2, n3;++		n0 = vsha256su1q_u32(vsha256su0q_u32(m0, m1), m2, m3);+		tmp = vaddq_u32(m0, vld1q_u32(&K[i]));+		abcd_save = abcd;+		abcd = vsha256hq_u32(abcd, efgh, tmp);+		efgh = vsha256h2q_u32(efgh, abcd_save, tmp);++		n1 = vsha256su1q_u32(vsha256su0q_u32(m1, m2), m3, n0);+		tmp = vaddq_u32(m1, vld1q_u32(&K[i + 4]));+		abcd_save = abcd;+		abcd = vsha256hq_u32(abcd, efgh, tmp);+		efgh = vsha256h2q_u32(efgh, abcd_save, tmp);++		n2 = vsha256su1q_u32(vsha256su0q_u32(m2, m3), n0, n1);+		tmp = vaddq_u32(m2, vld1q_u32(&K[i + 8]));+		abcd_save = abcd;+		abcd = vsha256hq_u32(abcd, efgh, tmp);+		efgh = vsha256h2q_u32(efgh, abcd_save, tmp);++		n3 = vsha256su1q_u32(vsha256su0q_u32(m3, n0), n1, n2);+		tmp = vaddq_u32(m3, vld1q_u32(&K[i + 12]));+		abcd_save = abcd;+		abcd = vsha256hq_u32(abcd, efgh, tmp);+		efgh = vsha256h2q_u32(efgh, abcd_save, tmp);++		m0 = n0; m1 = n1; m2 = n2; m3 = n3;+	}++	/* ... and the last four, where there is no more schedule to extend */+	tmp = vaddq_u32(m0, vld1q_u32(&K[48]));+	abcd_save = abcd;+	abcd = vsha256hq_u32(abcd, efgh, tmp);+	efgh = vsha256h2q_u32(efgh, abcd_save, tmp);++	tmp = vaddq_u32(m1, vld1q_u32(&K[52]));+	abcd_save = abcd;+	abcd = vsha256hq_u32(abcd, efgh, tmp);+	efgh = vsha256h2q_u32(efgh, abcd_save, tmp);++	tmp = vaddq_u32(m2, vld1q_u32(&K[56]));+	abcd_save = abcd;+	abcd = vsha256hq_u32(abcd, efgh, tmp);+	efgh = vsha256h2q_u32(efgh, abcd_save, tmp);++	tmp = vaddq_u32(m3, vld1q_u32(&K[60]));+	abcd_save = abcd;+	abcd = vsha256hq_u32(abcd, efgh, tmp);+	efgh = vsha256h2q_u32(efgh, abcd_save, tmp);++	vst1q_u32(state, vaddq_u32(abcd, abcd_prev));+	vst1q_u32(state + 4, vaddq_u32(efgh, efgh_prev));+}++/*+ * The SHA-2 instructions are optional in ARMv8.0.  They are always there on+ * Apple silicon; elsewhere the kernel reports them.+ */+int crypton_sha256_armv8_available(void)+{+#if defined(__APPLE__)+	return 1;+#elif defined(__linux__)+	return (getauxval(AT_HWCAP) & HWCAP_SHA2) != 0;+#else+	return 0;+#endif+}
+ cbits/sha3_armv8.c view
@@ -0,0 +1,172 @@+/*+ * Keccak-f[1600] using the ARMv8.2 SHA-3 instructions.+ *+ * crypton_sha3.c runs the permutation in plain C, a round at a time over a+ * table of rotation amounts and lane positions.  AArch64 has four+ * instructions that exist for exactly this:+ *+ *   EOR3  a ^ b ^ c            the column parities of theta+ *   RAX1  a ^ ROL(b, 1)        the rest of theta+ *   XAR   ROR(a ^ b, n)        theta's exclusive or and rho's rotation at once+ *   BCAX  a ^ (b & ~c)         chi+ *+ * They work on 128-bit registers and the permutation has twenty-five 64-bit+ * lanes, so each lane sits in the low half of a register and the high half+ * goes unused.  Rho and pi move one lane of every row into every other row,+ * so the round cannot be done in place: the twenty-five rotated words are+ * computed first and chi then writes the state from them.+ *+ * The body is generated from the definitions in FIPS 202 rather than copied+ * in: the rotation amounts are the triangular numbers modulo 64, pi sends+ * lane (x, y) to (y, 2x + 3y), and the script that worked those out checked+ * the result against the published digests of the empty string and of "abc"+ * before emitting any of this.+ */++#include <stdint.h>+#include <arm_neon.h>+#if defined(__APPLE__)+#include <sys/sysctl.h>+#elif defined(__linux__)+#include <sys/auxv.h>+#include <asm/hwcap.h>+#endif++/*+ * The SHA-3 instructions are an ARMv8.2 extension, so a translation unit+ * compiled for the baseline may not use them; see sha256_armv8.c for the whole+ * of that argument.  The flag for a build without attributes already asks for+ * "+sha3", which the SHA-512 path needed.+ */+#include "crypton_armv8_target.h"++static const uint64_t rc[24] = {+	0x0000000000000001ULL, 0x0000000000008082ULL, 0x800000000000808aULL,+	0x8000000080008000ULL, 0x000000000000808bULL, 0x0000000080000001ULL,+	0x8000000080008081ULL, 0x8000000000008009ULL, 0x000000000000008aULL,+	0x0000000000000088ULL, 0x0000000080008009ULL, 0x000000008000000aULL,+	0x000000008000808bULL, 0x800000000000008bULL, 0x8000000000008089ULL,+	0x8000000000008003ULL, 0x8000000000008002ULL, 0x8000000000000080ULL,+	0x000000000000800aULL, 0x800000008000000aULL, 0x8000000080008081ULL,+	0x8000000000008080ULL, 0x0000000080000001ULL, 0x8000000080008008ULL,+};++#define ROUND(k)                                                             \+	do {                                                                 \+	/* theta: the parity of each column, and what it adds */           \+	c[0] = veor3q_u64(a[0], a[5], a[10]);                              \+	c[0] = veor3q_u64(c[0], a[15], a[20]);                             \+	c[1] = veor3q_u64(a[1], a[6], a[11]);                              \+	c[1] = veor3q_u64(c[1], a[16], a[21]);                             \+	c[2] = veor3q_u64(a[2], a[7], a[12]);                              \+	c[2] = veor3q_u64(c[2], a[17], a[22]);                             \+	c[3] = veor3q_u64(a[3], a[8], a[13]);                              \+	c[3] = veor3q_u64(c[3], a[18], a[23]);                             \+	c[4] = veor3q_u64(a[4], a[9], a[14]);                              \+	c[4] = veor3q_u64(c[4], a[19], a[24]);                             \+	d[0] = vrax1q_u64(c[4], c[1]);                                     \+	d[1] = vrax1q_u64(c[0], c[2]);                                     \+	d[2] = vrax1q_u64(c[1], c[3]);                                     \+	d[3] = vrax1q_u64(c[2], c[4]);                                     \+	d[4] = vrax1q_u64(c[3], c[0]);                                     \+	/* theta's exclusive or, rho's rotation and pi's move, in one */   \+	b[0 ] = veorq_u64(a[0 ], d[0]);                                    \+	b[1 ] = vxarq_u64(a[6 ], d[1], 20);                                \+	b[2 ] = vxarq_u64(a[12], d[2], 21);                                \+	b[3 ] = vxarq_u64(a[18], d[3], 43);                                \+	b[4 ] = vxarq_u64(a[24], d[4], 50);                                \+	b[5 ] = vxarq_u64(a[3 ], d[3], 36);                                \+	b[6 ] = vxarq_u64(a[9 ], d[4], 44);                                \+	b[7 ] = vxarq_u64(a[10], d[0], 61);                                \+	b[8 ] = vxarq_u64(a[16], d[1], 19);                                \+	b[9 ] = vxarq_u64(a[22], d[2],  3);                                \+	b[10] = vxarq_u64(a[1 ], d[1], 63);                                \+	b[11] = vxarq_u64(a[7 ], d[2], 58);                                \+	b[12] = vxarq_u64(a[13], d[3], 39);                                \+	b[13] = vxarq_u64(a[19], d[4], 56);                                \+	b[14] = vxarq_u64(a[20], d[0], 46);                                \+	b[15] = vxarq_u64(a[4 ], d[4], 37);                                \+	b[16] = vxarq_u64(a[5 ], d[0], 28);                                \+	b[17] = vxarq_u64(a[11], d[1], 54);                                \+	b[18] = vxarq_u64(a[17], d[2], 49);                                \+	b[19] = vxarq_u64(a[23], d[3],  8);                                \+	b[20] = vxarq_u64(a[2 ], d[2],  2);                                \+	b[21] = vxarq_u64(a[8 ], d[3],  9);                                \+	b[22] = vxarq_u64(a[14], d[4], 25);                                \+	b[23] = vxarq_u64(a[15], d[0], 23);                                \+	b[24] = vxarq_u64(a[21], d[1], 62);                                \+	/* chi, along each row */                                          \+	a[0 ] = vbcaxq_u64(b[0 ], b[2 ], b[1 ]);                           \+	a[1 ] = vbcaxq_u64(b[1 ], b[3 ], b[2 ]);                           \+	a[2 ] = vbcaxq_u64(b[2 ], b[4 ], b[3 ]);                           \+	a[3 ] = vbcaxq_u64(b[3 ], b[0 ], b[4 ]);                           \+	a[4 ] = vbcaxq_u64(b[4 ], b[1 ], b[0 ]);                           \+	a[5 ] = vbcaxq_u64(b[5 ], b[7 ], b[6 ]);                           \+	a[6 ] = vbcaxq_u64(b[6 ], b[8 ], b[7 ]);                           \+	a[7 ] = vbcaxq_u64(b[7 ], b[9 ], b[8 ]);                           \+	a[8 ] = vbcaxq_u64(b[8 ], b[5 ], b[9 ]);                           \+	a[9 ] = vbcaxq_u64(b[9 ], b[6 ], b[5 ]);                           \+	a[10] = vbcaxq_u64(b[10], b[12], b[11]);                           \+	a[11] = vbcaxq_u64(b[11], b[13], b[12]);                           \+	a[12] = vbcaxq_u64(b[12], b[14], b[13]);                           \+	a[13] = vbcaxq_u64(b[13], b[10], b[14]);                           \+	a[14] = vbcaxq_u64(b[14], b[11], b[10]);                           \+	a[15] = vbcaxq_u64(b[15], b[17], b[16]);                           \+	a[16] = vbcaxq_u64(b[16], b[18], b[17]);                           \+	a[17] = vbcaxq_u64(b[17], b[19], b[18]);                           \+	a[18] = vbcaxq_u64(b[18], b[15], b[19]);                           \+	a[19] = vbcaxq_u64(b[19], b[16], b[15]);                           \+	a[20] = vbcaxq_u64(b[20], b[22], b[21]);                           \+	a[21] = vbcaxq_u64(b[21], b[23], b[22]);                           \+	a[22] = vbcaxq_u64(b[22], b[24], b[23]);                           \+	a[23] = vbcaxq_u64(b[23], b[20], b[24]);                           \+	a[24] = vbcaxq_u64(b[24], b[21], b[20]);                           \+	/* iota */                                                         \+		a[0] = veorq_u64(a[0], vld1q_dup_u64(&rc[k]));                \+	} while (0)++/* the twenty-four rounds over the state, in place */+CRYPTON_TARGET_ARMV8_SHA3+void crypton_sha3_armv8_permute(uint64_t state[25])+{+	uint64x2_t a[25], b[25], c[5], d[5];+	int i, round;++	for (i = 0; i < 25; i++)+		a[i] = vld1q_dup_u64(&state[i]);++	/* four rounds to an iteration: a round is a chain -- the column+	 * parities wait for the last chi of the round before -- so giving the+	 * processor more than one of them to look at is worth something.  One+	 * round an iteration measured 802 MB/s of SHA3-256, two 949 and four+	 * 991, against 551 for the plain C */+	for (round = 0; round < 24; round += 4) {+		ROUND(round);+		ROUND(round + 1);+		ROUND(round + 2);+		ROUND(round + 3);+	}++	for (i = 0; i < 25; i++)+		state[i] = vgetq_lane_u64(a[i], 0);+}++/*+ * Whether the extension is there.  It is on Apple silicon; elsewhere the+ * kernel reports it.+ */+int crypton_sha3_armv8_available(void)+{+#if defined(__APPLE__)+	int v = 0;+	size_t n = sizeof(v);++	if (sysctlbyname("hw.optional.arm.FEAT_SHA3", &v, &n, NULL, 0) != 0)+		return 0;+	return v != 0;+#elif defined(__linux__)+	return (getauxval(AT_HWCAP) & HWCAP_SHA3) != 0;+#else+	return 0;+#endif+}
+ cbits/sha512_armv8.c view
@@ -0,0 +1,157 @@+/*+ * SHA-512 using the ARMv8.2 SHA-512 extension.+ *+ * The same idea as sha256_armv8.c: SHA512H, SHA512H2, SHA512SU0 and+ * SHA512SU1 do two rounds at a time and compute the message schedule+ * alongside.  This extension is a good deal less common than the SHA-256+ * one -- it arrived in ARMv8.2 and is optional there -- so the runtime+ * check matters more here, and it is asked rather than assumed even on+ * Apple, where the SHA-256 one is taken for granted.+ *+ * SHA-384 and the truncated SHA-512/t variants share the compression+ * function, so they come along.+ */++#include <stdint.h>+#include <arm_neon.h>+#if defined(__linux__)+#include <sys/auxv.h>+#include <asm/hwcap.h>+#endif+#if defined(__APPLE__)+#include <sys/sysctl.h>+#include <string.h>+#endif++/*+ * The instructions are an extension, so a translation unit compiled for+ * baseline ARMv8-A may not use them; mark the function that does.  The+ * SHA-512 instructions live behind "+sha3" in both GCC and clang.+ */+#include "crypton_armv8_target.h"++static const uint64_t K[80] = {+	0x428a2f98d728ae22ULL, 0x7137449123ef65cdULL, 0xb5c0fbcfec4d3b2fULL,+	0xe9b5dba58189dbbcULL, 0x3956c25bf348b538ULL, 0x59f111f1b605d019ULL,+	0x923f82a4af194f9bULL, 0xab1c5ed5da6d8118ULL, 0xd807aa98a3030242ULL,+	0x12835b0145706fbeULL, 0x243185be4ee4b28cULL, 0x550c7dc3d5ffb4e2ULL,+	0x72be5d74f27b896fULL, 0x80deb1fe3b1696b1ULL, 0x9bdc06a725c71235ULL,+	0xc19bf174cf692694ULL, 0xe49b69c19ef14ad2ULL, 0xefbe4786384f25e3ULL,+	0x0fc19dc68b8cd5b5ULL, 0x240ca1cc77ac9c65ULL, 0x2de92c6f592b0275ULL,+	0x4a7484aa6ea6e483ULL, 0x5cb0a9dcbd41fbd4ULL, 0x76f988da831153b5ULL,+	0x983e5152ee66dfabULL, 0xa831c66d2db43210ULL, 0xb00327c898fb213fULL,+	0xbf597fc7beef0ee4ULL, 0xc6e00bf33da88fc2ULL, 0xd5a79147930aa725ULL,+	0x06ca6351e003826fULL, 0x142929670a0e6e70ULL, 0x27b70a8546d22ffcULL,+	0x2e1b21385c26c926ULL, 0x4d2c6dfc5ac42aedULL, 0x53380d139d95b3dfULL,+	0x650a73548baf63deULL, 0x766a0abb3c77b2a8ULL, 0x81c2c92e47edaee6ULL,+	0x92722c851482353bULL, 0xa2bfe8a14cf10364ULL, 0xa81a664bbc423001ULL,+	0xc24b8b70d0f89791ULL, 0xc76c51a30654be30ULL, 0xd192e819d6ef5218ULL,+	0xd69906245565a910ULL, 0xf40e35855771202aULL, 0x106aa07032bbd1b8ULL,+	0x19a4c116b8d2d0c8ULL, 0x1e376c085141ab53ULL, 0x2748774cdf8eeb99ULL,+	0x34b0bcb5e19b48a8ULL, 0x391c0cb3c5c95a63ULL, 0x4ed8aa4ae3418acbULL,+	0x5b9cca4f7763e373ULL, 0x682e6ff3d6b2b8a3ULL, 0x748f82ee5defb2fcULL,+	0x78a5636f43172f60ULL, 0x84c87814a1f0ab72ULL, 0x8cc702081a6439ecULL,+	0x90befffa23631e28ULL, 0xa4506cebde82bde9ULL, 0xbef9a3f7b2c67915ULL,+	0xc67178f2e372532bULL, 0xca273eceea26619cULL, 0xd186b8c721c0c207ULL,+	0xeada7dd6cde0eb1eULL, 0xf57d4f7fee6ed178ULL, 0x06f067aa72176fbaULL,+	0x0a637dc5a2c898a6ULL, 0x113f9804bef90daeULL, 0x1b710b35131c471bULL,+	0x28db77f523047d84ULL, 0x32caab7b40c72493ULL, 0x3c9ebe0a15c9bebcULL,+	0x431d67c49c100d4cULL, 0x4cc5d4becb3e42b6ULL, 0x597f299cfc657e2aULL,+	0x5fcb6fab3ad6faecULL, 0x6c44198c4a475817ULL,+};++/*+ * One 128-byte block.  `state` is the eight words of chaining value in host+ * order, `buf` the block as it arrived, which SHA-512 reads big-endian.+ *+ * ab, cd, ef and gh hold the working variables in pairs.  Each step covers+ * two rounds and rotates which pair plays which part, so four steps return+ * to the start; a group of eight steps is one pass over the schedule.+ */+CRYPTON_TARGET_ARMV8_SHA3+void crypton_sha512_armv8_do_chunk(uint64_t state[8], const uint8_t buf[128])+{+	uint64x2_t ab, cd, ef, gh, ab0, cd0, ef0, gh0;+	uint64x2_t s[8];+	int i;++	ab0 = ab = vld1q_u64(state);+	cd0 = cd = vld1q_u64(state + 2);+	ef0 = ef = vld1q_u64(state + 4);+	gh0 = gh = vld1q_u64(state + 6);++	for (i = 0; i < 8; i++)+		s[i] = vreinterpretq_u64_u8(vrev64q_u8(+		    vld1q_u8(buf + 16 * i)));++/* two rounds; A, B, C, D is a rotation of gh, ef, cd, ab */+#define RND(A, B, C, D, sv, ki)                                             \+	do {                                                                \+		uint64x2_t is_ = vaddq_u64((sv), vld1q_u64(&K[ki]));        \+		uint64x2_t sum_ = vaddq_u64(vextq_u64(is_, is_, 1), (A));   \+		uint64x2_t im_ = vsha512hq_u64(sum_, vextq_u64((B), (A), 1),\+		                               vextq_u64((C), (B), 1));     \+		(A) = vsha512h2q_u64(im_, (C), (D));                        \+		(C) = vaddq_u64((C), im_);                                  \+	} while (0)++/* extend the schedule in place, for the next sixteen rounds */+#define SCHED(j)                                                            \+	s[j] = vsha512su1q_u64(vsha512su0q_u64(s[j], s[((j) + 1) & 7]),     \+	                       s[((j) + 7) & 7],                            \+	                       vextq_u64(s[((j) + 4) & 7], s[((j) + 5) & 7], 1))++#define PASS(base)                                       \+	SCHED(0); RND(gh, ef, cd, ab, s[0], (base) +  0); \+	SCHED(1); RND(ef, cd, ab, gh, s[1], (base) +  2); \+	SCHED(2); RND(cd, ab, gh, ef, s[2], (base) +  4); \+	SCHED(3); RND(ab, gh, ef, cd, s[3], (base) +  6); \+	SCHED(4); RND(gh, ef, cd, ab, s[4], (base) +  8); \+	SCHED(5); RND(ef, cd, ab, gh, s[5], (base) + 10); \+	SCHED(6); RND(cd, ab, gh, ef, s[6], (base) + 12); \+	SCHED(7); RND(ab, gh, ef, cd, s[7], (base) + 14)++	/* rounds 0..15 run straight off the message */+	RND(gh, ef, cd, ab, s[0],  0);+	RND(ef, cd, ab, gh, s[1],  2);+	RND(cd, ab, gh, ef, s[2],  4);+	RND(ab, gh, ef, cd, s[3],  6);+	RND(gh, ef, cd, ab, s[4],  8);+	RND(ef, cd, ab, gh, s[5], 10);+	RND(cd, ab, gh, ef, s[6], 12);+	RND(ab, gh, ef, cd, s[7], 14);++	PASS(16);+	PASS(32);+	PASS(48);+	PASS(64);++#undef PASS+#undef SCHED+#undef RND++	vst1q_u64(state,     vaddq_u64(ab, ab0));+	vst1q_u64(state + 2, vaddq_u64(cd, cd0));+	vst1q_u64(state + 4, vaddq_u64(ef, ef0));+	vst1q_u64(state + 6, vaddq_u64(gh, gh0));+}++/*+ * Whether the extension is there.  Unlike the SHA-256 one this is not+ * something to take for granted anywhere, so both platforms are asked.+ */+int crypton_sha512_armv8_available(void)+{+#if defined(__APPLE__)+	int v = 0;+	size_t n = sizeof(v);++	if (sysctlbyname("hw.optional.arm.FEAT_SHA512", &v, &n, NULL, 0) != 0)+		return 0;+	return v != 0;+#elif defined(__linux__)+	return (getauxval(AT_HWCAP) & HWCAP_SHA512) != 0;+#else+	return 0;+#endif+}
+ cbits/tests/ct/README view
@@ -0,0 +1,84 @@+Does the code that handles a secret run in time independent of it?++memcheck already follows undefined bytes through arithmetic and reports the+moment one of them decides a branch or an address.  That is the same question,+so each driver here declares its secret undefined and runs under valgrind:+every report names a place where a secret reached a branch or an index.  The+technique is Adam Langley's ctgrind.++ct_canary.c is the calibration.  It branches on a secret and indexes a table+with one, so it must report; if it does not, the marking is not reaching the+code and the silence of every other driver in that run means nothing.  Round+five learned this the hard way with ThreadSanitizer, which saw nothing through+GHC's runtime including a deliberate race.++What is marked secret in each driver is the thing the caller would call a+private key, and nothing else.  A modulus, a peer's public key, a nonce and a+message are public and stay defined; so does each answer, which is declared+public again before anything looks at it.++  canary   a branch and a table index on a secret -- must report+  powm     crypton_powm_sec, the exponent being an RSA private key+  p256     both scalar multiplications and the scalar inversion+  x25519   the scalar+  ed25519  the private key, through signing+  decaf    Ed448 signing and X448, both private scalars+  chapoly  the ChaCha20 key, the plaintext, and the Poly1305 key+  aes      the AES key and the plaintext, through ECB and GCM+  aes_armv8  the same driver again, on AArch64, against the instructions++The aes driver is built against cbits/aes/generic.c and cbits/aes/gf.c on+purpose, rather than whatever the machine offers.  AES-NI and the ARMv8+instructions do not look anything up and would report nothing, which would say+nothing about the table-driven code every other machine runs.  That code is+variable-time by construction -- a table index is a byte of the state -- and+so is the table-driven GHASH beside it.  A report from the aes driver is+therefore expected and is a property of those implementations, not a defect+found in them; it is here so that the size of it is written down rather than+assumed.  Everything else is expected to be silent.++On AArch64 the same driver is then built a second time, as aes_armv8, against+cbits/aes/armv8.c with the crypto extension turned on.  AESE, AESMC and PMULL+look nothing up and branch on nothing, so that run must report nothing at all+-- not "nothing outside known.txt", nothing; a table site appearing there+would mean the dispatch had not picked the instructions.  The two runs keep+each other honest: the table-driven one has to report and the instruction one+has to be silent, and either going the wrong way says the run is not+measuring what it claims to.++Until that was added the harness ran only on x86-64, so crypton's AArch64 AES+and GHASH had never been put to it -- which was noticed when the GHASH was+rewritten.++What round eight found+----------------------++Of the eight drivers, five were silent: the RSA exponentiation, X25519,+Ed25519, ChaCha20 and Poly1305 never let a private key decide a branch or an+address.  The AES driver reported from the tables, as it was built to.++The other two reported, five places between them, and every one of them an+assert():++  crypton_p256_modmul          assert(top <= 1), assert(top == 0)+  crypton_gf_448_strong_reduce two asserts on a carry and a borrow+  crypton_gf_invert            assert(ret), that what was inverted had an+                               inverse++The first four check an invariant of a reduction rather than anything about+the data, so they hold whatever the input is.  The fifth holds because the two+callers that ask for it are inverting a projective z, which is never zero for+a point on the curve.  Either way the branch goes the same way every time and+no timing follows from it.  They are reported at all because+crypton's C is compiled without NDEBUG, so assert() is live in a released+library.  Twenty-eight assertions ship that way, none of them with a side+effect, and defining NDEBUG measured no faster on P-256, so whether to keep+them is a question about what a library should do when an internal invariant+fails -- abort the process, or carry on -- rather than one about speed.  They+are listed in known.txt and the job passes with them.++The decision was to keep them: an internal invariant that fails in a+cryptographic library is better met with an abort than with a wrong answer+carried onwards.  So this is settled rather than open, and the five entries in+known.txt are permanent.  What is not permanent is anything else appearing+beside them.
+ cbits/tests/ct/ct.h view
@@ -0,0 +1,53 @@+/* Marking secrets for valgrind.+ *+ * memcheck already follows undefined bytes through arithmetic and complains+ * the moment one decides a branch or an address.  That is the same question+ * as "does this run in time independent of the secret", so a secret declared+ * undefined turns memcheck into a checker for it.  The technique is Adam+ * Langley's ctgrind.+ *+ * Without CRYPTON_CT_VALGRIND the macros vanish and the drivers still build+ * and run, which is how they are kept honest on a machine with no valgrind.+ */+#ifndef CRYPTON_TESTS_CT_H+#define CRYPTON_TESTS_CT_H++#ifdef CRYPTON_CT_VALGRIND+#include <valgrind/memcheck.h>+/* this memory is a secret: report any branch or index that depends on it */+#define CT_SECRET(p, n) VALGRIND_MAKE_MEM_UNDEFINED((p), (n))+/* and this is the answer, which the caller is allowed to look at */+#define CT_PUBLIC(p, n) VALGRIND_MAKE_MEM_DEFINED((p), (n))+#else+#define CT_SECRET(p, n) ((void)(p), (void)(n))+#define CT_PUBLIC(p, n) ((void)(p), (void)(n))+#endif++#include <stdint.h>+#include <stdio.h>++/* A deterministic filler, so that a report names the same operation on every+ * run.  It is not random and does not need to be. */+static uint64_t ct_s0 = 0x243f6a8885a308d3ULL, ct_s1 = 0x13198a2e03707344ULL;+static uint64_t ct_rnd(void) {+    uint64_t x = ct_s0, y = ct_s1;+    ct_s0 = y;+    x ^= x << 23;+    ct_s1 = x ^ y ^ (x >> 17) ^ (y >> 26);+    return ct_s1 + y;+}+static void ct_fill(void *p, size_t n) {+    uint8_t *q = (uint8_t *)p;+    size_t i;+    for (i = 0; i < n; i++) q[i] = (uint8_t)(ct_rnd() >> 24);+}+/* Look at the answer, so that nothing above is optimized away.  Whatever is+ * handed here has been declared public first. */+static void ct_sink(const void *p, size_t n) {+    const uint8_t *q = (const uint8_t *)p;+    size_t i;+    uint8_t acc = 0;+    for (i = 0; i < n; i++) acc ^= q[i];+    if (acc == 0xa5 && n == (size_t)-1) printf("unreachable\n");+}+#endif
+ cbits/tests/ct/ct_aes.c view
@@ -0,0 +1,35 @@+/* AES, and AES-GCM.  The key is the secret, and so is the plaintext.+ *+ * Whether this reports depends on which implementation the machine selected.+ * AES-NI and the ARMv8 instructions do not look anything up; the generic C+ * is table-driven and is variable-time by construction, which is a property+ * of that code rather than a defect in it.  See cbits/tests/ct/README. */+#include "tests/ct/ct.h"+#include <string.h>+#include "crypton_aes.h"++int main(void) {+    aes_key k;+    aes_gcm_key gk;+    uint8_t key[32], pt[256], ct[256 + 16], iv[12];++    ct_fill(key, sizeof key);+    ct_fill(pt, sizeof pt);+    ct_fill(iv, sizeof iv);+    CT_SECRET(key, sizeof key);+    CT_SECRET(pt, sizeof pt);++    crypton_aes_initkey(&k, key, sizeof key);+    crypton_aes_encrypt_ecb((aes_block *)ct, &k, (aes_block *)pt,+                            sizeof pt / 16);+    CT_PUBLIC(ct, sizeof pt);+    ct_sink(ct, sizeof pt);++    crypton_aes_gcm_key_init(&gk, &k);+    /* the output takes the ciphertext and then the tag */+    crypton_aes_gcm_full_encrypt(ct, &gk, &k, iv, sizeof iv, NULL, 0,+                                 pt, sizeof pt, 16);+    CT_PUBLIC(ct, sizeof ct);+    ct_sink(ct, sizeof ct);+    return 0;+}
+ cbits/tests/ct/ct_aes_armv8.c view
@@ -0,0 +1,12 @@+/* The same driver as ct_aes.c, built against the AArch64 implementation+ * instead of the table-driven C.+ *+ * AESE, AESMC and PMULL look nothing up and branch on nothing, so this one+ * must report nothing at all -- not "nothing unknown", nothing.  The+ * table-driven run of the same driver is what keeps that honest: if the+ * marking stopped reaching the code, that run would fall silent and fail,+ * and a silence here would mean no more than a silence there.+ *+ * Until this existed the constant-time harness ran only on x86-64, so+ * crypton's AArch64 AES and GHASH had never been put to it. */+#include "ct_aes.c"
+ cbits/tests/ct/ct_canary.c view
@@ -0,0 +1,21 @@+/* The calibration.  This one is deliberately not constant time: it branches+ * on a secret byte and indexes a table with another.  If it reports nothing,+ * the marking is not reaching the code and every other driver's silence in+ * this run means nothing either -- which is the whole reason it is here. */+#include "tests/ct/ct.h"++static const uint8_t table[256] = {1};++int main(void) {+    uint8_t secret[32], out[2];++    ct_fill(secret, sizeof secret);+    CT_SECRET(secret, sizeof secret);++    out[0] = secret[0] & 1 ? 0x5a : 0xa5;   /* a branch on the secret */+    out[1] = table[secret[1]];              /* an address from the secret */++    CT_PUBLIC(out, sizeof out);+    ct_sink(out, sizeof out);+    return 0;+}
+ cbits/tests/ct/ct_chapoly.c view
@@ -0,0 +1,33 @@+/* ChaCha20 and Poly1305.  The key is the secret, and so is the plaintext. */+#include "tests/ct/ct.h"+#include <string.h>+#include "crypton_chacha.h"+#include "crypton_poly1305.h"++int main(void) {+    crypton_chacha_context ctx;+    poly1305_ctx pctx;+    poly1305_key pkey;+    poly1305_mac mac;+    uint8_t key[32], iv[12], pt[256], ct[256];++    ct_fill(key, sizeof key);+    ct_fill(iv, sizeof iv);+    ct_fill(pt, sizeof pt);+    ct_fill(pkey, sizeof pkey);+    CT_SECRET(key, sizeof key);+    CT_SECRET(pt, sizeof pt);+    CT_SECRET(pkey, sizeof pkey);++    crypton_chacha_init(&ctx, 20, sizeof key, key, sizeof iv, iv);+    crypton_chacha_combine(ct, &ctx, pt, sizeof pt);+    CT_PUBLIC(ct, sizeof ct);          /* the ciphertext goes on the wire */+    ct_sink(ct, sizeof ct);++    crypton_poly1305_init(&pctx, &pkey);+    crypton_poly1305_update(&pctx, ct, sizeof ct);+    crypton_poly1305_finalize(mac, &pctx);+    CT_PUBLIC(mac, sizeof mac);+    ct_sink(mac, sizeof mac);+    return 0;+}
+ cbits/tests/ct/ct_decaf.c view
@@ -0,0 +1,36 @@+/* X448 and Ed448.  The scalar and the private key are the secrets. */+#include "tests/ct/ct.h"+#include <string.h>+#include "decaf/ed448.h"+#include "decaf/point_448.h"++int main(void) {+    uint8_t priv[CRYPTON_DECAF_EDDSA_448_PRIVATE_BYTES];+    uint8_t pub[CRYPTON_DECAF_EDDSA_448_PUBLIC_BYTES];+    uint8_t sig[CRYPTON_DECAF_EDDSA_448_SIGNATURE_BYTES];+    uint8_t xs[CRYPTON_DECAF_X448_PRIVATE_BYTES];+    uint8_t xb[CRYPTON_DECAF_X448_PUBLIC_BYTES];+    uint8_t xo[CRYPTON_DECAF_X448_PUBLIC_BYTES];+    uint8_t msg[64];++    ct_fill(priv, sizeof priv);+    ct_fill(msg, sizeof msg);+    ct_fill(xs, sizeof xs);+    ct_fill(xb, sizeof xb);+    CT_SECRET(priv, sizeof priv);+    CT_SECRET(xs, sizeof xs);++    crypton_decaf_ed448_derive_public_key(pub, priv);+    CT_PUBLIC(pub, sizeof pub);+    crypton_decaf_ed448_sign(sig, priv, pub, msg, sizeof msg, 0, NULL, 0);+    CT_PUBLIC(sig, sizeof sig);+    ct_sink(sig, sizeof sig);++    crypton_decaf_x448_derive_public_key(xo, xs);+    CT_PUBLIC(xo, sizeof xo);+    ct_sink(xo, sizeof xo);+    (void)crypton_decaf_x448(xo, xb, xs);+    CT_PUBLIC(xo, sizeof xo);+    ct_sink(xo, sizeof xo);+    return 0;+}
+ cbits/tests/ct/ct_ed25519.c view

file too large to diff

+ cbits/tests/ct/ct_p256.c view

file too large to diff

+ cbits/tests/ct/ct_powm.c view

file too large to diff

+ cbits/tests/ct/ct_x25519.c view

file too large to diff

+ cbits/tests/ct/known.txt view

file too large to diff

+ cbits/tests/ct/run.sh view

file too large to diff

+ cbits/tests/endian/README view

file too large to diff

+ cbits/tests/endian/endian.c view

file too large to diff

+ cbits/tests/endian/run.sh view

file too large to diff

+ cbits/tests/endian/vectors.txt view

file too large to diff

+ cbits/tests/fuzz/README view

file too large to diff

+ cbits/tests/fuzz/corpus/aead-authentic.bin view

file too large to diff

+ cbits/tests/fuzz/corpus/ed25519-tampered.bin view

file too large to diff

+ cbits/tests/fuzz/corpus/ed25519-valid.bin view

file too large to diff

+ cbits/tests/fuzz/corpus/p256-on-curve.bin view

file too large to diff

+ cbits/tests/fuzz/fuzz.h view

file too large to diff

+ cbits/tests/fuzz/fuzz_aead.c view

file too large to diff

+ cbits/tests/fuzz/fuzz_canary.c view

file too large to diff

+ cbits/tests/fuzz/fuzz_decaf.c view

file too large to diff

+ cbits/tests/fuzz/fuzz_ed25519.c view

file too large to diff

+ cbits/tests/fuzz/fuzz_p256.c view

file too large to diff

+ cbits/tests/fuzz/run.sh view

file too large to diff

+ cbits/tests/fuzz/standalone.c view

file too large to diff

+ cbits/tests/perf/floors.txt view

file too large to diff

+ cbits/tests/perf/run.sh view

file too large to diff

+ cbits/tests/perf/throughput.c view

file too large to diff

+ cbits/tests/scrub/README view

file too large to diff

+ cbits/tests/scrub/known.txt view

file too large to diff

+ cbits/tests/scrub/run.sh view

file too large to diff

+ cbits/tests/scrub/scrub.c view

file too large to diff

+ cbits/tests/width/ed448_width.c view

file too large to diff

+ cbits/tests/width/p256_width.c view

file too large to diff

+ cbits/tests/width/run.sh view

file too large to diff

+ cbits/tests/width/x25519_width.c view

file too large to diff

crypton.cabal view

file too large to diff

+ tests/AFISSpec.hs view

file too large to diff

− tests/BCrypt.hs

file too large to diff

− tests/BCryptPBKDF.hs

file too large to diff

tests/BlockCipher.hs view

file too large to diff

+ tests/BlockCipher/AES/CBC.hs view

file too large to diff

+ tests/BlockCipher/AES/CCM.hs view

file too large to diff

+ tests/BlockCipher/AES/CTR.hs view

file too large to diff

+ tests/BlockCipher/AES/ECB.hs view

file too large to diff

+ tests/BlockCipher/AES/GCM.hs view

file too large to diff

+ tests/BlockCipher/AES/GCMLong.hs view

file too large to diff

+ tests/BlockCipher/AES/OCB3.hs view

file too large to diff

+ tests/BlockCipher/AES/XTS.hs view

file too large to diff

+ tests/BlockCipher/AESGCMSIVSpec.hs view

file too large to diff

+ tests/BlockCipher/AESSpec.hs view

file too large to diff

+ tests/BlockCipher/BlowfishSpec.hs view

file too large to diff

+ tests/BlockCipher/CAST5Spec.hs view

file too large to diff

+ tests/BlockCipher/CamelliaSpec.hs view

file too large to diff

+ tests/BlockCipher/DESSpec.hs view

file too large to diff

+ tests/BlockCipher/ModesSpec.hs view

file too large to diff

+ tests/BlockCipher/TripleDESSpec.hs view

file too large to diff

+ tests/BlockCipher/TwofishSpec.hs view

file too large to diff

− tests/ChaCha.hs

file too large to diff

− tests/ChaChaPoly1305.hs

file too large to diff

+ tests/ConstructHash/MiyaguchiPreneelSpec.hs view

file too large to diff

+ tests/Curve25519Spec.hs view

file too large to diff

+ tests/Curve448Spec.hs view

file too large to diff

− tests/ECC.hs

file too large to diff

− tests/ECC/Edwards25519.hs

file too large to diff

+ tests/ECC/Edwards25519Spec.hs view

file too large to diff

+ tests/ECCSpec.hs view

file too large to diff

− tests/ECDSA.hs

file too large to diff

+ tests/ECDSASpec.hs view

file too large to diff

+ tests/Ed25519Spec.hs view

file too large to diff

+ tests/Ed448Spec.hs view

file too large to diff

+ tests/EdDSASpec.hs view

file too large to diff

− tests/Hash.hs

file too large to diff

+ tests/HashSpec.hs view

file too large to diff

tests/Imports.hs view

file too large to diff

− tests/KAT_AES.hs

file too large to diff

− tests/KAT_AES/KATCBC.hs

file too large to diff

− tests/KAT_AES/KATCCM.hs

file too large to diff

− tests/KAT_AES/KATECB.hs

file too large to diff

− tests/KAT_AES/KATGCM.hs

file too large to diff

− tests/KAT_AES/KATOCB3.hs

file too large to diff

− tests/KAT_AES/KATXTS.hs

file too large to diff

− tests/KAT_AESGCMSIV.hs

file too large to diff

− tests/KAT_AFIS.hs

file too large to diff

− tests/KAT_Argon2.hs

file too large to diff

− tests/KAT_Blake2.hs

file too large to diff

− tests/KAT_Blowfish.hs

file too large to diff

− tests/KAT_CAST5.hs

file too large to diff

− tests/KAT_CMAC.hs

file too large to diff

− tests/KAT_Camellia.hs

file too large to diff

− tests/KAT_Curve25519.hs

file too large to diff

− tests/KAT_Curve448.hs

file too large to diff

− tests/KAT_DES.hs

file too large to diff

− tests/KAT_Ed25519.hs

file too large to diff

− tests/KAT_Ed448.hs

file too large to diff

− tests/KAT_EdDSA.hs

file too large to diff

− tests/KAT_HKDF.hs

file too large to diff

− tests/KAT_HMAC.hs

file too large to diff

− tests/KAT_KMAC.hs

file too large to diff

− tests/KAT_MiyaguchiPreneel.hs

file too large to diff

− tests/KAT_OTP.hs

file too large to diff

− tests/KAT_PBKDF2.hs

file too large to diff

− tests/KAT_PubKey.hs

file too large to diff

− tests/KAT_PubKey/DSA.hs

file too large to diff

− tests/KAT_PubKey/ECC.hs

file too large to diff

− tests/KAT_PubKey/ECDSA.hs

file too large to diff

− tests/KAT_PubKey/OAEP.hs

file too large to diff

− tests/KAT_PubKey/P256.hs

file too large to diff

− tests/KAT_PubKey/PSS.hs

file too large to diff

− tests/KAT_PubKey/RSA.hs

file too large to diff

− tests/KAT_PubKey/Rabin.hs

file too large to diff

− tests/KAT_RC4.hs

file too large to diff

− tests/KAT_Scrypt.hs

file too large to diff

− tests/KAT_TripleDES.hs

file too large to diff

− tests/KAT_Twofish.hs

file too large to diff

+ tests/KDF/Argon2Spec.hs view

file too large to diff

+ tests/KDF/BCryptPBKDFSpec.hs view

file too large to diff

+ tests/KDF/BCryptSpec.hs view

file too large to diff

+ tests/KDF/HKDFSpec.hs view

file too large to diff

+ tests/KDF/PBKDF2Spec.hs view

file too large to diff

+ tests/KDF/ScryptSpec.hs view

file too large to diff

+ tests/MAC/Blake2Spec.hs view

file too large to diff

+ tests/MAC/CMACSpec.hs view

file too large to diff

+ tests/MAC/HMACSpec.hs view

file too large to diff

+ tests/MAC/KMACSpec.hs view

file too large to diff

+ tests/MAC/Poly1305Spec.hs view

file too large to diff

+ tests/MAC/Poly1305Vectors.hs view

file too large to diff

− tests/Number.hs

file too large to diff

− tests/Number/F2m.hs

file too large to diff

+ tests/Number/F2mSpec.hs view

file too large to diff

+ tests/NumberSpec.hs view

file too large to diff

+ tests/OTPSpec.hs view

file too large to diff

− tests/Padding.hs

file too large to diff

+ tests/PaddingSpec.hs view

file too large to diff

− tests/Poly1305.hs

file too large to diff

+ tests/PubKey/DHSpec.hs view

file too large to diff

+ tests/PubKey/DSASpec.hs view

file too large to diff

+ tests/PubKey/ECCSpec.hs view

file too large to diff

+ tests/PubKey/ECDSASpec.hs view

file too large to diff

+ tests/PubKey/ElGamalSpec.hs view

file too large to diff

+ tests/PubKey/MGF1Spec.hs view

file too large to diff

+ tests/PubKey/OAEPSpec.hs view

file too large to diff

+ tests/PubKey/P256Spec.hs view

file too large to diff

+ tests/PubKey/PSSSpec.hs view

file too large to diff

+ tests/PubKey/RSASpec.hs view

file too large to diff

+ tests/PubKey/RabinSpec.hs view

file too large to diff

+ tests/PubKey/SecrecySpec.hs view

file too large to diff

+ tests/RuntimeSpec.hs view

file too large to diff

− tests/Salsa.hs

file too large to diff

+ tests/Spec.hs view

file too large to diff

+ tests/StreamCipher/ChaChaPoly1305Spec.hs view

file too large to diff

+ tests/StreamCipher/ChaChaSpec.hs view

file too large to diff

+ tests/StreamCipher/RC4Spec.hs view

file too large to diff

+ tests/StreamCipher/SalsaSpec.hs view

file too large to diff

+ tests/StreamCipher/XSalsaSpec.hs view

file too large to diff

− tests/Tests.hs

file too large to diff

tests/Utils.hs view

file too large to diff

− tests/XSalsa.hs

file too large to diff